2021-11-03T16:51:49.885 INFO:root:teuthology version: 1.1.0-c56135d1 2021-11-03T16:51:49.889 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/sage-2021-11-03_16:24:16-orch:cephadm-wip-sage-testing-2021-11-03-1039-distro-basic-smithi/6481880 --verbose --name sage-2021-11-03_16:24:16-orch:cephadm-wip-sage-testing-2021-11-03-1039-distro-basic-smithi --description orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} --owner scheduled_sage@teuthology -- /home/teuthworker/archive/sage-2021-11-03_16:24:16-orch:cephadm-wip-sage-testing-2021-11-03-1039-distro-basic-smithi/6481880/orig.config.yaml 2021-11-03T16:51:49.961 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2021-11-03T16:51:50.030 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/sage-2021-11-03_16:24:16-orch:cephadm-wip-sage-testing-2021-11-03-1039-distro-basic-smithi/6481880 branch: wip-sage-testing-2021-11-03-1039 description: orch:cephadm/mds_upgrade_sequence/{bluestore-bitmap centos_8.3_container_tools_3.0 conf/{client mds mon osd} overrides/{pg-warn whitelist_health whitelist_wrongly_marked_down} roles tasks/{0-v16.2.4 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-verify} 2-client 3-upgrade-with-workload 4-verify}} email: null first_in_suite: false job_id: '6481880' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/pacific v16.2.4 ' name: sage-2021-11-03_16:24:16-orch:cephadm-wip-sage-testing-2021-11-03-1039-distro-basic-smithi nuke-on-error: true os_type: centos os_version: '8.3' overrides: admin_socket: branch: wip-sage-testing-2021-11-03-1039 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 15m rados osd op timeout: 15m global: mon pg warn min per osd: 0 mds: debug mds: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 15m rados osd op timeout: 15m mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) sha1: 6e90212d8510e2d506395a8c82057d0d1787f3a4 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: sha1: 6e90212d8510e2d506395a8c82057d0d1787f3a4 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-sage-testing-2021-11-03-1039 sha1: 6e90212d8510e2d506395a8c82057d0d1787f3a4 owner: scheduled_sage@teuthology priority: 75 repo: git://git.ceph.com/ceph-ci.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 sha1: 6e90212d8510e2d506395a8c82057d0d1787f3a4 sleep_before_teardown: 0 suite: orch:cephadm suite_branch: wip-sage-testing-2021-11-03-1039 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_6e90212d8510e2d506395a8c82057d0d1787f3a4/qa suite_relpath: qa suite_repo: git://git.ceph.com/ceph-ci.git suite_sha1: 6e90212d8510e2d506395a8c82057d0d1787f3a4 targets: smithi027.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDwK7+tjTmjvBGrAd6+Ql7k/jgnGqy6rCixxYbChhhMfhBrcnY2HwQB1GExeC/Zv0gLxNexnR94EgywqAOiICYtnYuBtofEkOtCswiSLjTdygpB0rB8scKtjcbZ4fVKwkL2T5yamFt4uFwtejqQOWNpTec36dxNHNU+RxvW6HEiKyueBnxhvf1MJ+Z8cYzZl5ZFWmtJOjtSL8mXoVtCXPrUvnPJFXrGwmYS/PUFJV9gleAtOQnb9yKyJwmCVObX8ehn5qveRm3FP3eH1QKvDBQEFYd4xtak4PDJ6Ryb1Qc6IW36d/coijplGu5fCnkhPMJLCet+kr+ZRtl9yuNfe3K83gmvmLG1QCEgYpPxaFWRTLuwgfVjOAcpNQnTf6LVAKzdKiWE4IsZP7U7v7n9Z4KHl/7olNAv03C23fJF7OSpt6Rmage8DgSeIxM+WvNkcLzmHcRduBXScCIOB3VoGEd5Xg2g5k+6D6ARIXNSQohjZ5pyRdb7K0jg2n/cYP+vvs= smithi115.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCyLfmoVJIGBUXkY1Y9AobjbhZ+wZy6pBsJPT8oE56bSGAFJX6x0MyWSHgtkRD03PLLkxISv1UJs5TDqQ2VFcFOVVG0K5rns0F/gR0FwFbzFV6Qnm3OeXYUQzCDtBXnhYas9IXlYatfMMDjXaEdNMGjZ31f+dF8/juqFQH2oP8+nrx5cUmTdphyTw3cWYMpAg5dw1apaG40Ql2C/9FzRbQTQAjHBsbBZPVS81zwzhbyEsTZUcZ3V4dw4JAh1zrRFgi6dSVovA/C++h70kkAQ7tCFEPsMiLby3OjfthQWZ7Gxs7q2wCDl8JLKln713qRrD1Pxe+44pwFsPhkoGgDeAs3tuP6skD+6EMaghvpyhGPzp8KEn2/AHS0ElgQlgHOe4mjaQO9/D9nYZrzh+BMG8BYgzFWlukb3fQBEgNOmQ0E4AL5+shUdThPcpqIGgg0buRQ+B02FYppN/EhY5Y+ZPDUEAyArt3Kx4J2mP2Xz3bmgmT2ZkU6WHkhurmESCHT9V0= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:3.0 - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: exclude_packages: - ceph-volume tag: v16.2.4 - cephadm: avoid_pacific_features: true cephadm_branch: v16.2.4 cephadm_git_url: https://github.com/ceph/ceph image: docker.io/ceph/ceph:v16.2.4 roleless: true - print: '**** done starting v16.2.4' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 2 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay false - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - ceph --format=json mds versions | jq -e ". | add == 4" - fs.pre_upgrade_save: null - kclient: null - print: '**** done kclient client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology_branch: master teuthology_sha1: c56135d151713269e811ede3163c9743c2e269de timestamp: 2021-11-03_16:24:16 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true ; do ceph orch ps ; ceph versions ; ceph fs dump; sleep 30 ; done - ceph orch ps - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: sage verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.6888 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2021-11-03T16:51:50.030 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_6e90212d8510e2d506395a8c82057d0d1787f3a4/qa; will attempt to use it 2021-11-03T16:51:50.031 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_6e90212d8510e2d506395a8c82057d0d1787f3a4/qa/tasks 2021-11-03T16:51:50.032 INFO:teuthology.run_tasks:Running task internal.check_packages... 2021-11-03T16:51:50.034 INFO:teuthology.task.internal:Checking packages... 2021-11-03T16:51:50.065 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '6e90212d8510e2d506395a8c82057d0d1787f3a4' 2021-11-03T16:51:50.065 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2021-11-03T16:51:50.066 INFO:teuthology.packaging:ref: None 2021-11-03T16:51:50.067 INFO:teuthology.packaging:tag: None 2021-11-03T16:51:50.067 INFO:teuthology.packaging:branch: wip-sage-testing-2021-11-03-1039 2021-11-03T16:51:50.068 INFO:teuthology.packaging:sha1: 6e90212d8510e2d506395a8c82057d0d1787f3a4 2021-11-03T16:51:50.069 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-sage-testing-2021-11-03-1039 2021-11-03T16:51:50.298 INFO:teuthology.task.internal:Found packages for ceph version 17.0.0-8776.g6e90212d 2021-11-03T16:51:50.298 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2021-11-03T16:51:50.314 INFO:teuthology.task.internal:no buildpackages task found 2021-11-03T16:51:50.315 INFO:teuthology.run_tasks:Running task internal.save_config... 2021-11-03T16:51:50.358 INFO:teuthology.task.internal:Saving configuration 2021-11-03T16:51:50.380 INFO:teuthology.run_tasks:Running task internal.check_lock... 2021-11-03T16:51:50.388 INFO:teuthology.task.internal.check_lock:Checking locks... 2021-11-03T16:51:50.412 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi027.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/sage-2021-11-03_16:24:16-orch:cephadm-wip-sage-testing-2021-11-03-1039-distro-basic-smithi/6481880', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2021-11-03 16:47:05.328980', 'locked_by': 'scheduled_sage@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBTxSWYxE/la0sO/pQggxsITUKdL6zX7gQMj7DzfyhpWhCBMgspMYYZJ3rs/YOWzmv2DgHPc5x0S9HyAZjLEjcY='} 2021-11-03T16:51:50.435 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi115.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/sage-2021-11-03_16:24:16-orch:cephadm-wip-sage-testing-2021-11-03-1039-distro-basic-smithi/6481880', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2021-11-03 16:47:05.327586', 'locked_by': 'scheduled_sage@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMpHr16kPHLu5VBIJ1gj8OYaeFJC3ElN2K4Yn6AxONShy5TUZl1toeaeWjJQAYP3a0d4RLcaXuXJP6jCW3UkQRA='} 2021-11-03T16:51:50.435 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2021-11-03T16:51:50.444 INFO:teuthology.task.internal:roles: ubuntu@smithi027.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2021-11-03T16:51:50.444 INFO:teuthology.task.internal:roles: ubuntu@smithi115.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2021-11-03T16:51:50.445 INFO:teuthology.run_tasks:Running task console_log... 2021-11-03T16:51:50.539 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f68257e0c80>, signals=[15]) 2021-11-03T16:51:50.541 INFO:teuthology.run_tasks:Running task internal.connect... 2021-11-03T16:51:50.562 INFO:teuthology.task.internal:Opening connections... 2021-11-03T16:51:50.563 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi027.front.sepia.ceph.com 2021-11-03T16:51:50.565 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-11-03T16:51:50.638 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi115.front.sepia.ceph.com 2021-11-03T16:51:50.639 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-11-03T16:51:50.711 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2021-11-03T16:51:50.719 DEBUG:teuthology.orchestra.run.smithi027:> uname -m 2021-11-03T16:51:50.744 INFO:teuthology.orchestra.run.smithi027.stdout:x86_64 2021-11-03T16:51:50.744 DEBUG:teuthology.orchestra.run.smithi027:> cat /etc/os-release 2021-11-03T16:51:50.800 INFO:teuthology.orchestra.run.smithi027.stdout:NAME="CentOS Linux" 2021-11-03T16:51:50.801 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION="8" 2021-11-03T16:51:50.801 INFO:teuthology.orchestra.run.smithi027.stdout:ID="centos" 2021-11-03T16:51:50.802 INFO:teuthology.orchestra.run.smithi027.stdout:ID_LIKE="rhel fedora" 2021-11-03T16:51:50.802 INFO:teuthology.orchestra.run.smithi027.stdout:VERSION_ID="8" 2021-11-03T16:51:50.802 INFO:teuthology.orchestra.run.smithi027.stdout:PLATFORM_ID="platform:el8" 2021-11-03T16:51:50.802 INFO:teuthology.orchestra.run.smithi027.stdout:PRETTY_NAME="CentOS Linux 8" 2021-11-03T16:51:50.803 INFO:teuthology.orchestra.run.smithi027.stdout:ANSI_COLOR="0;31" 2021-11-03T16:51:50.803 INFO:teuthology.orchestra.run.smithi027.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-11-03T16:51:50.803 INFO:teuthology.orchestra.run.smithi027.stdout:HOME_URL="https://centos.org/" 2021-11-03T16:51:50.803 INFO:teuthology.orchestra.run.smithi027.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-11-03T16:51:50.804 INFO:teuthology.orchestra.run.smithi027.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-11-03T16:51:50.804 INFO:teuthology.orchestra.run.smithi027.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-11-03T16:51:50.805 INFO:teuthology.lock.ops:Updating smithi027.front.sepia.ceph.com on lock server 2021-11-03T16:51:50.824 DEBUG:teuthology.orchestra.run.smithi115:> uname -m 2021-11-03T16:51:50.840 INFO:teuthology.orchestra.run.smithi115.stdout:x86_64 2021-11-03T16:51:50.841 DEBUG:teuthology.orchestra.run.smithi115:> cat /etc/os-release 2021-11-03T16:51:50.895 INFO:teuthology.orchestra.run.smithi115.stdout:NAME="CentOS Linux" 2021-11-03T16:51:50.895 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION="8" 2021-11-03T16:51:50.895 INFO:teuthology.orchestra.run.smithi115.stdout:ID="centos" 2021-11-03T16:51:50.896 INFO:teuthology.orchestra.run.smithi115.stdout:ID_LIKE="rhel fedora" 2021-11-03T16:51:50.896 INFO:teuthology.orchestra.run.smithi115.stdout:VERSION_ID="8" 2021-11-03T16:51:50.896 INFO:teuthology.orchestra.run.smithi115.stdout:PLATFORM_ID="platform:el8" 2021-11-03T16:51:50.896 INFO:teuthology.orchestra.run.smithi115.stdout:PRETTY_NAME="CentOS Linux 8" 2021-11-03T16:51:50.897 INFO:teuthology.orchestra.run.smithi115.stdout:ANSI_COLOR="0;31" 2021-11-03T16:51:50.897 INFO:teuthology.orchestra.run.smithi115.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2021-11-03T16:51:50.897 INFO:teuthology.orchestra.run.smithi115.stdout:HOME_URL="https://centos.org/" 2021-11-03T16:51:50.897 INFO:teuthology.orchestra.run.smithi115.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2021-11-03T16:51:50.898 INFO:teuthology.orchestra.run.smithi115.stdout:CENTOS_MANTISBT_PROJECT="CentOS-8" 2021-11-03T16:51:50.898 INFO:teuthology.orchestra.run.smithi115.stdout:CENTOS_MANTISBT_PROJECT_VERSION="8" 2021-11-03T16:51:50.898 INFO:teuthology.lock.ops:Updating smithi115.front.sepia.ceph.com on lock server 2021-11-03T16:51:50.916 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2021-11-03T16:51:50.927 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2021-11-03T16:51:50.946 INFO:teuthology.task.internal:Checking for old test directory... 2021-11-03T16:51:50.947 DEBUG:teuthology.orchestra.run.smithi027:> test '!' -e /home/ubuntu/cephtest 2021-11-03T16:51:50.950 DEBUG:teuthology.orchestra.run.smithi115:> test '!' -e /home/ubuntu/cephtest 2021-11-03T16:51:50.963 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2021-11-03T16:51:50.975 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2021-11-03T16:51:50.976 DEBUG:teuthology.orchestra.run.smithi027:> test -z $(ls -A /var/lib/ceph) 2021-11-03T16:51:51.005 DEBUG:teuthology.orchestra.run.smithi115:> test -z $(ls -A /var/lib/ceph) 2021-11-03T16:51:51.032 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2021-11-03T16:51:51.090 INFO:teuthology.run_tasks:Running task kernel... 2021-11-03T16:51:51.107 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2021-11-03T16:51:51.108 INFO:teuthology.task.kernel:config {'host.a': {'kdb': True, 'sha1': 'distro'}, 'host.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2021-11-03T16:51:51.108 DEBUG:teuthology.orchestra.run.smithi027:> uname -r 2021-11-03T16:51:51.124 INFO:teuthology.orchestra.run.smithi027.stdout:4.18.0-305.10.2.el8_4.x86_64 2021-11-03T16:51:51.125 INFO:teuthology.task.kernel:Running kernel on smithi027: 4.18.0-305.10.2.el8_4.x86_64 2021-11-03T16:51:51.125 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum install -y kernel 2021-11-03T16:51:53.609 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS-8 - AppStream 28 MB/s | 9.5 MB 00:00 2021-11-03T16:51:55.347 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS-8 - Base 45 MB/s | 7.5 MB 00:00 2021-11-03T16:51:56.504 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS-8 - Extras 137 kB/s | 10 kB 00:00 2021-11-03T16:51:58.320 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Linux 8 - AppStream 5.4 MB/s | 9.5 MB 00:01 2021-11-03T16:52:00.417 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Linux 8 - BaseOS 15 MB/s | 7.5 MB 00:00 2021-11-03T16:52:01.845 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Linux 8 - Extras 31 kB/s | 10 kB 00:00 2021-11-03T16:52:02.000 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS-8 - PowerTools 25 MB/s | 2.4 MB 00:00 2021-11-03T16:52:02.551 INFO:teuthology.orchestra.run.smithi027.stdout:Copr repo for python3-asyncssh owned by ceph 32 kB/s | 3.5 kB 00:00 2021-11-03T16:52:03.388 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 14 MB/s | 11 MB 00:00 2021-11-03T16:52:05.738 INFO:teuthology.orchestra.run.smithi027.stdout:lab-extras 4.5 MB/s | 24 kB 00:00 2021-11-03T16:52:07.858 INFO:teuthology.orchestra.run.smithi027.stdout:Package kernel-4.18.0-305.10.2.el8_4.x86_64 is already installed. 2021-11-03T16:52:07.858 INFO:teuthology.orchestra.run.smithi027.stdout:Package kernel-4.18.0-240.el8.x86_64 is already installed. 2021-11-03T16:52:08.202 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2021-11-03T16:52:08.203 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T16:52:08.203 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2021-11-03T16:52:08.203 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T16:52:08.204 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2021-11-03T16:52:08.204 INFO:teuthology.orchestra.run.smithi027.stdout: kernel x86_64 4.18.0-305.19.1.el8_4 CentOS-Base 5.9 M 2021-11-03T16:52:08.204 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2021-11-03T16:52:08.204 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-core x86_64 4.18.0-305.19.1.el8_4 CentOS-Base 36 M 2021-11-03T16:52:08.205 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-modules x86_64 4.18.0-305.19.1.el8_4 CentOS-Base 28 M 2021-11-03T16:52:08.205 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T16:52:08.205 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2021-11-03T16:52:08.205 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T16:52:08.206 INFO:teuthology.orchestra.run.smithi027.stdout:Install 3 Packages 2021-11-03T16:52:08.206 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T16:52:08.206 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 70 M 2021-11-03T16:52:08.206 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 88 M 2021-11-03T16:52:08.206 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2021-11-03T16:52:08.395 INFO:teuthology.orchestra.run.smithi027.stdout:(1/3): kernel-4.18.0-305.19.1.el8_4.x86_64.rpm 31 MB/s | 5.9 MB 00:00 2021-11-03T16:52:09.318 INFO:teuthology.orchestra.run.smithi027.stdout:(2/3): kernel-core-4.18.0-305.19.1.el8_4.x86_64 32 MB/s | 36 MB 00:01 2021-11-03T16:52:09.531 INFO:teuthology.orchestra.run.smithi027.stdout:(3/3): kernel-modules-4.18.0-305.19.1.el8_4.x86 21 MB/s | 28 MB 00:01 2021-11-03T16:52:09.535 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2021-11-03T16:52:09.535 INFO:teuthology.orchestra.run.smithi027.stdout:Total 52 MB/s | 70 MB 00:01 2021-11-03T16:52:09.653 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2021-11-03T16:52:10.022 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2021-11-03T16:52:10.023 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2021-11-03T16:52:11.241 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2021-11-03T16:52:11.266 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2021-11-03T16:52:13.868 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2021-11-03T16:52:14.364 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : kernel-core-4.18.0-305.19.1.el8_4.x86_64 1/3 2021-11-03T16:52:17.758 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-core-4.18.0-305.19.1.el8_4.x86_64 1/3 2021-11-03T16:52:18.080 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : kernel-modules-4.18.0-305.19.1.el8_4.x86_64 2/3 2021-11-03T16:52:21.272 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-modules-4.18.0-305.19.1.el8_4.x86_64 2/3 2021-11-03T16:52:21.486 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : kernel-4.18.0-305.19.1.el8_4.x86_64 3/3 2021-11-03T16:52:43.897 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-core-4.18.0-305.19.1.el8_4.x86_64 3/3 2021-11-03T16:52:44.257 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: kernel-4.18.0-305.19.1.el8_4.x86_64 3/3 2021-11-03T16:52:44.257 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : kernel-4.18.0-305.19.1.el8_4.x86_64 1/3 2021-11-03T16:52:44.258 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : kernel-core-4.18.0-305.19.1.el8_4.x86_64 2/3 2021-11-03T16:52:45.109 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : kernel-modules-4.18.0-305.19.1.el8_4.x86_64 3/3 2021-11-03T16:52:45.109 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T16:52:45.110 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2021-11-03T16:52:45.110 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-4.18.0-305.19.1.el8_4.x86_64 2021-11-03T16:52:45.111 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-core-4.18.0-305.19.1.el8_4.x86_64 2021-11-03T16:52:45.112 INFO:teuthology.orchestra.run.smithi027.stdout: kernel-modules-4.18.0-305.19.1.el8_4.x86_64 2021-11-03T16:52:45.112 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T16:52:45.112 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2021-11-03T16:52:45.185 DEBUG:teuthology.orchestra.run.smithi027:> rpm -q kernel --last | head -n 1 2021-11-03T16:52:45.338 INFO:teuthology.orchestra.run.smithi027.stdout:kernel-4.18.0-305.19.1.el8_4.x86_64 Wed 03 Nov 2021 04:52:21 PM UTC 2021-11-03T16:52:45.340 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-305.19.1.el8_4.x86_64 2021-11-03T16:52:45.340 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-305.10.2.el8_4.x86_64 Expected: 4.18.0-305.19.1.el8_4.x86_64 2021-11-03T16:52:45.341 DEBUG:teuthology.orchestra.run.smithi115:> uname -r 2021-11-03T16:52:45.359 INFO:teuthology.orchestra.run.smithi115.stdout:4.18.0-305.10.2.el8_4.x86_64 2021-11-03T16:52:45.359 INFO:teuthology.task.kernel:Running kernel on smithi115: 4.18.0-305.10.2.el8_4.x86_64 2021-11-03T16:52:45.359 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum install -y kernel 2021-11-03T16:52:47.666 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS-8 - AppStream 36 MB/s | 9.5 MB 00:00 2021-11-03T16:52:49.324 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS-8 - Base 38 MB/s | 7.5 MB 00:00 2021-11-03T16:52:50.389 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS-8 - Extras 152 kB/s | 10 kB 00:00 2021-11-03T16:52:51.988 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Linux 8 - AppStream 6.1 MB/s | 9.5 MB 00:01 2021-11-03T16:52:54.104 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Linux 8 - BaseOS 12 MB/s | 7.5 MB 00:00 2021-11-03T16:52:55.445 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Linux 8 - Extras 31 kB/s | 10 kB 00:00 2021-11-03T16:52:55.621 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS-8 - PowerTools 20 MB/s | 2.4 MB 00:00 2021-11-03T16:52:56.120 INFO:teuthology.orchestra.run.smithi115.stdout:Copr repo for python3-asyncssh owned by ceph 44 kB/s | 3.5 kB 00:00 2021-11-03T16:52:56.899 INFO:teuthology.orchestra.run.smithi115.stdout:Extra Packages for Enterprise Linux 15 MB/s | 11 MB 00:00 2021-11-03T16:52:59.095 INFO:teuthology.orchestra.run.smithi115.stdout:lab-extras 4.6 MB/s | 24 kB 00:00 2021-11-03T16:53:01.097 INFO:teuthology.orchestra.run.smithi115.stdout:Package kernel-4.18.0-305.10.2.el8_4.x86_64 is already installed. 2021-11-03T16:53:01.097 INFO:teuthology.orchestra.run.smithi115.stdout:Package kernel-4.18.0-240.el8.x86_64 is already installed. 2021-11-03T16:53:01.415 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2021-11-03T16:53:01.416 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T16:53:01.416 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2021-11-03T16:53:01.417 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T16:53:01.417 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2021-11-03T16:53:01.417 INFO:teuthology.orchestra.run.smithi115.stdout: kernel x86_64 4.18.0-305.19.1.el8_4 CentOS-Base 5.9 M 2021-11-03T16:53:01.417 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2021-11-03T16:53:01.418 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-core x86_64 4.18.0-305.19.1.el8_4 CentOS-Base 36 M 2021-11-03T16:53:01.418 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-modules x86_64 4.18.0-305.19.1.el8_4 CentOS-Base 28 M 2021-11-03T16:53:01.418 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T16:53:01.418 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2021-11-03T16:53:01.418 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T16:53:01.419 INFO:teuthology.orchestra.run.smithi115.stdout:Install 3 Packages 2021-11-03T16:53:01.419 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T16:53:01.419 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 70 M 2021-11-03T16:53:01.419 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 88 M 2021-11-03T16:53:01.420 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2021-11-03T16:53:01.617 INFO:teuthology.orchestra.run.smithi115.stdout:(1/3): kernel-4.18.0-305.19.1.el8_4.x86_64.rpm 29 MB/s | 5.9 MB 00:00 2021-11-03T16:53:02.521 INFO:teuthology.orchestra.run.smithi115.stdout:(2/3): kernel-modules-4.18.0-305.19.1.el8_4.x86 25 MB/s | 28 MB 00:01 2021-11-03T16:53:02.834 INFO:teuthology.orchestra.run.smithi115.stdout:(3/3): kernel-core-4.18.0-305.19.1.el8_4.x86_64 25 MB/s | 36 MB 00:01 2021-11-03T16:53:02.838 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2021-11-03T16:53:02.838 INFO:teuthology.orchestra.run.smithi115.stdout:Total 49 MB/s | 70 MB 00:01 2021-11-03T16:53:02.950 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2021-11-03T16:53:03.276 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2021-11-03T16:53:03.277 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2021-11-03T16:53:04.466 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2021-11-03T16:53:04.499 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2021-11-03T16:53:06.930 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2021-11-03T16:53:07.437 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : kernel-core-4.18.0-305.19.1.el8_4.x86_64 1/3 2021-11-03T16:53:10.645 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-core-4.18.0-305.19.1.el8_4.x86_64 1/3 2021-11-03T16:53:10.987 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : kernel-modules-4.18.0-305.19.1.el8_4.x86_64 2/3 2021-11-03T16:53:13.955 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-modules-4.18.0-305.19.1.el8_4.x86_64 2/3 2021-11-03T16:53:14.163 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : kernel-4.18.0-305.19.1.el8_4.x86_64 3/3 2021-11-03T16:53:34.622 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-core-4.18.0-305.19.1.el8_4.x86_64 3/3 2021-11-03T16:53:34.954 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: kernel-4.18.0-305.19.1.el8_4.x86_64 3/3 2021-11-03T16:53:34.954 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : kernel-4.18.0-305.19.1.el8_4.x86_64 1/3 2021-11-03T16:53:34.955 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : kernel-core-4.18.0-305.19.1.el8_4.x86_64 2/3 2021-11-03T16:53:35.353 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : kernel-modules-4.18.0-305.19.1.el8_4.x86_64 3/3 2021-11-03T16:53:35.354 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T16:53:35.354 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2021-11-03T16:53:35.354 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-4.18.0-305.19.1.el8_4.x86_64 2021-11-03T16:53:35.355 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-core-4.18.0-305.19.1.el8_4.x86_64 2021-11-03T16:53:35.355 INFO:teuthology.orchestra.run.smithi115.stdout: kernel-modules-4.18.0-305.19.1.el8_4.x86_64 2021-11-03T16:53:35.355 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T16:53:35.355 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2021-11-03T16:53:35.415 DEBUG:teuthology.orchestra.run.smithi115:> rpm -q kernel --last | head -n 1 2021-11-03T16:53:35.557 INFO:teuthology.orchestra.run.smithi115.stdout:kernel-4.18.0-305.19.1.el8_4.x86_64 Wed 03 Nov 2021 04:53:13 PM UTC 2021-11-03T16:53:35.558 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-305.19.1.el8_4.x86_64 2021-11-03T16:53:35.559 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-305.10.2.el8_4.x86_64 Expected: 4.18.0-305.19.1.el8_4.x86_64 2021-11-03T16:53:35.559 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2021-11-03T16:53:35.560 DEBUG:teuthology.task.kernel:src is distro, skipping download 2021-11-03T16:53:35.560 DEBUG:teuthology.task.kernel:src is distro, skipping download 2021-11-03T16:53:35.561 INFO:teuthology.task.kernel:Installing distro kernel on host.a... 2021-11-03T16:53:35.561 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi027.front.sepia.ceph.com, path=None, version=distro) 2021-11-03T16:53:35.562 DEBUG:teuthology.orchestra.run.smithi027:> rpm -q kernel --last | head -n 1 2021-11-03T16:53:35.708 INFO:teuthology.orchestra.run.smithi027.stdout:kernel-4.18.0-305.19.1.el8_4.x86_64 Wed 03 Nov 2021 04:52:21 PM UTC 2021-11-03T16:53:35.710 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-305.19.1.el8_4.x86_64 2021-11-03T16:53:35.710 DEBUG:teuthology.orchestra.run.smithi027:> sudo rpm -qi grub2-tools 2021-11-03T16:53:35.741 INFO:teuthology.orchestra.run.smithi027.stdout:Name : grub2-tools 2021-11-03T16:53:35.742 INFO:teuthology.orchestra.run.smithi027.stdout:Epoch : 1 2021-11-03T16:53:35.742 INFO:teuthology.orchestra.run.smithi027.stdout:Version : 2.02 2021-11-03T16:53:35.742 INFO:teuthology.orchestra.run.smithi027.stdout:Release : 90.el8 2021-11-03T16:53:35.743 INFO:teuthology.orchestra.run.smithi027.stdout:Architecture: x86_64 2021-11-03T16:53:35.743 INFO:teuthology.orchestra.run.smithi027.stdout:Install Date: Wed 11 Aug 2021 06:38:04 PM UTC 2021-11-03T16:53:35.743 INFO:teuthology.orchestra.run.smithi027.stdout:Group : System Environment/Base 2021-11-03T16:53:35.743 INFO:teuthology.orchestra.run.smithi027.stdout:Size : 9197207 2021-11-03T16:53:35.743 INFO:teuthology.orchestra.run.smithi027.stdout:License : GPLv3+ 2021-11-03T16:53:35.744 INFO:teuthology.orchestra.run.smithi027.stdout:Signature : RSA/SHA256, Tue 08 Sep 2020 07:14:39 PM UTC, Key ID 05b555b38483c65d 2021-11-03T16:53:35.744 INFO:teuthology.orchestra.run.smithi027.stdout:Source RPM : grub2-2.02-90.el8.src.rpm 2021-11-03T16:53:35.744 INFO:teuthology.orchestra.run.smithi027.stdout:Build Date : Tue 08 Sep 2020 07:00:29 PM UTC 2021-11-03T16:53:35.744 INFO:teuthology.orchestra.run.smithi027.stdout:Build Host : kbuilder.bsys.centos.org 2021-11-03T16:53:35.745 INFO:teuthology.orchestra.run.smithi027.stdout:Relocations : (not relocatable) 2021-11-03T16:53:35.745 INFO:teuthology.orchestra.run.smithi027.stdout:Packager : CentOS BuildSystem 2021-11-03T16:53:35.745 INFO:teuthology.orchestra.run.smithi027.stdout:Vendor : CentOS 2021-11-03T16:53:35.745 INFO:teuthology.orchestra.run.smithi027.stdout:URL : http://www.gnu.org/software/grub/ 2021-11-03T16:53:35.746 INFO:teuthology.orchestra.run.smithi027.stdout:Summary : Support tools for GRUB. 2021-11-03T16:53:35.746 INFO:teuthology.orchestra.run.smithi027.stdout:Description : 2021-11-03T16:53:35.746 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T16:53:35.746 INFO:teuthology.orchestra.run.smithi027.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2021-11-03T16:53:35.746 INFO:teuthology.orchestra.run.smithi027.stdout:customizable bootloader with modular architecture. It supports a rich 2021-11-03T16:53:35.747 INFO:teuthology.orchestra.run.smithi027.stdout:variety of kernel formats, file systems, computer architectures and 2021-11-03T16:53:35.747 INFO:teuthology.orchestra.run.smithi027.stdout:hardware devices. 2021-11-03T16:53:35.747 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T16:53:35.747 INFO:teuthology.orchestra.run.smithi027.stdout:This subpackage provides tools for support of all platforms. 2021-11-03T16:53:35.749 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2021-11-03T16:53:35.749 INFO:teuthology.task.kernel:Updating grub on smithi027 to boot 4.18.0-305.19.1.el8_4.x86_64 2021-11-03T16:53:35.749 DEBUG:teuthology.orchestra.run.smithi027:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2021-11-03T16:53:36.633 INFO:teuthology.orchestra.run.smithi027.stderr:Generating grub configuration file ... 2021-11-03T16:53:38.420 INFO:teuthology.orchestra.run.smithi027.stderr:done 2021-11-03T16:53:38.422 DEBUG:teuthology.orchestra.run.smithi027:> mktemp 2021-11-03T16:53:38.467 INFO:teuthology.orchestra.run.smithi027.stdout:/tmp/tmp.yxqVxxzpNP 2021-11-03T16:53:38.467 DEBUG:teuthology.orchestra.run.smithi027:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.yxqVxxzpNP 2021-11-03T16:53:38.549 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0666 /tmp/tmp.yxqVxxzpNP 2021-11-03T16:53:38.700 DEBUG:teuthology.orchestra.remote:smithi027:/tmp/tmp.yxqVxxzpNP is 6KB 2021-11-03T16:53:38.749 DEBUG:teuthology.orchestra.run.smithi027:> rm -fr /tmp/tmp.yxqVxxzpNP 2021-11-03T16:53:38.763 DEBUG:teuthology.orchestra.run.smithi027:> sudo /bin/ls /boot/loader/entries || true 2021-11-03T16:53:38.836 INFO:teuthology.orchestra.run.smithi027.stdout:ecc66dffcd264e5181894e69c53ff13c-0-rescue.conf 2021-11-03T16:53:38.836 INFO:teuthology.orchestra.run.smithi027.stdout:ecc66dffcd264e5181894e69c53ff13c-4.18.0-240.el8.x86_64.conf 2021-11-03T16:53:38.837 INFO:teuthology.orchestra.run.smithi027.stdout:ecc66dffcd264e5181894e69c53ff13c-4.18.0-305.10.2.el8_4.x86_64.conf 2021-11-03T16:53:38.837 INFO:teuthology.orchestra.run.smithi027.stdout:ecc66dffcd264e5181894e69c53ff13c-4.18.0-305.19.1.el8_4.x86_64.conf 2021-11-03T16:53:38.838 DEBUG:teuthology.orchestra.run.smithi027:> sudo grub2-set-default ecc66dffcd264e5181894e69c53ff13c-4.18.0-305.19.1.el8_4.x86_64 2021-11-03T16:53:38.979 DEBUG:teuthology.orchestra.run.smithi027:> sudo shutdown -r now 2021-11-03T16:53:38.982 INFO:teuthology.task.kernel:Installing distro kernel on host.b... 2021-11-03T16:53:38.983 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi115.front.sepia.ceph.com, path=None, version=distro) 2021-11-03T16:53:38.983 DEBUG:teuthology.orchestra.run.smithi115:> rpm -q kernel --last | head -n 1 2021-11-03T16:53:39.122 INFO:teuthology.orchestra.run.smithi115.stdout:kernel-4.18.0-305.19.1.el8_4.x86_64 Wed 03 Nov 2021 04:53:13 PM UTC 2021-11-03T16:53:39.124 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-305.19.1.el8_4.x86_64 2021-11-03T16:53:39.125 DEBUG:teuthology.orchestra.run.smithi115:> sudo rpm -qi grub2-tools 2021-11-03T16:53:39.154 INFO:teuthology.orchestra.run.smithi115.stdout:Name : grub2-tools 2021-11-03T16:53:39.155 INFO:teuthology.orchestra.run.smithi115.stdout:Epoch : 1 2021-11-03T16:53:39.155 INFO:teuthology.orchestra.run.smithi115.stdout:Version : 2.02 2021-11-03T16:53:39.155 INFO:teuthology.orchestra.run.smithi115.stdout:Release : 90.el8 2021-11-03T16:53:39.155 INFO:teuthology.orchestra.run.smithi115.stdout:Architecture: x86_64 2021-11-03T16:53:39.156 INFO:teuthology.orchestra.run.smithi115.stdout:Install Date: Wed 11 Aug 2021 06:38:04 PM UTC 2021-11-03T16:53:39.156 INFO:teuthology.orchestra.run.smithi115.stdout:Group : System Environment/Base 2021-11-03T16:53:39.156 INFO:teuthology.orchestra.run.smithi115.stdout:Size : 9197207 2021-11-03T16:53:39.156 INFO:teuthology.orchestra.run.smithi115.stdout:License : GPLv3+ 2021-11-03T16:53:39.157 INFO:teuthology.orchestra.run.smithi115.stdout:Signature : RSA/SHA256, Tue 08 Sep 2020 07:14:39 PM UTC, Key ID 05b555b38483c65d 2021-11-03T16:53:39.157 INFO:teuthology.orchestra.run.smithi115.stdout:Source RPM : grub2-2.02-90.el8.src.rpm 2021-11-03T16:53:39.157 INFO:teuthology.orchestra.run.smithi115.stdout:Build Date : Tue 08 Sep 2020 07:00:29 PM UTC 2021-11-03T16:53:39.157 INFO:teuthology.orchestra.run.smithi115.stdout:Build Host : kbuilder.bsys.centos.org 2021-11-03T16:53:39.158 INFO:teuthology.orchestra.run.smithi115.stdout:Relocations : (not relocatable) 2021-11-03T16:53:39.158 INFO:teuthology.orchestra.run.smithi115.stdout:Packager : CentOS BuildSystem 2021-11-03T16:53:39.158 INFO:teuthology.orchestra.run.smithi115.stdout:Vendor : CentOS 2021-11-03T16:53:39.158 INFO:teuthology.orchestra.run.smithi115.stdout:URL : http://www.gnu.org/software/grub/ 2021-11-03T16:53:39.158 INFO:teuthology.orchestra.run.smithi115.stdout:Summary : Support tools for GRUB. 2021-11-03T16:53:39.159 INFO:teuthology.orchestra.run.smithi115.stdout:Description : 2021-11-03T16:53:39.159 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T16:53:39.159 INFO:teuthology.orchestra.run.smithi115.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2021-11-03T16:53:39.159 INFO:teuthology.orchestra.run.smithi115.stdout:customizable bootloader with modular architecture. It supports a rich 2021-11-03T16:53:39.160 INFO:teuthology.orchestra.run.smithi115.stdout:variety of kernel formats, file systems, computer architectures and 2021-11-03T16:53:39.160 INFO:teuthology.orchestra.run.smithi115.stdout:hardware devices. 2021-11-03T16:53:39.160 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T16:53:39.160 INFO:teuthology.orchestra.run.smithi115.stdout:This subpackage provides tools for support of all platforms. 2021-11-03T16:53:39.161 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2021-11-03T16:53:39.162 INFO:teuthology.task.kernel:Updating grub on smithi115 to boot 4.18.0-305.19.1.el8_4.x86_64 2021-11-03T16:53:39.162 DEBUG:teuthology.orchestra.run.smithi115:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2021-11-03T16:53:40.671 INFO:teuthology.orchestra.run.smithi115.stderr:Generating grub configuration file ... 2021-11-03T16:53:42.590 INFO:teuthology.orchestra.run.smithi115.stderr:done 2021-11-03T16:53:42.591 DEBUG:teuthology.orchestra.run.smithi115:> mktemp 2021-11-03T16:53:42.639 INFO:teuthology.orchestra.run.smithi115.stdout:/tmp/tmp.ZvXiAGz7H2 2021-11-03T16:53:42.640 DEBUG:teuthology.orchestra.run.smithi115:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.ZvXiAGz7H2 2021-11-03T16:53:42.671 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0666 /tmp/tmp.ZvXiAGz7H2 2021-11-03T16:53:42.825 DEBUG:teuthology.orchestra.remote:smithi115:/tmp/tmp.ZvXiAGz7H2 is 6KB 2021-11-03T16:53:42.873 DEBUG:teuthology.orchestra.run.smithi115:> rm -fr /tmp/tmp.ZvXiAGz7H2 2021-11-03T16:53:42.887 DEBUG:teuthology.orchestra.run.smithi115:> sudo /bin/ls /boot/loader/entries || true 2021-11-03T16:53:42.955 INFO:teuthology.orchestra.run.smithi115.stdout:ecc66dffcd264e5181894e69c53ff13c-0-rescue.conf 2021-11-03T16:53:42.956 INFO:teuthology.orchestra.run.smithi115.stdout:ecc66dffcd264e5181894e69c53ff13c-4.18.0-240.el8.x86_64.conf 2021-11-03T16:53:42.956 INFO:teuthology.orchestra.run.smithi115.stdout:ecc66dffcd264e5181894e69c53ff13c-4.18.0-305.10.2.el8_4.x86_64.conf 2021-11-03T16:53:42.956 INFO:teuthology.orchestra.run.smithi115.stdout:ecc66dffcd264e5181894e69c53ff13c-4.18.0-305.19.1.el8_4.x86_64.conf 2021-11-03T16:53:42.957 DEBUG:teuthology.orchestra.run.smithi115:> sudo grub2-set-default ecc66dffcd264e5181894e69c53ff13c-4.18.0-305.19.1.el8_4.x86_64 2021-11-03T16:53:43.130 DEBUG:teuthology.orchestra.run.smithi115:> sudo shutdown -r now 2021-11-03T16:54:13.162 INFO:teuthology.misc:Re-opening connections... 2021-11-03T16:54:13.163 INFO:teuthology.misc:trying to connect to ubuntu@smithi027.front.sepia.ceph.com 2021-11-03T16:54:13.164 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2021-11-03T16:54:13.165 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-11-03T16:55:31.687 DEBUG:teuthology.orchestra.remote:timed out 2021-11-03T16:55:31.689 INFO:teuthology.misc:trying to connect to ubuntu@smithi115.front.sepia.ceph.com 2021-11-03T16:55:31.690 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2021-11-03T16:55:31.691 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-11-03T16:55:32.019 DEBUG:teuthology.orchestra.run.smithi115:> true 2021-11-03T16:55:32.379 DEBUG:teuthology.misc:waited 79.21583342552185 2021-11-03T16:55:33.381 INFO:teuthology.misc:trying to connect to ubuntu@smithi027.front.sepia.ceph.com 2021-11-03T16:55:33.381 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2021-11-03T16:55:33.382 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-11-03T16:55:33.725 DEBUG:teuthology.orchestra.run.smithi027:> true 2021-11-03T16:55:34.002 DEBUG:teuthology.misc:waited 80.83830690383911 2021-11-03T16:55:35.003 INFO:teuthology.task.kernel:Checking client host.a for new kernel version... 2021-11-03T16:55:35.003 INFO:teuthology.task.kernel:Checking kernel version of host.a, want "4.18.0-305.19.1.el8_4.x86_64"... 2021-11-03T16:55:35.004 DEBUG:teuthology.orchestra.run.smithi027:> uname -r 2021-11-03T16:55:35.021 INFO:teuthology.orchestra.run.smithi027.stdout:4.18.0-305.19.1.el8_4.x86_64 2021-11-03T16:55:35.022 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-305.19.1.el8_4.x86_64 vs 4.18.0-305.19.1.el8_4.x86_64 2021-11-03T16:55:35.022 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2021-11-03T16:55:35.022 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2021-11-03T16:55:35.023 INFO:teuthology.task.kernel:Checking client host.b for new kernel version... 2021-11-03T16:55:35.023 INFO:teuthology.task.kernel:Checking kernel version of host.b, want "4.18.0-305.19.1.el8_4.x86_64"... 2021-11-03T16:55:35.023 DEBUG:teuthology.orchestra.run.smithi115:> uname -r 2021-11-03T16:55:35.094 INFO:teuthology.orchestra.run.smithi115.stdout:4.18.0-305.19.1.el8_4.x86_64 2021-11-03T16:55:35.095 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-305.19.1.el8_4.x86_64 vs 4.18.0-305.19.1.el8_4.x86_64 2021-11-03T16:55:35.095 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2021-11-03T16:55:35.095 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2021-11-03T16:55:36.097 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2021-11-03T16:55:36.097 DEBUG:teuthology.orchestra.run.smithi027:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2021-11-03T16:55:36.171 INFO:teuthology.orchestra.run.smithi027.stdout:ttyS1 2021-11-03T16:55:36.182 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2021-11-03T16:55:36.183 DEBUG:teuthology.orchestra.run.smithi115:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2021-11-03T16:55:36.266 INFO:teuthology.orchestra.run.smithi115.stdout:ttyS1 2021-11-03T16:55:36.311 INFO:teuthology.run_tasks:Running task internal.base... 2021-11-03T16:55:36.322 INFO:teuthology.task.internal:Creating test directory... 2021-11-03T16:55:36.323 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2021-11-03T16:55:36.325 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2021-11-03T16:55:36.340 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2021-11-03T16:55:36.367 INFO:teuthology.run_tasks:Running task internal.archive... 2021-11-03T16:55:36.451 INFO:teuthology.task.internal:Creating archive directory... 2021-11-03T16:55:36.451 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2021-11-03T16:55:36.454 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2021-11-03T16:55:36.494 INFO:teuthology.run_tasks:Running task internal.coredump... 2021-11-03T16:55:36.538 INFO:teuthology.task.internal:Enabling coredump saving... 2021-11-03T16:55:36.539 DEBUG:teuthology.orchestra.run.smithi027:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2021-11-03T16:55:36.542 DEBUG:teuthology.orchestra.run.smithi115:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2021-11-03T16:55:36.567 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-11-03T16:55:36.570 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-11-03T16:55:36.577 INFO:teuthology.orchestra.run.smithi115.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-11-03T16:55:36.581 INFO:teuthology.orchestra.run.smithi027.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2021-11-03T16:55:36.583 INFO:teuthology.run_tasks:Running task internal.sudo... 2021-11-03T16:55:36.593 INFO:teuthology.task.internal:Configuring sudo... 2021-11-03T16:55:36.594 DEBUG:teuthology.orchestra.run.smithi027:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2021-11-03T16:55:36.625 DEBUG:teuthology.orchestra.run.smithi115:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2021-11-03T16:55:36.650 INFO:teuthology.run_tasks:Running task internal.syslog... 2021-11-03T16:55:36.669 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2021-11-03T16:55:36.670 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2021-11-03T16:55:36.693 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2021-11-03T16:55:36.707 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2021-11-03T16:55:36.789 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2021-11-03T16:55:36.838 DEBUG:teuthology.orchestra.run.smithi027:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2021-11-03T16:55:36.918 DEBUG:teuthology.orchestra.run.smithi027:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2021-11-03T16:55:36.961 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2021-11-03T16:55:36.962 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2021-11-03T16:55:37.026 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2021-11-03T16:55:37.066 DEBUG:teuthology.orchestra.run.smithi115:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2021-11-03T16:55:37.118 DEBUG:teuthology.orchestra.run.smithi115:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2021-11-03T16:55:37.197 DEBUG:teuthology.orchestra.run.smithi115:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2021-11-03T16:55:37.237 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2021-11-03T16:55:37.237 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2021-11-03T16:55:37.300 DEBUG:teuthology.orchestra.run.smithi027:> sudo service rsyslog restart 2021-11-03T16:55:37.303 DEBUG:teuthology.orchestra.run.smithi115:> sudo service rsyslog restart 2021-11-03T16:55:37.347 INFO:teuthology.orchestra.run.smithi027.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2021-11-03T16:55:37.399 INFO:teuthology.orchestra.run.smithi115.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2021-11-03T16:55:37.784 INFO:teuthology.run_tasks:Running task internal.timer... 2021-11-03T16:55:37.854 INFO:teuthology.task.internal:Starting timer... 2021-11-03T16:55:37.854 INFO:teuthology.run_tasks:Running task pcp... 2021-11-03T16:55:37.944 INFO:teuthology.run_tasks:Running task selinux... 2021-11-03T16:55:38.026 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2021-11-03T16:55:38.027 DEBUG:teuthology.orchestra.run.smithi027:> sudo service auditd rotate 2021-11-03T16:55:38.089 INFO:teuthology.orchestra.run.smithi027.stdout:Rotating logs: [ OK ] 2021-11-03T16:55:38.091 DEBUG:teuthology.orchestra.run.smithi115:> sudo service auditd rotate 2021-11-03T16:55:38.141 INFO:teuthology.orchestra.run.smithi115.stdout:Rotating logs: [ OK ] 2021-11-03T16:55:38.143 DEBUG:teuthology.task.selinux:Getting current SELinux state 2021-11-03T16:55:38.143 DEBUG:teuthology.orchestra.run.smithi027:> /usr/sbin/getenforce 2021-11-03T16:55:38.167 INFO:teuthology.orchestra.run.smithi027.stdout:Permissive 2021-11-03T16:55:38.167 DEBUG:teuthology.orchestra.run.smithi115:> /usr/sbin/getenforce 2021-11-03T16:55:38.187 INFO:teuthology.orchestra.run.smithi115.stdout:Permissive 2021-11-03T16:55:38.187 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi027.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi115.front.sepia.ceph.com': 'permissive'} 2021-11-03T16:55:38.188 DEBUG:teuthology.orchestra.run.smithi027:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|scontext=system_u:system_r:logrotate_t:s0\)' 2021-11-03T16:55:38.232 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-03T16:55:38.233 DEBUG:teuthology.orchestra.run.smithi115:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\|comm="rhsmcertd-worke"\|comm="setroubleshootd"\|comm="rpm"\|tcontext=system_u:object_r:container_runtime_exec_t:s0\|comm="ksmtuned"\|comm="sssd"\|scontext=system_u:system_r:logrotate_t:s0\)' 2021-11-03T16:55:38.255 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-03T16:55:38.256 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2021-11-03T16:55:38.257 DEBUG:teuthology.orchestra.run.smithi027:> sudo /usr/sbin/setenforce permissive 2021-11-03T16:55:38.298 DEBUG:teuthology.orchestra.run.smithi115:> sudo /usr/sbin/setenforce permissive 2021-11-03T16:55:38.323 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2021-11-03T16:55:38.335 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master was just updated or references a specific commit; assuming it is current 2021-11-03T16:55:38.335 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to origin/master 2021-11-03T16:55:38.354 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2021-11-03T16:55:38.356 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi027.front.sepia.ceph.com,smithi115.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2021-11-03T16:55:38.728 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2021-11-03T16:55:39.555 INFO:teuthology.task.ansible.out:[WARNING]: file /home/teuthworker/src/git.ceph.com_git_ceph-cm- ansible_master/roles/testnode/tasks/setup-opensuse.yml is empty and had no tasks to include 2021-11-03T16:55:42.431 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2021-11-03T16:55:42.553 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:55:42 +0000 (0:00:00.139) 0:00:00.139 **** 2021-11-03T16:55:42.564 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:55:42 +0000 (0:00:00.011) 0:00:00.151 **** 2021-11-03T16:55:43.992 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi027.front.sepia.ceph.com] 2021-11-03T16:55:44.040 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:55:44 +0000 (0:00:01.476) 0:00:01.627 **** 2021-11-03T16:55:44.307 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2021-11-03T16:55:44.308 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2021-11-03T16:55:44.328 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:55:44 +0000 (0:00:00.287) 0:00:01.914 **** 2021-11-03T16:55:44.707 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** 2021-11-03T16:55:44.708 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2021-11-03T16:55:44.729 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:55:44 +0000 (0:00:00.401) 0:00:02.316 **** 2021-11-03T16:55:44.798 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:55:44.819 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:55:44 +0000 (0:00:00.089) 0:00:02.406 **** 2021-11-03T16:55:44.993 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi115.front.sepia.ceph.com] 2021-11-03T16:55:45.040 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:55:45 +0000 (0:00:00.220) 0:00:02.627 **** 2021-11-03T16:55:45.077 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** 2021-11-03T16:55:45.079 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-11-03T16:55:45.100 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:55:45 +0000 (0:00:00.060) 0:00:02.687 **** 2021-11-03T16:55:45.231 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2021-11-03T16:55:45.233 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2021-11-03T16:55:45.256 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:55:45 +0000 (0:00:00.155) 0:00:02.842 **** 2021-11-03T16:55:45.552 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** 2021-11-03T16:55:45.554 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2021-11-03T16:55:45.573 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:55:45 +0000 (0:00:00.317) 0:00:03.160 **** 2021-11-03T16:55:45.651 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2021-11-03T16:55:45.653 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:55:45.675 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:55:45 +0000 (0:00:00.102) 0:00:03.262 **** 2021-11-03T16:55:45.860 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** 2021-11-03T16:55:45.861 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-11-03T16:55:45.881 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:55:45 +0000 (0:00:00.205) 0:00:03.468 **** 2021-11-03T16:55:46.024 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1635958545.145491-28890-12944865649533/source", "state": "file", "uid": 0} 2021-11-03T16:55:46.052 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:55:46 +0000 (0:00:00.171) 0:00:03.639 **** 2021-11-03T16:55:46.482 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Add authorized keys for the ansible user.] ************* 2021-11-03T16:55:46.484 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2021-11-03T16:55:46.486 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** changed: [smithi115.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1635958545.925735-28945-30159647771868/source", "state": "file", "uid": 0} 2021-11-03T16:55:46.515 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:55:46 +0000 (0:00:00.462) 0:00:04.101 **** 2021-11-03T16:55:46.802 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Add authorized keys for the ansible user.] ************* 2021-11-03T16:55:46.804 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "comment": null, "exclusive": true, "follow": false, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv 2021-11-03T16:55:46.806 INFO:teuthology.task.ansible.out:4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2021-11-03T16:55:46.870 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** skipping: no hosts matched 2021-11-03T16:55:46.872 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2021-11-03T16:55:46.885 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* 2021-11-03T16:55:46.887 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:55:46 +0000 (0:00:00.370) 0:00:04.472 **** 2021-11-03T16:55:47.419 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] 2021-11-03T16:55:47.427 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] 2021-11-03T16:55:47.443 INFO:teuthology.task.ansible.out: TASK [set_fact] **************************************************************** 2021-11-03T16:55:47.445 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:55:47 +0000 (0:00:00.558) 0:00:05.030 **** 2021-11-03T16:55:47.468 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"ansible_facts": {"ran_from_cephlab_playbook": true}, "changed": false} 2021-11-03T16:55:47.479 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"ansible_facts": {"ran_from_cephlab_playbook": true}, "changed": false} 2021-11-03T16:55:47.484 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2021-11-03T16:55:47.673 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:55:47 +0000 (0:00:00.229) 0:00:05.259 **** 2021-11-03T16:55:47.693 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:55:47 +0000 (0:00:00.020) 0:00:05.279 **** 2021-11-03T16:55:48.313 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi027.front.sepia.ceph.com] 2021-11-03T16:55:48.322 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] 2021-11-03T16:55:48.381 INFO:teuthology.task.ansible.out:[WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. 2021-11-03T16:55:48.383 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:55:48 +0000 (0:00:00.688) 0:00:05.968 **** 2021-11-03T16:55:48.404 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:55:48 +0000 (0:00:00.022) 0:00:05.990 **** 2021-11-03T16:55:49.022 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2021-11-03T16:55:49.024 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.187031", "end": "2021-11-03 16:55:48.999756", "rc": 0, "start": "2021-11-03 16:55:48.812725", "stderr": "", "stderr_lines": [], "stdout": "d8709f98c8dde30910ea11812e94aaab26c7240d", "stdout_lines": ["d8709f98c8dde30910ea11812e94aaab26c7240d"]} 2021-11-03T16:55:49.044 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:55:49 +0000 (0:00:00.640) 0:00:06.630 **** 2021-11-03T16:55:49.194 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.376291", "end": "2021-11-03 16:55:49.169657", "rc": 0, "start": "2021-11-03 16:55:48.793366", "stderr": "", "stderr_lines": [], "stdout": "d8709f98c8dde30910ea11812e94aaab26c7240d", "stdout_lines": ["d8709f98c8dde30910ea11812e94aaab26c7240d"]} 2021-11-03T16:55:49.217 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:55:49 +0000 (0:00:00.173) 0:00:06.804 **** 2021-11-03T16:55:49.235 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2021-11-03T16:55:49.237 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.012119", "end": "2021-11-03 16:55:49.222163", "failed_when_result": false, "rc": 0, "start": "2021-11-03 16:55:49.210044", "stderr": "", "stderr_lines": [], "stdout": "c29274177b79f72740fc714fa28b55400cd0737f", "stdout_lines": ["c29274177b79f72740fc714fa28b55400cd0737f"]} 2021-11-03T16:55:49.258 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:55:49 +0000 (0:00:00.040) 0:00:06.844 **** 2021-11-03T16:55:49.326 INFO:teuthology.task.ansible.out: TASK [Determine if we can skip users and pubkeys updates] ********************** skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:55:49.350 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:55:49 +0000 (0:00:00.092) 0:00:06.936 **** 2021-11-03T16:55:49.418 INFO:teuthology.task.ansible.out: TASK [Merge extra_admin_users into managed_admin_users] ************************ 2021-11-03T16:55:49.420 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:55:49.440 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:55:49 +0000 (0:00:00.090) 0:00:07.027 **** 2021-11-03T16:55:49.447 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2021-11-03T16:55:49.448 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.014516", "end": "2021-11-03 16:55:49.411124", "failed_when_result": false, "rc": 0, "start": "2021-11-03 16:55:49.396608", "stderr": "", "stderr_lines": [], "stdout": "c29274177b79f72740fc714fa28b55400cd0737f", "stdout_lines": ["c29274177b79f72740fc714fa28b55400cd0737f"]} 2021-11-03T16:55:49.472 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:55:49 +0000 (0:00:00.031) 0:00:07.058 **** 2021-11-03T16:55:49.500 INFO:teuthology.task.ansible.out: TASK [Remove managed_admin_users from managed_users] *************************** 2021-11-03T16:55:49.502 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:55:49.527 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:55:49 +0000 (0:00:00.054) 0:00:07.113 **** 2021-11-03T16:55:49.544 INFO:teuthology.task.ansible.out: TASK [Determine if we can skip users and pubkeys updates] ********************** 2021-11-03T16:55:49.546 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:55:49.567 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:55:49 +0000 (0:00:00.040) 0:00:07.154 **** 2021-11-03T16:55:49.594 INFO:teuthology.task.ansible.out: TASK [Filter the managed_users list] ******************************************* 2021-11-03T16:55:49.595 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:55:49.618 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:55:49 +0000 (0:00:00.050) 0:00:07.204 **** 2021-11-03T16:55:49.636 INFO:teuthology.task.ansible.out: TASK [Merge extra_admin_users into managed_admin_users] ************************ 2021-11-03T16:55:49.637 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:55:49.658 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:55:49 +0000 (0:00:00.040) 0:00:07.245 **** 2021-11-03T16:55:49.686 INFO:teuthology.task.ansible.out: TASK [Filter the managed_admin_users list] ************************************* 2021-11-03T16:55:49.688 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:55:49.709 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:55:49 +0000 (0:00:00.051) 0:00:07.296 **** 2021-11-03T16:55:49.727 INFO:teuthology.task.ansible.out: TASK [Remove managed_admin_users from managed_users] *************************** 2021-11-03T16:55:49.729 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:55:49.753 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:55:49 +0000 (0:00:00.040) 0:00:07.337 **** 2021-11-03T16:55:49.821 INFO:teuthology.task.ansible.out: TASK [Filter the managed_users list] ******************************************* 2021-11-03T16:55:49.823 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:55:49.843 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:55:49 +0000 (0:00:00.092) 0:00:07.430 **** 2021-11-03T16:55:49.913 INFO:teuthology.task.ansible.out: TASK [Filter the managed_admin_users list] ************************************* 2021-11-03T16:55:49.915 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:55:49.936 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:55:49 +0000 (0:00:00.092) 0:00:07.523 **** 2021-11-03T16:55:49.944 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2021-11-03T16:55:49.944 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-11-03T16:55:49.968 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:55:49 +0000 (0:00:00.031) 0:00:07.554 **** 2021-11-03T16:55:50.182 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-11-03T16:55:50.204 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:55:50 +0000 (0:00:00.236) 0:00:07.791 **** 2021-11-03T16:55:50.253 INFO:teuthology.task.ansible.out: TASK [Create all admin users with sudo access.] ******************************** 2021-11-03T16:55:50.254 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2021-11-03T16:55:50.441 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2021-11-03T16:55:50.494 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2021-11-03T16:55:50.643 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2021-11-03T16:55:50.706 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2021-11-03T16:55:50.832 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2021-11-03T16:55:50.906 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2021-11-03T16:55:51.023 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2021-11-03T16:55:51.107 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1104} 2021-11-03T16:55:51.223 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2021-11-03T16:55:51.318 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1105} 2021-11-03T16:55:51.414 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2021-11-03T16:55:51.520 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1106} 2021-11-03T16:55:51.615 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@desktop QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratza@desktop QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898"}, "move_home": false, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "uid": 1108} 2021-11-03T16:55:51.731 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1107} 2021-11-03T16:55:51.816 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2021-11-03T16:55:51.932 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@desktop QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/jcaratzas", "item": {"name": "jcaratzas", "ovpn": "jcaratza@desktop QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898"}, "move_home": false, "name": "jcaratzas", "shell": "/bin/bash", "state": "present", "uid": 1108} 2021-11-03T16:55:52.006 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2021-11-03T16:55:52.143 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2021-11-03T16:55:52.207 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2021-11-03T16:55:52.343 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2021-11-03T16:55:52.397 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2021-11-03T16:55:52.544 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2021-11-03T16:55:52.597 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1113} 2021-11-03T16:55:52.755 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2021-11-03T16:55:52.787 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1114} 2021-11-03T16:55:52.953 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1113} 2021-11-03T16:55:52.985 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1115} 2021-11-03T16:55:53.162 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1114} 2021-11-03T16:55:53.185 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1116} 2021-11-03T16:55:53.361 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1115} 2021-11-03T16:55:53.373 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1117} 2021-11-03T16:55:53.572 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1116} 2021-11-03T16:55:53.573 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "move_home": false, "name": "cmeno", "shell": "/bin/bash", "state": "present", "uid": 1119} 2021-11-03T16:55:53.772 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1117} 2021-11-03T16:55:53.773 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1120} 2021-11-03T16:55:53.972 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1122} 2021-11-03T16:55:53.973 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/cmeno", "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "move_home": false, "name": "cmeno", "shell": "/bin/bash", "state": "present", "uid": 1119} 2021-11-03T16:55:54.160 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1123} 2021-11-03T16:55:54.182 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1120} 2021-11-03T16:55:54.360 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1124} 2021-11-03T16:55:54.381 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1122} 2021-11-03T16:55:54.559 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1125} 2021-11-03T16:55:54.580 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1123} 2021-11-03T16:55:54.757 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1126} 2021-11-03T16:55:54.790 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1124} 2021-11-03T16:55:54.946 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1127} 2021-11-03T16:55:54.990 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1125} 2021-11-03T16:55:55.150 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1128} 2021-11-03T16:55:55.192 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1126} 2021-11-03T16:55:55.349 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1129} 2021-11-03T16:55:55.402 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1127} 2021-11-03T16:55:55.539 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1130} 2021-11-03T16:55:55.603 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1128} 2021-11-03T16:55:55.740 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1131} 2021-11-03T16:55:55.803 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1129} 2021-11-03T16:55:55.938 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1132} 2021-11-03T16:55:56.016 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1130} 2021-11-03T16:55:56.142 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1134} 2021-11-03T16:55:56.215 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1131} 2021-11-03T16:55:56.331 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1135} 2021-11-03T16:55:56.426 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1132} 2021-11-03T16:55:56.531 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1136} 2021-11-03T16:55:56.626 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1134} 2021-11-03T16:55:56.720 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1137} 2021-11-03T16:55:56.835 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1135} 2021-11-03T16:55:56.919 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1138} 2021-11-03T16:55:57.033 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1136} 2021-11-03T16:55:57.107 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-11-03T16:55:57.232 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1137} 2021-11-03T16:55:57.305 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-11-03T16:55:57.440 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1138} 2021-11-03T16:55:57.503 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1141} 2021-11-03T16:55:57.649 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1139} 2021-11-03T16:55:57.692 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1142} 2021-11-03T16:55:57.847 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1140} 2021-11-03T16:55:57.890 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1143} 2021-11-03T16:55:58.056 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1141} 2021-11-03T16:55:58.078 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1144} 2021-11-03T16:55:58.254 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1142} 2021-11-03T16:55:58.276 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1145} 2021-11-03T16:55:58.463 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1143} 2021-11-03T16:55:58.464 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1146} 2021-11-03T16:55:58.663 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-11-03T16:55:58.665 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1144} 2021-11-03T16:55:58.851 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-11-03T16:55:58.863 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1145} 2021-11-03T16:55:59.050 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1149} 2021-11-03T16:55:59.072 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1146} 2021-11-03T16:55:59.238 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1150} 2021-11-03T16:55:59.271 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1147} 2021-11-03T16:55:59.437 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1151} 2021-11-03T16:55:59.469 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1148} 2021-11-03T16:55:59.626 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1152} 2021-11-03T16:55:59.679 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1149} 2021-11-03T16:55:59.825 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1153} 2021-11-03T16:55:59.878 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1150} 2021-11-03T16:56:00.014 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@snowmon CjHOevVvBm+PwhjnhZYM7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@snowmon CjHOevVvBm+PwhjnhZYM7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1154} 2021-11-03T16:56:00.077 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1151} 2021-11-03T16:56:00.213 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1155} 2021-11-03T16:56:00.286 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1152} 2021-11-03T16:56:00.402 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1156} 2021-11-03T16:56:00.486 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1153} 2021-11-03T16:56:00.601 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1157} 2021-11-03T16:56:00.688 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@snowmon CjHOevVvBm+PwhjnhZYM7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@snowmon CjHOevVvBm+PwhjnhZYM7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1154} 2021-11-03T16:56:00.794 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1158} 2021-11-03T16:56:00.900 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1155} 2021-11-03T16:56:01.004 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1159} 2021-11-03T16:56:01.098 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1156} 2021-11-03T16:56:01.192 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1160} 2021-11-03T16:56:01.308 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1157} 2021-11-03T16:56:01.394 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1161} 2021-11-03T16:56:01.509 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1158} 2021-11-03T16:56:01.593 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1162} 2021-11-03T16:56:01.719 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1159} 2021-11-03T16:56:01.783 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1138} 2021-11-03T16:56:01.919 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1160} 2021-11-03T16:56:01.982 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1163} 2021-11-03T16:56:02.129 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1161} 2021-11-03T16:56:02.172 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1164} 2021-11-03T16:56:02.338 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1162} 2021-11-03T16:56:02.371 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1165} 2021-11-03T16:56:02.537 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1138} 2021-11-03T16:56:02.570 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1166} 2021-11-03T16:56:02.736 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1163} 2021-11-03T16:56:02.768 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1167} 2021-11-03T16:56:02.945 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1164} 2021-11-03T16:56:02.956 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1168} 2021-11-03T16:56:03.143 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1165} 2021-11-03T16:56:03.154 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1169} 2021-11-03T16:56:03.341 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1170} 2021-11-03T16:56:03.352 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1166} 2021-11-03T16:56:03.541 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1171} 2021-11-03T16:56:03.553 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1167} 2021-11-03T16:56:03.741 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1172} 2021-11-03T16:56:03.774 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1168} 2021-11-03T16:56:03.940 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1173} 2021-11-03T16:56:03.972 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1169} 2021-11-03T16:56:04.128 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1174} 2021-11-03T16:56:04.171 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1170} 2021-11-03T16:56:04.327 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1175} 2021-11-03T16:56:04.380 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1171} 2021-11-03T16:56:04.526 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1176} 2021-11-03T16:56:04.590 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1172} 2021-11-03T16:56:04.715 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1177} 2021-11-03T16:56:04.789 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1173} 2021-11-03T16:56:04.914 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1178} 2021-11-03T16:56:04.988 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1174} 2021-11-03T16:56:05.113 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1179} 2021-11-03T16:56:05.197 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1175} 2021-11-03T16:56:05.303 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1180} 2021-11-03T16:56:05.398 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1176} 2021-11-03T16:56:05.503 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1181} 2021-11-03T16:56:05.608 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1177} 2021-11-03T16:56:05.692 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1182} 2021-11-03T16:56:05.807 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1178} 2021-11-03T16:56:05.892 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1185} 2021-11-03T16:56:06.018 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1179} 2021-11-03T16:56:06.081 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1186} 2021-11-03T16:56:06.219 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1180} 2021-11-03T16:56:06.283 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1187} 2021-11-03T16:56:06.419 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1181} 2021-11-03T16:56:06.473 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1188} 2021-11-03T16:56:06.630 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1182} 2021-11-03T16:56:06.673 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1189} 2021-11-03T16:56:06.831 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1185} 2021-11-03T16:56:06.863 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1190} 2021-11-03T16:56:07.029 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1186} 2021-11-03T16:56:07.061 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1191} 2021-11-03T16:56:07.239 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1187} 2021-11-03T16:56:07.250 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1192} 2021-11-03T16:56:07.438 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1188} 2021-11-03T16:56:07.451 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1193} 2021-11-03T16:56:07.648 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1194} 2021-11-03T16:56:07.649 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1189} 2021-11-03T16:56:07.842 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1195} 2021-11-03T16:56:07.843 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1190} 2021-11-03T16:56:08.042 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1196} 2021-11-03T16:56:08.052 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1191} 2021-11-03T16:56:08.230 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1197} 2021-11-03T16:56:08.262 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1192} 2021-11-03T16:56:08.438 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1198} 2021-11-03T16:56:08.605 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1193} 2021-11-03T16:56:08.648 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1199} 2021-11-03T16:56:08.671 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1194} 2021-11-03T16:56:08.844 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1146} 2021-11-03T16:56:08.878 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1195} 2021-11-03T16:56:09.044 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1200} 2021-11-03T16:56:09.076 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1196} 2021-11-03T16:56:09.243 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1201} 2021-11-03T16:56:09.286 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1197} 2021-11-03T16:56:09.445 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1202} 2021-11-03T16:56:09.498 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1198} 2021-11-03T16:56:09.645 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1203} 2021-11-03T16:56:09.702 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1199} 2021-11-03T16:56:09.840 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1204} 2021-11-03T16:56:09.914 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1146} 2021-11-03T16:56:10.040 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1205} 2021-11-03T16:56:10.115 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 1200} 2021-11-03T16:56:10.230 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1206} 2021-11-03T16:56:10.325 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100"}, "move_home": false, "name": "rraja", "shell": "/bin/bash", "state": "present", "uid": 1201} 2021-11-03T16:56:10.440 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1207} 2021-11-03T16:56:10.525 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/toabctl", "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "move_home": false, "name": "toabctl", "shell": "/bin/bash", "state": "present", "uid": 1202} 2021-11-03T16:56:10.629 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "move_home": false, "name": "khiremat", "shell": "/bin/bash", "state": "present", "uid": 1208} 2021-11-03T16:56:10.734 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/mgfritch", "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "move_home": false, "name": "mgfritch", "shell": "/bin/bash", "state": "present", "uid": 1203} 2021-11-03T16:56:10.817 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "move_home": false, "name": "kapandya", "shell": "/bin/bash", "state": "present", "uid": 1209} 2021-11-03T16:56:10.933 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/xiubli", "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "move_home": false, "name": "xiubli", "shell": "/bin/bash", "state": "present", "uid": 1204} 2021-11-03T16:56:11.018 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "move_home": false, "name": "prasad", "shell": "/bin/bash", "state": "present", "uid": 1210} 2021-11-03T16:56:11.134 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/jafaj", "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "move_home": false, "name": "jafaj", "shell": "/bin/bash", "state": "present", "uid": 1205} 2021-11-03T16:56:11.229 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "move_home": false, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "uid": 1211} 2021-11-03T16:56:11.344 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/jschmid", "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "move_home": false, "name": "jschmid", "shell": "/bin/bash", "state": "present", "uid": 1206} 2021-11-03T16:56:11.419 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "move_home": false, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "uid": 1212} 2021-11-03T16:56:11.554 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/kachau", "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "move_home": false, "name": "kachau", "shell": "/bin/bash", "state": "present", "uid": 1207} 2021-11-03T16:56:11.618 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "move_home": false, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "uid": 1213} 2021-11-03T16:56:11.755 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/khiremat", "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "move_home": false, "name": "khiremat", "shell": "/bin/bash", "state": "present", "uid": 1208} 2021-11-03T16:56:11.808 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/prateek", "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "move_home": false, "name": "prateek", "shell": "/bin/bash", "state": "present", "uid": 1214} 2021-11-03T16:56:11.957 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/kapandya", "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "move_home": false, "name": "kapandya", "shell": "/bin/bash", "state": "present", "uid": 1209} 2021-11-03T16:56:12.009 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1215, "groups": "sudo", "home": "/home/jelopez", "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "move_home": false, "name": "jelopez", "shell": "/bin/bash", "state": "present", "uid": 1215} 2021-11-03T16:56:12.155 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/prasad", "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "move_home": false, "name": "prasad", "shell": "/bin/bash", "state": "present", "uid": 1210} 2021-11-03T16:56:12.198 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1216, "groups": "sudo", "home": "/home/dcain", "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "move_home": false, "name": "dcain", "shell": "/bin/bash", "state": "present", "uid": 1216} 2021-11-03T16:56:12.365 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/vikhyat", "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "move_home": false, "name": "vikhyat", "shell": "/bin/bash", "state": "present", "uid": 1211} 2021-11-03T16:56:12.399 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1217, "groups": "sudo", "home": "/home/sunny", "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "move_home": false, "name": "sunny", "shell": "/bin/bash", "state": "present", "uid": 1217} 2021-11-03T16:56:12.576 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/shraddhaag", "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "move_home": false, "name": "shraddhaag", "shell": "/bin/bash", "state": "present", "uid": 1212} 2021-11-03T16:56:12.598 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1218, "groups": "sudo", "home": "/home/xuxuehan", "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "move_home": false, "name": "xuxuehan", "shell": "/bin/bash", "state": "present", "uid": 1218} 2021-11-03T16:56:12.775 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/gkyratsas", "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "move_home": false, "name": "gkyratsas", "shell": "/bin/bash", "state": "present", "uid": 1213} 2021-11-03T16:56:12.798 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1219, "groups": "sudo", "home": "/home/aclewett", "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "move_home": false, "name": "aclewett", "shell": "/bin/bash", "state": "present", "uid": 1219} 2021-11-03T16:56:12.988 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/prateek", "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "move_home": false, "name": "prateek", "shell": "/bin/bash", "state": "present", "uid": 1214} 2021-11-03T16:56:12.989 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1220, "groups": "sudo", "home": "/home/cblum", "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "move_home": false, "name": "cblum", "shell": "/bin/bash", "state": "present", "uid": 1220} 2021-11-03T16:56:13.190 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1221, "groups": "sudo", "home": "/home/mlatimer", "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "move_home": false, "name": "mlatimer", "shell": "/bin/bash", "state": "present", "uid": 1221} 2021-11-03T16:56:13.192 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1215, "groups": "sudo", "home": "/home/jelopez", "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "move_home": false, "name": "jelopez", "shell": "/bin/bash", "state": "present", "uid": 1215} 2021-11-03T16:56:13.380 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1222, "groups": "sudo", "home": "/home/shyamsundarr", "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "move_home": false, "name": "shyamsundarr", "shell": "/bin/bash", "state": "present", "uid": 1222} 2021-11-03T16:56:13.393 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1216, "groups": "sudo", "home": "/home/dcain", "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "move_home": false, "name": "dcain", "shell": "/bin/bash", "state": "present", "uid": 1216} 2021-11-03T16:56:13.595 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1223, "groups": "sudo", "home": "/home/shuzhenyi", "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "move_home": false, "name": "shuzhenyi", "shell": "/bin/bash", "state": "present", "uid": 1223} 2021-11-03T16:56:13.599 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1217, "groups": "sudo", "home": "/home/sunny", "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "move_home": false, "name": "sunny", "shell": "/bin/bash", "state": "present", "uid": 1217} 2021-11-03T16:56:13.804 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1224, "groups": "sudo", "home": "/home/mdipalma", "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "move_home": false, "name": "mdipalma", "shell": "/bin/bash", "state": "present", "uid": 1224} 2021-11-03T16:56:13.824 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1218, "groups": "sudo", "home": "/home/xuxuehan", "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "move_home": false, "name": "xuxuehan", "shell": "/bin/bash", "state": "present", "uid": 1218} 2021-11-03T16:56:14.004 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1225, "groups": "sudo", "home": "/home/kchang", "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "move_home": false, "name": "kchang", "shell": "/bin/bash", "state": "present", "uid": 1225} 2021-11-03T16:56:14.026 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1219, "groups": "sudo", "home": "/home/aclewett", "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "move_home": false, "name": "aclewett", "shell": "/bin/bash", "state": "present", "uid": 1219} 2021-11-03T16:56:14.194 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1226, "groups": "sudo", "home": "/home/vereddy", "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "move_home": false, "name": "vereddy", "shell": "/bin/bash", "state": "present", "uid": 1226} 2021-11-03T16:56:14.227 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1220, "groups": "sudo", "home": "/home/cblum", "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "move_home": false, "name": "cblum", "shell": "/bin/bash", "state": "present", "uid": 1220} 2021-11-03T16:56:14.405 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1227, "groups": "sudo", "home": "/home/benhanokh", "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "move_home": false, "name": "benhanokh", "shell": "/bin/bash", "state": "present", "uid": 1227} 2021-11-03T16:56:14.427 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1221, "groups": "sudo", "home": "/home/mlatimer", "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "move_home": false, "name": "mlatimer", "shell": "/bin/bash", "state": "present", "uid": 1221} 2021-11-03T16:56:14.599 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1228, "groups": "sudo", "home": "/home/jsalomon", "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "move_home": false, "name": "jsalomon", "shell": "/bin/bash", "state": "present", "uid": 1228} 2021-11-03T16:56:14.632 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1222, "groups": "sudo", "home": "/home/shyamsundarr", "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "move_home": false, "name": "shyamsundarr", "shell": "/bin/bash", "state": "present", "uid": 1222} 2021-11-03T16:56:14.791 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1229, "groups": "sudo", "home": "/home/jolmomar", "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "move_home": false, "name": "jolmomar", "shell": "/bin/bash", "state": "present", "uid": 1229} 2021-11-03T16:56:14.834 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1223, "groups": "sudo", "home": "/home/shuzhenyi", "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "move_home": false, "name": "shuzhenyi", "shell": "/bin/bash", "state": "present", "uid": 1223} 2021-11-03T16:56:14.983 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1231, "groups": "sudo", "home": "/home/julpark", "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "move_home": false, "name": "julpark", "shell": "/bin/bash", "state": "present", "uid": 1231} 2021-11-03T16:56:15.046 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1224, "groups": "sudo", "home": "/home/mdipalma", "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "move_home": false, "name": "mdipalma", "shell": "/bin/bash", "state": "present", "uid": 1224} 2021-11-03T16:56:15.183 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1232, "groups": "sudo", "home": "/home/pdhange", "item": {"name": "pdhange", "ovpn": "pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535"}, "move_home": false, "name": "pdhange", "shell": "/bin/bash", "state": "present", "uid": 1232} 2021-11-03T16:56:15.247 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1225, "groups": "sudo", "home": "/home/kchang", "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "move_home": false, "name": "kchang", "shell": "/bin/bash", "state": "present", "uid": 1225} 2021-11-03T16:56:15.374 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1233, "groups": "sudo", "home": "/home/dpivonka", "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4"}, "move_home": false, "name": "dpivonka", "shell": "/bin/bash", "state": "present", "uid": 1233} 2021-11-03T16:56:15.448 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1226, "groups": "sudo", "home": "/home/vereddy", "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "move_home": false, "name": "vereddy", "shell": "/bin/bash", "state": "present", "uid": 1226} 2021-11-03T16:56:15.574 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1234, "groups": "sudo", "home": "/home/adking", "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "move_home": false, "name": "adking", "shell": "/bin/bash", "state": "present", "uid": 1234} 2021-11-03T16:56:15.659 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1227, "groups": "sudo", "home": "/home/benhanokh", "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "move_home": false, "name": "benhanokh", "shell": "/bin/bash", "state": "present", "uid": 1227} 2021-11-03T16:56:15.775 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1235, "groups": "sudo", "home": "/home/amathuri", "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "move_home": false, "name": "amathuri", "shell": "/bin/bash", "state": "present", "uid": 1235} 2021-11-03T16:56:15.859 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1228, "groups": "sudo", "home": "/home/jsalomon", "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "move_home": false, "name": "jsalomon", "shell": "/bin/bash", "state": "present", "uid": 1228} 2021-11-03T16:56:15.964 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a28ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1236, "groups": "sudo", "home": "/home/nizamial09", "item": {"name": "nizamial09", "ovpn": "nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a28ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409"}, "move_home": false, "name": "nizamial09", "shell": "/bin/bash", "state": "present", "uid": 1236} 2021-11-03T16:56:16.081 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1229, "groups": "sudo", "home": "/home/jolmomar", "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "move_home": false, "name": "jolmomar", "shell": "/bin/bash", "state": "present", "uid": 1229} 2021-11-03T16:56:16.168 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1237, "groups": "sudo", "home": "/home/dachary", "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "move_home": false, "name": "dachary", "shell": "/bin/bash", "state": "present", "uid": 1237} 2021-11-03T16:56:16.286 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1231, "groups": "sudo", "home": "/home/julpark", "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "move_home": false, "name": "julpark", "shell": "/bin/bash", "state": "present", "uid": 1231} 2021-11-03T16:56:16.369 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'amctagga', 'ovpn': 'amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1238, "groups": "sudo", "home": "/home/amctagga", "item": {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, "move_home": false, "name": "amctagga", "shell": "/bin/bash", "state": "present", "uid": 1238} 2021-11-03T16:56:16.495 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1232, "groups": "sudo", "home": "/home/pdhange", "item": {"name": "pdhange", "ovpn": "pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535"}, "move_home": false, "name": "pdhange", "shell": "/bin/bash", "state": "present", "uid": 1232} 2021-11-03T16:56:16.580 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'kevinz', 'ovpn': 'kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1239, "groups": "sudo", "home": "/home/kevinz", "item": {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, "move_home": false, "name": "kevinz", "shell": "/bin/bash", "state": "present", "uid": 1239} 2021-11-03T16:56:16.695 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1233, "groups": "sudo", "home": "/home/dpivonka", "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4"}, "move_home": false, "name": "dpivonka", "shell": "/bin/bash", "state": "present", "uid": 1233} 2021-11-03T16:56:16.758 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'liu-chunmei', 'ovpn': 'liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1240, "groups": "sudo", "home": "/home/liu-chunmei", "item": {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c"}, "move_home": false, "name": "liu-chunmei", "shell": "/bin/bash", "state": "present", "uid": 1240} 2021-11-03T16:56:16.894 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1234, "groups": "sudo", "home": "/home/adking", "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "move_home": false, "name": "adking", "shell": "/bin/bash", "state": "present", "uid": 1234} 2021-11-03T16:56:16.959 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'cyx1231st', 'ovpn': 'cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1241, "groups": "sudo", "home": "/home/cyx1231st", "item": {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a"}, "move_home": false, "name": "cyx1231st", "shell": "/bin/bash", "state": "present", "uid": 1241} 2021-11-03T16:56:17.105 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1235, "groups": "sudo", "home": "/home/amathuri", "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "move_home": false, "name": "amathuri", "shell": "/bin/bash", "state": "present", "uid": 1235} 2021-11-03T16:56:17.158 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'sunnagar', 'ovpn': 'sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1242, "groups": "sudo", "home": "/home/sunnagar", "item": {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, "move_home": false, "name": "sunnagar", "shell": "/bin/bash", "state": "present", "uid": 1242} 2021-11-03T16:56:17.305 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a28ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1236, "groups": "sudo", "home": "/home/nizamial09", "item": {"name": "nizamial09", "ovpn": "nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a28ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409"}, "move_home": false, "name": "nizamial09", "shell": "/bin/bash", "state": "present", "uid": 1236} 2021-11-03T16:56:17.368 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'lflores', 'ovpn': 'lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1243, "groups": "sudo", "home": "/home/lflores", "item": {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, "move_home": false, "name": "lflores", "shell": "/bin/bash", "state": "present", "uid": 1243} 2021-11-03T16:56:17.516 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1237, "groups": "sudo", "home": "/home/dachary", "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "move_home": false, "name": "dachary", "shell": "/bin/bash", "state": "present", "uid": 1237} 2021-11-03T16:56:17.558 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'jsawaya', 'ovpn': 'jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1244, "groups": "sudo", "home": "/home/jsawaya", "item": {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, "move_home": false, "name": "jsawaya", "shell": "/bin/bash", "state": "present", "uid": 1244} 2021-11-03T16:56:17.718 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'amctagga', 'ovpn': 'amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1238, "groups": "sudo", "home": "/home/amctagga", "item": {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, "move_home": false, "name": "amctagga", "shell": "/bin/bash", "state": "present", "uid": 1238} 2021-11-03T16:56:17.760 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'markw8973240', 'ovpn': 'shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1245, "groups": "sudo", "home": "/home/markw8973240", "item": {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, "move_home": false, "name": "markw8973240", "shell": "/bin/bash", "state": "present", "uid": 1245} 2021-11-03T16:56:17.927 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'kevinz', 'ovpn': 'kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1239, "groups": "sudo", "home": "/home/kevinz", "item": {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, "move_home": false, "name": "kevinz", "shell": "/bin/bash", "state": "present", "uid": 1239} 2021-11-03T16:56:17.959 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'luisrico', 'ovpn': 'lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1246, "groups": "sudo", "home": "/home/luisrico", "item": {"name": "luisrico", "ovpn": "lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, "move_home": false, "name": "luisrico", "shell": "/bin/bash", "state": "present", "uid": 1246} 2021-11-03T16:56:18.126 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'liu-chunmei', 'ovpn': 'liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1240, "groups": "sudo", "home": "/home/liu-chunmei", "item": {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c"}, "move_home": false, "name": "liu-chunmei", "shell": "/bin/bash", "state": "present", "uid": 1240} 2021-11-03T16:56:18.148 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'matan', 'ovpn': 'matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1247, "groups": "sudo", "home": "/home/matan", "item": {"name": "matan", "ovpn": "matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n"}, "move_home": false, "name": "matan", "shell": "/bin/bash", "state": "present", "uid": 1247} 2021-11-03T16:56:18.326 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'cyx1231st', 'ovpn': 'cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1241, "groups": "sudo", "home": "/home/cyx1231st", "item": {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a"}, "move_home": false, "name": "cyx1231st", "shell": "/bin/bash", "state": "present", "uid": 1241} 2021-11-03T16:56:18.349 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'ozeneva', 'ovpn': 'ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1248, "groups": "sudo", "home": "/home/ozeneva", "item": {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, "move_home": false, "name": "ozeneva", "shell": "/bin/bash", "state": "present", "uid": 1248} 2021-11-03T16:56:18.537 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'sunnagar', 'ovpn': 'sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1242, "groups": "sudo", "home": "/home/sunnagar", "item": {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, "move_home": false, "name": "sunnagar", "shell": "/bin/bash", "state": "present", "uid": 1242} 2021-11-03T16:56:18.539 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'sudhanaruna', 'ovpn': 'ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1249, "groups": "sudo", "home": "/home/sudhanaruna", "item": {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, "move_home": false, "name": "sudhanaruna", "shell": "/bin/bash", "state": "present", "uid": 1249} 2021-11-03T16:56:18.739 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'zl31wang', 'ovpn': 'zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1250, "groups": "sudo", "home": "/home/zl31wang", "item": {"name": "zl31wang", "ovpn": "zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, "move_home": false, "name": "zl31wang", "shell": "/bin/bash", "state": "present", "uid": 1250} 2021-11-03T16:56:18.740 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'lflores', 'ovpn': 'lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1243, "groups": "sudo", "home": "/home/lflores", "item": {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, "move_home": false, "name": "lflores", "shell": "/bin/bash", "state": "present", "uid": 1243} 2021-11-03T16:56:18.937 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'mayagilad', 'ovpn': 'mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1251, "groups": "sudo", "home": "/home/mayagilad", "item": {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}, "move_home": false, "name": "mayagilad", "shell": "/bin/bash", "state": "present", "uid": 1251} 2021-11-03T16:56:18.949 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'jsawaya', 'ovpn': 'jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1244, "groups": "sudo", "home": "/home/jsawaya", "item": {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, "move_home": false, "name": "jsawaya", "shell": "/bin/bash", "state": "present", "uid": 1244} 2021-11-03T16:56:19.136 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'arjsharm', 'ovpn': 'arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1252, "groups": "sudo", "home": "/home/arjsharm", "item": {"name": "arjsharm", "ovpn": "arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39"}, "move_home": false, "name": "arjsharm", "shell": "/bin/bash", "state": "present", "uid": 1252} 2021-11-03T16:56:19.147 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'markw8973240', 'ovpn': 'shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1245, "groups": "sudo", "home": "/home/markw8973240", "item": {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, "move_home": false, "name": "markw8973240", "shell": "/bin/bash", "state": "present", "uid": 1245} 2021-11-03T16:56:19.324 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'melissali', 'ovpn': 'melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1253, "groups": "sudo", "home": "/home/melissali", "item": {"name": "melissali", "ovpn": "melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07"}, "move_home": false, "name": "melissali", "shell": "/bin/bash", "state": "present", "uid": 1253} 2021-11-03T16:56:19.356 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'luisrico', 'ovpn': 'lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1246, "groups": "sudo", "home": "/home/luisrico", "item": {"name": "luisrico", "ovpn": "lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, "move_home": false, "name": "luisrico", "shell": "/bin/bash", "state": "present", "uid": 1246} 2021-11-03T16:56:19.566 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'matan', 'ovpn': 'matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1247, "groups": "sudo", "home": "/home/matan", "item": {"name": "matan", "ovpn": "matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n"}, "move_home": false, "name": "matan", "shell": "/bin/bash", "state": "present", "uid": 1247} 2021-11-03T16:56:19.776 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'ozeneva', 'ovpn': 'ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1248, "groups": "sudo", "home": "/home/ozeneva", "item": {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, "move_home": false, "name": "ozeneva", "shell": "/bin/bash", "state": "present", "uid": 1248} 2021-11-03T16:56:19.985 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'sudhanaruna', 'ovpn': 'ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1249, "groups": "sudo", "home": "/home/sudhanaruna", "item": {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, "move_home": false, "name": "sudhanaruna", "shell": "/bin/bash", "state": "present", "uid": 1249} 2021-11-03T16:56:20.195 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'zl31wang', 'ovpn': 'zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1250, "groups": "sudo", "home": "/home/zl31wang", "item": {"name": "zl31wang", "ovpn": "zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, "move_home": false, "name": "zl31wang", "shell": "/bin/bash", "state": "present", "uid": 1250} 2021-11-03T16:56:20.394 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'mayagilad', 'ovpn': 'mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1251, "groups": "sudo", "home": "/home/mayagilad", "item": {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}, "move_home": false, "name": "mayagilad", "shell": "/bin/bash", "state": "present", "uid": 1251} 2021-11-03T16:56:20.603 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'arjsharm', 'ovpn': 'arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1252, "groups": "sudo", "home": "/home/arjsharm", "item": {"name": "arjsharm", "ovpn": "arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39"}, "move_home": false, "name": "arjsharm", "shell": "/bin/bash", "state": "present", "uid": 1252} 2021-11-03T16:56:20.801 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'melissali', 'ovpn': 'melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1253, "groups": "sudo", "home": "/home/melissali", "item": {"name": "melissali", "ovpn": "melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07"}, "move_home": false, "name": "melissali", "shell": "/bin/bash", "state": "present", "uid": 1253} 2021-11-03T16:56:20.937 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item={'name': 'ionutbalutoiu', 'ovpn': 'ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10102, "groups": "sudo", "home": "/home/ionutbalutoiu", "item": {"name": "ionutbalutoiu", "ovpn": "ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1"}, "name": "ionutbalutoiu", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10102} 2021-11-03T16:56:22.535 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'name': 'ionutbalutoiu', 'ovpn': 'ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10102, "groups": "sudo", "home": "/home/ionutbalutoiu", "item": {"name": "ionutbalutoiu", "ovpn": "ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1"}, "name": "ionutbalutoiu", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10102} 2021-11-03T16:56:22.681 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item={'name': 'pdiazbou', 'ovpn': 'pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10103, "groups": "sudo", "home": "/home/pdiazbou", "item": {"name": "pdiazbou", "ovpn": "pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4"}, "name": "pdiazbou", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2021-11-03T16:56:24.249 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item={'name': 'xuwang', 'ovpn': 'xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/xuwang", "item": {"name": "xuwang", "ovpn": "xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8"}, "name": "xuwang", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10104} 2021-11-03T16:56:24.437 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'name': 'pdiazbou', 'ovpn': 'pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10103, "groups": "sudo", "home": "/home/pdiazbou", "item": {"name": "pdiazbou", "ovpn": "pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4"}, "name": "pdiazbou", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2021-11-03T16:56:25.809 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item={'name': 'blaine', 'ovpn': 'blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10105, "groups": "sudo", "home": "/home/blaine", "item": {"name": "blaine", "ovpn": "blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547"}, "name": "blaine", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10105} 2021-11-03T16:56:25.937 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:56:25 +0000 (0:00:35.732) 0:00:43.524 **** 2021-11-03T16:56:26.006 INFO:teuthology.task.ansible.out: TASK [Create all users without sudo access.] *********************************** 2021-11-03T16:56:26.029 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:56:26 +0000 (0:00:00.091) 0:00:43.615 **** 2021-11-03T16:56:26.122 INFO:teuthology.task.ansible.out: TASK [Merge managed_users and managed_admin users] ***************************** 2021-11-03T16:56:26.125 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratza@desktop 2021-11-03T16:56:26.127 INFO:teuthology.task.ansible.out: QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b26 2021-11-03T16:56:26.129 INFO:teuthology.task.ansible.out:5c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed20 2021-11-03T16:56:26.130 INFO:teuthology.task.ansible.out:82caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name" 2021-11-03T16:56:26.132 INFO:teuthology.task.ansible.out:: "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915 2021-11-03T16:56:26.133 INFO:teuthology.task.ansible.out:d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@snowmon CjHOevVvBm+PwhjnhZ 2021-11-03T16:56:26.135 INFO:teuthology.task.ansible.out:YM7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56"}, {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a9 2021-11-03T16:56:26.136 INFO:teuthology.task.ansible.out:5579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 2021-11-03T16:56:26.137 INFO:teuthology.task.ansible.out: 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp 2021-11-03T16:56:26.139 INFO:teuthology.task.ansible.out:", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3 2021-11-03T16:56:26.140 INFO:teuthology.task.ansible.out:624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9Jj 2021-11-03T16:56:26.142 INFO:teuthology.task.ansible.out:enDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, 2021-11-03T16:56:26.142 INFO:teuthology.task.ansible.out: {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, {"name": "shyamsundarr", "ovpn": "sh 2021-11-03T16:56:26.144 INFO:teuthology.task.ansible.out:yamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, {"name": "pdhange", "ovpn": "pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535"}, {"name": "dpivonka", "ovpn": "dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4"}, {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, {"name": "nizamial09", "ovpn": "nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a 2021-11-03T16:56:26.146 INFO:teuthology.task.ansible.out:28ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409"}, {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c"}, {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a"}, {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, {"name": "luisrico", "ovpn": "lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, {"name": "matan", "ovpn": "matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n"}, {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, {"name": "zl31wang", "ovpn": " 2021-11-03T16:56:26.147 INFO:teuthology.task.ansible.out:zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}, {"name": "arjsharm", "ovpn": "arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39"}, {"name": "melissali", "ovpn": "melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07"}, {"name": "ionutbalutoiu", "ovpn": "ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1"}, {"name": "pdiazbou", "ovpn": "pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4"}, {"name": "xuwang", "ovpn": "xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8"}, {"name": "blaine", "ovpn": "blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547"}]}, "changed": false} TASK [Create all admin users with sudo access.] ******************************** changed: [smithi027.front.sepia.ceph.com] => (item={'name': 'xuwang', 'ovpn': 'xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/xuwang", "item": {"name": "xuwang", "ovpn": "xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8"}, "name": "xuwang", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10104} 2021-11-03T16:56:26.170 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:56:26 +0000 (0:00:00.141) 0:00:43.757 **** 2021-11-03T16:56:27.118 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2021-11-03T16:56:27.120 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"after": "d8709f98c8dde30910ea11812e94aaab26c7240d", "attempts": 1, "before": "d8709f98c8dde30910ea11812e94aaab26c7240d", "changed": false, "remote_url_changed": false} 2021-11-03T16:56:27.140 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:56:27 +0000 (0:00:00.970) 0:00:44.727 **** 2021-11-03T16:56:27.469 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-11-03T16:56:27.472 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDdkebJGXKQWBG/7wZG334P+EAdC+5VNMrjLjXhaZZbt3V9vKDQCodcU3zSOjMa/GDGg2WHQdO/vioCc6O/m61KTbgLDaaPSzHH16j/e1SaU8BIgKKU42AUBJzpPeaZ2wCC1i12Hq8ULwjEW29IDsnMortzlCSQmaTKc3DN2nCXZWQXA1rSiQ2AhIq5WNJY9S36lsg5tkex4xUajyNW4vPubLtkSv+WrzNK7ZJ8zfzP6eJEJ+Uj7CLdanBCgwoLGSPfGA5c/gyUT7mHibOUwvFB5sFVsozMPp2PXzUUIv70adJpOskLYfKY7TUSuaB4AUOPu6NVauzVIl3Tsn+uvHqvtn8KC3BPpyH7+JiFQWEt+aO+/ynprfoqCjktuf07SXo0ewoVYT2S++JEBa/N2hKRkzIoSPJ+Rvoazc8GZIjXpQRyhMdTV/LCSeyUTJ7lDQVrQVhUIHKnMm+hZmf4tLQKZvNAThO984CpY93ST5Ww07fjRLMXNZGuaz+zHKNexhWSPKr+ggChnZCOLl/F6zbDnBZoCaHR6hH4+QLz0K/FAgxN2Nn00RiOFkmIlzEioxMcwC/peTl3WzRfqPINdTftiTzcFi/c2QmlHhnxsueIwNhHOn4frkGkFmexdDrXRStERzSBJ13X4kWduPmQsn+bx5/7xwqC6M4bXpkYFOefOw== aschoen@redhat.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2021-11-03T16:56:27.692 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2021-11-03T16:56:27.807 INFO:teuthology.task.ansible.out: TASK [Create all admin users with sudo access.] ******************************** 2021-11-03T16:56:27.809 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'name': 'blaine', 'ovpn': 'blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10105, "groups": "sudo", "home": "/home/blaine", "item": {"name": "blaine", "ovpn": "blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547"}, "name": "blaine", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10105} 2021-11-03T16:56:27.925 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-11-03T16:56:27.926 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2021-11-03T16:56:27.947 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:56:27 +0000 (0:00:00.806) 0:00:45.534 **** 2021-11-03T16:56:28.010 INFO:teuthology.task.ansible.out: TASK [Create all users without sudo access.] *********************************** 2021-11-03T16:56:28.031 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:56:28 +0000 (0:00:00.084) 0:00:45.618 **** 2021-11-03T16:56:28.101 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw 2021-11-03T16:56:28.103 INFO:teuthology.task.ansible.out:8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZtIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} 2021-11-03T16:56:28.261 INFO:teuthology.task.ansible.out: TASK [Merge managed_users and managed_admin users] ***************************** 2021-11-03T16:56:28.262 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "jcaratzas", "ovpn": "jcaratza@desktop 2021-11-03T16:56:28.264 INFO:teuthology.task.ansible.out: QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898"}, {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b26 2021-11-03T16:56:28.266 INFO:teuthology.task.ansible.out:5c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed20 2021-11-03T16:56:28.267 INFO:teuthology.task.ansible.out:82caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name" 2021-11-03T16:56:28.269 INFO:teuthology.task.ansible.out:: "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915 2021-11-03T16:56:28.270 INFO:teuthology.task.ansible.out:d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, {"name": "ifed01", "ovpn": "ifed01@snowmon CjHOevVvBm+PwhjnhZ 2021-11-03T16:56:28.272 INFO:teuthology.task.ansible.out:YM7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56"}, {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a9 2021-11-03T16:56:28.272 INFO:teuthology.task.ansible.out:5579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 2021-11-03T16:56:28.274 INFO:teuthology.task.ansible.out: 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp 2021-11-03T16:56:28.276 INFO:teuthology.task.ansible.out:", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3 2021-11-03T16:56:28.277 INFO:teuthology.task.ansible.out:624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100"}, {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9Jj 2021-11-03T16:56:28.279 INFO:teuthology.task.ansible.out:enDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, 2021-11-03T16:56:28.280 INFO:teuthology.task.ansible.out: {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, {"name": "shyamsundarr", "ovpn": "sh 2021-11-03T16:56:28.281 INFO:teuthology.task.ansible.out:yamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, {"name": "pdhange", "ovpn": "pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535"}, {"name": "dpivonka", "ovpn": "dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4"}, {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, {"name": "nizamial09", "ovpn": "nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a 2021-11-03T16:56:28.283 INFO:teuthology.task.ansible.out:28ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409"}, {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c"}, {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a"}, {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, {"name": "luisrico", "ovpn": "lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, {"name": "matan", "ovpn": "matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n"}, {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, {"name": "zl31wang", "ovpn": " 2021-11-03T16:56:28.285 INFO:teuthology.task.ansible.out:zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}, {"name": "arjsharm", "ovpn": "arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39"}, {"name": "melissali", "ovpn": "melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07"}, {"name": "ionutbalutoiu", "ovpn": "ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1"}, {"name": "pdiazbou", "ovpn": "pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4"}, {"name": "xuwang", "ovpn": "xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8"}, {"name": "blaine", "ovpn": "blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547"}]}, "changed": false} 2021-11-03T16:56:28.307 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:56:28 +0000 (0:00:00.275) 0:00:45.893 **** 2021-11-03T16:56:28.313 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-11-03T16:56:28.314 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2021-11-03T16:56:28.513 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+ 2021-11-03T16:56:28.515 INFO:teuthology.task.ansible.out:o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2021-11-03T16:56:28.713 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2021-11-03T16:56:28.920 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@desktop QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@desktop QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAV78mqEy/VkB2BthIE/m7iE5sbTjTRS0KuH7AB0KPvZBsDIw5cL8jKyxiaPC85+aOkBa4LrP0TUFBzN87Snt+PLMz2QHBOvlSTfUuRkHVo3vWJFzVuUSzzddkYds3hAxUriAHKv6CRSy1yFBxPQQHTSATUHKWqihMElbUCZrrrfH8A7J9vwixzCSLg4KbINjfa77rxefcZwugUqiYiVUWUSOb023f+QEaDm7OP+/uzz3SjF1Y0LMwa75yctttIfkRBKjY7uFCuZdKc3l2A/fdPxbFwlZmIox3LLngdTzHMVXGbVHqq9Ixo8FSAXiI1Buy2LTMoXMJwQYSvzsv7xNwujKVlUjoE9cOOIdzCC5AziwSiJB4+tNh73c5tRUzMMC98HLOhPJriSPvOt1pMEdWDX3/evcyxvRgJUKgYnmQG7n8v8O/CLyG0sAJDmGC3U9S/3fUDUjZIKWSsc85Q3xRaJwZJIS2VZwH0zG1Gmynih8PPq7VeDx7tKuNm4uTYT8= bigjust@desktop-jsho3l9.lan", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2021-11-03T16:56:28.963 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2021-11-03T16:56:28.965 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"after": "d8709f98c8dde30910ea11812e94aaab26c7240d", "attempts": 1, "before": "d8709f98c8dde30910ea11812e94aaab26c7240d", "changed": false, "remote_url_changed": false} 2021-11-03T16:56:28.984 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:56:28 +0000 (0:00:00.677) 0:00:46.571 **** 2021-11-03T16:56:29.126 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-11-03T16:56:29.128 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2021-11-03T16:56:29.138 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:56:29.315 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDdkebJGXKQWBG/7wZG334P+EAdC+5VNMrjLjXhaZZbt3V9vKDQCodcU3zSOjMa/GDGg2WHQdO/vioCc6O/m61KTbgLDaaPSzHH16j/e1SaU8BIgKKU42AUBJzpPeaZ2wCC1i12Hq8ULwjEW29IDsnMortzlCSQmaTKc3DN2nCXZWQXA1rSiQ2AhIq5WNJY9S36lsg5tkex4xUajyNW4vPubLtkSv+WrzNK7ZJ8zfzP6eJEJ+Uj7CLdanBCgwoLGSPfGA5c/gyUT7mHibOUwvFB5sFVsozMPp2PXzUUIv70adJpOskLYfKY7TUSuaB4AUOPu6NVauzVIl3Tsn+uvHqvtn8KC3BPpyH7+JiFQWEt+aO+/ynprfoqCjktuf07SXo0ewoVYT2S++JEBa/N2hKRkzIoSPJ+Rvoazc8GZIjXpQRyhMdTV/LCSeyUTJ7lDQVrQVhUIHKnMm+hZmf4tLQKZvNAThO984CpY93ST5Ww07fjRLMXNZGuaz+zHKNexhWSPKr+ggChnZCOLl/F6zbDnBZoCaHR6hH4+QLz0K/FAgxN2Nn00RiOFkmIlzEioxMcwC/peTl3WzRfqPINdTftiTzcFi/c2QmlHhnxsueIwNhHOn4frkGkFmexdDrXRStERzSBJ13X4kWduPmQsn+bx5/7xwqC6M4bXpkYFOefOw== aschoen@redhat.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2021-11-03T16:56:29.348 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMor 2021-11-03T16:56:29.350 INFO:teuthology.task.ansible.out:bnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2021-11-03T16:56:29.547 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2021-11-03T16:56:29.549 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvG6+jM8aulc5CID/yE6AXPXczxq8tMgY+2sdo/ihWpgDZuFwGL4hwj8nyh5cTqZ2LEa4kRkUUIoH9avLCFN5pDBVvcj1/O06ZGnkq/hV6791UiPF5S5RGmz3MGxv0RDfjA0OLFav7u1s+ZJ6z2W1Tpy/W+g/fMYqbPc6rV+pm3Uo16+BWz/diZH8smqcty4chnnCy6gqOrcAxCZLpeyjX6Ydn3fnTLTAmhoQ9eUz7LjP4ZnFD5zL6jrSBlJ5WNTLevUzvS8eL7RuP8D3Qfn8ukIzPXG2vs594FhSc/HnvubKricvKMm0P0ki4Z0vM9VruQn7Tbeqdwpi1IfgyiAMX aemerson@work-yubi\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv8/FJn7hz2WieoQFUGXS0bhwSwL1LCI9kkjWYNkWb8IJPrBetlkr5TsvHEKDPZGF5Sv8LsYA6njPnbOGyAHpYlnv/Gz4Hwq3hLgmXCs6JGWBQv8NS+pjwnG6Nh6JfFI0ANeg652Rk9Qt9uLYeKVKmHXXrVZAD4NMvJGfEvneQzuwwpylUmV80WAdaQw/H0EP4NScZhNhknwAs/+KPFZH+RFoUj/Cb3poIFjwyviIgyal5hjAEsoczCW3hAla/9IyC2VDS027WPiPFmBFaYK5lYYLStBQdRx+6amH76XScLFy0FEPLL5qiN0ClAptvh1oIURgipdXZwaxpf06AXG8qw== aemerson@electric", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemer 2021-11-03T16:56:29.550 INFO:teuthology.task.ansible.out:son", "validate_certs": true} 2021-11-03T16:56:29.559 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:56:29.560 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:56:29.767 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2021-11-03T16:56:29.769 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2021-11-03T16:56:29.967 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtS 2021-11-03T16:56:29.969 INFO:teuthology.task.ansible.out:oAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2021-11-03T16:56:29.989 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZ 2021-11-03T16:56:29.990 INFO:teuthology.task.ansible.out:tIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOoIm6sOytKwqNf9FSQn0Olu72ZRDjAYeNFQVWTn2PIB ed25519-hplaptop", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "validate_certs": true} 2021-11-03T16:56:30.176 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2021-11-03T16:56:30.187 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2021-11-03T16:56:30.385 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2021-11-03T16:56:30.406 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+ 2021-11-03T16:56:30.408 INFO:teuthology.task.ansible.out:o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2021-11-03T16:56:30.586 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2021-11-03T16:56:30.618 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2021-11-03T16:56:30.794 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9 2021-11-03T16:56:30.796 INFO:teuthology.task.ansible.out:CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2021-11-03T16:56:30.837 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@desktop QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@desktop QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDAV78mqEy/VkB2BthIE/m7iE5sbTjTRS0KuH7AB0KPvZBsDIw5cL8jKyxiaPC85+aOkBa4LrP0TUFBzN87Snt+PLMz2QHBOvlSTfUuRkHVo3vWJFzVuUSzzddkYds3hAxUriAHKv6CRSy1yFBxPQQHTSATUHKWqihMElbUCZrrrfH8A7J9vwixzCSLg4KbINjfa77rxefcZwugUqiYiVUWUSOb023f+QEaDm7OP+/uzz3SjF1Y0LMwa75yctttIfkRBKjY7uFCuZdKc3l2A/fdPxbFwlZmIox3LLngdTzHMVXGbVHqq9Ixo8FSAXiI1Buy2LTMoXMJwQYSvzsv7xNwujKVlUjoE9cOOIdzCC5AziwSiJB4+tNh73c5tRUzMMC98HLOhPJriSPvOt1pMEdWDX3/evcyxvRgJUKgYnmQG7n8v8O/CLyG0sAJDmGC3U9S/3fUDUjZIKWSsc85Q3xRaJwZJIS2VZwH0zG1Gmynih8PPq7VeDx7tKuNm4uTYT8= bigjust@desktop-jsho3l9.lan", "key_options": null, "keyfile": "/home/jcaratzas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcaratzas", "validate_certs": true} 2021-11-03T16:56:31.004 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBY 2021-11-03T16:56:31.006 INFO:teuthology.task.ansible.out:B0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2021-11-03T16:56:31.048 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2021-11-03T16:56:31.059 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:56:31.216 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2021-11-03T16:56:31.269 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uTfMor 2021-11-03T16:56:31.270 INFO:teuthology.task.ansible.out:bnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2021-11-03T16:56:31.425 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2021-11-03T16:56:31.479 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvG6+jM8aulc5CID/yE6AXPXczxq8tMgY+2sdo/ihWpgDZuFwGL4hwj8nyh5cTqZ2LEa4kRkUUIoH9avLCFN5pDBVvcj1/O06ZGnkq/hV6791UiPF5S5RGmz3MGxv0RDfjA0OLFav7u1s+ZJ6z2W1Tpy/W+g/fMYqbPc6rV+pm3Uo16+BWz/diZH8smqcty4chnnCy6gqOrcAxCZLpeyjX6Ydn3fnTLTAmhoQ9eUz7LjP4ZnFD5zL6jrSBlJ5WNTLevUzvS8eL7RuP8D3Qfn8ukIzPXG2vs594FhSc/HnvubKricvKMm0P0ki4Z0vM9VruQn7Tbeqdwpi1IfgyiAMX aemerson@work-yubi\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAv8/FJn7hz2WieoQFUGXS0bhwSwL1LCI9kkjWYNkWb8IJPrBetlkr5TsvHEKDPZGF5Sv8LsYA6njPnbOGyAHpYlnv/Gz4Hwq3hLgmXCs6JGWBQv8NS+pjwnG6Nh6JfFI0ANeg652Rk9Qt9uLYeKVKmHXXrVZAD4NMvJGfEvneQzuwwpylUmV80WAdaQw/H0EP4NScZhNhknwAs/+KPFZH+RFoUj/Cb3poIFjwyviIgyal5hjAEsoczCW3hAla/9IyC2VDS027WPiPFmBFaYK5lYYLStBQdRx+6amH76XScLFy0FEPLL5qiN0ClAptvh1oIURgipdXZwaxpf06AXG8qw== aemerson@electric", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemer 2021-11-03T16:56:31.481 INFO:teuthology.task.ansible.out:son", "validate_certs": true} 2021-11-03T16:56:31.490 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:56:31.502 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:56:31.628 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlI 2021-11-03T16:56:31.630 INFO:teuthology.task.ansible.out:Qv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrSX8nDL2WaHe0b/rjE3TZhj5VwbrG6nn/hyWmm2dWcUqotSp973yD2G4tQL9UygeSkDGmjb6whrz2mcAFJ5P4oLtIcCLyZC550dcpwV3BtP/cMAIGE6xeEARKiP8KlVGFNDRg9/l1aAd08Bepe+rJyAYD7IxOOBfgKJMKbeOWnT40RTfwiD5CXNwjzQuhzZDEbm+8zjTChFM4P1EkfR3c+0X3MYpA1AsRQY0ElvFtcn6G0yPKR+obGdIs+ydGupKh1+peGWLemHMA5e62Bs4/Lwtyebyp7AOguC3ZBJ3TZWL4Vn1q6z21cQe6AuowjPrneZWC4ifFB6OUQJqM+R00pHBcajcvsfpYOblZvTJHM726Ek1vahOYfcbEMpqWlVYo+WPRUilzl5cI3ReyAcwphebqKPbhmzCuyJw3lWBKIVDhWjEFzGkxV/iFU5rIXFm/Z0JGNFFxmY1U4GlA4lFXtL7OvEdMsiZo1x0cNmFWLu0k9fQBtiQI19ZBiA7FPKc= yehuda@crestview", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2021-11-03T16:56:31.714 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2021-11-03T16:56:31.839 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2021-11-03T16:56:31.933 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+2vtS 2021-11-03T16:56:31.935 INFO:teuthology.task.ansible.out:oAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2021-11-03T16:56:32.037 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2021-11-03T16:56:32.152 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2021-11-03T16:56:32.246 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2021-11-03T16:56:32.383 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "key": "ecdsa-sha2-nistp521 AAAAE2VjZHNhLXNoYTItbmlzdHA1MjEAAAAIbmlzdHA1MjEAAACFBAHT7CHaceRLB5mJv9SYKog1wBXsavzH3G3WCBEvVyK76JIslDudZm2ycJv66r+VlGaD8sFI5Mb/KKFMKcKesLRX+wHqZg9MlkoJgm7WGoRQ5I0M1Vt7TL3uJkVCaGZHtO7BXqzTx53C1AaJ1HN+DG2MGy5mm6tHwL901UVqG4EzjBnjxA== christinameno@boxtop", "key_options": null, "keyfile": "/home/cmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cmeno", "validate_certs": true} 2021-11-03T16:56:32.445 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2021-11-03T16:56:32.601 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2021-11-03T16:56:32.654 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2021-11-03T16:56:32.819 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1Z9 2021-11-03T16:56:32.821 INFO:teuthology.task.ansible.out:CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2021-11-03T16:56:32.865 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQv 2021-11-03T16:56:32.867 INFO:teuthology.task.ansible.out:UxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2021-11-03T16:56:33.066 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2021-11-03T16:56:33.171 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkRhoBY 2021-11-03T16:56:33.173 INFO:teuthology.task.ansible.out:B0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2021-11-03T16:56:33.277 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2021-11-03T16:56:33.393 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2021-11-03T16:56:33.477 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2021-11-03T16:56:33.613 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2021-11-03T16:56:33.687 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2021-11-03T16:56:33.822 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlI 2021-11-03T16:56:33.824 INFO:teuthology.task.ansible.out:Qv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwmih46TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrSX8nDL2WaHe0b/rjE3TZhj5VwbrG6nn/hyWmm2dWcUqotSp973yD2G4tQL9UygeSkDGmjb6whrz2mcAFJ5P4oLtIcCLyZC550dcpwV3BtP/cMAIGE6xeEARKiP8KlVGFNDRg9/l1aAd08Bepe+rJyAYD7IxOOBfgKJMKbeOWnT40RTfwiD5CXNwjzQuhzZDEbm+8zjTChFM4P1EkfR3c+0X3MYpA1AsRQY0ElvFtcn6G0yPKR+obGdIs+ydGupKh1+peGWLemHMA5e62Bs4/Lwtyebyp7AOguC3ZBJ3TZWL4Vn1q6z21cQe6AuowjPrneZWC4ifFB6OUQJqM+R00pHBcajcvsfpYOblZvTJHM726Ek1vahOYfcbEMpqWlVYo+WPRUilzl5cI3ReyAcwphebqKPbhmzCuyJw3lWBKIVDhWjEFzGkxV/iFU5rIXFm/Z0JGNFFxmY1U4GlA4lFXtL7OvEdMsiZo1x0cNmFWLu0k9fQBtiQI19ZBiA7FPKc= yehuda@crestview", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2021-11-03T16:56:33.887 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-11-03T16:56:34.044 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2021-11-03T16:56:34.098 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2021-11-03T16:56:34.276 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2021-11-03T16:56:34.319 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIKepJjIovBTCkdVyvAdptljhw9bYcwtIqV/ojdbpeuJtxDo9UyrEkNoLeVpZgOcrbgrkOE7AEixo3qB9PbD8z5NRUleV7wipSh3QwrDmSylSVReJKOwYdxUKkzesZYBGIGYzAbTKqcVzPUWCeW314+xmv90XkO/+1vgbC5uoXfbfnaiDHQimBom8QV5Egr+EhhcuCqMEqaVSvjMr1T0e6IL9gUens6Mys6czI37go6QvmgEzkBbDJJ64PsiOOFHVNk4DJVJ1c7TRtsnpIHM1YVFjsH1AnAL6UufbgNSFf5dziUB/Bz815P1/jsOGpHkUKE37DbHU5xtLGl3MmimMt4ndcG6EggBGEYBwfEWPRAsZs3qngssrNH5W0CzbysLaoovzq5VqS8SdLMu5XrRLBEB0uWWzLq2bobYymk3t/oby5zrHJV78xAuLmldqjvlPZSdkgwWNv9YBLkteBVr2HhjtuashJ6amNPbA4kCGhXMpeRhfvfqs31CBEKbTV+A8ODWbWAoQxLCcqS24x0yMi73N0UzuqDTaJMcE8ODJvOxcLNaV688QDbrC/UehpX/Bp/DM7fLHj4YhkRpZxkkN5w2kb/bBbWya4DrffhHKqn9CH1z5ynj0237PhTwCXjtRn0JzpeRgN8OraFgR2H3uPc9XNqlkFxxOBK0lRngbxzw== owasserm@redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2021-11-03T16:56:34.486 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2021-11-03T16:56:34.519 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1 2021-11-03T16:56:34.521 INFO:teuthology.task.ansible.out:OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2021-11-03T16:56:34.708 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2021-11-03T16:56:34.730 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyK 2021-11-03T16:56:34.732 INFO:teuthology.task.ansible.out:z2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2021-11-03T16:56:34.920 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2021-11-03T16:56:34.931 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2021-11-03T16:56:35.142 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2CfQv 2021-11-03T16:56:35.144 INFO:teuthology.task.ansible.out:UxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": nul 2021-11-03T16:56:35.146 INFO:teuthology.task.ansible.out:l, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2021-11-03T16:56:35.342 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2021-11-03T16:56:35.353 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2021-11-03T16:56:35.552 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNH 2021-11-03T16:56:35.554 INFO:teuthology.task.ansible.out:ORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDISEXvGI2oJLTRR15ePZLkmdYjaQJxt9V/yBwKdGqYvnPzmLOpdmVpdQJo7ZOWwdp5tJsCy4q5uX5vIUv6Gyi3uVSGmZnrKUbsqoDNwAX5vx/gFBINIs/ZXXyD4fpUT8aoZeq1XmXdIaWG0nImsrtRnRxKP9NGZEcW8z0WPwgF0Ck+RH0qZUix6Bfbgy7vMXnivuDv+NdjEwyszcrPWvf6Krr0am9Q0+9nj/mjvk6CZLl+CH3HNdkhZuNs+qMG4XW8KhLvQB2p790XVlTHuVU4jWrFmjymggZxkhoCY/ukHH6wi2Pbx6yjjKOj8iIw1xKabJzJ7S8UKT23A41d918DUCUuEG66pN6Z4dnv+fZJxj5iEo1GdycSWIc4SpLTK3z4zDZypVKIzjI4o8VwQaN48HkVjUdTICTvGQfytFi3jX1I5nxIZiiPcxMQLR5hQggP8pn2XtvXpomn0aBc8tVuuZJLVq4zCVnhlVB6IACmb71evjFGOsoSuEsaTufUr48= prsrivas@t480.lab.blr.redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-11-03T16:56:35.574 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2021-11-03T16:56:35.763 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjE 2021-11-03T16:56:35.765 INFO:teuthology.task.ansible.out:fdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+d 2021-11-03T16:56:35.767 INFO:teuthology.task.ansible.out:TzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2021-11-03T16:56:35.775 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2021-11-03T16:56:35.966 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2021-11-03T16:56:35.999 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2021-11-03T16:56:36.168 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2021-11-03T16:56:36.222 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-11-03T16:56:36.379 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2021-11-03T16:56:36.444 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2021-11-03T16:56:36.591 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2021-11-03T16:56:36.665 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIKepJjIovBTCkdVyvAdptljhw9bYcwtIqV/ojdbpeuJtxDo9UyrEkNoLeVpZgOcrbgrkOE7AEixo3qB9PbD8z5NRUleV7wipSh3QwrDmSylSVReJKOwYdxUKkzesZYBGIGYzAbTKqcVzPUWCeW314+xmv90XkO/+1vgbC5uoXfbfnaiDHQimBom8QV5Egr+EhhcuCqMEqaVSvjMr1T0e6IL9gUens6Mys6czI37go6QvmgEzkBbDJJ64PsiOOFHVNk4DJVJ1c7TRtsnpIHM1YVFjsH1AnAL6UufbgNSFf5dziUB/Bz815P1/jsOGpHkUKE37DbHU5xtLGl3MmimMt4ndcG6EggBGEYBwfEWPRAsZs3qngssrNH5W0CzbysLaoovzq5VqS8SdLMu5XrRLBEB0uWWzLq2bobYymk3t/oby5zrHJV78xAuLmldqjvlPZSdkgwWNv9YBLkteBVr2HhjtuashJ6amNPbA4kCGhXMpeRhfvfqs31CBEKbTV+A8ODWbWAoQxLCcqS24x0yMi73N0UzuqDTaJMcE8ODJvOxcLNaV688QDbrC/UehpX/Bp/DM7fLHj4YhkRpZxkkN5w2kb/bBbWya4DrffhHKqn9CH1z5ynj0237PhTwCXjtRn0JzpeRgN8OraFgR2H3uPc9XNqlkFxxOBK0lRngbxzw== owasserm@redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2021-11-03T16:56:36.801 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2021-11-03T16:56:36.875 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6AKY1 2021-11-03T16:56:36.877 INFO:teuthology.task.ansible.out:OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2021-11-03T16:56:36.990 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/home 2021-11-03T16:56:36.992 INFO:teuthology.task.ansible.out:/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2021-11-03T16:56:37.096 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2RuqiZyK 2021-11-03T16:56:37.097 INFO:teuthology.task.ansible.out:z2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2021-11-03T16:56:37.203 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@snowmon CjHOevVvBm+PwhjnhZYM7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@snowmon CjHOevVvBm+PwhjnhZYM7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2021-11-03T16:56:37.319 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2021-11-03T16:56:37.403 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnOq6OHTRJuv7n0GZygBue/AE2q47pFM3KyrvmZxPRNmwtXn2r3ngHQQyc6DLZdFIv9Yn0VWu1KB/ryCdN3Xy7HAxBfHdKjtQ4vvfsWUbQF2JjhEjrsl+usgpqK1Tz6yuL4uAtFgLTN9o6gcH2bsQ15w+rEHCiYVzaGp7Cj+y+/VPBRIEHETSUZxwqiDH57uFTHxJrBIn6yhCorbn5QOjfmHUGmkxl3Geph++LbuQeha5F/DHaylMulsYZZBDEhIG8G2coDmVz8Cp1gw0kBWv63JHKnXaJboGXty2Zj36O/wdZKTE+7cV6JhxfJczhZBTb2cImIhspGwdHfnFgBoXz myoungwon@fg2server", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2021-11-03T16:56:37.539 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2021-11-03T16:56:37.623 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": " 2021-11-03T16:56:37.625 INFO:teuthology.task.ansible.out:gabrioux", "validate_certs": true} 2021-11-03T16:56:37.749 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2021-11-03T16:56:37.825 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2021-11-03T16:56:37.971 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNH 2021-11-03T16:56:37.973 INFO:teuthology.task.ansible.out:ORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDISEXvGI2oJLTRR15ePZLkmdYjaQJxt9V/yBwKdGqYvnPzmLOpdmVpdQJo7ZOWwdp5tJsCy4q5uX5vIUv6Gyi3uVSGmZnrKUbsqoDNwAX5vx/gFBINIs/ZXXyD4fpUT8aoZeq1XmXdIaWG0nImsrtRnRxKP9NGZEcW8z0WPwgF0Ck+RH0qZUix6Bfbgy7vMXnivuDv+NdjEwyszcrPWvf6Krr0am9Q0+9nj/mjvk6CZLl+CH3HNdkhZuNs+qMG4XW8KhLvQB2p790XVlTHuVU4jWrFmjymggZxkhoCY/ukHH6wi2Pbx6yjjKOj8iIw1xKabJzJ7S8UKT23A41d918DUCUuEG66pN6Z4dnv+fZJxj5iEo1GdycSWIc4SpLTK3z4zDZypVKIzjI4o8VwQaN48HkVjUdTICTvGQfytFi3jX1I5nxIZiiPcxMQLR5hQggP8pn2XtvXpomn0aBc8tVuuZJLVq4zCVnhlVB6IACmb71evjFGOsoSuEsaTufUr48= prsrivas@t480.lab.blr.redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-11-03T16:56:38.034 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2021-11-03T16:56:38.182 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+zmfjE 2021-11-03T16:56:38.183 INFO:teuthology.task.ansible.out:fdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbBmS+d 2021-11-03T16:56:38.185 INFO:teuthology.task.ansible.out:TzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2021-11-03T16:56:38.246 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kk 2021-11-03T16:56:38.248 INFO:teuthology.task.ansible.out:ke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2021-11-03T16:56:38.403 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2021-11-03T16:56:38.445 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2021-11-03T16:56:38.612 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2021-11-03T16:56:38.654 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2021-11-03T16:56:38.832 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2021-11-03T16:56:38.864 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2021-11-03T16:56:39.043 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2021-11-03T16:56:39.065 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-11-03T16:56:39.255 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2021-11-03T16:56:39.369 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2021-11-03T16:56:39.480 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2021-11-03T16:56:39.482 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6NWn9EzQJoeXkUbaNclYcR9Gga7HFhhXMGxdo/9ypYwlEzQTA6FnEreG3s1EmlQ4hofuc0kMhrjh0YeJZMrETWMoA0SmoX+6iR26UWVtOTk7i1v2TqkTY7llmQneHqQgmZEBXK0jF1HVP5kHY5dhagtcpfsGY5dc0eGUjfqAyh/xrCxciOE/JQ03tDcKIc5eOQIqv46JqnUS90Zr/Cov2qGKKHTa/VO1a2viOmDC/uHi5xXvII1Kh9rHp1oZ2Wm3fiGgT8xZaTJFFhWpdq3AdKfkEaV93yQcCdaUsGBHwDHcPOhxpBZpqZvcCR3I/tigDav4QViB1PRN7JnTIWCIznfrQK3r2Frl6BVFDvydwgRVS43ctEKPOJj8ELMUWX+fpjsKrA9Kn7nUU35ti7qIWKg1Cnz3lzVA29TsEg10E+nT4SVKZtRSWw/2AX0bsfxfSY30yjw5PP+bjnVh/bOOkqR2UkPBj4eeOKMD8nV1an79xP0eH+t36INaJhq6qwxE= nojha@nojha-mac", "key_options": null, "keyfile": "/home 2021-11-03T16:56:39.484 INFO:teuthology.task.ansible.out:/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2021-11-03T16:56:39.689 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2021-11-03T16:56:39.711 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@snowmon CjHOevVvBm+PwhjnhZYM7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@snowmon CjHOevVvBm+PwhjnhZYM7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2021-11-03T16:56:39.888 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2021-11-03T16:56:39.926 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCnOq6OHTRJuv7n0GZygBue/AE2q47pFM3KyrvmZxPRNmwtXn2r3ngHQQyc6DLZdFIv9Yn0VWu1KB/ryCdN3Xy7HAxBfHdKjtQ4vvfsWUbQF2JjhEjrsl+usgpqK1Tz6yuL4uAtFgLTN9o6gcH2bsQ15w+rEHCiYVzaGp7Cj+y+/VPBRIEHETSUZxwqiDH57uFTHxJrBIn6yhCorbn5QOjfmHUGmkxl3Geph++LbuQeha5F/DHaylMulsYZZBDEhIG8G2coDmVz8Cp1gw0kBWv63JHKnXaJboGXty2Zj36O/wdZKTE+7cV6JhxfJczhZBTb2cImIhspGwdHfnFgBoXz myoungwon@fg2server", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2021-11-03T16:56:40.093 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2021-11-03T16:56:40.146 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": " 2021-11-03T16:56:40.147 INFO:teuthology.task.ansible.out:gabrioux", "validate_certs": true} 2021-11-03T16:56:40.304 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2021-11-03T16:56:40.357 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2021-11-03T16:56:40.515 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2021-11-03T16:56:40.569 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2021-11-03T16:56:40.715 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2021-11-03T16:56:40.789 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS33Kk 2021-11-03T16:56:40.790 INFO:teuthology.task.ansible.out:ke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2021-11-03T16:56:40.925 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2021-11-03T16:56:41.009 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2021-11-03T16:56:41.128 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2021-11-03T16:56:41.222 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2021-11-03T16:56:41.337 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDklgWHrL9TMaMxIyN9MxVR6nR3xh9zfj0iR63/yOzaslX0WoVBk7CYTsRlH2GFVctvixVwrWbQgMaKe49utal/3FA6sMl8S2DTrtgC1a/exUTvU0U/+ZhyUlkyjQfRgtYOUwodNupL5NsC8VZqZshgmgXC34uye3bplx3VwvpUt9r6QxJN3/9Dbi6HxudhDCLfmvROhwQYHaFF+GglcbyjAt3uvsuwXVKBd1/vMQwbcUo9/zQlrnmWBRuqTNkXGrZauWigoi/hyuWYQ3MUbRc3LVpJncMv7BEtcj8fCVvPB09SFee3nSeTtFkm01GvRcYd3HBz/YL4Rz92wXl9VIxqs8aaJBzy+MD0NO496Rfqtz6YuZ1m7AxDIccDW+OpcPgYwaUKFXNTuVLWZXhkS0p4swefcKWRlQFF3BLOebN/Kr9jq1MFa+5pX7Gj4WF5uCIX0/fx40KVZNhfl3qywBy/d5MKwXO7PCHMiGcu2FIXdZ334MxdgiIVoX54PpRP4hs= smanjara@localhost.localdomain", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2021-11-03T16:56:41.431 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2021-11-03T16:56:41.546 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2021-11-03T16:56:41.641 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2021-11-03T16:56:41.746 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2021-11-03T16:56:41.860 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2021-11-03T16:56:41.948 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDQu/CuhCIFVGCLbA0cjn+qqcwX12cRcFA1sHyJtkj6Ltfmge8FnfgAsex/LaPJwbtbBiVl9xRcazHNfBaDw5DxMRSJF+e7WTydZC2Jm4qIj4D3Lr+imGPrJTK+kjN0QXRXknGsNn36oRF/I41cGqYRGwawFGSPZBHuav+ehDk1doTrbDlll4zMUWm2iXTcBIcJo4NR1LoE2c9/MPfbH7jKm5ke7aS0ets+EANEplkeClfCVAashtIF+Bz6a9qk4F2n0baLT/s3IQkXCwwO2fxxY03x8gWiB1n4E7sousEB+J1Skl/z0bwmaiY0C+raPoV/FyZxQuh0ocvdU7fnQbKuNoGE/Usfm9Y3g8eH81b/k4fMJL2lqXHkPft6zUV9bJsp3v1iGEkIf2+ER/QLczteNz6kw/uTPQctMEDXM17GEnBXISVrQcPSW3i93eFwWacs8iT3G4bjEQvpxSBQrhDdCCoxy8bVWqkVeDWw5u2q/3WL9Ov1NrY+G4/DYb2qONNQds0vTHCGvemU5GNjtaeCbojkZ7Z3KOeAY8080kKrsO7fve+ZMACCIFD+se8kF7NN4ZJASSqq9lJpBWvtbc0O8jCcVGNuN7exXjrcbiN1L/9HkARV+KEJU3jbXObAhAnfAaeRWBwmTfZodCu+fDd/Jor6l36tjD4kC4/CPrDtyQ== alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2021-11-03T16:56:42.073 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2021-11-03T16:56:42.157 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2021-11-03T16:56:42.282 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2021-11-03T16:56:42.366 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2021-11-03T16:56:42.491 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2021-11-03T16:56:42.564 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDp0xz0ml2XuoQe70TxVeTo3Ib5xMinDrYNJ5eq5XYNa8ZLGcEByeDK4myzixTloAo31Rb/yDyGqv6kQ6nFChihO2aab3Ni/bsMOaJat1scrznaIYXGvLvw5KYx/qBDUxb3qW0vO0N/bB3dUPS1 2021-11-03T16:56:42.566 INFO:teuthology.task.ansible.out:lXTJaVf+KuljUk7v38ov2gsPWlofRYZNkPGQfUKXFEXc39OVj+Ex5LkehH/Z1LwjRYJa7jAdSm8hE5m8m6WborrzAA4/iFuLbhtM/Bqm4boSqa7koBVux422QX/HblSQ2KAeT1GFOS7xdcO6ndyp6mIpCZWunTjZwSLyK20NvIV/AzdAro4GZWi0D34PPtbSG344ZT1dSLeDRhYVkDjS6uutrmw9C+Ns1bo8s1ydhxL2XEsU14SWRWlbyrMF9PXyJOsaNiBNWGwc9lTly0ZlmPnN3l0AC3RIKOstkEYmI+TMj21r1tltz48e+6dH1an51wwnhNRYiODd3hbS67tmIg05BuSU5+whJ93wzXw2rC4FLE= ylifshit@localhost.localdomain", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2021-11-03T16:56:42.701 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2021-11-03T16:56:42.764 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2021-11-03T16:56:42.921 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2021-11-03T16:56:42.975 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2021-11-03T16:56:43.132 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2021-11-03T16:56:43.174 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2021-11-03T16:56:43.351 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2021-11-03T16:56:43.394 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq 2021-11-03T16:56:43.396 INFO:teuthology.task.ansible.out:2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2021-11-03T16:56:43.560 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2021-11-03T16:56:43.602 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdqbYcbJYtgCrAbLo7PcS82caXxcT8cpX0RkULaFXfaiUDs8Foy6Yf/XoEHIdpgw8kX6qBdydYXxti6HFhYDNsL5KleKu/yA+bVKlvri39CRNkNxFHAUL2CRBT5Hefj1AJQBm/BKVEN4/eb05OlJJ808Qb/aNycot/YwgpNvUIaO2yTMA0meq82wNdZmUcmyLB9QXz4QInTcoAsxq70KxX95YC938rTWpbg1hM8gzLkJJt/OD/Z7TMYv/Sc5KGOFqo930AmObQ5KPmffkWGYbLT65TxlCULnvIr63lWHdGTj5mYjib/v9ncu24az9GYRWYcicV0B/def+KVrrIp3Mz ideepika@teuthology", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2021-11-03T16:56:43.779 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2021-11-03T16:56:43.811 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2021-11-03T16:56:43.989 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDklgWHrL9TMaMxIyN9MxVR6nR3xh9zfj0iR63/yOzaslX0WoVBk7CYTsRlH2GFVctvixVwrWbQgMaKe49utal/3FA6sMl8S2DTrtgC1a/exUTvU0U/+ZhyUlkyjQfRgtYOUwodNupL5NsC8VZqZshgmgXC34uye3bplx3VwvpUt9r6QxJN3/9Dbi6HxudhDCLfmvROhwQYHaFF+GglcbyjAt3uvsuwXVKBd1/vMQwbcUo9/zQlrnmWBRuqTNkXGrZauWigoi/hyuWYQ3MUbRc3LVpJncMv7BEtcj8fCVvPB09SFee3nSeTtFkm01GvRcYd3HBz/YL4Rz92wXl9VIxqs8aaJBzy+MD0NO496Rfqtz6YuZ1m7AxDIccDW+OpcPgYwaUKFXNTuVLWZXhkS0p4swefcKWRlQFF3BLOebN/Kr9jq1MFa+5pX7Gj4WF5uCIX0/fx40KVZNhfl3qywBy/d5MKwXO7PCHMiGcu2FIXdZ334MxdgiIVoX54PpRP4hs= smanjara@localhost.localdomain", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2021-11-03T16:56:44.021 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrGYBGPDQqn+2jnxKhDurSrAJvTT8dZMYGVCt6NbjpFhDFYtyvrF8Jv5NKBQynG18LwscN7a57L0Oq0e4jOEPRz//CDcAfP+L6p45L4BdUzTysN4+rQ7waMnIiYp7VjSswhGH/BnYIVKwmDqzM2fo1w/oLIT6mgwUYqAdceeswAkXBt7jgagL8ox3E4oHBqER7f60AHdTviuW953t5luVQiMjyF2fOwZuo2ycufIzBee0CwS+xyRuJQwdgD3HL4Qs2vtBCX+H8n3wC6A9JrMA5tzCDr5VZKBJZURIniwvaXfJodjO4SvBL5jAfXoMqriv8ZWHbap2fk82YE8/uPqNvSISjixJEBgQinklZ+iAOYm5tfPbhPM/8ErpN/yJ0EoGRn+nj6X6nAjXAIk3AENopeqtM7mLZ26RFIuEV5ld7eqUvAfiy6Hk2LH9Q1Z4CXaxD0gDGzUxgdAaaAgLAR4btWi2n7Tb7ZNDdOfctg2ya9ftzZBOHvCaVXSpJCry/GMU= skoduri@localhost.localdomain", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2021-11-03T16:56:44.208 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2021-11-03T16:56:44.220 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2021-11-03T16:56:44.418 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2021-11-03T16:56:44.430 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2021-11-03T16:56:44.649 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDQu/CuhCIFVGCLbA0cjn+qqcwX12cRcFA1sHyJtkj6Ltfmge8FnfgAsex/LaPJwbtbBiVl9xRcazHNfBaDw5DxMRSJF+e7WTydZC2Jm4qIj4D3Lr+imGPrJTK+kjN0QXRXknGsNn36oRF/I41cGqYRGwawFGSPZBHuav+ehDk1doTrbDlll4zMUWm2iXTcBIcJo4NR1LoE2c9/MPfbH7jKm5ke7aS0ets+EANEplkeClfCVAashtIF+Bz6a9qk4F2n0baLT/s3IQkXCwwO2fxxY03x8gWiB1n4E7sousEB+J1Skl/z0bwmaiY0C+raPoV/FyZxQuh0ocvdU7fnQbKuNoGE/Usfm9Y3g8eH81b/k4fMJL2lqXHkPft6zUV9bJsp3v1iGEkIf2+ER/QLczteNz6kw/uTPQctMEDXM17GEnBXISVrQcPSW3i93eFwWacs8iT3G4bjEQvpxSBQrhDdCCoxy8bVWqkVeDWw5u2q/3WL9Ov1NrY+G4/DYb2qONNQds0vTHCGvemU5GNjtaeCbojkZ7Z3KOeAY8080kKrsO7fve+ZMACCIFD+se8kF7NN4ZJASSqq9lJpBWvtbc0O8jCcVGNuN7exXjrcbiN1L/9HkARV+KEJU3jbXObAhAnfAaeRWBwmTfZodCu+fDd/Jor6l36tjD4kC4/CPrDtyQ== alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2021-11-03T16:56:44.660 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2021-11-03T16:56:44.869 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2021-11-03T16:56:44.871 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2021-11-03T16:56:45.067 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDS3HkgHb6z+gi4tN5rBL/N+WLJgN8RtUokt0YnRjpor2epRbmqgmEFRbRFqjLK9LiTuLxmkwol1/G56AJms9ixue33M6jVs73nc259LDxn2qj/NCtxeU7xdjSDuhKyItgM0yuiFjWOKSYTghNtwoR26PX7SQVbycD4NN1ZuwUbdMQA31VrLvQR7d0lYKlis2auH2Xb6WifotoEyszZz1KYRwzJvOHtHku/jAyNmTD/8kAsuCjEgUPSyyehAfdWn59tY2gM7aFyj/dtRoAghekv5gkov9zx+zjN6D2LSN7e+DRti05iJMifA3nfmdj7f8aHzXDtsDL0XNkmmT3g9N2OQ8WL4ESOFASXPPj9JGUROealgnaQ+Y64evDClhbt2L7FLbgj5sgKb+XYhpGnAiC+v3t1Rq3HZrstOYf2qSYguIo+zW9gHwzJDqH5YNCpSF4ZDCfkJ94ngl5ptQmZ/FcrH2hBs2cMoyK9EyDpcI9hBeACTzu19kaB6DrOC+UTvBgYL2sNbCvi+caIqhENJICZ21rI4u4EZqu7PVb+Y3CLgSjTEgclGYBZoxlrT/X2QHrIfZF7lLGeeh0bureedCrBg4b61SL7ZU3Me9x1iWOZMSJgVvg+qPSrzZiHbA46k8i0af5Q+geL1BuuGy76zbYd/KR0KiNZnXjdlHyRz76rpQ== kamoltatsirivadhna@Ksirivad-MacBook-Pro.loc 2021-11-03T16:56:45.069 INFO:teuthology.task.ansible.out:al", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2021-11-03T16:56:45.079 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2021-11-03T16:56:45.276 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2021-11-03T16:56:45.298 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDDp0xz0ml2XuoQe70TxVeTo3Ib5xMinDrYNJ5eq5XYNa8ZLGcEByeDK4myzixTloAo31Rb/yDyGqv6kQ6nFChihO2aab3Ni/bsMOaJat1scrznaIYXGvLvw5KYx/qBDUxb3qW0vO0N/bB3dUPS1 2021-11-03T16:56:45.300 INFO:teuthology.task.ansible.out:lXTJaVf+KuljUk7v38ov2gsPWlofRYZNkPGQfUKXFEXc39OVj+Ex5LkehH/Z1LwjRYJa7jAdSm8hE5m8m6WborrzAA4/iFuLbhtM/Bqm4boSqa7koBVux422QX/HblSQ2KAeT1GFOS7xdcO6ndyp6mIpCZWunTjZwSLyK20NvIV/AzdAro4GZWi0D34PPtbSG344ZT1dSLeDRhYVkDjS6uutrmw9C+Ns1bo8s1ydhxL2XEsU14SWRWlbyrMF9PXyJOsaNiBNWGwc9lTly0ZlmPnN3l0AC3RIKOstkEYmI+TMj21r1tltz48e+6dH1an51wwnhNRYiODd3hbS67tmIg05BuSU5+whJ93wzXw2rC4FLE= ylifshit@localhost.localdomain", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2021-11-03T16:56:45.484 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2021-11-03T16:56:45.516 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2021-11-03T16:56:45.694 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2021-11-03T16:56:45.739 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2021-11-03T16:56:45.896 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2021-11-03T16:56:45.960 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2021-11-03T16:56:46.108 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2021-11-03T16:56:46.182 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfbsmXq 2021-11-03T16:56:46.184 INFO:teuthology.task.ansible.out:2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2021-11-03T16:56:46.391 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDj6DxTZNodVnwJZrnSZ/jv9Cw4uYLb8MGpccmOvtwl93sVBvqq597AYnBqYiV4VBbKU3GVFbT41zkplia1iCfXoddttCOyWmEKviBXRnAsd5f80dxL0pfzrBJvCaH4NAfYgVw9JFWsvT981b+nB6vum4Vz8Dw0PM5GPnVUjsZNFkpmDPUBL4yHPmMCGP2WtUurJVD3Gm9t72XlG6Dl/vB9Qw/i4IY1SoCCbuhhNgE7IN8JXeGmVJbRwDeUlP+Q8Ti7iedV7YB/W1opt6eT5EPzX8NcxTc0c7rrDgsUWjFQYRSup0GLK9ofdHs0E2RbPPO5OSc5Z2+B1IEV7fZeyJwAyHwUgwx7sSGy7aROdvinaj/hOBglcgISyQ2CY7LTV8NGWJquo7yRCJXhZVIHfgqfBcrgWdNbr+vNBXnWe5PCtLhV8ryHPSJ/7Ru+7voQxgvwux5SdZwxXTbWDKni1f6/8bNrhRx6bmLK9xuYAgt2FhlIivI7L5kQF4etcBiaf68= deepika@x1cabon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdqbYcbJYtgCrAbLo7PcS82caXxcT8cpX0RkULaFXfaiUDs8Foy6Yf/XoEHIdpgw8kX6qBdydYXxti6HFhYDNsL5KleKu/yA+bVKlvri39CRNkNxFHAUL2CRBT5Hefj1AJQBm/BKVEN4/eb05OlJJ808Qb/aNycot/YwgpNvUIaO2yTMA0meq82wNdZmUcmyLB9QXz4QInTcoAsxq70KxX95YC938rTWpbg1hM8gzLkJJt/OD/Z7TMYv/Sc5KGOFqo930AmObQ5KPmffkWGYbLT65TxlCULnvIr63lWHdGTj5mYjib/v9ncu24az9GYRWYcicV0B/def+KVrrIp3Mz ideepika@teuthology", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2021-11-03T16:56:46.454 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2021-11-03T16:56:46.601 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2021-11-03T16:56:46.720 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDISEXvGI2oJLTRR15ePZLkmdYjaQJxt9V/ 2021-11-03T16:56:46.722 INFO:teuthology.task.ansible.out:yBwKdGqYvnPzmLOpdmVpdQJo7ZOWwdp5tJsCy4q5uX5vIUv6Gyi3uVSGmZnrKUbsqoDNwAX5vx/gFBINIs/ZXXyD4fpUT8aoZeq1XmXdIaWG0nImsrtRnRxKP9NGZEcW8z0WPwgF0Ck+RH0qZUix6Bfbgy7vMXnivuDv+NdjEwyszcrPWvf6Krr0am9Q0+9nj/mjvk6CZLl+CH3HNdkhZuNs+qMG4XW8KhLvQB2p790XVlTHuVU4jWrFmjymggZxkhoCY/ukHH6wi2Pbx6yjjKOj8iIw1xKabJzJ7S8UKT23A41d918DUCUuEG66pN6Z4dnv+fZJxj5iEo1GdycSWIc4SpLTK3z4zDZypVKIzjI4o8VwQaN48HkVjUdTICTvGQfytFi3jX1I5nxIZiiPcxMQLR5hQggP8pn2XtvXpomn0aBc8tVuuZJLVq4zCVnhlVB6IACmb71evjFGOsoSuEsaTufUr48= prsrivas@t480.lab.blr.redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-11-03T16:56:46.815 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCrGYBGPDQqn+2jnxKhDurSrAJvTT8dZMYGVCt6NbjpFhDFYtyvrF8Jv5NKBQynG18LwscN7a57L0Oq0e4jOEPRz//CDcAfP+L6p45L4BdUzTysN4+rQ7waMnIiYp7VjSswhGH/BnYIVKwmDqzM2fo1w/oLIT6mgwUYqAdceeswAkXBt7jgagL8ox3E4oHBqER7f60AHdTviuW953t5luVQiMjyF2fOwZuo2ycufIzBee0CwS+xyRuJQwdgD3HL4Qs2vtBCX+H8n3wC6A9JrMA5tzCDr5VZKBJZURIniwvaXfJodjO4SvBL5jAfXoMqriv8ZWHbap2fk82YE8/uPqNvSISjixJEBgQinklZ+iAOYm5tfPbhPM/8ErpN/yJ0EoGRn+nj6X6nAjXAIk3AENopeqtM7mLZ26RFIuEV5ld7eqUvAfiy6Hk2LH9Q1Z4CXaxD0gDGzUxgdAaaAgLAR4btWi2n7Tb7ZNDdOfctg2ya9ftzZBOHvCaVXSpJCry/GMU= skoduri@localhost.localdomain", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2021-11-03T16:56:46.951 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOJg56HsyNa5vHHb52G6O5vtv9I2tdlNMjFlG3SVCCu/bVZwVeGzp1ekuNGU10owmnIvhlSldMjLn2wwTwiDjbvYJVVCPvzK63vCUcMevRsvr9EyU16JdAgrmrGMyuYYEb+wh8XWI9Dc72o84fMaBYFSSer1S9nUBL+KVLVhIe7culgADimrDgDVh2t7u2JHBI0m0nKFGJZNcIyYf+GKOM+n7SaQXIYIytvSteU5BvUIKPsDd4z5lVlWiKO3kG6jF2RnKFV6KWBWI9m8YGItdGu25mXELXN9MOlApZF3AoGTmCE2Web7uDVGvO6rz6b+dc7RXTOlbWIhtMoRqQf6mz zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2021-11-03T16:56:47.035 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2021-11-03T16:56:47.175 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2021-11-03T16:56:47.248 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2021-11-03T16:56:47.395 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2021-11-03T16:56:47.459 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2021-11-03T16:56:47.617 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2021-11-03T16:56:47.670 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2021-11-03T16:56:47.827 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5bCXQ/B16AKl2oevC93R0U+3OrTyd3XEypouJy4LjIzXLX/Go1C1ViERwDAeHmqiUTqJUL+KZ0kMo8U7zlhNBk9PucbMZCqdqx5hkiCAo4y65lEXmuPGimRU2mK9TpcDTLWoyrD/EEhwzT6gmw3ytVTxi7qAb0tQUCQB+pW4+Zr5u6HZzgu9kLGxp7zsGtMA84Zqm5ql72mGuS0bUN1ajm8LWBetFuzOOW6YGfnKt29cLV2h1itVewA9DKM5G70B91eK4LjoNddCkqVA8L+v6LMQNU2pejUzPap98hmFUpQMNxlWQ0+B7UjfbkHWe+mIsSkdr3FfCfqWvsL0RiHcZ root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2021-11-03T16:56:47.890 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDS3HkgHb6z+gi4tN5rBL/N+WLJgN8RtUokt0YnRjpor2epRbmqgmEFRbRFqjLK9LiTuLxmkwol1/G56AJms9ixue33M6jVs73nc259LDxn2qj/NCtxeU7xdjSDuhKyItgM0yuiFjWOKSYTghNtwoR26PX7SQVbycD4NN1ZuwUbdMQA31VrLvQR7d0lYKlis2auH2Xb6WifotoEyszZz1KYRwzJvOHtHku/jAyNmTD/8kAsuCjEgUPSyyehAfdWn59tY2gM7aFyj/dtRoAghekv5gkov9zx+zjN6D2LSN7e+DRti05iJMifA3nfmdj7f8aHzXDtsDL0XNkmmT3g9N2OQ8WL4ESOFASXPPj9JGUROealgnaQ+Y64evDClhbt2L7FLbgj5sgKb+XYhpGnAiC+v3t1Rq3HZrstOYf2qSYguIo+zW9gHwzJDqH5YNCpSF4ZDCfkJ94ngl5ptQmZ/FcrH2hBs2cMoyK9EyDpcI9hBeACTzu19kaB6DrOC+UTvBgYL2sNbCvi+caIqhENJICZ21rI4u4EZqu7PVb+Y3CLgSjTEgclGYBZoxlrT/X2QHrIfZF7lLGeeh0bureedCrBg4b61SL7ZU3Me9x1iWOZMSJgVvg+qPSrzZiHbA46k8i0af5Q+geL1BuuGy76zbYd/KR0KiNZnXjdlHyRz76rpQ== kamoltatsirivadhna@Ksirivad-MacBook-Pro.loc 2021-11-03T16:56:47.891 INFO:teuthology.task.ansible.out:al", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2021-11-03T16:56:48.042 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2021-11-03T16:56:48.095 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2021-11-03T16:56:48.251 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2021-11-03T16:56:48.325 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2021-11-03T16:56:48.460 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} 2021-11-03T16:56:48.544 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2021-11-03T16:56:48.669 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2021-11-03T16:56:48.783 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2021-11-03T16:56:48.877 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqsDnRqAeNPi5xiIfFl2i2zqcwLTvcAso3R9+QAfSChw4+wcQzmMV8uQz1jt8UReefg2QL6bwvcQPj5Zw7ADqUjfxm1K0ezgH6RrtHgRZ1SVseTvJUvzmkDZ2gr2ToZAJYsaV7gBLJLZ7TOiQ6jzrtitiKqJZdpZfzadjShecY1TdMo8bSS7XfbRcwri40u82TRM825B4L//QuhMI7akPqa3iJbGEAQPhkGLI3kz19ryVG1UwvAkkt7TQ68zz+piXSmB2CMRti51shT4NuAOq6Qe9y2Obicf88ItkYxvySMQ5jWi2bo7Zhwuq9jUH8lK6I6PHrYc8LOsR9gE/hqkXxTQrkOXEDtHN1mr/KKLrXO0GeBdgh8Gj4gcwgd1RUCE+T9q09WO9j7c0piWK9Fs1+q1FjH06/AfmP4qob1TlJMHhtPyOp+lk2XvABP+sqW4vyrgmXu5vJU+Hgq3X9KE7bKh4WkCem24Np4uA+irhpvZmIoHyINsYW46lGfwJvM8= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2021-11-03T16:56:48.997 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2021-11-03T16:56:49.104 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} 2021-11-03T16:56:49.208 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2021-11-03T16:56:49.324 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2021-11-03T16:56:49.429 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDISEXvGI2oJLTRR15ePZLkmdYjaQJxt9V/ 2021-11-03T16:56:49.431 INFO:teuthology.task.ansible.out:yBwKdGqYvnPzmLOpdmVpdQJo7ZOWwdp5tJsCy4q5uX5vIUv6Gyi3uVSGmZnrKUbsqoDNwAX5vx/gFBINIs/ZXXyD4fpUT8aoZeq1XmXdIaWG0nImsrtRnRxKP9NGZEcW8z0WPwgF0Ck+RH0qZUix6Bfbgy7vMXnivuDv+NdjEwyszcrPWvf6Krr0am9Q0+9nj/mjvk6CZLl+CH3HNdkhZuNs+qMG4XW8KhLvQB2p790XVlTHuVU4jWrFmjymggZxkhoCY/ukHH6wi2Pbx6yjjKOj8iIw1xKabJzJ7S8UKT23A41d918DUCUuEG66pN6Z4dnv+fZJxj5iEo1GdycSWIc4SpLTK3z4zDZypVKIzjI4o8VwQaN48HkVjUdTICTvGQfytFi3jX1I5nxIZiiPcxMQLR5hQggP8pn2XtvXpomn0aBc8tVuuZJLVq4zCVnhlVB6IACmb71evjFGOsoSuEsaTufUr48= prsrivas@t480.lab.blr.redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2021-11-03T16:56:49.534 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} 2021-11-03T16:56:49.639 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDOJg56HsyNa5vHHb52G6O5vtv9I2tdlNMjFlG3SVCCu/bVZwVeGzp1ekuNGU10owmnIvhlSldMjLn2wwTwiDjbvYJVVCPvzK63vCUcMevRsvr9EyU16JdAgrmrGMyuYYEb+wh8XWI9Dc72o84fMaBYFSSer1S9nUBL+KVLVhIe7culgADimrDgDVh2t7u2JHBI0m0nKFGJZNcIyYf+GKOM+n7SaQXIYIytvSteU5BvUIKPsDd4z5lVlWiKO3kG6jF2RnKFV6KWBWI9m8YGItdGu25mXELXN9MOlApZF3AoGTmCE2Web7uDVGvO6rz6b+dc7RXTOlbWIhtMoRqQf6mz zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2021-11-03T16:56:49.755 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} 2021-11-03T16:56:49.849 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2021-11-03T16:56:49.957 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC9sZe9p48s6RSpluK1R97X9tsN3tWiidPbdyXvyMUXifYQb3nth9wMWUakrVBp+QGEkqALLIf3so3MfgbRvYexzNfHvRT7Lxn8D/isiCpWa4aLM/0alprV+RI38dR0mI3IVb3aIEYmgp2IY4yzybWTHCmS1VUrUJJ0pOfXOssq0xgo3qeaSQBHYRKasqR8GWAD5MWpG6/kg6gxI7o0aHVVTXy5ADB7NS1LEZr06xxNm4exeYvl9TzXrbGZWf2gC8ZpXhmFHI80A+TKEZRU6wmcju7G69nyEEF8/ebcBKmZB4eTsDN7qR7lc9dxKxiiZcJXc/MWy4bSu69NhOeuq2xx7bdexS8OHKdgYH9gsJolu5zgN20IHzaRgswO3XIe3nh2Ud8wntwBCwmJ9bf5YByITGjpOhntkvWAP3AXkQ1B4HXCdlUNW+4pMexDe2D05LHOwxcS8RVIH6ELu8rWe/QBA+DFcyZz4xE55pbfVt3FCEFrAxUn5qkyzszwPMjtK4XAnn9kmnw2B0k6XLZOZd+n7W8ege6wih+gwfDhYMZX0qzi3HdAK7nk5vEOFibCztLaogzfc6fdFRx8OIWJQXqx5I1641FU3UE6sVxku4IMDTq2GVyUWKdSN6y8gBiDPVe3xu4qM4xghSnaAmfB45IrcNvibBjikNva2FHt8lcutQ== prateek@prateek-Dell", "key_options": null, "keyfile": "/home/prateek/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prateek", "validate_certs": true} 2021-11-03T16:56:50.065 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2mEyxfkW1u84u5BocLCAXESj2qObWkxUd20cmPolhJYlIWLOafa69Dw5A9CjFBp0U6Jx5d0JWcwmAc5ZeffQ7t7vzmtHCZnpKkyZhN9bu6vuCh4N+I643SI/K7tdbB2E8ZE6uI7ORMAan5+t0bNSVolUVq6X1w7dIA5El54K10WNdiwgRKW6e3anNbJ3rBLGEk7gawZqDDOqTvVQmOBGWek3BjSReDvg3rkmTQLxJkiKdxnx4/s7gzH1xTpsS/OludIuSxF0BK064ZcT1t09qcjnT4Os1inV2GfYswt+UhpgqrHqXjEs0Z3y4Oy77QBKL1rDf93CP4a4+eJjqXe+v tbechtold@suse.com", "key_options": null, "keyfile": "/home/toabctl/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "toabctl", "validate_certs": true} 2021-11-03T16:56:50.162 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFmmTp8+YDXU27xnSPxhWwouslpkvp1eULVp63taoXceDOnH8EMWv0wiEjN/P+Fcf/UB+iU8bELyHubZiqnOUr9f2pGX4jkJ6Yty1p5BUK4my7Si5J0/JawEDVQm4HpFWMKUVKYumvz3UFuBoINHVTqr1cLIDAt7mdeKfJqFB45AqxDUIPP4d8iznNCw+QO2CMxk0o/wMNHbygZZrHAiDngXQxwhe6oR17RfolhREy0JjHgojRjxgagpG7O8WW3OPI5Outnz5Pvraas5q7w+b3xbB9vdTObW/Nii5VOpTdIsFzBvp46hLoRSIcB8pRLUrxPJBPOY7G0zbopDRLB2zLNGKIH2zxEHJOeY928uUbWT22b9mGu/L95FFY7hmeRAicCUeixMTgXgczeQlHDn5eqQzLI3PNP7JX/0dauCD27GRKsDUnpIDj9RqgCQDVueIQQMC6QMa9dYQmps0r9Ljp8R84WjYFRP9oOI+POOzbgmRO0JCVnPqz9MKyGVby9/WFBHcvG6xmwACez00oqe62z+rhcavGXViX5CxGMV6vUTXugBa4ZkXlzs+djzYWfZ33ZKVeRIwj2Pu1PRTGzB8e2BaKqJMN0FVtzpV4hom4Y/9vIPyDpjRjNuKzrkYW1M8PxBxn+EJUPMLhr3jFMCo6HSxHoFXy8BzZoFkZWqap3w== JCL@IT-MINI-JCL.local", "key_options": null, "keyfile": "/home/jelopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jelopez", "validate_certs": true} 2021-11-03T16:56:50.289 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCitWIN7jm5KEmB6jqcA+ufKRLodxkJ1GIdaaueCdeXdMHgWNH9FUDCUJk5dcA11MxQO15DRH/cdhb3QHPGiMv4XrBeK6HHYwGttzkcUEz1EDUhpowOrnFETbPa7K39ObmRcgdvoGXOSrNZ4mvptCTz9Xb0MF+Gcg5iJ4g+qZzIVa2/ueecYY8ZorbaH4m/NWUub6Rf01Q04HfxTvKrUFswrbCoE/Aigz/Oh88RiS854XoTvV7rE1tKbyKkaungwMAaQSNnZkq4hi8nbp6pzTxJLV7N/ZYMhVpdxSbWcghFg6cy7aeoCzZMN97WcDhpdfbhRT0dbqCkMmHV6t0usDDnOss2v68UCzkxuA1Hj4uy8rRHjXBtYqI6xsrwCjvmgP05B14TpmFdG+3GdRHx2+QlhnGj7nuPf1KoWW/f3tD85DXoEmVl6RJ4KTrMeEVlOn3kdMLVH0y2hIqPTYMP5cMVNi0EdfeoBdd9kxMKMvoSVBAgmLzGVv17o1lNvVS6V1kyt7gME2PfNR2Np+U1cJVJfUJRTm28mHXBVoDXG9s0zv8+LpQ/gVPS+vsPAfxWWYGW8ZmEV3ZorDOZycSPybjHkPFzDAdUhPG0Db0o+F9ug+22F7pvifsdDZ42zqGGLbyzfIQE3AtyI7LR7Sc5tpN9rKTeSBYJ0yNcDKEj5FUHDw== mfritch@suse.com", "key_options": null, "keyfile": "/home/mgfritch/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mgfritch", "validate_certs": true} 2021-11-03T16:56:50.385 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdbJDYY43Zu0EKAGgrSKEobyXIxdhT0AeTT3yBoNY+YdYDScimJsQzK4hKNJIn1xC3klr0dSnsqeJru+SA4GMhbdcsmVyudhutCw75R/vFoRHOwaeVS2SMiBAtDbAYGQ3dh6LapkmXeLURcycNbebUV4HM9XsYqnuL8J2XyY4tRKgHrAd6OQw4jsXDOxwTRlHM822N/2kgv8iEwiXAJoUoymbr8mnfSxuuaH0oiKBwuaK1wuaDrlJyXkXNslPq89/1f8hJfjixY2dQdW7Xk5m3t5ptQxWENpw79OMRJ4cDW4G7LrtFONom5KQ+9TBZJIhNgJobkqI4qwinrWF8qMW5 dcain@fullsteam", "key_options": null, "keyfile": "/home/dcain/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dcain", "validate_certs": true} 2021-11-03T16:56:50.512 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5bCXQ/B16AKl2oevC93R0U+3OrTyd3XEypouJy4LjIzXLX/Go1C1ViERwDAeHmqiUTqJUL+KZ0kMo8U7zlhNBk9PucbMZCqdqx5hkiCAo4y65lEXmuPGimRU2mK9TpcDTLWoyrD/EEhwzT6gmw3ytVTxi7qAb0tQUCQB+pW4+Zr5u6HZzgu9kLGxp7zsGtMA84Zqm5ql72mGuS0bUN1ajm8LWBetFuzOOW6YGfnKt29cLV2h1itVewA9DKM5G70B91eK4LjoNddCkqVA8L+v6LMQNU2pejUzPap98hmFUpQMNxlWQ0+B7UjfbkHWe+mIsSkdr3FfCfqWvsL0RiHcZ root@lxbceph0", "key_options": null, "keyfile": "/home/xiubli/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiubli", "validate_certs": true} 2021-11-03T16:56:50.596 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs2+6XwRPdiDHT+63Yab00VBulg96/OLEORtNUvm7TDRMpcx4OD2oOnbsE4rG3dOdXNgT17H15c6MWFX2a2uC2h1j4hJTa+dG567HrYXRHG60Kwg5Sviiuq/ETPoH/r3OSvHs0HJ4Y9qYPQ5j0k7gG1XgIY3QHDWNRsXiyRXP9FEfoaOKABDTUoLPjqRX+4ExJd1Nz6DLxqIlre8wfqF5L1l12vec8JU3HB5NSRa87HxKFtLLIAtw7B9H/HggqX5wWVEnXq2qN4o6OOePhaO5cjWAZxOtT+AJTyHkeyaWLwIGOHBVZS5BKti6jLc+PxQxAflYlu9X3xh/2FlA+9H+ABZwMHE3ruUfTVbviYBbtccMCvkzcvFgnvaUCHjd7Iv/T8/JHNfBgFgigcX0Vei0+PZQ5oC5b7YpKT241NA8DCGZNgr1KuL9LazkvbkbpDcaHUOrkWsmSW1RRMC6ffYkbAI9QiBJFpImDAWP1u+1yCGhzPKBfGNt91GcLCD6ML90= sunny@devBox", "key_options": null, "keyfile": "/home/sunny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunny", "validate_certs": true} 2021-11-03T16:56:50.721 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDT+6B5Fr99p8xe9Zo5x8ULcdeA6ne727iVJp/xaguGwGHj6AhuK+R/CjznnHk0yA+6kiblNzjvbnsuzO8T9c8Ayv5KU07MxP1y1pymdYBpEh3NTXmhB6QOTCNTQbT14PL6MVgzK17SqarFCYXoEGuyK9jaBdN9QM76vhEaXc/2Osg+d8boJTkOk7lOPHbzeOCbQU3d+jX0VFOgAQS5OFz3Ug/X7nB0D0LN8bt5FlCZiGO95Iwls/SyPDbVXc9ot8oh4ME7kd7sElokBdYdXPauibRxWKjSDn+hV6kZfSzqwnMFaZ1TC+Av6zNsBuZucNSp3/Ws3T5i9i+21eyS4+qVE8IiX2aVKYuC9afb2lPLbdG2Iy1CqiaeGPkZfiGT9P8iQPOkqae45859UzFAd/XzKjBcmus3lSArKG1EyZl03HQ5P2J60pErCXq8W9mPjxAze2ReLRXJVEgTXnmdhCptcg6PZ4d/k4BEddbdnoX6/BEgFxLEJ64HpTeTVcDXHpipCpxFSRvgyPH/wQGKAvt8AfKBhmeZs7LuyGOhJiOon4DDaShHpE9wTnJ5qlnyg44+u6NCwelGPMAkHKEO4OgcjSsyXm+HYUpxG2s4poloIJzvU6oa3dvJsMZFhcb8CsW75sBYE+YngS/1fg1WU88ZOdaa1yF4F87pVZycpZqY1w== jan@jfsuselaptop", "key_options": null, "keyfile": "/home/jafaj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jafaj", "validate_certs": true} 2021-11-03T16:56:50.805 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4pIcNiZKVIfT9GxyTe/B8G2GyHKxrAle3iExEzLH5bBCKtBrottX1y66KBzQSsrAb0z9h+A+uxlswlY6VM/gNos9VRB6x4NRB4ADY1Bku+JsKs5aXwRB1zz2ueZhrFN9hJwMl1heLICx85ksHBdm0q+JPdUAV+/LBdr0H/IjWveV34WL3Awf7QikzyKhzUciMB2gN9PJXTW0bl8PP56y60ys++L3nMSRJ+15uPcWl8MS3D2vABII0Tu+3PsxzrTTNodElyQ6G/fwyFfWhdp7fkidjR7mDDx8NyWfCl2rKejXv9LnVC0wb3tFoQqvwdz6shN4E51EIEGXHcLUeRpBhL2pFOqIZ0qfnmLIQVZ88RKMTgo42CgtmS5UzxCLOyKOh9GxF7DDEajTinJFJJVbNJ5xF494H2geSnmP1o1AecGsGbCZ4ndhS9unSfIZLAyv1J3wZlAlMsvxXgY7yrh1srUiASVyKK8jba1IT+v3TXR8kQUCNsLKKbvuX/l0IXME= xuxuehan@bogon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBd4eSizc2zAkWDy3mFZUhELO2wXDgkVxDNM/Nw2zT1li3L+6YOjFfdT4NOxpkg5laLd54bdKb5GVRA6iKqAO+aqo53xHHJGC8OY5eJaUPjtpypgIWO4Qzbos2ZzsC+nX5jvi8O9nM+JkabSW5iFv3gGPTqHixFYkxK2NIBa/UtXFOhmb1wZ6CG5Q+WObCosBUEMWokC8ntCADfhi6qDsO2fol6mYUhYEE/0nhQ5uLRrVIM5QM5dhrQVEeAs4KcYpmPDw6q6AKgx/f5zObh5OSJFTVsgx2cqzdQVJVDDx2OD2QVRtZ9fgvI2EGmKSwe1TUQOdqa5bmsVIp2Jfi8MC1 xuxuehan@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwIokXSvPYP3TRlSw0SBgmMFW5jVltGf4c6CqjIW4KKCgsL4c4Vs0FztRyDGc9p0krZUoGJCZHErVToXm6/lSq8JlfrlLX4X4yh37fUifWqc2UTJGeH8kLaP3WEt9vT8eQ9h8t2GqlMlGQU8KzVAdC+c5IwrbGRsS6S7D09/y0rklkhdJ6nuj8cssN/2TzLF9YIvVcFZHdyznk84eB7TgpPKD88MOpTnEgWFlD4vAWEuHpkuxgj1KI9LscgSFcW9/RxLeRxcV0GypZ8jPmczoCnin/ZOstCTpbcmjL 2021-11-03T16:56:50.807 INFO:teuthology.task.ansible.out:d+LW1kgGXXQSkf09iHeJ/cyCw6ooZOkynVwwmI9Uz2ZaMedCUBpG0bKclIerJ9jn+CoTs3U3Orhd9+6MVtCHX9VNJbVa4q0KCS1sp5S3HDrYkgUyawUxdYorrLZcqfRHWWo1luiridmuQdr8lQ6dvNpz4rhsCrAG6S1eeaC/s+o0Rzpv1YxU64JzquZ6WNR+W3U6Bg3iYkSoDCK9byf0rsptyL8= luqixi@luqixideMacBook-Air.local", "key_options": null, "keyfile": "/home/xuxuehan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuxuehan", "validate_certs": true} 2021-11-03T16:56:50.931 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo8wE/b2/EDwV018ENfcENVfXSF7vnXAwBSzNVScgxdqr2DVKTWDzhoI0m5CXTqI2HLnJVFDTgN3t5/fFcmwmSAP8ic8wU1JznSPZXN7qEQrcEsU/cwHeuzyutibg1ItTWWtoXYV8SFnVom8NF3UwXF8I+Vj6VF1EStWCcClgPjLbiOm8vIgaouXtbUIFv25GpadXyUtwiHEq1NKUPLgka8Usm8ibyS9hY9KmNWlD9AfmnwAImc8Urn2dErcsfAltxoGUHCI7ZBHGpzg+6dpynNR+giJMrjwWzREaCHmiJX3g6jmHuuY54eSOs6cxqi3E4jNHVWXPe/tEF3l0klEvl jschmid@suse.com", "key_options": null, "keyfile": "/home/jschmid/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jschmid", "validate_certs": true} 2021-11-03T16:56:51.015 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMOuxxTZyxz/TEVA+ODgtLrNcSsUhKE64BZg/Ub96LLffQXQc9SYT/0YI4wxu2vSsky05PnVSON7jqGxunCt0+AdRqxXhW4ttmRIOz8rq8SiMnnLayV34nIF1G0aYTkgY4ej4+VmSVIHU3bb5mp/Q3D0hxQwDPdgOJWNyW9oYerHbnx6lVgchvsZHB9VO09GNHcAsyUz4p+xBf9iR7eS74p4dZmUbth/oBBqpOa4NgNr69iyZ9a/pNDLxaVQoIpDFrevk5UU5nUHQq2frFNxF/Y2Ja1EBXCavvPF/m5v3J3xNEu3SI/GbLFCPKCKeBDpXJbe1QqTBKJjMl9qGmKEJ3 aclewett@mac", "key_options": null, "keyfile": "/home/aclewett/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aclewett", "validate_certs": true} 2021-11-03T16:56:51.144 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDnQfkkxYWLhMdoQ4RsZxWTXBpbzsssgeRfvV5q+n0mW60dxQ+9rjDXUTsRkmh45B5EhNWlKwyIUcQ5HpgJk6XQvYU6MPboWjriCTngmoUbwNIjOjyazkBbxfbwUiRDb+x1EyVm4Kt0rgj3nhqqxf3krqO2SlKgsVaY9b7+NTfshXb9T73/RRLtvwUOFkjp2B+w0SQs7Ka1GenmHSY/6xP25Mb8q9QW/2nuf1mV9rTd5l2ZHREzgG7gQFhjgUq3w6zCCfcCGwrPb4omqd9xiYLDw8l1vfQ7UPbuOqK1rFM67r7JQDG3yyJjCdWgRUI/ysbbXj0fyT+QOPZGK618V8yZPVlq5loQzYI39zAWZU9L9X9bkQCsJfjU5Yo9f/zOaecnWfUxC84NKpbG7k6XACdUIB9JGGtL6RPtgBU+k6Ts7EemIUCyMYm/GX+PYU3GnmmdqW9yBeyZoldwDCWLq05vjLXj+R3J2UrMl5x+fhP+KEFd1ZgNc7PV+CF35/ux8UE= kachau@localhost.localdomain", "key_options": null, "keyfile": "/home/kachau/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kachau", "validate_certs": true} 2021-11-03T16:56:51.228 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdTLDprr3DXtsgiqrB/miglJwSkY7BAv+OW9sC0ZHQf8Y4dFoUtr7q9fd6W7ehKL6aMjya/8eQSMPRoEIZFD8f2KPCuE74Wgwgv2cpAkFmLjcPtjixqLo3LGFCyEJWGLQxjlthdzx59mpCWgBsEJQmlKP36JIYjILIMtXtsETIUGFa8QPIn9sMdIgutPPf3yB/6DHcR7C6yIAuvbJactEsiCt4C8+uJinvCii3PzYOMAEd+mictPGk8nOCQFZrI+ZmwTz4UbJylxFqnX8lhhoPIUEfNAXlb3lkZWH1LaFOxarbmupuPYcg2rT0+kFtyNr3/mAx0FO01jADYooqcW1Z cardno:000604936806", "key_options": null, "keyfile": "/home/cblum/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cblum", "validate_certs": true} 2021-11-03T16:56:51.364 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQujcXnq9qOCmTWs44GwKOw48BxZGBmo9AeFk3KPRx33Cz5hS7UHPYvJy03KdoMySgxJtuxsOSvjJSPxpXesTtArO5giW8RLyuVsu9q5j/GhPKyABttuGQyexiRokrHQFLVifzgqwsUARwHarWH16Oa1n6fZPFNqSH56c872zS4Pwqgkzx99NKRWh/B+fIk8VmDzzP1qvQAnUXDeTeTOzapAL+8fSNTp3QnhmZbvCHYwUxfSqJyzq1wBL+517WrvzvX7yCY9B0wzm72sIO4daV6UDyAL1B72QpB04vEZZ/skwS/0jmokfBn43HHCMPy/Mxywfi54alFJzIu4pti30V kravishankar@localhost.localdomain", "key_options": null, "keyfile": "/home/khiremat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "khiremat", "validate_certs": true} 2021-11-03T16:56:51.437 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDPUCIVfNQOOG+uAeKWo7tLAdgGO/oQu4exTE3XLfyTVdd8u6pbQCmIDFEnbGgMtlYuWJlCJtxJIzEZJ5HIwf+po3y7kb7HxEeWnEb9DsqJB4nCRE8fHnNLgF3bh9gvE7sOZdNXEKqtsnGcaaS81oXQ6QgkFGQzlOBOqxj7jIvad7AtNWD3WbJi/uTVDsWcukU51tn/aEW+hHMD7Y5QJkLH7O0PrCeEZCUCWoiMP7sNwACBym0DsqX+POkGJOaGx9EeNhGVA4B6o3xYXJPqmFbgMWMfG3/Y3SiheZjxzySMQ7FKW4Q817NaWqbjtLZQsSZw7JNqNFkcdTzVZwwCOJ1AAecajpniKeQLnyiWQ3Z/EkSGGwtDj67wo7uqT85tpOyLfYhQm6Zp+xSh+I6s0YJ0xO/huLKzeT5D1r5vVMk2VdcLD3CCxXUwZgaMUxYAC2Wq6mpIkxVZFQLbGuA/dWkQbFsrpbf7ES+kh+uzp4R6dQ29OjXbVcSfF2vyIuA+VUKoyj370YOZQhs50yuAJ4BSHlEgoJdRzi3XUX48h45XHiZcCY3X8LTX6a+0KAIgqeAWdFhRgQtAxoRYjhOxC3k/boBc8sEQ3o4pkxnhtE5Y/tnM2CPkPMX9HWYirNf9GyZTYcR/5rDf4ETQ18G68WLaNK758OuUPjSkwttcyLd4dw== mlatimer@suse.com", "key_options": null, "keyfile": "/home/mlatimer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mlatimer", "validate_certs": true} 2021-11-03T16:56:51.583 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCoqsDnRqAeNPi5xiIfFl2i2zqcwLTvcAso3R9+QAfSChw4+wcQzmMV8uQz1jt8UReefg2QL6bwvcQPj5Zw7ADqUjfxm1K0ezgH6RrtHgRZ1SVseTvJUvzmkDZ2gr2ToZAJYsaV7gBLJLZ7TOiQ6jzrtitiKqJZdpZfzadjShecY1TdMo8bSS7XfbRcwri40u82TRM825B4L//QuhMI7akPqa3iJbGEAQPhkGLI3kz19ryVG1UwvAkkt7TQ68zz+piXSmB2CMRti51shT4NuAOq6Qe9y2Obicf88ItkYxvySMQ5jWi2bo7Zhwuq9jUH8lK6I6PHrYc8LOsR9gE/hqkXxTQrkOXEDtHN1mr/KKLrXO0GeBdgh8Gj4gcwgd1RUCE+T9q09WO9j7c0piWK9Fs1+q1FjH06/AfmP4qob1TlJMHhtPyOp+lk2XvABP+sqW4vyrgmXu5vJU+Hgq3X9KE7bKh4WkCem24Np4uA+irhpvZmIoHyINsYW46lGfwJvM8= kapandya@localhost.localdomain", "key_options": null, "keyfile": "/home/kapandya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kapandya", "validate_certs": true} 2021-11-03T16:56:51.657 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYh28a9R3a7jpRfPqVAL9EcD+S6KkZzKTfCmRn4s8Fg39kUZsgnWUZQ3jUxKSeBA00te7WMR8CsIEzulm7f+/egNuPcvzNI/SaaCDtVMaxqlEuW85ll8uBQi3s4SLmOu9oS+5oid4cqTrGTcG6uhl29oWS/53SxoGh+T+y6GesO2cYRuTpcUHxGMvIgZe/bKtVGdP1y7rQd62e8emb9UupMBxaHpkYj9AaMuLjovHbBgecWbe4PlzPWNzl+eITVk2d6CxxIem4/bw9VHkRjQZlhL2nbwRgtczLYhlaZAd4ePEtsqA+7nTDjTDhXXHVIf1pDAsJbZEdgQCSGH3FzxtrQ2hZJrH/NIKdHk/OtI06e7POSGj7PcHzTvMjQSFnTpJXinxwzn0lvFSlmABN/00lIE0p2ZXfPs6dgWJ17JnTq6dMUoTiGhfKphPHyfSXJCzfa2VB+lQQ5eAaa9cRXy9pLmWGHXVGz2hsBMtyYmTjR/wr8kdTzeK+Jk4opBtGgcxDustg59b9x/jvgoc3HCdtnkqLTug2KmutB6kb+4KnVO07GsSp0RmNtDmCRcXs9YN/eia9ICtIGDWPc3YDRppsw1Vktza/+ZW5vGzIKILZTo4SNGcg19zQw4Qxd9eLkXfB66vE2CQvA90CGo7mcye+UWjjXLr7i0cDEszAloQ9aQ== srangana@redhat.com", "key_options": null, "keyfile": "/home/shyamsundarr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyamsundarr", "validate_certs": true} 2021-11-03T16:56:51.792 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNEEz49M5KdLcp0cbKrE9iihrH5PEhsPgnAxac+aQwvnTstOOlUOyR+TvHiRa7sgXHuFX/UXo+k3Qmhs5U/sMZWZmmN3Jb3fKT78lSBbGnByclfPoGVTTLOHxwo+RFE79Sp3e30oshJ3DwqYgiX6JjM8B2Y7d5Tv0Ss5uEZswvmCEjdTLDSsoxrsOjVxBpuArOfCxes8PnKDdq21mWldpP66iwFel2GgOGSc85iVHKcOE2OpVMazMzXoMSMFoOpCULKjFfXBuDIqkXwJmtLJaiCgcVmTR56KXh7JpZ7vAeJ9ttv9qgOckqKdI3jEjeDT+TrdLlRWghBE7FRkm0vmun prasad.krishnan@macprasad", "key_options": null, "keyfile": "/home/prasad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prasad", "validate_certs": true} 2021-11-03T16:56:51.866 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRZyOnj6D9tCw5o4+ujpos5Uj1O0XVvea798DsXPRh8bEE1pLkARQJLv4iW6LsG3x/CXruahmwT+6C/VHppIsbgtpadMWdCTPYrsjctPK0/N9nr4kQEL6XeK3/GHFnwqYcc+LySpNNfbKrgN4qWOwdZBhQjSV7K7FJd4ZYSm4wp6TVbvKKp7o+JdeM2zhstfC9AcCtBv/TpxixFFq/+y8/Z0PwqN1YyHduDOr1dr1DWn4ZzrkhiTYfPOltx0jweD6zTJoTAJDqae7kBA+3XH3dUGX3n7Tchsa3bnIKVz9sLQvH/vPro61I+Xig6MQh77JsylNvpwb1LLOEzDf7dxl bigwhite@wrjj37deMacBook-Pro-2.local", "key_options": null, "keyfile": "/home/shuzhenyi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shuzhenyi", "validate_certs": true} 2021-11-03T16:56:52.002 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCu9CW99ySPXSKQpLbefu1A0KqbUQIzvUOJizIu4azEgdlOqUnOMR/GC6jgOdiVL5mNb/U5bg7yQuEHgPAtmPRotvbI6/FSwIAR9+GgiuKSOJCa4s7QzWbHwv9g7wS2SLURw8UERSzlLlT8KQyGj+GkBhToWp4PTXC7te61c8XJNGF6giLmIAky4SH8uPI1zeLwwviJQomjFWZZ1RTbcRJxvGPjPtuUKDlF7QgXTLE9ZgsNopusC6PfRd7N8Udkl2OYPAzA+Jun0IRnQyjRXG3c2sYXQR8YYGICsxvnm4l81vKdxRL34K7LqBUHQXUcoCjJy7+/2/1dca/AMHgi4sZwgeYQ/U9jZg5jbm7o4MpDUFsS8W6OO/ZPiuYTswGBNRXhPmbGFnfa9dBXBS0p+buHF8PcTSLH0Dp2nhHkK1ehGmBFVPw6nPj3wXA9BKuNc+g7AiNUyRKyCWroFJ0VwUQGbI6ZMccO2cI7QpYmIqPCl3am/vUSNq4j0m/iURR7+t0= vikhyat@redhat", "key_options": null, "keyfile": "/home/vikhyat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vikhyat", "validate_certs": true} 2021-11-03T16:56:52.081 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEcI3X9mNxtHRSvi2gVFavu21kRCWDd1erXW3d/1zWnoUvRxe9ft2xA1c3Bmo067GqhFudPcXoYJvz3atAGR9k3n1UTtTJe95XRZ1Zmn/boi6mn7I2oRVFpW/KjxaiM3iN2SHKknMtcs6qRTioapYCU7u924oc0izqPBvkrC73cfVF4emH1ERWEcQBTxWub6N6Mn3xaDmJ5xnGlNvDY70CC9osoa2VRNJM7GU9eUQClHirPbSooDvvVKfTGD7YfMH76j6PdDR/a/cgNeiMevtMbeFGhu1SOEHb1ihCNoGq2vQFPPWwwTCzMI05XuUEc4iaxw/kbGBMpWbmIvswIMQrRwgwLZk67GGpf/dEeutI6WseeNKNuHnTzzx43AW9G2coGq0RKE7OuA5Cqj0CO7Kcf5UA4AqIIroLETVBrrEV6qzpkD74QBZXPl7gbSZIyxLpvtQoMfSSzN+Mj184LmQoURCqIT+yM5gddF5/S5I+rpw81vBj5M9ZskyzVQAo0H8= michelledipalma@mdipalma-mac", "key_options": null, "keyfile": "/home/mdipalma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mdipalma", "validate_certs": true} 2021-11-03T16:56:52.228 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCipD0d1tI/Cff3AeOZTQ9eb6pZ/NOCAOw7MZG/JgpCs8+T3WqiUEYefsb2PuSa85vV2GC7kq0M9ux7+U+vZZf05geYiU2S9eTucLEu2+YE+RaAvIK+ZB134/33JSPGIVubXgGWZw/ZFCEYXmYk3kyEg1YDIk/BEaoG7PsDiSB4j0W7/WiV2bEEdSlTXt5C1/kH0qn4H9wQMiBSpdMgdaRWw60HjObDd6LASeUVgabbyclypravZdpYBnits4ayoN8VG1aZe4Qhm1SzP/CfxAAYfxo9NbqSBWJmn3iSN8UKHhCFrRCTaYic3z3usr8784VsA7z/LxK0fs6xjhoxRTa3 shraddha.agrawal000@gmail.com", "key_options": null, "keyfile": "/home/shraddhaag/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shraddhaag", "validate_certs": true} 2021-11-03T16:56:52.292 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+tqgRGdlbb7NMefZVPCtbzVZBapxmmPcKaUtqT7E/9AKRoHuroZQ7KJQfO15POpIfEja7BugBt5F95aW2Bd9/xhulzC4/JjiZYc6ST5Na03BrcudI3YcktoGR8Mm7ZVzSojvm0SKtLLHM5Su17JpCC7BitMdao1mETlUOklEXc6pgbr9XHOayo1zeuos8DB90iA8TQ2soR/POD1Rix4GKniCHXz10+iXQFYPsif1HsQ3rh8Vi1M0cTq98bMSmfsKODil8KIxf+lvLCL4GAgDYEyxnstiTFh5GAcIMOOo2dl1fDb/Oa981CxWU+xxJD+M7AEJVeyAqqAnShvZU6y1L kiefer@kiefer-uchi", "key_options": null, "keyfile": "/home/kchang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchang", "validate_certs": true} 2021-11-03T16:56:52.459 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpE5V9JG1WhkHSYqjqaijBb1MLyjcKQlmj5WQ8w5832n6x9J4yDO8wI4qmiX3XeiXPlmW6XjAq5keyukrQe/yVI6NDXzgRxaHDmNj1AR73gyUVqRXpJprrqqsnLHawIRovrBD/HNhTv/2G8xG3BHakac1+qj7tZe4Jjx0aG1HbRhhJ5Q98EgvTkDAhktnlvF9OXXC1WI7QirHx0pRHa40SYyFLfOQDbQSCzb1TAAY5NGn3SwK3s2mWeoEsAJpCLrkNfSTbR3Z0mylAJUn+PayqhJegjwKmLaONjx8ZHXN970JHgojBBGOD30J/NnJQ7N6eRl+ekjpHmLAYGbe2oCwD gkyratsas@suse.com", "key_options": null, "keyfile": "/home/gkyratsas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gkyratsas", "validate_certs": true} 2021-11-03T16:56:52.502 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDjj3bbbLvd4IHeFrSJyM9N6uKXA7ElQrzfcisKRGjUPON8IsAWkXQeAnHoBsIQv7tRGudXOrK+tbfv7crLI0/CxL+o0cCVW8bxuTEtdz07ZFsND9cppCc7Yq+wHsg7nQ/n0QfxnlYc0T9pATXUoT+aW4c3NQoDbsupLSkNs1C+G/MEBzz6iH6tgrv43OAOUccfQbC+IYx4RIKL5MteEjVrZanNELIG+jsYe+CZLdsONk1nR8+xKQw19Sm/jkKdl1K9NXoVPEBQmFXrYl6G+7tCdWeU3Vq24lM1X5p1mVspcG6FyyctcuP/8+zhL2L208DrXlOVPcqHR/4FEf3Wmk89 vereddy@vereddy.remote.csb", "key_options": null, "keyfile": "/home/vereddy/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vereddy", "validate_certs": true} 2021-11-03T16:56:52.689 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC9sZe9p48s6RSpluK1R97X9tsN3tWiidPbdyXvyMUXifYQb3nth9wMWUakrVBp+QGEkqALLIf3so3MfgbRvYexzNfHvRT7Lxn8D/isiCpWa4aLM/0alprV+RI38dR0mI3IVb3aIEYmgp2IY4yzybWTHCmS1VUrUJJ0pOfXOssq0xgo3qeaSQBHYRKasqR8GWAD5MWpG6/kg6gxI7o0aHVVTXy5ADB7NS1LEZr06xxNm4exeYvl9TzXrbGZWf2gC8ZpXhmFHI80A+TKEZRU6wmcju7G69nyEEF8/ebcBKmZB4eTsDN7qR7lc9dxKxiiZcJXc/MWy4bSu69NhOeuq2xx7bdexS8OHKdgYH9gsJolu5zgN20IHzaRgswO3XIe3nh2Ud8wntwBCwmJ9bf5YByITGjpOhntkvWAP3AXkQ1B4HXCdlUNW+4pMexDe2D05LHOwxcS8RVIH6ELu8rWe/QBA+DFcyZz4xE55pbfVt3FCEFrAxUn5qkyzszwPMjtK4XAnn9kmnw2B0k6XLZOZd+n7W8ege6wih+gwfDhYMZX0qzi3HdAK7nk5vEOFibCztLaogzfc6fdFRx8OIWJQXqx5I1641FU3UE6sVxku4IMDTq2GVyUWKdSN6y8gBiDPVe3xu4qM4xghSnaAmfB45IrcNvibBjikNva2FHt8lcutQ== prateek@prateek-Dell", "key_options": null, "keyfile": "/home/prateek/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prateek", "validate_certs": true} 2021-11-03T16:56:52.711 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+EkUbnaeC7Dqtq2N4/dX9v+WGlrgRS2ygHhlbZy4KHbd6+Q9T5NAWtqnFwqcbFpYYUBwMvz8zLo6HYN6gNskRN0qTiFKPQdtnydF0Tx1KHrdGGcX3G2lxyGH5P8nrqReXAPPvRiVc2/Mj7APCTmY2JftkSS1qYcTleeueivVyho6z6uD1AyC7OXANhtn7kGeIY/NvVKBhDr+3TuPmpPwPCQMcJUyZWCHYyp47v2vzNBy2Etp7WjGwOnfZ5ULiJGjH6PCXx0YfpmTUOPVkNbV6Om+0YJ3fcLlmQ/2nsdsWfOKvoAr+1uzO81h7ZBX9LVEitj8YU3WeSCV2vqS2VvOpq5NyHXWGq+ZkOCfhy4Z9MvMG+6hqp4p4z1JMpKsZ191kP7eAQua41GkTSQPlEfF+um1UV6ZjNzhB68utOXyGDoDcxct9oOdnhhjwu5W4/0t9iBHuMn02iHRYVrUWOH2FXVysR0p5x/L2xdvNPiTgFAZ2eN+EFWmdan4e9mAAw3M= gbenhano@localhost.localdomain", "key_options": null, "keyfile": "/home/benhanokh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "benhanokh", "validate_certs": true} 2021-11-03T16:56:52.909 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFmmTp8+YDXU27xnSPxhWwouslpkvp1eULVp63taoXceDOnH8EMWv0wiEjN/P+Fcf/UB+iU8bELyHubZiqnOUr9f2pGX4jkJ6Yty1p5BUK4my7Si5J0/JawEDVQm4HpFWMKUVKYumvz3UFuBoINHVTqr1cLIDAt7mdeKfJqFB45AqxDUIPP4d8iznNCw+QO2CMxk0o/wMNHbygZZrHAiDngXQxwhe6oR17RfolhREy0JjHgojRjxgagpG7O8WW3OPI5Outnz5Pvraas5q7w+b3xbB9vdTObW/Nii5VOpTdIsFzBvp46hLoRSIcB8pRLUrxPJBPOY7G0zbopDRLB2zLNGKIH2zxEHJOeY928uUbWT22b9mGu/L95FFY7hmeRAicCUeixMTgXgczeQlHDn5eqQzLI3PNP7JX/0dauCD27GRKsDUnpIDj9RqgCQDVueIQQMC6QMa9dYQmps0r9Ljp8R84WjYFRP9oOI+POOzbgmRO0JCVnPqz9MKyGVby9/WFBHcvG6xmwACez00oqe62z+rhcavGXViX5CxGMV6vUTXugBa4ZkXlzs+djzYWfZ33ZKVeRIwj2Pu1PRTGzB8e2BaKqJMN0FVtzpV4hom4Y/9vIPyDpjRjNuKzrkYW1M8PxBxn+EJUPMLhr3jFMCo6HSxHoFXy8BzZoFkZWqap3w== JCL@IT-MINI-JCL.local", "key_options": null, "keyfile": "/home/jelopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jelopez", "validate_certs": true} 2021-11-03T16:56:52.930 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBYtdHKBXQcXQFkEW8fSwlFK/xZM16VcFY288ABI/DPfejTStHhxiRvmlsWzMy0Ar5WadZ8FU1ptjA3MKBEY1Do1r1Ul369k3zHCvxXyg9QGGMNPYfHmnNeoF4GEgoJpuounPs0mkLmrPTR+cR+/XuaZZt38nsad8M/zJetnolvqCWpcV87D7+Rt8F32J6kGDknC4ikEe8I0n7F+goOmJhlgkqegbKEeJ/EMnOItwVZHFwaC+sxOq4IlYp2f1aZMKGAtr5+ICwYH1SMCcvnp3qWL8O//t2tvguZgQxzIs6Nf5GaxwdQkh4CvjbWiCkIbGFJ57YICS6k++KMXMbEnwh jsalomon@Josh-laptop.tlv.redhat.com", "key_options": null, "keyfile": "/home/jsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsalomon", "validate_certs": true} 2021-11-03T16:56:53.117 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdbJDYY43Zu0EKAGgrSKEobyXIxdhT0AeTT3yBoNY+YdYDScimJsQzK4hKNJIn1xC3klr0dSnsqeJru+SA4GMhbdcsmVyudhutCw75R/vFoRHOwaeVS2SMiBAtDbAYGQ3dh6LapkmXeLURcycNbebUV4HM9XsYqnuL8J2XyY4tRKgHrAd6OQw4jsXDOxwTRlHM822N/2kgv8iEwiXAJoUoymbr8mnfSxuuaH0oiKBwuaK1wuaDrlJyXkXNslPq89/1f8hJfjixY2dQdW7Xk5m3t5ptQxWENpw79OMRJ4cDW4G7LrtFONom5KQ+9TBZJIhNgJobkqI4qwinrWF8qMW5 dcain@fullsteam", "key_options": null, "keyfile": "/home/dcain/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dcain", "validate_certs": true} 2021-11-03T16:56:53.149 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9ltUNitnn0FNwMeiGkBnozpBN5ziVR2MreaqkfqTQY2TgI24559xuILyy+TnAKumhemZ8i8nIQ876Iso85OzIXbUbRM45hpL/bMQX7pWyMIRNeFKrCDwf18OT0DPc8aeDOAFQ6YoN0j4awUENjdZuuAW80cfcjoQO3m7ORF8wBF0IvMWwPDIQQrz+Li31ECgXg6VC7K3e+BNHlfNQ/2/ahS0+V4J6JYFsylZL0r2tydhDMd6rR8QA9c9BgeBz7+KWobGtiZVe7KReTzJ0O38PxNd8br3aS50j5BiN63gch8eFxPAROaUMnuiavu6OTDy4/aeL3Zc5WpUy3VyyumWf jolmomar@localhost.localdomain", "key_options": null, "keyfile": "/home/jolmomar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jolmomar", "validate_certs": true} 2021-11-03T16:56:53.339 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDs2+6XwRPdiDHT+63Yab00VBulg96/OLEORtNUvm7TDRMpcx4OD2oOnbsE4rG3dOdXNgT17H15c6MWFX2a2uC2h1j4hJTa+dG567HrYXRHG60Kwg5Sviiuq/ETPoH/r3OSvHs0HJ4Y9qYPQ5j0k7gG1XgIY3QHDWNRsXiyRXP9FEfoaOKABDTUoLPjqRX+4ExJd1Nz6DLxqIlre8wfqF5L1l12vec8JU3HB5NSRa87HxKFtLLIAtw7B9H/HggqX5wWVEnXq2qN4o6OOePhaO5cjWAZxOtT+AJTyHkeyaWLwIGOHBVZS5BKti6jLc+PxQxAflYlu9X3xh/2FlA+9H+ABZwMHE3ruUfTVbviYBbtccMCvkzcvFgnvaUCHjd7Iv/T8/JHNfBgFgigcX0Vei0+PZQ5oC5b7YpKT241NA8DCGZNgr1KuL9LazkvbkbpDcaHUOrkWsmSW1RRMC6ffYkbAI9QiBJFpImDAWP1u+1yCGhzPKBfGNt91GcLCD6ML90= sunny@devBox", "key_options": null, "keyfile": "/home/sunny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunny", "validate_certs": true} 2021-11-03T16:56:53.360 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoefxOfeArToGucfWG7y1tyLF2I1J/IkBSlmxxEJyHyDUB6ElXjYIJReoiPfmB0xLEmINpmt8QA4XN/8UbR1r9LFOexlZZbX1LSbihi11LXCvfIJWqkHVwIyKHRTu/duMrntBFVENJ0LVf0GSWVTtllXaDA+fMkp6/ra/hlRHQbx19UXy/9+Pogj8Xl+SntnMGiHlX8FFq2A2tUqhdQSf/1GW1HfUIa5WtwdEqIpKpAgMM7sOW9i3QAmLAMo8HiKrArSiy7jrRUODCqsTJsfifxEbmnOS0ZtmDh0eeWL/Dyqbz99Rg4WbFNeK8oyZYr7D6bEAhd2az2A+wycPx4Nq7 julpark@julpark.remote.csb", "key_options": null, "keyfile": "/home/julpark/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "julpark", "validate_certs": true} 2021-11-03T16:56:53.562 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4pIcNiZKVIfT9GxyTe/B8G2GyHKxrAle3iExEzLH5bBCKtBrottX1y66KBzQSsrAb0z9h+A+uxlswlY6VM/gNos9VRB6x4NRB4ADY1Bku+JsKs5aXwRB1zz2ueZhrFN9hJwMl1heLICx85ksHBdm0q+JPdUAV+/LBdr0H/IjWveV34WL3Awf7QikzyKhzUciMB2gN9PJXTW0bl8PP56y60ys++L3nMSRJ+15uPcWl8MS3D2vABII0Tu+3PsxzrTTNodElyQ6G/fwyFfWhdp7fkidjR7mDDx8NyWfCl2rKejXv9LnVC0wb3tFoQqvwdz6shN4E51EIEGXHcLUeRpBhL2pFOqIZ0qfnmLIQVZ88RKMTgo42CgtmS5UzxCLOyKOh9GxF7DDEajTinJFJJVbNJ5xF494H2geSnmP1o1AecGsGbCZ4ndhS9unSfIZLAyv1J3wZlAlMsvxXgY7yrh1srUiASVyKK8jba1IT+v3TXR8kQUCNsLKKbvuX/l0IXME= xuxuehan@bogon\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBd4eSizc2zAkWDy3mFZUhELO2wXDgkVxDNM/Nw2zT1li3L+6YOjFfdT4NOxpkg5laLd54bdKb5GVRA6iKqAO+aqo53xHHJGC8OY5eJaUPjtpypgIWO4Qzbos2ZzsC+nX5jvi8O9nM+JkabSW5iFv3gGPTqHixFYkxK2NIBa/UtXFOhmb1wZ6CG5Q+WObCosBUEMWokC8ntCADfhi6qDsO2fol6mYUhYEE/0nhQ5uLRrVIM5QM5dhrQVEeAs4KcYpmPDw6q6AKgx/f5zObh5OSJFTVsgx2cqzdQVJVDDx2OD2QVRtZ9fgvI2EGmKSwe1TUQOdqa5bmsVIp2Jfi8MC1 xuxuehan@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwIokXSvPYP3TRlSw0SBgmMFW5jVltGf4c6CqjIW4KKCgsL4c4Vs0FztRyDGc9p0krZUoGJCZHErVToXm6/lSq8JlfrlLX4X4yh37fUifWqc2UTJGeH8kLaP3WEt9vT8eQ9h8t2GqlMlGQU8KzVAdC+c5IwrbGRsS6S7D09/y0rklkhdJ6nuj8cssN/2TzLF9YIvVcFZHdyznk84eB7TgpPKD88MOpTnEgWFlD4vAWEuHpkuxgj1KI9LscgSFcW9/RxLeRxcV0GypZ8jPmczoCnin/ZOstCTpbcmjL 2021-11-03T16:56:53.563 INFO:teuthology.task.ansible.out:d+LW1kgGXXQSkf09iHeJ/cyCw6ooZOkynVwwmI9Uz2ZaMedCUBpG0bKclIerJ9jn+CoTs3U3Orhd9+6MVtCHX9VNJbVa4q0KCS1sp5S3HDrYkgUyawUxdYorrLZcqfRHWWo1luiridmuQdr8lQ6dvNpz4rhsCrAG6S1eeaC/s+o0Rzpv1YxU64JzquZ6WNR+W3U6Bg3iYkSoDCK9byf0rsptyL8= luqixi@luqixideMacBook-Air.local", "key_options": null, "keyfile": "/home/xuxuehan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuxuehan", "validate_certs": true} 2021-11-03T16:56:53.572 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdhange", "ovpn": "pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD03otezqmbR7v9dq62mWH5Kkg692jdmDZk6Ve4vux0A0YhxBuXDCyBWLE1Z4lLNxQahvK0pGsgokp+Uk+Q6aU+ZnF/E2jLp6YWerFXIHY3M7byObJx9+QuVBfNI0bZBkV3+vSjhSCGOYU+TrTHsKZqhsTcejZjmL3gFLmWfU6dTh0sX05qz97cdJl3CMPhCIR3KjtkiPygj417/N7wet/Efv1t79TvyS7dtK1oz+zRLEpKi/fZjvpXb8xec7as72Y0XexsWMDyZJ/N8PijTQ5L6EfLMV7ae/bWGzNAKsoC1dHVY7q/Q4pzktC6zU0Rc9sDRMtrgFg+q+04BfveSLfxTb9tqf7SbSvh7IAG1eKiEuDT/uy3T6Uiw+CiZ0eIqYzZscO/aDjUArGbS8eyaycybdYCBK9Jz38i0mYKkXg855yoVuIExr6MXieYwyVoVW8gbx1bAslaafrzra2dkAFByOJ0sbosPFRgR2+boUSxD6IaE+ALgno4T/0CI2RMiH0= prashant@cephdev\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4m0LAcC7XiFiXFtYcCvZqX1CtyV9JU+VUEawZJpWrGTOr8c+vIexq7+WKg+eIUMot4+dtfwG8lByS1VofglIe5/quv17Xj7dO/hF2WkV8O7RYt7NhzmrKg54yB73pbRpvqzGceWWDvGRsWQGzbY7ARqOp4FPIDJvgXABCkak+hmjXyXpvXN418CBDbDmxHWwFb9QtB/MpQFiFrfl1EMDj8DaXZNWgo5J/qytic6LgiyPHhHfhSylk4cl9TGftNatobpMjqBT0O2/plk/qf+7pHPuvGSgihjijdpEPzVfctXE2OmsQAIZgkzXeCqts+s6ZYdUKq1FvpV5rMM8ZLqfCkDjEzFENJS68YYE2EjxAEgMzj7h/sWc4B5I5pxkAqVAQSGjYRu6rCCbFz8PXUdTyW+Vs1YeylnlBMiCa4xE4gwvW3HlWEnumtyURd1PRENrGJsJ/0uX6bpltbSg4sGACRYpLPJMm3OhqI3Jm/J/e2NLjfCw+svU2447xnfJOh68= pdhange@vaquita\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCoeyVaMjhcGrWQB+4Rq+SSIi8+OY/kx2s1vofiGt1gu3RNtFmG0Df1VITJJ32dbzaW7kTNxaRvhldNd/n7yEMssG+Wg+a5WKvSudlbVMmscbSXdFuVbcCW1jNrTGCYrxL7Sn5e7NQEQfAsCbfXwTcYBw33NdBVovC+gs2V3sZGvX3Y0akZONNuVg1mhu0L4m1/5IegJh005KeilwetHA2A5Dr10RG6oyIgT7t4z5gZXsrOiG3UQBXaekILYV9jxMWoFXGkVLc8TpL9SUAdxE3Owc0u1XiPqYYlEN+zDvE0554Zj8Oc2ZO6HRXCZXQFvzXeCD12YswcOfkIDmJRF7GT prashant@mauli", "key_options": null, "k 2021-11-03T16:56:53.574 INFO:teuthology.task.ansible.out:eyfile": "/home/pdhange/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdhange", "validate_certs": true} 2021-11-03T16:56:53.772 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDMOuxxTZyxz/TEVA+ODgtLrNcSsUhKE64BZg/Ub96LLffQXQc9SYT/0YI4wxu2vSsky05PnVSON7jqGxunCt0+AdRqxXhW4ttmRIOz8rq8SiMnnLayV34nIF1G0aYTkgY4ej4+VmSVIHU3bb5mp/Q3D0hxQwDPdgOJWNyW9oYerHbnx6lVgchvsZHB9VO09GNHcAsyUz4p+xBf9iR7eS74p4dZmUbth/oBBqpOa4NgNr69iyZ9a/pNDLxaVQoIpDFrevk5UU5nUHQq2frFNxF/Y2Ja1EBXCavvPF/m5v3J3xNEu3SI/GbLFCPKCKeBDpXJbe1QqTBKJjMl9qGmKEJ3 aclewett@mac", "key_options": null, "keyfile": "/home/aclewett/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aclewett", "validate_certs": true} 2021-11-03T16:56:53.783 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDg7xW5+Mk7pG5MjLxOc7GTvDTCPBJj2L3MEQZIapmmo6t8uPODwDewXncSzrKDH6zFVE4TTTD9fJdu+BhyT+jCFgXrEOMMabj7vJsg0wO0SWqr+4NECFPhD9y4ewfYiewBeXisclA5Bto91v93AtYpZBCdmPYSi0BEcPVc6S3iBUD0PzIrL62I/kBAtm0f/bScRiT7XvQJXvkbEObW2yS2XhUv2qgq1VlFvO1jl3B85KMTHVb8+LrKpQWOKfRfxJ74c51y6tnEjZLFO6Xn0JevqOhd8gPQIXueFZOM0g0pNp1mbtGaaBlEIibbrzo2p9kzsdvJeHMq1XuMIi1/Qyj9 dpivonka@dhcp-41-144.bos.redhat.com", "key_options": null, "keyfile": "/home/dpivonka/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dpivonka", "validate_certs": true} 2021-11-03T16:56:53.993 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCdTLDprr3DXtsgiqrB/miglJwSkY7BAv+OW9sC0ZHQf8Y4dFoUtr7q9fd6W7ehKL6aMjya/8eQSMPRoEIZFD8f2KPCuE74Wgwgv2cpAkFmLjcPtjixqLo3LGFCyEJWGLQxjlthdzx59mpCWgBsEJQmlKP36JIYjILIMtXtsETIUGFa8QPIn9sMdIgutPPf3yB/6DHcR7C6yIAuvbJactEsiCt4C8+uJinvCii3PzYOMAEd+mictPGk8nOCQFZrI+ZmwTz4UbJylxFqnX8lhhoPIUEfNAXlb3lkZWH1LaFOxarbmupuPYcg2rT0+kFtyNr3/mAx0FO01jADYooqcW1Z cardno:000604936806", "key_options": null, "keyfile": "/home/cblum/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cblum", "validate_certs": true} 2021-11-03T16:56:53.995 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0m+8wna4RnrOcuCUj971EBe4cFiTXVkEWkIsUX3yeCUwMaS/ZjaoOjcdYX7cYpCp1ilo4FSO5BsochU0RiMbQEwLGcE0zB2q1ac8+yYKCuWWyknig2t4bOReu5OrXvl6mj/9R+p7clFlYhwIoNG5ZZ/2eTbo3nH7gWgaOKaPbTtvEGg/5R/qjQKg8OTC8oix6kLsaNTisQJ78Zd7b4ukg9bO0jlq/2UhQOBM072sU1/kJpWTkcA0xxm3nsC4rWPwapgz7UwBmMdyx116GZgWdfQecTLKO7b8pyJw21npk2ntUmeMi7WPZGX6VI3IpaAhBjW3KiATAb8JbAQh8F9bmejkw6x7zrP3hYKQOg3A+U8DKRRo/Ad+f0zz9gjC42DUvMEOr09RBdxZcEAPY7H/YRWEqbA2zPX3Gfth4b7/letzM3m9ToUvVfSt1CdjaZD+gIia9pCgDMb+j1j2w9reG4F7Us+W/1f5pYCyFyrm4XO3AdoYM9/FOAOR3vffTIQU= adking@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJOefD/VJfS15LRyuVDLluKv93Zgocsl8BhiDMimvBNbrGWc795v2+eevWuwaL/SlL44SnsNZ9Mz8FVn9YuFoQiHLg7cHZNLg9EJh5SMXzZzwLZHnF02MdI2CU6U++9nmh8FtBrdBWKzZ7o+01wOw+52dYOxvP7IJ7C/vjwOLgfigEN8Nd1PrxALVNIQz7fsiet0n3odk9PJfoqTu3IloXm9qbQiPFyfMA6b8a20RdLZCJDGGG5rd6gtsyeFHoAPZEz8FEUlNkow7NM7ZhdxOz/fWXyMZ27KmY8WSAgflbxysMzexyifkGzypiE3NQ10hdPGBgoYsHrAN4Qmte1/Z4aI5nxrNzLy/uNc+L/Bp/0XkLj9zfoiFFOReXvlz/laAV6LpbHyacr+iCwv9rlI7echXmvC6mSJXfBPuyR1uromUXpYRhEkn5uiOsCIQCuYpx2jeil7JV+LJlF62lLSndOv4g12Gq2TEZ4Bhsr2902kftPkLFenyfDTh3A2qGGNM= adking@localhost.localdomain", "key_options": null, "keyfile": "/home/adking/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adking", "validate_certs": true} 2021-11-03T16:56:54.203 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCs6q2b5NcUsNXl+w1Mrc7idNvGdwWkL0XRX5zTw22UNpQyqPkSrRTFKi4NxoY8ucv8JoLnaJL7hYSRHKTh7SqH78uPLZX0FLtoHvpYLgUgJzzJ5AOW5uekIhCMCm+3egmFnEBwFcNxtGnU2es02ByXNEXRxt4/LQ0f8K6lMqxVRSKYcltvWzlA7WFFdTnYMunbCYOdpw88RXm2pYjGIHRBnie7+pFiFSS6K/mkzQcfLiJmahgLSaYslI0ho1vQd8xfNLmZ700C5F4VTUlT/mJK/dI2LF3OfaPTsyqKNzOchrjLFx5b6dhZ4WZyqu7VKsHNm+xHZwT8VP3LPGSVVBdmEsiCsmV4g4QTI5ssZLi7siZDpaFp72nVAIO1i6kBGuiF3or+J4LmIh8ZRKzTc0DRDOw7S+IkWu1fv2UhYg83z6wrz4HeczI+bnolFFWW6HgGV3MFaUeRTRvrbxw2DPoEGQ+cjP0wsjyEhxlScwdh8W8IbVpNl+7mBe45YGUH2hg83Lst5QQxD4PHelriu40moWRHFKCnKfNesb4L1uqu29o8f+/PgpLdrWE/P5zIKe2ReWJyScYAI7jTMlFw+xgf+0TKryk44NMy90l3ei0umlx4qsFbbEv+lhzAALrUJ/da7fsQw7eIVykmcscBeYr99/YSYk4MGdk4jVCiHsQkdw== amathuri@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJ3ch+pU88/tX5TVVFl/1pPief8drYLRkK4BtfJk+xVNxX9qkDtOtiLAU+OugYF1/xk9hE9zTbm3UBL23unrzhlR80t8f2FiF1YFhHVzeZ/p/OunmkU0RFQzkMVuwKVOUNulw9FwpEZ/0Bjfxvaims0SBWaCzBgE5g0R+Ofdq/bVF82K8vU5Laa3KZ/cCdGzFgW00cBvPAKIpQ93g8TRqsCsFvR37nY9v5SU1hJ9araEp6LRA86tUt3EFtxD2tRcmqKGtCb0kOr+rRXWVGiaUBiedHxFudIy0+TNj4fNrJwpoC0hcBO9ek4Bh6hM7Zqzum9HpkPGlz84jkm+r+xMThaZCh8iFkgxyK+bw8B8smaVJD04ob9Wb4RYoFYfQ7TEkPkO9gr+UsSesbP9fWfucemUqwK0htOOtpc1YRwdOPcdPJWAfx9J5+YVL8BTWYBaTdmHslOKeUGM5fCob6QxC6Er9xweys9jh7MXttSpe013bY7P4ZDRF8uWvygpsytPE= amathuri@redhat.com", "key_ 2021-11-03T16:56:54.205 INFO:teuthology.task.ansible.out:options": null, "keyfile": "/home/amathuri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amathuri", "validate_certs": true} ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDPUCIVfNQOOG+uAeKWo7tLAdgGO/oQu4exTE3XLfyTVdd8u6pbQCmIDFEnbGgMtlYuWJlCJtxJIzEZJ5HIwf+po3y7kb7HxEeWnEb9DsqJB4nCRE8fHnNLgF3bh9gvE7sOZdNXEKqtsnGcaaS81oXQ6QgkFGQzlOBOqxj7jIvad7AtNWD3WbJi/uTVDsWcukU51tn/aEW+hHMD7Y5QJkLH7O0PrCeEZCUCWoiMP7sNwACBym0DsqX+POkGJOaGx9EeNhGVA4B6o3xYXJPqmFbgMWMfG3/Y3SiheZjxzySMQ7FKW4Q817NaWqbjtLZQsSZw7JNqNFkcdTzVZwwCOJ1AAecajpniKeQLnyiWQ3Z/EkSGGwtDj67wo7uqT85tpOyLfYhQm6Zp+xSh+I6s0YJ0xO/huLKzeT5D1r5vVMk2VdcLD3CCxXUwZgaMUxYAC2Wq6mpIkxVZFQLbGuA/dWkQbFsrpbf7ES+kh+uzp4R6dQ29OjXbVcSfF2vyIuA+VUKoyj370YOZQhs50yuAJ4BSHlEgoJdRzi3XUX48h45XHiZcCY3X8LTX6a+0KAIgqeAWdFhRgQtAxoRYjhOxC3k/boBc8sEQ3o4pkxnhtE5Y/tnM2CPkPMX9HWYirNf9GyZTYcR/5rDf4ETQ18G68WLaNK758OuUPjSkwttcyLd4dw== mlatimer@suse.com", "key_options": null, "keyfile": "/home/mlatimer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mlatimer", "validate_certs": true} 2021-11-03T16:56:54.414 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a28ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a28ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCt4sQTrDDg562IWY9Bw2fxFR7Y7b5lfo9egHa1qNFR5K4gt8lUwQXgzcxZK5dgSTZXHDGxrbYKWsdb8NbpgU/YjSqDUIptMZrhTI6pdHKZ2usY+nieaJj0NIZP/S7M93BbJeoNBYACjvDUr+vXEg6ZL3uAj+zFXr/4te+p4NqJBl4bSpvjcRK12O+F6HE3Q93j0Vlfs8IbXP4vcPcqOW+2usXw7JPdfjwuACJxrLTx/0mBsv8qXJDfOpPIHAMA/9SdTAzhs6XMo+fOwVOKkV+ki2fO3sl8pE16aFOkI3cYX36nkYYpZ2bJLnIPcLv37i3Nypl8eNLA4gzjkDf1o/KLO5vXOs76H189WGLqjh0SHAx/3KO6mrIhTq8Bv6YWoT3ntg22fXaxeajNhHcQ0bKZtGOAb0W1FcMRbSTkgEFAassYxlpTtuaCDIuA4fUVKW97Zu62eDDsSUtTWak/Yf940qAbptPPAw4ljmrvZRcjM9y6mGU8lKi5H4QoEOfbPok= nia@fedora", "key_options": null, "keyfile": "/home/nizamial09/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nizamial09", "validate_certs": true} 2021-11-03T16:56:54.425 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYh28a9R3a7jpRfPqVAL9EcD+S6KkZzKTfCmRn4s8Fg39kUZsgnWUZQ3jUxKSeBA00te7WMR8CsIEzulm7f+/egNuPcvzNI/SaaCDtVMaxqlEuW85ll8uBQi3s4SLmOu9oS+5oid4cqTrGTcG6uhl29oWS/53SxoGh+T+y6GesO2cYRuTpcUHxGMvIgZe/bKtVGdP1y7rQd62e8emb9UupMBxaHpkYj9AaMuLjovHbBgecWbe4PlzPWNzl+eITVk2d6CxxIem4/bw9VHkRjQZlhL2nbwRgtczLYhlaZAd4ePEtsqA+7nTDjTDhXXHVIf1pDAsJbZEdgQCSGH3FzxtrQ2hZJrH/NIKdHk/OtI06e7POSGj7PcHzTvMjQSFnTpJXinxwzn0lvFSlmABN/00lIE0p2ZXfPs6dgWJ17JnTq6dMUoTiGhfKphPHyfSXJCzfa2VB+lQQ5eAaa9cRXy9pLmWGHXVGz2hsBMtyYmTjR/wr8kdTzeK+Jk4opBtGgcxDustg59b9x/jvgoc3HCdtnkqLTug2KmutB6kb+4KnVO07GsSp0RmNtDmCRcXs9YN/eia9ICtIGDWPc3YDRppsw1Vktza/+ZW5vGzIKILZTo4SNGcg19zQw4Qxd9eLkXfB66vE2CQvA90CGo7mcye+UWjjXLr7i0cDEszAloQ9aQ== srangana@redhat.com", "key_options": null, "keyfile": "/home/shyamsundarr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyamsundarr", "validate_certs": true} 2021-11-03T16:56:54.635 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC//ZCaEeVN/J09FtsvuZ4eEs4rTBtYoQyG+ZpuWkJLL90VyJeYrZzRsgvveKW2W9oFxCFcMy7IBhLes0TZ/X/4Ox9eUmP/eyqK8fBqT3K1F1W/uOmKLFhlQhkxxj+p9BOMBD5trfES/QJQ6UFSYIS2WhWaitDgONjoLMR6wXZ9r7yZeQBxbmG4wGI0Cwx4Wgg+I59UbE/5iUfnX++gN3rXrPnv9V0yFeciS4HvXRFXdrIzvQFHcZMmpOOmGruHXN/ME2dZCG+kaZhZdI6kICSlOwpU7TU1lv99zjBCeEzGfFkjv041ykKdNVR3qE0k5u0AOMqxJBORL/woSCdGvrBXqju2G/2ohl3B4zD6TAve+JLfWOV5C20NFRPPbTd53JaNpV9OQqwOfVniowzSYYBdedjm9sq6YwhJt8iBXze3OA8ZO0GPXo7PC2MyrrUtu6uu3DZqU4x0cl8iJxqOlx5pH0Gydj8nuUG0+X6wHZzPXZRe5M7/RzmvXch9koG6FQMiikNjUkK8Guz4NMBdAYUpxYkdPGBhiuELoaAcgIVzSGoZTuICpNYwAxBHk6q18G8Qd8vaeUdGdim4YvwycsSOwFE47wg/XmpCHrNJUZzzB9UD7Kn84TG82LAmCuAzpzcp0uPjsBckbXzj9Zt0hRgF80yDgUazsQkbgtd2p8KxIQ== loic", "key_options": null, "keyfile": "/home/dachary/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dachary", "validate_certs": true} 2021-11-03T16:56:54.637 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRZyOnj6D9tCw5o4+ujpos5Uj1O0XVvea798DsXPRh8bEE1pLkARQJLv4iW6LsG3x/CXruahmwT+6C/VHppIsbgtpadMWdCTPYrsjctPK0/N9nr4kQEL6XeK3/GHFnwqYcc+LySpNNfbKrgN4qWOwdZBhQjSV7K7FJd4ZYSm4wp6TVbvKKp7o+JdeM2zhstfC9AcCtBv/TpxixFFq/+y8/Z0PwqN1YyHduDOr1dr1DWn4ZzrkhiTYfPOltx0jweD6zTJoTAJDqae7kBA+3XH3dUGX3n7Tchsa3bnIKVz9sLQvH/vPro61I+Xig6MQh77JsylNvpwb1LLOEzDf7dxl bigwhite@wrjj37deMacBook-Pro-2.local", "key_options": null, "keyfile": "/home/shuzhenyi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shuzhenyi", "validate_certs": true} 2021-11-03T16:56:54.845 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'amctagga', 'ovpn': 'amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDwVh9KqjV0b+65aBxQV66IUwNo3D5zTbN8U6Kh6eE4QtaIVe5Mmvi2SBbGMSWofE0zIIw2JmsI5LzdOoCyZl/3gNDPHQd6D9o0xRHQv94giZ2a5K+mW79foAxsuppTjV/Fbg8EZ/19W91+BqMWotlMR66ka0RPT/6U8V5mzXoydSminViO/B6q7ZYzIIgGZqaTsSxCMvz5BAzzeEaGfSlGxw5MOye9sF/tDppuRKG4rMUZULEWGuxgXAtabtXCduocAArqAY3BRJEgj7JetPdPNjipGrULyYQmT7GeknvP/waC0sPm5o8t7taJca7Vu9qKKMKPAqiK4KhlrW/yVBFliT2CjAdVuYvGj87LCcEn6NMeF+4ONwDz++YwjrAWQT2zHUJPvLnYOZPiD1LUm3F34cEHD1y4Uf+oeiYgriUMauskfFn+ShbTmWWooFRFyNobzweg9aMBZSkA+kFFifqOV1FwHTOYg342NumRGB8svK0vvTUID7AU1Ev8YXYqCEGbb0UHth/1pEbAF103enLTtPZ6qKua74rE0qJ89OF9dZXv4JY9+8Juyps7bi2XEyHNm1nHZOY5tqdY0eKFmSGJMoWBDzGgdMjmXlWp4jnY2fC619nZqxMrCQpeT70CqnBhtBemUuu6jHYUbbrPjnXIkH3FZ7/qvZt0jF0IMP9FGw== amctagga@redhat.com", "key_options": null, "keyfile": "/home/amctagga/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amctagga", "validate_certs": true} 2021-11-03T16:56:54.856 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDEcI3X9mNxtHRSvi2gVFavu21kRCWDd1erXW3d/1zWnoUvRxe9ft2xA1c3Bmo067GqhFudPcXoYJvz3atAGR9k3n1UTtTJe95XRZ1Zmn/boi6mn7I2oRVFpW/KjxaiM3iN2SHKknMtcs6qRTioapYCU7u924oc0izqPBvkrC73cfVF4emH1ERWEcQBTxWub6N6Mn3xaDmJ5xnGlNvDY70CC9osoa2VRNJM7GU9eUQClHirPbSooDvvVKfTGD7YfMH76j6PdDR/a/cgNeiMevtMbeFGhu1SOEHb1ihCNoGq2vQFPPWwwTCzMI05XuUEc4iaxw/kbGBMpWbmIvswIMQrRwgwLZk67GGpf/dEeutI6WseeNKNuHnTzzx43AW9G2coGq0RKE7OuA5Cqj0CO7Kcf5UA4AqIIroLETVBrrEV6qzpkD74QBZXPl7gbSZIyxLpvtQoMfSSzN+Mj184LmQoURCqIT+yM5gddF5/S5I+rpw81vBj5M9ZskyzVQAo0H8= michelledipalma@mdipalma-mac", "key_options": null, "keyfile": "/home/mdipalma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mdipalma", "validate_certs": true} 2021-11-03T16:56:55.046 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'kevinz', 'ovpn': 'kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDtiHTDuQk8EDH5R+JfDMucI4ZpOzKN5SLD+j35QLLySgGXholoDxtqjHnmRjM1rpj5SIaXjHBZdJi7iax+1MRywHYBi5BfTtzYPgA+NH8mt+XkcdkEuFPcYxGh6fMQDQWb2g9Euze/6qi0W9mbYXvvb1/Q3bhVCXg1UgKD42BndWEpbMYFfzJLZWPtOD/lg81pwUmf/4BpPFv52sy87lQVfkkvcUliLObR4S0TI5hPLYxBS+RXmHYKgvPMB2QJ9wum62lJ+YM1MxPygS0OWbIB6UQGXmvxRXA4osXkxJ8ksPPCupOgFNiiYro91cQh3mmi6iHc9kMz5R1aNGGOfDdlnLx9DRSIiNcacjXEVNFaoM0x441f4GeU2ZR8/7uq2dfk7QjPlh1eJGxG9pzRiQ8JCDF1car0eYwXdMTsQoQqpIi5imqcELdcqGGp9FhzFElgUwpf15JYlkouVhS6hq4LDjhLNk7dDT8fmCEnHOPcbxaTgN7J3idnJK6zeIniuuk= root@iZj6ce071s2zz3reioxn93Z", "key_options": null, "keyfile": "/home/kevinz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kevinz", "validate_certs": true} 2021-11-03T16:56:55.078 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+tqgRGdlbb7NMefZVPCtbzVZBapxmmPcKaUtqT7E/9AKRoHuroZQ7KJQfO15POpIfEja7BugBt5F95aW2Bd9/xhulzC4/JjiZYc6ST5Na03BrcudI3YcktoGR8Mm7ZVzSojvm0SKtLLHM5Su17JpCC7BitMdao1mETlUOklEXc6pgbr9XHOayo1zeuos8DB90iA8TQ2soR/POD1Rix4GKniCHXz10+iXQFYPsif1HsQ3rh8Vi1M0cTq98bMSmfsKODil8KIxf+lvLCL4GAgDYEyxnstiTFh5GAcIMOOo2dl1fDb/Oa981CxWU+xxJD+M7AEJVeyAqqAnShvZU6y1L kiefer@kiefer-uchi", "key_options": null, "keyfile": "/home/kchang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchang", "validate_certs": true} 2021-11-03T16:56:55.255 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'liu-chunmei', 'ovpn': 'liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1YAaxAvlRgu72huAggHpoDKSgtF/b00ACfUX4Wk5NyLFbYI3PAQapHNcNL4mIsH8pc6YkdjcJ5N/F12aEupJGQ0Q06OS324xapnaBoUy1hQl9IvhtPqTLEwR0X5xLIAhNBLIEJDNVg13MKwr0u5YVUXdrQRu7Kj9RXAPBahNZFo/Ed+cnR3LICQWliHeHFrk/DU2DUHol3a5wAPxrowAG0AxAecP6/7rASMmitvLQwdQcfIJAhWYgfyXzudg+gOmAEgv3O3U4QOikgmkm/EabUXLJnlTj6yuhtuuctdJr73UT3wLGAGmphLKCkug27kb1nTNoDa+PW/+grP63zhU1 chunmei@otccldstore05", "key_options": null, "keyfile": "/home/liu-chunmei/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "liu-chunmei", "validate_certs": true} 2021-11-03T16:56:55.287 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDjj3bbbLvd4IHeFrSJyM9N6uKXA7ElQrzfcisKRGjUPON8IsAWkXQeAnHoBsIQv7tRGudXOrK+tbfv7crLI0/CxL+o0cCVW8bxuTEtdz07ZFsND9cppCc7Yq+wHsg7nQ/n0QfxnlYc0T9pATXUoT+aW4c3NQoDbsupLSkNs1C+G/MEBzz6iH6tgrv43OAOUccfQbC+IYx4RIKL5MteEjVrZanNELIG+jsYe+CZLdsONk1nR8+xKQw19Sm/jkKdl1K9NXoVPEBQmFXrYl6G+7tCdWeU3Vq24lM1X5p1mVspcG6FyyctcuP/8+zhL2L208DrXlOVPcqHR/4FEf3Wmk89 vereddy@vereddy.remote.csb", "key_options": null, "keyfile": "/home/vereddy/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vereddy", "validate_certs": true} 2021-11-03T16:56:55.466 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'cyx1231st', 'ovpn': 'cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDLbPdDb+XHOD2fo8g3RacATj0cG07uELwUvqQP35y+JmhNRSb/ttaysFLA5ss0y77TiXx22saP3ECEm5SnuPg5IERkPQmXiw84ATTNfLnzx2G5H223SLaTQiGfSgniBCJHVPCJfNahJiurlZoOAlwDz6SKXTUiivVrFJ22iTBDCQUO6HFQKjE7aM3RAo5mZXKVlTXzM6WOd8vlzk3HMzV0/91JR6H86Lt9MEzXT4dOzqmtxd8gS6U+vA+50zOIspdK+orTRJ0PO2Fn8c1RSYM8g7uqQtpDcOEHderW8TRsqBbDxpyslK6hRX5urUNBX506XE6JNoViLNvCofgseCGj yingxin@yingxin-server", "key_options": null, "keyfile": "/home/cyx1231st/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cyx1231st", "validate_certs": true} 2021-11-03T16:56:55.509 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC+EkUbnaeC7Dqtq2N4/dX9v+WGlrgRS2ygHhlbZy4KHbd6+Q9T5NAWtqnFwqcbFpYYUBwMvz8zLo6HYN6gNskRN0qTiFKPQdtnydF0Tx1KHrdGGcX3G2lxyGH5P8nrqReXAPPvRiVc2/Mj7APCTmY2JftkSS1qYcTleeueivVyho6z6uD1AyC7OXANhtn7kGeIY/NvVKBhDr+3TuPmpPwPCQMcJUyZWCHYyp47v2vzNBy2Etp7WjGwOnfZ5ULiJGjH6PCXx0YfpmTUOPVkNbV6Om+0YJ3fcLlmQ/2nsdsWfOKvoAr+1uzO81h7ZBX9LVEitj8YU3WeSCV2vqS2VvOpq5NyHXWGq+ZkOCfhy4Z9MvMG+6hqp4p4z1JMpKsZ191kP7eAQua41GkTSQPlEfF+um1UV6ZjNzhB68utOXyGDoDcxct9oOdnhhjwu5W4/0t9iBHuMn02iHRYVrUWOH2FXVysR0p5x/L2xdvNPiTgFAZ2eN+EFWmdan4e9mAAw3M= gbenhano@localhost.localdomain", "key_options": null, "keyfile": "/home/benhanokh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "benhanokh", "validate_certs": true} 2021-11-03T16:56:55.678 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'sunnagar', 'ovpn': 'sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdqw+8an6CHqsagFqXfuzQi1zCFHCh0e9pDtYz+ylA7L+7vnJLjTR7km0bb4Z6VTUh7h0/nUfF6faPzD2KGmTKvS+mTpeOUaN/H8O8j8qqh90f8z4T8F8L3DwqQbPf/R0izXmTgz0FZObFHmj+G8a+D4ndxF80EPzKkML/G0Hg2GEVdYzazSY4imSY2f56ul6qByBknhFshaTo2DUDcdbDDokF6lO6WgXhY0MJrT24CUSMyZmyi+3hF7vowW2dfbJ+SuDft4XmPDnIxRhDWWvi12jZb8Le7SWYzUlu7fAGbWe63VJqNh992myI61aZshApDD24+1xqIoo+Z/XkQD3l sunnagar@localhost.localdomain", "key_options": null, "keyfile": "/home/sunnagar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunnagar", "validate_certs": true} 2021-11-03T16:56:55.722 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBYtdHKBXQcXQFkEW8fSwlFK/xZM16VcFY288ABI/DPfejTStHhxiRvmlsWzMy0Ar5WadZ8FU1ptjA3MKBEY1Do1r1Ul369k3zHCvxXyg9QGGMNPYfHmnNeoF4GEgoJpuounPs0mkLmrPTR+cR+/XuaZZt38nsad8M/zJetnolvqCWpcV87D7+Rt8F32J6kGDknC4ikEe8I0n7F+goOmJhlgkqegbKEeJ/EMnOItwVZHFwaC+sxOq4IlYp2f1aZMKGAtr5+ICwYH1SMCcvnp3qWL8O//t2tvguZgQxzIs6Nf5GaxwdQkh4CvjbWiCkIbGFJ57YICS6k++KMXMbEnwh jsalomon@Josh-laptop.tlv.redhat.com", "key_options": null, "keyfile": "/home/jsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsalomon", "validate_certs": true} 2021-11-03T16:56:55.893 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'lflores', 'ovpn': 'lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDTlsh9TFwLCeeD2RR+56VhRZ0hp9CHj72U3Z1gbMpxpcg6PYEDzb+0qPVQp1/cIhcwwy0sm13yFl/A5i0iCRpyqB2MbyaXjj0HgkNinxLBFkPSiW2C83lfAyAhG/gLk+FcmWE4+xVrE29aoSM0JtcNtp+q83UGdnFcvhFe0pgJbt5vITo3ZMJND/UZrzeewyK8Vy4Rvpfy8aHYi2Eq+7WQrRHwUwxn0CwsxldnUfmZtu58d+Rtt78LRF0mvcScNlC7q5D+8htmYwkWkK639w5IB3xPv+Z5cV49Psi24Q+9BIr9Zo9/OnnCTauAi5ivDlh+kk/iBksAE8rQpMhvfuSFnNoq2O87aPQvNQHdRCcOa7FVIDcuocu6AB8UP4cGcJreT4+BW5Zbk6vQeJZhmj9yExpRj9MuONy5793nKuAMPn9lOjuXRfK7A5Tb3OOhpEJK1/MjtdZz4mNmFFfFIZ24hgTZO0pOiMwDf/tQ6BG7H8NS9g3QmXPyZajUocw6Szs= lflores@redhat.com", "key_options": null, "keyfile": "/home/lflores/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lflores", "validate_certs": true} 2021-11-03T16:56:55.936 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9ltUNitnn0FNwMeiGkBnozpBN5ziVR2MreaqkfqTQY2TgI24559xuILyy+TnAKumhemZ8i8nIQ876Iso85OzIXbUbRM45hpL/bMQX7pWyMIRNeFKrCDwf18OT0DPc8aeDOAFQ6YoN0j4awUENjdZuuAW80cfcjoQO3m7ORF8wBF0IvMWwPDIQQrz+Li31ECgXg6VC7K3e+BNHlfNQ/2/ahS0+V4J6JYFsylZL0r2tydhDMd6rR8QA9c9BgeBz7+KWobGtiZVe7KReTzJ0O38PxNd8br3aS50j5BiN63gch8eFxPAROaUMnuiavu6OTDy4/aeL3Zc5WpUy3VyyumWf jolmomar@localhost.localdomain", "key_options": null, "keyfile": "/home/jolmomar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jolmomar", "validate_certs": true} 2021-11-03T16:56:56.101 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'jsawaya', 'ovpn': 'jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGq2g8OFN5msjIn5CNB2BRss9eIyJRk3r+7RQcnBOQid jsawaya@redhat.com", "key_options": null, "keyfile": "/home/jsawaya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsawaya", "validate_certs": true} 2021-11-03T16:56:56.155 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoefxOfeArToGucfWG7y1tyLF2I1J/IkBSlmxxEJyHyDUB6ElXjYIJReoiPfmB0xLEmINpmt8QA4XN/8UbR1r9LFOexlZZbX1LSbihi11LXCvfIJWqkHVwIyKHRTu/duMrntBFVENJ0LVf0GSWVTtllXaDA+fMkp6/ra/hlRHQbx19UXy/9+Pogj8Xl+SntnMGiHlX8FFq2A2tUqhdQSf/1GW1HfUIa5WtwdEqIpKpAgMM7sOW9i3QAmLAMo8HiKrArSiy7jrRUODCqsTJsfifxEbmnOS0ZtmDh0eeWL/Dyqbz99Rg4WbFNeK8oyZYr7D6bEAhd2az2A+wycPx4Nq7 julpark@julpark.remote.csb", "key_options": null, "keyfile": "/home/julpark/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "julpark", "validate_certs": true} 2021-11-03T16:56:56.324 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'markw8973240', 'ovpn': 'shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNIdsy+yA0QjwkvjzTiH4lKB+txlh4oGkgC0r1l9swmFvD+MuzNNkD+oB/YscT51tJXsP5/wZDZMuk60xds4Os2ejoIBvzdGBgZ1G2B33etHPmzgTFE7ZegBkMlvWYmrJ9YS+iaMvXn0MMtqfE4zq0kemVAA4NBfFZMhHPJMKCg7Gj31MR0S0b7dcPx7Jnpg4A45oHr0qnufeqZYQaHGgo+LO2xdoV4V2EswWr7YG+b1wI6DzFZBbNsGyQd+cE1BzQ8NmETPNNpDPk5dCyBS7U+4ZTf/WbWk3+Cn+Ky47iWfxcscupQRkCjmTYI/uCMhnrpGU0zEHkbFwLTS6wqDtDmVHm++xof3xSsGJ2cE/vZySZGHqdxqVXfT8gt17h35rME8eX3TU392rjfK6kjEUD3UgwJJp4NFzQIEXi2HTmDYx/Vi9nuId3V1NrDz98QLPUd1ijiy1DuzI0BJPE/DTDmsb7hNXNIcAV8VRc1243W3eaee/Pdg/wmwfkrgjTU3s= shuaiwang@Shuais-MacBook-Pro.local", "key_options": null, "keyfile": "/home/markw8973240/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "markw8973240", "validate_certs": true} 2021-11-03T16:56:56.378 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdhange", "ovpn": "pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQD03otezqmbR7v9dq62mWH5Kkg692jdmDZk6Ve4vux0A0YhxBuXDCyBWLE1Z4lLNxQahvK0pGsgokp+Uk+Q6aU+ZnF/E2jLp6YWerFXIHY3M7byObJx9+QuVBfNI0bZBkV3+vSjhSCGOYU+TrTHsKZqhsTcejZjmL3gFLmWfU6dTh0sX05qz97cdJl3CMPhCIR3KjtkiPygj417/N7wet/Efv1t79TvyS7dtK1oz+zRLEpKi/fZjvpXb8xec7as72Y0XexsWMDyZJ/N8PijTQ5L6EfLMV7ae/bWGzNAKsoC1dHVY7q/Q4pzktC6zU0Rc9sDRMtrgFg+q+04BfveSLfxTb9tqf7SbSvh7IAG1eKiEuDT/uy3T6Uiw+CiZ0eIqYzZscO/aDjUArGbS8eyaycybdYCBK9Jz38i0mYKkXg855yoVuIExr6MXieYwyVoVW8gbx1bAslaafrzra2dkAFByOJ0sbosPFRgR2+boUSxD6IaE+ALgno4T/0CI2RMiH0= prashant@cephdev\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC4m0LAcC7XiFiXFtYcCvZqX1CtyV9JU+VUEawZJpWrGTOr8c+vIexq7+WKg+eIUMot4+dtfwG8lByS1VofglIe5/quv17Xj7dO/hF2WkV8O7RYt7NhzmrKg54yB73pbRpvqzGceWWDvGRsWQGzbY7ARqOp4FPIDJvgXABCkak+hmjXyXpvXN418CBDbDmxHWwFb9QtB/MpQFiFrfl1EMDj8DaXZNWgo5J/qytic6LgiyPHhHfhSylk4cl9TGftNatobpMjqBT0O2/plk/qf+7pHPuvGSgihjijdpEPzVfctXE2OmsQAIZgkzXeCqts+s6ZYdUKq1FvpV5rMM8ZLqfCkDjEzFENJS68YYE2EjxAEgMzj7h/sWc4B5I5pxkAqVAQSGjYRu6rCCbFz8PXUdTyW+Vs1YeylnlBMiCa4xE4gwvW3HlWEnumtyURd1PRENrGJsJ/0uX6bpltbSg4sGACRYpLPJMm3OhqI3Jm/J/e2NLjfCw+svU2447xnfJOh68= pdhange@vaquita\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCoeyVaMjhcGrWQB+4Rq+SSIi8+OY/kx2s1vofiGt1gu3RNtFmG0Df1VITJJ32dbzaW7kTNxaRvhldNd/n7yEMssG+Wg+a5WKvSudlbVMmscbSXdFuVbcCW1jNrTGCYrxL7Sn5e7NQEQfAsCbfXwTcYBw33NdBVovC+gs2V3sZGvX3Y0akZONNuVg1mhu0L4m1/5IegJh005KeilwetHA2A5Dr10RG6oyIgT7t4z5gZXsrOiG3UQBXaekILYV9jxMWoFXGkVLc8TpL9SUAdxE3Owc0u1XiPqYYlEN+zDvE0554Zj8Oc2ZO6HRXCZXQFvzXeCD12YswcOfkIDmJRF7GT prashant@mauli", "key_options": null, "k 2021-11-03T16:56:56.380 INFO:teuthology.task.ansible.out:eyfile": "/home/pdhange/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdhange", "validate_certs": true} 2021-11-03T16:56:56.545 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'luisrico', 'ovpn': 'lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "luisrico", "ovpn": "lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+ZvZuogU+WfyGj4BR2xYllNoQqAoxQ6AM0IYddBV2pL1xgTM/SopuAxDRP0q2stpozMtjVjlvuiIgr/caqyTuOgYtYCvB09ppefqdnCDrJJBCOKpjUEnCpAKcBMMuz9V32Zqt5FgpIi0Mu7Gp/QCvB97imxEkoUunpdX/8Ri/xz0nJO+QqoJX5wOn91jG49XnkMCLzx+MuafhT90WFO9CV1sIVREZS+X2C7ACQuhx9Qwk7R4BXt2f3M4dA0K2nLwaAnVW50M1y6Gs50Wl+ojDR1itySOxv8YpwyRlRTOzQTyVMnG0Qsaxxwp+hy4b/vQW3NIUvHYdTyvqALTQdGHT lricomor@lricomor-mac", "key_options": null, "keyfile": "/home/luisrico/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "luisrico", "validate_certs": true} 2021-11-03T16:56:56.597 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDg7xW5+Mk7pG5MjLxOc7GTvDTCPBJj2L3MEQZIapmmo6t8uPODwDewXncSzrKDH6zFVE4TTTD9fJdu+BhyT+jCFgXrEOMMabj7vJsg0wO0SWqr+4NECFPhD9y4ewfYiewBeXisclA5Bto91v93AtYpZBCdmPYSi0BEcPVc6S3iBUD0PzIrL62I/kBAtm0f/bScRiT7XvQJXvkbEObW2yS2XhUv2qgq1VlFvO1jl3B85KMTHVb8+LrKpQWOKfRfxJ74c51y6tnEjZLFO6Xn0JevqOhd8gPQIXueFZOM0g0pNp1mbtGaaBlEIibbrzo2p9kzsdvJeHMq1XuMIi1/Qyj9 dpivonka@dhcp-41-144.bos.redhat.com", "key_options": null, "keyfile": "/home/dpivonka/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dpivonka", "validate_certs": true} 2021-11-03T16:56:56.773 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item={'name': 'matan', 'ovpn': 'matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "matan", "ovpn": "matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKjxTgyhL1yFriyNvvuhUc6SE8+6f6lT6FTF8cZL9uiz matan.brz@gmail.com\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIM8l7qvSQR7fnZdQ1yB1F1jgde0OmLcwcDFzc/o2dYBb matan.brz@gmail.com", "key_options": null, "keyfile": "/home/matan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "matan", "validate_certs": true} 2021-11-03T16:56:56.806 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC0m+8wna4RnrOcuCUj971EBe4cFiTXVkEWkIsUX3yeCUwMaS/ZjaoOjcdYX7cYpCp1ilo4FSO5BsochU0RiMbQEwLGcE0zB2q1ac8+yYKCuWWyknig2t4bOReu5OrXvl6mj/9R+p7clFlYhwIoNG5ZZ/2eTbo3nH7gWgaOKaPbTtvEGg/5R/qjQKg8OTC8oix6kLsaNTisQJ78Zd7b4ukg9bO0jlq/2UhQOBM072sU1/kJpWTkcA0xxm3nsC4rWPwapgz7UwBmMdyx116GZgWdfQecTLKO7b8pyJw21npk2ntUmeMi7WPZGX6VI3IpaAhBjW3KiATAb8JbAQh8F9bmejkw6x7zrP3hYKQOg3A+U8DKRRo/Ad+f0zz9gjC42DUvMEOr09RBdxZcEAPY7H/YRWEqbA2zPX3Gfth4b7/letzM3m9ToUvVfSt1CdjaZD+gIia9pCgDMb+j1j2w9reG4F7Us+W/1f5pYCyFyrm4XO3AdoYM9/FOAOR3vffTIQU= adking@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJOefD/VJfS15LRyuVDLluKv93Zgocsl8BhiDMimvBNbrGWc795v2+eevWuwaL/SlL44SnsNZ9Mz8FVn9YuFoQiHLg7cHZNLg9EJh5SMXzZzwLZHnF02MdI2CU6U++9nmh8FtBrdBWKzZ7o+01wOw+52dYOxvP7IJ7C/vjwOLgfigEN8Nd1PrxALVNIQz7fsiet0n3odk9PJfoqTu3IloXm9qbQiPFyfMA6b8a20RdLZCJDGGG5rd6gtsyeFHoAPZEz8FEUlNkow7NM7ZhdxOz/fWXyMZ27KmY8WSAgflbxysMzexyifkGzypiE3NQ10hdPGBgoYsHrAN4Qmte1/Z4aI5nxrNzLy/uNc+L/Bp/0XkLj9zfoiFFOReXvlz/laAV6LpbHyacr+iCwv9rlI7echXmvC6mSJXfBPuyR1uromUXpYRhEkn5uiOsCIQCuYpx2jeil7JV+LJlF62lLSndOv4g12Gq2TEZ4Bhsr2902kftPkLFenyfDTh3A2qGGNM= adking@localhost.localdomain", "key_options": null, "keyfile": "/home/adking/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adking", "validate_certs": true} 2021-11-03T16:56:56.983 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'ozeneva', 'ovpn': 'ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCkFTEUxQkQ1cplOLOf86Teu4qTYGkMqWtjIwzsU/AcimjFolSI5fJ3B72PH7bhQKkSnOlTjiOos2LrD5i/XgxkJD6icVWwHO11lczsq/8K5x0ZMJbJaiT3QphBB4yItoge2Y4/7g4o/78Ne+U7eZ1L5fgeO1fExtIaEXK1lS8pVlpZQXyproTIzp1WLHoaXb38Mxc5jAtuvEKfEycdX3c8k/9ti2qBCZnmjPH05WOxkqCRyNBT+bSJZyFMYt9RDUuycn3BYVSXTAzPW0qwgvKY5vij+qiFoxij7nKNMH7w6C0arolCcJxhiMyEF3UbTuOxgVONWyqFk/vScGGSYj9ytQYjv0L1cFyOVhlpbu+tnptulfQ3faMpjROE/eYxTIoY7OPDgWbFqEUJ4eEhKwAC7aYIYUhvoI/8k+4u64nsyEJQdHXmB+OY3WE5TegYUCuqSsdPY74eO1o8dDgxF363eggDSBmYqSSs+Tpu9TNhJG7iJFX1tiAPwmKvjVrJOac= ozeneva@omri", "key_options": null, "keyfile": "/home/ozeneva/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ozeneva", "validate_certs": true} 2021-11-03T16:56:57.015 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCs6q2b5NcUsNXl+w1Mrc7idNvGdwWkL0XRX5zTw22UNpQyqPkSrRTFKi4NxoY8ucv8JoLnaJL7hYSRHKTh7SqH78uPLZX0FLtoHvpYLgUgJzzJ5AOW5uekIhCMCm+3egmFnEBwFcNxtGnU2es02ByXNEXRxt4/LQ0f8K6lMqxVRSKYcltvWzlA7WFFdTnYMunbCYOdpw88RXm2pYjGIHRBnie7+pFiFSS6K/mkzQcfLiJmahgLSaYslI0ho1vQd8xfNLmZ700C5F4VTUlT/mJK/dI2LF3OfaPTsyqKNzOchrjLFx5b6dhZ4WZyqu7VKsHNm+xHZwT8VP3LPGSVVBdmEsiCsmV4g4QTI5ssZLi7siZDpaFp72nVAIO1i6kBGuiF3or+J4LmIh8ZRKzTc0DRDOw7S+IkWu1fv2UhYg83z6wrz4HeczI+bnolFFWW6HgGV3MFaUeRTRvrbxw2DPoEGQ+cjP0wsjyEhxlScwdh8W8IbVpNl+7mBe45YGUH2hg83Lst5QQxD4PHelriu40moWRHFKCnKfNesb4L1uqu29o8f+/PgpLdrWE/P5zIKe2ReWJyScYAI7jTMlFw+xgf+0TKryk44NMy90l3ei0umlx4qsFbbEv+lhzAALrUJ/da7fsQw7eIVykmcscBeYr99/YSYk4MGdk4jVCiHsQkdw== amathuri@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDJ3ch+pU88/tX5TVVFl/1pPief8drYLRkK4BtfJk+xVNxX9qkDtOtiLAU+OugYF1/xk9hE9zTbm3UBL23unrzhlR80t8f2FiF1YFhHVzeZ/p/OunmkU0RFQzkMVuwKVOUNulw9FwpEZ/0Bjfxvaims0SBWaCzBgE5g0R+Ofdq/bVF82K8vU5Laa3KZ/cCdGzFgW00cBvPAKIpQ93g8TRqsCsFvR37nY9v5SU1hJ9araEp6LRA86tUt3EFtxD2tRcmqKGtCb0kOr+rRXWVGiaUBiedHxFudIy0+TNj4fNrJwpoC0hcBO9ek4Bh6hM7Zqzum9HpkPGlz84jkm+r+xMThaZCh8iFkgxyK+bw8B8smaVJD04ob9Wb4RYoFYfQ7TEkPkO9gr+UsSesbP9fWfucemUqwK0htOOtpc1YRwdOPcdPJWAfx9J5+YVL8BTWYBaTdmHslOKeUGM5fCob6QxC6Er9xweys9jh7MXttSpe013bY7P4ZDRF8uWvygpsytPE= amathuri@redhat.com", "key_ 2021-11-03T16:56:57.017 INFO:teuthology.task.ansible.out:options": null, "keyfile": "/home/amathuri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amathuri", "validate_certs": true} 2021-11-03T16:56:57.206 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'sudhanaruna', 'ovpn': 'ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDepyzA+MeQT5oRM52SxmUUKVAzkcrH/NqFxoCpBAjUSioTMaB1h0PhHE00nvHnKRvqWdPl8LCBWS3Ncjh30r9lwHaP+oPy4iLQEc7D2Mwn7WyPh5cq9N3UF25C0AlMG744k1tBNb97qy73OJI+oc5pAj4vUpCC1Ut9pMP5E5pQCx/kwgSOpC7mk0dYr/H5FxjPb0ca55eisFTLXraHPSChdrHjlYC5XpOZ9csilTwFdF0fJdu0nMDwIxGvwTOaUcIcfNLkpGB2Sj/pAM4L9pVsIbeE3C8d978MZ8Ff+ksQ0T8eNzew2ZjSkSuV+oR1sBCJYZ7mbDQuYLKAs5DanO1VFJvgjBJm1zP2ONu+g0fn/j04qIosuudziwoG/dOTkFo+6dBvF5hQi7TbZOIUVFr5dBlGS+O4eoK0gi/wpzun227E4dadY9uB1wiZ/PTD/5og+gxWG32RkHqqX3ALlK4m8JKzrq214nZ7Dt9CNewiUkR15uRkDasGIdiu6qb4Ez8= ArunaSudhan@sudhan-linux", "key_options": null, "keyfile": "/home/sudhanaruna/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sudhanaruna", "validate_certs": true} 2021-11-03T16:56:57.239 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a28ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a28ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCt4sQTrDDg562IWY9Bw2fxFR7Y7b5lfo9egHa1qNFR5K4gt8lUwQXgzcxZK5dgSTZXHDGxrbYKWsdb8NbpgU/YjSqDUIptMZrhTI6pdHKZ2usY+nieaJj0NIZP/S7M93BbJeoNBYACjvDUr+vXEg6ZL3uAj+zFXr/4te+p4NqJBl4bSpvjcRK12O+F6HE3Q93j0Vlfs8IbXP4vcPcqOW+2usXw7JPdfjwuACJxrLTx/0mBsv8qXJDfOpPIHAMA/9SdTAzhs6XMo+fOwVOKkV+ki2fO3sl8pE16aFOkI3cYX36nkYYpZ2bJLnIPcLv37i3Nypl8eNLA4gzjkDf1o/KLO5vXOs76H189WGLqjh0SHAx/3KO6mrIhTq8Bv6YWoT3ntg22fXaxeajNhHcQ0bKZtGOAb0W1FcMRbSTkgEFAassYxlpTtuaCDIuA4fUVKW97Zu62eDDsSUtTWak/Yf940qAbptPPAw4ljmrvZRcjM9y6mGU8lKi5H4QoEOfbPok= nia@fedora", "key_options": null, "keyfile": "/home/nizamial09/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nizamial09", "validate_certs": true} 2021-11-03T16:56:57.409 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'zl31wang', 'ovpn': 'zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zl31wang", "ovpn": "zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDuHIpWAz6Sk3ATLvTG4KS24JUzgqQBEm4EesPgf1cEG+xK4gISUixaVC9OSNS/KR+8/yoWAhmw992p8cp7mWXxqrRC9kIrthLWvD6Jjz3HTXgarO2oNibeVu27kK/Fm2vvUNACARBkoC5YXd6IuoZOEg1bJu06DLw5Y7pGMqZRVDOrE8WUDwr27roZSgU2/N+18Y1LYn4gwSvWpuC/IotLyGSpW7/dgrZAIreRoYboMdTSlIjjuzeWvYx/AnZmkeEo0IhfyuBRl7ZQwxSFYHn9Z1+iQkt+O9hEZYSZuYRuyFPRqHNMKCEj1iYVRMcwbf/x2/dhjfmbWmjOMJoNNwm+DpixHHbId6Idx82GlAtCgQSTe9aG783oCeZYuVUr1plIRjeEsdjY9UBRYKNknEAChKjMSbxysvO3Dy4yrU2r8qo1iBS1fi4HUg12asCLrJX7vSCvOhOn7ltqDTmt5UiLSn+eCzXpOTBDWz7McB28FoYyR+Rwz5GOH3ITFhPyhXc= wzl@manjaro-wzl", "key_options": null, "keyfile": "/home/zl31wang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zl31wang", "validate_certs": true} 2021-11-03T16:56:57.451 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC//ZCaEeVN/J09FtsvuZ4eEs4rTBtYoQyG+ZpuWkJLL90VyJeYrZzRsgvveKW2W9oFxCFcMy7IBhLes0TZ/X/4Ox9eUmP/eyqK8fBqT3K1F1W/uOmKLFhlQhkxxj+p9BOMBD5trfES/QJQ6UFSYIS2WhWaitDgONjoLMR6wXZ9r7yZeQBxbmG4wGI0Cwx4Wgg+I59UbE/5iUfnX++gN3rXrPnv9V0yFeciS4HvXRFXdrIzvQFHcZMmpOOmGruHXN/ME2dZCG+kaZhZdI6kICSlOwpU7TU1lv99zjBCeEzGfFkjv041ykKdNVR3qE0k5u0AOMqxJBORL/woSCdGvrBXqju2G/2ohl3B4zD6TAve+JLfWOV5C20NFRPPbTd53JaNpV9OQqwOfVniowzSYYBdedjm9sq6YwhJt8iBXze3OA8ZO0GPXo7PC2MyrrUtu6uu3DZqU4x0cl8iJxqOlx5pH0Gydj8nuUG0+X6wHZzPXZRe5M7/RzmvXch9koG6FQMiikNjUkK8Guz4NMBdAYUpxYkdPGBhiuELoaAcgIVzSGoZTuICpNYwAxBHk6q18G8Qd8vaeUdGdim4YvwycsSOwFE47wg/XmpCHrNJUZzzB9UD7Kn84TG82LAmCuAzpzcp0uPjsBckbXzj9Zt0hRgF80yDgUazsQkbgtd2p8KxIQ== loic", "key_options": null, "keyfile": "/home/dachary/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dachary", "validate_certs": true} 2021-11-03T16:56:57.641 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'mayagilad', 'ovpn': 'mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIi8JMOVD91QR7EXBkhxeAsB2UO2gyWqYPiP6HjJO/TzzMKf7QtO928XzdORwQ1/jm2hDF1kaNPaU/XIHD0hHSvfICS4ELQcBPa5uPya6d5Kn5fVlpGqBcXBxKGbHB01PGjn0214eupt2LebWxkm7TP/BGDHKXXaoR748AlbyyG2L3TaNoGkwQiEoZe3CWgH4qkrf2CEpyfOa7BM7U0h+lceJxIGe+Yo6Ll14zyngDBa059E+ndFTpkQ6vECII7FdB1+A4Orhu40XHZgyVnlRudD1l80pIVnZves8N2zHZhQo8hp0sB5oY8HENiB09j7Xa6YQ/c1c2wm9E2xJHSNo7muFiHz48vQEgvNmu0lZdbVwXIy3IgLEBHjs6Kyqjtg618V4UaoP/LBOqtWtLd6TvGJ3qBlNyvhO5gZ2DGeLMN2WWoHxfl3VM+QGCGI2dnzUyHhfG1VBDYj8fzWXOuUVpVCMUw8fEg8GWmYLDwm3SbdMACWU87J0kAmaDPKcwwL6AwtWJZr/p89sSypl6y50JlOoDrbDbxrIbHQgbf1tcEgPOuLUN+O06qpi+OxKMtFdbvz+NQ4V4CpXpvMMQZj5AbuHp0EJ5Vme/Kcmpl/xoYGyFDzOEGR8lw6LMz+CpQ4GtnK5XXe1Xn3TJh6Tn2tVyS20p7artemGQQquCN8iQOw== mayag@gett.com", "key_options": null, "keyfile": "/home/mayagilad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mayagilad", "validate_certs": true} 2021-11-03T16:56:57.674 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'amctagga', 'ovpn': 'amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDwVh9KqjV0b+65aBxQV66IUwNo3D5zTbN8U6Kh6eE4QtaIVe5Mmvi2SBbGMSWofE0zIIw2JmsI5LzdOoCyZl/3gNDPHQd6D9o0xRHQv94giZ2a5K+mW79foAxsuppTjV/Fbg8EZ/19W91+BqMWotlMR66ka0RPT/6U8V5mzXoydSminViO/B6q7ZYzIIgGZqaTsSxCMvz5BAzzeEaGfSlGxw5MOye9sF/tDppuRKG4rMUZULEWGuxgXAtabtXCduocAArqAY3BRJEgj7JetPdPNjipGrULyYQmT7GeknvP/waC0sPm5o8t7taJca7Vu9qKKMKPAqiK4KhlrW/yVBFliT2CjAdVuYvGj87LCcEn6NMeF+4ONwDz++YwjrAWQT2zHUJPvLnYOZPiD1LUm3F34cEHD1y4Uf+oeiYgriUMauskfFn+ShbTmWWooFRFyNobzweg9aMBZSkA+kFFifqOV1FwHTOYg342NumRGB8svK0vvTUID7AU1Ev8YXYqCEGbb0UHth/1pEbAF103enLTtPZ6qKua74rE0qJ89OF9dZXv4JY9+8Juyps7bi2XEyHNm1nHZOY5tqdY0eKFmSGJMoWBDzGgdMjmXlWp4jnY2fC619nZqxMrCQpeT70CqnBhtBemUuu6jHYUbbrPjnXIkH3FZ7/qvZt0jF0IMP9FGw== amctagga@redhat.com", "key_options": null, "keyfile": "/home/amctagga/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amctagga", "validate_certs": true} 2021-11-03T16:56:57.852 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'arjsharm', 'ovpn': 'arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "arjsharm", "ovpn": "arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFG1CoPZEh4KBANdJZIzLsutHsF6tAYZTdDdEW+5E+8YyNHOy42D8iBLg+eCAbIl4SFmJTsPNtyO4L8/Ut79tkG3usA9nuGoAwGYuctZUTgOiTiV61940Y+NTvwan9vFNoknVi/3qIGNSXdm90I1aWRTKLCXRbGA1qvu+qyJv93O8LFaPni2TJmtAebY/xf0fHhW2fURUoeoCsoobrTkPA8SojT0A3SM+31iOD7sxbmsqLBR790wz/FHh152PiqpupIijS9TaZgGuVpejM3rgAC7H2kVmKJlrmDlmfcbKLS6BBi+uUDi6ta8whnDRq03nP/4LJE7ZDIxX3I4mHztZeJggURlsfpJdo99+VHkeWlGDBIP41QgD4UX1neuMcIO3nuEnZmLeJyvo9JOR/I7AudbQCDPjSMNbrF+el70mWLhvPe33NA3x6eOBRPc3r+qrSwMhXEPatZlbywJRY+O6yC2TYqdlUKsUEVA/D0Iw7tO+pf3yH3Y/iUDJJLvViRZs= Arjun@localhost.localdomain", "key_options": null, "keyfile": "/home/arjsharm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "arjsharm", "validate_certs": true} 2021-11-03T16:56:57.895 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'kevinz', 'ovpn': 'kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDtiHTDuQk8EDH5R+JfDMucI4ZpOzKN5SLD+j35QLLySgGXholoDxtqjHnmRjM1rpj5SIaXjHBZdJi7iax+1MRywHYBi5BfTtzYPgA+NH8mt+XkcdkEuFPcYxGh6fMQDQWb2g9Euze/6qi0W9mbYXvvb1/Q3bhVCXg1UgKD42BndWEpbMYFfzJLZWPtOD/lg81pwUmf/4BpPFv52sy87lQVfkkvcUliLObR4S0TI5hPLYxBS+RXmHYKgvPMB2QJ9wum62lJ+YM1MxPygS0OWbIB6UQGXmvxRXA4osXkxJ8ksPPCupOgFNiiYro91cQh3mmi6iHc9kMz5R1aNGGOfDdlnLx9DRSIiNcacjXEVNFaoM0x441f4GeU2ZR8/7uq2dfk7QjPlh1eJGxG9pzRiQ8JCDF1car0eYwXdMTsQoQqpIi5imqcELdcqGGp9FhzFElgUwpf15JYlkouVhS6hq4LDjhLNk7dDT8fmCEnHOPcbxaTgN7J3idnJK6zeIniuuk= root@iZj6ce071s2zz3reioxn93Z", "key_options": null, "keyfile": "/home/kevinz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kevinz", "validate_certs": true} 2021-11-03T16:56:58.071 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'melissali', 'ovpn': 'melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "melissali", "ovpn": "melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCXZDAUNEK4+/6Y+ywEtFnabnfvZmzzHhO6y4PbRHgx8SoQ8+7au/5ElB0vjjiJJSChnDymdqP0UzwcShzhWA8is6K9c6tXUHg6NcgV5tN2xYYmGIIWwm66Z1kYH6u1x2Jkv9LO04rXgebkzXPPYK401agCWjkjUv5acZE12+M/B6d4h2lpFjzBqZ/mmRNp0tHdCapdG/nS1i6GRi3Yi3fQ89CbiZArG9268DSEnLe+npeyBMVdDkWpVIytQ6pU1+g0prrePIVaMvrK3yEooduD4HeLr4liqkINj6rCs90xedPcFi0++UaLFRcgWh3OOnrI952Oe19dx5hlbmcnMPGjaXnkng2keWFyZMOQRRti5l8C/TXPUTaYGWEoqFEEabIKnCura0NWuZHpeF9xEK0tnvNVeauuUfkPxWQ7PH8QQ/9XjOkKbmy0+2q9tug8RC5i5NwcRE4W3ijM7pLwhWMdxkmIOe3w1taPq8HU95hMQchl2749oj8DRyVePVK9aH0= melissali@melissali-ubuntu", "key_options": null, "keyfile": "/home/melissali/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "melissali", "validate_certs": true} 2021-11-03T16:56:58.114 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'liu-chunmei', 'ovpn': 'liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1YAaxAvlRgu72huAggHpoDKSgtF/b00ACfUX4Wk5NyLFbYI3PAQapHNcNL4mIsH8pc6YkdjcJ5N/F12aEupJGQ0Q06OS324xapnaBoUy1hQl9IvhtPqTLEwR0X5xLIAhNBLIEJDNVg13MKwr0u5YVUXdrQRu7Kj9RXAPBahNZFo/Ed+cnR3LICQWliHeHFrk/DU2DUHol3a5wAPxrowAG0AxAecP6/7rASMmitvLQwdQcfIJAhWYgfyXzudg+gOmAEgv3O3U4QOikgmkm/EabUXLJnlTj6yuhtuuctdJr73UT3wLGAGmphLKCkug27kb1nTNoDa+PW/+grP63zhU1 chunmei@otccldstore05", "key_options": null, "keyfile": "/home/liu-chunmei/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "liu-chunmei", "validate_certs": true} 2021-11-03T16:56:58.291 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item={'name': 'ionutbalutoiu', 'ovpn': 'ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ionutbalutoiu", "ovpn": "ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYdmNB3np8OLc49B4cAh59CdNYIb4fAq2XBQ1sbOZMMZmhpFobyy+i6s4lBnovMNWAB6HmpsECHrKRU4IFl9lQ7vV+++J08SLoAkl6G0Gk9tmbgEs+/ovCZg+qUfZA1ZccSAPHmtqfZOiJcnlIlhQq2BoFnLUwmpeIQ4H5D+Y2q7DMdyGkdCenH80aD5KuiwLIyiBL9q7RYz0U5cdCO3wDFU4+5XuZiT8VYDUgp3mWPpAiPdGsJmCn41tpt3PViPDQlKh4vLW0ieQUJI/v2f6oHbM9YkxSoVlwLcrjTv+Nl+dVymqVTF4UaT1kz+ZnoH6khKZxSHvKm9YHc4LNZKYoVdyCxdZvBfxbEbXpT702vLqDNGEfZSlzJs5kVqP30m8fI/yzTQeD7tgjzzrKWdLZacyo+9cWeH6nFfZAQuZ9ug1lv3UAs5x6MsidEH5u6Iymi8BE+mUsOXnSpSfUDDo5n6+CMVe93brdW21iSX9Hbuz5RKB6xile1ROHpvpi1u7VQ9Hp3FIZFFiix9S5jniyB2OqjGKTO6RULfPsJo8sXCLq9YKq9zrkpMv5wURMQKnjtS5OHUWh0cde3U/VikPIfLkzEsfLV8ICAksyuWHQJGVUMxhPmSTKtRsPScbklekT+n65rexDt4Qr73YOy9BUIPV4Mo+kkl4ugQd6VCKJ6w== ibalutoiu@cloudbasesolutions.com", "key_options": null, "keyfile": "/home/ionutbalutoiu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ionutbalutoiu", "validate_certs": true} 2021-11-03T16:56:58.334 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'cyx1231st', 'ovpn': 'cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDLbPdDb+XHOD2fo8g3RacATj0cG07uELwUvqQP35y+JmhNRSb/ttaysFLA5ss0y77TiXx22saP3ECEm5SnuPg5IERkPQmXiw84ATTNfLnzx2G5H223SLaTQiGfSgniBCJHVPCJfNahJiurlZoOAlwDz6SKXTUiivVrFJ22iTBDCQUO6HFQKjE7aM3RAo5mZXKVlTXzM6WOd8vlzk3HMzV0/91JR6H86Lt9MEzXT4dOzqmtxd8gS6U+vA+50zOIspdK+orTRJ0PO2Fn8c1RSYM8g7uqQtpDcOEHderW8TRsqBbDxpyslK6hRX5urUNBX506XE6JNoViLNvCofgseCGj yingxin@yingxin-server", "key_options": null, "keyfile": "/home/cyx1231st/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cyx1231st", "validate_certs": true} 2021-11-03T16:56:58.512 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item={'name': 'pdiazbou', 'ovpn': 'pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdiazbou", "ovpn": "pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDuyLB5rnyjlglcHVCoCSG7pDVJQcTYq/I2yGr3ohK9p6boWxk+zW/a2Q7+qNdKk2DtUC4WvvZ76BU56FC6/LPIUKSJLbN/Xv9/dnZcqKf+ZVlCQgMrE8yGoWt8WLtejJ1iBj0EdoDfYwDiL09ygUim1L1hNxFDOVq9RA2Lo6Tqk5lcAgjoCnU4NZY2YzTXOUSZTzsv/1bd6WVudKHJuWF6ZPb+mbbMoWqwnUZCHwjW8PgTXaPhk1929P0ftWT4ncfKnCc46LXf8OkvzQgCrLyh8964tGKPWppp3wNg2AJ/zFvRY8kkJZSxOawp9Gu8+kCNwWUmLvE9bqYJKT7EMDrU+8TMUEWLFl5fGoxoLWD113PPBcFrpUhvRAJa625eec8a/inv4cqPf6dY6qyXAReXURJm7LnHT4fDq+F6op8+tmZT6Tzkw4+YMUd9h0XDJ4pnPQpk5tjWER/HuWm73+ialXo+ERQWnnVCB1IXq1FEOeF8YAslKC7aSYIZjG9uUkM= peristocles@localhost.localdomain", "key_options": null, "keyfile": "/home/pdiazbou/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdiazbou", "validate_certs": true} 2021-11-03T16:56:58.575 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'sunnagar', 'ovpn': 'sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdqw+8an6CHqsagFqXfuzQi1zCFHCh0e9pDtYz+ylA7L+7vnJLjTR7km0bb4Z6VTUh7h0/nUfF6faPzD2KGmTKvS+mTpeOUaN/H8O8j8qqh90f8z4T8F8L3DwqQbPf/R0izXmTgz0FZObFHmj+G8a+D4ndxF80EPzKkML/G0Hg2GEVdYzazSY4imSY2f56ul6qByBknhFshaTo2DUDcdbDDokF6lO6WgXhY0MJrT24CUSMyZmyi+3hF7vowW2dfbJ+SuDft4XmPDnIxRhDWWvi12jZb8Le7SWYzUlu7fAGbWe63VJqNh992myI61aZshApDD24+1xqIoo+Z/XkQD3l sunnagar@localhost.localdomain", "key_options": null, "keyfile": "/home/sunnagar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sunnagar", "validate_certs": true} 2021-11-03T16:56:58.724 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item={'name': 'xuwang', 'ovpn': 'xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuwang", "ovpn": "xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2iBYL614KRYvFhow408yzMljYRdfyMmbnkUVN8waja4RCsmxTpvIU1aBQ02bMTsY/0n+WMWAbI1D6C1HbuG+JbFYBLSgPCTSG8FIpukFU8y1by8piPu2/ZBD1XdFcPO6u1Nr/tZC08EXhfqvnDAVJUyg6rLLs5+TFKAiPSuv5GyrMRWvMI8eoPk+4u2r/ThxXZTDaIfJhPXlULMLL5ZxR4KZU+WtWbCGSFDGjSugCqdTzpZyX5jUq9C9JB83RxojCtI5n9rCoXLcGOvW9HBnVMxhS56zE8BwxLxLhGy6S1lOoyBPzu7w6XIc160JjCK5iAmlXRqG93nWEVrhZkxCz wangxu298@foxmail.com", "key_options": null, "keyfile": "/home/xuwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuwang", "validate_certs": true} 2021-11-03T16:56:58.787 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'lflores', 'ovpn': 'lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDTlsh9TFwLCeeD2RR+56VhRZ0hp9CHj72U3Z1gbMpxpcg6PYEDzb+0qPVQp1/cIhcwwy0sm13yFl/A5i0iCRpyqB2MbyaXjj0HgkNinxLBFkPSiW2C83lfAyAhG/gLk+FcmWE4+xVrE29aoSM0JtcNtp+q83UGdnFcvhFe0pgJbt5vITo3ZMJND/UZrzeewyK8Vy4Rvpfy8aHYi2Eq+7WQrRHwUwxn0CwsxldnUfmZtu58d+Rtt78LRF0mvcScNlC7q5D+8htmYwkWkK639w5IB3xPv+Z5cV49Psi24Q+9BIr9Zo9/OnnCTauAi5ivDlh+kk/iBksAE8rQpMhvfuSFnNoq2O87aPQvNQHdRCcOa7FVIDcuocu6AB8UP4cGcJreT4+BW5Zbk6vQeJZhmj9yExpRj9MuONy5793nKuAMPn9lOjuXRfK7A5Tb3OOhpEJK1/MjtdZz4mNmFFfFIZ24hgTZO0pOiMwDf/tQ6BG7H8NS9g3QmXPyZajUocw6Szs= lflores@redhat.com", "key_options": null, "keyfile": "/home/lflores/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lflores", "validate_certs": true} 2021-11-03T16:56:58.935 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item={'name': 'blaine', 'ovpn': 'blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "blaine", "ovpn": "blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKSdu3rI8o3qId+GU5KYsxB9CLPTODJFiwy+ww6SgrNKgPfiaSUsZAvcQwoxHfuaEXj6VGaOS6HyFFmvHKhzbecSQCzPIucIuKJJDA7VTaA9Jet0aT2b8Vk4p0Ud9j4WoausmxEdBQC+ZpJxSuqajplejYK1Yi3YEqXSCvM6Md76HCustS8efPvzLsLYttDd34Lw0/zp7ALE8hrepLYJgzQS5Z6btvQEjL7lvtHk80/Y+M6ZIQAl6nPcjb8u+wfs/lxcRA5i94GsZhEoYAeMQptJNvOaA5vYK4QtNyV6lyjGVJl4RUUAtLKKp4JE/kdx7sa0Gy99L+u4MjzBoP5B7SkdJnJx7FioXEen5l5DpiJ6xRixw+SjbqEffW4HHI5+1k6jP6kuHPbWfcrw693+aiFVDQA+myzfWo7iDTjY2H5CtnCh+wr1t3VktXhoyPob/XfMPzMNg4W8nA9KVePwzEXSy2Ro0t71GiFQqBviUuqfktqz4a52PqGyFHtskhFAH3H/h6JYM88vfc7Bw7P/u4eVaoCNgNIbLA/kmf689Q1H2KENj2SX7CDwNL9GCOy/pg89qai63+PFT8258zTi0TywWduP1D0DXXtIB3e5K1yK4Zt6VCinXZAze4bsT+z8y4f9W2Xs/FhhtUmo3mOiDJuaGmuWRzHQQTjL//v/Rimw== blaine@brgardne-mac", "key_options": null, "keyfile": "/home/blaine/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "blaine", "validate_certs": true} 2021-11-03T16:56:59.125 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'jsawaya', 'ovpn': 'jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGq2g8OFN5msjIn5CNB2BRss9eIyJRk3r+7RQcnBOQid jsawaya@redhat.com", "key_options": null, "keyfile": "/home/jsawaya/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jsawaya", "validate_certs": true} 2021-11-03T16:56:59.147 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:56:59 +0000 (0:00:30.162) 0:01:16.733 **** 2021-11-03T16:56:59.219 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'markw8973240', 'ovpn': 'shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDNIdsy+yA0QjwkvjzTiH4lKB+txlh4oGkgC0r1l9swmFvD+MuzNNkD+oB/YscT51tJXsP5/wZDZMuk60xds4Os2ejoIBvzdGBgZ1G2B33etHPmzgTFE7ZegBkMlvWYmrJ9YS+iaMvXn0MMtqfE4zq0kemVAA4NBfFZMhHPJMKCg7Gj31MR0S0b7dcPx7Jnpg4A45oHr0qnufeqZYQaHGgo+LO2xdoV4V2EswWr7YG+b1wI6DzFZBbNsGyQd+cE1BzQ8NmETPNNpDPk5dCyBS7U+4ZTf/WbWk3+Cn+Ky47iWfxcscupQRkCjmTYI/uCMhnrpGU0zEHkbFwLTS6wqDtDmVHm++xof3xSsGJ2cE/vZySZGHqdxqVXfT8gt17h35rME8eX3TU392rjfK6kjEUD3UgwJJp4NFzQIEXi2HTmDYx/Vi9nuId3V1NrDz98QLPUd1ijiy1DuzI0BJPE/DTDmsb7hNXNIcAV8VRc1243W3eaee/Pdg/wmwfkrgjTU3s= shuaiwang@Shuais-MacBook-Pro.local", "key_options": null, "keyfile": "/home/markw8973240/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "markw8973240", "validate_certs": true} 2021-11-03T16:56:59.231 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-11-03T16:56:59.232 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:56:59.242 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:56:59.243 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:56:59.253 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:56:59.264 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:56:59.266 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:56:59.276 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:56:59.287 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@desktop QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@desktop QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:56:59.288 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:56:59.424 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-11-03T16:56:59.426 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'luisrico', 'ovpn': 'lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "luisrico", "ovpn": "lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+ZvZuogU+WfyGj4BR2xYllNoQqAoxQ6AM0IYddBV2pL1xgTM/SopuAxDRP0q2stpozMtjVjlvuiIgr/caqyTuOgYtYCvB09ppefqdnCDrJJBCOKpjUEnCpAKcBMMuz9V32Zqt5FgpIi0Mu7Gp/QCvB97imxEkoUunpdX/8Ri/xz0nJO+QqoJX5wOn91jG49XnkMCLzx+MuafhT90WFO9CV1sIVREZS+X2C7ACQuhx9Qwk7R4BXt2f3M4dA0K2nLwaAnVW50M1y6Gs50Wl+ojDR1itySOxv8YpwyRlRTOzQTyVMnG0Qsaxxwp+hy4b/vQW3NIUvHYdTyvqALTQdGHT lricomor@lricomor-mac", "key_options": null, "keyfile": "/home/luisrico/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "luisrico", "validate_certs": true} 2021-11-03T16:56:59.511 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-11-03T16:56:59.513 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e 2021-11-03T16:56:59.515 INFO:teuthology.task.ansible.out:1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:56:59.524 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:56:59.644 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** changed: [smithi027.front.sepia.ceph.com] => (item={'name': 'matan', 'ovpn': 'matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "matan", "ovpn": "matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIKjxTgyhL1yFriyNvvuhUc6SE8+6f6lT6FTF8cZL9uiz matan.brz@gmail.com\nssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIM8l7qvSQR7fnZdQ1yB1F1jgde0OmLcwcDFzc/o2dYBb matan.brz@gmail.com", "key_options": null, "keyfile": "/home/matan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "matan", "validate_certs": true} 2021-11-03T16:56:59.728 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-11-03T16:56:59.730 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2021-11-03T16:56:59.877 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-11-03T16:56:59.879 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'ozeneva', 'ovpn': 'ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCkFTEUxQkQ1cplOLOf86Teu4qTYGkMqWtjIwzsU/AcimjFolSI5fJ3B72PH7bhQKkSnOlTjiOos2LrD5i/XgxkJD6icVWwHO11lczsq/8K5x0ZMJbJaiT3QphBB4yItoge2Y4/7g4o/78Ne+U7eZ1L5fgeO1fExtIaEXK1lS8pVlpZQXyproTIzp1WLHoaXb38Mxc5jAtuvEKfEycdX3c8k/9ti2qBCZnmjPH05WOxkqCRyNBT+bSJZyFMYt9RDUuycn3BYVSXTAzPW0qwgvKY5vij+qiFoxij7nKNMH7w6C0arolCcJxhiMyEF3UbTuOxgVONWyqFk/vScGGSYj9ytQYjv0L1cFyOVhlpbu+tnptulfQ3faMpjROE/eYxTIoY7OPDgWbFqEUJ4eEhKwAC7aYIYUhvoI/8k+4u64nsyEJQdHXmB+OY3WE5TegYUCuqSsdPY74eO1o8dDgxF363eggDSBmYqSSs+Tpu9TNhJG7iJFX1tiAPwmKvjVrJOac= ozeneva@omri", "key_options": null, "keyfile": "/home/ozeneva/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ozeneva", "validate_certs": true} 2021-11-03T16:56:59.931 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-11-03T16:56:59.933 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2021-11-03T16:56:59.942 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:56:59.953 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:56:59.965 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:56:59.976 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:56:59.978 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:56:59.998 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.000 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.012 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.021 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.032 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.034 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.044 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.046 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.056 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.067 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.069 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.083 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.095 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'sudhanaruna', 'ovpn': 'ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDepyzA+MeQT5oRM52SxmUUKVAzkcrH/NqFxoCpBAjUSioTMaB1h0PhHE00nvHnKRvqWdPl8LCBWS3Ncjh30r9lwHaP+oPy4iLQEc7D2Mwn7WyPh5cq9N3UF25C0AlMG744k1tBNb97qy73OJI+oc5pAj4vUpCC1Ut9pMP5E5pQCx/kwgSOpC7mk0dYr/H5FxjPb0ca55eisFTLXraHPSChdrHjlYC5XpOZ9csilTwFdF0fJdu0nMDwIxGvwTOaUcIcfNLkpGB2Sj/pAM4L9pVsIbeE3C8d978MZ8Ff+ksQ0T8eNzew2ZjSkSuV+oR1sBCJYZ7mbDQuYLKAs5DanO1VFJvgjBJm1zP2ONu+g0fn/j04qIosuudziwoG/dOTkFo+6dBvF5hQi7TbZOIUVFr5dBlGS+O4eoK0gi/wpzun227E4dadY9uB1wiZ/PTD/5og+gxWG32RkHqqX3ALlK4m8JKzrq214nZ7Dt9CNewiUkR15uRkDasGIdiu6qb4Ez8= ArunaSudhan@sudhan-linux", "key_options": null, "keyfile": "/home/sudhanaruna/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sudhanaruna", "validate_certs": true} 2021-11-03T16:57:00.107 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.119 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.130 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.132 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.142 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.143 INFO:teuthology.task.ansible.out: 2021-11-03T16:57:00.154 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.156 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.165 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.167 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.177 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.179 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.189 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.200 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.202 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.211 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.213 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@snowmon CjHOevVvBm+PwhjnhZYM7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@snowmon CjHOevVvBm+PwhjnhZYM7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.223 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.234 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.236 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.246 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.248 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.257 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.259 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.269 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.280 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.282 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.292 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.304 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.315 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.317 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.326 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.328 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.338 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'zl31wang', 'ovpn': 'zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zl31wang", "ovpn": "zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDuHIpWAz6Sk3ATLvTG4KS24JUzgqQBEm4EesPgf1cEG+xK4gISUixaVC9OSNS/KR+8/yoWAhmw992p8cp7mWXxqrRC9kIrthLWvD6Jjz3HTXgarO2oNibeVu27kK/Fm2vvUNACARBkoC5YXd6IuoZOEg1bJu06DLw5Y7pGMqZRVDOrE8WUDwr27roZSgU2/N+18Y1LYn4gwSvWpuC/IotLyGSpW7/dgrZAIreRoYboMdTSlIjjuzeWvYx/AnZmkeEo0IhfyuBRl7ZQwxSFYHn9Z1+iQkt+O9hEZYSZuYRuyFPRqHNMKCEj1iYVRMcwbf/x2/dhjfmbWmjOMJoNNwm+DpixHHbId6Idx82GlAtCgQSTe9aG783oCeZYuVUr1plIRjeEsdjY9UBRYKNknEAChKjMSbxysvO3Dy4yrU2r8qo1iBS1fi4HUg12asCLrJX7vSCvOhOn7ltqDTmt5UiLSn+eCzXpOTBDWz7McB28FoYyR+Rwz5GOH3ITFhPyhXc= wzl@manjaro-wzl", "key_options": null, "keyfile": "/home/zl31wang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zl31wang", "validate_certs": true} 2021-11-03T16:57:00.340 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.350 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.352 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.373 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.374 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.376 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.387 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.389 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.399 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.411 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.413 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.422 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.424 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.434 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.445 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.446 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.456 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.467 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.469 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.479 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.490 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.491 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.502 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.513 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.524 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.526 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.536 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.547 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.549 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.560 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.571 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.572 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'mayagilad', 'ovpn': 'mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIi8JMOVD91QR7EXBkhxeAsB2UO2gyWqYPiP6HjJO/TzzMKf7QtO928XzdORwQ1/jm2hDF1kaNPaU/XIHD0hHSvfICS4ELQcBPa5uPya6d5Kn5fVlpGqBcXBxKGbHB01PGjn0214eupt2LebWxkm7TP/BGDHKXXaoR748AlbyyG2L3TaNoGkwQiEoZe3CWgH4qkrf2CEpyfOa7BM7U0h+lceJxIGe+Yo6Ll14zyngDBa059E+ndFTpkQ6vECII7FdB1+A4Orhu40XHZgyVnlRudD1l80pIVnZves8N2zHZhQo8hp0sB5oY8HENiB09j7Xa6YQ/c1c2wm9E2xJHSNo7muFiHz48vQEgvNmu0lZdbVwXIy3IgLEBHjs6Kyqjtg618V4UaoP/LBOqtWtLd6TvGJ3qBlNyvhO5gZ2DGeLMN2WWoHxfl3VM+QGCGI2dnzUyHhfG1VBDYj8fzWXOuUVpVCMUw8fEg8GWmYLDwm3SbdMACWU87J0kAmaDPKcwwL6AwtWJZr/p89sSypl6y50JlOoDrbDbxrIbHQgbf1tcEgPOuLUN+O06qpi+OxKMtFdbvz+NQ4V4CpXpvMMQZj5AbuHp0EJ5Vme/Kcmpl/xoYGyFDzOEGR8lw6LMz+CpQ4GtnK5XXe1Xn3TJh6Tn2tVyS20p7artemGQQquCN8iQOw== mayag@gett.com", "key_options": null, "keyfile": "/home/mayagilad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mayagilad", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.583 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.594 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.596 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.606 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.617 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.618 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.630 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.642 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.654 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.676 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.678 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.692 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.701 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.702 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.714 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.717 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.726 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.728 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.738 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.740 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.750 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.761 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.763 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.774 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.776 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a28ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a28ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.786 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.798 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'amctagga', 'ovpn': 'amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.800 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'kevinz', 'ovpn': 'kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, "skip_reason": "Conditional result was False"} TASK [users : Update authorized_keys using the keys repo] ********************** 2021-11-03T16:57:00.801 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'arjsharm', 'ovpn': 'arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "arjsharm", "ovpn": "arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDFG1CoPZEh4KBANdJZIzLsutHsF6tAYZTdDdEW+5E+8YyNHOy42D8iBLg+eCAbIl4SFmJTsPNtyO4L8/Ut79tkG3usA9nuGoAwGYuctZUTgOiTiV61940Y+NTvwan9vFNoknVi/3qIGNSXdm90I1aWRTKLCXRbGA1qvu+qyJv93O8LFaPni2TJmtAebY/xf0fHhW2fURUoeoCsoobrTkPA8SojT0A3SM+31iOD7sxbmsqLBR790wz/FHh152PiqpupIijS9TaZgGuVpejM3rgAC7H2kVmKJlrmDlmfcbKLS6BBi+uUDi6ta8whnDRq03nP/4LJE7ZDIxX3I4mHztZeJggURlsfpJdo99+VHkeWlGDBIP41QgD4UX1neuMcIO3nuEnZmLeJyvo9JOR/I7AudbQCDPjSMNbrF+el70mWLhvPe33NA3x6eOBRPc3r+qrSwMhXEPatZlbywJRY+O6yC2TYqdlUKsUEVA/D0Iw7tO+pf3yH3Y/iUDJJLvViRZs= Arjun@localhost.localdomain", "key_options": null, "keyfile": "/home/arjsharm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "arjsharm", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-11-03T16:57:00.802 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'liu-chunmei', 'ovpn': 'liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.814 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'cyx1231st', 'ovpn': 'cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.825 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'sunnagar', 'ovpn': 'sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.826 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'lflores', 'ovpn': 'lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.837 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'jsawaya', 'ovpn': 'jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, "skip_reason": "Conditional result was False"} skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'markw8973240', 'ovpn': 'shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.848 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'luisrico', 'ovpn': 'lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "luisrico", "ovpn": "lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.859 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'matan', 'ovpn': 'matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "matan", "ovpn": "matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.860 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'ozeneva', 'ovpn': 'ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.870 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'sudhanaruna', 'ovpn': 'ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.872 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'zl31wang', 'ovpn': 'zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zl31wang", "ovpn": "zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.882 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'mayagilad', 'ovpn': 'mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.893 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'arjsharm', 'ovpn': 'arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "arjsharm", "ovpn": "arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.904 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'melissali', 'ovpn': 'melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "melissali", "ovpn": "melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.905 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'ionutbalutoiu', 'ovpn': 'ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ionutbalutoiu", "ovpn": "ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.916 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'pdiazbou', 'ovpn': 'pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdiazbou", "ovpn": "pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.927 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'xuwang', 'ovpn': 'xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuwang", "ovpn": "xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.928 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'name': 'blaine', 'ovpn': 'blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "blaine", "ovpn": "blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:00.981 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:00 +0000 (0:00:01.834) 0:01:18.567 **** 2021-11-03T16:57:01.008 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-11-03T16:57:01.010 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'melissali', 'ovpn': 'melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "melissali", "ovpn": "melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCXZDAUNEK4+/6Y+ywEtFnabnfvZmzzHhO6y4PbRHgx8SoQ8+7au/5ElB0vjjiJJSChnDymdqP0UzwcShzhWA8is6K9c6tXUHg6NcgV5tN2xYYmGIIWwm66Z1kYH6u1x2Jkv9LO04rXgebkzXPPYK401agCWjkjUv5acZE12+M/B6d4h2lpFjzBqZ/mmRNp0tHdCapdG/nS1i6GRi3Yi3fQ89CbiZArG9268DSEnLe+npeyBMVdDkWpVIytQ6pU1+g0prrePIVaMvrK3yEooduD4HeLr4liqkINj6rCs90xedPcFi0++UaLFRcgWh3OOnrI952Oe19dx5hlbmcnMPGjaXnkng2keWFyZMOQRRti5l8C/TXPUTaYGWEoqFEEabIKnCura0NWuZHpeF9xEK0tnvNVeauuUfkPxWQ7PH8QQ/9XjOkKbmy0+2q9tug8RC5i5NwcRE4W3ijM7pLwhWMdxkmIOe3w1taPq8HU95hMQchl2749oj8DRyVePVK9aH0= melissali@melissali-ubuntu", "key_options": null, "keyfile": "/home/melissali/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "melissali", "validate_certs": true} 2021-11-03T16:57:01.051 INFO:teuthology.task.ansible.out: TASK [Filter the revoked_users list] ******************************************* skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:01.073 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:01 +0000 (0:00:00.092) 0:01:18.660 **** 2021-11-03T16:57:01.228 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-11-03T16:57:01.230 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'name': 'ionutbalutoiu', 'ovpn': 'ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ionutbalutoiu", "ovpn": "ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCYdmNB3np8OLc49B4cAh59CdNYIb4fAq2XBQ1sbOZMMZmhpFobyy+i6s4lBnovMNWAB6HmpsECHrKRU4IFl9lQ7vV+++J08SLoAkl6G0Gk9tmbgEs+/ovCZg+qUfZA1ZccSAPHmtqfZOiJcnlIlhQq2BoFnLUwmpeIQ4H5D+Y2q7DMdyGkdCenH80aD5KuiwLIyiBL9q7RYz0U5cdCO3wDFU4+5XuZiT8VYDUgp3mWPpAiPdGsJmCn41tpt3PViPDQlKh4vLW0ieQUJI/v2f6oHbM9YkxSoVlwLcrjTv+Nl+dVymqVTF4UaT1kz+ZnoH6khKZxSHvKm9YHc4LNZKYoVdyCxdZvBfxbEbXpT702vLqDNGEfZSlzJs5kVqP30m8fI/yzTQeD7tgjzzrKWdLZacyo+9cWeH6nFfZAQuZ9ug1lv3UAs5x6MsidEH5u6Iymi8BE+mUsOXnSpSfUDDo5n6+CMVe93brdW21iSX9Hbuz5RKB6xile1ROHpvpi1u7VQ9Hp3FIZFFiix9S5jniyB2OqjGKTO6RULfPsJo8sXCLq9YKq9zrkpMv5wURMQKnjtS5OHUWh0cde3U/VikPIfLkzEsfLV8ICAksyuWHQJGVUMxhPmSTKtRsPScbklekT+n65rexDt4Qr73YOy9BUIPV4Mo+kkl4ugQd6VCKJ6w== ibalutoiu@cloudbasesolutions.com", "key_options": null, "keyfile": "/home/ionutbalutoiu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ionutbalutoiu", "validate_certs": true} 2021-11-03T16:57:01.354 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-11-03T16:57:01.356 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2021-11-03T16:57:01.438 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-11-03T16:57:01.440 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'name': 'pdiazbou', 'ovpn': 'pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdiazbou", "ovpn": "pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDuyLB5rnyjlglcHVCoCSG7pDVJQcTYq/I2yGr3ohK9p6boWxk+zW/a2Q7+qNdKk2DtUC4WvvZ76BU56FC6/LPIUKSJLbN/Xv9/dnZcqKf+ZVlCQgMrE8yGoWt8WLtejJ1iBj0EdoDfYwDiL09ygUim1L1hNxFDOVq9RA2Lo6Tqk5lcAgjoCnU4NZY2YzTXOUSZTzsv/1bd6WVudKHJuWF6ZPb+mbbMoWqwnUZCHwjW8PgTXaPhk1929P0ftWT4ncfKnCc46LXf8OkvzQgCrLyh8964tGKPWppp3wNg2AJ/zFvRY8kkJZSxOawp9Gu8+kCNwWUmLvE9bqYJKT7EMDrU+8TMUEWLFl5fGoxoLWD113PPBcFrpUhvRAJa625eec8a/inv4cqPf6dY6qyXAReXURJm7LnHT4fDq+F6op8+tmZT6Tzkw4+YMUd9h0XDJ4pnPQpk5tjWER/HuWm73+ialXo+ERQWnnVCB1IXq1FEOeF8YAslKC7aSYIZjG9uUkM= peristocles@localhost.localdomain", "key_options": null, "keyfile": "/home/pdiazbou/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdiazbou", "validate_certs": true} 2021-11-03T16:57:01.554 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-11-03T16:57:01.555 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2021-11-03T16:57:01.661 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-11-03T16:57:01.663 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'name': 'xuwang', 'ovpn': 'xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xuwang", "ovpn": "xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2iBYL614KRYvFhow408yzMljYRdfyMmbnkUVN8waja4RCsmxTpvIU1aBQ02bMTsY/0n+WMWAbI1D6C1HbuG+JbFYBLSgPCTSG8FIpukFU8y1by8piPu2/ZBD1XdFcPO6u1Nr/tZC08EXhfqvnDAVJUyg6rLLs5+TFKAiPSuv5GyrMRWvMI8eoPk+4u2r/ThxXZTDaIfJhPXlULMLL5ZxR4KZU+WtWbCGSFDGjSugCqdTzpZyX5jUq9C9JB83RxojCtI5n9rCoXLcGOvW9HBnVMxhS56zE8BwxLxLhGy6S1lOoyBPzu7w6XIc160JjCK5iAmlXRqG93nWEVrhZkxCz wangxu298@foxmail.com", "key_options": null, "keyfile": "/home/xuwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xuwang", "validate_certs": true} 2021-11-03T16:57:01.745 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-11-03T16:57:01.747 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2021-11-03T16:57:01.881 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2021-11-03T16:57:01.883 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'name': 'blaine', 'ovpn': 'blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "blaine", "ovpn": "blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKSdu3rI8o3qId+GU5KYsxB9CLPTODJFiwy+ww6SgrNKgPfiaSUsZAvcQwoxHfuaEXj6VGaOS6HyFFmvHKhzbecSQCzPIucIuKJJDA7VTaA9Jet0aT2b8Vk4p0Ud9j4WoausmxEdBQC+ZpJxSuqajplejYK1Yi3YEqXSCvM6Md76HCustS8efPvzLsLYttDd34Lw0/zp7ALE8hrepLYJgzQS5Z6btvQEjL7lvtHk80/Y+M6ZIQAl6nPcjb8u+wfs/lxcRA5i94GsZhEoYAeMQptJNvOaA5vYK4QtNyV6lyjGVJl4RUUAtLKKp4JE/kdx7sa0Gy99L+u4MjzBoP5B7SkdJnJx7FioXEen5l5DpiJ6xRixw+SjbqEffW4HHI5+1k6jP6kuHPbWfcrw693+aiFVDQA+myzfWo7iDTjY2H5CtnCh+wr1t3VktXhoyPob/XfMPzMNg4W8nA9KVePwzEXSy2Ro0t71GiFQqBviUuqfktqz4a52PqGyFHtskhFAH3H/h6JYM88vfc7Bw7P/u4eVaoCNgNIbLA/kmf689Q1H2KENj2SX7CDwNL9GCOy/pg89qai63+PFT8258zTi0TywWduP1D0DXXtIB3e5K1yK4Zt6VCinXZAze4bsT+z8y4f9W2Xs/FhhtUmo3mOiDJuaGmuWRzHQQTjL//v/Rimw== blaine@brgardne-mac", "key_options": null, "keyfile": "/home/blaine/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "blaine", "validate_certs": true} 2021-11-03T16:57:01.971 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-11-03T16:57:01.972 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2021-11-03T16:57:01.994 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:01 +0000 (0:00:00.921) 0:01:19.581 **** 2021-11-03T16:57:02.078 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-11-03T16:57:02.080 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'andrewschoen', 'ovpn': 'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:02.090 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'zack', 'ovpn': 'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:02.091 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'kdreyer', 'ovpn': 'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:02.101 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'dmick', 'ovpn': 'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\ndmick@hplaptop 49Wf5LFRBuobkfWp7OoAdQ 1401efaf459e0b7673f03b546eeff2c8144219c674eb3c94aa23e5af5ffc66b9\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:02.113 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'sage', 'ovpn': 'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:02.114 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'dgalloway', 'ovpn': 'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:02.124 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'akraitma', 'ovpn': 'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:02.125 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'jcaratzas', 'ovpn': 'jcaratza@desktop QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcaratzas", "ovpn": "jcaratza@desktop QI358a0dizGe6T42hCYj1A 139ab306be981d0a669ec2288a2529503882deffceb28bdfc9bde2b70a825898"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:02.136 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-11-03T16:57:02.138 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'trociny', 'ovpn': 'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:02.326 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-11-03T16:57:02.328 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2021-11-03T16:57:02.349 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'smithfarm', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf', 'ovpn': 'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2021-11-03T16:57:02.360 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'mbenjamin', 'ovpn': 'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:02.362 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'aemerson', 'ovpn': 'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01\naemerson@electric TjQxreDKF6bsIehyQnX4aQ a73b7d56b95ba3d4dda5ded995b9cb98eaaf5d2db7d148c340aa4f4c68cf26d7\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:02.529 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-11-03T16:57:02.530 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2021-11-03T16:57:02.572 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-11-03T16:57:02.574 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'amaredia', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour', 'ovpn': 'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2021-11-03T16:57:02.720 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-11-03T16:57:02.722 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "name": "gmeno", "state": "absent"} 2021-11-03T16:57:02.784 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2021-11-03T16:57:02.785 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'name': 'tserlin', 'key': 'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet', 'ovpn': 'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2021-11-03T16:57:02.795 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'dis', 'ovpn': 'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:02.806 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'gregf', 'ovpn': 'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:02.817 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'joshd', 'ovpn': 'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:02.829 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'cmeno', 'ovpn': 'cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cmeno", "ovpn": "cmeno@boxtop KD0XJIGun8xcnNI83pqrsg 79079fe78e1cf401cccb14692c9d16f63e2010c57cfbf56c487387affca90aed"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:02.830 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'ivancich', 'ovpn': 'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:02.840 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'yuriw', 'ovpn': 'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH2 WhMAZ+zZ/PPXOc+4HEDaZg 2d3e28b31edc3f2be43f6b73150dedc9c9df6e316d29afc1416d6b96a982e217\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:02.851 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'tamil', 'ovpn': 'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:02.853 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'jowilkin', 'ovpn': 'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:02.863 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'bhubbard', 'ovpn': 'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:02.874 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'yehudasa', 'ovpn': 'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\nyehuda@crestview yUJ7nsAXRv78MgIyxJ93Yw 5b872feb20cbd895a5d8c87956f9cbd491471654b1a29aa8c13826cd95a44c40\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:02.875 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'dang', 'ovpn': 'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:02.885 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'branto', 'ovpn': 'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:02.887 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'xiaoxichen', 'ovpn': 'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:02.896 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'ffilz', 'ovpn': 'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:02.907 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'joao', 'ovpn': 'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:02.909 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'nhm', 'ovpn': 'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:02.919 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-11-03T16:57:02.920 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "name": "alfredodeza", "state": "absent"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'mkidd', 'ovpn': 'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'jlopez', 'ovpn': 'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:02.931 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'haomaiwang', 'ovpn': 'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:02.933 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'kchai', 'ovpn': 'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:02.942 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:02.954 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'dfuller', 'ovpn': 'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'owasserm', 'ovpn': 'owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@orit-desktop zi8oe8INwK6IZrtUz6lg0w b5f4f96232a77a260f7a16244af873df2ff143c2bde0dbaabf02e9952b8f38ca\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:02.965 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'vasu', 'ovpn': 'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:02.976 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'smohan', 'ovpn': 'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:02.978 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'abhi', 'ovpn': 'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:02.988 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'cbodley', 'ovpn': 'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:02.989 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'mwatts', 'ovpn': 'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.000 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6\nprsrivas@t480 jciDaqMz2kZbSRymwpJiqA 68baba83dd966fe9536bc9a822428710b10a28781f695bc3fb750482a94b0e8c\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.001 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'pdonnell', 'ovpn': 'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.012 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'jlayton', 'ovpn': 'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.026 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'rzarzynski', 'ovpn': 'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.028 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'rdias', 'ovpn': 'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'vshankar', 'ovpn': 'vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride eYOkK3Yj10T5PFLIUu5DKw f6d9d714fb6a97e51a4c8d2a4fb7ed0c0e87caedfd0fa813b209846abd9a8a20"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.039 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'akupczyk', 'ovpn': 'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.050 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'nojha', 'ovpn': 'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e\nnojha@nojha-mac aucFu11aJL4iZg9ch76Lmg 6f14a860471fabc125d167c7af8d4729c26bc93bfab5e73d5457ebf54d32d526\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.052 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'ifed01', 'ovpn': 'ifed01@snowmon CjHOevVvBm+PwhjnhZYM7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@snowmon CjHOevVvBm+PwhjnhZYM7g 7393d3a95b1874764fc4e7ae2d481f1859b336e16e1ba53c49d126585e11ba56"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.063 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'myoungwon', 'ovpn': 'myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@fg2server p5CiGtdZolEwS1dCiRJjkg cd2491f835be1474a7802db607cf9a31c59e1031001e1dfb8fe240616dda9a71"}, "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'gabrioux', 'ovpn': 'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.073 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'leseb', 'ovpn': 'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.075 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'hchen', 'ovpn': 'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.086 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'jcollin', 'ovpn': 'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.088 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'xxg', 'ovpn': 'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.098 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'pcuzner', 'ovpn': 'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.109 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'mkogan', 'ovpn': 'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.111 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** ok: [smithi115.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'vumrao', 'ovpn': 'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.123 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'henrix', 'ovpn': 'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.125 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'pbs1108', 'ovpn': 'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.134 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'clacroix', 'ovpn': 'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.136 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'epuertat', 'ovpn': 'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.146 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'tdehler', 'ovpn': 'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.148 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'laura', 'ovpn': 'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.158 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'yaarit', 'ovpn': 'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.169 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'rpavani1998', 'ovpn': 'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.171 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'rishabh', 'ovpn': 'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.182 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'skrah', 'ovpn': 'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.195 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'smanjara', 'ovpn': 'smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora OfpUShUejOW2W/R4FlKQ7Q fab77ef619f730a5aaf255e4167aa7a35a7e5450d12820923d969036a6326b0f"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.198 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'bengland', 'ovpn': 'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'pnawracay', 'ovpn': 'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.207 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'alfonsomthd', 'ovpn': 'alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost iG3hmp+gS/4aOL4Vp/Nj4w ef1480c222cb2d540cf932ef2f27967891b768c2ad90b59a12a5279ceaa18b47"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.218 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'oliveiradan', 'ovpn': 'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.220 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'swagner', 'ovpn': 'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.230 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'yuvalif', 'ovpn': 'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e\nyuvalif@medea gy0T/Hj/6DnVSKyOKD0/ag 468fe695fad8610a8f2eba6d1015949e4e6dc3d62c50ecf03579928d45a8d2ae\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.243 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'kkeithle', 'ovpn': 'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'emmericp', 'ovpn': 'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.253 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'mchangir', 'ovpn': 'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.255 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'sjust', 'ovpn': 'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.265 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'ideepika', 'ovpn': 'deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@x1carbon hyRj1/9UjWhVzhav5+LNKA e3f7d2882c2b1a410dbb5458c08f280814d652e34c87e96f8a2e8bef1928926f"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.276 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'gsalomon', 'ovpn': 'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.277 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'soumyakoduri', 'ovpn': 'skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@localhost w/vFHwM/x4hUGlKJCteKiA 97933f044fdd8b3cf57ad744ce032728b06f354b0d719790def82b2f5b9a6133"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.288 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'kyr', 'ovpn': 'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'sseshasa', 'ovpn': 'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.299 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'rfriedma', 'ovpn': 'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.302 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'lmb', 'ovpn': 'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.311 INFO:teuthology.task.ansible.out: 2021-11-03T16:57:03.312 INFO:teuthology.task.ansible.out:TASK [Remove revoked users] **************************************************** ok: [smithi115.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2021-11-03T16:57:03.313 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'ksirivad', 'ovpn': 'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc\nksirivad@macbook 0ivvJOZVAywT15aJCDF2xw 0066940a99b2d37dcd856c8622f48e0d8cb59c092f8f96cc2352ec0bc979ce45\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.313 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'ofriedma', 'ovpn': 'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.325 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'h4iresis', 'ovpn': 'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.336 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'shyukri', 'ovpn': 'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'scarvalhojr', 'ovpn': 'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.347 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'rakesh', 'ovpn': 'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.349 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'ryg-', 'ovpn': 'ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat Y8asVyh6Qu+EEmWlRd1zXA d4923cf7be8e59728ea290aaec7670903b39b6444e5d303016338ddf5ab7837f"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.359 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'prsrivas', 'ovpn': 'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.362 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'zdover', 'ovpn': 'zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@lenovot5301 88ArNhsAMXq/nkSC9tKDig 131b8214bbea52345220522f8dd13c768bc5c1e41e4d164e7138719cfc3295d3"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.370 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'rraja', 'ovpn': 'rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@p50 k0lrZNIKWMH/VrZupJ9diA 03bfe34ee6e9a541e10426a3cc1e1046655420d4d0a79ce5667bec1b469a0100"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.382 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'toabctl', 'ovpn': 'toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "toabctl", "ovpn": "toabctl@minze a3I/7JakMEEANsNXCYM5XA b9715003902937328801993a359da43c61ab0be8d17f92371a48dca0821fb13f"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.384 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'mgfritch', 'ovpn': 'mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mgfritch", "ovpn": "mgfritch@laptop 94prpsaAkBwTzMmF9JR4WA 3adb8aad97b1f78c74d3ae91f21f4b165749843dacabad74f0039212c5a5db39"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.395 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'xiubli', 'ovpn': 'root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiubli", "ovpn": "root@lxbceph0 /BNUvfmLPqzfbIz/Jzlqkw 3e109dcb826d45a6ee7e8195fc025b257b3d369e135400bf0a4bf503d48503e3\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.397 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'jafaj', 'ovpn': 'jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jafaj", "ovpn": "jan@jfsuselaptop ddAKCkWSc1rEhQadU15jkA 90ddc588ca94ab776b539a1c5246a216dfbebabba3709ae803d386f978b65c6f"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.406 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'jschmid', 'ovpn': 'jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jschmid", "ovpn": "jschmid@zulu BqyepNoWzZbbsslJbvCuYg 4ce9b431fdd8a06856cfe00b88665da11dde1cacd72d40bf3dbe54ff633066ef"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.408 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'kachau', 'ovpn': 'kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kachau", "ovpn": "kachau@thinkpad dX7+Zp6Z+rMSZTbxL52c7g 59d0d7be858e4a3769eaca4563399989298004a271f60b08cc36bf4130e089fe"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.418 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'khiremat', 'ovpn': 'khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "khiremat", "ovpn": "khiremat@kotresh-T490s 7o9JjenDm1h34Zmd2SWO2g 791981cd84bd4c80bf3ff6076ebc64b3ebf8432903e36d1a49bbee27561f8315"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.419 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'kapandya', 'ovpn': 'kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kapandya", "ovpn": "kapandya@thinkpad g4JoonWShBhyS6fEdaHh7A 2990a987bf7afee6e3c8b5236b3dac813a57dbdce47ec63965c40b1b7bdafd67"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.430 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'prasad', 'ovpn': 'prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prasad", "ovpn": "prasad@macprasad tJYuW/3nDX8ikaLyJQdpAQ 1544f87809a170bb61069b588630467b05283a1347945b9545205b0a28110f89"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.431 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'vikhyat', 'ovpn': 'vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vikhyat", "ovpn": "vikhyat@redhat ae77Y7dK1BxLEfQvAOJo8A a62d65a2aa09a75dca299ce0424caf6bcbc52696b22aa1567432aac34d78f18f"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.442 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'shraddhaag', 'ovpn': 'shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shraddhaag", "ovpn": "shraddhaag@asus w1lmx5nwk8QQ/4u3UfGsJQ 98c5840f994a5e35b234375b07b1782f9a92bdcd5cf73e5847027045ab88e6d2"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.453 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'gkyratsas', 'ovpn': 'gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gkyratsas", "ovpn": "gkyratsas@suse.com PO5Ut+2Z4kCMjaumOyA+1A 70fafe30ccb801c8c0a059a7eb3a842a2d08015b828c63bc1a702caf056014c7"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.455 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'prateek', 'ovpn': 'prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prateek", "ovpn": "prateek@linux kj4pU4yLf+76XxY0mvoIwg c24292e824e3e43ebf7465f1079449ce80bf26311426ae3914cdf1fe8d591c67"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.464 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'jelopez', 'ovpn': 'jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jelopez", "ovpn": "jelopez@imac XbzY2mjr57lSPExJp0ZBgA 4f5dfad692c25293cd58393bd8ddae91b4dc0b2561c860072e0665933531208e"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.466 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'dcain', 'ovpn': 'dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dcain", "ovpn": "dcain@fullsteam 7vptxC1ZWCXLUV3FJwWQAg 2d73728e16c5f724e5f28eb326c24b8e13e7b361bcbaf506809f539e80223389"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.476 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'sunny', 'ovpn': 'sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunny", "ovpn": "sunny@devBox 81r/AKNOWhvaD0ESYy6tTg 58b86902f858bc3f1edb12cfc41ce10ed163985db2f7c253de08b7f1c310fe11"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.487 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'xuxuehan', 'ovpn': 'xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuxuehan", "ovpn": "xuxuehan@xuxuehan-PC aJDFkJ1+XzbovMyvRiFCAQ 5a71908d9bc8372ce0fff3bdbce4ea0e27eb11af0ed30e573f93d7a0b4335279\nxuxuehan@xuxuehan-notebook LzP6obPSYuao0iCJLQ/92A 3af116b690da581c77912a0c5fe184c84ca78526f67331db15714bc4ffddbcdc\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.489 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'aclewett', 'ovpn': 'aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "aclewett", "ovpn": "aclewett@mac bP0KgOkhW1GxTiugD3jMQA 90009b65da8175e083ee0e79d4953268f44493b98c2b532854da8d9e1d14e798"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.499 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'cblum', 'ovpn': 'cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cblum", "ovpn": "cblum@ ad4mZ9Cg0u6H8AdARvQUZw 4e3cec570cbe42786e5190de4194c2937f1073aef62a25b61bd85e05ee23db56"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.500 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** ok: [smithi115.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'mlatimer', 'ovpn': 'mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mlatimer", "ovpn": "mlatimer@lx5510 3lc4nP664Dmg63yr8yKG7w 3778fb16fb62d1a7d61f41cf2c2e72fd9acaae2805de2dc34497ef41ee1ad64f"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.511 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'shyamsundarr', 'ovpn': 'shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyamsundarr", "ovpn": "shyamsundarr@bartimaeus HUEzMC00f5f/tdydddWGNw e9032244bda086ecca6abc4f525296f9a125e4bbdb768d8c76c6d53d02f475d5"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.523 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'shuzhenyi', 'ovpn': 'bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shuzhenyi", "ovpn": "bigwhite@wrjj37deMacBook-Pro-2.local VoTYmzsOPazeWcH+DOHXKw d7502e96b37ebc2ee5e0c14c6026f10e2e72b49be70dd1c5c7e9751fafcd8501"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.525 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'mdipalma', 'ovpn': 'mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mdipalma", "ovpn": "mdipalma@mdmac IVn9Oy/SxZEmDgSWSeo3jA ea4dc08cb39ba55ade357c4949bf3f7d4d0322171ff5b5dac3c06d47cdc38f14"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.534 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'kchang', 'ovpn': 'kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchang", "ovpn": "kchang@kiefer-uchi Vy8tg3hSizdX5fIAJFgG/A 912a49a77b5722fdf4b9e304ee329c7b808728d73bb1e1f70cf55bd1221a3da3"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.536 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'vereddy', 'ovpn': 'vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vereddy", "ovpn": "vereddy@vereddy.remote.csb 8jHiMDBfgQxu/8+MTlPR1A 07898add1a48c50647de4a81ad23db0c8545202cb56d3172db9b22df32c6b4a2"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.546 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'benhanokh', 'ovpn': 'gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "benhanokh", "ovpn": "gbenhano@localhost.localdomain uMf63iCCcqDt1rUnOSujaw 8c554c50bf8e27d3373515168e59f4ed3e4c48bace5689838268569948545ed5"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.557 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'jsalomon', 'ovpn': 'jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsalomon", "ovpn": "jsalomon@Josh-laptop 9V9NVwrt57LxXiqRh5Q7PQ 4cf0b0dfa39cf1e72f51bbfed19b4d88bdb286d09e54a99eb35721e33b1ae7b0"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.559 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'jolmomar', 'ovpn': 'jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jolmomar", "ovpn": "jolmomar@thinkpad CnoIzb/786lZh1NINj0xAA e60eeb560654da4b84977a4f9dbccd599bf9460980985ee5726595eb4a46e640"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.569 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'julpark', 'ovpn': 'julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "julpark", "ovpn": "julpark@thinkpad svj+nOCQG95zuBUwdJoluQ 7252f05ef02b16fd74d931d6cf937304afbb098dbdbde13d377e884fe35e2fb8"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.570 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'pdhange', 'ovpn': 'pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdhange", "ovpn": "pdhange@mauli hqKEk1OuI2Wmi1CAcJk5CQ f49d854572b5efe9514b4d741e336adb3534d00b22b0a505142d82eccc2f7535"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.581 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'dpivonka', 'ovpn': 'dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dpivonka", "ovpn": "dpivonka@thinkpad 6vBnL22Dfp8d9grCNtSoHw c8270453301dbec76f8070809078087ff892c74360b80a82002d58c50af19ec4"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.591 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'adking', 'ovpn': 'adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adking", "ovpn": "adking@laptop1 VkjYUD1hJpR8sFhrH6MIcg 28c610bc9f7321de315923288d24650bdb4bc90593b423f1f59db16d67b6d625"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.596 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'amathuri', 'ovpn': 'amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amathuri", "ovpn": "amathuri@thinkpad WDHZL4VspBmlz5dzRlAGUQ 7f2c5d8e2554b869a09fdc20d6e4aabffa8200ef8e2b8d1bc951bfc673757267\namathuri@lenovothinkpad MhlrEG4X+iKq2d2GxwMbmA 812cfea500340140144ecb27f7851b86d8daff658f173189fceec8e61b59cbf2\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.604 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'nizamial09', 'ovpn': 'nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a28ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nizamial09", "ovpn": "nia@redhat OA+N80Ps0Ua7PfuVF4+M8w a9b0ecf7a28ef91e0653b0193f3f95967d6a671eff21ee8a99620a5a972ba409"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.606 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'dachary', 'ovpn': 'loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dachary", "ovpn": "loic@tulipe 1NGxatlPDW6m9FLwzG2Gng cc19cfe2d2101909e9e0ae6b207d7e8dccafd8ec1f4594bdd006dda6b67a80b5"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.615 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'amctagga', 'ovpn': 'amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amctagga", "ovpn": "amctagga@redhat20 DsFh00Ttjtomgs5nlTX3SA 713280e601e14a8fecfe190490e362f5f97bba3517b2bcfd4abd5230a45fbaa1"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.626 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'kevinz', 'ovpn': 'kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kevinz", "ovpn": "kevinz@linaroceph a2GchLhkaayg78xuUnI0sA b545e84634e614099b00d50e593933eabf0793035eaa439b6d82ec669f69de7f"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.627 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'liu-chunmei', 'ovpn': 'liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "liu-chunmei", "ovpn": "liu-chunmei@otccldstore gKMLlCSH99LCDse00ja0Kg 2c140368bca0e48102330631f91e5299298461a24cab483b9765293eafb2050c"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.637 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'cyx1231st', 'ovpn': 'cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cyx1231st", "ovpn": "cyx1231st@yingxin-server C3qUn5JXYFehgyfol2lnhw c5f306797eac838797ea99b07bd0f5e8e185052887b82ba5c343a42eb271747a"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.639 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'sunnagar', 'ovpn': 'sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sunnagar", "ovpn": "sunil@sepia-lab j6h8NzyTGEE6WHYT3uiOXA eda7763cdd544d8e7198cefee77f38641d44c8a6aa4d37b8b1429620641809ae"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.648 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'lflores', 'ovpn': 'lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lflores", "ovpn": "lflores@fedora CqF8+5Scni+GHCfHkm3AAA 0771c83aa58141af4704f4b01bba4490638ee2033fdabe5a2a4b8e06768b29c2"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.659 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'jsawaya', 'ovpn': 'jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jsawaya", "ovpn": "jsawaya@thinkpad jEJaJNljo4e2sHstFCNljQ d0042e4c40344f7fa350d3e0b748686aea00c4757ffbe32a41f3b119463d9f46"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.661 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'markw8973240', 'ovpn': 'shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "markw8973240", "ovpn": "shuai@mac d7Tqhwnt5D7+jeeqKQ375w e6ac2fb0750350a007d00bba1354989402199dc012150a84805be57a92e6f06d"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.671 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'luisrico', 'ovpn': 'lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "luisrico", "ovpn": "lricomor@lricomor-mac DI4SNq2EznedeK8Cztt2ug e05bc16840e8c3c45f641852101ad921abc248013ec89a599c27c275767b5347"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.673 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'matan', 'ovpn': 'matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "matan", "ovpn": "matan@main-win vEAxj3ZM9DjxBi/jfjc+jQ 9c30b8fcef9bdc804090947a92e1e60cc21d3f280f261bc3f4cecac64541dc6e\nmatan@thinkpad JJuwaWbac2lbcH5pCGON1w 521fe30e3a950915a1b538913c1af76198c4133018b7869d348740c708b38a7d\n"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.683 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'ozeneva', 'ovpn': 'ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ozeneva", "ovpn": "ozeneva@omri qB3om9JUxgVneOibFeRq6Q 92277afcd0bce888f60dceab4b2e8695ebbf7241dba09e3350a12c4fffd6ebd5"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.694 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'sudhanaruna', 'ovpn': 'ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sudhanaruna", "ovpn": "ArunaSudhan@sudhan-linux t/fhf5WR4vQxW3F/Lx7Atw 09248b17c9abfe802ca4be1b93ad318138f54fb60b073ff857c2effadb27a997"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.696 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'zl31wang', 'ovpn': 'zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zl31wang", "ovpn": "zl31wang@sjtu Et6K7OVsdRTnvOY5iuvUxA a7f66c06b405df269ac8f28a4b2c9f00d087091082c53ed5e03517e3811d9fc2"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.706 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'mayagilad', 'ovpn': 'mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mayagilad", "ovpn": "mayagilad@mayagilad-ThinkPad-T480 AGw0uOFkRU6SX8Kp3nyGuA d29a792aad9cc6038dfc2e3471f6556b50ef73562b1bc6231c39d8daf20a19d0"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.708 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** ok: [smithi115.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'arjsharm', 'ovpn': 'arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "arjsharm", "ovpn": "arjun@thinkpad_local 06szrDO7vKFS7BSicSbDsw e1284059b3e0604191a0f35e2841f22835b0e75249e637d8297edbee458ccf39"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.718 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'melissali', 'ovpn': 'melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "melissali", "ovpn": "melissali@melissali-ubuntu pooAecOHrOtZ3gXewVMxnw 36860a4078c773f47e1a58d2f120ec79e468142120ff6a816cb6ffbb9fd9ef07"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.720 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'ionutbalutoiu', 'ovpn': 'ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ionutbalutoiu", "ovpn": "ionutbalutoiu@macbook ruUdfqkSfEhKctGGLeQvkw 0a6cab2332041d56ca8b1216242195af2ad8d0ec6360fd6c1dd080f5f43240e1"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.730 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'pdiazbou', 'ovpn': 'pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdiazbou", "ovpn": "pdiazbou@thinkpad BNtoPcISV462wNeprZDOcg f6faca031e6549014132b4c66e53721c9d934a09d0d9d0f36e2dd31f90be1ce4"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.732 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'xuwang', 'ovpn': 'xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xuwang", "ovpn": "xuwang@laptop 2U0uRjvVh8eE4iSkFzK2Kg 2a9db1c375a032cc0a892ca2d0721f9d7303e2db5a7eab647870d7c2022ee2e8"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.742 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'name': 'blaine', 'ovpn': 'blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "blaine", "ovpn": "blaine@laptop RkDSJBkoRKI0UaxdQuMNCw abcfdade65dd772c066c0c63e712fb03240d676d72eb341186b0c0a94d4a4547"}, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.783 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:03 +0000 (0:00:01.789) 0:01:21.370 **** 2021-11-03T16:57:03.853 INFO:teuthology.task.ansible.out: TASK [Filter the revoked_users list] ******************************************* 2021-11-03T16:57:03.855 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:03.875 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:03 +0000 (0:00:00.091) 0:01:21.462 **** 2021-11-03T16:57:03.903 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-11-03T16:57:03.905 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2021-11-03T16:57:04.093 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2021-11-03T16:57:04.167 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2021-11-03T16:57:04.305 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2021-11-03T16:57:04.370 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2021-11-03T16:57:04.496 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2021-11-03T16:57:04.571 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2021-11-03T16:57:04.687 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2021-11-03T16:57:04.777 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2021-11-03T16:57:04.873 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2021-11-03T16:57:04.966 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2021-11-03T16:57:05.072 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2021-11-03T16:57:05.167 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2021-11-03T16:57:05.274 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2021-11-03T16:57:05.369 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2021-11-03T16:57:05.465 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2021-11-03T16:57:05.565 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=gmeno) => {"ansible_loop_var": "item", "changed": false, "item": "gmeno", "name": "gmeno", "state": "absent"} 2021-11-03T16:57:05.659 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2021-11-03T16:57:05.776 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=alfredodeza) => {"ansible_loop_var": "item", "changed": false, "item": "alfredodeza", "name": "alfredodeza", "state": "absent"} 2021-11-03T16:57:05.851 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2021-11-03T16:57:05.979 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2021-11-03T16:57:06.032 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2021-11-03T16:57:06.181 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2021-11-03T16:57:06.224 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2021-11-03T16:57:06.383 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2021-11-03T16:57:06.415 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2021-11-03T16:57:06.584 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2021-11-03T16:57:06.606 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2021-11-03T16:57:06.786 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2021-11-03T16:57:06.808 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2021-11-03T16:57:06.987 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2021-11-03T16:57:07.000 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2021-11-03T16:57:07.200 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} ok: [smithi027.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2021-11-03T16:57:07.390 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2021-11-03T16:57:07.402 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2021-11-03T16:57:07.582 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2021-11-03T16:57:07.604 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2021-11-03T16:57:07.772 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2021-11-03T16:57:07.806 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2021-11-03T16:57:07.963 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2021-11-03T16:57:08.007 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2021-11-03T16:57:08.155 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "name": "pranith", "state": "absent"} 2021-11-03T16:57:08.198 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2021-11-03T16:57:08.344 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "name": "dorinda", "state": "absent"} 2021-11-03T16:57:08.408 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2021-11-03T16:57:08.534 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=zyan) => {"ansible_loop_var": "item", "changed": false, "item": "zyan", "name": "zyan", "state": "absent"} 2021-11-03T16:57:08.608 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=jj) => {"ansible_loop_var": "item", "changed": false, "item": "jj", "name": "jj", "state": "absent"} 2021-11-03T16:57:08.724 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=jdillaman) => {"ansible_loop_var": "item", "changed": false, "item": "jdillaman", "name": "jdillaman", "state": "absent"} 2021-11-03T16:57:08.810 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=amarangone) => {"ansible_loop_var": "item", "changed": false, "item": "amarangone", "name": "amarangone", "state": "absent"} 2021-11-03T16:57:09.018 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=oprypin) => {"ansible_loop_var": "item", "changed": false, "item": "oprypin", "name": "oprypin", "state": "absent"} 2021-11-03T16:57:09.216 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=adamyanova) => {"ansible_loop_var": "item", "changed": false, "item": "adamyanova", "name": "adamyanova", "state": "absent"} 2021-11-03T16:57:09.414 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=sbillah) => {"ansible_loop_var": "item", "changed": false, "item": "sbillah", "name": "sbillah", "state": "absent"} 2021-11-03T16:57:09.612 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=onyb) => {"ansible_loop_var": "item", "changed": false, "item": "onyb", "name": "onyb", "state": "absent"} 2021-11-03T16:57:09.814 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=jwilliamson) => {"ansible_loop_var": "item", "changed": false, "item": "jwilliamson", "name": "jwilliamson", "state": "absent"} 2021-11-03T16:57:10.023 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=kmroz) => {"ansible_loop_var": "item", "changed": false, "item": "kmroz", "name": "kmroz", "state": "absent"} 2021-11-03T16:57:10.181 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item=davidz) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "davidz", "name": "davidz", "remove": false, "state": "absent"} 2021-11-03T16:57:10.223 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=shehbazj) => {"ansible_loop_var": "item", "changed": false, "item": "shehbazj", "name": "shehbazj", "state": "absent"} 2021-11-03T16:57:10.422 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=abhishekvrshny) => {"ansible_loop_var": "item", "changed": false, "item": "abhishekvrshny", "name": "abhishekvrshny", "state": "absent"} 2021-11-03T16:57:10.621 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=asheplyakov) => {"ansible_loop_var": "item", "changed": false, "item": "asheplyakov", "name": "asheplyakov", "state": "absent"} 2021-11-03T16:57:10.819 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=liupan) => {"ansible_loop_var": "item", "changed": false, "item": "liupan", "name": "liupan", "state": "absent"} 2021-11-03T16:57:11.016 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=adeza) => {"ansible_loop_var": "item", "changed": false, "item": "adeza", "name": "adeza", "state": "absent"} 2021-11-03T16:57:11.215 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=pranith) => {"ansible_loop_var": "item", "changed": false, "item": "pranith", "name": "pranith", "state": "absent"} 2021-11-03T16:57:11.414 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=dorinda) => {"ansible_loop_var": "item", "changed": false, "item": "dorinda", "name": "dorinda", "state": "absent"} 2021-11-03T16:57:11.613 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=zyan) => {"ansible_loop_var": "item", "changed": false, "item": "zyan", "name": "zyan", "state": "absent"} 2021-11-03T16:57:11.626 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item=wusui) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "wusui", "name": "wusui", "remove": false, "state": "absent"} 2021-11-03T16:57:11.813 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=jdillaman) => {"ansible_loop_var": "item", "changed": false, "item": "jdillaman", "name": "jdillaman", "state": "absent"} 2021-11-03T16:57:13.120 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item=nwatkins) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "nwatkins", "name": "nwatkins", "remove": false, "state": "absent"} 2021-11-03T16:57:13.411 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item=davidz) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "davidz", "name": "davidz", "remove": false, "state": "absent"} 2021-11-03T16:57:14.572 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item=sidharthanup) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "sidharthanup", "name": "sidharthanup", "remove": false, "state": "absent"} 2021-11-03T16:57:15.099 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item=wusui) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "wusui", "name": "wusui", "remove": false, "state": "absent"} 2021-11-03T16:57:15.994 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item=varsha) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "varsha", "name": "varsha", "remove": false, "state": "absent"} 2021-11-03T16:57:16.755 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item=nwatkins) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "nwatkins", "name": "nwatkins", "remove": false, "state": "absent"} 2021-11-03T16:57:17.392 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item=hmunjulu) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "hmunjulu", "name": "hmunjulu", "remove": false, "state": "absent"} 2021-11-03T16:57:17.601 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2021-11-03T16:57:17.802 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2021-11-03T16:57:17.990 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2021-11-03T16:57:18.179 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2021-11-03T16:57:18.379 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} changed: [smithi027.front.sepia.ceph.com] => (item=sidharthanup) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "sidharthanup", "name": "sidharthanup", "remove": false, "state": "absent"} 2021-11-03T16:57:18.559 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2021-11-03T16:57:18.749 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2021-11-03T16:57:18.938 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2021-11-03T16:57:19.137 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "name": "jbainbri", "state": "absent"} 2021-11-03T16:57:19.327 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "name": "kdhananj", "state": "absent"} 2021-11-03T16:57:19.388 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:19 +0000 (0:00:15.512) 0:01:36.974 **** 2021-11-03T16:57:19.940 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2021-11-03T16:57:19.941 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => {"changed": true, "checksum": "2551e778b6ddae713f4992ad172b228017a89982", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "bd234e5f7b7bb28068648cde28f6f057", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1635958639.4837525-5917-251376456448333/source", "state": "file", "uid": 0} 2021-11-03T16:57:19.964 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:19 +0000 (0:00:00.576) 0:01:37.550 **** 2021-11-03T16:57:20.034 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-11-03T16:57:20.036 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item=varsha) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "varsha", "name": "varsha", "remove": false, "state": "absent"} TASK [common : Log the OS name, version and release] *************************** ok: [smithi115.front.sepia.ceph.com] => { "msg": "Host smithi115.front.sepia.ceph.com is running CentOS 8.3 (n/a)" } 2021-11-03T16:57:20.058 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:20 +0000 (0:00:00.094) 0:01:37.645 **** 2021-11-03T16:57:20.129 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** ok: [smithi115.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml) => {"ansible_facts": {"nrpe_selinux_packages": ["python3-libsemanage", "python3-policycoreutils"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml"} 2021-11-03T16:57:20.151 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:20 +0000 (0:00:00.092) 0:01:37.738 **** 2021-11-03T16:57:20.233 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2021-11-03T16:57:20.234 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:20.255 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:20 +0000 (0:00:00.103) 0:01:37.842 **** 2021-11-03T16:57:20.579 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.088424", "end": "2021-11-03 16:57:20.556567", "rc": 0, "start": "2021-11-03 16:57:20.468143", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2021-11-03T16:57:20.600 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:20 +0000 (0:00:00.344) 0:01:38.187 **** 2021-11-03T16:57:20.671 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:20.693 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:20 +0000 (0:00:00.093) 0:01:38.280 **** 2021-11-03T16:57:20.762 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2021-11-03T16:57:20.764 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:20.785 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:20 +0000 (0:00:00.091) 0:01:38.371 **** 2021-11-03T16:57:20.853 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2021-11-03T16:57:20.855 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:20.880 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:20 +0000 (0:00:00.095) 0:01:38.467 **** 2021-11-03T16:57:20.952 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:20.975 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:20 +0000 (0:00:00.094) 0:01:38.561 **** 2021-11-03T16:57:21.044 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2021-11-03T16:57:21.046 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:21.070 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:21 +0000 (0:00:00.092) 0:01:38.654 **** 2021-11-03T16:57:21.138 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2021-11-03T16:57:21.140 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:21.160 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:21 +0000 (0:00:00.092) 0:01:38.747 **** 2021-11-03T16:57:21.241 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2021-11-03T16:57:21.243 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi115.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-11-03T16:57:21.265 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:21 +0000 (0:00:00.105) 0:01:38.852 **** 2021-11-03T16:57:21.348 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2021-11-03T16:57:21.350 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:21.371 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:21 +0000 (0:00:00.105) 0:01:38.958 **** 2021-11-03T16:57:21.439 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2021-11-03T16:57:21.441 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:21.461 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:21 +0000 (0:00:00.090) 0:01:39.048 **** 2021-11-03T16:57:21.541 INFO:teuthology.task.ansible.out: TASK [common : Set the releasever] ********************************************* 2021-11-03T16:57:21.542 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:21.563 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:21 +0000 (0:00:00.101) 0:01:39.150 **** 2021-11-03T16:57:21.632 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** skipping: [smithi115.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-11-03T16:57:21.654 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:21 +0000 (0:00:00.091) 0:01:39.241 **** 2021-11-03T16:57:21.692 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-11-03T16:57:21.694 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item=hmunjulu) => {"ansible_loop_var": "item", "changed": true, "force": false, "item": "hmunjulu", "name": "hmunjulu", "remove": false, "state": "absent"} 2021-11-03T16:57:21.736 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2021-11-03T16:57:21.738 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:21.762 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:21 +0000 (0:00:00.107) 0:01:39.348 **** 2021-11-03T16:57:21.830 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2021-11-03T16:57:21.832 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-11-03T16:57:21.853 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:21 +0000 (0:00:00.091) 0:01:39.440 **** 2021-11-03T16:57:21.902 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-11-03T16:57:21.904 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2021-11-03T16:57:21.924 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:21.946 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:21 +0000 (0:00:00.092) 0:01:39.533 **** 2021-11-03T16:57:22.015 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2021-11-03T16:57:22.017 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:22.038 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:22 +0000 (0:00:00.092) 0:01:39.625 **** 2021-11-03T16:57:22.107 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-11-03T16:57:22.109 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2021-11-03T16:57:22.119 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2021-11-03T16:57:22.121 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:22.142 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:22 +0000 (0:00:00.103) 0:01:39.728 **** 2021-11-03T16:57:22.211 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2021-11-03T16:57:22.213 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:22.234 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:22 +0000 (0:00:00.091) 0:01:39.820 **** 2021-11-03T16:57:22.305 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* skipping: [smithi115.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-11-03T16:57:22.327 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:22 +0000 (0:00:00.093) 0:01:39.914 **** 2021-11-03T16:57:22.334 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-11-03T16:57:22.336 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2021-11-03T16:57:22.399 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:22.421 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:22 +0000 (0:00:00.094) 0:01:40.008 **** 2021-11-03T16:57:22.492 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2021-11-03T16:57:22.493 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:22.514 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:22 +0000 (0:00:00.092) 0:01:40.100 **** 2021-11-03T16:57:22.522 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** ok: [smithi027.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2021-11-03T16:57:22.586 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2021-11-03T16:57:22.588 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:22.609 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:22 +0000 (0:00:00.095) 0:01:40.196 **** 2021-11-03T16:57:22.678 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2021-11-03T16:57:22.680 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:22.701 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:22 +0000 (0:00:00.091) 0:01:40.287 **** 2021-11-03T16:57:22.729 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-11-03T16:57:22.731 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2021-11-03T16:57:22.774 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2021-11-03T16:57:22.776 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:22.806 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:22 +0000 (0:00:00.105) 0:01:40.393 **** 2021-11-03T16:57:22.878 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2021-11-03T16:57:22.880 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:22.901 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:22 +0000 (0:00:00.094) 0:01:40.488 **** 2021-11-03T16:57:22.920 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-11-03T16:57:22.922 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2021-11-03T16:57:22.985 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2021-11-03T16:57:22.987 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-11-03T16:57:23.016 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:23 +0000 (0:00:00.114) 0:01:40.602 **** 2021-11-03T16:57:23.086 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2021-11-03T16:57:23.088 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:23.119 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:23 +0000 (0:00:00.103) 0:01:40.706 **** 2021-11-03T16:57:23.128 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-11-03T16:57:23.130 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2021-11-03T16:57:23.182 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:23.204 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:23 +0000 (0:00:00.084) 0:01:40.790 **** 2021-11-03T16:57:23.315 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-11-03T16:57:23.317 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=fche) => {"ansible_loop_var": "item", "changed": false, "item": "fche", "name": "fche", "state": "absent"} 2021-11-03T16:57:23.464 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** ok: [smithi115.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-11-03T16:57:23.485 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:23 +0000 (0:00:00.281) 0:01:41.072 **** 2021-11-03T16:57:23.523 INFO:teuthology.task.ansible.out: TASK [Remove revoked users] **************************************************** 2021-11-03T16:57:23.525 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=jbainbri) => {"ansible_loop_var": "item", "changed": false, "item": "jbainbri", "name": "jbainbri", "state": "absent"} 2021-11-03T16:57:23.726 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=kdhananj) => {"ansible_loop_var": "item", "changed": false, "item": "kdhananj", "name": "kdhananj", "state": "absent"} 2021-11-03T16:57:23.786 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:23 +0000 (0:00:00.300) 0:01:41.372 **** 2021-11-03T16:57:24.263 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2021-11-03T16:57:24.265 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "86c0d58bde7ad4cfaafe592d2fefd92c920c42ca", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 236, "state": "file", "uid": 0} 2021-11-03T16:57:24.352 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2021-11-03T16:57:24.355 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "checksum": "2551e778b6ddae713f4992ad172b228017a89982", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "bd234e5f7b7bb28068648cde28f6f057", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1635958643.8810806-6624-225264777026616/source", "state": "file", "uid": 0} 2021-11-03T16:57:24.377 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:24 +0000 (0:00:00.591) 0:01:41.964 **** 2021-11-03T16:57:24.451 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2021-11-03T16:57:24.453 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => { "msg": "Host smithi027.front.sepia.ceph.com is running CentOS 8.3 (n/a)" } 2021-11-03T16:57:24.475 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:24 +0000 (0:00:00.097) 0:01:42.061 **** 2021-11-03T16:57:24.558 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** ok: [smithi027.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml) => {"ansible_facts": {"nrpe_selinux_packages": ["python3-libsemanage", "python3-policycoreutils"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/centos_8.yml"} 2021-11-03T16:57:24.579 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:24 +0000 (0:00:00.104) 0:01:42.166 **** 2021-11-03T16:57:24.649 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2021-11-03T16:57:24.651 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:24.683 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:24 +0000 (0:00:00.103) 0:01:42.270 **** 2021-11-03T16:57:24.714 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2021-11-03T16:57:24.715 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a1bad5509ed35b895c2061841236b1cfe10619ae", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 262, "state": "file", "uid": 0} 2021-11-03T16:57:24.756 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:24 +0000 (0:00:00.072) 0:01:42.342 **** 2021-11-03T16:57:24.998 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.080847", "end": "2021-11-03 16:57:24.982078", "rc": 0, "start": "2021-11-03 16:57:24.901231", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2021-11-03T16:57:25.021 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:25 +0000 (0:00:00.265) 0:01:42.607 **** 2021-11-03T16:57:25.092 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:25.118 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:25 +0000 (0:00:00.097) 0:01:42.705 **** 2021-11-03T16:57:25.187 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:25.210 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:25 +0000 (0:00:00.091) 0:01:42.796 **** 2021-11-03T16:57:25.280 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2021-11-03T16:57:25.282 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "3c9ba7bacd1e30bed014cefd1fb4eab0f0f8fa10", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 484, "state": "file", "uid": 0} TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:25.304 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:25 +0000 (0:00:00.094) 0:01:42.890 **** 2021-11-03T16:57:25.374 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2021-11-03T16:57:25.375 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:25.395 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:25 +0000 (0:00:00.091) 0:01:42.982 **** 2021-11-03T16:57:25.466 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:25.488 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:25 +0000 (0:00:00.092) 0:01:43.075 **** 2021-11-03T16:57:25.581 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:25.604 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:25 +0000 (0:00:00.116) 0:01:43.191 **** 2021-11-03T16:57:25.685 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2021-11-03T16:57:25.687 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi027.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-11-03T16:57:25.711 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:25 +0000 (0:00:00.107) 0:01:43.298 **** 2021-11-03T16:57:25.741 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2021-11-03T16:57:25.743 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "d542c3e2c30a0d3eb7d454775342c302c0ded15b", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 540, "state": "file", "uid": 0} 2021-11-03T16:57:25.769 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:25 +0000 (0:00:00.057) 0:01:43.355 **** 2021-11-03T16:57:25.788 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:25.813 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:25 +0000 (0:00:00.043) 0:01:43.399 **** 2021-11-03T16:57:25.842 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-11-03T16:57:25.844 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:25.868 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:25 +0000 (0:00:00.055) 0:01:43.455 **** 2021-11-03T16:57:25.887 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2021-11-03T16:57:25.889 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:25.911 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:25 +0000 (0:00:00.043) 0:01:43.498 **** 2021-11-03T16:57:25.940 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-11-03T16:57:25.942 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:25.964 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:25 +0000 (0:00:00.052) 0:01:43.551 **** 2021-11-03T16:57:25.982 INFO:teuthology.task.ansible.out: TASK [common : Set the releasever] ********************************************* 2021-11-03T16:57:25.983 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:26.007 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:26 +0000 (0:00:00.042) 0:01:43.593 **** 2021-11-03T16:57:26.035 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2021-11-03T16:57:26.036 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:26.076 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:26 +0000 (0:00:00.068) 0:01:43.662 **** 2021-11-03T16:57:26.103 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2021-11-03T16:57:26.105 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-11-03T16:57:26.127 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:26 +0000 (0:00:00.051) 0:01:43.713 **** 2021-11-03T16:57:26.144 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2021-11-03T16:57:26.146 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:26.168 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:26 +0000 (0:00:00.041) 0:01:43.755 **** 2021-11-03T16:57:26.196 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:26.221 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:26 +0000 (0:00:00.052) 0:01:43.808 **** 2021-11-03T16:57:26.270 INFO:teuthology.task.ansible.out: TASK [common : Mask sleep units] *********************************************** 2021-11-03T16:57:26.272 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item=sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "sleep.target", "skip_reason": "Conditional result was False"} skipping: [smithi115.front.sepia.ceph.com] => (item=suspend.target) => {"ansible_loop_var": "item", "changed": false, "item": "suspend.target", "skip_reason": "Conditional result was False"} 2021-11-03T16:57:26.282 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item=hibernate.target) => {"ansible_loop_var": "item", "changed": false, "item": "hibernate.target", "skip_reason": "Conditional result was False"} skipping: [smithi115.front.sepia.ceph.com] => (item=hybrid-sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "hybrid-sleep.target", "skip_reason": "Conditional result was False"} 2021-11-03T16:57:26.312 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:26 +0000 (0:00:00.091) 0:01:43.899 **** 2021-11-03T16:57:26.320 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** skipping: [smithi027.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-11-03T16:57:26.346 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:26 +0000 (0:00:00.032) 0:01:43.932 **** 2021-11-03T16:57:26.385 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-11-03T16:57:26.387 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:26.410 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:26 +0000 (0:00:00.064) 0:01:43.996 **** 2021-11-03T16:57:26.428 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2021-11-03T16:57:26.430 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:26.451 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:26 +0000 (0:00:00.041) 0:01:44.038 **** 2021-11-03T16:57:26.479 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:26.505 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:26 +0000 (0:00:00.053) 0:01:44.091 **** 2021-11-03T16:57:26.525 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2021-11-03T16:57:26.527 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:26.552 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:26 +0000 (0:00:00.046) 0:01:44.138 **** 2021-11-03T16:57:26.581 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:26.604 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:26 +0000 (0:00:00.052) 0:01:44.190 **** 2021-11-03T16:57:26.632 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2021-11-03T16:57:26.633 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:26.660 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:26 +0000 (0:00:00.056) 0:01:44.247 **** 2021-11-03T16:57:26.680 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2021-11-03T16:57:26.682 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:26.707 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:26 +0000 (0:00:00.046) 0:01:44.294 **** 2021-11-03T16:57:26.736 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2021-11-03T16:57:26.738 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:26.758 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:26 +0000 (0:00:00.051) 0:01:44.345 **** 2021-11-03T16:57:26.776 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2021-11-03T16:57:26.779 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:26.800 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:26 +0000 (0:00:00.041) 0:01:44.387 **** 2021-11-03T16:57:26.829 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2021-11-03T16:57:26.831 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2021-11-03T16:57:26.853 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:26 +0000 (0:00:00.052) 0:01:44.440 **** 2021-11-03T16:57:26.933 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-11-03T16:57:26.935 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:26.962 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:26 +0000 (0:00:00.108) 0:01:44.548 **** 2021-11-03T16:57:27.042 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:27.065 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:27 +0000 (0:00:00.102) 0:01:44.651 **** 2021-11-03T16:57:27.133 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2021-11-03T16:57:27.135 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:27.166 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:27 +0000 (0:00:00.101) 0:01:44.753 **** 2021-11-03T16:57:27.239 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2021-11-03T16:57:27.241 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:27.262 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:27 +0000 (0:00:00.095) 0:01:44.849 **** 2021-11-03T16:57:27.331 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2021-11-03T16:57:27.333 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:27.353 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:27 +0000 (0:00:00.090) 0:01:44.939 **** 2021-11-03T16:57:27.421 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2021-11-03T16:57:27.423 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:27.443 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:27 +0000 (0:00:00.090) 0:01:45.030 **** 2021-11-03T16:57:27.512 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2021-11-03T16:57:27.514 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} TASK [common : Set the hardware clock] ***************************************** changed: [smithi115.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.522921", "end": "2021-11-03 16:57:27.500336", "rc": 0, "start": "2021-11-03 16:57:26.977415", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-03T16:57:27.535 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:27 +0000 (0:00:00.091) 0:01:45.122 **** 2021-11-03T16:57:27.556 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:27 +0000 (0:00:00.020) 0:01:45.142 **** 2021-11-03T16:57:27.605 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2021-11-03T16:57:27.607 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:27.632 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:27 +0000 (0:00:00.075) 0:01:45.218 **** 2021-11-03T16:57:27.702 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2021-11-03T16:57:27.704 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:27.726 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:27 +0000 (0:00:00.094) 0:01:45.312 **** 2021-11-03T16:57:27.994 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** ok: [smithi027.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-11-03T16:57:28.015 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:28 +0000 (0:00:00.289) 0:01:45.602 **** 2021-11-03T16:57:28.558 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2021-11-03T16:57:28.560 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'key': 'epel', 'value': {'name': 'Extra Packages for Enterprise Linux', 'mirrorlist': 'file:///etc/yum.repos.d/epel-mirrorlist', 'failovermethod': 'priority', 'enabled': '1', 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "86c0d58bde7ad4cfaafe592d2fefd92c920c42ca", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 236, "state": "file", "uid": 0} 2021-11-03T16:57:29.003 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'key': 'epel-testing', 'value': {'name': 'Extra Packages for Enterprise Linux - Testing', 'mirrorlist': 'file:///etc/yum.repos.d/epel-testing-mirrorlist', 'failovermethod': 'priority', 'enabled': 0, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a1bad5509ed35b895c2061841236b1cfe10619ae", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 262, "state": "file", "uid": 0} 2021-11-03T16:57:29.026 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:29 +0000 (0:00:01.011) 0:01:46.613 **** 2021-11-03T16:57:29.568 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2021-11-03T16:57:29.569 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "3c9ba7bacd1e30bed014cefd1fb4eab0f0f8fa10", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 484, "state": "file", "uid": 0} 2021-11-03T16:57:30.027 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "d542c3e2c30a0d3eb7d454775342c302c0ded15b", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 540, "state": "file", "uid": 0} 2021-11-03T16:57:30.050 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:30 +0000 (0:00:01.023) 0:01:47.637 **** 2021-11-03T16:57:30.120 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2021-11-03T16:57:30.121 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:30.142 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:30 +0000 (0:00:00.091) 0:01:47.728 **** 2021-11-03T16:57:30.210 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-11-03T16:57:30.212 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:30.232 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:30 +0000 (0:00:00.090) 0:01:47.819 **** 2021-11-03T16:57:30.301 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2021-11-03T16:57:30.303 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:30.329 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:30 +0000 (0:00:00.096) 0:01:47.915 **** 2021-11-03T16:57:30.398 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2021-11-03T16:57:30.400 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:30.419 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:30 +0000 (0:00:00.090) 0:01:48.006 **** 2021-11-03T16:57:30.490 INFO:teuthology.task.ansible.out: TASK [common : Mask sleep units] *********************************************** skipping: [smithi027.front.sepia.ceph.com] => (item=sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "sleep.target", "skip_reason": "Conditional result was False"} 2021-11-03T16:57:30.501 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=suspend.target) => {"ansible_loop_var": "item", "changed": false, "item": "suspend.target", "skip_reason": "Conditional result was False"} 2021-11-03T16:57:30.503 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=hibernate.target) => {"ansible_loop_var": "item", "changed": false, "item": "hibernate.target", "skip_reason": "Conditional result was False"} 2021-11-03T16:57:30.512 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=hybrid-sleep.target) => {"ansible_loop_var": "item", "changed": false, "item": "hybrid-sleep.target", "skip_reason": "Conditional result was False"} 2021-11-03T16:57:30.535 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:30 +0000 (0:00:00.115) 0:01:48.122 **** 2021-11-03T16:57:30.603 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2021-11-03T16:57:30.605 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:30.626 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:30 +0000 (0:00:00.090) 0:01:48.212 **** 2021-11-03T16:57:30.696 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:30.718 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:30 +0000 (0:00:00.091) 0:01:48.304 **** 2021-11-03T16:57:30.786 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2021-11-03T16:57:30.788 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:30.810 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:30 +0000 (0:00:00.092) 0:01:48.397 **** 2021-11-03T16:57:30.880 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2021-11-03T16:57:30.882 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:30.903 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:30 +0000 (0:00:00.092) 0:01:48.489 **** 2021-11-03T16:57:30.971 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2021-11-03T16:57:30.973 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:30.993 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:30 +0000 (0:00:00.090) 0:01:48.579 **** 2021-11-03T16:57:31.314 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2021-11-03T16:57:31.316 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": []} 2021-11-03T16:57:31.338 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:31 +0000 (0:00:00.344) 0:01:48.924 **** 2021-11-03T16:57:31.408 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:31.430 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:31 +0000 (0:00:00.092) 0:01:49.017 **** 2021-11-03T16:57:31.489 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2021-11-03T16:57:31.490 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:31.512 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:31 +0000 (0:00:00.082) 0:01:49.099 **** 2021-11-03T16:57:31.522 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2021-11-03T16:57:31.523 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.322917", "end": "2021-11-03 16:57:31.500330", "rc": 0, "start": "2021-11-03 16:57:31.177413", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-03T16:57:31.557 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:31 +0000 (0:00:00.042) 0:01:49.142 **** 2021-11-03T16:57:31.595 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:31.619 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:31 +0000 (0:00:00.063) 0:01:49.206 **** 2021-11-03T16:57:31.690 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2021-11-03T16:57:31.691 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:31.712 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:31 +0000 (0:00:00.092) 0:01:49.298 **** 2021-11-03T16:57:32.196 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2021-11-03T16:57:32.198 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2021-11-03T16:57:32.218 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:32 +0000 (0:00:00.506) 0:01:49.805 **** 2021-11-03T16:57:33.868 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2021-11-03T16:57:33.869 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": []} 2021-11-03T16:57:33.890 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:33 +0000 (0:00:01.671) 0:01:51.477 **** 2021-11-03T16:57:34.420 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** ok: [smithi115.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2021-11-03T16:57:34.866 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2021-11-03T16:57:35.282 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2021-11-03T16:57:35.306 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:35 +0000 (0:00:01.415) 0:01:52.892 **** 2021-11-03T16:57:35.427 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2021-11-03T16:57:35.429 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": []} 2021-11-03T16:57:35.451 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:35 +0000 (0:00:00.145) 0:01:53.038 **** 2021-11-03T16:57:35.490 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2021-11-03T16:57:35.492 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2021-11-03T16:57:35.514 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:35 +0000 (0:00:00.062) 0:01:53.101 **** 2021-11-03T16:57:35.542 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2021-11-03T16:57:35.543 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:35.565 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:35 +0000 (0:00:00.050) 0:01:53.151 **** 2021-11-03T16:57:35.635 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2021-11-03T16:57:35.637 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:35.657 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:35 +0000 (0:00:00.092) 0:01:53.244 **** 2021-11-03T16:57:35.726 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2021-11-03T16:57:35.728 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:35.748 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:35 +0000 (0:00:00.090) 0:01:53.335 **** 2021-11-03T16:57:35.816 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2021-11-03T16:57:35.818 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:35.840 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:35 +0000 (0:00:00.092) 0:01:53.427 **** 2021-11-03T16:57:36.005 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-11-03T16:57:36.007 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 13355, "state": "file", "uid": 0} 2021-11-03T16:57:36.375 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2021-11-03T16:57:36.377 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2021-11-03T16:57:36.398 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:36 +0000 (0:00:00.557) 0:01:53.985 **** 2021-11-03T16:57:36.426 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-11-03T16:57:36.428 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2021-11-03T16:57:36.835 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "2fbc67b36d59f66fc2621f3ca2dcf60dea2167a5", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3612, "state": "file", "uid": 0} 2021-11-03T16:57:36.894 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:36 +0000 (0:00:00.495) 0:01:54.481 **** 2021-11-03T16:57:36.932 INFO:teuthology.task.ansible.out: TASK [common : Include dnf_system vars] **************************************** 2021-11-03T16:57:36.934 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/dnf_systems.yml"], "changed": false} 2021-11-03T16:57:36.954 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:36 +0000 (0:00:00.059) 0:01:54.541 **** 2021-11-03T16:57:38.127 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2021-11-03T16:57:38.129 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": []} 2021-11-03T16:57:38.152 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:38 +0000 (0:00:01.198) 0:01:55.739 **** 2021-11-03T16:57:38.159 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2021-11-03T16:57:38.160 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2021-11-03T16:57:38.188 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:38 +0000 (0:00:00.035) 0:01:55.774 **** 2021-11-03T16:57:38.238 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** skipping: [smithi115.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-11-03T16:57:38.249 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-11-03T16:57:38.260 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-11-03T16:57:38.271 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-11-03T16:57:38.295 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:38 +0000 (0:00:00.107) 0:01:55.882 **** 2021-11-03T16:57:38.647 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** 2021-11-03T16:57:38.649 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2021-11-03T16:57:39.095 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2021-11-03T16:57:39.542 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2021-11-03T16:57:39.567 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:39 +0000 (0:00:01.271) 0:01:57.154 **** 2021-11-03T16:57:39.773 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2021-11-03T16:57:39.775 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2021-11-03T16:57:39.798 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:39 +0000 (0:00:00.230) 0:01:57.384 **** 2021-11-03T16:57:39.929 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2021-11-03T16:57:39.931 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": []} 2021-11-03T16:57:39.958 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:39 +0000 (0:00:00.160) 0:01:57.544 **** 2021-11-03T16:57:39.985 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (non-RHEL/CentOS)] ******** 2021-11-03T16:57:39.987 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:40.007 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:40 +0000 (0:00:00.049) 0:01:57.594 **** 2021-11-03T16:57:40.278 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-11-03T16:57:40.280 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "00d0404b545463be4a65f475e0870e07a48f35b6", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 13355, "state": "file", "uid": 0} 2021-11-03T16:57:40.458 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2021-11-03T16:57:40.460 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2021-11-03T16:57:40.480 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:40 +0000 (0:00:00.473) 0:01:58.067 **** 2021-11-03T16:57:40.706 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-11-03T16:57:40.708 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2021-11-03T16:57:40.949 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2021-11-03T16:57:40.951 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2021-11-03T16:57:40.980 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:40 +0000 (0:00:00.499) 0:01:58.567 **** 2021-11-03T16:57:41.022 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2021-11-03T16:57:41.024 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "DAEMON_OPTS", "skip_reason": "Conditional result was False"} 2021-11-03T16:57:41.034 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "NRPE_OPTS", "skip_reason": "Conditional result was False"} 2021-11-03T16:57:41.057 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:41 +0000 (0:00:00.077) 0:01:58.644 **** 2021-11-03T16:57:41.144 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2021-11-03T16:57:41.146 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "2fbc67b36d59f66fc2621f3ca2dcf60dea2167a5", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3612, "state": "file", "uid": 0} 2021-11-03T16:57:41.170 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:41 +0000 (0:00:00.112) 0:01:58.757 **** 2021-11-03T16:57:41.210 INFO:teuthology.task.ansible.out: TASK [common : Include dnf_system vars] **************************************** 2021-11-03T16:57:41.212 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/dnf_systems.yml"], "changed": false} 2021-11-03T16:57:41.232 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:41 +0000 (0:00:00.061) 0:01:58.819 **** 2021-11-03T16:57:41.260 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2021-11-03T16:57:41.262 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-11-03T16:57:41.284 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:41 +0000 (0:00:00.052) 0:01:58.871 **** 2021-11-03T16:57:41.438 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2021-11-03T16:57:41.440 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2021-11-03T16:57:41.465 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:41 +0000 (0:00:00.180) 0:01:59.052 **** 2021-11-03T16:57:41.504 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2021-11-03T16:57:41.506 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-11-03T16:57:41.527 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-11-03T16:57:41.529 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} TASK [common : Check firewalld status] ***************************************** fatal: [smithi115.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2021-11-03T16:57:41.551 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:41 +0000 (0:00:00.085) 0:01:59.137 **** 2021-11-03T16:57:41.559 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2021-11-03T16:57:41.560 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2021-11-03T16:57:41.583 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:41 +0000 (0:00:00.032) 0:01:59.170 **** 2021-11-03T16:57:41.590 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2021-11-03T16:57:41.592 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:41.615 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:41 +0000 (0:00:00.031) 0:01:59.202 **** 2021-11-03T16:57:42.073 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2021-11-03T16:57:42.075 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "checksum": "aa9150d4582f919d7a8504c01c8d86f05e3eeb5d", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1184, "state": "file", "uid": 0} 2021-11-03T16:57:42.099 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:42 +0000 (0:00:00.484) 0:01:59.686 **** 2021-11-03T16:57:42.731 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2021-11-03T16:57:42.733 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveState": "active", "AllowedCPUs": "", "AllowedMemoryNodes": "", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "DevicePolicy": "auto", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "EnvironmentFiles": "/etc/sysconfig/nrpe (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1254", "ExecMainStartTimestamp": "Wed 2021-11-03 16:55:28 UTC", "ExecMainStartTimestampMonotonic": "19960126", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_SSL_OPT ; ignore_errors=no ; start_time=[Wed 2021-11-03 16:55:28 UTC] ; stop_time=[n/a] ; pid=1254 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FileDescriptorStoreMax": "0", "GID": "984", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "L 2021-11-03T16:57:42.734 INFO:teuthology.task.ansible.out:imitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "262144", "LimitNOFILESoft": "1024", "LimitNPROC": "127245", "LimitNPROCSoft": "127245", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127245", "LimitSIGPENDINGSoft": "127245", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "MainPID": "1254", "MemoryAccounting": "yes", "MemoryCurrent": "700416", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Nice": "0", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "PIDFile": "/run/nrpe/nrpe.pid", "PermissionsStartOnly": "no", "RemainAfterExit": "no", "Restart": "on-abort", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeMaxUSec": "infinity", "SecureBits": "0", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StatusErrno": "0", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "203592", "TimeoutStar 2021-11-03T16:57:42.736 INFO:teuthology.task.ansible.out:tUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Type": "simple", "UID": "988", "UMask": "0022", "WatchdogTimestamp": "Wed 2021-11-03 16:55:28 UTC", "WatchdogTimestampMonotonic": "19960161", "WatchdogUSec": "0"}} 2021-11-03T16:57:42.754 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:42 +0000 (0:00:00.654) 0:02:00.341 **** 2021-11-03T16:57:42.938 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2021-11-03T16:57:42.940 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.001723", "end": "2021-11-03 16:57:42.925765", "rc": 0, "start": "2021-11-03 16:57:42.924042", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2021-11-03T16:57:42.961 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:42 +0000 (0:00:00.206) 0:02:00.547 **** 2021-11-03T16:57:43.307 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2021-11-03T16:57:43.310 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": []} 2021-11-03T16:57:43.330 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:43 +0000 (0:00:00.369) 0:02:00.917 **** 2021-11-03T16:57:43.369 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (non-RHEL/CentOS)] ******** 2021-11-03T16:57:43.371 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:43.396 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:43 +0000 (0:00:00.066) 0:02:00.983 **** 2021-11-03T16:57:43.864 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2021-11-03T16:57:43.865 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2021-11-03T16:57:43.886 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:43 +0000 (0:00:00.489) 0:02:01.473 **** 2021-11-03T16:57:44.362 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2021-11-03T16:57:44.364 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2021-11-03T16:57:44.385 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:44 +0000 (0:00:00.498) 0:02:01.971 **** 2021-11-03T16:57:44.422 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2021-11-03T16:57:44.424 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=DAEMON_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "DAEMON_OPTS", "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item=NRPE_OPTS) => {"ansible_loop_var": "item", "changed": false, "item": "NRPE_OPTS", "skip_reason": "Conditional result was False"} 2021-11-03T16:57:44.444 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:44 +0000 (0:00:00.059) 0:02:02.031 **** 2021-11-03T16:57:44.568 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2021-11-03T16:57:44.569 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": []} 2021-11-03T16:57:44.591 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:44 +0000 (0:00:00.146) 0:02:02.178 **** 2021-11-03T16:57:44.650 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2021-11-03T16:57:44.652 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-11-03T16:57:44.674 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:44 +0000 (0:00:00.082) 0:02:02.261 **** 2021-11-03T16:57:44.920 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** fatal: [smithi027.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2021-11-03T16:57:44.942 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:44 +0000 (0:00:00.267) 0:02:02.528 **** 2021-11-03T16:57:44.979 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2021-11-03T16:57:44.981 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:57:45.007 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:45 +0000 (0:00:00.065) 0:02:02.594 **** 2021-11-03T16:57:45.492 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2021-11-03T16:57:45.493 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "checksum": "aa9150d4582f919d7a8504c01c8d86f05e3eeb5d", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1184, "state": "file", "uid": 0} 2021-11-03T16:57:45.515 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:45 +0000 (0:00:00.507) 0:02:03.101 **** 2021-11-03T16:57:45.836 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveState": "active", "AllowedCPUs": "", "AllowedMemoryNodes": "", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "DevicePolicy": "auto", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "EnvironmentFiles": "/etc/sysconfig/nrpe (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "1246", "ExecMainStartTimestamp": "Wed 2021-11-03 16:55:13 UTC", "ExecMainStartTimestampMonotonic": "23707782", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_SSL_OPT ; ignore_errors=no ; start_time=[Wed 2021-11-03 16:55:13 UTC] ; stop_time=[n/a] ; pid=1246 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FileDescriptorStoreMax": "0", "GID": "984", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "1844674407370955 2021-11-03T16:57:45.837 INFO:teuthology.task.ansible.out:1615", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "262144", "LimitNOFILESoft": "1024", "LimitNPROC": "127269", "LimitNPROCSoft": "127269", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127269", "LimitSIGPENDINGSoft": "127269", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "MainPID": "1246", "MemoryAccounting": "yes", "MemoryCurrent": "667648", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Nice": "0", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "-500", "PIDFile": "/run/nrpe/nrpe.pid", "PermissionsStartOnly": "no", "RemainAfterExit": "no", "Restart": "on-abort", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeMaxUSec": "infinity", "SecureBits": "0", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StatusErrno": "0", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no 2021-11-03T16:57:45.839 INFO:teuthology.task.ansible.out:", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "203631", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min", "TimerSlackNSec": "50000", "Type": "simple", "UID": "988", "UMask": "0022", "WatchdogTimestamp": "Wed 2021-11-03 16:55:13 UTC", "WatchdogTimestampMonotonic": "23708237", "WatchdogUSec": "0"}} 2021-11-03T16:57:45.858 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:45 +0000 (0:00:00.343) 0:02:03.445 **** 2021-11-03T16:57:46.053 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2021-11-03T16:57:46.055 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.001838", "end": "2021-11-03 16:57:46.036313", "rc": 0, "start": "2021-11-03 16:57:46.034475", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2021-11-03T16:57:46.080 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:46 +0000 (0:00:00.221) 0:02:03.667 **** 2021-11-03T16:57:47.778 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2021-11-03T16:57:47.780 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": []} 2021-11-03T16:57:47.800 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:57:47 +0000 (0:00:01.719) 0:02:05.387 **** 2021-11-03T16:58:21.534 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2021-11-03T16:58:21.536 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: selinux-policy-targeted-3.14.3-67.el8_4.2.noarch", "Installed: selinux-policy-3.14.3-67.el8_4.2.noarch", "Removed: selinux-policy-3.14.3-67.el8.noarch", "Removed: selinux-policy-targeted-3.14.3-67.el8.noarch"]} 2021-11-03T16:58:21.559 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:21 +0000 (0:00:33.758) 0:02:39.145 **** 2021-11-03T16:58:22.311 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2021-11-03T16:58:22.313 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2021-11-03T16:58:22.333 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:22 +0000 (0:00:00.774) 0:02:39.920 **** 2021-11-03T16:58:26.536 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2021-11-03T16:58:26.538 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: selinux-policy-targeted-3.14.3-67.el8_4.2.noarch", "Installed: selinux-policy-3.14.3-67.el8_4.2.noarch", "Removed: selinux-policy-3.14.3-67.el8.noarch", "Removed: selinux-policy-targeted-3.14.3-67.el8.noarch"]} 2021-11-03T16:58:26.559 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:26 +0000 (0:00:04.226) 0:02:44.146 **** 2021-11-03T16:58:27.018 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2021-11-03T16:58:27.020 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2021-11-03T16:58:27.040 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:27 +0000 (0:00:00.481) 0:02:44.627 **** 2021-11-03T16:58:37.288 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2021-11-03T16:58:37.290 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:14.762280", "end": "2021-11-03 16:58:37.271777", "failed_when_result": false, "rc": 0, "start": "2021-11-03 16:58:22.509497", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2021-11-03T16:58:37.311 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:37 +0000 (0:00:10.271) 0:02:54.898 **** 2021-11-03T16:58:37.765 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2021-11-03T16:58:37.767 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2021-11-03T16:58:37.787 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:37 +0000 (0:00:00.475) 0:02:55.374 **** 2021-11-03T16:58:37.991 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2021-11-03T16:58:37.993 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.016006", "end": "2021-11-03 16:58:37.976071", "rc": 0, "start": "2021-11-03 16:58:37.960065", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-03T16:58:38.014 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:38 +0000 (0:00:00.226) 0:02:55.601 **** 2021-11-03T16:58:38.219 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2021-11-03T16:58:38.221 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.012717", "end": "2021-11-03 16:58:38.198026", "rc": 0, "start": "2021-11-03 16:58:38.185309", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-03T16:58:38.241 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:38 +0000 (0:00:00.226) 0:02:55.828 **** 2021-11-03T16:58:42.651 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2021-11-03T16:58:42.653 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:15.412638", "end": "2021-11-03 16:58:42.632356", "failed_when_result": false, "rc": 0, "start": "2021-11-03 16:58:27.219718", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2021-11-03T16:58:42.674 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:42 +0000 (0:00:04.433) 0:03:00.261 **** 2021-11-03T16:58:43.150 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** 2021-11-03T16:58:43.152 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2021-11-03T16:58:43.174 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:43 +0000 (0:00:00.499) 0:03:00.761 **** 2021-11-03T16:58:43.393 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2021-11-03T16:58:43.395 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.015772", "end": "2021-11-03 16:58:43.372360", "rc": 0, "start": "2021-11-03 16:58:43.356588", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-03T16:58:43.417 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:43 +0000 (0:00:00.242) 0:03:01.003 **** 2021-11-03T16:58:43.624 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2021-11-03T16:58:43.626 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.012650", "end": "2021-11-03 16:58:43.610861", "rc": 0, "start": "2021-11-03 16:58:43.598211", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-03T16:58:43.648 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:43 +0000 (0:00:00.231) 0:03:01.234 **** 2021-11-03T16:58:53.096 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2021-11-03T16:58:53.098 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:14.660514", "end": "2021-11-03 16:58:53.074689", "rc": 0, "start": "2021-11-03 16:58:38.414175", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-03T16:58:53.120 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:53 +0000 (0:00:09.471) 0:03:10.706 **** 2021-11-03T16:58:53.314 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2021-11-03T16:58:53.316 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2021-11-03T16:58:53.335 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:53 +0000 (0:00:00.215) 0:03:10.922 **** 2021-11-03T16:58:53.362 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** 2021-11-03T16:58:53.364 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi"], "dash_stream": "", "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/dnf_systems.yml"], "changed": false} 2021-11-03T16:58:53.384 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:53 +0000 (0:00:00.048) 0:03:10.971 **** 2021-11-03T16:58:53.422 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2021-11-03T16:58:53.424 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2021-11-03T16:58:53.446 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:53 +0000 (0:00:00.061) 0:03:11.033 **** 2021-11-03T16:58:53.483 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2021-11-03T16:58:53.485 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml) => {"ansible_facts": {"common_yum_repos": {"apache-arrow": {"baseurl": "https://apache.jfrog.io/artifactory/arrow/centos/8/$basearch", "enabled": 1, "gpgcheck": 1, "name": "Apache Arrow"}, "lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "copr_repos": ["ceph/python3-asyncssh"], "epel_packages": [], "nfs_service": "nfs-server", "ntp_service_name": "chronyd", "packages": ["redhat-lsb-core", "dnf-utils", "sysstat", "libedit", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse-libs", "openssl", "libuuid", "podman", "podman-docker", "attr", "ant", "lsof", "gettext", "bc", "xfsdump", "blktrace", "usbredir", "libev-devel", "valgrind", "nfs-utils", "ncurses-devel", "gcc", "git", "python3-nose", "python3-virtualenv", "genisoimage", "qemu-img", "qemu-kvm-core", "qemu-kvm-block-rbd", "libacl-devel", "dbench", "autoconf", "gdb", "iozone"], "packages_to_upgrade": ["libgcrypt"], "yum_mirrorlists": ["CentOS-AppStream-mirrorlist", "CentOS-Base-mirrorlist", "CentOS-Extras-mirrorlist", "CentOS-PowerTools-mirrorlist"], "yum_repos": {"CentOS-AppStream": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist", "name": "CentOS-$releasever - AppStream"}, "CentOS-Base": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Base-mirrorlist", "name": "CentOS-$releasever - Base"}, "CentOS-Extras": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Extras-mirrorlist", "name": "CentOS-$releasever - Extras"}, "CentOS-PowerTools": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/R 2021-11-03T16:58:53.487 INFO:teuthology.task.ansible.out:PM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-PowerTools-mirrorlist", "name": "CentOS-$releasever - PowerTools"}}}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml"} 2021-11-03T16:58:53.507 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:53 +0000 (0:00:00.060) 0:03:11.093 **** 2021-11-03T16:58:53.544 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2021-11-03T16:58:53.545 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2021-11-03T16:58:53.565 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:53 +0000 (0:00:00.058) 0:03:11.152 **** 2021-11-03T16:58:53.748 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2021-11-03T16:58:53.750 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2021-11-03T16:58:53.772 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:53 +0000 (0:00:00.206) 0:03:11.358 **** 2021-11-03T16:58:53.955 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2021-11-03T16:58:53.957 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "gid": 1254, "name": "ubuntu", "state": "present", "system": false} 2021-11-03T16:58:53.977 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:53 +0000 (0:00:00.205) 0:03:11.564 **** 2021-11-03T16:58:54.004 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2021-11-03T16:58:54.005 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:58:54.026 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:54 +0000 (0:00:00.048) 0:03:11.613 **** 2021-11-03T16:58:54.052 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2021-11-03T16:58:54.053 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:58:54.073 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:54 +0000 (0:00:00.047) 0:03:11.660 **** 2021-11-03T16:58:54.288 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2021-11-03T16:58:54.289 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2021-11-03T16:58:54.310 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:54 +0000 (0:00:00.236) 0:03:11.896 **** 2021-11-03T16:58:54.833 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2021-11-03T16:58:54.835 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2021-11-03T16:58:54.855 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:54 +0000 (0:00:00.545) 0:03:12.442 **** 2021-11-03T16:58:54.882 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2021-11-03T16:58:54.884 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi115"}, "changed": false} 2021-11-03T16:58:54.904 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:54 +0000 (0:00:00.048) 0:03:12.491 **** 2021-11-03T16:58:55.945 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2021-11-03T16:58:55.948 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi115.front.sepia.ceph.com", "ansible_hostname": "smithi115", "ansible_nodename": "smithi115"}, "changed": true, "name": "smithi115"} 2021-11-03T16:58:55.967 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:55 +0000 (0:00:01.062) 0:03:13.554 **** 2021-11-03T16:58:56.426 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2021-11-03T16:58:56.428 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "checksum": "a5e550b2aa74f6355e926d70d73a50ad9ab4b4be", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1043, "state": "file", "uid": 0} 2021-11-03T16:58:56.448 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:56 +0000 (0:00:00.480) 0:03:14.034 **** 2021-11-03T16:58:56.896 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2021-11-03T16:58:56.898 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2021-11-03T16:58:56.919 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:56 +0000 (0:00:00.471) 0:03:14.505 **** 2021-11-03T16:58:57.453 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2021-11-03T16:58:57.455 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2021-11-03T16:58:57.475 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:57 +0000 (0:00:00.556) 0:03:15.062 **** 2021-11-03T16:58:57.952 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2021-11-03T16:58:57.954 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2021-11-03T16:58:57.984 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:57 +0000 (0:00:00.509) 0:03:15.571 **** 2021-11-03T16:58:58.179 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-11-03T16:58:58.181 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2021-11-03T16:58:58.201 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:58 +0000 (0:00:00.216) 0:03:15.788 **** 2021-11-03T16:58:58.394 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-11-03T16:58:58.396 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "gid": 36, "name": "kvm", "state": "present", "system": false} 2021-11-03T16:58:58.416 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:58 +0000 (0:00:00.214) 0:03:16.002 **** 2021-11-03T16:58:58.631 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-11-03T16:58:58.632 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1254, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2021-11-03T16:58:58.655 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:58 +0000 (0:00:00.239) 0:03:16.242 **** 2021-11-03T16:58:59.110 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-11-03T16:58:59.112 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2021-11-03T16:58:59.132 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:59 +0000 (0:00:00.476) 0:03:16.718 **** 2021-11-03T16:58:59.316 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** 2021-11-03T16:58:59.317 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:15.466296", "end": "2021-11-03 16:58:59.295834", "rc": 0, "start": "2021-11-03 16:58:43.829538", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-03T16:58:59.340 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:59 +0000 (0:00:00.207) 0:03:16.926 **** 2021-11-03T16:58:59.546 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** 2021-11-03T16:58:59.548 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2021-11-03T16:58:59.569 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:59 +0000 (0:00:00.229) 0:03:17.156 **** 2021-11-03T16:58:59.600 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} TASK [testnode : Include package type specific vars.] ************************** ok: [smithi027.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi"], "dash_stream": "", "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/dnf_systems.yml"], "changed": false} 2021-11-03T16:58:59.632 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:59 +0000 (0:00:00.062) 0:03:17.219 **** 2021-11-03T16:58:59.655 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:59 +0000 (0:00:00.022) 0:03:17.241 **** 2021-11-03T16:58:59.672 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2021-11-03T16:58:59.674 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2021-11-03T16:58:59.697 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:59 +0000 (0:00:00.041) 0:03:17.283 **** 2021-11-03T16:58:59.736 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2021-11-03T16:58:59.738 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml) => {"ansible_facts": {"common_yum_repos": {"apache-arrow": {"baseurl": "https://apache.jfrog.io/artifactory/arrow/centos/8/$basearch", "enabled": 1, "gpgcheck": 1, "name": "Apache Arrow"}, "lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "copr_repos": ["ceph/python3-asyncssh"], "epel_packages": [], "nfs_service": "nfs-server", "ntp_service_name": "chronyd", "packages": ["redhat-lsb-core", "dnf-utils", "sysstat", "libedit", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse-libs", "openssl", "libuuid", "podman", "podman-docker", "attr", "ant", "lsof", "gettext", "bc", "xfsdump", "blktrace", "usbredir", "libev-devel", "valgrind", "nfs-utils", "ncurses-devel", "gcc", "git", "python3-nose", "python3-virtualenv", "genisoimage", "qemu-img", "qemu-kvm-core", "qemu-kvm-block-rbd", "libacl-devel", "dbench", "autoconf", "gdb", "iozone"], "packages_to_upgrade": ["libgcrypt"], "yum_mirrorlists": ["CentOS-AppStream-mirrorlist", "CentOS-Base-mirrorlist", "CentOS-Extras-mirrorlist", "CentOS-PowerTools-mirrorlist"], "yum_repos": {"CentOS-AppStream": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist", "name": "CentOS-$releasever - AppStream"}, "CentOS-Base": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Base-mirrorlist", "name": "CentOS-$releasever - Base"}, "CentOS-Extras": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Extras-mirrorlist", "name": "CentOS-$releasever - Extras"}, "CentOS-PowerTools": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/R 2021-11-03T16:58:59.740 INFO:teuthology.task.ansible.out:PM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-PowerTools-mirrorlist", "name": "CentOS-$releasever - PowerTools"}}}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8.yml"} 2021-11-03T16:58:59.762 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:59 +0000 (0:00:00.065) 0:03:17.348 **** 2021-11-03T16:58:59.803 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2021-11-03T16:58:59.805 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2021-11-03T16:58:59.834 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:58:59 +0000 (0:00:00.072) 0:03:17.420 **** 2021-11-03T16:59:00.029 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2021-11-03T16:59:00.031 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2021-11-03T16:59:00.050 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:00 +0000 (0:00:00.216) 0:03:17.637 **** 2021-11-03T16:59:00.253 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2021-11-03T16:59:00.255 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "gid": 1254, "name": "ubuntu", "state": "present", "system": false} 2021-11-03T16:59:00.275 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:00 +0000 (0:00:00.224) 0:03:17.862 **** 2021-11-03T16:59:00.303 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2021-11-03T16:59:00.305 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:59:00.325 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:00 +0000 (0:00:00.049) 0:03:17.911 **** 2021-11-03T16:59:00.352 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:59:00.374 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:00 +0000 (0:00:00.049) 0:03:17.960 **** 2021-11-03T16:59:00.606 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** ok: [smithi027.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2021-11-03T16:59:00.628 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:00 +0000 (0:00:00.254) 0:03:18.215 **** 2021-11-03T16:59:01.179 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2021-11-03T16:59:01.181 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2021-11-03T16:59:01.201 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:01 +0000 (0:00:00.572) 0:03:18.788 **** 2021-11-03T16:59:01.239 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2021-11-03T16:59:01.241 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi027"}, "changed": false} 2021-11-03T16:59:01.262 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:01 +0000 (0:00:00.060) 0:03:18.848 **** 2021-11-03T16:59:01.880 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ 2021-11-03T16:59:01.882 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi027.front.sepia.ceph.com", "ansible_hostname": "smithi027", "ansible_nodename": "smithi027"}, "changed": true, "name": "smithi027"} 2021-11-03T16:59:01.901 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:01 +0000 (0:00:00.639) 0:03:19.488 **** 2021-11-03T16:59:02.074 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-11-03T16:59:02.076 INFO:teuthology.task.ansible.out:[WARNING]: Consider using the yum, dnf or zypper module rather than running 'rpm'. If you need to use command because yum, dnf or zypper is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [smithi115.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:02.222189", "end": "2021-11-03 16:59:02.048844", "rc": 0, "start": "2021-11-03 16:58:59.826655", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-03T16:59:02.099 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:02 +0000 (0:00:00.197) 0:03:19.685 **** 2021-11-03T16:59:02.136 INFO:teuthology.task.ansible.out: TASK [testnode : Clean up Stream distro-provided repos] ************************ 2021-11-03T16:59:02.138 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:59:02.158 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:02 +0000 (0:00:00.059) 0:03:19.745 **** 2021-11-03T16:59:02.195 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-11-03T16:59:02.197 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:59:02.218 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:02 +0000 (0:00:00.059) 0:03:19.805 **** 2021-11-03T16:59:02.266 INFO:teuthology.task.ansible.out: TASK [testnode : Include CentOS Stream specific variables] ********************* 2021-11-03T16:59:02.268 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8_stream.yml) => {"ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8_stream.yml", "skip_reason": "Conditional result was False"} 2021-11-03T16:59:02.289 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:02 +0000 (0:00:00.070) 0:03:19.876 **** 2021-11-03T16:59:02.358 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2021-11-03T16:59:02.360 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "checksum": "a5e550b2aa74f6355e926d70d73a50ad9ab4b4be", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1043, "state": "file", "uid": 0} 2021-11-03T16:59:02.382 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:02 +0000 (0:00:00.092) 0:03:19.969 **** 2021-11-03T16:59:02.764 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2021-11-03T16:59:02.766 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=CentOS-AppStream-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "aa5ad73629677ef17172417e50d2cdadccb7a3f2", "dest": "/etc/yum.repos.d/CentOS-AppStream-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-AppStream-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-AppStream-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 397, "state": "file", "uid": 0} 2021-11-03T16:59:02.838 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2021-11-03T16:59:02.840 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2021-11-03T16:59:02.863 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:02 +0000 (0:00:00.480) 0:03:20.449 **** 2021-11-03T16:59:03.183 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2021-11-03T16:59:03.185 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=CentOS-Base-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "4ed3c0272e6c4ec99169cd68f0b43d40bdd4b296", "dest": "/etc/yum.repos.d/CentOS-Base-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-Base-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Base-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 380, "state": "file", "uid": 0} 2021-11-03T16:59:03.416 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2021-11-03T16:59:03.418 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2021-11-03T16:59:03.439 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:03 +0000 (0:00:00.576) 0:03:21.026 **** 2021-11-03T16:59:03.623 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2021-11-03T16:59:03.625 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=CentOS-Extras-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "a862f43d3b3c7b8b592aa7dce23a6d013ab20bc3", "dest": "/etc/yum.repos.d/CentOS-Extras-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-Extras-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Extras-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 382, "state": "file", "uid": 0} 2021-11-03T16:59:03.927 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2021-11-03T16:59:03.929 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2021-11-03T16:59:03.956 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:03 +0000 (0:00:00.516) 0:03:21.543 **** 2021-11-03T16:59:04.050 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2021-11-03T16:59:04.052 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=CentOS-PowerTools-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "20107eac0ba9a84b318fa5f29b02a387051bff98", "dest": "/etc/yum.repos.d/CentOS-PowerTools-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-PowerTools-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-PowerTools-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 397, "state": "file", "uid": 0} 2021-11-03T16:59:04.077 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:04 +0000 (0:00:00.120) 0:03:21.663 **** 2021-11-03T16:59:04.158 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-11-03T16:59:04.159 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2021-11-03T16:59:04.181 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:04 +0000 (0:00:00.104) 0:03:21.768 **** 2021-11-03T16:59:04.376 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-11-03T16:59:04.378 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "gid": 36, "name": "kvm", "state": "present", "system": false} 2021-11-03T16:59:04.398 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:04 +0000 (0:00:00.216) 0:03:21.984 **** 2021-11-03T16:59:04.541 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2021-11-03T16:59:04.542 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'key': 'lab-extras', 'value': {'name': 'lab-extras', 'baseurl': 'http://apt-mirror.front.sepia.ceph.com/lab-extras/8/', 'enabled': 1, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a0e2ac0628a264e14b96b1c57627a3aa344a6c52", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 203, "state": "file", "uid": 0} 2021-11-03T16:59:04.625 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-11-03T16:59:04.626 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1254, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2021-11-03T16:59:04.647 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:04 +0000 (0:00:00.249) 0:03:22.233 **** 2021-11-03T16:59:05.018 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2021-11-03T16:59:05.020 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item={'key': 'apache-arrow', 'value': {'name': 'Apache Arrow', 'baseurl': 'https://apache.jfrog.io/artifactory/arrow/centos/8/$basearch', 'enabled': 1, 'gpgcheck': 1}}) => {"ansible_loop_var": "item", "changed": true, "checksum": "86b924db6e383ddad921a207c09a708e6cca6a67", "dest": "/etc/yum.repos.d/apache-arrow.repo", "gid": 0, "group": "root", "item": {"key": "apache-arrow", "value": {"baseurl": "https://apache.jfrog.io/artifactory/arrow/centos/8/$basearch", "enabled": 1, "gpgcheck": 1, "name": "Apache Arrow"}}, "md5sum": "183d0889f7213a1e38c7f48965d7549e", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 215, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1635958744.5746233-17680-205137415944086/source", "state": "file", "uid": 0} 2021-11-03T16:59:05.043 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:05 +0000 (0:00:00.395) 0:03:22.629 **** 2021-11-03T16:59:05.133 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-11-03T16:59:05.135 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2021-11-03T16:59:05.157 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:05 +0000 (0:00:00.114) 0:03:22.744 **** 2021-11-03T16:59:05.521 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2021-11-03T16:59:05.523 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'key': 'CentOS-AppStream', 'value': {'name': 'CentOS-$releasever - AppStream', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "9b147c879e95cd23f3bbde027da102febcaad1a8", "dest": "/etc/yum.repos.d/CentOS-AppStream.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-AppStream", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist", "name": "CentOS-$releasever - AppStream"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-AppStream.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 289, "state": "file", "uid": 0} 2021-11-03T16:59:05.638 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-11-03T16:59:05.640 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2021-11-03T16:59:05.663 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:05 +0000 (0:00:00.506) 0:03:23.250 **** 2021-11-03T16:59:05.953 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2021-11-03T16:59:05.955 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'key': 'CentOS-Base', 'value': {'name': 'CentOS-$releasever - Base', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-Base-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "77e4590ed0ecec056fa44f4e65af60e110822b85", "dest": "/etc/yum.repos.d/CentOS-Base.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-Base", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Base-mirrorlist", "name": "CentOS-$releasever - Base"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Base.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 274, "state": "file", "uid": 0} 2021-11-03T16:59:06.390 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'key': 'CentOS-Extras', 'value': {'name': 'CentOS-$releasever - Extras', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-Extras-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "2856826614c7f1cb9409228f6c41319a023ca754", "dest": "/etc/yum.repos.d/CentOS-Extras.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-Extras", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Extras-mirrorlist", "name": "CentOS-$releasever - Extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 280, "state": "file", "uid": 0} 2021-11-03T16:59:06.818 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'key': 'CentOS-PowerTools', 'value': {'name': 'CentOS-$releasever - PowerTools', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-PowerTools-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "76441d1028936932e965a46c12ad393f767201d3", "dest": "/etc/yum.repos.d/CentOS-PowerTools.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-PowerTools", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-PowerTools-mirrorlist", "name": "CentOS-$releasever - PowerTools"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-PowerTools.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 292, "state": "file", "uid": 0} 2021-11-03T16:59:06.844 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:06 +0000 (0:00:01.180) 0:03:24.431 **** 2021-11-03T16:59:08.315 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-11-03T16:59:08.317 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:02.454880", "end": "2021-11-03 16:59:08.296325", "rc": 0, "start": "2021-11-03 16:59:05.841445", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-03T16:59:08.338 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:08 +0000 (0:00:01.494) 0:03:25.925 **** 2021-11-03T16:59:08.365 INFO:teuthology.task.ansible.out: TASK [testnode : Clean up Stream distro-provided repos] ************************ 2021-11-03T16:59:08.367 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:59:08.388 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:08 +0000 (0:00:00.048) 0:03:25.974 **** 2021-11-03T16:59:08.415 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-11-03T16:59:08.417 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:59:08.439 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:08 +0000 (0:00:00.051) 0:03:26.025 **** 2021-11-03T16:59:08.478 INFO:teuthology.task.ansible.out: TASK [testnode : Include CentOS Stream specific variables] ********************* 2021-11-03T16:59:08.479 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8_stream.yml) => {"ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_8_stream.yml", "skip_reason": "Conditional result was False"} 2021-11-03T16:59:08.502 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:08 +0000 (0:00:00.063) 0:03:26.088 **** 2021-11-03T16:59:08.998 INFO:teuthology.task.ansible.out: TASK [testnode : Configure local mirrorlists] ********************************** 2021-11-03T16:59:09.000 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=CentOS-AppStream-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "aa5ad73629677ef17172417e50d2cdadccb7a3f2", "dest": "/etc/yum.repos.d/CentOS-AppStream-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-AppStream-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-AppStream-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 397, "state": "file", "uid": 0} 2021-11-03T16:59:09.437 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=CentOS-Base-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "4ed3c0272e6c4ec99169cd68f0b43d40bdd4b296", "dest": "/etc/yum.repos.d/CentOS-Base-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-Base-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Base-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 380, "state": "file", "uid": 0} 2021-11-03T16:59:09.885 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=CentOS-Extras-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "a862f43d3b3c7b8b592aa7dce23a6d013ab20bc3", "dest": "/etc/yum.repos.d/CentOS-Extras-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-Extras-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Extras-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 382, "state": "file", "uid": 0} 2021-11-03T16:59:10.322 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=CentOS-PowerTools-mirrorlist) => {"ansible_loop_var": "item", "changed": false, "checksum": "20107eac0ba9a84b318fa5f29b02a387051bff98", "dest": "/etc/yum.repos.d/CentOS-PowerTools-mirrorlist", "gid": 0, "group": "root", "item": "CentOS-PowerTools-mirrorlist", "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-PowerTools-mirrorlist", "secontext": "system_u:object_r:system_conf_t:s0", "size": 397, "state": "file", "uid": 0} 2021-11-03T16:59:10.346 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:10 +0000 (0:00:01.844) 0:03:27.933 **** 2021-11-03T16:59:10.830 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2021-11-03T16:59:10.832 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'key': 'lab-extras', 'value': {'name': 'lab-extras', 'baseurl': 'http://apt-mirror.front.sepia.ceph.com/lab-extras/8/', 'enabled': 1, 'gpgcheck': 0}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "a0e2ac0628a264e14b96b1c57627a3aa344a6c52", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/8/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 203, "state": "file", "uid": 0} 2021-11-03T16:59:11.323 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'apache-arrow', 'value': {'name': 'Apache Arrow', 'baseurl': 'https://apache.jfrog.io/artifactory/arrow/centos/8/$basearch', 'enabled': 1, 'gpgcheck': 1}}) => {"ansible_loop_var": "item", "changed": true, "checksum": "86b924db6e383ddad921a207c09a708e6cca6a67", "dest": "/etc/yum.repos.d/apache-arrow.repo", "gid": 0, "group": "root", "item": {"key": "apache-arrow", "value": {"baseurl": "https://apache.jfrog.io/artifactory/arrow/centos/8/$basearch", "enabled": 1, "gpgcheck": 1, "name": "Apache Arrow"}}, "md5sum": "183d0889f7213a1e38c7f48965d7549e", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 215, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1635958750.8606496-18160-236176365228891/source", "state": "file", "uid": 0} 2021-11-03T16:59:11.347 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:11 +0000 (0:00:01.000) 0:03:28.934 **** 2021-11-03T16:59:11.842 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2021-11-03T16:59:11.844 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'key': 'CentOS-AppStream', 'value': {'name': 'CentOS-$releasever - AppStream', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "9b147c879e95cd23f3bbde027da102febcaad1a8", "dest": "/etc/yum.repos.d/CentOS-AppStream.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-AppStream", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-AppStream-mirrorlist", "name": "CentOS-$releasever - AppStream"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-AppStream.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 289, "state": "file", "uid": 0} 2021-11-03T16:59:12.279 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'key': 'CentOS-Base', 'value': {'name': 'CentOS-$releasever - Base', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-Base-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "77e4590ed0ecec056fa44f4e65af60e110822b85", "dest": "/etc/yum.repos.d/CentOS-Base.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-Base", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Base-mirrorlist", "name": "CentOS-$releasever - Base"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Base.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 274, "state": "file", "uid": 0} 2021-11-03T16:59:12.725 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'key': 'CentOS-Extras', 'value': {'name': 'CentOS-$releasever - Extras', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-Extras-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "2856826614c7f1cb9409228f6c41319a023ca754", "dest": "/etc/yum.repos.d/CentOS-Extras.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-Extras", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-Extras-mirrorlist", "name": "CentOS-$releasever - Extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-Extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 280, "state": "file", "uid": 0} 2021-11-03T16:59:13.174 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'key': 'CentOS-PowerTools', 'value': {'name': 'CentOS-$releasever - PowerTools', 'mirrorlist': 'file:///etc/yum.repos.d/CentOS-PowerTools-mirrorlist', 'gpgcheck': 1, 'enabled': 1, 'gpgkey': 'file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial'}}) => {"ansible_loop_var": "item", "changed": false, "checksum": "76441d1028936932e965a46c12ad393f767201d3", "dest": "/etc/yum.repos.d/CentOS-PowerTools.repo", "gid": 0, "group": "root", "item": {"key": "CentOS-PowerTools", "value": {"enabled": 1, "gpgcheck": 1, "gpgkey": "file:///etc/pki/rpm-gpg/RPM-GPG-KEY-centosofficial", "mirrorlist": "file:///etc/yum.repos.d/CentOS-PowerTools-mirrorlist", "name": "CentOS-$releasever - PowerTools"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/CentOS-PowerTools.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 292, "state": "file", "uid": 0} 2021-11-03T16:59:13.199 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:13 +0000 (0:00:01.851) 0:03:30.785 **** 2021-11-03T16:59:37.407 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2021-11-03T16:59:37.408 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item=ceph/python3-asyncssh) => {"ansible_loop_var": "item", "changed": true, "cmd": ["dnf", "-y", "copr", "enable", "ceph/python3-asyncssh"], "delta": "0:00:30.356165", "end": "2021-11-03 16:59:37.382848", "item": "ceph/python3-asyncssh", "rc": 0, "start": "2021-11-03 16:59:07.026683", "stderr": "Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.", "stderr_lines": ["Enabling a Copr repository. Please note that this repository is not part", "of the main distribution, and quality may vary.", "", "The Fedora Project does not exercise any power over the contents of", "this repository beyond the rules outlined in the Copr FAQ at", ",", "and packages are not held to any quality or security level.", "", "Please do not file bug reports about these packages in Fedora", "Bugzilla. In case of problems, contact the owner of this repository."], "stdout": "Repository successfully enabled.", "stdout_lines": ["Repository successfully enabled."], "warnings": ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]} [WARNING]: Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'comma 2021-11-03T16:59:37.411 INFO:teuthology.task.ansible.out:nd_warnings=False' in ansible.cfg to get rid of this message. 2021-11-03T16:59:37.431 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:37 +0000 (0:00:24.232) 0:03:55.018 **** 2021-11-03T16:59:40.361 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2021-11-03T16:59:40.363 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": []} 2021-11-03T16:59:40.384 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:40 +0000 (0:00:02.952) 0:03:57.970 **** 2021-11-03T16:59:41.223 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2021-11-03T16:59:41.225 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": true, "item": "https://download.ceph.com/keys/release.asc"} 2021-11-03T16:59:41.667 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": true, "item": "https://download.ceph.com/keys/autobuild.asc"} 2021-11-03T16:59:43.023 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item=https://dist.apache.org/repos/dist/dev/arrow/KEYS) => {"ansible_loop_var": "item", "changed": true, "item": "https://dist.apache.org/repos/dist/dev/arrow/KEYS"} 2021-11-03T16:59:43.045 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:43 +0000 (0:00:02.660) 0:04:00.631 **** 2021-11-03T16:59:43.439 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2021-11-03T16:59:43.441 INFO:teuthology.task.ansible.out:[WARNING]: Consider using the yum module rather than running 'yum'. If you need to use command because yum is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [smithi115.front.sepia.ceph.com] => {"changed": true, "cmd": "yum clean all", "delta": "0:00:00.184055", "end": "2021-11-03 16:59:43.417871", "rc": 0, "start": "2021-11-03 16:59:43.233816", "stderr": "", "stderr_lines": [], "stdout": "78 files removed", "stdout_lines": ["78 files removed"]} 2021-11-03T16:59:43.464 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:43 +0000 (0:00:00.419) 0:04:01.051 **** 2021-11-03T16:59:43.502 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-11-03T16:59:43.504 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"ansible_facts": {"skip_packaging": false}, "changed": false} 2021-11-03T16:59:43.525 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:43 +0000 (0:00:00.060) 0:04:01.112 **** 2021-11-03T16:59:43.563 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2021-11-03T16:59:43.565 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:59:43.586 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:43 +0000 (0:00:00.061) 0:04:01.173 **** 2021-11-03T16:59:43.624 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:59:43.647 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:43 +0000 (0:00:00.061) 0:04:01.234 **** 2021-11-03T16:59:43.853 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2021-11-03T16:59:43.855 INFO:teuthology.task.ansible.out:fatal: [smithi115.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.007551", "end": "2021-11-03 16:59:43.834994", "msg": "non-zero return code", "rc": 1, "start": "2021-11-03 16:59:43.827443", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2021-11-03T16:59:43.877 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:43 +0000 (0:00:00.229) 0:04:01.464 **** 2021-11-03T16:59:43.915 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2021-11-03T16:59:43.919 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T16:59:43.938 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 16:59:43 +0000 (0:00:00.060) 0:04:01.524 **** 2021-11-03T17:00:01.427 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-11-03T17:00:01.429 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: librados2-1:12.2.7-9.el8.x86_64", "Removed: librbd1-1:12.2.7-9.el8.x86_64", "Removed: qemu-kvm-block-rbd-15:4.2.0-48.module_el8.4.0+783+f8734d30.x86_64"]} 2021-11-03T17:00:01.449 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:01 +0000 (0:00:17.511) 0:04:19.036 **** 2021-11-03T17:00:04.129 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* changed: [smithi115.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: boost-program-options-1.66.0-10.el8.x86_64", "Removed: boost-random-1.66.0-10.el8.x86_64"]} 2021-11-03T17:00:04.153 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:04 +0000 (0:00:02.702) 0:04:21.738 **** 2021-11-03T17:00:08.669 INFO:teuthology.task.ansible.out: TASK [testnode : Enable copr repos] ******************************************** 2021-11-03T17:00:08.671 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item=ceph/python3-asyncssh) => {"ansible_loop_var": "item", "changed": true, "cmd": ["dnf", "-y", "copr", "enable", "ceph/python3-asyncssh"], "delta": "0:00:55.263268", "end": "2021-11-03 17:00:08.650257", "item": "ceph/python3-asyncssh", "rc": 0, "start": "2021-11-03 16:59:13.386989", "stderr": "Enabling a Copr repository. Please note that this repository is not part\nof the main distribution, and quality may vary.\n\nThe Fedora Project does not exercise any power over the contents of\nthis repository beyond the rules outlined in the Copr FAQ at\n,\nand packages are not held to any quality or security level.\n\nPlease do not file bug reports about these packages in Fedora\nBugzilla. In case of problems, contact the owner of this repository.", "stderr_lines": ["Enabling a Copr repository. Please note that this repository is not part", "of the main distribution, and quality may vary.", "", "The Fedora Project does not exercise any power over the contents of", "this repository beyond the rules outlined in the Copr FAQ at", ",", "and packages are not held to any quality or security level.", "", "Please do not file bug reports about these packages in Fedora", "Bugzilla. In case of problems, contact the owner of this repository."], "stdout": "Repository successfully enabled.", "stdout_lines": ["Repository successfully enabled."], "warnings": ["Consider using the dnf module rather than running 'dnf'. If you need to use command because dnf is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message."]} 2021-11-03T17:00:08.693 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:08 +0000 (0:00:04.541) 0:04:26.280 **** 2021-11-03T17:00:11.193 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-11-03T17:00:11.196 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: qemu-kvm-core-15:4.2.0-48.module_el8.4.0+885+5e18b468.3.x86_64", "Installed: qemu-img-15:4.2.0-48.module_el8.4.0+885+5e18b468.3.x86_64", "Installed: librados2-1:12.2.7-9.el8.x86_64", "Installed: qemu-kvm-block-rbd-15:4.2.0-48.module_el8.4.0+885+5e18b468.3.x86_64", "Installed: boost-program-options-1.66.0-10.el8.x86_64", "Installed: librbd1-1:12.2.7-9.el8.x86_64", "Installed: boost-random-1.66.0-10.el8.x86_64", "Installed: qemu-kvm-common-15:4.2.0-48.module_el8.4.0+885+5e18b468.3.x86_64", "Removed: qemu-kvm-core-15:4.2.0-48.module_el8.4.0+783+f8734d30.x86_64", "Removed: qemu-img-15:4.2.0-48.module_el8.4.0+783+f8734d30.x86_64", "Removed: qemu-kvm-common-15:4.2.0-48.module_el8.4.0+783+f8734d30.x86_64"]} 2021-11-03T17:00:11.237 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:11 +0000 (0:00:02.532) 0:04:28.813 **** 2021-11-03T17:00:11.266 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:11.286 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:11 +0000 (0:00:00.060) 0:04:28.873 **** 2021-11-03T17:00:11.742 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2021-11-03T17:00:11.744 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": []} 2021-11-03T17:00:11.767 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:11 +0000 (0:00:00.480) 0:04:29.354 **** 2021-11-03T17:00:12.287 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2021-11-03T17:00:12.289 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": true, "item": "https://download.ceph.com/keys/release.asc"} 2021-11-03T17:00:12.744 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": true, "item": "https://download.ceph.com/keys/autobuild.asc"} 2021-11-03T17:00:12.983 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-11-03T17:00:12.985 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": []} 2021-11-03T17:00:13.007 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:13 +0000 (0:00:01.239) 0:04:30.594 **** 2021-11-03T17:00:14.129 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2021-11-03T17:00:14.130 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item=https://dist.apache.org/repos/dist/dev/arrow/KEYS) => {"ansible_loop_var": "item", "changed": true, "item": "https://dist.apache.org/repos/dist/dev/arrow/KEYS"} 2021-11-03T17:00:14.153 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:14 +0000 (0:00:01.145) 0:04:31.739 **** 2021-11-03T17:00:14.567 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2021-11-03T17:00:14.569 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "cmd": "yum clean all", "delta": "0:00:00.198891", "end": "2021-11-03 17:00:14.549560", "rc": 0, "start": "2021-11-03 17:00:14.350669", "stderr": "", "stderr_lines": [], "stdout": "78 files removed", "stdout_lines": ["78 files removed"]} 2021-11-03T17:00:14.590 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:14 +0000 (0:00:00.437) 0:04:32.177 **** 2021-11-03T17:00:14.628 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-11-03T17:00:14.630 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"ansible_facts": {"skip_packaging": false}, "changed": false} 2021-11-03T17:00:14.650 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:14 +0000 (0:00:00.059) 0:04:32.237 **** 2021-11-03T17:00:14.658 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-11-03T17:00:14.659 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": []} 2021-11-03T17:00:14.682 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:14 +0000 (0:00:00.031) 0:04:32.268 **** 2021-11-03T17:00:14.690 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2021-11-03T17:00:14.691 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:14.719 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:14 +0000 (0:00:00.037) 0:04:32.305 **** 2021-11-03T17:00:14.761 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2021-11-03T17:00:14.763 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:14.787 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:14 +0000 (0:00:00.067) 0:04:32.373 **** 2021-11-03T17:00:14.995 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2021-11-03T17:00:14.997 INFO:teuthology.task.ansible.out:fatal: [smithi027.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.007806", "end": "2021-11-03 17:00:14.978557", "msg": "non-zero return code", "rc": 1, "start": "2021-11-03 17:00:14.970751", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring TASK [testnode : Stop and disable firewalld] *********************************** 2021-11-03T17:00:14.997 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveState": "inactive", "AllowedCPUs": "", "AllowedMemoryNodes": "", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUAffinity": "", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "ControlPID": "0", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "DevicePolicy": "auto", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "EnvironmentFiles": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FileDescriptorStoreMax": "0", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounti 2021-11-03T17:00:15.000 INFO:teuthology.task.ansible.out:ng": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "262144", "LimitNOFILESoft": "1024", "LimitNPROC": "127245", "LimitNPROCSoft": "127245", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127245", "LimitSIGPENDINGSoft": "127245", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "MainPID": "0", "MemoryAccounting": "yes", "MemoryCurrent": "[not set]", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Nice": "0", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "PermissionsStartOnly": "no", "RemainAfterExit": "no", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeMaxUSec": "infinity", "SecureBits": "0", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "null", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StatusErrno": "0", "SyslogFacility": "3", "Syslog 2021-11-03T17:00:15.001 INFO:teuthology.task.ansible.out:Level": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "203592", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Type": "dbus", "UID": "[not set]", "UMask": "0022", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-11-03T17:00:15.019 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:15 +0000 (0:00:00.232) 0:04:32.606 **** 2021-11-03T17:00:15.021 INFO:teuthology.task.ansible.out: 2021-11-03T17:00:15.041 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:15 +0000 (0:00:00.021) 0:04:32.628 **** 2021-11-03T17:00:15.043 INFO:teuthology.task.ansible.out: 2021-11-03T17:00:15.060 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** 2021-11-03T17:00:15.061 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:15.091 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:15 +0000 (0:00:00.050) 0:04:32.678 **** 2021-11-03T17:00:15.340 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2021-11-03T17:00:15.342 INFO:teuthology.task.ansible.out:fatal: [smithi115.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2021-11-03T17:00:15.363 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:15 +0000 (0:00:00.272) 0:04:32.950 **** 2021-11-03T17:00:15.992 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2021-11-03T17:00:15.993 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2021-11-03T17:00:16.014 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:16 +0000 (0:00:00.650) 0:04:33.601 **** 2021-11-03T17:00:16.041 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:16.064 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:16 +0000 (0:00:00.049) 0:04:33.651 **** 2021-11-03T17:00:16.092 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2021-11-03T17:00:16.094 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:16.114 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:16 +0000 (0:00:00.050) 0:04:33.701 **** 2021-11-03T17:00:16.142 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2021-11-03T17:00:16.144 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:16.164 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:16 +0000 (0:00:00.049) 0:04:33.751 **** 2021-11-03T17:00:16.191 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** 2021-11-03T17:00:16.193 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:16.220 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:16 +0000 (0:00:00.055) 0:04:33.806 **** 2021-11-03T17:00:16.247 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2021-11-03T17:00:16.249 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:16.269 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:16 +0000 (0:00:00.049) 0:04:33.855 **** 2021-11-03T17:00:16.296 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2021-11-03T17:00:16.297 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:16.318 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:16 +0000 (0:00:00.049) 0:04:33.905 **** 2021-11-03T17:00:16.346 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2021-11-03T17:00:16.379 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:16 +0000 (0:00:00.060) 0:04:33.966 **** 2021-11-03T17:00:16.409 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2021-11-03T17:00:16.411 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:16.433 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:16 +0000 (0:00:00.053) 0:04:34.020 **** 2021-11-03T17:00:16.460 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2021-11-03T17:00:16.462 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:16.482 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:16 +0000 (0:00:00.048) 0:04:34.069 **** 2021-11-03T17:00:16.520 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2021-11-03T17:00:16.522 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} skipping: [smithi115.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2021-11-03T17:00:16.542 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:16 +0000 (0:00:00.060) 0:04:34.129 **** 2021-11-03T17:00:16.570 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2021-11-03T17:00:16.572 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:16.593 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:16 +0000 (0:00:00.050) 0:04:34.179 **** 2021-11-03T17:00:16.620 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2021-11-03T17:00:16.642 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:16 +0000 (0:00:00.048) 0:04:34.228 **** 2021-11-03T17:00:16.668 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2021-11-03T17:00:16.670 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:16.690 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:16 +0000 (0:00:00.048) 0:04:34.277 **** 2021-11-03T17:00:16.717 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2021-11-03T17:00:16.719 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:16.740 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:16 +0000 (0:00:00.049) 0:04:34.326 **** 2021-11-03T17:00:16.767 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:16.789 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:16 +0000 (0:00:00.049) 0:04:34.376 **** 2021-11-03T17:00:16.816 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-11-03T17:00:16.818 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:16.838 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:16 +0000 (0:00:00.048) 0:04:34.425 **** 2021-11-03T17:00:16.865 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2021-11-03T17:00:16.867 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:16.887 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:16 +0000 (0:00:00.049) 0:04:34.474 **** 2021-11-03T17:00:16.915 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-11-03T17:00:16.917 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:16.937 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:16 +0000 (0:00:00.049) 0:04:34.524 **** 2021-11-03T17:00:16.965 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2021-11-03T17:00:16.966 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:16.986 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:16 +0000 (0:00:00.049) 0:04:34.573 **** 2021-11-03T17:00:17.013 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages with --no-install-recommends] **************** 2021-11-03T17:00:17.015 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:17.035 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:17 +0000 (0:00:00.049) 0:04:34.622 **** 2021-11-03T17:00:17.063 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2021-11-03T17:00:17.065 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:17.085 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:17 +0000 (0:00:00.049) 0:04:34.672 **** 2021-11-03T17:00:17.112 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2021-11-03T17:00:17.114 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:17.133 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:17 +0000 (0:00:00.048) 0:04:34.720 **** 2021-11-03T17:00:17.160 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2021-11-03T17:00:17.162 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:17.182 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:17 +0000 (0:00:00.048) 0:04:34.768 **** 2021-11-03T17:00:17.208 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2021-11-03T17:00:17.210 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:17.230 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:17 +0000 (0:00:00.047) 0:04:34.816 **** 2021-11-03T17:00:17.256 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:17.278 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:17 +0000 (0:00:00.048) 0:04:34.864 **** 2021-11-03T17:00:17.315 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:17.340 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:17 +0000 (0:00:00.062) 0:04:34.927 **** 2021-11-03T17:00:17.367 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:17.390 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:17 +0000 (0:00:00.049) 0:04:34.976 **** 2021-11-03T17:00:17.417 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-11-03T17:00:17.419 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:17.439 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:17 +0000 (0:00:00.049) 0:04:35.026 **** 2021-11-03T17:00:17.467 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-11-03T17:00:17.469 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:17.489 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:17 +0000 (0:00:00.049) 0:04:35.076 **** 2021-11-03T17:00:17.516 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-11-03T17:00:17.518 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:17.538 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:17 +0000 (0:00:00.049) 0:04:35.125 **** 2021-11-03T17:00:17.566 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-11-03T17:00:17.569 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:17.588 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:17 +0000 (0:00:00.049) 0:04:35.175 **** 2021-11-03T17:00:17.615 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** 2021-11-03T17:00:17.617 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:17.637 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:17 +0000 (0:00:00.049) 0:04:35.224 **** 2021-11-03T17:00:17.675 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:17.698 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:17 +0000 (0:00:00.061) 0:04:35.285 **** 2021-11-03T17:00:17.727 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-11-03T17:00:17.728 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:17.749 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:17 +0000 (0:00:00.050) 0:04:35.336 **** 2021-11-03T17:00:17.777 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-11-03T17:00:17.779 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:17.799 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:17 +0000 (0:00:00.049) 0:04:35.385 **** 2021-11-03T17:00:17.827 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-11-03T17:00:17.829 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:17.850 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:17 +0000 (0:00:00.051) 0:04:35.436 **** 2021-11-03T17:00:17.878 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-11-03T17:00:17.881 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:17.910 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:17 +0000 (0:00:00.060) 0:04:35.497 **** 2021-11-03T17:00:17.939 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-11-03T17:00:17.940 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:17.966 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:17 +0000 (0:00:00.056) 0:04:35.553 **** 2021-11-03T17:00:17.995 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:18.019 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:18 +0000 (0:00:00.052) 0:04:35.606 **** 2021-11-03T17:00:18.047 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-11-03T17:00:18.049 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:18.070 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:18 +0000 (0:00:00.050) 0:04:35.656 **** 2021-11-03T17:00:18.285 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-11-03T17:00:18.286 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/bin/lspci", "gid": 0, "group": "root", "mode": "0777", "owner": "root", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 11, "src": "/sbin/lspci", "state": "link", "uid": 0} 2021-11-03T17:00:18.308 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:18 +0000 (0:00:00.238) 0:04:35.895 **** 2021-11-03T17:00:18.611 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-11-03T17:00:18.613 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/ltp/testcases/bin", "secontext": "unconfined_u:object_r:lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2021-11-03T17:00:18.632 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:18 +0000 (0:00:00.324) 0:04:36.219 **** 2021-11-03T17:00:18.829 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-11-03T17:00:18.831 INFO:teuthology.task.ansible.out:[WARNING]: Cannot set fs attributes on a non-existent symlink target. follow should be set to False to avoid this. ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/lib/ltp/testcases/bin/fsstress", "src": "/usr/bin/fsstress"} 2021-11-03T17:00:18.853 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:18 +0000 (0:00:00.220) 0:04:36.440 **** 2021-11-03T17:00:18.854 INFO:teuthology.task.ansible.out: 2021-11-03T17:00:18.881 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-11-03T17:00:18.883 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:18.904 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:18 +0000 (0:00:00.050) 0:04:36.490 **** 2021-11-03T17:00:18.932 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:18.956 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:18 +0000 (0:00:00.052) 0:04:36.543 **** 2021-11-03T17:00:18.985 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:19.008 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:19 +0000 (0:00:00.051) 0:04:36.594 **** 2021-11-03T17:00:19.035 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-11-03T17:00:19.037 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:19.057 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:19 +0000 (0:00:00.049) 0:04:36.644 **** 2021-11-03T17:00:19.084 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:19.113 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:19 +0000 (0:00:00.055) 0:04:36.699 **** 2021-11-03T17:00:19.143 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-11-03T17:00:19.145 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:19.166 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:19 +0000 (0:00:00.053) 0:04:36.753 **** 2021-11-03T17:00:19.195 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-11-03T17:00:19.197 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:19.217 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:19 +0000 (0:00:00.050) 0:04:36.803 **** 2021-11-03T17:00:19.245 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-11-03T17:00:19.246 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:19.273 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:19 +0000 (0:00:00.055) 0:04:36.859 **** 2021-11-03T17:00:19.300 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-11-03T17:00:19.301 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:19.322 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:19 +0000 (0:00:00.049) 0:04:36.909 **** 2021-11-03T17:00:19.350 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2021-11-03T17:00:19.352 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:19.375 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:19 +0000 (0:00:00.052) 0:04:36.962 **** 2021-11-03T17:00:19.407 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2021-11-03T17:00:19.408 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:19.429 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:19 +0000 (0:00:00.054) 0:04:37.016 **** 2021-11-03T17:00:19.458 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-11-03T17:00:19.460 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:19.480 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:19 +0000 (0:00:00.050) 0:04:37.067 **** 2021-11-03T17:00:19.507 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-11-03T17:00:19.509 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:19.532 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:19 +0000 (0:00:00.051) 0:04:37.119 **** 2021-11-03T17:00:19.561 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:19.588 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:19 +0000 (0:00:00.055) 0:04:37.175 **** 2021-11-03T17:00:19.618 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2021-11-03T17:00:19.620 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:19.643 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:19 +0000 (0:00:00.054) 0:04:37.229 **** 2021-11-03T17:00:19.680 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2021-11-03T17:00:19.682 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:19.703 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:19 +0000 (0:00:00.059) 0:04:37.289 **** 2021-11-03T17:00:19.730 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2021-11-03T17:00:19.732 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:19.752 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:19 +0000 (0:00:00.049) 0:04:37.339 **** 2021-11-03T17:00:19.779 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:19.802 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:19 +0000 (0:00:00.049) 0:04:37.388 **** 2021-11-03T17:00:19.829 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:19.851 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:19 +0000 (0:00:00.049) 0:04:37.438 **** 2021-11-03T17:00:19.878 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2021-11-03T17:00:19.880 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:19.900 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:19 +0000 (0:00:00.049) 0:04:37.487 **** 2021-11-03T17:00:19.927 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:19.949 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:19 +0000 (0:00:00.048) 0:04:37.536 **** 2021-11-03T17:00:19.976 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2021-11-03T17:00:19.978 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:19.999 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:19 +0000 (0:00:00.049) 0:04:37.586 **** 2021-11-03T17:00:20.027 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-11-03T17:00:20.028 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2021-11-03T17:00:20.049 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:20 +0000 (0:00:00.049) 0:04:37.635 **** 2021-11-03T17:00:20.086 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2021-11-03T17:00:20.088 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2021-11-03T17:00:20.097 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2021-11-03T17:00:20.109 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2021-11-03T17:00:20.110 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2021-11-03T17:00:20.120 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2021-11-03T17:00:20.122 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2021-11-03T17:00:20.132 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2021-11-03T17:00:20.157 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:20 +0000 (0:00:00.108) 0:04:37.744 **** 2021-11-03T17:00:20.351 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2021-11-03T17:00:20.353 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003284", "end": "2021-11-03 17:00:20.330461", "rc": 0, "start": "2021-11-03 17:00:20.327177", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/virtualenv/bin/teuthology-lock"]} 2021-11-03T17:00:20.376 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:20 +0000 (0:00:00.219) 0:04:37.963 **** 2021-11-03T17:00:20.403 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2021-11-03T17:00:20.405 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:20.427 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:20 +0000 (0:00:00.051) 0:04:38.014 **** 2021-11-03T17:00:20.455 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2021-11-03T17:00:20.457 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:20.478 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:20 +0000 (0:00:00.050) 0:04:38.065 **** 2021-11-03T17:00:20.505 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2021-11-03T17:00:20.507 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:20.528 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:20 +0000 (0:00:00.049) 0:04:38.115 **** 2021-11-03T17:00:20.557 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2021-11-03T17:00:20.559 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:20.582 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:20 +0000 (0:00:00.054) 0:04:38.169 **** 2021-11-03T17:00:22.212 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2021-11-03T17:00:22.214 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": []} 2021-11-03T17:00:22.234 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:22 +0000 (0:00:01.652) 0:04:39.821 **** 2021-11-03T17:00:22.274 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-11-03T17:00:22.276 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930249576448, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227111713, 'block_used': 12983395, 'inode_total': 61054976, 'inode_available': 60985860, 'inode_used': 69116, 'uuid': 'fcc12580-8f3b-4cec-aef7-e7427fa239b7'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227111713, "block_size": 4096, "block_total": 240095108, "block_used": 12983395, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60985860, "inode_total": 61054976, "inode_used": 69116, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930249576448, "size_total": 983429562368, "uuid": "fcc12580-8f3b-4cec-aef7-e7427fa239b7"}} 2021-11-03T17:00:22.298 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:22 +0000 (0:00:00.063) 0:04:39.885 **** 2021-11-03T17:00:22.492 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2021-11-03T17:00:22.494 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.005191", "end": "2021-11-03 17:00:22.475748", "rc": 0, "start": "2021-11-03 17:00:22.470557", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-03T17:00:22.515 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:22 +0000 (0:00:00.216) 0:04:40.101 **** 2021-11-03T17:00:22.554 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2021-11-03T17:00:22.556 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930249576448, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227111713, 'block_used': 12983395, 'inode_total': 61054976, 'inode_available': 60985860, 'inode_used': 69116, 'uuid': 'fcc12580-8f3b-4cec-aef7-e7427fa239b7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227111713, "block_size": 4096, "block_total": 240095108, "block_used": 12983395, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60985860, "inode_total": 61054976, "inode_used": 69116, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930249576448, "size_total": 983429562368, "uuid": "fcc12580-8f3b-4cec-aef7-e7427fa239b7"}, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:22.577 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:22 +0000 (0:00:00.062) 0:04:40.164 **** 2021-11-03T17:00:22.605 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2021-11-03T17:00:22.627 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:22 +0000 (0:00:00.049) 0:04:40.213 **** 2021-11-03T17:00:24.475 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-11-03T17:00:24.477 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-IGzsxcOOgLDtduhbqPlboh3vQGdpccChgTsDHggCsCgZUoy8XFFozc7EWowDpF4b'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.632457", "end": "2021-11-03 17:00:24.449711", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-IGzsxcOOgLDtduhbqPlboh3vQGdpccChgTsDHggCsCgZUoy8XFFozc7EWowDpF4b"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-03 17:00:22.817254", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-11-03T17:00:25.732 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'virtual': 1, 'links': {'ids': ['lvm-pv-uuid-B05rN8-xzPE-NtnF-BEVp-YMcc-ajOg-67JQAN', 'nvme-INTEL_SSDPEDMD400G4_PHFT620400UU400BGN', 'nvme-nvme.8086-50484654363230343030555534303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': [], 'labels': [], 'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4']}, 'vendor': None, 'model': 'INTEL SSDPEDMD400G4', 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': 'none', 'sectors': '781422768', 'sectorsize': '512', 'size': '372.61 GB', 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4']}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.065253", "end": "2021-11-03 17:00:25.704480", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-B05rN8-xzPE-NtnF-BEVp-YMcc-ajOg-67JQAN", "nvme-INTEL_SSDPEDMD400G4_PHFT620400UU400BGN", "nvme-nvme.8086-50484654363230343030555534303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-03 17:00:24.639227", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! 2021-11-03T17:00:25.734 INFO:teuthology.task.ansible.out:You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-11-03T17:00:26.977 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-IGzsxcOOgLDtduhbqPlboh3vQGdpccCh8tlOaNuYUYv2oQoFQIG7W03R1XjRLYoM'], 'uuids': ['98284e88-1610-40fa-9e80-18603c9cb0f2'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '31252480', 'sectorsize': '512', 'size': '14.90 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.056807", "end": "2021-11-03 17:00:26.952223", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-IGzsxcOOgLDtduhbqPlboh3vQGdpccCh8tlOaNuYUYv2oQoFQIG7W03R1XjRLYoM"], "labels": [], "masters": [], "uuids": ["98284e88-1610-40fa-9e80-18603c9cb0f2"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-03 17:00:25.895416", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-11-03T17:00:28.397 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-IGzsxcOOgLDtduhbqPlboh3vQGdpccChecYiH9ft0CIGAeqE64MPJi4CDPc4Fipn'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.229880", "end": "2021-11-03 17:00:28.368557", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-IGzsxcOOgLDtduhbqPlboh3vQGdpccChecYiH9ft0CIGAeqE64MPJi4CDPc4Fipn"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-03 17:00:27.138677", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-11-03T17:00:29.819 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-IGzsxcOOgLDtduhbqPlboh3vQGdpccCh3K4hn6YgrSwcfYjpF433uEMPQ5N1UnRk'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.229888", "end": "2021-11-03 17:00:29.784951", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-IGzsxcOOgLDtduhbqPlboh3vQGdpccCh3K4hn6YgrSwcfYjpF433uEMPQ5N1UnRk"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-03 17:00:28.555063", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-11-03T17:00:29.831 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'virtual': 1, 'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QYT7', 'scsi-0ATA_ST1000NM0033-9ZM_Z1W5QYT7', 'scsi-1ATA_ST1000NM0033-9ZM173_Z1W5QYT7', 'scsi-35000c50091e302fc', 'scsi-SATA_ST1000NM0033-9ZM_Z1W5QYT7', 'wwn-0x5000c50091e302fc'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': 'ATA', 'model': 'ST1000NM0033-9ZM', 'sas_address': None, 'sas_device_handle': None, 'serial': 'Z1W5QYT7', 'removable': '0', 'support_discard': '0', 'wwn': '0x5000c50091e302fc', 'partitions': {'sda1': {'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QYT7-part1', 'scsi-0ATA_ST1000NM0033-9ZM_Z1W5QYT7-part1', 'scsi-1ATA_ST1000NM0033-9ZM173_Z1W5QYT7-part1', 'scsi-35000c50091e302fc-part1', 'scsi-SATA_ST1000NM0033-9ZM_Z1W5QYT7-part1', 'wwn-0x5000c50091e302fc-part1'], 'uuids': ['fcc12580-8f3b-4cec-aef7-e7427fa239b7'], 'labels': [], 'masters': []}, 'start': '2048', 'sectors': '1953522688', 'sectorsize': 512, 'size': '931.51 GB', 'uuid': 'fcc12580-8f3b-4cec-aef7-e7427fa239b7', 'holders': []}}, 'rotational': '1', 'scheduler_mode': 'mq-deadline', 'sectors': '1953525168', 'sectorsize': '512', 'size': '931.51 GB', 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'holders': []}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QYT7", "scsi-0ATA_ST1000NM0033-9ZM_Z1W5QYT7", "scsi-1ATA_ST1000NM0033-9ZM173_Z1W5QYT7", "scsi-35000c50091e302fc", "scsi-SATA_ST1000NM0033-9ZM_Z1W5QYT7", "wwn-0x5000c50091e302fc"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QYT7-part1", "scsi-0ATA_ST1000NM0033-9ZM_Z1W5QYT7-part1", "scsi-1ATA_ST1000NM0033-9ZM173_ 2021-11-03T17:00:29.832 INFO:teuthology.task.ansible.out:Z1W5QYT7-part1", "scsi-35000c50091e302fc-part1", "scsi-SATA_ST1000NM0033-9ZM_Z1W5QYT7-part1", "wwn-0x5000c50091e302fc-part1"], "labels": [], "masters": [], "uuids": ["fcc12580-8f3b-4cec-aef7-e7427fa239b7"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "fcc12580-8f3b-4cec-aef7-e7427fa239b7"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5QYT7", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50091e302fc"}}, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:31.075 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-IGzsxcOOgLDtduhbqPlboh3vQGdpccChpUgf55s7rGstNvmFqzfeL7BOHWp2AAOJ'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.048873", "end": "2021-11-03 17:00:31.052725", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-IGzsxcOOgLDtduhbqPlboh3vQGdpccChpUgf55s7rGstNvmFqzfeL7BOHWp2AAOJ"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-03 17:00:30.003852", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-11-03T17:00:31.116 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:31 +0000 (0:00:08.489) 0:04:48.703 **** 2021-11-03T17:00:31.483 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-11-03T17:00:31.485 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-IGzsxcOOgLDtduhbqPlboh3vQGdpccChgTsDHggCsCgZUoy8XFFozc7EWowDpF4b'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.144106", "end": "2021-11-03 17:00:31.461129", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-IGzsxcOOgLDtduhbqPlboh3vQGdpccChgTsDHggCsCgZUoy8XFFozc7EWowDpF4b"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-03 17:00:31.317023", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.141388 s, 816 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.141388 s, 816 MB/s"], "stdout": "", "stdout_lines": []} 2021-11-03T17:00:31.696 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'virtual': 1, 'links': {'ids': ['lvm-pv-uuid-B05rN8-xzPE-NtnF-BEVp-YMcc-ajOg-67JQAN', 'nvme-INTEL_SSDPEDMD400G4_PHFT620400UU400BGN', 'nvme-nvme.8086-50484654363230343030555534303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': [], 'labels': [], 'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4']}, 'vendor': None, 'model': 'INTEL SSDPEDMD400G4', 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': 'none', 'sectors': '781422768', 'sectorsize': '512', 'size': '372.61 GB', 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4']}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.038680", "end": "2021-11-03 17:00:31.671335", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-B05rN8-xzPE-NtnF-BEVp-YMcc-ajOg-67JQAN", "nvme-INTEL_SSDPEDMD400G4_PHFT620400UU400BGN", "nvme-nvme.8086-50484654363230343030555534303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-03 17:00:31.632655", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.0361208 s, 3.2 GB/s", "std 2021-11-03T17:00:31.698 INFO:teuthology.task.ansible.out:err_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.0361208 s, 3.2 GB/s"], "stdout": "", "stdout_lines": []} 2021-11-03T17:00:32.011 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-IGzsxcOOgLDtduhbqPlboh3vQGdpccCh8tlOaNuYUYv2oQoFQIG7W03R1XjRLYoM'], 'uuids': ['98284e88-1610-40fa-9e80-18603c9cb0f2'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '31252480', 'sectorsize': '512', 'size': '14.90 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.143411", "end": "2021-11-03 17:00:31.991137", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-IGzsxcOOgLDtduhbqPlboh3vQGdpccCh8tlOaNuYUYv2oQoFQIG7W03R1XjRLYoM"], "labels": [], "masters": [], "uuids": ["98284e88-1610-40fa-9e80-18603c9cb0f2"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-03 17:00:31.847726", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.140867 s, 819 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.140867 s, 819 MB/s"], "stdout": "", "stdout_lines": []} 2021-11-03T17:00:32.327 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-IGzsxcOOgLDtduhbqPlboh3vQGdpccChecYiH9ft0CIGAeqE64MPJi4CDPc4Fipn'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.141425", "end": "2021-11-03 17:00:32.304910", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-IGzsxcOOgLDtduhbqPlboh3vQGdpccChecYiH9ft0CIGAeqE64MPJi4CDPc4Fipn"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-03 17:00:32.163485", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.138987 s, 830 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.138987 s, 830 MB/s"], "stdout": "", "stdout_lines": []} 2021-11-03T17:00:32.642 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-IGzsxcOOgLDtduhbqPlboh3vQGdpccCh3K4hn6YgrSwcfYjpF433uEMPQ5N1UnRk'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.147556", "end": "2021-11-03 17:00:32.624249", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-IGzsxcOOgLDtduhbqPlboh3vQGdpccCh3K4hn6YgrSwcfYjpF433uEMPQ5N1UnRk"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-03 17:00:32.476693", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.14492 s, 796 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.14492 s, 796 MB/s"], "stdout": "", "stdout_lines": []} 2021-11-03T17:00:32.654 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'virtual': 1, 'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QYT7', 'scsi-0ATA_ST1000NM0033-9ZM_Z1W5QYT7', 'scsi-1ATA_ST1000NM0033-9ZM173_Z1W5QYT7', 'scsi-35000c50091e302fc', 'scsi-SATA_ST1000NM0033-9ZM_Z1W5QYT7', 'wwn-0x5000c50091e302fc'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': 'ATA', 'model': 'ST1000NM0033-9ZM', 'sas_address': None, 'sas_device_handle': None, 'serial': 'Z1W5QYT7', 'removable': '0', 'support_discard': '0', 'wwn': '0x5000c50091e302fc', 'partitions': {'sda1': {'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W5QYT7-part1', 'scsi-0ATA_ST1000NM0033-9ZM_Z1W5QYT7-part1', 'scsi-1ATA_ST1000NM0033-9ZM173_Z1W5QYT7-part1', 'scsi-35000c50091e302fc-part1', 'scsi-SATA_ST1000NM0033-9ZM_Z1W5QYT7-part1', 'wwn-0x5000c50091e302fc-part1'], 'uuids': ['fcc12580-8f3b-4cec-aef7-e7427fa239b7'], 'labels': [], 'masters': []}, 'start': '2048', 'sectors': '1953522688', 'sectorsize': 512, 'size': '931.51 GB', 'uuid': 'fcc12580-8f3b-4cec-aef7-e7427fa239b7', 'holders': []}}, 'rotational': '1', 'scheduler_mode': 'mq-deadline', 'sectors': '1953525168', 'sectorsize': '512', 'size': '931.51 GB', 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'holders': []}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QYT7", "scsi-0ATA_ST1000NM0033-9ZM_Z1W5QYT7", "scsi-1ATA_ST1000NM0033-9ZM173_Z1W5QYT7", "scsi-35000c50091e302fc", "scsi-SATA_ST1000NM0033-9ZM_Z1W5QYT7", "wwn-0x5000c50091e302fc"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5QYT7-part1", "scsi-0ATA_ST1000NM0033-9ZM_Z1W5QYT7-part1", "scsi-1ATA_ST1000NM0033-9ZM173_ 2021-11-03T17:00:32.656 INFO:teuthology.task.ansible.out:Z1W5QYT7-part1", "scsi-35000c50091e302fc-part1", "scsi-SATA_ST1000NM0033-9ZM_Z1W5QYT7-part1", "wwn-0x5000c50091e302fc-part1"], "labels": [], "masters": [], "uuids": ["fcc12580-8f3b-4cec-aef7-e7427fa239b7"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "fcc12580-8f3b-4cec-aef7-e7427fa239b7"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5QYT7", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c50091e302fc"}}, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:32.971 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-IGzsxcOOgLDtduhbqPlboh3vQGdpccChpUgf55s7rGstNvmFqzfeL7BOHWp2AAOJ'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.143303", "end": "2021-11-03 17:00:32.949875", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-IGzsxcOOgLDtduhbqPlboh3vQGdpccChpUgf55s7rGstNvmFqzfeL7BOHWp2AAOJ"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-03 17:00:32.806572", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.14067 s, 820 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.14067 s, 820 MB/s"], "stdout": "", "stdout_lines": []} 2021-11-03T17:00:32.999 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:32 +0000 (0:00:01.882) 0:04:50.585 **** 2021-11-03T17:00:33.235 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2021-11-03T17:00:33.237 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.047871", "end": "2021-11-03 17:00:33.210884", "rc": 0, "start": "2021-11-03 17:00:33.163013", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-03T17:00:33.257 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:33 +0000 (0:00:00.258) 0:04:50.844 **** 2021-11-03T17:00:33.580 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2021-11-03T17:00:33.581 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.136099", "end": "2021-11-03 17:00:33.560003", "rc": 0, "start": "2021-11-03 17:00:33.423904", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-03T17:00:33.602 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:33 +0000 (0:00:00.344) 0:04:51.189 **** 2021-11-03T17:00:33.630 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2021-11-03T17:00:33.654 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:33 +0000 (0:00:00.052) 0:04:51.241 **** 2021-11-03T17:00:33.681 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2021-11-03T17:00:33.683 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:33.706 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:33 +0000 (0:00:00.052) 0:04:51.293 **** 2021-11-03T17:00:33.734 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2021-11-03T17:00:33.736 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:33.758 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:33 +0000 (0:00:00.051) 0:04:51.344 **** 2021-11-03T17:00:33.787 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:33.811 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:33 +0000 (0:00:00.053) 0:04:51.398 **** 2021-11-03T17:00:33.839 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2021-11-03T17:00:33.841 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:33.872 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:33 +0000 (0:00:00.060) 0:04:51.459 **** 2021-11-03T17:00:34.059 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2021-11-03T17:00:34.061 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-11-03T17:00:34.081 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:34 +0000 (0:00:00.209) 0:04:51.668 **** 2021-11-03T17:00:34.129 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-11-03T17:00:34.132 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930249576448, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227111713, 'block_used': 12983395, 'inode_total': 61054976, 'inode_available': 60985860, 'inode_used': 69116, 'uuid': 'fcc12580-8f3b-4cec-aef7-e7427fa239b7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227111713, "block_size": 4096, "block_total": 240095108, "block_used": 12983395, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60985860, "inode_total": 61054976, "inode_used": 69116, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930249576448, "size_total": 983429562368, "uuid": "fcc12580-8f3b-4cec-aef7-e7427fa239b7"}, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:34.154 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:34 +0000 (0:00:00.073) 0:04:51.741 **** 2021-11-03T17:00:34.181 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2021-11-03T17:00:34.183 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:34.203 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:34 +0000 (0:00:00.048) 0:04:51.790 **** 2021-11-03T17:00:34.241 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:34.263 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:34 +0000 (0:00:00.060) 0:04:51.850 **** 2021-11-03T17:00:34.301 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-11-03T17:00:34.303 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:34.323 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:34 +0000 (0:00:00.060) 0:04:51.910 **** 2021-11-03T17:00:34.350 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2021-11-03T17:00:34.352 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:34.372 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:34 +0000 (0:00:00.048) 0:04:51.959 **** 2021-11-03T17:00:34.410 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:34.432 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:34 +0000 (0:00:00.059) 0:04:52.018 **** 2021-11-03T17:00:34.459 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2021-11-03T17:00:34.461 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:34.482 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:34 +0000 (0:00:00.049) 0:04:52.068 **** 2021-11-03T17:00:35.159 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2021-11-03T17:00:35.161 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2021-11-03T17:00:35.189 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:35 +0000 (0:00:00.707) 0:04:52.776 **** 2021-11-03T17:00:35.837 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* changed: [smithi115.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-11-03T17:00:36.161 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-11-03T17:00:36.505 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-11-03T17:00:36.837 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-11-03T17:00:37.184 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2021-11-03T17:00:37.206 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:37 +0000 (0:00:02.016) 0:04:54.793 **** 2021-11-03T17:00:37.413 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-11-03T17:00:37.415 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2021-11-03T17:00:37.436 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:37 +0000 (0:00:00.229) 0:04:55.022 **** 2021-11-03T17:00:37.661 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-11-03T17:00:37.663 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-11-03T17:00:37.830 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-11-03T17:00:38.007 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-11-03T17:00:38.175 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-11-03T17:00:38.186 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:38.211 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:38 +0000 (0:00:00.775) 0:04:55.797 **** 2021-11-03T17:00:38.238 INFO:teuthology.task.ansible.out: TASK [testnode : Create filesystems] ******************************************* 2021-11-03T17:00:38.240 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:38.260 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:38 +0000 (0:00:00.049) 0:04:55.846 **** 2021-11-03T17:00:38.288 INFO:teuthology.task.ansible.out: TASK [testnode : Mount filesystems] ******************************************** 2021-11-03T17:00:38.289 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:38.310 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:38 +0000 (0:00:00.049) 0:04:55.896 **** 2021-11-03T17:00:38.503 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2021-11-03T17:00:38.505 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2021-11-03T17:00:38.525 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:38 +0000 (0:00:00.215) 0:04:56.112 **** 2021-11-03T17:00:38.562 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2021-11-03T17:00:38.564 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2021-11-03T17:00:38.587 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:38 +0000 (0:00:00.061) 0:04:56.173 **** 2021-11-03T17:00:39.102 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2021-11-03T17:00:39.104 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => {"changed": true} 2021-11-03T17:00:39.126 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:39 +0000 (0:00:00.539) 0:04:56.713 **** 2021-11-03T17:00:39.686 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2021-11-03T17:00:39.688 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2021-11-03T17:00:39.710 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:39 +0000 (0:00:00.583) 0:04:57.296 **** 2021-11-03T17:00:39.737 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd] ****************************************** skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:39.758 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:39 +0000 (0:00:00.048) 0:04:57.345 **** 2021-11-03T17:00:39.785 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd conf file] ******************************** skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:39.805 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:39 +0000 (0:00:00.047) 0:04:57.392 **** 2021-11-03T17:00:39.835 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-11-03T17:00:39.837 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:39.856 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:39 +0000 (0:00:00.050) 0:04:57.442 **** 2021-11-03T17:00:39.882 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:39.903 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:39 +0000 (0:00:00.046) 0:04:57.489 **** 2021-11-03T17:00:39.930 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2021-11-03T17:00:39.931 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:39.951 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:39 +0000 (0:00:00.048) 0:04:57.537 **** 2021-11-03T17:00:39.977 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2021-11-03T17:00:39.978 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:39.997 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:39 +0000 (0:00:00.046) 0:04:57.584 **** 2021-11-03T17:00:40.024 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2021-11-03T17:00:40.026 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:40.049 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:40 +0000 (0:00:00.052) 0:04:57.636 **** 2021-11-03T17:00:40.078 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2021-11-03T17:00:40.080 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:40.099 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:40 +0000 (0:00:00.049) 0:04:57.686 **** 2021-11-03T17:00:40.127 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2021-11-03T17:00:40.128 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:40.159 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:40 +0000 (0:00:00.059) 0:04:57.745 **** 2021-11-03T17:00:40.189 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2021-11-03T17:00:40.191 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:40.213 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:40 +0000 (0:00:00.054) 0:04:57.800 **** 2021-11-03T17:00:40.240 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2021-11-03T17:00:40.242 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:40.262 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:40 +0000 (0:00:00.049) 0:04:57.849 **** 2021-11-03T17:00:40.289 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2021-11-03T17:00:40.291 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:40.311 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:40 +0000 (0:00:00.048) 0:04:57.898 **** 2021-11-03T17:00:40.338 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2021-11-03T17:00:40.340 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:40.360 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:40 +0000 (0:00:00.049) 0:04:57.947 **** 2021-11-03T17:00:40.387 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:40.409 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:40 +0000 (0:00:00.048) 0:04:57.996 **** 2021-11-03T17:00:40.436 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:40.457 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:40 +0000 (0:00:00.048) 0:04:58.044 **** 2021-11-03T17:00:40.475 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-11-03T17:00:40.476 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: librados2-1:12.2.7-9.el8.x86_64", "Removed: librbd1-1:12.2.7-9.el8.x86_64", "Removed: qemu-kvm-block-rbd-15:4.2.0-48.module_el8.4.0+783+f8734d30.x86_64"]} 2021-11-03T17:00:40.503 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:40 +0000 (0:00:00.045) 0:04:58.090 **** 2021-11-03T17:00:40.917 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2021-11-03T17:00:40.919 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "checksum": "a7ecef00073feab34f868065d34e9dac55134241", "dest": "/etc/chrony.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/chrony.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 215, "state": "file", "uid": 0} 2021-11-03T17:00:40.941 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:40 +0000 (0:00:00.437) 0:04:58.527 **** 2021-11-03T17:00:41.249 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2021-11-03T17:00:41.250 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "chronyd", "state": "started", "status": {"ActiveState": "active", "AllowedCPUs": "", "AllowedMemoryNodes": "", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "ControlGroup": "/system.slice/chronyd.service", "ControlPID": "0", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "DevicePolicy": "auto", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "EnvironmentFiles": "/etc/sysconfig/chronyd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "775", "ExecMainStartTimestamp": "Wed 2021-11-03 16:55:20 UTC", "ExecMainStartTimestampMonotonic": "12271586", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/chronyd ; argv[]=/usr/sbin/chronyd $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartPost": "{ path=/usr/libexec/chrony-helper ; argv[]=/usr/libexec/chrony-helper update-daemon ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FileDescriptorStoreMax": "0", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSI 2021-11-03T17:00:41.251 INFO:teuthology.task.ansible.out:ZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "262144", "LimitNOFILESoft": "1024", "LimitNPROC": "127245", "LimitNPROCSoft": "127245", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127245", "LimitSIGPENDINGSoft": "127245", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "MainPID": "775", "MemoryAccounting": "yes", "MemoryCurrent": "2076672", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Nice": "0", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "PIDFile": "/run/chrony/chronyd.pid", "PermissionsStartOnly": "no", "RemainAfterExit": "no", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeMaxUSec": "infinity", "SecureBits": "0", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StatusErrno": "0", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "203592", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Type": "forking", "UID": "[not set]", "UMask": "0022", "WatchdogTimestamp": "Wed 2021-11-03 1 2021-11-03T17:00:41.253 INFO:teuthology.task.ansible.out:6:55:20 UTC", "WatchdogTimestampMonotonic": "12271601", "WatchdogUSec": "0"}} 2021-11-03T17:00:41.272 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:41 +0000 (0:00:00.331) 0:04:58.859 **** 2021-11-03T17:00:41.299 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-11-03T17:00:41.301 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2021-11-03T17:00:41.321 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:41 +0000 (0:00:00.048) 0:04:58.907 **** 2021-11-03T17:00:41.358 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-11-03T17:00:41.360 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"ansible_facts": {"ansible_python_interpreter": "/usr/bin/python3", "pip_executable": "pip3", "pip_version": "python3-pip"}, "changed": false} 2021-11-03T17:00:41.381 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:41 +0000 (0:00:00.060) 0:04:58.968 **** 2021-11-03T17:00:41.408 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:41.430 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:41 +0000 (0:00:00.049) 0:04:59.017 **** 2021-11-03T17:00:41.457 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2021-11-03T17:00:41.459 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:41.479 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:41 +0000 (0:00:00.048) 0:04:59.066 **** 2021-11-03T17:00:41.507 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2021-11-03T17:00:41.507 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:41.529 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:41 +0000 (0:00:00.049) 0:04:59.116 **** 2021-11-03T17:00:41.725 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2021-11-03T17:00:41.727 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "gid": 1254, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2021-11-03T17:00:41.747 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:41 +0000 (0:00:00.218) 0:04:59.334 **** 2021-11-03T17:00:42.225 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2021-11-03T17:00:42.226 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1254, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2021-11-03T17:00:42.247 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:42 +0000 (0:00:00.499) 0:04:59.834 **** 2021-11-03T17:00:43.303 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-11-03T17:00:43.305 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Removed: boost-program-options-1.66.0-10.el8.x86_64", "Removed: boost-random-1.66.0-10.el8.x86_64"]} 2021-11-03T17:00:43.326 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:43 +0000 (0:00:01.078) 0:05:00.913 **** 2021-11-03T17:00:43.540 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2021-11-03T17:00:43.542 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip3", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.\n", "stderr_lines": ["WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead."], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages\nRequirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages", "Requirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)"], "version": null, "virtualenv": null} 2021-11-03T17:00:43.565 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:43 +0000 (0:00:00.238) 0:05:01.151 **** 2021-11-03T17:00:43.592 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2021-11-03T17:00:43.593 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:43.613 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:43 +0000 (0:00:00.048) 0:05:01.200 **** 2021-11-03T17:00:43.640 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2021-11-03T17:00:43.642 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2021-11-03T17:00:43.662 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:43 +0000 (0:00:00.048) 0:05:01.249 **** 2021-11-03T17:00:43.845 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2021-11-03T17:00:43.847 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2021-11-03T17:00:43.878 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:43 +0000 (0:00:00.216) 0:05:01.465 **** 2021-11-03T17:00:43.909 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2021-11-03T17:00:43.910 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:43.931 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:43 +0000 (0:00:00.052) 0:05:01.517 **** 2021-11-03T17:00:43.957 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2021-11-03T17:00:43.959 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2021-11-03T17:00:43.980 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:43 +0000 (0:00:00.048) 0:05:01.566 **** 2021-11-03T17:00:44.006 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2021-11-03T17:00:44.007 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:44.030 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:44 +0000 (0:00:00.050) 0:05:01.616 **** 2021-11-03T17:00:44.224 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** changed: [smithi115.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2021-11-03T17:00:44.245 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:44 +0000 (0:00:00.215) 0:05:01.832 **** 2021-11-03T17:00:44.439 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2021-11-03T17:00:44.441 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2021-11-03T17:00:44.461 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:44 +0000 (0:00:00.215) 0:05:02.047 **** 2021-11-03T17:00:44.499 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2021-11-03T17:00:44.501 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:44.521 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:44 +0000 (0:00:00.060) 0:05:02.108 **** 2021-11-03T17:00:44.560 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2021-11-03T17:00:44.562 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} skipping: [smithi115.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} 2021-11-03T17:00:44.571 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2021-11-03T17:00:44.573 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} skipping: [smithi115.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2021-11-03T17:00:44.596 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:44 +0000 (0:00:00.074) 0:05:02.183 **** 2021-11-03T17:00:44.633 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2021-11-03T17:00:44.635 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:44.644 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:44.646 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:44.656 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'enp3s0f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:44.657 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:44.687 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:44 +0000 (0:00:00.091) 0:05:02.274 **** 2021-11-03T17:00:44.717 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:44.742 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:44 +0000 (0:00:00.054) 0:05:02.328 **** 2021-11-03T17:00:44.769 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2021-11-03T17:00:44.771 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:44.791 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:44 +0000 (0:00:00.049) 0:05:02.378 **** 2021-11-03T17:00:44.818 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2021-11-03T17:00:44.820 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:44.842 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:44 +0000 (0:00:00.050) 0:05:02.429 **** 2021-11-03T17:00:44.869 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2021-11-03T17:00:44.871 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:44.891 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:44 +0000 (0:00:00.049) 0:05:02.478 **** 2021-11-03T17:00:44.918 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2021-11-03T17:00:44.920 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:44.942 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:44 +0000 (0:00:00.051) 0:05:02.529 **** 2021-11-03T17:00:44.970 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-11-03T17:00:44.972 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:44.994 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:44 +0000 (0:00:00.051) 0:05:02.581 **** 2021-11-03T17:00:45.032 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2021-11-03T17:00:45.034 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:45.056 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:45 +0000 (0:00:00.061) 0:05:02.643 **** 2021-11-03T17:00:45.083 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-11-03T17:00:45.085 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:45.105 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:45 +0000 (0:00:00.049) 0:05:02.692 **** 2021-11-03T17:00:45.311 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-11-03T17:00:45.313 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.004851", "end": "2021-11-03 17:00:45.286010", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2021-11-03 17:00:45.281159", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-03T17:00:45.478 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.002974", "end": "2021-11-03 17:00:45.457951", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2021-11-03 17:00:45.454977", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-03T17:00:45.655 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.002736", "end": "2021-11-03 17:00:45.627771", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2021-11-03 17:00:45.625035", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2021-11-03T17:00:46.826 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:01.004133", "end": "2021-11-03 17:00:46.797728", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2021-11-03 17:00:45.793595", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-03T17:00:46.849 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:46 +0000 (0:00:01.743) 0:05:04.436 **** 2021-11-03T17:00:46.877 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2021-11-03T17:00:46.901 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:46 +0000 (0:00:00.051) 0:05:04.488 **** 2021-11-03T17:00:46.928 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2021-11-03T17:00:46.930 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:50.926 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-11-03T17:00:50.928 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: qemu-kvm-core-15:4.2.0-48.module_el8.4.0+885+5e18b468.3.x86_64", "Installed: qemu-img-15:4.2.0-48.module_el8.4.0+885+5e18b468.3.x86_64", "Installed: librados2-1:12.2.7-9.el8.x86_64", "Installed: qemu-kvm-block-rbd-15:4.2.0-48.module_el8.4.0+885+5e18b468.3.x86_64", "Installed: boost-program-options-1.66.0-10.el8.x86_64", "Installed: librbd1-1:12.2.7-9.el8.x86_64", "Installed: boost-random-1.66.0-10.el8.x86_64", "Installed: qemu-kvm-common-15:4.2.0-48.module_el8.4.0+885+5e18b468.3.x86_64", "Removed: qemu-kvm-core-15:4.2.0-48.module_el8.4.0+783+f8734d30.x86_64", "Removed: qemu-img-15:4.2.0-48.module_el8.4.0+783+f8734d30.x86_64", "Removed: qemu-kvm-common-15:4.2.0-48.module_el8.4.0+783+f8734d30.x86_64"]} 2021-11-03T17:00:50.948 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:50 +0000 (0:00:04.046) 0:05:08.534 **** 2021-11-03T17:00:50.985 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2021-11-03T17:00:50.986 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:51.006 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:50 +0000 (0:00:00.058) 0:05:08.593 **** 2021-11-03T17:00:52.791 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-11-03T17:00:52.793 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": []} 2021-11-03T17:00:52.813 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:52 +0000 (0:00:01.806) 0:05:10.399 **** 2021-11-03T17:00:54.539 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-11-03T17:00:54.542 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": []} 2021-11-03T17:00:54.561 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:54 +0000 (0:00:01.748) 0:05:12.148 **** 2021-11-03T17:00:54.894 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveState": "inactive", "AllowedCPUs": "", "AllowedMemoryNodes": "", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUAffinity": "", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "ControlPID": "0", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "DevicePolicy": "auto", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "EnvironmentFiles": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FileDescriptorStoreMax": "0", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", 2021-11-03T17:00:54.896 INFO:teuthology.task.ansible.out: "LimitFSIZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "262144", "LimitNOFILESoft": "1024", "LimitNPROC": "127269", "LimitNPROCSoft": "127269", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127269", "LimitSIGPENDINGSoft": "127269", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "MainPID": "0", "MemoryAccounting": "yes", "MemoryCurrent": "[not set]", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Nice": "0", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "PermissionsStartOnly": "no", "RemainAfterExit": "no", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeMaxUSec": "infinity", "SecureBits": "0", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "null", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StatusErrno": "0", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "[not set]", "TasksMax": "203631", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Type": "dbus", "UID": "[not set]", "UMask": "0022", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2021-11-03T17:00:54.916 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:54 +0000 (0:00:00.355) 0:05:12.503 **** 2021-11-03T17:00:55.229 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ 2021-11-03T17:00:55.231 INFO:teuthology.task.ansible.out:fatal: [smithi027.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2021-11-03T17:00:55.252 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:55 +0000 (0:00:00.335) 0:05:12.838 **** 2021-11-03T17:00:55.551 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2021-11-03T17:00:55.553 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2021-11-03T17:00:55.574 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:55 +0000 (0:00:00.321) 0:05:13.160 **** 2021-11-03T17:00:55.603 INFO:teuthology.task.ansible.out: TASK [testnode : Install abrt] ************************************************* 2021-11-03T17:00:55.604 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:55.625 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:55 +0000 (0:00:00.051) 0:05:13.212 **** 2021-11-03T17:00:55.653 INFO:teuthology.task.ansible.out: TASK [testnode : Enable abrt-auto-reporting] *********************************** 2021-11-03T17:00:55.655 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:55.675 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:55 +0000 (0:00:00.049) 0:05:13.261 **** 2021-11-03T17:00:55.702 INFO:teuthology.task.ansible.out: TASK [testnode : Set OpenGPGCheck in abrt-action-save-package-data.conf] ******* 2021-11-03T17:00:55.704 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:55.724 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:55 +0000 (0:00:00.048) 0:05:13.310 **** 2021-11-03T17:00:55.762 INFO:teuthology.task.ansible.out: TASK [testnode : Set ProcessUnpackaged in abrt-action-save-package-data.conf] *** skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:55.783 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:55 +0000 (0:00:00.059) 0:05:13.369 **** 2021-11-03T17:00:55.820 INFO:teuthology.task.ansible.out: TASK [testnode : Restart abrtd] ************************************************ 2021-11-03T17:00:55.822 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:55.842 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:55 +0000 (0:00:00.059) 0:05:13.429 **** 2021-11-03T17:00:55.869 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2021-11-03T17:00:55.871 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:55.891 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:55 +0000 (0:00:00.048) 0:05:13.478 **** 2021-11-03T17:00:55.918 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2021-11-03T17:00:55.940 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:55 +0000 (0:00:00.048) 0:05:13.526 **** 2021-11-03T17:00:55.967 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2021-11-03T17:00:55.969 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:55.989 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:55 +0000 (0:00:00.049) 0:05:13.576 **** 2021-11-03T17:00:56.016 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2021-11-03T17:00:56.018 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:56.038 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:56 +0000 (0:00:00.049) 0:05:13.625 **** 2021-11-03T17:00:56.076 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2021-11-03T17:00:56.078 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2021-11-03T17:00:56.099 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:56 +0000 (0:00:00.060) 0:05:13.685 **** 2021-11-03T17:00:56.126 INFO:teuthology.task.ansible.out: 2021-11-03T17:00:56.127 INFO:teuthology.task.ansible.out:TASK [testnode : Install python-apt] ******************************************* skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:56.148 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:56 +0000 (0:00:00.049) 0:05:13.734 **** 2021-11-03T17:00:56.175 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2021-11-03T17:00:56.197 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:56 +0000 (0:00:00.049) 0:05:13.784 **** 2021-11-03T17:00:56.224 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2021-11-03T17:00:56.226 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:56.246 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:56 +0000 (0:00:00.049) 0:05:13.833 **** 2021-11-03T17:00:56.284 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2021-11-03T17:00:56.286 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:56.306 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:56 +0000 (0:00:00.059) 0:05:13.893 **** 2021-11-03T17:00:56.344 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2021-11-03T17:00:56.346 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:56.369 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:56 +0000 (0:00:00.062) 0:05:13.956 **** 2021-11-03T17:00:56.397 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:56.422 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:56 +0000 (0:00:00.053) 0:05:14.009 **** 2021-11-03T17:00:56.702 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:56.726 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:56 +0000 (0:00:00.303) 0:05:14.312 **** 2021-11-03T17:00:56.754 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:56.776 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:56 +0000 (0:00:00.049) 0:05:14.362 **** 2021-11-03T17:00:56.803 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2021-11-03T17:00:56.805 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:56.826 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:56 +0000 (0:00:00.050) 0:05:14.413 **** 2021-11-03T17:00:56.855 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages with --no-install-recommends] **************** 2021-11-03T17:00:56.857 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:56.877 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:56 +0000 (0:00:00.050) 0:05:14.463 **** 2021-11-03T17:00:56.903 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2021-11-03T17:00:56.905 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:56.927 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:56 +0000 (0:00:00.050) 0:05:14.513 **** 2021-11-03T17:00:56.954 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2021-11-03T17:00:56.956 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:56.976 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:56 +0000 (0:00:00.048) 0:05:14.562 **** 2021-11-03T17:00:57.002 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2021-11-03T17:00:57.004 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:57.025 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:57 +0000 (0:00:00.049) 0:05:14.611 **** 2021-11-03T17:00:57.052 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2021-11-03T17:00:57.054 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:57.073 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:57 +0000 (0:00:00.048) 0:05:14.660 **** 2021-11-03T17:00:57.100 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2021-11-03T17:00:57.102 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:57.123 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:57 +0000 (0:00:00.048) 0:05:14.709 **** 2021-11-03T17:00:57.150 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:57.172 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:57 +0000 (0:00:00.049) 0:05:14.759 **** 2021-11-03T17:00:57.199 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:57.221 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:57 +0000 (0:00:00.049) 0:05:14.808 **** 2021-11-03T17:00:57.249 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2021-11-03T17:00:57.250 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:57.270 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:57 +0000 (0:00:00.048) 0:05:14.857 **** 2021-11-03T17:00:57.297 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2021-11-03T17:00:57.299 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:57.319 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:57 +0000 (0:00:00.049) 0:05:14.906 **** 2021-11-03T17:00:57.347 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2021-11-03T17:00:57.348 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:57.368 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:57 +0000 (0:00:00.048) 0:05:14.955 **** 2021-11-03T17:00:57.395 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2021-11-03T17:00:57.397 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:57.417 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:57 +0000 (0:00:00.048) 0:05:15.004 **** 2021-11-03T17:00:57.444 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:57.466 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:57 +0000 (0:00:00.049) 0:05:15.053 **** 2021-11-03T17:00:57.493 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2021-11-03T17:00:57.495 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:57.515 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:57 +0000 (0:00:00.049) 0:05:15.102 **** 2021-11-03T17:00:57.543 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2021-11-03T17:00:57.545 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:57.564 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:57 +0000 (0:00:00.049) 0:05:15.151 **** 2021-11-03T17:00:57.591 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2021-11-03T17:00:57.593 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:57.614 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:57 +0000 (0:00:00.049) 0:05:15.200 **** 2021-11-03T17:00:57.641 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:57.664 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:57 +0000 (0:00:00.050) 0:05:15.251 **** 2021-11-03T17:00:57.692 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2021-11-03T17:00:57.694 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:57.715 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:57 +0000 (0:00:00.050) 0:05:15.301 **** 2021-11-03T17:00:57.742 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2021-11-03T17:00:57.744 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:57.764 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:57 +0000 (0:00:00.049) 0:05:15.351 **** 2021-11-03T17:00:57.802 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2021-11-03T17:00:57.804 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:57.827 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:57 +0000 (0:00:00.062) 0:05:15.413 **** 2021-11-03T17:00:57.855 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-11-03T17:00:57.857 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:57.877 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:57 +0000 (0:00:00.050) 0:05:15.463 **** 2021-11-03T17:00:58.097 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-11-03T17:00:58.099 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/bin/lspci", "gid": 0, "group": "root", "mode": "0777", "owner": "root", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 11, "src": "/sbin/lspci", "state": "link", "uid": 0} 2021-11-03T17:00:58.122 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:58 +0000 (0:00:00.243) 0:05:15.707 **** 2021-11-03T17:00:58.338 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-11-03T17:00:58.340 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/ltp/testcases/bin", "secontext": "unconfined_u:object_r:lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2021-11-03T17:00:58.361 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:58 +0000 (0:00:00.240) 0:05:15.948 **** 2021-11-03T17:00:58.572 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2021-11-03T17:00:58.574 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/lib/ltp/testcases/bin/fsstress", "src": "/usr/bin/fsstress"} 2021-11-03T17:00:58.601 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:58 +0000 (0:00:00.238) 0:05:16.187 **** 2021-11-03T17:00:58.629 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:58.660 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:58 +0000 (0:00:00.059) 0:05:16.246 **** 2021-11-03T17:00:58.688 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2021-11-03T17:00:58.690 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:58.712 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:58 +0000 (0:00:00.052) 0:05:16.299 **** 2021-11-03T17:00:58.742 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2021-11-03T17:00:58.744 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:58.766 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:58 +0000 (0:00:00.053) 0:05:16.353 **** 2021-11-03T17:00:58.796 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-11-03T17:00:58.798 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:58.820 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:58 +0000 (0:00:00.053) 0:05:16.407 **** 2021-11-03T17:00:58.851 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-11-03T17:00:58.852 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:58.884 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:58 +0000 (0:00:00.064) 0:05:16.471 **** 2021-11-03T17:00:58.912 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:58.934 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:58 +0000 (0:00:00.050) 0:05:16.521 **** 2021-11-03T17:00:58.963 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2021-11-03T17:00:58.965 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:58.986 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:58 +0000 (0:00:00.051) 0:05:16.573 **** 2021-11-03T17:00:59.014 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2021-11-03T17:00:59.016 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:59.040 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:59 +0000 (0:00:00.053) 0:05:16.626 **** 2021-11-03T17:00:59.068 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:59.091 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:59 +0000 (0:00:00.051) 0:05:16.677 **** 2021-11-03T17:00:59.118 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2021-11-03T17:00:59.120 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:59.142 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:59 +0000 (0:00:00.051) 0:05:16.729 **** 2021-11-03T17:00:59.170 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2021-11-03T17:00:59.172 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:59.192 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:59 +0000 (0:00:00.049) 0:05:16.779 **** 2021-11-03T17:00:59.220 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2021-11-03T17:00:59.221 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:59.242 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:59 +0000 (0:00:00.049) 0:05:16.829 **** 2021-11-03T17:00:59.269 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:59.293 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:59 +0000 (0:00:00.050) 0:05:16.880 **** 2021-11-03T17:00:59.322 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:59.347 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:59 +0000 (0:00:00.053) 0:05:16.933 **** 2021-11-03T17:00:59.386 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2021-11-03T17:00:59.388 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:59.413 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:59 +0000 (0:00:00.067) 0:05:17.000 **** 2021-11-03T17:00:59.441 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2021-11-03T17:00:59.444 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:59.472 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:59 +0000 (0:00:00.059) 0:05:17.059 **** 2021-11-03T17:00:59.499 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2021-11-03T17:00:59.501 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:59.522 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:59 +0000 (0:00:00.049) 0:05:17.108 **** 2021-11-03T17:00:59.549 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2021-11-03T17:00:59.551 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:59.575 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:59 +0000 (0:00:00.053) 0:05:17.161 **** 2021-11-03T17:00:59.602 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2021-11-03T17:00:59.603 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:59.624 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:59 +0000 (0:00:00.048) 0:05:17.210 **** 2021-11-03T17:00:59.651 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2021-11-03T17:00:59.653 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:59.673 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:59 +0000 (0:00:00.048) 0:05:17.259 **** 2021-11-03T17:00:59.699 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2021-11-03T17:00:59.701 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:59.722 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:59 +0000 (0:00:00.048) 0:05:17.308 **** 2021-11-03T17:00:59.750 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2021-11-03T17:00:59.751 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:00:59.771 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:59 +0000 (0:00:00.049) 0:05:17.358 **** 2021-11-03T17:00:59.809 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-11-03T17:00:59.811 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2021-11-03T17:00:59.831 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:59 +0000 (0:00:00.060) 0:05:17.418 **** 2021-11-03T17:00:59.871 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2021-11-03T17:00:59.872 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2021-11-03T17:00:59.882 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2021-11-03T17:00:59.893 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2021-11-03T17:00:59.894 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2021-11-03T17:00:59.904 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} 2021-11-03T17:00:59.914 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2021-11-03T17:00:59.916 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2021-11-03T17:00:59.941 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:00:59 +0000 (0:00:00.109) 0:05:17.528 **** 2021-11-03T17:01:00.147 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2021-11-03T17:01:00.149 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003305", "end": "2021-11-03 17:01:00.126646", "rc": 0, "start": "2021-11-03 17:01:00.123341", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/virtualenv/bin/teuthology-lock"]} 2021-11-03T17:01:00.174 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:00 +0000 (0:00:00.232) 0:05:17.761 **** 2021-11-03T17:01:00.202 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2021-11-03T17:01:00.204 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:00.228 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:00 +0000 (0:00:00.053) 0:05:17.815 **** 2021-11-03T17:01:00.256 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:00.283 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:00 +0000 (0:00:00.054) 0:05:17.869 **** 2021-11-03T17:01:00.310 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2021-11-03T17:01:00.312 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:00.333 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:00 +0000 (0:00:00.050) 0:05:17.920 **** 2021-11-03T17:01:00.361 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2021-11-03T17:01:00.363 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:00.384 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:00 +0000 (0:00:00.051) 0:05:17.971 **** 2021-11-03T17:01:02.127 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2021-11-03T17:01:02.129 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": []} 2021-11-03T17:01:02.149 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:02 +0000 (0:00:01.764) 0:05:19.736 **** 2021-11-03T17:01:02.187 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-11-03T17:01:02.189 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930249461760, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227111685, 'block_used': 12983423, 'inode_total': 61054976, 'inode_available': 60985860, 'inode_used': 69116, 'uuid': 'fcc12580-8f3b-4cec-aef7-e7427fa239b7'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227111685, "block_size": 4096, "block_total": 240095108, "block_used": 12983423, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60985860, "inode_total": 61054976, "inode_used": 69116, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930249461760, "size_total": 983429562368, "uuid": "fcc12580-8f3b-4cec-aef7-e7427fa239b7"}} 2021-11-03T17:01:02.224 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:02 +0000 (0:00:00.075) 0:05:19.811 **** 2021-11-03T17:01:02.430 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2021-11-03T17:01:02.432 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.005230", "end": "2021-11-03 17:01:02.411961", "rc": 0, "start": "2021-11-03 17:01:02.406731", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-03T17:01:02.454 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:02 +0000 (0:00:00.229) 0:05:20.040 **** 2021-11-03T17:01:02.492 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2021-11-03T17:01:02.493 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930249461760, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227111685, 'block_used': 12983423, 'inode_total': 61054976, 'inode_available': 60985860, 'inode_used': 69116, 'uuid': 'fcc12580-8f3b-4cec-aef7-e7427fa239b7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227111685, "block_size": 4096, "block_total": 240095108, "block_used": 12983423, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60985860, "inode_total": 61054976, "inode_used": 69116, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930249461760, "size_total": 983429562368, "uuid": "fcc12580-8f3b-4cec-aef7-e7427fa239b7"}, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:02.516 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:02 +0000 (0:00:00.062) 0:05:20.103 **** 2021-11-03T17:01:02.544 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2021-11-03T17:01:02.565 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:02 +0000 (0:00:00.049) 0:05:20.152 **** 2021-11-03T17:01:04.345 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2021-11-03T17:01:04.346 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-kBvyGKfbce35ZCwg0Im3hUO4Uud7ClF1UJtaLnaoX8huX4geRm1HhmzsaISSWxdu'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.547197", "end": "2021-11-03 17:01:04.310704", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-kBvyGKfbce35ZCwg0Im3hUO4Uud7ClF1UJtaLnaoX8huX4geRm1HhmzsaISSWxdu"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-03 17:01:02.763507", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-11-03T17:01:05.601 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'virtual': 1, 'links': {'ids': ['lvm-pv-uuid-0UkHoz-2yHW-RswC-03Cf-O1ga-LdOY-tzAPah', 'nvme-INTEL_SSDPEDMD400G4_CVFT5331001C400BGN', 'nvme-nvme.8086-43564654353333313030314334303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': [], 'labels': [], 'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4']}, 'vendor': None, 'model': 'INTEL SSDPEDMD400G4', 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': 'none', 'sectors': '781422768', 'sectorsize': '512', 'size': '372.61 GB', 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4']}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.063746", "end": "2021-11-03 17:01:05.577576", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-0UkHoz-2yHW-RswC-03Cf-O1ga-LdOY-tzAPah", "nvme-INTEL_SSDPEDMD400G4_CVFT5331001C400BGN", "nvme-nvme.8086-43564654353333313030314334303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-03 17:01:04.513830", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nGPT data structures destroyed! 2021-11-03T17:01:05.603 INFO:teuthology.task.ansible.out:You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-11-03T17:01:06.846 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-kBvyGKfbce35ZCwg0Im3hUO4Uud7ClF1vm8VTb5dv8ZDV38QsRPcEX6tBqdCWqeE'], 'uuids': ['75e39281-f282-494f-b2f0-47fb819e33b3'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '31252480', 'sectorsize': '512', 'size': '14.90 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.047504", "end": "2021-11-03 17:01:06.817658", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-kBvyGKfbce35ZCwg0Im3hUO4Uud7ClF1vm8VTb5dv8ZDV38QsRPcEX6tBqdCWqeE"], "labels": [], "masters": [], "uuids": ["75e39281-f282-494f-b2f0-47fb819e33b3"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-03 17:01:05.770154", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-11-03T17:01:08.091 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-kBvyGKfbce35ZCwg0Im3hUO4Uud7ClF1R12MX5MaemT32hc3dGYRqPglqEg88F2l'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.056191", "end": "2021-11-03 17:01:08.069341", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-kBvyGKfbce35ZCwg0Im3hUO4Uud7ClF1R12MX5MaemT32hc3dGYRqPglqEg88F2l"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-03 17:01:07.013150", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-11-03T17:01:09.340 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-kBvyGKfbce35ZCwg0Im3hUO4Uud7ClF1dfJWmudcUyMVyCPU6Z39ULXrZZsyBPlM'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.049435", "end": "2021-11-03 17:01:09.311120", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-kBvyGKfbce35ZCwg0Im3hUO4Uud7ClF1dfJWmudcUyMVyCPU6Z39ULXrZZsyBPlM"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-03 17:01:08.261685", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-11-03T17:01:09.341 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'virtual': 1, 'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W4H3N7', 'scsi-0ATA_ST1000NM0033-9ZM_Z1W4H3N7', 'scsi-1ATA_ST1000NM0033-9ZM173_Z1W4H3N7', 'scsi-35000c500871b45cd', 'scsi-SATA_ST1000NM0033-9ZM_Z1W4H3N7', 'wwn-0x5000c500871b45cd'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': 'ATA', 'model': 'ST1000NM0033-9ZM', 'sas_address': None, 'sas_device_handle': None, 'serial': 'Z1W4H3N7', 'removable': '0', 'support_discard': '0', 'wwn': '0x5000c500871b45cd', 'partitions': {'sda1': {'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W4H3N7-part1', 'scsi-0ATA_ST1000NM0033-9ZM_Z1W4H3N7-part1', 'scsi-1ATA_ST1000NM0033-9ZM173_Z1W4H3N7-part1', 'scsi-35000c500871b45cd-part1', 'scsi-SATA_ST1000NM0033-9ZM_Z1W4H3N7-part1', 'wwn-0x5000c500871b45cd-part1'], 'uuids': ['fcc12580-8f3b-4cec-aef7-e7427fa239b7'], 'labels': [], 'masters': []}, 'start': '2048', 'sectors': '1953522688', 'sectorsize': 512, 'size': '931.51 GB', 'uuid': 'fcc12580-8f3b-4cec-aef7-e7427fa239b7', 'holders': []}}, 'rotational': '1', 'scheduler_mode': 'mq-deadline', 'sectors': '1953525168', 'sectorsize': '512', 'size': '931.51 GB', 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'holders': []}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4H3N7", "scsi-0ATA_ST1000NM0033-9ZM_Z1W4H3N7", "scsi-1ATA_ST1000NM0033-9ZM173_Z1W4H3N7", "scsi-35000c500871b45cd", "scsi-SATA_ST1000NM0033-9ZM_Z1W4H3N7", "wwn-0x5000c500871b45cd"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4H3N7-part1", "scsi-0ATA_ST1000NM0033-9ZM_Z1W4H3N7-part1", "scsi-1ATA_ST1000NM0033-9ZM173_ 2021-11-03T17:01:09.343 INFO:teuthology.task.ansible.out:Z1W4H3N7-part1", "scsi-35000c500871b45cd-part1", "scsi-SATA_ST1000NM0033-9ZM_Z1W4H3N7-part1", "wwn-0x5000c500871b45cd-part1"], "labels": [], "masters": [], "uuids": ["fcc12580-8f3b-4cec-aef7-e7427fa239b7"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "fcc12580-8f3b-4cec-aef7-e7427fa239b7"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4H3N7", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500871b45cd"}}, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:10.618 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-kBvyGKfbce35ZCwg0Im3hUO4Uud7ClF1gJd8hg3h22d7CNDWX9DG6kRfHHtqT3W2'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.075628", "end": "2021-11-03 17:01:10.593394", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-kBvyGKfbce35ZCwg0Im3hUO4Uud7ClF1gJd8hg3h22d7CNDWX9DG6kRfHHtqT3W2"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-03 17:01:09.517766", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot or after you\nrun partprobe(8) or kpartx(8)\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot or after you", "run partprobe(8) or kpartx(8)", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2021-11-03T17:01:10.645 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:10 +0000 (0:00:08.080) 0:05:28.232 **** 2021-11-03T17:01:11.026 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2021-11-03T17:01:11.028 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'dm-1', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_2', 'dm-uuid-LVM-kBvyGKfbce35ZCwg0Im3hUO4Uud7ClF1UJtaLnaoX8huX4geRm1HhmzsaISSWxdu'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.150607", "end": "2021-11-03 17:01:11.009052", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-kBvyGKfbce35ZCwg0Im3hUO4Uud7ClF1UJtaLnaoX8huX4geRm1HhmzsaISSWxdu"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-03 17:01:10.858445", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.14777 s, 781 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.14777 s, 781 MB/s"], "stdout": "", "stdout_lines": []} 2021-11-03T17:01:11.270 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'nvme0n1', 'value': {'virtual': 1, 'links': {'ids': ['lvm-pv-uuid-0UkHoz-2yHW-RswC-03Cf-O1ga-LdOY-tzAPah', 'nvme-INTEL_SSDPEDMD400G4_CVFT5331001C400BGN', 'nvme-nvme.8086-43564654353333313030314334303042474e-494e54454c205353445045444d443430304734-00000001'], 'uuids': [], 'labels': [], 'masters': ['dm-0', 'dm-1', 'dm-2', 'dm-3', 'dm-4']}, 'vendor': None, 'model': 'INTEL SSDPEDMD400G4', 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': 'none', 'sectors': '781422768', 'sectorsize': '512', 'size': '372.61 GB', 'host': 'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', 'holders': ['vg_nvme-lv_2', 'vg_nvme-lv_5', 'vg_nvme-lv_3', 'vg_nvme-lv_1', 'vg_nvme-lv_4']}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.039155", "end": "2021-11-03 17:01:11.228408", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_2", "vg_nvme-lv_5", "vg_nvme-lv_3", "vg_nvme-lv_1", "vg_nvme-lv_4"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-0UkHoz-2yHW-RswC-03Cf-O1ga-LdOY-tzAPah", "nvme-INTEL_SSDPEDMD400G4_CVFT5331001C400BGN", "nvme-nvme.8086-43564654353333313030314334303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-03 17:01:11.189253", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.036358 s, 3.2 GB/s", "stde 2021-11-03T17:01:11.277 INFO:teuthology.task.ansible.out:rr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.036358 s, 3.2 GB/s"], "stdout": "", "stdout_lines": []} 2021-11-03T17:01:11.604 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'dm-4', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_5', 'dm-uuid-LVM-kBvyGKfbce35ZCwg0Im3hUO4Uud7ClF1vm8VTb5dv8ZDV38QsRPcEX6tBqdCWqeE'], 'uuids': ['75e39281-f282-494f-b2f0-47fb819e33b3'], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '31252480', 'sectorsize': '512', 'size': '14.90 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.150822", "end": "2021-11-03 17:01:11.580358", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-kBvyGKfbce35ZCwg0Im3hUO4Uud7ClF1vm8VTb5dv8ZDV38QsRPcEX6tBqdCWqeE"], "labels": [], "masters": [], "uuids": ["75e39281-f282-494f-b2f0-47fb819e33b3"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-03 17:01:11.429536", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.147983 s, 779 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.147983 s, 779 MB/s"], "stdout": "", "stdout_lines": []} 2021-11-03T17:01:11.918 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'dm-2', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_3', 'dm-uuid-LVM-kBvyGKfbce35ZCwg0Im3hUO4Uud7ClF1R12MX5MaemT32hc3dGYRqPglqEg88F2l'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.141816", "end": "2021-11-03 17:01:11.900194", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-kBvyGKfbce35ZCwg0Im3hUO4Uud7ClF1R12MX5MaemT32hc3dGYRqPglqEg88F2l"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-03 17:01:11.758378", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.13898 s, 830 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.13898 s, 830 MB/s"], "stdout": "", "stdout_lines": []} 2021-11-03T17:01:12.245 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'dm-0', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_1', 'dm-uuid-LVM-kBvyGKfbce35ZCwg0Im3hUO4Uud7ClF1dfJWmudcUyMVyCPU6Z39ULXrZZsyBPlM'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.143912", "end": "2021-11-03 17:01:12.224236", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-kBvyGKfbce35ZCwg0Im3hUO4Uud7ClF1dfJWmudcUyMVyCPU6Z39ULXrZZsyBPlM"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-03 17:01:12.080324", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.141186 s, 817 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.141186 s, 817 MB/s"], "stdout": "", "stdout_lines": []} 2021-11-03T17:01:12.257 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'key': 'sda', 'value': {'virtual': 1, 'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W4H3N7', 'scsi-0ATA_ST1000NM0033-9ZM_Z1W4H3N7', 'scsi-1ATA_ST1000NM0033-9ZM173_Z1W4H3N7', 'scsi-35000c500871b45cd', 'scsi-SATA_ST1000NM0033-9ZM_Z1W4H3N7', 'wwn-0x5000c500871b45cd'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': 'ATA', 'model': 'ST1000NM0033-9ZM', 'sas_address': None, 'sas_device_handle': None, 'serial': 'Z1W4H3N7', 'removable': '0', 'support_discard': '0', 'wwn': '0x5000c500871b45cd', 'partitions': {'sda1': {'links': {'ids': ['ata-ST1000NM0033-9ZM173_Z1W4H3N7-part1', 'scsi-0ATA_ST1000NM0033-9ZM_Z1W4H3N7-part1', 'scsi-1ATA_ST1000NM0033-9ZM173_Z1W4H3N7-part1', 'scsi-35000c500871b45cd-part1', 'scsi-SATA_ST1000NM0033-9ZM_Z1W4H3N7-part1', 'wwn-0x5000c500871b45cd-part1'], 'uuids': ['fcc12580-8f3b-4cec-aef7-e7427fa239b7'], 'labels': [], 'masters': []}, 'start': '2048', 'sectors': '1953522688', 'sectorsize': 512, 'size': '931.51 GB', 'uuid': 'fcc12580-8f3b-4cec-aef7-e7427fa239b7', 'holders': []}}, 'rotational': '1', 'scheduler_mode': 'mq-deadline', 'sectors': '1953525168', 'sectorsize': '512', 'size': '931.51 GB', 'host': 'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', 'holders': []}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4H3N7", "scsi-0ATA_ST1000NM0033-9ZM_Z1W4H3N7", "scsi-1ATA_ST1000NM0033-9ZM173_Z1W4H3N7", "scsi-35000c500871b45cd", "scsi-SATA_ST1000NM0033-9ZM_Z1W4H3N7", "wwn-0x5000c500871b45cd"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W4H3N7-part1", "scsi-0ATA_ST1000NM0033-9ZM_Z1W4H3N7-part1", "scsi-1ATA_ST1000NM0033-9ZM173_ 2021-11-03T17:01:12.259 INFO:teuthology.task.ansible.out:Z1W4H3N7-part1", "scsi-35000c500871b45cd-part1", "scsi-SATA_ST1000NM0033-9ZM_Z1W4H3N7-part1", "wwn-0x5000c500871b45cd-part1"], "labels": [], "masters": [], "uuids": ["fcc12580-8f3b-4cec-aef7-e7427fa239b7"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "fcc12580-8f3b-4cec-aef7-e7427fa239b7"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "mq-deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W4H3N7", "size": "931.51 GB", "support_discard": "0", "vendor": "ATA", "virtual": 1, "wwn": "0x5000c500871b45cd"}}, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:12.570 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'dm-3', 'value': {'virtual': 1, 'links': {'ids': ['dm-name-vg_nvme-lv_4', 'dm-uuid-LVM-kBvyGKfbce35ZCwg0Im3hUO4Uud7ClF1gJd8hg3h22d7CNDWX9DG6kRfHHtqT3W2'], 'uuids': [], 'labels': [], 'masters': []}, 'vendor': None, 'model': None, 'sas_address': None, 'sas_device_handle': None, 'removable': '0', 'support_discard': '512', 'partitions': {}, 'rotational': '0', 'scheduler_mode': '', 'sectors': '187490304', 'sectorsize': '512', 'size': '89.40 GB', 'host': '', 'holders': []}}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.139644", "end": "2021-11-03 17:01:12.549738", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-kBvyGKfbce35ZCwg0Im3hUO4Uud7ClF1gJd8hg3h22d7CNDWX9DG6kRfHHtqT3W2"], "labels": [], "masters": [], "uuids": []}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2021-11-03 17:01:12.410094", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB, 110 MiB) copied, 0.136866 s, 843 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB, 110 MiB) copied, 0.136866 s, 843 MB/s"], "stdout": "", "stdout_lines": []} 2021-11-03T17:01:12.599 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:12 +0000 (0:00:01.953) 0:05:30.186 **** 2021-11-03T17:01:12.847 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2021-11-03T17:01:12.849 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.049134", "end": "2021-11-03 17:01:12.818882", "rc": 0, "start": "2021-11-03 17:01:12.769748", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-03T17:01:12.870 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:12 +0000 (0:00:00.270) 0:05:30.456 **** 2021-11-03T17:01:13.109 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2021-11-03T17:01:13.111 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.051327", "end": "2021-11-03 17:01:13.092942", "rc": 0, "start": "2021-11-03 17:01:13.041615", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-03T17:01:13.142 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:13 +0000 (0:00:00.272) 0:05:30.728 **** 2021-11-03T17:01:13.173 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2021-11-03T17:01:13.199 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:13 +0000 (0:00:00.057) 0:05:30.786 **** 2021-11-03T17:01:13.227 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2021-11-03T17:01:13.229 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:13.253 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:13 +0000 (0:00:00.053) 0:05:30.839 **** 2021-11-03T17:01:13.281 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:13.302 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:13 +0000 (0:00:00.049) 0:05:30.889 **** 2021-11-03T17:01:13.329 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-11-03T17:01:13.331 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:13.356 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:13 +0000 (0:00:00.053) 0:05:30.942 **** 2021-11-03T17:01:13.384 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2021-11-03T17:01:13.385 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:13.406 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:13 +0000 (0:00:00.050) 0:05:30.993 **** 2021-11-03T17:01:13.603 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2021-11-03T17:01:13.605 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2021-11-03T17:01:13.627 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:13 +0000 (0:00:00.219) 0:05:31.213 **** 2021-11-03T17:01:13.665 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2021-11-03T17:01:13.668 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'mount': '/', 'device': '/dev/sda1', 'fstype': 'ext4', 'options': 'rw,seclabel,relatime', 'size_total': 983429562368, 'size_available': 930249461760, 'block_size': 4096, 'block_total': 240095108, 'block_available': 227111685, 'block_used': 12983423, 'inode_total': 61054976, 'inode_available': 60985860, 'inode_used': 69116, 'uuid': 'fcc12580-8f3b-4cec-aef7-e7427fa239b7'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227111685, "block_size": 4096, "block_total": 240095108, "block_used": 12983423, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60985860, "inode_total": 61054976, "inode_used": 69116, "mount": "/", "options": "rw,seclabel,relatime", "size_available": 930249461760, "size_total": 983429562368, "uuid": "fcc12580-8f3b-4cec-aef7-e7427fa239b7"}, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:13.690 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:13 +0000 (0:00:00.063) 0:05:31.277 **** 2021-11-03T17:01:13.730 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2021-11-03T17:01:13.732 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:13.753 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:13 +0000 (0:00:00.063) 0:05:31.340 **** 2021-11-03T17:01:13.784 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-11-03T17:01:13.786 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:13.811 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:13 +0000 (0:00:00.057) 0:05:31.397 **** 2021-11-03T17:01:13.838 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-11-03T17:01:13.840 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:13.860 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:13 +0000 (0:00:00.049) 0:05:31.447 **** 2021-11-03T17:01:13.888 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2021-11-03T17:01:13.889 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:13.909 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:13 +0000 (0:00:00.049) 0:05:31.496 **** 2021-11-03T17:01:13.936 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2021-11-03T17:01:13.938 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:13.960 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:13 +0000 (0:00:00.050) 0:05:31.547 **** 2021-11-03T17:01:13.989 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2021-11-03T17:01:13.990 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:14.011 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:14 +0000 (0:00:00.050) 0:05:31.597 **** 2021-11-03T17:01:14.480 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2021-11-03T17:01:14.482 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'vg_nvme', 'value': {'pvs': '/dev/nvme0n1'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2021-11-03T17:01:14.503 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:14 +0000 (0:00:00.492) 0:05:32.090 **** 2021-11-03T17:01:14.954 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2021-11-03T17:01:14.955 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-11-03T17:01:15.304 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-11-03T17:01:15.640 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-11-03T17:01:15.986 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2021-11-03T17:01:16.349 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2021-11-03T17:01:16.373 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:16 +0000 (0:00:01.870) 0:05:33.960 **** 2021-11-03T17:01:16.580 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2021-11-03T17:01:16.582 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2021-11-03T17:01:16.604 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:16 +0000 (0:00:00.230) 0:05:34.190 **** 2021-11-03T17:01:16.830 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2021-11-03T17:01:16.832 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'lv_1', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-11-03T17:01:17.010 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'lv_2', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-11-03T17:01:17.189 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'lv_3', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-11-03T17:01:17.368 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => (item={'key': 'lv_4', 'value': {'vg': 'vg_nvme', 'size': '89.4G', 'scratch_dev': True}}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2021-11-03T17:01:17.378 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'key': 'lv_5', 'value': {'vg': 'vg_nvme', 'size': '14.9G'}}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:17.402 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:17 +0000 (0:00:00.798) 0:05:34.989 **** 2021-11-03T17:01:17.430 INFO:teuthology.task.ansible.out: TASK [testnode : Create filesystems] ******************************************* 2021-11-03T17:01:17.432 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:17.452 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:17 +0000 (0:00:00.049) 0:05:35.039 **** 2021-11-03T17:01:17.481 INFO:teuthology.task.ansible.out: TASK [testnode : Mount filesystems] ******************************************** 2021-11-03T17:01:17.483 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:17.507 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:17 +0000 (0:00:00.054) 0:05:35.093 **** 2021-11-03T17:01:17.712 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2021-11-03T17:01:17.714 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2021-11-03T17:01:17.734 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:17 +0000 (0:00:00.227) 0:05:35.321 **** 2021-11-03T17:01:17.773 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** ok: [smithi027.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2021-11-03T17:01:17.795 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:17 +0000 (0:00:00.060) 0:05:35.381 **** 2021-11-03T17:01:18.084 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* changed: [smithi027.front.sepia.ceph.com] => {"changed": true} 2021-11-03T17:01:18.106 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:18 +0000 (0:00:00.311) 0:05:35.693 **** 2021-11-03T17:01:18.445 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2021-11-03T17:01:18.447 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2021-11-03T17:01:18.469 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:18 +0000 (0:00:00.362) 0:05:36.056 **** 2021-11-03T17:01:18.497 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd] ****************************************** 2021-11-03T17:01:18.499 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:18.520 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:18 +0000 (0:00:00.050) 0:05:36.106 **** 2021-11-03T17:01:18.547 INFO:teuthology.task.ansible.out: TASK [testnode : Install cachefilesd conf file] ******************************** 2021-11-03T17:01:18.550 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:18.571 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:18 +0000 (0:00:00.051) 0:05:36.158 **** 2021-11-03T17:01:18.599 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-11-03T17:01:18.601 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:18.620 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:18 +0000 (0:00:00.049) 0:05:36.207 **** 2021-11-03T17:01:18.649 INFO:teuthology.task.ansible.out: TASK [testnode : Restart cachefilesd] ****************************************** 2021-11-03T17:01:18.650 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:18.671 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:18 +0000 (0:00:00.050) 0:05:36.257 **** 2021-11-03T17:01:18.699 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** 2021-11-03T17:01:18.701 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:18.725 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:18 +0000 (0:00:00.053) 0:05:36.311 **** 2021-11-03T17:01:18.752 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2021-11-03T17:01:18.753 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:18.775 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:18 +0000 (0:00:00.050) 0:05:36.361 **** 2021-11-03T17:01:18.803 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2021-11-03T17:01:18.805 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:18.828 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:18 +0000 (0:00:00.052) 0:05:36.414 **** 2021-11-03T17:01:18.855 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2021-11-03T17:01:18.857 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:18.878 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:18 +0000 (0:00:00.050) 0:05:36.464 **** 2021-11-03T17:01:18.907 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2021-11-03T17:01:18.909 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:18.929 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:18 +0000 (0:00:00.051) 0:05:36.516 **** 2021-11-03T17:01:18.956 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2021-11-03T17:01:18.958 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:18.979 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:18 +0000 (0:00:00.049) 0:05:36.565 **** 2021-11-03T17:01:19.006 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2021-11-03T17:01:19.008 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:19.028 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:19 +0000 (0:00:00.049) 0:05:36.615 **** 2021-11-03T17:01:19.066 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:19.088 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:19 +0000 (0:00:00.059) 0:05:36.674 **** 2021-11-03T17:01:19.115 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2021-11-03T17:01:19.117 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:19.137 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:19 +0000 (0:00:00.049) 0:05:36.724 **** 2021-11-03T17:01:19.175 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** 2021-11-03T17:01:19.176 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:19.200 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:19 +0000 (0:00:00.062) 0:05:36.787 **** 2021-11-03T17:01:19.228 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2021-11-03T17:01:19.230 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:19.250 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:19 +0000 (0:00:00.050) 0:05:36.837 **** 2021-11-03T17:01:19.725 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2021-11-03T17:01:19.728 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "checksum": "a7ecef00073feab34f868065d34e9dac55134241", "dest": "/etc/chrony.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/chrony.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 215, "state": "file", "uid": 0} 2021-11-03T17:01:19.747 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:19 +0000 (0:00:00.496) 0:05:37.334 **** 2021-11-03T17:01:20.069 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2021-11-03T17:01:20.070 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "chronyd", "state": "started", "status": {"ActiveState": "active", "AllowedCPUs": "", "AllowedMemoryNodes": "", "BlockIOAccounting": "no", "BlockIOWeight": "[not set]", "CPUAccounting": "no", "CPUAffinity": "", "CPUQuotaPerSecUSec": "infinity", "CPUQuotaPeriodUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "[not set]", "CPUUsageNSec": "[not set]", "CPUWeight": "[not set]", "ControlGroup": "/system.slice/chronyd.service", "ControlPID": "0", "DefaultMemoryLow": "0", "DefaultMemoryMin": "0", "Delegate": "no", "DevicePolicy": "auto", "EffectiveCPUs": "", "EffectiveMemoryNodes": "", "EnvironmentFiles": "/etc/sysconfig/chronyd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "878", "ExecMainStartTimestamp": "Wed 2021-11-03 16:55:05 UTC", "ExecMainStartTimestampMonotonic": "15632961", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/chronyd ; argv[]=/usr/sbin/chronyd $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStartPost": "{ path=/usr/libexec/chrony-helper ; argv[]=/usr/libexec/chrony-helper update-daemon ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FileDescriptorStoreMax": "0", "GID": "[not set]", "GuessMainPID": "yes", "IOAccounting": "no", "IOSchedulingClass": "0", "IOSchedulingPriority": "0", "IOWeight": "[not set]", "IPAccounting": "no", "IPEgressBytes": "18446744073709551615", "IPEgressPackets": "18446744073709551615", "IPIngressBytes": "18446744073709551615", "IPIngressPackets": "18446744073709551615", "LimitAS": "infinity", "LimitASSoft": "infinity", "LimitCORE": "infinity", "LimitCORESoft": "infinity", "LimitCPU": "infinity", "LimitCPUSoft": "infinity", "LimitDATA": "infinity", "LimitDATASoft": "infinity", "LimitFSIZE": "infinity", "LimitFSI 2021-11-03T17:01:20.072 INFO:teuthology.task.ansible.out:ZESoft": "infinity", "LimitLOCKS": "infinity", "LimitLOCKSSoft": "infinity", "LimitMEMLOCK": "65536", "LimitMEMLOCKSoft": "65536", "LimitMSGQUEUE": "819200", "LimitMSGQUEUESoft": "819200", "LimitNICE": "0", "LimitNICESoft": "0", "LimitNOFILE": "262144", "LimitNOFILESoft": "1024", "LimitNPROC": "127269", "LimitNPROCSoft": "127269", "LimitRSS": "infinity", "LimitRSSSoft": "infinity", "LimitRTPRIO": "0", "LimitRTPRIOSoft": "0", "LimitRTTIME": "infinity", "LimitRTTIMESoft": "infinity", "LimitSIGPENDING": "127269", "LimitSIGPENDINGSoft": "127269", "LimitSTACK": "infinity", "LimitSTACKSoft": "8388608", "LogLevelMax": "-1", "LogRateLimitBurst": "0", "LogRateLimitIntervalUSec": "0", "MainPID": "878", "MemoryAccounting": "yes", "MemoryCurrent": "1961984", "MemoryHigh": "infinity", "MemoryLimit": "infinity", "MemoryLow": "0", "MemoryMax": "infinity", "MemoryMin": "0", "MemorySwapMax": "infinity", "NFileDescriptorStore": "0", "NRestarts": "0", "NUMAMask": "", "NUMAPolicy": "n/a", "Nice": "0", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "PIDFile": "/run/chrony/chronyd.pid", "PermissionsStartOnly": "no", "RemainAfterExit": "no", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeMaxUSec": "infinity", "SecureBits": "0", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardInputData": "", "StandardOutput": "journal", "StartupBlockIOWeight": "[not set]", "StartupCPUShares": "[not set]", "StartupCPUWeight": "[not set]", "StartupIOWeight": "[not set]", "StatusErrno": "0", "SyslogFacility": "3", "SyslogLevel": "6", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "yes", "TasksCurrent": "1", "TasksMax": "203631", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Type": "forking", "UID": "[not set]", "UMask": "0022", "WatchdogTimestamp": "Wed 2021-11-03 1 2021-11-03T17:01:20.074 INFO:teuthology.task.ansible.out:6:55:05 UTC", "WatchdogTimestampMonotonic": "15632985", "WatchdogUSec": "0"}} 2021-11-03T17:01:20.092 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:20 +0000 (0:00:00.344) 0:05:37.679 **** 2021-11-03T17:01:20.119 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-11-03T17:01:20.121 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"ansible_facts": {"pip_executable": "pip", "pip_version": "python-pip"}, "changed": false} 2021-11-03T17:01:20.141 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:20 +0000 (0:00:00.049) 0:05:37.728 **** 2021-11-03T17:01:20.183 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-11-03T17:01:20.185 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"ansible_facts": {"ansible_python_interpreter": "/usr/bin/python3", "pip_executable": "pip3", "pip_version": "python3-pip"}, "changed": false} 2021-11-03T17:01:20.205 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:20 +0000 (0:00:00.063) 0:05:37.792 **** 2021-11-03T17:01:20.233 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2021-11-03T17:01:20.235 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:20.255 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:20 +0000 (0:00:00.049) 0:05:37.841 **** 2021-11-03T17:01:20.282 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2021-11-03T17:01:20.283 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:20.303 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:20 +0000 (0:00:00.048) 0:05:37.890 **** 2021-11-03T17:01:20.341 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on zypper based systems.] ****************** 2021-11-03T17:01:20.343 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:20.362 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:20 +0000 (0:00:00.059) 0:05:37.949 **** 2021-11-03T17:01:20.568 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2021-11-03T17:01:20.570 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "gid": 1254, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2021-11-03T17:01:20.591 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:20 +0000 (0:00:00.228) 0:05:38.178 **** 2021-11-03T17:01:21.077 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2021-11-03T17:01:21.078 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1254, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2021-11-03T17:01:21.108 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:21 +0000 (0:00:00.517) 0:05:38.695 **** 2021-11-03T17:01:22.192 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2021-11-03T17:01:22.194 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip3", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.\n", "stderr_lines": ["WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead."], "stdout": "Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages\nRequirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied: remoto>=0.0.35 in /usr/local/lib/python3.6/site-packages", "Requirement already satisfied: execnet in /usr/local/lib/python3.6/site-packages (from remoto>=0.0.35)"], "version": null, "virtualenv": null} 2021-11-03T17:01:22.214 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:22 +0000 (0:00:01.105) 0:05:39.800 **** 2021-11-03T17:01:22.242 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2021-11-03T17:01:22.244 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:22.265 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:22 +0000 (0:00:00.050) 0:05:39.851 **** 2021-11-03T17:01:22.292 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2021-11-03T17:01:22.293 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"ansible_facts": {"interface": "ens1f0"}, "changed": false} 2021-11-03T17:01:22.313 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:22 +0000 (0:00:00.048) 0:05:39.900 **** 2021-11-03T17:01:22.497 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2021-11-03T17:01:22.499 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2021-11-03T17:01:22.519 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:22 +0000 (0:00:00.206) 0:05:40.106 **** 2021-11-03T17:01:22.547 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:22.570 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:22 +0000 (0:00:00.049) 0:05:40.156 **** 2021-11-03T17:01:22.597 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2021-11-03T17:01:22.599 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2021-11-03T17:01:22.620 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:22 +0000 (0:00:00.050) 0:05:40.207 **** 2021-11-03T17:01:22.648 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2021-11-03T17:01:22.649 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:22.670 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:22 +0000 (0:00:00.049) 0:05:40.257 **** 2021-11-03T17:01:22.864 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2021-11-03T17:01:22.866 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2021-11-03T17:01:22.888 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:22 +0000 (0:00:00.217) 0:05:40.475 **** 2021-11-03T17:01:23.081 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2021-11-03T17:01:23.084 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2021-11-03T17:01:23.104 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:23 +0000 (0:00:00.215) 0:05:40.690 **** 2021-11-03T17:01:23.131 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:23.153 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:23 +0000 (0:00:00.049) 0:05:40.740 **** 2021-11-03T17:01:23.191 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2021-11-03T17:01:23.193 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item=ens1f0) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False"} 2021-11-03T17:01:23.202 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=ens1f1) => {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False"} 2021-11-03T17:01:23.204 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2021-11-03T17:01:23.226 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:23 +0000 (0:00:00.073) 0:05:40.813 **** 2021-11-03T17:01:23.265 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2021-11-03T17:01:23.267 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'lo', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:23.276 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'ens1f0', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:23.278 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'ens1f1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "ens1f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:23.288 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno2', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi027.front.sepia.ceph.com] => (item={'changed': False, 'skipped': True, 'skip_reason': 'Conditional result was False', 'item': 'eno1', 'ansible_loop_var': 'item'}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:23.311 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:23 +0000 (0:00:00.084) 0:05:40.898 **** 2021-11-03T17:01:23.338 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2021-11-03T17:01:23.340 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:23.361 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:23 +0000 (0:00:00.050) 0:05:40.948 **** 2021-11-03T17:01:23.388 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2021-11-03T17:01:23.390 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:23.411 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:23 +0000 (0:00:00.049) 0:05:40.998 **** 2021-11-03T17:01:23.439 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2021-11-03T17:01:23.441 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:23.464 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:23 +0000 (0:00:00.052) 0:05:41.050 **** 2021-11-03T17:01:23.502 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2021-11-03T17:01:23.503 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:23.527 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:23 +0000 (0:00:00.063) 0:05:41.114 **** 2021-11-03T17:01:23.555 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2021-11-03T17:01:23.557 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:23.580 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:23 +0000 (0:00:00.052) 0:05:41.167 **** 2021-11-03T17:01:23.608 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-11-03T17:01:23.610 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:23.634 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:23 +0000 (0:00:00.054) 0:05:41.221 **** 2021-11-03T17:01:23.662 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2021-11-03T17:01:23.664 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:23.688 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:23 +0000 (0:00:00.053) 0:05:41.275 **** 2021-11-03T17:01:23.715 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2021-11-03T17:01:23.717 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:23.737 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:23 +0000 (0:00:00.048) 0:05:41.323 **** 2021-11-03T17:01:23.961 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2021-11-03T17:01:23.962 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.005455", "end": "2021-11-03 17:01:23.932798", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2021-11-03 17:01:23.927343", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-03T17:01:24.137 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.003034", "end": "2021-11-03 17:01:24.114804", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2021-11-03 17:01:24.111770", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-03T17:01:25.317 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:01.004284", "end": "2021-11-03 17:01:25.292567", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2021-11-03 17:01:24.288283", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2021-11-03T17:01:25.504 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.003371", "end": "2021-11-03 17:01:25.485865", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2021-11-03 17:01:25.482494", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-03T17:01:25.527 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:25 +0000 (0:00:01.790) 0:05:43.114 **** 2021-11-03T17:01:25.554 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2021-11-03T17:01:25.577 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:25 +0000 (0:00:00.049) 0:05:43.163 **** 2021-11-03T17:01:25.604 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2021-11-03T17:01:25.606 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:25.671 INFO:teuthology.task.ansible.out: PLAY [testnodes,senta,vossi,jenkins_slaves] ************************************ 2021-11-03T17:01:25.688 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:25 +0000 (0:00:00.111) 0:05:43.275 **** 2021-11-03T17:01:25.717 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:25 +0000 (0:00:00.028) 0:05:43.303 **** 2021-11-03T17:01:26.344 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi027.front.sepia.ceph.com] 2021-11-03T17:01:26.376 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] 2021-11-03T17:01:26.403 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:26 +0000 (0:00:00.686) 0:05:43.990 **** 2021-11-03T17:01:26.449 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:26 +0000 (0:00:00.045) 0:05:44.035 **** 2021-11-03T17:01:26.455 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-11-03T17:01:26.457 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:26.481 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:26 +0000 (0:00:00.032) 0:05:44.068 **** 2021-11-03T17:01:26.489 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:26.515 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:26 +0000 (0:00:00.032) 0:05:44.101 **** 2021-11-03T17:01:26.521 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-11-03T17:01:26.522 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"ansible_facts": {"package_manager": "yum"}, "changed": false} 2021-11-03T17:01:26.549 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:26 +0000 (0:00:00.034) 0:05:44.136 **** 2021-11-03T17:01:26.557 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"ansible_facts": {"package_manager": "yum"}, "changed": false} 2021-11-03T17:01:26.581 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:26 +0000 (0:00:00.032) 0:05:44.168 **** 2021-11-03T17:01:26.599 INFO:teuthology.task.ansible.out: TASK [container-host : Including distro specific variables] ******************** 2021-11-03T17:01:26.601 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/yum_systems.yml) => {"ansible_facts": {"container_packages": ["podman", "podman-docker"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/yum_systems.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/yum_systems.yml"} 2021-11-03T17:01:26.627 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:26 +0000 (0:00:00.045) 0:05:44.214 **** 2021-11-03T17:01:26.636 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/yum_systems.yml) => {"ansible_facts": {"container_packages": ["podman", "podman-docker"]}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/yum_systems.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/container-host/vars/yum_systems.yml"} 2021-11-03T17:01:26.667 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:26 +0000 (0:00:00.040) 0:05:44.254 **** 2021-11-03T17:01:32.307 INFO:teuthology.task.ansible.out: TASK [container-host : Install container packages] ***************************** 2021-11-03T17:01:32.309 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: podman-docker-3.2.3-0.11.module_el8.4.0+942+d25aada8.noarch", "Installed: podman-catatonit-3.2.3-0.11.module_el8.4.0+942+d25aada8.x86_64", "Installed: podman-3.2.3-0.11.module_el8.4.0+942+d25aada8.x86_64", "Removed: podman-3.0.1-7.module_el8.4.0+830+8027e1c4.x86_64", "Removed: podman-catatonit-3.0.1-7.module_el8.4.0+830+8027e1c4.x86_64", "Removed: podman-docker-3.0.1-7.module_el8.4.0+830+8027e1c4.noarch"]} 2021-11-03T17:01:32.329 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:32 +0000 (0:00:05.661) 0:05:49.916 **** 2021-11-03T17:01:32.356 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-11-03T17:01:32.358 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"ansible_facts": {"container_service_conf": "/etc/containers/registries.conf"}, "changed": false} 2021-11-03T17:01:32.379 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:32 +0000 (0:00:00.050) 0:05:49.966 **** 2021-11-03T17:01:32.407 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-11-03T17:01:32.409 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:32.429 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:32 +0000 (0:00:00.050) 0:05:50.016 **** 2021-11-03T17:01:32.561 INFO:teuthology.task.ansible.out: TASK [container-host : Install container packages] ***************************** 2021-11-03T17:01:32.563 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "msg": "", "rc": 0, "results": ["Installed: podman-docker-3.2.3-0.11.module_el8.4.0+942+d25aada8.noarch", "Installed: podman-catatonit-3.2.3-0.11.module_el8.4.0+942+d25aada8.x86_64", "Installed: podman-3.2.3-0.11.module_el8.4.0+942+d25aada8.x86_64", "Removed: podman-3.0.1-7.module_el8.4.0+830+8027e1c4.x86_64", "Removed: podman-catatonit-3.0.1-7.module_el8.4.0+830+8027e1c4.x86_64", "Removed: podman-docker-3.0.1-7.module_el8.4.0+830+8027e1c4.noarch"]} 2021-11-03T17:01:32.586 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:32 +0000 (0:00:00.156) 0:05:50.172 **** 2021-11-03T17:01:32.615 INFO:teuthology.task.ansible.out: TASK [container-host : set_fact] *********************************************** 2021-11-03T17:01:32.617 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"ansible_facts": {"container_service_conf": "/etc/containers/registries.conf"}, "changed": false} 2021-11-03T17:01:32.637 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:32 +0000 (0:00:00.051) 0:05:50.224 **** 2021-11-03T17:01:32.664 INFO:teuthology.task.ansible.out: TASK [container-host : Create container_mirror_cert_paths] ********************* 2021-11-03T17:01:32.666 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": false, "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 4096, "state": "directory", "uid": 0} TASK [container-host : set_fact] *********************************************** skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:32.689 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:32 +0000 (0:00:00.051) 0:05:50.275 **** 2021-11-03T17:01:32.852 INFO:teuthology.task.ansible.out: TASK [container-host : Create container_mirror_cert_paths] ********************* 2021-11-03T17:01:32.854 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": false, "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 4096, "state": "directory", "uid": 0} 2021-11-03T17:01:32.886 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:32 +0000 (0:00:00.197) 0:05:50.473 **** 2021-11-03T17:01:32.926 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": false, "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 4096, "state": "directory", "uid": 0} 2021-11-03T17:01:33.126 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": false, "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0755", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 4096, "state": "directory", "uid": 0} 2021-11-03T17:01:33.150 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:33 +0000 (0:00:00.263) 0:05:50.737 **** 2021-11-03T17:01:33.369 INFO:teuthology.task.ansible.out: TASK [container-host : Copy docker-mirror.front.sepia.ceph.com:5000 self-signed cert] *** 2021-11-03T17:01:33.371 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": false, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0644", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "secontext": "system_u:object_r:cert_t:s0", "size": 2334, "state": "file", "uid": 0} 2021-11-03T17:01:33.651 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": false, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0644", "owner": "root", "path": "/etc/docker/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "secontext": "system_u:object_r:cert_t:s0", "size": 2334, "state": "file", "uid": 0} 2021-11-03T17:01:33.798 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": false, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0644", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "secontext": "system_u:object_r:etc_t:s0", "size": 2334, "state": "file", "uid": 0} 2021-11-03T17:01:33.821 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:33 +0000 (0:00:00.671) 0:05:51.408 **** 2021-11-03T17:01:34.092 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => (item=/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000) => {"ansible_loop_var": "item", "changed": false, "checksum": "bcdfbb1de91ef82e0e4ed8af725c9ad52593eb79", "dest": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "gid": 0, "group": "root", "item": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000", "mode": "0644", "owner": "root", "path": "/etc/containers/certs.d/docker-mirror.front.sepia.ceph.com:5000/docker-mirror.crt", "secontext": "system_u:object_r:etc_t:s0", "size": 2334, "state": "file", "uid": 0} 2021-11-03T17:01:34.117 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:34 +0000 (0:00:00.295) 0:05:51.704 **** 2021-11-03T17:01:35.440 INFO:teuthology.task.ansible.out: TASK [container-host : Ensure git is installed] ******************************** 2021-11-03T17:01:35.442 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": []} 2021-11-03T17:01:35.463 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:35 +0000 (0:00:01.346) 0:05:53.050 **** 2021-11-03T17:01:35.838 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "msg": "Nothing to do", "rc": 0, "results": []} 2021-11-03T17:01:35.861 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:35 +0000 (0:00:00.398) 0:05:53.448 **** 2021-11-03T17:01:37.923 INFO:teuthology.task.ansible.out: TASK [container-host : Install registries-conf-ctl] **************************** 2021-11-03T17:01:37.924 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip3", "install", "-U", "git+https://github.com/sebastian-philipp/registries-conf-ctl"], "name": ["git+https://github.com/sebastian-philipp/registries-conf-ctl"], "requirements": null, "state": "latest", "stderr": "WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.\n", "stderr_lines": ["WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead."], "stdout": "Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl\n Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-n0qmzefu-build\nRequirement already up-to-date: toml in /usr/local/lib/python3.6/site-packages (from registries-conf-ctl==1.0)\nRequirement already up-to-date: docopt in /usr/local/lib/python3.6/site-packages (from registries-conf-ctl==1.0)\nInstalling collected packages: registries-conf-ctl\n Found existing installation: registries-conf-ctl 1.0\n Uninstalling registries-conf-ctl-1.0:\n Successfully uninstalled registries-conf-ctl-1.0\n Running setup.py install for registries-conf-ctl: started\n Running setup.py install for registries-conf-ctl: finished with status 'done'\nSuccessfully installed registries-conf-ctl-1.0\n", "stdout_lines": ["Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl", " Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-n0qmzefu-build", "Requirement already up-to-date: toml in /usr/local/lib/python3.6/site-packages (from registries-conf-ctl==1.0)", "Requirement already up-to-date: docopt in /usr/local/lib/python3.6/site-packages (from registries-conf-ctl==1.0)", "Installing collected packages: registries-conf-ctl", " Found existing installation: registries-conf-ctl 1.0", " Uninstalling registries-conf-ctl-1.0:", " Successfully uninstalled registries-conf-ctl-1.0", " Running setup.py in 2021-11-03T17:01:37.926 INFO:teuthology.task.ansible.out:stall for registries-conf-ctl: started", " Running setup.py install for registries-conf-ctl: finished with status 'done'", "Successfully installed registries-conf-ctl-1.0"], "version": null, "virtualenv": null} 2021-11-03T17:01:37.946 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:37 +0000 (0:00:02.084) 0:05:55.532 **** 2021-11-03T17:01:37.974 INFO:teuthology.task.ansible.out: TASK [container-host : Check for docker's daemon.json] ************************* 2021-11-03T17:01:37.976 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:37.999 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:37 +0000 (0:00:00.053) 0:05:55.586 **** 2021-11-03T17:01:38.036 INFO:teuthology.task.ansible.out: TASK [container-host : Create /etc/containers/registries.conf if necessary] **** 2021-11-03T17:01:38.038 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:38.069 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:38 +0000 (0:00:00.070) 0:05:55.656 **** 2021-11-03T17:01:38.365 INFO:teuthology.task.ansible.out: TASK [container-host : Install registries-conf-ctl] **************************** 2021-11-03T17:01:38.367 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip3", "install", "-U", "git+https://github.com/sebastian-philipp/registries-conf-ctl"], "name": ["git+https://github.com/sebastian-philipp/registries-conf-ctl"], "requirements": null, "state": "latest", "stderr": "WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead.\n", "stderr_lines": ["WARNING: Running pip install with root privileges is generally not a good idea. Try `pip3 install --user` instead."], "stdout": "Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl\n Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-uiaqfxyy-build\nRequirement already up-to-date: toml in /usr/local/lib/python3.6/site-packages (from registries-conf-ctl==1.0)\nRequirement already up-to-date: docopt in /usr/local/lib/python3.6/site-packages (from registries-conf-ctl==1.0)\nInstalling collected packages: registries-conf-ctl\n Found existing installation: registries-conf-ctl 1.0\n Uninstalling registries-conf-ctl-1.0:\n Successfully uninstalled registries-conf-ctl-1.0\n Running setup.py install for registries-conf-ctl: started\n Running setup.py install for registries-conf-ctl: finished with status 'done'\nSuccessfully installed registries-conf-ctl-1.0\n", "stdout_lines": ["Collecting git+https://github.com/sebastian-philipp/registries-conf-ctl", " Cloning https://github.com/sebastian-philipp/registries-conf-ctl to ./pip-uiaqfxyy-build", "Requirement already up-to-date: toml in /usr/local/lib/python3.6/site-packages (from registries-conf-ctl==1.0)", "Requirement already up-to-date: docopt in /usr/local/lib/python3.6/site-packages (from registries-conf-ctl==1.0)", "Installing collected packages: registries-conf-ctl", " Found existing installation: registries-conf-ctl 1.0", " Uninstalling registries-conf-ctl-1.0:", " Successfully uninstalled registries-conf-ctl-1.0", " Running setup.py in 2021-11-03T17:01:38.369 INFO:teuthology.task.ansible.out:stall for registries-conf-ctl: started", " Running setup.py install for registries-conf-ctl: finished with status 'done'", "Successfully installed registries-conf-ctl-1.0"], "version": null, "virtualenv": null} 2021-11-03T17:01:38.389 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:38 +0000 (0:00:00.319) 0:05:55.976 **** 2021-11-03T17:01:38.408 INFO:teuthology.task.ansible.out: TASK [container-host : Add local docker.io registry mirror] ******************** 2021-11-03T17:01:38.410 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => {"changed": true, "cmd": ["registries-conf-ctl", "add-mirror", "docker.io", "docker-mirror.front.sepia.ceph.com:5000"], "delta": "0:00:00.149785", "end": "2021-11-03 17:01:38.396418", "rc": 0, "start": "2021-11-03 17:01:38.246633", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-03T17:01:38.431 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:38 +0000 (0:00:00.041) 0:05:56.018 **** 2021-11-03T17:01:38.438 INFO:teuthology.task.ansible.out: TASK [container-host : Check for docker's daemon.json] ************************* 2021-11-03T17:01:38.439 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:38.462 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:38 +0000 (0:00:00.030) 0:05:56.049 **** 2021-11-03T17:01:38.469 INFO:teuthology.task.ansible.out: TASK [container-host : Restart docker service] ********************************* 2021-11-03T17:01:38.471 INFO:teuthology.task.ansible.out:skipping: [smithi115.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:38.493 INFO:teuthology.task.ansible.out: TASK [container-host : Create /etc/containers/registries.conf if necessary] **** skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:38.515 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:38 +0000 (0:00:00.052) 0:05:56.101 **** 2021-11-03T17:01:38.867 INFO:teuthology.task.ansible.out: TASK [container-host : Add local docker.io registry mirror] ******************** 2021-11-03T17:01:38.869 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "cmd": ["registries-conf-ctl", "add-mirror", "docker.io", "docker-mirror.front.sepia.ceph.com:5000"], "delta": "0:00:00.159442", "end": "2021-11-03 17:01:38.854952", "rc": 0, "start": "2021-11-03 17:01:38.695510", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2021-11-03T17:01:38.889 INFO:teuthology.task.ansible.out:Wednesday 03 November 2021 17:01:38 +0000 (0:00:00.374) 0:05:56.476 **** 2021-11-03T17:01:38.917 INFO:teuthology.task.ansible.out: TASK [container-host : Restart docker service] ********************************* 2021-11-03T17:01:38.919 INFO:teuthology.task.ansible.out:skipping: [smithi027.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2021-11-03T17:01:38.985 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** skipping: no hosts matched 2021-11-03T17:01:38.986 INFO:teuthology.task.ansible.out: PLAY [paddles] ***************************************************************** skipping: no hosts matched 2021-11-03T17:01:38.988 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** skipping: no hosts matched 2021-11-03T17:01:38.989 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2021-11-03T17:01:39.004 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* Wednesday 03 November 2021 17:01:38 +0000 (0:00:00.114) 0:05:56.590 **** 2021-11-03T17:01:39.663 INFO:teuthology.task.ansible.out:ok: [smithi027.front.sepia.ceph.com] 2021-11-03T17:01:39.673 INFO:teuthology.task.ansible.out:ok: [smithi115.front.sepia.ceph.com] 2021-11-03T17:01:39.690 INFO:teuthology.task.ansible.out: TASK [Touch /ceph-qa-ready] **************************************************** Wednesday 03 November 2021 17:01:39 +0000 (0:00:00.686) 0:05:57.276 **** 2021-11-03T17:01:39.892 INFO:teuthology.task.ansible.out:changed: [smithi027.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2021-11-03T17:01:39.903 INFO:teuthology.task.ansible.out:changed: [smithi115.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2021-11-03T17:01:39.906 INFO:teuthology.task.ansible.out: PLAY RECAP ********************************************************************* 2021-11-03T17:01:39.908 INFO:teuthology.task.ansible.out:smithi027.front.sepia.ceph.com : ok=135 changed=43 unreachable=0 failed=0 skipped=184 rescued=0 ignored=3 smithi115.front.sepia.ceph.com : ok=135 changed=43 unreachable=0 failed=0 skipped=184 rescued=0 ignored=3 Wednesday 03 November 2021 17:01:39 +0000 (0:00:00.216) 0:05:57.493 **** =============================================================================== Create all admin users with sudo access. ------------------------------- 35.73s common : nrpe - Ensure SELinux policy is up to date -------------------- 33.76s users : Update authorized_keys using the keys repo --------------------- 30.16s testnode : Enable copr repos ------------------------------------------- 24.23s Remove revoked users --------------------------------------------------- 15.51s common : nrpe - Remove SELinux policy package -------------------------- 10.27s common : nrpe - Load SELinux policy package ----------------------------- 9.47s testnode : Zap all non-root disks --------------------------------------- 8.08s container-host : Install container packages ----------------------------- 5.66s testnode : Install GPG -------------------------------------------------- 2.53s container-host : Install registries-conf-ctl ---------------------------- 2.08s testnode : Blow away lingering OSD data and FSIDs ----------------------- 1.95s testnode : Create logical volume(s) ------------------------------------- 1.87s testnode : Configure version specific repos in /etc/yum.repos.d/ -------- 1.85s testnode : Configure local mirrorlists ---------------------------------- 1.84s testnode : Remove packages ---------------------------------------------- 1.81s testnode : List any leftover Ceph artifacts from previous jobs ---------- 1.79s users : Update authorized_keys for each user with literal keys ---------- 1.79s testnode : Make sure rpm dependencies are installed --------------------- 1.76s testnode : Upgrade packages --------------------- 2021-11-03T17:01:39.909 INFO:teuthology.task.ansible.out:------------------------ 1.75s 2021-11-03T17:01:40.307 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi027.front.sepia.ceph.com'), Remote(name='ubuntu@smithi115.front.sepia.ceph.com')] 2021-11-03T17:01:40.309 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi027.front.sepia.ceph.com' 2021-11-03T17:01:40.311 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi027.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-11-03T17:01:40.387 DEBUG:teuthology.orchestra.run.smithi027:> true 2021-11-03T17:01:40.472 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi115.front.sepia.ceph.com' 2021-11-03T17:01:40.472 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi115.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2021-11-03T17:01:40.543 DEBUG:teuthology.orchestra.run.smithi115:> true 2021-11-03T17:01:40.622 INFO:teuthology.run_tasks:Running task clock... 2021-11-03T17:01:40.634 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2021-11-03T17:01:40.635 INFO:teuthology.orchestra.run:Running command with timeout 360 2021-11-03T17:01:40.637 DEBUG:teuthology.orchestra.run.smithi027:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-11-03T17:01:40.672 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2021-11-03T17:01:40.689 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2021-11-03T17:01:40.723 INFO:teuthology.orchestra.run.smithi027.stderr:sudo: ntpd: command not found 2021-11-03T17:01:40.734 INFO:teuthology.orchestra.run.smithi027.stdout:506 Cannot talk to daemon 2021-11-03T17:01:40.748 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntp.service: Unit ntp.service not found. 2021-11-03T17:01:40.762 INFO:teuthology.orchestra.run.smithi027.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2021-11-03T17:01:40.825 INFO:teuthology.orchestra.run.smithi027.stderr:bash: ntpq: command not found 2021-11-03T17:01:40.828 INFO:teuthology.orchestra.run.smithi027.stdout:210 Number of sources = 4 2021-11-03T17:01:40.828 INFO:teuthology.orchestra.run.smithi027.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2021-11-03T17:01:40.828 INFO:teuthology.orchestra.run.smithi027.stdout:=============================================================================== 2021-11-03T17:01:40.828 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2021-11-03T17:01:40.829 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2021-11-03T17:01:40.829 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2021-11-03T17:01:40.829 INFO:teuthology.orchestra.run.smithi027.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2021-11-03T17:01:40.829 INFO:teuthology.orchestra.run:Running command with timeout 360 2021-11-03T17:01:40.830 DEBUG:teuthology.orchestra.run.smithi115:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2021-11-03T17:01:40.860 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2021-11-03T17:01:40.874 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2021-11-03T17:01:40.907 INFO:teuthology.orchestra.run.smithi115.stderr:sudo: ntpd: command not found 2021-11-03T17:01:40.917 INFO:teuthology.orchestra.run.smithi115.stdout:506 Cannot talk to daemon 2021-11-03T17:01:40.930 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to start ntp.service: Unit ntp.service not found. 2021-11-03T17:01:40.944 INFO:teuthology.orchestra.run.smithi115.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2021-11-03T17:01:41.004 INFO:teuthology.orchestra.run.smithi115.stderr:bash: ntpq: command not found 2021-11-03T17:01:41.007 INFO:teuthology.orchestra.run.smithi115.stdout:210 Number of sources = 4 2021-11-03T17:01:41.008 INFO:teuthology.orchestra.run.smithi115.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2021-11-03T17:01:41.008 INFO:teuthology.orchestra.run.smithi115.stdout:=============================================================================== 2021-11-03T17:01:41.008 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2021-11-03T17:01:41.008 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2021-11-03T17:01:41.009 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2021-11-03T17:01:41.009 INFO:teuthology.orchestra.run.smithi115.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2021-11-03T17:01:41.009 INFO:teuthology.run_tasks:Running task pexec... 2021-11-03T17:01:41.021 INFO:teuthology.task.pexec:Executing custom commands... 2021-11-03T17:01:41.021 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi027.front.sepia.ceph.com 2021-11-03T17:01:41.022 DEBUG:teuthology.orchestra.run.smithi027:> TESTDIR=/home/ubuntu/cephtest bash -s 2021-11-03T17:01:41.023 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi115.front.sepia.ceph.com 2021-11-03T17:01:41.023 DEBUG:teuthology.orchestra.run.smithi115:> TESTDIR=/home/ubuntu/cephtest bash -s 2021-11-03T17:01:41.533 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:44 ago on Wed 03 Nov 2021 04:59:57 PM UTC. 2021-11-03T17:01:41.536 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:05 ago on Wed 03 Nov 2021 05:00:36 PM UTC. 2021-11-03T17:01:42.623 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2021-11-03T17:01:42.623 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:01:42.624 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2021-11-03T17:01:42.624 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:01:42.624 INFO:teuthology.orchestra.run.smithi115.stdout:Resetting modules: 2021-11-03T17:01:42.624 INFO:teuthology.orchestra.run.smithi115.stdout: container-tools 2021-11-03T17:01:42.625 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:01:42.625 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2021-11-03T17:01:42.625 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:01:42.625 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:01:42.683 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2021-11-03T17:01:42.684 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:01:42.685 INFO:teuthology.orchestra.run.smithi027.stdout: Package Architecture Version Repository Size 2021-11-03T17:01:42.685 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:01:42.685 INFO:teuthology.orchestra.run.smithi027.stdout:Resetting modules: 2021-11-03T17:01:42.685 INFO:teuthology.orchestra.run.smithi027.stdout: container-tools 2021-11-03T17:01:42.685 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:01:42.687 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2021-11-03T17:01:42.687 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:01:42.687 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:01:42.829 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2021-11-03T17:01:42.899 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2021-11-03T17:01:43.301 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:46 ago on Wed 03 Nov 2021 04:59:57 PM UTC. 2021-11-03T17:01:43.398 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:07 ago on Wed 03 Nov 2021 05:00:36 PM UTC. 2021-11-03T17:01:44.358 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2021-11-03T17:01:44.361 INFO:teuthology.orchestra.run.smithi115.stdout:======================================================================================================== 2021-11-03T17:01:44.361 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2021-11-03T17:01:44.362 INFO:teuthology.orchestra.run.smithi115.stdout:======================================================================================================== 2021-11-03T17:01:44.362 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrading: 2021-11-03T17:01:44.362 INFO:teuthology.orchestra.run.smithi115.stdout: containernetworking-plugins x86_64 0.9.1-1.module_el8.4.0+786+4668b267 CentOS-AppStream 20 M 2021-11-03T17:01:44.362 INFO:teuthology.orchestra.run.smithi115.stdout: criu x86_64 3.15-1.module_el8.4.0+786+4668b267 CentOS-AppStream 511 k 2021-11-03T17:01:44.363 INFO:teuthology.orchestra.run.smithi115.stdout: libslirp x86_64 4.3.1-1.module_el8.4.0+786+4668b267 CentOS-AppStream 69 k 2021-11-03T17:01:44.363 INFO:teuthology.orchestra.run.smithi115.stdout: slirp4netns x86_64 1.1.8-1.module_el8.4.0+786+4668b267 CentOS-AppStream 51 k 2021-11-03T17:01:44.363 INFO:teuthology.orchestra.run.smithi115.stdout:Installing group/module packages: 2021-11-03T17:01:44.363 INFO:teuthology.orchestra.run.smithi115.stdout: buildah x86_64 1.19.7-1.module_el8.4.0+786+4668b267 CentOS-AppStream 6.9 M 2021-11-03T17:01:44.364 INFO:teuthology.orchestra.run.smithi115.stdout: cockpit-podman noarch 29-2.module_el8.4.0+786+4668b267 CentOS-AppStream 1.1 M 2021-11-03T17:01:44.364 INFO:teuthology.orchestra.run.smithi115.stdout: skopeo x86_64 1:1.2.2-7.module_el8.4.0+786+4668b267 CentOS-AppStream 5.9 M 2021-11-03T17:01:44.364 INFO:teuthology.orchestra.run.smithi115.stdout: toolbox noarch 0.0.8-1.module_el8.4.0+786+4668b267 CentOS-AppStream 16 k 2021-11-03T17:01:44.364 INFO:teuthology.orchestra.run.smithi115.stdout: udica noarch 0.2.4-1.module_el8.4.0+786+4668b267 CentOS-AppStream 51 k 2021-11-03T17:01:44.364 INFO:teuthology.orchestra.run.smithi115.stdout:Downgrading: 2021-11-03T17:01:44.365 INFO:teuthology.orchestra.run.smithi115.stdout: conmon x86_64 2:2.0.26-1.module_el8.4.0+786+4668b267 CentOS-AppStream 51 k 2021-11-03T17:01:44.365 INFO:teuthology.orchestra.run.smithi115.stdout: container-selinux noarch 2:2.158.0-1.module_el8.4.0+786+4668b267 CentOS-AppStream 51 k 2021-11-03T17:01:44.365 INFO:teuthology.orchestra.run.smithi115.stdout: containers-common x86_64 1:1.2.2-7.module_el8.4.0+786+4668b267 CentOS-AppStream 97 k 2021-11-03T17:01:44.365 INFO:teuthology.orchestra.run.smithi115.stdout: crun x86_64 0.18-2.module_el8.4.0+786+4668b267 CentOS-AppStream 185 k 2021-11-03T17:01:44.365 INFO:teuthology.orchestra.run.smithi115.stdout: fuse-overlayfs x86_64 1.4.0-2.module_el8.4.0+786+4668b267 CentOS-AppStream 72 k 2021-11-03T17:01:44.366 INFO:teuthology.orchestra.run.smithi115.stdout: podman x86_64 3.0.1-6.module_el8.4.0+786+4668b267 CentOS-AppStream 11 M 2021-11-03T17:01:44.366 INFO:teuthology.orchestra.run.smithi115.stdout: podman-catatonit x86_64 3.0.1-6.module_el8.4.0+786+4668b267 CentOS-AppStream 320 k 2021-11-03T17:01:44.366 INFO:teuthology.orchestra.run.smithi115.stdout: podman-docker noarch 3.0.1-6.module_el8.4.0+786+4668b267 CentOS-AppStream 55 k 2021-11-03T17:01:44.366 INFO:teuthology.orchestra.run.smithi115.stdout: runc x86_64 1.0.0-71.rc92.module_el8.4.0+833+9763146c CentOS-AppStream 3.1 M 2021-11-03T17:01:44.366 INFO:teuthology.orchestra.run.smithi115.stdout:Installing module profiles: 2021-11-03T17:01:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: container-tools/common 2021-11-03T17:01:44.367 INFO:teuthology.orchestra.run.smithi115.stdout:Enabling module streams: 2021-11-03T17:01:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: container-tools 3.0 2021-11-03T17:01:44.367 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:01:44.368 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2021-11-03T17:01:44.368 INFO:teuthology.orchestra.run.smithi115.stdout:======================================================================================================== 2021-11-03T17:01:44.368 INFO:teuthology.orchestra.run.smithi115.stdout:Install 5 Packages 2021-11-03T17:01:44.368 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrade 4 Packages 2021-11-03T17:01:44.368 INFO:teuthology.orchestra.run.smithi115.stdout:Downgrade 9 Packages 2021-11-03T17:01:44.369 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:01:44.369 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 49 M 2021-11-03T17:01:44.370 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2021-11-03T17:01:44.403 INFO:teuthology.orchestra.run.smithi115.stdout:(1/18): conmon-2.0.26-1.module_el8.4.0+786+4668 1.3 MB/s | 51 kB 00:00 2021-11-03T17:01:44.409 INFO:teuthology.orchestra.run.smithi115.stdout:(2/18): container-selinux-2.158.0-1.module_el8. 1.1 MB/s | 51 kB 00:00 2021-11-03T17:01:44.412 INFO:teuthology.orchestra.run.smithi115.stdout:(3/18): crun-0.18-2.module_el8.4.0+786+4668b267 21 MB/s | 185 kB 00:00 2021-11-03T17:01:44.412 INFO:teuthology.orchestra.run.smithi115.stdout:(4/18): fuse-overlayfs-1.4.0-2.module_el8.4.0+7 22 MB/s | 72 kB 00:00 2021-11-03T17:01:44.430 INFO:teuthology.orchestra.run.smithi115.stdout:(5/18): containers-common-1.2.2-7.module_el8.4. 1.5 MB/s | 97 kB 00:00 2021-11-03T17:01:44.432 INFO:teuthology.orchestra.run.smithi115.stdout:(6/18): podman-catatonit-3.0.1-6.module_el8.4.0 16 MB/s | 320 kB 00:00 2021-11-03T17:01:44.439 INFO:teuthology.orchestra.run.smithi115.stdout:(7/18): podman-docker-3.0.1-6.module_el8.4.0+78 6.0 MB/s | 55 kB 00:00 2021-11-03T17:01:44.557 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2021-11-03T17:01:44.560 INFO:teuthology.orchestra.run.smithi027.stdout:======================================================================================================== 2021-11-03T17:01:44.561 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2021-11-03T17:01:44.561 INFO:teuthology.orchestra.run.smithi027.stdout:======================================================================================================== 2021-11-03T17:01:44.561 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrading: 2021-11-03T17:01:44.561 INFO:teuthology.orchestra.run.smithi027.stdout: containernetworking-plugins x86_64 0.9.1-1.module_el8.4.0+786+4668b267 CentOS-AppStream 20 M 2021-11-03T17:01:44.562 INFO:teuthology.orchestra.run.smithi027.stdout: criu x86_64 3.15-1.module_el8.4.0+786+4668b267 CentOS-AppStream 511 k 2021-11-03T17:01:44.562 INFO:teuthology.orchestra.run.smithi027.stdout: libslirp x86_64 4.3.1-1.module_el8.4.0+786+4668b267 CentOS-AppStream 69 k 2021-11-03T17:01:44.562 INFO:teuthology.orchestra.run.smithi027.stdout: slirp4netns x86_64 1.1.8-1.module_el8.4.0+786+4668b267 CentOS-AppStream 51 k 2021-11-03T17:01:44.562 INFO:teuthology.orchestra.run.smithi027.stdout:Installing group/module packages: 2021-11-03T17:01:44.563 INFO:teuthology.orchestra.run.smithi027.stdout: buildah x86_64 1.19.7-1.module_el8.4.0+786+4668b267 CentOS-AppStream 6.9 M 2021-11-03T17:01:44.563 INFO:teuthology.orchestra.run.smithi027.stdout: cockpit-podman noarch 29-2.module_el8.4.0+786+4668b267 CentOS-AppStream 1.1 M 2021-11-03T17:01:44.563 INFO:teuthology.orchestra.run.smithi027.stdout: skopeo x86_64 1:1.2.2-7.module_el8.4.0+786+4668b267 CentOS-AppStream 5.9 M 2021-11-03T17:01:44.563 INFO:teuthology.orchestra.run.smithi027.stdout: toolbox noarch 0.0.8-1.module_el8.4.0+786+4668b267 CentOS-AppStream 16 k 2021-11-03T17:01:44.564 INFO:teuthology.orchestra.run.smithi027.stdout: udica noarch 0.2.4-1.module_el8.4.0+786+4668b267 CentOS-AppStream 51 k 2021-11-03T17:01:44.564 INFO:teuthology.orchestra.run.smithi027.stdout:Downgrading: 2021-11-03T17:01:44.564 INFO:teuthology.orchestra.run.smithi027.stdout: conmon x86_64 2:2.0.26-1.module_el8.4.0+786+4668b267 CentOS-AppStream 51 k 2021-11-03T17:01:44.564 INFO:teuthology.orchestra.run.smithi027.stdout: container-selinux noarch 2:2.158.0-1.module_el8.4.0+786+4668b267 CentOS-AppStream 51 k 2021-11-03T17:01:44.564 INFO:teuthology.orchestra.run.smithi027.stdout: containers-common x86_64 1:1.2.2-7.module_el8.4.0+786+4668b267 CentOS-AppStream 97 k 2021-11-03T17:01:44.565 INFO:teuthology.orchestra.run.smithi027.stdout: crun x86_64 0.18-2.module_el8.4.0+786+4668b267 CentOS-AppStream 185 k 2021-11-03T17:01:44.565 INFO:teuthology.orchestra.run.smithi027.stdout: fuse-overlayfs x86_64 1.4.0-2.module_el8.4.0+786+4668b267 CentOS-AppStream 72 k 2021-11-03T17:01:44.565 INFO:teuthology.orchestra.run.smithi027.stdout: podman x86_64 3.0.1-6.module_el8.4.0+786+4668b267 CentOS-AppStream 11 M 2021-11-03T17:01:44.565 INFO:teuthology.orchestra.run.smithi027.stdout: podman-catatonit x86_64 3.0.1-6.module_el8.4.0+786+4668b267 CentOS-AppStream 320 k 2021-11-03T17:01:44.566 INFO:teuthology.orchestra.run.smithi027.stdout: podman-docker noarch 3.0.1-6.module_el8.4.0+786+4668b267 CentOS-AppStream 55 k 2021-11-03T17:01:44.566 INFO:teuthology.orchestra.run.smithi027.stdout: runc x86_64 1.0.0-71.rc92.module_el8.4.0+833+9763146c CentOS-AppStream 3.1 M 2021-11-03T17:01:44.566 INFO:teuthology.orchestra.run.smithi027.stdout:Installing module profiles: 2021-11-03T17:01:44.566 INFO:teuthology.orchestra.run.smithi027.stdout: container-tools/common 2021-11-03T17:01:44.567 INFO:teuthology.orchestra.run.smithi027.stdout:Enabling module streams: 2021-11-03T17:01:44.567 INFO:teuthology.orchestra.run.smithi027.stdout: container-tools 3.0 2021-11-03T17:01:44.567 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:01:44.567 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2021-11-03T17:01:44.567 INFO:teuthology.orchestra.run.smithi027.stdout:======================================================================================================== 2021-11-03T17:01:44.568 INFO:teuthology.orchestra.run.smithi027.stdout:Install 5 Packages 2021-11-03T17:01:44.568 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrade 4 Packages 2021-11-03T17:01:44.568 INFO:teuthology.orchestra.run.smithi027.stdout:Downgrade 9 Packages 2021-11-03T17:01:44.568 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:01:44.569 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 49 M 2021-11-03T17:01:44.569 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2021-11-03T17:01:44.605 INFO:teuthology.orchestra.run.smithi115.stdout:(8/18): runc-1.0.0-71.rc92.module_el8.4.0+833+9 18 MB/s | 3.1 MB 00:00 2021-11-03T17:01:44.639 INFO:teuthology.orchestra.run.smithi027.stdout:(1/18): conmon-2.0.26-1.module_el8.4.0+786+4668 685 kB/s | 51 kB 00:00 2021-11-03T17:01:44.644 INFO:teuthology.orchestra.run.smithi027.stdout:(2/18): container-selinux-2.158.0-1.module_el8. 637 kB/s | 51 kB 00:00 2021-11-03T17:01:44.667 INFO:teuthology.orchestra.run.smithi027.stdout:(3/18): containers-common-1.2.2-7.module_el8.4. 950 kB/s | 97 kB 00:00 2021-11-03T17:01:44.668 INFO:teuthology.orchestra.run.smithi027.stdout:(4/18): crun-0.18-2.module_el8.4.0+786+4668b267 6.1 MB/s | 185 kB 00:00 2021-11-03T17:01:44.669 INFO:teuthology.orchestra.run.smithi027.stdout:(5/18): fuse-overlayfs-1.4.0-2.module_el8.4.0+7 2.9 MB/s | 72 kB 00:00 2021-11-03T17:01:44.674 INFO:teuthology.orchestra.run.smithi115.stdout:(9/18): cockpit-podman-29-2.module_el8.4.0+786+ 16 MB/s | 1.1 MB 00:00 2021-11-03T17:01:44.677 INFO:teuthology.orchestra.run.smithi027.stdout:(6/18): podman-docker-3.0.1-6.module_el8.4.0+78 7.2 MB/s | 55 kB 00:00 2021-11-03T17:01:44.707 INFO:teuthology.orchestra.run.smithi027.stdout:(7/18): podman-catatonit-3.0.1-6.module_el8.4.0 8.2 MB/s | 320 kB 00:00 2021-11-03T17:01:44.953 INFO:teuthology.orchestra.run.smithi115.stdout:(10/18): buildah-1.19.7-1.module_el8.4.0+786+46 13 MB/s | 6.9 MB 00:00 2021-11-03T17:01:45.028 INFO:teuthology.orchestra.run.smithi115.stdout:(11/18): podman-3.0.1-6.module_el8.4.0+786+4668 18 MB/s | 11 MB 00:00 2021-11-03T17:01:45.029 INFO:teuthology.orchestra.run.smithi115.stdout:(12/18): toolbox-0.0.8-1.module_el8.4.0+786+466 208 kB/s | 16 kB 00:00 2021-11-03T17:01:45.046 INFO:teuthology.orchestra.run.smithi115.stdout:(13/18): udica-0.2.4-1.module_el8.4.0+786+4668b 2.8 MB/s | 51 kB 00:00 2021-11-03T17:01:45.063 INFO:teuthology.orchestra.run.smithi027.stdout:(8/18): runc-1.0.0-71.rc92.module_el8.4.0+833+9 7.9 MB/s | 3.1 MB 00:00 2021-11-03T17:01:45.098 INFO:teuthology.orchestra.run.smithi115.stdout:(14/18): skopeo-1.2.2-7.module_el8.4.0+786+4668 14 MB/s | 5.9 MB 00:00 2021-11-03T17:01:45.099 INFO:teuthology.orchestra.run.smithi115.stdout:(15/18): criu-3.15-1.module_el8.4.0+786+4668b26 9.4 MB/s | 511 kB 00:00 2021-11-03T17:01:45.118 INFO:teuthology.orchestra.run.smithi115.stdout:(16/18): libslirp-4.3.1-1.module_el8.4.0+786+46 3.4 MB/s | 69 kB 00:00 2021-11-03T17:01:45.119 INFO:teuthology.orchestra.run.smithi115.stdout:(17/18): slirp4netns-1.1.8-1.module_el8.4.0+786 2.5 MB/s | 51 kB 00:00 2021-11-03T17:01:45.182 INFO:teuthology.orchestra.run.smithi027.stdout:(9/18): cockpit-podman-29-2.module_el8.4.0+786+ 9.1 MB/s | 1.1 MB 00:00 2021-11-03T17:01:45.381 INFO:teuthology.orchestra.run.smithi027.stdout:(10/18): buildah-1.19.7-1.module_el8.4.0+786+46 10 MB/s | 6.9 MB 00:00 2021-11-03T17:01:45.396 INFO:teuthology.orchestra.run.smithi027.stdout:(11/18): toolbox-0.0.8-1.module_el8.4.0+786+466 1.0 MB/s | 16 kB 00:00 2021-11-03T17:01:45.411 INFO:teuthology.orchestra.run.smithi027.stdout:(12/18): udica-0.2.4-1.module_el8.4.0+786+4668b 3.4 MB/s | 51 kB 00:00 2021-11-03T17:01:45.542 INFO:teuthology.orchestra.run.smithi027.stdout:(13/18): podman-3.0.1-6.module_el8.4.0+786+4668 13 MB/s | 11 MB 00:00 2021-11-03T17:01:45.583 INFO:teuthology.orchestra.run.smithi027.stdout:(14/18): criu-3.15-1.module_el8.4.0+786+4668b26 12 MB/s | 511 kB 00:00 2021-11-03T17:01:45.609 INFO:teuthology.orchestra.run.smithi027.stdout:(15/18): skopeo-1.2.2-7.module_el8.4.0+786+4668 14 MB/s | 5.9 MB 00:00 2021-11-03T17:01:45.610 INFO:teuthology.orchestra.run.smithi027.stdout:(16/18): libslirp-4.3.1-1.module_el8.4.0+786+46 2.5 MB/s | 69 kB 00:00 2021-11-03T17:01:45.627 INFO:teuthology.orchestra.run.smithi027.stdout:(17/18): slirp4netns-1.1.8-1.module_el8.4.0+786 2.8 MB/s | 51 kB 00:00 2021-11-03T17:01:46.265 INFO:teuthology.orchestra.run.smithi027.stdout:(18/18): containernetworking-plugins-0.9.1-1.mo 23 MB/s | 20 MB 00:00 2021-11-03T17:01:46.268 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2021-11-03T17:01:46.269 INFO:teuthology.orchestra.run.smithi027.stdout:Total 29 MB/s | 49 MB 00:01 2021-11-03T17:01:46.300 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2021-11-03T17:01:46.348 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2021-11-03T17:01:46.348 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2021-11-03T17:01:46.506 INFO:teuthology.orchestra.run.smithi115.stdout:(18/18): containernetworking-plugins-0.9.1-1.mo 13 MB/s | 20 MB 00:01 2021-11-03T17:01:46.509 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2021-11-03T17:01:46.510 INFO:teuthology.orchestra.run.smithi115.stdout:Total 23 MB/s | 49 MB 00:02 2021-11-03T17:01:46.537 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2021-11-03T17:01:46.580 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2021-11-03T17:01:46.581 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2021-11-03T17:01:46.837 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2021-11-03T17:01:46.841 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2021-11-03T17:01:47.062 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2021-11-03T17:01:47.065 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2021-11-03T17:01:47.988 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2021-11-03T17:01:48.036 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: fuse-overlayfs-1.4.0-2.module_el8.4.0+786+4668b267.x 1/1 2021-11-03T17:01:48.132 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2021-11-03T17:01:48.160 INFO:teuthology.orchestra.run.smithi027.stdout: Downgrading : fuse-overlayfs-1.4.0-2.module_el8.4.0+786+4668b267 1/31 2021-11-03T17:01:48.168 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: fuse-overlayfs-1.4.0-2.module_el8.4.0+786+4668b267 1/31 2021-11-03T17:01:48.170 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: fuse-overlayfs-1.4.0-2.module_el8.4.0+786+4668b267.x 1/1 2021-11-03T17:01:48.182 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: container-selinux-2:2.158.0-1.module_el8.4.0+786+4 2/31 2021-11-03T17:01:48.326 INFO:teuthology.orchestra.run.smithi027.stdout: Downgrading : container-selinux-2:2.158.0-1.module_el8.4.0+786+4 2/31 2021-11-03T17:01:48.329 INFO:teuthology.orchestra.run.smithi115.stdout: Downgrading : fuse-overlayfs-1.4.0-2.module_el8.4.0+786+4668b267 1/31 2021-11-03T17:01:48.338 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: fuse-overlayfs-1.4.0-2.module_el8.4.0+786+4668b267 1/31 2021-11-03T17:01:48.357 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: container-selinux-2:2.158.0-1.module_el8.4.0+786+4 2/31 2021-11-03T17:01:48.496 INFO:teuthology.orchestra.run.smithi115.stdout: Downgrading : container-selinux-2:2.158.0-1.module_el8.4.0+786+4 2/31 2021-11-03T17:02:03.719 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: container-selinux-2:2.158.0-1.module_el8.4.0+786+4 2/31 2021-11-03T17:02:03.894 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : libslirp-4.3.1-1.module_el8.4.0+786+4668b267.x86_6 3/31 2021-11-03T17:02:04.056 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : slirp4netns-1.1.8-1.module_el8.4.0+786+4668b267.x8 4/31 2021-11-03T17:02:04.280 INFO:teuthology.orchestra.run.smithi115.stdout: Downgrading : containers-common-1:1.2.2-7.module_el8.4.0+786+466 5/31 2021-11-03T17:02:04.800 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : criu-3.15-1.module_el8.4.0+786+4668b267.x86_64 6/31 2021-11-03T17:02:04.968 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: container-selinux-2:2.158.0-1.module_el8.4.0+786+4 2/31 2021-11-03T17:02:05.107 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : libslirp-4.3.1-1.module_el8.4.0+786+4668b267.x86_6 3/31 2021-11-03T17:02:05.268 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : slirp4netns-1.1.8-1.module_el8.4.0+786+4668b267.x8 4/31 2021-11-03T17:02:05.527 INFO:teuthology.orchestra.run.smithi027.stdout: Downgrading : containers-common-1:1.2.2-7.module_el8.4.0+786+466 5/31 2021-11-03T17:02:06.056 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : criu-3.15-1.module_el8.4.0+786+4668b267.x86_64 6/31 2021-11-03T17:02:06.601 INFO:teuthology.orchestra.run.smithi115.stdout: Downgrading : runc-1.0.0-71.rc92.module_el8.4.0+833+9763146c.x86 7/31 2021-11-03T17:02:06.809 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : containernetworking-plugins-0.9.1-1.module_el8.4.0 8/31 2021-11-03T17:02:06.957 INFO:teuthology.orchestra.run.smithi115.stdout: Downgrading : crun-0.18-2.module_el8.4.0+786+4668b267.x86_64 9/31 2021-11-03T17:02:07.134 INFO:teuthology.orchestra.run.smithi115.stdout: Downgrading : conmon-2:2.0.26-1.module_el8.4.0+786+4668b267.x86_ 10/31 2021-11-03T17:02:07.997 INFO:teuthology.orchestra.run.smithi027.stdout: Downgrading : runc-1.0.0-71.rc92.module_el8.4.0+833+9763146c.x86 7/31 2021-11-03T17:02:08.233 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : containernetworking-plugins-0.9.1-1.module_el8.4.0 8/31 2021-11-03T17:02:08.373 INFO:teuthology.orchestra.run.smithi027.stdout: Downgrading : crun-0.18-2.module_el8.4.0+786+4668b267.x86_64 9/31 2021-11-03T17:02:08.551 INFO:teuthology.orchestra.run.smithi027.stdout: Downgrading : conmon-2:2.0.26-1.module_el8.4.0+786+4668b267.x86_ 10/31 2021-11-03T17:02:08.572 INFO:teuthology.orchestra.run.smithi115.stdout: Downgrading : podman-catatonit-3.0.1-6.module_el8.4.0+786+4668b2 11/31 2021-11-03T17:02:08.862 INFO:teuthology.orchestra.run.smithi115.stdout: Downgrading : podman-3.0.1-6.module_el8.4.0+786+4668b267.x86_64 12/31 2021-11-03T17:02:09.064 INFO:teuthology.orchestra.run.smithi115.stdout: Downgrading : podman-docker-3.0.1-6.module_el8.4.0+786+4668b267. 13/31 2021-11-03T17:02:09.194 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cockpit-podman-29-2.module_el8.4.0+786+4668b267.no 14/31 2021-11-03T17:02:09.896 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : toolbox-0.0.8-1.module_el8.4.0+786+4668b267.noarch 15/31 2021-11-03T17:02:10.064 INFO:teuthology.orchestra.run.smithi027.stdout: Downgrading : podman-catatonit-3.0.1-6.module_el8.4.0+786+4668b2 11/31 2021-11-03T17:02:10.348 INFO:teuthology.orchestra.run.smithi027.stdout: Downgrading : podman-3.0.1-6.module_el8.4.0+786+4668b267.x86_64 12/31 2021-11-03T17:02:10.565 INFO:teuthology.orchestra.run.smithi027.stdout: Downgrading : podman-docker-3.0.1-6.module_el8.4.0+786+4668b267. 13/31 2021-11-03T17:02:10.581 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : buildah-1.19.7-1.module_el8.4.0+786+4668b267.x86_6 16/31 2021-11-03T17:02:10.735 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cockpit-podman-29-2.module_el8.4.0+786+4668b267.no 14/31 2021-11-03T17:02:10.750 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : skopeo-1:1.2.2-7.module_el8.4.0+786+4668b267.x86_6 17/31 2021-11-03T17:02:10.908 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : udica-0.2.4-1.module_el8.4.0+786+4668b267.noarch 18/31 2021-11-03T17:02:11.013 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : podman-docker-3.2.3-0.11.module_el8.4.0+942+d25aad 19/31 2021-11-03T17:02:11.209 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : podman-3.2.3-0.11.module_el8.4.0+942+d25aada8.x86_ 20/31 2021-11-03T17:02:11.375 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : containers-common-1:1.2.2-10.module_el8.4.0+830+80 21/31 2021-11-03T17:02:11.477 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : toolbox-0.0.8-1.module_el8.4.0+786+4668b267.noarch 15/31 2021-11-03T17:02:11.517 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : slirp4netns-1.1.8-1.module_el8.4.0+641+6116a774.x8 22/31 2021-11-03T17:02:11.650 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : runc-1.0.0-73.rc93.module_el8.4.0+830+8027e1c4.x86 23/31 2021-11-03T17:02:11.650 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : container-selinux-2:2.162.0-1.module_el8.4.0+830+8 24/31 2021-11-03T17:02:11.784 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: container-selinux-2:2.162.0-1.module_el8.4.0+830+8 24/31 2021-11-03T17:02:11.909 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : podman-catatonit-3.2.3-0.11.module_el8.4.0+942+d25 25/31 2021-11-03T17:02:12.064 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : criu-3.15-1.module_el8.4.0+641+6116a774.x86_64 26/31 2021-11-03T17:02:12.200 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : libslirp-4.3.1-1.module_el8.4.0+575+63b40ad7.x86_6 27/31 2021-11-03T17:02:12.333 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : crun-0.18-2.module_el8.4.0+830+8027e1c4.x86_64 28/31 2021-11-03T17:02:12.483 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : buildah-1.19.7-1.module_el8.4.0+786+4668b267.x86_6 16/31 2021-11-03T17:02:12.560 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : fuse-overlayfs-1.4.0-3.module_el8.4.0+830+8027e1c4 29/31 2021-11-03T17:02:12.651 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : skopeo-1:1.2.2-7.module_el8.4.0+786+4668b267.x86_6 17/31 2021-11-03T17:02:12.693 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : conmon-2:2.0.26-3.module_el8.4.0+830+8027e1c4.x86_ 30/31 2021-11-03T17:02:12.807 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : udica-0.2.4-1.module_el8.4.0+786+4668b267.noarch 18/31 2021-11-03T17:02:12.858 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : containernetworking-plugins-0.9.1-1.module_el8.4.0 31/31 2021-11-03T17:02:12.913 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : podman-docker-3.2.3-0.11.module_el8.4.0+942+d25aad 19/31 2021-11-03T17:02:13.083 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : podman-3.2.3-0.11.module_el8.4.0+942+d25aada8.x86_ 20/31 2021-11-03T17:02:13.287 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : containers-common-1:1.2.2-10.module_el8.4.0+830+80 21/31 2021-11-03T17:02:13.425 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : slirp4netns-1.1.8-1.module_el8.4.0+641+6116a774.x8 22/31 2021-11-03T17:02:13.558 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : runc-1.0.0-73.rc93.module_el8.4.0+830+8027e1c4.x86 23/31 2021-11-03T17:02:13.558 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : container-selinux-2:2.162.0-1.module_el8.4.0+830+8 24/31 2021-11-03T17:02:13.733 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: container-selinux-2:2.162.0-1.module_el8.4.0+830+8 24/31 2021-11-03T17:02:13.841 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : podman-catatonit-3.2.3-0.11.module_el8.4.0+942+d25 25/31 2021-11-03T17:02:14.011 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : criu-3.15-1.module_el8.4.0+641+6116a774.x86_64 26/31 2021-11-03T17:02:14.149 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : libslirp-4.3.1-1.module_el8.4.0+575+63b40ad7.x86_6 27/31 2021-11-03T17:02:14.316 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : crun-0.18-2.module_el8.4.0+830+8027e1c4.x86_64 28/31 2021-11-03T17:02:14.441 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : fuse-overlayfs-1.4.0-3.module_el8.4.0+830+8027e1c4 29/31 2021-11-03T17:02:14.599 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : conmon-2:2.0.26-3.module_el8.4.0+830+8027e1c4.x86_ 30/31 2021-11-03T17:02:14.757 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : containernetworking-plugins-0.9.1-1.module_el8.4.0 31/31 2021-11-03T17:02:31.595 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: container-selinux-2:2.158.0-1.module_el8.4.0+786+4 31/31 2021-11-03T17:02:32.082 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: containernetworking-plugins-0.9.1-1.module_el8.4.0 31/31 2021-11-03T17:02:32.083 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : conmon-2:2.0.26-1.module_el8.4.0+786+4668b267.x86_ 1/31 2021-11-03T17:02:32.083 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : conmon-2:2.0.26-3.module_el8.4.0+830+8027e1c4.x86_ 2/31 2021-11-03T17:02:32.083 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : container-selinux-2:2.158.0-1.module_el8.4.0+786+4 3/31 2021-11-03T17:02:32.084 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : container-selinux-2:2.162.0-1.module_el8.4.0+830+8 4/31 2021-11-03T17:02:32.084 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : containers-common-1:1.2.2-7.module_el8.4.0+786+466 5/31 2021-11-03T17:02:32.084 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : containers-common-1:1.2.2-10.module_el8.4.0+830+80 6/31 2021-11-03T17:02:32.085 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : crun-0.18-2.module_el8.4.0+786+4668b267.x86_64 7/31 2021-11-03T17:02:32.085 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : crun-0.18-2.module_el8.4.0+830+8027e1c4.x86_64 8/31 2021-11-03T17:02:32.085 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : fuse-overlayfs-1.4.0-2.module_el8.4.0+786+4668b267 9/31 2021-11-03T17:02:32.085 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : fuse-overlayfs-1.4.0-3.module_el8.4.0+830+8027e1c4 10/31 2021-11-03T17:02:32.086 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : podman-3.0.1-6.module_el8.4.0+786+4668b267.x86_64 11/31 2021-11-03T17:02:32.086 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : podman-3.2.3-0.11.module_el8.4.0+942+d25aada8.x86_ 12/31 2021-11-03T17:02:32.086 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : podman-catatonit-3.0.1-6.module_el8.4.0+786+4668b2 13/31 2021-11-03T17:02:32.086 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : podman-catatonit-3.2.3-0.11.module_el8.4.0+942+d25 14/31 2021-11-03T17:02:32.087 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : podman-docker-3.0.1-6.module_el8.4.0+786+4668b267. 15/31 2021-11-03T17:02:32.087 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : podman-docker-3.2.3-0.11.module_el8.4.0+942+d25aad 16/31 2021-11-03T17:02:32.087 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : runc-1.0.0-71.rc92.module_el8.4.0+833+9763146c.x86 17/31 2021-11-03T17:02:32.087 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : runc-1.0.0-73.rc93.module_el8.4.0+830+8027e1c4.x86 18/31 2021-11-03T17:02:32.088 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : buildah-1.19.7-1.module_el8.4.0+786+4668b267.x86_6 19/31 2021-11-03T17:02:32.088 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cockpit-podman-29-2.module_el8.4.0+786+4668b267.no 20/31 2021-11-03T17:02:32.088 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : skopeo-1:1.2.2-7.module_el8.4.0+786+4668b267.x86_6 21/31 2021-11-03T17:02:32.088 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : toolbox-0.0.8-1.module_el8.4.0+786+4668b267.noarch 22/31 2021-11-03T17:02:32.088 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : udica-0.2.4-1.module_el8.4.0+786+4668b267.noarch 23/31 2021-11-03T17:02:32.089 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : containernetworking-plugins-0.9.1-1.module_el8.4.0 24/31 2021-11-03T17:02:32.089 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : containernetworking-plugins-0.9.1-1.module_el8.4.0 25/31 2021-11-03T17:02:32.089 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : criu-3.15-1.module_el8.4.0+786+4668b267.x86_64 26/31 2021-11-03T17:02:32.089 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : criu-3.15-1.module_el8.4.0+641+6116a774.x86_64 27/31 2021-11-03T17:02:32.090 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libslirp-4.3.1-1.module_el8.4.0+786+4668b267.x86_6 28/31 2021-11-03T17:02:32.090 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libslirp-4.3.1-1.module_el8.4.0+575+63b40ad7.x86_6 29/31 2021-11-03T17:02:32.090 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : slirp4netns-1.1.8-1.module_el8.4.0+786+4668b267.x8 30/31 2021-11-03T17:02:32.663 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : slirp4netns-1.1.8-1.module_el8.4.0+641+6116a774.x8 31/31 2021-11-03T17:02:32.664 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:02:32.664 INFO:teuthology.orchestra.run.smithi115.stdout:Upgraded: 2021-11-03T17:02:32.664 INFO:teuthology.orchestra.run.smithi115.stdout: containernetworking-plugins-0.9.1-1.module_el8.4.0+786+4668b267.x86_64 2021-11-03T17:02:32.665 INFO:teuthology.orchestra.run.smithi115.stdout: criu-3.15-1.module_el8.4.0+786+4668b267.x86_64 2021-11-03T17:02:32.665 INFO:teuthology.orchestra.run.smithi115.stdout: libslirp-4.3.1-1.module_el8.4.0+786+4668b267.x86_64 2021-11-03T17:02:32.665 INFO:teuthology.orchestra.run.smithi115.stdout: slirp4netns-1.1.8-1.module_el8.4.0+786+4668b267.x86_64 2021-11-03T17:02:32.666 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:02:32.666 INFO:teuthology.orchestra.run.smithi115.stdout:Downgraded: 2021-11-03T17:02:32.667 INFO:teuthology.orchestra.run.smithi115.stdout: conmon-2:2.0.26-1.module_el8.4.0+786+4668b267.x86_64 2021-11-03T17:02:32.667 INFO:teuthology.orchestra.run.smithi115.stdout: container-selinux-2:2.158.0-1.module_el8.4.0+786+4668b267.noarch 2021-11-03T17:02:32.667 INFO:teuthology.orchestra.run.smithi115.stdout: containers-common-1:1.2.2-7.module_el8.4.0+786+4668b267.x86_64 2021-11-03T17:02:32.667 INFO:teuthology.orchestra.run.smithi115.stdout: crun-0.18-2.module_el8.4.0+786+4668b267.x86_64 2021-11-03T17:02:32.668 INFO:teuthology.orchestra.run.smithi115.stdout: fuse-overlayfs-1.4.0-2.module_el8.4.0+786+4668b267.x86_64 2021-11-03T17:02:32.668 INFO:teuthology.orchestra.run.smithi115.stdout: podman-3.0.1-6.module_el8.4.0+786+4668b267.x86_64 2021-11-03T17:02:32.668 INFO:teuthology.orchestra.run.smithi115.stdout: podman-catatonit-3.0.1-6.module_el8.4.0+786+4668b267.x86_64 2021-11-03T17:02:32.668 INFO:teuthology.orchestra.run.smithi115.stdout: podman-docker-3.0.1-6.module_el8.4.0+786+4668b267.noarch 2021-11-03T17:02:32.669 INFO:teuthology.orchestra.run.smithi115.stdout: runc-1.0.0-71.rc92.module_el8.4.0+833+9763146c.x86_64 2021-11-03T17:02:32.669 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:02:32.669 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2021-11-03T17:02:32.670 INFO:teuthology.orchestra.run.smithi115.stdout: buildah-1.19.7-1.module_el8.4.0+786+4668b267.x86_64 2021-11-03T17:02:32.670 INFO:teuthology.orchestra.run.smithi115.stdout: cockpit-podman-29-2.module_el8.4.0+786+4668b267.noarch 2021-11-03T17:02:32.670 INFO:teuthology.orchestra.run.smithi115.stdout: skopeo-1:1.2.2-7.module_el8.4.0+786+4668b267.x86_64 2021-11-03T17:02:32.671 INFO:teuthology.orchestra.run.smithi115.stdout: toolbox-0.0.8-1.module_el8.4.0+786+4668b267.noarch 2021-11-03T17:02:32.671 INFO:teuthology.orchestra.run.smithi115.stdout: udica-0.2.4-1.module_el8.4.0+786+4668b267.noarch 2021-11-03T17:02:32.671 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:02:32.672 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2021-11-03T17:02:32.883 DEBUG:teuthology.parallel:result is None 2021-11-03T17:02:34.574 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: container-selinux-2:2.158.0-1.module_el8.4.0+786+4 31/31 2021-11-03T17:02:35.117 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: containernetworking-plugins-0.9.1-1.module_el8.4.0 31/31 2021-11-03T17:02:35.117 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : conmon-2:2.0.26-1.module_el8.4.0+786+4668b267.x86_ 1/31 2021-11-03T17:02:35.117 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : conmon-2:2.0.26-3.module_el8.4.0+830+8027e1c4.x86_ 2/31 2021-11-03T17:02:35.117 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : container-selinux-2:2.158.0-1.module_el8.4.0+786+4 3/31 2021-11-03T17:02:35.118 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : container-selinux-2:2.162.0-1.module_el8.4.0+830+8 4/31 2021-11-03T17:02:35.118 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : containers-common-1:1.2.2-7.module_el8.4.0+786+466 5/31 2021-11-03T17:02:35.118 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : containers-common-1:1.2.2-10.module_el8.4.0+830+80 6/31 2021-11-03T17:02:35.119 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : crun-0.18-2.module_el8.4.0+786+4668b267.x86_64 7/31 2021-11-03T17:02:35.119 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : crun-0.18-2.module_el8.4.0+830+8027e1c4.x86_64 8/31 2021-11-03T17:02:35.119 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : fuse-overlayfs-1.4.0-2.module_el8.4.0+786+4668b267 9/31 2021-11-03T17:02:35.119 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : fuse-overlayfs-1.4.0-3.module_el8.4.0+830+8027e1c4 10/31 2021-11-03T17:02:35.120 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : podman-3.0.1-6.module_el8.4.0+786+4668b267.x86_64 11/31 2021-11-03T17:02:35.120 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : podman-3.2.3-0.11.module_el8.4.0+942+d25aada8.x86_ 12/31 2021-11-03T17:02:35.120 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : podman-catatonit-3.0.1-6.module_el8.4.0+786+4668b2 13/31 2021-11-03T17:02:35.121 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : podman-catatonit-3.2.3-0.11.module_el8.4.0+942+d25 14/31 2021-11-03T17:02:35.121 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : podman-docker-3.0.1-6.module_el8.4.0+786+4668b267. 15/31 2021-11-03T17:02:35.122 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : podman-docker-3.2.3-0.11.module_el8.4.0+942+d25aad 16/31 2021-11-03T17:02:35.122 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : runc-1.0.0-71.rc92.module_el8.4.0+833+9763146c.x86 17/31 2021-11-03T17:02:35.122 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : runc-1.0.0-73.rc93.module_el8.4.0+830+8027e1c4.x86 18/31 2021-11-03T17:02:35.123 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : buildah-1.19.7-1.module_el8.4.0+786+4668b267.x86_6 19/31 2021-11-03T17:02:35.123 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cockpit-podman-29-2.module_el8.4.0+786+4668b267.no 20/31 2021-11-03T17:02:35.123 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : skopeo-1:1.2.2-7.module_el8.4.0+786+4668b267.x86_6 21/31 2021-11-03T17:02:35.124 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : toolbox-0.0.8-1.module_el8.4.0+786+4668b267.noarch 22/31 2021-11-03T17:02:35.124 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : udica-0.2.4-1.module_el8.4.0+786+4668b267.noarch 23/31 2021-11-03T17:02:35.126 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : containernetworking-plugins-0.9.1-1.module_el8.4.0 24/31 2021-11-03T17:02:35.127 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : containernetworking-plugins-0.9.1-1.module_el8.4.0 25/31 2021-11-03T17:02:35.127 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : criu-3.15-1.module_el8.4.0+786+4668b267.x86_64 26/31 2021-11-03T17:02:35.128 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : criu-3.15-1.module_el8.4.0+641+6116a774.x86_64 27/31 2021-11-03T17:02:35.128 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libslirp-4.3.1-1.module_el8.4.0+786+4668b267.x86_6 28/31 2021-11-03T17:02:35.129 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libslirp-4.3.1-1.module_el8.4.0+575+63b40ad7.x86_6 29/31 2021-11-03T17:02:35.129 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : slirp4netns-1.1.8-1.module_el8.4.0+786+4668b267.x8 30/31 2021-11-03T17:02:35.679 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : slirp4netns-1.1.8-1.module_el8.4.0+641+6116a774.x8 31/31 2021-11-03T17:02:35.680 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:02:35.680 INFO:teuthology.orchestra.run.smithi027.stdout:Upgraded: 2021-11-03T17:02:35.680 INFO:teuthology.orchestra.run.smithi027.stdout: containernetworking-plugins-0.9.1-1.module_el8.4.0+786+4668b267.x86_64 2021-11-03T17:02:35.681 INFO:teuthology.orchestra.run.smithi027.stdout: criu-3.15-1.module_el8.4.0+786+4668b267.x86_64 2021-11-03T17:02:35.681 INFO:teuthology.orchestra.run.smithi027.stdout: libslirp-4.3.1-1.module_el8.4.0+786+4668b267.x86_64 2021-11-03T17:02:35.681 INFO:teuthology.orchestra.run.smithi027.stdout: slirp4netns-1.1.8-1.module_el8.4.0+786+4668b267.x86_64 2021-11-03T17:02:35.681 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:02:35.682 INFO:teuthology.orchestra.run.smithi027.stdout:Downgraded: 2021-11-03T17:02:35.682 INFO:teuthology.orchestra.run.smithi027.stdout: conmon-2:2.0.26-1.module_el8.4.0+786+4668b267.x86_64 2021-11-03T17:02:35.682 INFO:teuthology.orchestra.run.smithi027.stdout: container-selinux-2:2.158.0-1.module_el8.4.0+786+4668b267.noarch 2021-11-03T17:02:35.683 INFO:teuthology.orchestra.run.smithi027.stdout: containers-common-1:1.2.2-7.module_el8.4.0+786+4668b267.x86_64 2021-11-03T17:02:35.683 INFO:teuthology.orchestra.run.smithi027.stdout: crun-0.18-2.module_el8.4.0+786+4668b267.x86_64 2021-11-03T17:02:35.683 INFO:teuthology.orchestra.run.smithi027.stdout: fuse-overlayfs-1.4.0-2.module_el8.4.0+786+4668b267.x86_64 2021-11-03T17:02:35.683 INFO:teuthology.orchestra.run.smithi027.stdout: podman-3.0.1-6.module_el8.4.0+786+4668b267.x86_64 2021-11-03T17:02:35.683 INFO:teuthology.orchestra.run.smithi027.stdout: podman-catatonit-3.0.1-6.module_el8.4.0+786+4668b267.x86_64 2021-11-03T17:02:35.684 INFO:teuthology.orchestra.run.smithi027.stdout: podman-docker-3.0.1-6.module_el8.4.0+786+4668b267.noarch 2021-11-03T17:02:35.684 INFO:teuthology.orchestra.run.smithi027.stdout: runc-1.0.0-71.rc92.module_el8.4.0+833+9763146c.x86_64 2021-11-03T17:02:35.684 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:02:35.684 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2021-11-03T17:02:35.685 INFO:teuthology.orchestra.run.smithi027.stdout: buildah-1.19.7-1.module_el8.4.0+786+4668b267.x86_64 2021-11-03T17:02:35.685 INFO:teuthology.orchestra.run.smithi027.stdout: cockpit-podman-29-2.module_el8.4.0+786+4668b267.noarch 2021-11-03T17:02:35.685 INFO:teuthology.orchestra.run.smithi027.stdout: skopeo-1:1.2.2-7.module_el8.4.0+786+4668b267.x86_64 2021-11-03T17:02:35.685 INFO:teuthology.orchestra.run.smithi027.stdout: toolbox-0.0.8-1.module_el8.4.0+786+4668b267.noarch 2021-11-03T17:02:35.686 INFO:teuthology.orchestra.run.smithi027.stdout: udica-0.2.4-1.module_el8.4.0+786+4668b267.noarch 2021-11-03T17:02:35.686 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:02:35.686 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2021-11-03T17:02:35.924 DEBUG:teuthology.parallel:result is None 2021-11-03T17:02:35.924 INFO:teuthology.run_tasks:Running task install... 2021-11-03T17:02:35.937 DEBUG:teuthology.task.install:project ceph 2021-11-03T17:02:35.938 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'sha1': '6e90212d8510e2d506395a8c82057d0d1787f3a4'}} 2021-11-03T17:02:35.938 DEBUG:teuthology.task.install:config {'exclude_packages': ['ceph-volume'], 'tag': 'v16.2.4', 'sha1': '6e90212d8510e2d506395a8c82057d0d1787f3a4'} 2021-11-03T17:02:35.938 INFO:teuthology.task.install:Using flavor: default 2021-11-03T17:02:35.945 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2021-11-03T17:02:35.946 INFO:teuthology.task.install:extra packages: [] 2021-11-03T17:02:35.946 DEBUG:teuthology.orchestra.run.smithi027:> sudo lsb_release -is 2021-11-03T17:02:35.981 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS 2021-11-03T17:02:35.983 DEBUG:teuthology.misc:System to be installed: CentOS 2021-11-03T17:02:35.984 DEBUG:teuthology.orchestra.run.smithi115:> sudo lsb_release -is 2021-11-03T17:02:35.985 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6e90212d8510e2d506395a8c82057d0d1787f3a4', 'tag': 'v16.2.4', 'wait_for_package': False} 2021-11-03T17:02:35.986 DEBUG:teuthology.orchestra.run.smithi027:> sudo lsb_release -is 2021-11-03T17:02:36.027 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS 2021-11-03T17:02:36.029 DEBUG:teuthology.misc:System to be installed: CentOS 2021-11-03T17:02:36.029 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6e90212d8510e2d506395a8c82057d0d1787f3a4', 'tag': 'v16.2.4', 'wait_for_package': False} 2021-11-03T17:02:36.029 DEBUG:teuthology.orchestra.run.smithi115:> sudo lsb_release -is 2021-11-03T17:02:36.055 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS 2021-11-03T17:02:36.056 DEBUG:teuthology.misc:System to be installed: CentOS 2021-11-03T17:02:36.057 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2021-11-03T17:02:36.057 INFO:teuthology.packaging:ref: None 2021-11-03T17:02:36.058 INFO:teuthology.packaging:tag: v16.2.4 2021-11-03T17:02:36.058 INFO:teuthology.packaging:branch: None 2021-11-03T17:02:36.058 INFO:teuthology.packaging:sha1: 6e90212d8510e2d506395a8c82057d0d1787f3a4 2021-11-03T17:02:36.103 DEBUG:teuthology.repo_utils:git ls-remote git://git.ceph.com/ceph-ci v16.2.4^{} -> 3cbe25cde3cfa028984618ad32de9edc4c1eaed0 2021-11-03T17:02:36.105 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3cbe25cde3cfa028984618ad32de9edc4c1eaed0 2021-11-03T17:02:36.155 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS 2021-11-03T17:02:36.156 DEBUG:teuthology.misc:System to be installed: CentOS 2021-11-03T17:02:36.156 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2021-11-03T17:02:36.157 INFO:teuthology.packaging:ref: None 2021-11-03T17:02:36.157 INFO:teuthology.packaging:tag: v16.2.4 2021-11-03T17:02:36.157 INFO:teuthology.packaging:branch: None 2021-11-03T17:02:36.157 INFO:teuthology.packaging:sha1: 6e90212d8510e2d506395a8c82057d0d1787f3a4 2021-11-03T17:02:36.191 DEBUG:teuthology.repo_utils:git ls-remote git://git.ceph.com/ceph-ci v16.2.4^{} -> 3cbe25cde3cfa028984618ad32de9edc4c1eaed0 2021-11-03T17:02:36.192 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3cbe25cde3cfa028984618ad32de9edc4c1eaed0 2021-11-03T17:02:36.339 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/pacific/3cbe25cde3cfa028984618ad32de9edc4c1eaed0/centos/8/flavors/default/ 2021-11-03T17:02:36.339 INFO:teuthology.task.install.rpm:Package version is 16.2.4-0 2021-11-03T17:02:36.383 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/pacific/3cbe25cde3cfa028984618ad32de9edc4c1eaed0/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/pacific/3cbe25cde3cfa028984618ad32de9edc4c1eaed0/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/pacific/3cbe25cde3cfa028984618ad32de9edc4c1eaed0/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2021-11-03T17:02:36.383 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2021-11-03T17:02:36.384 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/yum.repos.d/ceph.repo 2021-11-03T17:02:36.410 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2021-11-03T17:02:36.410 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2021-11-03T17:02:36.411 INFO:teuthology.packaging:ref: None 2021-11-03T17:02:36.411 INFO:teuthology.packaging:tag: v16.2.4 2021-11-03T17:02:36.411 INFO:teuthology.packaging:branch: None 2021-11-03T17:02:36.411 INFO:teuthology.packaging:sha1: 6e90212d8510e2d506395a8c82057d0d1787f3a4 2021-11-03T17:02:36.412 DEBUG:teuthology.orchestra.run.smithi027:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v16.2.4/;g' /etc/yum.repos.d/ceph.repo ; fi 2021-11-03T17:02:36.417 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/pacific/3cbe25cde3cfa028984618ad32de9edc4c1eaed0/centos/8/flavors/default/ 2021-11-03T17:02:36.418 INFO:teuthology.task.install.rpm:Package version is 16.2.4-0 2021-11-03T17:02:36.443 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/pacific/3cbe25cde3cfa028984618ad32de9edc4c1eaed0/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/pacific/3cbe25cde3cfa028984618ad32de9edc4c1eaed0/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/pacific/3cbe25cde3cfa028984618ad32de9edc4c1eaed0/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2021-11-03T17:02:36.443 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2021-11-03T17:02:36.444 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/yum.repos.d/ceph.repo 2021-11-03T17:02:36.472 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2021-11-03T17:02:36.473 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2021-11-03T17:02:36.473 INFO:teuthology.packaging:ref: None 2021-11-03T17:02:36.474 INFO:teuthology.packaging:tag: v16.2.4 2021-11-03T17:02:36.474 INFO:teuthology.packaging:branch: None 2021-11-03T17:02:36.475 INFO:teuthology.packaging:sha1: 6e90212d8510e2d506395a8c82057d0d1787f3a4 2021-11-03T17:02:36.475 DEBUG:teuthology.orchestra.run.smithi115:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v16.2.4/;g' /etc/yum.repos.d/ceph.repo ; fi 2021-11-03T17:02:36.477 DEBUG:teuthology.orchestra.run.smithi027:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2021-11-03T17:02:36.541 DEBUG:teuthology.orchestra.run.smithi115:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2021-11-03T17:02:36.552 DEBUG:teuthology.orchestra.run.smithi027:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2021-11-03T17:02:36.617 INFO:teuthology.orchestra.run.smithi027.stdout:check_obsoletes = 1 2021-11-03T17:02:36.619 DEBUG:teuthology.orchestra.run.smithi115:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2021-11-03T17:02:36.620 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum clean all 2021-11-03T17:02:36.684 INFO:teuthology.orchestra.run.smithi115.stdout:check_obsoletes = 1 2021-11-03T17:02:36.685 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum clean all 2021-11-03T17:02:36.869 INFO:teuthology.orchestra.run.smithi027.stdout:87 files removed 2021-11-03T17:02:36.890 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-radosgw 2021-11-03T17:02:36.926 INFO:teuthology.orchestra.run.smithi115.stdout:87 files removed 2021-11-03T17:02:36.946 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-radosgw 2021-11-03T17:02:37.262 INFO:teuthology.orchestra.run.smithi027.stdout:ceph packages for x86_64 2.7 MB/s | 74 kB 00:00 2021-11-03T17:02:37.296 INFO:teuthology.orchestra.run.smithi115.stdout:ceph packages for x86_64 2.8 MB/s | 74 kB 00:00 2021-11-03T17:02:37.320 INFO:teuthology.orchestra.run.smithi027.stdout:ceph noarch packages 626 kB/s | 15 kB 00:00 2021-11-03T17:02:37.352 INFO:teuthology.orchestra.run.smithi115.stdout:ceph noarch packages 647 kB/s | 15 kB 00:00 2021-11-03T17:02:37.370 INFO:teuthology.orchestra.run.smithi027.stdout:ceph source packages 71 kB/s | 1.7 kB 00:00 2021-11-03T17:02:37.399 INFO:teuthology.orchestra.run.smithi115.stdout:ceph source packages 76 kB/s | 1.7 kB 00:00 2021-11-03T17:02:37.623 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS-8 - AppStream 41 MB/s | 9.5 MB 00:00 2021-11-03T17:02:37.673 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS-8 - AppStream 37 MB/s | 9.5 MB 00:00 2021-11-03T17:02:39.319 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS-8 - Base 50 MB/s | 7.5 MB 00:00 2021-11-03T17:02:39.453 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS-8 - Base 49 MB/s | 7.5 MB 00:00 2021-11-03T17:02:40.411 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS-8 - Extras 162 kB/s | 10 kB 00:00 2021-11-03T17:02:40.605 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS-8 - Extras 162 kB/s | 10 kB 00:00 2021-11-03T17:02:43.668 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Linux 8 - AppStream 3.2 MB/s | 9.5 MB 00:03 2021-11-03T17:02:44.562 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Linux 8 - AppStream 2.3 MB/s | 9.5 MB 00:04 2021-11-03T17:02:45.690 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Linux 8 - BaseOS 16 MB/s | 7.5 MB 00:00 2021-11-03T17:02:46.613 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Linux 8 - BaseOS 15 MB/s | 7.5 MB 00:00 2021-11-03T17:02:47.120 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS Linux 8 - Extras 30 kB/s | 10 kB 00:00 2021-11-03T17:02:47.282 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS-8 - PowerTools 23 MB/s | 2.4 MB 00:00 2021-11-03T17:02:47.828 INFO:teuthology.orchestra.run.smithi027.stdout:Copr repo for python3-asyncssh owned by ceph 33 kB/s | 3.5 kB 00:00 2021-11-03T17:02:47.847 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS Linux 8 - Extras 65 kB/s | 10 kB 00:00 2021-11-03T17:02:48.006 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS-8 - PowerTools 23 MB/s | 2.4 MB 00:00 2021-11-03T17:02:48.549 INFO:teuthology.orchestra.run.smithi115.stdout:Copr repo for python3-asyncssh owned by ceph 46 kB/s | 3.5 kB 00:00 2021-11-03T17:02:49.061 INFO:teuthology.orchestra.run.smithi027.stdout:Apache Arrow 133 kB/s | 156 kB 00:01 2021-11-03T17:02:49.810 INFO:teuthology.orchestra.run.smithi027.stdout:Extra Packages for Enterprise Linux 16 MB/s | 11 MB 00:00 2021-11-03T17:02:50.239 INFO:teuthology.orchestra.run.smithi115.stdout:Apache Arrow 95 kB/s | 156 kB 00:01 2021-11-03T17:02:50.977 INFO:teuthology.orchestra.run.smithi115.stdout:Extra Packages for Enterprise Linux 16 MB/s | 11 MB 00:00 2021-11-03T17:02:52.154 INFO:teuthology.orchestra.run.smithi027.stdout:lab-extras 4.3 MB/s | 24 kB 00:00 2021-11-03T17:02:53.233 INFO:teuthology.orchestra.run.smithi115.stdout:lab-extras 4.5 MB/s | 24 kB 00:00 2021-11-03T17:02:54.641 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2021-11-03T17:02:54.644 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:02:54.644 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2021-11-03T17:02:54.645 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:02:54.645 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2021-11-03T17:02:54.645 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw x86_64 2:16.2.4-0.el8 ceph 13 M 2021-11-03T17:02:54.645 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrading: 2021-11-03T17:02:54.646 INFO:teuthology.orchestra.run.smithi027.stdout: librados2 x86_64 2:16.2.4-0.el8 ceph 3.7 M 2021-11-03T17:02:54.646 INFO:teuthology.orchestra.run.smithi027.stdout: librbd1 x86_64 2:16.2.4-0.el8 ceph 4.0 M 2021-11-03T17:02:54.646 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2021-11-03T17:02:54.646 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base x86_64 2:16.2.4-0.el8 ceph 6.0 M 2021-11-03T17:02:54.647 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common x86_64 2:16.2.4-0.el8 ceph 24 M 2021-11-03T17:02:54.647 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux x86_64 2:16.2.4-0.el8 ceph 25 k 2021-11-03T17:02:54.647 INFO:teuthology.orchestra.run.smithi027.stdout: daxctl-libs x86_64 71.1-2.el8 CentOS-Base 41 k 2021-11-03T17:02:54.647 INFO:teuthology.orchestra.run.smithi027.stdout: fmt x86_64 6.2.1-1.el8 epel 116 k 2021-11-03T17:02:54.647 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2021-11-03T17:02:54.648 INFO:teuthology.orchestra.run.smithi027.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2021-11-03T17:02:54.648 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2 x86_64 2:16.2.4-0.el8 ceph 801 k 2021-11-03T17:02:54.648 INFO:teuthology.orchestra.run.smithi027.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2021-11-03T17:02:54.648 INFO:teuthology.orchestra.run.smithi027.stdout: libpmemobj x86_64 1.6.1-1.el8 CentOS-AppStream 145 k 2021-11-03T17:02:54.649 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq x86_64 0.9.0-3.el8 CentOS-Base 47 k 2021-11-03T17:02:54.649 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1 x86_64 2:16.2.4-0.el8 ceph 486 k 2021-11-03T17:02:54.649 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka x86_64 0.11.4-1.el8 CentOS-AppStream 353 k 2021-11-03T17:02:54.649 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2 x86_64 2:16.2.4-0.el8 ceph 4.1 M 2021-11-03T17:02:54.650 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2021-11-03T17:02:54.650 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust x86_64 2.8.1-11.el8 CentOS-AppStream 259 k 2021-11-03T17:02:54.650 INFO:teuthology.orchestra.run.smithi027.stdout: ndctl-libs x86_64 71.1-2.el8 CentOS-Base 78 k 2021-11-03T17:02:54.650 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse x86_64 2:16.2.4-0.el8 ceph 44 k 2021-11-03T17:02:54.651 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common x86_64 2:16.2.4-0.el8 ceph 80 k 2021-11-03T17:02:54.651 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs x86_64 2:16.2.4-0.el8 ceph 213 k 2021-11-03T17:02:54.651 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable noarch 0.7.2-14.el8 CentOS-AppStream 44 k 2021-11-03T17:02:54.651 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados x86_64 2:16.2.4-0.el8 ceph 387 k 2021-11-03T17:02:54.652 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd x86_64 2:16.2.4-0.el8 ceph 368 k 2021-11-03T17:02:54.652 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw x86_64 2:16.2.4-0.el8 ceph 114 k 2021-11-03T17:02:54.652 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:02:54.652 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2021-11-03T17:02:54.652 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:02:54.653 INFO:teuthology.orchestra.run.smithi027.stdout:Install 25 Packages 2021-11-03T17:02:54.653 INFO:teuthology.orchestra.run.smithi027.stdout:Upgrade 2 Packages 2021-11-03T17:02:54.653 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:02:54.654 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 59 M 2021-11-03T17:02:54.654 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2021-11-03T17:02:54.862 INFO:teuthology.orchestra.run.smithi027.stdout:(1/27): ceph-base-16.2.4-0.el8.x86_64.rpm 28 MB/s | 6.0 MB 00:00 2021-11-03T17:02:54.865 INFO:teuthology.orchestra.run.smithi027.stdout:(2/27): ceph-selinux-16.2.4-0.el8.x86_64.rpm 12 MB/s | 25 kB 00:00 2021-11-03T17:02:54.888 INFO:teuthology.orchestra.run.smithi027.stdout:(3/27): libcephfs2-16.2.4-0.el8.x86_64.rpm 34 MB/s | 801 kB 00:00 2021-11-03T17:02:54.903 INFO:teuthology.orchestra.run.smithi027.stdout:(4/27): libradosstriper1-16.2.4-0.el8.x86_64.rp 33 MB/s | 486 kB 00:00 2021-11-03T17:02:55.025 INFO:teuthology.orchestra.run.smithi027.stdout:(5/27): librgw2-16.2.4-0.el8.x86_64.rpm 33 MB/s | 4.1 MB 00:00 2021-11-03T17:02:55.027 INFO:teuthology.orchestra.run.smithi027.stdout:(6/27): python3-ceph-argparse-16.2.4-0.el8.x86_ 19 MB/s | 44 kB 00:00 2021-11-03T17:02:55.030 INFO:teuthology.orchestra.run.smithi027.stdout:(7/27): python3-ceph-common-16.2.4-0.el8.x86_64 34 MB/s | 80 kB 00:00 2021-11-03T17:02:55.132 INFO:teuthology.orchestra.run.smithi027.stdout:(8/27): ceph-radosgw-16.2.4-0.el8.x86_64.rpm 26 MB/s | 13 MB 00:00 2021-11-03T17:02:55.134 INFO:teuthology.orchestra.run.smithi027.stdout:(9/27): python3-cephfs-16.2.4-0.el8.x86_64.rpm 2.0 MB/s | 213 kB 00:00 2021-11-03T17:02:55.145 INFO:teuthology.orchestra.run.smithi027.stdout:(10/27): python3-rados-16.2.4-0.el8.x86_64.rpm 32 MB/s | 387 kB 00:00 2021-11-03T17:02:55.146 INFO:teuthology.orchestra.run.smithi027.stdout:(11/27): python3-rbd-16.2.4-0.el8.x86_64.rpm 28 MB/s | 368 kB 00:00 2021-11-03T17:02:55.148 INFO:teuthology.orchestra.run.smithi027.stdout:(12/27): python3-rgw-16.2.4-0.el8.x86_64.rpm 33 MB/s | 114 kB 00:00 2021-11-03T17:02:55.179 INFO:teuthology.orchestra.run.smithi027.stdout:(13/27): libpmemobj-1.6.1-1.el8.x86_64.rpm 4.4 MB/s | 145 kB 00:00 2021-11-03T17:02:55.184 INFO:teuthology.orchestra.run.smithi027.stdout:(14/27): lttng-ust-2.8.1-11.el8.x86_64.rpm 57 MB/s | 259 kB 00:00 2021-11-03T17:02:55.186 INFO:teuthology.orchestra.run.smithi027.stdout:(15/27): python3-prettytable-0.7.2-14.el8.noarc 25 MB/s | 44 kB 00:00 2021-11-03T17:02:55.187 INFO:teuthology.orchestra.run.smithi027.stdout:(16/27): daxctl-libs-71.1-2.el8.x86_64.rpm 28 MB/s | 41 kB 00:00 2021-11-03T17:02:55.189 INFO:teuthology.orchestra.run.smithi027.stdout:(17/27): librabbitmq-0.9.0-3.el8.x86_64.rpm 30 MB/s | 47 kB 00:00 2021-11-03T17:02:55.192 INFO:teuthology.orchestra.run.smithi027.stdout:(18/27): ndctl-libs-71.1-2.el8.x86_64.rpm 38 MB/s | 78 kB 00:00 2021-11-03T17:02:55.193 INFO:teuthology.orchestra.run.smithi027.stdout:(19/27): librdkafka-0.11.4-1.el8.x86_64.rpm 8.7 MB/s | 353 kB 00:00 2021-11-03T17:02:55.425 INFO:teuthology.orchestra.run.smithi027.stdout:(20/27): ceph-common-16.2.4-0.el8.x86_64.rpm 31 MB/s | 24 MB 00:00 2021-11-03T17:02:55.426 INFO:teuthology.orchestra.run.smithi027.stdout:(21/27): fmt-6.2.1-1.el8.x86_64.rpm 496 kB/s | 116 kB 00:00 2021-11-03T17:02:55.462 INFO:teuthology.orchestra.run.smithi027.stdout:(22/27): liboath-2.6.2-3.el8.x86_64.rpm 1.7 MB/s | 59 kB 00:00 2021-11-03T17:02:55.482 INFO:teuthology.orchestra.run.smithi027.stdout:(23/27): gperftools-libs-2.7-9.el8.x86_64.rpm 1.0 MB/s | 306 kB 00:00 2021-11-03T17:02:55.497 INFO:teuthology.orchestra.run.smithi027.stdout:(24/27): libunwind-1.3.1-3.el8.x86_64.rpm 2.0 MB/s | 75 kB 00:00 2021-11-03T17:02:55.538 INFO:teuthology.orchestra.run.smithi027.stdout:(25/27): librados2-16.2.4-0.el8.x86_64.rpm 64 MB/s | 3.7 MB 00:00 2021-11-03T17:02:55.554 INFO:teuthology.orchestra.run.smithi027.stdout:(26/27): leveldb-1.22-1.el8.x86_64.rpm 1.4 MB/s | 181 kB 00:00 2021-11-03T17:02:55.571 INFO:teuthology.orchestra.run.smithi027.stdout:(27/27): librbd1-16.2.4-0.el8.x86_64.rpm 54 MB/s | 4.0 MB 00:00 2021-11-03T17:02:55.576 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2021-11-03T17:02:55.576 INFO:teuthology.orchestra.run.smithi027.stdout:Total 63 MB/s | 59 MB 00:00 2021-11-03T17:02:55.588 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2021-11-03T17:02:55.608 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2021-11-03T17:02:55.611 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:02:55.612 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2021-11-03T17:02:55.612 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:02:55.612 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2021-11-03T17:02:55.612 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-radosgw x86_64 2:16.2.4-0.el8 ceph 13 M 2021-11-03T17:02:55.612 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrading: 2021-11-03T17:02:55.613 INFO:teuthology.orchestra.run.smithi115.stdout: librados2 x86_64 2:16.2.4-0.el8 ceph 3.7 M 2021-11-03T17:02:55.613 INFO:teuthology.orchestra.run.smithi115.stdout: librbd1 x86_64 2:16.2.4-0.el8 ceph 4.0 M 2021-11-03T17:02:55.613 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2021-11-03T17:02:55.613 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-base x86_64 2:16.2.4-0.el8 ceph 6.0 M 2021-11-03T17:02:55.614 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-common x86_64 2:16.2.4-0.el8 ceph 24 M 2021-11-03T17:02:55.614 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-selinux x86_64 2:16.2.4-0.el8 ceph 25 k 2021-11-03T17:02:55.614 INFO:teuthology.orchestra.run.smithi115.stdout: daxctl-libs x86_64 71.1-2.el8 CentOS-Base 41 k 2021-11-03T17:02:55.614 INFO:teuthology.orchestra.run.smithi115.stdout: fmt x86_64 6.2.1-1.el8 epel 116 k 2021-11-03T17:02:55.615 INFO:teuthology.orchestra.run.smithi115.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2021-11-03T17:02:55.615 INFO:teuthology.orchestra.run.smithi115.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2021-11-03T17:02:55.615 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs2 x86_64 2:16.2.4-0.el8 ceph 801 k 2021-11-03T17:02:55.615 INFO:teuthology.orchestra.run.smithi115.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2021-11-03T17:02:55.615 INFO:teuthology.orchestra.run.smithi115.stdout: libpmemobj x86_64 1.6.1-1.el8 CentOS-AppStream 145 k 2021-11-03T17:02:55.616 INFO:teuthology.orchestra.run.smithi115.stdout: librabbitmq x86_64 0.9.0-3.el8 CentOS-Base 47 k 2021-11-03T17:02:55.616 INFO:teuthology.orchestra.run.smithi115.stdout: libradosstriper1 x86_64 2:16.2.4-0.el8 ceph 486 k 2021-11-03T17:02:55.616 INFO:teuthology.orchestra.run.smithi115.stdout: librdkafka x86_64 0.11.4-1.el8 CentOS-AppStream 353 k 2021-11-03T17:02:55.616 INFO:teuthology.orchestra.run.smithi115.stdout: librgw2 x86_64 2:16.2.4-0.el8 ceph 4.1 M 2021-11-03T17:02:55.617 INFO:teuthology.orchestra.run.smithi115.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2021-11-03T17:02:55.617 INFO:teuthology.orchestra.run.smithi115.stdout: lttng-ust x86_64 2.8.1-11.el8 CentOS-AppStream 259 k 2021-11-03T17:02:55.617 INFO:teuthology.orchestra.run.smithi115.stdout: ndctl-libs x86_64 71.1-2.el8 CentOS-Base 78 k 2021-11-03T17:02:55.617 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-argparse x86_64 2:16.2.4-0.el8 ceph 44 k 2021-11-03T17:02:55.617 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-common x86_64 2:16.2.4-0.el8 ceph 80 k 2021-11-03T17:02:55.618 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cephfs x86_64 2:16.2.4-0.el8 ceph 213 k 2021-11-03T17:02:55.618 INFO:teuthology.orchestra.run.smithi115.stdout: python3-prettytable noarch 0.7.2-14.el8 CentOS-AppStream 44 k 2021-11-03T17:02:55.618 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rados x86_64 2:16.2.4-0.el8 ceph 387 k 2021-11-03T17:02:55.618 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rbd x86_64 2:16.2.4-0.el8 ceph 368 k 2021-11-03T17:02:55.619 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rgw x86_64 2:16.2.4-0.el8 ceph 114 k 2021-11-03T17:02:55.619 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:02:55.619 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2021-11-03T17:02:55.619 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:02:55.619 INFO:teuthology.orchestra.run.smithi115.stdout:Install 25 Packages 2021-11-03T17:02:55.620 INFO:teuthology.orchestra.run.smithi115.stdout:Upgrade 2 Packages 2021-11-03T17:02:55.620 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:02:55.621 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 59 M 2021-11-03T17:02:55.621 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2021-11-03T17:02:55.640 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2021-11-03T17:02:55.641 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2021-11-03T17:02:55.756 INFO:teuthology.orchestra.run.smithi115.stdout:(1/27): ceph-base-16.2.4-0.el8.x86_64.rpm 43 MB/s | 6.0 MB 00:00 2021-11-03T17:02:55.760 INFO:teuthology.orchestra.run.smithi115.stdout:(2/27): ceph-selinux-16.2.4-0.el8.x86_64.rpm 6.3 MB/s | 25 kB 00:00 2021-11-03T17:02:55.779 INFO:teuthology.orchestra.run.smithi115.stdout:(3/27): libcephfs2-16.2.4-0.el8.x86_64.rpm 40 MB/s | 801 kB 00:00 2021-11-03T17:02:55.794 INFO:teuthology.orchestra.run.smithi115.stdout:(4/27): libradosstriper1-16.2.4-0.el8.x86_64.rp 34 MB/s | 486 kB 00:00 2021-11-03T17:02:55.895 INFO:teuthology.orchestra.run.smithi115.stdout:(5/27): librgw2-16.2.4-0.el8.x86_64.rpm 40 MB/s | 4.1 MB 00:00 2021-11-03T17:02:55.905 INFO:teuthology.orchestra.run.smithi115.stdout:(6/27): python3-ceph-argparse-16.2.4-0.el8.x86_ 4.4 MB/s | 44 kB 00:00 2021-11-03T17:02:55.914 INFO:teuthology.orchestra.run.smithi115.stdout:(7/27): python3-ceph-common-16.2.4-0.el8.x86_64 8.7 MB/s | 80 kB 00:00 2021-11-03T17:02:55.925 INFO:teuthology.orchestra.run.smithi115.stdout:(8/27): python3-cephfs-16.2.4-0.el8.x86_64.rpm 19 MB/s | 213 kB 00:00 2021-11-03T17:02:55.939 INFO:teuthology.orchestra.run.smithi115.stdout:(9/27): python3-rados-16.2.4-0.el8.x86_64.rpm 28 MB/s | 387 kB 00:00 2021-11-03T17:02:55.953 INFO:teuthology.orchestra.run.smithi115.stdout:(10/27): python3-rbd-16.2.4-0.el8.x86_64.rpm 25 MB/s | 368 kB 00:00 2021-11-03T17:02:55.964 INFO:teuthology.orchestra.run.smithi115.stdout:(11/27): python3-rgw-16.2.4-0.el8.x86_64.rpm 10 MB/s | 114 kB 00:00 2021-11-03T17:02:55.978 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2021-11-03T17:02:55.982 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2021-11-03T17:02:55.997 INFO:teuthology.orchestra.run.smithi115.stdout:(12/27): libpmemobj-1.6.1-1.el8.x86_64.rpm 4.5 MB/s | 145 kB 00:00 2021-11-03T17:02:56.002 INFO:teuthology.orchestra.run.smithi115.stdout:(13/27): librdkafka-0.11.4-1.el8.x86_64.rpm 61 MB/s | 353 kB 00:00 2021-11-03T17:02:56.006 INFO:teuthology.orchestra.run.smithi115.stdout:(14/27): lttng-ust-2.8.1-11.el8.x86_64.rpm 67 MB/s | 259 kB 00:00 2021-11-03T17:02:56.008 INFO:teuthology.orchestra.run.smithi115.stdout:(15/27): python3-prettytable-0.7.2-14.el8.noarc 35 MB/s | 44 kB 00:00 2021-11-03T17:02:56.009 INFO:teuthology.orchestra.run.smithi115.stdout:(16/27): daxctl-libs-71.1-2.el8.x86_64.rpm 33 MB/s | 41 kB 00:00 2021-11-03T17:02:56.010 INFO:teuthology.orchestra.run.smithi115.stdout:(17/27): librabbitmq-0.9.0-3.el8.x86_64.rpm 36 MB/s | 47 kB 00:00 2021-11-03T17:02:56.012 INFO:teuthology.orchestra.run.smithi115.stdout:(18/27): ndctl-libs-71.1-2.el8.x86_64.rpm 44 MB/s | 78 kB 00:00 2021-11-03T17:02:56.134 INFO:teuthology.orchestra.run.smithi115.stdout:(19/27): ceph-radosgw-16.2.4-0.el8.x86_64.rpm 24 MB/s | 13 MB 00:00 2021-11-03T17:02:56.303 INFO:teuthology.orchestra.run.smithi115.stdout:(20/27): ceph-common-16.2.4-0.el8.x86_64.rpm 35 MB/s | 24 MB 00:00 2021-11-03T17:02:56.366 INFO:teuthology.orchestra.run.smithi115.stdout:(21/27): fmt-6.2.1-1.el8.x86_64.rpm 330 kB/s | 116 kB 00:00 2021-11-03T17:02:56.382 INFO:teuthology.orchestra.run.smithi115.stdout:(22/27): gperftools-libs-2.7-9.el8.x86_64.rpm 1.2 MB/s | 306 kB 00:00 2021-11-03T17:02:56.390 INFO:teuthology.orchestra.run.smithi115.stdout:(23/27): liboath-2.6.2-3.el8.x86_64.rpm 2.4 MB/s | 59 kB 00:00 2021-11-03T17:02:56.410 INFO:teuthology.orchestra.run.smithi115.stdout:(24/27): libunwind-1.3.1-3.el8.x86_64.rpm 2.6 MB/s | 75 kB 00:00 2021-11-03T17:02:56.448 INFO:teuthology.orchestra.run.smithi115.stdout:(25/27): librados2-16.2.4-0.el8.x86_64.rpm 65 MB/s | 3.7 MB 00:00 2021-11-03T17:02:56.472 INFO:teuthology.orchestra.run.smithi115.stdout:(26/27): librbd1-16.2.4-0.el8.x86_64.rpm 66 MB/s | 4.0 MB 00:00 2021-11-03T17:02:56.477 INFO:teuthology.orchestra.run.smithi115.stdout:(27/27): leveldb-1.22-1.el8.x86_64.rpm 1.0 MB/s | 181 kB 00:00 2021-11-03T17:02:56.480 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2021-11-03T17:02:56.481 INFO:teuthology.orchestra.run.smithi115.stdout:Total 68 MB/s | 59 MB 00:00 2021-11-03T17:02:56.492 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2021-11-03T17:02:56.539 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2021-11-03T17:02:56.539 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2021-11-03T17:02:56.863 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2021-11-03T17:02:56.866 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2021-11-03T17:02:57.198 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2021-11-03T17:02:57.383 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : fmt-6.2.1-1.el8.x86_64 1/29 2021-11-03T17:02:57.531 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/29 2021-11-03T17:02:57.676 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/29 2021-11-03T17:02:58.032 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2021-11-03T17:02:58.093 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/29 2021-11-03T17:02:58.178 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : fmt-6.2.1-1.el8.x86_64 1/29 2021-11-03T17:02:58.243 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librados2-2:16.2.4-0.el8.x86_64 4/29 2021-11-03T17:02:58.335 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-2:16.2.4-0.el8.x86_64 4/29 2021-11-03T17:02:58.407 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/29 2021-11-03T17:02:58.505 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 3/29 2021-11-03T17:02:58.546 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rados-2:16.2.4-0.el8.x86_64 5/29 2021-11-03T17:02:58.684 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs2-2:16.2.4-0.el8.x86_64 6/29 2021-11-03T17:02:58.754 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephfs2-2:16.2.4-0.el8.x86_64 6/29 2021-11-03T17:02:58.887 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 3/29 2021-11-03T17:02:58.893 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 7/29 2021-11-03T17:02:59.034 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 8/29 2021-11-03T17:02:59.039 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : librados2-2:16.2.4-0.el8.x86_64 4/29 2021-11-03T17:02:59.120 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados2-2:16.2.4-0.el8.x86_64 4/29 2021-11-03T17:02:59.338 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rados-2:16.2.4-0.el8.x86_64 5/29 2021-11-03T17:02:59.458 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 8/29 2021-11-03T17:02:59.480 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephfs2-2:16.2.4-0.el8.x86_64 6/29 2021-11-03T17:02:59.530 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephfs2-2:16.2.4-0.el8.x86_64 6/29 2021-11-03T17:02:59.601 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librgw2-2:16.2.4-0.el8.x86_64 9/29 2021-11-03T17:02:59.661 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librgw2-2:16.2.4-0.el8.x86_64 9/29 2021-11-03T17:02:59.815 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : leveldb-1.22-1.el8.x86_64 10/29 2021-11-03T17:02:59.959 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : daxctl-libs-71.1-2.el8.x86_64 11/29 2021-11-03T17:03:00.010 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: daxctl-libs-71.1-2.el8.x86_64 11/29 2021-11-03T17:03:00.012 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 7/29 2021-11-03T17:03:00.163 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 8/29 2021-11-03T17:03:00.200 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-argparse-2:16.2.4-0.el8.x86_64 12/29 2021-11-03T17:03:00.403 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cephfs-2:16.2.4-0.el8.x86_64 13/29 2021-11-03T17:03:00.570 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ndctl-libs-71.1-2.el8.x86_64 14/29 2021-11-03T17:03:00.571 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 8/29 2021-11-03T17:03:00.645 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ndctl-libs-71.1-2.el8.x86_64 14/29 2021-11-03T17:03:00.713 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librgw2-2:16.2.4-0.el8.x86_64 9/29 2021-11-03T17:03:00.779 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librgw2-2:16.2.4-0.el8.x86_64 9/29 2021-11-03T17:03:00.801 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libpmemobj-1.6.1-1.el8.x86_64 15/29 2021-11-03T17:03:00.910 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : leveldb-1.22-1.el8.x86_64 10/29 2021-11-03T17:03:01.097 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : daxctl-libs-71.1-2.el8.x86_64 11/29 2021-11-03T17:03:01.144 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: daxctl-libs-71.1-2.el8.x86_64 11/29 2021-11-03T17:03:01.241 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libpmemobj-1.6.1-1.el8.x86_64 15/29 2021-11-03T17:03:01.318 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ceph-argparse-2:16.2.4-0.el8.x86_64 12/29 2021-11-03T17:03:01.409 INFO:teuthology.orchestra.run.smithi027.stdout: Upgrading : librbd1-2:16.2.4-0.el8.x86_64 16/29 2021-11-03T17:03:01.488 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cephfs-2:16.2.4-0.el8.x86_64 13/29 2021-11-03T17:03:01.494 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-2:16.2.4-0.el8.x86_64 16/29 2021-11-03T17:03:01.655 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ndctl-libs-71.1-2.el8.x86_64 14/29 2021-11-03T17:03:01.728 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ndctl-libs-71.1-2.el8.x86_64 14/29 2021-11-03T17:03:01.763 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rbd-2:16.2.4-0.el8.x86_64 17/29 2021-11-03T17:03:01.871 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libpmemobj-1.6.1-1.el8.x86_64 15/29 2021-11-03T17:03:01.963 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rgw-2:16.2.4-0.el8.x86_64 18/29 2021-11-03T17:03:02.100 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libradosstriper1-2:16.2.4-0.el8.x86_64 19/29 2021-11-03T17:03:02.186 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libradosstriper1-2:16.2.4-0.el8.x86_64 19/29 2021-11-03T17:03:02.297 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libpmemobj-1.6.1-1.el8.x86_64 15/29 2021-11-03T17:03:02.353 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 20/29 2021-11-03T17:03:02.471 INFO:teuthology.orchestra.run.smithi115.stdout: Upgrading : librbd1-2:16.2.4-0.el8.x86_64 16/29 2021-11-03T17:03:02.527 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 21/29 2021-11-03T17:03:02.551 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librbd1-2:16.2.4-0.el8.x86_64 16/29 2021-11-03T17:03:02.699 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 22/29 2021-11-03T17:03:02.707 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rbd-2:16.2.4-0.el8.x86_64 17/29 2021-11-03T17:03:02.892 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-ceph-common-2:16.2.4-0.el8.x86_64 23/29 2021-11-03T17:03:02.915 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rgw-2:16.2.4-0.el8.x86_64 18/29 2021-11-03T17:03:03.037 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libradosstriper1-2:16.2.4-0.el8.x86_64 19/29 2021-11-03T17:03:03.095 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libradosstriper1-2:16.2.4-0.el8.x86_64 19/29 2021-11-03T17:03:03.271 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 20/29 2021-11-03T17:03:03.446 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 21/29 2021-11-03T17:03:03.649 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 22/29 2021-11-03T17:03:03.796 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-ceph-common-2:16.2.4-0.el8.x86_64 23/29 2021-11-03T17:03:07.391 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:16.2.4-0.el8.x86_64 24/29 2021-11-03T17:03:07.601 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-common-2:16.2.4-0.el8.x86_64 24/29 2021-11-03T17:03:07.867 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-common-2:16.2.4-0.el8.x86_64 24/29 2021-11-03T17:03:08.071 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-common-2:16.2.4-0.el8.x86_64 24/29 2021-11-03T17:03:08.342 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-common-2:16.2.4-0.el8.x86_64 24/29 2021-11-03T17:03:08.571 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-base-2:16.2.4-0.el8.x86_64 25/29 2021-11-03T17:03:08.770 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-common-2:16.2.4-0.el8.x86_64 24/29 2021-11-03T17:03:09.034 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-base-2:16.2.4-0.el8.x86_64 25/29 2021-11-03T17:03:09.036 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-base-2:16.2.4-0.el8.x86_64 25/29 2021-11-03T17:03:09.139 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-base-2:16.2.4-0.el8.x86_64 25/29 2021-11-03T17:03:09.166 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-selinux-2:16.2.4-0.el8.x86_64 26/29 2021-11-03T17:03:09.261 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-selinux-2:16.2.4-0.el8.x86_64 26/29 2021-11-03T17:03:24.717 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-selinux-2:16.2.4-0.el8.x86_64 26/29 2021-11-03T17:03:24.718 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /sys 2021-11-03T17:03:24.718 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /proc 2021-11-03T17:03:24.719 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /dev 2021-11-03T17:03:24.719 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /run 2021-11-03T17:03:24.719 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /mnt 2021-11-03T17:03:24.720 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /var/tmp 2021-11-03T17:03:24.720 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /home 2021-11-03T17:03:24.721 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /tmp 2021-11-03T17:03:24.721 INFO:teuthology.orchestra.run.smithi115.stdout:skipping the directory /dev 2021-11-03T17:03:24.722 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:03:25.289 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-selinux-2:16.2.4-0.el8.x86_64 26/29 2021-11-03T17:03:25.290 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /sys 2021-11-03T17:03:25.290 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /proc 2021-11-03T17:03:25.291 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /dev 2021-11-03T17:03:25.291 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /run 2021-11-03T17:03:25.291 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /mnt 2021-11-03T17:03:25.291 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /var/tmp 2021-11-03T17:03:25.292 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /home 2021-11-03T17:03:25.292 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /tmp 2021-11-03T17:03:25.292 INFO:teuthology.orchestra.run.smithi027.stdout:skipping the directory /dev 2021-11-03T17:03:25.292 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:03:26.033 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-radosgw-2:16.2.4-0.el8.x86_64 27/29 2021-11-03T17:03:26.096 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-radosgw-2:16.2.4-0.el8.x86_64 27/29 2021-11-03T17:03:26.096 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 28/29 2021-11-03T17:03:26.268 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 28/29 2021-11-03T17:03:26.269 INFO:teuthology.orchestra.run.smithi115.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 29/29 2021-11-03T17:03:26.669 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-radosgw-2:16.2.4-0.el8.x86_64 27/29 2021-11-03T17:03:26.747 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-radosgw-2:16.2.4-0.el8.x86_64 27/29 2021-11-03T17:03:26.748 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 28/29 2021-11-03T17:03:26.933 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 28/29 2021-11-03T17:03:26.934 INFO:teuthology.orchestra.run.smithi027.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 29/29 2021-11-03T17:03:27.056 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 29/29 2021-11-03T17:03:27.056 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-base-2:16.2.4-0.el8.x86_64 1/29 2021-11-03T17:03:27.056 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-common-2:16.2.4-0.el8.x86_64 2/29 2021-11-03T17:03:27.056 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-radosgw-2:16.2.4-0.el8.x86_64 3/29 2021-11-03T17:03:27.057 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-selinux-2:16.2.4-0.el8.x86_64 4/29 2021-11-03T17:03:27.057 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephfs2-2:16.2.4-0.el8.x86_64 5/29 2021-11-03T17:03:27.057 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libradosstriper1-2:16.2.4-0.el8.x86_64 6/29 2021-11-03T17:03:27.058 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librgw2-2:16.2.4-0.el8.x86_64 7/29 2021-11-03T17:03:27.058 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ceph-argparse-2:16.2.4-0.el8.x86_64 8/29 2021-11-03T17:03:27.058 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-ceph-common-2:16.2.4-0.el8.x86_64 9/29 2021-11-03T17:03:27.058 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cephfs-2:16.2.4-0.el8.x86_64 10/29 2021-11-03T17:03:27.058 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rados-2:16.2.4-0.el8.x86_64 11/29 2021-11-03T17:03:27.059 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rbd-2:16.2.4-0.el8.x86_64 12/29 2021-11-03T17:03:27.059 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rgw-2:16.2.4-0.el8.x86_64 13/29 2021-11-03T17:03:27.059 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libpmemobj-1.6.1-1.el8.x86_64 14/29 2021-11-03T17:03:27.059 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 15/29 2021-11-03T17:03:27.060 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 16/29 2021-11-03T17:03:27.060 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 17/29 2021-11-03T17:03:27.060 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : daxctl-libs-71.1-2.el8.x86_64 18/29 2021-11-03T17:03:27.060 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 19/29 2021-11-03T17:03:27.061 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ndctl-libs-71.1-2.el8.x86_64 20/29 2021-11-03T17:03:27.061 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : fmt-6.2.1-1.el8.x86_64 21/29 2021-11-03T17:03:27.061 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 22/29 2021-11-03T17:03:27.061 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : leveldb-1.22-1.el8.x86_64 23/29 2021-11-03T17:03:27.061 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 24/29 2021-11-03T17:03:27.062 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 25/29 2021-11-03T17:03:27.062 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados2-2:16.2.4-0.el8.x86_64 26/29 2021-11-03T17:03:27.062 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 27/29 2021-11-03T17:03:27.062 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librbd1-2:16.2.4-0.el8.x86_64 28/29 2021-11-03T17:03:27.639 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 29/29 2021-11-03T17:03:27.639 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-base-2:16.2.4-0.el8.x86_64 1/29 2021-11-03T17:03:27.640 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-common-2:16.2.4-0.el8.x86_64 2/29 2021-11-03T17:03:27.640 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-radosgw-2:16.2.4-0.el8.x86_64 3/29 2021-11-03T17:03:27.640 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-selinux-2:16.2.4-0.el8.x86_64 4/29 2021-11-03T17:03:27.641 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs2-2:16.2.4-0.el8.x86_64 5/29 2021-11-03T17:03:27.641 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libradosstriper1-2:16.2.4-0.el8.x86_64 6/29 2021-11-03T17:03:27.641 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librgw2-2:16.2.4-0.el8.x86_64 7/29 2021-11-03T17:03:27.641 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-argparse-2:16.2.4-0.el8.x86_64 8/29 2021-11-03T17:03:27.642 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-ceph-common-2:16.2.4-0.el8.x86_64 9/29 2021-11-03T17:03:27.642 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cephfs-2:16.2.4-0.el8.x86_64 10/29 2021-11-03T17:03:27.642 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rados-2:16.2.4-0.el8.x86_64 11/29 2021-11-03T17:03:27.643 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rbd-2:16.2.4-0.el8.x86_64 12/29 2021-11-03T17:03:27.643 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rgw-2:16.2.4-0.el8.x86_64 13/29 2021-11-03T17:03:27.643 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libpmemobj-1.6.1-1.el8.x86_64 14/29 2021-11-03T17:03:27.643 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 15/29 2021-11-03T17:03:27.644 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 16/29 2021-11-03T17:03:27.644 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 17/29 2021-11-03T17:03:27.644 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : daxctl-libs-71.1-2.el8.x86_64 18/29 2021-11-03T17:03:27.644 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 19/29 2021-11-03T17:03:27.645 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ndctl-libs-71.1-2.el8.x86_64 20/29 2021-11-03T17:03:27.645 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : fmt-6.2.1-1.el8.x86_64 21/29 2021-11-03T17:03:27.645 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 22/29 2021-11-03T17:03:27.645 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : leveldb-1.22-1.el8.x86_64 23/29 2021-11-03T17:03:27.646 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 24/29 2021-11-03T17:03:27.646 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 25/29 2021-11-03T17:03:27.646 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-2:16.2.4-0.el8.x86_64 26/29 2021-11-03T17:03:27.647 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 27/29 2021-11-03T17:03:27.647 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-2:16.2.4-0.el8.x86_64 28/29 2021-11-03T17:03:27.777 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 29/29 2021-11-03T17:03:27.778 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:03:27.778 INFO:teuthology.orchestra.run.smithi115.stdout:Upgraded: 2021-11-03T17:03:27.778 INFO:teuthology.orchestra.run.smithi115.stdout: librados2-2:16.2.4-0.el8.x86_64 librbd1-2:16.2.4-0.el8.x86_64 2021-11-03T17:03:27.779 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:03:27.779 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2021-11-03T17:03:27.779 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-base-2:16.2.4-0.el8.x86_64 2021-11-03T17:03:27.779 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-common-2:16.2.4-0.el8.x86_64 2021-11-03T17:03:27.779 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-radosgw-2:16.2.4-0.el8.x86_64 2021-11-03T17:03:27.780 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-selinux-2:16.2.4-0.el8.x86_64 2021-11-03T17:03:27.780 INFO:teuthology.orchestra.run.smithi115.stdout: daxctl-libs-71.1-2.el8.x86_64 2021-11-03T17:03:27.780 INFO:teuthology.orchestra.run.smithi115.stdout: fmt-6.2.1-1.el8.x86_64 2021-11-03T17:03:27.780 INFO:teuthology.orchestra.run.smithi115.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2021-11-03T17:03:27.781 INFO:teuthology.orchestra.run.smithi115.stdout: leveldb-1.22-1.el8.x86_64 2021-11-03T17:03:27.781 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs2-2:16.2.4-0.el8.x86_64 2021-11-03T17:03:27.781 INFO:teuthology.orchestra.run.smithi115.stdout: liboath-2.6.2-3.el8.x86_64 2021-11-03T17:03:27.781 INFO:teuthology.orchestra.run.smithi115.stdout: libpmemobj-1.6.1-1.el8.x86_64 2021-11-03T17:03:27.781 INFO:teuthology.orchestra.run.smithi115.stdout: librabbitmq-0.9.0-3.el8.x86_64 2021-11-03T17:03:27.782 INFO:teuthology.orchestra.run.smithi115.stdout: libradosstriper1-2:16.2.4-0.el8.x86_64 2021-11-03T17:03:27.782 INFO:teuthology.orchestra.run.smithi115.stdout: librdkafka-0.11.4-1.el8.x86_64 2021-11-03T17:03:27.782 INFO:teuthology.orchestra.run.smithi115.stdout: librgw2-2:16.2.4-0.el8.x86_64 2021-11-03T17:03:27.782 INFO:teuthology.orchestra.run.smithi115.stdout: libunwind-1.3.1-3.el8.x86_64 2021-11-03T17:03:27.783 INFO:teuthology.orchestra.run.smithi115.stdout: lttng-ust-2.8.1-11.el8.x86_64 2021-11-03T17:03:27.783 INFO:teuthology.orchestra.run.smithi115.stdout: ndctl-libs-71.1-2.el8.x86_64 2021-11-03T17:03:27.783 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-argparse-2:16.2.4-0.el8.x86_64 2021-11-03T17:03:27.783 INFO:teuthology.orchestra.run.smithi115.stdout: python3-ceph-common-2:16.2.4-0.el8.x86_64 2021-11-03T17:03:27.784 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cephfs-2:16.2.4-0.el8.x86_64 2021-11-03T17:03:27.784 INFO:teuthology.orchestra.run.smithi115.stdout: python3-prettytable-0.7.2-14.el8.noarch 2021-11-03T17:03:27.784 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rados-2:16.2.4-0.el8.x86_64 2021-11-03T17:03:27.784 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rbd-2:16.2.4-0.el8.x86_64 2021-11-03T17:03:27.785 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rgw-2:16.2.4-0.el8.x86_64 2021-11-03T17:03:27.785 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:03:27.785 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2021-11-03T17:03:27.982 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-test 2021-11-03T17:03:28.449 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:35 ago on Wed 03 Nov 2021 05:02:53 PM UTC. 2021-11-03T17:03:28.544 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 29/29 2021-11-03T17:03:28.544 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:03:28.544 INFO:teuthology.orchestra.run.smithi027.stdout:Upgraded: 2021-11-03T17:03:28.545 INFO:teuthology.orchestra.run.smithi027.stdout: librados2-2:16.2.4-0.el8.x86_64 librbd1-2:16.2.4-0.el8.x86_64 2021-11-03T17:03:28.545 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:03:28.545 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2021-11-03T17:03:28.545 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-base-2:16.2.4-0.el8.x86_64 2021-11-03T17:03:28.545 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-common-2:16.2.4-0.el8.x86_64 2021-11-03T17:03:28.546 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-radosgw-2:16.2.4-0.el8.x86_64 2021-11-03T17:03:28.546 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-selinux-2:16.2.4-0.el8.x86_64 2021-11-03T17:03:28.546 INFO:teuthology.orchestra.run.smithi027.stdout: daxctl-libs-71.1-2.el8.x86_64 2021-11-03T17:03:28.546 INFO:teuthology.orchestra.run.smithi027.stdout: fmt-6.2.1-1.el8.x86_64 2021-11-03T17:03:28.547 INFO:teuthology.orchestra.run.smithi027.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2021-11-03T17:03:28.547 INFO:teuthology.orchestra.run.smithi027.stdout: leveldb-1.22-1.el8.x86_64 2021-11-03T17:03:28.548 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs2-2:16.2.4-0.el8.x86_64 2021-11-03T17:03:28.548 INFO:teuthology.orchestra.run.smithi027.stdout: liboath-2.6.2-3.el8.x86_64 2021-11-03T17:03:28.548 INFO:teuthology.orchestra.run.smithi027.stdout: libpmemobj-1.6.1-1.el8.x86_64 2021-11-03T17:03:28.548 INFO:teuthology.orchestra.run.smithi027.stdout: librabbitmq-0.9.0-3.el8.x86_64 2021-11-03T17:03:28.549 INFO:teuthology.orchestra.run.smithi027.stdout: libradosstriper1-2:16.2.4-0.el8.x86_64 2021-11-03T17:03:28.549 INFO:teuthology.orchestra.run.smithi027.stdout: librdkafka-0.11.4-1.el8.x86_64 2021-11-03T17:03:28.549 INFO:teuthology.orchestra.run.smithi027.stdout: librgw2-2:16.2.4-0.el8.x86_64 2021-11-03T17:03:28.549 INFO:teuthology.orchestra.run.smithi027.stdout: libunwind-1.3.1-3.el8.x86_64 2021-11-03T17:03:28.550 INFO:teuthology.orchestra.run.smithi027.stdout: lttng-ust-2.8.1-11.el8.x86_64 2021-11-03T17:03:28.550 INFO:teuthology.orchestra.run.smithi027.stdout: ndctl-libs-71.1-2.el8.x86_64 2021-11-03T17:03:28.550 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-argparse-2:16.2.4-0.el8.x86_64 2021-11-03T17:03:28.550 INFO:teuthology.orchestra.run.smithi027.stdout: python3-ceph-common-2:16.2.4-0.el8.x86_64 2021-11-03T17:03:28.551 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cephfs-2:16.2.4-0.el8.x86_64 2021-11-03T17:03:28.551 INFO:teuthology.orchestra.run.smithi027.stdout: python3-prettytable-0.7.2-14.el8.noarch 2021-11-03T17:03:28.551 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rados-2:16.2.4-0.el8.x86_64 2021-11-03T17:03:28.551 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rbd-2:16.2.4-0.el8.x86_64 2021-11-03T17:03:28.552 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rgw-2:16.2.4-0.el8.x86_64 2021-11-03T17:03:28.552 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:03:28.552 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2021-11-03T17:03:28.746 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-test 2021-11-03T17:03:29.237 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:00:37 ago on Wed 03 Nov 2021 05:02:52 PM UTC. 2021-11-03T17:03:29.447 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2021-11-03T17:03:29.448 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:03:29.448 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2021-11-03T17:03:29.448 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:03:29.448 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2021-11-03T17:03:29.449 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-test x86_64 2:16.2.4-0.el8 ceph 52 M 2021-11-03T17:03:29.449 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2021-11-03T17:03:29.449 INFO:teuthology.orchestra.run.smithi115.stdout: jq x86_64 1.5-12.el8 CentOS-AppStream 161 k 2021-11-03T17:03:29.449 INFO:teuthology.orchestra.run.smithi115.stdout: libcephsqlite x86_64 2:16.2.4-0.el8 ceph 169 k 2021-11-03T17:03:29.450 INFO:teuthology.orchestra.run.smithi115.stdout: libxslt x86_64 1.1.32-6.el8 CentOS-Base 250 k 2021-11-03T17:03:29.450 INFO:teuthology.orchestra.run.smithi115.stdout: oniguruma x86_64 6.8.2-2.el8 CentOS-AppStream 187 k 2021-11-03T17:03:29.450 INFO:teuthology.orchestra.run.smithi115.stdout: socat x86_64 1.7.3.3-2.el8 CentOS-AppStream 302 k 2021-11-03T17:03:29.450 INFO:teuthology.orchestra.run.smithi115.stdout: xmlstarlet x86_64 1.6.1-11.el8 epel 69 k 2021-11-03T17:03:29.451 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:03:29.451 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2021-11-03T17:03:29.451 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:03:29.451 INFO:teuthology.orchestra.run.smithi115.stdout:Install 7 Packages 2021-11-03T17:03:29.452 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:03:29.452 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 53 M 2021-11-03T17:03:29.453 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 229 M 2021-11-03T17:03:29.453 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2021-11-03T17:03:29.481 INFO:teuthology.orchestra.run.smithi115.stdout:(1/7): libcephsqlite-16.2.4-0.el8.x86_64.rpm 5.4 MB/s | 169 kB 00:00 2021-11-03T17:03:29.492 INFO:teuthology.orchestra.run.smithi115.stdout:(2/7): jq-1.5-12.el8.x86_64.rpm 3.8 MB/s | 161 kB 00:00 2021-11-03T17:03:29.496 INFO:teuthology.orchestra.run.smithi115.stdout:(3/7): socat-1.7.3.3-2.el8.x86_64.rpm 66 MB/s | 302 kB 00:00 2021-11-03T17:03:29.500 INFO:teuthology.orchestra.run.smithi115.stdout:(4/7): libxslt-1.1.32-6.el8.x86_64.rpm 68 MB/s | 250 kB 00:00 2021-11-03T17:03:29.512 INFO:teuthology.orchestra.run.smithi115.stdout:(5/7): oniguruma-6.8.2-2.el8.x86_64.rpm 5.9 MB/s | 187 kB 00:00 2021-11-03T17:03:29.623 INFO:teuthology.orchestra.run.smithi115.stdout:(6/7): xmlstarlet-1.6.1-11.el8.x86_64.rpm 563 kB/s | 69 kB 00:00 2021-11-03T17:03:30.318 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2021-11-03T17:03:30.319 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:03:30.319 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2021-11-03T17:03:30.320 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:03:30.320 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2021-11-03T17:03:30.320 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test x86_64 2:16.2.4-0.el8 ceph 52 M 2021-11-03T17:03:30.320 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2021-11-03T17:03:30.321 INFO:teuthology.orchestra.run.smithi027.stdout: jq x86_64 1.5-12.el8 CentOS-AppStream 161 k 2021-11-03T17:03:30.321 INFO:teuthology.orchestra.run.smithi027.stdout: libcephsqlite x86_64 2:16.2.4-0.el8 ceph 169 k 2021-11-03T17:03:30.321 INFO:teuthology.orchestra.run.smithi027.stdout: libxslt x86_64 1.1.32-6.el8 CentOS-Base 250 k 2021-11-03T17:03:30.321 INFO:teuthology.orchestra.run.smithi027.stdout: oniguruma x86_64 6.8.2-2.el8 CentOS-AppStream 187 k 2021-11-03T17:03:30.322 INFO:teuthology.orchestra.run.smithi027.stdout: socat x86_64 1.7.3.3-2.el8 CentOS-AppStream 302 k 2021-11-03T17:03:30.322 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet x86_64 1.6.1-11.el8 epel 69 k 2021-11-03T17:03:30.323 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:03:30.323 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2021-11-03T17:03:30.323 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:03:30.323 INFO:teuthology.orchestra.run.smithi027.stdout:Install 7 Packages 2021-11-03T17:03:30.324 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:03:30.324 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 53 M 2021-11-03T17:03:30.325 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 229 M 2021-11-03T17:03:30.325 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2021-11-03T17:03:30.347 INFO:teuthology.orchestra.run.smithi027.stdout:(1/7): libcephsqlite-16.2.4-0.el8.x86_64.rpm 6.6 MB/s | 169 kB 00:00 2021-11-03T17:03:30.373 INFO:teuthology.orchestra.run.smithi027.stdout:(2/7): jq-1.5-12.el8.x86_64.rpm 3.1 MB/s | 161 kB 00:00 2021-11-03T17:03:30.381 INFO:teuthology.orchestra.run.smithi027.stdout:(3/7): socat-1.7.3.3-2.el8.x86_64.rpm 39 MB/s | 302 kB 00:00 2021-11-03T17:03:30.386 INFO:teuthology.orchestra.run.smithi027.stdout:(4/7): libxslt-1.1.32-6.el8.x86_64.rpm 49 MB/s | 250 kB 00:00 2021-11-03T17:03:30.388 INFO:teuthology.orchestra.run.smithi027.stdout:(5/7): oniguruma-6.8.2-2.el8.x86_64.rpm 4.5 MB/s | 187 kB 00:00 2021-11-03T17:03:30.422 INFO:teuthology.orchestra.run.smithi115.stdout:(7/7): ceph-test-16.2.4-0.el8.x86_64.rpm 53 MB/s | 52 MB 00:00 2021-11-03T17:03:30.426 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2021-11-03T17:03:30.427 INFO:teuthology.orchestra.run.smithi115.stdout:Total 54 MB/s | 53 MB 00:00 2021-11-03T17:03:30.435 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2021-11-03T17:03:30.459 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2021-11-03T17:03:30.460 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2021-11-03T17:03:30.484 INFO:teuthology.orchestra.run.smithi027.stdout:(6/7): xmlstarlet-1.6.1-11.el8.x86_64.rpm 706 kB/s | 69 kB 00:00 2021-11-03T17:03:30.740 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2021-11-03T17:03:30.746 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2021-11-03T17:03:31.245 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2021-11-03T17:03:31.314 INFO:teuthology.orchestra.run.smithi027.stdout:(7/7): ceph-test-16.2.4-0.el8.x86_64.rpm 52 MB/s | 52 MB 00:00 2021-11-03T17:03:31.318 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2021-11-03T17:03:31.319 INFO:teuthology.orchestra.run.smithi027.stdout:Total 53 MB/s | 53 MB 00:00 2021-11-03T17:03:31.327 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2021-11-03T17:03:31.355 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2021-11-03T17:03:31.356 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2021-11-03T17:03:31.400 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libxslt-1.1.32-6.el8.x86_64 1/7 2021-11-03T17:03:31.591 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : xmlstarlet-1.6.1-11.el8.x86_64 2/7 2021-11-03T17:03:31.655 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2021-11-03T17:03:31.658 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2021-11-03T17:03:31.767 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 3/7 2021-11-03T17:03:32.170 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2021-11-03T17:03:32.316 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libxslt-1.1.32-6.el8.x86_64 1/7 2021-11-03T17:03:32.469 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : xmlstarlet-1.6.1-11.el8.x86_64 2/7 2021-11-03T17:03:32.659 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 3/7 2021-11-03T17:03:32.813 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 4/7 2021-11-03T17:03:32.825 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 4/7 2021-11-03T17:03:32.892 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 4/7 2021-11-03T17:03:32.959 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 4/7 2021-11-03T17:03:33.072 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : jq-1.5-12.el8.x86_64 5/7 2021-11-03T17:03:33.155 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : jq-1.5-12.el8.x86_64 5/7 2021-11-03T17:03:33.263 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephsqlite-2:16.2.4-0.el8.x86_64 6/7 2021-11-03T17:03:33.289 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephsqlite-2:16.2.4-0.el8.x86_64 6/7 2021-11-03T17:03:38.060 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephsqlite-2:16.2.4-0.el8.x86_64 6/7 2021-11-03T17:03:38.354 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephsqlite-2:16.2.4-0.el8.x86_64 6/7 2021-11-03T17:03:38.578 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-test-2:16.2.4-0.el8.x86_64 7/7 2021-11-03T17:03:38.826 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-test-2:16.2.4-0.el8.x86_64 7/7 2021-11-03T17:03:38.826 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-test-2:16.2.4-0.el8.x86_64 1/7 2021-11-03T17:03:38.826 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephsqlite-2:16.2.4-0.el8.x86_64 2/7 2021-11-03T17:03:38.827 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : jq-1.5-12.el8.x86_64 3/7 2021-11-03T17:03:38.827 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 4/7 2021-11-03T17:03:38.827 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 5/7 2021-11-03T17:03:38.827 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libxslt-1.1.32-6.el8.x86_64 6/7 2021-11-03T17:03:39.213 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : xmlstarlet-1.6.1-11.el8.x86_64 7/7 2021-11-03T17:03:39.213 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:03:39.214 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2021-11-03T17:03:39.214 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-test-2:16.2.4-0.el8.x86_64 jq-1.5-12.el8.x86_64 2021-11-03T17:03:39.214 INFO:teuthology.orchestra.run.smithi027.stdout: libcephsqlite-2:16.2.4-0.el8.x86_64 libxslt-1.1.32-6.el8.x86_64 2021-11-03T17:03:39.214 INFO:teuthology.orchestra.run.smithi027.stdout: oniguruma-6.8.2-2.el8.x86_64 socat-1.7.3.3-2.el8.x86_64 2021-11-03T17:03:39.215 INFO:teuthology.orchestra.run.smithi027.stdout: xmlstarlet-1.6.1-11.el8.x86_64 2021-11-03T17:03:39.215 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:03:39.215 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2021-11-03T17:03:39.303 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph 2021-11-03T17:03:39.598 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-test-2:16.2.4-0.el8.x86_64 7/7 2021-11-03T17:03:39.780 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:00:47 ago on Wed 03 Nov 2021 05:02:52 PM UTC. 2021-11-03T17:03:39.833 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-test-2:16.2.4-0.el8.x86_64 7/7 2021-11-03T17:03:39.833 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-test-2:16.2.4-0.el8.x86_64 1/7 2021-11-03T17:03:39.833 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephsqlite-2:16.2.4-0.el8.x86_64 2/7 2021-11-03T17:03:39.833 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : jq-1.5-12.el8.x86_64 3/7 2021-11-03T17:03:39.834 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 4/7 2021-11-03T17:03:39.834 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 5/7 2021-11-03T17:03:39.834 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libxslt-1.1.32-6.el8.x86_64 6/7 2021-11-03T17:03:40.170 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : xmlstarlet-1.6.1-11.el8.x86_64 7/7 2021-11-03T17:03:40.171 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:03:40.171 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2021-11-03T17:03:40.171 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-test-2:16.2.4-0.el8.x86_64 jq-1.5-12.el8.x86_64 2021-11-03T17:03:40.172 INFO:teuthology.orchestra.run.smithi115.stdout: libcephsqlite-2:16.2.4-0.el8.x86_64 libxslt-1.1.32-6.el8.x86_64 2021-11-03T17:03:40.172 INFO:teuthology.orchestra.run.smithi115.stdout: oniguruma-6.8.2-2.el8.x86_64 socat-1.7.3.3-2.el8.x86_64 2021-11-03T17:03:40.172 INFO:teuthology.orchestra.run.smithi115.stdout: xmlstarlet-1.6.1-11.el8.x86_64 2021-11-03T17:03:40.172 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:03:40.173 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2021-11-03T17:03:40.262 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph 2021-11-03T17:03:40.719 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:00:47 ago on Wed 03 Nov 2021 05:02:53 PM UTC. 2021-11-03T17:03:40.859 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2021-11-03T17:03:40.863 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:03:40.863 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2021-11-03T17:03:40.864 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:03:40.864 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2021-11-03T17:03:40.865 INFO:teuthology.orchestra.run.smithi027.stdout: ceph x86_64 2:16.2.4-0.el8 ceph 6.3 k 2021-11-03T17:03:40.865 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2021-11-03T17:03:40.865 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds x86_64 2:16.2.4-0.el8 ceph 2.4 M 2021-11-03T17:03:40.866 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr x86_64 2:16.2.4-0.el8 ceph 1.6 M 2021-11-03T17:03:40.866 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core noarch 2:16.2.4-0.el8 ceph-noarch 220 k 2021-11-03T17:03:40.866 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon x86_64 2:16.2.4-0.el8 ceph 5.0 M 2021-11-03T17:03:40.867 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd x86_64 2:16.2.4-0.el8 ceph 18 M 2021-11-03T17:03:40.867 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2021-11-03T17:03:40.867 INFO:teuthology.orchestra.run.smithi027.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2021-11-03T17:03:40.867 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cffi x86_64 1.11.5-5.el8 CentOS-Base 237 k 2021-11-03T17:03:40.867 INFO:teuthology.orchestra.run.smithi027.stdout: python3-chardet noarch 3.0.4-7.el8 CentOS-Base 195 k 2021-11-03T17:03:40.868 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2021-11-03T17:03:40.868 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2021-11-03T17:03:40.868 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cryptography x86_64 3.2.1-4.el8 CentOS-Base 559 k 2021-11-03T17:03:40.868 INFO:teuthology.orchestra.run.smithi027.stdout: python3-idna noarch 2.5-5.el8 CentOS-Base 97 k 2021-11-03T17:03:40.870 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2021-11-03T17:03:40.870 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2021-11-03T17:03:40.870 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2021-11-03T17:03:40.871 INFO:teuthology.orchestra.run.smithi027.stdout: python3-lxml x86_64 4.2.3-2.el8 CentOS-AppStream 1.5 M 2021-11-03T17:03:40.871 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako noarch 1.0.6-13.el8 CentOS-AppStream 157 k 2021-11-03T17:03:40.871 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe x86_64 0.23-19.el8 CentOS-AppStream 39 k 2021-11-03T17:03:40.871 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2021-11-03T17:03:40.871 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2021-11-03T17:03:40.872 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2021-11-03T17:03:40.872 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 CentOS-AppStream 103 k 2021-11-03T17:03:40.872 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pycparser noarch 2.14-14.el8 CentOS-Base 109 k 2021-11-03T17:03:40.872 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pysocks noarch 1.6.8-3.el8 CentOS-Base 34 k 2021-11-03T17:03:40.873 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz noarch 2017.2-9.el8 CentOS-AppStream 54 k 2021-11-03T17:03:40.873 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests noarch 2.20.0-2.1.el8_1 CentOS-Base 123 k 2021-11-03T17:03:40.874 INFO:teuthology.orchestra.run.smithi027.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2021-11-03T17:03:40.874 INFO:teuthology.orchestra.run.smithi027.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2021-11-03T17:03:40.876 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2021-11-03T17:03:40.876 INFO:teuthology.orchestra.run.smithi027.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2021-11-03T17:03:40.876 INFO:teuthology.orchestra.run.smithi027.stdout: python3-urllib3 noarch 1.24.2-5.el8 CentOS-Base 177 k 2021-11-03T17:03:40.877 INFO:teuthology.orchestra.run.smithi027.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2021-11-03T17:03:40.877 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webencodings noarch 0.5.1-6.el8 CentOS-AppStream 27 k 2021-11-03T17:03:40.877 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2021-11-03T17:03:40.877 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2021-11-03T17:03:40.878 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug noarch 0.12.2-4.el8 CentOS-AppStream 457 k 2021-11-03T17:03:40.878 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2021-11-03T17:03:40.878 INFO:teuthology.orchestra.run.smithi027.stdout:Installing weak dependencies: 2021-11-03T17:03:40.878 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cssselect noarch 0.9.2-10.el8 epel 40 k 2021-11-03T17:03:40.879 INFO:teuthology.orchestra.run.smithi027.stdout: python3-html5lib noarch 1:0.999999999-6.el8 CentOS-AppStream 214 k 2021-11-03T17:03:40.879 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:03:40.879 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2021-11-03T17:03:40.879 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:03:40.879 INFO:teuthology.orchestra.run.smithi027.stdout:Install 41 Packages 2021-11-03T17:03:40.880 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:03:40.882 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 33 M 2021-11-03T17:03:40.882 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 119 M 2021-11-03T17:03:40.883 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2021-11-03T17:03:40.890 INFO:teuthology.orchestra.run.smithi027.stdout:(1/41): ceph-16.2.4-0.el8.x86_64.rpm 307 kB/s | 6.3 kB 00:00 2021-11-03T17:03:40.933 INFO:teuthology.orchestra.run.smithi027.stdout:(2/41): ceph-mds-16.2.4-0.el8.x86_64.rpm 38 MB/s | 2.4 MB 00:00 2021-11-03T17:03:40.976 INFO:teuthology.orchestra.run.smithi027.stdout:(3/41): ceph-mgr-16.2.4-0.el8.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2021-11-03T17:03:40.991 INFO:teuthology.orchestra.run.smithi027.stdout:(4/41): ceph-mgr-modules-core-16.2.4-0.el8.noar 15 MB/s | 220 kB 00:00 2021-11-03T17:03:41.023 INFO:teuthology.orchestra.run.smithi027.stdout:(5/41): python3-html5lib-0.999999999-6.el8.noar 6.4 MB/s | 214 kB 00:00 2021-11-03T17:03:41.045 INFO:teuthology.orchestra.run.smithi027.stdout:(6/41): python3-lxml-4.2.3-2.el8.x86_64.rpm 71 MB/s | 1.5 MB 00:00 2021-11-03T17:03:41.048 INFO:teuthology.orchestra.run.smithi027.stdout:(7/41): python3-mako-1.0.6-13.el8.noarch.rpm 55 MB/s | 157 kB 00:00 2021-11-03T17:03:41.049 INFO:teuthology.orchestra.run.smithi027.stdout:(8/41): python3-markupsafe-0.23-19.el8.x86_64.r 30 MB/s | 39 kB 00:00 2021-11-03T17:03:41.052 INFO:teuthology.orchestra.run.smithi027.stdout:(9/41): python3-pyOpenSSL-19.0.0-1.el8.noarch.r 41 MB/s | 103 kB 00:00 2021-11-03T17:03:41.053 INFO:teuthology.orchestra.run.smithi027.stdout:(10/41): python3-pytz-2017.2-9.el8.noarch.rpm 34 MB/s | 54 kB 00:00 2021-11-03T17:03:41.055 INFO:teuthology.orchestra.run.smithi027.stdout:(11/41): python3-webencodings-0.5.1-6.el8.noarc 20 MB/s | 27 kB 00:00 2021-11-03T17:03:41.062 INFO:teuthology.orchestra.run.smithi027.stdout:(12/41): python3-werkzeug-0.12.2-4.el8.noarch.r 69 MB/s | 457 kB 00:00 2021-11-03T17:03:41.079 INFO:teuthology.orchestra.run.smithi027.stdout:(13/41): ceph-mon-16.2.4-0.el8.x86_64.rpm 26 MB/s | 5.0 MB 00:00 2021-11-03T17:03:41.080 INFO:teuthology.orchestra.run.smithi027.stdout:(14/41): python3-cffi-1.11.5-5.el8.x86_64.rpm 13 MB/s | 237 kB 00:00 2021-11-03T17:03:41.094 INFO:teuthology.orchestra.run.smithi027.stdout:(15/41): python3-cryptography-3.2.1-4.el8.x86_6 64 MB/s | 559 kB 00:00 2021-11-03T17:03:41.096 INFO:teuthology.orchestra.run.smithi027.stdout:(16/41): python3-idna-2.5-5.el8.noarch.rpm 44 MB/s | 97 kB 00:00 2021-11-03T17:03:41.098 INFO:teuthology.orchestra.run.smithi027.stdout:(17/41): python3-pycparser-2.14-14.el8.noarch.r 49 MB/s | 109 kB 00:00 2021-11-03T17:03:41.100 INFO:teuthology.orchestra.run.smithi027.stdout:(18/41): python3-pysocks-1.6.8-3.el8.noarch.rpm 25 MB/s | 34 kB 00:00 2021-11-03T17:03:41.102 INFO:teuthology.orchestra.run.smithi027.stdout:(19/41): python3-requests-2.20.0-2.1.el8_1.noar 52 MB/s | 123 kB 00:00 2021-11-03T17:03:41.105 INFO:teuthology.orchestra.run.smithi027.stdout:(20/41): python3-urllib3-1.24.2-5.el8.noarch.rp 59 MB/s | 177 kB 00:00 2021-11-03T17:03:41.114 INFO:teuthology.orchestra.run.smithi027.stdout:(21/41): python3-chardet-3.0.4-7.el8.noarch.rpm 5.4 MB/s | 195 kB 00:00 2021-11-03T17:03:41.278 INFO:teuthology.orchestra.run.smithi027.stdout:(22/41): ceph-osd-16.2.4-0.el8.x86_64.rpm 53 MB/s | 18 MB 00:00 2021-11-03T17:03:41.279 INFO:teuthology.orchestra.run.smithi027.stdout:(23/41): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 252 kB/s | 44 kB 00:00 2021-11-03T17:03:41.293 INFO:teuthology.orchestra.run.smithi027.stdout:(24/41): python3-beautifulsoup4-4.6.3-2.el8.1.n 1.0 MB/s | 185 kB 00:00 2021-11-03T17:03:41.332 INFO:teuthology.orchestra.run.smithi027.stdout:(25/41): python3-cssselect-0.9.2-10.el8.noarch. 1.0 MB/s | 40 kB 00:00 2021-11-03T17:03:41.376 INFO:teuthology.orchestra.run.smithi027.stdout:(26/41): python3-jaraco-6.2-6.el8.noarch.rpm 254 kB/s | 11 kB 00:00 2021-11-03T17:03:41.399 INFO:teuthology.orchestra.run.smithi027.stdout:(27/41): python3-cherrypy-18.4.0-1.el8.noarch.r 3.1 MB/s | 384 kB 00:00 2021-11-03T17:03:41.414 INFO:teuthology.orchestra.run.smithi027.stdout:(28/41): python3-jaraco-functools-2.0-4.el8.noa 518 kB/s | 18 kB 00:00 2021-11-03T17:03:41.417 INFO:teuthology.orchestra.run.smithi027.stdout:(29/41): python3-cheroot-8.5.2-1.el8.noarch.rpm 1.2 MB/s | 173 kB 00:00 2021-11-03T17:03:41.458 INFO:teuthology.orchestra.run.smithi027.stdout:(30/41): python3-logutils-0.3.5-11.el8.noarch.r 827 kB/s | 49 kB 00:00 2021-11-03T17:03:41.463 INFO:teuthology.orchestra.run.smithi027.stdout:(31/41): python3-more-itertools-7.2.0-3.el8.noa 1.1 MB/s | 59 kB 00:00 2021-11-03T17:03:41.493 INFO:teuthology.orchestra.run.smithi027.stdout:(32/41): python3-portend-2.6-1.el8.noarch.rpm 457 kB/s | 16 kB 00:00 2021-11-03T17:03:41.497 INFO:teuthology.orchestra.run.smithi027.stdout:(33/41): python3-pecan-1.3.2-9.el8.noarch.rpm 3.5 MB/s | 283 kB 00:00 2021-11-03T17:03:41.502 INFO:teuthology.orchestra.run.smithi027.stdout:(34/41): python3-simplegeneric-0.8.1-17.el8.noa 474 kB/s | 19 kB 00:00 2021-11-03T17:03:41.528 INFO:teuthology.orchestra.run.smithi027.stdout:(35/41): python3-singledispatch-3.4.0.3-18.el8. 672 kB/s | 24 kB 00:00 2021-11-03T17:03:41.530 INFO:teuthology.orchestra.run.smithi027.stdout:(36/41): python3-tempora-1.14.1-5.el8.noarch.rp 893 kB/s | 29 kB 00:00 2021-11-03T17:03:41.541 INFO:teuthology.orchestra.run.smithi027.stdout:(37/41): python3-trustme-0.6.0-4.el8.noarch.rpm 722 kB/s | 27 kB 00:00 2021-11-03T17:03:41.570 INFO:teuthology.orchestra.run.smithi027.stdout:(38/41): python3-waitress-1.2.1-2.el8.1.noarch. 5.7 MB/s | 241 kB 00:00 2021-11-03T17:03:41.580 INFO:teuthology.orchestra.run.smithi027.stdout:(39/41): python3-webtest-2.0.33-1.el8.noarch.rp 2.1 MB/s | 85 kB 00:00 2021-11-03T17:03:41.601 INFO:teuthology.orchestra.run.smithi027.stdout:(40/41): python3-zc-lockfile-2.0-2.el8.noarch.r 731 kB/s | 23 kB 00:00 2021-11-03T17:03:41.606 INFO:teuthology.orchestra.run.smithi027.stdout:(41/41): python3-webob-1.8.5-1.el8.1.noarch.rpm 3.2 MB/s | 251 kB 00:00 2021-11-03T17:03:41.611 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2021-11-03T17:03:41.611 INFO:teuthology.orchestra.run.smithi027.stdout:Total 45 MB/s | 33 MB 00:00 2021-11-03T17:03:41.641 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2021-11-03T17:03:41.687 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2021-11-03T17:03:41.687 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2021-11-03T17:03:41.716 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2021-11-03T17:03:41.720 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:03:41.720 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2021-11-03T17:03:41.720 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:03:41.721 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2021-11-03T17:03:41.721 INFO:teuthology.orchestra.run.smithi115.stdout: ceph x86_64 2:16.2.4-0.el8 ceph 6.3 k 2021-11-03T17:03:41.721 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2021-11-03T17:03:41.721 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mds x86_64 2:16.2.4-0.el8 ceph 2.4 M 2021-11-03T17:03:41.722 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr x86_64 2:16.2.4-0.el8 ceph 1.6 M 2021-11-03T17:03:41.722 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-modules-core noarch 2:16.2.4-0.el8 ceph-noarch 220 k 2021-11-03T17:03:41.722 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mon x86_64 2:16.2.4-0.el8 ceph 5.0 M 2021-11-03T17:03:41.722 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-osd x86_64 2:16.2.4-0.el8 ceph 18 M 2021-11-03T17:03:41.723 INFO:teuthology.orchestra.run.smithi115.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2021-11-03T17:03:41.723 INFO:teuthology.orchestra.run.smithi115.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2021-11-03T17:03:41.723 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cffi x86_64 1.11.5-5.el8 CentOS-Base 237 k 2021-11-03T17:03:41.723 INFO:teuthology.orchestra.run.smithi115.stdout: python3-chardet noarch 3.0.4-7.el8 CentOS-Base 195 k 2021-11-03T17:03:41.724 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2021-11-03T17:03:41.724 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2021-11-03T17:03:41.724 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cryptography x86_64 3.2.1-4.el8 CentOS-Base 559 k 2021-11-03T17:03:41.724 INFO:teuthology.orchestra.run.smithi115.stdout: python3-idna noarch 2.5-5.el8 CentOS-Base 97 k 2021-11-03T17:03:41.724 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2021-11-03T17:03:41.725 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2021-11-03T17:03:41.725 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2021-11-03T17:03:41.725 INFO:teuthology.orchestra.run.smithi115.stdout: python3-lxml x86_64 4.2.3-2.el8 CentOS-AppStream 1.5 M 2021-11-03T17:03:41.725 INFO:teuthology.orchestra.run.smithi115.stdout: python3-mako noarch 1.0.6-13.el8 CentOS-AppStream 157 k 2021-11-03T17:03:41.726 INFO:teuthology.orchestra.run.smithi115.stdout: python3-markupsafe x86_64 0.23-19.el8 CentOS-AppStream 39 k 2021-11-03T17:03:41.726 INFO:teuthology.orchestra.run.smithi115.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2021-11-03T17:03:41.726 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2021-11-03T17:03:41.726 INFO:teuthology.orchestra.run.smithi115.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2021-11-03T17:03:41.727 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 CentOS-AppStream 103 k 2021-11-03T17:03:41.727 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pycparser noarch 2.14-14.el8 CentOS-Base 109 k 2021-11-03T17:03:41.727 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pysocks noarch 1.6.8-3.el8 CentOS-Base 34 k 2021-11-03T17:03:41.727 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytz noarch 2017.2-9.el8 CentOS-AppStream 54 k 2021-11-03T17:03:41.727 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests noarch 2.20.0-2.1.el8_1 CentOS-Base 123 k 2021-11-03T17:03:41.728 INFO:teuthology.orchestra.run.smithi115.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2021-11-03T17:03:41.728 INFO:teuthology.orchestra.run.smithi115.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2021-11-03T17:03:41.728 INFO:teuthology.orchestra.run.smithi115.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2021-11-03T17:03:41.728 INFO:teuthology.orchestra.run.smithi115.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2021-11-03T17:03:41.729 INFO:teuthology.orchestra.run.smithi115.stdout: python3-urllib3 noarch 1.24.2-5.el8 CentOS-Base 177 k 2021-11-03T17:03:41.729 INFO:teuthology.orchestra.run.smithi115.stdout: python3-waitress noarch 1.2.1-2.el8.1 epel 241 k 2021-11-03T17:03:41.729 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webencodings noarch 0.5.1-6.el8 CentOS-AppStream 27 k 2021-11-03T17:03:41.729 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2021-11-03T17:03:41.729 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2021-11-03T17:03:41.730 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug noarch 0.12.2-4.el8 CentOS-AppStream 457 k 2021-11-03T17:03:41.730 INFO:teuthology.orchestra.run.smithi115.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2021-11-03T17:03:41.731 INFO:teuthology.orchestra.run.smithi115.stdout:Installing weak dependencies: 2021-11-03T17:03:41.731 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cssselect noarch 0.9.2-10.el8 epel 40 k 2021-11-03T17:03:41.731 INFO:teuthology.orchestra.run.smithi115.stdout: python3-html5lib noarch 1:0.999999999-6.el8 CentOS-AppStream 214 k 2021-11-03T17:03:41.731 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:03:41.732 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2021-11-03T17:03:41.732 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:03:41.732 INFO:teuthology.orchestra.run.smithi115.stdout:Install 41 Packages 2021-11-03T17:03:41.732 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:03:41.733 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 33 M 2021-11-03T17:03:41.733 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 119 M 2021-11-03T17:03:41.734 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2021-11-03T17:03:41.745 INFO:teuthology.orchestra.run.smithi115.stdout:(1/41): ceph-16.2.4-0.el8.x86_64.rpm 325 kB/s | 6.3 kB 00:00 2021-11-03T17:03:41.798 INFO:teuthology.orchestra.run.smithi115.stdout:(2/41): ceph-mgr-16.2.4-0.el8.x86_64.rpm 22 MB/s | 1.6 MB 00:00 2021-11-03T17:03:41.806 INFO:teuthology.orchestra.run.smithi115.stdout:(3/41): ceph-mds-16.2.4-0.el8.x86_64.rpm 30 MB/s | 2.4 MB 00:00 2021-11-03T17:03:41.812 INFO:teuthology.orchestra.run.smithi115.stdout:(4/41): ceph-mgr-modules-core-16.2.4-0.el8.noar 34 MB/s | 220 kB 00:00 2021-11-03T17:03:41.843 INFO:teuthology.orchestra.run.smithi115.stdout:(5/41): python3-html5lib-0.999999999-6.el8.noar 6.7 MB/s | 214 kB 00:00 2021-11-03T17:03:41.864 INFO:teuthology.orchestra.run.smithi115.stdout:(6/41): python3-lxml-4.2.3-2.el8.x86_64.rpm 74 MB/s | 1.5 MB 00:00 2021-11-03T17:03:41.867 INFO:teuthology.orchestra.run.smithi115.stdout:(7/41): python3-mako-1.0.6-13.el8.noarch.rpm 58 MB/s | 157 kB 00:00 2021-11-03T17:03:41.880 INFO:teuthology.orchestra.run.smithi115.stdout:(8/41): ceph-mon-16.2.4-0.el8.x86_64.rpm 37 MB/s | 5.0 MB 00:00 2021-11-03T17:03:41.881 INFO:teuthology.orchestra.run.smithi115.stdout:(9/41): python3-markupsafe-0.23-19.el8.x86_64.r 2.7 MB/s | 39 kB 00:00 2021-11-03T17:03:41.887 INFO:teuthology.orchestra.run.smithi115.stdout:(10/41): python3-pytz-2017.2-9.el8.noarch.rpm 9.8 MB/s | 54 kB 00:00 2021-11-03T17:03:41.888 INFO:teuthology.orchestra.run.smithi115.stdout:(11/41): python3-webencodings-0.5.1-6.el8.noarc 19 MB/s | 27 kB 00:00 2021-11-03T17:03:41.895 INFO:teuthology.orchestra.run.smithi115.stdout:(12/41): python3-werkzeug-0.12.2-4.el8.noarch.r 68 MB/s | 457 kB 00:00 2021-11-03T17:03:41.899 INFO:teuthology.orchestra.run.smithi115.stdout:(13/41): python3-cffi-1.11.5-5.el8.x86_64.rpm 62 MB/s | 237 kB 00:00 2021-11-03T17:03:41.902 INFO:teuthology.orchestra.run.smithi115.stdout:(14/41): python3-chardet-3.0.4-7.el8.noarch.rpm 62 MB/s | 195 kB 00:00 2021-11-03T17:03:41.911 INFO:teuthology.orchestra.run.smithi115.stdout:(15/41): python3-cryptography-3.2.1-4.el8.x86_6 65 MB/s | 559 kB 00:00 2021-11-03T17:03:41.912 INFO:teuthology.orchestra.run.smithi115.stdout:(16/41): python3-pyOpenSSL-19.0.0-1.el8.noarch. 3.2 MB/s | 103 kB 00:00 2021-11-03T17:03:41.913 INFO:teuthology.orchestra.run.smithi115.stdout:(17/41): python3-idna-2.5-5.el8.noarch.rpm 43 MB/s | 97 kB 00:00 2021-11-03T17:03:41.914 INFO:teuthology.orchestra.run.smithi115.stdout:(18/41): python3-pycparser-2.14-14.el8.noarch.r 50 MB/s | 109 kB 00:00 2021-11-03T17:03:41.915 INFO:teuthology.orchestra.run.smithi115.stdout:(19/41): python3-pysocks-1.6.8-3.el8.noarch.rpm 21 MB/s | 34 kB 00:00 2021-11-03T17:03:41.918 INFO:teuthology.orchestra.run.smithi115.stdout:(20/41): python3-requests-2.20.0-2.1.el8_1.noar 36 MB/s | 123 kB 00:00 2021-11-03T17:03:41.919 INFO:teuthology.orchestra.run.smithi115.stdout:(21/41): python3-urllib3-1.24.2-5.el8.noarch.rp 46 MB/s | 177 kB 00:00 2021-11-03T17:03:41.940 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2021-11-03T17:03:41.943 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2021-11-03T17:03:42.130 INFO:teuthology.orchestra.run.smithi115.stdout:(22/41): ceph-osd-16.2.4-0.el8.x86_64.rpm 55 MB/s | 18 MB 00:00 2021-11-03T17:03:42.131 INFO:teuthology.orchestra.run.smithi115.stdout:(23/41): python3-bcrypt-3.1.6-2.el8.1.x86_64.rp 205 kB/s | 44 kB 00:00 2021-11-03T17:03:42.131 INFO:teuthology.orchestra.run.smithi115.stdout:(24/41): python3-beautifulsoup4-4.6.3-2.el8.1.n 872 kB/s | 185 kB 00:00 2021-11-03T17:03:42.156 INFO:teuthology.orchestra.run.smithi115.stdout:(25/41): python3-cssselect-0.9.2-10.el8.noarch. 1.6 MB/s | 40 kB 00:00 2021-11-03T17:03:42.181 INFO:teuthology.orchestra.run.smithi115.stdout:(26/41): python3-jaraco-6.2-6.el8.noarch.rpm 455 kB/s | 11 kB 00:00 2021-11-03T17:03:42.206 INFO:teuthology.orchestra.run.smithi115.stdout:(27/41): python3-jaraco-functools-2.0-4.el8.noa 710 kB/s | 18 kB 00:00 2021-11-03T17:03:42.216 INFO:teuthology.orchestra.run.smithi115.stdout:(28/41): python3-cheroot-8.5.2-1.el8.noarch.rpm 2.0 MB/s | 173 kB 00:00 2021-11-03T17:03:42.231 INFO:teuthology.orchestra.run.smithi115.stdout:(29/41): python3-logutils-0.3.5-11.el8.noarch.r 1.9 MB/s | 49 kB 00:00 2021-11-03T17:03:42.244 INFO:teuthology.orchestra.run.smithi115.stdout:(30/41): python3-more-itertools-7.2.0-3.el8.noa 2.0 MB/s | 59 kB 00:00 2021-11-03T17:03:42.272 INFO:teuthology.orchestra.run.smithi115.stdout:(31/41): python3-portend-2.6-1.el8.noarch.rpm 558 kB/s | 16 kB 00:00 2021-11-03T17:03:42.280 INFO:teuthology.orchestra.run.smithi115.stdout:(32/41): python3-pecan-1.3.2-9.el8.noarch.rpm 5.6 MB/s | 283 kB 00:00 2021-11-03T17:03:42.300 INFO:teuthology.orchestra.run.smithi115.stdout:(33/41): python3-simplegeneric-0.8.1-17.el8.noa 675 kB/s | 19 kB 00:00 2021-11-03T17:03:42.305 INFO:teuthology.orchestra.run.smithi115.stdout:(34/41): python3-singledispatch-3.4.0.3-18.el8. 976 kB/s | 24 kB 00:00 2021-11-03T17:03:42.326 INFO:teuthology.orchestra.run.smithi115.stdout:(35/41): python3-cherrypy-18.4.0-1.el8.noarch.r 1.9 MB/s | 384 kB 00:00 2021-11-03T17:03:42.328 INFO:teuthology.orchestra.run.smithi115.stdout:(36/41): python3-tempora-1.14.1-5.el8.noarch.rp 1.0 MB/s | 29 kB 00:00 2021-11-03T17:03:42.330 INFO:teuthology.orchestra.run.smithi115.stdout:(37/41): python3-trustme-0.6.0-4.el8.noarch.rpm 1.1 MB/s | 27 kB 00:00 2021-11-03T17:03:42.356 INFO:teuthology.orchestra.run.smithi115.stdout:(38/41): python3-webtest-2.0.33-1.el8.noarch.rp 3.2 MB/s | 85 kB 00:00 2021-11-03T17:03:42.357 INFO:teuthology.orchestra.run.smithi115.stdout:(39/41): python3-waitress-1.2.1-2.el8.1.noarch. 7.5 MB/s | 241 kB 00:00 2021-11-03T17:03:42.381 INFO:teuthology.orchestra.run.smithi115.stdout:(40/41): python3-zc-lockfile-2.0-2.el8.noarch.r 933 kB/s | 23 kB 00:00 2021-11-03T17:03:42.385 INFO:teuthology.orchestra.run.smithi115.stdout:(41/41): python3-webob-1.8.5-1.el8.1.noarch.rpm 4.3 MB/s | 251 kB 00:00 2021-11-03T17:03:42.390 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2021-11-03T17:03:42.390 INFO:teuthology.orchestra.run.smithi115.stdout:Total 50 MB/s | 33 MB 00:00 2021-11-03T17:03:42.416 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2021-11-03T17:03:42.457 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2021-11-03T17:03:42.457 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2021-11-03T17:03:42.691 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2021-11-03T17:03:42.694 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2021-11-03T17:03:43.941 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2021-11-03T17:03:44.012 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2021-11-03T17:03:44.228 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/41 2021-11-03T17:03:44.293 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/41 2021-11-03T17:03:44.449 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/41 2021-11-03T17:03:44.517 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/41 2021-11-03T17:03:44.628 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-idna-2.5-5.el8.noarch 3/41 2021-11-03T17:03:44.671 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-idna-2.5-5.el8.noarch 3/41 2021-11-03T17:03:44.846 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 4/41 2021-11-03T17:03:44.895 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 4/41 2021-11-03T17:03:45.027 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 5/41 2021-11-03T17:03:45.111 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-waitress-1.2.1-2.el8.1.noarch 5/41 2021-11-03T17:03:45.183 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 6/41 2021-11-03T17:03:45.267 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 6/41 2021-11-03T17:03:45.330 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 7/41 2021-11-03T17:03:45.425 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 7/41 2021-11-03T17:03:45.489 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 8/41 2021-11-03T17:03:45.598 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 8/41 2021-11-03T17:03:45.625 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 9/41 2021-11-03T17:03:45.769 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 9/41 2021-11-03T17:03:45.781 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 10/41 2021-11-03T17:03:45.927 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 10/41 2021-11-03T17:03:45.927 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cssselect-0.9.2-10.el8.noarch 11/41 2021-11-03T17:03:46.071 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cssselect-0.9.2-10.el8.noarch 11/41 2021-11-03T17:03:46.166 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pysocks-1.6.8-3.el8.noarch 12/41 2021-11-03T17:03:46.321 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pysocks-1.6.8-3.el8.noarch 12/41 2021-11-03T17:03:46.418 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-urllib3-1.24.2-5.el8.noarch 13/41 2021-11-03T17:03:46.550 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-urllib3-1.24.2-5.el8.noarch 13/41 2021-11-03T17:03:46.649 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 14/41 2021-11-03T17:03:46.795 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 14/41 2021-11-03T17:03:47.037 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 15/41 2021-11-03T17:03:47.273 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cryptography-3.2.1-4.el8.x86_64 16/41 2021-11-03T17:03:47.333 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 15/41 2021-11-03T17:03:47.427 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 17/41 2021-11-03T17:03:47.593 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cryptography-3.2.1-4.el8.x86_64 16/41 2021-11-03T17:03:47.646 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 18/41 2021-11-03T17:03:47.770 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 17/41 2021-11-03T17:03:47.871 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 19/41 2021-11-03T17:03:47.994 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 18/41 2021-11-03T17:03:48.103 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 20/41 2021-11-03T17:03:48.247 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 19/41 2021-11-03T17:03:48.291 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-chardet-3.0.4-7.el8.noarch 21/41 2021-11-03T17:03:48.496 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 20/41 2021-11-03T17:03:48.548 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-requests-2.20.0-2.1.el8_1.noarch 22/41 2021-11-03T17:03:48.713 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-chardet-3.0.4-7.el8.noarch 21/41 2021-11-03T17:03:48.755 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 23/41 2021-11-03T17:03:48.982 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-webencodings-0.5.1-6.el8.noarch 24/41 2021-11-03T17:03:48.991 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-requests-2.20.0-2.1.el8_1.noarch 22/41 2021-11-03T17:03:49.191 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 23/41 2021-11-03T17:03:49.199 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-html5lib-1:0.999999999-6.el8.noarch 25/41 2021-11-03T17:03:49.387 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-webencodings-0.5.1-6.el8.noarch 24/41 2021-11-03T17:03:49.562 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 26/41 2021-11-03T17:03:49.753 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-html5lib-1:0.999999999-6.el8.noarch 25/41 2021-11-03T17:03:49.770 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-lxml-4.2.3-2.el8.x86_64 27/41 2021-11-03T17:03:49.925 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 28/41 2021-11-03T17:03:50.116 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 26/41 2021-11-03T17:03:50.130 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 29/41 2021-11-03T17:03:50.349 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 30/41 2021-11-03T17:03:50.360 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-lxml-4.2.3-2.el8.x86_64 27/41 2021-11-03T17:03:50.552 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 28/41 2021-11-03T17:03:50.640 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-portend-2.6-1.el8.noarch 31/41 2021-11-03T17:03:50.757 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 29/41 2021-11-03T17:03:50.865 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 32/41 2021-11-03T17:03:50.934 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 30/41 2021-11-03T17:03:51.075 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 33/41 2021-11-03T17:03:51.217 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-portend-2.6-1.el8.noarch 31/41 2021-11-03T17:03:51.417 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 32/41 2021-11-03T17:03:51.444 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 34/41 2021-11-03T17:03:51.630 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 33/41 2021-11-03T17:03:51.772 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 35/41 2021-11-03T17:03:52.123 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-modules-core-2:16.2.4-0.el8.noarch 36/41 2021-11-03T17:03:52.140 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 34/41 2021-11-03T17:03:52.294 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-2:16.2.4-0.el8.x86_64 37/41 2021-11-03T17:03:52.517 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 35/41 2021-11-03T17:03:52.871 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-modules-core-2:16.2.4-0.el8.noarch 36/41 2021-11-03T17:03:53.040 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-2:16.2.4-0.el8.x86_64 37/41 2021-11-03T17:03:53.815 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-2:16.2.4-0.el8.x86_64 37/41 2021-11-03T17:03:54.035 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-osd-2:16.2.4-0.el8.x86_64 38/41 2021-11-03T17:03:54.467 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-osd-2:16.2.4-0.el8.x86_64 38/41 2021-11-03T17:03:54.635 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mon-2:16.2.4-0.el8.x86_64 39/41 2021-11-03T17:03:54.659 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-2:16.2.4-0.el8.x86_64 37/41 2021-11-03T17:03:54.859 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mon-2:16.2.4-0.el8.x86_64 39/41 2021-11-03T17:03:54.865 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-osd-2:16.2.4-0.el8.x86_64 38/41 2021-11-03T17:03:55.201 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mds-2:16.2.4-0.el8.x86_64 40/41 2021-11-03T17:03:55.218 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mds-2:16.2.4-0.el8.x86_64 40/41 2021-11-03T17:03:55.324 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-osd-2:16.2.4-0.el8.x86_64 38/41 2021-11-03T17:03:55.325 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-2:16.2.4-0.el8.x86_64 41/41 2021-11-03T17:03:55.540 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mon-2:16.2.4-0.el8.x86_64 39/41 2021-11-03T17:03:55.716 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-2:16.2.4-0.el8.x86_64 41/41 2021-11-03T17:03:55.717 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-2:16.2.4-0.el8.x86_64 1/41 2021-11-03T17:03:55.717 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mds-2:16.2.4-0.el8.x86_64 2/41 2021-11-03T17:03:55.717 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-2:16.2.4-0.el8.x86_64 3/41 2021-11-03T17:03:55.718 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mon-2:16.2.4-0.el8.x86_64 4/41 2021-11-03T17:03:55.718 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-osd-2:16.2.4-0.el8.x86_64 5/41 2021-11-03T17:03:55.718 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-modules-core-2:16.2.4-0.el8.noarch 6/41 2021-11-03T17:03:55.718 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-html5lib-1:0.999999999-6.el8.noarch 7/41 2021-11-03T17:03:55.719 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-lxml-4.2.3-2.el8.x86_64 8/41 2021-11-03T17:03:55.719 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 9/41 2021-11-03T17:03:55.719 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 10/41 2021-11-03T17:03:55.719 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 11/41 2021-11-03T17:03:55.720 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 12/41 2021-11-03T17:03:55.720 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-webencodings-0.5.1-6.el8.noarch 13/41 2021-11-03T17:03:55.720 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 14/41 2021-11-03T17:03:55.720 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 15/41 2021-11-03T17:03:55.721 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-chardet-3.0.4-7.el8.noarch 16/41 2021-11-03T17:03:55.721 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cryptography-3.2.1-4.el8.x86_64 17/41 2021-11-03T17:03:55.721 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-idna-2.5-5.el8.noarch 18/41 2021-11-03T17:03:55.721 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 19/41 2021-11-03T17:03:55.722 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pysocks-1.6.8-3.el8.noarch 20/41 2021-11-03T17:03:55.722 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-requests-2.20.0-2.1.el8_1.noarch 21/41 2021-11-03T17:03:55.722 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-urllib3-1.24.2-5.el8.noarch 22/41 2021-11-03T17:03:55.722 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 23/41 2021-11-03T17:03:55.723 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 24/41 2021-11-03T17:03:55.723 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 25/41 2021-11-03T17:03:55.723 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 26/41 2021-11-03T17:03:55.723 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cssselect-0.9.2-10.el8.noarch 27/41 2021-11-03T17:03:55.723 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 28/41 2021-11-03T17:03:55.724 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 29/41 2021-11-03T17:03:55.724 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 30/41 2021-11-03T17:03:55.724 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 31/41 2021-11-03T17:03:55.724 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 32/41 2021-11-03T17:03:55.725 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-portend-2.6-1.el8.noarch 33/41 2021-11-03T17:03:55.725 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 34/41 2021-11-03T17:03:55.725 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 35/41 2021-11-03T17:03:55.725 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 36/41 2021-11-03T17:03:55.726 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 37/41 2021-11-03T17:03:55.726 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 38/41 2021-11-03T17:03:55.726 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 39/41 2021-11-03T17:03:55.726 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 40/41 2021-11-03T17:03:55.775 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mon-2:16.2.4-0.el8.x86_64 39/41 2021-11-03T17:03:55.952 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mds-2:16.2.4-0.el8.x86_64 40/41 2021-11-03T17:03:55.969 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mds-2:16.2.4-0.el8.x86_64 40/41 2021-11-03T17:03:56.143 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-2:16.2.4-0.el8.x86_64 41/41 2021-11-03T17:03:56.550 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-2:16.2.4-0.el8.x86_64 41/41 2021-11-03T17:03:56.550 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-2:16.2.4-0.el8.x86_64 1/41 2021-11-03T17:03:56.551 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mds-2:16.2.4-0.el8.x86_64 2/41 2021-11-03T17:03:56.551 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-2:16.2.4-0.el8.x86_64 3/41 2021-11-03T17:03:56.552 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mon-2:16.2.4-0.el8.x86_64 4/41 2021-11-03T17:03:56.552 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-osd-2:16.2.4-0.el8.x86_64 5/41 2021-11-03T17:03:56.552 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-modules-core-2:16.2.4-0.el8.noarch 6/41 2021-11-03T17:03:56.553 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-html5lib-1:0.999999999-6.el8.noarch 7/41 2021-11-03T17:03:56.553 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-lxml-4.2.3-2.el8.x86_64 8/41 2021-11-03T17:03:56.553 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 9/41 2021-11-03T17:03:56.553 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 10/41 2021-11-03T17:03:56.553 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 11/41 2021-11-03T17:03:56.554 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 12/41 2021-11-03T17:03:56.554 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-webencodings-0.5.1-6.el8.noarch 13/41 2021-11-03T17:03:56.554 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 14/41 2021-11-03T17:03:56.554 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 15/41 2021-11-03T17:03:56.555 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-chardet-3.0.4-7.el8.noarch 16/41 2021-11-03T17:03:56.555 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cryptography-3.2.1-4.el8.x86_64 17/41 2021-11-03T17:03:56.555 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-idna-2.5-5.el8.noarch 18/41 2021-11-03T17:03:56.555 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 19/41 2021-11-03T17:03:56.555 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pysocks-1.6.8-3.el8.noarch 20/41 2021-11-03T17:03:56.556 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-requests-2.20.0-2.1.el8_1.noarch 21/41 2021-11-03T17:03:56.556 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-urllib3-1.24.2-5.el8.noarch 22/41 2021-11-03T17:03:56.556 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 23/41 2021-11-03T17:03:56.556 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 24/41 2021-11-03T17:03:56.557 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 25/41 2021-11-03T17:03:56.557 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 26/41 2021-11-03T17:03:56.557 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cssselect-0.9.2-10.el8.noarch 27/41 2021-11-03T17:03:56.557 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 28/41 2021-11-03T17:03:56.557 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 29/41 2021-11-03T17:03:56.558 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 30/41 2021-11-03T17:03:56.558 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 31/41 2021-11-03T17:03:56.558 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 32/41 2021-11-03T17:03:56.558 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-portend-2.6-1.el8.noarch 33/41 2021-11-03T17:03:56.558 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 34/41 2021-11-03T17:03:56.559 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 35/41 2021-11-03T17:03:56.559 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 36/41 2021-11-03T17:03:56.559 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 37/41 2021-11-03T17:03:56.559 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-waitress-1.2.1-2.el8.1.noarch 38/41 2021-11-03T17:03:56.560 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 39/41 2021-11-03T17:03:56.560 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 40/41 2021-11-03T17:03:56.561 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 41/41 2021-11-03T17:03:56.561 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:03:56.561 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2021-11-03T17:03:56.561 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-2:16.2.4-0.el8.x86_64 2021-11-03T17:03:56.562 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mds-2:16.2.4-0.el8.x86_64 2021-11-03T17:03:56.562 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-2:16.2.4-0.el8.x86_64 2021-11-03T17:03:56.562 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-modules-core-2:16.2.4-0.el8.noarch 2021-11-03T17:03:56.562 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mon-2:16.2.4-0.el8.x86_64 2021-11-03T17:03:56.563 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-osd-2:16.2.4-0.el8.x86_64 2021-11-03T17:03:56.563 INFO:teuthology.orchestra.run.smithi115.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2021-11-03T17:03:56.563 INFO:teuthology.orchestra.run.smithi115.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2021-11-03T17:03:56.563 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cffi-1.11.5-5.el8.x86_64 2021-11-03T17:03:56.564 INFO:teuthology.orchestra.run.smithi115.stdout: python3-chardet-3.0.4-7.el8.noarch 2021-11-03T17:03:56.564 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cheroot-8.5.2-1.el8.noarch 2021-11-03T17:03:56.564 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2021-11-03T17:03:56.564 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cryptography-3.2.1-4.el8.x86_64 2021-11-03T17:03:56.565 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cssselect-0.9.2-10.el8.noarch 2021-11-03T17:03:56.565 INFO:teuthology.orchestra.run.smithi115.stdout: python3-html5lib-1:0.999999999-6.el8.noarch 2021-11-03T17:03:56.565 INFO:teuthology.orchestra.run.smithi115.stdout: python3-idna-2.5-5.el8.noarch 2021-11-03T17:03:56.565 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-6.2-6.el8.noarch 2021-11-03T17:03:56.565 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2021-11-03T17:03:56.566 INFO:teuthology.orchestra.run.smithi115.stdout: python3-logutils-0.3.5-11.el8.noarch 2021-11-03T17:03:56.566 INFO:teuthology.orchestra.run.smithi115.stdout: python3-lxml-4.2.3-2.el8.x86_64 2021-11-03T17:03:56.566 INFO:teuthology.orchestra.run.smithi115.stdout: python3-mako-1.0.6-13.el8.noarch 2021-11-03T17:03:56.567 INFO:teuthology.orchestra.run.smithi115.stdout: python3-markupsafe-0.23-19.el8.x86_64 2021-11-03T17:03:56.567 INFO:teuthology.orchestra.run.smithi115.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2021-11-03T17:03:56.567 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pecan-1.3.2-9.el8.noarch 2021-11-03T17:03:56.567 INFO:teuthology.orchestra.run.smithi115.stdout: python3-portend-2.6-1.el8.noarch 2021-11-03T17:03:56.567 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2021-11-03T17:03:56.568 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pycparser-2.14-14.el8.noarch 2021-11-03T17:03:56.568 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pysocks-1.6.8-3.el8.noarch 2021-11-03T17:03:56.568 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pytz-2017.2-9.el8.noarch 2021-11-03T17:03:56.568 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-2.20.0-2.1.el8_1.noarch 2021-11-03T17:03:56.568 INFO:teuthology.orchestra.run.smithi115.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2021-11-03T17:03:56.569 INFO:teuthology.orchestra.run.smithi115.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2021-11-03T17:03:56.569 INFO:teuthology.orchestra.run.smithi115.stdout: python3-tempora-1.14.1-5.el8.noarch 2021-11-03T17:03:56.569 INFO:teuthology.orchestra.run.smithi115.stdout: python3-trustme-0.6.0-4.el8.noarch 2021-11-03T17:03:56.569 INFO:teuthology.orchestra.run.smithi115.stdout: python3-urllib3-1.24.2-5.el8.noarch 2021-11-03T17:03:56.570 INFO:teuthology.orchestra.run.smithi115.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2021-11-03T17:03:56.570 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webencodings-0.5.1-6.el8.noarch 2021-11-03T17:03:56.570 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webob-1.8.5-1.el8.1.noarch 2021-11-03T17:03:56.571 INFO:teuthology.orchestra.run.smithi115.stdout: python3-webtest-2.0.33-1.el8.noarch 2021-11-03T17:03:56.571 INFO:teuthology.orchestra.run.smithi115.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2021-11-03T17:03:56.571 INFO:teuthology.orchestra.run.smithi115.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2021-11-03T17:03:56.571 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:03:56.571 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2021-11-03T17:03:56.744 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-base 2021-11-03T17:03:57.201 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:04 ago on Wed 03 Nov 2021 05:02:53 PM UTC. 2021-11-03T17:03:57.424 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 41/41 2021-11-03T17:03:57.424 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:03:57.424 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2021-11-03T17:03:57.425 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-2:16.2.4-0.el8.x86_64 2021-11-03T17:03:57.425 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mds-2:16.2.4-0.el8.x86_64 2021-11-03T17:03:57.425 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-2:16.2.4-0.el8.x86_64 2021-11-03T17:03:57.425 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-modules-core-2:16.2.4-0.el8.noarch 2021-11-03T17:03:57.426 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mon-2:16.2.4-0.el8.x86_64 2021-11-03T17:03:57.426 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-osd-2:16.2.4-0.el8.x86_64 2021-11-03T17:03:57.426 INFO:teuthology.orchestra.run.smithi027.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2021-11-03T17:03:57.426 INFO:teuthology.orchestra.run.smithi027.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2021-11-03T17:03:57.427 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cffi-1.11.5-5.el8.x86_64 2021-11-03T17:03:57.427 INFO:teuthology.orchestra.run.smithi027.stdout: python3-chardet-3.0.4-7.el8.noarch 2021-11-03T17:03:57.427 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cheroot-8.5.2-1.el8.noarch 2021-11-03T17:03:57.427 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2021-11-03T17:03:57.428 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cryptography-3.2.1-4.el8.x86_64 2021-11-03T17:03:57.428 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cssselect-0.9.2-10.el8.noarch 2021-11-03T17:03:57.428 INFO:teuthology.orchestra.run.smithi027.stdout: python3-html5lib-1:0.999999999-6.el8.noarch 2021-11-03T17:03:57.428 INFO:teuthology.orchestra.run.smithi027.stdout: python3-idna-2.5-5.el8.noarch 2021-11-03T17:03:57.428 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-6.2-6.el8.noarch 2021-11-03T17:03:57.429 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2021-11-03T17:03:57.429 INFO:teuthology.orchestra.run.smithi027.stdout: python3-logutils-0.3.5-11.el8.noarch 2021-11-03T17:03:57.429 INFO:teuthology.orchestra.run.smithi027.stdout: python3-lxml-4.2.3-2.el8.x86_64 2021-11-03T17:03:57.429 INFO:teuthology.orchestra.run.smithi027.stdout: python3-mako-1.0.6-13.el8.noarch 2021-11-03T17:03:57.430 INFO:teuthology.orchestra.run.smithi027.stdout: python3-markupsafe-0.23-19.el8.x86_64 2021-11-03T17:03:57.430 INFO:teuthology.orchestra.run.smithi027.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2021-11-03T17:03:57.430 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pecan-1.3.2-9.el8.noarch 2021-11-03T17:03:57.430 INFO:teuthology.orchestra.run.smithi027.stdout: python3-portend-2.6-1.el8.noarch 2021-11-03T17:03:57.431 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2021-11-03T17:03:57.431 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pycparser-2.14-14.el8.noarch 2021-11-03T17:03:57.431 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pysocks-1.6.8-3.el8.noarch 2021-11-03T17:03:57.431 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pytz-2017.2-9.el8.noarch 2021-11-03T17:03:57.431 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-2.20.0-2.1.el8_1.noarch 2021-11-03T17:03:57.432 INFO:teuthology.orchestra.run.smithi027.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2021-11-03T17:03:57.432 INFO:teuthology.orchestra.run.smithi027.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2021-11-03T17:03:57.432 INFO:teuthology.orchestra.run.smithi027.stdout: python3-tempora-1.14.1-5.el8.noarch 2021-11-03T17:03:57.432 INFO:teuthology.orchestra.run.smithi027.stdout: python3-trustme-0.6.0-4.el8.noarch 2021-11-03T17:03:57.433 INFO:teuthology.orchestra.run.smithi027.stdout: python3-urllib3-1.24.2-5.el8.noarch 2021-11-03T17:03:57.433 INFO:teuthology.orchestra.run.smithi027.stdout: python3-waitress-1.2.1-2.el8.1.noarch 2021-11-03T17:03:57.433 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webencodings-0.5.1-6.el8.noarch 2021-11-03T17:03:57.433 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webob-1.8.5-1.el8.1.noarch 2021-11-03T17:03:57.433 INFO:teuthology.orchestra.run.smithi027.stdout: python3-webtest-2.0.33-1.el8.noarch 2021-11-03T17:03:57.434 INFO:teuthology.orchestra.run.smithi027.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2021-11-03T17:03:57.434 INFO:teuthology.orchestra.run.smithi027.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2021-11-03T17:03:57.434 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:03:57.435 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2021-11-03T17:03:57.615 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-base 2021-11-03T17:03:57.918 INFO:teuthology.orchestra.run.smithi115.stdout:Package ceph-base-2:16.2.4-0.el8.x86_64 is already installed. 2021-11-03T17:03:58.099 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:06 ago on Wed 03 Nov 2021 05:02:52 PM UTC. 2021-11-03T17:03:58.199 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2021-11-03T17:03:58.200 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2021-11-03T17:03:58.200 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2021-11-03T17:03:58.255 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install cephadm 2021-11-03T17:03:58.704 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:05 ago on Wed 03 Nov 2021 05:02:53 PM UTC. 2021-11-03T17:03:58.878 INFO:teuthology.orchestra.run.smithi027.stdout:Package ceph-base-2:16.2.4-0.el8.x86_64 is already installed. 2021-11-03T17:03:59.173 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2021-11-03T17:03:59.174 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2021-11-03T17:03:59.175 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2021-11-03T17:03:59.561 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install cephadm 2021-11-03T17:03:59.708 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2021-11-03T17:03:59.709 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:03:59.709 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2021-11-03T17:03:59.709 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:03:59.710 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2021-11-03T17:03:59.710 INFO:teuthology.orchestra.run.smithi115.stdout: cephadm noarch 2:16.2.4-0.el8 ceph-noarch 71 k 2021-11-03T17:03:59.710 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:03:59.711 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2021-11-03T17:03:59.711 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:03:59.711 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2021-11-03T17:03:59.711 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:03:59.712 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 71 k 2021-11-03T17:03:59.712 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 290 k 2021-11-03T17:03:59.712 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2021-11-03T17:03:59.721 INFO:teuthology.orchestra.run.smithi115.stdout:cephadm-16.2.4-0.el8.noarch.rpm 5.9 MB/s | 71 kB 00:00 2021-11-03T17:03:59.721 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2021-11-03T17:03:59.722 INFO:teuthology.orchestra.run.smithi115.stdout:Total 5.3 MB/s | 71 kB 00:00 2021-11-03T17:03:59.722 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2021-11-03T17:03:59.730 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2021-11-03T17:03:59.730 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2021-11-03T17:03:59.731 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2021-11-03T17:03:59.735 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2021-11-03T17:03:59.780 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2021-11-03T17:04:00.044 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:08 ago on Wed 03 Nov 2021 05:02:52 PM UTC. 2021-11-03T17:04:01.160 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2021-11-03T17:04:01.161 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:04:01.161 INFO:teuthology.orchestra.run.smithi027.stdout: Package Architecture Version Repository Size 2021-11-03T17:04:01.162 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:04:01.162 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2021-11-03T17:04:01.162 INFO:teuthology.orchestra.run.smithi027.stdout: cephadm noarch 2:16.2.4-0.el8 ceph-noarch 71 k 2021-11-03T17:04:01.162 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:04:01.163 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2021-11-03T17:04:01.163 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:04:01.163 INFO:teuthology.orchestra.run.smithi027.stdout:Install 1 Package 2021-11-03T17:04:01.163 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:04:01.163 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 71 k 2021-11-03T17:04:01.164 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 290 k 2021-11-03T17:04:01.164 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2021-11-03T17:04:01.173 INFO:teuthology.orchestra.run.smithi027.stdout:cephadm-16.2.4-0.el8.noarch.rpm 6.1 MB/s | 71 kB 00:00 2021-11-03T17:04:01.174 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2021-11-03T17:04:01.175 INFO:teuthology.orchestra.run.smithi027.stdout:Total 5.5 MB/s | 71 kB 00:00 2021-11-03T17:04:01.175 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2021-11-03T17:04:01.182 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2021-11-03T17:04:01.183 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2021-11-03T17:04:01.184 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2021-11-03T17:04:01.188 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2021-11-03T17:04:01.231 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2021-11-03T17:04:02.110 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: cephadm-2:16.2.4-0.el8.noarch 1/1 2021-11-03T17:04:02.363 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : cephadm-2:16.2.4-0.el8.noarch 1/1 2021-11-03T17:04:02.543 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: cephadm-2:16.2.4-0.el8.noarch 1/1 2021-11-03T17:04:02.832 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : cephadm-2:16.2.4-0.el8.noarch 1/1 2021-11-03T17:04:02.832 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:02.833 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2021-11-03T17:04:02.833 INFO:teuthology.orchestra.run.smithi115.stdout: cephadm-2:16.2.4-0.el8.noarch 2021-11-03T17:04:02.833 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:02.833 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2021-11-03T17:04:02.967 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-immutable-object-cache 2021-11-03T17:04:03.437 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:10 ago on Wed 03 Nov 2021 05:02:53 PM UTC. 2021-11-03T17:04:03.551 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: cephadm-2:16.2.4-0.el8.noarch 1/1 2021-11-03T17:04:03.819 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : cephadm-2:16.2.4-0.el8.noarch 1/1 2021-11-03T17:04:04.006 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: cephadm-2:16.2.4-0.el8.noarch 1/1 2021-11-03T17:04:04.304 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : cephadm-2:16.2.4-0.el8.noarch 1/1 2021-11-03T17:04:04.304 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:04:04.304 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2021-11-03T17:04:04.305 INFO:teuthology.orchestra.run.smithi027.stdout: cephadm-2:16.2.4-0.el8.noarch 2021-11-03T17:04:04.305 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:04:04.305 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2021-11-03T17:04:04.427 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2021-11-03T17:04:04.428 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:04:04.428 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repo Size 2021-11-03T17:04:04.428 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:04:04.429 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2021-11-03T17:04:04.429 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-immutable-object-cache x86_64 2:16.2.4-0.el8 ceph 163 k 2021-11-03T17:04:04.429 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:04.429 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2021-11-03T17:04:04.429 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:04:04.430 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2021-11-03T17:04:04.430 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:04.430 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 163 k 2021-11-03T17:04:04.430 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 587 k 2021-11-03T17:04:04.431 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2021-11-03T17:04:04.443 INFO:teuthology.orchestra.run.smithi115.stdout:ceph-immutable-object-cache-16.2.4-0.el8.x86_64 11 MB/s | 163 kB 00:00 2021-11-03T17:04:04.443 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2021-11-03T17:04:04.444 INFO:teuthology.orchestra.run.smithi115.stdout:Total 10 MB/s | 163 kB 00:00 2021-11-03T17:04:04.444 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2021-11-03T17:04:04.457 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2021-11-03T17:04:04.457 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2021-11-03T17:04:04.473 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-immutable-object-cache 2021-11-03T17:04:04.504 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2021-11-03T17:04:04.507 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2021-11-03T17:04:04.627 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2021-11-03T17:04:04.866 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-immutable-object-cache-2:16.2.4-0.el8.x86_64 1/1 2021-11-03T17:04:04.966 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:12 ago on Wed 03 Nov 2021 05:02:52 PM UTC. 2021-11-03T17:04:05.231 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.2.4-0.el8.x86_64 1/1 2021-11-03T17:04:05.501 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-immutable-object-cache-2:16.2.4-0.el8.x86_64 1/1 2021-11-03T17:04:05.502 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:05.502 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2021-11-03T17:04:05.502 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-immutable-object-cache-2:16.2.4-0.el8.x86_64 2021-11-03T17:04:05.503 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:05.503 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2021-11-03T17:04:05.887 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr 2021-11-03T17:04:06.224 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2021-11-03T17:04:06.226 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:04:06.226 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repo Size 2021-11-03T17:04:06.227 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:04:06.227 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2021-11-03T17:04:06.228 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-immutable-object-cache x86_64 2:16.2.4-0.el8 ceph 163 k 2021-11-03T17:04:06.228 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:04:06.229 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2021-11-03T17:04:06.229 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:04:06.230 INFO:teuthology.orchestra.run.smithi027.stdout:Install 1 Package 2021-11-03T17:04:06.230 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:04:06.230 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 163 k 2021-11-03T17:04:06.231 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 587 k 2021-11-03T17:04:06.231 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2021-11-03T17:04:06.239 INFO:teuthology.orchestra.run.smithi027.stdout:ceph-immutable-object-cache-16.2.4-0.el8.x86_64 12 MB/s | 163 kB 00:00 2021-11-03T17:04:06.240 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2021-11-03T17:04:06.241 INFO:teuthology.orchestra.run.smithi027.stdout:Total 11 MB/s | 163 kB 00:00 2021-11-03T17:04:06.241 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2021-11-03T17:04:06.255 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2021-11-03T17:04:06.255 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2021-11-03T17:04:06.305 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2021-11-03T17:04:06.308 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2021-11-03T17:04:06.407 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:13 ago on Wed 03 Nov 2021 05:02:53 PM UTC. 2021-11-03T17:04:06.425 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2021-11-03T17:04:06.634 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-immutable-object-cache-2:16.2.4-0.el8.x86_64 1/1 2021-11-03T17:04:07.019 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.2.4-0.el8.x86_64 1/1 2021-11-03T17:04:07.321 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-immutable-object-cache-2:16.2.4-0.el8.x86_64 1/1 2021-11-03T17:04:07.322 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:04:07.322 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2021-11-03T17:04:07.323 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-immutable-object-cache-2:16.2.4-0.el8.x86_64 2021-11-03T17:04:07.323 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:04:07.323 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2021-11-03T17:04:07.480 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-mgr 2021-11-03T17:04:07.669 INFO:teuthology.orchestra.run.smithi115.stdout:Package ceph-mgr-2:16.2.4-0.el8.x86_64 is already installed. 2021-11-03T17:04:07.947 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2021-11-03T17:04:07.948 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2021-11-03T17:04:07.948 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2021-11-03T17:04:07.962 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:15 ago on Wed 03 Nov 2021 05:02:52 PM UTC. 2021-11-03T17:04:08.185 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr-dashboard 2021-11-03T17:04:08.649 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:15 ago on Wed 03 Nov 2021 05:02:53 PM UTC. 2021-11-03T17:04:08.735 INFO:teuthology.orchestra.run.smithi027.stdout:Package ceph-mgr-2:16.2.4-0.el8.x86_64 is already installed. 2021-11-03T17:04:09.023 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2021-11-03T17:04:09.024 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2021-11-03T17:04:09.025 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2021-11-03T17:04:09.078 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-mgr-dashboard 2021-11-03T17:04:09.556 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:17 ago on Wed 03 Nov 2021 05:02:52 PM UTC. 2021-11-03T17:04:09.652 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2021-11-03T17:04:09.653 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:04:09.653 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2021-11-03T17:04:09.654 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:04:09.654 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2021-11-03T17:04:09.654 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-dashboard noarch 2:16.2.4-0.el8 ceph-noarch 2.7 M 2021-11-03T17:04:09.654 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2021-11-03T17:04:09.655 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-grafana-dashboards noarch 2:16.2.4-0.el8 ceph-noarch 24 k 2021-11-03T17:04:09.655 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-prometheus-alerts noarch 2:16.2.4-0.el8 ceph-noarch 8.9 k 2021-11-03T17:04:09.655 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jwt noarch 1.6.1-2.el8 CentOS-Base 43 k 2021-11-03T17:04:09.655 INFO:teuthology.orchestra.run.smithi115.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2021-11-03T17:04:09.656 INFO:teuthology.orchestra.run.smithi115.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2021-11-03T17:04:09.656 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:09.656 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2021-11-03T17:04:09.656 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:04:09.656 INFO:teuthology.orchestra.run.smithi115.stdout:Install 6 Packages 2021-11-03T17:04:09.657 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:09.658 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 3.0 M 2021-11-03T17:04:09.658 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 15 M 2021-11-03T17:04:09.658 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2021-11-03T17:04:09.675 INFO:teuthology.orchestra.run.smithi115.stdout:(1/6): ceph-grafana-dashboards-16.2.4-0.el8.noa 1.2 MB/s | 24 kB 00:00 2021-11-03T17:04:09.682 INFO:teuthology.orchestra.run.smithi115.stdout:(2/6): ceph-prometheus-alerts-16.2.4-0.el8.noar 337 kB/s | 8.9 kB 00:00 2021-11-03T17:04:09.715 INFO:teuthology.orchestra.run.smithi115.stdout:(3/6): ceph-mgr-dashboard-16.2.4-0.el8.noarch.r 45 MB/s | 2.7 MB 00:00 2021-11-03T17:04:09.715 INFO:teuthology.orchestra.run.smithi115.stdout:(4/6): python3-jwt-1.6.1-2.el8.noarch.rpm 1.1 MB/s | 43 kB 00:00 2021-11-03T17:04:09.756 INFO:teuthology.orchestra.run.smithi115.stdout:(5/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 459 kB/s | 34 kB 00:00 2021-11-03T17:04:09.834 INFO:teuthology.orchestra.run.smithi115.stdout:(6/6): python3-routes-2.4.1-12.el8.noarch.rpm 1.6 MB/s | 196 kB 00:00 2021-11-03T17:04:09.837 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2021-11-03T17:04:09.838 INFO:teuthology.orchestra.run.smithi115.stdout:Total 16 MB/s | 3.0 MB 00:00 2021-11-03T17:04:09.840 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2021-11-03T17:04:09.849 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2021-11-03T17:04:09.850 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2021-11-03T17:04:09.917 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2021-11-03T17:04:09.920 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2021-11-03T17:04:10.175 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2021-11-03T17:04:10.381 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2021-11-03T17:04:10.575 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2021-11-03T17:04:10.626 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2021-11-03T17:04:10.627 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:04:10.627 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2021-11-03T17:04:10.628 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:04:10.628 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2021-11-03T17:04:10.628 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard noarch 2:16.2.4-0.el8 ceph-noarch 2.7 M 2021-11-03T17:04:10.629 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2021-11-03T17:04:10.629 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards noarch 2:16.2.4-0.el8 ceph-noarch 24 k 2021-11-03T17:04:10.630 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts noarch 2:16.2.4-0.el8 ceph-noarch 8.9 k 2021-11-03T17:04:10.630 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jwt noarch 1.6.1-2.el8 CentOS-Base 43 k 2021-11-03T17:04:10.630 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2021-11-03T17:04:10.631 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2021-11-03T17:04:10.631 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:04:10.631 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2021-11-03T17:04:10.632 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:04:10.632 INFO:teuthology.orchestra.run.smithi027.stdout:Install 6 Packages 2021-11-03T17:04:10.632 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:04:10.633 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 3.0 M 2021-11-03T17:04:10.633 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 15 M 2021-11-03T17:04:10.633 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2021-11-03T17:04:10.695 INFO:teuthology.orchestra.run.smithi027.stdout:(1/6): ceph-grafana-dashboards-16.2.4-0.el8.noa 376 kB/s | 24 kB 00:00 2021-11-03T17:04:10.695 INFO:teuthology.orchestra.run.smithi027.stdout:(2/6): ceph-prometheus-alerts-16.2.4-0.el8.noar 139 kB/s | 8.9 kB 00:00 2021-11-03T17:04:10.753 INFO:teuthology.orchestra.run.smithi027.stdout:(3/6): ceph-mgr-dashboard-16.2.4-0.el8.noarch.r 22 MB/s | 2.7 MB 00:00 2021-11-03T17:04:10.766 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2021-11-03T17:04:10.819 INFO:teuthology.orchestra.run.smithi027.stdout:(4/6): python3-jwt-1.6.1-2.el8.noarch.rpm 342 kB/s | 43 kB 00:00 2021-11-03T17:04:10.819 INFO:teuthology.orchestra.run.smithi027.stdout:(5/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 269 kB/s | 34 kB 00:00 2021-11-03T17:04:10.912 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-prometheus-alerts-2:16.2.4-0.el8.noarch 4/6 2021-11-03T17:04:10.981 INFO:teuthology.orchestra.run.smithi027.stdout:(6/6): python3-routes-2.4.1-12.el8.noarch.rpm 862 kB/s | 196 kB 00:00 2021-11-03T17:04:10.985 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2021-11-03T17:04:10.986 INFO:teuthology.orchestra.run.smithi027.stdout:Total 8.3 MB/s | 3.0 MB 00:00 2021-11-03T17:04:10.988 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2021-11-03T17:04:10.999 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2021-11-03T17:04:10.999 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2021-11-03T17:04:11.092 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2021-11-03T17:04:11.096 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2021-11-03T17:04:11.927 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-grafana-dashboards-2:16.2.4-0.el8.noarch 5/6 2021-11-03T17:04:11.984 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2021-11-03T17:04:12.189 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-dashboard-2:16.2.4-0.el8.noarch 6/6 2021-11-03T17:04:12.212 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 1/6 2021-11-03T17:04:12.422 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.2.4-0.el8.noarch 6/6 2021-11-03T17:04:12.422 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-grafana-dashboards-2:16.2.4-0.el8.noarch 1/6 2021-11-03T17:04:12.422 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-dashboard-2:16.2.4-0.el8.noarch 2/6 2021-11-03T17:04:12.423 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-prometheus-alerts-2:16.2.4-0.el8.noarch 3/6 2021-11-03T17:04:12.423 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2021-11-03T17:04:12.423 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2021-11-03T17:04:12.441 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 2/6 2021-11-03T17:04:12.627 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 3/6 2021-11-03T17:04:12.708 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2021-11-03T17:04:12.708 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:12.709 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2021-11-03T17:04:12.709 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-grafana-dashboards-2:16.2.4-0.el8.noarch 2021-11-03T17:04:12.709 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-dashboard-2:16.2.4-0.el8.noarch 2021-11-03T17:04:12.709 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-prometheus-alerts-2:16.2.4-0.el8.noarch 2021-11-03T17:04:12.709 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jwt-1.6.1-2.el8.noarch 2021-11-03T17:04:12.710 INFO:teuthology.orchestra.run.smithi115.stdout: python3-repoze-lru-0.7-6.el8.noarch 2021-11-03T17:04:12.710 INFO:teuthology.orchestra.run.smithi115.stdout: python3-routes-2.4.1-12.el8.noarch 2021-11-03T17:04:12.710 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:12.710 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2021-11-03T17:04:12.773 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr-diskprediction-local 2021-11-03T17:04:12.783 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-prometheus-alerts-2:16.2.4-0.el8.noarch 4/6 2021-11-03T17:04:13.224 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:20 ago on Wed 03 Nov 2021 05:02:53 PM UTC. 2021-11-03T17:04:13.889 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-grafana-dashboards-2:16.2.4-0.el8.noarch 5/6 2021-11-03T17:04:14.154 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-dashboard-2:16.2.4-0.el8.noarch 6/6 2021-11-03T17:04:14.232 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2021-11-03T17:04:14.233 INFO:teuthology.orchestra.run.smithi115.stdout:===================================================================================================== 2021-11-03T17:04:14.234 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2021-11-03T17:04:14.234 INFO:teuthology.orchestra.run.smithi115.stdout:===================================================================================================== 2021-11-03T17:04:14.234 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2021-11-03T17:04:14.235 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-diskprediction-local noarch 2:16.2.4-0.el8 ceph-noarch 8.6 M 2021-11-03T17:04:14.235 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2021-11-03T17:04:14.235 INFO:teuthology.orchestra.run.smithi115.stdout: libgfortran x86_64 8.4.1-1.el8 CentOS-Base 639 k 2021-11-03T17:04:14.235 INFO:teuthology.orchestra.run.smithi115.stdout: libquadmath x86_64 8.4.1-1.el8 CentOS-Base 169 k 2021-11-03T17:04:14.236 INFO:teuthology.orchestra.run.smithi115.stdout: openblas x86_64 0.3.12-1.el8 CentOS-AppStream 4.6 M 2021-11-03T17:04:14.236 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-threads x86_64 0.3.12-1.el8 CentOS-AppStream 4.7 M 2021-11-03T17:04:14.236 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy x86_64 1:1.14.3-9.el8 CentOS-AppStream 3.7 M 2021-11-03T17:04:14.236 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 CentOS-AppStream 225 k 2021-11-03T17:04:14.236 INFO:teuthology.orchestra.run.smithi115.stdout: python3-scipy x86_64 1.0.0-20.module_el8.4.0+790+083e3d81 CentOS-AppStream 15 M 2021-11-03T17:04:14.237 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:14.237 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2021-11-03T17:04:14.237 INFO:teuthology.orchestra.run.smithi115.stdout:===================================================================================================== 2021-11-03T17:04:14.237 INFO:teuthology.orchestra.run.smithi115.stdout:Install 8 Packages 2021-11-03T17:04:14.238 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:14.238 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 37 M 2021-11-03T17:04:14.239 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 197 M 2021-11-03T17:04:14.239 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2021-11-03T17:04:14.337 INFO:teuthology.orchestra.run.smithi115.stdout:(1/8): openblas-0.3.12-1.el8.x86_64.rpm 46 MB/s | 4.6 MB 00:00 2021-11-03T17:04:14.393 INFO:teuthology.orchestra.run.smithi115.stdout:(2/8): python3-numpy-1.14.3-9.el8.x86_64.rpm 67 MB/s | 3.7 MB 00:00 2021-11-03T17:04:14.398 INFO:teuthology.orchestra.run.smithi115.stdout:(3/8): python3-numpy-f2py-1.14.3-9.el8.x86_64.r 43 MB/s | 225 kB 00:00 2021-11-03T17:04:14.401 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.2.4-0.el8.noarch 6/6 2021-11-03T17:04:14.401 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-grafana-dashboards-2:16.2.4-0.el8.noarch 1/6 2021-11-03T17:04:14.402 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-dashboard-2:16.2.4-0.el8.noarch 2/6 2021-11-03T17:04:14.402 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-prometheus-alerts-2:16.2.4-0.el8.noarch 3/6 2021-11-03T17:04:14.402 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 4/6 2021-11-03T17:04:14.402 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 5/6 2021-11-03T17:04:14.443 INFO:teuthology.orchestra.run.smithi115.stdout:(4/8): ceph-mgr-diskprediction-local-16.2.4-0.e 41 MB/s | 8.6 MB 00:00 2021-11-03T17:04:14.519 INFO:teuthology.orchestra.run.smithi115.stdout:(5/8): openblas-threads-0.3.12-1.el8.x86_64.rpm 17 MB/s | 4.7 MB 00:00 2021-11-03T17:04:14.535 INFO:teuthology.orchestra.run.smithi115.stdout:(6/8): libquadmath-8.4.1-1.el8.x86_64.rpm 10 MB/s | 169 kB 00:00 2021-11-03T17:04:14.566 INFO:teuthology.orchestra.run.smithi115.stdout:(7/8): libgfortran-8.4.1-1.el8.x86_64.rpm 5.1 MB/s | 639 kB 00:00 2021-11-03T17:04:14.703 INFO:teuthology.orchestra.run.smithi115.stdout:(8/8): python3-scipy-1.0.0-20.module_el8.4.0+79 48 MB/s | 15 MB 00:00 2021-11-03T17:04:14.707 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2021-11-03T17:04:14.707 INFO:teuthology.orchestra.run.smithi115.stdout:Total 79 MB/s | 37 MB 00:00 2021-11-03T17:04:14.712 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 6/6 2021-11-03T17:04:14.712 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:04:14.713 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2021-11-03T17:04:14.713 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-grafana-dashboards-2:16.2.4-0.el8.noarch 2021-11-03T17:04:14.713 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-dashboard-2:16.2.4-0.el8.noarch 2021-11-03T17:04:14.713 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-prometheus-alerts-2:16.2.4-0.el8.noarch 2021-11-03T17:04:14.714 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jwt-1.6.1-2.el8.noarch 2021-11-03T17:04:14.714 INFO:teuthology.orchestra.run.smithi027.stdout: python3-repoze-lru-0.7-6.el8.noarch 2021-11-03T17:04:14.714 INFO:teuthology.orchestra.run.smithi027.stdout: python3-routes-2.4.1-12.el8.noarch 2021-11-03T17:04:14.714 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:04:14.714 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2021-11-03T17:04:14.721 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2021-11-03T17:04:14.741 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2021-11-03T17:04:14.741 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2021-11-03T17:04:14.790 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-mgr-diskprediction-local 2021-11-03T17:04:15.043 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2021-11-03T17:04:15.046 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2021-11-03T17:04:15.270 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:23 ago on Wed 03 Nov 2021 05:02:52 PM UTC. 2021-11-03T17:04:15.593 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2021-11-03T17:04:15.738 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 1/8 2021-11-03T17:04:15.876 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 1/8 2021-11-03T17:04:16.088 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 2/8 2021-11-03T17:04:16.365 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2021-11-03T17:04:16.366 INFO:teuthology.orchestra.run.smithi027.stdout:===================================================================================================== 2021-11-03T17:04:16.366 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2021-11-03T17:04:16.367 INFO:teuthology.orchestra.run.smithi027.stdout:===================================================================================================== 2021-11-03T17:04:16.367 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2021-11-03T17:04:16.368 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local noarch 2:16.2.4-0.el8 ceph-noarch 8.6 M 2021-11-03T17:04:16.368 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2021-11-03T17:04:16.369 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran x86_64 8.4.1-1.el8 CentOS-Base 639 k 2021-11-03T17:04:16.369 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath x86_64 8.4.1-1.el8 CentOS-Base 169 k 2021-11-03T17:04:16.369 INFO:teuthology.orchestra.run.smithi027.stdout: openblas x86_64 0.3.12-1.el8 CentOS-AppStream 4.6 M 2021-11-03T17:04:16.370 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-threads x86_64 0.3.12-1.el8 CentOS-AppStream 4.7 M 2021-11-03T17:04:16.370 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy x86_64 1:1.14.3-9.el8 CentOS-AppStream 3.7 M 2021-11-03T17:04:16.371 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 CentOS-AppStream 225 k 2021-11-03T17:04:16.371 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy x86_64 1.0.0-20.module_el8.4.0+790+083e3d81 CentOS-AppStream 15 M 2021-11-03T17:04:16.372 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:04:16.372 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2021-11-03T17:04:16.373 INFO:teuthology.orchestra.run.smithi027.stdout:===================================================================================================== 2021-11-03T17:04:16.373 INFO:teuthology.orchestra.run.smithi027.stdout:Install 8 Packages 2021-11-03T17:04:16.374 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:04:16.376 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 37 M 2021-11-03T17:04:16.376 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 197 M 2021-11-03T17:04:16.376 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2021-11-03T17:04:16.484 INFO:teuthology.orchestra.run.smithi027.stdout:(1/8): openblas-0.3.12-1.el8.x86_64.rpm 40 MB/s | 4.6 MB 00:00 2021-11-03T17:04:16.513 INFO:teuthology.orchestra.run.smithi027.stdout:(2/8): ceph-mgr-diskprediction-local-16.2.4-0.e 59 MB/s | 8.6 MB 00:00 2021-11-03T17:04:16.583 INFO:teuthology.orchestra.run.smithi027.stdout:(3/8): python3-numpy-1.14.3-9.el8.x86_64.rpm 38 MB/s | 3.7 MB 00:00 2021-11-03T17:04:16.584 INFO:teuthology.orchestra.run.smithi027.stdout:(4/8): python3-numpy-f2py-1.14.3-9.el8.x86_64.r 3.1 MB/s | 225 kB 00:00 2021-11-03T17:04:16.602 INFO:teuthology.orchestra.run.smithi027.stdout:(5/8): openblas-threads-0.3.12-1.el8.x86_64.rpm 20 MB/s | 4.7 MB 00:00 2021-11-03T17:04:16.612 INFO:teuthology.orchestra.run.smithi027.stdout:(6/8): libquadmath-8.4.1-1.el8.x86_64.rpm 17 MB/s | 169 kB 00:00 2021-11-03T17:04:16.612 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 2/8 2021-11-03T17:04:16.641 INFO:teuthology.orchestra.run.smithi027.stdout:(7/8): libgfortran-8.4.1-1.el8.x86_64.rpm 11 MB/s | 639 kB 00:00 2021-11-03T17:04:16.788 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openblas-0.3.12-1.el8.x86_64 3/8 2021-11-03T17:04:16.885 INFO:teuthology.orchestra.run.smithi027.stdout:(8/8): python3-scipy-1.0.0-20.module_el8.4.0+79 48 MB/s | 15 MB 00:00 2021-11-03T17:04:16.889 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2021-11-03T17:04:16.890 INFO:teuthology.orchestra.run.smithi027.stdout:Total 71 MB/s | 37 MB 00:00 2021-11-03T17:04:16.905 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2021-11-03T17:04:16.927 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2021-11-03T17:04:16.927 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2021-11-03T17:04:17.247 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2021-11-03T17:04:17.250 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2021-11-03T17:04:17.331 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 3/8 2021-11-03T17:04:17.496 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 4/8 2021-11-03T17:04:17.811 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2021-11-03T17:04:17.982 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 1/8 2021-11-03T17:04:18.124 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 1/8 2021-11-03T17:04:18.210 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 4/8 2021-11-03T17:04:18.307 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 2/8 2021-11-03T17:04:18.797 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 5/8 2021-11-03T17:04:18.862 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 2/8 2021-11-03T17:04:19.024 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-0.3.12-1.el8.x86_64 3/8 2021-11-03T17:04:19.071 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2021-11-03T17:04:19.607 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 3/8 2021-11-03T17:04:19.799 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 4/8 2021-11-03T17:04:20.542 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 4/8 2021-11-03T17:04:20.913 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 5/8 2021-11-03T17:04:21.041 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2021-11-03T17:04:21.174 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2021-11-03T17:04:22.338 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-scipy-1.0.0-20.module_el8.4.0+790+083e3d81.x 7/8 2021-11-03T17:04:22.495 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-diskprediction-local-2:16.2.4-0.el8.noarch 8/8 2021-11-03T17:04:22.770 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.2.4-0.el8.noarch 8/8 2021-11-03T17:04:22.771 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.2.4-0.el8.noarch 1/8 2021-11-03T17:04:22.771 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 2/8 2021-11-03T17:04:22.771 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 3/8 2021-11-03T17:04:22.771 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 4/8 2021-11-03T17:04:22.772 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 5/8 2021-11-03T17:04:22.772 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-scipy-1.0.0-20.module_el8.4.0+790+083e3d81.x 6/8 2021-11-03T17:04:22.772 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 7/8 2021-11-03T17:04:23.081 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 8/8 2021-11-03T17:04:23.081 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:23.082 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2021-11-03T17:04:23.082 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-diskprediction-local-2:16.2.4-0.el8.noarch 2021-11-03T17:04:23.082 INFO:teuthology.orchestra.run.smithi115.stdout: libgfortran-8.4.1-1.el8.x86_64 2021-11-03T17:04:23.082 INFO:teuthology.orchestra.run.smithi115.stdout: libquadmath-8.4.1-1.el8.x86_64 2021-11-03T17:04:23.083 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-0.3.12-1.el8.x86_64 2021-11-03T17:04:23.083 INFO:teuthology.orchestra.run.smithi115.stdout: openblas-threads-0.3.12-1.el8.x86_64 2021-11-03T17:04:23.083 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2021-11-03T17:04:23.083 INFO:teuthology.orchestra.run.smithi115.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2021-11-03T17:04:23.084 INFO:teuthology.orchestra.run.smithi115.stdout: python3-scipy-1.0.0-20.module_el8.4.0+790+083e3d81.x86_64 2021-11-03T17:04:23.084 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:23.084 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2021-11-03T17:04:23.158 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr-rook 2021-11-03T17:04:23.279 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 6/8 2021-11-03T17:04:23.607 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:30 ago on Wed 03 Nov 2021 05:02:53 PM UTC. 2021-11-03T17:04:24.659 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-scipy-1.0.0-20.module_el8.4.0+790+083e3d81.x 7/8 2021-11-03T17:04:24.823 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-diskprediction-local-2:16.2.4-0.el8.noarch 8/8 2021-11-03T17:04:25.113 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.2.4-0.el8.noarch 8/8 2021-11-03T17:04:25.113 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.2.4-0.el8.noarch 1/8 2021-11-03T17:04:25.113 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 2/8 2021-11-03T17:04:25.114 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 3/8 2021-11-03T17:04:25.114 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 4/8 2021-11-03T17:04:25.114 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 5/8 2021-11-03T17:04:25.114 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-scipy-1.0.0-20.module_el8.4.0+790+083e3d81.x 6/8 2021-11-03T17:04:25.115 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 7/8 2021-11-03T17:04:25.274 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2021-11-03T17:04:25.275 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:04:25.276 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2021-11-03T17:04:25.276 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:04:25.276 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2021-11-03T17:04:25.276 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-rook noarch 2:16.2.4-0.el8 ceph-noarch 27 k 2021-11-03T17:04:25.277 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2021-11-03T17:04:25.277 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2021-11-03T17:04:25.277 INFO:teuthology.orchestra.run.smithi115.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2021-11-03T17:04:25.277 INFO:teuthology.orchestra.run.smithi115.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2021-11-03T17:04:25.278 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpatch noarch 1.21-2.el8 CentOS-AppStream 27 k 2021-11-03T17:04:25.278 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpointer noarch 1.10-11.el8 CentOS-AppStream 20 k 2021-11-03T17:04:25.278 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2021-11-03T17:04:25.279 INFO:teuthology.orchestra.run.smithi115.stdout: python3-oauthlib noarch 2.1.0-1.el8 CentOS-Base 155 k 2021-11-03T17:04:25.279 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1 noarch 0.3.7-6.el8 CentOS-AppStream 126 k 2021-11-03T17:04:25.279 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 CentOS-AppStream 110 k 2021-11-03T17:04:25.280 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 CentOS-Base 43 k 2021-11-03T17:04:25.280 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rsa noarch 4.7.2-1.el8 epel 61 k 2021-11-03T17:04:25.280 INFO:teuthology.orchestra.run.smithi115.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2021-11-03T17:04:25.280 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:25.280 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2021-11-03T17:04:25.281 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:04:25.281 INFO:teuthology.orchestra.run.smithi115.stdout:Install 13 Packages 2021-11-03T17:04:25.281 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:25.282 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 1.9 M 2021-11-03T17:04:25.282 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 24 M 2021-11-03T17:04:25.282 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2021-11-03T17:04:25.299 INFO:teuthology.orchestra.run.smithi115.stdout:(1/13): ceph-mgr-rook-16.2.4-0.el8.noarch.rpm 1.3 MB/s | 27 kB 00:00 2021-11-03T17:04:25.323 INFO:teuthology.orchestra.run.smithi115.stdout:(2/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 612 kB/s | 27 kB 00:00 2021-11-03T17:04:25.324 INFO:teuthology.orchestra.run.smithi115.stdout:(3/13): python3-pyasn1-modules-0.3.7-6.el8.noar 48 MB/s | 110 kB 00:00 2021-11-03T17:04:25.328 INFO:teuthology.orchestra.run.smithi115.stdout:(4/13): python3-jsonpointer-1.10-11.el8.noarch. 402 kB/s | 20 kB 00:00 2021-11-03T17:04:25.329 INFO:teuthology.orchestra.run.smithi115.stdout:(5/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 46 MB/s | 155 kB 00:00 2021-11-03T17:04:25.331 INFO:teuthology.orchestra.run.smithi115.stdout:(6/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 4.0 MB/s | 126 kB 00:00 2021-11-03T17:04:25.331 INFO:teuthology.orchestra.run.smithi115.stdout:(7/13): python3-requests-oauthlib-1.0.0-1.el8.n 22 MB/s | 43 kB 00:00 2021-11-03T17:04:25.431 INFO:teuthology.orchestra.run.smithi115.stdout:(8/13): python3-cachetools-3.1.1-4.el8.noarch.r 322 kB/s | 33 kB 00:00 2021-11-03T17:04:25.434 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 8/8 2021-11-03T17:04:25.435 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:04:25.435 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2021-11-03T17:04:25.435 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-diskprediction-local-2:16.2.4-0.el8.noarch 2021-11-03T17:04:25.435 INFO:teuthology.orchestra.run.smithi027.stdout: libgfortran-8.4.1-1.el8.x86_64 2021-11-03T17:04:25.436 INFO:teuthology.orchestra.run.smithi027.stdout: libquadmath-8.4.1-1.el8.x86_64 2021-11-03T17:04:25.436 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-0.3.12-1.el8.x86_64 2021-11-03T17:04:25.436 INFO:teuthology.orchestra.run.smithi027.stdout: openblas-threads-0.3.12-1.el8.x86_64 2021-11-03T17:04:25.436 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2021-11-03T17:04:25.437 INFO:teuthology.orchestra.run.smithi027.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2021-11-03T17:04:25.437 INFO:teuthology.orchestra.run.smithi027.stdout: python3-scipy-1.0.0-20.module_el8.4.0+790+083e3d81.x86_64 2021-11-03T17:04:25.437 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:04:25.437 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2021-11-03T17:04:25.438 INFO:teuthology.orchestra.run.smithi115.stdout:(9/13): python3-certifi-2018.10.15-7.el8.noarch 152 kB/s | 16 kB 00:00 2021-11-03T17:04:25.479 INFO:teuthology.orchestra.run.smithi115.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 548 kB/s | 81 kB 00:00 2021-11-03T17:04:25.507 INFO:teuthology.orchestra.run.smithi115.stdout:(11/13): python3-rsa-4.7.2-1.el8.noarch.rpm 887 kB/s | 61 kB 00:00 2021-11-03T17:04:25.513 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-mgr-rook 2021-11-03T17:04:25.518 INFO:teuthology.orchestra.run.smithi115.stdout:(12/13): python3-websocket-client-0.56.0-5.el8. 1.5 MB/s | 61 kB 00:00 2021-11-03T17:04:25.589 INFO:teuthology.orchestra.run.smithi115.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 7.4 MB/s | 1.2 MB 00:00 2021-11-03T17:04:25.592 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2021-11-03T17:04:25.592 INFO:teuthology.orchestra.run.smithi115.stdout:Total 6.1 MB/s | 1.9 MB 00:00 2021-11-03T17:04:25.602 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2021-11-03T17:04:25.617 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2021-11-03T17:04:25.618 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2021-11-03T17:04:25.689 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2021-11-03T17:04:25.692 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2021-11-03T17:04:25.991 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:33 ago on Wed 03 Nov 2021 05:02:52 PM UTC. 2021-11-03T17:04:26.190 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2021-11-03T17:04:26.425 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2021-11-03T17:04:26.614 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2021-11-03T17:04:26.814 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-rsa-4.7.2-1.el8.noarch 3/13 2021-11-03T17:04:26.993 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2021-11-03T17:04:27.063 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2021-11-03T17:04:27.065 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:04:27.065 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2021-11-03T17:04:27.065 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:04:27.065 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2021-11-03T17:04:27.066 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook noarch 2:16.2.4-0.el8 ceph-noarch 27 k 2021-11-03T17:04:27.066 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2021-11-03T17:04:27.066 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2021-11-03T17:04:27.066 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2021-11-03T17:04:27.067 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2021-11-03T17:04:27.067 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch noarch 1.21-2.el8 CentOS-AppStream 27 k 2021-11-03T17:04:27.067 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer noarch 1.10-11.el8 CentOS-AppStream 20 k 2021-11-03T17:04:27.067 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2021-11-03T17:04:27.068 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib noarch 2.1.0-1.el8 CentOS-Base 155 k 2021-11-03T17:04:27.068 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1 noarch 0.3.7-6.el8 CentOS-AppStream 126 k 2021-11-03T17:04:27.068 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 CentOS-AppStream 110 k 2021-11-03T17:04:27.068 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 CentOS-Base 43 k 2021-11-03T17:04:27.068 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa noarch 4.7.2-1.el8 epel 61 k 2021-11-03T17:04:27.069 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2021-11-03T17:04:27.069 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:04:27.069 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2021-11-03T17:04:27.069 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:04:27.070 INFO:teuthology.orchestra.run.smithi027.stdout:Install 13 Packages 2021-11-03T17:04:27.070 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:04:27.070 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 1.9 M 2021-11-03T17:04:27.071 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 24 M 2021-11-03T17:04:27.071 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2021-11-03T17:04:27.088 INFO:teuthology.orchestra.run.smithi027.stdout:(1/13): ceph-mgr-rook-16.2.4-0.el8.noarch.rpm 1.3 MB/s | 27 kB 00:00 2021-11-03T17:04:27.103 INFO:teuthology.orchestra.run.smithi027.stdout:(2/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 755 kB/s | 27 kB 00:00 2021-11-03T17:04:27.107 INFO:teuthology.orchestra.run.smithi027.stdout:(3/13): python3-pyasn1-modules-0.3.7-6.el8.noar 46 MB/s | 110 kB 00:00 2021-11-03T17:04:27.109 INFO:teuthology.orchestra.run.smithi027.stdout:(4/13): python3-oauthlib-2.1.0-1.el8.noarch.rpm 57 MB/s | 155 kB 00:00 2021-11-03T17:04:27.111 INFO:teuthology.orchestra.run.smithi027.stdout:(5/13): python3-requests-oauthlib-1.0.0-1.el8.n 32 MB/s | 43 kB 00:00 2021-11-03T17:04:27.111 INFO:teuthology.orchestra.run.smithi027.stdout:(6/13): python3-jsonpointer-1.10-11.el8.noarch. 462 kB/s | 20 kB 00:00 2021-11-03T17:04:27.118 INFO:teuthology.orchestra.run.smithi027.stdout:(7/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 4.1 MB/s | 126 kB 00:00 2021-11-03T17:04:27.151 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2021-11-03T17:04:27.185 INFO:teuthology.orchestra.run.smithi027.stdout:(8/13): python3-certifi-2018.10.15-7.el8.noarch 223 kB/s | 16 kB 00:00 2021-11-03T17:04:27.185 INFO:teuthology.orchestra.run.smithi027.stdout:(9/13): python3-cachetools-3.1.1-4.el8.noarch.r 444 kB/s | 33 kB 00:00 2021-11-03T17:04:27.215 INFO:teuthology.orchestra.run.smithi027.stdout:(10/13): python3-google-auth-1.1.1-10.el8.noarc 837 kB/s | 81 kB 00:00 2021-11-03T17:04:27.234 INFO:teuthology.orchestra.run.smithi027.stdout:(11/13): python3-rsa-4.7.2-1.el8.noarch.rpm 1.2 MB/s | 61 kB 00:00 2021-11-03T17:04:27.240 INFO:teuthology.orchestra.run.smithi027.stdout:(12/13): python3-websocket-client-0.56.0-5.el8. 2.4 MB/s | 61 kB 00:00 2021-11-03T17:04:27.336 INFO:teuthology.orchestra.run.smithi027.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 7.7 MB/s | 1.2 MB 00:00 2021-11-03T17:04:27.338 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2021-11-03T17:04:27.339 INFO:teuthology.orchestra.run.smithi027.stdout:Total 7.0 MB/s | 1.9 MB 00:00 2021-11-03T17:04:27.350 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2021-11-03T17:04:27.366 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2021-11-03T17:04:27.367 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2021-11-03T17:04:27.372 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2021-11-03T17:04:27.447 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2021-11-03T17:04:27.450 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2021-11-03T17:04:27.649 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2021-11-03T17:04:27.849 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2021-11-03T17:04:28.665 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2021-11-03T17:04:28.915 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2021-11-03T17:04:28.932 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2021-11-03T17:04:29.121 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 2/13 2021-11-03T17:04:29.167 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2021-11-03T17:04:29.317 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2021-11-03T17:04:29.388 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-rsa-4.7.2-1.el8.noarch 3/13 2021-11-03T17:04:29.467 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2021-11-03T17:04:29.573 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 4/13 2021-11-03T17:04:29.635 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-rook-2:16.2.4-0.el8.noarch 13/13 2021-11-03T17:04:29.751 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 5/13 2021-11-03T17:04:29.883 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-rook-2:16.2.4-0.el8.noarch 13/13 2021-11-03T17:04:29.883 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-rook-2:16.2.4-0.el8.noarch 1/13 2021-11-03T17:04:29.884 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2021-11-03T17:04:29.884 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2021-11-03T17:04:29.884 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2021-11-03T17:04:29.885 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2021-11-03T17:04:29.885 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2021-11-03T17:04:29.886 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2021-11-03T17:04:29.886 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2021-11-03T17:04:29.887 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2021-11-03T17:04:29.887 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2021-11-03T17:04:29.887 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2021-11-03T17:04:29.888 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-rsa-4.7.2-1.el8.noarch 12/13 2021-11-03T17:04:29.990 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 6/13 2021-11-03T17:04:30.254 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2021-11-03T17:04:30.255 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:30.255 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2021-11-03T17:04:30.256 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-rook-2:16.2.4-0.el8.noarch 2021-11-03T17:04:30.256 INFO:teuthology.orchestra.run.smithi115.stdout: python3-cachetools-3.1.1-4.el8.noarch 2021-11-03T17:04:30.256 INFO:teuthology.orchestra.run.smithi115.stdout: python3-certifi-2018.10.15-7.el8.noarch 2021-11-03T17:04:30.257 INFO:teuthology.orchestra.run.smithi115.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2021-11-03T17:04:30.257 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpatch-1.21-2.el8.noarch 2021-11-03T17:04:30.257 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jsonpointer-1.10-11.el8.noarch 2021-11-03T17:04:30.257 INFO:teuthology.orchestra.run.smithi115.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2021-11-03T17:04:30.258 INFO:teuthology.orchestra.run.smithi115.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2021-11-03T17:04:30.258 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2021-11-03T17:04:30.259 INFO:teuthology.orchestra.run.smithi115.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2021-11-03T17:04:30.259 INFO:teuthology.orchestra.run.smithi115.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2021-11-03T17:04:30.259 INFO:teuthology.orchestra.run.smithi115.stdout: python3-rsa-4.7.2-1.el8.noarch 2021-11-03T17:04:30.259 INFO:teuthology.orchestra.run.smithi115.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2021-11-03T17:04:30.260 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:30.260 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2021-11-03T17:04:30.268 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 7/13 2021-11-03T17:04:30.319 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-mgr-cephadm 2021-11-03T17:04:30.509 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 8/13 2021-11-03T17:04:30.774 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:37 ago on Wed 03 Nov 2021 05:02:53 PM UTC. 2021-11-03T17:04:31.665 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 9/13 2021-11-03T17:04:31.780 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2021-11-03T17:04:31.780 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:04:31.780 INFO:teuthology.orchestra.run.smithi115.stdout: Package Arch Version Repository Size 2021-11-03T17:04:31.781 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:04:31.781 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2021-11-03T17:04:31.781 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-cephadm noarch 2:16.2.4-0.el8 ceph-noarch 89 k 2021-11-03T17:04:31.781 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2021-11-03T17:04:31.782 INFO:teuthology.orchestra.run.smithi115.stdout: python3-apipkg noarch 1.5-6.el8 epel 21 k 2021-11-03T17:04:31.782 INFO:teuthology.orchestra.run.smithi115.stdout: python3-babel noarch 2.5.1-5.el8 CentOS-AppStream 4.8 M 2021-11-03T17:04:31.782 INFO:teuthology.orchestra.run.smithi115.stdout: python3-execnet noarch 1.7.1-1.el8 epel 355 k 2021-11-03T17:04:31.783 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 CentOS-AppStream 538 k 2021-11-03T17:04:31.783 INFO:teuthology.orchestra.run.smithi115.stdout: python3-remoto noarch 1.2.1-1.el8 epel 49 k 2021-11-03T17:04:31.783 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:31.783 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2021-11-03T17:04:31.783 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:04:31.784 INFO:teuthology.orchestra.run.smithi115.stdout:Install 6 Packages 2021-11-03T17:04:31.784 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:31.785 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 5.8 M 2021-11-03T17:04:31.785 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 24 M 2021-11-03T17:04:31.785 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2021-11-03T17:04:31.801 INFO:teuthology.orchestra.run.smithi115.stdout:(1/6): ceph-mgr-cephadm-16.2.4-0.el8.noarch.rpm 4.6 MB/s | 89 kB 00:00 2021-11-03T17:04:31.839 INFO:teuthology.orchestra.run.smithi115.stdout:(2/6): python3-jinja2-2.10.1-2.el8_0.noarch.rpm 9.3 MB/s | 538 kB 00:00 2021-11-03T17:04:31.880 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 10/13 2021-11-03T17:04:31.881 INFO:teuthology.orchestra.run.smithi115.stdout:(3/6): python3-babel-2.5.1-5.el8.noarch.rpm 48 MB/s | 4.8 MB 00:00 2021-11-03T17:04:31.884 INFO:teuthology.orchestra.run.smithi115.stdout:(4/6): python3-apipkg-1.5-6.el8.noarch.rpm 257 kB/s | 21 kB 00:00 2021-11-03T17:04:32.006 INFO:teuthology.orchestra.run.smithi115.stdout:(5/6): python3-execnet-1.7.1-1.el8.noarch.rpm 2.1 MB/s | 355 kB 00:00 2021-11-03T17:04:32.064 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 11/13 2021-11-03T17:04:32.102 INFO:teuthology.orchestra.run.smithi115.stdout:(6/6): python3-remoto-1.2.1-1.el8.noarch.rpm 222 kB/s | 49 kB 00:00 2021-11-03T17:04:32.104 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2021-11-03T17:04:32.105 INFO:teuthology.orchestra.run.smithi115.stdout:Total 18 MB/s | 5.8 MB 00:00 2021-11-03T17:04:32.109 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2021-11-03T17:04:32.118 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2021-11-03T17:04:32.119 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2021-11-03T17:04:32.201 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2021-11-03T17:04:32.204 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2021-11-03T17:04:32.240 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 12/13 2021-11-03T17:04:32.406 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-rook-2:16.2.4-0.el8.noarch 13/13 2021-11-03T17:04:32.501 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2021-11-03T17:04:32.684 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-rook-2:16.2.4-0.el8.noarch 13/13 2021-11-03T17:04:32.684 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-rook-2:16.2.4-0.el8.noarch 1/13 2021-11-03T17:04:32.684 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 2/13 2021-11-03T17:04:32.684 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 3/13 2021-11-03T17:04:32.685 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 4/13 2021-11-03T17:04:32.685 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 5/13 2021-11-03T17:04:32.685 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 6/13 2021-11-03T17:04:32.685 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 7/13 2021-11-03T17:04:32.686 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 8/13 2021-11-03T17:04:32.686 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 9/13 2021-11-03T17:04:32.686 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 10/13 2021-11-03T17:04:32.686 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 11/13 2021-11-03T17:04:32.687 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-rsa-4.7.2-1.el8.noarch 12/13 2021-11-03T17:04:32.731 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 1/6 2021-11-03T17:04:32.931 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 2/6 2021-11-03T17:04:33.060 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 13/13 2021-11-03T17:04:33.061 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:04:33.061 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2021-11-03T17:04:33.061 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-rook-2:16.2.4-0.el8.noarch 2021-11-03T17:04:33.061 INFO:teuthology.orchestra.run.smithi027.stdout: python3-cachetools-3.1.1-4.el8.noarch 2021-11-03T17:04:33.062 INFO:teuthology.orchestra.run.smithi027.stdout: python3-certifi-2018.10.15-7.el8.noarch 2021-11-03T17:04:33.062 INFO:teuthology.orchestra.run.smithi027.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2021-11-03T17:04:33.062 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpatch-1.21-2.el8.noarch 2021-11-03T17:04:33.062 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jsonpointer-1.10-11.el8.noarch 2021-11-03T17:04:33.062 INFO:teuthology.orchestra.run.smithi027.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2021-11-03T17:04:33.063 INFO:teuthology.orchestra.run.smithi027.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2021-11-03T17:04:33.063 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2021-11-03T17:04:33.063 INFO:teuthology.orchestra.run.smithi027.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2021-11-03T17:04:33.063 INFO:teuthology.orchestra.run.smithi027.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2021-11-03T17:04:33.064 INFO:teuthology.orchestra.run.smithi027.stdout: python3-rsa-4.7.2-1.el8.noarch 2021-11-03T17:04:33.064 INFO:teuthology.orchestra.run.smithi027.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2021-11-03T17:04:33.064 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:04:33.064 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2021-11-03T17:04:33.131 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-mgr-cephadm 2021-11-03T17:04:33.610 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:41 ago on Wed 03 Nov 2021 05:02:52 PM UTC. 2021-11-03T17:04:33.861 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-remoto-1.2.1-1.el8.noarch 3/6 2021-11-03T17:04:34.188 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 4/6 2021-11-03T17:04:34.399 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 5/6 2021-11-03T17:04:34.609 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-mgr-cephadm-2:16.2.4-0.el8.noarch 6/6 2021-11-03T17:04:34.720 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2021-11-03T17:04:34.722 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:04:34.722 INFO:teuthology.orchestra.run.smithi027.stdout: Package Arch Version Repository Size 2021-11-03T17:04:34.723 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:04:34.723 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2021-11-03T17:04:34.723 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-cephadm noarch 2:16.2.4-0.el8 ceph-noarch 89 k 2021-11-03T17:04:34.723 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2021-11-03T17:04:34.724 INFO:teuthology.orchestra.run.smithi027.stdout: python3-apipkg noarch 1.5-6.el8 epel 21 k 2021-11-03T17:04:34.724 INFO:teuthology.orchestra.run.smithi027.stdout: python3-babel noarch 2.5.1-5.el8 CentOS-AppStream 4.8 M 2021-11-03T17:04:34.724 INFO:teuthology.orchestra.run.smithi027.stdout: python3-execnet noarch 1.7.1-1.el8 epel 355 k 2021-11-03T17:04:34.724 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 CentOS-AppStream 538 k 2021-11-03T17:04:34.724 INFO:teuthology.orchestra.run.smithi027.stdout: python3-remoto noarch 1.2.1-1.el8 epel 49 k 2021-11-03T17:04:34.725 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:04:34.725 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2021-11-03T17:04:34.725 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:04:34.725 INFO:teuthology.orchestra.run.smithi027.stdout:Install 6 Packages 2021-11-03T17:04:34.726 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:04:34.726 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 5.8 M 2021-11-03T17:04:34.727 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 24 M 2021-11-03T17:04:34.727 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2021-11-03T17:04:34.746 INFO:teuthology.orchestra.run.smithi027.stdout:(1/6): ceph-mgr-cephadm-16.2.4-0.el8.noarch.rpm 3.9 MB/s | 89 kB 00:00 2021-11-03T17:04:34.784 INFO:teuthology.orchestra.run.smithi027.stdout:(2/6): python3-jinja2-2.10.1-2.el8_0.noarch.rpm 8.7 MB/s | 538 kB 00:00 2021-11-03T17:04:34.822 INFO:teuthology.orchestra.run.smithi027.stdout:(3/6): python3-babel-2.5.1-5.el8.noarch.rpm 49 MB/s | 4.8 MB 00:00 2021-11-03T17:04:34.822 INFO:teuthology.orchestra.run.smithi027.stdout:(4/6): python3-apipkg-1.5-6.el8.noarch.rpm 279 kB/s | 21 kB 00:00 2021-11-03T17:04:34.845 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-mgr-cephadm-2:16.2.4-0.el8.noarch 6/6 2021-11-03T17:04:34.846 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-mgr-cephadm-2:16.2.4-0.el8.noarch 1/6 2021-11-03T17:04:34.846 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 2/6 2021-11-03T17:04:34.846 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 3/6 2021-11-03T17:04:34.847 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 4/6 2021-11-03T17:04:34.847 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 5/6 2021-11-03T17:04:34.870 INFO:teuthology.orchestra.run.smithi027.stdout:(5/6): python3-remoto-1.2.1-1.el8.noarch.rpm 1.0 MB/s | 49 kB 00:00 2021-11-03T17:04:34.962 INFO:teuthology.orchestra.run.smithi027.stdout:(6/6): python3-execnet-1.7.1-1.el8.noarch.rpm 2.0 MB/s | 355 kB 00:00 2021-11-03T17:04:34.964 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2021-11-03T17:04:34.964 INFO:teuthology.orchestra.run.smithi027.stdout:Total 24 MB/s | 5.8 MB 00:00 2021-11-03T17:04:34.969 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2021-11-03T17:04:34.979 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2021-11-03T17:04:34.979 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2021-11-03T17:04:35.066 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2021-11-03T17:04:35.069 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2021-11-03T17:04:35.136 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : python3-remoto-1.2.1-1.el8.noarch 6/6 2021-11-03T17:04:35.137 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:35.137 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2021-11-03T17:04:35.137 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-mgr-cephadm-2:16.2.4-0.el8.noarch python3-apipkg-1.5-6.el8.noarch 2021-11-03T17:04:35.138 INFO:teuthology.orchestra.run.smithi115.stdout: python3-babel-2.5.1-5.el8.noarch python3-execnet-1.7.1-1.el8.noarch 2021-11-03T17:04:35.138 INFO:teuthology.orchestra.run.smithi115.stdout: python3-jinja2-2.10.1-2.el8_0.noarch python3-remoto-1.2.1-1.el8.noarch 2021-11-03T17:04:35.138 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:35.138 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2021-11-03T17:04:35.197 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install ceph-fuse 2021-11-03T17:04:35.383 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2021-11-03T17:04:35.642 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 1/6 2021-11-03T17:04:35.701 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:42 ago on Wed 03 Nov 2021 05:02:53 PM UTC. 2021-11-03T17:04:35.872 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 2/6 2021-11-03T17:04:36.684 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2021-11-03T17:04:36.685 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:04:36.686 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2021-11-03T17:04:36.686 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:04:36.686 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2021-11-03T17:04:36.686 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-fuse x86_64 2:16.2.4-0.el8 ceph 885 k 2021-11-03T17:04:36.687 INFO:teuthology.orchestra.run.smithi115.stdout:Installing dependencies: 2021-11-03T17:04:36.687 INFO:teuthology.orchestra.run.smithi115.stdout: fuse x86_64 2.9.7-12.el8 CentOS-Base 82 k 2021-11-03T17:04:36.687 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:36.687 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2021-11-03T17:04:36.688 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:04:36.688 INFO:teuthology.orchestra.run.smithi115.stdout:Install 2 Packages 2021-11-03T17:04:36.688 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:36.688 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 967 k 2021-11-03T17:04:36.689 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 3.2 M 2021-11-03T17:04:36.689 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2021-11-03T17:04:36.712 INFO:teuthology.orchestra.run.smithi115.stdout:(1/2): ceph-fuse-16.2.4-0.el8.x86_64.rpm 34 MB/s | 885 kB 00:00 2021-11-03T17:04:36.723 INFO:teuthology.orchestra.run.smithi115.stdout:(2/2): fuse-2.9.7-12.el8.x86_64.rpm 2.2 MB/s | 82 kB 00:00 2021-11-03T17:04:36.724 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2021-11-03T17:04:36.725 INFO:teuthology.orchestra.run.smithi115.stdout:Total 24 MB/s | 967 kB 00:00 2021-11-03T17:04:36.727 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2021-11-03T17:04:36.748 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2021-11-03T17:04:36.749 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2021-11-03T17:04:36.826 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2021-11-03T17:04:36.829 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2021-11-03T17:04:36.878 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-remoto-1.2.1-1.el8.noarch 3/6 2021-11-03T17:04:37.015 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2021-11-03T17:04:37.219 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 4/6 2021-11-03T17:04:37.317 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : fuse-2.9.7-12.el8.x86_64 1/2 2021-11-03T17:04:37.471 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 5/6 2021-11-03T17:04:37.491 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : ceph-fuse-2:16.2.4-0.el8.x86_64 2/2 2021-11-03T17:04:37.647 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-mgr-cephadm-2:16.2.4-0.el8.noarch 6/6 2021-11-03T17:04:37.825 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: ceph-fuse-2:16.2.4-0.el8.x86_64 2/2 2021-11-03T17:04:37.826 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : ceph-fuse-2:16.2.4-0.el8.x86_64 1/2 2021-11-03T17:04:37.921 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-mgr-cephadm-2:16.2.4-0.el8.noarch 6/6 2021-11-03T17:04:37.921 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-mgr-cephadm-2:16.2.4-0.el8.noarch 1/6 2021-11-03T17:04:37.922 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 2/6 2021-11-03T17:04:37.922 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 3/6 2021-11-03T17:04:37.922 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 4/6 2021-11-03T17:04:37.923 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 5/6 2021-11-03T17:04:38.139 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 2/2 2021-11-03T17:04:38.139 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:38.140 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2021-11-03T17:04:38.140 INFO:teuthology.orchestra.run.smithi115.stdout: ceph-fuse-2:16.2.4-0.el8.x86_64 fuse-2.9.7-12.el8.x86_64 2021-11-03T17:04:38.141 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:38.141 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2021-11-03T17:04:38.234 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : python3-remoto-1.2.1-1.el8.noarch 6/6 2021-11-03T17:04:38.235 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:04:38.235 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2021-11-03T17:04:38.235 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-mgr-cephadm-2:16.2.4-0.el8.noarch python3-apipkg-1.5-6.el8.noarch 2021-11-03T17:04:38.236 INFO:teuthology.orchestra.run.smithi027.stdout: python3-babel-2.5.1-5.el8.noarch python3-execnet-1.7.1-1.el8.noarch 2021-11-03T17:04:38.236 INFO:teuthology.orchestra.run.smithi027.stdout: python3-jinja2-2.10.1-2.el8_0.noarch python3-remoto-1.2.1-1.el8.noarch 2021-11-03T17:04:38.237 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:04:38.237 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2021-11-03T17:04:38.274 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install librados-devel 2021-11-03T17:04:38.287 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install ceph-fuse 2021-11-03T17:04:38.739 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:45 ago on Wed 03 Nov 2021 05:02:53 PM UTC. 2021-11-03T17:04:38.770 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:46 ago on Wed 03 Nov 2021 05:02:52 PM UTC. 2021-11-03T17:04:39.759 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2021-11-03T17:04:39.760 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:04:39.761 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2021-11-03T17:04:39.761 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:04:39.761 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2021-11-03T17:04:39.761 INFO:teuthology.orchestra.run.smithi115.stdout: librados-devel x86_64 2:16.2.4-0.el8 ceph 131 k 2021-11-03T17:04:39.761 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:39.762 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2021-11-03T17:04:39.762 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:04:39.762 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2021-11-03T17:04:39.762 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:39.763 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 131 k 2021-11-03T17:04:39.763 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 439 k 2021-11-03T17:04:39.763 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2021-11-03T17:04:39.773 INFO:teuthology.orchestra.run.smithi115.stdout:librados-devel-16.2.4-0.el8.x86_64.rpm 11 MB/s | 131 kB 00:00 2021-11-03T17:04:39.773 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2021-11-03T17:04:39.774 INFO:teuthology.orchestra.run.smithi115.stdout:Total 9.5 MB/s | 131 kB 00:00 2021-11-03T17:04:39.774 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2021-11-03T17:04:39.781 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2021-11-03T17:04:39.782 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2021-11-03T17:04:39.831 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2021-11-03T17:04:39.834 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2021-11-03T17:04:39.852 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2021-11-03T17:04:39.853 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:04:39.854 INFO:teuthology.orchestra.run.smithi027.stdout: Package Architecture Version Repository Size 2021-11-03T17:04:39.854 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:04:39.854 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2021-11-03T17:04:39.854 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse x86_64 2:16.2.4-0.el8 ceph 885 k 2021-11-03T17:04:39.855 INFO:teuthology.orchestra.run.smithi027.stdout:Installing dependencies: 2021-11-03T17:04:39.855 INFO:teuthology.orchestra.run.smithi027.stdout: fuse x86_64 2.9.7-12.el8 CentOS-Base 82 k 2021-11-03T17:04:39.856 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:04:39.856 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2021-11-03T17:04:39.856 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:04:39.857 INFO:teuthology.orchestra.run.smithi027.stdout:Install 2 Packages 2021-11-03T17:04:39.857 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:04:39.857 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 967 k 2021-11-03T17:04:39.858 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 3.2 M 2021-11-03T17:04:39.858 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2021-11-03T17:04:39.880 INFO:teuthology.orchestra.run.smithi027.stdout:(1/2): ceph-fuse-16.2.4-0.el8.x86_64.rpm 33 MB/s | 885 kB 00:00 2021-11-03T17:04:39.891 INFO:teuthology.orchestra.run.smithi027.stdout:(2/2): fuse-2.9.7-12.el8.x86_64.rpm 2.2 MB/s | 82 kB 00:00 2021-11-03T17:04:39.893 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2021-11-03T17:04:39.894 INFO:teuthology.orchestra.run.smithi027.stdout:Total 24 MB/s | 967 kB 00:00 2021-11-03T17:04:39.896 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2021-11-03T17:04:39.920 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2021-11-03T17:04:39.920 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2021-11-03T17:04:39.959 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2021-11-03T17:04:40.002 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2021-11-03T17:04:40.006 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2021-11-03T17:04:40.139 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : librados-devel-2:16.2.4-0.el8.x86_64 1/1 2021-11-03T17:04:40.172 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2021-11-03T17:04:40.385 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: librados-devel-2:16.2.4-0.el8.x86_64 1/1 2021-11-03T17:04:40.477 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : fuse-2.9.7-12.el8.x86_64 1/2 2021-11-03T17:04:40.654 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : ceph-fuse-2:16.2.4-0.el8.x86_64 2/2 2021-11-03T17:04:40.707 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : librados-devel-2:16.2.4-0.el8.x86_64 1/1 2021-11-03T17:04:40.707 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:40.707 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2021-11-03T17:04:40.707 INFO:teuthology.orchestra.run.smithi115.stdout: librados-devel-2:16.2.4-0.el8.x86_64 2021-11-03T17:04:40.708 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:40.708 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2021-11-03T17:04:40.859 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install libcephfs2 2021-11-03T17:04:41.006 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: ceph-fuse-2:16.2.4-0.el8.x86_64 2/2 2021-11-03T17:04:41.006 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : ceph-fuse-2:16.2.4-0.el8.x86_64 1/2 2021-11-03T17:04:41.292 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 2/2 2021-11-03T17:04:41.293 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:04:41.293 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2021-11-03T17:04:41.293 INFO:teuthology.orchestra.run.smithi027.stdout: ceph-fuse-2:16.2.4-0.el8.x86_64 fuse-2.9.7-12.el8.x86_64 2021-11-03T17:04:41.293 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:04:41.294 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2021-11-03T17:04:41.339 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:48 ago on Wed 03 Nov 2021 05:02:53 PM UTC. 2021-11-03T17:04:41.480 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install librados-devel 2021-11-03T17:04:41.965 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:49 ago on Wed 03 Nov 2021 05:02:52 PM UTC. 2021-11-03T17:04:42.058 INFO:teuthology.orchestra.run.smithi115.stdout:Package libcephfs2-2:16.2.4-0.el8.x86_64 is already installed. 2021-11-03T17:04:42.333 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2021-11-03T17:04:42.334 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2021-11-03T17:04:42.334 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2021-11-03T17:04:42.377 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install libcephfs-devel 2021-11-03T17:04:42.830 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:49 ago on Wed 03 Nov 2021 05:02:53 PM UTC. 2021-11-03T17:04:43.062 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2021-11-03T17:04:43.063 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:04:43.063 INFO:teuthology.orchestra.run.smithi027.stdout: Package Architecture Version Repository Size 2021-11-03T17:04:43.064 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:04:43.064 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2021-11-03T17:04:43.064 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel x86_64 2:16.2.4-0.el8 ceph 131 k 2021-11-03T17:04:43.064 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:04:43.065 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2021-11-03T17:04:43.065 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:04:43.065 INFO:teuthology.orchestra.run.smithi027.stdout:Install 1 Package 2021-11-03T17:04:43.065 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:04:43.065 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 131 k 2021-11-03T17:04:43.066 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 439 k 2021-11-03T17:04:43.066 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2021-11-03T17:04:43.077 INFO:teuthology.orchestra.run.smithi027.stdout:librados-devel-16.2.4-0.el8.x86_64.rpm 9.9 MB/s | 131 kB 00:00 2021-11-03T17:04:43.078 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2021-11-03T17:04:43.078 INFO:teuthology.orchestra.run.smithi027.stdout:Total 9.0 MB/s | 131 kB 00:00 2021-11-03T17:04:43.079 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2021-11-03T17:04:43.085 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2021-11-03T17:04:43.086 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2021-11-03T17:04:43.139 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2021-11-03T17:04:43.142 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2021-11-03T17:04:43.257 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2021-11-03T17:04:43.437 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : librados-devel-2:16.2.4-0.el8.x86_64 1/1 2021-11-03T17:04:43.695 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: librados-devel-2:16.2.4-0.el8.x86_64 1/1 2021-11-03T17:04:43.828 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2021-11-03T17:04:43.830 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:04:43.830 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2021-11-03T17:04:43.831 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:04:43.831 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2021-11-03T17:04:43.832 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs-devel x86_64 2:16.2.4-0.el8 ceph 24 k 2021-11-03T17:04:43.833 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:43.833 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2021-11-03T17:04:43.834 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:04:43.834 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2021-11-03T17:04:43.834 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:43.834 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 24 k 2021-11-03T17:04:43.835 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 94 k 2021-11-03T17:04:43.835 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2021-11-03T17:04:43.840 INFO:teuthology.orchestra.run.smithi115.stdout:libcephfs-devel-16.2.4-0.el8.x86_64.rpm 2.2 MB/s | 24 kB 00:00 2021-11-03T17:04:43.841 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2021-11-03T17:04:43.841 INFO:teuthology.orchestra.run.smithi115.stdout:Total 2.0 MB/s | 24 kB 00:00 2021-11-03T17:04:43.842 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2021-11-03T17:04:43.846 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2021-11-03T17:04:43.846 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2021-11-03T17:04:43.848 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2021-11-03T17:04:43.851 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2021-11-03T17:04:43.898 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2021-11-03T17:04:44.007 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : librados-devel-2:16.2.4-0.el8.x86_64 1/1 2021-11-03T17:04:44.008 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:04:44.008 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2021-11-03T17:04:44.009 INFO:teuthology.orchestra.run.smithi027.stdout: librados-devel-2:16.2.4-0.el8.x86_64 2021-11-03T17:04:44.009 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:04:44.009 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2021-11-03T17:04:44.064 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : libcephfs-devel-2:16.2.4-0.el8.x86_64 1/1 2021-11-03T17:04:44.157 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install libcephfs2 2021-11-03T17:04:44.284 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: libcephfs-devel-2:16.2.4-0.el8.x86_64 1/1 2021-11-03T17:04:44.533 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : libcephfs-devel-2:16.2.4-0.el8.x86_64 1/1 2021-11-03T17:04:44.534 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:44.534 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2021-11-03T17:04:44.534 INFO:teuthology.orchestra.run.smithi115.stdout: libcephfs-devel-2:16.2.4-0.el8.x86_64 2021-11-03T17:04:44.534 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:44.535 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2021-11-03T17:04:44.610 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install librados2 2021-11-03T17:04:44.640 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:52 ago on Wed 03 Nov 2021 05:02:52 PM UTC. 2021-11-03T17:04:45.066 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:52 ago on Wed 03 Nov 2021 05:02:53 PM UTC. 2021-11-03T17:04:45.413 INFO:teuthology.orchestra.run.smithi027.stdout:Package libcephfs2-2:16.2.4-0.el8.x86_64 is already installed. 2021-11-03T17:04:45.703 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2021-11-03T17:04:45.704 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2021-11-03T17:04:45.704 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2021-11-03T17:04:45.750 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install libcephfs-devel 2021-11-03T17:04:45.791 INFO:teuthology.orchestra.run.smithi115.stdout:Package librados2-2:16.2.4-0.el8.x86_64 is already installed. 2021-11-03T17:04:46.071 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2021-11-03T17:04:46.072 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2021-11-03T17:04:46.073 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2021-11-03T17:04:46.118 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install librbd1 2021-11-03T17:04:46.231 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:54 ago on Wed 03 Nov 2021 05:02:52 PM UTC. 2021-11-03T17:04:46.572 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:53 ago on Wed 03 Nov 2021 05:02:53 PM UTC. 2021-11-03T17:04:47.282 INFO:teuthology.orchestra.run.smithi115.stdout:Package librbd1-2:16.2.4-0.el8.x86_64 is already installed. 2021-11-03T17:04:47.291 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2021-11-03T17:04:47.291 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:04:47.291 INFO:teuthology.orchestra.run.smithi027.stdout: Package Architecture Version Repository Size 2021-11-03T17:04:47.292 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:04:47.292 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2021-11-03T17:04:47.292 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel x86_64 2:16.2.4-0.el8 ceph 24 k 2021-11-03T17:04:47.292 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:04:47.293 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2021-11-03T17:04:47.293 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:04:47.293 INFO:teuthology.orchestra.run.smithi027.stdout:Install 1 Package 2021-11-03T17:04:47.293 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:04:47.294 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 24 k 2021-11-03T17:04:47.294 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 94 k 2021-11-03T17:04:47.294 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2021-11-03T17:04:47.304 INFO:teuthology.orchestra.run.smithi027.stdout:libcephfs-devel-16.2.4-0.el8.x86_64.rpm 2.0 MB/s | 24 kB 00:00 2021-11-03T17:04:47.306 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2021-11-03T17:04:47.306 INFO:teuthology.orchestra.run.smithi027.stdout:Total 1.8 MB/s | 24 kB 00:00 2021-11-03T17:04:47.306 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2021-11-03T17:04:47.310 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2021-11-03T17:04:47.311 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2021-11-03T17:04:47.312 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2021-11-03T17:04:47.316 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2021-11-03T17:04:47.370 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2021-11-03T17:04:47.544 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : libcephfs-devel-2:16.2.4-0.el8.x86_64 1/1 2021-11-03T17:04:47.560 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2021-11-03T17:04:47.560 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2021-11-03T17:04:47.561 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2021-11-03T17:04:47.604 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install python3-rados 2021-11-03T17:04:47.778 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: libcephfs-devel-2:16.2.4-0.el8.x86_64 1/1 2021-11-03T17:04:48.059 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:55 ago on Wed 03 Nov 2021 05:02:53 PM UTC. 2021-11-03T17:04:48.090 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : libcephfs-devel-2:16.2.4-0.el8.x86_64 1/1 2021-11-03T17:04:48.091 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:04:48.091 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2021-11-03T17:04:48.091 INFO:teuthology.orchestra.run.smithi027.stdout: libcephfs-devel-2:16.2.4-0.el8.x86_64 2021-11-03T17:04:48.092 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:04:48.092 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2021-11-03T17:04:48.166 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install librados2 2021-11-03T17:04:48.653 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:56 ago on Wed 03 Nov 2021 05:02:52 PM UTC. 2021-11-03T17:04:48.782 INFO:teuthology.orchestra.run.smithi115.stdout:Package python3-rados-2:16.2.4-0.el8.x86_64 is already installed. 2021-11-03T17:04:49.054 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2021-11-03T17:04:49.055 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2021-11-03T17:04:49.055 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2021-11-03T17:04:49.099 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install python3-rgw 2021-11-03T17:04:49.421 INFO:teuthology.orchestra.run.smithi027.stdout:Package librados2-2:16.2.4-0.el8.x86_64 is already installed. 2021-11-03T17:04:49.554 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:56 ago on Wed 03 Nov 2021 05:02:53 PM UTC. 2021-11-03T17:04:49.713 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2021-11-03T17:04:49.715 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2021-11-03T17:04:49.715 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2021-11-03T17:04:49.766 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install librbd1 2021-11-03T17:04:50.252 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:58 ago on Wed 03 Nov 2021 05:02:52 PM UTC. 2021-11-03T17:04:50.290 INFO:teuthology.orchestra.run.smithi115.stdout:Package python3-rgw-2:16.2.4-0.el8.x86_64 is already installed. 2021-11-03T17:04:50.561 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2021-11-03T17:04:50.562 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2021-11-03T17:04:50.563 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2021-11-03T17:04:50.608 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install python3-cephfs 2021-11-03T17:04:51.037 INFO:teuthology.orchestra.run.smithi027.stdout:Package librbd1-2:16.2.4-0.el8.x86_64 is already installed. 2021-11-03T17:04:51.060 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:58 ago on Wed 03 Nov 2021 05:02:53 PM UTC. 2021-11-03T17:04:51.331 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2021-11-03T17:04:51.333 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2021-11-03T17:04:51.333 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2021-11-03T17:04:51.385 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install python3-rados 2021-11-03T17:04:51.773 INFO:teuthology.orchestra.run.smithi115.stdout:Package python3-cephfs-2:16.2.4-0.el8.x86_64 is already installed. 2021-11-03T17:04:51.869 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:01:59 ago on Wed 03 Nov 2021 05:02:52 PM UTC. 2021-11-03T17:04:52.049 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2021-11-03T17:04:52.050 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2021-11-03T17:04:52.050 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2021-11-03T17:04:52.101 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install python3-rbd 2021-11-03T17:04:52.559 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:01:59 ago on Wed 03 Nov 2021 05:02:53 PM UTC. 2021-11-03T17:04:52.654 INFO:teuthology.orchestra.run.smithi027.stdout:Package python3-rados-2:16.2.4-0.el8.x86_64 is already installed. 2021-11-03T17:04:52.939 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2021-11-03T17:04:52.941 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2021-11-03T17:04:52.941 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2021-11-03T17:04:52.988 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install python3-rgw 2021-11-03T17:04:53.469 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:02:01 ago on Wed 03 Nov 2021 05:02:52 PM UTC. 2021-11-03T17:04:53.623 INFO:teuthology.orchestra.run.smithi115.stdout:Package python3-rbd-2:16.2.4-0.el8.x86_64 is already installed. 2021-11-03T17:04:53.898 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2021-11-03T17:04:53.899 INFO:teuthology.orchestra.run.smithi115.stdout:Nothing to do. 2021-11-03T17:04:53.899 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2021-11-03T17:04:53.952 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install rbd-fuse 2021-11-03T17:04:54.237 INFO:teuthology.orchestra.run.smithi027.stdout:Package python3-rgw-2:16.2.4-0.el8.x86_64 is already installed. 2021-11-03T17:04:54.405 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:02:01 ago on Wed 03 Nov 2021 05:02:53 PM UTC. 2021-11-03T17:04:54.524 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2021-11-03T17:04:54.525 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2021-11-03T17:04:54.525 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2021-11-03T17:04:54.569 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install python3-cephfs 2021-11-03T17:04:55.050 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:02:03 ago on Wed 03 Nov 2021 05:02:52 PM UTC. 2021-11-03T17:04:55.416 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2021-11-03T17:04:55.417 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:04:55.417 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2021-11-03T17:04:55.418 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:04:55.418 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2021-11-03T17:04:55.418 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-fuse x86_64 2:16.2.4-0.el8 ceph 89 k 2021-11-03T17:04:55.418 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:55.419 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2021-11-03T17:04:55.419 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:04:55.419 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2021-11-03T17:04:55.420 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:55.420 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 89 k 2021-11-03T17:04:55.420 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 298 k 2021-11-03T17:04:55.421 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2021-11-03T17:04:55.429 INFO:teuthology.orchestra.run.smithi115.stdout:rbd-fuse-16.2.4-0.el8.x86_64.rpm 7.7 MB/s | 89 kB 00:00 2021-11-03T17:04:55.429 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2021-11-03T17:04:55.430 INFO:teuthology.orchestra.run.smithi115.stdout:Total 6.9 MB/s | 89 kB 00:00 2021-11-03T17:04:55.431 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2021-11-03T17:04:55.442 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2021-11-03T17:04:55.442 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2021-11-03T17:04:55.493 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2021-11-03T17:04:55.497 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2021-11-03T17:04:55.601 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2021-11-03T17:04:55.777 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-fuse-2:16.2.4-0.el8.x86_64 1/1 2021-11-03T17:04:56.022 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: rbd-fuse-2:16.2.4-0.el8.x86_64 1/1 2021-11-03T17:04:56.196 INFO:teuthology.orchestra.run.smithi027.stdout:Package python3-cephfs-2:16.2.4-0.el8.x86_64 is already installed. 2021-11-03T17:04:56.308 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-fuse-2:16.2.4-0.el8.x86_64 1/1 2021-11-03T17:04:56.308 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:56.309 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2021-11-03T17:04:56.309 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-fuse-2:16.2.4-0.el8.x86_64 2021-11-03T17:04:56.309 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:56.309 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2021-11-03T17:04:56.461 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install rbd-mirror 2021-11-03T17:04:56.495 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2021-11-03T17:04:56.496 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2021-11-03T17:04:56.496 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2021-11-03T17:04:56.851 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install python3-rbd 2021-11-03T17:04:56.921 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:02:03 ago on Wed 03 Nov 2021 05:02:53 PM UTC. 2021-11-03T17:04:57.339 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:02:05 ago on Wed 03 Nov 2021 05:02:52 PM UTC. 2021-11-03T17:04:57.920 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2021-11-03T17:04:57.921 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:04:57.922 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2021-11-03T17:04:57.922 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:04:57.922 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2021-11-03T17:04:57.922 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-mirror x86_64 2:16.2.4-0.el8 ceph 3.7 M 2021-11-03T17:04:57.923 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:57.923 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2021-11-03T17:04:57.923 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:04:57.923 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2021-11-03T17:04:57.924 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:57.924 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 3.7 M 2021-11-03T17:04:57.925 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 18 M 2021-11-03T17:04:57.925 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2021-11-03T17:04:57.993 INFO:teuthology.orchestra.run.smithi115.stdout:rbd-mirror-16.2.4-0.el8.x86_64.rpm 52 MB/s | 3.7 MB 00:00 2021-11-03T17:04:57.994 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2021-11-03T17:04:57.995 INFO:teuthology.orchestra.run.smithi115.stdout:Total 51 MB/s | 3.7 MB 00:00 2021-11-03T17:04:57.995 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2021-11-03T17:04:58.009 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2021-11-03T17:04:58.009 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2021-11-03T17:04:58.073 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2021-11-03T17:04:58.077 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2021-11-03T17:04:58.111 INFO:teuthology.orchestra.run.smithi027.stdout:Package python3-rbd-2:16.2.4-0.el8.x86_64 is already installed. 2021-11-03T17:04:58.402 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2021-11-03T17:04:58.403 INFO:teuthology.orchestra.run.smithi027.stdout:Nothing to do. 2021-11-03T17:04:58.403 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2021-11-03T17:04:58.445 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install rbd-fuse 2021-11-03T17:04:58.542 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2021-11-03T17:04:58.729 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-mirror-2:16.2.4-0.el8.x86_64 1/1 2021-11-03T17:04:58.932 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:02:06 ago on Wed 03 Nov 2021 05:02:52 PM UTC. 2021-11-03T17:04:59.078 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: rbd-mirror-2:16.2.4-0.el8.x86_64 1/1 2021-11-03T17:04:59.372 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-mirror-2:16.2.4-0.el8.x86_64 1/1 2021-11-03T17:04:59.372 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:59.373 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2021-11-03T17:04:59.373 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-mirror-2:16.2.4-0.el8.x86_64 2021-11-03T17:04:59.373 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:04:59.373 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2021-11-03T17:04:59.514 DEBUG:teuthology.orchestra.run.smithi115:> sudo yum -y install rbd-nbd 2021-11-03T17:04:59.981 INFO:teuthology.orchestra.run.smithi115.stdout:Last metadata expiration check: 0:02:06 ago on Wed 03 Nov 2021 05:02:53 PM UTC. 2021-11-03T17:05:00.082 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2021-11-03T17:05:00.083 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:05:00.083 INFO:teuthology.orchestra.run.smithi027.stdout: Package Architecture Version Repository Size 2021-11-03T17:05:00.083 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:05:00.084 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2021-11-03T17:05:00.084 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse x86_64 2:16.2.4-0.el8 ceph 89 k 2021-11-03T17:05:00.084 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:05:00.084 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2021-11-03T17:05:00.085 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:05:00.085 INFO:teuthology.orchestra.run.smithi027.stdout:Install 1 Package 2021-11-03T17:05:00.085 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:05:00.086 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 89 k 2021-11-03T17:05:00.086 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 298 k 2021-11-03T17:05:00.086 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2021-11-03T17:05:00.096 INFO:teuthology.orchestra.run.smithi027.stdout:rbd-fuse-16.2.4-0.el8.x86_64.rpm 6.9 MB/s | 89 kB 00:00 2021-11-03T17:05:00.097 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2021-11-03T17:05:00.097 INFO:teuthology.orchestra.run.smithi027.stdout:Total 6.3 MB/s | 89 kB 00:00 2021-11-03T17:05:00.098 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2021-11-03T17:05:00.110 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2021-11-03T17:05:00.110 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2021-11-03T17:05:00.163 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2021-11-03T17:05:00.166 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2021-11-03T17:05:00.275 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2021-11-03T17:05:00.468 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-fuse-2:16.2.4-0.el8.x86_64 1/1 2021-11-03T17:05:00.750 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: rbd-fuse-2:16.2.4-0.el8.x86_64 1/1 2021-11-03T17:05:00.995 INFO:teuthology.orchestra.run.smithi115.stdout:Dependencies resolved. 2021-11-03T17:05:00.998 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:05:00.999 INFO:teuthology.orchestra.run.smithi115.stdout: Package Architecture Version Repository Size 2021-11-03T17:05:00.999 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:05:00.999 INFO:teuthology.orchestra.run.smithi115.stdout:Installing: 2021-11-03T17:05:01.000 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-nbd x86_64 2:16.2.4-0.el8 ceph 173 k 2021-11-03T17:05:01.000 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:05:01.000 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction Summary 2021-11-03T17:05:01.001 INFO:teuthology.orchestra.run.smithi115.stdout:================================================================================ 2021-11-03T17:05:01.001 INFO:teuthology.orchestra.run.smithi115.stdout:Install 1 Package 2021-11-03T17:05:01.001 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:05:01.001 INFO:teuthology.orchestra.run.smithi115.stdout:Total download size: 173 k 2021-11-03T17:05:01.002 INFO:teuthology.orchestra.run.smithi115.stdout:Installed size: 649 k 2021-11-03T17:05:01.002 INFO:teuthology.orchestra.run.smithi115.stdout:Downloading Packages: 2021-11-03T17:05:01.009 INFO:teuthology.orchestra.run.smithi115.stdout:rbd-nbd-16.2.4-0.el8.x86_64.rpm 13 MB/s | 173 kB 00:00 2021-11-03T17:05:01.010 INFO:teuthology.orchestra.run.smithi115.stdout:-------------------------------------------------------------------------------- 2021-11-03T17:05:01.010 INFO:teuthology.orchestra.run.smithi115.stdout:Total 12 MB/s | 173 kB 00:00 2021-11-03T17:05:01.011 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction check 2021-11-03T17:05:01.024 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction check succeeded. 2021-11-03T17:05:01.024 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction test 2021-11-03T17:05:01.055 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-fuse-2:16.2.4-0.el8.x86_64 1/1 2021-11-03T17:05:01.056 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:05:01.056 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2021-11-03T17:05:01.056 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-fuse-2:16.2.4-0.el8.x86_64 2021-11-03T17:05:01.056 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:05:01.057 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2021-11-03T17:05:01.073 INFO:teuthology.orchestra.run.smithi115.stdout:Transaction test succeeded. 2021-11-03T17:05:01.076 INFO:teuthology.orchestra.run.smithi115.stdout:Running transaction 2021-11-03T17:05:01.211 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install rbd-mirror 2021-11-03T17:05:01.224 INFO:teuthology.orchestra.run.smithi115.stdout: Preparing : 1/1 2021-11-03T17:05:01.385 INFO:teuthology.orchestra.run.smithi115.stdout: Installing : rbd-nbd-2:16.2.4-0.el8.x86_64 1/1 2021-11-03T17:05:01.621 INFO:teuthology.orchestra.run.smithi115.stdout: Running scriptlet: rbd-nbd-2:16.2.4-0.el8.x86_64 1/1 2021-11-03T17:05:01.692 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:02:09 ago on Wed 03 Nov 2021 05:02:52 PM UTC. 2021-11-03T17:05:01.890 INFO:teuthology.orchestra.run.smithi115.stdout: Verifying : rbd-nbd-2:16.2.4-0.el8.x86_64 1/1 2021-11-03T17:05:01.890 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:05:01.890 INFO:teuthology.orchestra.run.smithi115.stdout:Installed: 2021-11-03T17:05:01.891 INFO:teuthology.orchestra.run.smithi115.stdout: rbd-nbd-2:16.2.4-0.el8.x86_64 2021-11-03T17:05:01.891 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:05:01.891 INFO:teuthology.orchestra.run.smithi115.stdout:Complete! 2021-11-03T17:05:02.033 DEBUG:teuthology.parallel:result is None 2021-11-03T17:05:02.775 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2021-11-03T17:05:02.776 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:05:02.776 INFO:teuthology.orchestra.run.smithi027.stdout: Package Architecture Version Repository Size 2021-11-03T17:05:02.776 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:05:02.777 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2021-11-03T17:05:02.777 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror x86_64 2:16.2.4-0.el8 ceph 3.7 M 2021-11-03T17:05:02.777 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:05:02.777 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2021-11-03T17:05:02.777 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:05:02.778 INFO:teuthology.orchestra.run.smithi027.stdout:Install 1 Package 2021-11-03T17:05:02.778 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:05:02.778 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 3.7 M 2021-11-03T17:05:02.778 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 18 M 2021-11-03T17:05:02.779 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2021-11-03T17:05:02.838 INFO:teuthology.orchestra.run.smithi027.stdout:rbd-mirror-16.2.4-0.el8.x86_64.rpm 60 MB/s | 3.7 MB 00:00 2021-11-03T17:05:02.839 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2021-11-03T17:05:02.840 INFO:teuthology.orchestra.run.smithi027.stdout:Total 59 MB/s | 3.7 MB 00:00 2021-11-03T17:05:02.840 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2021-11-03T17:05:02.855 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2021-11-03T17:05:02.855 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2021-11-03T17:05:02.924 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2021-11-03T17:05:02.927 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2021-11-03T17:05:03.452 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2021-11-03T17:05:03.669 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-mirror-2:16.2.4-0.el8.x86_64 1/1 2021-11-03T17:05:04.049 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: rbd-mirror-2:16.2.4-0.el8.x86_64 1/1 2021-11-03T17:05:04.339 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-mirror-2:16.2.4-0.el8.x86_64 1/1 2021-11-03T17:05:04.339 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:05:04.339 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2021-11-03T17:05:04.339 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-mirror-2:16.2.4-0.el8.x86_64 2021-11-03T17:05:04.340 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:05:04.340 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2021-11-03T17:05:04.476 DEBUG:teuthology.orchestra.run.smithi027:> sudo yum -y install rbd-nbd 2021-11-03T17:05:04.967 INFO:teuthology.orchestra.run.smithi027.stdout:Last metadata expiration check: 0:02:12 ago on Wed 03 Nov 2021 05:02:52 PM UTC. 2021-11-03T17:05:06.030 INFO:teuthology.orchestra.run.smithi027.stdout:Dependencies resolved. 2021-11-03T17:05:06.031 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:05:06.032 INFO:teuthology.orchestra.run.smithi027.stdout: Package Architecture Version Repository Size 2021-11-03T17:05:06.032 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:05:06.032 INFO:teuthology.orchestra.run.smithi027.stdout:Installing: 2021-11-03T17:05:06.032 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd x86_64 2:16.2.4-0.el8 ceph 173 k 2021-11-03T17:05:06.033 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:05:06.033 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction Summary 2021-11-03T17:05:06.033 INFO:teuthology.orchestra.run.smithi027.stdout:================================================================================ 2021-11-03T17:05:06.033 INFO:teuthology.orchestra.run.smithi027.stdout:Install 1 Package 2021-11-03T17:05:06.034 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:05:06.034 INFO:teuthology.orchestra.run.smithi027.stdout:Total download size: 173 k 2021-11-03T17:05:06.034 INFO:teuthology.orchestra.run.smithi027.stdout:Installed size: 649 k 2021-11-03T17:05:06.034 INFO:teuthology.orchestra.run.smithi027.stdout:Downloading Packages: 2021-11-03T17:05:06.045 INFO:teuthology.orchestra.run.smithi027.stdout:rbd-nbd-16.2.4-0.el8.x86_64.rpm 12 MB/s | 173 kB 00:00 2021-11-03T17:05:06.046 INFO:teuthology.orchestra.run.smithi027.stdout:-------------------------------------------------------------------------------- 2021-11-03T17:05:06.047 INFO:teuthology.orchestra.run.smithi027.stdout:Total 11 MB/s | 173 kB 00:00 2021-11-03T17:05:06.047 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction check 2021-11-03T17:05:06.060 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction check succeeded. 2021-11-03T17:05:06.061 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction test 2021-11-03T17:05:06.115 INFO:teuthology.orchestra.run.smithi027.stdout:Transaction test succeeded. 2021-11-03T17:05:06.119 INFO:teuthology.orchestra.run.smithi027.stdout:Running transaction 2021-11-03T17:05:06.244 INFO:teuthology.orchestra.run.smithi027.stdout: Preparing : 1/1 2021-11-03T17:05:06.417 INFO:teuthology.orchestra.run.smithi027.stdout: Installing : rbd-nbd-2:16.2.4-0.el8.x86_64 1/1 2021-11-03T17:05:06.675 INFO:teuthology.orchestra.run.smithi027.stdout: Running scriptlet: rbd-nbd-2:16.2.4-0.el8.x86_64 1/1 2021-11-03T17:05:06.989 INFO:teuthology.orchestra.run.smithi027.stdout: Verifying : rbd-nbd-2:16.2.4-0.el8.x86_64 1/1 2021-11-03T17:05:06.990 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:05:06.990 INFO:teuthology.orchestra.run.smithi027.stdout:Installed: 2021-11-03T17:05:06.990 INFO:teuthology.orchestra.run.smithi027.stdout: rbd-nbd-2:16.2.4-0.el8.x86_64 2021-11-03T17:05:06.991 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:05:06.991 INFO:teuthology.orchestra.run.smithi027.stdout:Complete! 2021-11-03T17:05:07.119 DEBUG:teuthology.parallel:result is None 2021-11-03T17:05:07.120 DEBUG:teuthology.orchestra.run.smithi027:> sudo lsb_release -is 2021-11-03T17:05:07.155 INFO:teuthology.orchestra.run.smithi027.stdout:CentOS 2021-11-03T17:05:07.156 DEBUG:teuthology.misc:System to be installed: CentOS 2021-11-03T17:05:07.157 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2021-11-03T17:05:07.157 INFO:teuthology.packaging:ref: None 2021-11-03T17:05:07.158 INFO:teuthology.packaging:tag: v16.2.4 2021-11-03T17:05:07.158 INFO:teuthology.packaging:branch: None 2021-11-03T17:05:07.158 INFO:teuthology.packaging:sha1: 6e90212d8510e2d506395a8c82057d0d1787f3a4 2021-11-03T17:05:07.188 DEBUG:teuthology.repo_utils:git ls-remote git://git.ceph.com/ceph-ci v16.2.4^{} -> 3cbe25cde3cfa028984618ad32de9edc4c1eaed0 2021-11-03T17:05:07.189 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3cbe25cde3cfa028984618ad32de9edc4c1eaed0 2021-11-03T17:05:07.408 DEBUG:teuthology.orchestra.run.smithi027:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2021-11-03T17:05:07.429 INFO:teuthology.orchestra.run.smithi027.stdout:16.2.4-0.el8 2021-11-03T17:05:07.430 INFO:teuthology.packaging:The installed version of ceph is 16.2.4-0.el8 2021-11-03T17:05:07.431 INFO:teuthology.task.install:The correct ceph version 16.2.4-0 is installed. 2021-11-03T17:05:07.431 DEBUG:teuthology.orchestra.run.smithi115:> sudo lsb_release -is 2021-11-03T17:05:07.472 INFO:teuthology.orchestra.run.smithi115.stdout:CentOS 2021-11-03T17:05:07.474 DEBUG:teuthology.misc:System to be installed: CentOS 2021-11-03T17:05:07.474 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2021-11-03T17:05:07.475 INFO:teuthology.packaging:ref: None 2021-11-03T17:05:07.475 INFO:teuthology.packaging:tag: v16.2.4 2021-11-03T17:05:07.476 INFO:teuthology.packaging:branch: None 2021-11-03T17:05:07.476 INFO:teuthology.packaging:sha1: 6e90212d8510e2d506395a8c82057d0d1787f3a4 2021-11-03T17:05:07.506 DEBUG:teuthology.repo_utils:git ls-remote git://git.ceph.com/ceph-ci v16.2.4^{} -> 3cbe25cde3cfa028984618ad32de9edc4c1eaed0 2021-11-03T17:05:07.508 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=3cbe25cde3cfa028984618ad32de9edc4c1eaed0 2021-11-03T17:05:07.718 DEBUG:teuthology.orchestra.run.smithi115:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2021-11-03T17:05:07.742 INFO:teuthology.orchestra.run.smithi115.stdout:16.2.4-0.el8 2021-11-03T17:05:07.742 INFO:teuthology.packaging:The installed version of ceph is 16.2.4-0.el8 2021-11-03T17:05:07.743 INFO:teuthology.task.install:The correct ceph version 16.2.4-0 is installed. 2021-11-03T17:05:07.743 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2021-11-03T17:05:07.744 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2021-11-03T17:05:07.745 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2021-11-03T17:05:07.774 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2021-11-03T17:05:07.774 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2021-11-03T17:05:07.809 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2021-11-03T17:05:07.810 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2021-11-03T17:05:07.810 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/daemon-helper 2021-11-03T17:05:07.841 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/daemon-helper 2021-11-03T17:05:07.908 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2021-11-03T17:05:07.909 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/daemon-helper 2021-11-03T17:05:07.936 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/daemon-helper 2021-11-03T17:05:08.001 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2021-11-03T17:05:08.002 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2021-11-03T17:05:08.002 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/usr/bin/adjust-ulimits 2021-11-03T17:05:08.030 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2021-11-03T17:05:08.121 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2021-11-03T17:05:08.121 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/usr/bin/adjust-ulimits 2021-11-03T17:05:08.147 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2021-11-03T17:05:08.212 INFO:teuthology.run_tasks:Running task cephadm... 2021-11-03T17:05:08.285 INFO:tasks.cephadm:Config: {'avoid_pacific_features': True, 'cephadm_branch': 'v16.2.4', 'cephadm_git_url': 'https://github.com/ceph/ceph', 'image': 'docker.io/ceph/ceph:v16.2.4', 'roleless': True, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': '15m', 'rados osd op timeout': '15m'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)'], 'sha1': '6e90212d8510e2d506395a8c82057d0d1787f3a4'} 2021-11-03T17:05:08.286 INFO:tasks.cephadm:Cluster image is docker.io/ceph/ceph:v16.2.4 2021-11-03T17:05:08.287 INFO:tasks.cephadm:Cluster fsid is 32ab977e-3cc8-11ec-8c28-001a4aab830c 2021-11-03T17:05:08.287 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2021-11-03T17:05:08.288 INFO:tasks.cephadm:No mon roles; fabricating mons 2021-11-03T17:05:08.288 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi027': '172.21.15.27', 'mon.smithi115': '172.21.15.115'} 2021-11-03T17:05:08.289 INFO:tasks.cephadm:Normalizing hostnames... 2021-11-03T17:05:08.289 DEBUG:teuthology.orchestra.run.smithi027:> sudo hostname $(hostname -s) 2021-11-03T17:05:08.319 DEBUG:teuthology.orchestra.run.smithi115:> sudo hostname $(hostname -s) 2021-11-03T17:05:08.345 INFO:tasks.cephadm:Downloading cephadm (repo https://github.com/ceph/ceph ref v16.2.4)... 2021-11-03T17:05:08.346 DEBUG:teuthology.orchestra.run.smithi027:> curl --silent https://raw.githubusercontent.com/ceph/ceph/v16.2.4/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2021-11-03T17:05:08.606 INFO:teuthology.orchestra.run.smithi027.stdout:-rw-rw-r--. 1 ubuntu ubuntu 292005 Nov 3 17:05 /home/ubuntu/cephtest/cephadm 2021-11-03T17:05:08.607 DEBUG:teuthology.orchestra.run.smithi115:> curl --silent https://raw.githubusercontent.com/ceph/ceph/v16.2.4/src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2021-11-03T17:05:08.714 INFO:teuthology.orchestra.run.smithi115.stdout:-rw-rw-r--. 1 ubuntu ubuntu 292005 Nov 3 17:05 /home/ubuntu/cephtest/cephadm 2021-11-03T17:05:08.715 DEBUG:teuthology.orchestra.run.smithi027:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2021-11-03T17:05:08.736 DEBUG:teuthology.orchestra.run.smithi115:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2021-11-03T17:05:08.774 DEBUG:teuthology.orchestra.run.smithi027:> sudo mkdir -p /etc/ceph 2021-11-03T17:05:08.807 DEBUG:teuthology.orchestra.run.smithi115:> sudo mkdir -p /etc/ceph 2021-11-03T17:05:08.835 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 777 /etc/ceph 2021-11-03T17:05:08.876 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 777 /etc/ceph 2021-11-03T17:05:08.905 INFO:tasks.cephadm:Writing seed config... 2021-11-03T17:05:08.907 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2021-11-03T17:05:08.907 INFO:tasks.cephadm: override: [client] debug client = 20 2021-11-03T17:05:08.908 INFO:tasks.cephadm: override: [client] debug ms = 1 2021-11-03T17:05:08.908 INFO:tasks.cephadm: override: [client] rados mon op timeout = 15m 2021-11-03T17:05:08.909 INFO:tasks.cephadm: override: [client] rados osd op timeout = 15m 2021-11-03T17:05:08.909 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2021-11-03T17:05:08.910 INFO:tasks.cephadm: override: [mds] debug mds = 20 2021-11-03T17:05:08.910 INFO:tasks.cephadm: override: [mds] debug ms = 1 2021-11-03T17:05:08.910 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2021-11-03T17:05:08.911 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2021-11-03T17:05:08.911 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2021-11-03T17:05:08.911 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2021-11-03T17:05:08.911 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2021-11-03T17:05:08.912 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 15m 2021-11-03T17:05:08.912 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 15m 2021-11-03T17:05:08.912 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2021-11-03T17:05:08.913 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2021-11-03T17:05:08.913 INFO:tasks.cephadm: override: [mon] debug mon = 20 2021-11-03T17:05:08.913 INFO:tasks.cephadm: override: [mon] debug ms = 1 2021-11-03T17:05:08.914 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2021-11-03T17:05:08.914 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2021-11-03T17:05:08.915 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2021-11-03T17:05:08.916 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2021-11-03T17:05:08.916 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2021-11-03T17:05:08.917 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2021-11-03T17:05:08.918 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2021-11-03T17:05:08.918 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2021-11-03T17:05:08.918 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2021-11-03T17:05:08.919 INFO:tasks.cephadm: override: [osd] debug ms = 1 2021-11-03T17:05:08.919 INFO:tasks.cephadm: override: [osd] debug osd = 20 2021-11-03T17:05:08.919 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2021-11-03T17:05:08.920 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2021-11-03T17:05:08.920 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2021-11-03T17:05:08.920 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2021-11-03T17:05:08.921 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2021-11-03T17:05:08.921 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2021-11-03T17:05:08.921 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2021-11-03T17:05:08.922 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2021-11-03T17:05:08.923 DEBUG:teuthology.orchestra.run.smithi027:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2021-11-03T17:05:08.939 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 32ab977e-3cc8-11ec-8c28-001a4aab830c mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 15m rados osd op timeout = 15m [mds] debug mds = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 15m rados osd op timeout = 15m 2021-11-03T17:05:08.940 DEBUG:teuthology.orchestra.run.smithi027:mon.smithi027> sudo journalctl -f -n 0 -u ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c@mon.smithi027.service 2021-11-03T17:05:08.982 INFO:tasks.cephadm:Bootstrapping... 2021-11-03T17:05:08.983 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 -v bootstrap --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.27 && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2021-11-03T17:05:09.009 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:-- Logs begin at Wed 2021-11-03 16:54:54 UTC. -- 2021-11-03T17:05:09.247 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman version --format {{.Client.Version}} 2021-11-03T17:05:10.082 INFO:teuthology.orchestra.run.smithi027.stderr:/bin/podman: 3.0.2-dev 2021-11-03T17:05:10.102 INFO:teuthology.orchestra.run.smithi027.stderr:Verifying podman|docker is present... 2021-11-03T17:05:10.102 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman version --format {{.Client.Version}} 2021-11-03T17:05:10.154 INFO:teuthology.orchestra.run.smithi027.stderr:/bin/podman: 3.0.2-dev 2021-11-03T17:05:10.174 INFO:teuthology.orchestra.run.smithi027.stderr:Verifying lvm2 is present... 2021-11-03T17:05:10.174 INFO:teuthology.orchestra.run.smithi027.stderr:Verifying time synchronization is in place... 2021-11-03T17:05:10.174 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: systemctl is-enabled chrony.service 2021-11-03T17:05:10.181 INFO:teuthology.orchestra.run.smithi027.stderr:systemctl: Failed to get unit file state for chrony.service: No such file or directory 2021-11-03T17:05:10.181 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: systemctl is-active chrony.service 2021-11-03T17:05:10.187 INFO:teuthology.orchestra.run.smithi027.stderr:systemctl: inactive 2021-11-03T17:05:10.188 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: systemctl is-enabled chronyd.service 2021-11-03T17:05:10.194 INFO:teuthology.orchestra.run.smithi027.stderr:systemctl: enabled 2021-11-03T17:05:10.195 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: systemctl is-active chronyd.service 2021-11-03T17:05:10.200 INFO:teuthology.orchestra.run.smithi027.stderr:systemctl: active 2021-11-03T17:05:10.201 INFO:teuthology.orchestra.run.smithi027.stderr:Unit chronyd.service is enabled and running 2021-11-03T17:05:10.202 INFO:teuthology.orchestra.run.smithi027.stderr:Repeating the final host check... 2021-11-03T17:05:10.202 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman version --format {{.Client.Version}} 2021-11-03T17:05:10.247 INFO:teuthology.orchestra.run.smithi027.stderr:/bin/podman: 3.0.2-dev 2021-11-03T17:05:10.261 INFO:teuthology.orchestra.run.smithi027.stderr:podman|docker (/bin/podman) is present 2021-11-03T17:05:10.262 INFO:teuthology.orchestra.run.smithi027.stderr:systemctl is present 2021-11-03T17:05:10.262 INFO:teuthology.orchestra.run.smithi027.stderr:lvcreate is present 2021-11-03T17:05:10.262 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: systemctl is-enabled chrony.service 2021-11-03T17:05:10.267 INFO:teuthology.orchestra.run.smithi027.stderr:systemctl: Failed to get unit file state for chrony.service: No such file or directory 2021-11-03T17:05:10.268 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: systemctl is-active chrony.service 2021-11-03T17:05:10.273 INFO:teuthology.orchestra.run.smithi027.stderr:systemctl: inactive 2021-11-03T17:05:10.274 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: systemctl is-enabled chronyd.service 2021-11-03T17:05:10.279 INFO:teuthology.orchestra.run.smithi027.stderr:systemctl: enabled 2021-11-03T17:05:10.279 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: systemctl is-active chronyd.service 2021-11-03T17:05:10.284 INFO:teuthology.orchestra.run.smithi027.stderr:systemctl: active 2021-11-03T17:05:10.286 INFO:teuthology.orchestra.run.smithi027.stderr:Unit chronyd.service is enabled and running 2021-11-03T17:05:10.286 INFO:teuthology.orchestra.run.smithi027.stderr:Host looks OK 2021-11-03T17:05:10.286 INFO:teuthology.orchestra.run.smithi027.stderr:Cluster fsid: 32ab977e-3cc8-11ec-8c28-001a4aab830c 2021-11-03T17:05:10.286 INFO:teuthology.orchestra.run.smithi027.stderr:Acquiring lock 140512085428712 on /run/cephadm/32ab977e-3cc8-11ec-8c28-001a4aab830c.lock 2021-11-03T17:05:10.287 INFO:teuthology.orchestra.run.smithi027.stderr:Lock 140512085428712 acquired on /run/cephadm/32ab977e-3cc8-11ec-8c28-001a4aab830c.lock 2021-11-03T17:05:10.287 INFO:teuthology.orchestra.run.smithi027.stderr:Verifying IP 172.21.15.27 port 3300 ... 2021-11-03T17:05:10.288 INFO:teuthology.orchestra.run.smithi027.stderr:Verifying IP 172.21.15.27 port 6789 ... 2021-11-03T17:05:10.288 INFO:teuthology.orchestra.run.smithi027.stderr:Base mon IP is 172.21.15.27, final addrv is [v2:172.21.15.27:3300,v1:172.21.15.27:6789] 2021-11-03T17:05:10.288 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /sbin/ip route ls 2021-11-03T17:05:10.289 INFO:teuthology.orchestra.run.smithi027.stderr:/sbin/ip: default via 172.21.15.254 dev ens1f0 proto dhcp metric 100 2021-11-03T17:05:10.289 INFO:teuthology.orchestra.run.smithi027.stderr:/sbin/ip: 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.27 metric 100 2021-11-03T17:05:10.290 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /sbin/ip -6 route ls 2021-11-03T17:05:10.291 INFO:teuthology.orchestra.run.smithi027.stderr:/sbin/ip: ::1 dev lo proto kernel metric 256 pref medium 2021-11-03T17:05:10.292 INFO:teuthology.orchestra.run.smithi027.stderr:/sbin/ip: fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2021-11-03T17:05:10.292 INFO:teuthology.orchestra.run.smithi027.stderr:/sbin/ip: default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1797sec hoplimit 64 pref medium 2021-11-03T17:05:10.292 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /sbin/ip -6 addr ls 2021-11-03T17:05:10.294 INFO:teuthology.orchestra.run.smithi027.stderr:/sbin/ip: 1: lo: mtu 65536 state UNKNOWN qlen 1000 2021-11-03T17:05:10.295 INFO:teuthology.orchestra.run.smithi027.stderr:/sbin/ip: inet6 ::1/128 scope host 2021-11-03T17:05:10.295 INFO:teuthology.orchestra.run.smithi027.stderr:/sbin/ip: valid_lft forever preferred_lft forever 2021-11-03T17:05:10.296 INFO:teuthology.orchestra.run.smithi027.stderr:/sbin/ip: 3: ens1f0: mtu 1500 state UP qlen 1000 2021-11-03T17:05:10.296 INFO:teuthology.orchestra.run.smithi027.stderr:/sbin/ip: inet6 fe80::ec4:7aff:febd:1606/64 scope link 2021-11-03T17:05:10.296 INFO:teuthology.orchestra.run.smithi027.stderr:/sbin/ip: valid_lft forever preferred_lft forever 2021-11-03T17:05:10.297 INFO:teuthology.orchestra.run.smithi027.stderr:Mon IP 172.21.15.27 is in CIDR network 172.21.0.0/20 2021-11-03T17:05:10.297 INFO:teuthology.orchestra.run.smithi027.stderr:- internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2021-11-03T17:05:10.298 INFO:teuthology.orchestra.run.smithi027.stderr:Pulling container image docker.io/ceph/ceph:v16.2.4... 2021-11-03T17:05:10.298 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman pull docker.io/ceph/ceph:v16.2.4 2021-11-03T17:05:10.386 INFO:teuthology.orchestra.run.smithi027.stderr:/bin/podman: Trying to pull docker.io/ceph/ceph:v16.2.4... 2021-11-03T17:05:10.747 INFO:teuthology.orchestra.run.smithi027.stderr:/bin/podman: Getting image source signatures 2021-11-03T17:05:10.858 INFO:teuthology.orchestra.run.smithi027.stderr:/bin/podman: Copying blob sha256:7a0437f04f83f084b7ed68ad9c4a4947e12fc4e1b006b38129bac89114ec3621 2021-11-03T17:05:10.908 INFO:teuthology.orchestra.run.smithi027.stderr:/bin/podman: Copying blob sha256:551101690791055467b1840be133b2e6b029d263771bc6c46a96bb7acdacac39 2021-11-03T17:05:17.999 INFO:teuthology.orchestra.run.smithi027.stderr:/bin/podman: Copying config sha256:8d91d370c2b86c07de46146aba8d36718eaefa69b1880c77fa312fda6efd7d29 2021-11-03T17:05:18.074 INFO:teuthology.orchestra.run.smithi027.stderr:/bin/podman: Writing manifest to image destination 2021-11-03T17:05:18.074 INFO:teuthology.orchestra.run.smithi027.stderr:/bin/podman: Storing signatures 2021-11-03T17:05:27.572 INFO:teuthology.orchestra.run.smithi027.stderr:/bin/podman: 8d91d370c2b86c07de46146aba8d36718eaefa69b1880c77fa312fda6efd7d29 2021-11-03T17:05:27.581 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi027 -e CEPH_USE_RANDOM_NONCE=1 docker.io/ceph/ceph:v16.2.4 --version 2021-11-03T17:05:28.313 INFO:teuthology.orchestra.run.smithi027.stderr:ceph: ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable) 2021-11-03T17:05:38.372 INFO:teuthology.orchestra.run.smithi027.stderr:Ceph version: ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable) 2021-11-03T17:05:38.373 INFO:teuthology.orchestra.run.smithi027.stderr:Extracting ceph user uid/gid from container image... 2021-11-03T17:05:38.373 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint stat --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi027 -e CEPH_USE_RANDOM_NONCE=1 docker.io/ceph/ceph:v16.2.4 -c %u %g /var/lib/ceph 2021-11-03T17:05:39.000 INFO:teuthology.orchestra.run.smithi027.stderr:stat: 167 167 2021-11-03T17:05:39.572 INFO:teuthology.orchestra.run.smithi027.stderr:Creating initial keys... 2021-11-03T17:05:39.572 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph-authtool --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi027 -e CEPH_USE_RANDOM_NONCE=1 docker.io/ceph/ceph:v16.2.4 --gen-print-key 2021-11-03T17:05:40.152 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-authtool: AQBkwYJhApaKCBAAdPjdJ9WcHDtzvKgTI4vGxw== 2021-11-03T17:05:40.754 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph-authtool --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi027 -e CEPH_USE_RANDOM_NONCE=1 docker.io/ceph/ceph:v16.2.4 --gen-print-key 2021-11-03T17:05:41.302 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-authtool: AQBlwYJhcfJ0ERAAxZtqLuMRgNnzRMbrCTZhbw== 2021-11-03T17:05:41.872 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph-authtool --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi027 -e CEPH_USE_RANDOM_NONCE=1 docker.io/ceph/ceph:v16.2.4 --gen-print-key 2021-11-03T17:05:42.462 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-authtool: AQBmwYJh66JoGRAA+oHE5uXthU1bOqTsljIHJA== 2021-11-03T17:05:43.003 INFO:teuthology.orchestra.run.smithi027.stderr:Creating initial monmap... 2021-11-03T17:05:43.003 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/monmaptool --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi027 -e CEPH_USE_RANDOM_NONCE=1 -v /tmp/ceph-tmp2pjd_c97:/tmp/monmap:z docker.io/ceph/ceph:v16.2.4 --create --clobber --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c --addv smithi027 [v2:172.21.15.27:3300,v1:172.21.15.27:6789] /tmp/monmap 2021-11-03T17:05:43.577 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: monmap file /tmp/monmap 2021-11-03T17:05:43.577 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: set fsid to 32ab977e-3cc8-11ec-8c28-001a4aab830c 2021-11-03T17:05:43.577 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/monmaptool: /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2021-11-03T17:05:44.192 INFO:teuthology.orchestra.run.smithi027.stderr:monmaptool for smithi027 [v2:172.21.15.27:3300,v1:172.21.15.27:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2021-11-03T17:05:44.192 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/monmaptool: set fsid to 32ab977e-3cc8-11ec-8c28-001a4aab830c 2021-11-03T17:05:44.192 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2021-11-03T17:05:44.193 INFO:teuthology.orchestra.run.smithi027.stderr: 2021-11-03T17:05:44.193 INFO:teuthology.orchestra.run.smithi027.stderr:Creating mon... 2021-11-03T17:05:44.193 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph-mon --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi027 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c:/var/log/ceph:z -v /var/lib/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c/mon.smithi027:/var/lib/ceph/mon/ceph-smithi027:z -v /tmp/ceph-tmpium3qiv9:/tmp/keyring:z -v /tmp/ceph-tmp2pjd_c97:/tmp/monmap:z docker.io/ceph/ceph:v16.2.4 --mkfs -i smithi027 --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -c /dev/null --monmap /tmp/monmap --keyring /tmp/keyring --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true 2021-11-03T17:05:44.762 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.726+0000 7f1af23cb700 0 set uid:gid to 167:167 (ceph:ceph) 2021-11-03T17:05:44.762 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.727+0000 7f1af23cb700 1 imported monmap: 2021-11-03T17:05:44.762 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: epoch 0 2021-11-03T17:05:44.763 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c 2021-11-03T17:05:44.763 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: last_changed 2021-11-03T17:05:43.573486+0000 2021-11-03T17:05:44.763 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: created 2021-11-03T17:05:43.573486+0000 2021-11-03T17:05:44.763 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: min_mon_release 0 (unknown) 2021-11-03T17:05:44.764 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: election_strategy: 1 2021-11-03T17:05:44.764 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: 0: [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon.smithi027 2021-11-03T17:05:44.764 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: 2021-11-03T17:05:44.764 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.727+0000 7f1af23cb700 0 /usr/bin/ceph-mon: set fsid to 32ab977e-3cc8-11ec-8c28-001a4aab830c 2021-11-03T17:05:44.764 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: RocksDB version: 6.8.1 2021-11-03T17:05:44.765 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: 2021-11-03T17:05:44.765 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2021-11-03T17:05:44.765 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Compile date May 13 2021 2021-11-03T17:05:44.765 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: DB SUMMARY 2021-11-03T17:05:44.766 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: 2021-11-03T17:05:44.766 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi027/store.db dir, Total Num: 0, files: 2021-11-03T17:05:44.766 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: 2021-11-03T17:05:44.766 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi027/store.db: 2021-11-03T17:05:44.766 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: 2021-11-03T17:05:44.767 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.error_if_exists: 0 2021-11-03T17:05:44.767 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.create_if_missing: 1 2021-11-03T17:05:44.767 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.paranoid_checks: 1 2021-11-03T17:05:44.767 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.env: 0x55883effd1c0 2021-11-03T17:05:44.768 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.fs: Posix File System 2021-11-03T17:05:44.768 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.info_log: 0x55884024cec0 2021-11-03T17:05:44.768 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.max_file_opening_threads: 16 2021-11-03T17:05:44.768 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.statistics: (nil) 2021-11-03T17:05:44.768 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.use_fsync: 0 2021-11-03T17:05:44.769 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.max_log_file_size: 0 2021-11-03T17:05:44.769 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.max_manifest_file_size: 1073741824 2021-11-03T17:05:44.769 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.log_file_time_to_roll: 0 2021-11-03T17:05:44.769 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.keep_log_file_num: 1000 2021-11-03T17:05:44.770 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.recycle_log_file_num: 0 2021-11-03T17:05:44.770 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.allow_fallocate: 1 2021-11-03T17:05:44.770 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.allow_mmap_reads: 0 2021-11-03T17:05:44.770 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.allow_mmap_writes: 0 2021-11-03T17:05:44.770 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.use_direct_reads: 0 2021-11-03T17:05:44.771 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2021-11-03T17:05:44.771 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.create_missing_column_families: 0 2021-11-03T17:05:44.771 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.db_log_dir: 2021-11-03T17:05:44.771 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi027/store.db 2021-11-03T17:05:44.771 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.table_cache_numshardbits: 6 2021-11-03T17:05:44.772 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.max_subcompactions: 1 2021-11-03T17:05:44.772 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.max_background_flushes: -1 2021-11-03T17:05:44.772 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.WAL_ttl_seconds: 0 2021-11-03T17:05:44.772 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.WAL_size_limit_MB: 0 2021-11-03T17:05:44.773 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2021-11-03T17:05:44.773 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.manifest_preallocation_size: 4194304 2021-11-03T17:05:44.773 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.is_fd_close_on_exec: 1 2021-11-03T17:05:44.773 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.advise_random_on_open: 1 2021-11-03T17:05:44.773 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.db_write_buffer_size: 0 2021-11-03T17:05:44.774 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.write_buffer_manager: 0x558840253530 2021-11-03T17:05:44.774 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.access_hint_on_compaction_start: 1 2021-11-03T17:05:44.774 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2021-11-03T17:05:44.774 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2021-11-03T17:05:44.774 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.use_adaptive_mutex: 0 2021-11-03T17:05:44.775 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.rate_limiter: (nil) 2021-11-03T17:05:44.775 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2021-11-03T17:05:44.775 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.wal_recovery_mode: 2 2021-11-03T17:05:44.775 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.enable_thread_tracking: 0 2021-11-03T17:05:44.776 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.enable_pipelined_write: 0 2021-11-03T17:05:44.776 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.unordered_write: 0 2021-11-03T17:05:44.776 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2021-11-03T17:05:44.776 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2021-11-03T17:05:44.776 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.write_thread_max_yield_usec: 100 2021-11-03T17:05:44.777 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2021-11-03T17:05:44.777 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.row_cache: None 2021-11-03T17:05:44.777 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.wal_filter: None 2021-11-03T17:05:44.777 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.avoid_flush_during_recovery: 0 2021-11-03T17:05:44.778 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.allow_ingest_behind: 0 2021-11-03T17:05:44.778 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.preserve_deletes: 0 2021-11-03T17:05:44.778 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.two_write_queues: 0 2021-11-03T17:05:44.778 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.manual_wal_flush: 0 2021-11-03T17:05:44.778 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.atomic_flush: 0 2021-11-03T17:05:44.779 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2021-11-03T17:05:44.779 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.persist_stats_to_disk: 0 2021-11-03T17:05:44.779 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.write_dbid_to_manifest: 0 2021-11-03T17:05:44.779 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.log_readahead_size: 0 2021-11-03T17:05:44.780 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.sst_file_checksum_func: Unknown 2021-11-03T17:05:44.780 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.max_background_jobs: 2 2021-11-03T17:05:44.780 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.max_background_compactions: -1 2021-11-03T17:05:44.780 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2021-11-03T17:05:44.780 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2021-11-03T17:05:44.781 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.delayed_write_rate : 16777216 2021-11-03T17:05:44.781 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.max_total_wal_size: 0 2021-11-03T17:05:44.781 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2021-11-03T17:05:44.781 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.stats_dump_period_sec: 600 2021-11-03T17:05:44.782 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.stats_persist_period_sec: 600 2021-11-03T17:05:44.782 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.stats_history_buffer_size: 1048576 2021-11-03T17:05:44.782 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.max_open_files: -1 2021-11-03T17:05:44.782 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.bytes_per_sync: 0 2021-11-03T17:05:44.782 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.wal_bytes_per_sync: 0 2021-11-03T17:05:44.783 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.strict_bytes_per_sync: 0 2021-11-03T17:05:44.783 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Options.compaction_readahead_size: 0 2021-11-03T17:05:44.783 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Compression algorithms supported: 2021-11-03T17:05:44.783 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: kZSTDNotFinalCompression supported: 0 2021-11-03T17:05:44.784 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: kZSTD supported: 0 2021-11-03T17:05:44.784 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: kXpressCompression supported: 0 2021-11-03T17:05:44.784 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: kLZ4HCCompression supported: 1 2021-11-03T17:05:44.784 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: kLZ4Compression supported: 1 2021-11-03T17:05:44.785 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: kBZip2Compression supported: 0 2021-11-03T17:05:44.785 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: kZlibCompression supported: 1 2021-11-03T17:05:44.785 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: kSnappyCompression supported: 1 2021-11-03T17:05:44.785 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: Fast CRC32 supported: Supported on x86 2021-11-03T17:05:44.785 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.728+0000 7f1af23cb700 4 rocksdb: [db_impl/db_impl_open.cc:273] Creating manifest 1 2021-11-03T17:05:44.786 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: 2021-11-03T17:05:44.786 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi027/store.db/MANIFEST-000001 2021-11-03T17:05:44.786 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: 2021-11-03T17:05:44.786 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2021-11-03T17:05:44.787 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: 2021-11-03T17:05:44.787 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2021-11-03T17:05:44.787 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.merge_operator: 2021-11-03T17:05:44.787 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.compaction_filter: None 2021-11-03T17:05:44.788 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.compaction_filter_factory: None 2021-11-03T17:05:44.788 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.memtable_factory: SkipListFactory 2021-11-03T17:05:44.788 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.table_factory: BlockBasedTable 2021-11-03T17:05:44.788 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55884015e120) 2021-11-03T17:05:44.788 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: cache_index_and_filter_blocks: 1 2021-11-03T17:05:44.789 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: cache_index_and_filter_blocks_with_high_priority: 0 2021-11-03T17:05:44.789 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: pin_l0_filter_and_index_blocks_in_cache: 0 2021-11-03T17:05:44.789 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: pin_top_level_index_and_filter: 1 2021-11-03T17:05:44.789 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: index_type: 0 2021-11-03T17:05:44.789 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: data_block_index_type: 0 2021-11-03T17:05:44.790 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: index_shortening: 1 2021-11-03T17:05:44.790 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: data_block_hash_table_util_ratio: 0.750000 2021-11-03T17:05:44.790 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: hash_index_allow_collision: 1 2021-11-03T17:05:44.790 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: checksum: 1 2021-11-03T17:05:44.791 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: no_block_cache: 0 2021-11-03T17:05:44.791 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: block_cache: 0x558840198d10 2021-11-03T17:05:44.791 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: block_cache_name: BinnedLRUCache 2021-11-03T17:05:44.791 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: block_cache_options: 2021-11-03T17:05:44.791 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: capacity : 536870912 2021-11-03T17:05:44.792 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: num_shard_bits : 4 2021-11-03T17:05:44.792 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: strict_capacity_limit : 0 2021-11-03T17:05:44.792 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: high_pri_pool_ratio: 0.000 2021-11-03T17:05:44.792 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: block_cache_compressed: (nil) 2021-11-03T17:05:44.793 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: persistent_cache: (nil) 2021-11-03T17:05:44.793 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: block_size: 4096 2021-11-03T17:05:44.793 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: block_size_deviation: 10 2021-11-03T17:05:44.793 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: block_restart_interval: 16 2021-11-03T17:05:44.793 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: index_block_restart_interval: 1 2021-11-03T17:05:44.794 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: metadata_block_size: 4096 2021-11-03T17:05:44.794 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: partition_filters: 0 2021-11-03T17:05:44.794 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: use_delta_encoding: 1 2021-11-03T17:05:44.794 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: filter_policy: rocksdb.BuiltinBloomFilter 2021-11-03T17:05:44.794 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: whole_key_filtering: 1 2021-11-03T17:05:44.795 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: verify_compression: 0 2021-11-03T17:05:44.795 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: read_amp_bytes_per_bit: 0 2021-11-03T17:05:44.795 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: format_version: 2 2021-11-03T17:05:44.795 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: enable_index_compression: 1 2021-11-03T17:05:44.796 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: block_align: 0 2021-11-03T17:05:44.796 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: 2021-11-03T17:05:44.796 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.write_buffer_size: 33554432 2021-11-03T17:05:44.796 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.max_write_buffer_number: 2 2021-11-03T17:05:44.796 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.compression: NoCompression 2021-11-03T17:05:44.797 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.bottommost_compression: Disabled 2021-11-03T17:05:44.797 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.prefix_extractor: nullptr 2021-11-03T17:05:44.797 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2021-11-03T17:05:44.797 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.num_levels: 7 2021-11-03T17:05:44.797 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2021-11-03T17:05:44.798 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2021-11-03T17:05:44.798 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2021-11-03T17:05:44.798 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2021-11-03T17:05:44.798 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2021-11-03T17:05:44.799 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2021-11-03T17:05:44.799 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2021-11-03T17:05:44.799 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2021-11-03T17:05:44.799 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2021-11-03T17:05:44.799 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.compression_opts.window_bits: -14 2021-11-03T17:05:44.800 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.compression_opts.level: 32767 2021-11-03T17:05:44.800 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.compression_opts.strategy: 0 2021-11-03T17:05:44.800 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2021-11-03T17:05:44.800 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2021-11-03T17:05:44.801 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.compression_opts.enabled: false 2021-11-03T17:05:44.801 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2021-11-03T17:05:44.801 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2021-11-03T17:05:44.801 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.level0_stop_writes_trigger: 36 2021-11-03T17:05:44.801 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.target_file_size_base: 67108864 2021-11-03T17:05:44.802 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.target_file_size_multiplier: 1 2021-11-03T17:05:44.802 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2021-11-03T17:05:44.802 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2021-11-03T17:05:44.802 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2021-11-03T17:05:44.803 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2021-11-03T17:05:44.803 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2021-11-03T17:05:44.803 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2021-11-03T17:05:44.803 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2021-11-03T17:05:44.803 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2021-11-03T17:05:44.804 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2021-11-03T17:05:44.804 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2021-11-03T17:05:44.804 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2021-11-03T17:05:44.804 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.max_compaction_bytes: 1677721600 2021-11-03T17:05:44.804 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.arena_block_size: 4194304 2021-11-03T17:05:44.805 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2021-11-03T17:05:44.805 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2021-11-03T17:05:44.805 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2021-11-03T17:05:44.805 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.disable_auto_compactions: 0 2021-11-03T17:05:44.806 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2021-11-03T17:05:44.806 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2021-11-03T17:05:44.806 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2021-11-03T17:05:44.806 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2021-11-03T17:05:44.806 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2021-11-03T17:05:44.807 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2021-11-03T17:05:44.807 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2021-11-03T17:05:44.807 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2021-11-03T17:05:44.807 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2021-11-03T17:05:44.808 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2021-11-03T17:05:44.808 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.table_properties_collectors: 2021-11-03T17:05:44.808 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.inplace_update_support: 0 2021-11-03T17:05:44.808 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.inplace_update_num_locks: 10000 2021-11-03T17:05:44.808 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2021-11-03T17:05:44.809 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.memtable_whole_key_filtering: 0 2021-11-03T17:05:44.809 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.memtable_huge_page_size: 0 2021-11-03T17:05:44.809 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.bloom_locality: 0 2021-11-03T17:05:44.809 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.max_successive_merges: 0 2021-11-03T17:05:44.810 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.optimize_filters_for_hits: 0 2021-11-03T17:05:44.810 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.paranoid_file_checks: 0 2021-11-03T17:05:44.810 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.force_consistency_checks: 0 2021-11-03T17:05:44.810 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.report_bg_io_stats: 0 2021-11-03T17:05:44.810 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.ttl: 2592000 2021-11-03T17:05:44.811 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: Options.periodic_compaction_seconds: 0 2021-11-03T17:05:44.811 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi027/store.db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2021-11-03T17:05:44.811 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: 2021-11-03T17:05:44.811 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.731+0000 7f1af23cb700 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 0 2021-11-03T17:05:44.812 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: 2021-11-03T17:05:44.812 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.732+0000 7f1af23cb700 4 rocksdb: DB pointer 0x55884025f800 2021-11-03T17:05:44.812 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.732+0000 7f1adb264700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2021-11-03T17:05:44.812 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.732+0000 7f1adb264700 4 rocksdb: [db_impl/db_impl.cc:851] 2021-11-03T17:05:44.812 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: ** DB Stats ** 2021-11-03T17:05:44.813 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: Uptime(secs): 0.0 total, 0.0 interval 2021-11-03T17:05:44.813 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2021-11-03T17:05:44.813 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2021-11-03T17:05:44.813 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2021-11-03T17:05:44.814 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2021-11-03T17:05:44.814 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2021-11-03T17:05:44.814 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2021-11-03T17:05:44.814 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: 2021-11-03T17:05:44.814 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2021-11-03T17:05:44.815 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-11-03T17:05:44.815 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-11-03T17:05:44.815 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2021-11-03T17:05:44.815 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2021-11-03T17:05:44.816 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: 2021-11-03T17:05:44.816 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2021-11-03T17:05:44.816 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-11-03T17:05:44.816 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-11-03T17:05:44.816 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: Uptime(secs): 0.0 total, 0.0 interval 2021-11-03T17:05:44.817 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: Flush(GB): cumulative 0.000, interval 0.000 2021-11-03T17:05:44.817 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: AddFile(GB): cumulative 0.000, interval 0.000 2021-11-03T17:05:44.817 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: AddFile(Total Files): cumulative 0, interval 0 2021-11-03T17:05:44.817 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: AddFile(L0 Files): cumulative 0, interval 0 2021-11-03T17:05:44.817 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: AddFile(Keys): cumulative 0, interval 0 2021-11-03T17:05:44.818 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-11-03T17:05:44.818 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-11-03T17:05:44.818 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2021-11-03T17:05:44.818 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: 2021-11-03T17:05:44.819 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: ** File Read Latency Histogram By Level [default] ** 2021-11-03T17:05:44.819 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: 2021-11-03T17:05:44.819 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2021-11-03T17:05:44.819 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-11-03T17:05:44.819 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-11-03T17:05:44.820 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2021-11-03T17:05:44.820 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2021-11-03T17:05:44.820 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: 2021-11-03T17:05:44.820 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: ** Compaction Stats [default] ** 2021-11-03T17:05:44.821 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-11-03T17:05:44.821 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-11-03T17:05:44.821 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: Uptime(secs): 0.0 total, 0.0 interval 2021-11-03T17:05:44.821 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: Flush(GB): cumulative 0.000, interval 0.000 2021-11-03T17:05:44.821 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: AddFile(GB): cumulative 0.000, interval 0.000 2021-11-03T17:05:44.822 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: AddFile(Total Files): cumulative 0, interval 0 2021-11-03T17:05:44.822 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: AddFile(L0 Files): cumulative 0, interval 0 2021-11-03T17:05:44.822 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: AddFile(Keys): cumulative 0, interval 0 2021-11-03T17:05:44.822 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-11-03T17:05:44.823 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-11-03T17:05:44.823 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2021-11-03T17:05:44.823 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: 2021-11-03T17:05:44.823 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: ** File Read Latency Histogram By Level [default] ** 2021-11-03T17:05:44.823 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: 2021-11-03T17:05:44.824 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.734+0000 7f1af23cb700 4 rocksdb: [db_impl/db_impl.cc:397] Shutdown: canceling all background work 2021-11-03T17:05:44.824 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.734+0000 7f1af23cb700 4 rocksdb: [db_impl/db_impl.cc:573] Shutdown complete 2021-11-03T17:05:44.824 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph-mon: debug 2021-11-03T17:05:44.734+0000 7f1af23cb700 0 /usr/bin/ceph-mon: created monfs at /var/lib/ceph/mon/ceph-smithi027 for mon.smithi027 2021-11-03T17:05:45.321 INFO:teuthology.orchestra.run.smithi027.stderr:create mon.smithi027 on 2021-11-03T17:05:45.322 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: install -d -m0770 -o 167 -g 167 /var/run/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c 2021-11-03T17:05:45.328 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: systemctl disable ceph.target 2021-11-03T17:05:45.336 INFO:teuthology.orchestra.run.smithi027.stderr:systemctl: Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2021-11-03T17:05:45.423 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: systemctl enable ceph.target 2021-11-03T17:05:45.429 INFO:teuthology.orchestra.run.smithi027.stderr:systemctl: Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2021-11-03T17:05:45.512 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: systemctl start ceph.target 2021-11-03T17:05:45.520 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: systemctl enable ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c.target 2021-11-03T17:05:45.526 INFO:teuthology.orchestra.run.smithi027.stderr:systemctl: Created symlink /etc/systemd/system/multi-user.target.wants/ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c.target → /etc/systemd/system/ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c.target. 2021-11-03T17:05:45.526 INFO:teuthology.orchestra.run.smithi027.stderr:systemctl: Created symlink /etc/systemd/system/ceph.target.wants/ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c.target → /etc/systemd/system/ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c.target. 2021-11-03T17:05:45.603 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: systemctl start ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c.target 2021-11-03T17:05:45.610 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: systemctl daemon-reload 2021-11-03T17:05:45.695 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: systemctl stop ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c@mon.smithi027 2021-11-03T17:05:45.719 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: systemctl reset-failed ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c@mon.smithi027 2021-11-03T17:05:45.725 INFO:teuthology.orchestra.run.smithi027.stderr:systemctl: Failed to reset failed state of unit ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c@mon.smithi027.service: Unit ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c@mon.smithi027.service not loaded. 2021-11-03T17:05:45.726 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: systemctl enable ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c@mon.smithi027 2021-11-03T17:05:45.732 INFO:teuthology.orchestra.run.smithi027.stderr:systemctl: Created symlink /etc/systemd/system/ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c.target.wants/ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c@mon.smithi027.service → /etc/systemd/system/ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c@.service. 2021-11-03T17:05:45.819 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: systemctl start ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c@mon.smithi027 2021-11-03T17:05:46.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:45 smithi027 systemd[1]: Starting Ceph mon.smithi027 for 32ab977e-3cc8-11ec-8c28-001a4aab830c... 2021-11-03T17:05:46.284 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: systemctl is-enabled firewalld.service 2021-11-03T17:05:46.292 INFO:teuthology.orchestra.run.smithi027.stderr:systemctl: disabled 2021-11-03T17:05:46.293 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: systemctl is-active firewalld.service 2021-11-03T17:05:46.300 INFO:teuthology.orchestra.run.smithi027.stderr:systemctl: inactive 2021-11-03T17:05:46.301 INFO:teuthology.orchestra.run.smithi027.stderr:firewalld.service is not enabled 2021-11-03T17:05:46.302 INFO:teuthology.orchestra.run.smithi027.stderr:Not possible to enable service . firewalld.service is not available 2021-11-03T17:05:46.302 INFO:teuthology.orchestra.run.smithi027.stderr:Not possible to open ports <[]>. firewalld.service is not available 2021-11-03T17:05:46.303 INFO:teuthology.orchestra.run.smithi027.stderr:Waiting for mon to start... 2021-11-03T17:05:46.303 INFO:teuthology.orchestra.run.smithi027.stderr:Waiting for mon... 2021-11-03T17:05:46.303 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi027 -e CEPH_USE_RANDOM_NONCE=1 -v /var/lib/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c/mon.smithi027:/var/lib/ceph/mon/ceph-smithi027:z -v /tmp/ceph-tmp3f8lj10x:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpq2rb3_h7:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 status 2021-11-03T17:05:46.537 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:46 smithi027 bash[31480]: 9d6e3ddb553ff72c702eaa115cd2113d2716b4ea69931a737b92fcd8f4010a9d 2021-11-03T17:05:46.537 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:46 smithi027 systemd[1]: Started Ceph mon.smithi027 for 32ab977e-3cc8-11ec-8c28-001a4aab830c. 2021-11-03T17:05:46.537 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:46 smithi027 conmon[31596]: cluster 2021-11-03T17:05:46.314324+0000 mon.smithi027 (mon.0) 0 : [INF] mkfs 32ab977e-3cc8-11ec-8c28-001a4aab830c 2021-11-03T17:05:46.538 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:46 smithi027 conmon[31596]: cluster 2021-11-03T17: 2021-11-03T17:05:46.538 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:46 smithi027 conmon[31596]: 05:46.297264+0000 mon.smithi027 (mon.0) 1 : cluster [INF] mon.smithi027 is new leader, mons smithi027 in quorum (ranks 0) 2021-11-03T17:05:46.961 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:46.963+0000 7f2b2cd8e700 1 Processor -- start 2021-11-03T17:05:46.962 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:46.963+0000 7f2b2cd8e700 1 -- start start 2021-11-03T17:05:46.962 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:46.963+0000 7f2b2cd8e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b280fd090 0x7f2b280fd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-03T17:05:46.962 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:46.963+0000 7f2b2cd8e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b280fda30 con 0x7f2b280fd090 2021-11-03T17:05:46.962 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:46.964+0000 7f2b277fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b280fd090 0x7f2b280fd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:05:46.963 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:46.964+0000 7f2b277fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b280fd090 0x7f2b280fd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53944/0 (socket says 172.21.15.27:53944) 2021-11-03T17:05:46.963 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:46.964+0000 7f2b277fe700 1 -- 172.21.15.27:0/3972061729 learned_addr learned my addr 172.21.15.27:0/3972061729 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:05:46.963 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:46.964+0000 7f2b277fe700 1 -- 172.21.15.27:0/3972061729 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b280fdb70 con 0x7f2b280fd090 2021-11-03T17:05:46.963 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:46.964+0000 7f2b277fe700 1 --2- 172.21.15.27:0/3972061729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b280fd090 0x7f2b280fd4b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7f2b1800b5f0 tx=0x7f2b18006ee0).ready entity=mon.0 client_cookie=8f364851acb4fa80 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:05:46.964 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:46.964+0000 7f2b267fc700 1 -- 172.21.15.27:0/3972061729 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2b1800d600 con 0x7f2b280fd090 2021-11-03T17:05:46.964 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:46.964+0000 7f2b267fc700 1 -- 172.21.15.27:0/3972061729 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b1800d760 con 0x7f2b280fd090 2021-11-03T17:05:46.964 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:46.965+0000 7f2b267fc700 1 -- 172.21.15.27:0/3972061729 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2b1800da30 con 0x7f2b280fd090 2021-11-03T17:05:46.964 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:46.965+0000 7f2b2cd8e700 1 -- 172.21.15.27:0/3972061729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b280fd090 msgr2=0x7f2b280fd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:05:46.964 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:46.965+0000 7f2b2cd8e700 1 --2- 172.21.15.27:0/3972061729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b280fd090 0x7f2b280fd4b0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7f2b1800b5f0 tx=0x7f2b18006ee0).stop 2021-11-03T17:05:46.965 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:46.965+0000 7f2b2cd8e700 1 -- 172.21.15.27:0/3972061729 shutdown_connections 2021-11-03T17:05:46.965 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:46.965+0000 7f2b2cd8e700 1 --2- 172.21.15.27:0/3972061729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b280fd090 0x7f2b280fd4b0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:05:46.965 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:46.965+0000 7f2b2cd8e700 1 -- 172.21.15.27:0/3972061729 >> 172.21.15.27:0/3972061729 conn(0x7f2b280f5850 msgr2=0x7f2b280f7c70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:05:46.965 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:46.965+0000 7f2b2cd8e700 1 -- 172.21.15.27:0/3972061729 shutdown_connections 2021-11-03T17:05:46.966 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:46.965+0000 7f2b2cd8e700 1 -- 172.21.15.27:0/3972061729 wait complete. 2021-11-03T17:05:46.966 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:46.966+0000 7f2b2cd8e700 1 Processor -- start 2021-11-03T17:05:46.966 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:46.966+0000 7f2b2cd8e700 1 -- start start 2021-11-03T17:05:46.966 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:46.966+0000 7f2b2cd8e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b280fd090 0x7f2b2810a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:05:46.967 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:46.966+0000 7f2b2cd8e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2810a530 con 0x7f2b280fd090 2021-11-03T17:05:46.967 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:46.966+0000 7f2b277fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b280fd090 0x7f2b2810a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:05:46.967 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:46.966+0000 7f2b277fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b280fd090 0x7f2b2810a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53946/0 (socket says 172.21.15.27:53946) 2021-11-03T17:05:46.967 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:46.966+0000 7f2b277fe700 1 -- 172.21.15.27:0/1067695527 learned_addr learned my addr 172.21.15.27:0/1067695527 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:05:46.968 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:46.966+0000 7f2b277fe700 1 -- 172.21.15.27:0/1067695527 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b1800b200 con 0x7f2b280fd090 2021-11-03T17:05:46.968 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:46.967+0000 7f2b277fe700 1 --2- 172.21.15.27:0/1067695527 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b280fd090 0x7f2b2810a040 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7f2b18006c70 tx=0x7f2b18016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:05:46.969 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:46.967+0000 7f2b0ffff700 1 -- 172.21.15.27:0/1067695527 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2b18011b40 con 0x7f2b280fd090 2021-11-03T17:05:46.970 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:46.967+0000 7f2b0ffff700 1 -- 172.21.15.27:0/1067695527 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b18011ca0 con 0x7f2b280fd090 2021-11-03T17:05:46.970 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:46.967+0000 7f2b2cd8e700 1 -- 172.21.15.27:0/1067695527 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b2810a730 con 0x7f2b280fd090 2021-11-03T17:05:46.970 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:46.967+0000 7f2b0ffff700 1 -- 172.21.15.27:0/1067695527 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2b1801d430 con 0x7f2b280fd090 2021-11-03T17:05:46.970 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:46.967+0000 7f2b2cd8e700 1 -- 172.21.15.27:0/1067695527 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b2810abc0 con 0x7f2b280fd090 2021-11-03T17:05:46.971 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:46.967+0000 7f2b0ffff700 1 -- 172.21.15.27:0/1067695527 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7f2b1800f070 con 0x7f2b280fd090 2021-11-03T17:05:46.971 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:46.967+0000 7f2b0ffff700 1 -- 172.21.15.27:0/1067695527 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f2b1801dc80 con 0x7f2b280fd090 2021-11-03T17:05:46.971 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:46.968+0000 7f2b2cd8e700 1 -- 172.21.15.27:0/1067695527 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b28045ad0 con 0x7f2b280fd090 2021-11-03T17:05:46.971 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:46.970+0000 7f2b0ffff700 1 -- 172.21.15.27:0/1067695527 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72097 (secure 0 0 0) 0x7f2b18016070 con 0x7f2b280fd090 2021-11-03T17:05:47.093 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:47.095+0000 7f2b2cd8e700 1 -- 172.21.15.27:0/1067695527 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f2b280008d0 con 0x7f2b280fd090 2021-11-03T17:05:47.093 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:47.095+0000 7f2b0ffff700 1 -- 172.21.15.27:0/1067695527 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7f2b18036070 con 0x7f2b280fd090 2021-11-03T17:05:47.093 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: cluster: 2021-11-03T17:05:47.094 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: id: 32ab977e-3cc8-11ec-8c28-001a4aab830c 2021-11-03T17:05:47.094 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: health: HEALTH_OK 2021-11-03T17:05:47.094 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:47.095 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: services: 2021-11-03T17:05:47.095 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: mon: 1 daemons, quorum smithi027 (age 0.787021s) 2021-11-03T17:05:47.095 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: mgr: no daemons active 2021-11-03T17:05:47.095 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: osd: 0 osds: 0 up, 0 in 2021-11-03T17:05:47.096 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:47.096 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: data: 2021-11-03T17:05:47.096 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: pools: 0 pools, 0 pgs 2021-11-03T17:05:47.096 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: objects: 0 objects, 0 B 2021-11-03T17:05:47.097 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: usage: 0 B used, 0 B / 0 B avail 2021-11-03T17:05:47.097 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: pgs: 2021-11-03T17:05:47.097 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:47.097 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:47.096+0000 7f2b2cd8e700 1 -- 172.21.15.27:0/1067695527 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b280fd090 msgr2=0x7f2b2810a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:05:47.097 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:47.096+0000 7f2b2cd8e700 1 --2- 172.21.15.27:0/1067695527 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b280fd090 0x7f2b2810a040 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7f2b18006c70 tx=0x7f2b18016000).stop 2021-11-03T17:05:47.098 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:47.096+0000 7f2b2cd8e700 1 -- 172.21.15.27:0/1067695527 shutdown_connections 2021-11-03T17:05:47.098 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:47.096+0000 7f2b2cd8e700 1 --2- 172.21.15.27:0/1067695527 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b280fd090 0x7f2b2810a040 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:05:47.098 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:47.096+0000 7f2b2cd8e700 1 -- 172.21.15.27:0/1067695527 >> 172.21.15.27:0/1067695527 conn(0x7f2b280f5850 msgr2=0x7f2b280fe940 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:05:47.098 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:47.096+0000 7f2b2cd8e700 1 -- 172.21.15.27:0/1067695527 shutdown_connections 2021-11-03T17:05:47.099 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:47.096+0000 7f2b2cd8e700 1 -- 172.21.15.27:0/1067695527 wait complete. 2021-11-03T17:05:47.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:47 smithi027 conmon[31596]: cluster 2021-11-03T17:05:46.309075+0000 2021-11-03T17:05:47.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:47 smithi027 conmon[31596]: mon.smithi027 (mon.0) 2 : cluster [INF] mon.smithi027 is new leader, mons smithi027 in quorum (ranks 0) 2021-11-03T17:05:47.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:47 smithi027 conmon[31596]: cluster 2021-11-03T17:05:46.311315+0000 mon.smithi027 (mon.0) 3 : cluster [DBG] monmap e1: 1 mons at {smithi027=[v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0]} 2021-11-03T17:05:47.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:47 smithi027 conmon[31596]: cluster 2021-11-03T17:05:46.320088+0000 mon.smithi027 (mon.0) 4 : cluster [DBG] fsmap 2021-11-03T17:05:47.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:47 smithi027 conmon[31596]: cluster 2021-11-03T17:05:46.324756+0000 mon.smithi027 (mon.0) 5 : cluster [DBG] osdmap e1: 0 total, 0 up, 0 in 2021-11-03T17:05:47.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:47 smithi027 conmon[31596]: cluster 2021-11-03T17:05:46.325316+0000 mon.smithi027 (mon.0) 6 : cluster [DBG] mgrmap e1: no daemons active 2021-11-03T17:05:47.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:47 smithi027 conmon[31596]: audit 2021-11-03T17:05:47.096063+0000 mon.smithi027 (mon.0) 7 : audit [DBG] from='client.? 172.21.15.27:0/1067695527' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2021-11-03T17:05:47.804 INFO:teuthology.orchestra.run.smithi027.stderr:mon is available 2021-11-03T17:05:47.804 INFO:teuthology.orchestra.run.smithi027.stderr:Assimilating anything we can from ceph.conf... 2021-11-03T17:05:47.804 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi027 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmp3f8lj10x:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpq2rb3_h7:/etc/ceph/ceph.conf:z -v /var/lib/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c/mon.smithi027:/var/lib/ceph/mon/ceph-smithi027:z docker.io/ceph/ceph:v16.2.4 config assimilate-conf -i /var/lib/ceph/mon/ceph-smithi027/config 2021-11-03T17:05:48.462 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.461+0000 7f0cbb7b7700 1 Processor -- start 2021-11-03T17:05:48.463 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.461+0000 7f0cbb7b7700 1 -- start start 2021-11-03T17:05:48.464 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.461+0000 7f0cbb7b7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0cb406fb80 0x7f0cb40fe850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-03T17:05:48.464 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.461+0000 7f0cbb7b7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb40fed40 con 0x7f0cb406fb80 2021-11-03T17:05:48.464 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.461+0000 7f0cb9553700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0cb406fb80 0x7f0cb40fe850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:05:48.465 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.462+0000 7f0cb9553700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0cb406fb80 0x7f0cb40fe850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53948/0 (socket says 172.21.15.27:53948) 2021-11-03T17:05:48.465 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.462+0000 7f0cb9553700 1 -- 172.21.15.27:0/1905989629 learned_addr learned my addr 172.21.15.27:0/1905989629 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:05:48.466 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.462+0000 7f0cb9553700 1 -- 172.21.15.27:0/1905989629 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb40fee80 con 0x7f0cb406fb80 2021-11-03T17:05:48.466 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.462+0000 7f0cb9553700 1 --2- 172.21.15.27:0/1905989629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0cb406fb80 0x7f0cb40fe850 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7f0cb000b5f0 tx=0x7f0cb0006ee0).ready entity=mon.0 client_cookie=c7ae60c4f3f02542 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:05:48.467 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.462+0000 7f0ca3fff700 1 -- 172.21.15.27:0/1905989629 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0cb000d600 con 0x7f0cb406fb80 2021-11-03T17:05:48.467 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.462+0000 7f0ca3fff700 1 -- 172.21.15.27:0/1905989629 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cb000d760 con 0x7f0cb406fb80 2021-11-03T17:05:48.467 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.462+0000 7f0ca3fff700 1 -- 172.21.15.27:0/1905989629 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0cb000da30 con 0x7f0cb406fb80 2021-11-03T17:05:48.468 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.463+0000 7f0cbb7b7700 1 -- 172.21.15.27:0/1905989629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0cb406fb80 msgr2=0x7f0cb40fe850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:05:48.468 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.463+0000 7f0cbb7b7700 1 --2- 172.21.15.27:0/1905989629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0cb406fb80 0x7f0cb40fe850 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7f0cb000b5f0 tx=0x7f0cb0006ee0).stop 2021-11-03T17:05:48.469 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.463+0000 7f0cbb7b7700 1 -- 172.21.15.27:0/1905989629 shutdown_connections 2021-11-03T17:05:48.469 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.463+0000 7f0cbb7b7700 1 --2- 172.21.15.27:0/1905989629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0cb406fb80 0x7f0cb40fe850 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:05:48.470 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.463+0000 7f0cbb7b7700 1 -- 172.21.15.27:0/1905989629 >> 172.21.15.27:0/1905989629 conn(0x7f0cb40f62f0 msgr2=0x7f0cb40f8710 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:05:48.470 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.463+0000 7f0cbb7b7700 1 -- 172.21.15.27:0/1905989629 shutdown_connections 2021-11-03T17:05:48.470 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.463+0000 7f0cbb7b7700 1 -- 172.21.15.27:0/1905989629 wait complete. 2021-11-03T17:05:48.471 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.463+0000 7f0cbb7b7700 1 Processor -- start 2021-11-03T17:05:48.471 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.463+0000 7f0cbb7b7700 1 -- start start 2021-11-03T17:05:48.472 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.464+0000 7f0cbb7b7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0cb406fb80 0x7f0cb410a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:05:48.472 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.464+0000 7f0cbb7b7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb410a6e0 con 0x7f0cb406fb80 2021-11-03T17:05:48.473 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.464+0000 7f0cb9553700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0cb406fb80 0x7f0cb410a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:05:48.473 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.464+0000 7f0cb9553700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0cb406fb80 0x7f0cb410a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53950/0 (socket says 172.21.15.27:53950) 2021-11-03T17:05:48.473 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.464+0000 7f0cb9553700 1 -- 172.21.15.27:0/2453419190 learned_addr learned my addr 172.21.15.27:0/2453419190 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:05:48.474 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.464+0000 7f0cb9553700 1 -- 172.21.15.27:0/2453419190 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb000b200 con 0x7f0cb406fb80 2021-11-03T17:05:48.474 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.464+0000 7f0cb9553700 1 --2- 172.21.15.27:0/2453419190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0cb406fb80 0x7f0cb410a1f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7f0cb0006c70 tx=0x7f0cb0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:05:48.475 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.464+0000 7f0ca1ffb700 1 -- 172.21.15.27:0/2453419190 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0cb0011c80 con 0x7f0cb406fb80 2021-11-03T17:05:48.475 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.464+0000 7f0ca1ffb700 1 -- 172.21.15.27:0/2453419190 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cb0011de0 con 0x7f0cb406fb80 2021-11-03T17:05:48.476 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.464+0000 7f0ca1ffb700 1 -- 172.21.15.27:0/2453419190 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0cb001d5d0 con 0x7f0cb406fb80 2021-11-03T17:05:48.476 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.464+0000 7f0cbb7b7700 1 -- 172.21.15.27:0/2453419190 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cb410a8e0 con 0x7f0cb406fb80 2021-11-03T17:05:48.477 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.465+0000 7f0cbb7b7700 1 -- 172.21.15.27:0/2453419190 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cb410ad70 con 0x7f0cb406fb80 2021-11-03T17:05:48.477 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.465+0000 7f0ca1ffb700 1 -- 172.21.15.27:0/2453419190 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7f0cb000f070 con 0x7f0cb406fb80 2021-11-03T17:05:48.477 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.465+0000 7f0ca1ffb700 1 -- 172.21.15.27:0/2453419190 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f0cb001dbc0 con 0x7f0cb406fb80 2021-11-03T17:05:48.478 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.466+0000 7f0cbb7b7700 1 -- 172.21.15.27:0/2453419190 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cb4045ad0 con 0x7f0cb406fb80 2021-11-03T17:05:48.479 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.468+0000 7f0ca1ffb700 1 -- 172.21.15.27:0/2453419190 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72097 (secure 0 0 0) 0x7f0cb0016070 con 0x7f0cb406fb80 2021-11-03T17:05:48.581 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.586+0000 7f0cbb7b7700 1 -- 172.21.15.27:0/2453419190 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f0cb4103e60 con 0x7f0cb406fb80 2021-11-03T17:05:48.586 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.590+0000 7f0ca1ffb700 1 -- 172.21.15.27:0/2453419190 <== mon.0 v2:172.21.15.27:3300/0 7 ==== config(25 keys) v1 ==== 983+0+0 (secure 0 0 0) 0x7f0cb0031880 con 0x7f0cb406fb80 2021-11-03T17:05:48.586 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.590+0000 7f0ca1ffb700 1 -- 172.21.15.27:0/2453419190 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+489 (secure 0 0 0) 0x7f0cb0036400 con 0x7f0cb406fb80 2021-11-03T17:05:48.586 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.587 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: [global] 2021-11-03T17:05:48.587 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: fsid = 32ab977e-3cc8-11ec-8c28-001a4aab830c 2021-11-03T17:05:48.588 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: log_to_journald = false 2021-11-03T17:05:48.588 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: mon_host = [v2:172.21.15.27:3300,v1:172.21.15.27:6789] 2021-11-03T17:05:48.588 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: mon_osd_allow_pg_remap = true 2021-11-03T17:05:48.588 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: mon_osd_allow_primary_affinity = true 2021-11-03T17:05:48.589 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: mon_warn_on_no_sortbitwise = false 2021-11-03T17:05:48.589 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: osd_crush_chooseleaf_type = 0 2021-11-03T17:05:48.591 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.592 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: [mgr] 2021-11-03T17:05:48.592 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: mgr/telemetry/nag = false 2021-11-03T17:05:48.592 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.592 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: [osd] 2021-11-03T17:05:48.593 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: mon_osd_backfillfull_ratio = 0.85 2021-11-03T17:05:48.593 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: mon_osd_full_ratio = 0.9 2021-11-03T17:05:48.593 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: mon_osd_nearfull_ratio = 0.8 2021-11-03T17:05:48.594 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: osd_map_max_advance = 10 2021-11-03T17:05:48.594 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: osd_memory_target_autotune = true 2021-11-03T17:05:48.594 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: osd_sloppy_crc = true 2021-11-03T17:05:48.595 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.591+0000 7f0cbb7b7700 1 -- 172.21.15.27:0/2453419190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0cb406fb80 msgr2=0x7f0cb410a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:05:48.596 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.591+0000 7f0cbb7b7700 1 --2- 172.21.15.27:0/2453419190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0cb406fb80 0x7f0cb410a1f0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7f0cb0006c70 tx=0x7f0cb0016000).stop 2021-11-03T17:05:48.596 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.591+0000 7f0cbb7b7700 1 -- 172.21.15.27:0/2453419190 shutdown_connections 2021-11-03T17:05:48.596 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.591+0000 7f0cbb7b7700 1 --2- 172.21.15.27:0/2453419190 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0cb406fb80 0x7f0cb410a1f0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:05:48.596 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.591+0000 7f0cbb7b7700 1 -- 172.21.15.27:0/2453419190 >> 172.21.15.27:0/2453419190 conn(0x7f0cb40f62f0 msgr2=0x7f0cb40f8090 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:05:48.597 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.591+0000 7f0cbb7b7700 1 -- 172.21.15.27:0/2453419190 shutdown_connections 2021-11-03T17:05:48.597 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:48.591+0000 7f0cbb7b7700 1 -- 172.21.15.27:0/2453419190 wait complete. 2021-11-03T17:05:49.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:48 smithi027 conmon[31596]: audit 2021-11-03T17:05:48.587546 2021-11-03T17:05:49.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:48 smithi027 conmon[31596]: +0000 mon.smithi027 (mon.0) 8 : audit [INF] from='client.? 172.21.15.27:0/2453419190' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2021-11-03T17:05:49.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:48 smithi027 conmon[31596]: audit 2021-11-03T17:05:48.590598+0000 mon.smithi027 (mon.0) 9 : audit [INF] from='client.? 172.21.15.27:0/2453419190' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2021-11-03T17:05:49.087 INFO:teuthology.orchestra.run.smithi027.stderr:Generating new minimal ceph.conf... 2021-11-03T17:05:49.087 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi027 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmp3f8lj10x:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpq2rb3_h7:/etc/ceph/ceph.conf:z -v /var/lib/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c/mon.smithi027:/var/lib/ceph/mon/ceph-smithi027:z docker.io/ceph/ceph:v16.2.4 config generate-minimal-conf -o /var/lib/ceph/mon/ceph-smithi027/config 2021-11-03T17:05:49.711 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.712+0000 7f9705be9700 1 Processor -- start 2021-11-03T17:05:49.712 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.712+0000 7f9705be9700 1 -- start start 2021-11-03T17:05:49.712 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.712+0000 7f9705be9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97000fc080 0x7f97000fc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-03T17:05:49.712 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.712+0000 7f9705be9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97000fc990 con 0x7f97000fc080 2021-11-03T17:05:49.713 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.712+0000 7f96ff7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97000fc080 0x7f97000fc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:05:49.713 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.712+0000 7f96ff7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97000fc080 0x7f97000fc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53952/0 (socket says 172.21.15.27:53952) 2021-11-03T17:05:49.713 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.712+0000 7f96ff7fe700 1 -- 172.21.15.27:0/32762203 learned_addr learned my addr 172.21.15.27:0/32762203 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:05:49.713 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.713+0000 7f96ff7fe700 1 -- 172.21.15.27:0/32762203 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97000fcad0 con 0x7f97000fc080 2021-11-03T17:05:49.714 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.713+0000 7f96ff7fe700 1 --2- 172.21.15.27:0/32762203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97000fc080 0x7f97000fc4a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f96f000b5f0 tx=0x7f96f0006ee0).ready entity=mon.0 client_cookie=2a33549af7227464 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:05:49.714 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.713+0000 7f96fe7fc700 1 -- 172.21.15.27:0/32762203 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f96f000d600 con 0x7f97000fc080 2021-11-03T17:05:49.714 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.713+0000 7f96fe7fc700 1 -- 172.21.15.27:0/32762203 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 983+0+0 (secure 0 0 0) 0x7f96f000dbb0 con 0x7f97000fc080 2021-11-03T17:05:49.715 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.713+0000 7f96fe7fc700 1 -- 172.21.15.27:0/32762203 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f96f0011bb0 con 0x7f97000fc080 2021-11-03T17:05:49.715 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.713+0000 7f9705be9700 1 -- 172.21.15.27:0/32762203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97000fc080 msgr2=0x7f97000fc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:05:49.715 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.713+0000 7f9705be9700 1 --2- 172.21.15.27:0/32762203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97000fc080 0x7f97000fc4a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f96f000b5f0 tx=0x7f96f0006ee0).stop 2021-11-03T17:05:49.715 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.713+0000 7f9705be9700 1 -- 172.21.15.27:0/32762203 shutdown_connections 2021-11-03T17:05:49.716 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.713+0000 7f9705be9700 1 --2- 172.21.15.27:0/32762203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97000fc080 0x7f97000fc4a0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:05:49.716 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.713+0000 7f9705be9700 1 -- 172.21.15.27:0/32762203 >> 172.21.15.27:0/32762203 conn(0x7f97000f5a60 msgr2=0x7f97000f7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:05:49.716 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.714+0000 7f9705be9700 1 -- 172.21.15.27:0/32762203 shutdown_connections 2021-11-03T17:05:49.716 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.714+0000 7f9705be9700 1 -- 172.21.15.27:0/32762203 wait complete. 2021-11-03T17:05:49.717 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.714+0000 7f9705be9700 1 Processor -- start 2021-11-03T17:05:49.717 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.714+0000 7f9705be9700 1 -- start start 2021-11-03T17:05:49.717 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.714+0000 7f9705be9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97000fc080 0x7f97001086c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:05:49.717 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.714+0000 7f9705be9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9700108bb0 con 0x7f97000fc080 2021-11-03T17:05:49.718 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.715+0000 7f96ff7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97000fc080 0x7f97001086c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:05:49.718 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.715+0000 7f96ff7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97000fc080 0x7f97001086c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53954/0 (socket says 172.21.15.27:53954) 2021-11-03T17:05:49.718 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.715+0000 7f96ff7fe700 1 -- 172.21.15.27:0/853798414 learned_addr learned my addr 172.21.15.27:0/853798414 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:05:49.718 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.715+0000 7f96ff7fe700 1 -- 172.21.15.27:0/853798414 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f000b200 con 0x7f97000fc080 2021-11-03T17:05:49.719 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.715+0000 7f96ff7fe700 1 --2- 172.21.15.27:0/853798414 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97000fc080 0x7f97001086c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f96f0006c70 tx=0x7f96f0006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:05:49.719 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.715+0000 7f96e7fff700 1 -- 172.21.15.27:0/853798414 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f96f001b410 con 0x7f97000fc080 2021-11-03T17:05:49.719 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.715+0000 7f96e7fff700 1 -- 172.21.15.27:0/853798414 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 983+0+0 (secure 0 0 0) 0x7f96f001b9c0 con 0x7f97000fc080 2021-11-03T17:05:49.719 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.715+0000 7f9705be9700 1 -- 172.21.15.27:0/853798414 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9700108db0 con 0x7f97000fc080 2021-11-03T17:05:49.720 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.715+0000 7f96e7fff700 1 -- 172.21.15.27:0/853798414 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f96f001a6f0 con 0x7f97000fc080 2021-11-03T17:05:49.720 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.715+0000 7f9705be9700 1 -- 172.21.15.27:0/853798414 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9700109240 con 0x7f97000fc080 2021-11-03T17:05:49.720 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.715+0000 7f96e7fff700 1 -- 172.21.15.27:0/853798414 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7f96f000f070 con 0x7f97000fc080 2021-11-03T17:05:49.720 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.715+0000 7f96e7fff700 1 -- 172.21.15.27:0/853798414 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f96f0025ad0 con 0x7f97000fc080 2021-11-03T17:05:49.720 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.717+0000 7f9705be9700 1 -- 172.21.15.27:0/853798414 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9700045ad0 con 0x7f97000fc080 2021-11-03T17:05:49.721 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.719+0000 7f96e7fff700 1 -- 172.21.15.27:0/853798414 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72097 (secure 0 0 0) 0x7f96f0016070 con 0x7f97000fc080 2021-11-03T17:05:49.834 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.839+0000 7f9705be9700 1 -- 172.21.15.27:0/853798414 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f97000fa810 con 0x7f97000fc080 2021-11-03T17:05:49.834 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.839+0000 7f96e7fff700 1 -- 172.21.15.27:0/853798414 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f96f0025d60 con 0x7f97000fc080 2021-11-03T17:05:49.835 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.840+0000 7f9705be9700 1 -- 172.21.15.27:0/853798414 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97000fc080 msgr2=0x7f97001086c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:05:49.836 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.840+0000 7f9705be9700 1 --2- 172.21.15.27:0/853798414 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97000fc080 0x7f97001086c0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f96f0006c70 tx=0x7f96f0006ee0).stop 2021-11-03T17:05:49.836 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.840+0000 7f9705be9700 1 -- 172.21.15.27:0/853798414 shutdown_connections 2021-11-03T17:05:49.836 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.840+0000 7f9705be9700 1 --2- 172.21.15.27:0/853798414 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97000fc080 0x7f97001086c0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:05:49.836 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.840+0000 7f9705be9700 1 -- 172.21.15.27:0/853798414 >> 172.21.15.27:0/853798414 conn(0x7f97000f5a60 msgr2=0x7f97000ff8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:05:49.837 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.840+0000 7f9705be9700 1 -- 172.21.15.27:0/853798414 shutdown_connections 2021-11-03T17:05:49.837 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:49.840+0000 7f9705be9700 1 -- 172.21.15.27:0/853798414 wait complete. 2021-11-03T17:05:50.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:49 smithi027 conmon[31596]: audit 2021-11-03T17:05:49.839682+0000 mon.smithi027 (mon.0) 10 : audit [DBG] from='client.? 172.21.15.27:0/853798414' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:05:50.328 INFO:teuthology.orchestra.run.smithi027.stderr:Restarting the monitor... 2021-11-03T17:05:50.328 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: systemctl restart ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c@mon.smithi027 2021-11-03T17:05:50.612 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:50 smithi027 systemd[1]: Stopping Ceph mon.smithi027 for 32ab977e-3cc8-11ec-8c28-001a4aab830c... 2021-11-03T17:05:50.613 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:50 smithi027 conmon[31596]: debug 2021-11-03T17:05:50.430+0000 7f70ebeb3700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.smithi027 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2021-11-03T17:05:50.613 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:50 smithi027 conmon[31596]: debug 2021-11-03T17:05:50.430+0000 7f70ebeb3700 -1 mon.smithi027@0(leader) e1 *** Got Signal Terminated *** 2021-11-03T17:05:51.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:50 smithi027 podman[32105]: 9d6e3ddb553ff72c702eaa115cd2113d2716b4ea69931a737b92fcd8f4010a9d 2021-11-03T17:05:51.391 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 systemd[1]: ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c@mon.smithi027.service: Succeeded. 2021-11-03T17:05:51.392 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 systemd[1]: Stopped Ceph mon.smithi027 for 32ab977e-3cc8-11ec-8c28-001a4aab830c. 2021-11-03T17:05:51.392 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 systemd[1]: Starting Ceph mon.smithi027 for 32ab977e-3cc8-11ec-8c28-001a4aab830c... 2021-11-03T17:05:51.507 INFO:teuthology.orchestra.run.smithi027.stderr:Setting mon public_network to 172.21.0.0/20 2021-11-03T17:05:51.507 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi027 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmp3f8lj10x:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpq2rb3_h7:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 config set mon public_network 172.21.0.0/20 2021-11-03T17:05:51.711 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.490+0000 7ff1cfaa0700 0 set uid:gid to 167:167 (ceph:ceph) 2021-11-03T17:05:51.711 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.490+0000 7ff1cfaa0700 0 ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable), process ceph-mon, pid 7 2021-11-03T17:05:51.711 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.490+0000 7ff1cfaa0700 0 pidfile_write: ignore empty --pid-file 2021-11-03T17:05:51.712 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.494+0000 7ff1cfaa0700 0 load: jerasure load: lrc load: isa 2021-11-03T17:05:51.712 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: RocksDB version: 6.8.1 2021-11-03T17:05:51.712 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: 2021-11-03T17:05:51.713 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.713 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2021-11-03T17:05:51.713 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Compile date May 13 2021 2021-11-03T17:05:51.713 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: DB SUMMARY 2021-11-03T17:05:51.714 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: 2021-11-03T17:05:51.714 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: CURRENT file: CURRENT 2021-11-03T17:05:51.714 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: 2021-11-03T17:05:51.715 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: IDENTITY file: IDENTITY 2021-11-03T17:05:51.715 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: 2021-11-03T17:05:51.715 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: MANIFEST file: MANIFEST-000005 size: 131 Bytes 2021-11-03T17:05:51.715 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: 2021-11-03T17:05:51.715 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi027/store.db dir, Total Num: 1, files: 000004.sst 2021-11-03T17:05:51.716 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: 2021-11-03T17:05:51.716 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi027/store.db: 000006.log size: 97387 ; 2021-11-03T17:05:51.716 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: 2021-11-03T17:05:51.716 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.error_if_exists: 0 2021-11-03T17:05:51.717 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.create_if_missing: 0 2021-11-03T17:05:51.717 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.paranoid_checks: 1 2021-11-03T17:05:51.717 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.env: 0x55d350c221c0 2021-11-03T17:05:51.717 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.fs: Posix File System 2021-11-03T17:05:51.718 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.info_log: 0x55d35318b960 2021-11-03T17:05:51.718 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.max_file_opening_threads: 16 2021-11-03T17:05:51.718 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.statistics: (nil) 2021-11-03T17:05:51.718 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.use_fsync: 0 2021-11-03T17:05:51.719 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.max_log_file_size: 0 2021-11-03T17:05:51.719 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.max_manifest_file_size: 1073741824 2021-11-03T17:05:51.719 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.log_file_time_to_roll: 0 2021-11-03T17:05:51.719 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.keep_log_file_num: 1000 2021-11-03T17:05:51.720 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.recycle_log_file_num: 0 2021-11-03T17:05:51.720 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.allow_fallocate: 1 2021-11-03T17:05:51.720 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.allow_mmap_reads: 0 2021-11-03T17:05:51.720 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.allow_mmap_writes: 0 2021-11-03T17:05:51.720 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.use_direct_reads: 0 2021-11-03T17:05:51.721 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2021-11-03T17:05:51.721 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.create_missing_column_families: 0 2021-11-03T17:05:51.721 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.db_log_dir: 2021-11-03T17:05:51.721 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi027/store.db 2021-11-03T17:05:51.722 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.table_cache_numshardbits: 6 2021-11-03T17:05:51.722 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.max_subcompactions: 1 2021-11-03T17:05:51.722 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.max_background_flushes: -1 2021-11-03T17:05:51.722 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.WAL_ttl_seconds: 0 2021-11-03T17:05:51.723 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.WAL_size_limit_MB: 0 2021-11-03T17:05:51.723 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2021-11-03T17:05:51.723 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.manifest_preallocation_size: 4194304 2021-11-03T17:05:51.723 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.is_fd_close_on_exec: 1 2021-11-03T17:05:51.723 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.advise_random_on_open: 1 2021-11-03T17:05:51.724 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.db_write_buffer_size: 0 2021-11-03T17:05:51.724 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.write_buffer_manager: 0x55d353e00030 2021-11-03T17:05:51.724 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.access_hint_on_compaction_start: 1 2021-11-03T17:05:51.724 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2021-11-03T17:05:51.725 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2021-11-03T17:05:51.725 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.use_adaptive_mutex: 0 2021-11-03T17:05:51.725 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.rate_limiter: (nil) 2021-11-03T17:05:51.727 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2021-11-03T17:05:51.727 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.wal_recovery_mode: 2 2021-11-03T17:05:51.728 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.enable_thread_tracking: 0 2021-11-03T17:05:51.728 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.enable_pipelined_write: 0 2021-11-03T17:05:51.728 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.unordered_write: 0 2021-11-03T17:05:51.728 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2021-11-03T17:05:51.729 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2021-11-03T17:05:51.729 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.write_thread_max_yield_usec: 100 2021-11-03T17:05:51.729 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2021-11-03T17:05:51.729 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.row_cache: None 2021-11-03T17:05:51.729 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.wal_filter: None 2021-11-03T17:05:51.730 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.avoid_flush_during_recovery: 0 2021-11-03T17:05:51.730 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.allow_ingest_behind: 0 2021-11-03T17:05:51.730 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.730 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.preserve_deletes: 0 2021-11-03T17:05:51.731 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.two_write_queues: 0 2021-11-03T17:05:51.731 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.manual_wal_flush: 0 2021-11-03T17:05:51.731 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.atomic_flush: 0 2021-11-03T17:05:51.731 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2021-11-03T17:05:51.731 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.persist_stats_to_disk: 0 2021-11-03T17:05:51.732 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.write_dbid_to_manifest: 0 2021-11-03T17:05:51.732 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.log_readahead_size: 0 2021-11-03T17:05:51.732 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.sst_file_checksum_func: Unknown 2021-11-03T17:05:51.732 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.max_background_jobs: 2 2021-11-03T17:05:51.733 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.max_background_compactions: -1 2021-11-03T17:05:51.733 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2021-11-03T17:05:51.733 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2021-11-03T17:05:51.733 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.delayed_write_rate : 16777216 2021-11-03T17:05:51.733 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.max_total_wal_size: 0 2021-11-03T17:05:51.734 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2021-11-03T17:05:51.734 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.stats_dump_period_sec: 600 2021-11-03T17:05:51.734 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.stats_persist_period_sec: 600 2021-11-03T17:05:51.734 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.stats_history_buffer_size: 1048576 2021-11-03T17:05:51.735 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.max_open_files: -1 2021-11-03T17:05:51.735 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.bytes_per_sync: 0 2021-11-03T17:05:51.735 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.wal_bytes_per_sync: 0 2021-11-03T17:05:51.735 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.strict_bytes_per_sync: 0 2021-11-03T17:05:51.735 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.compaction_readahead_size: 0 2021-11-03T17:05:51.736 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Compression algorithms supported: 2021-11-03T17:05:51.736 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: 2021-11-03T17:05:51.736 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: kZSTDNotFinalCompression supported: 0 2021-11-03T17:05:51.736 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: kZSTD supported: 0 2021-11-03T17:05:51.737 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: kXpressCompression supported: 0 2021-11-03T17:05:51.737 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: kLZ4HCCompression supported: 1 2021-11-03T17:05:51.737 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: kLZ4Compression supported: 1 2021-11-03T17:05:51.737 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.737 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: kBZip2Compression supported: 0 2021-11-03T17:05:51.738 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: kZlibCompression supported: 1 2021-11-03T17:05:51.738 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: kSnappyCompression supported: 1 2021-11-03T17:05:51.738 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Fast CRC32 supported: Supported on x86 2021-11-03T17:05:51.738 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi027/store.db/MANIFEST-000005 2021-11-03T17:05:51.739 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: 2021-11-03T17:05:51.739 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2021-11-03T17:05:51.739 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: 2021-11-03T17:05:51.739 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2021-11-03T17:05:51.739 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.740 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.merge_operator: 2021-11-03T17:05:51.740 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.compaction_filter: None 2021-11-03T17:05:51.740 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.compaction_filter_factory: None 2021-11-03T17:05:51.740 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.memtable_factory: SkipListFactory 2021-11-03T17:05:51.741 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.table_factory: BlockBasedTable 2021-11-03T17:05:51.741 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d35309c128) 2021-11-03T17:05:51.741 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: cache_index_and_filter_blocks: 1 2021-11-03T17:05:51.741 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: cache_index_and_filter_blocks_with_high_priority: 0 2021-11-03T17:05:51.742 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: pin_l0_filter_and_index_blocks_in_cache: 0 2021-11-03T17:05:51.742 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: pin_top_level_index_and_filter: 1 2021-11-03T17:05:51.742 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: index_type: 0 2021-11-03T17:05:51.742 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: data_block_index_type: 0 2021-11-03T17:05:51.742 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: index_shortening: 1 2021-11-03T17:05:51.743 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: data_block_hash_table_util_ratio: 0.750000 2021-11-03T17:05:51.743 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: hash_index_allow_collision: 1 2021-11-03T17:05:51.743 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: checksum: 1 2021-11-03T17:05:51.743 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: no_block_cache: 0 2021-11-03T17:05:51.744 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: block_cache: 0x55d3530d6f10 2021-11-03T17:05:51.744 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: block_cache_name: BinnedLRUCache 2021-11-03T17:05:51.744 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: block_cache_options: 2021-11-03T17:05:51.744 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: capacity : 536870912 2021-11-03T17:05:51.744 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: num_shard_bits : 4 2021-11-03T17:05:51.745 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: strict_capacity_limit : 0 2021-11-03T17:05:51.745 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: high_pri_pool_ratio: 0.000 2021-11-03T17:05:51.745 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: block_cache_compressed: (nil) 2021-11-03T17:05:51.745 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: persistent_cache: (nil) 2021-11-03T17:05:51.746 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: block_size: 4096 2021-11-03T17:05:51.746 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: block_size_deviation: 10 2021-11-03T17:05:51.746 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: block_restart_interval: 16 2021-11-03T17:05:51.746 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: index_block_restart_interval: 1 2021-11-03T17:05:51.747 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: metadata_block_size: 4096 2021-11-03T17:05:51.747 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: partition_filters: 0 2021-11-03T17:05:51.747 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: use_delta_encoding: 1 2021-11-03T17:05:51.747 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: filter_policy: rocksdb.BuiltinBloomFilter 2021-11-03T17:05:51.747 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: whole_key_filtering: 1 2021-11-03T17:05:51.748 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: verify_compression: 0 2021-11-03T17:05:51.748 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: read_amp_bytes_per_bit: 0 2021-11-03T17:05:51.748 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: format_version: 2 2021-11-03T17:05:51.748 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: enable_index_compression: 1 2021-11-03T17:05:51.749 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: block_align: 0 2021-11-03T17:05:51.749 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: 2021-11-03T17:05:51.749 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.write_buffer_size: 33554432 2021-11-03T17:05:51.749 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.max_write_buffer_number: 2 2021-11-03T17:05:51.749 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.compression: NoCompression 2021-11-03T17:05:51.750 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.bottommost_compression: Disabled 2021-11-03T17:05:51.750 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.prefix_extractor: nullptr 2021-11-03T17:05:51.750 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2021-11-03T17:05:51.750 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.num_levels: 7 2021-11-03T17:05:51.751 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2021-11-03T17:05:51.751 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2021-11-03T17:05:51.751 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2021-11-03T17:05:51.751 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2021-11-03T17:05:51.751 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2021-11-03T17:05:51.752 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2021-11-03T17:05:51.752 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2021-11-03T17:05:51.752 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2021-11-03T17:05:51.752 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2021-11-03T17:05:51.753 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.compression_opts.window_bits: -14 2021-11-03T17:05:51.753 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.compression_opts.level: 32767 2021-11-03T17:05:51.753 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.compression_opts.strategy: 0 2021-11-03T17:05:51.753 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2021-11-03T17:05:51.753 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2021-11-03T17:05:51.754 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.compression_opts.enabled: false 2021-11-03T17:05:51.754 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2021-11-03T17:05:51.754 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2021-11-03T17:05:51.754 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.level0_stop_writes_trigger: 36 2021-11-03T17:05:51.755 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.target_file_size_base: 67108864 2021-11-03T17:05:51.755 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.target_file_size_multiplier: 1 2021-11-03T17:05:51.755 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2021-11-03T17:05:51.755 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2021-11-03T17:05:51.756 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2021-11-03T17:05:51.756 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2021-11-03T17:05:51.756 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2021-11-03T17:05:51.756 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2021-11-03T17:05:51.756 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2021-11-03T17:05:51.757 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2021-11-03T17:05:51.757 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2021-11-03T17:05:51.757 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2021-11-03T17:05:51.757 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2021-11-03T17:05:51.758 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.max_compaction_bytes: 1677721600 2021-11-03T17:05:51.758 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: 2021-11-03T17:05:51.758 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.arena_block_size: 4194304 2021-11-03T17:05:51.758 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2021-11-03T17:05:51.758 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2021-11-03T17:05:51.759 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2021-11-03T17:05:51.759 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.disable_auto_compactions: 0 2021-11-03T17:05:51.759 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2021-11-03T17:05:51.759 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2021-11-03T17:05:51.760 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2021-11-03T17:05:51.760 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2021-11-03T17:05:51.760 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2021-11-03T17:05:51.760 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2021-11-03T17:05:51.761 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2021-11-03T17:05:51.761 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2021-11-03T17:05:51.761 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2021-11-03T17:05:51.761 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2021-11-03T17:05:51.762 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.table_properties_collectors: 2021-11-03T17:05:51.762 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.inplace_update_support: 0 2021-11-03T17:05:51.762 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.inplace_update_num_locks: 10000 2021-11-03T17:05:51.762 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2021-11-03T17:05:51.763 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.memtable_whole_key_filtering: 0 2021-11-03T17:05:51.763 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.memtable_huge_page_size: 0 2021-11-03T17:05:51.763 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.bloom_locality: 0 2021-11-03T17:05:51.763 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.max_successive_merges: 0 2021-11-03T17:05:51.764 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.optimize_filters_for_hits: 0 2021-11-03T17:05:51.764 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.paranoid_file_checks: 0 2021-11-03T17:05:51.764 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.force_consistency_checks: 0 2021-11-03T17:05:51.764 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.report_bg_io_stats: 0 2021-11-03T17:05:51.764 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.ttl: 2592000 2021-11-03T17:05:51.765 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.495+0000 7ff1cfaa0700 4 rocksdb: Options.periodic_compaction_seconds: 0 2021-11-03T17:05:51.765 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.496+0000 7ff1cfaa0700 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi027/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 5, log_number is 4,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2021-11-03T17:05:51.765 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: 2021-11-03T17:05:51.765 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.765 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: 2021-11-03T17:05:51.496+0000 7ff1cfaa0700 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 4 2021-11-03T17:05:51.766 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: 2021-11-03T17:05:51.766 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.496+0000 7ff1cfaa0700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1635959151496613, "job": 1, "event": "recovery_started", "log_files": [6]} 2021-11-03T17:05:51.766 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.496+0000 7ff1cfaa0700 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #6 mode 2 2021-11-03T17:05:51.766 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.496+0000 7ff1cfaa0700 3 rocksdb: [le/block_based/filter_policy.cc:584] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2021-11-03T17:05:51.767 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.497+0000 7ff1cfaa0700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1635959151497502, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 7, "file_size": 93057, "table_properties": {"data_size": 91157, "index_size": 377, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 709, "raw_key_size": 12317, "raw_average_key_size": 49, "raw_value_size": 84313, "raw_average_value_size": 337, "num_data_blocks": 12, "num_entries": 250, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1635959151, "oldest_key_time": 3, "file_creation_time": 0}} 2021-11-03T17:05:51.767 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.497+0000 7ff1cfaa0700 4 rocksdb: [version_set.cc:3826] Creating manifest 8 2021-11-03T17:05:51.767 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: 2021-11-03T17:05:51.767 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.497+0000 7ff1cfaa0700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1635959151497883, "job": 1, "event": "recovery_finished"} 2021-11-03T17:05:51.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.498+0000 7ff1cfaa0700 4 rocksdb: DB pointer 0x55d35319d800 2021-11-03T17:05:51.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.498+0000 7ff1b68b8700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2021-11-03T17:05:51.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.498+0000 7ff1b68b8700 4 rocksdb: [db_impl/db_impl.cc:851] 2021-11-03T17:05:51.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: ** DB Stats ** 2021-11-03T17:05:51.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: Uptime(secs): 0.0 total, 0.0 interval 2021-11-03T17:05:51.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2021-11-03T17:05:51.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2021-11-03T17:05:51.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2021-11-03T17:05:51.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2021-11-03T17:05:51.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2021-11-03T17:05:51.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2021-11-03T17:05:51.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: 2021-11-03T17:05:51.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: ** Compaction Stats [default] ** 2021-11-03T17:05:51.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-11-03T17:05:51.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-11-03T17:05:51.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: L0 2/0 92.57 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 157.1 0.00 0.00 1 0.001 0 0 2021-11-03T17:05:51.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: Sum 2/0 92.57 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 157.1 0.00 0.00 1 0.001 0 0 2021-11-03T17:05:51.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 157.1 0.00 0.00 1 0.001 0 0 2021-11-03T17:05:51.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: 2021-11-03T17:05:51.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: ** Compaction Stats [default] ** 2021-11-03T17:05:51.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-11-03T17:05:51.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-11-03T17:05:51.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 157.1 0.00 0.00 1 0.001 0 0 2021-11-03T17:05:51.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: Uptime(secs): 0.0 total, 0.0 interval 2021-11-03T17:05:51.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: Flush(GB): cumulative 0.000, interval 0.000 2021-11-03T17:05:51.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: AddFile(GB): cumulative 0.000, interval 0.000 2021-11-03T17:05:51.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: AddFile(Total Files): cumulative 0, interval 0 2021-11-03T17:05:51.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: AddFile(L0 Files): cumulative 0, interval 0 2021-11-03T17:05:51.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: AddFile(Keys): cumulative 0, interval 0 2021-11-03T17:05:51.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: Cumulative compaction: 0.00 GB write, 32.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-11-03T17:05:51.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: Interval compaction: 0.00 GB write, 32.57 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-11-03T17:05:51.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2021-11-03T17:05:51.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: 2021-11-03T17:05:51.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: ** File Read Latency Histogram By Level [default] ** 2021-11-03T17:05:51.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: 2021-11-03T17:05:51.776 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: ** Compaction Stats [default] ** 2021-11-03T17:05:51.776 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-11-03T17:05:51.776 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-11-03T17:05:51.776 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: L0 2/0 92.57 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 157.1 0.00 0.00 1 0.001 0 0 2021-11-03T17:05:51.777 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: Sum 2/0 92.57 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 157.1 0.00 0.00 1 0.001 0 0 2021-11-03T17:05:51.777 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2021-11-03T17:05:51.777 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: 2021-11-03T17:05:51.777 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: ** Compaction Stats [default] ** 2021-11-03T17:05:51.777 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-11-03T17:05:51.778 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-11-03T17:05:51.778 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 157.1 0.00 0.00 1 0.001 0 0 2021-11-03T17:05:51.778 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: Uptime(secs): 0.0 total, 0.0 interval 2021-11-03T17:05:51.778 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: Flush(GB): cumulative 0.000, interval 0.000 2021-11-03T17:05:51.779 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: AddFile(GB): cumulative 0.000, interval 0.000 2021-11-03T17:05:51.779 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: AddFile(Total Files): cumulative 0, interval 0 2021-11-03T17:05:51.779 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: AddFile(L0 Files): cumulative 0, interval 0 2021-11-03T17:05:51.779 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: AddFile(Keys): cumulative 0, interval 0 2021-11-03T17:05:51.779 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: Cumulative compaction: 0.00 GB write, 32.24 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-11-03T17:05:51.780 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-11-03T17:05:51.780 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2021-11-03T17:05:51.780 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: 2021-11-03T17:05:51.780 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: ** File Read Latency Histogram By Level [default] ** 2021-11-03T17:05:51.781 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: 2021-11-03T17:05:51.781 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.498+0000 7ff1cfaa0700 0 starting mon.smithi027 rank 0 at public addrs [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] at bind addrs [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi027 fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c 2021-11-03T17:05:51.781 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.499+0000 7ff1cfaa0700 1 mon.smithi027@-1(???) e1 preinit fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c 2021-11-03T17:05:51.781 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.499+0000 7ff1cfaa0700 0 mon.smithi027@-1(???).mds e1 new map 2021-11-03T17:05:51.781 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.782 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: 2021-11-03T17:05:51.499+0000 7ff1cfaa0700 0 mon.smithi027@-1(???).mds e1 print_map 2021-11-03T17:05:51.782 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: e1 2021-11-03T17:05:51.782 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:05:51.782 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:05:51.783 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: legacy client fscid: -1 2021-11-03T17:05:51.783 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: 2021-11-03T17:05:51.783 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: No filesystems configured 2021-11-03T17:05:51.783 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: 2021-11-03T17:05:51.783 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.499+0000 7ff1cfaa0700 0 mon.smithi027@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2021-11-03T17:05:51.784 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.784 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: 2021-11-03T17:05:51.499+0000 7ff1cfaa0700 0 mon.smithi027@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2021-11-03T17:05:51.784 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.499+0000 7ff1cfaa0700 0 mon.smithi027@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2021-11-03T17:05:51.784 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.499+0000 7ff1cfaa0700 0 mon.smithi027@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2021-11-03T17:05:51.785 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: audit 2021-11-03T17:05:51.785 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: 2021-11-03T17:05:49.839682 2021-11-03T17:05:51.785 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: +0000 mon.smithi027 (mon.0) 10 : audit 2021-11-03T17:05:51.785 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: [DBG] from='client.? 172.21.15.27:0/853798414' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:05:51.785 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: debug 2021-11-03T17:05:51.500+0000 7ff1cfaa0700 1 mon.smithi027@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2021-11-03T17:05:51.786 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 bash[32208]: 060d5938214f76bc057d33ac29fdfd199f0136f33cb61527d59b00179ee25f36 2021-11-03T17:05:51.786 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 systemd[1]: Started Ceph mon.smithi027 for 32ab977e-3cc8-11ec-8c28-001a4aab830c. 2021-11-03T17:05:51.786 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: cluster 2021-11-03T17:05:51.504094+0000 mon.smithi027 ( 2021-11-03T17:05:51.786 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: mon.0) 1 : cluster [INF] mon.smithi027 is new leader, mons smithi027 in quorum (ranks 0) 2021-11-03T17:05:51.787 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: cluster 2021-11-03T17:05:51.504137+0000 mon.smithi027 (mon.0 2021-11-03T17:05:51.787 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: ) 2 : cluster [DBG] monmap e1: 1 mons at {smithi027=[v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0]} 2021-11-03T17:05:51.787 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: cluster 2021-11-03T17:05:51.504167 2021-11-03T17:05:51.787 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: +0000 mon.smithi027 (mon.0) 3 : cluster [DBG] 2021-11-03T17:05:51.787 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: fsmap 2021-11-03T17:05:51.788 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: cluster 2021-11-03 2021-11-03T17:05:51.788 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: T17:05:51.504190+0000 2021-11-03T17:05:51.788 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: mon.smithi027 (mon.0) 4 : 2021-11-03T17:05:51.788 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: cluster [DBG] osdmap e1: 0 total, 0 up, 0 in 2021-11-03T17:05:51.788 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: cluster 2021-11-03T17: 2021-11-03T17:05:51.789 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: 05:51.504536+0000 mon.smithi027 (mon 2021-11-03T17:05:51.789 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: .0) 5 : cluster [DBG] 2021-11-03T17:05:51.789 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:51 smithi027 conmon[32322]: mgrmap e1: no daemons active 2021-11-03T17:05:52.162 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.165+0000 7f062824c700 1 Processor -- start 2021-11-03T17:05:52.163 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.166+0000 7f062824c700 1 -- start start 2021-11-03T17:05:52.163 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.166+0000 7f062824c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06200fd280 0x7f06200fd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-03T17:05:52.163 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.166+0000 7f062824c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06200fdb90 con 0x7f06200fd280 2021-11-03T17:05:52.164 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.166+0000 7f0625fe8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06200fd280 0x7f06200fd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:05:52.164 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.166+0000 7f0625fe8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06200fd280 0x7f06200fd6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53956/0 (socket says 172.21.15.27:53956) 2021-11-03T17:05:52.164 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.166+0000 7f0625fe8700 1 -- 172.21.15.27:0/4077964423 learned_addr learned my addr 172.21.15.27:0/4077964423 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:05:52.164 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.167+0000 7f0625fe8700 1 -- 172.21.15.27:0/4077964423 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06200fdcd0 con 0x7f06200fd280 2021-11-03T17:05:52.164 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.167+0000 7f0625fe8700 1 --2- 172.21.15.27:0/4077964423 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06200fd280 0x7f06200fd6a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7f061c00b5f0 tx=0x7f061c006ee0).ready entity=mon.0 client_cookie=bd847c6193f5f09 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:05:52.165 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.167+0000 7f0624fe6700 1 -- 172.21.15.27:0/4077964423 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f061c00d600 con 0x7f06200fd280 2021-11-03T17:05:52.165 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.167+0000 7f0624fe6700 1 -- 172.21.15.27:0/4077964423 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 983+0+0 (secure 0 0 0) 0x7f061c00dbb0 con 0x7f06200fd280 2021-11-03T17:05:52.165 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.167+0000 7f0624fe6700 1 -- 172.21.15.27:0/4077964423 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f061c011bb0 con 0x7f06200fd280 2021-11-03T17:05:52.168 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.168+0000 7f062824c700 1 -- 172.21.15.27:0/4077964423 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06200fd280 msgr2=0x7f06200fd6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:05:52.168 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.168+0000 7f062824c700 1 --2- 172.21.15.27:0/4077964423 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06200fd280 0x7f06200fd6a0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7f061c00b5f0 tx=0x7f061c006ee0).stop 2021-11-03T17:05:52.169 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.168+0000 7f062824c700 1 -- 172.21.15.27:0/4077964423 shutdown_connections 2021-11-03T17:05:52.169 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.168+0000 7f062824c700 1 --2- 172.21.15.27:0/4077964423 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06200fd280 0x7f06200fd6a0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:05:52.169 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.168+0000 7f062824c700 1 -- 172.21.15.27:0/4077964423 >> 172.21.15.27:0/4077964423 conn(0x7f06200f5a60 msgr2=0x7f06200f7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:05:52.169 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.168+0000 7f062824c700 1 -- 172.21.15.27:0/4077964423 shutdown_connections 2021-11-03T17:05:52.170 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.168+0000 7f062824c700 1 -- 172.21.15.27:0/4077964423 wait complete. 2021-11-03T17:05:52.170 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.169+0000 7f062824c700 1 Processor -- start 2021-11-03T17:05:52.170 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.169+0000 7f062824c700 1 -- start start 2021-11-03T17:05:52.170 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.169+0000 7f062824c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06200fd280 0x7f062010a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:05:52.170 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.169+0000 7f062824c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062010adf0 con 0x7f06200fd280 2021-11-03T17:05:52.171 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.169+0000 7f0625fe8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06200fd280 0x7f062010a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:05:52.171 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.169+0000 7f0625fe8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06200fd280 0x7f062010a900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53958/0 (socket says 172.21.15.27:53958) 2021-11-03T17:05:52.171 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.169+0000 7f0625fe8700 1 -- 172.21.15.27:0/3020903776 learned_addr learned my addr 172.21.15.27:0/3020903776 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:05:52.171 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.169+0000 7f0625fe8700 1 -- 172.21.15.27:0/3020903776 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f061c00b200 con 0x7f06200fd280 2021-11-03T17:05:52.171 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.169+0000 7f0625fe8700 1 --2- 172.21.15.27:0/3020903776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06200fd280 0x7f062010a900 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7f061c006a30 tx=0x7f061c006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:05:52.172 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.169+0000 7f060e7fc700 1 -- 172.21.15.27:0/3020903776 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f061c01b570 con 0x7f06200fd280 2021-11-03T17:05:52.172 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.169+0000 7f062824c700 1 -- 172.21.15.27:0/3020903776 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f062010aff0 con 0x7f06200fd280 2021-11-03T17:05:52.172 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.169+0000 7f060e7fc700 1 -- 172.21.15.27:0/3020903776 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 983+0+0 (secure 0 0 0) 0x7f061c01bb20 con 0x7f06200fd280 2021-11-03T17:05:52.172 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.169+0000 7f060e7fc700 1 -- 172.21.15.27:0/3020903776 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f061c01aa00 con 0x7f06200fd280 2021-11-03T17:05:52.173 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.170+0000 7f062824c700 1 -- 172.21.15.27:0/3020903776 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f062010b480 con 0x7f06200fd280 2021-11-03T17:05:52.173 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.170+0000 7f060e7fc700 1 -- 172.21.15.27:0/3020903776 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7f061c00f070 con 0x7f06200fd280 2021-11-03T17:05:52.173 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.170+0000 7f060e7fc700 1 -- 172.21.15.27:0/3020903776 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f061c024c50 con 0x7f06200fd280 2021-11-03T17:05:52.173 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.171+0000 7f062824c700 1 -- 172.21.15.27:0/3020903776 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0620045ad0 con 0x7f06200fd280 2021-11-03T17:05:52.173 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.173+0000 7f060e7fc700 1 -- 172.21.15.27:0/3020903776 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72097 (secure 0 0 0) 0x7f061c016070 con 0x7f06200fd280 2021-11-03T17:05:52.286 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.291+0000 7f062824c700 1 -- 172.21.15.27:0/3020903776 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f06200fa640 con 0x7f06200fd280 2021-11-03T17:05:52.289 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.294+0000 7f060e7fc700 1 -- 172.21.15.27:0/3020903776 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f061c024e10 con 0x7f06200fd280 2021-11-03T17:05:52.290 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.294+0000 7f062824c700 1 -- 172.21.15.27:0/3020903776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06200fd280 msgr2=0x7f062010a900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:05:52.290 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.294+0000 7f062824c700 1 --2- 172.21.15.27:0/3020903776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06200fd280 0x7f062010a900 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7f061c006a30 tx=0x7f061c006ee0).stop 2021-11-03T17:05:52.290 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.294+0000 7f062824c700 1 -- 172.21.15.27:0/3020903776 shutdown_connections 2021-11-03T17:05:52.291 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.294+0000 7f062824c700 1 --2- 172.21.15.27:0/3020903776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06200fd280 0x7f062010a900 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:05:52.291 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.294+0000 7f062824c700 1 -- 172.21.15.27:0/3020903776 >> 172.21.15.27:0/3020903776 conn(0x7f06200f5a60 msgr2=0x7f06200ff180 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:05:52.291 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.295+0000 7f062824c700 1 -- 172.21.15.27:0/3020903776 shutdown_connections 2021-11-03T17:05:52.291 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:52.295+0000 7f062824c700 1 -- 172.21.15.27:0/3020903776 wait complete. 2021-11-03T17:05:52.861 INFO:teuthology.orchestra.run.smithi027.stderr:Wrote config to /etc/ceph/ceph.conf 2021-11-03T17:05:52.862 INFO:teuthology.orchestra.run.smithi027.stderr:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2021-11-03T17:05:52.862 INFO:teuthology.orchestra.run.smithi027.stderr:Creating mgr... 2021-11-03T17:05:52.862 INFO:teuthology.orchestra.run.smithi027.stderr:Verifying port 9283 ... 2021-11-03T17:05:52.863 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: systemctl daemon-reload 2021-11-03T17:05:52.957 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: systemctl stop ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c@mgr.smithi027.ufhhvb 2021-11-03T17:05:52.982 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: systemctl reset-failed ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c@mgr.smithi027.ufhhvb 2021-11-03T17:05:52.987 INFO:teuthology.orchestra.run.smithi027.stderr:systemctl: Failed to reset failed state of unit ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c@mgr.smithi027.ufhhvb.service: Unit ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c@mgr.smithi027.ufhhvb.service not loaded. 2021-11-03T17:05:52.988 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: systemctl enable ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c@mgr.smithi027.ufhhvb 2021-11-03T17:05:52.994 INFO:teuthology.orchestra.run.smithi027.stderr:systemctl: Created symlink /etc/systemd/system/ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c.target.wants/ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c@mgr.smithi027.ufhhvb.service → /etc/systemd/system/ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c@.service. 2021-11-03T17:05:53.077 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: systemctl start ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c@mgr.smithi027.ufhhvb 2021-11-03T17:05:53.741 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:53 smithi027 conmon[32322]: audit 2021-11-03T17:05:52.294422+0000 2021-11-03T17:05:53.742 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:53 smithi027 conmon[32322]: mon.smithi027 (mon.0) 6 : audit [INF] from='client.? 172.21.15.27:0/3020903776' entity='client.admin' 2021-11-03T17:05:53.748 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: systemctl is-enabled firewalld.service 2021-11-03T17:05:53.756 INFO:teuthology.orchestra.run.smithi027.stderr:systemctl: disabled 2021-11-03T17:05:53.756 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: systemctl is-active firewalld.service 2021-11-03T17:05:53.764 INFO:teuthology.orchestra.run.smithi027.stderr:systemctl: inactive 2021-11-03T17:05:53.765 INFO:teuthology.orchestra.run.smithi027.stderr:firewalld.service is not enabled 2021-11-03T17:05:53.765 INFO:teuthology.orchestra.run.smithi027.stderr:Not possible to enable service . firewalld.service is not available 2021-11-03T17:05:53.765 INFO:teuthology.orchestra.run.smithi027.stderr:Not possible to open ports <[]>. firewalld.service is not available 2021-11-03T17:05:53.765 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: systemctl is-enabled firewalld.service 2021-11-03T17:05:53.772 INFO:teuthology.orchestra.run.smithi027.stderr:systemctl: disabled 2021-11-03T17:05:53.773 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: systemctl is-active firewalld.service 2021-11-03T17:05:53.780 INFO:teuthology.orchestra.run.smithi027.stderr:systemctl: inactive 2021-11-03T17:05:53.781 INFO:teuthology.orchestra.run.smithi027.stderr:firewalld.service is not enabled 2021-11-03T17:05:53.782 INFO:teuthology.orchestra.run.smithi027.stderr:Not possible to open ports <[9283]>. firewalld.service is not available 2021-11-03T17:05:53.782 INFO:teuthology.orchestra.run.smithi027.stderr:Waiting for mgr to start... 2021-11-03T17:05:53.782 INFO:teuthology.orchestra.run.smithi027.stderr:Waiting for mgr... 2021-11-03T17:05:53.782 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi027 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmp3f8lj10x:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpq2rb3_h7:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 status -f json-pretty 2021-11-03T17:05:54.503 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.504+0000 7f3109302700 1 Processor -- start 2021-11-03T17:05:54.504 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.504+0000 7f3109302700 1 -- start start 2021-11-03T17:05:54.504 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.504+0000 7f3109302700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31040fd260 0x7f31040fd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-03T17:05:54.504 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.504+0000 7f3109302700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31040fdb70 con 0x7f31040fd260 2021-11-03T17:05:54.505 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.505+0000 7f3102ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31040fd260 0x7f31040fd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:05:54.505 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.505+0000 7f3102ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31040fd260 0x7f31040fd680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53964/0 (socket says 172.21.15.27:53964) 2021-11-03T17:05:54.505 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.505+0000 7f3102ffd700 1 -- 172.21.15.27:0/1044905090 learned_addr learned my addr 172.21.15.27:0/1044905090 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:05:54.505 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.505+0000 7f3102ffd700 1 -- 172.21.15.27:0/1044905090 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31040fdcb0 con 0x7f31040fd260 2021-11-03T17:05:54.506 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.505+0000 7f3102ffd700 1 --2- 172.21.15.27:0/1044905090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31040fd260 0x7f31040fd680 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f30f400b5f0 tx=0x7f30f4006ee0).ready entity=mon.0 client_cookie=22b819813e450933 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:05:54.506 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.505+0000 7f3101ffb700 1 -- 172.21.15.27:0/1044905090 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f30f400d600 con 0x7f31040fd260 2021-11-03T17:05:54.506 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.505+0000 7f3101ffb700 1 -- 172.21.15.27:0/1044905090 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 983+0+0 (secure 0 0 0) 0x7f30f400dbb0 con 0x7f31040fd260 2021-11-03T17:05:54.506 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.505+0000 7f3101ffb700 1 -- 172.21.15.27:0/1044905090 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f30f4011bb0 con 0x7f31040fd260 2021-11-03T17:05:54.507 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.506+0000 7f3109302700 1 -- 172.21.15.27:0/1044905090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31040fd260 msgr2=0x7f31040fd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:05:54.507 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.506+0000 7f3109302700 1 --2- 172.21.15.27:0/1044905090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31040fd260 0x7f31040fd680 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7f30f400b5f0 tx=0x7f30f4006ee0).stop 2021-11-03T17:05:54.507 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.506+0000 7f3109302700 1 -- 172.21.15.27:0/1044905090 shutdown_connections 2021-11-03T17:05:54.507 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.506+0000 7f3109302700 1 --2- 172.21.15.27:0/1044905090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31040fd260 0x7f31040fd680 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:05:54.508 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.506+0000 7f3109302700 1 -- 172.21.15.27:0/1044905090 >> 172.21.15.27:0/1044905090 conn(0x7f31040f5a60 msgr2=0x7f31040f7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:05:54.508 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.506+0000 7f3109302700 1 -- 172.21.15.27:0/1044905090 shutdown_connections 2021-11-03T17:05:54.508 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.506+0000 7f3109302700 1 -- 172.21.15.27:0/1044905090 wait complete. 2021-11-03T17:05:54.508 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.507+0000 7f3109302700 1 Processor -- start 2021-11-03T17:05:54.508 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.507+0000 7f3109302700 1 -- start start 2021-11-03T17:05:54.509 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.507+0000 7f3109302700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31040fd260 0x7f310410a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:05:54.509 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.507+0000 7f3109302700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f310410ae20 con 0x7f31040fd260 2021-11-03T17:05:54.509 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.507+0000 7f3102ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31040fd260 0x7f310410a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:05:54.509 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.507+0000 7f3102ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31040fd260 0x7f310410a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53966/0 (socket says 172.21.15.27:53966) 2021-11-03T17:05:54.510 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.507+0000 7f3102ffd700 1 -- 172.21.15.27:0/3086094785 learned_addr learned my addr 172.21.15.27:0/3086094785 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:05:54.510 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.508+0000 7f3102ffd700 1 -- 172.21.15.27:0/3086094785 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f400b200 con 0x7f31040fd260 2021-11-03T17:05:54.510 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.508+0000 7f3102ffd700 1 --2- 172.21.15.27:0/3086094785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31040fd260 0x7f310410a930 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f30f4006a30 tx=0x7f30f4006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:05:54.510 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.508+0000 7f30eb7fe700 1 -- 172.21.15.27:0/3086094785 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f30f401b410 con 0x7f31040fd260 2021-11-03T17:05:54.511 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.508+0000 7f30eb7fe700 1 -- 172.21.15.27:0/3086094785 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 983+0+0 (secure 0 0 0) 0x7f30f401b9c0 con 0x7f31040fd260 2021-11-03T17:05:54.511 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.508+0000 7f3109302700 1 -- 172.21.15.27:0/3086094785 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f310410b020 con 0x7f31040fd260 2021-11-03T17:05:54.511 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.508+0000 7f30eb7fe700 1 -- 172.21.15.27:0/3086094785 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f30f401a780 con 0x7f31040fd260 2021-11-03T17:05:54.511 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.508+0000 7f3109302700 1 -- 172.21.15.27:0/3086094785 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f310410b4b0 con 0x7f31040fd260 2021-11-03T17:05:54.511 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.509+0000 7f30eb7fe700 1 -- 172.21.15.27:0/3086094785 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7f30f400f070 con 0x7f31040fd260 2021-11-03T17:05:54.512 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.509+0000 7f30eb7fe700 1 -- 172.21.15.27:0/3086094785 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f30f40247b0 con 0x7f31040fd260 2021-11-03T17:05:54.513 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.509+0000 7f3109302700 1 -- 172.21.15.27:0/3086094785 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3104045ad0 con 0x7f31040fd260 2021-11-03T17:05:54.513 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.512+0000 7f30eb7fe700 1 -- 172.21.15.27:0/3086094785 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72097 (secure 0 0 0) 0x7f30f4016070 con 0x7f31040fd260 2021-11-03T17:05:54.631 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.636+0000 7f3109302700 1 -- 172.21.15.27:0/3086094785 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f3104104550 con 0x7f31040fd260 2021-11-03T17:05:54.631 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.636+0000 7f30eb7fe700 1 -- 172.21.15.27:0/3086094785 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1230 (secure 0 0 0) 0x7f30f401aa40 con 0x7f31040fd260 2021-11-03T17:05:54.638 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.638 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: { 2021-11-03T17:05:54.639 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "fsid": "32ab977e-3cc8-11ec-8c28-001a4aab830c", 2021-11-03T17:05:54.639 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "health": { 2021-11-03T17:05:54.639 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2021-11-03T17:05:54.639 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "checks": {}, 2021-11-03T17:05:54.640 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "mutes": [] 2021-11-03T17:05:54.640 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: }, 2021-11-03T17:05:54.640 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "election_epoch": 5, 2021-11-03T17:05:54.640 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "quorum": [ 2021-11-03T17:05:54.641 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 0 2021-11-03T17:05:54.641 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: ], 2021-11-03T17:05:54.641 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "quorum_names": [ 2021-11-03T17:05:54.641 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "smithi027" 2021-11-03T17:05:54.642 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: ], 2021-11-03T17:05:54.642 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "quorum_age": 3, 2021-11-03T17:05:54.642 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "monmap": { 2021-11-03T17:05:54.642 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "epoch": 1, 2021-11-03T17:05:54.642 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "min_mon_release_name": "pacific", 2021-11-03T17:05:54.643 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "num_mons": 1 2021-11-03T17:05:54.643 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: }, 2021-11-03T17:05:54.643 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "osdmap": { 2021-11-03T17:05:54.643 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "epoch": 1, 2021-11-03T17:05:54.644 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "num_osds": 0, 2021-11-03T17:05:54.644 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "num_up_osds": 0, 2021-11-03T17:05:54.644 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "osd_up_since": 0, 2021-11-03T17:05:54.644 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "num_in_osds": 0, 2021-11-03T17:05:54.645 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "osd_in_since": 0, 2021-11-03T17:05:54.645 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2021-11-03T17:05:54.645 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: }, 2021-11-03T17:05:54.645 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "pgmap": { 2021-11-03T17:05:54.646 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "pgs_by_state": [], 2021-11-03T17:05:54.646 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "num_pgs": 0, 2021-11-03T17:05:54.646 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "num_pools": 0, 2021-11-03T17:05:54.646 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "num_objects": 0, 2021-11-03T17:05:54.647 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "data_bytes": 0, 2021-11-03T17:05:54.647 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "bytes_used": 0, 2021-11-03T17:05:54.647 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "bytes_avail": 0, 2021-11-03T17:05:54.647 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "bytes_total": 0 2021-11-03T17:05:54.647 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: }, 2021-11-03T17:05:54.648 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "fsmap": { 2021-11-03T17:05:54.648 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "epoch": 1, 2021-11-03T17:05:54.648 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "by_rank": [], 2021-11-03T17:05:54.648 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "up:standby": 0 2021-11-03T17:05:54.648 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: }, 2021-11-03T17:05:54.649 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "mgrmap": { 2021-11-03T17:05:54.649 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "available": false, 2021-11-03T17:05:54.649 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "num_standbys": 0, 2021-11-03T17:05:54.650 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "modules": [ 2021-11-03T17:05:54.650 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "iostat", 2021-11-03T17:05:54.650 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "restful" 2021-11-03T17:05:54.650 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: ], 2021-11-03T17:05:54.651 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "services": {} 2021-11-03T17:05:54.651 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: }, 2021-11-03T17:05:54.651 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "servicemap": { 2021-11-03T17:05:54.651 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "epoch": 1, 2021-11-03T17:05:54.652 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "modified": "2021-11-03T17:05:46.314793+0000", 2021-11-03T17:05:54.652 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "services": {} 2021-11-03T17:05:54.652 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: }, 2021-11-03T17:05:54.652 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "progress_events": {} 2021-11-03T17:05:54.652 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: } 2021-11-03T17:05:54.654 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.637+0000 7f3109302700 1 -- 172.21.15.27:0/3086094785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31040fd260 msgr2=0x7f310410a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:05:54.654 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.637+0000 7f3109302700 1 --2- 172.21.15.27:0/3086094785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31040fd260 0x7f310410a930 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f30f4006a30 tx=0x7f30f4006ee0).stop 2021-11-03T17:05:54.654 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.637+0000 7f3109302700 1 -- 172.21.15.27:0/3086094785 shutdown_connections 2021-11-03T17:05:54.655 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.637+0000 7f3109302700 1 --2- 172.21.15.27:0/3086094785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31040fd260 0x7f310410a930 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:05:54.655 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.637+0000 7f3109302700 1 -- 172.21.15.27:0/3086094785 >> 172.21.15.27:0/3086094785 conn(0x7f31040f5a60 msgr2=0x7f31040ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:05:54.655 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.637+0000 7f3109302700 1 -- 172.21.15.27:0/3086094785 shutdown_connections 2021-11-03T17:05:54.655 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:54.637+0000 7f3109302700 1 -- 172.21.15.27:0/3086094785 wait complete. 2021-11-03T17:05:55.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:54 smithi027 conmon[32322]: audit 2021-11-03T17:05:54.637058+0000 2021-11-03T17:05:55.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:54 smithi027 conmon[32322]: mon.smithi027 (mon.0) 7 : audit [DBG] from='client.? 172.21.15.27:0/3086094785' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2021-11-03T17:05:55.194 INFO:teuthology.orchestra.run.smithi027.stderr:mgr not available, waiting (1/15)... 2021-11-03T17:05:57.196 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi027 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmp3f8lj10x:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpq2rb3_h7:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 status -f json-pretty 2021-11-03T17:05:57.857 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.860+0000 7fdeeda53700 1 Processor -- start 2021-11-03T17:05:57.857 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.860+0000 7fdeeda53700 1 -- start start 2021-11-03T17:05:57.858 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.860+0000 7fdeeda53700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdee8071c20 0x7fdee80701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-03T17:05:57.858 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.860+0000 7fdeeda53700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee8072040 con 0x7fdee8071c20 2021-11-03T17:05:57.858 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.860+0000 7fdee6ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdee8071c20 0x7fdee80701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:05:57.858 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.860+0000 7fdee6ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdee8071c20 0x7fdee80701c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53968/0 (socket says 172.21.15.27:53968) 2021-11-03T17:05:57.859 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.860+0000 7fdee6ffd700 1 -- 172.21.15.27:0/1451028765 learned_addr learned my addr 172.21.15.27:0/1451028765 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:05:57.859 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.861+0000 7fdee6ffd700 1 -- 172.21.15.27:0/1451028765 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee80706b0 con 0x7fdee8071c20 2021-11-03T17:05:57.859 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.861+0000 7fdee6ffd700 1 --2- 172.21.15.27:0/1451028765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdee8071c20 0x7fdee80701c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fded800b5f0 tx=0x7fded8006ee0).ready entity=mon.0 client_cookie=631555718ee57530 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:05:57.859 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.861+0000 7fdee5ffb700 1 -- 172.21.15.27:0/1451028765 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fded800d600 con 0x7fdee8071c20 2021-11-03T17:05:57.860 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.861+0000 7fdee5ffb700 1 -- 172.21.15.27:0/1451028765 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 983+0+0 (secure 0 0 0) 0x7fded800dbb0 con 0x7fdee8071c20 2021-11-03T17:05:57.860 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.861+0000 7fdee5ffb700 1 -- 172.21.15.27:0/1451028765 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fded8011bb0 con 0x7fdee8071c20 2021-11-03T17:05:57.863 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.862+0000 7fdeeda53700 1 -- 172.21.15.27:0/1451028765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdee8071c20 msgr2=0x7fdee80701c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:05:57.863 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.862+0000 7fdeeda53700 1 --2- 172.21.15.27:0/1451028765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdee8071c20 0x7fdee80701c0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fded800b5f0 tx=0x7fded8006ee0).stop 2021-11-03T17:05:57.864 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.862+0000 7fdeeda53700 1 -- 172.21.15.27:0/1451028765 shutdown_connections 2021-11-03T17:05:57.864 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.862+0000 7fdeeda53700 1 --2- 172.21.15.27:0/1451028765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdee8071c20 0x7fdee80701c0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:05:57.864 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.862+0000 7fdeeda53700 1 -- 172.21.15.27:0/1451028765 >> 172.21.15.27:0/1451028765 conn(0x7fdee80f73e0 msgr2=0x7fdee80f9800 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:05:57.864 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.862+0000 7fdeeda53700 1 -- 172.21.15.27:0/1451028765 shutdown_connections 2021-11-03T17:05:57.865 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.862+0000 7fdeeda53700 1 -- 172.21.15.27:0/1451028765 wait complete. 2021-11-03T17:05:57.865 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.863+0000 7fdeeda53700 1 Processor -- start 2021-11-03T17:05:57.865 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.863+0000 7fdeeda53700 1 -- start start 2021-11-03T17:05:57.865 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.863+0000 7fdeeda53700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdee8071c20 0x7fdee810cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:05:57.866 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.863+0000 7fdeeda53700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee810d020 con 0x7fdee8071c20 2021-11-03T17:05:57.866 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.863+0000 7fdee6ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdee8071c20 0x7fdee810cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:05:57.866 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.863+0000 7fdee6ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdee8071c20 0x7fdee810cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53970/0 (socket says 172.21.15.27:53970) 2021-11-03T17:05:57.866 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.863+0000 7fdee6ffd700 1 -- 172.21.15.27:0/3584616138 learned_addr learned my addr 172.21.15.27:0/3584616138 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:05:57.867 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.863+0000 7fdee6ffd700 1 -- 172.21.15.27:0/3584616138 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded800b200 con 0x7fdee8071c20 2021-11-03T17:05:57.867 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.863+0000 7fdee6ffd700 1 --2- 172.21.15.27:0/3584616138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdee8071c20 0x7fdee810cb30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fded800c010 tx=0x7fded8006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:05:57.867 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.864+0000 7fdecf7fe700 1 -- 172.21.15.27:0/3584616138 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fded801b410 con 0x7fdee8071c20 2021-11-03T17:05:57.867 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.864+0000 7fdecf7fe700 1 -- 172.21.15.27:0/3584616138 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 983+0+0 (secure 0 0 0) 0x7fded801b9c0 con 0x7fdee8071c20 2021-11-03T17:05:57.867 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.864+0000 7fdeeda53700 1 -- 172.21.15.27:0/3584616138 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee810d220 con 0x7fdee8071c20 2021-11-03T17:05:57.868 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.864+0000 7fdecf7fe700 1 -- 172.21.15.27:0/3584616138 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fded801a740 con 0x7fdee8071c20 2021-11-03T17:05:57.868 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.864+0000 7fdecf7fe700 1 -- 172.21.15.27:0/3584616138 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 1) v1 ==== 484+0+0 (secure 0 0 0) 0x7fded800f070 con 0x7fdee8071c20 2021-11-03T17:05:57.868 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.865+0000 7fdeeda53700 1 -- 172.21.15.27:0/3584616138 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee810d6b0 con 0x7fdee8071c20 2021-11-03T17:05:57.868 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.865+0000 7fdecf7fe700 1 -- 172.21.15.27:0/3584616138 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fded801ae80 con 0x7fdee8071c20 2021-11-03T17:05:57.868 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.865+0000 7fdeeda53700 1 -- 172.21.15.27:0/3584616138 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee8045ad0 con 0x7fdee8071c20 2021-11-03T17:05:57.869 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.868+0000 7fdecf7fe700 1 -- 172.21.15.27:0/3584616138 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72097 (secure 0 0 0) 0x7fded8016070 con 0x7fdee8071c20 2021-11-03T17:05:57.996 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.993+0000 7fdeeda53700 1 -- 172.21.15.27:0/3584616138 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fdee8101d90 con 0x7fdee8071c20 2021-11-03T17:05:57.997 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.994+0000 7fdecf7fe700 1 -- 172.21.15.27:0/3584616138 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1230 (secure 0 0 0) 0x7fdee8101d90 con 0x7fdee8071c20 2021-11-03T17:05:57.997 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.997 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: { 2021-11-03T17:05:57.997 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "fsid": "32ab977e-3cc8-11ec-8c28-001a4aab830c", 2021-11-03T17:05:57.998 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "health": { 2021-11-03T17:05:57.998 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2021-11-03T17:05:57.998 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "checks": {}, 2021-11-03T17:05:57.998 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "mutes": [] 2021-11-03T17:05:57.998 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: }, 2021-11-03T17:05:57.999 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "election_epoch": 5, 2021-11-03T17:05:57.999 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "quorum": [ 2021-11-03T17:05:57.999 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 0 2021-11-03T17:05:57.999 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: ], 2021-11-03T17:05:58.000 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "quorum_names": [ 2021-11-03T17:05:58.000 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "smithi027" 2021-11-03T17:05:58.000 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: ], 2021-11-03T17:05:58.000 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "quorum_age": 6, 2021-11-03T17:05:58.001 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "monmap": { 2021-11-03T17:05:58.001 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "epoch": 1, 2021-11-03T17:05:58.001 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "min_mon_release_name": "pacific", 2021-11-03T17:05:58.001 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "num_mons": 1 2021-11-03T17:05:58.001 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: }, 2021-11-03T17:05:58.002 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "osdmap": { 2021-11-03T17:05:58.002 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "epoch": 1, 2021-11-03T17:05:58.002 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "num_osds": 0, 2021-11-03T17:05:58.002 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "num_up_osds": 0, 2021-11-03T17:05:58.003 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "osd_up_since": 0, 2021-11-03T17:05:58.003 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "num_in_osds": 0, 2021-11-03T17:05:58.003 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "osd_in_since": 0, 2021-11-03T17:05:58.003 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2021-11-03T17:05:58.004 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: }, 2021-11-03T17:05:58.004 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "pgmap": { 2021-11-03T17:05:58.004 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "pgs_by_state": [], 2021-11-03T17:05:58.004 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "num_pgs": 0, 2021-11-03T17:05:58.004 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "num_pools": 0, 2021-11-03T17:05:58.005 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "num_objects": 0, 2021-11-03T17:05:58.005 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "data_bytes": 0, 2021-11-03T17:05:58.005 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "bytes_used": 0, 2021-11-03T17:05:58.005 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "bytes_avail": 0, 2021-11-03T17:05:58.006 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "bytes_total": 0 2021-11-03T17:05:58.006 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: }, 2021-11-03T17:05:58.006 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "fsmap": { 2021-11-03T17:05:58.006 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "epoch": 1, 2021-11-03T17:05:58.006 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "by_rank": [], 2021-11-03T17:05:58.007 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "up:standby": 0 2021-11-03T17:05:58.007 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: }, 2021-11-03T17:05:58.007 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "mgrmap": { 2021-11-03T17:05:58.007 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "available": false, 2021-11-03T17:05:58.008 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "num_standbys": 0, 2021-11-03T17:05:58.008 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "modules": [ 2021-11-03T17:05:58.008 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "iostat", 2021-11-03T17:05:58.008 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "restful" 2021-11-03T17:05:58.008 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: ], 2021-11-03T17:05:58.009 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "services": {} 2021-11-03T17:05:58.009 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: }, 2021-11-03T17:05:58.009 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "servicemap": { 2021-11-03T17:05:58.009 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "epoch": 1, 2021-11-03T17:05:58.009 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "modified": "2021-11-03T17:05:46.314793+0000", 2021-11-03T17:05:58.010 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "services": {} 2021-11-03T17:05:58.010 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: }, 2021-11-03T17:05:58.010 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "progress_events": {} 2021-11-03T17:05:58.010 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: } 2021-11-03T17:05:58.010 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.995+0000 7fdeeda53700 1 -- 172.21.15.27:0/3584616138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdee8071c20 msgr2=0x7fdee810cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:05:58.011 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.995+0000 7fdeeda53700 1 --2- 172.21.15.27:0/3584616138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdee8071c20 0x7fdee810cb30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fded800c010 tx=0x7fded8006ee0).stop 2021-11-03T17:05:58.011 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.995+0000 7fdeeda53700 1 -- 172.21.15.27:0/3584616138 shutdown_connections 2021-11-03T17:05:58.011 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.995+0000 7fdeeda53700 1 --2- 172.21.15.27:0/3584616138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdee8071c20 0x7fdee810cb30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:05:58.011 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.995+0000 7fdeeda53700 1 -- 172.21.15.27:0/3584616138 >> 172.21.15.27:0/3584616138 conn(0x7fdee80f73e0 msgr2=0x7fdee80f8ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:05:58.012 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.995+0000 7fdeeda53700 1 -- 172.21.15.27:0/3584616138 shutdown_connections 2021-11-03T17:05:58.012 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:05:57.995+0000 7fdeeda53700 1 -- 172.21.15.27:0/3584616138 wait complete. 2021-11-03T17:05:58.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:58 smithi027 conmon[32322]: audit 2021-11-03T17:05:57.995736+0000 mon.smithi027 (mon 2021-11-03T17:05:58.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:58 smithi027 conmon[32322]: .0) 8 : audit [DBG] from='client.? 172.21.15.27:0/3584616138' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2021-11-03T17:05:58.559 INFO:teuthology.orchestra.run.smithi027.stderr:mgr not available, waiting (2/15)... 2021-11-03T17:06:00.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: cluster 2021-11-03T17:05:59.689384+0000 mon.smithi027 (mon.0) 9 : cluster [INF] Activating manager daemon smithi027.ufhhvb 2021-11-03T17:06:00.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: cluster 2021-11-03T17:05:59.691824+0000 mon.smithi027 (mon.0) 10 : cluster [DBG] mgrmap e2: smithi027.ufhhvb(active, starting, since 0.00253298s) 2021-11-03T17:06:00.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: audit 2021-11-03 2021-11-03T17:06:00.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: T17:05:59.692919+0000 mon.smithi027 (mon.0) 2021-11-03T17:06:00.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: 11 : audit [DBG] from='mgr.14100 172.21.15.27:0/607929312' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata"}]: dispatch 2021-11-03T17:06:00.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: audit 2021 2021-11-03T17:06:00.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: -11-03T17:05:59.693143 2021-11-03T17:06:00.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: +0000 mon.smithi027 (mon.0) 2021-11-03T17:06:00.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: 12 : audit [DBG] from='mgr.14100 172.21.15.27:0/607929312' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata"}]: dispatch 2021-11-03T17:06:00.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: audit 2021 2021-11-03T17:06:00.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: -11-03T17:05:59 2021-11-03T17:06:00.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: .693373+0000 mon.smithi027 (mon. 2021-11-03T17:06:00.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: 0) 13 : audit [DBG] 2021-11-03T17:06:00.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: from='mgr.14100 172.21.15.27:0/607929312' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata"}]: dispatch 2021-11-03T17:06:00.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: audit 2021-11 2021-11-03T17:06:00.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: -03T17:05:59. 2021-11-03T17:06:00.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: 693526+0000 mon.smithi027 (mon.0 2021-11-03T17:06:00.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: ) 14 : audit [DBG] from='mgr.14100 172.21.15.27:0/607929312' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2021-11-03T17:06:00.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: 2021-11-03T17:06:00.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: audit 2021 2021-11-03T17:06:00.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: -11-03T17:05 2021-11-03T17:06:00.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: :59.693824+0000 mon.smithi027 (mon 2021-11-03T17:06:00.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: .0) 15 : audit [DBG] 2021-11-03T17:06:00.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: from='mgr.14100 172.21.15.27:0/607929312' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mgr metadata", "who": "smithi027.ufhhvb", "id": "smithi027.ufhhvb"}]: dispatch 2021-11-03T17:06:00.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: cluster 2021-11-03T17:06:00.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: 2021-11-03T17 2021-11-03T17:06:00.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: :05:59.695966+0000 mon.smithi027 2021-11-03T17:06:00.026 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: (mon.0) 16 : cluster 2021-11-03T17:06:00.026 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: [INF] Manager daemon smithi027.ufhhvb is now available 2021-11-03T17:06:00.026 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: audit 2021 2021-11-03T17:06:00.026 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: -11-03T17:05:59 2021-11-03T17:06:00.027 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: .704343+0000 mon.smithi027 (mon. 2021-11-03T17:06:00.027 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: 0) 17 : audit [INF] 2021-11-03T17:06:00.027 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: from='mgr.14100 172.21.15.27:0/607929312' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:06:00.027 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: audit 2021 2021-11-03T17:06:00.027 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: -11-03T17:05 2021-11-03T17:06:00.028 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: :59.709420+0000 mon.smithi027 ( 2021-11-03T17:06:00.028 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: mon.0) 18 : audit 2021-11-03T17:06:00.028 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: [INF] from='mgr.14100 172.21.15.27:0/607929312' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:06:00.028 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: audit 2021-11-03T17:06:00.028 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: 2021-11-03T17: 2021-11-03T17:06:00.029 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: 05:59.710643+0000 mon.smithi027 2021-11-03T17:06:00.029 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: (mon.0) 19 : audit 2021-11-03T17:06:00.029 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: [INF] from='mgr.14100 172.21.15.27:0/607929312' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:06:00.029 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: audit 2021 2021-11-03T17:06:00.029 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: -11-03T17:05 2021-11-03T17:06:00.030 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: :59.715657+0000 mon.smithi027 ( 2021-11-03T17:06:00.030 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: mon.0) 20 : audit 2021-11-03T17:06:00.030 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: [INF] from='mgr.14100 172.21.15.27:0/607929312' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:06:00.030 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: audit 2021-11-03T17:06:00.031 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: 2021-11-03T17:05 2021-11-03T17:06:00.031 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: :59.717701+0000 mon.smithi027 ( 2021-11-03T17:06:00.031 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: mon.0) 21 : audit 2021-11-03T17:06:00.031 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:05:59 smithi027 conmon[32322]: [INF] from='mgr.14100 172.21.15.27:0/607929312' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:06:00.562 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi027 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmp3f8lj10x:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpq2rb3_h7:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 status -f json-pretty 2021-11-03T17:06:01.272 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.272+0000 7fc18ab79700 1 Processor -- start 2021-11-03T17:06:01.272 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.272+0000 7fc18ab79700 1 -- start start 2021-11-03T17:06:01.272 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.272+0000 7fc18ab79700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc18406fb60 0x7fc1840fe840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:01.273 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.272+0000 7fc18ab79700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1840fed30 con 0x7fc18406fb60 2021-11-03T17:06:01.273 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.273+0000 7fc188915700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc18406fb60 0x7fc1840fe840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:01.273 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.273+0000 7fc188915700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc18406fb60 0x7fc1840fe840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53990/0 (socket says 172.21.15.27:53990) 2021-11-03T17:06:01.273 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.273+0000 7fc188915700 1 -- 172.21.15.27:0/162033668 learned_addr learned my addr 172.21.15.27:0/162033668 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:01.274 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.273+0000 7fc188915700 1 -- 172.21.15.27:0/162033668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1840fee70 con 0x7fc18406fb60 2021-11-03T17:06:01.274 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.273+0000 7fc188915700 1 --2- 172.21.15.27:0/162033668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc18406fb60 0x7fc1840fe840 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fc17c009960 tx=0x7fc17c0092d0).ready entity=mon.0 client_cookie=e4911d2053dd87d6 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:01.274 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.273+0000 7fc1837fe700 1 -- 172.21.15.27:0/162033668 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc17c00b9c0 con 0x7fc18406fb60 2021-11-03T17:06:01.274 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.273+0000 7fc1837fe700 1 -- 172.21.15.27:0/162033668 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 983+0+0 (secure 0 0 0) 0x7fc17c011400 con 0x7fc18406fb60 2021-11-03T17:06:01.274 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.273+0000 7fc1837fe700 1 -- 172.21.15.27:0/162033668 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc17c010580 con 0x7fc18406fb60 2021-11-03T17:06:01.275 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.274+0000 7fc18ab79700 1 -- 172.21.15.27:0/162033668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc18406fb60 msgr2=0x7fc1840fe840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:01.275 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.274+0000 7fc18ab79700 1 --2- 172.21.15.27:0/162033668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc18406fb60 0x7fc1840fe840 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fc17c009960 tx=0x7fc17c0092d0).stop 2021-11-03T17:06:01.275 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.274+0000 7fc18ab79700 1 -- 172.21.15.27:0/162033668 shutdown_connections 2021-11-03T17:06:01.275 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.274+0000 7fc18ab79700 1 --2- 172.21.15.27:0/162033668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc18406fb60 0x7fc1840fe840 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:01.276 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.274+0000 7fc18ab79700 1 -- 172.21.15.27:0/162033668 >> 172.21.15.27:0/162033668 conn(0x7fc1840f6260 msgr2=0x7fc1840f86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:01.276 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.274+0000 7fc18ab79700 1 -- 172.21.15.27:0/162033668 shutdown_connections 2021-11-03T17:06:01.276 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.274+0000 7fc18ab79700 1 -- 172.21.15.27:0/162033668 wait complete. 2021-11-03T17:06:01.276 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.274+0000 7fc18ab79700 1 Processor -- start 2021-11-03T17:06:01.277 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.274+0000 7fc18ab79700 1 -- start start 2021-11-03T17:06:01.277 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.275+0000 7fc18ab79700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc18406fb60 0x7fc1840fc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:01.277 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.275+0000 7fc18ab79700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1840fa740 con 0x7fc18406fb60 2021-11-03T17:06:01.277 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.275+0000 7fc188915700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc18406fb60 0x7fc1840fc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:01.277 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.275+0000 7fc188915700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc18406fb60 0x7fc1840fc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53992/0 (socket says 172.21.15.27:53992) 2021-11-03T17:06:01.278 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.275+0000 7fc188915700 1 -- 172.21.15.27:0/247250851 learned_addr learned my addr 172.21.15.27:0/247250851 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:01.278 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.275+0000 7fc188915700 1 -- 172.21.15.27:0/247250851 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc17c009570 con 0x7fc18406fb60 2021-11-03T17:06:01.278 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.275+0000 7fc188915700 1 --2- 172.21.15.27:0/247250851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc18406fb60 0x7fc1840fc2c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7fc17c00bac0 tx=0x7fc17c0092d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:01.278 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.275+0000 7fc1817fa700 1 -- 172.21.15.27:0/247250851 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc17c0107f0 con 0x7fc18406fb60 2021-11-03T17:06:01.279 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.275+0000 7fc1817fa700 1 -- 172.21.15.27:0/247250851 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 983+0+0 (secure 0 0 0) 0x7fc17c010da0 con 0x7fc18406fb60 2021-11-03T17:06:01.279 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.275+0000 7fc18ab79700 1 -- 172.21.15.27:0/247250851 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1840fa8e0 con 0x7fc18406fb60 2021-11-03T17:06:01.279 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.275+0000 7fc1817fa700 1 -- 172.21.15.27:0/247250851 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc17c01ad20 con 0x7fc18406fb60 2021-11-03T17:06:01.279 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.275+0000 7fc18ab79700 1 -- 172.21.15.27:0/247250851 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1840fad70 con 0x7fc18406fb60 2021-11-03T17:06:01.280 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.276+0000 7fc1817fa700 1 -- 172.21.15.27:0/247250851 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 3) v1 ==== 36029+0+0 (secure 0 0 0) 0x7fc17c00f070 con 0x7fc18406fb60 2021-11-03T17:06:01.281 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.277+0000 7fc1817fa700 1 --2- 172.21.15.27:0/247250851 >> [v2:172.21.15.27:6800/388626148,v1:172.21.15.27:6801/388626148] conn(0x7fc17002fc70 0x7fc170032130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:01.281 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.277+0000 7fc18ab79700 1 -- 172.21.15.27:0/247250851 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc184045ad0 con 0x7fc18406fb60 2021-11-03T17:06:01.281 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.277+0000 7fc183fff700 1 --2- 172.21.15.27:0/247250851 >> [v2:172.21.15.27:6800/388626148,v1:172.21.15.27:6801/388626148] conn(0x7fc17002fc70 0x7fc170032130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:01.281 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.277+0000 7fc1817fa700 1 -- 172.21.15.27:0/247250851 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fc17c0418d0 con 0x7fc18406fb60 2021-11-03T17:06:01.282 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.277+0000 7fc183fff700 1 --2- 172.21.15.27:0/247250851 >> [v2:172.21.15.27:6800/388626148,v1:172.21.15.27:6801/388626148] conn(0x7fc17002fc70 0x7fc170032130 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7fc1780096a0 tx=0x7fc178006b40).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:01.282 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.281+0000 7fc1817fa700 1 -- 172.21.15.27:0/247250851 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fc17c018500 con 0x7fc18406fb60 2021-11-03T17:06:01.684 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.679+0000 7fc18ab79700 1 -- 172.21.15.27:0/247250851 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fc184027a00 con 0x7fc18406fb60 2021-11-03T17:06:01.684 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.680+0000 7fc1817fa700 1 -- 172.21.15.27:0/247250851 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1230 (secure 0 0 0) 0x7fc17c02b9f0 con 0x7fc18406fb60 2021-11-03T17:06:01.684 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.684 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: { 2021-11-03T17:06:01.685 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "fsid": "32ab977e-3cc8-11ec-8c28-001a4aab830c", 2021-11-03T17:06:01.685 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "health": { 2021-11-03T17:06:01.685 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "status": "HEALTH_OK", 2021-11-03T17:06:01.685 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "checks": {}, 2021-11-03T17:06:01.686 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "mutes": [] 2021-11-03T17:06:01.686 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: }, 2021-11-03T17:06:01.686 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "election_epoch": 5, 2021-11-03T17:06:01.686 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "quorum": [ 2021-11-03T17:06:01.687 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 0 2021-11-03T17:06:01.687 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: ], 2021-11-03T17:06:01.687 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "quorum_names": [ 2021-11-03T17:06:01.687 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "smithi027" 2021-11-03T17:06:01.688 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: ], 2021-11-03T17:06:01.688 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "quorum_age": 10, 2021-11-03T17:06:01.688 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "monmap": { 2021-11-03T17:06:01.688 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "epoch": 1, 2021-11-03T17:06:01.689 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "min_mon_release_name": "pacific", 2021-11-03T17:06:01.689 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "num_mons": 1 2021-11-03T17:06:01.689 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: }, 2021-11-03T17:06:01.689 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "osdmap": { 2021-11-03T17:06:01.690 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "epoch": 1, 2021-11-03T17:06:01.690 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "num_osds": 0, 2021-11-03T17:06:01.690 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "num_up_osds": 0, 2021-11-03T17:06:01.690 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "osd_up_since": 0, 2021-11-03T17:06:01.690 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "num_in_osds": 0, 2021-11-03T17:06:01.691 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "osd_in_since": 0, 2021-11-03T17:06:01.691 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "num_remapped_pgs": 0 2021-11-03T17:06:01.691 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: }, 2021-11-03T17:06:01.691 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "pgmap": { 2021-11-03T17:06:01.692 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "pgs_by_state": [], 2021-11-03T17:06:01.692 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "num_pgs": 0, 2021-11-03T17:06:01.692 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "num_pools": 0, 2021-11-03T17:06:01.692 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "num_objects": 0, 2021-11-03T17:06:01.693 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "data_bytes": 0, 2021-11-03T17:06:01.693 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "bytes_used": 0, 2021-11-03T17:06:01.693 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "bytes_avail": 0, 2021-11-03T17:06:01.693 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "bytes_total": 0 2021-11-03T17:06:01.693 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: }, 2021-11-03T17:06:01.694 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "fsmap": { 2021-11-03T17:06:01.694 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "epoch": 1, 2021-11-03T17:06:01.694 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "by_rank": [], 2021-11-03T17:06:01.694 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "up:standby": 0 2021-11-03T17:06:01.695 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: }, 2021-11-03T17:06:01.695 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "mgrmap": { 2021-11-03T17:06:01.695 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "available": true, 2021-11-03T17:06:01.695 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "num_standbys": 0, 2021-11-03T17:06:01.696 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "modules": [ 2021-11-03T17:06:01.696 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "iostat", 2021-11-03T17:06:01.696 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "restful" 2021-11-03T17:06:01.696 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: ], 2021-11-03T17:06:01.696 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "services": {} 2021-11-03T17:06:01.697 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: }, 2021-11-03T17:06:01.697 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "servicemap": { 2021-11-03T17:06:01.697 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "epoch": 1, 2021-11-03T17:06:01.697 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "modified": "2021-11-03T17:05:46.314793+0000", 2021-11-03T17:06:01.698 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "services": {} 2021-11-03T17:06:01.698 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: }, 2021-11-03T17:06:01.698 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "progress_events": {} 2021-11-03T17:06:01.699 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: } 2021-11-03T17:06:01.699 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.682+0000 7fc18ab79700 1 -- 172.21.15.27:0/247250851 >> [v2:172.21.15.27:6800/388626148,v1:172.21.15.27:6801/388626148] conn(0x7fc17002fc70 msgr2=0x7fc170032130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:01.699 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.682+0000 7fc18ab79700 1 --2- 172.21.15.27:0/247250851 >> [v2:172.21.15.27:6800/388626148,v1:172.21.15.27:6801/388626148] conn(0x7fc17002fc70 0x7fc170032130 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7fc1780096a0 tx=0x7fc178006b40).stop 2021-11-03T17:06:01.699 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.682+0000 7fc18ab79700 1 -- 172.21.15.27:0/247250851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc18406fb60 msgr2=0x7fc1840fc2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:01.700 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.682+0000 7fc18ab79700 1 --2- 172.21.15.27:0/247250851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc18406fb60 0x7fc1840fc2c0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7fc17c00bac0 tx=0x7fc17c0092d0).stop 2021-11-03T17:06:01.700 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.682+0000 7fc18ab79700 1 -- 172.21.15.27:0/247250851 shutdown_connections 2021-11-03T17:06:01.700 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.682+0000 7fc18ab79700 1 --2- 172.21.15.27:0/247250851 >> [v2:172.21.15.27:6800/388626148,v1:172.21.15.27:6801/388626148] conn(0x7fc17002fc70 0x7fc170032130 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:01.700 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.682+0000 7fc18ab79700 1 --2- 172.21.15.27:0/247250851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc18406fb60 0x7fc1840fc2c0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:01.700 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.682+0000 7fc18ab79700 1 -- 172.21.15.27:0/247250851 >> 172.21.15.27:0/247250851 conn(0x7fc1840f6260 msgr2=0x7fc1840f7d30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:01.701 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.682+0000 7fc18ab79700 1 -- 172.21.15.27:0/247250851 shutdown_connections 2021-11-03T17:06:01.701 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:01.682+0000 7fc18ab79700 1 -- 172.21.15.27:0/247250851 wait complete. 2021-11-03T17:06:01.701 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:01 smithi027 conmon[32322]: cluster 2021-11-03T17:06:00.694712+0000 mon.smithi027 (mon.0) 22 : cluster [DBG] mgrmap e3: smithi027.ufhhvb(active, since 1.00542s) 2021-11-03T17:06:01.702 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:01 smithi027 conmon[32322]: audit 2021-11-03T17:06:01.681354+0000 mon.smithi027 (mon.0) 23 : audit [DBG] from='client.? 172.21.15.27:0/247250851' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2021-11-03T17:06:02.326 INFO:teuthology.orchestra.run.smithi027.stderr:mgr is available 2021-11-03T17:06:02.327 INFO:teuthology.orchestra.run.smithi027.stderr:Enabling cephadm module... 2021-11-03T17:06:02.327 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi027 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmp3f8lj10x:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpq2rb3_h7:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 mgr module enable cephadm 2021-11-03T17:06:03.033 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.033+0000 7fb128d60700 1 Processor -- start 2021-11-03T17:06:03.033 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.033+0000 7fb128d60700 1 -- start start 2021-11-03T17:06:03.033 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.033+0000 7fb128d60700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1240fab40 0x7fb1240faf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:03.034 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.033+0000 7fb128d60700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1240fb4e0 con 0x7fb1240fab40 2021-11-03T17:06:03.034 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.033+0000 7fb12259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1240fab40 0x7fb1240faf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:03.034 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.033+0000 7fb12259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1240fab40 0x7fb1240faf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53996/0 (socket says 172.21.15.27:53996) 2021-11-03T17:06:03.034 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.033+0000 7fb12259c700 1 -- 172.21.15.27:0/3701337619 learned_addr learned my addr 172.21.15.27:0/3701337619 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:03.035 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.034+0000 7fb12259c700 1 -- 172.21.15.27:0/3701337619 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1240fb620 con 0x7fb1240fab40 2021-11-03T17:06:03.035 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.034+0000 7fb12259c700 1 --2- 172.21.15.27:0/3701337619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1240fab40 0x7fb1240faf60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7fb11400b5f0 tx=0x7fb114006ee0).ready entity=mon.0 client_cookie=788291eea66d3ff2 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:03.035 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.034+0000 7fb12159a700 1 -- 172.21.15.27:0/3701337619 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb11400d600 con 0x7fb1240fab40 2021-11-03T17:06:03.035 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.034+0000 7fb12159a700 1 -- 172.21.15.27:0/3701337619 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 983+0+0 (secure 0 0 0) 0x7fb11400dbb0 con 0x7fb1240fab40 2021-11-03T17:06:03.036 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.034+0000 7fb12159a700 1 -- 172.21.15.27:0/3701337619 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb114011bb0 con 0x7fb1240fab40 2021-11-03T17:06:03.036 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.035+0000 7fb128d60700 1 -- 172.21.15.27:0/3701337619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1240fab40 msgr2=0x7fb1240faf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:03.036 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.035+0000 7fb128d60700 1 --2- 172.21.15.27:0/3701337619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1240fab40 0x7fb1240faf60 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7fb11400b5f0 tx=0x7fb114006ee0).stop 2021-11-03T17:06:03.036 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.035+0000 7fb128d60700 1 -- 172.21.15.27:0/3701337619 shutdown_connections 2021-11-03T17:06:03.037 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.035+0000 7fb128d60700 1 --2- 172.21.15.27:0/3701337619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1240fab40 0x7fb1240faf60 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:03.037 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.035+0000 7fb128d60700 1 -- 172.21.15.27:0/3701337619 >> 172.21.15.27:0/3701337619 conn(0x7fb1240f62a0 msgr2=0x7fb1240f86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:03.037 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.035+0000 7fb128d60700 1 -- 172.21.15.27:0/3701337619 shutdown_connections 2021-11-03T17:06:03.037 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.035+0000 7fb128d60700 1 -- 172.21.15.27:0/3701337619 wait complete. 2021-11-03T17:06:03.037 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.035+0000 7fb128d60700 1 Processor -- start 2021-11-03T17:06:03.038 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.035+0000 7fb128d60700 1 -- start start 2021-11-03T17:06:03.038 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.036+0000 7fb128d60700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1240fab40 0x7fb124100e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:03.038 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.036+0000 7fb128d60700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1240ff2d0 con 0x7fb1240fab40 2021-11-03T17:06:03.038 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.036+0000 7fb12259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1240fab40 0x7fb124100e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:03.039 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.036+0000 7fb12259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1240fab40 0x7fb124100e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:53998/0 (socket says 172.21.15.27:53998) 2021-11-03T17:06:03.039 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.036+0000 7fb12259c700 1 -- 172.21.15.27:0/144755400 learned_addr learned my addr 172.21.15.27:0/144755400 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:03.039 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.036+0000 7fb12259c700 1 -- 172.21.15.27:0/144755400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb11400b200 con 0x7fb1240fab40 2021-11-03T17:06:03.039 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.036+0000 7fb12259c700 1 --2- 172.21.15.27:0/144755400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1240fab40 0x7fb124100e50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fb11400db50 tx=0x7fb114006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:03.040 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.036+0000 7fb112ffd700 1 -- 172.21.15.27:0/144755400 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb11401b410 con 0x7fb1240fab40 2021-11-03T17:06:03.040 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.036+0000 7fb112ffd700 1 -- 172.21.15.27:0/144755400 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 983+0+0 (secure 0 0 0) 0x7fb11401b9c0 con 0x7fb1240fab40 2021-11-03T17:06:03.040 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.036+0000 7fb112ffd700 1 -- 172.21.15.27:0/144755400 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb11401a6d0 con 0x7fb1240fab40 2021-11-03T17:06:03.040 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.036+0000 7fb128d60700 1 -- 172.21.15.27:0/144755400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1240ff470 con 0x7fb1240fab40 2021-11-03T17:06:03.040 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.037+0000 7fb128d60700 1 -- 172.21.15.27:0/144755400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1240ff900 con 0x7fb1240fab40 2021-11-03T17:06:03.042 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.038+0000 7fb128d60700 1 -- 172.21.15.27:0/144755400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb124045ad0 con 0x7fb1240fab40 2021-11-03T17:06:03.042 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.038+0000 7fb112ffd700 1 -- 172.21.15.27:0/144755400 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 4) v1 ==== 36109+0+0 (secure 0 0 0) 0x7fb11400f070 con 0x7fb1240fab40 2021-11-03T17:06:03.042 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.038+0000 7fb112ffd700 1 --2- 172.21.15.27:0/144755400 >> [v2:172.21.15.27:6800/388626148,v1:172.21.15.27:6801/388626148] conn(0x7fb10802fdb0 0x7fb108032270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:03.042 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.038+0000 7fb112ffd700 1 -- 172.21.15.27:0/144755400 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fb1140412f0 con 0x7fb1240fab40 2021-11-03T17:06:03.043 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.039+0000 7fb121d9b700 1 --2- 172.21.15.27:0/144755400 >> [v2:172.21.15.27:6800/388626148,v1:172.21.15.27:6801/388626148] conn(0x7fb10802fdb0 0x7fb108032270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:03.043 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.039+0000 7fb121d9b700 1 --2- 172.21.15.27:0/144755400 >> [v2:172.21.15.27:6800/388626148,v1:172.21.15.27:6801/388626148] conn(0x7fb10802fdb0 0x7fb108032270 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fb1180096a0 tx=0x7fb118006b40).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:03.043 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.043+0000 7fb112ffd700 1 -- 172.21.15.27:0/144755400 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fb114019d40 con 0x7fb1240fab40 2021-11-03T17:06:03.400 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.404+0000 7fb128d60700 1 -- 172.21.15.27:0/144755400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fb124104af0 con 0x7fb1240fab40 2021-11-03T17:06:03.501 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.505+0000 7fb112ffd700 1 -- 172.21.15.27:0/144755400 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 5) v1 ==== 36120+0+0 (secure 0 0 0) 0x7fb11401a850 con 0x7fb1240fab40 2021-11-03T17:06:03.502 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.506+0000 7fb112ffd700 1 -- 172.21.15.27:0/144755400 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fb114023b50 con 0x7fb1240fab40 2021-11-03T17:06:03.505 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.508+0000 7fb128d60700 1 -- 172.21.15.27:0/144755400 >> [v2:172.21.15.27:6800/388626148,v1:172.21.15.27:6801/388626148] conn(0x7fb10802fdb0 msgr2=0x7fb108032270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:03.506 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.508+0000 7fb128d60700 1 --2- 172.21.15.27:0/144755400 >> [v2:172.21.15.27:6800/388626148,v1:172.21.15.27:6801/388626148] conn(0x7fb10802fdb0 0x7fb108032270 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fb1180096a0 tx=0x7fb118006b40).stop 2021-11-03T17:06:03.506 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.509+0000 7fb128d60700 1 -- 172.21.15.27:0/144755400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1240fab40 msgr2=0x7fb124100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:03.506 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.509+0000 7fb128d60700 1 --2- 172.21.15.27:0/144755400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1240fab40 0x7fb124100e50 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fb11400db50 tx=0x7fb114006ee0).stop 2021-11-03T17:06:03.507 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.509+0000 7fb128d60700 1 -- 172.21.15.27:0/144755400 shutdown_connections 2021-11-03T17:06:03.507 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.509+0000 7fb128d60700 1 --2- 172.21.15.27:0/144755400 >> [v2:172.21.15.27:6800/388626148,v1:172.21.15.27:6801/388626148] conn(0x7fb10802fdb0 0x7fb108032270 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:03.507 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.509+0000 7fb128d60700 1 --2- 172.21.15.27:0/144755400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1240fab40 0x7fb124100e50 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:03.507 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.509+0000 7fb128d60700 1 -- 172.21.15.27:0/144755400 >> 172.21.15.27:0/144755400 conn(0x7fb1240f62a0 msgr2=0x7fb12406f7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:03.508 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.509+0000 7fb128d60700 1 -- 172.21.15.27:0/144755400 shutdown_connections 2021-11-03T17:06:03.508 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:03.509+0000 7fb128d60700 1 -- 172.21.15.27:0/144755400 wait complete. 2021-11-03T17:06:03.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:03 smithi027 conmon[32322]: cluster 2021-11-03T17:06:02.507074+0000 mon.smithi027 (mon.0) 24 : cluster [DBG] mgrmap e4: smithi027.ufhhvb(active, since 2s) 2021-11-03T17:06:03.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:03 smithi027 conmon[32322]: audit 2021-11-03T17:06:03.405881+0000 mon.smithi027 (mon.0) 25 : audit [INF] from='client.? 172.21.15.27:0/144755400' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2021-11-03T17:06:04.134 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi027 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmp3f8lj10x:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpq2rb3_h7:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 mgr stat 2021-11-03T17:06:04.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:04 smithi027 conmon[32322]: audit 2021-11-03T17:06:03.508315+0000 2021-11-03T17:06:04.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:04 smithi027 conmon[32322]: mon.smithi027 (mon.0) 26 : audit [INF] from='client.? 172.21.15.27:0/144755400' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2021-11-03T17:06:04.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:04 smithi027 conmon[32322]: cluster 2021-11-03T17:06:03.508383+0000 mon.smithi027 (mon.0) 27 : cluster [DBG] mgrmap e5: smithi027.ufhhvb(active, since 3s) 2021-11-03T17:06:04.801 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.802+0000 7f073c8c6700 1 Processor -- start 2021-11-03T17:06:04.802 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.802+0000 7f073c8c6700 1 -- start start 2021-11-03T17:06:04.802 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.802+0000 7f073c8c6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f073806fbc0 0x7f07380fe630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:04.802 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.802+0000 7f073c8c6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07380feb20 con 0x7f073806fbc0 2021-11-03T17:06:04.802 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.802+0000 7f0736d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f073806fbc0 0x7f07380fe630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:04.803 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.802+0000 7f0736d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f073806fbc0 0x7f07380fe630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54012/0 (socket says 172.21.15.27:54012) 2021-11-03T17:06:04.803 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.802+0000 7f0736d9d700 1 -- 172.21.15.27:0/356473636 learned_addr learned my addr 172.21.15.27:0/356473636 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:04.803 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.803+0000 7f0736d9d700 1 -- 172.21.15.27:0/356473636 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07380fec60 con 0x7f073806fbc0 2021-11-03T17:06:04.803 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.803+0000 7f0736d9d700 1 --2- 172.21.15.27:0/356473636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f073806fbc0 0x7f07380fe630 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f072c00b5f0 tx=0x7f072c006ee0).ready entity=mon.0 client_cookie=6251ba753e0b71ea server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:04.804 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.803+0000 7f0735d9b700 1 -- 172.21.15.27:0/356473636 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f072c00d600 con 0x7f073806fbc0 2021-11-03T17:06:04.804 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.803+0000 7f0735d9b700 1 -- 172.21.15.27:0/356473636 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 983+0+0 (secure 0 0 0) 0x7f072c00dbb0 con 0x7f073806fbc0 2021-11-03T17:06:04.804 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.803+0000 7f0735d9b700 1 -- 172.21.15.27:0/356473636 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f072c011bb0 con 0x7f073806fbc0 2021-11-03T17:06:04.804 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.804+0000 7f073c8c6700 1 -- 172.21.15.27:0/356473636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f073806fbc0 msgr2=0x7f07380fe630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:04.805 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.804+0000 7f073c8c6700 1 --2- 172.21.15.27:0/356473636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f073806fbc0 0x7f07380fe630 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f072c00b5f0 tx=0x7f072c006ee0).stop 2021-11-03T17:06:04.805 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.804+0000 7f073c8c6700 1 -- 172.21.15.27:0/356473636 shutdown_connections 2021-11-03T17:06:04.805 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.804+0000 7f073c8c6700 1 --2- 172.21.15.27:0/356473636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f073806fbc0 0x7f07380fe630 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:04.805 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.804+0000 7f073c8c6700 1 -- 172.21.15.27:0/356473636 >> 172.21.15.27:0/356473636 conn(0x7f07380f6110 msgr2=0x7f07380f8530 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:04.805 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.804+0000 7f073c8c6700 1 -- 172.21.15.27:0/356473636 shutdown_connections 2021-11-03T17:06:04.806 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.804+0000 7f073c8c6700 1 -- 172.21.15.27:0/356473636 wait complete. 2021-11-03T17:06:04.806 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.804+0000 7f073c8c6700 1 Processor -- start 2021-11-03T17:06:04.806 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.805+0000 7f073c8c6700 1 -- start start 2021-11-03T17:06:04.806 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.805+0000 7f073c8c6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f073806fbc0 0x7f07380fc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:04.807 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.805+0000 7f073c8c6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07380fa530 con 0x7f073806fbc0 2021-11-03T17:06:04.807 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.805+0000 7f0736d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f073806fbc0 0x7f07380fc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:04.808 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.805+0000 7f0736d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f073806fbc0 0x7f07380fc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54014/0 (socket says 172.21.15.27:54014) 2021-11-03T17:06:04.808 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.805+0000 7f0736d9d700 1 -- 172.21.15.27:0/2158952254 learned_addr learned my addr 172.21.15.27:0/2158952254 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:04.809 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.805+0000 7f0736d9d700 1 -- 172.21.15.27:0/2158952254 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f072c00b200 con 0x7f073806fbc0 2021-11-03T17:06:04.809 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.805+0000 7f0736d9d700 1 --2- 172.21.15.27:0/2158952254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f073806fbc0 0x7f07380fc0b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f072c006cb0 tx=0x7f072c006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:04.809 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.805+0000 7f071f7fe700 1 -- 172.21.15.27:0/2158952254 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f072c01b570 con 0x7f073806fbc0 2021-11-03T17:06:04.810 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.806+0000 7f073c8c6700 1 -- 172.21.15.27:0/2158952254 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07380fa6d0 con 0x7f073806fbc0 2021-11-03T17:06:04.810 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.806+0000 7f071f7fe700 1 -- 172.21.15.27:0/2158952254 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 983+0+0 (secure 0 0 0) 0x7f072c01bb20 con 0x7f073806fbc0 2021-11-03T17:06:04.810 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.806+0000 7f071f7fe700 1 -- 172.21.15.27:0/2158952254 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f072c01a960 con 0x7f073806fbc0 2021-11-03T17:06:04.810 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.806+0000 7f073c8c6700 1 -- 172.21.15.27:0/2158952254 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07380fab60 con 0x7f073806fbc0 2021-11-03T17:06:04.811 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.806+0000 7f071f7fe700 1 -- 172.21.15.27:0/2158952254 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 36120+0+0 (secure 0 0 0) 0x7f072c00f070 con 0x7f073806fbc0 2021-11-03T17:06:04.811 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.807+0000 7f071f7fe700 1 --2- 172.21.15.27:0/2158952254 >> [v2:172.21.15.27:6800/388626148,v1:172.21.15.27:6801/388626148] conn(0x7f072402fd50 0x7f0724032210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:04.811 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.807+0000 7f071f7fe700 1 -- 172.21.15.27:0/2158952254 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f072c041110 con 0x7f073806fbc0 2021-11-03T17:06:04.811 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.807+0000 7f073659c700 1 -- 172.21.15.27:0/2158952254 >> [v2:172.21.15.27:6800/388626148,v1:172.21.15.27:6801/388626148] conn(0x7f072402fd50 msgr2=0x7f0724032210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/388626148 2021-11-03T17:06:04.812 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.807+0000 7f073659c700 1 --2- 172.21.15.27:0/2158952254 >> [v2:172.21.15.27:6800/388626148,v1:172.21.15.27:6801/388626148] conn(0x7f072402fd50 0x7f0724032210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2021-11-03T17:06:04.812 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.807+0000 7f073c8c6700 1 -- 172.21.15.27:0/2158952254 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0738045ad0 con 0x7f073806fbc0 2021-11-03T17:06:04.812 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:04.811+0000 7f071f7fe700 1 -- 172.21.15.27:0/2158952254 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f072c016070 con 0x7f073806fbc0 2021-11-03T17:06:05.005 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:05.008+0000 7f073659c700 1 -- 172.21.15.27:0/2158952254 >> [v2:172.21.15.27:6800/388626148,v1:172.21.15.27:6801/388626148] conn(0x7f072402fd50 msgr2=0x7f0724032210 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/388626148 2021-11-03T17:06:05.006 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:05.008+0000 7f073659c700 1 --2- 172.21.15.27:0/2158952254 >> [v2:172.21.15.27:6800/388626148,v1:172.21.15.27:6801/388626148] conn(0x7f072402fd50 0x7f0724032210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2021-11-03T17:06:05.195 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:05.198+0000 7f073c8c6700 1 -- 172.21.15.27:0/2158952254 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f07381045b0 con 0x7f073806fbc0 2021-11-03T17:06:05.195 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:05.198+0000 7f071f7fe700 1 -- 172.21.15.27:0/2158952254 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7f072c010be0 con 0x7f073806fbc0 2021-11-03T17:06:05.195 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: { 2021-11-03T17:06:05.196 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "epoch": 5, 2021-11-03T17:06:05.196 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "available": true, 2021-11-03T17:06:05.196 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "active_name": "smithi027.ufhhvb", 2021-11-03T17:06:05.196 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "num_standby": 0 2021-11-03T17:06:05.197 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: } 2021-11-03T17:06:05.198 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:05.200+0000 7f073c8c6700 1 -- 172.21.15.27:0/2158952254 >> [v2:172.21.15.27:6800/388626148,v1:172.21.15.27:6801/388626148] conn(0x7f072402fd50 msgr2=0x7f0724032210 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2021-11-03T17:06:05.198 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:05.200+0000 7f073c8c6700 1 --2- 172.21.15.27:0/2158952254 >> [v2:172.21.15.27:6800/388626148,v1:172.21.15.27:6801/388626148] conn(0x7f072402fd50 0x7f0724032210 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:06:05.199 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:05.200+0000 7f073c8c6700 1 -- 172.21.15.27:0/2158952254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f073806fbc0 msgr2=0x7f07380fc0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:05.199 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:05.200+0000 7f073c8c6700 1 --2- 172.21.15.27:0/2158952254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f073806fbc0 0x7f07380fc0b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f072c006cb0 tx=0x7f072c006ee0).stop 2021-11-03T17:06:05.199 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:05.200+0000 7f073c8c6700 1 -- 172.21.15.27:0/2158952254 shutdown_connections 2021-11-03T17:06:05.199 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:05.200+0000 7f073c8c6700 1 --2- 172.21.15.27:0/2158952254 >> [v2:172.21.15.27:6800/388626148,v1:172.21.15.27:6801/388626148] conn(0x7f072402fd50 0x7f0724032210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:06:05.200 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:05.200+0000 7f073c8c6700 1 --2- 172.21.15.27:0/2158952254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f073806fbc0 0x7f07380fc0b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:05.200 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:05.200+0000 7f073c8c6700 1 -- 172.21.15.27:0/2158952254 >> 172.21.15.27:0/2158952254 conn(0x7f07380f6110 msgr2=0x7f07380f7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:05.200 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:05.201+0000 7f073c8c6700 1 -- 172.21.15.27:0/2158952254 shutdown_connections 2021-11-03T17:06:05.201 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:05.201+0000 7f073c8c6700 1 -- 172.21.15.27:0/2158952254 wait complete. 2021-11-03T17:06:05.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:05 smithi027 conmon[32322]: audit 2021-11-03T17:06:05.199969+0000 mon.smithi027 2021-11-03T17:06:05.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:05 smithi027 conmon[32322]: (mon.0) 28 : audit [DBG] from='client.? 172.21.15.27:0/2158952254' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2021-11-03T17:06:05.926 INFO:teuthology.orchestra.run.smithi027.stderr:Waiting for the mgr to restart... 2021-11-03T17:06:05.926 INFO:teuthology.orchestra.run.smithi027.stderr:Waiting for mgr epoch 5... 2021-11-03T17:06:05.926 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi027 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmp3f8lj10x:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpq2rb3_h7:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 tell mgr mgr_status 2021-11-03T17:06:06.596 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.598+0000 7f4c3c6a7700 1 Processor -- start 2021-11-03T17:06:06.597 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.598+0000 7f4c3c6a7700 1 -- start start 2021-11-03T17:06:06.597 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.598+0000 7f4c3c6a7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c340faa80 0x7f4c340faea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:06.597 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.598+0000 7f4c3c6a7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c340fb390 con 0x7f4c340faa80 2021-11-03T17:06:06.598 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.598+0000 7f4c3a443700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c340faa80 0x7f4c340faea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:06.598 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.598+0000 7f4c3a443700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c340faa80 0x7f4c340faea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54024/0 (socket says 172.21.15.27:54024) 2021-11-03T17:06:06.598 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.598+0000 7f4c3a443700 1 -- 172.21.15.27:0/2059650719 learned_addr learned my addr 172.21.15.27:0/2059650719 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:06.598 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.599+0000 7f4c3a443700 1 -- 172.21.15.27:0/2059650719 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c340fb4d0 con 0x7f4c340faa80 2021-11-03T17:06:06.599 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.599+0000 7f4c3a443700 1 --2- 172.21.15.27:0/2059650719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c340faa80 0x7f4c340faea0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f4c3000b5f0 tx=0x7f4c30006ee0).ready entity=mon.0 client_cookie=bdd9e1ea5f991d37 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:06.599 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.599+0000 7f4c39441700 1 -- 172.21.15.27:0/2059650719 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4c3000d600 con 0x7f4c340faa80 2021-11-03T17:06:06.599 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.599+0000 7f4c39441700 1 -- 172.21.15.27:0/2059650719 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 983+0+0 (secure 0 0 0) 0x7f4c3000dbb0 con 0x7f4c340faa80 2021-11-03T17:06:06.599 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.599+0000 7f4c39441700 1 -- 172.21.15.27:0/2059650719 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4c30011bb0 con 0x7f4c340faa80 2021-11-03T17:06:06.600 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.600+0000 7f4c3c6a7700 1 -- 172.21.15.27:0/2059650719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c340faa80 msgr2=0x7f4c340faea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:06.600 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.600+0000 7f4c3c6a7700 1 --2- 172.21.15.27:0/2059650719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c340faa80 0x7f4c340faea0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f4c3000b5f0 tx=0x7f4c30006ee0).stop 2021-11-03T17:06:06.600 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.600+0000 7f4c3c6a7700 1 -- 172.21.15.27:0/2059650719 shutdown_connections 2021-11-03T17:06:06.600 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.600+0000 7f4c3c6a7700 1 --2- 172.21.15.27:0/2059650719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c340faa80 0x7f4c340faea0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:06.601 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.600+0000 7f4c3c6a7700 1 -- 172.21.15.27:0/2059650719 >> 172.21.15.27:0/2059650719 conn(0x7f4c340f6260 msgr2=0x7f4c340f86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:06.601 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.600+0000 7f4c3c6a7700 1 -- 172.21.15.27:0/2059650719 shutdown_connections 2021-11-03T17:06:06.601 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.600+0000 7f4c3c6a7700 1 -- 172.21.15.27:0/2059650719 wait complete. 2021-11-03T17:06:06.603 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.601+0000 7f4c3c6a7700 1 Processor -- start 2021-11-03T17:06:06.604 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.601+0000 7f4c3c6a7700 1 -- start start 2021-11-03T17:06:06.604 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.601+0000 7f4c3c6a7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c340faa80 0x7f4c3410d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:06.604 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.601+0000 7f4c3c6a7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c3406fc00 con 0x7f4c340faa80 2021-11-03T17:06:06.604 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.601+0000 7f4c3a443700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c340faa80 0x7f4c3410d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:06.605 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.601+0000 7f4c3a443700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c340faa80 0x7f4c3410d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54026/0 (socket says 172.21.15.27:54026) 2021-11-03T17:06:06.605 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.601+0000 7f4c3a443700 1 -- 172.21.15.27:0/1138345790 learned_addr learned my addr 172.21.15.27:0/1138345790 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:06.605 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.602+0000 7f4c3a443700 1 -- 172.21.15.27:0/1138345790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c3000b200 con 0x7f4c340faa80 2021-11-03T17:06:06.606 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.602+0000 7f4c3a443700 1 --2- 172.21.15.27:0/1138345790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c340faa80 0x7f4c3410d580 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f4c30003000 tx=0x7f4c30006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:06.606 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.602+0000 7f4c22ffd700 1 -- 172.21.15.27:0/1138345790 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4c3000d9c0 con 0x7f4c340faa80 2021-11-03T17:06:06.606 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.602+0000 7f4c3c6a7700 1 -- 172.21.15.27:0/1138345790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c3406fe00 con 0x7f4c340faa80 2021-11-03T17:06:06.606 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.602+0000 7f4c22ffd700 1 -- 172.21.15.27:0/1138345790 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 983+0+0 (secure 0 0 0) 0x7f4c3001a9d0 con 0x7f4c340faa80 2021-11-03T17:06:06.607 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.602+0000 7f4c3c6a7700 1 -- 172.21.15.27:0/1138345790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c3410dc30 con 0x7f4c340faa80 2021-11-03T17:06:06.607 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.602+0000 7f4c22ffd700 1 -- 172.21.15.27:0/1138345790 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4c3001b880 con 0x7f4c340faa80 2021-11-03T17:06:06.607 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.603+0000 7f4c22ffd700 1 -- 172.21.15.27:0/1138345790 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 5) v1 ==== 36120+0+0 (secure 0 0 0) 0x7f4c3000f070 con 0x7f4c340faa80 2021-11-03T17:06:06.607 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.604+0000 7f4c22ffd700 1 --2- 172.21.15.27:0/1138345790 >> [v2:172.21.15.27:6800/388626148,v1:172.21.15.27:6801/388626148] conn(0x7f4c1c02fdb0 0x7f4c1c032270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:06.607 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.604+0000 7f4c22ffd700 1 -- 172.21.15.27:0/1138345790 --> [v2:172.21.15.27:6800/388626148,v1:172.21.15.27:6801/388626148] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4c1c032930 con 0x7f4c1c02fdb0 2021-11-03T17:06:06.608 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.604+0000 7f4c22ffd700 1 -- 172.21.15.27:0/1138345790 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f4c30040fd0 con 0x7f4c340faa80 2021-11-03T17:06:06.608 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.604+0000 7f4c39c42700 1 -- 172.21.15.27:0/1138345790 >> [v2:172.21.15.27:6800/388626148,v1:172.21.15.27:6801/388626148] conn(0x7f4c1c02fdb0 msgr2=0x7f4c1c032270 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/388626148 2021-11-03T17:06:06.608 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.604+0000 7f4c39c42700 1 --2- 172.21.15.27:0/1138345790 >> [v2:172.21.15.27:6800/388626148,v1:172.21.15.27:6801/388626148] conn(0x7f4c1c02fdb0 0x7f4c1c032270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2021-11-03T17:06:06.801 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.805+0000 7f4c39c42700 1 -- 172.21.15.27:0/1138345790 >> [v2:172.21.15.27:6800/388626148,v1:172.21.15.27:6801/388626148] conn(0x7f4c1c02fdb0 msgr2=0x7f4c1c032270 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/388626148 2021-11-03T17:06:06.801 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:06.805+0000 7f4c39c42700 1 --2- 172.21.15.27:0/1138345790 >> [v2:172.21.15.27:6800/388626148,v1:172.21.15.27:6801/388626148] conn(0x7f4c1c02fdb0 0x7f4c1c032270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2021-11-03T17:06:07.202 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:07.206+0000 7f4c39c42700 1 -- 172.21.15.27:0/1138345790 >> [v2:172.21.15.27:6800/388626148,v1:172.21.15.27:6801/388626148] conn(0x7f4c1c02fdb0 msgr2=0x7f4c1c032270 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/388626148 2021-11-03T17:06:07.202 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:07.206+0000 7f4c39c42700 1 --2- 172.21.15.27:0/1138345790 >> [v2:172.21.15.27:6800/388626148,v1:172.21.15.27:6801/388626148] conn(0x7f4c1c02fdb0 0x7f4c1c032270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2021-11-03T17:06:08.003 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:08.007+0000 7f4c39c42700 1 -- 172.21.15.27:0/1138345790 >> [v2:172.21.15.27:6800/388626148,v1:172.21.15.27:6801/388626148] conn(0x7f4c1c02fdb0 msgr2=0x7f4c1c032270 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/388626148 2021-11-03T17:06:08.003 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:08.007+0000 7f4c39c42700 1 --2- 172.21.15.27:0/1138345790 >> [v2:172.21.15.27:6800/388626148,v1:172.21.15.27:6801/388626148] conn(0x7f4c1c02fdb0 0x7f4c1c032270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2021-11-03T17:06:09.599 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:09.602+0000 7f4c22ffd700 1 -- 172.21.15.27:0/1138345790 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mgrmap(e 6) v1 ==== 35970+0+0 (secure 0 0 0) 0x7f4c30010dd0 con 0x7f4c340faa80 2021-11-03T17:06:09.599 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:09.602+0000 7f4c22ffd700 1 -- 172.21.15.27:0/1138345790 >> [v2:172.21.15.27:6800/388626148,v1:172.21.15.27:6801/388626148] conn(0x7f4c1c02fdb0 msgr2=0x7f4c1c032270 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2021-11-03T17:06:09.600 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:09.602+0000 7f4c22ffd700 1 --2- 172.21.15.27:0/1138345790 >> [v2:172.21.15.27:6800/388626148,v1:172.21.15.27:6801/388626148] conn(0x7f4c1c02fdb0 0x7f4c1c032270 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:06:10.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:09 smithi027 conmon[32322]: cluster 2021-11-03T17:06:09.549434+0000 mon.smithi027 (mon.0) 29 : cluster [INF] Active manager daemon smithi027.ufhhvb restarted 2021-11-03T17:06:10.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:09 smithi027 conmon[32322]: cluster 2021-11-03T17:06:09.549819+0000 mon.smithi027 (mon.0) 30 : cluster [INF] Activating manager daemon smithi027.ufhhvb 2021-11-03T17:06:10.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:09 smithi027 conmon[32322]: cluster 2021-11-03T17:06:09.553858+0000 mon.smithi027 (mon.0) 31 : cluster [DBG] osdmap e2: 0 total, 0 up, 0 in 2021-11-03T17:06:10.610 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:10.613+0000 7f4c22ffd700 1 -- 172.21.15.27:0/1138345790 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 7) v1 ==== 36040+0+0 (secure 0 0 0) 0x7f4c30042780 con 0x7f4c340faa80 2021-11-03T17:06:10.611 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:10.614+0000 7f4c22ffd700 1 --2- 172.21.15.27:0/1138345790 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f4c1c033810 0x7f4c1c035c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:10.611 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:10.614+0000 7f4c22ffd700 1 -- 172.21.15.27:0/1138345790 --> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4c1c032930 con 0x7f4c1c033810 2021-11-03T17:06:10.611 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:10.614+0000 7f4c39c42700 1 --2- 172.21.15.27:0/1138345790 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f4c1c033810 0x7f4c1c035c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:10.612 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:10.615+0000 7f4c39c42700 1 --2- 172.21.15.27:0/1138345790 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f4c1c033810 0x7f4c1c035c00 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7f4c240036d0 tx=0x7f4c24006d20).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:10.612 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:10.616+0000 7f4c22ffd700 1 -- 172.21.15.27:0/1138345790 <== mgr.14114 v2:172.21.15.27:6800/774407793 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6438 (secure 0 0 0) 0x7f4c1c032930 con 0x7f4c1c033810 2021-11-03T17:06:10.622 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:10.626+0000 7f4c3c6a7700 1 -- 172.21.15.27:0/1138345790 --> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f4c14000dc0 con 0x7f4c1c033810 2021-11-03T17:06:10.624 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:10.627+0000 7f4c22ffd700 1 -- 172.21.15.27:0/1138345790 <== mgr.14114 v2:172.21.15.27:6800/774407793 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f4c14000dc0 con 0x7f4c1c033810 2021-11-03T17:06:10.624 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: { 2021-11-03T17:06:10.625 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "mgrmap_epoch": 7, 2021-11-03T17:06:10.625 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "initialized": true 2021-11-03T17:06:10.625 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: } 2021-11-03T17:06:10.625 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:10.627+0000 7f4c3c6a7700 1 -- 172.21.15.27:0/1138345790 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f4c1c033810 msgr2=0x7f4c1c035c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:10.626 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:10.627+0000 7f4c3c6a7700 1 --2- 172.21.15.27:0/1138345790 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f4c1c033810 0x7f4c1c035c00 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7f4c240036d0 tx=0x7f4c24006d20).stop 2021-11-03T17:06:10.626 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:10.627+0000 7f4c3c6a7700 1 -- 172.21.15.27:0/1138345790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c340faa80 msgr2=0x7f4c3410d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:10.626 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:10.627+0000 7f4c3c6a7700 1 --2- 172.21.15.27:0/1138345790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c340faa80 0x7f4c3410d580 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f4c30003000 tx=0x7f4c30006ee0).stop 2021-11-03T17:06:10.626 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:10.627+0000 7f4c3c6a7700 1 -- 172.21.15.27:0/1138345790 shutdown_connections 2021-11-03T17:06:10.626 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:10.627+0000 7f4c3c6a7700 1 --2- 172.21.15.27:0/1138345790 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f4c1c033810 0x7f4c1c035c00 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:10.627 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:10.627+0000 7f4c3c6a7700 1 --2- 172.21.15.27:0/1138345790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c340faa80 0x7f4c3410d580 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:10.627 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:10.627+0000 7f4c3c6a7700 1 --2- 172.21.15.27:0/1138345790 >> [v2:172.21.15.27:6800/388626148,v1:172.21.15.27:6801/388626148] conn(0x7f4c1c02fdb0 0x7f4c1c032270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:06:10.627 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:10.627+0000 7f4c3c6a7700 1 -- 172.21.15.27:0/1138345790 >> 172.21.15.27:0/1138345790 conn(0x7f4c340f6260 msgr2=0x7f4c340f7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:10.627 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:10.627+0000 7f4c3c6a7700 1 -- 172.21.15.27:0/1138345790 shutdown_connections 2021-11-03T17:06:10.628 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:10.627+0000 7f4c3c6a7700 1 -- 172.21.15.27:0/1138345790 wait complete. 2021-11-03T17:06:11.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: cluster 2021-11-03T17:06:09.605141+0000 mon.smithi027 (mon.0) 32 : cluster 2021-11-03T17:06:11.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: [DBG] mgrmap e6: smithi027.ufhhvb(active, starting, since 0.055391s) 2021-11-03T17:06:11.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: audit 2021-11-03T17:06:09.608016+0000 mon.smithi027 (mon.0) 33 : audit [DBG] from='mgr.14114 172.21.15.27:0/784375816' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2021-11-03T17:06:11.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: audit 2021-11- 2021-11-03T17:06:11.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: 03T17:06:09.608858+0000 mon.smithi027 (mon.0) 34 : audit [DBG] from='mgr.14114 172.21.15.27:0/784375816' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mgr metadata", "who": "smithi027.ufhhvb", "id": "smithi027.ufhhvb"}]: dispatch 2021-11-03T17:06:11.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: audit 2021-11 2021-11-03T17:06:11.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: -03T17:06:09.609711+0000 mon.smithi027 (mon.0) 35 2021-11-03T17:06:11.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: : audit [DBG] from='mgr.14114 172.21.15.27:0/784375816' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata"}]: dispatch 2021-11-03T17:06:11.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: audit 2021-11 2021-11-03T17:06:11.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: -03T17:06:09.609880+0000 mon.smithi027 (mon.0) 36 : 2021-11-03T17:06:11.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: audit [DBG] from='mgr.14114 172.21.15.27:0/784375816' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata"}]: dispatch 2021-11-03T17:06:11.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: audit 2021-11 2021-11-03T17:06:11.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: -03T17:06:09.610026+0000 mon.smithi027 (mon.0) 37 : 2021-11-03T17:06:11.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: audit [DBG] from='mgr.14114 172.21.15.27:0/784375816' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata"}]: dispatch 2021-11-03T17:06:11.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: cluster 2021-11 2021-11-03T17:06:11.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: -03T17:06:09.612168+0000 mon.smithi027 (mon. 2021-11-03T17:06:11.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: 0) 38 : cluster [INF] Manager daemon smithi027.ufhhvb is now available 2021-11-03T17:06:11.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: audit 2021-11 2021-11-03T17:06:11.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: -03T17:06:09.628936+0000 mon.smithi027 (mon.0) 39 : audit 2021-11-03T17:06:11.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: [INF] from='mgr.14114 172.21.15.27:0/784375816' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:06:11.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: audit 2021- 2021-11-03T17:06:11.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: 11-03T17:06:09.631761+0000 mon.smithi027 (mon.0 2021-11-03T17:06:11.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: ) 40 : audit [INF] from='mgr.14114 172.21.15.27:0/784375816' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:06:11.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: audit 2021-11 2021-11-03T17:06:11.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: -03T17:06:09.637004+0000 mon.smithi027 (mon.0) 41 2021-11-03T17:06:11.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: : audit [INF] from='mgr.14114 172.21.15.27:0/784375816' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:06:11.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: audit 2021-11 2021-11-03T17:06:11.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: -03T17:06:09.760575+0000 mon.smithi027 (mon.0) 2021-11-03T17:06:11.026 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: 42 : audit [INF] from='mgr.14114 172.21.15.27:0/784375816' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:06:11.026 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: audit 2021- 2021-11-03T17:06:11.026 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: 11-03T17:06:09.761294+0000 mon.smithi027 ( 2021-11-03T17:06:11.026 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: mon.0) 43 : audit 2021-11-03T17:06:11.027 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: [DBG] from='mgr.14114 172.21.15.27:0/784375816' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:06:11.027 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: audit 2021-11 2021-11-03T17:06:11.027 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: -03T17:06:09.762973+0000 mon.smithi027 (mon.0) 2021-11-03T17:06:11.027 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: 44 : audit [DBG] from='mgr.14114 172.21.15.27:0/784375816' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:06:11.027 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: audit 2021- 2021-11-03T17:06:11.028 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: 11-03T17:06:09.765031+0000 mon.smithi027 ( 2021-11-03T17:06:11.028 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: mon.0) 45 : audit [DBG] 2021-11-03T17:06:11.028 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: from='mgr.14114 172.21.15.27:0/784375816' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:06:11.028 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: audit 2021- 2021-11-03T17:06:11.029 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: 11-03T17:06:09.776109+0000 mon.smithi027 (mon 2021-11-03T17:06:11.029 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: .0) 46 : audit [INF] from='mgr.14114 172.21.15.27:0/784375816' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:06:11.029 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: 2021-11-03T17:06:11.029 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: audit 2021-11 2021-11-03T17:06:11.030 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: -03T17:06:09.777330+0000 mon.smithi027 (mon.0 2021-11-03T17:06:11.030 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:10 smithi027 conmon[32322]: ) 47 : audit [INF] from='mgr.14114 172.21.15.27:0/784375816' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:06:11.392 INFO:teuthology.orchestra.run.smithi027.stderr:mgr epoch 5 is available 2021-11-03T17:06:11.393 INFO:teuthology.orchestra.run.smithi027.stderr:Setting orchestrator backend to cephadm... 2021-11-03T17:06:11.393 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi027 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmp3f8lj10x:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpq2rb3_h7:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 orch set backend cephadm 2021-11-03T17:06:11.724 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:11 smithi027 conmon[32322]: audit 2021-11-03T17:06:10.617174+0000 mgr.smithi027.ufhhvb (mgr.14114) 1 : audit [DBG] from='client.14118 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2021-11-03T17:06:11.724 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:11 smithi027 conmon[32322]: cluster 2021-11-03T17:06:10.618045+0000 mon.smithi027 (mon.0) 48 : cluster [DBG] mgrmap e7: smithi027.ufhhvb(active, since 1.06828s) 2021-11-03T17:06:11.725 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:11 smithi027 conmon[32322]: audit 2021-11-03T17:06:10.628410+0000 mgr.smithi027.ufhhvb (mgr.14114) 2 : audit [DBG] from='client.14118 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2021-11-03T17:06:12.194 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.192+0000 7fc414956700 1 Processor -- start 2021-11-03T17:06:12.194 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.192+0000 7fc414956700 1 -- start start 2021-11-03T17:06:12.194 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.193+0000 7fc414956700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4100faaa0 0x7fc4100faec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:12.195 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.193+0000 7fc414956700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4100fb3b0 con 0x7fc4100faaa0 2021-11-03T17:06:12.195 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.193+0000 7fc40e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4100faaa0 0x7fc4100faec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:12.195 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.193+0000 7fc40e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4100faaa0 0x7fc4100faec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54058/0 (socket says 172.21.15.27:54058) 2021-11-03T17:06:12.195 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.193+0000 7fc40e59c700 1 -- 172.21.15.27:0/2650008351 learned_addr learned my addr 172.21.15.27:0/2650008351 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:12.196 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.193+0000 7fc40e59c700 1 -- 172.21.15.27:0/2650008351 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4100fb4f0 con 0x7fc4100faaa0 2021-11-03T17:06:12.196 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.194+0000 7fc40e59c700 1 --2- 172.21.15.27:0/2650008351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4100faaa0 0x7fc4100faec0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fc40800b5f0 tx=0x7fc408006ee0).ready entity=mon.0 client_cookie=cb6c1ba69f6992c7 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:12.196 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.194+0000 7fc40d59a700 1 -- 172.21.15.27:0/2650008351 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc40800d600 con 0x7fc4100faaa0 2021-11-03T17:06:12.196 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.194+0000 7fc40d59a700 1 -- 172.21.15.27:0/2650008351 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 983+0+0 (secure 0 0 0) 0x7fc40800dbb0 con 0x7fc4100faaa0 2021-11-03T17:06:12.196 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.194+0000 7fc40d59a700 1 -- 172.21.15.27:0/2650008351 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc408011bb0 con 0x7fc4100faaa0 2021-11-03T17:06:12.197 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.194+0000 7fc414956700 1 -- 172.21.15.27:0/2650008351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4100faaa0 msgr2=0x7fc4100faec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:12.197 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.194+0000 7fc414956700 1 --2- 172.21.15.27:0/2650008351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4100faaa0 0x7fc4100faec0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fc40800b5f0 tx=0x7fc408006ee0).stop 2021-11-03T17:06:12.197 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.194+0000 7fc414956700 1 -- 172.21.15.27:0/2650008351 shutdown_connections 2021-11-03T17:06:12.197 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.194+0000 7fc414956700 1 --2- 172.21.15.27:0/2650008351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4100faaa0 0x7fc4100faec0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:12.198 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.194+0000 7fc414956700 1 -- 172.21.15.27:0/2650008351 >> 172.21.15.27:0/2650008351 conn(0x7fc4100f62e0 msgr2=0x7fc4100f8700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:12.198 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.194+0000 7fc414956700 1 -- 172.21.15.27:0/2650008351 shutdown_connections 2021-11-03T17:06:12.198 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.194+0000 7fc414956700 1 -- 172.21.15.27:0/2650008351 wait complete. 2021-11-03T17:06:12.198 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.195+0000 7fc414956700 1 Processor -- start 2021-11-03T17:06:12.199 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.195+0000 7fc414956700 1 -- start start 2021-11-03T17:06:12.199 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.195+0000 7fc414956700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4100faaa0 0x7fc41010d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:12.199 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.195+0000 7fc414956700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41006fbe0 con 0x7fc4100faaa0 2021-11-03T17:06:12.199 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.195+0000 7fc40e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4100faaa0 0x7fc41010d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:12.200 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.195+0000 7fc40e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4100faaa0 0x7fc41010d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54060/0 (socket says 172.21.15.27:54060) 2021-11-03T17:06:12.200 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.195+0000 7fc40e59c700 1 -- 172.21.15.27:0/1261961145 learned_addr learned my addr 172.21.15.27:0/1261961145 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:12.200 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.196+0000 7fc40e59c700 1 -- 172.21.15.27:0/1261961145 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc40800b200 con 0x7fc4100faaa0 2021-11-03T17:06:12.200 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.196+0000 7fc40e59c700 1 --2- 172.21.15.27:0/1261961145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4100faaa0 0x7fc41010d610 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fc408003000 tx=0x7fc408006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:12.200 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.196+0000 7fc406ffd700 1 -- 172.21.15.27:0/1261961145 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc408011e30 con 0x7fc4100faaa0 2021-11-03T17:06:12.201 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.196+0000 7fc406ffd700 1 -- 172.21.15.27:0/1261961145 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 983+0+0 (secure 0 0 0) 0x7fc40800d9c0 con 0x7fc4100faaa0 2021-11-03T17:06:12.201 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.196+0000 7fc414956700 1 -- 172.21.15.27:0/1261961145 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc41006fde0 con 0x7fc4100faaa0 2021-11-03T17:06:12.201 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.196+0000 7fc406ffd700 1 -- 172.21.15.27:0/1261961145 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc40801b740 con 0x7fc4100faaa0 2021-11-03T17:06:12.201 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.196+0000 7fc414956700 1 -- 172.21.15.27:0/1261961145 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc41010dcc0 con 0x7fc4100faaa0 2021-11-03T17:06:12.202 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.197+0000 7fc406ffd700 1 -- 172.21.15.27:0/1261961145 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 7) v1 ==== 36040+0+0 (secure 0 0 0) 0x7fc40800f070 con 0x7fc4100faaa0 2021-11-03T17:06:12.202 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.197+0000 7fc406ffd700 1 --2- 172.21.15.27:0/1261961145 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7fc3f002fca0 0x7fc3f0032160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:12.202 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.197+0000 7fc406ffd700 1 -- 172.21.15.27:0/1261961145 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7fc408040fd0 con 0x7fc4100faaa0 2021-11-03T17:06:12.202 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.197+0000 7fc40dd9b700 1 --2- 172.21.15.27:0/1261961145 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7fc3f002fca0 0x7fc3f0032160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:12.203 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.197+0000 7fc414956700 1 -- 172.21.15.27:0/1261961145 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc410059100 con 0x7fc4100faaa0 2021-11-03T17:06:12.203 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.197+0000 7fc40dd9b700 1 --2- 172.21.15.27:0/1261961145 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7fc3f002fca0 0x7fc3f0032160 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fc3f80096a0 tx=0x7fc3f8006b40).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:12.203 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.202+0000 7fc406ffd700 1 -- 172.21.15.27:0/1261961145 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fc408016070 con 0x7fc4100faaa0 2021-11-03T17:06:12.502 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.506+0000 7fc406ffd700 1 -- 172.21.15.27:0/1261961145 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 8) v1 ==== 36120+0+0 (secure 0 0 0) 0x7fc408024790 con 0x7fc4100faaa0 2021-11-03T17:06:12.545 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.549+0000 7fc414956700 1 -- 172.21.15.27:0/1261961145 --> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4100fc930 con 0x7fc3f002fca0 2021-11-03T17:06:12.553 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.557+0000 7fc406ffd700 1 -- 172.21.15.27:0/1261961145 <== mgr.14114 v2:172.21.15.27:6800/774407793 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc4100fc930 con 0x7fc3f002fca0 2021-11-03T17:06:12.555 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.559+0000 7fc414956700 1 -- 172.21.15.27:0/1261961145 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7fc3f002fca0 msgr2=0x7fc3f0032160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:12.556 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.559+0000 7fc414956700 1 --2- 172.21.15.27:0/1261961145 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7fc3f002fca0 0x7fc3f0032160 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fc3f80096a0 tx=0x7fc3f8006b40).stop 2021-11-03T17:06:12.556 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.559+0000 7fc414956700 1 -- 172.21.15.27:0/1261961145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4100faaa0 msgr2=0x7fc41010d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:12.556 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.559+0000 7fc414956700 1 --2- 172.21.15.27:0/1261961145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4100faaa0 0x7fc41010d610 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7fc408003000 tx=0x7fc408006ee0).stop 2021-11-03T17:06:12.556 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.559+0000 7fc414956700 1 -- 172.21.15.27:0/1261961145 shutdown_connections 2021-11-03T17:06:12.557 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.559+0000 7fc414956700 1 --2- 172.21.15.27:0/1261961145 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7fc3f002fca0 0x7fc3f0032160 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:12.557 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.559+0000 7fc414956700 1 --2- 172.21.15.27:0/1261961145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4100faaa0 0x7fc41010d610 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:12.557 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.559+0000 7fc414956700 1 -- 172.21.15.27:0/1261961145 >> 172.21.15.27:0/1261961145 conn(0x7fc4100f62e0 msgr2=0x7fc4100f7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:12.557 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.559+0000 7fc414956700 1 -- 172.21.15.27:0/1261961145 shutdown_connections 2021-11-03T17:06:12.558 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:12.559+0000 7fc414956700 1 -- 172.21.15.27:0/1261961145 wait complete. 2021-11-03T17:06:13.192 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi027 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmp3f8lj10x:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpq2rb3_h7:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 cephadm set-user root 2021-11-03T17:06:13.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:13 smithi027 conmon[32322]: cluster 2021-11-03T17:06:12.508775+0000 mon.smithi027 (mon 2021-11-03T17:06:13.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:13 smithi027 conmon[32322]: .0) 49 : cluster [DBG] mgrmap e8: smithi027.ufhhvb(active, since 2s) 2021-11-03T17:06:13.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:13 smithi027 conmon[32322]: audit 2021-11-03T17:06:12.558558+0000 mon.smithi027 (mon.0) 50 : audit [INF] from='mgr.14114 172.21.15.27:0/784375816' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:06:13.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:13 smithi027 conmon[32322]: audit 2021-11-03T17:06:12.562381+0000 mon.smithi027 (mon.0) 51 : audit [DBG] from='mgr.14114 172.21.15.27:0/784375816' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:06:13.877 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.877+0000 7efe2c649700 1 Processor -- start 2021-11-03T17:06:13.877 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.877+0000 7efe2c649700 1 -- start start 2021-11-03T17:06:13.878 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.877+0000 7efe2c649700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe2406fb80 0x7efe240fe850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:13.878 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.877+0000 7efe2c649700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe240fed40 con 0x7efe2406fb80 2021-11-03T17:06:13.878 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.877+0000 7efe2a3e5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe2406fb80 0x7efe240fe850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:13.878 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.878+0000 7efe2a3e5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe2406fb80 0x7efe240fe850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54064/0 (socket says 172.21.15.27:54064) 2021-11-03T17:06:13.879 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.878+0000 7efe2a3e5700 1 -- 172.21.15.27:0/3871402435 learned_addr learned my addr 172.21.15.27:0/3871402435 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:13.879 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.878+0000 7efe2a3e5700 1 -- 172.21.15.27:0/3871402435 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe240fee80 con 0x7efe2406fb80 2021-11-03T17:06:13.879 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.878+0000 7efe2a3e5700 1 --2- 172.21.15.27:0/3871402435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe2406fb80 0x7efe240fe850 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7efe14009960 tx=0x7efe140092d0).ready entity=mon.0 client_cookie=6439f486a0e6c89 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:13.879 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.878+0000 7efe293e3700 1 -- 172.21.15.27:0/3871402435 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efe1400ba50 con 0x7efe2406fb80 2021-11-03T17:06:13.880 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.878+0000 7efe293e3700 1 -- 172.21.15.27:0/3871402435 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 983+0+0 (secure 0 0 0) 0x7efe14011400 con 0x7efe2406fb80 2021-11-03T17:06:13.880 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.878+0000 7efe293e3700 1 -- 172.21.15.27:0/3871402435 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efe14010580 con 0x7efe2406fb80 2021-11-03T17:06:13.880 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.879+0000 7efe2c649700 1 -- 172.21.15.27:0/3871402435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe2406fb80 msgr2=0x7efe240fe850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:13.880 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.879+0000 7efe2c649700 1 --2- 172.21.15.27:0/3871402435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe2406fb80 0x7efe240fe850 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7efe14009960 tx=0x7efe140092d0).stop 2021-11-03T17:06:13.880 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.879+0000 7efe2c649700 1 -- 172.21.15.27:0/3871402435 shutdown_connections 2021-11-03T17:06:13.881 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.879+0000 7efe2c649700 1 --2- 172.21.15.27:0/3871402435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe2406fb80 0x7efe240fe850 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:13.881 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.879+0000 7efe2c649700 1 -- 172.21.15.27:0/3871402435 >> 172.21.15.27:0/3871402435 conn(0x7efe240f62f0 msgr2=0x7efe240f8710 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:13.881 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.879+0000 7efe2c649700 1 -- 172.21.15.27:0/3871402435 shutdown_connections 2021-11-03T17:06:13.881 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.879+0000 7efe2c649700 1 -- 172.21.15.27:0/3871402435 wait complete. 2021-11-03T17:06:13.882 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.879+0000 7efe2c649700 1 Processor -- start 2021-11-03T17:06:13.882 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.880+0000 7efe2c649700 1 -- start start 2021-11-03T17:06:13.882 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.880+0000 7efe2c649700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe2406fb80 0x7efe2410a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:13.882 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.880+0000 7efe2c649700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2410ae40 con 0x7efe2406fb80 2021-11-03T17:06:13.882 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.880+0000 7efe2a3e5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe2406fb80 0x7efe2410a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:13.883 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.880+0000 7efe2a3e5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe2406fb80 0x7efe2410a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54066/0 (socket says 172.21.15.27:54066) 2021-11-03T17:06:13.883 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.880+0000 7efe2a3e5700 1 -- 172.21.15.27:0/477578666 learned_addr learned my addr 172.21.15.27:0/477578666 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:13.883 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.880+0000 7efe2a3e5700 1 -- 172.21.15.27:0/477578666 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe14009570 con 0x7efe2406fb80 2021-11-03T17:06:13.883 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.880+0000 7efe2a3e5700 1 --2- 172.21.15.27:0/477578666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe2406fb80 0x7efe2410a950 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7efe14010770 tx=0x7efe140092d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:13.884 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.880+0000 7efe22ffd700 1 -- 172.21.15.27:0/477578666 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efe140107f0 con 0x7efe2406fb80 2021-11-03T17:06:13.884 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.880+0000 7efe22ffd700 1 -- 172.21.15.27:0/477578666 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 983+0+0 (secure 0 0 0) 0x7efe14010da0 con 0x7efe2406fb80 2021-11-03T17:06:13.884 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.880+0000 7efe2c649700 1 -- 172.21.15.27:0/477578666 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe2410b040 con 0x7efe2406fb80 2021-11-03T17:06:13.884 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.880+0000 7efe22ffd700 1 -- 172.21.15.27:0/477578666 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7efe1401ad90 con 0x7efe2406fb80 2021-11-03T17:06:13.885 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.880+0000 7efe2c649700 1 -- 172.21.15.27:0/477578666 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe2410b4d0 con 0x7efe2406fb80 2021-11-03T17:06:13.885 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.881+0000 7efe22ffd700 1 -- 172.21.15.27:0/477578666 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) v1 ==== 36120+0+0 (secure 0 0 0) 0x7efe1400f070 con 0x7efe2406fb80 2021-11-03T17:06:13.886 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.882+0000 7efe22ffd700 1 --2- 172.21.15.27:0/477578666 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7efe1002fdb0 0x7efe10032270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:13.886 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.882+0000 7efe22ffd700 1 -- 172.21.15.27:0/477578666 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7efe14041910 con 0x7efe2406fb80 2021-11-03T17:06:13.886 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.882+0000 7efe2c649700 1 -- 172.21.15.27:0/477578666 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe24045ad0 con 0x7efe2406fb80 2021-11-03T17:06:13.887 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.882+0000 7efe29be4700 1 --2- 172.21.15.27:0/477578666 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7efe1002fdb0 0x7efe10032270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:13.887 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.882+0000 7efe29be4700 1 --2- 172.21.15.27:0/477578666 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7efe1002fdb0 0x7efe10032270 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7efe180096a0 tx=0x7efe18006b40).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:13.887 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:13.886+0000 7efe22ffd700 1 -- 172.21.15.27:0/477578666 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7efe14010950 con 0x7efe2406fb80 2021-11-03T17:06:14.170 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:14.174+0000 7efe2c649700 1 -- 172.21.15.27:0/477578666 --> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7efe24104920 con 0x7efe1002fdb0 2021-11-03T17:06:14.171 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:14.175+0000 7efe22ffd700 1 -- 172.21.15.27:0/477578666 <== mgr.14114 v2:172.21.15.27:6800/774407793 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7efe24104920 con 0x7efe1002fdb0 2021-11-03T17:06:14.171 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: value unchanged 2021-11-03T17:06:14.173 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:14.176+0000 7efe2c649700 1 -- 172.21.15.27:0/477578666 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7efe1002fdb0 msgr2=0x7efe10032270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:14.173 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:14.176+0000 7efe2c649700 1 --2- 172.21.15.27:0/477578666 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7efe1002fdb0 0x7efe10032270 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7efe180096a0 tx=0x7efe18006b40).stop 2021-11-03T17:06:14.174 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:14.176+0000 7efe2c649700 1 -- 172.21.15.27:0/477578666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe2406fb80 msgr2=0x7efe2410a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:14.174 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:14.176+0000 7efe2c649700 1 --2- 172.21.15.27:0/477578666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe2406fb80 0x7efe2410a950 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7efe14010770 tx=0x7efe140092d0).stop 2021-11-03T17:06:14.174 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:14.177+0000 7efe2c649700 1 -- 172.21.15.27:0/477578666 shutdown_connections 2021-11-03T17:06:14.175 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:14.177+0000 7efe2c649700 1 --2- 172.21.15.27:0/477578666 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7efe1002fdb0 0x7efe10032270 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:14.175 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:14.177+0000 7efe2c649700 1 --2- 172.21.15.27:0/477578666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe2406fb80 0x7efe2410a950 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:14.175 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:14.177+0000 7efe2c649700 1 -- 172.21.15.27:0/477578666 >> 172.21.15.27:0/477578666 conn(0x7efe240f62f0 msgr2=0x7efe240f7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:14.176 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:14.177+0000 7efe2c649700 1 -- 172.21.15.27:0/477578666 shutdown_connections 2021-11-03T17:06:14.176 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:14.177+0000 7efe2c649700 1 -- 172.21.15.27:0/477578666 wait complete. 2021-11-03T17:06:14.749 INFO:teuthology.orchestra.run.smithi027.stderr:Generating ssh key... 2021-11-03T17:06:14.750 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi027 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmp3f8lj10x:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpq2rb3_h7:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 cephadm generate-key 2021-11-03T17:06:14.757 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:14 smithi027 conmon[32322]: audit 2021-11-03T17:06:12.552030+0000 mgr.smithi027.ufhhvb ( 2021-11-03T17:06:14.758 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:14 smithi027 conmon[32322]: mgr.14114) 3 : audit [DBG] from='client.14124 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:06:15.415 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.416+0000 7fd9fbe76700 1 Processor -- start 2021-11-03T17:06:15.416 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.416+0000 7fd9fbe76700 1 -- start start 2021-11-03T17:06:15.416 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.416+0000 7fd9fbe76700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9f406fb40 0x7fd9f4100c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:15.416 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.416+0000 7fd9fbe76700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f40fc680 con 0x7fd9f406fb40 2021-11-03T17:06:15.417 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.416+0000 7fd9f9c12700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9f406fb40 0x7fd9f4100c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:15.417 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.416+0000 7fd9f9c12700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9f406fb40 0x7fd9f4100c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54070/0 (socket says 172.21.15.27:54070) 2021-11-03T17:06:15.417 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.416+0000 7fd9f9c12700 1 -- 172.21.15.27:0/4010152170 learned_addr learned my addr 172.21.15.27:0/4010152170 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:15.417 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.417+0000 7fd9f9c12700 1 -- 172.21.15.27:0/4010152170 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f4101180 con 0x7fd9f406fb40 2021-11-03T17:06:15.417 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.417+0000 7fd9f9c12700 1 --2- 172.21.15.27:0/4010152170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9f406fb40 0x7fd9f4100c90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fd9f000b5f0 tx=0x7fd9f0006ee0).ready entity=mon.0 client_cookie=cc8eee3cc53d354d server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:15.418 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.417+0000 7fd9f8c10700 1 -- 172.21.15.27:0/4010152170 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd9f000d600 con 0x7fd9f406fb40 2021-11-03T17:06:15.418 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.417+0000 7fd9f8c10700 1 -- 172.21.15.27:0/4010152170 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 983+0+0 (secure 0 0 0) 0x7fd9f000dbb0 con 0x7fd9f406fb40 2021-11-03T17:06:15.418 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.417+0000 7fd9f8c10700 1 -- 172.21.15.27:0/4010152170 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd9f0011bb0 con 0x7fd9f406fb40 2021-11-03T17:06:15.418 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.417+0000 7fd9fbe76700 1 -- 172.21.15.27:0/4010152170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9f406fb40 msgr2=0x7fd9f4100c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:15.419 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.417+0000 7fd9fbe76700 1 --2- 172.21.15.27:0/4010152170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9f406fb40 0x7fd9f4100c90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7fd9f000b5f0 tx=0x7fd9f0006ee0).stop 2021-11-03T17:06:15.419 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.418+0000 7fd9fbe76700 1 -- 172.21.15.27:0/4010152170 shutdown_connections 2021-11-03T17:06:15.419 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.418+0000 7fd9fbe76700 1 --2- 172.21.15.27:0/4010152170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9f406fb40 0x7fd9f4100c90 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:15.419 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.418+0000 7fd9fbe76700 1 -- 172.21.15.27:0/4010152170 >> 172.21.15.27:0/4010152170 conn(0x7fd9f40f62f0 msgr2=0x7fd9f40f8710 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:15.420 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.418+0000 7fd9fbe76700 1 -- 172.21.15.27:0/4010152170 shutdown_connections 2021-11-03T17:06:15.420 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.418+0000 7fd9fbe76700 1 -- 172.21.15.27:0/4010152170 wait complete. 2021-11-03T17:06:15.420 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.418+0000 7fd9fbe76700 1 Processor -- start 2021-11-03T17:06:15.420 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.418+0000 7fd9fbe76700 1 -- start start 2021-11-03T17:06:15.421 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.418+0000 7fd9fbe76700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9f406fb40 0x7fd9f410cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:15.421 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.418+0000 7fd9fbe76700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9f410d040 con 0x7fd9f406fb40 2021-11-03T17:06:15.421 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.419+0000 7fd9f9c12700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9f406fb40 0x7fd9f410cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:15.421 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.419+0000 7fd9f9c12700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9f406fb40 0x7fd9f410cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54072/0 (socket says 172.21.15.27:54072) 2021-11-03T17:06:15.422 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.419+0000 7fd9f9c12700 1 -- 172.21.15.27:0/948200727 learned_addr learned my addr 172.21.15.27:0/948200727 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:15.422 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.419+0000 7fd9f9c12700 1 -- 172.21.15.27:0/948200727 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9f000b200 con 0x7fd9f406fb40 2021-11-03T17:06:15.422 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.419+0000 7fd9f9c12700 1 --2- 172.21.15.27:0/948200727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9f406fb40 0x7fd9f410cb50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fd9f0006cb0 tx=0x7fd9f0006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:15.422 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.419+0000 7fd9e27fc700 1 -- 172.21.15.27:0/948200727 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd9f001b410 con 0x7fd9f406fb40 2021-11-03T17:06:15.423 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.419+0000 7fd9e27fc700 1 -- 172.21.15.27:0/948200727 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 983+0+0 (secure 0 0 0) 0x7fd9f001b9c0 con 0x7fd9f406fb40 2021-11-03T17:06:15.423 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.419+0000 7fd9e27fc700 1 -- 172.21.15.27:0/948200727 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd9f001a6f0 con 0x7fd9f406fb40 2021-11-03T17:06:15.423 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.419+0000 7fd9fbe76700 1 -- 172.21.15.27:0/948200727 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9f410d240 con 0x7fd9f406fb40 2021-11-03T17:06:15.423 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.419+0000 7fd9fbe76700 1 -- 172.21.15.27:0/948200727 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9f410d6d0 con 0x7fd9f406fb40 2021-11-03T17:06:15.423 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.420+0000 7fd9e27fc700 1 -- 172.21.15.27:0/948200727 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) v1 ==== 36120+0+0 (secure 0 0 0) 0x7fd9f000f070 con 0x7fd9f406fb40 2021-11-03T17:06:15.424 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.420+0000 7fd9e27fc700 1 --2- 172.21.15.27:0/948200727 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7fd9dc02fdb0 0x7fd9dc032270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:15.424 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.420+0000 7fd9e27fc700 1 -- 172.21.15.27:0/948200727 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7fd9f0040e90 con 0x7fd9f406fb40 2021-11-03T17:06:15.424 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.420+0000 7fd9f9411700 1 --2- 172.21.15.27:0/948200727 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7fd9dc02fdb0 0x7fd9dc032270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:15.424 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.420+0000 7fd9fbe76700 1 -- 172.21.15.27:0/948200727 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9f4045ad0 con 0x7fd9f406fb40 2021-11-03T17:06:15.425 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.421+0000 7fd9f9411700 1 --2- 172.21.15.27:0/948200727 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7fd9dc02fdb0 0x7fd9dc032270 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fd9e40096a0 tx=0x7fd9e4006b40).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:15.425 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.425+0000 7fd9e27fc700 1 -- 172.21.15.27:0/948200727 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fd9f0016070 con 0x7fd9f406fb40 2021-11-03T17:06:15.708 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:15.712+0000 7fd9fbe76700 1 -- 172.21.15.27:0/948200727 --> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9f40008d0 con 0x7fd9dc02fdb0 2021-11-03T17:06:15.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:15 smithi027 conmon[32322]: audit 2021-11-03T17:06:14.176344+0000 mgr.smithi027.ufhhvb ( 2021-11-03T17:06:15.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:15 smithi027 conmon[32322]: mgr.14114) 4 : audit [DBG] from='client.14126 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:06:16.470 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:16.475+0000 7fd9e27fc700 1 -- 172.21.15.27:0/948200727 <== mgr.14114 v2:172.21.15.27:6800/774407793 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd9f40008d0 con 0x7fd9dc02fdb0 2021-11-03T17:06:16.473 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:16.477+0000 7fd9fbe76700 1 -- 172.21.15.27:0/948200727 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7fd9dc02fdb0 msgr2=0x7fd9dc032270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:16.474 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:16.477+0000 7fd9fbe76700 1 --2- 172.21.15.27:0/948200727 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7fd9dc02fdb0 0x7fd9dc032270 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fd9e40096a0 tx=0x7fd9e4006b40).stop 2021-11-03T17:06:16.474 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:16.477+0000 7fd9fbe76700 1 -- 172.21.15.27:0/948200727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9f406fb40 msgr2=0x7fd9f410cb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:16.474 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:16.477+0000 7fd9fbe76700 1 --2- 172.21.15.27:0/948200727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9f406fb40 0x7fd9f410cb50 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fd9f0006cb0 tx=0x7fd9f0006ee0).stop 2021-11-03T17:06:16.474 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:16.477+0000 7fd9fbe76700 1 -- 172.21.15.27:0/948200727 shutdown_connections 2021-11-03T17:06:16.475 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:16.477+0000 7fd9fbe76700 1 --2- 172.21.15.27:0/948200727 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7fd9dc02fdb0 0x7fd9dc032270 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:16.475 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:16.477+0000 7fd9fbe76700 1 --2- 172.21.15.27:0/948200727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9f406fb40 0x7fd9f410cb50 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:16.475 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:16.477+0000 7fd9fbe76700 1 -- 172.21.15.27:0/948200727 >> 172.21.15.27:0/948200727 conn(0x7fd9f40f62f0 msgr2=0x7fd9f40f7e30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:16.475 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:16.477+0000 7fd9fbe76700 1 -- 172.21.15.27:0/948200727 shutdown_connections 2021-11-03T17:06:16.476 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:16.477+0000 7fd9fbe76700 1 -- 172.21.15.27:0/948200727 wait complete. 2021-11-03T17:06:17.083 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi027 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmp3f8lj10x:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpq2rb3_h7:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 cephadm get-pub-key 2021-11-03T17:06:17.741 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.741+0000 7f21c9d69700 1 Processor -- start 2021-11-03T17:06:17.741 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.741+0000 7f21c9d69700 1 -- start start 2021-11-03T17:06:17.741 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.741+0000 7f21c9d69700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c40fba50 0x7f21c40fbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:17.741 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.741+0000 7f21c9d69700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c40fc360 con 0x7f21c40fba50 2021-11-03T17:06:17.742 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.741+0000 7f21c8d67700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c40fba50 0x7f21c40fbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:17.742 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.742+0000 7f21c8d67700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c40fba50 0x7f21c40fbe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54076/0 (socket says 172.21.15.27:54076) 2021-11-03T17:06:17.742 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.742+0000 7f21c8d67700 1 -- 172.21.15.27:0/475927305 learned_addr learned my addr 172.21.15.27:0/475927305 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:17.742 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.742+0000 7f21c8d67700 1 -- 172.21.15.27:0/475927305 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21c40fc4a0 con 0x7f21c40fba50 2021-11-03T17:06:17.743 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.742+0000 7f21c8d67700 1 --2- 172.21.15.27:0/475927305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c40fba50 0x7f21c40fbe70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f21bc009960 tx=0x7f21bc0092d0).ready entity=mon.0 client_cookie=3398daee2237cb05 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:17.743 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.742+0000 7f21c37fe700 1 -- 172.21.15.27:0/475927305 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f21bc00b9c0 con 0x7f21c40fba50 2021-11-03T17:06:17.743 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.742+0000 7f21c37fe700 1 -- 172.21.15.27:0/475927305 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 983+0+0 (secure 0 0 0) 0x7f21bc011400 con 0x7f21c40fba50 2021-11-03T17:06:17.743 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.742+0000 7f21c37fe700 1 -- 172.21.15.27:0/475927305 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f21bc010580 con 0x7f21c40fba50 2021-11-03T17:06:17.744 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.743+0000 7f21c9d69700 1 -- 172.21.15.27:0/475927305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c40fba50 msgr2=0x7f21c40fbe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:17.744 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.743+0000 7f21c9d69700 1 --2- 172.21.15.27:0/475927305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c40fba50 0x7f21c40fbe70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f21bc009960 tx=0x7f21bc0092d0).stop 2021-11-03T17:06:17.744 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.743+0000 7f21c9d69700 1 -- 172.21.15.27:0/475927305 shutdown_connections 2021-11-03T17:06:17.744 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.743+0000 7f21c9d69700 1 --2- 172.21.15.27:0/475927305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c40fba50 0x7f21c40fbe70 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:17.745 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.743+0000 7f21c9d69700 1 -- 172.21.15.27:0/475927305 >> 172.21.15.27:0/475927305 conn(0x7f21c40f7270 msgr2=0x7f21c40f9690 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:17.745 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.743+0000 7f21c9d69700 1 -- 172.21.15.27:0/475927305 shutdown_connections 2021-11-03T17:06:17.745 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.743+0000 7f21c9d69700 1 -- 172.21.15.27:0/475927305 wait complete. 2021-11-03T17:06:17.745 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.743+0000 7f21c9d69700 1 Processor -- start 2021-11-03T17:06:17.745 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.743+0000 7f21c9d69700 1 -- start start 2021-11-03T17:06:17.746 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.744+0000 7f21c9d69700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c40fba50 0x7f21c4071d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:17.746 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.744+0000 7f21c9d69700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21c4070180 con 0x7f21c40fba50 2021-11-03T17:06:17.746 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.744+0000 7f21c8d67700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c40fba50 0x7f21c4071d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:17.746 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.744+0000 7f21c8d67700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c40fba50 0x7f21c4071d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54078/0 (socket says 172.21.15.27:54078) 2021-11-03T17:06:17.747 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.744+0000 7f21c8d67700 1 -- 172.21.15.27:0/1842352515 learned_addr learned my addr 172.21.15.27:0/1842352515 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:17.747 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.744+0000 7f21c8d67700 1 -- 172.21.15.27:0/1842352515 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21bc009570 con 0x7f21c40fba50 2021-11-03T17:06:17.747 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.744+0000 7f21c8d67700 1 --2- 172.21.15.27:0/1842352515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c40fba50 0x7f21c4071d00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f21bc00b5c0 tx=0x7f21bc0092d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:17.747 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.744+0000 7f21c17fa700 1 -- 172.21.15.27:0/1842352515 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f21bc010840 con 0x7f21c40fba50 2021-11-03T17:06:17.748 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.744+0000 7f21c17fa700 1 -- 172.21.15.27:0/1842352515 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 983+0+0 (secure 0 0 0) 0x7f21bc011e90 con 0x7f21c40fba50 2021-11-03T17:06:17.748 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.744+0000 7f21c17fa700 1 -- 172.21.15.27:0/1842352515 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f21bc01ac10 con 0x7f21c40fba50 2021-11-03T17:06:17.748 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.744+0000 7f21c9d69700 1 -- 172.21.15.27:0/1842352515 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21c4070320 con 0x7f21c40fba50 2021-11-03T17:06:17.748 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.745+0000 7f21c9d69700 1 -- 172.21.15.27:0/1842352515 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21c40707b0 con 0x7f21c40fba50 2021-11-03T17:06:17.748 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.745+0000 7f21c17fa700 1 -- 172.21.15.27:0/1842352515 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) v1 ==== 36120+0+0 (secure 0 0 0) 0x7f21bc02a070 con 0x7f21c40fba50 2021-11-03T17:06:17.749 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.745+0000 7f21c17fa700 1 --2- 172.21.15.27:0/1842352515 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f21b002fdb0 0x7f21b0032270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:17.749 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.745+0000 7f21c17fa700 1 -- 172.21.15.27:0/1842352515 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7f21bc041b50 con 0x7f21c40fba50 2021-11-03T17:06:17.749 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.745+0000 7f21c3fff700 1 --2- 172.21.15.27:0/1842352515 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f21b002fdb0 0x7f21b0032270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:17.749 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.746+0000 7f21c9d69700 1 -- 172.21.15.27:0/1842352515 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21c4059100 con 0x7f21c40fba50 2021-11-03T17:06:17.750 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.746+0000 7f21c3fff700 1 --2- 172.21.15.27:0/1842352515 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f21b002fdb0 0x7f21b0032270 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f21b80096a0 tx=0x7f21b8006b40).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:17.750 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:17.750+0000 7f21c17fa700 1 -- 172.21.15.27:0/1842352515 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f21bc01e080 con 0x7f21c40fba50 2021-11-03T17:06:17.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:17 smithi027 conmon[32322]: cephadm 2021-11-03T17:06: 2021-11-03T17:06:17.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:17 smithi027 conmon[32322]: 15.713906+0000 mgr.smithi027.ufhhvb (mgr.14114) 6 : cephadm [INF] Generating ssh key... 2021-11-03T17:06:17.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:17 smithi027 conmon[32322]: audit 2021-11-03T17:06 2021-11-03T17:06:17.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:17 smithi027 conmon[32322]: :16.473584+0000 mon.smithi027 (mon.0) 52 : audit 2021-11-03T17:06:17.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:17 smithi027 conmon[32322]: [INF] from='mgr.14114 172.21.15.27:0/784375816' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:06:17.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:17 smithi027 conmon[32322]: audit 2021-11-03T17:06:17.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:17 smithi027 conmon[32322]: 2021-11-03T 2021-11-03T17:06:17.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:17 smithi027 conmon[32322]: 17:06:16.475349+0000 2021-11-03T17:06:17.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:17 smithi027 conmon[32322]: mon.smithi027 (mon.0) 53 : audit 2021-11-03T17:06:17.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:17 smithi027 conmon[32322]: [INF] from='mgr.14114 172.21.15.27:0/784375816' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:06:18.036 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:18.039+0000 7f21c9d69700 1 -- 172.21.15.27:0/1842352515 --> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f21c40fd810 con 0x7f21b002fdb0 2021-11-03T17:06:18.036 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:18.040+0000 7f21c17fa700 1 -- 172.21.15.27:0/1842352515 <== mgr.14114 v2:172.21.15.27:6800/774407793 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f21c40fd810 con 0x7f21b002fdb0 2021-11-03T17:06:18.037 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDB1JsmfoPcV548a0JNZ2YyxCK9yzt2y9p0rsuzBHbYmRl1XJ3MOKqMKr5HzZggpgtGKfmVCwyPqy2tA7e0IymbWo09P0jgLkfcod2MPsHNCUa5457jRnsGWCaDxdB4WplXtk5FrkEWr8STePgS/tdb24gnd/xoI0gfANvzEUGFD6+y9T8M3mRIZ9WGEuUPfPZj5lk6QXTwQBMlsV291a/OuJWURBWh3tQH5A/n9+Z2NK1ViqKQaEoB5/MVcFjOb9+xtT5vegzPdSsTZxfmBDEGSwX/e+sQtno3lBzcIXDnkhZ4g3u0vmxaLT5eH64zz44Ub84ucPaytxE1qhZ09Ss0gdtMyT7se/wCLBpuK7s/o5u+u35h1YZ4vJs/iHgCn5zK8+Vk8kUKBTrxOwZJTxMeCVOOWVsUk7smDfrnkC6ZlrXDVJ61BMXPOktAVlIjxXjhKR7A12i7K6DWa1HUwcIrjNUmzOP1oAMwxaas/KxxN3VM3BY4qRSywGLnKVOU8P0= ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c 2021-11-03T17:06:18.040 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:18.041+0000 7f21c9d69700 1 -- 172.21.15.27:0/1842352515 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f21b002fdb0 msgr2=0x7f21b0032270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:18.040 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:18.041+0000 7f21c9d69700 1 --2- 172.21.15.27:0/1842352515 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f21b002fdb0 0x7f21b0032270 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f21b80096a0 tx=0x7f21b8006b40).stop 2021-11-03T17:06:18.041 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:18.041+0000 7f21c9d69700 1 -- 172.21.15.27:0/1842352515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c40fba50 msgr2=0x7f21c4071d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:18.041 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:18.041+0000 7f21c9d69700 1 --2- 172.21.15.27:0/1842352515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c40fba50 0x7f21c4071d00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f21bc00b5c0 tx=0x7f21bc0092d0).stop 2021-11-03T17:06:18.041 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:18.041+0000 7f21c9d69700 1 -- 172.21.15.27:0/1842352515 shutdown_connections 2021-11-03T17:06:18.042 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:18.041+0000 7f21c9d69700 1 --2- 172.21.15.27:0/1842352515 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f21b002fdb0 0x7f21b0032270 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:18.042 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:18.042+0000 7f21c9d69700 1 --2- 172.21.15.27:0/1842352515 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21c40fba50 0x7f21c4071d00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:18.043 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:18.042+0000 7f21c9d69700 1 -- 172.21.15.27:0/1842352515 >> 172.21.15.27:0/1842352515 conn(0x7f21c40f7270 msgr2=0x7f21c40f8db0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:18.043 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:18.042+0000 7f21c9d69700 1 -- 172.21.15.27:0/1842352515 shutdown_connections 2021-11-03T17:06:18.044 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:18.042+0000 7f21c9d69700 1 -- 172.21.15.27:0/1842352515 wait complete. 2021-11-03T17:06:18.674 INFO:teuthology.orchestra.run.smithi027.stderr:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2021-11-03T17:06:18.675 INFO:teuthology.orchestra.run.smithi027.stderr:Adding key to root@localhost authorized_keys... 2021-11-03T17:06:18.675 INFO:teuthology.orchestra.run.smithi027.stderr:Adding host smithi027... 2021-11-03T17:06:18.676 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi027 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmp3f8lj10x:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpq2rb3_h7:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 orch host add smithi027 172.21.15.27 2021-11-03T17:06:18.888 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:18 smithi027 conmon[32322]: audit 2021-11-03T17:06:18.040431+0000 mgr.smithi027.ufhhvb 2021-11-03T17:06:18.889 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:18 smithi027 conmon[32322]: (mgr.14114) 7 : audit [DBG] from='client.14130 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:06:19.341 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.344+0000 7f88b2e18700 1 Processor -- start 2021-11-03T17:06:19.341 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.344+0000 7f88b2e18700 1 -- start start 2021-11-03T17:06:19.342 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.344+0000 7f88b2e18700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88ac0fd2a0 0x7f88ac0fd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:19.342 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.344+0000 7f88b2e18700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac0fdbb0 con 0x7f88ac0fd2a0 2021-11-03T17:06:19.342 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.344+0000 7f88b0bb4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88ac0fd2a0 0x7f88ac0fd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:19.342 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.344+0000 7f88b0bb4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88ac0fd2a0 0x7f88ac0fd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54082/0 (socket says 172.21.15.27:54082) 2021-11-03T17:06:19.343 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.344+0000 7f88b0bb4700 1 -- 172.21.15.27:0/1242173550 learned_addr learned my addr 172.21.15.27:0/1242173550 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:19.343 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.344+0000 7f88b0bb4700 1 -- 172.21.15.27:0/1242173550 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88ac0fdcf0 con 0x7f88ac0fd2a0 2021-11-03T17:06:19.343 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.344+0000 7f88b0bb4700 1 --2- 172.21.15.27:0/1242173550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88ac0fd2a0 0x7f88ac0fd6c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f88a000b5f0 tx=0x7f88a0006ee0).ready entity=mon.0 client_cookie=f091d40b29acb514 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:19.343 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.345+0000 7f88ab7fe700 1 -- 172.21.15.27:0/1242173550 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f88a000d600 con 0x7f88ac0fd2a0 2021-11-03T17:06:19.344 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.345+0000 7f88ab7fe700 1 -- 172.21.15.27:0/1242173550 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 983+0+0 (secure 0 0 0) 0x7f88a000dbb0 con 0x7f88ac0fd2a0 2021-11-03T17:06:19.344 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.345+0000 7f88ab7fe700 1 -- 172.21.15.27:0/1242173550 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f88a0011bb0 con 0x7f88ac0fd2a0 2021-11-03T17:06:19.344 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.345+0000 7f88b2e18700 1 -- 172.21.15.27:0/1242173550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88ac0fd2a0 msgr2=0x7f88ac0fd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:19.344 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.345+0000 7f88b2e18700 1 --2- 172.21.15.27:0/1242173550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88ac0fd2a0 0x7f88ac0fd6c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f88a000b5f0 tx=0x7f88a0006ee0).stop 2021-11-03T17:06:19.345 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.345+0000 7f88b2e18700 1 -- 172.21.15.27:0/1242173550 shutdown_connections 2021-11-03T17:06:19.345 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.345+0000 7f88b2e18700 1 --2- 172.21.15.27:0/1242173550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88ac0fd2a0 0x7f88ac0fd6c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:19.345 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.345+0000 7f88b2e18700 1 -- 172.21.15.27:0/1242173550 >> 172.21.15.27:0/1242173550 conn(0x7f88ac0f5a60 msgr2=0x7f88ac0f7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:19.345 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.345+0000 7f88b2e18700 1 -- 172.21.15.27:0/1242173550 shutdown_connections 2021-11-03T17:06:19.345 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.345+0000 7f88b2e18700 1 -- 172.21.15.27:0/1242173550 wait complete. 2021-11-03T17:06:19.348 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.346+0000 7f88b2e18700 1 Processor -- start 2021-11-03T17:06:19.348 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.346+0000 7f88b2e18700 1 -- start start 2021-11-03T17:06:19.349 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.347+0000 7f88b2e18700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88ac0fd2a0 0x7f88ac10a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:19.349 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.347+0000 7f88b2e18700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88ac10adc0 con 0x7f88ac0fd2a0 2021-11-03T17:06:19.349 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.347+0000 7f88b0bb4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88ac0fd2a0 0x7f88ac10a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:19.349 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.347+0000 7f88b0bb4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88ac0fd2a0 0x7f88ac10a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54084/0 (socket says 172.21.15.27:54084) 2021-11-03T17:06:19.350 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.347+0000 7f88b0bb4700 1 -- 172.21.15.27:0/2010616547 learned_addr learned my addr 172.21.15.27:0/2010616547 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:19.350 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.347+0000 7f88b0bb4700 1 -- 172.21.15.27:0/2010616547 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a000b200 con 0x7f88ac0fd2a0 2021-11-03T17:06:19.350 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.347+0000 7f88b0bb4700 1 --2- 172.21.15.27:0/2010616547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88ac0fd2a0 0x7f88ac10a8d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f88a0003000 tx=0x7f88a0006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:19.350 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.347+0000 7f88a97fa700 1 -- 172.21.15.27:0/2010616547 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f88a000d9c0 con 0x7f88ac0fd2a0 2021-11-03T17:06:19.351 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.347+0000 7f88a97fa700 1 -- 172.21.15.27:0/2010616547 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 983+0+0 (secure 0 0 0) 0x7f88a001a9d0 con 0x7f88ac0fd2a0 2021-11-03T17:06:19.351 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.347+0000 7f88b2e18700 1 -- 172.21.15.27:0/2010616547 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88ac10afc0 con 0x7f88ac0fd2a0 2021-11-03T17:06:19.351 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.347+0000 7f88a97fa700 1 -- 172.21.15.27:0/2010616547 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f88a001b880 con 0x7f88ac0fd2a0 2021-11-03T17:06:19.351 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.347+0000 7f88b2e18700 1 -- 172.21.15.27:0/2010616547 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88ac10b450 con 0x7f88ac0fd2a0 2021-11-03T17:06:19.352 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.348+0000 7f88a97fa700 1 -- 172.21.15.27:0/2010616547 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) v1 ==== 36120+0+0 (secure 0 0 0) 0x7f88a000f070 con 0x7f88ac0fd2a0 2021-11-03T17:06:19.352 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.348+0000 7f88a97fa700 1 --2- 172.21.15.27:0/2010616547 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f889002fdb0 0x7f8890032270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:19.352 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.348+0000 7f88a97fa700 1 -- 172.21.15.27:0/2010616547 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7f88a00410a0 con 0x7f88ac0fd2a0 2021-11-03T17:06:19.352 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.348+0000 7f88abfff700 1 --2- 172.21.15.27:0/2010616547 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f889002fdb0 0x7f8890032270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:19.352 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.349+0000 7f88b2e18700 1 -- 172.21.15.27:0/2010616547 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88ac045ad0 con 0x7f88ac0fd2a0 2021-11-03T17:06:19.353 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.349+0000 7f88abfff700 1 --2- 172.21.15.27:0/2010616547 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f889002fdb0 0x7f8890032270 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f88980096a0 tx=0x7f8898006b40).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:19.353 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.353+0000 7f88a97fa700 1 -- 172.21.15.27:0/2010616547 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f88a0016080 con 0x7f88ac0fd2a0 2021-11-03T17:06:19.679 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:19.684+0000 7f88b2e18700 1 -- 172.21.15.27:0/2010616547 --> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi027", "addr": "172.21.15.27", "target": ["mon-mgr", ""]}) v1 -- 0x7f88ac10bcb0 con 0x7f889002fdb0 2021-11-03T17:06:20.352 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: Added host 'smithi027' 2021-11-03T17:06:20.352 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:20.356+0000 7f88a97fa700 1 -- 172.21.15.27:0/2010616547 <== mgr.14114 v2:172.21.15.27:6800/774407793 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+23 (secure 0 0 0) 0x7f88ac10bcb0 con 0x7f889002fdb0 2021-11-03T17:06:20.354 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:20.358+0000 7f88b2e18700 1 -- 172.21.15.27:0/2010616547 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f889002fdb0 msgr2=0x7f8890032270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:20.355 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:20.358+0000 7f88b2e18700 1 --2- 172.21.15.27:0/2010616547 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f889002fdb0 0x7f8890032270 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f88980096a0 tx=0x7f8898006b40).stop 2021-11-03T17:06:20.355 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:20.359+0000 7f88b2e18700 1 -- 172.21.15.27:0/2010616547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88ac0fd2a0 msgr2=0x7f88ac10a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:20.355 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:20.359+0000 7f88b2e18700 1 --2- 172.21.15.27:0/2010616547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88ac0fd2a0 0x7f88ac10a8d0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f88a0003000 tx=0x7f88a0006ee0).stop 2021-11-03T17:06:20.356 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:20.359+0000 7f88b2e18700 1 -- 172.21.15.27:0/2010616547 shutdown_connections 2021-11-03T17:06:20.356 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:20.359+0000 7f88b2e18700 1 --2- 172.21.15.27:0/2010616547 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f889002fdb0 0x7f8890032270 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:20.356 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:20.359+0000 7f88b2e18700 1 --2- 172.21.15.27:0/2010616547 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88ac0fd2a0 0x7f88ac10a8d0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:20.356 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:20.359+0000 7f88b2e18700 1 -- 172.21.15.27:0/2010616547 >> 172.21.15.27:0/2010616547 conn(0x7f88ac0f5a60 msgr2=0x7f88ac0ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:20.357 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:20.359+0000 7f88b2e18700 1 -- 172.21.15.27:0/2010616547 shutdown_connections 2021-11-03T17:06:20.357 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:20.359+0000 7f88b2e18700 1 -- 172.21.15.27:0/2010616547 wait complete. 2021-11-03T17:06:21.048 INFO:teuthology.orchestra.run.smithi027.stderr:Deploying mon service with default placement... 2021-11-03T17:06:21.049 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi027 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmp3f8lj10x:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpq2rb3_h7:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 orch apply mon 2021-11-03T17:06:21.721 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.722+0000 7fcb0c706700 1 Processor -- start 2021-11-03T17:06:21.721 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.722+0000 7fcb0c706700 1 -- start start 2021-11-03T17:06:21.722 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.722+0000 7fcb0c706700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb040fc080 0x7fcb040fc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:21.722 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.722+0000 7fcb0c706700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb040fc990 con 0x7fcb040fc080 2021-11-03T17:06:21.722 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.722+0000 7fcb0a4a2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb040fc080 0x7fcb040fc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:21.722 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.722+0000 7fcb0a4a2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb040fc080 0x7fcb040fc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54098/0 (socket says 172.21.15.27:54098) 2021-11-03T17:06:21.723 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.722+0000 7fcb0a4a2700 1 -- 172.21.15.27:0/385081939 learned_addr learned my addr 172.21.15.27:0/385081939 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:21.723 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.723+0000 7fcb0a4a2700 1 -- 172.21.15.27:0/385081939 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb040fcad0 con 0x7fcb040fc080 2021-11-03T17:06:21.723 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.723+0000 7fcb0a4a2700 1 --2- 172.21.15.27:0/385081939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb040fc080 0x7fcb040fc4a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fcb0000b5f0 tx=0x7fcb00006ee0).ready entity=mon.0 client_cookie=3d94668ebd6194cd server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:21.724 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.723+0000 7fcb094a0700 1 -- 172.21.15.27:0/385081939 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb0000d600 con 0x7fcb040fc080 2021-11-03T17:06:21.724 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.723+0000 7fcb094a0700 1 -- 172.21.15.27:0/385081939 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 983+0+0 (secure 0 0 0) 0x7fcb0000dbb0 con 0x7fcb040fc080 2021-11-03T17:06:21.724 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.723+0000 7fcb094a0700 1 -- 172.21.15.27:0/385081939 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb00011bb0 con 0x7fcb040fc080 2021-11-03T17:06:21.724 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.723+0000 7fcb0c706700 1 -- 172.21.15.27:0/385081939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb040fc080 msgr2=0x7fcb040fc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:21.725 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.723+0000 7fcb0c706700 1 --2- 172.21.15.27:0/385081939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb040fc080 0x7fcb040fc4a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fcb0000b5f0 tx=0x7fcb00006ee0).stop 2021-11-03T17:06:21.725 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.724+0000 7fcb0c706700 1 -- 172.21.15.27:0/385081939 shutdown_connections 2021-11-03T17:06:21.725 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.724+0000 7fcb0c706700 1 --2- 172.21.15.27:0/385081939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb040fc080 0x7fcb040fc4a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:21.726 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.724+0000 7fcb0c706700 1 -- 172.21.15.27:0/385081939 >> 172.21.15.27:0/385081939 conn(0x7fcb040f5a60 msgr2=0x7fcb040f7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:21.726 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.724+0000 7fcb0c706700 1 -- 172.21.15.27:0/385081939 shutdown_connections 2021-11-03T17:06:21.726 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.724+0000 7fcb0c706700 1 -- 172.21.15.27:0/385081939 wait complete. 2021-11-03T17:06:21.727 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.724+0000 7fcb0c706700 1 Processor -- start 2021-11-03T17:06:21.727 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.724+0000 7fcb0c706700 1 -- start start 2021-11-03T17:06:21.727 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.725+0000 7fcb0c706700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb040fc080 0x7fcb0410fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:21.727 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.725+0000 7fcb0c706700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb0410b430 con 0x7fcb040fc080 2021-11-03T17:06:21.728 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.725+0000 7fcb0a4a2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb040fc080 0x7fcb0410fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:21.728 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.725+0000 7fcb0a4a2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb040fc080 0x7fcb0410fa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54100/0 (socket says 172.21.15.27:54100) 2021-11-03T17:06:21.728 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.725+0000 7fcb0a4a2700 1 -- 172.21.15.27:0/72004934 learned_addr learned my addr 172.21.15.27:0/72004934 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:21.728 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.725+0000 7fcb0a4a2700 1 -- 172.21.15.27:0/72004934 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb0000b200 con 0x7fcb040fc080 2021-11-03T17:06:21.729 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.725+0000 7fcb0a4a2700 1 --2- 172.21.15.27:0/72004934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb040fc080 0x7fcb0410fa40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fcb00003000 tx=0x7fcb00006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:21.729 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.725+0000 7fcaf2ffd700 1 -- 172.21.15.27:0/72004934 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb00011e30 con 0x7fcb040fc080 2021-11-03T17:06:21.729 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.725+0000 7fcaf2ffd700 1 -- 172.21.15.27:0/72004934 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 983+0+0 (secure 0 0 0) 0x7fcb0000d9c0 con 0x7fcb040fc080 2021-11-03T17:06:21.729 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.725+0000 7fcb0c706700 1 -- 172.21.15.27:0/72004934 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb040fa9f0 con 0x7fcb040fc080 2021-11-03T17:06:21.730 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.725+0000 7fcaf2ffd700 1 -- 172.21.15.27:0/72004934 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcb0001b740 con 0x7fcb040fc080 2021-11-03T17:06:21.730 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.725+0000 7fcb0c706700 1 -- 172.21.15.27:0/72004934 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb0410ff30 con 0x7fcb040fc080 2021-11-03T17:06:21.730 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.726+0000 7fcaf2ffd700 1 -- 172.21.15.27:0/72004934 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) v1 ==== 36120+0+0 (secure 0 0 0) 0x7fcb0000f070 con 0x7fcb040fc080 2021-11-03T17:06:21.731 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.726+0000 7fcaf2ffd700 1 --2- 172.21.15.27:0/72004934 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7fcaec02fd50 0x7fcaec032210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:21.732 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.726+0000 7fcaf2ffd700 1 -- 172.21.15.27:0/72004934 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7fcb00040ff0 con 0x7fcb040fc080 2021-11-03T17:06:21.732 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.726+0000 7fcb09ca1700 1 --2- 172.21.15.27:0/72004934 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7fcaec02fd50 0x7fcaec032210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:21.732 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.727+0000 7fcb09ca1700 1 --2- 172.21.15.27:0/72004934 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7fcaec02fd50 0x7fcaec032210 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fcaf40096a0 tx=0x7fcaf4006b40).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:21.732 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.727+0000 7fcb0c706700 1 -- 172.21.15.27:0/72004934 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb04045ad0 con 0x7fcb040fc080 2021-11-03T17:06:21.733 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:21.731+0000 7fcaf2ffd700 1 -- 172.21.15.27:0/72004934 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fcb00024410 con 0x7fcb040fc080 2021-11-03T17:06:21.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:21 smithi027 conmon[32322]: audit 2021-11-03T17:06:19.685231+0000 mgr.smithi027.ufhhvb (mgr.14114) 8 : audit [DBG] from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi027", "addr": "172.21.15.27", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:06:21.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:21 smithi027 conmon[32322]: audit 2021-11-03T17:06:20.355772+0000 mon.smithi027 (mon.0) 54 : audit [INF] from='mgr.14114 172.21.15.27:0/784375816' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:06:21.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:21 smithi027 conmon[32322]: cephadm 2021-11-03T17:06:20.356102+0000 mgr.smithi027.ufhhvb (mgr.14114) 9 : cephadm [INF] Added host smithi027 2021-11-03T17:06:21.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:21 smithi027 conmon[32322]: audit 2021-11-03T17:06:20.356382+0000 mon.smithi027 (mon.0) 55 : audit [DBG] from='mgr.14114 172.21.15.27:0/784375816' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:06:22.071 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:22.076+0000 7fcb0c706700 1 -- 172.21.15.27:0/72004934 --> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb040fb940 con 0x7fcaec02fd50 2021-11-03T17:06:22.076 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:22.081+0000 7fcaf2ffd700 1 -- 172.21.15.27:0/72004934 <== mgr.14114 v2:172.21.15.27:6800/774407793 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fcb040fb940 con 0x7fcaec02fd50 2021-11-03T17:06:22.076 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: Scheduled mon update... 2021-11-03T17:06:22.078 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:22.083+0000 7fcb0c706700 1 -- 172.21.15.27:0/72004934 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7fcaec02fd50 msgr2=0x7fcaec032210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:22.079 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:22.083+0000 7fcb0c706700 1 --2- 172.21.15.27:0/72004934 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7fcaec02fd50 0x7fcaec032210 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fcaf40096a0 tx=0x7fcaf4006b40).stop 2021-11-03T17:06:22.079 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:22.083+0000 7fcb0c706700 1 -- 172.21.15.27:0/72004934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb040fc080 msgr2=0x7fcb0410fa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:22.080 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:22.083+0000 7fcb0c706700 1 --2- 172.21.15.27:0/72004934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb040fc080 0x7fcb0410fa40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fcb00003000 tx=0x7fcb00006ee0).stop 2021-11-03T17:06:22.080 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:22.083+0000 7fcb0c706700 1 -- 172.21.15.27:0/72004934 shutdown_connections 2021-11-03T17:06:22.080 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:22.083+0000 7fcb0c706700 1 --2- 172.21.15.27:0/72004934 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7fcaec02fd50 0x7fcaec032210 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:22.080 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:22.083+0000 7fcb0c706700 1 --2- 172.21.15.27:0/72004934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb040fc080 0x7fcb0410fa40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:22.080 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:22.083+0000 7fcb0c706700 1 -- 172.21.15.27:0/72004934 >> 172.21.15.27:0/72004934 conn(0x7fcb040f5a60 msgr2=0x7fcb040f74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:22.081 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:22.083+0000 7fcb0c706700 1 -- 172.21.15.27:0/72004934 shutdown_connections 2021-11-03T17:06:22.081 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:22.083+0000 7fcb0c706700 1 -- 172.21.15.27:0/72004934 wait complete. 2021-11-03T17:06:22.816 INFO:teuthology.orchestra.run.smithi027.stderr:Deploying mgr service with default placement... 2021-11-03T17:06:22.816 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi027 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmp3f8lj10x:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpq2rb3_h7:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 orch apply mgr 2021-11-03T17:06:23.364 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:23 smithi027 conmon[32322]: audit 2021-11-03T17:06:22.077510+0000 mgr.smithi027.ufhhvb (mgr.14114) 10 : audit [DBG] from='client.14134 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:06:23.364 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:23 smithi027 conmon[32322]: cephadm 2021-11-03T17:06:22.078416+0000 mgr.smithi027.ufhhvb (mgr.14114) 11 : cephadm [INF] Saving service mon spec with placement count:5 2021-11-03T17:06:23.364 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:23 smithi027 conmon[32322]: audit 2021-11-03T17:06:22.081445+0000 mon.smithi027 (mon.0) 56 : audit [INF] from='mgr.14114 172.21.15.27:0/784375816' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:06:23.836 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.832+0000 7f89f66ac700 1 Processor -- start 2021-11-03T17:06:23.837 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.832+0000 7f89f66ac700 1 -- start start 2021-11-03T17:06:23.837 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.833+0000 7f89f66ac700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89f006fb60 0x7f89f00fe840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:23.838 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.833+0000 7f89f66ac700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f00fed30 con 0x7f89f006fb60 2021-11-03T17:06:23.838 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.833+0000 7f89effff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89f006fb60 0x7f89f00fe840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:23.839 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.833+0000 7f89effff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89f006fb60 0x7f89f00fe840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54104/0 (socket says 172.21.15.27:54104) 2021-11-03T17:06:23.839 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.833+0000 7f89effff700 1 -- 172.21.15.27:0/3934050327 learned_addr learned my addr 172.21.15.27:0/3934050327 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:23.840 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.833+0000 7f89effff700 1 -- 172.21.15.27:0/3934050327 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f00fee70 con 0x7f89f006fb60 2021-11-03T17:06:23.840 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.833+0000 7f89effff700 1 --2- 172.21.15.27:0/3934050327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89f006fb60 0x7f89f00fe840 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f89e400b5f0 tx=0x7f89e4006ee0).ready entity=mon.0 client_cookie=2297775e01de41f4 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:23.840 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.834+0000 7f89eeffd700 1 -- 172.21.15.27:0/3934050327 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f89e400d600 con 0x7f89f006fb60 2021-11-03T17:06:23.841 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.834+0000 7f89eeffd700 1 -- 172.21.15.27:0/3934050327 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f89e400dbf0 con 0x7f89f006fb60 2021-11-03T17:06:23.841 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.834+0000 7f89eeffd700 1 -- 172.21.15.27:0/3934050327 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f89e4011bf0 con 0x7f89f006fb60 2021-11-03T17:06:23.842 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.834+0000 7f89f66ac700 1 -- 172.21.15.27:0/3934050327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89f006fb60 msgr2=0x7f89f00fe840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:23.842 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.834+0000 7f89f66ac700 1 --2- 172.21.15.27:0/3934050327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89f006fb60 0x7f89f00fe840 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f89e400b5f0 tx=0x7f89e4006ee0).stop 2021-11-03T17:06:23.843 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.834+0000 7f89f66ac700 1 -- 172.21.15.27:0/3934050327 shutdown_connections 2021-11-03T17:06:23.843 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.834+0000 7f89f66ac700 1 --2- 172.21.15.27:0/3934050327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89f006fb60 0x7f89f00fe840 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:23.844 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.834+0000 7f89f66ac700 1 -- 172.21.15.27:0/3934050327 >> 172.21.15.27:0/3934050327 conn(0x7f89f00f6260 msgr2=0x7f89f00f86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:23.844 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.835+0000 7f89f66ac700 1 -- 172.21.15.27:0/3934050327 shutdown_connections 2021-11-03T17:06:23.844 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.835+0000 7f89f66ac700 1 -- 172.21.15.27:0/3934050327 wait complete. 2021-11-03T17:06:23.845 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.835+0000 7f89f66ac700 1 Processor -- start 2021-11-03T17:06:23.845 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.835+0000 7f89f66ac700 1 -- start start 2021-11-03T17:06:23.846 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.836+0000 7f89f66ac700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89f006fb60 0x7f89f010a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:23.846 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.836+0000 7f89f66ac700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f010ae40 con 0x7f89f006fb60 2021-11-03T17:06:23.847 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.836+0000 7f89effff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89f006fb60 0x7f89f010a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:23.847 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.836+0000 7f89effff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89f006fb60 0x7f89f010a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54106/0 (socket says 172.21.15.27:54106) 2021-11-03T17:06:23.848 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.836+0000 7f89effff700 1 -- 172.21.15.27:0/1283110440 learned_addr learned my addr 172.21.15.27:0/1283110440 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:23.848 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.836+0000 7f89effff700 1 -- 172.21.15.27:0/1283110440 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e400b200 con 0x7f89f006fb60 2021-11-03T17:06:23.849 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.836+0000 7f89effff700 1 --2- 172.21.15.27:0/1283110440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89f006fb60 0x7f89f010a950 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f89e400bdc0 tx=0x7f89e4006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:23.849 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.836+0000 7f89ecff9700 1 -- 172.21.15.27:0/1283110440 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f89e401b410 con 0x7f89f006fb60 2021-11-03T17:06:23.849 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.836+0000 7f89ecff9700 1 -- 172.21.15.27:0/1283110440 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f89e401ba00 con 0x7f89f006fb60 2021-11-03T17:06:23.850 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.836+0000 7f89f66ac700 1 -- 172.21.15.27:0/1283110440 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89f010b040 con 0x7f89f006fb60 2021-11-03T17:06:23.850 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.836+0000 7f89ecff9700 1 -- 172.21.15.27:0/1283110440 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f89e401a6b0 con 0x7f89f006fb60 2021-11-03T17:06:23.851 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.837+0000 7f89f66ac700 1 -- 172.21.15.27:0/1283110440 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f010b4d0 con 0x7f89f006fb60 2021-11-03T17:06:23.851 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.838+0000 7f89f66ac700 1 -- 172.21.15.27:0/1283110440 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89f0045ad0 con 0x7f89f006fb60 2021-11-03T17:06:23.852 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.838+0000 7f89ecff9700 1 -- 172.21.15.27:0/1283110440 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) v1 ==== 36120+0+0 (secure 0 0 0) 0x7f89e400f070 con 0x7f89f006fb60 2021-11-03T17:06:23.852 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.838+0000 7f89ecff9700 1 --2- 172.21.15.27:0/1283110440 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f89dc02fdb0 0x7f89dc032270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:23.853 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.838+0000 7f89ecff9700 1 -- 172.21.15.27:0/1283110440 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7f89e40418e0 con 0x7f89f006fb60 2021-11-03T17:06:23.853 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.839+0000 7f89ef7fe700 1 --2- 172.21.15.27:0/1283110440 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f89dc02fdb0 0x7f89dc032270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:23.854 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.839+0000 7f89ef7fe700 1 --2- 172.21.15.27:0/1283110440 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f89dc02fdb0 0x7f89dc032270 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f89e00096a0 tx=0x7f89e0006b40).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:23.855 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:23.842+0000 7f89ecff9700 1 -- 172.21.15.27:0/1283110440 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f89e4006a70 con 0x7f89f006fb60 2021-11-03T17:06:24.202 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:24.206+0000 7f89f66ac700 1 -- 172.21.15.27:0/1283110440 --> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7f89f010bd30 con 0x7f89dc02fdb0 2021-11-03T17:06:24.206 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:24.211+0000 7f89ecff9700 1 -- 172.21.15.27:0/1283110440 <== mgr.14114 v2:172.21.15.27:6800/774407793 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f89f010bd30 con 0x7f89dc02fdb0 2021-11-03T17:06:24.206 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: Scheduled mgr update... 2021-11-03T17:06:24.209 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:24.213+0000 7f89f66ac700 1 -- 172.21.15.27:0/1283110440 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f89dc02fdb0 msgr2=0x7f89dc032270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:24.209 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:24.213+0000 7f89f66ac700 1 --2- 172.21.15.27:0/1283110440 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f89dc02fdb0 0x7f89dc032270 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f89e00096a0 tx=0x7f89e0006b40).stop 2021-11-03T17:06:24.209 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:24.213+0000 7f89f66ac700 1 -- 172.21.15.27:0/1283110440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89f006fb60 msgr2=0x7f89f010a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:24.209 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:24.213+0000 7f89f66ac700 1 --2- 172.21.15.27:0/1283110440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89f006fb60 0x7f89f010a950 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f89e400bdc0 tx=0x7f89e4006ee0).stop 2021-11-03T17:06:24.210 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:24.213+0000 7f89f66ac700 1 -- 172.21.15.27:0/1283110440 shutdown_connections 2021-11-03T17:06:24.210 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:24.213+0000 7f89f66ac700 1 --2- 172.21.15.27:0/1283110440 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f89dc02fdb0 0x7f89dc032270 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:24.210 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:24.213+0000 7f89f66ac700 1 --2- 172.21.15.27:0/1283110440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f89f006fb60 0x7f89f010a950 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:24.210 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:24.213+0000 7f89f66ac700 1 -- 172.21.15.27:0/1283110440 >> 172.21.15.27:0/1283110440 conn(0x7f89f00f6260 msgr2=0x7f89f00f7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:24.211 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:24.213+0000 7f89f66ac700 1 -- 172.21.15.27:0/1283110440 shutdown_connections 2021-11-03T17:06:24.211 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:24.213+0000 7f89f66ac700 1 -- 172.21.15.27:0/1283110440 wait complete. 2021-11-03T17:06:24.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:24 smithi027 conmon[32322]: audit 2021-11-03T17:06:23.321350+0000 mon.smithi027 (mon.0) 57 : audit [INF] from='mgr.14114 172.21.15.27:0/784375816' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:06:24.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:24 smithi027 conmon[32322]: audit 2021-11-03T17:06:23.592162+0000 mon.smithi027 (mon.0) 58 : audit [INF] from='mgr.14114 172.21.15.27:0/784375816' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:06:24.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:24 smithi027 conmon[32322]: audit 2021-11-03T17:06:24.211260+0000 mon.smithi027 (mon.0) 59 : audit [INF] from='mgr.14114 172.21.15.27:0/784375816' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:06:25.007 INFO:teuthology.orchestra.run.smithi027.stderr:Deploying crash service with default placement... 2021-11-03T17:06:25.007 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi027 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmp3f8lj10x:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpq2rb3_h7:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 orch apply crash 2021-11-03T17:06:25.722 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:25 smithi027 conmon[32322]: audit 2021-11-03T17:06:24.208466+0000 mgr.smithi027.ufhhvb (mgr.14114) 12 : audit [DBG] from='client.14136 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:06:25.722 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:25 smithi027 conmon[32322]: cephadm 2021-11-03T17:06:24.209068+0000 mgr.smithi027.ufhhvb (mgr.14114) 13 : cephadm [INF] Saving service mgr spec with placement count:2 2021-11-03T17:06:25.722 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:25 smithi027 conmon[32322]: audit 2021-11-03T17:06:24.495906+0000 mon.smithi027 (mon.0) 60 : audit [INF] from='mgr.14114 172.21.15.27:0/784375816' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:06:26.096 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.097+0000 7f9d8d304700 1 Processor -- start 2021-11-03T17:06:26.096 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.097+0000 7f9d8d304700 1 -- start start 2021-11-03T17:06:26.097 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.097+0000 7f9d8d304700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d880fd260 0x7f9d880fd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:26.097 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.097+0000 7f9d8d304700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d880fdb70 con 0x7f9d880fd260 2021-11-03T17:06:26.098 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.098+0000 7f9d86ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d880fd260 0x7f9d880fd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:26.098 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.098+0000 7f9d86ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d880fd260 0x7f9d880fd680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54110/0 (socket says 172.21.15.27:54110) 2021-11-03T17:06:26.098 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.098+0000 7f9d86ffd700 1 -- 172.21.15.27:0/2593382631 learned_addr learned my addr 172.21.15.27:0/2593382631 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:26.099 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.098+0000 7f9d86ffd700 1 -- 172.21.15.27:0/2593382631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d880fdcb0 con 0x7f9d880fd260 2021-11-03T17:06:26.099 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.098+0000 7f9d86ffd700 1 --2- 172.21.15.27:0/2593382631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d880fd260 0x7f9d880fd680 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f9d7800b5f0 tx=0x7f9d78006ee0).ready entity=mon.0 client_cookie=e7138b4c4bf1e721 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:26.099 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.098+0000 7f9d85ffb700 1 -- 172.21.15.27:0/2593382631 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9d7800d600 con 0x7f9d880fd260 2021-11-03T17:06:26.100 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.098+0000 7f9d85ffb700 1 -- 172.21.15.27:0/2593382631 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f9d7800dbf0 con 0x7f9d880fd260 2021-11-03T17:06:26.100 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.098+0000 7f9d85ffb700 1 -- 172.21.15.27:0/2593382631 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9d78011bf0 con 0x7f9d880fd260 2021-11-03T17:06:26.100 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.099+0000 7f9d8d304700 1 -- 172.21.15.27:0/2593382631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d880fd260 msgr2=0x7f9d880fd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:26.100 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.099+0000 7f9d8d304700 1 --2- 172.21.15.27:0/2593382631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d880fd260 0x7f9d880fd680 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f9d7800b5f0 tx=0x7f9d78006ee0).stop 2021-11-03T17:06:26.101 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.099+0000 7f9d8d304700 1 -- 172.21.15.27:0/2593382631 shutdown_connections 2021-11-03T17:06:26.101 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.099+0000 7f9d8d304700 1 --2- 172.21.15.27:0/2593382631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d880fd260 0x7f9d880fd680 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:26.101 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.099+0000 7f9d8d304700 1 -- 172.21.15.27:0/2593382631 >> 172.21.15.27:0/2593382631 conn(0x7f9d880f5a60 msgr2=0x7f9d880f7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:26.101 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.099+0000 7f9d8d304700 1 -- 172.21.15.27:0/2593382631 shutdown_connections 2021-11-03T17:06:26.101 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.099+0000 7f9d8d304700 1 -- 172.21.15.27:0/2593382631 wait complete. 2021-11-03T17:06:26.102 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.100+0000 7f9d8d304700 1 Processor -- start 2021-11-03T17:06:26.102 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.100+0000 7f9d8d304700 1 -- start start 2021-11-03T17:06:26.102 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.100+0000 7f9d8d304700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d880fd260 0x7f9d88108710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:26.103 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.100+0000 7f9d8d304700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d88108c00 con 0x7f9d880fd260 2021-11-03T17:06:26.103 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.100+0000 7f9d86ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d880fd260 0x7f9d88108710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:26.103 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.100+0000 7f9d86ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d880fd260 0x7f9d88108710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54112/0 (socket says 172.21.15.27:54112) 2021-11-03T17:06:26.103 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.100+0000 7f9d86ffd700 1 -- 172.21.15.27:0/2732181892 learned_addr learned my addr 172.21.15.27:0/2732181892 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:26.104 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.100+0000 7f9d86ffd700 1 -- 172.21.15.27:0/2732181892 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d7800b200 con 0x7f9d880fd260 2021-11-03T17:06:26.104 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.101+0000 7f9d86ffd700 1 --2- 172.21.15.27:0/2732181892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d880fd260 0x7f9d88108710 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f9d78011e30 tx=0x7f9d78006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:26.104 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.101+0000 7f9d7f7fe700 1 -- 172.21.15.27:0/2732181892 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9d7801b410 con 0x7f9d880fd260 2021-11-03T17:06:26.105 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.101+0000 7f9d7f7fe700 1 -- 172.21.15.27:0/2732181892 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f9d7801ba00 con 0x7f9d880fd260 2021-11-03T17:06:26.105 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.101+0000 7f9d8d304700 1 -- 172.21.15.27:0/2732181892 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d88108e00 con 0x7f9d880fd260 2021-11-03T17:06:26.105 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.101+0000 7f9d7f7fe700 1 -- 172.21.15.27:0/2732181892 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f9d7801a750 con 0x7f9d880fd260 2021-11-03T17:06:26.107 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.101+0000 7f9d8d304700 1 -- 172.21.15.27:0/2732181892 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d88109290 con 0x7f9d880fd260 2021-11-03T17:06:26.107 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.102+0000 7f9d8d304700 1 -- 172.21.15.27:0/2732181892 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d88045ad0 con 0x7f9d880fd260 2021-11-03T17:06:26.107 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.103+0000 7f9d7f7fe700 1 -- 172.21.15.27:0/2732181892 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) v1 ==== 36120+0+0 (secure 0 0 0) 0x7f9d7802b070 con 0x7f9d880fd260 2021-11-03T17:06:26.107 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.103+0000 7f9d7f7fe700 1 --2- 172.21.15.27:0/2732181892 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f9d680341f0 0x7f9d680366b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:26.108 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.103+0000 7f9d7f7fe700 1 -- 172.21.15.27:0/2732181892 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7f9d78041930 con 0x7f9d880fd260 2021-11-03T17:06:26.108 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.103+0000 7f9d867fc700 1 --2- 172.21.15.27:0/2732181892 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f9d680341f0 0x7f9d680366b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:26.108 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.103+0000 7f9d867fc700 1 --2- 172.21.15.27:0/2732181892 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f9d680341f0 0x7f9d680366b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f9d700096a0 tx=0x7f9d70006b40).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:26.108 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.107+0000 7f9d7f7fe700 1 -- 172.21.15.27:0/2732181892 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f9d78023ba0 con 0x7f9d880fd260 2021-11-03T17:06:26.446 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.451+0000 7f9d8d304700 1 -- 172.21.15.27:0/2732181892 --> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d88109af0 con 0x7f9d680341f0 2021-11-03T17:06:26.450 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.455+0000 7f9d7f7fe700 1 -- 172.21.15.27:0/2732181892 <== mgr.14114 v2:172.21.15.27:6800/774407793 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f9d88109af0 con 0x7f9d680341f0 2021-11-03T17:06:26.450 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: Scheduled crash update... 2021-11-03T17:06:26.453 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.457+0000 7f9d8d304700 1 -- 172.21.15.27:0/2732181892 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f9d680341f0 msgr2=0x7f9d680366b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:26.453 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.457+0000 7f9d8d304700 1 --2- 172.21.15.27:0/2732181892 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f9d680341f0 0x7f9d680366b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f9d700096a0 tx=0x7f9d70006b40).stop 2021-11-03T17:06:26.453 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.457+0000 7f9d8d304700 1 -- 172.21.15.27:0/2732181892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d880fd260 msgr2=0x7f9d88108710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:26.454 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.457+0000 7f9d8d304700 1 --2- 172.21.15.27:0/2732181892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d880fd260 0x7f9d88108710 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f9d78011e30 tx=0x7f9d78006ee0).stop 2021-11-03T17:06:26.454 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.457+0000 7f9d8d304700 1 -- 172.21.15.27:0/2732181892 shutdown_connections 2021-11-03T17:06:26.454 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.457+0000 7f9d8d304700 1 --2- 172.21.15.27:0/2732181892 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f9d680341f0 0x7f9d680366b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:26.455 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.457+0000 7f9d8d304700 1 --2- 172.21.15.27:0/2732181892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9d880fd260 0x7f9d88108710 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:26.455 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.457+0000 7f9d8d304700 1 -- 172.21.15.27:0/2732181892 >> 172.21.15.27:0/2732181892 conn(0x7f9d880f5a60 msgr2=0x7f9d880f7220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:26.455 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.457+0000 7f9d8d304700 1 -- 172.21.15.27:0/2732181892 shutdown_connections 2021-11-03T17:06:26.455 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:26.457+0000 7f9d8d304700 1 -- 172.21.15.27:0/2732181892 wait complete. 2021-11-03T17:06:26.990 INFO:teuthology.orchestra.run.smithi027.stderr:Enabling mgr prometheus module... 2021-11-03T17:06:26.990 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi027 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmp3f8lj10x:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpq2rb3_h7:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 mgr module enable prometheus 2021-11-03T17:06:27.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:27 smithi027 conmon[32322]: audit 2021-11-03T17:06:26.452465+0000 2021-11-03T17:06:27.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:27 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14114) 14 : audit [DBG] from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:06:27.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:27 smithi027 conmon[32322]: cephadm 2021-11-03T17:06:26.453192+0000 mgr.smithi027.ufhhvb (mgr.14114) 15 : cephadm [INF] Saving service crash spec with placement * 2021-11-03T17:06:27.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:27 smithi027 conmon[32322]: audit 2021-11-03T17:06:26.455461+0000 mon.smithi027 (mon.0) 61 : audit [INF] from='mgr.14114 172.21.15.27:0/784375816' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:06:27.649 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.650+0000 7fb56740a700 1 Processor -- start 2021-11-03T17:06:27.650 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.650+0000 7fb56740a700 1 -- start start 2021-11-03T17:06:27.650 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.650+0000 7fb56740a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb56006fb00 0x7fb5600fe8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:27.650 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.650+0000 7fb56740a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb56006ff20 con 0x7fb56006fb00 2021-11-03T17:06:27.651 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.650+0000 7fb5651a6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb56006fb00 0x7fb5600fe8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:27.651 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.650+0000 7fb5651a6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb56006fb00 0x7fb5600fe8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54116/0 (socket says 172.21.15.27:54116) 2021-11-03T17:06:27.651 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.650+0000 7fb5651a6700 1 -- 172.21.15.27:0/470937686 learned_addr learned my addr 172.21.15.27:0/470937686 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:27.651 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.650+0000 7fb5651a6700 1 -- 172.21.15.27:0/470937686 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5600fedb0 con 0x7fb56006fb00 2021-11-03T17:06:27.652 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.651+0000 7fb5651a6700 1 --2- 172.21.15.27:0/470937686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb56006fb00 0x7fb5600fe8c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fb55c00b5f0 tx=0x7fb55c006ee0).ready entity=mon.0 client_cookie=f2c1f28f6b34e11 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:27.652 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.651+0000 7fb54ffff700 1 -- 172.21.15.27:0/470937686 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb55c00d600 con 0x7fb56006fb00 2021-11-03T17:06:27.652 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.651+0000 7fb54ffff700 1 -- 172.21.15.27:0/470937686 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7fb55c00dbf0 con 0x7fb56006fb00 2021-11-03T17:06:27.652 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.651+0000 7fb54ffff700 1 -- 172.21.15.27:0/470937686 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb55c011bf0 con 0x7fb56006fb00 2021-11-03T17:06:27.652 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.651+0000 7fb56740a700 1 -- 172.21.15.27:0/470937686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb56006fb00 msgr2=0x7fb5600fe8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:27.653 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.651+0000 7fb56740a700 1 --2- 172.21.15.27:0/470937686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb56006fb00 0x7fb5600fe8c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fb55c00b5f0 tx=0x7fb55c006ee0).stop 2021-11-03T17:06:27.653 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.651+0000 7fb56740a700 1 -- 172.21.15.27:0/470937686 shutdown_connections 2021-11-03T17:06:27.653 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.651+0000 7fb56740a700 1 --2- 172.21.15.27:0/470937686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb56006fb00 0x7fb5600fe8c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:27.653 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.651+0000 7fb56740a700 1 -- 172.21.15.27:0/470937686 >> 172.21.15.27:0/470937686 conn(0x7fb5600f62e0 msgr2=0x7fb5600f8700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:27.654 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.652+0000 7fb56740a700 1 -- 172.21.15.27:0/470937686 shutdown_connections 2021-11-03T17:06:27.654 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.652+0000 7fb56740a700 1 -- 172.21.15.27:0/470937686 wait complete. 2021-11-03T17:06:27.654 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.652+0000 7fb56740a700 1 Processor -- start 2021-11-03T17:06:27.654 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.652+0000 7fb56740a700 1 -- start start 2021-11-03T17:06:27.654 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.652+0000 7fb56740a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb56006fb00 0x7fb5600fc340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:27.655 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.652+0000 7fb56740a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5600fa7c0 con 0x7fb56006fb00 2021-11-03T17:06:27.655 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.652+0000 7fb5651a6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb56006fb00 0x7fb5600fc340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:27.655 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.652+0000 7fb5651a6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb56006fb00 0x7fb5600fc340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54118/0 (socket says 172.21.15.27:54118) 2021-11-03T17:06:27.655 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.652+0000 7fb5651a6700 1 -- 172.21.15.27:0/3119440655 learned_addr learned my addr 172.21.15.27:0/3119440655 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:27.656 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.653+0000 7fb5651a6700 1 -- 172.21.15.27:0/3119440655 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55c00b200 con 0x7fb56006fb00 2021-11-03T17:06:27.656 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.653+0000 7fb5651a6700 1 --2- 172.21.15.27:0/3119440655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb56006fb00 0x7fb5600fc340 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fb55c011de0 tx=0x7fb55c006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:27.656 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.653+0000 7fb54dffb700 1 -- 172.21.15.27:0/3119440655 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb55c01b410 con 0x7fb56006fb00 2021-11-03T17:06:27.656 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.653+0000 7fb54dffb700 1 -- 172.21.15.27:0/3119440655 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7fb55c01ba00 con 0x7fb56006fb00 2021-11-03T17:06:27.656 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.653+0000 7fb56740a700 1 -- 172.21.15.27:0/3119440655 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5600fa960 con 0x7fb56006fb00 2021-11-03T17:06:27.657 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.653+0000 7fb54dffb700 1 -- 172.21.15.27:0/3119440655 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb55c01a770 con 0x7fb56006fb00 2021-11-03T17:06:27.657 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.653+0000 7fb56740a700 1 -- 172.21.15.27:0/3119440655 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5600fadf0 con 0x7fb56006fb00 2021-11-03T17:06:27.657 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.654+0000 7fb54dffb700 1 -- 172.21.15.27:0/3119440655 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 8) v1 ==== 36120+0+0 (secure 0 0 0) 0x7fb55c02b070 con 0x7fb56006fb00 2021-11-03T17:06:27.657 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.654+0000 7fb54dffb700 1 --2- 172.21.15.27:0/3119440655 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7fb54802fdb0 0x7fb548032270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:27.657 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.654+0000 7fb54dffb700 1 -- 172.21.15.27:0/3119440655 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7fb55c041560 con 0x7fb56006fb00 2021-11-03T17:06:27.658 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.654+0000 7fb5649a5700 1 --2- 172.21.15.27:0/3119440655 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7fb54802fdb0 0x7fb548032270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:27.658 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.654+0000 7fb56740a700 1 -- 172.21.15.27:0/3119440655 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb560059100 con 0x7fb56006fb00 2021-11-03T17:06:27.658 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.655+0000 7fb5649a5700 1 --2- 172.21.15.27:0/3119440655 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7fb54802fdb0 0x7fb548032270 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fb5500096a0 tx=0x7fb550006b40).ready entity=mgr.14114 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:27.659 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:27.659+0000 7fb54dffb700 1 -- 172.21.15.27:0/3119440655 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fb55c01f080 con 0x7fb56006fb00 2021-11-03T17:06:28.013 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:28.018+0000 7fb56740a700 1 -- 172.21.15.27:0/3119440655 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "prometheus"} v 0) v1 -- 0x7fb560045ad0 con 0x7fb56006fb00 2021-11-03T17:06:28.453 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:28.458+0000 7fb54dffb700 1 -- 172.21.15.27:0/3119440655 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 9) v1 ==== 36134+0+0 (secure 0 0 0) 0x7fb55c00f070 con 0x7fb56006fb00 2021-11-03T17:06:28.454 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:28.460+0000 7fb54dffb700 1 -- 172.21.15.27:0/3119440655 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "prometheus"}]=0 v9) v1 ==== 89+0+0 (secure 0 0 0) 0x7fb55c023930 con 0x7fb56006fb00 2021-11-03T17:06:28.458 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:28.462+0000 7fb56740a700 1 -- 172.21.15.27:0/3119440655 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7fb54802fdb0 msgr2=0x7fb548032270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:28.458 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:28.462+0000 7fb56740a700 1 --2- 172.21.15.27:0/3119440655 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7fb54802fdb0 0x7fb548032270 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7fb5500096a0 tx=0x7fb550006b40).stop 2021-11-03T17:06:28.458 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:28.462+0000 7fb56740a700 1 -- 172.21.15.27:0/3119440655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb56006fb00 msgr2=0x7fb5600fc340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:28.459 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:28.462+0000 7fb56740a700 1 --2- 172.21.15.27:0/3119440655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb56006fb00 0x7fb5600fc340 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fb55c011de0 tx=0x7fb55c006ee0).stop 2021-11-03T17:06:28.459 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:28.462+0000 7fb56740a700 1 -- 172.21.15.27:0/3119440655 shutdown_connections 2021-11-03T17:06:28.459 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:28.462+0000 7fb56740a700 1 --2- 172.21.15.27:0/3119440655 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7fb54802fdb0 0x7fb548032270 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:28.459 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:28.462+0000 7fb56740a700 1 --2- 172.21.15.27:0/3119440655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb56006fb00 0x7fb5600fc340 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:28.459 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:28.462+0000 7fb56740a700 1 -- 172.21.15.27:0/3119440655 >> 172.21.15.27:0/3119440655 conn(0x7fb5600f62e0 msgr2=0x7fb5600f7d30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:28.460 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:28.463+0000 7fb56740a700 1 -- 172.21.15.27:0/3119440655 shutdown_connections 2021-11-03T17:06:28.460 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:28.463+0000 7fb56740a700 1 -- 172.21.15.27:0/3119440655 wait complete. 2021-11-03T17:06:28.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:28 smithi027 conmon[32322]: audit 2021-11-03T17:06:28.018781+0000 mon.smithi027 (mon.0) 62 : audit [INF] from='client.? 172.21.15.27:0/3119440655' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2021-11-03T17:06:29.006 INFO:teuthology.orchestra.run.smithi027.stderr:Deploying prometheus service with default placement... 2021-11-03T17:06:29.007 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi027 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmp3f8lj10x:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpq2rb3_h7:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 orch apply prometheus 2021-11-03T17:06:29.691 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.691+0000 7f60692b5700 1 Processor -- start 2021-11-03T17:06:29.692 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.691+0000 7f60692b5700 1 -- start start 2021-11-03T17:06:29.692 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.691+0000 7f60692b5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f606406fb60 0x7f60640fe690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:29.693 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.691+0000 7f60692b5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60640feb80 con 0x7f606406fb60 2021-11-03T17:06:29.693 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.691+0000 7f6063fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f606406fb60 0x7f60640fe690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:29.694 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.692+0000 7f6063fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f606406fb60 0x7f60640fe690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54132/0 (socket says 172.21.15.27:54132) 2021-11-03T17:06:29.694 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.692+0000 7f6063fff700 1 -- 172.21.15.27:0/140350401 learned_addr learned my addr 172.21.15.27:0/140350401 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:29.694 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.692+0000 7f6063fff700 1 -- 172.21.15.27:0/140350401 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60640fecc0 con 0x7f606406fb60 2021-11-03T17:06:29.695 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.692+0000 7f6063fff700 1 --2- 172.21.15.27:0/140350401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f606406fb60 0x7f60640fe690 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f605400b5f0 tx=0x7f6054006ee0).ready entity=mon.0 client_cookie=38d901e97ab5c79b server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:29.695 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.692+0000 7f6062ffd700 1 -- 172.21.15.27:0/140350401 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f605400d600 con 0x7f606406fb60 2021-11-03T17:06:29.695 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.692+0000 7f6062ffd700 1 -- 172.21.15.27:0/140350401 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f605400dbf0 con 0x7f606406fb60 2021-11-03T17:06:29.695 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.692+0000 7f6062ffd700 1 -- 172.21.15.27:0/140350401 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6054011bf0 con 0x7f606406fb60 2021-11-03T17:06:29.696 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.693+0000 7f60692b5700 1 -- 172.21.15.27:0/140350401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f606406fb60 msgr2=0x7f60640fe690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:29.696 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.693+0000 7f60692b5700 1 --2- 172.21.15.27:0/140350401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f606406fb60 0x7f60640fe690 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f605400b5f0 tx=0x7f6054006ee0).stop 2021-11-03T17:06:29.696 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.693+0000 7f60692b5700 1 -- 172.21.15.27:0/140350401 shutdown_connections 2021-11-03T17:06:29.696 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.693+0000 7f60692b5700 1 --2- 172.21.15.27:0/140350401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f606406fb60 0x7f60640fe690 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:29.696 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.693+0000 7f60692b5700 1 -- 172.21.15.27:0/140350401 >> 172.21.15.27:0/140350401 conn(0x7f60640f6110 msgr2=0x7f60640f8530 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:29.697 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.693+0000 7f60692b5700 1 -- 172.21.15.27:0/140350401 shutdown_connections 2021-11-03T17:06:29.697 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.693+0000 7f60692b5700 1 -- 172.21.15.27:0/140350401 wait complete. 2021-11-03T17:06:29.697 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.694+0000 7f60692b5700 1 Processor -- start 2021-11-03T17:06:29.697 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.694+0000 7f60692b5700 1 -- start start 2021-11-03T17:06:29.698 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.694+0000 7f60692b5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f606406fb60 0x7f60640fc110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:29.698 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.694+0000 7f60692b5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60640fa590 con 0x7f606406fb60 2021-11-03T17:06:29.698 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.694+0000 7f6063fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f606406fb60 0x7f60640fc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:29.698 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.694+0000 7f6063fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f606406fb60 0x7f60640fc110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54134/0 (socket says 172.21.15.27:54134) 2021-11-03T17:06:29.699 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.694+0000 7f6063fff700 1 -- 172.21.15.27:0/2506884824 learned_addr learned my addr 172.21.15.27:0/2506884824 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:29.699 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.694+0000 7f6063fff700 1 -- 172.21.15.27:0/2506884824 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f605400b200 con 0x7f606406fb60 2021-11-03T17:06:29.699 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.694+0000 7f6063fff700 1 --2- 172.21.15.27:0/2506884824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f606406fb60 0x7f60640fc110 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f6054006800 tx=0x7f6054006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:29.699 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.695+0000 7f6060ff9700 1 -- 172.21.15.27:0/2506884824 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f605400dec0 con 0x7f606406fb60 2021-11-03T17:06:29.699 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.695+0000 7f60692b5700 1 -- 172.21.15.27:0/2506884824 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60640fa730 con 0x7f606406fb60 2021-11-03T17:06:29.700 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.695+0000 7f6060ff9700 1 -- 172.21.15.27:0/2506884824 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f605401b8a0 con 0x7f606406fb60 2021-11-03T17:06:29.700 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.695+0000 7f6060ff9700 1 -- 172.21.15.27:0/2506884824 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f605401a7f0 con 0x7f606406fb60 2021-11-03T17:06:29.700 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.695+0000 7f60692b5700 1 -- 172.21.15.27:0/2506884824 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60640fabc0 con 0x7f606406fb60 2021-11-03T17:06:29.700 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.696+0000 7f6060ff9700 1 -- 172.21.15.27:0/2506884824 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 9) v1 ==== 36134+0+0 (secure 0 0 0) 0x7f605402b070 con 0x7f606406fb60 2021-11-03T17:06:29.701 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.696+0000 7f6060ff9700 1 --2- 172.21.15.27:0/2506884824 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f604402fe00 0x7f60440322c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:29.701 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.696+0000 7f6060ff9700 1 -- 172.21.15.27:0/2506884824 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 893+0+0 (secure 0 0 0) 0x7f60540415b0 con 0x7f606406fb60 2021-11-03T17:06:29.701 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.696+0000 7f60637fe700 1 -- 172.21.15.27:0/2506884824 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f604402fe00 msgr2=0x7f60440322c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/774407793 2021-11-03T17:06:29.701 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.696+0000 7f60637fe700 1 --2- 172.21.15.27:0/2506884824 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f604402fe00 0x7f60440322c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2021-11-03T17:06:29.701 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.696+0000 7f60692b5700 1 -- 172.21.15.27:0/2506884824 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6064059100 con 0x7f606406fb60 2021-11-03T17:06:29.702 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.700+0000 7f6060ff9700 1 -- 172.21.15.27:0/2506884824 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f605401f020 con 0x7f606406fb60 2021-11-03T17:06:29.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:29 smithi027 conmon[32322]: audit 2021-11-03T17:06:28.460344+0000 mon.smithi027 (mon.0) 63 : audit [INF] 2021-11-03T17:06:29.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:29 smithi027 conmon[32322]: from='client.? 172.21.15.27:0/3119440655' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2021-11-03T17:06:29.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:29 smithi027 conmon[32322]: cluster 2021-11-03T17:06:28.460409+0000 mon.smithi027 (mon.0) 64 : cluster [DBG] mgrmap e9: smithi027.ufhhvb(active, since 18s) 2021-11-03T17:06:29.892 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.897+0000 7f60637fe700 1 -- 172.21.15.27:0/2506884824 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f604402fe00 msgr2=0x7f60440322c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/774407793 2021-11-03T17:06:29.892 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:29.897+0000 7f60637fe700 1 --2- 172.21.15.27:0/2506884824 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f604402fe00 0x7f60440322c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2021-11-03T17:06:30.066 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:30.070+0000 7f60692b5700 1 -- 172.21.15.27:0/2506884824 --> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7f60640ff9f0 con 0x7f604402fe00 2021-11-03T17:06:30.293 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:30.298+0000 7f60637fe700 1 -- 172.21.15.27:0/2506884824 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f604402fe00 msgr2=0x7f60440322c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/774407793 2021-11-03T17:06:30.293 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:30.298+0000 7f60637fe700 1 --2- 172.21.15.27:0/2506884824 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f604402fe00 0x7f60440322c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2021-11-03T17:06:31.094 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:31.099+0000 7f60637fe700 1 -- 172.21.15.27:0/2506884824 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f604402fe00 msgr2=0x7f60440322c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/774407793 2021-11-03T17:06:31.094 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:31.099+0000 7f60637fe700 1 --2- 172.21.15.27:0/2506884824 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f604402fe00 0x7f60440322c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2021-11-03T17:06:32.696 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:32.700+0000 7f60637fe700 1 -- 172.21.15.27:0/2506884824 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f604402fe00 msgr2=0x7f60440322c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/774407793 2021-11-03T17:06:32.696 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:32.700+0000 7f60637fe700 1 --2- 172.21.15.27:0/2506884824 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f604402fe00 0x7f60440322c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2021-11-03T17:06:34.459 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:34.462+0000 7f6060ff9700 1 -- 172.21.15.27:0/2506884824 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 10) v1 ==== 35984+0+0 (secure 0 0 0) 0x7f605400f070 con 0x7f606406fb60 2021-11-03T17:06:34.459 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:34.462+0000 7f6060ff9700 1 -- 172.21.15.27:0/2506884824 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f604402fe00 msgr2=0x7f60440322c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2021-11-03T17:06:34.459 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:34.462+0000 7f6060ff9700 1 --2- 172.21.15.27:0/2506884824 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f604402fe00 0x7f60440322c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:06:35.461 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:35.463+0000 7f6060ff9700 1 -- 172.21.15.27:0/2506884824 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mgrmap(e 11) v1 ==== 36054+0+0 (secure 0 0 0) 0x7f6054043360 con 0x7f606406fb60 2021-11-03T17:06:35.461 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:35.464+0000 7f6060ff9700 1 --2- 172.21.15.27:0/2506884824 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f6044033bb0 0x7f6044035fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:35.462 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:35.464+0000 7f6060ff9700 1 -- 172.21.15.27:0/2506884824 --> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7f60640ff9f0 con 0x7f6044033bb0 2021-11-03T17:06:35.462 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:35.464+0000 7f60637fe700 1 --2- 172.21.15.27:0/2506884824 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f6044033bb0 0x7f6044035fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:35.462 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:35.465+0000 7f60637fe700 1 --2- 172.21.15.27:0/2506884824 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f6044033bb0 0x7f6044035fa0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7f604c006fd0 tx=0x7f604c006d30).ready entity=mgr.14142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:35.465 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:35.469+0000 7f6060ff9700 1 -- 172.21.15.27:0/2506884824 <== mgr.14142 v2:172.21.15.27:6800/3506673123 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7f60640ff9f0 con 0x7f6044033bb0 2021-11-03T17:06:35.466 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: Scheduled prometheus update... 2021-11-03T17:06:35.469 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:35.472+0000 7f60692b5700 1 -- 172.21.15.27:0/2506884824 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f6044033bb0 msgr2=0x7f6044035fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:35.469 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:35.472+0000 7f60692b5700 1 --2- 172.21.15.27:0/2506884824 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f6044033bb0 0x7f6044035fa0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 rx=0x7f604c006fd0 tx=0x7f604c006d30).stop 2021-11-03T17:06:35.469 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:35.472+0000 7f60692b5700 1 -- 172.21.15.27:0/2506884824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f606406fb60 msgr2=0x7f60640fc110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:35.469 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:35.472+0000 7f60692b5700 1 --2- 172.21.15.27:0/2506884824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f606406fb60 0x7f60640fc110 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f6054006800 tx=0x7f6054006ee0).stop 2021-11-03T17:06:35.470 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:35.472+0000 7f60692b5700 1 -- 172.21.15.27:0/2506884824 shutdown_connections 2021-11-03T17:06:35.470 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:35.472+0000 7f60692b5700 1 --2- 172.21.15.27:0/2506884824 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f6044033bb0 0x7f6044035fa0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:35.470 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:35.472+0000 7f60692b5700 1 --2- 172.21.15.27:0/2506884824 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f606406fb60 0x7f60640fc110 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:35.470 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:35.472+0000 7f60692b5700 1 --2- 172.21.15.27:0/2506884824 >> [v2:172.21.15.27:6800/774407793,v1:172.21.15.27:6801/774407793] conn(0x7f604402fe00 0x7f60440322c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:06:35.471 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:35.472+0000 7f60692b5700 1 -- 172.21.15.27:0/2506884824 >> 172.21.15.27:0/2506884824 conn(0x7f60640f6110 msgr2=0x7f60640f7b60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:35.471 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:35.472+0000 7f60692b5700 1 -- 172.21.15.27:0/2506884824 shutdown_connections 2021-11-03T17:06:35.471 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:35.472+0000 7f60692b5700 1 -- 172.21.15.27:0/2506884824 wait complete. 2021-11-03T17:06:35.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:35 smithi027 conmon[32322]: cluster 2021-11-03T17:06:34.410301 2021-11-03T17:06:35.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:35 smithi027 conmon[32322]: +0000 mon.smithi027 (mon.0) 65 : cluster [INF] Active manager daemon smithi027.ufhhvb restarted 2021-11-03T17:06:35.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:35 smithi027 conmon[32322]: cluster 2021-11-03T17:06:34.410741+0000 mon.smithi027 (mon.0) 66 : cluster [INF] Activating manager daemon smithi027.ufhhvb 2021-11-03T17:06:35.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:35 smithi027 conmon[32322]: cluster 2021-11-03T17:06:34.412927+0000 mon.smithi027 (mon.0) 67 : cluster [DBG] osdmap e3: 0 total, 0 up, 0 in 2021-11-03T17:06:35.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:35 smithi027 conmon[32322]: cluster 2021-11-03T17:06:34.464171+0000 mon.smithi027 (mon.0) 68 : cluster [DBG] mgrmap e10: smithi027.ufhhvb(active, starting, since 0.0535348s) 2021-11-03T17:06:35.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:35 smithi027 conmon[32322]: audit 2021-11-03T17:06:34.465401+0000 mon.smithi027 (mon.0) 69 : audit [DBG] from='mgr.14142 172.21.15.27:0/2657360545' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2021-11-03T17:06:35.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:35 smithi027 conmon[32322]: audit 2021-11-03T17:06:34.465987+0000 mon.smithi027 (mon.0) 70 : audit [DBG] from='mgr.14142 172.21.15.27:0/2657360545' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mgr metadata", "who": "smithi027.ufhhvb", "id": "smithi027.ufhhvb"}]: dispatch 2021-11-03T17:06:35.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:35 smithi027 conmon[32322]: audit 2021-11-03T17:06:34.466850+0000 mon.smithi027 (mon.0) 71 : audit [DBG] from='mgr.14142 172.21.15.27:0/2657360545' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata"}]: dispatch 2021-11-03T17:06:35.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:35 smithi027 conmon[32322]: audit 2021-11-03T17:06:34.467028+0000 mon.smithi027 (mon.0) 72 : audit [DBG] from='mgr.14142 172.21.15.27:0/2657360545' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata"}]: dispatch 2021-11-03T17:06:35.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:35 smithi027 conmon[32322]: audit 2021-11-03T17:06:34.467160+0000 mon.smithi027 2021-11-03T17:06:35.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:35 smithi027 conmon[32322]: (mon.0) 73 : audit [DBG] from='mgr.14142 172.21.15.27:0/2657360545' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata"}]: dispatch 2021-11-03T17:06:35.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:35 smithi027 conmon[32322]: cluster 2021-11-03T17:06:34.469360+0000 mon.smithi027 (mon.0) 74 : cluster [INF] Manager daemon smithi027.ufhhvb is now available 2021-11-03T17:06:35.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:35 smithi027 conmon[32322]: audit 2021-11-03T17:06:34.601712+0000 2021-11-03T17:06:35.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:35 smithi027 conmon[32322]: mon.smithi027 (mon.0) 75 : audit [DBG] from='mgr.14142 172.21.15.27:0/2657360545' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:06:35.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:35 smithi027 conmon[32322]: audit 2021-11-03T17:06:34 2021-11-03T17:06:35.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:35 smithi027 conmon[32322]: .603555+0000 mon.smithi027 (mon.0) 76 : audit [DBG] from='mgr.14142 172.21.15.27:0/2657360545' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:06:35.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:35 smithi027 conmon[32322]: audit 2021-11-03 2021-11-03T17:06:35.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:35 smithi027 conmon[32322]: T17:06:34.620699+0000 mon.smithi027 (mon.0) 77 2021-11-03T17:06:35.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:35 smithi027 conmon[32322]: : audit [INF] from='mgr.14142 172.21.15.27:0/2657360545' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:06:35.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:35 smithi027 conmon[32322]: audit 2021- 2021-11-03T17:06:35.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:35 smithi027 conmon[32322]: 11-03T17:06:34.623967+0000 mon.smithi027 ( 2021-11-03T17:06:35.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:35 smithi027 conmon[32322]: mon.0) 78 : audit [INF] from='mgr.14142 172.21.15.27:0/2657360545' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:06:36.225 INFO:teuthology.orchestra.run.smithi027.stderr:Deploying grafana service with default placement... 2021-11-03T17:06:36.226 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi027 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmp3f8lj10x:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpq2rb3_h7:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 orch apply grafana 2021-11-03T17:06:36.495 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:36 smithi027 conmon[32322]: cluster 2021-11-03T17:06:35.467498+0000 mon.smithi027 (mon.0) 79 : cluster [DBG] mgrmap e11: smithi027.ufhhvb(active, since 1.05686s) 2021-11-03T17:06:36.495 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:36 smithi027 conmon[32322]: cephadm 2021-11-03T17:06:35.468677+0000 mgr.smithi027.ufhhvb (mgr.14142) 2 : cephadm [INF] Saving service prometheus spec with placement count:1 2021-11-03T17:06:36.495 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:36 smithi027 conmon[32322]: audit 2021-11-03T17:06:35.470796+0000 mon.smithi027 (mon.0) 80 : audit [INF] from='mgr.14142 172.21.15.27:0/2657360545' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:06:36.495 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:36 smithi027 conmon[32322]: audit 2021-11-03T17:06:36.197830+0000 mon.smithi027 (mon.0) 81 : audit [INF] from='mgr.14142 172.21.15.27:0/2657360545' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:06:36.923 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.923+0000 7ff1bed63700 1 Processor -- start 2021-11-03T17:06:36.923 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.923+0000 7ff1bed63700 1 -- start start 2021-11-03T17:06:36.924 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.923+0000 7ff1bed63700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1b8071c20 0x7ff1b80701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:36.924 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.923+0000 7ff1bed63700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b8072040 con 0x7ff1b8071c20 2021-11-03T17:06:36.924 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.924+0000 7ff1bdd61700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1b8071c20 0x7ff1b80701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:36.924 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.924+0000 7ff1bdd61700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1b8071c20 0x7ff1b80701c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54178/0 (socket says 172.21.15.27:54178) 2021-11-03T17:06:36.925 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.924+0000 7ff1bdd61700 1 -- 172.21.15.27:0/2832944765 learned_addr learned my addr 172.21.15.27:0/2832944765 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:36.925 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.924+0000 7ff1bdd61700 1 -- 172.21.15.27:0/2832944765 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b80706b0 con 0x7ff1b8071c20 2021-11-03T17:06:36.925 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.924+0000 7ff1bdd61700 1 --2- 172.21.15.27:0/2832944765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1b8071c20 0x7ff1b80701c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7ff1ac00b5f0 tx=0x7ff1ac006ee0).ready entity=mon.0 client_cookie=eec3a63922a0fe86 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:36.925 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.924+0000 7ff1bcd5f700 1 -- 172.21.15.27:0/2832944765 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff1ac00d600 con 0x7ff1b8071c20 2021-11-03T17:06:36.925 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.924+0000 7ff1bcd5f700 1 -- 172.21.15.27:0/2832944765 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7ff1ac00dbf0 con 0x7ff1b8071c20 2021-11-03T17:06:36.926 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.924+0000 7ff1bcd5f700 1 -- 172.21.15.27:0/2832944765 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff1ac011bf0 con 0x7ff1b8071c20 2021-11-03T17:06:36.926 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.925+0000 7ff1bed63700 1 -- 172.21.15.27:0/2832944765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1b8071c20 msgr2=0x7ff1b80701c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:36.926 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.925+0000 7ff1bed63700 1 --2- 172.21.15.27:0/2832944765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1b8071c20 0x7ff1b80701c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7ff1ac00b5f0 tx=0x7ff1ac006ee0).stop 2021-11-03T17:06:36.926 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.925+0000 7ff1bed63700 1 -- 172.21.15.27:0/2832944765 shutdown_connections 2021-11-03T17:06:36.927 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.925+0000 7ff1bed63700 1 --2- 172.21.15.27:0/2832944765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1b8071c20 0x7ff1b80701c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:36.927 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.925+0000 7ff1bed63700 1 -- 172.21.15.27:0/2832944765 >> 172.21.15.27:0/2832944765 conn(0x7ff1b80f7270 msgr2=0x7ff1b80f9690 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:36.927 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.925+0000 7ff1bed63700 1 -- 172.21.15.27:0/2832944765 shutdown_connections 2021-11-03T17:06:36.927 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.925+0000 7ff1bed63700 1 -- 172.21.15.27:0/2832944765 wait complete. 2021-11-03T17:06:36.928 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.926+0000 7ff1bed63700 1 Processor -- start 2021-11-03T17:06:36.928 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.926+0000 7ff1bed63700 1 -- start start 2021-11-03T17:06:36.928 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.926+0000 7ff1bed63700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1b8071c20 0x7ff1b810ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:36.928 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.926+0000 7ff1bed63700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b810cef0 con 0x7ff1b8071c20 2021-11-03T17:06:36.929 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.926+0000 7ff1bdd61700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1b8071c20 0x7ff1b810ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:36.929 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.926+0000 7ff1bdd61700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1b8071c20 0x7ff1b810ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54180/0 (socket says 172.21.15.27:54180) 2021-11-03T17:06:36.929 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.926+0000 7ff1bdd61700 1 -- 172.21.15.27:0/1852515106 learned_addr learned my addr 172.21.15.27:0/1852515106 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:36.929 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.926+0000 7ff1bdd61700 1 -- 172.21.15.27:0/1852515106 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ac00b200 con 0x7ff1b8071c20 2021-11-03T17:06:36.929 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.926+0000 7ff1bdd61700 1 --2- 172.21.15.27:0/1852515106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1b8071c20 0x7ff1b810ca00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7ff1ac00c010 tx=0x7ff1ac006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:36.930 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.927+0000 7ff1aa7fc700 1 -- 172.21.15.27:0/1852515106 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff1ac00dec0 con 0x7ff1b8071c20 2021-11-03T17:06:36.930 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.927+0000 7ff1aa7fc700 1 -- 172.21.15.27:0/1852515106 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7ff1ac01b8a0 con 0x7ff1b8071c20 2021-11-03T17:06:36.930 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.927+0000 7ff1bed63700 1 -- 172.21.15.27:0/1852515106 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1b810d0f0 con 0x7ff1b8071c20 2021-11-03T17:06:36.931 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.927+0000 7ff1aa7fc700 1 -- 172.21.15.27:0/1852515106 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff1ac01a730 con 0x7ff1b8071c20 2021-11-03T17:06:36.931 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.927+0000 7ff1bed63700 1 -- 172.21.15.27:0/1852515106 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1b810d580 con 0x7ff1b8071c20 2021-11-03T17:06:36.932 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.928+0000 7ff1aa7fc700 1 -- 172.21.15.27:0/1852515106 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 11) v1 ==== 36054+0+0 (secure 0 0 0) 0x7ff1ac02b070 con 0x7ff1b8071c20 2021-11-03T17:06:36.933 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.928+0000 7ff1aa7fc700 1 --2- 172.21.15.27:0/1852515106 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7ff1a002fd70 0x7ff1a0032230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:36.933 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.928+0000 7ff1aa7fc700 1 -- 172.21.15.27:0/1852515106 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (secure 0 0 0) 0x7ff1ac0413c0 con 0x7ff1b8071c20 2021-11-03T17:06:36.933 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.928+0000 7ff1bd560700 1 --2- 172.21.15.27:0/1852515106 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7ff1a002fd70 0x7ff1a0032230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:36.933 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.928+0000 7ff1bd560700 1 --2- 172.21.15.27:0/1852515106 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7ff1a002fd70 0x7ff1a0032230 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7ff1b40096a0 tx=0x7ff1b4006b40).ready entity=mgr.14142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:36.934 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.929+0000 7ff1bed63700 1 -- 172.21.15.27:0/1852515106 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1b8045ad0 con 0x7ff1b8071c20 2021-11-03T17:06:36.934 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:36.933+0000 7ff1aa7fc700 1 -- 172.21.15.27:0/1852515106 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7ff1ac01f080 con 0x7ff1b8071c20 2021-11-03T17:06:37.193 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:37.197+0000 7ff1aa7fc700 1 -- 172.21.15.27:0/1852515106 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 12) v1 ==== 36195+0+0 (secure 0 0 0) 0x7ff1ac00f070 con 0x7ff1b8071c20 2021-11-03T17:06:37.279 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:37.282+0000 7ff1bed63700 1 -- 172.21.15.27:0/1852515106 --> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1b81069d0 con 0x7ff1a002fd70 2021-11-03T17:06:37.285 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:37.289+0000 7ff1aa7fc700 1 -- 172.21.15.27:0/1852515106 <== mgr.14142 v2:172.21.15.27:6800/3506673123 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7ff1b81069d0 con 0x7ff1a002fd70 2021-11-03T17:06:37.285 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: Scheduled grafana update... 2021-11-03T17:06:37.288 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:37.291+0000 7ff1bed63700 1 -- 172.21.15.27:0/1852515106 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7ff1a002fd70 msgr2=0x7ff1a0032230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:37.288 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:37.291+0000 7ff1bed63700 1 --2- 172.21.15.27:0/1852515106 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7ff1a002fd70 0x7ff1a0032230 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7ff1b40096a0 tx=0x7ff1b4006b40).stop 2021-11-03T17:06:37.289 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:37.291+0000 7ff1bed63700 1 -- 172.21.15.27:0/1852515106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1b8071c20 msgr2=0x7ff1b810ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:37.289 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:37.291+0000 7ff1bed63700 1 --2- 172.21.15.27:0/1852515106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1b8071c20 0x7ff1b810ca00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7ff1ac00c010 tx=0x7ff1ac006ee0).stop 2021-11-03T17:06:37.289 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:37.291+0000 7ff1bed63700 1 -- 172.21.15.27:0/1852515106 shutdown_connections 2021-11-03T17:06:37.289 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:37.291+0000 7ff1bed63700 1 --2- 172.21.15.27:0/1852515106 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7ff1a002fd70 0x7ff1a0032230 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:37.290 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:37.291+0000 7ff1bed63700 1 --2- 172.21.15.27:0/1852515106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1b8071c20 0x7ff1b810ca00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:37.290 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:37.291+0000 7ff1bed63700 1 -- 172.21.15.27:0/1852515106 >> 172.21.15.27:0/1852515106 conn(0x7ff1b80f7270 msgr2=0x7ff1b80f8d70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:37.290 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:37.291+0000 7ff1bed63700 1 -- 172.21.15.27:0/1852515106 shutdown_connections 2021-11-03T17:06:37.290 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:37.291+0000 7ff1bed63700 1 -- 172.21.15.27:0/1852515106 wait complete. 2021-11-03T17:06:38.080 INFO:teuthology.orchestra.run.smithi027.stderr:Deploying node-exporter service with default placement... 2021-11-03T17:06:38.080 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi027 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmp3f8lj10x:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpq2rb3_h7:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 orch apply node-exporter 2021-11-03T17:06:38.337 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:38 smithi027 conmon[32322]: cluster 2021-11-03T17:06:37.200707+0000 mon.smithi027 (mon.0) 82 : cluster [DBG] mgrmap e12: smithi027.ufhhvb(active, since 2s) 2021-11-03T17:06:38.338 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:38 smithi027 conmon[32322]: audit 2021-11-03T17:06:37.285268+0000 mgr.smithi027.ufhhvb (mgr.14142) 3 : audit [DBG] from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:06:38.338 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:38 smithi027 conmon[32322]: cephadm 2021-11-03T17:06:37.285855+0000 mgr.smithi027.ufhhvb (mgr.14142) 4 : cephadm [INF] Saving service grafana spec with placement count:1 2021-11-03T17:06:38.338 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:38 smithi027 conmon[32322]: audit 2021-11-03T17:06:37.290770+0000 mon.smithi027 (mon.0) 83 : audit [INF] from='mgr.14142 172.21.15.27:0/2657360545' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:06:38.730 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.730+0000 7f93f6fad700 1 Processor -- start 2021-11-03T17:06:38.731 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.730+0000 7f93f6fad700 1 -- start start 2021-11-03T17:06:38.731 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.730+0000 7f93f6fad700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93f006fbc0 0x7f93f00fe7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:38.731 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.730+0000 7f93f6fad700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f00fecd0 con 0x7f93f006fbc0 2021-11-03T17:06:38.732 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.731+0000 7f93f4d49700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93f006fbc0 0x7f93f00fe7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:38.732 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.731+0000 7f93f4d49700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93f006fbc0 0x7f93f00fe7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54184/0 (socket says 172.21.15.27:54184) 2021-11-03T17:06:38.732 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.731+0000 7f93f4d49700 1 -- 172.21.15.27:0/4017076359 learned_addr learned my addr 172.21.15.27:0/4017076359 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:38.732 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.731+0000 7f93f4d49700 1 -- 172.21.15.27:0/4017076359 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93f00fee10 con 0x7f93f006fbc0 2021-11-03T17:06:38.732 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.731+0000 7f93f4d49700 1 --2- 172.21.15.27:0/4017076359 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93f006fbc0 0x7f93f00fe7e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f93e8009960 tx=0x7f93e80092d0).ready entity=mon.0 client_cookie=b2876a9ce6b5ecb3 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:38.733 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.731+0000 7f93ef7fe700 1 -- 172.21.15.27:0/4017076359 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f93e800b9c0 con 0x7f93f006fbc0 2021-11-03T17:06:38.733 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.731+0000 7f93ef7fe700 1 -- 172.21.15.27:0/4017076359 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f93e8011400 con 0x7f93f006fbc0 2021-11-03T17:06:38.733 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.731+0000 7f93ef7fe700 1 -- 172.21.15.27:0/4017076359 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f93e8010580 con 0x7f93f006fbc0 2021-11-03T17:06:38.733 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.732+0000 7f93f6fad700 1 -- 172.21.15.27:0/4017076359 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93f006fbc0 msgr2=0x7f93f00fe7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:38.734 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.732+0000 7f93f6fad700 1 --2- 172.21.15.27:0/4017076359 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93f006fbc0 0x7f93f00fe7e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f93e8009960 tx=0x7f93e80092d0).stop 2021-11-03T17:06:38.734 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.732+0000 7f93f6fad700 1 -- 172.21.15.27:0/4017076359 shutdown_connections 2021-11-03T17:06:38.734 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.732+0000 7f93f6fad700 1 --2- 172.21.15.27:0/4017076359 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93f006fbc0 0x7f93f00fe7e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:38.735 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.732+0000 7f93f6fad700 1 -- 172.21.15.27:0/4017076359 >> 172.21.15.27:0/4017076359 conn(0x7f93f00f6260 msgr2=0x7f93f00f86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:38.735 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.732+0000 7f93f6fad700 1 -- 172.21.15.27:0/4017076359 shutdown_connections 2021-11-03T17:06:38.735 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.732+0000 7f93f6fad700 1 -- 172.21.15.27:0/4017076359 wait complete. 2021-11-03T17:06:38.735 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.733+0000 7f93f6fad700 1 Processor -- start 2021-11-03T17:06:38.736 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.733+0000 7f93f6fad700 1 -- start start 2021-11-03T17:06:38.736 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.733+0000 7f93f6fad700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93f006fbc0 0x7f93f00fc260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:38.736 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.733+0000 7f93f6fad700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93f00fa6e0 con 0x7f93f006fbc0 2021-11-03T17:06:38.736 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.733+0000 7f93f4d49700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93f006fbc0 0x7f93f00fc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:38.737 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.733+0000 7f93f4d49700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93f006fbc0 0x7f93f00fc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54186/0 (socket says 172.21.15.27:54186) 2021-11-03T17:06:38.737 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.733+0000 7f93f4d49700 1 -- 172.21.15.27:0/3080695890 learned_addr learned my addr 172.21.15.27:0/3080695890 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:38.737 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.733+0000 7f93f4d49700 1 -- 172.21.15.27:0/3080695890 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e8009570 con 0x7f93f006fbc0 2021-11-03T17:06:38.737 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.733+0000 7f93f4d49700 1 --2- 172.21.15.27:0/3080695890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93f006fbc0 0x7f93f00fc260 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f93e8006f00 tx=0x7f93e80092d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:38.738 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.734+0000 7f93ed7fa700 1 -- 172.21.15.27:0/3080695890 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f93e8010840 con 0x7f93f006fbc0 2021-11-03T17:06:38.738 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.734+0000 7f93f6fad700 1 -- 172.21.15.27:0/3080695890 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93f00fa880 con 0x7f93f006fbc0 2021-11-03T17:06:38.738 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.734+0000 7f93ed7fa700 1 -- 172.21.15.27:0/3080695890 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f93e8010e30 con 0x7f93f006fbc0 2021-11-03T17:06:38.739 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.734+0000 7f93f6fad700 1 -- 172.21.15.27:0/3080695890 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93f00fad10 con 0x7f93f006fbc0 2021-11-03T17:06:38.739 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.734+0000 7f93ed7fa700 1 -- 172.21.15.27:0/3080695890 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f93e801bda0 con 0x7f93f006fbc0 2021-11-03T17:06:38.740 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.735+0000 7f93ed7fa700 1 -- 172.21.15.27:0/3080695890 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 12) v1 ==== 36195+0+0 (secure 0 0 0) 0x7f93e800f070 con 0x7f93f006fbc0 2021-11-03T17:06:38.741 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.735+0000 7f93ed7fa700 1 --2- 172.21.15.27:0/3080695890 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f93d002feb0 0x7f93d0032370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:38.741 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.735+0000 7f93ed7fa700 1 -- 172.21.15.27:0/3080695890 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f93e8041ba0 con 0x7f93f006fbc0 2021-11-03T17:06:38.742 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.735+0000 7f93f6fad700 1 -- 172.21.15.27:0/3080695890 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93f0045ad0 con 0x7f93f006fbc0 2021-11-03T17:06:38.742 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.735+0000 7f93effff700 1 --2- 172.21.15.27:0/3080695890 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f93d002feb0 0x7f93d0032370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:38.742 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.735+0000 7f93effff700 1 --2- 172.21.15.27:0/3080695890 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f93d002feb0 0x7f93d0032370 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f93e00096a0 tx=0x7f93e0006b40).ready entity=mgr.14142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:38.742 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:38.739+0000 7f93ed7fa700 1 -- 172.21.15.27:0/3080695890 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f93e8024400 con 0x7f93f006fbc0 2021-11-03T17:06:39.075 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:39.079+0000 7f93f6fad700 1 -- 172.21.15.27:0/3080695890 --> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f93f00ffb40 con 0x7f93d002feb0 2021-11-03T17:06:39.079 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:39.083+0000 7f93ed7fa700 1 -- 172.21.15.27:0/3080695890 <== mgr.14142 v2:172.21.15.27:6800/3506673123 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f93f00ffb40 con 0x7f93d002feb0 2021-11-03T17:06:39.079 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: Scheduled node-exporter update... 2021-11-03T17:06:39.082 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:39.085+0000 7f93f6fad700 1 -- 172.21.15.27:0/3080695890 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f93d002feb0 msgr2=0x7f93d0032370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:39.082 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:39.085+0000 7f93f6fad700 1 --2- 172.21.15.27:0/3080695890 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f93d002feb0 0x7f93d0032370 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f93e00096a0 tx=0x7f93e0006b40).stop 2021-11-03T17:06:39.082 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:39.085+0000 7f93f6fad700 1 -- 172.21.15.27:0/3080695890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93f006fbc0 msgr2=0x7f93f00fc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:39.083 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:39.085+0000 7f93f6fad700 1 --2- 172.21.15.27:0/3080695890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93f006fbc0 0x7f93f00fc260 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f93e8006f00 tx=0x7f93e80092d0).stop 2021-11-03T17:06:39.083 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:39.085+0000 7f93f6fad700 1 -- 172.21.15.27:0/3080695890 shutdown_connections 2021-11-03T17:06:39.083 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:39.085+0000 7f93f6fad700 1 --2- 172.21.15.27:0/3080695890 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f93d002feb0 0x7f93d0032370 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:39.083 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:39.085+0000 7f93f6fad700 1 --2- 172.21.15.27:0/3080695890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93f006fbc0 0x7f93f00fc260 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:39.083 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:39.085+0000 7f93f6fad700 1 -- 172.21.15.27:0/3080695890 >> 172.21.15.27:0/3080695890 conn(0x7f93f00f6260 msgr2=0x7f93f00f7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:39.084 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:39.085+0000 7f93f6fad700 1 -- 172.21.15.27:0/3080695890 shutdown_connections 2021-11-03T17:06:39.084 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:39.085+0000 7f93f6fad700 1 -- 172.21.15.27:0/3080695890 wait complete. 2021-11-03T17:06:39.617 INFO:teuthology.orchestra.run.smithi027.stderr:Deploying alertmanager service with default placement... 2021-11-03T17:06:39.617 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi027 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmp3f8lj10x:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpq2rb3_h7:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 orch apply alertmanager 2021-11-03T17:06:40.252 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.246+0000 7f5767ef0700 1 Processor -- start 2021-11-03T17:06:40.252 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.246+0000 7f5767ef0700 1 -- start start 2021-11-03T17:06:40.253 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.246+0000 7f5767ef0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57600fd240 0x7f57600fd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:40.253 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.246+0000 7f5767ef0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57600fdb50 con 0x7f57600fd240 2021-11-03T17:06:40.254 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.246+0000 7f5765c8c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57600fd240 0x7f57600fd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:40.254 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.246+0000 7f5765c8c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57600fd240 0x7f57600fd660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54190/0 (socket says 172.21.15.27:54190) 2021-11-03T17:06:40.254 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.246+0000 7f5765c8c700 1 -- 172.21.15.27:0/4227635446 learned_addr learned my addr 172.21.15.27:0/4227635446 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:40.255 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.246+0000 7f5765c8c700 1 -- 172.21.15.27:0/4227635446 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57600fdc90 con 0x7f57600fd240 2021-11-03T17:06:40.255 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.247+0000 7f5765c8c700 1 --2- 172.21.15.27:0/4227635446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57600fd240 0x7f57600fd660 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f575c00b5f0 tx=0x7f575c006ee0).ready entity=mon.0 client_cookie=612469e136b669ed server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:40.255 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.247+0000 7f5764c8a700 1 -- 172.21.15.27:0/4227635446 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f575c00d600 con 0x7f57600fd240 2021-11-03T17:06:40.255 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.247+0000 7f5764c8a700 1 -- 172.21.15.27:0/4227635446 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f575c00dbf0 con 0x7f57600fd240 2021-11-03T17:06:40.255 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.247+0000 7f5764c8a700 1 -- 172.21.15.27:0/4227635446 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f575c011bf0 con 0x7f57600fd240 2021-11-03T17:06:40.256 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.247+0000 7f5767ef0700 1 -- 172.21.15.27:0/4227635446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57600fd240 msgr2=0x7f57600fd660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:40.256 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.247+0000 7f5767ef0700 1 --2- 172.21.15.27:0/4227635446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57600fd240 0x7f57600fd660 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f575c00b5f0 tx=0x7f575c006ee0).stop 2021-11-03T17:06:40.256 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.247+0000 7f5767ef0700 1 -- 172.21.15.27:0/4227635446 shutdown_connections 2021-11-03T17:06:40.256 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.247+0000 7f5767ef0700 1 --2- 172.21.15.27:0/4227635446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57600fd240 0x7f57600fd660 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:40.257 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.247+0000 7f5767ef0700 1 -- 172.21.15.27:0/4227635446 >> 172.21.15.27:0/4227635446 conn(0x7f57600f5a60 msgr2=0x7f57600f7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:40.257 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.248+0000 7f5767ef0700 1 -- 172.21.15.27:0/4227635446 shutdown_connections 2021-11-03T17:06:40.257 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.248+0000 7f5767ef0700 1 -- 172.21.15.27:0/4227635446 wait complete. 2021-11-03T17:06:40.257 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.248+0000 7f5767ef0700 1 Processor -- start 2021-11-03T17:06:40.258 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.248+0000 7f5767ef0700 1 -- start start 2021-11-03T17:06:40.258 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.248+0000 7f5767ef0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57600fd240 0x7f576010a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:40.258 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.248+0000 7f5767ef0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576010ae20 con 0x7f57600fd240 2021-11-03T17:06:40.258 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.248+0000 7f5765c8c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57600fd240 0x7f576010a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:40.258 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.248+0000 7f5765c8c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57600fd240 0x7f576010a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54192/0 (socket says 172.21.15.27:54192) 2021-11-03T17:06:40.259 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.248+0000 7f5765c8c700 1 -- 172.21.15.27:0/543593440 learned_addr learned my addr 172.21.15.27:0/543593440 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:40.259 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.249+0000 7f5765c8c700 1 -- 172.21.15.27:0/543593440 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f575c00b200 con 0x7f57600fd240 2021-11-03T17:06:40.259 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.249+0000 7f5765c8c700 1 --2- 172.21.15.27:0/543593440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57600fd240 0x7f576010a930 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f575c011660 tx=0x7f575c006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:40.259 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.249+0000 7f574e7fc700 1 -- 172.21.15.27:0/543593440 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f575c01b570 con 0x7f57600fd240 2021-11-03T17:06:40.260 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.249+0000 7f574e7fc700 1 -- 172.21.15.27:0/543593440 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f575c01bb60 con 0x7f57600fd240 2021-11-03T17:06:40.260 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.249+0000 7f5767ef0700 1 -- 172.21.15.27:0/543593440 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f576010b020 con 0x7f57600fd240 2021-11-03T17:06:40.260 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.249+0000 7f574e7fc700 1 -- 172.21.15.27:0/543593440 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f575c01a790 con 0x7f57600fd240 2021-11-03T17:06:40.260 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.249+0000 7f5767ef0700 1 -- 172.21.15.27:0/543593440 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f576010b4b0 con 0x7f57600fd240 2021-11-03T17:06:40.261 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.250+0000 7f574e7fc700 1 -- 172.21.15.27:0/543593440 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 12) v1 ==== 36195+0+0 (secure 0 0 0) 0x7f575c02b070 con 0x7f57600fd240 2021-11-03T17:06:40.261 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.250+0000 7f574e7fc700 1 --2- 172.21.15.27:0/543593440 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f574802fe60 0x7f5748032320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:40.261 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.250+0000 7f574e7fc700 1 -- 172.21.15.27:0/543593440 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f575c041660 con 0x7f57600fd240 2021-11-03T17:06:40.261 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.250+0000 7f5767ef0700 1 -- 172.21.15.27:0/543593440 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5760045ad0 con 0x7f57600fd240 2021-11-03T17:06:40.261 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.250+0000 7f576548b700 1 --2- 172.21.15.27:0/543593440 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f574802fe60 0x7f5748032320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:40.262 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.251+0000 7f576548b700 1 --2- 172.21.15.27:0/543593440 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f574802fe60 0x7f5748032320 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f57500096a0 tx=0x7f5750006b40).ready entity=mgr.14142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:40.262 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.255+0000 7f574e7fc700 1 -- 172.21.15.27:0/543593440 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f575c016070 con 0x7f57600fd240 2021-11-03T17:06:40.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:40 smithi027 conmon[32322]: audit 2021-11-03T17:06:39. 2021-11-03T17:06:40.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:40 smithi027 conmon[32322]: 081627+0000 mgr.smithi027.ufhhvb (mgr.14142) 5 : audit [DBG] from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:06:40.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:40 smithi027 conmon[32322]: cephadm 2021-11-03T17:06:39.082349+0000 mgr.smithi027.ufhhvb (mgr.14142) 6 : cephadm [INF] Saving service node-exporter spec with placement * 2021-11-03T17:06:40.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:40 smithi027 conmon[32322]: audit 2021-11-03T17:06:39.084725+0000 mon.smithi027 (mon.0) 84 : audit [INF] from='mgr.14142 172.21.15.27:0/2657360545' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:06:40.592 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.596+0000 7f5767ef0700 1 -- 172.21.15.27:0/543593440 --> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7f576010bdd0 con 0x7f574802fe60 2021-11-03T17:06:40.596 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.600+0000 7f574e7fc700 1 -- 172.21.15.27:0/543593440 <== mgr.14142 v2:172.21.15.27:6800/3506673123 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f576010bdd0 con 0x7f574802fe60 2021-11-03T17:06:40.597 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: Scheduled alertmanager update... 2021-11-03T17:06:40.599 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.602+0000 7f5767ef0700 1 -- 172.21.15.27:0/543593440 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f574802fe60 msgr2=0x7f5748032320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:40.600 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.602+0000 7f5767ef0700 1 --2- 172.21.15.27:0/543593440 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f574802fe60 0x7f5748032320 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f57500096a0 tx=0x7f5750006b40).stop 2021-11-03T17:06:40.600 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.602+0000 7f5767ef0700 1 -- 172.21.15.27:0/543593440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57600fd240 msgr2=0x7f576010a930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:40.600 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.602+0000 7f5767ef0700 1 --2- 172.21.15.27:0/543593440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57600fd240 0x7f576010a930 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f575c011660 tx=0x7f575c006ee0).stop 2021-11-03T17:06:40.601 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.602+0000 7f5767ef0700 1 -- 172.21.15.27:0/543593440 shutdown_connections 2021-11-03T17:06:40.601 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.602+0000 7f5767ef0700 1 --2- 172.21.15.27:0/543593440 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f574802fe60 0x7f5748032320 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:40.601 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.602+0000 7f5767ef0700 1 --2- 172.21.15.27:0/543593440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57600fd240 0x7f576010a930 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:40.602 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.602+0000 7f5767ef0700 1 -- 172.21.15.27:0/543593440 >> 172.21.15.27:0/543593440 conn(0x7f57600f5a60 msgr2=0x7f57600ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:40.602 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.603+0000 7f5767ef0700 1 -- 172.21.15.27:0/543593440 shutdown_connections 2021-11-03T17:06:40.602 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:40.603+0000 7f5767ef0700 1 -- 172.21.15.27:0/543593440 wait complete. 2021-11-03T17:06:41.096 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi027 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmp3f8lj10x:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpq2rb3_h7:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 config set mgr mgr/cephadm/container_init True --force 2021-11-03T17:06:41.749 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.749+0000 7f0e0267d700 1 Processor -- start 2021-11-03T17:06:41.750 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.749+0000 7f0e0267d700 1 -- start start 2021-11-03T17:06:41.750 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.749+0000 7f0e0267d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dfc0fc1f0 0x7f0dfc0fc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:41.750 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.749+0000 7f0e0267d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc0fcb00 con 0x7f0dfc0fc1f0 2021-11-03T17:06:41.750 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.749+0000 7f0dfbfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dfc0fc1f0 0x7f0dfc0fc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:41.751 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.749+0000 7f0dfbfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dfc0fc1f0 0x7f0dfc0fc610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54196/0 (socket says 172.21.15.27:54196) 2021-11-03T17:06:41.751 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.749+0000 7f0dfbfff700 1 -- 172.21.15.27:0/124494909 learned_addr learned my addr 172.21.15.27:0/124494909 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:41.751 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.750+0000 7f0dfbfff700 1 -- 172.21.15.27:0/124494909 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dfc0fcc40 con 0x7f0dfc0fc1f0 2021-11-03T17:06:41.751 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.750+0000 7f0dfbfff700 1 --2- 172.21.15.27:0/124494909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dfc0fc1f0 0x7f0dfc0fc610 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f0df000b5f0 tx=0x7f0df0006ee0).ready entity=mon.0 client_cookie=f25cf4171fca1e63 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:41.752 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.750+0000 7f0dfaffd700 1 -- 172.21.15.27:0/124494909 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0df000d600 con 0x7f0dfc0fc1f0 2021-11-03T17:06:41.752 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.750+0000 7f0dfaffd700 1 -- 172.21.15.27:0/124494909 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f0df000dbf0 con 0x7f0dfc0fc1f0 2021-11-03T17:06:41.752 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.750+0000 7f0dfaffd700 1 -- 172.21.15.27:0/124494909 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0df0011bf0 con 0x7f0dfc0fc1f0 2021-11-03T17:06:41.752 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.751+0000 7f0e0267d700 1 -- 172.21.15.27:0/124494909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dfc0fc1f0 msgr2=0x7f0dfc0fc610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:41.752 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.751+0000 7f0e0267d700 1 --2- 172.21.15.27:0/124494909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dfc0fc1f0 0x7f0dfc0fc610 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f0df000b5f0 tx=0x7f0df0006ee0).stop 2021-11-03T17:06:41.753 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.751+0000 7f0e0267d700 1 -- 172.21.15.27:0/124494909 shutdown_connections 2021-11-03T17:06:41.753 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.751+0000 7f0e0267d700 1 --2- 172.21.15.27:0/124494909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dfc0fc1f0 0x7f0dfc0fc610 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:41.753 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.751+0000 7f0e0267d700 1 -- 172.21.15.27:0/124494909 >> 172.21.15.27:0/124494909 conn(0x7f0dfc0f5a60 msgr2=0x7f0dfc0f7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:41.753 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.751+0000 7f0e0267d700 1 -- 172.21.15.27:0/124494909 shutdown_connections 2021-11-03T17:06:41.754 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.751+0000 7f0e0267d700 1 -- 172.21.15.27:0/124494909 wait complete. 2021-11-03T17:06:41.754 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.751+0000 7f0e0267d700 1 Processor -- start 2021-11-03T17:06:41.754 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.751+0000 7f0e0267d700 1 -- start start 2021-11-03T17:06:41.754 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.752+0000 7f0e0267d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dfc0fc1f0 0x7f0dfc10cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:41.754 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.752+0000 7f0e0267d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0dfc10d030 con 0x7f0dfc0fc1f0 2021-11-03T17:06:41.755 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.752+0000 7f0dfbfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dfc0fc1f0 0x7f0dfc10cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:41.755 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.752+0000 7f0dfbfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dfc0fc1f0 0x7f0dfc10cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54198/0 (socket says 172.21.15.27:54198) 2021-11-03T17:06:41.755 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.752+0000 7f0dfbfff700 1 -- 172.21.15.27:0/152430580 learned_addr learned my addr 172.21.15.27:0/152430580 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:41.755 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.752+0000 7f0dfbfff700 1 -- 172.21.15.27:0/152430580 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df000b200 con 0x7f0dfc0fc1f0 2021-11-03T17:06:41.756 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.752+0000 7f0dfbfff700 1 --2- 172.21.15.27:0/152430580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dfc0fc1f0 0x7f0dfc10cb40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f0df0011e30 tx=0x7f0df0006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:41.756 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.752+0000 7f0df8ff9700 1 -- 172.21.15.27:0/152430580 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0df001b410 con 0x7f0dfc0fc1f0 2021-11-03T17:06:41.756 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.752+0000 7f0df8ff9700 1 -- 172.21.15.27:0/152430580 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f0df001ba00 con 0x7f0dfc0fc1f0 2021-11-03T17:06:41.756 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.752+0000 7f0df8ff9700 1 -- 172.21.15.27:0/152430580 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0df001a750 con 0x7f0dfc0fc1f0 2021-11-03T17:06:41.756 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.752+0000 7f0e0267d700 1 -- 172.21.15.27:0/152430580 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dfc10d230 con 0x7f0dfc0fc1f0 2021-11-03T17:06:41.757 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.752+0000 7f0e0267d700 1 -- 172.21.15.27:0/152430580 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0dfc10d6c0 con 0x7f0dfc0fc1f0 2021-11-03T17:06:41.757 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.753+0000 7f0df8ff9700 1 -- 172.21.15.27:0/152430580 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 12) v1 ==== 36195+0+0 (secure 0 0 0) 0x7f0df002b070 con 0x7f0dfc0fc1f0 2021-11-03T17:06:41.757 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.753+0000 7f0df8ff9700 1 --2- 172.21.15.27:0/152430580 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f0de802feb0 0x7f0de8032370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:41.757 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.753+0000 7f0df8ff9700 1 -- 172.21.15.27:0/152430580 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f0df0041640 con 0x7f0dfc0fc1f0 2021-11-03T17:06:41.758 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.754+0000 7f0e0267d700 1 -- 172.21.15.27:0/152430580 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dfc045ad0 con 0x7f0dfc0fc1f0 2021-11-03T17:06:41.759 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.754+0000 7f0dfb7fe700 1 --2- 172.21.15.27:0/152430580 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f0de802feb0 0x7f0de8032370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:41.759 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.754+0000 7f0dfb7fe700 1 --2- 172.21.15.27:0/152430580 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f0de802feb0 0x7f0de8032370 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f0dec0096a0 tx=0x7f0dec006b40).ready entity=mgr.14142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:41.759 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:41.758+0000 7f0df8ff9700 1 -- 172.21.15.27:0/152430580 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f0df001f020 con 0x7f0dfc0fc1f0 2021-11-03T17:06:42.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:41 smithi027 conmon[32322]: audit 2021-11-03T17:06:40.599126 2021-11-03T17:06:42.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:41 smithi027 conmon[32322]: +0000 mgr.smithi027.ufhhvb (mgr.14142) 7 : audit [DBG] from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:06:42.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:41 smithi027 conmon[32322]: cephadm 2021-11-03T17:06:40.599703+0000 mgr.smithi027.ufhhvb (mgr.14142) 8 : cephadm [INF] Saving service alertmanager spec with placement count:1 2021-11-03T17:06:42.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:41 smithi027 conmon[32322]: audit 2021-11-03T17:06:40.601999+0000 mon.smithi027 (mon.0) 85 : audit [INF] from='mgr.14142 172.21.15.27:0/2657360545' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:06:42.038 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:42.041+0000 7f0e0267d700 1 -- 172.21.15.27:0/152430580 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f0dfc0008d0 con 0x7f0dfc0fc1f0 2021-11-03T17:06:42.040 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:42.044+0000 7f0df8ff9700 1 -- 172.21.15.27:0/152430580 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v9)=0 v9) v1 ==== 142+0+0 (secure 0 0 0) 0x7f0df0016070 con 0x7f0dfc0fc1f0 2021-11-03T17:06:42.043 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:42.046+0000 7f0e0267d700 1 -- 172.21.15.27:0/152430580 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f0de802feb0 msgr2=0x7f0de8032370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:42.043 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:42.046+0000 7f0e0267d700 1 --2- 172.21.15.27:0/152430580 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f0de802feb0 0x7f0de8032370 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f0dec0096a0 tx=0x7f0dec006b40).stop 2021-11-03T17:06:42.043 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:42.046+0000 7f0e0267d700 1 -- 172.21.15.27:0/152430580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dfc0fc1f0 msgr2=0x7f0dfc10cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:42.043 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:42.046+0000 7f0e0267d700 1 --2- 172.21.15.27:0/152430580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dfc0fc1f0 0x7f0dfc10cb40 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f0df0011e30 tx=0x7f0df0006ee0).stop 2021-11-03T17:06:42.044 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:42.046+0000 7f0e0267d700 1 -- 172.21.15.27:0/152430580 shutdown_connections 2021-11-03T17:06:42.045 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:42.046+0000 7f0e0267d700 1 --2- 172.21.15.27:0/152430580 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f0de802feb0 0x7f0de8032370 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:42.045 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:42.046+0000 7f0e0267d700 1 --2- 172.21.15.27:0/152430580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0dfc0fc1f0 0x7f0dfc10cb40 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:42.045 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:42.046+0000 7f0e0267d700 1 -- 172.21.15.27:0/152430580 >> 172.21.15.27:0/152430580 conn(0x7f0dfc0f5a60 msgr2=0x7f0dfc0f7570 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:42.045 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:42.046+0000 7f0e0267d700 1 -- 172.21.15.27:0/152430580 shutdown_connections 2021-11-03T17:06:42.046 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:42.047+0000 7f0e0267d700 1 -- 172.21.15.27:0/152430580 wait complete. 2021-11-03T17:06:42.580 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi027 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmp3f8lj10x:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpq2rb3_h7:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 config set mgr mgr/dashboard/ssl_server_port 8443 2021-11-03T17:06:43.214 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.214+0000 7fe08759e700 1 Processor -- start 2021-11-03T17:06:43.214 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.214+0000 7fe08759e700 1 -- start start 2021-11-03T17:06:43.215 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.215+0000 7fe08759e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0880fd090 0x7fe0880fd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:43.215 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.215+0000 7fe08759e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0880fda30 con 0x7fe0880fd090 2021-11-03T17:06:43.215 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.215+0000 7fe08659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0880fd090 0x7fe0880fd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:43.215 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.215+0000 7fe08659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0880fd090 0x7fe0880fd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54202/0 (socket says 172.21.15.27:54202) 2021-11-03T17:06:43.216 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.215+0000 7fe08659c700 1 -- 172.21.15.27:0/3390753427 learned_addr learned my addr 172.21.15.27:0/3390753427 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:43.216 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.215+0000 7fe08659c700 1 -- 172.21.15.27:0/3390753427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0880fdb70 con 0x7fe0880fd090 2021-11-03T17:06:43.216 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.215+0000 7fe08659c700 1 --2- 172.21.15.27:0/3390753427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0880fd090 0x7fe0880fd4b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fe07800b5f0 tx=0x7fe078006ee0).ready entity=mon.0 client_cookie=7a5397e022e38e1e server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:43.216 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.215+0000 7fe08559a700 1 -- 172.21.15.27:0/3390753427 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe07800d600 con 0x7fe0880fd090 2021-11-03T17:06:43.217 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.216+0000 7fe08559a700 1 -- 172.21.15.27:0/3390753427 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7fe07800dbf0 con 0x7fe0880fd090 2021-11-03T17:06:43.217 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.216+0000 7fe08559a700 1 -- 172.21.15.27:0/3390753427 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe078011bf0 con 0x7fe0880fd090 2021-11-03T17:06:43.217 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.216+0000 7fe08759e700 1 -- 172.21.15.27:0/3390753427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0880fd090 msgr2=0x7fe0880fd4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:43.217 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.216+0000 7fe08759e700 1 --2- 172.21.15.27:0/3390753427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0880fd090 0x7fe0880fd4b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fe07800b5f0 tx=0x7fe078006ee0).stop 2021-11-03T17:06:43.218 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.216+0000 7fe08759e700 1 -- 172.21.15.27:0/3390753427 shutdown_connections 2021-11-03T17:06:43.218 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.216+0000 7fe08759e700 1 --2- 172.21.15.27:0/3390753427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0880fd090 0x7fe0880fd4b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:43.218 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.216+0000 7fe08759e700 1 -- 172.21.15.27:0/3390753427 >> 172.21.15.27:0/3390753427 conn(0x7fe0880f5850 msgr2=0x7fe0880f7c70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:43.218 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.216+0000 7fe08759e700 1 -- 172.21.15.27:0/3390753427 shutdown_connections 2021-11-03T17:06:43.219 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.216+0000 7fe08759e700 1 -- 172.21.15.27:0/3390753427 wait complete. 2021-11-03T17:06:43.219 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.217+0000 7fe08759e700 1 Processor -- start 2021-11-03T17:06:43.219 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.217+0000 7fe08759e700 1 -- start start 2021-11-03T17:06:43.219 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.217+0000 7fe08759e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0880fd090 0x7fe0880fc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:43.220 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.217+0000 7fe08759e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0880fade0 con 0x7fe0880fd090 2021-11-03T17:06:43.220 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.217+0000 7fe08659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0880fd090 0x7fe0880fc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:43.220 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.217+0000 7fe08659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0880fd090 0x7fe0880fc960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54204/0 (socket says 172.21.15.27:54204) 2021-11-03T17:06:43.220 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.217+0000 7fe08659c700 1 -- 172.21.15.27:0/3012942804 learned_addr learned my addr 172.21.15.27:0/3012942804 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:43.221 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.217+0000 7fe08659c700 1 -- 172.21.15.27:0/3012942804 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe07800b200 con 0x7fe0880fd090 2021-11-03T17:06:43.221 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.217+0000 7fe08659c700 1 --2- 172.21.15.27:0/3012942804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0880fd090 0x7fe0880fc960 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fe078011e00 tx=0x7fe078006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:43.221 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.217+0000 7fe076ffd700 1 -- 172.21.15.27:0/3012942804 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe07801b410 con 0x7fe0880fd090 2021-11-03T17:06:43.221 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.217+0000 7fe076ffd700 1 -- 172.21.15.27:0/3012942804 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7fe07801ba00 con 0x7fe0880fd090 2021-11-03T17:06:43.222 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.217+0000 7fe08759e700 1 -- 172.21.15.27:0/3012942804 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0880faf80 con 0x7fe0880fd090 2021-11-03T17:06:43.222 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.218+0000 7fe076ffd700 1 -- 172.21.15.27:0/3012942804 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe07801a810 con 0x7fe0880fd090 2021-11-03T17:06:43.222 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.218+0000 7fe08759e700 1 -- 172.21.15.27:0/3012942804 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0880fb410 con 0x7fe0880fd090 2021-11-03T17:06:43.222 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.218+0000 7fe076ffd700 1 -- 172.21.15.27:0/3012942804 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 12) v1 ==== 36195+0+0 (secure 0 0 0) 0x7fe07802b070 con 0x7fe0880fd090 2021-11-03T17:06:43.223 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.218+0000 7fe076ffd700 1 --2- 172.21.15.27:0/3012942804 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7fe06c02ff00 0x7fe06c0323c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:43.224 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.218+0000 7fe076ffd700 1 -- 172.21.15.27:0/3012942804 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fe078041750 con 0x7fe0880fd090 2021-11-03T17:06:43.224 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.219+0000 7fe085d9b700 1 --2- 172.21.15.27:0/3012942804 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7fe06c02ff00 0x7fe06c0323c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:43.224 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.219+0000 7fe085d9b700 1 --2- 172.21.15.27:0/3012942804 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7fe06c02ff00 0x7fe06c0323c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fe07c0096a0 tx=0x7fe07c006b40).ready entity=mgr.14142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:43.224 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.219+0000 7fe08759e700 1 -- 172.21.15.27:0/3012942804 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe088059100 con 0x7fe0880fd090 2021-11-03T17:06:43.225 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.224+0000 7fe076ffd700 1 -- 172.21.15.27:0/3012942804 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fe07801f020 con 0x7fe0880fd090 2021-11-03T17:06:43.504 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.507+0000 7fe08759e700 1 -- 172.21.15.27:0/3012942804 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7fe088045ad0 con 0x7fe0880fd090 2021-11-03T17:06:43.506 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.510+0000 7fe076ffd700 1 -- 172.21.15.27:0/3012942804 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v10)=0 v10) v1 ==== 130+0+0 (secure 0 0 0) 0x7fe07802b8f0 con 0x7fe0880fd090 2021-11-03T17:06:43.510 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.512+0000 7fe08759e700 1 -- 172.21.15.27:0/3012942804 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7fe06c02ff00 msgr2=0x7fe06c0323c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:43.511 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.512+0000 7fe08759e700 1 --2- 172.21.15.27:0/3012942804 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7fe06c02ff00 0x7fe06c0323c0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fe07c0096a0 tx=0x7fe07c006b40).stop 2021-11-03T17:06:43.511 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.512+0000 7fe08759e700 1 -- 172.21.15.27:0/3012942804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0880fd090 msgr2=0x7fe0880fc960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:43.512 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.512+0000 7fe08759e700 1 --2- 172.21.15.27:0/3012942804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0880fd090 0x7fe0880fc960 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fe078011e00 tx=0x7fe078006ee0).stop 2021-11-03T17:06:43.512 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.512+0000 7fe08759e700 1 -- 172.21.15.27:0/3012942804 shutdown_connections 2021-11-03T17:06:43.513 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.513+0000 7fe08759e700 1 --2- 172.21.15.27:0/3012942804 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7fe06c02ff00 0x7fe06c0323c0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:43.513 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.513+0000 7fe08759e700 1 --2- 172.21.15.27:0/3012942804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0880fd090 0x7fe0880fc960 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:43.514 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.513+0000 7fe08759e700 1 -- 172.21.15.27:0/3012942804 >> 172.21.15.27:0/3012942804 conn(0x7fe0880f5850 msgr2=0x7fe0880ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:43.514 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.513+0000 7fe08759e700 1 -- 172.21.15.27:0/3012942804 shutdown_connections 2021-11-03T17:06:43.515 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:43.513+0000 7fe08759e700 1 -- 172.21.15.27:0/3012942804 wait complete. 2021-11-03T17:06:43.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:43 smithi027 conmon[32322]: audit 2021-11-03T17:06:42. 2021-11-03T17:06:43.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:43 smithi027 conmon[32322]: 046249+0000 mon.smithi027 (mon.0) 86 : audit [INF] from='client.? 172.21.15.27:0/152430580' entity='client.admin' 2021-11-03T17:06:44.025 INFO:teuthology.orchestra.run.smithi027.stderr:Enabling the dashboard module... 2021-11-03T17:06:44.025 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi027 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmp3f8lj10x:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpq2rb3_h7:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 mgr module enable dashboard 2021-11-03T17:06:44.661 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.663+0000 7fc69c904700 1 Processor -- start 2021-11-03T17:06:44.662 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.664+0000 7fc69c904700 1 -- start start 2021-11-03T17:06:44.662 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.664+0000 7fc69c904700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc698071c40 0x7fc6980701e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:44.662 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.664+0000 7fc69c904700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc698072060 con 0x7fc698071c40 2021-11-03T17:06:44.663 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.664+0000 7fc69659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc698071c40 0x7fc6980701e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:44.663 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.664+0000 7fc69659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc698071c40 0x7fc6980701e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54208/0 (socket says 172.21.15.27:54208) 2021-11-03T17:06:44.663 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.664+0000 7fc69659c700 1 -- 172.21.15.27:0/1434035259 learned_addr learned my addr 172.21.15.27:0/1434035259 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:44.663 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.664+0000 7fc69659c700 1 -- 172.21.15.27:0/1434035259 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6980706d0 con 0x7fc698071c40 2021-11-03T17:06:44.663 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.665+0000 7fc69659c700 1 --2- 172.21.15.27:0/1434035259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc698071c40 0x7fc6980701e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fc68c00b5f0 tx=0x7fc68c006ee0).ready entity=mon.0 client_cookie=460e341704c4221b server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:44.664 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.665+0000 7fc69559a700 1 -- 172.21.15.27:0/1434035259 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc68c00d600 con 0x7fc698071c40 2021-11-03T17:06:44.664 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.665+0000 7fc69559a700 1 -- 172.21.15.27:0/1434035259 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7fc68c00dbf0 con 0x7fc698071c40 2021-11-03T17:06:44.664 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.665+0000 7fc69559a700 1 -- 172.21.15.27:0/1434035259 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc68c011bf0 con 0x7fc698071c40 2021-11-03T17:06:44.664 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.665+0000 7fc69c904700 1 -- 172.21.15.27:0/1434035259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc698071c40 msgr2=0x7fc6980701e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:44.665 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.665+0000 7fc69c904700 1 --2- 172.21.15.27:0/1434035259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc698071c40 0x7fc6980701e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fc68c00b5f0 tx=0x7fc68c006ee0).stop 2021-11-03T17:06:44.665 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.666+0000 7fc69c904700 1 -- 172.21.15.27:0/1434035259 shutdown_connections 2021-11-03T17:06:44.668 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.666+0000 7fc69c904700 1 --2- 172.21.15.27:0/1434035259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc698071c40 0x7fc6980701e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:44.669 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.666+0000 7fc69c904700 1 -- 172.21.15.27:0/1434035259 >> 172.21.15.27:0/1434035259 conn(0x7fc6980f73d0 msgr2=0x7fc6980f97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:44.669 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.666+0000 7fc69c904700 1 -- 172.21.15.27:0/1434035259 shutdown_connections 2021-11-03T17:06:44.669 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.666+0000 7fc69c904700 1 -- 172.21.15.27:0/1434035259 wait complete. 2021-11-03T17:06:44.669 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.666+0000 7fc69c904700 1 Processor -- start 2021-11-03T17:06:44.670 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.667+0000 7fc69c904700 1 -- start start 2021-11-03T17:06:44.670 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.667+0000 7fc69c904700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc698071c40 0x7fc69810cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:44.670 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.667+0000 7fc69c904700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc69810d060 con 0x7fc698071c40 2021-11-03T17:06:44.670 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.667+0000 7fc69659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc698071c40 0x7fc69810cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:44.670 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.667+0000 7fc69659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc698071c40 0x7fc69810cb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54210/0 (socket says 172.21.15.27:54210) 2021-11-03T17:06:44.671 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.667+0000 7fc69659c700 1 -- 172.21.15.27:0/3016188635 learned_addr learned my addr 172.21.15.27:0/3016188635 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:44.671 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.667+0000 7fc69659c700 1 -- 172.21.15.27:0/3016188635 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc68c00b200 con 0x7fc698071c40 2021-11-03T17:06:44.671 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.667+0000 7fc69659c700 1 --2- 172.21.15.27:0/3016188635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc698071c40 0x7fc69810cb70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fc68c006800 tx=0x7fc68c006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:44.671 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.667+0000 7fc686ffd700 1 -- 172.21.15.27:0/3016188635 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc68c00dec0 con 0x7fc698071c40 2021-11-03T17:06:44.672 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.667+0000 7fc686ffd700 1 -- 172.21.15.27:0/3016188635 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7fc68c01b8a0 con 0x7fc698071c40 2021-11-03T17:06:44.672 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.667+0000 7fc69c904700 1 -- 172.21.15.27:0/3016188635 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc69810d260 con 0x7fc698071c40 2021-11-03T17:06:44.672 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.667+0000 7fc686ffd700 1 -- 172.21.15.27:0/3016188635 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc68c01a690 con 0x7fc698071c40 2021-11-03T17:06:44.672 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.668+0000 7fc69c904700 1 -- 172.21.15.27:0/3016188635 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc69810d6f0 con 0x7fc698071c40 2021-11-03T17:06:44.673 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.668+0000 7fc686ffd700 1 -- 172.21.15.27:0/3016188635 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 12) v1 ==== 36195+0+0 (secure 0 0 0) 0x7fc68c02b070 con 0x7fc698071c40 2021-11-03T17:06:44.673 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.668+0000 7fc686ffd700 1 --2- 172.21.15.27:0/3016188635 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7fc67c02feb0 0x7fc67c032370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:44.673 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.668+0000 7fc686ffd700 1 -- 172.21.15.27:0/3016188635 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (secure 0 0 0) 0x7fc68c041810 con 0x7fc698071c40 2021-11-03T17:06:44.673 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.669+0000 7fc695d9b700 1 --2- 172.21.15.27:0/3016188635 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7fc67c02feb0 0x7fc67c032370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:44.673 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.669+0000 7fc69c904700 1 -- 172.21.15.27:0/3016188635 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc698045ad0 con 0x7fc698071c40 2021-11-03T17:06:44.674 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.669+0000 7fc695d9b700 1 --2- 172.21.15.27:0/3016188635 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7fc67c02feb0 0x7fc67c032370 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fc6880096a0 tx=0x7fc688006b40).ready entity=mgr.14142 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:44.674 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:44.673+0000 7fc686ffd700 1 -- 172.21.15.27:0/3016188635 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fc68c01f020 con 0x7fc698071c40 2021-11-03T17:06:44.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:44 smithi027 conmon[32322]: audit 2021-11-03T17:06:43.512292+0000 mon.smithi027 (mon 2021-11-03T17:06:44.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:44 smithi027 conmon[32322]: .0) 87 : audit [INF] from='client.? 172.21.15.27:0/3012942804' entity='client.admin' 2021-11-03T17:06:45.027 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:45.031+0000 7fc69c904700 1 -- 172.21.15.27:0/3016188635 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7fc698101d80 con 0x7fc698071c40 2021-11-03T17:06:45.513 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:45.516+0000 7fc686ffd700 1 -- 172.21.15.27:0/3016188635 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 13) v1 ==== 36208+0+0 (secure 0 0 0) 0x7fc68c00f070 con 0x7fc698071c40 2021-11-03T17:06:45.513 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:45.518+0000 7fc686ffd700 1 -- 172.21.15.27:0/3016188635 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v13) v1 ==== 88+0+0 (secure 0 0 0) 0x7fc68c02ead0 con 0x7fc698071c40 2021-11-03T17:06:45.518 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:45.521+0000 7fc69c904700 1 -- 172.21.15.27:0/3016188635 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7fc67c02feb0 msgr2=0x7fc67c032370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:45.518 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:45.521+0000 7fc69c904700 1 --2- 172.21.15.27:0/3016188635 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7fc67c02feb0 0x7fc67c032370 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fc6880096a0 tx=0x7fc688006b40).stop 2021-11-03T17:06:45.519 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:45.521+0000 7fc69c904700 1 -- 172.21.15.27:0/3016188635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc698071c40 msgr2=0x7fc69810cb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:45.519 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:45.521+0000 7fc69c904700 1 --2- 172.21.15.27:0/3016188635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc698071c40 0x7fc69810cb70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fc68c006800 tx=0x7fc68c006ee0).stop 2021-11-03T17:06:45.519 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:45.521+0000 7fc69c904700 1 -- 172.21.15.27:0/3016188635 shutdown_connections 2021-11-03T17:06:45.519 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:45.521+0000 7fc69c904700 1 --2- 172.21.15.27:0/3016188635 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7fc67c02feb0 0x7fc67c032370 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:45.519 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:45.521+0000 7fc69c904700 1 --2- 172.21.15.27:0/3016188635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc698071c40 0x7fc69810cb70 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:45.520 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:45.521+0000 7fc69c904700 1 -- 172.21.15.27:0/3016188635 >> 172.21.15.27:0/3016188635 conn(0x7fc6980f73d0 msgr2=0x7fc6980f8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:45.520 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:45.521+0000 7fc69c904700 1 -- 172.21.15.27:0/3016188635 shutdown_connections 2021-11-03T17:06:45.520 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:45.521+0000 7fc69c904700 1 -- 172.21.15.27:0/3016188635 wait complete. 2021-11-03T17:06:45.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:45 smithi027 conmon[32322]: audit 2021-11-03T17:06:45.033533+0000 mon.smithi027 ( 2021-11-03T17:06:45.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:45 smithi027 conmon[32322]: mon.0) 88 : audit [INF] from='client.? 172.21.15.27:0/3016188635' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2021-11-03T17:06:46.126 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi027 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmp3f8lj10x:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpq2rb3_h7:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 mgr stat 2021-11-03T17:06:46.762 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.763+0000 7f8b18701700 1 Processor -- start 2021-11-03T17:06:46.763 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.763+0000 7f8b18701700 1 -- start start 2021-11-03T17:06:46.763 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.763+0000 7f8b18701700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b100faa80 0x7f8b100faea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:46.763 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.763+0000 7f8b18701700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b100fb390 con 0x7f8b100faa80 2021-11-03T17:06:46.763 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.763+0000 7f8b1649d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b100faa80 0x7f8b100faea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:46.764 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.763+0000 7f8b1649d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b100faa80 0x7f8b100faea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54224/0 (socket says 172.21.15.27:54224) 2021-11-03T17:06:46.764 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.763+0000 7f8b1649d700 1 -- 172.21.15.27:0/3601866455 learned_addr learned my addr 172.21.15.27:0/3601866455 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:46.764 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.764+0000 7f8b1649d700 1 -- 172.21.15.27:0/3601866455 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b100fb4d0 con 0x7f8b100faa80 2021-11-03T17:06:46.764 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.764+0000 7f8b1649d700 1 --2- 172.21.15.27:0/3601866455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b100faa80 0x7f8b100faea0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f8b0c00b5f0 tx=0x7f8b0c006ee0).ready entity=mon.0 client_cookie=56f598044fc99e99 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:46.764 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.764+0000 7f8b1549b700 1 -- 172.21.15.27:0/3601866455 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8b0c00d600 con 0x7f8b100faa80 2021-11-03T17:06:46.765 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.764+0000 7f8b1549b700 1 -- 172.21.15.27:0/3601866455 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f8b0c00dbf0 con 0x7f8b100faa80 2021-11-03T17:06:46.765 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.764+0000 7f8b1549b700 1 -- 172.21.15.27:0/3601866455 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8b0c011bf0 con 0x7f8b100faa80 2021-11-03T17:06:46.765 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.764+0000 7f8b18701700 1 -- 172.21.15.27:0/3601866455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b100faa80 msgr2=0x7f8b100faea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:46.765 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.764+0000 7f8b18701700 1 --2- 172.21.15.27:0/3601866455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b100faa80 0x7f8b100faea0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f8b0c00b5f0 tx=0x7f8b0c006ee0).stop 2021-11-03T17:06:46.766 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.765+0000 7f8b18701700 1 -- 172.21.15.27:0/3601866455 shutdown_connections 2021-11-03T17:06:46.766 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.765+0000 7f8b18701700 1 --2- 172.21.15.27:0/3601866455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b100faa80 0x7f8b100faea0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:46.766 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.765+0000 7f8b18701700 1 -- 172.21.15.27:0/3601866455 >> 172.21.15.27:0/3601866455 conn(0x7f8b100f6260 msgr2=0x7f8b100f86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:46.766 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.765+0000 7f8b18701700 1 -- 172.21.15.27:0/3601866455 shutdown_connections 2021-11-03T17:06:46.767 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.765+0000 7f8b18701700 1 -- 172.21.15.27:0/3601866455 wait complete. 2021-11-03T17:06:46.767 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.765+0000 7f8b18701700 1 Processor -- start 2021-11-03T17:06:46.767 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.765+0000 7f8b18701700 1 -- start start 2021-11-03T17:06:46.767 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.766+0000 7f8b18701700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b100faa80 0x7f8b1010d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:46.767 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.766+0000 7f8b18701700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b1006fbe0 con 0x7f8b100faa80 2021-11-03T17:06:46.768 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.766+0000 7f8b1649d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b100faa80 0x7f8b1010d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:46.768 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.766+0000 7f8b1649d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b100faa80 0x7f8b1010d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54226/0 (socket says 172.21.15.27:54226) 2021-11-03T17:06:46.768 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.766+0000 7f8b1649d700 1 -- 172.21.15.27:0/4079525977 learned_addr learned my addr 172.21.15.27:0/4079525977 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:46.768 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.766+0000 7f8b1649d700 1 -- 172.21.15.27:0/4079525977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b0c00b200 con 0x7f8b100faa80 2021-11-03T17:06:46.769 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.766+0000 7f8b1649d700 1 --2- 172.21.15.27:0/4079525977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b100faa80 0x7f8b1010d5c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f8b0c011e30 tx=0x7f8b0c006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:46.769 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.766+0000 7f8afeffd700 1 -- 172.21.15.27:0/4079525977 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8b0c01b410 con 0x7f8b100faa80 2021-11-03T17:06:46.769 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.766+0000 7f8afeffd700 1 -- 172.21.15.27:0/4079525977 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f8b0c01ba00 con 0x7f8b100faa80 2021-11-03T17:06:46.771 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.766+0000 7f8b18701700 1 -- 172.21.15.27:0/4079525977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b1006fde0 con 0x7f8b100faa80 2021-11-03T17:06:46.771 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.766+0000 7f8afeffd700 1 -- 172.21.15.27:0/4079525977 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8b0c01a7b0 con 0x7f8b100faa80 2021-11-03T17:06:46.771 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.766+0000 7f8b18701700 1 -- 172.21.15.27:0/4079525977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b1010dc70 con 0x7f8b100faa80 2021-11-03T17:06:46.771 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.767+0000 7f8afeffd700 1 -- 172.21.15.27:0/4079525977 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) v1 ==== 36208+0+0 (secure 0 0 0) 0x7f8b0c02b070 con 0x7f8b100faa80 2021-11-03T17:06:46.772 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.767+0000 7f8afeffd700 1 --2- 172.21.15.27:0/4079525977 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f8af802ff00 0x7f8af80323c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:46.772 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.767+0000 7f8afeffd700 1 -- 172.21.15.27:0/4079525977 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f8b0c041780 con 0x7f8b100faa80 2021-11-03T17:06:46.772 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.767+0000 7f8b15c9c700 1 -- 172.21.15.27:0/4079525977 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f8af802ff00 msgr2=0x7f8af80323c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3506673123 2021-11-03T17:06:46.772 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.767+0000 7f8b15c9c700 1 --2- 172.21.15.27:0/4079525977 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f8af802ff00 0x7f8af80323c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2021-11-03T17:06:46.773 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.768+0000 7f8b18701700 1 -- 172.21.15.27:0/4079525977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b10059100 con 0x7f8b100faa80 2021-11-03T17:06:46.773 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.772+0000 7f8afeffd700 1 -- 172.21.15.27:0/4079525977 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f8b0c01f080 con 0x7f8b100faa80 2021-11-03T17:06:46.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:46 smithi027 conmon[32322]: audit 2021-11-03T17:06:45.519344+0000 mon.smithi027 (mon. 2021-11-03T17:06:46.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:46 smithi027 conmon[32322]: 0) 89 : audit [INF] from='client.? 172.21.15.27:0/3016188635' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2021-11-03T17:06:46.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:46 smithi027 conmon[32322]: cluster 2021-11-03T17:06:45.519440+0000 mon.smithi027 (mon.0) 90 : cluster [DBG] mgrmap e13: smithi027.ufhhvb(active, since 11s) 2021-11-03T17:06:46.964 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.968+0000 7f8b15c9c700 1 -- 172.21.15.27:0/4079525977 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f8af802ff00 msgr2=0x7f8af80323c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3506673123 2021-11-03T17:06:46.964 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:46.968+0000 7f8b15c9c700 1 --2- 172.21.15.27:0/4079525977 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f8af802ff00 0x7f8af80323c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2021-11-03T17:06:47.162 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:47.165+0000 7f8b18701700 1 -- 172.21.15.27:0/4079525977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f8b100fc7d0 con 0x7f8b100faa80 2021-11-03T17:06:47.163 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:47.165+0000 7f8afeffd700 1 -- 172.21.15.27:0/4079525977 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v13) v1 ==== 56+0+104 (secure 0 0 0) 0x7f8b0c016070 con 0x7f8b100faa80 2021-11-03T17:06:47.163 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: { 2021-11-03T17:06:47.163 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "epoch": 13, 2021-11-03T17:06:47.163 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "available": true, 2021-11-03T17:06:47.164 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "active_name": "smithi027.ufhhvb", 2021-11-03T17:06:47.164 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "num_standby": 0 2021-11-03T17:06:47.164 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: } 2021-11-03T17:06:47.166 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:47.167+0000 7f8b18701700 1 -- 172.21.15.27:0/4079525977 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f8af802ff00 msgr2=0x7f8af80323c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2021-11-03T17:06:47.166 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:47.167+0000 7f8b18701700 1 --2- 172.21.15.27:0/4079525977 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f8af802ff00 0x7f8af80323c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:06:47.166 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:47.167+0000 7f8b18701700 1 -- 172.21.15.27:0/4079525977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b100faa80 msgr2=0x7f8b1010d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:47.167 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:47.167+0000 7f8b18701700 1 --2- 172.21.15.27:0/4079525977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b100faa80 0x7f8b1010d5c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f8b0c011e30 tx=0x7f8b0c006ee0).stop 2021-11-03T17:06:47.167 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:47.167+0000 7f8b18701700 1 -- 172.21.15.27:0/4079525977 shutdown_connections 2021-11-03T17:06:47.167 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:47.167+0000 7f8b18701700 1 --2- 172.21.15.27:0/4079525977 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f8af802ff00 0x7f8af80323c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:06:47.167 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:47.167+0000 7f8b18701700 1 --2- 172.21.15.27:0/4079525977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b100faa80 0x7f8b1010d5c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:47.167 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:47.167+0000 7f8b18701700 1 -- 172.21.15.27:0/4079525977 >> 172.21.15.27:0/4079525977 conn(0x7f8b100f6260 msgr2=0x7f8b100f7c80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:47.168 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:47.168+0000 7f8b18701700 1 -- 172.21.15.27:0/4079525977 shutdown_connections 2021-11-03T17:06:47.168 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:47.168+0000 7f8b18701700 1 -- 172.21.15.27:0/4079525977 wait complete. 2021-11-03T17:06:47.670 INFO:teuthology.orchestra.run.smithi027.stderr:Waiting for the mgr to restart... 2021-11-03T17:06:47.670 INFO:teuthology.orchestra.run.smithi027.stderr:Waiting for mgr epoch 13... 2021-11-03T17:06:47.671 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi027 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmp3f8lj10x:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpq2rb3_h7:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 tell mgr mgr_status 2021-11-03T17:06:47.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:47 smithi027 conmon[32322]: audit 2021-11-03T17:06:47.167271+0000 mon.smithi027 (mon 2021-11-03T17:06:47.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:47 smithi027 conmon[32322]: .0) 91 : audit [DBG] from='client.? 172.21.15.27:0/4079525977' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2021-11-03T17:06:48.332 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.331+0000 7f272fa2c700 1 Processor -- start 2021-11-03T17:06:48.332 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.332+0000 7f272fa2c700 1 -- start start 2021-11-03T17:06:48.332 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.332+0000 7f272fa2c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f272806fb40 0x7f2728100c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:48.333 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.332+0000 7f272fa2c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27280fc650 con 0x7f272806fb40 2021-11-03T17:06:48.333 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.332+0000 7f272d7c8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f272806fb40 0x7f2728100c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:48.333 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.332+0000 7f272d7c8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f272806fb40 0x7f2728100c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54234/0 (socket says 172.21.15.27:54234) 2021-11-03T17:06:48.333 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.332+0000 7f272d7c8700 1 -- 172.21.15.27:0/2136759957 learned_addr learned my addr 172.21.15.27:0/2136759957 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:48.333 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.333+0000 7f272d7c8700 1 -- 172.21.15.27:0/2136759957 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2728101150 con 0x7f272806fb40 2021-11-03T17:06:48.334 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.333+0000 7f272d7c8700 1 --2- 172.21.15.27:0/2136759957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f272806fb40 0x7f2728100c60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f272400b5f0 tx=0x7f2724006ee0).ready entity=mon.0 client_cookie=1acc8717e840b214 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:48.334 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.333+0000 7f271bfff700 1 -- 172.21.15.27:0/2136759957 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f272400d600 con 0x7f272806fb40 2021-11-03T17:06:48.334 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.333+0000 7f271bfff700 1 -- 172.21.15.27:0/2136759957 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f272400dbf0 con 0x7f272806fb40 2021-11-03T17:06:48.334 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.333+0000 7f271bfff700 1 -- 172.21.15.27:0/2136759957 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2724011bf0 con 0x7f272806fb40 2021-11-03T17:06:48.335 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.334+0000 7f272fa2c700 1 -- 172.21.15.27:0/2136759957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f272806fb40 msgr2=0x7f2728100c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:48.335 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.334+0000 7f272fa2c700 1 --2- 172.21.15.27:0/2136759957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f272806fb40 0x7f2728100c60 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f272400b5f0 tx=0x7f2724006ee0).stop 2021-11-03T17:06:48.335 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.334+0000 7f272fa2c700 1 -- 172.21.15.27:0/2136759957 shutdown_connections 2021-11-03T17:06:48.335 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.334+0000 7f272fa2c700 1 --2- 172.21.15.27:0/2136759957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f272806fb40 0x7f2728100c60 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:48.335 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.334+0000 7f272fa2c700 1 -- 172.21.15.27:0/2136759957 >> 172.21.15.27:0/2136759957 conn(0x7f27280f6260 msgr2=0x7f27280f86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:48.336 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.334+0000 7f272fa2c700 1 -- 172.21.15.27:0/2136759957 shutdown_connections 2021-11-03T17:06:48.336 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.334+0000 7f272fa2c700 1 -- 172.21.15.27:0/2136759957 wait complete. 2021-11-03T17:06:48.336 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.335+0000 7f272fa2c700 1 Processor -- start 2021-11-03T17:06:48.336 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.335+0000 7f272fa2c700 1 -- start start 2021-11-03T17:06:48.337 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.335+0000 7f272fa2c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f272806fb40 0x7f272810cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:48.337 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.335+0000 7f272fa2c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272810cfa0 con 0x7f272806fb40 2021-11-03T17:06:48.337 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.335+0000 7f272d7c8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f272806fb40 0x7f272810cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:48.337 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.335+0000 7f272d7c8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f272806fb40 0x7f272810cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54236/0 (socket says 172.21.15.27:54236) 2021-11-03T17:06:48.338 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.335+0000 7f272d7c8700 1 -- 172.21.15.27:0/443191504 learned_addr learned my addr 172.21.15.27:0/443191504 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:48.338 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.335+0000 7f272d7c8700 1 -- 172.21.15.27:0/443191504 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f272400b200 con 0x7f272806fb40 2021-11-03T17:06:48.338 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.335+0000 7f272d7c8700 1 --2- 172.21.15.27:0/443191504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f272806fb40 0x7f272810cab0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f2724006800 tx=0x7f2724006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:48.338 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.336+0000 7f2719ffb700 1 -- 172.21.15.27:0/443191504 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f272401b410 con 0x7f272806fb40 2021-11-03T17:06:48.339 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.336+0000 7f2719ffb700 1 -- 172.21.15.27:0/443191504 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f272401ba00 con 0x7f272806fb40 2021-11-03T17:06:48.340 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.336+0000 7f272fa2c700 1 -- 172.21.15.27:0/443191504 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f272810d1a0 con 0x7f272806fb40 2021-11-03T17:06:48.340 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.336+0000 7f2719ffb700 1 -- 172.21.15.27:0/443191504 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f272401a6e0 con 0x7f272806fb40 2021-11-03T17:06:48.340 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.336+0000 7f272fa2c700 1 -- 172.21.15.27:0/443191504 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f272810d630 con 0x7f272806fb40 2021-11-03T17:06:48.341 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.337+0000 7f2719ffb700 1 -- 172.21.15.27:0/443191504 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 13) v1 ==== 36208+0+0 (secure 0 0 0) 0x7f272402b070 con 0x7f272806fb40 2021-11-03T17:06:48.341 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.337+0000 7f2719ffb700 1 --2- 172.21.15.27:0/443191504 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f271002feb0 0x7f2710032370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:48.341 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.337+0000 7f2719ffb700 1 -- 172.21.15.27:0/443191504 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1065+0+0 (secure 0 0 0) 0x7f2724042600 con 0x7f272806fb40 2021-11-03T17:06:48.341 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.337+0000 7f272fa2c700 1 -- 172.21.15.27:0/443191504 --> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2728059100 con 0x7f271002feb0 2021-11-03T17:06:48.341 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.337+0000 7f272cfc7700 1 -- 172.21.15.27:0/443191504 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f271002feb0 msgr2=0x7f2710032370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3506673123 2021-11-03T17:06:48.342 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.337+0000 7f272cfc7700 1 --2- 172.21.15.27:0/443191504 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f271002feb0 0x7f2710032370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2021-11-03T17:06:48.534 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.538+0000 7f272cfc7700 1 -- 172.21.15.27:0/443191504 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f271002feb0 msgr2=0x7f2710032370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3506673123 2021-11-03T17:06:48.535 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.538+0000 7f272cfc7700 1 --2- 172.21.15.27:0/443191504 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f271002feb0 0x7f2710032370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2021-11-03T17:06:48.935 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.939+0000 7f272cfc7700 1 -- 172.21.15.27:0/443191504 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f271002feb0 msgr2=0x7f2710032370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3506673123 2021-11-03T17:06:48.935 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:48.939+0000 7f272cfc7700 1 --2- 172.21.15.27:0/443191504 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f271002feb0 0x7f2710032370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2021-11-03T17:06:49.736 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:49.740+0000 7f272cfc7700 1 -- 172.21.15.27:0/443191504 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f271002feb0 msgr2=0x7f2710032370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3506673123 2021-11-03T17:06:49.736 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:49.740+0000 7f272cfc7700 1 --2- 172.21.15.27:0/443191504 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f271002feb0 0x7f2710032370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2021-11-03T17:06:51.338 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:51.341+0000 7f272cfc7700 1 -- 172.21.15.27:0/443191504 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f271002feb0 msgr2=0x7f2710032370 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6800/3506673123 2021-11-03T17:06:51.338 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:51.341+0000 7f272cfc7700 1 --2- 172.21.15.27:0/443191504 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f271002feb0 0x7f2710032370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2021-11-03T17:06:52.514 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:52.517+0000 7f2719ffb700 1 -- 172.21.15.27:0/443191504 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mgrmap(e 14) v1 ==== 35997+0+0 (secure 0 0 0) 0x7f272400f070 con 0x7f272806fb40 2021-11-03T17:06:52.514 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:52.518+0000 7f2719ffb700 1 -- 172.21.15.27:0/443191504 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f271002feb0 msgr2=0x7f2710032370 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2021-11-03T17:06:52.515 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:52.518+0000 7f2719ffb700 1 --2- 172.21.15.27:0/443191504 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f271002feb0 0x7f2710032370 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:06:52.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:52 smithi027 conmon[32322]: cluster 2021-11-03T17:06:51.527577+0000 mon.smithi027 (mon.0) 92 : cluster [INF] Active manager daemon smithi027.ufhhvb restarted 2021-11-03T17:06:52.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:52 smithi027 conmon[32322]: cluster 2021-11-03T17:06:51.528029+0000 mon.smithi027 (mon.0) 93 : cluster [INF] Activating manager daemon smithi027.ufhhvb 2021-11-03T17:06:52.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:52 smithi027 conmon[32322]: cluster 2021-11-03T17:06:51.534055+0000 mon.smithi027 (mon.0) 94 : cluster [DBG] osdmap e4: 0 total, 0 up, 0 in 2021-11-03T17:06:53.516 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:53.519+0000 7f2719ffb700 1 -- 172.21.15.27:0/443191504 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mgrmap(e 15) v1 ==== 36067+0+0 (secure 0 0 0) 0x7f2724041280 con 0x7f272806fb40 2021-11-03T17:06:53.517 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:53.519+0000 7f2719ffb700 1 --2- 172.21.15.27:0/443191504 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f2710033920 0x7f2710035d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:53.517 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:53.519+0000 7f2719ffb700 1 -- 172.21.15.27:0/443191504 --> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f2728059100 con 0x7f2710033920 2021-11-03T17:06:53.517 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:53.519+0000 7f272cfc7700 1 --2- 172.21.15.27:0/443191504 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f2710033920 0x7f2710035d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:53.517 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:53.520+0000 7f272cfc7700 1 --2- 172.21.15.27:0/443191504 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f2710033920 0x7f2710035d10 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7f271c006fd0 tx=0x7f271c006cf0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:53.518 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:53.520+0000 7f2719ffb700 1 -- 172.21.15.27:0/443191504 <== mgr.14162 v2:172.21.15.27:6800/3802716887 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6438 (secure 0 0 0) 0x7f2728059100 con 0x7f2710033920 2021-11-03T17:06:53.533 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:53.535+0000 7f272fa2c700 1 -- 172.21.15.27:0/443191504 --> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f2728045ad0 con 0x7f2710033920 2021-11-03T17:06:53.533 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:53.536+0000 7f2719ffb700 1 -- 172.21.15.27:0/443191504 <== mgr.14162 v2:172.21.15.27:6800/3802716887 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f2728045ad0 con 0x7f2710033920 2021-11-03T17:06:53.534 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: { 2021-11-03T17:06:53.534 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "mgrmap_epoch": 15, 2021-11-03T17:06:53.534 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: "initialized": true 2021-11-03T17:06:53.535 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: } 2021-11-03T17:06:53.535 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:53.536+0000 7f272fa2c700 1 -- 172.21.15.27:0/443191504 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f2710033920 msgr2=0x7f2710035d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:53.535 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:53.536+0000 7f272fa2c700 1 --2- 172.21.15.27:0/443191504 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f2710033920 0x7f2710035d10 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 rx=0x7f271c006fd0 tx=0x7f271c006cf0).stop 2021-11-03T17:06:53.535 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:53.536+0000 7f272fa2c700 1 -- 172.21.15.27:0/443191504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f272806fb40 msgr2=0x7f272810cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:53.535 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:53.536+0000 7f272fa2c700 1 --2- 172.21.15.27:0/443191504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f272806fb40 0x7f272810cab0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f2724006800 tx=0x7f2724006ee0).stop 2021-11-03T17:06:53.536 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:53.536+0000 7f272fa2c700 1 -- 172.21.15.27:0/443191504 shutdown_connections 2021-11-03T17:06:53.536 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:53.536+0000 7f272fa2c700 1 --2- 172.21.15.27:0/443191504 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f2710033920 0x7f2710035d10 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:53.536 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:53.536+0000 7f272fa2c700 1 --2- 172.21.15.27:0/443191504 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f272806fb40 0x7f272810cab0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:53.536 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:53.536+0000 7f272fa2c700 1 --2- 172.21.15.27:0/443191504 >> [v2:172.21.15.27:6800/3506673123,v1:172.21.15.27:6801/3506673123] conn(0x7f271002feb0 0x7f2710032370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:06:53.537 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:53.536+0000 7f272fa2c700 1 -- 172.21.15.27:0/443191504 >> 172.21.15.27:0/443191504 conn(0x7f27280f6260 msgr2=0x7f27280f7cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:53.537 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:53.536+0000 7f272fa2c700 1 -- 172.21.15.27:0/443191504 shutdown_connections 2021-11-03T17:06:53.537 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:53.536+0000 7f272fa2c700 1 -- 172.21.15.27:0/443191504 wait complete. 2021-11-03T17:06:53.777 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:53 smithi027 conmon[32322]: cluster 2021-11-03T17:06:52.520576+0000 mon.smithi027 (mon.0) 95 : cluster [DBG] mgrmap e14: smithi027.ufhhvb(active, starting, since 0.992626s) 2021-11-03T17:06:53.841 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:53 smithi027 conmon[32322]: audit 2021-11-03T17:06:52.521665+0000 mon.smithi027 (mon.0) 96 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2021-11-03T17:06:53.841 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:53 smithi027 conmon[32322]: audit 2021-11-03T17:06:52.522598+0000 mon.smithi027 (mon.0) 97 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mgr metadata", "who": "smithi027.ufhhvb", "id": "smithi027.ufhhvb"}]: dispatch 2021-11-03T17:06:53.842 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:53 smithi027 conmon[32322]: audit 2021-11-03T17:06:52.523583+0000 mon.smithi027 (mon.0) 98 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata"}]: dispatch 2021-11-03T17:06:53.842 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:53 smithi027 conmon[32322]: audit 2021-11-03T17:06:52.523791+0000 mon.smithi027 (mon.0) 99 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata"}]: dispatch 2021-11-03T17:06:53.842 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:53 smithi027 conmon[32322]: audit 2021-11-03T17:06:52.523996+0000 mon.smithi027 (mon.0) 100 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata"}]: dispatch 2021-11-03T17:06:53.842 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:53 smithi027 conmon[32322]: cluster 2021-11-03T17:06:52.538523+0000 mon.smithi027 (mon.0) 101 : cluster [INF] Manager daemon smithi027.ufhhvb is now available 2021-11-03T17:06:53.843 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:53 smithi027 conmon[32322]: audit 2021-11 2021-11-03T17:06:53.843 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:53 smithi027 conmon[32322]: -03T17:06:52.681503+0000 mon.smithi027 (mon.0) 102 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:06:53.843 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:53 smithi027 conmon[32322]: audit 2021-11-03T17:06:52.684011+0000 mon.smithi027 (mon.0) 103 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:06:53.843 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:53 smithi027 conmon[32322]: audit 2021-11-03T17:06:52.704054+0000 mon.smithi027 (mon.0) 104 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:06:53.843 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:53 smithi027 conmon[32322]: audit 2021-11-03T17:06:52.713956+0000 mon.smithi027 (mon.0) 105 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:06:54.203 INFO:teuthology.orchestra.run.smithi027.stderr:mgr epoch 13 is available 2021-11-03T17:06:54.203 INFO:teuthology.orchestra.run.smithi027.stderr:Generating a dashboard self-signed certificate... 2021-11-03T17:06:54.204 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi027 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmp3f8lj10x:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpq2rb3_h7:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 dashboard create-self-signed-cert 2021-11-03T17:06:54.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:54 smithi027 conmon[32322]: audit 2021-11-03T17:06:53.522073+0000 mgr.smithi027.ufhhvb 2021-11-03T17:06:54.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:54 smithi027 conmon[32322]: (mgr.14162) 1 : audit [DBG] from='client.14166 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2021-11-03T17:06:54.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:54 smithi027 conmon[32322]: cluster 2021-11-03T17:06:53.522903+0000 mon.smithi027 (mon.0) 106 : cluster [DBG] mgrmap e15: smithi027.ufhhvb(active, since 1.99496s) 2021-11-03T17:06:54.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:54 smithi027 conmon[32322]: audit 2021-11-03T17:06:53.537623+0000 mgr.smithi027.ufhhvb (mgr.14162) 2 : audit [DBG] from='client.14166 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2021-11-03T17:06:54.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:54 smithi027 conmon[32322]: audit 2021-11-03T17:06:54.210177+0000 mon.smithi027 (mon.0) 107 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:06:54.911 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.914+0000 7fb86209c700 1 Processor -- start 2021-11-03T17:06:54.911 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.914+0000 7fb86209c700 1 -- start start 2021-11-03T17:06:54.911 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.914+0000 7fb86209c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb85c06fbc0 0x7fb85c0fe630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:54.912 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.914+0000 7fb86209c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c0feb20 con 0x7fb85c06fbc0 2021-11-03T17:06:54.912 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.915+0000 7fb86109a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb85c06fbc0 0x7fb85c0fe630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:54.912 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.915+0000 7fb86109a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb85c06fbc0 0x7fb85c0fe630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54284/0 (socket says 172.21.15.27:54284) 2021-11-03T17:06:54.912 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.915+0000 7fb86109a700 1 -- 172.21.15.27:0/181954782 learned_addr learned my addr 172.21.15.27:0/181954782 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:54.913 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.915+0000 7fb86109a700 1 -- 172.21.15.27:0/181954782 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb85c0fec60 con 0x7fb85c06fbc0 2021-11-03T17:06:54.913 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.915+0000 7fb86109a700 1 --2- 172.21.15.27:0/181954782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb85c06fbc0 0x7fb85c0fe630 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fb85800b5f0 tx=0x7fb858006ee0).ready entity=mon.0 client_cookie=36a4f403b48d5082 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:54.913 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.915+0000 7fb84ffff700 1 -- 172.21.15.27:0/181954782 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb85800d600 con 0x7fb85c06fbc0 2021-11-03T17:06:54.914 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.915+0000 7fb84ffff700 1 -- 172.21.15.27:0/181954782 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7fb85800dbf0 con 0x7fb85c06fbc0 2021-11-03T17:06:54.916 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.915+0000 7fb84ffff700 1 -- 172.21.15.27:0/181954782 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb858011bf0 con 0x7fb85c06fbc0 2021-11-03T17:06:54.916 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.916+0000 7fb86209c700 1 -- 172.21.15.27:0/181954782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb85c06fbc0 msgr2=0x7fb85c0fe630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:54.916 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.916+0000 7fb86209c700 1 --2- 172.21.15.27:0/181954782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb85c06fbc0 0x7fb85c0fe630 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fb85800b5f0 tx=0x7fb858006ee0).stop 2021-11-03T17:06:54.916 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.916+0000 7fb86209c700 1 -- 172.21.15.27:0/181954782 shutdown_connections 2021-11-03T17:06:54.917 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.916+0000 7fb86209c700 1 --2- 172.21.15.27:0/181954782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb85c06fbc0 0x7fb85c0fe630 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:54.917 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.916+0000 7fb86209c700 1 -- 172.21.15.27:0/181954782 >> 172.21.15.27:0/181954782 conn(0x7fb85c0f6110 msgr2=0x7fb85c0f8530 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:54.917 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.916+0000 7fb86209c700 1 -- 172.21.15.27:0/181954782 shutdown_connections 2021-11-03T17:06:54.917 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.916+0000 7fb86209c700 1 -- 172.21.15.27:0/181954782 wait complete. 2021-11-03T17:06:54.917 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.917+0000 7fb86209c700 1 Processor -- start 2021-11-03T17:06:54.918 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.917+0000 7fb86209c700 1 -- start start 2021-11-03T17:06:54.918 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.917+0000 7fb86209c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb85c06fbc0 0x7fb85c0fc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:54.918 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.917+0000 7fb86209c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85c0fa530 con 0x7fb85c06fbc0 2021-11-03T17:06:54.918 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.917+0000 7fb86109a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb85c06fbc0 0x7fb85c0fc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:54.919 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.917+0000 7fb86109a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb85c06fbc0 0x7fb85c0fc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54286/0 (socket says 172.21.15.27:54286) 2021-11-03T17:06:54.919 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.917+0000 7fb86109a700 1 -- 172.21.15.27:0/261958244 learned_addr learned my addr 172.21.15.27:0/261958244 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:54.919 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.918+0000 7fb86109a700 1 -- 172.21.15.27:0/261958244 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb85800b200 con 0x7fb85c06fbc0 2021-11-03T17:06:54.919 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.918+0000 7fb86109a700 1 --2- 172.21.15.27:0/261958244 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb85c06fbc0 0x7fb85c0fc0b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fb858011e00 tx=0x7fb858006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:54.920 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.918+0000 7fb84dffb700 1 -- 172.21.15.27:0/261958244 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb85801b410 con 0x7fb85c06fbc0 2021-11-03T17:06:54.920 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.918+0000 7fb84dffb700 1 -- 172.21.15.27:0/261958244 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7fb85801ba00 con 0x7fb85c06fbc0 2021-11-03T17:06:54.920 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.918+0000 7fb84dffb700 1 -- 172.21.15.27:0/261958244 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb85801a780 con 0x7fb85c06fbc0 2021-11-03T17:06:54.920 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.918+0000 7fb86209c700 1 -- 172.21.15.27:0/261958244 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb85c0fa6d0 con 0x7fb85c06fbc0 2021-11-03T17:06:54.920 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.918+0000 7fb86209c700 1 -- 172.21.15.27:0/261958244 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb85c0fab60 con 0x7fb85c06fbc0 2021-11-03T17:06:54.921 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.919+0000 7fb84dffb700 1 -- 172.21.15.27:0/261958244 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) v1 ==== 36208+0+0 (secure 0 0 0) 0x7fb85802b070 con 0x7fb85c06fbc0 2021-11-03T17:06:54.921 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.919+0000 7fb84dffb700 1 --2- 172.21.15.27:0/261958244 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fb84402feb0 0x7fb844032370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:54.921 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.919+0000 7fb84dffb700 1 -- 172.21.15.27:0/261958244 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1237+0+0 (secure 0 0 0) 0x7fb8580417a0 con 0x7fb85c06fbc0 2021-11-03T17:06:54.922 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.919+0000 7fb860899700 1 --2- 172.21.15.27:0/261958244 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fb84402feb0 0x7fb844032370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:54.922 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.920+0000 7fb860899700 1 --2- 172.21.15.27:0/261958244 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fb84402feb0 0x7fb844032370 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fb8500096a0 tx=0x7fb850006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:54.922 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.920+0000 7fb86209c700 1 -- 172.21.15.27:0/261958244 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb85c059100 con 0x7fb85c06fbc0 2021-11-03T17:06:54.923 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:54.924+0000 7fb84dffb700 1 -- 172.21.15.27:0/261958244 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fb858016030 con 0x7fb85c06fbc0 2021-11-03T17:06:55.224 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:55.228+0000 7fb86209c700 1 -- 172.21.15.27:0/261958244 --> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fb85c0ff990 con 0x7fb84402feb0 2021-11-03T17:06:55.398 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:55.402+0000 7fb84dffb700 1 -- 172.21.15.27:0/261958244 <== mgr.14162 v2:172.21.15.27:6800/3802716887 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fb85c0ff990 con 0x7fb84402feb0 2021-11-03T17:06:55.398 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: Self-signed certificate created 2021-11-03T17:06:55.400 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:55.404+0000 7fb86209c700 1 -- 172.21.15.27:0/261958244 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fb84402feb0 msgr2=0x7fb844032370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:55.400 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:55.404+0000 7fb86209c700 1 --2- 172.21.15.27:0/261958244 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fb84402feb0 0x7fb844032370 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fb8500096a0 tx=0x7fb850006b40).stop 2021-11-03T17:06:55.401 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:55.404+0000 7fb86209c700 1 -- 172.21.15.27:0/261958244 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb85c06fbc0 msgr2=0x7fb85c0fc0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:55.401 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:55.404+0000 7fb86209c700 1 --2- 172.21.15.27:0/261958244 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb85c06fbc0 0x7fb85c0fc0b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fb858011e00 tx=0x7fb858006ee0).stop 2021-11-03T17:06:55.401 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:55.404+0000 7fb86209c700 1 -- 172.21.15.27:0/261958244 shutdown_connections 2021-11-03T17:06:55.402 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:55.404+0000 7fb86209c700 1 --2- 172.21.15.27:0/261958244 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fb84402feb0 0x7fb844032370 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:55.402 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:55.404+0000 7fb86209c700 1 --2- 172.21.15.27:0/261958244 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb85c06fbc0 0x7fb85c0fc0b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:55.402 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:55.404+0000 7fb86209c700 1 -- 172.21.15.27:0/261958244 >> 172.21.15.27:0/261958244 conn(0x7fb85c0f6110 msgr2=0x7fb85c0f7b80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:55.402 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:55.404+0000 7fb86209c700 1 -- 172.21.15.27:0/261958244 shutdown_connections 2021-11-03T17:06:55.403 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:55.404+0000 7fb86209c700 1 -- 172.21.15.27:0/261958244 wait complete. 2021-11-03T17:06:55.672 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:55 smithi027 conmon[32322]: cluster 2021-11-03T17:06:54.526460+0000 mon.smithi027 (mon.0) 108 : cluster [DBG] mgrmap e16: smithi027.ufhhvb(active, since 2s) 2021-11-03T17:06:55.673 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:55 smithi027 conmon[32322]: audit 2021-11-03T17:06:55.400363+0000 mon.smithi027 (mon.0) 109 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:06:55.673 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:55 smithi027 conmon[32322]: audit 2021-11-03T17:06:55.402922+0000 mon.smithi027 (mon.0) 110 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:06:56.061 INFO:teuthology.orchestra.run.smithi027.stderr:Creating initial admin user... 2021-11-03T17:06:56.062 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi027 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmp3f8lj10x:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpq2rb3_h7:/etc/ceph/ceph.conf:z -v /tmp/ceph-tmpd_m0xhhj:/tmp/dashboard.pw:z docker.io/ceph/ceph:v16.2.4 dashboard ac-user-create admin -i /tmp/dashboard.pw administrator --force-password --pwd-update-required 2021-11-03T17:06:56.752 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.753+0000 7f070b434700 1 Processor -- start 2021-11-03T17:06:56.752 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.753+0000 7f070b434700 1 -- start start 2021-11-03T17:06:56.753 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.754+0000 7f070b434700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f070406fb80 0x7f07040fe820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:56.753 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.754+0000 7f070b434700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07040fed10 con 0x7f070406fb80 2021-11-03T17:06:56.753 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.754+0000 7f07091d0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f070406fb80 0x7f07040fe820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:56.753 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.754+0000 7f07091d0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f070406fb80 0x7f07040fe820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54290/0 (socket says 172.21.15.27:54290) 2021-11-03T17:06:56.753 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.754+0000 7f07091d0700 1 -- 172.21.15.27:0/1648982779 learned_addr learned my addr 172.21.15.27:0/1648982779 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:56.754 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.754+0000 7f07091d0700 1 -- 172.21.15.27:0/1648982779 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07040fee50 con 0x7f070406fb80 2021-11-03T17:06:56.754 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.754+0000 7f07091d0700 1 --2- 172.21.15.27:0/1648982779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f070406fb80 0x7f07040fe820 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f070000b5f0 tx=0x7f0700006ee0).ready entity=mon.0 client_cookie=aa32a51e1d24d071 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:56.754 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.754+0000 7f06f3fff700 1 -- 172.21.15.27:0/1648982779 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f070000d600 con 0x7f070406fb80 2021-11-03T17:06:56.754 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.755+0000 7f06f3fff700 1 -- 172.21.15.27:0/1648982779 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f070000dbf0 con 0x7f070406fb80 2021-11-03T17:06:56.755 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.755+0000 7f06f3fff700 1 -- 172.21.15.27:0/1648982779 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0700011c20 con 0x7f070406fb80 2021-11-03T17:06:56.755 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.755+0000 7f070b434700 1 -- 172.21.15.27:0/1648982779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f070406fb80 msgr2=0x7f07040fe820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:56.755 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.755+0000 7f070b434700 1 --2- 172.21.15.27:0/1648982779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f070406fb80 0x7f07040fe820 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f070000b5f0 tx=0x7f0700006ee0).stop 2021-11-03T17:06:56.755 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.755+0000 7f070b434700 1 -- 172.21.15.27:0/1648982779 shutdown_connections 2021-11-03T17:06:56.756 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.755+0000 7f070b434700 1 --2- 172.21.15.27:0/1648982779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f070406fb80 0x7f07040fe820 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:56.756 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.755+0000 7f070b434700 1 -- 172.21.15.27:0/1648982779 >> 172.21.15.27:0/1648982779 conn(0x7f07040f6260 msgr2=0x7f07040f86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:56.756 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.755+0000 7f070b434700 1 -- 172.21.15.27:0/1648982779 shutdown_connections 2021-11-03T17:06:56.756 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.755+0000 7f070b434700 1 -- 172.21.15.27:0/1648982779 wait complete. 2021-11-03T17:06:56.756 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.756+0000 7f070b434700 1 Processor -- start 2021-11-03T17:06:56.757 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.756+0000 7f070b434700 1 -- start start 2021-11-03T17:06:56.757 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.756+0000 7f070b434700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f070406fb80 0x7f07040fc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:56.757 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.756+0000 7f070b434700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07040fa720 con 0x7f070406fb80 2021-11-03T17:06:56.757 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.756+0000 7f07091d0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f070406fb80 0x7f07040fc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:56.758 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.756+0000 7f07091d0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f070406fb80 0x7f07040fc2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54292/0 (socket says 172.21.15.27:54292) 2021-11-03T17:06:56.758 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.756+0000 7f07091d0700 1 -- 172.21.15.27:0/3765699131 learned_addr learned my addr 172.21.15.27:0/3765699131 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:56.758 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.757+0000 7f07091d0700 1 -- 172.21.15.27:0/3765699131 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f070000b200 con 0x7f070406fb80 2021-11-03T17:06:56.758 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.757+0000 7f07091d0700 1 --2- 172.21.15.27:0/3765699131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f070406fb80 0x7f07040fc2a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f0700006800 tx=0x7f0700006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:56.758 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.757+0000 7f06f1ffb700 1 -- 172.21.15.27:0/3765699131 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f070001b570 con 0x7f070406fb80 2021-11-03T17:06:56.759 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.757+0000 7f070b434700 1 -- 172.21.15.27:0/3765699131 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07040fa8c0 con 0x7f070406fb80 2021-11-03T17:06:56.759 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.757+0000 7f06f1ffb700 1 -- 172.21.15.27:0/3765699131 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f070001bb60 con 0x7f070406fb80 2021-11-03T17:06:56.759 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.757+0000 7f06f1ffb700 1 -- 172.21.15.27:0/3765699131 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f070001a890 con 0x7f070406fb80 2021-11-03T17:06:56.760 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.757+0000 7f070b434700 1 -- 172.21.15.27:0/3765699131 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07040fad50 con 0x7f070406fb80 2021-11-03T17:06:56.761 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.758+0000 7f06f1ffb700 1 -- 172.21.15.27:0/3765699131 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) v1 ==== 36208+0+0 (secure 0 0 0) 0x7f070002b070 con 0x7f070406fb80 2021-11-03T17:06:56.761 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.758+0000 7f06f1ffb700 1 --2- 172.21.15.27:0/3765699131 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f06ec02feb0 0x7f06ec032370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:56.761 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.758+0000 7f06f1ffb700 1 -- 172.21.15.27:0/3765699131 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1237+0+0 (secure 0 0 0) 0x7f0700041990 con 0x7f070406fb80 2021-11-03T17:06:56.761 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.758+0000 7f07089cf700 1 --2- 172.21.15.27:0/3765699131 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f06ec02feb0 0x7f06ec032370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:56.762 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.758+0000 7f070b434700 1 -- 172.21.15.27:0/3765699131 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0704059100 con 0x7f070406fb80 2021-11-03T17:06:56.762 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.759+0000 7f07089cf700 1 --2- 172.21.15.27:0/3765699131 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f06ec02feb0 0x7f06ec032370 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f06f40096a0 tx=0x7f06f4006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:56.762 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:56.763+0000 7f06f1ffb700 1 -- 172.21.15.27:0/3765699131 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f070001f070 con 0x7f070406fb80 2021-11-03T17:06:56.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:56 smithi027 conmon[32322]: audit 2021-11-03T17:06:55.229896+0000 mgr.smithi027.ufhhvb 2021-11-03T17:06:56.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:56 smithi027 conmon[32322]: (mgr.14162) 3 : audit [DBG] from='client.14172 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:06:57.073 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:57.078+0000 7f070b434700 1 -- 172.21.15.27:0/3765699131 --> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f0704058f20 con 0x7f06ec02feb0 2021-11-03T17:06:57.319 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:57.324+0000 7f06f1ffb700 1 -- 172.21.15.27:0/3765699131 <== mgr.14162 v2:172.21.15.27:6800/3802716887 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f0704058f20 con 0x7f06ec02feb0 2021-11-03T17:06:57.320 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: {"username": "admin", "password": "$2b$12$cie4qERZk6.06pGn9hsuPeHgL4LW1.uhv8P/x9W8Lr1lzDtjbXk8u", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1635959217, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2021-11-03T17:06:57.322 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:57.326+0000 7f070b434700 1 -- 172.21.15.27:0/3765699131 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f06ec02feb0 msgr2=0x7f06ec032370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:57.322 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:57.326+0000 7f070b434700 1 --2- 172.21.15.27:0/3765699131 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f06ec02feb0 0x7f06ec032370 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f06f40096a0 tx=0x7f06f4006b40).stop 2021-11-03T17:06:57.322 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:57.326+0000 7f070b434700 1 -- 172.21.15.27:0/3765699131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f070406fb80 msgr2=0x7f07040fc2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:57.323 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:57.326+0000 7f070b434700 1 --2- 172.21.15.27:0/3765699131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f070406fb80 0x7f07040fc2a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f0700006800 tx=0x7f0700006ee0).stop 2021-11-03T17:06:57.323 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:57.326+0000 7f070b434700 1 -- 172.21.15.27:0/3765699131 shutdown_connections 2021-11-03T17:06:57.323 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:57.326+0000 7f070b434700 1 --2- 172.21.15.27:0/3765699131 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f06ec02feb0 0x7f06ec032370 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:57.323 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:57.326+0000 7f070b434700 1 --2- 172.21.15.27:0/3765699131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f070406fb80 0x7f07040fc2a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:57.324 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:57.326+0000 7f070b434700 1 -- 172.21.15.27:0/3765699131 >> 172.21.15.27:0/3765699131 conn(0x7f07040f6260 msgr2=0x7f07040f7c80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:57.324 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:57.326+0000 7f070b434700 1 -- 172.21.15.27:0/3765699131 shutdown_connections 2021-11-03T17:06:57.324 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:57.326+0000 7f070b434700 1 -- 172.21.15.27:0/3765699131 wait complete. 2021-11-03T17:06:57.827 INFO:teuthology.orchestra.run.smithi027.stderr:Fetching dashboard port number... 2021-11-03T17:06:57.828 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /bin/podman run --rm --ipc=host --no-hosts --net=host --entrypoint /usr/bin/ceph --init -e CONTAINER_IMAGE=docker.io/ceph/ceph:v16.2.4 -e NODE_NAME=smithi027 -e CEPH_USE_RANDOM_NONCE=1 -v /var/log/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c:/var/log/ceph:z -v /tmp/ceph-tmp3f8lj10x:/etc/ceph/ceph.client.admin.keyring:z -v /tmp/ceph-tmpq2rb3_h7:/etc/ceph/ceph.conf:z docker.io/ceph/ceph:v16.2.4 config get mgr mgr/dashboard/ssl_server_port 2021-11-03T17:06:58.495 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.496+0000 7f19676fd700 1 Processor -- start 2021-11-03T17:06:58.495 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.496+0000 7f19676fd700 1 -- start start 2021-11-03T17:06:58.496 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.496+0000 7f19676fd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19600fc1d0 0x7f19600fc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:58.496 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.496+0000 7f19676fd700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19600fcae0 con 0x7f19600fc1d0 2021-11-03T17:06:58.496 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.496+0000 7f1965499700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19600fc1d0 0x7f19600fc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:58.497 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.497+0000 7f1965499700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19600fc1d0 0x7f19600fc5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54300/0 (socket says 172.21.15.27:54300) 2021-11-03T17:06:58.497 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.497+0000 7f1965499700 1 -- 172.21.15.27:0/549667189 learned_addr learned my addr 172.21.15.27:0/549667189 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:58.497 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.497+0000 7f1965499700 1 -- 172.21.15.27:0/549667189 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19600fcc20 con 0x7f19600fc1d0 2021-11-03T17:06:58.497 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.497+0000 7f1965499700 1 --2- 172.21.15.27:0/549667189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19600fc1d0 0x7f19600fc5f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f1958009960 tx=0x7f19580092d0).ready entity=mon.0 client_cookie=c78426514bd1e38e server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:58.497 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.497+0000 7f195ffff700 1 -- 172.21.15.27:0/549667189 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f195800ba50 con 0x7f19600fc1d0 2021-11-03T17:06:58.498 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.497+0000 7f195ffff700 1 -- 172.21.15.27:0/549667189 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f195800bbb0 con 0x7f19600fc1d0 2021-11-03T17:06:58.498 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.497+0000 7f195ffff700 1 -- 172.21.15.27:0/549667189 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f19580103a0 con 0x7f19600fc1d0 2021-11-03T17:06:58.498 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.498+0000 7f19676fd700 1 -- 172.21.15.27:0/549667189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19600fc1d0 msgr2=0x7f19600fc5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:58.498 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.498+0000 7f19676fd700 1 --2- 172.21.15.27:0/549667189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19600fc1d0 0x7f19600fc5f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f1958009960 tx=0x7f19580092d0).stop 2021-11-03T17:06:58.499 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.498+0000 7f19676fd700 1 -- 172.21.15.27:0/549667189 shutdown_connections 2021-11-03T17:06:58.499 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.498+0000 7f19676fd700 1 --2- 172.21.15.27:0/549667189 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19600fc1d0 0x7f19600fc5f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:58.499 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.498+0000 7f19676fd700 1 -- 172.21.15.27:0/549667189 >> 172.21.15.27:0/549667189 conn(0x7f19600f5a60 msgr2=0x7f19600f7e80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:58.499 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.498+0000 7f19676fd700 1 -- 172.21.15.27:0/549667189 shutdown_connections 2021-11-03T17:06:58.500 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.498+0000 7f19676fd700 1 -- 172.21.15.27:0/549667189 wait complete. 2021-11-03T17:06:58.500 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.498+0000 7f19676fd700 1 Processor -- start 2021-11-03T17:06:58.500 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.498+0000 7f19676fd700 1 -- start start 2021-11-03T17:06:58.500 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.499+0000 7f19676fd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19600fc1d0 0x7f196010cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:58.501 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.499+0000 7f19676fd700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196010d090 con 0x7f19600fc1d0 2021-11-03T17:06:58.501 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.499+0000 7f1965499700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19600fc1d0 0x7f196010cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:58.501 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.499+0000 7f1965499700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19600fc1d0 0x7f196010cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54302/0 (socket says 172.21.15.27:54302) 2021-11-03T17:06:58.501 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.499+0000 7f1965499700 1 -- 172.21.15.27:0/4055157807 learned_addr learned my addr 172.21.15.27:0/4055157807 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:06:58.502 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.499+0000 7f1965499700 1 -- 172.21.15.27:0/4055157807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1958009570 con 0x7f19600fc1d0 2021-11-03T17:06:58.502 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.499+0000 7f1965499700 1 --2- 172.21.15.27:0/4055157807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19600fc1d0 0x7f196010cba0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f195800bf50 tx=0x7f19580092d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:58.502 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.499+0000 7f195dffb700 1 -- 172.21.15.27:0/4055157807 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f19580107e0 con 0x7f19600fc1d0 2021-11-03T17:06:58.502 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.499+0000 7f195dffb700 1 -- 172.21.15.27:0/4055157807 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f1958010dd0 con 0x7f19600fc1d0 2021-11-03T17:06:58.502 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.499+0000 7f19676fd700 1 -- 172.21.15.27:0/4055157807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f196010d290 con 0x7f19600fc1d0 2021-11-03T17:06:58.503 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.499+0000 7f195dffb700 1 -- 172.21.15.27:0/4055157807 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f195801ac70 con 0x7f19600fc1d0 2021-11-03T17:06:58.503 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.500+0000 7f19676fd700 1 -- 172.21.15.27:0/4055157807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f196010d720 con 0x7f19600fc1d0 2021-11-03T17:06:58.503 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.500+0000 7f195dffb700 1 -- 172.21.15.27:0/4055157807 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 16) v1 ==== 36208+0+0 (secure 0 0 0) 0x7f195800f070 con 0x7f19600fc1d0 2021-11-03T17:06:58.504 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.501+0000 7f195dffb700 1 --2- 172.21.15.27:0/4055157807 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f194c02ff00 0x7f194c0323c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:06:58.505 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.501+0000 7f195dffb700 1 -- 172.21.15.27:0/4055157807 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1237+0+0 (secure 0 0 0) 0x7f1958041f70 con 0x7f19600fc1d0 2021-11-03T17:06:58.505 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.501+0000 7f19676fd700 1 -- 172.21.15.27:0/4055157807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1960045ad0 con 0x7f19600fc1d0 2021-11-03T17:06:58.505 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.501+0000 7f1964c98700 1 --2- 172.21.15.27:0/4055157807 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f194c02ff00 0x7f194c0323c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:06:58.506 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.501+0000 7f1964c98700 1 --2- 172.21.15.27:0/4055157807 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f194c02ff00 0x7f194c0323c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f19540096a0 tx=0x7f1954006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:06:58.506 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.505+0000 7f195dffb700 1 -- 172.21.15.27:0/4055157807 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f195801e020 con 0x7f19600fc1d0 2021-11-03T17:06:58.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:58 smithi027 conmon[32322]: audit 2021-11-03T17:06:57.079537+0000 mgr.smithi027.ufhhvb (mgr 2021-11-03T17:06:58.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:58 smithi027 conmon[32322]: .14162) 4 : audit [DBG] from='client.14174 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:06:58.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:58 smithi027 conmon[32322]: audit 2021-11-03T17:06:57.324760+0000 mon.smithi027 (mon.0) 111 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:06:58.788 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.793+0000 7f19676fd700 1 -- 172.21.15.27:0/4055157807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f1960101d80 con 0x7f19600fc1d0 2021-11-03T17:06:58.788 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.793+0000 7f195dffb700 1 -- 172.21.15.27:0/4055157807 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v10) v1 ==== 112+0+5 (secure 0 0 0) 0x7f1958018500 con 0x7f19600fc1d0 2021-11-03T17:06:58.789 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 8443 2021-11-03T17:06:58.791 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.795+0000 7f19676fd700 1 -- 172.21.15.27:0/4055157807 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f194c02ff00 msgr2=0x7f194c0323c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:58.791 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.795+0000 7f19676fd700 1 --2- 172.21.15.27:0/4055157807 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f194c02ff00 0x7f194c0323c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f19540096a0 tx=0x7f1954006b40).stop 2021-11-03T17:06:58.791 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.795+0000 7f19676fd700 1 -- 172.21.15.27:0/4055157807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19600fc1d0 msgr2=0x7f196010cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:06:58.792 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.795+0000 7f19676fd700 1 --2- 172.21.15.27:0/4055157807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19600fc1d0 0x7f196010cba0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f195800bf50 tx=0x7f19580092d0).stop 2021-11-03T17:06:58.792 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.795+0000 7f19676fd700 1 -- 172.21.15.27:0/4055157807 shutdown_connections 2021-11-03T17:06:58.792 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.795+0000 7f19676fd700 1 --2- 172.21.15.27:0/4055157807 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f194c02ff00 0x7f194c0323c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:58.792 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.795+0000 7f19676fd700 1 --2- 172.21.15.27:0/4055157807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f19600fc1d0 0x7f196010cba0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:06:58.793 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.795+0000 7f19676fd700 1 -- 172.21.15.27:0/4055157807 >> 172.21.15.27:0/4055157807 conn(0x7f19600f5a60 msgr2=0x7f19600f7520 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:06:58.793 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.795+0000 7f19676fd700 1 -- 172.21.15.27:0/4055157807 shutdown_connections 2021-11-03T17:06:58.793 INFO:teuthology.orchestra.run.smithi027.stderr:/usr/bin/ceph: 2021-11-03T17:06:58.795+0000 7f19676fd700 1 -- 172.21.15.27:0/4055157807 wait complete. 2021-11-03T17:06:59.444 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: systemctl is-enabled firewalld.service 2021-11-03T17:06:59.453 INFO:teuthology.orchestra.run.smithi027.stderr:systemctl: disabled 2021-11-03T17:06:59.453 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: systemctl is-active firewalld.service 2021-11-03T17:06:59.460 INFO:teuthology.orchestra.run.smithi027.stderr:systemctl: inactive 2021-11-03T17:06:59.461 INFO:teuthology.orchestra.run.smithi027.stderr:firewalld.service is not enabled 2021-11-03T17:06:59.461 INFO:teuthology.orchestra.run.smithi027.stderr:Not possible to open ports <[8443]>. firewalld.service is not available 2021-11-03T17:06:59.464 INFO:teuthology.orchestra.run.smithi027.stderr:Ceph Dashboard is now available at: 2021-11-03T17:06:59.465 INFO:teuthology.orchestra.run.smithi027.stderr: 2021-11-03T17:06:59.465 INFO:teuthology.orchestra.run.smithi027.stderr: URL: https://smithi027.front.sepia.ceph.com:8443/ 2021-11-03T17:06:59.465 INFO:teuthology.orchestra.run.smithi027.stderr: User: admin 2021-11-03T17:06:59.465 INFO:teuthology.orchestra.run.smithi027.stderr: Password: 4q4y81vj72 2021-11-03T17:06:59.466 INFO:teuthology.orchestra.run.smithi027.stderr: 2021-11-03T17:06:59.466 INFO:teuthology.orchestra.run.smithi027.stderr:You can access the Ceph CLI with: 2021-11-03T17:06:59.466 INFO:teuthology.orchestra.run.smithi027.stderr: 2021-11-03T17:06:59.466 INFO:teuthology.orchestra.run.smithi027.stderr: sudo /home/ubuntu/cephtest/cephadm shell --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2021-11-03T17:06:59.467 INFO:teuthology.orchestra.run.smithi027.stderr: 2021-11-03T17:06:59.467 INFO:teuthology.orchestra.run.smithi027.stderr:Please consider enabling telemetry to help improve Ceph: 2021-11-03T17:06:59.467 INFO:teuthology.orchestra.run.smithi027.stderr: 2021-11-03T17:06:59.467 INFO:teuthology.orchestra.run.smithi027.stderr: ceph telemetry on 2021-11-03T17:06:59.467 INFO:teuthology.orchestra.run.smithi027.stderr: 2021-11-03T17:06:59.468 INFO:teuthology.orchestra.run.smithi027.stderr:For more information see: 2021-11-03T17:06:59.468 INFO:teuthology.orchestra.run.smithi027.stderr: 2021-11-03T17:06:59.468 INFO:teuthology.orchestra.run.smithi027.stderr: https://docs.ceph.com/docs/pacific/mgr/telemetry/ 2021-11-03T17:06:59.468 INFO:teuthology.orchestra.run.smithi027.stderr: 2021-11-03T17:06:59.469 INFO:teuthology.orchestra.run.smithi027.stderr:Bootstrap complete. 2021-11-03T17:06:59.469 INFO:teuthology.orchestra.run.smithi027.stderr:Releasing lock 140512085428712 on /run/cephadm/32ab977e-3cc8-11ec-8c28-001a4aab830c.lock 2021-11-03T17:06:59.469 INFO:teuthology.orchestra.run.smithi027.stderr:Lock 140512085428712 released on /run/cephadm/32ab977e-3cc8-11ec-8c28-001a4aab830c.lock 2021-11-03T17:06:59.504 INFO:tasks.cephadm:Fetching config... 2021-11-03T17:06:59.505 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2021-11-03T17:06:59.505 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2021-11-03T17:06:59.560 INFO:tasks.cephadm:Fetching client.admin keyring... 2021-11-03T17:06:59.560 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2021-11-03T17:06:59.561 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2021-11-03T17:06:59.615 INFO:tasks.cephadm:Fetching mon keyring... 2021-11-03T17:06:59.615 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2021-11-03T17:06:59.616 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd if=/var/lib/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c/mon.smithi027/keyring of=/dev/stdout 2021-11-03T17:06:59.679 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:06:59 smithi027 conmon[32322]: audit 2021-11-03T17:06:58.793892+0000 mon.smithi027 (mon.0) 112 : audit [DBG] from='client.? 172.21.15.27:0/4055157807' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2021-11-03T17:06:59.683 INFO:tasks.cephadm:Fetching pub ssh key... 2021-11-03T17:06:59.684 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2021-11-03T17:06:59.684 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2021-11-03T17:06:59.738 INFO:tasks.cephadm:Installing pub ssh key for root users... 2021-11-03T17:06:59.739 DEBUG:teuthology.orchestra.run.smithi027:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDB1JsmfoPcV548a0JNZ2YyxCK9yzt2y9p0rsuzBHbYmRl1XJ3MOKqMKr5HzZggpgtGKfmVCwyPqy2tA7e0IymbWo09P0jgLkfcod2MPsHNCUa5457jRnsGWCaDxdB4WplXtk5FrkEWr8STePgS/tdb24gnd/xoI0gfANvzEUGFD6+y9T8M3mRIZ9WGEuUPfPZj5lk6QXTwQBMlsV291a/OuJWURBWh3tQH5A/n9+Z2NK1ViqKQaEoB5/MVcFjOb9+xtT5vegzPdSsTZxfmBDEGSwX/e+sQtno3lBzcIXDnkhZ4g3u0vmxaLT5eH64zz44Ub84ucPaytxE1qhZ09Ss0gdtMyT7se/wCLBpuK7s/o5u+u35h1YZ4vJs/iHgCn5zK8+Vk8kUKBTrxOwZJTxMeCVOOWVsUk7smDfrnkC6ZlrXDVJ61BMXPOktAVlIjxXjhKR7A12i7K6DWa1HUwcIrjNUmzOP1oAMwxaas/KxxN3VM3BY4qRSywGLnKVOU8P0= ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2021-11-03T17:06:59.814 INFO:teuthology.orchestra.run.smithi027.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDB1JsmfoPcV548a0JNZ2YyxCK9yzt2y9p0rsuzBHbYmRl1XJ3MOKqMKr5HzZggpgtGKfmVCwyPqy2tA7e0IymbWo09P0jgLkfcod2MPsHNCUa5457jRnsGWCaDxdB4WplXtk5FrkEWr8STePgS/tdb24gnd/xoI0gfANvzEUGFD6+y9T8M3mRIZ9WGEuUPfPZj5lk6QXTwQBMlsV291a/OuJWURBWh3tQH5A/n9+Z2NK1ViqKQaEoB5/MVcFjOb9+xtT5vegzPdSsTZxfmBDEGSwX/e+sQtno3lBzcIXDnkhZ4g3u0vmxaLT5eH64zz44Ub84ucPaytxE1qhZ09Ss0gdtMyT7se/wCLBpuK7s/o5u+u35h1YZ4vJs/iHgCn5zK8+Vk8kUKBTrxOwZJTxMeCVOOWVsUk7smDfrnkC6ZlrXDVJ61BMXPOktAVlIjxXjhKR7A12i7K6DWa1HUwcIrjNUmzOP1oAMwxaas/KxxN3VM3BY4qRSywGLnKVOU8P0= ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c 2021-11-03T17:06:59.826 DEBUG:teuthology.orchestra.run.smithi115:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDB1JsmfoPcV548a0JNZ2YyxCK9yzt2y9p0rsuzBHbYmRl1XJ3MOKqMKr5HzZggpgtGKfmVCwyPqy2tA7e0IymbWo09P0jgLkfcod2MPsHNCUa5457jRnsGWCaDxdB4WplXtk5FrkEWr8STePgS/tdb24gnd/xoI0gfANvzEUGFD6+y9T8M3mRIZ9WGEuUPfPZj5lk6QXTwQBMlsV291a/OuJWURBWh3tQH5A/n9+Z2NK1ViqKQaEoB5/MVcFjOb9+xtT5vegzPdSsTZxfmBDEGSwX/e+sQtno3lBzcIXDnkhZ4g3u0vmxaLT5eH64zz44Ub84ucPaytxE1qhZ09Ss0gdtMyT7se/wCLBpuK7s/o5u+u35h1YZ4vJs/iHgCn5zK8+Vk8kUKBTrxOwZJTxMeCVOOWVsUk7smDfrnkC6ZlrXDVJ61BMXPOktAVlIjxXjhKR7A12i7K6DWa1HUwcIrjNUmzOP1oAMwxaas/KxxN3VM3BY4qRSywGLnKVOU8P0= ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2021-11-03T17:06:59.868 INFO:teuthology.orchestra.run.smithi115.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDB1JsmfoPcV548a0JNZ2YyxCK9yzt2y9p0rsuzBHbYmRl1XJ3MOKqMKr5HzZggpgtGKfmVCwyPqy2tA7e0IymbWo09P0jgLkfcod2MPsHNCUa5457jRnsGWCaDxdB4WplXtk5FrkEWr8STePgS/tdb24gnd/xoI0gfANvzEUGFD6+y9T8M3mRIZ9WGEuUPfPZj5lk6QXTwQBMlsV291a/OuJWURBWh3tQH5A/n9+Z2NK1ViqKQaEoB5/MVcFjOb9+xtT5vegzPdSsTZxfmBDEGSwX/e+sQtno3lBzcIXDnkhZ4g3u0vmxaLT5eH64zz44Ub84ucPaytxE1qhZ09Ss0gdtMyT7se/wCLBpuK7s/o5u+u35h1YZ4vJs/iHgCn5zK8+Vk8kUKBTrxOwZJTxMeCVOOWVsUk7smDfrnkC6ZlrXDVJ61BMXPOktAVlIjxXjhKR7A12i7K6DWa1HUwcIrjNUmzOP1oAMwxaas/KxxN3VM3BY4qRSywGLnKVOU8P0= ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c 2021-11-03T17:06:59.881 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph config set mgr mgr/cephadm/allow_ptrace true 2021-11-03T17:07:01.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:01 smithi027 conmon[32322]: cluster 2021-11-03T17:07:00.328004+0000 mon.smithi027 (mon.0) 113 : cluster [DBG] mgrmap e17: smithi027.ufhhvb(active, since 8s) 2021-11-03T17:07:02.049 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.053+0000 7f2c6f59e700 1 -- 172.21.15.27:0/1952805064 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c700f62a0 msgr2=0x7f2c700f66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:07:02.050 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.053+0000 7f2c6f59e700 1 --2- 172.21.15.27:0/1952805064 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c700f62a0 0x7f2c700f66c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f2c60009fb0 tx=0x7f2c60009b30).stop 2021-11-03T17:07:02.050 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.053+0000 7f2c6f59e700 1 -- 172.21.15.27:0/1952805064 shutdown_connections 2021-11-03T17:07:02.050 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.053+0000 7f2c6f59e700 1 --2- 172.21.15.27:0/1952805064 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c700f62a0 0x7f2c700f66c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:07:02.050 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.053+0000 7f2c6f59e700 1 -- 172.21.15.27:0/1952805064 >> 172.21.15.27:0/1952805064 conn(0x7f2c700f1aa0 msgr2=0x7f2c700f3ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:07:02.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.053+0000 7f2c6f59e700 1 -- 172.21.15.27:0/1952805064 shutdown_connections 2021-11-03T17:07:02.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.053+0000 7f2c6f59e700 1 -- 172.21.15.27:0/1952805064 wait complete. 2021-11-03T17:07:02.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.053+0000 7f2c6f59e700 1 Processor -- start 2021-11-03T17:07:02.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.053+0000 7f2c6f59e700 1 -- start start 2021-11-03T17:07:02.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.053+0000 7f2c6f59e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c700f62a0 0x7f2c7006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:07:02.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.053+0000 7f2c6f59e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c70069e20 con 0x7f2c700f62a0 2021-11-03T17:07:02.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.054+0000 7f2c6e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c700f62a0 0x7f2c7006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:07:02.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.054+0000 7f2c6e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c700f62a0 0x7f2c7006b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54308/0 (socket says 172.21.15.27:54308) 2021-11-03T17:07:02.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.054+0000 7f2c6e59c700 1 -- 172.21.15.27:0/121551522 learned_addr learned my addr 172.21.15.27:0/121551522 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:07:02.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.054+0000 7f2c6e59c700 1 -- 172.21.15.27:0/121551522 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c60005040 con 0x7f2c700f62a0 2021-11-03T17:07:02.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.054+0000 7f2c6e59c700 1 --2- 172.21.15.27:0/121551522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c700f62a0 0x7f2c7006b9a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f2c60009f80 tx=0x7f2c6000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:07:02.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.054+0000 7f2c5effd700 1 -- 172.21.15.27:0/121551522 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2c60009320 con 0x7f2c700f62a0 2021-11-03T17:07:02.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.054+0000 7f2c5effd700 1 -- 172.21.15.27:0/121551522 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f2c60013680 con 0x7f2c700f62a0 2021-11-03T17:07:02.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.054+0000 7f2c6f59e700 1 -- 172.21.15.27:0/121551522 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c70069fc0 con 0x7f2c700f62a0 2021-11-03T17:07:02.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.054+0000 7f2c5effd700 1 -- 172.21.15.27:0/121551522 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f2c6001eda0 con 0x7f2c700f62a0 2021-11-03T17:07:02.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.054+0000 7f2c6f59e700 1 -- 172.21.15.27:0/121551522 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c7006a450 con 0x7f2c700f62a0 2021-11-03T17:07:02.055 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.055+0000 7f2c5effd700 1 -- 172.21.15.27:0/121551522 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f2c6001e420 con 0x7f2c700f62a0 2021-11-03T17:07:02.056 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.055+0000 7f2c5effd700 1 --2- 172.21.15.27:0/121551522 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f2c5802ff60 0x7f2c58032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:07:02.056 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.055+0000 7f2c5effd700 1 -- 172.21.15.27:0/121551522 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1237+0+0 (secure 0 0 0) 0x7f2c60041090 con 0x7f2c700f62a0 2021-11-03T17:07:02.056 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.055+0000 7f2c6dd9b700 1 --2- 172.21.15.27:0/121551522 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f2c5802ff60 0x7f2c58032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:07:02.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.055+0000 7f2c6f59e700 1 -- 172.21.15.27:0/121551522 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c70059100 con 0x7f2c700f62a0 2021-11-03T17:07:02.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.056+0000 7f2c6dd9b700 1 --2- 172.21.15.27:0/121551522 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f2c5802ff60 0x7f2c58032420 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f2c640096a0 tx=0x7f2c64006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:07:02.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.060+0000 7f2c5effd700 1 -- 172.21.15.27:0/121551522 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f2c6001e740 con 0x7f2c700f62a0 2021-11-03T17:07:02.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.227+0000 7f2c6f59e700 1 -- 172.21.15.27:0/121551522 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f2c700fbe30 con 0x7f2c700f62a0 2021-11-03T17:07:02.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.230+0000 7f2c5effd700 1 -- 172.21.15.27:0/121551522 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v11)=0 v11) v1 ==== 125+0+0 (secure 0 0 0) 0x7f2c6001cb10 con 0x7f2c700f62a0 2021-11-03T17:07:02.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.233+0000 7f2c6f59e700 1 -- 172.21.15.27:0/121551522 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f2c5802ff60 msgr2=0x7f2c58032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:07:02.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.233+0000 7f2c6f59e700 1 --2- 172.21.15.27:0/121551522 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f2c5802ff60 0x7f2c58032420 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f2c640096a0 tx=0x7f2c64006b40).stop 2021-11-03T17:07:02.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.233+0000 7f2c6f59e700 1 -- 172.21.15.27:0/121551522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c700f62a0 msgr2=0x7f2c7006b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:07:02.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.233+0000 7f2c6f59e700 1 --2- 172.21.15.27:0/121551522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c700f62a0 0x7f2c7006b9a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f2c60009f80 tx=0x7f2c6000b040).stop 2021-11-03T17:07:02.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.233+0000 7f2c6f59e700 1 -- 172.21.15.27:0/121551522 shutdown_connections 2021-11-03T17:07:02.230 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.233+0000 7f2c6f59e700 1 --2- 172.21.15.27:0/121551522 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f2c5802ff60 0x7f2c58032420 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:07:02.230 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.233+0000 7f2c6f59e700 1 --2- 172.21.15.27:0/121551522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c700f62a0 0x7f2c7006b9a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:07:02.230 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.233+0000 7f2c6f59e700 1 -- 172.21.15.27:0/121551522 >> 172.21.15.27:0/121551522 conn(0x7f2c700f1aa0 msgr2=0x7f2c700f2510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:07:02.231 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.233+0000 7f2c6f59e700 1 -- 172.21.15.27:0/121551522 shutdown_connections 2021-11-03T17:07:02.231 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:02.233+0000 7f2c6f59e700 1 -- 172.21.15.27:0/121551522 wait complete. 2021-11-03T17:07:02.871 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi115 2021-11-03T17:07:02.872 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2021-11-03T17:07:02.872 DEBUG:teuthology.orchestra.run.smithi115:> dd of=/etc/ceph/ceph.conf 2021-11-03T17:07:02.890 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2021-11-03T17:07:02.890 DEBUG:teuthology.orchestra.run.smithi115:> dd of=/etc/ceph/ceph.client.admin.keyring 2021-11-03T17:07:02.947 INFO:tasks.cephadm:Adding host smithi115 to orchestrator... 2021-11-03T17:07:02.948 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph orch host add smithi115 2021-11-03T17:07:03.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:03 smithi027 conmon[32322]: audit 2021-11-03T17:07:02.231272+0000 mon.smithi027 (mon.0) 114 : audit [INF] from='client.? 172.21.15.27:0/121551522' entity='client.admin' 2021-11-03T17:07:05.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:05 smithi027 conmon[32322]: audit 2021-11-03T17:07:04.138981+0000 mon.smithi027 (mon.0) 115 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:07:05.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:05 smithi027 conmon[32322]: audit 2021-11-03T17:07:04.442409+0000 mon.smithi027 (mon.0) 116 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:07:05.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:05 smithi027 conmon[32322]: cephadm 2021-11-03T17:07:04.450876+0000 mgr.smithi027.ufhhvb (mgr.14162) 5 : cephadm [INF] Deploying daemon alertmanager.smithi027 on smithi027 2021-11-03T17:07:13.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:13 smithi027 conmon[32322]: audit 2021-11-03T17:07:12.099802 2021-11-03T17:07:13.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:13 smithi027 conmon[32322]: +0000 mon.smithi027 (mon.0) 117 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:07:13.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:13 smithi027 conmon[32322]: audit 2021-11-03T17:07:12.100815+0000 mon.smithi027 (mon.0) 118 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi027", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2021-11-03T17:07:13.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:13 smithi027 conmon[32322]: audit 2021-11-03T17:07:12.102463+0000 mon.smithi027 (mon.0) 119 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi027", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2021-11-03T17:07:13.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:13 smithi027 conmon[32322]: audit 2021-11-03T17:07:12.103086+0000 mon.smithi027 (mon.0) 120 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:07:13.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:13 smithi027 conmon[32322]: cephadm 2021-11-03T17:07:12.103674+0000 mgr.smithi027.ufhhvb (mgr.14162) 6 : cephadm [INF] Deploying daemon crash.smithi027 on smithi027 2021-11-03T17:07:13.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:13 smithi027 conmon[32322]: 2021-11-03T17:07:13.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:13 smithi027 conmon[32322]: cluster 2021-11-03T17:07:12.524744+0000 mgr.smithi027.ufhhvb (mgr.14162) 7 : 2021-11-03T17:07:13.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:13 smithi027 conmon[32322]: cluster [DBG] pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:07:15.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:14 smithi027 conmon[32322]: cluster 2021-11-03T17:07:14.524984+0000 mgr.smithi027.ufhhvb (mgr.14162) 2021-11-03T17:07:15.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:14 smithi027 conmon[32322]: 8 : cluster [DBG] pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:07:15.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:14 smithi027 conmon[32322]: audit 2021-11-03T17:07:14.588527+0000 mon.smithi027 (mon.0) 121 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:07:16.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:15 smithi027 conmon[32322]: audit 2021-11-03T17:07:14.735124+0000 2021-11-03T17:07:16.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:15 smithi027 conmon[32322]: mon.smithi027 (mon.0) 122 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:07:16.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:15 smithi027 conmon[32322]: audit 2021-11-03T17:07:14.739544+0000 mon.smithi027 (mon.0) 123 : 2021-11-03T17:07:16.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:15 smithi027 conmon[32322]: audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:07:16.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:15 smithi027 conmon[32322]: audit 2021-11-03T17:07 2021-11-03T17:07:16.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:15 smithi027 conmon[32322]: :14.739951+0000 mon.smithi027 (mon.0) 124 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2021-11-03T17:07:16.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:15 smithi027 conmon[32322]: audit 2021- 2021-11-03T17:07:16.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:15 smithi027 conmon[32322]: 11-03T17:07:14.740419+0000 mgr.smithi027.ufhhvb ( 2021-11-03T17:07:16.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:15 smithi027 conmon[32322]: mgr.14162) 9 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2021-11-03T17:07:16.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:15 smithi027 conmon[32322]: audit 2021-11-03T17: 2021-11-03T17:07:16.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:15 smithi027 conmon[32322]: 07:14.745491+0000 mon.smithi027 (mon.0) 125 : 2021-11-03T17:07:16.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:15 smithi027 conmon[32322]: audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:07:16.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:15 smithi027 conmon[32322]: cephadm 2021-11-03T17:07:16.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:15 smithi027 conmon[32322]: 2021-11-03T17:07:14.749207+0000 mgr.smithi027.ufhhvb (mgr.14162) 10 2021-11-03T17:07:16.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:15 smithi027 conmon[32322]: : cephadm [INF] Deploying daemon grafana.smithi027 on smithi027 2021-11-03T17:07:17.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:17 smithi027 conmon[32322]: cluster 2021-11-03T17:07:16.525374+0000 mgr.smithi027.ufhhvb (mgr 2021-11-03T17:07:17.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:17 smithi027 conmon[32322]: .14162) 11 : cluster [DBG] pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:07:19.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:18 smithi027 conmon[32322]: cluster 2021-11-03T17:07:18.525582+0000 2021-11-03T17:07:19.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:18 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14162) 12 : cluster [DBG] pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:07:21.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:20 smithi027 conmon[32322]: cluster 2021-11-03T17:07:20.525856+0000 mgr.smithi027.ufhhvb 2021-11-03T17:07:21.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:20 smithi027 conmon[32322]: (mgr.14162) 13 : cluster [DBG] pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:07:23.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:22 smithi027 conmon[32322]: cluster 2021-11-03T17:07:22.526101+0000 mgr.smithi027.ufhhvb (mgr.14162) 14 : cluster [DBG] pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:07:24.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:24 smithi027 conmon[32322]: cluster 2021-11-03T17:07:24.526453+0000 mgr.smithi027.ufhhvb (mgr.14162) 15 : cluster [DBG] pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:07:28.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:27 smithi027 conmon[32322]: cluster 2021-11-03T17:07:26.526890 2021-11-03T17:07:28.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:27 smithi027 conmon[32322]: +0000 mgr.smithi027.ufhhvb (mgr.14162) 16 : cluster [DBG] pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:07:29.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:28 smithi027 conmon[32322]: cluster 2021-11-03T17:07:28.527150+0000 mgr.smithi027.ufhhvb (mgr.14162) 17 : cluster [DBG] pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:07:30.954 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:30 smithi027 conmon[32322]: cluster 2021-11-03T17:07:30. 2021-11-03T17:07:30.954 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:30 smithi027 conmon[32322]: 527396+0000 mgr.smithi027.ufhhvb (mgr.14162) 18 : cluster [DBG] pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:07:32.272 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:32.275+0000 7f0fabfbf700 1 -- 172.21.15.115:0/3952609787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0fa40f5410 msgr2=0x7f0fa40f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:07:32.273 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:32.275+0000 7f0fabfbf700 1 --2- 172.21.15.115:0/3952609787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0fa40f5410 0x7f0fa40f5830 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f0fa0009fb0 tx=0x7f0fa0009b30).stop 2021-11-03T17:07:32.273 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:32.275+0000 7f0fabfbf700 1 -- 172.21.15.115:0/3952609787 shutdown_connections 2021-11-03T17:07:32.273 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:32.275+0000 7f0fabfbf700 1 --2- 172.21.15.115:0/3952609787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0fa40f5410 0x7f0fa40f5830 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:07:32.274 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:32.275+0000 7f0fabfbf700 1 -- 172.21.15.115:0/3952609787 >> 172.21.15.115:0/3952609787 conn(0x7f0fa40f0b70 msgr2=0x7f0fa40f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:07:32.274 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:32.275+0000 7f0fabfbf700 1 -- 172.21.15.115:0/3952609787 shutdown_connections 2021-11-03T17:07:32.274 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:32.275+0000 7f0fabfbf700 1 -- 172.21.15.115:0/3952609787 wait complete. 2021-11-03T17:07:32.274 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:32.275+0000 7f0fabfbf700 1 Processor -- start 2021-11-03T17:07:32.275 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:32.275+0000 7f0fabfbf700 1 -- start start 2021-11-03T17:07:32.275 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:32.276+0000 7f0fabfbf700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0fa40f5410 0x7f0fa4105610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:07:32.275 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:32.276+0000 7f0fabfbf700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fa4105b00 con 0x7f0fa40f5410 2021-11-03T17:07:32.275 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:32.276+0000 7f0fa9d5b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0fa40f5410 0x7f0fa4105610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:07:32.276 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:32.276+0000 7f0fa9d5b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0fa40f5410 0x7f0fa4105610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.115:47356/0 (socket says 172.21.15.115:47356) 2021-11-03T17:07:32.276 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:32.276+0000 7f0fa9d5b700 1 -- 172.21.15.115:0/1133208883 learned_addr learned my addr 172.21.15.115:0/1133208883 (peer_addr_for_me v2:172.21.15.115:0/0) 2021-11-03T17:07:32.276 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:32.276+0000 7f0fa9d5b700 1 -- 172.21.15.115:0/1133208883 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fa0005040 con 0x7f0fa40f5410 2021-11-03T17:07:32.276 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:32.276+0000 7f0fa9d5b700 1 --2- 172.21.15.115:0/1133208883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0fa40f5410 0x7f0fa4105610 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f0fa00089f0 tx=0x7f0fa000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:07:32.277 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:32.276+0000 7f0f9a7fc700 1 -- 172.21.15.115:0/1133208883 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0fa0009320 con 0x7f0fa40f5410 2021-11-03T17:07:32.277 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:32.276+0000 7f0fabfbf700 1 -- 172.21.15.115:0/1133208883 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fa4105d00 con 0x7f0fa40f5410 2021-11-03T17:07:32.277 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:32.276+0000 7f0f9a7fc700 1 -- 172.21.15.115:0/1133208883 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f0fa0013ec0 con 0x7f0fa40f5410 2021-11-03T17:07:32.277 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:32.276+0000 7f0f9a7fc700 1 -- 172.21.15.115:0/1133208883 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0fa0014d30 con 0x7f0fa40f5410 2021-11-03T17:07:32.278 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:32.276+0000 7f0fabfbf700 1 -- 172.21.15.115:0/1133208883 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fa4106190 con 0x7f0fa40f5410 2021-11-03T17:07:32.278 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:32.277+0000 7f0f9a7fc700 1 -- 172.21.15.115:0/1133208883 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f0fa00263a0 con 0x7f0fa40f5410 2021-11-03T17:07:32.278 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:32.277+0000 7f0fabfbf700 1 -- 172.21.15.115:0/1133208883 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fa4045ad0 con 0x7f0fa40f5410 2021-11-03T17:07:32.278 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:32.277+0000 7f0f9a7fc700 1 --2- 172.21.15.115:0/1133208883 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f0f9002ff60 0x7f0f90032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:07:32.279 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:32.277+0000 7f0f9a7fc700 1 -- 172.21.15.115:0/1133208883 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1237+0+0 (secure 0 0 0) 0x7f0fa0040ff0 con 0x7f0fa40f5410 2021-11-03T17:07:32.279 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:32.277+0000 7f0fa955a700 1 --2- 172.21.15.115:0/1133208883 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f0f9002ff60 0x7f0f90032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:07:32.279 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:32.278+0000 7f0fa955a700 1 --2- 172.21.15.115:0/1133208883 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f0f9002ff60 0x7f0f90032420 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f0f940096a0 tx=0x7f0f94006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:07:32.280 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:32.282+0000 7f0f9a7fc700 1 -- 172.21.15.115:0/1133208883 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f0fa001db70 con 0x7f0fa40f5410 2021-11-03T17:07:32.359 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:32 smithi027 conmon[32322]: audit 2021-11-03T17:07:31.327730+0000 2021-11-03T17:07:32.360 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:32 smithi027 conmon[32322]: mon.smithi027 (mon.0) 126 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:07:32.360 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:32 smithi027 conmon[32322]: cephadm 2021-11-03T17:07:31. 2021-11-03T17:07:32.360 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:32 smithi027 conmon[32322]: 329148+0000 mgr.smithi027.ufhhvb (mgr.14162) 19 : cephadm [INF] Deploying daemon node-exporter.smithi027 on smithi027 2021-11-03T17:07:32.447 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:32.452+0000 7f0fabfbf700 1 -- 172.21.15.115:0/1133208883 --> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi115", "target": ["mon-mgr", ""]}) v1 -- 0x7f0fa40f1d70 con 0x7f0f9002ff60 2021-11-03T17:07:33.094 INFO:teuthology.orchestra.run.smithi115.stdout:Added host 'smithi115' 2021-11-03T17:07:33.094 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:33.098+0000 7f0f9a7fc700 1 -- 172.21.15.115:0/1133208883 <== mgr.14162 v2:172.21.15.27:6800/3802716887 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+23 (secure 0 0 0) 0x7f0fa40f1d70 con 0x7f0f9002ff60 2021-11-03T17:07:33.095 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:33.100+0000 7f0fabfbf700 1 -- 172.21.15.115:0/1133208883 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f0f9002ff60 msgr2=0x7f0f90032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:07:33.096 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:33.100+0000 7f0fabfbf700 1 --2- 172.21.15.115:0/1133208883 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f0f9002ff60 0x7f0f90032420 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f0f940096a0 tx=0x7f0f94006b40).stop 2021-11-03T17:07:33.096 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:33.100+0000 7f0fabfbf700 1 -- 172.21.15.115:0/1133208883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0fa40f5410 msgr2=0x7f0fa4105610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:07:33.096 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:33.100+0000 7f0fabfbf700 1 --2- 172.21.15.115:0/1133208883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0fa40f5410 0x7f0fa4105610 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f0fa00089f0 tx=0x7f0fa000b040).stop 2021-11-03T17:07:33.096 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:33.100+0000 7f0fabfbf700 1 -- 172.21.15.115:0/1133208883 shutdown_connections 2021-11-03T17:07:33.097 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:33.100+0000 7f0fabfbf700 1 --2- 172.21.15.115:0/1133208883 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f0f9002ff60 0x7f0f90032420 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:07:33.097 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:33.100+0000 7f0fabfbf700 1 --2- 172.21.15.115:0/1133208883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0fa40f5410 0x7f0fa4105610 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:07:33.097 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:33.100+0000 7f0fabfbf700 1 -- 172.21.15.115:0/1133208883 >> 172.21.15.115:0/1133208883 conn(0x7f0fa40f0b70 msgr2=0x7f0fa40f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:07:33.097 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:33.100+0000 7f0fabfbf700 1 -- 172.21.15.115:0/1133208883 shutdown_connections 2021-11-03T17:07:33.098 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:33.100+0000 7f0fabfbf700 1 -- 172.21.15.115:0/1133208883 wait complete. 2021-11-03T17:07:33.393 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:33 smithi027 conmon[32322]: audit 2021-11-03T17:07:32.453689+0000 2021-11-03T17:07:33.393 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:33 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14162) 20 : audit [DBG] from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi115", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:07:33.394 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:33 smithi027 conmon[32322]: cluster 2021-11-03T17:07: 2021-11-03T17:07:33.394 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:33 smithi027 conmon[32322]: 32.527642+0000 mgr.smithi027.ufhhvb (mgr.14162) 21 : cluster [DBG] pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:07:33.394 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:33 smithi027 conmon[32322]: audit 2021-11 2021-11-03T17:07:33.394 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:33 smithi027 conmon[32322]: -03T17:07:33.097763+0000 mon.smithi027 ( 2021-11-03T17:07:33.395 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:33 smithi027 conmon[32322]: mon.0) 127 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:07:33.732 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph orch host ls --format=json 2021-11-03T17:07:34.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:34 smithi027 conmon[32322]: cephadm 2021-11-03T17:07:33.098098 2021-11-03T17:07:34.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:34 smithi027 conmon[32322]: +0000 mgr.smithi027.ufhhvb (mgr.14162) 22 : cephadm [INF] Added host smithi115 2021-11-03T17:07:35.543 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.546+0000 7fa1e759e700 1 -- 172.21.15.115:0/708397757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1e80f62a0 msgr2=0x7fa1e80f66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:07:35.543 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.546+0000 7fa1e759e700 1 --2- 172.21.15.115:0/708397757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1e80f62a0 0x7fa1e80f66c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fa1d8009fb0 tx=0x7fa1d8009b30).stop 2021-11-03T17:07:35.543 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.546+0000 7fa1e759e700 1 -- 172.21.15.115:0/708397757 shutdown_connections 2021-11-03T17:07:35.544 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.546+0000 7fa1e759e700 1 --2- 172.21.15.115:0/708397757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1e80f62a0 0x7fa1e80f66c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:07:35.544 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.546+0000 7fa1e759e700 1 -- 172.21.15.115:0/708397757 >> 172.21.15.115:0/708397757 conn(0x7fa1e80f1ac0 msgr2=0x7fa1e80f3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:07:35.544 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.546+0000 7fa1e759e700 1 -- 172.21.15.115:0/708397757 shutdown_connections 2021-11-03T17:07:35.544 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.546+0000 7fa1e759e700 1 -- 172.21.15.115:0/708397757 wait complete. 2021-11-03T17:07:35.545 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.547+0000 7fa1e759e700 1 Processor -- start 2021-11-03T17:07:35.545 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.547+0000 7fa1e759e700 1 -- start start 2021-11-03T17:07:35.545 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.547+0000 7fa1e759e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1e80f62a0 0x7fa1e81064e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:07:35.545 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.547+0000 7fa1e759e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e81069d0 con 0x7fa1e80f62a0 2021-11-03T17:07:35.546 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.547+0000 7fa1e659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1e80f62a0 0x7fa1e81064e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:07:35.546 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.547+0000 7fa1e659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1e80f62a0 0x7fa1e81064e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.115:47362/0 (socket says 172.21.15.115:47362) 2021-11-03T17:07:35.546 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.547+0000 7fa1e659c700 1 -- 172.21.15.115:0/1807642961 learned_addr learned my addr 172.21.15.115:0/1807642961 (peer_addr_for_me v2:172.21.15.115:0/0) 2021-11-03T17:07:35.546 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.547+0000 7fa1e659c700 1 -- 172.21.15.115:0/1807642961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d8005040 con 0x7fa1e80f62a0 2021-11-03T17:07:35.547 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.547+0000 7fa1e659c700 1 --2- 172.21.15.115:0/1807642961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1e80f62a0 0x7fa1e81064e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fa1d8009f80 tx=0x7fa1d800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:07:35.547 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.548+0000 7fa1d6ffd700 1 -- 172.21.15.115:0/1807642961 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa1d8009320 con 0x7fa1e80f62a0 2021-11-03T17:07:35.547 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.548+0000 7fa1e759e700 1 -- 172.21.15.115:0/1807642961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e8106bd0 con 0x7fa1e80f62a0 2021-11-03T17:07:35.547 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.548+0000 7fa1d6ffd700 1 -- 172.21.15.115:0/1807642961 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7fa1d8013680 con 0x7fa1e80f62a0 2021-11-03T17:07:35.548 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.548+0000 7fa1d6ffd700 1 -- 172.21.15.115:0/1807642961 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa1d801eda0 con 0x7fa1e80f62a0 2021-11-03T17:07:35.548 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.548+0000 7fa1e759e700 1 -- 172.21.15.115:0/1807642961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1e8107060 con 0x7fa1e80f62a0 2021-11-03T17:07:35.549 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.548+0000 7fa1d6ffd700 1 -- 172.21.15.115:0/1807642961 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fa1d801e420 con 0x7fa1e80f62a0 2021-11-03T17:07:35.549 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.548+0000 7fa1e759e700 1 -- 172.21.15.115:0/1807642961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1e8045ad0 con 0x7fa1e80f62a0 2021-11-03T17:07:35.549 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.549+0000 7fa1d6ffd700 1 --2- 172.21.15.115:0/1807642961 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fa1d002ff60 0x7fa1d0032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:07:35.550 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.549+0000 7fa1d6ffd700 1 -- 172.21.15.115:0/1807642961 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1237+0+0 (secure 0 0 0) 0x7fa1d8040fd0 con 0x7fa1e80f62a0 2021-11-03T17:07:35.550 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.549+0000 7fa1e5d9b700 1 --2- 172.21.15.115:0/1807642961 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fa1d002ff60 0x7fa1d0032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:07:35.550 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.549+0000 7fa1e5d9b700 1 --2- 172.21.15.115:0/1807642961 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fa1d002ff60 0x7fa1d0032420 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fa1dc0096a0 tx=0x7fa1dc006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:07:35.550 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.554+0000 7fa1d6ffd700 1 -- 172.21.15.115:0/1807642961 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fa1d801e740 con 0x7fa1e80f62a0 2021-11-03T17:07:35.605 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:35 smithi027 conmon[32322]: cluster 2021-11-03T17:07:34.527896+0000 mgr.smithi027.ufhhvb (mgr.14162) 23 : cluster 2021-11-03T17:07:35.606 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:35 smithi027 conmon[32322]: [DBG] pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:07:35.606 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:35 smithi027 conmon[32322]: audit 2021-11-03T17:07:35.212427+0000 mon.smithi027 (mon.0) 128 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:07:35.718 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.723+0000 7fa1e759e700 1 -- 172.21.15.115:0/1807642961 --> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa1e80f2c40 con 0x7fa1d002ff60 2021-11-03T17:07:35.719 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.724+0000 7fa1d6ffd700 1 -- 172.21.15.115:0/1807642961 <== mgr.14162 v2:172.21.15.27:6800/3802716887 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+156 (secure 0 0 0) 0x7fa1e80f2c40 con 0x7fa1d002ff60 2021-11-03T17:07:35.719 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:07:35.719 INFO:teuthology.orchestra.run.smithi115.stdout:[{"addr": "172.21.15.27", "hostname": "smithi027", "labels": [], "status": ""}, {"addr": "smithi115", "hostname": "smithi115", "labels": [], "status": ""}] 2021-11-03T17:07:35.720 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.726+0000 7fa1e759e700 1 -- 172.21.15.115:0/1807642961 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fa1d002ff60 msgr2=0x7fa1d0032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:07:35.721 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.726+0000 7fa1e759e700 1 --2- 172.21.15.115:0/1807642961 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fa1d002ff60 0x7fa1d0032420 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fa1dc0096a0 tx=0x7fa1dc006b40).stop 2021-11-03T17:07:35.721 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.726+0000 7fa1e759e700 1 -- 172.21.15.115:0/1807642961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1e80f62a0 msgr2=0x7fa1e81064e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:07:35.721 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.726+0000 7fa1e759e700 1 --2- 172.21.15.115:0/1807642961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1e80f62a0 0x7fa1e81064e0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fa1d8009f80 tx=0x7fa1d800b040).stop 2021-11-03T17:07:35.721 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.726+0000 7fa1e759e700 1 -- 172.21.15.115:0/1807642961 shutdown_connections 2021-11-03T17:07:35.722 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.726+0000 7fa1e759e700 1 --2- 172.21.15.115:0/1807642961 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fa1d002ff60 0x7fa1d0032420 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:07:35.722 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.726+0000 7fa1e759e700 1 --2- 172.21.15.115:0/1807642961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1e80f62a0 0x7fa1e81064e0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:07:35.722 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.726+0000 7fa1e759e700 1 -- 172.21.15.115:0/1807642961 >> 172.21.15.115:0/1807642961 conn(0x7fa1e80f1ac0 msgr2=0x7fa1e80f2530 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:07:35.722 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.726+0000 7fa1e759e700 1 -- 172.21.15.115:0/1807642961 shutdown_connections 2021-11-03T17:07:35.723 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:35.726+0000 7fa1e759e700 1 -- 172.21.15.115:0/1807642961 wait complete. 2021-11-03T17:07:36.300 INFO:tasks.cephadm:Setting crush tunables to default 2021-11-03T17:07:36.301 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph osd crush tunables default 2021-11-03T17:07:36.403 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:36 smithi027 conmon[32322]: cephadm 2021-11-03T17:07:35.219754+0000 2021-11-03T17:07:36.404 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:36 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14162) 24 : cephadm [INF] Deploying daemon prometheus.smithi027 on smithi027 2021-11-03T17:07:37.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:37 smithi027 conmon[32322]: audit 2021-11-03T17:07:35.724450+0000 2021-11-03T17:07:37.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:37 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14162) 25 : audit [DBG] from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2021-11-03T17:07:37.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:37 smithi027 conmon[32322]: cluster 2021-11-03T17:07:36.528195+0000 mgr.smithi027.ufhhvb (mgr.14162) 26 : cluster [DBG] pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:07:38.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:38 smithi027 conmon[32322]: cluster 2021-11-03T17:07:38.528608+0000 mgr.smithi027.ufhhvb (mgr.14162) 27 : cluster 2021-11-03T17:07:38.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:38 smithi027 conmon[32322]: [DBG] pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:07:40.668 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:40 smithi027 conmon[32322]: cluster 2021-11-03T17:07:40.528969 2021-11-03T17:07:40.669 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:40 smithi027 conmon[32322]: +0000 mgr.smithi027.ufhhvb (mgr.14162) 28 : cluster [DBG] pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:07:42.929 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:42 smithi027 conmon[32322]: cluster 2021-11-03T17:07:42.529243+0000 mgr.smithi027.ufhhvb ( 2021-11-03T17:07:42.929 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:42 smithi027 conmon[32322]: mgr.14162) 29 : cluster [DBG] pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:07:43.320 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.305+0000 7f6c985fd700 1 -- 172.21.15.27:0/3969233323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c900f5410 msgr2=0x7f6c900f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:07:43.320 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.305+0000 7f6c985fd700 1 --2- 172.21.15.27:0/3969233323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c900f5410 0x7f6c900f5830 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f6c8c009fb0 tx=0x7f6c8c009b30).stop 2021-11-03T17:07:43.320 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.305+0000 7f6c985fd700 1 -- 172.21.15.27:0/3969233323 shutdown_connections 2021-11-03T17:07:43.321 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.305+0000 7f6c985fd700 1 --2- 172.21.15.27:0/3969233323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c900f5410 0x7f6c900f5830 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:07:43.321 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.305+0000 7f6c985fd700 1 -- 172.21.15.27:0/3969233323 >> 172.21.15.27:0/3969233323 conn(0x7f6c900f0b70 msgr2=0x7f6c900f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:07:43.321 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.305+0000 7f6c985fd700 1 -- 172.21.15.27:0/3969233323 shutdown_connections 2021-11-03T17:07:43.321 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.305+0000 7f6c985fd700 1 -- 172.21.15.27:0/3969233323 wait complete. 2021-11-03T17:07:43.322 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.306+0000 7f6c985fd700 1 Processor -- start 2021-11-03T17:07:43.322 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.306+0000 7f6c985fd700 1 -- start start 2021-11-03T17:07:43.322 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.306+0000 7f6c985fd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c900f5410 0x7f6c90106710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:07:43.322 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.306+0000 7f6c985fd700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c90106c00 con 0x7f6c900f5410 2021-11-03T17:07:43.322 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.306+0000 7f6c96399700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c900f5410 0x7f6c90106710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:07:43.323 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.306+0000 7f6c96399700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c900f5410 0x7f6c90106710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54390/0 (socket says 172.21.15.27:54390) 2021-11-03T17:07:43.323 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.306+0000 7f6c96399700 1 -- 172.21.15.27:0/3968195058 learned_addr learned my addr 172.21.15.27:0/3968195058 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:07:43.323 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.306+0000 7f6c96399700 1 -- 172.21.15.27:0/3968195058 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c8c005040 con 0x7f6c900f5410 2021-11-03T17:07:43.323 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.306+0000 7f6c96399700 1 --2- 172.21.15.27:0/3968195058 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c900f5410 0x7f6c90106710 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f6c8c000c00 tx=0x7f6c8c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:07:43.324 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.306+0000 7f6c86ffd700 1 -- 172.21.15.27:0/3968195058 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6c8c0095f0 con 0x7f6c900f5410 2021-11-03T17:07:43.324 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.306+0000 7f6c86ffd700 1 -- 172.21.15.27:0/3968195058 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f6c8c007e50 con 0x7f6c900f5410 2021-11-03T17:07:43.324 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.307+0000 7f6c985fd700 1 -- 172.21.15.27:0/3968195058 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c90106e00 con 0x7f6c900f5410 2021-11-03T17:07:43.324 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.307+0000 7f6c86ffd700 1 -- 172.21.15.27:0/3968195058 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6c8c014e70 con 0x7f6c900f5410 2021-11-03T17:07:43.324 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.307+0000 7f6c985fd700 1 -- 172.21.15.27:0/3968195058 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c90107290 con 0x7f6c900f5410 2021-11-03T17:07:43.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.307+0000 7f6c86ffd700 1 -- 172.21.15.27:0/3968195058 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f6c8c025070 con 0x7f6c900f5410 2021-11-03T17:07:43.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.308+0000 7f6c985fd700 1 -- 172.21.15.27:0/3968195058 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c90045ad0 con 0x7f6c900f5410 2021-11-03T17:07:43.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.308+0000 7f6c86ffd700 1 --2- 172.21.15.27:0/3968195058 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f6c7c02ffb0 0x7f6c7c032470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:07:43.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.308+0000 7f6c86ffd700 1 -- 172.21.15.27:0/3968195058 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1237+0+0 (secure 0 0 0) 0x7f6c8c0408e0 con 0x7f6c900f5410 2021-11-03T17:07:43.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.308+0000 7f6c95b98700 1 --2- 172.21.15.27:0/3968195058 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f6c7c02ffb0 0x7f6c7c032470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:07:43.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.308+0000 7f6c95b98700 1 --2- 172.21.15.27:0/3968195058 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f6c7c02ffb0 0x7f6c7c032470 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f6c800096a0 tx=0x7f6c80006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:07:43.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.312+0000 7f6c86ffd700 1 -- 172.21.15.27:0/3968195058 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f6c8c00f4e0 con 0x7f6c900f5410 2021-11-03T17:07:43.473 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.479+0000 7f6c985fd700 1 -- 172.21.15.27:0/3968195058 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f6c900faf80 con 0x7f6c900f5410 2021-11-03T17:07:43.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.639+0000 7f6c86ffd700 1 -- 172.21.15.27:0/3968195058 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v5) v1 ==== 124+0+0 (secure 0 0 0) 0x7f6c8c00f300 con 0x7f6c900f5410 2021-11-03T17:07:43.638 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.643+0000 7f6c985fd700 1 -- 172.21.15.27:0/3968195058 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f6c7c02ffb0 msgr2=0x7f6c7c032470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:07:43.639 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.643+0000 7f6c985fd700 1 --2- 172.21.15.27:0/3968195058 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f6c7c02ffb0 0x7f6c7c032470 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f6c800096a0 tx=0x7f6c80006b40).stop 2021-11-03T17:07:43.639 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.643+0000 7f6c985fd700 1 -- 172.21.15.27:0/3968195058 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c900f5410 msgr2=0x7f6c90106710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:07:43.639 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.643+0000 7f6c985fd700 1 --2- 172.21.15.27:0/3968195058 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c900f5410 0x7f6c90106710 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f6c8c000c00 tx=0x7f6c8c00b040).stop 2021-11-03T17:07:43.639 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.644+0000 7f6c985fd700 1 -- 172.21.15.27:0/3968195058 shutdown_connections 2021-11-03T17:07:43.640 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.644+0000 7f6c985fd700 1 --2- 172.21.15.27:0/3968195058 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f6c7c02ffb0 0x7f6c7c032470 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:07:43.640 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.644+0000 7f6c985fd700 1 --2- 172.21.15.27:0/3968195058 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c900f5410 0x7f6c90106710 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:07:43.640 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.644+0000 7f6c985fd700 1 -- 172.21.15.27:0/3968195058 >> 172.21.15.27:0/3968195058 conn(0x7f6c900f0b70 msgr2=0x7f6c900f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:07:43.640 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.644+0000 7f6c985fd700 1 -- 172.21.15.27:0/3968195058 shutdown_connections 2021-11-03T17:07:43.641 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:07:43.644+0000 7f6c985fd700 1 -- 172.21.15.27:0/3968195058 wait complete. 2021-11-03T17:07:43.641 INFO:teuthology.orchestra.run.smithi027.stderr:adjusted tunables profile to default 2021-11-03T17:07:43.762 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:43 smithi027 conmon[32322]: audit 2021-11-03T17:07:43.479751+0000 mon.smithi027 ( 2021-11-03T17:07:43.762 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:43 smithi027 conmon[32322]: mon.0) 129 : audit [INF] from='client.? 172.21.15.27:0/3968195058' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2021-11-03T17:07:45.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:44 smithi027 conmon[32322]: audit 2021-11-03T17:07:43.639322+0000 mon.smithi027 (mon.0) 130 : audit [INF] from='client.? 172.21.15.27:0/3968195058' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2021-11-03T17:07:45.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:44 smithi027 conmon[32322]: cluster 2021-11-03T17:07:43.639454+0000 mon.smithi027 (mon.0) 131 : cluster [DBG] osdmap e5: 0 total, 0 up, 0 in 2021-11-03T17:07:45.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:44 smithi027 conmon[32322]: cluster 2021-11-03 2021-11-03T17:07:45.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:44 smithi027 conmon[32322]: T17:07:44.529597+0000 2021-11-03T17:07:45.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:44 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14162) 30 : 2021-11-03T17:07:45.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:44 smithi027 conmon[32322]: cluster [DBG] pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:07:46.014 INFO:tasks.cephadm:Adding mon.smithi027 on smithi027 2021-11-03T17:07:46.015 INFO:tasks.cephadm:Adding mon.smithi115 on smithi115 2021-11-03T17:07:46.015 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph orch apply mon '2;smithi027:172.21.15.27=smithi027;smithi115:172.21.15.115=smithi115' 2021-11-03T17:07:47.826 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:47.829+0000 7fa6c551d700 1 -- 172.21.15.115:0/2321795215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6c00f5410 msgr2=0x7fa6c00f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:07:47.826 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:47.829+0000 7fa6c551d700 1 --2- 172.21.15.115:0/2321795215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6c00f5410 0x7fa6c00f5830 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fa6b0009960 tx=0x7fa6b0004d10).stop 2021-11-03T17:07:47.827 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:47.829+0000 7fa6c551d700 1 -- 172.21.15.115:0/2321795215 shutdown_connections 2021-11-03T17:07:47.827 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:47.829+0000 7fa6c551d700 1 --2- 172.21.15.115:0/2321795215 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6c00f5410 0x7fa6c00f5830 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:07:47.827 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:47.829+0000 7fa6c551d700 1 -- 172.21.15.115:0/2321795215 >> 172.21.15.115:0/2321795215 conn(0x7fa6c00f0b70 msgr2=0x7fa6c00f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:07:47.827 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:47.829+0000 7fa6c551d700 1 -- 172.21.15.115:0/2321795215 shutdown_connections 2021-11-03T17:07:47.827 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:47.829+0000 7fa6c551d700 1 -- 172.21.15.115:0/2321795215 wait complete. 2021-11-03T17:07:47.828 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:47.830+0000 7fa6c551d700 1 Processor -- start 2021-11-03T17:07:47.828 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:47.830+0000 7fa6c551d700 1 -- start start 2021-11-03T17:07:47.828 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:47.830+0000 7fa6c551d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6c00f5410 0x7fa6c0109410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:07:47.828 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:47.830+0000 7fa6c551d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c0060240 con 0x7fa6c00f5410 2021-11-03T17:07:47.829 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:47.830+0000 7fa6beffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6c00f5410 0x7fa6c0109410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:07:47.829 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:47.830+0000 7fa6beffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6c00f5410 0x7fa6c0109410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.115:47368/0 (socket says 172.21.15.115:47368) 2021-11-03T17:07:47.829 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:47.830+0000 7fa6beffd700 1 -- 172.21.15.115:0/2505859191 learned_addr learned my addr 172.21.15.115:0/2505859191 (peer_addr_for_me v2:172.21.15.115:0/0) 2021-11-03T17:07:47.830 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:47.831+0000 7fa6beffd700 1 -- 172.21.15.115:0/2505859191 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b0009640 con 0x7fa6c00f5410 2021-11-03T17:07:47.830 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:47.831+0000 7fa6beffd700 1 --2- 172.21.15.115:0/2505859191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6c00f5410 0x7fa6c0109410 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fa6b0009610 tx=0x7fa6b0004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:07:47.830 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:47.831+0000 7fa6b77fe700 1 -- 172.21.15.115:0/2505859191 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa6b0018070 con 0x7fa6c00f5410 2021-11-03T17:07:47.830 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:47.831+0000 7fa6b77fe700 1 -- 172.21.15.115:0/2505859191 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7fa6b000b7e0 con 0x7fa6c00f5410 2021-11-03T17:07:47.830 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:47.831+0000 7fa6c551d700 1 -- 172.21.15.115:0/2505859191 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c0060440 con 0x7fa6c00f5410 2021-11-03T17:07:47.831 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:47.831+0000 7fa6b77fe700 1 -- 172.21.15.115:0/2505859191 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fa6b001d420 con 0x7fa6c00f5410 2021-11-03T17:07:47.831 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:47.831+0000 7fa6c551d700 1 -- 172.21.15.115:0/2505859191 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c0109ac0 con 0x7fa6c00f5410 2021-11-03T17:07:47.832 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:47.832+0000 7fa6b77fe700 1 -- 172.21.15.115:0/2505859191 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7fa6b0024060 con 0x7fa6c00f5410 2021-11-03T17:07:47.832 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:47.832+0000 7fa6b77fe700 1 --2- 172.21.15.115:0/2505859191 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fa6ac02ff60 0x7fa6ac032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:07:47.833 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:47.832+0000 7fa6b77fe700 1 -- 172.21.15.115:0/2505859191 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1237+0+0 (secure 0 0 0) 0x7fa6b00411a0 con 0x7fa6c00f5410 2021-11-03T17:07:47.833 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:47.832+0000 7fa6c551d700 1 -- 172.21.15.115:0/2505859191 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6c0045ad0 con 0x7fa6c00f5410 2021-11-03T17:07:47.833 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:47.832+0000 7fa6be7fc700 1 --2- 172.21.15.115:0/2505859191 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fa6ac02ff60 0x7fa6ac032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:07:47.833 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:47.833+0000 7fa6be7fc700 1 --2- 172.21.15.115:0/2505859191 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fa6ac02ff60 0x7fa6ac032420 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fa6a80096a0 tx=0x7fa6a8006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:07:47.834 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:47.837+0000 7fa6b77fe700 1 -- 172.21.15.115:0/2505859191 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fa6b0027440 con 0x7fa6c00f5410 2021-11-03T17:07:47.860 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:47 smithi027 conmon[32322]: cluster 2021-11-03T17:07:46.529832+0000 mgr.smithi027.ufhhvb (mgr 2021-11-03T17:07:47.860 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:47 smithi027 conmon[32322]: .14162) 31 : cluster [DBG] pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:07:48.005 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:48.010+0000 7fa6c551d700 1 -- 172.21.15.115:0/2505859191 --> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi027:172.21.15.27=smithi027;smithi115:172.21.15.115=smithi115", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6c00f1e20 con 0x7fa6ac02ff60 2021-11-03T17:07:48.012 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:48.018+0000 7fa6b77fe700 1 -- 172.21.15.115:0/2505859191 <== mgr.14162 v2:172.21.15.27:6800/3802716887 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fa6c00f1e20 con 0x7fa6ac02ff60 2021-11-03T17:07:48.013 INFO:teuthology.orchestra.run.smithi115.stdout:Scheduled mon update... 2021-11-03T17:07:48.014 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:48.019+0000 7fa6c551d700 1 -- 172.21.15.115:0/2505859191 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fa6ac02ff60 msgr2=0x7fa6ac032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:07:48.014 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:48.019+0000 7fa6c551d700 1 --2- 172.21.15.115:0/2505859191 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fa6ac02ff60 0x7fa6ac032420 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fa6a80096a0 tx=0x7fa6a8006b40).stop 2021-11-03T17:07:48.014 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:48.019+0000 7fa6c551d700 1 -- 172.21.15.115:0/2505859191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6c00f5410 msgr2=0x7fa6c0109410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:07:48.015 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:48.019+0000 7fa6c551d700 1 --2- 172.21.15.115:0/2505859191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6c00f5410 0x7fa6c0109410 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fa6b0009610 tx=0x7fa6b0004d10).stop 2021-11-03T17:07:48.015 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:48.019+0000 7fa6c551d700 1 -- 172.21.15.115:0/2505859191 shutdown_connections 2021-11-03T17:07:48.015 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:48.019+0000 7fa6c551d700 1 --2- 172.21.15.115:0/2505859191 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fa6ac02ff60 0x7fa6ac032420 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:07:48.016 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:48.019+0000 7fa6c551d700 1 --2- 172.21.15.115:0/2505859191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6c00f5410 0x7fa6c0109410 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:07:48.016 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:48.019+0000 7fa6c551d700 1 -- 172.21.15.115:0/2505859191 >> 172.21.15.115:0/2505859191 conn(0x7fa6c00f0b70 msgr2=0x7fa6c00f1660 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:07:48.016 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:48.019+0000 7fa6c551d700 1 -- 172.21.15.115:0/2505859191 shutdown_connections 2021-11-03T17:07:48.016 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:48.019+0000 7fa6c551d700 1 -- 172.21.15.115:0/2505859191 wait complete. 2021-11-03T17:07:48.606 DEBUG:teuthology.orchestra.run.smithi115:mon.smithi115> sudo journalctl -f -n 0 -u ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c@mon.smithi115.service 2021-11-03T17:07:48.609 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2021-11-03T17:07:48.610 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph mon dump -f json 2021-11-03T17:07:48.635 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:-- Logs begin at Wed 2021-11-03 16:55:13 UTC. -- 2021-11-03T17:07:49.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:49 smithi027 conmon[32322]: audit 2021-11-03T17:07:48.011697+0000 mgr.smithi027.ufhhvb (mgr.14162) 32 : audit [DBG] from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi027:172.21.15.27=smithi027;smithi115:172.21.15.115=smithi115", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:07:49.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:49 smithi027 conmon[32322]: cephadm 2021-11-03T17:07:48.013029+0000 mgr.smithi027.ufhhvb (mgr.14162) 33 : cephadm [INF] Saving service mon spec with placement smithi027:172.21.15.27=smithi027;smithi115:172.21.15.115=smithi115;count:2 2021-11-03T17:07:49.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:49 smithi027 conmon[32322]: audit 2021-11-03T17:07:48.018055+0000 mon.smithi027 (mon.0) 132 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:07:49.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:49 smithi027 conmon[32322]: cluster 2021-11-03T17:07:48.530057+0000 mgr.smithi027.ufhhvb (mgr.14162) 34 : cluster [DBG] pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:07:49.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:49 smithi027 conmon[32322]: audit 2021-11-03T17:07:48.694231+0000 mon.smithi027 (mon.0) 133 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:07:49.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:49 smithi027 conmon[32322]: audit 2021-11-03T17:07:48.694811+0000 mon.smithi027 (mon.0) 134 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:07:49.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:49 smithi027 conmon[32322]: audit 2021-11-03T 2021-11-03T17:07:49.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:49 smithi027 conmon[32322]: 17:07:48.960524+0000 mon.smithi027 (mon.0) 135 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:07:50.593 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.596+0000 7ff998ddf700 1 -- 172.21.15.115:0/2616001321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9940f6510 msgr2=0x7ff9940f6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:07:50.594 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.596+0000 7ff998ddf700 1 --2- 172.21.15.115:0/2616001321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9940f6510 0x7ff9940f6930 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7ff984009fb0 tx=0x7ff984009b30).stop 2021-11-03T17:07:50.594 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.596+0000 7ff998ddf700 1 -- 172.21.15.115:0/2616001321 shutdown_connections 2021-11-03T17:07:50.594 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.596+0000 7ff998ddf700 1 --2- 172.21.15.115:0/2616001321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9940f6510 0x7ff9940f6930 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:07:50.594 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.596+0000 7ff998ddf700 1 -- 172.21.15.115:0/2616001321 >> 172.21.15.115:0/2616001321 conn(0x7ff9940f1c70 msgr2=0x7ff9940f4090 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:07:50.595 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.596+0000 7ff998ddf700 1 -- 172.21.15.115:0/2616001321 shutdown_connections 2021-11-03T17:07:50.595 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.596+0000 7ff998ddf700 1 -- 172.21.15.115:0/2616001321 wait complete. 2021-11-03T17:07:50.595 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.596+0000 7ff998ddf700 1 Processor -- start 2021-11-03T17:07:50.595 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.597+0000 7ff998ddf700 1 -- start start 2021-11-03T17:07:50.596 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.597+0000 7ff998ddf700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9940f6510 0x7ff99406b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:07:50.596 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.597+0000 7ff998ddf700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff994069e20 con 0x7ff9940f6510 2021-11-03T17:07:50.596 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.597+0000 7ff99259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9940f6510 0x7ff99406b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:07:50.596 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.597+0000 7ff99259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9940f6510 0x7ff99406b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.115:47374/0 (socket says 172.21.15.115:47374) 2021-11-03T17:07:50.597 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.597+0000 7ff99259c700 1 -- 172.21.15.115:0/3613694668 learned_addr learned my addr 172.21.15.115:0/3613694668 (peer_addr_for_me v2:172.21.15.115:0/0) 2021-11-03T17:07:50.597 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.597+0000 7ff99259c700 1 -- 172.21.15.115:0/3613694668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff984005040 con 0x7ff9940f6510 2021-11-03T17:07:50.597 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.597+0000 7ff99259c700 1 --2- 172.21.15.115:0/3613694668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9940f6510 0x7ff99406b9a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7ff9840089f0 tx=0x7ff98400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:07:50.597 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.597+0000 7ff982ffd700 1 -- 172.21.15.115:0/3613694668 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff984009320 con 0x7ff9940f6510 2021-11-03T17:07:50.598 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.597+0000 7ff998ddf700 1 -- 172.21.15.115:0/3613694668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff994069fc0 con 0x7ff9940f6510 2021-11-03T17:07:50.598 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.597+0000 7ff982ffd700 1 -- 172.21.15.115:0/3613694668 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7ff984013ec0 con 0x7ff9940f6510 2021-11-03T17:07:50.598 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.597+0000 7ff982ffd700 1 -- 172.21.15.115:0/3613694668 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff984014d30 con 0x7ff9940f6510 2021-11-03T17:07:50.598 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.598+0000 7ff998ddf700 1 -- 172.21.15.115:0/3613694668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff99406a450 con 0x7ff9940f6510 2021-11-03T17:07:50.599 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.598+0000 7ff982ffd700 1 -- 172.21.15.115:0/3613694668 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7ff9840095f0 con 0x7ff9940f6510 2021-11-03T17:07:50.599 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.598+0000 7ff998ddf700 1 -- 172.21.15.115:0/3613694668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff994059100 con 0x7ff9940f6510 2021-11-03T17:07:50.599 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.598+0000 7ff982ffd700 1 --2- 172.21.15.115:0/3613694668 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7ff97c02ff60 0x7ff97c032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:07:50.600 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.598+0000 7ff982ffd700 1 -- 172.21.15.115:0/3613694668 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1237+0+0 (secure 0 0 0) 0x7ff984040f90 con 0x7ff9940f6510 2021-11-03T17:07:50.600 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.599+0000 7ff991d9b700 1 --2- 172.21.15.115:0/3613694668 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7ff97c02ff60 0x7ff97c032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:07:50.602 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.599+0000 7ff991d9b700 1 --2- 172.21.15.115:0/3613694668 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7ff97c02ff60 0x7ff97c032420 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7ff9880096a0 tx=0x7ff988006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:07:50.602 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.603+0000 7ff982ffd700 1 -- 172.21.15.115:0/3613694668 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7ff984014420 con 0x7ff9940f6510 2021-11-03T17:07:50.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:50 smithi027 conmon[32322]: audit 2021-11-03T17:07:49.524787+0000 mon.smithi027 (mon.0) 136 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:07:50.817 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.822+0000 7ff998ddf700 1 -- 172.21.15.115:0/3613694668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff9940fbf90 con 0x7ff9940f6510 2021-11-03T17:07:50.818 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.823+0000 7ff982ffd700 1 -- 172.21.15.115:0/3613694668 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7ff98401d980 con 0x7ff9940f6510 2021-11-03T17:07:50.819 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:07:50.819 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":1,"fsid":"32ab977e-3cc8-11ec-8c28-001a4aab830c","modified":"2021-11-03T17:05:43.573486Z","created":"2021-11-03T17:05:43.573486Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi027","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:3300","nonce":0},{"type":"v1","addr":"172.21.15.27:6789","nonce":0}]},"addr":"172.21.15.27:6789/0","public_addr":"172.21.15.27:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2021-11-03T17:07:50.820 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.825+0000 7ff998ddf700 1 -- 172.21.15.115:0/3613694668 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7ff97c02ff60 msgr2=0x7ff97c032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:07:50.820 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.825+0000 7ff998ddf700 1 --2- 172.21.15.115:0/3613694668 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7ff97c02ff60 0x7ff97c032420 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7ff9880096a0 tx=0x7ff988006b40).stop 2021-11-03T17:07:50.820 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.825+0000 7ff998ddf700 1 -- 172.21.15.115:0/3613694668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9940f6510 msgr2=0x7ff99406b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:07:50.821 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.825+0000 7ff998ddf700 1 --2- 172.21.15.115:0/3613694668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9940f6510 0x7ff99406b9a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7ff9840089f0 tx=0x7ff98400b040).stop 2021-11-03T17:07:50.821 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.825+0000 7ff998ddf700 1 -- 172.21.15.115:0/3613694668 shutdown_connections 2021-11-03T17:07:50.821 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.825+0000 7ff998ddf700 1 --2- 172.21.15.115:0/3613694668 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7ff97c02ff60 0x7ff97c032420 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:07:50.821 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.825+0000 7ff998ddf700 1 --2- 172.21.15.115:0/3613694668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9940f6510 0x7ff99406b9a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:07:50.822 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.825+0000 7ff998ddf700 1 -- 172.21.15.115:0/3613694668 >> 172.21.15.115:0/3613694668 conn(0x7ff9940f1c70 msgr2=0x7ff9940f2760 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:07:50.822 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.825+0000 7ff998ddf700 1 -- 172.21.15.115:0/3613694668 shutdown_connections 2021-11-03T17:07:50.822 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:50.825+0000 7ff998ddf700 1 -- 172.21.15.115:0/3613694668 wait complete. 2021-11-03T17:07:50.822 INFO:teuthology.orchestra.run.smithi115.stderr:dumped monmap epoch 1 2021-11-03T17:07:52.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:51 smithi027 conmon[32322]: cluster 2021-11-03T17:07:50.530259+0000 2021-11-03T17:07:52.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:51 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14162) 35 : cluster [DBG] pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:07:52.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:51 smithi027 conmon[32322]: audit 2021-11-03T17:07:50.823669+0000 mon.smithi027 (mon.0) 137 : audit [DBG] from='client.? 172.21.15.115:0/3613694668' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2021-11-03T17:07:53.152 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2021-11-03T17:07:53.153 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph mon dump -f json 2021-11-03T17:07:53.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:53 smithi027 conmon[32322]: audit 2021-11-03T17:07:52.095500+0000 mon.smithi027 (mon.0 2021-11-03T17:07:53.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:53 smithi027 conmon[32322]: ) 138 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:07:53.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:53 smithi027 conmon[32322]: cluster 2021-11-03T17:07:52.530638+0000 mgr.smithi027.ufhhvb (mgr.14162) 36 : cluster [DBG] pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:07:53.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:53 smithi027 conmon[32322]: audit 2021-11-03T17: 2021-11-03T17:07:53.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:53 smithi027 conmon[32322]: 07:52.712308+0000 mon.smithi027 (mon.0) 139 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:07:53.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:53 smithi027 conmon[32322]: audit 2021-11-03T17:07:52.715867+0000 mon.smithi027 ( 2021-11-03T17:07:53.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:53 smithi027 conmon[32322]: mon.0) 140 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:07:54.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:54 smithi027 conmon[32322]: audit 2021-11-03T17:07:53.220593+0000 mon.smithi027 (mon.0) 2021-11-03T17:07:54.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:54 smithi027 conmon[32322]: 141 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:07:54.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:54 smithi027 conmon[32322]: audit 2021-11-03T17:07:53.520545+0000 mon.smithi027 (mon.0) 142 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:07:54.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:54 smithi027 conmon[32322]: audit 2021-11-03T 2021-11-03T17:07:54.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:54 smithi027 conmon[32322]: 17:07:53.523473+0000 mon.smithi027 (mon.0) 143 : audit 2021-11-03T17:07:54.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:54 smithi027 conmon[32322]: [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:07:54.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:54 smithi027 conmon[32322]: audit 2021-11-03T17:07:54.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:54 smithi027 conmon[32322]: 2021-11-03T17:07:53.524861 2021-11-03T17:07:54.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:54 smithi027 conmon[32322]: +0000 mon.smithi027 (mon.0) 144 : audit [INF] 2021-11-03T17:07:54.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:54 smithi027 conmon[32322]: from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi115", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2021-11-03T17:07:54.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:54 smithi027 conmon[32322]: audit 2021-11-03T17:07 2021-11-03T17:07:54.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:54 smithi027 conmon[32322]: :53.526953+0000 mon.smithi027 (mon.0) 145 : audit 2021-11-03T17:07:54.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:54 smithi027 conmon[32322]: [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi115", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2021-11-03T17:07:54.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:54 smithi027 conmon[32322]: audit 2021-11-03T17:07:54.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:54 smithi027 conmon[32322]: 2021-11-03T17:07:53.527868+0000 2021-11-03T17:07:54.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:54 smithi027 conmon[32322]: mon.smithi027 (mon.0) 146 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:07:54.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:54 smithi027 conmon[32322]: cephadm 2021-11-03T17:07:54.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:54 smithi027 conmon[32322]: 2021-11-03T17:07:53.528977+0000 mgr.smithi027.ufhhvb ( 2021-11-03T17:07:54.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:54 smithi027 conmon[32322]: mgr.14162) 37 : cephadm [INF] Deploying daemon crash.smithi115 on smithi115 2021-11-03T17:07:55.048 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.051+0000 7f855bfff700 1 -- 172.21.15.115:0/2911579593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f855c0f51a0 msgr2=0x7f855c0f55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:07:55.049 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.051+0000 7f855bfff700 1 --2- 172.21.15.115:0/2911579593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f855c0f51a0 0x7f855c0f55c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f854c009fb0 tx=0x7f854c009b30).stop 2021-11-03T17:07:55.049 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.051+0000 7f855bfff700 1 -- 172.21.15.115:0/2911579593 shutdown_connections 2021-11-03T17:07:55.050 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.051+0000 7f855bfff700 1 --2- 172.21.15.115:0/2911579593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f855c0f51a0 0x7f855c0f55c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:07:55.050 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.051+0000 7f855bfff700 1 -- 172.21.15.115:0/2911579593 >> 172.21.15.115:0/2911579593 conn(0x7f855c0f09a0 msgr2=0x7f855c0f2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:07:55.050 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.051+0000 7f855bfff700 1 -- 172.21.15.115:0/2911579593 shutdown_connections 2021-11-03T17:07:55.050 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.051+0000 7f855bfff700 1 -- 172.21.15.115:0/2911579593 wait complete. 2021-11-03T17:07:55.050 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.051+0000 7f855bfff700 1 Processor -- start 2021-11-03T17:07:55.051 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.051+0000 7f855bfff700 1 -- start start 2021-11-03T17:07:55.051 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.052+0000 7f855bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f855c0f51a0 0x7f855c106490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:07:55.051 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.052+0000 7f855bfff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855c106980 con 0x7f855c0f51a0 2021-11-03T17:07:55.051 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.052+0000 7f855affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f855c0f51a0 0x7f855c106490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:07:55.052 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.052+0000 7f855affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f855c0f51a0 0x7f855c106490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.115:47380/0 (socket says 172.21.15.115:47380) 2021-11-03T17:07:55.052 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.052+0000 7f855affd700 1 -- 172.21.15.115:0/2525465379 learned_addr learned my addr 172.21.15.115:0/2525465379 (peer_addr_for_me v2:172.21.15.115:0/0) 2021-11-03T17:07:55.052 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.052+0000 7f855affd700 1 -- 172.21.15.115:0/2525465379 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f854c005040 con 0x7f855c0f51a0 2021-11-03T17:07:55.053 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.052+0000 7f855affd700 1 --2- 172.21.15.115:0/2525465379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f855c0f51a0 0x7f855c106490 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f854c009f80 tx=0x7f854c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:07:55.053 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.052+0000 7f85437fe700 1 -- 172.21.15.115:0/2525465379 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f854c009320 con 0x7f855c0f51a0 2021-11-03T17:07:55.053 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.052+0000 7f85437fe700 1 -- 172.21.15.115:0/2525465379 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f854c013680 con 0x7f855c0f51a0 2021-11-03T17:07:55.053 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.052+0000 7f855bfff700 1 -- 172.21.15.115:0/2525465379 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f855c106b80 con 0x7f855c0f51a0 2021-11-03T17:07:55.054 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.052+0000 7f85437fe700 1 -- 172.21.15.115:0/2525465379 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f854c01eda0 con 0x7f855c0f51a0 2021-11-03T17:07:55.054 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.052+0000 7f855bfff700 1 -- 172.21.15.115:0/2525465379 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f855c107010 con 0x7f855c0f51a0 2021-11-03T17:07:55.055 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.053+0000 7f85437fe700 1 -- 172.21.15.115:0/2525465379 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f854c014420 con 0x7f855c0f51a0 2021-11-03T17:07:55.055 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.053+0000 7f855bfff700 1 -- 172.21.15.115:0/2525465379 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f855c045ad0 con 0x7f855c0f51a0 2021-11-03T17:07:55.056 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.053+0000 7f85437fe700 1 --2- 172.21.15.115:0/2525465379 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f854402ffb0 0x7f8544032470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:07:55.056 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.053+0000 7f85437fe700 1 -- 172.21.15.115:0/2525465379 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1237+0+0 (secure 0 0 0) 0x7f854c041210 con 0x7f855c0f51a0 2021-11-03T17:07:55.056 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.054+0000 7f855a7fc700 1 --2- 172.21.15.115:0/2525465379 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f854402ffb0 0x7f8544032470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:07:55.056 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.054+0000 7f855a7fc700 1 --2- 172.21.15.115:0/2525465379 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f854402ffb0 0x7f8544032470 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f85500096a0 tx=0x7f8550006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:07:55.056 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.058+0000 7f85437fe700 1 -- 172.21.15.115:0/2525465379 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f854c00faf0 con 0x7f855c0f51a0 2021-11-03T17:07:55.279 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.281+0000 7f855bfff700 1 -- 172.21.15.115:0/2525465379 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f855c107ec0 con 0x7f855c0f51a0 2021-11-03T17:07:55.280 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.282+0000 7f85437fe700 1 -- 172.21.15.115:0/2525465379 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f854c00faf0 con 0x7f855c0f51a0 2021-11-03T17:07:55.281 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:07:55.281 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":1,"fsid":"32ab977e-3cc8-11ec-8c28-001a4aab830c","modified":"2021-11-03T17:05:43.573486Z","created":"2021-11-03T17:05:43.573486Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi027","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:3300","nonce":0},{"type":"v1","addr":"172.21.15.27:6789","nonce":0}]},"addr":"172.21.15.27:6789/0","public_addr":"172.21.15.27:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2021-11-03T17:07:55.282 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.284+0000 7f855bfff700 1 -- 172.21.15.115:0/2525465379 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f854402ffb0 msgr2=0x7f8544032470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:07:55.283 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.284+0000 7f855bfff700 1 --2- 172.21.15.115:0/2525465379 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f854402ffb0 0x7f8544032470 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f85500096a0 tx=0x7f8550006b40).stop 2021-11-03T17:07:55.283 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.284+0000 7f855bfff700 1 -- 172.21.15.115:0/2525465379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f855c0f51a0 msgr2=0x7f855c106490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:07:55.283 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.284+0000 7f855bfff700 1 --2- 172.21.15.115:0/2525465379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f855c0f51a0 0x7f855c106490 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f854c009f80 tx=0x7f854c00b040).stop 2021-11-03T17:07:55.284 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.284+0000 7f855bfff700 1 -- 172.21.15.115:0/2525465379 shutdown_connections 2021-11-03T17:07:55.284 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.284+0000 7f855bfff700 1 --2- 172.21.15.115:0/2525465379 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f854402ffb0 0x7f8544032470 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:07:55.284 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.284+0000 7f855bfff700 1 --2- 172.21.15.115:0/2525465379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f855c0f51a0 0x7f855c106490 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:07:55.284 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.284+0000 7f855bfff700 1 -- 172.21.15.115:0/2525465379 >> 172.21.15.115:0/2525465379 conn(0x7f855c0f09a0 msgr2=0x7f855c05fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:07:55.285 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.284+0000 7f855bfff700 1 -- 172.21.15.115:0/2525465379 shutdown_connections 2021-11-03T17:07:55.285 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:55.284+0000 7f855bfff700 1 -- 172.21.15.115:0/2525465379 wait complete. 2021-11-03T17:07:55.285 INFO:teuthology.orchestra.run.smithi115.stderr:dumped monmap epoch 1 2021-11-03T17:07:55.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:55 smithi027 conmon[32322]: cluster 2021-11-03T17:07:54.530966+0000 mgr.smithi027.ufhhvb ( 2021-11-03T17:07:55.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:55 smithi027 conmon[32322]: mgr.14162) 38 : cluster [DBG] pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:07:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:56 smithi027 conmon[32322]: audit 2021-11-03T17:07:55.283708+0000 mon.smithi027 (mon 2021-11-03T17:07:56.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:56 smithi027 conmon[32322]: .0) 147 : audit [DBG] from='client.? 172.21.15.115:0/2525465379' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2021-11-03T17:07:57.024 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2021-11-03T17:07:57.025 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph mon dump -f json 2021-11-03T17:07:58.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:57 smithi027 conmon[32322]: cluster 2021-11-03T17:07:56.531360+0000 mgr.smithi027.ufhhvb (mgr.14162) 39 : cluster 2021-11-03T17:07:58.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:57 smithi027 conmon[32322]: [DBG] pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:07:58.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:57 smithi027 conmon[32322]: audit 2021-11-03T17:07:57.038731+0000 mon.smithi027 (mon.0) 148 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:07:58.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:57 smithi027 conmon[32322]: audit 2021-11-03T17:07:57.040318+0000 mon.smithi027 (mon.0) 149 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi115.tcyodz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2021-11-03T17:07:58.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:57 smithi027 conmon[32322]: audit 2021-11-03T17:07:58.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:57 smithi027 conmon[32322]: 2021-11-03T17:07:57.045042+0000 mon.smithi027 (mon.0) 150 : 2021-11-03T17:07:58.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:57 smithi027 conmon[32322]: audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi115.tcyodz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2021-11-03T17:07:58.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:57 smithi027 conmon[32322]: audit 2021-11- 2021-11-03T17:07:58.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:57 smithi027 conmon[32322]: 03T17:07:57.045953+0000 mon.smithi027 (mon.0) 2021-11-03T17:07:58.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:57 smithi027 conmon[32322]: 151 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mgr services"}]: dispatch 2021-11-03T17:07:58.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:57 smithi027 conmon[32322]: audit 2021-11-03 2021-11-03T17:07:58.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:57 smithi027 conmon[32322]: T17:07:57.047107+0000 mon.smithi027 (mon.0) 152 2021-11-03T17:07:58.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:57 smithi027 conmon[32322]: : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:07:59.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:58 smithi027 conmon[32322]: cephadm 2021-11-03T17:07:57.048139+0000 mgr.smithi027.ufhhvb (mgr.14162 2021-11-03T17:07:59.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:58 smithi027 conmon[32322]: ) 40 : cephadm [INF] Deploying daemon mgr.smithi115.tcyodz on smithi115 2021-11-03T17:07:59.320 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.323+0000 7f7ff9aae700 1 -- 172.21.15.115:0/4066685670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ff40f51a0 msgr2=0x7f7ff40f55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:07:59.320 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.323+0000 7f7ff9aae700 1 --2- 172.21.15.115:0/4066685670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ff40f51a0 0x7f7ff40f55c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f7fe8009fb0 tx=0x7f7fe8009b30).stop 2021-11-03T17:07:59.320 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.323+0000 7f7ff9aae700 1 -- 172.21.15.115:0/4066685670 shutdown_connections 2021-11-03T17:07:59.321 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.323+0000 7f7ff9aae700 1 --2- 172.21.15.115:0/4066685670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ff40f51a0 0x7f7ff40f55c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:07:59.321 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.323+0000 7f7ff9aae700 1 -- 172.21.15.115:0/4066685670 >> 172.21.15.115:0/4066685670 conn(0x7f7ff40f09a0 msgr2=0x7f7ff40f2dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:07:59.321 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.323+0000 7f7ff9aae700 1 -- 172.21.15.115:0/4066685670 shutdown_connections 2021-11-03T17:07:59.321 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.323+0000 7f7ff9aae700 1 -- 172.21.15.115:0/4066685670 wait complete. 2021-11-03T17:07:59.322 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.323+0000 7f7ff9aae700 1 Processor -- start 2021-11-03T17:07:59.322 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.324+0000 7f7ff9aae700 1 -- start start 2021-11-03T17:07:59.322 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.324+0000 7f7ff9aae700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ff40f51a0 0x7f7ff40601b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:07:59.322 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.324+0000 7f7ff9aae700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff41091e0 con 0x7f7ff40f51a0 2021-11-03T17:07:59.324 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.324+0000 7f7ff8aac700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ff40f51a0 0x7f7ff40601b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:07:59.324 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.324+0000 7f7ff8aac700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ff40f51a0 0x7f7ff40601b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.115:47386/0 (socket says 172.21.15.115:47386) 2021-11-03T17:07:59.325 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.324+0000 7f7ff8aac700 1 -- 172.21.15.115:0/215147358 learned_addr learned my addr 172.21.15.115:0/215147358 (peer_addr_for_me v2:172.21.15.115:0/0) 2021-11-03T17:07:59.325 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.325+0000 7f7ff8aac700 1 -- 172.21.15.115:0/215147358 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe8005040 con 0x7f7ff40f51a0 2021-11-03T17:07:59.325 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.325+0000 7f7ff8aac700 1 --2- 172.21.15.115:0/215147358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ff40f51a0 0x7f7ff40601b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f7fe8009f80 tx=0x7f7fe800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:07:59.326 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.325+0000 7f7ff17fa700 1 -- 172.21.15.115:0/215147358 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7fe8009320 con 0x7f7ff40f51a0 2021-11-03T17:07:59.326 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.325+0000 7f7ff17fa700 1 -- 172.21.15.115:0/215147358 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f7fe8013680 con 0x7f7ff40f51a0 2021-11-03T17:07:59.326 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.325+0000 7f7ff17fa700 1 -- 172.21.15.115:0/215147358 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7fe801eda0 con 0x7f7ff40f51a0 2021-11-03T17:07:59.326 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.325+0000 7f7ff9aae700 1 -- 172.21.15.115:0/215147358 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ff4109320 con 0x7f7ff40f51a0 2021-11-03T17:07:59.326 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.325+0000 7f7ff9aae700 1 -- 172.21.15.115:0/215147358 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ff4109810 con 0x7f7ff40f51a0 2021-11-03T17:07:59.327 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.326+0000 7f7ff17fa700 1 -- 172.21.15.115:0/215147358 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f7fe801e420 con 0x7f7ff40f51a0 2021-11-03T17:07:59.327 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.326+0000 7f7ff17fa700 1 --2- 172.21.15.115:0/215147358 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f7fe402ff60 0x7f7fe4032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:07:59.327 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.326+0000 7f7ff9aae700 1 -- 172.21.15.115:0/215147358 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ff4045ad0 con 0x7f7ff40f51a0 2021-11-03T17:07:59.327 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.326+0000 7f7ff17fa700 1 -- 172.21.15.115:0/215147358 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1237+0+0 (secure 0 0 0) 0x7f7fe8041070 con 0x7f7ff40f51a0 2021-11-03T17:07:59.328 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.327+0000 7f7ff3fff700 1 --2- 172.21.15.115:0/215147358 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f7fe402ff60 0x7f7fe4032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:07:59.328 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.327+0000 7f7ff3fff700 1 --2- 172.21.15.115:0/215147358 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f7fe402ff60 0x7f7fe4032420 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f7fe00096a0 tx=0x7f7fe0006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:07:59.328 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.331+0000 7f7ff17fa700 1 -- 172.21.15.115:0/215147358 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f7fe801e780 con 0x7f7ff40f51a0 2021-11-03T17:07:59.552 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.556+0000 7f7ff9aae700 1 -- 172.21.15.115:0/215147358 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7ff410a040 con 0x7f7ff40f51a0 2021-11-03T17:07:59.553 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.556+0000 7f7ff17fa700 1 -- 172.21.15.115:0/215147358 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f7fe801cb10 con 0x7f7ff40f51a0 2021-11-03T17:07:59.554 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:07:59.554 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":1,"fsid":"32ab977e-3cc8-11ec-8c28-001a4aab830c","modified":"2021-11-03T17:05:43.573486Z","created":"2021-11-03T17:05:43.573486Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi027","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:3300","nonce":0},{"type":"v1","addr":"172.21.15.27:6789","nonce":0}]},"addr":"172.21.15.27:6789/0","public_addr":"172.21.15.27:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2021-11-03T17:07:59.555 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.558+0000 7f7ff9aae700 1 -- 172.21.15.115:0/215147358 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f7fe402ff60 msgr2=0x7f7fe4032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:07:59.555 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.558+0000 7f7ff9aae700 1 --2- 172.21.15.115:0/215147358 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f7fe402ff60 0x7f7fe4032420 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f7fe00096a0 tx=0x7f7fe0006b40).stop 2021-11-03T17:07:59.555 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.558+0000 7f7ff9aae700 1 -- 172.21.15.115:0/215147358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ff40f51a0 msgr2=0x7f7ff40601b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:07:59.556 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.558+0000 7f7ff9aae700 1 --2- 172.21.15.115:0/215147358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ff40f51a0 0x7f7ff40601b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f7fe8009f80 tx=0x7f7fe800b040).stop 2021-11-03T17:07:59.556 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.558+0000 7f7ff9aae700 1 -- 172.21.15.115:0/215147358 shutdown_connections 2021-11-03T17:07:59.556 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.558+0000 7f7ff9aae700 1 --2- 172.21.15.115:0/215147358 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f7fe402ff60 0x7f7fe4032420 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:07:59.556 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.558+0000 7f7ff9aae700 1 --2- 172.21.15.115:0/215147358 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ff40f51a0 0x7f7ff40601b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:07:59.557 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.558+0000 7f7ff9aae700 1 -- 172.21.15.115:0/215147358 >> 172.21.15.115:0/215147358 conn(0x7f7ff40f09a0 msgr2=0x7f7ff40f13e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:07:59.557 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.558+0000 7f7ff9aae700 1 -- 172.21.15.115:0/215147358 shutdown_connections 2021-11-03T17:07:59.557 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:07:59.558+0000 7f7ff9aae700 1 -- 172.21.15.115:0/215147358 wait complete. 2021-11-03T17:07:59.557 INFO:teuthology.orchestra.run.smithi115.stderr:dumped monmap epoch 1 2021-11-03T17:08:00.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:07:59 smithi027 conmon[32322]: cluster 2021-11-03T17:07:58.531730+0000 mgr.smithi027.ufhhvb (mgr.14162) 41 : cluster [DBG] pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:01.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:00 smithi027 conmon[32322]: audit 2021-11-03T17:07:59.558053+0000 mon.smithi027 (mon.0 2021-11-03T17:08:01.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:00 smithi027 conmon[32322]: ) 153 : audit [DBG] from='client.? 172.21.15.115:0/215147358' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2021-11-03T17:08:01.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:00 smithi027 conmon[32322]: audit 2021-11-03T17:08:00.104467+0000 mon.smithi027 (mon.0) 154 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:01.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:00 smithi027 conmon[32322]: audit 2021-11-03T17:08:00.106103+0000 mon.smithi027 (mon.0) 155 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-11-03T17:08:01.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:00 smithi027 conmon[32322]: audit 2021-11-03T17:08:00.107079+0000 mon.smithi027 (mon.0) 156 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:08:01.232 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2021-11-03T17:08:01.233 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph mon dump -f json 2021-11-03T17:08:02.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:01 smithi027 conmon[32322]: cephadm 2021-11-03T17:08:00.108004+0000 mgr.smithi027.ufhhvb (mgr.14162) 42 : cephadm [INF] Deploying daemon mon.smithi115 on smithi115 2021-11-03T17:08:02.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:01 smithi027 conmon[32322]: cluster 2021-11-03T17:08:00.532070+0000 mgr.smithi027.ufhhvb (mgr.14162) 43 : cluster [DBG] pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:03.687 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.689+0000 7f51b57c2700 1 -- 172.21.15.115:0/955721852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51b00f6a80 msgr2=0x7f51b00f6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:08:03.687 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.689+0000 7f51b57c2700 1 --2- 172.21.15.115:0/955721852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51b00f6a80 0x7f51b00f6ea0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f51a0009fb0 tx=0x7f51a0009b30).stop 2021-11-03T17:08:03.687 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.689+0000 7f51b57c2700 1 -- 172.21.15.115:0/955721852 shutdown_connections 2021-11-03T17:08:03.688 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.689+0000 7f51b57c2700 1 --2- 172.21.15.115:0/955721852 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51b00f6a80 0x7f51b00f6ea0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:08:03.688 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.689+0000 7f51b57c2700 1 -- 172.21.15.115:0/955721852 >> 172.21.15.115:0/955721852 conn(0x7f51b00f02e0 msgr2=0x7f51b00f2700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:08:03.688 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.690+0000 7f51b57c2700 1 -- 172.21.15.115:0/955721852 shutdown_connections 2021-11-03T17:08:03.688 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.690+0000 7f51b57c2700 1 -- 172.21.15.115:0/955721852 wait complete. 2021-11-03T17:08:03.689 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.690+0000 7f51b57c2700 1 Processor -- start 2021-11-03T17:08:03.689 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.690+0000 7f51b57c2700 1 -- start start 2021-11-03T17:08:03.689 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.691+0000 7f51b57c2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51b00f6a80 0x7f51b0105570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:08:03.689 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.691+0000 7f51b57c2700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b0105a60 con 0x7f51b00f6a80 2021-11-03T17:08:03.689 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.691+0000 7f51aeffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51b00f6a80 0x7f51b0105570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:08:03.690 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.691+0000 7f51aeffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51b00f6a80 0x7f51b0105570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.115:47396/0 (socket says 172.21.15.115:47396) 2021-11-03T17:08:03.690 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.691+0000 7f51aeffd700 1 -- 172.21.15.115:0/3002360860 learned_addr learned my addr 172.21.15.115:0/3002360860 (peer_addr_for_me v2:172.21.15.115:0/0) 2021-11-03T17:08:03.690 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.691+0000 7f51aeffd700 1 -- 172.21.15.115:0/3002360860 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a0005040 con 0x7f51b00f6a80 2021-11-03T17:08:03.692 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.691+0000 7f51aeffd700 1 --2- 172.21.15.115:0/3002360860 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51b00f6a80 0x7f51b0105570 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f51a0009f80 tx=0x7f51a000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:08:03.692 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.692+0000 7f51977fe700 1 -- 172.21.15.115:0/3002360860 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f51a0009320 con 0x7f51b00f6a80 2021-11-03T17:08:03.692 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.692+0000 7f51977fe700 1 -- 172.21.15.115:0/3002360860 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f51a0013680 con 0x7f51b00f6a80 2021-11-03T17:08:03.692 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.692+0000 7f51b57c2700 1 -- 172.21.15.115:0/3002360860 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b0105c60 con 0x7f51b00f6a80 2021-11-03T17:08:03.693 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.692+0000 7f51977fe700 1 -- 172.21.15.115:0/3002360860 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f51a001eda0 con 0x7f51b00f6a80 2021-11-03T17:08:03.693 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.692+0000 7f51b57c2700 1 -- 172.21.15.115:0/3002360860 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b01060f0 con 0x7f51b00f6a80 2021-11-03T17:08:03.693 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.693+0000 7f51977fe700 1 -- 172.21.15.115:0/3002360860 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 17) v1 ==== 36269+0+0 (secure 0 0 0) 0x7f51a0014420 con 0x7f51b00f6a80 2021-11-03T17:08:03.693 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.693+0000 7f51b57c2700 1 -- 172.21.15.115:0/3002360860 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51b0045ad0 con 0x7f51b00f6a80 2021-11-03T17:08:03.693 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.693+0000 7f51977fe700 1 --2- 172.21.15.115:0/3002360860 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f519802ff60 0x7f5198032420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:08:03.694 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.693+0000 7f51977fe700 1 -- 172.21.15.115:0/3002360860 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1237+0+0 (secure 0 0 0) 0x7f51a0041240 con 0x7f51b00f6a80 2021-11-03T17:08:03.694 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.693+0000 7f51ae7fc700 1 --2- 172.21.15.115:0/3002360860 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f519802ff60 0x7f5198032420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:08:03.694 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.693+0000 7f51ae7fc700 1 --2- 172.21.15.115:0/3002360860 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f519802ff60 0x7f5198032420 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f51a40096a0 tx=0x7f51a4006b40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:08:03.695 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.698+0000 7f51977fe700 1 -- 172.21.15.115:0/3002360860 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f51a001c6e0 con 0x7f51b00f6a80 2021-11-03T17:08:03.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:03 smithi115 systemd[1]: Starting Ceph mon.smithi115 for 32ab977e-3cc8-11ec-8c28-001a4aab830c... 2021-11-03T17:08:03.921 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.925+0000 7f51b57c2700 1 -- 172.21.15.115:0/3002360860 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f51b0106fa0 con 0x7f51b00f6a80 2021-11-03T17:08:03.922 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.925+0000 7f51977fe700 1 -- 172.21.15.115:0/3002360860 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+690 (secure 0 0 0) 0x7f51a001c320 con 0x7f51b00f6a80 2021-11-03T17:08:03.923 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:08:03.924 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":1,"fsid":"32ab977e-3cc8-11ec-8c28-001a4aab830c","modified":"2021-11-03T17:05:43.573486Z","created":"2021-11-03T17:05:43.573486Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi027","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:3300","nonce":0},{"type":"v1","addr":"172.21.15.27:6789","nonce":0}]},"addr":"172.21.15.27:6789/0","public_addr":"172.21.15.27:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2021-11-03T17:08:03.925 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.927+0000 7f51b57c2700 1 -- 172.21.15.115:0/3002360860 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f519802ff60 msgr2=0x7f5198032420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:08:03.926 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.927+0000 7f51b57c2700 1 --2- 172.21.15.115:0/3002360860 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f519802ff60 0x7f5198032420 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f51a40096a0 tx=0x7f51a4006b40).stop 2021-11-03T17:08:03.926 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.927+0000 7f51b57c2700 1 -- 172.21.15.115:0/3002360860 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51b00f6a80 msgr2=0x7f51b0105570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:08:03.926 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.927+0000 7f51b57c2700 1 --2- 172.21.15.115:0/3002360860 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51b00f6a80 0x7f51b0105570 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f51a0009f80 tx=0x7f51a000b040).stop 2021-11-03T17:08:03.927 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.927+0000 7f51b57c2700 1 -- 172.21.15.115:0/3002360860 shutdown_connections 2021-11-03T17:08:03.927 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.927+0000 7f51b57c2700 1 --2- 172.21.15.115:0/3002360860 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f519802ff60 0x7f5198032420 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:08:03.927 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.927+0000 7f51b57c2700 1 --2- 172.21.15.115:0/3002360860 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51b00f6a80 0x7f51b0105570 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:08:03.928 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.927+0000 7f51b57c2700 1 -- 172.21.15.115:0/3002360860 >> 172.21.15.115:0/3002360860 conn(0x7f51b00f02e0 msgr2=0x7f51b00f0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:08:03.928 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.927+0000 7f51b57c2700 1 -- 172.21.15.115:0/3002360860 shutdown_connections 2021-11-03T17:08:03.929 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:03.927+0000 7f51b57c2700 1 -- 172.21.15.115:0/3002360860 wait complete. 2021-11-03T17:08:03.929 INFO:teuthology.orchestra.run.smithi115.stderr:dumped monmap epoch 1 2021-11-03T17:08:04.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:03 smithi027 conmon[32322]: cluster 2021-11-03T17:08:02.532490+0000 mgr.smithi027.ufhhvb (mgr.14162) 44 : cluster [DBG] pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:04.618 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.359+0000 7f7b59cee700 0 set uid:gid to 167:167 (ceph:ceph) 2021-11-03T17:08:04.619 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.359+0000 7f7b59cee700 0 ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable), process ceph-mon, pid 7 2021-11-03T17:08:04.619 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.359+0000 7f7b59cee700 0 pidfile_write: ignore empty --pid-file 2021-11-03T17:08:04.619 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 0 load: jerasure load: lrc load: isa 2021-11-03T17:08:04.620 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: RocksDB version: 6.8.1 2021-11-03T17:08:04.620 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: 2021-11-03T17:08:04.620 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Git sha rocksdb_build_git_sha:@0@ 2021-11-03T17:08:04.620 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Compile date May 13 2021 2021-11-03T17:08:04.621 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: DB SUMMARY 2021-11-03T17:08:04.621 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: 2021-11-03T17:08:04.621 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: CURRENT file: CURRENT 2021-11-03T17:08:04.621 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: 2021-11-03T17:08:04.622 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: IDENTITY file: IDENTITY 2021-11-03T17:08:04.622 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: 2021-11-03T17:08:04.622 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: MANIFEST file: MANIFEST-000001 size: 13 Bytes 2021-11-03T17:08:04.622 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: 2021-11-03T17:08:04.624 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi115/store.db dir, Total Num: 0, files: 2021-11-03T17:08:04.625 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: 2021-11-03T17:08:04.625 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi115/store.db: 000003.log size: 511 ; 2021-11-03T17:08:04.626 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: 2021-11-03T17:08:04.626 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.error_if_exists: 0 2021-11-03T17:08:04.627 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.create_if_missing: 0 2021-11-03T17:08:04.628 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.paranoid_checks: 1 2021-11-03T17:08:04.628 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.env: 0x557d251911c0 2021-11-03T17:08:04.629 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.fs: Posix File System 2021-11-03T17:08:04.629 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.info_log: 0x557d27387960 2021-11-03T17:08:04.630 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.max_file_opening_threads: 16 2021-11-03T17:08:04.630 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.statistics: (nil) 2021-11-03T17:08:04.630 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.use_fsync: 0 2021-11-03T17:08:04.631 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.max_log_file_size: 0 2021-11-03T17:08:04.631 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.max_manifest_file_size: 1073741824 2021-11-03T17:08:04.632 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.log_file_time_to_roll: 0 2021-11-03T17:08:04.632 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.keep_log_file_num: 1000 2021-11-03T17:08:04.632 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.recycle_log_file_num: 0 2021-11-03T17:08:04.633 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.allow_fallocate: 1 2021-11-03T17:08:04.633 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.allow_mmap_reads: 0 2021-11-03T17:08:04.634 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: 2021-11-03T17:08:04.634 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.allow_mmap_writes: 0 2021-11-03T17:08:04.634 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.use_direct_reads: 0 2021-11-03T17:08:04.635 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2021-11-03T17:08:04.635 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.create_missing_column_families: 0 2021-11-03T17:08:04.636 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.db_log_dir: 2021-11-03T17:08:04.636 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi115/store.db 2021-11-03T17:08:04.636 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.table_cache_numshardbits: 6 2021-11-03T17:08:04.637 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.max_subcompactions: 1 2021-11-03T17:08:04.638 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.max_background_flushes: -1 2021-11-03T17:08:04.638 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.WAL_ttl_seconds: 0 2021-11-03T17:08:04.639 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.WAL_size_limit_MB: 0 2021-11-03T17:08:04.639 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2021-11-03T17:08:04.639 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.manifest_preallocation_size: 4194304 2021-11-03T17:08:04.640 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.is_fd_close_on_exec: 1 2021-11-03T17:08:04.640 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.advise_random_on_open: 1 2021-11-03T17:08:04.640 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.db_write_buffer_size: 0 2021-11-03T17:08:04.641 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: 2021-11-03T17:08:04.641 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.write_buffer_manager: 0x557d27ffc030 2021-11-03T17:08:04.641 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.access_hint_on_compaction_start: 1 2021-11-03T17:08:04.642 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2021-11-03T17:08:04.642 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2021-11-03T17:08:04.643 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.use_adaptive_mutex: 0 2021-11-03T17:08:04.643 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.rate_limiter: (nil) 2021-11-03T17:08:04.646 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2021-11-03T17:08:04.647 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.wal_recovery_mode: 2 2021-11-03T17:08:04.647 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.enable_thread_tracking: 0 2021-11-03T17:08:04.648 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.enable_pipelined_write: 0 2021-11-03T17:08:04.649 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.unordered_write: 0 2021-11-03T17:08:04.649 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2021-11-03T17:08:04.650 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2021-11-03T17:08:04.650 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.write_thread_max_yield_usec: 100 2021-11-03T17:08:04.651 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2021-11-03T17:08:04.651 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.652 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.row_cache: None 2021-11-03T17:08:04.652 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.wal_filter: None 2021-11-03T17:08:04.652 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.avoid_flush_during_recovery: 0 2021-11-03T17:08:04.653 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.allow_ingest_behind: 0 2021-11-03T17:08:04.653 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.preserve_deletes: 0 2021-11-03T17:08:04.653 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.two_write_queues: 0 2021-11-03T17:08:04.654 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.manual_wal_flush: 0 2021-11-03T17:08:04.654 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.atomic_flush: 0 2021-11-03T17:08:04.655 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2021-11-03T17:08:04.655 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.persist_stats_to_disk: 0 2021-11-03T17:08:04.656 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.write_dbid_to_manifest: 0 2021-11-03T17:08:04.656 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.log_readahead_size: 0 2021-11-03T17:08:04.657 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.sst_file_checksum_func: Unknown 2021-11-03T17:08:04.657 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.max_background_jobs: 2 2021-11-03T17:08:04.657 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.max_background_compactions: -1 2021-11-03T17:08:04.658 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2021-11-03T17:08:04.658 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2021-11-03T17:08:04.660 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.delayed_write_rate : 16777216 2021-11-03T17:08:04.660 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.max_total_wal_size: 0 2021-11-03T17:08:04.661 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2021-11-03T17:08:04.661 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.stats_dump_period_sec: 600 2021-11-03T17:08:04.661 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.stats_persist_period_sec: 600 2021-11-03T17:08:04.662 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.stats_history_buffer_size: 1048576 2021-11-03T17:08:04.662 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.max_open_files: -1 2021-11-03T17:08:04.663 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.bytes_per_sync: 0 2021-11-03T17:08:04.663 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.wal_bytes_per_sync: 0 2021-11-03T17:08:04.664 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.strict_bytes_per_sync: 0 2021-11-03T17:08:04.664 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.compaction_readahead_size: 0 2021-11-03T17:08:04.664 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Compression algorithms supported: 2021-11-03T17:08:04.665 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: kZSTDNotFinalCompression supported: 0 2021-11-03T17:08:04.665 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: kZSTD supported: 0 2021-11-03T17:08:04.665 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: kXpressCompression supported: 0 2021-11-03T17:08:04.665 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: kLZ4HCCompression supported: 1 2021-11-03T17:08:04.665 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: kLZ4Compression supported: 1 2021-11-03T17:08:04.666 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: kBZip2Compression supported: 0 2021-11-03T17:08:04.666 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: kZlibCompression supported: 1 2021-11-03T17:08:04.666 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: kSnappyCompression supported: 1 2021-11-03T17:08:04.666 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Fast CRC32 supported: Supported on x86 2021-11-03T17:08:04.667 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: [version_set.cc:4413] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi115/store.db/MANIFEST-000001 2021-11-03T17:08:04.667 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: 2021-11-03T17:08:04.667 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: [column_family.cc:552] --------------- Options for column family [default]: 2021-11-03T17:08:04.667 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: 2021-11-03T17:08:04.667 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2021-11-03T17:08:04.668 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.merge_operator: 2021-11-03T17:08:04.668 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.compaction_filter: None 2021-11-03T17:08:04.668 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.compaction_filter_factory: None 2021-11-03T17:08:04.668 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.memtable_factory: SkipListFactory 2021-11-03T17:08:04.669 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.364+0000 7f7b59cee700 4 rocksdb: Options.table_factory: BlockBasedTable 2021-11-03T17:08:04.669 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x557d27298128) 2021-11-03T17:08:04.669 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: cache_index_and_filter_blocks: 1 2021-11-03T17:08:04.669 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: cache_index_and_filter_blocks_with_high_priority: 0 2021-11-03T17:08:04.669 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: pin_l0_filter_and_index_blocks_in_cache: 0 2021-11-03T17:08:04.670 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: pin_top_level_index_and_filter: 1 2021-11-03T17:08:04.670 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: index_type: 0 2021-11-03T17:08:04.670 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: data_block_index_type: 0 2021-11-03T17:08:04.670 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: index_shortening: 1 2021-11-03T17:08:04.671 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: data_block_hash_table_util_ratio: 0.750000 2021-11-03T17:08:04.671 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: hash_index_allow_collision: 1 2021-11-03T17:08:04.671 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: checksum: 1 2021-11-03T17:08:04.671 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: no_block_cache: 0 2021-11-03T17:08:04.671 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: block_cache: 0x557d272d2f10 2021-11-03T17:08:04.672 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: block_cache_name: BinnedLRUCache 2021-11-03T17:08:04.672 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: block_cache_options: 2021-11-03T17:08:04.672 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: capacity : 536870912 2021-11-03T17:08:04.672 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: num_shard_bits : 4 2021-11-03T17:08:04.673 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: strict_capacity_limit : 0 2021-11-03T17:08:04.673 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: high_pri_pool_ratio: 0.000 2021-11-03T17:08:04.673 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: block_cache_compressed: (nil) 2021-11-03T17:08:04.673 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: persistent_cache: (nil) 2021-11-03T17:08:04.673 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: block_size: 4096 2021-11-03T17:08:04.674 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: block_size_deviation: 10 2021-11-03T17:08:04.674 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: block_restart_interval: 16 2021-11-03T17:08:04.674 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: index_block_restart_interval: 1 2021-11-03T17:08:04.675 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: metadata_block_size: 4096 2021-11-03T17:08:04.675 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: partition_filters: 0 2021-11-03T17:08:04.675 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: use_delta_encoding: 1 2021-11-03T17:08:04.676 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: filter_policy: rocksdb.BuiltinBloomFilter 2021-11-03T17:08:04.676 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: whole_key_filtering: 1 2021-11-03T17:08:04.677 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: verify_compression: 0 2021-11-03T17:08:04.677 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: read_amp_bytes_per_bit: 0 2021-11-03T17:08:04.678 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: format_version: 2 2021-11-03T17:08:04.678 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: enable_index_compression: 1 2021-11-03T17:08:04.678 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: block_align: 0 2021-11-03T17:08:04.679 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: 2021-11-03T17:08:04.679 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.write_buffer_size: 33554432 2021-11-03T17:08:04.680 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: 2021-11-03T17:08:04.680 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.max_write_buffer_number: 2 2021-11-03T17:08:04.681 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.compression: NoCompression 2021-11-03T17:08:04.681 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.bottommost_compression: Disabled 2021-11-03T17:08:04.682 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.prefix_extractor: nullptr 2021-11-03T17:08:04.682 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2021-11-03T17:08:04.682 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.num_levels: 7 2021-11-03T17:08:04.683 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2021-11-03T17:08:04.683 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2021-11-03T17:08:04.683 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2021-11-03T17:08:04.684 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2021-11-03T17:08:04.684 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2021-11-03T17:08:04.685 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2021-11-03T17:08:04.685 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2021-11-03T17:08:04.685 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2021-11-03T17:08:04.686 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2021-11-03T17:08:04.686 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.compression_opts.window_bits: -14 2021-11-03T17:08:04.687 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.compression_opts.level: 32767 2021-11-03T17:08:04.687 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.compression_opts.strategy: 0 2021-11-03T17:08:04.687 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2021-11-03T17:08:04.688 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2021-11-03T17:08:04.688 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.compression_opts.enabled: false 2021-11-03T17:08:04.689 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2021-11-03T17:08:04.689 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2021-11-03T17:08:04.690 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.level0_stop_writes_trigger: 36 2021-11-03T17:08:04.690 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.target_file_size_base: 67108864 2021-11-03T17:08:04.690 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.target_file_size_multiplier: 1 2021-11-03T17:08:04.691 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2021-11-03T17:08:04.691 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2021-11-03T17:08:04.692 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2021-11-03T17:08:04.692 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2021-11-03T17:08:04.692 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2021-11-03T17:08:04.693 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2021-11-03T17:08:04.693 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2021-11-03T17:08:04.693 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2021-11-03T17:08:04.694 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2021-11-03T17:08:04.694 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2021-11-03T17:08:04.695 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2021-11-03T17:08:04.695 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.max_compaction_bytes: 1677721600 2021-11-03T17:08:04.695 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.arena_block_size: 4194304 2021-11-03T17:08:04.696 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2021-11-03T17:08:04.696 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2021-11-03T17:08:04.696 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: 2021-11-03T17:08:04.697 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2021-11-03T17:08:04.697 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.disable_auto_compactions: 0 2021-11-03T17:08:04.698 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2021-11-03T17:08:04.698 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2021-11-03T17:08:04.698 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2021-11-03T17:08:04.699 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2021-11-03T17:08:04.699 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2021-11-03T17:08:04.699 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2021-11-03T17:08:04.700 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2021-11-03T17:08:04.700 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2021-11-03T17:08:04.701 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2021-11-03T17:08:04.701 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2021-11-03T17:08:04.701 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.table_properties_collectors: 2021-11-03T17:08:04.702 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.inplace_update_support: 0 2021-11-03T17:08:04.702 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.inplace_update_num_locks: 10000 2021-11-03T17:08:04.703 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2021-11-03T17:08:04.703 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.memtable_whole_key_filtering: 0 2021-11-03T17:08:04.704 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.memtable_huge_page_size: 0 2021-11-03T17:08:04.704 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.bloom_locality: 0 2021-11-03T17:08:04.704 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.max_successive_merges: 0 2021-11-03T17:08:04.705 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.optimize_filters_for_hits: 0 2021-11-03T17:08:04.705 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.paranoid_file_checks: 0 2021-11-03T17:08:04.705 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.force_consistency_checks: 0 2021-11-03T17:08:04.705 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.report_bg_io_stats: 0 2021-11-03T17:08:04.706 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.ttl: 2592000 2021-11-03T17:08:04.706 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: Options.periodic_compaction_seconds: 0 2021-11-03T17:08:04.706 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: [version_set.cc:4568] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi115/store.db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2021-11-03T17:08:04.707 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: 2021-11-03T17:08:04.707 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: [version_set.cc:4577] Column family [default] (ID 0), log number is 0 2021-11-03T17:08:04.708 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: 2021-11-03T17:08:04.708 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1635959284367161, "job": 1, "event": "recovery_started", "log_files": [3]} 2021-11-03T17:08:04.708 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.709 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: 2021-11-03T17:08:04.365+0000 7f7b59cee700 4 rocksdb: [db_impl/db_impl_open.cc:760] Recovering log #3 mode 2 2021-11-03T17:08:04.709 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.365+0000 7f7b59cee700 3 rocksdb: [le/block_based/filter_policy.cc:584] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2021-11-03T17:08:04.710 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.366+0000 7f7b59cee700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1635959284367779, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 4, "file_size": 1437, "table_properties": {"data_size": 523, "index_size": 40, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 0, "index_value_is_delta_encoded": 0, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1635959284, "oldest_key_time": 3, "file_creation_time": 0}} 2021-11-03T17:08:04.710 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.366+0000 7f7b59cee700 4 rocksdb: [version_set.cc:3826] Creating manifest 5 2021-11-03T17:08:04.710 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: 2021-11-03T17:08:04.711 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.366+0000 7f7b59cee700 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1635959284368263, "job": 1, "event": "recovery_finished"} 2021-11-03T17:08:04.711 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.367+0000 7f7b59cee700 4 rocksdb: DB pointer 0x557d27399800 2021-11-03T17:08:04.712 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.367+0000 7f7b40b06700 4 rocksdb: [db_impl/db_impl.cc:850] ------- DUMPING STATS ------- 2021-11-03T17:08:04.712 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.367+0000 7f7b40b06700 4 rocksdb: [db_impl/db_impl.cc:851] 2021-11-03T17:08:04.713 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: ** DB Stats ** 2021-11-03T17:08:04.713 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: Uptime(secs): 0.0 total, 0.0 interval 2021-11-03T17:08:04.713 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2021-11-03T17:08:04.714 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2021-11-03T17:08:04.714 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2021-11-03T17:08:04.714 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2021-11-03T17:08:04.715 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2021-11-03T17:08:04.715 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2021-11-03T17:08:04.715 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: 2021-11-03T17:08:04.716 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: ** Compaction Stats [default] ** 2021-11-03T17:08:04.716 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-11-03T17:08:04.717 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-11-03T17:08:04.717 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: L0 1/0 1.40 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.4 0.00 0.00 1 0.001 0 0 2021-11-03T17:08:04.718 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: Sum 1/0 1.40 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.4 0.00 0.00 1 0.001 0 0 2021-11-03T17:08:04.718 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.4 0.00 0.00 1 0.001 0 0 2021-11-03T17:08:04.718 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: 2021-11-03T17:08:04.718 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: ** Compaction Stats [default] ** 2021-11-03T17:08:04.719 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-11-03T17:08:04.719 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-11-03T17:08:04.719 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.4 0.00 0.00 1 0.001 0 0 2021-11-03T17:08:04.719 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: Uptime(secs): 0.0 total, 0.0 interval 2021-11-03T17:08:04.720 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: Flush(GB): cumulative 0.000, interval 0.000 2021-11-03T17:08:04.720 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: AddFile(GB): cumulative 0.000, interval 0.000 2021-11-03T17:08:04.720 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: AddFile(Total Files): cumulative 0, interval 0 2021-11-03T17:08:04.720 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: AddFile(L0 Files): cumulative 0, interval 0 2021-11-03T17:08:04.721 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: AddFile(Keys): cumulative 0, interval 0 2021-11-03T17:08:04.721 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: Cumulative compaction: 0.00 GB write, 0.48 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-11-03T17:08:04.721 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: Interval compaction: 0.00 GB write, 0.48 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-11-03T17:08:04.721 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2021-11-03T17:08:04.722 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: 2021-11-03T17:08:04.722 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: ** File Read Latency Histogram By Level [default] ** 2021-11-03T17:08:04.722 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: 2021-11-03T17:08:04.722 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: ** Compaction Stats [default] ** 2021-11-03T17:08:04.722 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-11-03T17:08:04.723 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-11-03T17:08:04.723 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: L0 1/0 1.40 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.4 0.00 0.00 1 0.001 0 0 2021-11-03T17:08:04.723 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: Sum 1/0 1.40 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.4 0.00 0.00 1 0.001 0 0 2021-11-03T17:08:04.723 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2021-11-03T17:08:04.724 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: 2021-11-03T17:08:04.724 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: ** Compaction Stats [default] ** 2021-11-03T17:08:04.724 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-11-03T17:08:04.724 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-11-03T17:08:04.725 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.4 0.00 0.00 1 0.001 0 0 2021-11-03T17:08:04.725 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: Uptime(secs): 0.0 total, 0.0 interval 2021-11-03T17:08:04.726 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: Flush(GB): cumulative 0.000, interval 0.000 2021-11-03T17:08:04.726 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: AddFile(GB): cumulative 0.000, interval 0.000 2021-11-03T17:08:04.727 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: AddFile(Total Files): cumulative 0, interval 0 2021-11-03T17:08:04.727 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: AddFile(L0 Files): cumulative 0, interval 0 2021-11-03T17:08:04.728 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: AddFile(Keys): cumulative 0, interval 0 2021-11-03T17:08:04.728 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: Cumulative compaction: 0.00 GB write, 0.47 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-11-03T17:08:04.728 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-11-03T17:08:04.729 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2021-11-03T17:08:04.729 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: 2021-11-03T17:08:04.729 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: ** File Read Latency Histogram By Level [default] ** 2021-11-03T17:08:04.729 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: 2021-11-03T17:08:04.730 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.368+0000 7f7b59cee700 0 mon.smithi115 does not exist in monmap, will attempt to join an existing cluster 2021-11-03T17:08:04.730 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.368+0000 7f7b59cee700 0 using public_addr v2:172.21.15.115:0/0 -> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] 2021-11-03T17:08:04.730 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.368+0000 7f7b59cee700 0 starting mon.smithi115 rank -1 at public addrs [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] at bind addrs [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi115 fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c 2021-11-03T17:08:04.730 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.370+0000 7f7b59cee700 1 mon.smithi115@-1(???) e0 preinit fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c 2021-11-03T17:08:04.731 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.396+0000 7f7b4330b700 0 mon.smithi115@-1(synchronizing).mds e1 new map 2021-11-03T17:08:04.731 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.396+0000 7f7b4330b700 0 mon.smithi115@-1(synchronizing).mds e1 print_map 2021-11-03T17:08:04.731 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: e1 2021-11-03T17:08:04.731 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:08:04.732 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:08:04.732 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: legacy client fscid: -1 2021-11-03T17:08:04.732 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: 2021-11-03T17:08:04.732 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: No filesystems configured 2021-11-03T17:08:04.732 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: 2021-11-03T17:08:04.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.396+0000 7f7b4330b700 1 mon.smithi115@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2021-11-03T17:08:04.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.396+0000 7f7b4330b700 1 mon.smithi115@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2021-11-03T17:08:04.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: 2021-11-03T17:08:04.396+0000 7f7b4330b700 1 mon.smithi115@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2021-11-03T17:08:04.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.396+0000 7f7b4330b700 1 mon.smithi115@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2021-11-03T17:08:04.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.396+0000 7f7b4330b700 1 mon.smithi115@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2021-11-03T17:08:04.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.396+0000 7f7b4330b700 1 mon.smithi115@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2021-11-03T17:08:04.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.396+0000 7f7b4330b700 1 mon.smithi115@-1(synchronizing).osd e5 e5: 0 total, 0 up, 0 in 2021-11-03T17:08:04.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.397+0000 7f7b4330b700 0 mon.smithi115@-1(synchronizing).osd e5 crush map has features 3314932999778484224, adjusting msgr requires 2021-11-03T17:08:04.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: 2021-11-03T17:08:04.397+0000 7f7b4330b700 0 mon.smithi115@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2021-11-03T17:08:04.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.397+0000 7f7b4330b700 0 mon.smithi115@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2021-11-03T17:08:04.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.397+0000 7f7b4330b700 0 mon.smithi115@-1(synchronizing).osd e5 crush map has features 288514050185494528, adjusting msgr requires 2021-11-03T17:08:04.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: cluster 2021-11-03T17:08:04.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: 2021-11-03T17:08:02.532490+0000 mgr.smithi027.ufhhvb (mgr 2021-11-03T17:08:04.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: .14162) 44 : cluster [DBG] pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:04.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 conmon[34670]: debug 2021-11-03T17:08:04.397+0000 7f7b4330b700 1 mon.smithi115@-1(synchronizing).paxosservice(auth 1..7) refresh upgraded, format 0 -> 3 2021-11-03T17:08:04.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 bash[34525]: 319323f0983442a469e49342deb68dcf097e9049c2caef2669ebf60c3a070d28 2021-11-03T17:08:04.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:04 smithi115 systemd[1]: Started Ceph mon.smithi115 for 32ab977e-3cc8-11ec-8c28-001a4aab830c. 2021-11-03T17:08:05.604 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2021-11-03T17:08:05.605 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph mon dump -f json 2021-11-03T17:08:09.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:09 smithi115 conmon[34670]: audit 2021-11-03T17:08:04.405010+0000 mon.smithi027 (mon.0) 159 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2021-11-03T17:08:09.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:09 smithi115 conmon[34670]: cluster 2021-11-03T17:08:04.405277+0000 mon.smithi027 (mon.0) 160 : cluster [INF] mon.smithi027 calling monitor election 2021-11-03T17:08:09.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:09 smithi115 conmon[34670]: audit 2021-11-03T17:08:04.406743+0000 mon.smithi027 (mon.0) 161 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2021-11-03T17:08:09.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:09 smithi115 conmon[34670]: cluster 2021-11-03T17:08:04.532929+0000 mgr.smithi027.ufhhvb (mgr.14162) 45 : cluster [DBG] pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:09.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:09 smithi115 conmon[34670]: audit 2021-11-03T17:08:05.151711+0000 mon.smithi027 (mon.0) 162 : audit [DBG] from='mgr.? 172.21.15.115:0/1081831706' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi115.tcyodz/crt"}]: dispatch 2021-11-03T17:08:09.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:09 smithi115 conmon[34670]: audit 2021-11-03T17:08:05.402178+0000 mon.smithi027 (mon.0) 163 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2021-11-03T17:08:09.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:09 smithi115 conmon[34670]: audit 2021-11-03T17:08:06.402265+0000 mon.smithi027 (mon.0) 164 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2021-11-03T17:08:09.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:09 smithi115 conmon[34670]: cluster 2021-11-03T17:08:06.403050+0000 mon.smithi115 (mon.1) 1 : cluster [INF] mon.smithi115 calling monitor election 2021-11-03T17:08:09.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:09 smithi115 conmon[34670]: cluster 2021-11-03T17:08:06.533322+0000 mgr.smithi027.ufhhvb (mgr.14162) 46 : cluster [DBG] pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:09.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:09 smithi115 conmon[34670]: audit 2021-11-03T17:08:07.402267+0000 mon.smithi027 (mon.0) 165 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2021-11-03T17:08:09.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:09 smithi115 conmon[34670]: audit 2021-11-03T17:08:08.402400+0000 mon.smithi027 (mon.0) 166 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2021-11-03T17:08:09.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:09 smithi115 conmon[34670]: cluster 2021-11-03T17:08:08.533705+0000 mgr.smithi027.ufhhvb (mgr.14162) 47 : cluster [DBG] pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:09.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:09 smithi115 conmon[34670]: audit 2021-11-03T17:08:09.402787+0000 mon.smithi027 (mon.0) 167 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2021-11-03T17:08:09.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:09 smithi115 conmon[34670]: cluster 2021-11-03T17:08:09.419580+0000 mon.smithi027 (mon.0) 168 : cluster [INF] mon.smithi027 is new leader, mons smithi027,smithi115 in quorum (ranks 0,1) 2021-11-03T17:08:09.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:09 smithi115 conmon[34670]: cluster 2021-11-03T17:08:09.422684+0000 mon.smithi027 (mon.0) 169 : cluster [DBG] monmap e2: 2 mons at {smithi027=[v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0],smithi115=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0]} 2021-11-03T17:08:09.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:09 smithi115 conmon[34670]: cluster 2021-11-03T17:08:09.422745+0000 mon.smithi027 (mon.0) 170 : cluster [DBG] fsmap 2021-11-03T17:08:09.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:09 smithi115 conmon[34670]: cluster 2021-11-03T17:08:09.422778+0000 mon.smithi027 (mon.0) 171 : cluster [DBG] osdmap e5: 0 total, 0 up, 0 in 2021-11-03T17:08:09.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:09 smithi115 conmon[34670]: cluster 2021-11-03T17:08:09.423003+0000 mon.smithi027 (mon.0) 172 : cluster [DBG] mgrmap e17: smithi027.ufhhvb(active, since 77s) 2021-11-03T17:08:09.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:09 smithi115 conmon[34670]: cluster 2021-11-03T17:08:09.423878+0000 mon.smithi027 (mon.0) 173 : cluster [DBG] Standby manager daemon smithi115.tcyodz started 2021-11-03T17:08:09.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:09 smithi115 conmon[34670]: audit 2021-11-03T17:08:09.425133+0000 mon.smithi027 (mon.0) 174 : audit [DBG] from='mgr.? 172.21.15.115:0/1081831706' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-11-03T17:08:09.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:09 smithi115 conmon[34670]: cluster 2021-11-03T17:08:09.426927+0000 mon.smithi027 (mon.0) 175 : cluster [INF] overall HEALTH_OK 2021-11-03T17:08:09.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:09 smithi115 conmon[34670]: audit 2021-11-03T17:08:09.427427+0000 mon.smithi027 (mon.0) 176 : audit [DBG] from='mgr.? 172.21.15.115:0/1081831706' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi115.tcyodz/key"}]: dispatch 2021-11-03T17:08:09.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:09 smithi115 conmon[34670]: audit 2021-11-03T17:08:09.429239+0000 mon.smithi027 (mon.0) 177 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:09.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:09 smithi115 conmon[34670]: audit 2021-11-03T17:08:09.429440+0000 mon.smithi027 (mon.0) 178 : audit [DBG] from='mgr.? 172.21.15.115:0/1081831706' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-11-03T17:08:09.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: audit 2021-11-03T17:08:04.405010+0000 mon.smithi027 (mon.0) 159 : audit [DBG] 2021-11-03T17:08:09.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2021-11-03T17:08:09.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: cluster 2021-11-03T17:08:04.405277+0000 mon.smithi027 (mon.0) 160 : cluster [INF] mon.smithi027 calling monitor election 2021-11-03T17:08:09.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: audit 2021-11-03T17:08:09.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: 2021-11-03 2021-11-03T17:08:09.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: T17:08:04.406743+0000 mon.smithi027 2021-11-03T17:08:09.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: (mon.0) 161 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2021-11-03T17:08:09.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: 2021-11-03T17:08:09.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: cluster 2021 2021-11-03T17:08:09.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: -11-03T17:08:04.532929+0000 2021-11-03T17:08:09.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14162) 45 : cluster [DBG] 2021-11-03T17:08:09.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:09.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: audit 2021-11-03T17 2021-11-03T17:08:09.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: :08:05.151711+0000 mon.smithi027 (mon. 2021-11-03T17:08:09.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: 0) 162 : audit [DBG] from='mgr.? 172.21.15.115:0/1081831706' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi115.tcyodz/crt"}]: dispatch 2021-11-03T17:08:09.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: audit 2021- 2021-11-03T17:08:09.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: 11-03T17:08:05.402178+0000 2021-11-03T17:08:09.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: mon.smithi027 (mon.0) 163 : audit [DBG] 2021-11-03T17:08:09.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2021-11-03T17:08:09.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: audit 2021-11-03T17:08:06 2021-11-03T17:08:09.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: .402265+0000 mon.smithi027 (mon.0) 164 : 2021-11-03T17:08:09.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2021-11-03T17:08:09.776 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: cluster 2021-11-03T17:08 2021-11-03T17:08:09.776 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: :06.403050+0000 mon.smithi115 (mon.1) 2021-11-03T17:08:09.776 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: 1 : cluster [INF] mon.smithi115 calling monitor election 2021-11-03T17:08:09.777 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: cluster 2021-11- 2021-11-03T17:08:09.777 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: 03T17:08:06.533322+0000 mgr.smithi027.ufhhvb 2021-11-03T17:08:09.777 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: (mgr.14162) 46 : cluster [DBG] pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:09.777 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: 2021-11-03T17:08:09.778 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: audit 2021-11-03T17:08:09.778 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: 2021-11-03T17:08:07.402267+0000 2021-11-03T17:08:09.778 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: mon.smithi027 (mon.0) 165 : audit [DBG] 2021-11-03T17:08:09.778 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2021-11-03T17:08:09.779 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: audit 2021-11-03T17:08: 2021-11-03T17:08:09.779 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: 08.402400+0000 mon.smithi027 (mon.0) 166 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2021-11-03T17:08:09.779 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: 2021-11-03T17:08:09.779 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: cluster 2021-11-03T17:08:09.780 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: 2021-11-03T17:08:08.533705+0000 2021-11-03T17:08:09.780 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14162) 47 : cluster [DBG] 2021-11-03T17:08:09.780 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:09.780 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: audit 2021-11-03T17:08 2021-11-03T17:08:09.781 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: :09.402787+0000 mon.smithi027 (mon.0 2021-11-03T17:08:09.781 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: ) 167 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2021-11-03T17:08:09.781 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: cluster 2021-11 2021-11-03T17:08:09.781 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: -03T17:08:09.419580+0000 2021-11-03T17:08:09.782 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: mon.smithi027 (mon.0) 168 : cluster [INF] 2021-11-03T17:08:09.782 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: mon.smithi027 is new leader, mons smithi027,smithi115 in quorum (ranks 0,1) 2021-11-03T17:08:09.782 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: cluster 2021-11-03T17: 2021-11-03T17:08:09.782 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: 08:09.422684+0000 mon.smithi027 (mon.0 2021-11-03T17:08:09.783 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: ) 169 : cluster [DBG] monmap e2: 2 mons at {smithi027=[v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0],smithi115=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0]} 2021-11-03T17:08:09.783 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: cluster 2021-11 2021-11-03T17:08:09.783 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: -03T17:08:09.422745+0000 2021-11-03T17:08:09.783 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: mon.smithi027 (mon.0) 170 : cluster [DBG] 2021-11-03T17:08:09.783 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: fsmap 2021-11-03T17:08:09.784 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: cluster 2021-11-03T17:08 2021-11-03T17:08:09.784 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: :09.422778+0000 mon.smithi027 (mon.0) 2021-11-03T17:08:09.784 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: 171 : cluster [DBG] osdmap e5: 0 total, 0 up, 0 in 2021-11-03T17:08:09.784 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: cluster 2021-11 2021-11-03T17:08:09.785 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: -03T17:08:09.423003+0000 2021-11-03T17:08:09.785 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: mon.smithi027 (mon.0) 172 : cluster [DBG] 2021-11-03T17:08:09.785 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: mgrmap e17: smithi027.ufhhvb(active, since 77s) 2021-11-03T17:08:09.786 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: cluster 2021-11-03T17:08 2021-11-03T17:08:09.786 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: :09.423878+0000 mon.smithi027 (mon.0) 2021-11-03T17:08:09.786 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: 173 : cluster [DBG] Standby manager daemon smithi115.tcyodz started 2021-11-03T17:08:09.786 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: audit 2021-11- 2021-11-03T17:08:09.787 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: 03T17:08:09.425133+0000 mon.smithi027 2021-11-03T17:08:09.787 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: (mon.0) 174 : audit [DBG] from='mgr.? 172.21.15.115:0/1081831706' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-11-03T17:08:09.787 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: cluster 2021-11-03 2021-11-03T17:08:09.788 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: T17:08:09.426927+0000 mon.smithi027 2021-11-03T17:08:09.788 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: (mon.0) 175 : cluster [INF] overall HEALTH_OK 2021-11-03T17:08:09.788 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: audit 2021-11 2021-11-03T17:08:09.788 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: -03T17:08:09.427427+0000 2021-11-03T17:08:09.788 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: mon.smithi027 (mon.0) 176 : audit [DBG] 2021-11-03T17:08:09.789 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: from='mgr.? 172.21.15.115:0/1081831706' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi115.tcyodz/key"}]: dispatch 2021-11-03T17:08:09.789 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: audit 2021-11-03T17:08 2021-11-03T17:08:09.789 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: :09.429239+0000 mon.smithi027 (mon.0) 2021-11-03T17:08:09.789 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: 177 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:09.790 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: audit 2021-11 2021-11-03T17:08:09.790 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: -03T17:08:09.429440+0000 2021-11-03T17:08:09.791 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: mon.smithi027 (mon.0) 178 : audit [DBG] 2021-11-03T17:08:09.791 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:09 smithi027 conmon[32322]: from='mgr.? 172.21.15.115:0/1081831706' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-11-03T17:08:10.339 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.338+0000 7eff9cad9700 1 -- 172.21.15.115:0/2809663984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff80003250 msgr2=0x7eff800056e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:08:10.339 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.338+0000 7eff9cad9700 1 --2- 172.21.15.115:0/2809663984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff80003250 0x7eff800056e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7eff980f61c0 tx=0x7eff8800b040).stop 2021-11-03T17:08:10.340 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.339+0000 7eff9cad9700 1 -- 172.21.15.115:0/2809663984 shutdown_connections 2021-11-03T17:08:10.340 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.339+0000 7eff9cad9700 1 --2- 172.21.15.115:0/2809663984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff80003250 0x7eff800056e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:08:10.340 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.339+0000 7eff9cad9700 1 -- 172.21.15.115:0/2809663984 >> 172.21.15.115:0/2809663984 conn(0x7eff980f0b70 msgr2=0x7eff980f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:08:10.340 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.339+0000 7eff9cad9700 1 -- 172.21.15.115:0/2809663984 shutdown_connections 2021-11-03T17:08:10.341 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.339+0000 7eff9cad9700 1 -- 172.21.15.115:0/2809663984 wait complete. 2021-11-03T17:08:10.341 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.340+0000 7eff9cad9700 1 Processor -- start 2021-11-03T17:08:10.341 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.340+0000 7eff9cad9700 1 -- start start 2021-11-03T17:08:10.341 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.340+0000 7eff9cad9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff80003250 0x7eff980fb090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:08:10.342 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.340+0000 7eff9cad9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff980f9510 0x7eff980f9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:08:10.342 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.340+0000 7eff9cad9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9810a330 con 0x7eff80003250 2021-11-03T17:08:10.342 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.340+0000 7eff9cad9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9810a470 con 0x7eff980f9510 2021-11-03T17:08:10.342 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.340+0000 7eff95d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff980f9510 0x7eff980f9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:08:10.343 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.340+0000 7eff95d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff980f9510 0x7eff980f9990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38562/0 (socket says 172.21.15.115:38562) 2021-11-03T17:08:10.343 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.340+0000 7eff95d9b700 1 -- 172.21.15.115:0/1194045692 learned_addr learned my addr 172.21.15.115:0/1194045692 (peer_addr_for_me v2:172.21.15.115:0/0) 2021-11-03T17:08:10.344 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.340+0000 7eff9659c700 1 --2- 172.21.15.115:0/1194045692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff80003250 0x7eff980fb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:08:10.344 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.340+0000 7eff95d9b700 1 -- 172.21.15.115:0/1194045692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff980f9510 msgr2=0x7eff980f9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2021-11-03T17:08:10.344 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.340+0000 7eff95d9b700 1 -- 172.21.15.115:0/1194045692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff980f9510 msgr2=0x7eff980f9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2021-11-03T17:08:10.345 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.340+0000 7eff95d9b700 1 --2- 172.21.15.115:0/1194045692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff980f9510 0x7eff980f9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2021-11-03T17:08:10.345 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.340+0000 7eff95d9b700 1 --2- 172.21.15.115:0/1194045692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff980f9510 0x7eff980f9990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2021-11-03T17:08:10.345 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.341+0000 7eff9659c700 1 -- 172.21.15.115:0/1194045692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff980f9510 msgr2=0x7eff980f9990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2021-11-03T17:08:10.345 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.341+0000 7eff9659c700 1 --2- 172.21.15.115:0/1194045692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff980f9510 0x7eff980f9990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:08:10.346 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.341+0000 7eff9659c700 1 -- 172.21.15.115:0/1194045692 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff88005040 con 0x7eff80003250 2021-11-03T17:08:10.346 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.341+0000 7eff9659c700 1 --2- 172.21.15.115:0/1194045692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff80003250 0x7eff980fb090 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7eff88019040 tx=0x7eff88002040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:08:10.346 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.341+0000 7eff86ffd700 1 -- 172.21.15.115:0/1194045692 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff88004840 con 0x7eff80003250 2021-11-03T17:08:10.347 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.341+0000 7eff86ffd700 1 -- 172.21.15.115:0/1194045692 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7eff88020440 con 0x7eff80003250 2021-11-03T17:08:10.347 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.341+0000 7eff9cad9700 1 -- 172.21.15.115:0/1194045692 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff980f9ee0 con 0x7eff80003250 2021-11-03T17:08:10.347 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.341+0000 7eff86ffd700 1 -- 172.21.15.115:0/1194045692 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff8801f600 con 0x7eff80003250 2021-11-03T17:08:10.347 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.341+0000 7eff9cad9700 1 -- 172.21.15.115:0/1194045692 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff98112ea0 con 0x7eff80003250 2021-11-03T17:08:10.347 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.342+0000 7eff9cad9700 1 -- 172.21.15.115:0/1194045692 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff98059100 con 0x7eff80003250 2021-11-03T17:08:10.348 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.342+0000 7eff86ffd700 1 -- 172.21.15.115:0/1194045692 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7eff880205b0 con 0x7eff80003250 2021-11-03T17:08:10.348 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.343+0000 7eff86ffd700 1 --2- 172.21.15.115:0/1194045692 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7eff7805c3b0 0x7eff7805e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:08:10.348 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.343+0000 7eff86ffd700 1 -- 172.21.15.115:0/1194045692 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1237+0+0 (secure 0 0 0) 0x7eff88077060 con 0x7eff80003250 2021-11-03T17:08:10.348 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.343+0000 7eff95d9b700 1 --2- 172.21.15.115:0/1194045692 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7eff7805c3b0 0x7eff7805e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:08:10.349 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.343+0000 7eff95d9b700 1 --2- 172.21.15.115:0/1194045692 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7eff7805c3b0 0x7eff7805e870 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7eff8c006cf0 tx=0x7eff8c008040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:08:10.349 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.347+0000 7eff86ffd700 1 -- 172.21.15.115:0/1194045692 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7eff8804ad50 con 0x7eff80003250 2021-11-03T17:08:10.559 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.563+0000 7eff9cad9700 1 -- 172.21.15.115:0/1194045692 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7eff98045ad0 con 0x7eff80003250 2021-11-03T17:08:10.560 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.564+0000 7eff86ffd700 1 -- 172.21.15.115:0/1194045692 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+968 (secure 0 0 0) 0x7eff8801b030 con 0x7eff80003250 2021-11-03T17:08:10.561 INFO:teuthology.orchestra.run.smithi115.stdout: 2021-11-03T17:08:10.561 INFO:teuthology.orchestra.run.smithi115.stdout:{"epoch":2,"fsid":"32ab977e-3cc8-11ec-8c28-001a4aab830c","modified":"2021-11-03T17:08:04.402191Z","created":"2021-11-03T17:05:43.573486Z","min_mon_release":16,"min_mon_release_name":"pacific","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging"],"optional":[]},"mons":[{"rank":0,"name":"smithi027","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:3300","nonce":0},{"type":"v1","addr":"172.21.15.27:6789","nonce":0}]},"addr":"172.21.15.27:6789/0","public_addr":"172.21.15.27:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi115","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:3300","nonce":0},{"type":"v1","addr":"172.21.15.115:6789","nonce":0}]},"addr":"172.21.15.115:6789/0","public_addr":"172.21.15.115:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2021-11-03T17:08:10.563 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.565+0000 7eff9cad9700 1 -- 172.21.15.115:0/1194045692 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7eff7805c3b0 msgr2=0x7eff7805e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:08:10.564 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.565+0000 7eff9cad9700 1 --2- 172.21.15.115:0/1194045692 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7eff7805c3b0 0x7eff7805e870 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7eff8c006cf0 tx=0x7eff8c008040).stop 2021-11-03T17:08:10.564 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.565+0000 7eff9cad9700 1 -- 172.21.15.115:0/1194045692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff80003250 msgr2=0x7eff980fb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:08:10.564 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.565+0000 7eff9cad9700 1 --2- 172.21.15.115:0/1194045692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff80003250 0x7eff980fb090 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7eff88019040 tx=0x7eff88002040).stop 2021-11-03T17:08:10.565 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.565+0000 7eff9cad9700 1 -- 172.21.15.115:0/1194045692 shutdown_connections 2021-11-03T17:08:10.565 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.565+0000 7eff9cad9700 1 --2- 172.21.15.115:0/1194045692 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7eff7805c3b0 0x7eff7805e870 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:08:10.565 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.565+0000 7eff9cad9700 1 --2- 172.21.15.115:0/1194045692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff80003250 0x7eff980fb090 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:08:10.565 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.565+0000 7eff9cad9700 1 --2- 172.21.15.115:0/1194045692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff980f9510 0x7eff980f9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:08:10.566 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.565+0000 7eff9cad9700 1 -- 172.21.15.115:0/1194045692 >> 172.21.15.115:0/1194045692 conn(0x7eff980f0b70 msgr2=0x7eff980f2e90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:08:10.566 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.565+0000 7eff9cad9700 1 -- 172.21.15.115:0/1194045692 shutdown_connections 2021-11-03T17:08:10.566 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:08:10.565+0000 7eff9cad9700 1 -- 172.21.15.115:0/1194045692 wait complete. 2021-11-03T17:08:10.566 INFO:teuthology.orchestra.run.smithi115.stderr:dumped monmap epoch 2 2021-11-03T17:08:10.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:10 smithi115 conmon[34670]: cephadm 2021-11-03T17:08:09.430393+0000 mgr.smithi027.ufhhvb (mgr.14162) 2021-11-03T17:08:10.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:10 smithi115 conmon[34670]: 48 : cephadm [INF] Deploying daemon node-exporter.smithi115 on smithi115 2021-11-03T17:08:10.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:10 smithi115 conmon[34670]: cluster 2021-11-03T17:08:09.483423+0000 mon.smithi027 (mon.0) 179 : cluster [DBG] mgrmap e18: smithi027.ufhhvb(active, since 77s), standbys: smithi115.tcyodz 2021-11-03T17:08:10.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:10 smithi115 conmon[34670]: audit 2021-11-03T17:08:09.483630+0000 mon.smithi027 (mon.0) 180 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mgr metadata", "who": "smithi115.tcyodz", "id": "smithi115.tcyodz"}]: dispatch 2021-11-03T17:08:10.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:10 smithi115 conmon[34670]: audit 2021-11-03T17:08:10.402736+0000 mon.smithi027 (mon.0) 181 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2021-11-03T17:08:10.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:10 smithi027 conmon[32322]: cephadm 2021-11-03T17:08:09.430393+0000 mgr.smithi027.ufhhvb (mgr 2021-11-03T17:08:10.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:10 smithi027 conmon[32322]: .14162) 48 : cephadm [INF] Deploying daemon node-exporter.smithi115 on smithi115 2021-11-03T17:08:10.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:10 smithi027 conmon[32322]: cluster 2021-11-03T17:08:09.483423+0000 mon.smithi027 (mon.0) 179 : cluster [DBG] mgrmap e18: smithi027.ufhhvb(active, since 77s), standbys: smithi115.tcyodz 2021-11-03T17:08:10.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:10 smithi027 conmon[32322]: audit 2021-11-03T17:08:09.483630+0000 mon.smithi027 (mon.0) 180 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mgr metadata", "who": "smithi115.tcyodz", "id": "smithi115.tcyodz"}]: dispatch 2021-11-03T17:08:10.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:10 smithi027 conmon[32322]: audit 2021- 2021-11-03T17:08:10.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:10 smithi027 conmon[32322]: 11-03T17:08:10.402736+0000 2021-11-03T17:08:10.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:10 smithi027 conmon[32322]: mon.smithi027 (mon.0) 181 : audit [DBG] 2021-11-03T17:08:10.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:10 smithi027 conmon[32322]: from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2021-11-03T17:08:11.584 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:11 smithi115 conmon[34670]: cluster 2021-11-03T17:08:10 2021-11-03T17:08:11.584 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:11 smithi115 conmon[34670]: .534024+0000 mgr.smithi027.ufhhvb (mgr.14162) 49 : cluster [DBG] pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:11.584 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:11 smithi115 conmon[34670]: audit 2021-11-03T17:08:10.565243+0000 mon.smithi027 (mon.0) 182 : audit [DBG] from='client.? 172.21.15.115:0/1194045692' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2021-11-03T17:08:11.604 INFO:tasks.cephadm:Generating final ceph.conf file... 2021-11-03T17:08:11.604 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph config generate-minimal-conf 2021-11-03T17:08:11.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:11 smithi027 conmon[32322]: cluster 2021-11-03T17:08:10.534024+0000 mgr.smithi027.ufhhvb (mgr. 2021-11-03T17:08:11.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:11 smithi027 conmon[32322]: 14162) 49 : cluster [DBG] pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:11.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:11 smithi027 conmon[32322]: audit 2021-11-03T17:08:10.565243+0000 mon.smithi027 (mon.0) 182 : audit [DBG] from='client.? 172.21.15.115:0/1194045692' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2021-11-03T17:08:12.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:12 smithi027 conmon[32322]: cluster 2021-11-03T17:08:12.534379+0000 mgr.smithi027.ufhhvb (mgr.14162) 50 : cluster [DBG] pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:12.862 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:12 smithi115 conmon[34670]: cluster 2021-11-03T17:08:12.534379+0000 mgr.smithi027.ufhhvb ( 2021-11-03T17:08:12.862 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:12 smithi115 conmon[34670]: mgr.14162) 50 : cluster [DBG] pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:13.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.484+0000 7f7cc49a5700 1 -- 172.21.15.27:0/1222396549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cc00f5410 msgr2=0x7f7cc00f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:08:13.480 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.484+0000 7f7cc49a5700 1 --2- 172.21.15.27:0/1222396549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cc00f5410 0x7f7cc00f5830 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f7cb0009fb0 tx=0x7f7cb0009b30).stop 2021-11-03T17:08:13.480 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.484+0000 7f7cc49a5700 1 -- 172.21.15.27:0/1222396549 shutdown_connections 2021-11-03T17:08:13.481 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.484+0000 7f7cc49a5700 1 --2- 172.21.15.27:0/1222396549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cc00f5410 0x7f7cc00f5830 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:08:13.481 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.484+0000 7f7cc49a5700 1 -- 172.21.15.27:0/1222396549 >> 172.21.15.27:0/1222396549 conn(0x7f7cc00f0b70 msgr2=0x7f7cc00f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:08:13.481 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.484+0000 7f7cc49a5700 1 -- 172.21.15.27:0/1222396549 shutdown_connections 2021-11-03T17:08:13.481 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.484+0000 7f7cc49a5700 1 -- 172.21.15.27:0/1222396549 wait complete. 2021-11-03T17:08:13.484 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.485+0000 7f7cc49a5700 1 Processor -- start 2021-11-03T17:08:13.484 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.485+0000 7f7cc49a5700 1 -- start start 2021-11-03T17:08:13.485 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.485+0000 7f7cc49a5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cc00f5410 0x7f7cc0109490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:08:13.485 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.485+0000 7f7cc49a5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cc0060240 0x7f7cc010b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:08:13.485 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.485+0000 7f7cc49a5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc010bf80 con 0x7f7cc0060240 2021-11-03T17:08:13.486 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.485+0000 7f7cc49a5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc010c0c0 con 0x7f7cc00f5410 2021-11-03T17:08:13.486 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.485+0000 7f7cbdd9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cc0060240 0x7f7cc010b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:08:13.486 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.485+0000 7f7cbdd9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cc0060240 0x7f7cc010b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54408/0 (socket says 172.21.15.27:54408) 2021-11-03T17:08:13.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.485+0000 7f7cbdd9b700 1 -- 172.21.15.27:0/3653274499 learned_addr learned my addr 172.21.15.27:0/3653274499 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:08:13.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.485+0000 7f7cbe59c700 1 --2- 172.21.15.27:0/3653274499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cc00f5410 0x7f7cc0109490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:08:13.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.486+0000 7f7cbe59c700 1 -- 172.21.15.27:0/3653274499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cc00f5410 msgr2=0x7f7cc0109490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2021-11-03T17:08:13.488 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.486+0000 7f7cbe59c700 1 -- 172.21.15.27:0/3653274499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cc00f5410 msgr2=0x7f7cc0109490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2021-11-03T17:08:13.488 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.486+0000 7f7cbe59c700 1 --2- 172.21.15.27:0/3653274499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cc00f5410 0x7f7cc0109490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2021-11-03T17:08:13.488 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.486+0000 7f7cbdd9b700 1 -- 172.21.15.27:0/3653274499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cc00f5410 msgr2=0x7f7cc0109490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:08:13.488 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.486+0000 7f7cbe59c700 1 --2- 172.21.15.27:0/3653274499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cc00f5410 0x7f7cc0109490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2021-11-03T17:08:13.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.486+0000 7f7cbdd9b700 1 --2- 172.21.15.27:0/3653274499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cc00f5410 0x7f7cc0109490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:08:13.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.486+0000 7f7cbdd9b700 1 -- 172.21.15.27:0/3653274499 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb0005040 con 0x7f7cc0060240 2021-11-03T17:08:13.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.486+0000 7f7cbdd9b700 1 --2- 172.21.15.27:0/3653274499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cc0060240 0x7f7cc010b990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f7cb40093c0 tx=0x7f7cb4006ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:08:13.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.486+0000 7f7caeffd700 1 -- 172.21.15.27:0/3653274499 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cb400e650 con 0x7f7cc0060240 2021-11-03T17:08:13.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.486+0000 7f7caeffd700 1 -- 172.21.15.27:0/3653274499 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f7cb400ec40 con 0x7f7cc0060240 2021-11-03T17:08:13.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.486+0000 7f7caeffd700 1 -- 172.21.15.27:0/3653274499 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7cb4011bb0 con 0x7f7cc0060240 2021-11-03T17:08:13.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.486+0000 7f7cc49a5700 1 -- 172.21.15.27:0/3653274499 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cc010c3a0 con 0x7f7cc0060240 2021-11-03T17:08:13.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.486+0000 7f7cc49a5700 1 -- 172.21.15.27:0/3653274499 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cc010c8e0 con 0x7f7cc0060240 2021-11-03T17:08:13.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.487+0000 7f7cc49a5700 1 -- 172.21.15.27:0/3653274499 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cc0045ad0 con 0x7f7cc0060240 2021-11-03T17:08:13.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.488+0000 7f7caeffd700 1 -- 172.21.15.27:0/3653274499 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f7cb4018030 con 0x7f7cc0060240 2021-11-03T17:08:13.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.488+0000 7f7caeffd700 1 --2- 172.21.15.27:0/3653274499 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f7ca805c400 0x7f7ca805e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:08:13.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.488+0000 7f7caeffd700 1 -- 172.21.15.27:0/3653274499 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1237+0+0 (secure 0 0 0) 0x7f7cb4074910 con 0x7f7cc0060240 2021-11-03T17:08:13.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.488+0000 7f7cbe59c700 1 --2- 172.21.15.27:0/3653274499 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f7ca805c400 0x7f7ca805e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:08:13.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.488+0000 7f7cbe59c700 1 --2- 172.21.15.27:0/3653274499 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f7ca805c400 0x7f7ca805e8c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f7cb0004dd0 tx=0x7f7cb000b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:08:13.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.494+0000 7f7caeffd700 1 -- 172.21.15.27:0/3653274499 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f7cb40452b0 con 0x7f7cc0060240 2021-11-03T17:08:13.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.665+0000 7f7cc49a5700 1 -- 172.21.15.27:0/3653274499 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f7cc00fae90 con 0x7f7cc0060240 2021-11-03T17:08:13.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.665+0000 7f7caeffd700 1 -- 172.21.15.27:0/3653274499 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v12) v1 ==== 76+0+225 (secure 0 0 0) 0x7f7cb4016020 con 0x7f7cc0060240 2021-11-03T17:08:13.661 INFO:teuthology.orchestra.run.smithi027.stdout:# minimal ceph.conf for 32ab977e-3cc8-11ec-8c28-001a4aab830c 2021-11-03T17:08:13.661 INFO:teuthology.orchestra.run.smithi027.stdout:[global] 2021-11-03T17:08:13.661 INFO:teuthology.orchestra.run.smithi027.stdout: fsid = 32ab977e-3cc8-11ec-8c28-001a4aab830c 2021-11-03T17:08:13.661 INFO:teuthology.orchestra.run.smithi027.stdout: mon_host = [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] 2021-11-03T17:08:13.662 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.667+0000 7f7cc49a5700 1 -- 172.21.15.27:0/3653274499 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f7ca805c400 msgr2=0x7f7ca805e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:08:13.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.667+0000 7f7cc49a5700 1 --2- 172.21.15.27:0/3653274499 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f7ca805c400 0x7f7ca805e8c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f7cb0004dd0 tx=0x7f7cb000b040).stop 2021-11-03T17:08:13.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.667+0000 7f7cc49a5700 1 -- 172.21.15.27:0/3653274499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cc0060240 msgr2=0x7f7cc010b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:08:13.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.667+0000 7f7cc49a5700 1 --2- 172.21.15.27:0/3653274499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cc0060240 0x7f7cc010b990 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f7cb40093c0 tx=0x7f7cb4006ee0).stop 2021-11-03T17:08:13.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.667+0000 7f7cc49a5700 1 -- 172.21.15.27:0/3653274499 shutdown_connections 2021-11-03T17:08:13.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.667+0000 7f7cc49a5700 1 --2- 172.21.15.27:0/3653274499 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f7ca805c400 0x7f7ca805e8c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:08:13.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.667+0000 7f7cc49a5700 1 --2- 172.21.15.27:0/3653274499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cc00f5410 0x7f7cc0109490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:08:13.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.667+0000 7f7cc49a5700 1 --2- 172.21.15.27:0/3653274499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cc0060240 0x7f7cc010b990 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:08:13.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.667+0000 7f7cc49a5700 1 -- 172.21.15.27:0/3653274499 >> 172.21.15.27:0/3653274499 conn(0x7f7cc00f0b70 msgr2=0x7f7cc00f15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:08:13.665 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.667+0000 7f7cc49a5700 1 -- 172.21.15.27:0/3653274499 shutdown_connections 2021-11-03T17:08:13.665 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:13.667+0000 7f7cc49a5700 1 -- 172.21.15.27:0/3653274499 wait complete. 2021-11-03T17:08:14.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:13 smithi115 conmon[34670]: audit 2021-11-03T17:08:12.956902+0000 mon.smithi027 (mon.0 2021-11-03T17:08:14.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:13 smithi115 conmon[34670]: ) 183 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:14.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:13 smithi115 conmon[34670]: audit 2021-11-03T17:08:12.957840+0000 mon.smithi027 (mon.0) 184 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:08:14.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:13 smithi115 conmon[34670]: audit 2021-11-03T17:08:13.666031+0000 mon.smithi027 (mon.0) 185 : audit [DBG] from='client.? 172.21.15.27:0/3653274499' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:08:14.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:13 smithi027 conmon[32322]: audit 2021-11-03T17:08:12.956902+0000 mon.smithi027 (mon. 2021-11-03T17:08:14.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:13 smithi027 conmon[32322]: 0) 183 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:14.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:13 smithi027 conmon[32322]: audit 2021-11-03T17:08:12.957840+0000 mon.smithi027 (mon.0) 184 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:08:14.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:13 smithi027 conmon[32322]: audit 2021-11-03T17:08:13.666031+0000 mon.smithi027 (mon.0) 185 : audit [DBG] from='client.? 172.21.15.27:0/3653274499' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:08:14.362 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2021-11-03T17:08:14.363 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2021-11-03T17:08:14.363 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.conf 2021-11-03T17:08:14.394 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2021-11-03T17:08:14.394 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2021-11-03T17:08:14.461 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2021-11-03T17:08:14.461 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/ceph.conf 2021-11-03T17:08:14.488 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2021-11-03T17:08:14.489 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2021-11-03T17:08:14.552 INFO:tasks.cephadm:Deploying OSDs... 2021-11-03T17:08:14.553 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2021-11-03T17:08:14.553 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/scratch_devs of=/dev/stdout 2021-11-03T17:08:14.569 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2021-11-03T17:08:14.569 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2021-11-03T17:08:14.625 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2021-11-03T17:08:14.625 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-11-03T17:08:14.626 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 6h/6d Inode: 110227 Links: 1 2021-11-03T17:08:14.626 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-11-03T17:08:14.626 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2021-11-03T17:08:14.626 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2021-11-03 17:07:47.913082331 +0000 2021-11-03T17:08:14.627 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2021-11-03 17:01:14.916827623 +0000 2021-11-03T17:08:14.627 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2021-11-03 17:01:14.916827623 +0000 2021-11-03T17:08:14.627 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: - 2021-11-03T17:08:14.636 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2021-11-03T17:08:14.636 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2021-11-03T17:08:14.636 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000235602 s, 2.2 MB/s 2021-11-03T17:08:14.639 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2021-11-03T17:08:14.695 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2021-11-03T17:08:14.695 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-11-03T17:08:14.695 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 6h/6d Inode: 111942 Links: 1 2021-11-03T17:08:14.696 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-11-03T17:08:14.696 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2021-11-03T17:08:14.696 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2021-11-03 17:07:47.913082331 +0000 2021-11-03T17:08:14.696 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2021-11-03 17:01:15.271818506 +0000 2021-11-03T17:08:14.696 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2021-11-03 17:01:15.271818506 +0000 2021-11-03T17:08:14.697 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: - 2021-11-03T17:08:14.705 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2021-11-03T17:08:14.705 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2021-11-03T17:08:14.705 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000218698 s, 2.3 MB/s 2021-11-03T17:08:14.708 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2021-11-03T17:08:14.761 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2021-11-03T17:08:14.762 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-11-03T17:08:14.762 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 6h/6d Inode: 111998 Links: 1 2021-11-03T17:08:14.762 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-11-03T17:08:14.763 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2021-11-03T17:08:14.763 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2021-11-03 17:07:47.914082305 +0000 2021-11-03T17:08:14.763 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2021-11-03 17:01:15.615809671 +0000 2021-11-03T17:08:14.763 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2021-11-03 17:01:15.615809671 +0000 2021-11-03T17:08:14.764 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: - 2021-11-03T17:08:14.771 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2021-11-03T17:08:14.772 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2021-11-03T17:08:14.772 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000219319 s, 2.3 MB/s 2021-11-03T17:08:14.775 DEBUG:teuthology.orchestra.run.smithi027:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2021-11-03T17:08:14.829 INFO:teuthology.orchestra.run.smithi027.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2021-11-03T17:08:14.831 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-11-03T17:08:14.831 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 6h/6d Inode: 112069 Links: 1 2021-11-03T17:08:14.831 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-11-03T17:08:14.831 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:device_t:s0 2021-11-03T17:08:14.832 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 2021-11-03 17:07:47.914082305 +0000 2021-11-03T17:08:14.832 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2021-11-03 17:01:15.958800862 +0000 2021-11-03T17:08:14.832 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2021-11-03 17:01:15.958800862 +0000 2021-11-03T17:08:14.832 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: - 2021-11-03T17:08:14.839 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records in 2021-11-03T17:08:14.839 INFO:teuthology.orchestra.run.smithi027.stderr:1+0 records out 2021-11-03T17:08:14.840 INFO:teuthology.orchestra.run.smithi027.stderr:512 bytes copied, 0.000186847 s, 2.7 MB/s 2021-11-03T17:08:14.842 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2021-11-03T17:08:14.842 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/scratch_devs of=/dev/stdout 2021-11-03T17:08:14.859 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2021-11-03T17:08:14.859 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2021-11-03T17:08:14.913 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2021-11-03T17:08:14.913 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-11-03T17:08:14.913 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 6h/6d Inode: 117300 Links: 1 2021-11-03T17:08:14.913 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-11-03T17:08:14.914 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2021-11-03T17:08:14.914 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2021-11-03 17:08:09.877713536 +0000 2021-11-03T17:08:14.915 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2021-11-03 17:00:35.814618399 +0000 2021-11-03T17:08:14.915 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2021-11-03 17:00:35.814618399 +0000 2021-11-03T17:08:14.915 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: - 2021-11-03T17:08:14.922 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2021-11-03T17:08:14.923 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2021-11-03T17:08:14.923 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000247895 s, 2.1 MB/s 2021-11-03T17:08:14.925 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2021-11-03T17:08:14.980 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2021-11-03T17:08:14.980 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-11-03T17:08:14.980 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 6h/6d Inode: 114454 Links: 1 2021-11-03T17:08:14.981 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-11-03T17:08:14.981 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2021-11-03T17:08:14.981 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2021-11-03 17:08:09.877713536 +0000 2021-11-03T17:08:14.981 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2021-11-03 17:00:36.142612265 +0000 2021-11-03T17:08:14.981 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2021-11-03 17:00:36.142612265 +0000 2021-11-03T17:08:14.982 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: - 2021-11-03T17:08:14.989 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2021-11-03T17:08:14.990 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2021-11-03T17:08:14.990 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000206284 s, 2.5 MB/s 2021-11-03T17:08:14.992 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2021-11-03T17:08:15.047 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2021-11-03T17:08:15.047 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-11-03T17:08:15.047 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 6h/6d Inode: 119810 Links: 1 2021-11-03T17:08:15.048 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-11-03T17:08:15.048 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2021-11-03T17:08:15.048 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2021-11-03 17:08:09.877713536 +0000 2021-11-03T17:08:15.048 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2021-11-03 17:00:36.476606020 +0000 2021-11-03T17:08:15.048 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2021-11-03 17:00:36.476606020 +0000 2021-11-03T17:08:15.049 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: - 2021-11-03T17:08:15.056 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2021-11-03T17:08:15.056 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2021-11-03T17:08:15.057 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000223252 s, 2.3 MB/s 2021-11-03T17:08:15.059 DEBUG:teuthology.orchestra.run.smithi115:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2021-11-03T17:08:15.114 INFO:teuthology.orchestra.run.smithi115.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2021-11-03T17:08:15.114 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2021-11-03T17:08:15.114 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 6h/6d Inode: 112621 Links: 1 2021-11-03T17:08:15.115 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2021-11-03T17:08:15.115 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:device_t:s0 2021-11-03T17:08:15.115 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 2021-11-03 17:08:09.878713516 +0000 2021-11-03T17:08:15.115 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2021-11-03 17:00:36.812599737 +0000 2021-11-03T17:08:15.116 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2021-11-03 17:00:36.812599737 +0000 2021-11-03T17:08:15.116 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: - 2021-11-03T17:08:15.123 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records in 2021-11-03T17:08:15.123 INFO:teuthology.orchestra.run.smithi115.stderr:1+0 records out 2021-11-03T17:08:15.124 INFO:teuthology.orchestra.run.smithi115.stderr:512 bytes copied, 0.000153528 s, 3.3 MB/s 2021-11-03T17:08:15.126 INFO:tasks.cephadm:Deploying osd.0 on smithi027 with /dev/vg_nvme/lv_4... 2021-11-03T17:08:15.126 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2021-11-03T17:08:15.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:15 smithi027 conmon[32322]: cluster 2021-11-03T17:08:14.534612+0000 mgr.smithi027.ufhhvb (mgr.14162) 51 : cluster [DBG] pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:15.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:15 smithi027 conmon[32322]: audit 2021-11-03T17:08:14.605911+0000 mon.smithi027 (mon.0) 186 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:15.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:15 smithi115 conmon[34670]: cluster 2021-11-03T17:08:14.534612+0000 mgr.smithi027.ufhhvb ( 2021-11-03T17:08:16.048 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:15 smithi115 conmon[34670]: mgr.14162) 51 : cluster [DBG] pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:16.049 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:15 smithi115 conmon[34670]: audit 2021-11-03T17:08:14.605911+0000 mon.smithi027 (mon.0) 186 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:17.511 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping: /dev/vg_nvme/lv_4 2021-11-03T17:08:17.511 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2021-11-03T17:08:17.512 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10+0 records in 2021-11-03T17:08:17.512 INFO:teuthology.orchestra.run.smithi027.stderr:10+0 records out 2021-11-03T17:08:17.512 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0108799 s, 964 MB/s 2021-11-03T17:08:17.512 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping successful for: 2021-11-03T17:08:17.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:17 smithi115 conmon[34670]: cluster 2021-11-03T17:08:16.534907+0000 mgr.smithi027.ufhhvb (mgr.14162) 52 2021-11-03T17:08:17.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:17 smithi115 conmon[34670]: : cluster [DBG] pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:17.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:17 smithi115 conmon[34670]: audit 2021-11-03T17:08:17.491406+0000 mon.smithi027 (mon.0) 187 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:18.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:17 smithi027 conmon[32322]: cluster 2021-11-03T17:08:16.534907+0000 mgr.smithi027.ufhhvb (mgr.14162) 52 : cluster [DBG] pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:18.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:17 smithi027 conmon[32322]: audit 2021-11-03T17:08:17.491406+0000 mon.smithi027 (mon.0) 187 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:18.047 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph orch daemon add osd smithi027:vg_nvme/lv_4 2021-11-03T17:08:19.188 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:18 smithi027 conmon[32322]: audit 2021-11-03T17:08:17.928064+0000 mon.smithi027 (mon.0) 188 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:19.188 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:18 smithi027 conmon[32322]: cephadm 2021-11-03T17:08:17.930072+0000 mgr.smithi027.ufhhvb (mgr.14162) 53 : cephadm [INF] Reconfiguring mon.smithi027 (unknown last config time)... 2021-11-03T17:08:19.189 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:18 smithi027 conmon[32322]: audit 2021-11-03T17:08:17.930292+0000 mon.smithi027 (mon.0) 189 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-11-03T17:08:19.189 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:18 smithi027 conmon[32322]: audit 2021-11-03T17:08:17.930957+0000 mon.smithi027 (mon.0) 190 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-11-03T17:08:19.189 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:18 smithi027 conmon[32322]: audit 2021-11-03T17:08:17.931583+0000 mon.smithi027 (mon.0) 191 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:08:19.190 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:18 smithi027 conmon[32322]: cephadm 2021-11-03T17:08:17.932137+0000 mgr.smithi027.ufhhvb (mgr.14162) 54 : cephadm [INF] Reconfiguring daemon mon.smithi027 on smithi027 2021-11-03T17:08:19.190 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:18 smithi027 conmon[32322]: cluster 2021-11-03T17:08:18.535134+0000 mgr.smithi027.ufhhvb (mgr.14162) 55 : cluster [DBG] pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:19.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:18 smithi115 conmon[34670]: audit 2021-11-03T17:08:17.928064+0000 2021-11-03T17:08:19.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:18 smithi115 conmon[34670]: mon.smithi027 (mon.0) 188 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:19.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:18 smithi115 conmon[34670]: cephadm 2021-11-03T17:08:17.930072+0000 mgr.smithi027.ufhhvb (mgr.14162) 2021-11-03T17:08:19.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:18 smithi115 conmon[34670]: 53 : cephadm [INF] Reconfiguring mon.smithi027 (unknown last config time)... 2021-11-03T17:08:19.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:18 smithi115 conmon[34670]: audit 2021-11-03T17 2021-11-03T17:08:19.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:18 smithi115 conmon[34670]: :08:17.930292+0000 mon.smithi027 (mon.0) 189 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-11-03T17:08:19.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:18 smithi115 conmon[34670]: 2021-11-03T17:08:19.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:18 smithi115 conmon[34670]: audit 2021- 2021-11-03T17:08:19.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:18 smithi115 conmon[34670]: 11-03T17:08:17.930957+0000 mon.smithi027 (mon.0) 190 2021-11-03T17:08:19.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:18 smithi115 conmon[34670]: : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-11-03T17:08:19.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:18 smithi115 conmon[34670]: audit 2021-11-03T 2021-11-03T17:08:19.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:18 smithi115 conmon[34670]: 17:08:17.931583+0000 mon.smithi027 (mon.0 2021-11-03T17:08:19.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:18 smithi115 conmon[34670]: ) 191 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:08:19.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:18 smithi115 conmon[34670]: cephadm 2021-11-03 2021-11-03T17:08:19.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:18 smithi115 conmon[34670]: T17:08:17.932137+0000 mgr.smithi027.ufhhvb (mgr 2021-11-03T17:08:19.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:18 smithi115 conmon[34670]: .14162) 54 : cephadm [INF] Reconfiguring daemon mon.smithi027 on smithi027 2021-11-03T17:08:19.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:18 smithi115 conmon[34670]: cluster 2021- 2021-11-03T17:08:19.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:18 smithi115 conmon[34670]: 11-03T17:08:18.535134+0000 mgr.smithi027.ufhhvb 2021-11-03T17:08:19.241 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:18 smithi115 conmon[34670]: (mgr.14162) 55 : cluster [DBG] pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:20.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:20.225+0000 7fe7a2e8c700 1 -- 172.21.15.27:0/409750712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe79c0f6860 msgr2=0x7fe79c0f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:08:20.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:20.225+0000 7fe7a2e8c700 1 --2- 172.21.15.27:0/409750712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe79c0f6860 0x7fe79c0f6cc0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fe790004660 tx=0x7fe790009b30).stop 2021-11-03T17:08:20.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:20.225+0000 7fe7a2e8c700 1 -- 172.21.15.27:0/409750712 shutdown_connections 2021-11-03T17:08:20.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:20.225+0000 7fe7a2e8c700 1 --2- 172.21.15.27:0/409750712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe79c0f6860 0x7fe79c0f6cc0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:08:20.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:20.225+0000 7fe7a2e8c700 1 --2- 172.21.15.27:0/409750712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe79c0f55c0 0x7fe79c0f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:08:20.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:20.225+0000 7fe7a2e8c700 1 -- 172.21.15.27:0/409750712 >> 172.21.15.27:0/409750712 conn(0x7fe79c0f0d40 msgr2=0x7fe79c0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:08:20.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:20.225+0000 7fe7a2e8c700 1 -- 172.21.15.27:0/409750712 shutdown_connections 2021-11-03T17:08:20.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:20.225+0000 7fe7a2e8c700 1 -- 172.21.15.27:0/409750712 wait complete. 2021-11-03T17:08:20.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:20.226+0000 7fe7a2e8c700 1 Processor -- start 2021-11-03T17:08:20.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:20.226+0000 7fe7a2e8c700 1 -- start start 2021-11-03T17:08:20.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:20.226+0000 7fe7a2e8c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe79c0f55c0 0x7fe79c106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:08:20.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:20.226+0000 7fe7a2e8c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe79c0f6860 0x7fe79c107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:08:20.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:20.226+0000 7fe7a2e8c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c107a10 con 0x7fe79c0f55c0 2021-11-03T17:08:20.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:20.226+0000 7fe7a2e8c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79c107b50 con 0x7fe79c0f6860 2021-11-03T17:08:20.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:20.226+0000 7fe7a0c28700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe79c0f55c0 0x7fe79c106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:08:20.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:20.226+0000 7fe7a0c28700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe79c0f55c0 0x7fe79c106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54418/0 (socket says 172.21.15.27:54418) 2021-11-03T17:08:20.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:20.226+0000 7fe7a0c28700 1 -- 172.21.15.27:0/486989890 learned_addr learned my addr 172.21.15.27:0/486989890 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:08:20.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:20.226+0000 7fe7a0c28700 1 -- 172.21.15.27:0/486989890 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe79c0f6860 msgr2=0x7fe79c107460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:08:20.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:20.226+0000 7fe7a0c28700 1 --2- 172.21.15.27:0/486989890 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe79c0f6860 0x7fe79c107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:08:20.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:20.226+0000 7fe7a0c28700 1 -- 172.21.15.27:0/486989890 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe790005040 con 0x7fe79c0f55c0 2021-11-03T17:08:20.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:20.227+0000 7fe7a0c28700 1 --2- 172.21.15.27:0/486989890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe79c0f55c0 0x7fe79c106f70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fe79400d7f0 tx=0x7fe794004e80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:08:20.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:20.227+0000 7fe7997fa700 1 -- 172.21.15.27:0/486989890 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe79400b4f0 con 0x7fe79c0f55c0 2021-11-03T17:08:20.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:20.227+0000 7fe7997fa700 1 -- 172.21.15.27:0/486989890 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7fe79400f040 con 0x7fe79c0f55c0 2021-11-03T17:08:20.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:20.227+0000 7fe7a2e8c700 1 -- 172.21.15.27:0/486989890 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe79c1084f0 con 0x7fe79c0f55c0 2021-11-03T17:08:20.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:20.227+0000 7fe7997fa700 1 -- 172.21.15.27:0/486989890 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe794013940 con 0x7fe79c0f55c0 2021-11-03T17:08:20.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:20.227+0000 7fe7a2e8c700 1 -- 172.21.15.27:0/486989890 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe79c108ab0 con 0x7fe79c0f55c0 2021-11-03T17:08:20.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:20.228+0000 7fe7a2e8c700 1 -- 172.21.15.27:0/486989890 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe79c045ad0 con 0x7fe79c0f55c0 2021-11-03T17:08:20.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:20.228+0000 7fe7997fa700 1 -- 172.21.15.27:0/486989890 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fe79401a030 con 0x7fe79c0f55c0 2021-11-03T17:08:20.230 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:20.228+0000 7fe7997fa700 1 --2- 172.21.15.27:0/486989890 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fe78405c2b0 0x7fe78405e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:08:20.230 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:20.229+0000 7fe7997fa700 1 -- 172.21.15.27:0/486989890 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1237+0+0 (secure 0 0 0) 0x7fe794074dd0 con 0x7fe79c0f55c0 2021-11-03T17:08:20.230 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:20.229+0000 7fe79bfff700 1 --2- 172.21.15.27:0/486989890 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fe78405c2b0 0x7fe78405e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:08:20.230 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:20.229+0000 7fe79bfff700 1 --2- 172.21.15.27:0/486989890 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fe78405c2b0 0x7fe78405e770 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fe790004e30 tx=0x7fe790009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:08:20.231 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:20.233+0000 7fe7997fa700 1 -- 172.21.15.27:0/486989890 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fe794048a00 con 0x7fe79c0f55c0 2021-11-03T17:08:20.429 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:20.434+0000 7fe7a2e8c700 1 -- 172.21.15.27:0/486989890 --> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fe79c0faca0 con 0x7fe78405c2b0 2021-11-03T17:08:20.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:20 smithi115 conmon[34670]: audit 2021-11-03T17:08:19.676177+0000 mon.smithi027 (mon.0) 192 2021-11-03T17:08:20.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:20 smithi115 conmon[34670]: : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:20.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:20 smithi115 conmon[34670]: cephadm 2021-11-03T17:08:19.676672+0000 mgr.smithi027.ufhhvb (mgr.14162) 56 : cephadm [INF] Reconfiguring mgr.smithi027.ufhhvb (unknown last config time)... 2021-11-03T17:08:20.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:20 smithi115 conmon[34670]: audit 2021-11-03T17:08:19.676913+0000 mon.smithi027 (mon.0) 193 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi027.ufhhvb", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2021-11-03T17:08:20.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:20 smithi115 conmon[34670]: 2021-11-03T17:08:20.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:20 smithi115 conmon[34670]: audit 2021-11-03 2021-11-03T17:08:20.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:20 smithi115 conmon[34670]: T17:08:19.677608+0000 mon.smithi027 (mon.0) 194 : audit 2021-11-03T17:08:20.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:20 smithi115 conmon[34670]: [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mgr services"}]: dispatch 2021-11-03T17:08:20.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:20 smithi115 conmon[34670]: audit 2021-11- 2021-11-03T17:08:20.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:20 smithi115 conmon[34670]: 03T17:08:19.678192+0000 mon.smithi027 ( 2021-11-03T17:08:20.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:20 smithi115 conmon[34670]: mon.0) 195 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:08:20.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:20 smithi115 conmon[34670]: cephadm 2021- 2021-11-03T17:08:20.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:20 smithi115 conmon[34670]: 11-03T17:08:19.678748+0000 mgr.smithi027.ufhhvb 2021-11-03T17:08:20.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:20 smithi115 conmon[34670]: (mgr.14162) 57 : cephadm [INF] Reconfiguring daemon mgr.smithi027.ufhhvb on smithi027 2021-11-03T17:08:20.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:20 smithi115 conmon[34670]: audit 2021-11-03T17: 2021-11-03T17:08:20.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:20 smithi115 conmon[34670]: 08:20.435990+0000 mgr.smithi027.ufhhvb (mgr.14162) 58 : audit 2021-11-03T17:08:20.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:20 smithi115 conmon[34670]: [DBG] from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:08:20.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:20 smithi115 conmon[34670]: audit 2021-11-03T17: 2021-11-03T17:08:20.991 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:20 smithi115 conmon[34670]: 08:20.436747+0000 mon.smithi027 (mon.0) 2021-11-03T17:08:20.991 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:20 smithi115 conmon[34670]: 196 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2021-11-03T17:08:20.991 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:20 smithi115 conmon[34670]: cephadm 2021-11-03T 2021-11-03T17:08:20.992 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:20 smithi115 conmon[34670]: 17:08:20.438078+0000 mgr.smithi027.ufhhvb (mgr.14162 2021-11-03T17:08:20.992 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:20 smithi115 conmon[34670]: ) 59 : cephadm [INF] Applying service osd.None on host smithi027... 2021-11-03T17:08:20.993 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:20 smithi115 conmon[34670]: audit 2021-11 2021-11-03T17:08:20.993 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:20 smithi115 conmon[34670]: -03T17:08:20.438287+0000 2021-11-03T17:08:20.994 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:20 smithi115 conmon[34670]: mon.smithi027 (mon.0) 197 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2021-11-03T17:08:20.994 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:20 smithi115 conmon[34670]: audit 2021- 2021-11-03T17:08:20.995 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:20 smithi115 conmon[34670]: 11-03T17:08:20.438851+0000 mon.smithi027 2021-11-03T17:08:20.995 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:20 smithi115 conmon[34670]: (mon.0) 198 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:08:20.995 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:20 smithi115 conmon[34670]: cluster 2021-11 2021-11-03T17:08:20.995 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:20 smithi115 conmon[34670]: -03T17:08:20.535377+0000 mgr.smithi027.ufhhvb 2021-11-03T17:08:20.996 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:20 smithi115 conmon[34670]: (mgr.14162) 60 : cluster [DBG] pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:21.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:20 smithi027 conmon[32322]: audit 2021-11-03T17:08:19.676177+0000 mon.smithi027 (mon.0) 192 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:21.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:20 smithi027 conmon[32322]: cephadm 2021-11-03T17:08:19.676672+0000 mgr.smithi027.ufhhvb (mgr.14162) 56 : cephadm [INF] Reconfiguring mgr.smithi027.ufhhvb (unknown last config time)... 2021-11-03T17:08:21.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:20 smithi027 conmon[32322]: audit 2021-11-03T17:08:19.676913+0000 mon.smithi027 (mon.0) 193 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi027.ufhhvb", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2021-11-03T17:08:21.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:20 smithi027 conmon[32322]: audit 2021-11-03T17:08:19.677608+0000 mon.smithi027 (mon.0) 194 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mgr services"}]: dispatch 2021-11-03T17:08:21.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:20 smithi027 conmon[32322]: audit 2021-11-03T17:08:19.678192+0000 mon.smithi027 (mon.0) 195 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:08:21.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:20 smithi027 conmon[32322]: cephadm 2021-11-03T17:08:19.678748+0000 mgr.smithi027.ufhhvb (mgr.14162) 57 : cephadm [INF] Reconfiguring daemon mgr.smithi027.ufhhvb on smithi027 2021-11-03T17:08:21.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:20 smithi027 conmon[32322]: audit 2021-11-03T17:08:20.435990+0000 mgr.smithi027.ufhhvb (mgr.14162) 58 : audit [DBG] from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:08:21.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:20 smithi027 conmon[32322]: audit 2021- 2021-11-03T17:08:21.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:20 smithi027 conmon[32322]: 11-03T17:08:20.436747+0000 mon.smithi027 (mon.0) 196 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2021-11-03T17:08:21.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:20 smithi027 conmon[32322]: cephadm 2021-11-03T17:08:20.438078+0000 mgr.smithi027.ufhhvb (mgr.14162) 59 : cephadm [INF] Applying service osd.None on host smithi027... 2021-11-03T17:08:21.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:20 smithi027 conmon[32322]: audit 2021-11-03T17:08:20.438287+0000 mon.smithi027 (mon.0) 197 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2021-11-03T17:08:21.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:20 smithi027 conmon[32322]: audit 2021-11-03T17:08:20.438851+0000 mon.smithi027 (mon.0) 198 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:08:21.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:20 smithi027 conmon[32322]: cluster 2021-11-03T17:08:20.535377+0000 mgr.smithi027.ufhhvb (mgr.14162) 60 : cluster [DBG] pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:22.692 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:22 smithi027 conmon[32322]: audit 2021-11-03T17:08:21.290865+0000 mon.smithi027 ( 2021-11-03T17:08:22.693 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:22 smithi027 conmon[32322]: mon.0) 199 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:22.693 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:22 smithi027 conmon[32322]: cephadm 2021-11-03T17:08:21.291620+0000 mgr.smithi027.ufhhvb (mgr.14162) 61 : cephadm [INF] Reconfiguring alertmanager.smithi027 (dependencies changed)... 2021-11-03T17:08:22.693 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:22 smithi027 conmon[32322]: cephadm 2021-11-03T17:08:21.294703+0000 mgr.smithi027.ufhhvb (mgr.14162) 62 : cephadm [INF] Reconfiguring daemon alertmanager.smithi027 on smithi027 2021-11-03T17:08:22.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:22 smithi115 conmon[34670]: audit 2021-11-03T17:08:21.290865+0000 mon.smithi027 (mon. 2021-11-03T17:08:22.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:22 smithi115 conmon[34670]: 0) 199 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:22.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:22 smithi115 conmon[34670]: cephadm 2021-11-03T17:08:21.291620+0000 mgr.smithi027.ufhhvb (mgr.14162) 61 : cephadm [INF] Reconfiguring alertmanager.smithi027 (dependencies changed)... 2021-11-03T17:08:22.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:22 smithi115 conmon[34670]: cephadm 2021-11-03T17:08:21. 2021-11-03T17:08:22.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:22 smithi115 conmon[34670]: 294703+0000 mgr.smithi027.ufhhvb (mgr.14162) 62 : cephadm [INF] 2021-11-03T17:08:22.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:22 smithi115 conmon[34670]: Reconfiguring daemon alertmanager.smithi027 on smithi027 2021-11-03T17:08:23.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:23 smithi027 conmon[32322]: cluster 2021-11-03T17:08:22.535599+0000 2021-11-03T17:08:23.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:23 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14162) 63 : cluster [DBG] pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:23.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:23 smithi115 conmon[34670]: cluster 2021-11-03T17:08:22.535599+0000 2021-11-03T17:08:23.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:23 smithi115 conmon[34670]: mgr.smithi027.ufhhvb (mgr.14162) 63 : cluster [DBG] pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:24.843 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:24 smithi027 conmon[32322]: cluster 2021-11-03T17:08:24.535869+0000 2021-11-03T17:08:24.844 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:24 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14162) 64 : cluster [DBG] pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:24.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:24 smithi115 conmon[34670]: cluster 2021-11-03T17:08:24.535869+0000 mgr.smithi027.ufhhvb (mgr.14162) 64 : cluster [DBG] pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:27.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:27 smithi115 conmon[34670]: audit 2021-11-03T17:08:26.109893+0000 mon.smithi027 (mon.0) 200 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:27.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:27 smithi115 conmon[34670]: 2021-11-03T17:08:27.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:27 smithi115 conmon[34670]: cephadm 2021-11-03T17:08:26.110466+0000 mgr.smithi027.ufhhvb (mgr.14162) 65 : cephadm [INF] Reconfiguring crash.smithi027 (monmap changed)... 2021-11-03T17:08:27.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:27 smithi115 conmon[34670]: audit 2021-11-03T17:08:26.110758+0000 mon.smithi027 (mon.0) 201 : audit 2021-11-03T17:08:27.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:27 smithi115 conmon[34670]: [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi027", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2021-11-03T17:08:27.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:27 smithi115 conmon[34670]: audit 2021-11-03T17: 2021-11-03T17:08:27.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:27 smithi115 conmon[34670]: 08:26.111477+0000 mon.smithi027 (mon.0) 202 : 2021-11-03T17:08:27.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:27 smithi115 conmon[34670]: audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:08:27.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:27 smithi115 conmon[34670]: cephadm 2021-11-03T17: 2021-11-03T17:08:27.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:27 smithi115 conmon[34670]: 08:26.111994+0000 mgr.smithi027.ufhhvb (mgr.14162) 66 : 2021-11-03T17:08:27.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:27 smithi115 conmon[34670]: cephadm [INF] Reconfiguring daemon crash.smithi027 on smithi027 2021-11-03T17:08:27.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:27 smithi115 conmon[34670]: cluster 2021-11-03T17 2021-11-03T17:08:27.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:27 smithi115 conmon[34670]: :08:26.536112+0000 mgr.smithi027.ufhhvb (mgr.14162) 2021-11-03T17:08:27.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:27 smithi115 conmon[34670]: 67 : cluster [DBG] pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:27.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:27 smithi115 conmon[34670]: audit 2021-11-03T 2021-11-03T17:08:27.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:27 smithi115 conmon[34670]: 17:08:27.067063+0000 mon.smithi027 (mon.0 2021-11-03T17:08:27.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:27 smithi115 conmon[34670]: ) 203 : audit [INF] from='client.? 172.21.15.27:0/780357757' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f6bbb8f-4e3c-401a-b1bd-69ba9cdd9d13"}]: dispatch 2021-11-03T17:08:27.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:27 smithi115 conmon[34670]: audit 2021-11- 2021-11-03T17:08:27.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:27 smithi115 conmon[34670]: 03T17:08:27.071309+0000 mon.smithi027 (mon 2021-11-03T17:08:27.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:27 smithi115 conmon[34670]: .0) 204 : audit [INF] from='client.? 172.21.15.27:0/780357757' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2f6bbb8f-4e3c-401a-b1bd-69ba9cdd9d13"}]': finished 2021-11-03T17:08:27.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:27 smithi115 conmon[34670]: cluster 2021- 2021-11-03T17:08:27.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:27 smithi115 conmon[34670]: 11-03T17:08:27.071363+0000 2021-11-03T17:08:27.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:27 smithi115 conmon[34670]: mon.smithi027 (mon.0) 205 : cluster [DBG] osdmap e6: 1 total, 0 up, 1 in 2021-11-03T17:08:27.490 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:27 smithi115 conmon[34670]: 2021-11-03T17:08:27.490 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:27 smithi115 conmon[34670]: audit 2021-11-03T17:08: 2021-11-03T17:08:27.490 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:27 smithi115 conmon[34670]: 27.071471+0000 mon.smithi027 (mon.0) 206 2021-11-03T17:08:27.490 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:27 smithi115 conmon[34670]: : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-11-03T17:08:27.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:27 smithi027 conmon[32322]: audit 2021-11-03T17:08:26.109893+0000 mon.smithi027 (mon.0) 200 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:27.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:27 smithi027 conmon[32322]: cephadm 2021-11-03T17:08:26.110466+0000 mgr.smithi027.ufhhvb (mgr.14162) 65 : cephadm [INF] Reconfiguring crash.smithi027 (monmap changed)... 2021-11-03T17:08:27.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:27 smithi027 conmon[32322]: audit 2021-11-03T17:08:26.110758+0000 mon.smithi027 (mon.0) 201 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi027", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2021-11-03T17:08:27.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:27 smithi027 conmon[32322]: audit 2021-11-03T17:08:26.111477+0000 mon.smithi027 (mon.0) 202 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:08:27.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:27 smithi027 conmon[32322]: cephadm 2021-11-03T17:08:26.111994+0000 mgr.smithi027.ufhhvb (mgr.14162) 66 : cephadm [INF] Reconfiguring daemon crash.smithi027 on smithi027 2021-11-03T17:08:27.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:27 smithi027 conmon[32322]: cluster 2021-11-03T17:08:26.536112+0000 mgr.smithi027.ufhhvb (mgr.14162) 67 : cluster [DBG] pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:27.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:27 smithi027 conmon[32322]: audit 2021-11-03T17:08:27.067063+0000 mon.smithi027 (mon.0) 203 : audit [INF] from='client.? 172.21.15.27:0/780357757' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2f6bbb8f-4e3c-401a-b1bd-69ba9cdd9d13"}]: dispatch 2021-11-03T17:08:27.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:27 smithi027 conmon[32322]: audit 2021-11-03T17:08:27.071309+0000 mon.smithi027 (mon.0) 204 : audit [INF] from='client.? 172.21.15.27:0/780357757' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2f6bbb8f-4e3c-401a-b1bd-69ba9cdd9d13"}]': finished 2021-11-03T17:08:27.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:27 smithi027 conmon[32322]: cluster 2021-11-03T17:08:27.071363+0000 mon.smithi027 (mon.0) 205 : cluster [DBG] osdmap e6: 1 total, 0 up, 1 in 2021-11-03T17:08:27.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:27 smithi027 conmon[32322]: audit 2021-11-03T17:08:27.071471+0000 mon.smithi027 (mon.0) 206 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-11-03T17:08:28.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:28 smithi115 conmon[34670]: audit 2021-11-03T17:08:27.578713+0000 mon.smithi027 (mon.0) 207 : audit [DBG] from='client.? 172.21.15.27:0/3796429611' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2021-11-03T17:08:29.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:28 smithi027 conmon[32322]: audit 2021-11-03T17:08:27.578713+0000 mon.smithi027 (mon.0) 207 : audit [DBG] 2021-11-03T17:08:29.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:28 smithi027 conmon[32322]: from='client.? 172.21.15.27:0/3796429611' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2021-11-03T17:08:29.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:29 smithi115 conmon[34670]: cluster 2021-11-03T17:08:28.536370+0000 mgr.smithi027.ufhhvb (mgr.14162) 68 : cluster [DBG] pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:30.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:29 smithi027 conmon[32322]: cluster 2021-11-03T17: 2021-11-03T17:08:30.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:29 smithi027 conmon[32322]: 08:28.536370+0000 mgr.smithi027.ufhhvb (mgr.14162) 68 : cluster [DBG] pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:31.805 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:31 smithi027 conmon[32322]: cluster 2021-11-03T17:08:30.536767+0000 2021-11-03T17:08:31.805 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:31 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14162) 69 : cluster [DBG] pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:31.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:31 smithi115 conmon[34670]: cluster 2021-11-03T17:08:30.536767+0000 mgr.smithi027.ufhhvb (mgr.14162 2021-11-03T17:08:31.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:31 smithi115 conmon[34670]: ) 69 : cluster [DBG] pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:33.753 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:33 smithi027 conmon[32322]: cluster 2021-11-03T17:08:32.537203+0000 2021-11-03T17:08:33.753 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:33 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14162) 70 : cluster [DBG] pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:33.754 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:33 smithi027 conmon[32322]: audit 2021-11-03T17:08:33.026421+0000 mon.smithi027 (mon.0) 208 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:33.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:33 smithi115 conmon[34670]: cluster 2021-11-03T17:08:32.537203+0000 mgr.smithi027.ufhhvb (mgr.14162) 70 : cluster [DBG] pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:33.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:33 smithi115 conmon[34670]: audit 2021-11-03T17:08:33.026421+0000 mon.smithi027 (mon.0) 208 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:34.801 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:34 smithi027 conmon[32322]: cephadm 2021-11-03T17:08:33.026961+0000 mgr.smithi027.ufhhvb (mgr 2021-11-03T17:08:34.802 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:34 smithi027 conmon[32322]: .14162) 71 : cephadm [INF] Reconfiguring grafana.smithi027 (dependencies changed)... 2021-11-03T17:08:34.802 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:34 smithi027 conmon[32322]: cephadm 2021-11-03T17:08:33.031151+0000 mgr.smithi027.ufhhvb (mgr.14162) 72 : cephadm [INF] Reconfiguring daemon grafana.smithi027 on smithi027 2021-11-03T17:08:34.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:34 smithi115 conmon[34670]: cephadm 2021-11-03T17:08:33.026961+0000 mgr.smithi027.ufhhvb (mgr.14162) 71 : cephadm [INF] Reconfiguring grafana.smithi027 (dependencies changed)... 2021-11-03T17:08:34.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:34 smithi115 conmon[34670]: cephadm 2021-11-03T17:08:33.031151+0000 mgr.smithi027.ufhhvb (mgr.14162) 72 : cephadm [INF] Reconfiguring daemon grafana.smithi027 on smithi027 2021-11-03T17:08:35.797 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:35 smithi027 conmon[32322]: cluster 2021-11-03T17:08:34.537538+0000 mgr.smithi027.ufhhvb (mgr. 2021-11-03T17:08:35.797 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:35 smithi027 conmon[32322]: 14162) 73 : cluster [DBG] pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:35.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:35 smithi115 conmon[34670]: cluster 2021-11-03T17:08:34.537538+0000 mgr.smithi027.ufhhvb (mgr.14162) 73 : cluster [DBG] pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:37.739 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:37 smithi027 conmon[32322]: cluster 2021-11-03T17:08:36.537901+0000 mgr.smithi027.ufhhvb (mgr.14162) 74 : cluster [DBG] 2021-11-03T17:08:37.739 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:37 smithi027 conmon[32322]: pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:37.740 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:37 smithi027 conmon[32322]: audit 2021-11-03T17:08:37.071945+0000 mon.smithi027 (mon.0) 209 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:37.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:37 smithi115 conmon[34670]: cluster 2021-11-03T17:08:36.537901+0000 mgr.smithi027.ufhhvb (mgr.14162) 74 : cluster [DBG] pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:37.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:37 smithi115 conmon[34670]: audit 2021-11-03T17:08:37.071945+0000 mon.smithi027 2021-11-03T17:08:37.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:37 smithi115 conmon[34670]: (mon.0) 209 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:38.699 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:38 smithi027 conmon[32322]: cephadm 2021-11-03T17:08:37.072720+0000 mgr.smithi027.ufhhvb (mgr.14162) 75 : cephadm [INF] Reconfiguring prometheus.smithi027 (dependencies changed)... 2021-11-03T17:08:38.699 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:38 smithi027 conmon[32322]: cephadm 2021-11-03T17:08:37.075239+0000 mgr.smithi027.ufhhvb (mgr.14162) 76 : cephadm [INF] Reconfiguring daemon prometheus.smithi027 on smithi027 2021-11-03T17:08:38.700 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:38 smithi027 conmon[32322]: audit 2021-11-03T17:08:38.421232+0000 mon.smithi027 (mon.0) 210 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2021-11-03T17:08:38.700 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:38 smithi027 conmon[32322]: audit 2021-11-03T17:08:38.422077+0000 mon.smithi027 (mon.0) 211 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:08:38.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:38 smithi115 conmon[34670]: cephadm 2021-11-03T17:08:37.072720+0000 mgr.smithi027.ufhhvb (mgr.14162) 75 : cephadm [INF] Reconfiguring prometheus.smithi027 (dependencies changed)... 2021-11-03T17:08:38.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:38 smithi115 conmon[34670]: cephadm 2021-11-03T 2021-11-03T17:08:38.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:38 smithi115 conmon[34670]: 17:08:37.075239+0000 mgr.smithi027.ufhhvb (mgr.14162) 76 : cephadm [INF] Reconfiguring daemon prometheus.smithi027 on smithi027 2021-11-03T17:08:38.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:38 smithi115 conmon[34670]: audit 2021-11-03T17:08:38.421232+0000 mon.smithi027 (mon.0) 210 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2021-11-03T17:08:38.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:38 smithi115 conmon[34670]: 2021-11-03T17:08:38.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:38 smithi115 conmon[34670]: audit 2021-11-03T17: 2021-11-03T17:08:38.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:38 smithi115 conmon[34670]: 08:38.422077+0000 mon.smithi027 (mon.0) 211 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:08:39.924 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:39 smithi027 conmon[32322]: cephadm 2021-11-03T17:08:38.422755+0000 mgr.smithi027.ufhhvb ( 2021-11-03T17:08:39.925 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:39 smithi027 conmon[32322]: mgr.14162) 77 : cephadm [INF] Deploying daemon osd.0 on smithi027 2021-11-03T17:08:39.925 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:39 smithi027 conmon[32322]: cluster 2021-11-03T17:08:38.538170+0000 mgr.smithi027.ufhhvb (mgr.14162) 78 : cluster [DBG] pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:39.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:39 smithi115 conmon[34670]: cephadm 2021-11-03T17:08:38.422755+0000 mgr.smithi027.ufhhvb (mgr.14162) 77 : cephadm [INF] Deploying daemon osd.0 on smithi027 2021-11-03T17:08:39.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:39 smithi115 conmon[34670]: cluster 2021-11-03T17: 2021-11-03T17:08:39.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:39 smithi115 conmon[34670]: 08:38.538170+0000 mgr.smithi027.ufhhvb (mgr.14162) 78 : cluster [DBG] pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:41.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:41 smithi115 conmon[34670]: cluster 2021-11-03T17:08:40.538647+0000 mgr.smithi027.ufhhvb (mgr.14162) 79 : cluster [DBG] pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:41.995 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:41 smithi027 conmon[32322]: cluster 2021-11-03T17:08:40.538647+0000 mgr.smithi027.ufhhvb (mgr. 2021-11-03T17:08:41.995 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:41 smithi027 conmon[32322]: 14162) 79 : cluster [DBG] pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:43.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:43 smithi115 conmon[34670]: audit 2021-11-03T17:08:42.366251+0000 mon.smithi027 (mon.0) 212 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:43.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:43 smithi115 conmon[34670]: cephadm 2021-11-03T17:08:42.366773+0000 mgr.smithi027.ufhhvb (mgr.14162) 80 : cephadm [INF] Reconfiguring crash.smithi115 (monmap changed)... 2021-11-03T17:08:43.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:43 smithi115 conmon[34670]: audit 2021-11-03T17:08:42.367021+0000 mon.smithi027 (mon.0) 213 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi115", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2021-11-03T17:08:43.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:43 smithi115 conmon[34670]: audit 2021-11-03T17:08:42.367674+0000 mon.smithi027 (mon.0) 214 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:08:43.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:43 smithi115 conmon[34670]: cephadm 2021-11-03T17:08:42.368150+0000 mgr.smithi027.ufhhvb (mgr.14162) 81 : cephadm [INF] Reconfiguring daemon crash.smithi115 on smithi115 2021-11-03T17:08:43.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:43 smithi115 conmon[34670]: cluster 2021-11-03T17:08:42.538919+0000 mgr.smithi027.ufhhvb (mgr.14162) 82 : cluster [DBG] pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:43.739 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:43 smithi027 conmon[32322]: audit 2021-11-03T17:08:42.366251+0000 mon.smithi027 (mon.0) 212 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:43.739 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:43 smithi027 conmon[32322]: cephadm 2021-11-03T17:08:42.366773+0000 mgr.smithi027.ufhhvb (mgr.14162) 80 : cephadm [INF] Reconfiguring crash.smithi115 (monmap changed)... 2021-11-03T17:08:43.739 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:43 smithi027 conmon[32322]: audit 2021-11-03T17:08:42.367021+0000 mon.smithi027 (mon.0) 213 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi115", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2021-11-03T17:08:43.740 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:43 smithi027 conmon[32322]: audit 2021-11-03T17:08:42.367674+0000 mon.smithi027 (mon.0) 214 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:08:43.740 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:43 smithi027 conmon[32322]: cephadm 2021-11-03T17:08:42.368150+0000 mgr.smithi027.ufhhvb (mgr.14162) 81 : cephadm [INF] Reconfiguring daemon crash.smithi115 on smithi115 2021-11-03T17:08:43.740 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:43 smithi027 conmon[32322]: cluster 2021-11-03T17:08:42.538919+0000 mgr.smithi027.ufhhvb (mgr.14162) 82 : cluster [DBG] pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:44.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:44 smithi115 conmon[34670]: audit 2021-11-03T17:08:43.878779+0000 mon.smithi027 (mon.0) 215 2021-11-03T17:08:44.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:44 smithi115 conmon[34670]: : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:44.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:44 smithi115 conmon[34670]: cephadm 2021-11-03T17:08:43.879316+0000 mgr.smithi027.ufhhvb (mgr.14162) 83 : cephadm [INF] Reconfiguring mgr.smithi115.tcyodz (monmap changed)... 2021-11-03T17:08:44.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:44 smithi115 conmon[34670]: audit 2021-11-03T17:08:43.879593+0000 mon.smithi027 (mon.0) 216 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi115.tcyodz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2021-11-03T17:08:44.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:44 smithi115 conmon[34670]: audit 2021-11-03T17:08:43.880316+0000 mon.smithi027 (mon.0) 217 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mgr services"}]: dispatch 2021-11-03T17:08:44.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:44 smithi115 conmon[34670]: audit 2021-11-03T17:08:43.880941+0000 mon.smithi027 (mon.0) 218 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:08:44.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:44 smithi115 conmon[34670]: cephadm 2021-11-03T17:08:43.881456+0000 mgr.smithi027.ufhhvb (mgr.14162) 84 : cephadm [INF] Reconfiguring daemon mgr.smithi115.tcyodz on smithi115 2021-11-03T17:08:44.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:44 smithi115 conmon[34670]: cluster 2021-11-03T17: 2021-11-03T17:08:44.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:44 smithi115 conmon[34670]: 08:44.539313+0000 mgr.smithi027.ufhhvb (mgr.14162) 85 : cluster [DBG] pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:45.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:44 smithi027 conmon[32322]: audit 2021-11-03T17:08:43.878779+0000 mon.smithi027 2021-11-03T17:08:45.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:44 smithi027 conmon[32322]: (mon.0) 215 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:45.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:44 smithi027 conmon[32322]: cephadm 2021-11-03T17:08:43.879316+0000 mgr.smithi027.ufhhvb (mgr.14162) 83 : cephadm [INF] Reconfiguring mgr.smithi115.tcyodz (monmap changed)... 2021-11-03T17:08:45.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:44 smithi027 conmon[32322]: audit 2021-11-03T17:08:43.879593+0000 mon.smithi027 (mon.0) 216 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi115.tcyodz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2021-11-03T17:08:45.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:44 smithi027 conmon[32322]: audit 2021-11-03T17:08:43.880316+0000 mon.smithi027 (mon.0) 217 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mgr services"}]: dispatch 2021-11-03T17:08:45.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:44 smithi027 conmon[32322]: audit 2021-11-03T17:08:43.880941+0000 mon.smithi027 (mon.0) 218 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:08:45.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:44 smithi027 conmon[32322]: cephadm 2021-11-03T17:08:43.881456+0000 mgr.smithi027.ufhhvb (mgr.14162) 84 : cephadm [INF] Reconfiguring daemon mgr.smithi115.tcyodz on smithi115 2021-11-03T17:08:45.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:44 smithi027 conmon[32322]: cluster 2021-11-03T17:08:44.539313+0000 mgr.smithi027.ufhhvb (mgr.14162) 85 : cluster [DBG] pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:45.803 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 0 on host 'smithi027' 2021-11-03T17:08:45.804 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:45.806+0000 7fe7997fa700 1 -- 172.21.15.27:0/486989890 <== mgr.14162 v2:172.21.15.27:6800/3802716887 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe79c0faca0 con 0x7fe78405c2b0 2021-11-03T17:08:45.805 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:45.809+0000 7fe7a2e8c700 1 -- 172.21.15.27:0/486989890 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fe78405c2b0 msgr2=0x7fe78405e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:08:45.805 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:45.809+0000 7fe7a2e8c700 1 --2- 172.21.15.27:0/486989890 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fe78405c2b0 0x7fe78405e770 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fe790004e30 tx=0x7fe790009b30).stop 2021-11-03T17:08:45.805 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:45.809+0000 7fe7a2e8c700 1 -- 172.21.15.27:0/486989890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe79c0f55c0 msgr2=0x7fe79c106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:08:45.805 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:45.809+0000 7fe7a2e8c700 1 --2- 172.21.15.27:0/486989890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe79c0f55c0 0x7fe79c106f70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7fe79400d7f0 tx=0x7fe794004e80).stop 2021-11-03T17:08:45.806 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:45.809+0000 7fe7a2e8c700 1 -- 172.21.15.27:0/486989890 shutdown_connections 2021-11-03T17:08:45.806 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:45.809+0000 7fe7a2e8c700 1 --2- 172.21.15.27:0/486989890 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fe78405c2b0 0x7fe78405e770 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:08:45.806 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:45.809+0000 7fe7a2e8c700 1 --2- 172.21.15.27:0/486989890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe79c0f55c0 0x7fe79c106f70 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:08:45.806 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:45.809+0000 7fe7a2e8c700 1 --2- 172.21.15.27:0/486989890 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe79c0f6860 0x7fe79c107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:08:45.806 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:45.809+0000 7fe7a2e8c700 1 -- 172.21.15.27:0/486989890 >> 172.21.15.27:0/486989890 conn(0x7fe79c0f0d40 msgr2=0x7fe79c05fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:08:45.807 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:45.809+0000 7fe7a2e8c700 1 -- 172.21.15.27:0/486989890 shutdown_connections 2021-11-03T17:08:45.807 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:45.809+0000 7fe7a2e8c700 1 -- 172.21.15.27:0/486989890 wait complete. 2021-11-03T17:08:46.274 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:46 smithi115 conmon[34670]: audit 2021-11-03T17:08:45.277756+0000 mon.smithi027 (mon.0) 219 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:46.274 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:46 smithi115 conmon[34670]: cephadm 2021-11-03T17: 2021-11-03T17:08:46.352 DEBUG:teuthology.orchestra.run.smithi027:osd.0> sudo journalctl -f -n 0 -u ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c@osd.0.service 2021-11-03T17:08:46.355 INFO:tasks.cephadm:Deploying osd.1 on smithi027 with /dev/vg_nvme/lv_3... 2021-11-03T17:08:46.356 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2021-11-03T17:08:46.383 INFO:journalctl@ceph.osd.0.smithi027.stdout:-- Logs begin at Wed 2021-11-03 16:54:54 UTC. -- 2021-11-03T17:08:46.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:46 smithi027 conmon[32322]: audit 2021-11-03T17:08:45. 2021-11-03T17:08:46.530 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:46 smithi027 conmon[32322]: 277756+0000 mon.smithi027 (mon.0) 219 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:46.530 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:46 smithi027 conmon[32322]: cephadm 2021-11- 2021-11-03T17:08:46.531 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:46 smithi027 conmon[32322]: 03T17:08:45.278284+0000 mgr.smithi027.ufhhvb (mgr.14162) 86 : 2021-11-03T17:08:46.531 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:46 smithi027 conmon[32322]: cephadm [INF] Reconfiguring mon.smithi115 (monmap changed)... 2021-11-03T17:08:46.531 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:46 smithi027 conmon[32322]: audit 2021-11-03T17: 2021-11-03T17:08:46.531 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:46 smithi027 conmon[32322]: 08:45.278526+0000 mon.smithi027 (mon.0 2021-11-03T17:08:46.532 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:46 smithi027 conmon[32322]: ) 220 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-11-03T17:08:46.532 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:46 smithi027 conmon[32322]: audit 2021-11-03T17 2021-11-03T17:08:46.532 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:46 smithi027 conmon[32322]: :08:45.279225+0000 mon.smithi027 (mon. 2021-11-03T17:08:46.532 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:46 smithi027 conmon[32322]: 0) 221 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-11-03T17:08:46.532 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:46 smithi027 conmon[32322]: audit 2021-11-03T 2021-11-03T17:08:46.533 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:46 smithi027 conmon[32322]: 17:08:45.279867+0000 mon.smithi027 2021-11-03T17:08:46.533 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:46 smithi027 conmon[32322]: (mon.0) 222 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:08:46.533 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:46 smithi027 conmon[32322]: cephadm 2021-11 2021-11-03T17:08:46.533 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:46 smithi027 conmon[32322]: -03T17:08:45.280423+0000 2021-11-03T17:08:46.534 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:46 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14162) 87 : cephadm [INF] 2021-11-03T17:08:46.534 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:46 smithi027 conmon[32322]: Reconfiguring daemon mon.smithi115 on smithi115 2021-11-03T17:08:46.534 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:46 smithi027 conmon[32322]: audit 2021 2021-11-03T17:08:46.534 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:46 smithi027 conmon[32322]: -11-03T17:08:45.802827+0000 mon.smithi027 (mon 2021-11-03T17:08:46.535 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:46 smithi027 conmon[32322]: .0) 223 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:46.535 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:46 smithi027 conmon[32322]: audit 2021-11-03T 2021-11-03T17:08:46.535 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:46 smithi027 conmon[32322]: 17:08:45.806972+0000 mon.smithi027 (mon 2021-11-03T17:08:46.535 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:46 smithi027 conmon[32322]: .0) 224 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:46.601 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:46 smithi115 conmon[34670]: 08:45.278284+0000 mgr.smithi027.ufhhvb (mgr.14162) 86 : cephadm [INF] Reconfiguring mon.smithi115 (monmap changed)... 2021-11-03T17:08:46.602 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:46 smithi115 conmon[34670]: audit 2021-11-03T17:08:45.278526+0000 mon.smithi027 (mon.0) 220 : audit 2021-11-03T17:08:46.602 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:46 smithi115 conmon[34670]: [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-11-03T17:08:46.602 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:46 smithi115 conmon[34670]: audit 2021 2021-11-03T17:08:46.602 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:46 smithi115 conmon[34670]: -11-03T17:08:45.279225+0000 2021-11-03T17:08:46.603 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:46 smithi115 conmon[34670]: mon.smithi027 (mon.0) 221 : audit [DBG] 2021-11-03T17:08:46.603 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:46 smithi115 conmon[34670]: from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-11-03T17:08:46.603 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:46 smithi115 conmon[34670]: audit 2021-11-03T 2021-11-03T17:08:46.603 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:46 smithi115 conmon[34670]: 17:08:45.279867+0000 mon.smithi027 ( 2021-11-03T17:08:46.603 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:46 smithi115 conmon[34670]: mon.0) 222 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:08:46.604 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:46 smithi115 conmon[34670]: cephadm 2021 2021-11-03T17:08:46.604 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:46 smithi115 conmon[34670]: -11-03T17:08: 2021-11-03T17:08:46.604 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:46 smithi115 conmon[34670]: 45.280423+0000 mgr.smithi027.ufhhvb (mgr.14162) 87 2021-11-03T17:08:46.604 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:46 smithi115 conmon[34670]: : cephadm [INF] Reconfiguring daemon mon.smithi115 on smithi115 2021-11-03T17:08:46.605 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:46 smithi115 conmon[34670]: audit 2021-11 2021-11-03T17:08:46.605 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:46 smithi115 conmon[34670]: -03T17:08:45.802827 2021-11-03T17:08:46.605 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:46 smithi115 conmon[34670]: +0000 mon.smithi027 (mon.0) 223 : audit 2021-11-03T17:08:46.605 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:46 smithi115 conmon[34670]: [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:46.605 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:46 smithi115 conmon[34670]: audit 2021-11- 2021-11-03T17:08:46.606 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:46 smithi115 conmon[34670]: 03T17:08:45.806972 2021-11-03T17:08:46.606 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:46 smithi115 conmon[34670]: +0000 mon.smithi027 (mon.0) 224 : audit 2021-11-03T17:08:46.606 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:46 smithi115 conmon[34670]: [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:47.880 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:47 smithi027 conmon[32322]: cluster 2021-11-03T17:08:46.539593+0000 mgr.smithi027.ufhhvb (mgr.14162) 88 : cluster [DBG] pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:47.880 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:47 smithi027 conmon[32322]: audit 2021-11-03T17:08:46.642616+0000 mon.smithi027 (mon.0) 225 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:47.881 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:47 smithi027 conmon[32322]: audit 2021-11-03T17:08:46.645550+0000 mon.smithi027 (mon.0) 226 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2021-11-03T17:08:47.881 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:47 smithi027 conmon[32322]: audit 2021-11-03T17:08:46.647265+0000 mon.smithi027 (mon.0) 227 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://172.21.15.27:9093"}]: dispatch 2021-11-03T17:08:47.881 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:47 smithi027 conmon[32322]: audit 2021-11-03T17:08:46.651364+0000 mon.smithi027 (mon.0) 228 : audit 2021-11-03T17:08:47.882 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:47 smithi027 conmon[32322]: [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:47.882 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:47 smithi027 conmon[32322]: audit 2021-11-03T17:08:46.655136+0000 mon.smithi027 (mon.0) 229 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2021-11-03T17:08:47.882 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:47 smithi027 conmon[32322]: audit 2021-11-03T17:08:46.662974+0000 mon.smithi027 (mon.0) 230 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://172.21.15.27:3000"}]: dispatch 2021-11-03T17:08:47.882 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:47 smithi027 conmon[32322]: audit 2021-11-03T17:08:46.667718+0000 mon.smithi027 (mon.0) 231 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:47.883 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:47 smithi027 conmon[32322]: audit 2021-11-03T17:08:46.671613+0000 mon.smithi027 (mon.0) 232 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2021-11-03T17:08:47.883 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:47 smithi027 conmon[32322]: audit 2021-11-03T17:08:46.677069+0000 mon.smithi027 (mon.0) 233 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://172.21.15.27:9095"}]: dispatch 2021-11-03T17:08:47.883 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:47 smithi027 conmon[32322]: audit 2021-11-03T17:08:46.681777+0000 mon.smithi027 (mon.0) 234 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:47.883 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:47 smithi027 conmon[32322]: audit 2021-11-03T17:08:46.684421+0000 mon.smithi027 (mon.0) 235 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:08:47.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:47 smithi115 conmon[34670]: cluster 2021-11-03T17:08:46.539593+0000 2021-11-03T17:08:47.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:47 smithi115 conmon[34670]: mgr.smithi027.ufhhvb (mgr.14162) 88 : cluster [DBG] pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:47.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:47 smithi115 conmon[34670]: audit 2021-11-03T17:08:46.642616+0000 mon.smithi027 (mon.0) 225 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:47.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:47 smithi115 conmon[34670]: audit 2021-11-03T17:08:46.645550+0000 mon.smithi027 (mon.0) 226 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2021-11-03T17:08:47.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:47 smithi115 conmon[34670]: audit 2021-11-03T17:08:46.647265+0000 mon.smithi027 (mon. 2021-11-03T17:08:47.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:47 smithi115 conmon[34670]: 0) 227 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://172.21.15.27:9093"}]: dispatch 2021-11-03T17:08:47.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:47 smithi115 conmon[34670]: audit 2021-11-03T17:08 2021-11-03T17:08:47.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:47 smithi115 conmon[34670]: :46.651364+0000 mon.smithi027 (mon.0) 228 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:47.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:47 smithi115 conmon[34670]: 2021-11-03T17:08:47.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:47 smithi115 conmon[34670]: audit 2021-11-03T17:08:47.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:47 smithi115 conmon[34670]: 2021-11-03T17:08:46 2021-11-03T17:08:47.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:47 smithi115 conmon[34670]: .655136+0000 mon.smithi027 (mon.0) 229 : 2021-11-03T17:08:47.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:47 smithi115 conmon[34670]: audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2021-11-03T17:08:47.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:47 smithi115 conmon[34670]: audit 2021-11- 2021-11-03T17:08:47.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:47 smithi115 conmon[34670]: 03T17:08:46.662974+0000 2021-11-03T17:08:47.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:47 smithi115 conmon[34670]: mon.smithi027 (mon.0) 230 : audit [INF] 2021-11-03T17:08:47.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:47 smithi115 conmon[34670]: from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://172.21.15.27:3000"}]: dispatch 2021-11-03T17:08:47.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:47 smithi115 conmon[34670]: audit 2021-11-03T 2021-11-03T17:08:47.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:47 smithi115 conmon[34670]: 17:08:46.667718+0000 2021-11-03T17:08:47.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:47 smithi115 conmon[34670]: mon.smithi027 (mon.0) 231 : audit 2021-11-03T17:08:47.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:47 smithi115 conmon[34670]: [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:47.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:47 smithi115 conmon[34670]: audit 2021-11 2021-11-03T17:08:47.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:47 smithi115 conmon[34670]: -03T17:08:46.671613 2021-11-03T17:08:47.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:47 smithi115 conmon[34670]: +0000 mon.smithi027 (mon.0) 232 : audit 2021-11-03T17:08:47.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:47 smithi115 conmon[34670]: [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2021-11-03T17:08:47.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:47 smithi115 conmon[34670]: audit 2021-11-03 2021-11-03T17:08:47.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:47 smithi115 conmon[34670]: T17:08:46.677069+0000 mon.smithi027 2021-11-03T17:08:47.991 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:47 smithi115 conmon[34670]: (mon.0) 233 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://172.21.15.27:9095"}]: dispatch 2021-11-03T17:08:47.991 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:47 smithi115 conmon[34670]: 2021-11-03T17:08:47.991 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:47 smithi115 conmon[34670]: audit 2021-11-03T 2021-11-03T17:08:47.991 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:47 smithi115 conmon[34670]: 17:08:46.681777+0000 mon.smithi027 2021-11-03T17:08:47.992 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:47 smithi115 conmon[34670]: (mon.0) 234 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:47.992 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:47 smithi115 conmon[34670]: 2021-11-03T17:08:47.992 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:47 smithi115 conmon[34670]: audit 2021-11-03T17:08:47.992 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:47 smithi115 conmon[34670]: 2021-11-03T17:08: 2021-11-03T17:08:47.992 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:47 smithi115 conmon[34670]: 46.684421+0000 mon.smithi027 (mon.0) 2021-11-03T17:08:47.993 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:47 smithi115 conmon[34670]: 235 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:08:48.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:48 smithi027 conmon[32322]: audit 2021-11-03T17:08:46.645906+0000 mgr.smithi027.ufhhvb (mgr.14162) 89 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2021-11-03T17:08:48.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:48 smithi027 conmon[32322]: audit 2021-11-03T17:08:46.647618+0000 mgr.smithi027.ufhhvb (mgr.14162) 90 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://172.21.15.27:9093"}]: dispatch 2021-11-03T17:08:48.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:48 smithi027 conmon[32322]: audit 2021-11-03T17:08:46.655615+0000 mgr.smithi027.ufhhvb (mgr.14162) 91 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2021-11-03T17:08:48.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:48 smithi027 conmon[32322]: audit 2021-11-03T17:08:46.663355+0000 mgr.smithi027.ufhhvb (mgr.14162) 92 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://172.21.15.27:3000"}]: dispatch 2021-11-03T17:08:48.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:48 smithi027 conmon[32322]: audit 2021-11-03T17:08:46.672023+0000 mgr.smithi027.ufhhvb (mgr.14162) 93 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2021-11-03T17:08:48.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:48 smithi027 conmon[32322]: audit 2021-11-03T17:08:46.677440+0000 mgr.smithi027.ufhhvb (mgr.14162) 94 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://172.21.15.27:9095"}]: dispatch 2021-11-03T17:08:48.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:48 smithi027 conmon[32322]: audit 2021-11-03T17:08:48.485851+0000 mon.smithi027 (mon.0) 236 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:48.880 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping: /dev/vg_nvme/lv_3 2021-11-03T17:08:48.880 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2021-11-03T17:08:48.880 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10+0 records in 2021-11-03T17:08:48.880 INFO:teuthology.orchestra.run.smithi027.stderr:10+0 records out 2021-11-03T17:08:48.881 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0117338 s, 894 MB/s 2021-11-03T17:08:48.881 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping successful for: 2021-11-03T17:08:48.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:48 smithi115 conmon[34670]: audit 2021-11-03T17:08:46.645906+0000 mgr.smithi027.ufhhvb (mgr.14162) 89 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2021-11-03T17:08:48.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:48 smithi115 conmon[34670]: audit 2021-11-03 2021-11-03T17:08:48.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:48 smithi115 conmon[34670]: T17:08:46.647618+0000 mgr.smithi027.ufhhvb (mgr.14162) 90 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://172.21.15.27:9093"}]: dispatch 2021-11-03T17:08:48.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:48 smithi115 conmon[34670]: audit 2021-11-03T17:08:46.655615+0000 mgr.smithi027.ufhhvb (mgr.14162) 91 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2021-11-03T17:08:48.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:48 smithi115 conmon[34670]: audit 2021-11-03T17:08:46.663355+0000 mgr.smithi027.ufhhvb (mgr.14162) 92 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://172.21.15.27:3000"}]: dispatch 2021-11-03T17:08:48.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:48 smithi115 conmon[34670]: audit 2021-11-03T17:08:46.672023+0000 mgr.smithi027.ufhhvb (mgr.14162) 93 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2021-11-03T17:08:48.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:48 smithi115 conmon[34670]: audit 2021-11-03T17:08:46.677440+0000 mgr.smithi027.ufhhvb (mgr.14162) 94 : audit [DBG] from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://172.21.15.27:9095"}]: dispatch 2021-11-03T17:08:48.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:48 smithi115 conmon[34670]: audit 2021-11-03T17:08:48.485851+0000 mon.smithi027 (mon.0) 236 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:49.681 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph orch daemon add osd smithi027:vg_nvme/lv_3 2021-11-03T17:08:49.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:49 smithi115 conmon[34670]: cluster 2021-11-03T17:08:48.539872+0000 mgr.smithi027.ufhhvb (mgr.14162) 95 : cluster [DBG] pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:50.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:49 smithi027 conmon[32322]: cluster 2021-11-03T17:08:48.539872+0000 mgr.smithi027.ufhhvb (mgr.14162) 95 : cluster [DBG] pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:50.019 INFO:journalctl@ceph.osd.0.smithi027.stdout:Nov 03 17:08:49 smithi027 conmon[50393]: debug 2021-11-03T17:08:49.793+0000 7faf66361080 -1 osd.0 0 log_to_monitors {default=true} 2021-11-03T17:08:50.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:50 smithi115 conmon[34670]: audit 2021-11-03T17:08:49.797670+0000 mon.smithi027 (mon.0) 2021-11-03T17:08:50.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:50 smithi115 conmon[34670]: 237 : audit [INF] from='osd.0 [v2:172.21.15.27:6802/3126282997,v1:172.21.15.27:6803/3126282997]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2021-11-03T17:08:51.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:50 smithi027 conmon[32322]: audit 2021-11-03T17:08:49.797670+0000 mon.smithi027 (mon.0) 237 : audit [INF] from='osd.0 [v2:172.21.15.27:6802/3126282997,v1:172.21.15.27:6803/3126282997]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2021-11-03T17:08:51.754 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:51.757+0000 7f03a3eca700 1 -- 172.21.15.27:0/435044575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f039c0f66d0 msgr2=0x7f039c0f6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:08:51.754 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:51.757+0000 7f03a3eca700 1 --2- 172.21.15.27:0/435044575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f039c0f66d0 0x7f039c0f6af0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f038c0099e0 tx=0x7f038c004d10).stop 2021-11-03T17:08:51.754 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:51.757+0000 7f03a3eca700 1 -- 172.21.15.27:0/435044575 shutdown_connections 2021-11-03T17:08:51.755 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:51.757+0000 7f03a3eca700 1 --2- 172.21.15.27:0/435044575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f039c0f7970 0x7f039c0f7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:08:51.755 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:51.757+0000 7f03a3eca700 1 --2- 172.21.15.27:0/435044575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f039c0f66d0 0x7f039c0f6af0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:08:51.755 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:51.757+0000 7f03a3eca700 1 -- 172.21.15.27:0/435044575 >> 172.21.15.27:0/435044575 conn(0x7f039c0f1e10 msgr2=0x7f039c0f4250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:08:51.756 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:51.757+0000 7f03a3eca700 1 -- 172.21.15.27:0/435044575 shutdown_connections 2021-11-03T17:08:51.756 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:51.757+0000 7f03a3eca700 1 -- 172.21.15.27:0/435044575 wait complete. 2021-11-03T17:08:51.756 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:51.758+0000 7f03a3eca700 1 Processor -- start 2021-11-03T17:08:51.756 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:51.758+0000 7f03a3eca700 1 -- start start 2021-11-03T17:08:51.756 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:51.758+0000 7f03a3eca700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f039c0f66d0 0x7f039c1070c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:08:51.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:51.758+0000 7f03a3eca700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f039c0f7970 0x7f039c1075b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:08:51.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:51.758+0000 7f03a3eca700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039c107b60 con 0x7f039c0f7970 2021-11-03T17:08:51.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:51.758+0000 7f03a3eca700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039c107ca0 con 0x7f039c0f66d0 2021-11-03T17:08:51.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:51.758+0000 7f03a1465700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f039c0f7970 0x7f039c1075b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:08:51.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:51.758+0000 7f03a1465700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f039c0f7970 0x7f039c1075b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54470/0 (socket says 172.21.15.27:54470) 2021-11-03T17:08:51.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:51.758+0000 7f03a1465700 1 -- 172.21.15.27:0/3663656666 learned_addr learned my addr 172.21.15.27:0/3663656666 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:08:51.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:51.758+0000 7f03a1c66700 1 --2- 172.21.15.27:0/3663656666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f039c0f66d0 0x7f039c1070c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:08:51.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:51.758+0000 7f03a1465700 1 -- 172.21.15.27:0/3663656666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f039c0f66d0 msgr2=0x7f039c1070c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:08:51.759 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:51.758+0000 7f03a1465700 1 --2- 172.21.15.27:0/3663656666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f039c0f66d0 0x7f039c1070c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:08:51.759 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:51.758+0000 7f03a1465700 1 -- 172.21.15.27:0/3663656666 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f038c009640 con 0x7f039c0f7970 2021-11-03T17:08:51.759 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:51.759+0000 7f03a1465700 1 --2- 172.21.15.27:0/3663656666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f039c0f7970 0x7f039c1075b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f03900026d0 tx=0x7f0390009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:08:51.759 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:51.759+0000 7f039a7fc700 1 -- 172.21.15.27:0/3663656666 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f03900079c0 con 0x7f039c0f7970 2021-11-03T17:08:51.759 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:51.759+0000 7f039a7fc700 1 -- 172.21.15.27:0/3663656666 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f039000f040 con 0x7f039c0f7970 2021-11-03T17:08:51.760 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:51.759+0000 7f039a7fc700 1 -- 172.21.15.27:0/3663656666 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0390014820 con 0x7f039c0f7970 2021-11-03T17:08:51.760 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:51.759+0000 7f03a3eca700 1 -- 172.21.15.27:0/3663656666 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f039c108640 con 0x7f039c0f7970 2021-11-03T17:08:51.760 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:51.759+0000 7f03a3eca700 1 -- 172.21.15.27:0/3663656666 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f039c108b80 con 0x7f039c0f7970 2021-11-03T17:08:51.761 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:51.760+0000 7f03a3eca700 1 -- 172.21.15.27:0/3663656666 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f039c045ad0 con 0x7f039c0f7970 2021-11-03T17:08:51.762 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:51.760+0000 7f039a7fc700 1 -- 172.21.15.27:0/3663656666 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f03900090d0 con 0x7f039c0f7970 2021-11-03T17:08:51.762 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:51.760+0000 7f039a7fc700 1 --2- 172.21.15.27:0/3663656666 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f038405c2b0 0x7f038405e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:08:51.762 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:51.761+0000 7f039a7fc700 1 -- 172.21.15.27:0/3663656666 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1569+0+0 (secure 0 0 0) 0x7f0390074ed0 con 0x7f039c0f7970 2021-11-03T17:08:51.762 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:51.761+0000 7f03a1c66700 1 --2- 172.21.15.27:0/3663656666 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f038405c2b0 0x7f038405e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:08:51.762 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:51.761+0000 7f03a1c66700 1 --2- 172.21.15.27:0/3663656666 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f038405c2b0 0x7f038405e770 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f038c003380 tx=0x7f038c004d10).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:08:51.763 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:51.764+0000 7f039a7fc700 1 -- 172.21.15.27:0/3663656666 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f0390048ba0 con 0x7f039c0f7970 2021-11-03T17:08:51.948 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:08:51.953+0000 7f03a3eca700 1 -- 172.21.15.27:0/3663656666 --> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f039c108e50 con 0x7f038405c2b0 2021-11-03T17:08:51.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:51 smithi115 conmon[34670]: cluster 2021-11-03T17:08:50.540234+0000 mgr.smithi027.ufhhvb (mgr.14162) 96 : cluster [DBG] pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:51.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:51 smithi115 conmon[34670]: audit 2021-11-03T17:08:51.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:51 smithi115 conmon[34670]: 2021-11-03T17:08:50.554499+0000 mon.smithi027 (mon.0) 238 : audit [INF] from='osd.0 [v2:172.21.15.27:6802/3126282997,v1:172.21.15.27:6803/3126282997]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2021-11-03T17:08:51.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:51 smithi115 conmon[34670]: cluster 2021-11-03T17:08:50 2021-11-03T17:08:51.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:51 smithi115 conmon[34670]: .554556+0000 mon.smithi027 (mon.0) 239 : cluster [DBG] osdmap e7: 1 total, 0 up, 1 in 2021-11-03T17:08:51.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:51 smithi115 conmon[34670]: audit 2021-11-03T17:08:50 2021-11-03T17:08:51.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:51 smithi115 conmon[34670]: .554651+0000 mon.smithi027 (mon.0) 240 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-11-03T17:08:51.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:51 smithi115 conmon[34670]: 2021-11-03T17:08:51.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:51 smithi115 conmon[34670]: audit 2021-11-03T17:08:51.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:51 smithi115 conmon[34670]: 2021-11-03T17:08:50 2021-11-03T17:08:51.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:51 smithi115 conmon[34670]: .554996+0000 mon.smithi027 (mon.0) 241 : audit 2021-11-03T17:08:51.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:51 smithi115 conmon[34670]: [INF] from='osd.0 [v2:172.21.15.27:6802/3126282997,v1:172.21.15.27:6803/3126282997]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2021-11-03T17:08:52.019 INFO:journalctl@ceph.osd.0.smithi027.stdout:Nov 03 17:08:51 smithi027 conmon[50393]: debug 2021-11-03T17:08:51.555+0000 7faf5c8d2700 -1 osd.0 0 waiting for initial osdmap 2021-11-03T17:08:52.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:51 smithi027 conmon[32322]: cluster 2021-11-03T17:08:50.540234+0000 mgr.smithi027.ufhhvb ( 2021-11-03T17:08:52.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:51 smithi027 conmon[32322]: mgr.14162) 96 : cluster [DBG] pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:52.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:51 smithi027 conmon[32322]: audit 2021-11-03T17:08:50.554499+0000 mon.smithi027 (mon.0) 238 : audit [INF] from='osd.0 [v2:172.21.15.27:6802/3126282997,v1:172.21.15.27:6803/3126282997]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2021-11-03T17:08:52.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:51 smithi027 conmon[32322]: cluster 2021-11-03T17:08:50.554556+0000 mon.smithi027 (mon.0) 239 : cluster [DBG] osdmap e7: 1 total, 0 up, 1 in 2021-11-03T17:08:52.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:51 smithi027 conmon[32322]: audit 2021-11-03T17:08:50.554651+0000 mon.smithi027 (mon.0) 240 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-11-03T17:08:52.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:51 smithi027 conmon[32322]: audit 2021-11-03T17:08:50.554996+0000 mon.smithi027 (mon.0) 241 : audit [INF] from='osd.0 [v2:172.21.15.27:6802/3126282997,v1:172.21.15.27:6803/3126282997]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2021-11-03T17:08:52.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:52 smithi027 conmon[32322]: audit 2021-11-03T17:08:51.552966+0000 mon.smithi027 (mon.0) 242 : audit [INF] from='osd.0 [v2:172.21.15.27:6802/3126282997,v1:172.21.15.27:6803/3126282997]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2021-11-03T17:08:52.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:52 smithi027 conmon[32322]: cluster 2021-11-03T17:08:51.553019+0000 mon.smithi027 (mon.0) 243 : cluster [DBG] osdmap e8: 1 total, 0 up, 1 in 2021-11-03T17:08:52.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:52 smithi027 conmon[32322]: audit 2021-11-03T17:08:51.553397+0000 mon.smithi027 (mon.0) 244 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-11-03T17:08:52.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:52 smithi027 conmon[32322]: audit 2021-11-03T17:08:51.557500+0000 mon.smithi027 (mon.0) 245 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-11-03T17:08:52.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:52 smithi027 conmon[32322]: audit 2021-11-03T17:08:51.955798+0000 mon.smithi027 (mon.0) 246 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2021-11-03T17:08:52.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:52 smithi027 conmon[32322]: audit 2021-11-03T17:08:51.957288+0000 mon.smithi027 (mon.0) 247 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2021-11-03T17:08:52.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:52 smithi027 conmon[32322]: audit 2021-11-03T17:08:51.957952+0000 mon.smithi027 (mon.0) 248 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:08:52.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:52 smithi027 conmon[32322]: audit 2021-11-03T17:08:52.300106+0000 mon.smithi027 (mon.0) 249 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:52.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:52 smithi115 conmon[34670]: audit 2021-11-03T17:08:51.552966+0000 mon.smithi027 (mon.0) 242 : audit [INF] from='osd.0 [v2:172.21.15.27:6802/3126282997,v1:172.21.15.27:6803/3126282997]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2021-11-03T17:08:52.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:52 smithi115 conmon[34670]: cluster 2021-11-03T17:08:51.553019+0000 mon.smithi027 (mon.0) 243 : cluster [DBG] osdmap e8: 1 total, 0 up, 1 in 2021-11-03T17:08:52.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:52 smithi115 conmon[34670]: audit 2021-11-03T17 2021-11-03T17:08:52.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:52 smithi115 conmon[34670]: :08:51.553397+0000 mon.smithi027 (mon 2021-11-03T17:08:52.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:52 smithi115 conmon[34670]: .0) 244 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-11-03T17:08:52.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:52 smithi115 conmon[34670]: audit 2021-11-03T17:08:52.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:52 smithi115 conmon[34670]: 2021-11-03T17:08: 2021-11-03T17:08:52.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:52 smithi115 conmon[34670]: 51.557500+0000 mon.smithi027 (mon.0) 245 2021-11-03T17:08:52.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:52 smithi115 conmon[34670]: : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-11-03T17:08:52.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:52 smithi115 conmon[34670]: audit 2021- 2021-11-03T17:08:52.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:52 smithi115 conmon[34670]: 11-03T17:08:51. 2021-11-03T17:08:52.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:52 smithi115 conmon[34670]: 955798+0000 mon.smithi027 (mon.0) 246 : audit 2021-11-03T17:08:52.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:52 smithi115 conmon[34670]: [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2021-11-03T17:08:52.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:52 smithi115 conmon[34670]: audit 2021-11 2021-11-03T17:08:52.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:52 smithi115 conmon[34670]: -03T17:08:51.957288 2021-11-03T17:08:52.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:52 smithi115 conmon[34670]: +0000 mon.smithi027 (mon.0) 247 : audit 2021-11-03T17:08:52.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:52 smithi115 conmon[34670]: [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2021-11-03T17:08:52.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:52 smithi115 conmon[34670]: audit 2021-11 2021-11-03T17:08:52.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:52 smithi115 conmon[34670]: -03T17:08:51.957952 2021-11-03T17:08:52.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:52 smithi115 conmon[34670]: +0000 mon.smithi027 (mon.0) 248 : audit [DBG] 2021-11-03T17:08:52.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:52 smithi115 conmon[34670]: from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:08:52.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:52 smithi115 conmon[34670]: audit 2021-11-03 2021-11-03T17:08:52.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:52 smithi115 conmon[34670]: T17:08:52.300106+0000 mon.smithi027 ( 2021-11-03T17:08:52.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:52 smithi115 conmon[34670]: mon.0) 249 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:08:53.637 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:53 smithi027 conmon[32322]: cluster 2021-11-03T17:08:50.763971+0000 osd.0 2021-11-03T17:08:53.638 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:53 smithi027 conmon[32322]: (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2021-11-03T17:08:53.638 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:53 smithi027 conmon[32322]: cluster 2021-11-03T17:08:50.764080+0000 osd.0 (osd.0) 2 : 2021-11-03T17:08:53.638 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:53 smithi027 conmon[32322]: cluster [DBG] purged_snaps scrub ok 2021-11-03T17:08:53.638 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:53 smithi027 conmon[32322]: audit 2021-11-03T17:08:53.639 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:53 smithi027 conmon[32322]: 2021-11-03T17:08:51.954958+0000 mgr.smithi027.ufhhvb ( 2021-11-03T17:08:53.639 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:53 smithi027 conmon[32322]: mgr.14162) 97 : audit [DBG] from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:08:53.639 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:53 smithi027 conmon[32322]: cephadm 2021-11-03 2021-11-03T17:08:53.639 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:53 smithi027 conmon[32322]: T17:08:51.957044+0000 mgr.smithi027.ufhhvb (mgr.14162) 98 : cephadm [INF] Applying service osd.None on host smithi027... 2021-11-03T17:08:53.640 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:53 smithi027 conmon[32322]: cluster 2021-11 2021-11-03T17:08:53.640 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:53 smithi027 conmon[32322]: -03T17:08:52.540522+0000 2021-11-03T17:08:53.640 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:53 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14162) 99 : cluster [DBG] 2021-11-03T17:08:53.640 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:53 smithi027 conmon[32322]: pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:53.640 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:53 smithi027 conmon[32322]: audit 2021-11-03T17:08:52.556323+0000 mon.smithi027 ( 2021-11-03T17:08:53.641 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:53 smithi027 conmon[32322]: mon.0) 250 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-11-03T17:08:53.641 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:53 smithi027 conmon[32322]: cluster 2021-11 2021-11-03T17:08:53.641 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:53 smithi027 conmon[32322]: -03T17:08:52.561012+0000 mon.smithi027 2021-11-03T17:08:53.641 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:53 smithi027 conmon[32322]: (mon.0) 251 : cluster [INF] osd.0 [v2:172.21.15.27:6802/3126282997,v1:172.21.15.27:6803/3126282997] boot 2021-11-03T17:08:53.642 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:53 smithi027 conmon[32322]: 2021-11-03T17:08:53.642 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:53 smithi027 conmon[32322]: cluster 2021 2021-11-03T17:08:53.642 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:53 smithi027 conmon[32322]: -11-03T17:08:52.561055+0000 mon.smithi027 2021-11-03T17:08:53.642 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:53 smithi027 conmon[32322]: (mon.0) 252 : cluster [DBG] 2021-11-03T17:08:53.642 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:53 smithi027 conmon[32322]: osdmap e9: 1 total, 1 up, 1 in 2021-11-03T17:08:53.643 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:53 smithi027 conmon[32322]: audit 2021-11-03T17:08:53.643 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:53 smithi027 conmon[32322]: 2021-11-03T17:08:52.561587+0000 mon.smithi027 2021-11-03T17:08:53.643 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:53 smithi027 conmon[32322]: (mon.0) 253 : audit [DBG] 2021-11-03T17:08:53.643 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:53 smithi027 conmon[32322]: from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-11-03T17:08:53.643 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:53 smithi027 conmon[32322]: audit 2021-11-03T17:08:53.644 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:53 smithi027 conmon[32322]: 2021-11-03T17:08:52.713250 2021-11-03T17:08:53.644 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:53 smithi027 conmon[32322]: +0000 mon.smithi027 (mon.0) 254 : audit 2021-11-03T17:08:53.644 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:53 smithi027 conmon[32322]: [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:08:53.644 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:53 smithi027 conmon[32322]: audit 2021 2021-11-03T17:08:53.644 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:53 smithi027 conmon[32322]: -11-03T17:08:52.716708+0000 mon.smithi027 2021-11-03T17:08:53.645 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:53 smithi027 conmon[32322]: (mon.0) 255 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:08:53.645 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:53 smithi027 conmon[32322]: 2021-11-03T17:08:53.957 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:53 smithi115 conmon[34670]: cluster 2021-11-03T17:08:50.763971+0000 osd.0 (osd.0) 1 : cluster [DBG] purged_snaps scrub starts 2021-11-03T17:08:53.958 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:53 smithi115 conmon[34670]: cluster 2021-11-03T17:08:50.764080+0000 osd.0 (osd.0) 2 : cluster [DBG] purged_snaps scrub ok 2021-11-03T17:08:53.958 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:53 smithi115 conmon[34670]: audit 2021-11-03T17:08:53.958 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:53 smithi115 conmon[34670]: 2021-11-03T17:08:51 2021-11-03T17:08:53.958 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:53 smithi115 conmon[34670]: .954958+0000 mgr.smithi027.ufhhvb (mgr.14162) 97 : 2021-11-03T17:08:53.959 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:53 smithi115 conmon[34670]: audit [DBG] from='client.14226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:08:53.959 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:53 smithi115 conmon[34670]: cephadm 2021-11- 2021-11-03T17:08:53.959 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:53 smithi115 conmon[34670]: 03T17:08:51.957044+0000 mgr.smithi027.ufhhvb 2021-11-03T17:08:53.959 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:53 smithi115 conmon[34670]: (mgr.14162) 98 : cephadm [INF] Applying service osd.None on host smithi027... 2021-11-03T17:08:53.960 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:53 smithi115 conmon[34670]: cluster 2021-11-03T 2021-11-03T17:08:53.960 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:53 smithi115 conmon[34670]: 17:08:52.540522+0000 mgr.smithi027.ufhhvb (mgr 2021-11-03T17:08:53.960 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:53 smithi115 conmon[34670]: .14162) 99 : cluster [DBG] pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2021-11-03T17:08:53.960 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:53 smithi115 conmon[34670]: audit 2021-11-03T17:08:53.960 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:53 smithi115 conmon[34670]: 2021-11-03T17:08: 2021-11-03T17:08:53.961 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:53 smithi115 conmon[34670]: 52.556323+0000 mon.smithi027 (mon.0) 250 : 2021-11-03T17:08:53.961 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:53 smithi115 conmon[34670]: audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-11-03T17:08:53.961 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:53 smithi115 conmon[34670]: cluster 2021-11- 2021-11-03T17:08:53.961 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:53 smithi115 conmon[34670]: 03T17:08:52.561012+0000 2021-11-03T17:08:53.961 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:53 smithi115 conmon[34670]: mon.smithi027 (mon.0) 251 : cluster [INF] osd.0 [v2:172.21.15.27:6802/3126282997,v1:172.21.15.27:6803/3126282997] boot 2021-11-03T17:08:53.962 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:53 smithi115 conmon[34670]: 2021-11-03T17:08:53.962 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:53 smithi115 conmon[34670]: cluster 2021-11-03T17 2021-11-03T17:08:53.962 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:53 smithi115 conmon[34670]: :08:52.561055+0000 mon.smithi027 ( 2021-11-03T17:08:53.962 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:53 smithi115 conmon[34670]: mon.0) 252 : cluster [DBG] osdmap e9: 1 total, 1 up, 1 in 2021-11-03T17:08:53.963 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:53 smithi115 conmon[34670]: audit 2021-11-03 2021-11-03T17:08:53.963 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:53 smithi115 conmon[34670]: T17:08:52.561587+0000 2021-11-03T17:08:53.963 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:53 smithi115 conmon[34670]: mon.smithi027 (mon.0) 253 : audit 2021-11-03T17:08:53.963 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:53 smithi115 conmon[34670]: [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-11-03T17:08:53.963 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:53 smithi115 conmon[34670]: audit 2021 2021-11-03T17:08:53.964 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:53 smithi115 conmon[34670]: -11-03T17:08:52 2021-11-03T17:08:53.964 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:53 smithi115 conmon[34670]: .713250+0000 mon.smithi027 (mon.0) 254 : 2021-11-03T17:08:53.964 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:53 smithi115 conmon[34670]: audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:08:53.964 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:53 smithi115 conmon[34670]: audit 2021- 2021-11-03T17:08:53.964 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:53 smithi115 conmon[34670]: 11-03T17:08:52. 2021-11-03T17:08:53.965 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:53 smithi115 conmon[34670]: 716708+0000 mon.smithi027 (mon.0) 255 : audit 2021-11-03T17:08:53.965 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:53 smithi115 conmon[34670]: [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:08:54.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:54 smithi115 conmon[34670]: cluster 2021-11-03T17:08:53.572566+0000 mon.smithi027 (mon.0) 2021-11-03T17:08:54.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:54 smithi115 conmon[34670]: 256 : cluster [DBG] osdmap e10: 1 total, 1 up, 1 in 2021-11-03T17:08:55.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:54 smithi027 conmon[32322]: cluster 2021-11-03T17:08:53.572566+0000 mon.smithi027 2021-11-03T17:08:55.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:54 smithi027 conmon[32322]: (mon.0) 256 : cluster [DBG] osdmap e10: 1 total, 1 up, 1 in 2021-11-03T17:08:55.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:55 smithi115 conmon[34670]: cluster 2021-11-03T17:08:54.540828+0000 mgr.smithi027.ufhhvb (mgr.14162) 100 : cluster [DBG] pgmap v60: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2021-11-03T17:08:55.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:55 smithi115 conmon[34670]: audit 2021-11-03T17:08:54.912473+0000 mon.smithi027 (mon.0) 257 : audit [INF] from='client.? 172.21.15.27:0/3903525267' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0942b897-5aa3-461a-a2d6-943d5739d963"}]: dispatch 2021-11-03T17:08:55.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:55 smithi115 conmon[34670]: audit 2021-11-03T17: 2021-11-03T17:08:55.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:55 smithi115 conmon[34670]: 08:54.916159+0000 mon.smithi027 (mon.0) 258 : audit 2021-11-03T17:08:55.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:55 smithi115 conmon[34670]: [INF] from='client.? 172.21.15.27:0/3903525267' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0942b897-5aa3-461a-a2d6-943d5739d963"}]': finished 2021-11-03T17:08:55.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:55 smithi115 conmon[34670]: cluster 2021-11 2021-11-03T17:08:55.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:55 smithi115 conmon[34670]: -03T17:08:54.916224 2021-11-03T17:08:55.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:55 smithi115 conmon[34670]: +0000 mon.smithi027 (mon.0) 259 : 2021-11-03T17:08:55.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:55 smithi115 conmon[34670]: cluster [DBG] osdmap e11: 2 total, 1 up, 2 in 2021-11-03T17:08:55.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:55 smithi115 conmon[34670]: audit 2021-11-03T17 2021-11-03T17:08:55.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:55 smithi115 conmon[34670]: :08:54.916335+0000 mon.smithi027 (mon.0) 2021-11-03T17:08:55.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:55 smithi115 conmon[34670]: 260 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-11-03T17:08:55.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:55 smithi115 conmon[34670]: audit 2021 2021-11-03T17:08:55.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:55 smithi115 conmon[34670]: -11-03T17:08:55 2021-11-03T17:08:55.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:55 smithi115 conmon[34670]: .443597+0000 mon.smithi027 (mon.0) 261 : audit 2021-11-03T17:08:55.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:55 smithi115 conmon[34670]: [DBG] from='client.? 172.21.15.27:0/222638394' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2021-11-03T17:08:56.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:55 smithi027 conmon[32322]: cluster 2021-11-03T17:08:54.540828+0000 mgr.smithi027.ufhhvb (mgr.14162) 100 : cluster [DBG] pgmap v60: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2021-11-03T17:08:56.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:55 smithi027 conmon[32322]: audit 2021-11-03T17:08:54.912473+0000 mon.smithi027 (mon.0) 257 : audit [INF] from='client.? 172.21.15.27:0/3903525267' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0942b897-5aa3-461a-a2d6-943d5739d963"}]: dispatch 2021-11-03T17:08:56.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:55 smithi027 conmon[32322]: audit 2021-11-03T17:08:54.916159+0000 mon.smithi027 (mon.0) 258 : audit [INF] from='client.? 172.21.15.27:0/3903525267' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0942b897-5aa3-461a-a2d6-943d5739d963"}]': finished 2021-11-03T17:08:56.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:55 smithi027 conmon[32322]: cluster 2021-11-03T17:08:54.916224+0000 mon.smithi027 (mon.0) 259 : cluster [DBG] osdmap e11: 2 total, 1 up, 2 in 2021-11-03T17:08:56.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:55 smithi027 conmon[32322]: audit 2021-11-03T17:08:54.916335+0000 mon.smithi027 (mon 2021-11-03T17:08:56.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:55 smithi027 conmon[32322]: .0) 260 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-11-03T17:08:56.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:55 smithi027 conmon[32322]: audit 2021-11-03T17:08:55.443597+0000 mon.smithi027 (mon.0) 261 : audit [DBG] from='client.? 172.21.15.27:0/222638394' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2021-11-03T17:08:57.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:57 smithi115 conmon[34670]: cluster 2021-11-03T17:08:56.541106+0000 2021-11-03T17:08:57.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:57 smithi115 conmon[34670]: mgr.smithi027.ufhhvb (mgr.14162) 101 : cluster [DBG] pgmap v62: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2021-11-03T17:08:58.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:57 smithi027 conmon[32322]: cluster 2021-11-03T17:08:56.541106+0000 mgr.smithi027.ufhhvb (mgr.14162) 101 : cluster [DBG] pgmap v62: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2021-11-03T17:08:59.733 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:59 smithi027 conmon[32322]: cluster 2021-11-03T17:08:58.541494+0000 2021-11-03T17:08:59.734 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:08:59 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14162) 102 : cluster [DBG] pgmap v63: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2021-11-03T17:08:59.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:08:59 smithi115 conmon[34670]: cluster 2021-11-03T17:08:58.541494+0000 mgr.smithi027.ufhhvb (mgr.14162) 102 : cluster [DBG] pgmap v63: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2021-11-03T17:09:01.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:01 smithi027 conmon[32322]: cluster 2021-11-03T17:09:00.541754+0000 mgr.smithi027.ufhhvb (mgr.14162) 103 : cluster [DBG] pgmap v64: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2021-11-03T17:09:01.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:01 smithi115 conmon[34670]: cluster 2021-11-03T17:09:00.541754+0000 mgr.smithi027.ufhhvb (mgr.14162) 103 : cluster 2021-11-03T17:09:01.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:01 smithi115 conmon[34670]: [DBG] pgmap v64: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2021-11-03T17:09:03.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:02 smithi115 conmon[34670]: cephadm 2021-11-03T17:09:01.854924+0000 mgr.smithi027.ufhhvb (mgr.14162) 104 : cephadm 2021-11-03T17:09:03.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:02 smithi115 conmon[34670]: [INF] Detected new or changed devices on smithi027 2021-11-03T17:09:03.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:02 smithi115 conmon[34670]: audit 2021-11-03T17:09:01.859689+0000 mon.smithi027 (mon.0) 262 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:03.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:02 smithi115 conmon[34670]: audit 2021-11-03T17:09:01.862591 2021-11-03T17:09:03.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:02 smithi115 conmon[34670]: +0000 mon.smithi027 (mon.0) 263 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:03.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:02 smithi115 conmon[34670]: audit 2021-11-03T 2021-11-03T17:09:03.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:02 smithi115 conmon[34670]: 17:09:01.865276+0000 mon.smithi027 (mon.0) 264 : audit 2021-11-03T17:09:03.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:02 smithi115 conmon[34670]: [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:09:03.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:02 smithi115 conmon[34670]: cluster 2021-11-03T17:09:03.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:02 smithi115 conmon[34670]: 2021-11-03T17:09:02.542178+0000 mgr.smithi027.ufhhvb 2021-11-03T17:09:03.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:02 smithi115 conmon[34670]: (mgr.14162) 105 : cluster [DBG] pgmap v65: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2021-11-03T17:09:03.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:02 smithi115 conmon[34670]: 2021-11-03T17:09:03.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:02 smithi027 conmon[32322]: cephadm 2021-11-03T17:09:01.854924+0000 mgr.smithi027.ufhhvb ( 2021-11-03T17:09:03.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:02 smithi027 conmon[32322]: mgr.14162) 104 : cephadm [INF] Detected new or changed devices on smithi027 2021-11-03T17:09:03.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:02 smithi027 conmon[32322]: audit 2021-11-03T17:09:01.859689+0000 mon.smithi027 (mon.0) 262 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:03.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:02 smithi027 conmon[32322]: audit 2021-11-03T17:09:01.862591+0000 mon.smithi027 (mon.0) 263 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:03.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:02 smithi027 conmon[32322]: audit 2021-11-03T17:09:01.865276+0000 mon.smithi027 (mon.0) 264 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:09:03.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:02 smithi027 conmon[32322]: cluster 2021-11-03T17:09:02.542178+0000 mgr.smithi027.ufhhvb (mgr.14162) 105 : cluster [DBG] pgmap v65: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2021-11-03T17:09:04.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:03 smithi027 conmon[32322]: audit 2021-11-03T17:09:03.038821+0000 mon.smithi027 (mon.0) 265 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2021-11-03T17:09:04.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:03 smithi027 conmon[32322]: audit 2021-11-03T17:09:03.039509+0000 mon.smithi027 (mon.0) 266 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:09:04.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:03 smithi027 conmon[32322]: cephadm 2021-11-03T17:09:03.040055+0000 mgr.smithi027.ufhhvb (mgr.14162) 106 : cephadm [INF] Deploying daemon osd.1 on smithi027 2021-11-03T17:09:04.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:03 smithi027 conmon[32322]: audit 2021-11-03T17:09:03.567537+0000 mon.smithi027 (mon.0) 267 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:04.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:03 smithi115 conmon[34670]: audit 2021-11-03T17:09:03.038821+0000 mon.smithi027 (mon.0) 265 : audit [INF] 2021-11-03T17:09:04.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:03 smithi115 conmon[34670]: from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2021-11-03T17:09:04.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:03 smithi115 conmon[34670]: audit 2021-11-03T17:09:03.039509+0000 mon.smithi027 (mon.0) 266 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:09:04.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:03 smithi115 conmon[34670]: cephadm 2021-11-03T17:09:03.040055+0000 mgr.smithi027.ufhhvb (mgr.14162) 106 : cephadm [INF] Deploying daemon osd.1 on smithi027 2021-11-03T17:09:04.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:03 smithi115 conmon[34670]: audit 2021-11-03T17:09:03.567537+0000 mon.smithi027 (mon.0) 267 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:05.199 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:04 smithi027 conmon[32322]: cluster 2021-11-03T17:09:04.542462+0000 mgr.smithi027.ufhhvb ( 2021-11-03T17:09:05.200 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:04 smithi027 conmon[32322]: mgr.14162) 107 : cluster [DBG] pgmap v66: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2021-11-03T17:09:05.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:04 smithi115 conmon[34670]: cluster 2021-11-03T17:09:04.542462+0000 mgr.smithi027.ufhhvb (mgr.14162) 107 : cluster [DBG] pgmap v66: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2021-11-03T17:09:07.685 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:07.688+0000 7f039a7fc700 1 -- 172.21.15.27:0/3663656666 <== mgr.14162 v2:172.21.15.27:6800/3802716887 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f039c108e50 con 0x7f038405c2b0 2021-11-03T17:09:07.686 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 1 on host 'smithi027' 2021-11-03T17:09:07.687 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:07.691+0000 7f03a3eca700 1 -- 172.21.15.27:0/3663656666 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f038405c2b0 msgr2=0x7f038405e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:09:07.687 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:07.691+0000 7f03a3eca700 1 --2- 172.21.15.27:0/3663656666 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f038405c2b0 0x7f038405e770 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f038c003380 tx=0x7f038c004d10).stop 2021-11-03T17:09:07.687 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:07.691+0000 7f03a3eca700 1 -- 172.21.15.27:0/3663656666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f039c0f7970 msgr2=0x7f039c1075b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:09:07.688 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:07.691+0000 7f03a3eca700 1 --2- 172.21.15.27:0/3663656666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f039c0f7970 0x7f039c1075b0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f03900026d0 tx=0x7f0390009de0).stop 2021-11-03T17:09:07.688 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:07.691+0000 7f03a3eca700 1 -- 172.21.15.27:0/3663656666 shutdown_connections 2021-11-03T17:09:07.688 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:07.691+0000 7f03a3eca700 1 --2- 172.21.15.27:0/3663656666 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f038405c2b0 0x7f038405e770 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:09:07.688 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:07.691+0000 7f03a3eca700 1 --2- 172.21.15.27:0/3663656666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f039c0f66d0 0x7f039c1070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:09:07.688 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:07.691+0000 7f03a3eca700 1 --2- 172.21.15.27:0/3663656666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f039c0f7970 0x7f039c1075b0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:09:07.689 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:07.691+0000 7f03a3eca700 1 -- 172.21.15.27:0/3663656666 >> 172.21.15.27:0/3663656666 conn(0x7f039c0f1e10 msgr2=0x7f039c0fb3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:09:07.689 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:07.691+0000 7f03a3eca700 1 -- 172.21.15.27:0/3663656666 shutdown_connections 2021-11-03T17:09:07.689 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:07.691+0000 7f03a3eca700 1 -- 172.21.15.27:0/3663656666 wait complete. 2021-11-03T17:09:07.869 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:07 smithi027 conmon[32322]: cluster 2021-11-03T17:09:06.542751+0000 mgr.smithi027.ufhhvb (mgr.14162 2021-11-03T17:09:07.869 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:07 smithi027 conmon[32322]: ) 108 : cluster [DBG] pgmap v67: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2021-11-03T17:09:07.870 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:07 smithi027 conmon[32322]: audit 2021-11-03T17:09:07.234764+0000 mon.smithi027 (mon.0) 268 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:07.870 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:07 smithi027 conmon[32322]: audit 2021-11-03T17:09:07.238456+0000 mon.smithi027 (mon.0) 269 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:07.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:07 smithi115 conmon[34670]: cluster 2021-11-03T17:09:06.542751+0000 mgr.smithi027.ufhhvb (mgr.14162) 108 : cluster [DBG] pgmap v67: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2021-11-03T17:09:07.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:07 smithi115 conmon[34670]: audit 2021-11-03T17:09:07.234764+0000 mon.smithi027 (mon.0) 268 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:07.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:07 smithi115 conmon[34670]: audit 2021-11-03T17:09:07 2021-11-03T17:09:07.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:07 smithi115 conmon[34670]: .238456+0000 mon.smithi027 (mon.0) 269 : audit [INF] 2021-11-03T17:09:07.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:07 smithi115 conmon[34670]: from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:08.384 DEBUG:teuthology.orchestra.run.smithi027:osd.1> sudo journalctl -f -n 0 -u ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c@osd.1.service 2021-11-03T17:09:08.387 INFO:tasks.cephadm:Deploying osd.2 on smithi027 with /dev/vg_nvme/lv_2... 2021-11-03T17:09:08.387 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2021-11-03T17:09:08.429 INFO:journalctl@ceph.osd.1.smithi027.stdout:-- Logs begin at Wed 2021-11-03 16:54:54 UTC. -- 2021-11-03T17:09:08.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:08 smithi027 conmon[32322]: audit 2021-11-03T17:09:07.679879+0000 mon.smithi027 (mon.0) 270 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:09:08.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:08 smithi027 conmon[32322]: audit 2021-11-03T17:09:07.683851+0000 mon.smithi027 (mon.0) 271 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:08.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:08 smithi027 conmon[32322]: audit 2021-11-03T17:09:07.688266+0000 mon.smithi027 (mon.0) 272 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:08.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:08 smithi115 conmon[34670]: audit 2021-11-03T17:09:07.679879+0000 mon.smithi027 (mon.0) 270 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:09:08.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:08 smithi115 conmon[34670]: audit 2021-11-03T17:09:08.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:08 smithi115 conmon[34670]: 2021-11-03T17:09:07.683851+0000 mon.smithi027 (mon.0) 271 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:08.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:08 smithi115 conmon[34670]: audit 2021-11-03T17:09:07.688266+0000 mon.smithi027 (mon. 2021-11-03T17:09:08.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:08 smithi115 conmon[34670]: 0) 272 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:09.701 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:09 smithi027 conmon[32322]: cluster 2021-11-03T17:09 2021-11-03T17:09:09.701 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:09 smithi027 conmon[32322]: :08.543068+0000 mgr.smithi027.ufhhvb (mgr.14162) 109 : cluster [DBG] pgmap v68: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2021-11-03T17:09:09.702 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:09 smithi027 conmon[32322]: audit 2021-11-03T17:09:09.393524+0000 mon.smithi027 (mon.0) 273 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:09.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:09 smithi115 conmon[34670]: cluster 2021-11-03T17:09:08.543068+0000 mgr.smithi027.ufhhvb (mgr.14162) 109 : cluster [DBG] pgmap v68: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2021-11-03T17:09:09.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:09 smithi115 conmon[34670]: audit 2021-11-03T17:09:09.393524+0000 mon.smithi027 (mon.0) 273 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:11.055 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping: /dev/vg_nvme/lv_2 2021-11-03T17:09:11.056 INFO:teuthology.orchestra.run.smithi027.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2021-11-03T17:09:11.056 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10+0 records in 2021-11-03T17:09:11.056 INFO:teuthology.orchestra.run.smithi027.stderr:10+0 records out 2021-11-03T17:09:11.056 INFO:teuthology.orchestra.run.smithi027.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0119307 s, 879 MB/s 2021-11-03T17:09:11.057 INFO:teuthology.orchestra.run.smithi027.stderr:--> Zapping successful for: 2021-11-03T17:09:11.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:11 smithi027 conmon[32322]: cluster 2021-11-03T17:09:10.543425+0000 mgr.smithi027.ufhhvb (mgr.14162) 110 : cluster [DBG] pgmap v69: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2021-11-03T17:09:11.769 INFO:journalctl@ceph.osd.1.smithi027.stdout:Nov 03 17:09:11 smithi027 conmon[56703]: debug 2021-11-03T17:09:11.670+0000 7fb3ffd23080 -1 osd.1 0 log_to_monitors {default=true} 2021-11-03T17:09:11.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:11 smithi115 conmon[34670]: cluster 2021-11-03T17:09:10.543425+0000 mgr.smithi027.ufhhvb (mgr.14162) 110 : cluster [DBG] pgmap v69: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2021-11-03T17:09:11.993 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph orch daemon add osd smithi027:vg_nvme/lv_2 2021-11-03T17:09:12.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:12 smithi027 conmon[32322]: audit 2021-11-03T17:09:11.674930 2021-11-03T17:09:12.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:12 smithi027 conmon[32322]: +0000 mon.smithi027 (mon.0) 274 : audit [INF] from='osd.1 [v2:172.21.15.27:6810/2723402359,v1:172.21.15.27:6811/2723402359]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2021-11-03T17:09:12.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:12 smithi115 conmon[34670]: audit 2021-11-03T17:09:11.674930+0000 mon.smithi027 (mon.0) 274 : audit [INF] from='osd.1 [v2:172.21.15.27:6810/2723402359,v1:172.21.15.27:6811/2723402359]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2021-11-03T17:09:13.894 INFO:journalctl@ceph.osd.1.smithi027.stdout:Nov 03 17:09:13 smithi027 conmon[56703]: debug 2021-11-03T17:09:13.564+0000 7fb3f6294700 -1 osd.1 0 waiting for initial osdmap 2021-11-03T17:09:13.894 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:13 smithi027 conmon[32322]: cluster 2021-11-03T17:09:12.543676+0000 mgr.smithi027.ufhhvb (mgr 2021-11-03T17:09:13.894 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:13 smithi027 conmon[32322]: .14162) 111 : cluster [DBG] pgmap v70: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2021-11-03T17:09:13.895 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:13 smithi027 conmon[32322]: audit 2021-11-03T17:09:12.560694+0000 mon.smithi027 (mon.0) 275 : audit [INF] from='osd.1 [v2:172.21.15.27:6810/2723402359,v1:172.21.15.27:6811/2723402359]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2021-11-03T17:09:13.895 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:13 smithi027 conmon[32322]: cluster 2021-11-03T17:09:12.560781+0000 mon.smithi027 (mon.0) 276 : cluster [DBG] osdmap e12: 2 total, 1 up, 2 in 2021-11-03T17:09:13.895 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:13 smithi027 conmon[32322]: audit 2021-11-03T17:09:12.560906+0000 mon.smithi027 (mon.0) 277 2021-11-03T17:09:13.895 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:13 smithi027 conmon[32322]: : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-11-03T17:09:13.896 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:13 smithi027 conmon[32322]: audit 2021-11-03T17:09:12.561292+0000 mon.smithi027 (mon.0) 278 : audit [INF] from='osd.1 [v2:172.21.15.27:6810/2723402359,v1:172.21.15.27:6811/2723402359]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2021-11-03T17:09:13.955 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:13 smithi115 conmon[34670]: cluster 2021-11-03T17:09:12.543676+0000 mgr.smithi027.ufhhvb (mgr.14162) 111 : cluster [DBG] pgmap v70: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2021-11-03T17:09:13.956 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:13 smithi115 conmon[34670]: audit 2021-11-03T17:09:12.560694+0000 mon.smithi027 (mon.0) 275 : audit [INF] from='osd.1 [v2:172.21.15.27:6810/2723402359,v1:172.21.15.27:6811/2723402359]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2021-11-03T17:09:13.956 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:13 smithi115 conmon[34670]: cluster 2021- 2021-11-03T17:09:13.956 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:13 smithi115 conmon[34670]: 11-03T17:09:12.560781+0000 mon.smithi027 (mon.0) 276 : cluster [DBG] 2021-11-03T17:09:13.957 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:13 smithi115 conmon[34670]: osdmap e12: 2 total, 1 up, 2 in 2021-11-03T17:09:13.957 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:13 smithi115 conmon[34670]: audit 2021-11- 2021-11-03T17:09:13.957 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:13 smithi115 conmon[34670]: 03T17:09:12.560906+0000 2021-11-03T17:09:13.957 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:13 smithi115 conmon[34670]: mon.smithi027 (mon.0) 277 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-11-03T17:09:13.958 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:13 smithi115 conmon[34670]: 2021-11-03T17:09:13.958 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:13 smithi115 conmon[34670]: audit 2021-11-03T 2021-11-03T17:09:13.958 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:13 smithi115 conmon[34670]: 17:09:12.561292+0000 mon.smithi027 (mon 2021-11-03T17:09:13.958 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:13 smithi115 conmon[34670]: .0) 278 : audit [INF] from='osd.1 [v2:172.21.15.27:6810/2723402359,v1:172.21.15.27:6811/2723402359]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2021-11-03T17:09:14.092 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:14.094+0000 7fcb92032700 1 -- 172.21.15.27:0/648211009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb8c0f6880 msgr2=0x7fcb8c0f6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:09:14.093 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:14.094+0000 7fcb92032700 1 --2- 172.21.15.27:0/648211009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb8c0f6880 0x7fcb8c0f6ce0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7fcb74004660 tx=0x7fcb74009b30).stop 2021-11-03T17:09:14.093 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:14.094+0000 7fcb92032700 1 -- 172.21.15.27:0/648211009 shutdown_connections 2021-11-03T17:09:14.093 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:14.094+0000 7fcb92032700 1 --2- 172.21.15.27:0/648211009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb8c0f6880 0x7fcb8c0f6ce0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:09:14.093 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:14.094+0000 7fcb92032700 1 --2- 172.21.15.27:0/648211009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb8c0f55e0 0x7fcb8c0f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:09:14.093 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:14.094+0000 7fcb92032700 1 -- 172.21.15.27:0/648211009 >> 172.21.15.27:0/648211009 conn(0x7fcb8c0f0d40 msgr2=0x7fcb8c0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:09:14.094 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:14.094+0000 7fcb92032700 1 -- 172.21.15.27:0/648211009 shutdown_connections 2021-11-03T17:09:14.094 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:14.094+0000 7fcb92032700 1 -- 172.21.15.27:0/648211009 wait complete. 2021-11-03T17:09:14.094 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:14.095+0000 7fcb92032700 1 Processor -- start 2021-11-03T17:09:14.094 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:14.095+0000 7fcb92032700 1 -- start start 2021-11-03T17:09:14.095 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:14.095+0000 7fcb92032700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb8c0f55e0 0x7fcb8c106f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:09:14.095 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:14.095+0000 7fcb92032700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb8c0f6880 0x7fcb8c107440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:09:14.095 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:14.095+0000 7fcb92032700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c107a80 con 0x7fcb8c0f6880 2021-11-03T17:09:14.095 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:14.095+0000 7fcb92032700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb8c108330 con 0x7fcb8c0f55e0 2021-11-03T17:09:14.096 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:14.095+0000 7fcb8affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb8c0f6880 0x7fcb8c107440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:09:14.096 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:14.095+0000 7fcb8affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb8c0f6880 0x7fcb8c107440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54522/0 (socket says 172.21.15.27:54522) 2021-11-03T17:09:14.096 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:14.095+0000 7fcb8affd700 1 -- 172.21.15.27:0/83744609 learned_addr learned my addr 172.21.15.27:0/83744609 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:09:14.096 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:14.095+0000 7fcb8b7fe700 1 --2- 172.21.15.27:0/83744609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb8c0f55e0 0x7fcb8c106f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:09:14.096 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:14.095+0000 7fcb8affd700 1 -- 172.21.15.27:0/83744609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb8c0f55e0 msgr2=0x7fcb8c106f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:09:14.097 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:14.095+0000 7fcb8affd700 1 --2- 172.21.15.27:0/83744609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb8c0f55e0 0x7fcb8c106f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:09:14.097 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:14.095+0000 7fcb8affd700 1 -- 172.21.15.27:0/83744609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb74005040 con 0x7fcb8c0f6880 2021-11-03T17:09:14.097 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:14.096+0000 7fcb8b7fe700 1 --2- 172.21.15.27:0/83744609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb8c0f55e0 0x7fcb8c106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:09:14.097 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:14.096+0000 7fcb8affd700 1 --2- 172.21.15.27:0/83744609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb8c0f6880 0x7fcb8c107440 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7fcb74009990 tx=0x7fcb74009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:09:14.098 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:14.096+0000 7fcb83fff700 1 -- 172.21.15.27:0/83744609 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb74018070 con 0x7fcb8c0f6880 2021-11-03T17:09:14.098 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:14.096+0000 7fcb83fff700 1 -- 172.21.15.27:0/83744609 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7fcb74013410 con 0x7fcb8c0f6880 2021-11-03T17:09:14.098 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:14.096+0000 7fcb83fff700 1 -- 172.21.15.27:0/83744609 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb7401c620 con 0x7fcb8c0f6880 2021-11-03T17:09:14.098 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:14.096+0000 7fcb92032700 1 -- 172.21.15.27:0/83744609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb8c1085b0 con 0x7fcb8c0f6880 2021-11-03T17:09:14.098 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:14.096+0000 7fcb92032700 1 -- 172.21.15.27:0/83744609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb8c108b10 con 0x7fcb8c0f6880 2021-11-03T17:09:14.100 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:14.097+0000 7fcb92032700 1 -- 172.21.15.27:0/83744609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb8c0f49d0 con 0x7fcb8c0f6880 2021-11-03T17:09:14.100 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:14.097+0000 7fcb83fff700 1 -- 172.21.15.27:0/83744609 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fcb7401c8e0 con 0x7fcb8c0f6880 2021-11-03T17:09:14.100 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:14.097+0000 7fcb83fff700 1 --2- 172.21.15.27:0/83744609 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fcb7805c2b0 0x7fcb7805e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:09:14.100 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:14.098+0000 7fcb83fff700 1 -- 172.21.15.27:0/83744609 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 2001+0+0 (secure 0 0 0) 0x7fcb74077170 con 0x7fcb8c0f6880 2021-11-03T17:09:14.101 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:14.098+0000 7fcb8b7fe700 1 --2- 172.21.15.27:0/83744609 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fcb7805c2b0 0x7fcb7805e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:09:14.101 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:14.098+0000 7fcb8b7fe700 1 --2- 172.21.15.27:0/83744609 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fcb7805c2b0 0x7fcb7805e770 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fcb84006fd0 tx=0x7fcb84008040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:09:14.101 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:14.101+0000 7fcb83fff700 1 -- 172.21.15.27:0/83744609 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fcb7404ab60 con 0x7fcb8c0f6880 2021-11-03T17:09:14.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:14.298+0000 7fcb92032700 1 -- 172.21.15.27:0/83744609 --> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb8c0facc0 con 0x7fcb7805c2b0 2021-11-03T17:09:14.872 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:14 smithi027 conmon[32322]: audit 2021-11-03T17:09:13. 2021-11-03T17:09:14.872 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:14 smithi027 conmon[32322]: 561793+0000 mon.smithi027 (mon.0) 279 : audit [INF] from='osd.1 [v2:172.21.15.27:6810/2723402359,v1:172.21.15.27:6811/2723402359]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2021-11-03T17:09:14.872 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:14 smithi027 conmon[32322]: cluster 2021-11-03T17:09:13.561862+0000 mon.smithi027 (mon.0) 280 : cluster [DBG] osdmap e13: 2 total, 1 up, 2 in 2021-11-03T17:09:14.873 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:14 smithi027 conmon[32322]: audit 2021-11-03T17:09:13.562310+0000 mon.smithi027 (mon.0) 281 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-11-03T17:09:14.873 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:14 smithi027 conmon[32322]: audit 2021-11-03T17:09:13.569084+0000 mon.smithi027 (mon.0) 282 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-11-03T17:09:14.873 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:14 smithi027 conmon[32322]: audit 2021-11-03T17:09:13.653928+0000 mon.smithi027 (mon.0) 283 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:14.873 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:14 smithi027 conmon[32322]: audit 2021-11-03T17:09:14.302272+0000 mon.smithi027 (mon.0) 284 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2021-11-03T17:09:14.873 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:14 smithi027 conmon[32322]: audit 2021-11-03T17:09:14.303654+0000 mon.smithi027 (mon.0) 285 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2021-11-03T17:09:14.874 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:14 smithi027 conmon[32322]: audit 2021-11-03T17:09:14.304249+0000 mon.smithi027 (mon.0) 286 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:09:14.874 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:14 smithi027 conmon[32322]: audit 2021-11-03T17:09:14.565453+0000 2021-11-03T17:09:14.874 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:14 smithi027 conmon[32322]: mon.smithi027 (mon.0) 287 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-11-03T17:09:14.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:14 smithi115 conmon[34670]: audit 2021-11-03T17:09:13.561793+0000 mon.smithi027 (mon.0) 279 : audit [INF] from='osd.1 [v2:172.21.15.27:6810/2723402359,v1:172.21.15.27:6811/2723402359]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2021-11-03T17:09:14.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:14 smithi115 conmon[34670]: cluster 2021-11-03T17:09:13.561862+0000 mon.smithi027 (mon.0) 280 : cluster [DBG] osdmap e13: 2 total, 1 up, 2 in 2021-11-03T17:09:14.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:14 smithi115 conmon[34670]: audit 2021- 2021-11-03T17:09:14.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:14 smithi115 conmon[34670]: 11-03T17:09:13.562310+0000 mon.smithi027 (mon.0) 2021-11-03T17:09:14.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:14 smithi115 conmon[34670]: 281 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-11-03T17:09:14.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:14 smithi115 conmon[34670]: audit 2021- 2021-11-03T17:09:14.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:14 smithi115 conmon[34670]: 11-03T17:09:13. 2021-11-03T17:09:14.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:14 smithi115 conmon[34670]: 569084+0000 mon.smithi027 (mon.0) 282 : audit [DBG] 2021-11-03T17:09:14.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:14 smithi115 conmon[34670]: from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-11-03T17:09:14.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:14 smithi115 conmon[34670]: audit 2021-11-03T 2021-11-03T17:09:14.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:14 smithi115 conmon[34670]: 17:09:13.653928+0000 mon.smithi027 ( 2021-11-03T17:09:14.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:14 smithi115 conmon[34670]: mon.0) 283 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:14.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:14 smithi115 conmon[34670]: audit 2021-11-03T17:09:14.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:14 smithi115 conmon[34670]: 2021-11-03T17:09 2021-11-03T17:09:14.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:14 smithi115 conmon[34670]: :14.302272+0000 mon.smithi027 (mon.0) 284 2021-11-03T17:09:14.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:14 smithi115 conmon[34670]: : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2021-11-03T17:09:14.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:14 smithi115 conmon[34670]: audit 2021-11 2021-11-03T17:09:14.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:14 smithi115 conmon[34670]: -03T17:09:14.303654+0000 mon.smithi027 (mon.0) 285 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2021-11-03T17:09:14.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:14 smithi115 conmon[34670]: audit 2021 2021-11-03T17:09:14.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:14 smithi115 conmon[34670]: -11-03T17:09:14 2021-11-03T17:09:14.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:14 smithi115 conmon[34670]: .304249+0000 mon.smithi027 (mon.0) 286 : audit 2021-11-03T17:09:14.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:14 smithi115 conmon[34670]: [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:09:14.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:14 smithi115 conmon[34670]: audit 2021-11 2021-11-03T17:09:14.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:14 smithi115 conmon[34670]: -03T17:09:14. 2021-11-03T17:09:14.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:14 smithi115 conmon[34670]: 565453+0000 mon.smithi027 (mon.0) 287 : audit 2021-11-03T17:09:14.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:14 smithi115 conmon[34670]: [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-11-03T17:09:15.671 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:15 smithi027 conmon[32322]: cluster 2021-11-03T17:09:12.720458+0000 2021-11-03T17:09:15.672 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:15 smithi027 conmon[32322]: osd.1 (osd.1) 1 : cluster [DBG] purged_snaps scrub starts 2021-11-03T17:09:15.672 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:15 smithi027 conmon[32322]: cluster 2021-11-03T17:09:12.720564+0000 osd.1 (osd.1) 2 : cluster [DBG] purged_snaps scrub ok 2021-11-03T17:09:15.672 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:15 smithi027 conmon[32322]: audit 2021-11-03T17:09:14.301472+0000 mgr.smithi027.ufhhvb (mgr.14162) 112 : audit [DBG] from='client.14244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:09:15.672 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:15 smithi027 conmon[32322]: cephadm 2021-11-03T17:09:14.303441+0000 mgr.smithi027.ufhhvb (mgr.14162) 113 : cephadm [INF] Applying service osd.None on host smithi027... 2021-11-03T17:09:15.673 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:15 smithi027 conmon[32322]: cluster 2021-11-03T17:09:14.543982+0000 mgr.smithi027.ufhhvb (mgr.14162) 114 : cluster [DBG] pgmap v73: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2021-11-03T17:09:15.673 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:15 smithi027 conmon[32322]: cluster 2021-11-03T17:09:14.569774+0000 mon.smithi027 (mon.0) 288 : cluster [INF] osd.1 [v2:172.21.15.27:6810/2723402359,v1:172.21.15.27:6811/2723402359] boot 2021-11-03T17:09:15.673 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:15 smithi027 conmon[32322]: cluster 2021-11-03T17:09:14.569820+0000 mon.smithi027 (mon.0) 289 : cluster [DBG] osdmap e14: 2 total, 2 up, 2 in 2021-11-03T17:09:15.673 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:15 smithi027 conmon[32322]: audit 2021-11-03T17:09:14.570334+0000 mon.smithi027 (mon.0) 290 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-11-03T17:09:15.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:15 smithi115 conmon[34670]: cluster 2021-11-03T17:09:12.720458+0000 osd.1 (osd. 2021-11-03T17:09:15.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:15 smithi115 conmon[34670]: 1) 1 : cluster [DBG] purged_snaps scrub starts 2021-11-03T17:09:15.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:15 smithi115 conmon[34670]: cluster 2021-11-03T17:09:12.720564+0000 osd.1 (osd.1) 2 : cluster [DBG] purged_snaps scrub ok 2021-11-03T17:09:15.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:15 smithi115 conmon[34670]: audit 2021-11-03T17:09:14.301472+0000 mgr.smithi027.ufhhvb 2021-11-03T17:09:15.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:15 smithi115 conmon[34670]: (mgr.14162) 112 : audit [DBG] from='client.14244 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi027:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:09:15.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:15 smithi115 conmon[34670]: cephadm 2021-11 2021-11-03T17:09:15.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:15 smithi115 conmon[34670]: -03T17:09:14.303441+0000 mgr.smithi027.ufhhvb ( 2021-11-03T17:09:15.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:15 smithi115 conmon[34670]: mgr.14162) 113 : cephadm [INF] Applying service osd.None on host smithi027... 2021-11-03T17:09:15.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:15 smithi115 conmon[34670]: cluster 2021- 2021-11-03T17:09:15.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:15 smithi115 conmon[34670]: 11-03T17:09:14.543982+0000 mgr.smithi027.ufhhvb ( 2021-11-03T17:09:15.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:15 smithi115 conmon[34670]: mgr.14162) 114 : cluster [DBG] pgmap v73: 0 pgs: ; 0 B data, 4.7 MiB used, 89 GiB / 89 GiB avail 2021-11-03T17:09:15.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:15 smithi115 conmon[34670]: cluster 2021 2021-11-03T17:09:15.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:15 smithi115 conmon[34670]: -11-03T17:09:14.569774+0000 2021-11-03T17:09:15.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:15 smithi115 conmon[34670]: mon.smithi027 (mon.0) 288 : cluster [INF] osd.1 [v2:172.21.15.27:6810/2723402359,v1:172.21.15.27:6811/2723402359] boot 2021-11-03T17:09:15.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:15 smithi115 conmon[34670]: cluster 2021 2021-11-03T17:09:15.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:15 smithi115 conmon[34670]: -11-03T17:09:14.569820+0000 2021-11-03T17:09:15.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:15 smithi115 conmon[34670]: mon.smithi027 (mon.0) 289 : cluster [DBG] 2021-11-03T17:09:15.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:15 smithi115 conmon[34670]: osdmap e14: 2 total, 2 up, 2 in 2021-11-03T17:09:15.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:15 smithi115 conmon[34670]: audit 2021-11-03T17:09:14. 2021-11-03T17:09:15.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:15 smithi115 conmon[34670]: 570334+0000 mon.smithi027 (mon.0) 290 : audit 2021-11-03T17:09:15.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:15 smithi115 conmon[34670]: [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-11-03T17:09:16.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:16 smithi027 conmon[32322]: cluster 2021-11-03T17:09:15.656393+0000 mon.smithi027 (mon.0) 291 : cluster [DBG] osdmap e15: 2 total, 2 up, 2 in 2021-11-03T17:09:16.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:16 smithi027 conmon[32322]: cluster 2021-11-03T17:09:16.544263+0000 mgr.smithi027.ufhhvb (mgr.14162) 115 : cluster [DBG] pgmap v76: 0 pgs: ; 0 B data, 9.5 MiB used, 179 GiB / 179 GiB avail 2021-11-03T17:09:16.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:16 smithi115 conmon[34670]: cluster 2021-11-03T17:09:15.656393+0000 mon.smithi027 (mon. 2021-11-03T17:09:16.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:16 smithi115 conmon[34670]: 0) 291 : cluster [DBG] osdmap e15: 2 total, 2 up, 2 in 2021-11-03T17:09:16.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:16 smithi115 conmon[34670]: cluster 2021-11-03T17:09:16.544263+0000 mgr.smithi027.ufhhvb (mgr.14162) 115 : cluster [DBG] 2021-11-03T17:09:16.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:16 smithi115 conmon[34670]: pgmap v76: 0 pgs: ; 0 B data, 9.5 MiB used, 179 GiB / 179 GiB avail 2021-11-03T17:09:18.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:18 smithi115 conmon[34670]: audit 2021-11-03T17:09:17.736641+0000 mon.smithi027 (mon 2021-11-03T17:09:18.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:18 smithi115 conmon[34670]: .0) 292 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:18.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:18 smithi115 conmon[34670]: audit 2021-11-03T17:09:17.740989+0000 mon.smithi027 (mon.0) 293 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:18.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:18 smithi115 conmon[34670]: audit 2021-11-03T17:09:17.743713+0000 mon.smithi027 (mon 2021-11-03T17:09:18.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:18 smithi115 conmon[34670]: .0) 294 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:09:18.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:18 smithi115 conmon[34670]: audit 2021-11-03T17:09 2021-11-03T17:09:18.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:18 smithi115 conmon[34670]: :18.428746+0000 mon.smithi027 (mon.0) 295 : audit 2021-11-03T17:09:18.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:18 smithi115 conmon[34670]: [INF] from='client.? 172.21.15.27:0/3793824698' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ea602d0-3e10-4f30-96cd-288e53a7a00a"}]: dispatch 2021-11-03T17:09:18.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:18 smithi115 conmon[34670]: audit 2021-11-03T17: 2021-11-03T17:09:18.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:18 smithi115 conmon[34670]: 09:18.432661+0000 mon.smithi027 (mon.0) 296 : 2021-11-03T17:09:18.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:18 smithi115 conmon[34670]: audit [INF] from='client.? 172.21.15.27:0/3793824698' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ea602d0-3e10-4f30-96cd-288e53a7a00a"}]': finished 2021-11-03T17:09:18.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:18 smithi115 conmon[34670]: cluster 2021-11-03T17: 2021-11-03T17:09:18.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:18 smithi115 conmon[34670]: 09:18.432709+0000 mon.smithi027 (mon.0) 2021-11-03T17:09:18.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:18 smithi115 conmon[34670]: 297 : cluster [DBG] osdmap e16: 3 total, 2 up, 3 in 2021-11-03T17:09:18.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:18 smithi115 conmon[34670]: audit 2021-11- 2021-11-03T17:09:18.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:18 smithi115 conmon[34670]: 03T17:09:18.432808+0000 mon.smithi027 (mon 2021-11-03T17:09:18.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:18 smithi115 conmon[34670]: .0) 298 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-11-03T17:09:18.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:18 smithi115 conmon[34670]: cluster 2021-11 2021-11-03T17:09:18.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:18 smithi115 conmon[34670]: -03T17:09:18.544571+0000 mgr.smithi027.ufhhvb ( 2021-11-03T17:09:18.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:18 smithi115 conmon[34670]: mgr.14162) 116 : cluster [DBG] pgmap v78: 0 pgs: ; 0 B data, 9.5 MiB used, 179 GiB / 179 GiB avail 2021-11-03T17:09:19.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:18 smithi027 conmon[32322]: audit 2021-11-03T17:09:17.736641+0000 mon.smithi027 (mon.0) 292 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:19.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:18 smithi027 conmon[32322]: audit 2021-11-03T17:09:17.740989+0000 mon.smithi027 (mon.0) 293 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:19.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:18 smithi027 conmon[32322]: audit 2021-11-03T17:09:17.743713+0000 mon.smithi027 (mon.0) 294 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:09:19.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:18 smithi027 conmon[32322]: audit 2021-11-03T17:09:18.428746+0000 mon.smithi027 (mon.0) 295 : audit [INF] from='client.? 172.21.15.27:0/3793824698' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "3ea602d0-3e10-4f30-96cd-288e53a7a00a"}]: dispatch 2021-11-03T17:09:19.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:18 smithi027 conmon[32322]: audit 2021-11-03T17:09:18.432661+0000 mon.smithi027 (mon.0) 296 : audit [INF] from='client.? 172.21.15.27:0/3793824698' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ea602d0-3e10-4f30-96cd-288e53a7a00a"}]': finished 2021-11-03T17:09:19.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:18 smithi027 conmon[32322]: cluster 2021-11-03T17:09:18.432709+0000 mon.smithi027 (mon.0) 297 : cluster [DBG] osdmap e16: 3 total, 2 up, 3 in 2021-11-03T17:09:19.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:18 smithi027 conmon[32322]: audit 2021-11-03T17:09: 2021-11-03T17:09:19.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:18 smithi027 conmon[32322]: 18.432808+0000 mon.smithi027 (mon.0) 298 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-11-03T17:09:19.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:18 smithi027 conmon[32322]: cluster 2021-11-03T17:09:18.544571+0000 mgr.smithi027.ufhhvb (mgr.14162) 116 : cluster [DBG] pgmap v78: 0 pgs: ; 0 B data, 9.5 MiB used, 179 GiB / 179 GiB avail 2021-11-03T17:09:19.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:19 smithi115 conmon[34670]: audit 2021-11-03T17:09:18.991384+0000 mon.smithi027 (mon.0) 299 : audit [DBG] from='client.? 172.21.15.27:0/316762171' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2021-11-03T17:09:20.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:19 smithi027 conmon[32322]: audit 2021-11-03T17:09:18.991384+0000 mon.smithi027 ( 2021-11-03T17:09:20.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:19 smithi027 conmon[32322]: mon.0) 299 : audit [DBG] from='client.? 172.21.15.27:0/316762171' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2021-11-03T17:09:21.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:20 smithi027 conmon[32322]: cluster 2021-11-03T17:09:20.544933+0000 2021-11-03T17:09:21.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:20 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14162) 117 : cluster [DBG] pgmap v79: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2021-11-03T17:09:21.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:20 smithi115 conmon[34670]: cluster 2021-11-03T17:09:20.544933+0000 mgr.smithi027.ufhhvb (mgr 2021-11-03T17:09:21.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:20 smithi115 conmon[34670]: .14162) 117 : cluster [DBG] pgmap v79: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2021-11-03T17:09:22.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:22 smithi027 conmon[32322]: audit 2021-11-03T17:09:21.430057+0000 mon.smithi027 (mon.0) 300 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:22.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:22 smithi027 conmon[32322]: audit 2021-11-03T17:09:21.706398+0000 mon.smithi027 (mon.0) 301 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:22.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:22 smithi115 conmon[34670]: audit 2021-11-03T17:09:21.430057+0000 mon.smithi027 (mon.0) 2021-11-03T17:09:22.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:22 smithi115 conmon[34670]: 300 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:22.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:22 smithi115 conmon[34670]: audit 2021-11-03T17:09:21.706398+0000 mon.smithi027 (mon.0) 301 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:23.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:23 smithi115 conmon[34670]: cluster 2021-11-03T17:09:22.545220+0000 mgr.smithi027.ufhhvb (mgr.14162) 118 : cluster [DBG] 2021-11-03T17:09:23.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:23 smithi115 conmon[34670]: pgmap v80: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2021-11-03T17:09:23.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:23 smithi027 conmon[32322]: cluster 2021-11-03T17:09:22.545220+0000 2021-11-03T17:09:23.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:23 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14162) 118 : cluster [DBG] pgmap v80: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2021-11-03T17:09:24.692 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:24 smithi027 conmon[32322]: cluster 2021-11-03T17:09:24.545621+0000 mgr.smithi027.ufhhvb (mgr.14162) 119 : cluster [DBG] pgmap v81: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2021-11-03T17:09:24.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:24 smithi115 conmon[34670]: cluster 2021-11-03T17:09:24.545621+0000 mgr.smithi027.ufhhvb (mgr.14162) 119 : 2021-11-03T17:09:24.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:24 smithi115 conmon[34670]: cluster [DBG] pgmap v81: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2021-11-03T17:09:26.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:25 smithi115 conmon[34670]: audit 2021-11-03T17:09:25.838508+0000 mon.smithi027 2021-11-03T17:09:26.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:25 smithi115 conmon[34670]: (mon.0) 302 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2021-11-03T17:09:26.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:25 smithi115 conmon[34670]: audit 2021-11-03T17:09:25.839213+0000 mon.smithi027 (mon.0) 2021-11-03T17:09:26.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:25 smithi115 conmon[34670]: 303 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:09:26.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:25 smithi027 conmon[32322]: audit 2021-11-03T17:09:25.838508+0000 mon.smithi027 (mon.0) 302 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2021-11-03T17:09:26.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:25 smithi027 conmon[32322]: 2021-11-03T17:09:26.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:25 smithi027 conmon[32322]: audit 2021-11-03T17:09:25.839213+0000 mon.smithi027 (mon.0) 303 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:09:27.608 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:27 smithi027 conmon[32322]: cephadm 2021-11-03T17:09:25.839798+0000 mgr.smithi027.ufhhvb (mgr.14162) 120 : cephadm [INF] Deploying daemon osd.2 on smithi027 2021-11-03T17:09:27.609 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:27 smithi027 conmon[32322]: cluster 2021-11-03T17:09:26.545910+0000 mgr.smithi027.ufhhvb (mgr.14162) 121 : cluster [DBG] pgmap v82: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2021-11-03T17:09:27.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:27 smithi115 conmon[34670]: cephadm 2021-11-03T17:09:25.839798+0000 mgr.smithi027.ufhhvb (mgr.14162 2021-11-03T17:09:27.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:27 smithi115 conmon[34670]: ) 120 : cephadm [INF] Deploying daemon osd.2 on smithi027 2021-11-03T17:09:27.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:27 smithi115 conmon[34670]: cluster 2021-11-03T17:09:26.545910+0000 mgr.smithi027.ufhhvb (mgr.14162) 121 : cluster [DBG] pgmap v82: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2021-11-03T17:09:29.426 INFO:teuthology.orchestra.run.smithi027.stdout:Created osd(s) 2 on host 'smithi027' 2021-11-03T17:09:29.427 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:29.429+0000 7fcb83fff700 1 -- 172.21.15.27:0/83744609 <== mgr.14162 v2:172.21.15.27:6800/3802716887 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fcb8c0facc0 con 0x7fcb7805c2b0 2021-11-03T17:09:29.427 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:29.432+0000 7fcb92032700 1 -- 172.21.15.27:0/83744609 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fcb7805c2b0 msgr2=0x7fcb7805e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:09:29.427 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:29.432+0000 7fcb92032700 1 --2- 172.21.15.27:0/83744609 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fcb7805c2b0 0x7fcb7805e770 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fcb84006fd0 tx=0x7fcb84008040).stop 2021-11-03T17:09:29.427 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:29.432+0000 7fcb92032700 1 -- 172.21.15.27:0/83744609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb8c0f6880 msgr2=0x7fcb8c107440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:09:29.428 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:29.432+0000 7fcb92032700 1 --2- 172.21.15.27:0/83744609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb8c0f6880 0x7fcb8c107440 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7fcb74009990 tx=0x7fcb74009b30).stop 2021-11-03T17:09:29.428 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:29.434+0000 7fcb92032700 1 -- 172.21.15.27:0/83744609 shutdown_connections 2021-11-03T17:09:29.429 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:29.434+0000 7fcb92032700 1 --2- 172.21.15.27:0/83744609 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fcb7805c2b0 0x7fcb7805e770 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:09:29.429 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:29.434+0000 7fcb92032700 1 --2- 172.21.15.27:0/83744609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb8c0f55e0 0x7fcb8c106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:09:29.429 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:29.434+0000 7fcb92032700 1 --2- 172.21.15.27:0/83744609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb8c0f6880 0x7fcb8c107440 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:09:29.429 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:29.434+0000 7fcb92032700 1 -- 172.21.15.27:0/83744609 >> 172.21.15.27:0/83744609 conn(0x7fcb8c0f0d40 msgr2=0x7fcb8c05ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:09:29.430 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:29.434+0000 7fcb92032700 1 -- 172.21.15.27:0/83744609 shutdown_connections 2021-11-03T17:09:29.430 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:09:29.434+0000 7fcb92032700 1 -- 172.21.15.27:0/83744609 wait complete. 2021-11-03T17:09:29.625 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:29 smithi027 conmon[32322]: cluster 2021-11-03T17:09:28.546223+0000 mgr.smithi027.ufhhvb (mgr.14162) 122 : cluster 2021-11-03T17:09:29.625 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:29 smithi027 conmon[32322]: [DBG] pgmap v83: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2021-11-03T17:09:29.626 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:29 smithi027 conmon[32322]: audit 2021-11-03T17:09:29.419323+0000 mon.smithi027 (mon.0) 304 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:09:29.626 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:29 smithi027 conmon[32322]: audit 2021-11-03T17:09:29.423306+0000 mon.smithi027 (mon.0) 305 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:29.626 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:29 smithi027 conmon[32322]: audit 2021-11-03T17:09:29.429149+0000 mon.smithi027 (mon.0) 306 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:29.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:29 smithi115 conmon[34670]: cluster 2021-11-03T17:09:28.546223+0000 mgr.smithi027.ufhhvb (mgr.14162) 122 : cluster [DBG] pgmap v83: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2021-11-03T17:09:29.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:29 smithi115 conmon[34670]: audit 2021-11-03T17:09:29.419323+0000 mon.smithi027 (mon.0) 304 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:09:29.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:29 smithi115 conmon[34670]: audit 2021-11-03T17:09:29.423306+0000 mon.smithi027 (mon.0) 305 2021-11-03T17:09:29.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:29 smithi115 conmon[34670]: : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:29.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:29 smithi115 conmon[34670]: audit 2021-11-03 2021-11-03T17:09:29.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:29 smithi115 conmon[34670]: T17:09:29.429149+0000 mon.smithi027 (mon.0) 306 : audit [INF] 2021-11-03T17:09:29.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:29 smithi115 conmon[34670]: from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:30.063 DEBUG:teuthology.orchestra.run.smithi027:osd.2> sudo journalctl -f -n 0 -u ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c@osd.2.service 2021-11-03T17:09:30.066 INFO:tasks.cephadm:Deploying osd.3 on smithi115 with /dev/vg_nvme/lv_4... 2021-11-03T17:09:30.067 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2021-11-03T17:09:30.096 INFO:journalctl@ceph.osd.2.smithi027.stdout:-- Logs begin at Wed 2021-11-03 16:54:54 UTC. -- 2021-11-03T17:09:31.800 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:31 smithi115 conmon[34670]: cluster 2021-11-03T17:09:30.546702+0000 2021-11-03T17:09:31.800 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:31 smithi115 conmon[34670]: mgr.smithi027.ufhhvb (mgr.14162) 123 : cluster [DBG] pgmap v84: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2021-11-03T17:09:32.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:31 smithi027 conmon[32322]: cluster 2021-11-03T17:09:30.546702+0000 2021-11-03T17:09:32.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:31 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14162) 123 : cluster [DBG] pgmap v84: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2021-11-03T17:09:32.216 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping: /dev/vg_nvme/lv_4 2021-11-03T17:09:32.216 INFO:teuthology.orchestra.run.smithi115.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2021-11-03T17:09:32.216 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10+0 records in 2021-11-03T17:09:32.217 INFO:teuthology.orchestra.run.smithi115.stderr:10+0 records out 2021-11-03T17:09:32.217 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110687 s, 947 MB/s 2021-11-03T17:09:32.217 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping successful for: 2021-11-03T17:09:32.892 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph orch daemon add osd smithi115:vg_nvme/lv_4 2021-11-03T17:09:33.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:33 smithi115 conmon[34670]: cluster 2021-11-03T17:09:32.546982+0000 mgr.smithi027.ufhhvb (mgr.14162) 124 : cluster [DBG] pgmap v85: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2021-11-03T17:09:33.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:33 smithi115 conmon[34670]: audit 2021-11-03T17:09:33.427097+0000 mon.smithi027 (mon.0) 307 : audit [INF] from='osd.2 [v2:172.21.15.27:6818/329182726,v1:172.21.15.27:6819/329182726]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2021-11-03T17:09:33.769 INFO:journalctl@ceph.osd.2.smithi027.stdout:Nov 03 17:09:33 smithi027 conmon[63140]: debug 2021-11-03T17:09:33.423+0000 7fc2a5618080 -1 osd.2 0 log_to_monitors {default=true} 2021-11-03T17:09:33.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:33 smithi027 conmon[32322]: cluster 2021-11-03T17:09:32.546982+0000 2021-11-03T17:09:33.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:33 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14162) 124 : cluster [DBG] pgmap v85: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2021-11-03T17:09:33.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:33 smithi027 conmon[32322]: audit 2021-11-03T17:09:33.427097+0000 mon.smithi027 (mon.0) 307 2021-11-03T17:09:33.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:33 smithi027 conmon[32322]: : audit [INF] from='osd.2 [v2:172.21.15.27:6818/329182726,v1:172.21.15.27:6819/329182726]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2021-11-03T17:09:34.585 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:34 smithi115 conmon[34670]: audit 2021-11-03T17:09:33.565739+0000 mon.smithi027 (mon.0) 308 : audit [INF] from='osd.2 [v2:172.21.15.27:6818/329182726,v1:172.21.15.27:6819/329182726]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2021-11-03T17:09:34.585 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:34 smithi115 conmon[34670]: 2021-11-03T17:09:34.586 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:34 smithi115 conmon[34670]: cluster 2021-11-03T17:09:33.565827+0000 mon.smithi027 (mon.0) 309 : cluster [DBG] osdmap e17: 3 total, 2 up, 3 in 2021-11-03T17:09:34.586 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:34 smithi115 conmon[34670]: audit 2021-11-03T17:09:33.565967+0000 mon.smithi027 (mon.0) 310 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-11-03T17:09:34.586 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:34 smithi115 conmon[34670]: audit 2021-11-03T17:09:33.566360+0000 mon.smithi027 (mon.0) 311 : audit [INF] from='osd.2 [v2:172.21.15.27:6818/329182726,v1:172.21.15.27:6819/329182726]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2021-11-03T17:09:34.586 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:34 smithi115 conmon[34670]: audit 2021-11-03T17:09:33.645949+0000 mon.smithi027 (mon.0) 312 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:34.769 INFO:journalctl@ceph.osd.2.smithi027.stdout:Nov 03 17:09:34 smithi027 conmon[63140]: debug 2021-11-03T17:09:34.569+0000 7fc29bb89700 -1 osd.2 0 waiting for initial osdmap 2021-11-03T17:09:34.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:34 smithi027 conmon[32322]: audit 2021-11-03T17:09:33. 2021-11-03T17:09:34.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:34 smithi027 conmon[32322]: 565739+0000 mon.smithi027 (mon.0) 308 : audit [INF] from='osd.2 [v2:172.21.15.27:6818/329182726,v1:172.21.15.27:6819/329182726]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2021-11-03T17:09:34.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:34 smithi027 conmon[32322]: cluster 2021-11-03T17:09:33.565827+0000 mon.smithi027 (mon.0) 309 : cluster [DBG] osdmap e17: 3 total, 2 up, 3 in 2021-11-03T17:09:34.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:34 smithi027 conmon[32322]: audit 2021-11-03T17:09:33.565967+0000 mon.smithi027 (mon.0) 310 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-11-03T17:09:34.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:34 smithi027 conmon[32322]: audit 2021-11-03T17:09:34.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:34 smithi027 conmon[32322]: 2021-11-03T17:09: 2021-11-03T17:09:34.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:34 smithi027 conmon[32322]: 33.566360+0000 mon.smithi027 (mon.0) 311 2021-11-03T17:09:34.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:34 smithi027 conmon[32322]: : audit [INF] from='osd.2 [v2:172.21.15.27:6818/329182726,v1:172.21.15.27:6819/329182726]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2021-11-03T17:09:34.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:34 smithi027 conmon[32322]: audit 2021-11-03T17:09:34.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:34 smithi027 conmon[32322]: 2021-11-03T17:09:33.645949 2021-11-03T17:09:34.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:34 smithi027 conmon[32322]: +0000 mon.smithi027 (mon.0) 312 : audit 2021-11-03T17:09:34.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:34 smithi027 conmon[32322]: [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:34.793 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:34.797+0000 7f859dcbb700 1 -- 172.21.15.115:0/786468463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85980f5ef0 msgr2=0x7f85980f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:09:34.793 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:34.797+0000 7f859dcbb700 1 --2- 172.21.15.115:0/786468463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85980f5ef0 0x7f85980f6920 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7f858c004660 tx=0x7f858c009b30).stop 2021-11-03T17:09:34.793 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:34.798+0000 7f859dcbb700 1 -- 172.21.15.115:0/786468463 shutdown_connections 2021-11-03T17:09:34.793 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:34.798+0000 7f859dcbb700 1 --2- 172.21.15.115:0/786468463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85980f5ef0 0x7f85980f6920 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:09:34.794 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:34.798+0000 7f859dcbb700 1 --2- 172.21.15.115:0/786468463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85980f55e0 0x7f85980f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:09:34.794 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:34.798+0000 7f859dcbb700 1 -- 172.21.15.115:0/786468463 >> 172.21.15.115:0/786468463 conn(0x7f85980f0de0 msgr2=0x7f85980f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:09:34.794 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:34.798+0000 7f859dcbb700 1 -- 172.21.15.115:0/786468463 shutdown_connections 2021-11-03T17:09:34.794 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:34.798+0000 7f859dcbb700 1 -- 172.21.15.115:0/786468463 wait complete. 2021-11-03T17:09:34.795 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:34.798+0000 7f859dcbb700 1 Processor -- start 2021-11-03T17:09:34.795 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:34.798+0000 7f859dcbb700 1 -- start start 2021-11-03T17:09:34.797 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:34.798+0000 7f859dcbb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85980f55e0 0x7f8598109c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:09:34.798 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:34.798+0000 7f859dcbb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85980f5ef0 0x7f859810c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:09:34.798 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:34.798+0000 7f859dcbb700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8598060500 con 0x7f85980f5ef0 2021-11-03T17:09:34.798 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:34.798+0000 7f859dcbb700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8598060640 con 0x7f85980f55e0 2021-11-03T17:09:34.798 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:34.799+0000 7f85977fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85980f55e0 0x7f8598109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:09:34.799 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:34.799+0000 7f85977fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85980f55e0 0x7f8598109c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38582/0 (socket says 172.21.15.115:38582) 2021-11-03T17:09:34.799 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:34.799+0000 7f85977fe700 1 -- 172.21.15.115:0/1295621900 learned_addr learned my addr 172.21.15.115:0/1295621900 (peer_addr_for_me v2:172.21.15.115:0/0) 2021-11-03T17:09:34.799 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:34.799+0000 7f8596ffd700 1 --2- 172.21.15.115:0/1295621900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85980f5ef0 0x7f859810c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:09:34.799 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:34.799+0000 7f85977fe700 1 -- 172.21.15.115:0/1295621900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85980f5ef0 msgr2=0x7f859810c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:09:34.800 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:34.799+0000 7f85977fe700 1 --2- 172.21.15.115:0/1295621900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85980f5ef0 0x7f859810c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:09:34.800 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:34.799+0000 7f85977fe700 1 -- 172.21.15.115:0/1295621900 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f858c005040 con 0x7f85980f55e0 2021-11-03T17:09:34.800 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:34.799+0000 7f85977fe700 1 --2- 172.21.15.115:0/1295621900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85980f55e0 0x7f8598109c70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7f85880026d0 tx=0x7f8588009de0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:09:34.800 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:34.799+0000 7f857ffff700 1 -- 172.21.15.115:0/1295621900 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85880079c0 con 0x7f85980f55e0 2021-11-03T17:09:34.800 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:34.799+0000 7f857ffff700 1 -- 172.21.15.115:0/1295621900 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f858800f040 con 0x7f85980f55e0 2021-11-03T17:09:34.801 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:34.799+0000 7f859dcbb700 1 -- 172.21.15.115:0/1295621900 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f859810c720 con 0x7f85980f55e0 2021-11-03T17:09:34.801 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:34.799+0000 7f857ffff700 1 -- 172.21.15.115:0/1295621900 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8588014820 con 0x7f85980f55e0 2021-11-03T17:09:34.801 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:34.799+0000 7f859dcbb700 1 -- 172.21.15.115:0/1295621900 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f859810cce0 con 0x7f85980f55e0 2021-11-03T17:09:34.801 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:34.800+0000 7f859dcbb700 1 -- 172.21.15.115:0/1295621900 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8598045ad0 con 0x7f85980f55e0 2021-11-03T17:09:34.802 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:34.800+0000 7f857ffff700 1 -- 172.21.15.115:0/1295621900 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f85880090d0 con 0x7f85980f55e0 2021-11-03T17:09:34.802 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:34.800+0000 7f857ffff700 1 --2- 172.21.15.115:0/1295621900 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f858005c2a0 0x7f858005e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:09:34.802 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:34.801+0000 7f857ffff700 1 -- 172.21.15.115:0/1295621900 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 2433+0+0 (secure 0 0 0) 0x7f8588075230 con 0x7f85980f55e0 2021-11-03T17:09:34.802 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:34.801+0000 7f8596ffd700 1 --2- 172.21.15.115:0/1295621900 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f858005c2a0 0x7f858005e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:09:34.802 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:34.801+0000 7f8596ffd700 1 --2- 172.21.15.115:0/1295621900 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f858005c2a0 0x7f858005e760 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f858c004e30 tx=0x7f858c009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:09:34.803 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:34.805+0000 7f857ffff700 1 -- 172.21.15.115:0/1295621900 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f8588048ba0 con 0x7f85980f55e0 2021-11-03T17:09:34.975 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:34.980+0000 7f859dcbb700 1 -- 172.21.15.115:0/1295621900 --> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f85980fdf40 con 0x7f858005c2a0 2021-11-03T17:09:35.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:35 smithi115 conmon[34670]: cluster 2021-11-03T17:09:34.547233+0000 mgr.smithi027.ufhhvb 2021-11-03T17:09:35.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:35 smithi115 conmon[34670]: (mgr.14162) 125 : cluster [DBG] pgmap v87: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2021-11-03T17:09:35.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:35 smithi115 conmon[34670]: audit 2021-11-03T17:09:34.565916+0000 mon.smithi027 (mon.0) 313 : audit [INF] from='osd.2 [v2:172.21.15.27:6818/329182726,v1:172.21.15.27:6819/329182726]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2021-11-03T17:09:35.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:35 smithi115 conmon[34670]: cluster 2021-11-03T17:09:34.565970+0000 mon.smithi027 (mon.0) 314 : cluster [DBG] osdmap e18: 3 total, 2 up, 3 in 2021-11-03T17:09:35.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:35 smithi115 conmon[34670]: audit 2021-11-03T17:09:34.566811+0000 mon.smithi027 (mon.0) 315 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-11-03T17:09:35.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:35 smithi115 conmon[34670]: audit 2021-11-03T17:09:34.574370 2021-11-03T17:09:35.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:35 smithi115 conmon[34670]: +0000 mon.smithi027 (mon.0) 316 : 2021-11-03T17:09:35.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:35 smithi115 conmon[34670]: audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-11-03T17:09:35.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:35 smithi115 conmon[34670]: audit 2021-11-03T17:09:35.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:35 smithi115 conmon[34670]: 2021-11-03T17:09 2021-11-03T17:09:35.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:35 smithi115 conmon[34670]: :34.982385+0000 mon.smithi027 ( 2021-11-03T17:09:35.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:35 smithi115 conmon[34670]: mon.0) 317 : audit [DBG] 2021-11-03T17:09:35.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:35 smithi115 conmon[34670]: from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2021-11-03T17:09:35.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:35 smithi115 conmon[34670]: audit 2021-11-03T17:09:35.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:35 smithi115 conmon[34670]: 2021-11-03T17: 2021-11-03T17:09:35.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:35 smithi115 conmon[34670]: 09:34.983981+0000 mon.smithi027 2021-11-03T17:09:35.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:35 smithi115 conmon[34670]: (mon.0) 318 : audit 2021-11-03T17:09:35.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:35 smithi115 conmon[34670]: [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2021-11-03T17:09:35.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:35 smithi115 conmon[34670]: audit 2021-11-03T17:09:35.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:35 smithi115 conmon[34670]: 2021-11-03T17:09 2021-11-03T17:09:35.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:35 smithi115 conmon[34670]: :34.984606+0000 mon.smithi027 ( 2021-11-03T17:09:35.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:35 smithi115 conmon[34670]: mon.0) 319 : audit [DBG] 2021-11-03T17:09:35.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:35 smithi115 conmon[34670]: from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:09:35.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:35 smithi115 conmon[34670]: audit 2021-11-03T17:09:35.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:35 smithi115 conmon[34670]: 2021-11-03T17: 2021-11-03T17:09:35.741 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:35 smithi115 conmon[34670]: 09:35.569564+0000 mon.smithi027 2021-11-03T17:09:35.741 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:35 smithi115 conmon[34670]: (mon.0) 320 : audit 2021-11-03T17:09:35.741 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:35 smithi115 conmon[34670]: [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-11-03T17:09:36.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:35 smithi027 conmon[32322]: cluster 2021-11-03T17:09:34.547233+0000 mgr.smithi027.ufhhvb 2021-11-03T17:09:36.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:35 smithi027 conmon[32322]: (mgr.14162) 125 : cluster [DBG] pgmap v87: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2021-11-03T17:09:36.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:35 smithi027 conmon[32322]: audit 2021-11-03T17:09:34.565916+0000 mon.smithi027 (mon.0) 313 : audit [INF] from='osd.2 [v2:172.21.15.27:6818/329182726,v1:172.21.15.27:6819/329182726]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]': finished 2021-11-03T17:09:36.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:35 smithi027 conmon[32322]: cluster 2021-11-03T17:09:34.565970+0000 mon.smithi027 (mon.0) 314 : cluster [DBG] osdmap e18: 3 total, 2 up, 3 in 2021-11-03T17:09:36.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:35 smithi027 conmon[32322]: audit 2021-11-03T17:09:34.566811+0000 mon.smithi027 (mon.0) 315 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-11-03T17:09:36.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:35 smithi027 conmon[32322]: audit 2021-11-03T17:09:34.574370+0000 mon.smithi027 (mon.0) 316 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-11-03T17:09:36.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:35 smithi027 conmon[32322]: audit 2021-11-03T17:09:34.982385+0000 mon.smithi027 (mon.0) 317 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2021-11-03T17:09:36.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:35 smithi027 conmon[32322]: audit 2021-11-03T17:09:34.983981+0000 mon.smithi027 (mon.0) 318 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2021-11-03T17:09:36.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:35 smithi027 conmon[32322]: audit 2021-11-03T17:09:34.984606+0000 mon.smithi027 (mon.0) 319 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:09:36.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:35 smithi027 conmon[32322]: audit 2021-11-03T17:09:35.569564+0000 mon.smithi027 (mon.0) 320 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-11-03T17:09:36.722 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:36 smithi115 conmon[34670]: cluster 2021-11-03T17:09:34.450723+0000 osd.2 (osd.2) 1 : cluster 2021-11-03T17:09:36.723 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:36 smithi115 conmon[34670]: [DBG] purged_snaps scrub starts 2021-11-03T17:09:36.723 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:36 smithi115 conmon[34670]: cluster 2021-11-03T17:09:34.450924+0000 osd.2 (osd.2) 2 : cluster [DBG] purged_snaps scrub ok 2021-11-03T17:09:36.723 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:36 smithi115 conmon[34670]: audit 2021-11-03T17:09:34.981602+0000 mgr.smithi027.ufhhvb (mgr.14162) 126 : audit [DBG] from='client.24133 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:09:36.723 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:36 smithi115 conmon[34670]: cephadm 2021-11-03T17:09:34.983759+0000 mgr.smithi027.ufhhvb (mgr.14162) 127 : cephadm [INF] Applying service osd.None on host smithi115... 2021-11-03T17:09:36.724 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:36 smithi115 conmon[34670]: cluster 2021-11-03T17:09:35.574367+0000 mon.smithi027 (mon.0) 321 : cluster [INF] osd.2 [v2:172.21.15.27:6818/329182726,v1:172.21.15.27:6819/329182726] boot 2021-11-03T17:09:36.724 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:36 smithi115 conmon[34670]: cluster 2021-11-03T17:09:35.574395+0000 mon.smithi027 (mon.0) 322 : cluster [DBG] osdmap e19: 3 total, 3 up, 3 in 2021-11-03T17:09:36.724 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:36 smithi115 conmon[34670]: audit 2021-11-03T17:09:35.574772+0000 mon.smithi027 (mon.0) 323 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-11-03T17:09:37.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:36 smithi027 conmon[32322]: cluster 2021-11-03T17:09:34.450723+0000 osd.2 (osd.2) 1 : cluster [DBG] purged_snaps scrub starts 2021-11-03T17:09:37.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:36 smithi027 conmon[32322]: cluster 2021-11-03T17:09:34.450924+0000 osd.2 (osd.2) 2 : cluster [DBG] purged_snaps scrub ok 2021-11-03T17:09:37.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:36 smithi027 conmon[32322]: audit 2021-11-03T17:09:34.981602+0000 mgr.smithi027.ufhhvb (mgr.14162) 126 : audit [DBG] from='client.24133 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:09:37.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:36 smithi027 conmon[32322]: cephadm 2021-11-03T17:09:34.983759+0000 mgr.smithi027.ufhhvb (mgr.14162) 127 : cephadm [INF] Applying service osd.None on host smithi115... 2021-11-03T17:09:37.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:36 smithi027 conmon[32322]: cluster 2021-11-03T17:09:35.574367+0000 mon.smithi027 (mon.0) 321 : cluster [INF] osd.2 [v2:172.21.15.27:6818/329182726,v1:172.21.15.27:6819/329182726] boot 2021-11-03T17:09:37.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:36 smithi027 conmon[32322]: cluster 2021-11-03T17:09:35.574395+0000 mon.smithi027 (mon 2021-11-03T17:09:37.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:36 smithi027 conmon[32322]: .0) 322 : cluster [DBG] osdmap e19: 3 total, 3 up, 3 in 2021-11-03T17:09:37.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:36 smithi027 conmon[32322]: audit 2021-11-03T17:09:35.574772+0000 mon.smithi027 (mon.0) 323 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-11-03T17:09:37.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:37 smithi115 conmon[34670]: cluster 2021-11-03T17:09:36.547491+0000 mgr.smithi027.ufhhvb (mgr.14162) 128 : cluster [DBG] pgmap v90: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2021-11-03T17:09:37.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:37 smithi115 conmon[34670]: cluster 2021-11-03T17:09:36.585484+0000 mon.smithi027 (mon.0) 324 : cluster [DBG] osdmap e20: 3 total, 3 up, 3 in 2021-11-03T17:09:37.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:37 smithi115 conmon[34670]: audit 2021-11-03T17:09:37.033541+0000 mon.smithi027 (mon.0) 325 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:37.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:37 smithi115 conmon[34670]: audit 2021-11-03T17:09:37.038709+0000 mon.smithi027 (mon.0) 326 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:37.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:37 smithi115 conmon[34670]: audit 2021-11-03T17:09:37.041410+0000 mon.smithi027 (mon.0) 327 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:09:37.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:37 smithi115 conmon[34670]: audit 2021-11-03T17:09:37.393793+0000 mon.smithi115 (mon.1) 2 : audit [INF] from='client.? 172.21.15.115:0/3596975042' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c517d190-0d21-4db0-994c-5f971d67f103"}]: dispatch 2021-11-03T17:09:37.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:37 smithi115 conmon[34670]: audit 2021-11-03T17:09:37.394162+0000 mon.smithi027 (mon.0) 328 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c517d190-0d21-4db0-994c-5f971d67f103"}]: dispatch 2021-11-03T17:09:37.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:37 smithi115 conmon[34670]: audit 2021-11-03T17:09:37.403381+0000 mon.smithi027 (mon.0) 329 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c517d190-0d21-4db0-994c-5f971d67f103"}]': finished 2021-11-03T17:09:37.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:37 smithi115 conmon[34670]: cluster 2021-11-03T17:09:37.403465+0000 mon.smithi027 (mon.0) 330 : cluster [DBG] osdmap e21: 4 total, 3 up, 4 in 2021-11-03T17:09:37.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:37 smithi115 conmon[34670]: 2021-11-03T17:09:37.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:37 smithi115 conmon[34670]: audit 2021-11-03T17:09:37.403589+0000 mon.smithi027 (mon.0) 331 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-11-03T17:09:38.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:37 smithi027 conmon[32322]: cluster 2021-11-03T17:09:36.547491+0000 mgr.smithi027.ufhhvb (mgr.14162) 128 : cluster [DBG] pgmap v90: 0 pgs: ; 0 B data, 9.6 MiB used, 179 GiB / 179 GiB avail 2021-11-03T17:09:38.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:37 smithi027 conmon[32322]: cluster 2021-11-03T17:09:36.585484+0000 mon.smithi027 (mon.0) 324 : cluster [DBG] osdmap e20: 3 total, 3 up, 3 in 2021-11-03T17:09:38.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:37 smithi027 conmon[32322]: audit 2021-11-03T17:09:37.033541+0000 mon.smithi027 (mon.0) 325 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:38.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:37 smithi027 conmon[32322]: audit 2021-11-03T17:09:37.038709 2021-11-03T17:09:38.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:37 smithi027 conmon[32322]: +0000 mon.smithi027 (mon.0) 326 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:38.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:37 smithi027 conmon[32322]: audit 2021-11-03T17:09:38.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:37 smithi027 conmon[32322]: 2021-11-03T17:09:37.041410+0000 2021-11-03T17:09:38.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:37 smithi027 conmon[32322]: mon.smithi027 (mon.0) 327 : audit [DBG] 2021-11-03T17:09:38.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:37 smithi027 conmon[32322]: from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:09:38.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:37 smithi027 conmon[32322]: audit 2021-11-03T17:09:38.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:37 smithi027 conmon[32322]: 2021-11-03T17:09:37 2021-11-03T17:09:38.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:37 smithi027 conmon[32322]: .393793+0000 mon.smithi115 (mon.1) 2 : 2021-11-03T17:09:38.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:37 smithi027 conmon[32322]: audit [INF] from='client.? 172.21.15.115:0/3596975042' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c517d190-0d21-4db0-994c-5f971d67f103"}]: dispatch 2021-11-03T17:09:38.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:37 smithi027 conmon[32322]: audit 2021-11- 2021-11-03T17:09:38.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:37 smithi027 conmon[32322]: 03T17:09:37.394162+0000 mon.smithi027 2021-11-03T17:09:38.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:37 smithi027 conmon[32322]: (mon.0) 328 : audit [INF] 2021-11-03T17:09:38.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:37 smithi027 conmon[32322]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c517d190-0d21-4db0-994c-5f971d67f103"}]: dispatch 2021-11-03T17:09:38.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:37 smithi027 conmon[32322]: audit 2021-11-03T17:09:38.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:37 smithi027 conmon[32322]: 2021-11-03T17:09:37.403381+0000 2021-11-03T17:09:38.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:37 smithi027 conmon[32322]: mon.smithi027 (mon.0) 329 : audit [INF] 2021-11-03T17:09:38.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:37 smithi027 conmon[32322]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c517d190-0d21-4db0-994c-5f971d67f103"}]': finished 2021-11-03T17:09:38.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:37 smithi027 conmon[32322]: cluster 2021-11-03T17:09:38.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:37 smithi027 conmon[32322]: 2021-11-03T17:09:37. 2021-11-03T17:09:38.026 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:37 smithi027 conmon[32322]: 403465+0000 mon.smithi027 (mon.0) 330 : cluster 2021-11-03T17:09:38.026 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:37 smithi027 conmon[32322]: [DBG] osdmap e21: 4 total, 3 up, 4 in 2021-11-03T17:09:38.026 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:37 smithi027 conmon[32322]: audit 2021-11-03T17: 2021-11-03T17:09:38.027 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:37 smithi027 conmon[32322]: 09:37.403589+0000 mon.smithi027 (mon. 2021-11-03T17:09:38.027 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:37 smithi027 conmon[32322]: 0) 331 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-11-03T17:09:38.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:38 smithi115 conmon[34670]: cephadm 2021-11-03T17:09:37.027005+0000 mgr.smithi027.ufhhvb 2021-11-03T17:09:38.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:38 smithi115 conmon[34670]: (mgr.14162) 129 : cephadm [INF] Detected new or changed devices on smithi027 2021-11-03T17:09:38.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:38 smithi115 conmon[34670]: audit 2021-11-03T17:09:37.907470+0000 mon.smithi115 (mon.1) 3 : audit [DBG] from='client.? 172.21.15.115:0/819936150' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2021-11-03T17:09:39.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:38 smithi027 conmon[32322]: cephadm 2021-11-03T17:09:37.027005+0000 mgr.smithi027.ufhhvb (mgr. 2021-11-03T17:09:39.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:38 smithi027 conmon[32322]: 14162) 129 : cephadm [INF] Detected new or changed devices on smithi027 2021-11-03T17:09:39.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:38 smithi027 conmon[32322]: audit 2021-11-03T17:09:37.907470+0000 mon.smithi115 (mon.1) 3 : audit [DBG] from='client.? 172.21.15.115:0/819936150' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2021-11-03T17:09:39.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:39 smithi115 conmon[34670]: cluster 2021-11-03T17:09:38.547733+0000 mgr.smithi027.ufhhvb (mgr. 2021-11-03T17:09:39.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:39 smithi115 conmon[34670]: 14162) 130 : cluster [DBG] pgmap v93: 0 pgs: ; 0 B data, 14 MiB used, 268 GiB / 268 GiB avail 2021-11-03T17:09:40.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:39 smithi027 conmon[32322]: cluster 2021-11-03T17:09:38.547733+0000 mgr.smithi027.ufhhvb 2021-11-03T17:09:40.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:39 smithi027 conmon[32322]: (mgr.14162) 130 : cluster [DBG] pgmap v93: 0 pgs: ; 0 B data, 14 MiB used, 268 GiB / 268 GiB avail 2021-11-03T17:09:41.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:41 smithi115 conmon[34670]: cluster 2021-11-03T17:09:40.548185+0000 mgr.smithi027.ufhhvb (mgr.14162) 131 2021-11-03T17:09:41.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:41 smithi115 conmon[34670]: : cluster [DBG] pgmap v94: 0 pgs: ; 0 B data, 14 MiB used, 268 GiB / 268 GiB avail 2021-11-03T17:09:41.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:41 smithi115 conmon[34670]: audit 2021-11-03T17:09:40.847643+0000 mon.smithi027 (mon.0) 332 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:41.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:41 smithi115 conmon[34670]: audit 2021-11-03T17:09:40.850889+0000 mon.smithi027 (mon.0) 333 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:42.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:41 smithi027 conmon[32322]: cluster 2021-11-03T17:09:40.548185+0000 mgr.smithi027.ufhhvb (mgr.14162 2021-11-03T17:09:42.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:41 smithi027 conmon[32322]: ) 131 : cluster [DBG] pgmap v94: 0 pgs: ; 0 B data, 14 MiB used, 268 GiB / 268 GiB avail 2021-11-03T17:09:42.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:41 smithi027 conmon[32322]: audit 2021-11-03T17:09:40.847643+0000 mon.smithi027 (mon.0) 332 : audit 2021-11-03T17:09:42.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:41 smithi027 conmon[32322]: [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:42.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:41 smithi027 conmon[32322]: audit 2021-11-03T17: 2021-11-03T17:09:42.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:41 smithi027 conmon[32322]: 09:40.850889+0000 mon.smithi027 (mon.0 2021-11-03T17:09:42.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:41 smithi027 conmon[32322]: ) 333 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:42.937 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:42 smithi027 conmon[32322]: cluster 2021-11-03T17:09:42.548719+0000 mgr.smithi027.ufhhvb (mgr.14162) 132 : cluster [DBG] pgmap v95: 0 pgs: ; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2021-11-03T17:09:42.938 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:42 smithi027 conmon[32322]: 2021-11-03T17:09:42.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:42 smithi115 conmon[34670]: cluster 2021-11-03T17:09:42.548719+0000 mgr.smithi027.ufhhvb (mgr 2021-11-03T17:09:42.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:42 smithi115 conmon[34670]: .14162) 132 : cluster [DBG] pgmap v95: 0 pgs: ; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2021-11-03T17:09:44.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:44 smithi115 conmon[34670]: audit 2021-11-03T17:09:44.459679+0000 mon.smithi027 (mon.0) 334 2021-11-03T17:09:44.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:44 smithi115 conmon[34670]: : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2021-11-03T17:09:44.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:44 smithi115 conmon[34670]: audit 2021-11-03T17:09:44.460673+0000 mon.smithi027 (mon.0) 335 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:09:44.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:44 smithi027 conmon[32322]: audit 2021-11-03T17:09:44.459679+0000 mon.smithi027 (mon.0) 2021-11-03T17:09:44.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:44 smithi027 conmon[32322]: 334 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2021-11-03T17:09:44.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:44 smithi027 conmon[32322]: audit 2021-11-03T17:09:44.460673+0000 mon.smithi027 (mon.0) 335 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:09:45.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:45 smithi115 conmon[34670]: cephadm 2021-11-03T17:09:44.461440+0000 mgr.smithi027.ufhhvb (mgr. 2021-11-03T17:09:45.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:45 smithi115 conmon[34670]: 14162) 133 : cephadm [INF] Deploying daemon osd.3 on smithi115 2021-11-03T17:09:45.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:45 smithi115 conmon[34670]: cluster 2021-11-03T17:09:44.549100+0000 mgr.smithi027.ufhhvb (mgr.14162) 134 : cluster [DBG] pgmap v96: 0 pgs: ; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2021-11-03T17:09:45.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:45 smithi027 conmon[32322]: cephadm 2021-11-03T17:09:44.461440+0000 mgr.smithi027.ufhhvb (mgr.14162) 133 : cephadm 2021-11-03T17:09:45.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:45 smithi027 conmon[32322]: [INF] Deploying daemon osd.3 on smithi115 2021-11-03T17:09:45.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:45 smithi027 conmon[32322]: cluster 2021-11-03T17:09:44.549100+0000 mgr.smithi027.ufhhvb (mgr.14162) 134 : cluster [DBG] pgmap v96: 0 pgs: ; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2021-11-03T17:09:47.838 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:47 smithi115 conmon[34670]: cluster 2021-11-03T17:09:46.549409+0000 mgr.smithi027.ufhhvb (mgr.14162 2021-11-03T17:09:47.838 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:47 smithi115 conmon[34670]: ) 135 : cluster [DBG] pgmap v97: 0 pgs: ; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2021-11-03T17:09:48.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:47 smithi027 conmon[32322]: cluster 2021-11-03T17:09:46.549409+0000 mgr.smithi027.ufhhvb (mgr.14162) 2021-11-03T17:09:48.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:47 smithi027 conmon[32322]: 135 : cluster [DBG] pgmap v97: 0 pgs: ; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2021-11-03T17:09:48.023 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 3 on host 'smithi115' 2021-11-03T17:09:48.023 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:48.026+0000 7f857ffff700 1 -- 172.21.15.115:0/1295621900 <== mgr.14162 v2:172.21.15.27:6800/3802716887 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f85980fdf40 con 0x7f858005c2a0 2021-11-03T17:09:48.024 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:48.028+0000 7f859dcbb700 1 -- 172.21.15.115:0/1295621900 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f858005c2a0 msgr2=0x7f858005e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:09:48.024 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:48.028+0000 7f859dcbb700 1 --2- 172.21.15.115:0/1295621900 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f858005c2a0 0x7f858005e760 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f858c004e30 tx=0x7f858c009b30).stop 2021-11-03T17:09:48.025 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:48.028+0000 7f859dcbb700 1 -- 172.21.15.115:0/1295621900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85980f55e0 msgr2=0x7f8598109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:09:48.025 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:48.028+0000 7f859dcbb700 1 --2- 172.21.15.115:0/1295621900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85980f55e0 0x7f8598109c70 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7f85880026d0 tx=0x7f8588009de0).stop 2021-11-03T17:09:48.025 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:48.028+0000 7f859dcbb700 1 -- 172.21.15.115:0/1295621900 shutdown_connections 2021-11-03T17:09:48.025 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:48.028+0000 7f859dcbb700 1 --2- 172.21.15.115:0/1295621900 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f858005c2a0 0x7f858005e760 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:09:48.026 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:48.028+0000 7f859dcbb700 1 --2- 172.21.15.115:0/1295621900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85980f55e0 0x7f8598109c70 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:09:48.026 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:48.028+0000 7f859dcbb700 1 --2- 172.21.15.115:0/1295621900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85980f5ef0 0x7f859810c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:09:48.026 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:48.028+0000 7f859dcbb700 1 -- 172.21.15.115:0/1295621900 >> 172.21.15.115:0/1295621900 conn(0x7f85980f0de0 msgr2=0x7f85980fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:09:48.026 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:48.028+0000 7f859dcbb700 1 -- 172.21.15.115:0/1295621900 shutdown_connections 2021-11-03T17:09:48.027 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:48.028+0000 7f859dcbb700 1 -- 172.21.15.115:0/1295621900 wait complete. 2021-11-03T17:09:48.670 DEBUG:teuthology.orchestra.run.smithi115:osd.3> sudo journalctl -f -n 0 -u ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c@osd.3.service 2021-11-03T17:09:48.672 INFO:tasks.cephadm:Deploying osd.4 on smithi115 with /dev/vg_nvme/lv_3... 2021-11-03T17:09:48.673 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2021-11-03T17:09:48.700 INFO:journalctl@ceph.osd.3.smithi115.stdout:-- Logs begin at Wed 2021-11-03 16:55:13 UTC. -- 2021-11-03T17:09:48.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:48 smithi115 conmon[34670]: audit 2021-11-03T17:09:48.017042+0000 mon.smithi027 ( 2021-11-03T17:09:48.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:48 smithi115 conmon[34670]: mon.0) 336 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:09:48.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:48 smithi115 conmon[34670]: audit 2021-11-03T17:09:48.021353+0000 mon.smithi027 (mon 2021-11-03T17:09:48.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:48 smithi115 conmon[34670]: .0) 337 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:48.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:48 smithi115 conmon[34670]: audit 2021 2021-11-03T17:09:48.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:48 smithi115 conmon[34670]: -11-03T17:09:48.026581+0000 mon.smithi027 (mon.0) 338 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:48.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:48 smithi115 conmon[34670]: 2021-11-03T17:09:48.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:48 smithi115 conmon[34670]: audit 2021 2021-11-03T17:09:48.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:48 smithi115 conmon[34670]: -11-03T17:09:48.028985+0000 mon.smithi027 (mon.0) 339 : 2021-11-03T17:09:48.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:48 smithi115 conmon[34670]: audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1}]: dispatch 2021-11-03T17:09:49.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:48 smithi027 conmon[32322]: audit 2021-11-03T17:09:48.017042+0000 mon.smithi027 ( 2021-11-03T17:09:49.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:48 smithi027 conmon[32322]: mon.0) 336 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:09:49.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:48 smithi027 conmon[32322]: audit 2021-11-03T17:09:48.021353+0000 mon.smithi027 (mon.0) 337 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:49.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:48 smithi027 conmon[32322]: audit 2021-11-03T17:09:48.026581+0000 mon.smithi027 (mon.0) 338 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:49.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:48 smithi027 conmon[32322]: audit 2021-11-03T17:09:48.028985+0000 mon.smithi027 (mon.0) 2021-11-03T17:09:49.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:48 smithi027 conmon[32322]: 339 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1}]: dispatch 2021-11-03T17:09:49.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:49 smithi115 conmon[34670]: cluster 2021-11-03T17:09:48.549815+0000 mgr.smithi027.ufhhvb (mgr.14162) 136 : cluster [DBG] pgmap v98: 0 pgs: ; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2021-11-03T17:09:49.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:49 smithi115 conmon[34670]: audit 2021-11-03T17:09:49.033179+0000 mon.smithi027 (mon.0) 340 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1}]': finished 2021-11-03T17:09:49.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:49 smithi115 conmon[34670]: cluster 2021-11-03T17:09:49.033243+0000 mon.smithi027 (mon.0) 341 : cluster [DBG] osdmap e22: 4 total, 3 up, 4 in 2021-11-03T17:09:49.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:49 smithi115 conmon[34670]: audit 2021-11-03T17:09:49.033376+0000 mon.smithi027 (mon.0) 342 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-11-03T17:09:49.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:49 smithi115 conmon[34670]: audit 2021-11-03T17:09:49.034257+0000 mon.smithi027 (mon.0) 343 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2021-11-03T17:09:50.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:49 smithi027 conmon[32322]: cluster 2021-11-03T17:09:48. 2021-11-03T17:09:50.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:49 smithi027 conmon[32322]: 549815+0000 mgr.smithi027.ufhhvb (mgr.14162) 136 : cluster [DBG] pgmap v98: 0 pgs: ; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2021-11-03T17:09:50.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:49 smithi027 conmon[32322]: audit 2021-11-03T17:09:49.033179+0000 mon.smithi027 (mon.0) 340 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "osd pool create", "format": "json", "pool": "device_health_metrics", "pg_num": 1, "pg_num_min": 1}]': finished 2021-11-03T17:09:50.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:49 smithi027 conmon[32322]: cluster 2021-11-03T17:09:49.033243+0000 mon.smithi027 (mon.0) 341 : cluster [DBG] osdmap e22: 4 total, 3 up, 4 in 2021-11-03T17:09:50.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:49 smithi027 conmon[32322]: audit 2021-11-03T17:09:49.033376+0000 mon.smithi027 2021-11-03T17:09:50.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:49 smithi027 conmon[32322]: (mon.0) 342 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-11-03T17:09:50.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:49 smithi027 conmon[32322]: audit 2021-11-03T17:09:49.034257+0000 mon.smithi027 (mon 2021-11-03T17:09:50.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:49 smithi027 conmon[32322]: .0) 343 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]: dispatch 2021-11-03T17:09:50.940 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping: /dev/vg_nvme/lv_3 2021-11-03T17:09:50.941 INFO:teuthology.orchestra.run.smithi115.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2021-11-03T17:09:50.941 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10+0 records in 2021-11-03T17:09:50.941 INFO:teuthology.orchestra.run.smithi115.stderr:10+0 records out 2021-11-03T17:09:50.941 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0125332 s, 837 MB/s 2021-11-03T17:09:50.942 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping successful for: 2021-11-03T17:09:51.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:51 smithi115 conmon[34670]: audit 2021-11-03T17:09:50.034784+0000 mon.smithi027 (mon.0) 344 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2021-11-03T17:09:51.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:51 smithi115 conmon[34670]: cluster 2021-11-03T17:09:50.034858+0000 mon.smithi027 (mon.0) 345 : cluster [DBG] osdmap e23: 4 total, 3 up, 4 in 2021-11-03T17:09:51.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:51 smithi115 conmon[34670]: audit 2021-11-03T17:09:50.035036+0000 mon.smithi027 (mon.0) 346 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-11-03T17:09:51.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:51 smithi115 conmon[34670]: cluster 2021-11-03T17:09:50.550152+0000 mgr.smithi027.ufhhvb (mgr.14162) 137 : cluster [DBG] pgmap v101: 1 pgs: 1 unknown; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2021-11-03T17:09:51.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:51 smithi115 conmon[34670]: audit 2021-11-03T17:09:50.997357+0000 mon.smithi027 (mon.0) 347 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:51.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:51 smithi027 conmon[32322]: audit 2021-11-03T17:09:50.034784+0000 mon.smithi027 (mon.0) 344 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": "device_health_metrics", "app": "mgr_devicehealth"}]': finished 2021-11-03T17:09:51.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:51 smithi027 conmon[32322]: cluster 2021-11-03T17:09:50.034858+0000 mon.smithi027 (mon.0) 345 : cluster [DBG] osdmap e23: 4 total, 3 up, 4 in 2021-11-03T17:09:51.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:51 smithi027 conmon[32322]: audit 2021-11-03T17:09:50.035036+0000 mon.smithi027 (mon.0) 346 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-11-03T17:09:51.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:51 smithi027 conmon[32322]: cluster 2021-11-03T17:09:50.550152+0000 mgr.smithi027.ufhhvb (mgr.14162) 137 : cluster [DBG] pgmap v101: 1 pgs: 1 unknown; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2021-11-03T17:09:51.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:51 smithi027 conmon[32322]: audit 2021-11-03T17:09:50.997357+0000 mon.smithi027 (mon.0) 347 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:51.611 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph orch daemon add osd smithi115:vg_nvme/lv_3 2021-11-03T17:09:52.123 INFO:journalctl@ceph.osd.3.smithi115.stdout:Nov 03 17:09:52 smithi115 conmon[41051]: debug 2021-11-03T17:09:52.014+0000 7f552fe2a080 -1 osd.3 0 log_to_monitors {default=true} 2021-11-03T17:09:52.124 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:52 smithi115 conmon[34670]: cluster 2021-11-03T17:09:51.040544+0000 mon.smithi027 (mon. 2021-11-03T17:09:52.124 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:52 smithi115 conmon[34670]: 0) 348 : cluster [DBG] osdmap e24: 4 total, 3 up, 4 in 2021-11-03T17:09:52.124 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:52 smithi115 conmon[34670]: audit 2021-11-03T17:09:51.040810+0000 mon.smithi027 (mon. 2021-11-03T17:09:52.124 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:52 smithi115 conmon[34670]: 0) 349 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-11-03T17:09:52.125 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:52 smithi115 conmon[34670]: audit 2021-11-03T 2021-11-03T17:09:52.125 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:52 smithi115 conmon[34670]: 17:09:51.868790+0000 mon.smithi027 (mon.0) 350 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:52.125 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:52 smithi115 conmon[34670]: audit 2021-11-03T17 2021-11-03T17:09:52.125 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:52 smithi115 conmon[34670]: :09:52.018553+0000 mon.smithi115 (mon.1) 4 : audit 2021-11-03T17:09:52.126 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:52 smithi115 conmon[34670]: [INF] from='osd.3 [v2:172.21.15.115:6800/4026713633,v1:172.21.15.115:6801/4026713633]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2021-11-03T17:09:52.126 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:52 smithi115 conmon[34670]: audit 2021-11-03T17:09:52.126 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:52 smithi115 conmon[34670]: 2021-11-03T17:09:52.019030+0000 mon.smithi027 (mon. 2021-11-03T17:09:52.126 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:52 smithi115 conmon[34670]: 0) 351 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2021-11-03T17:09:52.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:52 smithi027 conmon[32322]: cluster 2021-11-03T17:09:51.040544+0000 mon.smithi027 2021-11-03T17:09:52.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:52 smithi027 conmon[32322]: (mon.0) 348 : cluster [DBG] osdmap e24: 4 total, 3 up, 4 in 2021-11-03T17:09:52.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:52 smithi027 conmon[32322]: audit 2021-11-03T17:09:51.040810+0000 mon.smithi027 (mon.0) 349 : audit 2021-11-03T17:09:52.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:52 smithi027 conmon[32322]: [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-11-03T17:09:52.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:52 smithi027 conmon[32322]: audit 2021-11-03T17:09:52.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:52 smithi027 conmon[32322]: 2021-11-03T17:09:51.868790+0000 2021-11-03T17:09:52.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:52 smithi027 conmon[32322]: mon.smithi027 (mon.0) 350 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:52.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:52 smithi027 conmon[32322]: audit 2021-11- 2021-11-03T17:09:52.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:52 smithi027 conmon[32322]: 03T17:09:52.018553+0000 mon.smithi115 (mon.1) 2021-11-03T17:09:52.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:52 smithi027 conmon[32322]: 4 : audit [INF] from='osd.3 [v2:172.21.15.115:6800/4026713633,v1:172.21.15.115:6801/4026713633]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2021-11-03T17:09:52.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:52 smithi027 conmon[32322]: audit 2021-11-03T17:09:52.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:52 smithi027 conmon[32322]: 2021-11-03T17: 2021-11-03T17:09:52.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:52 smithi027 conmon[32322]: 09:52.019030+0000 mon.smithi027 (mon.0) 2021-11-03T17:09:52.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:52 smithi027 conmon[32322]: 351 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2021-11-03T17:09:53.164 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:53 smithi115 conmon[34670]: cluster 2021-11-03T17:09:52.550654+0000 mgr.smithi027.ufhhvb (mgr.14162) 138 : cluster 2021-11-03T17:09:53.164 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:53 smithi115 conmon[34670]: [DBG] pgmap v103: 1 pgs: 1 creating+activating; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2021-11-03T17:09:53.164 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:53 smithi115 conmon[34670]: audit 2021-11-03T17:09:52.720120+0000 mon.smithi027 (mon.0) 352 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:09:53.165 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:53 smithi115 conmon[34670]: audit 2021-11-03T17:09:52.720446+0000 mon.smithi027 (mon.0) 353 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:09:53.165 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:53 smithi115 conmon[34670]: audit 2021-11-03T17:09:52.871540+0000 mon.smithi027 (mon.0) 354 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2021-11-03T17:09:53.165 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:53 smithi115 conmon[34670]: cluster 2021-11-03T17:09:52.871612+0000 mon.smithi027 (mon.0) 355 : cluster [DBG] osdmap e25: 4 total, 3 up, 4 in 2021-11-03T17:09:53.165 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:53 smithi115 conmon[34670]: audit 2021-11-03T17:09:52.871758+0000 mon.smithi027 (mon.0) 356 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-11-03T17:09:53.166 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:53 smithi115 conmon[34670]: audit 2021-11-03T17:09:52.872241+0000 mon.smithi115 (mon.1) 5 : audit [INF] from='osd.3 [v2:172.21.15.115:6800/4026713633,v1:172.21.15.115:6801/4026713633]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2021-11-03T17:09:53.166 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:53 smithi115 conmon[34670]: audit 2021-11-03T17:09:52.872668+0000 mon.smithi027 (mon.0) 357 : audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2021-11-03T17:09:53.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:53 smithi027 conmon[32322]: cluster 2021-11-03T17:09:52.550654+0000 mgr.smithi027.ufhhvb (mgr. 2021-11-03T17:09:53.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:53 smithi027 conmon[32322]: 14162) 138 : cluster [DBG] pgmap v103: 1 pgs: 1 creating+activating; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2021-11-03T17:09:53.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:53 smithi027 conmon[32322]: audit 2021-11-03T17:09:52.720120+0000 mon.smithi027 (mon.0) 352 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:09:53.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:53 smithi027 conmon[32322]: audit 2021-11-03T17:09:52. 2021-11-03T17:09:53.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:53 smithi027 conmon[32322]: 720446+0000 mon.smithi027 (mon.0) 353 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:09:53.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:53 smithi027 conmon[32322]: audit 2021-11-03T 2021-11-03T17:09:53.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:53 smithi027 conmon[32322]: 17:09:52.871540+0000 mon.smithi027 (mon.0) 354 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2021-11-03T17:09:53.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:53 smithi027 conmon[32322]: cluster 2021-11- 2021-11-03T17:09:53.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:53 smithi027 conmon[32322]: 03T17:09:52.871612+0000 mon.smithi027 (mon.0) 355 : cluster [DBG] osdmap e25: 4 total, 3 up, 4 in 2021-11-03T17:09:53.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:53 smithi027 conmon[32322]: 2021-11-03T17:09:53.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:53 smithi027 conmon[32322]: audit 2021-11-03T17:09:52.871758+0000 mon.smithi027 (mon.0 2021-11-03T17:09:53.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:53 smithi027 conmon[32322]: ) 356 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-11-03T17:09:53.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:53 smithi027 conmon[32322]: audit 2021-11-03T17:09:53.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:53 smithi027 conmon[32322]: 2021-11-03T17: 2021-11-03T17:09:53.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:53 smithi027 conmon[32322]: 09:52.872241+0000 mon.smithi115 (mon.1) 5 : 2021-11-03T17:09:53.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:53 smithi027 conmon[32322]: audit [INF] from='osd.3 [v2:172.21.15.115:6800/4026713633,v1:172.21.15.115:6801/4026713633]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2021-11-03T17:09:53.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:53 smithi027 conmon[32322]: audit 2021-11-03T17:09:53.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:53 smithi027 conmon[32322]: 2021-11-03T17: 2021-11-03T17:09:53.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:53 smithi027 conmon[32322]: 09:52.872668+0000 mon.smithi027 (mon.0) 357 : 2021-11-03T17:09:53.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:53 smithi027 conmon[32322]: audit [INF] from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2021-11-03T17:09:54.082 INFO:journalctl@ceph.osd.3.smithi115.stdout:Nov 03 17:09:53 smithi115 conmon[41051]: debug 2021-11-03T17:09:53.874+0000 7f552639b700 -1 osd.3 0 waiting for initial osdmap 2021-11-03T17:09:54.285 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:54.289+0000 7fa2f8e2b700 1 -- 172.21.15.115:0/1139304635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2f40f66d0 msgr2=0x7fa2f40f6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:09:54.285 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:54.289+0000 7fa2f8e2b700 1 --2- 172.21.15.115:0/1139304635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2f40f66d0 0x7fa2f40f6b30 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fa2e80046e0 tx=0x7fa2e8009b30).stop 2021-11-03T17:09:54.285 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:54.289+0000 7fa2f8e2b700 1 -- 172.21.15.115:0/1139304635 shutdown_connections 2021-11-03T17:09:54.286 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:54.289+0000 7fa2f8e2b700 1 --2- 172.21.15.115:0/1139304635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2f40f66d0 0x7fa2f40f6b30 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:09:54.286 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:54.289+0000 7fa2f8e2b700 1 --2- 172.21.15.115:0/1139304635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2f40f5430 0x7fa2f40f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:09:54.286 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:54.289+0000 7fa2f8e2b700 1 -- 172.21.15.115:0/1139304635 >> 172.21.15.115:0/1139304635 conn(0x7fa2f40f0b90 msgr2=0x7fa2f40f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:09:54.286 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:54.289+0000 7fa2f8e2b700 1 -- 172.21.15.115:0/1139304635 shutdown_connections 2021-11-03T17:09:54.287 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:54.289+0000 7fa2f8e2b700 1 -- 172.21.15.115:0/1139304635 wait complete. 2021-11-03T17:09:54.290 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:54.289+0000 7fa2f8e2b700 1 Processor -- start 2021-11-03T17:09:54.290 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:54.290+0000 7fa2f8e2b700 1 -- start start 2021-11-03T17:09:54.290 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:54.290+0000 7fa2f8e2b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2f40f5430 0x7fa2f4104b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:09:54.290 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:54.290+0000 7fa2f8e2b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2f40f66d0 0x7fa2f4105040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:09:54.291 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:54.290+0000 7fa2f8e2b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f41055f0 con 0x7fa2f40f66d0 2021-11-03T17:09:54.291 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:54.290+0000 7fa2f8e2b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f4105730 con 0x7fa2f40f5430 2021-11-03T17:09:54.291 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:54.290+0000 7fa2f37fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2f40f5430 0x7fa2f4104b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:09:54.291 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:54.290+0000 7fa2f37fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2f40f5430 0x7fa2f4104b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38624/0 (socket says 172.21.15.115:38624) 2021-11-03T17:09:54.292 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:54.290+0000 7fa2f37fe700 1 -- 172.21.15.115:0/3212230075 learned_addr learned my addr 172.21.15.115:0/3212230075 (peer_addr_for_me v2:172.21.15.115:0/0) 2021-11-03T17:09:54.292 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:54.290+0000 7fa2f2ffd700 1 --2- 172.21.15.115:0/3212230075 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2f40f66d0 0x7fa2f4105040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:09:54.292 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:54.290+0000 7fa2f37fe700 1 -- 172.21.15.115:0/3212230075 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2f40f66d0 msgr2=0x7fa2f4105040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:09:54.292 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:54.290+0000 7fa2f37fe700 1 --2- 172.21.15.115:0/3212230075 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2f40f66d0 0x7fa2f4105040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:09:54.293 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:54.290+0000 7fa2f37fe700 1 -- 172.21.15.115:0/3212230075 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e8005040 con 0x7fa2f40f5430 2021-11-03T17:09:54.293 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:54.290+0000 7fa2f37fe700 1 --2- 172.21.15.115:0/3212230075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2f40f5430 0x7fa2f4104b50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fa2e400d7f0 tx=0x7fa2e4004e80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:09:54.293 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:54.290+0000 7fa2dbfff700 1 -- 172.21.15.115:0/3212230075 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2e400b4f0 con 0x7fa2f40f5430 2021-11-03T17:09:54.293 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:54.290+0000 7fa2dbfff700 1 -- 172.21.15.115:0/3212230075 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7fa2e400f040 con 0x7fa2f40f5430 2021-11-03T17:09:54.293 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:54.290+0000 7fa2f8e2b700 1 -- 172.21.15.115:0/3212230075 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2f41060d0 con 0x7fa2f40f5430 2021-11-03T17:09:54.294 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:54.290+0000 7fa2dbfff700 1 -- 172.21.15.115:0/3212230075 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2e4013940 con 0x7fa2f40f5430 2021-11-03T17:09:54.294 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:54.290+0000 7fa2f8e2b700 1 -- 172.21.15.115:0/3212230075 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2f4106690 con 0x7fa2f40f5430 2021-11-03T17:09:54.294 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:54.291+0000 7fa2f8e2b700 1 -- 172.21.15.115:0/3212230075 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2f4045ad0 con 0x7fa2f40f5430 2021-11-03T17:09:54.295 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:54.292+0000 7fa2dbfff700 1 -- 172.21.15.115:0/3212230075 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fa2e401a030 con 0x7fa2f40f5430 2021-11-03T17:09:54.295 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:54.292+0000 7fa2dbfff700 1 --2- 172.21.15.115:0/3212230075 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fa2dc05c300 0x7fa2dc05e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:09:54.295 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:54.292+0000 7fa2dbfff700 1 -- 172.21.15.115:0/3212230075 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3375+0+0 (secure 0 0 0) 0x7fa2e4075cf0 con 0x7fa2f40f5430 2021-11-03T17:09:54.296 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:54.293+0000 7fa2f2ffd700 1 --2- 172.21.15.115:0/3212230075 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fa2dc05c300 0x7fa2dc05e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:09:54.296 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:54.293+0000 7fa2f2ffd700 1 --2- 172.21.15.115:0/3212230075 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fa2dc05c300 0x7fa2dc05e7c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fa2e8000c00 tx=0x7fa2e8009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:09:54.297 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:54.296+0000 7fa2dbfff700 1 -- 172.21.15.115:0/3212230075 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fa2e4049190 con 0x7fa2f40f5430 2021-11-03T17:09:54.478 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:09:54.482+0000 7fa2f8e2b700 1 -- 172.21.15.115:0/3212230075 --> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2f4106910 con 0x7fa2dc05c300 2021-11-03T17:09:55.163 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:54 smithi115 conmon[34670]: audit 2021-11-03T17:09:53.873192+0000 mon.smithi027 ( 2021-11-03T17:09:55.163 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:54 smithi115 conmon[34670]: mon.0) 358 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2021-11-03T17:09:55.163 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:54 smithi115 conmon[34670]: cluster 2021-11-03T17:09:53.873313+0000 mon.smithi027 (mon.0) 359 : cluster [DBG] osdmap e26: 4 total, 3 up, 4 in 2021-11-03T17:09:55.164 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:54 smithi115 conmon[34670]: audit 2021-11-03T17:09:53.873526+0000 mon.smithi027 (mon.0) 360 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-11-03T17:09:55.164 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:54 smithi115 conmon[34670]: audit 2021-11-03T17:09:53.876680+0000 mon.smithi027 (mon.0) 361 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-11-03T17:09:55.164 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:54 smithi115 conmon[34670]: audit 2021-11-03T17:09:54.485443+0000 mgr.smithi027.ufhhvb (mgr.14162) 139 : audit [DBG] from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:09:55.164 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:54 smithi115 conmon[34670]: audit 2021-11-03T17:09:54.486389+0000 mon.smithi027 (mon.0) 362 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2021-11-03T17:09:55.165 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:54 smithi115 conmon[34670]: cephadm 2021-11-03T17:09:54.488087+0000 mgr.smithi027.ufhhvb (mgr.14162) 140 : cephadm [INF] Applying service osd.None on host smithi115... 2021-11-03T17:09:55.165 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:54 smithi115 conmon[34670]: audit 2021-11-03T17:09:54.488392+0000 mon.smithi027 (mon.0) 363 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2021-11-03T17:09:55.165 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:54 smithi115 conmon[34670]: audit 2021-11-03T17:09:54.489278+0000 mon.smithi027 (mon.0) 364 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:09:55.165 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:54 smithi115 conmon[34670]: cluster 2021-11-03T17:09:54.550974+0000 mgr.smithi027.ufhhvb (mgr.14162) 141 : cluster [DBG] pgmap v106: 1 pgs: 1 creating+activating; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2021-11-03T17:09:55.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:54 smithi027 conmon[32322]: audit 2021-11-03T17:09:53. 2021-11-03T17:09:55.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:54 smithi027 conmon[32322]: 873192+0000 mon.smithi027 (mon.0) 358 : audit [INF] from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2021-11-03T17:09:55.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:54 smithi027 conmon[32322]: cluster 2021-11-03T17:09:55.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:54 smithi027 conmon[32322]: 2021-11-03T17:09:53.873313+0000 mon.smithi027 (mon.0) 359 : cluster 2021-11-03T17:09:55.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:54 smithi027 conmon[32322]: [DBG] osdmap e26: 4 total, 3 up, 4 in 2021-11-03T17:09:55.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:54 smithi027 conmon[32322]: audit 2021-11-03T17: 2021-11-03T17:09:55.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:54 smithi027 conmon[32322]: 09:53.873526+0000 mon.smithi027 (mon.0 2021-11-03T17:09:55.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:54 smithi027 conmon[32322]: ) 360 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-11-03T17:09:55.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:54 smithi027 conmon[32322]: audit 2021-11-03T17:09:55.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:54 smithi027 conmon[32322]: 2021-11-03T17:09:53 2021-11-03T17:09:55.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:54 smithi027 conmon[32322]: .876680+0000 mon.smithi027 (mon.0) 361 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-11-03T17:09:55.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:54 smithi027 conmon[32322]: 2021-11-03T17:09:55.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:54 smithi027 conmon[32322]: audit 2021-11-03T17:09:55.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:54 smithi027 conmon[32322]: 2021-11-03T17:09:54.485443+0000 mgr.smithi027.ufhhvb (mgr.14162) 139 : audit 2021-11-03T17:09:55.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:54 smithi027 conmon[32322]: [DBG] from='client.24151 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:09:55.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:54 smithi027 conmon[32322]: audit 2021-11 2021-11-03T17:09:55.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:54 smithi027 conmon[32322]: -03T17:09:54.486389+0000 mon.smithi027 (mon.0) 362 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2021-11-03T17:09:55.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:54 smithi027 conmon[32322]: cephadm 2021-11-03T17:09:54.488087+0000 2021-11-03T17:09:55.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:54 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14162) 140 : cephadm 2021-11-03T17:09:55.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:54 smithi027 conmon[32322]: [INF] Applying service osd.None on host smithi115... 2021-11-03T17:09:55.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:54 smithi027 conmon[32322]: 2021-11-03T17:09:55.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:54 smithi027 conmon[32322]: audit 2021-11 2021-11-03T17:09:55.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:54 smithi027 conmon[32322]: -03T17:09:54.488392+0000 2021-11-03T17:09:55.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:54 smithi027 conmon[32322]: mon.smithi027 (mon.0) 363 : audit 2021-11-03T17:09:55.275 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:54 smithi027 conmon[32322]: [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2021-11-03T17:09:55.275 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:54 smithi027 conmon[32322]: audit 2021-11-03T17:09:55.275 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:54 smithi027 conmon[32322]: 2021-11-03T17:09:54.489278+0000 2021-11-03T17:09:55.275 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:54 smithi027 conmon[32322]: mon.smithi027 (mon.0) 364 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:09:55.275 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:54 smithi027 conmon[32322]: cluster 2021-11 2021-11-03T17:09:55.276 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:54 smithi027 conmon[32322]: -03T17:09:54.550974+0000 mgr.smithi027.ufhhvb (mgr.14162) 141 : cluster [DBG] pgmap v106: 1 pgs: 1 creating+activating; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2021-11-03T17:09:56.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:55 smithi115 conmon[34670]: cluster 2021-11-03T17:09:53.016767+0000 osd.3 (osd.3) 1 : cluster [DBG] purged_snaps scrub starts 2021-11-03T17:09:56.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:55 smithi115 conmon[34670]: cluster 2021-11-03T17:09:53.016881+0000 osd.3 (osd.3) 2 : cluster [DBG] purged_snaps scrub ok 2021-11-03T17:09:56.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:55 smithi115 conmon[34670]: cluster 2021-11-03T17:09:54.880131+0000 mon.smithi027 (mon.0) 365 : cluster [INF] osd.3 [v2:172.21.15.115:6800/4026713633,v1:172.21.15.115:6801/4026713633] boot 2021-11-03T17:09:56.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:55 smithi115 conmon[34670]: cluster 2021-11-03T17:09:54.880216+0000 mon.smithi027 (mon.0) 366 : cluster [DBG] osdmap e27: 4 total, 4 up, 4 in 2021-11-03T17:09:56.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:55 smithi115 conmon[34670]: audit 2021-11-03T17:09:54.880669+0000 mon.smithi027 (mon.0) 367 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-11-03T17:09:56.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:55 smithi115 conmon[34670]: cephadm 2021-11-03T17:09:55.210262+0000 mgr.smithi027.ufhhvb (mgr.14162) 142 : cephadm [INF] Detected new or changed devices on smithi115 2021-11-03T17:09:56.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:55 smithi115 conmon[34670]: audit 2021-11-03T17:09:55.215843+0000 mon.smithi027 (mon.0) 368 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:56.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:55 smithi115 conmon[34670]: audit 2021-11-03T17:09:55.219925+0000 mon.smithi027 (mon.0) 369 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:56.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:55 smithi115 conmon[34670]: audit 2021-11-03T17:09:55.223588+0000 mon.smithi027 (mon.0) 370 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:09:56.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:55 smithi027 conmon[32322]: cluster 2021-11-03T17:09:53.016767+0000 osd.3 ( 2021-11-03T17:09:56.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:55 smithi027 conmon[32322]: osd.3) 1 : cluster [DBG] purged_snaps scrub starts 2021-11-03T17:09:56.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:55 smithi027 conmon[32322]: cluster 2021-11-03T17:09:53.016881+0000 osd.3 (osd.3) 2 : cluster [DBG] purged_snaps scrub ok 2021-11-03T17:09:56.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:55 smithi027 conmon[32322]: cluster 2021-11-03T17:09:54.880131+0000 mon.smithi027 (mon.0) 365 : cluster [INF] osd.3 [v2:172.21.15.115:6800/4026713633,v1:172.21.15.115:6801/4026713633] boot 2021-11-03T17:09:56.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:55 smithi027 conmon[32322]: cluster 2021-11-03T17:09:54.880216+0000 mon.smithi027 (mon.0) 366 : 2021-11-03T17:09:56.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:55 smithi027 conmon[32322]: cluster [DBG] osdmap e27: 4 total, 4 up, 4 in 2021-11-03T17:09:56.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:55 smithi027 conmon[32322]: audit 2021-11-03T17:09:54.880669 2021-11-03T17:09:56.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:55 smithi027 conmon[32322]: +0000 mon.smithi027 (mon.0) 367 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-11-03T17:09:56.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:55 smithi027 conmon[32322]: cephadm 2021-11 2021-11-03T17:09:56.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:55 smithi027 conmon[32322]: -03T17:09:55.210262+0000 mgr.smithi027.ufhhvb ( 2021-11-03T17:09:56.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:55 smithi027 conmon[32322]: mgr.14162) 142 : cephadm [INF] Detected new or changed devices on smithi115 2021-11-03T17:09:56.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:55 smithi027 conmon[32322]: 2021-11-03T17:09:56.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:55 smithi027 conmon[32322]: audit 2021 2021-11-03T17:09:56.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:55 smithi027 conmon[32322]: -11-03T17:09:55.215843+0000 mon.smithi027 2021-11-03T17:09:56.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:55 smithi027 conmon[32322]: (mon.0) 368 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:56.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:55 smithi027 conmon[32322]: 2021-11-03T17:09:56.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:55 smithi027 conmon[32322]: audit 2021-11-03T17:09:56.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:55 smithi027 conmon[32322]: 2021-11-03T17:09:55.219925+0000 2021-11-03T17:09:56.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:55 smithi027 conmon[32322]: mon.smithi027 (mon.0) 369 : audit [INF] 2021-11-03T17:09:56.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:55 smithi027 conmon[32322]: from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:09:56.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:55 smithi027 conmon[32322]: audit 2021-11-03T17:09:56.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:55 smithi027 conmon[32322]: 2021-11-03T17:09:55.223588 2021-11-03T17:09:56.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:55 smithi027 conmon[32322]: +0000 mon.smithi027 (mon.0) 370 : audit 2021-11-03T17:09:56.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:55 smithi027 conmon[32322]: [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:09:57.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:56 smithi115 conmon[34670]: cluster 2021-11-03T17:09:55.881016+0000 mon.smithi027 (mon.0) 371 : cluster [DBG] osdmap e28: 4 total, 4 up, 4 in 2021-11-03T17:09:57.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:56 smithi115 conmon[34670]: cluster 2021-11-03T17:09:56.551294+0000 mgr.smithi027.ufhhvb (mgr.14162) 143 : cluster [DBG] pgmap v109: 1 pgs: 1 remapped+peering; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2021-11-03T17:09:57.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:56 smithi115 conmon[34670]: cluster 2021-11-03T17:09:56.879617+0000 mon.smithi027 (mon.0) 372 : cluster [DBG] osdmap e29: 4 total, 4 up, 4 in 2021-11-03T17:09:57.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:56 smithi027 conmon[32322]: cluster 2021-11-03T17:09 2021-11-03T17:09:57.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:56 smithi027 conmon[32322]: :55.881016+0000 mon.smithi027 (mon.0) 371 : cluster [DBG] osdmap e28: 4 total, 4 up, 4 in 2021-11-03T17:09:57.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:56 smithi027 conmon[32322]: cluster 2021-11-03T17:09:56.551294+0000 mgr.smithi027.ufhhvb (mgr.14162) 143 : cluster [DBG] pgmap v109: 1 pgs: 1 remapped+peering; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2021-11-03T17:09:57.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:56 smithi027 conmon[32322]: cluster 2021-11-03T17:09:56.879617+0000 mon.smithi027 (mon.0) 372 : cluster [DBG] osdmap e29: 4 total, 4 up, 4 in 2021-11-03T17:09:58.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:57 smithi115 conmon[34670]: audit 2021-11-03T17:09:57.032437+0000 mon.smithi115 (mon.1) 6 : audit [INF] from='client.? 172.21.15.115:0/448728764' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aeeb2c74-919d-4ac7-8c7a-1dd273225f59"}]: dispatch 2021-11-03T17:09:58.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:57 smithi115 conmon[34670]: audit 2021-11-03T17:09:57.033028+0000 mon.smithi027 (mon.0) 373 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aeeb2c74-919d-4ac7-8c7a-1dd273225f59"}]: dispatch 2021-11-03T17:09:58.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:57 smithi115 conmon[34670]: audit 2021-11-03T17:09:57.036919+0000 mon.smithi027 (mon.0) 374 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aeeb2c74-919d-4ac7-8c7a-1dd273225f59"}]': finished 2021-11-03T17:09:58.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:57 smithi115 conmon[34670]: cluster 2021-11-03T17:09:57.036991+0000 mon.smithi027 (mon.0) 375 : cluster [DBG] osdmap e30: 5 total, 4 up, 5 in 2021-11-03T17:09:58.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:57 smithi115 conmon[34670]: audit 2021-11-03T17:09:57.037109+0000 mon.smithi027 (mon.0) 376 : 2021-11-03T17:09:58.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:57 smithi115 conmon[34670]: audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-11-03T17:09:58.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:57 smithi115 conmon[34670]: audit 2021-11-03T17:09 2021-11-03T17:09:58.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:57 smithi115 conmon[34670]: :57.534990+0000 mon.smithi115 (mon.1) 2021-11-03T17:09:58.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:57 smithi115 conmon[34670]: 7 : audit [DBG] from='client.? 172.21.15.115:0/4187934096' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2021-11-03T17:09:58.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:57 smithi027 conmon[32322]: audit 2021-11-03T17:09:57.032437+0000 mon.smithi115 ( 2021-11-03T17:09:58.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:57 smithi027 conmon[32322]: mon.1) 6 : audit [INF] from='client.? 172.21.15.115:0/448728764' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aeeb2c74-919d-4ac7-8c7a-1dd273225f59"}]: dispatch 2021-11-03T17:09:58.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:57 smithi027 conmon[32322]: audit 2021-11-03T17:09:57.033028+0000 2021-11-03T17:09:58.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:57 smithi027 conmon[32322]: mon.smithi027 (mon.0) 373 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aeeb2c74-919d-4ac7-8c7a-1dd273225f59"}]: dispatch 2021-11-03T17:09:58.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:57 smithi027 conmon[32322]: audit 2021-11 2021-11-03T17:09:58.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:57 smithi027 conmon[32322]: -03T17:09:57 2021-11-03T17:09:58.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:57 smithi027 conmon[32322]: .036919+0000 mon.smithi027 (mon.0) 374 2021-11-03T17:09:58.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:57 smithi027 conmon[32322]: : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aeeb2c74-919d-4ac7-8c7a-1dd273225f59"}]': finished 2021-11-03T17:09:58.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:57 smithi027 conmon[32322]: cluster 2021-11 2021-11-03T17:09:58.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:57 smithi027 conmon[32322]: -03T17:09:57. 2021-11-03T17:09:58.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:57 smithi027 conmon[32322]: 036991+0000 mon.smithi027 (mon.0) 375 : 2021-11-03T17:09:58.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:57 smithi027 conmon[32322]: cluster [DBG] osdmap e30: 5 total, 4 up, 5 in 2021-11-03T17:09:58.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:57 smithi027 conmon[32322]: audit 2021 2021-11-03T17:09:58.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:57 smithi027 conmon[32322]: -11-03T17:09 2021-11-03T17:09:58.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:57 smithi027 conmon[32322]: :57.037109+0000 mon.smithi027 (mon.0 2021-11-03T17:09:58.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:57 smithi027 conmon[32322]: ) 376 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-11-03T17:09:58.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:57 smithi027 conmon[32322]: audit 2021-11-03T17:09:58.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:57 smithi027 conmon[32322]: 2021-11-03T17: 2021-11-03T17:09:58.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:57 smithi027 conmon[32322]: 09:57.534990+0000 mon.smithi115 (mon.1 2021-11-03T17:09:58.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:57 smithi027 conmon[32322]: ) 7 : audit [DBG] from='client.? 172.21.15.115:0/4187934096' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2021-11-03T17:09:59.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:58 smithi115 conmon[34670]: cluster 2021-11-03T17:09:58.551576+0000 mgr.smithi027.ufhhvb (mgr.14162) 144 : cluster [DBG] pgmap v112: 1 pgs: 1 remapped+peering; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2021-11-03T17:09:59.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:09:58 smithi115 conmon[34670]: 2021-11-03T17:09:59.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:58 smithi027 conmon[32322]: cluster 2021-11-03T17:09:58.551576+0000 mgr.smithi027.ufhhvb ( 2021-11-03T17:09:59.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:09:58 smithi027 conmon[32322]: mgr.14162) 144 : cluster [DBG] pgmap v112: 1 pgs: 1 remapped+peering; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2021-11-03T17:10:00.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:00 smithi115 conmon[34670]: audit 2021-11-03T17:09:59.049605+0000 mon.smithi027 (mon.0) 377 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:10:00.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:00 smithi115 conmon[34670]: audit 2021-11-03T17:09:59.055278+0000 mon.smithi027 (mon.0) 378 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:10:00.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:00 smithi115 conmon[34670]: cluster 2021-11-03T17:10:00.000157+0000 mon.smithi027 (mon.0) 379 : cluster [INF] overall HEALTH_OK 2021-11-03T17:10:00.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:00 smithi027 conmon[32322]: audit 2021-11-03T17:09:59.049605+0000 mon.smithi027 (mon.0) 377 : audit 2021-11-03T17:10:00.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:00 smithi027 conmon[32322]: [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:10:00.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:00 smithi027 conmon[32322]: audit 2021-11-03T17:09:59.055278+0000 mon.smithi027 (mon.0) 378 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:10:00.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:00 smithi027 conmon[32322]: cluster 2021-11-03T17:10:00.000157+0000 mon.smithi027 (mon.0) 379 2021-11-03T17:10:00.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:00 smithi027 conmon[32322]: : cluster [INF] overall HEALTH_OK 2021-11-03T17:10:01.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:01 smithi115 conmon[34670]: cluster 2021-11-03T17:10:00.551947+0000 mgr.smithi027.ufhhvb (mgr.14162 2021-11-03T17:10:01.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:01 smithi115 conmon[34670]: ) 145 : cluster [DBG] pgmap v113: 1 pgs: 1 remapped+peering; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2021-11-03T17:10:01.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:01 smithi027 conmon[32322]: cluster 2021-11-03T17:10:00.551947+0000 mgr.smithi027.ufhhvb (mgr.14162) 145 : 2021-11-03T17:10:01.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:01 smithi027 conmon[32322]: cluster [DBG] pgmap v113: 1 pgs: 1 remapped+peering; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2021-11-03T17:10:02.938 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:02 smithi027 conmon[32322]: cluster 2021-11-03T17:10:02.552346+0000 mgr.smithi027.ufhhvb (mgr.14162) 146 : cluster [DBG] pgmap v114: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2021-11-03T17:10:02.950 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:02 smithi115 conmon[34670]: cluster 2021-11-03T17:10:02.552346+0000 mgr.smithi027.ufhhvb (mgr.14162) 146 : cluster [DBG] pgmap v114: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2021-11-03T17:10:04.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:04 smithi115 conmon[34670]: audit 2021-11-03T17:10:04.290667+0000 mon.smithi027 (mon.0) 380 : audit 2021-11-03T17:10:04.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:04 smithi115 conmon[34670]: [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2021-11-03T17:10:04.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:04 smithi115 conmon[34670]: audit 2021-11-03T17:10:04.291694+0000 mon.smithi027 (mon.0) 381 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:10:04.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:04 smithi027 conmon[32322]: audit 2021-11-03T17:10:04.290667+0000 mon.smithi027 (mon.0) 380 : audit 2021-11-03T17:10:04.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:04 smithi027 conmon[32322]: [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2021-11-03T17:10:04.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:04 smithi027 conmon[32322]: audit 2021-11-03T17:10:04.291694+0000 mon.smithi027 (mon.0) 381 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:10:05.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:05 smithi115 conmon[34670]: cephadm 2021-11-03T17:10:04.292512+0000 mgr.smithi027.ufhhvb (mgr.14162) 147 : cephadm [INF] Deploying daemon osd.4 on smithi115 2021-11-03T17:10:05.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:05 smithi115 conmon[34670]: cluster 2021-11-03T17:10:04.552861+0000 mgr.smithi027.ufhhvb (mgr.14162) 148 : cluster [DBG] pgmap v115: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2021-11-03T17:10:05.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:05 smithi027 conmon[32322]: cephadm 2021-11-03T17:10:04.292512+0000 mgr.smithi027.ufhhvb (mgr.14162) 147 : cephadm [INF] 2021-11-03T17:10:05.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:05 smithi027 conmon[32322]: Deploying daemon osd.4 on smithi115 2021-11-03T17:10:05.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:05 smithi027 conmon[32322]: cluster 2021-11-03T17:10:04.552861+0000 mgr.smithi027.ufhhvb (mgr.14162) 148 : cluster [DBG] pgmap v115: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2021-11-03T17:10:07.770 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 4 on host 'smithi115' 2021-11-03T17:10:07.772 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:07.774+0000 7fa2dbfff700 1 -- 172.21.15.115:0/3212230075 <== mgr.14162 v2:172.21.15.27:6800/3802716887 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa2f4106910 con 0x7fa2dc05c300 2021-11-03T17:10:07.772 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:07.776+0000 7fa2f8e2b700 1 -- 172.21.15.115:0/3212230075 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fa2dc05c300 msgr2=0x7fa2dc05e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:07.772 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:07.776+0000 7fa2f8e2b700 1 --2- 172.21.15.115:0/3212230075 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fa2dc05c300 0x7fa2dc05e7c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fa2e8000c00 tx=0x7fa2e8009b30).stop 2021-11-03T17:10:07.772 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:07.776+0000 7fa2f8e2b700 1 -- 172.21.15.115:0/3212230075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2f40f5430 msgr2=0x7fa2f4104b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:07.773 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:07.776+0000 7fa2f8e2b700 1 --2- 172.21.15.115:0/3212230075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2f40f5430 0x7fa2f4104b50 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fa2e400d7f0 tx=0x7fa2e4004e80).stop 2021-11-03T17:10:07.773 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:07.776+0000 7fa2f8e2b700 1 -- 172.21.15.115:0/3212230075 shutdown_connections 2021-11-03T17:10:07.773 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:07.776+0000 7fa2f8e2b700 1 --2- 172.21.15.115:0/3212230075 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fa2dc05c300 0x7fa2dc05e7c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:07.773 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:07.776+0000 7fa2f8e2b700 1 --2- 172.21.15.115:0/3212230075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2f40f5430 0x7fa2f4104b50 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:07.774 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:07.776+0000 7fa2f8e2b700 1 --2- 172.21.15.115:0/3212230075 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2f40f66d0 0x7fa2f4105040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:07.774 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:07.776+0000 7fa2f8e2b700 1 -- 172.21.15.115:0/3212230075 >> 172.21.15.115:0/3212230075 conn(0x7fa2f40f0b90 msgr2=0x7fa2f40fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:10:07.774 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:07.776+0000 7fa2f8e2b700 1 -- 172.21.15.115:0/3212230075 shutdown_connections 2021-11-03T17:10:07.774 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:07.776+0000 7fa2f8e2b700 1 -- 172.21.15.115:0/3212230075 wait complete. 2021-11-03T17:10:07.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:07 smithi115 conmon[34670]: cluster 2021-11-03T17:10:06.553182+0000 mgr.smithi027.ufhhvb (mgr. 2021-11-03T17:10:07.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:07 smithi115 conmon[34670]: 14162) 149 : cluster [DBG] pgmap v116: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2021-11-03T17:10:08.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:07 smithi027 conmon[32322]: cluster 2021-11-03T17:10:06.553182+0000 mgr.smithi027.ufhhvb (mgr 2021-11-03T17:10:08.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:07 smithi027 conmon[32322]: .14162) 149 : cluster [DBG] pgmap v116: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2021-11-03T17:10:08.416 DEBUG:teuthology.orchestra.run.smithi115:osd.4> sudo journalctl -f -n 0 -u ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c@osd.4.service 2021-11-03T17:10:08.418 INFO:tasks.cephadm:Deploying osd.5 on smithi115 with /dev/vg_nvme/lv_2... 2021-11-03T17:10:08.419 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2021-11-03T17:10:08.446 INFO:journalctl@ceph.osd.4.smithi115.stdout:-- Logs begin at Wed 2021-11-03 16:55:13 UTC. -- 2021-11-03T17:10:08.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:08 smithi115 conmon[34670]: audit 2021-11-03T17:10:07.763968+0000 mon.smithi027 (mon.0) 382 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:10:08.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:08 smithi115 conmon[34670]: audit 2021-11-03T17:10:07.768872+0000 mon.smithi027 (mon.0) 383 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:10:08.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:08 smithi115 conmon[34670]: audit 2021-11-03T17:10:07.773864+0000 mon.smithi027 (mon.0) 384 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:10:09.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:08 smithi027 conmon[32322]: audit 2021-11-03T17:10:07.763968+0000 mon.smithi027 (mon.0) 382 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:10:09.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:08 smithi027 conmon[32322]: audit 2021-11-03T17:10:07.768872+0000 mon.smithi027 (mon.0) 383 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:10:09.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:08 smithi027 conmon[32322]: audit 2021-11-03T17:10:07.773864+0000 mon.smithi027 (mon.0) 384 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:10:09.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:09 smithi115 conmon[34670]: cluster 2021-11-03T17:10:08.553535+0000 mgr.smithi027.ufhhvb (mgr.14162) 150 : cluster [DBG] pgmap v117: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2021-11-03T17:10:10.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:09 smithi027 conmon[32322]: cluster 2021-11-03T17:10:08.553535+0000 mgr.smithi027.ufhhvb ( 2021-11-03T17:10:10.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:09 smithi027 conmon[32322]: mgr.14162) 150 : cluster [DBG] pgmap v117: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2021-11-03T17:10:10.852 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping: /dev/vg_nvme/lv_2 2021-11-03T17:10:10.853 INFO:teuthology.orchestra.run.smithi115.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2021-11-03T17:10:10.853 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10+0 records in 2021-11-03T17:10:10.853 INFO:teuthology.orchestra.run.smithi115.stderr:10+0 records out 2021-11-03T17:10:10.854 INFO:teuthology.orchestra.run.smithi115.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0121816 s, 861 MB/s 2021-11-03T17:10:10.854 INFO:teuthology.orchestra.run.smithi115.stderr:--> Zapping successful for: 2021-11-03T17:10:11.466 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph orch daemon add osd smithi115:vg_nvme/lv_2 2021-11-03T17:10:11.698 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:11 smithi115 conmon[34670]: cluster 2021-11-03T17:10:10.553834+0000 mgr.smithi027.ufhhvb (mgr.14162) 151 : cluster [DBG] pgmap v118: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2021-11-03T17:10:11.699 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:11 smithi115 conmon[34670]: audit 2021-11-03T17:10:10.844338+0000 mon.smithi027 (mon.0) 385 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:10:11.949 INFO:journalctl@ceph.osd.4.smithi115.stdout:Nov 03 17:10:11 smithi115 conmon[45920]: debug 2021-11-03T17:10:11.747+0000 7fef58429080 -1 osd.4 0 log_to_monitors {default=true} 2021-11-03T17:10:12.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:11 smithi027 conmon[32322]: cluster 2021-11-03T17:10:10. 2021-11-03T17:10:12.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:11 smithi027 conmon[32322]: 553834+0000 mgr.smithi027.ufhhvb (mgr.14162) 151 : cluster [DBG] pgmap v118: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2021-11-03T17:10:12.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:11 smithi027 conmon[32322]: audit 2021-11-03T17:10:10.844338+0000 mon.smithi027 (mon.0) 385 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:10:12.937 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:12 smithi027 conmon[32322]: audit 2021-11-03T17:10:11.569222+0000 mon.smithi027 (mon.0) 386 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:10:12.938 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:12 smithi027 conmon[32322]: audit 2021-11-03T17:10:11.750853+0000 mon.smithi115 (mon.1) 8 : audit [INF] from='osd.4 [v2:172.21.15.115:6808/1601394331,v1:172.21.15.115:6809/1601394331]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2021-11-03T17:10:12.938 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:12 smithi027 conmon[32322]: audit 2021-11-03T17:10:11.751353 2021-11-03T17:10:12.938 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:12 smithi027 conmon[32322]: +0000 mon.smithi027 (mon.0) 387 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2021-11-03T17:10:12.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:12 smithi115 conmon[34670]: audit 2021-11-03T17:10:11.569222 2021-11-03T17:10:12.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:12 smithi115 conmon[34670]: +0000 mon.smithi027 (mon.0) 386 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:10:12.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:12 smithi115 conmon[34670]: audit 2021-11-03T17:10:11.750853+0000 mon.smithi115 (mon.1) 8 : audit [INF] from='osd.4 [v2:172.21.15.115:6808/1601394331,v1:172.21.15.115:6809/1601394331]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2021-11-03T17:10:12.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:12 smithi115 conmon[34670]: audit 2021-11-03T17:10:11.751353+0000 mon.smithi027 (mon.0) 387 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2021-11-03T17:10:13.803 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:13.806+0000 7fe12614a700 1 -- 172.21.15.115:0/3927394137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1200f5ef0 msgr2=0x7fe1200f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:13.803 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:13.806+0000 7fe12614a700 1 --2- 172.21.15.115:0/3927394137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1200f5ef0 0x7fe1200f6920 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fe114004660 tx=0x7fe114009b30).stop 2021-11-03T17:10:13.803 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:13.806+0000 7fe12614a700 1 -- 172.21.15.115:0/3927394137 shutdown_connections 2021-11-03T17:10:13.804 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:13.806+0000 7fe12614a700 1 --2- 172.21.15.115:0/3927394137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1200f5ef0 0x7fe1200f6920 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:13.804 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:13.806+0000 7fe12614a700 1 --2- 172.21.15.115:0/3927394137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1200f55e0 0x7fe1200f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:13.804 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:13.806+0000 7fe12614a700 1 -- 172.21.15.115:0/3927394137 >> 172.21.15.115:0/3927394137 conn(0x7fe1200f0de0 msgr2=0x7fe1200f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:10:13.804 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:13.806+0000 7fe12614a700 1 -- 172.21.15.115:0/3927394137 shutdown_connections 2021-11-03T17:10:13.805 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:13.806+0000 7fe12614a700 1 -- 172.21.15.115:0/3927394137 wait complete. 2021-11-03T17:10:13.805 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:13.806+0000 7fe12614a700 1 Processor -- start 2021-11-03T17:10:13.805 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:13.807+0000 7fe12614a700 1 -- start start 2021-11-03T17:10:13.805 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:13.807+0000 7fe12614a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1200f55e0 0x7fe120109bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:10:13.806 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:13.807+0000 7fe12614a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1200f5ef0 0x7fe12010c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:10:13.806 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:13.807+0000 7fe12614a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe120060500 con 0x7fe1200f5ef0 2021-11-03T17:10:13.806 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:13.807+0000 7fe12614a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe120060640 con 0x7fe1200f55e0 2021-11-03T17:10:13.806 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:13.807+0000 7fe11f7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1200f55e0 0x7fe120109bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:10:13.807 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:13.807+0000 7fe11f7fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1200f55e0 0x7fe120109bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38686/0 (socket says 172.21.15.115:38686) 2021-11-03T17:10:13.807 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:13.807+0000 7fe11f7fe700 1 -- 172.21.15.115:0/2020115612 learned_addr learned my addr 172.21.15.115:0/2020115612 (peer_addr_for_me v2:172.21.15.115:0/0) 2021-11-03T17:10:13.807 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:13.807+0000 7fe11effd700 1 --2- 172.21.15.115:0/2020115612 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1200f5ef0 0x7fe12010c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:10:13.807 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:13.807+0000 7fe11f7fe700 1 -- 172.21.15.115:0/2020115612 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1200f5ef0 msgr2=0x7fe12010c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:13.808 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:13.807+0000 7fe11f7fe700 1 --2- 172.21.15.115:0/2020115612 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1200f5ef0 0x7fe12010c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:13.808 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:13.807+0000 7fe11f7fe700 1 -- 172.21.15.115:0/2020115612 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe114005040 con 0x7fe1200f55e0 2021-11-03T17:10:13.808 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:13.807+0000 7fe11f7fe700 1 --2- 172.21.15.115:0/2020115612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1200f55e0 0x7fe120109bf0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fe11000eb70 tx=0x7fe11000b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:10:13.808 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:13.807+0000 7fe107fff700 1 -- 172.21.15.115:0/2020115612 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1100092e0 con 0x7fe1200f55e0 2021-11-03T17:10:13.808 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:13.807+0000 7fe107fff700 1 -- 172.21.15.115:0/2020115612 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7fe11000f040 con 0x7fe1200f55e0 2021-11-03T17:10:13.809 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:13.807+0000 7fe12614a700 1 -- 172.21.15.115:0/2020115612 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe12010c6a0 con 0x7fe1200f55e0 2021-11-03T17:10:13.809 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:13.807+0000 7fe107fff700 1 -- 172.21.15.115:0/2020115612 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe11000c910 con 0x7fe1200f55e0 2021-11-03T17:10:13.809 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:13.807+0000 7fe12614a700 1 -- 172.21.15.115:0/2020115612 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe12010cc60 con 0x7fe1200f55e0 2021-11-03T17:10:13.809 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:13.808+0000 7fe12614a700 1 -- 172.21.15.115:0/2020115612 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe120045ad0 con 0x7fe1200f55e0 2021-11-03T17:10:13.811 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:13.809+0000 7fe107fff700 1 -- 172.21.15.115:0/2020115612 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fe11001b030 con 0x7fe1200f55e0 2021-11-03T17:10:13.811 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:13.809+0000 7fe107fff700 1 --2- 172.21.15.115:0/2020115612 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fe10805c2b0 0x7fe10805e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:10:13.811 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:13.809+0000 7fe107fff700 1 -- 172.21.15.115:0/2020115612 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3807+0+0 (secure 0 0 0) 0x7fe1100758b0 con 0x7fe1200f55e0 2021-11-03T17:10:13.812 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:13.809+0000 7fe11effd700 1 --2- 172.21.15.115:0/2020115612 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fe10805c2b0 0x7fe10805e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:10:13.812 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:13.810+0000 7fe11effd700 1 --2- 172.21.15.115:0/2020115612 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fe10805c2b0 0x7fe10805e770 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fe114004d80 tx=0x7fe11400b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:10:13.812 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:13.813+0000 7fe107fff700 1 -- 172.21.15.115:0/2020115612 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fe110048c70 con 0x7fe1200f55e0 2021-11-03T17:10:13.956 INFO:journalctl@ceph.osd.4.smithi115.stdout:Nov 03 17:10:13 smithi115 conmon[45920]: debug 2021-11-03T17:10:13.582+0000 7fef5019d700 -1 osd.4 0 waiting for initial osdmap 2021-11-03T17:10:13.957 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:13 smithi115 conmon[34670]: cluster 2021-11-03T17:10:12.554339+0000 2021-11-03T17:10:13.957 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:13 smithi115 conmon[34670]: mgr.smithi027.ufhhvb (mgr.14162) 152 : cluster [DBG] pgmap v119: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2021-11-03T17:10:13.957 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:13 smithi115 conmon[34670]: audit 2021-11-03T17:10:12.574988+0000 mon.smithi027 (mon.0) 388 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2021-11-03T17:10:13.957 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:13 smithi115 conmon[34670]: cluster 2021-11-03T17:10:12.575108+0000 mon.smithi027 (mon.0) 389 : cluster [DBG] osdmap e31: 5 total, 4 up, 5 in 2021-11-03T17:10:13.958 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:13 smithi115 conmon[34670]: audit 2021-11-03T17:10:12.575434+0000 mon.smithi027 (mon.0) 390 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-11-03T17:10:13.958 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:13 smithi115 conmon[34670]: audit 2021-11-03T17:10:12.577387+0000 mon.smithi115 (mon.1) 9 : audit [INF] from='osd.4 [v2:172.21.15.115:6808/1601394331,v1:172.21.15.115:6809/1601394331]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2021-11-03T17:10:13.958 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:13 smithi115 conmon[34670]: audit 2021-11-03T17:10:12.577812+0000 mon.smithi027 (mon.0) 391 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2021-11-03T17:10:13.990 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:13.995+0000 7fe12614a700 1 -- 172.21.15.115:0/2020115612 --> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fe1200fdf40 con 0x7fe10805c2b0 2021-11-03T17:10:14.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:13 smithi027 conmon[32322]: cluster 2021-11-03T17:10:12.554339+0000 mgr.smithi027.ufhhvb (mgr.14162) 152 : 2021-11-03T17:10:14.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:13 smithi027 conmon[32322]: cluster [DBG] pgmap v119: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2021-11-03T17:10:14.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:13 smithi027 conmon[32322]: audit 2021-11-03T17:10:12.574988+0000 mon.smithi027 (mon.0) 388 : audit [INF] 2021-11-03T17:10:14.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:13 smithi027 conmon[32322]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2021-11-03T17:10:14.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:13 smithi027 conmon[32322]: cluster 2021 2021-11-03T17:10:14.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:13 smithi027 conmon[32322]: -11-03T17:10:12 2021-11-03T17:10:14.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:13 smithi027 conmon[32322]: .575108+0000 mon.smithi027 (mon.0 2021-11-03T17:10:14.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:13 smithi027 conmon[32322]: ) 389 : cluster [DBG] osdmap e31: 5 total, 4 up, 5 in 2021-11-03T17:10:14.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:13 smithi027 conmon[32322]: audit 2021- 2021-11-03T17:10:14.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:13 smithi027 conmon[32322]: 11-03T17:10:12.575434+0000 2021-11-03T17:10:14.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:13 smithi027 conmon[32322]: mon.smithi027 (mon.0) 390 : 2021-11-03T17:10:14.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:13 smithi027 conmon[32322]: audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-11-03T17:10:14.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:13 smithi027 conmon[32322]: audit 2021 2021-11-03T17:10:14.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:13 smithi027 conmon[32322]: -11-03T17:10:12 2021-11-03T17:10:14.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:13 smithi027 conmon[32322]: .577387+0000 mon.smithi115 (mon. 2021-11-03T17:10:14.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:13 smithi027 conmon[32322]: 1) 9 : audit [INF] 2021-11-03T17:10:14.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:13 smithi027 conmon[32322]: from='osd.4 [v2:172.21.15.115:6808/1601394331,v1:172.21.15.115:6809/1601394331]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2021-11-03T17:10:14.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:13 smithi027 conmon[32322]: audit 2021-11-03 2021-11-03T17:10:14.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:13 smithi027 conmon[32322]: T17:10:12.577812+0000 mon.smithi027 (mon.0) 391 : audit [INF] from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2021-11-03T17:10:14.925 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:14 smithi115 conmon[34670]: audit 2021-11-03T17:10:13.577099+0000 mon.smithi027 (mon.0) 2021-11-03T17:10:14.925 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:14 smithi115 conmon[34670]: 392 : audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2021-11-03T17:10:14.926 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:14 smithi115 conmon[34670]: cluster 2021-11-03T17:10:13.577240+0000 mon.smithi027 (mon.0) 393 : cluster [DBG] osdmap e32: 5 total, 4 up, 5 in 2021-11-03T17:10:14.926 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:14 smithi115 conmon[34670]: audit 2021-11-03T17:10:13.577736+0000 mon.smithi027 (mon.0) 394 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-11-03T17:10:14.926 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:14 smithi115 conmon[34670]: audit 2021-11-03T17:10:13.582590+0000 mon.smithi027 (mon.0) 395 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-11-03T17:10:14.927 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:14 smithi115 conmon[34670]: audit 2021-11-03T17:10:13.998026+0000 mon.smithi027 (mon.0) 396 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2021-11-03T17:10:14.927 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:14 smithi115 conmon[34670]: audit 2021-11-03T17:10:14.000406+0000 mon.smithi027 (mon.0) 397 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2021-11-03T17:10:14.927 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:14 smithi115 conmon[34670]: audit 2021-11-03T17:10:14.001266+0000 mon.smithi027 (mon.0) 398 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:10:15.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:14 smithi027 conmon[32322]: audit 2021-11-03T17:10:13.577099+0000 mon.smithi027 (mon.0) 392 : 2021-11-03T17:10:15.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:14 smithi027 conmon[32322]: audit [INF] from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2021-11-03T17:10:15.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:14 smithi027 conmon[32322]: cluster 2021-11-03T17:10:13.577240+0000 mon.smithi027 (mon.0) 393 : cluster [DBG] osdmap e32: 5 total, 4 up, 5 in 2021-11-03T17:10:15.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:14 smithi027 conmon[32322]: audit 2021-11-03T17:10: 2021-11-03T17:10:15.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:14 smithi027 conmon[32322]: 13.577736+0000 mon.smithi027 (mon. 2021-11-03T17:10:15.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:14 smithi027 conmon[32322]: 0) 394 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-11-03T17:10:15.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:14 smithi027 conmon[32322]: audit 2021- 2021-11-03T17:10:15.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:14 smithi027 conmon[32322]: 11-03T17:10:13.582590+0000 2021-11-03T17:10:15.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:14 smithi027 conmon[32322]: mon.smithi027 (mon.0) 395 : 2021-11-03T17:10:15.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:14 smithi027 conmon[32322]: audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-11-03T17:10:15.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:14 smithi027 conmon[32322]: audit 2021 2021-11-03T17:10:15.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:14 smithi027 conmon[32322]: -11-03T17:10:13. 2021-11-03T17:10:15.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:14 smithi027 conmon[32322]: 998026+0000 mon.smithi027 (mon.0 2021-11-03T17:10:15.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:14 smithi027 conmon[32322]: ) 396 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2021-11-03T17:10:15.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:14 smithi027 conmon[32322]: audit 2021 2021-11-03T17:10:15.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:14 smithi027 conmon[32322]: -11-03T17:10:14.000406 2021-11-03T17:10:15.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:14 smithi027 conmon[32322]: +0000 mon.smithi027 ( 2021-11-03T17:10:15.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:14 smithi027 conmon[32322]: mon.0) 397 : audit 2021-11-03T17:10:15.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:14 smithi027 conmon[32322]: [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2021-11-03T17:10:15.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:14 smithi027 conmon[32322]: audit 2021 2021-11-03T17:10:15.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:14 smithi027 conmon[32322]: -11-03T17:10:14 2021-11-03T17:10:15.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:14 smithi027 conmon[32322]: .001266+0000 mon.smithi027 (mon. 2021-11-03T17:10:15.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:14 smithi027 conmon[32322]: 0) 398 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:10:15.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:14 smithi027 conmon[32322]: 2021-11-03T17:10:15.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:15 smithi115 conmon[34670]: cluster 2021-11-03T17:10:12.728020+0000 osd.4 (osd.4) 1 2021-11-03T17:10:15.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:15 smithi115 conmon[34670]: : cluster [DBG] purged_snaps scrub starts 2021-11-03T17:10:15.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:15 smithi115 conmon[34670]: cluster 2021-11-03T17:10:12.728116+0000 osd.4 (osd.4) 2 : cluster [DBG] purged_snaps scrub ok 2021-11-03T17:10:15.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:15 smithi115 conmon[34670]: audit 2021-11-03T17:10:13.997056+0000 mgr.smithi027.ufhhvb (mgr.14162) 153 : audit [DBG] from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:10:15.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:15 smithi115 conmon[34670]: cephadm 2021-11-03T17:10:13.999579+0000 mgr.smithi027.ufhhvb (mgr.14162) 154 : cephadm [INF] Applying service osd.None on host smithi115... 2021-11-03T17:10:15.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:15 smithi115 conmon[34670]: cluster 2021-11-03T17:10:14.554743+0000 mgr.smithi027.ufhhvb (mgr.14162) 155 : cluster [DBG] pgmap v122: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2021-11-03T17:10:15.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:15 smithi115 conmon[34670]: cluster 2021-11-03T17:10:14.578587+0000 mon.smithi027 (mon.0) 399 : cluster [INF] osd.4 [v2:172.21.15.115:6808/1601394331,v1:172.21.15.115:6809/1601394331] boot 2021-11-03T17:10:15.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:15 smithi115 conmon[34670]: cluster 2021-11-03T17:10:14.578640+0000 mon.smithi027 (mon.0) 400 : cluster [DBG] osdmap e33: 5 total, 5 up, 5 in 2021-11-03T17:10:15.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:15 smithi115 conmon[34670]: audit 2021-11-03T17:10:14.578789+0000 mon.smithi027 (mon.0) 401 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-11-03T17:10:15.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:15 smithi115 conmon[34670]: audit 2021-11-03T17:10:15.223605+0000 mon.smithi027 (mon.0) 402 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:10:16.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:15 smithi027 conmon[32322]: cluster 2021-11-03T17:10:12.728020+0000 osd.4 (osd.4) 2021-11-03T17:10:16.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:15 smithi027 conmon[32322]: 1 : cluster [DBG] purged_snaps scrub starts 2021-11-03T17:10:16.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:15 smithi027 conmon[32322]: cluster 2021-11-03T17:10:12.728116+0000 osd.4 (osd.4) 2 : cluster [DBG] purged_snaps scrub ok 2021-11-03T17:10:16.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:15 smithi027 conmon[32322]: audit 2021-11-03T17:10:16.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:15 smithi027 conmon[32322]: 2021-11-03T17: 2021-11-03T17:10:16.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:15 smithi027 conmon[32322]: 10:13.997056+0000 mgr.smithi027.ufhhvb ( 2021-11-03T17:10:16.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:15 smithi027 conmon[32322]: mgr.14162) 153 : audit [DBG] 2021-11-03T17:10:16.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:15 smithi027 conmon[32322]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi115:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:10:16.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:15 smithi027 conmon[32322]: cephadm 2021 2021-11-03T17:10:16.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:15 smithi027 conmon[32322]: -11-03T17:10:13.999579 2021-11-03T17:10:16.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:15 smithi027 conmon[32322]: +0000 mgr.smithi027.ufhhvb (mgr.14162) 154 : cephadm 2021-11-03T17:10:16.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:15 smithi027 conmon[32322]: [INF] Applying service osd.None on host smithi115... 2021-11-03T17:10:16.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:15 smithi027 conmon[32322]: cluster 2021 2021-11-03T17:10:16.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:15 smithi027 conmon[32322]: -11-03T17:10:14. 2021-11-03T17:10:16.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:15 smithi027 conmon[32322]: 554743+0000 mgr.smithi027.ufhhvb (mgr.14162) 2021-11-03T17:10:16.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:15 smithi027 conmon[32322]: 155 : cluster [DBG] pgmap v122: 1 pgs: 1 active+clean; 0 B data, 20 MiB used, 358 GiB / 358 GiB avail 2021-11-03T17:10:16.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:15 smithi027 conmon[32322]: cluster 2021 2021-11-03T17:10:16.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:15 smithi027 conmon[32322]: -11-03T17:10:14.578587+0000 2021-11-03T17:10:16.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:15 smithi027 conmon[32322]: mon.smithi027 (mon.0) 399 : cluster 2021-11-03T17:10:16.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:15 smithi027 conmon[32322]: [INF] osd.4 [v2:172.21.15.115:6808/1601394331,v1:172.21.15.115:6809/1601394331] boot 2021-11-03T17:10:16.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:15 smithi027 conmon[32322]: cluster 2021 2021-11-03T17:10:16.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:15 smithi027 conmon[32322]: -11-03T17:10:14 2021-11-03T17:10:16.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:15 smithi027 conmon[32322]: .578640+0000 mon.smithi027 (mon. 2021-11-03T17:10:16.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:15 smithi027 conmon[32322]: 0) 400 : cluster [DBG] osdmap e33: 5 total, 5 up, 5 in 2021-11-03T17:10:16.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:15 smithi027 conmon[32322]: 2021-11-03T17:10:16.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:15 smithi027 conmon[32322]: audit 2021 2021-11-03T17:10:16.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:15 smithi027 conmon[32322]: -11-03T17:10:14. 2021-11-03T17:10:16.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:15 smithi027 conmon[32322]: 578789+0000 mon.smithi027 (mon.0) 2021-11-03T17:10:16.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:15 smithi027 conmon[32322]: 401 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-11-03T17:10:16.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:15 smithi027 conmon[32322]: audit 2021 2021-11-03T17:10:16.026 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:15 smithi027 conmon[32322]: -11-03T17:10:15. 2021-11-03T17:10:16.026 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:15 smithi027 conmon[32322]: 223605+0000 mon.smithi027 (mon.0 2021-11-03T17:10:16.026 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:15 smithi027 conmon[32322]: ) 402 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:10:16.026 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:15 smithi027 conmon[32322]: 2021-11-03T17:10:16.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:16 smithi115 conmon[34670]: cephadm 2021-11-03T17:10:15.218449+0000 mgr.smithi027.ufhhvb (mgr.14162) 156 : cephadm [INF] Detected new or changed devices on smithi115 2021-11-03T17:10:16.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:16 smithi115 conmon[34670]: cluster 2021-11-03T17:10:15.591678+0000 mon.smithi027 (mon.0) 403 : cluster [DBG] osdmap e34: 5 total, 5 up, 5 in 2021-11-03T17:10:16.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:16 smithi115 conmon[34670]: audit 2021-11-03T17:10:15.799734+0000 mon.smithi027 (mon.0) 404 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:10:16.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:16 smithi115 conmon[34670]: audit 2021-11-03T17:10:15.803435+0000 mon.smithi027 (mon.0) 405 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:10:16.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:16 smithi115 conmon[34670]: audit 2021-11-03T17:10:15.810113+0000 mon.smithi027 (mon.0) 406 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:10:17.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:16 smithi027 conmon[32322]: cephadm 2021-11-03T17:10:15.218449+0000 mgr.smithi027.ufhhvb (mgr.14162) 156 : cephadm 2021-11-03T17:10:17.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:16 smithi027 conmon[32322]: [INF] Detected new or changed devices on smithi115 2021-11-03T17:10:17.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:16 smithi027 conmon[32322]: cluster 2021-11-03T17:10:15.591678+0000 mon.smithi027 (mon.0) 403 : cluster [DBG] osdmap e34: 5 total, 5 up, 5 in 2021-11-03T17:10:17.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:16 smithi027 conmon[32322]: audit 2021 2021-11-03T17:10:17.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:16 smithi027 conmon[32322]: -11-03T17:10:15.799734+0000 mon.smithi027 (mon.0) 404 : audit 2021-11-03T17:10:17.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:16 smithi027 conmon[32322]: [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:10:17.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:16 smithi027 conmon[32322]: audit 2021- 2021-11-03T17:10:17.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:16 smithi027 conmon[32322]: 11-03T17:10:15.803435+0000 mon.smithi027 (mon.0) 2021-11-03T17:10:17.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:16 smithi027 conmon[32322]: 405 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:10:17.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:16 smithi027 conmon[32322]: audit 2021-11- 2021-11-03T17:10:17.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:16 smithi027 conmon[32322]: 03T17:10:15.810113+0000 2021-11-03T17:10:17.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:16 smithi027 conmon[32322]: mon.smithi027 (mon.0) 406 : audit 2021-11-03T17:10:17.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:16 smithi027 conmon[32322]: [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:10:17.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:17 smithi115 conmon[34670]: cluster 2021-11-03T17:10:16.555133+0000 mgr.smithi027.ufhhvb (mgr.14162) 157 : 2021-11-03T17:10:17.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:17 smithi115 conmon[34670]: cluster [DBG] pgmap v125: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2021-11-03T17:10:17.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:17 smithi115 conmon[34670]: audit 2021-11-03T17:10:16.844986+0000 mon.smithi115 (mon.1) 10 : audit [INF] from='client.? 172.21.15.115:0/3066047859' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "047cfc7d-01d6-4859-8dc8-01d3811b9060"}]: dispatch 2021-11-03T17:10:17.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:17 smithi115 conmon[34670]: audit 2021-11-03T17:10:16.845574+0000 mon.smithi027 (mon.0) 407 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "047cfc7d-01d6-4859-8dc8-01d3811b9060"}]: dispatch 2021-11-03T17:10:17.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:17 smithi115 conmon[34670]: audit 2021-11-03T17:10:16 2021-11-03T17:10:17.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:17 smithi115 conmon[34670]: .849587+0000 mon.smithi027 (mon.0) 408 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "047cfc7d-01d6-4859-8dc8-01d3811b9060"}]': finished 2021-11-03T17:10:17.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:17 smithi115 conmon[34670]: cluster 2021-11 2021-11-03T17:10:17.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:17 smithi115 conmon[34670]: -03T17:10:16.849659+0000 mon.smithi027 (mon. 2021-11-03T17:10:17.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:17 smithi115 conmon[34670]: 0) 409 : cluster [DBG] osdmap e35: 6 total, 5 up, 6 in 2021-11-03T17:10:17.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:17 smithi115 conmon[34670]: audit 2021-11-03T17:10:17.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:17 smithi115 conmon[34670]: 2021-11-03T17:10 2021-11-03T17:10:17.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:17 smithi115 conmon[34670]: :16.849804+0000 mon.smithi027 (mon.0) 410 2021-11-03T17:10:17.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:17 smithi115 conmon[34670]: : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-11-03T17:10:17.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:17 smithi115 conmon[34670]: audit 2021- 2021-11-03T17:10:17.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:17 smithi115 conmon[34670]: 11-03T17:10: 2021-11-03T17:10:17.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:17 smithi115 conmon[34670]: 17.346901+0000 mon.smithi115 (mon.1) 11 2021-11-03T17:10:17.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:17 smithi115 conmon[34670]: : audit [DBG] from='client.? 172.21.15.115:0/2385429351' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2021-11-03T17:10:18.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:17 smithi027 conmon[32322]: cluster 2021-11-03T17:10:16.555133+0000 mgr.smithi027.ufhhvb (mgr.14162) 157 : 2021-11-03T17:10:18.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:17 smithi027 conmon[32322]: cluster [DBG] pgmap v125: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2021-11-03T17:10:18.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:17 smithi027 conmon[32322]: audit 2021-11-03T17:10:16.844986+0000 mon.smithi115 (mon.1) 10 : audit [INF] from='client.? 172.21.15.115:0/3066047859' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "047cfc7d-01d6-4859-8dc8-01d3811b9060"}]: dispatch 2021-11-03T17:10:18.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:17 smithi027 conmon[32322]: audit 2021-11-03T17: 2021-11-03T17:10:18.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:17 smithi027 conmon[32322]: 10:16.845574+0000 mon.smithi027 (mon.0) 407 : audit 2021-11-03T17:10:18.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:17 smithi027 conmon[32322]: [INF] from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "047cfc7d-01d6-4859-8dc8-01d3811b9060"}]: dispatch 2021-11-03T17:10:18.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:17 smithi027 conmon[32322]: audit 2021- 2021-11-03T17:10:18.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:17 smithi027 conmon[32322]: 11-03T17:10:16.849587+0000 mon.smithi027 (mon.0) 2021-11-03T17:10:18.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:17 smithi027 conmon[32322]: 408 : audit [INF] from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "047cfc7d-01d6-4859-8dc8-01d3811b9060"}]': finished 2021-11-03T17:10:18.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:17 smithi027 conmon[32322]: cluster 2021-11-03T17 2021-11-03T17:10:18.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:17 smithi027 conmon[32322]: :10:16.849659+0000 mon.smithi027 ( 2021-11-03T17:10:18.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:17 smithi027 conmon[32322]: mon.0) 409 : cluster [DBG] 2021-11-03T17:10:18.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:17 smithi027 conmon[32322]: osdmap e35: 6 total, 5 up, 6 in 2021-11-03T17:10:18.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:17 smithi027 conmon[32322]: audit 2021- 2021-11-03T17:10:18.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:17 smithi027 conmon[32322]: 11-03T17:10:16.849804 2021-11-03T17:10:18.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:17 smithi027 conmon[32322]: +0000 mon.smithi027 (mon.0) 410 2021-11-03T17:10:18.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:17 smithi027 conmon[32322]: : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-11-03T17:10:18.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:17 smithi027 conmon[32322]: audit 2021 2021-11-03T17:10:18.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:17 smithi027 conmon[32322]: -11-03T17:10:17 2021-11-03T17:10:18.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:17 smithi027 conmon[32322]: .346901+0000 mon.smithi115 (mon.1 2021-11-03T17:10:18.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:17 smithi027 conmon[32322]: ) 11 : audit [DBG] from='client.? 172.21.15.115:0/2385429351' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2021-11-03T17:10:18.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:17 smithi027 conmon[32322]: 2021-11-03T17:10:18.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:18 smithi115 conmon[34670]: cluster 2021-11-03T17:10:18.555674+0000 mgr.smithi027.ufhhvb (mgr.14162) 158 2021-11-03T17:10:18.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:18 smithi115 conmon[34670]: : cluster [DBG] pgmap v127: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2021-11-03T17:10:19.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:18 smithi027 conmon[32322]: cluster 2021-11-03T17:10:18.555674+0000 mgr.smithi027.ufhhvb (mgr.14162) 158 : 2021-11-03T17:10:19.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:18 smithi027 conmon[32322]: cluster [DBG] pgmap v127: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2021-11-03T17:10:20.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:20 smithi115 conmon[34670]: cluster 2021-11-03T17:10:20.556151+0000 mgr.smithi027.ufhhvb ( 2021-11-03T17:10:20.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:20 smithi115 conmon[34670]: mgr.14162) 159 : cluster [DBG] pgmap v128: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2021-11-03T17:10:21.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:20 smithi027 conmon[32322]: cluster 2021-11-03T17:10:20.556151+0000 mgr.smithi027.ufhhvb (mgr.14162) 159 : cluster 2021-11-03T17:10:21.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:20 smithi027 conmon[32322]: [DBG] pgmap v128: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2021-11-03T17:10:22.937 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:22 smithi027 conmon[32322]: cluster 2021-11-03T17:10:22.556583+0000 mgr.smithi027.ufhhvb (mgr.14162) 160 : cluster [DBG] pgmap v129: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2021-11-03T17:10:22.949 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:22 smithi115 conmon[34670]: cluster 2021-11-03T17:10:22.556583+0000 mgr.smithi027.ufhhvb (mgr.14162) 160 : cluster [DBG] pgmap v129: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2021-11-03T17:10:24.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:24 smithi115 conmon[34670]: audit 2021-11-03T17:10:24.095130+0000 mon.smithi027 (mon.0) 411 : 2021-11-03T17:10:24.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:24 smithi115 conmon[34670]: audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2021-11-03T17:10:24.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:24 smithi115 conmon[34670]: audit 2021-11-03T17:10:24.096192+0000 mon.smithi027 (mon.0) 412 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:10:24.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:24 smithi027 conmon[32322]: audit 2021-11-03T17:10:24.095130+0000 mon.smithi027 (mon.0) 2021-11-03T17:10:24.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:24 smithi027 conmon[32322]: 411 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2021-11-03T17:10:24.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:24 smithi027 conmon[32322]: audit 2021-11-03T17:10:24.096192+0000 mon.smithi027 (mon.0) 412 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:10:25.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:25 smithi115 conmon[34670]: cephadm 2021-11-03T17:10:24.097025+0000 mgr.smithi027.ufhhvb (mgr.14162) 161 2021-11-03T17:10:25.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:25 smithi115 conmon[34670]: : cephadm [INF] Deploying daemon osd.5 on smithi115 2021-11-03T17:10:25.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:25 smithi115 conmon[34670]: cluster 2021-11-03T17:10:24.557098+0000 mgr.smithi027.ufhhvb (mgr.14162) 162 : cluster [DBG] pgmap v130: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2021-11-03T17:10:25.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:25 smithi027 conmon[32322]: cephadm 2021-11-03T17:10:24.097025+0000 mgr.smithi027.ufhhvb (mgr.14162) 161 2021-11-03T17:10:25.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:25 smithi027 conmon[32322]: : cephadm [INF] Deploying daemon osd.5 on smithi115 2021-11-03T17:10:25.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:25 smithi027 conmon[32322]: cluster 2021-11-03T17:10:24.557098+0000 mgr.smithi027.ufhhvb (mgr.14162) 162 : cluster [DBG] pgmap v130: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2021-11-03T17:10:27.715 INFO:teuthology.orchestra.run.smithi115.stdout:Created osd(s) 5 on host 'smithi115' 2021-11-03T17:10:27.716 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:27.719+0000 7fe107fff700 1 -- 172.21.15.115:0/2020115612 <== mgr.14162 v2:172.21.15.27:6800/3802716887 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe1200fdf40 con 0x7fe10805c2b0 2021-11-03T17:10:27.717 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:27.721+0000 7fe12614a700 1 -- 172.21.15.115:0/2020115612 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fe10805c2b0 msgr2=0x7fe10805e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:27.717 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:27.721+0000 7fe12614a700 1 --2- 172.21.15.115:0/2020115612 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fe10805c2b0 0x7fe10805e770 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fe114004d80 tx=0x7fe11400b040).stop 2021-11-03T17:10:27.717 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:27.721+0000 7fe12614a700 1 -- 172.21.15.115:0/2020115612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1200f55e0 msgr2=0x7fe120109bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:27.717 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:27.721+0000 7fe12614a700 1 --2- 172.21.15.115:0/2020115612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1200f55e0 0x7fe120109bf0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fe11000eb70 tx=0x7fe11000b040).stop 2021-11-03T17:10:27.718 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:27.722+0000 7fe12614a700 1 -- 172.21.15.115:0/2020115612 shutdown_connections 2021-11-03T17:10:27.718 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:27.722+0000 7fe12614a700 1 --2- 172.21.15.115:0/2020115612 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fe10805c2b0 0x7fe10805e770 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:27.718 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:27.722+0000 7fe12614a700 1 --2- 172.21.15.115:0/2020115612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1200f55e0 0x7fe120109bf0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:27.718 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:27.722+0000 7fe12614a700 1 --2- 172.21.15.115:0/2020115612 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1200f5ef0 0x7fe12010c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:27.719 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:27.722+0000 7fe12614a700 1 -- 172.21.15.115:0/2020115612 >> 172.21.15.115:0/2020115612 conn(0x7fe1200f0de0 msgr2=0x7fe1200fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:10:27.719 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:27.722+0000 7fe12614a700 1 -- 172.21.15.115:0/2020115612 shutdown_connections 2021-11-03T17:10:27.719 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:27.722+0000 7fe12614a700 1 -- 172.21.15.115:0/2020115612 wait complete. 2021-11-03T17:10:27.881 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:27 smithi115 conmon[34670]: cluster 2021-11-03T17:10:26.557470+0000 2021-11-03T17:10:27.882 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:27 smithi115 conmon[34670]: mgr.smithi027.ufhhvb (mgr.14162) 163 : cluster [DBG] pgmap v131: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2021-11-03T17:10:28.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:27 smithi027 conmon[32322]: cluster 2021-11-03T17:10:26.557470+0000 mgr.smithi027.ufhhvb (mgr.14162) 163 : cluster 2021-11-03T17:10:28.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:27 smithi027 conmon[32322]: [DBG] pgmap v131: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2021-11-03T17:10:28.319 DEBUG:teuthology.orchestra.run.smithi115:osd.5> sudo journalctl -f -n 0 -u ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c@osd.5.service 2021-11-03T17:10:28.323 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2021-11-03T17:10:28.323 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph osd stat -f json 2021-11-03T17:10:28.351 INFO:journalctl@ceph.osd.5.smithi115.stdout:-- Logs begin at Wed 2021-11-03 16:55:13 UTC. -- 2021-11-03T17:10:28.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:28 smithi027 conmon[32322]: audit 2021-11-03T17:10:27.708639 2021-11-03T17:10:28.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:28 smithi027 conmon[32322]: +0000 mon.smithi027 (mon.0) 413 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:10:28.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:28 smithi027 conmon[32322]: audit 2021-11-03T17:10:27.713045+0000 mon.smithi027 (mon.0) 414 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:10:28.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:28 smithi027 conmon[32322]: audit 2021-11-03T17:10:27.719428+0000 mon.smithi027 (mon.0) 415 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:10:28.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:28 smithi115 conmon[34670]: audit 2021-11-03T17:10:27.708639+0000 mon.smithi027 (mon.0) 413 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:10:28.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:28 smithi115 conmon[34670]: audit 2021-11-03T17:10:27.713045+0000 mon.smithi027 (mon.0) 414 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:10:28.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:28 smithi115 conmon[34670]: audit 2021-11-03T17:10:27.719428+0000 mon.smithi027 (mon.0) 415 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:10:29.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:29 smithi115 conmon[34670]: cluster 2021-11-03T17:10:28.557900+0000 mgr.smithi027.ufhhvb (mgr. 2021-11-03T17:10:29.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:29 smithi115 conmon[34670]: 14162) 164 : cluster [DBG] pgmap v132: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2021-11-03T17:10:29.993 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:29 smithi027 conmon[32322]: cluster 2021-11-03T17:10:28.557900+0000 mgr.smithi027.ufhhvb ( 2021-11-03T17:10:29.993 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:29 smithi027 conmon[32322]: mgr.14162) 164 : cluster [DBG] pgmap v132: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2021-11-03T17:10:30.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.196+0000 7f9e31691700 1 -- 172.21.15.27:0/2284189899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e2c0f7980 msgr2=0x7f9e2c0f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:30.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.196+0000 7f9e31691700 1 --2- 172.21.15.27:0/2284189899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e2c0f7980 0x7f9e2c0f7de0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f9e14004660 tx=0x7f9e14009b30).stop 2021-11-03T17:10:30.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.196+0000 7f9e31691700 1 -- 172.21.15.27:0/2284189899 shutdown_connections 2021-11-03T17:10:30.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.196+0000 7f9e31691700 1 --2- 172.21.15.27:0/2284189899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e2c0f7980 0x7f9e2c0f7de0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:30.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.196+0000 7f9e31691700 1 --2- 172.21.15.27:0/2284189899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e2c0f66e0 0x7f9e2c0f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:30.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.196+0000 7f9e31691700 1 -- 172.21.15.27:0/2284189899 >> 172.21.15.27:0/2284189899 conn(0x7f9e2c0f1e40 msgr2=0x7f9e2c0f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:10:30.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.196+0000 7f9e31691700 1 -- 172.21.15.27:0/2284189899 shutdown_connections 2021-11-03T17:10:30.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.196+0000 7f9e31691700 1 -- 172.21.15.27:0/2284189899 wait complete. 2021-11-03T17:10:30.197 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.197+0000 7f9e31691700 1 Processor -- start 2021-11-03T17:10:30.197 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.197+0000 7f9e31691700 1 -- start start 2021-11-03T17:10:30.198 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.197+0000 7f9e31691700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e2c0f66e0 0x7f9e2c106f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:10:30.198 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.197+0000 7f9e31691700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e2c0f7980 0x7f9e2c107400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:10:30.198 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.197+0000 7f9e31691700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2c1079b0 con 0x7f9e2c0f66e0 2021-11-03T17:10:30.198 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.197+0000 7f9e31691700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e2c107af0 con 0x7f9e2c0f7980 2021-11-03T17:10:30.199 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.197+0000 7f9e2affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e2c0f66e0 0x7f9e2c106f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:10:30.199 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.197+0000 7f9e2affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e2c0f66e0 0x7f9e2c106f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54638/0 (socket says 172.21.15.27:54638) 2021-11-03T17:10:30.199 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.197+0000 7f9e2affd700 1 -- 172.21.15.27:0/117904323 learned_addr learned my addr 172.21.15.27:0/117904323 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:10:30.199 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.197+0000 7f9e2a7fc700 1 --2- 172.21.15.27:0/117904323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e2c0f7980 0x7f9e2c107400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:10:30.200 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.197+0000 7f9e2affd700 1 -- 172.21.15.27:0/117904323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e2c0f7980 msgr2=0x7f9e2c107400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:30.200 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.197+0000 7f9e2affd700 1 --2- 172.21.15.27:0/117904323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e2c0f7980 0x7f9e2c107400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:30.200 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.197+0000 7f9e2affd700 1 -- 172.21.15.27:0/117904323 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e14005040 con 0x7f9e2c0f66e0 2021-11-03T17:10:30.200 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.197+0000 7f9e2affd700 1 --2- 172.21.15.27:0/117904323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e2c0f66e0 0x7f9e2c106f10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f9e2400d8d0 tx=0x7f9e24004f00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:10:30.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.197+0000 7f9e237fe700 1 -- 172.21.15.27:0/117904323 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e2400b4b0 con 0x7f9e2c0f66e0 2021-11-03T17:10:30.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.198+0000 7f9e31691700 1 -- 172.21.15.27:0/117904323 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e2c108490 con 0x7f9e2c0f66e0 2021-11-03T17:10:30.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.198+0000 7f9e237fe700 1 -- 172.21.15.27:0/117904323 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f9e2400f040 con 0x7f9e2c0f66e0 2021-11-03T17:10:30.202 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.198+0000 7f9e237fe700 1 -- 172.21.15.27:0/117904323 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e24013940 con 0x7f9e2c0f66e0 2021-11-03T17:10:30.202 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.198+0000 7f9e31691700 1 -- 172.21.15.27:0/117904323 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e2c108a50 con 0x7f9e2c0f66e0 2021-11-03T17:10:30.202 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.198+0000 7f9e31691700 1 -- 172.21.15.27:0/117904323 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e2c045ad0 con 0x7f9e2c0f66e0 2021-11-03T17:10:30.203 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.199+0000 7f9e237fe700 1 -- 172.21.15.27:0/117904323 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f9e2401a030 con 0x7f9e2c0f66e0 2021-11-03T17:10:30.203 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.199+0000 7f9e237fe700 1 --2- 172.21.15.27:0/117904323 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f9e1805c300 0x7f9e1805e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:10:30.204 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.199+0000 7f9e237fe700 1 -- 172.21.15.27:0/117904323 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4202+0+0 (secure 0 0 0) 0x7f9e240759f0 con 0x7f9e2c0f66e0 2021-11-03T17:10:30.204 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.199+0000 7f9e2a7fc700 1 --2- 172.21.15.27:0/117904323 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f9e1805c300 0x7f9e1805e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:10:30.204 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.200+0000 7f9e2a7fc700 1 --2- 172.21.15.27:0/117904323 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f9e1805c300 0x7f9e1805e7c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f9e14005010 tx=0x7f9e14009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:10:30.205 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.203+0000 7f9e237fe700 1 -- 172.21.15.27:0/117904323 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f9e24048a30 con 0x7f9e2c0f66e0 2021-11-03T17:10:30.356 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.359+0000 7f9e31691700 1 -- 172.21.15.27:0/117904323 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f9e2c027a00 con 0x7f9e2c0f66e0 2021-11-03T17:10:30.356 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.360+0000 7f9e237fe700 1 -- 172.21.15.27:0/117904323 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v35) v1 ==== 74+0+130 (secure 0 0 0) 0x7f9e24018090 con 0x7f9e2c0f66e0 2021-11-03T17:10:30.356 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:10:30.357 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.361+0000 7f9e31691700 1 -- 172.21.15.27:0/117904323 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f9e1805c300 msgr2=0x7f9e1805e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:30.358 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.361+0000 7f9e31691700 1 --2- 172.21.15.27:0/117904323 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f9e1805c300 0x7f9e1805e7c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f9e14005010 tx=0x7f9e14009b30).stop 2021-11-03T17:10:30.358 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.361+0000 7f9e31691700 1 -- 172.21.15.27:0/117904323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e2c0f66e0 msgr2=0x7f9e2c106f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:30.358 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.361+0000 7f9e31691700 1 --2- 172.21.15.27:0/117904323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e2c0f66e0 0x7f9e2c106f10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f9e2400d8d0 tx=0x7f9e24004f00).stop 2021-11-03T17:10:30.359 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.361+0000 7f9e31691700 1 -- 172.21.15.27:0/117904323 shutdown_connections 2021-11-03T17:10:30.359 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.361+0000 7f9e31691700 1 --2- 172.21.15.27:0/117904323 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f9e1805c300 0x7f9e1805e7c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:30.359 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.362+0000 7f9e31691700 1 --2- 172.21.15.27:0/117904323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e2c0f66e0 0x7f9e2c106f10 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:30.359 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.362+0000 7f9e31691700 1 --2- 172.21.15.27:0/117904323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e2c0f7980 0x7f9e2c107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:30.359 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.362+0000 7f9e31691700 1 -- 172.21.15.27:0/117904323 >> 172.21.15.27:0/117904323 conn(0x7f9e2c0f1e40 msgr2=0x7f9e2c0fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:10:30.360 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.362+0000 7f9e31691700 1 -- 172.21.15.27:0/117904323 shutdown_connections 2021-11-03T17:10:30.360 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:30.362+0000 7f9e31691700 1 -- 172.21.15.27:0/117904323 wait complete. 2021-11-03T17:10:30.981 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":35,"num_osds":6,"num_up_osds":5,"osd_up_since":1635959414,"num_in_osds":6,"osd_in_since":1635959416,"num_remapped_pgs":0} 2021-11-03T17:10:31.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:31 smithi115 conmon[34670]: audit 2021-11-03T17:10:30.037645+0000 mon.smithi027 (mon.0) 416 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:10:31.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:31 smithi115 conmon[34670]: audit 2021-11-03T17:10:30.361741+0000 mon.smithi027 (mon.0) 417 : audit [DBG] from='client.? 172.21.15.27:0/117904323' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2021-11-03T17:10:31.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:31 smithi115 conmon[34670]: cluster 2021-11-03T17:10:30.558183+0000 mgr.smithi027.ufhhvb (mgr.14162) 165 : cluster [DBG] pgmap v133: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2021-11-03T17:10:31.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:31 smithi027 conmon[32322]: audit 2021-11-03T17:10:30.037645+0000 mon.smithi027 (mon.0) 416 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:10:31.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:31 smithi027 conmon[32322]: audit 2021-11-03T17:10:30.361741+0000 mon.smithi027 (mon.0) 417 : audit [DBG] from='client.? 172.21.15.27:0/117904323' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2021-11-03T17:10:31.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:31 smithi027 conmon[32322]: cluster 2021-11-03T17:10:30.558183+0000 mgr.smithi027.ufhhvb (mgr.14162) 165 : cluster [DBG] pgmap v133: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2021-11-03T17:10:31.679 INFO:journalctl@ceph.osd.5.smithi115.stdout:Nov 03 17:10:31 smithi115 conmon[50881]: debug 2021-11-03T17:10:31.654+0000 7f797c119080 -1 osd.5 0 log_to_monitors {default=true} 2021-11-03T17:10:31.983 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph osd stat -f json 2021-11-03T17:10:32.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:32 smithi115 conmon[34670]: audit 2021-11-03T17:10:31.657661+0000 mon.smithi115 (mon.1) 12 : audit 2021-11-03T17:10:32.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:32 smithi115 conmon[34670]: [INF] from='osd.5 [v2:172.21.15.115:6816/2350455075,v1:172.21.15.115:6817/2350455075]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2021-11-03T17:10:32.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:32 smithi115 conmon[34670]: audit 2021-11-03T17:10:31.658162+0000 mon.smithi027 (mon.0) 418 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2021-11-03T17:10:32.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:32 smithi027 conmon[32322]: audit 2021-11-03T17:10:31.657661+0000 mon.smithi115 (mon.1) 12 : audit [INF] from='osd.5 [v2:172.21.15.115:6816/2350455075,v1:172.21.15.115:6817/2350455075]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2021-11-03T17:10:32.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:32 smithi027 conmon[32322]: audit 2021-11-03T17:10:31.658162+0000 mon.smithi027 (mon.0) 418 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2021-11-03T17:10:33.676 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:33 smithi027 conmon[32322]: cluster 2021-11-03T17:10:32.558487+0000 mgr.smithi027.ufhhvb ( 2021-11-03T17:10:33.676 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:33 smithi027 conmon[32322]: mgr.14162) 166 : cluster [DBG] pgmap v134: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2021-11-03T17:10:33.677 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:33 smithi027 conmon[32322]: audit 2021-11-03T17:10:32.561537+0000 mon.smithi027 (mon.0) 419 : 2021-11-03T17:10:33.677 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:33 smithi027 conmon[32322]: audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2021-11-03T17:10:33.677 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:33 smithi027 conmon[32322]: cluster 2021-11-03T17:10:32.561599 2021-11-03T17:10:33.677 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:33 smithi027 conmon[32322]: +0000 mon.smithi027 (mon.0) 420 : cluster [DBG] osdmap e36: 6 total, 5 up, 6 in 2021-11-03T17:10:33.678 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:33 smithi027 conmon[32322]: audit 2021-11 2021-11-03T17:10:33.678 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:33 smithi027 conmon[32322]: -03T17:10:32.561896+0000 mon.smithi027 (mon.0) 421 2021-11-03T17:10:33.678 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:33 smithi027 conmon[32322]: : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-11-03T17:10:33.678 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:33 smithi027 conmon[32322]: audit 2021-11-03T17:10:32.565477+0000 mon.smithi115 2021-11-03T17:10:33.679 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:33 smithi027 conmon[32322]: (mon.1) 13 : audit [INF] from='osd.5 [v2:172.21.15.115:6816/2350455075,v1:172.21.15.115:6817/2350455075]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2021-11-03T17:10:33.679 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:33 smithi027 conmon[32322]: audit 2021-11-03T17:10:32.567681+0000 2021-11-03T17:10:33.679 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:33 smithi027 conmon[32322]: mon.smithi027 (mon.0) 422 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2021-11-03T17:10:33.679 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:33 smithi027 conmon[32322]: audit 2021-11-03T17: 2021-11-03T17:10:33.680 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:33 smithi027 conmon[32322]: 10:33.297668+0000 mon.smithi027 (mon.0) 423 : audit 2021-11-03T17:10:33.680 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:33 smithi027 conmon[32322]: [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:10:33.680 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:33 smithi027 conmon[32322]: audit 2021-11-03T17:10:33.680 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:33 smithi027 conmon[32322]: 2021-11-03T17:10:33.301159+0000 mon.smithi027 (mon.0) 424 : audit 2021-11-03T17:10:33.681 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:33 smithi027 conmon[32322]: [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:10:33.681 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:33 smithi027 conmon[32322]: audit 2021-11-03 2021-11-03T17:10:33.681 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:33 smithi027 conmon[32322]: T17:10:33.304369+0000 mon.smithi027 (mon.0) 425 : audit 2021-11-03T17:10:33.681 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:33 smithi027 conmon[32322]: [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:10:33.682 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:33 smithi027 conmon[32322]: audit 2021-11-03T17:10:33.308799+0000 mon.smithi027 ( 2021-11-03T17:10:33.682 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:33 smithi027 conmon[32322]: mon.0) 426 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:10:33.807 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.811+0000 7faa1caad700 1 -- 172.21.15.27:0/1677659768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa180f66e0 msgr2=0x7faa180f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:33.807 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.811+0000 7faa1caad700 1 --2- 172.21.15.27:0/1677659768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa180f66e0 0x7faa180f6b00 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7faa10009a30 tx=0x7faa10004c50).stop 2021-11-03T17:10:33.808 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.811+0000 7faa1caad700 1 -- 172.21.15.27:0/1677659768 shutdown_connections 2021-11-03T17:10:33.808 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.811+0000 7faa1caad700 1 --2- 172.21.15.27:0/1677659768 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa180f7980 0x7faa180f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:33.808 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.811+0000 7faa1caad700 1 --2- 172.21.15.27:0/1677659768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa180f66e0 0x7faa180f6b00 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:33.808 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.811+0000 7faa1caad700 1 -- 172.21.15.27:0/1677659768 >> 172.21.15.27:0/1677659768 conn(0x7faa180f1e40 msgr2=0x7faa180f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:10:33.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.811+0000 7faa1caad700 1 -- 172.21.15.27:0/1677659768 shutdown_connections 2021-11-03T17:10:33.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.811+0000 7faa1caad700 1 -- 172.21.15.27:0/1677659768 wait complete. 2021-11-03T17:10:33.812 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.812+0000 7faa1caad700 1 Processor -- start 2021-11-03T17:10:33.812 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.812+0000 7faa1caad700 1 -- start start 2021-11-03T17:10:33.813 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.812+0000 7faa1caad700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa180f66e0 0x7faa1806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:10:33.813 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.812+0000 7faa1caad700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa180f7980 0x7faa18069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:10:33.813 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.812+0000 7faa1caad700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1806a380 con 0x7faa180f7980 2021-11-03T17:10:33.813 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.812+0000 7faa1caad700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa1806a4c0 con 0x7faa180f66e0 2021-11-03T17:10:33.813 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.812+0000 7faa15d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa180f7980 0x7faa18069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:10:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.812+0000 7faa15d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa180f7980 0x7faa18069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54648/0 (socket says 172.21.15.27:54648) 2021-11-03T17:10:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.812+0000 7faa15d9b700 1 -- 172.21.15.27:0/2050439266 learned_addr learned my addr 172.21.15.27:0/2050439266 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:10:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.812+0000 7faa1659c700 1 --2- 172.21.15.27:0/2050439266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa180f66e0 0x7faa1806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:10:33.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.812+0000 7faa15d9b700 1 -- 172.21.15.27:0/2050439266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa180f66e0 msgr2=0x7faa1806b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:33.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.812+0000 7faa15d9b700 1 --2- 172.21.15.27:0/2050439266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa180f66e0 0x7faa1806b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:33.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.812+0000 7faa15d9b700 1 -- 172.21.15.27:0/2050439266 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa00005040 con 0x7faa180f7980 2021-11-03T17:10:33.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.812+0000 7faa1659c700 1 --2- 172.21.15.27:0/2050439266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa180f66e0 0x7faa1806b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:10:33.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.812+0000 7faa15d9b700 1 --2- 172.21.15.27:0/2050439266 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa180f7980 0x7faa18069e20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7faa00004960 tx=0x7faa00009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:10:33.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.813+0000 7faa0effd700 1 -- 172.21.15.27:0/2050439266 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa00007e50 con 0x7faa180f7980 2021-11-03T17:10:33.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.813+0000 7faa0effd700 1 -- 172.21.15.27:0/2050439266 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7faa0000f040 con 0x7faa180f7980 2021-11-03T17:10:33.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.813+0000 7faa1caad700 1 -- 172.21.15.27:0/2050439266 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa10009640 con 0x7faa180f7980 2021-11-03T17:10:33.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.813+0000 7faa0effd700 1 -- 172.21.15.27:0/2050439266 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa00014960 con 0x7faa180f7980 2021-11-03T17:10:33.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.813+0000 7faa1caad700 1 -- 172.21.15.27:0/2050439266 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa1806abd0 con 0x7faa180f7980 2021-11-03T17:10:33.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.814+0000 7faa1caad700 1 -- 172.21.15.27:0/2050439266 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa18045ad0 con 0x7faa180f7980 2021-11-03T17:10:33.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.814+0000 7faa0effd700 1 -- 172.21.15.27:0/2050439266 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7faa00014ba0 con 0x7faa180f7980 2021-11-03T17:10:33.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.814+0000 7faa0effd700 1 --2- 172.21.15.27:0/2050439266 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7faa0405c2b0 0x7faa0405e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:10:33.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.814+0000 7faa0effd700 1 -- 172.21.15.27:0/2050439266 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 4239+0+0 (secure 0 0 0) 0x7faa00075c70 con 0x7faa180f7980 2021-11-03T17:10:33.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.814+0000 7faa1659c700 1 --2- 172.21.15.27:0/2050439266 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7faa0405c2b0 0x7faa0405e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:10:33.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.815+0000 7faa1659c700 1 --2- 172.21.15.27:0/2050439266 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7faa0405c2b0 0x7faa0405e770 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7faa10009a00 tx=0x7faa10004c50).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:10:33.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.818+0000 7faa0effd700 1 -- 172.21.15.27:0/2050439266 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7faa000472b0 con 0x7faa180f7980 2021-11-03T17:10:33.955 INFO:journalctl@ceph.osd.5.smithi115.stdout:Nov 03 17:10:33 smithi115 conmon[50881]: debug 2021-11-03T17:10:33.578+0000 7f797268a700 -1 osd.5 0 waiting for initial osdmap 2021-11-03T17:10:33.956 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:33 smithi115 conmon[34670]: cluster 2021-11-03T17:10:32.558487+0000 mgr.smithi027.ufhhvb (mgr.14162) 166 : cluster 2021-11-03T17:10:33.956 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:33 smithi115 conmon[34670]: [DBG] pgmap v134: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2021-11-03T17:10:33.956 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:33 smithi115 conmon[34670]: audit 2021-11-03T17:10:32.561537+0000 mon.smithi027 (mon.0) 419 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2021-11-03T17:10:33.957 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:33 smithi115 conmon[34670]: cluster 2021-11-03T17:10:33.957 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:33 smithi115 conmon[34670]: 2021-11-03T17:10:32.561599+0000 mon.smithi027 (mon.0) 420 : cluster [DBG] osdmap e36: 6 total, 5 up, 6 in 2021-11-03T17:10:33.963 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:33 smithi115 conmon[34670]: audit 2021-11-03T17:10:33.963 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:33 smithi115 conmon[34670]: 2021-11-03T17:10:32.561896 2021-11-03T17:10:33.964 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:33 smithi115 conmon[34670]: +0000 mon.smithi027 (mon.0) 421 : audit 2021-11-03T17:10:33.964 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:33 smithi115 conmon[34670]: [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-11-03T17:10:33.964 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:33 smithi115 conmon[34670]: audit 2021-11-03T17:10:33.964 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:33 smithi115 conmon[34670]: 2021-11-03T17:10: 2021-11-03T17:10:33.965 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:33 smithi115 conmon[34670]: 32.565477+0000 mon.smithi115 (mon.1) 13 2021-11-03T17:10:33.965 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:33 smithi115 conmon[34670]: : audit [INF] from='osd.5 [v2:172.21.15.115:6816/2350455075,v1:172.21.15.115:6817/2350455075]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2021-11-03T17:10:33.965 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:33 smithi115 conmon[34670]: audit 2021-11-03 2021-11-03T17:10:33.965 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:33 smithi115 conmon[34670]: T17:10:32.567681+0000 mon.smithi027 ( 2021-11-03T17:10:33.966 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:33 smithi115 conmon[34670]: mon.0) 422 : audit [INF] from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2021-11-03T17:10:33.966 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:33 smithi115 conmon[34670]: audit 2021-11-03T17:10:33.966 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:33 smithi115 conmon[34670]: 2021-11-03T17:10:33. 2021-11-03T17:10:33.966 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:33 smithi115 conmon[34670]: 297668+0000 mon.smithi027 (mon.0) 423 : audit 2021-11-03T17:10:33.967 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:33 smithi115 conmon[34670]: [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:10:33.967 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:33 smithi115 conmon[34670]: audit 2021-11-03T17: 2021-11-03T17:10:33.967 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:33 smithi115 conmon[34670]: 10:33.301159+0000 mon.smithi027 (mon.0) 2021-11-03T17:10:33.968 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:33 smithi115 conmon[34670]: 424 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:10:33.968 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:33 smithi115 conmon[34670]: audit 2021-11- 2021-11-03T17:10:33.968 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:33 smithi115 conmon[34670]: 03T17:10:33.304369+0000 mon.smithi027 ( 2021-11-03T17:10:33.968 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:33 smithi115 conmon[34670]: mon.0) 425 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:10:33.968 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:33 smithi115 conmon[34670]: audit 2021-11 2021-11-03T17:10:33.969 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:33 smithi115 conmon[34670]: -03T17:10:33.308799+0000 mon.smithi027 2021-11-03T17:10:33.969 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:33 smithi115 conmon[34670]: (mon.0) 426 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:10:33.969 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:33 smithi115 conmon[34670]: 2021-11-03T17:10:33.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.976+0000 7faa1caad700 1 -- 172.21.15.27:0/2050439266 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7faa18027a00 con 0x7faa180f7980 2021-11-03T17:10:33.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.976+0000 7faa0effd700 1 -- 172.21.15.27:0/2050439266 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v37) v1 ==== 74+0+130 (secure 0 0 0) 0x7faa0004c7e0 con 0x7faa180f7980 2021-11-03T17:10:33.972 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:10:33.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.978+0000 7faa1caad700 1 -- 172.21.15.27:0/2050439266 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7faa0405c2b0 msgr2=0x7faa0405e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:33.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.978+0000 7faa1caad700 1 --2- 172.21.15.27:0/2050439266 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7faa0405c2b0 0x7faa0405e770 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7faa10009a00 tx=0x7faa10004c50).stop 2021-11-03T17:10:33.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.978+0000 7faa1caad700 1 -- 172.21.15.27:0/2050439266 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa180f7980 msgr2=0x7faa18069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:33.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.978+0000 7faa1caad700 1 --2- 172.21.15.27:0/2050439266 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa180f7980 0x7faa18069e20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7faa00004960 tx=0x7faa00009b30).stop 2021-11-03T17:10:33.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.978+0000 7faa1caad700 1 -- 172.21.15.27:0/2050439266 shutdown_connections 2021-11-03T17:10:33.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.978+0000 7faa1caad700 1 --2- 172.21.15.27:0/2050439266 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7faa0405c2b0 0x7faa0405e770 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:33.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.978+0000 7faa1caad700 1 --2- 172.21.15.27:0/2050439266 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa180f66e0 0x7faa1806b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:33.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.978+0000 7faa1caad700 1 --2- 172.21.15.27:0/2050439266 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa180f7980 0x7faa18069e20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:33.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.978+0000 7faa1caad700 1 -- 172.21.15.27:0/2050439266 >> 172.21.15.27:0/2050439266 conn(0x7faa180f1e40 msgr2=0x7faa180fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:10:33.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.978+0000 7faa1caad700 1 -- 172.21.15.27:0/2050439266 shutdown_connections 2021-11-03T17:10:33.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:33.978+0000 7faa1caad700 1 -- 172.21.15.27:0/2050439266 wait complete. 2021-11-03T17:10:34.564 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":37,"num_osds":6,"num_up_osds":5,"osd_up_since":1635959414,"num_in_osds":6,"osd_in_since":1635959416,"num_remapped_pgs":0} 2021-11-03T17:10:34.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:34 smithi115 conmon[34670]: cephadm 2021-11-03T17:10:33.292897+0000 mgr.smithi027.ufhhvb (mgr.14162) 167 : cephadm [INF] Detected new or changed devices on smithi115 2021-11-03T17:10:34.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:34 smithi115 conmon[34670]: audit 2021-11-03T17:10:33.573890+0000 2021-11-03T17:10:34.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:34 smithi115 conmon[34670]: mon.smithi027 (mon.0) 427 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2021-11-03T17:10:34.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:34 smithi115 conmon[34670]: cluster 2021-11-03T17:10:33.573977+0000 mon.smithi027 (mon.0) 428 : cluster [DBG] osdmap e37: 6 total, 5 up, 6 in 2021-11-03T17:10:34.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:34 smithi115 conmon[34670]: audit 2021-11-03T17:10:33.574457+0000 mon.smithi027 2021-11-03T17:10:34.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:34 smithi115 conmon[34670]: (mon.0) 429 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-11-03T17:10:34.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:34 smithi115 conmon[34670]: audit 2021-11 2021-11-03T17:10:34.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:34 smithi115 conmon[34670]: -03T17:10:33.577016 2021-11-03T17:10:34.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:34 smithi115 conmon[34670]: +0000 mon.smithi027 (mon.0) 430 : 2021-11-03T17:10:34.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:34 smithi115 conmon[34670]: audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-11-03T17:10:34.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:34 smithi115 conmon[34670]: audit 2021-11-03 2021-11-03T17:10:34.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:34 smithi115 conmon[34670]: T17:10:33.978078+0000 mon.smithi027 (mon. 2021-11-03T17:10:34.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:34 smithi115 conmon[34670]: 0) 431 : audit [DBG] from='client.? 172.21.15.27:0/2050439266' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2021-11-03T17:10:34.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:34 smithi115 conmon[34670]: audit 2021- 2021-11-03T17:10:34.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:34 smithi115 conmon[34670]: 11-03T17:10:34.577020+0000 2021-11-03T17:10:34.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:34 smithi115 conmon[34670]: mon.smithi027 (mon.0) 432 : audit 2021-11-03T17:10:34.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:34 smithi115 conmon[34670]: [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-11-03T17:10:35.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:34 smithi027 conmon[32322]: cephadm 2021-11-03T17:10:33.292897+0000 mgr.smithi027.ufhhvb (mgr.14162) 167 : cephadm [INF] Detected new or changed devices on smithi115 2021-11-03T17:10:35.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:34 smithi027 conmon[32322]: audit 2021-11-03T17:10:33.573890+0000 mon.smithi027 (mon.0) 427 : audit [INF] from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]': finished 2021-11-03T17:10:35.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:34 smithi027 conmon[32322]: cluster 2021-11-03T17:10:33.573977+0000 mon.smithi027 (mon.0) 428 : cluster [DBG] osdmap e37: 6 total, 5 up, 6 in 2021-11-03T17:10:35.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:34 smithi027 conmon[32322]: audit 2021-11-03T17:10:33.574457+0000 mon.smithi027 (mon.0) 429 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-11-03T17:10:35.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:34 smithi027 conmon[32322]: audit 2021-11-03T17:10:33.577016+0000 mon.smithi027 (mon.0) 430 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-11-03T17:10:35.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:34 smithi027 conmon[32322]: audit 2021-11-03T17:10:33.978078+0000 mon.smithi027 (mon.0) 431 : audit [DBG] from='client.? 172.21.15.27:0/2050439266' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2021-11-03T17:10:35.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:34 smithi027 conmon[32322]: audit 2021-11-03T17: 2021-11-03T17:10:35.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:34 smithi027 conmon[32322]: 10:34.577020+0000 mon.smithi027 (mon.0) 432 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-11-03T17:10:35.565 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph osd stat -f json 2021-11-03T17:10:35.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:35 smithi115 conmon[34670]: cluster 2021-11-03T17:10:32.668205+0000 osd.5 (osd.5) 1 : cluster [DBG] 2021-11-03T17:10:35.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:35 smithi115 conmon[34670]: purged_snaps scrub starts 2021-11-03T17:10:35.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:35 smithi115 conmon[34670]: cluster 2021-11-03T17:10:32.668300+0000 osd.5 (osd.5) 2 : cluster [DBG] purged_snaps scrub ok 2021-11-03T17:10:35.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:35 smithi115 conmon[34670]: cluster 2021-11-03 2021-11-03T17:10:35.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:35 smithi115 conmon[34670]: T17:10:34.558867+0000 mgr.smithi027.ufhhvb (mgr.14162) 168 : cluster [DBG] pgmap v137: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2021-11-03T17:10:35.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:35 smithi115 conmon[34670]: cluster 2021-11 2021-11-03T17:10:35.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:35 smithi115 conmon[34670]: -03T17:10:34.580807+0000 mon.smithi027 2021-11-03T17:10:35.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:35 smithi115 conmon[34670]: (mon.0) 433 : cluster [INF] 2021-11-03T17:10:35.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:35 smithi115 conmon[34670]: osd.5 [v2:172.21.15.115:6816/2350455075,v1:172.21.15.115:6817/2350455075] boot 2021-11-03T17:10:35.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:35 smithi115 conmon[34670]: cluster 2021- 2021-11-03T17:10:35.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:35 smithi115 conmon[34670]: 11-03T17:10:34.580845+0000 2021-11-03T17:10:35.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:35 smithi115 conmon[34670]: mon.smithi027 (mon.0) 434 : cluster 2021-11-03T17:10:35.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:35 smithi115 conmon[34670]: [DBG] osdmap e38: 6 total, 6 up, 6 in 2021-11-03T17:10:35.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:35 smithi115 conmon[34670]: audit 2021- 2021-11-03T17:10:35.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:35 smithi115 conmon[34670]: 11-03T17:10:34.581373 2021-11-03T17:10:35.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:35 smithi115 conmon[34670]: +0000 mon.smithi027 (mon.0) 435 : 2021-11-03T17:10:35.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:35 smithi115 conmon[34670]: audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-11-03T17:10:36.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:35 smithi027 conmon[32322]: cluster 2021-11-03T17:10:32.668205+0000 osd.5 (osd.5) 1 : 2021-11-03T17:10:36.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:35 smithi027 conmon[32322]: cluster [DBG] purged_snaps scrub starts 2021-11-03T17:10:36.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:35 smithi027 conmon[32322]: cluster 2021-11-03T17:10:32.668300+0000 osd.5 (osd.5) 2 : cluster [DBG] purged_snaps scrub ok 2021-11-03T17:10:36.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:35 smithi027 conmon[32322]: cluster 2021-11-03T17:10:34.558867+0000 mgr.smithi027.ufhhvb (mgr.14162) 168 : cluster [DBG] pgmap v137: 1 pgs: 1 active+clean; 0 B data, 25 MiB used, 447 GiB / 447 GiB avail 2021-11-03T17:10:36.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:35 smithi027 conmon[32322]: cluster 2021-11-03T17:10:34.580807+0000 mon.smithi027 (mon.0) 433 : cluster [INF] osd.5 [v2:172.21.15.115:6816/2350455075,v1:172.21.15.115:6817/2350455075] boot 2021-11-03T17:10:36.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:35 smithi027 conmon[32322]: cluster 2021-11 2021-11-03T17:10:36.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:35 smithi027 conmon[32322]: -03T17:10:34.580845+0000 mon.smithi027 (mon.0) 434 : cluster [DBG] osdmap e38: 6 total, 6 up, 6 in 2021-11-03T17:10:36.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:35 smithi027 conmon[32322]: audit 2021-11 2021-11-03T17:10:36.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:35 smithi027 conmon[32322]: -03T17:10:34.581373+0000 mon.smithi027 (mon.0) 2021-11-03T17:10:36.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:35 smithi027 conmon[32322]: 435 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-11-03T17:10:36.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:36 smithi027 conmon[32322]: cluster 2021-11-03T17:10:35.588652+0000 mon.smithi027 (mon. 2021-11-03T17:10:36.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:36 smithi027 conmon[32322]: 0) 436 : cluster [DBG] osdmap e39: 6 total, 6 up, 6 in 2021-11-03T17:10:36.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:36 smithi115 conmon[34670]: cluster 2021-11-03T17:10:35.588652+0000 mon.smithi027 (mon.0) 436 : cluster [DBG] osdmap e39: 6 total, 6 up, 6 in 2021-11-03T17:10:37.474 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.478+0000 7f52dd891700 1 -- 172.21.15.27:0/1119288214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52d80f6530 msgr2=0x7f52d80f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:37.474 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.478+0000 7f52dd891700 1 --2- 172.21.15.27:0/1119288214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52d80f6530 0x7f52d80f6950 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f52cc004660 tx=0x7f52cc009b30).stop 2021-11-03T17:10:37.475 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.478+0000 7f52dd891700 1 -- 172.21.15.27:0/1119288214 shutdown_connections 2021-11-03T17:10:37.475 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.478+0000 7f52dd891700 1 --2- 172.21.15.27:0/1119288214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52d80f77d0 0x7f52d80f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:37.475 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.478+0000 7f52dd891700 1 --2- 172.21.15.27:0/1119288214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52d80f6530 0x7f52d80f6950 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:37.475 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.478+0000 7f52dd891700 1 -- 172.21.15.27:0/1119288214 >> 172.21.15.27:0/1119288214 conn(0x7f52d80f1cd0 msgr2=0x7f52d80f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:10:37.476 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.478+0000 7f52dd891700 1 -- 172.21.15.27:0/1119288214 shutdown_connections 2021-11-03T17:10:37.476 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.478+0000 7f52dd891700 1 -- 172.21.15.27:0/1119288214 wait complete. 2021-11-03T17:10:37.476 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.478+0000 7f52dd891700 1 Processor -- start 2021-11-03T17:10:37.476 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.478+0000 7f52dd891700 1 -- start start 2021-11-03T17:10:37.478 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.478+0000 7f52dd891700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52d80f6530 0x7f52d8106db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:10:37.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.478+0000 7f52dd891700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52d80f77d0 0x7f52d81072a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:10:37.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.478+0000 7f52dd891700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d81078a0 con 0x7f52d80f77d0 2021-11-03T17:10:37.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.478+0000 7f52dd891700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d81080e0 con 0x7f52d80f6530 2021-11-03T17:10:37.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.479+0000 7f52d7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52d80f77d0 0x7f52d81072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:10:37.480 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.479+0000 7f52d7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52d80f77d0 0x7f52d81072a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54672/0 (socket says 172.21.15.27:54672) 2021-11-03T17:10:37.480 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.479+0000 7f52d7fff700 1 -- 172.21.15.27:0/390926948 learned_addr learned my addr 172.21.15.27:0/390926948 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:10:37.480 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.479+0000 7f52dc88f700 1 --2- 172.21.15.27:0/390926948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52d80f6530 0x7f52d8106db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:10:37.480 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.479+0000 7f52d7fff700 1 -- 172.21.15.27:0/390926948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52d80f6530 msgr2=0x7f52d8106db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:37.481 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.479+0000 7f52d7fff700 1 --2- 172.21.15.27:0/390926948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52d80f6530 0x7f52d8106db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:37.481 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.479+0000 7f52d7fff700 1 -- 172.21.15.27:0/390926948 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52cc005040 con 0x7f52d80f77d0 2021-11-03T17:10:37.481 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.479+0000 7f52d7fff700 1 --2- 172.21.15.27:0/390926948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52d80f77d0 0x7f52d81072a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f52c40026d0 tx=0x7f52c4009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:10:37.481 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.479+0000 7f52d57fa700 1 -- 172.21.15.27:0/390926948 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52c40079c0 con 0x7f52d80f77d0 2021-11-03T17:10:37.482 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.479+0000 7f52d57fa700 1 -- 172.21.15.27:0/390926948 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f52c400f040 con 0x7f52d80f77d0 2021-11-03T17:10:37.482 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.479+0000 7f52dd891700 1 -- 172.21.15.27:0/390926948 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52d81083c0 con 0x7f52d80f77d0 2021-11-03T17:10:37.482 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.479+0000 7f52d57fa700 1 -- 172.21.15.27:0/390926948 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52c4014820 con 0x7f52d80f77d0 2021-11-03T17:10:37.482 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.479+0000 7f52dd891700 1 -- 172.21.15.27:0/390926948 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52d8108900 con 0x7f52d80f77d0 2021-11-03T17:10:37.482 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.480+0000 7f52dd891700 1 -- 172.21.15.27:0/390926948 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52d8059100 con 0x7f52d80f77d0 2021-11-03T17:10:37.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.481+0000 7f52d57fa700 1 -- 172.21.15.27:0/390926948 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f52c40090d0 con 0x7f52d80f77d0 2021-11-03T17:10:37.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.481+0000 7f52d57fa700 1 --2- 172.21.15.27:0/390926948 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f52c805c300 0x7f52c805e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:10:37.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.481+0000 7f52d57fa700 1 -- 172.21.15.27:0/390926948 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4519+0+0 (secure 0 0 0) 0x7f52c4075c10 con 0x7f52d80f77d0 2021-11-03T17:10:37.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.481+0000 7f52dc88f700 1 --2- 172.21.15.27:0/390926948 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f52c805c300 0x7f52c805e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:10:37.484 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.482+0000 7f52dc88f700 1 --2- 172.21.15.27:0/390926948 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f52c805c300 0x7f52c805e7c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f52cc013620 tx=0x7f52cc009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:10:37.484 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.485+0000 7f52d57fa700 1 -- 172.21.15.27:0/390926948 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f52c4048ce0 con 0x7f52d80f77d0 2021-11-03T17:10:37.642 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.646+0000 7f52dd891700 1 -- 172.21.15.27:0/390926948 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f52d8045ad0 con 0x7f52d80f77d0 2021-11-03T17:10:37.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.647+0000 7f52d57fa700 1 -- 172.21.15.27:0/390926948 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v39) v1 ==== 74+0+130 (secure 0 0 0) 0x7f52c404c300 con 0x7f52d80f77d0 2021-11-03T17:10:37.643 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:10:37.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.648+0000 7f52dd891700 1 -- 172.21.15.27:0/390926948 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f52c805c300 msgr2=0x7f52c805e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:37.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.648+0000 7f52dd891700 1 --2- 172.21.15.27:0/390926948 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f52c805c300 0x7f52c805e7c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f52cc013620 tx=0x7f52cc009b30).stop 2021-11-03T17:10:37.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.648+0000 7f52dd891700 1 -- 172.21.15.27:0/390926948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52d80f77d0 msgr2=0x7f52d81072a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:37.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.648+0000 7f52dd891700 1 --2- 172.21.15.27:0/390926948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52d80f77d0 0x7f52d81072a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f52c40026d0 tx=0x7f52c4009de0).stop 2021-11-03T17:10:37.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.648+0000 7f52dd891700 1 -- 172.21.15.27:0/390926948 shutdown_connections 2021-11-03T17:10:37.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.648+0000 7f52dd891700 1 --2- 172.21.15.27:0/390926948 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f52c805c300 0x7f52c805e7c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:37.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.648+0000 7f52dd891700 1 --2- 172.21.15.27:0/390926948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52d80f6530 0x7f52d8106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:37.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.648+0000 7f52dd891700 1 --2- 172.21.15.27:0/390926948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52d80f77d0 0x7f52d81072a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:37.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.649+0000 7f52dd891700 1 -- 172.21.15.27:0/390926948 >> 172.21.15.27:0/390926948 conn(0x7f52d80f1cd0 msgr2=0x7f52d80fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:10:37.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.649+0000 7f52dd891700 1 -- 172.21.15.27:0/390926948 shutdown_connections 2021-11-03T17:10:37.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:37.649+0000 7f52dd891700 1 -- 172.21.15.27:0/390926948 wait complete. 2021-11-03T17:10:37.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:37 smithi115 conmon[34670]: cluster 2021-11-03T17:10:36.559192+0000 mgr.smithi027.ufhhvb (mgr.14162) 169 : cluster [DBG] pgmap v140: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:10:38.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:37 smithi027 conmon[32322]: cluster 2021-11-03T17:10:36.559192+0000 mgr.smithi027.ufhhvb 2021-11-03T17:10:38.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:37 smithi027 conmon[32322]: (mgr.14162) 169 : cluster [DBG] pgmap v140: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:10:38.237 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":39,"num_osds":6,"num_up_osds":6,"osd_up_since":1635959434,"num_in_osds":6,"osd_in_since":1635959416,"num_remapped_pgs":0} 2021-11-03T17:10:38.238 INFO:tasks.cephadm:Setting up client nodes... 2021-11-03T17:10:38.239 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2021-11-03T17:10:38.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:38 smithi115 conmon[34670]: audit 2021-11-03T17:10:37.648350+0000 mon.smithi027 (mon.0 2021-11-03T17:10:38.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:38 smithi115 conmon[34670]: ) 437 : audit [DBG] from='client.? 172.21.15.27:0/390926948' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2021-11-03T17:10:39.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:38 smithi027 conmon[32322]: audit 2021-11-03T17:10:37.648350+0000 mon.smithi027 (mon.0) 437 : audit [DBG] from='client.? 172.21.15.27:0/390926948' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2021-11-03T17:10:39.890 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:39 smithi027 conmon[32322]: cluster 2021-11-03T17:10:38.559511+0000 mgr.smithi027.ufhhvb (mgr.14162 2021-11-03T17:10:39.890 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:39 smithi027 conmon[32322]: ) 170 : cluster [DBG] pgmap v141: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:10:39.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:39 smithi115 conmon[34670]: cluster 2021-11-03T17:10:38.559511+0000 mgr.smithi027.ufhhvb (mgr.14162) 170 : cluster [DBG] pgmap v141: 1 pgs: 1 active+clean; 0 B data, 30 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:10:40.086 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.085+0000 7fd82c9d8700 1 -- 172.21.15.27:0/248942090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8280f68e0 msgr2=0x7fd8280f6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:40.087 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.085+0000 7fd82c9d8700 1 --2- 172.21.15.27:0/248942090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8280f68e0 0x7fd8280f6d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fd81c004660 tx=0x7fd81c009b30).stop 2021-11-03T17:10:40.087 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.085+0000 7fd82c9d8700 1 -- 172.21.15.27:0/248942090 shutdown_connections 2021-11-03T17:10:40.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.085+0000 7fd82c9d8700 1 --2- 172.21.15.27:0/248942090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8280f68e0 0x7fd8280f6d40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:40.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.085+0000 7fd82c9d8700 1 --2- 172.21.15.27:0/248942090 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8280f5640 0x7fd8280f5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:40.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.085+0000 7fd82c9d8700 1 -- 172.21.15.27:0/248942090 >> 172.21.15.27:0/248942090 conn(0x7fd8280f0da0 msgr2=0x7fd8280f31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:10:40.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.085+0000 7fd82c9d8700 1 -- 172.21.15.27:0/248942090 shutdown_connections 2021-11-03T17:10:40.090 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.085+0000 7fd82c9d8700 1 -- 172.21.15.27:0/248942090 wait complete. 2021-11-03T17:10:40.090 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.086+0000 7fd82c9d8700 1 Processor -- start 2021-11-03T17:10:40.091 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.086+0000 7fd82c9d8700 1 -- start start 2021-11-03T17:10:40.091 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.086+0000 7fd82c9d8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8280f5640 0x7fd828109c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:10:40.092 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.086+0000 7fd82c9d8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8280f68e0 0x7fd82810c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:10:40.092 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.086+0000 7fd82c9d8700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8280605a0 con 0x7fd8280f68e0 2021-11-03T17:10:40.093 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.086+0000 7fd82c9d8700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8280606e0 con 0x7fd8280f5640 2021-11-03T17:10:40.093 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.086+0000 7fd825d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8280f68e0 0x7fd82810c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:10:40.093 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.086+0000 7fd825d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8280f68e0 0x7fd82810c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54688/0 (socket says 172.21.15.27:54688) 2021-11-03T17:10:40.094 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.086+0000 7fd825d9b700 1 -- 172.21.15.27:0/1216123725 learned_addr learned my addr 172.21.15.27:0/1216123725 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:10:40.094 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.086+0000 7fd82659c700 1 --2- 172.21.15.27:0/1216123725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8280f5640 0x7fd828109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:10:40.095 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.087+0000 7fd825d9b700 1 -- 172.21.15.27:0/1216123725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8280f5640 msgr2=0x7fd828109c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:40.095 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.087+0000 7fd825d9b700 1 --2- 172.21.15.27:0/1216123725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8280f5640 0x7fd828109c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:40.096 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.087+0000 7fd825d9b700 1 -- 172.21.15.27:0/1216123725 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd81c005040 con 0x7fd8280f68e0 2021-11-03T17:10:40.096 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.087+0000 7fd825d9b700 1 --2- 172.21.15.27:0/1216123725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8280f68e0 0x7fd82810c170 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fd81c009830 tx=0x7fd81c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:10:40.097 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.087+0000 7fd816ffd700 1 -- 172.21.15.27:0/1216123725 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd81c018070 con 0x7fd8280f68e0 2021-11-03T17:10:40.097 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.087+0000 7fd816ffd700 1 -- 172.21.15.27:0/1216123725 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7fd81c013410 con 0x7fd8280f68e0 2021-11-03T17:10:40.098 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.087+0000 7fd816ffd700 1 -- 172.21.15.27:0/1216123725 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd81c01dea0 con 0x7fd8280f68e0 2021-11-03T17:10:40.098 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.087+0000 7fd82c9d8700 1 -- 172.21.15.27:0/1216123725 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd82810c6c0 con 0x7fd8280f68e0 2021-11-03T17:10:40.098 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.087+0000 7fd82c9d8700 1 -- 172.21.15.27:0/1216123725 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd82810cbf0 con 0x7fd8280f68e0 2021-11-03T17:10:40.099 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.088+0000 7fd82c9d8700 1 -- 172.21.15.27:0/1216123725 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8280f4a30 con 0x7fd8280f68e0 2021-11-03T17:10:40.099 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.089+0000 7fd816ffd700 1 -- 172.21.15.27:0/1216123725 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fd81c023020 con 0x7fd8280f68e0 2021-11-03T17:10:40.100 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.089+0000 7fd816ffd700 1 --2- 172.21.15.27:0/1216123725 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fd81005c2a0 0x7fd81005e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:10:40.100 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.089+0000 7fd816ffd700 1 -- 172.21.15.27:0/1216123725 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4519+0+0 (secure 0 0 0) 0x7fd81c077070 con 0x7fd8280f68e0 2021-11-03T17:10:40.101 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.089+0000 7fd82659c700 1 --2- 172.21.15.27:0/1216123725 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fd81005c2a0 0x7fd81005e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:10:40.101 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.089+0000 7fd82659c700 1 --2- 172.21.15.27:0/1216123725 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fd81005c2a0 0x7fd81005e760 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fd828060220 tx=0x7fd81800b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:10:40.102 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.093+0000 7fd816ffd700 1 -- 172.21.15.27:0/1216123725 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fd81c04a110 con 0x7fd8280f68e0 2021-11-03T17:10:40.307 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.312+0000 7fd82c9d8700 1 -- 172.21.15.27:0/1216123725 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fd828045ad0 con 0x7fd8280f68e0 2021-11-03T17:10:40.311 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.315+0000 7fd816ffd700 1 -- 172.21.15.27:0/1216123725 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v15) v1 ==== 170+0+59 (secure 0 0 0) 0x7fd81c024340 con 0x7fd8280f68e0 2021-11-03T17:10:40.311 INFO:teuthology.orchestra.run.smithi027.stdout:[client.0] 2021-11-03T17:10:40.312 INFO:teuthology.orchestra.run.smithi027.stdout: key = AQCQwoJhBSS2EhAACxq4RFqKoy6Ub2L6wPLZQQ== 2021-11-03T17:10:40.313 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.317+0000 7fd82c9d8700 1 -- 172.21.15.27:0/1216123725 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fd81005c2a0 msgr2=0x7fd81005e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:40.313 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.317+0000 7fd82c9d8700 1 --2- 172.21.15.27:0/1216123725 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fd81005c2a0 0x7fd81005e760 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fd828060220 tx=0x7fd81800b040).stop 2021-11-03T17:10:40.313 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.317+0000 7fd82c9d8700 1 -- 172.21.15.27:0/1216123725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8280f68e0 msgr2=0x7fd82810c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:40.314 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.317+0000 7fd82c9d8700 1 --2- 172.21.15.27:0/1216123725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8280f68e0 0x7fd82810c170 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7fd81c009830 tx=0x7fd81c009b30).stop 2021-11-03T17:10:40.314 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.317+0000 7fd82c9d8700 1 -- 172.21.15.27:0/1216123725 shutdown_connections 2021-11-03T17:10:40.314 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.317+0000 7fd82c9d8700 1 --2- 172.21.15.27:0/1216123725 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fd81005c2a0 0x7fd81005e760 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:40.314 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.317+0000 7fd82c9d8700 1 --2- 172.21.15.27:0/1216123725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8280f5640 0x7fd828109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:40.315 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.317+0000 7fd82c9d8700 1 --2- 172.21.15.27:0/1216123725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8280f68e0 0x7fd82810c170 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:40.315 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.317+0000 7fd82c9d8700 1 -- 172.21.15.27:0/1216123725 >> 172.21.15.27:0/1216123725 conn(0x7fd8280f0da0 msgr2=0x7fd8280fa330 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:10:40.315 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.317+0000 7fd82c9d8700 1 -- 172.21.15.27:0/1216123725 shutdown_connections 2021-11-03T17:10:40.315 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:40.317+0000 7fd82c9d8700 1 -- 172.21.15.27:0/1216123725 wait complete. 2021-11-03T17:10:40.907 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:40 smithi027 conmon[32322]: audit 2021-11-03T17:10:40.313766+0000 mon.smithi027 (mon. 2021-11-03T17:10:40.907 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:40 smithi027 conmon[32322]: 0) 438 : audit [INF] from='client.? 172.21.15.27:0/1216123725' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2021-11-03T17:10:40.907 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:40 smithi027 conmon[32322]: audit 2021-11-03T17:10:40.317059+0000 mon.smithi027 (mon.0) 439 : audit [INF] from='client.? 172.21.15.27:0/1216123725' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2021-11-03T17:10:40.922 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2021-11-03T17:10:40.923 DEBUG:teuthology.orchestra.run.smithi027:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2021-11-03T17:10:40.923 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2021-11-03T17:10:40.963 DEBUG:teuthology.orchestra.run.smithi115:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2021-11-03T17:10:40.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:40 smithi115 conmon[34670]: audit 2021-11-03T17:10:40.313766+0000 mon.smithi027 (mon.0) 438 : audit [INF] from='client.? 172.21.15.27:0/1216123725' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2021-11-03T17:10:40.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:40 smithi115 conmon[34670]: audit 2021-11-03T17:10:40.317059+0000 mon.smithi027 (mon.0) 439 : audit [INF] from='client.? 172.21.15.27:0/1216123725' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2021-11-03T17:10:41.732 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:41 smithi115 conmon[34670]: cluster 2021-11-03T17:10:40.559819+0000 mgr.smithi027.ufhhvb (mgr.14162) 171 : cluster [DBG] pgmap v142: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:10:42.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:41 smithi027 conmon[32322]: cluster 2021-11-03T17:10:40.559819+0000 mgr.smithi027.ufhhvb (mgr.14162) 171 : cluster [DBG] pgmap v142: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:10:42.657 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:42 smithi115 conmon[34670]: cluster 2021-11-03T17:10:42.560196+0000 mgr.smithi027.ufhhvb (mgr.14162) 172 : cluster 2021-11-03T17:10:42.658 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:42 smithi115 conmon[34670]: [DBG] pgmap v143: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:10:42.799 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:42.803+0000 7f1ad2db7700 1 -- 172.21.15.115:0/1451039156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1acc0f79e0 msgr2=0x7f1acc0f7e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:42.799 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:42.803+0000 7f1ad2db7700 1 --2- 172.21.15.115:0/1451039156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1acc0f79e0 0x7f1acc0f7e40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f1ab8004660 tx=0x7f1ab8009b30).stop 2021-11-03T17:10:42.800 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:42.803+0000 7f1ad2db7700 1 -- 172.21.15.115:0/1451039156 shutdown_connections 2021-11-03T17:10:42.800 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:42.803+0000 7f1ad2db7700 1 --2- 172.21.15.115:0/1451039156 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1acc0f79e0 0x7f1acc0f7e40 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:42.800 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:42.803+0000 7f1ad2db7700 1 --2- 172.21.15.115:0/1451039156 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1acc0f6740 0x7f1acc0f6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:42.800 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:42.803+0000 7f1ad2db7700 1 -- 172.21.15.115:0/1451039156 >> 172.21.15.115:0/1451039156 conn(0x7f1acc0f1e80 msgr2=0x7f1acc0f42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:10:42.801 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:42.803+0000 7f1ad2db7700 1 -- 172.21.15.115:0/1451039156 shutdown_connections 2021-11-03T17:10:42.801 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:42.803+0000 7f1ad2db7700 1 -- 172.21.15.115:0/1451039156 wait complete. 2021-11-03T17:10:42.801 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:42.804+0000 7f1ad2db7700 1 Processor -- start 2021-11-03T17:10:42.801 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:42.804+0000 7f1ad2db7700 1 -- start start 2021-11-03T17:10:42.803 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:42.804+0000 7f1ad2db7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1acc0f6740 0x7f1acc06b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:10:42.804 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:42.804+0000 7f1ad2db7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1acc0f79e0 0x7f1acc069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:10:42.804 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:42.804+0000 7f1ad2db7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc06a380 con 0x7f1acc0f79e0 2021-11-03T17:10:42.804 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:42.804+0000 7f1ad2db7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc06a4c0 con 0x7f1acc0f6740 2021-11-03T17:10:42.804 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:42.804+0000 7f1ad0b53700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1acc0f6740 0x7f1acc06b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:10:42.805 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:42.804+0000 7f1ad0b53700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1acc0f6740 0x7f1acc06b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.115:38806/0 (socket says 172.21.15.115:38806) 2021-11-03T17:10:42.805 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:42.804+0000 7f1ad0b53700 1 -- 172.21.15.115:0/3694043646 learned_addr learned my addr 172.21.15.115:0/3694043646 (peer_addr_for_me v2:172.21.15.115:0/0) 2021-11-03T17:10:42.805 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:42.804+0000 7f1acbfff700 1 --2- 172.21.15.115:0/3694043646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1acc0f79e0 0x7f1acc069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:10:42.805 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:42.804+0000 7f1ad0b53700 1 -- 172.21.15.115:0/3694043646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1acc0f79e0 msgr2=0x7f1acc069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:42.806 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:42.804+0000 7f1ad0b53700 1 --2- 172.21.15.115:0/3694043646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1acc0f79e0 0x7f1acc069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:42.806 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:42.804+0000 7f1ad0b53700 1 -- 172.21.15.115:0/3694043646 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab8005040 con 0x7f1acc0f6740 2021-11-03T17:10:42.806 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:42.804+0000 7f1acbfff700 1 --2- 172.21.15.115:0/3694043646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1acc0f79e0 0x7f1acc069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:10:42.806 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:42.804+0000 7f1ad0b53700 1 --2- 172.21.15.115:0/3694043646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1acc0f6740 0x7f1acc06b9a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f1ac0002700 tx=0x7f1ac0009d80).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:10:42.807 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:42.805+0000 7f1ac97fa700 1 -- 172.21.15.115:0/3694043646 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ac0009320 con 0x7f1acc0f6740 2021-11-03T17:10:42.807 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:42.805+0000 7f1ad2db7700 1 -- 172.21.15.115:0/3694043646 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1acc06a7a0 con 0x7f1acc0f6740 2021-11-03T17:10:42.807 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:42.805+0000 7f1ac97fa700 1 -- 172.21.15.115:0/3694043646 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f1ac000f040 con 0x7f1acc0f6740 2021-11-03T17:10:42.807 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:42.805+0000 7f1ac97fa700 1 -- 172.21.15.115:0/3694043646 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ac0014860 con 0x7f1acc0f6740 2021-11-03T17:10:42.807 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:42.805+0000 7f1ad2db7700 1 -- 172.21.15.115:0/3694043646 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1acc10ef80 con 0x7f1acc0f6740 2021-11-03T17:10:42.808 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:42.806+0000 7f1ad2db7700 1 -- 172.21.15.115:0/3694043646 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1acc045ad0 con 0x7f1acc0f6740 2021-11-03T17:10:42.808 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:42.806+0000 7f1ac97fa700 1 -- 172.21.15.115:0/3694043646 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f1ac0009480 con 0x7f1acc0f6740 2021-11-03T17:10:42.808 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:42.806+0000 7f1ac97fa700 1 --2- 172.21.15.115:0/3694043646 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f1abc05c2b0 0x7f1abc05e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:10:42.808 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:42.806+0000 7f1ac97fa700 1 -- 172.21.15.115:0/3694043646 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4519+0+0 (secure 0 0 0) 0x7f1ac0075e70 con 0x7f1acc0f6740 2021-11-03T17:10:42.809 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:42.807+0000 7f1acbfff700 1 --2- 172.21.15.115:0/3694043646 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f1abc05c2b0 0x7f1abc05e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:10:42.809 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:42.807+0000 7f1acbfff700 1 --2- 172.21.15.115:0/3694043646 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f1abc05c2b0 0x7f1abc05e770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f1ab8004e30 tx=0x7f1ab8009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:10:42.809 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:42.810+0000 7f1ac97fa700 1 -- 172.21.15.115:0/3694043646 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f1ac0048e90 con 0x7f1acc0f6740 2021-11-03T17:10:42.937 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:42 smithi027 conmon[32322]: cluster 2021-11-03T17:10:42.560196+0000 mgr.smithi027.ufhhvb (mgr.14162) 172 : cluster [DBG] pgmap v143: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:10:43.009 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:43.013+0000 7f1ad2db7700 1 -- 172.21.15.115:0/3694043646 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f1acc027a00 con 0x7f1acc0f6740 2021-11-03T17:10:43.013 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:43.018+0000 7f1ac97fa700 1 -- 172.21.15.115:0/3694043646 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7f1ac004c4b0 con 0x7f1acc0f6740 2021-11-03T17:10:43.014 INFO:teuthology.orchestra.run.smithi115.stdout:[client.1] 2021-11-03T17:10:43.014 INFO:teuthology.orchestra.run.smithi115.stdout: key = AQCTwoJhZEXvABAAIGWv/W+zYbad2phRElL42A== 2021-11-03T17:10:43.015 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:43.019+0000 7f1ad2db7700 1 -- 172.21.15.115:0/3694043646 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f1abc05c2b0 msgr2=0x7f1abc05e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:43.015 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:43.019+0000 7f1ad2db7700 1 --2- 172.21.15.115:0/3694043646 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f1abc05c2b0 0x7f1abc05e770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f1ab8004e30 tx=0x7f1ab8009b30).stop 2021-11-03T17:10:43.016 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:43.019+0000 7f1ad2db7700 1 -- 172.21.15.115:0/3694043646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1acc0f6740 msgr2=0x7f1acc06b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:43.016 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:43.019+0000 7f1ad2db7700 1 --2- 172.21.15.115:0/3694043646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1acc0f6740 0x7f1acc06b9a0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f1ac0002700 tx=0x7f1ac0009d80).stop 2021-11-03T17:10:43.016 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:43.019+0000 7f1ad2db7700 1 -- 172.21.15.115:0/3694043646 shutdown_connections 2021-11-03T17:10:43.016 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:43.019+0000 7f1ad2db7700 1 --2- 172.21.15.115:0/3694043646 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f1abc05c2b0 0x7f1abc05e770 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:43.016 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:43.019+0000 7f1ad2db7700 1 --2- 172.21.15.115:0/3694043646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1acc0f6740 0x7f1acc06b9a0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:43.017 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:43.019+0000 7f1ad2db7700 1 --2- 172.21.15.115:0/3694043646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1acc0f79e0 0x7f1acc069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:43.017 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:43.019+0000 7f1ad2db7700 1 -- 172.21.15.115:0/3694043646 >> 172.21.15.115:0/3694043646 conn(0x7f1acc0f1e80 msgr2=0x7f1acc0fb430 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:10:43.017 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:43.019+0000 7f1ad2db7700 1 -- 172.21.15.115:0/3694043646 shutdown_connections 2021-11-03T17:10:43.017 INFO:teuthology.orchestra.run.smithi115.stderr:2021-11-03T17:10:43.020+0000 7f1ad2db7700 1 -- 172.21.15.115:0/3694043646 wait complete. 2021-11-03T17:10:43.651 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2021-11-03T17:10:43.651 DEBUG:teuthology.orchestra.run.smithi115:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2021-11-03T17:10:43.652 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2021-11-03T17:10:43.728 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph osd dump --format=json 2021-11-03T17:10:43.956 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:43 smithi115 conmon[34670]: audit 2021-11-03T17:10:43.014905+0000 mon.smithi115 (mon.1) 14 2021-11-03T17:10:43.956 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:43 smithi115 conmon[34670]: : audit [INF] from='client.? 172.21.15.115:0/3694043646' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2021-11-03T17:10:43.956 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:43 smithi115 conmon[34670]: audit 2021-11-03T17:10:43.015494+0000 mon.smithi027 (mon.0) 440 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2021-11-03T17:10:43.956 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:43 smithi115 conmon[34670]: audit 2021- 2021-11-03T17:10:43.957 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:43 smithi115 conmon[34670]: 11-03T17:10:43.019365+0000 mon.smithi027 (mon.0) 441 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2021-11-03T17:10:44.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:43 smithi027 conmon[32322]: audit 2021-11-03T17:10:43.014905+0000 mon.smithi115 (mon.1) 14 : audit 2021-11-03T17:10:44.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:43 smithi027 conmon[32322]: [INF] from='client.? 172.21.15.115:0/3694043646' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2021-11-03T17:10:44.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:43 smithi027 conmon[32322]: audit 2021-11-03T17:10:43.015494+0000 mon.smithi027 (mon.0) 440 : audit [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2021-11-03T17:10:44.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:43 smithi027 conmon[32322]: audit 2021-11-03T17: 2021-11-03T17:10:44.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:43 smithi027 conmon[32322]: 10:43.019365+0000 mon.smithi027 (mon.0) 441 : audit [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2021-11-03T17:10:44.029 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c/mon.smithi027/config 2021-11-03T17:10:44.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:44 smithi115 conmon[34670]: cluster 2021-11-03T17:10:44.560509+0000 mgr.smithi027.ufhhvb (mgr.14162) 173 : cluster [DBG] pgmap v144: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:10:45.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:44 smithi027 conmon[32322]: cluster 2021-11-03T17:10:44.560509+0000 mgr.smithi027.ufhhvb (mgr.14162) 173 : cluster [DBG] pgmap v144: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:10:45.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.678+0000 7fab02dd2700 1 -- 172.21.15.27:0/1615720851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faafc060810 msgr2=0x7faafc0fb820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:45.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.678+0000 7fab02dd2700 1 --2- 172.21.15.27:0/1615720851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faafc060810 0x7faafc0fb820 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7faaf0004660 tx=0x7faaf0009b30).stop 2021-11-03T17:10:45.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.678+0000 7fab02dd2700 1 -- 172.21.15.27:0/1615720851 shutdown_connections 2021-11-03T17:10:45.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.678+0000 7fab02dd2700 1 --2- 172.21.15.27:0/1615720851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faafc0fbd10 0x7faafc0fe100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:45.676 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.678+0000 7fab02dd2700 1 --2- 172.21.15.27:0/1615720851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faafc060810 0x7faafc0fb820 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:45.676 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.678+0000 7fab02dd2700 1 -- 172.21.15.27:0/1615720851 >> 172.21.15.27:0/1615720851 conn(0x7faafc0f0e20 msgr2=0x7faafc0f3260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:10:45.676 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.678+0000 7fab02dd2700 1 -- 172.21.15.27:0/1615720851 shutdown_connections 2021-11-03T17:10:45.676 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.678+0000 7fab02dd2700 1 -- 172.21.15.27:0/1615720851 wait complete. 2021-11-03T17:10:45.677 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.678+0000 7fab02dd2700 1 Processor -- start 2021-11-03T17:10:45.677 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.678+0000 7fab02dd2700 1 -- start start 2021-11-03T17:10:45.677 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.678+0000 7fab02dd2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faafc060810 0x7faafc109260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:10:45.677 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.678+0000 7fab02dd2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faafc0fbd10 0x7faafc109750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:10:45.678 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.678+0000 7fab02dd2700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faafc109d00 con 0x7faafc060810 2021-11-03T17:10:45.678 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.678+0000 7fab02dd2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faafc109e40 con 0x7faafc0fbd10 2021-11-03T17:10:45.678 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.678+0000 7fab00b6e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faafc060810 0x7faafc109260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:10:45.678 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.678+0000 7fab00b6e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faafc060810 0x7faafc109260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54694/0 (socket says 172.21.15.27:54694) 2021-11-03T17:10:45.678 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.678+0000 7fab00b6e700 1 -- 172.21.15.27:0/1472910999 learned_addr learned my addr 172.21.15.27:0/1472910999 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:10:45.679 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.678+0000 7faafbfff700 1 --2- 172.21.15.27:0/1472910999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faafc0fbd10 0x7faafc109750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:10:45.679 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.678+0000 7fab00b6e700 1 -- 172.21.15.27:0/1472910999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faafc0fbd10 msgr2=0x7faafc109750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:45.679 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.678+0000 7fab00b6e700 1 --2- 172.21.15.27:0/1472910999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faafc0fbd10 0x7faafc109750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:45.679 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.678+0000 7fab00b6e700 1 -- 172.21.15.27:0/1472910999 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaf0005040 con 0x7faafc060810 2021-11-03T17:10:45.680 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.678+0000 7faafbfff700 1 --2- 172.21.15.27:0/1472910999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faafc0fbd10 0x7faafc109750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:10:45.680 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.678+0000 7fab00b6e700 1 --2- 172.21.15.27:0/1472910999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faafc060810 0x7faafc109260 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7faaf0009990 tx=0x7faaf0009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:10:45.680 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.678+0000 7faaf97fa700 1 -- 172.21.15.27:0/1472910999 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faaf0018070 con 0x7faafc060810 2021-11-03T17:10:45.680 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.678+0000 7faaf97fa700 1 -- 172.21.15.27:0/1472910999 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7faaf0013410 con 0x7faafc060810 2021-11-03T17:10:45.681 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.678+0000 7faaf97fa700 1 -- 172.21.15.27:0/1472910999 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faaf001c620 con 0x7faafc060810 2021-11-03T17:10:45.681 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.678+0000 7fab02dd2700 1 -- 172.21.15.27:0/1472910999 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faafc10a780 con 0x7faafc060810 2021-11-03T17:10:45.681 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.678+0000 7fab02dd2700 1 -- 172.21.15.27:0/1472910999 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faafc10ace0 con 0x7faafc060810 2021-11-03T17:10:45.682 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.679+0000 7fab02dd2700 1 -- 172.21.15.27:0/1472910999 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faafc045ad0 con 0x7faafc060810 2021-11-03T17:10:45.683 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.679+0000 7faaf97fa700 1 -- 172.21.15.27:0/1472910999 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7faaf001d420 con 0x7faafc060810 2021-11-03T17:10:45.683 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.680+0000 7faaf97fa700 1 --2- 172.21.15.27:0/1472910999 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7faaec05c2a0 0x7faaec05e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:10:45.683 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.680+0000 7faaf97fa700 1 -- 172.21.15.27:0/1472910999 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4519+0+0 (secure 0 0 0) 0x7faaf0077a10 con 0x7faafc060810 2021-11-03T17:10:45.683 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.680+0000 7faafbfff700 1 --2- 172.21.15.27:0/1472910999 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7faaec05c2a0 0x7faaec05e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:10:45.683 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.680+0000 7faafbfff700 1 --2- 172.21.15.27:0/1472910999 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7faaec05c2a0 0x7faaec05e760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7faafc0605e0 tx=0x7faae8009040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:10:45.684 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.685+0000 7faaf97fa700 1 -- 172.21.15.27:0/1472910999 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7faaf004aab0 con 0x7faafc060810 2021-11-03T17:10:45.839 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.843+0000 7fab02dd2700 1 -- 172.21.15.27:0/1472910999 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faafc027a00 con 0x7faafc060810 2021-11-03T17:10:45.840 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.844+0000 7faaf97fa700 1 -- 172.21.15.27:0/1472910999 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+10887 (secure 0 0 0) 0x7faaf004e0d0 con 0x7faafc060810 2021-11-03T17:10:45.840 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:10:45.840 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":39,"fsid":"32ab977e-3cc8-11ec-8c28-001a4aab830c","created":"2021-11-03T17:05:46.314150+0000","modified":"2021-11-03T17:10:35.584298+0000","last_up_change":"2021-11-03T17:10:34.577218+0000","last_in_change":"2021-11-03T17:10:16.846149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-03T17:09:48.488849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"2f6bbb8f-4e3c-401a-b1bd-69ba9cdd9d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3126282997},{"type":"v1","addr":"172.21.15.27:6803","nonce":3126282997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3126282997},{"type":"v1","addr":"172.21.15.27:6805","nonce":3126282997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":3126282997},{"type":"v1","addr":"172.21.15.27:6809","nonce":3126282997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3126282997},{"type":"v1","addr":"172.21.15.27:6807","nonce":3126282997}]},"public_addr":"172.21.15.27:6803/3126282997","cluster_addr":"172.21.15.27:6805/3126282997","heartbeat_back_addr":"172.21.15.27:6809/3126282997","heartbeat_front_addr":"172.21.15.27:6807/3126282997","state":["exists","up"]},{"osd":1,"uuid":"0942b897-5aa3-461a-a2d6-943d5739d963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2723402359},{"type":"v1","addr":"172.21.15.27:6811","nonce":2723402359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2723402359},{"type":"v1","addr":"172.21.15.27:6813","nonce":2723402359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2723402359},{"type":"v1","addr":"172.21.15.27:6817","nonce":2723402359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2723402359},{"type":"v1","addr":"172.21.15.27:6815","nonce":2723402359}]},"public_addr":"172.21.15.27:6811/2723402359","cluster_addr":"172.21.15.27:6813/2723402359","heartbeat_back_addr":"172.21.15.27:6817/2723402359","heartbeat_front_addr":"172.21.15.27:6815/2723402359","state":["exists","up"]},{"osd":2,"uuid":"3ea602d0-3e10-4f30-96cd-288e53a7a00a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":329182726},{"type":"v1","addr":"172.21.15.27:6819","nonce":329182726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":329182726},{"type":"v1","addr":"172.21.15.27:6821","nonce":329182726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":329182726},{"type":"v1","addr":"172.21.15.27:6825","nonce":329182726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":329182726},{"type":"v1","addr":"172.21.15.27:6823","nonce":329182726}]},"public_addr":"172.21.15.27:6819/329182726","cluster_addr":"172.21.15.27:6821/329182726","heartbeat_back_addr":"172.21.15.27:6825/329182726","heartbeat_front_addr":"172.21.15.27:6823/329182726","state":["exists","up"]},{"osd":3,"uuid":"c517d190-0d21-4db0-994c-5f971d67f103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":4026713633},{"type":"v1","addr":"172.21.15.115:6801","nonce":4026713633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":4026713633},{"type":"v1","addr":"172.21.15.115:6803","nonce":4026713633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":4026713633},{"type":"v1","addr":"172.21.15.115:6807","nonce":4026713633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":4026713633},{"type":"v1","addr":"172.21.15.115:6805","nonce":4026713633}]},"public_addr":"172.21.15.115:6801/4026713633","cluster_addr":"172.21.15.115:6803/4026713633","heartbeat_back_addr":"172.21.15.115:6807/4026713633","heartbeat_front_addr":"172.21.15.115:6805/4026713633","state":["exists","up"]},{"osd":4,"uuid":"aeeb2c74-919d-4ac7-8c7a-1dd273225f59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1601394331},{"type":"v1","addr":"172.21.15.115:6809","nonce":1601394331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1601394331},{"type":"v1","addr":"172.21.15.115:6811","nonce":1601394331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1601394331},{"type":"v1","addr":"172.21.15.115:6815","nonce":1601394331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1601394331},{"type":"v1","addr":"172.21.15.115:6813","nonce":1601394331}]},"public_addr":"172.21.15.115:6809/1601394331","cluster_addr":"172.21.15.115:6811/1601394331","heartbeat_back_addr":"172.21.15.115:6815/1601394331","heartbeat_front_addr":"172.21.15.115:6813/1601394331","state":["exists","up"]},{"osd":5,"uuid":"047cfc7d-01d6-4859-8dc8-01d3811b9060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":2350455075},{"type":"v1","addr":"172.21.15.115:6817","nonce":2350455075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":2350455075},{"type":"v1","addr":"172.21.15.115:6819","nonce":2350455075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":2350455075},{"type":"v1","addr":"172.21.15.115:6823","nonce":2350455075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":2350455075},{"type":"v1","addr":"172.21.15.115:6821","nonce":2350455075}]},"public_addr":"172.21.15.115:6817/2350455075","cluster_addr":"172.21.15.115:6819/2350455075","heartbeat_back_addr":"172.21.15.115:6823/2350455075","heartbeat_front_addr":"172.21.15.115:6821/2350455075","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-11-03T17:08:50.764082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-11-03T17:09:12.720566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-11-03T17:09:34.450930+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-11-03T17:09:53.016884+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-11-03T17:10:12.728118+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-11-03T17:10:32.668302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6801/3506673123":"2021-11-04T17:06:51.527604+0000","172.21.15.27:6800/3506673123":"2021-11-04T17:06:51.527604+0000","172.21.15.27:0/92526005":"2021-11-04T17:06:51.527604+0000","172.21.15.27:0/1091817081":"2021-11-04T17:06:51.527604+0000","172.21.15.27:0/3225036173":"2021-11-04T17:06:34.410322+0000","172.21.15.27:6801/774407793":"2021-11-04T17:06:34.410322+0000","172.21.15.27:6800/774407793":"2021-11-04T17:06:34.410322+0000","172.21.15.27:0/3947386121":"2021-11-04T17:06:09.549455+0000","172.21.15.27:6800/388626148":"2021-11-04T17:06:09.549455+0000","172.21.15.27:0/2008079603":"2021-11-04T17:06:09.549455+0000","172.21.15.27:0/2202850722":"2021-11-04T17:06:34.410322+0000","172.21.15.27:6801/388626148":"2021-11-04T17:06:09.549455+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-11-03T17:10:45.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.845+0000 7fab02dd2700 1 -- 172.21.15.27:0/1472910999 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7faaec05c2a0 msgr2=0x7faaec05e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:45.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.845+0000 7fab02dd2700 1 --2- 172.21.15.27:0/1472910999 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7faaec05c2a0 0x7faaec05e760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7faafc0605e0 tx=0x7faae8009040).stop 2021-11-03T17:10:45.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.845+0000 7fab02dd2700 1 -- 172.21.15.27:0/1472910999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faafc060810 msgr2=0x7faafc109260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:45.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.845+0000 7fab02dd2700 1 --2- 172.21.15.27:0/1472910999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faafc060810 0x7faafc109260 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7faaf0009990 tx=0x7faaf0009b30).stop 2021-11-03T17:10:45.843 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.845+0000 7fab02dd2700 1 -- 172.21.15.27:0/1472910999 shutdown_connections 2021-11-03T17:10:45.843 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.845+0000 7fab02dd2700 1 --2- 172.21.15.27:0/1472910999 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7faaec05c2a0 0x7faaec05e760 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:45.843 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.846+0000 7fab02dd2700 1 --2- 172.21.15.27:0/1472910999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faafc060810 0x7faafc109260 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:45.843 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.846+0000 7fab02dd2700 1 --2- 172.21.15.27:0/1472910999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faafc0fbd10 0x7faafc109750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:45.843 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.846+0000 7fab02dd2700 1 -- 172.21.15.27:0/1472910999 >> 172.21.15.27:0/1472910999 conn(0x7faafc0f0e20 msgr2=0x7faafc0f3260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:10:45.844 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.846+0000 7fab02dd2700 1 -- 172.21.15.27:0/1472910999 shutdown_connections 2021-11-03T17:10:45.844 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:45.846+0000 7fab02dd2700 1 -- 172.21.15.27:0/1472910999 wait complete. 2021-11-03T17:10:46.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:45 smithi115 conmon[34670]: audit 2021-11-03T17:10:45.845172+0000 mon.smithi027 (mon.0) 442 : audit [DBG] from='client.? 172.21.15.27:0/1472910999' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2021-11-03T17:10:46.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:45 smithi027 conmon[32322]: audit 2021-11-03T17:10:45.845172+0000 mon.smithi027 (mon.0) 442 : audit 2021-11-03T17:10:46.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:45 smithi027 conmon[32322]: [DBG] from='client.? 172.21.15.27:0/1472910999' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2021-11-03T17:10:46.424 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-03T17:09:48.488849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}] 2021-11-03T17:10:46.424 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph osd pool get device_health_metrics pg_num 2021-11-03T17:10:46.717 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c/mon.smithi027/config 2021-11-03T17:10:47.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:47 smithi027 conmon[32322]: cluster 2021-11-03T17:10:46.560856+0000 mgr.smithi027.ufhhvb (mgr.14162) 174 : cluster [DBG] pgmap v145: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:10:47.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:47 smithi115 conmon[34670]: cluster 2021-11-03T17:10:46.560856+0000 mgr.smithi027.ufhhvb (mgr.14162) 174 : cluster [DBG] pgmap v145: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:10:48.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.446+0000 7fe51719f700 1 -- 172.21.15.27:0/2075690797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5100f7a80 msgr2=0x7fe5100f7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:48.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.446+0000 7fe51719f700 1 --2- 172.21.15.27:0/2075690797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5100f7a80 0x7fe5100f7ee0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7fe4fc004660 tx=0x7fe4fc009b30).stop 2021-11-03T17:10:48.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.447+0000 7fe51719f700 1 -- 172.21.15.27:0/2075690797 shutdown_connections 2021-11-03T17:10:48.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.447+0000 7fe51719f700 1 --2- 172.21.15.27:0/2075690797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5100f7a80 0x7fe5100f7ee0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:48.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.447+0000 7fe51719f700 1 --2- 172.21.15.27:0/2075690797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5100f67e0 0x7fe5100f6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:48.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.447+0000 7fe51719f700 1 -- 172.21.15.27:0/2075690797 >> 172.21.15.27:0/2075690797 conn(0x7fe5100f1f20 msgr2=0x7fe5100f4360 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:10:48.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.447+0000 7fe51719f700 1 -- 172.21.15.27:0/2075690797 shutdown_connections 2021-11-03T17:10:48.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.447+0000 7fe51719f700 1 -- 172.21.15.27:0/2075690797 wait complete. 2021-11-03T17:10:48.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.447+0000 7fe51719f700 1 Processor -- start 2021-11-03T17:10:48.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.447+0000 7fe51719f700 1 -- start start 2021-11-03T17:10:48.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.447+0000 7fe51719f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5100f67e0 0x7fe51006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:10:48.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.447+0000 7fe51719f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5100f7a80 0x7fe510069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:10:48.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.447+0000 7fe51719f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51006a380 con 0x7fe5100f67e0 2021-11-03T17:10:48.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.447+0000 7fe51719f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51006a4c0 con 0x7fe5100f7a80 2021-11-03T17:10:48.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.448+0000 7fe514f3b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5100f67e0 0x7fe51006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:10:48.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.448+0000 7fe514f3b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5100f67e0 0x7fe51006b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54704/0 (socket says 172.21.15.27:54704) 2021-11-03T17:10:48.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.448+0000 7fe514f3b700 1 -- 172.21.15.27:0/2432651801 learned_addr learned my addr 172.21.15.27:0/2432651801 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:10:48.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.448+0000 7fe50ffff700 1 --2- 172.21.15.27:0/2432651801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5100f7a80 0x7fe510069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:10:48.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.448+0000 7fe514f3b700 1 -- 172.21.15.27:0/2432651801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5100f7a80 msgr2=0x7fe510069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:48.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.448+0000 7fe514f3b700 1 --2- 172.21.15.27:0/2432651801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5100f7a80 0x7fe510069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:48.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.448+0000 7fe514f3b700 1 -- 172.21.15.27:0/2432651801 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4fc005040 con 0x7fe5100f67e0 2021-11-03T17:10:48.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.448+0000 7fe514f3b700 1 --2- 172.21.15.27:0/2432651801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5100f67e0 0x7fe51006b9a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7fe504002700 tx=0x7fe504009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:10:48.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.448+0000 7fe50d7fa700 1 -- 172.21.15.27:0/2432651801 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5040079c0 con 0x7fe5100f67e0 2021-11-03T17:10:48.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.448+0000 7fe50d7fa700 1 -- 172.21.15.27:0/2432651801 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7fe50400f040 con 0x7fe5100f67e0 2021-11-03T17:10:48.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.448+0000 7fe51719f700 1 -- 172.21.15.27:0/2432651801 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe51006a7a0 con 0x7fe5100f67e0 2021-11-03T17:10:48.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.448+0000 7fe50d7fa700 1 -- 172.21.15.27:0/2432651801 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe504014860 con 0x7fe5100f67e0 2021-11-03T17:10:48.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.448+0000 7fe51719f700 1 -- 172.21.15.27:0/2432651801 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe51010f060 con 0x7fe5100f67e0 2021-11-03T17:10:48.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.449+0000 7fe51719f700 1 -- 172.21.15.27:0/2432651801 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe510045ad0 con 0x7fe5100f67e0 2021-11-03T17:10:48.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.449+0000 7fe50d7fa700 1 -- 172.21.15.27:0/2432651801 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fe50401b060 con 0x7fe5100f67e0 2021-11-03T17:10:48.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.450+0000 7fe50d7fa700 1 --2- 172.21.15.27:0/2432651801 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fe50005c2a0 0x7fe50005e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:10:48.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.450+0000 7fe50d7fa700 1 -- 172.21.15.27:0/2432651801 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4519+0+0 (secure 0 0 0) 0x7fe504075900 con 0x7fe5100f67e0 2021-11-03T17:10:48.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.450+0000 7fe50ffff700 1 --2- 172.21.15.27:0/2432651801 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fe50005c2a0 0x7fe50005e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:10:48.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.450+0000 7fe50ffff700 1 --2- 172.21.15.27:0/2432651801 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fe50005c2a0 0x7fe50005e760 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fe4fc005010 tx=0x7fe4fc009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:10:48.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.454+0000 7fe50d7fa700 1 -- 172.21.15.27:0/2432651801 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fe5040489a0 con 0x7fe5100f67e0 2021-11-03T17:10:48.606 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.611+0000 7fe51719f700 1 -- 172.21.15.27:0/2432651801 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fe5100ff0a0 con 0x7fe5100f67e0 2021-11-03T17:10:48.607 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.611+0000 7fe50d7fa700 1 -- 172.21.15.27:0/2432651801 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v39) v1 ==== 110+0+10 (secure 0 0 0) 0x7fe50404bfc0 con 0x7fe5100f67e0 2021-11-03T17:10:48.607 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2021-11-03T17:10:48.608 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.613+0000 7fe51719f700 1 -- 172.21.15.27:0/2432651801 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fe50005c2a0 msgr2=0x7fe50005e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:48.608 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.613+0000 7fe51719f700 1 --2- 172.21.15.27:0/2432651801 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fe50005c2a0 0x7fe50005e760 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fe4fc005010 tx=0x7fe4fc009b30).stop 2021-11-03T17:10:48.608 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.613+0000 7fe51719f700 1 -- 172.21.15.27:0/2432651801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5100f67e0 msgr2=0x7fe51006b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:48.609 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.613+0000 7fe51719f700 1 --2- 172.21.15.27:0/2432651801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5100f67e0 0x7fe51006b9a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7fe504002700 tx=0x7fe504009d80).stop 2021-11-03T17:10:48.609 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.613+0000 7fe51719f700 1 -- 172.21.15.27:0/2432651801 shutdown_connections 2021-11-03T17:10:48.609 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.613+0000 7fe51719f700 1 --2- 172.21.15.27:0/2432651801 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fe50005c2a0 0x7fe50005e760 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:48.609 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.613+0000 7fe51719f700 1 --2- 172.21.15.27:0/2432651801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe5100f67e0 0x7fe51006b9a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:48.610 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.613+0000 7fe51719f700 1 --2- 172.21.15.27:0/2432651801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe5100f7a80 0x7fe510069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:48.610 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.613+0000 7fe51719f700 1 -- 172.21.15.27:0/2432651801 >> 172.21.15.27:0/2432651801 conn(0x7fe5100f1f20 msgr2=0x7fe5100fb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:10:48.610 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.613+0000 7fe51719f700 1 -- 172.21.15.27:0/2432651801 shutdown_connections 2021-11-03T17:10:48.610 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:48.613+0000 7fe51719f700 1 -- 172.21.15.27:0/2432651801 wait complete. 2021-11-03T17:10:49.187 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2021-11-03T17:10:49.188 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2021-11-03T17:10:49.188 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph mgr dump --format=json 2021-11-03T17:10:49.478 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c/mon.smithi027/config 2021-11-03T17:10:49.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:49 smithi115 conmon[34670]: cluster 2021-11-03T17:10:48.561160+0000 mgr.smithi027.ufhhvb (mgr.14162) 175 : cluster [DBG] pgmap v146: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:10:49.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:49 smithi115 conmon[34670]: audit 2021-11-03T17:10:48.612257+0000 mon.smithi027 (mon.0) 443 : audit [DBG] from='client.? 172.21.15.27:0/2432651801' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2021-11-03T17:10:50.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:49 smithi027 conmon[32322]: cluster 2021-11-03T17:10:48.561160+0000 2021-11-03T17:10:50.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:49 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14162) 175 : cluster [DBG] pgmap v146: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:10:50.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:49 smithi027 conmon[32322]: audit 2021-11-03T17:10:48.612257+0000 mon.smithi027 (mon.0) 443 : audit [DBG] from='client.? 172.21.15.27:0/2432651801' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2021-11-03T17:10:51.049 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.052+0000 7f5d32cd9700 1 -- 172.21.15.27:0/2725162436 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d2c060810 msgr2=0x7f5d2c0f9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:51.049 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.052+0000 7f5d32cd9700 1 --2- 172.21.15.27:0/2725162436 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d2c060810 0x7f5d2c0f9400 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f5d240099e0 tx=0x7f5d24004d10).stop 2021-11-03T17:10:51.049 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.052+0000 7f5d32cd9700 1 -- 172.21.15.27:0/2725162436 shutdown_connections 2021-11-03T17:10:51.049 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.052+0000 7f5d32cd9700 1 --2- 172.21.15.27:0/2725162436 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d2c0f9980 0x7f5d2c0fbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:51.050 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.052+0000 7f5d32cd9700 1 --2- 172.21.15.27:0/2725162436 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d2c060810 0x7f5d2c0f9400 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:51.050 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.052+0000 7f5d32cd9700 1 -- 172.21.15.27:0/2725162436 >> 172.21.15.27:0/2725162436 conn(0x7f5d2c0f0e20 msgr2=0x7f5d2c0f3240 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:10:51.050 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.052+0000 7f5d32cd9700 1 -- 172.21.15.27:0/2725162436 shutdown_connections 2021-11-03T17:10:51.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.052+0000 7f5d32cd9700 1 -- 172.21.15.27:0/2725162436 wait complete. 2021-11-03T17:10:51.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.053+0000 7f5d32cd9700 1 Processor -- start 2021-11-03T17:10:51.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.053+0000 7f5d32cd9700 1 -- start start 2021-11-03T17:10:51.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.053+0000 7f5d32cd9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d2c060810 0x7f5d2c0f6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:10:51.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.053+0000 7f5d32cd9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d2c0f9980 0x7f5d2c0f5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:10:51.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.053+0000 7f5d32cd9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c0f5860 con 0x7f5d2c0f9980 2021-11-03T17:10:51.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.053+0000 7f5d32cd9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d2c0f59a0 con 0x7f5d2c060810 2021-11-03T17:10:51.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.053+0000 7f5d2bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d2c0f9980 0x7f5d2c0f5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:10:51.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.053+0000 7f5d2bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d2c0f9980 0x7f5d2c0f5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54716/0 (socket says 172.21.15.27:54716) 2021-11-03T17:10:51.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.053+0000 7f5d2bfff700 1 -- 172.21.15.27:0/3860674703 learned_addr learned my addr 172.21.15.27:0/3860674703 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:10:51.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.053+0000 7f5d2bfff700 1 -- 172.21.15.27:0/3860674703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d2c060810 msgr2=0x7f5d2c0f6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:51.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.053+0000 7f5d2bfff700 1 --2- 172.21.15.27:0/3860674703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d2c060810 0x7f5d2c0f6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:10:51.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.053+0000 7f5d2bfff700 1 -- 172.21.15.27:0/3860674703 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d24009640 con 0x7f5d2c0f9980 2021-11-03T17:10:51.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.053+0000 7f5d2bfff700 1 --2- 172.21.15.27:0/3860674703 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d2c0f9980 0x7f5d2c0f5300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f5d20002700 tx=0x7f5d20009d60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:10:51.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.054+0000 7f5d297fa700 1 -- 172.21.15.27:0/3860674703 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d20009320 con 0x7f5d2c0f9980 2021-11-03T17:10:51.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.054+0000 7f5d297fa700 1 -- 172.21.15.27:0/3860674703 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f5d2000f040 con 0x7f5d2c0f9980 2021-11-03T17:10:51.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.054+0000 7f5d32cd9700 1 -- 172.21.15.27:0/3860674703 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d2c0f5c80 con 0x7f5d2c0f9980 2021-11-03T17:10:51.055 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.054+0000 7f5d297fa700 1 -- 172.21.15.27:0/3860674703 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5d20014740 con 0x7f5d2c0f9980 2021-11-03T17:10:51.055 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.054+0000 7f5d32cd9700 1 -- 172.21.15.27:0/3860674703 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d2c10f1e0 con 0x7f5d2c0f9980 2021-11-03T17:10:51.056 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.055+0000 7f5d32cd9700 1 -- 172.21.15.27:0/3860674703 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d2c045ad0 con 0x7f5d2c0f9980 2021-11-03T17:10:51.056 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.055+0000 7f5d297fa700 1 -- 172.21.15.27:0/3860674703 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f5d200077b0 con 0x7f5d2c0f9980 2021-11-03T17:10:51.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.055+0000 7f5d297fa700 1 --2- 172.21.15.27:0/3860674703 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f5d1405c2a0 0x7f5d1405e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:10:51.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.055+0000 7f5d297fa700 1 -- 172.21.15.27:0/3860674703 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4519+0+0 (secure 0 0 0) 0x7f5d200760a0 con 0x7f5d2c0f9980 2021-11-03T17:10:51.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.055+0000 7f5d30a75700 1 --2- 172.21.15.27:0/3860674703 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f5d1405c2a0 0x7f5d1405e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:10:51.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.056+0000 7f5d30a75700 1 --2- 172.21.15.27:0/3860674703 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f5d1405c2a0 0x7f5d1405e760 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f5d24000f80 tx=0x7f5d24004d10).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:10:51.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.059+0000 7f5d297fa700 1 -- 172.21.15.27:0/3860674703 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f5d200490c0 con 0x7f5d2c0f9980 2021-11-03T17:10:51.248 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.254+0000 7f5d32cd9700 1 -- 172.21.15.27:0/3860674703 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f5d2c027a00 con 0x7f5d2c0f9980 2021-11-03T17:10:51.251 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.256+0000 7f5d297fa700 1 -- 172.21.15.27:0/3860674703 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v18) v1 ==== 74+0+143498 (secure 0 0 0) 0x7f5d2004c6e0 con 0x7f5d2c0f9980 2021-11-03T17:10:51.254 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:10:51.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.259+0000 7f5d32cd9700 1 -- 172.21.15.27:0/3860674703 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f5d1405c2a0 msgr2=0x7f5d1405e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:51.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.259+0000 7f5d32cd9700 1 --2- 172.21.15.27:0/3860674703 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f5d1405c2a0 0x7f5d1405e760 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f5d24000f80 tx=0x7f5d24004d10).stop 2021-11-03T17:10:51.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.259+0000 7f5d32cd9700 1 -- 172.21.15.27:0/3860674703 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d2c0f9980 msgr2=0x7f5d2c0f5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:51.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.259+0000 7f5d32cd9700 1 --2- 172.21.15.27:0/3860674703 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d2c0f9980 0x7f5d2c0f5300 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f5d20002700 tx=0x7f5d20009d60).stop 2021-11-03T17:10:51.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.259+0000 7f5d32cd9700 1 -- 172.21.15.27:0/3860674703 shutdown_connections 2021-11-03T17:10:51.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.259+0000 7f5d32cd9700 1 --2- 172.21.15.27:0/3860674703 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f5d1405c2a0 0x7f5d1405e760 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:51.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.259+0000 7f5d32cd9700 1 --2- 172.21.15.27:0/3860674703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5d2c060810 0x7f5d2c0f6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:10:51.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.259+0000 7f5d32cd9700 1 --2- 172.21.15.27:0/3860674703 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5d2c0f9980 0x7f5d2c0f5300 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:51.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.259+0000 7f5d32cd9700 1 -- 172.21.15.27:0/3860674703 >> 172.21.15.27:0/3860674703 conn(0x7f5d2c0f0e20 msgr2=0x7f5d2c0f3210 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:10:51.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.259+0000 7f5d32cd9700 1 -- 172.21.15.27:0/3860674703 shutdown_connections 2021-11-03T17:10:51.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:51.259+0000 7f5d32cd9700 1 -- 172.21.15.27:0/3860674703 wait complete. 2021-11-03T17:10:51.865 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:51 smithi027 conmon[32322]: cluster 2021-11-03T17:10:50.561486+0000 mgr.smithi027.ufhhvb (mgr.14162) 176 : cluster [DBG] pgmap v147: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:10:51.865 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:51 smithi027 conmon[32322]: audit 2021-11-03T17:10:51.254792+0000 mon.smithi027 (mon.0) 444 : audit [DBG] from='client.? 172.21.15.27:0/3860674703' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2021-11-03T17:10:51.878 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":18,"active_gid":14162,"active_name":"smithi027.ufhhvb","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6800","nonce":3802716887},{"type":"v1","addr":"172.21.15.27:6801","nonce":3802716887}]},"active_addr":"172.21.15.27:6801/3802716887","active_change":"2021-11-03T17:06:51.527934+0000","active_mgr_features":4540138297136906239,"available":true,"standbys":[{"gid":14194,"name":"smithi115.tcyodz","mgr_features":4540138297136906239,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.7.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"haproxy","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Registry to which we should normalize unqualified image names","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"str","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/alertmanager:v0.20.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"docker.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"docker.io/ceph/ceph-grafana:6.7.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"haproxy","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/node-exporter:v0.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"docker.io/prom/prometheus:v2.18.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Registry to which we should normalize unqualified image names","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository url","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"str","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://smithi027.front.sepia.ceph.com:8443/","prometheus":"http://smithi027.front.sepia.ceph.com:9283/"},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"last_failure_osd_epoch":4,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":1421495022}]},{"addrvec":[{"type":"v2","addr":"172.21.15.27:0","nonce":11136840}]}]}} 2021-11-03T17:10:51.883 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2021-11-03T17:10:51.884 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2021-11-03T17:10:51.884 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph osd dump --format=json 2021-11-03T17:10:51.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:51 smithi115 conmon[34670]: cluster 2021-11-03T17:10:50.561486+0000 mgr.smithi027.ufhhvb (mgr.14162) 176 : cluster [DBG] 2021-11-03T17:10:51.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:51 smithi115 conmon[34670]: pgmap v147: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:10:51.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:51 smithi115 conmon[34670]: audit 2021-11-03T17:10:51.254792+0000 mon.smithi027 (mon.0) 444 : audit [DBG] from='client.? 172.21.15.27:0/3860674703' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2021-11-03T17:10:52.174 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c/mon.smithi027/config 2021-11-03T17:10:53.605 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:53 smithi027 conmon[32322]: cluster 2021-11-03T17:10:52.561741+0000 mgr.smithi027.ufhhvb 2021-11-03T17:10:53.606 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:53 smithi027 conmon[32322]: (mgr.14162) 177 : cluster [DBG] pgmap v148: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:10:53.606 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:53 smithi027 conmon[32322]: audit 2021-11-03T17:10:52.723727+0000 mon.smithi027 (mon.0) 445 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:10:53.606 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:53 smithi027 conmon[32322]: audit 2021-11-03T17:10:52.723940+0000 mon.smithi027 (mon.0) 446 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:10:53.792 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.793+0000 7f7f9942c700 1 -- 172.21.15.27:0/2558943769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f940f67e0 msgr2=0x7f7f940f6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:53.792 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.793+0000 7f7f9942c700 1 --2- 172.21.15.27:0/2558943769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f940f67e0 0x7f7f940f6c00 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f7f84004660 tx=0x7f7f84009b30).stop 2021-11-03T17:10:53.792 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.793+0000 7f7f9942c700 1 -- 172.21.15.27:0/2558943769 shutdown_connections 2021-11-03T17:10:53.792 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.793+0000 7f7f9942c700 1 --2- 172.21.15.27:0/2558943769 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f940f7a80 0x7f7f940f7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:53.793 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.793+0000 7f7f9942c700 1 --2- 172.21.15.27:0/2558943769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f940f67e0 0x7f7f940f6c00 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:53.793 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.793+0000 7f7f9942c700 1 -- 172.21.15.27:0/2558943769 >> 172.21.15.27:0/2558943769 conn(0x7f7f940f1f20 msgr2=0x7f7f940f4360 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:10:53.793 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.793+0000 7f7f9942c700 1 -- 172.21.15.27:0/2558943769 shutdown_connections 2021-11-03T17:10:53.793 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.793+0000 7f7f9942c700 1 -- 172.21.15.27:0/2558943769 wait complete. 2021-11-03T17:10:53.794 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.794+0000 7f7f9942c700 1 Processor -- start 2021-11-03T17:10:53.794 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.794+0000 7f7f9942c700 1 -- start start 2021-11-03T17:10:53.794 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.794+0000 7f7f9942c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f940f67e0 0x7f7f9406b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:10:53.794 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.794+0000 7f7f9942c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f940f7a80 0x7f7f94069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:10:53.795 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.794+0000 7f7f9942c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9406a380 con 0x7f7f940f7a80 2021-11-03T17:10:53.795 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.794+0000 7f7f9942c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9406a4c0 con 0x7f7f940f67e0 2021-11-03T17:10:53.795 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.794+0000 7f7f927fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f940f7a80 0x7f7f94069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:10:53.795 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.794+0000 7f7f927fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f940f7a80 0x7f7f94069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54726/0 (socket says 172.21.15.27:54726) 2021-11-03T17:10:53.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.794+0000 7f7f927fc700 1 -- 172.21.15.27:0/3313582390 learned_addr learned my addr 172.21.15.27:0/3313582390 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:10:53.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.794+0000 7f7f92ffd700 1 --2- 172.21.15.27:0/3313582390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f940f67e0 0x7f7f9406b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:10:53.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.794+0000 7f7f927fc700 1 -- 172.21.15.27:0/3313582390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f940f67e0 msgr2=0x7f7f9406b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:53.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.794+0000 7f7f927fc700 1 --2- 172.21.15.27:0/3313582390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f940f67e0 0x7f7f9406b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:53.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.794+0000 7f7f927fc700 1 -- 172.21.15.27:0/3313582390 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f84005040 con 0x7f7f940f7a80 2021-11-03T17:10:53.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.795+0000 7f7f927fc700 1 --2- 172.21.15.27:0/3313582390 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f940f7a80 0x7f7f94069e20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f7f7c0026d0 tx=0x7f7f7c009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:10:53.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.795+0000 7f7f8b7fe700 1 -- 172.21.15.27:0/3313582390 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f7c0079c0 con 0x7f7f940f7a80 2021-11-03T17:10:53.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.795+0000 7f7f8b7fe700 1 -- 172.21.15.27:0/3313582390 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f7f7c00f040 con 0x7f7f940f7a80 2021-11-03T17:10:53.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.795+0000 7f7f9942c700 1 -- 172.21.15.27:0/3313582390 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f9406a7a0 con 0x7f7f940f7a80 2021-11-03T17:10:53.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.795+0000 7f7f8b7fe700 1 -- 172.21.15.27:0/3313582390 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f7c014820 con 0x7f7f940f7a80 2021-11-03T17:10:53.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.795+0000 7f7f9942c700 1 -- 172.21.15.27:0/3313582390 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f9410f0e0 con 0x7f7f940f7a80 2021-11-03T17:10:53.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.796+0000 7f7f9942c700 1 -- 172.21.15.27:0/3313582390 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f94045ad0 con 0x7f7f940f7a80 2021-11-03T17:10:53.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.796+0000 7f7f8b7fe700 1 -- 172.21.15.27:0/3313582390 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f7f7c014980 con 0x7f7f940f7a80 2021-11-03T17:10:53.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.796+0000 7f7f8b7fe700 1 --2- 172.21.15.27:0/3313582390 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f7f8005c2a0 0x7f7f8005e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:10:53.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.797+0000 7f7f8b7fe700 1 -- 172.21.15.27:0/3313582390 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4519+0+0 (secure 0 0 0) 0x7f7f7c075a30 con 0x7f7f940f7a80 2021-11-03T17:10:53.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.797+0000 7f7f92ffd700 1 --2- 172.21.15.27:0/3313582390 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f7f8005c2a0 0x7f7f8005e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:10:53.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.797+0000 7f7f92ffd700 1 --2- 172.21.15.27:0/3313582390 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f7f8005c2a0 0x7f7f8005e760 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f7f84013620 tx=0x7f7f84009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:10:53.800 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.800+0000 7f7f8b7fe700 1 -- 172.21.15.27:0/3313582390 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f7f7c048b80 con 0x7f7f940f7a80 2021-11-03T17:10:53.954 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.959+0000 7f7f9942c700 1 -- 172.21.15.27:0/3313582390 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f94027a00 con 0x7f7f940f7a80 2021-11-03T17:10:53.955 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.960+0000 7f7f8b7fe700 1 -- 172.21.15.27:0/3313582390 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+10887 (secure 0 0 0) 0x7f7f7c04c1a0 con 0x7f7f940f7a80 2021-11-03T17:10:53.955 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:10:53.956 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":39,"fsid":"32ab977e-3cc8-11ec-8c28-001a4aab830c","created":"2021-11-03T17:05:46.314150+0000","modified":"2021-11-03T17:10:35.584298+0000","last_up_change":"2021-11-03T17:10:34.577218+0000","last_in_change":"2021-11-03T17:10:16.846149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-03T17:09:48.488849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"2f6bbb8f-4e3c-401a-b1bd-69ba9cdd9d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3126282997},{"type":"v1","addr":"172.21.15.27:6803","nonce":3126282997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3126282997},{"type":"v1","addr":"172.21.15.27:6805","nonce":3126282997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":3126282997},{"type":"v1","addr":"172.21.15.27:6809","nonce":3126282997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3126282997},{"type":"v1","addr":"172.21.15.27:6807","nonce":3126282997}]},"public_addr":"172.21.15.27:6803/3126282997","cluster_addr":"172.21.15.27:6805/3126282997","heartbeat_back_addr":"172.21.15.27:6809/3126282997","heartbeat_front_addr":"172.21.15.27:6807/3126282997","state":["exists","up"]},{"osd":1,"uuid":"0942b897-5aa3-461a-a2d6-943d5739d963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2723402359},{"type":"v1","addr":"172.21.15.27:6811","nonce":2723402359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2723402359},{"type":"v1","addr":"172.21.15.27:6813","nonce":2723402359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2723402359},{"type":"v1","addr":"172.21.15.27:6817","nonce":2723402359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2723402359},{"type":"v1","addr":"172.21.15.27:6815","nonce":2723402359}]},"public_addr":"172.21.15.27:6811/2723402359","cluster_addr":"172.21.15.27:6813/2723402359","heartbeat_back_addr":"172.21.15.27:6817/2723402359","heartbeat_front_addr":"172.21.15.27:6815/2723402359","state":["exists","up"]},{"osd":2,"uuid":"3ea602d0-3e10-4f30-96cd-288e53a7a00a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":329182726},{"type":"v1","addr":"172.21.15.27:6819","nonce":329182726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":329182726},{"type":"v1","addr":"172.21.15.27:6821","nonce":329182726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":329182726},{"type":"v1","addr":"172.21.15.27:6825","nonce":329182726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":329182726},{"type":"v1","addr":"172.21.15.27:6823","nonce":329182726}]},"public_addr":"172.21.15.27:6819/329182726","cluster_addr":"172.21.15.27:6821/329182726","heartbeat_back_addr":"172.21.15.27:6825/329182726","heartbeat_front_addr":"172.21.15.27:6823/329182726","state":["exists","up"]},{"osd":3,"uuid":"c517d190-0d21-4db0-994c-5f971d67f103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":4026713633},{"type":"v1","addr":"172.21.15.115:6801","nonce":4026713633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":4026713633},{"type":"v1","addr":"172.21.15.115:6803","nonce":4026713633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":4026713633},{"type":"v1","addr":"172.21.15.115:6807","nonce":4026713633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":4026713633},{"type":"v1","addr":"172.21.15.115:6805","nonce":4026713633}]},"public_addr":"172.21.15.115:6801/4026713633","cluster_addr":"172.21.15.115:6803/4026713633","heartbeat_back_addr":"172.21.15.115:6807/4026713633","heartbeat_front_addr":"172.21.15.115:6805/4026713633","state":["exists","up"]},{"osd":4,"uuid":"aeeb2c74-919d-4ac7-8c7a-1dd273225f59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1601394331},{"type":"v1","addr":"172.21.15.115:6809","nonce":1601394331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1601394331},{"type":"v1","addr":"172.21.15.115:6811","nonce":1601394331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1601394331},{"type":"v1","addr":"172.21.15.115:6815","nonce":1601394331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1601394331},{"type":"v1","addr":"172.21.15.115:6813","nonce":1601394331}]},"public_addr":"172.21.15.115:6809/1601394331","cluster_addr":"172.21.15.115:6811/1601394331","heartbeat_back_addr":"172.21.15.115:6815/1601394331","heartbeat_front_addr":"172.21.15.115:6813/1601394331","state":["exists","up"]},{"osd":5,"uuid":"047cfc7d-01d6-4859-8dc8-01d3811b9060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":2350455075},{"type":"v1","addr":"172.21.15.115:6817","nonce":2350455075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":2350455075},{"type":"v1","addr":"172.21.15.115:6819","nonce":2350455075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":2350455075},{"type":"v1","addr":"172.21.15.115:6823","nonce":2350455075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":2350455075},{"type":"v1","addr":"172.21.15.115:6821","nonce":2350455075}]},"public_addr":"172.21.15.115:6817/2350455075","cluster_addr":"172.21.15.115:6819/2350455075","heartbeat_back_addr":"172.21.15.115:6823/2350455075","heartbeat_front_addr":"172.21.15.115:6821/2350455075","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-11-03T17:08:50.764082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-11-03T17:09:12.720566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-11-03T17:09:34.450930+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-11-03T17:09:53.016884+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-11-03T17:10:12.728118+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-11-03T17:10:32.668302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6801/3506673123":"2021-11-04T17:06:51.527604+0000","172.21.15.27:6800/3506673123":"2021-11-04T17:06:51.527604+0000","172.21.15.27:0/92526005":"2021-11-04T17:06:51.527604+0000","172.21.15.27:0/1091817081":"2021-11-04T17:06:51.527604+0000","172.21.15.27:0/3225036173":"2021-11-04T17:06:34.410322+0000","172.21.15.27:6801/774407793":"2021-11-04T17:06:34.410322+0000","172.21.15.27:6800/774407793":"2021-11-04T17:06:34.410322+0000","172.21.15.27:0/3947386121":"2021-11-04T17:06:09.549455+0000","172.21.15.27:6800/388626148":"2021-11-04T17:06:09.549455+0000","172.21.15.27:0/2008079603":"2021-11-04T17:06:09.549455+0000","172.21.15.27:0/2202850722":"2021-11-04T17:06:34.410322+0000","172.21.15.27:6801/388626148":"2021-11-04T17:06:09.549455+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-11-03T17:10:53.956 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:53 smithi115 conmon[34670]: cluster 2021-11-03T17:10:52.561741+0000 mgr.smithi027.ufhhvb (mgr.14162) 177 : cluster [DBG] pgmap v148: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:10:53.957 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:53 smithi115 conmon[34670]: audit 2021-11-03T17:10:52.723727+0000 mon.smithi027 (mon.0) 445 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:10:53.957 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:53 smithi115 conmon[34670]: audit 2021-11-03T17:10:52.723940+0000 mon.smithi027 (mon.0) 446 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:10:53.957 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:53 smithi115 conmon[34670]: 2021-11-03T17:10:53.958 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.962+0000 7f7f9942c700 1 -- 172.21.15.27:0/3313582390 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f7f8005c2a0 msgr2=0x7f7f8005e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:53.958 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.962+0000 7f7f9942c700 1 --2- 172.21.15.27:0/3313582390 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f7f8005c2a0 0x7f7f8005e760 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f7f84013620 tx=0x7f7f84009b30).stop 2021-11-03T17:10:53.959 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.962+0000 7f7f9942c700 1 -- 172.21.15.27:0/3313582390 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f940f7a80 msgr2=0x7f7f94069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:53.959 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.962+0000 7f7f9942c700 1 --2- 172.21.15.27:0/3313582390 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f940f7a80 0x7f7f94069e20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f7f7c0026d0 tx=0x7f7f7c009de0).stop 2021-11-03T17:10:53.959 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.962+0000 7f7f9942c700 1 -- 172.21.15.27:0/3313582390 shutdown_connections 2021-11-03T17:10:53.959 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.962+0000 7f7f9942c700 1 --2- 172.21.15.27:0/3313582390 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f7f8005c2a0 0x7f7f8005e760 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:53.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.962+0000 7f7f9942c700 1 --2- 172.21.15.27:0/3313582390 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f940f67e0 0x7f7f9406b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:53.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.962+0000 7f7f9942c700 1 --2- 172.21.15.27:0/3313582390 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f940f7a80 0x7f7f94069e20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:53.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.962+0000 7f7f9942c700 1 -- 172.21.15.27:0/3313582390 >> 172.21.15.27:0/3313582390 conn(0x7f7f940f1f20 msgr2=0x7f7f940fb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:10:53.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.962+0000 7f7f9942c700 1 -- 172.21.15.27:0/3313582390 shutdown_connections 2021-11-03T17:10:53.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:53.962+0000 7f7f9942c700 1 -- 172.21.15.27:0/3313582390 wait complete. 2021-11-03T17:10:54.536 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2021-11-03T17:10:54.537 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph osd dump --format=json 2021-11-03T17:10:54.828 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c/mon.smithi027/config 2021-11-03T17:10:54.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:54 smithi115 conmon[34670]: audit 2021-11-03T17:10:53.960429+0000 mon.smithi027 (mon.0) 447 : audit [DBG] from='client.? 172.21.15.27:0/3313582390' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2021-11-03T17:10:55.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:54 smithi027 conmon[32322]: audit 2021-11-03T17:10:53.960429+0000 mon.smithi027 2021-11-03T17:10:55.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:54 smithi027 conmon[32322]: (mon.0) 447 : audit [DBG] from='client.? 172.21.15.27:0/3313582390' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2021-11-03T17:10:55.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:55 smithi027 conmon[32322]: cluster 2021-11-03T17:10:54.562039 2021-11-03T17:10:55.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:55 smithi027 conmon[32322]: +0000 mgr.smithi027.ufhhvb (mgr.14162) 178 : cluster [DBG] pgmap v149: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:10:55.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:55 smithi115 conmon[34670]: cluster 2021-11-03T17:10:54.562039+0000 mgr.smithi027.ufhhvb (mgr.14162) 178 : cluster [DBG] pgmap v149: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:10:56.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.451+0000 7f481b59e700 1 -- 172.21.15.27:0/2874952708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f481c0f57f0 msgr2=0x7f481c0f5c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:56.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.451+0000 7f481b59e700 1 --2- 172.21.15.27:0/2874952708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f481c0f57f0 0x7f481c0f5c70 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f4804004660 tx=0x7f4804009b30).stop 2021-11-03T17:10:56.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.451+0000 7f481b59e700 1 -- 172.21.15.27:0/2874952708 shutdown_connections 2021-11-03T17:10:56.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.451+0000 7f481b59e700 1 --2- 172.21.15.27:0/2874952708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f481c0f57f0 0x7f481c0f5c70 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:56.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.451+0000 7f481b59e700 1 --2- 172.21.15.27:0/2874952708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f481c0f6af0 0x7f481c0f6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:56.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.451+0000 7f481b59e700 1 -- 172.21.15.27:0/2874952708 >> 172.21.15.27:0/2874952708 conn(0x7f481c0f0400 msgr2=0x7f481c0f2820 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:10:56.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.451+0000 7f481b59e700 1 -- 172.21.15.27:0/2874952708 shutdown_connections 2021-11-03T17:10:56.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.451+0000 7f481b59e700 1 -- 172.21.15.27:0/2874952708 wait complete. 2021-11-03T17:10:56.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.452+0000 7f481b59e700 1 Processor -- start 2021-11-03T17:10:56.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.452+0000 7f481b59e700 1 -- start start 2021-11-03T17:10:56.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.452+0000 7f481b59e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f481c0f57f0 0x7f481c106f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:10:56.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.452+0000 7f481b59e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f481c0f6af0 0x7f481c1073f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:10:56.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.452+0000 7f481b59e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481c1079a0 con 0x7f481c0f57f0 2021-11-03T17:10:56.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.452+0000 7f481b59e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481c107ae0 con 0x7f481c0f6af0 2021-11-03T17:10:56.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.452+0000 7f481a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f481c0f57f0 0x7f481c106f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:10:56.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.452+0000 7f481a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f481c0f57f0 0x7f481c106f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54734/0 (socket says 172.21.15.27:54734) 2021-11-03T17:10:56.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.452+0000 7f481a59c700 1 -- 172.21.15.27:0/1482174642 learned_addr learned my addr 172.21.15.27:0/1482174642 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:10:56.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.452+0000 7f4819d9b700 1 --2- 172.21.15.27:0/1482174642 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f481c0f6af0 0x7f481c1073f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:10:56.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.452+0000 7f481a59c700 1 -- 172.21.15.27:0/1482174642 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f481c0f6af0 msgr2=0x7f481c1073f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:56.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.452+0000 7f481a59c700 1 --2- 172.21.15.27:0/1482174642 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f481c0f6af0 0x7f481c1073f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:56.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.452+0000 7f481a59c700 1 -- 172.21.15.27:0/1482174642 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4804005040 con 0x7f481c0f57f0 2021-11-03T17:10:56.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.452+0000 7f4819d9b700 1 --2- 172.21.15.27:0/1482174642 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f481c0f6af0 0x7f481c1073f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:10:56.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.453+0000 7f481a59c700 1 --2- 172.21.15.27:0/1482174642 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f481c0f57f0 0x7f481c106f00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f480c002700 tx=0x7f480c009d60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:10:56.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.453+0000 7f4812ffd700 1 -- 172.21.15.27:0/1482174642 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f480c009320 con 0x7f481c0f57f0 2021-11-03T17:10:56.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.453+0000 7f4812ffd700 1 -- 172.21.15.27:0/1482174642 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f480c00f040 con 0x7f481c0f57f0 2021-11-03T17:10:56.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.453+0000 7f481b59e700 1 -- 172.21.15.27:0/1482174642 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f481c108480 con 0x7f481c0f57f0 2021-11-03T17:10:56.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.453+0000 7f4812ffd700 1 -- 172.21.15.27:0/1482174642 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f480c0146f0 con 0x7f481c0f57f0 2021-11-03T17:10:56.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.453+0000 7f481b59e700 1 -- 172.21.15.27:0/1482174642 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f481c108a40 con 0x7f481c0f57f0 2021-11-03T17:10:56.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.454+0000 7f481b59e700 1 -- 172.21.15.27:0/1482174642 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f481c045ad0 con 0x7f481c0f57f0 2021-11-03T17:10:56.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.454+0000 7f4812ffd700 1 -- 172.21.15.27:0/1482174642 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f480c009480 con 0x7f481c0f57f0 2021-11-03T17:10:56.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.454+0000 7f4812ffd700 1 --2- 172.21.15.27:0/1482174642 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f480805c2b0 0x7f480805e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:10:56.457 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.454+0000 7f4812ffd700 1 -- 172.21.15.27:0/1482174642 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4519+0+0 (secure 0 0 0) 0x7f480c0761e0 con 0x7f481c0f57f0 2021-11-03T17:10:56.457 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.455+0000 7f4819d9b700 1 --2- 172.21.15.27:0/1482174642 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f480805c2b0 0x7f480805e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:10:56.457 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.455+0000 7f4819d9b700 1 --2- 172.21.15.27:0/1482174642 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f480805c2b0 0x7f480805e770 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f4804004e30 tx=0x7f4804009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:10:56.457 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.458+0000 7f4812ffd700 1 -- 172.21.15.27:0/1482174642 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f480c0491a0 con 0x7f481c0f57f0 2021-11-03T17:10:56.612 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.617+0000 7f481b59e700 1 -- 172.21.15.27:0/1482174642 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f481c0fef40 con 0x7f481c0f57f0 2021-11-03T17:10:56.612 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.617+0000 7f4812ffd700 1 -- 172.21.15.27:0/1482174642 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+10887 (secure 0 0 0) 0x7f480c04c7c0 con 0x7f481c0f57f0 2021-11-03T17:10:56.613 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:10:56.613 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":39,"fsid":"32ab977e-3cc8-11ec-8c28-001a4aab830c","created":"2021-11-03T17:05:46.314150+0000","modified":"2021-11-03T17:10:35.584298+0000","last_up_change":"2021-11-03T17:10:34.577218+0000","last_in_change":"2021-11-03T17:10:16.846149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-03T17:09:48.488849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}}],"osds":[{"osd":0,"uuid":"2f6bbb8f-4e3c-401a-b1bd-69ba9cdd9d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3126282997},{"type":"v1","addr":"172.21.15.27:6803","nonce":3126282997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3126282997},{"type":"v1","addr":"172.21.15.27:6805","nonce":3126282997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":3126282997},{"type":"v1","addr":"172.21.15.27:6809","nonce":3126282997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3126282997},{"type":"v1","addr":"172.21.15.27:6807","nonce":3126282997}]},"public_addr":"172.21.15.27:6803/3126282997","cluster_addr":"172.21.15.27:6805/3126282997","heartbeat_back_addr":"172.21.15.27:6809/3126282997","heartbeat_front_addr":"172.21.15.27:6807/3126282997","state":["exists","up"]},{"osd":1,"uuid":"0942b897-5aa3-461a-a2d6-943d5739d963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":28,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2723402359},{"type":"v1","addr":"172.21.15.27:6811","nonce":2723402359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2723402359},{"type":"v1","addr":"172.21.15.27:6813","nonce":2723402359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2723402359},{"type":"v1","addr":"172.21.15.27:6817","nonce":2723402359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2723402359},{"type":"v1","addr":"172.21.15.27:6815","nonce":2723402359}]},"public_addr":"172.21.15.27:6811/2723402359","cluster_addr":"172.21.15.27:6813/2723402359","heartbeat_back_addr":"172.21.15.27:6817/2723402359","heartbeat_front_addr":"172.21.15.27:6815/2723402359","state":["exists","up"]},{"osd":2,"uuid":"3ea602d0-3e10-4f30-96cd-288e53a7a00a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":329182726},{"type":"v1","addr":"172.21.15.27:6819","nonce":329182726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":329182726},{"type":"v1","addr":"172.21.15.27:6821","nonce":329182726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":329182726},{"type":"v1","addr":"172.21.15.27:6825","nonce":329182726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":329182726},{"type":"v1","addr":"172.21.15.27:6823","nonce":329182726}]},"public_addr":"172.21.15.27:6819/329182726","cluster_addr":"172.21.15.27:6821/329182726","heartbeat_back_addr":"172.21.15.27:6825/329182726","heartbeat_front_addr":"172.21.15.27:6823/329182726","state":["exists","up"]},{"osd":3,"uuid":"c517d190-0d21-4db0-994c-5f971d67f103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":4026713633},{"type":"v1","addr":"172.21.15.115:6801","nonce":4026713633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":4026713633},{"type":"v1","addr":"172.21.15.115:6803","nonce":4026713633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":4026713633},{"type":"v1","addr":"172.21.15.115:6807","nonce":4026713633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":4026713633},{"type":"v1","addr":"172.21.15.115:6805","nonce":4026713633}]},"public_addr":"172.21.15.115:6801/4026713633","cluster_addr":"172.21.15.115:6803/4026713633","heartbeat_back_addr":"172.21.15.115:6807/4026713633","heartbeat_front_addr":"172.21.15.115:6805/4026713633","state":["exists","up"]},{"osd":4,"uuid":"aeeb2c74-919d-4ac7-8c7a-1dd273225f59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1601394331},{"type":"v1","addr":"172.21.15.115:6809","nonce":1601394331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1601394331},{"type":"v1","addr":"172.21.15.115:6811","nonce":1601394331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1601394331},{"type":"v1","addr":"172.21.15.115:6815","nonce":1601394331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1601394331},{"type":"v1","addr":"172.21.15.115:6813","nonce":1601394331}]},"public_addr":"172.21.15.115:6809/1601394331","cluster_addr":"172.21.15.115:6811/1601394331","heartbeat_back_addr":"172.21.15.115:6815/1601394331","heartbeat_front_addr":"172.21.15.115:6813/1601394331","state":["exists","up"]},{"osd":5,"uuid":"047cfc7d-01d6-4859-8dc8-01d3811b9060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":2350455075},{"type":"v1","addr":"172.21.15.115:6817","nonce":2350455075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":2350455075},{"type":"v1","addr":"172.21.15.115:6819","nonce":2350455075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":2350455075},{"type":"v1","addr":"172.21.15.115:6823","nonce":2350455075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":2350455075},{"type":"v1","addr":"172.21.15.115:6821","nonce":2350455075}]},"public_addr":"172.21.15.115:6817/2350455075","cluster_addr":"172.21.15.115:6819/2350455075","heartbeat_back_addr":"172.21.15.115:6823/2350455075","heartbeat_front_addr":"172.21.15.115:6821/2350455075","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-11-03T17:08:50.764082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-11-03T17:09:12.720566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-11-03T17:09:34.450930+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-11-03T17:09:53.016884+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-11-03T17:10:12.728118+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-11-03T17:10:32.668302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6801/3506673123":"2021-11-04T17:06:51.527604+0000","172.21.15.27:6800/3506673123":"2021-11-04T17:06:51.527604+0000","172.21.15.27:0/92526005":"2021-11-04T17:06:51.527604+0000","172.21.15.27:0/1091817081":"2021-11-04T17:06:51.527604+0000","172.21.15.27:0/3225036173":"2021-11-04T17:06:34.410322+0000","172.21.15.27:6801/774407793":"2021-11-04T17:06:34.410322+0000","172.21.15.27:6800/774407793":"2021-11-04T17:06:34.410322+0000","172.21.15.27:0/3947386121":"2021-11-04T17:06:09.549455+0000","172.21.15.27:6800/388626148":"2021-11-04T17:06:09.549455+0000","172.21.15.27:0/2008079603":"2021-11-04T17:06:09.549455+0000","172.21.15.27:0/2202850722":"2021-11-04T17:06:34.410322+0000","172.21.15.27:6801/388626148":"2021-11-04T17:06:09.549455+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-11-03T17:10:56.614 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.619+0000 7f481b59e700 1 -- 172.21.15.27:0/1482174642 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f480805c2b0 msgr2=0x7f480805e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:56.614 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.619+0000 7f481b59e700 1 --2- 172.21.15.27:0/1482174642 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f480805c2b0 0x7f480805e770 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f4804004e30 tx=0x7f4804009b30).stop 2021-11-03T17:10:56.615 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.619+0000 7f481b59e700 1 -- 172.21.15.27:0/1482174642 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f481c0f57f0 msgr2=0x7f481c106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:56.615 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.619+0000 7f481b59e700 1 --2- 172.21.15.27:0/1482174642 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f481c0f57f0 0x7f481c106f00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f480c002700 tx=0x7f480c009d60).stop 2021-11-03T17:10:56.615 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.619+0000 7f481b59e700 1 -- 172.21.15.27:0/1482174642 shutdown_connections 2021-11-03T17:10:56.615 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.619+0000 7f481b59e700 1 --2- 172.21.15.27:0/1482174642 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f480805c2b0 0x7f480805e770 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:56.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.619+0000 7f481b59e700 1 --2- 172.21.15.27:0/1482174642 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f481c0f57f0 0x7f481c106f00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:56.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.619+0000 7f481b59e700 1 --2- 172.21.15.27:0/1482174642 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f481c0f6af0 0x7f481c1073f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:56.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.619+0000 7f481b59e700 1 -- 172.21.15.27:0/1482174642 >> 172.21.15.27:0/1482174642 conn(0x7f481c0f0400 msgr2=0x7f481c0fa250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:10:56.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.619+0000 7f481b59e700 1 -- 172.21.15.27:0/1482174642 shutdown_connections 2021-11-03T17:10:56.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:56.619+0000 7f481b59e700 1 -- 172.21.15.27:0/1482174642 wait complete. 2021-11-03T17:10:57.187 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph tell osd.0 flush_pg_stats 2021-11-03T17:10:57.476 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c/mon.smithi027/config 2021-11-03T17:10:57.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:57 smithi115 conmon[34670]: cluster 2021-11-03T17:10:56.562357+0000 mgr.smithi027.ufhhvb (mgr.14162) 179 : cluster [DBG] pgmap v150: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:10:57.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:57 smithi115 conmon[34670]: audit 2021-11-03T17:10:56.617704+0000 mon.smithi027 (mon.0) 448 : audit [DBG] from='client.? 172.21.15.27:0/1482174642' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2021-11-03T17:10:58.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:57 smithi027 conmon[32322]: cluster 2021-11-03T17:10:56.562357+0000 mgr.smithi027.ufhhvb (mgr.14162) 179 : cluster [DBG] pgmap v150: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:10:58.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:57 smithi027 conmon[32322]: audit 2021-11-03T17:10:56.617704+0000 mon.smithi027 (mon.0) 448 : audit [DBG] from='client.? 172.21.15.27:0/1482174642' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2021-11-03T17:10:59.104 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.109+0000 7fd82d212700 1 -- 172.21.15.27:0/1789237580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8280f7a80 msgr2=0x7fd8280f7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:59.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.109+0000 7fd82d212700 1 --2- 172.21.15.27:0/1789237580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8280f7a80 0x7fd8280f7ee0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fd81c004660 tx=0x7fd81c009b30).stop 2021-11-03T17:10:59.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.110+0000 7fd82d212700 1 -- 172.21.15.27:0/1789237580 shutdown_connections 2021-11-03T17:10:59.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.110+0000 7fd82d212700 1 --2- 172.21.15.27:0/1789237580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8280f7a80 0x7fd8280f7ee0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:59.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.110+0000 7fd82d212700 1 --2- 172.21.15.27:0/1789237580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8280f67e0 0x7fd8280f6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:59.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.110+0000 7fd82d212700 1 -- 172.21.15.27:0/1789237580 >> 172.21.15.27:0/1789237580 conn(0x7fd8280f1f20 msgr2=0x7fd8280f4360 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:10:59.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.110+0000 7fd82d212700 1 -- 172.21.15.27:0/1789237580 shutdown_connections 2021-11-03T17:10:59.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.110+0000 7fd82d212700 1 -- 172.21.15.27:0/1789237580 wait complete. 2021-11-03T17:10:59.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.110+0000 7fd82d212700 1 Processor -- start 2021-11-03T17:10:59.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.110+0000 7fd82d212700 1 -- start start 2021-11-03T17:10:59.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.110+0000 7fd82d212700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8280f67e0 0x7fd828106fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:10:59.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.110+0000 7fd82d212700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8280f7a80 0x7fd828107490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:10:59.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.110+0000 7fd82d212700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd828107a40 con 0x7fd8280f7a80 2021-11-03T17:10:59.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.110+0000 7fd82d212700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd828107b80 con 0x7fd8280f67e0 2021-11-03T17:10:59.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.111+0000 7fd82659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8280f7a80 0x7fd828107490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:10:59.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.111+0000 7fd82659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8280f7a80 0x7fd828107490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54746/0 (socket says 172.21.15.27:54746) 2021-11-03T17:10:59.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.111+0000 7fd82659c700 1 -- 172.21.15.27:0/1806601671 learned_addr learned my addr 172.21.15.27:0/1806601671 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:10:59.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.111+0000 7fd826d9d700 1 --2- 172.21.15.27:0/1806601671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8280f67e0 0x7fd828106fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:10:59.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.111+0000 7fd82659c700 1 -- 172.21.15.27:0/1806601671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8280f67e0 msgr2=0x7fd828106fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:59.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.111+0000 7fd82659c700 1 --2- 172.21.15.27:0/1806601671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8280f67e0 0x7fd828106fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:59.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.111+0000 7fd82659c700 1 -- 172.21.15.27:0/1806601671 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd81c005040 con 0x7fd8280f7a80 2021-11-03T17:10:59.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.111+0000 7fd82659c700 1 --2- 172.21.15.27:0/1806601671 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8280f7a80 0x7fd828107490 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fd81c000c00 tx=0x7fd81c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:10:59.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.111+0000 7fd80f7fe700 1 -- 172.21.15.27:0/1806601671 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd81c019070 con 0x7fd8280f7a80 2021-11-03T17:10:59.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.111+0000 7fd80f7fe700 1 -- 172.21.15.27:0/1806601671 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7fd81c007e50 con 0x7fd8280f7a80 2021-11-03T17:10:59.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.111+0000 7fd80f7fe700 1 -- 172.21.15.27:0/1806601671 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd81c014cd0 con 0x7fd8280f7a80 2021-11-03T17:10:59.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.111+0000 7fd82d212700 1 -- 172.21.15.27:0/1806601671 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8281084c0 con 0x7fd8280f7a80 2021-11-03T17:10:59.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.111+0000 7fd82d212700 1 -- 172.21.15.27:0/1806601671 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd828108a20 con 0x7fd8280f7a80 2021-11-03T17:10:59.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.112+0000 7fd82d212700 1 -- 172.21.15.27:0/1806601671 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd82805dcf0 con 0x7fd8280f7a80 2021-11-03T17:10:59.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.112+0000 7fd80f7fe700 1 -- 172.21.15.27:0/1806601671 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fd81c025070 con 0x7fd8280f7a80 2021-11-03T17:10:59.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.113+0000 7fd80f7fe700 1 --2- 172.21.15.27:0/1806601671 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fd81005c2a0 0x7fd81005e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:10:59.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.113+0000 7fd80f7fe700 1 -- 172.21.15.27:0/1806601671 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4519+0+0 (secure 0 0 0) 0x7fd81c076a70 con 0x7fd8280f7a80 2021-11-03T17:10:59.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.113+0000 7fd826d9d700 1 --2- 172.21.15.27:0/1806601671 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fd81005c2a0 0x7fd81005e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:10:59.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.113+0000 7fd80f7fe700 1 --2- 172.21.15.27:0/1806601671 >> [v2:172.21.15.27:6802/3126282997,v1:172.21.15.27:6803/3126282997] conn(0x7fd810061fc0 0x7fd810064400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:10:59.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.113+0000 7fd80f7fe700 1 -- 172.21.15.27:0/1806601671 --> [v2:172.21.15.27:6802/3126282997,v1:172.21.15.27:6803/3126282997] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd810064a80 con 0x7fd810061fc0 2021-11-03T17:10:59.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.113+0000 7fd80f7fe700 1 -- 172.21.15.27:0/1806601671 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd81c07b220 con 0x7fd8280f7a80 2021-11-03T17:10:59.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.113+0000 7fd82759e700 1 --2- 172.21.15.27:0/1806601671 >> [v2:172.21.15.27:6802/3126282997,v1:172.21.15.27:6803/3126282997] conn(0x7fd810061fc0 0x7fd810064400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:10:59.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.113+0000 7fd826d9d700 1 --2- 172.21.15.27:0/1806601671 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fd81005c2a0 0x7fd81005e760 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fd818009fd0 tx=0x7fd81800b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:10:59.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.113+0000 7fd82759e700 1 --2- 172.21.15.27:0/1806601671 >> [v2:172.21.15.27:6802/3126282997,v1:172.21.15.27:6803/3126282997] conn(0x7fd810061fc0 0x7fd810064400 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:10:59.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.114+0000 7fd80f7fe700 1 -- 172.21.15.27:0/1806601671 <== osd.0 v2:172.21.15.27:6802/3126282997 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7fd810064a80 con 0x7fd810061fc0 2021-11-03T17:10:59.123 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.128+0000 7fd82d212700 1 -- 172.21.15.27:0/1806601671 --> [v2:172.21.15.27:6802/3126282997,v1:172.21.15.27:6803/3126282997] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd828108ed0 con 0x7fd810061fc0 2021-11-03T17:10:59.123 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.128+0000 7fd80f7fe700 1 -- 172.21.15.27:0/1806601671 <== osd.0 v2:172.21.15.27:6802/3126282997 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd828108ed0 con 0x7fd810061fc0 2021-11-03T17:10:59.125 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.129+0000 7fd82d212700 1 -- 172.21.15.27:0/1806601671 >> [v2:172.21.15.27:6802/3126282997,v1:172.21.15.27:6803/3126282997] conn(0x7fd810061fc0 msgr2=0x7fd810064400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:59.125 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.129+0000 7fd82d212700 1 --2- 172.21.15.27:0/1806601671 >> [v2:172.21.15.27:6802/3126282997,v1:172.21.15.27:6803/3126282997] conn(0x7fd810061fc0 0x7fd810064400 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:59.125 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.129+0000 7fd82d212700 1 -- 172.21.15.27:0/1806601671 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fd81005c2a0 msgr2=0x7fd81005e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:59.125 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.129+0000 7fd82d212700 1 --2- 172.21.15.27:0/1806601671 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fd81005c2a0 0x7fd81005e760 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7fd818009fd0 tx=0x7fd81800b040).stop 2021-11-03T17:10:59.126 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.129+0000 7fd82d212700 1 -- 172.21.15.27:0/1806601671 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8280f7a80 msgr2=0x7fd828107490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:10:59.126 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.129+0000 7fd82d212700 1 --2- 172.21.15.27:0/1806601671 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8280f7a80 0x7fd828107490 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fd81c000c00 tx=0x7fd81c00b040).stop 2021-11-03T17:10:59.126 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.129+0000 7fd82d212700 1 -- 172.21.15.27:0/1806601671 shutdown_connections 2021-11-03T17:10:59.126 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.129+0000 7fd82d212700 1 --2- 172.21.15.27:0/1806601671 >> [v2:172.21.15.27:6802/3126282997,v1:172.21.15.27:6803/3126282997] conn(0x7fd810061fc0 0x7fd810064400 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:59.127 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.129+0000 7fd82d212700 1 --2- 172.21.15.27:0/1806601671 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fd81005c2a0 0x7fd81005e760 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:59.127 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.129+0000 7fd82d212700 1 --2- 172.21.15.27:0/1806601671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8280f67e0 0x7fd828106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:59.127 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.129+0000 7fd82d212700 1 --2- 172.21.15.27:0/1806601671 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8280f7a80 0x7fd828107490 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:10:59.127 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.129+0000 7fd82d212700 1 -- 172.21.15.27:0/1806601671 >> 172.21.15.27:0/1806601671 conn(0x7fd8280f1f20 msgr2=0x7fd8280fb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:10:59.128 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.129+0000 7fd82d212700 1 -- 172.21.15.27:0/1806601671 shutdown_connections 2021-11-03T17:10:59.128 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:10:59.129+0000 7fd82d212700 1 -- 172.21.15.27:0/1806601671 wait complete. 2021-11-03T17:10:59.718 INFO:teuthology.orchestra.run.smithi027.stdout:38654705693 2021-11-03T17:10:59.718 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph tell osd.1 flush_pg_stats 2021-11-03T17:10:59.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:10:59 smithi115 conmon[34670]: cluster 2021-11-03T17:10:58.562841+0000 mgr.smithi027.ufhhvb (mgr.14162) 180 : cluster [DBG] pgmap v151: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:00.004 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c/mon.smithi027/config 2021-11-03T17:11:00.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:10:59 smithi027 conmon[32322]: cluster 2021-11-03T17:10:58.562841+0000 mgr.smithi027.ufhhvb (mgr.14162) 180 : cluster [DBG] pgmap v151: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:01.608 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.611+0000 7fe036d16700 1 -- 172.21.15.27:0/248451252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0300f56e0 msgr2=0x7fe0300f5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:01.608 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.611+0000 7fe036d16700 1 --2- 172.21.15.27:0/248451252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0300f56e0 0x7fe0300f5b00 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7fe024004660 tx=0x7fe024009b30).stop 2021-11-03T17:11:01.608 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.611+0000 7fe036d16700 1 -- 172.21.15.27:0/248451252 shutdown_connections 2021-11-03T17:11:01.609 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.611+0000 7fe036d16700 1 --2- 172.21.15.27:0/248451252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0300f6980 0x7fe0300f6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:01.609 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.611+0000 7fe036d16700 1 --2- 172.21.15.27:0/248451252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0300f56e0 0x7fe0300f5b00 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:01.609 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.611+0000 7fe036d16700 1 -- 172.21.15.27:0/248451252 >> 172.21.15.27:0/248451252 conn(0x7fe0300f0e20 msgr2=0x7fe0300f3260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:01.609 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.611+0000 7fe036d16700 1 -- 172.21.15.27:0/248451252 shutdown_connections 2021-11-03T17:11:01.609 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.611+0000 7fe036d16700 1 -- 172.21.15.27:0/248451252 wait complete. 2021-11-03T17:11:01.610 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.612+0000 7fe036d16700 1 Processor -- start 2021-11-03T17:11:01.610 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.612+0000 7fe036d16700 1 -- start start 2021-11-03T17:11:01.610 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.612+0000 7fe036d16700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0300f56e0 0x7fe0301070a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:01.610 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.612+0000 7fe036d16700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0300f6980 0x7fe030107590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:01.611 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.612+0000 7fe036d16700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe030107b40 con 0x7fe0300f56e0 2021-11-03T17:11:01.611 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.612+0000 7fe036d16700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe030107c80 con 0x7fe0300f6980 2021-11-03T17:11:01.611 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.612+0000 7fe034ab2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0300f56e0 0x7fe0301070a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:01.611 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.612+0000 7fe034ab2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0300f56e0 0x7fe0301070a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54756/0 (socket says 172.21.15.27:54756) 2021-11-03T17:11:01.611 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.612+0000 7fe034ab2700 1 -- 172.21.15.27:0/3878374267 learned_addr learned my addr 172.21.15.27:0/3878374267 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:11:01.612 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.613+0000 7fe034ab2700 1 -- 172.21.15.27:0/3878374267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0300f6980 msgr2=0x7fe030107590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:01.612 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.613+0000 7fe034ab2700 1 --2- 172.21.15.27:0/3878374267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0300f6980 0x7fe030107590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:11:01.612 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.613+0000 7fe034ab2700 1 -- 172.21.15.27:0/3878374267 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe024005040 con 0x7fe0300f56e0 2021-11-03T17:11:01.612 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.613+0000 7fe034ab2700 1 --2- 172.21.15.27:0/3878374267 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0300f56e0 0x7fe0301070a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7fe024007a60 tx=0x7fe024009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:01.613 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.613+0000 7fe02d7fa700 1 -- 172.21.15.27:0/3878374267 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe024018070 con 0x7fe0300f56e0 2021-11-03T17:11:01.613 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.613+0000 7fe036d16700 1 -- 172.21.15.27:0/3878374267 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0301085c0 con 0x7fe0300f56e0 2021-11-03T17:11:01.613 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.613+0000 7fe02d7fa700 1 -- 172.21.15.27:0/3878374267 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7fe024013410 con 0x7fe0300f56e0 2021-11-03T17:11:01.613 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.613+0000 7fe02d7fa700 1 -- 172.21.15.27:0/3878374267 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe02401c620 con 0x7fe0300f56e0 2021-11-03T17:11:01.614 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.613+0000 7fe036d16700 1 -- 172.21.15.27:0/3878374267 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe030108b20 con 0x7fe0300f56e0 2021-11-03T17:11:01.615 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.614+0000 7fe036d16700 1 -- 172.21.15.27:0/3878374267 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe03005dcf0 con 0x7fe0300f56e0 2021-11-03T17:11:01.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.614+0000 7fe02d7fa700 1 -- 172.21.15.27:0/3878374267 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fe02401d940 con 0x7fe0300f56e0 2021-11-03T17:11:01.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.615+0000 7fe02d7fa700 1 --2- 172.21.15.27:0/3878374267 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fe02005c2a0 0x7fe02005e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:01.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.615+0000 7fe02d7fa700 1 -- 172.21.15.27:0/3878374267 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4519+0+0 (secure 0 0 0) 0x7fe024078300 con 0x7fe0300f56e0 2021-11-03T17:11:01.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.615+0000 7fe02ffff700 1 --2- 172.21.15.27:0/3878374267 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fe02005c2a0 0x7fe02005e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:01.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.615+0000 7fe02d7fa700 1 --2- 172.21.15.27:0/3878374267 >> [v2:172.21.15.27:6810/2723402359,v1:172.21.15.27:6811/2723402359] conn(0x7fe020061fc0 0x7fe020064400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:01.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.615+0000 7fe02d7fa700 1 -- 172.21.15.27:0/3878374267 --> [v2:172.21.15.27:6810/2723402359,v1:172.21.15.27:6811/2723402359] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe020064a80 con 0x7fe020061fc0 2021-11-03T17:11:01.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.615+0000 7fe02d7fa700 1 -- 172.21.15.27:0/3878374267 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe024078640 con 0x7fe0300f56e0 2021-11-03T17:11:01.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.615+0000 7fe0352b3700 1 --2- 172.21.15.27:0/3878374267 >> [v2:172.21.15.27:6810/2723402359,v1:172.21.15.27:6811/2723402359] conn(0x7fe020061fc0 0x7fe020064400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:01.618 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.615+0000 7fe02ffff700 1 --2- 172.21.15.27:0/3878374267 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fe02005c2a0 0x7fe02005e760 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fe01c0047d0 tx=0x7fe01c007a40).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:01.618 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.615+0000 7fe0352b3700 1 --2- 172.21.15.27:0/3878374267 >> [v2:172.21.15.27:6810/2723402359,v1:172.21.15.27:6811/2723402359] conn(0x7fe020061fc0 0x7fe020064400 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:01.618 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.616+0000 7fe02d7fa700 1 -- 172.21.15.27:0/3878374267 <== osd.1 v2:172.21.15.27:6810/2723402359 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7fe020064a80 con 0x7fe020061fc0 2021-11-03T17:11:01.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.630+0000 7fe036d16700 1 -- 172.21.15.27:0/3878374267 --> [v2:172.21.15.27:6810/2723402359,v1:172.21.15.27:6811/2723402359] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe030108f80 con 0x7fe020061fc0 2021-11-03T17:11:01.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.631+0000 7fe02d7fa700 1 -- 172.21.15.27:0/3878374267 <== osd.1 v2:172.21.15.27:6810/2723402359 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe030108f80 con 0x7fe020061fc0 2021-11-03T17:11:01.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.631+0000 7fe036d16700 1 -- 172.21.15.27:0/3878374267 >> [v2:172.21.15.27:6810/2723402359,v1:172.21.15.27:6811/2723402359] conn(0x7fe020061fc0 msgr2=0x7fe020064400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:01.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.631+0000 7fe036d16700 1 --2- 172.21.15.27:0/3878374267 >> [v2:172.21.15.27:6810/2723402359,v1:172.21.15.27:6811/2723402359] conn(0x7fe020061fc0 0x7fe020064400 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:01.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.631+0000 7fe036d16700 1 -- 172.21.15.27:0/3878374267 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fe02005c2a0 msgr2=0x7fe02005e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:01.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.631+0000 7fe036d16700 1 --2- 172.21.15.27:0/3878374267 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fe02005c2a0 0x7fe02005e760 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7fe01c0047d0 tx=0x7fe01c007a40).stop 2021-11-03T17:11:01.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.631+0000 7fe036d16700 1 -- 172.21.15.27:0/3878374267 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0300f56e0 msgr2=0x7fe0301070a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:01.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.631+0000 7fe036d16700 1 --2- 172.21.15.27:0/3878374267 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0300f56e0 0x7fe0301070a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7fe024007a60 tx=0x7fe024009b30).stop 2021-11-03T17:11:01.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.631+0000 7fe036d16700 1 -- 172.21.15.27:0/3878374267 shutdown_connections 2021-11-03T17:11:01.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.631+0000 7fe036d16700 1 --2- 172.21.15.27:0/3878374267 >> [v2:172.21.15.27:6810/2723402359,v1:172.21.15.27:6811/2723402359] conn(0x7fe020061fc0 0x7fe020064400 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:01.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.631+0000 7fe036d16700 1 --2- 172.21.15.27:0/3878374267 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fe02005c2a0 0x7fe02005e760 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:01.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.631+0000 7fe036d16700 1 --2- 172.21.15.27:0/3878374267 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0300f56e0 0x7fe0301070a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:01.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.631+0000 7fe036d16700 1 --2- 172.21.15.27:0/3878374267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0300f6980 0x7fe030107590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:11:01.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.631+0000 7fe036d16700 1 -- 172.21.15.27:0/3878374267 >> 172.21.15.27:0/3878374267 conn(0x7fe0300f0e20 msgr2=0x7fe0300603a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:01.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.631+0000 7fe036d16700 1 -- 172.21.15.27:0/3878374267 shutdown_connections 2021-11-03T17:11:01.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:01.631+0000 7fe036d16700 1 -- 172.21.15.27:0/3878374267 wait complete. 2021-11-03T17:11:01.845 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:01 smithi027 conmon[32322]: cluster 2021-11-03T17:11:00.563160+0000 2021-11-03T17:11:01.845 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:01 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14162) 181 : cluster [DBG] pgmap v152: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:01.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:01 smithi115 conmon[34670]: cluster 2021-11-03T17:11:00.563160+0000 mgr.smithi027.ufhhvb (mgr.14162) 181 : cluster [DBG] pgmap v152: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:02.226 INFO:teuthology.orchestra.run.smithi027.stdout:60129542168 2021-11-03T17:11:02.226 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph tell osd.2 flush_pg_stats 2021-11-03T17:11:02.516 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c/mon.smithi027/config 2021-11-03T17:11:03.903 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:03 smithi027 conmon[32322]: cluster 2021-11-03T17:11:02.563456+0000 mgr.smithi027.ufhhvb (mgr.14162) 182 : cluster [DBG] pgmap v153: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:03.955 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:03 smithi115 conmon[34670]: cluster 2021-11-03T17:11:02.563456+0000 mgr.smithi027.ufhhvb (mgr.14162) 182 : cluster [DBG] pgmap v153: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:04.096 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.101+0000 7f5cd2479700 1 -- 172.21.15.27:0/930549888 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ccc0f56e0 msgr2=0x7f5ccc0f5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:04.097 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.101+0000 7f5cd2479700 1 --2- 172.21.15.27:0/930549888 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ccc0f56e0 0x7f5ccc0f5b00 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f5cbc004660 tx=0x7f5cbc009b30).stop 2021-11-03T17:11:04.097 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.101+0000 7f5cd2479700 1 -- 172.21.15.27:0/930549888 shutdown_connections 2021-11-03T17:11:04.097 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.101+0000 7f5cd2479700 1 --2- 172.21.15.27:0/930549888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5ccc0f6980 0x7f5ccc0f6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:04.098 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.101+0000 7f5cd2479700 1 --2- 172.21.15.27:0/930549888 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ccc0f56e0 0x7f5ccc0f5b00 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:04.098 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.101+0000 7f5cd2479700 1 -- 172.21.15.27:0/930549888 >> 172.21.15.27:0/930549888 conn(0x7f5ccc0f0e20 msgr2=0x7f5ccc0f3260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:04.098 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.101+0000 7f5cd2479700 1 -- 172.21.15.27:0/930549888 shutdown_connections 2021-11-03T17:11:04.098 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.101+0000 7f5cd2479700 1 -- 172.21.15.27:0/930549888 wait complete. 2021-11-03T17:11:04.099 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.102+0000 7f5cd2479700 1 Processor -- start 2021-11-03T17:11:04.099 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.102+0000 7f5cd2479700 1 -- start start 2021-11-03T17:11:04.101 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.102+0000 7f5cd2479700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5ccc0f56e0 0x7f5ccc105fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:04.102 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.102+0000 7f5cd2479700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ccc0f6980 0x7f5ccc1064d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:04.102 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.102+0000 7f5cd2479700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ccc106a80 con 0x7f5ccc0f6980 2021-11-03T17:11:04.102 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.102+0000 7f5cd2479700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ccc106bc0 con 0x7f5ccc0f56e0 2021-11-03T17:11:04.102 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.102+0000 7f5ccb7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ccc0f6980 0x7f5ccc1064d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:04.103 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.102+0000 7f5ccb7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ccc0f6980 0x7f5ccc1064d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54768/0 (socket says 172.21.15.27:54768) 2021-11-03T17:11:04.103 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.102+0000 7f5ccb7fe700 1 -- 172.21.15.27:0/2973704068 learned_addr learned my addr 172.21.15.27:0/2973704068 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:11:04.103 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.102+0000 7f5ccbfff700 1 --2- 172.21.15.27:0/2973704068 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5ccc0f56e0 0x7f5ccc105fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:04.103 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.103+0000 7f5ccb7fe700 1 -- 172.21.15.27:0/2973704068 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5ccc0f56e0 msgr2=0x7f5ccc105fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:04.104 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.103+0000 7f5ccb7fe700 1 --2- 172.21.15.27:0/2973704068 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5ccc0f56e0 0x7f5ccc105fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:04.104 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.103+0000 7f5ccb7fe700 1 -- 172.21.15.27:0/2973704068 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cbc005040 con 0x7f5ccc0f6980 2021-11-03T17:11:04.104 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.103+0000 7f5ccb7fe700 1 --2- 172.21.15.27:0/2973704068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ccc0f6980 0x7f5ccc1064d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f5cb40026d0 tx=0x7f5cb4009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:04.104 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.103+0000 7f5cc8ff9700 1 -- 172.21.15.27:0/2973704068 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cb40079c0 con 0x7f5ccc0f6980 2021-11-03T17:11:04.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.103+0000 7f5cd2479700 1 -- 172.21.15.27:0/2973704068 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ccc107560 con 0x7f5ccc0f6980 2021-11-03T17:11:04.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.103+0000 7f5cc8ff9700 1 -- 172.21.15.27:0/2973704068 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f5cb400f040 con 0x7f5ccc0f6980 2021-11-03T17:11:04.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.103+0000 7f5cc8ff9700 1 -- 172.21.15.27:0/2973704068 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cb4014820 con 0x7f5ccc0f6980 2021-11-03T17:11:04.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.103+0000 7f5cd2479700 1 -- 172.21.15.27:0/2973704068 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ccc107b20 con 0x7f5ccc0f6980 2021-11-03T17:11:04.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.104+0000 7f5cd2479700 1 -- 172.21.15.27:0/2973704068 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5ccc05dcf0 con 0x7f5ccc0f6980 2021-11-03T17:11:04.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.104+0000 7f5cc8ff9700 1 -- 172.21.15.27:0/2973704068 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f5cb40090d0 con 0x7f5ccc0f6980 2021-11-03T17:11:04.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.104+0000 7f5cc8ff9700 1 --2- 172.21.15.27:0/2973704068 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f5cb805e4b0 0x7f5cb8060970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:04.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.104+0000 7f5cc8ff9700 1 -- 172.21.15.27:0/2973704068 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4519+0+0 (secure 0 0 0) 0x7f5cb4075bc0 con 0x7f5ccc0f6980 2021-11-03T17:11:04.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.105+0000 7f5ccbfff700 1 --2- 172.21.15.27:0/2973704068 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f5cb805e4b0 0x7f5cb8060970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:04.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.105+0000 7f5cc8ff9700 1 --2- 172.21.15.27:0/2973704068 >> [v2:172.21.15.27:6818/329182726,v1:172.21.15.27:6819/329182726] conn(0x7f5cb80641d0 0x7f5cb8066610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:04.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.105+0000 7f5cc8ff9700 1 -- 172.21.15.27:0/2973704068 --> [v2:172.21.15.27:6818/329182726,v1:172.21.15.27:6819/329182726] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5cb8066c90 con 0x7f5cb80641d0 2021-11-03T17:11:04.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.105+0000 7f5cc8ff9700 1 -- 172.21.15.27:0/2973704068 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5cb407b220 con 0x7f5ccc0f6980 2021-11-03T17:11:04.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.105+0000 7f5cd0a16700 1 --2- 172.21.15.27:0/2973704068 >> [v2:172.21.15.27:6818/329182726,v1:172.21.15.27:6819/329182726] conn(0x7f5cb80641d0 0x7f5cb8066610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:04.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.105+0000 7f5ccbfff700 1 --2- 172.21.15.27:0/2973704068 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f5cb805e4b0 0x7f5cb8060970 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f5cbc005010 tx=0x7f5cbc009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:04.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.105+0000 7f5cd0a16700 1 --2- 172.21.15.27:0/2973704068 >> [v2:172.21.15.27:6818/329182726,v1:172.21.15.27:6819/329182726] conn(0x7f5cb80641d0 0x7f5cb8066610 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:04.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.106+0000 7f5cc8ff9700 1 -- 172.21.15.27:0/2973704068 <== osd.2 v2:172.21.15.27:6818/329182726 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7f5cb8066c90 con 0x7f5cb80641d0 2021-11-03T17:11:04.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.120+0000 7f5cd2479700 1 -- 172.21.15.27:0/2973704068 --> [v2:172.21.15.27:6818/329182726,v1:172.21.15.27:6819/329182726] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5ccc107fe0 con 0x7f5cb80641d0 2021-11-03T17:11:04.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.120+0000 7f5cc8ff9700 1 -- 172.21.15.27:0/2973704068 <== osd.2 v2:172.21.15.27:6818/329182726 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5ccc107fe0 con 0x7f5cb80641d0 2021-11-03T17:11:04.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.121+0000 7f5cd2479700 1 -- 172.21.15.27:0/2973704068 >> [v2:172.21.15.27:6818/329182726,v1:172.21.15.27:6819/329182726] conn(0x7f5cb80641d0 msgr2=0x7f5cb8066610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:04.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.121+0000 7f5cd2479700 1 --2- 172.21.15.27:0/2973704068 >> [v2:172.21.15.27:6818/329182726,v1:172.21.15.27:6819/329182726] conn(0x7f5cb80641d0 0x7f5cb8066610 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:04.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.121+0000 7f5cd2479700 1 -- 172.21.15.27:0/2973704068 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f5cb805e4b0 msgr2=0x7f5cb8060970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:04.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.121+0000 7f5cd2479700 1 --2- 172.21.15.27:0/2973704068 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f5cb805e4b0 0x7f5cb8060970 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f5cbc005010 tx=0x7f5cbc009b30).stop 2021-11-03T17:11:04.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.121+0000 7f5cd2479700 1 -- 172.21.15.27:0/2973704068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ccc0f6980 msgr2=0x7f5ccc1064d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:04.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.121+0000 7f5cd2479700 1 --2- 172.21.15.27:0/2973704068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ccc0f6980 0x7f5ccc1064d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f5cb40026d0 tx=0x7f5cb4009de0).stop 2021-11-03T17:11:04.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.121+0000 7f5cd2479700 1 -- 172.21.15.27:0/2973704068 shutdown_connections 2021-11-03T17:11:04.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.121+0000 7f5cd2479700 1 --2- 172.21.15.27:0/2973704068 >> [v2:172.21.15.27:6818/329182726,v1:172.21.15.27:6819/329182726] conn(0x7f5cb80641d0 0x7f5cb8066610 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:04.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.121+0000 7f5cd2479700 1 --2- 172.21.15.27:0/2973704068 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f5cb805e4b0 0x7f5cb8060970 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:04.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.121+0000 7f5cd2479700 1 --2- 172.21.15.27:0/2973704068 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5ccc0f56e0 0x7f5ccc105fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:04.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.121+0000 7f5cd2479700 1 --2- 172.21.15.27:0/2973704068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ccc0f6980 0x7f5ccc1064d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:04.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.121+0000 7f5cd2479700 1 -- 172.21.15.27:0/2973704068 >> 172.21.15.27:0/2973704068 conn(0x7f5ccc0f0e20 msgr2=0x7f5ccc0fa3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:04.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.121+0000 7f5cd2479700 1 -- 172.21.15.27:0/2973704068 shutdown_connections 2021-11-03T17:11:04.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:04.121+0000 7f5cd2479700 1 -- 172.21.15.27:0/2973704068 wait complete. 2021-11-03T17:11:04.718 INFO:teuthology.orchestra.run.smithi027.stdout:81604378645 2021-11-03T17:11:04.719 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph tell osd.3 flush_pg_stats 2021-11-03T17:11:05.003 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c/mon.smithi027/config 2021-11-03T17:11:05.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:05 smithi115 conmon[34670]: cluster 2021-11-03T17:11:04.563821+0000 mgr.smithi027.ufhhvb (mgr.14162) 183 : cluster [DBG] pgmap v154: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:06.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:05 smithi027 conmon[32322]: cluster 2021-11-03T17:11:04.563821+0000 mgr.smithi027.ufhhvb (mgr.14162) 183 : cluster [DBG] pgmap v154: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:06.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.526+0000 7f0f9eb3c700 1 -- 172.21.15.27:0/1090121487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f980f56e0 msgr2=0x7f0f980f5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:06.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.526+0000 7f0f9eb3c700 1 --2- 172.21.15.27:0/1090121487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f980f56e0 0x7f0f980f5b00 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f0f8c004660 tx=0x7f0f8c009b30).stop 2021-11-03T17:11:06.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.526+0000 7f0f9eb3c700 1 -- 172.21.15.27:0/1090121487 shutdown_connections 2021-11-03T17:11:06.526 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.526+0000 7f0f9eb3c700 1 --2- 172.21.15.27:0/1090121487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f980f6980 0x7f0f980f6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:11:06.526 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.526+0000 7f0f9eb3c700 1 --2- 172.21.15.27:0/1090121487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f980f56e0 0x7f0f980f5b00 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:06.526 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.526+0000 7f0f9eb3c700 1 -- 172.21.15.27:0/1090121487 >> 172.21.15.27:0/1090121487 conn(0x7f0f980f0e20 msgr2=0x7f0f980f3260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:06.526 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.526+0000 7f0f9eb3c700 1 -- 172.21.15.27:0/1090121487 shutdown_connections 2021-11-03T17:11:06.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.526+0000 7f0f9eb3c700 1 -- 172.21.15.27:0/1090121487 wait complete. 2021-11-03T17:11:06.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.527+0000 7f0f9eb3c700 1 Processor -- start 2021-11-03T17:11:06.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.527+0000 7f0f9eb3c700 1 -- start start 2021-11-03T17:11:06.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.527+0000 7f0f9eb3c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f980f56e0 0x7f0f981070e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:06.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.527+0000 7f0f9eb3c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f980f6980 0x7f0f981075d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:06.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.527+0000 7f0f9eb3c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f98107c10 con 0x7f0f980f56e0 2021-11-03T17:11:06.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.527+0000 7f0f9eb3c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f981084c0 con 0x7f0f980f6980 2021-11-03T17:11:06.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.527+0000 7f0f9c8d8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f980f56e0 0x7f0f981070e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:06.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.527+0000 7f0f9c8d8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f980f56e0 0x7f0f981070e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54778/0 (socket says 172.21.15.27:54778) 2021-11-03T17:11:06.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.527+0000 7f0f9c8d8700 1 -- 172.21.15.27:0/3946837045 learned_addr learned my addr 172.21.15.27:0/3946837045 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:11:06.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.527+0000 7f0f97fff700 1 --2- 172.21.15.27:0/3946837045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f980f6980 0x7f0f981075d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:06.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.528+0000 7f0f9c8d8700 1 -- 172.21.15.27:0/3946837045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f980f6980 msgr2=0x7f0f981075d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:06.530 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.528+0000 7f0f9c8d8700 1 --2- 172.21.15.27:0/3946837045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f980f6980 0x7f0f981075d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:06.530 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.528+0000 7f0f9c8d8700 1 -- 172.21.15.27:0/3946837045 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f8c005040 con 0x7f0f980f56e0 2021-11-03T17:11:06.530 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.528+0000 7f0f9c8d8700 1 --2- 172.21.15.27:0/3946837045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f980f56e0 0x7f0f981070e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f0f8c007a60 tx=0x7f0f8c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:06.530 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.528+0000 7f0f957fa700 1 -- 172.21.15.27:0/3946837045 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f8c018070 con 0x7f0f980f56e0 2021-11-03T17:11:06.531 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.528+0000 7f0f957fa700 1 -- 172.21.15.27:0/3946837045 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f0f8c013410 con 0x7f0f980f56e0 2021-11-03T17:11:06.531 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.528+0000 7f0f957fa700 1 -- 172.21.15.27:0/3946837045 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f8c01c620 con 0x7f0f980f56e0 2021-11-03T17:11:06.531 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.528+0000 7f0f9eb3c700 1 -- 172.21.15.27:0/3946837045 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f98108740 con 0x7f0f980f56e0 2021-11-03T17:11:06.531 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.528+0000 7f0f9eb3c700 1 -- 172.21.15.27:0/3946837045 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f98108c50 con 0x7f0f980f56e0 2021-11-03T17:11:06.532 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.529+0000 7f0f9eb3c700 1 -- 172.21.15.27:0/3946837045 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0f9805dcf0 con 0x7f0f980f56e0 2021-11-03T17:11:06.532 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.529+0000 7f0f957fa700 1 -- 172.21.15.27:0/3946837045 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f0f8c01d940 con 0x7f0f980f56e0 2021-11-03T17:11:06.533 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.529+0000 7f0f957fa700 1 --2- 172.21.15.27:0/3946837045 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f0f8805c2a0 0x7f0f8805e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:06.534 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.530+0000 7f0f957fa700 1 -- 172.21.15.27:0/3946837045 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4519+0+0 (secure 0 0 0) 0x7f0f8c077b30 con 0x7f0f980f56e0 2021-11-03T17:11:06.534 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.530+0000 7f0f97fff700 1 --2- 172.21.15.27:0/3946837045 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f0f8805c2a0 0x7f0f8805e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:06.534 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.530+0000 7f0f957fa700 1 --2- 172.21.15.27:0/3946837045 >> [v2:172.21.15.115:6800/4026713633,v1:172.21.15.115:6801/4026713633] conn(0x7f0f88061fc0 0x7f0f88064400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:06.534 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.530+0000 7f0f957fa700 1 -- 172.21.15.27:0/3946837045 --> [v2:172.21.15.115:6800/4026713633,v1:172.21.15.115:6801/4026713633] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0f88064a80 con 0x7f0f88061fc0 2021-11-03T17:11:06.535 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.530+0000 7f0f957fa700 1 -- 172.21.15.27:0/3946837045 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f8c07c1c0 con 0x7f0f980f56e0 2021-11-03T17:11:06.535 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.530+0000 7f0f97fff700 1 --2- 172.21.15.27:0/3946837045 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f0f8805c2a0 0x7f0f8805e760 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f0f8400b790 tx=0x7f0f84008040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:06.535 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.530+0000 7f0f9d0d9700 1 --2- 172.21.15.27:0/3946837045 >> [v2:172.21.15.115:6800/4026713633,v1:172.21.15.115:6801/4026713633] conn(0x7f0f88061fc0 0x7f0f88064400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:06.535 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.530+0000 7f0f9d0d9700 1 --2- 172.21.15.27:0/3946837045 >> [v2:172.21.15.115:6800/4026713633,v1:172.21.15.115:6801/4026713633] conn(0x7f0f88061fc0 0x7f0f88064400 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:06.535 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.531+0000 7f0f957fa700 1 -- 172.21.15.27:0/3946837045 <== osd.3 v2:172.21.15.115:6800/4026713633 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7f0f88064a80 con 0x7f0f88061fc0 2021-11-03T17:11:06.541 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.546+0000 7f0f9eb3c700 1 -- 172.21.15.27:0/3946837045 --> [v2:172.21.15.115:6800/4026713633,v1:172.21.15.115:6801/4026713633] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0f981091c0 con 0x7f0f88061fc0 2021-11-03T17:11:06.542 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.546+0000 7f0f957fa700 1 -- 172.21.15.27:0/3946837045 <== osd.3 v2:172.21.15.115:6800/4026713633 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0f981091c0 con 0x7f0f88061fc0 2021-11-03T17:11:06.543 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.547+0000 7f0f9eb3c700 1 -- 172.21.15.27:0/3946837045 >> [v2:172.21.15.115:6800/4026713633,v1:172.21.15.115:6801/4026713633] conn(0x7f0f88061fc0 msgr2=0x7f0f88064400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:06.544 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.547+0000 7f0f9eb3c700 1 --2- 172.21.15.27:0/3946837045 >> [v2:172.21.15.115:6800/4026713633,v1:172.21.15.115:6801/4026713633] conn(0x7f0f88061fc0 0x7f0f88064400 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:06.544 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.547+0000 7f0f9eb3c700 1 -- 172.21.15.27:0/3946837045 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f0f8805c2a0 msgr2=0x7f0f8805e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:06.544 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.547+0000 7f0f9eb3c700 1 --2- 172.21.15.27:0/3946837045 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f0f8805c2a0 0x7f0f8805e760 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f0f8400b790 tx=0x7f0f84008040).stop 2021-11-03T17:11:06.544 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.547+0000 7f0f9eb3c700 1 -- 172.21.15.27:0/3946837045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f980f56e0 msgr2=0x7f0f981070e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:06.545 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.547+0000 7f0f9eb3c700 1 --2- 172.21.15.27:0/3946837045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f980f56e0 0x7f0f981070e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f0f8c007a60 tx=0x7f0f8c009b30).stop 2021-11-03T17:11:06.545 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.547+0000 7f0f9eb3c700 1 -- 172.21.15.27:0/3946837045 shutdown_connections 2021-11-03T17:11:06.545 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.547+0000 7f0f9eb3c700 1 --2- 172.21.15.27:0/3946837045 >> [v2:172.21.15.115:6800/4026713633,v1:172.21.15.115:6801/4026713633] conn(0x7f0f88061fc0 0x7f0f88064400 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:06.545 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.547+0000 7f0f9eb3c700 1 --2- 172.21.15.27:0/3946837045 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f0f8805c2a0 0x7f0f8805e760 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:06.546 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.547+0000 7f0f9eb3c700 1 --2- 172.21.15.27:0/3946837045 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f980f56e0 0x7f0f981070e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:06.546 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.547+0000 7f0f9eb3c700 1 --2- 172.21.15.27:0/3946837045 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f980f6980 0x7f0f981075d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:06.546 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.547+0000 7f0f9eb3c700 1 -- 172.21.15.27:0/3946837045 >> 172.21.15.27:0/3946837045 conn(0x7f0f980f0e20 msgr2=0x7f0f9805fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:06.546 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.547+0000 7f0f9eb3c700 1 -- 172.21.15.27:0/3946837045 shutdown_connections 2021-11-03T17:11:06.547 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:06.547+0000 7f0f9eb3c700 1 -- 172.21.15.27:0/3946837045 wait complete. 2021-11-03T17:11:07.135 INFO:teuthology.orchestra.run.smithi027.stdout:115964117008 2021-11-03T17:11:07.136 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph tell osd.4 flush_pg_stats 2021-11-03T17:11:07.425 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c/mon.smithi027/config 2021-11-03T17:11:07.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:07 smithi115 conmon[34670]: cluster 2021-11-03T17:11:06.564143+0000 mgr.smithi027.ufhhvb (mgr.14162) 184 : cluster [DBG] pgmap v155: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:08.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:07 smithi027 conmon[32322]: cluster 2021-11-03T17:11:06.564143+0000 2021-11-03T17:11:08.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:07 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14162) 184 : cluster [DBG] pgmap v155: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:09.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.108+0000 7fda2bd48700 1 -- 172.21.15.27:0/2120607455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda240f67c0 msgr2=0x7fda240f6be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:09.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.108+0000 7fda2bd48700 1 --2- 172.21.15.27:0/2120607455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda240f67c0 0x7fda240f6be0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7fda1c0099e0 tx=0x7fda1c004d10).stop 2021-11-03T17:11:09.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.108+0000 7fda2bd48700 1 -- 172.21.15.27:0/2120607455 shutdown_connections 2021-11-03T17:11:09.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.108+0000 7fda2bd48700 1 --2- 172.21.15.27:0/2120607455 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda240f7a60 0x7fda240f7ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:11:09.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.108+0000 7fda2bd48700 1 --2- 172.21.15.27:0/2120607455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda240f67c0 0x7fda240f6be0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:09.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.108+0000 7fda2bd48700 1 -- 172.21.15.27:0/2120607455 >> 172.21.15.27:0/2120607455 conn(0x7fda240f1f20 msgr2=0x7fda240f4340 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:09.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.108+0000 7fda2bd48700 1 -- 172.21.15.27:0/2120607455 shutdown_connections 2021-11-03T17:11:09.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.108+0000 7fda2bd48700 1 -- 172.21.15.27:0/2120607455 wait complete. 2021-11-03T17:11:09.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.109+0000 7fda2bd48700 1 Processor -- start 2021-11-03T17:11:09.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.109+0000 7fda2bd48700 1 -- start start 2021-11-03T17:11:09.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.109+0000 7fda2bd48700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda240f67c0 0x7fda2406b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:09.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.109+0000 7fda2bd48700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda240f7a60 0x7fda24069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:09.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.109+0000 7fda2bd48700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2406a410 con 0x7fda240f7a60 2021-11-03T17:11:09.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.109+0000 7fda2bd48700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2406a550 con 0x7fda240f67c0 2021-11-03T17:11:09.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.109+0000 7fda292e3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda240f7a60 0x7fda24069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:09.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.109+0000 7fda292e3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda240f7a60 0x7fda24069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54792/0 (socket says 172.21.15.27:54792) 2021-11-03T17:11:09.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.109+0000 7fda292e3700 1 -- 172.21.15.27:0/2137234080 learned_addr learned my addr 172.21.15.27:0/2137234080 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:11:09.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.109+0000 7fda29ae4700 1 --2- 172.21.15.27:0/2137234080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda240f67c0 0x7fda2406b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:09.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.109+0000 7fda292e3700 1 -- 172.21.15.27:0/2137234080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda240f67c0 msgr2=0x7fda2406b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:09.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.109+0000 7fda292e3700 1 --2- 172.21.15.27:0/2137234080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda240f67c0 0x7fda2406b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:09.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.109+0000 7fda292e3700 1 -- 172.21.15.27:0/2137234080 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda1c009640 con 0x7fda240f7a60 2021-11-03T17:11:09.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.109+0000 7fda29ae4700 1 --2- 172.21.15.27:0/2137234080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda240f67c0 0x7fda2406b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:11:09.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.110+0000 7fda292e3700 1 --2- 172.21.15.27:0/2137234080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda240f7a60 0x7fda24069e20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fda1800b410 tx=0x7fda18006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:09.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.110+0000 7fda227fc700 1 -- 172.21.15.27:0/2137234080 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda180125f0 con 0x7fda240f7a60 2021-11-03T17:11:09.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.110+0000 7fda227fc700 1 -- 172.21.15.27:0/2137234080 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7fda18012be0 con 0x7fda240f7a60 2021-11-03T17:11:09.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.110+0000 7fda2bd48700 1 -- 172.21.15.27:0/2137234080 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda2406a830 con 0x7fda240f7a60 2021-11-03T17:11:09.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.110+0000 7fda227fc700 1 -- 172.21.15.27:0/2137234080 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda18002a00 con 0x7fda240f7a60 2021-11-03T17:11:09.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.110+0000 7fda2bd48700 1 -- 172.21.15.27:0/2137234080 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda2410f250 con 0x7fda240f7a60 2021-11-03T17:11:09.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.111+0000 7fda2bd48700 1 -- 172.21.15.27:0/2137234080 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fda2405dd30 con 0x7fda240f7a60 2021-11-03T17:11:09.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.111+0000 7fda227fc700 1 -- 172.21.15.27:0/2137234080 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fda1800f030 con 0x7fda240f7a60 2021-11-03T17:11:09.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.111+0000 7fda227fc700 1 --2- 172.21.15.27:0/2137234080 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fda0c05c2a0 0x7fda0c05e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:09.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.112+0000 7fda227fc700 1 -- 172.21.15.27:0/2137234080 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4519+0+0 (secure 0 0 0) 0x7fda18075fe0 con 0x7fda240f7a60 2021-11-03T17:11:09.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.112+0000 7fda29ae4700 1 --2- 172.21.15.27:0/2137234080 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fda0c05c2a0 0x7fda0c05e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:09.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.112+0000 7fda227fc700 1 --2- 172.21.15.27:0/2137234080 >> [v2:172.21.15.115:6808/1601394331,v1:172.21.15.115:6809/1601394331] conn(0x7fda0c062010 0x7fda0c064450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:09.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.112+0000 7fda227fc700 1 -- 172.21.15.27:0/2137234080 --> [v2:172.21.15.115:6808/1601394331,v1:172.21.15.115:6809/1601394331] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fda0c064ad0 con 0x7fda0c062010 2021-11-03T17:11:09.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.112+0000 7fda227fc700 1 -- 172.21.15.27:0/2137234080 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda18078650 con 0x7fda240f7a60 2021-11-03T17:11:09.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.112+0000 7fda29ae4700 1 --2- 172.21.15.27:0/2137234080 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fda0c05c2a0 0x7fda0c05e760 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fda1c009f50 tx=0x7fda1c004d10).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:09.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.112+0000 7fda2a2e5700 1 --2- 172.21.15.27:0/2137234080 >> [v2:172.21.15.115:6808/1601394331,v1:172.21.15.115:6809/1601394331] conn(0x7fda0c062010 0x7fda0c064450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:09.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.112+0000 7fda2a2e5700 1 --2- 172.21.15.27:0/2137234080 >> [v2:172.21.15.115:6808/1601394331,v1:172.21.15.115:6809/1601394331] conn(0x7fda0c062010 0x7fda0c064450 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:09.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.113+0000 7fda227fc700 1 -- 172.21.15.27:0/2137234080 <== osd.4 v2:172.21.15.115:6808/1601394331 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7fda0c064ad0 con 0x7fda0c062010 2021-11-03T17:11:09.123 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.127+0000 7fda2bd48700 1 -- 172.21.15.27:0/2137234080 --> [v2:172.21.15.115:6808/1601394331,v1:172.21.15.115:6809/1601394331] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fda24058150 con 0x7fda0c062010 2021-11-03T17:11:09.124 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.128+0000 7fda227fc700 1 -- 172.21.15.27:0/2137234080 <== osd.4 v2:172.21.15.115:6808/1601394331 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fda24058150 con 0x7fda0c062010 2021-11-03T17:11:09.125 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.128+0000 7fda2bd48700 1 -- 172.21.15.27:0/2137234080 >> [v2:172.21.15.115:6808/1601394331,v1:172.21.15.115:6809/1601394331] conn(0x7fda0c062010 msgr2=0x7fda0c064450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:09.125 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.128+0000 7fda2bd48700 1 --2- 172.21.15.27:0/2137234080 >> [v2:172.21.15.115:6808/1601394331,v1:172.21.15.115:6809/1601394331] conn(0x7fda0c062010 0x7fda0c064450 crc :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:09.125 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.128+0000 7fda2bd48700 1 -- 172.21.15.27:0/2137234080 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fda0c05c2a0 msgr2=0x7fda0c05e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:09.126 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.128+0000 7fda2bd48700 1 --2- 172.21.15.27:0/2137234080 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fda0c05c2a0 0x7fda0c05e760 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7fda1c009f50 tx=0x7fda1c004d10).stop 2021-11-03T17:11:09.126 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.128+0000 7fda2bd48700 1 -- 172.21.15.27:0/2137234080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda240f7a60 msgr2=0x7fda24069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:09.126 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.128+0000 7fda2bd48700 1 --2- 172.21.15.27:0/2137234080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda240f7a60 0x7fda24069e20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fda1800b410 tx=0x7fda18006ef0).stop 2021-11-03T17:11:09.126 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.128+0000 7fda2bd48700 1 -- 172.21.15.27:0/2137234080 shutdown_connections 2021-11-03T17:11:09.127 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.128+0000 7fda2bd48700 1 --2- 172.21.15.27:0/2137234080 >> [v2:172.21.15.115:6808/1601394331,v1:172.21.15.115:6809/1601394331] conn(0x7fda0c062010 0x7fda0c064450 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:09.127 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.128+0000 7fda2bd48700 1 --2- 172.21.15.27:0/2137234080 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fda0c05c2a0 0x7fda0c05e760 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:09.127 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.128+0000 7fda2bd48700 1 --2- 172.21.15.27:0/2137234080 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda240f67c0 0x7fda2406b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:09.127 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.128+0000 7fda2bd48700 1 --2- 172.21.15.27:0/2137234080 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda240f7a60 0x7fda24069e20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:09.128 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.128+0000 7fda2bd48700 1 -- 172.21.15.27:0/2137234080 >> 172.21.15.27:0/2137234080 conn(0x7fda240f1f20 msgr2=0x7fda240fb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:09.128 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.128+0000 7fda2bd48700 1 -- 172.21.15.27:0/2137234080 shutdown_connections 2021-11-03T17:11:09.128 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:09.128+0000 7fda2bd48700 1 -- 172.21.15.27:0/2137234080 wait complete. 2021-11-03T17:11:09.700 INFO:teuthology.orchestra.run.smithi027.stdout:141733920781 2021-11-03T17:11:09.701 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph tell osd.5 flush_pg_stats 2021-11-03T17:11:09.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:09 smithi115 conmon[34670]: cluster 2021-11-03T17:11:08.564587+0000 mgr.smithi027.ufhhvb (mgr.14162) 185 : cluster [DBG] pgmap v156: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:09.989 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c/mon.smithi027/config 2021-11-03T17:11:10.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:09 smithi027 conmon[32322]: cluster 2021-11-03T17:11:08.564587+0000 mgr.smithi027.ufhhvb (mgr.14162) 185 : cluster [DBG] pgmap v156: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:11.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.558+0000 7f49eb2a7700 1 -- 172.21.15.27:0/2189493201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49e40f7a80 msgr2=0x7f49e40f7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:11.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.558+0000 7f49eb2a7700 1 --2- 172.21.15.27:0/2189493201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49e40f7a80 0x7f49e40f7ee0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f49d8004660 tx=0x7f49d8009b30).stop 2021-11-03T17:11:11.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.558+0000 7f49eb2a7700 1 -- 172.21.15.27:0/2189493201 shutdown_connections 2021-11-03T17:11:11.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.558+0000 7f49eb2a7700 1 --2- 172.21.15.27:0/2189493201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49e40f7a80 0x7f49e40f7ee0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:11.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.558+0000 7f49eb2a7700 1 --2- 172.21.15.27:0/2189493201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49e40f67e0 0x7f49e40f6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:11.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.558+0000 7f49eb2a7700 1 -- 172.21.15.27:0/2189493201 >> 172.21.15.27:0/2189493201 conn(0x7f49e40f1f40 msgr2=0x7f49e40f4360 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:11.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.558+0000 7f49eb2a7700 1 -- 172.21.15.27:0/2189493201 shutdown_connections 2021-11-03T17:11:11.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.558+0000 7f49eb2a7700 1 -- 172.21.15.27:0/2189493201 wait complete. 2021-11-03T17:11:11.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.558+0000 7f49eb2a7700 1 Processor -- start 2021-11-03T17:11:11.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.558+0000 7f49eb2a7700 1 -- start start 2021-11-03T17:11:11.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.559+0000 7f49eb2a7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49e40f67e0 0x7f49e406b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:11.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.559+0000 7f49eb2a7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49e40f7a80 0x7f49e4069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:11.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.559+0000 7f49eb2a7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e406a380 con 0x7f49e40f7a80 2021-11-03T17:11:11.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.559+0000 7f49eb2a7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49e406a4c0 con 0x7f49e40f67e0 2021-11-03T17:11:11.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.559+0000 7f49e8842700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49e40f7a80 0x7f49e4069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:11.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.559+0000 7f49e8842700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49e40f7a80 0x7f49e4069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54802/0 (socket says 172.21.15.27:54802) 2021-11-03T17:11:11.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.559+0000 7f49e8842700 1 -- 172.21.15.27:0/2831722805 learned_addr learned my addr 172.21.15.27:0/2831722805 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:11:11.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.559+0000 7f49e9043700 1 --2- 172.21.15.27:0/2831722805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49e40f67e0 0x7f49e406b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:11.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.559+0000 7f49e8842700 1 -- 172.21.15.27:0/2831722805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49e40f67e0 msgr2=0x7f49e406b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:11.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.559+0000 7f49e8842700 1 --2- 172.21.15.27:0/2831722805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49e40f67e0 0x7f49e406b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:11.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.559+0000 7f49e8842700 1 -- 172.21.15.27:0/2831722805 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d8005040 con 0x7f49e40f7a80 2021-11-03T17:11:11.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.559+0000 7f49e9043700 1 --2- 172.21.15.27:0/2831722805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49e40f67e0 0x7f49e406b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:11:11.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.559+0000 7f49e8842700 1 --2- 172.21.15.27:0/2831722805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49e40f7a80 0x7f49e4069e20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f49d8009990 tx=0x7f49d8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:11.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.559+0000 7f49e1ffb700 1 -- 172.21.15.27:0/2831722805 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49d8018070 con 0x7f49e40f7a80 2021-11-03T17:11:11.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.559+0000 7f49e1ffb700 1 -- 172.21.15.27:0/2831722805 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f49d8013410 con 0x7f49e40f7a80 2021-11-03T17:11:11.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.559+0000 7f49e1ffb700 1 -- 172.21.15.27:0/2831722805 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49d801c620 con 0x7f49e40f7a80 2021-11-03T17:11:11.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.559+0000 7f49eb2a7700 1 -- 172.21.15.27:0/2831722805 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49e406a740 con 0x7f49e40f7a80 2021-11-03T17:11:11.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.560+0000 7f49eb2a7700 1 -- 172.21.15.27:0/2831722805 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49e406aca0 con 0x7f49e40f7a80 2021-11-03T17:11:11.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.560+0000 7f49eb2a7700 1 -- 172.21.15.27:0/2831722805 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f49e405dd10 con 0x7f49e40f7a80 2021-11-03T17:11:11.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.561+0000 7f49e1ffb700 1 -- 172.21.15.27:0/2831722805 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f49d801d420 con 0x7f49e40f7a80 2021-11-03T17:11:11.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.561+0000 7f49e1ffb700 1 --2- 172.21.15.27:0/2831722805 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f49cc05c2a0 0x7f49cc05e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:11.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.561+0000 7f49e1ffb700 1 -- 172.21.15.27:0/2831722805 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4519+0+0 (secure 0 0 0) 0x7f49d8077a40 con 0x7f49e40f7a80 2021-11-03T17:11:11.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.561+0000 7f49e9043700 1 --2- 172.21.15.27:0/2831722805 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f49cc05c2a0 0x7f49cc05e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:11.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.561+0000 7f49e1ffb700 1 --2- 172.21.15.27:0/2831722805 >> [v2:172.21.15.115:6816/2350455075,v1:172.21.15.115:6817/2350455075] conn(0x7f49cc061fc0 0x7f49cc064400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:11.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.561+0000 7f49e1ffb700 1 -- 172.21.15.27:0/2831722805 --> [v2:172.21.15.115:6816/2350455075,v1:172.21.15.115:6817/2350455075] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f49cc064a80 con 0x7f49cc061fc0 2021-11-03T17:11:11.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.561+0000 7f49e1ffb700 1 -- 172.21.15.27:0/2831722805 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49d807c220 con 0x7f49e40f7a80 2021-11-03T17:11:11.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.562+0000 7f49e9043700 1 --2- 172.21.15.27:0/2831722805 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f49cc05c2a0 0x7f49cc05e760 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f49e406b730 tx=0x7f49dc009040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:11.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.562+0000 7f49e9844700 1 --2- 172.21.15.27:0/2831722805 >> [v2:172.21.15.115:6816/2350455075,v1:172.21.15.115:6817/2350455075] conn(0x7f49cc061fc0 0x7f49cc064400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:11.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.562+0000 7f49e9844700 1 --2- 172.21.15.27:0/2831722805 >> [v2:172.21.15.115:6816/2350455075,v1:172.21.15.115:6817/2350455075] conn(0x7f49cc061fc0 0x7f49cc064400 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:11.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.563+0000 7f49e1ffb700 1 -- 172.21.15.27:0/2831722805 <== osd.5 v2:172.21.15.115:6816/2350455075 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23065 (crc 0 0 0) 0x7f49cc064a80 con 0x7f49cc061fc0 2021-11-03T17:11:11.573 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.577+0000 7f49eb2a7700 1 -- 172.21.15.27:0/2831722805 --> [v2:172.21.15.115:6816/2350455075,v1:172.21.15.115:6817/2350455075] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f49e40581d0 con 0x7f49cc061fc0 2021-11-03T17:11:11.573 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.578+0000 7f49e1ffb700 1 -- 172.21.15.27:0/2831722805 <== osd.5 v2:172.21.15.115:6816/2350455075 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f49e40581d0 con 0x7f49cc061fc0 2021-11-03T17:11:11.575 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.578+0000 7f49eb2a7700 1 -- 172.21.15.27:0/2831722805 >> [v2:172.21.15.115:6816/2350455075,v1:172.21.15.115:6817/2350455075] conn(0x7f49cc061fc0 msgr2=0x7f49cc064400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:11.575 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.578+0000 7f49eb2a7700 1 --2- 172.21.15.27:0/2831722805 >> [v2:172.21.15.115:6816/2350455075,v1:172.21.15.115:6817/2350455075] conn(0x7f49cc061fc0 0x7f49cc064400 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:11.575 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.578+0000 7f49eb2a7700 1 -- 172.21.15.27:0/2831722805 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f49cc05c2a0 msgr2=0x7f49cc05e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:11.575 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.578+0000 7f49eb2a7700 1 --2- 172.21.15.27:0/2831722805 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f49cc05c2a0 0x7f49cc05e760 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f49e406b730 tx=0x7f49dc009040).stop 2021-11-03T17:11:11.576 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.578+0000 7f49eb2a7700 1 -- 172.21.15.27:0/2831722805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49e40f7a80 msgr2=0x7f49e4069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:11.576 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.578+0000 7f49eb2a7700 1 --2- 172.21.15.27:0/2831722805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49e40f7a80 0x7f49e4069e20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f49d8009990 tx=0x7f49d8009b30).stop 2021-11-03T17:11:11.576 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.578+0000 7f49eb2a7700 1 -- 172.21.15.27:0/2831722805 shutdown_connections 2021-11-03T17:11:11.576 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.578+0000 7f49eb2a7700 1 --2- 172.21.15.27:0/2831722805 >> [v2:172.21.15.115:6816/2350455075,v1:172.21.15.115:6817/2350455075] conn(0x7f49cc061fc0 0x7f49cc064400 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:11.577 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.578+0000 7f49eb2a7700 1 --2- 172.21.15.27:0/2831722805 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f49cc05c2a0 0x7f49cc05e760 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:11.577 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.578+0000 7f49eb2a7700 1 --2- 172.21.15.27:0/2831722805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49e40f67e0 0x7f49e406b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:11.577 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.578+0000 7f49eb2a7700 1 --2- 172.21.15.27:0/2831722805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49e40f7a80 0x7f49e4069e20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:11.577 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.578+0000 7f49eb2a7700 1 -- 172.21.15.27:0/2831722805 >> 172.21.15.27:0/2831722805 conn(0x7f49e40f1f40 msgr2=0x7f49e40fb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:11.577 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.578+0000 7f49eb2a7700 1 -- 172.21.15.27:0/2831722805 shutdown_connections 2021-11-03T17:11:11.578 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:11.578+0000 7f49eb2a7700 1 -- 172.21.15.27:0/2831722805 wait complete. 2021-11-03T17:11:11.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:11 smithi115 conmon[34670]: cluster 2021-11-03T17:11:10.564876+0000 mgr.smithi027.ufhhvb (mgr.14162) 186 : cluster [DBG] pgmap v157: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:12.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:11 smithi027 conmon[32322]: cluster 2021-11-03T17:11:10.564876+0000 mgr.smithi027.ufhhvb (mgr.14162) 186 : cluster [DBG] pgmap v157: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:12.142 INFO:teuthology.orchestra.run.smithi027.stdout:163208757259 2021-11-03T17:11:12.142 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph osd last-stat-seq osd.0 2021-11-03T17:11:12.434 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c/mon.smithi027/config 2021-11-03T17:11:13.894 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:13 smithi027 conmon[32322]: cluster 2021-11-03T17:11:12.565190+0000 2021-11-03T17:11:13.894 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:13 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14162) 187 : cluster [DBG] pgmap v158: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:13.955 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:13 smithi115 conmon[34670]: cluster 2021-11-03T17:11:12.565190+0000 mgr.smithi027.ufhhvb (mgr.14162) 187 : cluster [DBG] pgmap v158: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:14.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.082+0000 7f5221753700 1 -- 172.21.15.27:0/1305051212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f521c060870 msgr2=0x7f521c0f93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:14.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.082+0000 7f5221753700 1 --2- 172.21.15.27:0/1305051212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f521c060870 0x7f521c0f93a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f52140099e0 tx=0x7f5214004d10).stop 2021-11-03T17:11:14.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.082+0000 7f5221753700 1 -- 172.21.15.27:0/1305051212 shutdown_connections 2021-11-03T17:11:14.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.082+0000 7f5221753700 1 --2- 172.21.15.27:0/1305051212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f521c0f9920 0x7f521c0fbd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:14.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.082+0000 7f5221753700 1 --2- 172.21.15.27:0/1305051212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f521c060870 0x7f521c0f93a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:14.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.082+0000 7f5221753700 1 -- 172.21.15.27:0/1305051212 >> 172.21.15.27:0/1305051212 conn(0x7f521c0f0e20 msgr2=0x7f521c0f3240 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:14.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.082+0000 7f5221753700 1 -- 172.21.15.27:0/1305051212 shutdown_connections 2021-11-03T17:11:14.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.082+0000 7f5221753700 1 -- 172.21.15.27:0/1305051212 wait complete. 2021-11-03T17:11:14.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.083+0000 7f5221753700 1 Processor -- start 2021-11-03T17:11:14.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.083+0000 7f5221753700 1 -- start start 2021-11-03T17:11:14.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.083+0000 7f5221753700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f521c060870 0x7f521c0f6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:14.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.083+0000 7f5221753700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f521c0f9920 0x7f521c0f52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:14.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.083+0000 7f5221753700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521c0f5800 con 0x7f521c060870 2021-11-03T17:11:14.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.083+0000 7f5221753700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f521c0f5940 con 0x7f521c0f9920 2021-11-03T17:11:14.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.083+0000 7f521affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f521c060870 0x7f521c0f6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:14.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.083+0000 7f521affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f521c060870 0x7f521c0f6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54812/0 (socket says 172.21.15.27:54812) 2021-11-03T17:11:14.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.083+0000 7f521affd700 1 -- 172.21.15.27:0/4178582506 learned_addr learned my addr 172.21.15.27:0/4178582506 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:11:14.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.083+0000 7f521a7fc700 1 --2- 172.21.15.27:0/4178582506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f521c0f9920 0x7f521c0f52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:14.086 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.083+0000 7f521affd700 1 -- 172.21.15.27:0/4178582506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f521c0f9920 msgr2=0x7f521c0f52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:14.086 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.083+0000 7f521affd700 1 --2- 172.21.15.27:0/4178582506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f521c0f9920 0x7f521c0f52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:14.086 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.083+0000 7f521affd700 1 -- 172.21.15.27:0/4178582506 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5214009640 con 0x7f521c060870 2021-11-03T17:11:14.086 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.083+0000 7f521a7fc700 1 --2- 172.21.15.27:0/4178582506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f521c0f9920 0x7f521c0f52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:11:14.086 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.083+0000 7f521affd700 1 --2- 172.21.15.27:0/4178582506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f521c060870 0x7f521c0f6e20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f521400b3b0 tx=0x7f5214004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:14.087 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.084+0000 7f52137fe700 1 -- 172.21.15.27:0/4178582506 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f521400e070 con 0x7f521c060870 2021-11-03T17:11:14.087 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.084+0000 7f52137fe700 1 -- 172.21.15.27:0/4178582506 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f5214007dc0 con 0x7f521c060870 2021-11-03T17:11:14.087 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.084+0000 7f52137fe700 1 -- 172.21.15.27:0/4178582506 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5214018070 con 0x7f521c060870 2021-11-03T17:11:14.087 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.084+0000 7f5221753700 1 -- 172.21.15.27:0/4178582506 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f521c0f5bc0 con 0x7f521c060870 2021-11-03T17:11:14.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.084+0000 7f5221753700 1 -- 172.21.15.27:0/4178582506 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f521c0f60a0 con 0x7f521c060870 2021-11-03T17:11:14.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.085+0000 7f5221753700 1 -- 172.21.15.27:0/4178582506 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f521c045ad0 con 0x7f521c060870 2021-11-03T17:11:14.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.085+0000 7f52137fe700 1 -- 172.21.15.27:0/4178582506 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f5214024060 con 0x7f521c060870 2021-11-03T17:11:14.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.085+0000 7f52137fe700 1 --2- 172.21.15.27:0/4178582506 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f520805c2a0 0x7f520805e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:14.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.085+0000 7f52137fe700 1 -- 172.21.15.27:0/4178582506 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4519+0+0 (secure 0 0 0) 0x7f52140772c0 con 0x7f521c060870 2021-11-03T17:11:14.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.085+0000 7f521a7fc700 1 --2- 172.21.15.27:0/4178582506 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f520805c2a0 0x7f520805e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:14.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.086+0000 7f521a7fc700 1 --2- 172.21.15.27:0/4178582506 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f520805c2a0 0x7f520805e760 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f521c060640 tx=0x7f520400b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:14.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.089+0000 7f52137fe700 1 -- 172.21.15.27:0/4178582506 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f521404a360 con 0x7f521c060870 2021-11-03T17:11:14.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.248+0000 7f5221753700 1 -- 172.21.15.27:0/4178582506 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f521c027a00 con 0x7f521c060870 2021-11-03T17:11:14.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.248+0000 7f52137fe700 1 -- 172.21.15.27:0/4178582506 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f521404d980 con 0x7f521c060870 2021-11-03T17:11:14.245 INFO:teuthology.orchestra.run.smithi027.stdout:38654705695 2021-11-03T17:11:14.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.250+0000 7f5221753700 1 -- 172.21.15.27:0/4178582506 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f520805c2a0 msgr2=0x7f520805e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:14.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.250+0000 7f5221753700 1 --2- 172.21.15.27:0/4178582506 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f520805c2a0 0x7f520805e760 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f521c060640 tx=0x7f520400b040).stop 2021-11-03T17:11:14.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.250+0000 7f5221753700 1 -- 172.21.15.27:0/4178582506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f521c060870 msgr2=0x7f521c0f6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:14.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.250+0000 7f5221753700 1 --2- 172.21.15.27:0/4178582506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f521c060870 0x7f521c0f6e20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f521400b3b0 tx=0x7f5214004d10).stop 2021-11-03T17:11:14.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.250+0000 7f5221753700 1 -- 172.21.15.27:0/4178582506 shutdown_connections 2021-11-03T17:11:14.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.250+0000 7f5221753700 1 --2- 172.21.15.27:0/4178582506 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f520805c2a0 0x7f520805e760 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:14.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.250+0000 7f5221753700 1 --2- 172.21.15.27:0/4178582506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f521c060870 0x7f521c0f6e20 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:14.248 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.250+0000 7f5221753700 1 --2- 172.21.15.27:0/4178582506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f521c0f9920 0x7f521c0f52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:14.248 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.250+0000 7f5221753700 1 -- 172.21.15.27:0/4178582506 >> 172.21.15.27:0/4178582506 conn(0x7f521c0f0e20 msgr2=0x7f521c0f3210 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:14.248 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.250+0000 7f5221753700 1 -- 172.21.15.27:0/4178582506 shutdown_connections 2021-11-03T17:11:14.248 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:14.250+0000 7f5221753700 1 -- 172.21.15.27:0/4178582506 wait complete. 2021-11-03T17:11:14.816 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705693 got 38654705695 for osd.0 2021-11-03T17:11:14.817 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph osd last-stat-seq osd.1 2021-11-03T17:11:14.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:14 smithi115 conmon[34670]: audit 2021-11-03T17:11:14.250365+0000 mon.smithi027 (mon.0) 449 : audit [DBG] from='client.? 172.21.15.27:0/4178582506' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2021-11-03T17:11:15.096 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:14 smithi027 conmon[32322]: audit 2021-11-03T17:11:14.250365+0000 mon.smithi027 (mon.0) 449 : audit [DBG] from='client.? 172.21.15.27:0/4178582506' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2021-11-03T17:11:15.105 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c/mon.smithi027/config 2021-11-03T17:11:15.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:15 smithi115 conmon[34670]: cluster 2021-11-03T17:11:14.565576+0000 mgr.smithi027.ufhhvb (mgr.14162) 188 : cluster [DBG] pgmap v159: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:16.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:15 smithi027 conmon[32322]: cluster 2021-11-03T17:11:14.565576+0000 2021-11-03T17:11:16.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:15 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14162) 188 : cluster [DBG] pgmap v159: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:16.680 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.683+0000 7f4f2fd48700 1 -- 172.21.15.27:0/646232808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f280f56e0 msgr2=0x7f4f280f5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:16.681 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.683+0000 7f4f2fd48700 1 --2- 172.21.15.27:0/646232808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f280f56e0 0x7f4f280f5b00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f4f200099e0 tx=0x7f4f20004d10).stop 2021-11-03T17:11:16.681 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.683+0000 7f4f2fd48700 1 -- 172.21.15.27:0/646232808 shutdown_connections 2021-11-03T17:11:16.681 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.683+0000 7f4f2fd48700 1 --2- 172.21.15.27:0/646232808 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f280f6980 0x7f4f280f6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:11:16.682 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.683+0000 7f4f2fd48700 1 --2- 172.21.15.27:0/646232808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f280f56e0 0x7f4f280f5b00 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:16.682 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.683+0000 7f4f2fd48700 1 -- 172.21.15.27:0/646232808 >> 172.21.15.27:0/646232808 conn(0x7f4f280f0e20 msgr2=0x7f4f280f3260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:16.682 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.683+0000 7f4f2fd48700 1 -- 172.21.15.27:0/646232808 shutdown_connections 2021-11-03T17:11:16.682 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.683+0000 7f4f2fd48700 1 -- 172.21.15.27:0/646232808 wait complete. 2021-11-03T17:11:16.683 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.684+0000 7f4f2fd48700 1 Processor -- start 2021-11-03T17:11:16.683 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.684+0000 7f4f2fd48700 1 -- start start 2021-11-03T17:11:16.683 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.684+0000 7f4f2fd48700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f280f56e0 0x7f4f28106150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:16.683 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.684+0000 7f4f2fd48700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f280f6980 0x7f4f28106640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:16.684 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.684+0000 7f4f2fd48700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f28106c80 con 0x7f4f280f56e0 2021-11-03T17:11:16.684 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.684+0000 7f4f2fd48700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f28107530 con 0x7f4f280f6980 2021-11-03T17:11:16.684 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.684+0000 7f4f2dae4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f280f56e0 0x7f4f28106150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:16.684 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.684+0000 7f4f2dae4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f280f56e0 0x7f4f28106150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54820/0 (socket says 172.21.15.27:54820) 2021-11-03T17:11:16.684 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.684+0000 7f4f2dae4700 1 -- 172.21.15.27:0/2842869862 learned_addr learned my addr 172.21.15.27:0/2842869862 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:11:16.685 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.684+0000 7f4f2dae4700 1 -- 172.21.15.27:0/2842869862 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f280f6980 msgr2=0x7f4f28106640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:16.685 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.684+0000 7f4f2dae4700 1 --2- 172.21.15.27:0/2842869862 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f280f6980 0x7f4f28106640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:11:16.685 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.684+0000 7f4f2dae4700 1 -- 172.21.15.27:0/2842869862 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f20009640 con 0x7f4f280f56e0 2021-11-03T17:11:16.685 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.684+0000 7f4f2dae4700 1 --2- 172.21.15.27:0/2842869862 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f280f56e0 0x7f4f28106150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f4f2000b530 tx=0x7f4f20004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:16.686 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.684+0000 7f4f267fc700 1 -- 172.21.15.27:0/2842869862 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f2000e030 con 0x7f4f280f56e0 2021-11-03T17:11:16.686 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.685+0000 7f4f267fc700 1 -- 172.21.15.27:0/2842869862 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f4f2001d8b0 con 0x7f4f280f56e0 2021-11-03T17:11:16.686 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.685+0000 7f4f2fd48700 1 -- 172.21.15.27:0/2842869862 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f281077b0 con 0x7f4f280f56e0 2021-11-03T17:11:16.686 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.685+0000 7f4f267fc700 1 -- 172.21.15.27:0/2842869862 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f20018070 con 0x7f4f280f56e0 2021-11-03T17:11:16.686 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.685+0000 7f4f2fd48700 1 -- 172.21.15.27:0/2842869862 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f28107c90 con 0x7f4f280f56e0 2021-11-03T17:11:16.688 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.685+0000 7f4f2fd48700 1 -- 172.21.15.27:0/2842869862 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f28059100 con 0x7f4f280f56e0 2021-11-03T17:11:16.688 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.686+0000 7f4f267fc700 1 -- 172.21.15.27:0/2842869862 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f4f2001da20 con 0x7f4f280f56e0 2021-11-03T17:11:16.688 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.686+0000 7f4f267fc700 1 --2- 172.21.15.27:0/2842869862 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f4f1005c2a0 0x7f4f1005e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:16.688 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.686+0000 7f4f267fc700 1 -- 172.21.15.27:0/2842869862 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4519+0+0 (secure 0 0 0) 0x7f4f200779d0 con 0x7f4f280f56e0 2021-11-03T17:11:16.689 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.686+0000 7f4f2d2e3700 1 --2- 172.21.15.27:0/2842869862 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f4f1005c2a0 0x7f4f1005e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:16.689 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.687+0000 7f4f2d2e3700 1 --2- 172.21.15.27:0/2842869862 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f4f1005c2a0 0x7f4f1005e760 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f4f1c009610 tx=0x7f4f1c006ab0).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:16.689 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.690+0000 7f4f267fc700 1 -- 172.21.15.27:0/2842869862 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f4f2004aa10 con 0x7f4f280f56e0 2021-11-03T17:11:16.847 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.851+0000 7f4f2fd48700 1 -- 172.21.15.27:0/2842869862 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f4f28045ad0 con 0x7f4f280f56e0 2021-11-03T17:11:16.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.852+0000 7f4f267fc700 1 -- 172.21.15.27:0/2842869862 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4f2004e030 con 0x7f4f280f56e0 2021-11-03T17:11:16.848 INFO:teuthology.orchestra.run.smithi027.stdout:60129542171 2021-11-03T17:11:16.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.853+0000 7f4f2fd48700 1 -- 172.21.15.27:0/2842869862 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f4f1005c2a0 msgr2=0x7f4f1005e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:16.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.853+0000 7f4f2fd48700 1 --2- 172.21.15.27:0/2842869862 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f4f1005c2a0 0x7f4f1005e760 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f4f1c009610 tx=0x7f4f1c006ab0).stop 2021-11-03T17:11:16.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.853+0000 7f4f2fd48700 1 -- 172.21.15.27:0/2842869862 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f280f56e0 msgr2=0x7f4f28106150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:16.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.853+0000 7f4f2fd48700 1 --2- 172.21.15.27:0/2842869862 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f280f56e0 0x7f4f28106150 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f4f2000b530 tx=0x7f4f20004d10).stop 2021-11-03T17:11:16.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.853+0000 7f4f2fd48700 1 -- 172.21.15.27:0/2842869862 shutdown_connections 2021-11-03T17:11:16.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.853+0000 7f4f2fd48700 1 --2- 172.21.15.27:0/2842869862 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f4f1005c2a0 0x7f4f1005e760 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:16.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.853+0000 7f4f2fd48700 1 --2- 172.21.15.27:0/2842869862 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f280f56e0 0x7f4f28106150 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:16.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.853+0000 7f4f2fd48700 1 --2- 172.21.15.27:0/2842869862 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f280f6980 0x7f4f28106640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:11:16.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.853+0000 7f4f2fd48700 1 -- 172.21.15.27:0/2842869862 >> 172.21.15.27:0/2842869862 conn(0x7f4f280f0e20 msgr2=0x7f4f280fa3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:16.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.853+0000 7f4f2fd48700 1 -- 172.21.15.27:0/2842869862 shutdown_connections 2021-11-03T17:11:16.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:16.853+0000 7f4f2fd48700 1 -- 172.21.15.27:0/2842869862 wait complete. 2021-11-03T17:11:17.475 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542168 got 60129542171 for osd.1 2021-11-03T17:11:17.476 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph osd last-stat-seq osd.2 2021-11-03T17:11:17.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:17 smithi027 conmon[32322]: cluster 2021-11-03T17:11:16.565961+0000 mgr.smithi027.ufhhvb (mgr.14162) 189 : cluster [DBG] pgmap v160: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:17.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:17 smithi027 conmon[32322]: audit 2021-11-03T17:11:16.853407+0000 mon.smithi027 (mon.0) 450 : audit [DBG] from='client.? 172.21.15.27:0/2842869862' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2021-11-03T17:11:17.770 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c/mon.smithi027/config 2021-11-03T17:11:17.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:17 smithi115 conmon[34670]: cluster 2021-11-03T17:11:16.565961+0000 mgr.smithi027.ufhhvb (mgr.14162) 189 : cluster [DBG] pgmap v160: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:17.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:17 smithi115 conmon[34670]: audit 2021-11-03T17:11:16.853407+0000 mon.smithi027 (mon.0) 450 : audit [DBG] from='client.? 172.21.15.27:0/2842869862' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2021-11-03T17:11:19.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.456+0000 7fd4075a8700 1 -- 172.21.15.27:0/2429442697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4000f56e0 msgr2=0x7fd4000f5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:19.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.456+0000 7fd4075a8700 1 --2- 172.21.15.27:0/2429442697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4000f56e0 0x7fd4000f5b00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7fd3f00099e0 tx=0x7fd3f0004d10).stop 2021-11-03T17:11:19.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.456+0000 7fd4075a8700 1 -- 172.21.15.27:0/2429442697 shutdown_connections 2021-11-03T17:11:19.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.456+0000 7fd4075a8700 1 --2- 172.21.15.27:0/2429442697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd4000f6980 0x7fd4000f6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:19.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.456+0000 7fd4075a8700 1 --2- 172.21.15.27:0/2429442697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4000f56e0 0x7fd4000f5b00 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:19.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.456+0000 7fd4075a8700 1 -- 172.21.15.27:0/2429442697 >> 172.21.15.27:0/2429442697 conn(0x7fd4000f0e20 msgr2=0x7fd4000f3260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:19.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.456+0000 7fd4075a8700 1 -- 172.21.15.27:0/2429442697 shutdown_connections 2021-11-03T17:11:19.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.456+0000 7fd4075a8700 1 -- 172.21.15.27:0/2429442697 wait complete. 2021-11-03T17:11:19.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.457+0000 7fd4075a8700 1 Processor -- start 2021-11-03T17:11:19.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.457+0000 7fd4075a8700 1 -- start start 2021-11-03T17:11:19.457 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.457+0000 7fd4075a8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd4000f56e0 0x7fd400109ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:19.457 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.457+0000 7fd4075a8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4000f6980 0x7fd40010c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:19.457 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.457+0000 7fd4075a8700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd400060920 con 0x7fd4000f6980 2021-11-03T17:11:19.457 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.457+0000 7fd4075a8700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd400060a60 con 0x7fd4000f56e0 2021-11-03T17:11:19.458 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.457+0000 7fd404b43700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4000f6980 0x7fd40010c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:19.458 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.457+0000 7fd404b43700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4000f6980 0x7fd40010c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54830/0 (socket says 172.21.15.27:54830) 2021-11-03T17:11:19.458 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.457+0000 7fd404b43700 1 -- 172.21.15.27:0/520938678 learned_addr learned my addr 172.21.15.27:0/520938678 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:11:19.458 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.457+0000 7fd405344700 1 --2- 172.21.15.27:0/520938678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd4000f56e0 0x7fd400109ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:19.458 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.458+0000 7fd404b43700 1 -- 172.21.15.27:0/520938678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd4000f56e0 msgr2=0x7fd400109ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:19.459 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.458+0000 7fd404b43700 1 --2- 172.21.15.27:0/520938678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd4000f56e0 0x7fd400109ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:19.459 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.458+0000 7fd404b43700 1 -- 172.21.15.27:0/520938678 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f0009640 con 0x7fd4000f6980 2021-11-03T17:11:19.459 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.458+0000 7fd404b43700 1 --2- 172.21.15.27:0/520938678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4000f6980 0x7fd40010c3f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fd3f40026d0 tx=0x7fd3f4009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:19.459 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.458+0000 7fd3fdffb700 1 -- 172.21.15.27:0/520938678 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3f4009320 con 0x7fd4000f6980 2021-11-03T17:11:19.460 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.458+0000 7fd3fdffb700 1 -- 172.21.15.27:0/520938678 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7fd3f400f040 con 0x7fd4000f6980 2021-11-03T17:11:19.460 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.458+0000 7fd4075a8700 1 -- 172.21.15.27:0/520938678 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd40010c9a0 con 0x7fd4000f6980 2021-11-03T17:11:19.460 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.458+0000 7fd3fdffb700 1 -- 172.21.15.27:0/520938678 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3f4014770 con 0x7fd4000f6980 2021-11-03T17:11:19.460 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.458+0000 7fd4075a8700 1 -- 172.21.15.27:0/520938678 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd40010ceb0 con 0x7fd4000f6980 2021-11-03T17:11:19.461 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.459+0000 7fd4075a8700 1 -- 172.21.15.27:0/520938678 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd400059100 con 0x7fd4000f6980 2021-11-03T17:11:19.461 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.459+0000 7fd3fdffb700 1 -- 172.21.15.27:0/520938678 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fd3f4009480 con 0x7fd4000f6980 2021-11-03T17:11:19.462 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.460+0000 7fd3fdffb700 1 --2- 172.21.15.27:0/520938678 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fd3e805c2a0 0x7fd3e805e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:19.462 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.460+0000 7fd3fdffb700 1 -- 172.21.15.27:0/520938678 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4519+0+0 (secure 0 0 0) 0x7fd3f40761d0 con 0x7fd4000f6980 2021-11-03T17:11:19.462 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.460+0000 7fd405344700 1 --2- 172.21.15.27:0/520938678 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fd3e805c2a0 0x7fd3e805e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:19.462 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.460+0000 7fd405344700 1 --2- 172.21.15.27:0/520938678 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fd3e805c2a0 0x7fd3e805e760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fd3f0009f50 tx=0x7fd3f0004d10).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:19.463 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.464+0000 7fd3fdffb700 1 -- 172.21.15.27:0/520938678 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fd3f40491f0 con 0x7fd4000f6980 2021-11-03T17:11:19.621 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.625+0000 7fd4075a8700 1 -- 172.21.15.27:0/520938678 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd400045ad0 con 0x7fd4000f6980 2021-11-03T17:11:19.621 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.625+0000 7fd3fdffb700 1 -- 172.21.15.27:0/520938678 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd3f404c810 con 0x7fd4000f6980 2021-11-03T17:11:19.621 INFO:teuthology.orchestra.run.smithi027.stdout:81604378648 2021-11-03T17:11:19.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.627+0000 7fd4075a8700 1 -- 172.21.15.27:0/520938678 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fd3e805c2a0 msgr2=0x7fd3e805e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:19.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.627+0000 7fd4075a8700 1 --2- 172.21.15.27:0/520938678 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fd3e805c2a0 0x7fd3e805e760 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fd3f0009f50 tx=0x7fd3f0004d10).stop 2021-11-03T17:11:19.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.627+0000 7fd4075a8700 1 -- 172.21.15.27:0/520938678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4000f6980 msgr2=0x7fd40010c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:19.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.627+0000 7fd4075a8700 1 --2- 172.21.15.27:0/520938678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4000f6980 0x7fd40010c3f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fd3f40026d0 tx=0x7fd3f4009de0).stop 2021-11-03T17:11:19.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.627+0000 7fd4075a8700 1 -- 172.21.15.27:0/520938678 shutdown_connections 2021-11-03T17:11:19.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.627+0000 7fd4075a8700 1 --2- 172.21.15.27:0/520938678 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fd3e805c2a0 0x7fd3e805e760 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:19.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.627+0000 7fd4075a8700 1 --2- 172.21.15.27:0/520938678 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd4000f56e0 0x7fd400109ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:19.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.627+0000 7fd4075a8700 1 --2- 172.21.15.27:0/520938678 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4000f6980 0x7fd40010c3f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:19.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.627+0000 7fd4075a8700 1 -- 172.21.15.27:0/520938678 >> 172.21.15.27:0/520938678 conn(0x7fd4000f0e20 msgr2=0x7fd4000fa3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:19.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.627+0000 7fd4075a8700 1 -- 172.21.15.27:0/520938678 shutdown_connections 2021-11-03T17:11:19.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:19.627+0000 7fd4075a8700 1 -- 172.21.15.27:0/520938678 wait complete. 2021-11-03T17:11:19.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:19 smithi115 conmon[34670]: cluster 2021-11-03T17:11:18.566292+0000 mgr.smithi027.ufhhvb (mgr.14162) 190 : cluster [DBG] pgmap v161: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:20.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:19 smithi027 conmon[32322]: cluster 2021-11-03T17:11:18.566292 2021-11-03T17:11:20.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:19 smithi027 conmon[32322]: +0000 mgr.smithi027.ufhhvb (mgr.14162) 190 : cluster [DBG] pgmap v161: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:20.199 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378645 got 81604378648 for osd.2 2021-11-03T17:11:20.200 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph osd last-stat-seq osd.3 2021-11-03T17:11:20.484 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c/mon.smithi027/config 2021-11-03T17:11:20.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:20 smithi115 conmon[34670]: audit 2021-11-03T17:11:19.627009+0000 mon.smithi027 (mon.0) 451 : audit 2021-11-03T17:11:20.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:20 smithi115 conmon[34670]: [DBG] from='client.? 172.21.15.27:0/520938678' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2021-11-03T17:11:21.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:20 smithi027 conmon[32322]: audit 2021-11-03T17:11:19.627009+0000 2021-11-03T17:11:21.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:20 smithi027 conmon[32322]: mon.smithi027 (mon.0) 451 : audit [DBG] from='client.? 172.21.15.27:0/520938678' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2021-11-03T17:11:21.844 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:21 smithi027 conmon[32322]: cluster 2021-11-03T17:11:20.566763+0000 mgr.smithi027.ufhhvb (mgr.14162) 191 : cluster [DBG] pgmap v162: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:21.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:21 smithi115 conmon[34670]: cluster 2021-11-03T17:11:20.566763+0000 mgr.smithi027.ufhhvb (mgr.14162) 191 2021-11-03T17:11:21.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:21 smithi115 conmon[34670]: : cluster [DBG] pgmap v162: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:22.020 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.023+0000 7f181b9e1700 1 -- 172.21.15.27:0/2395629306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18140f67e0 msgr2=0x7f18140f6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:22.020 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.023+0000 7f181b9e1700 1 --2- 172.21.15.27:0/2395629306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18140f67e0 0x7f18140f6c00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f180c0099e0 tx=0x7f180c004d10).stop 2021-11-03T17:11:22.021 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.023+0000 7f181b9e1700 1 -- 172.21.15.27:0/2395629306 shutdown_connections 2021-11-03T17:11:22.021 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.023+0000 7f181b9e1700 1 --2- 172.21.15.27:0/2395629306 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18140f7a80 0x7f18140f7ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:22.021 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.023+0000 7f181b9e1700 1 --2- 172.21.15.27:0/2395629306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18140f67e0 0x7f18140f6c00 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:22.021 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.023+0000 7f181b9e1700 1 -- 172.21.15.27:0/2395629306 >> 172.21.15.27:0/2395629306 conn(0x7f18140f1f20 msgr2=0x7f18140f4360 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:22.022 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.023+0000 7f181b9e1700 1 -- 172.21.15.27:0/2395629306 shutdown_connections 2021-11-03T17:11:22.022 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.023+0000 7f181b9e1700 1 -- 172.21.15.27:0/2395629306 wait complete. 2021-11-03T17:11:22.022 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.023+0000 7f181b9e1700 1 Processor -- start 2021-11-03T17:11:22.022 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.023+0000 7f181b9e1700 1 -- start start 2021-11-03T17:11:22.023 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.024+0000 7f181b9e1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18140f67e0 0x7f18141071d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:22.023 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.024+0000 7f181b9e1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18140f7a80 0x7f18141076c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:22.023 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.024+0000 7f181b9e1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1814107c70 con 0x7f18140f67e0 2021-11-03T17:11:22.023 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.024+0000 7f181b9e1700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1814107db0 con 0x7f18140f7a80 2021-11-03T17:11:22.024 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.024+0000 7f181977d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18140f67e0 0x7f18141071d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:22.024 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.024+0000 7f181977d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18140f67e0 0x7f18141071d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54838/0 (socket says 172.21.15.27:54838) 2021-11-03T17:11:22.024 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.024+0000 7f181977d700 1 -- 172.21.15.27:0/758443727 learned_addr learned my addr 172.21.15.27:0/758443727 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:11:22.024 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.024+0000 7f1818f7c700 1 --2- 172.21.15.27:0/758443727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18140f7a80 0x7f18141076c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:22.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.024+0000 7f181977d700 1 -- 172.21.15.27:0/758443727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18140f7a80 msgr2=0x7f18141076c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:22.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.024+0000 7f181977d700 1 --2- 172.21.15.27:0/758443727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18140f7a80 0x7f18141076c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:22.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.024+0000 7f181977d700 1 -- 172.21.15.27:0/758443727 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f180c009640 con 0x7f18140f67e0 2021-11-03T17:11:22.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.024+0000 7f181977d700 1 --2- 172.21.15.27:0/758443727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18140f67e0 0x7f18141071d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f180c00b3b0 tx=0x7f180c004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:22.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.024+0000 7f1811ffb700 1 -- 172.21.15.27:0/758443727 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f180c00e070 con 0x7f18140f67e0 2021-11-03T17:11:22.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.024+0000 7f1811ffb700 1 -- 172.21.15.27:0/758443727 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f180c0049b0 con 0x7f18140f67e0 2021-11-03T17:11:22.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.024+0000 7f181b9e1700 1 -- 172.21.15.27:0/758443727 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18141086f0 con 0x7f18140f67e0 2021-11-03T17:11:22.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.024+0000 7f1811ffb700 1 -- 172.21.15.27:0/758443727 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f180c018070 con 0x7f18140f67e0 2021-11-03T17:11:22.027 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.025+0000 7f181b9e1700 1 -- 172.21.15.27:0/758443727 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1814108c50 con 0x7f18140f67e0 2021-11-03T17:11:22.028 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.025+0000 7f181b9e1700 1 -- 172.21.15.27:0/758443727 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1814045ad0 con 0x7f18140f67e0 2021-11-03T17:11:22.028 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.026+0000 7f1811ffb700 1 -- 172.21.15.27:0/758443727 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f180c024060 con 0x7f18140f67e0 2021-11-03T17:11:22.028 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.026+0000 7f1811ffb700 1 --2- 172.21.15.27:0/758443727 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f17fc05c2a0 0x7f17fc05e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:22.028 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.026+0000 7f1811ffb700 1 -- 172.21.15.27:0/758443727 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4519+0+0 (secure 0 0 0) 0x7f180c0773b0 con 0x7f18140f67e0 2021-11-03T17:11:22.029 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.026+0000 7f1818f7c700 1 --2- 172.21.15.27:0/758443727 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f17fc05c2a0 0x7f17fc05e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:22.029 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.027+0000 7f1818f7c700 1 --2- 172.21.15.27:0/758443727 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f17fc05c2a0 0x7f17fc05e760 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f1808007f80 tx=0x7f180800b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:22.029 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.031+0000 7f1811ffb700 1 -- 172.21.15.27:0/758443727 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f180c04a390 con 0x7f18140f67e0 2021-11-03T17:11:22.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.189+0000 7f181b9e1700 1 -- 172.21.15.27:0/758443727 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f1814027a00 con 0x7f18140f67e0 2021-11-03T17:11:22.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.189+0000 7f1811ffb700 1 -- 172.21.15.27:0/758443727 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f180c04d9b0 con 0x7f18140f67e0 2021-11-03T17:11:22.186 INFO:teuthology.orchestra.run.smithi027.stdout:115964117010 2021-11-03T17:11:22.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.191+0000 7f181b9e1700 1 -- 172.21.15.27:0/758443727 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f17fc05c2a0 msgr2=0x7f17fc05e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:22.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.191+0000 7f181b9e1700 1 --2- 172.21.15.27:0/758443727 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f17fc05c2a0 0x7f17fc05e760 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f1808007f80 tx=0x7f180800b040).stop 2021-11-03T17:11:22.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.191+0000 7f181b9e1700 1 -- 172.21.15.27:0/758443727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18140f67e0 msgr2=0x7f18141071d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:22.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.191+0000 7f181b9e1700 1 --2- 172.21.15.27:0/758443727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18140f67e0 0x7f18141071d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f180c00b3b0 tx=0x7f180c004d10).stop 2021-11-03T17:11:22.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.191+0000 7f181b9e1700 1 -- 172.21.15.27:0/758443727 shutdown_connections 2021-11-03T17:11:22.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.191+0000 7f181b9e1700 1 --2- 172.21.15.27:0/758443727 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f17fc05c2a0 0x7f17fc05e760 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:22.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.191+0000 7f181b9e1700 1 --2- 172.21.15.27:0/758443727 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18140f67e0 0x7f18141071d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:22.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.191+0000 7f181b9e1700 1 --2- 172.21.15.27:0/758443727 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18140f7a80 0x7f18141076c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:22.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.191+0000 7f181b9e1700 1 -- 172.21.15.27:0/758443727 >> 172.21.15.27:0/758443727 conn(0x7f18140f1f20 msgr2=0x7f18140fb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:22.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.191+0000 7f181b9e1700 1 -- 172.21.15.27:0/758443727 shutdown_connections 2021-11-03T17:11:22.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:22.191+0000 7f181b9e1700 1 -- 172.21.15.27:0/758443727 wait complete. 2021-11-03T17:11:22.762 INFO:tasks.cephadm.ceph_manager.ceph:need seq 115964117008 got 115964117010 for osd.3 2021-11-03T17:11:22.763 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph osd last-stat-seq osd.4 2021-11-03T17:11:22.937 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:22 smithi027 conmon[32322]: audit 2021-11-03T17:11:22.190820+0000 mon.smithi027 (mon. 2021-11-03T17:11:22.938 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:22 smithi027 conmon[32322]: 0) 452 : audit [DBG] from='client.? 172.21.15.27:0/758443727' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2021-11-03T17:11:22.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:22 smithi115 conmon[34670]: audit 2021-11-03T17:11:22.190820+0000 mon.smithi027 (mon.0) 452 : 2021-11-03T17:11:22.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:22 smithi115 conmon[34670]: audit [DBG] from='client.? 172.21.15.27:0/758443727' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2021-11-03T17:11:23.052 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c/mon.smithi027/config 2021-11-03T17:11:23.955 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:23 smithi115 conmon[34670]: cluster 2021-11-03T17:11:22.567178+0000 mgr.smithi027.ufhhvb 2021-11-03T17:11:23.956 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:23 smithi115 conmon[34670]: (mgr.14162) 192 : cluster [DBG] pgmap v163: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:24.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:23 smithi027 conmon[32322]: cluster 2021-11-03T17:11:22.567178 2021-11-03T17:11:24.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:23 smithi027 conmon[32322]: +0000 mgr.smithi027.ufhhvb (mgr.14162) 192 : cluster [DBG] pgmap v163: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:24.655 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.657+0000 7fcbc8b31700 1 -- 172.21.15.27:0/245261804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbc40f56e0 msgr2=0x7fcbc40f5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:24.655 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.657+0000 7fcbc8b31700 1 --2- 172.21.15.27:0/245261804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbc40f56e0 0x7fcbc40f5b00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fcbbc004660 tx=0x7fcbbc009b30).stop 2021-11-03T17:11:24.655 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.657+0000 7fcbc8b31700 1 -- 172.21.15.27:0/245261804 shutdown_connections 2021-11-03T17:11:24.656 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.657+0000 7fcbc8b31700 1 --2- 172.21.15.27:0/245261804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbc40f6980 0x7fcbc40f6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:24.656 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.657+0000 7fcbc8b31700 1 --2- 172.21.15.27:0/245261804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbc40f56e0 0x7fcbc40f5b00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:24.656 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.657+0000 7fcbc8b31700 1 -- 172.21.15.27:0/245261804 >> 172.21.15.27:0/245261804 conn(0x7fcbc40f0e20 msgr2=0x7fcbc40f3260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:24.656 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.657+0000 7fcbc8b31700 1 -- 172.21.15.27:0/245261804 shutdown_connections 2021-11-03T17:11:24.657 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.657+0000 7fcbc8b31700 1 -- 172.21.15.27:0/245261804 wait complete. 2021-11-03T17:11:24.657 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.658+0000 7fcbc8b31700 1 Processor -- start 2021-11-03T17:11:24.657 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.658+0000 7fcbc8b31700 1 -- start start 2021-11-03T17:11:24.657 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.658+0000 7fcbc8b31700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbc40f56e0 0x7fcbc41070d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:24.658 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.658+0000 7fcbc8b31700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbc40f6980 0x7fcbc41075c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:24.658 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.658+0000 7fcbc8b31700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc4107b70 con 0x7fcbc40f56e0 2021-11-03T17:11:24.658 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.658+0000 7fcbc8b31700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbc4107cb0 con 0x7fcbc40f6980 2021-11-03T17:11:24.658 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.658+0000 7fcbc259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbc40f56e0 0x7fcbc41070d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:24.658 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.658+0000 7fcbc259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbc40f56e0 0x7fcbc41070d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54848/0 (socket says 172.21.15.27:54848) 2021-11-03T17:11:24.659 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.658+0000 7fcbc259c700 1 -- 172.21.15.27:0/3983755222 learned_addr learned my addr 172.21.15.27:0/3983755222 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:11:24.659 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.658+0000 7fcbc1d9b700 1 --2- 172.21.15.27:0/3983755222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbc40f6980 0x7fcbc41075c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:24.659 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.658+0000 7fcbc259c700 1 -- 172.21.15.27:0/3983755222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbc40f6980 msgr2=0x7fcbc41075c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:24.659 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.658+0000 7fcbc259c700 1 --2- 172.21.15.27:0/3983755222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbc40f6980 0x7fcbc41075c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:24.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.658+0000 7fcbc259c700 1 -- 172.21.15.27:0/3983755222 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbbc005040 con 0x7fcbc40f56e0 2021-11-03T17:11:24.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.658+0000 7fcbc259c700 1 --2- 172.21.15.27:0/3983755222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbc40f56e0 0x7fcbc41070d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7fcbbc004e30 tx=0x7fcbbc009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:24.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.659+0000 7fcbbaffd700 1 -- 172.21.15.27:0/3983755222 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbbc018070 con 0x7fcbc40f56e0 2021-11-03T17:11:24.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.659+0000 7fcbbaffd700 1 -- 172.21.15.27:0/3983755222 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7fcbbc013410 con 0x7fcbc40f56e0 2021-11-03T17:11:24.661 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.659+0000 7fcbc8b31700 1 -- 172.21.15.27:0/3983755222 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbc41085f0 con 0x7fcbc40f56e0 2021-11-03T17:11:24.661 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.659+0000 7fcbbaffd700 1 -- 172.21.15.27:0/3983755222 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbbc01c3c0 con 0x7fcbc40f56e0 2021-11-03T17:11:24.661 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.659+0000 7fcbc8b31700 1 -- 172.21.15.27:0/3983755222 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbc4108b50 con 0x7fcbc40f56e0 2021-11-03T17:11:24.662 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.660+0000 7fcbc8b31700 1 -- 172.21.15.27:0/3983755222 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbc4045ad0 con 0x7fcbc40f56e0 2021-11-03T17:11:24.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.660+0000 7fcbbaffd700 1 -- 172.21.15.27:0/3983755222 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fcbbc01d420 con 0x7fcbc40f56e0 2021-11-03T17:11:24.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.660+0000 7fcbbaffd700 1 --2- 172.21.15.27:0/3983755222 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fcbb005c2a0 0x7fcbb005e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:24.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.660+0000 7fcbbaffd700 1 -- 172.21.15.27:0/3983755222 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4519+0+0 (secure 0 0 0) 0x7fcbbc077850 con 0x7fcbc40f56e0 2021-11-03T17:11:24.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.660+0000 7fcbc1d9b700 1 --2- 172.21.15.27:0/3983755222 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fcbb005c2a0 0x7fcbb005e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:24.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.661+0000 7fcbc1d9b700 1 --2- 172.21.15.27:0/3983755222 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fcbb005c2a0 0x7fcbb005e760 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fcbac004a70 tx=0x7fcbac00b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:24.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.665+0000 7fcbbaffd700 1 -- 172.21.15.27:0/3983755222 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fcbbc04a8f0 con 0x7fcbc40f56e0 2021-11-03T17:11:24.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.820+0000 7fcbc8b31700 1 -- 172.21.15.27:0/3983755222 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fcbc41092a0 con 0x7fcbc40f56e0 2021-11-03T17:11:24.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.821+0000 7fcbbaffd700 1 -- 172.21.15.27:0/3983755222 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fcbbc01d6b0 con 0x7fcbc40f56e0 2021-11-03T17:11:24.817 INFO:teuthology.orchestra.run.smithi027.stdout:141733920784 2021-11-03T17:11:24.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.822+0000 7fcbc8b31700 1 -- 172.21.15.27:0/3983755222 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fcbb005c2a0 msgr2=0x7fcbb005e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:24.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.822+0000 7fcbc8b31700 1 --2- 172.21.15.27:0/3983755222 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fcbb005c2a0 0x7fcbb005e760 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fcbac004a70 tx=0x7fcbac00b040).stop 2021-11-03T17:11:24.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.822+0000 7fcbc8b31700 1 -- 172.21.15.27:0/3983755222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbc40f56e0 msgr2=0x7fcbc41070d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:24.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.822+0000 7fcbc8b31700 1 --2- 172.21.15.27:0/3983755222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbc40f56e0 0x7fcbc41070d0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7fcbbc004e30 tx=0x7fcbbc009b30).stop 2021-11-03T17:11:24.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.822+0000 7fcbc8b31700 1 -- 172.21.15.27:0/3983755222 shutdown_connections 2021-11-03T17:11:24.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.822+0000 7fcbc8b31700 1 --2- 172.21.15.27:0/3983755222 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fcbb005c2a0 0x7fcbb005e760 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:24.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.822+0000 7fcbc8b31700 1 --2- 172.21.15.27:0/3983755222 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbc40f56e0 0x7fcbc41070d0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:24.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.822+0000 7fcbc8b31700 1 --2- 172.21.15.27:0/3983755222 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbc40f6980 0x7fcbc41075c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:24.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.822+0000 7fcbc8b31700 1 -- 172.21.15.27:0/3983755222 >> 172.21.15.27:0/3983755222 conn(0x7fcbc40f0e20 msgr2=0x7fcbc405fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:24.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.822+0000 7fcbc8b31700 1 -- 172.21.15.27:0/3983755222 shutdown_connections 2021-11-03T17:11:24.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:24.822+0000 7fcbc8b31700 1 -- 172.21.15.27:0/3983755222 wait complete. 2021-11-03T17:11:25.432 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920781 got 141733920784 for osd.4 2021-11-03T17:11:25.432 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph osd last-stat-seq osd.5 2021-11-03T17:11:25.722 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c/mon.smithi027/config 2021-11-03T17:11:25.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:25 smithi027 conmon[32322]: cluster 2021-11-03T17:11:24.567461 2021-11-03T17:11:25.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:25 smithi027 conmon[32322]: +0000 mgr.smithi027.ufhhvb (mgr.14162) 193 : cluster [DBG] pgmap v164: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:25.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:25 smithi027 conmon[32322]: audit 2021-11-03T17:11:24.822465+0000 mon.smithi027 (mon.0) 453 : audit [DBG] from='client.? 172.21.15.27:0/3983755222' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2021-11-03T17:11:25.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:25 smithi115 conmon[34670]: cluster 2021-11-03T17:11:24.567461+0000 mgr.smithi027.ufhhvb (mgr.14162) 193 : cluster [DBG] pgmap v164: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:25.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:25 smithi115 conmon[34670]: 2021-11-03T17:11:25.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:25 smithi115 conmon[34670]: audit 2021-11-03T17:11:24.822465+0000 mon.smithi027 (mon.0) 453 : audit [DBG] from='client.? 172.21.15.27:0/3983755222' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2021-11-03T17:11:27.354 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.357+0000 7f9c8646a700 1 -- 172.21.15.27:0/580919012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c800f7a80 msgr2=0x7f9c800f7ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:27.354 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.357+0000 7f9c8646a700 1 --2- 172.21.15.27:0/580919012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c800f7a80 0x7f9c800f7ee0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f9c70004660 tx=0x7f9c70009b30).stop 2021-11-03T17:11:27.354 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.357+0000 7f9c8646a700 1 -- 172.21.15.27:0/580919012 shutdown_connections 2021-11-03T17:11:27.355 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.357+0000 7f9c8646a700 1 --2- 172.21.15.27:0/580919012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c800f7a80 0x7f9c800f7ee0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:27.355 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.357+0000 7f9c8646a700 1 --2- 172.21.15.27:0/580919012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c800f67e0 0x7f9c800f6c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:27.355 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.357+0000 7f9c8646a700 1 -- 172.21.15.27:0/580919012 >> 172.21.15.27:0/580919012 conn(0x7f9c800f1f20 msgr2=0x7f9c800f4360 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:27.355 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.357+0000 7f9c8646a700 1 -- 172.21.15.27:0/580919012 shutdown_connections 2021-11-03T17:11:27.356 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.357+0000 7f9c8646a700 1 -- 172.21.15.27:0/580919012 wait complete. 2021-11-03T17:11:27.356 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.358+0000 7f9c8646a700 1 Processor -- start 2021-11-03T17:11:27.356 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.358+0000 7f9c8646a700 1 -- start start 2021-11-03T17:11:27.356 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.358+0000 7f9c8646a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c800f67e0 0x7f9c80106fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:27.357 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.358+0000 7f9c8646a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c800f7a80 0x7f9c80107490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:27.357 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.358+0000 7f9c8646a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c80107a40 con 0x7f9c800f7a80 2021-11-03T17:11:27.357 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.358+0000 7f9c8646a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c80107b80 con 0x7f9c800f67e0 2021-11-03T17:11:27.357 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.359+0000 7f9c7f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c800f7a80 0x7f9c80107490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:27.357 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.359+0000 7f9c7f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c800f7a80 0x7f9c80107490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54858/0 (socket says 172.21.15.27:54858) 2021-11-03T17:11:27.358 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.359+0000 7f9c7f7fe700 1 -- 172.21.15.27:0/1546767471 learned_addr learned my addr 172.21.15.27:0/1546767471 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:11:27.358 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.359+0000 7f9c7ffff700 1 --2- 172.21.15.27:0/1546767471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c800f67e0 0x7f9c80106fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:27.358 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.359+0000 7f9c7f7fe700 1 -- 172.21.15.27:0/1546767471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c800f67e0 msgr2=0x7f9c80106fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:27.358 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.359+0000 7f9c7f7fe700 1 --2- 172.21.15.27:0/1546767471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c800f67e0 0x7f9c80106fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:27.359 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.359+0000 7f9c7f7fe700 1 -- 172.21.15.27:0/1546767471 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c70005040 con 0x7f9c800f7a80 2021-11-03T17:11:27.359 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.359+0000 7f9c7f7fe700 1 --2- 172.21.15.27:0/1546767471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c800f7a80 0x7f9c80107490 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f9c70000c00 tx=0x7f9c7000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:27.359 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.359+0000 7f9c7cff9700 1 -- 172.21.15.27:0/1546767471 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c70019070 con 0x7f9c800f7a80 2021-11-03T17:11:27.359 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.359+0000 7f9c7cff9700 1 -- 172.21.15.27:0/1546767471 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f9c70007e50 con 0x7f9c800f7a80 2021-11-03T17:11:27.359 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.359+0000 7f9c8646a700 1 -- 172.21.15.27:0/1546767471 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c801084c0 con 0x7f9c800f7a80 2021-11-03T17:11:27.360 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.359+0000 7f9c7cff9700 1 -- 172.21.15.27:0/1546767471 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c70014d00 con 0x7f9c800f7a80 2021-11-03T17:11:27.360 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.359+0000 7f9c8646a700 1 -- 172.21.15.27:0/1546767471 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c80108a20 con 0x7f9c800f7a80 2021-11-03T17:11:27.361 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.360+0000 7f9c8646a700 1 -- 172.21.15.27:0/1546767471 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c80045ad0 con 0x7f9c800f7a80 2021-11-03T17:11:27.361 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.360+0000 7f9c7cff9700 1 -- 172.21.15.27:0/1546767471 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f9c70025070 con 0x7f9c800f7a80 2021-11-03T17:11:27.361 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.361+0000 7f9c7cff9700 1 --2- 172.21.15.27:0/1546767471 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f9c6805c2a0 0x7f9c6805e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:27.362 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.361+0000 7f9c7cff9700 1 -- 172.21.15.27:0/1546767471 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4519+0+0 (secure 0 0 0) 0x7f9c70076c90 con 0x7f9c800f7a80 2021-11-03T17:11:27.362 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.361+0000 7f9c7ffff700 1 --2- 172.21.15.27:0/1546767471 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f9c6805c2a0 0x7f9c6805e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:27.362 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.361+0000 7f9c7ffff700 1 --2- 172.21.15.27:0/1546767471 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f9c6805c2a0 0x7f9c6805e760 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f9c800f78e0 tx=0x7f9c7400b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:27.362 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.365+0000 7f9c7cff9700 1 -- 172.21.15.27:0/1546767471 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f9c70049cb0 con 0x7f9c800f7a80 2021-11-03T17:11:27.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.523+0000 7f9c8646a700 1 -- 172.21.15.27:0/1546767471 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f9c80027a00 con 0x7f9c800f7a80 2021-11-03T17:11:27.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.523+0000 7f9c7cff9700 1 -- 172.21.15.27:0/1546767471 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9c7004d2d0 con 0x7f9c800f7a80 2021-11-03T17:11:27.519 INFO:teuthology.orchestra.run.smithi027.stdout:163208757262 2021-11-03T17:11:27.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.525+0000 7f9c8646a700 1 -- 172.21.15.27:0/1546767471 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f9c6805c2a0 msgr2=0x7f9c6805e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:27.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.525+0000 7f9c8646a700 1 --2- 172.21.15.27:0/1546767471 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f9c6805c2a0 0x7f9c6805e760 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f9c800f78e0 tx=0x7f9c7400b040).stop 2021-11-03T17:11:27.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.525+0000 7f9c8646a700 1 -- 172.21.15.27:0/1546767471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c800f7a80 msgr2=0x7f9c80107490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:27.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.525+0000 7f9c8646a700 1 --2- 172.21.15.27:0/1546767471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c800f7a80 0x7f9c80107490 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f9c70000c00 tx=0x7f9c7000b040).stop 2021-11-03T17:11:27.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.525+0000 7f9c8646a700 1 -- 172.21.15.27:0/1546767471 shutdown_connections 2021-11-03T17:11:27.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.525+0000 7f9c8646a700 1 --2- 172.21.15.27:0/1546767471 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f9c6805c2a0 0x7f9c6805e760 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:27.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.525+0000 7f9c8646a700 1 --2- 172.21.15.27:0/1546767471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c800f67e0 0x7f9c80106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:27.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.525+0000 7f9c8646a700 1 --2- 172.21.15.27:0/1546767471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c800f7a80 0x7f9c80107490 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:27.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.525+0000 7f9c8646a700 1 -- 172.21.15.27:0/1546767471 >> 172.21.15.27:0/1546767471 conn(0x7f9c800f1f20 msgr2=0x7f9c800fb4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:27.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.525+0000 7f9c8646a700 1 -- 172.21.15.27:0/1546767471 shutdown_connections 2021-11-03T17:11:27.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:27.525+0000 7f9c8646a700 1 -- 172.21.15.27:0/1546767471 wait complete. 2021-11-03T17:11:27.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:27 smithi115 conmon[34670]: cluster 2021-11-03T17:11:26.567783+0000 mgr.smithi027.ufhhvb (mgr.14162) 194 : cluster 2021-11-03T17:11:27.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:27 smithi115 conmon[34670]: [DBG] pgmap v165: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:27.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:27 smithi115 conmon[34670]: audit 2021-11-03T17:11:27.524112+0000 mon.smithi027 (mon.0) 454 : audit [DBG] from='client.? 172.21.15.27:0/1546767471' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2021-11-03T17:11:28.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:27 smithi027 conmon[32322]: cluster 2021-11-03T17:11: 2021-11-03T17:11:28.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:27 smithi027 conmon[32322]: 26.567783+0000 mgr.smithi027.ufhhvb (mgr.14162) 194 : cluster [DBG] pgmap v165: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:28.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:27 smithi027 conmon[32322]: audit 2021-11-03T17:11:27.524112+0000 mon.smithi027 (mon.0) 454 : audit [DBG] from='client.? 172.21.15.27:0/1546767471' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2021-11-03T17:11:28.090 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757259 got 163208757262 for osd.5 2021-11-03T17:11:28.090 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2021-11-03T17:11:28.091 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph pg dump --format=json 2021-11-03T17:11:28.381 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c/mon.smithi027/config 2021-11-03T17:11:29.848 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:29 smithi027 conmon[32322]: cluster 2021-11-03T17:11:28.568255+0000 mgr.smithi027.ufhhvb ( 2021-11-03T17:11:29.849 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:29 smithi027 conmon[32322]: mgr.14162) 195 : cluster [DBG] pgmap v166: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:29.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:29 smithi115 conmon[34670]: cluster 2021-11-03T17:11:28.568255+0000 mgr.smithi027.ufhhvb (mgr.14162) 195 : cluster [DBG] pgmap v166: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:30.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.040+0000 7f9297b15700 1 -- 172.21.15.27:0/3963178870 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92900f56e0 msgr2=0x7f92900f5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:30.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.040+0000 7f9297b15700 1 --2- 172.21.15.27:0/3963178870 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92900f56e0 0x7f92900f5b00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f928c004660 tx=0x7f928c009b30).stop 2021-11-03T17:11:30.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.040+0000 7f9297b15700 1 -- 172.21.15.27:0/3963178870 shutdown_connections 2021-11-03T17:11:30.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.040+0000 7f9297b15700 1 --2- 172.21.15.27:0/3963178870 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92900f6980 0x7f92900f6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:30.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.040+0000 7f9297b15700 1 --2- 172.21.15.27:0/3963178870 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92900f56e0 0x7f92900f5b00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:30.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.040+0000 7f9297b15700 1 -- 172.21.15.27:0/3963178870 >> 172.21.15.27:0/3963178870 conn(0x7f92900f0e20 msgr2=0x7f92900f3260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:30.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.040+0000 7f9297b15700 1 -- 172.21.15.27:0/3963178870 shutdown_connections 2021-11-03T17:11:30.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.040+0000 7f9297b15700 1 -- 172.21.15.27:0/3963178870 wait complete. 2021-11-03T17:11:30.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.040+0000 7f9297b15700 1 Processor -- start 2021-11-03T17:11:30.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.040+0000 7f9297b15700 1 -- start start 2021-11-03T17:11:30.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.041+0000 7f9297b15700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92900f56e0 0x7f9290109d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:30.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.041+0000 7f9297b15700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92900f6980 0x7f929010c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:30.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.041+0000 7f9297b15700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9290060940 con 0x7f92900f6980 2021-11-03T17:11:30.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.041+0000 7f9297b15700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9290060a80 con 0x7f92900f56e0 2021-11-03T17:11:30.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.041+0000 7f92950b0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92900f6980 0x7f929010c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:30.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.041+0000 7f92950b0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92900f6980 0x7f929010c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54868/0 (socket says 172.21.15.27:54868) 2021-11-03T17:11:30.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.041+0000 7f92950b0700 1 -- 172.21.15.27:0/2881667140 learned_addr learned my addr 172.21.15.27:0/2881667140 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:11:30.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.041+0000 7f92958b1700 1 --2- 172.21.15.27:0/2881667140 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92900f56e0 0x7f9290109d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:30.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.041+0000 7f92950b0700 1 -- 172.21.15.27:0/2881667140 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92900f56e0 msgr2=0x7f9290109d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:30.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.041+0000 7f92950b0700 1 --2- 172.21.15.27:0/2881667140 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92900f56e0 0x7f9290109d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:30.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.041+0000 7f92950b0700 1 -- 172.21.15.27:0/2881667140 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f928c005040 con 0x7f92900f6980 2021-11-03T17:11:30.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.041+0000 7f92950b0700 1 --2- 172.21.15.27:0/2881667140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92900f6980 0x7f929010c260 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f928000e7f0 tx=0x7f928000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:30.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.041+0000 7f92867fc700 1 -- 172.21.15.27:0/2881667140 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92800092e0 con 0x7f92900f6980 2021-11-03T17:11:30.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.041+0000 7f92867fc700 1 -- 172.21.15.27:0/2881667140 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f928000f040 con 0x7f92900f6980 2021-11-03T17:11:30.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.041+0000 7f9297b15700 1 -- 172.21.15.27:0/2881667140 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f929010c810 con 0x7f92900f6980 2021-11-03T17:11:30.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.041+0000 7f92867fc700 1 -- 172.21.15.27:0/2881667140 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92800146c0 con 0x7f92900f6980 2021-11-03T17:11:30.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.041+0000 7f9297b15700 1 -- 172.21.15.27:0/2881667140 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f929010cdd0 con 0x7f92900f6980 2021-11-03T17:11:30.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.042+0000 7f9297b15700 1 -- 172.21.15.27:0/2881667140 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9290045ad0 con 0x7f92900f6980 2021-11-03T17:11:30.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.043+0000 7f92867fc700 1 -- 172.21.15.27:0/2881667140 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f928001b030 con 0x7f92900f6980 2021-11-03T17:11:30.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.043+0000 7f92867fc700 1 --2- 172.21.15.27:0/2881667140 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f927c05c2a0 0x7f927c05e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:30.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.043+0000 7f92867fc700 1 -- 172.21.15.27:0/2881667140 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4519+0+0 (secure 0 0 0) 0x7f9280075910 con 0x7f92900f6980 2021-11-03T17:11:30.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.043+0000 7f92958b1700 1 --2- 172.21.15.27:0/2881667140 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f927c05c2a0 0x7f927c05e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:30.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.044+0000 7f92958b1700 1 --2- 172.21.15.27:0/2881667140 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f927c05c2a0 0x7f927c05e760 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f928c000c00 tx=0x7f928c00b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:30.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.047+0000 7f92867fc700 1 -- 172.21.15.27:0/2881667140 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f9280048980 con 0x7f92900f6980 2021-11-03T17:11:30.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.200+0000 7f9297b15700 1 -- 172.21.15.27:0/2881667140 --> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9290057f10 con 0x7f927c05c2a0 2021-11-03T17:11:30.196 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.202+0000 7f92867fc700 1 -- 172.21.15.27:0/2881667140 <== mgr.14162 v2:172.21.15.27:6800/3802716887 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27776 (secure 0 0 0) 0x7f9290057f10 con 0x7f927c05c2a0 2021-11-03T17:11:30.197 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:11:30.199 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.203+0000 7f9297b15700 1 -- 172.21.15.27:0/2881667140 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f927c05c2a0 msgr2=0x7f927c05e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:30.199 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.203+0000 7f9297b15700 1 --2- 172.21.15.27:0/2881667140 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f927c05c2a0 0x7f927c05e760 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f928c000c00 tx=0x7f928c00b040).stop 2021-11-03T17:11:30.199 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.203+0000 7f9297b15700 1 -- 172.21.15.27:0/2881667140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92900f6980 msgr2=0x7f929010c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:30.199 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.203+0000 7f9297b15700 1 --2- 172.21.15.27:0/2881667140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92900f6980 0x7f929010c260 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f928000e7f0 tx=0x7f928000b040).stop 2021-11-03T17:11:30.200 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.204+0000 7f9297b15700 1 -- 172.21.15.27:0/2881667140 shutdown_connections 2021-11-03T17:11:30.200 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.204+0000 7f9297b15700 1 --2- 172.21.15.27:0/2881667140 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f927c05c2a0 0x7f927c05e760 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:30.200 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.204+0000 7f9297b15700 1 --2- 172.21.15.27:0/2881667140 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92900f56e0 0x7f9290109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:30.200 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.204+0000 7f9297b15700 1 --2- 172.21.15.27:0/2881667140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92900f6980 0x7f929010c260 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:30.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.204+0000 7f9297b15700 1 -- 172.21.15.27:0/2881667140 >> 172.21.15.27:0/2881667140 conn(0x7f92900f0e20 msgr2=0x7f92900fa3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:30.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.204+0000 7f9297b15700 1 -- 172.21.15.27:0/2881667140 shutdown_connections 2021-11-03T17:11:30.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:30.204+0000 7f9297b15700 1 -- 172.21.15.27:0/2881667140 wait complete. 2021-11-03T17:11:30.202 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2021-11-03T17:11:30.807 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":166,"stamp":"2021-11-03T17:11:28.568015+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":31272,"kb_used_data":2040,"kb_used_omap":0,"kb_used_meta":29184,"kb_avail":562439640,"statfs":{"total":575970213888,"available":575938191360,"internally_reserved":0,"allocated":2088960,"data_stored":778212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29884416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":"34","reported_epoch":"38","state":"active+clean","last_fresh":"2021-11-03T17:10:34.822363+0000","last_change":"2021-11-03T17:10:14.821826+0000","last_active":"2021-11-03T17:10:34.822363+0000","last_peered":"2021-11-03T17:10:34.822363+0000","last_clean":"2021-11-03T17:10:34.822363+0000","last_became_active":"2021-11-03T17:10:14.821427+0000","last_became_peered":"2021-11-03T17:10:14.821427+0000","last_unstale":"2021-11-03T17:10:34.822363+0000","last_undegraded":"2021-11-03T17:10:34.822363+0000","last_fullsized":"2021-11-03T17:10:34.822363+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-03T17:09:49.024850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-03T17:09:49.024850+0000","last_clean_scrub_stamp":"2021-11-03T17:09:49.024850+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":38,"seq":163208757262,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5084,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":4736,"kb_avail":93740068,"statfs":{"total":95995035648,"available":95989829632,"internally_reserved":0,"allocated":348160,"data_stored":129702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4849664},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]}]},{"osd":4,"up_from":33,"seq":141733920785,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5148,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":93740004,"statfs":{"total":95995035648,"available":95989764096,"internally_reserved":0,"allocated":348160,"data_stored":129702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 3 17:11:15 2021","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.35799999999999998}]},{"osd":1,"last update":"Wed Nov 3 17:11:15 2021","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.40799999999999997}]},{"osd":2,"last update":"Wed Nov 3 17:11:15 2021","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.39000000000000001}]},{"osd":3,"last update":"Wed Nov 3 17:11:15 2021","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.32400000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]}]},{"osd":3,"up_from":27,"seq":115964117012,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5212,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":93739940,"statfs":{"total":95995035648,"available":95989698560,"internally_reserved":0,"allocated":348160,"data_stored":129702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 3 17:10:57 2021","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.496},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.36899999999999999}]},{"osd":1,"last update":"Wed Nov 3 17:10:57 2021","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.42699999999999999}]},{"osd":2,"last update":"Wed Nov 3 17:10:57 2021","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.47599999999999998}]},{"osd":4,"last update":"Wed Nov 3 17:11:16 2021","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.51800000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]}]},{"osd":2,"up_from":19,"seq":81604378650,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5212,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":93739940,"statfs":{"total":95995035648,"available":95989698560,"internally_reserved":0,"allocated":348160,"data_stored":129702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 3 17:10:36 2021","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.186,"5min":0.186,"15min":0.186},"max":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.33700000000000002}]},{"osd":1,"last update":"Wed Nov 3 17:10:36 2021","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.5,"5min":0.5,"15min":0.5},"last":0.32100000000000001}]},{"osd":3,"last update":"Wed Nov 3 17:10:54 2021","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.48699999999999999}]},{"osd":4,"last update":"Wed Nov 3 17:11:15 2021","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.6360000000000001,"5min":2.6360000000000001,"15min":2.6360000000000001},"last":0.61199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]}]},{"osd":0,"up_from":9,"seq":38654705699,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5340,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":93739812,"statfs":{"total":95995035648,"available":95989567488,"internally_reserved":0,"allocated":348160,"data_stored":129702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 3 17:11:16 2021","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32200000000000001,"15min":0.32100000000000001},"min":{"1min":0.23899999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.46500000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.26400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.501,"5min":0.501,"15min":0.501},"last":0.45800000000000002}]},{"osd":2,"last update":"Wed Nov 3 17:10:38 2021","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.502}]},{"osd":3,"last update":"Wed Nov 3 17:10:56 2021","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.752},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.83099999999999996}]},{"osd":4,"last update":"Wed Nov 3 17:11:15 2021","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.85999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]}]},{"osd":1,"up_from":14,"seq":60129542173,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5276,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":93739876,"statfs":{"total":95995035648,"available":95989633024,"internally_reserved":0,"allocated":348160,"data_stored":129702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 3 17:11:17 2021","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.32600000000000001,"15min":0.32400000000000001},"min":{"1min":0.22800000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.46100000000000002,"5min":0.53500000000000003,"15min":0.53500000000000003},"last":0.318},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"min":{"1min":0.26600000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.54500000000000004,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.29199999999999998}]},{"osd":2,"last update":"Wed Nov 3 17:10:39 2021","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"last":0.36399999999999999}]},{"osd":3,"last update":"Wed Nov 3 17:10:57 2021","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.75},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.72899999999999998}]},{"osd":4,"last update":"Wed Nov 3 17:11:17 2021","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.73599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2021-11-03T17:11:30.808 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph pg dump --format=json 2021-11-03T17:11:31.097 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c/mon.smithi027/config 2021-11-03T17:11:31.874 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:31 smithi027 conmon[32322]: audit 2021-11-03T17:11:30.201273+0000 mgr.smithi027.ufhhvb (mgr.14162) 196 : audit [DBG] from='client.14388 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2021-11-03T17:11:31.874 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:31 smithi027 conmon[32322]: cluster 2021-11-03T17:11:30.568668+0000 mgr.smithi027.ufhhvb (mgr.14162) 197 : cluster [DBG] pgmap v167: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:31.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:31 smithi115 conmon[34670]: audit 2021-11-03T17:11:30.201273+0000 mgr.smithi027.ufhhvb (mgr.14162) 196 : audit [DBG] 2021-11-03T17:11:31.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:31 smithi115 conmon[34670]: from='client.14388 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2021-11-03T17:11:31.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:31 smithi115 conmon[34670]: cluster 2021-11-03T17:11:30.568668+0000 mgr.smithi027.ufhhvb (mgr.14162) 197 : cluster [DBG] pgmap v167: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:32.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.713+0000 7fa05985c700 1 -- 172.21.15.27:0/2414602767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0540f67c0 msgr2=0x7fa0540f6c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:32.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.713+0000 7fa05985c700 1 --2- 172.21.15.27:0/2414602767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0540f67c0 0x7fa0540f6c20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7fa040004660 tx=0x7fa040009b30).stop 2021-11-03T17:11:32.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.713+0000 7fa05985c700 1 -- 172.21.15.27:0/2414602767 shutdown_connections 2021-11-03T17:11:32.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.713+0000 7fa05985c700 1 --2- 172.21.15.27:0/2414602767 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0540f67c0 0x7fa0540f6c20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:32.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.713+0000 7fa05985c700 1 --2- 172.21.15.27:0/2414602767 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0540f5520 0x7fa0540f5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:32.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.713+0000 7fa05985c700 1 -- 172.21.15.27:0/2414602767 >> 172.21.15.27:0/2414602767 conn(0x7fa0540f0ca0 msgr2=0x7fa0540f30c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:32.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.714+0000 7fa05985c700 1 -- 172.21.15.27:0/2414602767 shutdown_connections 2021-11-03T17:11:32.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.714+0000 7fa05985c700 1 -- 172.21.15.27:0/2414602767 wait complete. 2021-11-03T17:11:32.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.714+0000 7fa05985c700 1 Processor -- start 2021-11-03T17:11:32.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.714+0000 7fa05985c700 1 -- start start 2021-11-03T17:11:32.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.714+0000 7fa05985c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0540f5520 0x7fa054106eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:32.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.714+0000 7fa05985c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0540f67c0 0x7fa0541073a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:32.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.714+0000 7fa05985c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa054107950 con 0x7fa0540f67c0 2021-11-03T17:11:32.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.714+0000 7fa05985c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa054107a90 con 0x7fa0540f5520 2021-11-03T17:11:32.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.715+0000 7fa053fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0540f67c0 0x7fa0541073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:32.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.715+0000 7fa053fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0540f67c0 0x7fa0541073a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54878/0 (socket says 172.21.15.27:54878) 2021-11-03T17:11:32.715 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.715+0000 7fa053fff700 1 -- 172.21.15.27:0/2075458105 learned_addr learned my addr 172.21.15.27:0/2075458105 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:11:32.715 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.715+0000 7fa05885a700 1 --2- 172.21.15.27:0/2075458105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0540f5520 0x7fa054106eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:32.715 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.715+0000 7fa053fff700 1 -- 172.21.15.27:0/2075458105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0540f5520 msgr2=0x7fa054106eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:32.715 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.715+0000 7fa053fff700 1 --2- 172.21.15.27:0/2075458105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0540f5520 0x7fa054106eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:32.716 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.715+0000 7fa053fff700 1 -- 172.21.15.27:0/2075458105 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa040005040 con 0x7fa0540f67c0 2021-11-03T17:11:32.716 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.715+0000 7fa053fff700 1 --2- 172.21.15.27:0/2075458105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0540f67c0 0x7fa0541073a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7fa040009990 tx=0x7fa040009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:32.716 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.715+0000 7fa0517fa700 1 -- 172.21.15.27:0/2075458105 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa040018070 con 0x7fa0540f67c0 2021-11-03T17:11:32.716 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.715+0000 7fa0517fa700 1 -- 172.21.15.27:0/2075458105 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7fa040013410 con 0x7fa0540f67c0 2021-11-03T17:11:32.716 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.715+0000 7fa05985c700 1 -- 172.21.15.27:0/2075458105 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0541083d0 con 0x7fa0540f67c0 2021-11-03T17:11:32.717 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.715+0000 7fa0517fa700 1 -- 172.21.15.27:0/2075458105 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa04001de80 con 0x7fa0540f67c0 2021-11-03T17:11:32.717 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.715+0000 7fa05985c700 1 -- 172.21.15.27:0/2075458105 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa054108930 con 0x7fa0540f67c0 2021-11-03T17:11:32.717 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.716+0000 7fa05985c700 1 -- 172.21.15.27:0/2075458105 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa054045ad0 con 0x7fa0540f67c0 2021-11-03T17:11:32.717 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.716+0000 7fa0517fa700 1 -- 172.21.15.27:0/2075458105 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fa04001d420 con 0x7fa0540f67c0 2021-11-03T17:11:32.718 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.717+0000 7fa0517fa700 1 --2- 172.21.15.27:0/2075458105 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fa04405c2b0 0x7fa04405e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:32.718 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.717+0000 7fa0517fa700 1 -- 172.21.15.27:0/2075458105 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4519+0+0 (secure 0 0 0) 0x7fa040077790 con 0x7fa0540f67c0 2021-11-03T17:11:32.718 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.717+0000 7fa05885a700 1 --2- 172.21.15.27:0/2075458105 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fa04405c2b0 0x7fa04405e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:32.718 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.717+0000 7fa05885a700 1 --2- 172.21.15.27:0/2075458105 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fa04405c2b0 0x7fa04405e770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fa0540f6620 tx=0x7fa04800b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:32.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.721+0000 7fa0517fa700 1 -- 172.21.15.27:0/2075458105 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fa04004a7b0 con 0x7fa0540f67c0 2021-11-03T17:11:32.871 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.875+0000 7fa05985c700 1 -- 172.21.15.27:0/2075458105 --> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa054108b50 con 0x7fa04405c2b0 2021-11-03T17:11:32.872 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.877+0000 7fa0517fa700 1 -- 172.21.15.27:0/2075458105 <== mgr.14162 v2:172.21.15.27:6800/3802716887 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+27734 (secure 0 0 0) 0x7fa054108b50 con 0x7fa04405c2b0 2021-11-03T17:11:32.873 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:11:32.873 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.878+0000 7fa05985c700 1 -- 172.21.15.27:0/2075458105 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fa04405c2b0 msgr2=0x7fa04405e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:32.874 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.878+0000 7fa05985c700 1 --2- 172.21.15.27:0/2075458105 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fa04405c2b0 0x7fa04405e770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fa0540f6620 tx=0x7fa04800b040).stop 2021-11-03T17:11:32.874 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.878+0000 7fa05985c700 1 -- 172.21.15.27:0/2075458105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0540f67c0 msgr2=0x7fa0541073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:32.874 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.878+0000 7fa05985c700 1 --2- 172.21.15.27:0/2075458105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0540f67c0 0x7fa0541073a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7fa040009990 tx=0x7fa040009b30).stop 2021-11-03T17:11:32.875 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.879+0000 7fa05985c700 1 -- 172.21.15.27:0/2075458105 shutdown_connections 2021-11-03T17:11:32.875 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.879+0000 7fa05985c700 1 --2- 172.21.15.27:0/2075458105 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fa04405c2b0 0x7fa04405e770 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:32.875 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.879+0000 7fa05985c700 1 --2- 172.21.15.27:0/2075458105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0540f5520 0x7fa054106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:32.875 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.879+0000 7fa05985c700 1 --2- 172.21.15.27:0/2075458105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0540f67c0 0x7fa0541073a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:32.875 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.879+0000 7fa05985c700 1 -- 172.21.15.27:0/2075458105 >> 172.21.15.27:0/2075458105 conn(0x7fa0540f0ca0 msgr2=0x7fa0540fa210 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:32.876 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.879+0000 7fa05985c700 1 -- 172.21.15.27:0/2075458105 shutdown_connections 2021-11-03T17:11:32.876 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:32.879+0000 7fa05985c700 1 -- 172.21.15.27:0/2075458105 wait complete. 2021-11-03T17:11:32.876 INFO:teuthology.orchestra.run.smithi027.stderr:dumped all 2021-11-03T17:11:33.465 INFO:teuthology.orchestra.run.smithi027.stdout:{"pg_ready":true,"pg_map":{"version":168,"stamp":"2021-11-03T17:11:32.568870+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":3,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":31272,"kb_used_data":2040,"kb_used_omap":0,"kb_used_meta":29184,"kb_avail":562439640,"statfs":{"total":575970213888,"available":575938191360,"internally_reserved":0,"allocated":2088960,"data_stored":778212,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":29884416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.0","version":"0'0","reported_seq":"34","reported_epoch":"38","state":"active+clean","last_fresh":"2021-11-03T17:10:34.822363+0000","last_change":"2021-11-03T17:10:14.821826+0000","last_active":"2021-11-03T17:10:34.822363+0000","last_peered":"2021-11-03T17:10:34.822363+0000","last_clean":"2021-11-03T17:10:34.822363+0000","last_became_active":"2021-11-03T17:10:14.821427+0000","last_became_peered":"2021-11-03T17:10:14.821427+0000","last_unstale":"2021-11-03T17:10:34.822363+0000","last_undegraded":"2021-11-03T17:10:34.822363+0000","last_fullsized":"2021-11-03T17:10:34.822363+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2021-11-03T17:09:49.024850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2021-11-03T17:09:49.024850+0000","last_clean_scrub_stamp":"2021-11-03T17:09:49.024850+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":38,"seq":163208757263,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5084,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":4736,"kb_avail":93740068,"statfs":{"total":95995035648,"available":95989829632,"internally_reserved":0,"allocated":348160,"data_stored":129702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4849664},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44400000000000001}]}]},{"osd":4,"up_from":33,"seq":141733920786,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5148,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":4800,"kb_avail":93740004,"statfs":{"total":95995035648,"available":95989764096,"internally_reserved":0,"allocated":348160,"data_stored":129702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4915200},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 3 17:11:15 2021","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.48499999999999999}]},{"osd":1,"last update":"Wed Nov 3 17:11:15 2021","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.49099999999999999}]},{"osd":2,"last update":"Wed Nov 3 17:11:15 2021","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.51000000000000001}]},{"osd":3,"last update":"Wed Nov 3 17:11:15 2021","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.442},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.54000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]}]},{"osd":3,"up_from":27,"seq":115964117013,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5212,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":93739940,"statfs":{"total":95995035648,"available":95989698560,"internally_reserved":0,"allocated":348160,"data_stored":129702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 3 17:10:57 2021","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.78100000000000003},"last":0.40899999999999997}]},{"osd":1,"last update":"Wed Nov 3 17:10:57 2021","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.501,"15min":0.501},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.377},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.73499999999999999,"5min":0.73499999999999999,"15min":0.73499999999999999},"last":0.38800000000000001}]},{"osd":2,"last update":"Wed Nov 3 17:10:57 2021","interfaces":[{"interface":"back","average":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.314}]},{"osd":4,"last update":"Wed Nov 3 17:11:16 2021","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"last":0.44600000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]}]},{"osd":2,"up_from":19,"seq":81604378651,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":5212,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":4864,"kb_avail":93739940,"statfs":{"total":95995035648,"available":95989698560,"internally_reserved":0,"allocated":348160,"data_stored":129702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":4980736},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 3 17:10:36 2021","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.186,"5min":0.186,"15min":0.186},"max":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"last":0.32500000000000001}]},{"osd":1,"last update":"Wed Nov 3 17:10:36 2021","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.5,"5min":0.5,"15min":0.5},"last":0.34000000000000002}]},{"osd":3,"last update":"Wed Nov 3 17:10:54 2021","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.624,"5min":0.624,"15min":0.624},"last":0.51900000000000002}]},{"osd":4,"last update":"Wed Nov 3 17:11:15 2021","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.501},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":2.6360000000000001,"5min":2.6360000000000001,"15min":2.6360000000000001},"last":0.46500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]}]},{"osd":0,"up_from":9,"seq":38654705699,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5340,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":4992,"kb_avail":93739812,"statfs":{"total":95995035648,"available":95989567488,"internally_reserved":0,"allocated":348160,"data_stored":129702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5111808},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Nov 3 17:11:16 2021","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.32200000000000001,"15min":0.32100000000000001},"min":{"1min":0.23899999999999999,"5min":0.20599999999999999,"15min":0.20599999999999999},"max":{"1min":0.46500000000000002,"5min":0.49399999999999999,"15min":0.49399999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"min":{"1min":0.26400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.501,"5min":0.501,"15min":0.501},"last":0.45800000000000002}]},{"osd":2,"last update":"Wed Nov 3 17:10:38 2021","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"last":0.502}]},{"osd":3,"last update":"Wed Nov 3 17:10:56 2021","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"last":0.752},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"last":0.83099999999999996}]},{"osd":4,"last update":"Wed Nov 3 17:11:15 2021","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.72799999999999998},"last":0.85999999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79200000000000004}]}]},{"osd":1,"up_from":14,"seq":60129542174,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":5276,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":4928,"kb_avail":93739876,"statfs":{"total":95995035648,"available":95989633024,"internally_reserved":0,"allocated":348160,"data_stored":129702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":5046272},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Nov 3 17:11:17 2021","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.32600000000000001,"15min":0.32400000000000001},"min":{"1min":0.22800000000000001,"5min":0.224,"15min":0.224},"max":{"1min":0.46100000000000002,"5min":0.53500000000000003,"15min":0.53500000000000003},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"min":{"1min":0.26600000000000001,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.54500000000000004,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.505}]},{"osd":2,"last update":"Wed Nov 3 17:10:39 2021","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"last":0.29599999999999999}]},{"osd":3,"last update":"Wed Nov 3 17:10:57 2021","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.436},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.55500000000000005}]},{"osd":4,"last update":"Wed Nov 3 17:11:17 2021","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.40799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2021-11-03T17:11:33.467 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2021-11-03T17:11:33.467 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2021-11-03T17:11:33.467 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2021-11-03T17:11:33.468 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph health --format=json 2021-11-03T17:11:33.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:33 smithi027 conmon[32322]: cluster 2021-11-03T17:11:32.569144+0000 2021-11-03T17:11:33.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:33 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14162) 198 : cluster [DBG] pgmap v168: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:33.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:33 smithi027 conmon[32322]: audit 2021-11-03T17:11:33.312230+0000 mon.smithi027 (mon.0) 455 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:11:33.771 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c/mon.smithi027/config 2021-11-03T17:11:33.955 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:33 smithi115 conmon[34670]: cluster 2021-11-03T17:11:32.569144+0000 mgr.smithi027.ufhhvb (mgr.14162 2021-11-03T17:11:33.955 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:33 smithi115 conmon[34670]: ) 198 : cluster [DBG] pgmap v168: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:33.955 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:33 smithi115 conmon[34670]: audit 2021-11-03T17:11:33.312230+0000 mon.smithi027 (mon.0) 455 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:11:34.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:34 smithi027 conmon[32322]: audit 2021-11-03T17:11:32.876594+0000 mgr.smithi027.ufhhvb ( 2021-11-03T17:11:34.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:34 smithi027 conmon[32322]: mgr.14162) 199 : audit [DBG] from='client.14392 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2021-11-03T17:11:34.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:34 smithi027 conmon[32322]: audit 2021-11-03T17:11:33.607894+0000 mon.smithi027 (mon.0) 456 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:11:34.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:34 smithi115 conmon[34670]: audit 2021-11-03T17:11:32.876594+0000 mgr.smithi027.ufhhvb (mgr.14162) 2021-11-03T17:11:34.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:34 smithi115 conmon[34670]: 199 : audit [DBG] from='client.14392 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2021-11-03T17:11:34.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:34 smithi115 conmon[34670]: audit 2021-11-03T17:11:33.607894+0000 mon.smithi027 (mon.0) 456 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:11:35.402 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.406+0000 7fc283b7c700 1 -- 172.21.15.27:0/1355661956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc27c060870 msgr2=0x7fc27c0f93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:35.403 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.406+0000 7fc283b7c700 1 --2- 172.21.15.27:0/1355661956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc27c060870 0x7fc27c0f93b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7fc274004660 tx=0x7fc274009b30).stop 2021-11-03T17:11:35.403 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.406+0000 7fc283b7c700 1 -- 172.21.15.27:0/1355661956 shutdown_connections 2021-11-03T17:11:35.403 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.406+0000 7fc283b7c700 1 --2- 172.21.15.27:0/1355661956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc27c0f9930 0x7fc27c0fbd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:11:35.403 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.406+0000 7fc283b7c700 1 --2- 172.21.15.27:0/1355661956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc27c060870 0x7fc27c0f93b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:35.404 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.406+0000 7fc283b7c700 1 -- 172.21.15.27:0/1355661956 >> 172.21.15.27:0/1355661956 conn(0x7fc27c0f0e10 msgr2=0x7fc27c0f3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:35.404 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.406+0000 7fc283b7c700 1 -- 172.21.15.27:0/1355661956 shutdown_connections 2021-11-03T17:11:35.404 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.406+0000 7fc283b7c700 1 -- 172.21.15.27:0/1355661956 wait complete. 2021-11-03T17:11:35.404 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.406+0000 7fc283b7c700 1 Processor -- start 2021-11-03T17:11:35.405 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.406+0000 7fc283b7c700 1 -- start start 2021-11-03T17:11:35.405 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.407+0000 7fc283b7c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc27c060870 0x7fc27c107030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:35.405 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.407+0000 7fc283b7c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc27c0f9930 0x7fc27c107520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:35.405 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.407+0000 7fc283b7c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27c107b60 con 0x7fc27c0f9930 2021-11-03T17:11:35.406 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.407+0000 7fc283b7c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc27c108410 con 0x7fc27c060870 2021-11-03T17:11:35.406 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.407+0000 7fc281117700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc27c0f9930 0x7fc27c107520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:35.406 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.407+0000 7fc281117700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc27c0f9930 0x7fc27c107520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54888/0 (socket says 172.21.15.27:54888) 2021-11-03T17:11:35.406 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.407+0000 7fc281117700 1 -- 172.21.15.27:0/1062737557 learned_addr learned my addr 172.21.15.27:0/1062737557 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:11:35.407 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.407+0000 7fc281918700 1 --2- 172.21.15.27:0/1062737557 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc27c060870 0x7fc27c107030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:35.407 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.407+0000 7fc281117700 1 -- 172.21.15.27:0/1062737557 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc27c060870 msgr2=0x7fc27c107030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:35.407 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.407+0000 7fc281117700 1 --2- 172.21.15.27:0/1062737557 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc27c060870 0x7fc27c107030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:35.407 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.407+0000 7fc281117700 1 -- 172.21.15.27:0/1062737557 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc274005040 con 0x7fc27c0f9930 2021-11-03T17:11:35.407 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.407+0000 7fc281117700 1 --2- 172.21.15.27:0/1062737557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc27c0f9930 0x7fc27c107520 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7fc2700093f0 tx=0x7fc270006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:35.408 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.407+0000 7fc27a7fc700 1 -- 172.21.15.27:0/1062737557 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc270013640 con 0x7fc27c0f9930 2021-11-03T17:11:35.408 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.407+0000 7fc283b7c700 1 -- 172.21.15.27:0/1062737557 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc27c1086f0 con 0x7fc27c0f9930 2021-11-03T17:11:35.408 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.407+0000 7fc27a7fc700 1 -- 172.21.15.27:0/1062737557 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7fc270013c30 con 0x7fc27c0f9930 2021-11-03T17:11:35.408 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.407+0000 7fc27a7fc700 1 -- 172.21.15.27:0/1062737557 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc270012be0 con 0x7fc27c0f9930 2021-11-03T17:11:35.409 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.407+0000 7fc283b7c700 1 -- 172.21.15.27:0/1062737557 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc27c108cb0 con 0x7fc27c0f9930 2021-11-03T17:11:35.410 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.408+0000 7fc283b7c700 1 -- 172.21.15.27:0/1062737557 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc27c059100 con 0x7fc27c0f9930 2021-11-03T17:11:35.410 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.409+0000 7fc27a7fc700 1 -- 172.21.15.27:0/1062737557 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fc27001a060 con 0x7fc27c0f9930 2021-11-03T17:11:35.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.409+0000 7fc27a7fc700 1 --2- 172.21.15.27:0/1062737557 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fc26405c2a0 0x7fc26405e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:35.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.409+0000 7fc27a7fc700 1 -- 172.21.15.27:0/1062737557 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4519+0+0 (secure 0 0 0) 0x7fc270075b40 con 0x7fc27c0f9930 2021-11-03T17:11:35.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.409+0000 7fc281918700 1 --2- 172.21.15.27:0/1062737557 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fc26405c2a0 0x7fc26405e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:35.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.409+0000 7fc281918700 1 --2- 172.21.15.27:0/1062737557 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fc26405c2a0 0x7fc26405e760 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fc274005010 tx=0x7fc274009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:35.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.413+0000 7fc27a7fc700 1 -- 172.21.15.27:0/1062737557 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fc270048b00 con 0x7fc27c0f9930 2021-11-03T17:11:35.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.632+0000 7fc283b7c700 1 -- 172.21.15.27:0/1062737557 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc27c0ff030 con 0x7fc27c0f9930 2021-11-03T17:11:35.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.632+0000 7fc27a7fc700 1 -- 172.21.15.27:0/1062737557 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+45 (secure 0 0 0) 0x7fc27004c120 con 0x7fc27c0f9930 2021-11-03T17:11:35.628 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:11:35.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.634+0000 7fc283b7c700 1 -- 172.21.15.27:0/1062737557 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fc26405c2a0 msgr2=0x7fc26405e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:35.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.634+0000 7fc283b7c700 1 --2- 172.21.15.27:0/1062737557 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fc26405c2a0 0x7fc26405e760 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7fc274005010 tx=0x7fc274009b30).stop 2021-11-03T17:11:35.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.634+0000 7fc283b7c700 1 -- 172.21.15.27:0/1062737557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc27c0f9930 msgr2=0x7fc27c107520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:35.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.634+0000 7fc283b7c700 1 --2- 172.21.15.27:0/1062737557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc27c0f9930 0x7fc27c107520 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7fc2700093f0 tx=0x7fc270006ec0).stop 2021-11-03T17:11:35.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.634+0000 7fc283b7c700 1 -- 172.21.15.27:0/1062737557 shutdown_connections 2021-11-03T17:11:35.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.634+0000 7fc283b7c700 1 --2- 172.21.15.27:0/1062737557 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fc26405c2a0 0x7fc26405e760 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:35.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.634+0000 7fc283b7c700 1 --2- 172.21.15.27:0/1062737557 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc27c060870 0x7fc27c107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:35.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.634+0000 7fc283b7c700 1 --2- 172.21.15.27:0/1062737557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc27c0f9930 0x7fc27c107520 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:35.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.634+0000 7fc283b7c700 1 -- 172.21.15.27:0/1062737557 >> 172.21.15.27:0/1062737557 conn(0x7fc27c0f0e10 msgr2=0x7fc27c0f3250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:35.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.634+0000 7fc283b7c700 1 -- 172.21.15.27:0/1062737557 shutdown_connections 2021-11-03T17:11:35.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:35.634+0000 7fc283b7c700 1 -- 172.21.15.27:0/1062737557 wait complete. 2021-11-03T17:11:35.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:35 smithi115 conmon[34670]: cluster 2021-11-03T17:11:34.569456+0000 mgr.smithi027.ufhhvb (mgr. 2021-11-03T17:11:35.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:35 smithi115 conmon[34670]: 14162) 200 : cluster [DBG] pgmap v169: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:36.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:35 smithi027 conmon[32322]: cluster 2021-11-03T17:11:34.569456+0000 2021-11-03T17:11:36.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:35 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14162) 200 : cluster [DBG] pgmap v169: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:36.284 INFO:teuthology.orchestra.run.smithi027.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2021-11-03T17:11:36.284 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2021-11-03T17:11:36.285 INFO:tasks.cephadm:Setup complete, yielding 2021-11-03T17:11:36.285 INFO:teuthology.run_tasks:Running task print... 2021-11-03T17:11:36.298 INFO:teuthology.task.print:**** done starting v16.2.4 2021-11-03T17:11:36.299 INFO:teuthology.run_tasks:Running task cephadm.shell... 2021-11-03T17:11:36.338 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi027.front.sepia.ceph.com 2021-11-03T17:11:36.339 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- bash -c 'ceph orch status' 2021-11-03T17:11:36.879 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:36 smithi027 conmon[32322]: audit 2021-11-03T17:11:35.632844+0000 mon.smithi027 (mon.0) 457 : audit [DBG] from='client.? 172.21.15.27:0/1062737557' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2021-11-03T17:11:36.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:36 smithi115 conmon[34670]: audit 2021-11-03T17:11:35.632844+0000 mon.smithi027 (mon.0) 457 2021-11-03T17:11:36.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:36 smithi115 conmon[34670]: : audit [DBG] from='client.? 172.21.15.27:0/1062737557' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2021-11-03T17:11:37.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:37 smithi115 conmon[34670]: cluster 2021-11-03T17:11:36.569777+0000 mgr.smithi027.ufhhvb ( 2021-11-03T17:11:37.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:37 smithi115 conmon[34670]: mgr.14162) 201 : cluster [DBG] pgmap v170: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:38.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:37 smithi027 conmon[32322]: cluster 2021-11-03T17:11:36.569777+0000 mgr.smithi027.ufhhvb (mgr.14162 2021-11-03T17:11:38.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:37 smithi027 conmon[32322]: ) 201 : cluster [DBG] pgmap v170: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:38.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.230+0000 7fb69ff1f700 1 -- 172.21.15.27:0/3611340660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6980f5ed0 msgr2=0x7fb6980f6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:38.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.230+0000 7fb69ff1f700 1 --2- 172.21.15.27:0/3611340660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6980f5ed0 0x7fb6980f6900 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7fb68c004660 tx=0x7fb68c009b30).stop 2021-11-03T17:11:38.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.230+0000 7fb69ff1f700 1 -- 172.21.15.27:0/3611340660 shutdown_connections 2021-11-03T17:11:38.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.230+0000 7fb69ff1f700 1 --2- 172.21.15.27:0/3611340660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6980f5ed0 0x7fb6980f6900 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:38.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.230+0000 7fb69ff1f700 1 --2- 172.21.15.27:0/3611340660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6980f55c0 0x7fb6980f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:11:38.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.230+0000 7fb69ff1f700 1 -- 172.21.15.27:0/3611340660 >> 172.21.15.27:0/3611340660 conn(0x7fb6980f0dc0 msgr2=0x7fb6980f31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:38.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.230+0000 7fb69ff1f700 1 -- 172.21.15.27:0/3611340660 shutdown_connections 2021-11-03T17:11:38.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.230+0000 7fb69ff1f700 1 -- 172.21.15.27:0/3611340660 wait complete. 2021-11-03T17:11:38.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.231+0000 7fb69ff1f700 1 Processor -- start 2021-11-03T17:11:38.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.231+0000 7fb69ff1f700 1 -- start start 2021-11-03T17:11:38.230 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.231+0000 7fb69ff1f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6980f55c0 0x7fb698109c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:38.230 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.231+0000 7fb69ff1f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6980f5ed0 0x7fb69810c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:38.230 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.231+0000 7fb69ff1f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb698060630 con 0x7fb6980f5ed0 2021-11-03T17:11:38.231 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.231+0000 7fb69ff1f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb698060770 con 0x7fb6980f55c0 2021-11-03T17:11:38.231 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.231+0000 7fb69d4ba700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6980f5ed0 0x7fb69810c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:38.231 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.231+0000 7fb69d4ba700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6980f5ed0 0x7fb69810c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54898/0 (socket says 172.21.15.27:54898) 2021-11-03T17:11:38.231 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.231+0000 7fb69d4ba700 1 -- 172.21.15.27:0/2701886815 learned_addr learned my addr 172.21.15.27:0/2701886815 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:11:38.231 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.231+0000 7fb69dcbb700 1 --2- 172.21.15.27:0/2701886815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6980f55c0 0x7fb698109c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:38.232 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.231+0000 7fb69d4ba700 1 -- 172.21.15.27:0/2701886815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6980f55c0 msgr2=0x7fb698109c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:38.232 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.231+0000 7fb69d4ba700 1 --2- 172.21.15.27:0/2701886815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6980f55c0 0x7fb698109c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:38.232 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.231+0000 7fb69d4ba700 1 -- 172.21.15.27:0/2701886815 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb68c005040 con 0x7fb6980f5ed0 2021-11-03T17:11:38.232 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.231+0000 7fb69d4ba700 1 --2- 172.21.15.27:0/2701886815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6980f5ed0 0x7fb69810c130 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7fb68c009990 tx=0x7fb68c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:38.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.232+0000 7fb6967fc700 1 -- 172.21.15.27:0/2701886815 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb68c018070 con 0x7fb6980f5ed0 2021-11-03T17:11:38.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.232+0000 7fb69ff1f700 1 -- 172.21.15.27:0/2701886815 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb69810c730 con 0x7fb6980f5ed0 2021-11-03T17:11:38.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.232+0000 7fb6967fc700 1 -- 172.21.15.27:0/2701886815 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7fb68c013410 con 0x7fb6980f5ed0 2021-11-03T17:11:38.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.232+0000 7fb6967fc700 1 -- 172.21.15.27:0/2701886815 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb68c01c620 con 0x7fb6980f5ed0 2021-11-03T17:11:38.234 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.232+0000 7fb69ff1f700 1 -- 172.21.15.27:0/2701886815 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb69810cc90 con 0x7fb6980f5ed0 2021-11-03T17:11:38.235 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.233+0000 7fb69ff1f700 1 -- 172.21.15.27:0/2701886815 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb698045ad0 con 0x7fb6980f5ed0 2021-11-03T17:11:38.235 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.233+0000 7fb6967fc700 1 -- 172.21.15.27:0/2701886815 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fb68c01d420 con 0x7fb6980f5ed0 2021-11-03T17:11:38.235 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.233+0000 7fb6967fc700 1 --2- 172.21.15.27:0/2701886815 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fb68005c2b0 0x7fb68005e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:38.236 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.233+0000 7fb6967fc700 1 -- 172.21.15.27:0/2701886815 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4519+0+0 (secure 0 0 0) 0x7fb68c077a40 con 0x7fb6980f5ed0 2021-11-03T17:11:38.236 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.233+0000 7fb69dcbb700 1 --2- 172.21.15.27:0/2701886815 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fb68005c2b0 0x7fb68005e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:38.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.234+0000 7fb69dcbb700 1 --2- 172.21.15.27:0/2701886815 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fb68005c2b0 0x7fb68005e770 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fb698060220 tx=0x7fb690008040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:38.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.237+0000 7fb6967fc700 1 -- 172.21.15.27:0/2701886815 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fb68c04aae0 con 0x7fb6980f5ed0 2021-11-03T17:11:38.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.426+0000 7fb69ff1f700 1 -- 172.21.15.27:0/2701886815 --> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb6980fdfa0 con 0x7fb68005c2b0 2021-11-03T17:11:38.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.427+0000 7fb6967fc700 1 -- 172.21.15.27:0/2701886815 <== mgr.14162 v2:172.21.15.27:6800/3802716887 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fb6980fdfa0 con 0x7fb68005c2b0 2021-11-03T17:11:38.422 INFO:teuthology.orchestra.run.smithi027.stdout:Backend: cephadm 2021-11-03T17:11:38.422 INFO:teuthology.orchestra.run.smithi027.stdout:Available: Yes 2021-11-03T17:11:38.423 INFO:teuthology.orchestra.run.smithi027.stdout:Paused: No 2021-11-03T17:11:38.424 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.428+0000 7fb69ff1f700 1 -- 172.21.15.27:0/2701886815 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fb68005c2b0 msgr2=0x7fb68005e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:38.424 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.428+0000 7fb69ff1f700 1 --2- 172.21.15.27:0/2701886815 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fb68005c2b0 0x7fb68005e770 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7fb698060220 tx=0x7fb690008040).stop 2021-11-03T17:11:38.424 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.428+0000 7fb69ff1f700 1 -- 172.21.15.27:0/2701886815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6980f5ed0 msgr2=0x7fb69810c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:38.425 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.428+0000 7fb69ff1f700 1 --2- 172.21.15.27:0/2701886815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6980f5ed0 0x7fb69810c130 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7fb68c009990 tx=0x7fb68c009b30).stop 2021-11-03T17:11:38.425 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.429+0000 7fb69ff1f700 1 -- 172.21.15.27:0/2701886815 shutdown_connections 2021-11-03T17:11:38.425 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.429+0000 7fb69ff1f700 1 --2- 172.21.15.27:0/2701886815 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fb68005c2b0 0x7fb68005e770 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:38.426 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.429+0000 7fb69ff1f700 1 --2- 172.21.15.27:0/2701886815 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6980f55c0 0x7fb698109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:38.426 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.429+0000 7fb69ff1f700 1 --2- 172.21.15.27:0/2701886815 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6980f5ed0 0x7fb69810c130 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:38.426 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.429+0000 7fb69ff1f700 1 -- 172.21.15.27:0/2701886815 >> 172.21.15.27:0/2701886815 conn(0x7fb6980f0dc0 msgr2=0x7fb6980fa350 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:38.426 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.429+0000 7fb69ff1f700 1 -- 172.21.15.27:0/2701886815 shutdown_connections 2021-11-03T17:11:38.427 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:38.429+0000 7fb69ff1f700 1 -- 172.21.15.27:0/2701886815 wait complete. 2021-11-03T17:11:38.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:38 smithi027 conmon[32322]: audit 2021-11-03T17:11:38.427605+0000 mgr.smithi027.ufhhvb (mgr.14162) 2021-11-03T17:11:38.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:38 smithi027 conmon[32322]: 202 : audit [DBG] from='client.14400 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:11:38.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:38 smithi027 conmon[32322]: cluster 2021-11-03T17:11:38.570262+0000 mgr.smithi027.ufhhvb (mgr.14162) 203 : cluster [DBG] pgmap v171: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:38.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:38 smithi115 conmon[34670]: audit 2021-11-03T17:11:38.427605+0000 mgr.smithi027.ufhhvb (mgr. 2021-11-03T17:11:38.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:38 smithi115 conmon[34670]: 14162) 202 : audit [DBG] from='client.14400 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:11:38.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:38 smithi115 conmon[34670]: cluster 2021-11-03T17:11:38.570262+0000 mgr.smithi027.ufhhvb (mgr.14162) 203 : cluster [DBG] pgmap v171: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:39.024 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- bash -c 'ceph orch ps' 2021-11-03T17:11:40.808 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:40.812+0000 7fc5e95d2700 1 -- 172.21.15.27:0/766802534 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5e40f77d0 msgr2=0x7fc5e40f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:40.808 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:40.812+0000 7fc5e95d2700 1 --2- 172.21.15.27:0/766802534 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5e40f77d0 0x7fc5e40f7c30 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7fc5cc004660 tx=0x7fc5cc009b30).stop 2021-11-03T17:11:40.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:40.812+0000 7fc5e95d2700 1 -- 172.21.15.27:0/766802534 shutdown_connections 2021-11-03T17:11:40.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:40.812+0000 7fc5e95d2700 1 --2- 172.21.15.27:0/766802534 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5e40f77d0 0x7fc5e40f7c30 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:40.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:40.813+0000 7fc5e95d2700 1 --2- 172.21.15.27:0/766802534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5e40f6530 0x7fc5e40f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:40.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:40.813+0000 7fc5e95d2700 1 -- 172.21.15.27:0/766802534 >> 172.21.15.27:0/766802534 conn(0x7fc5e40f1cd0 msgr2=0x7fc5e40f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:40.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:40.813+0000 7fc5e95d2700 1 -- 172.21.15.27:0/766802534 shutdown_connections 2021-11-03T17:11:40.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:40.813+0000 7fc5e95d2700 1 -- 172.21.15.27:0/766802534 wait complete. 2021-11-03T17:11:40.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:40.813+0000 7fc5e95d2700 1 Processor -- start 2021-11-03T17:11:40.811 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:40.813+0000 7fc5e95d2700 1 -- start start 2021-11-03T17:11:40.813 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:40.814+0000 7fc5e95d2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5e40f6530 0x7fc5e406b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:40.813 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:40.814+0000 7fc5e95d2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5e40f77d0 0x7fc5e4069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:40.813 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:40.814+0000 7fc5e95d2700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e406a380 con 0x7fc5e40f77d0 2021-11-03T17:11:40.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:40.814+0000 7fc5e95d2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5e406a4c0 con 0x7fc5e40f6530 2021-11-03T17:11:40.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:40.814+0000 7fc5e37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5e40f77d0 0x7fc5e4069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:40.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:40.814+0000 7fc5e37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5e40f77d0 0x7fc5e4069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54908/0 (socket says 172.21.15.27:54908) 2021-11-03T17:11:40.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:40.814+0000 7fc5e37fe700 1 -- 172.21.15.27:0/2688278615 learned_addr learned my addr 172.21.15.27:0/2688278615 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:11:40.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:40.814+0000 7fc5e3fff700 1 --2- 172.21.15.27:0/2688278615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5e40f6530 0x7fc5e406b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:40.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:40.814+0000 7fc5e37fe700 1 -- 172.21.15.27:0/2688278615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5e40f6530 msgr2=0x7fc5e406b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:40.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:40.814+0000 7fc5e37fe700 1 --2- 172.21.15.27:0/2688278615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5e40f6530 0x7fc5e406b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:40.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:40.814+0000 7fc5e37fe700 1 -- 172.21.15.27:0/2688278615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5cc005040 con 0x7fc5e40f77d0 2021-11-03T17:11:40.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:40.814+0000 7fc5e3fff700 1 --2- 172.21.15.27:0/2688278615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5e40f6530 0x7fc5e406b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:11:40.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:40.814+0000 7fc5e37fe700 1 --2- 172.21.15.27:0/2688278615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5e40f77d0 0x7fc5e4069e20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7fc5cc009990 tx=0x7fc5cc009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:40.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:40.814+0000 7fc5e0ff9700 1 -- 172.21.15.27:0/2688278615 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5cc018070 con 0x7fc5e40f77d0 2021-11-03T17:11:40.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:40.814+0000 7fc5e0ff9700 1 -- 172.21.15.27:0/2688278615 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7fc5cc013410 con 0x7fc5e40f77d0 2021-11-03T17:11:40.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:40.814+0000 7fc5e95d2700 1 -- 172.21.15.27:0/2688278615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5e406a740 con 0x7fc5e40f77d0 2021-11-03T17:11:40.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:40.814+0000 7fc5e0ff9700 1 -- 172.21.15.27:0/2688278615 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5cc01c3c0 con 0x7fc5e40f77d0 2021-11-03T17:11:40.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:40.814+0000 7fc5e95d2700 1 -- 172.21.15.27:0/2688278615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5e406aca0 con 0x7fc5e40f77d0 2021-11-03T17:11:40.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:40.815+0000 7fc5e95d2700 1 -- 172.21.15.27:0/2688278615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5e4045ad0 con 0x7fc5e40f77d0 2021-11-03T17:11:40.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:40.816+0000 7fc5e0ff9700 1 -- 172.21.15.27:0/2688278615 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fc5cc01d420 con 0x7fc5e40f77d0 2021-11-03T17:11:40.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:40.816+0000 7fc5e0ff9700 1 --2- 172.21.15.27:0/2688278615 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fc5d005c300 0x7fc5d005e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:40.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:40.816+0000 7fc5e0ff9700 1 -- 172.21.15.27:0/2688278615 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4519+0+0 (secure 0 0 0) 0x7fc5cc077880 con 0x7fc5e40f77d0 2021-11-03T17:11:40.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:40.816+0000 7fc5e3fff700 1 --2- 172.21.15.27:0/2688278615 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fc5d005c300 0x7fc5d005e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:40.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:40.817+0000 7fc5e3fff700 1 --2- 172.21.15.27:0/2688278615 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fc5d005c300 0x7fc5d005e7c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fc5d4009880 tx=0x7fc5d400b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:40.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:40.820+0000 7fc5e0ff9700 1 -- 172.21.15.27:0/2688278615 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fc5cc04a8a0 con 0x7fc5e40f77d0 2021-11-03T17:11:40.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:40 smithi115 conmon[34670]: cluster 2021-11-03T17:11:40.570665+0000 mgr.smithi027.ufhhvb (mgr.14162) 204 : cluster 2021-11-03T17:11:40.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:40 smithi115 conmon[34670]: [DBG] pgmap v172: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:40.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:41.003+0000 7fc5e95d2700 1 -- 172.21.15.27:0/2688278615 --> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5e40fee00 con 0x7fc5d005c300 2021-11-03T17:11:41.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:41.013+0000 7fc5e0ff9700 1 -- 172.21.15.27:0/2688278615 <== mgr.14162 v2:172.21.15.27:6800/3802716887 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2160 (secure 0 0 0) 0x7fc5e40fee00 con 0x7fc5d005c300 2021-11-03T17:11:41.008 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE VERSION IMAGE ID CONTAINER ID 2021-11-03T17:11:41.009 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (3m) 89s ago 4m 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:11:41.009 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (4m) 89s ago 4m 16.2.4 8d91d370c2b8 d494472a47c0 2021-11-03T17:11:41.009 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (3m) 70s ago 3m 16.2.4 8d91d370c2b8 f84fd4042530 2021-11-03T17:11:41.010 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (3m) 89s ago 4m 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:11:41.010 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:9283 running (5m) 89s ago 5m 16.2.4 8d91d370c2b8 999298a5769c 2021-11-03T17:11:41.010 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8443,9283 running (3m) 70s ago 3m 16.2.4 8d91d370c2b8 722d63932db9 2021-11-03T17:11:41.010 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (5m) 89s ago 5m 16.2.4 8d91d370c2b8 060d5938214f 2021-11-03T17:11:41.011 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (3m) 70s ago 3m 16.2.4 8d91d370c2b8 319323f09834 2021-11-03T17:11:41.011 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (4m) 89s ago 4m 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:11:41.011 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (3m) 70s ago 3m 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:11:41.011 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (2m) 89s ago 2m 16.2.4 8d91d370c2b8 17f79a815a0f 2021-11-03T17:11:41.012 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (2m) 89s ago 2m 16.2.4 8d91d370c2b8 422be0cac0db 2021-11-03T17:11:41.012 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (2m) 89s ago 2m 16.2.4 8d91d370c2b8 c211ef5551c9 2021-11-03T17:11:41.012 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (113s) 70s ago 113s 16.2.4 8d91d370c2b8 b1486ec1b619 2021-11-03T17:11:41.012 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (93s) 70s ago 93s 16.2.4 8d91d370c2b8 8b91230c69e5 2021-11-03T17:11:41.012 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (73s) 70s ago 73s 16.2.4 8d91d370c2b8 4a54c4d6b6f5 2021-11-03T17:11:41.013 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (2m) 89s ago 3m 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:11:41.014 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:41.015+0000 7fc5e95d2700 1 -- 172.21.15.27:0/2688278615 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fc5d005c300 msgr2=0x7fc5d005e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:41.014 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:41.015+0000 7fc5e95d2700 1 --2- 172.21.15.27:0/2688278615 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fc5d005c300 0x7fc5d005e7c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fc5d4009880 tx=0x7fc5d400b040).stop 2021-11-03T17:11:41.014 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:41.015+0000 7fc5e95d2700 1 -- 172.21.15.27:0/2688278615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5e40f77d0 msgr2=0x7fc5e4069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:41.014 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:41.015+0000 7fc5e95d2700 1 --2- 172.21.15.27:0/2688278615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5e40f77d0 0x7fc5e4069e20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7fc5cc009990 tx=0x7fc5cc009b30).stop 2021-11-03T17:11:41.015 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:41.015+0000 7fc5e95d2700 1 -- 172.21.15.27:0/2688278615 shutdown_connections 2021-11-03T17:11:41.015 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:41.015+0000 7fc5e95d2700 1 --2- 172.21.15.27:0/2688278615 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fc5d005c300 0x7fc5d005e7c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:41.015 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:41.015+0000 7fc5e95d2700 1 --2- 172.21.15.27:0/2688278615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc5e40f6530 0x7fc5e406b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:41.015 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:41.015+0000 7fc5e95d2700 1 --2- 172.21.15.27:0/2688278615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc5e40f77d0 0x7fc5e4069e20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:41.016 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:41.015+0000 7fc5e95d2700 1 -- 172.21.15.27:0/2688278615 >> 172.21.15.27:0/2688278615 conn(0x7fc5e40f1cd0 msgr2=0x7fc5e40fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:41.016 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:41.015+0000 7fc5e95d2700 1 -- 172.21.15.27:0/2688278615 shutdown_connections 2021-11-03T17:11:41.016 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:41.015+0000 7fc5e95d2700 1 -- 172.21.15.27:0/2688278615 wait complete. 2021-11-03T17:11:41.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:40 smithi027 conmon[32322]: cluster 2021-11-03T17:11:40.570665+0000 mgr.smithi027.ufhhvb (mgr.14162) 204 : cluster [DBG] pgmap v172: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:41.582 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- bash -c 'ceph orch ls' 2021-11-03T17:11:42.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:42 smithi027 conmon[32322]: audit 2021-11-03T17:11:41.004396+0000 mgr.smithi027.ufhhvb (mgr.14162) 205 : audit [DBG] from='client.14404 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:11:42.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:42 smithi115 conmon[34670]: audit 2021-11-03T17:11:41.004396+0000 mgr.smithi027.ufhhvb (mgr.14162) 205 : audit [DBG] from='client.14404 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:11:43.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.446+0000 7fe6e5796700 1 -- 172.21.15.27:0/398746281 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6e00602b0 msgr2=0x7fe6e00f9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:43.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.446+0000 7fe6e5796700 1 --2- 172.21.15.27:0/398746281 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6e00602b0 0x7fe6e00f9180 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7fe6d0004660 tx=0x7fe6d0009b30).stop 2021-11-03T17:11:43.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.446+0000 7fe6e5796700 1 -- 172.21.15.27:0/398746281 shutdown_connections 2021-11-03T17:11:43.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.446+0000 7fe6e5796700 1 --2- 172.21.15.27:0/398746281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6e00f9700 0x7fe6e00fbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:11:43.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.446+0000 7fe6e5796700 1 --2- 172.21.15.27:0/398746281 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6e00602b0 0x7fe6e00f9180 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:43.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.446+0000 7fe6e5796700 1 -- 172.21.15.27:0/398746281 >> 172.21.15.27:0/398746281 conn(0x7fe6e00f0ba0 msgr2=0x7fe6e00f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:43.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.446+0000 7fe6e5796700 1 -- 172.21.15.27:0/398746281 shutdown_connections 2021-11-03T17:11:43.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.446+0000 7fe6e5796700 1 -- 172.21.15.27:0/398746281 wait complete. 2021-11-03T17:11:43.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.447+0000 7fe6e5796700 1 Processor -- start 2021-11-03T17:11:43.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.447+0000 7fe6e5796700 1 -- start start 2021-11-03T17:11:43.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.447+0000 7fe6e5796700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6e00602b0 0x7fe6e0106e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:43.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.447+0000 7fe6e5796700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6e00f9700 0x7fe6e0107300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:43.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.447+0000 7fe6e5796700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e0107990 con 0x7fe6e00f9700 2021-11-03T17:11:43.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.447+0000 7fe6e5796700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e0108110 con 0x7fe6e00602b0 2021-11-03T17:11:43.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.447+0000 7fe6df7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6e00f9700 0x7fe6e0107300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:43.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.448+0000 7fe6df7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6e00f9700 0x7fe6e0107300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54918/0 (socket says 172.21.15.27:54918) 2021-11-03T17:11:43.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.448+0000 7fe6df7fe700 1 -- 172.21.15.27:0/657574672 learned_addr learned my addr 172.21.15.27:0/657574672 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:11:43.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.448+0000 7fe6dffff700 1 --2- 172.21.15.27:0/657574672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6e00602b0 0x7fe6e0106e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:43.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.448+0000 7fe6df7fe700 1 -- 172.21.15.27:0/657574672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6e00602b0 msgr2=0x7fe6e0106e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:43.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.448+0000 7fe6df7fe700 1 --2- 172.21.15.27:0/657574672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6e00602b0 0x7fe6e0106e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:43.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.448+0000 7fe6df7fe700 1 -- 172.21.15.27:0/657574672 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6d0005040 con 0x7fe6e00f9700 2021-11-03T17:11:43.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.448+0000 7fe6dffff700 1 --2- 172.21.15.27:0/657574672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6e00602b0 0x7fe6e0106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:11:43.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.448+0000 7fe6df7fe700 1 --2- 172.21.15.27:0/657574672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6e00f9700 0x7fe6e0107300 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7fe6c80093f0 tx=0x7fe6c8006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:43.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.448+0000 7fe6dcff9700 1 -- 172.21.15.27:0/657574672 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6c8009ea0 con 0x7fe6e00f9700 2021-11-03T17:11:43.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.448+0000 7fe6e5796700 1 -- 172.21.15.27:0/657574672 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6e0108390 con 0x7fe6e00f9700 2021-11-03T17:11:43.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.448+0000 7fe6dcff9700 1 -- 172.21.15.27:0/657574672 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7fe6c8013ad0 con 0x7fe6e00f9700 2021-11-03T17:11:43.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.448+0000 7fe6dcff9700 1 -- 172.21.15.27:0/657574672 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6c800da80 con 0x7fe6e00f9700 2021-11-03T17:11:43.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.448+0000 7fe6e5796700 1 -- 172.21.15.27:0/657574672 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6e0108950 con 0x7fe6e00f9700 2021-11-03T17:11:43.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.449+0000 7fe6e5796700 1 -- 172.21.15.27:0/657574672 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6e0045ad0 con 0x7fe6e00f9700 2021-11-03T17:11:43.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.450+0000 7fe6dcff9700 1 -- 172.21.15.27:0/657574672 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fe6c801a060 con 0x7fe6e00f9700 2021-11-03T17:11:43.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.450+0000 7fe6dcff9700 1 --2- 172.21.15.27:0/657574672 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fe6cc05c300 0x7fe6cc05e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:43.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.450+0000 7fe6dcff9700 1 -- 172.21.15.27:0/657574672 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4519+0+0 (secure 0 0 0) 0x7fe6c8075ff0 con 0x7fe6e00f9700 2021-11-03T17:11:43.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.450+0000 7fe6dffff700 1 --2- 172.21.15.27:0/657574672 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fe6cc05c300 0x7fe6cc05e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:43.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.451+0000 7fe6dffff700 1 --2- 172.21.15.27:0/657574672 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fe6cc05c300 0x7fe6cc05e7c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fe6d0004e30 tx=0x7fe6d0009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:43.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.454+0000 7fe6dcff9700 1 -- 172.21.15.27:0/657574672 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fe6c80490a0 con 0x7fe6e00f9700 2021-11-03T17:11:43.637 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.642+0000 7fe6e5796700 1 -- 172.21.15.27:0/657574672 --> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6e0108bd0 con 0x7fe6cc05c300 2021-11-03T17:11:43.642 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.647+0000 7fe6dcff9700 1 -- 172.21.15.27:0/657574672 <== mgr.14162 v2:172.21.15.27:6800/3802716887 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1116 (secure 0 0 0) 0x7fe6e0108bd0 con 0x7fe6cc05c300 2021-11-03T17:11:43.642 INFO:teuthology.orchestra.run.smithi027.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2021-11-03T17:11:43.642 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager 1/1 92s ago 5m count:1 2021-11-03T17:11:43.643 INFO:teuthology.orchestra.run.smithi027.stdout:crash 2/2 92s ago 5m * 2021-11-03T17:11:43.643 INFO:teuthology.orchestra.run.smithi027.stdout:grafana 1/1 92s ago 5m count:1 2021-11-03T17:11:43.643 INFO:teuthology.orchestra.run.smithi027.stdout:mgr 2/2 92s ago 5m count:2 2021-11-03T17:11:43.644 INFO:teuthology.orchestra.run.smithi027.stdout:mon 2/2 92s ago 3m smithi027:172.21.15.27=smithi027;smithi115:172.21.15.115=smithi115;count:2 2021-11-03T17:11:43.644 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter 2/2 92s ago 5m * 2021-11-03T17:11:43.644 INFO:teuthology.orchestra.run.smithi027.stdout:osd.unmanaged 6/6 92s ago - 2021-11-03T17:11:43.644 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus 1/1 92s ago 5m count:1 2021-11-03T17:11:43.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.649+0000 7fe6e5796700 1 -- 172.21.15.27:0/657574672 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fe6cc05c300 msgr2=0x7fe6cc05e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:43.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.649+0000 7fe6e5796700 1 --2- 172.21.15.27:0/657574672 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fe6cc05c300 0x7fe6cc05e7c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fe6d0004e30 tx=0x7fe6d0009b30).stop 2021-11-03T17:11:43.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.649+0000 7fe6e5796700 1 -- 172.21.15.27:0/657574672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6e00f9700 msgr2=0x7fe6e0107300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:43.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.649+0000 7fe6e5796700 1 --2- 172.21.15.27:0/657574672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6e00f9700 0x7fe6e0107300 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7fe6c80093f0 tx=0x7fe6c8006ec0).stop 2021-11-03T17:11:43.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.649+0000 7fe6e5796700 1 -- 172.21.15.27:0/657574672 shutdown_connections 2021-11-03T17:11:43.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.649+0000 7fe6e5796700 1 --2- 172.21.15.27:0/657574672 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fe6cc05c300 0x7fe6cc05e7c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:43.648 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.649+0000 7fe6e5796700 1 --2- 172.21.15.27:0/657574672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6e00602b0 0x7fe6e0106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:43.648 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.649+0000 7fe6e5796700 1 --2- 172.21.15.27:0/657574672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6e00f9700 0x7fe6e0107300 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:43.648 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.649+0000 7fe6e5796700 1 -- 172.21.15.27:0/657574672 >> 172.21.15.27:0/657574672 conn(0x7fe6e00f0ba0 msgr2=0x7fe6e00f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:43.648 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.649+0000 7fe6e5796700 1 -- 172.21.15.27:0/657574672 shutdown_connections 2021-11-03T17:11:43.649 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:43.649+0000 7fe6e5796700 1 -- 172.21.15.27:0/657574672 wait complete. 2021-11-03T17:11:43.955 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:43 smithi115 conmon[34670]: cluster 2021-11-03T17:11:42.570937+0000 mgr.smithi027.ufhhvb (mgr.14162 2021-11-03T17:11:43.956 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:43 smithi115 conmon[34670]: ) 206 : cluster [DBG] pgmap v173: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:44.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:43 smithi027 conmon[32322]: cluster 2021-11-03T17:11:42. 2021-11-03T17:11:44.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:43 smithi027 conmon[32322]: 570937+0000 mgr.smithi027.ufhhvb (mgr.14162) 206 : cluster [DBG] pgmap v173: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:44.247 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- bash -c 'ceph orch host ls' 2021-11-03T17:11:45.965 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:45 smithi027 conmon[32322]: audit 2021-11-03T17:11:43.643604+0000 2021-11-03T17:11:45.966 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:45 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14162) 207 : audit [DBG] from='client.14408 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:11:45.966 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:45 smithi027 conmon[32322]: cluster 2021-11-03T17:11:44.571241+0000 mgr.smithi027.ufhhvb (mgr.14162) 208 : cluster [DBG] pgmap v174: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:45.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:45 smithi115 conmon[34670]: audit 2021-11-03T17:11:43.643604+0000 mgr.smithi027.ufhhvb (mgr.14162) 207 : audit [DBG] 2021-11-03T17:11:45.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:45 smithi115 conmon[34670]: from='client.14408 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:11:45.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:45 smithi115 conmon[34670]: cluster 2021-11-03T17:11:44.571241+0000 mgr.smithi027.ufhhvb (mgr.14162) 208 : cluster [DBG] pgmap v174: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:46.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.154+0000 7f40c1952700 1 -- 172.21.15.27:0/3541497596 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40bc0f55e0 msgr2=0x7f40bc0f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:46.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.154+0000 7f40c1952700 1 --2- 172.21.15.27:0/3541497596 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40bc0f55e0 0x7f40bc0f5a00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f40b40099e0 tx=0x7f40b4004d10).stop 2021-11-03T17:11:46.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.154+0000 7f40c1952700 1 -- 172.21.15.27:0/3541497596 shutdown_connections 2021-11-03T17:11:46.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.154+0000 7f40c1952700 1 --2- 172.21.15.27:0/3541497596 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40bc0f5ef0 0x7f40bc0f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:11:46.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.154+0000 7f40c1952700 1 --2- 172.21.15.27:0/3541497596 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40bc0f55e0 0x7f40bc0f5a00 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:46.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.154+0000 7f40c1952700 1 -- 172.21.15.27:0/3541497596 >> 172.21.15.27:0/3541497596 conn(0x7f40bc0f0de0 msgr2=0x7f40bc0f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:46.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.154+0000 7f40c1952700 1 -- 172.21.15.27:0/3541497596 shutdown_connections 2021-11-03T17:11:46.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.154+0000 7f40c1952700 1 -- 172.21.15.27:0/3541497596 wait complete. 2021-11-03T17:11:46.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.155+0000 7f40c1952700 1 Processor -- start 2021-11-03T17:11:46.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.155+0000 7f40c1952700 1 -- start start 2021-11-03T17:11:46.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.155+0000 7f40c1952700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40bc0f55e0 0x7f40bc1092e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:46.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.155+0000 7f40c1952700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40bc0f5ef0 0x7f40bc1097d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:46.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.155+0000 7f40c1952700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc109e10 con 0x7f40bc0f5ef0 2021-11-03T17:11:46.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.155+0000 7f40c1952700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc10a6c0 con 0x7f40bc0f55e0 2021-11-03T17:11:46.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.155+0000 7f40ba7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40bc0f5ef0 0x7f40bc1097d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:46.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.155+0000 7f40ba7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40bc0f5ef0 0x7f40bc1097d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54928/0 (socket says 172.21.15.27:54928) 2021-11-03T17:11:46.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.155+0000 7f40ba7fc700 1 -- 172.21.15.27:0/2219795955 learned_addr learned my addr 172.21.15.27:0/2219795955 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:11:46.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.155+0000 7f40baffd700 1 --2- 172.21.15.27:0/2219795955 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40bc0f55e0 0x7f40bc1092e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:46.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.155+0000 7f40ba7fc700 1 -- 172.21.15.27:0/2219795955 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40bc0f55e0 msgr2=0x7f40bc1092e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:46.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.155+0000 7f40ba7fc700 1 --2- 172.21.15.27:0/2219795955 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40bc0f55e0 0x7f40bc1092e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:46.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.155+0000 7f40ba7fc700 1 -- 172.21.15.27:0/2219795955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b4009640 con 0x7f40bc0f5ef0 2021-11-03T17:11:46.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.156+0000 7f40ba7fc700 1 --2- 172.21.15.27:0/2219795955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40bc0f5ef0 0x7f40bc1097d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f40a400b410 tx=0x7f40a4006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:46.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.156+0000 7f40b37fe700 1 -- 172.21.15.27:0/2219795955 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40a40125f0 con 0x7f40bc0f5ef0 2021-11-03T17:11:46.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.156+0000 7f40b37fe700 1 -- 172.21.15.27:0/2219795955 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f40a4012be0 con 0x7f40bc0f5ef0 2021-11-03T17:11:46.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.156+0000 7f40c1952700 1 -- 172.21.15.27:0/2219795955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40bc10a9a0 con 0x7f40bc0f5ef0 2021-11-03T17:11:46.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.156+0000 7f40b37fe700 1 -- 172.21.15.27:0/2219795955 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f40a4002a00 con 0x7f40bc0f5ef0 2021-11-03T17:11:46.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.156+0000 7f40c1952700 1 -- 172.21.15.27:0/2219795955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40bc10aee0 con 0x7f40bc0f5ef0 2021-11-03T17:11:46.160 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.157+0000 7f40c1952700 1 -- 172.21.15.27:0/2219795955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40bc0f4a70 con 0x7f40bc0f5ef0 2021-11-03T17:11:46.161 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.157+0000 7f40b37fe700 1 -- 172.21.15.27:0/2219795955 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f40a400f030 con 0x7f40bc0f5ef0 2021-11-03T17:11:46.161 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.157+0000 7f40b37fe700 1 --2- 172.21.15.27:0/2219795955 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f40a805c2a0 0x7f40a805e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:46.161 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.157+0000 7f40b37fe700 1 -- 172.21.15.27:0/2219795955 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4519+0+0 (secure 0 0 0) 0x7f40a40760d0 con 0x7f40bc0f5ef0 2021-11-03T17:11:46.162 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.157+0000 7f40baffd700 1 --2- 172.21.15.27:0/2219795955 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f40a805c2a0 0x7f40a805e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:46.162 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.158+0000 7f40baffd700 1 --2- 172.21.15.27:0/2219795955 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f40a805c2a0 0x7f40a805e760 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f40b4000f80 tx=0x7f40b4004d10).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:46.162 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.161+0000 7f40b37fe700 1 -- 172.21.15.27:0/2219795955 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f40a4049090 con 0x7f40bc0f5ef0 2021-11-03T17:11:46.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.344+0000 7f40c1952700 1 -- 172.21.15.27:0/2219795955 --> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f40bc0fab60 con 0x7f40a805c2a0 2021-11-03T17:11:46.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.346+0000 7f40b37fe700 1 -- 172.21.15.27:0/2219795955 <== mgr.14162 v2:172.21.15.27:6800/3802716887 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+126 (secure 0 0 0) 0x7f40bc0fab60 con 0x7f40a805c2a0 2021-11-03T17:11:46.342 INFO:teuthology.orchestra.run.smithi027.stdout:HOST ADDR LABELS STATUS 2021-11-03T17:11:46.342 INFO:teuthology.orchestra.run.smithi027.stdout:smithi027 172.21.15.27 2021-11-03T17:11:46.342 INFO:teuthology.orchestra.run.smithi027.stdout:smithi115 smithi115 2021-11-03T17:11:46.343 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.347+0000 7f40c1952700 1 -- 172.21.15.27:0/2219795955 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f40a805c2a0 msgr2=0x7f40a805e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:46.344 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.347+0000 7f40c1952700 1 --2- 172.21.15.27:0/2219795955 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f40a805c2a0 0x7f40a805e760 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f40b4000f80 tx=0x7f40b4004d10).stop 2021-11-03T17:11:46.344 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.347+0000 7f40c1952700 1 -- 172.21.15.27:0/2219795955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40bc0f5ef0 msgr2=0x7f40bc1097d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:46.344 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.347+0000 7f40c1952700 1 --2- 172.21.15.27:0/2219795955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40bc0f5ef0 0x7f40bc1097d0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f40a400b410 tx=0x7f40a4006ef0).stop 2021-11-03T17:11:46.344 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.347+0000 7f40c1952700 1 -- 172.21.15.27:0/2219795955 shutdown_connections 2021-11-03T17:11:46.345 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.347+0000 7f40c1952700 1 --2- 172.21.15.27:0/2219795955 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f40a805c2a0 0x7f40a805e760 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:46.345 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.347+0000 7f40c1952700 1 --2- 172.21.15.27:0/2219795955 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40bc0f55e0 0x7f40bc1092e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:46.345 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.347+0000 7f40c1952700 1 --2- 172.21.15.27:0/2219795955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40bc0f5ef0 0x7f40bc1097d0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:46.345 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.347+0000 7f40c1952700 1 -- 172.21.15.27:0/2219795955 >> 172.21.15.27:0/2219795955 conn(0x7f40bc0f0de0 msgr2=0x7f40bc05ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:46.345 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.347+0000 7f40c1952700 1 -- 172.21.15.27:0/2219795955 shutdown_connections 2021-11-03T17:11:46.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:46.347+0000 7f40c1952700 1 -- 172.21.15.27:0/2219795955 wait complete. 2021-11-03T17:11:46.921 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- bash -c 'ceph orch device ls' 2021-11-03T17:11:47.737 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:47 smithi027 conmon[32322]: audit 2021-11-03T17:11:46.346778+0000 mgr.smithi027.ufhhvb (mgr.14162) 209 : audit [DBG] from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:11:47.738 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:47 smithi027 conmon[32322]: cluster 2021-11-03T17:11:46.571763+0000 mgr.smithi027.ufhhvb (mgr.14162) 210 : cluster [DBG] pgmap v175: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:47.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:47 smithi115 conmon[34670]: audit 2021-11-03T17:11:46.346778+0000 mgr.smithi027.ufhhvb (mgr.14162) 209 : audit 2021-11-03T17:11:47.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:47 smithi115 conmon[34670]: [DBG] from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:11:47.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:47 smithi115 conmon[34670]: cluster 2021-11-03T17:11:46.571763+0000 mgr.smithi027.ufhhvb (mgr.14162) 210 : cluster [DBG] pgmap v175: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:48.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.787+0000 7f7efa518700 1 -- 172.21.15.27:0/3387679551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ef40f6ff0 msgr2=0x7f7ef40f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:48.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.787+0000 7f7efa518700 1 --2- 172.21.15.27:0/3387679551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ef40f6ff0 0x7f7ef40f7a20 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f7ee4004660 tx=0x7f7ee4009b30).stop 2021-11-03T17:11:48.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.787+0000 7f7efa518700 1 -- 172.21.15.27:0/3387679551 shutdown_connections 2021-11-03T17:11:48.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.787+0000 7f7efa518700 1 --2- 172.21.15.27:0/3387679551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ef40f6ff0 0x7f7ef40f7a20 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:48.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.787+0000 7f7efa518700 1 --2- 172.21.15.27:0/3387679551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ef40f66e0 0x7f7ef40f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:48.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.787+0000 7f7efa518700 1 -- 172.21.15.27:0/3387679551 >> 172.21.15.27:0/3387679551 conn(0x7f7ef40f1f00 msgr2=0x7f7ef40f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:48.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.787+0000 7f7efa518700 1 -- 172.21.15.27:0/3387679551 shutdown_connections 2021-11-03T17:11:48.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.787+0000 7f7efa518700 1 -- 172.21.15.27:0/3387679551 wait complete. 2021-11-03T17:11:48.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.788+0000 7f7efa518700 1 Processor -- start 2021-11-03T17:11:48.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.788+0000 7f7efa518700 1 -- start start 2021-11-03T17:11:48.788 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.788+0000 7f7efa518700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ef40f66e0 0x7f7ef406b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:48.789 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.788+0000 7f7efa518700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ef40f6ff0 0x7f7ef4069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:48.789 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.788+0000 7f7efa518700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef406a380 con 0x7f7ef40f66e0 2021-11-03T17:11:48.789 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.788+0000 7f7efa518700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef406a4c0 con 0x7f7ef40f6ff0 2021-11-03T17:11:48.789 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.788+0000 7f7ef3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ef40f66e0 0x7f7ef406b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:48.790 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.788+0000 7f7ef3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ef40f66e0 0x7f7ef406b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54936/0 (socket says 172.21.15.27:54936) 2021-11-03T17:11:48.790 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.788+0000 7f7ef3fff700 1 -- 172.21.15.27:0/725083343 learned_addr learned my addr 172.21.15.27:0/725083343 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:11:48.790 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.788+0000 7f7ef37fe700 1 --2- 172.21.15.27:0/725083343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ef40f6ff0 0x7f7ef4069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:48.790 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.789+0000 7f7ef3fff700 1 -- 172.21.15.27:0/725083343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ef40f6ff0 msgr2=0x7f7ef4069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:48.791 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.789+0000 7f7ef3fff700 1 --2- 172.21.15.27:0/725083343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ef40f6ff0 0x7f7ef4069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:48.791 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.789+0000 7f7ef3fff700 1 -- 172.21.15.27:0/725083343 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ee4005040 con 0x7f7ef40f66e0 2021-11-03T17:11:48.791 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.789+0000 7f7ef3fff700 1 --2- 172.21.15.27:0/725083343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ef40f66e0 0x7f7ef406b9a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f7ee800e7f0 tx=0x7f7ee800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:48.791 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.789+0000 7f7ef0ff9700 1 -- 172.21.15.27:0/725083343 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ee80092e0 con 0x7f7ef40f66e0 2021-11-03T17:11:48.792 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.789+0000 7f7efa518700 1 -- 172.21.15.27:0/725083343 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ef406a7a0 con 0x7f7ef40f66e0 2021-11-03T17:11:48.792 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.789+0000 7f7ef0ff9700 1 -- 172.21.15.27:0/725083343 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f7ee800f040 con 0x7f7ef40f66e0 2021-11-03T17:11:48.792 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.789+0000 7f7ef0ff9700 1 -- 172.21.15.27:0/725083343 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ee80146c0 con 0x7f7ef40f66e0 2021-11-03T17:11:48.792 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.789+0000 7f7efa518700 1 -- 172.21.15.27:0/725083343 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ef410ef20 con 0x7f7ef40f66e0 2021-11-03T17:11:48.793 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.790+0000 7f7efa518700 1 -- 172.21.15.27:0/725083343 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ef4045ad0 con 0x7f7ef40f66e0 2021-11-03T17:11:48.793 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.790+0000 7f7ef0ff9700 1 -- 172.21.15.27:0/725083343 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f7ee801b030 con 0x7f7ef40f66e0 2021-11-03T17:11:48.793 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.791+0000 7f7ef0ff9700 1 --2- 172.21.15.27:0/725083343 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f7edc05c2b0 0x7f7edc05e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:48.793 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.791+0000 7f7ef0ff9700 1 -- 172.21.15.27:0/725083343 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4519+0+0 (secure 0 0 0) 0x7f7ee80758d0 con 0x7f7ef40f66e0 2021-11-03T17:11:48.794 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.791+0000 7f7ef37fe700 1 --2- 172.21.15.27:0/725083343 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f7edc05c2b0 0x7f7edc05e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:48.794 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.791+0000 7f7ef37fe700 1 --2- 172.21.15.27:0/725083343 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f7edc05c2b0 0x7f7edc05e770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f7ee4005010 tx=0x7f7ee4016040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:48.795 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.795+0000 7f7ef0ff9700 1 -- 172.21.15.27:0/725083343 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f7ee80489a0 con 0x7f7ef40f66e0 2021-11-03T17:11:48.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.980+0000 7f7efa518700 1 -- 172.21.15.27:0/725083343 --> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ef40ff050 con 0x7f7edc05c2b0 2021-11-03T17:11:48.978 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.982+0000 7f7ef0ff9700 1 -- 172.21.15.27:0/725083343 <== mgr.14162 v2:172.21.15.27:6800/3802716887 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+279 (secure 0 0 0) 0x7f7ef40ff050 con 0x7f7edc05c2b0 2021-11-03T17:11:48.978 INFO:teuthology.orchestra.run.smithi027.stdout:Hostname Path Type Serial Size Health Ident Fault Available 2021-11-03T17:11:48.978 INFO:teuthology.orchestra.run.smithi027.stdout:smithi027 /dev/nvme0n1 ssd CVFT5331001C400BGN 400G Unknown N/A N/A No 2021-11-03T17:11:48.979 INFO:teuthology.orchestra.run.smithi027.stdout:smithi115 /dev/nvme0n1 ssd PHFT620400UU400BGN 400G Unknown N/A N/A No 2021-11-03T17:11:48.980 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.984+0000 7f7efa518700 1 -- 172.21.15.27:0/725083343 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f7edc05c2b0 msgr2=0x7f7edc05e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:48.980 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.984+0000 7f7efa518700 1 --2- 172.21.15.27:0/725083343 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f7edc05c2b0 0x7f7edc05e770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f7ee4005010 tx=0x7f7ee4016040).stop 2021-11-03T17:11:48.980 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.984+0000 7f7efa518700 1 -- 172.21.15.27:0/725083343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ef40f66e0 msgr2=0x7f7ef406b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:48.980 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.984+0000 7f7efa518700 1 --2- 172.21.15.27:0/725083343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ef40f66e0 0x7f7ef406b9a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f7ee800e7f0 tx=0x7f7ee800b040).stop 2021-11-03T17:11:48.981 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.984+0000 7f7efa518700 1 -- 172.21.15.27:0/725083343 shutdown_connections 2021-11-03T17:11:48.981 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.984+0000 7f7efa518700 1 --2- 172.21.15.27:0/725083343 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f7edc05c2b0 0x7f7edc05e770 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:48.981 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.984+0000 7f7efa518700 1 --2- 172.21.15.27:0/725083343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ef40f66e0 0x7f7ef406b9a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:48.981 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.984+0000 7f7efa518700 1 --2- 172.21.15.27:0/725083343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ef40f6ff0 0x7f7ef4069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:48.982 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.984+0000 7f7efa518700 1 -- 172.21.15.27:0/725083343 >> 172.21.15.27:0/725083343 conn(0x7f7ef40f1f00 msgr2=0x7f7ef40fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:48.982 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.984+0000 7f7efa518700 1 -- 172.21.15.27:0/725083343 shutdown_connections 2021-11-03T17:11:48.982 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:48.984+0000 7f7efa518700 1 -- 172.21.15.27:0/725083343 wait complete. 2021-11-03T17:11:49.521 INFO:teuthology.run_tasks:Running task cephadm.shell... 2021-11-03T17:11:49.532 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi027.front.sepia.ceph.com 2021-11-03T17:11:49.533 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- bash -c 'ceph fs volume create cephfs --placement=4' 2021-11-03T17:11:49.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:49 smithi027 conmon[32322]: cluster 2021-11-03T17:11:48.572122+0000 mgr.smithi027.ufhhvb (mgr.14162) 211 : cluster [DBG] pgmap v176: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:49.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:49 smithi115 conmon[34670]: cluster 2021-11-03T17:11:48.572122+0000 mgr.smithi027.ufhhvb (mgr.14162) 211 : cluster [DBG] pgmap v176: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:50.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:50 smithi027 conmon[32322]: audit 2021-11-03T17:11:48.982949+0000 mgr.smithi027.ufhhvb (mgr 2021-11-03T17:11:50.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:50 smithi027 conmon[32322]: .14162) 212 : audit [DBG] from='client.14416 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:11:50.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:50 smithi115 conmon[34670]: audit 2021-11-03T17:11:48.982949+0000 mgr.smithi027.ufhhvb (mgr.14162) 212 : audit 2021-11-03T17:11:50.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:50 smithi115 conmon[34670]: [DBG] from='client.14416 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:11:51.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:51.373+0000 7f1a42ec5700 1 -- 172.21.15.27:0/211508750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a3c0f77d0 msgr2=0x7f1a3c0f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:51.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:51.373+0000 7f1a42ec5700 1 --2- 172.21.15.27:0/211508750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a3c0f77d0 0x7f1a3c0f7c30 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f1a2c004660 tx=0x7f1a2c009b30).stop 2021-11-03T17:11:51.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:51.373+0000 7f1a42ec5700 1 -- 172.21.15.27:0/211508750 shutdown_connections 2021-11-03T17:11:51.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:51.373+0000 7f1a42ec5700 1 --2- 172.21.15.27:0/211508750 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a3c0f77d0 0x7f1a3c0f7c30 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:51.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:51.373+0000 7f1a42ec5700 1 --2- 172.21.15.27:0/211508750 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a3c0f6530 0x7f1a3c0f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:51.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:51.373+0000 7f1a42ec5700 1 -- 172.21.15.27:0/211508750 >> 172.21.15.27:0/211508750 conn(0x7f1a3c0f1cd0 msgr2=0x7f1a3c0f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:51.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:51.374+0000 7f1a42ec5700 1 -- 172.21.15.27:0/211508750 shutdown_connections 2021-11-03T17:11:51.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:51.374+0000 7f1a42ec5700 1 -- 172.21.15.27:0/211508750 wait complete. 2021-11-03T17:11:51.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:51.374+0000 7f1a42ec5700 1 Processor -- start 2021-11-03T17:11:51.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:51.374+0000 7f1a42ec5700 1 -- start start 2021-11-03T17:11:51.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:51.374+0000 7f1a42ec5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a3c0f6530 0x7f1a3c06b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:51.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:51.374+0000 7f1a42ec5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a3c0f77d0 0x7f1a3c069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:51.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:51.374+0000 7f1a42ec5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c06a380 con 0x7f1a3c0f6530 2021-11-03T17:11:51.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:51.374+0000 7f1a42ec5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c06a4c0 con 0x7f1a3c0f77d0 2021-11-03T17:11:51.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:51.374+0000 7f1a41ec3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a3c0f6530 0x7f1a3c06b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:51.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:51.374+0000 7f1a41ec3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a3c0f6530 0x7f1a3c06b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54946/0 (socket says 172.21.15.27:54946) 2021-11-03T17:11:51.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:51.374+0000 7f1a41ec3700 1 -- 172.21.15.27:0/492627871 learned_addr learned my addr 172.21.15.27:0/492627871 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:11:51.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:51.374+0000 7f1a416c2700 1 --2- 172.21.15.27:0/492627871 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a3c0f77d0 0x7f1a3c069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:51.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:51.375+0000 7f1a41ec3700 1 -- 172.21.15.27:0/492627871 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a3c0f77d0 msgr2=0x7f1a3c069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:51.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:51.375+0000 7f1a41ec3700 1 --2- 172.21.15.27:0/492627871 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a3c0f77d0 0x7f1a3c069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:51.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:51.375+0000 7f1a41ec3700 1 -- 172.21.15.27:0/492627871 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a2c005040 con 0x7f1a3c0f6530 2021-11-03T17:11:51.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:51.375+0000 7f1a41ec3700 1 --2- 172.21.15.27:0/492627871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a3c0f6530 0x7f1a3c06b9a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f1a38002700 tx=0x7f1a38009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:51.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:51.375+0000 7f1a327fc700 1 -- 172.21.15.27:0/492627871 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a380079c0 con 0x7f1a3c0f6530 2021-11-03T17:11:51.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:51.375+0000 7f1a327fc700 1 -- 172.21.15.27:0/492627871 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f1a3800f040 con 0x7f1a3c0f6530 2021-11-03T17:11:51.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:51.375+0000 7f1a42ec5700 1 -- 172.21.15.27:0/492627871 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a3c06a7a0 con 0x7f1a3c0f6530 2021-11-03T17:11:51.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:51.375+0000 7f1a327fc700 1 -- 172.21.15.27:0/492627871 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a38014860 con 0x7f1a3c0f6530 2021-11-03T17:11:51.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:51.375+0000 7f1a42ec5700 1 -- 172.21.15.27:0/492627871 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a3c10ed40 con 0x7f1a3c0f6530 2021-11-03T17:11:51.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:51.376+0000 7f1a42ec5700 1 -- 172.21.15.27:0/492627871 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a3c045ad0 con 0x7f1a3c0f6530 2021-11-03T17:11:51.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:51.376+0000 7f1a327fc700 1 -- 172.21.15.27:0/492627871 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f1a3801b060 con 0x7f1a3c0f6530 2021-11-03T17:11:51.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:51.377+0000 7f1a327fc700 1 --2- 172.21.15.27:0/492627871 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f1a2805c300 0x7f1a2805e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:51.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:51.377+0000 7f1a327fc700 1 -- 172.21.15.27:0/492627871 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4519+0+0 (secure 0 0 0) 0x7f1a380759e0 con 0x7f1a3c0f6530 2021-11-03T17:11:51.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:51.377+0000 7f1a416c2700 1 --2- 172.21.15.27:0/492627871 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f1a2805c300 0x7f1a2805e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:51.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:51.377+0000 7f1a416c2700 1 --2- 172.21.15.27:0/492627871 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f1a2805c300 0x7f1a2805e7c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f1a2c005010 tx=0x7f1a2c009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:51.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:51.380+0000 7f1a327fc700 1 -- 172.21.15.27:0/492627871 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f1a38048a00 con 0x7f1a3c0f6530 2021-11-03T17:11:51.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:51.567+0000 7f1a42ec5700 1 -- 172.21.15.27:0/492627871 --> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a3c0fee30 con 0x7f1a2805c300 2021-11-03T17:11:52.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:52 smithi027 conmon[32322]: cluster 2021-11-03T17:11:50.572403+0000 mgr.smithi027.ufhhvb (mgr.14162) 213 : cluster [DBG] pgmap v177: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:52.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:52 smithi115 conmon[34670]: cluster 2021-11-03T17:11:50.572403+0000 mgr.smithi027.ufhhvb (mgr.14162) 213 : cluster [DBG] pgmap v177: 1 pgs: 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:53.094 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:53.098+0000 7f1a327fc700 1 -- 172.21.15.27:0/492627871 <== mgr.14162 v2:172.21.15.27:6800/3802716887 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1a3c0fee30 con 0x7f1a2805c300 2021-11-03T17:11:53.096 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:53.100+0000 7f1a42ec5700 1 -- 172.21.15.27:0/492627871 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f1a2805c300 msgr2=0x7f1a2805e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:53.096 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:53.100+0000 7f1a42ec5700 1 --2- 172.21.15.27:0/492627871 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f1a2805c300 0x7f1a2805e7c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f1a2c005010 tx=0x7f1a2c009b30).stop 2021-11-03T17:11:53.096 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:53.100+0000 7f1a42ec5700 1 -- 172.21.15.27:0/492627871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a3c0f6530 msgr2=0x7f1a3c06b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:53.097 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:53.100+0000 7f1a42ec5700 1 --2- 172.21.15.27:0/492627871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a3c0f6530 0x7f1a3c06b9a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f1a38002700 tx=0x7f1a38009d80).stop 2021-11-03T17:11:53.097 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:53.100+0000 7f1a42ec5700 1 -- 172.21.15.27:0/492627871 shutdown_connections 2021-11-03T17:11:53.097 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:53.100+0000 7f1a42ec5700 1 --2- 172.21.15.27:0/492627871 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f1a2805c300 0x7f1a2805e7c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:53.097 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:53.100+0000 7f1a42ec5700 1 --2- 172.21.15.27:0/492627871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a3c0f6530 0x7f1a3c06b9a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:53.098 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:53.100+0000 7f1a42ec5700 1 --2- 172.21.15.27:0/492627871 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a3c0f77d0 0x7f1a3c069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:53.098 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:53.100+0000 7f1a42ec5700 1 -- 172.21.15.27:0/492627871 >> 172.21.15.27:0/492627871 conn(0x7f1a3c0f1cd0 msgr2=0x7f1a3c0fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:53.098 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:53.100+0000 7f1a42ec5700 1 -- 172.21.15.27:0/492627871 shutdown_connections 2021-11-03T17:11:53.098 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:53.100+0000 7f1a42ec5700 1 -- 172.21.15.27:0/492627871 wait complete. 2021-11-03T17:11:53.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:53 smithi027 conmon[32322]: audit 2021-11-03T17:11:51.569733+0000 2021-11-03T17:11:53.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:53 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14162) 214 : audit [DBG] from='client.14420 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:11:53.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:53 smithi027 conmon[32322]: audit 2021-11-03T17: 2021-11-03T17:11:53.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:53 smithi027 conmon[32322]: 11:51.570241+0000 mon.smithi027 (mon.0) 458 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2021-11-03T17:11:53.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:53 smithi027 conmon[32322]: audit 2021-11-03T 2021-11-03T17:11:53.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:53 smithi027 conmon[32322]: 17:11:52.062415+0000 mon.smithi027 (mon 2021-11-03T17:11:53.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:53 smithi027 conmon[32322]: .0) 459 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2021-11-03T17:11:53.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:53 smithi027 conmon[32322]: cluster 2021-11-03 2021-11-03T17:11:53.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:53 smithi027 conmon[32322]: T17:11:52.062458+0000 mon.smithi027 2021-11-03T17:11:53.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:53 smithi027 conmon[32322]: (mon.0) 460 : cluster [DBG] osdmap e40: 6 total, 6 up, 6 in 2021-11-03T17:11:53.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:53 smithi027 conmon[32322]: 2021-11-03T17:11:53.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:53 smithi027 conmon[32322]: audit 2021- 2021-11-03T17:11:53.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:53 smithi027 conmon[32322]: 11-03T17:11:52 2021-11-03T17:11:53.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:53 smithi027 conmon[32322]: .063635+0000 mon.smithi027 (mon.0) 461 2021-11-03T17:11:53.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:53 smithi027 conmon[32322]: : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2021-11-03T17:11:53.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:53 smithi027 conmon[32322]: cluster 2021-11-03T17: 2021-11-03T17:11:53.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:53 smithi027 conmon[32322]: 11:52.572737+0000 mgr.smithi027.ufhhvb (mgr 2021-11-03T17:11:53.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:53 smithi027 conmon[32322]: .14162) 215 : cluster [DBG] pgmap v179: 33 pgs: 32 unknown, 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:53.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:53 smithi027 conmon[32322]: audit 2021- 2021-11-03T17:11:53.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:53 smithi027 conmon[32322]: 11-03T17:11:52. 2021-11-03T17:11:53.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:53 smithi027 conmon[32322]: 725713+0000 mon.smithi027 (mon.0) 462 : 2021-11-03T17:11:53.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:53 smithi027 conmon[32322]: audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:11:53.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:53 smithi027 conmon[32322]: audit 2021-11-03T17:11:53.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:53 smithi027 conmon[32322]: 2021-11 2021-11-03T17:11:53.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:53 smithi027 conmon[32322]: -03T17:11:52.725913+0000 mon.smithi027 (mon. 2021-11-03T17:11:53.275 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:53 smithi027 conmon[32322]: 0) 463 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:11:53.275 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:53 smithi027 conmon[32322]: debug 2021-11-03T17:11:53.076+0000 7ff1b90bd700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-11-03T17:11:53.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:53 smithi115 conmon[34670]: audit 2021-11-03T17:11:51.569733+0000 mgr.smithi027.ufhhvb (mgr.14162) 214 : audit [DBG] from='client.14420 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:11:53.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:53 smithi115 conmon[34670]: audit 2021-11-03T17:11:51.570241+0000 mon.smithi027 2021-11-03T17:11:53.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:53 smithi115 conmon[34670]: (mon.0) 458 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2021-11-03T17:11:53.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:53 smithi115 conmon[34670]: audit 2021-11-03T17:11:52.062415+0000 mon.smithi027 (mon.0) 459 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2021-11-03T17:11:53.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:53 smithi115 conmon[34670]: cluster 2021-11-03T17:11:52.062458+0000 mon.smithi027 (mon.0) 460 : cluster 2021-11-03T17:11:53.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:53 smithi115 conmon[34670]: [DBG] osdmap e40: 6 total, 6 up, 6 in 2021-11-03T17:11:53.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:53 smithi115 conmon[34670]: audit 2021-11-03T17:11:52 2021-11-03T17:11:53.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:53 smithi115 conmon[34670]: .063635+0000 mon.smithi027 (mon.0) 461 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2021-11-03T17:11:53.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:53 smithi115 conmon[34670]: cluster 2021 2021-11-03T17:11:53.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:53 smithi115 conmon[34670]: -11-03T17:11:52.572737+0000 2021-11-03T17:11:53.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:53 smithi115 conmon[34670]: mgr.smithi027.ufhhvb (mgr.14162) 215 : cluster [DBG] pgmap v179: 33 pgs: 32 unknown, 1 active+clean; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:53.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:53 smithi115 conmon[34670]: 2021-11-03T17:11:53.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:53 smithi115 conmon[34670]: audit 2021-11-03T17:11:53.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:53 smithi115 conmon[34670]: 2021-11-03T17:11:52.725713+0000 mon.smithi027 (mon. 2021-11-03T17:11:53.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:53 smithi115 conmon[34670]: 0) 462 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:11:53.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:53 smithi115 conmon[34670]: audit 2021 2021-11-03T17:11:53.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:53 smithi115 conmon[34670]: -11-03T17:11:52.725913 2021-11-03T17:11:53.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:53 smithi115 conmon[34670]: +0000 mon.smithi027 (mon.0) 463 : audit [INF] 2021-11-03T17:11:53.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:53 smithi115 conmon[34670]: from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:11:53.744 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- bash -c 'ceph fs dump' 2021-11-03T17:11:54.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: audit 2021-11-03T17:11:53.072113+0000 mon.smithi027 (mon.0) 464 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2021-11-03T17:11:54.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: cluster 2021-11-03T17 2021-11-03T17:11:54.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: :11:53.072292+0000 mon.smithi027 (mon.0) 465 : cluster [DBG] osdmap e41: 6 total, 6 up, 6 in 2021-11-03T17:11:54.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: audit 2021-11-03T17:11:53.077488+0000 mon.smithi027 (mon.0) 466 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2021-11-03T17:11:54.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: cluster 2021-11-03T17 2021-11-03T17:11:54.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: :11:53.078350+0000 mon.smithi027 ( 2021-11-03T17:11:54.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: mon.0) 467 : cluster [ERR] Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-11-03T17:11:54.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: 2021-11-03T17:11:54.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: cluster 2021-11 2021-11-03T17:11:54.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: -03T17:11:53.078383 2021-11-03T17:11:54.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: +0000 mon.smithi027 (mon.0) 468 : 2021-11-03T17:11:54.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: cluster [WRN] Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2021-11-03T17:11:54.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: cluster 2021 2021-11-03T17:11:54.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: -11-03T17:11:53 2021-11-03T17:11:54.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: .088427+0000 mon.smithi027 (mon.0) 469 2021-11-03T17:11:54.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: : cluster [DBG] osdmap e42: 6 total, 6 up, 6 in 2021-11-03T17:11:54.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: audit 2021-11-03T17:11:54.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: 2021-11-03T17:11:53 2021-11-03T17:11:54.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: .089262+0000 mon.smithi027 (mon.0) 470 : audit [INF] 2021-11-03T17:11:54.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2021-11-03T17:11:54.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: cluster 2021 2021-11-03T17:11:54.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: -11-03T17:11:53 2021-11-03T17:11:54.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: .089355+0000 mon.smithi027 (mon.0 2021-11-03T17:11:54.241 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: ) 471 : cluster [DBG] fsmap cephfs:0 2021-11-03T17:11:54.241 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: cephadm 2021- 2021-11-03T17:11:54.241 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: 11-03T17:11:53. 2021-11-03T17:11:54.241 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: 092208+0000 mgr.smithi027.ufhhvb (mgr.14162) 2021-11-03T17:11:54.242 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: 216 : cephadm [INF] Saving service mds.cephfs spec with placement count:4 2021-11-03T17:11:54.242 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: audit 2021 2021-11-03T17:11:54.242 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: -11-03T17:11:53. 2021-11-03T17:11:54.242 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: 098997+0000 mon.smithi027 (mon.0 2021-11-03T17:11:54.243 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: ) 472 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:11:54.243 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: audit 2021 2021-11-03T17:11:54.243 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: -11-03T17:11:53 2021-11-03T17:11:54.243 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: .100469+0000 mon.smithi027 (mon.0) 2021-11-03T17:11:54.243 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: 473 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:11:54.244 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: audit 2021- 2021-11-03T17:11:54.244 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: 11-03T17:11:53. 2021-11-03T17:11:54.244 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: 111675+0000 mon.smithi027 (mon.0) 2021-11-03T17:11:54.244 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: 474 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:11:54.245 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: audit 2021 2021-11-03T17:11:54.245 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: -11-03T17:11:53 2021-11-03T17:11:54.245 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: .119576+0000 mon.smithi027 (mon.0 2021-11-03T17:11:54.246 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: ) 475 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:11:54.246 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: audit 2021- 2021-11-03T17:11:54.246 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: 11-03T17:11:53. 2021-11-03T17:11:54.246 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: 120367+0000 mon.smithi027 (mon.0) 2021-11-03T17:11:54.247 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: 476 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi115.zsfgnv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2021-11-03T17:11:54.247 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: audit 2021 2021-11-03T17:11:54.247 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: -11-03T17:11:53 2021-11-03T17:11:54.247 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: .123430+0000 mon.smithi027 (mon.0) 2021-11-03T17:11:54.248 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: 477 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi115.zsfgnv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2021-11-03T17:11:54.248 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: audit 2021 2021-11-03T17:11:54.248 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: -11-03T17:11:53 2021-11-03T17:11:54.248 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: .124226+0000 mon.smithi027 (mon.0 2021-11-03T17:11:54.249 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: ) 478 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:11:54.249 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: 2021-11-03T17:11:54.249 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: cephadm 2021 2021-11-03T17:11:54.250 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: -11-03T17:11:53 2021-11-03T17:11:54.250 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: .125196+0000 mgr.smithi027.ufhhvb (mgr. 2021-11-03T17:11:54.250 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: 14162) 217 : cephadm [INF] Deploying daemon mds.cephfs.smithi115.zsfgnv on smithi115 2021-11-03T17:11:54.250 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:54 smithi115 conmon[34670]: 2021-11-03T17:11:54.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:54 smithi027 conmon[32322]: audit 2021-11-03T 2021-11-03T17:11:54.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:54 smithi027 conmon[32322]: 17:11:53.072113+0000 mon.smithi027 (mon.0) 464 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2021-11-03T17:11:54.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:54 smithi027 conmon[32322]: cluster 2021-11-03T17:11:53.072292+0000 mon.smithi027 (mon.0) 465 : cluster [DBG] osdmap e41: 6 total, 6 up, 6 in 2021-11-03T17:11:54.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:54 smithi027 conmon[32322]: audit 2021-11-03T17:11:53.077488+0000 mon.smithi027 (mon.0) 466 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2021-11-03T17:11:54.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:54 smithi027 conmon[32322]: cluster 2021-11-03T17:11:53.078350+0000 mon.smithi027 (mon.0) 467 : cluster [ERR] Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2021-11-03T17:11:54.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:54 smithi027 conmon[32322]: cluster 2021-11-03T17:11:53.078383+0000 mon.smithi027 (mon.0) 468 : cluster [WRN] Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2021-11-03T17:11:54.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:54 smithi027 conmon[32322]: cluster 2021-11-03T17:11:53.088427+0000 mon.smithi027 (mon.0) 469 : cluster [DBG] osdmap e42: 6 total, 6 up, 6 in 2021-11-03T17:11:54.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:54 smithi027 conmon[32322]: audit 2021-11-03T17:11:53.089262+0000 mon.smithi027 (mon.0) 470 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2021-11-03T17:11:54.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:54 smithi027 conmon[32322]: cluster 2021-11-03T17:11:53.089355+0000 mon.smithi027 (mon.0) 471 : cluster [DBG] fsmap cephfs:0 2021-11-03T17:11:54.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:54 smithi027 conmon[32322]: cephadm 2021-11-03T17:11:53.092208+0000 mgr.smithi027.ufhhvb (mgr.14162) 216 : cephadm [INF] Saving service mds.cephfs spec with placement count:4 2021-11-03T17:11:54.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:54 smithi027 conmon[32322]: audit 2021-11-03T17:11:53.098997+0000 mon.smithi027 (mon.0) 472 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:11:54.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:54 smithi027 conmon[32322]: audit 2021-11-03T17:11:53.100469+0000 mon.smithi027 (mon.0) 473 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:11:54.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:54 smithi027 conmon[32322]: audit 2021-11-03T17:11:53.111675+0000 mon.smithi027 (mon.0) 474 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:11:54.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:54 smithi027 conmon[32322]: audit 2021-11-03T17:11:53.119576+0000 mon.smithi027 (mon.0) 475 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:11:54.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:54 smithi027 conmon[32322]: audit 2021-11-03T17:11:53.120367+0000 mon.smithi027 (mon.0) 476 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi115.zsfgnv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2021-11-03T17:11:54.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:54 smithi027 conmon[32322]: audit 2021-11-03T17: 2021-11-03T17:11:54.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:54 smithi027 conmon[32322]: 11:53.123430+0000 mon.smithi027 (mon.0) 477 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi115.zsfgnv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2021-11-03T17:11:54.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:54 smithi027 conmon[32322]: audit 2021-11-03T17:11:53.124226+0000 mon.smithi027 (mon.0) 478 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:11:54.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:54 smithi027 conmon[32322]: cephadm 2021-11 2021-11-03T17:11:54.524 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:54 smithi027 conmon[32322]: -03T17:11:53.125196+0000 2021-11-03T17:11:54.524 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:54 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14162) 217 : cephadm 2021-11-03T17:11:54.524 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:54 smithi027 conmon[32322]: [INF] Deploying daemon mds.cephfs.smithi115.zsfgnv on smithi115 2021-11-03T17:11:55.414 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:55 smithi027 conmon[32322]: cluster 2021-11-03T17:11:54.087250+0000 mon.smithi027 (mon.0 2021-11-03T17:11:55.415 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:55 smithi027 conmon[32322]: ) 479 : cluster [DBG] osdmap e43: 6 total, 6 up, 6 in 2021-11-03T17:11:55.415 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:55 smithi027 conmon[32322]: cluster 2021-11-03T17:11:54.573087+0000 mgr.smithi027.ufhhvb (mgr. 2021-11-03T17:11:55.415 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:55 smithi027 conmon[32322]: 14162) 218 : cluster [DBG] pgmap v183: 65 pgs: 6 creating+peering, 4 active+clean, 55 unknown; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:55.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:55 smithi115 conmon[34670]: cluster 2021-11-03T17:11:54.087250+0000 mon.smithi027 (mon.0) 479 : cluster [DBG] osdmap e43: 6 total, 6 up, 6 in 2021-11-03T17:11:55.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:55 smithi115 conmon[34670]: cluster 2021-11-03T17:11:54.573087+0000 mgr.smithi027.ufhhvb (mgr.14162) 218 : cluster [DBG] pgmap v183: 65 pgs: 6 creating+peering, 4 active+clean, 55 unknown; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:55.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.626+0000 7fc0cdc87700 1 -- 172.21.15.27:0/3023114284 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c80f5ef0 msgr2=0x7fc0c80f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:55.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.626+0000 7fc0cdc87700 1 --2- 172.21.15.27:0/3023114284 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c80f5ef0 0x7fc0c80f6920 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7fc0bc004660 tx=0x7fc0bc009b30).stop 2021-11-03T17:11:55.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.626+0000 7fc0cdc87700 1 -- 172.21.15.27:0/3023114284 shutdown_connections 2021-11-03T17:11:55.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.626+0000 7fc0cdc87700 1 --2- 172.21.15.27:0/3023114284 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c80f5ef0 0x7fc0c80f6920 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:55.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.626+0000 7fc0cdc87700 1 --2- 172.21.15.27:0/3023114284 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0c80f55e0 0x7fc0c80f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:55.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.626+0000 7fc0cdc87700 1 -- 172.21.15.27:0/3023114284 >> 172.21.15.27:0/3023114284 conn(0x7fc0c80f0de0 msgr2=0x7fc0c80f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:55.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.626+0000 7fc0cdc87700 1 -- 172.21.15.27:0/3023114284 shutdown_connections 2021-11-03T17:11:55.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.626+0000 7fc0cdc87700 1 -- 172.21.15.27:0/3023114284 wait complete. 2021-11-03T17:11:55.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.627+0000 7fc0cdc87700 1 Processor -- start 2021-11-03T17:11:55.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.627+0000 7fc0cdc87700 1 -- start start 2021-11-03T17:11:55.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.627+0000 7fc0cdc87700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0c80f55e0 0x7fc0c8105e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:55.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.627+0000 7fc0cdc87700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c80f5ef0 0x7fc0c8106350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:55.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.627+0000 7fc0cdc87700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c8106950 con 0x7fc0c80f5ef0 2021-11-03T17:11:55.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.627+0000 7fc0cdc87700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0c8107160 con 0x7fc0c80f55e0 2021-11-03T17:11:55.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.627+0000 7fc0c6ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c80f5ef0 0x7fc0c8106350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:55.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.627+0000 7fc0c6ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c80f5ef0 0x7fc0c8106350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54958/0 (socket says 172.21.15.27:54958) 2021-11-03T17:11:55.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.627+0000 7fc0c77fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0c80f55e0 0x7fc0c8105e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:55.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.627+0000 7fc0c6ffd700 1 -- 172.21.15.27:0/3279128759 learned_addr learned my addr 172.21.15.27:0/3279128759 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:11:55.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.627+0000 7fc0c6ffd700 1 -- 172.21.15.27:0/3279128759 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0c80f55e0 msgr2=0x7fc0c8105e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:55.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.627+0000 7fc0c6ffd700 1 --2- 172.21.15.27:0/3279128759 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0c80f55e0 0x7fc0c8105e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:55.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.627+0000 7fc0c6ffd700 1 -- 172.21.15.27:0/3279128759 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0bc005040 con 0x7fc0c80f5ef0 2021-11-03T17:11:55.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.627+0000 7fc0c6ffd700 1 --2- 172.21.15.27:0/3279128759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c80f5ef0 0x7fc0c8106350 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7fc0bc000c00 tx=0x7fc0bc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:55.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.628+0000 7fc0affff700 1 -- 172.21.15.27:0/3279128759 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0bc019070 con 0x7fc0c80f5ef0 2021-11-03T17:11:55.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.628+0000 7fc0affff700 1 -- 172.21.15.27:0/3279128759 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7fc0bc007e50 con 0x7fc0c80f5ef0 2021-11-03T17:11:55.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.628+0000 7fc0cdc87700 1 -- 172.21.15.27:0/3279128759 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0c8107380 con 0x7fc0c80f5ef0 2021-11-03T17:11:55.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.628+0000 7fc0affff700 1 -- 172.21.15.27:0/3279128759 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0bc014cd0 con 0x7fc0c80f5ef0 2021-11-03T17:11:55.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.628+0000 7fc0cdc87700 1 -- 172.21.15.27:0/3279128759 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0c81078e0 con 0x7fc0c80f5ef0 2021-11-03T17:11:55.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.629+0000 7fc0cdc87700 1 -- 172.21.15.27:0/3279128759 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0c80f4a70 con 0x7fc0c80f5ef0 2021-11-03T17:11:55.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.629+0000 7fc0affff700 1 -- 172.21.15.27:0/3279128759 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fc0bc025070 con 0x7fc0c80f5ef0 2021-11-03T17:11:55.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.629+0000 7fc0affff700 1 --2- 172.21.15.27:0/3279128759 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fc0b005c2b0 0x7fc0b005e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:55.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.629+0000 7fc0affff700 1 -- 172.21.15.27:0/3279128759 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fc0bc076e80 con 0x7fc0c80f5ef0 2021-11-03T17:11:55.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.629+0000 7fc0c77fe700 1 --2- 172.21.15.27:0/3279128759 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fc0b005c2b0 0x7fc0b005e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:55.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.630+0000 7fc0c77fe700 1 --2- 172.21.15.27:0/3279128759 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fc0b005c2b0 0x7fc0b005e770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fc0c8105bd0 tx=0x7fc0b800b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:55.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.634+0000 7fc0affff700 1 -- 172.21.15.27:0/3279128759 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fc0bc049b60 con 0x7fc0c80f5ef0 2021-11-03T17:11:55.831 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.835+0000 7fc0cdc87700 1 -- 172.21.15.27:0/3279128759 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc0c8045ad0 con 0x7fc0c80f5ef0 2021-11-03T17:11:55.831 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.836+0000 7fc0affff700 1 -- 172.21.15.27:0/3279128759 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1003 (secure 0 0 0) 0x7fc0bc04d180 con 0x7fc0c80f5ef0 2021-11-03T17:11:55.832 INFO:teuthology.orchestra.run.smithi027.stdout:e2 2021-11-03T17:11:55.832 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:11:55.832 INFO:teuthology.orchestra.run.smithi027.stdout:compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:11:55.833 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:11:55.833 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:11:55.833 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:11:55.834 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:11:55.834 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 2 2021-11-03T17:11:55.834 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 2021-11-03T17:11:55.834 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:11:55.835 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:11:53.078248+0000 2021-11-03T17:11:55.835 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:11:55.835 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:11:55.835 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:11:55.836 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:11:55.836 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:11:55.836 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:11:55.836 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:11:55.836 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 0 2021-11-03T17:11:55.837 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:11:55.837 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:11:55.837 INFO:teuthology.orchestra.run.smithi027.stdout:in 2021-11-03T17:11:55.837 INFO:teuthology.orchestra.run.smithi027.stdout:up {} 2021-11-03T17:11:55.838 INFO:teuthology.orchestra.run.smithi027.stdout:failed 2021-11-03T17:11:55.838 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:11:55.838 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:11:55.838 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:11:55.838 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:11:55.839 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:11:55.839 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:11:55.839 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 0 2021-11-03T17:11:55.839 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:11:55.840 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:11:55.840 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.837+0000 7fc0cdc87700 1 -- 172.21.15.27:0/3279128759 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fc0b005c2b0 msgr2=0x7fc0b005e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:55.841 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.837+0000 7fc0cdc87700 1 --2- 172.21.15.27:0/3279128759 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fc0b005c2b0 0x7fc0b005e770 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7fc0c8105bd0 tx=0x7fc0b800b040).stop 2021-11-03T17:11:55.841 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.837+0000 7fc0cdc87700 1 -- 172.21.15.27:0/3279128759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c80f5ef0 msgr2=0x7fc0c8106350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:55.841 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.837+0000 7fc0cdc87700 1 --2- 172.21.15.27:0/3279128759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c80f5ef0 0x7fc0c8106350 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7fc0bc000c00 tx=0x7fc0bc00b040).stop 2021-11-03T17:11:55.841 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.837+0000 7fc0cdc87700 1 -- 172.21.15.27:0/3279128759 shutdown_connections 2021-11-03T17:11:55.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.837+0000 7fc0cdc87700 1 --2- 172.21.15.27:0/3279128759 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fc0b005c2b0 0x7fc0b005e770 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:55.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.837+0000 7fc0cdc87700 1 --2- 172.21.15.27:0/3279128759 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0c80f55e0 0x7fc0c8105e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:55.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.837+0000 7fc0cdc87700 1 --2- 172.21.15.27:0/3279128759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0c80f5ef0 0x7fc0c8106350 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:55.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.837+0000 7fc0cdc87700 1 -- 172.21.15.27:0/3279128759 >> 172.21.15.27:0/3279128759 conn(0x7fc0c80f0de0 msgr2=0x7fc0c80fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:55.843 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.837+0000 7fc0cdc87700 1 -- 172.21.15.27:0/3279128759 shutdown_connections 2021-11-03T17:11:55.843 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:55.837+0000 7fc0cdc87700 1 -- 172.21.15.27:0/3279128759 wait complete. 2021-11-03T17:11:55.843 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 2 2021-11-03T17:11:56.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:56 smithi027 conmon[32322]: cluster 2021-11-03T17:11:55.088859+0000 mon.smithi027 (mon.0) 480 : cluster [DBG] osdmap e44: 6 total, 6 up, 6 in 2021-11-03T17:11:56.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:56 smithi027 conmon[32322]: audit 2021-11-03T17:11:55.659167+0000 mon.smithi027 (mon.0) 481 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:11:56.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:56 smithi027 conmon[32322]: audit 2021-11-03T17:11:55.659964+0000 mon.smithi027 (mon.0) 482 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi027.jdwohn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2021-11-03T17:11:56.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:56 smithi027 conmon[32322]: audit 2021-11-03T17:11:55.663329+0000 mon.smithi027 (mon.0) 483 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi027.jdwohn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2021-11-03T17:11:56.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:56 smithi027 conmon[32322]: audit 2021-11-03T17:11:55.664022+0000 mon.smithi027 (mon.0) 484 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:11:56.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:56 smithi027 conmon[32322]: audit 2021-11-03T17:11:55.837312+0000 mon.smithi027 (mon.0) 485 : audit [DBG] from='client.? 172.21.15.27:0/3279128759' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:11:56.419 INFO:teuthology.run_tasks:Running task cephadm.shell... 2021-11-03T17:11:56.431 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi027.front.sepia.ceph.com 2021-11-03T17:11:56.431 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- bash -c 'ceph fs set cephfs max_mds 2' 2021-11-03T17:11:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:56 smithi115 conmon[34670]: cluster 2021-11-03T17:11:55.088859+0000 mon.smithi027 (mon.0) 480 2021-11-03T17:11:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:56 smithi115 conmon[34670]: : cluster [DBG] osdmap e44: 6 total, 6 up, 6 in 2021-11-03T17:11:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:56 smithi115 conmon[34670]: audit 2021-11-03T17:11:55.659167+0000 mon.smithi027 (mon.0) 481 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:11:56.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:56 smithi115 conmon[34670]: audit 2021-11-03T17:11:55.659964+0000 mon.smithi027 (mon.0) 482 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi027.jdwohn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2021-11-03T17:11:56.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:56 smithi115 conmon[34670]: audit 2021-11-03T17:11:55.663329+0000 mon.smithi027 (mon.0) 483 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi027.jdwohn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2021-11-03T17:11:56.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:56 smithi115 conmon[34670]: audit 2021-11-03T17:11:55.664022+0000 mon.smithi027 (mon.0) 484 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:11:56.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:56 smithi115 conmon[34670]: audit 2021-11-03T17:11:55.837312+0000 mon.smithi027 (mon.0) 485 : audit [DBG] from='client.? 172.21.15.27:0/3279128759' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:11:57.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:57 smithi027 conmon[32322]: cephadm 2021-11-03T17:11:55.664774 2021-11-03T17:11:57.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:57 smithi027 conmon[32322]: +0000 mgr.smithi027.ufhhvb (mgr.14162) 219 : cephadm [INF] Deploying daemon mds.cephfs.smithi027.jdwohn on smithi027 2021-11-03T17:11:57.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:57 smithi027 conmon[32322]: cluster 2021-11-03T17:11:56.573497+0000 mgr.smithi027.ufhhvb (mgr.14162) 220 : cluster [DBG] pgmap v185: 65 pgs: 6 creating+peering, 50 active+clean, 9 unknown; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:57.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:57 smithi115 conmon[34670]: cephadm 2021-11-03T17:11:55.664774+0000 mgr.smithi027.ufhhvb (mgr.14162) 219 : cephadm [INF] 2021-11-03T17:11:57.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:57 smithi115 conmon[34670]: Deploying daemon mds.cephfs.smithi027.jdwohn on smithi027 2021-11-03T17:11:57.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:57 smithi115 conmon[34670]: cluster 2021-11-03T17:11:56.573497+0000 mgr.smithi027.ufhhvb (mgr.14162) 220 : cluster [DBG] pgmap v185: 65 pgs: 6 creating+peering, 50 active+clean, 9 unknown; 0 B data, 31 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:58.905 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:58.907+0000 7f4b0ec91700 1 -- 172.21.15.27:0/1830120753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b080f5430 msgr2=0x7f4b080f5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:58.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:58.907+0000 7f4b0ec91700 1 --2- 172.21.15.27:0/1830120753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b080f5430 0x7f4b080f5850 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f4b040045a0 tx=0x7f4b04009b30).stop 2021-11-03T17:11:58.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:58.907+0000 7f4b0ec91700 1 -- 172.21.15.27:0/1830120753 shutdown_connections 2021-11-03T17:11:58.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:58.907+0000 7f4b0ec91700 1 --2- 172.21.15.27:0/1830120753 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b080f66d0 0x7f4b080f6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:58.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:58.907+0000 7f4b0ec91700 1 --2- 172.21.15.27:0/1830120753 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b080f5430 0x7f4b080f5850 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:58.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:58.907+0000 7f4b0ec91700 1 -- 172.21.15.27:0/1830120753 >> 172.21.15.27:0/1830120753 conn(0x7f4b080f0b90 msgr2=0x7f4b080f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:58.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:58.907+0000 7f4b0ec91700 1 -- 172.21.15.27:0/1830120753 shutdown_connections 2021-11-03T17:11:58.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:58.907+0000 7f4b0ec91700 1 -- 172.21.15.27:0/1830120753 wait complete. 2021-11-03T17:11:58.908 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:58.908+0000 7f4b0ec91700 1 Processor -- start 2021-11-03T17:11:58.908 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:58.908+0000 7f4b0ec91700 1 -- start start 2021-11-03T17:11:58.908 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:58.909+0000 7f4b0ec91700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b080f5430 0x7f4b080f9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:58.908 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:58.909+0000 7f4b0ec91700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b080f66d0 0x7f4b080f8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:58.908 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:58.909+0000 7f4b0ec91700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b080f8790 con 0x7f4b080f5430 2021-11-03T17:11:58.909 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:58.909+0000 7f4b0ec91700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b080f88d0 con 0x7f4b080f66d0 2021-11-03T17:11:58.909 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:58.909+0000 7f4b0dc8f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b080f5430 0x7f4b080f9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:58.909 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:58.909+0000 7f4b0dc8f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b080f5430 0x7f4b080f9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54966/0 (socket says 172.21.15.27:54966) 2021-11-03T17:11:58.909 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:58.909+0000 7f4b0dc8f700 1 -- 172.21.15.27:0/1816877029 learned_addr learned my addr 172.21.15.27:0/1816877029 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:11:58.912 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:58.909+0000 7f4b0d48e700 1 --2- 172.21.15.27:0/1816877029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b080f66d0 0x7f4b080f8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:58.912 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:58.910+0000 7f4b0dc8f700 1 -- 172.21.15.27:0/1816877029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b080f66d0 msgr2=0x7f4b080f8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:58.912 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:58.910+0000 7f4b0dc8f700 1 --2- 172.21.15.27:0/1816877029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b080f66d0 0x7f4b080f8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:58.912 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:58.910+0000 7f4b0dc8f700 1 -- 172.21.15.27:0/1816877029 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af8005040 con 0x7f4b080f5430 2021-11-03T17:11:58.913 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:58.910+0000 7f4b0dc8f700 1 --2- 172.21.15.27:0/1816877029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b080f5430 0x7f4b080f9db0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f4b040133e0 tx=0x7f4b0400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:58.913 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:58.910+0000 7f4afe7fc700 1 -- 172.21.15.27:0/1816877029 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b0400e070 con 0x7f4b080f5430 2021-11-03T17:11:58.913 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:58.911+0000 7f4b0ec91700 1 -- 172.21.15.27:0/1816877029 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b04005040 con 0x7f4b080f5430 2021-11-03T17:11:58.913 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:58.911+0000 7f4afe7fc700 1 -- 172.21.15.27:0/1816877029 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f4b04007ba0 con 0x7f4b080f5430 2021-11-03T17:11:58.914 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:58.911+0000 7f4afe7fc700 1 -- 172.21.15.27:0/1816877029 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b04019070 con 0x7f4b080f5430 2021-11-03T17:11:58.914 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:58.911+0000 7f4b0ec91700 1 -- 172.21.15.27:0/1816877029 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b080f8ea0 con 0x7f4b080f5430 2021-11-03T17:11:58.914 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:58.912+0000 7f4b0ec91700 1 -- 172.21.15.27:0/1816877029 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b08059100 con 0x7f4b080f5430 2021-11-03T17:11:58.914 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:58.912+0000 7f4afe7fc700 1 -- 172.21.15.27:0/1816877029 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f4b04025070 con 0x7f4b080f5430 2021-11-03T17:11:58.914 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:58.912+0000 7f4afe7fc700 1 --2- 172.21.15.27:0/1816877029 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f4af405c2f0 0x7f4af405e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:11:58.915 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:58.912+0000 7f4afe7fc700 1 -- 172.21.15.27:0/1816877029 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f4b04076b70 con 0x7f4b080f5430 2021-11-03T17:11:58.915 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:58.913+0000 7f4b0d48e700 1 --2- 172.21.15.27:0/1816877029 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f4af405c2f0 0x7f4af405e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:11:58.915 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:58.913+0000 7f4b0d48e700 1 --2- 172.21.15.27:0/1816877029 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f4af405c2f0 0x7f4af405e7b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f4af8009fd0 tx=0x7f4af800b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:11:58.916 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:58.917+0000 7f4afe7fc700 1 -- 172.21.15.27:0/1816877029 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f4b04049850 con 0x7f4b080f5430 2021-11-03T17:11:58.925 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:58 smithi027 conmon[32322]: cluster 2021-11-03T17: 2021-11-03T17:11:58.926 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:58 smithi027 conmon[32322]: 11:58.573862+0000 mgr.smithi027.ufhhvb (mgr.14162) 221 : cluster [DBG] pgmap v186: 65 pgs: 65 active+clean; 0 B data, 32 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:58.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:58 smithi115 conmon[34670]: cluster 2021-11-03T17:11:58.573862+0000 mgr.smithi027.ufhhvb (mgr.14162) 221 : cluster [DBG] pgmap v186: 65 pgs: 65 active+clean; 0 B data, 32 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:11:59.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:59.112+0000 7f4b0ec91700 1 -- 172.21.15.27:0/1816877029 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7f4b080fdca0 con 0x7f4b080f5430 2021-11-03T17:11:59.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:59.675+0000 7f4afe7fc700 1 -- 172.21.15.27:0/1816877029 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v3) v1 ==== 105+0+0 (secure 0 0 0) 0x7f4b0404ce70 con 0x7f4b080f5430 2021-11-03T17:11:59.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:59.677+0000 7f4b0ec91700 1 -- 172.21.15.27:0/1816877029 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f4af405c2f0 msgr2=0x7f4af405e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:59.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:59.677+0000 7f4b0ec91700 1 --2- 172.21.15.27:0/1816877029 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f4af405c2f0 0x7f4af405e7b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f4af8009fd0 tx=0x7f4af800b040).stop 2021-11-03T17:11:59.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:59.677+0000 7f4b0ec91700 1 -- 172.21.15.27:0/1816877029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b080f5430 msgr2=0x7f4b080f9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:11:59.676 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:59.677+0000 7f4b0ec91700 1 --2- 172.21.15.27:0/1816877029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b080f5430 0x7f4b080f9db0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f4b040133e0 tx=0x7f4b0400b040).stop 2021-11-03T17:11:59.676 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:59.677+0000 7f4b0ec91700 1 -- 172.21.15.27:0/1816877029 shutdown_connections 2021-11-03T17:11:59.677 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:59.677+0000 7f4b0ec91700 1 --2- 172.21.15.27:0/1816877029 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f4af405c2f0 0x7f4af405e7b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:59.677 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:59.677+0000 7f4b0ec91700 1 --2- 172.21.15.27:0/1816877029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b080f5430 0x7f4b080f9db0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:59.677 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:59.677+0000 7f4b0ec91700 1 --2- 172.21.15.27:0/1816877029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b080f66d0 0x7f4b080f8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:11:59.678 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:59.677+0000 7f4b0ec91700 1 -- 172.21.15.27:0/1816877029 >> 172.21.15.27:0/1816877029 conn(0x7f4b080f0b90 msgr2=0x7f4b0805f620 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:11:59.678 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:59.678+0000 7f4b0ec91700 1 -- 172.21.15.27:0/1816877029 shutdown_connections 2021-11-03T17:11:59.678 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:11:59.678+0000 7f4b0ec91700 1 -- 172.21.15.27:0/1816877029 wait complete. 2021-11-03T17:11:59.926 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:59 smithi115 conmon[34670]: audit 2021-11-03T17:11:59.114554+0000 mon.smithi027 (mon.0) 486 : audit [INF] from='client.? 172.21.15.27:0/1816877029' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2021-11-03T17:11:59.927 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:59 smithi115 conmon[34670]: audit 2021-11-03T17:11:59.320624+0000 mon.smithi027 (mon.0) 487 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:11:59.927 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:59 smithi115 conmon[34670]: audit 2021-11-03T17:11:59.321389+0000 mon.smithi027 (mon.0) 488 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi115.semywj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2021-11-03T17:11:59.927 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:59 smithi115 conmon[34670]: audit 2021-11-03T17:11:59.324092+0000 mon.smithi027 (mon.0) 489 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi115.semywj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2021-11-03T17:11:59.928 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:59 smithi115 conmon[34670]: audit 2021-11-03T17:11:59.324736+0000 mon.smithi027 (mon.0) 490 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:11:59.928 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:11:59 smithi115 conmon[34670]: cephadm 2021-11-03T17:11:59.325406+0000 mgr.smithi027.ufhhvb (mgr.14162) 222 : cephadm [INF] Deploying daemon mds.cephfs.smithi115.semywj on smithi115 2021-11-03T17:12:00.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:59 smithi027 conmon[32322]: audit 2021-11-03T17:11:59.114554 2021-11-03T17:12:00.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:59 smithi027 conmon[32322]: +0000 mon.smithi027 (mon.0) 486 : audit [INF] from='client.? 172.21.15.27:0/1816877029' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2021-11-03T17:12:00.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:59 smithi027 conmon[32322]: audit 2021-11-03T17:11:59.320624+0000 mon.smithi027 (mon.0) 487 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:12:00.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:59 smithi027 conmon[32322]: audit 2021-11-03T17:11:59.321389+0000 mon.smithi027 (mon.0) 488 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi115.semywj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2021-11-03T17:12:00.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:59 smithi027 conmon[32322]: audit 2021-11-03T17:11:59.324092+0000 mon.smithi027 (mon 2021-11-03T17:12:00.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:59 smithi027 conmon[32322]: .0) 489 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi115.semywj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2021-11-03T17:12:00.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:59 smithi027 conmon[32322]: audit 2021-11-03T17:11:59 2021-11-03T17:12:00.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:59 smithi027 conmon[32322]: .324736+0000 mon.smithi027 (mon.0) 490 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:12:00.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:59 smithi027 conmon[32322]: 2021-11-03T17:12:00.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:59 smithi027 conmon[32322]: cephadm 2021-11-03T17:12:00.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:59 smithi027 conmon[32322]: 2021-11-03T17:11:59.325406+0000 2021-11-03T17:12:00.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:59 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14162) 222 : cephadm [INF] 2021-11-03T17:12:00.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:11:59 smithi027 conmon[32322]: Deploying daemon mds.cephfs.smithi115.semywj on smithi115 2021-11-03T17:12:00.391 INFO:teuthology.run_tasks:Running task cephadm.shell... 2021-11-03T17:12:00.401 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi027.front.sepia.ceph.com 2021-11-03T17:12:00.402 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- bash -c 'ceph fs set cephfs allow_standby_replay false' 2021-11-03T17:12:00.930 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:00 smithi027 conmon[32322]: audit 2021-11-03T17:11:59.676592+0000 mon.smithi027 (mon.0) 491 : audit [INF] from='client.? 172.21.15.27:0/1816877029' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2021-11-03T17:12:00.931 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:00 smithi027 conmon[32322]: cluster 2021-11-03T17:11:59.676649+0000 mon.smithi027 (mon.0) 492 : cluster [DBG] mds.? [v2:172.21.15.115:6824/1621245036,v1:172.21.15.115:6825/1621245036] up:boot 2021-11-03T17:12:00.931 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:00 smithi027 conmon[32322]: cluster 2021-11-03T17:11:59.676696+0000 mon.smithi027 (mon.0) 493 : cluster [INF] daemon mds.cephfs.smithi115.zsfgnv assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2021-11-03T17:12:00.931 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:00 smithi027 conmon[32322]: cluster 2021-11-03T17:11:59.676803+0000 mon.smithi027 (mon.0) 494 : cluster [INF] Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2021-11-03T17:12:00.932 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:00 smithi027 conmon[32322]: cluster 2021-11-03T17:11:59.677025+0000 mon.smithi027 (mon.0) 495 : cluster [DBG] fsmap cephfs:0 1 up:standby 2021-11-03T17:12:00.933 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:00 smithi027 conmon[32322]: audit 2021-11-03T17:11:59.677178+0000 mon.smithi027 (mon.0) 496 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.zsfgnv"}]: dispatch 2021-11-03T17:12:00.933 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:00 smithi027 conmon[32322]: cluster 2021-11-03 2021-11-03T17:12:00.933 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:00 smithi027 conmon[32322]: T17:11:59.680255+0000 mon.smithi027 (mon.0) 497 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi115.zsfgnv=up:creating} 2021-11-03T17:12:00.934 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:00 smithi027 conmon[32322]: cluster 2021-11-03T17:11:59.693473+0000 mon.smithi027 (mon.0) 498 : cluster [INF] daemon mds.cephfs.smithi115.zsfgnv is now active in filesystem cephfs as rank 0 2021-11-03T17:12:00.934 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:00 smithi027 conmon[32322]: cluster 2021-11-03T17:12:00.574215+0000 mgr.smithi027.ufhhvb (mgr.14162) 223 : cluster [DBG] pgmap v187: 65 pgs: 65 active+clean; 0 B data, 32 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:12:00.979 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:00 smithi115 conmon[34670]: audit 2021-11-03T17:11:59.676592+0000 mon.smithi027 (mon.0) 491 : audit [INF] from='client.? 172.21.15.27:0/1816877029' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2021-11-03T17:12:00.979 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:00 smithi115 conmon[34670]: cluster 2021-11-03T17:11:59.676649+0000 mon.smithi027 (mon.0) 492 : cluster [DBG] mds.? [v2:172.21.15.115:6824/1621245036,v1:172.21.15.115:6825/1621245036] up:boot 2021-11-03T17:12:00.979 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:00 smithi115 conmon[34670]: cluster 2021-11-03T17:11:59.676696+0000 mon.smithi027 (mon.0) 493 : cluster [INF] daemon mds.cephfs.smithi115.zsfgnv assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2021-11-03T17:12:00.980 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:00 smithi115 conmon[34670]: cluster 2021-11-03T17:11:59.676803+0000 mon.smithi027 (mon.0) 494 : cluster [INF] Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2021-11-03T17:12:00.980 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:00 smithi115 conmon[34670]: cluster 2021-11-03T17:11:59.677025+0000 mon.smithi027 (mon.0) 495 : cluster [DBG] fsmap cephfs:0 1 up:standby 2021-11-03T17:12:00.980 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:00 smithi115 conmon[34670]: audit 2021-11-03T17:11:59.677178+0000 mon.smithi027 (mon.0) 496 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.zsfgnv"}]: dispatch 2021-11-03T17:12:00.981 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:00 smithi115 conmon[34670]: cluster 2021-11-03T 2021-11-03T17:12:00.981 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:00 smithi115 conmon[34670]: 17:11:59.680255+0000 mon.smithi027 (mon.0) 497 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi115.zsfgnv=up:creating} 2021-11-03T17:12:00.981 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:00 smithi115 conmon[34670]: cluster 2021-11-03T17:11:59.693473+0000 mon.smithi027 (mon.0) 498 : cluster [INF] daemon mds.cephfs.smithi115.zsfgnv is now active in filesystem cephfs as rank 0 2021-11-03T17:12:00.982 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:00 smithi115 conmon[34670]: cluster 2021-11-03T17:12: 2021-11-03T17:12:00.982 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:00 smithi115 conmon[34670]: 00.574215+0000 mgr.smithi027.ufhhvb (mgr.14162) 223 : cluster [DBG] 2021-11-03T17:12:00.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:00 smithi115 conmon[34670]: pgmap v187: 65 pgs: 65 active+clean; 0 B data, 32 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:12:01.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:01 smithi115 conmon[34670]: cluster 2021-11-03T17:12:00.690981+0000 2021-11-03T17:12:01.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:01 smithi115 conmon[34670]: mon.smithi027 (mon.0) 499 : cluster [DBG] mds.? [v2:172.21.15.115:6824/1621245036,v1:172.21.15.115:6825/1621245036] up:active 2021-11-03T17:12:01.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:01 smithi115 conmon[34670]: cluster 2021-11-03T17:12:00.691046+0000 mon.smithi027 (mon.0) 500 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi115.zsfgnv=up:active} 2021-11-03T17:12:02.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:01 smithi027 conmon[32322]: cluster 2021-11-03T17:12:00. 2021-11-03T17:12:02.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:01 smithi027 conmon[32322]: 690981+0000 mon.smithi027 (mon.0) 499 : cluster [DBG] mds.? [v2:172.21.15.115:6824/1621245036,v1:172.21.15.115:6825/1621245036] up:active 2021-11-03T17:12:02.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:01 smithi027 conmon[32322]: cluster 2021-11-03T17:12:00.691046+0000 mon.smithi027 (mon.0) 500 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi115.zsfgnv=up:active} 2021-11-03T17:12:02.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:02.236+0000 7f9e0272b700 1 -- 172.21.15.27:0/242396736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9dfc0f7980 msgr2=0x7f9dfc0f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:02.234 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:02.236+0000 7f9e0272b700 1 --2- 172.21.15.27:0/242396736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9dfc0f7980 0x7f9dfc0f7de0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f9df0004660 tx=0x7f9df0009b30).stop 2021-11-03T17:12:02.234 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:02.236+0000 7f9e0272b700 1 -- 172.21.15.27:0/242396736 shutdown_connections 2021-11-03T17:12:02.234 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:02.236+0000 7f9e0272b700 1 --2- 172.21.15.27:0/242396736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9dfc0f7980 0x7f9dfc0f7de0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:02.235 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:02.236+0000 7f9e0272b700 1 --2- 172.21.15.27:0/242396736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dfc0f66e0 0x7f9dfc0f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:02.235 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:02.236+0000 7f9e0272b700 1 -- 172.21.15.27:0/242396736 >> 172.21.15.27:0/242396736 conn(0x7f9dfc0f1e40 msgr2=0x7f9dfc0f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:12:02.235 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:02.236+0000 7f9e0272b700 1 -- 172.21.15.27:0/242396736 shutdown_connections 2021-11-03T17:12:02.235 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:02.236+0000 7f9e0272b700 1 -- 172.21.15.27:0/242396736 wait complete. 2021-11-03T17:12:02.236 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:02.236+0000 7f9e0272b700 1 Processor -- start 2021-11-03T17:12:02.236 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:02.236+0000 7f9e0272b700 1 -- start start 2021-11-03T17:12:02.236 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:02.237+0000 7f9e0272b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9dfc0f66e0 0x7f9dfc106ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:02.236 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:02.237+0000 7f9e0272b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dfc0f7980 0x7f9dfc1073d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:02.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:02.237+0000 7f9e0272b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc107980 con 0x7f9dfc0f66e0 2021-11-03T17:12:02.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:02.237+0000 7f9e0272b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dfc107ac0 con 0x7f9dfc0f7980 2021-11-03T17:12:02.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:02.237+0000 7f9dfbfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9dfc0f66e0 0x7f9dfc106ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:02.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:02.237+0000 7f9dfbfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9dfc0f66e0 0x7f9dfc106ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:54984/0 (socket says 172.21.15.27:54984) 2021-11-03T17:12:02.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:02.237+0000 7f9dfbfff700 1 -- 172.21.15.27:0/484966016 learned_addr learned my addr 172.21.15.27:0/484966016 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:12:02.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:02.237+0000 7f9dfb7fe700 1 --2- 172.21.15.27:0/484966016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dfc0f7980 0x7f9dfc1073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:02.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:02.237+0000 7f9dfbfff700 1 -- 172.21.15.27:0/484966016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dfc0f7980 msgr2=0x7f9dfc1073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:02.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:02.237+0000 7f9dfbfff700 1 --2- 172.21.15.27:0/484966016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dfc0f7980 0x7f9dfc1073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:02.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:02.237+0000 7f9dfbfff700 1 -- 172.21.15.27:0/484966016 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9df0005040 con 0x7f9dfc0f66e0 2021-11-03T17:12:02.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:02.237+0000 7f9dfb7fe700 1 --2- 172.21.15.27:0/484966016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dfc0f7980 0x7f9dfc1073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:12:02.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:02.237+0000 7f9dfbfff700 1 --2- 172.21.15.27:0/484966016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9dfc0f66e0 0x7f9dfc106ee0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f9dec00e7c0 tx=0x7f9dec00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:12:02.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:02.237+0000 7f9df8ff9700 1 -- 172.21.15.27:0/484966016 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9dec0092e0 con 0x7f9dfc0f66e0 2021-11-03T17:12:02.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:02.237+0000 7f9df8ff9700 1 -- 172.21.15.27:0/484966016 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f9dec00f040 con 0x7f9dfc0f66e0 2021-11-03T17:12:02.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:02.237+0000 7f9e0272b700 1 -- 172.21.15.27:0/484966016 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dfc108460 con 0x7f9dfc0f66e0 2021-11-03T17:12:02.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:02.237+0000 7f9df8ff9700 1 -- 172.21.15.27:0/484966016 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9dec0146c0 con 0x7f9dfc0f66e0 2021-11-03T17:12:02.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:02.238+0000 7f9e0272b700 1 -- 172.21.15.27:0/484966016 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dfc108a20 con 0x7f9dfc0f66e0 2021-11-03T17:12:02.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:02.239+0000 7f9e0272b700 1 -- 172.21.15.27:0/484966016 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9dfc045ad0 con 0x7f9dfc0f66e0 2021-11-03T17:12:02.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:02.239+0000 7f9df8ff9700 1 -- 172.21.15.27:0/484966016 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f9dec01b030 con 0x7f9dfc0f66e0 2021-11-03T17:12:02.242 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:02.239+0000 7f9df8ff9700 1 --2- 172.21.15.27:0/484966016 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f9de405c2a0 0x7f9de405e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:02.242 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:02.239+0000 7f9df8ff9700 1 -- 172.21.15.27:0/484966016 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f9dec075ec0 con 0x7f9dfc0f66e0 2021-11-03T17:12:02.242 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:02.240+0000 7f9dfb7fe700 1 --2- 172.21.15.27:0/484966016 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f9de405c2a0 0x7f9de405e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:02.242 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:02.240+0000 7f9dfb7fe700 1 --2- 172.21.15.27:0/484966016 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f9de405c2a0 0x7f9de405e760 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f9df0004d80 tx=0x7f9df000b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:12:02.243 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:02.246+0000 7f9df8ff9700 1 -- 172.21.15.27:0/484966016 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f9dec048ba0 con 0x7f9dfc0f66e0 2021-11-03T17:12:02.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:02.456+0000 7f9e0272b700 1 -- 172.21.15.27:0/484966016 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9dfc027a00 con 0x7f9dfc0f66e0 2021-11-03T17:12:03.015 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:03.019+0000 7f9df8ff9700 1 -- 172.21.15.27:0/484966016 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9dec0190c0 con 0x7f9dfc0f66e0 2021-11-03T17:12:03.017 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:03.021+0000 7f9e0272b700 1 -- 172.21.15.27:0/484966016 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f9de405c2a0 msgr2=0x7f9de405e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:03.017 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:03.021+0000 7f9e0272b700 1 --2- 172.21.15.27:0/484966016 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f9de405c2a0 0x7f9de405e760 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f9df0004d80 tx=0x7f9df000b040).stop 2021-11-03T17:12:03.018 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:03.021+0000 7f9e0272b700 1 -- 172.21.15.27:0/484966016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9dfc0f66e0 msgr2=0x7f9dfc106ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:03.018 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:03.021+0000 7f9e0272b700 1 --2- 172.21.15.27:0/484966016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9dfc0f66e0 0x7f9dfc106ee0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f9dec00e7c0 tx=0x7f9dec00b040).stop 2021-11-03T17:12:03.018 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:03.021+0000 7f9e0272b700 1 -- 172.21.15.27:0/484966016 shutdown_connections 2021-11-03T17:12:03.019 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:03.021+0000 7f9e0272b700 1 --2- 172.21.15.27:0/484966016 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f9de405c2a0 0x7f9de405e760 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:03.019 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:03.021+0000 7f9e0272b700 1 --2- 172.21.15.27:0/484966016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9dfc0f66e0 0x7f9dfc106ee0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:03.019 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:03.021+0000 7f9e0272b700 1 --2- 172.21.15.27:0/484966016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9dfc0f7980 0x7f9dfc1073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:03.019 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:03.021+0000 7f9e0272b700 1 -- 172.21.15.27:0/484966016 >> 172.21.15.27:0/484966016 conn(0x7f9dfc0f1e40 msgr2=0x7f9dfc0fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:12:03.020 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:03.021+0000 7f9e0272b700 1 -- 172.21.15.27:0/484966016 shutdown_connections 2021-11-03T17:12:03.020 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:03.021+0000 7f9e0272b700 1 -- 172.21.15.27:0/484966016 wait complete. 2021-11-03T17:12:03.039 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:02 smithi027 conmon[32322]: audit 2021-11-03T17:12:01.803961+0000 mon.smithi027 ( 2021-11-03T17:12:03.039 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:02 smithi027 conmon[32322]: mon.0) 501 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:12:03.040 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:02 smithi027 conmon[32322]: audit 2021-11-03T17:12:01.804764+0000 mon.smithi027 (mon.0) 502 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi027.fksixo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2021-11-03T17:12:03.040 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:02 smithi027 conmon[32322]: audit 2021-11-03T17:12:01.807863+0000 mon.smithi027 (mon.0) 503 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi027.fksixo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2021-11-03T17:12:03.040 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:02 smithi027 conmon[32322]: audit 2021-11-03T17:12:01.808600+0000 mon.smithi027 (mon.0) 504 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:12:03.041 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:02 smithi027 conmon[32322]: cephadm 2021-11-03T17:12:01.809286+0000 mgr.smithi027.ufhhvb (mgr.14162) 224 : cephadm [INF] Deploying daemon mds.cephfs.smithi027.fksixo on smithi027 2021-11-03T17:12:03.041 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:02 smithi027 conmon[32322]: audit 2021-11-03T17:12:02.458765+0000 mon.smithi027 (mon.0) 505 : audit [INF] from='client.? 172.21.15.27:0/484966016' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2021-11-03T17:12:03.041 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:02 smithi027 conmon[32322]: cluster 2021-11-03T17:12:02.574679+0000 mgr.smithi027.ufhhvb (mgr.14162) 225 : cluster [DBG] pgmap v188: 65 pgs: 65 active+clean; 1.7 KiB data, 32 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:12:03.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:02 smithi115 conmon[34670]: audit 2021-11-03T17:12:01.803961+0000 mon.smithi027 (mon.0) 501 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:12:03.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:02 smithi115 conmon[34670]: audit 2021-11-03T17:12:01.804764+0000 2021-11-03T17:12:03.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:02 smithi115 conmon[34670]: mon.smithi027 (mon.0) 502 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi027.fksixo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2021-11-03T17:12:03.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:02 smithi115 conmon[34670]: audit 2021-11-03T17:12:01.807863+0000 mon.smithi027 (mon.0) 503 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi027.fksixo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2021-11-03T17:12:03.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:02 smithi115 conmon[34670]: audit 2021-11-03T17:12:01.808600+0000 mon.smithi027 (mon.0) 504 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:12:03.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:02 smithi115 conmon[34670]: cephadm 2021-11-03T17:12:01.809286+0000 mgr.smithi027.ufhhvb (mgr.14162) 224 : cephadm [INF] Deploying daemon mds.cephfs.smithi027.fksixo on smithi027 2021-11-03T17:12:03.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:02 smithi115 conmon[34670]: audit 2021-11-03T17:12:02.458765+0000 mon.smithi027 (mon.0) 505 : audit [INF] from='client.? 172.21.15.27:0/484966016' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2021-11-03T17:12:03.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:02 smithi115 conmon[34670]: cluster 2021-11-03T17:12:02.574679+0000 mgr.smithi027.ufhhvb (mgr.14162) 225 : cluster [DBG] pgmap v188: 65 pgs: 65 active+clean; 1.7 KiB data, 32 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:12:03.945 INFO:teuthology.run_tasks:Running task cephadm.shell... 2021-11-03T17:12:03.955 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi027.front.sepia.ceph.com 2021-11-03T17:12:03.956 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- bash -c 'ceph fs dump' 2021-11-03T17:12:04.169 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:04 smithi027 conmon[32322]: audit 2021-11-03T17:12:03.020891+0000 mon.smithi027 (mon.0) 506 : audit [INF] from='client.? 172.21.15.27:0/484966016' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2021-11-03T17:12:04.169 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:04 smithi027 conmon[32322]: cluster 2021-11-03T17:12:03.020979+0000 mon.smithi027 (mon.0) 507 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi115.zsfgnv=up:active} 2021-11-03T17:12:04.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:04 smithi115 conmon[34670]: audit 2021-11-03T17:12:03.020891+0000 mon.smithi027 (mon.0) 506 : audit 2021-11-03T17:12:04.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:04 smithi115 conmon[34670]: [INF] from='client.? 172.21.15.27:0/484966016' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2021-11-03T17:12:04.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:04 smithi115 conmon[34670]: cluster 2021-11-03T17:12:03.020979+0000 mon.smithi027 (mon.0) 507 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi115.zsfgnv=up:active} 2021-11-03T17:12:05.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:05 smithi027 conmon[32322]: cluster 2021-11-03T17:12: 2021-11-03T17:12:05.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:05 smithi027 conmon[32322]: 04.022634+0000 mon.smithi027 (mon.0) 508 : cluster [DBG] mds.? [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] up:boot 2021-11-03T17:12:05.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:05 smithi027 conmon[32322]: cluster 2021-11-03T17:12:04.022682+0000 mon.smithi027 (mon.0) 509 : cluster [INF] daemon mds.cephfs.smithi027.jdwohn assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2021-11-03T17:12:05.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:05 smithi027 conmon[32322]: cluster 2021-11-03T17:12:04.022787+0000 mon.smithi027 (mon.0) 510 : cluster [WRN] Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2021-11-03T17:12:05.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:05 smithi027 conmon[32322]: cluster 2021-11-03T17:12:04.022808+0000 mon.smithi027 (mon.0) 511 : cluster [INF] Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2021-11-03T17:12:05.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:05 smithi027 conmon[32322]: cluster 2021-11-03T17:12:04.023232+0000 mon.smithi027 (mon.0) 512 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi115.zsfgnv=up:active} 1 up:standby 2021-11-03T17:12:05.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:05 smithi027 conmon[32322]: audit 2021-11-03T17:12:04.023328+0000 mon.smithi027 (mon.0) 513 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.jdwohn"}]: dispatch 2021-11-03T17:12:05.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:05 smithi027 conmon[32322]: cluster 2021-11-03T17:12:04.029539+0000 mon.smithi027 (mon.0) 514 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi115.zsfgnv=up:active,1=cephfs.smithi027.jdwohn=up:creating} 2021-11-03T17:12:05.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:05 smithi027 conmon[32322]: cluster 2021-11-03T17:12:04.043743+0000 mon.smithi027 (mon.0) 515 : cluster [INF] daemon mds.cephfs.smithi027.jdwohn is now active in filesystem cephfs as rank 1 2021-11-03T17:12:05.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:05 smithi027 conmon[32322]: cluster 2021-11-03T17:12:04.575042+0000 mgr.smithi027.ufhhvb (mgr.14162) 226 : cluster [DBG] pgmap v189: 65 pgs: 65 active+clean; 2.2 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 3.5 KiB/s wr, 2 op/s 2021-11-03T17:12:05.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:05 smithi115 conmon[34670]: cluster 2021 2021-11-03T17:12:05.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:05 smithi115 conmon[34670]: -11-03T17:12:04.022634+0000 mon.smithi027 (mon.0) 508 : cluster [DBG] mds.? [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] up:boot 2021-11-03T17:12:05.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:05 smithi115 conmon[34670]: cluster 2021-11-03T17:12:04.022682+0000 2021-11-03T17:12:05.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:05 smithi115 conmon[34670]: mon.smithi027 (mon.0) 509 : cluster [INF] daemon mds.cephfs.smithi027.jdwohn assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2021-11-03T17:12:05.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:05 smithi115 conmon[34670]: cluster 2021-11-03T17 2021-11-03T17:12:05.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:05 smithi115 conmon[34670]: :12:04.022787+0000 mon.smithi027 (mon.0) 510 : cluster [WRN] Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2021-11-03T17:12:05.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:05 smithi115 conmon[34670]: cluster 2021-11-03T17:12: 2021-11-03T17:12:05.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:05 smithi115 conmon[34670]: 04.022808+0000 mon.smithi027 (mon.0) 511 : cluster [INF] Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2021-11-03T17:12:05.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:05 smithi115 conmon[34670]: cluster 2021-11 2021-11-03T17:12:05.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:05 smithi115 conmon[34670]: -03T17:12:04.023232+0000 mon.smithi027 (mon.0 2021-11-03T17:12:05.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:05 smithi115 conmon[34670]: ) 512 : cluster [DBG] fsmap cephfs:1 {0=cephfs.smithi115.zsfgnv=up:active} 1 up:standby 2021-11-03T17:12:05.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:05 smithi115 conmon[34670]: audit 2021-11-03T17:12:05.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:05 smithi115 conmon[34670]: 2021- 2021-11-03T17:12:05.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:05 smithi115 conmon[34670]: 11-03T17:12:04.023328 2021-11-03T17:12:05.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:05 smithi115 conmon[34670]: +0000 mon.smithi027 (mon.0) 2021-11-03T17:12:05.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:05 smithi115 conmon[34670]: 513 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.jdwohn"}]: dispatch 2021-11-03T17:12:05.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:05 smithi115 conmon[34670]: cluster 2021- 2021-11-03T17:12:05.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:05 smithi115 conmon[34670]: 11-03T17:12:04.029539+0000 mon.smithi027 2021-11-03T17:12:05.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:05 smithi115 conmon[34670]: (mon.0) 514 : cluster [DBG] 2021-11-03T17:12:05.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:05 smithi115 conmon[34670]: fsmap cephfs:2 {0=cephfs.smithi115.zsfgnv=up:active,1=cephfs.smithi027.jdwohn=up:creating} 2021-11-03T17:12:05.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:05 smithi115 conmon[34670]: cluster 2021-11-03T17:12:05.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:05 smithi115 conmon[34670]: 2021-11-03T17:12:04.043743+0000 mon.smithi027 (mon.0) 515 : cluster [INF] 2021-11-03T17:12:05.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:05 smithi115 conmon[34670]: daemon mds.cephfs.smithi027.jdwohn is now active in filesystem cephfs as rank 1 2021-11-03T17:12:05.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:05 smithi115 conmon[34670]: cluster 2021-11-03T17:12:05.490 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:05 smithi115 conmon[34670]: 2021-11-03T17:12:04. 2021-11-03T17:12:05.490 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:05 smithi115 conmon[34670]: 575042+0000 mgr.smithi027.ufhhvb (mgr.14162) 226 : cluster 2021-11-03T17:12:05.490 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:05 smithi115 conmon[34670]: [DBG] pgmap v189: 65 pgs: 65 active+clean; 2.2 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 3.5 KiB/s wr, 2 op/s 2021-11-03T17:12:06.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.192+0000 7f95334e6700 1 -- 172.21.15.27:0/3482338467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f952c0f66e0 msgr2=0x7f952c0f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:06.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.192+0000 7f95334e6700 1 --2- 172.21.15.27:0/3482338467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f952c0f66e0 0x7f952c0f6b00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f9528004660 tx=0x7f9528009b30).stop 2021-11-03T17:12:06.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.192+0000 7f95334e6700 1 -- 172.21.15.27:0/3482338467 shutdown_connections 2021-11-03T17:12:06.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.192+0000 7f95334e6700 1 --2- 172.21.15.27:0/3482338467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f952c0f7980 0x7f952c0f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:06.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.192+0000 7f95334e6700 1 --2- 172.21.15.27:0/3482338467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f952c0f66e0 0x7f952c0f6b00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:06.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.192+0000 7f95334e6700 1 -- 172.21.15.27:0/3482338467 >> 172.21.15.27:0/3482338467 conn(0x7f952c0f1e40 msgr2=0x7f952c0f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:12:06.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.192+0000 7f95334e6700 1 -- 172.21.15.27:0/3482338467 shutdown_connections 2021-11-03T17:12:06.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.192+0000 7f95334e6700 1 -- 172.21.15.27:0/3482338467 wait complete. 2021-11-03T17:12:06.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.193+0000 7f95334e6700 1 Processor -- start 2021-11-03T17:12:06.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.193+0000 7f95334e6700 1 -- start start 2021-11-03T17:12:06.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.193+0000 7f95334e6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f952c0f66e0 0x7f952c06b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:06.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.193+0000 7f95334e6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f952c0f7980 0x7f952c069e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:06.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.193+0000 7f95334e6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952c10ac20 con 0x7f952c0f7980 2021-11-03T17:12:06.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.193+0000 7f95334e6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952c10ad60 con 0x7f952c0f66e0 2021-11-03T17:12:06.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.193+0000 7f9531282700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f952c0f66e0 0x7f952c06b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:06.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.193+0000 7f9531282700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f952c0f66e0 0x7f952c06b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.27:46182/0 (socket says 172.21.15.27:46182) 2021-11-03T17:12:06.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.193+0000 7f9531282700 1 -- 172.21.15.27:0/2181012853 learned_addr learned my addr 172.21.15.27:0/2181012853 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:12:06.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.193+0000 7f9530a81700 1 --2- 172.21.15.27:0/2181012853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f952c0f7980 0x7f952c069e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:06.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.193+0000 7f9531282700 1 -- 172.21.15.27:0/2181012853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f952c0f7980 msgr2=0x7f952c069e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:06.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.193+0000 7f9531282700 1 --2- 172.21.15.27:0/2181012853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f952c0f7980 0x7f952c069e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:06.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.193+0000 7f9531282700 1 -- 172.21.15.27:0/2181012853 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9528005040 con 0x7f952c0f66e0 2021-11-03T17:12:06.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.193+0000 7f9530a81700 1 --2- 172.21.15.27:0/2181012853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f952c0f7980 0x7f952c069e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:12:06.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.193+0000 7f9531282700 1 --2- 172.21.15.27:0/2181012853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f952c0f66e0 0x7f952c06b9c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f9528014040 tx=0x7f952800b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:12:06.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.194+0000 7f9521ffb700 1 -- 172.21.15.27:0/2181012853 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9528019070 con 0x7f952c0f66e0 2021-11-03T17:12:06.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.194+0000 7f95334e6700 1 -- 172.21.15.27:0/2181012853 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f952c10afe0 con 0x7f952c0f66e0 2021-11-03T17:12:06.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.194+0000 7f9521ffb700 1 -- 172.21.15.27:0/2181012853 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f9528007e50 con 0x7f952c0f66e0 2021-11-03T17:12:06.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.194+0000 7f9521ffb700 1 -- 172.21.15.27:0/2181012853 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9528014d30 con 0x7f952c0f66e0 2021-11-03T17:12:06.196 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.194+0000 7f95334e6700 1 -- 172.21.15.27:0/2181012853 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f952c10b540 con 0x7f952c0f66e0 2021-11-03T17:12:06.197 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.195+0000 7f95334e6700 1 -- 172.21.15.27:0/2181012853 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f952c0f5ad0 con 0x7f952c0f66e0 2021-11-03T17:12:06.197 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.195+0000 7f9521ffb700 1 -- 172.21.15.27:0/2181012853 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f9528025070 con 0x7f952c0f66e0 2021-11-03T17:12:06.197 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.196+0000 7f9521ffb700 1 --2- 172.21.15.27:0/2181012853 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f951805c2b0 0x7f951805e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:06.198 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.196+0000 7f9521ffb700 1 -- 172.21.15.27:0/2181012853 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f9528076f90 con 0x7f952c0f66e0 2021-11-03T17:12:06.198 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.196+0000 7f9530a81700 1 --2- 172.21.15.27:0/2181012853 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f951805c2b0 0x7f951805e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:06.198 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.196+0000 7f9530a81700 1 --2- 172.21.15.27:0/2181012853 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f951805c2b0 0x7f951805e770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f951c007fc0 tx=0x7f951c00b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:12:06.198 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.200+0000 7f9521ffb700 1 -- 172.21.15.27:0/2181012853 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f9528049c70 con 0x7f952c0f66e0 2021-11-03T17:12:06.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:06 smithi027 conmon[32322]: cluster 2021-11-03T17:12:05.033215+0000 mon.smithi027 (mon.0) 516 : cluster [DBG] mds.? [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] up:active 2021-11-03T17:12:06.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:06 smithi027 conmon[32322]: cluster 2021-11-03T17:12:05.033283+0000 mon.smithi027 (mon.0) 517 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi115.zsfgnv=up:active,1=cephfs.smithi027.jdwohn=up:active} 2021-11-03T17:12:06.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:06 smithi027 conmon[32322]: audit 2021-11-03T17:12:05.624602+0000 mon.smithi027 (mon.0) 518 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:12:06.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:06 smithi027 conmon[32322]: audit 2021-11-03T17:12:05.625398+0000 mon.smithi027 (mon.0) 519 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:12:06.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.416+0000 7f95334e6700 1 -- 172.21.15.27:0/2181012853 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f952c06aba0 con 0x7f952c0f66e0 2021-11-03T17:12:06.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.416+0000 7f9521ffb700 1 -- 172.21.15.27:0/2181012853 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 9 v9) v1 ==== 75+0+1311 (secure 0 0 0) 0x7f952804d290 con 0x7f952c0f66e0 2021-11-03T17:12:06.412 INFO:teuthology.orchestra.run.smithi027.stdout:e9 2021-11-03T17:12:06.412 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:12:06.412 INFO:teuthology.orchestra.run.smithi027.stdout:compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:12:06.413 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:12:06.413 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:12:06.413 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:12:06.413 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:12:06.414 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 9 2021-11-03T17:12:06.414 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 2021-11-03T17:12:06.414 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:12:06.414 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:12:05.026191+0000 2021-11-03T17:12:06.415 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:12:06.415 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:12:06.415 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:12:06.415 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:12:06.415 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:12:06.416 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:12:06.416 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:12:06.416 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 0 2021-11-03T17:12:06.416 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:12:06.417 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 2 2021-11-03T17:12:06.417 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:12:06.417 INFO:teuthology.orchestra.run.smithi027.stdout:up {0=24273,1=14436} 2021-11-03T17:12:06.417 INFO:teuthology.orchestra.run.smithi027.stdout:failed 2021-11-03T17:12:06.417 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:12:06.418 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:12:06.418 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:12:06.418 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:12:06.418 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:12:06.419 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:12:06.419 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:12:06.419 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{0:24273} state up:active seq 3 join_fscid=1 addr [v2:172.21.15.115:6824/1621245036,v1:172.21.15.115:6825/1621245036]] 2021-11-03T17:12:06.419 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 3 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060]] 2021-11-03T17:12:06.419 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:12:06.420 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:12:06.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.418+0000 7f95334e6700 1 -- 172.21.15.27:0/2181012853 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f951805c2b0 msgr2=0x7f951805e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:06.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.418+0000 7f95334e6700 1 --2- 172.21.15.27:0/2181012853 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f951805c2b0 0x7f951805e770 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f951c007fc0 tx=0x7f951c00b040).stop 2021-11-03T17:12:06.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.418+0000 7f95334e6700 1 -- 172.21.15.27:0/2181012853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f952c0f66e0 msgr2=0x7f952c06b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:06.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.418+0000 7f95334e6700 1 --2- 172.21.15.27:0/2181012853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f952c0f66e0 0x7f952c06b9c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f9528014040 tx=0x7f952800b040).stop 2021-11-03T17:12:06.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.418+0000 7f95334e6700 1 -- 172.21.15.27:0/2181012853 shutdown_connections 2021-11-03T17:12:06.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.418+0000 7f95334e6700 1 --2- 172.21.15.27:0/2181012853 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f951805c2b0 0x7f951805e770 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:06.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.418+0000 7f95334e6700 1 --2- 172.21.15.27:0/2181012853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f952c0f66e0 0x7f952c06b9c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:06.423 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.418+0000 7f95334e6700 1 --2- 172.21.15.27:0/2181012853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f952c0f7980 0x7f952c069e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:06.423 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.419+0000 7f95334e6700 1 -- 172.21.15.27:0/2181012853 >> 172.21.15.27:0/2181012853 conn(0x7f952c0f1e40 msgr2=0x7f952c0fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:12:06.423 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.419+0000 7f95334e6700 1 -- 172.21.15.27:0/2181012853 shutdown_connections 2021-11-03T17:12:06.423 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:06.419+0000 7f95334e6700 1 -- 172.21.15.27:0/2181012853 wait complete. 2021-11-03T17:12:06.423 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 9 2021-11-03T17:12:06.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:06 smithi115 conmon[34670]: cluster 2021-11-03T17:12:05.033215+0000 mon.smithi027 (mon.0 2021-11-03T17:12:06.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:06 smithi115 conmon[34670]: ) 516 : cluster [DBG] mds.? [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] up:active 2021-11-03T17:12:06.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:06 smithi115 conmon[34670]: cluster 2021-11-03T17:12:05.033283+0000 mon.smithi027 (mon.0) 517 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi115.zsfgnv=up:active,1=cephfs.smithi027.jdwohn=up:active} 2021-11-03T17:12:06.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:06 smithi115 conmon[34670]: audit 2021-11-03T17:12:05.624602+0000 mon.smithi027 (mon.0) 2021-11-03T17:12:06.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:06 smithi115 conmon[34670]: 518 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:12:06.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:06 smithi115 conmon[34670]: audit 2021-11 2021-11-03T17:12:06.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:06 smithi115 conmon[34670]: -03T17:12:05.625398+0000 mon.smithi027 2021-11-03T17:12:06.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:06 smithi115 conmon[34670]: (mon.0) 519 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:12:06.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:06 smithi115 conmon[34670]: 2021-11-03T17:12:07.259 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2021-11-03T17:12:07.285 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:07 smithi027 conmon[32322]: audit 2021-11-03T17:12:06.417242+0000 2021-11-03T17:12:07.286 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:07 smithi027 conmon[32322]: mon.smithi115 (mon.1) 15 : audit [DBG] from='client.? 172.21.15.27:0/2181012853' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:12:07.286 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:07 smithi027 conmon[32322]: cluster 2021-11-03T17:12:06.575518+0000 mgr.smithi027.ufhhvb (mgr.14162) 227 : cluster [DBG] pgmap v190: 65 pgs: 65 active+clean; 3.6 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 4.7 KiB/s wr, 10 op/s 2021-11-03T17:12:07.287 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:07 smithi027 conmon[32322]: cluster 2021-11-03T17:12:06.622528+0000 mon.smithi027 (mon.0) 520 : cluster [INF] Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2021-11-03T17:12:07.287 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:07 smithi027 conmon[32322]: cluster 2021-11-03T17:12:06.622561+0000 mon.smithi027 (mon.0) 521 : cluster [INF] Cluster is now healthy 2021-11-03T17:12:07.287 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:07 smithi027 conmon[32322]: cluster 2021-11-03T17:12:06.625676+0000 mon.smithi027 (mon.0) 522 : cluster [DBG] mds.? [v2:172.21.15.115:6826/3367380088,v1:172.21.15.115:6827/3367380088] up:boot 2021-11-03T17:12:07.288 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:07 smithi027 conmon[32322]: cluster 2021-11-03T17:12:06.625728+0000 mon.smithi027 (mon.0) 523 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi115.zsfgnv=up:active,1=cephfs.smithi027.jdwohn=up:active} 1 up:standby 2021-11-03T17:12:07.288 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:07 smithi027 conmon[32322]: audit 2021-11-03T17:12:06.625870+0000 mon.smithi027 (mon.0) 524 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.semywj"}]: dispatch 2021-11-03T17:12:07.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:07 smithi115 conmon[34670]: audit 2021-11-03T17:12:06.417242+0000 mon.smithi115 (mon.1) 15 : audit [DBG] from='client.? 172.21.15.27:0/2181012853' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:12:07.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:07 smithi115 conmon[34670]: cluster 2021-11-03T17:12:06.575518+0000 mgr.smithi027.ufhhvb (mgr.14162) 227 : cluster [DBG] pgmap v190: 65 pgs: 65 active+clean; 3.6 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 4.7 KiB/s wr, 10 op/s 2021-11-03T17:12:07.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:07 smithi115 conmon[34670]: cluster 2021-11-03T17:12:06.622528+0000 mon.smithi027 (mon.0) 520 : cluster [INF] Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2021-11-03T17:12:07.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:07 smithi115 conmon[34670]: cluster 2021-11-03T17:12:06.622561+0000 mon.smithi027 (mon.0) 521 : cluster [INF] Cluster is now healthy 2021-11-03T17:12:07.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:07 smithi115 conmon[34670]: cluster 2021-11-03T17:12:06.625676+0000 mon.smithi027 (mon.0) 522 : cluster [DBG] mds.? [v2:172.21.15.115:6826/3367380088,v1:172.21.15.115:6827/3367380088] up:boot 2021-11-03T17:12:07.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:07 smithi115 conmon[34670]: cluster 2021-11-03T17:12:06.625728+0000 mon.smithi027 (mon.0) 523 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi115.zsfgnv=up:active,1=cephfs.smithi027.jdwohn=up:active} 1 up:standby 2021-11-03T17:12:07.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:07 smithi115 conmon[34670]: audit 2021-11-03T17:12:06.625870+0000 mon.smithi027 (mon.0) 524 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.semywj"}]: dispatch 2021-11-03T17:12:08.934 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:08 smithi027 conmon[32322]: audit 2021-11-03T17:12:07.646093+0000 mon.smithi027 (mon.0) 525 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:12:08.934 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:08 smithi027 conmon[32322]: cluster 2021-11-03T17:12:08.576124+0000 mgr.smithi027.ufhhvb (mgr.14162) 228 : cluster [DBG] pgmap v191: 65 pgs: 65 active+clean; 3.6 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 3.2 KiB/s wr, 7 op/s 2021-11-03T17:12:08.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:08 smithi115 conmon[34670]: audit 2021-11-03T17:12:07.646093+0000 mon.smithi027 (mon.0) 525 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:12:08.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:08 smithi115 conmon[34670]: cluster 2021-11-03T17:12:08.576124+0000 mgr.smithi027.ufhhvb (mgr.14162) 228 : cluster [DBG] pgmap v191: 65 pgs: 65 active+clean; 3.6 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 3.2 KiB/s wr, 7 op/s 2021-11-03T17:12:09.469 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.470+0000 7fe09bfff700 1 -- 172.21.15.27:0/2098632030 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe09c0f5440 msgr2=0x7fe09c0f5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:09.470 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.470+0000 7fe09bfff700 1 --2- 172.21.15.27:0/2098632030 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe09c0f5440 0x7fe09c0f5860 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7fe0900046e0 tx=0x7fe090009b30).stop 2021-11-03T17:12:09.471 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.470+0000 7fe09bfff700 1 -- 172.21.15.27:0/2098632030 shutdown_connections 2021-11-03T17:12:09.471 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.470+0000 7fe09bfff700 1 --2- 172.21.15.27:0/2098632030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe09c0f66e0 0x7fe09c0f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:09.471 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.470+0000 7fe09bfff700 1 --2- 172.21.15.27:0/2098632030 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe09c0f5440 0x7fe09c0f5860 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:09.472 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.470+0000 7fe09bfff700 1 -- 172.21.15.27:0/2098632030 >> 172.21.15.27:0/2098632030 conn(0x7fe09c0f0ba0 msgr2=0x7fe09c0f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:12:09.472 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.470+0000 7fe09bfff700 1 -- 172.21.15.27:0/2098632030 shutdown_connections 2021-11-03T17:12:09.473 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.470+0000 7fe09bfff700 1 -- 172.21.15.27:0/2098632030 wait complete. 2021-11-03T17:12:09.473 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.471+0000 7fe09bfff700 1 Processor -- start 2021-11-03T17:12:09.474 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.471+0000 7fe09bfff700 1 -- start start 2021-11-03T17:12:09.474 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.471+0000 7fe09bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe09c0f5440 0x7fe09c108fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:09.475 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.471+0000 7fe09bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe09c0f66e0 0x7fe09c1094b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:09.475 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.471+0000 7fe09bfff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c109a60 con 0x7fe09c0f66e0 2021-11-03T17:12:09.476 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.471+0000 7fe09bfff700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c109ba0 con 0x7fe09c0f5440 2021-11-03T17:12:09.476 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.471+0000 7fe09a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe09c0f66e0 0x7fe09c1094b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:09.477 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.471+0000 7fe09a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe09c0f66e0 0x7fe09c1094b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55028/0 (socket says 172.21.15.27:55028) 2021-11-03T17:12:09.477 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.471+0000 7fe09a7fc700 1 -- 172.21.15.27:0/1997483361 learned_addr learned my addr 172.21.15.27:0/1997483361 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:12:09.477 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.471+0000 7fe09affd700 1 --2- 172.21.15.27:0/1997483361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe09c0f5440 0x7fe09c108fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:09.477 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.472+0000 7fe09a7fc700 1 -- 172.21.15.27:0/1997483361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe09c0f5440 msgr2=0x7fe09c108fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:09.478 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.472+0000 7fe09a7fc700 1 --2- 172.21.15.27:0/1997483361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe09c0f5440 0x7fe09c108fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:09.478 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.472+0000 7fe09a7fc700 1 -- 172.21.15.27:0/1997483361 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe090005040 con 0x7fe09c0f66e0 2021-11-03T17:12:09.478 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.472+0000 7fe09a7fc700 1 --2- 172.21.15.27:0/1997483361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe09c0f66e0 0x7fe09c1094b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fe08c0026d0 tx=0x7fe08c009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:12:09.478 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.472+0000 7fe0837fe700 1 -- 172.21.15.27:0/1997483361 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe08c0079c0 con 0x7fe09c0f66e0 2021-11-03T17:12:09.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.472+0000 7fe09bfff700 1 -- 172.21.15.27:0/1997483361 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe09c10a540 con 0x7fe09c0f66e0 2021-11-03T17:12:09.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.472+0000 7fe0837fe700 1 -- 172.21.15.27:0/1997483361 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7fe08c00f040 con 0x7fe09c0f66e0 2021-11-03T17:12:09.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.472+0000 7fe0837fe700 1 -- 172.21.15.27:0/1997483361 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe08c014820 con 0x7fe09c0f66e0 2021-11-03T17:12:09.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.472+0000 7fe09bfff700 1 -- 172.21.15.27:0/1997483361 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe09c10ab00 con 0x7fe09c0f66e0 2021-11-03T17:12:09.480 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.473+0000 7fe09bfff700 1 -- 172.21.15.27:0/1997483361 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe09c045ad0 con 0x7fe09c0f66e0 2021-11-03T17:12:09.480 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.473+0000 7fe0837fe700 1 -- 172.21.15.27:0/1997483361 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fe08c0090d0 con 0x7fe09c0f66e0 2021-11-03T17:12:09.480 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.474+0000 7fe0837fe700 1 --2- 172.21.15.27:0/1997483361 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fe08405c300 0x7fe08405e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:09.481 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.474+0000 7fe0837fe700 1 -- 172.21.15.27:0/1997483361 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fe08c075f00 con 0x7fe09c0f66e0 2021-11-03T17:12:09.481 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.474+0000 7fe09affd700 1 --2- 172.21.15.27:0/1997483361 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fe08405c300 0x7fe08405e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:09.481 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.474+0000 7fe09affd700 1 --2- 172.21.15.27:0/1997483361 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fe08405c300 0x7fe08405e7c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fe0900046b0 tx=0x7fe090009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:12:09.482 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.478+0000 7fe0837fe700 1 -- 172.21.15.27:0/1997483361 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fe08c048c90 con 0x7fe09c0f66e0 2021-11-03T17:12:09.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.700+0000 7fe09bfff700 1 -- 172.21.15.27:0/1997483361 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe09c027a00 con 0x7fe09c0f66e0 2021-11-03T17:12:09.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.700+0000 7fe0837fe700 1 -- 172.21.15.27:0/1997483361 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+2946 (secure 0 0 0) 0x7fe08c04c2b0 con 0x7fe09c0f66e0 2021-11-03T17:12:09.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.702+0000 7fe09bfff700 1 -- 172.21.15.27:0/1997483361 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fe08405c300 msgr2=0x7fe08405e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:09.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.702+0000 7fe09bfff700 1 --2- 172.21.15.27:0/1997483361 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fe08405c300 0x7fe08405e7c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fe0900046b0 tx=0x7fe090009b30).stop 2021-11-03T17:12:09.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.702+0000 7fe09bfff700 1 -- 172.21.15.27:0/1997483361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe09c0f66e0 msgr2=0x7fe09c1094b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:09.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.702+0000 7fe09bfff700 1 --2- 172.21.15.27:0/1997483361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe09c0f66e0 0x7fe09c1094b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fe08c0026d0 tx=0x7fe08c009de0).stop 2021-11-03T17:12:09.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.702+0000 7fe09bfff700 1 -- 172.21.15.27:0/1997483361 shutdown_connections 2021-11-03T17:12:09.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.702+0000 7fe09bfff700 1 --2- 172.21.15.27:0/1997483361 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fe08405c300 0x7fe08405e7c0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:09.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.702+0000 7fe09bfff700 1 --2- 172.21.15.27:0/1997483361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe09c0f5440 0x7fe09c108fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:09.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.702+0000 7fe09bfff700 1 --2- 172.21.15.27:0/1997483361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe09c0f66e0 0x7fe09c1094b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:09.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.702+0000 7fe09bfff700 1 -- 172.21.15.27:0/1997483361 >> 172.21.15.27:0/1997483361 conn(0x7fe09c0f0ba0 msgr2=0x7fe09c0fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:12:09.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.702+0000 7fe09bfff700 1 -- 172.21.15.27:0/1997483361 shutdown_connections 2021-11-03T17:12:09.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:09.702+0000 7fe09bfff700 1 -- 172.21.15.27:0/1997483361 wait complete. 2021-11-03T17:12:09.700 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 11 2021-11-03T17:12:09.706 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:12:10.670 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- bash -c 'ceph --format=json mds versions | jq -e ". | add == 4"' 2021-11-03T17:12:10.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:10 smithi115 conmon[34670]: cluster 2021-11-03T17:12:09.650786+0000 mon.smithi027 (mon.0) 526 : cluster [DBG] mds.? [v2:172.21.15.27:6828/1337044739,v1:172.21.15.27:6829/1337044739] up:boot 2021-11-03T17:12:10.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:10 smithi115 conmon[34670]: cluster 2021-11-03T17:12:09.650851+0000 mon.smithi027 (mon.0) 527 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi115.zsfgnv=up:active,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:12:10.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:10 smithi115 conmon[34670]: audit 2021-11-03T17:12:09.650981+0000 mon.smithi027 (mon.0) 528 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.fksixo"}]: dispatch 2021-11-03T17:12:10.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:10 smithi115 conmon[34670]: audit 2021-11-03T17:12:09.700901+0000 mon.smithi027 (mon.0) 529 : audit [DBG] from='client.? 172.21.15.27:0/1997483361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2021-11-03T17:12:10.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:10 smithi115 conmon[34670]: cluster 2021-11-03T17:12:10.576582+0000 mgr.smithi027.ufhhvb (mgr.14162) 229 : cluster [DBG] pgmap v192: 65 pgs: 65 active+clean; 3.6 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s wr, 5 op/s 2021-11-03T17:12:11.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:10 smithi027 conmon[32322]: cluster 2021-11-03T17:12:09.650786+0000 mon.smithi027 2021-11-03T17:12:11.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:10 smithi027 conmon[32322]: (mon.0) 526 : cluster [DBG] mds.? [v2:172.21.15.27:6828/1337044739,v1:172.21.15.27:6829/1337044739] up:boot 2021-11-03T17:12:11.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:10 smithi027 conmon[32322]: cluster 2021-11-03T17:12:09.650851+0000 mon.smithi027 (mon.0) 527 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi115.zsfgnv=up:active,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:12:11.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:10 smithi027 conmon[32322]: audit 2021-11-03T17:12:09.650981+0000 mon.smithi027 (mon.0) 528 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.fksixo"}]: dispatch 2021-11-03T17:12:11.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:10 smithi027 conmon[32322]: audit 2021-11-03T17:12:09.700901+0000 mon.smithi027 (mon.0) 529 : audit [DBG] from='client.? 172.21.15.27:0/1997483361' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2021-11-03T17:12:11.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:10 smithi027 conmon[32322]: cluster 2021-11-03T17:12:10.576582+0000 mgr.smithi027.ufhhvb (mgr.14162) 229 : cluster [DBG] pgmap v192: 65 pgs: 65 active+clean; 3.6 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s wr, 5 op/s 2021-11-03T17:12:11.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:11 smithi115 conmon[34670]: cluster 2021-11-03T17:12:10.656078+0000 mon.smithi027 (mon.0) 530 : cluster [DBG] mds.? [v2:172.21.15.115:6826/3367380088,v1:172.21.15.115:6827/3367380088] up:standby 2021-11-03T17:12:11.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:11 smithi115 conmon[34670]: cluster 2021-11-03T17:12:10.656140+0000 mon.smithi027 (mon.0) 531 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi115.zsfgnv=up:active,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:12:12.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:11 smithi027 conmon[32322]: cluster 2021-11-03T17:12:10.656078+0000 mon.smithi027 (mon.0) 530 : cluster [DBG] mds.? [v2:172.21.15.115:6826/3367380088,v1:172.21.15.115:6827/3367380088] up:standby 2021-11-03T17:12:12.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:11 smithi027 conmon[32322]: cluster 2021-11-03T17:12:10.656140+0000 mon.smithi027 (mon.0) 531 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi115.zsfgnv=up:active,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:12:12.736 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.739+0000 7fab84f7c700 1 -- 172.21.15.27:0/3943263911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab800f4d60 msgr2=0x7fab800f5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:12.736 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.739+0000 7fab84f7c700 1 --2- 172.21.15.27:0/3943263911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab800f4d60 0x7fab800f5180 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7fab68004660 tx=0x7fab68009b30).stop 2021-11-03T17:12:12.737 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.739+0000 7fab84f7c700 1 -- 172.21.15.27:0/3943263911 shutdown_connections 2021-11-03T17:12:12.737 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.739+0000 7fab84f7c700 1 --2- 172.21.15.27:0/3943263911 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab800f5f60 0x7fab800f63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:12.737 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.739+0000 7fab84f7c700 1 --2- 172.21.15.27:0/3943263911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab800f4d60 0x7fab800f5180 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:12.737 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.739+0000 7fab84f7c700 1 -- 172.21.15.27:0/3943263911 >> 172.21.15.27:0/3943263911 conn(0x7fab800f04c0 msgr2=0x7fab800f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:12:12.738 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.739+0000 7fab84f7c700 1 -- 172.21.15.27:0/3943263911 shutdown_connections 2021-11-03T17:12:12.738 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.739+0000 7fab84f7c700 1 -- 172.21.15.27:0/3943263911 wait complete. 2021-11-03T17:12:12.738 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.740+0000 7fab84f7c700 1 Processor -- start 2021-11-03T17:12:12.739 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.740+0000 7fab84f7c700 1 -- start start 2021-11-03T17:12:12.739 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.740+0000 7fab84f7c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab800f4d60 0x7fab80105e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:12.739 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.740+0000 7fab84f7c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab800f5f60 0x7fab80106350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:12.739 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.740+0000 7fab84f7c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab80106950 con 0x7fab800f4d60 2021-11-03T17:12:12.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.740+0000 7fab84f7c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab80107160 con 0x7fab800f5f60 2021-11-03T17:12:12.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.740+0000 7fab7e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab800f4d60 0x7fab80105e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:12.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.740+0000 7fab7e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab800f4d60 0x7fab80105e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55038/0 (socket says 172.21.15.27:55038) 2021-11-03T17:12:12.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.740+0000 7fab7e59c700 1 -- 172.21.15.27:0/4040526302 learned_addr learned my addr 172.21.15.27:0/4040526302 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:12:12.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.740+0000 7fab7dd9b700 1 --2- 172.21.15.27:0/4040526302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab800f5f60 0x7fab80106350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:12.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.740+0000 7fab7e59c700 1 -- 172.21.15.27:0/4040526302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab800f5f60 msgr2=0x7fab80106350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:12.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.740+0000 7fab7e59c700 1 --2- 172.21.15.27:0/4040526302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab800f5f60 0x7fab80106350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:12.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.740+0000 7fab7e59c700 1 -- 172.21.15.27:0/4040526302 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab68005040 con 0x7fab800f4d60 2021-11-03T17:12:12.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.740+0000 7fab7e59c700 1 --2- 172.21.15.27:0/4040526302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab800f4d60 0x7fab80105e60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7fab68000c00 tx=0x7fab6800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:12:12.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.741+0000 7fab76ffd700 1 -- 172.21.15.27:0/4040526302 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab6801b070 con 0x7fab800f4d60 2021-11-03T17:12:12.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.741+0000 7fab84f7c700 1 -- 172.21.15.27:0/4040526302 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab80107380 con 0x7fab800f4d60 2021-11-03T17:12:12.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.741+0000 7fab76ffd700 1 -- 172.21.15.27:0/4040526302 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7fab68007e50 con 0x7fab800f4d60 2021-11-03T17:12:12.744 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.741+0000 7fab76ffd700 1 -- 172.21.15.27:0/4040526302 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab68014d00 con 0x7fab800f4d60 2021-11-03T17:12:12.744 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.741+0000 7fab84f7c700 1 -- 172.21.15.27:0/4040526302 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab801078e0 con 0x7fab800f4d60 2021-11-03T17:12:12.745 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.742+0000 7fab84f7c700 1 -- 172.21.15.27:0/4040526302 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab800f4150 con 0x7fab800f4d60 2021-11-03T17:12:12.746 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.742+0000 7fab76ffd700 1 -- 172.21.15.27:0/4040526302 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fab68027070 con 0x7fab800f4d60 2021-11-03T17:12:12.746 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.742+0000 7fab76ffd700 1 --2- 172.21.15.27:0/4040526302 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fab6c05c2b0 0x7fab6c05e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:12.746 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.743+0000 7fab76ffd700 1 -- 172.21.15.27:0/4040526302 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fab68078d90 con 0x7fab800f4d60 2021-11-03T17:12:12.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.743+0000 7fab7dd9b700 1 --2- 172.21.15.27:0/4040526302 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fab6c05c2b0 0x7fab6c05e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:12.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.743+0000 7fab7dd9b700 1 --2- 172.21.15.27:0/4040526302 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fab6c05c2b0 0x7fab6c05e770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fab70004fd0 tx=0x7fab7000b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:12:12.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.747+0000 7fab76ffd700 1 -- 172.21.15.27:0/4040526302 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fab6804ba70 con 0x7fab800f4d60 2021-11-03T17:12:12.992 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.997+0000 7fab84f7c700 1 -- 172.21.15.27:0/4040526302 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7fab800f7a30 con 0x7fab800f4d60 2021-11-03T17:12:12.992 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.997+0000 7fab76ffd700 1 -- 172.21.15.27:0/4040526302 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v12) v1 ==== 78+0+86 (secure 0 0 0) 0x7fab6804f090 con 0x7fab800f4d60 2021-11-03T17:12:12.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.999+0000 7fab84f7c700 1 -- 172.21.15.27:0/4040526302 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fab6c05c2b0 msgr2=0x7fab6c05e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:12.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.999+0000 7fab84f7c700 1 --2- 172.21.15.27:0/4040526302 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fab6c05c2b0 0x7fab6c05e770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fab70004fd0 tx=0x7fab7000b040).stop 2021-11-03T17:12:12.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.999+0000 7fab84f7c700 1 -- 172.21.15.27:0/4040526302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab800f4d60 msgr2=0x7fab80105e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:12.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.999+0000 7fab84f7c700 1 --2- 172.21.15.27:0/4040526302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab800f4d60 0x7fab80105e60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7fab68000c00 tx=0x7fab6800b040).stop 2021-11-03T17:12:12.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.999+0000 7fab84f7c700 1 -- 172.21.15.27:0/4040526302 shutdown_connections 2021-11-03T17:12:12.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.999+0000 7fab84f7c700 1 --2- 172.21.15.27:0/4040526302 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fab6c05c2b0 0x7fab6c05e770 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:12.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.999+0000 7fab84f7c700 1 --2- 172.21.15.27:0/4040526302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab800f4d60 0x7fab80105e60 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:12.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.999+0000 7fab84f7c700 1 --2- 172.21.15.27:0/4040526302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab800f5f60 0x7fab80106350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:12.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.999+0000 7fab84f7c700 1 -- 172.21.15.27:0/4040526302 >> 172.21.15.27:0/4040526302 conn(0x7fab800f04c0 msgr2=0x7fab800fab50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:12:12.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.999+0000 7fab84f7c700 1 -- 172.21.15.27:0/4040526302 shutdown_connections 2021-11-03T17:12:12.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:12.999+0000 7fab84f7c700 1 -- 172.21.15.27:0/4040526302 wait complete. 2021-11-03T17:12:13.005 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:12:13.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:13 smithi027 conmon[32322]: audit 2021-11-03T17:12:12.476249+0000 mon.smithi027 (mon.0) 532 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:12:13.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:13 smithi027 conmon[32322]: audit 2021-11-03T17:12:12.479342+0000 mon.smithi027 (mon.0) 533 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:12:13.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:13 smithi027 conmon[32322]: audit 2021-11-03T17:12:12.483309+0000 mon.smithi027 (mon.0) 534 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:12:13.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:13 smithi027 conmon[32322]: cluster 2021-11-03T17:12:12.577076+0000 mgr.smithi027.ufhhvb (mgr.14162) 230 : cluster [DBG] pgmap v193: 65 pgs: 65 active+clean; 3.6 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.9 KiB/s wr, 4 op/s 2021-11-03T17:12:13.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:13 smithi027 conmon[32322]: audit 2021-11-03T17:12:12.998156+0000 mon.smithi027 (mon.0) 535 : audit [DBG] from='client.? 172.21.15.27:0/4040526302' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2021-11-03T17:12:13.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:13 smithi115 conmon[34670]: audit 2021-11-03T17:12:12.476249+0000 mon.smithi027 (mon.0) 532 : audit [INF] 2021-11-03T17:12:13.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:13 smithi115 conmon[34670]: from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:12:13.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:13 smithi115 conmon[34670]: audit 2021-11-03T17:12:12.479342+0000 mon.smithi027 (mon.0) 533 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:12:13.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:13 smithi115 conmon[34670]: audit 2021-11 2021-11-03T17:12:13.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:13 smithi115 conmon[34670]: -03T17:12:12.483309+0000 mon.smithi027 (mon.0) 534 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:12:13.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:13 smithi115 conmon[34670]: cluster 2021-11-03T17:12:13.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:13 smithi115 conmon[34670]: 2021-11-03T17:12 2021-11-03T17:12:13.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:13 smithi115 conmon[34670]: :12.577076+0000 mgr.smithi027.ufhhvb (mgr. 2021-11-03T17:12:13.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:13 smithi115 conmon[34670]: 14162) 230 : cluster [DBG] pgmap v193: 65 pgs: 65 active+clean; 3.6 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.9 KiB/s wr, 4 op/s 2021-11-03T17:12:13.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:13 smithi115 conmon[34670]: audit 2021- 2021-11-03T17:12:13.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:13 smithi115 conmon[34670]: 11-03T17:12:12.998156+0000 2021-11-03T17:12:13.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:13 smithi115 conmon[34670]: mon.smithi027 (mon.0) 535 : audit 2021-11-03T17:12:13.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:13 smithi115 conmon[34670]: [DBG] from='client.? 172.21.15.27:0/4040526302' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2021-11-03T17:12:13.899 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2021-11-03T17:12:13.912 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2021-11-03T17:12:14.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.090+0000 7fc2b2999700 1 -- 172.21.15.27:0/422503559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2ac0f55a0 msgr2=0x7fc2ac0f59c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:14.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.090+0000 7fc2b2999700 1 --2- 172.21.15.27:0/422503559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2ac0f55a0 0x7fc2ac0f59c0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7fc29c004660 tx=0x7fc29c009b30).stop 2021-11-03T17:12:14.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.090+0000 7fc2b2999700 1 -- 172.21.15.27:0/422503559 shutdown_connections 2021-11-03T17:12:14.090 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.090+0000 7fc2b2999700 1 --2- 172.21.15.27:0/422503559 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2ac0f6840 0x7fc2ac0f6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:14.090 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.090+0000 7fc2b2999700 1 --2- 172.21.15.27:0/422503559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2ac0f55a0 0x7fc2ac0f59c0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:14.090 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.090+0000 7fc2b2999700 1 -- 172.21.15.27:0/422503559 >> 172.21.15.27:0/422503559 conn(0x7fc2ac0f0ce0 msgr2=0x7fc2ac0f3120 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:12:14.090 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.090+0000 7fc2b2999700 1 -- 172.21.15.27:0/422503559 shutdown_connections 2021-11-03T17:12:14.091 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.090+0000 7fc2b2999700 1 -- 172.21.15.27:0/422503559 wait complete. 2021-11-03T17:12:14.091 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.091+0000 7fc2b2999700 1 Processor -- start 2021-11-03T17:12:14.091 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.091+0000 7fc2b2999700 1 -- start start 2021-11-03T17:12:14.091 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.091+0000 7fc2b2999700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2ac0f55a0 0x7fc2ac106e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:14.092 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.091+0000 7fc2b2999700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2ac0f6840 0x7fc2ac107310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:14.092 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.091+0000 7fc2b2999700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ac1078c0 con 0x7fc2ac0f6840 2021-11-03T17:12:14.092 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.091+0000 7fc2b2999700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2ac107a00 con 0x7fc2ac0f55a0 2021-11-03T17:12:14.092 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.091+0000 7fc2ab7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2ac0f6840 0x7fc2ac107310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:14.093 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.091+0000 7fc2abfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2ac0f55a0 0x7fc2ac106e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:14.093 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.091+0000 7fc2ab7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2ac0f6840 0x7fc2ac107310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55050/0 (socket says 172.21.15.27:55050) 2021-11-03T17:12:14.093 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.091+0000 7fc2ab7fe700 1 -- 172.21.15.27:0/2252101805 learned_addr learned my addr 172.21.15.27:0/2252101805 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:12:14.093 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.092+0000 7fc2abfff700 1 -- 172.21.15.27:0/2252101805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2ac0f6840 msgr2=0x7fc2ac107310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:14.093 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.092+0000 7fc2abfff700 1 --2- 172.21.15.27:0/2252101805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2ac0f6840 0x7fc2ac107310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:14.094 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.092+0000 7fc2abfff700 1 -- 172.21.15.27:0/2252101805 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc29c005040 con 0x7fc2ac0f55a0 2021-11-03T17:12:14.094 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.092+0000 7fc2ab7fe700 1 --2- 172.21.15.27:0/2252101805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2ac0f6840 0x7fc2ac107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-03T17:12:14.094 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.092+0000 7fc2abfff700 1 --2- 172.21.15.27:0/2252101805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2ac0f55a0 0x7fc2ac106e20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fc29c004660 tx=0x7fc29c00b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:12:14.094 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.092+0000 7fc2a8ff9700 1 -- 172.21.15.27:0/2252101805 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc29c019070 con 0x7fc2ac0f55a0 2021-11-03T17:12:14.095 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.092+0000 7fc2a8ff9700 1 -- 172.21.15.27:0/2252101805 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7fc29c007c40 con 0x7fc2ac0f55a0 2021-11-03T17:12:14.095 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.092+0000 7fc2a8ff9700 1 -- 172.21.15.27:0/2252101805 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc29c01ebd0 con 0x7fc2ac0f55a0 2021-11-03T17:12:14.095 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.092+0000 7fc2b2999700 1 -- 172.21.15.27:0/2252101805 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2ac10c440 con 0x7fc2ac0f55a0 2021-11-03T17:12:14.096 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.092+0000 7fc2b2999700 1 -- 172.21.15.27:0/2252101805 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2ac10ca00 con 0x7fc2ac0f55a0 2021-11-03T17:12:14.096 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.093+0000 7fc2b2999700 1 -- 172.21.15.27:0/2252101805 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2ac0fdd60 con 0x7fc2ac0f55a0 2021-11-03T17:12:14.097 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.093+0000 7fc2a8ff9700 1 -- 172.21.15.27:0/2252101805 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fc29c01ed30 con 0x7fc2ac0f55a0 2021-11-03T17:12:14.097 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.094+0000 7fc2a8ff9700 1 --2- 172.21.15.27:0/2252101805 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fc29805c2a0 0x7fc29805e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:14.097 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.094+0000 7fc2a8ff9700 1 -- 172.21.15.27:0/2252101805 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fc29c0771b0 con 0x7fc2ac0f55a0 2021-11-03T17:12:14.097 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.094+0000 7fc2ab7fe700 1 --2- 172.21.15.27:0/2252101805 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fc29805c2a0 0x7fc29805e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:14.098 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.094+0000 7fc2ab7fe700 1 --2- 172.21.15.27:0/2252101805 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fc29805c2a0 0x7fc29805e760 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fc294007a30 tx=0x7fc29400b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:12:14.098 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.098+0000 7fc2a8ff9700 1 -- 172.21.15.27:0/2252101805 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fc29c049f10 con 0x7fc2ac0f55a0 2021-11-03T17:12:14.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.260+0000 7fc2b2999700 1 -- 172.21.15.27:0/2252101805 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc2ac10d360 con 0x7fc2ac0f55a0 2021-11-03T17:12:14.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.261+0000 7fc2a8ff9700 1 -- 172.21.15.27:0/2252101805 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+14062 (secure 0 0 0) 0x7fc29c04d530 con 0x7fc2ac0f55a0 2021-11-03T17:12:14.257 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:12:14.257 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":44,"fsid":"32ab977e-3cc8-11ec-8c28-001a4aab830c","created":"2021-11-03T17:05:46.314150+0000","modified":"2021-11-03T17:11:55.085265+0000","last_up_change":"2021-11-03T17:10:34.577218+0000","last_in_change":"2021-11-03T17:10:16.846149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2021-11-03T17:09:48.488849+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2021-11-03T17:11:52.008416+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"43","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2021-11-03T17:11:52.507226+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2f6bbb8f-4e3c-401a-b1bd-69ba9cdd9d13","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6802","nonce":3126282997},{"type":"v1","addr":"172.21.15.27:6803","nonce":3126282997}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6804","nonce":3126282997},{"type":"v1","addr":"172.21.15.27:6805","nonce":3126282997}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6808","nonce":3126282997},{"type":"v1","addr":"172.21.15.27:6809","nonce":3126282997}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6806","nonce":3126282997},{"type":"v1","addr":"172.21.15.27:6807","nonce":3126282997}]},"public_addr":"172.21.15.27:6803/3126282997","cluster_addr":"172.21.15.27:6805/3126282997","heartbeat_back_addr":"172.21.15.27:6809/3126282997","heartbeat_front_addr":"172.21.15.27:6807/3126282997","state":["exists","up"]},{"osd":1,"uuid":"0942b897-5aa3-461a-a2d6-943d5739d963","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6810","nonce":2723402359},{"type":"v1","addr":"172.21.15.27:6811","nonce":2723402359}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6812","nonce":2723402359},{"type":"v1","addr":"172.21.15.27:6813","nonce":2723402359}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6816","nonce":2723402359},{"type":"v1","addr":"172.21.15.27:6817","nonce":2723402359}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6814","nonce":2723402359},{"type":"v1","addr":"172.21.15.27:6815","nonce":2723402359}]},"public_addr":"172.21.15.27:6811/2723402359","cluster_addr":"172.21.15.27:6813/2723402359","heartbeat_back_addr":"172.21.15.27:6817/2723402359","heartbeat_front_addr":"172.21.15.27:6815/2723402359","state":["exists","up"]},{"osd":2,"uuid":"3ea602d0-3e10-4f30-96cd-288e53a7a00a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6818","nonce":329182726},{"type":"v1","addr":"172.21.15.27:6819","nonce":329182726}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6820","nonce":329182726},{"type":"v1","addr":"172.21.15.27:6821","nonce":329182726}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6824","nonce":329182726},{"type":"v1","addr":"172.21.15.27:6825","nonce":329182726}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6822","nonce":329182726},{"type":"v1","addr":"172.21.15.27:6823","nonce":329182726}]},"public_addr":"172.21.15.27:6819/329182726","cluster_addr":"172.21.15.27:6821/329182726","heartbeat_back_addr":"172.21.15.27:6825/329182726","heartbeat_front_addr":"172.21.15.27:6823/329182726","state":["exists","up"]},{"osd":3,"uuid":"c517d190-0d21-4db0-994c-5f971d67f103","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":27,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6800","nonce":4026713633},{"type":"v1","addr":"172.21.15.115:6801","nonce":4026713633}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6802","nonce":4026713633},{"type":"v1","addr":"172.21.15.115:6803","nonce":4026713633}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6806","nonce":4026713633},{"type":"v1","addr":"172.21.15.115:6807","nonce":4026713633}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6804","nonce":4026713633},{"type":"v1","addr":"172.21.15.115:6805","nonce":4026713633}]},"public_addr":"172.21.15.115:6801/4026713633","cluster_addr":"172.21.15.115:6803/4026713633","heartbeat_back_addr":"172.21.15.115:6807/4026713633","heartbeat_front_addr":"172.21.15.115:6805/4026713633","state":["exists","up"]},{"osd":4,"uuid":"aeeb2c74-919d-4ac7-8c7a-1dd273225f59","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6808","nonce":1601394331},{"type":"v1","addr":"172.21.15.115:6809","nonce":1601394331}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6810","nonce":1601394331},{"type":"v1","addr":"172.21.15.115:6811","nonce":1601394331}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6814","nonce":1601394331},{"type":"v1","addr":"172.21.15.115:6815","nonce":1601394331}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6812","nonce":1601394331},{"type":"v1","addr":"172.21.15.115:6813","nonce":1601394331}]},"public_addr":"172.21.15.115:6809/1601394331","cluster_addr":"172.21.15.115:6811/1601394331","heartbeat_back_addr":"172.21.15.115:6815/1601394331","heartbeat_front_addr":"172.21.15.115:6813/1601394331","state":["exists","up"]},{"osd":5,"uuid":"047cfc7d-01d6-4859-8dc8-01d3811b9060","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6816","nonce":2350455075},{"type":"v1","addr":"172.21.15.115:6817","nonce":2350455075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6818","nonce":2350455075},{"type":"v1","addr":"172.21.15.115:6819","nonce":2350455075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6822","nonce":2350455075},{"type":"v1","addr":"172.21.15.115:6823","nonce":2350455075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6820","nonce":2350455075},{"type":"v1","addr":"172.21.15.115:6821","nonce":2350455075}]},"public_addr":"172.21.15.115:6817/2350455075","cluster_addr":"172.21.15.115:6819/2350455075","heartbeat_back_addr":"172.21.15.115:6823/2350455075","heartbeat_front_addr":"172.21.15.115:6821/2350455075","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-11-03T17:08:50.764082+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-11-03T17:09:12.720566+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-11-03T17:09:34.450930+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-11-03T17:09:53.016884+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-11-03T17:10:12.728118+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138297136906239,"old_weight":0,"last_purged_snaps_scrub":"2021-11-03T17:10:32.668302+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.27:6801/3506673123":"2021-11-04T17:06:51.527604+0000","172.21.15.27:6800/3506673123":"2021-11-04T17:06:51.527604+0000","172.21.15.27:0/92526005":"2021-11-04T17:06:51.527604+0000","172.21.15.27:0/1091817081":"2021-11-04T17:06:51.527604+0000","172.21.15.27:0/3225036173":"2021-11-04T17:06:34.410322+0000","172.21.15.27:6801/774407793":"2021-11-04T17:06:34.410322+0000","172.21.15.27:6800/774407793":"2021-11-04T17:06:34.410322+0000","172.21.15.27:0/3947386121":"2021-11-04T17:06:09.549455+0000","172.21.15.27:6800/388626148":"2021-11-04T17:06:09.549455+0000","172.21.15.27:0/2008079603":"2021-11-04T17:06:09.549455+0000","172.21.15.27:0/2202850722":"2021-11-04T17:06:34.410322+0000","172.21.15.27:6801/388626148":"2021-11-04T17:06:09.549455+0000"},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2021-11-03T17:12:14.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.263+0000 7fc2b2999700 1 -- 172.21.15.27:0/2252101805 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fc29805c2a0 msgr2=0x7fc29805e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:14.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.263+0000 7fc2b2999700 1 --2- 172.21.15.27:0/2252101805 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fc29805c2a0 0x7fc29805e760 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fc294007a30 tx=0x7fc29400b040).stop 2021-11-03T17:12:14.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.263+0000 7fc2b2999700 1 -- 172.21.15.27:0/2252101805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2ac0f55a0 msgr2=0x7fc2ac106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:14.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.263+0000 7fc2b2999700 1 --2- 172.21.15.27:0/2252101805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2ac0f55a0 0x7fc2ac106e20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fc29c004660 tx=0x7fc29c00b040).stop 2021-11-03T17:12:14.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.263+0000 7fc2b2999700 1 -- 172.21.15.27:0/2252101805 shutdown_connections 2021-11-03T17:12:14.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.263+0000 7fc2b2999700 1 --2- 172.21.15.27:0/2252101805 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fc29805c2a0 0x7fc29805e760 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:14.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.263+0000 7fc2b2999700 1 --2- 172.21.15.27:0/2252101805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc2ac0f55a0 0x7fc2ac106e20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:14.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.263+0000 7fc2b2999700 1 --2- 172.21.15.27:0/2252101805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc2ac0f6840 0x7fc2ac107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:14.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.263+0000 7fc2b2999700 1 -- 172.21.15.27:0/2252101805 >> 172.21.15.27:0/2252101805 conn(0x7fc2ac0f0ce0 msgr2=0x7fc2ac05fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:12:14.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.263+0000 7fc2b2999700 1 -- 172.21.15.27:0/2252101805 shutdown_connections 2021-11-03T17:12:14.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.263+0000 7fc2b2999700 1 -- 172.21.15.27:0/2252101805 wait complete. 2021-11-03T17:12:14.271 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2021-11-03T17:09:48.488849+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '24', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2021-11-03T17:11:52.008416+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '43', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2021-11-03T17:11:52.507226+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2021-11-03T17:12:14.271 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2021-11-03T17:12:14.391 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.396+0000 7feba91c7700 1 -- 172.21.15.27:0/1257453184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feba40f5580 msgr2=0x7feba40f59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:14.392 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.396+0000 7feba91c7700 1 --2- 172.21.15.27:0/1257453184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feba40f5580 0x7feba40f59a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7feb94004660 tx=0x7feb94009b30).stop 2021-11-03T17:12:14.392 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.396+0000 7feba91c7700 1 -- 172.21.15.27:0/1257453184 shutdown_connections 2021-11-03T17:12:14.392 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.396+0000 7feba91c7700 1 --2- 172.21.15.27:0/1257453184 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feba40f6820 0x7feba40f6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:14.392 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.396+0000 7feba91c7700 1 --2- 172.21.15.27:0/1257453184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feba40f5580 0x7feba40f59a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:14.393 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.396+0000 7feba91c7700 1 -- 172.21.15.27:0/1257453184 >> 172.21.15.27:0/1257453184 conn(0x7feba40f0ce0 msgr2=0x7feba40f3100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:12:14.393 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.396+0000 7feba91c7700 1 -- 172.21.15.27:0/1257453184 shutdown_connections 2021-11-03T17:12:14.393 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.396+0000 7feba91c7700 1 -- 172.21.15.27:0/1257453184 wait complete. 2021-11-03T17:12:14.396 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.397+0000 7feba91c7700 1 Processor -- start 2021-11-03T17:12:14.396 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.397+0000 7feba91c7700 1 -- start start 2021-11-03T17:12:14.397 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.397+0000 7feba91c7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feba40f5580 0x7feba405ff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:14.397 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.397+0000 7feba91c7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feba40f6820 0x7feba410e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:14.397 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.398+0000 7feba91c7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba4107a00 con 0x7feba40f6820 2021-11-03T17:12:14.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.398+0000 7feba91c7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba410e510 con 0x7feba40f5580 2021-11-03T17:12:14.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.398+0000 7feba259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feba40f6820 0x7feba410e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:14.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.398+0000 7feba259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feba40f6820 0x7feba410e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55060/0 (socket says 172.21.15.27:55060) 2021-11-03T17:12:14.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.398+0000 7feba259c700 1 -- 172.21.15.27:0/3571299442 learned_addr learned my addr 172.21.15.27:0/3571299442 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:12:14.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.398+0000 7feba2d9d700 1 --2- 172.21.15.27:0/3571299442 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feba40f5580 0x7feba405ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:14.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.398+0000 7feba259c700 1 -- 172.21.15.27:0/3571299442 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feba40f5580 msgr2=0x7feba405ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:14.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.398+0000 7feba259c700 1 --2- 172.21.15.27:0/3571299442 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feba40f5580 0x7feba405ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:14.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.398+0000 7feba259c700 1 -- 172.21.15.27:0/3571299442 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb94005040 con 0x7feba40f6820 2021-11-03T17:12:14.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.398+0000 7feba259c700 1 --2- 172.21.15.27:0/3571299442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feba40f6820 0x7feba410e020 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7feb9800e7f0 tx=0x7feb9800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:12:14.400 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.398+0000 7feb8b7fe700 1 -- 172.21.15.27:0/3571299442 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb980092e0 con 0x7feba40f6820 2021-11-03T17:12:14.400 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.398+0000 7feb8b7fe700 1 -- 172.21.15.27:0/3571299442 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7feb9800f040 con 0x7feba40f6820 2021-11-03T17:12:14.400 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.398+0000 7feba91c7700 1 -- 172.21.15.27:0/3571299442 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feba410e6b0 con 0x7feba40f6820 2021-11-03T17:12:14.400 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.398+0000 7feb8b7fe700 1 -- 172.21.15.27:0/3571299442 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb980146c0 con 0x7feba40f6820 2021-11-03T17:12:14.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.398+0000 7feba91c7700 1 -- 172.21.15.27:0/3571299442 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feba410ec40 con 0x7feba40f6820 2021-11-03T17:12:14.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.399+0000 7feba91c7700 1 -- 172.21.15.27:0/3571299442 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feba4046330 con 0x7feba40f6820 2021-11-03T17:12:14.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.400+0000 7feb8b7fe700 1 -- 172.21.15.27:0/3571299442 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7feb9801b030 con 0x7feba40f6820 2021-11-03T17:12:14.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.400+0000 7feb8b7fe700 1 --2- 172.21.15.27:0/3571299442 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7feb8c05c2a0 0x7feb8c05e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:14.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.400+0000 7feb8b7fe700 1 -- 172.21.15.27:0/3571299442 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5349+0+0 (secure 0 0 0) 0x7feb98075b70 con 0x7feba40f6820 2021-11-03T17:12:14.402 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.400+0000 7feba2d9d700 1 --2- 172.21.15.27:0/3571299442 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7feb8c05c2a0 0x7feb8c05e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:14.402 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.401+0000 7feba2d9d700 1 --2- 172.21.15.27:0/3571299442 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7feb8c05c2a0 0x7feb8c05e760 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7feb94004d80 tx=0x7feb94016040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:12:14.402 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.404+0000 7feb8b7fe700 1 -- 172.21.15.27:0/3571299442 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7feb98048900 con 0x7feba40f6820 2021-11-03T17:12:14.578 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.582+0000 7feba91c7700 1 -- 172.21.15.27:0/3571299442 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7feba4027a00 con 0x7feba40f6820 2021-11-03T17:12:14.578 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.583+0000 7feb8b7fe700 1 -- 172.21.15.27:0/3571299442 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v44) v1 ==== 110+0+10 (secure 0 0 0) 0x7feb98019020 con 0x7feba40f6820 2021-11-03T17:12:14.579 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 1 2021-11-03T17:12:14.580 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.584+0000 7feba91c7700 1 -- 172.21.15.27:0/3571299442 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7feb8c05c2a0 msgr2=0x7feb8c05e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:14.580 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.584+0000 7feba91c7700 1 --2- 172.21.15.27:0/3571299442 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7feb8c05c2a0 0x7feb8c05e760 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7feb94004d80 tx=0x7feb94016040).stop 2021-11-03T17:12:14.580 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.584+0000 7feba91c7700 1 -- 172.21.15.27:0/3571299442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feba40f6820 msgr2=0x7feba410e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:14.581 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.584+0000 7feba91c7700 1 --2- 172.21.15.27:0/3571299442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feba40f6820 0x7feba410e020 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7feb9800e7f0 tx=0x7feb9800b040).stop 2021-11-03T17:12:14.581 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.584+0000 7feba91c7700 1 -- 172.21.15.27:0/3571299442 shutdown_connections 2021-11-03T17:12:14.581 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.584+0000 7feba91c7700 1 --2- 172.21.15.27:0/3571299442 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7feb8c05c2a0 0x7feb8c05e760 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:14.581 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.584+0000 7feba91c7700 1 --2- 172.21.15.27:0/3571299442 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feba40f5580 0x7feba405ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:14.581 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.584+0000 7feba91c7700 1 --2- 172.21.15.27:0/3571299442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feba40f6820 0x7feba410e020 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:14.582 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.584+0000 7feba91c7700 1 -- 172.21.15.27:0/3571299442 >> 172.21.15.27:0/3571299442 conn(0x7feba40f0ce0 msgr2=0x7feba40fa270 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:12:14.582 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.585+0000 7feba91c7700 1 -- 172.21.15.27:0/3571299442 shutdown_connections 2021-11-03T17:12:14.582 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.585+0000 7feba91c7700 1 -- 172.21.15.27:0/3571299442 wait complete. 2021-11-03T17:12:14.590 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:14 smithi027 conmon[32322]: audit 2021-11-03T17:12:14.261601+0000 mon.smithi115 (mon.1) 2021-11-03T17:12:14.590 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:14 smithi027 conmon[32322]: 16 : audit [DBG] from='client.? 172.21.15.27:0/2252101805' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2021-11-03T17:12:14.592 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2021-11-03T17:12:14.721 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.724+0000 7feacf9b9700 1 -- 172.21.15.27:0/2101332379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feac80f7b40 msgr2=0x7feac80f7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:14.721 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.724+0000 7feacf9b9700 1 --2- 172.21.15.27:0/2101332379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feac80f7b40 0x7feac80f7fa0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7feab8004660 tx=0x7feab8009b30).stop 2021-11-03T17:12:14.722 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.724+0000 7feacf9b9700 1 -- 172.21.15.27:0/2101332379 shutdown_connections 2021-11-03T17:12:14.722 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.724+0000 7feacf9b9700 1 --2- 172.21.15.27:0/2101332379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feac80f7b40 0x7feac80f7fa0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:14.722 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.724+0000 7feacf9b9700 1 --2- 172.21.15.27:0/2101332379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feac80f68a0 0x7feac80f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:14.722 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.724+0000 7feacf9b9700 1 -- 172.21.15.27:0/2101332379 >> 172.21.15.27:0/2101332379 conn(0x7feac80f1fc0 msgr2=0x7feac80f4420 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:12:14.723 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.724+0000 7feacf9b9700 1 -- 172.21.15.27:0/2101332379 shutdown_connections 2021-11-03T17:12:14.723 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.724+0000 7feacf9b9700 1 -- 172.21.15.27:0/2101332379 wait complete. 2021-11-03T17:12:14.723 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.725+0000 7feacf9b9700 1 Processor -- start 2021-11-03T17:12:14.723 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.725+0000 7feacf9b9700 1 -- start start 2021-11-03T17:12:14.723 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.725+0000 7feacf9b9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feac80f68a0 0x7feac806b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:14.724 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.725+0000 7feacf9b9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feac80f7b40 0x7feac8069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:14.724 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.725+0000 7feacf9b9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac806bd70 con 0x7feac80f7b40 2021-11-03T17:12:14.724 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.725+0000 7feacf9b9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac806a310 con 0x7feac80f68a0 2021-11-03T17:12:14.724 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.725+0000 7feaccf54700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feac80f7b40 0x7feac8069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:14.725 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.725+0000 7feaccf54700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feac80f7b40 0x7feac8069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55070/0 (socket says 172.21.15.27:55070) 2021-11-03T17:12:14.725 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.725+0000 7feaccf54700 1 -- 172.21.15.27:0/4264537522 learned_addr learned my addr 172.21.15.27:0/4264537522 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:12:14.725 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.725+0000 7feacd755700 1 --2- 172.21.15.27:0/4264537522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feac80f68a0 0x7feac806b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:14.725 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.725+0000 7feaccf54700 1 -- 172.21.15.27:0/4264537522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feac80f68a0 msgr2=0x7feac806b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:14.725 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.725+0000 7feaccf54700 1 --2- 172.21.15.27:0/4264537522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feac80f68a0 0x7feac806b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:14.726 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.725+0000 7feaccf54700 1 -- 172.21.15.27:0/4264537522 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab8005040 con 0x7feac80f7b40 2021-11-03T17:12:14.726 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.726+0000 7feaccf54700 1 --2- 172.21.15.27:0/4264537522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feac80f7b40 0x7feac8069e20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7feab8000c00 tx=0x7feab800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:12:14.726 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.726+0000 7feabdffb700 1 -- 172.21.15.27:0/4264537522 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feab8019070 con 0x7feac80f7b40 2021-11-03T17:12:14.726 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.726+0000 7feacf9b9700 1 -- 172.21.15.27:0/4264537522 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feac806a530 con 0x7feac80f7b40 2021-11-03T17:12:14.727 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.726+0000 7feabdffb700 1 -- 172.21.15.27:0/4264537522 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7feab8007e50 con 0x7feac80f7b40 2021-11-03T17:12:14.727 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.726+0000 7feabdffb700 1 -- 172.21.15.27:0/4264537522 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feab8014cd0 con 0x7feac80f7b40 2021-11-03T17:12:14.727 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.726+0000 7feacf9b9700 1 -- 172.21.15.27:0/4264537522 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac806aa90 con 0x7feac80f7b40 2021-11-03T17:12:14.728 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.727+0000 7feacf9b9700 1 -- 172.21.15.27:0/4264537522 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feac805ea10 con 0x7feac80f7b40 2021-11-03T17:12:14.728 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.728+0000 7feabdffb700 1 -- 172.21.15.27:0/4264537522 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7feab8025070 con 0x7feac80f7b40 2021-11-03T17:12:14.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.728+0000 7feabdffb700 1 --2- 172.21.15.27:0/4264537522 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7feab405c2a0 0x7feab405e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:14.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.728+0000 7feabdffb700 1 -- 172.21.15.27:0/4264537522 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5349+0+0 (secure 0 0 0) 0x7feab8076e40 con 0x7feac80f7b40 2021-11-03T17:12:14.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.728+0000 7feacd755700 1 --2- 172.21.15.27:0/4264537522 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7feab405c2a0 0x7feab405e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:14.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.729+0000 7feacd755700 1 --2- 172.21.15.27:0/4264537522 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7feab405c2a0 0x7feab405e760 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7feac4009fd0 tx=0x7feac400b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:12:14.730 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.733+0000 7feabdffb700 1 -- 172.21.15.27:0/4264537522 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7feab8049a80 con 0x7feac80f7b40 2021-11-03T17:12:14.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:14 smithi115 conmon[34670]: audit 2021-11-03T17:12:14.261601+0000 mon.smithi115 (mon.1) 16 : audit [DBG] from='client.? 172.21.15.27:0/2252101805' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2021-11-03T17:12:14.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.901+0000 7feacf9b9700 1 -- 172.21.15.27:0/4264537522 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7feac8027a00 con 0x7feac80f7b40 2021-11-03T17:12:14.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.902+0000 7feabdffb700 1 -- 172.21.15.27:0/4264537522 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v44) v1 ==== 107+0+11 (secure 0 0 0) 0x7feab8023030 con 0x7feac80f7b40 2021-11-03T17:12:14.897 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2021-11-03T17:12:14.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.904+0000 7feacf9b9700 1 -- 172.21.15.27:0/4264537522 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7feab405c2a0 msgr2=0x7feab405e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:14.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.904+0000 7feacf9b9700 1 --2- 172.21.15.27:0/4264537522 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7feab405c2a0 0x7feab405e760 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7feac4009fd0 tx=0x7feac400b040).stop 2021-11-03T17:12:14.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.904+0000 7feacf9b9700 1 -- 172.21.15.27:0/4264537522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feac80f7b40 msgr2=0x7feac8069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:14.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.904+0000 7feacf9b9700 1 --2- 172.21.15.27:0/4264537522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feac80f7b40 0x7feac8069e20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7feab8000c00 tx=0x7feab800b040).stop 2021-11-03T17:12:14.900 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.904+0000 7feacf9b9700 1 -- 172.21.15.27:0/4264537522 shutdown_connections 2021-11-03T17:12:14.900 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.904+0000 7feacf9b9700 1 --2- 172.21.15.27:0/4264537522 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7feab405c2a0 0x7feab405e760 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:14.900 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.904+0000 7feacf9b9700 1 --2- 172.21.15.27:0/4264537522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feac80f68a0 0x7feac806b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:14.900 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.904+0000 7feacf9b9700 1 --2- 172.21.15.27:0/4264537522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feac80f7b40 0x7feac8069e20 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:14.900 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.904+0000 7feacf9b9700 1 -- 172.21.15.27:0/4264537522 >> 172.21.15.27:0/4264537522 conn(0x7feac80f1fc0 msgr2=0x7feac80fb590 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:12:14.901 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.904+0000 7feacf9b9700 1 -- 172.21.15.27:0/4264537522 shutdown_connections 2021-11-03T17:12:14.901 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:14.904+0000 7feacf9b9700 1 -- 172.21.15.27:0/4264537522 wait complete. 2021-11-03T17:12:14.910 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2021-11-03T17:12:15.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.038+0000 7f0e4359e700 1 -- 172.21.15.27:0/1458205410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e440f56b0 msgr2=0x7f0e440f5b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:15.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.038+0000 7f0e4359e700 1 --2- 172.21.15.27:0/1458205410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e440f56b0 0x7f0e440f5b30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f0e38004660 tx=0x7f0e38009b30).stop 2021-11-03T17:12:15.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.038+0000 7f0e4359e700 1 -- 172.21.15.27:0/1458205410 shutdown_connections 2021-11-03T17:12:15.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.038+0000 7f0e4359e700 1 --2- 172.21.15.27:0/1458205410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e440f56b0 0x7f0e440f5b30 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:15.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.038+0000 7f0e4359e700 1 --2- 172.21.15.27:0/1458205410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e440f69b0 0x7f0e440f6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:15.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.038+0000 7f0e4359e700 1 -- 172.21.15.27:0/1458205410 >> 172.21.15.27:0/1458205410 conn(0x7f0e440f02c0 msgr2=0x7f0e440f26e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:12:15.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.038+0000 7f0e4359e700 1 -- 172.21.15.27:0/1458205410 shutdown_connections 2021-11-03T17:12:15.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.038+0000 7f0e4359e700 1 -- 172.21.15.27:0/1458205410 wait complete. 2021-11-03T17:12:15.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.039+0000 7f0e4359e700 1 Processor -- start 2021-11-03T17:12:15.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.039+0000 7f0e4359e700 1 -- start start 2021-11-03T17:12:15.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.039+0000 7f0e4359e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e440f56b0 0x7f0e44109aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:15.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.039+0000 7f0e4359e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e440f69b0 0x7f0e4410bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:15.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.039+0000 7f0e4359e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e440f5340 con 0x7f0e440f56b0 2021-11-03T17:12:15.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.039+0000 7f0e4359e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e440f5480 con 0x7f0e440f69b0 2021-11-03T17:12:15.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.039+0000 7f0e4259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e440f56b0 0x7f0e44109aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:15.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.039+0000 7f0e4259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e440f56b0 0x7f0e44109aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55078/0 (socket says 172.21.15.27:55078) 2021-11-03T17:12:15.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.039+0000 7f0e4259c700 1 -- 172.21.15.27:0/3583239969 learned_addr learned my addr 172.21.15.27:0/3583239969 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:12:15.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.039+0000 7f0e41d9b700 1 --2- 172.21.15.27:0/3583239969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e440f69b0 0x7f0e4410bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:15.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.039+0000 7f0e4259c700 1 -- 172.21.15.27:0/3583239969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e440f69b0 msgr2=0x7f0e4410bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:15.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.039+0000 7f0e4259c700 1 --2- 172.21.15.27:0/3583239969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e440f69b0 0x7f0e4410bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:15.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.039+0000 7f0e4259c700 1 -- 172.21.15.27:0/3583239969 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e38005040 con 0x7f0e440f56b0 2021-11-03T17:12:15.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.040+0000 7f0e4259c700 1 --2- 172.21.15.27:0/3583239969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e440f56b0 0x7f0e44109aa0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f0e3400c790 tx=0x7f0e34007d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:12:15.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.040+0000 7f0e32ffd700 1 -- 172.21.15.27:0/3583239969 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e34013630 con 0x7f0e440f56b0 2021-11-03T17:12:15.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.040+0000 7f0e32ffd700 1 -- 172.21.15.27:0/3583239969 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f0e34012840 con 0x7f0e440f56b0 2021-11-03T17:12:15.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.040+0000 7f0e4359e700 1 -- 172.21.15.27:0/3583239969 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e4410c550 con 0x7f0e440f56b0 2021-11-03T17:12:15.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.040+0000 7f0e32ffd700 1 -- 172.21.15.27:0/3583239969 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e34002760 con 0x7f0e440f56b0 2021-11-03T17:12:15.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.040+0000 7f0e4359e700 1 -- 172.21.15.27:0/3583239969 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e4410cae0 con 0x7f0e440f56b0 2021-11-03T17:12:15.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.041+0000 7f0e4359e700 1 -- 172.21.15.27:0/3583239969 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e44045ad0 con 0x7f0e440f56b0 2021-11-03T17:12:15.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.041+0000 7f0e32ffd700 1 -- 172.21.15.27:0/3583239969 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f0e34010070 con 0x7f0e440f56b0 2021-11-03T17:12:15.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.041+0000 7f0e32ffd700 1 --2- 172.21.15.27:0/3583239969 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f0e2c05c2b0 0x7f0e2c05e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:15.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.042+0000 7f0e32ffd700 1 -- 172.21.15.27:0/3583239969 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f0e34075fd0 con 0x7f0e440f56b0 2021-11-03T17:12:15.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.042+0000 7f0e41d9b700 1 --2- 172.21.15.27:0/3583239969 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f0e2c05c2b0 0x7f0e2c05e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:15.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.042+0000 7f0e41d9b700 1 --2- 172.21.15.27:0/3583239969 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f0e2c05c2b0 0x7f0e2c05e770 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f0e38004630 tx=0x7f0e38009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:12:15.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.046+0000 7f0e32ffd700 1 -- 172.21.15.27:0/3583239969 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f0e34048c50 con 0x7f0e440f56b0 2021-11-03T17:12:15.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.205+0000 7f0e4359e700 1 -- 172.21.15.27:0/3583239969 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f0e44027a00 con 0x7f0e440f56b0 2021-11-03T17:12:15.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.206+0000 7f0e32ffd700 1 -- 172.21.15.27:0/3583239969 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v44) v1 ==== 107+0+11 (secure 0 0 0) 0x7f0e3404c270 con 0x7f0e440f56b0 2021-11-03T17:12:15.202 INFO:teuthology.orchestra.run.smithi027.stdout:pg_num: 32 2021-11-03T17:12:15.203 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.207+0000 7f0e4359e700 1 -- 172.21.15.27:0/3583239969 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f0e2c05c2b0 msgr2=0x7f0e2c05e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:15.203 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.207+0000 7f0e4359e700 1 --2- 172.21.15.27:0/3583239969 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f0e2c05c2b0 0x7f0e2c05e770 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f0e38004630 tx=0x7f0e38009b30).stop 2021-11-03T17:12:15.204 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.207+0000 7f0e4359e700 1 -- 172.21.15.27:0/3583239969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e440f56b0 msgr2=0x7f0e44109aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:15.204 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.207+0000 7f0e4359e700 1 --2- 172.21.15.27:0/3583239969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e440f56b0 0x7f0e44109aa0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f0e3400c790 tx=0x7f0e34007d90).stop 2021-11-03T17:12:15.204 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.208+0000 7f0e4359e700 1 -- 172.21.15.27:0/3583239969 shutdown_connections 2021-11-03T17:12:15.204 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.208+0000 7f0e4359e700 1 --2- 172.21.15.27:0/3583239969 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f0e2c05c2b0 0x7f0e2c05e770 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:15.204 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.208+0000 7f0e4359e700 1 --2- 172.21.15.27:0/3583239969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e440f56b0 0x7f0e44109aa0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:15.205 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.208+0000 7f0e4359e700 1 --2- 172.21.15.27:0/3583239969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e440f69b0 0x7f0e4410bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:15.205 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.208+0000 7f0e4359e700 1 -- 172.21.15.27:0/3583239969 >> 172.21.15.27:0/3583239969 conn(0x7f0e440f02c0 msgr2=0x7f0e440fa110 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:12:15.205 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.208+0000 7f0e4359e700 1 -- 172.21.15.27:0/3583239969 shutdown_connections 2021-11-03T17:12:15.205 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.208+0000 7f0e4359e700 1 -- 172.21.15.27:0/3583239969 wait complete. 2021-11-03T17:12:15.214 DEBUG:teuthology.orchestra.run.smithi027:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2021-11-03T17:12:15.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.337+0000 7f5674869700 1 -- 172.21.15.27:0/1914051814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56700f6840 msgr2=0x7f56700f6ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:15.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.337+0000 7f5674869700 1 --2- 172.21.15.27:0/1914051814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56700f6840 0x7f56700f6ca0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f5664004660 tx=0x7f5664009b30).stop 2021-11-03T17:12:15.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.337+0000 7f5674869700 1 -- 172.21.15.27:0/1914051814 shutdown_connections 2021-11-03T17:12:15.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.337+0000 7f5674869700 1 --2- 172.21.15.27:0/1914051814 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56700f6840 0x7f56700f6ca0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:15.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.337+0000 7f5674869700 1 --2- 172.21.15.27:0/1914051814 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56700f55a0 0x7f56700f59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:15.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.337+0000 7f5674869700 1 -- 172.21.15.27:0/1914051814 >> 172.21.15.27:0/1914051814 conn(0x7f56700f0ce0 msgr2=0x7f56700f3120 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:12:15.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.337+0000 7f5674869700 1 -- 172.21.15.27:0/1914051814 shutdown_connections 2021-11-03T17:12:15.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.337+0000 7f5674869700 1 -- 172.21.15.27:0/1914051814 wait complete. 2021-11-03T17:12:15.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.337+0000 7f5674869700 1 Processor -- start 2021-11-03T17:12:15.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.338+0000 7f5674869700 1 -- start start 2021-11-03T17:12:15.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.338+0000 7f5674869700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56700f55a0 0x7f5670105da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:15.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.338+0000 7f5674869700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56700f6840 0x7f5670106290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:15.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.338+0000 7f5674869700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5670106890 con 0x7f56700f6840 2021-11-03T17:12:15.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.338+0000 7f5674869700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56701069d0 con 0x7f56700f55a0 2021-11-03T17:12:15.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.338+0000 7f566dd9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56700f6840 0x7f5670106290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:15.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.338+0000 7f566dd9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56700f6840 0x7f5670106290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55090/0 (socket says 172.21.15.27:55090) 2021-11-03T17:12:15.344 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.338+0000 7f566dd9b700 1 -- 172.21.15.27:0/700553309 learned_addr learned my addr 172.21.15.27:0/700553309 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:12:15.344 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.338+0000 7f566e59c700 1 --2- 172.21.15.27:0/700553309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56700f55a0 0x7f5670105da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:15.344 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.338+0000 7f566dd9b700 1 -- 172.21.15.27:0/700553309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56700f55a0 msgr2=0x7f5670105da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:15.345 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.338+0000 7f566dd9b700 1 --2- 172.21.15.27:0/700553309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56700f55a0 0x7f5670105da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:15.345 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.338+0000 7f566dd9b700 1 -- 172.21.15.27:0/700553309 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5664005040 con 0x7f56700f6840 2021-11-03T17:12:15.345 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.338+0000 7f566dd9b700 1 --2- 172.21.15.27:0/700553309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56700f6840 0x7f5670106290 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f5664009990 tx=0x7f5664009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:12:15.345 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.338+0000 7f565effd700 1 -- 172.21.15.27:0/700553309 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5664018070 con 0x7f56700f6840 2021-11-03T17:12:15.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.338+0000 7f565effd700 1 -- 172.21.15.27:0/700553309 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f5664013410 con 0x7f56700f6840 2021-11-03T17:12:15.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.338+0000 7f5674869700 1 -- 172.21.15.27:0/700553309 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f567010b380 con 0x7f56700f6840 2021-11-03T17:12:15.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.338+0000 7f565effd700 1 -- 172.21.15.27:0/700553309 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f566401de80 con 0x7f56700f6840 2021-11-03T17:12:15.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.338+0000 7f5674869700 1 -- 172.21.15.27:0/700553309 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f567010b8e0 con 0x7f56700f6840 2021-11-03T17:12:15.347 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.339+0000 7f5674869700 1 -- 172.21.15.27:0/700553309 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5670027a00 con 0x7f56700f6840 2021-11-03T17:12:15.347 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.340+0000 7f565effd700 1 -- 172.21.15.27:0/700553309 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f566401d420 con 0x7f56700f6840 2021-11-03T17:12:15.347 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.340+0000 7f565effd700 1 --2- 172.21.15.27:0/700553309 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f565805c2a0 0x7f565805e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:15.347 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.340+0000 7f565effd700 1 -- 172.21.15.27:0/700553309 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f56640779e0 con 0x7f56700f6840 2021-11-03T17:12:15.348 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.340+0000 7f566e59c700 1 --2- 172.21.15.27:0/700553309 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f565805c2a0 0x7f565805e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:15.348 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.340+0000 7f566e59c700 1 --2- 172.21.15.27:0/700553309 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f565805c2a0 0x7f565805e760 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f5670107290 tx=0x7f566000b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:12:15.348 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.345+0000 7f565effd700 1 -- 172.21.15.27:0/700553309 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f566404a740 con 0x7f56700f6840 2021-11-03T17:12:15.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.562+0000 7f5674869700 1 -- 172.21.15.27:0/700553309 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f56700fdd60 con 0x7f56700f6840 2021-11-03T17:12:15.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.563+0000 7f565effd700 1 -- 172.21.15.27:0/700553309 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+2944 (secure 0 0 0) 0x7f566404dd60 con 0x7f56700f6840 2021-11-03T17:12:15.559 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:12:15.560 INFO:teuthology.orchestra.run.smithi027.stdout:{"epoch":13,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14448,"name":"cephfs.smithi027.fksixo","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.27:6829/1337044739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6828","nonce":1337044739},{"type":"v1","addr":"172.21.15.27:6829","nonce":1337044739}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":13},{"gid":24289,"name":"cephfs.smithi115.semywj","rank":-1,"incarnation":0,"state":"up:standby","state_seq":3,"addr":"172.21.15.115:6827/3367380088","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6826","nonce":3367380088},{"type":"v1","addr":"172.21.15.115:6827","nonce":3367380088}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0,"epoch":12}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2021-11-03T17:11:53.078170+0000","modified":"2021-11-03T17:12:05.026191+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":24273,"mds_1":14436},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14436":{"gid":14436,"name":"cephfs.smithi027.jdwohn","rank":1,"incarnation":8,"state":"up:active","state_seq":3,"addr":"172.21.15.27:6827/536348060","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.27:6826","nonce":536348060},{"type":"v1","addr":"172.21.15.27:6827","nonce":536348060}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0},"gid_24273":{"gid":24273,"name":"cephfs.smithi115.zsfgnv","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.115:6825/1621245036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.115:6824","nonce":1621245036},{"type":"v1","addr":"172.21.15.115:6825","nonce":1621245036}]},"join_fscid":1,"export_targets":[],"features":4540138297136906239,"flags":0}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":1},"id":1}]} 2021-11-03T17:12:15.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.564+0000 7f5674869700 1 -- 172.21.15.27:0/700553309 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f565805c2a0 msgr2=0x7f565805e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:15.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.564+0000 7f5674869700 1 --2- 172.21.15.27:0/700553309 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f565805c2a0 0x7f565805e760 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f5670107290 tx=0x7f566000b040).stop 2021-11-03T17:12:15.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.564+0000 7f5674869700 1 -- 172.21.15.27:0/700553309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56700f6840 msgr2=0x7f5670106290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:15.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.564+0000 7f5674869700 1 --2- 172.21.15.27:0/700553309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56700f6840 0x7f5670106290 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f5664009990 tx=0x7f5664009b30).stop 2021-11-03T17:12:15.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.565+0000 7f5674869700 1 -- 172.21.15.27:0/700553309 shutdown_connections 2021-11-03T17:12:15.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.565+0000 7f5674869700 1 --2- 172.21.15.27:0/700553309 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f565805c2a0 0x7f565805e760 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:15.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.565+0000 7f5674869700 1 --2- 172.21.15.27:0/700553309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56700f55a0 0x7f5670105da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:15.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.565+0000 7f5674869700 1 --2- 172.21.15.27:0/700553309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56700f6840 0x7f5670106290 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:15.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.565+0000 7f5674869700 1 -- 172.21.15.27:0/700553309 >> 172.21.15.27:0/700553309 conn(0x7f56700f0ce0 msgr2=0x7f56700fa290 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:12:15.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.565+0000 7f5674869700 1 -- 172.21.15.27:0/700553309 shutdown_connections 2021-11-03T17:12:15.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:15.565+0000 7f5674869700 1 -- 172.21.15.27:0/700553309 wait complete. 2021-11-03T17:12:15.564 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 13 2021-11-03T17:12:15.571 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 9, 'max_mds': 2, 'flags': 18} 2021-11-03T17:12:15.572 INFO:teuthology.run_tasks:Running task kclient... 2021-11-03T17:12:15.596 INFO:tasks.kclient:Mounting kernel clients... 2021-11-03T17:12:15.597 INFO:tasks.kclient:config is {'client.0': {}, 'client.1': {}} 2021-11-03T17:12:15.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-03T17:12:15.598 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2021-11-03T17:12:15.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-03T17:12:15.615 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link delete ceph-brx 2021-11-03T17:12:15.681 INFO:teuthology.orchestra.run.smithi027.stderr:Cannot find device "ceph-brx" 2021-11-03T17:12:15.682 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-03T17:12:15.683 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-03T17:12:15.684 DEBUG:teuthology.orchestra.run.smithi115:> ip netns list 2021-11-03T17:12:15.704 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-03T17:12:15.704 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link delete ceph-brx 2021-11-03T17:12:15.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:15 smithi115 conmon[34670]: cluster 2021-11-03T17:12:14.482056+0000 mon.smithi027 (mon.0) 536 : cluster [DBG] mds.? [v2:172.21.15.27:6828/1337044739,v1:172.21.15.27:6829/1337044739] up:standby 2021-11-03T17:12:15.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:15 smithi115 conmon[34670]: cluster 2021-11-03T17:12:14.482124+0000 2021-11-03T17:12:15.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:15 smithi115 conmon[34670]: mon.smithi027 (mon.0) 537 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi115.zsfgnv=up:active,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:12:15.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:15 smithi115 conmon[34670]: audit 2021-11-03T17:12:14.510550+0000 mon.smithi027 (mon.0) 538 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:12:15.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:15 smithi115 conmon[34670]: cluster 2021-11-03T17:12:14.577473+0000 mgr.smithi027.ufhhvb (mgr 2021-11-03T17:12:15.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:15 smithi115 conmon[34670]: .14162) 231 : cluster [DBG] pgmap v194: 65 pgs: 65 active+clean; 3.6 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s wr, 4 op/s 2021-11-03T17:12:15.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:15 smithi115 conmon[34670]: audit 2021-11-03 2021-11-03T17:12:15.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:15 smithi115 conmon[34670]: T17:12:14.583568+0000 mon.smithi027 (mon.0) 539 : audit 2021-11-03T17:12:15.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:15 smithi115 conmon[34670]: [DBG] from='client.? 172.21.15.27:0/3571299442' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2021-11-03T17:12:15.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:15 smithi115 conmon[34670]: audit 2021-11-03T17:12: 2021-11-03T17:12:15.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:15 smithi115 conmon[34670]: 14.902581+0000 mon.smithi027 (mon.0) 540 : 2021-11-03T17:12:15.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:15 smithi115 conmon[34670]: audit [DBG] from='client.? 172.21.15.27:0/4264537522' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2021-11-03T17:12:15.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:15 smithi115 conmon[34670]: audit 2021-11-03T17 2021-11-03T17:12:15.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:15 smithi115 conmon[34670]: :12:15.206727+0000 mon.smithi027 (mon.0 2021-11-03T17:12:15.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:15 smithi115 conmon[34670]: ) 541 : audit [DBG] from='client.? 172.21.15.27:0/3583239969' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2021-11-03T17:12:15.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:15 smithi027 conmon[32322]: cluster 2021-11-03T17:12:14.482056+0000 mon.smithi027 ( 2021-11-03T17:12:15.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:15 smithi027 conmon[32322]: mon.0) 536 : cluster [DBG] mds.? [v2:172.21.15.27:6828/1337044739,v1:172.21.15.27:6829/1337044739] up:standby 2021-11-03T17:12:15.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:15 smithi027 conmon[32322]: cluster 2021-11-03T17:12:14.482124+0000 mon.smithi027 (mon.0) 537 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi115.zsfgnv=up:active,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:12:15.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:15 smithi027 conmon[32322]: audit 2021-11-03T17:12:14.510550+0000 mon.smithi027 (mon. 2021-11-03T17:12:15.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:15 smithi027 conmon[32322]: 0) 538 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:12:15.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:15 smithi027 conmon[32322]: cluster 2021-11-03T17:12:15.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:15 smithi027 conmon[32322]: 2021-11-03T17:12:14. 2021-11-03T17:12:15.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:15 smithi027 conmon[32322]: 577473+0000 mgr.smithi027.ufhhvb (mgr.14162) 231 : cluster [DBG] 2021-11-03T17:12:15.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:15 smithi027 conmon[32322]: pgmap v194: 65 pgs: 65 active+clean; 3.6 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s wr, 4 op/s 2021-11-03T17:12:15.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:15 smithi027 conmon[32322]: audit 2021-11-03T17:12:14.583568+0000 mon.smithi027 (mon.0) 539 : audit [DBG] from='client.? 172.21.15.27:0/3571299442' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]: dispatch 2021-11-03T17:12:15.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:15 smithi027 conmon[32322]: 2021-11-03T17:12:15.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:15 smithi027 conmon[32322]: audit 2021- 2021-11-03T17:12:15.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:15 smithi027 conmon[32322]: 11-03T17:12:14.902581+0000 mon.smithi027 (mon.0) 540 : 2021-11-03T17:12:15.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:15 smithi027 conmon[32322]: audit [DBG] from='client.? 172.21.15.27:0/4264537522' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2021-11-03T17:12:15.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:15 smithi027 conmon[32322]: audit 2021-11-03T17:12:15.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:15 smithi027 conmon[32322]: 2021-11-03T 2021-11-03T17:12:15.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:15 smithi027 conmon[32322]: 17:12:15.206727+0000 mon.smithi027 (mon.0) 2021-11-03T17:12:15.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:15 smithi027 conmon[32322]: 541 : audit [DBG] from='client.? 172.21.15.27:0/3583239969' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2021-11-03T17:12:15.773 DEBUG:teuthology.orchestra.run:got remote process result: 1 2021-11-03T17:12:15.774 INFO:teuthology.orchestra.run.smithi115.stderr:Cannot find device "ceph-brx" 2021-11-03T17:12:15.775 INFO:tasks.kclient:client.0 config is {} 2021-11-03T17:12:15.775 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph fs ls 2021-11-03T17:12:16.122 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c/mon.smithi027/config 2021-11-03T17:12:16.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:16 smithi027 conmon[32322]: audit 2021-11-03T17:12:15.563421+0000 mon.smithi027 (mon. 2021-11-03T17:12:16.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:16 smithi027 conmon[32322]: 0) 542 : audit [DBG] from='client.? 172.21.15.27:0/700553309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2021-11-03T17:12:16.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:16 smithi115 conmon[34670]: audit 2021-11-03T17:12:15.563421+0000 mon.smithi027 (mon.0) 542 : audit [DBG] from='client.? 172.21.15.27:0/700553309' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2021-11-03T17:12:17.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:17 smithi027 conmon[32322]: cluster 2021-11-03T17:12:16.577931+0000 mgr.smithi027.ufhhvb (mgr.14162) 232 : cluster [DBG] pgmap v195: 65 pgs: 65 active+clean; 3.6 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 3 op/s 2021-11-03T17:12:17.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:17 smithi115 conmon[34670]: cluster 2021-11-03T17:12:16.577931+0000 mgr.smithi027.ufhhvb (mgr.14162) 232 : cluster [DBG] pgmap v195: 65 pgs: 65 active+clean; 3.6 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 3 op/s 2021-11-03T17:12:17.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:17.996+0000 7f62f514a700 1 -- 172.21.15.27:0/4022161138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62f00f7a60 msgr2=0x7f62f00f7ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:17.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:17.996+0000 7f62f514a700 1 --2- 172.21.15.27:0/4022161138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62f00f7a60 0x7f62f00f7ec0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f62e4004660 tx=0x7f62e4009b30).stop 2021-11-03T17:12:17.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:17.996+0000 7f62f514a700 1 -- 172.21.15.27:0/4022161138 shutdown_connections 2021-11-03T17:12:17.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:17.996+0000 7f62f514a700 1 --2- 172.21.15.27:0/4022161138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62f00f7a60 0x7f62f00f7ec0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:17.992 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:17.996+0000 7f62f514a700 1 --2- 172.21.15.27:0/4022161138 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62f00f67c0 0x7f62f00f6be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:17.992 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:17.996+0000 7f62f514a700 1 -- 172.21.15.27:0/4022161138 >> 172.21.15.27:0/4022161138 conn(0x7f62f00f1f20 msgr2=0x7f62f00f4340 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:12:17.992 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:17.996+0000 7f62f514a700 1 -- 172.21.15.27:0/4022161138 shutdown_connections 2021-11-03T17:12:17.992 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:17.996+0000 7f62f514a700 1 -- 172.21.15.27:0/4022161138 wait complete. 2021-11-03T17:12:17.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:17.997+0000 7f62f514a700 1 Processor -- start 2021-11-03T17:12:17.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:17.997+0000 7f62f514a700 1 -- start start 2021-11-03T17:12:17.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:17.997+0000 7f62f514a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62f00f67c0 0x7f62f0106fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:17.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:17.997+0000 7f62f514a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62f00f7a60 0x7f62f01074d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:17.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:17.997+0000 7f62f514a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f0107a80 con 0x7f62f00f7a60 2021-11-03T17:12:17.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:17.997+0000 7f62f514a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62f0107bc0 con 0x7f62f00f67c0 2021-11-03T17:12:17.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:17.997+0000 7f62ee59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62f00f7a60 0x7f62f01074d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:17.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:17.997+0000 7f62ee59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62f00f7a60 0x7f62f01074d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55100/0 (socket says 172.21.15.27:55100) 2021-11-03T17:12:17.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:17.997+0000 7f62ee59c700 1 -- 172.21.15.27:0/3240867106 learned_addr learned my addr 172.21.15.27:0/3240867106 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:12:17.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:17.997+0000 7f62eed9d700 1 --2- 172.21.15.27:0/3240867106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62f00f67c0 0x7f62f0106fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:17.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:17.997+0000 7f62ee59c700 1 -- 172.21.15.27:0/3240867106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62f00f67c0 msgr2=0x7f62f0106fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:17.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:17.997+0000 7f62ee59c700 1 --2- 172.21.15.27:0/3240867106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62f00f67c0 0x7f62f0106fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:17.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:17.997+0000 7f62ee59c700 1 -- 172.21.15.27:0/3240867106 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e4005040 con 0x7f62f00f7a60 2021-11-03T17:12:17.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:17.997+0000 7f62ee59c700 1 --2- 172.21.15.27:0/3240867106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62f00f7a60 0x7f62f01074d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f62e4009990 tx=0x7f62e4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:12:17.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:17.999+0000 7f62d77fe700 1 -- 172.21.15.27:0/3240867106 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62e4018070 con 0x7f62f00f7a60 2021-11-03T17:12:17.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:17.999+0000 7f62f514a700 1 -- 172.21.15.27:0/3240867106 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62f0108500 con 0x7f62f00f7a60 2021-11-03T17:12:17.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:17.999+0000 7f62f514a700 1 -- 172.21.15.27:0/3240867106 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62f0108a60 con 0x7f62f00f7a60 2021-11-03T17:12:17.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:17.999+0000 7f62d77fe700 1 -- 172.21.15.27:0/3240867106 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7f62e4013410 con 0x7f62f00f7a60 2021-11-03T17:12:17.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:17.999+0000 7f62d77fe700 1 -- 172.21.15.27:0/3240867106 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f62e401dc90 con 0x7f62f00f7a60 2021-11-03T17:12:18.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:18.000+0000 7f62d77fe700 1 -- 172.21.15.27:0/3240867106 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f62e4027440 con 0x7f62f00f7a60 2021-11-03T17:12:18.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:18.000+0000 7f62d77fe700 1 --2- 172.21.15.27:0/3240867106 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f62d805c2a0 0x7f62d805e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:18.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:18.000+0000 7f62d77fe700 1 -- 172.21.15.27:0/3240867106 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f62e4077880 con 0x7f62f00f7a60 2021-11-03T17:12:18.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:18.000+0000 7f62eed9d700 1 --2- 172.21.15.27:0/3240867106 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f62d805c2a0 0x7f62d805e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:18.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:18.001+0000 7f62eed9d700 1 --2- 172.21.15.27:0/3240867106 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f62d805c2a0 0x7f62d805e760 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f62f00f78c0 tx=0x7f62e000b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:12:18.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:18.002+0000 7f62f514a700 1 -- 172.21.15.27:0/3240867106 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62d0004fa0 con 0x7f62f00f7a60 2021-11-03T17:12:18.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:18.006+0000 7f62d77fe700 1 -- 172.21.15.27:0/3240867106 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f62e404a500 con 0x7f62f00f7a60 2021-11-03T17:12:18.200 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:18.205+0000 7f62f514a700 1 -- 172.21.15.27:0/3240867106 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f62d0005e80 con 0x7f62f00f7a60 2021-11-03T17:12:18.200 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:18.205+0000 7f62d77fe700 1 -- 172.21.15.27:0/3240867106 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v13) v1 ==== 53+0+83 (secure 0 0 0) 0x7f62e404db20 con 0x7f62f00f7a60 2021-11-03T17:12:18.201 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2021-11-03T17:12:18.202 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:18.207+0000 7f62f514a700 1 -- 172.21.15.27:0/3240867106 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f62d805c2a0 msgr2=0x7f62d805e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:18.202 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:18.207+0000 7f62f514a700 1 --2- 172.21.15.27:0/3240867106 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f62d805c2a0 0x7f62d805e760 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f62f00f78c0 tx=0x7f62e000b040).stop 2021-11-03T17:12:18.202 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:18.207+0000 7f62f514a700 1 -- 172.21.15.27:0/3240867106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62f00f7a60 msgr2=0x7f62f01074d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:18.202 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:18.207+0000 7f62f514a700 1 --2- 172.21.15.27:0/3240867106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62f00f7a60 0x7f62f01074d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f62e4009990 tx=0x7f62e4009b30).stop 2021-11-03T17:12:18.203 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:18.207+0000 7f62f514a700 1 -- 172.21.15.27:0/3240867106 shutdown_connections 2021-11-03T17:12:18.203 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:18.207+0000 7f62f514a700 1 --2- 172.21.15.27:0/3240867106 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f62d805c2a0 0x7f62d805e760 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:18.203 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:18.207+0000 7f62f514a700 1 --2- 172.21.15.27:0/3240867106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62f00f67c0 0x7f62f0106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:18.203 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:18.207+0000 7f62f514a700 1 --2- 172.21.15.27:0/3240867106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62f00f7a60 0x7f62f01074d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:18.204 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:18.207+0000 7f62f514a700 1 -- 172.21.15.27:0/3240867106 >> 172.21.15.27:0/3240867106 conn(0x7f62f00f1f20 msgr2=0x7f62f00fb4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:12:18.204 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:18.207+0000 7f62f514a700 1 -- 172.21.15.27:0/3240867106 shutdown_connections 2021-11-03T17:12:18.204 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:18.207+0000 7f62f514a700 1 -- 172.21.15.27:0/3240867106 wait complete. 2021-11-03T17:12:18.831 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2021-11-03T17:12:18.832 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2021-11-03T17:12:18.832 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi027.front.sepia.ceph.com 2021-11-03T17:12:18.832 INFO:tasks.cephfs.mount:self.client.name = client.0 2021-11-03T17:12:18.833 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2021-11-03T17:12:18.833 INFO:tasks.cephfs.mount:self.cephfs_name = None 2021-11-03T17:12:18.833 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2021-11-03T17:12:18.833 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2021-11-03T17:12:18.834 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2021-11-03T17:12:18.834 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-03T17:12:18.835 DEBUG:teuthology.orchestra.run.smithi027:> ip addr 2021-11-03T17:12:18.851 INFO:teuthology.orchestra.run.smithi027.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2021-11-03T17:12:18.852 INFO:teuthology.orchestra.run.smithi027.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2021-11-03T17:12:18.852 INFO:teuthology.orchestra.run.smithi027.stdout: inet 127.0.0.1/8 scope host lo 2021-11-03T17:12:18.852 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2021-11-03T17:12:18.852 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 ::1/128 scope host 2021-11-03T17:12:18.853 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2021-11-03T17:12:18.853 INFO:teuthology.orchestra.run.smithi027.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2021-11-03T17:12:18.853 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c8 brd ff:ff:ff:ff:ff:ff 2021-11-03T17:12:18.853 INFO:teuthology.orchestra.run.smithi027.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2021-11-03T17:12:18.854 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:bd:16:06 brd ff:ff:ff:ff:ff:ff 2021-11-03T17:12:18.854 INFO:teuthology.orchestra.run.smithi027.stdout: inet 172.21.15.27/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2021-11-03T17:12:18.854 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft 42170sec preferred_lft 42170sec 2021-11-03T17:12:18.854 INFO:teuthology.orchestra.run.smithi027.stdout: inet6 fe80::ec4:7aff:febd:1606/64 scope link 2021-11-03T17:12:18.855 INFO:teuthology.orchestra.run.smithi027.stdout: valid_lft forever preferred_lft forever 2021-11-03T17:12:18.855 INFO:teuthology.orchestra.run.smithi027.stdout:4: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2021-11-03T17:12:18.855 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:6b:fe:c9 brd ff:ff:ff:ff:ff:ff 2021-11-03T17:12:18.855 INFO:teuthology.orchestra.run.smithi027.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2021-11-03T17:12:18.856 INFO:teuthology.orchestra.run.smithi027.stdout: link/ether 0c:c4:7a:bd:16:07 brd ff:ff:ff:ff:ff:ff 2021-11-03T17:12:18.856 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2021-11-03T17:12:18.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-03T17:12:18.857 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2021-11-03T17:12:18.858 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2021-11-03T17:12:18.858 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add name ceph-brx type bridge 2021-11-03T17:12:18.858 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr flush dev ceph-brx 2021-11-03T17:12:18.859 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set ceph-brx up 2021-11-03T17:12:18.859 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2021-11-03T17:12:18.860 DEBUG:teuthology.orchestra.run.smithi027:> ') 2021-11-03T17:12:19.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-03T17:12:19.020 DEBUG:teuthology.orchestra.run.smithi027:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2021-11-03T17:12:19.045 INFO:teuthology.orchestra.run.smithi027.stdout:1 2021-11-03T17:12:19.047 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-03T17:12:19.047 DEBUG:teuthology.orchestra.run.smithi027:> route 2021-11-03T17:12:19.117 INFO:teuthology.orchestra.run.smithi027.stdout:Kernel IP routing table 2021-11-03T17:12:19.117 INFO:teuthology.orchestra.run.smithi027.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2021-11-03T17:12:19.117 INFO:teuthology.orchestra.run.smithi027.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2021-11-03T17:12:19.117 INFO:teuthology.orchestra.run.smithi027.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2021-11-03T17:12:19.118 INFO:teuthology.orchestra.run.smithi027.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2021-11-03T17:12:19.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-03T17:12:19.119 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2021-11-03T17:12:19.119 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2021-11-03T17:12:19.119 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2021-11-03T17:12:19.120 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2021-11-03T17:12:19.120 DEBUG:teuthology.orchestra.run.smithi027:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2021-11-03T17:12:19.120 DEBUG:teuthology.orchestra.run.smithi027:> ') 2021-11-03T17:12:19.191 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:18 smithi027 conmon[32322]: audit 2021-11-03T17:12: 2021-11-03T17:12:19.192 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:18 smithi027 conmon[32322]: 17.995727+0000 mon.smithi027 (mon.0) 543 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:12:19.192 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:18 smithi027 conmon[32322]: audit 2021-11-03T17:12:17.999454+0000 mon.smithi027 (mon.0) 544 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:12:19.192 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:18 smithi027 conmon[32322]: audit 2021-11-03T17:12:18.206219+0000 mon.smithi027 (mon.0) 545 : audit [DBG] from='client.? 172.21.15.27:0/3240867106' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2021-11-03T17:12:19.193 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:18 smithi027 conmon[32322]: cluster 2021-11-03T17:12:18.578540+0000 mgr.smithi027.ufhhvb (mgr.14162) 233 : cluster [DBG] pgmap v196: 65 pgs: 65 active+clean; 3.6 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2021-11-03T17:12:19.365 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-03T17:12:19.365 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list 2021-11-03T17:12:19.420 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-03T17:12:19.421 DEBUG:teuthology.orchestra.run.smithi027:> ip netns list-id 2021-11-03T17:12:19.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-03T17:12:19.475 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2021-11-03T17:12:19.475 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2021-11-03T17:12:19.475 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2021-11-03T17:12:19.476 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2021-11-03T17:12:19.476 DEBUG:teuthology.orchestra.run.smithi027:> ') 2021-11-03T17:12:19.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:18 smithi115 conmon[34670]: audit 2021-11-03T17:12:17.995727+0000 mon.smithi027 (mon.0) 543 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:12:19.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:18 smithi115 conmon[34670]: 2021-11-03T17:12:19.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:18 smithi115 conmon[34670]: audit 2021-11-03T17:12:17.999454+0000 mon.smithi027 (mon.0) 544 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:12:19.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:18 smithi115 conmon[34670]: audit 2021-11-03T17:12:18.206219+0000 mon.smithi027 (mon.0) 545 : audit [DBG] from='client.? 172.21.15.27:0/3240867106' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2021-11-03T17:12:19.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:18 smithi115 conmon[34670]: cluster 2021-11-03T17:12:18.578540+0000 mgr.smithi027.ufhhvb (mgr.14162) 233 : cluster [DBG] pgmap v196: 65 pgs: 65 active+clean; 3.6 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2021-11-03T17:12:19.564 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2021-11-03T17:12:19.565 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-03T17:12:19.565 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2021-11-03T17:12:19.565 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2021-11-03T17:12:19.566 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2021-11-03T17:12:19.566 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2021-11-03T17:12:19.566 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2021-11-03T17:12:19.567 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2021-11-03T17:12:19.567 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2021-11-03T17:12:19.568 DEBUG:teuthology.orchestra.run.smithi027:> ') 2021-11-03T17:12:19.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-03T17:12:19.798 DEBUG:teuthology.orchestra.run.smithi027:> (cd / && exec bash -c ' 2021-11-03T17:12:19.798 DEBUG:teuthology.orchestra.run.smithi027:> set -e 2021-11-03T17:12:19.798 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set brx.0 up 2021-11-03T17:12:19.799 DEBUG:teuthology.orchestra.run.smithi027:> sudo ip link set dev brx.0 master ceph-brx 2021-11-03T17:12:19.799 DEBUG:teuthology.orchestra.run.smithi027:> ') 2021-11-03T17:12:19.907 INFO:teuthology.orchestra.run:Running command with timeout 60 2021-11-03T17:12:19.908 DEBUG:teuthology.orchestra.run.smithi027:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2021-11-03T17:12:19.938 INFO:teuthology.orchestra.run.smithi027.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2021-11-03T17:12:19.938 INFO:teuthology.orchestra.run:Running command with timeout 60 2021-11-03T17:12:19.939 DEBUG:teuthology.orchestra.run.smithi027:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2021-11-03T17:12:19.992 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-11-03T17:12:19.992 DEBUG:teuthology.orchestra.run.smithi027:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2021-11-03T17:12:20.101 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2021-11-03T17:12:20.102 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2021-11-03T17:12:20.244 INFO:teuthology.orchestra.run.smithi027.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,norbytes 2021-11-03T17:12:20.244 INFO:teuthology.orchestra.run.smithi027.stdout:mount.ceph: options "norequire_active_mds,name=0,norbytes" will pass to kernel. 2021-11-03T17:12:20.259 INFO:tasks.cephfs.kernel_mount:mount command passed 2021-11-03T17:12:20.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-03T17:12:20.259 DEBUG:teuthology.orchestra.run.smithi027:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2021-11-03T17:12:20.289 INFO:tasks.kclient:client.1 config is {} 2021-11-03T17:12:20.289 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -- ceph fs ls 2021-11-03T17:12:20.616 INFO:teuthology.orchestra.run.smithi027.stderr:Inferring config /var/lib/ceph/32ab977e-3cc8-11ec-8c28-001a4aab830c/mon.smithi027/config 2021-11-03T17:12:20.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:20 smithi115 conmon[34670]: cluster 2021-11-03T17:12:20.578968+0000 mgr.smithi027.ufhhvb (mgr.14162) 234 : cluster [DBG] pgmap v197: 65 pgs: 65 active+clean; 3.8 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2021-11-03T17:12:21.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:20 smithi027 conmon[32322]: cluster 2021-11-03T17:12:20.578968+0000 mgr.smithi027.ufhhvb (mgr.14162) 234 : cluster [DBG] pgmap v197: 65 pgs: 65 active+clean; 3.8 KiB data, 32 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2021-11-03T17:12:22.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.376+0000 7fbfbc867700 1 -- 172.21.15.27:0/3582851829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfb80f6af0 msgr2=0x7fbfb80f6f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:22.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.376+0000 7fbfbc867700 1 --2- 172.21.15.27:0/3582851829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfb80f6af0 0x7fbfb80f6f10 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7fbfa80099e0 tx=0x7fbfa8004d10).stop 2021-11-03T17:12:22.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.376+0000 7fbfbc867700 1 -- 172.21.15.27:0/3582851829 shutdown_connections 2021-11-03T17:12:22.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.376+0000 7fbfbc867700 1 --2- 172.21.15.27:0/3582851829 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfb80f57f0 0x7fbfb80f5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:12:22.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.376+0000 7fbfbc867700 1 --2- 172.21.15.27:0/3582851829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfb80f6af0 0x7fbfb80f6f10 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:22.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.376+0000 7fbfbc867700 1 -- 172.21.15.27:0/3582851829 >> 172.21.15.27:0/3582851829 conn(0x7fbfb80f0400 msgr2=0x7fbfb80f2820 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:12:22.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.376+0000 7fbfbc867700 1 -- 172.21.15.27:0/3582851829 shutdown_connections 2021-11-03T17:12:22.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.376+0000 7fbfbc867700 1 -- 172.21.15.27:0/3582851829 wait complete. 2021-11-03T17:12:22.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.377+0000 7fbfbc867700 1 Processor -- start 2021-11-03T17:12:22.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.377+0000 7fbfbc867700 1 -- start start 2021-11-03T17:12:22.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.377+0000 7fbfbc867700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfb80f57f0 0x7fbfb8105f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:22.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.377+0000 7fbfbc867700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfb80f6af0 0x7fbfb8106480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:22.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.377+0000 7fbfbc867700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb8106ac0 con 0x7fbfb80f6af0 2021-11-03T17:12:22.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.377+0000 7fbfbc867700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb8107370 con 0x7fbfb80f57f0 2021-11-03T17:12:22.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.377+0000 7fbfb659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfb80f6af0 0x7fbfb8106480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:22.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.377+0000 7fbfb659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfb80f6af0 0x7fbfb8106480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55116/0 (socket says 172.21.15.27:55116) 2021-11-03T17:12:22.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.377+0000 7fbfb659c700 1 -- 172.21.15.27:0/1301778410 learned_addr learned my addr 172.21.15.27:0/1301778410 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:12:22.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.378+0000 7fbfb6d9d700 1 --2- 172.21.15.27:0/1301778410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfb80f57f0 0x7fbfb8105f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:22.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.378+0000 7fbfb659c700 1 -- 172.21.15.27:0/1301778410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfb80f57f0 msgr2=0x7fbfb8105f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:22.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.378+0000 7fbfb659c700 1 --2- 172.21.15.27:0/1301778410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfb80f57f0 0x7fbfb8105f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:22.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.378+0000 7fbfb659c700 1 -- 172.21.15.27:0/1301778410 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa8009640 con 0x7fbfb80f6af0 2021-11-03T17:12:22.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.378+0000 7fbfb659c700 1 --2- 172.21.15.27:0/1301778410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfb80f6af0 0x7fbfb8106480 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7fbfac0093f0 tx=0x7fbfac006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:12:22.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.378+0000 7fbf9f7fe700 1 -- 172.21.15.27:0/1301778410 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfac009ea0 con 0x7fbfb80f6af0 2021-11-03T17:12:22.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.378+0000 7fbf9f7fe700 1 -- 172.21.15.27:0/1301778410 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7fbfac013ad0 con 0x7fbfb80f6af0 2021-11-03T17:12:22.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.378+0000 7fbfbc867700 1 -- 172.21.15.27:0/1301778410 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfb8107650 con 0x7fbfb80f6af0 2021-11-03T17:12:22.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.378+0000 7fbf9f7fe700 1 -- 172.21.15.27:0/1301778410 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfac00da80 con 0x7fbfb80f6af0 2021-11-03T17:12:22.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.378+0000 7fbfbc867700 1 -- 172.21.15.27:0/1301778410 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb8107b90 con 0x7fbfb80f6af0 2021-11-03T17:12:22.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.379+0000 7fbfbc867700 1 -- 172.21.15.27:0/1301778410 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfb8059100 con 0x7fbfb80f6af0 2021-11-03T17:12:22.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.379+0000 7fbf9f7fe700 1 -- 172.21.15.27:0/1301778410 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fbfac01a060 con 0x7fbfb80f6af0 2021-11-03T17:12:22.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.379+0000 7fbf9f7fe700 1 --2- 172.21.15.27:0/1301778410 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fbfa005c2b0 0x7fbfa005e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:22.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.380+0000 7fbf9f7fe700 1 -- 172.21.15.27:0/1301778410 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fbfac075e90 con 0x7fbfb80f6af0 2021-11-03T17:12:22.382 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.380+0000 7fbfb6d9d700 1 --2- 172.21.15.27:0/1301778410 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fbfa005c2b0 0x7fbfa005e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:22.382 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.380+0000 7fbfb6d9d700 1 --2- 172.21.15.27:0/1301778410 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fbfa005c2b0 0x7fbfa005e770 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fbfa8000f80 tx=0x7fbfa8004d10).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:12:22.382 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.384+0000 7fbf9f7fe700 1 -- 172.21.15.27:0/1301778410 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fbfac048b90 con 0x7fbfb80f6af0 2021-11-03T17:12:22.574 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.578+0000 7fbfbc867700 1 -- 172.21.15.27:0/1301778410 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbfb80008d0 con 0x7fbfb80f6af0 2021-11-03T17:12:22.574 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.579+0000 7fbf9f7fe700 1 -- 172.21.15.27:0/1301778410 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v13) v1 ==== 53+0+83 (secure 0 0 0) 0x7fbfac04c1b0 con 0x7fbfb80f6af0 2021-11-03T17:12:22.575 INFO:teuthology.orchestra.run.smithi027.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2021-11-03T17:12:22.575 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.580+0000 7fbfbc867700 1 -- 172.21.15.27:0/1301778410 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fbfa005c2b0 msgr2=0x7fbfa005e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:22.576 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.580+0000 7fbfbc867700 1 --2- 172.21.15.27:0/1301778410 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fbfa005c2b0 0x7fbfa005e770 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fbfa8000f80 tx=0x7fbfa8004d10).stop 2021-11-03T17:12:22.576 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.580+0000 7fbfbc867700 1 -- 172.21.15.27:0/1301778410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfb80f6af0 msgr2=0x7fbfb8106480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:22.576 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.580+0000 7fbfbc867700 1 --2- 172.21.15.27:0/1301778410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfb80f6af0 0x7fbfb8106480 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7fbfac0093f0 tx=0x7fbfac006ec0).stop 2021-11-03T17:12:22.576 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.580+0000 7fbfbc867700 1 -- 172.21.15.27:0/1301778410 shutdown_connections 2021-11-03T17:12:22.577 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.580+0000 7fbfbc867700 1 --2- 172.21.15.27:0/1301778410 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fbfa005c2b0 0x7fbfa005e770 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:22.577 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.580+0000 7fbfbc867700 1 --2- 172.21.15.27:0/1301778410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbfb80f57f0 0x7fbfb8105f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:22.577 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.580+0000 7fbfbc867700 1 --2- 172.21.15.27:0/1301778410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbfb80f6af0 0x7fbfb8106480 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:22.578 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.581+0000 7fbfbc867700 1 -- 172.21.15.27:0/1301778410 >> 172.21.15.27:0/1301778410 conn(0x7fbfb80f0400 msgr2=0x7fbfb80fa250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:12:22.578 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.581+0000 7fbfbc867700 1 -- 172.21.15.27:0/1301778410 shutdown_connections 2021-11-03T17:12:22.578 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:22.581+0000 7fbfbc867700 1 -- 172.21.15.27:0/1301778410 wait complete. 2021-11-03T17:12:23.182 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2021-11-03T17:12:23.182 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2021-11-03T17:12:23.182 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi115.front.sepia.ceph.com 2021-11-03T17:12:23.183 INFO:tasks.cephfs.mount:self.client.name = client.1 2021-11-03T17:12:23.183 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2021-11-03T17:12:23.183 INFO:tasks.cephfs.mount:self.cephfs_name = None 2021-11-03T17:12:23.184 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2021-11-03T17:12:23.184 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2021-11-03T17:12:23.184 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2021-11-03T17:12:23.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-03T17:12:23.185 DEBUG:teuthology.orchestra.run.smithi115:> ip addr 2021-11-03T17:12:23.203 INFO:teuthology.orchestra.run.smithi115.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2021-11-03T17:12:23.203 INFO:teuthology.orchestra.run.smithi115.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2021-11-03T17:12:23.203 INFO:teuthology.orchestra.run.smithi115.stdout: inet 127.0.0.1/8 scope host lo 2021-11-03T17:12:23.204 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2021-11-03T17:12:23.204 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 ::1/128 scope host 2021-11-03T17:12:23.204 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2021-11-03T17:12:23.204 INFO:teuthology.orchestra.run.smithi115.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2021-11-03T17:12:23.205 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:d8:a5:44 brd ff:ff:ff:ff:ff:ff 2021-11-03T17:12:23.205 INFO:teuthology.orchestra.run.smithi115.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2021-11-03T17:12:23.205 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:d8:a5:45 brd ff:ff:ff:ff:ff:ff 2021-11-03T17:12:23.205 INFO:teuthology.orchestra.run.smithi115.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2021-11-03T17:12:23.206 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:8f:ce:8e brd ff:ff:ff:ff:ff:ff 2021-11-03T17:12:23.206 INFO:teuthology.orchestra.run.smithi115.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2021-11-03T17:12:23.206 INFO:teuthology.orchestra.run.smithi115.stdout: link/ether 0c:c4:7a:8f:ce:8f brd ff:ff:ff:ff:ff:ff 2021-11-03T17:12:23.206 INFO:teuthology.orchestra.run.smithi115.stdout: inet 172.21.15.115/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2021-11-03T17:12:23.206 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft 42182sec preferred_lft 42182sec 2021-11-03T17:12:23.207 INFO:teuthology.orchestra.run.smithi115.stdout: inet6 fe80::ec4:7aff:fe8f:ce8f/64 scope link noprefixroute 2021-11-03T17:12:23.207 INFO:teuthology.orchestra.run.smithi115.stdout: valid_lft forever preferred_lft forever 2021-11-03T17:12:23.207 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2021-11-03T17:12:23.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-03T17:12:23.208 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec bash -c ' 2021-11-03T17:12:23.208 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2021-11-03T17:12:23.209 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link add name ceph-brx type bridge 2021-11-03T17:12:23.209 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip addr flush dev ceph-brx 2021-11-03T17:12:23.209 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link set ceph-brx up 2021-11-03T17:12:23.210 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2021-11-03T17:12:23.210 DEBUG:teuthology.orchestra.run.smithi115:> ') 2021-11-03T17:12:23.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:23 smithi027 conmon[32322]: cluster 2021-11-03T17:12:22.579493+0000 mgr.smithi027.ufhhvb 2021-11-03T17:12:23.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:23 smithi027 conmon[32322]: (mgr.14162) 235 : cluster [DBG] pgmap v198: 65 pgs: 65 active+clean; 4.8 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2021-11-03T17:12:23.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:23 smithi027 conmon[32322]: audit 2021-11-03T17:12:22.579684 2021-11-03T17:12:23.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:23 smithi027 conmon[32322]: +0000 mon.smithi027 (mon.0) 546 : audit [DBG] from='client.? 172.21.15.27:0/1301778410' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2021-11-03T17:12:23.283 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:23 smithi115 conmon[34670]: cluster 2021-11-03T17:12:22.579493+0000 mgr.smithi027.ufhhvb (mgr.14162) 235 : cluster [DBG] pgmap v198: 65 pgs: 65 active+clean; 4.8 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2021-11-03T17:12:23.283 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:23 smithi115 conmon[34670]: audit 2021-11-03T17:12:22.579684+0000 mon.smithi027 (mon.0) 546 : audit [DBG] from='client.? 172.21.15.27:0/1301778410' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2021-11-03T17:12:23.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-03T17:12:23.386 DEBUG:teuthology.orchestra.run.smithi115:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2021-11-03T17:12:23.451 INFO:teuthology.orchestra.run.smithi115.stdout:1 2021-11-03T17:12:23.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-03T17:12:23.453 DEBUG:teuthology.orchestra.run.smithi115:> route 2021-11-03T17:12:23.529 INFO:teuthology.orchestra.run.smithi115.stdout:Kernel IP routing table 2021-11-03T17:12:23.530 INFO:teuthology.orchestra.run.smithi115.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2021-11-03T17:12:23.530 INFO:teuthology.orchestra.run.smithi115.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2021-11-03T17:12:23.530 INFO:teuthology.orchestra.run.smithi115.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2021-11-03T17:12:23.531 INFO:teuthology.orchestra.run.smithi115.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2021-11-03T17:12:23.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-03T17:12:23.532 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec bash -c ' 2021-11-03T17:12:23.532 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2021-11-03T17:12:23.532 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2021-11-03T17:12:23.533 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2021-11-03T17:12:23.533 DEBUG:teuthology.orchestra.run.smithi115:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2021-11-03T17:12:23.533 DEBUG:teuthology.orchestra.run.smithi115:> ') 2021-11-03T17:12:23.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-03T17:12:23.768 DEBUG:teuthology.orchestra.run.smithi115:> ip netns list 2021-11-03T17:12:23.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-03T17:12:23.783 DEBUG:teuthology.orchestra.run.smithi115:> ip netns list-id 2021-11-03T17:12:23.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-03T17:12:23.837 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec bash -c ' 2021-11-03T17:12:23.837 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2021-11-03T17:12:23.838 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2021-11-03T17:12:23.838 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2021-11-03T17:12:23.839 DEBUG:teuthology.orchestra.run.smithi115:> ') 2021-11-03T17:12:23.925 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2021-11-03T17:12:23.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-03T17:12:23.926 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec bash -c ' 2021-11-03T17:12:23.926 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2021-11-03T17:12:23.927 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2021-11-03T17:12:23.927 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2021-11-03T17:12:23.927 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2021-11-03T17:12:23.928 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2021-11-03T17:12:23.928 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2021-11-03T17:12:23.928 DEBUG:teuthology.orchestra.run.smithi115:> ') 2021-11-03T17:12:24.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-03T17:12:24.146 DEBUG:teuthology.orchestra.run.smithi115:> (cd / && exec bash -c ' 2021-11-03T17:12:24.146 DEBUG:teuthology.orchestra.run.smithi115:> set -e 2021-11-03T17:12:24.146 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link set brx.0 up 2021-11-03T17:12:24.147 DEBUG:teuthology.orchestra.run.smithi115:> sudo ip link set dev brx.0 master ceph-brx 2021-11-03T17:12:24.147 DEBUG:teuthology.orchestra.run.smithi115:> ') 2021-11-03T17:12:24.245 INFO:teuthology.orchestra.run:Running command with timeout 60 2021-11-03T17:12:24.246 DEBUG:teuthology.orchestra.run.smithi115:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2021-11-03T17:12:24.265 INFO:teuthology.orchestra.run.smithi115.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2021-11-03T17:12:24.265 INFO:teuthology.orchestra.run:Running command with timeout 60 2021-11-03T17:12:24.266 DEBUG:teuthology.orchestra.run.smithi115:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2021-11-03T17:12:24.319 INFO:teuthology.orchestra.run:Running command with timeout 1800 2021-11-03T17:12:24.320 DEBUG:teuthology.orchestra.run.smithi115:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2021-11-03T17:12:24.427 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2021-11-03T17:12:24.428 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,name=1,norbytes" will pass to kernel. 2021-11-03T17:12:24.558 INFO:teuthology.orchestra.run.smithi115.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,norbytes 2021-11-03T17:12:24.558 INFO:teuthology.orchestra.run.smithi115.stdout:mount.ceph: options "norequire_active_mds,name=1,norbytes" will pass to kernel. 2021-11-03T17:12:24.563 INFO:tasks.cephfs.kernel_mount:mount command passed 2021-11-03T17:12:24.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2021-11-03T17:12:24.565 DEBUG:teuthology.orchestra.run.smithi115:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2021-11-03T17:12:24.593 INFO:teuthology.run_tasks:Running task print... 2021-11-03T17:12:24.604 INFO:teuthology.task.print:**** done kclient client 2021-11-03T17:12:24.605 INFO:teuthology.run_tasks:Running task parallel... 2021-11-03T17:12:24.647 INFO:teuthology.task.parallel:starting parallel... 2021-11-03T17:12:24.648 INFO:teuthology.task.parallel:In parallel, running task sequential... 2021-11-03T17:12:24.649 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2021-11-03T17:12:24.649 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi027.front.sepia.ceph.com 2021-11-03T17:12:24.650 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -e sha1=6e90212d8510e2d506395a8c82057d0d1787f3a4 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2021-11-03T17:12:24.650 INFO:teuthology.task.parallel:In parallel, running task sequential... 2021-11-03T17:12:24.651 INFO:teuthology.task.sequential:In sequential, running task workunit... 2021-11-03T17:12:24.654 INFO:tasks.workunit:Pulling workunits from ref 6e90212d8510e2d506395a8c82057d0d1787f3a4 2021-11-03T17:12:24.656 INFO:tasks.workunit:Making a separate scratch dir for every client... 2021-11-03T17:12:24.656 INFO:tasks.workunit:timeout=3h 2021-11-03T17:12:24.657 INFO:tasks.workunit:cleanup=True 2021-11-03T17:12:24.657 DEBUG:teuthology.orchestra.run.smithi027:> stat -- /home/ubuntu/cephtest/mnt.0 2021-11-03T17:12:24.679 INFO:teuthology.orchestra.run.smithi027.stdout: File: /home/ubuntu/cephtest/mnt.0 2021-11-03T17:12:24.679 INFO:teuthology.orchestra.run.smithi027.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2021-11-03T17:12:24.680 INFO:teuthology.orchestra.run.smithi027.stdout:Device: 77h/119d Inode: 1 Links: 2 2021-11-03T17:12:24.680 INFO:teuthology.orchestra.run.smithi027.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2021-11-03T17:12:24.680 INFO:teuthology.orchestra.run.smithi027.stdout:Context: system_u:object_r:cephfs_t:s0 2021-11-03T17:12:24.680 INFO:teuthology.orchestra.run.smithi027.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2021-11-03T17:12:24.681 INFO:teuthology.orchestra.run.smithi027.stdout:Modify: 2021-11-03 17:11:59.681821302 +0000 2021-11-03T17:12:24.681 INFO:teuthology.orchestra.run.smithi027.stdout:Change: 2021-11-03 17:12:20.288060892 +0000 2021-11-03T17:12:24.681 INFO:teuthology.orchestra.run.smithi027.stdout: Birth: - 2021-11-03T17:12:24.681 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2021-11-03T17:12:24.682 DEBUG:teuthology.orchestra.run.smithi027:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2021-11-03T17:12:24.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:24 smithi115 conmon[34670]: cluster 2021-11-03T17:12:24.580041+0000 mgr.smithi027.ufhhvb (mgr 2021-11-03T17:12:24.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:24 smithi115 conmon[34670]: .14162) 236 : cluster [DBG] pgmap v199: 65 pgs: 65 active+clean; 4.8 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2021-11-03T17:12:24.756 DEBUG:teuthology.orchestra.run.smithi115:> stat -- /home/ubuntu/cephtest/mnt.1 2021-11-03T17:12:24.773 INFO:teuthology.orchestra.run.smithi115.stdout: File: /home/ubuntu/cephtest/mnt.1 2021-11-03T17:12:24.773 INFO:teuthology.orchestra.run.smithi115.stdout: Size: 1 Blocks: 0 IO Block: 65536 directory 2021-11-03T17:12:24.773 INFO:teuthology.orchestra.run.smithi115.stdout:Device: 78h/120d Inode: 1 Links: 3 2021-11-03T17:12:24.773 INFO:teuthology.orchestra.run.smithi115.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2021-11-03T17:12:24.774 INFO:teuthology.orchestra.run.smithi115.stdout:Context: system_u:object_r:cephfs_t:s0 2021-11-03T17:12:24.774 INFO:teuthology.orchestra.run.smithi115.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2021-11-03T17:12:24.774 INFO:teuthology.orchestra.run.smithi115.stdout:Modify: 2021-11-03 17:12:24.751946241 +0000 2021-11-03T17:12:24.774 INFO:teuthology.orchestra.run.smithi115.stdout:Change: 2021-11-03 17:12:24.751946241 +0000 2021-11-03T17:12:24.775 INFO:teuthology.orchestra.run.smithi115.stdout: Birth: - 2021-11-03T17:12:24.775 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2021-11-03T17:12:24.775 DEBUG:teuthology.orchestra.run.smithi115:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2021-11-03T17:12:24.847 DEBUG:teuthology.orchestra.run.smithi027:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone git://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 6e90212d8510e2d506395a8c82057d0d1787f3a4 2021-11-03T17:12:24.848 DEBUG:teuthology.orchestra.run.smithi115:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone git://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout 6e90212d8510e2d506395a8c82057d0d1787f3a4 2021-11-03T17:12:24.864 INFO:tasks.workunit.client.0.smithi027.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2021-11-03T17:12:24.902 INFO:tasks.workunit.client.1.smithi115.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2021-11-03T17:12:25.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:24 smithi027 conmon[32322]: cluster 2021-11-03T17:12: 2021-11-03T17:12:25.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:24 smithi027 conmon[32322]: 24.580041+0000 mgr.smithi027.ufhhvb (mgr.14162) 236 : cluster [DBG] pgmap v199: 65 pgs: 65 active+clean; 4.8 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2021-11-03T17:12:26.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.553+0000 7fda14eca700 1 -- 172.21.15.27:0/832648464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda100f66e0 msgr2=0x7fda100f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:26.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.553+0000 7fda14eca700 1 --2- 172.21.15.27:0/832648464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda100f66e0 0x7fda100f6b40 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7fda00004660 tx=0x7fda00009b30).stop 2021-11-03T17:12:26.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.553+0000 7fda14eca700 1 -- 172.21.15.27:0/832648464 shutdown_connections 2021-11-03T17:12:26.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.553+0000 7fda14eca700 1 --2- 172.21.15.27:0/832648464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda100f66e0 0x7fda100f6b40 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:26.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.553+0000 7fda14eca700 1 --2- 172.21.15.27:0/832648464 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda100f5440 0x7fda100f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:26.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.553+0000 7fda14eca700 1 -- 172.21.15.27:0/832648464 >> 172.21.15.27:0/832648464 conn(0x7fda100f0ba0 msgr2=0x7fda100f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:12:26.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.553+0000 7fda14eca700 1 -- 172.21.15.27:0/832648464 shutdown_connections 2021-11-03T17:12:26.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.553+0000 7fda14eca700 1 -- 172.21.15.27:0/832648464 wait complete. 2021-11-03T17:12:26.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.554+0000 7fda14eca700 1 Processor -- start 2021-11-03T17:12:26.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.554+0000 7fda14eca700 1 -- start start 2021-11-03T17:12:26.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.554+0000 7fda14eca700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda100f5440 0x7fda10108f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:26.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.554+0000 7fda14eca700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda100f66e0 0x7fda10109440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:26.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.554+0000 7fda14eca700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda101099f0 con 0x7fda100f5440 2021-11-03T17:12:26.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.554+0000 7fda14eca700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda10109b30 con 0x7fda100f66e0 2021-11-03T17:12:26.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.554+0000 7fda0f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda100f5440 0x7fda10108f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:26.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.554+0000 7fda0f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda100f5440 0x7fda10108f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55126/0 (socket says 172.21.15.27:55126) 2021-11-03T17:12:26.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.554+0000 7fda0f7fe700 1 -- 172.21.15.27:0/3659661351 learned_addr learned my addr 172.21.15.27:0/3659661351 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:12:26.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.555+0000 7fda0effd700 1 --2- 172.21.15.27:0/3659661351 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda100f66e0 0x7fda10109440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:26.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.555+0000 7fda0f7fe700 1 -- 172.21.15.27:0/3659661351 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda100f66e0 msgr2=0x7fda10109440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:26.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.555+0000 7fda0f7fe700 1 --2- 172.21.15.27:0/3659661351 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda100f66e0 0x7fda10109440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:26.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.555+0000 7fda0f7fe700 1 -- 172.21.15.27:0/3659661351 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda00005040 con 0x7fda100f5440 2021-11-03T17:12:26.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.555+0000 7fda0f7fe700 1 --2- 172.21.15.27:0/3659661351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda100f5440 0x7fda10108f50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7fda04002700 tx=0x7fda04009d60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:12:26.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.555+0000 7fd9f7fff700 1 -- 172.21.15.27:0/3659661351 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda040079c0 con 0x7fda100f5440 2021-11-03T17:12:26.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.555+0000 7fd9f7fff700 1 -- 172.21.15.27:0/3659661351 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7fda0400f040 con 0x7fda100f5440 2021-11-03T17:12:26.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.555+0000 7fda14eca700 1 -- 172.21.15.27:0/3659661351 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda1010a4d0 con 0x7fda100f5440 2021-11-03T17:12:26.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.555+0000 7fd9f7fff700 1 -- 172.21.15.27:0/3659661351 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda04014750 con 0x7fda100f5440 2021-11-03T17:12:26.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.555+0000 7fda14eca700 1 -- 172.21.15.27:0/3659661351 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda1010aa90 con 0x7fda100f5440 2021-11-03T17:12:26.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.556+0000 7fda14eca700 1 -- 172.21.15.27:0/3659661351 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda10045ad0 con 0x7fda100f5440 2021-11-03T17:12:26.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.557+0000 7fd9f7fff700 1 -- 172.21.15.27:0/3659661351 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fda0401b060 con 0x7fda100f5440 2021-11-03T17:12:26.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.557+0000 7fd9f7fff700 1 --2- 172.21.15.27:0/3659661351 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fd9f805c300 0x7fd9f805e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:26.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.557+0000 7fd9f7fff700 1 -- 172.21.15.27:0/3659661351 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fda04075d40 con 0x7fda100f5440 2021-11-03T17:12:26.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.557+0000 7fda0effd700 1 --2- 172.21.15.27:0/3659661351 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fd9f805c300 0x7fd9f805e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:26.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.557+0000 7fda0effd700 1 --2- 172.21.15.27:0/3659661351 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fd9f805c300 0x7fd9f805e7c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fda00005010 tx=0x7fda00009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:12:26.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.561+0000 7fd9f7fff700 1 -- 172.21.15.27:0/3659661351 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fda040489c0 con 0x7fda100f5440 2021-11-03T17:12:26.716 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.719+0000 7fda14eca700 1 -- 172.21.15.27:0/3659661351 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7fda1010b130 con 0x7fda100f5440 2021-11-03T17:12:26.717 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.720+0000 7fd9f7fff700 1 -- 172.21.15.27:0/3659661351 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v16)=0 v16) v1 ==== 155+0+0 (secure 0 0 0) 0x7fda0404bfe0 con 0x7fda100f5440 2021-11-03T17:12:26.718 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.721+0000 7fda14eca700 1 -- 172.21.15.27:0/3659661351 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fd9f805c300 msgr2=0x7fd9f805e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:26.718 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.721+0000 7fda14eca700 1 --2- 172.21.15.27:0/3659661351 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fd9f805c300 0x7fd9f805e7c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fda00005010 tx=0x7fda00009b30).stop 2021-11-03T17:12:26.718 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.721+0000 7fda14eca700 1 -- 172.21.15.27:0/3659661351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda100f5440 msgr2=0x7fda10108f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:26.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.721+0000 7fda14eca700 1 --2- 172.21.15.27:0/3659661351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda100f5440 0x7fda10108f50 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7fda04002700 tx=0x7fda04009d60).stop 2021-11-03T17:12:26.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.721+0000 7fda14eca700 1 -- 172.21.15.27:0/3659661351 shutdown_connections 2021-11-03T17:12:26.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.721+0000 7fda14eca700 1 --2- 172.21.15.27:0/3659661351 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fd9f805c300 0x7fd9f805e7c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:26.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.721+0000 7fda14eca700 1 --2- 172.21.15.27:0/3659661351 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda100f5440 0x7fda10108f50 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:26.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.721+0000 7fda14eca700 1 --2- 172.21.15.27:0/3659661351 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda100f66e0 0x7fda10109440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:26.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.721+0000 7fda14eca700 1 -- 172.21.15.27:0/3659661351 >> 172.21.15.27:0/3659661351 conn(0x7fda100f0ba0 msgr2=0x7fda1005fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:12:26.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.722+0000 7fda14eca700 1 -- 172.21.15.27:0/3659661351 shutdown_connections 2021-11-03T17:12:26.721 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:26.722+0000 7fda14eca700 1 -- 172.21.15.27:0/3659661351 wait complete. 2021-11-03T17:12:26.934 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:26 smithi027 conmon[32322]: cluster 2021-11-03T17:12:26.580490+0000 mgr.smithi027.ufhhvb (mgr.14162) 237 : cluster [DBG] pgmap v200: 65 pgs: 65 active+clean; 9.1 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 853 B/s wr, 0 op/s 2021-11-03T17:12:26.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:26 smithi115 conmon[34670]: cluster 2021-11-03T17:12:26.580490+0000 mgr.smithi027.ufhhvb (mgr.14162) 237 : cluster [DBG] pgmap v200: 65 pgs: 65 active+clean; 9.1 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 853 B/s wr, 0 op/s 2021-11-03T17:12:26.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:26 smithi115 conmon[34670]: 2021-11-03T17:12:27.382 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -e sha1=6e90212d8510e2d506395a8c82057d0d1787f3a4 -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2021-11-03T17:12:28.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:28 smithi115 conmon[34670]: cluster 2021-11-03T17:12:28.580993+0000 mgr.smithi027.ufhhvb (mgr.14162) 238 : cluster [DBG] pgmap v201: 65 pgs: 65 active+clean; 9.1 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 853 B/s wr, 0 op/s 2021-11-03T17:12:29.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:28 smithi027 conmon[32322]: cluster 2021-11-03T17:12:28.580993+0000 mgr.smithi027.ufhhvb (mgr 2021-11-03T17:12:29.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:28 smithi027 conmon[32322]: .14162) 238 : cluster [DBG] pgmap v201: 65 pgs: 65 active+clean; 9.1 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 853 B/s wr, 0 op/s 2021-11-03T17:12:29.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.301+0000 7fc8e26a9700 1 -- 172.21.15.27:0/4225538673 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8dc0f6530 msgr2=0x7fc8dc0f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:29.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.301+0000 7fc8e26a9700 1 --2- 172.21.15.27:0/4225538673 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8dc0f6530 0x7fc8dc0f6950 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7fc8d8004660 tx=0x7fc8d8009b30).stop 2021-11-03T17:12:29.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.301+0000 7fc8e26a9700 1 -- 172.21.15.27:0/4225538673 shutdown_connections 2021-11-03T17:12:29.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.301+0000 7fc8e26a9700 1 --2- 172.21.15.27:0/4225538673 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8dc0f77d0 0x7fc8dc0f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:12:29.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.301+0000 7fc8e26a9700 1 --2- 172.21.15.27:0/4225538673 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8dc0f6530 0x7fc8dc0f6950 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:29.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.301+0000 7fc8e26a9700 1 -- 172.21.15.27:0/4225538673 >> 172.21.15.27:0/4225538673 conn(0x7fc8dc0f1cd0 msgr2=0x7fc8dc0f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:12:29.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.302+0000 7fc8e26a9700 1 -- 172.21.15.27:0/4225538673 shutdown_connections 2021-11-03T17:12:29.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.302+0000 7fc8e26a9700 1 -- 172.21.15.27:0/4225538673 wait complete. 2021-11-03T17:12:29.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.302+0000 7fc8e26a9700 1 Processor -- start 2021-11-03T17:12:29.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.302+0000 7fc8e26a9700 1 -- start start 2021-11-03T17:12:29.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.302+0000 7fc8e26a9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8dc0f6530 0x7fc8dc106da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:29.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.302+0000 7fc8e26a9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8dc0f77d0 0x7fc8dc107290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:29.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.302+0000 7fc8e26a9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8dc107920 con 0x7fc8dc0f6530 2021-11-03T17:12:29.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.302+0000 7fc8e26a9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8dc1080d0 con 0x7fc8dc0f77d0 2021-11-03T17:12:29.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.302+0000 7fc8e16a7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8dc0f6530 0x7fc8dc106da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:29.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.302+0000 7fc8e16a7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8dc0f6530 0x7fc8dc106da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55134/0 (socket says 172.21.15.27:55134) 2021-11-03T17:12:29.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.302+0000 7fc8e16a7700 1 -- 172.21.15.27:0/3086614458 learned_addr learned my addr 172.21.15.27:0/3086614458 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:12:29.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.303+0000 7fc8e0ea6700 1 --2- 172.21.15.27:0/3086614458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8dc0f77d0 0x7fc8dc107290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:29.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.303+0000 7fc8e16a7700 1 -- 172.21.15.27:0/3086614458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8dc0f77d0 msgr2=0x7fc8dc107290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:29.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.303+0000 7fc8e16a7700 1 --2- 172.21.15.27:0/3086614458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8dc0f77d0 0x7fc8dc107290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:29.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.303+0000 7fc8e16a7700 1 -- 172.21.15.27:0/3086614458 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d8005040 con 0x7fc8dc0f6530 2021-11-03T17:12:29.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.303+0000 7fc8e0ea6700 1 --2- 172.21.15.27:0/3086614458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8dc0f77d0 0x7fc8dc107290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:12:29.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.303+0000 7fc8e16a7700 1 --2- 172.21.15.27:0/3086614458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8dc0f6530 0x7fc8dc106da0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fc8d8007a60 tx=0x7fc8d8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:12:29.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.303+0000 7fc8d1ffb700 1 -- 172.21.15.27:0/3086614458 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8d8018070 con 0x7fc8dc0f6530 2021-11-03T17:12:29.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.303+0000 7fc8e26a9700 1 -- 172.21.15.27:0/3086614458 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8dc108350 con 0x7fc8dc0f6530 2021-11-03T17:12:29.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.303+0000 7fc8d1ffb700 1 -- 172.21.15.27:0/3086614458 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7fc8d8013410 con 0x7fc8dc0f6530 2021-11-03T17:12:29.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.303+0000 7fc8d1ffb700 1 -- 172.21.15.27:0/3086614458 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8d801c620 con 0x7fc8dc0f6530 2021-11-03T17:12:29.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.303+0000 7fc8e26a9700 1 -- 172.21.15.27:0/3086614458 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8dc1088b0 con 0x7fc8dc0f6530 2021-11-03T17:12:29.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.304+0000 7fc8e26a9700 1 -- 172.21.15.27:0/3086614458 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8dc059100 con 0x7fc8dc0f6530 2021-11-03T17:12:29.307 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.304+0000 7fc8d1ffb700 1 -- 172.21.15.27:0/3086614458 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fc8d801d940 con 0x7fc8dc0f6530 2021-11-03T17:12:29.307 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.305+0000 7fc8d1ffb700 1 --2- 172.21.15.27:0/3086614458 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fc8c805c300 0x7fc8c805e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:29.307 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.305+0000 7fc8d1ffb700 1 -- 172.21.15.27:0/3086614458 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fc8d8077e70 con 0x7fc8dc0f6530 2021-11-03T17:12:29.307 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.305+0000 7fc8e0ea6700 1 --2- 172.21.15.27:0/3086614458 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fc8c805c300 0x7fc8c805e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:29.308 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.305+0000 7fc8e0ea6700 1 --2- 172.21.15.27:0/3086614458 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fc8c805c300 0x7fc8c805e7c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fc8dc106b30 tx=0x7fc8cc008040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:12:29.308 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.309+0000 7fc8d1ffb700 1 -- 172.21.15.27:0/3086614458 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fc8d804ab50 con 0x7fc8dc0f6530 2021-11-03T17:12:29.466 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.470+0000 7fc8e26a9700 1 -- 172.21.15.27:0/3086614458 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7fc8dc045ad0 con 0x7fc8dc0f6530 2021-11-03T17:12:29.466 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.470+0000 7fc8d1ffb700 1 -- 172.21.15.27:0/3086614458 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v16)=0 v16) v1 ==== 163+0+0 (secure 0 0 0) 0x7fc8d804e170 con 0x7fc8dc0f6530 2021-11-03T17:12:29.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.472+0000 7fc8e26a9700 1 -- 172.21.15.27:0/3086614458 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fc8c805c300 msgr2=0x7fc8c805e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:29.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.472+0000 7fc8e26a9700 1 --2- 172.21.15.27:0/3086614458 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fc8c805c300 0x7fc8c805e7c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fc8dc106b30 tx=0x7fc8cc008040).stop 2021-11-03T17:12:29.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.472+0000 7fc8e26a9700 1 -- 172.21.15.27:0/3086614458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8dc0f6530 msgr2=0x7fc8dc106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:29.469 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.472+0000 7fc8e26a9700 1 --2- 172.21.15.27:0/3086614458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8dc0f6530 0x7fc8dc106da0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fc8d8007a60 tx=0x7fc8d8009b30).stop 2021-11-03T17:12:29.469 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.472+0000 7fc8e26a9700 1 -- 172.21.15.27:0/3086614458 shutdown_connections 2021-11-03T17:12:29.469 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.472+0000 7fc8e26a9700 1 --2- 172.21.15.27:0/3086614458 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fc8c805c300 0x7fc8c805e7c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:29.470 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.472+0000 7fc8e26a9700 1 --2- 172.21.15.27:0/3086614458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8dc0f6530 0x7fc8dc106da0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:29.470 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.472+0000 7fc8e26a9700 1 --2- 172.21.15.27:0/3086614458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8dc0f77d0 0x7fc8dc107290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:29.470 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.472+0000 7fc8e26a9700 1 -- 172.21.15.27:0/3086614458 >> 172.21.15.27:0/3086614458 conn(0x7fc8dc0f1cd0 msgr2=0x7fc8dc0fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:12:29.470 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.472+0000 7fc8e26a9700 1 -- 172.21.15.27:0/3086614458 shutdown_connections 2021-11-03T17:12:29.471 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:29.472+0000 7fc8e26a9700 1 -- 172.21.15.27:0/3086614458 wait complete. 2021-11-03T17:12:30.182 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -e sha1=6e90212d8510e2d506395a8c82057d0d1787f3a4 -- bash -c 'ceph config set global log_to_journald false --force' 2021-11-03T17:12:30.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:30 smithi115 conmon[34670]: cluster 2021-11-03T17:12:30.581648+0000 mgr.smithi027.ufhhvb (mgr.14162) 239 : cluster [DBG] pgmap v202: 65 pgs: 65 active+clean; 9.1 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2021-11-03T17:12:31.015 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:30 smithi027 conmon[32322]: cluster 2021-11-03T17:12:30.581648+0000 mgr.smithi027.ufhhvb (mgr.14162) 239 : cluster [DBG] pgmap v202: 65 pgs: 65 active+clean; 9.1 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2021-11-03T17:12:32.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.035+0000 7fd7a56db700 1 -- 172.21.15.27:0/3395117636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7a0060270 msgr2=0x7fd7a00f91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:32.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.035+0000 7fd7a56db700 1 --2- 172.21.15.27:0/3395117636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7a0060270 0x7fd7a00f91c0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7fd790004660 tx=0x7fd790009b30).stop 2021-11-03T17:12:32.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.035+0000 7fd7a56db700 1 -- 172.21.15.27:0/3395117636 shutdown_connections 2021-11-03T17:12:32.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.035+0000 7fd7a56db700 1 --2- 172.21.15.27:0/3395117636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7a00f96b0 0x7fd7a00fbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:32.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.035+0000 7fd7a56db700 1 --2- 172.21.15.27:0/3395117636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7a0060270 0x7fd7a00f91c0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:32.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.035+0000 7fd7a56db700 1 -- 172.21.15.27:0/3395117636 >> 172.21.15.27:0/3395117636 conn(0x7fd7a00f0ba0 msgr2=0x7fd7a00f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:12:32.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.035+0000 7fd7a56db700 1 -- 172.21.15.27:0/3395117636 shutdown_connections 2021-11-03T17:12:32.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.035+0000 7fd7a56db700 1 -- 172.21.15.27:0/3395117636 wait complete. 2021-11-03T17:12:32.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.036+0000 7fd7a56db700 1 Processor -- start 2021-11-03T17:12:32.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.036+0000 7fd7a56db700 1 -- start start 2021-11-03T17:12:32.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.036+0000 7fd7a56db700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7a0060270 0x7fd7a00f6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:32.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.036+0000 7fd7a56db700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7a00f96b0 0x7fd7a00f50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:32.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.036+0000 7fd7a56db700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a00f5620 con 0x7fd7a00f96b0 2021-11-03T17:12:32.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.036+0000 7fd7a56db700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7a00f5760 con 0x7fd7a0060270 2021-11-03T17:12:32.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.036+0000 7fd79f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7a00f96b0 0x7fd7a00f50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:32.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.036+0000 7fd79f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7a00f96b0 0x7fd7a00f50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55146/0 (socket says 172.21.15.27:55146) 2021-11-03T17:12:32.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.036+0000 7fd79f7fe700 1 -- 172.21.15.27:0/1993210566 learned_addr learned my addr 172.21.15.27:0/1993210566 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:12:32.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.036+0000 7fd79ffff700 1 --2- 172.21.15.27:0/1993210566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7a0060270 0x7fd7a00f6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:32.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.037+0000 7fd79f7fe700 1 -- 172.21.15.27:0/1993210566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7a0060270 msgr2=0x7fd7a00f6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:32.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.037+0000 7fd79f7fe700 1 --2- 172.21.15.27:0/1993210566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7a0060270 0x7fd7a00f6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:32.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.037+0000 7fd79f7fe700 1 -- 172.21.15.27:0/1993210566 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd790005040 con 0x7fd7a00f96b0 2021-11-03T17:12:32.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.037+0000 7fd79f7fe700 1 --2- 172.21.15.27:0/1993210566 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7a00f96b0 0x7fd7a00f50c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7fd794002700 tx=0x7fd794009d60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:12:32.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.037+0000 7fd79cff9700 1 -- 172.21.15.27:0/1993210566 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7940079c0 con 0x7fd7a00f96b0 2021-11-03T17:12:32.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.037+0000 7fd79cff9700 1 -- 172.21.15.27:0/1993210566 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(25 keys) v1 ==== 1047+0+0 (secure 0 0 0) 0x7fd79400f040 con 0x7fd7a00f96b0 2021-11-03T17:12:32.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.037+0000 7fd79cff9700 1 -- 172.21.15.27:0/1993210566 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7940146e0 con 0x7fd7a00f96b0 2021-11-03T17:12:32.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.037+0000 7fd7a56db700 1 -- 172.21.15.27:0/1993210566 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7a00f5a40 con 0x7fd7a00f96b0 2021-11-03T17:12:32.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.037+0000 7fd7a56db700 1 -- 172.21.15.27:0/1993210566 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7a010ede0 con 0x7fd7a00f96b0 2021-11-03T17:12:32.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.038+0000 7fd7a56db700 1 -- 172.21.15.27:0/1993210566 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7a0045ad0 con 0x7fd7a00f96b0 2021-11-03T17:12:32.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.038+0000 7fd79cff9700 1 -- 172.21.15.27:0/1993210566 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fd79401b060 con 0x7fd7a00f96b0 2021-11-03T17:12:32.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.039+0000 7fd79cff9700 1 --2- 172.21.15.27:0/1993210566 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fd78805c300 0x7fd78805e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:32.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.039+0000 7fd79cff9700 1 -- 172.21.15.27:0/1993210566 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fd794075d10 con 0x7fd7a00f96b0 2021-11-03T17:12:32.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.039+0000 7fd79ffff700 1 --2- 172.21.15.27:0/1993210566 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fd78805c300 0x7fd78805e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:32.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.039+0000 7fd79ffff700 1 --2- 172.21.15.27:0/1993210566 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fd78805c300 0x7fd78805e7c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7fd790005010 tx=0x7fd790009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:12:32.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.043+0000 7fd79cff9700 1 -- 172.21.15.27:0/1993210566 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fd794048990 con 0x7fd7a00f96b0 2021-11-03T17:12:32.200 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.204+0000 7fd7a56db700 1 -- 172.21.15.27:0/1993210566 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7fd7a0027a00 con 0x7fd7a00f96b0 2021-11-03T17:12:32.206 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.210+0000 7fd79cff9700 1 -- 172.21.15.27:0/1993210566 <== mon.0 v2:172.21.15.27:3300/0 7 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fd794048990 con 0x7fd7a00f96b0 2021-11-03T17:12:32.206 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.211+0000 7fd79cff9700 1 -- 172.21.15.27:0/1993210566 <== mon.0 v2:172.21.15.27:3300/0 8 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v17)=0 v17) v1 ==== 135+0+0 (secure 0 0 0) 0x7fd79404bfb0 con 0x7fd7a00f96b0 2021-11-03T17:12:32.209 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.213+0000 7fd7a56db700 1 -- 172.21.15.27:0/1993210566 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fd78805c300 msgr2=0x7fd78805e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:32.209 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.213+0000 7fd7a56db700 1 --2- 172.21.15.27:0/1993210566 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fd78805c300 0x7fd78805e7c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7fd790005010 tx=0x7fd790009b30).stop 2021-11-03T17:12:32.209 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.213+0000 7fd7a56db700 1 -- 172.21.15.27:0/1993210566 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7a00f96b0 msgr2=0x7fd7a00f50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:32.209 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.213+0000 7fd7a56db700 1 --2- 172.21.15.27:0/1993210566 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7a00f96b0 0x7fd7a00f50c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7fd794002700 tx=0x7fd794009d60).stop 2021-11-03T17:12:32.210 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.213+0000 7fd7a56db700 1 -- 172.21.15.27:0/1993210566 shutdown_connections 2021-11-03T17:12:32.210 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.213+0000 7fd7a56db700 1 --2- 172.21.15.27:0/1993210566 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fd78805c300 0x7fd78805e7c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:32.210 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.213+0000 7fd7a56db700 1 --2- 172.21.15.27:0/1993210566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7a0060270 0x7fd7a00f6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:32.211 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.213+0000 7fd7a56db700 1 --2- 172.21.15.27:0/1993210566 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7a00f96b0 0x7fd7a00f50c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:32.211 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.213+0000 7fd7a56db700 1 -- 172.21.15.27:0/1993210566 >> 172.21.15.27:0/1993210566 conn(0x7fd7a00f0ba0 msgr2=0x7fd7a00f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:12:32.211 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.213+0000 7fd7a56db700 1 -- 172.21.15.27:0/1993210566 shutdown_connections 2021-11-03T17:12:32.211 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:32.213+0000 7fd7a56db700 1 -- 172.21.15.27:0/1993210566 wait complete. 2021-11-03T17:12:32.807 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -e sha1=6e90212d8510e2d506395a8c82057d0d1787f3a4 -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2021-11-03T17:12:33.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:33 smithi115 conmon[34670]: audit 2021-11-03T17:12:32.211686+0000 mon.smithi027 (mon.0) 547 : audit [INF] from='client.? 172.21.15.27:0/1993210566' entity='client.admin' 2021-11-03T17:12:33.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:33 smithi115 conmon[34670]: audit 2021-11-03T17:12:32.215567+0000 mon.smithi027 (mon.0) 548 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:12:33.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:33 smithi115 conmon[34670]: audit 2021-11-03T17:12:32.223115+0000 mon.smithi027 (mon.0) 549 : audit 2021-11-03T17:12:33.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:33 smithi115 conmon[34670]: [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:12:33.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:33 smithi115 conmon[34670]: cluster 2021-11 2021-11-03T17:12:33.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:33 smithi115 conmon[34670]: -03T17:12:32.582069+0000 mgr.smithi027.ufhhvb (mgr.14162) 240 2021-11-03T17:12:33.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:33 smithi115 conmon[34670]: : cluster [DBG] pgmap v203: 65 pgs: 65 active+clean; 9.1 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 938 B/s wr, 0 op/s 2021-11-03T17:12:33.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:33 smithi027 conmon[32322]: audit 2021-11-03T17:12:32.211686+0000 mon.smithi027 (mon. 2021-11-03T17:12:33.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:33 smithi027 conmon[32322]: 0) 547 : audit [INF] from='client.? 172.21.15.27:0/1993210566' entity='client.admin' 2021-11-03T17:12:33.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:33 smithi027 conmon[32322]: audit 2021-11-03T17:12:32.215567+0000 2021-11-03T17:12:33.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:33 smithi027 conmon[32322]: mon.smithi027 (mon.0) 548 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:12:33.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:33 smithi027 conmon[32322]: audit 2021- 2021-11-03T17:12:33.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:33 smithi027 conmon[32322]: 11-03T17:12: 2021-11-03T17:12:33.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:33 smithi027 conmon[32322]: 32.223115+0000 mon.smithi027 (mon.0) 549 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:12:33.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:33 smithi027 conmon[32322]: cluster 2021-11-03T17:12: 2021-11-03T17:12:33.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:33 smithi027 conmon[32322]: 32.582069+0000 mgr.smithi027.ufhhvb (mgr.14162) 240 : cluster [DBG] 2021-11-03T17:12:33.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:33 smithi027 conmon[32322]: pgmap v203: 65 pgs: 65 active+clean; 9.1 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 938 B/s wr, 0 op/s 2021-11-03T17:12:34.639 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.642+0000 7fa591aef700 1 -- 172.21.15.27:0/511116712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa58c0f6860 msgr2=0x7fa58c0f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:34.640 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.642+0000 7fa591aef700 1 --2- 172.21.15.27:0/511116712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa58c0f6860 0x7fa58c0f6cc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7fa580004660 tx=0x7fa580009b30).stop 2021-11-03T17:12:34.640 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.642+0000 7fa591aef700 1 -- 172.21.15.27:0/511116712 shutdown_connections 2021-11-03T17:12:34.640 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.642+0000 7fa591aef700 1 --2- 172.21.15.27:0/511116712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa58c0f6860 0x7fa58c0f6cc0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:34.640 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.642+0000 7fa591aef700 1 --2- 172.21.15.27:0/511116712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa58c0f55c0 0x7fa58c0f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:34.641 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.642+0000 7fa591aef700 1 -- 172.21.15.27:0/511116712 >> 172.21.15.27:0/511116712 conn(0x7fa58c0f0d20 msgr2=0x7fa58c0f3140 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:12:34.641 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.642+0000 7fa591aef700 1 -- 172.21.15.27:0/511116712 shutdown_connections 2021-11-03T17:12:34.641 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.642+0000 7fa591aef700 1 -- 172.21.15.27:0/511116712 wait complete. 2021-11-03T17:12:34.641 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.643+0000 7fa591aef700 1 Processor -- start 2021-11-03T17:12:34.641 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.643+0000 7fa591aef700 1 -- start start 2021-11-03T17:12:34.642 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.643+0000 7fa591aef700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa58c0f55c0 0x7fa58c0f9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:34.642 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.643+0000 7fa591aef700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa58c0f6860 0x7fa58c0f83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:34.642 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.643+0000 7fa591aef700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c0f8920 con 0x7fa58c0f55c0 2021-11-03T17:12:34.642 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.643+0000 7fa591aef700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa58c0f8a60 con 0x7fa58c0f6860 2021-11-03T17:12:34.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.643+0000 7fa58b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa58c0f55c0 0x7fa58c0f9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:34.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.643+0000 7fa58b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa58c0f55c0 0x7fa58c0f9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55154/0 (socket says 172.21.15.27:55154) 2021-11-03T17:12:34.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.643+0000 7fa58b7fe700 1 -- 172.21.15.27:0/2730636057 learned_addr learned my addr 172.21.15.27:0/2730636057 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:12:34.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.643+0000 7fa58b7fe700 1 -- 172.21.15.27:0/2730636057 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa58c0f6860 msgr2=0x7fa58c0f83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:34.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.643+0000 7fa58b7fe700 1 --2- 172.21.15.27:0/2730636057 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa58c0f6860 0x7fa58c0f83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:12:34.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.643+0000 7fa58b7fe700 1 -- 172.21.15.27:0/2730636057 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa580005040 con 0x7fa58c0f55c0 2021-11-03T17:12:34.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.643+0000 7fa58b7fe700 1 --2- 172.21.15.27:0/2730636057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa58c0f55c0 0x7fa58c0f9f40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7fa57c00c790 tx=0x7fa57c007d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:12:34.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.644+0000 7fa573fff700 1 -- 172.21.15.27:0/2730636057 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa57c00e680 con 0x7fa58c0f55c0 2021-11-03T17:12:34.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.644+0000 7fa573fff700 1 -- 172.21.15.27:0/2730636057 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fa57c00ec90 con 0x7fa58c0f55c0 2021-11-03T17:12:34.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.644+0000 7fa591aef700 1 -- 172.21.15.27:0/2730636057 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa58c0f8d40 con 0x7fa58c0f55c0 2021-11-03T17:12:34.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.644+0000 7fa573fff700 1 -- 172.21.15.27:0/2730636057 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa57c012d00 con 0x7fa58c0f55c0 2021-11-03T17:12:34.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.644+0000 7fa591aef700 1 -- 172.21.15.27:0/2730636057 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa58c10ef70 con 0x7fa58c0f55c0 2021-11-03T17:12:34.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.645+0000 7fa591aef700 1 -- 172.21.15.27:0/2730636057 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa58c045ad0 con 0x7fa58c0f55c0 2021-11-03T17:12:34.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.645+0000 7fa573fff700 1 -- 172.21.15.27:0/2730636057 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fa57c010070 con 0x7fa58c0f55c0 2021-11-03T17:12:34.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.645+0000 7fa573fff700 1 --2- 172.21.15.27:0/2730636057 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fa57405c2b0 0x7fa57405e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:34.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.645+0000 7fa573fff700 1 -- 172.21.15.27:0/2730636057 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fa57c075ef0 con 0x7fa58c0f55c0 2021-11-03T17:12:34.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.645+0000 7fa58affd700 1 --2- 172.21.15.27:0/2730636057 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fa57405c2b0 0x7fa57405e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:34.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.646+0000 7fa58affd700 1 --2- 172.21.15.27:0/2730636057 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fa57405c2b0 0x7fa57405e770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fa580004e30 tx=0x7fa580009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:12:34.648 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.649+0000 7fa573fff700 1 -- 172.21.15.27:0/2730636057 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fa57c048c50 con 0x7fa58c0f55c0 2021-11-03T17:12:34.839 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.843+0000 7fa591aef700 1 -- 172.21.15.27:0/2730636057 --> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:6e90212d8510e2d506395a8c82057d0d1787f3a4", "target": ["mon-mgr", ""]}) v1 -- 0x7fa58c05fe90 con 0x7fa57405c2b0 2021-11-03T17:12:34.844 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.848+0000 7fa573fff700 1 -- 172.21.15.27:0/2730636057 <== mgr.14162 v2:172.21.15.27:6800/3802716887 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7fa58c05fe90 con 0x7fa57405c2b0 2021-11-03T17:12:34.844 INFO:teuthology.orchestra.run.smithi027.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:6e90212d8510e2d506395a8c82057d0d1787f3a4 2021-11-03T17:12:34.846 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.850+0000 7fa591aef700 1 -- 172.21.15.27:0/2730636057 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fa57405c2b0 msgr2=0x7fa57405e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:34.847 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.850+0000 7fa591aef700 1 --2- 172.21.15.27:0/2730636057 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fa57405c2b0 0x7fa57405e770 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7fa580004e30 tx=0x7fa580009b30).stop 2021-11-03T17:12:34.847 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.850+0000 7fa591aef700 1 -- 172.21.15.27:0/2730636057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa58c0f55c0 msgr2=0x7fa58c0f9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:34.847 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.850+0000 7fa591aef700 1 --2- 172.21.15.27:0/2730636057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa58c0f55c0 0x7fa58c0f9f40 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7fa57c00c790 tx=0x7fa57c007d90).stop 2021-11-03T17:12:34.847 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.850+0000 7fa591aef700 1 -- 172.21.15.27:0/2730636057 shutdown_connections 2021-11-03T17:12:34.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.851+0000 7fa591aef700 1 --2- 172.21.15.27:0/2730636057 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fa57405c2b0 0x7fa57405e770 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:34.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.851+0000 7fa591aef700 1 --2- 172.21.15.27:0/2730636057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa58c0f55c0 0x7fa58c0f9f40 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:34.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.851+0000 7fa591aef700 1 --2- 172.21.15.27:0/2730636057 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa58c0f6860 0x7fa58c0f83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:12:34.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.851+0000 7fa591aef700 1 -- 172.21.15.27:0/2730636057 >> 172.21.15.27:0/2730636057 conn(0x7fa58c0f0d20 msgr2=0x7fa58c05f780 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:12:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.851+0000 7fa591aef700 1 -- 172.21.15.27:0/2730636057 shutdown_connections 2021-11-03T17:12:34.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:34.851+0000 7fa591aef700 1 -- 172.21.15.27:0/2730636057 wait complete. 2021-11-03T17:12:34.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:34 smithi115 conmon[34670]: cluster 2021-11-03T17:12:34.582434+0000 mgr.smithi027.ufhhvb (mgr.14162) 241 : cluster [DBG] pgmap v204: 65 pgs: 65 active+clean; 9.1 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2021-11-03T17:12:35.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:34 smithi027 conmon[32322]: cluster 2021-11-03T17:12:34.582434+0000 mgr.smithi027.ufhhvb (mgr.14162) 241 : cluster [DBG] pgmap v204: 65 pgs: 65 active+clean; 9.1 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2021-11-03T17:12:35.518 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2021-11-03T17:12:35.519 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi027.front.sepia.ceph.com 2021-11-03T17:12:35.520 DEBUG:teuthology.orchestra.run.smithi027:> sudo /home/ubuntu/cephtest/cephadm --image docker.io/ceph/ceph:v16.2.4 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c -e sha1=6e90212d8510e2d506395a8c82057d0d1787f3a4 -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true ; do ceph orch ps ; ceph versions ; ceph fs dump; sleep 30 ; done' 2021-11-03T17:12:36.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:35 smithi115 conmon[34670]: audit 2021-11-03T17:12:34.845687+0000 mgr.smithi027.ufhhvb (mgr.14162) 242 : audit [DBG] from='client.14506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:6e90212d8510e2d506395a8c82057d0d1787f3a4", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:12:36.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:35 smithi115 conmon[34670]: cephadm 2021-11-03T17:12:34.846306+0000 mgr.smithi027.ufhhvb 2021-11-03T17:12:36.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:35 smithi115 conmon[34670]: (mgr.14162) 243 : cephadm [INF] Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:6e90212d8510e2d506395a8c82057d0d1787f3a4 2021-11-03T17:12:36.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:35 smithi115 conmon[34670]: audit 2021-11-03T17:12:34.849897+0000 mon.smithi027 (mon.0) 550 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:12:36.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:35 smithi115 conmon[34670]: audit 2021-11-03T17:12:34.850387+0000 mon.smithi027 (mon.0) 551 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:12:36.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:35 smithi115 conmon[34670]: audit 2021-11-03T17:12:35.335075+0000 mon.smithi027 (mon.0) 552 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:12:36.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:35 smithi115 conmon[34670]: cephadm 2021-11-03T17:12:35.338746+0000 mgr.smithi027.ufhhvb (mgr.14162) 244 : cephadm [INF] Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:6e90212d8510e2d506395a8c82057d0d1787f3a4 2021-11-03T17:12:36.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:35 smithi027 conmon[32322]: audit 2021-11-03T17 2021-11-03T17:12:36.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:35 smithi027 conmon[32322]: :12:34.845687+0000 mgr.smithi027.ufhhvb (mgr.14162) 242 : audit [DBG] from='client.14506 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:6e90212d8510e2d506395a8c82057d0d1787f3a4", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:12:36.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:35 smithi027 conmon[32322]: cephadm 2021-11-03T17:12:34.846306+0000 mgr.smithi027.ufhhvb (mgr.14162) 243 : cephadm [INF] Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:6e90212d8510e2d506395a8c82057d0d1787f3a4 2021-11-03T17:12:36.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:35 smithi027 conmon[32322]: audit 2021-11-03T17:12:34.849897+0000 mon.smithi027 (mon.0) 550 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:12:36.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:35 smithi027 conmon[32322]: audit 2021-11-03T17:12:34.850387+0000 mon.smithi027 (mon.0) 551 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:12:36.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:35 smithi027 conmon[32322]: audit 2021-11-03T17:12: 2021-11-03T17:12:36.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:35 smithi027 conmon[32322]: 35.335075+0000 mon.smithi027 (mon.0) 552 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:12:36.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:35 smithi027 conmon[32322]: cephadm 2021- 2021-11-03T17:12:36.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:35 smithi027 conmon[32322]: 11-03T17:12:35 2021-11-03T17:12:36.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:35 smithi027 conmon[32322]: .338746+0000 mgr.smithi027.ufhhvb (mgr. 2021-11-03T17:12:36.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:35 smithi027 conmon[32322]: 14162) 244 : cephadm [INF] Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:6e90212d8510e2d506395a8c82057d0d1787f3a4 2021-11-03T17:12:36.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:35 smithi027 conmon[32322]: 2021-11-03T17:12:37.172 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:36 smithi027 conmon[32322]: cluster 2021-11-03T17:12: 2021-11-03T17:12:37.172 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:36 smithi027 conmon[32322]: 36.583008+0000 mgr.smithi027.ufhhvb (mgr.14162) 245 : cluster [DBG] pgmap v205: 65 pgs: 65 active+clean; 9.1 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2021-11-03T17:12:37.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:36 smithi115 conmon[34670]: cluster 2021-11-03T17:12:36.583008+0000 mgr.smithi027.ufhhvb (mgr.14162) 245 : cluster [DBG] pgmap v205: 65 pgs: 65 active+clean; 9.1 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2021-11-03T17:12:37.639 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.642+0000 7f09a575a700 1 -- 172.21.15.27:0/3889575549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09a0060270 msgr2=0x7f09a00f91b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:37.639 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.642+0000 7f09a575a700 1 --2- 172.21.15.27:0/3889575549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09a0060270 0x7f09a00f91b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f0990004660 tx=0x7f0990009b30).stop 2021-11-03T17:12:37.639 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.642+0000 7f09a575a700 1 -- 172.21.15.27:0/3889575549 shutdown_connections 2021-11-03T17:12:37.639 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.642+0000 7f09a575a700 1 --2- 172.21.15.27:0/3889575549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09a00f96a0 0x7f09a00fbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:37.640 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.642+0000 7f09a575a700 1 --2- 172.21.15.27:0/3889575549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09a0060270 0x7f09a00f91b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:37.640 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.642+0000 7f09a575a700 1 -- 172.21.15.27:0/3889575549 >> 172.21.15.27:0/3889575549 conn(0x7f09a00f0b90 msgr2=0x7f09a00f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:12:37.640 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.642+0000 7f09a575a700 1 -- 172.21.15.27:0/3889575549 shutdown_connections 2021-11-03T17:12:37.640 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.642+0000 7f09a575a700 1 -- 172.21.15.27:0/3889575549 wait complete. 2021-11-03T17:12:37.641 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.643+0000 7f09a575a700 1 Processor -- start 2021-11-03T17:12:37.641 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.643+0000 7f09a575a700 1 -- start start 2021-11-03T17:12:37.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.643+0000 7f09a575a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09a0060270 0x7f09a00f6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:37.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.643+0000 7f09a575a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09a00f96a0 0x7f09a00f50b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:37.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.643+0000 7f09a575a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a00f5610 con 0x7f09a00f96a0 2021-11-03T17:12:37.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.643+0000 7f09a575a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09a00f5750 con 0x7f09a0060270 2021-11-03T17:12:37.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.643+0000 7f099f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09a00f96a0 0x7f09a00f50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:37.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.643+0000 7f099f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09a00f96a0 0x7f09a00f50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55180/0 (socket says 172.21.15.27:55180) 2021-11-03T17:12:37.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.643+0000 7f099f7fe700 1 -- 172.21.15.27:0/3371524465 learned_addr learned my addr 172.21.15.27:0/3371524465 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:12:37.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.644+0000 7f099ffff700 1 --2- 172.21.15.27:0/3371524465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09a0060270 0x7f09a00f6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:37.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.644+0000 7f099f7fe700 1 -- 172.21.15.27:0/3371524465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09a0060270 msgr2=0x7f09a00f6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:37.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.644+0000 7f099f7fe700 1 --2- 172.21.15.27:0/3371524465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09a0060270 0x7f09a00f6c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:37.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.644+0000 7f099f7fe700 1 -- 172.21.15.27:0/3371524465 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0990005040 con 0x7f09a00f96a0 2021-11-03T17:12:37.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.644+0000 7f099f7fe700 1 --2- 172.21.15.27:0/3371524465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09a00f96a0 0x7f09a00f50b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f099400c510 tx=0x7f0994007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:12:37.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.644+0000 7f099cff9700 1 -- 172.21.15.27:0/3371524465 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f099400f650 con 0x7f09a00f96a0 2021-11-03T17:12:37.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.644+0000 7f099cff9700 1 -- 172.21.15.27:0/3371524465 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f099400fc60 con 0x7f09a00f96a0 2021-11-03T17:12:37.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.644+0000 7f099cff9700 1 -- 172.21.15.27:0/3371524465 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0994012c20 con 0x7f09a00f96a0 2021-11-03T17:12:37.648 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.644+0000 7f09a575a700 1 -- 172.21.15.27:0/3371524465 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09a00f5a30 con 0x7f09a00f96a0 2021-11-03T17:12:37.648 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.644+0000 7f09a575a700 1 -- 172.21.15.27:0/3371524465 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09a010edb0 con 0x7f09a00f96a0 2021-11-03T17:12:37.648 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.645+0000 7f09a575a700 1 -- 172.21.15.27:0/3371524465 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09a0045ad0 con 0x7f09a00f96a0 2021-11-03T17:12:37.648 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.646+0000 7f099cff9700 1 -- 172.21.15.27:0/3371524465 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f0994010070 con 0x7f09a00f96a0 2021-11-03T17:12:37.649 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.646+0000 7f099cff9700 1 --2- 172.21.15.27:0/3371524465 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f098805c300 0x7f098805e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:37.649 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.646+0000 7f099cff9700 1 -- 172.21.15.27:0/3371524465 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f0994075a60 con 0x7f09a00f96a0 2021-11-03T17:12:37.649 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.646+0000 7f099ffff700 1 --2- 172.21.15.27:0/3371524465 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f098805c300 0x7f098805e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:37.649 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.647+0000 7f099ffff700 1 --2- 172.21.15.27:0/3371524465 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f098805c300 0x7f098805e7c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f0990004630 tx=0x7f0990009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:12:37.650 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.653+0000 7f099cff9700 1 -- 172.21.15.27:0/3371524465 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f09940452b0 con 0x7f09a00f96a0 2021-11-03T17:12:37.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.904+0000 7f09a575a700 1 -- 172.21.15.27:0/3371524465 --> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f09a00fee90 con 0x7f098805c300 2021-11-03T17:12:37.900 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.905+0000 7f099cff9700 1 -- 172.21.15.27:0/3371524465 <== mgr.14162 v2:172.21.15.27:6800/3802716887 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+278 (secure 0 0 0) 0x7f09a00fee90 con 0x7f098805c300 2021-11-03T17:12:37.902 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.907+0000 7f09a575a700 1 -- 172.21.15.27:0/3371524465 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f098805c300 msgr2=0x7f098805e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:37.903 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.907+0000 7f09a575a700 1 --2- 172.21.15.27:0/3371524465 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f098805c300 0x7f098805e7c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f0990004630 tx=0x7f0990009b30).stop 2021-11-03T17:12:37.903 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.907+0000 7f09a575a700 1 -- 172.21.15.27:0/3371524465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09a00f96a0 msgr2=0x7f09a00f50b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:37.903 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.907+0000 7f09a575a700 1 --2- 172.21.15.27:0/3371524465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09a00f96a0 0x7f09a00f50b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f099400c510 tx=0x7f0994007df0).stop 2021-11-03T17:12:37.904 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.907+0000 7f09a575a700 1 -- 172.21.15.27:0/3371524465 shutdown_connections 2021-11-03T17:12:37.904 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.907+0000 7f09a575a700 1 --2- 172.21.15.27:0/3371524465 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f098805c300 0x7f098805e7c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:37.904 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.907+0000 7f09a575a700 1 --2- 172.21.15.27:0/3371524465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09a0060270 0x7f09a00f6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:37.904 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.907+0000 7f09a575a700 1 --2- 172.21.15.27:0/3371524465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09a00f96a0 0x7f09a00f50b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:37.904 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.907+0000 7f09a575a700 1 -- 172.21.15.27:0/3371524465 >> 172.21.15.27:0/3371524465 conn(0x7f09a00f0b90 msgr2=0x7f09a00f2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:12:37.905 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.907+0000 7f09a575a700 1 -- 172.21.15.27:0/3371524465 shutdown_connections 2021-11-03T17:12:37.905 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:37.907+0000 7f09a575a700 1 -- 172.21.15.27:0/3371524465 wait complete. 2021-11-03T17:12:37.912 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:12:38.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.011+0000 7feae9acc700 1 -- 172.21.15.27:0/2335902037 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feae40f5430 msgr2=0x7feae40f5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:38.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.011+0000 7feae9acc700 1 --2- 172.21.15.27:0/2335902037 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feae40f5430 0x7feae40f5850 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7fead8004660 tx=0x7fead8009b30).stop 2021-11-03T17:12:38.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.011+0000 7feae9acc700 1 -- 172.21.15.27:0/2335902037 shutdown_connections 2021-11-03T17:12:38.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.011+0000 7feae9acc700 1 --2- 172.21.15.27:0/2335902037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feae40f66d0 0x7feae40f6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:38.009 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.011+0000 7feae9acc700 1 --2- 172.21.15.27:0/2335902037 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feae40f5430 0x7feae40f5850 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:38.009 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.011+0000 7feae9acc700 1 -- 172.21.15.27:0/2335902037 >> 172.21.15.27:0/2335902037 conn(0x7feae40f0b90 msgr2=0x7feae40f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:12:38.009 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.012+0000 7feae9acc700 1 -- 172.21.15.27:0/2335902037 shutdown_connections 2021-11-03T17:12:38.010 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.012+0000 7feae9acc700 1 -- 172.21.15.27:0/2335902037 wait complete. 2021-11-03T17:12:38.010 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.012+0000 7feae9acc700 1 Processor -- start 2021-11-03T17:12:38.010 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.012+0000 7feae9acc700 1 -- start start 2021-11-03T17:12:38.013 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.013+0000 7feae9acc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feae40f5430 0x7feae4109ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:38.013 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.013+0000 7feae9acc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feae40f66d0 0x7feae410bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:38.013 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.013+0000 7feae9acc700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae40603c0 con 0x7feae40f5430 2021-11-03T17:12:38.013 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.013+0000 7feae9acc700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feae4060500 con 0x7feae40f66d0 2021-11-03T17:12:38.014 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.013+0000 7feae8aca700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feae40f5430 0x7feae4109ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:38.014 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.013+0000 7feae8aca700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feae40f5430 0x7feae4109ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55188/0 (socket says 172.21.15.27:55188) 2021-11-03T17:12:38.014 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.013+0000 7feae8aca700 1 -- 172.21.15.27:0/3805789387 learned_addr learned my addr 172.21.15.27:0/3805789387 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:12:38.014 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.013+0000 7feae3fff700 1 --2- 172.21.15.27:0/3805789387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feae40f66d0 0x7feae410bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:38.015 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.013+0000 7feae8aca700 1 -- 172.21.15.27:0/3805789387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feae40f66d0 msgr2=0x7feae410bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:38.015 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.013+0000 7feae8aca700 1 --2- 172.21.15.27:0/3805789387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feae40f66d0 0x7feae410bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:38.015 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.013+0000 7feae8aca700 1 -- 172.21.15.27:0/3805789387 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead8005040 con 0x7feae40f5430 2021-11-03T17:12:38.015 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.013+0000 7feae8aca700 1 --2- 172.21.15.27:0/3805789387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feae40f5430 0x7feae4109ab0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7fead8013a70 tx=0x7fead8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:12:38.016 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.013+0000 7feae17fa700 1 -- 172.21.15.27:0/3805789387 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fead8018070 con 0x7feae40f5430 2021-11-03T17:12:38.016 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.013+0000 7feae9acc700 1 -- 172.21.15.27:0/3805789387 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feae410c500 con 0x7feae40f5430 2021-11-03T17:12:38.016 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.013+0000 7feae17fa700 1 -- 172.21.15.27:0/3805789387 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fead80097d0 con 0x7feae40f5430 2021-11-03T17:12:38.016 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.013+0000 7feae17fa700 1 -- 172.21.15.27:0/3805789387 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fead801de80 con 0x7feae40f5430 2021-11-03T17:12:38.017 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.013+0000 7feae9acc700 1 -- 172.21.15.27:0/3805789387 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feae410ca30 con 0x7feae40f5430 2021-11-03T17:12:38.017 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.014+0000 7feae9acc700 1 -- 172.21.15.27:0/3805789387 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feae4059100 con 0x7feae40f5430 2021-11-03T17:12:38.017 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.015+0000 7feae17fa700 1 -- 172.21.15.27:0/3805789387 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fead8007ba0 con 0x7feae40f5430 2021-11-03T17:12:38.017 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.015+0000 7feae17fa700 1 --2- 172.21.15.27:0/3805789387 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fead405c300 0x7fead405e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:38.018 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.015+0000 7feae17fa700 1 -- 172.21.15.27:0/3805789387 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fead8077af0 con 0x7feae40f5430 2021-11-03T17:12:38.018 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.015+0000 7feae3fff700 1 --2- 172.21.15.27:0/3805789387 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fead405c300 0x7fead405e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:38.018 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.016+0000 7feae3fff700 1 --2- 172.21.15.27:0/3805789387 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fead405c300 0x7fead405e7c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fead0004c40 tx=0x7fead000b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:12:38.019 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.020+0000 7feae17fa700 1 -- 172.21.15.27:0/3805789387 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fead804a7d0 con 0x7feae40f5430 2021-11-03T17:12:38.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.224+0000 7feae9acc700 1 -- 172.21.15.27:0/3805789387 --> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7feae40fdd70 con 0x7fead405c300 2021-11-03T17:12:38.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.238+0000 7feae17fa700 1 -- 172.21.15.27:0/3805789387 <== mgr.14162 v2:172.21.15.27:6800/3802716887 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2684 (secure 0 0 0) 0x7feae40fdd70 con 0x7fead405c300 2021-11-03T17:12:38.234 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE VERSION IMAGE ID CONTAINER ID 2021-11-03T17:12:38.234 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (4m) 20s ago 5m 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:12:38.234 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (5m) 20s ago 5m 16.2.4 8d91d370c2b8 d494472a47c0 2021-11-03T17:12:38.235 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (4m) 23s ago 4m 16.2.4 8d91d370c2b8 f84fd4042530 2021-11-03T17:12:38.235 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (4m) 20s ago 5m 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:12:38.235 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (33s) 20s ago 32s 16.2.4 8d91d370c2b8 509a42d9cc82 2021-11-03T17:12:38.235 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (39s) 20s ago 38s 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:12:38.236 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (36s) 23s ago 36s 16.2.4 8d91d370c2b8 bfd990d6415e 2021-11-03T17:12:38.236 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (43s) 23s ago 42s 16.2.4 8d91d370c2b8 f8e6ab2a5ed4 2021-11-03T17:12:38.236 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:9283 running (6m) 20s ago 6m 16.2.4 8d91d370c2b8 999298a5769c 2021-11-03T17:12:38.236 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8443,9283 running (4m) 23s ago 4m 16.2.4 8d91d370c2b8 722d63932db9 2021-11-03T17:12:38.237 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (6m) 20s ago 6m 16.2.4 8d91d370c2b8 060d5938214f 2021-11-03T17:12:38.237 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (4m) 23s ago 4m 16.2.4 8d91d370c2b8 319323f09834 2021-11-03T17:12:38.237 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (5m) 20s ago 5m 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:12:38.237 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (4m) 23s ago 4m 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:12:38.238 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (3m) 20s ago 3m 16.2.4 8d91d370c2b8 17f79a815a0f 2021-11-03T17:12:38.238 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (3m) 20s ago 3m 16.2.4 8d91d370c2b8 422be0cac0db 2021-11-03T17:12:38.238 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (3m) 20s ago 3m 16.2.4 8d91d370c2b8 c211ef5551c9 2021-11-03T17:12:38.238 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (2m) 23s ago 2m 16.2.4 8d91d370c2b8 b1486ec1b619 2021-11-03T17:12:38.238 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (2m) 23s ago 2m 16.2.4 8d91d370c2b8 8b91230c69e5 2021-11-03T17:12:38.239 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (2m) 23s ago 2m 16.2.4 8d91d370c2b8 4a54c4d6b6f5 2021-11-03T17:12:38.239 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (3m) 20s ago 4m 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:12:38.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.239+0000 7feae9acc700 1 -- 172.21.15.27:0/3805789387 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fead405c300 msgr2=0x7fead405e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:38.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.239+0000 7feae9acc700 1 --2- 172.21.15.27:0/3805789387 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fead405c300 0x7fead405e7c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7fead0004c40 tx=0x7fead000b040).stop 2021-11-03T17:12:38.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.239+0000 7feae9acc700 1 -- 172.21.15.27:0/3805789387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feae40f5430 msgr2=0x7feae4109ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:38.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.239+0000 7feae9acc700 1 --2- 172.21.15.27:0/3805789387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feae40f5430 0x7feae4109ab0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7fead8013a70 tx=0x7fead8009b30).stop 2021-11-03T17:12:38.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.239+0000 7feae9acc700 1 -- 172.21.15.27:0/3805789387 shutdown_connections 2021-11-03T17:12:38.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.239+0000 7feae9acc700 1 --2- 172.21.15.27:0/3805789387 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fead405c300 0x7fead405e7c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:38.242 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.240+0000 7feae9acc700 1 --2- 172.21.15.27:0/3805789387 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feae40f5430 0x7feae4109ab0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:38.242 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.240+0000 7feae9acc700 1 --2- 172.21.15.27:0/3805789387 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feae40f66d0 0x7feae410bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:38.242 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.240+0000 7feae9acc700 1 -- 172.21.15.27:0/3805789387 >> 172.21.15.27:0/3805789387 conn(0x7feae40f0b90 msgr2=0x7feae40fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:12:38.242 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.240+0000 7feae9acc700 1 -- 172.21.15.27:0/3805789387 shutdown_connections 2021-11-03T17:12:38.243 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.240+0000 7feae9acc700 1 -- 172.21.15.27:0/3805789387 wait complete. 2021-11-03T17:12:38.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.342+0000 7f4df495d700 1 -- 172.21.15.27:0/1745039728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4df00f5430 msgr2=0x7f4df00f5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:38.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.342+0000 7f4df495d700 1 --2- 172.21.15.27:0/1745039728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4df00f5430 0x7f4df00f5850 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f4de0004660 tx=0x7f4de0009b30).stop 2021-11-03T17:12:38.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.343+0000 7f4df495d700 1 -- 172.21.15.27:0/1745039728 shutdown_connections 2021-11-03T17:12:38.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.343+0000 7f4df495d700 1 --2- 172.21.15.27:0/1745039728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4df00f66d0 0x7f4df00f6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:12:38.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.343+0000 7f4df495d700 1 --2- 172.21.15.27:0/1745039728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4df00f5430 0x7f4df00f5850 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:38.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.343+0000 7f4df495d700 1 -- 172.21.15.27:0/1745039728 >> 172.21.15.27:0/1745039728 conn(0x7f4df00f0b90 msgr2=0x7f4df00f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:12:38.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.343+0000 7f4df495d700 1 -- 172.21.15.27:0/1745039728 shutdown_connections 2021-11-03T17:12:38.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.343+0000 7f4df495d700 1 -- 172.21.15.27:0/1745039728 wait complete. 2021-11-03T17:12:38.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.343+0000 7f4df495d700 1 Processor -- start 2021-11-03T17:12:38.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.343+0000 7f4df495d700 1 -- start start 2021-11-03T17:12:38.343 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.343+0000 7f4df495d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4df00f5430 0x7f4df0109ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:38.344 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.344+0000 7f4df495d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4df00f66d0 0x7f4df010bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:38.344 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.344+0000 7f4df495d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df0060470 con 0x7f4df00f66d0 2021-11-03T17:12:38.344 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.344+0000 7f4df495d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df00605b0 con 0x7f4df00f5430 2021-11-03T17:12:38.345 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.344+0000 7f4deeffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4df00f66d0 0x7f4df010bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:38.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.344+0000 7f4deeffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4df00f66d0 0x7f4df010bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55200/0 (socket says 172.21.15.27:55200) 2021-11-03T17:12:38.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.344+0000 7f4deeffd700 1 -- 172.21.15.27:0/2925989259 learned_addr learned my addr 172.21.15.27:0/2925989259 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:12:38.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.344+0000 7f4def7fe700 1 --2- 172.21.15.27:0/2925989259 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4df00f5430 0x7f4df0109ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:38.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.344+0000 7f4deeffd700 1 -- 172.21.15.27:0/2925989259 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4df00f5430 msgr2=0x7f4df0109ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:38.347 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.344+0000 7f4deeffd700 1 --2- 172.21.15.27:0/2925989259 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4df00f5430 0x7f4df0109ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:38.347 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.344+0000 7f4deeffd700 1 -- 172.21.15.27:0/2925989259 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de0005040 con 0x7f4df00f66d0 2021-11-03T17:12:38.347 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.344+0000 7f4deeffd700 1 --2- 172.21.15.27:0/2925989259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4df00f66d0 0x7f4df010bfb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f4de400b410 tx=0x7f4de4006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:12:38.347 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.344+0000 7f4dd7fff700 1 -- 172.21.15.27:0/2925989259 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4de40125f0 con 0x7f4df00f66d0 2021-11-03T17:12:38.348 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.344+0000 7f4df495d700 1 -- 172.21.15.27:0/2925989259 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4df010c610 con 0x7f4df00f66d0 2021-11-03T17:12:38.348 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.344+0000 7f4dd7fff700 1 -- 172.21.15.27:0/2925989259 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f4de400be30 con 0x7f4df00f66d0 2021-11-03T17:12:38.348 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.345+0000 7f4dd7fff700 1 -- 172.21.15.27:0/2925989259 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4de4002a70 con 0x7f4df00f66d0 2021-11-03T17:12:38.348 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.345+0000 7f4df495d700 1 -- 172.21.15.27:0/2925989259 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df010cba0 con 0x7f4df00f66d0 2021-11-03T17:12:38.348 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.346+0000 7f4dd7fff700 1 -- 172.21.15.27:0/2925989259 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f4de400f030 con 0x7f4df00f66d0 2021-11-03T17:12:38.349 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.346+0000 7f4df495d700 1 -- 172.21.15.27:0/2925989259 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4df0059100 con 0x7f4df00f66d0 2021-11-03T17:12:38.349 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.346+0000 7f4dd7fff700 1 --2- 172.21.15.27:0/2925989259 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f4dd805c300 0x7f4dd805e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:38.349 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.346+0000 7f4dd7fff700 1 -- 172.21.15.27:0/2925989259 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f4de40762d0 con 0x7f4df00f66d0 2021-11-03T17:12:38.349 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.346+0000 7f4def7fe700 1 --2- 172.21.15.27:0/2925989259 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f4dd805c300 0x7f4dd805e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:38.350 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.347+0000 7f4def7fe700 1 --2- 172.21.15.27:0/2925989259 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f4dd805c300 0x7f4dd805e7c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f4de0013690 tx=0x7f4de0009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:12:38.350 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.352+0000 7f4dd7fff700 1 -- 172.21.15.27:0/2925989259 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f4de4048fb0 con 0x7f4df00f66d0 2021-11-03T17:12:38.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.603+0000 7f4df495d700 1 -- 172.21.15.27:0/2925989259 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4df0045ad0 con 0x7f4df00f66d0 2021-11-03T17:12:38.600 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.603+0000 7f4dd7fff700 1 -- 172.21.15.27:0/2925989259 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+573 (secure 0 0 0) 0x7f4de404c5d0 con 0x7f4df00f66d0 2021-11-03T17:12:38.600 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:12:38.600 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:12:38.601 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 2 2021-11-03T17:12:38.601 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:12:38.601 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:12:38.601 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 2 2021-11-03T17:12:38.602 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:12:38.602 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:12:38.602 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 6 2021-11-03T17:12:38.602 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:12:38.603 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:12:38.603 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:12:38.603 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:12:38.603 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:12:38.604 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 14 2021-11-03T17:12:38.604 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:12:38.604 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:12:38.605 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.605+0000 7f4df495d700 1 -- 172.21.15.27:0/2925989259 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f4dd805c300 msgr2=0x7f4dd805e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:38.605 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.605+0000 7f4df495d700 1 --2- 172.21.15.27:0/2925989259 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f4dd805c300 0x7f4dd805e7c0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f4de0013690 tx=0x7f4de0009b30).stop 2021-11-03T17:12:38.606 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.605+0000 7f4df495d700 1 -- 172.21.15.27:0/2925989259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4df00f66d0 msgr2=0x7f4df010bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:38.606 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.605+0000 7f4df495d700 1 --2- 172.21.15.27:0/2925989259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4df00f66d0 0x7f4df010bfb0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f4de400b410 tx=0x7f4de4006ef0).stop 2021-11-03T17:12:38.606 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.605+0000 7f4df495d700 1 -- 172.21.15.27:0/2925989259 shutdown_connections 2021-11-03T17:12:38.606 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.605+0000 7f4df495d700 1 --2- 172.21.15.27:0/2925989259 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f4dd805c300 0x7f4dd805e7c0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:38.607 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.605+0000 7f4df495d700 1 --2- 172.21.15.27:0/2925989259 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4df00f5430 0x7f4df0109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:38.607 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.605+0000 7f4df495d700 1 --2- 172.21.15.27:0/2925989259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4df00f66d0 0x7f4df010bfb0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:38.607 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.605+0000 7f4df495d700 1 -- 172.21.15.27:0/2925989259 >> 172.21.15.27:0/2925989259 conn(0x7f4df00f0b90 msgr2=0x7f4df00fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:12:38.607 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.605+0000 7f4df495d700 1 -- 172.21.15.27:0/2925989259 shutdown_connections 2021-11-03T17:12:38.608 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.605+0000 7f4df495d700 1 -- 172.21.15.27:0/2925989259 wait complete. 2021-11-03T17:12:38.721 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.725+0000 7f5e8c95c700 1 -- 172.21.15.27:0/3303743477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e880f5430 msgr2=0x7f5e880f5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:38.722 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.725+0000 7f5e8c95c700 1 --2- 172.21.15.27:0/3303743477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e880f5430 0x7f5e880f5850 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f5e78004660 tx=0x7f5e78009b30).stop 2021-11-03T17:12:38.722 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.725+0000 7f5e8c95c700 1 -- 172.21.15.27:0/3303743477 shutdown_connections 2021-11-03T17:12:38.722 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.725+0000 7f5e8c95c700 1 --2- 172.21.15.27:0/3303743477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e880f66d0 0x7f5e880f6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:38.722 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.725+0000 7f5e8c95c700 1 --2- 172.21.15.27:0/3303743477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e880f5430 0x7f5e880f5850 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:38.723 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.725+0000 7f5e8c95c700 1 -- 172.21.15.27:0/3303743477 >> 172.21.15.27:0/3303743477 conn(0x7f5e880f0b90 msgr2=0x7f5e880f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:12:38.723 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.725+0000 7f5e8c95c700 1 -- 172.21.15.27:0/3303743477 shutdown_connections 2021-11-03T17:12:38.723 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.725+0000 7f5e8c95c700 1 -- 172.21.15.27:0/3303743477 wait complete. 2021-11-03T17:12:38.723 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.725+0000 7f5e8c95c700 1 Processor -- start 2021-11-03T17:12:38.724 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.726+0000 7f5e8c95c700 1 -- start start 2021-11-03T17:12:38.726 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.726+0000 7f5e8c95c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e880f5430 0x7f5e88109ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:38.726 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.726+0000 7f5e8c95c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e880f66d0 0x7f5e8810bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:38.726 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.726+0000 7f5e8c95c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e880603c0 con 0x7f5e880f5430 2021-11-03T17:12:38.727 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.726+0000 7f5e8c95c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e88060500 con 0x7f5e880f66d0 2021-11-03T17:12:38.727 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.726+0000 7f5e877fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e880f5430 0x7f5e88109ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:38.727 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.726+0000 7f5e877fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e880f5430 0x7f5e88109ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55208/0 (socket says 172.21.15.27:55208) 2021-11-03T17:12:38.727 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.726+0000 7f5e877fe700 1 -- 172.21.15.27:0/2263972858 learned_addr learned my addr 172.21.15.27:0/2263972858 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:12:38.728 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.726+0000 7f5e86ffd700 1 --2- 172.21.15.27:0/2263972858 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e880f66d0 0x7f5e8810bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:38.728 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.726+0000 7f5e877fe700 1 -- 172.21.15.27:0/2263972858 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e880f66d0 msgr2=0x7f5e8810bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:38.728 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.726+0000 7f5e877fe700 1 --2- 172.21.15.27:0/2263972858 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e880f66d0 0x7f5e8810bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:38.728 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.726+0000 7f5e877fe700 1 -- 172.21.15.27:0/2263972858 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e78005040 con 0x7f5e880f5430 2021-11-03T17:12:38.728 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.726+0000 7f5e877fe700 1 --2- 172.21.15.27:0/2263972858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e880f5430 0x7f5e88109ab0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f5e78013a70 tx=0x7f5e78009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:12:38.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.727+0000 7f5e6ffff700 1 -- 172.21.15.27:0/2263972858 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e78018070 con 0x7f5e880f5430 2021-11-03T17:12:38.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.727+0000 7f5e6ffff700 1 -- 172.21.15.27:0/2263972858 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f5e780097d0 con 0x7f5e880f5430 2021-11-03T17:12:38.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.727+0000 7f5e8c95c700 1 -- 172.21.15.27:0/2263972858 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e8810c500 con 0x7f5e880f5430 2021-11-03T17:12:38.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.727+0000 7f5e6ffff700 1 -- 172.21.15.27:0/2263972858 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e7801de80 con 0x7f5e880f5430 2021-11-03T17:12:38.730 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.727+0000 7f5e8c95c700 1 -- 172.21.15.27:0/2263972858 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e8810ca30 con 0x7f5e880f5430 2021-11-03T17:12:38.730 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.728+0000 7f5e8c95c700 1 -- 172.21.15.27:0/2263972858 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e88045ad0 con 0x7f5e880f5430 2021-11-03T17:12:38.730 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.729+0000 7f5e6ffff700 1 -- 172.21.15.27:0/2263972858 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f5e78007ba0 con 0x7f5e880f5430 2021-11-03T17:12:38.730 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.729+0000 7f5e6ffff700 1 --2- 172.21.15.27:0/2263972858 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f5e7005c300 0x7f5e7005e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:12:38.731 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.729+0000 7f5e6ffff700 1 -- 172.21.15.27:0/2263972858 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f5e78078030 con 0x7f5e880f5430 2021-11-03T17:12:38.731 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.729+0000 7f5e86ffd700 1 --2- 172.21.15.27:0/2263972858 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f5e7005c300 0x7f5e7005e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:12:38.731 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.730+0000 7f5e86ffd700 1 --2- 172.21.15.27:0/2263972858 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f5e7005c300 0x7f5e7005e7c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f5e7c004c40 tx=0x7f5e7c00b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:12:38.732 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.733+0000 7f5e6ffff700 1 -- 172.21.15.27:0/2263972858 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f5e7804ae00 con 0x7f5e880f5430 2021-11-03T17:12:38.940 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.944+0000 7f5e8c95c700 1 -- 172.21.15.27:0/2263972858 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5e880fdd70 con 0x7f5e880f5430 2021-11-03T17:12:38.940 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.944+0000 7f5e6ffff700 1 -- 172.21.15.27:0/2263972858 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1627 (secure 0 0 0) 0x7f5e7804e420 con 0x7f5e880f5430 2021-11-03T17:12:38.941 INFO:teuthology.orchestra.run.smithi027.stdout:e13 2021-11-03T17:12:38.941 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:12:38.941 INFO:teuthology.orchestra.run.smithi027.stdout:compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:12:38.941 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:12:38.942 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:12:38.942 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:12:38.942 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:12:38.942 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 9 2021-11-03T17:12:38.943 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 2021-11-03T17:12:38.943 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:12:38.943 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:12:05.026191+0000 2021-11-03T17:12:38.943 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:12:38.943 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:12:38.944 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:12:38.944 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:12:38.944 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:12:38.944 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:12:38.944 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:12:38.945 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 0 2021-11-03T17:12:38.945 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:12:38.945 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 2 2021-11-03T17:12:38.945 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:12:38.946 INFO:teuthology.orchestra.run.smithi027.stdout:up {0=24273,1=14436} 2021-11-03T17:12:38.946 INFO:teuthology.orchestra.run.smithi027.stdout:failed 2021-11-03T17:12:38.946 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:12:38.946 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:12:38.947 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:12:38.947 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:12:38.947 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:12:38.947 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:12:38.947 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:12:38.948 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{0:24273} state up:active seq 3 join_fscid=1 addr [v2:172.21.15.115:6824/1621245036,v1:172.21.15.115:6825/1621245036]] 2021-11-03T17:12:38.948 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 3 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060]] 2021-11-03T17:12:38.948 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:12:38.948 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:12:38.948 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:12:38.949 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:12:38.949 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14448} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6828/1337044739,v1:172.21.15.27:6829/1337044739]] 2021-11-03T17:12:38.949 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:24289} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.115:6826/3367380088,v1:172.21.15.115:6827/3367380088]] 2021-11-03T17:12:38.950 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.946+0000 7f5e8c95c700 1 -- 172.21.15.27:0/2263972858 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f5e7005c300 msgr2=0x7f5e7005e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:38.950 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.946+0000 7f5e8c95c700 1 --2- 172.21.15.27:0/2263972858 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f5e7005c300 0x7f5e7005e7c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f5e7c004c40 tx=0x7f5e7c00b040).stop 2021-11-03T17:12:38.951 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.946+0000 7f5e8c95c700 1 -- 172.21.15.27:0/2263972858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e880f5430 msgr2=0x7f5e88109ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:12:38.951 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.946+0000 7f5e8c95c700 1 --2- 172.21.15.27:0/2263972858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e880f5430 0x7f5e88109ab0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f5e78013a70 tx=0x7f5e78009b30).stop 2021-11-03T17:12:38.951 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.946+0000 7f5e8c95c700 1 -- 172.21.15.27:0/2263972858 shutdown_connections 2021-11-03T17:12:38.951 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.946+0000 7f5e8c95c700 1 --2- 172.21.15.27:0/2263972858 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f5e7005c300 0x7f5e7005e7c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:38.952 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.946+0000 7f5e8c95c700 1 --2- 172.21.15.27:0/2263972858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5e880f5430 0x7f5e88109ab0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:38.952 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.946+0000 7f5e8c95c700 1 --2- 172.21.15.27:0/2263972858 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5e880f66d0 0x7f5e8810bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:12:38.952 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.946+0000 7f5e8c95c700 1 -- 172.21.15.27:0/2263972858 >> 172.21.15.27:0/2263972858 conn(0x7f5e880f0b90 msgr2=0x7f5e880fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:12:38.952 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.947+0000 7f5e8c95c700 1 -- 172.21.15.27:0/2263972858 shutdown_connections 2021-11-03T17:12:38.953 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:12:38.947+0000 7f5e8c95c700 1 -- 172.21.15.27:0/2263972858 wait complete. 2021-11-03T17:12:38.953 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 13 2021-11-03T17:12:38.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:38 smithi115 conmon[34670]: audit 2021-11-03T17:12:37.906396+0000 mgr.smithi027.ufhhvb (mgr.14162) 246 : audit 2021-11-03T17:12:38.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:38 smithi115 conmon[34670]: [DBG] from='client.14510 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:12:38.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:38 smithi115 conmon[34670]: audit 2021-11-03T17:12:38.227188+0000 mgr.smithi027.ufhhvb (mgr.14162) 247 : audit [DBG] from='client.14514 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:12:38.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:38 smithi115 conmon[34670]: cluster 2021-11-03 2021-11-03T17:12:38.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:38 smithi115 conmon[34670]: T17:12:38.583337+0000 mgr.smithi027.ufhhvb (mgr.14162) 248 : cluster [DBG] pgmap v206: 65 pgs: 65 active+clean; 9.1 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2021-11-03T17:12:38.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:38 smithi115 conmon[34670]: audit 2021-11-03T17:12: 2021-11-03T17:12:38.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:38 smithi115 conmon[34670]: 38.605233+0000 mon.smithi027 (mon.0) 553 : 2021-11-03T17:12:38.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:38 smithi115 conmon[34670]: audit [DBG] from='client.? 172.21.15.27:0/2925989259' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:12:39.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:38 smithi027 conmon[32322]: audit 2021-11-03T17:12:37.906396+0000 mgr.smithi027.ufhhvb (mgr.14162) 246 : audit [DBG] from='client.14510 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:12:39.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:38 smithi027 conmon[32322]: audit 2021-11-03T17:12:38.227188+0000 mgr.smithi027.ufhhvb (mgr.14162) 247 : audit [DBG] from='client.14514 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:12:39.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:38 smithi027 conmon[32322]: cluster 2021-11-03T17:12:38.583337+0000 mgr.smithi027.ufhhvb (mgr.14162) 248 : cluster [DBG] pgmap v206: 65 pgs: 65 active+clean; 9.1 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2021-11-03T17:12:39.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:38 smithi027 conmon[32322]: audit 2021-11-03T17:12:38.605233+0000 mon.smithi027 (mon.0) 553 : audit [DBG] from='client.? 172.21.15.27:0/2925989259' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:12:39.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:39 smithi115 conmon[34670]: audit 2021-11-03T17:12:38.946112+0000 mon.smithi027 (mon.0) 554 : 2021-11-03T17:12:39.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:39 smithi115 conmon[34670]: audit [DBG] from='client.? 172.21.15.27:0/2263972858' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:12:40.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:39 smithi027 conmon[32322]: audit 2021-11-03T17:12:38.946112+0000 mon.smithi027 ( 2021-11-03T17:12:40.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:39 smithi027 conmon[32322]: mon.0) 554 : audit [DBG] from='client.? 172.21.15.27:0/2263972858' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:12:40.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:40 smithi115 conmon[34670]: cluster 2021-11-03T17:12:40.583743+0000 mgr.smithi027.ufhhvb (mgr.14162) 249 : cluster [DBG] 2021-11-03T17:12:40.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:40 smithi115 conmon[34670]: pgmap v207: 65 pgs: 65 active+clean; 9.1 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2021-11-03T17:12:41.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:40 smithi027 conmon[32322]: cluster 2021-11-03T17:12:40 2021-11-03T17:12:41.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:40 smithi027 conmon[32322]: .583743+0000 mgr.smithi027.ufhhvb (mgr.14162) 249 : cluster [DBG] pgmap v207: 65 pgs: 65 active+clean; 9.1 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2021-11-03T17:12:43.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:43 smithi027 conmon[32322]: cluster 2021-11-03T17:12:42.584216+0000 mgr.smithi027.ufhhvb (mgr 2021-11-03T17:12:43.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:43 smithi027 conmon[32322]: .14162) 250 : cluster [DBG] pgmap v208: 65 pgs: 65 active+clean; 9.1 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:12:43.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:43 smithi115 conmon[34670]: cluster 2021-11-03T17:12:42.584216+0000 mgr.smithi027.ufhhvb (mgr.14162) 250 : cluster [DBG] pgmap v208: 65 pgs: 65 active+clean; 9.1 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:12:43.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:43 smithi115 conmon[34670]: 2021-11-03T17:12:44.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:44 smithi115 conmon[34670]: cluster 2021-11-03T17:12:44.584602+0000 2021-11-03T17:12:44.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:44 smithi115 conmon[34670]: mgr.smithi027.ufhhvb (mgr.14162) 251 : cluster [DBG] pgmap v209: 65 pgs: 65 active+clean; 9.1 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:12:45.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:44 smithi027 conmon[32322]: cluster 2021-11-03T17:12:44.584602+0000 mgr.smithi027.ufhhvb (mgr.14162) 251 : cluster 2021-11-03T17:12:45.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:44 smithi027 conmon[32322]: [DBG] pgmap v209: 65 pgs: 65 active+clean; 9.1 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:12:46.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:46 smithi115 conmon[34670]: cluster 2021-11-03T17:12:46.585141 2021-11-03T17:12:46.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:46 smithi115 conmon[34670]: +0000 mgr.smithi027.ufhhvb (mgr.14162) 252 : cluster [DBG] pgmap v210: 65 pgs: 65 active+clean; 9.1 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:12:47.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:46 smithi027 conmon[32322]: cluster 2021-11-03T17:12 2021-11-03T17:12:47.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:46 smithi027 conmon[32322]: :46.585141+0000 mgr.smithi027.ufhhvb (mgr.14162) 252 : cluster [DBG] pgmap v210: 65 pgs: 65 active+clean; 9.1 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:12:48.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:48 smithi115 conmon[34670]: cluster 2021-11-03T17:12:48.585551+0000 mgr.smithi027.ufhhvb (mgr.14162) 253 : cluster 2021-11-03T17:12:48.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:48 smithi115 conmon[34670]: [DBG] pgmap v211: 65 pgs: 65 active+clean; 9.1 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:12:49.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:48 smithi027 conmon[32322]: cluster 2021-11-03T17:12:48.585551+0000 mgr.smithi027.ufhhvb (mgr.14162) 253 : cluster [DBG] pgmap v211: 65 pgs: 65 active+clean; 9.1 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:12:50.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:50 smithi115 conmon[34670]: cluster 2021-11-03T17:12:50.585949+0000 mgr.smithi027.ufhhvb (mgr.14162) 254 : cluster [DBG] pgmap v212: 65 pgs: 65 active+clean; 9.1 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:12:50.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:50 smithi115 conmon[34670]: 2021-11-03T17:12:51.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:50 smithi027 conmon[32322]: cluster 2021-11-03T17:12: 2021-11-03T17:12:51.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:50 smithi027 conmon[32322]: 50.585949+0000 mgr.smithi027.ufhhvb (mgr.14162) 254 : cluster [DBG] pgmap v212: 65 pgs: 65 active+clean; 9.1 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:12:53.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:53 smithi027 conmon[32322]: cluster 2021-11-03T17:12:52.586428+0000 mgr.smithi027.ufhhvb (mgr.14162) 255 : cluster [DBG] pgmap v213: 65 pgs: 65 active+clean; 9.1 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:12:53.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:53 smithi027 conmon[32322]: audit 2021-11-03T17:12:52.748117+0000 mon.smithi027 (mon.0) 555 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:12:53.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:53 smithi027 conmon[32322]: audit 2021-11-03T17:12:52.748413+0000 mon.smithi027 (mon.0) 556 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:12:53.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:53 smithi115 conmon[34670]: cluster 2021-11-03T17:12:52.586428+0000 mgr.smithi027.ufhhvb (mgr 2021-11-03T17:12:53.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:53 smithi115 conmon[34670]: .14162) 255 : cluster [DBG] pgmap v213: 65 pgs: 65 active+clean; 9.1 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:12:53.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:53 smithi115 conmon[34670]: audit 2021-11-03T17:12:52.748117+0000 mon.smithi027 (mon.0) 555 : audit [INF] 2021-11-03T17:12:53.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:53 smithi115 conmon[34670]: from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:12:53.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:53 smithi115 conmon[34670]: audit 2021-11-03T17:12:53.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:53 smithi115 conmon[34670]: 2021-11-03T17:12:52.748413+0000 mon.smithi027 2021-11-03T17:12:53.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:53 smithi115 conmon[34670]: (mon.0) 556 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:12:54.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:54 smithi115 conmon[34670]: cluster 2021-11-03T17:12:54.586889+0000 mgr.smithi027.ufhhvb 2021-11-03T17:12:54.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:54 smithi115 conmon[34670]: (mgr.14162) 256 : cluster [DBG] pgmap v214: 65 pgs: 65 active+clean; 9.1 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:12:55.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:54 smithi027 conmon[32322]: cluster 2021-11-03T17:12:54. 2021-11-03T17:12:55.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:54 smithi027 conmon[32322]: 586889+0000 mgr.smithi027.ufhhvb (mgr.14162) 256 : cluster [DBG] pgmap v214: 65 pgs: 65 active+clean; 9.1 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:12:56.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:56 smithi115 conmon[34670]: cluster 2021-11-03T17:12:56.587357+0000 2021-11-03T17:12:56.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:56 smithi115 conmon[34670]: mgr.smithi027.ufhhvb (mgr.14162) 257 : cluster [DBG] pgmap v215: 65 pgs: 65 active+clean; 9.1 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:12:57.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:56 smithi027 conmon[32322]: cluster 2021-11-03T17 2021-11-03T17:12:57.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:56 smithi027 conmon[32322]: :12:56.587357+0000 mgr.smithi027.ufhhvb (mgr.14162) 257 : cluster [DBG] pgmap v215: 65 pgs: 65 active+clean; 9.1 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:12:58.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:58 smithi115 conmon[34670]: cluster 2021-11-03T17:12:58.587766+0000 mgr.smithi027.ufhhvb 2021-11-03T17:12:58.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:12:58 smithi115 conmon[34670]: (mgr.14162) 258 : cluster [DBG] pgmap v216: 65 pgs: 65 active+clean; 9.1 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:12:59.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:58 smithi027 conmon[32322]: cluster 2021-11-03T17:12:58.587766+0000 mgr.smithi027.ufhhvb 2021-11-03T17:12:59.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:12:58 smithi027 conmon[32322]: (mgr.14162) 258 : cluster [DBG] pgmap v216: 65 pgs: 65 active+clean; 9.1 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:13:00.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:00 smithi115 conmon[34670]: cluster 2021-11-03T17:13:00.588157+0000 mgr.smithi027.ufhhvb ( 2021-11-03T17:13:00.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:00 smithi115 conmon[34670]: mgr.14162) 259 : cluster [DBG] pgmap v217: 65 pgs: 65 active+clean; 9.1 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:13:01.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:00 smithi027 conmon[32322]: cluster 2021-11-03T17:13:00 2021-11-03T17:13:01.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:00 smithi027 conmon[32322]: .588157+0000 mgr.smithi027.ufhhvb (mgr.14162) 259 : cluster [DBG] pgmap v217: 65 pgs: 65 active+clean; 9.1 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2021-11-03T17:13:03.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:03 smithi027 conmon[32322]: cluster 2021-11-03T17:13:02.588664+0000 mgr.smithi027.ufhhvb (mgr.14162) 260 : cluster [DBG] pgmap v218: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2021-11-03T17:13:03.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:03 smithi115 conmon[34670]: cluster 2021-11-03T17:13:02.588664+0000 mgr.smithi027.ufhhvb ( 2021-11-03T17:13:03.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:03 smithi115 conmon[34670]: mgr.14162) 260 : cluster [DBG] pgmap v218: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2021-11-03T17:13:04.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:04 smithi115 conmon[34670]: cluster 2021-11-03T17:13:04.589112+0000 mgr.smithi027.ufhhvb 2021-11-03T17:13:04.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:04 smithi115 conmon[34670]: (mgr.14162) 261 : cluster [DBG] pgmap v219: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2021-11-03T17:13:05.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:04 smithi027 conmon[32322]: cluster 2021-11-03T17:13:04.589112+0000 mgr.smithi027.ufhhvb (mgr.14162) 261 : cluster [DBG] pgmap v219: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2021-11-03T17:13:06.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:06 smithi115 conmon[34670]: cluster 2021-11-03T17:13:06.589676+0000 mgr.smithi027.ufhhvb (mgr.14162) 262 : cluster [DBG] pgmap v220: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2021-11-03T17:13:07.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:06 smithi027 conmon[32322]: cluster 2021-11-03T17:13:06. 2021-11-03T17:13:07.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:06 smithi027 conmon[32322]: 589676+0000 mgr.smithi027.ufhhvb (mgr.14162) 262 : cluster [DBG] pgmap v220: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2021-11-03T17:13:07.652 INFO:tasks.workunit.client.1.smithi115.stderr:Note: switching to '6e90212d8510e2d506395a8c82057d0d1787f3a4'. 2021-11-03T17:13:07.653 INFO:tasks.workunit.client.1.smithi115.stderr: 2021-11-03T17:13:07.653 INFO:tasks.workunit.client.1.smithi115.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2021-11-03T17:13:07.653 INFO:tasks.workunit.client.1.smithi115.stderr:changes and commit them, and you can discard any commits you make in this 2021-11-03T17:13:07.653 INFO:tasks.workunit.client.1.smithi115.stderr:state without impacting any branches by switching back to a branch. 2021-11-03T17:13:07.654 INFO:tasks.workunit.client.1.smithi115.stderr: 2021-11-03T17:13:07.654 INFO:tasks.workunit.client.1.smithi115.stderr:If you want to create a new branch to retain commits you create, you may 2021-11-03T17:13:07.654 INFO:tasks.workunit.client.1.smithi115.stderr:do so (now or later) by using -c with the switch command. Example: 2021-11-03T17:13:07.654 INFO:tasks.workunit.client.1.smithi115.stderr: 2021-11-03T17:13:07.655 INFO:tasks.workunit.client.1.smithi115.stderr: git switch -c 2021-11-03T17:13:07.655 INFO:tasks.workunit.client.1.smithi115.stderr: 2021-11-03T17:13:07.655 INFO:tasks.workunit.client.1.smithi115.stderr:Or undo this operation with: 2021-11-03T17:13:07.655 INFO:tasks.workunit.client.1.smithi115.stderr: 2021-11-03T17:13:07.655 INFO:tasks.workunit.client.1.smithi115.stderr: git switch - 2021-11-03T17:13:07.656 INFO:tasks.workunit.client.1.smithi115.stderr: 2021-11-03T17:13:07.656 INFO:tasks.workunit.client.1.smithi115.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2021-11-03T17:13:07.656 INFO:tasks.workunit.client.1.smithi115.stderr: 2021-11-03T17:13:07.656 INFO:tasks.workunit.client.1.smithi115.stderr:HEAD is now at 6e90212d851 Merge branch 'test-container-tools-3.1' of https://github.com/liewegas/ceph into wip-sage-testing-2021-11-03-1039 2021-11-03T17:13:07.658 DEBUG:teuthology.orchestra.run.smithi115:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2021-11-03T17:13:07.684 INFO:tasks.workunit.client.1.smithi115.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2021-11-03T17:13:07.686 INFO:tasks.workunit.client.1.smithi115.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2021-11-03T17:13:07.686 INFO:tasks.workunit.client.1.smithi115.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2021-11-03T17:13:08.188 INFO:tasks.workunit.client.1.smithi115.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2021-11-03T17:13:08.253 INFO:tasks.workunit.client.1.smithi115.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2021-11-03T17:13:08.290 INFO:tasks.workunit.client.1.smithi115.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2021-11-03T17:13:08.291 INFO:tasks.workunit.client.1.smithi115.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2021-11-03T17:13:08.291 INFO:tasks.workunit.client.1.smithi115.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2021-11-03T17:13:08.326 INFO:tasks.workunit.client.1.smithi115.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2021-11-03T17:13:08.329 DEBUG:teuthology.orchestra.run.smithi115:> set -ex 2021-11-03T17:13:08.329 DEBUG:teuthology.orchestra.run.smithi115:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2021-11-03T17:13:08.384 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2021-11-03T17:13:08.388 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2021-11-03T17:13:08.388 DEBUG:teuthology.orchestra.run.smithi115:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6e90212d8510e2d506395a8c82057d0d1787f3a4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2021-11-03T17:13:08.445 INFO:tasks.workunit.client.1.smithi115.stderr:+ mkdir -p fsstress 2021-11-03T17:13:08.446 INFO:tasks.workunit.client.1.smithi115.stderr:+ pushd fsstress 2021-11-03T17:13:08.447 INFO:tasks.workunit.client.1.smithi115.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2021-11-03T17:13:08.447 INFO:tasks.workunit.client.1.smithi115.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2021-11-03T17:13:08.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:08 smithi115 conmon[34670]: cluster 2021-11-03T17:13:08.590141+0000 mgr.smithi027.ufhhvb (mgr.14162) 263 : cluster [DBG] pgmap v221: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2021-11-03T17:13:09.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:08 smithi027 conmon[32322]: cluster 2021-11-03T17:13:09.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:08 smithi027 conmon[32322]: 2021-11-03T17:13:08.590141+0000 mgr.smithi027.ufhhvb (mgr.14162) 263 : cluster [DBG] pgmap v221: 65 pgs: 65 active+clean; 12 KiB data, 33 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2021-11-03T17:13:09.093 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.097+0000 7f28cec3e700 1 -- 172.21.15.27:0/1745912090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28c80f8520 msgr2=0x7f28c80f46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:13:09.094 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.097+0000 7f28cec3e700 1 --2- 172.21.15.27:0/1745912090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28c80f8520 0x7f28c80f46a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f28b80045a0 tx=0x7f28b8009b30).stop 2021-11-03T17:13:09.094 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.097+0000 7f28cec3e700 1 -- 172.21.15.27:0/1745912090 shutdown_connections 2021-11-03T17:13:09.094 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.097+0000 7f28cec3e700 1 --2- 172.21.15.27:0/1745912090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28c80f8520 0x7f28c80f46a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:09.094 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.097+0000 7f28cec3e700 1 --2- 172.21.15.27:0/1745912090 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28c80f7b80 0x7f28c80f7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:09.095 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.097+0000 7f28cec3e700 1 -- 172.21.15.27:0/1745912090 >> 172.21.15.27:0/1745912090 conn(0x7f28c80f0320 msgr2=0x7f28c80f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:13:09.095 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.098+0000 7f28cec3e700 1 -- 172.21.15.27:0/1745912090 shutdown_connections 2021-11-03T17:13:09.095 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.098+0000 7f28cec3e700 1 -- 172.21.15.27:0/1745912090 wait complete. 2021-11-03T17:13:09.096 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.098+0000 7f28cec3e700 1 Processor -- start 2021-11-03T17:13:09.097 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.098+0000 7f28cec3e700 1 -- start start 2021-11-03T17:13:09.097 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.099+0000 7f28cec3e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28c80f7b80 0x7f28c80f6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:13:09.097 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.099+0000 7f28cec3e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28c80f8520 0x7f28c80f7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:13:09.097 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.099+0000 7f28cec3e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c81146d0 con 0x7f28c80f8520 2021-11-03T17:13:09.098 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.099+0000 7f28cec3e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28c8114840 con 0x7f28c80f7b80 2021-11-03T17:13:09.098 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.099+0000 7f28cdc3c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28c80f7b80 0x7f28c80f6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:13:09.098 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.099+0000 7f28cdc3c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28c80f7b80 0x7f28c80f6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.27:46384/0 (socket says 172.21.15.27:46384) 2021-11-03T17:13:09.098 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.099+0000 7f28cdc3c700 1 -- 172.21.15.27:0/3817030227 learned_addr learned my addr 172.21.15.27:0/3817030227 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:13:09.099 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.099+0000 7f28cdc3c700 1 -- 172.21.15.27:0/3817030227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28c80f8520 msgr2=0x7f28c80f7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:13:09.099 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.099+0000 7f28cdc3c700 1 --2- 172.21.15.27:0/3817030227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28c80f8520 0x7f28c80f7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:13:09.099 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.099+0000 7f28cdc3c700 1 -- 172.21.15.27:0/3817030227 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28b8005040 con 0x7f28c80f7b80 2021-11-03T17:13:09.099 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.100+0000 7f28cdc3c700 1 --2- 172.21.15.27:0/3817030227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28c80f7b80 0x7f28c80f6f70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f28c4002700 tx=0x7f28c4009cc0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:13:09.100 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.101+0000 7f28be7fc700 1 -- 172.21.15.27:0/3817030227 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28c4009320 con 0x7f28c80f7b80 2021-11-03T17:13:09.102 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.101+0000 7f28be7fc700 1 -- 172.21.15.27:0/3817030227 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f28c400f040 con 0x7f28c80f7b80 2021-11-03T17:13:09.102 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.101+0000 7f28be7fc700 1 -- 172.21.15.27:0/3817030227 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28c4014800 con 0x7f28c80f7b80 2021-11-03T17:13:09.103 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.102+0000 7f28cec3e700 1 -- 172.21.15.27:0/3817030227 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28c80f5b60 con 0x7f28c80f7b80 2021-11-03T17:13:09.103 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.102+0000 7f28cec3e700 1 -- 172.21.15.27:0/3817030227 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28c80f60f0 con 0x7f28c80f7b80 2021-11-03T17:13:09.103 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.103+0000 7f28cec3e700 1 -- 172.21.15.27:0/3817030227 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28c8059100 con 0x7f28c80f7b80 2021-11-03T17:13:09.103 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.103+0000 7f28be7fc700 1 -- 172.21.15.27:0/3817030227 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f28c4009480 con 0x7f28c80f7b80 2021-11-03T17:13:09.104 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.104+0000 7f28be7fc700 1 --2- 172.21.15.27:0/3817030227 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f28b405c340 0x7f28b405e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:13:09.104 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.104+0000 7f28be7fc700 1 -- 172.21.15.27:0/3817030227 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f28c4076210 con 0x7f28c80f7b80 2021-11-03T17:13:09.104 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.104+0000 7f28cd43b700 1 --2- 172.21.15.27:0/3817030227 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f28b405c340 0x7f28b405e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:13:09.104 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.105+0000 7f28cd43b700 1 --2- 172.21.15.27:0/3817030227 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f28b405c340 0x7f28b405e800 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f28b8000c00 tx=0x7f28b800b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:13:09.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.108+0000 7f28be7fc700 1 -- 172.21.15.27:0/3817030227 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f28c4048ef0 con 0x7f28c80f7b80 2021-11-03T17:13:09.359 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.364+0000 7f28cec3e700 1 -- 172.21.15.27:0/3817030227 --> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f28c8068910 con 0x7f28b405c340 2021-11-03T17:13:09.362 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.366+0000 7f28be7fc700 1 -- 172.21.15.27:0/3817030227 <== mgr.14162 v2:172.21.15.27:6800/3802716887 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+278 (secure 0 0 0) 0x7f28c8068910 con 0x7f28b405c340 2021-11-03T17:13:09.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.368+0000 7f28cec3e700 1 -- 172.21.15.27:0/3817030227 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f28b405c340 msgr2=0x7f28b405e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:13:09.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.368+0000 7f28cec3e700 1 --2- 172.21.15.27:0/3817030227 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f28b405c340 0x7f28b405e800 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f28b8000c00 tx=0x7f28b800b040).stop 2021-11-03T17:13:09.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.368+0000 7f28cec3e700 1 -- 172.21.15.27:0/3817030227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28c80f7b80 msgr2=0x7f28c80f6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:13:09.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.368+0000 7f28cec3e700 1 --2- 172.21.15.27:0/3817030227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28c80f7b80 0x7f28c80f6f70 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f28c4002700 tx=0x7f28c4009cc0).stop 2021-11-03T17:13:09.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.373+0000 7f28cec3e700 1 -- 172.21.15.27:0/3817030227 shutdown_connections 2021-11-03T17:13:09.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.373+0000 7f28cec3e700 1 --2- 172.21.15.27:0/3817030227 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f28b405c340 0x7f28b405e800 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:09.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.373+0000 7f28cec3e700 1 --2- 172.21.15.27:0/3817030227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f28c80f7b80 0x7f28c80f6f70 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:09.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.373+0000 7f28cec3e700 1 --2- 172.21.15.27:0/3817030227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f28c80f8520 0x7f28c80f7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:13:09.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.373+0000 7f28cec3e700 1 -- 172.21.15.27:0/3817030227 >> 172.21.15.27:0/3817030227 conn(0x7f28c80f0320 msgr2=0x7f28c80fb410 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:13:09.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.373+0000 7f28cec3e700 1 -- 172.21.15.27:0/3817030227 shutdown_connections 2021-11-03T17:13:09.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.373+0000 7f28cec3e700 1 -- 172.21.15.27:0/3817030227 wait complete. 2021-11-03T17:13:09.381 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:13:09.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.530+0000 7fa75a751700 1 -- 172.21.15.27:0/2517532163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7540f55c0 msgr2=0x7fa7540f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:13:09.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.530+0000 7fa75a751700 1 --2- 172.21.15.27:0/2517532163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7540f55c0 0x7fa7540f59e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7fa744004660 tx=0x7fa744009b30).stop 2021-11-03T17:13:09.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.530+0000 7fa75a751700 1 -- 172.21.15.27:0/2517532163 shutdown_connections 2021-11-03T17:13:09.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.530+0000 7fa75a751700 1 --2- 172.21.15.27:0/2517532163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7540f6860 0x7fa7540f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:09.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.530+0000 7fa75a751700 1 --2- 172.21.15.27:0/2517532163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7540f55c0 0x7fa7540f59e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:09.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.530+0000 7fa75a751700 1 -- 172.21.15.27:0/2517532163 >> 172.21.15.27:0/2517532163 conn(0x7fa7540f0d40 msgr2=0x7fa7540f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:13:09.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.530+0000 7fa75a751700 1 -- 172.21.15.27:0/2517532163 shutdown_connections 2021-11-03T17:13:09.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.530+0000 7fa75a751700 1 -- 172.21.15.27:0/2517532163 wait complete. 2021-11-03T17:13:09.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.531+0000 7fa75a751700 1 Processor -- start 2021-11-03T17:13:09.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.531+0000 7fa75a751700 1 -- start start 2021-11-03T17:13:09.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.531+0000 7fa75a751700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7540f55c0 0x7fa754106f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:13:09.530 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.531+0000 7fa75a751700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7540f6860 0x7fa754107420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:13:09.530 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.531+0000 7fa75a751700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7541079d0 con 0x7fa7540f55c0 2021-11-03T17:13:09.530 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.531+0000 7fa75a751700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa754107b10 con 0x7fa7540f6860 2021-11-03T17:13:09.530 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.531+0000 7fa753fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7540f55c0 0x7fa754106f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:13:09.530 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.531+0000 7fa753fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7540f55c0 0x7fa754106f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55226/0 (socket says 172.21.15.27:55226) 2021-11-03T17:13:09.531 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.531+0000 7fa753fff700 1 -- 172.21.15.27:0/3540692150 learned_addr learned my addr 172.21.15.27:0/3540692150 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:13:09.532 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.532+0000 7fa7537fe700 1 --2- 172.21.15.27:0/3540692150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7540f6860 0x7fa754107420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:13:09.533 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.532+0000 7fa7537fe700 1 -- 172.21.15.27:0/3540692150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7540f55c0 msgr2=0x7fa754106f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:13:09.533 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.532+0000 7fa7537fe700 1 --2- 172.21.15.27:0/3540692150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7540f55c0 0x7fa754106f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:09.533 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.532+0000 7fa7537fe700 1 -- 172.21.15.27:0/3540692150 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa744005040 con 0x7fa7540f6860 2021-11-03T17:13:09.533 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.532+0000 7fa7537fe700 1 --2- 172.21.15.27:0/3540692150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7540f6860 0x7fa754107420 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fa74800e7c0 tx=0x7fa74800b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:13:09.534 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.532+0000 7fa750ff9700 1 -- 172.21.15.27:0/3540692150 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7480092e0 con 0x7fa7540f6860 2021-11-03T17:13:09.534 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.532+0000 7fa75a751700 1 -- 172.21.15.27:0/3540692150 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7541084b0 con 0x7fa7540f6860 2021-11-03T17:13:09.534 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.532+0000 7fa750ff9700 1 -- 172.21.15.27:0/3540692150 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fa74800f040 con 0x7fa7540f6860 2021-11-03T17:13:09.534 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.532+0000 7fa750ff9700 1 -- 172.21.15.27:0/3540692150 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa748014880 con 0x7fa7540f6860 2021-11-03T17:13:09.535 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.532+0000 7fa75a751700 1 -- 172.21.15.27:0/3540692150 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa754108a70 con 0x7fa7540f6860 2021-11-03T17:13:09.535 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.533+0000 7fa75a751700 1 -- 172.21.15.27:0/3540692150 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa754045ad0 con 0x7fa7540f6860 2021-11-03T17:13:09.535 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.534+0000 7fa750ff9700 1 -- 172.21.15.27:0/3540692150 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fa74801b030 con 0x7fa7540f6860 2021-11-03T17:13:09.535 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.534+0000 7fa750ff9700 1 --2- 172.21.15.27:0/3540692150 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fa73c05c2a0 0x7fa73c05e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:13:09.536 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.534+0000 7fa750ff9700 1 -- 172.21.15.27:0/3540692150 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fa748075d10 con 0x7fa7540f6860 2021-11-03T17:13:09.536 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.534+0000 7fa753fff700 1 --2- 172.21.15.27:0/3540692150 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fa73c05c2a0 0x7fa73c05e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:13:09.536 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.535+0000 7fa753fff700 1 --2- 172.21.15.27:0/3540692150 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fa73c05c2a0 0x7fa73c05e760 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fa744005010 tx=0x7fa744016040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:13:09.537 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.538+0000 7fa750ff9700 1 -- 172.21.15.27:0/3540692150 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fa7480489f0 con 0x7fa7540f6860 2021-11-03T17:13:09.607 INFO:tasks.workunit.client.1.smithi115.stderr:+ tar xzf ltp-full.tgz 2021-11-03T17:13:09.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.885+0000 7fa75a751700 1 -- 172.21.15.27:0/3540692150 --> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7540faca0 con 0x7fa73c05c2a0 2021-11-03T17:13:09.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.902+0000 7fa750ff9700 1 -- 172.21.15.27:0/3540692150 <== mgr.14162 v2:172.21.15.27:6800/3802716887 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2684 (secure 0 0 0) 0x7fa7540faca0 con 0x7fa73c05c2a0 2021-11-03T17:13:09.899 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE VERSION IMAGE ID CONTAINER ID 2021-11-03T17:13:09.900 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (4m) 51s ago 5m 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:13:09.900 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (5m) 51s ago 5m 16.2.4 8d91d370c2b8 d494472a47c0 2021-11-03T17:13:09.900 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (5m) 55s ago 5m 16.2.4 8d91d370c2b8 f84fd4042530 2021-11-03T17:13:09.900 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (4m) 51s ago 5m 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:13:09.900 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (65s) 51s ago 64s 16.2.4 8d91d370c2b8 509a42d9cc82 2021-11-03T17:13:09.901 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (71s) 51s ago 70s 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:13:09.901 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (68s) 55s ago 68s 16.2.4 8d91d370c2b8 bfd990d6415e 2021-11-03T17:13:09.901 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (74s) 55s ago 74s 16.2.4 8d91d370c2b8 f8e6ab2a5ed4 2021-11-03T17:13:09.901 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:9283 running (7m) 51s ago 7m 16.2.4 8d91d370c2b8 999298a5769c 2021-11-03T17:13:09.902 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8443,9283 running (5m) 55s ago 5m 16.2.4 8d91d370c2b8 722d63932db9 2021-11-03T17:13:09.902 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (7m) 51s ago 7m 16.2.4 8d91d370c2b8 060d5938214f 2021-11-03T17:13:09.902 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (5m) 55s ago 5m 16.2.4 8d91d370c2b8 319323f09834 2021-11-03T17:13:09.902 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (5m) 51s ago 5m 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:13:09.902 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (4m) 55s ago 4m 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:13:09.903 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (4m) 51s ago 4m 16.2.4 8d91d370c2b8 17f79a815a0f 2021-11-03T17:13:09.903 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (4m) 51s ago 4m 16.2.4 8d91d370c2b8 422be0cac0db 2021-11-03T17:13:09.903 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (3m) 51s ago 3m 16.2.4 8d91d370c2b8 c211ef5551c9 2021-11-03T17:13:09.903 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (3m) 55s ago 3m 16.2.4 8d91d370c2b8 b1486ec1b619 2021-11-03T17:13:09.904 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (3m) 55s ago 3m 16.2.4 8d91d370c2b8 8b91230c69e5 2021-11-03T17:13:09.904 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (2m) 55s ago 2m 16.2.4 8d91d370c2b8 4a54c4d6b6f5 2021-11-03T17:13:09.904 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (4m) 51s ago 5m 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:13:09.905 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.904+0000 7fa75a751700 1 -- 172.21.15.27:0/3540692150 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fa73c05c2a0 msgr2=0x7fa73c05e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:13:09.905 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.904+0000 7fa75a751700 1 --2- 172.21.15.27:0/3540692150 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fa73c05c2a0 0x7fa73c05e760 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7fa744005010 tx=0x7fa744016040).stop 2021-11-03T17:13:09.905 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.904+0000 7fa75a751700 1 -- 172.21.15.27:0/3540692150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7540f6860 msgr2=0x7fa754107420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:13:09.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.904+0000 7fa75a751700 1 --2- 172.21.15.27:0/3540692150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7540f6860 0x7fa754107420 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fa74800e7c0 tx=0x7fa74800b040).stop 2021-11-03T17:13:09.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.904+0000 7fa75a751700 1 -- 172.21.15.27:0/3540692150 shutdown_connections 2021-11-03T17:13:09.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.904+0000 7fa75a751700 1 --2- 172.21.15.27:0/3540692150 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fa73c05c2a0 0x7fa73c05e760 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:09.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.904+0000 7fa75a751700 1 --2- 172.21.15.27:0/3540692150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7540f55c0 0x7fa754106f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:09.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.904+0000 7fa75a751700 1 --2- 172.21.15.27:0/3540692150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7540f6860 0x7fa754107420 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:09.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.904+0000 7fa75a751700 1 -- 172.21.15.27:0/3540692150 >> 172.21.15.27:0/3540692150 conn(0x7fa7540f0d40 msgr2=0x7fa75405fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:13:09.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.905+0000 7fa75a751700 1 -- 172.21.15.27:0/3540692150 shutdown_connections 2021-11-03T17:13:09.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:09.905+0000 7fa75a751700 1 -- 172.21.15.27:0/3540692150 wait complete. 2021-11-03T17:13:09.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:09 smithi115 conmon[34670]: audit 2021-11-03T17:13:09.367426+0000 mgr.smithi027.ufhhvb ( 2021-11-03T17:13:09.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:09 smithi115 conmon[34670]: mgr.14162) 264 : audit [DBG] from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:13:10.015 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.019+0000 7fb30359e700 1 -- 172.21.15.27:0/2726162111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3040f5430 msgr2=0x7fb3040f5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:13:10.015 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.019+0000 7fb30359e700 1 --2- 172.21.15.27:0/2726162111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3040f5430 0x7fb3040f5850 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7fb2f4004660 tx=0x7fb2f4009b30).stop 2021-11-03T17:13:10.016 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.019+0000 7fb30359e700 1 -- 172.21.15.27:0/2726162111 shutdown_connections 2021-11-03T17:13:10.016 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.019+0000 7fb30359e700 1 --2- 172.21.15.27:0/2726162111 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3040f66d0 0x7fb3040f6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:10.016 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.019+0000 7fb30359e700 1 --2- 172.21.15.27:0/2726162111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3040f5430 0x7fb3040f5850 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:10.016 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.019+0000 7fb30359e700 1 -- 172.21.15.27:0/2726162111 >> 172.21.15.27:0/2726162111 conn(0x7fb3040f0b90 msgr2=0x7fb3040f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:13:10.017 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.019+0000 7fb30359e700 1 -- 172.21.15.27:0/2726162111 shutdown_connections 2021-11-03T17:13:10.017 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.019+0000 7fb30359e700 1 -- 172.21.15.27:0/2726162111 wait complete. 2021-11-03T17:13:10.020 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.020+0000 7fb30359e700 1 Processor -- start 2021-11-03T17:13:10.020 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.020+0000 7fb30359e700 1 -- start start 2021-11-03T17:13:10.020 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.020+0000 7fb30359e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3040f5430 0x7fb30410beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:13:10.021 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.020+0000 7fb30359e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3040f66d0 0x7fb30410e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:13:10.021 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.020+0000 7fb30359e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3040602b0 con 0x7fb3040f5430 2021-11-03T17:13:10.022 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.020+0000 7fb30359e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3040603f0 con 0x7fb3040f66d0 2021-11-03T17:13:10.022 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.021+0000 7fb301d9b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3040f66d0 0x7fb30410e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:13:10.023 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.021+0000 7fb30259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3040f5430 0x7fb30410beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:13:10.023 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.021+0000 7fb30259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3040f5430 0x7fb30410beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55236/0 (socket says 172.21.15.27:55236) 2021-11-03T17:13:10.024 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.021+0000 7fb30259c700 1 -- 172.21.15.27:0/1560869065 learned_addr learned my addr 172.21.15.27:0/1560869065 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:13:10.024 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.021+0000 7fb30259c700 1 -- 172.21.15.27:0/1560869065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3040f66d0 msgr2=0x7fb30410e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:13:10.024 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.021+0000 7fb30259c700 1 --2- 172.21.15.27:0/1560869065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3040f66d0 0x7fb30410e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:10.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.021+0000 7fb30259c700 1 -- 172.21.15.27:0/1560869065 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f4005040 con 0x7fb3040f5430 2021-11-03T17:13:10.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.021+0000 7fb30259c700 1 --2- 172.21.15.27:0/1560869065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3040f5430 0x7fb30410beb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7fb2f4013a70 tx=0x7fb2f4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:13:10.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.021+0000 7fb2f2ffd700 1 -- 172.21.15.27:0/1560869065 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2f4018070 con 0x7fb3040f5430 2021-11-03T17:13:10.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.021+0000 7fb2f2ffd700 1 -- 172.21.15.27:0/1560869065 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fb2f40097d0 con 0x7fb3040f5430 2021-11-03T17:13:10.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.021+0000 7fb2f2ffd700 1 -- 172.21.15.27:0/1560869065 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2f401de80 con 0x7fb3040f5430 2021-11-03T17:13:10.027 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.021+0000 7fb30359e700 1 -- 172.21.15.27:0/1560869065 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb30410e8a0 con 0x7fb3040f5430 2021-11-03T17:13:10.027 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.022+0000 7fb30359e700 1 -- 172.21.15.27:0/1560869065 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb30410ebf0 con 0x7fb3040f5430 2021-11-03T17:13:10.027 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.023+0000 7fb30359e700 1 -- 172.21.15.27:0/1560869065 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb304045ad0 con 0x7fb3040f5430 2021-11-03T17:13:10.028 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:09 smithi027 conmon[32322]: audit 2021-11-03T17:13:09. 2021-11-03T17:13:10.028 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:09 smithi027 conmon[32322]: 367426+0000 mgr.smithi027.ufhhvb (mgr.14162) 264 : audit [DBG] from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:13:10.029 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.023+0000 7fb2f2ffd700 1 -- 172.21.15.27:0/1560869065 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fb2f4007ba0 con 0x7fb3040f5430 2021-11-03T17:13:10.029 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.024+0000 7fb2f2ffd700 1 --2- 172.21.15.27:0/1560869065 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fb2ec05c300 0x7fb2ec05e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:13:10.029 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.024+0000 7fb2f2ffd700 1 -- 172.21.15.27:0/1560869065 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fb2f4077ad0 con 0x7fb3040f5430 2021-11-03T17:13:10.030 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.025+0000 7fb301d9b700 1 --2- 172.21.15.27:0/1560869065 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fb2ec05c300 0x7fb2ec05e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:13:10.030 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.026+0000 7fb301d9b700 1 --2- 172.21.15.27:0/1560869065 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fb2ec05c300 0x7fb2ec05e7c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fb2f8004c40 tx=0x7fb2f800b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:13:10.030 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.032+0000 7fb2f2ffd700 1 -- 172.21.15.27:0/1560869065 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fb2f404a7b0 con 0x7fb3040f5430 2021-11-03T17:13:10.291 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.296+0000 7fb30359e700 1 -- 172.21.15.27:0/1560869065 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb3040fdbf0 con 0x7fb3040f5430 2021-11-03T17:13:10.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.296+0000 7fb2f2ffd700 1 -- 172.21.15.27:0/1560869065 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+573 (secure 0 0 0) 0x7fb2f404ddd0 con 0x7fb3040f5430 2021-11-03T17:13:10.293 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:13:10.293 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:13:10.293 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 2 2021-11-03T17:13:10.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:13:10.294 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:13:10.294 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 2 2021-11-03T17:13:10.294 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:13:10.295 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:13:10.295 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 6 2021-11-03T17:13:10.295 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:13:10.295 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:13:10.296 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:13:10.296 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:13:10.296 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:13:10.296 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 14 2021-11-03T17:13:10.296 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:13:10.297 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:13:10.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.298+0000 7fb30359e700 1 -- 172.21.15.27:0/1560869065 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fb2ec05c300 msgr2=0x7fb2ec05e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:13:10.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.298+0000 7fb30359e700 1 --2- 172.21.15.27:0/1560869065 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fb2ec05c300 0x7fb2ec05e7c0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7fb2f8004c40 tx=0x7fb2f800b040).stop 2021-11-03T17:13:10.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.298+0000 7fb30359e700 1 -- 172.21.15.27:0/1560869065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3040f5430 msgr2=0x7fb30410beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:13:10.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.298+0000 7fb30359e700 1 --2- 172.21.15.27:0/1560869065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3040f5430 0x7fb30410beb0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7fb2f4013a70 tx=0x7fb2f4009b30).stop 2021-11-03T17:13:10.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.298+0000 7fb30359e700 1 -- 172.21.15.27:0/1560869065 shutdown_connections 2021-11-03T17:13:10.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.298+0000 7fb30359e700 1 --2- 172.21.15.27:0/1560869065 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fb2ec05c300 0x7fb2ec05e7c0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:10.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.298+0000 7fb30359e700 1 --2- 172.21.15.27:0/1560869065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3040f5430 0x7fb30410beb0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:10.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.298+0000 7fb30359e700 1 --2- 172.21.15.27:0/1560869065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3040f66d0 0x7fb30410e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:10.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.298+0000 7fb30359e700 1 -- 172.21.15.27:0/1560869065 >> 172.21.15.27:0/1560869065 conn(0x7fb3040f0b90 msgr2=0x7fb3040fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:13:10.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.298+0000 7fb30359e700 1 -- 172.21.15.27:0/1560869065 shutdown_connections 2021-11-03T17:13:10.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.298+0000 7fb30359e700 1 -- 172.21.15.27:0/1560869065 wait complete. 2021-11-03T17:13:10.397 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.401+0000 7f09f9526700 1 -- 172.21.15.27:0/2406787977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09f40f5440 msgr2=0x7f09f40f5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:13:10.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.401+0000 7f09f9526700 1 --2- 172.21.15.27:0/2406787977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09f40f5440 0x7f09f40f5860 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f09e4004660 tx=0x7f09e4009b30).stop 2021-11-03T17:13:10.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.401+0000 7f09f9526700 1 -- 172.21.15.27:0/2406787977 shutdown_connections 2021-11-03T17:13:10.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.401+0000 7f09f9526700 1 --2- 172.21.15.27:0/2406787977 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09f40f66e0 0x7f09f40f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:10.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.401+0000 7f09f9526700 1 --2- 172.21.15.27:0/2406787977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09f40f5440 0x7f09f40f5860 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:10.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.401+0000 7f09f9526700 1 -- 172.21.15.27:0/2406787977 >> 172.21.15.27:0/2406787977 conn(0x7f09f40f0ba0 msgr2=0x7f09f40f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:13:10.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.401+0000 7f09f9526700 1 -- 172.21.15.27:0/2406787977 shutdown_connections 2021-11-03T17:13:10.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.401+0000 7f09f9526700 1 -- 172.21.15.27:0/2406787977 wait complete. 2021-11-03T17:13:10.400 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.402+0000 7f09f9526700 1 Processor -- start 2021-11-03T17:13:10.400 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.402+0000 7f09f9526700 1 -- start start 2021-11-03T17:13:10.402 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.402+0000 7f09f9526700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09f40f5440 0x7f09f4105cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:13:10.402 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.402+0000 7f09f9526700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09f40f66e0 0x7f09f41061c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:13:10.402 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.402+0000 7f09f9526700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f4106770 con 0x7f09f40f66e0 2021-11-03T17:13:10.403 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.402+0000 7f09f9526700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f4106f70 con 0x7f09f40f5440 2021-11-03T17:13:10.403 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.402+0000 7f09f3fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09f40f5440 0x7f09f4105cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:13:10.403 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.403+0000 7f09f37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09f40f66e0 0x7f09f41061c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:13:10.403 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.403+0000 7f09f3fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09f40f5440 0x7f09f4105cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.27:46408/0 (socket says 172.21.15.27:46408) 2021-11-03T17:13:10.404 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.403+0000 7f09f3fff700 1 -- 172.21.15.27:0/2706181593 learned_addr learned my addr 172.21.15.27:0/2706181593 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:13:10.404 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.403+0000 7f09f37fe700 1 -- 172.21.15.27:0/2706181593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09f40f5440 msgr2=0x7f09f4105cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:13:10.404 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.403+0000 7f09f37fe700 1 --2- 172.21.15.27:0/2706181593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09f40f5440 0x7f09f4105cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:10.404 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.403+0000 7f09f37fe700 1 -- 172.21.15.27:0/2706181593 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09e4005040 con 0x7f09f40f66e0 2021-11-03T17:13:10.405 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.403+0000 7f09f3fff700 1 --2- 172.21.15.27:0/2706181593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09f40f5440 0x7f09f4105cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-03T17:13:10.405 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.403+0000 7f09f37fe700 1 --2- 172.21.15.27:0/2706181593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09f40f66e0 0x7f09f41061c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f09e800e7c0 tx=0x7f09e800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:13:10.405 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.403+0000 7f09f0ff9700 1 -- 172.21.15.27:0/2706181593 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09e80092e0 con 0x7f09f40f66e0 2021-11-03T17:13:10.406 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.403+0000 7f09f0ff9700 1 -- 172.21.15.27:0/2706181593 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f09e800f040 con 0x7f09f40f66e0 2021-11-03T17:13:10.406 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.403+0000 7f09f9526700 1 -- 172.21.15.27:0/2706181593 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09f4107250 con 0x7f09f40f66e0 2021-11-03T17:13:10.406 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.403+0000 7f09f0ff9700 1 -- 172.21.15.27:0/2706181593 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09e8014880 con 0x7f09f40f66e0 2021-11-03T17:13:10.406 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.403+0000 7f09f9526700 1 -- 172.21.15.27:0/2706181593 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09f4107810 con 0x7f09f40f66e0 2021-11-03T17:13:10.407 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.404+0000 7f09f9526700 1 -- 172.21.15.27:0/2706181593 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09f4059100 con 0x7f09f40f66e0 2021-11-03T17:13:10.407 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.405+0000 7f09f0ff9700 1 -- 172.21.15.27:0/2706181593 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f09e801b030 con 0x7f09f40f66e0 2021-11-03T17:13:10.407 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.406+0000 7f09f0ff9700 1 --2- 172.21.15.27:0/2706181593 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f09dc05c300 0x7f09dc05e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:13:10.407 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.406+0000 7f09f0ff9700 1 -- 172.21.15.27:0/2706181593 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f09e80759f0 con 0x7f09f40f66e0 2021-11-03T17:13:10.408 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.406+0000 7f09f3fff700 1 --2- 172.21.15.27:0/2706181593 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f09dc05c300 0x7f09dc05e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:13:10.408 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.407+0000 7f09f3fff700 1 --2- 172.21.15.27:0/2706181593 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f09dc05c300 0x7f09dc05e7c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f09e4005010 tx=0x7f09e4016040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:13:10.409 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.413+0000 7f09f0ff9700 1 -- 172.21.15.27:0/2706181593 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f09e8077e60 con 0x7f09f40f66e0 2021-11-03T17:13:10.610 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.615+0000 7f09f9526700 1 -- 172.21.15.27:0/2706181593 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f09f4045ad0 con 0x7f09f40f66e0 2021-11-03T17:13:10.613 INFO:teuthology.orchestra.run.smithi027.stdout:e13 2021-11-03T17:13:10.613 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:13:10.613 INFO:teuthology.orchestra.run.smithi027.stdout:compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:13:10.614 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:13:10.614 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:13:10.615 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:13:10.615 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:13:10.615 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 9 2021-11-03T17:13:10.616 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 2021-11-03T17:13:10.616 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:13:10.616 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:12:05.026191+0000 2021-11-03T17:13:10.616 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:13:10.617 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:13:10.617 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:13:10.617 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:13:10.617 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:13:10.618 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:13:10.618 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:13:10.618 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 0 2021-11-03T17:13:10.618 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:13:10.618 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 2 2021-11-03T17:13:10.619 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:13:10.619 INFO:teuthology.orchestra.run.smithi027.stdout:up {0=24273,1=14436} 2021-11-03T17:13:10.619 INFO:teuthology.orchestra.run.smithi027.stdout:failed 2021-11-03T17:13:10.619 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:13:10.620 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:13:10.620 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:13:10.620 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:13:10.620 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:13:10.621 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:13:10.621 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:13:10.621 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{0:24273} state up:active seq 3 join_fscid=1 addr [v2:172.21.15.115:6824/1621245036,v1:172.21.15.115:6825/1621245036]] 2021-11-03T17:13:10.621 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 3 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060]] 2021-11-03T17:13:10.621 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:13:10.622 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:13:10.622 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:13:10.622 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:13:10.622 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14448} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6828/1337044739,v1:172.21.15.27:6829/1337044739]] 2021-11-03T17:13:10.623 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:24289} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.115:6826/3367380088,v1:172.21.15.115:6827/3367380088]] 2021-11-03T17:13:10.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.615+0000 7f09f0ff9700 1 -- 172.21.15.27:0/2706181593 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 13 v13) v1 ==== 76+0+1627 (secure 0 0 0) 0x7f09e8019020 con 0x7f09f40f66e0 2021-11-03T17:13:10.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.617+0000 7f09f9526700 1 -- 172.21.15.27:0/2706181593 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f09dc05c300 msgr2=0x7f09dc05e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:13:10.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.618+0000 7f09f9526700 1 --2- 172.21.15.27:0/2706181593 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f09dc05c300 0x7f09dc05e7c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f09e4005010 tx=0x7f09e4016040).stop 2021-11-03T17:13:10.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.618+0000 7f09f9526700 1 -- 172.21.15.27:0/2706181593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09f40f66e0 msgr2=0x7f09f41061c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:13:10.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.618+0000 7f09f9526700 1 --2- 172.21.15.27:0/2706181593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09f40f66e0 0x7f09f41061c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f09e800e7c0 tx=0x7f09e800b040).stop 2021-11-03T17:13:10.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.618+0000 7f09f9526700 1 -- 172.21.15.27:0/2706181593 shutdown_connections 2021-11-03T17:13:10.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.618+0000 7f09f9526700 1 --2- 172.21.15.27:0/2706181593 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f09dc05c300 0x7f09dc05e7c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:10.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.618+0000 7f09f9526700 1 --2- 172.21.15.27:0/2706181593 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09f40f5440 0x7f09f4105cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:10.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.618+0000 7f09f9526700 1 --2- 172.21.15.27:0/2706181593 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09f40f66e0 0x7f09f41061c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:10.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.618+0000 7f09f9526700 1 -- 172.21.15.27:0/2706181593 >> 172.21.15.27:0/2706181593 conn(0x7f09f40f0ba0 msgr2=0x7f09f40fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:13:10.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.618+0000 7f09f9526700 1 -- 172.21.15.27:0/2706181593 shutdown_connections 2021-11-03T17:13:10.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:10.618+0000 7f09f9526700 1 -- 172.21.15.27:0/2706181593 wait complete. 2021-11-03T17:13:10.627 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 13 2021-11-03T17:13:10.938 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:10 smithi027 conmon[32322]: audit 2021-11-03T17:13:09.887962+0000 mgr.smithi027.ufhhvb 2021-11-03T17:13:10.939 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:10 smithi027 conmon[32322]: (mgr.14162) 265 : audit [DBG] from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:13:10.939 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:10 smithi027 conmon[32322]: audit 2021-11-03T17:13:10.298053+0000 mon.smithi027 (mon.0) 557 : audit [DBG] from='client.? 172.21.15.27:0/1560869065' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:13:10.939 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:10 smithi027 conmon[32322]: cluster 2021-11-03T17:13:10.939 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:10 smithi027 conmon[32322]: 2021-11-03T17:13:10.590649+0000 mgr.smithi027.ufhhvb (mgr.14162) 266 : cluster 2021-11-03T17:13:10.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:10 smithi027 conmon[32322]: [DBG] pgmap v222: 65 pgs: 65 active+clean; 12 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2021-11-03T17:13:10.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:10 smithi027 conmon[32322]: audit 2021-11-03 2021-11-03T17:13:10.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:10 smithi027 conmon[32322]: T17:13:10.617049+0000 mon.smithi027 ( 2021-11-03T17:13:10.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:10 smithi027 conmon[32322]: mon.0) 558 : audit [DBG] from='client.? 172.21.15.27:0/2706181593' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:13:10.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:10 smithi027 conmon[32322]: 2021-11-03T17:13:10.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:10 smithi115 conmon[34670]: audit 2021-11-03T17:13:09.887962+0000 mgr.smithi027.ufhhvb (mgr 2021-11-03T17:13:10.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:10 smithi115 conmon[34670]: .14162) 265 : audit [DBG] from='client.24359 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:13:10.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:10 smithi115 conmon[34670]: audit 2021-11-03T17:13:10.298053+0000 mon.smithi027 (mon.0) 557 : audit [DBG] from='client.? 172.21.15.27:0/1560869065' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:13:10.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:10 smithi115 conmon[34670]: cluster 2021-11-03T17:13:10.590649+0000 mgr.smithi027.ufhhvb (mgr.14162) 266 : cluster [DBG] pgmap v222: 65 pgs: 65 active+clean; 12 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2021-11-03T17:13:10.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:10 smithi115 conmon[34670]: audit 2021-11-03T17:13:10.617049 2021-11-03T17:13:10.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:10 smithi115 conmon[34670]: +0000 mon.smithi027 (mon.0) 558 : audit [DBG] from='client.? 172.21.15.27:0/2706181593' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:13:13.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:13 smithi027 conmon[32322]: cluster 2021-11-03T17:13:12.591076+0000 2021-11-03T17:13:13.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:13 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14162) 267 : cluster [DBG] pgmap v223: 65 pgs: 65 active+clean; 2.1 MiB data, 36 MiB used, 536 GiB / 536 GiB avail; 178 KiB/s wr, 1 op/s 2021-11-03T17:13:13.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:13 smithi115 conmon[34670]: cluster 2021-11-03T17:13:12.591076+0000 mgr.smithi027.ufhhvb (mgr.14162) 267 : cluster [DBG] pgmap v223: 65 pgs: 65 active+clean; 2.1 MiB data, 36 MiB used, 536 GiB / 536 GiB avail; 178 KiB/s wr, 1 op/s 2021-11-03T17:13:14.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:14 smithi115 conmon[34670]: cluster 2021-11-03T17:13:14.591799+0000 mgr.smithi027.ufhhvb (mgr.14162) 268 : cluster [DBG] pgmap v224: 65 pgs: 65 active+clean; 2.1 MiB data, 39 MiB used, 536 GiB / 536 GiB avail; 178 KiB/s wr, 0 op/s 2021-11-03T17:13:15.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:14 smithi027 conmon[32322]: cluster 2021-11-03T17:13:14.591799+0000 mgr.smithi027.ufhhvb (mgr.14162) 268 : cluster [DBG] pgmap v224: 65 pgs: 65 active+clean; 2.1 MiB data, 39 MiB used, 536 GiB / 536 GiB avail; 178 KiB/s wr, 0 op/s 2021-11-03T17:13:15.435 INFO:tasks.workunit.client.0.smithi027.stderr:Updating files: 78% (7718/9786) Updating files: 79% (7731/9786) Updating files: 80% (7829/9786) Updating files: 81% (7927/9786) Updating files: 82% (8025/9786) Updating files: 83% (8123/9786) Updating files: 84% (8221/9786) Updating files: 85% (8319/9786) Updating files: 86% (8416/9786) Updating files: 87% (8514/9786) Updating files: 88% (8612/9786) Updating files: 89% (8710/9786) Updating files: 90% (8808/9786) Updating files: 91% (8906/9786) Updating files: 92% (9004/9786) Updating files: 93% (9101/9786) Updating files: 94% (9199/9786) Updating files: 95% (9297/9786) Updating files: 96% (9395/9786) Updating files: 97% (9493/9786) Updating files: 98% (9591/9786) Updating files: 99% (9689/9786) Updating files: 100% (9786/9786) Updating files: 100% (9786/9786), done. 2021-11-03T17:13:16.550 INFO:tasks.workunit.client.0.smithi027.stderr:Note: switching to '6e90212d8510e2d506395a8c82057d0d1787f3a4'. 2021-11-03T17:13:16.550 INFO:tasks.workunit.client.0.smithi027.stderr: 2021-11-03T17:13:16.550 INFO:tasks.workunit.client.0.smithi027.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2021-11-03T17:13:16.550 INFO:tasks.workunit.client.0.smithi027.stderr:changes and commit them, and you can discard any commits you make in this 2021-11-03T17:13:16.551 INFO:tasks.workunit.client.0.smithi027.stderr:state without impacting any branches by switching back to a branch. 2021-11-03T17:13:16.551 INFO:tasks.workunit.client.0.smithi027.stderr: 2021-11-03T17:13:16.551 INFO:tasks.workunit.client.0.smithi027.stderr:If you want to create a new branch to retain commits you create, you may 2021-11-03T17:13:16.551 INFO:tasks.workunit.client.0.smithi027.stderr:do so (now or later) by using -c with the switch command. Example: 2021-11-03T17:13:16.552 INFO:tasks.workunit.client.0.smithi027.stderr: 2021-11-03T17:13:16.552 INFO:tasks.workunit.client.0.smithi027.stderr: git switch -c 2021-11-03T17:13:16.552 INFO:tasks.workunit.client.0.smithi027.stderr: 2021-11-03T17:13:16.552 INFO:tasks.workunit.client.0.smithi027.stderr:Or undo this operation with: 2021-11-03T17:13:16.552 INFO:tasks.workunit.client.0.smithi027.stderr: 2021-11-03T17:13:16.553 INFO:tasks.workunit.client.0.smithi027.stderr: git switch - 2021-11-03T17:13:16.553 INFO:tasks.workunit.client.0.smithi027.stderr: 2021-11-03T17:13:16.553 INFO:tasks.workunit.client.0.smithi027.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2021-11-03T17:13:16.553 INFO:tasks.workunit.client.0.smithi027.stderr: 2021-11-03T17:13:16.554 INFO:tasks.workunit.client.0.smithi027.stderr:HEAD is now at 6e90212d851 Merge branch 'test-container-tools-3.1' of https://github.com/liewegas/ceph into wip-sage-testing-2021-11-03-1039 2021-11-03T17:13:16.556 DEBUG:teuthology.orchestra.run.smithi027:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2021-11-03T17:13:16.624 INFO:tasks.workunit.client.0.smithi027.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2021-11-03T17:13:16.625 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2021-11-03T17:13:16.626 INFO:tasks.workunit.client.0.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2021-11-03T17:13:16.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:16 smithi115 conmon[34670]: cluster 2021-11-03T17:13:15.682352+0000 mon.smithi027 (mon.0) 2021-11-03T17:13:16.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:16 smithi115 conmon[34670]: 559 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi115.zsfgnv=up:active,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:13:16.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:16 smithi115 conmon[34670]: cluster 2021-11-03T17:13:16.592269+0000 mgr.smithi027.ufhhvb (mgr.14162) 269 : cluster [DBG] pgmap v225: 65 pgs: 65 active+clean; 41 MiB data, 170 MiB used, 536 GiB / 536 GiB avail; 3.5 MiB/s wr, 283 op/s 2021-11-03T17:13:17.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:16 smithi027 conmon[32322]: cluster 2021-11-03T17:13:15.682352+0000 mon.smithi027 (mon 2021-11-03T17:13:17.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:16 smithi027 conmon[32322]: .0) 559 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi115.zsfgnv=up:active,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:13:17.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:16 smithi027 conmon[32322]: cluster 2021-11-03T17:13:16.592269+0000 mgr.smithi027.ufhhvb (mgr.14162) 269 : 2021-11-03T17:13:17.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:16 smithi027 conmon[32322]: cluster [DBG] pgmap v225: 65 pgs: 65 active+clean; 41 MiB data, 170 MiB used, 536 GiB / 536 GiB avail; 3.5 MiB/s wr, 283 op/s 2021-11-03T17:13:18.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:18 smithi115 conmon[34670]: cluster 2021-11-03T17:13:18.592914+0000 mgr.smithi027.ufhhvb (mgr.14162) 270 2021-11-03T17:13:18.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:18 smithi115 conmon[34670]: : cluster [DBG] pgmap v226: 65 pgs: 65 active+clean; 48 MiB data, 195 MiB used, 536 GiB / 536 GiB avail; 4.2 MiB/s wr, 354 op/s 2021-11-03T17:13:19.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:18 smithi027 conmon[32322]: cluster 2021-11-03T17:13:18.592914+0000 mgr.smithi027.ufhhvb (mgr.14162) 270 : cluster [DBG] pgmap v226: 65 pgs: 65 active+clean; 48 MiB data, 195 MiB used, 536 GiB / 536 GiB avail; 4.2 MiB/s wr, 354 op/s 2021-11-03T17:13:19.901 INFO:tasks.workunit.client.0.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2021-11-03T17:13:20.137 INFO:tasks.workunit.client.0.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2021-11-03T17:13:20.232 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2021-11-03T17:13:20.233 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2021-11-03T17:13:20.234 INFO:tasks.workunit.client.0.smithi027.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2021-11-03T17:13:20.353 INFO:tasks.workunit.client.0.smithi027.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2021-11-03T17:13:20.356 DEBUG:teuthology.orchestra.run.smithi027:> set -ex 2021-11-03T17:13:20.356 DEBUG:teuthology.orchestra.run.smithi027:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2021-11-03T17:13:20.373 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2021-11-03T17:13:20.374 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2021-11-03T17:13:20.375 DEBUG:teuthology.orchestra.run.smithi027:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=6e90212d8510e2d506395a8c82057d0d1787f3a4 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2021-11-03T17:13:20.435 INFO:tasks.workunit.client.0.smithi027.stderr:+ mkdir -p fsstress 2021-11-03T17:13:20.437 INFO:tasks.workunit.client.0.smithi027.stderr:+ pushd fsstress 2021-11-03T17:13:20.438 INFO:tasks.workunit.client.0.smithi027.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2021-11-03T17:13:20.438 INFO:tasks.workunit.client.0.smithi027.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2021-11-03T17:13:20.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:20 smithi115 conmon[34670]: cluster 2021-11-03T17:13:20.593345+0000 mgr.smithi027.ufhhvb (mgr.14162) 271 : cluster 2021-11-03T17:13:20.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:20 smithi115 conmon[34670]: [DBG] pgmap v227: 65 pgs: 65 active+clean; 49 MiB data, 230 MiB used, 536 GiB / 536 GiB avail; 4.3 MiB/s wr, 354 op/s 2021-11-03T17:13:21.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:20 smithi027 conmon[32322]: cluster 2021-11-03T17:13:20.593345+0000 mgr.smithi027.ufhhvb (mgr 2021-11-03T17:13:21.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:20 smithi027 conmon[32322]: .14162) 271 : cluster [DBG] pgmap v227: 65 pgs: 65 active+clean; 49 MiB data, 230 MiB used, 536 GiB / 536 GiB avail; 4.3 MiB/s wr, 354 op/s 2021-11-03T17:13:22.780 INFO:tasks.workunit.client.0.smithi027.stderr:+ tar xzf ltp-full.tgz 2021-11-03T17:13:23.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:23 smithi027 conmon[32322]: cluster 2021-11-03T17:13:22.593920+0000 mgr.smithi027.ufhhvb (mgr.14162) 272 : cluster [DBG] 2021-11-03T17:13:23.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:23 smithi027 conmon[32322]: pgmap v228: 65 pgs: 65 active+clean; 58 MiB data, 243 MiB used, 536 GiB / 536 GiB avail; 5.1 MiB/s wr, 359 op/s 2021-11-03T17:13:23.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:23 smithi115 conmon[34670]: cluster 2021-11-03T17:13:22.593920 2021-11-03T17:13:23.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:23 smithi115 conmon[34670]: +0000 mgr.smithi027.ufhhvb (mgr.14162) 272 : cluster [DBG] pgmap v228: 65 pgs: 65 active+clean; 58 MiB data, 243 MiB used, 536 GiB / 536 GiB avail; 5.1 MiB/s wr, 359 op/s 2021-11-03T17:13:24.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:24 smithi115 conmon[34670]: cluster 2021-11-03T17:13:24.594299+0000 mgr.smithi027.ufhhvb (mgr.14162) 273 2021-11-03T17:13:24.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:24 smithi115 conmon[34670]: : cluster [DBG] pgmap v229: 65 pgs: 65 active+clean; 58 MiB data, 255 MiB used, 536 GiB / 536 GiB avail; 4.9 MiB/s wr, 359 op/s 2021-11-03T17:13:25.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:24 smithi027 conmon[32322]: cluster 2021-11-03T17:13:24.594299+0000 mgr.smithi027.ufhhvb (mgr. 2021-11-03T17:13:25.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:24 smithi027 conmon[32322]: 14162) 273 : cluster [DBG] pgmap v229: 65 pgs: 65 active+clean; 58 MiB data, 255 MiB used, 536 GiB / 536 GiB avail; 4.9 MiB/s wr, 359 op/s 2021-11-03T17:13:25.285 INFO:tasks.workunit.client.1.smithi115.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2021-11-03T17:13:25.286 INFO:tasks.workunit.client.1.smithi115.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2021-11-03T17:13:25.286 INFO:tasks.workunit.client.1.smithi115.stderr:+ make 2021-11-03T17:13:25.848 INFO:tasks.workunit.client.1.smithi115.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2021-11-03T17:13:26.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:26 smithi115 conmon[34670]: cluster 2021-11-03T17:13:26.594977+0000 mgr.smithi027.ufhhvb 2021-11-03T17:13:26.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:26 smithi115 conmon[34670]: (mgr.14162) 274 : cluster [DBG] pgmap v230: 65 pgs: 65 active+clean; 71 MiB data, 281 MiB used, 536 GiB / 536 GiB avail; 6.0 MiB/s wr, 383 op/s 2021-11-03T17:13:26.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:26 smithi115 conmon[34670]: audit 2021-11-03T17:13:26 2021-11-03T17:13:26.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:26 smithi115 conmon[34670]: .673997+0000 mon.smithi027 (mon.0) 560 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:13:26.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:26 smithi115 conmon[34670]: audit 2021-11-03T17:13:26.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:26 smithi115 conmon[34670]: 2021-11-03T17:13:26.675383+0000 mon.smithi027 (mon 2021-11-03T17:13:26.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:26 smithi115 conmon[34670]: .0) 561 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:13:27.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:26 smithi027 conmon[32322]: cluster 2021-11-03T17:13:26.594977 2021-11-03T17:13:27.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:26 smithi027 conmon[32322]: +0000 mgr.smithi027.ufhhvb (mgr.14162) 274 : cluster [DBG] pgmap v230: 65 pgs: 65 active+clean; 71 MiB data, 281 MiB used, 536 GiB / 536 GiB avail; 6.0 MiB/s wr, 383 op/s 2021-11-03T17:13:27.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:26 smithi027 conmon[32322]: audit 2021-11-03T17:13:26.673997+0000 mon.smithi027 2021-11-03T17:13:27.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:26 smithi027 conmon[32322]: (mon.0) 560 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:13:27.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:26 smithi027 conmon[32322]: audit 2021-11- 2021-11-03T17:13:27.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:26 smithi027 conmon[32322]: 03T17:13:26.675383+0000 mon.smithi027 (mon.0) 561 : 2021-11-03T17:13:27.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:26 smithi027 conmon[32322]: audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:13:27.033 INFO:tasks.workunit.client.1.smithi115.stderr:++ readlink -f fsstress 2021-11-03T17:13:27.033 INFO:tasks.workunit.client.1.smithi115.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2021-11-03T17:13:27.034 INFO:tasks.workunit.client.1.smithi115.stderr:+ popd 2021-11-03T17:13:27.034 INFO:tasks.workunit.client.1.smithi115.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2021-11-03T17:13:27.035 INFO:tasks.workunit.client.1.smithi115.stdout:~/cephtest/mnt.1/client.1/tmp 2021-11-03T17:13:27.035 INFO:tasks.workunit.client.1.smithi115.stderr:+ popd 2021-11-03T17:13:27.035 INFO:tasks.workunit.client.1.smithi115.stderr:++ mktemp -d -p . 2021-11-03T17:13:27.040 INFO:tasks.workunit.client.1.smithi115.stderr:+ T=./tmp.ibnEjVpTuX 2021-11-03T17:13:27.040 INFO:tasks.workunit.client.1.smithi115.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.ibnEjVpTuX -l 1 -n 1000 -p 10 -v 2021-11-03T17:13:27.057 INFO:tasks.workunit.client.1.smithi115.stdout:seed = 1635924650 2021-11-03T17:13:27.087 INFO:tasks.workunit.client.1.smithi115.stdout:0/0: chown . 942850 1 2021-11-03T17:13:27.087 INFO:tasks.workunit.client.1.smithi115.stdout:0/1: chown . 15774 1 2021-11-03T17:13:27.088 INFO:tasks.workunit.client.1.smithi115.stdout:0/2: write - no filename 2021-11-03T17:13:27.088 INFO:tasks.workunit.client.1.smithi115.stdout:0/3: write - no filename 2021-11-03T17:13:27.096 INFO:tasks.workunit.client.1.smithi115.stdout:1/0: link - no file 2021-11-03T17:13:27.096 INFO:tasks.workunit.client.1.smithi115.stdout:1/1: dwrite - no filename 2021-11-03T17:13:27.096 INFO:tasks.workunit.client.1.smithi115.stdout:1/2: unlink - no file 2021-11-03T17:13:27.123 INFO:tasks.workunit.client.1.smithi115.stdout:0/4: creat f0 x:0 0 0 2021-11-03T17:13:27.123 INFO:tasks.workunit.client.1.smithi115.stdout:0/5: dread - f0 zero size 2021-11-03T17:13:27.132 INFO:tasks.workunit.client.1.smithi115.stdout:2/0: rename - no filename 2021-11-03T17:13:27.132 INFO:tasks.workunit.client.1.smithi115.stdout:2/1: dwrite - no filename 2021-11-03T17:13:27.132 INFO:tasks.workunit.client.1.smithi115.stdout:2/2: write - no filename 2021-11-03T17:13:27.132 INFO:tasks.workunit.client.1.smithi115.stdout:2/3: link - no file 2021-11-03T17:13:27.133 INFO:tasks.workunit.client.1.smithi115.stdout:2/4: write - no filename 2021-11-03T17:13:27.133 INFO:tasks.workunit.client.1.smithi115.stdout:2/5: readlink - no filename 2021-11-03T17:13:27.133 INFO:tasks.workunit.client.1.smithi115.stdout:2/6: rename - no filename 2021-11-03T17:13:27.134 INFO:tasks.workunit.client.1.smithi115.stdout:1/3: creat f0 x:0 0 0 2021-11-03T17:13:27.134 INFO:tasks.workunit.client.1.smithi115.stdout:1/4: mknod c1 1 2021-11-03T17:13:27.135 INFO:tasks.workunit.client.1.smithi115.stdout:0/6: creat f1 x:0 0 0 2021-11-03T17:13:27.136 INFO:tasks.workunit.client.1.smithi115.stdout:3/0: write - no filename 2021-11-03T17:13:27.138 INFO:tasks.workunit.client.1.smithi115.stdout:1/5: creat f2 x:0 0 0 2021-11-03T17:13:27.139 INFO:tasks.workunit.client.1.smithi115.stdout:0/7: creat f2 x:0 0 0 2021-11-03T17:13:27.139 INFO:tasks.workunit.client.1.smithi115.stdout:0/8: dread - f1 zero size 2021-11-03T17:13:27.140 INFO:tasks.workunit.client.1.smithi115.stdout:4/0: chown . 3 1 2021-11-03T17:13:27.140 INFO:tasks.workunit.client.1.smithi115.stdout:4/1: link - no file 2021-11-03T17:13:27.140 INFO:tasks.workunit.client.1.smithi115.stdout:4/2: dwrite - no filename 2021-11-03T17:13:27.140 INFO:tasks.workunit.client.1.smithi115.stdout:4/3: write - no filename 2021-11-03T17:13:27.141 INFO:tasks.workunit.client.1.smithi115.stdout:4/4: mknod c0 1 2021-11-03T17:13:27.141 INFO:tasks.workunit.client.1.smithi115.stdout:3/1: symlink l0 0 2021-11-03T17:13:27.142 INFO:tasks.workunit.client.1.smithi115.stdout:1/6: creat f3 x:0 0 0 2021-11-03T17:13:27.143 INFO:tasks.workunit.client.1.smithi115.stdout:1/7: write f2 [229501,51667] 0 2021-11-03T17:13:27.145 INFO:tasks.workunit.client.1.smithi115.stdout:4/5: symlink l1 0 2021-11-03T17:13:27.151 INFO:tasks.workunit.client.1.smithi115.stdout:3/2: symlink l1 0 2021-11-03T17:13:27.154 INFO:tasks.workunit.client.1.smithi115.stdout:6/0: dread - no filename 2021-11-03T17:13:27.154 INFO:tasks.workunit.client.1.smithi115.stdout:6/1: chown . 2516977 1 2021-11-03T17:13:27.155 INFO:tasks.workunit.client.1.smithi115.stdout:5/0: creat f0 x:0 0 0 2021-11-03T17:13:27.162 INFO:tasks.workunit.client.1.smithi115.stdout:4/6: creat f2 x:0 0 0 2021-11-03T17:13:27.163 INFO:tasks.workunit.client.1.smithi115.stdout:3/3: creat f2 x:0 0 0 2021-11-03T17:13:27.163 INFO:tasks.workunit.client.1.smithi115.stdout:7/0: truncate - no filename 2021-11-03T17:13:27.163 INFO:tasks.workunit.client.1.smithi115.stdout:7/1: stat - no entries 2021-11-03T17:13:27.163 INFO:tasks.workunit.client.1.smithi115.stdout:7/2: dwrite - no filename 2021-11-03T17:13:27.164 INFO:tasks.workunit.client.1.smithi115.stdout:7/3: read - no filename 2021-11-03T17:13:27.164 INFO:tasks.workunit.client.1.smithi115.stdout:7/4: fdatasync - no filename 2021-11-03T17:13:27.164 INFO:tasks.workunit.client.1.smithi115.stdout:7/5: truncate - no filename 2021-11-03T17:13:27.164 INFO:tasks.workunit.client.1.smithi115.stdout:7/6: chown . 15324 1 2021-11-03T17:13:27.165 INFO:tasks.workunit.client.1.smithi115.stdout:7/7: read - no filename 2021-11-03T17:13:27.165 INFO:tasks.workunit.client.1.smithi115.stdout:7/8: read - no filename 2021-11-03T17:13:27.165 INFO:tasks.workunit.client.1.smithi115.stdout:7/9: rmdir - no directory 2021-11-03T17:13:27.165 INFO:tasks.workunit.client.1.smithi115.stdout:7/10: chown . 909 1 2021-11-03T17:13:27.165 INFO:tasks.workunit.client.1.smithi115.stdout:7/11: getdents . 0 2021-11-03T17:13:27.166 INFO:tasks.workunit.client.1.smithi115.stdout:6/2: mkdir d0 0 2021-11-03T17:13:27.166 INFO:tasks.workunit.client.1.smithi115.stdout:6/3: dwrite - no filename 2021-11-03T17:13:27.168 INFO:tasks.workunit.client.1.smithi115.stdout:3/4: unlink f2 0 2021-11-03T17:13:27.169 INFO:tasks.workunit.client.1.smithi115.stdout:7/12: symlink l0 0 2021-11-03T17:13:27.170 INFO:tasks.workunit.client.1.smithi115.stdout:7/13: dwrite - no filename 2021-11-03T17:13:27.170 INFO:tasks.workunit.client.1.smithi115.stdout:7/14: dread - no filename 2021-11-03T17:13:27.180 INFO:tasks.workunit.client.1.smithi115.stdout:6/4: mkdir d0/d1 0 2021-11-03T17:13:27.180 INFO:tasks.workunit.client.1.smithi115.stdout:3/5: symlink l3 0 2021-11-03T17:13:27.180 INFO:tasks.workunit.client.1.smithi115.stdout:3/6: dread - no filename 2021-11-03T17:13:27.181 INFO:tasks.workunit.client.1.smithi115.stdout:7/15: creat f1 x:0 0 0 2021-11-03T17:13:27.181 INFO:tasks.workunit.client.1.smithi115.stdout:7/16: stat f1 0 2021-11-03T17:13:27.181 INFO:tasks.workunit.client.1.smithi115.stdout:8/0: creat f0 x:0 0 0 2021-11-03T17:13:27.185 INFO:tasks.workunit.client.1.smithi115.stdout:6/5: mkdir d0/d1/d2 0 2021-11-03T17:13:27.185 INFO:tasks.workunit.client.1.smithi115.stdout:6/6: stat d0 0 2021-11-03T17:13:27.185 INFO:tasks.workunit.client.1.smithi115.stdout:6/7: mknod d0/d1/d2/c3 1 2021-11-03T17:13:27.186 INFO:tasks.workunit.client.1.smithi115.stdout:3/7: symlink l4 0 2021-11-03T17:13:27.186 INFO:tasks.workunit.client.1.smithi115.stdout:3/8: dwrite - no filename 2021-11-03T17:13:27.188 INFO:tasks.workunit.client.1.smithi115.stdout:9/0: dwrite - no filename 2021-11-03T17:13:27.188 INFO:tasks.workunit.client.1.smithi115.stdout:9/1: read - no filename 2021-11-03T17:13:27.188 INFO:tasks.workunit.client.1.smithi115.stdout:9/2: chown . 392115489 1 2021-11-03T17:13:27.189 INFO:tasks.workunit.client.1.smithi115.stdout:9/3: write - no filename 2021-11-03T17:13:27.189 INFO:tasks.workunit.client.1.smithi115.stdout:9/4: dread - no filename 2021-11-03T17:13:27.193 INFO:tasks.workunit.client.1.smithi115.stdout:7/17: symlink l2 0 2021-11-03T17:13:27.194 INFO:tasks.workunit.client.1.smithi115.stdout:6/8: creat d0/f4 x:0 0 0 2021-11-03T17:13:27.194 INFO:tasks.workunit.client.1.smithi115.stdout:6/9: dread - d0/f4 zero size 2021-11-03T17:13:27.194 INFO:tasks.workunit.client.1.smithi115.stdout:6/10: truncate d0/f4 801039 0 2021-11-03T17:13:27.196 INFO:tasks.workunit.client.1.smithi115.stdout:3/9: creat f5 x:0 0 0 2021-11-03T17:13:27.196 INFO:tasks.workunit.client.1.smithi115.stdout:3/10: truncate f5 105171 0 2021-11-03T17:13:27.197 INFO:tasks.workunit.client.1.smithi115.stdout:3/11: write f5 [338943,75214] 0 2021-11-03T17:13:27.198 INFO:tasks.workunit.client.1.smithi115.stdout:9/5: creat f0 x:0 0 0 2021-11-03T17:13:27.206 INFO:tasks.workunit.client.1.smithi115.stdout:6/11: rename d0/f4 to d0/d1/f5 0 2021-11-03T17:13:27.206 INFO:tasks.workunit.client.1.smithi115.stdout:7/18: mkdir d3 0 2021-11-03T17:13:27.206 INFO:tasks.workunit.client.1.smithi115.stdout:3/12: rename l3 to l6 0 2021-11-03T17:13:27.206 INFO:tasks.workunit.client.1.smithi115.stdout:9/6: mkdir d1 0 2021-11-03T17:13:27.213 INFO:tasks.workunit.client.1.smithi115.stdout:9/7: creat d1/f2 x:0 0 0 2021-11-03T17:13:27.214 INFO:tasks.workunit.client.1.smithi115.stdout:9/8: stat d1/f2 0 2021-11-03T17:13:27.214 INFO:tasks.workunit.client.1.smithi115.stdout:9/9: readlink - no filename 2021-11-03T17:13:27.214 INFO:tasks.workunit.client.1.smithi115.stdout:9/10: dread - d1/f2 zero size 2021-11-03T17:13:27.217 INFO:tasks.workunit.client.1.smithi115.stdout:9/11: unlink f0 0 2021-11-03T17:13:27.217 INFO:tasks.workunit.client.1.smithi115.stdout:9/12: chown d1 432923 1 2021-11-03T17:13:27.649 INFO:tasks.workunit.client.1.smithi115.stdout:5/1: dwrite f0 [0,4194304] 0 2021-11-03T17:13:27.801 INFO:tasks.workunit.client.1.smithi115.stdout:6/12: dread d0/d1/f5 [0,4194304] 0 2021-11-03T17:13:27.801 INFO:tasks.workunit.client.1.smithi115.stdout:6/13: stat d0/d1/d2 0 2021-11-03T17:13:28.001 INFO:tasks.workunit.client.1.smithi115.stdout:0/9: dwrite f0 [0,4194304] 0 2021-11-03T17:13:28.002 INFO:tasks.workunit.client.1.smithi115.stdout:0/10: readlink - no filename 2021-11-03T17:13:28.002 INFO:tasks.workunit.client.1.smithi115.stdout:0/11: mknod c3 1 2021-11-03T17:13:28.002 INFO:tasks.workunit.client.1.smithi115.stdout:0/12: readlink - no filename 2021-11-03T17:13:28.032 INFO:tasks.workunit.client.1.smithi115.stdout:7/19: dwrite f1 [0,4194304] 0 2021-11-03T17:13:28.033 INFO:tasks.workunit.client.1.smithi115.stdout:7/20: readlink l0 0 2021-11-03T17:13:28.039 INFO:tasks.workunit.client.1.smithi115.stdout:7/21: write f1 [2836269,26414] 0 2021-11-03T17:13:28.104 INFO:tasks.workunit.client.1.smithi115.stdout:8/1: dwrite f0 [0,4194304] 0 2021-11-03T17:13:28.347 INFO:tasks.workunit.client.1.smithi115.stdout:1/8: fsync f3 0 2021-11-03T17:13:28.347 INFO:tasks.workunit.client.1.smithi115.stdout:1/9: rmdir - no directory 2021-11-03T17:13:28.348 INFO:tasks.workunit.client.1.smithi115.stdout:1/10: dread - f3 zero size 2021-11-03T17:13:28.348 INFO:tasks.workunit.client.1.smithi115.stdout:1/11: mknod c4 1 2021-11-03T17:13:28.352 INFO:tasks.workunit.client.1.smithi115.stdout:1/12: dread f2 [0,4194304] 0 2021-11-03T17:13:28.352 INFO:tasks.workunit.client.1.smithi115.stdout:1/13: mknod c5 1 2021-11-03T17:13:28.352 INFO:tasks.workunit.client.1.smithi115.stdout:1/14: mknod c6 1 2021-11-03T17:13:28.352 INFO:tasks.workunit.client.1.smithi115.stdout:1/15: write f3 [985806,128880] 0 2021-11-03T17:13:28.394 INFO:tasks.workunit.client.1.smithi115.stdout:1/16: dwrite f0 [0,4194304] 0 2021-11-03T17:13:28.395 INFO:tasks.workunit.client.1.smithi115.stdout:1/17: truncate f0 4941078 0 2021-11-03T17:13:28.441 INFO:tasks.workunit.client.1.smithi115.stdout:6/14: dwrite d0/d1/f5 [0,4194304] 0 2021-11-03T17:13:28.460 INFO:tasks.workunit.client.1.smithi115.stdout:6/15: dread d0/d1/f5 [0,4194304] 0 2021-11-03T17:13:28.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:28 smithi115 conmon[34670]: cephadm 2021-11-03T17:13:26.674385+0000 mgr.smithi027.ufhhvb (mgr.14162) 275 : cephadm [INF] Upgrade: Target is version 17.0.0-8776-g6e90212d (unknown) 2021-11-03T17:13:28.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:28 smithi115 conmon[34670]: cephadm 2021-11-03T17:13:26.674618+0000 mgr.smithi027.ufhhvb (mgr.14162) 276 : cephadm [INF] Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:72989db4ddcdff183d79e135290eb63333fd8eb1af14b7c17a49eac7dfadef2c, digests ['quay.ceph.io/ceph-ci/ceph@sha256:72989db4ddcdff183d79e135290eb63333fd8eb1af14b7c17a49eac7dfadef2c'] 2021-11-03T17:13:28.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:28 smithi115 conmon[34670]: cephadm 2021-11-03T17:13:26.676328+0000 mgr.smithi027.ufhhvb (mgr.14162) 277 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.smithi027.ufhhvb) 2021-11-03T17:13:28.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:28 smithi115 conmon[34670]: cephadm 2021-11-03T17:13:26.981649+0000 mgr.smithi027.ufhhvb (mgr.14162) 278 : cephadm [INF] Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:72989db4ddcdff183d79e135290eb63333fd8eb1af14b7c17a49eac7dfadef2c on smithi115 2021-11-03T17:13:28.490 INFO:tasks.workunit.client.1.smithi115.stdout:6/16: dread d0/d1/f5 [0,4194304] 0 2021-11-03T17:13:28.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:28 smithi027 conmon[32322]: cephadm 2021-11-03T17:13:26.674385+0000 mgr.smithi027.ufhhvb (mgr.14162) 275 : cephadm [INF] Upgrade: Target is version 17.0.0-8776-g6e90212d (unknown) 2021-11-03T17:13:28.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:28 smithi027 conmon[32322]: cephadm 2021-11-03T17:13:26.674618+0000 mgr.smithi027.ufhhvb (mgr.14162) 276 : cephadm [INF] Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:72989db4ddcdff183d79e135290eb63333fd8eb1af14b7c17a49eac7dfadef2c, digests ['quay.ceph.io/ceph-ci/ceph@sha256:72989db4ddcdff183d79e135290eb63333fd8eb1af14b7c17a49eac7dfadef2c'] 2021-11-03T17:13:28.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:28 smithi027 conmon[32322]: cephadm 2021-11-03T17:13:26.676328+0000 mgr.smithi027.ufhhvb (mgr.14162) 277 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.smithi027.ufhhvb) 2021-11-03T17:13:28.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:28 smithi027 conmon[32322]: cephadm 2021-11-03T17:13:26.981649+0000 mgr.smithi027.ufhhvb (mgr.14162) 278 : cephadm [INF] Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:72989db4ddcdff183d79e135290eb63333fd8eb1af14b7c17a49eac7dfadef2c on smithi115 2021-11-03T17:13:28.541 INFO:tasks.workunit.client.1.smithi115.stdout:6/17: dwrite d0/d1/f5 [0,4194304] 0 2021-11-03T17:13:28.623 INFO:tasks.workunit.client.1.smithi115.stdout:5/2: dwrite f0 [0,4194304] 0 2021-11-03T17:13:28.623 INFO:tasks.workunit.client.1.smithi115.stdout:5/3: chown f0 179224526 1 2021-11-03T17:13:28.640 INFO:tasks.workunit.client.1.smithi115.stdout:5/4: dwrite f0 [0,4194304] 0 2021-11-03T17:13:28.640 INFO:tasks.workunit.client.1.smithi115.stdout:3/13: fsync f5 0 2021-11-03T17:13:28.654 INFO:tasks.workunit.client.1.smithi115.stdout:3/14: dwrite f5 [0,4194304] 0 2021-11-03T17:13:28.667 INFO:tasks.workunit.client.1.smithi115.stdout:3/15: dwrite f5 [0,4194304] 0 2021-11-03T17:13:28.830 INFO:tasks.workunit.client.1.smithi115.stdout:0/13: symlink l4 0 2021-11-03T17:13:28.830 INFO:tasks.workunit.client.1.smithi115.stdout:0/14: chown l4 0 1 2021-11-03T17:13:29.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:29 smithi115 conmon[34670]: cluster 2021-11-03T17:13:28.595341+0000 mgr.smithi027.ufhhvb (mgr.14162) 279 : cluster [DBG] 2021-11-03T17:13:29.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:29 smithi115 conmon[34670]: pgmap v231: 65 pgs: 65 active+clean; 94 MiB data, 339 MiB used, 536 GiB / 536 GiB avail; 682 B/s rd, 4.6 MiB/s wr, 226 op/s 2021-11-03T17:13:29.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:29 smithi027 conmon[32322]: cluster 2021-11-03T17:13:28.595341+0000 mgr.smithi027.ufhhvb (mgr 2021-11-03T17:13:29.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:29 smithi027 conmon[32322]: .14162) 279 : cluster [DBG] pgmap v231: 65 pgs: 65 active+clean; 94 MiB data, 339 MiB used, 536 GiB / 536 GiB avail; 682 B/s rd, 4.6 MiB/s wr, 226 op/s 2021-11-03T17:13:29.934 INFO:tasks.workunit.client.1.smithi115.stdout:8/2: creat f1 x:0 0 0 2021-11-03T17:13:29.935 INFO:tasks.workunit.client.1.smithi115.stdout:8/3: write f1 [5591,67466] 0 2021-11-03T17:13:29.935 INFO:tasks.workunit.client.1.smithi115.stdout:8/4: mknod c2 1 2021-11-03T17:13:29.935 INFO:tasks.workunit.client.1.smithi115.stdout:8/5: stat f1 0 2021-11-03T17:13:30.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:30 smithi115 conmon[34670]: cluster 2021-11-03T17:13:30.595952 2021-11-03T17:13:30.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:30 smithi115 conmon[34670]: +0000 mgr.smithi027.ufhhvb (mgr.14162) 280 : cluster [DBG] pgmap v232: 65 pgs: 65 active+clean; 142 MiB data, 540 MiB used, 536 GiB / 536 GiB avail; 706 KiB/s rd, 8.8 MiB/s wr, 496 op/s 2021-11-03T17:13:31.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:30 smithi027 conmon[32322]: cluster 2021-11-03T17:13:30.595952+0000 mgr.smithi027.ufhhvb (mgr. 2021-11-03T17:13:31.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:30 smithi027 conmon[32322]: 14162) 280 : cluster [DBG] pgmap v232: 65 pgs: 65 active+clean; 142 MiB data, 540 MiB used, 536 GiB / 536 GiB avail; 706 KiB/s rd, 8.8 MiB/s wr, 496 op/s 2021-11-03T17:13:32.538 INFO:tasks.workunit.client.1.smithi115.stdout:1/18: mkdir d7 0 2021-11-03T17:13:32.987 INFO:tasks.workunit.client.1.smithi115.stdout:6/18: write d0/d1/f5 [4275989,29553] 0 2021-11-03T17:13:33.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:33 smithi027 conmon[32322]: cluster 2021-11-03T17:13:32.596577+0000 mgr.smithi027.ufhhvb ( 2021-11-03T17:13:33.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:33 smithi027 conmon[32322]: mgr.14162) 281 : cluster [DBG] pgmap v233: 65 pgs: 65 active+clean; 163 MiB data, 655 MiB used, 536 GiB / 536 GiB avail; 706 KiB/s rd, 11 MiB/s wr, 694 op/s 2021-11-03T17:13:33.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:33 smithi115 conmon[34670]: cluster 2021-11-03T17:13:32.596577+0000 mgr.smithi027.ufhhvb (mgr.14162) 281 : 2021-11-03T17:13:33.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:33 smithi115 conmon[34670]: cluster [DBG] pgmap v233: 65 pgs: 65 active+clean; 163 MiB data, 655 MiB used, 536 GiB / 536 GiB avail; 706 KiB/s rd, 11 MiB/s wr, 694 op/s 2021-11-03T17:13:33.543 INFO:tasks.workunit.client.1.smithi115.stdout:5/5: creat f1 x:0 0 0 2021-11-03T17:13:33.543 INFO:tasks.workunit.client.1.smithi115.stdout:5/6: mknod c2 1 2021-11-03T17:13:33.545 INFO:tasks.workunit.client.1.smithi115.stdout:3/16: getdents . 0 2021-11-03T17:13:33.546 INFO:tasks.workunit.client.1.smithi115.stdout:3/17: readlink l4 0 2021-11-03T17:13:33.546 INFO:tasks.workunit.client.1.smithi115.stdout:3/18: rmdir - no directory 2021-11-03T17:13:33.547 INFO:tasks.workunit.client.1.smithi115.stdout:0/15: symlink l5 0 2021-11-03T17:13:33.547 INFO:tasks.workunit.client.1.smithi115.stdout:0/16: truncate f2 993338 0 2021-11-03T17:13:33.549 INFO:tasks.workunit.client.1.smithi115.stdout:8/6: symlink l3 0 2021-11-03T17:13:33.549 INFO:tasks.workunit.client.1.smithi115.stdout:8/7: write f0 [4551190,36883] 0 2021-11-03T17:13:33.550 INFO:tasks.workunit.client.1.smithi115.stdout:8/8: readlink l3 0 2021-11-03T17:13:33.550 INFO:tasks.workunit.client.1.smithi115.stdout:1/19: creat d7/f8 x:0 0 0 2021-11-03T17:13:33.551 INFO:tasks.workunit.client.1.smithi115.stdout:6/19: link d0/d1/f5 d0/f6 0 2021-11-03T17:13:33.555 INFO:tasks.workunit.client.1.smithi115.stdout:5/7: creat f3 x:0 0 0 2021-11-03T17:13:33.556 INFO:tasks.workunit.client.1.smithi115.stdout:3/19: rename l0 to l7 0 2021-11-03T17:13:33.556 INFO:tasks.workunit.client.1.smithi115.stdout:0/17: creat f6 x:0 0 0 2021-11-03T17:13:33.557 INFO:tasks.workunit.client.1.smithi115.stdout:8/9: unlink f1 0 2021-11-03T17:13:33.559 INFO:tasks.workunit.client.1.smithi115.stdout:1/20: creat d7/f9 x:0 0 0 2021-11-03T17:13:33.559 INFO:tasks.workunit.client.1.smithi115.stdout:1/21: truncate d7/f8 715111 0 2021-11-03T17:13:33.560 INFO:tasks.workunit.client.1.smithi115.stdout:0/18: creat f7 x:0 0 0 2021-11-03T17:13:33.562 INFO:tasks.workunit.client.1.smithi115.stdout:8/10: mkdir d4 0 2021-11-03T17:13:33.562 INFO:tasks.workunit.client.1.smithi115.stdout:1/22: creat d7/fa x:0 0 0 2021-11-03T17:13:33.566 INFO:tasks.workunit.client.1.smithi115.stdout:0/19: unlink f1 0 2021-11-03T17:13:33.566 INFO:tasks.workunit.client.1.smithi115.stdout:8/11: write f0 [969731,108169] 0 2021-11-03T17:13:33.566 INFO:tasks.workunit.client.1.smithi115.stdout:1/23: creat d7/fb x:0 0 0 2021-11-03T17:13:33.570 INFO:tasks.workunit.client.1.smithi115.stdout:1/24: mkdir d7/dc 0 2021-11-03T17:13:33.571 INFO:tasks.workunit.client.1.smithi115.stdout:1/25: mkdir d7/dc/dd 0 2021-11-03T17:13:33.571 INFO:tasks.workunit.client.1.smithi115.stdout:1/26: chown d7 235 1 2021-11-03T17:13:33.572 INFO:tasks.workunit.client.1.smithi115.stdout:1/27: write d7/fb [238543,73977] 0 2021-11-03T17:13:33.572 INFO:tasks.workunit.client.1.smithi115.stdout:1/28: write f3 [1151074,38967] 0 2021-11-03T17:13:33.573 INFO:tasks.workunit.client.1.smithi115.stdout:6/20: dread d0/f6 [0,4194304] 0 2021-11-03T17:13:33.575 INFO:tasks.workunit.client.1.smithi115.stdout:5/8: dwrite f3 [0,4194304] 0 2021-11-03T17:13:33.576 INFO:tasks.workunit.client.1.smithi115.stdout:5/9: write f1 [929839,38400] 0 2021-11-03T17:13:33.576 INFO:tasks.workunit.client.1.smithi115.stdout:5/10: readlink - no filename 2021-11-03T17:13:33.576 INFO:tasks.workunit.client.1.smithi115.stdout:5/11: chown f0 3 1 2021-11-03T17:13:33.582 INFO:tasks.workunit.client.1.smithi115.stdout:5/12: symlink l4 0 2021-11-03T17:13:33.583 INFO:tasks.workunit.client.1.smithi115.stdout:1/29: dread f0 [0,4194304] 0 2021-11-03T17:13:33.597 INFO:tasks.workunit.client.1.smithi115.stdout:5/13: dwrite f0 [0,4194304] 0 2021-11-03T17:13:33.605 INFO:tasks.workunit.client.1.smithi115.stdout:0/20: dwrite f2 [0,4194304] 0 2021-11-03T17:13:33.605 INFO:tasks.workunit.client.1.smithi115.stdout:8/12: dread f0 [0,4194304] 0 2021-11-03T17:13:33.607 INFO:tasks.workunit.client.1.smithi115.stdout:5/14: dread f3 [0,4194304] 0 2021-11-03T17:13:33.612 INFO:tasks.workunit.client.1.smithi115.stdout:8/13: write f0 [200766,75083] 0 2021-11-03T17:13:33.612 INFO:tasks.workunit.client.1.smithi115.stdout:8/14: write f0 [4461385,122317] 0 2021-11-03T17:13:33.612 INFO:tasks.workunit.client.1.smithi115.stdout:8/15: getdents d4 0 2021-11-03T17:13:33.619 INFO:tasks.workunit.client.1.smithi115.stdout:0/21: creat f8 x:0 0 0 2021-11-03T17:13:33.621 INFO:tasks.workunit.client.1.smithi115.stdout:5/15: symlink l5 0 2021-11-03T17:13:33.622 INFO:tasks.workunit.client.1.smithi115.stdout:8/16: creat d4/f5 x:0 0 0 2021-11-03T17:13:33.630 INFO:tasks.workunit.client.1.smithi115.stdout:0/22: mkdir d9 0 2021-11-03T17:13:33.631 INFO:tasks.workunit.client.1.smithi115.stdout:0/23: dread - f8 zero size 2021-11-03T17:13:33.636 INFO:tasks.workunit.client.1.smithi115.stdout:0/24: symlink d9/la 0 2021-11-03T17:13:33.637 INFO:tasks.workunit.client.1.smithi115.stdout:5/16: dwrite f0 [0,4194304] 0 2021-11-03T17:13:33.642 INFO:tasks.workunit.client.1.smithi115.stdout:0/25: mkdir d9/db 0 2021-11-03T17:13:33.642 INFO:tasks.workunit.client.1.smithi115.stdout:0/26: write f6 [627577,18865] 0 2021-11-03T17:13:33.647 INFO:tasks.workunit.client.1.smithi115.stdout:5/17: rename f1 to f6 0 2021-11-03T17:13:33.784 INFO:tasks.workunit.client.1.smithi115.stdout:5/18: write f3 [1188662,106224] 0 2021-11-03T17:13:33.785 INFO:tasks.workunit.client.1.smithi115.stdout:0/27: dwrite f0 [0,4194304] 0 2021-11-03T17:13:33.785 INFO:tasks.workunit.client.1.smithi115.stdout:0/28: dread - f8 zero size 2021-11-03T17:13:33.785 INFO:tasks.workunit.client.1.smithi115.stdout:0/29: rename f6 to d9/fc 0 2021-11-03T17:13:33.785 INFO:tasks.workunit.client.1.smithi115.stdout:8/17: dwrite f0 [4194304,4194304] 0 2021-11-03T17:13:33.785 INFO:tasks.workunit.client.1.smithi115.stdout:0/30: symlink d9/db/ld 0 2021-11-03T17:13:33.786 INFO:tasks.workunit.client.1.smithi115.stdout:8/18: mkdir d4/d6 0 2021-11-03T17:13:33.786 INFO:tasks.workunit.client.1.smithi115.stdout:0/31: symlink d9/le 0 2021-11-03T17:13:33.786 INFO:tasks.workunit.client.1.smithi115.stdout:8/19: unlink f0 0 2021-11-03T17:13:33.786 INFO:tasks.workunit.client.1.smithi115.stdout:8/20: symlink d4/l7 0 2021-11-03T17:13:33.787 INFO:tasks.workunit.client.1.smithi115.stdout:8/21: stat l3 0 2021-11-03T17:13:33.787 INFO:tasks.workunit.client.1.smithi115.stdout:8/22: read - d4/f5 zero size 2021-11-03T17:13:33.787 INFO:tasks.workunit.client.1.smithi115.stdout:8/23: creat d4/d6/f8 x:0 0 0 2021-11-03T17:13:33.787 INFO:tasks.workunit.client.1.smithi115.stdout:8/24: rename l3 to d4/l9 0 2021-11-03T17:13:33.788 INFO:tasks.workunit.client.1.smithi115.stdout:0/32: dwrite f8 [0,4194304] 0 2021-11-03T17:13:33.788 INFO:tasks.workunit.client.1.smithi115.stdout:0/33: fdatasync f0 0 2021-11-03T17:13:33.788 INFO:tasks.workunit.client.1.smithi115.stdout:8/25: dwrite d4/d6/f8 [0,4194304] 0 2021-11-03T17:13:33.789 INFO:tasks.workunit.client.1.smithi115.stdout:8/26: chown d4/l7 33989577 1 2021-11-03T17:13:33.789 INFO:tasks.workunit.client.1.smithi115.stdout:0/34: dwrite f2 [0,4194304] 0 2021-11-03T17:13:33.789 INFO:tasks.workunit.client.1.smithi115.stdout:0/35: write f7 [215486,81229] 0 2021-11-03T17:13:33.789 INFO:tasks.workunit.client.1.smithi115.stdout:0/36: chown l5 587275692 1 2021-11-03T17:13:33.789 INFO:tasks.workunit.client.1.smithi115.stdout:0/37: dread f0 [0,4194304] 0 2021-11-03T17:13:33.790 INFO:tasks.workunit.client.1.smithi115.stdout:8/27: dwrite d4/f5 [0,4194304] 0 2021-11-03T17:13:33.790 INFO:tasks.workunit.client.1.smithi115.stdout:0/38: dread f7 [0,4194304] 0 2021-11-03T17:13:33.790 INFO:tasks.workunit.client.1.smithi115.stdout:0/39: chown d9/db/ld 38 1 2021-11-03T17:13:33.790 INFO:tasks.workunit.client.1.smithi115.stdout:8/28: creat d4/fa x:0 0 0 2021-11-03T17:13:33.790 INFO:tasks.workunit.client.1.smithi115.stdout:0/40: mkdir d9/db/df 0 2021-11-03T17:13:33.791 INFO:tasks.workunit.client.1.smithi115.stdout:8/29: creat d4/d6/fb x:0 0 0 2021-11-03T17:13:33.791 INFO:tasks.workunit.client.1.smithi115.stdout:8/30: fdatasync d4/d6/f8 0 2021-11-03T17:13:33.792 INFO:tasks.workunit.client.1.smithi115.stdout:0/41: dwrite f0 [0,4194304] 0 2021-11-03T17:13:33.793 INFO:tasks.workunit.client.1.smithi115.stdout:0/42: getdents d9/db/df 0 2021-11-03T17:13:33.793 INFO:tasks.workunit.client.1.smithi115.stdout:0/43: chown l5 180217818 1 2021-11-03T17:13:33.793 INFO:tasks.workunit.client.1.smithi115.stdout:0/44: write f0 [2358248,108024] 0 2021-11-03T17:13:33.794 INFO:tasks.workunit.client.1.smithi115.stdout:0/45: chown f7 1083182 1 2021-11-03T17:13:33.798 INFO:tasks.workunit.client.1.smithi115.stdout:0/46: symlink d9/db/l10 0 2021-11-03T17:13:33.805 INFO:tasks.workunit.client.1.smithi115.stdout:0/47: creat d9/db/df/f11 x:0 0 0 2021-11-03T17:13:33.831 INFO:tasks.workunit.client.1.smithi115.stdout:0/48: dwrite f2 [0,4194304] 0 2021-11-03T17:13:33.834 INFO:tasks.workunit.client.1.smithi115.stdout:0/49: creat d9/db/df/f12 x:0 0 0 2021-11-03T17:13:33.834 INFO:tasks.workunit.client.1.smithi115.stdout:0/50: write f7 [973280,62436] 0 2021-11-03T17:13:33.834 INFO:tasks.workunit.client.1.smithi115.stdout:0/51: readlink d9/la 0 2021-11-03T17:13:33.835 INFO:tasks.workunit.client.1.smithi115.stdout:0/52: mknod d9/db/df/c13 1 2021-11-03T17:13:33.835 INFO:tasks.workunit.client.1.smithi115.stdout:0/53: write d9/db/df/f12 [657410,108646] 0 2021-11-03T17:13:33.840 INFO:tasks.workunit.client.1.smithi115.stdout:0/54: mkdir d9/db/d14 0 2021-11-03T17:13:33.842 INFO:tasks.workunit.client.1.smithi115.stdout:0/55: rename d9/db/d14 to d9/d15 0 2021-11-03T17:13:33.846 INFO:tasks.workunit.client.1.smithi115.stdout:0/56: creat d9/d15/f16 x:0 0 0 2021-11-03T17:13:33.848 INFO:tasks.workunit.client.1.smithi115.stdout:0/57: mkdir d9/d17 0 2021-11-03T17:13:33.852 INFO:tasks.workunit.client.1.smithi115.stdout:0/58: unlink d9/db/l10 0 2021-11-03T17:13:33.857 INFO:tasks.workunit.client.1.smithi115.stdout:0/59: unlink d9/la 0 2021-11-03T17:13:33.859 INFO:tasks.workunit.client.1.smithi115.stdout:0/60: link d9/db/ld d9/d15/l18 0 2021-11-03T17:13:33.862 INFO:tasks.workunit.client.1.smithi115.stdout:0/61: creat d9/db/f19 x:0 0 0 2021-11-03T17:13:33.881 INFO:tasks.workunit.client.1.smithi115.stdout:0/62: dwrite d9/db/df/f12 [0,4194304] 0 2021-11-03T17:13:33.888 INFO:tasks.workunit.client.1.smithi115.stdout:0/63: mkdir d9/d15/d1a 0 2021-11-03T17:13:33.888 INFO:tasks.workunit.client.1.smithi115.stdout:0/64: chown d9/db/df/f12 350000061 1 2021-11-03T17:13:33.888 INFO:tasks.workunit.client.1.smithi115.stdout:0/65: stat d9/db/df/f11 0 2021-11-03T17:13:33.952 INFO:tasks.workunit.client.1.smithi115.stdout:7/22: truncate f1 2538928 0 2021-11-03T17:13:33.956 INFO:tasks.workunit.client.1.smithi115.stdout:6/21: getdents d0 0 2021-11-03T17:13:33.960 INFO:tasks.workunit.client.1.smithi115.stdout:6/22: creat d0/f7 x:0 0 0 2021-11-03T17:13:33.966 INFO:tasks.workunit.client.1.smithi115.stdout:6/23: write d0/f6 [964987,2186] 0 2021-11-03T17:13:33.967 INFO:tasks.workunit.client.1.smithi115.stdout:1/30: fsync d7/fb 0 2021-11-03T17:13:33.967 INFO:tasks.workunit.client.1.smithi115.stdout:3/20: truncate f5 1749039 0 2021-11-03T17:13:33.968 INFO:tasks.workunit.client.1.smithi115.stdout:1/31: dread f0 [0,4194304] 0 2021-11-03T17:13:33.974 INFO:tasks.workunit.client.1.smithi115.stdout:1/32: creat d7/fe x:0 0 0 2021-11-03T17:13:33.985 INFO:tasks.workunit.client.1.smithi115.stdout:5/19: fsync f6 0 2021-11-03T17:13:33.986 INFO:tasks.workunit.client.1.smithi115.stdout:6/24: dread d0/d1/f5 [0,4194304] 0 2021-11-03T17:13:33.989 INFO:tasks.workunit.client.1.smithi115.stdout:1/33: dwrite d7/fb [0,4194304] 0 2021-11-03T17:13:33.989 INFO:tasks.workunit.client.1.smithi115.stdout:1/34: write d7/fa [793676,3483] 0 2021-11-03T17:13:33.989 INFO:tasks.workunit.client.1.smithi115.stdout:1/35: dread - d7/f9 zero size 2021-11-03T17:13:33.990 INFO:tasks.workunit.client.1.smithi115.stdout:5/20: fsync f3 0 2021-11-03T17:13:33.991 INFO:tasks.workunit.client.1.smithi115.stdout:6/25: creat d0/d1/f8 x:0 0 0 2021-11-03T17:13:34.003 INFO:tasks.workunit.client.1.smithi115.stdout:5/21: creat f7 x:0 0 0 2021-11-03T17:13:34.006 INFO:tasks.workunit.client.1.smithi115.stdout:5/22: unlink l4 0 2021-11-03T17:13:34.016 INFO:tasks.workunit.client.1.smithi115.stdout:6/26: dwrite d0/f7 [0,4194304] 0 2021-11-03T17:13:34.016 INFO:tasks.workunit.client.1.smithi115.stdout:6/27: rename d0 to d0/d1/d9 22 2021-11-03T17:13:34.016 INFO:tasks.workunit.client.1.smithi115.stdout:1/36: dwrite d7/f9 [0,4194304] 0 2021-11-03T17:13:34.021 INFO:tasks.workunit.client.1.smithi115.stdout:8/31: getdents d4 0 2021-11-03T17:13:34.023 INFO:tasks.workunit.client.1.smithi115.stdout:5/23: dwrite f0 [0,4194304] 0 2021-11-03T17:13:34.025 INFO:tasks.workunit.client.1.smithi115.stdout:5/24: dread f6 [0,4194304] 0 2021-11-03T17:13:34.025 INFO:tasks.workunit.client.1.smithi115.stdout:5/25: mknod c8 1 2021-11-03T17:13:34.027 INFO:tasks.workunit.client.1.smithi115.stdout:6/28: creat d0/d1/fa x:0 0 0 2021-11-03T17:13:34.027 INFO:tasks.workunit.client.1.smithi115.stdout:6/29: stat d0/d1/d2 0 2021-11-03T17:13:34.029 INFO:tasks.workunit.client.1.smithi115.stdout:6/30: write d0/d1/f5 [2791618,73943] 0 2021-11-03T17:13:34.036 INFO:tasks.workunit.client.1.smithi115.stdout:1/37: rename d7/fb to d7/dc/ff 0 2021-11-03T17:13:34.041 INFO:tasks.workunit.client.1.smithi115.stdout:8/32: dwrite d4/d6/f8 [0,4194304] 0 2021-11-03T17:13:34.041 INFO:tasks.workunit.client.1.smithi115.stdout:1/38: rename f3 to d7/dc/dd/f10 0 2021-11-03T17:13:34.047 INFO:tasks.workunit.client.1.smithi115.stdout:0/66: getdents d9/d15 0 2021-11-03T17:13:34.055 INFO:tasks.workunit.client.1.smithi115.stdout:0/67: creat d9/db/f1b x:0 0 0 2021-11-03T17:13:34.058 INFO:tasks.workunit.client.1.smithi115.stdout:0/68: unlink d9/d15/l18 0 2021-11-03T17:13:34.064 INFO:tasks.workunit.client.1.smithi115.stdout:0/69: symlink d9/db/l1c 0 2021-11-03T17:13:34.065 INFO:tasks.workunit.client.1.smithi115.stdout:8/33: dwrite d4/d6/f8 [4194304,4194304] 0 2021-11-03T17:13:34.066 INFO:tasks.workunit.client.1.smithi115.stdout:8/34: stat d4/l7 0 2021-11-03T17:13:34.066 INFO:tasks.workunit.client.1.smithi115.stdout:8/35: write d4/fa [673729,94688] 0 2021-11-03T17:13:34.066 INFO:tasks.workunit.client.1.smithi115.stdout:8/36: read - d4/d6/fb zero size 2021-11-03T17:13:34.066 INFO:tasks.workunit.client.1.smithi115.stdout:8/37: mknod d4/cc 1 2021-11-03T17:13:34.067 INFO:tasks.workunit.client.1.smithi115.stdout:1/39: dwrite f2 [0,4194304] 0 2021-11-03T17:13:34.068 INFO:tasks.workunit.client.1.smithi115.stdout:7/23: unlink f1 0 2021-11-03T17:13:34.069 INFO:tasks.workunit.client.1.smithi115.stdout:7/24: read - no filename 2021-11-03T17:13:34.071 INFO:tasks.workunit.client.1.smithi115.stdout:1/40: dread d7/dc/ff [0,4194304] 0 2021-11-03T17:13:34.076 INFO:tasks.workunit.client.1.smithi115.stdout:8/38: link d4/d6/fb d4/fd 0 2021-11-03T17:13:34.078 INFO:tasks.workunit.client.1.smithi115.stdout:3/21: write f5 [237295,74634] 0 2021-11-03T17:13:34.081 INFO:tasks.workunit.client.1.smithi115.stdout:5/26: fsync f7 0 2021-11-03T17:13:34.084 INFO:tasks.workunit.client.1.smithi115.stdout:5/27: write f3 [1046001,87714] 0 2021-11-03T17:13:34.084 INFO:tasks.workunit.client.1.smithi115.stdout:5/28: dread - f7 zero size 2021-11-03T17:13:34.087 INFO:tasks.workunit.client.1.smithi115.stdout:0/70: dwrite d9/db/df/f12 [0,4194304] 0 2021-11-03T17:13:34.088 INFO:tasks.workunit.client.1.smithi115.stdout:6/31: getdents d0/d1 0 2021-11-03T17:13:34.095 INFO:tasks.workunit.client.1.smithi115.stdout:7/25: rename l2 to d3/l4 0 2021-11-03T17:13:34.095 INFO:tasks.workunit.client.1.smithi115.stdout:7/26: chown l0 6116248 1 2021-11-03T17:13:34.095 INFO:tasks.workunit.client.1.smithi115.stdout:7/27: write - no filename 2021-11-03T17:13:34.096 INFO:tasks.workunit.client.1.smithi115.stdout:1/41: creat d7/f11 x:0 0 0 2021-11-03T17:13:34.096 INFO:tasks.workunit.client.1.smithi115.stdout:1/42: fsync d7/f9 0 2021-11-03T17:13:34.097 INFO:tasks.workunit.client.1.smithi115.stdout:1/43: truncate d7/fe 1013328 0 2021-11-03T17:13:34.101 INFO:tasks.workunit.client.1.smithi115.stdout:8/39: symlink d4/d6/le 0 2021-11-03T17:13:34.107 INFO:tasks.workunit.client.1.smithi115.stdout:5/29: rename f3 to f9 0 2021-11-03T17:13:34.109 INFO:tasks.workunit.client.1.smithi115.stdout:1/44: dwrite d7/f8 [0,4194304] 0 2021-11-03T17:13:34.113 INFO:tasks.workunit.client.1.smithi115.stdout:0/71: mkdir d9/d15/d1a/d1d 0 2021-11-03T17:13:34.114 INFO:tasks.workunit.client.1.smithi115.stdout:0/72: chown d9/d15 56300 1 2021-11-03T17:13:34.115 INFO:tasks.workunit.client.1.smithi115.stdout:6/32: link d0/f6 d0/d1/fb 0 2021-11-03T17:13:34.115 INFO:tasks.workunit.client.1.smithi115.stdout:6/33: stat d0/d1/fb 0 2021-11-03T17:13:34.116 INFO:tasks.workunit.client.1.smithi115.stdout:7/28: symlink d3/l5 0 2021-11-03T17:13:34.116 INFO:tasks.workunit.client.1.smithi115.stdout:7/29: chown l0 756402123 1 2021-11-03T17:13:34.117 INFO:tasks.workunit.client.1.smithi115.stdout:7/30: fsync - no filename 2021-11-03T17:13:34.117 INFO:tasks.workunit.client.1.smithi115.stdout:7/31: read - no filename 2021-11-03T17:13:34.118 INFO:tasks.workunit.client.1.smithi115.stdout:8/40: unlink d4/d6/fb 0 2021-11-03T17:13:34.118 INFO:tasks.workunit.client.1.smithi115.stdout:8/41: write d4/fa [1034373,16051] 0 2021-11-03T17:13:34.118 INFO:tasks.workunit.client.1.smithi115.stdout:8/42: stat d4/l7 0 2021-11-03T17:13:34.119 INFO:tasks.workunit.client.1.smithi115.stdout:5/30: creat fa x:0 0 0 2021-11-03T17:13:34.119 INFO:tasks.workunit.client.1.smithi115.stdout:5/31: readlink l5 0 2021-11-03T17:13:34.120 INFO:tasks.workunit.client.1.smithi115.stdout:5/32: chown f6 100 1 2021-11-03T17:13:34.120 INFO:tasks.workunit.client.1.smithi115.stdout:5/33: write f6 [1388044,42738] 0 2021-11-03T17:13:34.122 INFO:tasks.workunit.client.1.smithi115.stdout:1/45: unlink f0 0 2021-11-03T17:13:34.122 INFO:tasks.workunit.client.1.smithi115.stdout:6/34: chown d0/d1/d2 18276 1 2021-11-03T17:13:34.124 INFO:tasks.workunit.client.1.smithi115.stdout:0/73: dread f2 [0,4194304] 0 2021-11-03T17:13:34.126 INFO:tasks.workunit.client.1.smithi115.stdout:8/43: link d4/d6/f8 d4/ff 0 2021-11-03T17:13:34.126 INFO:tasks.workunit.client.1.smithi115.stdout:8/44: write d4/fa [1778007,84153] 0 2021-11-03T17:13:34.128 INFO:tasks.workunit.client.1.smithi115.stdout:1/46: symlink d7/l12 0 2021-11-03T17:13:34.131 INFO:tasks.workunit.client.1.smithi115.stdout:0/74: rename d9/db/f19 to d9/d15/f1e 0 2021-11-03T17:13:34.132 INFO:tasks.workunit.client.1.smithi115.stdout:0/75: read d9/fc [624829,64622] 0 2021-11-03T17:13:34.136 INFO:tasks.workunit.client.1.smithi115.stdout:0/76: write f8 [824812,83525] 0 2021-11-03T17:13:34.142 INFO:tasks.workunit.client.1.smithi115.stdout:5/34: dwrite f6 [0,4194304] 0 2021-11-03T17:13:34.142 INFO:tasks.workunit.client.1.smithi115.stdout:5/35: readlink l5 0 2021-11-03T17:13:34.144 INFO:tasks.workunit.client.1.smithi115.stdout:1/47: creat d7/dc/dd/f13 x:0 0 0 2021-11-03T17:13:34.146 INFO:tasks.workunit.client.1.smithi115.stdout:5/36: dread f9 [0,4194304] 0 2021-11-03T17:13:34.146 INFO:tasks.workunit.client.1.smithi115.stdout:5/37: rmdir - no directory 2021-11-03T17:13:34.147 INFO:tasks.workunit.client.1.smithi115.stdout:5/38: mknod cb 1 2021-11-03T17:13:34.148 INFO:tasks.workunit.client.1.smithi115.stdout:8/45: getdents d4/d6 0 2021-11-03T17:13:34.156 INFO:tasks.workunit.client.1.smithi115.stdout:8/46: dread d4/ff [4194304,4194304] 0 2021-11-03T17:13:34.156 INFO:tasks.workunit.client.1.smithi115.stdout:5/39: symlink lc 0 2021-11-03T17:13:34.157 INFO:tasks.workunit.client.1.smithi115.stdout:5/40: chown fa 6 1 2021-11-03T17:13:34.157 INFO:tasks.workunit.client.1.smithi115.stdout:5/41: write fa [725885,428] 0 2021-11-03T17:13:34.162 INFO:tasks.workunit.client.1.smithi115.stdout:1/48: getdents d7/dc/dd 0 2021-11-03T17:13:34.163 INFO:tasks.workunit.client.1.smithi115.stdout:1/49: write d7/f8 [235092,38704] 0 2021-11-03T17:13:34.164 INFO:tasks.workunit.client.1.smithi115.stdout:1/50: chown d7/f11 78179 1 2021-11-03T17:13:34.166 INFO:tasks.workunit.client.1.smithi115.stdout:8/47: write d4/d6/f8 [2904780,67608] 0 2021-11-03T17:13:34.167 INFO:tasks.workunit.client.1.smithi115.stdout:5/42: creat fd x:0 0 0 2021-11-03T17:13:34.168 INFO:tasks.workunit.client.1.smithi115.stdout:5/43: write f0 [83963,115568] 0 2021-11-03T17:13:34.170 INFO:tasks.workunit.client.1.smithi115.stdout:5/44: rename f6 to fe 0 2021-11-03T17:13:34.171 INFO:tasks.workunit.client.1.smithi115.stdout:8/48: symlink d4/l10 0 2021-11-03T17:13:34.173 INFO:tasks.workunit.client.1.smithi115.stdout:5/45: unlink lc 0 2021-11-03T17:13:34.174 INFO:tasks.workunit.client.1.smithi115.stdout:1/51: dread f2 [0,4194304] 0 2021-11-03T17:13:34.175 INFO:tasks.workunit.client.1.smithi115.stdout:5/46: rename fa to ff 0 2021-11-03T17:13:34.177 INFO:tasks.workunit.client.1.smithi115.stdout:1/52: getdents d7 0 2021-11-03T17:13:34.181 INFO:tasks.workunit.client.1.smithi115.stdout:1/53: creat d7/f14 x:0 0 0 2021-11-03T17:13:34.184 INFO:tasks.workunit.client.1.smithi115.stdout:1/54: creat d7/dc/f15 x:0 0 0 2021-11-03T17:13:34.188 INFO:tasks.workunit.client.1.smithi115.stdout:1/55: write d7/fa [544139,105632] 0 2021-11-03T17:13:34.189 INFO:tasks.workunit.client.1.smithi115.stdout:1/56: truncate d7/f14 338433 0 2021-11-03T17:13:34.192 INFO:tasks.workunit.client.1.smithi115.stdout:1/57: creat d7/dc/f16 x:0 0 0 2021-11-03T17:13:34.193 INFO:tasks.workunit.client.1.smithi115.stdout:5/47: dwrite fd [0,4194304] 0 2021-11-03T17:13:34.193 INFO:tasks.workunit.client.1.smithi115.stdout:5/48: chown fe 37484538 1 2021-11-03T17:13:34.193 INFO:tasks.workunit.client.1.smithi115.stdout:5/49: chown ff 128540 1 2021-11-03T17:13:34.194 INFO:tasks.workunit.client.1.smithi115.stdout:5/50: fsync ff 0 2021-11-03T17:13:34.194 INFO:tasks.workunit.client.1.smithi115.stdout:1/58: chown d7/f9 40653169 1 2021-11-03T17:13:34.194 INFO:tasks.workunit.client.1.smithi115.stdout:1/59: mknod d7/dc/dd/c17 1 2021-11-03T17:13:34.196 INFO:tasks.workunit.client.1.smithi115.stdout:5/51: symlink l10 0 2021-11-03T17:13:34.196 INFO:tasks.workunit.client.1.smithi115.stdout:5/52: stat f0 0 2021-11-03T17:13:34.196 INFO:tasks.workunit.client.1.smithi115.stdout:5/53: fsync fd 0 2021-11-03T17:13:34.198 INFO:tasks.workunit.client.1.smithi115.stdout:1/60: rename d7/dc/dd/f10 to d7/dc/dd/f18 0 2021-11-03T17:13:34.199 INFO:tasks.workunit.client.1.smithi115.stdout:5/54: creat f11 x:0 0 0 2021-11-03T17:13:34.203 INFO:tasks.workunit.client.1.smithi115.stdout:1/61: write d7/f8 [3044450,80187] 0 2021-11-03T17:13:34.207 INFO:tasks.workunit.client.1.smithi115.stdout:1/62: symlink d7/dc/l19 0 2021-11-03T17:13:34.207 INFO:tasks.workunit.client.1.smithi115.stdout:1/63: dread - d7/dc/f15 zero size 2021-11-03T17:13:34.209 INFO:tasks.workunit.client.1.smithi115.stdout:1/64: mkdir d7/dc/d1a 0 2021-11-03T17:13:34.223 INFO:tasks.workunit.client.1.smithi115.stdout:1/65: dwrite d7/dc/ff [0,4194304] 0 2021-11-03T17:13:34.224 INFO:tasks.workunit.client.1.smithi115.stdout:1/66: read d7/dc/dd/f18 [456742,79343] 0 2021-11-03T17:13:34.224 INFO:tasks.workunit.client.1.smithi115.stdout:1/67: chown d7/dc/d1a 31 1 2021-11-03T17:13:34.225 INFO:tasks.workunit.client.1.smithi115.stdout:1/68: dread - d7/dc/f15 zero size 2021-11-03T17:13:34.226 INFO:tasks.workunit.client.1.smithi115.stdout:1/69: write d7/f8 [2548009,100219] 0 2021-11-03T17:13:34.227 INFO:tasks.workunit.client.1.smithi115.stdout:1/70: mknod d7/dc/d1a/c1b 1 2021-11-03T17:13:34.228 INFO:tasks.workunit.client.1.smithi115.stdout:1/71: dread d7/fe [0,4194304] 0 2021-11-03T17:13:34.229 INFO:tasks.workunit.client.1.smithi115.stdout:1/72: write d7/dc/ff [1527119,48961] 0 2021-11-03T17:13:34.231 INFO:tasks.workunit.client.1.smithi115.stdout:1/73: symlink d7/dc/l1c 0 2021-11-03T17:13:34.248 INFO:tasks.workunit.client.1.smithi115.stdout:1/74: dwrite d7/f14 [0,4194304] 0 2021-11-03T17:13:34.253 INFO:tasks.workunit.client.1.smithi115.stdout:1/75: getdents d7 0 2021-11-03T17:13:34.253 INFO:tasks.workunit.client.1.smithi115.stdout:1/76: write d7/dc/f16 [393833,67352] 0 2021-11-03T17:13:34.256 INFO:tasks.workunit.client.1.smithi115.stdout:1/77: rename d7/l12 to d7/l1d 0 2021-11-03T17:13:34.258 INFO:tasks.workunit.client.1.smithi115.stdout:1/78: fsync d7/dc/f16 0 2021-11-03T17:13:34.271 INFO:tasks.workunit.client.1.smithi115.stdout:1/79: dread d7/f14 [0,4194304] 0 2021-11-03T17:13:34.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:34 smithi115 conmon[34670]: cluster 2021-11-03T17:13:34.596958+0000 2021-11-03T17:13:34.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:34 smithi115 conmon[34670]: mgr.smithi027.ufhhvb (mgr.14162) 282 : cluster [DBG] pgmap v234: 65 pgs: 65 active+clean; 165 MiB data, 680 MiB used, 536 GiB / 536 GiB avail; 707 KiB/s rd, 11 MiB/s wr, 689 op/s 2021-11-03T17:13:35.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:34 smithi027 conmon[32322]: cluster 2021-11-03T17:13:34.596958+0000 2021-11-03T17:13:35.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:34 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14162) 282 : cluster [DBG] pgmap v234: 65 pgs: 65 active+clean; 165 MiB data, 680 MiB used, 536 GiB / 536 GiB avail; 707 KiB/s rd, 11 MiB/s wr, 689 op/s 2021-11-03T17:13:36.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:36 smithi115 conmon[34670]: cluster 2021-11-03T17:13:36.597525+0000 mgr.smithi027.ufhhvb (mgr.14162) 2021-11-03T17:13:36.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:36 smithi115 conmon[34670]: 283 : cluster [DBG] pgmap v235: 65 pgs: 65 active+clean; 231 MiB data, 862 MiB used, 536 GiB / 536 GiB avail; 4.5 MiB/s rd, 19 MiB/s wr, 713 op/s 2021-11-03T17:13:37.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:36 smithi027 conmon[32322]: cluster 2021-11-03T17:13:36.597525 2021-11-03T17:13:37.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:36 smithi027 conmon[32322]: +0000 mgr.smithi027.ufhhvb (mgr.14162) 283 : cluster [DBG] pgmap v235: 65 pgs: 65 active+clean; 231 MiB data, 862 MiB used, 536 GiB / 536 GiB avail; 4.5 MiB/s rd, 19 MiB/s wr, 713 op/s 2021-11-03T17:13:38.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:38 smithi115 conmon[34670]: cluster 2021-11-03T17:13:38.597943+0000 mgr.smithi027.ufhhvb (mgr.14162) 284 : cluster 2021-11-03T17:13:38.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:38 smithi115 conmon[34670]: [DBG] pgmap v236: 65 pgs: 65 active+clean; 233 MiB data, 888 MiB used, 536 GiB / 536 GiB avail; 5.1 MiB/s rd, 19 MiB/s wr, 690 op/s 2021-11-03T17:13:39.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:38 smithi027 conmon[32322]: cluster 2021-11-03T17:13:38.597943+0000 mgr.smithi027.ufhhvb (mgr.14162) 284 : 2021-11-03T17:13:39.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:38 smithi027 conmon[32322]: cluster [DBG] pgmap v236: 65 pgs: 65 active+clean; 233 MiB data, 888 MiB used, 536 GiB / 536 GiB avail; 5.1 MiB/s rd, 19 MiB/s wr, 690 op/s 2021-11-03T17:13:40.324 INFO:tasks.workunit.client.1.smithi115.stdout:2/7: sync 2021-11-03T17:13:40.324 INFO:tasks.workunit.client.1.smithi115.stdout:2/8: write - no filename 2021-11-03T17:13:40.384 INFO:tasks.workunit.client.1.smithi115.stdout:6/35: sync 2021-11-03T17:13:40.384 INFO:tasks.workunit.client.1.smithi115.stdout:7/32: sync 2021-11-03T17:13:40.385 INFO:tasks.workunit.client.1.smithi115.stdout:5/55: sync 2021-11-03T17:13:40.385 INFO:tasks.workunit.client.1.smithi115.stdout:0/77: sync 2021-11-03T17:13:40.385 INFO:tasks.workunit.client.1.smithi115.stdout:4/7: sync 2021-11-03T17:13:40.385 INFO:tasks.workunit.client.1.smithi115.stdout:3/22: sync 2021-11-03T17:13:40.386 INFO:tasks.workunit.client.1.smithi115.stdout:8/49: sync 2021-11-03T17:13:40.386 INFO:tasks.workunit.client.1.smithi115.stdout:7/33: chown d3 4 1 2021-11-03T17:13:40.386 INFO:tasks.workunit.client.1.smithi115.stdout:7/34: write - no filename 2021-11-03T17:13:40.387 INFO:tasks.workunit.client.1.smithi115.stdout:5/56: rename ff to f12 0 2021-11-03T17:13:40.388 INFO:tasks.workunit.client.1.smithi115.stdout:4/8: rename f2 to f3 0 2021-11-03T17:13:40.388 INFO:tasks.workunit.client.1.smithi115.stdout:4/9: dread - f3 zero size 2021-11-03T17:13:40.389 INFO:tasks.workunit.client.1.smithi115.stdout:7/35: rename d3/l4 to d3/l6 0 2021-11-03T17:13:40.390 INFO:tasks.workunit.client.1.smithi115.stdout:7/36: dwrite - no filename 2021-11-03T17:13:40.391 INFO:tasks.workunit.client.1.smithi115.stdout:0/78: mkdir d9/d15/d1f 0 2021-11-03T17:13:40.393 INFO:tasks.workunit.client.1.smithi115.stdout:6/36: creat d0/d1/d2/fc x:0 0 0 2021-11-03T17:13:40.394 INFO:tasks.workunit.client.1.smithi115.stdout:5/57: truncate fd 1874864 0 2021-11-03T17:13:40.395 INFO:tasks.workunit.client.1.smithi115.stdout:8/50: truncate d4/fa 1838624 0 2021-11-03T17:13:40.398 INFO:tasks.workunit.client.1.smithi115.stdout:4/10: symlink l4 0 2021-11-03T17:13:40.398 INFO:tasks.workunit.client.1.smithi115.stdout:4/11: fdatasync f3 0 2021-11-03T17:13:40.398 INFO:tasks.workunit.client.1.smithi115.stdout:4/12: dread - f3 zero size 2021-11-03T17:13:40.398 INFO:tasks.workunit.client.1.smithi115.stdout:4/13: chown l1 31481 1 2021-11-03T17:13:40.399 INFO:tasks.workunit.client.1.smithi115.stdout:4/14: write f3 [302090,76210] 0 2021-11-03T17:13:40.399 INFO:tasks.workunit.client.1.smithi115.stdout:4/15: write f3 [1196829,29275] 0 2021-11-03T17:13:40.399 INFO:tasks.workunit.client.1.smithi115.stdout:7/37: symlink d3/l7 0 2021-11-03T17:13:40.399 INFO:tasks.workunit.client.1.smithi115.stdout:7/38: write - no filename 2021-11-03T17:13:40.400 INFO:tasks.workunit.client.1.smithi115.stdout:7/39: dread - no filename 2021-11-03T17:13:40.400 INFO:tasks.workunit.client.1.smithi115.stdout:7/40: chown l0 3126981 1 2021-11-03T17:13:40.401 INFO:tasks.workunit.client.1.smithi115.stdout:0/79: creat d9/db/f20 x:0 0 0 2021-11-03T17:13:40.401 INFO:tasks.workunit.client.1.smithi115.stdout:0/80: readlink d9/le 0 2021-11-03T17:13:40.401 INFO:tasks.workunit.client.1.smithi115.stdout:6/37: dread - d0/d1/fa zero size 2021-11-03T17:13:40.401 INFO:tasks.workunit.client.1.smithi115.stdout:6/38: readlink - no filename 2021-11-03T17:13:40.402 INFO:tasks.workunit.client.1.smithi115.stdout:3/23: rename l6 to l8 0 2021-11-03T17:13:40.405 INFO:tasks.workunit.client.1.smithi115.stdout:3/24: dread f5 [0,4194304] 0 2021-11-03T17:13:40.406 INFO:tasks.workunit.client.1.smithi115.stdout:3/25: write f5 [2160778,30039] 0 2021-11-03T17:13:40.409 INFO:tasks.workunit.client.1.smithi115.stdout:5/58: symlink l13 0 2021-11-03T17:13:40.410 INFO:tasks.workunit.client.1.smithi115.stdout:4/16: creat f5 x:0 0 0 2021-11-03T17:13:40.410 INFO:tasks.workunit.client.1.smithi115.stdout:6/39: write d0/d1/f8 [403062,32611] 0 2021-11-03T17:13:40.411 INFO:tasks.workunit.client.1.smithi115.stdout:5/59: symlink l14 0 2021-11-03T17:13:40.411 INFO:tasks.workunit.client.1.smithi115.stdout:5/60: mknod c15 1 2021-11-03T17:13:40.412 INFO:tasks.workunit.client.1.smithi115.stdout:6/40: truncate d0/d1/f5 314562 0 2021-11-03T17:13:40.413 INFO:tasks.workunit.client.1.smithi115.stdout:6/41: creat d0/d1/d2/fd x:0 0 0 2021-11-03T17:13:40.414 INFO:tasks.workunit.client.1.smithi115.stdout:5/61: write f0 [3268819,48067] 0 2021-11-03T17:13:40.415 INFO:tasks.workunit.client.1.smithi115.stdout:6/42: rename d0/d1/fa to d0/d1/fe 0 2021-11-03T17:13:40.415 INFO:tasks.workunit.client.1.smithi115.stdout:6/43: truncate d0/d1/d2/fc 68577 0 2021-11-03T17:13:40.416 INFO:tasks.workunit.client.1.smithi115.stdout:3/26: fsync f5 0 2021-11-03T17:13:40.416 INFO:tasks.workunit.client.1.smithi115.stdout:0/81: dwrite d9/db/f1b [0,4194304] 0 2021-11-03T17:13:40.418 INFO:tasks.workunit.client.1.smithi115.stdout:5/62: rename l5 to l16 0 2021-11-03T17:13:40.418 INFO:tasks.workunit.client.1.smithi115.stdout:5/63: fdatasync f9 0 2021-11-03T17:13:40.422 INFO:tasks.workunit.client.1.smithi115.stdout:9/13: sync 2021-11-03T17:13:40.422 INFO:tasks.workunit.client.1.smithi115.stdout:9/14: fsync d1/f2 0 2021-11-03T17:13:40.425 INFO:tasks.workunit.client.1.smithi115.stdout:1/80: sync 2021-11-03T17:13:40.435 INFO:tasks.workunit.client.1.smithi115.stdout:5/64: creat f17 x:0 0 0 2021-11-03T17:13:40.435 INFO:tasks.workunit.client.1.smithi115.stdout:5/65: mknod c18 1 2021-11-03T17:13:40.435 INFO:tasks.workunit.client.1.smithi115.stdout:4/17: getdents . 0 2021-11-03T17:13:40.438 INFO:tasks.workunit.client.1.smithi115.stdout:6/44: write d0/d1/fb [338838,90968] 0 2021-11-03T17:13:40.438 INFO:tasks.workunit.client.1.smithi115.stdout:6/45: truncate d0/d1/d2/fc 881750 0 2021-11-03T17:13:40.440 INFO:tasks.workunit.client.1.smithi115.stdout:6/46: creat d0/d1/d2/ff x:0 0 0 2021-11-03T17:13:40.448 INFO:tasks.workunit.client.1.smithi115.stdout:5/66: dread fe [0,4194304] 0 2021-11-03T17:13:40.449 INFO:tasks.workunit.client.1.smithi115.stdout:5/67: creat f19 x:0 0 0 2021-11-03T17:13:40.450 INFO:tasks.workunit.client.1.smithi115.stdout:5/68: write fe [55038,116268] 0 2021-11-03T17:13:40.463 INFO:tasks.workunit.client.1.smithi115.stdout:9/15: dwrite d1/f2 [0,4194304] 0 2021-11-03T17:13:40.463 INFO:tasks.workunit.client.1.smithi115.stdout:4/18: dwrite f5 [0,4194304] 0 2021-11-03T17:13:40.467 INFO:tasks.workunit.client.1.smithi115.stdout:4/19: creat f6 x:0 0 0 2021-11-03T17:13:40.469 INFO:tasks.workunit.client.1.smithi115.stdout:0/82: dwrite d9/db/df/f12 [0,4194304] 0 2021-11-03T17:13:40.469 INFO:tasks.workunit.client.1.smithi115.stdout:8/51: dwrite d4/fa [0,4194304] 0 2021-11-03T17:13:40.470 INFO:tasks.workunit.client.1.smithi115.stdout:3/27: truncate f5 851271 0 2021-11-03T17:13:40.472 INFO:tasks.workunit.client.1.smithi115.stdout:0/83: link d9/db/ld d9/d15/l21 0 2021-11-03T17:13:40.474 INFO:tasks.workunit.client.1.smithi115.stdout:6/47: dwrite d0/d1/f8 [0,4194304] 0 2021-11-03T17:13:40.479 INFO:tasks.workunit.client.1.smithi115.stdout:5/69: dwrite f7 [0,4194304] 0 2021-11-03T17:13:40.479 INFO:tasks.workunit.client.1.smithi115.stdout:9/16: dread d1/f2 [0,4194304] 0 2021-11-03T17:13:40.479 INFO:tasks.workunit.client.1.smithi115.stdout:9/17: stat d1/f2 0 2021-11-03T17:13:40.479 INFO:tasks.workunit.client.1.smithi115.stdout:9/18: chown d1 2961079 1 2021-11-03T17:13:40.480 INFO:tasks.workunit.client.1.smithi115.stdout:8/52: creat d4/f11 x:0 0 0 2021-11-03T17:13:40.480 INFO:tasks.workunit.client.1.smithi115.stdout:3/28: creat f9 x:0 0 0 2021-11-03T17:13:40.481 INFO:tasks.workunit.client.1.smithi115.stdout:9/19: write d1/f2 [3718741,127110] 0 2021-11-03T17:13:40.481 INFO:tasks.workunit.client.1.smithi115.stdout:9/20: stat d1 0 2021-11-03T17:13:40.481 INFO:tasks.workunit.client.1.smithi115.stdout:0/84: creat d9/d15/d1a/d1d/f22 x:0 0 0 2021-11-03T17:13:40.482 INFO:tasks.workunit.client.1.smithi115.stdout:0/85: write f2 [1972581,109146] 0 2021-11-03T17:13:40.488 INFO:tasks.workunit.client.1.smithi115.stdout:5/70: dread f0 [0,4194304] 0 2021-11-03T17:13:40.489 INFO:tasks.workunit.client.1.smithi115.stdout:5/71: stat f17 0 2021-11-03T17:13:40.490 INFO:tasks.workunit.client.1.smithi115.stdout:9/21: symlink d1/l3 0 2021-11-03T17:13:40.492 INFO:tasks.workunit.client.1.smithi115.stdout:0/86: unlink f2 0 2021-11-03T17:13:40.494 INFO:tasks.workunit.client.1.smithi115.stdout:5/72: getdents . 0 2021-11-03T17:13:40.494 INFO:tasks.workunit.client.1.smithi115.stdout:5/73: dread - f17 zero size 2021-11-03T17:13:40.494 INFO:tasks.workunit.client.1.smithi115.stdout:4/20: dwrite f3 [0,4194304] 0 2021-11-03T17:13:40.495 INFO:tasks.workunit.client.1.smithi115.stdout:4/21: fdatasync f5 0 2021-11-03T17:13:40.495 INFO:tasks.workunit.client.1.smithi115.stdout:4/22: stat l4 0 2021-11-03T17:13:40.496 INFO:tasks.workunit.client.1.smithi115.stdout:3/29: truncate f5 1061382 0 2021-11-03T17:13:40.501 INFO:tasks.workunit.client.1.smithi115.stdout:5/74: dread fe [0,4194304] 0 2021-11-03T17:13:40.501 INFO:tasks.workunit.client.1.smithi115.stdout:0/87: rename d9/fc to d9/f23 0 2021-11-03T17:13:40.504 INFO:tasks.workunit.client.1.smithi115.stdout:3/30: read f5 [536057,31652] 0 2021-11-03T17:13:40.505 INFO:tasks.workunit.client.1.smithi115.stdout:3/31: read - f9 zero size 2021-11-03T17:13:40.510 INFO:tasks.workunit.client.1.smithi115.stdout:9/22: dwrite d1/f2 [4194304,4194304] 0 2021-11-03T17:13:40.511 INFO:tasks.workunit.client.1.smithi115.stdout:9/23: write d1/f2 [5444723,36707] 0 2021-11-03T17:13:40.512 INFO:tasks.workunit.client.1.smithi115.stdout:4/23: mkdir d7 0 2021-11-03T17:13:40.512 INFO:tasks.workunit.client.1.smithi115.stdout:8/53: rename d4/ff to d4/f12 0 2021-11-03T17:13:40.512 INFO:tasks.workunit.client.1.smithi115.stdout:0/88: creat d9/d15/d1a/d1d/f24 x:0 0 0 2021-11-03T17:13:40.512 INFO:tasks.workunit.client.1.smithi115.stdout:3/32: getdents . 0 2021-11-03T17:13:40.513 INFO:tasks.workunit.client.1.smithi115.stdout:3/33: stat l7 0 2021-11-03T17:13:40.513 INFO:tasks.workunit.client.1.smithi115.stdout:3/34: mknod ca 1 2021-11-03T17:13:40.515 INFO:tasks.workunit.client.1.smithi115.stdout:9/24: symlink d1/l4 0 2021-11-03T17:13:40.518 INFO:tasks.workunit.client.1.smithi115.stdout:8/54: creat d4/d6/f13 x:0 0 0 2021-11-03T17:13:40.519 INFO:tasks.workunit.client.1.smithi115.stdout:4/24: rename f3 to d7/f8 0 2021-11-03T17:13:40.520 INFO:tasks.workunit.client.1.smithi115.stdout:4/25: write f5 [1913563,16993] 0 2021-11-03T17:13:40.521 INFO:tasks.workunit.client.1.smithi115.stdout:4/26: chown f6 1 1 2021-11-03T17:13:40.521 INFO:tasks.workunit.client.1.smithi115.stdout:0/89: rename d9/db/df/f11 to d9/f25 0 2021-11-03T17:13:40.522 INFO:tasks.workunit.client.1.smithi115.stdout:0/90: stat d9/db/f20 0 2021-11-03T17:13:40.522 INFO:tasks.workunit.client.1.smithi115.stdout:0/91: mknod d9/d15/d1a/d1d/c26 1 2021-11-03T17:13:40.522 INFO:tasks.workunit.client.1.smithi115.stdout:0/92: truncate d9/d15/d1a/d1d/f22 713255 0 2021-11-03T17:13:40.523 INFO:tasks.workunit.client.1.smithi115.stdout:4/27: truncate d7/f8 3574596 0 2021-11-03T17:13:40.524 INFO:tasks.workunit.client.1.smithi115.stdout:0/93: symlink d9/d15/d1a/l27 0 2021-11-03T17:13:40.525 INFO:tasks.workunit.client.1.smithi115.stdout:4/28: symlink d7/l9 0 2021-11-03T17:13:40.532 INFO:tasks.workunit.client.1.smithi115.stdout:3/35: dwrite f5 [0,4194304] 0 2021-11-03T17:13:40.532 INFO:tasks.workunit.client.1.smithi115.stdout:3/36: write f9 [997647,28118] 0 2021-11-03T17:13:40.535 INFO:tasks.workunit.client.1.smithi115.stdout:9/25: dread d1/f2 [0,4194304] 0 2021-11-03T17:13:40.540 INFO:tasks.workunit.client.1.smithi115.stdout:0/94: dread f8 [0,4194304] 0 2021-11-03T17:13:40.542 INFO:tasks.workunit.client.1.smithi115.stdout:9/26: getdents d1 0 2021-11-03T17:13:40.544 INFO:tasks.workunit.client.1.smithi115.stdout:8/55: dwrite d4/d6/f8 [0,4194304] 0 2021-11-03T17:13:40.546 INFO:tasks.workunit.client.1.smithi115.stdout:0/95: symlink d9/l28 0 2021-11-03T17:13:40.547 INFO:tasks.workunit.client.1.smithi115.stdout:0/96: chown d9/d15/d1a/l27 13379 1 2021-11-03T17:13:40.547 INFO:tasks.workunit.client.1.smithi115.stdout:0/97: stat d9/d15/f16 0 2021-11-03T17:13:40.548 INFO:tasks.workunit.client.1.smithi115.stdout:9/27: mkdir d1/d5 0 2021-11-03T17:13:40.549 INFO:tasks.workunit.client.1.smithi115.stdout:0/98: rmdir d9/d17 0 2021-11-03T17:13:40.550 INFO:tasks.workunit.client.1.smithi115.stdout:9/28: creat d1/d5/f6 x:0 0 0 2021-11-03T17:13:40.551 INFO:tasks.workunit.client.1.smithi115.stdout:8/56: link d4/fa d4/f14 0 2021-11-03T17:13:40.553 INFO:tasks.workunit.client.1.smithi115.stdout:9/29: creat d1/f7 x:0 0 0 2021-11-03T17:13:40.554 INFO:tasks.workunit.client.1.smithi115.stdout:9/30: creat d1/d5/f8 x:0 0 0 2021-11-03T17:13:40.557 INFO:tasks.workunit.client.1.smithi115.stdout:5/75: dwrite fd [0,4194304] 0 2021-11-03T17:13:40.558 INFO:tasks.workunit.client.1.smithi115.stdout:3/37: dwrite f5 [4194304,4194304] 0 2021-11-03T17:13:40.559 INFO:tasks.workunit.client.1.smithi115.stdout:5/76: write fd [1682880,126840] 0 2021-11-03T17:13:40.561 INFO:tasks.workunit.client.1.smithi115.stdout:8/57: write d4/f12 [4095532,40917] 0 2021-11-03T17:13:40.561 INFO:tasks.workunit.client.1.smithi115.stdout:0/99: dread f0 [0,4194304] 0 2021-11-03T17:13:40.562 INFO:tasks.workunit.client.1.smithi115.stdout:0/100: write d9/d15/f1e [600496,26096] 0 2021-11-03T17:13:40.567 INFO:tasks.workunit.client.1.smithi115.stdout:5/77: mkdir d1a 0 2021-11-03T17:13:40.570 INFO:tasks.workunit.client.1.smithi115.stdout:3/38: dread f5 [0,4194304] 0 2021-11-03T17:13:40.570 INFO:tasks.workunit.client.1.smithi115.stdout:3/39: mknod cb 1 2021-11-03T17:13:40.570 INFO:tasks.workunit.client.1.smithi115.stdout:0/101: creat d9/d15/d1a/d1d/f29 x:0 0 0 2021-11-03T17:13:40.571 INFO:tasks.workunit.client.1.smithi115.stdout:9/31: dwrite d1/d5/f6 [0,4194304] 0 2021-11-03T17:13:40.572 INFO:tasks.workunit.client.1.smithi115.stdout:9/32: read - d1/f7 zero size 2021-11-03T17:13:40.572 INFO:tasks.workunit.client.1.smithi115.stdout:5/78: mkdir d1a/d1b 0 2021-11-03T17:13:40.572 INFO:tasks.workunit.client.1.smithi115.stdout:5/79: fsync f0 0 2021-11-03T17:13:40.573 INFO:tasks.workunit.client.1.smithi115.stdout:8/58: symlink d4/l15 0 2021-11-03T17:13:40.573 INFO:tasks.workunit.client.1.smithi115.stdout:8/59: readlink d4/l10 0 2021-11-03T17:13:40.579 INFO:tasks.workunit.client.1.smithi115.stdout:9/33: rename d1/f2 to d1/f9 0 2021-11-03T17:13:40.579 INFO:tasks.workunit.client.1.smithi115.stdout:8/60: rename d4/d6/f13 to d4/d6/f16 0 2021-11-03T17:13:40.580 INFO:tasks.workunit.client.1.smithi115.stdout:9/34: dread - d1/f7 zero size 2021-11-03T17:13:40.581 INFO:tasks.workunit.client.1.smithi115.stdout:8/61: truncate d4/f5 518968 0 2021-11-03T17:13:40.581 INFO:tasks.workunit.client.1.smithi115.stdout:8/62: mknod d4/c17 1 2021-11-03T17:13:40.582 INFO:tasks.workunit.client.1.smithi115.stdout:8/63: mkdir d4/d6/d18 0 2021-11-03T17:13:40.587 INFO:tasks.workunit.client.1.smithi115.stdout:9/35: write d1/d5/f6 [5237178,93884] 0 2021-11-03T17:13:40.587 INFO:tasks.workunit.client.1.smithi115.stdout:9/36: mknod d1/d5/ca 1 2021-11-03T17:13:40.587 INFO:tasks.workunit.client.1.smithi115.stdout:9/37: chown d1 556 1 2021-11-03T17:13:40.588 INFO:tasks.workunit.client.1.smithi115.stdout:9/38: mknod d1/cb 1 2021-11-03T17:13:40.588 INFO:tasks.workunit.client.1.smithi115.stdout:9/39: chown d1/f7 419650 1 2021-11-03T17:13:40.589 INFO:tasks.workunit.client.1.smithi115.stdout:9/40: link d1/d5/f6 d1/d5/fc 0 2021-11-03T17:13:40.590 INFO:tasks.workunit.client.1.smithi115.stdout:0/102: truncate d9/d15/d1a/d1d/f22 248093 0 2021-11-03T17:13:40.590 INFO:tasks.workunit.client.1.smithi115.stdout:0/103: chown d9/d15/d1a/l27 19511940 1 2021-11-03T17:13:40.591 INFO:tasks.workunit.client.1.smithi115.stdout:0/104: mknod d9/db/df/c2a 1 2021-11-03T17:13:40.591 INFO:tasks.workunit.client.1.smithi115.stdout:0/105: mknod d9/db/df/c2b 1 2021-11-03T17:13:40.591 INFO:tasks.workunit.client.1.smithi115.stdout:9/41: symlink d1/ld 0 2021-11-03T17:13:40.592 INFO:tasks.workunit.client.1.smithi115.stdout:9/42: chown d1/ld 29607825 1 2021-11-03T17:13:40.592 INFO:tasks.workunit.client.1.smithi115.stdout:0/106: getdents d9/db/df 0 2021-11-03T17:13:40.594 INFO:tasks.workunit.client.1.smithi115.stdout:9/43: link d1/d5/fc d1/d5/fe 0 2021-11-03T17:13:40.595 INFO:tasks.workunit.client.1.smithi115.stdout:8/64: dread d4/f14 [0,4194304] 0 2021-11-03T17:13:40.596 INFO:tasks.workunit.client.1.smithi115.stdout:8/65: mknod d4/d6/d18/c19 1 2021-11-03T17:13:40.596 INFO:tasks.workunit.client.1.smithi115.stdout:0/107: mkdir d9/d15/d2c 0 2021-11-03T17:13:40.598 INFO:tasks.workunit.client.1.smithi115.stdout:3/40: dwrite f5 [4194304,4194304] 0 2021-11-03T17:13:40.598 INFO:tasks.workunit.client.1.smithi115.stdout:5/80: dwrite f12 [0,4194304] 0 2021-11-03T17:13:40.599 INFO:tasks.workunit.client.1.smithi115.stdout:5/81: getdents d1a/d1b 0 2021-11-03T17:13:40.599 INFO:tasks.workunit.client.1.smithi115.stdout:5/82: stat l10 0 2021-11-03T17:13:40.600 INFO:tasks.workunit.client.1.smithi115.stdout:9/44: write d1/d5/f6 [1386773,100708] 0 2021-11-03T17:13:40.600 INFO:tasks.workunit.client.1.smithi115.stdout:3/41: write f5 [5080755,119303] 0 2021-11-03T17:13:40.601 INFO:tasks.workunit.client.1.smithi115.stdout:8/66: mknod d4/c1a 1 2021-11-03T17:13:40.603 INFO:tasks.workunit.client.1.smithi115.stdout:5/83: dread fe [0,4194304] 0 2021-11-03T17:13:40.603 INFO:tasks.workunit.client.1.smithi115.stdout:5/84: fdatasync f7 0 2021-11-03T17:13:40.604 INFO:tasks.workunit.client.1.smithi115.stdout:5/85: mknod d1a/c1c 1 2021-11-03T17:13:40.609 INFO:tasks.workunit.client.1.smithi115.stdout:5/86: creat d1a/f1d x:0 0 0 2021-11-03T17:13:40.626 INFO:tasks.workunit.client.1.smithi115.stdout:5/87: dread f7 [0,4194304] 0 2021-11-03T17:13:40.627 INFO:tasks.workunit.client.1.smithi115.stdout:8/67: dwrite d4/f11 [0,4194304] 0 2021-11-03T17:13:40.630 INFO:tasks.workunit.client.1.smithi115.stdout:3/42: dwrite f5 [0,4194304] 0 2021-11-03T17:13:40.634 INFO:tasks.workunit.client.1.smithi115.stdout:3/43: write f9 [378552,17297] 0 2021-11-03T17:13:40.634 INFO:tasks.workunit.client.1.smithi115.stdout:3/44: readlink l8 0 2021-11-03T17:13:40.663 INFO:tasks.workunit.client.1.smithi115.stdout:3/45: dwrite f5 [4194304,4194304] 0 2021-11-03T17:13:40.667 INFO:tasks.workunit.client.1.smithi115.stdout:3/46: dread f9 [0,4194304] 0 2021-11-03T17:13:40.667 INFO:tasks.workunit.client.1.smithi115.stdout:3/47: getdents . 0 2021-11-03T17:13:40.667 INFO:tasks.workunit.client.1.smithi115.stdout:3/48: chown l8 5 1 2021-11-03T17:13:40.668 INFO:tasks.workunit.client.1.smithi115.stdout:3/49: truncate f5 8595463 0 2021-11-03T17:13:40.669 INFO:tasks.workunit.client.1.smithi115.stdout:3/50: dread f9 [0,4194304] 0 2021-11-03T17:13:40.674 INFO:tasks.workunit.client.1.smithi115.stdout:5/88: link f17 d1a/f1e 0 2021-11-03T17:13:40.674 INFO:tasks.workunit.client.1.smithi115.stdout:5/89: readlink l14 0 2021-11-03T17:13:40.674 INFO:tasks.workunit.client.1.smithi115.stdout:5/90: write f11 [333060,14108] 0 2021-11-03T17:13:40.696 INFO:tasks.workunit.client.1.smithi115.stdout:8/68: mknod d4/c1b 1 2021-11-03T17:13:40.715 INFO:tasks.workunit.client.1.smithi115.stdout:3/51: creat fc x:0 0 0 2021-11-03T17:13:40.721 INFO:tasks.workunit.client.1.smithi115.stdout:5/91: creat d1a/f1f x:0 0 0 2021-11-03T17:13:40.725 INFO:tasks.workunit.client.1.smithi115.stdout:8/69: mknod d4/c1c 1 2021-11-03T17:13:40.726 INFO:tasks.workunit.client.1.smithi115.stdout:8/70: readlink d4/l10 0 2021-11-03T17:13:40.731 INFO:tasks.workunit.client.1.smithi115.stdout:3/52: rename l4 to ld 0 2021-11-03T17:13:40.733 INFO:tasks.workunit.client.1.smithi115.stdout:5/92: creat d1a/f20 x:0 0 0 2021-11-03T17:13:40.733 INFO:tasks.workunit.client.1.smithi115.stdout:5/93: getdents d1a/d1b 0 2021-11-03T17:13:40.733 INFO:tasks.workunit.client.1.smithi115.stdout:5/94: mknod d1a/c21 1 2021-11-03T17:13:40.734 INFO:tasks.workunit.client.1.smithi115.stdout:5/95: write f0 [86758,54088] 0 2021-11-03T17:13:40.740 INFO:tasks.workunit.client.1.smithi115.stdout:8/71: unlink d4/l9 0 2021-11-03T17:13:40.741 INFO:tasks.workunit.client.1.smithi115.stdout:8/72: creat d4/d6/f1d x:0 0 0 2021-11-03T17:13:40.741 INFO:tasks.workunit.client.1.smithi115.stdout:8/73: write d4/d6/f1d [843514,106171] 0 2021-11-03T17:13:40.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:40.753+0000 7f52fe46c700 1 -- 172.21.15.27:0/535509803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52f80f4bc0 msgr2=0x7f52f80f4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:13:40.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:40.753+0000 7f52fe46c700 1 --2- 172.21.15.27:0/535509803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52f80f4bc0 0x7f52f80f4fe0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f52f4004660 tx=0x7f52f4009b30).stop 2021-11-03T17:13:40.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:40.753+0000 7f52fe46c700 1 -- 172.21.15.27:0/535509803 shutdown_connections 2021-11-03T17:13:40.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:40.753+0000 7f52fe46c700 1 --2- 172.21.15.27:0/535509803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52f80f5e60 0x7f52f80f62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:40.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:40.753+0000 7f52fe46c700 1 --2- 172.21.15.27:0/535509803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52f80f4bc0 0x7f52f80f4fe0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:40.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:40.753+0000 7f52fe46c700 1 -- 172.21.15.27:0/535509803 >> 172.21.15.27:0/535509803 conn(0x7f52f80f0320 msgr2=0x7f52f80f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:13:40.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:40.753+0000 7f52fe46c700 1 -- 172.21.15.27:0/535509803 shutdown_connections 2021-11-03T17:13:40.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:40.753+0000 7f52fe46c700 1 -- 172.21.15.27:0/535509803 wait complete. 2021-11-03T17:13:40.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:40.754+0000 7f52fe46c700 1 Processor -- start 2021-11-03T17:13:40.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:40.754+0000 7f52fe46c700 1 -- start start 2021-11-03T17:13:40.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:40.755+0000 7f52fe46c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52f80f5e60 0x7f52f8110df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:13:40.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:40.755+0000 7f52fe46c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52f81112e0 0x7f52f81136f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:13:40.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:40.755+0000 7f52fe46c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f8113cf0 con 0x7f52f81112e0 2021-11-03T17:13:40.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:40.755+0000 7f52fe46c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f8113e60 con 0x7f52f80f5e60 2021-11-03T17:13:40.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:40.755+0000 7f52fd46a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52f80f5e60 0x7f52f8110df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:13:40.755 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:40.755+0000 7f52fd46a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52f80f5e60 0x7f52f8110df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.27:46434/0 (socket says 172.21.15.27:46434) 2021-11-03T17:13:40.755 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:40.755+0000 7f52fd46a700 1 -- 172.21.15.27:0/3031268914 learned_addr learned my addr 172.21.15.27:0/3031268914 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:13:40.755 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:40.755+0000 7f52fcc69700 1 --2- 172.21.15.27:0/3031268914 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52f81112e0 0x7f52f81136f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:13:40.755 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:40.755+0000 7f52fd46a700 1 -- 172.21.15.27:0/3031268914 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52f81112e0 msgr2=0x7f52f81136f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:13:40.756 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:40.755+0000 7f52fd46a700 1 --2- 172.21.15.27:0/3031268914 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52f81112e0 0x7f52f81136f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:40.756 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:40.755+0000 7f52fd46a700 1 -- 172.21.15.27:0/3031268914 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f4005040 con 0x7f52f80f5e60 2021-11-03T17:13:40.756 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:40.756+0000 7f52fd46a700 1 --2- 172.21.15.27:0/3031268914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52f80f5e60 0x7f52f8110df0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f52f4011000 tx=0x7f52f4009b30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:13:40.756 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:40.756+0000 7f52edffb700 1 -- 172.21.15.27:0/3031268914 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52f4013380 con 0x7f52f80f5e60 2021-11-03T17:13:40.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:40.756+0000 7f52edffb700 1 -- 172.21.15.27:0/3031268914 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f52f4007e70 con 0x7f52f80f5e60 2021-11-03T17:13:40.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:40.756+0000 7f52edffb700 1 -- 172.21.15.27:0/3031268914 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52f401ed40 con 0x7f52f80f5e60 2021-11-03T17:13:40.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:40.757+0000 7f52fe46c700 1 -- 172.21.15.27:0/3031268914 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52f8068990 con 0x7f52f80f5e60 2021-11-03T17:13:40.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:40.757+0000 7f52fe46c700 1 -- 172.21.15.27:0/3031268914 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52f8068ec0 con 0x7f52f80f5e60 2021-11-03T17:13:40.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:40.759+0000 7f52edffb700 1 -- 172.21.15.27:0/3031268914 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f52f401e440 con 0x7f52f80f5e60 2021-11-03T17:13:40.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:40.759+0000 7f52fe46c700 1 -- 172.21.15.27:0/3031268914 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52f8069900 con 0x7f52f80f5e60 2021-11-03T17:13:40.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:40.760+0000 7f52edffb700 1 --2- 172.21.15.27:0/3031268914 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f52e405e500 0x7f52e40609c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:13:40.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:40.760+0000 7f52edffb700 1 -- 172.21.15.27:0/3031268914 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f52f4077530 con 0x7f52f80f5e60 2021-11-03T17:13:40.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:40.760+0000 7f52fcc69700 1 --2- 172.21.15.27:0/3031268914 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f52e405e500 0x7f52e40609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:13:40.759 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:40.760+0000 7f52fcc69700 1 --2- 172.21.15.27:0/3031268914 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f52e405e500 0x7f52e40609c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f52f805eff0 tx=0x7f52e800b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:13:40.759 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:40.763+0000 7f52edffb700 1 -- 172.21.15.27:0/3031268914 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f52f8069900 con 0x7f52f80f5e60 2021-11-03T17:13:40.762 INFO:tasks.workunit.client.1.smithi115.stdout:8/74: dwrite d4/f11 [4194304,4194304] 0 2021-11-03T17:13:40.764 INFO:tasks.workunit.client.1.smithi115.stdout:8/75: creat d4/f1e x:0 0 0 2021-11-03T17:13:40.765 INFO:tasks.workunit.client.1.smithi115.stdout:8/76: creat d4/f1f x:0 0 0 2021-11-03T17:13:40.765 INFO:tasks.workunit.client.1.smithi115.stdout:8/77: readlink d4/l10 0 2021-11-03T17:13:40.766 INFO:tasks.workunit.client.1.smithi115.stdout:8/78: chown d4/fa 76194 1 2021-11-03T17:13:40.768 INFO:tasks.workunit.client.1.smithi115.stdout:8/79: dread d4/f14 [0,4194304] 0 2021-11-03T17:13:40.768 INFO:tasks.workunit.client.1.smithi115.stdout:8/80: mknod d4/d6/d18/c20 1 2021-11-03T17:13:40.785 INFO:tasks.workunit.client.1.smithi115.stdout:8/81: dwrite d4/f1e [0,4194304] 0 2021-11-03T17:13:40.787 INFO:tasks.workunit.client.1.smithi115.stdout:8/82: getdents d4/d6 0 2021-11-03T17:13:40.809 INFO:tasks.workunit.client.1.smithi115.stdout:8/83: dwrite d4/f11 [4194304,4194304] 0 2021-11-03T17:13:40.821 INFO:tasks.workunit.client.1.smithi115.stdout:8/84: dread d4/f11 [0,4194304] 0 2021-11-03T17:13:40.841 INFO:tasks.workunit.client.1.smithi115.stdout:8/85: dwrite d4/f1f [0,4194304] 0 2021-11-03T17:13:40.842 INFO:tasks.workunit.client.1.smithi115.stdout:8/86: write d4/f1f [3841513,74901] 0 2021-11-03T17:13:40.843 INFO:tasks.workunit.client.1.smithi115.stdout:8/87: mknod d4/d6/d18/c21 1 2021-11-03T17:13:40.843 INFO:tasks.workunit.client.1.smithi115.stdout:8/88: mknod d4/d6/d18/c22 1 2021-11-03T17:13:40.844 INFO:tasks.workunit.client.1.smithi115.stdout:8/89: write d4/d6/f1d [969960,14389] 0 2021-11-03T17:13:40.851 INFO:tasks.workunit.client.1.smithi115.stdout:8/90: dread d4/f1f [0,4194304] 0 2021-11-03T17:13:40.852 INFO:tasks.workunit.client.1.smithi115.stdout:8/91: write d4/f12 [759218,124686] 0 2021-11-03T17:13:40.859 INFO:tasks.workunit.client.1.smithi115.stdout:8/92: dread d4/f12 [4194304,4194304] 0 2021-11-03T17:13:40.862 INFO:tasks.workunit.client.1.smithi115.stdout:8/93: dread d4/f14 [0,4194304] 0 2021-11-03T17:13:40.863 INFO:tasks.workunit.client.1.smithi115.stdout:8/94: write d4/f1f [2362883,106473] 0 2021-11-03T17:13:40.863 INFO:tasks.workunit.client.1.smithi115.stdout:8/95: chown d4/d6 3 1 2021-11-03T17:13:40.864 INFO:tasks.workunit.client.1.smithi115.stdout:8/96: rename d4/f11 to d4/d6/d18/f23 0 2021-11-03T17:13:40.865 INFO:tasks.workunit.client.1.smithi115.stdout:8/97: chown d4/l7 215 1 2021-11-03T17:13:40.881 INFO:tasks.workunit.client.1.smithi115.stdout:8/98: dwrite d4/f14 [0,4194304] 0 2021-11-03T17:13:40.883 INFO:tasks.workunit.client.1.smithi115.stdout:8/99: truncate d4/d6/d18/f23 6005243 0 2021-11-03T17:13:40.884 INFO:tasks.workunit.client.1.smithi115.stdout:8/100: symlink d4/d6/l24 0 2021-11-03T17:13:40.894 INFO:tasks.workunit.client.1.smithi115.stdout:8/101: dread d4/f1e [0,4194304] 0 2021-11-03T17:13:40.900 INFO:tasks.workunit.client.1.smithi115.stdout:8/102: dread d4/f1f [0,4194304] 0 2021-11-03T17:13:40.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:40 smithi115 conmon[34670]: cluster 2021-11-03T17:13:40.598338+0000 mgr.smithi027.ufhhvb (mgr.14162) 285 : cluster [DBG] pgmap v237: 65 pgs: 65 active+clean; 233 MiB data, 885 MiB used, 536 GiB / 536 GiB avail; 5.1 MiB/s rd, 18 MiB/s wr, 565 op/s 2021-11-03T17:13:40.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.003+0000 7f52fe46c700 1 -- 172.21.15.27:0/3031268914 --> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f52f8062ba0 con 0x7f52e405e500 2021-11-03T17:13:41.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.004+0000 7f52edffb700 1 -- 172.21.15.27:0/3031268914 <== mgr.14162 v2:172.21.15.27:6800/3802716887 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+372 (secure 0 0 0) 0x7f52f8062ba0 con 0x7f52e405e500 2021-11-03T17:13:41.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.006+0000 7f52fe46c700 1 -- 172.21.15.27:0/3031268914 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f52e405e500 msgr2=0x7f52e40609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:13:41.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.006+0000 7f52fe46c700 1 --2- 172.21.15.27:0/3031268914 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f52e405e500 0x7f52e40609c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f52f805eff0 tx=0x7f52e800b040).stop 2021-11-03T17:13:41.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.006+0000 7f52fe46c700 1 -- 172.21.15.27:0/3031268914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52f80f5e60 msgr2=0x7f52f8110df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:13:41.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.006+0000 7f52fe46c700 1 --2- 172.21.15.27:0/3031268914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52f80f5e60 0x7f52f8110df0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f52f4011000 tx=0x7f52f4009b30).stop 2021-11-03T17:13:41.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.006+0000 7f52fe46c700 1 -- 172.21.15.27:0/3031268914 shutdown_connections 2021-11-03T17:13:41.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.006+0000 7f52fe46c700 1 --2- 172.21.15.27:0/3031268914 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f52e405e500 0x7f52e40609c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:41.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.006+0000 7f52fe46c700 1 --2- 172.21.15.27:0/3031268914 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52f80f5e60 0x7f52f8110df0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:41.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.006+0000 7f52fe46c700 1 --2- 172.21.15.27:0/3031268914 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52f81112e0 0x7f52f81136f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:41.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.006+0000 7f52fe46c700 1 -- 172.21.15.27:0/3031268914 >> 172.21.15.27:0/3031268914 conn(0x7f52f80f0320 msgr2=0x7f52f80f98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:13:41.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.006+0000 7f52fe46c700 1 -- 172.21.15.27:0/3031268914 shutdown_connections 2021-11-03T17:13:41.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.006+0000 7f52fe46c700 1 -- 172.21.15.27:0/3031268914 wait complete. 2021-11-03T17:13:41.012 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:13:41.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:40 smithi027 conmon[32322]: cluster 2021-11-03T17:13: 2021-11-03T17:13:41.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:40 smithi027 conmon[32322]: 40.598338+0000 mgr.smithi027.ufhhvb (mgr.14162) 285 : cluster [DBG] pgmap v237: 65 pgs: 65 active+clean; 233 MiB data, 885 MiB used, 536 GiB / 536 GiB avail; 5.1 MiB/s rd, 18 MiB/s wr, 565 op/s 2021-11-03T17:13:41.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.111+0000 7f6995fb7700 1 -- 172.21.15.27:0/2173267120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69900f66e0 msgr2=0x7f69900f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:13:41.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.111+0000 7f6995fb7700 1 --2- 172.21.15.27:0/2173267120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69900f66e0 0x7f69900f6b00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f6980004660 tx=0x7f6980009b30).stop 2021-11-03T17:13:41.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.111+0000 7f6995fb7700 1 -- 172.21.15.27:0/2173267120 shutdown_connections 2021-11-03T17:13:41.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.111+0000 7f6995fb7700 1 --2- 172.21.15.27:0/2173267120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69900f7980 0x7f69900f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:41.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.111+0000 7f6995fb7700 1 --2- 172.21.15.27:0/2173267120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69900f66e0 0x7f69900f6b00 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:41.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.111+0000 7f6995fb7700 1 -- 172.21.15.27:0/2173267120 >> 172.21.15.27:0/2173267120 conn(0x7f69900f1e40 msgr2=0x7f69900f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:13:41.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.111+0000 7f6995fb7700 1 -- 172.21.15.27:0/2173267120 shutdown_connections 2021-11-03T17:13:41.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.111+0000 7f6995fb7700 1 -- 172.21.15.27:0/2173267120 wait complete. 2021-11-03T17:13:41.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.111+0000 7f6995fb7700 1 Processor -- start 2021-11-03T17:13:41.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.111+0000 7f6995fb7700 1 -- start start 2021-11-03T17:13:41.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.112+0000 7f6995fb7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69900f66e0 0x7f699006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:13:41.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.112+0000 7f6995fb7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69900f7980 0x7f6990069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:13:41.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.112+0000 7f6995fb7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699006a380 con 0x7f69900f66e0 2021-11-03T17:13:41.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.112+0000 7f6995fb7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f699006a4c0 con 0x7f69900f7980 2021-11-03T17:13:41.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.112+0000 7f698f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69900f66e0 0x7f699006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:13:41.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.112+0000 7f698f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69900f66e0 0x7f699006b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55278/0 (socket says 172.21.15.27:55278) 2021-11-03T17:13:41.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.112+0000 7f698f7fe700 1 -- 172.21.15.27:0/2624997977 learned_addr learned my addr 172.21.15.27:0/2624997977 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:13:41.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.112+0000 7f698effd700 1 --2- 172.21.15.27:0/2624997977 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69900f7980 0x7f6990069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:13:41.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.112+0000 7f698f7fe700 1 -- 172.21.15.27:0/2624997977 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69900f7980 msgr2=0x7f6990069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:13:41.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.112+0000 7f698f7fe700 1 --2- 172.21.15.27:0/2624997977 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69900f7980 0x7f6990069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:41.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.112+0000 7f698f7fe700 1 -- 172.21.15.27:0/2624997977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6980005040 con 0x7f69900f66e0 2021-11-03T17:13:41.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.112+0000 7f698effd700 1 --2- 172.21.15.27:0/2624997977 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69900f7980 0x7f6990069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:13:41.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.112+0000 7f698f7fe700 1 --2- 172.21.15.27:0/2624997977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69900f66e0 0x7f699006b9a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f6980014040 tx=0x7f698000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:13:41.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.112+0000 7f6977fff700 1 -- 172.21.15.27:0/2624997977 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6980019070 con 0x7f69900f66e0 2021-11-03T17:13:41.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.113+0000 7f6995fb7700 1 -- 172.21.15.27:0/2624997977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f699006a740 con 0x7f69900f66e0 2021-11-03T17:13:41.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.113+0000 7f6977fff700 1 -- 172.21.15.27:0/2624997977 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f6980007e70 con 0x7f69900f66e0 2021-11-03T17:13:41.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.113+0000 7f6977fff700 1 -- 172.21.15.27:0/2624997977 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6980014d50 con 0x7f69900f66e0 2021-11-03T17:13:41.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.113+0000 7f6995fb7700 1 -- 172.21.15.27:0/2624997977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f699006aca0 con 0x7f69900f66e0 2021-11-03T17:13:41.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.114+0000 7f6995fb7700 1 -- 172.21.15.27:0/2624997977 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6990045ad0 con 0x7f69900f66e0 2021-11-03T17:13:41.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.114+0000 7f6977fff700 1 -- 172.21.15.27:0/2624997977 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f6980025070 con 0x7f69900f66e0 2021-11-03T17:13:41.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.114+0000 7f6977fff700 1 --2- 172.21.15.27:0/2624997977 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f697805c2b0 0x7f697805e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:13:41.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.114+0000 7f6977fff700 1 -- 172.21.15.27:0/2624997977 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f6980076f60 con 0x7f69900f66e0 2021-11-03T17:13:41.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.115+0000 7f698effd700 1 --2- 172.21.15.27:0/2624997977 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f697805c2b0 0x7f697805e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:13:41.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.115+0000 7f698effd700 1 --2- 172.21.15.27:0/2624997977 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f697805c2b0 0x7f697805e770 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f6984009170 tx=0x7f698400b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:13:41.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.119+0000 7f6977fff700 1 -- 172.21.15.27:0/2624997977 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f69800a32b0 con 0x7f69900f66e0 2021-11-03T17:13:41.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.311+0000 7f6995fb7700 1 -- 172.21.15.27:0/2624997977 --> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f69900fefe0 con 0x7f697805c2b0 2021-11-03T17:13:41.319 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.323+0000 7f6977fff700 1 -- 172.21.15.27:0/2624997977 <== mgr.14162 v2:172.21.15.27:6800/3802716887 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2728 (secure 0 0 0) 0x7f69900fefe0 con 0x7f697805c2b0 2021-11-03T17:13:41.319 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE VERSION IMAGE ID CONTAINER ID 2021-11-03T17:13:41.319 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (5m) 83s ago 6m 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:13:41.320 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (6m) 83s ago 6m 16.2.4 8d91d370c2b8 d494472a47c0 2021-11-03T17:13:41.320 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (5m) 86s ago 5m 16.2.4 8d91d370c2b8 f84fd4042530 2021-11-03T17:13:41.320 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (5m) 83s ago 6m 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:13:41.320 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (96s) 83s ago 95s 16.2.4 8d91d370c2b8 509a42d9cc82 2021-11-03T17:13:41.321 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (102s) 83s ago 102s 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:13:41.321 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (100s) 86s ago 99s 16.2.4 8d91d370c2b8 bfd990d6415e 2021-11-03T17:13:41.321 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (106s) 86s ago 105s 16.2.4 8d91d370c2b8 f8e6ab2a5ed4 2021-11-03T17:13:41.321 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:9283 running (7m) 83s ago 7m 16.2.4 8d91d370c2b8 999298a5769c 2021-11-03T17:13:41.322 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8443,9283 running (5m) 86s ago 5m 16.2.4 8d91d370c2b8 722d63932db9 2021-11-03T17:13:41.322 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (7m) 83s ago 7m 16.2.4 8d91d370c2b8 060d5938214f 2021-11-03T17:13:41.322 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (5m) 86s ago 5m 16.2.4 8d91d370c2b8 319323f09834 2021-11-03T17:13:41.322 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (6m) 83s ago 6m 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:13:41.323 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (5m) 86s ago 5m 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:13:41.323 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (4m) 83s ago 4m 16.2.4 8d91d370c2b8 17f79a815a0f 2021-11-03T17:13:41.323 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (4m) 83s ago 4m 16.2.4 8d91d370c2b8 422be0cac0db 2021-11-03T17:13:41.323 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (4m) 83s ago 4m 16.2.4 8d91d370c2b8 c211ef5551c9 2021-11-03T17:13:41.324 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (3m) 86s ago 3m 16.2.4 8d91d370c2b8 b1486ec1b619 2021-11-03T17:13:41.324 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (3m) 86s ago 3m 16.2.4 8d91d370c2b8 8b91230c69e5 2021-11-03T17:13:41.324 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (3m) 86s ago 3m 16.2.4 8d91d370c2b8 4a54c4d6b6f5 2021-11-03T17:13:41.324 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (4m) 83s ago 5m 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:13:41.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.326+0000 7f6995fb7700 1 -- 172.21.15.27:0/2624997977 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f697805c2b0 msgr2=0x7f697805e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:13:41.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.326+0000 7f6995fb7700 1 --2- 172.21.15.27:0/2624997977 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f697805c2b0 0x7f697805e770 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f6984009170 tx=0x7f698400b040).stop 2021-11-03T17:13:41.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.326+0000 7f6995fb7700 1 -- 172.21.15.27:0/2624997977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69900f66e0 msgr2=0x7f699006b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:13:41.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.326+0000 7f6995fb7700 1 --2- 172.21.15.27:0/2624997977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69900f66e0 0x7f699006b9a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f6980014040 tx=0x7f698000b040).stop 2021-11-03T17:13:41.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.326+0000 7f6995fb7700 1 -- 172.21.15.27:0/2624997977 shutdown_connections 2021-11-03T17:13:41.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.326+0000 7f6995fb7700 1 --2- 172.21.15.27:0/2624997977 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f697805c2b0 0x7f697805e770 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:41.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.326+0000 7f6995fb7700 1 --2- 172.21.15.27:0/2624997977 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69900f66e0 0x7f699006b9a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:41.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.326+0000 7f6995fb7700 1 --2- 172.21.15.27:0/2624997977 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69900f7980 0x7f6990069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:41.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.326+0000 7f6995fb7700 1 -- 172.21.15.27:0/2624997977 >> 172.21.15.27:0/2624997977 conn(0x7f69900f1e40 msgr2=0x7f69900fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:13:41.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.326+0000 7f6995fb7700 1 -- 172.21.15.27:0/2624997977 shutdown_connections 2021-11-03T17:13:41.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.326+0000 7f6995fb7700 1 -- 172.21.15.27:0/2624997977 wait complete. 2021-11-03T17:13:41.433 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.435+0000 7f00aa7db700 1 -- 172.21.15.27:0/3361269274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00a40f66d0 msgr2=0x7f00a40f6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:13:41.433 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.435+0000 7f00aa7db700 1 --2- 172.21.15.27:0/3361269274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00a40f66d0 0x7f00a40f6b30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f0094004660 tx=0x7f0094009b30).stop 2021-11-03T17:13:41.433 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.436+0000 7f00aa7db700 1 -- 172.21.15.27:0/3361269274 shutdown_connections 2021-11-03T17:13:41.434 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.436+0000 7f00aa7db700 1 --2- 172.21.15.27:0/3361269274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00a40f66d0 0x7f00a40f6b30 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:41.434 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.436+0000 7f00aa7db700 1 --2- 172.21.15.27:0/3361269274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00a40f5430 0x7f00a40f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:41.434 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.436+0000 7f00aa7db700 1 -- 172.21.15.27:0/3361269274 >> 172.21.15.27:0/3361269274 conn(0x7f00a40f0b90 msgr2=0x7f00a40f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:13:41.435 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.436+0000 7f00aa7db700 1 -- 172.21.15.27:0/3361269274 shutdown_connections 2021-11-03T17:13:41.435 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.436+0000 7f00aa7db700 1 -- 172.21.15.27:0/3361269274 wait complete. 2021-11-03T17:13:41.435 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.436+0000 7f00aa7db700 1 Processor -- start 2021-11-03T17:13:41.435 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.436+0000 7f00aa7db700 1 -- start start 2021-11-03T17:13:41.436 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.437+0000 7f00aa7db700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00a40f5430 0x7f00a41099d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:13:41.436 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.437+0000 7f00aa7db700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00a40f66d0 0x7f00a410bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:13:41.436 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.437+0000 7f00aa7db700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a40603e0 con 0x7f00a40f5430 2021-11-03T17:13:41.436 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.437+0000 7f00aa7db700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a4060520 con 0x7f00a40f66d0 2021-11-03T17:13:41.436 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.437+0000 7f00a97d9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00a40f5430 0x7f00a41099d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:13:41.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.437+0000 7f00a97d9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00a40f5430 0x7f00a41099d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55288/0 (socket says 172.21.15.27:55288) 2021-11-03T17:13:41.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.437+0000 7f00a97d9700 1 -- 172.21.15.27:0/3742235370 learned_addr learned my addr 172.21.15.27:0/3742235370 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:13:41.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.437+0000 7f00a8fd8700 1 --2- 172.21.15.27:0/3742235370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00a40f66d0 0x7f00a410bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:13:41.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.437+0000 7f00a97d9700 1 -- 172.21.15.27:0/3742235370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00a40f66d0 msgr2=0x7f00a410bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:13:41.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.437+0000 7f00a97d9700 1 --2- 172.21.15.27:0/3742235370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00a40f66d0 0x7f00a410bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:41.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.437+0000 7f00a97d9700 1 -- 172.21.15.27:0/3742235370 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0094005040 con 0x7f00a40f5430 2021-11-03T17:13:41.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.437+0000 7f00a97d9700 1 --2- 172.21.15.27:0/3742235370 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00a40f5430 0x7f00a41099d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f00a000e7c0 tx=0x7f00a000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:13:41.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.437+0000 7f0099ffb700 1 -- 172.21.15.27:0/3742235370 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00a00092e0 con 0x7f00a40f5430 2021-11-03T17:13:41.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.437+0000 7f0099ffb700 1 -- 172.21.15.27:0/3742235370 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f00a000f040 con 0x7f00a40f5430 2021-11-03T17:13:41.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.437+0000 7f00aa7db700 1 -- 172.21.15.27:0/3742235370 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00a410c480 con 0x7f00a40f5430 2021-11-03T17:13:41.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.437+0000 7f0099ffb700 1 -- 172.21.15.27:0/3742235370 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00a00147e0 con 0x7f00a40f5430 2021-11-03T17:13:41.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.438+0000 7f00aa7db700 1 -- 172.21.15.27:0/3742235370 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00a410ca40 con 0x7f00a40f5430 2021-11-03T17:13:41.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.439+0000 7f00aa7db700 1 -- 172.21.15.27:0/3742235370 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00a4045ad0 con 0x7f00a40f5430 2021-11-03T17:13:41.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.439+0000 7f0099ffb700 1 -- 172.21.15.27:0/3742235370 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f00a001b030 con 0x7f00a40f5430 2021-11-03T17:13:41.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.439+0000 7f0099ffb700 1 --2- 172.21.15.27:0/3742235370 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f009005c300 0x7f009005e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:13:41.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.439+0000 7f0099ffb700 1 -- 172.21.15.27:0/3742235370 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f00a0075d80 con 0x7f00a40f5430 2021-11-03T17:13:41.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.439+0000 7f00a8fd8700 1 --2- 172.21.15.27:0/3742235370 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f009005c300 0x7f009005e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:13:41.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.440+0000 7f00a8fd8700 1 --2- 172.21.15.27:0/3742235370 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f009005c300 0x7f009005e7c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f0094005010 tx=0x7f0094016040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:13:41.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.443+0000 7f0099ffb700 1 -- 172.21.15.27:0/3742235370 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f00a0048a60 con 0x7f00a40f5430 2021-11-03T17:13:41.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.707+0000 7f00aa7db700 1 -- 172.21.15.27:0/3742235370 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f00a4027a00 con 0x7f00a40f5430 2021-11-03T17:13:41.704 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.708+0000 7f0099ffb700 1 -- 172.21.15.27:0/3742235370 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+573 (secure 0 0 0) 0x7f00a0077070 con 0x7f00a40f5430 2021-11-03T17:13:41.704 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:13:41.704 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:13:41.705 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 2 2021-11-03T17:13:41.705 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:13:41.705 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:13:41.705 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 2 2021-11-03T17:13:41.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:13:41.706 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:13:41.706 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 6 2021-11-03T17:13:41.706 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:13:41.707 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:13:41.707 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:13:41.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:13:41.707 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:13:41.708 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 14 2021-11-03T17:13:41.708 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:13:41.708 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:13:41.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.709+0000 7f00aa7db700 1 -- 172.21.15.27:0/3742235370 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f009005c300 msgr2=0x7f009005e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:13:41.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.709+0000 7f00aa7db700 1 --2- 172.21.15.27:0/3742235370 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f009005c300 0x7f009005e7c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f0094005010 tx=0x7f0094016040).stop 2021-11-03T17:13:41.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.709+0000 7f00aa7db700 1 -- 172.21.15.27:0/3742235370 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00a40f5430 msgr2=0x7f00a41099d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:13:41.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.709+0000 7f00aa7db700 1 --2- 172.21.15.27:0/3742235370 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00a40f5430 0x7f00a41099d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f00a000e7c0 tx=0x7f00a000b040).stop 2021-11-03T17:13:41.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.709+0000 7f00aa7db700 1 -- 172.21.15.27:0/3742235370 shutdown_connections 2021-11-03T17:13:41.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.709+0000 7f00aa7db700 1 --2- 172.21.15.27:0/3742235370 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f009005c300 0x7f009005e7c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:41.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.709+0000 7f00aa7db700 1 --2- 172.21.15.27:0/3742235370 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00a40f5430 0x7f00a41099d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:41.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.709+0000 7f00aa7db700 1 --2- 172.21.15.27:0/3742235370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00a40f66d0 0x7f00a410bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:41.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.709+0000 7f00aa7db700 1 -- 172.21.15.27:0/3742235370 >> 172.21.15.27:0/3742235370 conn(0x7f00a40f0b90 msgr2=0x7f00a40fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:13:41.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.710+0000 7f00aa7db700 1 -- 172.21.15.27:0/3742235370 shutdown_connections 2021-11-03T17:13:41.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.710+0000 7f00aa7db700 1 -- 172.21.15.27:0/3742235370 wait complete. 2021-11-03T17:13:41.823 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.827+0000 7f1273dfc700 1 -- 172.21.15.27:0/2112226433 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f126c100fd0 msgr2=0x7f126c1033c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:13:41.823 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.827+0000 7f1273dfc700 1 --2- 172.21.15.27:0/2112226433 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f126c100fd0 0x7f126c1033c0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f125c004660 tx=0x7f125c009b30).stop 2021-11-03T17:13:41.824 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.827+0000 7f1273dfc700 1 -- 172.21.15.27:0/2112226433 shutdown_connections 2021-11-03T17:13:41.824 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.827+0000 7f1273dfc700 1 --2- 172.21.15.27:0/2112226433 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f126c100fd0 0x7f126c1033c0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:41.824 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.827+0000 7f1273dfc700 1 --2- 172.21.15.27:0/2112226433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f126c0681d0 0x7f126c100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:41.824 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.827+0000 7f1273dfc700 1 -- 172.21.15.27:0/2112226433 >> 172.21.15.27:0/2112226433 conn(0x7f126c0636c0 msgr2=0x7f126c065ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:13:41.825 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.827+0000 7f1273dfc700 1 -- 172.21.15.27:0/2112226433 shutdown_connections 2021-11-03T17:13:41.825 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.827+0000 7f1273dfc700 1 -- 172.21.15.27:0/2112226433 wait complete. 2021-11-03T17:13:41.826 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.828+0000 7f1273dfc700 1 Processor -- start 2021-11-03T17:13:41.827 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.828+0000 7f1273dfc700 1 -- start start 2021-11-03T17:13:41.827 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.828+0000 7f1273dfc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f126c0681d0 0x7f126c1118b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:13:41.827 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.828+0000 7f1273dfc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f126c100fd0 0x7f126c111da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:13:41.827 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.828+0000 7f1273dfc700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126c112350 con 0x7f126c0681d0 2021-11-03T17:13:41.828 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.828+0000 7f1273dfc700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126c112490 con 0x7f126c100fd0 2021-11-03T17:13:41.828 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.828+0000 7f1271b98700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f126c0681d0 0x7f126c1118b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:13:41.828 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.828+0000 7f1271b98700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f126c0681d0 0x7f126c1118b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55298/0 (socket says 172.21.15.27:55298) 2021-11-03T17:13:41.828 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.828+0000 7f1271b98700 1 -- 172.21.15.27:0/2408941787 learned_addr learned my addr 172.21.15.27:0/2408941787 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:13:41.828 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.829+0000 7f1271b98700 1 -- 172.21.15.27:0/2408941787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f126c100fd0 msgr2=0x7f126c111da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:13:41.829 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.829+0000 7f1271b98700 1 --2- 172.21.15.27:0/2408941787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f126c100fd0 0x7f126c111da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:13:41.829 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.829+0000 7f1271b98700 1 -- 172.21.15.27:0/2408941787 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f125c005040 con 0x7f126c0681d0 2021-11-03T17:13:41.829 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.830+0000 7f1271b98700 1 --2- 172.21.15.27:0/2408941787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f126c0681d0 0x7f126c1118b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f126800e7f0 tx=0x7f126800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:13:41.829 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.830+0000 7f12627fc700 1 -- 172.21.15.27:0/2408941787 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12680092e0 con 0x7f126c0681d0 2021-11-03T17:13:41.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.830+0000 7f1273dfc700 1 -- 172.21.15.27:0/2408941787 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f126c112e30 con 0x7f126c0681d0 2021-11-03T17:13:41.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.830+0000 7f1273dfc700 1 -- 172.21.15.27:0/2408941787 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f126c1133f0 con 0x7f126c0681d0 2021-11-03T17:13:41.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.830+0000 7f12627fc700 1 -- 172.21.15.27:0/2408941787 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f126800f040 con 0x7f126c0681d0 2021-11-03T17:13:41.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.830+0000 7f12627fc700 1 -- 172.21.15.27:0/2408941787 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1268014750 con 0x7f126c0681d0 2021-11-03T17:13:41.831 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.831+0000 7f1273dfc700 1 -- 172.21.15.27:0/2408941787 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1254004fa0 con 0x7f126c0681d0 2021-11-03T17:13:41.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.833+0000 7f12627fc700 1 -- 172.21.15.27:0/2408941787 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f126801b030 con 0x7f126c0681d0 2021-11-03T17:13:41.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.833+0000 7f12627fc700 1 --2- 172.21.15.27:0/2408941787 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f125805c2a0 0x7f125805e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:13:41.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.833+0000 7f12627fc700 1 -- 172.21.15.27:0/2408941787 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f1268075c90 con 0x7f126c0681d0 2021-11-03T17:13:41.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.833+0000 7f1271397700 1 --2- 172.21.15.27:0/2408941787 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f125805c2a0 0x7f125805e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:13:41.833 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.834+0000 7f1271397700 1 --2- 172.21.15.27:0/2408941787 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f125805c2a0 0x7f125805e760 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f125c000c00 tx=0x7f125c00b040).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:13:41.833 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:41.838+0000 7f12627fc700 1 -- 172.21.15.27:0/2408941787 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f1268048a00 con 0x7f126c0681d0 2021-11-03T17:13:41.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:41 smithi115 conmon[34670]: audit 2021-11-03T17:13:41.005403+0000 mgr.smithi027.ufhhvb (mgr 2021-11-03T17:13:41.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:41 smithi115 conmon[34670]: .14162) 286 : audit [DBG] from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:13:41.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:41 smithi115 conmon[34670]: audit 2021-11-03T17:13:41.313545+0000 mgr.smithi027.ufhhvb (mgr.14162) 287 : audit [DBG] from='client.14544 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:13:42.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:41 smithi027 conmon[32322]: audit 2021-11-03T17:13:41.005403+0000 mgr.smithi027.ufhhvb (mgr.14162) 286 2021-11-03T17:13:42.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:41 smithi027 conmon[32322]: : audit [DBG] from='client.24371 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:13:42.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:41 smithi027 conmon[32322]: audit 2021-11-03T17:13:41.313545+0000 mgr.smithi027.ufhhvb (mgr.14162) 287 : audit [DBG] from='client.14544 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:13:42.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:42.041+0000 7f1273dfc700 1 -- 172.21.15.27:0/2408941787 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1254005940 con 0x7f126c0681d0 2021-11-03T17:13:42.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:42.041+0000 7f12627fc700 1 -- 172.21.15.27:0/2408941787 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 14 v14) v1 ==== 76+0+1645 (secure 0 0 0) 0x7f1268019070 con 0x7f126c0681d0 2021-11-03T17:13:42.037 INFO:teuthology.orchestra.run.smithi027.stdout:e14 2021-11-03T17:13:42.037 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:13:42.038 INFO:teuthology.orchestra.run.smithi027.stdout:compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:13:42.038 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:13:42.038 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:13:42.039 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:13:42.039 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:13:42.039 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 14 2021-11-03T17:13:42.039 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 2021-11-03T17:13:42.040 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:13:42.040 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:13:15.679383+0000 2021-11-03T17:13:42.040 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:13:42.040 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:13:42.040 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:13:42.041 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:13:42.041 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:13:42.041 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:13:42.041 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:13:42.041 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 0 2021-11-03T17:13:42.042 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:13:42.042 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 2 2021-11-03T17:13:42.042 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:13:42.042 INFO:teuthology.orchestra.run.smithi027.stdout:up {0=24273,1=14436} 2021-11-03T17:13:42.042 INFO:teuthology.orchestra.run.smithi027.stdout:failed 2021-11-03T17:13:42.043 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:13:42.043 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:13:42.043 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:13:42.043 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:13:42.043 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:13:42.044 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:13:42.044 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:13:42.044 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{0:24273} state up:active seq 3 export targets 1 join_fscid=1 addr [v2:172.21.15.115:6824/1621245036,v1:172.21.15.115:6825/1621245036]] 2021-11-03T17:13:42.044 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 3 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060]] 2021-11-03T17:13:42.045 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:13:42.045 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:13:42.045 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:13:42.045 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:13:42.046 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14448} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6828/1337044739,v1:172.21.15.27:6829/1337044739]] 2021-11-03T17:13:42.046 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:24289} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.115:6826/3367380088,v1:172.21.15.115:6827/3367380088]] 2021-11-03T17:13:42.047 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:42.043+0000 7f1273dfc700 1 -- 172.21.15.27:0/2408941787 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f125805c2a0 msgr2=0x7f125805e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:13:42.047 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:42.043+0000 7f1273dfc700 1 --2- 172.21.15.27:0/2408941787 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f125805c2a0 0x7f125805e760 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f125c000c00 tx=0x7f125c00b040).stop 2021-11-03T17:13:42.047 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:42.043+0000 7f1273dfc700 1 -- 172.21.15.27:0/2408941787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f126c0681d0 msgr2=0x7f126c1118b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:13:42.048 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:42.043+0000 7f1273dfc700 1 --2- 172.21.15.27:0/2408941787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f126c0681d0 0x7f126c1118b0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f126800e7f0 tx=0x7f126800b040).stop 2021-11-03T17:13:42.048 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:42.043+0000 7f1273dfc700 1 -- 172.21.15.27:0/2408941787 shutdown_connections 2021-11-03T17:13:42.048 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:42.043+0000 7f1273dfc700 1 --2- 172.21.15.27:0/2408941787 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f125805c2a0 0x7f125805e760 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:42.048 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:42.043+0000 7f1273dfc700 1 --2- 172.21.15.27:0/2408941787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f126c0681d0 0x7f126c1118b0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:13:42.049 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:42.043+0000 7f1273dfc700 1 --2- 172.21.15.27:0/2408941787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f126c100fd0 0x7f126c111da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:13:42.049 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:42.043+0000 7f1273dfc700 1 -- 172.21.15.27:0/2408941787 >> 172.21.15.27:0/2408941787 conn(0x7f126c0636c0 msgr2=0x7f126c065ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:13:42.049 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:42.043+0000 7f1273dfc700 1 -- 172.21.15.27:0/2408941787 shutdown_connections 2021-11-03T17:13:42.049 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:13:42.043+0000 7f1273dfc700 1 -- 172.21.15.27:0/2408941787 wait complete. 2021-11-03T17:13:42.050 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 14 2021-11-03T17:13:42.124 INFO:tasks.workunit.client.1.smithi115.stdout:2/9: sync 2021-11-03T17:13:42.124 INFO:tasks.workunit.client.1.smithi115.stdout:2/10: read - no filename 2021-11-03T17:13:42.125 INFO:tasks.workunit.client.1.smithi115.stdout:2/11: truncate - no filename 2021-11-03T17:13:42.125 INFO:tasks.workunit.client.1.smithi115.stdout:2/12: dread - no filename 2021-11-03T17:13:42.125 INFO:tasks.workunit.client.1.smithi115.stdout:2/13: rename - no filename 2021-11-03T17:13:42.125 INFO:tasks.workunit.client.1.smithi115.stdout:2/14: rename - no filename 2021-11-03T17:13:42.125 INFO:tasks.workunit.client.1.smithi115.stdout:2/15: rename - no filename 2021-11-03T17:13:42.126 INFO:tasks.workunit.client.1.smithi115.stdout:2/16: mknod c0 1 2021-11-03T17:13:42.126 INFO:tasks.workunit.client.1.smithi115.stdout:2/17: symlink l1 0 2021-11-03T17:13:42.126 INFO:tasks.workunit.client.1.smithi115.stdout:2/18: write - no filename 2021-11-03T17:13:42.127 INFO:tasks.workunit.client.1.smithi115.stdout:2/19: fdatasync - no filename 2021-11-03T17:13:42.127 INFO:tasks.workunit.client.1.smithi115.stdout:2/20: unlink l1 0 2021-11-03T17:13:42.128 INFO:tasks.workunit.client.1.smithi115.stdout:2/21: dwrite - no filename 2021-11-03T17:13:42.128 INFO:tasks.workunit.client.1.smithi115.stdout:2/22: mknod c2 1 2021-11-03T17:13:42.128 INFO:tasks.workunit.client.1.smithi115.stdout:2/23: chown . 213 1 2021-11-03T17:13:42.128 INFO:tasks.workunit.client.1.smithi115.stdout:2/24: mknod c3 1 2021-11-03T17:13:42.128 INFO:tasks.workunit.client.1.smithi115.stdout:2/25: chown . 20992620 1 2021-11-03T17:13:42.129 INFO:tasks.workunit.client.1.smithi115.stdout:2/26: unlink - no file 2021-11-03T17:13:42.129 INFO:tasks.workunit.client.1.smithi115.stdout:2/27: truncate - no filename 2021-11-03T17:13:42.129 INFO:tasks.workunit.client.1.smithi115.stdout:2/28: fdatasync - no filename 2021-11-03T17:13:42.129 INFO:tasks.workunit.client.1.smithi115.stdout:2/29: rmdir - no directory 2021-11-03T17:13:42.129 INFO:tasks.workunit.client.1.smithi115.stdout:2/30: chown . 1274 1 2021-11-03T17:13:42.131 INFO:tasks.workunit.client.1.smithi115.stdout:2/31: getdents . 0 2021-11-03T17:13:42.131 INFO:tasks.workunit.client.1.smithi115.stdout:2/32: fdatasync - no filename 2021-11-03T17:13:42.131 INFO:tasks.workunit.client.1.smithi115.stdout:2/33: stat - no entries 2021-11-03T17:13:42.131 INFO:tasks.workunit.client.1.smithi115.stdout:2/34: fsync - no filename 2021-11-03T17:13:42.132 INFO:tasks.workunit.client.1.smithi115.stdout:2/35: chown . 14054 1 2021-11-03T17:13:42.132 INFO:tasks.workunit.client.1.smithi115.stdout:2/36: rename - no filename 2021-11-03T17:13:42.132 INFO:tasks.workunit.client.1.smithi115.stdout:2/37: mkdir d4 0 2021-11-03T17:13:42.133 INFO:tasks.workunit.client.1.smithi115.stdout:2/38: chown d4 0 1 2021-11-03T17:13:42.133 INFO:tasks.workunit.client.1.smithi115.stdout:2/39: write - no filename 2021-11-03T17:13:42.133 INFO:tasks.workunit.client.1.smithi115.stdout:2/40: chown d4 151794 1 2021-11-03T17:13:42.133 INFO:tasks.workunit.client.1.smithi115.stdout:2/41: dread - no filename 2021-11-03T17:13:42.134 INFO:tasks.workunit.client.1.smithi115.stdout:2/42: rename d4 to d4/d5 22 2021-11-03T17:13:42.937 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:42 smithi027 conmon[32322]: audit 2021-11-03T17:13:41.709213+0000 mon.smithi027 ( 2021-11-03T17:13:42.937 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:42 smithi027 conmon[32322]: mon.0) 562 : audit [DBG] from='client.? 172.21.15.27:0/3742235370' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:13:42.938 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:42 smithi027 conmon[32322]: audit 2021-11-03T17:13:42.042691+0000 mon.smithi027 (mon.0) 563 : audit [DBG] from='client.? 172.21.15.27:0/2408941787' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:13:42.938 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:42 smithi027 conmon[32322]: cluster 2021-11-03T 2021-11-03T17:13:42.938 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:42 smithi027 conmon[32322]: 17:13:42.598765+0000 mgr.smithi027.ufhhvb (mgr.14162 2021-11-03T17:13:42.938 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:42 smithi027 conmon[32322]: ) 288 : cluster [DBG] pgmap v238: 65 pgs: 65 active+clean; 270 MiB data, 945 MiB used, 535 GiB / 536 GiB avail; 7.9 MiB/s rd, 18 MiB/s wr, 248 op/s 2021-11-03T17:13:42.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:42 smithi115 conmon[34670]: audit 2021-11-03T17:13:41.709213 2021-11-03T17:13:42.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:42 smithi115 conmon[34670]: +0000 mon.smithi027 (mon.0) 562 : audit [DBG] from='client.? 172.21.15.27:0/3742235370' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:13:42.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:42 smithi115 conmon[34670]: audit 2021-11-03T17:13:42.042691+0000 mon.smithi027 2021-11-03T17:13:42.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:42 smithi115 conmon[34670]: (mon.0) 563 : audit [DBG] from='client.? 172.21.15.27:0/2408941787' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:13:42.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:42 smithi115 conmon[34670]: cluster 2021-11 2021-11-03T17:13:42.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:42 smithi115 conmon[34670]: -03T17:13:42.598765+0000 mgr.smithi027.ufhhvb (mgr. 2021-11-03T17:13:42.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:42 smithi115 conmon[34670]: 14162) 288 : cluster [DBG] pgmap v238: 65 pgs: 65 active+clean; 270 MiB data, 945 MiB used, 535 GiB / 536 GiB avail; 7.9 MiB/s rd, 18 MiB/s wr, 248 op/s 2021-11-03T17:13:44.283 INFO:tasks.workunit.client.1.smithi115.stdout:7/41: sync 2021-11-03T17:13:44.284 INFO:tasks.workunit.client.1.smithi115.stdout:1/81: sync 2021-11-03T17:13:44.284 INFO:tasks.workunit.client.1.smithi115.stdout:0/108: sync 2021-11-03T17:13:44.284 INFO:tasks.workunit.client.1.smithi115.stdout:5/96: sync 2021-11-03T17:13:44.284 INFO:tasks.workunit.client.1.smithi115.stdout:3/53: sync 2021-11-03T17:13:44.284 INFO:tasks.workunit.client.1.smithi115.stdout:9/45: sync 2021-11-03T17:13:44.285 INFO:tasks.workunit.client.1.smithi115.stdout:2/43: sync 2021-11-03T17:13:44.285 INFO:tasks.workunit.client.1.smithi115.stdout:8/103: sync 2021-11-03T17:13:44.285 INFO:tasks.workunit.client.1.smithi115.stdout:6/48: sync 2021-11-03T17:13:44.286 INFO:tasks.workunit.client.1.smithi115.stdout:2/44: write - no filename 2021-11-03T17:13:44.286 INFO:tasks.workunit.client.1.smithi115.stdout:4/29: sync 2021-11-03T17:13:44.286 INFO:tasks.workunit.client.1.smithi115.stdout:4/30: mknod d7/ca 1 2021-11-03T17:13:44.287 INFO:tasks.workunit.client.1.smithi115.stdout:8/104: write d4/f12 [6957541,64411] 0 2021-11-03T17:13:44.287 INFO:tasks.workunit.client.1.smithi115.stdout:3/54: link l8 le 0 2021-11-03T17:13:44.288 INFO:tasks.workunit.client.1.smithi115.stdout:1/82: getdents d7/dc/dd 0 2021-11-03T17:13:44.288 INFO:tasks.workunit.client.1.smithi115.stdout:1/83: fdatasync d7/f14 0 2021-11-03T17:13:44.288 INFO:tasks.workunit.client.1.smithi115.stdout:1/84: readlink d7/dc/l1c 0 2021-11-03T17:13:44.289 INFO:tasks.workunit.client.1.smithi115.stdout:7/42: symlink d3/l8 0 2021-11-03T17:13:44.296 INFO:tasks.workunit.client.1.smithi115.stdout:5/97: symlink d1a/l22 0 2021-11-03T17:13:44.296 INFO:tasks.workunit.client.1.smithi115.stdout:5/98: stat d1a/f20 0 2021-11-03T17:13:44.296 INFO:tasks.workunit.client.1.smithi115.stdout:2/45: mkdir d4/d6 0 2021-11-03T17:13:44.297 INFO:tasks.workunit.client.1.smithi115.stdout:2/46: mknod d4/d6/c7 1 2021-11-03T17:13:44.297 INFO:tasks.workunit.client.1.smithi115.stdout:2/47: getdents d4/d6 0 2021-11-03T17:13:44.297 INFO:tasks.workunit.client.1.smithi115.stdout:2/48: link - no file 2021-11-03T17:13:44.297 INFO:tasks.workunit.client.1.smithi115.stdout:4/31: symlink d7/lb 0 2021-11-03T17:13:44.298 INFO:tasks.workunit.client.1.smithi115.stdout:8/105: link d4/l10 d4/d6/d18/l25 0 2021-11-03T17:13:44.298 INFO:tasks.workunit.client.1.smithi115.stdout:3/55: stat le 0 2021-11-03T17:13:44.299 INFO:tasks.workunit.client.1.smithi115.stdout:3/56: write f5 [5020999,2488] 0 2021-11-03T17:13:44.299 INFO:tasks.workunit.client.1.smithi115.stdout:1/85: mkdir d7/dc/d1e 0 2021-11-03T17:13:44.299 INFO:tasks.workunit.client.1.smithi115.stdout:1/86: chown d7/dc/l1c 3079 1 2021-11-03T17:13:44.300 INFO:tasks.workunit.client.1.smithi115.stdout:1/87: write d7/dc/dd/f18 [1716776,30011] 0 2021-11-03T17:13:44.300 INFO:tasks.workunit.client.1.smithi115.stdout:7/43: creat d3/f9 x:0 0 0 2021-11-03T17:13:44.302 INFO:tasks.workunit.client.1.smithi115.stdout:5/99: mkdir d1a/d23 0 2021-11-03T17:13:44.304 INFO:tasks.workunit.client.1.smithi115.stdout:4/32: mkdir d7/dc 0 2021-11-03T17:13:44.304 INFO:tasks.workunit.client.1.smithi115.stdout:4/33: chown l1 1382680 1 2021-11-03T17:13:44.305 INFO:tasks.workunit.client.1.smithi115.stdout:6/49: dwrite d0/d1/f8 [0,4194304] 0 2021-11-03T17:13:44.305 INFO:tasks.workunit.client.1.smithi115.stdout:6/50: write d0/d1/d2/fd [254253,101714] 0 2021-11-03T17:13:44.306 INFO:tasks.workunit.client.1.smithi115.stdout:6/51: fdatasync d0/f7 0 2021-11-03T17:13:44.306 INFO:tasks.workunit.client.1.smithi115.stdout:8/106: creat d4/f26 x:0 0 0 2021-11-03T17:13:44.306 INFO:tasks.workunit.client.1.smithi115.stdout:1/88: truncate f2 1966194 0 2021-11-03T17:13:44.308 INFO:tasks.workunit.client.1.smithi115.stdout:7/44: mkdir d3/da 0 2021-11-03T17:13:44.309 INFO:tasks.workunit.client.1.smithi115.stdout:7/45: mknod d3/cb 1 2021-11-03T17:13:44.309 INFO:tasks.workunit.client.1.smithi115.stdout:7/46: truncate d3/f9 480380 0 2021-11-03T17:13:44.310 INFO:tasks.workunit.client.1.smithi115.stdout:5/100: unlink f7 0 2021-11-03T17:13:44.312 INFO:tasks.workunit.client.1.smithi115.stdout:1/89: truncate d7/dc/ff 2985358 0 2021-11-03T17:13:44.313 INFO:tasks.workunit.client.1.smithi115.stdout:7/47: creat d3/fc x:0 0 0 2021-11-03T17:13:44.313 INFO:tasks.workunit.client.1.smithi115.stdout:7/48: truncate d3/f9 1141730 0 2021-11-03T17:13:44.314 INFO:tasks.workunit.client.1.smithi115.stdout:4/34: write f5 [3537219,89892] 0 2021-11-03T17:13:44.314 INFO:tasks.workunit.client.1.smithi115.stdout:1/90: write d7/f8 [3690305,91035] 0 2021-11-03T17:13:44.315 INFO:tasks.workunit.client.1.smithi115.stdout:1/91: readlink d7/dc/l1c 0 2021-11-03T17:13:44.315 INFO:tasks.workunit.client.1.smithi115.stdout:5/101: mkdir d1a/d23/d24 0 2021-11-03T17:13:44.315 INFO:tasks.workunit.client.1.smithi115.stdout:7/49: dread d3/f9 [0,4194304] 0 2021-11-03T17:13:44.316 INFO:tasks.workunit.client.1.smithi115.stdout:1/92: dread d7/fe [0,4194304] 0 2021-11-03T17:13:44.316 INFO:tasks.workunit.client.1.smithi115.stdout:6/52: mkdir d0/d1/d10 0 2021-11-03T17:13:44.318 INFO:tasks.workunit.client.1.smithi115.stdout:5/102: symlink d1a/l25 0 2021-11-03T17:13:44.319 INFO:tasks.workunit.client.1.smithi115.stdout:3/57: dwrite fc [0,4194304] 0 2021-11-03T17:13:44.319 INFO:tasks.workunit.client.1.smithi115.stdout:3/58: readlink le 0 2021-11-03T17:13:44.320 INFO:tasks.workunit.client.1.smithi115.stdout:3/59: fdatasync fc 0 2021-11-03T17:13:44.320 INFO:tasks.workunit.client.1.smithi115.stdout:3/60: chown f5 173648006 1 2021-11-03T17:13:44.320 INFO:tasks.workunit.client.1.smithi115.stdout:3/61: truncate f9 1228577 0 2021-11-03T17:13:44.320 INFO:tasks.workunit.client.1.smithi115.stdout:3/62: stat l7 0 2021-11-03T17:13:44.321 INFO:tasks.workunit.client.1.smithi115.stdout:3/63: write f9 [201544,110701] 0 2021-11-03T17:13:44.321 INFO:tasks.workunit.client.1.smithi115.stdout:7/50: symlink d3/ld 0 2021-11-03T17:13:44.323 INFO:tasks.workunit.client.1.smithi115.stdout:7/51: dread d3/f9 [0,4194304] 0 2021-11-03T17:13:44.324 INFO:tasks.workunit.client.1.smithi115.stdout:5/103: rename f17 to d1a/d23/f26 0 2021-11-03T17:13:44.325 INFO:tasks.workunit.client.1.smithi115.stdout:1/93: dread d7/f14 [0,4194304] 0 2021-11-03T17:13:44.326 INFO:tasks.workunit.client.1.smithi115.stdout:4/35: dread d7/f8 [0,4194304] 0 2021-11-03T17:13:44.330 INFO:tasks.workunit.client.1.smithi115.stdout:2/49: getdents d4 0 2021-11-03T17:13:44.330 INFO:tasks.workunit.client.1.smithi115.stdout:2/50: dread - no filename 2021-11-03T17:13:44.330 INFO:tasks.workunit.client.1.smithi115.stdout:2/51: stat d4 0 2021-11-03T17:13:44.331 INFO:tasks.workunit.client.1.smithi115.stdout:2/52: dread - no filename 2021-11-03T17:13:44.331 INFO:tasks.workunit.client.1.smithi115.stdout:2/53: mknod d4/d6/c8 1 2021-11-03T17:13:44.331 INFO:tasks.workunit.client.1.smithi115.stdout:2/54: dwrite - no filename 2021-11-03T17:13:44.331 INFO:tasks.workunit.client.1.smithi115.stdout:2/55: dread - no filename 2021-11-03T17:13:44.332 INFO:tasks.workunit.client.1.smithi115.stdout:2/56: write - no filename 2021-11-03T17:13:44.332 INFO:tasks.workunit.client.1.smithi115.stdout:2/57: dread - no filename 2021-11-03T17:13:44.332 INFO:tasks.workunit.client.1.smithi115.stdout:2/58: readlink - no filename 2021-11-03T17:13:44.332 INFO:tasks.workunit.client.1.smithi115.stdout:2/59: mknod d4/d6/c9 1 2021-11-03T17:13:44.333 INFO:tasks.workunit.client.1.smithi115.stdout:2/60: truncate - no filename 2021-11-03T17:13:44.333 INFO:tasks.workunit.client.1.smithi115.stdout:2/61: write - no filename 2021-11-03T17:13:44.333 INFO:tasks.workunit.client.1.smithi115.stdout:2/62: chown d4/d6 156939 1 2021-11-03T17:13:44.335 INFO:tasks.workunit.client.1.smithi115.stdout:1/94: fsync d7/dc/ff 0 2021-11-03T17:13:44.336 INFO:tasks.workunit.client.1.smithi115.stdout:3/64: creat ff x:0 0 0 2021-11-03T17:13:44.336 INFO:tasks.workunit.client.1.smithi115.stdout:3/65: truncate ff 892384 0 2021-11-03T17:13:44.337 INFO:tasks.workunit.client.1.smithi115.stdout:7/52: mkdir d3/da/de 0 2021-11-03T17:13:44.338 INFO:tasks.workunit.client.1.smithi115.stdout:7/53: dread d3/f9 [0,4194304] 0 2021-11-03T17:13:44.338 INFO:tasks.workunit.client.1.smithi115.stdout:6/53: dwrite d0/d1/fe [0,4194304] 0 2021-11-03T17:13:44.338 INFO:tasks.workunit.client.1.smithi115.stdout:5/104: rmdir d1a/d23 39 2021-11-03T17:13:44.339 INFO:tasks.workunit.client.1.smithi115.stdout:5/105: stat d1a/d1b 0 2021-11-03T17:13:44.342 INFO:tasks.workunit.client.1.smithi115.stdout:4/36: rmdir d7 39 2021-11-03T17:13:44.343 INFO:tasks.workunit.client.1.smithi115.stdout:4/37: mknod d7/dc/cd 1 2021-11-03T17:13:44.344 INFO:tasks.workunit.client.1.smithi115.stdout:4/38: write f5 [1889736,107575] 0 2021-11-03T17:13:44.344 INFO:tasks.workunit.client.1.smithi115.stdout:4/39: mknod d7/dc/ce 1 2021-11-03T17:13:44.346 INFO:tasks.workunit.client.1.smithi115.stdout:2/63: creat d4/fa x:0 0 0 2021-11-03T17:13:44.348 INFO:tasks.workunit.client.1.smithi115.stdout:3/66: creat f10 x:0 0 0 2021-11-03T17:13:44.348 INFO:tasks.workunit.client.1.smithi115.stdout:3/67: dread - f10 zero size 2021-11-03T17:13:44.349 INFO:tasks.workunit.client.1.smithi115.stdout:3/68: mknod c11 1 2021-11-03T17:13:44.356 INFO:tasks.workunit.client.1.smithi115.stdout:6/54: rmdir d0/d1 39 2021-11-03T17:13:44.356 INFO:tasks.workunit.client.1.smithi115.stdout:6/55: mknod d0/d1/d2/c11 1 2021-11-03T17:13:44.358 INFO:tasks.workunit.client.1.smithi115.stdout:4/40: symlink d7/lf 0 2021-11-03T17:13:44.358 INFO:tasks.workunit.client.1.smithi115.stdout:4/41: fsync d7/f8 0 2021-11-03T17:13:44.358 INFO:tasks.workunit.client.1.smithi115.stdout:4/42: chown d7/lf 16698197 1 2021-11-03T17:13:44.358 INFO:tasks.workunit.client.1.smithi115.stdout:4/43: mknod d7/c10 1 2021-11-03T17:13:44.359 INFO:tasks.workunit.client.1.smithi115.stdout:5/106: creat d1a/d23/d24/f27 x:0 0 0 2021-11-03T17:13:44.362 INFO:tasks.workunit.client.1.smithi115.stdout:6/56: creat d0/d1/d2/f12 x:0 0 0 2021-11-03T17:13:44.362 INFO:tasks.workunit.client.1.smithi115.stdout:5/107: dread fe [0,4194304] 0 2021-11-03T17:13:44.362 INFO:tasks.workunit.client.1.smithi115.stdout:5/108: read fe [2551459,5234] 0 2021-11-03T17:13:44.363 INFO:tasks.workunit.client.1.smithi115.stdout:5/109: getdents d1a/d1b 0 2021-11-03T17:13:44.364 INFO:tasks.workunit.client.1.smithi115.stdout:4/44: rename f6 to d7/dc/f11 0 2021-11-03T17:13:44.365 INFO:tasks.workunit.client.1.smithi115.stdout:1/95: dwrite d7/f9 [0,4194304] 0 2021-11-03T17:13:44.365 INFO:tasks.workunit.client.1.smithi115.stdout:1/96: readlink d7/l1d 0 2021-11-03T17:13:44.366 INFO:tasks.workunit.client.1.smithi115.stdout:2/64: mkdir d4/db 0 2021-11-03T17:13:44.366 INFO:tasks.workunit.client.1.smithi115.stdout:2/65: rename d4/db to d4/db/dc 22 2021-11-03T17:13:44.367 INFO:tasks.workunit.client.1.smithi115.stdout:2/66: readlink - no filename 2021-11-03T17:13:44.369 INFO:tasks.workunit.client.1.smithi115.stdout:6/57: mkdir d0/d1/d10/d13 0 2021-11-03T17:13:44.370 INFO:tasks.workunit.client.1.smithi115.stdout:6/58: mknod d0/d1/d10/d13/c14 1 2021-11-03T17:13:44.370 INFO:tasks.workunit.client.1.smithi115.stdout:7/54: truncate d3/f9 690653 0 2021-11-03T17:13:44.371 INFO:tasks.workunit.client.1.smithi115.stdout:5/110: rmdir d1a/d23 39 2021-11-03T17:13:44.372 INFO:tasks.workunit.client.1.smithi115.stdout:4/45: creat d7/dc/f12 x:0 0 0 2021-11-03T17:13:44.378 INFO:tasks.workunit.client.1.smithi115.stdout:4/46: creat d7/dc/f13 x:0 0 0 2021-11-03T17:13:44.379 INFO:tasks.workunit.client.1.smithi115.stdout:4/47: mknod d7/c14 1 2021-11-03T17:13:44.379 INFO:tasks.workunit.client.1.smithi115.stdout:2/67: link d4/fa d4/d6/fd 0 2021-11-03T17:13:44.380 INFO:tasks.workunit.client.1.smithi115.stdout:6/59: rename d0/f7 to d0/f15 0 2021-11-03T17:13:44.381 INFO:tasks.workunit.client.1.smithi115.stdout:6/60: chown d0/d1/d2/f12 18841 1 2021-11-03T17:13:44.381 INFO:tasks.workunit.client.1.smithi115.stdout:6/61: chown d0/f6 54891 1 2021-11-03T17:13:44.381 INFO:tasks.workunit.client.1.smithi115.stdout:6/62: chown d0/d1 267409 1 2021-11-03T17:13:44.381 INFO:tasks.workunit.client.1.smithi115.stdout:6/63: write d0/d1/d2/ff [241333,37747] 0 2021-11-03T17:13:44.382 INFO:tasks.workunit.client.1.smithi115.stdout:1/97: dwrite d7/dc/f15 [0,4194304] 0 2021-11-03T17:13:44.382 INFO:tasks.workunit.client.1.smithi115.stdout:1/98: mknod d7/dc/dd/c1f 1 2021-11-03T17:13:44.383 INFO:tasks.workunit.client.1.smithi115.stdout:1/99: read d7/dc/dd/f18 [536222,62637] 0 2021-11-03T17:13:44.383 INFO:tasks.workunit.client.1.smithi115.stdout:4/48: symlink d7/dc/l15 0 2021-11-03T17:13:44.384 INFO:tasks.workunit.client.1.smithi115.stdout:2/68: creat d4/fe x:0 0 0 2021-11-03T17:13:44.384 INFO:tasks.workunit.client.1.smithi115.stdout:2/69: dread - d4/fa zero size 2021-11-03T17:13:44.386 INFO:tasks.workunit.client.1.smithi115.stdout:5/111: rmdir d1a/d1b 0 2021-11-03T17:13:44.387 INFO:tasks.workunit.client.1.smithi115.stdout:1/100: getdents d7/dc 0 2021-11-03T17:13:44.390 INFO:tasks.workunit.client.1.smithi115.stdout:4/49: link l1 d7/l16 0 2021-11-03T17:13:44.393 INFO:tasks.workunit.client.1.smithi115.stdout:2/70: symlink d4/db/lf 0 2021-11-03T17:13:44.394 INFO:tasks.workunit.client.1.smithi115.stdout:2/71: fsync d4/fe 0 2021-11-03T17:13:44.395 INFO:tasks.workunit.client.1.smithi115.stdout:5/112: mkdir d1a/d28 0 2021-11-03T17:13:44.396 INFO:tasks.workunit.client.1.smithi115.stdout:5/113: write d1a/f1d [799483,33331] 0 2021-11-03T17:13:44.397 INFO:tasks.workunit.client.1.smithi115.stdout:1/101: rename d7/dc/d1e to d7/dc/d20 0 2021-11-03T17:13:44.398 INFO:tasks.workunit.client.1.smithi115.stdout:1/102: read d7/fa [691672,111083] 0 2021-11-03T17:13:44.399 INFO:tasks.workunit.client.1.smithi115.stdout:5/114: rename d1a/d23/f26 to d1a/d28/f29 0 2021-11-03T17:13:44.399 INFO:tasks.workunit.client.1.smithi115.stdout:5/115: stat d1a/d28/f29 0 2021-11-03T17:13:44.400 INFO:tasks.workunit.client.1.smithi115.stdout:1/103: symlink d7/dc/d1a/l21 0 2021-11-03T17:13:44.407 INFO:tasks.workunit.client.1.smithi115.stdout:4/50: dwrite d7/dc/f13 [0,4194304] 0 2021-11-03T17:13:44.408 INFO:tasks.workunit.client.1.smithi115.stdout:6/64: dwrite d0/d1/d2/fd [0,4194304] 0 2021-11-03T17:13:44.412 INFO:tasks.workunit.client.1.smithi115.stdout:5/116: getdents d1a/d23 0 2021-11-03T17:13:44.413 INFO:tasks.workunit.client.1.smithi115.stdout:5/117: write f0 [3068412,104469] 0 2021-11-03T17:13:44.413 INFO:tasks.workunit.client.1.smithi115.stdout:5/118: fdatasync f11 0 2021-11-03T17:13:44.414 INFO:tasks.workunit.client.1.smithi115.stdout:5/119: getdents d1a/d23 0 2021-11-03T17:13:44.415 INFO:tasks.workunit.client.1.smithi115.stdout:4/51: creat d7/dc/f17 x:0 0 0 2021-11-03T17:13:44.416 INFO:tasks.workunit.client.1.smithi115.stdout:1/104: dwrite d7/dc/dd/f13 [0,4194304] 0 2021-11-03T17:13:44.419 INFO:tasks.workunit.client.1.smithi115.stdout:1/105: write d7/f14 [3315281,54737] 0 2021-11-03T17:13:44.428 INFO:tasks.workunit.client.1.smithi115.stdout:1/106: dread d7/dc/dd/f13 [0,4194304] 0 2021-11-03T17:13:44.428 INFO:tasks.workunit.client.1.smithi115.stdout:1/107: readlink d7/dc/d1a/l21 0 2021-11-03T17:13:44.429 INFO:tasks.workunit.client.1.smithi115.stdout:5/120: dread f0 [0,4194304] 0 2021-11-03T17:13:44.429 INFO:tasks.workunit.client.1.smithi115.stdout:1/108: getdents d7 0 2021-11-03T17:13:44.430 INFO:tasks.workunit.client.1.smithi115.stdout:1/109: fsync d7/dc/dd/f18 0 2021-11-03T17:13:44.430 INFO:tasks.workunit.client.1.smithi115.stdout:1/110: chown d7/fa 3 1 2021-11-03T17:13:44.430 INFO:tasks.workunit.client.1.smithi115.stdout:1/111: write d7/f14 [185357,97651] 0 2021-11-03T17:13:44.433 INFO:tasks.workunit.client.1.smithi115.stdout:5/121: dread fe [0,4194304] 0 2021-11-03T17:13:44.434 INFO:tasks.workunit.client.1.smithi115.stdout:5/122: rename f12 to d1a/d23/d24/f2a 0 2021-11-03T17:13:44.435 INFO:tasks.workunit.client.1.smithi115.stdout:5/123: chown l13 458 1 2021-11-03T17:13:44.435 INFO:tasks.workunit.client.1.smithi115.stdout:5/124: mknod d1a/d23/c2b 1 2021-11-03T17:13:44.436 INFO:tasks.workunit.client.1.smithi115.stdout:4/52: dwrite d7/dc/f12 [0,4194304] 0 2021-11-03T17:13:44.436 INFO:tasks.workunit.client.1.smithi115.stdout:4/53: stat d7/dc 0 2021-11-03T17:13:44.437 INFO:tasks.workunit.client.1.smithi115.stdout:4/54: chown d7/dc/f12 474 1 2021-11-03T17:13:44.437 INFO:tasks.workunit.client.1.smithi115.stdout:4/55: read d7/dc/f12 [2783527,2829] 0 2021-11-03T17:13:44.437 INFO:tasks.workunit.client.1.smithi115.stdout:4/56: readlink d7/l9 0 2021-11-03T17:13:44.437 INFO:tasks.workunit.client.1.smithi115.stdout:4/57: mknod d7/dc/c18 1 2021-11-03T17:13:44.438 INFO:tasks.workunit.client.1.smithi115.stdout:5/125: creat d1a/d28/f2c x:0 0 0 2021-11-03T17:13:44.438 INFO:tasks.workunit.client.1.smithi115.stdout:5/126: mknod d1a/d23/c2d 1 2021-11-03T17:13:44.438 INFO:tasks.workunit.client.1.smithi115.stdout:1/112: truncate d7/f14 835193 0 2021-11-03T17:13:44.440 INFO:tasks.workunit.client.1.smithi115.stdout:4/58: truncate f5 1856510 0 2021-11-03T17:13:44.440 INFO:tasks.workunit.client.1.smithi115.stdout:4/59: chown d7/dc/l15 4750746 1 2021-11-03T17:13:44.441 INFO:tasks.workunit.client.1.smithi115.stdout:4/60: chown d7/l16 5 1 2021-11-03T17:13:44.441 INFO:tasks.workunit.client.1.smithi115.stdout:5/127: symlink d1a/d28/l2e 0 2021-11-03T17:13:44.442 INFO:tasks.workunit.client.1.smithi115.stdout:4/61: write d7/dc/f13 [2215805,35774] 0 2021-11-03T17:13:44.443 INFO:tasks.workunit.client.1.smithi115.stdout:5/128: symlink d1a/l2f 0 2021-11-03T17:13:44.443 INFO:tasks.workunit.client.1.smithi115.stdout:5/129: write f19 [524431,96843] 0 2021-11-03T17:13:44.443 INFO:tasks.workunit.client.1.smithi115.stdout:5/130: chown d1a/f1d 504 1 2021-11-03T17:13:44.444 INFO:tasks.workunit.client.1.smithi115.stdout:5/131: chown d1a/f1f 5509 1 2021-11-03T17:13:44.444 INFO:tasks.workunit.client.1.smithi115.stdout:5/132: readlink d1a/d28/l2e 0 2021-11-03T17:13:44.444 INFO:tasks.workunit.client.1.smithi115.stdout:5/133: fsync d1a/d23/d24/f27 0 2021-11-03T17:13:44.444 INFO:tasks.workunit.client.1.smithi115.stdout:5/134: chown d1a/d28/f29 1 1 2021-11-03T17:13:44.446 INFO:tasks.workunit.client.1.smithi115.stdout:1/113: dread d7/f9 [0,4194304] 0 2021-11-03T17:13:44.446 INFO:tasks.workunit.client.1.smithi115.stdout:1/114: chown d7 76 1 2021-11-03T17:13:44.447 INFO:tasks.workunit.client.1.smithi115.stdout:5/135: rmdir d1a/d23/d24 39 2021-11-03T17:13:44.449 INFO:tasks.workunit.client.1.smithi115.stdout:1/115: link d7/dc/l19 d7/dc/d1a/l22 0 2021-11-03T17:13:44.449 INFO:tasks.workunit.client.1.smithi115.stdout:1/116: truncate d7/f11 957295 0 2021-11-03T17:13:44.449 INFO:tasks.workunit.client.1.smithi115.stdout:1/117: mknod d7/dc/dd/c23 1 2021-11-03T17:13:44.449 INFO:tasks.workunit.client.1.smithi115.stdout:1/118: readlink d7/dc/l19 0 2021-11-03T17:13:44.450 INFO:tasks.workunit.client.1.smithi115.stdout:5/136: dread fe [0,4194304] 0 2021-11-03T17:13:44.451 INFO:tasks.workunit.client.1.smithi115.stdout:1/119: unlink d7/dc/dd/f13 0 2021-11-03T17:13:44.451 INFO:tasks.workunit.client.1.smithi115.stdout:4/62: write f5 [1143380,113672] 0 2021-11-03T17:13:44.454 INFO:tasks.workunit.client.1.smithi115.stdout:5/137: mkdir d1a/d23/d30 0 2021-11-03T17:13:44.455 INFO:tasks.workunit.client.1.smithi115.stdout:1/120: mkdir d7/dc/dd/d24 0 2021-11-03T17:13:44.455 INFO:tasks.workunit.client.1.smithi115.stdout:4/63: creat d7/dc/f19 x:0 0 0 2021-11-03T17:13:44.455 INFO:tasks.workunit.client.1.smithi115.stdout:4/64: write d7/dc/f11 [487042,79887] 0 2021-11-03T17:13:44.458 INFO:tasks.workunit.client.1.smithi115.stdout:5/138: creat d1a/d28/f31 x:0 0 0 2021-11-03T17:13:44.458 INFO:tasks.workunit.client.1.smithi115.stdout:5/139: write d1a/d23/d24/f27 [1034971,32180] 0 2021-11-03T17:13:44.459 INFO:tasks.workunit.client.1.smithi115.stdout:1/121: link d7/l1d d7/dc/d1a/l25 0 2021-11-03T17:13:44.459 INFO:tasks.workunit.client.1.smithi115.stdout:1/122: chown d7/dc/l1c 2 1 2021-11-03T17:13:44.459 INFO:tasks.workunit.client.1.smithi115.stdout:1/123: write d7/f9 [4209988,65811] 0 2021-11-03T17:13:44.460 INFO:tasks.workunit.client.1.smithi115.stdout:5/140: creat d1a/d23/d24/f32 x:0 0 0 2021-11-03T17:13:44.461 INFO:tasks.workunit.client.1.smithi115.stdout:1/124: mkdir d7/dc/dd/d24/d26 0 2021-11-03T17:13:44.461 INFO:tasks.workunit.client.1.smithi115.stdout:1/125: mknod d7/dc/d20/c27 1 2021-11-03T17:13:44.461 INFO:tasks.workunit.client.1.smithi115.stdout:1/126: readlink d7/dc/d1a/l22 0 2021-11-03T17:13:44.468 INFO:tasks.workunit.client.1.smithi115.stdout:4/65: dread d7/dc/f13 [0,4194304] 0 2021-11-03T17:13:44.484 INFO:tasks.workunit.client.1.smithi115.stdout:5/141: dwrite d1a/d23/d24/f32 [0,4194304] 0 2021-11-03T17:13:44.484 INFO:tasks.workunit.client.1.smithi115.stdout:5/142: fsync f0 0 2021-11-03T17:13:44.485 INFO:tasks.workunit.client.1.smithi115.stdout:5/143: write d1a/d28/f29 [688328,85316] 0 2021-11-03T17:13:44.485 INFO:tasks.workunit.client.1.smithi115.stdout:1/127: dwrite d7/fa [0,4194304] 0 2021-11-03T17:13:44.485 INFO:tasks.workunit.client.1.smithi115.stdout:1/128: stat d7/f11 0 2021-11-03T17:13:44.485 INFO:tasks.workunit.client.1.smithi115.stdout:1/129: write d7/dc/ff [2999626,17658] 0 2021-11-03T17:13:44.486 INFO:tasks.workunit.client.1.smithi115.stdout:4/66: dread f5 [0,4194304] 0 2021-11-03T17:13:44.487 INFO:tasks.workunit.client.1.smithi115.stdout:4/67: mkdir d7/dc/d1a 0 2021-11-03T17:13:44.493 INFO:tasks.workunit.client.1.smithi115.stdout:5/144: dread d1a/d23/d24/f2a [0,4194304] 0 2021-11-03T17:13:44.494 INFO:tasks.workunit.client.1.smithi115.stdout:1/130: dread d7/f8 [0,4194304] 0 2021-11-03T17:13:44.496 INFO:tasks.workunit.client.1.smithi115.stdout:5/145: write fe [406247,44731] 0 2021-11-03T17:13:44.497 INFO:tasks.workunit.client.1.smithi115.stdout:1/131: rmdir d7/dc 39 2021-11-03T17:13:44.497 INFO:tasks.workunit.client.1.smithi115.stdout:1/132: fsync d7/fe 0 2021-11-03T17:13:44.497 INFO:tasks.workunit.client.1.smithi115.stdout:1/133: write d7/fe [308849,116737] 0 2021-11-03T17:13:44.498 INFO:tasks.workunit.client.1.smithi115.stdout:1/134: fdatasync d7/dc/f15 0 2021-11-03T17:13:44.498 INFO:tasks.workunit.client.1.smithi115.stdout:1/135: write d7/dc/dd/f18 [40711,96423] 0 2021-11-03T17:13:44.501 INFO:tasks.workunit.client.1.smithi115.stdout:5/146: dread d1a/d23/d24/f2a [0,4194304] 0 2021-11-03T17:13:44.501 INFO:tasks.workunit.client.1.smithi115.stdout:5/147: fsync d1a/d28/f29 0 2021-11-03T17:13:44.507 INFO:tasks.workunit.client.1.smithi115.stdout:5/148: creat d1a/d23/d24/f33 x:0 0 0 2021-11-03T17:13:44.509 INFO:tasks.workunit.client.1.smithi115.stdout:4/68: dwrite d7/dc/f11 [0,4194304] 0 2021-11-03T17:13:44.521 INFO:tasks.workunit.client.1.smithi115.stdout:4/69: dread f5 [0,4194304] 0 2021-11-03T17:13:44.521 INFO:tasks.workunit.client.1.smithi115.stdout:4/70: mknod d7/c1b 1 2021-11-03T17:13:44.522 INFO:tasks.workunit.client.1.smithi115.stdout:4/71: stat d7/dc 0 2021-11-03T17:13:44.523 INFO:tasks.workunit.client.1.smithi115.stdout:4/72: write d7/dc/f13 [2980841,41125] 0 2021-11-03T17:13:44.523 INFO:tasks.workunit.client.1.smithi115.stdout:4/73: chown d7/dc/f17 1866 1 2021-11-03T17:13:44.523 INFO:tasks.workunit.client.1.smithi115.stdout:4/74: write d7/dc/f13 [4668205,55856] 0 2021-11-03T17:13:44.523 INFO:tasks.workunit.client.1.smithi115.stdout:4/75: truncate d7/dc/f12 3190157 0 2021-11-03T17:13:44.524 INFO:tasks.workunit.client.1.smithi115.stdout:4/76: unlink d7/lf 0 2021-11-03T17:13:44.524 INFO:tasks.workunit.client.1.smithi115.stdout:1/136: dwrite d7/f9 [0,4194304] 0 2021-11-03T17:13:44.525 INFO:tasks.workunit.client.1.smithi115.stdout:4/77: truncate d7/f8 49472 0 2021-11-03T17:13:44.527 INFO:tasks.workunit.client.1.smithi115.stdout:5/149: rename d1a/d23/d24 to d1a/d28/d34 0 2021-11-03T17:13:44.527 INFO:tasks.workunit.client.1.smithi115.stdout:5/150: readlink l14 0 2021-11-03T17:13:44.527 INFO:tasks.workunit.client.1.smithi115.stdout:5/151: stat d1a/d23/d30 0 2021-11-03T17:13:44.528 INFO:tasks.workunit.client.1.smithi115.stdout:1/137: link d7/f11 d7/dc/dd/d24/f28 0 2021-11-03T17:13:44.531 INFO:tasks.workunit.client.1.smithi115.stdout:1/138: symlink d7/dc/dd/d24/d26/l29 0 2021-11-03T17:13:44.533 INFO:tasks.workunit.client.1.smithi115.stdout:4/78: rmdir d7 39 2021-11-03T17:13:44.534 INFO:tasks.workunit.client.1.smithi115.stdout:1/139: mkdir d7/dc/d2a 0 2021-11-03T17:13:44.535 INFO:tasks.workunit.client.1.smithi115.stdout:4/79: creat d7/dc/d1a/f1c x:0 0 0 2021-11-03T17:13:44.535 INFO:tasks.workunit.client.1.smithi115.stdout:4/80: mknod d7/dc/c1d 1 2021-11-03T17:13:44.536 INFO:tasks.workunit.client.1.smithi115.stdout:1/140: read d7/f11 [665962,9603] 0 2021-11-03T17:13:44.536 INFO:tasks.workunit.client.1.smithi115.stdout:4/81: symlink d7/dc/d1a/l1e 0 2021-11-03T17:13:44.537 INFO:tasks.workunit.client.1.smithi115.stdout:1/141: truncate d7/fa 963242 0 2021-11-03T17:13:44.537 INFO:tasks.workunit.client.1.smithi115.stdout:1/142: stat d7/dc/f16 0 2021-11-03T17:13:44.539 INFO:tasks.workunit.client.1.smithi115.stdout:1/143: unlink d7/dc/dd/f18 0 2021-11-03T17:13:44.542 INFO:tasks.workunit.client.1.smithi115.stdout:1/144: rename d7/f11 to d7/dc/dd/d24/f2b 0 2021-11-03T17:13:44.542 INFO:tasks.workunit.client.1.smithi115.stdout:5/152: dwrite d1a/f20 [0,4194304] 0 2021-11-03T17:13:44.545 INFO:tasks.workunit.client.1.smithi115.stdout:5/153: mkdir d1a/d23/d35 0 2021-11-03T17:13:44.546 INFO:tasks.workunit.client.1.smithi115.stdout:5/154: unlink f19 0 2021-11-03T17:13:44.549 INFO:tasks.workunit.client.1.smithi115.stdout:4/82: dread d7/dc/f12 [0,4194304] 0 2021-11-03T17:13:44.549 INFO:tasks.workunit.client.1.smithi115.stdout:4/83: mknod d7/dc/c1f 1 2021-11-03T17:13:44.550 INFO:tasks.workunit.client.1.smithi115.stdout:4/84: truncate d7/dc/f11 3344277 0 2021-11-03T17:13:44.550 INFO:tasks.workunit.client.1.smithi115.stdout:4/85: chown d7/dc/d1a/l1e 82265 1 2021-11-03T17:13:44.551 INFO:tasks.workunit.client.1.smithi115.stdout:4/86: mknod d7/c20 1 2021-11-03T17:13:44.553 INFO:tasks.workunit.client.1.smithi115.stdout:1/145: read d7/fa [763716,27942] 0 2021-11-03T17:13:44.555 INFO:tasks.workunit.client.1.smithi115.stdout:1/146: link f2 d7/dc/dd/d24/d26/f2c 0 2021-11-03T17:13:44.556 INFO:tasks.workunit.client.1.smithi115.stdout:1/147: mkdir d7/dc/d2a/d2d 0 2021-11-03T17:13:44.569 INFO:tasks.workunit.client.1.smithi115.stdout:4/87: dread d7/dc/f11 [0,4194304] 0 2021-11-03T17:13:44.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:44 smithi115 conmon[34670]: cluster 2021-11-03T17:13:44.599191+0000 mgr.smithi027.ufhhvb (mgr.14162) 289 2021-11-03T17:13:44.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:44 smithi115 conmon[34670]: : cluster [DBG] pgmap v239: 65 pgs: 65 active+clean; 279 MiB data, 978 MiB used, 535 GiB / 536 GiB avail; 8.6 MiB/s rd, 17 MiB/s wr, 52 op/s 2021-11-03T17:13:45.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:44 smithi027 conmon[32322]: cluster 2021-11-03T17:13:44.599191 2021-11-03T17:13:45.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:44 smithi027 conmon[32322]: +0000 mgr.smithi027.ufhhvb (mgr.14162) 289 : cluster [DBG] pgmap v239: 65 pgs: 65 active+clean; 279 MiB data, 978 MiB used, 535 GiB / 536 GiB avail; 8.6 MiB/s rd, 17 MiB/s wr, 52 op/s 2021-11-03T17:13:45.856 INFO:tasks.workunit.client.0.smithi027.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2021-11-03T17:13:45.857 INFO:tasks.workunit.client.0.smithi027.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2021-11-03T17:13:45.858 INFO:tasks.workunit.client.0.smithi027.stderr:+ make 2021-11-03T17:13:45.904 INFO:tasks.workunit.client.0.smithi027.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2021-11-03T17:13:46.114 INFO:tasks.workunit.client.0.smithi027.stderr:++ readlink -f fsstress 2021-11-03T17:13:46.115 INFO:tasks.workunit.client.0.smithi027.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2021-11-03T17:13:46.115 INFO:tasks.workunit.client.0.smithi027.stderr:+ popd 2021-11-03T17:13:46.116 INFO:tasks.workunit.client.0.smithi027.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2021-11-03T17:13:46.117 INFO:tasks.workunit.client.0.smithi027.stderr:+ popd 2021-11-03T17:13:46.118 INFO:tasks.workunit.client.0.smithi027.stdout:~/cephtest/mnt.0/client.0/tmp 2021-11-03T17:13:46.118 INFO:tasks.workunit.client.0.smithi027.stderr:++ mktemp -d -p . 2021-11-03T17:13:46.119 INFO:tasks.workunit.client.0.smithi027.stderr:+ T=./tmp.4PQBNuAN7C 2021-11-03T17:13:46.119 INFO:tasks.workunit.client.0.smithi027.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.4PQBNuAN7C -l 1 -n 1000 -p 10 -v 2021-11-03T17:13:46.120 INFO:tasks.workunit.client.0.smithi027.stdout:seed = 1635986893 2021-11-03T17:13:46.125 INFO:tasks.workunit.client.0.smithi027.stdout:0/0: symlink l0 0 2021-11-03T17:13:46.126 INFO:tasks.workunit.client.0.smithi027.stdout:2/0: stat - no entries 2021-11-03T17:13:46.126 INFO:tasks.workunit.client.0.smithi027.stdout:2/1: getdents . 0 2021-11-03T17:13:46.126 INFO:tasks.workunit.client.0.smithi027.stdout:2/2: dwrite - no filename 2021-11-03T17:13:46.127 INFO:tasks.workunit.client.0.smithi027.stdout:1/0: symlink l0 0 2021-11-03T17:13:46.128 INFO:tasks.workunit.client.0.smithi027.stdout:0/1: link l0 l1 0 2021-11-03T17:13:46.128 INFO:tasks.workunit.client.0.smithi027.stdout:0/2: read - no filename 2021-11-03T17:13:46.129 INFO:tasks.workunit.client.0.smithi027.stdout:3/0: unlink - no file 2021-11-03T17:13:46.129 INFO:tasks.workunit.client.0.smithi027.stdout:3/1: dread - no filename 2021-11-03T17:13:46.129 INFO:tasks.workunit.client.0.smithi027.stdout:3/2: read - no filename 2021-11-03T17:13:46.130 INFO:tasks.workunit.client.0.smithi027.stdout:3/3: write - no filename 2021-11-03T17:13:46.130 INFO:tasks.workunit.client.0.smithi027.stdout:3/4: write - no filename 2021-11-03T17:13:46.130 INFO:tasks.workunit.client.0.smithi027.stdout:3/5: chown . 464331070 1 2021-11-03T17:13:46.130 INFO:tasks.workunit.client.0.smithi027.stdout:3/6: dread - no filename 2021-11-03T17:13:46.131 INFO:tasks.workunit.client.0.smithi027.stdout:3/7: write - no filename 2021-11-03T17:13:46.131 INFO:tasks.workunit.client.0.smithi027.stdout:2/3: mkdir d0 0 2021-11-03T17:13:46.132 INFO:tasks.workunit.client.0.smithi027.stdout:2/4: rename d0 to d0/d1 22 2021-11-03T17:13:46.132 INFO:tasks.workunit.client.0.smithi027.stdout:1/1: creat f1 x:0 0 0 2021-11-03T17:13:46.132 INFO:tasks.workunit.client.0.smithi027.stdout:0/3: symlink l2 0 2021-11-03T17:13:46.135 INFO:tasks.workunit.client.0.smithi027.stdout:3/8: creat f0 x:0 0 0 2021-11-03T17:13:46.135 INFO:tasks.workunit.client.0.smithi027.stdout:3/9: dread - f0 zero size 2021-11-03T17:13:46.136 INFO:tasks.workunit.client.0.smithi027.stdout:5/0: rename - no filename 2021-11-03T17:13:46.137 INFO:tasks.workunit.client.0.smithi027.stdout:5/1: rename - no filename 2021-11-03T17:13:46.138 INFO:tasks.workunit.client.0.smithi027.stdout:4/0: creat f0 x:0 0 0 2021-11-03T17:13:46.138 INFO:tasks.workunit.client.0.smithi027.stdout:4/1: dread - f0 zero size 2021-11-03T17:13:46.140 INFO:tasks.workunit.client.0.smithi027.stdout:3/10: mkdir d1 0 2021-11-03T17:13:46.140 INFO:tasks.workunit.client.0.smithi027.stdout:3/11: write f0 [226068,44284] 0 2021-11-03T17:13:46.141 INFO:tasks.workunit.client.0.smithi027.stdout:3/12: chown f0 45 1 2021-11-03T17:13:46.141 INFO:tasks.workunit.client.0.smithi027.stdout:3/13: write f0 [709012,23592] 0 2021-11-03T17:13:46.141 INFO:tasks.workunit.client.0.smithi027.stdout:6/0: dwrite - no filename 2021-11-03T17:13:46.142 INFO:tasks.workunit.client.0.smithi027.stdout:5/2: creat f0 x:0 0 0 2021-11-03T17:13:46.164 INFO:tasks.workunit.client.0.smithi027.stdout:0/4: getdents . 0 2021-11-03T17:13:46.164 INFO:tasks.workunit.client.0.smithi027.stdout:0/5: chown l2 472 1 2021-11-03T17:13:46.167 INFO:tasks.workunit.client.0.smithi027.stdout:3/14: symlink d1/l2 0 2021-11-03T17:13:46.169 INFO:tasks.workunit.client.0.smithi027.stdout:4/2: symlink l1 0 2021-11-03T17:13:46.173 INFO:tasks.workunit.client.0.smithi027.stdout:1/2: getdents . 0 2021-11-03T17:13:46.173 INFO:tasks.workunit.client.0.smithi027.stdout:1/3: chown f1 735477 1 2021-11-03T17:13:46.174 INFO:tasks.workunit.client.0.smithi027.stdout:1/4: chown f1 1377031 1 2021-11-03T17:13:46.174 INFO:tasks.workunit.client.0.smithi027.stdout:1/5: write f1 [66920,37466] 0 2021-11-03T17:13:46.174 INFO:tasks.workunit.client.0.smithi027.stdout:1/6: stat l0 0 2021-11-03T17:13:46.180 INFO:tasks.workunit.client.0.smithi027.stdout:7/0: dread - no filename 2021-11-03T17:13:46.182 INFO:tasks.workunit.client.0.smithi027.stdout:6/1: creat f0 x:0 0 0 2021-11-03T17:13:46.182 INFO:tasks.workunit.client.0.smithi027.stdout:6/2: chown f0 1 1 2021-11-03T17:13:46.183 INFO:tasks.workunit.client.0.smithi027.stdout:6/3: chown f0 28815416 1 2021-11-03T17:13:46.184 INFO:tasks.workunit.client.0.smithi027.stdout:0/6: rename l1 to l3 0 2021-11-03T17:13:46.184 INFO:tasks.workunit.client.0.smithi027.stdout:0/7: mknod c4 1 2021-11-03T17:13:46.185 INFO:tasks.workunit.client.0.smithi027.stdout:0/8: dwrite - no filename 2021-11-03T17:13:46.196 INFO:tasks.workunit.client.0.smithi027.stdout:3/15: symlink d1/l3 0 2021-11-03T17:13:46.206 INFO:tasks.workunit.client.0.smithi027.stdout:1/7: rename f1 to f2 0 2021-11-03T17:13:46.207 INFO:tasks.workunit.client.0.smithi027.stdout:8/0: mknod c0 1 2021-11-03T17:13:46.207 INFO:tasks.workunit.client.0.smithi027.stdout:8/1: write - no filename 2021-11-03T17:13:46.208 INFO:tasks.workunit.client.0.smithi027.stdout:8/2: mknod c1 1 2021-11-03T17:13:46.208 INFO:tasks.workunit.client.0.smithi027.stdout:8/3: mknod c2 1 2021-11-03T17:13:46.208 INFO:tasks.workunit.client.0.smithi027.stdout:8/4: getdents . 0 2021-11-03T17:13:46.208 INFO:tasks.workunit.client.0.smithi027.stdout:8/5: getdents . 0 2021-11-03T17:13:46.208 INFO:tasks.workunit.client.0.smithi027.stdout:8/6: mknod c3 1 2021-11-03T17:13:46.209 INFO:tasks.workunit.client.0.smithi027.stdout:8/7: rename - no filename 2021-11-03T17:13:46.209 INFO:tasks.workunit.client.0.smithi027.stdout:8/8: dread - no filename 2021-11-03T17:13:46.209 INFO:tasks.workunit.client.0.smithi027.stdout:8/9: truncate - no filename 2021-11-03T17:13:46.209 INFO:tasks.workunit.client.0.smithi027.stdout:8/10: write - no filename 2021-11-03T17:13:46.209 INFO:tasks.workunit.client.0.smithi027.stdout:8/11: fdatasync - no filename 2021-11-03T17:13:46.210 INFO:tasks.workunit.client.0.smithi027.stdout:8/12: dwrite - no filename 2021-11-03T17:13:46.213 INFO:tasks.workunit.client.0.smithi027.stdout:6/4: link f0 f1 0 2021-11-03T17:13:46.213 INFO:tasks.workunit.client.0.smithi027.stdout:7/1: mkdir d0 0 2021-11-03T17:13:46.214 INFO:tasks.workunit.client.0.smithi027.stdout:7/2: write - no filename 2021-11-03T17:13:46.214 INFO:tasks.workunit.client.0.smithi027.stdout:7/3: fsync - no filename 2021-11-03T17:13:46.214 INFO:tasks.workunit.client.0.smithi027.stdout:7/4: dwrite - no filename 2021-11-03T17:13:46.214 INFO:tasks.workunit.client.0.smithi027.stdout:7/5: fsync - no filename 2021-11-03T17:13:46.215 INFO:tasks.workunit.client.0.smithi027.stdout:7/6: fdatasync - no filename 2021-11-03T17:13:46.217 INFO:tasks.workunit.client.0.smithi027.stdout:0/9: rename l0 to l5 0 2021-11-03T17:13:46.217 INFO:tasks.workunit.client.0.smithi027.stdout:0/10: dread - no filename 2021-11-03T17:13:46.218 INFO:tasks.workunit.client.0.smithi027.stdout:0/11: dwrite - no filename 2021-11-03T17:13:46.218 INFO:tasks.workunit.client.0.smithi027.stdout:0/12: read - no filename 2021-11-03T17:13:46.218 INFO:tasks.workunit.client.0.smithi027.stdout:8/13: symlink l4 0 2021-11-03T17:13:46.219 INFO:tasks.workunit.client.0.smithi027.stdout:6/5: creat f2 x:0 0 0 2021-11-03T17:13:46.220 INFO:tasks.workunit.client.0.smithi027.stdout:7/7: symlink d0/l1 0 2021-11-03T17:13:46.220 INFO:tasks.workunit.client.0.smithi027.stdout:7/8: dread - no filename 2021-11-03T17:13:46.222 INFO:tasks.workunit.client.0.smithi027.stdout:0/13: creat f6 x:0 0 0 2021-11-03T17:13:46.226 INFO:tasks.workunit.client.0.smithi027.stdout:8/14: creat f5 x:0 0 0 2021-11-03T17:13:46.226 INFO:tasks.workunit.client.0.smithi027.stdout:8/15: truncate f5 276089 0 2021-11-03T17:13:46.227 INFO:tasks.workunit.client.0.smithi027.stdout:7/9: mkdir d0/d2 0 2021-11-03T17:13:46.227 INFO:tasks.workunit.client.0.smithi027.stdout:7/10: dread - no filename 2021-11-03T17:13:46.227 INFO:tasks.workunit.client.0.smithi027.stdout:7/11: fdatasync - no filename 2021-11-03T17:13:46.227 INFO:tasks.workunit.client.0.smithi027.stdout:7/12: write - no filename 2021-11-03T17:13:46.228 INFO:tasks.workunit.client.0.smithi027.stdout:7/13: dread - no filename 2021-11-03T17:13:46.228 INFO:tasks.workunit.client.0.smithi027.stdout:7/14: dread - no filename 2021-11-03T17:13:46.228 INFO:tasks.workunit.client.0.smithi027.stdout:7/15: truncate - no filename 2021-11-03T17:13:46.228 INFO:tasks.workunit.client.0.smithi027.stdout:7/16: write - no filename 2021-11-03T17:13:46.229 INFO:tasks.workunit.client.0.smithi027.stdout:7/17: read - no filename 2021-11-03T17:13:46.229 INFO:tasks.workunit.client.0.smithi027.stdout:7/18: chown d0/d2 14740963 1 2021-11-03T17:13:46.229 INFO:tasks.workunit.client.0.smithi027.stdout:7/19: chown d0 159291 1 2021-11-03T17:13:46.229 INFO:tasks.workunit.client.0.smithi027.stdout:8/16: write f5 [789596,112838] 0 2021-11-03T17:13:46.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:46 smithi115 conmon[34670]: cluster 2021-11-03T17:13:46.599645+0000 mgr.smithi027.ufhhvb (mgr.14162) 290 : cluster [DBG] pgmap v240: 65 pgs: 65 active+clean; 321 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 15 MiB/s rd, 23 MiB/s wr, 75 op/s 2021-11-03T17:13:47.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:46 smithi027 conmon[32322]: cluster 2021-11-03T17:13:46.599645+0000 mgr.smithi027.ufhhvb (mgr.14162) 290 : cluster [DBG] pgmap v240: 65 pgs: 65 active+clean; 321 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 15 MiB/s rd, 23 MiB/s wr, 75 op/s 2021-11-03T17:13:47.857 INFO:tasks.workunit.client.0.smithi027.stdout:3/16: fsync f0 0 2021-11-03T17:13:47.933 INFO:tasks.workunit.client.0.smithi027.stdout:6/6: dwrite f0 [0,4194304] 0 2021-11-03T17:13:48.070 INFO:tasks.workunit.client.0.smithi027.stdout:6/7: stat f1 0 2021-11-03T17:13:48.070 INFO:tasks.workunit.client.0.smithi027.stdout:9/0: stat - no entries 2021-11-03T17:13:48.071 INFO:tasks.workunit.client.0.smithi027.stdout:9/1: dwrite - no filename 2021-11-03T17:13:48.071 INFO:tasks.workunit.client.0.smithi027.stdout:9/2: write - no filename 2021-11-03T17:13:48.128 INFO:tasks.workunit.client.0.smithi027.stdout:6/8: dwrite f0 [0,4194304] 0 2021-11-03T17:13:48.129 INFO:tasks.workunit.client.0.smithi027.stdout:6/9: rmdir - no directory 2021-11-03T17:13:48.131 INFO:tasks.workunit.client.0.smithi027.stdout:6/10: read f1 [4106126,85728] 0 2021-11-03T17:13:48.151 INFO:tasks.workunit.client.0.smithi027.stdout:6/11: dwrite f0 [0,4194304] 0 2021-11-03T17:13:48.281 INFO:tasks.workunit.client.0.smithi027.stdout:8/17: read f5 [321991,46969] 0 2021-11-03T17:13:48.283 INFO:tasks.workunit.client.0.smithi027.stdout:8/18: dread f5 [0,4194304] 0 2021-11-03T17:13:48.284 INFO:tasks.workunit.client.0.smithi027.stdout:8/19: read f5 [809387,110677] 0 2021-11-03T17:13:48.284 INFO:tasks.workunit.client.0.smithi027.stdout:8/20: truncate f5 1513107 0 2021-11-03T17:13:48.357 INFO:tasks.workunit.client.0.smithi027.stdout:4/3: fsync f0 0 2021-11-03T17:13:48.358 INFO:tasks.workunit.client.0.smithi027.stdout:4/4: write f0 [802469,14406] 0 2021-11-03T17:13:48.361 INFO:tasks.workunit.client.0.smithi027.stdout:5/3: getdents . 0 2021-11-03T17:13:48.361 INFO:tasks.workunit.client.0.smithi027.stdout:5/4: dread - f0 zero size 2021-11-03T17:13:48.362 INFO:tasks.workunit.client.0.smithi027.stdout:5/5: mknod c1 1 2021-11-03T17:13:48.367 INFO:tasks.workunit.client.0.smithi027.stdout:0/14: rename l3 to l7 0 2021-11-03T17:13:48.368 INFO:tasks.workunit.client.0.smithi027.stdout:0/15: mknod c8 1 2021-11-03T17:13:48.368 INFO:tasks.workunit.client.0.smithi027.stdout:0/16: dread - f6 zero size 2021-11-03T17:13:48.368 INFO:tasks.workunit.client.0.smithi027.stdout:0/17: chown f6 160 1 2021-11-03T17:13:48.369 INFO:tasks.workunit.client.0.smithi027.stdout:1/8: getdents . 0 2021-11-03T17:13:48.374 INFO:tasks.workunit.client.0.smithi027.stdout:1/9: dread f2 [0,4194304] 0 2021-11-03T17:13:48.374 INFO:tasks.workunit.client.0.smithi027.stdout:1/10: chown f2 59651419 1 2021-11-03T17:13:48.377 INFO:tasks.workunit.client.0.smithi027.stdout:5/6: dwrite f0 [0,4194304] 0 2021-11-03T17:13:48.384 INFO:tasks.workunit.client.0.smithi027.stdout:5/7: dread f0 [0,4194304] 0 2021-11-03T17:13:48.384 INFO:tasks.workunit.client.0.smithi027.stdout:5/8: fdatasync f0 0 2021-11-03T17:13:48.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:48 smithi115 conmon[34670]: cluster 2021-11-03T17:13:48.600010+0000 mgr.smithi027.ufhhvb (mgr.14162) 291 : cluster [DBG] pgmap v241: 65 pgs: 65 active+clean; 333 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 17 MiB/s wr, 235 op/s 2021-11-03T17:13:49.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:48 smithi027 conmon[32322]: cluster 2021-11-03T17:13:48.600010+0000 2021-11-03T17:13:49.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:48 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14162) 291 : cluster [DBG] pgmap v241: 65 pgs: 65 active+clean; 333 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 17 MiB/s wr, 235 op/s 2021-11-03T17:13:49.214 INFO:tasks.workunit.client.1.smithi115.stdout:9/46: sync 2021-11-03T17:13:49.214 INFO:tasks.workunit.client.1.smithi115.stdout:0/109: sync 2021-11-03T17:13:49.214 INFO:tasks.workunit.client.1.smithi115.stdout:3/69: sync 2021-11-03T17:13:49.215 INFO:tasks.workunit.client.1.smithi115.stdout:9/47: chown d1/l3 0 1 2021-11-03T17:13:49.215 INFO:tasks.workunit.client.1.smithi115.stdout:0/110: chown l5 1249061845 1 2021-11-03T17:13:49.215 INFO:tasks.workunit.client.1.smithi115.stdout:9/48: chown d1/f9 60 1 2021-11-03T17:13:49.216 INFO:tasks.workunit.client.1.smithi115.stdout:3/70: creat f12 x:0 0 0 2021-11-03T17:13:49.216 INFO:tasks.workunit.client.1.smithi115.stdout:3/71: chown fc 732499 1 2021-11-03T17:13:49.216 INFO:tasks.workunit.client.1.smithi115.stdout:3/72: dread ff [0,4194304] 0 2021-11-03T17:13:49.217 INFO:tasks.workunit.client.1.smithi115.stdout:3/73: chown ff 229227067 1 2021-11-03T17:13:49.217 INFO:tasks.workunit.client.1.smithi115.stdout:0/111: rename d9/db/l1c to d9/d15/d1a/l2d 0 2021-11-03T17:13:49.219 INFO:tasks.workunit.client.1.smithi115.stdout:0/112: creat d9/d15/d1a/f2e x:0 0 0 2021-11-03T17:13:49.219 INFO:tasks.workunit.client.1.smithi115.stdout:0/113: write d9/f23 [1016286,98966] 0 2021-11-03T17:13:49.219 INFO:tasks.workunit.client.1.smithi115.stdout:0/114: mknod d9/db/c2f 1 2021-11-03T17:13:49.239 INFO:tasks.workunit.client.1.smithi115.stdout:9/49: dwrite d1/f9 [0,4194304] 0 2021-11-03T17:13:49.239 INFO:tasks.workunit.client.1.smithi115.stdout:9/50: fsync d1/d5/fe 0 2021-11-03T17:13:49.240 INFO:tasks.workunit.client.1.smithi115.stdout:9/51: chown d1/l4 160913 1 2021-11-03T17:13:49.240 INFO:tasks.workunit.client.1.smithi115.stdout:9/52: creat d1/d5/ff x:0 0 0 2021-11-03T17:13:49.243 INFO:tasks.workunit.client.1.smithi115.stdout:3/74: dwrite f5 [4194304,4194304] 0 2021-11-03T17:13:49.243 INFO:tasks.workunit.client.1.smithi115.stdout:3/75: write f10 [981007,99464] 0 2021-11-03T17:13:49.244 INFO:tasks.workunit.client.1.smithi115.stdout:3/76: mkdir d13 0 2021-11-03T17:13:49.244 INFO:tasks.workunit.client.1.smithi115.stdout:3/77: chown f12 24989465 1 2021-11-03T17:13:49.249 INFO:tasks.workunit.client.1.smithi115.stdout:9/53: dread d1/f9 [0,4194304] 0 2021-11-03T17:13:49.251 INFO:tasks.workunit.client.1.smithi115.stdout:9/54: write d1/d5/fe [4467376,23366] 0 2021-11-03T17:13:49.251 INFO:tasks.workunit.client.1.smithi115.stdout:9/55: rename d1 to d1/d10 22 2021-11-03T17:13:49.252 INFO:tasks.workunit.client.1.smithi115.stdout:9/56: rmdir d1 39 2021-11-03T17:13:49.252 INFO:tasks.workunit.client.1.smithi115.stdout:9/57: truncate d1/d5/f8 336778 0 2021-11-03T17:13:49.254 INFO:tasks.workunit.client.1.smithi115.stdout:9/58: unlink d1/l3 0 2021-11-03T17:13:49.257 INFO:tasks.workunit.client.1.smithi115.stdout:9/59: fsync d1/d5/f6 0 2021-11-03T17:13:49.259 INFO:tasks.workunit.client.1.smithi115.stdout:9/60: symlink d1/d5/l11 0 2021-11-03T17:13:49.265 INFO:tasks.workunit.client.1.smithi115.stdout:3/78: dwrite f10 [0,4194304] 0 2021-11-03T17:13:49.267 INFO:tasks.workunit.client.1.smithi115.stdout:3/79: write f5 [4039073,43028] 0 2021-11-03T17:13:49.268 INFO:tasks.workunit.client.1.smithi115.stdout:3/80: write fc [109942,80133] 0 2021-11-03T17:13:49.282 INFO:tasks.workunit.client.1.smithi115.stdout:9/61: dwrite d1/f9 [8388608,4194304] 0 2021-11-03T17:13:49.282 INFO:tasks.workunit.client.1.smithi115.stdout:9/62: chown d1/ld 99849 1 2021-11-03T17:13:49.283 INFO:tasks.workunit.client.1.smithi115.stdout:9/63: creat d1/d5/f12 x:0 0 0 2021-11-03T17:13:49.295 INFO:tasks.workunit.client.1.smithi115.stdout:3/81: dwrite f10 [0,4194304] 0 2021-11-03T17:13:49.299 INFO:tasks.workunit.client.1.smithi115.stdout:9/64: dread d1/d5/fc [0,4194304] 0 2021-11-03T17:13:49.299 INFO:tasks.workunit.client.1.smithi115.stdout:9/65: write d1/d5/ff [60291,68982] 0 2021-11-03T17:13:49.305 INFO:tasks.workunit.client.1.smithi115.stdout:9/66: getdents d1/d5 0 2021-11-03T17:13:49.307 INFO:tasks.workunit.client.1.smithi115.stdout:9/67: creat d1/f13 x:0 0 0 2021-11-03T17:13:49.307 INFO:tasks.workunit.client.1.smithi115.stdout:9/68: chown d1/d5/f12 0 1 2021-11-03T17:13:49.307 INFO:tasks.workunit.client.1.smithi115.stdout:9/69: chown d1/f7 56597 1 2021-11-03T17:13:49.308 INFO:tasks.workunit.client.1.smithi115.stdout:9/70: stat d1/d5/f8 0 2021-11-03T17:13:49.308 INFO:tasks.workunit.client.1.smithi115.stdout:9/71: dread - d1/d5/f12 zero size 2021-11-03T17:13:49.308 INFO:tasks.workunit.client.1.smithi115.stdout:9/72: chown d1/f7 31600237 1 2021-11-03T17:13:49.308 INFO:tasks.workunit.client.1.smithi115.stdout:9/73: mknod d1/c14 1 2021-11-03T17:13:49.310 INFO:tasks.workunit.client.1.smithi115.stdout:3/82: dread fc [0,4194304] 0 2021-11-03T17:13:49.312 INFO:tasks.workunit.client.1.smithi115.stdout:3/83: creat d13/f14 x:0 0 0 2021-11-03T17:13:49.313 INFO:tasks.workunit.client.1.smithi115.stdout:3/84: truncate fc 373469 0 2021-11-03T17:13:49.318 INFO:tasks.workunit.client.1.smithi115.stdout:3/85: rmdir d13 39 2021-11-03T17:13:49.319 INFO:tasks.workunit.client.1.smithi115.stdout:3/86: symlink d13/l15 0 2021-11-03T17:13:49.320 INFO:tasks.workunit.client.1.smithi115.stdout:3/87: rename d13/f14 to d13/f16 0 2021-11-03T17:13:49.339 INFO:tasks.workunit.client.1.smithi115.stdout:3/88: dwrite f5 [4194304,4194304] 0 2021-11-03T17:13:49.341 INFO:tasks.workunit.client.1.smithi115.stdout:3/89: rename le to d13/l17 0 2021-11-03T17:13:49.341 INFO:tasks.workunit.client.1.smithi115.stdout:3/90: symlink d13/l18 0 2021-11-03T17:13:49.342 INFO:tasks.workunit.client.1.smithi115.stdout:3/91: creat d13/f19 x:0 0 0 2021-11-03T17:13:49.358 INFO:tasks.workunit.client.1.smithi115.stdout:3/92: dwrite f9 [0,4194304] 0 2021-11-03T17:13:49.360 INFO:tasks.workunit.client.1.smithi115.stdout:3/93: symlink d13/l1a 0 2021-11-03T17:13:49.367 INFO:tasks.workunit.client.1.smithi115.stdout:3/94: rmdir d13 39 2021-11-03T17:13:49.367 INFO:tasks.workunit.client.1.smithi115.stdout:3/95: read - f12 zero size 2021-11-03T17:13:49.385 INFO:tasks.workunit.client.1.smithi115.stdout:3/96: dwrite fc [0,4194304] 0 2021-11-03T17:13:49.385 INFO:tasks.workunit.client.1.smithi115.stdout:3/97: fdatasync d13/f19 0 2021-11-03T17:13:49.387 INFO:tasks.workunit.client.1.smithi115.stdout:3/98: creat d13/f1b x:0 0 0 2021-11-03T17:13:49.387 INFO:tasks.workunit.client.1.smithi115.stdout:3/99: creat d13/f1c x:0 0 0 2021-11-03T17:13:49.388 INFO:tasks.workunit.client.1.smithi115.stdout:3/100: stat d13/f19 0 2021-11-03T17:13:49.403 INFO:tasks.workunit.client.1.smithi115.stdout:3/101: dwrite ff [0,4194304] 0 2021-11-03T17:13:49.404 INFO:tasks.workunit.client.1.smithi115.stdout:3/102: chown d13 76 1 2021-11-03T17:13:49.404 INFO:tasks.workunit.client.1.smithi115.stdout:3/103: stat ff 0 2021-11-03T17:13:49.441 INFO:tasks.workunit.client.1.smithi115.stdout:3/104: dwrite fc [0,4194304] 0 2021-11-03T17:13:49.442 INFO:tasks.workunit.client.1.smithi115.stdout:3/105: rename f5 to d13/f1d 0 2021-11-03T17:13:49.443 INFO:tasks.workunit.client.1.smithi115.stdout:3/106: creat d13/f1e x:0 0 0 2021-11-03T17:13:49.444 INFO:tasks.workunit.client.1.smithi115.stdout:3/107: dread - d13/f1e zero size 2021-11-03T17:13:49.444 INFO:tasks.workunit.client.1.smithi115.stdout:3/108: fdatasync fc 0 2021-11-03T17:13:49.444 INFO:tasks.workunit.client.1.smithi115.stdout:3/109: readlink l1 0 2021-11-03T17:13:49.445 INFO:tasks.workunit.client.1.smithi115.stdout:3/110: creat d13/f1f x:0 0 0 2021-11-03T17:13:49.446 INFO:tasks.workunit.client.1.smithi115.stdout:3/111: write ff [1543453,1955] 0 2021-11-03T17:13:49.446 INFO:tasks.workunit.client.1.smithi115.stdout:3/112: mkdir d13/d20 0 2021-11-03T17:13:49.447 INFO:tasks.workunit.client.1.smithi115.stdout:3/113: creat d13/f21 x:0 0 0 2021-11-03T17:13:49.454 INFO:tasks.workunit.client.1.smithi115.stdout:3/114: dread f9 [0,4194304] 0 2021-11-03T17:13:49.455 INFO:tasks.workunit.client.1.smithi115.stdout:3/115: readlink d13/l18 0 2021-11-03T17:13:49.455 INFO:tasks.workunit.client.1.smithi115.stdout:3/116: write d13/f16 [679719,94872] 0 2021-11-03T17:13:49.455 INFO:tasks.workunit.client.1.smithi115.stdout:3/117: dread - d13/f1b zero size 2021-11-03T17:13:49.455 INFO:tasks.workunit.client.1.smithi115.stdout:3/118: write d13/f1c [138728,3574] 0 2021-11-03T17:13:49.456 INFO:tasks.workunit.client.1.smithi115.stdout:3/119: creat d13/f22 x:0 0 0 2021-11-03T17:13:49.456 INFO:tasks.workunit.client.1.smithi115.stdout:3/120: creat d13/f23 x:0 0 0 2021-11-03T17:13:49.464 INFO:tasks.workunit.client.1.smithi115.stdout:3/121: getdents d13 0 2021-11-03T17:13:49.464 INFO:tasks.workunit.client.1.smithi115.stdout:3/122: mknod d13/d20/c24 1 2021-11-03T17:13:49.465 INFO:tasks.workunit.client.1.smithi115.stdout:3/123: creat d13/f25 x:0 0 0 2021-11-03T17:13:49.469 INFO:tasks.workunit.client.1.smithi115.stdout:3/124: rmdir d13 39 2021-11-03T17:13:49.470 INFO:tasks.workunit.client.1.smithi115.stdout:3/125: getdents d13/d20 0 2021-11-03T17:13:49.486 INFO:tasks.workunit.client.1.smithi115.stdout:3/126: dwrite d13/f16 [0,4194304] 0 2021-11-03T17:13:49.488 INFO:tasks.workunit.client.1.smithi115.stdout:3/127: dread - d13/f1f zero size 2021-11-03T17:13:49.488 INFO:tasks.workunit.client.1.smithi115.stdout:3/128: truncate d13/f1e 466679 0 2021-11-03T17:13:49.488 INFO:tasks.workunit.client.1.smithi115.stdout:3/129: chown f10 14 1 2021-11-03T17:13:49.489 INFO:tasks.workunit.client.1.smithi115.stdout:3/130: creat d13/f26 x:0 0 0 2021-11-03T17:13:49.889 INFO:tasks.workunit.client.1.smithi115.stdout:4/88: sync 2021-11-03T17:13:49.889 INFO:tasks.workunit.client.1.smithi115.stdout:1/148: sync 2021-11-03T17:13:49.889 INFO:tasks.workunit.client.1.smithi115.stdout:8/107: sync 2021-11-03T17:13:49.890 INFO:tasks.workunit.client.1.smithi115.stdout:6/65: sync 2021-11-03T17:13:49.890 INFO:tasks.workunit.client.1.smithi115.stdout:2/72: sync 2021-11-03T17:13:49.890 INFO:tasks.workunit.client.1.smithi115.stdout:7/55: sync 2021-11-03T17:13:49.890 INFO:tasks.workunit.client.1.smithi115.stdout:4/89: chown d7/l9 245692 1 2021-11-03T17:13:49.890 INFO:tasks.workunit.client.1.smithi115.stdout:8/108: dread - d4/f26 zero size 2021-11-03T17:13:49.891 INFO:tasks.workunit.client.1.smithi115.stdout:7/56: read - d3/fc zero size 2021-11-03T17:13:49.891 INFO:tasks.workunit.client.1.smithi115.stdout:1/149: fsync d7/fe 0 2021-11-03T17:13:49.891 INFO:tasks.workunit.client.1.smithi115.stdout:4/90: dread - d7/dc/f19 zero size 2021-11-03T17:13:49.891 INFO:tasks.workunit.client.1.smithi115.stdout:1/150: fdatasync d7/dc/f16 0 2021-11-03T17:13:49.892 INFO:tasks.workunit.client.1.smithi115.stdout:8/109: fdatasync d4/d6/d18/f23 0 2021-11-03T17:13:49.892 INFO:tasks.workunit.client.1.smithi115.stdout:4/91: truncate d7/dc/f19 67185 0 2021-11-03T17:13:49.892 INFO:tasks.workunit.client.1.smithi115.stdout:2/73: write d4/fa [671328,29561] 0 2021-11-03T17:13:49.893 INFO:tasks.workunit.client.1.smithi115.stdout:1/151: rmdir d7/dc/d20 0 2021-11-03T17:13:49.899 INFO:tasks.workunit.client.1.smithi115.stdout:4/92: mkdir d7/dc/d1a/d21 0 2021-11-03T17:13:49.899 INFO:tasks.workunit.client.1.smithi115.stdout:2/74: creat d4/f10 x:0 0 0 2021-11-03T17:13:49.899 INFO:tasks.workunit.client.1.smithi115.stdout:1/152: getdents d7/dc/dd/d24 0 2021-11-03T17:13:49.899 INFO:tasks.workunit.client.1.smithi115.stdout:1/153: fsync d7/dc/dd/d24/f28 0 2021-11-03T17:13:49.900 INFO:tasks.workunit.client.1.smithi115.stdout:4/93: creat d7/dc/d1a/d21/f22 x:0 0 0 2021-11-03T17:13:49.900 INFO:tasks.workunit.client.1.smithi115.stdout:4/94: dread - d7/dc/d1a/f1c zero size 2021-11-03T17:13:49.900 INFO:tasks.workunit.client.1.smithi115.stdout:4/95: truncate d7/dc/d1a/d21/f22 8505 0 2021-11-03T17:13:49.900 INFO:tasks.workunit.client.1.smithi115.stdout:4/96: write d7/dc/d1a/d21/f22 [802164,23207] 0 2021-11-03T17:13:49.901 INFO:tasks.workunit.client.1.smithi115.stdout:8/110: dread d4/f12 [0,4194304] 0 2021-11-03T17:13:49.902 INFO:tasks.workunit.client.1.smithi115.stdout:6/66: dread d0/f15 [0,4194304] 0 2021-11-03T17:13:49.902 INFO:tasks.workunit.client.1.smithi115.stdout:6/67: dread - d0/d1/d2/f12 zero size 2021-11-03T17:13:49.902 INFO:tasks.workunit.client.1.smithi115.stdout:6/68: chown d0/d1/d2/fc 14300 1 2021-11-03T17:13:49.902 INFO:tasks.workunit.client.1.smithi115.stdout:6/69: fsync d0/d1/f5 0 2021-11-03T17:13:49.903 INFO:tasks.workunit.client.1.smithi115.stdout:6/70: dread - d0/d1/d2/f12 zero size 2021-11-03T17:13:49.903 INFO:tasks.workunit.client.1.smithi115.stdout:6/71: dread - d0/d1/d2/f12 zero size 2021-11-03T17:13:49.903 INFO:tasks.workunit.client.1.smithi115.stdout:8/111: creat d4/f27 x:0 0 0 2021-11-03T17:13:49.903 INFO:tasks.workunit.client.1.smithi115.stdout:8/112: chown d4/l7 15 1 2021-11-03T17:13:49.905 INFO:tasks.workunit.client.1.smithi115.stdout:1/154: dread f2 [0,4194304] 0 2021-11-03T17:13:49.908 INFO:tasks.workunit.client.1.smithi115.stdout:6/72: link d0/d1/d2/ff d0/d1/d10/f16 0 2021-11-03T17:13:49.911 INFO:tasks.workunit.client.1.smithi115.stdout:6/73: getdents d0 0 2021-11-03T17:13:49.911 INFO:tasks.workunit.client.1.smithi115.stdout:6/74: write d0/d1/d2/fc [1480260,83093] 0 2021-11-03T17:13:49.912 INFO:tasks.workunit.client.1.smithi115.stdout:7/57: dwrite d3/f9 [0,4194304] 0 2021-11-03T17:13:49.912 INFO:tasks.workunit.client.1.smithi115.stdout:8/113: dread d4/f1f [0,4194304] 0 2021-11-03T17:13:49.912 INFO:tasks.workunit.client.1.smithi115.stdout:8/114: fdatasync d4/d6/f8 0 2021-11-03T17:13:49.912 INFO:tasks.workunit.client.1.smithi115.stdout:7/58: write d3/fc [695220,114885] 0 2021-11-03T17:13:49.912 INFO:tasks.workunit.client.1.smithi115.stdout:7/59: stat d3/da/de 0 2021-11-03T17:13:49.913 INFO:tasks.workunit.client.1.smithi115.stdout:7/60: mknod d3/da/cf 1 2021-11-03T17:13:49.914 INFO:tasks.workunit.client.1.smithi115.stdout:1/155: dread f2 [0,4194304] 0 2021-11-03T17:13:49.915 INFO:tasks.workunit.client.1.smithi115.stdout:5/155: sync 2021-11-03T17:13:49.915 INFO:tasks.workunit.client.1.smithi115.stdout:4/97: dwrite d7/dc/f17 [0,4194304] 0 2021-11-03T17:13:49.916 INFO:tasks.workunit.client.1.smithi115.stdout:6/75: creat d0/f17 x:0 0 0 2021-11-03T17:13:49.918 INFO:tasks.workunit.client.1.smithi115.stdout:8/115: creat d4/f28 x:0 0 0 2021-11-03T17:13:49.919 INFO:tasks.workunit.client.1.smithi115.stdout:7/61: rename d3/l5 to d3/l10 0 2021-11-03T17:13:49.919 INFO:tasks.workunit.client.1.smithi115.stdout:1/156: stat d7/dc/d1a/l25 0 2021-11-03T17:13:49.921 INFO:tasks.workunit.client.1.smithi115.stdout:5/156: symlink d1a/d23/d30/l36 0 2021-11-03T17:13:49.924 INFO:tasks.workunit.client.1.smithi115.stdout:4/98: mkdir d7/dc/d1a/d23 0 2021-11-03T17:13:49.927 INFO:tasks.workunit.client.1.smithi115.stdout:8/116: mknod d4/c29 1 2021-11-03T17:13:49.928 INFO:tasks.workunit.client.1.smithi115.stdout:6/76: mkdir d0/d18 0 2021-11-03T17:13:49.928 INFO:tasks.workunit.client.1.smithi115.stdout:1/157: unlink d7/fa 0 2021-11-03T17:13:49.931 INFO:tasks.workunit.client.1.smithi115.stdout:6/77: dread d0/d1/d2/ff [0,4194304] 0 2021-11-03T17:13:49.931 INFO:tasks.workunit.client.1.smithi115.stdout:5/157: rename d1a/l25 to d1a/d28/d34/l37 0 2021-11-03T17:13:49.934 INFO:tasks.workunit.client.1.smithi115.stdout:4/99: rmdir d7/dc/d1a/d23 0 2021-11-03T17:13:49.935 INFO:tasks.workunit.client.1.smithi115.stdout:1/158: symlink d7/dc/dd/d24/d26/l2e 0 2021-11-03T17:13:49.936 INFO:tasks.workunit.client.1.smithi115.stdout:5/158: dread d1a/d28/d34/f2a [0,4194304] 0 2021-11-03T17:13:49.936 INFO:tasks.workunit.client.1.smithi115.stdout:8/117: getdents d4/d6 0 2021-11-03T17:13:49.937 INFO:tasks.workunit.client.1.smithi115.stdout:4/100: write d7/dc/f11 [489875,11397] 0 2021-11-03T17:13:49.938 INFO:tasks.workunit.client.1.smithi115.stdout:5/159: unlink d1a/f20 0 2021-11-03T17:13:49.938 INFO:tasks.workunit.client.1.smithi115.stdout:5/160: readlink l16 0 2021-11-03T17:13:49.938 INFO:tasks.workunit.client.1.smithi115.stdout:5/161: write d1a/f1e [1398311,44197] 0 2021-11-03T17:13:49.939 INFO:tasks.workunit.client.1.smithi115.stdout:6/78: creat d0/d1/f19 x:0 0 0 2021-11-03T17:13:49.939 INFO:tasks.workunit.client.1.smithi115.stdout:4/101: link d7/l16 d7/dc/d1a/l24 0 2021-11-03T17:13:49.940 INFO:tasks.workunit.client.1.smithi115.stdout:1/159: dread d7/f8 [0,4194304] 0 2021-11-03T17:13:49.940 INFO:tasks.workunit.client.1.smithi115.stdout:7/62: dwrite d3/f9 [4194304,4194304] 0 2021-11-03T17:13:49.941 INFO:tasks.workunit.client.1.smithi115.stdout:6/79: rmdir d0/d18 0 2021-11-03T17:13:49.942 INFO:tasks.workunit.client.1.smithi115.stdout:8/118: dread d4/d6/d18/f23 [4194304,4194304] 0 2021-11-03T17:13:49.944 INFO:tasks.workunit.client.1.smithi115.stdout:4/102: unlink f5 0 2021-11-03T17:13:49.944 INFO:tasks.workunit.client.1.smithi115.stdout:4/103: chown d7/l16 53363614 1 2021-11-03T17:13:49.944 INFO:tasks.workunit.client.1.smithi115.stdout:4/104: chown d7/l16 106645 1 2021-11-03T17:13:49.946 INFO:tasks.workunit.client.1.smithi115.stdout:1/160: unlink d7/dc/dd/d24/d26/l2e 0 2021-11-03T17:13:49.951 INFO:tasks.workunit.client.1.smithi115.stdout:1/161: creat d7/dc/f2f x:0 0 0 2021-11-03T17:13:49.951 INFO:tasks.workunit.client.1.smithi115.stdout:1/162: mknod d7/dc/d2a/d2d/c30 1 2021-11-03T17:13:49.951 INFO:tasks.workunit.client.1.smithi115.stdout:6/80: dread d0/d1/f8 [0,4194304] 0 2021-11-03T17:13:49.952 INFO:tasks.workunit.client.1.smithi115.stdout:6/81: fsync d0/d1/d10/f16 0 2021-11-03T17:13:49.952 INFO:tasks.workunit.client.1.smithi115.stdout:6/82: write d0/d1/d2/ff [672548,70282] 0 2021-11-03T17:13:49.952 INFO:tasks.workunit.client.1.smithi115.stdout:6/83: mknod d0/c1a 1 2021-11-03T17:13:49.952 INFO:tasks.workunit.client.1.smithi115.stdout:6/84: fdatasync d0/d1/f8 0 2021-11-03T17:13:49.953 INFO:tasks.workunit.client.1.smithi115.stdout:1/163: truncate f2 1993694 0 2021-11-03T17:13:49.954 INFO:tasks.workunit.client.1.smithi115.stdout:1/164: creat d7/dc/d2a/f31 x:0 0 0 2021-11-03T17:13:49.955 INFO:tasks.workunit.client.1.smithi115.stdout:1/165: mkdir d7/dc/d2a/d2d/d32 0 2021-11-03T17:13:49.957 INFO:tasks.workunit.client.1.smithi115.stdout:1/166: creat d7/dc/d2a/f33 x:0 0 0 2021-11-03T17:13:49.958 INFO:tasks.workunit.client.1.smithi115.stdout:1/167: read - d7/dc/f2f zero size 2021-11-03T17:13:49.958 INFO:tasks.workunit.client.1.smithi115.stdout:5/162: dwrite d1a/d28/f31 [0,4194304] 0 2021-11-03T17:13:49.959 INFO:tasks.workunit.client.1.smithi115.stdout:8/119: dread d4/f1f [0,4194304] 0 2021-11-03T17:13:49.963 INFO:tasks.workunit.client.1.smithi115.stdout:5/163: read d1a/d28/f31 [906141,65693] 0 2021-11-03T17:13:49.964 INFO:tasks.workunit.client.1.smithi115.stdout:5/164: write f11 [595640,45182] 0 2021-11-03T17:13:49.964 INFO:tasks.workunit.client.1.smithi115.stdout:5/165: chown fd 3700117 1 2021-11-03T17:13:49.964 INFO:tasks.workunit.client.1.smithi115.stdout:8/120: creat d4/d6/f2a x:0 0 0 2021-11-03T17:13:49.969 INFO:tasks.workunit.client.1.smithi115.stdout:7/63: dread d3/fc [0,4194304] 0 2021-11-03T17:13:49.977 INFO:tasks.workunit.client.1.smithi115.stdout:4/105: dwrite d7/dc/f12 [0,4194304] 0 2021-11-03T17:13:49.978 INFO:tasks.workunit.client.1.smithi115.stdout:7/64: write d3/fc [314160,31440] 0 2021-11-03T17:13:49.978 INFO:tasks.workunit.client.1.smithi115.stdout:7/65: fdatasync d3/f9 0 2021-11-03T17:13:49.979 INFO:tasks.workunit.client.1.smithi115.stdout:4/106: rename d7/dc/f19 to d7/dc/d1a/d21/f25 0 2021-11-03T17:13:49.980 INFO:tasks.workunit.client.1.smithi115.stdout:4/107: truncate d7/dc/f13 3769497 0 2021-11-03T17:13:49.981 INFO:tasks.workunit.client.1.smithi115.stdout:4/108: symlink d7/l26 0 2021-11-03T17:13:49.982 INFO:tasks.workunit.client.1.smithi115.stdout:4/109: symlink d7/l27 0 2021-11-03T17:13:49.984 INFO:tasks.workunit.client.1.smithi115.stdout:6/85: dwrite d0/d1/d2/fd [0,4194304] 0 2021-11-03T17:13:49.984 INFO:tasks.workunit.client.1.smithi115.stdout:6/86: mknod d0/d1/d10/d13/c1b 1 2021-11-03T17:13:49.987 INFO:tasks.workunit.client.1.smithi115.stdout:6/87: dread d0/d1/f8 [0,4194304] 0 2021-11-03T17:13:49.988 INFO:tasks.workunit.client.1.smithi115.stdout:1/168: dwrite d7/dc/f2f [0,4194304] 0 2021-11-03T17:13:49.989 INFO:tasks.workunit.client.1.smithi115.stdout:5/166: dwrite d1a/d28/d34/f27 [0,4194304] 0 2021-11-03T17:13:49.989 INFO:tasks.workunit.client.1.smithi115.stdout:6/88: mkdir d0/d1/d10/d13/d1c 0 2021-11-03T17:13:49.990 INFO:tasks.workunit.client.1.smithi115.stdout:6/89: fdatasync d0/d1/d2/fd 0 2021-11-03T17:13:49.990 INFO:tasks.workunit.client.1.smithi115.stdout:8/121: rmdir d4 39 2021-11-03T17:13:49.991 INFO:tasks.workunit.client.1.smithi115.stdout:8/122: dread - d4/d6/f2a zero size 2021-11-03T17:13:49.997 INFO:tasks.workunit.client.1.smithi115.stdout:4/110: dread d7/dc/f17 [0,4194304] 0 2021-11-03T17:13:49.997 INFO:tasks.workunit.client.1.smithi115.stdout:4/111: write d7/f8 [519214,12987] 0 2021-11-03T17:13:49.997 INFO:tasks.workunit.client.1.smithi115.stdout:8/123: rename d4/f12 to d4/d6/f2b 0 2021-11-03T17:13:49.999 INFO:tasks.workunit.client.1.smithi115.stdout:8/124: creat d4/d6/f2c x:0 0 0 2021-11-03T17:13:50.000 INFO:tasks.workunit.client.1.smithi115.stdout:8/125: rename d4/f1f to d4/d6/d18/f2d 0 2021-11-03T17:13:50.001 INFO:tasks.workunit.client.1.smithi115.stdout:8/126: symlink d4/d6/l2e 0 2021-11-03T17:13:50.002 INFO:tasks.workunit.client.1.smithi115.stdout:6/90: fdatasync d0/d1/d10/f16 0 2021-11-03T17:13:50.002 INFO:tasks.workunit.client.1.smithi115.stdout:6/91: mknod d0/d1/d10/c1d 1 2021-11-03T17:13:50.003 INFO:tasks.workunit.client.1.smithi115.stdout:6/92: fdatasync d0/d1/fe 0 2021-11-03T17:13:50.003 INFO:tasks.workunit.client.1.smithi115.stdout:4/112: getdents d7/dc/d1a/d21 0 2021-11-03T17:13:50.049 INFO:tasks.workunit.client.1.smithi115.stdout:5/167: dwrite d1a/f1d [0,4194304] 0 2021-11-03T17:13:50.050 INFO:tasks.workunit.client.1.smithi115.stdout:7/66: dwrite d3/fc [0,4194304] 0 2021-11-03T17:13:50.050 INFO:tasks.workunit.client.1.smithi115.stdout:1/169: dwrite d7/f14 [0,4194304] 0 2021-11-03T17:13:50.051 INFO:tasks.workunit.client.1.smithi115.stdout:1/170: read - d7/dc/d2a/f33 zero size 2021-11-03T17:13:50.051 INFO:tasks.workunit.client.1.smithi115.stdout:1/171: fsync f2 0 2021-11-03T17:13:50.053 INFO:tasks.workunit.client.1.smithi115.stdout:6/93: dwrite d0/d1/d10/f16 [0,4194304] 0 2021-11-03T17:13:50.053 INFO:tasks.workunit.client.1.smithi115.stdout:4/113: dwrite d7/dc/f13 [4194304,4194304] 0 2021-11-03T17:13:50.054 INFO:tasks.workunit.client.1.smithi115.stdout:4/114: write d7/dc/d1a/f1c [7390,90078] 0 2021-11-03T17:13:50.054 INFO:tasks.workunit.client.1.smithi115.stdout:4/115: mknod d7/dc/d1a/d21/c28 1 2021-11-03T17:13:50.055 INFO:tasks.workunit.client.1.smithi115.stdout:8/127: dwrite d4/d6/f2a [0,4194304] 0 2021-11-03T17:13:50.055 INFO:tasks.workunit.client.1.smithi115.stdout:5/168: symlink d1a/l38 0 2021-11-03T17:13:50.056 INFO:tasks.workunit.client.1.smithi115.stdout:7/67: write d3/fc [106357,70632] 0 2021-11-03T17:13:50.056 INFO:tasks.workunit.client.1.smithi115.stdout:7/68: truncate d3/fc 4888329 0 2021-11-03T17:13:50.057 INFO:tasks.workunit.client.1.smithi115.stdout:6/94: write d0/d1/d2/fd [982328,73639] 0 2021-11-03T17:13:50.061 INFO:tasks.workunit.client.1.smithi115.stdout:8/128: unlink d4/f14 0 2021-11-03T17:13:50.061 INFO:tasks.workunit.client.1.smithi115.stdout:8/129: read - d4/f26 zero size 2021-11-03T17:13:50.065 INFO:tasks.workunit.client.1.smithi115.stdout:6/95: mkdir d0/d1/d10/d13/d1c/d1e 0 2021-11-03T17:13:50.067 INFO:tasks.workunit.client.1.smithi115.stdout:6/96: write d0/d1/d2/ff [3397397,8502] 0 2021-11-03T17:13:50.068 INFO:tasks.workunit.client.1.smithi115.stdout:6/97: stat d0 0 2021-11-03T17:13:50.071 INFO:tasks.workunit.client.1.smithi115.stdout:6/98: link d0/d1/f5 d0/d1/d2/f1f 0 2021-11-03T17:13:50.072 INFO:tasks.workunit.client.1.smithi115.stdout:6/99: creat d0/d1/d10/d13/d1c/d1e/f20 x:0 0 0 2021-11-03T17:13:50.073 INFO:tasks.workunit.client.1.smithi115.stdout:6/100: creat d0/d1/d10/d13/d1c/f21 x:0 0 0 2021-11-03T17:13:50.077 INFO:tasks.workunit.client.1.smithi115.stdout:5/169: dwrite d1a/f1d [0,4194304] 0 2021-11-03T17:13:50.078 INFO:tasks.workunit.client.1.smithi115.stdout:7/69: dwrite d3/fc [0,4194304] 0 2021-11-03T17:13:50.078 INFO:tasks.workunit.client.1.smithi115.stdout:7/70: chown d3/l8 112966 1 2021-11-03T17:13:50.079 INFO:tasks.workunit.client.1.smithi115.stdout:7/71: mknod d3/da/de/c11 1 2021-11-03T17:13:50.079 INFO:tasks.workunit.client.1.smithi115.stdout:5/170: symlink d1a/d28/d34/l39 0 2021-11-03T17:13:50.080 INFO:tasks.workunit.client.1.smithi115.stdout:5/171: mknod d1a/d28/c3a 1 2021-11-03T17:13:50.080 INFO:tasks.workunit.client.1.smithi115.stdout:5/172: truncate d1a/d28/d34/f32 4199582 0 2021-11-03T17:13:50.082 INFO:tasks.workunit.client.1.smithi115.stdout:7/72: rmdir d3/da/de 0 2021-11-03T17:13:50.084 INFO:tasks.workunit.client.1.smithi115.stdout:4/116: truncate d7/f8 48619 0 2021-11-03T17:13:50.084 INFO:tasks.workunit.client.1.smithi115.stdout:5/173: link f0 d1a/d23/d35/f3b 0 2021-11-03T17:13:50.086 INFO:tasks.workunit.client.1.smithi115.stdout:4/117: symlink d7/dc/d1a/d21/l29 0 2021-11-03T17:13:50.096 INFO:tasks.workunit.client.1.smithi115.stdout:5/174: dread f9 [0,4194304] 0 2021-11-03T17:13:50.097 INFO:tasks.workunit.client.1.smithi115.stdout:7/73: dread d3/fc [0,4194304] 0 2021-11-03T17:13:50.098 INFO:tasks.workunit.client.1.smithi115.stdout:8/130: truncate d4/fa 4665922 0 2021-11-03T17:13:50.098 INFO:tasks.workunit.client.1.smithi115.stdout:8/131: write d4/f28 [564883,127975] 0 2021-11-03T17:13:50.098 INFO:tasks.workunit.client.1.smithi115.stdout:8/132: chown d4/d6/l24 254712 1 2021-11-03T17:13:50.098 INFO:tasks.workunit.client.1.smithi115.stdout:8/133: fdatasync d4/d6/f16 0 2021-11-03T17:13:50.099 INFO:tasks.workunit.client.1.smithi115.stdout:8/134: getdents d4/d6 0 2021-11-03T17:13:50.099 INFO:tasks.workunit.client.1.smithi115.stdout:8/135: fsync d4/d6/f1d 0 2021-11-03T17:13:50.100 INFO:tasks.workunit.client.1.smithi115.stdout:8/136: mknod d4/c2f 1 2021-11-03T17:13:50.100 INFO:tasks.workunit.client.1.smithi115.stdout:4/118: write d7/f8 [860488,15710] 0 2021-11-03T17:13:50.106 INFO:tasks.workunit.client.1.smithi115.stdout:6/101: dwrite d0/d1/f5 [0,4194304] 0 2021-11-03T17:13:50.106 INFO:tasks.workunit.client.1.smithi115.stdout:6/102: chown d0/f6 146 1 2021-11-03T17:13:50.108 INFO:tasks.workunit.client.1.smithi115.stdout:5/175: dwrite d1a/f1f [0,4194304] 0 2021-11-03T17:13:50.108 INFO:tasks.workunit.client.1.smithi115.stdout:5/176: readlink l14 0 2021-11-03T17:13:50.108 INFO:tasks.workunit.client.1.smithi115.stdout:5/177: readlink l16 0 2021-11-03T17:13:50.109 INFO:tasks.workunit.client.1.smithi115.stdout:6/103: rmdir d0 39 2021-11-03T17:13:50.109 INFO:tasks.workunit.client.1.smithi115.stdout:6/104: read - d0/f17 zero size 2021-11-03T17:13:50.112 INFO:tasks.workunit.client.1.smithi115.stdout:6/105: creat d0/f22 x:0 0 0 2021-11-03T17:13:50.112 INFO:tasks.workunit.client.1.smithi115.stdout:6/106: rename d0/d1 to d0/d1/d10/d13/d1c/d1e/d23 22 2021-11-03T17:13:50.120 INFO:tasks.workunit.client.1.smithi115.stdout:7/74: dwrite d3/f9 [8388608,4194304] 0 2021-11-03T17:13:50.120 INFO:tasks.workunit.client.1.smithi115.stdout:7/75: readlink d3/l10 0 2021-11-03T17:13:50.124 INFO:tasks.workunit.client.1.smithi115.stdout:4/119: truncate d7/dc/f12 541380 0 2021-11-03T17:13:50.124 INFO:tasks.workunit.client.1.smithi115.stdout:4/120: fdatasync d7/dc/d1a/d21/f22 0 2021-11-03T17:13:50.134 INFO:tasks.workunit.client.1.smithi115.stdout:8/137: dread d4/f1e [0,4194304] 0 2021-11-03T17:13:50.135 INFO:tasks.workunit.client.1.smithi115.stdout:5/178: dwrite d1a/f1f [0,4194304] 0 2021-11-03T17:13:50.136 INFO:tasks.workunit.client.1.smithi115.stdout:8/138: mkdir d4/d6/d18/d30 0 2021-11-03T17:13:50.137 INFO:tasks.workunit.client.1.smithi115.stdout:4/121: write d7/dc/f12 [1159735,82811] 0 2021-11-03T17:13:50.138 INFO:tasks.workunit.client.1.smithi115.stdout:4/122: rename d7/dc to d7/dc/d2a 22 2021-11-03T17:13:50.141 INFO:tasks.workunit.client.1.smithi115.stdout:8/139: dread d4/d6/d18/f2d [0,4194304] 0 2021-11-03T17:13:50.145 INFO:tasks.workunit.client.1.smithi115.stdout:7/76: dwrite d3/f9 [4194304,4194304] 0 2021-11-03T17:13:50.146 INFO:tasks.workunit.client.1.smithi115.stdout:7/77: creat d3/da/f12 x:0 0 0 2021-11-03T17:13:50.147 INFO:tasks.workunit.client.1.smithi115.stdout:7/78: creat d3/f13 x:0 0 0 2021-11-03T17:13:50.147 INFO:tasks.workunit.client.1.smithi115.stdout:7/79: readlink l0 0 2021-11-03T17:13:50.149 INFO:tasks.workunit.client.1.smithi115.stdout:6/107: dread d0/d1/d10/f16 [0,4194304] 0 2021-11-03T17:13:50.150 INFO:tasks.workunit.client.1.smithi115.stdout:6/108: dread - d0/f22 zero size 2021-11-03T17:13:50.150 INFO:tasks.workunit.client.1.smithi115.stdout:7/80: rename d3/da/f12 to d3/f14 0 2021-11-03T17:13:50.157 INFO:tasks.workunit.client.1.smithi115.stdout:4/123: truncate d7/f8 704671 0 2021-11-03T17:13:50.158 INFO:tasks.workunit.client.1.smithi115.stdout:4/124: chown d7/dc/d1a/d21 1 1 2021-11-03T17:13:50.158 INFO:tasks.workunit.client.1.smithi115.stdout:4/125: readlink l1 0 2021-11-03T17:13:50.158 INFO:tasks.workunit.client.1.smithi115.stdout:5/179: dwrite d1a/d28/d34/f32 [0,4194304] 0 2021-11-03T17:13:50.159 INFO:tasks.workunit.client.1.smithi115.stdout:4/126: write d7/f8 [1099364,21798] 0 2021-11-03T17:13:50.162 INFO:tasks.workunit.client.1.smithi115.stdout:7/81: getdents d3 0 2021-11-03T17:13:50.164 INFO:tasks.workunit.client.1.smithi115.stdout:5/180: creat d1a/f3c x:0 0 0 2021-11-03T17:13:50.168 INFO:tasks.workunit.client.1.smithi115.stdout:5/181: write d1a/f1d [428438,92063] 0 2021-11-03T17:13:50.169 INFO:tasks.workunit.client.1.smithi115.stdout:8/140: dwrite d4/f5 [0,4194304] 0 2021-11-03T17:13:50.170 INFO:tasks.workunit.client.1.smithi115.stdout:5/182: rename d1a/d23/d35 to d1a/d3d 0 2021-11-03T17:13:50.171 INFO:tasks.workunit.client.1.smithi115.stdout:5/183: mknod d1a/d23/d30/c3e 1 2021-11-03T17:13:50.173 INFO:tasks.workunit.client.1.smithi115.stdout:8/141: creat d4/d6/d18/d30/f31 x:0 0 0 2021-11-03T17:13:50.181 INFO:tasks.workunit.client.1.smithi115.stdout:5/184: dread fe [0,4194304] 0 2021-11-03T17:13:50.181 INFO:tasks.workunit.client.1.smithi115.stdout:5/185: chown d1a/f1e 17457 1 2021-11-03T17:13:50.181 INFO:tasks.workunit.client.1.smithi115.stdout:7/82: dwrite d3/f14 [0,4194304] 0 2021-11-03T17:13:50.184 INFO:tasks.workunit.client.1.smithi115.stdout:5/186: dread d1a/d3d/f3b [0,4194304] 0 2021-11-03T17:13:50.185 INFO:tasks.workunit.client.1.smithi115.stdout:5/187: chown d1a/l22 0 1 2021-11-03T17:13:50.187 INFO:tasks.workunit.client.1.smithi115.stdout:5/188: rename d1a/d23/d30 to d1a/d3f 0 2021-11-03T17:13:50.187 INFO:tasks.workunit.client.1.smithi115.stdout:5/189: readlink d1a/d28/l2e 0 2021-11-03T17:13:50.191 INFO:tasks.workunit.client.1.smithi115.stdout:7/83: dread d3/f14 [0,4194304] 0 2021-11-03T17:13:50.192 INFO:tasks.workunit.client.1.smithi115.stdout:7/84: creat d3/da/f15 x:0 0 0 2021-11-03T17:13:50.192 INFO:tasks.workunit.client.1.smithi115.stdout:7/85: mknod d3/da/c16 1 2021-11-03T17:13:50.193 INFO:tasks.workunit.client.1.smithi115.stdout:7/86: write d3/f13 [735007,24577] 0 2021-11-03T17:13:50.194 INFO:tasks.workunit.client.1.smithi115.stdout:4/127: dwrite d7/dc/f11 [0,4194304] 0 2021-11-03T17:13:50.196 INFO:tasks.workunit.client.1.smithi115.stdout:4/128: mkdir d7/d2b 0 2021-11-03T17:13:50.196 INFO:tasks.workunit.client.1.smithi115.stdout:4/129: getdents d7/d2b 0 2021-11-03T17:13:50.197 INFO:tasks.workunit.client.1.smithi115.stdout:8/142: dwrite d4/f28 [0,4194304] 0 2021-11-03T17:13:50.197 INFO:tasks.workunit.client.1.smithi115.stdout:8/143: readlink d4/d6/l24 0 2021-11-03T17:13:50.198 INFO:tasks.workunit.client.1.smithi115.stdout:4/130: write d7/dc/f11 [4023477,127760] 0 2021-11-03T17:13:50.201 INFO:tasks.workunit.client.1.smithi115.stdout:8/144: creat d4/d6/d18/d30/f32 x:0 0 0 2021-11-03T17:13:50.203 INFO:tasks.workunit.client.1.smithi115.stdout:4/131: rename d7/l27 to d7/dc/d1a/l2c 0 2021-11-03T17:13:50.203 INFO:tasks.workunit.client.1.smithi115.stdout:4/132: mknod d7/d2b/c2d 1 2021-11-03T17:13:50.203 INFO:tasks.workunit.client.1.smithi115.stdout:4/133: stat d7 0 2021-11-03T17:13:50.203 INFO:tasks.workunit.client.1.smithi115.stdout:8/145: mkdir d4/d6/d18/d30/d33 0 2021-11-03T17:13:50.208 INFO:tasks.workunit.client.1.smithi115.stdout:8/146: write d4/f5 [483152,60352] 0 2021-11-03T17:13:50.208 INFO:tasks.workunit.client.1.smithi115.stdout:7/87: dwrite d3/fc [0,4194304] 0 2021-11-03T17:13:50.211 INFO:tasks.workunit.client.1.smithi115.stdout:7/88: unlink d3/fc 0 2021-11-03T17:13:50.212 INFO:tasks.workunit.client.1.smithi115.stdout:7/89: symlink d3/l17 0 2021-11-03T17:13:50.220 INFO:tasks.workunit.client.1.smithi115.stdout:7/90: write d3/f9 [12042264,24675] 0 2021-11-03T17:13:50.221 INFO:tasks.workunit.client.1.smithi115.stdout:7/91: mkdir d3/da/d18 0 2021-11-03T17:13:50.224 INFO:tasks.workunit.client.1.smithi115.stdout:7/92: dread d3/f14 [0,4194304] 0 2021-11-03T17:13:50.228 INFO:tasks.workunit.client.1.smithi115.stdout:4/134: dwrite d7/dc/f13 [4194304,4194304] 0 2021-11-03T17:13:50.229 INFO:tasks.workunit.client.1.smithi115.stdout:8/147: dwrite d4/d6/f16 [0,4194304] 0 2021-11-03T17:13:50.230 INFO:tasks.workunit.client.1.smithi115.stdout:4/135: creat d7/dc/d1a/f2e x:0 0 0 2021-11-03T17:13:50.231 INFO:tasks.workunit.client.1.smithi115.stdout:8/148: creat d4/d6/d18/d30/f34 x:0 0 0 2021-11-03T17:13:50.233 INFO:tasks.workunit.client.1.smithi115.stdout:4/136: unlink d7/f8 0 2021-11-03T17:13:50.250 INFO:tasks.workunit.client.1.smithi115.stdout:4/137: dwrite d7/dc/d1a/d21/f22 [0,4194304] 0 2021-11-03T17:13:50.251 INFO:tasks.workunit.client.1.smithi115.stdout:4/138: write d7/dc/f13 [1486726,128181] 0 2021-11-03T17:13:50.252 INFO:tasks.workunit.client.1.smithi115.stdout:4/139: mknod d7/dc/d1a/d21/c2f 1 2021-11-03T17:13:50.252 INFO:tasks.workunit.client.1.smithi115.stdout:4/140: stat d7/dc/d1a/f1c 0 2021-11-03T17:13:50.253 INFO:tasks.workunit.client.1.smithi115.stdout:4/141: creat d7/f30 x:0 0 0 2021-11-03T17:13:50.253 INFO:tasks.workunit.client.1.smithi115.stdout:4/142: chown d7/lb 7893 1 2021-11-03T17:13:50.254 INFO:tasks.workunit.client.1.smithi115.stdout:4/143: link d7/l16 d7/dc/d1a/d21/l31 0 2021-11-03T17:13:50.254 INFO:tasks.workunit.client.1.smithi115.stdout:4/144: write d7/dc/d1a/f2e [691148,94191] 0 2021-11-03T17:13:50.255 INFO:tasks.workunit.client.1.smithi115.stdout:4/145: mkdir d7/dc/d1a/d21/d32 0 2021-11-03T17:13:50.255 INFO:tasks.workunit.client.1.smithi115.stdout:4/146: readlink d7/dc/d1a/l1e 0 2021-11-03T17:13:50.256 INFO:tasks.workunit.client.1.smithi115.stdout:4/147: read d7/dc/f12 [453736,14172] 0 2021-11-03T17:13:50.256 INFO:tasks.workunit.client.1.smithi115.stdout:4/148: getdents d7/dc 0 2021-11-03T17:13:50.257 INFO:tasks.workunit.client.1.smithi115.stdout:4/149: symlink d7/dc/l33 0 2021-11-03T17:13:50.258 INFO:tasks.workunit.client.1.smithi115.stdout:4/150: symlink d7/dc/l34 0 2021-11-03T17:13:50.260 INFO:tasks.workunit.client.1.smithi115.stdout:4/151: unlink d7/l16 0 2021-11-03T17:13:50.260 INFO:tasks.workunit.client.1.smithi115.stdout:7/93: dwrite d3/f13 [0,4194304] 0 2021-11-03T17:13:50.262 INFO:tasks.workunit.client.1.smithi115.stdout:7/94: truncate d3/f14 4123888 0 2021-11-03T17:13:50.263 INFO:tasks.workunit.client.1.smithi115.stdout:7/95: mknod d3/da/c19 1 2021-11-03T17:13:50.263 INFO:tasks.workunit.client.1.smithi115.stdout:7/96: rename d3 to d3/da/d18/d1a 22 2021-11-03T17:13:50.264 INFO:tasks.workunit.client.1.smithi115.stdout:4/152: link d7/l26 d7/dc/l35 0 2021-11-03T17:13:50.267 INFO:tasks.workunit.client.1.smithi115.stdout:8/149: dwrite d4/d6/d18/d30/f34 [0,4194304] 0 2021-11-03T17:13:50.268 INFO:tasks.workunit.client.1.smithi115.stdout:8/150: read d4/d6/d18/f2d [2142987,121728] 0 2021-11-03T17:13:50.270 INFO:tasks.workunit.client.1.smithi115.stdout:8/151: creat d4/d6/d18/d30/f35 x:0 0 0 2021-11-03T17:13:50.292 INFO:tasks.workunit.client.1.smithi115.stdout:4/153: dwrite d7/dc/f11 [4194304,4194304] 0 2021-11-03T17:13:50.301 INFO:tasks.workunit.client.1.smithi115.stdout:7/97: dwrite d3/f14 [4194304,4194304] 0 2021-11-03T17:13:50.302 INFO:tasks.workunit.client.1.smithi115.stdout:7/98: creat d3/da/f1b x:0 0 0 2021-11-03T17:13:50.302 INFO:tasks.workunit.client.1.smithi115.stdout:8/152: dwrite d4/d6/d18/d30/f32 [0,4194304] 0 2021-11-03T17:13:50.303 INFO:tasks.workunit.client.1.smithi115.stdout:7/99: creat d3/da/d18/f1c x:0 0 0 2021-11-03T17:13:50.303 INFO:tasks.workunit.client.1.smithi115.stdout:7/100: chown d3/da/d18/f1c 52 1 2021-11-03T17:13:50.304 INFO:tasks.workunit.client.1.smithi115.stdout:7/101: chown d3/l6 0 1 2021-11-03T17:13:50.306 INFO:tasks.workunit.client.1.smithi115.stdout:8/153: fsync d4/fd 0 2021-11-03T17:13:50.306 INFO:tasks.workunit.client.1.smithi115.stdout:8/154: write d4/d6/d18/d30/f35 [694270,11514] 0 2021-11-03T17:13:50.307 INFO:tasks.workunit.client.1.smithi115.stdout:7/102: rename d3/l17 to d3/da/l1d 0 2021-11-03T17:13:50.309 INFO:tasks.workunit.client.1.smithi115.stdout:8/155: rename d4/f5 to d4/d6/d18/d30/f36 0 2021-11-03T17:13:50.310 INFO:tasks.workunit.client.1.smithi115.stdout:8/156: write d4/f1e [2227976,58762] 0 2021-11-03T17:13:50.310 INFO:tasks.workunit.client.1.smithi115.stdout:7/103: creat d3/f1e x:0 0 0 2021-11-03T17:13:50.311 INFO:tasks.workunit.client.1.smithi115.stdout:8/157: write d4/d6/d18/f23 [1659881,71936] 0 2021-11-03T17:13:50.312 INFO:tasks.workunit.client.1.smithi115.stdout:8/158: getdents d4/d6/d18/d30/d33 0 2021-11-03T17:13:50.312 INFO:tasks.workunit.client.1.smithi115.stdout:8/159: dread - d4/f26 zero size 2021-11-03T17:13:50.312 INFO:tasks.workunit.client.1.smithi115.stdout:8/160: truncate d4/d6/d18/d30/f35 1686013 0 2021-11-03T17:13:50.312 INFO:tasks.workunit.client.1.smithi115.stdout:8/161: fdatasync d4/d6/d18/d30/f31 0 2021-11-03T17:13:50.320 INFO:tasks.workunit.client.1.smithi115.stdout:4/154: dwrite d7/dc/f11 [4194304,4194304] 0 2021-11-03T17:13:50.322 INFO:tasks.workunit.client.1.smithi115.stdout:4/155: mkdir d7/dc/d36 0 2021-11-03T17:13:50.323 INFO:tasks.workunit.client.1.smithi115.stdout:4/156: mkdir d7/dc/d1a/d21/d37 0 2021-11-03T17:13:50.324 INFO:tasks.workunit.client.1.smithi115.stdout:4/157: stat d7/dc/d1a/l24 0 2021-11-03T17:13:50.325 INFO:tasks.workunit.client.1.smithi115.stdout:4/158: link d7/dc/d1a/f1c d7/dc/d1a/f38 0 2021-11-03T17:13:50.326 INFO:tasks.workunit.client.1.smithi115.stdout:4/159: link d7/l9 d7/dc/d36/l39 0 2021-11-03T17:13:50.326 INFO:tasks.workunit.client.1.smithi115.stdout:4/160: write d7/dc/d1a/f2e [177812,108673] 0 2021-11-03T17:13:50.327 INFO:tasks.workunit.client.1.smithi115.stdout:4/161: creat d7/dc/d1a/d21/d32/f3a x:0 0 0 2021-11-03T17:13:50.328 INFO:tasks.workunit.client.1.smithi115.stdout:4/162: truncate d7/dc/d1a/f1c 23062 0 2021-11-03T17:13:50.330 INFO:tasks.workunit.client.1.smithi115.stdout:4/163: rename d7/dc/d1a/l24 to d7/dc/d36/l3b 0 2021-11-03T17:13:50.331 INFO:tasks.workunit.client.1.smithi115.stdout:4/164: stat d7/d2b 0 2021-11-03T17:13:50.336 INFO:tasks.workunit.client.1.smithi115.stdout:7/104: dwrite d3/da/f1b [0,4194304] 0 2021-11-03T17:13:50.336 INFO:tasks.workunit.client.1.smithi115.stdout:7/105: write d3/f1e [576764,30387] 0 2021-11-03T17:13:50.336 INFO:tasks.workunit.client.1.smithi115.stdout:7/106: readlink l0 0 2021-11-03T17:13:50.337 INFO:tasks.workunit.client.1.smithi115.stdout:7/107: readlink d3/l7 0 2021-11-03T17:13:50.338 INFO:tasks.workunit.client.1.smithi115.stdout:8/162: dwrite d4/d6/f2b [0,4194304] 0 2021-11-03T17:13:50.338 INFO:tasks.workunit.client.1.smithi115.stdout:8/163: mknod d4/d6/d18/c37 1 2021-11-03T17:13:50.345 INFO:tasks.workunit.client.1.smithi115.stdout:8/164: dread d4/d6/f16 [0,4194304] 0 2021-11-03T17:13:50.345 INFO:tasks.workunit.client.1.smithi115.stdout:8/165: getdents d4/d6/d18/d30/d33 0 2021-11-03T17:13:50.347 INFO:tasks.workunit.client.1.smithi115.stdout:8/166: mkdir d4/d38 0 2021-11-03T17:13:50.369 INFO:tasks.workunit.client.1.smithi115.stdout:8/167: dwrite d4/d6/d18/f23 [0,4194304] 0 2021-11-03T17:13:50.391 INFO:tasks.workunit.client.1.smithi115.stdout:8/168: dwrite d4/d6/f2b [0,4194304] 0 2021-11-03T17:13:50.398 INFO:tasks.workunit.client.1.smithi115.stdout:8/169: dread d4/fa [0,4194304] 0 2021-11-03T17:13:50.399 INFO:tasks.workunit.client.1.smithi115.stdout:8/170: symlink d4/d6/d18/d30/l39 0 2021-11-03T17:13:50.401 INFO:tasks.workunit.client.1.smithi115.stdout:8/171: rmdir d4/d38 0 2021-11-03T17:13:50.403 INFO:tasks.workunit.client.1.smithi115.stdout:8/172: dread d4/d6/d18/d30/f35 [0,4194304] 0 2021-11-03T17:13:50.404 INFO:tasks.workunit.client.1.smithi115.stdout:8/173: chown d4/d6/f2b 558697 1 2021-11-03T17:13:50.410 INFO:tasks.workunit.client.1.smithi115.stdout:8/174: dread d4/d6/d18/d30/f36 [0,4194304] 0 2021-11-03T17:13:50.412 INFO:tasks.workunit.client.1.smithi115.stdout:8/175: rename d4/f1e to d4/d6/d18/d30/f3a 0 2021-11-03T17:13:50.419 INFO:tasks.workunit.client.1.smithi115.stdout:8/176: dread d4/d6/f2b [0,4194304] 0 2021-11-03T17:13:50.423 INFO:tasks.workunit.client.1.smithi115.stdout:8/177: dread d4/d6/d18/d30/f36 [0,4194304] 0 2021-11-03T17:13:50.423 INFO:tasks.workunit.client.1.smithi115.stdout:8/178: mknod d4/d6/d18/d30/c3b 1 2021-11-03T17:13:50.429 INFO:tasks.workunit.client.1.smithi115.stdout:8/179: write d4/d6/d18/d30/f34 [2742149,30648] 0 2021-11-03T17:13:50.451 INFO:tasks.workunit.client.1.smithi115.stdout:8/180: dwrite d4/d6/d18/d30/f31 [0,4194304] 0 2021-11-03T17:13:50.454 INFO:tasks.workunit.client.1.smithi115.stdout:8/181: fdatasync d4/d6/d18/d30/f34 0 2021-11-03T17:13:50.456 INFO:tasks.workunit.client.1.smithi115.stdout:8/182: dread d4/d6/d18/d30/f35 [0,4194304] 0 2021-11-03T17:13:50.457 INFO:tasks.workunit.client.1.smithi115.stdout:8/183: creat d4/d6/f3c x:0 0 0 2021-11-03T17:13:50.459 INFO:tasks.workunit.client.1.smithi115.stdout:8/184: link d4/d6/d18/d30/f31 d4/d6/d18/f3d 0 2021-11-03T17:13:50.460 INFO:tasks.workunit.client.1.smithi115.stdout:8/185: symlink d4/d6/l3e 0 2021-11-03T17:13:50.461 INFO:tasks.workunit.client.1.smithi115.stdout:8/186: mkdir d4/d6/d3f 0 2021-11-03T17:13:50.462 INFO:tasks.workunit.client.1.smithi115.stdout:8/187: symlink d4/d6/d18/d30/d33/l40 0 2021-11-03T17:13:50.462 INFO:tasks.workunit.client.1.smithi115.stdout:8/188: stat d4/l7 0 2021-11-03T17:13:50.485 INFO:tasks.workunit.client.1.smithi115.stdout:8/189: dwrite d4/d6/d18/f23 [4194304,4194304] 0 2021-11-03T17:13:50.491 INFO:tasks.workunit.client.1.smithi115.stdout:8/190: getdents d4/d6 0 2021-11-03T17:13:50.494 INFO:tasks.workunit.client.1.smithi115.stdout:8/191: rename d4/d6/d18/d30/l39 to d4/d6/l41 0 2021-11-03T17:13:50.494 INFO:tasks.workunit.client.1.smithi115.stdout:8/192: rmdir d4 39 2021-11-03T17:13:50.495 INFO:tasks.workunit.client.1.smithi115.stdout:8/193: creat d4/d6/f42 x:0 0 0 2021-11-03T17:13:50.497 INFO:tasks.workunit.client.1.smithi115.stdout:8/194: read d4/d6/f8 [4258958,98782] 0 2021-11-03T17:13:50.499 INFO:tasks.workunit.client.1.smithi115.stdout:8/195: unlink d4/d6/le 0 2021-11-03T17:13:50.521 INFO:tasks.workunit.client.1.smithi115.stdout:8/196: dwrite d4/d6/d18/d30/f34 [0,4194304] 0 2021-11-03T17:13:50.522 INFO:tasks.workunit.client.1.smithi115.stdout:8/197: mkdir d4/d6/d3f/d43 0 2021-11-03T17:13:50.540 INFO:tasks.workunit.client.1.smithi115.stdout:8/198: dwrite d4/d6/d18/d30/f3a [0,4194304] 0 2021-11-03T17:13:50.540 INFO:tasks.workunit.client.1.smithi115.stdout:8/199: chown d4/fd 54 1 2021-11-03T17:13:50.541 INFO:tasks.workunit.client.1.smithi115.stdout:8/200: mkdir d4/d6/d18/d44 0 2021-11-03T17:13:50.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:50 smithi115 conmon[34670]: cluster 2021-11-03T17:13:50.600391+0000 mgr.smithi027.ufhhvb (mgr 2021-11-03T17:13:50.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:50 smithi115 conmon[34670]: .14162) 292 : cluster [DBG] pgmap v242: 65 pgs: 65 active+clean; 423 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 28 MiB/s wr, 626 op/s 2021-11-03T17:13:51.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:50 smithi027 conmon[32322]: cluster 2021-11-03T17:13:50.600391+0000 mgr.smithi027.ufhhvb (mgr 2021-11-03T17:13:51.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:50 smithi027 conmon[32322]: .14162) 292 : cluster [DBG] pgmap v242: 65 pgs: 65 active+clean; 423 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 28 MiB/s wr, 626 op/s 2021-11-03T17:13:52.837 INFO:tasks.workunit.client.0.smithi027.stdout:3/17: unlink d1/l2 0 2021-11-03T17:13:53.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:53 smithi027 conmon[32322]: cluster 2021-11-03T17:13:52 2021-11-03T17:13:53.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:53 smithi027 conmon[32322]: .600772+0000 mgr.smithi027.ufhhvb (mgr.14162) 293 : cluster [DBG] pgmap v243: 65 pgs: 65 active+clean; 489 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 34 MiB/s wr, 832 op/s 2021-11-03T17:13:53.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:53 smithi027 conmon[32322]: audit 2021-11-03T17:13:52.753699 2021-11-03T17:13:53.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:53 smithi027 conmon[32322]: +0000 mon.smithi027 (mon.0) 564 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:13:53.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:53 smithi027 conmon[32322]: audit 2021-11- 2021-11-03T17:13:53.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:53 smithi027 conmon[32322]: 03T17:13:52.753901+0000 mon.smithi027 (mon.0) 565 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:13:53.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:53 smithi027 conmon[32322]: 2021-11-03T17:13:53.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:53 smithi115 conmon[34670]: cluster 2021-11-03T17:13:52.600772+0000 mgr.smithi027.ufhhvb (mgr.14162) 293 : 2021-11-03T17:13:53.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:53 smithi115 conmon[34670]: cluster [DBG] pgmap v243: 65 pgs: 65 active+clean; 489 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 34 MiB/s wr, 832 op/s 2021-11-03T17:13:53.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:53 smithi115 conmon[34670]: audit 2021-11-03T17:13:52.753699+0000 mon.smithi027 (mon.0) 564 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:13:53.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:53 smithi115 conmon[34670]: audit 2021-11-03T17:13: 2021-11-03T17:13:53.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:53 smithi115 conmon[34670]: 52.753901+0000 mon.smithi027 (mon.0) 565 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:13:53.844 INFO:tasks.workunit.client.0.smithi027.stdout:9/3: symlink l0 0 2021-11-03T17:13:54.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:54 smithi115 conmon[34670]: cluster 2021-11-03T17:13:54.601099+0000 mgr.smithi027.ufhhvb (mgr.14162) 2021-11-03T17:13:54.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:54 smithi115 conmon[34670]: 294 : cluster [DBG] pgmap v244: 65 pgs: 65 active+clean; 497 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 32 MiB/s wr, 810 op/s 2021-11-03T17:13:55.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:54 smithi027 conmon[32322]: cluster 2021-11-03T17:13:54.601099+0000 mgr.smithi027.ufhhvb 2021-11-03T17:13:55.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:54 smithi027 conmon[32322]: (mgr.14162) 294 : cluster [DBG] pgmap v244: 65 pgs: 65 active+clean; 497 MiB data, 1.6 GiB used, 535 GiB / 536 GiB avail; 16 MiB/s rd, 32 MiB/s wr, 810 op/s 2021-11-03T17:13:55.247 INFO:tasks.workunit.client.0.smithi027.stdout:4/5: creat f2 x:0 0 0 2021-11-03T17:13:55.248 INFO:tasks.workunit.client.0.smithi027.stdout:4/6: mknod c3 1 2021-11-03T17:13:55.248 INFO:tasks.workunit.client.0.smithi027.stdout:0/18: mkdir d9 0 2021-11-03T17:13:55.249 INFO:tasks.workunit.client.0.smithi027.stdout:1/11: rename l0 to l3 0 2021-11-03T17:13:55.249 INFO:tasks.workunit.client.0.smithi027.stdout:1/12: rmdir - no directory 2021-11-03T17:13:55.249 INFO:tasks.workunit.client.0.smithi027.stdout:1/13: chown l3 3056489 1 2021-11-03T17:13:55.250 INFO:tasks.workunit.client.0.smithi027.stdout:1/14: dread f2 [0,4194304] 0 2021-11-03T17:13:55.252 INFO:tasks.workunit.client.0.smithi027.stdout:5/9: mkdir d2 0 2021-11-03T17:13:55.253 INFO:tasks.workunit.client.0.smithi027.stdout:5/10: read f0 [3908768,88822] 0 2021-11-03T17:13:55.256 INFO:tasks.workunit.client.0.smithi027.stdout:3/18: symlink d1/l4 0 2021-11-03T17:13:56.273 INFO:tasks.workunit.client.0.smithi027.stdout:9/4: symlink l1 0 2021-11-03T17:13:56.275 INFO:tasks.workunit.client.0.smithi027.stdout:6/12: link f2 f3 0 2021-11-03T17:13:56.275 INFO:tasks.workunit.client.0.smithi027.stdout:6/13: mknod c4 1 2021-11-03T17:13:56.276 INFO:tasks.workunit.client.0.smithi027.stdout:8/21: rename f5 to f6 0 2021-11-03T17:13:56.277 INFO:tasks.workunit.client.0.smithi027.stdout:4/7: rename f0 to f4 0 2021-11-03T17:13:56.278 INFO:tasks.workunit.client.0.smithi027.stdout:0/19: mkdir d9/da 0 2021-11-03T17:13:56.279 INFO:tasks.workunit.client.0.smithi027.stdout:0/20: read - f6 zero size 2021-11-03T17:13:56.279 INFO:tasks.workunit.client.0.smithi027.stdout:1/15: symlink l4 0 2021-11-03T17:13:56.281 INFO:tasks.workunit.client.0.smithi027.stdout:3/19: rename d1/l3 to d1/l5 0 2021-11-03T17:13:56.285 INFO:tasks.workunit.client.0.smithi027.stdout:9/5: symlink l2 0 2021-11-03T17:13:56.285 INFO:tasks.workunit.client.0.smithi027.stdout:9/6: fdatasync - no filename 2021-11-03T17:13:56.286 INFO:tasks.workunit.client.0.smithi027.stdout:9/7: chown l0 3186 1 2021-11-03T17:13:56.286 INFO:tasks.workunit.client.0.smithi027.stdout:9/8: write - no filename 2021-11-03T17:13:56.304 INFO:tasks.workunit.client.0.smithi027.stdout:3/20: dwrite f0 [0,4194304] 0 2021-11-03T17:13:56.394 INFO:tasks.workunit.client.0.smithi027.stdout:6/14: truncate f0 1932431 0 2021-11-03T17:13:56.395 INFO:tasks.workunit.client.0.smithi027.stdout:8/22: mkdir d7 0 2021-11-03T17:13:56.397 INFO:tasks.workunit.client.0.smithi027.stdout:8/23: dread f6 [0,4194304] 0 2021-11-03T17:13:56.400 INFO:tasks.workunit.client.0.smithi027.stdout:8/24: write f6 [480827,59192] 0 2021-11-03T17:13:56.471 INFO:tasks.workunit.client.0.smithi027.stdout:0/21: mkdir d9/da/db 0 2021-11-03T17:13:56.472 INFO:tasks.workunit.client.0.smithi027.stdout:0/22: mknod d9/cc 1 2021-11-03T17:13:56.472 INFO:tasks.workunit.client.0.smithi027.stdout:0/23: chown d9 3 1 2021-11-03T17:13:56.472 INFO:tasks.workunit.client.0.smithi027.stdout:0/24: readlink l2 0 2021-11-03T17:13:56.473 INFO:tasks.workunit.client.0.smithi027.stdout:1/16: creat f5 x:0 0 0 2021-11-03T17:13:56.474 INFO:tasks.workunit.client.0.smithi027.stdout:9/9: rename l2 to l3 0 2021-11-03T17:13:56.474 INFO:tasks.workunit.client.0.smithi027.stdout:9/10: chown l0 29261834 1 2021-11-03T17:13:56.474 INFO:tasks.workunit.client.0.smithi027.stdout:9/11: truncate - no filename 2021-11-03T17:13:56.477 INFO:tasks.workunit.client.0.smithi027.stdout:6/15: write f2 [572944,74567] 0 2021-11-03T17:13:56.480 INFO:tasks.workunit.client.0.smithi027.stdout:0/25: creat d9/da/fd x:0 0 0 2021-11-03T17:13:56.480 INFO:tasks.workunit.client.0.smithi027.stdout:0/26: chown d9/da 199562 1 2021-11-03T17:13:56.481 INFO:tasks.workunit.client.0.smithi027.stdout:0/27: chown l7 8117 1 2021-11-03T17:13:56.481 INFO:tasks.workunit.client.0.smithi027.stdout:0/28: stat f6 0 2021-11-03T17:13:56.482 INFO:tasks.workunit.client.0.smithi027.stdout:5/11: truncate f0 3047178 0 2021-11-03T17:13:56.492 INFO:tasks.workunit.client.0.smithi027.stdout:9/12: symlink l4 0 2021-11-03T17:13:56.495 INFO:tasks.workunit.client.0.smithi027.stdout:1/17: dwrite f5 [0,4194304] 0 2021-11-03T17:13:56.496 INFO:tasks.workunit.client.0.smithi027.stdout:0/29: unlink f6 0 2021-11-03T17:13:56.497 INFO:tasks.workunit.client.0.smithi027.stdout:4/8: write f4 [1415532,28108] 0 2021-11-03T17:13:56.497 INFO:tasks.workunit.client.0.smithi027.stdout:1/18: dread f2 [0,4194304] 0 2021-11-03T17:13:56.500 INFO:tasks.workunit.client.0.smithi027.stdout:8/25: link f6 d7/f8 0 2021-11-03T17:13:56.500 INFO:tasks.workunit.client.0.smithi027.stdout:8/26: rename d7 to d7/d9 22 2021-11-03T17:13:56.500 INFO:tasks.workunit.client.0.smithi027.stdout:8/27: rename d7 to d7/da 22 2021-11-03T17:13:56.505 INFO:tasks.workunit.client.0.smithi027.stdout:9/13: rename l0 to l5 0 2021-11-03T17:13:56.505 INFO:tasks.workunit.client.0.smithi027.stdout:9/14: mknod c6 1 2021-11-03T17:13:56.506 INFO:tasks.workunit.client.0.smithi027.stdout:3/21: dwrite f0 [4194304,4194304] 0 2021-11-03T17:13:56.507 INFO:tasks.workunit.client.0.smithi027.stdout:6/16: unlink f3 0 2021-11-03T17:13:56.507 INFO:tasks.workunit.client.0.smithi027.stdout:6/17: fdatasync f1 0 2021-11-03T17:13:56.511 INFO:tasks.workunit.client.0.smithi027.stdout:5/12: write f0 [2446479,92928] 0 2021-11-03T17:13:56.514 INFO:tasks.workunit.client.0.smithi027.stdout:0/30: creat d9/da/fe x:0 0 0 2021-11-03T17:13:56.514 INFO:tasks.workunit.client.0.smithi027.stdout:0/31: readlink l2 0 2021-11-03T17:13:56.514 INFO:tasks.workunit.client.0.smithi027.stdout:0/32: write d9/da/fd [872645,44726] 0 2021-11-03T17:13:56.516 INFO:tasks.workunit.client.0.smithi027.stdout:4/9: getdents . 0 2021-11-03T17:13:56.516 INFO:tasks.workunit.client.0.smithi027.stdout:4/10: readlink l1 0 2021-11-03T17:13:56.518 INFO:tasks.workunit.client.0.smithi027.stdout:1/19: mkdir d6 0 2021-11-03T17:13:56.521 INFO:tasks.workunit.client.0.smithi027.stdout:8/28: creat d7/fb x:0 0 0 2021-11-03T17:13:56.521 INFO:tasks.workunit.client.0.smithi027.stdout:8/29: readlink l4 0 2021-11-03T17:13:56.525 INFO:tasks.workunit.client.0.smithi027.stdout:9/15: rename l4 to l7 0 2021-11-03T17:13:56.526 INFO:tasks.workunit.client.0.smithi027.stdout:9/16: fsync - no filename 2021-11-03T17:13:56.526 INFO:tasks.workunit.client.0.smithi027.stdout:9/17: truncate - no filename 2021-11-03T17:13:56.527 INFO:tasks.workunit.client.0.smithi027.stdout:8/30: fdatasync f6 0 2021-11-03T17:13:56.527 INFO:tasks.workunit.client.0.smithi027.stdout:8/31: stat d7 0 2021-11-03T17:13:56.528 INFO:tasks.workunit.client.0.smithi027.stdout:3/22: creat d1/f6 x:0 0 0 2021-11-03T17:13:56.528 INFO:tasks.workunit.client.0.smithi027.stdout:3/23: read - d1/f6 zero size 2021-11-03T17:13:56.532 INFO:tasks.workunit.client.0.smithi027.stdout:0/33: rmdir d9/da/db 0 2021-11-03T17:13:56.532 INFO:tasks.workunit.client.0.smithi027.stdout:0/34: readlink l2 0 2021-11-03T17:13:56.534 INFO:tasks.workunit.client.0.smithi027.stdout:4/11: rename f2 to f5 0 2021-11-03T17:13:56.534 INFO:tasks.workunit.client.0.smithi027.stdout:4/12: write f5 [339095,52855] 0 2021-11-03T17:13:56.534 INFO:tasks.workunit.client.0.smithi027.stdout:4/13: rmdir - no directory 2021-11-03T17:13:56.535 INFO:tasks.workunit.client.0.smithi027.stdout:4/14: write f5 [546572,65883] 0 2021-11-03T17:13:56.535 INFO:tasks.workunit.client.0.smithi027.stdout:3/24: write f0 [4322152,53766] 0 2021-11-03T17:13:56.535 INFO:tasks.workunit.client.0.smithi027.stdout:9/18: creat f8 x:0 0 0 2021-11-03T17:13:56.537 INFO:tasks.workunit.client.0.smithi027.stdout:5/13: dwrite f0 [0,4194304] 0 2021-11-03T17:13:56.537 INFO:tasks.workunit.client.0.smithi027.stdout:1/20: dwrite f2 [0,4194304] 0 2021-11-03T17:13:56.539 INFO:tasks.workunit.client.0.smithi027.stdout:6/18: unlink f1 0 2021-11-03T17:13:56.539 INFO:tasks.workunit.client.0.smithi027.stdout:6/19: chown f0 121605620 1 2021-11-03T17:13:56.542 INFO:tasks.workunit.client.0.smithi027.stdout:4/15: write f4 [2310246,80878] 0 2021-11-03T17:13:56.543 INFO:tasks.workunit.client.0.smithi027.stdout:9/19: symlink l9 0 2021-11-03T17:13:56.543 INFO:tasks.workunit.client.0.smithi027.stdout:9/20: fdatasync f8 0 2021-11-03T17:13:56.544 INFO:tasks.workunit.client.0.smithi027.stdout:5/14: creat d2/f3 x:0 0 0 2021-11-03T17:13:56.545 INFO:tasks.workunit.client.0.smithi027.stdout:5/15: truncate d2/f3 681073 0 2021-11-03T17:13:56.547 INFO:tasks.workunit.client.0.smithi027.stdout:1/21: creat d6/f7 x:0 0 0 2021-11-03T17:13:56.548 INFO:tasks.workunit.client.0.smithi027.stdout:1/22: fsync f5 0 2021-11-03T17:13:56.548 INFO:tasks.workunit.client.0.smithi027.stdout:1/23: fdatasync f2 0 2021-11-03T17:13:56.548 INFO:tasks.workunit.client.0.smithi027.stdout:6/20: write f2 [1323202,118006] 0 2021-11-03T17:13:56.550 INFO:tasks.workunit.client.0.smithi027.stdout:9/21: unlink l1 0 2021-11-03T17:13:56.550 INFO:tasks.workunit.client.0.smithi027.stdout:9/22: readlink l5 0 2021-11-03T17:13:56.551 INFO:tasks.workunit.client.0.smithi027.stdout:0/35: getdents d9/da 0 2021-11-03T17:13:56.551 INFO:tasks.workunit.client.0.smithi027.stdout:0/36: mknod d9/cf 1 2021-11-03T17:13:56.555 INFO:tasks.workunit.client.0.smithi027.stdout:1/24: dread f5 [0,4194304] 0 2021-11-03T17:13:56.556 INFO:tasks.workunit.client.0.smithi027.stdout:5/16: mkdir d2/d4 0 2021-11-03T17:13:56.557 INFO:tasks.workunit.client.0.smithi027.stdout:6/21: link f2 f5 0 2021-11-03T17:13:56.559 INFO:tasks.workunit.client.0.smithi027.stdout:9/23: stat l5 0 2021-11-03T17:13:56.559 INFO:tasks.workunit.client.0.smithi027.stdout:9/24: dread - f8 zero size 2021-11-03T17:13:56.559 INFO:tasks.workunit.client.0.smithi027.stdout:9/25: truncate f8 407857 0 2021-11-03T17:13:56.560 INFO:tasks.workunit.client.0.smithi027.stdout:0/37: creat d9/da/f10 x:0 0 0 2021-11-03T17:13:56.561 INFO:tasks.workunit.client.0.smithi027.stdout:3/25: dread f0 [0,4194304] 0 2021-11-03T17:13:56.561 INFO:tasks.workunit.client.0.smithi027.stdout:8/32: dwrite d7/f8 [0,4194304] 0 2021-11-03T17:13:56.561 INFO:tasks.workunit.client.0.smithi027.stdout:3/26: chown d1/l4 1920 1 2021-11-03T17:13:56.562 INFO:tasks.workunit.client.0.smithi027.stdout:3/27: truncate d1/f6 7148 0 2021-11-03T17:13:56.562 INFO:tasks.workunit.client.0.smithi027.stdout:1/25: rename d6/f7 to d6/f8 0 2021-11-03T17:13:56.562 INFO:tasks.workunit.client.0.smithi027.stdout:3/28: write f0 [4924747,80059] 0 2021-11-03T17:13:56.563 INFO:tasks.workunit.client.0.smithi027.stdout:6/22: fdatasync f5 0 2021-11-03T17:13:56.564 INFO:tasks.workunit.client.0.smithi027.stdout:6/23: chown f0 63015 1 2021-11-03T17:13:56.564 INFO:tasks.workunit.client.0.smithi027.stdout:6/24: truncate f5 2267103 0 2021-11-03T17:13:56.565 INFO:tasks.workunit.client.0.smithi027.stdout:5/17: rename f0 to d2/d4/f5 0 2021-11-03T17:13:56.568 INFO:tasks.workunit.client.0.smithi027.stdout:9/26: rename l9 to la 0 2021-11-03T17:13:56.568 INFO:tasks.workunit.client.0.smithi027.stdout:9/27: write f8 [1340604,73597] 0 2021-11-03T17:13:56.575 INFO:tasks.workunit.client.0.smithi027.stdout:8/33: symlink d7/lc 0 2021-11-03T17:13:56.575 INFO:tasks.workunit.client.0.smithi027.stdout:8/34: chown d7/fb 132462 1 2021-11-03T17:13:56.578 INFO:tasks.workunit.client.0.smithi027.stdout:5/18: creat d2/d4/f6 x:0 0 0 2021-11-03T17:13:56.579 INFO:tasks.workunit.client.0.smithi027.stdout:5/19: read d2/d4/f5 [3903356,11654] 0 2021-11-03T17:13:56.579 INFO:tasks.workunit.client.0.smithi027.stdout:5/20: chown d2/d4/f6 16 1 2021-11-03T17:13:56.581 INFO:tasks.workunit.client.0.smithi027.stdout:1/26: rmdir d6 39 2021-11-03T17:13:56.581 INFO:tasks.workunit.client.0.smithi027.stdout:0/38: dwrite d9/da/f10 [0,4194304] 0 2021-11-03T17:13:56.581 INFO:tasks.workunit.client.0.smithi027.stdout:4/16: truncate f5 470716 0 2021-11-03T17:13:56.587 INFO:tasks.workunit.client.0.smithi027.stdout:8/35: creat d7/fd x:0 0 0 2021-11-03T17:13:56.588 INFO:tasks.workunit.client.0.smithi027.stdout:9/28: dread f8 [0,4194304] 0 2021-11-03T17:13:56.589 INFO:tasks.workunit.client.0.smithi027.stdout:3/29: dwrite d1/f6 [0,4194304] 0 2021-11-03T17:13:56.590 INFO:tasks.workunit.client.0.smithi027.stdout:3/30: chown f0 0 1 2021-11-03T17:13:56.590 INFO:tasks.workunit.client.0.smithi027.stdout:4/17: dread f5 [0,4194304] 0 2021-11-03T17:13:56.591 INFO:tasks.workunit.client.0.smithi027.stdout:5/21: getdents d2/d4 0 2021-11-03T17:13:56.592 INFO:tasks.workunit.client.0.smithi027.stdout:1/27: dread f2 [0,4194304] 0 2021-11-03T17:13:56.596 INFO:tasks.workunit.client.0.smithi027.stdout:0/39: fsync d9/da/fe 0 2021-11-03T17:13:56.599 INFO:tasks.workunit.client.0.smithi027.stdout:9/29: mkdir db 0 2021-11-03T17:13:56.601 INFO:tasks.workunit.client.0.smithi027.stdout:3/31: creat d1/f7 x:0 0 0 2021-11-03T17:13:56.601 INFO:tasks.workunit.client.0.smithi027.stdout:3/32: mknod d1/c8 1 2021-11-03T17:13:56.602 INFO:tasks.workunit.client.0.smithi027.stdout:5/22: creat d2/f7 x:0 0 0 2021-11-03T17:13:56.605 INFO:tasks.workunit.client.0.smithi027.stdout:1/28: rename l4 to d6/l9 0 2021-11-03T17:13:56.606 INFO:tasks.workunit.client.0.smithi027.stdout:8/36: truncate f6 25283 0 2021-11-03T17:13:56.607 INFO:tasks.workunit.client.0.smithi027.stdout:8/37: stat d7/f8 0 2021-11-03T17:13:56.608 INFO:tasks.workunit.client.0.smithi027.stdout:9/30: truncate f8 954267 0 2021-11-03T17:13:56.608 INFO:tasks.workunit.client.0.smithi027.stdout:9/31: mknod db/cc 1 2021-11-03T17:13:56.608 INFO:tasks.workunit.client.0.smithi027.stdout:9/32: stat f8 0 2021-11-03T17:13:56.609 INFO:tasks.workunit.client.0.smithi027.stdout:4/18: mkdir d6 0 2021-11-03T17:13:56.609 INFO:tasks.workunit.client.0.smithi027.stdout:4/19: rename d6 to d6/d7 22 2021-11-03T17:13:56.612 INFO:tasks.workunit.client.0.smithi027.stdout:1/29: unlink f5 0 2021-11-03T17:13:56.613 INFO:tasks.workunit.client.0.smithi027.stdout:1/30: chown d6 62964 1 2021-11-03T17:13:56.613 INFO:tasks.workunit.client.0.smithi027.stdout:1/31: fsync d6/f8 0 2021-11-03T17:13:56.614 INFO:tasks.workunit.client.0.smithi027.stdout:8/38: creat d7/fe x:0 0 0 2021-11-03T17:13:56.614 INFO:tasks.workunit.client.0.smithi027.stdout:8/39: readlink d7/lc 0 2021-11-03T17:13:56.615 INFO:tasks.workunit.client.0.smithi027.stdout:1/32: rename d6/f8 to d6/fa 0 2021-11-03T17:13:56.617 INFO:tasks.workunit.client.0.smithi027.stdout:1/33: symlink d6/lb 0 2021-11-03T17:13:56.620 INFO:tasks.workunit.client.0.smithi027.stdout:5/23: rmdir d2 39 2021-11-03T17:13:56.620 INFO:tasks.workunit.client.0.smithi027.stdout:5/24: write d2/f7 [995083,97200] 0 2021-11-03T17:13:56.622 INFO:tasks.workunit.client.0.smithi027.stdout:9/33: write f8 [1636105,24334] 0 2021-11-03T17:13:56.624 INFO:tasks.workunit.client.0.smithi027.stdout:5/25: symlink d2/l8 0 2021-11-03T17:13:56.626 INFO:tasks.workunit.client.0.smithi027.stdout:9/34: creat db/fd x:0 0 0 2021-11-03T17:13:56.626 INFO:tasks.workunit.client.0.smithi027.stdout:0/40: dwrite d9/da/f10 [0,4194304] 0 2021-11-03T17:13:56.626 INFO:tasks.workunit.client.0.smithi027.stdout:9/35: mknod db/ce 1 2021-11-03T17:13:56.631 INFO:tasks.workunit.client.0.smithi027.stdout:9/36: rename l3 to db/lf 0 2021-11-03T17:13:56.633 INFO:tasks.workunit.client.0.smithi027.stdout:4/20: dwrite f5 [0,4194304] 0 2021-11-03T17:13:56.633 INFO:tasks.workunit.client.0.smithi027.stdout:4/21: chown d6 4070 1 2021-11-03T17:13:56.636 INFO:tasks.workunit.client.0.smithi027.stdout:9/37: write f8 [54370,22730] 0 2021-11-03T17:13:56.636 INFO:tasks.workunit.client.0.smithi027.stdout:9/38: truncate db/fd 638208 0 2021-11-03T17:13:56.637 INFO:tasks.workunit.client.0.smithi027.stdout:4/22: creat d6/f8 x:0 0 0 2021-11-03T17:13:56.637 INFO:tasks.workunit.client.0.smithi027.stdout:4/23: write d6/f8 [490600,42493] 0 2021-11-03T17:13:56.638 INFO:tasks.workunit.client.0.smithi027.stdout:4/24: mkdir d6/d9 0 2021-11-03T17:13:56.639 INFO:tasks.workunit.client.0.smithi027.stdout:4/25: fdatasync f5 0 2021-11-03T17:13:56.639 INFO:tasks.workunit.client.0.smithi027.stdout:4/26: symlink d6/d9/la 0 2021-11-03T17:13:56.645 INFO:tasks.workunit.client.0.smithi027.stdout:4/27: write f5 [674020,62840] 0 2021-11-03T17:13:56.650 INFO:tasks.workunit.client.0.smithi027.stdout:5/26: dwrite d2/d4/f5 [0,4194304] 0 2021-11-03T17:13:56.650 INFO:tasks.workunit.client.0.smithi027.stdout:5/27: readlink d2/l8 0 2021-11-03T17:13:56.650 INFO:tasks.workunit.client.0.smithi027.stdout:5/28: mknod d2/d4/c9 1 2021-11-03T17:13:56.650 INFO:tasks.workunit.client.0.smithi027.stdout:8/40: dwrite d7/fb [0,4194304] 0 2021-11-03T17:13:56.651 INFO:tasks.workunit.client.0.smithi027.stdout:5/29: write d2/d4/f6 [354885,48392] 0 2021-11-03T17:13:56.651 INFO:tasks.workunit.client.0.smithi027.stdout:1/34: dwrite d6/fa [0,4194304] 0 2021-11-03T17:13:56.654 INFO:tasks.workunit.client.0.smithi027.stdout:1/35: dread f2 [0,4194304] 0 2021-11-03T17:13:56.655 INFO:tasks.workunit.client.0.smithi027.stdout:5/30: creat d2/d4/fa x:0 0 0 2021-11-03T17:13:56.656 INFO:tasks.workunit.client.0.smithi027.stdout:8/41: rmdir d7 39 2021-11-03T17:13:56.656 INFO:tasks.workunit.client.0.smithi027.stdout:8/42: write d7/fe [994672,53984] 0 2021-11-03T17:13:56.656 INFO:tasks.workunit.client.0.smithi027.stdout:8/43: write d7/f8 [460319,22266] 0 2021-11-03T17:13:56.657 INFO:tasks.workunit.client.0.smithi027.stdout:8/44: stat d7/fe 0 2021-11-03T17:13:56.657 INFO:tasks.workunit.client.0.smithi027.stdout:5/31: dread d2/f7 [0,4194304] 0 2021-11-03T17:13:56.660 INFO:tasks.workunit.client.0.smithi027.stdout:9/39: dread f8 [0,4194304] 0 2021-11-03T17:13:56.661 INFO:tasks.workunit.client.0.smithi027.stdout:0/41: truncate d9/da/f10 322618 0 2021-11-03T17:13:56.661 INFO:tasks.workunit.client.0.smithi027.stdout:0/42: write d9/da/fe [397499,107200] 0 2021-11-03T17:13:56.662 INFO:tasks.workunit.client.0.smithi027.stdout:8/45: dread d7/fb [0,4194304] 0 2021-11-03T17:13:56.673 INFO:tasks.workunit.client.0.smithi027.stdout:1/36: creat d6/fc x:0 0 0 2021-11-03T17:13:56.674 INFO:tasks.workunit.client.0.smithi027.stdout:1/37: chown d6 113 1 2021-11-03T17:13:56.675 INFO:tasks.workunit.client.0.smithi027.stdout:5/32: creat d2/d4/fb x:0 0 0 2021-11-03T17:13:56.675 INFO:tasks.workunit.client.0.smithi027.stdout:5/33: mknod d2/d4/cc 1 2021-11-03T17:13:56.675 INFO:tasks.workunit.client.0.smithi027.stdout:0/43: rmdir d9/da 39 2021-11-03T17:13:56.676 INFO:tasks.workunit.client.0.smithi027.stdout:0/44: write d9/da/f10 [593285,10519] 0 2021-11-03T17:13:56.677 INFO:tasks.workunit.client.0.smithi027.stdout:8/46: rmdir d7 39 2021-11-03T17:13:56.677 INFO:tasks.workunit.client.0.smithi027.stdout:8/47: mknod d7/cf 1 2021-11-03T17:13:56.678 INFO:tasks.workunit.client.0.smithi027.stdout:8/48: write d7/fe [1475998,59911] 0 2021-11-03T17:13:56.678 INFO:tasks.workunit.client.0.smithi027.stdout:9/40: dwrite db/fd [0,4194304] 0 2021-11-03T17:13:56.678 INFO:tasks.workunit.client.0.smithi027.stdout:9/41: mknod db/c10 1 2021-11-03T17:13:56.679 INFO:tasks.workunit.client.0.smithi027.stdout:1/38: creat d6/fd x:0 0 0 2021-11-03T17:13:56.679 INFO:tasks.workunit.client.0.smithi027.stdout:1/39: chown f2 34 1 2021-11-03T17:13:56.682 INFO:tasks.workunit.client.0.smithi027.stdout:5/34: getdents d2/d4 0 2021-11-03T17:13:56.685 INFO:tasks.workunit.client.0.smithi027.stdout:8/49: mkdir d7/d10 0 2021-11-03T17:13:56.688 INFO:tasks.workunit.client.0.smithi027.stdout:1/40: getdents d6 0 2021-11-03T17:13:56.690 INFO:tasks.workunit.client.0.smithi027.stdout:5/35: creat d2/fd x:0 0 0 2021-11-03T17:13:56.692 INFO:tasks.workunit.client.0.smithi027.stdout:8/50: symlink d7/l11 0 2021-11-03T17:13:56.694 INFO:tasks.workunit.client.0.smithi027.stdout:8/51: rename d7/fd to d7/d10/f12 0 2021-11-03T17:13:56.697 INFO:tasks.workunit.client.0.smithi027.stdout:8/52: rename d7/lc to d7/d10/l13 0 2021-11-03T17:13:56.697 INFO:tasks.workunit.client.0.smithi027.stdout:8/53: chown d7 7904664 1 2021-11-03T17:13:56.697 INFO:tasks.workunit.client.0.smithi027.stdout:5/36: write d2/d4/f5 [831197,122429] 0 2021-11-03T17:13:56.700 INFO:tasks.workunit.client.0.smithi027.stdout:5/37: rename d2/fd to d2/d4/fe 0 2021-11-03T17:13:56.702 INFO:tasks.workunit.client.0.smithi027.stdout:5/38: mkdir d2/d4/df 0 2021-11-03T17:13:56.703 INFO:tasks.workunit.client.0.smithi027.stdout:5/39: symlink d2/d4/l10 0 2021-11-03T17:13:56.705 INFO:tasks.workunit.client.0.smithi027.stdout:5/40: creat d2/d4/df/f11 x:0 0 0 2021-11-03T17:13:56.706 INFO:tasks.workunit.client.0.smithi027.stdout:5/41: mkdir d2/d12 0 2021-11-03T17:13:56.706 INFO:tasks.workunit.client.0.smithi027.stdout:5/42: write d2/d4/fa [114159,101407] 0 2021-11-03T17:13:56.707 INFO:tasks.workunit.client.0.smithi027.stdout:5/43: write d2/d4/df/f11 [318018,129204] 0 2021-11-03T17:13:56.707 INFO:tasks.workunit.client.0.smithi027.stdout:5/44: truncate d2/d4/fa 478357 0 2021-11-03T17:13:56.816 INFO:tasks.workunit.client.0.smithi027.stdout:7/20: sync 2021-11-03T17:13:56.816 INFO:tasks.workunit.client.0.smithi027.stdout:2/5: sync 2021-11-03T17:13:56.816 INFO:tasks.workunit.client.0.smithi027.stdout:2/6: write - no filename 2021-11-03T17:13:56.817 INFO:tasks.workunit.client.0.smithi027.stdout:2/7: dwrite - no filename 2021-11-03T17:13:56.817 INFO:tasks.workunit.client.0.smithi027.stdout:2/8: dwrite - no filename 2021-11-03T17:13:56.817 INFO:tasks.workunit.client.0.smithi027.stdout:7/21: mknod d0/d2/c3 1 2021-11-03T17:13:56.817 INFO:tasks.workunit.client.0.smithi027.stdout:7/22: stat d0/l1 0 2021-11-03T17:13:56.818 INFO:tasks.workunit.client.0.smithi027.stdout:7/23: mknod d0/c4 1 2021-11-03T17:13:56.818 INFO:tasks.workunit.client.0.smithi027.stdout:2/9: mkdir d0/d2 0 2021-11-03T17:13:56.818 INFO:tasks.workunit.client.0.smithi027.stdout:2/10: dread - no filename 2021-11-03T17:13:56.819 INFO:tasks.workunit.client.0.smithi027.stdout:7/24: creat d0/d2/f5 x:0 0 0 2021-11-03T17:13:56.820 INFO:tasks.workunit.client.0.smithi027.stdout:2/11: creat d0/d2/f3 x:0 0 0 2021-11-03T17:13:56.820 INFO:tasks.workunit.client.0.smithi027.stdout:2/12: fdatasync d0/d2/f3 0 2021-11-03T17:13:56.820 INFO:tasks.workunit.client.0.smithi027.stdout:2/13: truncate d0/d2/f3 947156 0 2021-11-03T17:13:56.822 INFO:tasks.workunit.client.0.smithi027.stdout:8/54: fsync d7/f8 0 2021-11-03T17:13:56.823 INFO:tasks.workunit.client.0.smithi027.stdout:8/55: chown d7/d10 0 1 2021-11-03T17:13:56.823 INFO:tasks.workunit.client.0.smithi027.stdout:9/42: dwrite db/fd [0,4194304] 0 2021-11-03T17:13:56.827 INFO:tasks.workunit.client.0.smithi027.stdout:0/45: dwrite d9/da/f10 [0,4194304] 0 2021-11-03T17:13:56.828 INFO:tasks.workunit.client.0.smithi027.stdout:9/43: stat l7 0 2021-11-03T17:13:56.834 INFO:tasks.workunit.client.0.smithi027.stdout:1/41: dwrite f2 [0,4194304] 0 2021-11-03T17:13:56.835 INFO:tasks.workunit.client.0.smithi027.stdout:0/46: stat d9/da/fd 0 2021-11-03T17:13:56.835 INFO:tasks.workunit.client.0.smithi027.stdout:0/47: readlink l2 0 2021-11-03T17:13:56.836 INFO:tasks.workunit.client.0.smithi027.stdout:9/44: truncate f8 1206099 0 2021-11-03T17:13:56.838 INFO:tasks.workunit.client.0.smithi027.stdout:1/42: mkdir d6/de 0 2021-11-03T17:13:56.839 INFO:tasks.workunit.client.0.smithi027.stdout:9/45: symlink db/l11 0 2021-11-03T17:13:56.843 INFO:tasks.workunit.client.0.smithi027.stdout:1/43: write f2 [4065208,111950] 0 2021-11-03T17:13:56.845 INFO:tasks.workunit.client.0.smithi027.stdout:5/45: dwrite d2/d4/fe [0,4194304] 0 2021-11-03T17:13:56.845 INFO:tasks.workunit.client.0.smithi027.stdout:7/25: dwrite d0/d2/f5 [0,4194304] 0 2021-11-03T17:13:56.849 INFO:tasks.workunit.client.0.smithi027.stdout:2/14: dwrite d0/d2/f3 [0,4194304] 0 2021-11-03T17:13:56.851 INFO:tasks.workunit.client.0.smithi027.stdout:8/56: rename d7/d10/l13 to d7/d10/l14 0 2021-11-03T17:13:56.851 INFO:tasks.workunit.client.0.smithi027.stdout:8/57: write f6 [787710,130962] 0 2021-11-03T17:13:56.854 INFO:tasks.workunit.client.0.smithi027.stdout:5/46: creat d2/d4/f13 x:0 0 0 2021-11-03T17:13:56.862 INFO:tasks.workunit.client.0.smithi027.stdout:5/47: dread d2/d4/f5 [0,4194304] 0 2021-11-03T17:13:56.862 INFO:tasks.workunit.client.0.smithi027.stdout:5/48: chown d2/d4 47252 1 2021-11-03T17:13:56.862 INFO:tasks.workunit.client.0.smithi027.stdout:5/49: rename d2 to d2/d12/d14 22 2021-11-03T17:13:56.863 INFO:tasks.workunit.client.0.smithi027.stdout:7/26: creat d0/d2/f6 x:0 0 0 2021-11-03T17:13:56.863 INFO:tasks.workunit.client.0.smithi027.stdout:7/27: dread - d0/d2/f6 zero size 2021-11-03T17:13:56.864 INFO:tasks.workunit.client.0.smithi027.stdout:7/28: write d0/d2/f6 [809076,61918] 0 2021-11-03T17:13:56.864 INFO:tasks.workunit.client.0.smithi027.stdout:2/15: fsync d0/d2/f3 0 2021-11-03T17:13:56.865 INFO:tasks.workunit.client.0.smithi027.stdout:0/48: dwrite d9/da/f10 [0,4194304] 0 2021-11-03T17:13:56.866 INFO:tasks.workunit.client.0.smithi027.stdout:1/44: dwrite d6/fc [0,4194304] 0 2021-11-03T17:13:56.875 INFO:tasks.workunit.client.0.smithi027.stdout:8/58: creat d7/d10/f15 x:0 0 0 2021-11-03T17:13:56.875 INFO:tasks.workunit.client.0.smithi027.stdout:8/59: fsync d7/d10/f12 0 2021-11-03T17:13:56.875 INFO:tasks.workunit.client.0.smithi027.stdout:8/60: truncate d7/d10/f15 791755 0 2021-11-03T17:13:56.876 INFO:tasks.workunit.client.0.smithi027.stdout:8/61: chown d7/d10/l14 10039 1 2021-11-03T17:13:56.876 INFO:tasks.workunit.client.0.smithi027.stdout:8/62: chown d7/d10/f12 1 1 2021-11-03T17:13:56.876 INFO:tasks.workunit.client.0.smithi027.stdout:8/63: stat d7/fe 0 2021-11-03T17:13:56.877 INFO:tasks.workunit.client.0.smithi027.stdout:7/29: symlink d0/d2/l7 0 2021-11-03T17:13:56.877 INFO:tasks.workunit.client.0.smithi027.stdout:7/30: mknod d0/d2/c8 1 2021-11-03T17:13:56.879 INFO:tasks.workunit.client.0.smithi027.stdout:0/49: rmdir d9/da 39 2021-11-03T17:13:56.885 INFO:tasks.workunit.client.0.smithi027.stdout:7/31: link d0/d2/l7 d0/d2/l9 0 2021-11-03T17:13:56.886 INFO:tasks.workunit.client.0.smithi027.stdout:2/16: link d0/d2/f3 d0/d2/f4 0 2021-11-03T17:13:56.892 INFO:tasks.workunit.client.0.smithi027.stdout:0/50: link l2 d9/da/l11 0 2021-11-03T17:13:56.892 INFO:tasks.workunit.client.0.smithi027.stdout:0/51: fsync d9/da/f10 0 2021-11-03T17:13:56.895 INFO:tasks.workunit.client.0.smithi027.stdout:1/45: truncate f2 1254410 0 2021-11-03T17:13:56.895 INFO:tasks.workunit.client.0.smithi027.stdout:1/46: stat d6 0 2021-11-03T17:13:56.897 INFO:tasks.workunit.client.0.smithi027.stdout:0/52: mkdir d9/d12 0 2021-11-03T17:13:56.897 INFO:tasks.workunit.client.0.smithi027.stdout:0/53: write d9/da/fe [907134,127953] 0 2021-11-03T17:13:56.898 INFO:tasks.workunit.client.0.smithi027.stdout:1/47: truncate d6/fa 735545 0 2021-11-03T17:13:56.899 INFO:tasks.workunit.client.0.smithi027.stdout:0/54: link d9/da/fe d9/f13 0 2021-11-03T17:13:56.901 INFO:tasks.workunit.client.0.smithi027.stdout:9/46: dwrite f8 [0,4194304] 0 2021-11-03T17:13:56.901 INFO:tasks.workunit.client.0.smithi027.stdout:0/55: link d9/da/l11 d9/l14 0 2021-11-03T17:13:56.903 INFO:tasks.workunit.client.0.smithi027.stdout:9/47: creat db/f12 x:0 0 0 2021-11-03T17:13:56.903 INFO:tasks.workunit.client.0.smithi027.stdout:8/64: dwrite f6 [0,4194304] 0 2021-11-03T17:13:56.904 INFO:tasks.workunit.client.0.smithi027.stdout:8/65: write d7/d10/f12 [727213,25480] 0 2021-11-03T17:13:56.904 INFO:tasks.workunit.client.0.smithi027.stdout:7/32: dwrite d0/d2/f5 [0,4194304] 0 2021-11-03T17:13:56.904 INFO:tasks.workunit.client.0.smithi027.stdout:0/56: truncate d9/da/fd 717602 0 2021-11-03T17:13:56.905 INFO:tasks.workunit.client.0.smithi027.stdout:9/48: creat db/f13 x:0 0 0 2021-11-03T17:13:56.905 INFO:tasks.workunit.client.0.smithi027.stdout:9/49: chown db/f13 1155 1 2021-11-03T17:13:56.907 INFO:tasks.workunit.client.0.smithi027.stdout:8/66: dread d7/fb [0,4194304] 0 2021-11-03T17:13:56.909 INFO:tasks.workunit.client.0.smithi027.stdout:2/17: rename d0/d2 to d0/d5 0 2021-11-03T17:13:56.909 INFO:tasks.workunit.client.0.smithi027.stdout:2/18: read d0/d5/f3 [535561,10669] 0 2021-11-03T17:13:56.910 INFO:tasks.workunit.client.0.smithi027.stdout:2/19: write d0/d5/f3 [3679138,46408] 0 2021-11-03T17:13:56.911 INFO:tasks.workunit.client.0.smithi027.stdout:7/33: getdents d0/d2 0 2021-11-03T17:13:56.912 INFO:tasks.workunit.client.0.smithi027.stdout:7/34: read d0/d2/f5 [3290102,125191] 0 2021-11-03T17:13:56.912 INFO:tasks.workunit.client.0.smithi027.stdout:7/35: mknod d0/ca 1 2021-11-03T17:13:56.914 INFO:tasks.workunit.client.0.smithi027.stdout:9/50: creat db/f14 x:0 0 0 2021-11-03T17:13:56.917 INFO:tasks.workunit.client.0.smithi027.stdout:1/48: write f2 [109942,72243] 0 2021-11-03T17:13:56.919 INFO:tasks.workunit.client.0.smithi027.stdout:7/36: dread d0/d2/f6 [0,4194304] 0 2021-11-03T17:13:56.919 INFO:tasks.workunit.client.0.smithi027.stdout:7/37: getdents d0/d2 0 2021-11-03T17:13:56.920 INFO:tasks.workunit.client.0.smithi027.stdout:8/67: creat d7/d10/f16 x:0 0 0 2021-11-03T17:13:56.921 INFO:tasks.workunit.client.0.smithi027.stdout:2/20: symlink d0/l6 0 2021-11-03T17:13:56.924 INFO:tasks.workunit.client.0.smithi027.stdout:1/49: creat d6/de/ff x:0 0 0 2021-11-03T17:13:56.924 INFO:tasks.workunit.client.0.smithi027.stdout:1/50: dread - d6/fd zero size 2021-11-03T17:13:56.925 INFO:tasks.workunit.client.0.smithi027.stdout:1/51: write d6/fd [735870,16778] 0 2021-11-03T17:13:56.927 INFO:tasks.workunit.client.0.smithi027.stdout:2/21: creat d0/f7 x:0 0 0 2021-11-03T17:13:56.928 INFO:tasks.workunit.client.0.smithi027.stdout:2/22: chown d0/d5 110 1 2021-11-03T17:13:56.934 INFO:tasks.workunit.client.0.smithi027.stdout:1/52: dread d6/fd [0,4194304] 0 2021-11-03T17:13:56.935 INFO:tasks.workunit.client.0.smithi027.stdout:9/51: dread db/fd [0,4194304] 0 2021-11-03T17:13:56.935 INFO:tasks.workunit.client.0.smithi027.stdout:9/52: symlink db/l15 0 2021-11-03T17:13:56.935 INFO:tasks.workunit.client.0.smithi027.stdout:9/53: creat db/f16 x:0 0 0 2021-11-03T17:13:56.936 INFO:tasks.workunit.client.0.smithi027.stdout:9/54: creat db/f17 x:0 0 0 2021-11-03T17:13:56.939 INFO:tasks.workunit.client.0.smithi027.stdout:1/53: dread d6/fa [0,4194304] 0 2021-11-03T17:13:56.941 INFO:tasks.workunit.client.0.smithi027.stdout:1/54: symlink d6/l10 0 2021-11-03T17:13:56.949 INFO:tasks.workunit.client.0.smithi027.stdout:0/57: dwrite d9/da/fd [0,4194304] 0 2021-11-03T17:13:56.950 INFO:tasks.workunit.client.0.smithi027.stdout:7/38: dwrite d0/d2/f6 [0,4194304] 0 2021-11-03T17:13:56.951 INFO:tasks.workunit.client.0.smithi027.stdout:1/55: read f2 [202148,123849] 0 2021-11-03T17:13:56.951 INFO:tasks.workunit.client.0.smithi027.stdout:1/56: write d6/de/ff [182452,117932] 0 2021-11-03T17:13:56.951 INFO:tasks.workunit.client.0.smithi027.stdout:7/39: read d0/d2/f6 [566481,115578] 0 2021-11-03T17:13:56.952 INFO:tasks.workunit.client.0.smithi027.stdout:8/68: getdents d7/d10 0 2021-11-03T17:13:56.952 INFO:tasks.workunit.client.0.smithi027.stdout:8/69: chown d7/d10/f16 21996 1 2021-11-03T17:13:56.954 INFO:tasks.workunit.client.0.smithi027.stdout:0/58: dread d9/da/fe [0,4194304] 0 2021-11-03T17:13:56.954 INFO:tasks.workunit.client.0.smithi027.stdout:0/59: mknod d9/c15 1 2021-11-03T17:13:56.954 INFO:tasks.workunit.client.0.smithi027.stdout:0/60: chown d9/da/fe 886829 1 2021-11-03T17:13:56.959 INFO:tasks.workunit.client.0.smithi027.stdout:7/40: dread d0/d2/f5 [0,4194304] 0 2021-11-03T17:13:56.960 INFO:tasks.workunit.client.0.smithi027.stdout:2/23: rmdir d0 39 2021-11-03T17:13:56.963 INFO:tasks.workunit.client.0.smithi027.stdout:0/61: read d9/f13 [694631,44708] 0 2021-11-03T17:13:56.963 INFO:tasks.workunit.client.0.smithi027.stdout:0/62: chown l7 3853194 1 2021-11-03T17:13:56.963 INFO:tasks.workunit.client.0.smithi027.stdout:0/63: fsync d9/da/fd 0 2021-11-03T17:13:56.963 INFO:tasks.workunit.client.0.smithi027.stdout:8/70: dread d7/f8 [0,4194304] 0 2021-11-03T17:13:56.964 INFO:tasks.workunit.client.0.smithi027.stdout:8/71: fdatasync d7/fb 0 2021-11-03T17:13:56.964 INFO:tasks.workunit.client.0.smithi027.stdout:9/55: getdents db 0 2021-11-03T17:13:56.968 INFO:tasks.workunit.client.0.smithi027.stdout:1/57: mkdir d6/d11 0 2021-11-03T17:13:56.972 INFO:tasks.workunit.client.0.smithi027.stdout:9/56: link l5 db/l18 0 2021-11-03T17:13:56.975 INFO:tasks.workunit.client.0.smithi027.stdout:9/57: mknod db/c19 1 2021-11-03T17:13:56.976 INFO:tasks.workunit.client.0.smithi027.stdout:9/58: dread - db/f13 zero size 2021-11-03T17:13:56.976 INFO:tasks.workunit.client.0.smithi027.stdout:9/59: read - db/f14 zero size 2021-11-03T17:13:56.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:56 smithi115 conmon[34670]: cluster 2021-11-03T17:13:56.601552+0000 mgr.smithi027.ufhhvb (mgr.14162) 295 2021-11-03T17:13:56.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:56 smithi115 conmon[34670]: : cluster [DBG] pgmap v245: 65 pgs: 65 active+clean; 518 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 33 MiB/s wr, 824 op/s 2021-11-03T17:13:56.993 INFO:tasks.workunit.client.0.smithi027.stdout:2/24: dwrite d0/f7 [0,4194304] 0 2021-11-03T17:13:56.998 INFO:tasks.workunit.client.0.smithi027.stdout:2/25: creat d0/d5/f8 x:0 0 0 2021-11-03T17:13:56.998 INFO:tasks.workunit.client.0.smithi027.stdout:2/26: write d0/d5/f3 [4898469,65873] 0 2021-11-03T17:13:57.003 INFO:tasks.workunit.client.0.smithi027.stdout:2/27: write d0/f7 [3837078,17361] 0 2021-11-03T17:13:57.009 INFO:tasks.workunit.client.0.smithi027.stdout:2/28: write d0/d5/f4 [2507905,41072] 0 2021-11-03T17:13:57.009 INFO:tasks.workunit.client.0.smithi027.stdout:0/64: dwrite d9/da/fd [0,4194304] 0 2021-11-03T17:13:57.010 INFO:tasks.workunit.client.0.smithi027.stdout:7/41: dwrite d0/d2/f6 [4194304,4194304] 0 2021-11-03T17:13:57.010 INFO:tasks.workunit.client.0.smithi027.stdout:9/60: dwrite db/f17 [0,4194304] 0 2021-11-03T17:13:57.011 INFO:tasks.workunit.client.0.smithi027.stdout:9/61: chown la 92935 1 2021-11-03T17:13:57.011 INFO:tasks.workunit.client.0.smithi027.stdout:1/58: dwrite d6/de/ff [0,4194304] 0 2021-11-03T17:13:57.012 INFO:tasks.workunit.client.0.smithi027.stdout:1/59: stat d6 0 2021-11-03T17:13:57.012 INFO:tasks.workunit.client.0.smithi027.stdout:0/65: creat d9/da/f16 x:0 0 0 2021-11-03T17:13:57.014 INFO:tasks.workunit.client.0.smithi027.stdout:7/42: symlink d0/lb 0 2021-11-03T17:13:57.016 INFO:tasks.workunit.client.0.smithi027.stdout:9/62: stat db/l11 0 2021-11-03T17:13:57.016 INFO:tasks.workunit.client.0.smithi027.stdout:9/63: write db/f14 [120389,1566] 0 2021-11-03T17:13:57.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:56 smithi027 conmon[32322]: cluster 2021-11-03T17:13:56.601552+0000 mgr.smithi027.ufhhvb ( 2021-11-03T17:13:57.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:56 smithi027 conmon[32322]: mgr.14162) 295 : cluster [DBG] pgmap v245: 65 pgs: 65 active+clean; 518 MiB data, 1.8 GiB used, 535 GiB / 536 GiB avail; 17 MiB/s rd, 33 MiB/s wr, 824 op/s 2021-11-03T17:13:57.019 INFO:tasks.workunit.client.0.smithi027.stdout:0/66: dread d9/da/fe [0,4194304] 0 2021-11-03T17:13:57.023 INFO:tasks.workunit.client.0.smithi027.stdout:9/64: dread db/f17 [0,4194304] 0 2021-11-03T17:13:57.024 INFO:tasks.workunit.client.0.smithi027.stdout:0/67: write d9/da/fe [856808,33933] 0 2021-11-03T17:13:57.027 INFO:tasks.workunit.client.0.smithi027.stdout:9/65: rename db/l18 to db/l1a 0 2021-11-03T17:13:57.030 INFO:tasks.workunit.client.0.smithi027.stdout:9/66: mkdir db/d1b 0 2021-11-03T17:13:57.033 INFO:tasks.workunit.client.0.smithi027.stdout:9/67: write db/f16 [239511,31406] 0 2021-11-03T17:13:57.033 INFO:tasks.workunit.client.0.smithi027.stdout:1/60: dwrite d6/fa [0,4194304] 0 2021-11-03T17:13:57.034 INFO:tasks.workunit.client.0.smithi027.stdout:2/29: dread d0/d5/f4 [0,4194304] 0 2021-11-03T17:13:57.035 INFO:tasks.workunit.client.0.smithi027.stdout:9/68: mkdir db/d1b/d1c 0 2021-11-03T17:13:57.041 INFO:tasks.workunit.client.0.smithi027.stdout:0/68: dread d9/da/fe [0,4194304] 0 2021-11-03T17:13:57.042 INFO:tasks.workunit.client.0.smithi027.stdout:7/43: dwrite d0/d2/f6 [0,4194304] 0 2021-11-03T17:13:57.042 INFO:tasks.workunit.client.0.smithi027.stdout:0/69: write d9/da/fe [1455820,130537] 0 2021-11-03T17:13:57.042 INFO:tasks.workunit.client.0.smithi027.stdout:0/70: fdatasync d9/da/fd 0 2021-11-03T17:13:57.043 INFO:tasks.workunit.client.0.smithi027.stdout:0/71: read d9/f13 [256257,105253] 0 2021-11-03T17:13:57.043 INFO:tasks.workunit.client.0.smithi027.stdout:0/72: fdatasync d9/da/f10 0 2021-11-03T17:13:57.043 INFO:tasks.workunit.client.0.smithi027.stdout:7/44: symlink d0/lc 0 2021-11-03T17:13:57.044 INFO:tasks.workunit.client.0.smithi027.stdout:7/45: creat d0/d2/fd x:0 0 0 2021-11-03T17:13:57.045 INFO:tasks.workunit.client.0.smithi027.stdout:7/46: symlink d0/le 0 2021-11-03T17:13:57.046 INFO:tasks.workunit.client.0.smithi027.stdout:7/47: creat d0/d2/ff x:0 0 0 2021-11-03T17:13:57.047 INFO:tasks.workunit.client.0.smithi027.stdout:7/48: readlink d0/d2/l7 0 2021-11-03T17:13:57.047 INFO:tasks.workunit.client.0.smithi027.stdout:7/49: mkdir d0/d2/d10 0 2021-11-03T17:13:57.049 INFO:tasks.workunit.client.0.smithi027.stdout:7/50: creat d0/d2/d10/f11 x:0 0 0 2021-11-03T17:13:57.051 INFO:tasks.workunit.client.0.smithi027.stdout:9/69: dread db/fd [0,4194304] 0 2021-11-03T17:13:57.051 INFO:tasks.workunit.client.0.smithi027.stdout:9/70: write db/f13 [845961,110356] 0 2021-11-03T17:13:57.051 INFO:tasks.workunit.client.0.smithi027.stdout:9/71: chown db/d1b 15435097 1 2021-11-03T17:13:57.051 INFO:tasks.workunit.client.0.smithi027.stdout:9/72: stat db/f17 0 2021-11-03T17:13:57.053 INFO:tasks.workunit.client.0.smithi027.stdout:1/61: dwrite d6/fa [0,4194304] 0 2021-11-03T17:13:57.054 INFO:tasks.workunit.client.0.smithi027.stdout:9/73: unlink db/f12 0 2021-11-03T17:13:57.055 INFO:tasks.workunit.client.0.smithi027.stdout:9/74: mkdir db/d1b/d1c/d1d 0 2021-11-03T17:13:57.060 INFO:tasks.workunit.client.0.smithi027.stdout:1/62: write d6/fc [833463,37391] 0 2021-11-03T17:13:57.071 INFO:tasks.workunit.client.0.smithi027.stdout:2/30: dwrite d0/f7 [0,4194304] 0 2021-11-03T17:13:57.071 INFO:tasks.workunit.client.0.smithi027.stdout:7/51: dread d0/d2/f6 [4194304,4194304] 0 2021-11-03T17:13:57.073 INFO:tasks.workunit.client.0.smithi027.stdout:2/31: link d0/d5/f4 d0/d5/f9 0 2021-11-03T17:13:57.074 INFO:tasks.workunit.client.0.smithi027.stdout:0/73: dwrite d9/da/f10 [0,4194304] 0 2021-11-03T17:13:57.075 INFO:tasks.workunit.client.0.smithi027.stdout:0/74: chown d9 1765989098 1 2021-11-03T17:13:57.075 INFO:tasks.workunit.client.0.smithi027.stdout:9/75: dwrite db/f17 [0,4194304] 0 2021-11-03T17:13:57.075 INFO:tasks.workunit.client.0.smithi027.stdout:0/75: write d9/da/fe [1290377,61938] 0 2021-11-03T17:13:57.079 INFO:tasks.workunit.client.0.smithi027.stdout:9/76: dread db/fd [0,4194304] 0 2021-11-03T17:13:57.081 INFO:tasks.workunit.client.0.smithi027.stdout:9/77: creat db/d1b/d1c/d1d/f1e x:0 0 0 2021-11-03T17:13:57.085 INFO:tasks.workunit.client.0.smithi027.stdout:9/78: read db/fd [2470429,22479] 0 2021-11-03T17:13:57.087 INFO:tasks.workunit.client.0.smithi027.stdout:9/79: creat db/f1f x:0 0 0 2021-11-03T17:13:57.091 INFO:tasks.workunit.client.0.smithi027.stdout:2/32: fsync d0/d5/f4 0 2021-11-03T17:13:57.095 INFO:tasks.workunit.client.0.smithi027.stdout:9/80: creat db/d1b/f20 x:0 0 0 2021-11-03T17:13:57.096 INFO:tasks.workunit.client.0.smithi027.stdout:0/76: dwrite d9/da/fe [0,4194304] 0 2021-11-03T17:13:57.096 INFO:tasks.workunit.client.0.smithi027.stdout:2/33: symlink d0/d5/la 0 2021-11-03T17:13:57.097 INFO:tasks.workunit.client.0.smithi027.stdout:2/34: fsync d0/f7 0 2021-11-03T17:13:57.099 INFO:tasks.workunit.client.0.smithi027.stdout:0/77: unlink l5 0 2021-11-03T17:13:57.099 INFO:tasks.workunit.client.0.smithi027.stdout:0/78: chown d9/da 5580535 1 2021-11-03T17:13:57.104 INFO:tasks.workunit.client.0.smithi027.stdout:2/35: rmdir d0/d5 39 2021-11-03T17:13:57.104 INFO:tasks.workunit.client.0.smithi027.stdout:2/36: readlink d0/l6 0 2021-11-03T17:13:57.104 INFO:tasks.workunit.client.0.smithi027.stdout:2/37: chown d0/f7 14277725 1 2021-11-03T17:13:57.105 INFO:tasks.workunit.client.0.smithi027.stdout:2/38: write d0/f7 [1685282,75215] 0 2021-11-03T17:13:57.106 INFO:tasks.workunit.client.0.smithi027.stdout:2/39: mknod d0/d5/cb 1 2021-11-03T17:13:57.106 INFO:tasks.workunit.client.0.smithi027.stdout:2/40: write d0/d5/f9 [1987087,28242] 0 2021-11-03T17:13:57.106 INFO:tasks.workunit.client.0.smithi027.stdout:2/41: chown d0/d5/f4 494279 1 2021-11-03T17:13:57.106 INFO:tasks.workunit.client.0.smithi027.stdout:9/81: getdents db/d1b 0 2021-11-03T17:13:57.107 INFO:tasks.workunit.client.0.smithi027.stdout:9/82: chown db/d1b/d1c/d1d 5485 1 2021-11-03T17:13:57.108 INFO:tasks.workunit.client.0.smithi027.stdout:2/42: rmdir d0 39 2021-11-03T17:13:57.114 INFO:tasks.workunit.client.0.smithi027.stdout:2/43: dread d0/d5/f3 [4194304,4194304] 0 2021-11-03T17:13:57.121 INFO:tasks.workunit.client.0.smithi027.stdout:0/79: dwrite d9/f13 [4194304,4194304] 0 2021-11-03T17:13:57.124 INFO:tasks.workunit.client.0.smithi027.stdout:0/80: write d9/f13 [2349876,74035] 0 2021-11-03T17:13:57.126 INFO:tasks.workunit.client.0.smithi027.stdout:0/81: creat d9/d12/f17 x:0 0 0 2021-11-03T17:13:57.127 INFO:tasks.workunit.client.0.smithi027.stdout:0/82: mkdir d9/da/d18 0 2021-11-03T17:13:57.136 INFO:tasks.workunit.client.0.smithi027.stdout:0/83: dread d9/da/fd [0,4194304] 0 2021-11-03T17:13:57.138 INFO:tasks.workunit.client.0.smithi027.stdout:0/84: symlink d9/da/d18/l19 0 2021-11-03T17:13:57.138 INFO:tasks.workunit.client.0.smithi027.stdout:0/85: chown d9/da/d18 13482868 1 2021-11-03T17:13:57.139 INFO:tasks.workunit.client.0.smithi027.stdout:0/86: chown d9/da/d18/l19 0 1 2021-11-03T17:13:57.139 INFO:tasks.workunit.client.0.smithi027.stdout:0/87: creat d9/f1a x:0 0 0 2021-11-03T17:13:57.140 INFO:tasks.workunit.client.0.smithi027.stdout:0/88: fdatasync d9/d12/f17 0 2021-11-03T17:13:57.142 INFO:tasks.workunit.client.0.smithi027.stdout:0/89: truncate d9/f13 6478 0 2021-11-03T17:13:57.419 INFO:tasks.workunit.client.0.smithi027.stdout:4/28: sync 2021-11-03T17:13:57.420 INFO:tasks.workunit.client.0.smithi027.stdout:8/72: sync 2021-11-03T17:13:57.420 INFO:tasks.workunit.client.0.smithi027.stdout:6/25: sync 2021-11-03T17:13:57.420 INFO:tasks.workunit.client.0.smithi027.stdout:3/33: sync 2021-11-03T17:13:57.420 INFO:tasks.workunit.client.0.smithi027.stdout:4/29: write f4 [2650367,41108] 0 2021-11-03T17:13:57.420 INFO:tasks.workunit.client.0.smithi027.stdout:5/50: sync 2021-11-03T17:13:57.422 INFO:tasks.workunit.client.0.smithi027.stdout:6/26: write f5 [1250390,14138] 0 2021-11-03T17:13:57.422 INFO:tasks.workunit.client.0.smithi027.stdout:6/27: stat f5 0 2021-11-03T17:13:57.422 INFO:tasks.workunit.client.0.smithi027.stdout:6/28: chown f5 46544897 1 2021-11-03T17:13:57.423 INFO:tasks.workunit.client.0.smithi027.stdout:3/34: write f0 [1135391,129742] 0 2021-11-03T17:13:57.423 INFO:tasks.workunit.client.0.smithi027.stdout:8/73: creat d7/f17 x:0 0 0 2021-11-03T17:13:57.423 INFO:tasks.workunit.client.0.smithi027.stdout:8/74: write d7/d10/f12 [573410,117195] 0 2021-11-03T17:13:57.423 INFO:tasks.workunit.client.0.smithi027.stdout:3/35: write f0 [6702136,49760] 0 2021-11-03T17:13:57.424 INFO:tasks.workunit.client.0.smithi027.stdout:3/36: readlink d1/l5 0 2021-11-03T17:13:57.424 INFO:tasks.workunit.client.0.smithi027.stdout:4/30: mkdir d6/db 0 2021-11-03T17:13:57.424 INFO:tasks.workunit.client.0.smithi027.stdout:8/75: write d7/f8 [2609862,53171] 0 2021-11-03T17:13:57.424 INFO:tasks.workunit.client.0.smithi027.stdout:5/51: creat d2/d4/df/f15 x:0 0 0 2021-11-03T17:13:57.426 INFO:tasks.workunit.client.0.smithi027.stdout:3/37: link d1/f6 d1/f9 0 2021-11-03T17:13:57.427 INFO:tasks.workunit.client.0.smithi027.stdout:5/52: symlink d2/d12/l16 0 2021-11-03T17:13:57.427 INFO:tasks.workunit.client.0.smithi027.stdout:5/53: chown d2/l8 583801 1 2021-11-03T17:13:57.428 INFO:tasks.workunit.client.0.smithi027.stdout:5/54: read - d2/d4/df/f15 zero size 2021-11-03T17:13:57.428 INFO:tasks.workunit.client.0.smithi027.stdout:5/55: fsync d2/d4/f5 0 2021-11-03T17:13:57.428 INFO:tasks.workunit.client.0.smithi027.stdout:5/56: fdatasync d2/f3 0 2021-11-03T17:13:57.431 INFO:tasks.workunit.client.0.smithi027.stdout:3/38: creat d1/fa x:0 0 0 2021-11-03T17:13:57.431 INFO:tasks.workunit.client.0.smithi027.stdout:4/31: dread f5 [0,4194304] 0 2021-11-03T17:13:57.432 INFO:tasks.workunit.client.0.smithi027.stdout:8/76: dread d7/d10/f12 [0,4194304] 0 2021-11-03T17:13:57.432 INFO:tasks.workunit.client.0.smithi027.stdout:8/77: getdents d7/d10 0 2021-11-03T17:13:57.432 INFO:tasks.workunit.client.0.smithi027.stdout:8/78: chown f6 23718700 1 2021-11-03T17:13:57.433 INFO:tasks.workunit.client.0.smithi027.stdout:6/29: truncate f5 1521846 0 2021-11-03T17:13:57.435 INFO:tasks.workunit.client.0.smithi027.stdout:5/57: unlink d2/d12/l16 0 2021-11-03T17:13:57.436 INFO:tasks.workunit.client.0.smithi027.stdout:3/39: mkdir d1/db 0 2021-11-03T17:13:57.436 INFO:tasks.workunit.client.0.smithi027.stdout:3/40: read - d1/f7 zero size 2021-11-03T17:13:57.438 INFO:tasks.workunit.client.0.smithi027.stdout:4/32: creat d6/db/fc x:0 0 0 2021-11-03T17:13:57.443 INFO:tasks.workunit.client.0.smithi027.stdout:4/33: dread f5 [0,4194304] 0 2021-11-03T17:13:57.459 INFO:tasks.workunit.client.0.smithi027.stdout:8/79: mkdir d7/d10/d18 0 2021-11-03T17:13:57.461 INFO:tasks.workunit.client.0.smithi027.stdout:6/30: getdents . 0 2021-11-03T17:13:57.461 INFO:tasks.workunit.client.0.smithi027.stdout:6/31: readlink - no filename 2021-11-03T17:13:57.466 INFO:tasks.workunit.client.0.smithi027.stdout:4/34: truncate f4 1061460 0 2021-11-03T17:13:57.468 INFO:tasks.workunit.client.0.smithi027.stdout:8/80: unlink d7/fb 0 2021-11-03T17:13:57.468 INFO:tasks.workunit.client.0.smithi027.stdout:8/81: chown d7/f17 0 1 2021-11-03T17:13:57.469 INFO:tasks.workunit.client.0.smithi027.stdout:6/32: creat f6 x:0 0 0 2021-11-03T17:13:57.473 INFO:tasks.workunit.client.0.smithi027.stdout:6/33: write f5 [1522631,82380] 0 2021-11-03T17:13:57.474 INFO:tasks.workunit.client.0.smithi027.stdout:4/35: link l1 d6/d9/ld 0 2021-11-03T17:13:57.474 INFO:tasks.workunit.client.0.smithi027.stdout:4/36: read d6/f8 [140584,74823] 0 2021-11-03T17:13:57.475 INFO:tasks.workunit.client.0.smithi027.stdout:4/37: mknod d6/ce 1 2021-11-03T17:13:57.479 INFO:tasks.workunit.client.0.smithi027.stdout:4/38: getdents d6/d9 0 2021-11-03T17:13:57.482 INFO:tasks.workunit.client.0.smithi027.stdout:4/39: creat d6/d9/ff x:0 0 0 2021-11-03T17:13:57.484 INFO:tasks.workunit.client.0.smithi027.stdout:4/40: symlink d6/d9/l10 0 2021-11-03T17:13:57.485 INFO:tasks.workunit.client.0.smithi027.stdout:4/41: truncate f4 549254 0 2021-11-03T17:13:57.489 INFO:tasks.workunit.client.0.smithi027.stdout:4/42: link f5 d6/d9/f11 0 2021-11-03T17:13:57.489 INFO:tasks.workunit.client.0.smithi027.stdout:4/43: read d6/f8 [133504,2347] 0 2021-11-03T17:13:57.490 INFO:tasks.workunit.client.0.smithi027.stdout:8/82: dwrite d7/d10/f16 [0,4194304] 0 2021-11-03T17:13:57.493 INFO:tasks.workunit.client.0.smithi027.stdout:8/83: getdents d7 0 2021-11-03T17:13:57.495 INFO:tasks.workunit.client.0.smithi027.stdout:4/44: dread f5 [0,4194304] 0 2021-11-03T17:13:57.495 INFO:tasks.workunit.client.0.smithi027.stdout:4/45: read - d6/d9/ff zero size 2021-11-03T17:13:57.495 INFO:tasks.workunit.client.0.smithi027.stdout:8/84: symlink d7/d10/d18/l19 0 2021-11-03T17:13:57.496 INFO:tasks.workunit.client.0.smithi027.stdout:8/85: read d7/fe [1389363,6391] 0 2021-11-03T17:13:57.497 INFO:tasks.workunit.client.0.smithi027.stdout:4/46: getdents d6 0 2021-11-03T17:13:57.498 INFO:tasks.workunit.client.0.smithi027.stdout:4/47: write f5 [3659051,16647] 0 2021-11-03T17:13:57.498 INFO:tasks.workunit.client.0.smithi027.stdout:6/34: dwrite f2 [0,4194304] 0 2021-11-03T17:13:57.500 INFO:tasks.workunit.client.0.smithi027.stdout:6/35: creat f7 x:0 0 0 2021-11-03T17:13:57.508 INFO:tasks.workunit.client.0.smithi027.stdout:9/83: sync 2021-11-03T17:13:57.510 INFO:tasks.workunit.client.0.smithi027.stdout:9/84: creat db/d1b/d1c/d1d/f21 x:0 0 0 2021-11-03T17:13:57.511 INFO:tasks.workunit.client.0.smithi027.stdout:9/85: chown db/fd 0 1 2021-11-03T17:13:57.511 INFO:tasks.workunit.client.0.smithi027.stdout:9/86: chown db/f14 3 1 2021-11-03T17:13:57.513 INFO:tasks.workunit.client.0.smithi027.stdout:9/87: mkdir db/d22 0 2021-11-03T17:13:57.513 INFO:tasks.workunit.client.0.smithi027.stdout:9/88: dread - db/d1b/d1c/d1d/f1e zero size 2021-11-03T17:13:57.514 INFO:tasks.workunit.client.0.smithi027.stdout:9/89: write db/fd [4623250,62123] 0 2021-11-03T17:13:57.514 INFO:tasks.workunit.client.0.smithi027.stdout:9/90: mkdir db/d1b/d23 0 2021-11-03T17:13:57.514 INFO:tasks.workunit.client.0.smithi027.stdout:9/91: mknod db/d1b/c24 1 2021-11-03T17:13:57.522 INFO:tasks.workunit.client.0.smithi027.stdout:8/86: dwrite d7/f17 [0,4194304] 0 2021-11-03T17:13:57.535 INFO:tasks.workunit.client.0.smithi027.stdout:2/44: sync 2021-11-03T17:13:57.535 INFO:tasks.workunit.client.0.smithi027.stdout:7/52: sync 2021-11-03T17:13:57.536 INFO:tasks.workunit.client.0.smithi027.stdout:0/90: sync 2021-11-03T17:13:57.536 INFO:tasks.workunit.client.0.smithi027.stdout:0/91: mknod d9/da/d18/c1b 1 2021-11-03T17:13:57.536 INFO:tasks.workunit.client.0.smithi027.stdout:7/53: mknod d0/d2/c12 1 2021-11-03T17:13:57.536 INFO:tasks.workunit.client.0.smithi027.stdout:0/92: truncate d9/d12/f17 362679 0 2021-11-03T17:13:57.538 INFO:tasks.workunit.client.0.smithi027.stdout:2/45: unlink d0/l6 0 2021-11-03T17:13:57.540 INFO:tasks.workunit.client.0.smithi027.stdout:7/54: creat d0/f13 x:0 0 0 2021-11-03T17:13:57.540 INFO:tasks.workunit.client.0.smithi027.stdout:7/55: dread - d0/d2/d10/f11 zero size 2021-11-03T17:13:57.540 INFO:tasks.workunit.client.0.smithi027.stdout:0/93: dread d9/da/fd [0,4194304] 0 2021-11-03T17:13:57.541 INFO:tasks.workunit.client.0.smithi027.stdout:0/94: dread d9/d12/f17 [0,4194304] 0 2021-11-03T17:13:57.542 INFO:tasks.workunit.client.0.smithi027.stdout:0/95: write d9/da/f10 [4576218,55701] 0 2021-11-03T17:13:57.542 INFO:tasks.workunit.client.0.smithi027.stdout:2/46: mkdir d0/dc 0 2021-11-03T17:13:57.542 INFO:tasks.workunit.client.0.smithi027.stdout:2/47: mknod d0/d5/cd 1 2021-11-03T17:13:57.543 INFO:tasks.workunit.client.0.smithi027.stdout:7/56: symlink d0/l14 0 2021-11-03T17:13:57.544 INFO:tasks.workunit.client.0.smithi027.stdout:7/57: chown d0/d2 300797 1 2021-11-03T17:13:57.544 INFO:tasks.workunit.client.0.smithi027.stdout:2/48: read d0/f7 [3173393,116617] 0 2021-11-03T17:13:57.544 INFO:tasks.workunit.client.0.smithi027.stdout:2/49: readlink d0/d5/la 0 2021-11-03T17:13:57.544 INFO:tasks.workunit.client.0.smithi027.stdout:0/96: symlink d9/d12/l1c 0 2021-11-03T17:13:57.545 INFO:tasks.workunit.client.0.smithi027.stdout:7/58: stat d0/d2/l9 0 2021-11-03T17:13:57.545 INFO:tasks.workunit.client.0.smithi027.stdout:7/59: write d0/d2/fd [694079,7151] 0 2021-11-03T17:13:57.545 INFO:tasks.workunit.client.0.smithi027.stdout:7/60: chown d0/d2/f6 7733 1 2021-11-03T17:13:57.545 INFO:tasks.workunit.client.0.smithi027.stdout:6/36: dwrite f2 [0,4194304] 0 2021-11-03T17:13:57.547 INFO:tasks.workunit.client.0.smithi027.stdout:9/92: dwrite db/f16 [0,4194304] 0 2021-11-03T17:13:57.547 INFO:tasks.workunit.client.0.smithi027.stdout:4/48: dwrite d6/f8 [0,4194304] 0 2021-11-03T17:13:57.547 INFO:tasks.workunit.client.0.smithi027.stdout:4/49: mknod d6/c12 1 2021-11-03T17:13:57.547 INFO:tasks.workunit.client.0.smithi027.stdout:8/87: dwrite d7/f17 [0,4194304] 0 2021-11-03T17:13:57.548 INFO:tasks.workunit.client.0.smithi027.stdout:2/50: rename d0/dc to d0/d5/de 0 2021-11-03T17:13:57.548 INFO:tasks.workunit.client.0.smithi027.stdout:6/37: write f2 [2789959,79823] 0 2021-11-03T17:13:57.548 INFO:tasks.workunit.client.0.smithi027.stdout:6/38: dread - f7 zero size 2021-11-03T17:13:57.549 INFO:tasks.workunit.client.0.smithi027.stdout:0/97: creat d9/d12/f1d x:0 0 0 2021-11-03T17:13:57.550 INFO:tasks.workunit.client.0.smithi027.stdout:7/61: rmdir d0/d2/d10 39 2021-11-03T17:13:57.551 INFO:tasks.workunit.client.0.smithi027.stdout:9/93: dread db/f16 [0,4194304] 0 2021-11-03T17:13:57.553 INFO:tasks.workunit.client.0.smithi027.stdout:9/94: write db/fd [2479266,54586] 0 2021-11-03T17:13:57.553 INFO:tasks.workunit.client.0.smithi027.stdout:8/88: dread f6 [0,4194304] 0 2021-11-03T17:13:57.553 INFO:tasks.workunit.client.0.smithi027.stdout:8/89: write d7/fe [175030,80680] 0 2021-11-03T17:13:57.560 INFO:tasks.workunit.client.0.smithi027.stdout:4/50: dread d6/f8 [0,4194304] 0 2021-11-03T17:13:57.560 INFO:tasks.workunit.client.0.smithi027.stdout:4/51: mknod d6/db/c13 1 2021-11-03T17:13:57.565 INFO:tasks.workunit.client.0.smithi027.stdout:2/51: symlink d0/d5/lf 0 2021-11-03T17:13:57.566 INFO:tasks.workunit.client.0.smithi027.stdout:2/52: chown d0/d5/f9 1207818 1 2021-11-03T17:13:57.566 INFO:tasks.workunit.client.0.smithi027.stdout:2/53: dread - d0/d5/f8 zero size 2021-11-03T17:13:57.570 INFO:tasks.workunit.client.0.smithi027.stdout:0/98: creat d9/d12/f1e x:0 0 0 2021-11-03T17:13:57.570 INFO:tasks.workunit.client.0.smithi027.stdout:0/99: truncate d9/f1a 297792 0 2021-11-03T17:13:57.571 INFO:tasks.workunit.client.0.smithi027.stdout:7/62: symlink d0/d2/d10/l15 0 2021-11-03T17:13:57.574 INFO:tasks.workunit.client.0.smithi027.stdout:9/95: creat db/d1b/d1c/f25 x:0 0 0 2021-11-03T17:13:57.575 INFO:tasks.workunit.client.0.smithi027.stdout:9/96: mknod db/d1b/d23/c26 1 2021-11-03T17:13:57.575 INFO:tasks.workunit.client.0.smithi027.stdout:9/97: chown db/d1b/d1c 0 1 2021-11-03T17:13:57.575 INFO:tasks.workunit.client.0.smithi027.stdout:9/98: mknod db/d1b/d1c/d1d/c27 1 2021-11-03T17:13:57.576 INFO:tasks.workunit.client.0.smithi027.stdout:8/90: dwrite d7/f17 [0,4194304] 0 2021-11-03T17:13:57.576 INFO:tasks.workunit.client.0.smithi027.stdout:4/52: symlink d6/db/l14 0 2021-11-03T17:13:57.577 INFO:tasks.workunit.client.0.smithi027.stdout:2/54: link d0/f7 d0/f10 0 2021-11-03T17:13:57.577 INFO:tasks.workunit.client.0.smithi027.stdout:2/55: stat d0/d5/f3 0 2021-11-03T17:13:57.577 INFO:tasks.workunit.client.0.smithi027.stdout:2/56: write d0/d5/f8 [725617,60528] 0 2021-11-03T17:13:57.579 INFO:tasks.workunit.client.0.smithi027.stdout:9/99: link db/l1a db/d1b/d1c/d1d/l28 0 2021-11-03T17:13:57.580 INFO:tasks.workunit.client.0.smithi027.stdout:9/100: fsync db/f14 0 2021-11-03T17:13:57.581 INFO:tasks.workunit.client.0.smithi027.stdout:8/91: mkdir d7/d10/d18/d1a 0 2021-11-03T17:13:57.581 INFO:tasks.workunit.client.0.smithi027.stdout:8/92: chown d7/d10/d18/d1a 2733 1 2021-11-03T17:13:57.582 INFO:tasks.workunit.client.0.smithi027.stdout:4/53: symlink d6/db/l15 0 2021-11-03T17:13:57.584 INFO:tasks.workunit.client.0.smithi027.stdout:2/57: rename d0/d5/f9 to d0/f11 0 2021-11-03T17:13:57.587 INFO:tasks.workunit.client.0.smithi027.stdout:8/93: creat d7/f1b x:0 0 0 2021-11-03T17:13:57.589 INFO:tasks.workunit.client.0.smithi027.stdout:6/39: truncate f5 354854 0 2021-11-03T17:13:57.592 INFO:tasks.workunit.client.0.smithi027.stdout:0/100: dwrite d9/d12/f1e [0,4194304] 0 2021-11-03T17:13:57.593 INFO:tasks.workunit.client.0.smithi027.stdout:7/63: dwrite d0/d2/f5 [0,4194304] 0 2021-11-03T17:13:57.593 INFO:tasks.workunit.client.0.smithi027.stdout:7/64: write d0/d2/ff [348978,2796] 0 2021-11-03T17:13:57.594 INFO:tasks.workunit.client.0.smithi027.stdout:9/101: rename la to db/d1b/l29 0 2021-11-03T17:13:57.594 INFO:tasks.workunit.client.0.smithi027.stdout:9/102: dread - db/d1b/d1c/d1d/f21 zero size 2021-11-03T17:13:57.595 INFO:tasks.workunit.client.0.smithi027.stdout:8/94: unlink d7/f17 0 2021-11-03T17:13:57.600 INFO:tasks.workunit.client.0.smithi027.stdout:0/101: creat d9/f1f x:0 0 0 2021-11-03T17:13:57.600 INFO:tasks.workunit.client.0.smithi027.stdout:0/102: chown d9/da/f16 313860 1 2021-11-03T17:13:57.602 INFO:tasks.workunit.client.0.smithi027.stdout:7/65: creat d0/f16 x:0 0 0 2021-11-03T17:13:57.602 INFO:tasks.workunit.client.0.smithi027.stdout:9/103: creat db/d22/f2a x:0 0 0 2021-11-03T17:13:57.603 INFO:tasks.workunit.client.0.smithi027.stdout:6/40: dread f0 [0,4194304] 0 2021-11-03T17:13:57.603 INFO:tasks.workunit.client.0.smithi027.stdout:0/103: creat d9/f20 x:0 0 0 2021-11-03T17:13:57.604 INFO:tasks.workunit.client.0.smithi027.stdout:0/104: chown d9/da/l11 192054 1 2021-11-03T17:13:57.605 INFO:tasks.workunit.client.0.smithi027.stdout:7/66: rename d0/d2/ff to d0/f17 0 2021-11-03T17:13:57.605 INFO:tasks.workunit.client.0.smithi027.stdout:2/58: dwrite d0/f10 [4194304,4194304] 0 2021-11-03T17:13:57.608 INFO:tasks.workunit.client.0.smithi027.stdout:4/54: getdents d6/db 0 2021-11-03T17:13:57.608 INFO:tasks.workunit.client.0.smithi027.stdout:4/55: read - d6/db/fc zero size 2021-11-03T17:13:57.608 INFO:tasks.workunit.client.0.smithi027.stdout:4/56: write d6/db/fc [903485,50102] 0 2021-11-03T17:13:57.610 INFO:tasks.workunit.client.0.smithi027.stdout:2/59: read d0/f7 [1108471,101784] 0 2021-11-03T17:13:57.611 INFO:tasks.workunit.client.0.smithi027.stdout:2/60: fsync d0/f11 0 2021-11-03T17:13:57.611 INFO:tasks.workunit.client.0.smithi027.stdout:2/61: mknod d0/d5/de/c12 1 2021-11-03T17:13:57.611 INFO:tasks.workunit.client.0.smithi027.stdout:2/62: chown d0/d5 491883 1 2021-11-03T17:13:57.613 INFO:tasks.workunit.client.0.smithi027.stdout:8/95: dwrite d7/f1b [0,4194304] 0 2021-11-03T17:13:57.614 INFO:tasks.workunit.client.0.smithi027.stdout:0/105: symlink d9/l21 0 2021-11-03T17:13:57.615 INFO:tasks.workunit.client.0.smithi027.stdout:0/106: mknod d9/d12/c22 1 2021-11-03T17:13:57.615 INFO:tasks.workunit.client.0.smithi027.stdout:0/107: write d9/d12/f17 [1352725,90342] 0 2021-11-03T17:13:57.615 INFO:tasks.workunit.client.0.smithi027.stdout:4/57: fsync d6/d9/f11 0 2021-11-03T17:13:57.616 INFO:tasks.workunit.client.0.smithi027.stdout:7/67: creat d0/f18 x:0 0 0 2021-11-03T17:13:57.617 INFO:tasks.workunit.client.0.smithi027.stdout:7/68: dread - d0/f16 zero size 2021-11-03T17:13:57.619 INFO:tasks.workunit.client.0.smithi027.stdout:2/63: link d0/f11 d0/d5/f13 0 2021-11-03T17:13:57.621 INFO:tasks.workunit.client.0.smithi027.stdout:7/69: write d0/d2/f6 [3379754,40503] 0 2021-11-03T17:13:57.622 INFO:tasks.workunit.client.0.smithi027.stdout:8/96: getdents d7 0 2021-11-03T17:13:57.625 INFO:tasks.workunit.client.0.smithi027.stdout:6/41: dwrite f7 [0,4194304] 0 2021-11-03T17:13:57.625 INFO:tasks.workunit.client.0.smithi027.stdout:8/97: creat d7/d10/d18/f1c x:0 0 0 2021-11-03T17:13:57.626 INFO:tasks.workunit.client.0.smithi027.stdout:8/98: fsync d7/f1b 0 2021-11-03T17:13:57.627 INFO:tasks.workunit.client.0.smithi027.stdout:4/58: mkdir d6/d9/d16 0 2021-11-03T17:13:57.627 INFO:tasks.workunit.client.0.smithi027.stdout:4/59: chown d6/d9 109946717 1 2021-11-03T17:13:57.632 INFO:tasks.workunit.client.0.smithi027.stdout:0/108: dwrite d9/f1a [0,4194304] 0 2021-11-03T17:13:57.633 INFO:tasks.workunit.client.0.smithi027.stdout:0/109: chown d9/da/d18/l19 26786 1 2021-11-03T17:13:57.633 INFO:tasks.workunit.client.0.smithi027.stdout:0/110: dread - d9/da/f16 zero size 2021-11-03T17:13:57.634 INFO:tasks.workunit.client.0.smithi027.stdout:7/70: dread d0/d2/fd [0,4194304] 0 2021-11-03T17:13:57.634 INFO:tasks.workunit.client.0.smithi027.stdout:7/71: write d0/f18 [89246,53769] 0 2021-11-03T17:13:57.636 INFO:tasks.workunit.client.0.smithi027.stdout:0/111: creat d9/f23 x:0 0 0 2021-11-03T17:13:57.637 INFO:tasks.workunit.client.0.smithi027.stdout:0/112: write d9/f13 [580123,11720] 0 2021-11-03T17:13:57.637 INFO:tasks.workunit.client.0.smithi027.stdout:0/113: stat d9/f1a 0 2021-11-03T17:13:57.637 INFO:tasks.workunit.client.0.smithi027.stdout:0/114: dread - d9/da/f16 zero size 2021-11-03T17:13:57.637 INFO:tasks.workunit.client.0.smithi027.stdout:0/115: chown d9/d12/f17 420381 1 2021-11-03T17:13:57.638 INFO:tasks.workunit.client.0.smithi027.stdout:8/99: write d7/f8 [3505536,75609] 0 2021-11-03T17:13:57.638 INFO:tasks.workunit.client.0.smithi027.stdout:2/64: dread d0/f10 [0,4194304] 0 2021-11-03T17:13:57.643 INFO:tasks.workunit.client.0.smithi027.stdout:4/60: dwrite d6/d9/ff [0,4194304] 0 2021-11-03T17:13:57.644 INFO:tasks.workunit.client.0.smithi027.stdout:9/104: rename db/d22 to db/d1b/d2b 0 2021-11-03T17:13:57.650 INFO:tasks.workunit.client.0.smithi027.stdout:0/116: creat d9/f24 x:0 0 0 2021-11-03T17:13:57.650 INFO:tasks.workunit.client.0.smithi027.stdout:0/117: truncate d9/f1f 752219 0 2021-11-03T17:13:57.652 INFO:tasks.workunit.client.0.smithi027.stdout:7/72: dwrite d0/f13 [0,4194304] 0 2021-11-03T17:13:57.652 INFO:tasks.workunit.client.0.smithi027.stdout:7/73: dread - d0/f16 zero size 2021-11-03T17:13:57.652 INFO:tasks.workunit.client.0.smithi027.stdout:7/74: readlink d0/d2/l7 0 2021-11-03T17:13:57.653 INFO:tasks.workunit.client.0.smithi027.stdout:8/100: creat d7/f1d x:0 0 0 2021-11-03T17:13:57.654 INFO:tasks.workunit.client.0.smithi027.stdout:2/65: symlink d0/d5/de/l14 0 2021-11-03T17:13:57.655 INFO:tasks.workunit.client.0.smithi027.stdout:4/61: truncate f5 1291796 0 2021-11-03T17:13:57.660 INFO:tasks.workunit.client.0.smithi027.stdout:4/62: read d6/d9/ff [2069012,21884] 0 2021-11-03T17:13:57.661 INFO:tasks.workunit.client.0.smithi027.stdout:9/105: mkdir db/d1b/d23/d2c 0 2021-11-03T17:13:57.662 INFO:tasks.workunit.client.0.smithi027.stdout:0/118: rename d9/da/d18/l19 to d9/da/d18/l25 0 2021-11-03T17:13:57.662 INFO:tasks.workunit.client.0.smithi027.stdout:0/119: fsync d9/f23 0 2021-11-03T17:13:57.669 INFO:tasks.workunit.client.0.smithi027.stdout:7/75: creat d0/d2/d10/f19 x:0 0 0 2021-11-03T17:13:57.670 INFO:tasks.workunit.client.0.smithi027.stdout:2/66: symlink d0/l15 0 2021-11-03T17:13:57.671 INFO:tasks.workunit.client.0.smithi027.stdout:8/101: dwrite d7/fe [0,4194304] 0 2021-11-03T17:13:57.671 INFO:tasks.workunit.client.0.smithi027.stdout:8/102: rename d7/d10 to d7/d10/d1e 22 2021-11-03T17:13:57.673 INFO:tasks.workunit.client.0.smithi027.stdout:4/63: mkdir d6/d9/d16/d17 0 2021-11-03T17:13:57.677 INFO:tasks.workunit.client.0.smithi027.stdout:7/76: unlink d0/f17 0 2021-11-03T17:13:57.680 INFO:tasks.workunit.client.0.smithi027.stdout:2/67: truncate d0/f11 855803 0 2021-11-03T17:13:57.684 INFO:tasks.workunit.client.0.smithi027.stdout:7/77: symlink d0/d2/l1a 0 2021-11-03T17:13:57.685 INFO:tasks.workunit.client.0.smithi027.stdout:0/120: dwrite d9/d12/f1d [0,4194304] 0 2021-11-03T17:13:57.685 INFO:tasks.workunit.client.0.smithi027.stdout:0/121: write d9/f20 [591888,72262] 0 2021-11-03T17:13:57.687 INFO:tasks.workunit.client.0.smithi027.stdout:9/106: dwrite db/d1b/d2b/f2a [0,4194304] 0 2021-11-03T17:13:57.688 INFO:tasks.workunit.client.0.smithi027.stdout:8/103: dwrite d7/fe [0,4194304] 0 2021-11-03T17:13:57.691 INFO:tasks.workunit.client.0.smithi027.stdout:8/104: read d7/d10/f15 [429479,82248] 0 2021-11-03T17:13:57.691 INFO:tasks.workunit.client.0.smithi027.stdout:8/105: readlink l4 0 2021-11-03T17:13:57.692 INFO:tasks.workunit.client.0.smithi027.stdout:4/64: dread d6/d9/f11 [0,4194304] 0 2021-11-03T17:13:57.692 INFO:tasks.workunit.client.0.smithi027.stdout:4/65: chown f5 2731284 1 2021-11-03T17:13:57.696 INFO:tasks.workunit.client.0.smithi027.stdout:9/107: symlink db/d1b/d23/d2c/l2d 0 2021-11-03T17:13:57.700 INFO:tasks.workunit.client.0.smithi027.stdout:0/122: rename d9/da/d18/l25 to d9/da/l26 0 2021-11-03T17:13:57.701 INFO:tasks.workunit.client.0.smithi027.stdout:0/123: mknod d9/d12/c27 1 2021-11-03T17:13:57.701 INFO:tasks.workunit.client.0.smithi027.stdout:0/124: chown d9/da/fe 4 1 2021-11-03T17:13:57.701 INFO:tasks.workunit.client.0.smithi027.stdout:7/78: getdents d0/d2/d10 0 2021-11-03T17:13:57.701 INFO:tasks.workunit.client.0.smithi027.stdout:7/79: chown d0/l1 546500460 1 2021-11-03T17:13:57.701 INFO:tasks.workunit.client.0.smithi027.stdout:7/80: write d0/f16 [611407,7570] 0 2021-11-03T17:13:57.702 INFO:tasks.workunit.client.0.smithi027.stdout:7/81: truncate d0/f18 379126 0 2021-11-03T17:13:57.702 INFO:tasks.workunit.client.0.smithi027.stdout:7/82: chown d0/d2/l7 953847 1 2021-11-03T17:13:57.702 INFO:tasks.workunit.client.0.smithi027.stdout:7/83: dread - d0/d2/d10/f11 zero size 2021-11-03T17:13:57.702 INFO:tasks.workunit.client.0.smithi027.stdout:7/84: stat d0/lb 0 2021-11-03T17:13:57.707 INFO:tasks.workunit.client.0.smithi027.stdout:2/68: dwrite d0/f7 [0,4194304] 0 2021-11-03T17:13:57.707 INFO:tasks.workunit.client.0.smithi027.stdout:2/69: mknod d0/c16 1 2021-11-03T17:13:57.707 INFO:tasks.workunit.client.0.smithi027.stdout:2/70: fsync d0/f11 0 2021-11-03T17:13:57.708 INFO:tasks.workunit.client.0.smithi027.stdout:2/71: write d0/f11 [1519418,78668] 0 2021-11-03T17:13:57.708 INFO:tasks.workunit.client.0.smithi027.stdout:2/72: write d0/d5/f8 [1478993,70015] 0 2021-11-03T17:13:57.708 INFO:tasks.workunit.client.0.smithi027.stdout:2/73: write d0/d5/f8 [1565945,104355] 0 2021-11-03T17:13:57.709 INFO:tasks.workunit.client.0.smithi027.stdout:4/66: dwrite d6/d9/ff [0,4194304] 0 2021-11-03T17:13:57.711 INFO:tasks.workunit.client.0.smithi027.stdout:7/85: creat d0/d2/d10/f1b x:0 0 0 2021-11-03T17:13:57.712 INFO:tasks.workunit.client.0.smithi027.stdout:9/108: symlink db/l2e 0 2021-11-03T17:13:57.712 INFO:tasks.workunit.client.0.smithi027.stdout:9/109: fsync db/fd 0 2021-11-03T17:13:57.714 INFO:tasks.workunit.client.0.smithi027.stdout:4/67: rename d6/db/l14 to d6/l18 0 2021-11-03T17:13:57.717 INFO:tasks.workunit.client.0.smithi027.stdout:7/86: unlink d0/d2/f6 0 2021-11-03T17:13:57.717 INFO:tasks.workunit.client.0.smithi027.stdout:7/87: fsync d0/d2/f5 0 2021-11-03T17:13:57.717 INFO:tasks.workunit.client.0.smithi027.stdout:0/125: dwrite d9/f1a [0,4194304] 0 2021-11-03T17:13:57.718 INFO:tasks.workunit.client.0.smithi027.stdout:2/74: fsync d0/f11 0 2021-11-03T17:13:57.719 INFO:tasks.workunit.client.0.smithi027.stdout:9/110: dread db/fd [0,4194304] 0 2021-11-03T17:13:57.719 INFO:tasks.workunit.client.0.smithi027.stdout:9/111: fdatasync db/f1f 0 2021-11-03T17:13:57.720 INFO:tasks.workunit.client.0.smithi027.stdout:4/68: creat d6/d9/d16/f19 x:0 0 0 2021-11-03T17:13:57.720 INFO:tasks.workunit.client.0.smithi027.stdout:4/69: fsync f5 0 2021-11-03T17:13:57.720 INFO:tasks.workunit.client.0.smithi027.stdout:4/70: mknod d6/d9/d16/c1a 1 2021-11-03T17:13:57.720 INFO:tasks.workunit.client.0.smithi027.stdout:7/88: mkdir d0/d2/d1c 0 2021-11-03T17:13:57.727 INFO:tasks.workunit.client.0.smithi027.stdout:0/126: creat d9/d12/f28 x:0 0 0 2021-11-03T17:13:57.729 INFO:tasks.workunit.client.0.smithi027.stdout:2/75: symlink d0/d5/l17 0 2021-11-03T17:13:57.730 INFO:tasks.workunit.client.0.smithi027.stdout:9/112: mkdir db/d1b/d1c/d1d/d2f 0 2021-11-03T17:13:57.734 INFO:tasks.workunit.client.0.smithi027.stdout:0/127: symlink d9/da/l29 0 2021-11-03T17:13:57.735 INFO:tasks.workunit.client.0.smithi027.stdout:2/76: write d0/f7 [4871552,66651] 0 2021-11-03T17:13:57.737 INFO:tasks.workunit.client.0.smithi027.stdout:0/128: dread d9/da/fe [0,4194304] 0 2021-11-03T17:13:57.737 INFO:tasks.workunit.client.0.smithi027.stdout:0/129: chown d9/d12/l1c 63514 1 2021-11-03T17:13:57.740 INFO:tasks.workunit.client.0.smithi027.stdout:4/71: dwrite f5 [0,4194304] 0 2021-11-03T17:13:57.760 INFO:tasks.workunit.client.0.smithi027.stdout:9/113: dwrite f8 [4194304,4194304] 0 2021-11-03T17:13:57.760 INFO:tasks.workunit.client.0.smithi027.stdout:9/114: mknod db/d1b/c30 1 2021-11-03T17:13:57.761 INFO:tasks.workunit.client.0.smithi027.stdout:2/77: dread d0/f7 [0,4194304] 0 2021-11-03T17:13:57.762 INFO:tasks.workunit.client.0.smithi027.stdout:0/130: dwrite d9/da/fd [0,4194304] 0 2021-11-03T17:13:57.764 INFO:tasks.workunit.client.0.smithi027.stdout:0/131: rename d9/f20 to d9/f2a 0 2021-11-03T17:13:57.767 INFO:tasks.workunit.client.0.smithi027.stdout:2/78: dread d0/d5/f3 [0,4194304] 0 2021-11-03T17:13:57.770 INFO:tasks.workunit.client.0.smithi027.stdout:0/132: rmdir d9/da/d18 0 2021-11-03T17:13:57.770 INFO:tasks.workunit.client.0.smithi027.stdout:0/133: truncate d9/d12/f1e 5057473 0 2021-11-03T17:13:57.771 INFO:tasks.workunit.client.0.smithi027.stdout:0/134: chown d9/d12/f1e 1664 1 2021-11-03T17:13:57.771 INFO:tasks.workunit.client.0.smithi027.stdout:0/135: chown d9/f23 163 1 2021-11-03T17:13:57.771 INFO:tasks.workunit.client.0.smithi027.stdout:9/115: dread db/f17 [0,4194304] 0 2021-11-03T17:13:57.772 INFO:tasks.workunit.client.0.smithi027.stdout:0/136: creat d9/f2b x:0 0 0 2021-11-03T17:13:57.772 INFO:tasks.workunit.client.0.smithi027.stdout:0/137: truncate d9/f13 1151100 0 2021-11-03T17:13:57.773 INFO:tasks.workunit.client.0.smithi027.stdout:0/138: write d9/f24 [70399,62517] 0 2021-11-03T17:13:57.773 INFO:tasks.workunit.client.0.smithi027.stdout:0/139: symlink d9/d12/l2c 0 2021-11-03T17:13:57.773 INFO:tasks.workunit.client.0.smithi027.stdout:2/79: dread d0/d5/f13 [0,4194304] 0 2021-11-03T17:13:57.774 INFO:tasks.workunit.client.0.smithi027.stdout:2/80: mknod d0/d5/de/c18 1 2021-11-03T17:13:57.775 INFO:tasks.workunit.client.0.smithi027.stdout:0/140: creat d9/da/f2d x:0 0 0 2021-11-03T17:13:57.777 INFO:tasks.workunit.client.0.smithi027.stdout:2/81: rename d0/d5/lf to d0/l19 0 2021-11-03T17:13:57.778 INFO:tasks.workunit.client.0.smithi027.stdout:4/72: truncate f5 4047954 0 2021-11-03T17:13:57.780 INFO:tasks.workunit.client.0.smithi027.stdout:0/141: creat d9/f2e x:0 0 0 2021-11-03T17:13:57.781 INFO:tasks.workunit.client.0.smithi027.stdout:4/73: creat d6/d9/f1b x:0 0 0 2021-11-03T17:13:57.787 INFO:tasks.workunit.client.0.smithi027.stdout:9/116: dwrite db/f16 [0,4194304] 0 2021-11-03T17:13:57.787 INFO:tasks.workunit.client.0.smithi027.stdout:9/117: chown db/d1b/d1c 440 1 2021-11-03T17:13:57.787 INFO:tasks.workunit.client.0.smithi027.stdout:9/118: write db/d1b/d1c/f25 [954772,79006] 0 2021-11-03T17:13:57.788 INFO:tasks.workunit.client.0.smithi027.stdout:9/119: chown db/d1b 13 1 2021-11-03T17:13:57.790 INFO:tasks.workunit.client.0.smithi027.stdout:9/120: symlink db/d1b/d1c/d1d/l31 0 2021-11-03T17:13:57.790 INFO:tasks.workunit.client.0.smithi027.stdout:9/121: dread - db/f1f zero size 2021-11-03T17:13:57.791 INFO:tasks.workunit.client.0.smithi027.stdout:9/122: dread db/f14 [0,4194304] 0 2021-11-03T17:13:57.792 INFO:tasks.workunit.client.0.smithi027.stdout:9/123: truncate db/d1b/f20 555147 0 2021-11-03T17:13:57.792 INFO:tasks.workunit.client.0.smithi027.stdout:9/124: mknod db/d1b/d2b/c32 1 2021-11-03T17:13:57.801 INFO:tasks.workunit.client.0.smithi027.stdout:2/82: dwrite d0/f11 [0,4194304] 0 2021-11-03T17:13:57.802 INFO:tasks.workunit.client.0.smithi027.stdout:2/83: fsync d0/d5/f13 0 2021-11-03T17:13:57.806 INFO:tasks.workunit.client.0.smithi027.stdout:2/84: dread d0/d5/f8 [0,4194304] 0 2021-11-03T17:13:57.806 INFO:tasks.workunit.client.0.smithi027.stdout:2/85: chown d0/d5/de 139457 1 2021-11-03T17:13:57.806 INFO:tasks.workunit.client.0.smithi027.stdout:2/86: chown d0/l19 0 1 2021-11-03T17:13:57.807 INFO:tasks.workunit.client.0.smithi027.stdout:4/74: dwrite d6/f8 [0,4194304] 0 2021-11-03T17:13:57.817 INFO:tasks.workunit.client.0.smithi027.stdout:2/87: symlink d0/d5/l1a 0 2021-11-03T17:13:57.817 INFO:tasks.workunit.client.0.smithi027.stdout:2/88: write d0/d5/f8 [2113402,75365] 0 2021-11-03T17:13:57.818 INFO:tasks.workunit.client.0.smithi027.stdout:4/75: symlink d6/l1c 0 2021-11-03T17:13:57.821 INFO:tasks.workunit.client.0.smithi027.stdout:4/76: truncate f5 4012247 0 2021-11-03T17:13:57.822 INFO:tasks.workunit.client.0.smithi027.stdout:4/77: chown d6/f8 64 1 2021-11-03T17:13:57.824 INFO:tasks.workunit.client.0.smithi027.stdout:4/78: rename d6/d9/ld to d6/d9/d16/d17/l1d 0 2021-11-03T17:13:57.825 INFO:tasks.workunit.client.0.smithi027.stdout:4/79: creat d6/d9/f1e x:0 0 0 2021-11-03T17:13:57.827 INFO:tasks.workunit.client.0.smithi027.stdout:4/80: creat d6/f1f x:0 0 0 2021-11-03T17:13:57.829 INFO:tasks.workunit.client.0.smithi027.stdout:4/81: symlink d6/l20 0 2021-11-03T17:13:57.847 INFO:tasks.workunit.client.0.smithi027.stdout:2/89: dwrite d0/f11 [0,4194304] 0 2021-11-03T17:13:57.848 INFO:tasks.workunit.client.0.smithi027.stdout:4/82: dwrite f4 [0,4194304] 0 2021-11-03T17:13:57.849 INFO:tasks.workunit.client.0.smithi027.stdout:4/83: write f5 [661625,41571] 0 2021-11-03T17:13:57.850 INFO:tasks.workunit.client.0.smithi027.stdout:4/84: truncate f4 4941929 0 2021-11-03T17:13:57.852 INFO:tasks.workunit.client.0.smithi027.stdout:2/90: mkdir d0/d1b 0 2021-11-03T17:13:57.855 INFO:tasks.workunit.client.0.smithi027.stdout:4/85: unlink d6/d9/ff 0 2021-11-03T17:13:57.855 INFO:tasks.workunit.client.0.smithi027.stdout:4/86: write d6/d9/f1e [853258,49966] 0 2021-11-03T17:13:57.856 INFO:tasks.workunit.client.0.smithi027.stdout:2/91: mkdir d0/d1b/d1c 0 2021-11-03T17:13:57.857 INFO:tasks.workunit.client.0.smithi027.stdout:4/87: mkdir d6/db/d21 0 2021-11-03T17:13:57.860 INFO:tasks.workunit.client.0.smithi027.stdout:2/92: unlink d0/f7 0 2021-11-03T17:13:57.863 INFO:tasks.workunit.client.0.smithi027.stdout:4/88: unlink d6/l20 0 2021-11-03T17:13:57.864 INFO:tasks.workunit.client.0.smithi027.stdout:4/89: truncate d6/db/fc 585763 0 2021-11-03T17:13:57.876 INFO:tasks.workunit.client.0.smithi027.stdout:4/90: truncate d6/f8 1043354 0 2021-11-03T17:13:57.883 INFO:tasks.workunit.client.0.smithi027.stdout:2/93: dwrite d0/f11 [0,4194304] 0 2021-11-03T17:13:57.887 INFO:tasks.workunit.client.0.smithi027.stdout:4/91: dread d6/d9/f1e [0,4194304] 0 2021-11-03T17:13:57.889 INFO:tasks.workunit.client.0.smithi027.stdout:4/92: write f4 [1100965,74817] 0 2021-11-03T17:13:57.889 INFO:tasks.workunit.client.0.smithi027.stdout:4/93: stat d6/d9 0 2021-11-03T17:13:57.889 INFO:tasks.workunit.client.0.smithi027.stdout:4/94: dread - d6/f1f zero size 2021-11-03T17:13:57.892 INFO:tasks.workunit.client.0.smithi027.stdout:2/94: truncate d0/d5/f4 2865299 0 2021-11-03T17:13:57.893 INFO:tasks.workunit.client.0.smithi027.stdout:1/63: sync 2021-11-03T17:13:57.896 INFO:tasks.workunit.client.0.smithi027.stdout:1/64: creat d6/d11/f12 x:0 0 0 2021-11-03T17:13:57.899 INFO:tasks.workunit.client.0.smithi027.stdout:1/65: dread f2 [0,4194304] 0 2021-11-03T17:13:57.900 INFO:tasks.workunit.client.0.smithi027.stdout:2/95: write d0/d5/f13 [1165537,9990] 0 2021-11-03T17:13:57.900 INFO:tasks.workunit.client.0.smithi027.stdout:2/96: fsync d0/d5/f8 0 2021-11-03T17:13:57.914 INFO:tasks.workunit.client.0.smithi027.stdout:1/66: dread d6/de/ff [0,4194304] 0 2021-11-03T17:13:57.915 INFO:tasks.workunit.client.0.smithi027.stdout:1/67: mkdir d6/de/d13 0 2021-11-03T17:13:57.917 INFO:tasks.workunit.client.0.smithi027.stdout:1/68: mkdir d6/d11/d14 0 2021-11-03T17:13:57.917 INFO:tasks.workunit.client.0.smithi027.stdout:1/69: chown l3 57 1 2021-11-03T17:13:57.917 INFO:tasks.workunit.client.0.smithi027.stdout:1/70: chown d6/de/d13 3501835 1 2021-11-03T17:13:57.918 INFO:tasks.workunit.client.0.smithi027.stdout:1/71: mkdir d6/de/d13/d15 0 2021-11-03T17:13:57.918 INFO:tasks.workunit.client.0.smithi027.stdout:1/72: chown d6 163402599 1 2021-11-03T17:13:57.919 INFO:tasks.workunit.client.0.smithi027.stdout:1/73: creat d6/de/d13/d15/f16 x:0 0 0 2021-11-03T17:13:57.920 INFO:tasks.workunit.client.0.smithi027.stdout:1/74: dread d6/fd [0,4194304] 0 2021-11-03T17:13:57.921 INFO:tasks.workunit.client.0.smithi027.stdout:1/75: creat d6/d11/d14/f17 x:0 0 0 2021-11-03T17:13:57.922 INFO:tasks.workunit.client.0.smithi027.stdout:2/97: dwrite d0/f10 [8388608,4194304] 0 2021-11-03T17:13:57.923 INFO:tasks.workunit.client.0.smithi027.stdout:1/76: link d6/d11/f12 d6/f18 0 2021-11-03T17:13:57.934 INFO:tasks.workunit.client.0.smithi027.stdout:2/98: dread d0/d5/f3 [0,4194304] 0 2021-11-03T17:13:57.935 INFO:tasks.workunit.client.0.smithi027.stdout:2/99: symlink d0/d1b/d1c/l1d 0 2021-11-03T17:13:57.941 INFO:tasks.workunit.client.0.smithi027.stdout:1/77: dwrite d6/f18 [0,4194304] 0 2021-11-03T17:13:57.941 INFO:tasks.workunit.client.0.smithi027.stdout:1/78: chown d6/f18 123807 1 2021-11-03T17:13:57.943 INFO:tasks.workunit.client.0.smithi027.stdout:1/79: write d6/fa [2663772,58767] 0 2021-11-03T17:13:57.944 INFO:tasks.workunit.client.0.smithi027.stdout:1/80: creat d6/de/f19 x:0 0 0 2021-11-03T17:13:57.949 INFO:tasks.workunit.client.0.smithi027.stdout:1/81: creat d6/de/f1a x:0 0 0 2021-11-03T17:13:57.953 INFO:tasks.workunit.client.0.smithi027.stdout:1/82: rmdir d6/d11 39 2021-11-03T17:13:57.954 INFO:tasks.workunit.client.0.smithi027.stdout:1/83: write d6/fd [621032,43561] 0 2021-11-03T17:13:57.957 INFO:tasks.workunit.client.0.smithi027.stdout:1/84: rmdir d6/de 39 2021-11-03T17:13:57.957 INFO:tasks.workunit.client.0.smithi027.stdout:1/85: rename d6/d11 to d6/d11/d14/d1b 22 2021-11-03T17:13:57.961 INFO:tasks.workunit.client.0.smithi027.stdout:2/100: dwrite d0/d5/f4 [0,4194304] 0 2021-11-03T17:13:57.964 INFO:tasks.workunit.client.0.smithi027.stdout:2/101: creat d0/d5/f1e x:0 0 0 2021-11-03T17:13:57.965 INFO:tasks.workunit.client.0.smithi027.stdout:2/102: write d0/d5/f3 [424252,1512] 0 2021-11-03T17:13:57.966 INFO:tasks.workunit.client.0.smithi027.stdout:2/103: chown d0/d5/la 569992 1 2021-11-03T17:13:57.967 INFO:tasks.workunit.client.0.smithi027.stdout:2/104: symlink d0/d1b/l1f 0 2021-11-03T17:13:57.967 INFO:tasks.workunit.client.0.smithi027.stdout:2/105: mknod d0/d1b/c20 1 2021-11-03T17:13:57.968 INFO:tasks.workunit.client.0.smithi027.stdout:2/106: write d0/d5/f1e [28506,15455] 0 2021-11-03T17:13:57.976 INFO:tasks.workunit.client.0.smithi027.stdout:1/86: dwrite d6/fc [0,4194304] 0 2021-11-03T17:13:57.978 INFO:tasks.workunit.client.0.smithi027.stdout:1/87: rename d6/fc to d6/de/f1c 0 2021-11-03T17:13:57.978 INFO:tasks.workunit.client.0.smithi027.stdout:1/88: read d6/de/ff [1107080,78645] 0 2021-11-03T17:13:57.979 INFO:tasks.workunit.client.0.smithi027.stdout:1/89: mknod d6/c1d 1 2021-11-03T17:13:57.979 INFO:tasks.workunit.client.0.smithi027.stdout:1/90: mknod d6/de/d13/d15/c1e 1 2021-11-03T17:13:57.980 INFO:tasks.workunit.client.0.smithi027.stdout:2/107: truncate d0/d5/f3 21575 0 2021-11-03T17:13:57.980 INFO:tasks.workunit.client.0.smithi027.stdout:1/91: mkdir d6/d11/d14/d1f 0 2021-11-03T17:13:57.982 INFO:tasks.workunit.client.0.smithi027.stdout:1/92: creat d6/de/d13/f20 x:0 0 0 2021-11-03T17:13:57.985 INFO:tasks.workunit.client.0.smithi027.stdout:1/93: creat d6/d11/d14/f21 x:0 0 0 2021-11-03T17:13:58.002 INFO:tasks.workunit.client.0.smithi027.stdout:1/94: dwrite f2 [0,4194304] 0 2021-11-03T17:13:58.005 INFO:tasks.workunit.client.0.smithi027.stdout:1/95: rename d6/d11/d14/f21 to d6/d11/d14/d1f/f22 0 2021-11-03T17:13:58.005 INFO:tasks.workunit.client.0.smithi027.stdout:1/96: chown d6/d11/d14/d1f 31438505 1 2021-11-03T17:13:58.006 INFO:tasks.workunit.client.0.smithi027.stdout:1/97: link d6/f18 d6/f23 0 2021-11-03T17:13:58.008 INFO:tasks.workunit.client.0.smithi027.stdout:1/98: write d6/f18 [3035327,80761] 0 2021-11-03T17:13:58.008 INFO:tasks.workunit.client.0.smithi027.stdout:1/99: truncate d6/f23 4343124 0 2021-11-03T17:13:58.008 INFO:tasks.workunit.client.0.smithi027.stdout:1/100: truncate d6/fd 913790 0 2021-11-03T17:13:58.009 INFO:tasks.workunit.client.0.smithi027.stdout:1/101: rename d6/d11/f12 to d6/d11/d14/d1f/f24 0 2021-11-03T17:13:58.010 INFO:tasks.workunit.client.0.smithi027.stdout:1/102: write d6/fd [822350,99274] 0 2021-11-03T17:13:58.011 INFO:tasks.workunit.client.0.smithi027.stdout:1/103: write d6/d11/d14/d1f/f24 [315663,31740] 0 2021-11-03T17:13:58.011 INFO:tasks.workunit.client.0.smithi027.stdout:1/104: chown d6/f18 687 1 2021-11-03T17:13:58.012 INFO:tasks.workunit.client.0.smithi027.stdout:1/105: dread - d6/de/d13/f20 zero size 2021-11-03T17:13:58.012 INFO:tasks.workunit.client.0.smithi027.stdout:2/108: dwrite d0/d5/f3 [0,4194304] 0 2021-11-03T17:13:58.012 INFO:tasks.workunit.client.0.smithi027.stdout:1/106: mkdir d6/d11/d14/d1f/d25 0 2021-11-03T17:13:58.013 INFO:tasks.workunit.client.0.smithi027.stdout:2/109: chown d0/l19 8 1 2021-11-03T17:13:58.013 INFO:tasks.workunit.client.0.smithi027.stdout:2/110: creat d0/d5/de/f21 x:0 0 0 2021-11-03T17:13:58.024 INFO:tasks.workunit.client.0.smithi027.stdout:2/111: dread d0/d5/f13 [0,4194304] 0 2021-11-03T17:13:58.026 INFO:tasks.workunit.client.0.smithi027.stdout:1/107: dwrite d6/de/f19 [0,4194304] 0 2021-11-03T17:13:58.044 INFO:tasks.workunit.client.0.smithi027.stdout:1/108: dwrite d6/fa [0,4194304] 0 2021-11-03T17:13:58.057 INFO:tasks.workunit.client.0.smithi027.stdout:1/109: dwrite d6/fd [0,4194304] 0 2021-11-03T17:13:58.058 INFO:tasks.workunit.client.0.smithi027.stdout:1/110: fdatasync d6/d11/d14/d1f/f24 0 2021-11-03T17:13:58.087 INFO:tasks.workunit.client.0.smithi027.stdout:1/111: dwrite d6/de/f19 [4194304,4194304] 0 2021-11-03T17:13:58.087 INFO:tasks.workunit.client.0.smithi027.stdout:1/112: mknod d6/d11/d14/d1f/d25/c26 1 2021-11-03T17:13:58.089 INFO:tasks.workunit.client.0.smithi027.stdout:1/113: link d6/de/f1a d6/d11/f27 0 2021-11-03T17:13:58.106 INFO:tasks.workunit.client.0.smithi027.stdout:1/114: dwrite d6/de/d13/f20 [0,4194304] 0 2021-11-03T17:13:58.107 INFO:tasks.workunit.client.0.smithi027.stdout:1/115: write d6/de/ff [2946409,110140] 0 2021-11-03T17:13:58.113 INFO:tasks.workunit.client.0.smithi027.stdout:1/116: mkdir d6/d11/d14/d1f/d25/d28 0 2021-11-03T17:13:58.115 INFO:tasks.workunit.client.0.smithi027.stdout:1/117: link d6/d11/d14/d1f/f22 d6/f29 0 2021-11-03T17:13:58.115 INFO:tasks.workunit.client.0.smithi027.stdout:1/118: mknod d6/d11/d14/c2a 1 2021-11-03T17:13:58.117 INFO:tasks.workunit.client.0.smithi027.stdout:1/119: unlink d6/de/d13/f20 0 2021-11-03T17:13:58.117 INFO:tasks.workunit.client.0.smithi027.stdout:1/120: fdatasync d6/f23 0 2021-11-03T17:13:58.118 INFO:tasks.workunit.client.0.smithi027.stdout:1/121: truncate d6/de/f1a 363071 0 2021-11-03T17:13:58.119 INFO:tasks.workunit.client.0.smithi027.stdout:1/122: creat d6/d11/d14/d1f/f2b x:0 0 0 2021-11-03T17:13:58.121 INFO:tasks.workunit.client.0.smithi027.stdout:1/123: creat d6/de/f2c x:0 0 0 2021-11-03T17:13:58.122 INFO:tasks.workunit.client.0.smithi027.stdout:1/124: write d6/fa [2410471,121470] 0 2021-11-03T17:13:58.124 INFO:tasks.workunit.client.0.smithi027.stdout:1/125: creat d6/d11/d14/d1f/d25/f2d x:0 0 0 2021-11-03T17:13:58.124 INFO:tasks.workunit.client.0.smithi027.stdout:1/126: dread - d6/de/f2c zero size 2021-11-03T17:13:58.126 INFO:tasks.workunit.client.0.smithi027.stdout:1/127: symlink d6/d11/d14/d1f/d25/l2e 0 2021-11-03T17:13:58.126 INFO:tasks.workunit.client.0.smithi027.stdout:1/128: dread - d6/d11/d14/f17 zero size 2021-11-03T17:13:58.127 INFO:tasks.workunit.client.0.smithi027.stdout:1/129: link d6/de/d13/d15/f16 d6/de/d13/d15/f2f 0 2021-11-03T17:13:58.128 INFO:tasks.workunit.client.0.smithi027.stdout:1/130: chown d6/d11 7220445 1 2021-11-03T17:13:58.140 INFO:tasks.workunit.client.0.smithi027.stdout:1/131: dwrite f2 [4194304,4194304] 0 2021-11-03T17:13:58.163 INFO:tasks.workunit.client.0.smithi027.stdout:1/132: dwrite d6/d11/d14/f17 [0,4194304] 0 2021-11-03T17:13:58.164 INFO:tasks.workunit.client.0.smithi027.stdout:1/133: dread - d6/d11/d14/d1f/f2b zero size 2021-11-03T17:13:58.164 INFO:tasks.workunit.client.0.smithi027.stdout:1/134: fsync d6/d11/d14/d1f/f22 0 2021-11-03T17:13:58.166 INFO:tasks.workunit.client.0.smithi027.stdout:1/135: symlink d6/d11/d14/l30 0 2021-11-03T17:13:58.166 INFO:tasks.workunit.client.0.smithi027.stdout:1/136: write d6/de/f1a [1288803,126706] 0 2021-11-03T17:13:58.166 INFO:tasks.workunit.client.0.smithi027.stdout:1/137: chown d6/d11/d14/d1f/d25/l2e 5 1 2021-11-03T17:13:58.180 INFO:tasks.workunit.client.0.smithi027.stdout:1/138: dwrite d6/fd [0,4194304] 0 2021-11-03T17:13:58.182 INFO:tasks.workunit.client.0.smithi027.stdout:1/139: mkdir d6/d11/d14/d1f/d31 0 2021-11-03T17:13:58.183 INFO:tasks.workunit.client.0.smithi027.stdout:1/140: mkdir d6/d11/d14/d1f/d25/d28/d32 0 2021-11-03T17:13:58.183 INFO:tasks.workunit.client.0.smithi027.stdout:1/141: write d6/d11/d14/d1f/f2b [544123,32085] 0 2021-11-03T17:13:58.184 INFO:tasks.workunit.client.0.smithi027.stdout:1/142: creat d6/d11/d14/d1f/d31/f33 x:0 0 0 2021-11-03T17:13:58.186 INFO:tasks.workunit.client.0.smithi027.stdout:1/143: write d6/fd [937957,129258] 0 2021-11-03T17:13:58.186 INFO:tasks.workunit.client.0.smithi027.stdout:1/144: truncate d6/d11/d14/d1f/d25/f2d 194996 0 2021-11-03T17:13:58.191 INFO:tasks.workunit.client.0.smithi027.stdout:1/145: dread f2 [4194304,4194304] 0 2021-11-03T17:13:58.712 INFO:tasks.workunit.client.0.smithi027.stdout:5/58: sync 2021-11-03T17:13:58.713 INFO:tasks.workunit.client.0.smithi027.stdout:3/41: sync 2021-11-03T17:13:58.713 INFO:tasks.workunit.client.0.smithi027.stdout:5/59: chown d2/f3 1557223 1 2021-11-03T17:13:58.714 INFO:tasks.workunit.client.0.smithi027.stdout:3/42: unlink f0 0 2021-11-03T17:13:58.714 INFO:tasks.workunit.client.0.smithi027.stdout:3/43: dread - d1/f7 zero size 2021-11-03T17:13:58.715 INFO:tasks.workunit.client.0.smithi027.stdout:5/60: symlink d2/d12/l17 0 2021-11-03T17:13:58.716 INFO:tasks.workunit.client.0.smithi027.stdout:3/44: creat d1/fc x:0 0 0 2021-11-03T17:13:58.717 INFO:tasks.workunit.client.0.smithi027.stdout:5/61: creat d2/f18 x:0 0 0 2021-11-03T17:13:58.750 INFO:tasks.workunit.client.0.smithi027.stdout:3/45: dwrite d1/f9 [0,4194304] 0 2021-11-03T17:13:58.751 INFO:tasks.workunit.client.0.smithi027.stdout:3/46: mknod d1/cd 1 2021-11-03T17:13:58.751 INFO:tasks.workunit.client.0.smithi027.stdout:3/47: readlink d1/l5 0 2021-11-03T17:13:58.751 INFO:tasks.workunit.client.0.smithi027.stdout:5/62: dwrite d2/d4/df/f11 [0,4194304] 0 2021-11-03T17:13:58.751 INFO:tasks.workunit.client.0.smithi027.stdout:5/63: dread - d2/d4/fb zero size 2021-11-03T17:13:58.753 INFO:tasks.workunit.client.0.smithi027.stdout:3/48: rename d1/l5 to d1/db/le 0 2021-11-03T17:13:58.753 INFO:tasks.workunit.client.0.smithi027.stdout:3/49: chown d1/db 0 1 2021-11-03T17:13:58.753 INFO:tasks.workunit.client.0.smithi027.stdout:3/50: mknod d1/cf 1 2021-11-03T17:13:58.753 INFO:tasks.workunit.client.0.smithi027.stdout:5/64: dread d2/f7 [0,4194304] 0 2021-11-03T17:13:58.754 INFO:tasks.workunit.client.0.smithi027.stdout:5/65: dread - d2/d4/f13 zero size 2021-11-03T17:13:58.754 INFO:tasks.workunit.client.0.smithi027.stdout:5/66: dread - d2/d4/f13 zero size 2021-11-03T17:13:58.754 INFO:tasks.workunit.client.0.smithi027.stdout:5/67: fsync d2/d4/df/f15 0 2021-11-03T17:13:58.754 INFO:tasks.workunit.client.0.smithi027.stdout:5/68: mknod d2/d4/c19 1 2021-11-03T17:13:58.777 INFO:tasks.workunit.client.0.smithi027.stdout:5/69: dwrite d2/d4/f5 [0,4194304] 0 2021-11-03T17:13:58.778 INFO:tasks.workunit.client.0.smithi027.stdout:5/70: write d2/f18 [1001086,84715] 0 2021-11-03T17:13:58.778 INFO:tasks.workunit.client.0.smithi027.stdout:5/71: chown d2 0 1 2021-11-03T17:13:58.778 INFO:tasks.workunit.client.0.smithi027.stdout:3/51: dwrite d1/fa [0,4194304] 0 2021-11-03T17:13:58.780 INFO:tasks.workunit.client.0.smithi027.stdout:6/42: sync 2021-11-03T17:13:58.781 INFO:tasks.workunit.client.0.smithi027.stdout:3/52: creat d1/f10 x:0 0 0 2021-11-03T17:13:58.781 INFO:tasks.workunit.client.0.smithi027.stdout:3/53: read - d1/fc zero size 2021-11-03T17:13:58.781 INFO:tasks.workunit.client.0.smithi027.stdout:3/54: write d1/fc [366390,66106] 0 2021-11-03T17:13:58.782 INFO:tasks.workunit.client.0.smithi027.stdout:3/55: fsync d1/f7 0 2021-11-03T17:13:58.782 INFO:tasks.workunit.client.0.smithi027.stdout:3/56: rename d1/db to d1/db/d11 22 2021-11-03T17:13:58.782 INFO:tasks.workunit.client.0.smithi027.stdout:3/57: chown d1/f6 1459736136 1 2021-11-03T17:13:58.783 INFO:tasks.workunit.client.0.smithi027.stdout:6/43: truncate f7 2622673 0 2021-11-03T17:13:58.783 INFO:tasks.workunit.client.0.smithi027.stdout:6/44: chown f2 5 1 2021-11-03T17:13:58.785 INFO:tasks.workunit.client.0.smithi027.stdout:3/58: rename d1/f10 to d1/db/f12 0 2021-11-03T17:13:58.785 INFO:tasks.workunit.client.0.smithi027.stdout:3/59: chown d1/fa 13 1 2021-11-03T17:13:58.787 INFO:tasks.workunit.client.0.smithi027.stdout:3/60: creat d1/f13 x:0 0 0 2021-11-03T17:13:58.788 INFO:tasks.workunit.client.0.smithi027.stdout:3/61: creat d1/f14 x:0 0 0 2021-11-03T17:13:58.800 INFO:tasks.workunit.client.0.smithi027.stdout:5/72: dwrite d2/d4/fe [0,4194304] 0 2021-11-03T17:13:58.803 INFO:tasks.workunit.client.0.smithi027.stdout:3/62: getdents d1/db 0 2021-11-03T17:13:58.809 INFO:tasks.workunit.client.0.smithi027.stdout:6/45: dwrite f2 [0,4194304] 0 2021-11-03T17:13:58.810 INFO:tasks.workunit.client.0.smithi027.stdout:6/46: fsync f5 0 2021-11-03T17:13:58.814 INFO:tasks.workunit.client.0.smithi027.stdout:3/63: dread d1/fa [0,4194304] 0 2021-11-03T17:13:58.814 INFO:tasks.workunit.client.0.smithi027.stdout:3/64: write d1/db/f12 [877103,34001] 0 2021-11-03T17:13:58.815 INFO:tasks.workunit.client.0.smithi027.stdout:3/65: chown d1/f14 14 1 2021-11-03T17:13:58.815 INFO:tasks.workunit.client.0.smithi027.stdout:3/66: mknod d1/db/c15 1 2021-11-03T17:13:58.819 INFO:tasks.workunit.client.0.smithi027.stdout:5/73: dwrite d2/d4/df/f15 [0,4194304] 0 2021-11-03T17:13:58.819 INFO:tasks.workunit.client.0.smithi027.stdout:5/74: mknod d2/d4/c1a 1 2021-11-03T17:13:58.820 INFO:tasks.workunit.client.0.smithi027.stdout:5/75: write d2/d4/f5 [962196,35407] 0 2021-11-03T17:13:58.842 INFO:tasks.workunit.client.0.smithi027.stdout:6/47: dwrite f7 [0,4194304] 0 2021-11-03T17:13:58.842 INFO:tasks.workunit.client.0.smithi027.stdout:6/48: write f0 [2738470,123497] 0 2021-11-03T17:13:58.843 INFO:tasks.workunit.client.0.smithi027.stdout:3/67: dwrite d1/f6 [0,4194304] 0 2021-11-03T17:13:58.844 INFO:tasks.workunit.client.0.smithi027.stdout:5/76: truncate d2/d4/df/f15 3313383 0 2021-11-03T17:13:58.845 INFO:tasks.workunit.client.0.smithi027.stdout:5/77: write d2/f7 [1663914,32956] 0 2021-11-03T17:13:58.845 INFO:tasks.workunit.client.0.smithi027.stdout:5/78: mknod d2/d4/df/c1b 1 2021-11-03T17:13:58.845 INFO:tasks.workunit.client.0.smithi027.stdout:5/79: truncate d2/f18 1337688 0 2021-11-03T17:13:58.845 INFO:tasks.workunit.client.0.smithi027.stdout:5/80: mknod d2/c1c 1 2021-11-03T17:13:58.846 INFO:tasks.workunit.client.0.smithi027.stdout:6/49: unlink f2 0 2021-11-03T17:13:58.846 INFO:tasks.workunit.client.0.smithi027.stdout:6/50: write f6 [624451,81426] 0 2021-11-03T17:13:58.847 INFO:tasks.workunit.client.0.smithi027.stdout:3/68: fdatasync d1/fc 0 2021-11-03T17:13:58.847 INFO:tasks.workunit.client.0.smithi027.stdout:3/69: mknod d1/c16 1 2021-11-03T17:13:58.849 INFO:tasks.workunit.client.0.smithi027.stdout:5/81: rmdir d2/d4/df 39 2021-11-03T17:13:58.849 INFO:tasks.workunit.client.0.smithi027.stdout:5/82: chown d2/d4/df 2789 1 2021-11-03T17:13:58.849 INFO:tasks.workunit.client.0.smithi027.stdout:5/83: mknod d2/c1d 1 2021-11-03T17:13:58.851 INFO:tasks.workunit.client.0.smithi027.stdout:6/51: unlink f5 0 2021-11-03T17:13:58.851 INFO:tasks.workunit.client.0.smithi027.stdout:6/52: write f6 [1509033,60985] 0 2021-11-03T17:13:58.853 INFO:tasks.workunit.client.0.smithi027.stdout:5/84: creat d2/f1e x:0 0 0 2021-11-03T17:13:58.853 INFO:tasks.workunit.client.0.smithi027.stdout:5/85: fsync d2/f3 0 2021-11-03T17:13:58.854 INFO:tasks.workunit.client.0.smithi027.stdout:5/86: mkdir d2/d1f 0 2021-11-03T17:13:58.856 INFO:tasks.workunit.client.0.smithi027.stdout:5/87: mkdir d2/d4/df/d20 0 2021-11-03T17:13:58.856 INFO:tasks.workunit.client.0.smithi027.stdout:5/88: rename d2/d4/df to d2/d4/df/d20/d21 22 2021-11-03T17:13:58.863 INFO:tasks.workunit.client.0.smithi027.stdout:6/53: dread f7 [0,4194304] 0 2021-11-03T17:13:58.865 INFO:tasks.workunit.client.0.smithi027.stdout:6/54: write f0 [2614592,89407] 0 2021-11-03T17:13:58.868 INFO:tasks.workunit.client.0.smithi027.stdout:3/70: dwrite d1/f13 [0,4194304] 0 2021-11-03T17:13:58.868 INFO:tasks.workunit.client.0.smithi027.stdout:3/71: chown d1/f6 13715199 1 2021-11-03T17:13:58.869 INFO:tasks.workunit.client.0.smithi027.stdout:3/72: chown d1/f13 271 1 2021-11-03T17:13:58.869 INFO:tasks.workunit.client.0.smithi027.stdout:3/73: write d1/f14 [814138,84633] 0 2021-11-03T17:13:58.870 INFO:tasks.workunit.client.0.smithi027.stdout:3/74: write d1/f13 [508566,107019] 0 2021-11-03T17:13:58.871 INFO:tasks.workunit.client.0.smithi027.stdout:6/55: dread f7 [0,4194304] 0 2021-11-03T17:13:58.872 INFO:tasks.workunit.client.0.smithi027.stdout:3/75: rmdir d1 39 2021-11-03T17:13:58.874 INFO:tasks.workunit.client.0.smithi027.stdout:3/76: creat d1/f17 x:0 0 0 2021-11-03T17:13:58.875 INFO:tasks.workunit.client.0.smithi027.stdout:3/77: creat d1/f18 x:0 0 0 2021-11-03T17:13:58.878 INFO:tasks.workunit.client.0.smithi027.stdout:6/56: dread f7 [0,4194304] 0 2021-11-03T17:13:58.878 INFO:tasks.workunit.client.0.smithi027.stdout:6/57: readlink - no filename 2021-11-03T17:13:58.879 INFO:tasks.workunit.client.0.smithi027.stdout:6/58: getdents . 0 2021-11-03T17:13:58.879 INFO:tasks.workunit.client.0.smithi027.stdout:6/59: chown f6 0 1 2021-11-03T17:13:58.880 INFO:tasks.workunit.client.0.smithi027.stdout:6/60: readlink - no filename 2021-11-03T17:13:58.880 INFO:tasks.workunit.client.0.smithi027.stdout:6/61: write f0 [2900941,89312] 0 2021-11-03T17:13:58.880 INFO:tasks.workunit.client.0.smithi027.stdout:6/62: write f0 [2364576,108855] 0 2021-11-03T17:13:58.887 INFO:tasks.workunit.client.0.smithi027.stdout:5/89: dwrite d2/f7 [0,4194304] 0 2021-11-03T17:13:58.889 INFO:tasks.workunit.client.0.smithi027.stdout:3/78: dread d1/f6 [0,4194304] 0 2021-11-03T17:13:58.891 INFO:tasks.workunit.client.0.smithi027.stdout:5/90: link d2/d12/l17 d2/d4/l22 0 2021-11-03T17:13:58.891 INFO:tasks.workunit.client.0.smithi027.stdout:5/91: write d2/f18 [1953034,57884] 0 2021-11-03T17:13:58.891 INFO:tasks.workunit.client.0.smithi027.stdout:5/92: mknod d2/c23 1 2021-11-03T17:13:58.892 INFO:tasks.workunit.client.0.smithi027.stdout:5/93: getdents d2/d1f 0 2021-11-03T17:13:58.897 INFO:tasks.workunit.client.0.smithi027.stdout:5/94: dread d2/d4/fe [0,4194304] 0 2021-11-03T17:13:58.898 INFO:tasks.workunit.client.0.smithi027.stdout:5/95: write d2/d4/fe [3117012,45071] 0 2021-11-03T17:13:58.898 INFO:tasks.workunit.client.0.smithi027.stdout:5/96: chown d2/d4/df 72 1 2021-11-03T17:13:58.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:58 smithi115 conmon[34670]: cluster 2021-11-03T17:13:58.602043+0000 mgr.smithi027.ufhhvb (mgr. 2021-11-03T17:13:58.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:13:58 smithi115 conmon[34670]: 14162) 296 : cluster [DBG] pgmap v246: 65 pgs: 65 active+clean; 556 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 32 MiB/s wr, 806 op/s 2021-11-03T17:13:59.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:58 smithi027 conmon[32322]: cluster 2021-11-03T17:13:58.602043+0000 mgr.smithi027.ufhhvb (mgr.14162) 2021-11-03T17:13:59.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:13:58 smithi027 conmon[32322]: 296 : cluster [DBG] pgmap v246: 65 pgs: 65 active+clean; 556 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 12 MiB/s rd, 32 MiB/s wr, 806 op/s 2021-11-03T17:13:59.051 INFO:tasks.workunit.client.0.smithi027.stdout:0/142: sync 2021-11-03T17:13:59.051 INFO:tasks.workunit.client.0.smithi027.stdout:7/89: sync 2021-11-03T17:13:59.051 INFO:tasks.workunit.client.0.smithi027.stdout:8/106: sync 2021-11-03T17:13:59.051 INFO:tasks.workunit.client.0.smithi027.stdout:9/125: sync 2021-11-03T17:13:59.052 INFO:tasks.workunit.client.0.smithi027.stdout:4/95: sync 2021-11-03T17:13:59.052 INFO:tasks.workunit.client.0.smithi027.stdout:0/143: chown d9/da/l29 199135379 1 2021-11-03T17:13:59.052 INFO:tasks.workunit.client.0.smithi027.stdout:9/126: dread - db/f1f zero size 2021-11-03T17:13:59.053 INFO:tasks.workunit.client.0.smithi027.stdout:7/90: symlink d0/l1d 0 2021-11-03T17:13:59.053 INFO:tasks.workunit.client.0.smithi027.stdout:8/107: creat d7/d10/d18/d1a/f1f x:0 0 0 2021-11-03T17:13:59.055 INFO:tasks.workunit.client.0.smithi027.stdout:4/96: symlink d6/l22 0 2021-11-03T17:13:59.055 INFO:tasks.workunit.client.0.smithi027.stdout:4/97: dread - d6/f1f zero size 2021-11-03T17:13:59.056 INFO:tasks.workunit.client.0.smithi027.stdout:4/98: chown d6/d9/f1e 0 1 2021-11-03T17:13:59.056 INFO:tasks.workunit.client.0.smithi027.stdout:0/144: link d9/da/l11 d9/l2f 0 2021-11-03T17:13:59.058 INFO:tasks.workunit.client.0.smithi027.stdout:8/108: mkdir d7/d10/d18/d20 0 2021-11-03T17:13:59.060 INFO:tasks.workunit.client.0.smithi027.stdout:7/91: mkdir d0/d2/d10/d1e 0 2021-11-03T17:13:59.061 INFO:tasks.workunit.client.0.smithi027.stdout:0/145: mkdir d9/da/d30 0 2021-11-03T17:13:59.062 INFO:tasks.workunit.client.0.smithi027.stdout:7/92: mkdir d0/d2/d1f 0 2021-11-03T17:13:59.067 INFO:tasks.workunit.client.0.smithi027.stdout:8/109: dread d7/f1b [0,4194304] 0 2021-11-03T17:13:59.074 INFO:tasks.workunit.client.0.smithi027.stdout:4/99: dwrite d6/d9/f1e [0,4194304] 0 2021-11-03T17:13:59.074 INFO:tasks.workunit.client.0.smithi027.stdout:8/110: dread d7/f8 [0,4194304] 0 2021-11-03T17:13:59.075 INFO:tasks.workunit.client.0.smithi027.stdout:8/111: write d7/d10/d18/d1a/f1f [458603,55214] 0 2021-11-03T17:13:59.076 INFO:tasks.workunit.client.0.smithi027.stdout:7/93: rmdir d0/d2 39 2021-11-03T17:13:59.080 INFO:tasks.workunit.client.0.smithi027.stdout:4/100: unlink l1 0 2021-11-03T17:13:59.080 INFO:tasks.workunit.client.0.smithi027.stdout:4/101: fsync d6/f1f 0 2021-11-03T17:13:59.080 INFO:tasks.workunit.client.0.smithi027.stdout:4/102: mknod d6/db/d21/c23 1 2021-11-03T17:13:59.081 INFO:tasks.workunit.client.0.smithi027.stdout:8/112: mkdir d7/d10/d21 0 2021-11-03T17:13:59.084 INFO:tasks.workunit.client.0.smithi027.stdout:8/113: symlink d7/l22 0 2021-11-03T17:13:59.084 INFO:tasks.workunit.client.0.smithi027.stdout:8/114: write d7/d10/f16 [4985973,66965] 0 2021-11-03T17:13:59.087 INFO:tasks.workunit.client.0.smithi027.stdout:8/115: read d7/d10/f15 [349803,18835] 0 2021-11-03T17:13:59.088 INFO:tasks.workunit.client.0.smithi027.stdout:8/116: stat d7/d10/d18/d20 0 2021-11-03T17:13:59.088 INFO:tasks.workunit.client.0.smithi027.stdout:8/117: fsync d7/d10/d18/d1a/f1f 0 2021-11-03T17:13:59.088 INFO:tasks.workunit.client.0.smithi027.stdout:8/118: chown d7/d10/d21 6140 1 2021-11-03T17:13:59.090 INFO:tasks.workunit.client.0.smithi027.stdout:8/119: creat d7/f23 x:0 0 0 2021-11-03T17:13:59.090 INFO:tasks.workunit.client.0.smithi027.stdout:8/120: fdatasync d7/fe 0 2021-11-03T17:13:59.090 INFO:tasks.workunit.client.0.smithi027.stdout:8/121: fsync d7/fe 0 2021-11-03T17:13:59.090 INFO:tasks.workunit.client.0.smithi027.stdout:8/122: mknod d7/d10/d21/c24 1 2021-11-03T17:13:59.091 INFO:tasks.workunit.client.0.smithi027.stdout:8/123: read - d7/f23 zero size 2021-11-03T17:13:59.094 INFO:tasks.workunit.client.0.smithi027.stdout:8/124: dread d7/d10/d18/d1a/f1f [0,4194304] 0 2021-11-03T17:13:59.094 INFO:tasks.workunit.client.0.smithi027.stdout:8/125: chown d7/d10/d18/d1a 202431 1 2021-11-03T17:13:59.103 INFO:tasks.workunit.client.0.smithi027.stdout:7/94: dwrite d0/d2/d10/f1b [0,4194304] 0 2021-11-03T17:13:59.103 INFO:tasks.workunit.client.0.smithi027.stdout:4/103: dwrite f4 [0,4194304] 0 2021-11-03T17:13:59.103 INFO:tasks.workunit.client.0.smithi027.stdout:4/104: chown d6/l18 20 1 2021-11-03T17:13:59.105 INFO:tasks.workunit.client.0.smithi027.stdout:7/95: mkdir d0/d2/d10/d1e/d20 0 2021-11-03T17:13:59.106 INFO:tasks.workunit.client.0.smithi027.stdout:4/105: mkdir d6/d24 0 2021-11-03T17:13:59.108 INFO:tasks.workunit.client.0.smithi027.stdout:4/106: link d6/f8 d6/db/d21/f25 0 2021-11-03T17:13:59.113 INFO:tasks.workunit.client.0.smithi027.stdout:8/126: dwrite d7/d10/d18/f1c [0,4194304] 0 2021-11-03T17:13:59.114 INFO:tasks.workunit.client.0.smithi027.stdout:7/96: dread d0/d2/d10/f1b [0,4194304] 0 2021-11-03T17:13:59.117 INFO:tasks.workunit.client.0.smithi027.stdout:8/127: creat d7/d10/d21/f25 x:0 0 0 2021-11-03T17:13:59.118 INFO:tasks.workunit.client.0.smithi027.stdout:8/128: dread - d7/d10/d21/f25 zero size 2021-11-03T17:13:59.118 INFO:tasks.workunit.client.0.smithi027.stdout:4/107: getdents d6 0 2021-11-03T17:13:59.120 INFO:tasks.workunit.client.0.smithi027.stdout:7/97: read - d0/d2/d10/f19 zero size 2021-11-03T17:13:59.121 INFO:tasks.workunit.client.0.smithi027.stdout:7/98: readlink d0/l1 0 2021-11-03T17:13:59.121 INFO:tasks.workunit.client.0.smithi027.stdout:7/99: read d0/d2/d10/f1b [1409408,77763] 0 2021-11-03T17:13:59.124 INFO:tasks.workunit.client.0.smithi027.stdout:8/129: rename d7/f23 to d7/d10/d18/d1a/f26 0 2021-11-03T17:13:59.125 INFO:tasks.workunit.client.0.smithi027.stdout:7/100: symlink d0/d2/d10/d1e/l21 0 2021-11-03T17:13:59.126 INFO:tasks.workunit.client.0.smithi027.stdout:7/101: write d0/f13 [347016,77409] 0 2021-11-03T17:13:59.127 INFO:tasks.workunit.client.0.smithi027.stdout:7/102: chown d0/l1 877968 1 2021-11-03T17:13:59.128 INFO:tasks.workunit.client.0.smithi027.stdout:4/108: rename d6/d9/f1e to d6/d9/d16/f26 0 2021-11-03T17:13:59.129 INFO:tasks.workunit.client.0.smithi027.stdout:8/130: mkdir d7/d10/d21/d27 0 2021-11-03T17:13:59.130 INFO:tasks.workunit.client.0.smithi027.stdout:8/131: chown d7/f1d 209830525 1 2021-11-03T17:13:59.131 INFO:tasks.workunit.client.0.smithi027.stdout:4/109: mkdir d6/db/d27 0 2021-11-03T17:13:59.132 INFO:tasks.workunit.client.0.smithi027.stdout:4/110: dread - d6/f1f zero size 2021-11-03T17:13:59.134 INFO:tasks.workunit.client.0.smithi027.stdout:8/132: rename d7/d10/d18/d20 to d7/d28 0 2021-11-03T17:13:59.137 INFO:tasks.workunit.client.0.smithi027.stdout:8/133: symlink d7/d28/l29 0 2021-11-03T17:13:59.137 INFO:tasks.workunit.client.0.smithi027.stdout:8/134: write d7/f1b [4955878,8884] 0 2021-11-03T17:13:59.137 INFO:tasks.workunit.client.0.smithi027.stdout:8/135: write d7/f8 [4985131,4242] 0 2021-11-03T17:13:59.138 INFO:tasks.workunit.client.0.smithi027.stdout:7/103: dread d0/f13 [0,4194304] 0 2021-11-03T17:13:59.139 INFO:tasks.workunit.client.0.smithi027.stdout:8/136: dread d7/d10/f15 [0,4194304] 0 2021-11-03T17:13:59.139 INFO:tasks.workunit.client.0.smithi027.stdout:7/104: read d0/f16 [310815,71179] 0 2021-11-03T17:13:59.142 INFO:tasks.workunit.client.0.smithi027.stdout:8/137: symlink d7/d10/d21/l2a 0 2021-11-03T17:13:59.142 INFO:tasks.workunit.client.0.smithi027.stdout:8/138: fdatasync d7/f1d 0 2021-11-03T17:13:59.142 INFO:tasks.workunit.client.0.smithi027.stdout:8/139: chown d7/d28 705696372 1 2021-11-03T17:13:59.143 INFO:tasks.workunit.client.0.smithi027.stdout:7/105: readlink d0/d2/d10/l15 0 2021-11-03T17:13:59.148 INFO:tasks.workunit.client.0.smithi027.stdout:7/106: dread d0/d2/d10/f1b [0,4194304] 0 2021-11-03T17:13:59.149 INFO:tasks.workunit.client.0.smithi027.stdout:7/107: symlink d0/d2/d10/d1e/l22 0 2021-11-03T17:13:59.149 INFO:tasks.workunit.client.0.smithi027.stdout:7/108: write d0/f13 [5142167,4447] 0 2021-11-03T17:13:59.150 INFO:tasks.workunit.client.0.smithi027.stdout:7/109: dread d0/f18 [0,4194304] 0 2021-11-03T17:13:59.151 INFO:tasks.workunit.client.0.smithi027.stdout:7/110: chown d0/d2/d10/d1e/d20 124460 1 2021-11-03T17:13:59.160 INFO:tasks.workunit.client.0.smithi027.stdout:8/140: dwrite d7/d10/f12 [0,4194304] 0 2021-11-03T17:13:59.162 INFO:tasks.workunit.client.0.smithi027.stdout:7/111: dread d0/f13 [0,4194304] 0 2021-11-03T17:13:59.162 INFO:tasks.workunit.client.0.smithi027.stdout:7/112: stat d0/d2/f5 0 2021-11-03T17:13:59.162 INFO:tasks.workunit.client.0.smithi027.stdout:7/113: truncate d0/d2/d10/f1b 4829235 0 2021-11-03T17:13:59.162 INFO:tasks.workunit.client.0.smithi027.stdout:7/114: read d0/f13 [3254016,71677] 0 2021-11-03T17:13:59.163 INFO:tasks.workunit.client.0.smithi027.stdout:8/141: creat d7/d10/f2b x:0 0 0 2021-11-03T17:13:59.165 INFO:tasks.workunit.client.0.smithi027.stdout:8/142: unlink d7/d10/d18/f1c 0 2021-11-03T17:13:59.167 INFO:tasks.workunit.client.0.smithi027.stdout:8/143: rename d7/l22 to d7/l2c 0 2021-11-03T17:13:59.196 INFO:tasks.workunit.client.0.smithi027.stdout:2/112: sync 2021-11-03T17:13:59.200 INFO:tasks.workunit.client.0.smithi027.stdout:8/144: rename d7/d28 to d7/d2d 0 2021-11-03T17:13:59.201 INFO:tasks.workunit.client.0.smithi027.stdout:8/145: fdatasync d7/d10/f2b 0 2021-11-03T17:13:59.202 INFO:tasks.workunit.client.0.smithi027.stdout:8/146: creat d7/d10/d18/d1a/f2e x:0 0 0 2021-11-03T17:13:59.202 INFO:tasks.workunit.client.0.smithi027.stdout:8/147: fsync d7/d10/d18/d1a/f1f 0 2021-11-03T17:13:59.204 INFO:tasks.workunit.client.0.smithi027.stdout:8/148: rmdir d7/d10/d21/d27 0 2021-11-03T17:13:59.204 INFO:tasks.workunit.client.0.smithi027.stdout:8/149: chown d7/d10/d18/d1a/f1f 4034 1 2021-11-03T17:13:59.214 INFO:tasks.workunit.client.0.smithi027.stdout:2/113: dwrite d0/d5/de/f21 [0,4194304] 0 2021-11-03T17:13:59.215 INFO:tasks.workunit.client.0.smithi027.stdout:2/114: fdatasync d0/d5/f8 0 2021-11-03T17:13:59.226 INFO:tasks.workunit.client.0.smithi027.stdout:8/150: link d7/l2c d7/l2f 0 2021-11-03T17:13:59.231 INFO:tasks.workunit.client.0.smithi027.stdout:2/115: rename d0/d1b/l1f to d0/d5/de/l22 0 2021-11-03T17:13:59.232 INFO:tasks.workunit.client.0.smithi027.stdout:8/151: symlink d7/d10/d18/d1a/l30 0 2021-11-03T17:13:59.233 INFO:tasks.workunit.client.0.smithi027.stdout:8/152: stat d7/d10/d18/d1a/f2e 0 2021-11-03T17:13:59.233 INFO:tasks.workunit.client.0.smithi027.stdout:8/153: mknod d7/c31 1 2021-11-03T17:13:59.233 INFO:tasks.workunit.client.0.smithi027.stdout:8/154: dread d7/d10/f15 [0,4194304] 0 2021-11-03T17:13:59.236 INFO:tasks.workunit.client.0.smithi027.stdout:8/155: creat d7/f32 x:0 0 0 2021-11-03T17:13:59.236 INFO:tasks.workunit.client.0.smithi027.stdout:8/156: readlink d7/d10/d18/d1a/l30 0 2021-11-03T17:13:59.238 INFO:tasks.workunit.client.0.smithi027.stdout:8/157: stat d7/l2c 0 2021-11-03T17:13:59.240 INFO:tasks.workunit.client.0.smithi027.stdout:2/116: dread d0/d5/de/f21 [0,4194304] 0 2021-11-03T17:13:59.240 INFO:tasks.workunit.client.0.smithi027.stdout:8/158: write f6 [2193727,107721] 0 2021-11-03T17:13:59.242 INFO:tasks.workunit.client.0.smithi027.stdout:1/146: sync 2021-11-03T17:13:59.244 INFO:tasks.workunit.client.0.smithi027.stdout:8/159: write d7/d10/f16 [167741,36389] 0 2021-11-03T17:13:59.244 INFO:tasks.workunit.client.0.smithi027.stdout:8/160: readlink l4 0 2021-11-03T17:13:59.245 INFO:tasks.workunit.client.0.smithi027.stdout:1/147: write d6/f18 [2658663,67883] 0 2021-11-03T17:13:59.250 INFO:tasks.workunit.client.0.smithi027.stdout:1/148: creat d6/d11/d14/d1f/d31/f34 x:0 0 0 2021-11-03T17:13:59.251 INFO:tasks.workunit.client.0.smithi027.stdout:1/149: getdents d6/d11 0 2021-11-03T17:13:59.253 INFO:tasks.workunit.client.0.smithi027.stdout:1/150: write d6/d11/d14/d1f/f24 [2651250,110599] 0 2021-11-03T17:13:59.264 INFO:tasks.workunit.client.0.smithi027.stdout:8/161: dwrite d7/fe [0,4194304] 0 2021-11-03T17:13:59.283 INFO:tasks.workunit.client.0.smithi027.stdout:1/151: dwrite d6/d11/d14/f17 [0,4194304] 0 2021-11-03T17:13:59.288 INFO:tasks.workunit.client.0.smithi027.stdout:8/162: dwrite f6 [0,4194304] 0 2021-11-03T17:13:59.358 INFO:tasks.workunit.client.0.smithi027.stdout:1/152: creat d6/d11/d14/f35 x:0 0 0 2021-11-03T17:13:59.359 INFO:tasks.workunit.client.0.smithi027.stdout:1/153: mknod d6/de/d13/d15/c36 1 2021-11-03T17:13:59.359 INFO:tasks.workunit.client.0.smithi027.stdout:1/154: chown d6/d11/d14/f17 850 1 2021-11-03T17:13:59.362 INFO:tasks.workunit.client.0.smithi027.stdout:8/163: creat d7/d10/d18/f33 x:0 0 0 2021-11-03T17:13:59.365 INFO:tasks.workunit.client.0.smithi027.stdout:1/155: mkdir d6/d11/d14/d1f/d37 0 2021-11-03T17:13:59.365 INFO:tasks.workunit.client.0.smithi027.stdout:1/156: truncate d6/de/f1c 5239195 0 2021-11-03T17:13:59.366 INFO:tasks.workunit.client.0.smithi027.stdout:1/157: write f2 [9395085,70018] 0 2021-11-03T17:13:59.366 INFO:tasks.workunit.client.0.smithi027.stdout:1/158: creat d6/de/d13/f38 x:0 0 0 2021-11-03T17:13:59.368 INFO:tasks.workunit.client.0.smithi027.stdout:1/159: symlink d6/d11/d14/d1f/d25/d28/l39 0 2021-11-03T17:13:59.378 INFO:tasks.workunit.client.0.smithi027.stdout:1/160: dread d6/fa [0,4194304] 0 2021-11-03T17:13:59.379 INFO:tasks.workunit.client.0.smithi027.stdout:1/161: truncate d6/de/ff 915585 0 2021-11-03T17:13:59.380 INFO:tasks.workunit.client.0.smithi027.stdout:1/162: stat d6/d11/d14/d1f/f22 0 2021-11-03T17:13:59.380 INFO:tasks.workunit.client.0.smithi027.stdout:1/163: dread - d6/d11/d14/f35 zero size 2021-11-03T17:13:59.380 INFO:tasks.workunit.client.0.smithi027.stdout:1/164: truncate d6/d11/f27 1282767 0 2021-11-03T17:13:59.383 INFO:tasks.workunit.client.0.smithi027.stdout:1/165: unlink d6/d11/d14/d1f/d31/f33 0 2021-11-03T17:13:59.385 INFO:tasks.workunit.client.0.smithi027.stdout:1/166: write f2 [6550888,70344] 0 2021-11-03T17:13:59.385 INFO:tasks.workunit.client.0.smithi027.stdout:1/167: chown d6/d11/d14/d1f 6004144 1 2021-11-03T17:13:59.385 INFO:tasks.workunit.client.0.smithi027.stdout:1/168: mknod d6/de/d13/d15/c3a 1 2021-11-03T17:13:59.385 INFO:tasks.workunit.client.0.smithi027.stdout:1/169: dread - d6/d11/d14/f35 zero size 2021-11-03T17:13:59.385 INFO:tasks.workunit.client.0.smithi027.stdout:1/170: dread - d6/de/d13/d15/f2f zero size 2021-11-03T17:13:59.387 INFO:tasks.workunit.client.0.smithi027.stdout:1/171: symlink d6/l3b 0 2021-11-03T17:13:59.388 INFO:tasks.workunit.client.0.smithi027.stdout:1/172: dread d6/d11/d14/d1f/d25/f2d [0,4194304] 0 2021-11-03T17:13:59.396 INFO:tasks.workunit.client.0.smithi027.stdout:1/173: dread d6/fd [0,4194304] 0 2021-11-03T17:13:59.396 INFO:tasks.workunit.client.0.smithi027.stdout:1/174: dread - d6/f29 zero size 2021-11-03T17:13:59.408 INFO:tasks.workunit.client.0.smithi027.stdout:1/175: dwrite d6/de/f2c [0,4194304] 0 2021-11-03T17:13:59.408 INFO:tasks.workunit.client.0.smithi027.stdout:1/176: truncate d6/de/f2c 4285607 0 2021-11-03T17:13:59.408 INFO:tasks.workunit.client.0.smithi027.stdout:1/177: stat d6/d11/d14/d1f/d25/d28/d32 0 2021-11-03T17:13:59.411 INFO:tasks.workunit.client.0.smithi027.stdout:1/178: creat d6/d11/d14/d1f/d25/d28/f3c x:0 0 0 2021-11-03T17:13:59.412 INFO:tasks.workunit.client.0.smithi027.stdout:1/179: write d6/d11/d14/f17 [638042,71537] 0 2021-11-03T17:14:00.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:00 smithi115 conmon[34670]: cluster 2021-11-03T17:13:59.693989+0000 mon.smithi027 (mon.0) 566 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi115.zsfgnv=up:active,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:14:00.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:00 smithi115 conmon[34670]: cluster 2021-11-03T17:14:00.602533+0000 mgr.smithi027.ufhhvb (mgr.14162) 297 : cluster [DBG] pgmap v247: 65 pgs: 65 active+clean; 649 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 19 MiB/s rd, 46 MiB/s wr, 660 op/s 2021-11-03T17:14:01.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:00 smithi027 conmon[32322]: cluster 2021-11-03T17:13:59.693989+0000 2021-11-03T17:14:01.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:00 smithi027 conmon[32322]: mon.smithi027 (mon.0) 566 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi115.zsfgnv=up:active,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:14:01.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:00 smithi027 conmon[32322]: cluster 2021-11-03T17:14:00.602533+0000 mgr.smithi027.ufhhvb ( 2021-11-03T17:14:01.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:00 smithi027 conmon[32322]: mgr.14162) 297 : cluster [DBG] pgmap v247: 65 pgs: 65 active+clean; 649 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 19 MiB/s rd, 46 MiB/s wr, 660 op/s 2021-11-03T17:14:01.060 INFO:tasks.workunit.client.0.smithi027.stdout:5/97: sync 2021-11-03T17:14:01.060 INFO:tasks.workunit.client.0.smithi027.stdout:5/98: chown d2/d4/l10 175195 1 2021-11-03T17:14:01.060 INFO:tasks.workunit.client.0.smithi027.stdout:5/99: mknod d2/d1f/c24 1 2021-11-03T17:14:01.063 INFO:tasks.workunit.client.0.smithi027.stdout:5/100: dread d2/f18 [0,4194304] 0 2021-11-03T17:14:01.066 INFO:tasks.workunit.client.0.smithi027.stdout:5/101: mkdir d2/d4/df/d20/d25 0 2021-11-03T17:14:01.067 INFO:tasks.workunit.client.0.smithi027.stdout:5/102: mkdir d2/d26 0 2021-11-03T17:14:01.069 INFO:tasks.workunit.client.0.smithi027.stdout:5/103: creat d2/d4/df/d20/d25/f27 x:0 0 0 2021-11-03T17:14:01.069 INFO:tasks.workunit.client.0.smithi027.stdout:5/104: write d2/f18 [1003141,39713] 0 2021-11-03T17:14:01.069 INFO:tasks.workunit.client.0.smithi027.stdout:5/105: chown d2/d4/f13 4934446 1 2021-11-03T17:14:01.069 INFO:tasks.workunit.client.0.smithi027.stdout:5/106: dread - d2/f1e zero size 2021-11-03T17:14:01.070 INFO:tasks.workunit.client.0.smithi027.stdout:5/107: fdatasync d2/d4/f6 0 2021-11-03T17:14:01.070 INFO:tasks.workunit.client.0.smithi027.stdout:5/108: chown d2/d4/df/f15 51 1 2021-11-03T17:14:01.071 INFO:tasks.workunit.client.0.smithi027.stdout:5/109: rename d2/d4/fb to d2/d1f/f28 0 2021-11-03T17:14:01.071 INFO:tasks.workunit.client.0.smithi027.stdout:5/110: mknod d2/d4/df/d20/d25/c29 1 2021-11-03T17:14:01.071 INFO:tasks.workunit.client.0.smithi027.stdout:5/111: getdents d2/d26 0 2021-11-03T17:14:01.071 INFO:tasks.workunit.client.0.smithi027.stdout:5/112: chown d2/f3 22491 1 2021-11-03T17:14:01.072 INFO:tasks.workunit.client.0.smithi027.stdout:5/113: creat d2/d12/f2a x:0 0 0 2021-11-03T17:14:01.073 INFO:tasks.workunit.client.0.smithi027.stdout:5/114: symlink d2/d4/df/d20/d25/l2b 0 2021-11-03T17:14:01.073 INFO:tasks.workunit.client.0.smithi027.stdout:5/115: write d2/d4/f6 [1044937,87686] 0 2021-11-03T17:14:01.073 INFO:tasks.workunit.client.0.smithi027.stdout:5/116: write d2/f7 [4366887,118676] 0 2021-11-03T17:14:01.074 INFO:tasks.workunit.client.0.smithi027.stdout:5/117: truncate d2/f3 1585197 0 2021-11-03T17:14:01.074 INFO:tasks.workunit.client.0.smithi027.stdout:5/118: chown d2/d12 2117 1 2021-11-03T17:14:01.074 INFO:tasks.workunit.client.0.smithi027.stdout:5/119: write d2/f7 [3703118,21405] 0 2021-11-03T17:14:01.075 INFO:tasks.workunit.client.0.smithi027.stdout:5/120: rename d2/d26 to d2/d2c 0 2021-11-03T17:14:01.077 INFO:tasks.workunit.client.0.smithi027.stdout:5/121: mkdir d2/d4/df/d2d 0 2021-11-03T17:14:01.077 INFO:tasks.workunit.client.0.smithi027.stdout:5/122: dread - d2/d12/f2a zero size 2021-11-03T17:14:01.088 INFO:tasks.workunit.client.0.smithi027.stdout:5/123: rmdir d2 39 2021-11-03T17:14:01.090 INFO:tasks.workunit.client.0.smithi027.stdout:5/124: creat d2/d4/df/f2e x:0 0 0 2021-11-03T17:14:01.092 INFO:tasks.workunit.client.0.smithi027.stdout:5/125: rename d2/d4/f6 to d2/d4/df/f2f 0 2021-11-03T17:14:01.093 INFO:tasks.workunit.client.0.smithi027.stdout:5/126: creat d2/d4/df/f30 x:0 0 0 2021-11-03T17:14:01.094 INFO:tasks.workunit.client.0.smithi027.stdout:5/127: symlink d2/d4/df/l31 0 2021-11-03T17:14:01.104 INFO:tasks.workunit.client.0.smithi027.stdout:3/79: sync 2021-11-03T17:14:01.104 INFO:tasks.workunit.client.0.smithi027.stdout:6/63: sync 2021-11-03T17:14:01.104 INFO:tasks.workunit.client.0.smithi027.stdout:6/64: chown f6 949226411 1 2021-11-03T17:14:01.105 INFO:tasks.workunit.client.0.smithi027.stdout:3/80: truncate d1/fa 2588052 0 2021-11-03T17:14:01.115 INFO:tasks.workunit.client.0.smithi027.stdout:3/81: write d1/fa [441772,89058] 0 2021-11-03T17:14:01.115 INFO:tasks.workunit.client.0.smithi027.stdout:3/82: chown d1/l4 69978139 1 2021-11-03T17:14:01.132 INFO:tasks.workunit.client.0.smithi027.stdout:3/83: dwrite d1/f14 [0,4194304] 0 2021-11-03T17:14:01.146 INFO:tasks.workunit.client.0.smithi027.stdout:3/84: dwrite d1/f17 [0,4194304] 0 2021-11-03T17:14:01.149 INFO:tasks.workunit.client.0.smithi027.stdout:3/85: creat d1/f19 x:0 0 0 2021-11-03T17:14:01.150 INFO:tasks.workunit.client.0.smithi027.stdout:3/86: write d1/f13 [2434724,22549] 0 2021-11-03T17:14:01.151 INFO:tasks.workunit.client.0.smithi027.stdout:3/87: truncate d1/f6 704413 0 2021-11-03T17:14:01.153 INFO:tasks.workunit.client.0.smithi027.stdout:3/88: rename d1/f19 to d1/f1a 0 2021-11-03T17:14:01.153 INFO:tasks.workunit.client.0.smithi027.stdout:3/89: chown d1/f9 1095737 1 2021-11-03T17:14:01.156 INFO:tasks.workunit.client.0.smithi027.stdout:3/90: truncate d1/f17 3756500 0 2021-11-03T17:14:01.159 INFO:tasks.workunit.client.0.smithi027.stdout:3/91: rmdir d1/db 39 2021-11-03T17:14:01.161 INFO:tasks.workunit.client.0.smithi027.stdout:3/92: symlink d1/l1b 0 2021-11-03T17:14:01.161 INFO:tasks.workunit.client.0.smithi027.stdout:3/93: chown d1/l4 355617 1 2021-11-03T17:14:01.162 INFO:tasks.workunit.client.0.smithi027.stdout:3/94: write d1/f9 [1468967,48975] 0 2021-11-03T17:14:01.163 INFO:tasks.workunit.client.0.smithi027.stdout:3/95: link d1/f13 d1/db/f1c 0 2021-11-03T17:14:01.165 INFO:tasks.workunit.client.0.smithi027.stdout:3/96: rename d1/fc to d1/f1d 0 2021-11-03T17:14:01.189 INFO:tasks.workunit.client.0.smithi027.stdout:3/97: dwrite d1/f13 [0,4194304] 0 2021-11-03T17:14:01.286 INFO:tasks.workunit.client.0.smithi027.stdout:4/111: sync 2021-11-03T17:14:01.745 INFO:tasks.workunit.client.0.smithi027.stdout:3/98: creat d1/db/f1e x:0 0 0 2021-11-03T17:14:01.746 INFO:tasks.workunit.client.0.smithi027.stdout:3/99: mknod d1/db/c1f 1 2021-11-03T17:14:01.749 INFO:tasks.workunit.client.0.smithi027.stdout:4/112: mkdir d6/d9/d28 0 2021-11-03T17:14:01.749 INFO:tasks.workunit.client.0.smithi027.stdout:4/113: chown d6/d9/l10 124 1 2021-11-03T17:14:01.749 INFO:tasks.workunit.client.0.smithi027.stdout:4/114: write f5 [4739004,64807] 0 2021-11-03T17:14:01.765 INFO:tasks.workunit.client.0.smithi027.stdout:4/115: dwrite d6/f1f [0,4194304] 0 2021-11-03T17:14:01.769 INFO:tasks.workunit.client.0.smithi027.stdout:4/116: rename d6/d9/la to d6/db/l29 0 2021-11-03T17:14:01.770 INFO:tasks.workunit.client.0.smithi027.stdout:4/117: write d6/d9/f11 [3912631,6083] 0 2021-11-03T17:14:01.771 INFO:tasks.workunit.client.0.smithi027.stdout:4/118: mknod d6/d9/c2a 1 2021-11-03T17:14:01.771 INFO:tasks.workunit.client.0.smithi027.stdout:4/119: dread - d6/d9/d16/f19 zero size 2021-11-03T17:14:01.772 INFO:tasks.workunit.client.0.smithi027.stdout:4/120: mkdir d6/db/d21/d2b 0 2021-11-03T17:14:01.774 INFO:tasks.workunit.client.0.smithi027.stdout:4/121: creat d6/db/d21/f2c x:0 0 0 2021-11-03T17:14:01.775 INFO:tasks.workunit.client.0.smithi027.stdout:4/122: write d6/d9/f11 [2543739,90618] 0 2021-11-03T17:14:01.776 INFO:tasks.workunit.client.0.smithi027.stdout:4/123: mkdir d6/db/d21/d2b/d2d 0 2021-11-03T17:14:01.778 INFO:tasks.workunit.client.0.smithi027.stdout:4/124: symlink d6/db/d21/d2b/d2d/l2e 0 2021-11-03T17:14:01.778 INFO:tasks.workunit.client.0.smithi027.stdout:4/125: chown d6/d9/l10 3 1 2021-11-03T17:14:01.778 INFO:tasks.workunit.client.0.smithi027.stdout:4/126: getdents d6/db/d27 0 2021-11-03T17:14:01.778 INFO:tasks.workunit.client.0.smithi027.stdout:4/127: write d6/d9/d16/f19 [675670,38073] 0 2021-11-03T17:14:01.779 INFO:tasks.workunit.client.0.smithi027.stdout:4/128: read d6/f1f [3421764,23738] 0 2021-11-03T17:14:01.779 INFO:tasks.workunit.client.0.smithi027.stdout:4/129: stat d6/l18 0 2021-11-03T17:14:01.780 INFO:tasks.workunit.client.0.smithi027.stdout:4/130: mknod d6/c2f 1 2021-11-03T17:14:01.780 INFO:tasks.workunit.client.0.smithi027.stdout:4/131: fdatasync d6/db/d21/f2c 0 2021-11-03T17:14:01.780 INFO:tasks.workunit.client.0.smithi027.stdout:4/132: rename d6/db/d21/d2b/d2d/l2e to d6/d9/d28/l30 0 2021-11-03T17:14:01.781 INFO:tasks.workunit.client.0.smithi027.stdout:4/133: creat d6/d9/d16/d17/f31 x:0 0 0 2021-11-03T17:14:01.783 INFO:tasks.workunit.client.0.smithi027.stdout:4/134: link f5 d6/d9/d16/f32 0 2021-11-03T17:14:01.785 INFO:tasks.workunit.client.0.smithi027.stdout:4/135: unlink f5 0 2021-11-03T17:14:01.785 INFO:tasks.workunit.client.0.smithi027.stdout:4/136: chown d6/d9/d16/d17/f31 1970723 1 2021-11-03T17:14:01.790 INFO:tasks.workunit.client.0.smithi027.stdout:4/137: link d6/db/l29 d6/d24/l33 0 2021-11-03T17:14:01.791 INFO:tasks.workunit.client.0.smithi027.stdout:4/138: fsync f4 0 2021-11-03T17:14:01.798 INFO:tasks.workunit.client.0.smithi027.stdout:4/139: write d6/d9/d16/f32 [2936469,88126] 0 2021-11-03T17:14:01.799 INFO:tasks.workunit.client.0.smithi027.stdout:4/140: write d6/d9/f11 [3046521,10867] 0 2021-11-03T17:14:01.799 INFO:tasks.workunit.client.0.smithi027.stdout:4/141: chown d6 6818733 1 2021-11-03T17:14:01.806 INFO:tasks.workunit.client.0.smithi027.stdout:4/142: write d6/d9/d16/f32 [4141600,33823] 0 2021-11-03T17:14:01.807 INFO:tasks.workunit.client.0.smithi027.stdout:4/143: truncate d6/db/d21/f25 1937585 0 2021-11-03T17:14:01.816 INFO:tasks.workunit.client.0.smithi027.stdout:4/144: dread d6/d9/d16/f26 [0,4194304] 0 2021-11-03T17:14:01.817 INFO:tasks.workunit.client.0.smithi027.stdout:4/145: creat d6/db/f34 x:0 0 0 2021-11-03T17:14:01.818 INFO:tasks.workunit.client.0.smithi027.stdout:4/146: creat d6/db/f35 x:0 0 0 2021-11-03T17:14:01.818 INFO:tasks.workunit.client.0.smithi027.stdout:4/147: readlink d6/l18 0 2021-11-03T17:14:01.819 INFO:tasks.workunit.client.0.smithi027.stdout:4/148: chown d6/d9/d16/d17/l1d 30691013 1 2021-11-03T17:14:01.822 INFO:tasks.workunit.client.0.smithi027.stdout:4/149: dread d6/db/d21/f25 [0,4194304] 0 2021-11-03T17:14:01.825 INFO:tasks.workunit.client.0.smithi027.stdout:4/150: rename d6/db/d27 to d6/d9/d16/d36 0 2021-11-03T17:14:01.826 INFO:tasks.workunit.client.0.smithi027.stdout:4/151: read d6/d9/f11 [4106618,110621] 0 2021-11-03T17:14:01.840 INFO:tasks.workunit.client.0.smithi027.stdout:4/152: dwrite d6/d9/d16/d17/f31 [0,4194304] 0 2021-11-03T17:14:01.840 INFO:tasks.workunit.client.0.smithi027.stdout:4/153: dread - d6/d9/f1b zero size 2021-11-03T17:14:01.843 INFO:tasks.workunit.client.0.smithi027.stdout:4/154: creat d6/db/d21/f37 x:0 0 0 2021-11-03T17:14:01.844 INFO:tasks.workunit.client.0.smithi027.stdout:4/155: write d6/f1f [1847063,124478] 0 2021-11-03T17:14:01.844 INFO:tasks.workunit.client.0.smithi027.stdout:4/156: write d6/db/f35 [938514,11891] 0 2021-11-03T17:14:01.847 INFO:tasks.workunit.client.0.smithi027.stdout:4/157: creat d6/db/d21/f38 x:0 0 0 2021-11-03T17:14:01.848 INFO:tasks.workunit.client.0.smithi027.stdout:4/158: creat d6/d9/d28/f39 x:0 0 0 2021-11-03T17:14:01.850 INFO:tasks.workunit.client.0.smithi027.stdout:4/159: unlink d6/d9/d28/f39 0 2021-11-03T17:14:01.851 INFO:tasks.workunit.client.0.smithi027.stdout:4/160: write d6/d9/d16/f19 [817568,59441] 0 2021-11-03T17:14:01.851 INFO:tasks.workunit.client.0.smithi027.stdout:4/161: write d6/db/d21/f38 [11319,84033] 0 2021-11-03T17:14:01.851 INFO:tasks.workunit.client.0.smithi027.stdout:4/162: truncate d6/db/d21/f38 772582 0 2021-11-03T17:14:01.852 INFO:tasks.workunit.client.0.smithi027.stdout:4/163: mkdir d6/db/d21/d2b/d3a 0 2021-11-03T17:14:01.858 INFO:tasks.workunit.client.0.smithi027.stdout:4/164: dread d6/d9/d16/f19 [0,4194304] 0 2021-11-03T17:14:01.858 INFO:tasks.workunit.client.0.smithi027.stdout:4/165: truncate d6/db/f34 1459 0 2021-11-03T17:14:01.859 INFO:tasks.workunit.client.0.smithi027.stdout:4/166: creat d6/d9/d28/f3b x:0 0 0 2021-11-03T17:14:01.859 INFO:tasks.workunit.client.0.smithi027.stdout:4/167: chown d6/db/l15 396504538 1 2021-11-03T17:14:01.860 INFO:tasks.workunit.client.0.smithi027.stdout:4/168: creat d6/d9/d28/f3c x:0 0 0 2021-11-03T17:14:01.879 INFO:tasks.workunit.client.0.smithi027.stdout:4/169: dwrite d6/d9/f11 [0,4194304] 0 2021-11-03T17:14:01.879 INFO:tasks.workunit.client.0.smithi027.stdout:4/170: truncate d6/db/fc 1127487 0 2021-11-03T17:14:01.883 INFO:tasks.workunit.client.0.smithi027.stdout:4/171: truncate d6/f1f 2548518 0 2021-11-03T17:14:01.885 INFO:tasks.workunit.client.0.smithi027.stdout:4/172: creat d6/d9/d16/f3d x:0 0 0 2021-11-03T17:14:01.972 INFO:tasks.workunit.client.0.smithi027.stdout:1/180: sync 2021-11-03T17:14:01.973 INFO:tasks.workunit.client.0.smithi027.stdout:1/181: truncate d6/de/d13/d15/f2f 501496 0 2021-11-03T17:14:01.974 INFO:tasks.workunit.client.0.smithi027.stdout:1/182: symlink d6/d11/d14/d1f/d37/l3d 0 2021-11-03T17:14:01.974 INFO:tasks.workunit.client.0.smithi027.stdout:1/183: chown f2 31 1 2021-11-03T17:14:01.975 INFO:tasks.workunit.client.0.smithi027.stdout:1/184: fdatasync d6/d11/d14/d1f/f2b 0 2021-11-03T17:14:01.988 INFO:tasks.workunit.client.0.smithi027.stdout:1/185: dwrite d6/de/f1a [0,4194304] 0 2021-11-03T17:14:02.055 INFO:tasks.workunit.client.0.smithi027.stdout:8/164: sync 2021-11-03T17:14:02.061 INFO:tasks.workunit.client.0.smithi027.stdout:8/165: truncate d7/d10/f12 1062824 0 2021-11-03T17:14:02.078 INFO:tasks.workunit.client.0.smithi027.stdout:8/166: dwrite d7/f8 [4194304,4194304] 0 2021-11-03T17:14:02.095 INFO:tasks.workunit.client.0.smithi027.stdout:8/167: dwrite f6 [4194304,4194304] 0 2021-11-03T17:14:02.099 INFO:tasks.workunit.client.0.smithi027.stdout:8/168: write d7/d10/f16 [2457701,55107] 0 2021-11-03T17:14:02.118 INFO:tasks.workunit.client.0.smithi027.stdout:8/169: dwrite d7/d10/f16 [0,4194304] 0 2021-11-03T17:14:02.118 INFO:tasks.workunit.client.0.smithi027.stdout:8/170: truncate d7/d10/d18/d1a/f2e 784272 0 2021-11-03T17:14:02.119 INFO:tasks.workunit.client.0.smithi027.stdout:8/171: creat d7/d10/f34 x:0 0 0 2021-11-03T17:14:02.131 INFO:tasks.workunit.client.0.smithi027.stdout:7/115: sync 2021-11-03T17:14:02.132 INFO:tasks.workunit.client.0.smithi027.stdout:0/146: sync 2021-11-03T17:14:02.132 INFO:tasks.workunit.client.0.smithi027.stdout:2/117: sync 2021-11-03T17:14:02.132 INFO:tasks.workunit.client.0.smithi027.stdout:9/127: sync 2021-11-03T17:14:02.132 INFO:tasks.workunit.client.0.smithi027.stdout:0/147: write d9/f24 [562547,8636] 0 2021-11-03T17:14:02.133 INFO:tasks.workunit.client.0.smithi027.stdout:0/148: fsync d9/f2e 0 2021-11-03T17:14:02.133 INFO:tasks.workunit.client.0.smithi027.stdout:0/149: dread - d9/f23 zero size 2021-11-03T17:14:02.134 INFO:tasks.workunit.client.0.smithi027.stdout:7/116: write d0/d2/fd [145387,101296] 0 2021-11-03T17:14:02.134 INFO:tasks.workunit.client.0.smithi027.stdout:8/172: dwrite d7/d10/d18/d1a/f2e [0,4194304] 0 2021-11-03T17:14:02.135 INFO:tasks.workunit.client.0.smithi027.stdout:8/173: truncate d7/d10/f34 498393 0 2021-11-03T17:14:02.136 INFO:tasks.workunit.client.0.smithi027.stdout:2/118: rename d0/f11 to d0/d5/de/f23 0 2021-11-03T17:14:02.136 INFO:tasks.workunit.client.0.smithi027.stdout:2/119: fdatasync d0/d5/f8 0 2021-11-03T17:14:02.138 INFO:tasks.workunit.client.0.smithi027.stdout:0/150: mkdir d9/da/d31 0 2021-11-03T17:14:02.138 INFO:tasks.workunit.client.0.smithi027.stdout:7/117: mknod d0/d2/d10/c23 1 2021-11-03T17:14:02.139 INFO:tasks.workunit.client.0.smithi027.stdout:7/118: fdatasync d0/f13 0 2021-11-03T17:14:02.139 INFO:tasks.workunit.client.0.smithi027.stdout:7/119: readlink d0/lb 0 2021-11-03T17:14:02.139 INFO:tasks.workunit.client.0.smithi027.stdout:7/120: write d0/f13 [5280645,84113] 0 2021-11-03T17:14:02.142 INFO:tasks.workunit.client.0.smithi027.stdout:2/120: creat d0/d1b/d1c/f24 x:0 0 0 2021-11-03T17:14:02.150 INFO:tasks.workunit.client.0.smithi027.stdout:7/121: creat d0/d2/d10/d1e/f24 x:0 0 0 2021-11-03T17:14:02.152 INFO:tasks.workunit.client.0.smithi027.stdout:9/128: dwrite db/f17 [0,4194304] 0 2021-11-03T17:14:02.152 INFO:tasks.workunit.client.0.smithi027.stdout:9/129: fdatasync db/d1b/d1c/d1d/f1e 0 2021-11-03T17:14:02.153 INFO:tasks.workunit.client.0.smithi027.stdout:9/130: write db/f14 [530037,111828] 0 2021-11-03T17:14:02.153 INFO:tasks.workunit.client.0.smithi027.stdout:9/131: chown db/d1b/d1c/d1d/d2f 41 1 2021-11-03T17:14:02.154 INFO:tasks.workunit.client.0.smithi027.stdout:9/132: unlink f8 0 2021-11-03T17:14:02.154 INFO:tasks.workunit.client.0.smithi027.stdout:9/133: stat db/f1f 0 2021-11-03T17:14:02.154 INFO:tasks.workunit.client.0.smithi027.stdout:9/134: fsync db/fd 0 2021-11-03T17:14:02.155 INFO:tasks.workunit.client.0.smithi027.stdout:9/135: mknod db/c33 1 2021-11-03T17:14:02.155 INFO:tasks.workunit.client.0.smithi027.stdout:9/136: mknod db/d1b/d1c/d1d/d2f/c34 1 2021-11-03T17:14:02.156 INFO:tasks.workunit.client.0.smithi027.stdout:9/137: creat db/f35 x:0 0 0 2021-11-03T17:14:02.156 INFO:tasks.workunit.client.0.smithi027.stdout:9/138: fdatasync db/f13 0 2021-11-03T17:14:02.157 INFO:tasks.workunit.client.0.smithi027.stdout:7/122: dread d0/f16 [0,4194304] 0 2021-11-03T17:14:02.158 INFO:tasks.workunit.client.0.smithi027.stdout:7/123: truncate d0/d2/f5 3568230 0 2021-11-03T17:14:02.161 INFO:tasks.workunit.client.0.smithi027.stdout:2/121: dwrite d0/d5/f8 [0,4194304] 0 2021-11-03T17:14:02.162 INFO:tasks.workunit.client.0.smithi027.stdout:7/124: link d0/d2/d10/f1b d0/f25 0 2021-11-03T17:14:02.162 INFO:tasks.workunit.client.0.smithi027.stdout:8/174: dwrite d7/d10/f15 [0,4194304] 0 2021-11-03T17:14:02.163 INFO:tasks.workunit.client.0.smithi027.stdout:2/122: symlink d0/d1b/l25 0 2021-11-03T17:14:02.164 INFO:tasks.workunit.client.0.smithi027.stdout:2/123: dread d0/d5/f1e [0,4194304] 0 2021-11-03T17:14:02.170 INFO:tasks.workunit.client.0.smithi027.stdout:7/125: dread d0/d2/d10/f1b [0,4194304] 0 2021-11-03T17:14:02.170 INFO:tasks.workunit.client.0.smithi027.stdout:7/126: stat d0/d2/l9 0 2021-11-03T17:14:02.171 INFO:tasks.workunit.client.0.smithi027.stdout:8/175: rename d7/fe to d7/f35 0 2021-11-03T17:14:02.174 INFO:tasks.workunit.client.0.smithi027.stdout:2/124: link d0/d5/de/f21 d0/d1b/d1c/f26 0 2021-11-03T17:14:02.181 INFO:tasks.workunit.client.0.smithi027.stdout:2/125: creat d0/d1b/f27 x:0 0 0 2021-11-03T17:14:02.184 INFO:tasks.workunit.client.0.smithi027.stdout:2/126: symlink d0/l28 0 2021-11-03T17:14:02.184 INFO:tasks.workunit.client.0.smithi027.stdout:2/127: readlink d0/d1b/l25 0 2021-11-03T17:14:02.189 INFO:tasks.workunit.client.0.smithi027.stdout:2/128: dread d0/d1b/d1c/f26 [0,4194304] 0 2021-11-03T17:14:02.189 INFO:tasks.workunit.client.0.smithi027.stdout:2/129: write d0/d5/f1e [493537,3104] 0 2021-11-03T17:14:02.189 INFO:tasks.workunit.client.0.smithi027.stdout:2/130: dread - d0/d1b/d1c/f24 zero size 2021-11-03T17:14:02.190 INFO:tasks.workunit.client.0.smithi027.stdout:2/131: readlink d0/d5/l17 0 2021-11-03T17:14:02.190 INFO:tasks.workunit.client.0.smithi027.stdout:2/132: mkdir d0/d1b/d29 0 2021-11-03T17:14:02.198 INFO:tasks.workunit.client.0.smithi027.stdout:8/176: dwrite d7/d10/f34 [0,4194304] 0 2021-11-03T17:14:02.199 INFO:tasks.workunit.client.0.smithi027.stdout:7/127: dwrite d0/d2/d10/f11 [0,4194304] 0 2021-11-03T17:14:02.199 INFO:tasks.workunit.client.0.smithi027.stdout:7/128: getdents d0/d2/d1f 0 2021-11-03T17:14:02.199 INFO:tasks.workunit.client.0.smithi027.stdout:7/129: fsync d0/f16 0 2021-11-03T17:14:02.199 INFO:tasks.workunit.client.0.smithi027.stdout:7/130: write d0/f16 [1577099,4871] 0 2021-11-03T17:14:02.199 INFO:tasks.workunit.client.0.smithi027.stdout:8/177: creat d7/f36 x:0 0 0 2021-11-03T17:14:02.200 INFO:tasks.workunit.client.0.smithi027.stdout:2/133: rename d0/d5/de to d0/d2a 0 2021-11-03T17:14:02.201 INFO:tasks.workunit.client.0.smithi027.stdout:2/134: chown d0/d1b/f27 7668 1 2021-11-03T17:14:02.201 INFO:tasks.workunit.client.0.smithi027.stdout:2/135: read - d0/d1b/f27 zero size 2021-11-03T17:14:02.205 INFO:tasks.workunit.client.0.smithi027.stdout:7/131: creat d0/d2/d10/d1e/d20/f26 x:0 0 0 2021-11-03T17:14:02.206 INFO:tasks.workunit.client.0.smithi027.stdout:2/136: creat d0/d1b/d29/f2b x:0 0 0 2021-11-03T17:14:02.207 INFO:tasks.workunit.client.0.smithi027.stdout:7/132: dread d0/d2/fd [0,4194304] 0 2021-11-03T17:14:02.216 INFO:tasks.workunit.client.0.smithi027.stdout:8/178: dwrite d7/d10/d18/d1a/f2e [0,4194304] 0 2021-11-03T17:14:02.218 INFO:tasks.workunit.client.0.smithi027.stdout:2/137: rmdir d0/d2a 39 2021-11-03T17:14:02.218 INFO:tasks.workunit.client.0.smithi027.stdout:2/138: write d0/d5/f8 [5236603,59661] 0 2021-11-03T17:14:02.219 INFO:tasks.workunit.client.0.smithi027.stdout:2/139: fsync d0/f10 0 2021-11-03T17:14:02.219 INFO:tasks.workunit.client.0.smithi027.stdout:2/140: truncate d0/d5/f1e 636295 0 2021-11-03T17:14:02.222 INFO:tasks.workunit.client.0.smithi027.stdout:2/141: truncate d0/f10 1975422 0 2021-11-03T17:14:02.223 INFO:tasks.workunit.client.0.smithi027.stdout:2/142: mkdir d0/d5/d2c 0 2021-11-03T17:14:02.224 INFO:tasks.workunit.client.0.smithi027.stdout:2/143: rmdir d0/d2a 39 2021-11-03T17:14:02.233 INFO:tasks.workunit.client.0.smithi027.stdout:7/133: dwrite d0/d2/d10/f1b [0,4194304] 0 2021-11-03T17:14:02.235 INFO:tasks.workunit.client.0.smithi027.stdout:7/134: creat d0/d2/d1c/f27 x:0 0 0 2021-11-03T17:14:02.238 INFO:tasks.workunit.client.0.smithi027.stdout:8/179: dwrite d7/d10/d18/d1a/f1f [0,4194304] 0 2021-11-03T17:14:02.243 INFO:tasks.workunit.client.0.smithi027.stdout:8/180: unlink d7/d10/l14 0 2021-11-03T17:14:02.249 INFO:tasks.workunit.client.0.smithi027.stdout:8/181: unlink d7/d10/f34 0 2021-11-03T17:14:02.250 INFO:tasks.workunit.client.0.smithi027.stdout:8/182: chown d7/d10/f16 263123028 1 2021-11-03T17:14:02.250 INFO:tasks.workunit.client.0.smithi027.stdout:8/183: mknod d7/c37 1 2021-11-03T17:14:02.250 INFO:tasks.workunit.client.0.smithi027.stdout:8/184: truncate d7/d10/f12 977642 0 2021-11-03T17:14:02.253 INFO:tasks.workunit.client.0.smithi027.stdout:2/144: dwrite d0/d2a/f23 [0,4194304] 0 2021-11-03T17:14:02.254 INFO:tasks.workunit.client.0.smithi027.stdout:8/185: dread d7/f1b [0,4194304] 0 2021-11-03T17:14:02.254 INFO:tasks.workunit.client.0.smithi027.stdout:8/186: readlink d7/l2f 0 2021-11-03T17:14:02.255 INFO:tasks.workunit.client.0.smithi027.stdout:2/145: rmdir d0/d1b/d1c 39 2021-11-03T17:14:02.259 INFO:tasks.workunit.client.0.smithi027.stdout:7/135: truncate d0/f16 435740 0 2021-11-03T17:14:02.262 INFO:tasks.workunit.client.0.smithi027.stdout:8/187: mkdir d7/d2d/d38 0 2021-11-03T17:14:02.265 INFO:tasks.workunit.client.0.smithi027.stdout:7/136: rmdir d0/d2/d1c 39 2021-11-03T17:14:02.268 INFO:tasks.workunit.client.0.smithi027.stdout:8/188: creat d7/d10/d21/f39 x:0 0 0 2021-11-03T17:14:02.269 INFO:tasks.workunit.client.0.smithi027.stdout:8/189: getdents d7/d10/d18/d1a 0 2021-11-03T17:14:02.270 INFO:tasks.workunit.client.0.smithi027.stdout:8/190: write d7/d10/f2b [628787,58811] 0 2021-11-03T17:14:02.275 INFO:tasks.workunit.client.0.smithi027.stdout:2/146: dwrite d0/f10 [0,4194304] 0 2021-11-03T17:14:02.275 INFO:tasks.workunit.client.0.smithi027.stdout:7/137: dread d0/d2/d10/f11 [0,4194304] 0 2021-11-03T17:14:02.276 INFO:tasks.workunit.client.0.smithi027.stdout:7/138: dread - d0/d2/d10/f19 zero size 2021-11-03T17:14:02.276 INFO:tasks.workunit.client.0.smithi027.stdout:7/139: truncate d0/f13 6395250 0 2021-11-03T17:14:02.276 INFO:tasks.workunit.client.0.smithi027.stdout:7/140: mknod d0/c28 1 2021-11-03T17:14:02.276 INFO:tasks.workunit.client.0.smithi027.stdout:7/141: write d0/d2/d10/f19 [988006,84060] 0 2021-11-03T17:14:02.277 INFO:tasks.workunit.client.0.smithi027.stdout:7/142: read d0/f13 [6345465,63400] 0 2021-11-03T17:14:02.277 INFO:tasks.workunit.client.0.smithi027.stdout:7/143: mknod d0/d2/d1c/c29 1 2021-11-03T17:14:02.277 INFO:tasks.workunit.client.0.smithi027.stdout:7/144: write d0/d2/d10/f1b [2875020,72364] 0 2021-11-03T17:14:02.278 INFO:tasks.workunit.client.0.smithi027.stdout:7/145: mknod d0/d2/d1c/c2a 1 2021-11-03T17:14:02.278 INFO:tasks.workunit.client.0.smithi027.stdout:7/146: chown d0/d2/d10/d1e/d20 225 1 2021-11-03T17:14:02.279 INFO:tasks.workunit.client.0.smithi027.stdout:2/147: symlink d0/l2d 0 2021-11-03T17:14:02.279 INFO:tasks.workunit.client.0.smithi027.stdout:7/147: truncate d0/f16 352807 0 2021-11-03T17:14:02.282 INFO:tasks.workunit.client.0.smithi027.stdout:2/148: symlink d0/d1b/l2e 0 2021-11-03T17:14:02.282 INFO:tasks.workunit.client.0.smithi027.stdout:2/149: fsync d0/d1b/d29/f2b 0 2021-11-03T17:14:02.285 INFO:tasks.workunit.client.0.smithi027.stdout:2/150: creat d0/d1b/d29/f2f x:0 0 0 2021-11-03T17:14:02.285 INFO:tasks.workunit.client.0.smithi027.stdout:2/151: fsync d0/d5/f13 0 2021-11-03T17:14:02.288 INFO:tasks.workunit.client.0.smithi027.stdout:2/152: creat d0/d2a/f30 x:0 0 0 2021-11-03T17:14:02.288 INFO:tasks.workunit.client.0.smithi027.stdout:2/153: readlink d0/d5/l1a 0 2021-11-03T17:14:02.288 INFO:tasks.workunit.client.0.smithi027.stdout:2/154: fdatasync d0/d5/f8 0 2021-11-03T17:14:02.289 INFO:tasks.workunit.client.0.smithi027.stdout:2/155: dread - d0/d2a/f30 zero size 2021-11-03T17:14:02.289 INFO:tasks.workunit.client.0.smithi027.stdout:2/156: read d0/d5/f13 [1917833,36664] 0 2021-11-03T17:14:02.291 INFO:tasks.workunit.client.0.smithi027.stdout:2/157: rename d0/d1b/l2e to d0/l31 0 2021-11-03T17:14:02.294 INFO:tasks.workunit.client.0.smithi027.stdout:2/158: link d0/l2d d0/d5/l32 0 2021-11-03T17:14:02.294 INFO:tasks.workunit.client.0.smithi027.stdout:2/159: write d0/d1b/f27 [617264,108516] 0 2021-11-03T17:14:02.295 INFO:tasks.workunit.client.0.smithi027.stdout:7/148: dwrite d0/d2/fd [0,4194304] 0 2021-11-03T17:14:02.296 INFO:tasks.workunit.client.0.smithi027.stdout:7/149: truncate d0/d2/d10/d1e/d20/f26 920109 0 2021-11-03T17:14:02.296 INFO:tasks.workunit.client.0.smithi027.stdout:7/150: fsync d0/d2/f5 0 2021-11-03T17:14:02.296 INFO:tasks.workunit.client.0.smithi027.stdout:2/160: unlink d0/l15 0 2021-11-03T17:14:02.297 INFO:tasks.workunit.client.0.smithi027.stdout:2/161: readlink d0/d2a/l14 0 2021-11-03T17:14:02.302 INFO:tasks.workunit.client.0.smithi027.stdout:7/151: creat d0/d2/d1c/f2b x:0 0 0 2021-11-03T17:14:02.306 INFO:tasks.workunit.client.0.smithi027.stdout:2/162: creat d0/d1b/d1c/f33 x:0 0 0 2021-11-03T17:14:02.306 INFO:tasks.workunit.client.0.smithi027.stdout:2/163: dread - d0/d1b/d29/f2f zero size 2021-11-03T17:14:02.306 INFO:tasks.workunit.client.0.smithi027.stdout:2/164: mknod d0/d1b/d29/c34 1 2021-11-03T17:14:02.307 INFO:tasks.workunit.client.0.smithi027.stdout:2/165: rename d0/d5 to d0/d5/d2c/d35 22 2021-11-03T17:14:02.307 INFO:tasks.workunit.client.0.smithi027.stdout:2/166: readlink d0/d1b/l25 0 2021-11-03T17:14:02.307 INFO:tasks.workunit.client.0.smithi027.stdout:7/152: symlink d0/d2/d1c/l2c 0 2021-11-03T17:14:02.308 INFO:tasks.workunit.client.0.smithi027.stdout:2/167: link d0/d1b/d1c/l1d d0/d1b/l36 0 2021-11-03T17:14:02.308 INFO:tasks.workunit.client.0.smithi027.stdout:7/153: rmdir d0/d2 39 2021-11-03T17:14:02.310 INFO:tasks.workunit.client.0.smithi027.stdout:2/168: rmdir d0/d5 39 2021-11-03T17:14:02.312 INFO:tasks.workunit.client.0.smithi027.stdout:7/154: truncate d0/f25 3889698 0 2021-11-03T17:14:02.312 INFO:tasks.workunit.client.0.smithi027.stdout:7/155: fdatasync d0/d2/d10/f1b 0 2021-11-03T17:14:02.312 INFO:tasks.workunit.client.0.smithi027.stdout:7/156: write d0/d2/d1c/f2b [75846,42321] 0 2021-11-03T17:14:02.312 INFO:tasks.workunit.client.0.smithi027.stdout:2/169: stat d0/d2a/l22 0 2021-11-03T17:14:02.313 INFO:tasks.workunit.client.0.smithi027.stdout:7/157: mkdir d0/d2d 0 2021-11-03T17:14:02.314 INFO:tasks.workunit.client.0.smithi027.stdout:2/170: getdents d0/d2a 0 2021-11-03T17:14:02.314 INFO:tasks.workunit.client.0.smithi027.stdout:2/171: mknod d0/d5/c37 1 2021-11-03T17:14:02.315 INFO:tasks.workunit.client.0.smithi027.stdout:2/172: mknod d0/d1b/d29/c38 1 2021-11-03T17:14:02.319 INFO:tasks.workunit.client.0.smithi027.stdout:7/158: rmdir d0/d2/d10 39 2021-11-03T17:14:02.320 INFO:tasks.workunit.client.0.smithi027.stdout:2/173: creat d0/d1b/d29/f39 x:0 0 0 2021-11-03T17:14:02.322 INFO:tasks.workunit.client.0.smithi027.stdout:2/174: symlink d0/d5/d2c/l3a 0 2021-11-03T17:14:02.323 INFO:tasks.workunit.client.0.smithi027.stdout:7/159: rename d0/d2/d1c/l2c to d0/d2/d10/l2e 0 2021-11-03T17:14:02.326 INFO:tasks.workunit.client.0.smithi027.stdout:2/175: creat d0/f3b x:0 0 0 2021-11-03T17:14:02.326 INFO:tasks.workunit.client.0.smithi027.stdout:2/176: write d0/d1b/d1c/f33 [919305,72337] 0 2021-11-03T17:14:02.326 INFO:tasks.workunit.client.0.smithi027.stdout:2/177: write d0/d2a/f21 [4750064,93540] 0 2021-11-03T17:14:02.326 INFO:tasks.workunit.client.0.smithi027.stdout:2/178: mknod d0/d2a/c3c 1 2021-11-03T17:14:02.327 INFO:tasks.workunit.client.0.smithi027.stdout:7/160: mkdir d0/d2d/d2f 0 2021-11-03T17:14:02.327 INFO:tasks.workunit.client.0.smithi027.stdout:7/161: dread - d0/d2/d1c/f27 zero size 2021-11-03T17:14:02.329 INFO:tasks.workunit.client.0.smithi027.stdout:2/179: creat d0/d5/d2c/f3d x:0 0 0 2021-11-03T17:14:02.329 INFO:tasks.workunit.client.0.smithi027.stdout:2/180: fsync d0/d5/f3 0 2021-11-03T17:14:02.329 INFO:tasks.workunit.client.0.smithi027.stdout:2/181: chown d0/d1b/d1c/f24 167215035 1 2021-11-03T17:14:02.329 INFO:tasks.workunit.client.0.smithi027.stdout:2/182: write d0/d2a/f30 [191378,18557] 0 2021-11-03T17:14:02.329 INFO:tasks.workunit.client.0.smithi027.stdout:2/183: truncate d0/d2a/f23 4367036 0 2021-11-03T17:14:02.331 INFO:tasks.workunit.client.0.smithi027.stdout:7/162: link d0/l1d d0/d2/d10/l30 0 2021-11-03T17:14:02.331 INFO:tasks.workunit.client.0.smithi027.stdout:7/163: fsync d0/d2/d10/f1b 0 2021-11-03T17:14:02.332 INFO:tasks.workunit.client.0.smithi027.stdout:7/164: mkdir d0/d2/d10/d1e/d20/d31 0 2021-11-03T17:14:02.333 INFO:tasks.workunit.client.0.smithi027.stdout:7/165: creat d0/d2/d10/d1e/d20/d31/f32 x:0 0 0 2021-11-03T17:14:02.334 INFO:tasks.workunit.client.0.smithi027.stdout:7/166: symlink d0/d2/l33 0 2021-11-03T17:14:02.336 INFO:tasks.workunit.client.0.smithi027.stdout:7/167: write d0/d2/d10/f1b [1242095,20600] 0 2021-11-03T17:14:02.336 INFO:tasks.workunit.client.0.smithi027.stdout:7/168: dread - d0/d2/d10/d1e/d20/d31/f32 zero size 2021-11-03T17:14:02.353 INFO:tasks.workunit.client.0.smithi027.stdout:2/184: dwrite d0/d5/f3 [0,4194304] 0 2021-11-03T17:14:02.353 INFO:tasks.workunit.client.0.smithi027.stdout:2/185: write d0/d1b/d29/f39 [762576,51879] 0 2021-11-03T17:14:02.354 INFO:tasks.workunit.client.0.smithi027.stdout:2/186: symlink d0/d5/d2c/l3e 0 2021-11-03T17:14:02.356 INFO:tasks.workunit.client.0.smithi027.stdout:2/187: link d0/l2d d0/d2a/l3f 0 2021-11-03T17:14:02.356 INFO:tasks.workunit.client.0.smithi027.stdout:2/188: readlink d0/d5/l32 0 2021-11-03T17:14:02.375 INFO:tasks.workunit.client.0.smithi027.stdout:2/189: dwrite d0/d1b/d29/f39 [0,4194304] 0 2021-11-03T17:14:02.391 INFO:tasks.workunit.client.0.smithi027.stdout:2/190: dwrite d0/d5/f4 [4194304,4194304] 0 2021-11-03T17:14:02.398 INFO:tasks.workunit.client.0.smithi027.stdout:2/191: dread d0/d5/f8 [0,4194304] 0 2021-11-03T17:14:02.399 INFO:tasks.workunit.client.0.smithi027.stdout:2/192: getdents d0/d5 0 2021-11-03T17:14:02.401 INFO:tasks.workunit.client.0.smithi027.stdout:2/193: rename d0/d2a/l14 to d0/d1b/l40 0 2021-11-03T17:14:02.402 INFO:tasks.workunit.client.0.smithi027.stdout:2/194: creat d0/f41 x:0 0 0 2021-11-03T17:14:02.402 INFO:tasks.workunit.client.0.smithi027.stdout:2/195: write d0/d2a/f30 [468493,103989] 0 2021-11-03T17:14:02.403 INFO:tasks.workunit.client.0.smithi027.stdout:2/196: truncate d0/d1b/d1c/f24 531626 0 2021-11-03T17:14:02.403 INFO:tasks.workunit.client.0.smithi027.stdout:2/197: dread - d0/d5/d2c/f3d zero size 2021-11-03T17:14:02.403 INFO:tasks.workunit.client.0.smithi027.stdout:2/198: rmdir d0/d1b/d1c 39 2021-11-03T17:14:02.405 INFO:tasks.workunit.client.0.smithi027.stdout:2/199: symlink d0/d5/l42 0 2021-11-03T17:14:02.906 INFO:tasks.workunit.client.0.smithi027.stdout:5/128: sync 2021-11-03T17:14:02.906 INFO:tasks.workunit.client.0.smithi027.stdout:6/65: sync 2021-11-03T17:14:02.906 INFO:tasks.workunit.client.0.smithi027.stdout:6/66: readlink - no filename 2021-11-03T17:14:02.907 INFO:tasks.workunit.client.0.smithi027.stdout:6/67: getdents . 0 2021-11-03T17:14:02.909 INFO:tasks.workunit.client.0.smithi027.stdout:5/129: unlink d2/d1f/f28 0 2021-11-03T17:14:02.909 INFO:tasks.workunit.client.0.smithi027.stdout:5/130: stat d2/d4/df/f30 0 2021-11-03T17:14:02.911 INFO:tasks.workunit.client.0.smithi027.stdout:5/131: creat d2/d12/f32 x:0 0 0 2021-11-03T17:14:02.911 INFO:tasks.workunit.client.0.smithi027.stdout:5/132: dread - d2/d4/df/f30 zero size 2021-11-03T17:14:02.911 INFO:tasks.workunit.client.0.smithi027.stdout:5/133: mknod d2/d4/c33 1 2021-11-03T17:14:02.912 INFO:tasks.workunit.client.0.smithi027.stdout:5/134: write d2/d4/df/f30 [761640,15127] 0 2021-11-03T17:14:02.934 INFO:tasks.workunit.client.0.smithi027.stdout:6/68: dwrite f7 [0,4194304] 0 2021-11-03T17:14:02.935 INFO:tasks.workunit.client.0.smithi027.stdout:5/135: dwrite d2/f7 [0,4194304] 0 2021-11-03T17:14:02.935 INFO:tasks.workunit.client.0.smithi027.stdout:5/136: chown d2/d4/df/d2d 12315798 1 2021-11-03T17:14:02.935 INFO:tasks.workunit.client.0.smithi027.stdout:5/137: rmdir d2 39 2021-11-03T17:14:02.938 INFO:tasks.workunit.client.0.smithi027.stdout:6/69: dread f0 [0,4194304] 0 2021-11-03T17:14:02.939 INFO:tasks.workunit.client.0.smithi027.stdout:5/138: write d2/d4/df/f11 [2783638,106619] 0 2021-11-03T17:14:02.940 INFO:tasks.workunit.client.0.smithi027.stdout:5/139: truncate d2/d4/fa 443615 0 2021-11-03T17:14:02.941 INFO:tasks.workunit.client.0.smithi027.stdout:6/70: dread f6 [0,4194304] 0 2021-11-03T17:14:02.942 INFO:tasks.workunit.client.0.smithi027.stdout:5/140: creat d2/d4/f34 x:0 0 0 2021-11-03T17:14:02.943 INFO:tasks.workunit.client.0.smithi027.stdout:6/71: creat f8 x:0 0 0 2021-11-03T17:14:02.944 INFO:tasks.workunit.client.0.smithi027.stdout:5/141: getdents d2/d4/df/d20 0 2021-11-03T17:14:02.946 INFO:tasks.workunit.client.0.smithi027.stdout:6/72: dread f6 [0,4194304] 0 2021-11-03T17:14:02.946 INFO:tasks.workunit.client.0.smithi027.stdout:6/73: stat f6 0 2021-11-03T17:14:02.947 INFO:tasks.workunit.client.0.smithi027.stdout:5/142: creat d2/d4/df/d2d/f35 x:0 0 0 2021-11-03T17:14:02.947 INFO:tasks.workunit.client.0.smithi027.stdout:5/143: stat d2/d4/df/f2f 0 2021-11-03T17:14:02.947 INFO:tasks.workunit.client.0.smithi027.stdout:6/74: write f0 [1902821,66679] 0 2021-11-03T17:14:02.948 INFO:tasks.workunit.client.0.smithi027.stdout:5/144: rmdir d2 39 2021-11-03T17:14:02.950 INFO:tasks.workunit.client.0.smithi027.stdout:6/75: unlink f8 0 2021-11-03T17:14:02.950 INFO:tasks.workunit.client.0.smithi027.stdout:6/76: mknod c9 1 2021-11-03T17:14:02.951 INFO:tasks.workunit.client.0.smithi027.stdout:6/77: creat fa x:0 0 0 2021-11-03T17:14:02.952 INFO:tasks.workunit.client.0.smithi027.stdout:6/78: symlink lb 0 2021-11-03T17:14:02.952 INFO:tasks.workunit.client.0.smithi027.stdout:6/79: chown f6 1 1 2021-11-03T17:14:02.963 INFO:tasks.workunit.client.0.smithi027.stdout:6/80: truncate f0 739689 0 2021-11-03T17:14:02.968 INFO:tasks.workunit.client.0.smithi027.stdout:5/145: dwrite d2/d4/df/f2f [0,4194304] 0 2021-11-03T17:14:02.968 INFO:tasks.workunit.client.0.smithi027.stdout:5/146: dread - d2/f1e zero size 2021-11-03T17:14:02.973 INFO:tasks.workunit.client.0.smithi027.stdout:6/81: read f0 [485483,24160] 0 2021-11-03T17:14:02.974 INFO:tasks.workunit.client.0.smithi027.stdout:5/147: creat d2/f36 x:0 0 0 2021-11-03T17:14:02.974 INFO:tasks.workunit.client.0.smithi027.stdout:5/148: mknod d2/c37 1 2021-11-03T17:14:02.975 INFO:tasks.workunit.client.0.smithi027.stdout:5/149: readlink d2/d4/df/l31 0 2021-11-03T17:14:02.975 INFO:tasks.workunit.client.0.smithi027.stdout:5/150: readlink d2/d4/df/d20/d25/l2b 0 2021-11-03T17:14:02.975 INFO:tasks.workunit.client.0.smithi027.stdout:5/151: write d2/d4/f13 [312659,130270] 0 2021-11-03T17:14:02.976 INFO:tasks.workunit.client.0.smithi027.stdout:6/82: mkdir dc 0 2021-11-03T17:14:02.977 INFO:tasks.workunit.client.0.smithi027.stdout:6/83: dread f0 [0,4194304] 0 2021-11-03T17:14:02.977 INFO:tasks.workunit.client.0.smithi027.stdout:6/84: write f0 [1779623,13095] 0 2021-11-03T17:14:02.978 INFO:tasks.workunit.client.0.smithi027.stdout:6/85: mknod dc/cd 1 2021-11-03T17:14:02.978 INFO:tasks.workunit.client.0.smithi027.stdout:6/86: creat dc/fe x:0 0 0 2021-11-03T17:14:02.990 INFO:tasks.workunit.client.0.smithi027.stdout:6/87: dread f7 [0,4194304] 0 2021-11-03T17:14:02.991 INFO:tasks.workunit.client.0.smithi027.stdout:5/152: dwrite d2/d4/df/d20/d25/f27 [0,4194304] 0 2021-11-03T17:14:02.991 INFO:tasks.workunit.client.0.smithi027.stdout:5/153: read - d2/f36 zero size 2021-11-03T17:14:02.992 INFO:tasks.workunit.client.0.smithi027.stdout:6/88: creat dc/ff x:0 0 0 2021-11-03T17:14:02.992 INFO:tasks.workunit.client.0.smithi027.stdout:6/89: chown dc/ff 0 1 2021-11-03T17:14:02.995 INFO:tasks.workunit.client.0.smithi027.stdout:5/154: rename d2/d4/fa to d2/d1f/f38 0 2021-11-03T17:14:02.995 INFO:tasks.workunit.client.0.smithi027.stdout:6/90: creat dc/f10 x:0 0 0 2021-11-03T17:14:03.000 INFO:tasks.workunit.client.0.smithi027.stdout:6/91: rename f6 to dc/f11 0 2021-11-03T17:14:03.000 INFO:tasks.workunit.client.0.smithi027.stdout:6/92: fdatasync dc/f11 0 2021-11-03T17:14:03.000 INFO:tasks.workunit.client.0.smithi027.stdout:6/93: chown fa 0 1 2021-11-03T17:14:03.001 INFO:tasks.workunit.client.0.smithi027.stdout:6/94: symlink dc/l12 0 2021-11-03T17:14:03.002 INFO:tasks.workunit.client.0.smithi027.stdout:6/95: chown fa 30769 1 2021-11-03T17:14:03.006 INFO:tasks.workunit.client.0.smithi027.stdout:6/96: dread f7 [0,4194304] 0 2021-11-03T17:14:03.011 INFO:tasks.workunit.client.0.smithi027.stdout:6/97: rmdir dc 39 2021-11-03T17:14:03.011 INFO:tasks.workunit.client.0.smithi027.stdout:6/98: dread - dc/ff zero size 2021-11-03T17:14:03.012 INFO:tasks.workunit.client.0.smithi027.stdout:6/99: fsync dc/ff 0 2021-11-03T17:14:03.012 INFO:tasks.workunit.client.0.smithi027.stdout:6/100: dread - dc/fe zero size 2021-11-03T17:14:03.012 INFO:tasks.workunit.client.0.smithi027.stdout:6/101: truncate fa 816191 0 2021-11-03T17:14:03.012 INFO:tasks.workunit.client.0.smithi027.stdout:6/102: chown dc/f10 1728556 1 2021-11-03T17:14:03.012 INFO:tasks.workunit.client.0.smithi027.stdout:6/103: chown f0 1829868533 1 2021-11-03T17:14:03.013 INFO:tasks.workunit.client.0.smithi027.stdout:6/104: symlink dc/l13 0 2021-11-03T17:14:03.013 INFO:tasks.workunit.client.0.smithi027.stdout:6/105: write fa [1553505,71867] 0 2021-11-03T17:14:03.022 INFO:tasks.workunit.client.0.smithi027.stdout:5/155: dwrite d2/d1f/f38 [0,4194304] 0 2021-11-03T17:14:03.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:03 smithi027 conmon[32322]: cluster 2021-11-03T17:14:02.603186+0000 mgr.smithi027.ufhhvb (mgr.14162) 298 : 2021-11-03T17:14:03.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:03 smithi027 conmon[32322]: cluster [DBG] pgmap v248: 65 pgs: 65 active+clean; 736 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 21 MiB/s rd, 48 MiB/s wr, 291 op/s 2021-11-03T17:14:03.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:03 smithi115 conmon[34670]: cluster 2021-11-03T17:14:02.603186+0000 mgr.smithi027.ufhhvb (mgr.14162) 298 : cluster [DBG] pgmap v248: 65 pgs: 65 active+clean; 736 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 21 MiB/s rd, 48 MiB/s wr, 291 op/s 2021-11-03T17:14:03.868 INFO:tasks.workunit.client.1.smithi115.stdout:6/109: sync 2021-11-03T17:14:03.869 INFO:tasks.workunit.client.1.smithi115.stdout:9/74: sync 2021-11-03T17:14:03.869 INFO:tasks.workunit.client.1.smithi115.stdout:3/131: sync 2021-11-03T17:14:03.869 INFO:tasks.workunit.client.1.smithi115.stdout:1/172: sync 2021-11-03T17:14:03.869 INFO:tasks.workunit.client.1.smithi115.stdout:5/190: sync 2021-11-03T17:14:03.869 INFO:tasks.workunit.client.1.smithi115.stdout:0/115: sync 2021-11-03T17:14:03.870 INFO:tasks.workunit.client.1.smithi115.stdout:4/165: sync 2021-11-03T17:14:03.870 INFO:tasks.workunit.client.1.smithi115.stdout:4/166: chown d7/l26 3089 1 2021-11-03T17:14:03.871 INFO:tasks.workunit.client.1.smithi115.stdout:3/132: dread - d13/f23 zero size 2021-11-03T17:14:03.871 INFO:tasks.workunit.client.1.smithi115.stdout:4/167: dread d7/dc/d1a/f2e [0,4194304] 0 2021-11-03T17:14:03.871 INFO:tasks.workunit.client.1.smithi115.stdout:4/168: write d7/f30 [448120,105004] 0 2021-11-03T17:14:03.873 INFO:tasks.workunit.client.1.smithi115.stdout:0/116: rename d9/l28 to d9/db/l30 0 2021-11-03T17:14:03.874 INFO:tasks.workunit.client.1.smithi115.stdout:5/191: mkdir d1a/d28/d40 0 2021-11-03T17:14:03.875 INFO:tasks.workunit.client.1.smithi115.stdout:5/192: readlink d1a/d28/l2e 0 2021-11-03T17:14:03.875 INFO:tasks.workunit.client.1.smithi115.stdout:5/193: fsync d1a/d28/d34/f32 0 2021-11-03T17:14:03.875 INFO:tasks.workunit.client.1.smithi115.stdout:5/194: dread - d1a/f3c zero size 2021-11-03T17:14:03.881 INFO:tasks.workunit.client.1.smithi115.stdout:1/173: symlink d7/dc/d2a/d2d/l34 0 2021-11-03T17:14:03.882 INFO:tasks.workunit.client.1.smithi115.stdout:4/169: mkdir d7/dc/d1a/d21/d32/d3c 0 2021-11-03T17:14:03.882 INFO:tasks.workunit.client.1.smithi115.stdout:4/170: rename d7/dc/d1a/d21/d32/d3c to d7/dc/d1a/d21/d32/d3c/d3d 22 2021-11-03T17:14:03.883 INFO:tasks.workunit.client.1.smithi115.stdout:4/171: chown d7/dc/d1a/d21/d32 3 1 2021-11-03T17:14:03.892 INFO:tasks.workunit.client.1.smithi115.stdout:0/117: dread f7 [0,4194304] 0 2021-11-03T17:14:03.894 INFO:tasks.workunit.client.1.smithi115.stdout:4/172: write d7/dc/f13 [6770671,1902] 0 2021-11-03T17:14:03.894 INFO:tasks.workunit.client.1.smithi115.stdout:0/118: symlink d9/d15/d1a/d1d/l31 0 2021-11-03T17:14:03.895 INFO:tasks.workunit.client.1.smithi115.stdout:0/119: mknod d9/db/c32 1 2021-11-03T17:14:03.896 INFO:tasks.workunit.client.1.smithi115.stdout:0/120: creat d9/d15/f33 x:0 0 0 2021-11-03T17:14:03.897 INFO:tasks.workunit.client.1.smithi115.stdout:0/121: fdatasync d9/d15/f1e 0 2021-11-03T17:14:03.923 INFO:tasks.workunit.client.1.smithi115.stdout:6/110: dwrite d0/d1/d2/f12 [0,4194304] 0 2021-11-03T17:14:03.924 INFO:tasks.workunit.client.1.smithi115.stdout:9/75: dwrite d1/d5/f6 [0,4194304] 0 2021-11-03T17:14:03.925 INFO:tasks.workunit.client.1.smithi115.stdout:4/173: dwrite d7/dc/d1a/d21/d32/f3a [0,4194304] 0 2021-11-03T17:14:03.925 INFO:tasks.workunit.client.1.smithi115.stdout:1/174: dwrite d7/dc/f15 [0,4194304] 0 2021-11-03T17:14:03.926 INFO:tasks.workunit.client.1.smithi115.stdout:0/122: dwrite d9/db/f1b [0,4194304] 0 2021-11-03T17:14:03.926 INFO:tasks.workunit.client.1.smithi115.stdout:0/123: dread - d9/d15/d1a/d1d/f29 zero size 2021-11-03T17:14:03.927 INFO:tasks.workunit.client.1.smithi115.stdout:5/195: dwrite d1a/d28/f31 [0,4194304] 0 2021-11-03T17:14:03.927 INFO:tasks.workunit.client.1.smithi115.stdout:0/124: chown d9/d15/f1e 196446022 1 2021-11-03T17:14:03.927 INFO:tasks.workunit.client.1.smithi115.stdout:3/133: dwrite d13/f16 [0,4194304] 0 2021-11-03T17:14:03.929 INFO:tasks.workunit.client.1.smithi115.stdout:4/174: chown d7/dc/f12 32933214 1 2021-11-03T17:14:03.930 INFO:tasks.workunit.client.1.smithi115.stdout:4/175: write d7/f30 [717200,108784] 0 2021-11-03T17:14:03.930 INFO:tasks.workunit.client.1.smithi115.stdout:6/111: dread d0/d1/d10/f16 [0,4194304] 0 2021-11-03T17:14:03.930 INFO:tasks.workunit.client.1.smithi115.stdout:1/175: creat d7/dc/d1a/f35 x:0 0 0 2021-11-03T17:14:03.931 INFO:tasks.workunit.client.1.smithi115.stdout:6/112: rmdir d0/d1/d2 39 2021-11-03T17:14:03.932 INFO:tasks.workunit.client.1.smithi115.stdout:6/113: truncate d0/f17 448933 0 2021-11-03T17:14:03.933 INFO:tasks.workunit.client.1.smithi115.stdout:4/176: link d7/dc/d1a/f2e d7/dc/d1a/d21/d32/d3c/f3e 0 2021-11-03T17:14:03.934 INFO:tasks.workunit.client.1.smithi115.stdout:1/176: truncate f2 522968 0 2021-11-03T17:14:03.935 INFO:tasks.workunit.client.1.smithi115.stdout:1/177: chown d7/dc/d2a/d2d 66145251 1 2021-11-03T17:14:03.935 INFO:tasks.workunit.client.1.smithi115.stdout:1/178: dread - d7/dc/d2a/f31 zero size 2021-11-03T17:14:03.935 INFO:tasks.workunit.client.1.smithi115.stdout:1/179: rename d7/dc to d7/dc/d2a/d36 22 2021-11-03T17:14:03.935 INFO:tasks.workunit.client.1.smithi115.stdout:1/180: write d7/dc/d2a/f33 [765293,32083] 0 2021-11-03T17:14:03.936 INFO:tasks.workunit.client.1.smithi115.stdout:9/76: dread d1/d5/fc [0,4194304] 0 2021-11-03T17:14:03.937 INFO:tasks.workunit.client.1.smithi115.stdout:9/77: truncate d1/f13 362039 0 2021-11-03T17:14:03.937 INFO:tasks.workunit.client.1.smithi115.stdout:5/196: dread d1a/d28/f31 [0,4194304] 0 2021-11-03T17:14:03.938 INFO:tasks.workunit.client.1.smithi115.stdout:5/197: chown d1a/d28/d34/f32 26259 1 2021-11-03T17:14:03.939 INFO:tasks.workunit.client.1.smithi115.stdout:5/198: read d1a/d28/d34/f2a [1641406,297] 0 2021-11-03T17:14:03.939 INFO:tasks.workunit.client.1.smithi115.stdout:5/199: chown d1a/d23 3973 1 2021-11-03T17:14:03.939 INFO:tasks.workunit.client.1.smithi115.stdout:5/200: fdatasync f0 0 2021-11-03T17:14:03.940 INFO:tasks.workunit.client.1.smithi115.stdout:2/75: sync 2021-11-03T17:14:03.940 INFO:tasks.workunit.client.1.smithi115.stdout:2/76: stat d4/db 0 2021-11-03T17:14:03.940 INFO:tasks.workunit.client.1.smithi115.stdout:2/77: write d4/fa [921946,20344] 0 2021-11-03T17:14:03.941 INFO:tasks.workunit.client.1.smithi115.stdout:4/177: write d7/dc/d1a/d21/f22 [1036183,1488] 0 2021-11-03T17:14:03.943 INFO:tasks.workunit.client.1.smithi115.stdout:1/181: symlink d7/dc/dd/d24/l37 0 2021-11-03T17:14:03.944 INFO:tasks.workunit.client.1.smithi115.stdout:6/114: dread d0/d1/fb [0,4194304] 0 2021-11-03T17:14:03.944 INFO:tasks.workunit.client.1.smithi115.stdout:6/115: stat d0/d1/d2/f1f 0 2021-11-03T17:14:03.945 INFO:tasks.workunit.client.1.smithi115.stdout:6/116: mknod d0/c24 1 2021-11-03T17:14:03.946 INFO:tasks.workunit.client.1.smithi115.stdout:5/201: rename l16 to d1a/d3f/l41 0 2021-11-03T17:14:03.948 INFO:tasks.workunit.client.1.smithi115.stdout:4/178: mkdir d7/dc/d1a/d21/d3f 0 2021-11-03T17:14:03.949 INFO:tasks.workunit.client.1.smithi115.stdout:1/182: mkdir d7/d38 0 2021-11-03T17:14:03.949 INFO:tasks.workunit.client.1.smithi115.stdout:0/125: dwrite d9/d15/d1a/d1d/f24 [0,4194304] 0 2021-11-03T17:14:03.950 INFO:tasks.workunit.client.1.smithi115.stdout:3/134: dwrite f12 [0,4194304] 0 2021-11-03T17:14:03.951 INFO:tasks.workunit.client.1.smithi115.stdout:5/202: truncate f11 344998 0 2021-11-03T17:14:03.952 INFO:tasks.workunit.client.1.smithi115.stdout:2/78: dread d4/d6/fd [0,4194304] 0 2021-11-03T17:14:03.952 INFO:tasks.workunit.client.1.smithi115.stdout:2/79: dread - d4/f10 zero size 2021-11-03T17:14:03.952 INFO:tasks.workunit.client.1.smithi115.stdout:2/80: write d4/fa [1518031,91055] 0 2021-11-03T17:14:03.953 INFO:tasks.workunit.client.1.smithi115.stdout:2/81: write d4/fe [205897,59143] 0 2021-11-03T17:14:03.953 INFO:tasks.workunit.client.1.smithi115.stdout:2/82: mknod d4/d6/c11 1 2021-11-03T17:14:03.956 INFO:tasks.workunit.client.1.smithi115.stdout:0/126: mkdir d9/d15/d2c/d34 0 2021-11-03T17:14:03.956 INFO:tasks.workunit.client.1.smithi115.stdout:0/127: write f7 [1574992,32560] 0 2021-11-03T17:14:03.957 INFO:tasks.workunit.client.1.smithi115.stdout:9/78: write d1/d5/f8 [532982,67146] 0 2021-11-03T17:14:03.957 INFO:tasks.workunit.client.1.smithi115.stdout:9/79: mknod d1/c15 1 2021-11-03T17:14:03.960 INFO:tasks.workunit.client.1.smithi115.stdout:5/203: rmdir d1a/d28/d40 0 2021-11-03T17:14:03.961 INFO:tasks.workunit.client.1.smithi115.stdout:1/183: dread d7/dc/f15 [0,4194304] 0 2021-11-03T17:14:03.961 INFO:tasks.workunit.client.1.smithi115.stdout:9/80: dread d1/f13 [0,4194304] 0 2021-11-03T17:14:03.962 INFO:tasks.workunit.client.1.smithi115.stdout:3/135: dread d13/f16 [0,4194304] 0 2021-11-03T17:14:03.962 INFO:tasks.workunit.client.1.smithi115.stdout:5/204: creat d1a/d3d/f42 x:0 0 0 2021-11-03T17:14:03.963 INFO:tasks.workunit.client.1.smithi115.stdout:5/205: truncate d1a/f3c 971521 0 2021-11-03T17:14:03.964 INFO:tasks.workunit.client.1.smithi115.stdout:1/184: truncate d7/dc/f15 4183887 0 2021-11-03T17:14:03.964 INFO:tasks.workunit.client.1.smithi115.stdout:1/185: chown d7/dc/dd/d24 45785742 1 2021-11-03T17:14:03.965 INFO:tasks.workunit.client.1.smithi115.stdout:1/186: dread d7/dc/dd/d24/f28 [0,4194304] 0 2021-11-03T17:14:03.967 INFO:tasks.workunit.client.1.smithi115.stdout:9/81: symlink d1/l16 0 2021-11-03T17:14:03.967 INFO:tasks.workunit.client.1.smithi115.stdout:4/179: dread d7/f30 [0,4194304] 0 2021-11-03T17:14:03.967 INFO:tasks.workunit.client.1.smithi115.stdout:4/180: readlink d7/dc/d1a/d21/l31 0 2021-11-03T17:14:03.968 INFO:tasks.workunit.client.1.smithi115.stdout:5/206: truncate d1a/d3d/f3b 1042306 0 2021-11-03T17:14:03.969 INFO:tasks.workunit.client.1.smithi115.stdout:1/187: symlink d7/l39 0 2021-11-03T17:14:03.969 INFO:tasks.workunit.client.1.smithi115.stdout:1/188: fsync f2 0 2021-11-03T17:14:03.971 INFO:tasks.workunit.client.1.smithi115.stdout:5/207: getdents d1a/d28/d34 0 2021-11-03T17:14:03.973 INFO:tasks.workunit.client.1.smithi115.stdout:0/128: dread d9/db/f1b [0,4194304] 0 2021-11-03T17:14:03.974 INFO:tasks.workunit.client.1.smithi115.stdout:0/129: write f7 [454681,96494] 0 2021-11-03T17:14:03.975 INFO:tasks.workunit.client.1.smithi115.stdout:6/117: dwrite d0/f17 [0,4194304] 0 2021-11-03T17:14:03.978 INFO:tasks.workunit.client.1.smithi115.stdout:1/189: creat d7/dc/d2a/d2d/d32/f3a x:0 0 0 2021-11-03T17:14:03.978 INFO:tasks.workunit.client.1.smithi115.stdout:5/208: symlink d1a/d28/l43 0 2021-11-03T17:14:03.979 INFO:tasks.workunit.client.1.smithi115.stdout:5/209: chown d1a/d28/l2e 9 1 2021-11-03T17:14:03.979 INFO:tasks.workunit.client.1.smithi115.stdout:2/83: dwrite d4/f10 [0,4194304] 0 2021-11-03T17:14:03.979 INFO:tasks.workunit.client.1.smithi115.stdout:2/84: write d4/d6/fd [1609301,29939] 0 2021-11-03T17:14:03.980 INFO:tasks.workunit.client.1.smithi115.stdout:0/130: dread d9/d15/f1e [0,4194304] 0 2021-11-03T17:14:03.982 INFO:tasks.workunit.client.1.smithi115.stdout:1/190: rename d7/dc/dd/d24/d26/f2c to d7/dc/dd/d24/d26/f3b 0 2021-11-03T17:14:03.984 INFO:tasks.workunit.client.1.smithi115.stdout:3/136: dwrite d13/f23 [0,4194304] 0 2021-11-03T17:14:03.985 INFO:tasks.workunit.client.1.smithi115.stdout:5/210: rename d1a/d3f to d1a/d28/d44 0 2021-11-03T17:14:03.985 INFO:tasks.workunit.client.1.smithi115.stdout:5/211: chown d1a/d28/f29 360 1 2021-11-03T17:14:03.986 INFO:tasks.workunit.client.1.smithi115.stdout:2/85: creat d4/d6/f12 x:0 0 0 2021-11-03T17:14:03.989 INFO:tasks.workunit.client.1.smithi115.stdout:0/131: creat d9/f35 x:0 0 0 2021-11-03T17:14:03.989 INFO:tasks.workunit.client.1.smithi115.stdout:0/132: write d9/f35 [440206,53425] 0 2021-11-03T17:14:03.989 INFO:tasks.workunit.client.1.smithi115.stdout:0/133: mknod d9/d15/d1f/c36 1 2021-11-03T17:14:03.989 INFO:tasks.workunit.client.1.smithi115.stdout:0/134: chown l4 70757350 1 2021-11-03T17:14:03.990 INFO:tasks.workunit.client.1.smithi115.stdout:0/135: dread d9/d15/d1a/d1d/f22 [0,4194304] 0 2021-11-03T17:14:03.990 INFO:tasks.workunit.client.1.smithi115.stdout:0/136: fdatasync d9/d15/d1a/f2e 0 2021-11-03T17:14:03.991 INFO:tasks.workunit.client.1.smithi115.stdout:4/181: dread d7/dc/f13 [0,4194304] 0 2021-11-03T17:14:03.991 INFO:tasks.workunit.client.1.smithi115.stdout:1/191: truncate d7/f9 2750408 0 2021-11-03T17:14:03.991 INFO:tasks.workunit.client.1.smithi115.stdout:4/182: mknod d7/dc/d1a/c40 1 2021-11-03T17:14:03.991 INFO:tasks.workunit.client.1.smithi115.stdout:1/192: readlink d7/dc/dd/d24/l37 0 2021-11-03T17:14:03.992 INFO:tasks.workunit.client.1.smithi115.stdout:3/137: creat d13/f27 x:0 0 0 2021-11-03T17:14:03.993 INFO:tasks.workunit.client.1.smithi115.stdout:5/212: creat d1a/d28/d34/f45 x:0 0 0 2021-11-03T17:14:03.996 INFO:tasks.workunit.client.1.smithi115.stdout:2/86: mkdir d4/d6/d13 0 2021-11-03T17:14:03.997 INFO:tasks.workunit.client.1.smithi115.stdout:2/87: stat d4/fe 0 2021-11-03T17:14:03.997 INFO:tasks.workunit.client.1.smithi115.stdout:0/137: getdents d9/d15/d1a/d1d 0 2021-11-03T17:14:03.997 INFO:tasks.workunit.client.1.smithi115.stdout:1/193: write d7/dc/ff [1820821,105208] 0 2021-11-03T17:14:03.999 INFO:tasks.workunit.client.1.smithi115.stdout:9/82: dwrite d1/d5/f12 [0,4194304] 0 2021-11-03T17:14:03.999 INFO:tasks.workunit.client.1.smithi115.stdout:6/118: dwrite d0/f15 [4194304,4194304] 0 2021-11-03T17:14:03.999 INFO:tasks.workunit.client.1.smithi115.stdout:6/119: fdatasync d0/d1/d2/f12 0 2021-11-03T17:14:04.000 INFO:tasks.workunit.client.1.smithi115.stdout:4/183: symlink d7/dc/d1a/d21/d32/l41 0 2021-11-03T17:14:04.000 INFO:tasks.workunit.client.1.smithi115.stdout:6/120: write d0/d1/d10/f16 [1044856,59273] 0 2021-11-03T17:14:04.009 INFO:tasks.workunit.client.1.smithi115.stdout:5/213: link d1a/l22 d1a/d3d/l46 0 2021-11-03T17:14:04.010 INFO:tasks.workunit.client.1.smithi115.stdout:2/88: creat d4/db/f14 x:0 0 0 2021-11-03T17:14:04.011 INFO:tasks.workunit.client.1.smithi115.stdout:0/138: mkdir d9/d37 0 2021-11-03T17:14:04.014 INFO:tasks.workunit.client.1.smithi115.stdout:1/194: rename d7/fe to d7/dc/d2a/d2d/f3c 0 2021-11-03T17:14:04.017 INFO:tasks.workunit.client.1.smithi115.stdout:9/83: rename d1/d5/fe to d1/d5/f17 0 2021-11-03T17:14:04.017 INFO:tasks.workunit.client.1.smithi115.stdout:9/84: fdatasync d1/f7 0 2021-11-03T17:14:04.018 INFO:tasks.workunit.client.1.smithi115.stdout:9/85: write d1/f9 [8631277,15546] 0 2021-11-03T17:14:04.018 INFO:tasks.workunit.client.1.smithi115.stdout:4/184: mkdir d7/dc/d36/d42 0 2021-11-03T17:14:04.021 INFO:tasks.workunit.client.1.smithi115.stdout:6/121: creat d0/d1/d10/d13/f25 x:0 0 0 2021-11-03T17:14:04.021 INFO:tasks.workunit.client.1.smithi115.stdout:6/122: dread - d0/d1/d10/d13/d1c/f21 zero size 2021-11-03T17:14:04.021 INFO:tasks.workunit.client.1.smithi115.stdout:6/123: readlink - no filename 2021-11-03T17:14:04.021 INFO:tasks.workunit.client.1.smithi115.stdout:6/124: chown d0/d1/d10/d13/d1c/f21 106 1 2021-11-03T17:14:04.026 INFO:tasks.workunit.client.1.smithi115.stdout:2/89: mknod d4/c15 1 2021-11-03T17:14:04.031 INFO:tasks.workunit.client.1.smithi115.stdout:0/139: dwrite d9/db/f20 [0,4194304] 0 2021-11-03T17:14:04.031 INFO:tasks.workunit.client.1.smithi115.stdout:0/140: fdatasync d9/d15/d1a/d1d/f22 0 2021-11-03T17:14:04.037 INFO:tasks.workunit.client.1.smithi115.stdout:1/195: creat d7/dc/d1a/f3d x:0 0 0 2021-11-03T17:14:04.038 INFO:tasks.workunit.client.1.smithi115.stdout:3/138: link d13/l15 d13/l28 0 2021-11-03T17:14:04.039 INFO:tasks.workunit.client.1.smithi115.stdout:6/125: symlink d0/d1/d10/l26 0 2021-11-03T17:14:04.040 INFO:tasks.workunit.client.1.smithi115.stdout:2/90: symlink d4/d6/l16 0 2021-11-03T17:14:04.040 INFO:tasks.workunit.client.1.smithi115.stdout:2/91: readlink d4/db/lf 0 2021-11-03T17:14:04.040 INFO:tasks.workunit.client.1.smithi115.stdout:2/92: getdents d4/d6/d13 0 2021-11-03T17:14:04.041 INFO:tasks.workunit.client.1.smithi115.stdout:2/93: dread - d4/d6/f12 zero size 2021-11-03T17:14:04.041 INFO:tasks.workunit.client.1.smithi115.stdout:6/126: write d0/d1/d10/f16 [403986,53343] 0 2021-11-03T17:14:04.042 INFO:tasks.workunit.client.1.smithi115.stdout:3/139: read f10 [3025714,61020] 0 2021-11-03T17:14:04.042 INFO:tasks.workunit.client.1.smithi115.stdout:0/141: rename d9/d15/d1a/d1d/l31 to d9/d15/d2c/d34/l38 0 2021-11-03T17:14:04.043 INFO:tasks.workunit.client.1.smithi115.stdout:0/142: truncate d9/f23 1568082 0 2021-11-03T17:14:04.043 INFO:tasks.workunit.client.1.smithi115.stdout:1/196: mkdir d7/dc/dd/d3e 0 2021-11-03T17:14:04.044 INFO:tasks.workunit.client.1.smithi115.stdout:2/94: creat d4/db/f17 x:0 0 0 2021-11-03T17:14:04.044 INFO:tasks.workunit.client.1.smithi115.stdout:2/95: mknod d4/d6/c18 1 2021-11-03T17:14:04.045 INFO:tasks.workunit.client.1.smithi115.stdout:0/143: dread d9/f23 [0,4194304] 0 2021-11-03T17:14:04.046 INFO:tasks.workunit.client.1.smithi115.stdout:4/185: dwrite d7/dc/d1a/d21/f22 [0,4194304] 0 2021-11-03T17:14:04.046 INFO:tasks.workunit.client.1.smithi115.stdout:4/186: mknod d7/dc/d36/d42/c43 1 2021-11-03T17:14:04.046 INFO:tasks.workunit.client.1.smithi115.stdout:6/127: dread d0/d1/d2/fc [0,4194304] 0 2021-11-03T17:14:04.047 INFO:tasks.workunit.client.1.smithi115.stdout:2/96: link d4/db/lf d4/db/l19 0 2021-11-03T17:14:04.047 INFO:tasks.workunit.client.1.smithi115.stdout:3/140: write d13/f23 [1438119,17977] 0 2021-11-03T17:14:04.048 INFO:tasks.workunit.client.1.smithi115.stdout:3/141: dread - d13/f1f zero size 2021-11-03T17:14:04.048 INFO:tasks.workunit.client.1.smithi115.stdout:4/187: mkdir d7/dc/d1a/d21/d37/d44 0 2021-11-03T17:14:04.048 INFO:tasks.workunit.client.1.smithi115.stdout:4/188: chown d7/lb 363620 1 2021-11-03T17:14:04.048 INFO:tasks.workunit.client.1.smithi115.stdout:6/128: creat d0/d1/d10/f27 x:0 0 0 2021-11-03T17:14:04.050 INFO:tasks.workunit.client.1.smithi115.stdout:2/97: creat d4/d6/f1a x:0 0 0 2021-11-03T17:14:04.050 INFO:tasks.workunit.client.1.smithi115.stdout:0/144: dread d9/db/f1b [0,4194304] 0 2021-11-03T17:14:04.050 INFO:tasks.workunit.client.1.smithi115.stdout:0/145: write f7 [140009,127197] 0 2021-11-03T17:14:04.050 INFO:tasks.workunit.client.1.smithi115.stdout:0/146: truncate d9/f25 834755 0 2021-11-03T17:14:04.051 INFO:tasks.workunit.client.1.smithi115.stdout:0/147: dread - d9/d15/d1a/d1d/f29 zero size 2021-11-03T17:14:04.051 INFO:tasks.workunit.client.1.smithi115.stdout:3/142: creat d13/d20/f29 x:0 0 0 2021-11-03T17:14:04.051 INFO:tasks.workunit.client.1.smithi115.stdout:6/129: dread d0/d1/d2/fc [0,4194304] 0 2021-11-03T17:14:04.051 INFO:tasks.workunit.client.1.smithi115.stdout:6/130: write d0/d1/d10/d13/f25 [746863,21914] 0 2021-11-03T17:14:04.052 INFO:tasks.workunit.client.1.smithi115.stdout:4/189: rename d7/dc/d1a/l1e to d7/dc/d1a/d21/d32/l45 0 2021-11-03T17:14:04.052 INFO:tasks.workunit.client.1.smithi115.stdout:4/190: truncate d7/f30 1147473 0 2021-11-03T17:14:04.053 INFO:tasks.workunit.client.1.smithi115.stdout:9/86: dwrite d1/f9 [4194304,4194304] 0 2021-11-03T17:14:04.053 INFO:tasks.workunit.client.1.smithi115.stdout:5/214: dwrite f11 [0,4194304] 0 2021-11-03T17:14:04.055 INFO:tasks.workunit.client.1.smithi115.stdout:4/191: creat d7/dc/d1a/d21/d32/f46 x:0 0 0 2021-11-03T17:14:04.056 INFO:tasks.workunit.client.1.smithi115.stdout:2/98: fsync d4/fe 0 2021-11-03T17:14:04.057 INFO:tasks.workunit.client.1.smithi115.stdout:9/87: mkdir d1/d18 0 2021-11-03T17:14:04.059 INFO:tasks.workunit.client.1.smithi115.stdout:1/197: dread d7/f14 [0,4194304] 0 2021-11-03T17:14:04.061 INFO:tasks.workunit.client.1.smithi115.stdout:8/201: sync 2021-11-03T17:14:04.061 INFO:tasks.workunit.client.1.smithi115.stdout:7/108: sync 2021-11-03T17:14:04.062 INFO:tasks.workunit.client.1.smithi115.stdout:3/143: mkdir d13/d2a 0 2021-11-03T17:14:04.065 INFO:tasks.workunit.client.1.smithi115.stdout:0/148: rmdir d9/d15/d1a 39 2021-11-03T17:14:04.065 INFO:tasks.workunit.client.1.smithi115.stdout:0/149: dread - d9/d15/d1a/f2e zero size 2021-11-03T17:14:04.065 INFO:tasks.workunit.client.1.smithi115.stdout:0/150: dread - d9/d15/f33 zero size 2021-11-03T17:14:04.068 INFO:tasks.workunit.client.1.smithi115.stdout:1/198: dread d7/dc/dd/d24/d26/f3b [0,4194304] 0 2021-11-03T17:14:04.071 INFO:tasks.workunit.client.1.smithi115.stdout:4/192: mkdir d7/dc/d1a/d21/d47 0 2021-11-03T17:14:04.072 INFO:tasks.workunit.client.1.smithi115.stdout:2/99: creat d4/db/f1b x:0 0 0 2021-11-03T17:14:04.074 INFO:tasks.workunit.client.1.smithi115.stdout:7/109: rename d3/l8 to d3/da/l1f 0 2021-11-03T17:14:04.075 INFO:tasks.workunit.client.1.smithi115.stdout:8/202: getdents d4/d6/d3f 0 2021-11-03T17:14:04.076 INFO:tasks.workunit.client.1.smithi115.stdout:3/144: creat d13/f2b x:0 0 0 2021-11-03T17:14:04.076 INFO:tasks.workunit.client.1.smithi115.stdout:6/131: dwrite d0/d1/d2/ff [0,4194304] 0 2021-11-03T17:14:04.077 INFO:tasks.workunit.client.1.smithi115.stdout:9/88: dwrite d1/d5/f8 [0,4194304] 0 2021-11-03T17:14:04.077 INFO:tasks.workunit.client.1.smithi115.stdout:9/89: mknod d1/c19 1 2021-11-03T17:14:04.077 INFO:tasks.workunit.client.1.smithi115.stdout:0/151: unlink d9/db/df/f12 0 2021-11-03T17:14:04.080 INFO:tasks.workunit.client.1.smithi115.stdout:2/100: rename d4/d6/fd to d4/d6/f1c 0 2021-11-03T17:14:04.080 INFO:tasks.workunit.client.1.smithi115.stdout:2/101: fdatasync d4/db/f1b 0 2021-11-03T17:14:04.080 INFO:tasks.workunit.client.1.smithi115.stdout:2/102: mknod d4/d6/c1d 1 2021-11-03T17:14:04.080 INFO:tasks.workunit.client.1.smithi115.stdout:2/103: chown d4/d6 8123200 1 2021-11-03T17:14:04.081 INFO:tasks.workunit.client.1.smithi115.stdout:2/104: write d4/d6/f12 [950403,118402] 0 2021-11-03T17:14:04.081 INFO:tasks.workunit.client.1.smithi115.stdout:2/105: truncate d4/f10 4652412 0 2021-11-03T17:14:04.081 INFO:tasks.workunit.client.1.smithi115.stdout:2/106: write d4/db/f14 [855817,6884] 0 2021-11-03T17:14:04.081 INFO:tasks.workunit.client.1.smithi115.stdout:7/110: symlink d3/l20 0 2021-11-03T17:14:04.082 INFO:tasks.workunit.client.1.smithi115.stdout:7/111: mknod d3/c21 1 2021-11-03T17:14:04.082 INFO:tasks.workunit.client.1.smithi115.stdout:7/112: chown d3/ld 390 1 2021-11-03T17:14:04.082 INFO:tasks.workunit.client.1.smithi115.stdout:7/113: stat d3/l20 0 2021-11-03T17:14:04.082 INFO:tasks.workunit.client.1.smithi115.stdout:7/114: write d3/f1e [862342,28283] 0 2021-11-03T17:14:04.083 INFO:tasks.workunit.client.1.smithi115.stdout:8/203: rename d4/d6/d18/d30/f32 to d4/d6/d3f/f45 0 2021-11-03T17:14:04.083 INFO:tasks.workunit.client.1.smithi115.stdout:8/204: chown d4/d6/d18/d30/f34 2096526707 1 2021-11-03T17:14:04.085 INFO:tasks.workunit.client.1.smithi115.stdout:6/132: creat d0/d1/d10/f28 x:0 0 0 2021-11-03T17:14:04.088 INFO:tasks.workunit.client.1.smithi115.stdout:9/90: getdents d1 0 2021-11-03T17:14:04.089 INFO:tasks.workunit.client.1.smithi115.stdout:0/152: creat d9/d15/d1f/f39 x:0 0 0 2021-11-03T17:14:04.090 INFO:tasks.workunit.client.1.smithi115.stdout:4/193: link d7/dc/d1a/d21/d32/f3a d7/dc/f48 0 2021-11-03T17:14:04.091 INFO:tasks.workunit.client.1.smithi115.stdout:4/194: chown d7/lb 0 1 2021-11-03T17:14:04.091 INFO:tasks.workunit.client.1.smithi115.stdout:7/115: dread d3/f13 [0,4194304] 0 2021-11-03T17:14:04.092 INFO:tasks.workunit.client.1.smithi115.stdout:7/116: chown d3/da/d18 48220 1 2021-11-03T17:14:04.095 INFO:tasks.workunit.client.1.smithi115.stdout:2/107: rename d4/db to d4/d6/d13/d1e 0 2021-11-03T17:14:04.095 INFO:tasks.workunit.client.1.smithi115.stdout:2/108: mknod d4/d6/d13/c1f 1 2021-11-03T17:14:04.095 INFO:tasks.workunit.client.1.smithi115.stdout:2/109: mknod d4/d6/d13/c20 1 2021-11-03T17:14:04.096 INFO:tasks.workunit.client.1.smithi115.stdout:8/205: dread d4/d6/d18/d30/f34 [0,4194304] 0 2021-11-03T17:14:04.096 INFO:tasks.workunit.client.1.smithi115.stdout:3/145: dwrite d13/f19 [0,4194304] 0 2021-11-03T17:14:04.097 INFO:tasks.workunit.client.1.smithi115.stdout:6/133: creat d0/d1/d2/f29 x:0 0 0 2021-11-03T17:14:04.097 INFO:tasks.workunit.client.1.smithi115.stdout:6/134: chown d0/f22 23840 1 2021-11-03T17:14:04.097 INFO:tasks.workunit.client.1.smithi115.stdout:9/91: mkdir d1/d1a 0 2021-11-03T17:14:04.099 INFO:tasks.workunit.client.1.smithi115.stdout:0/153: creat d9/d15/f3a x:0 0 0 2021-11-03T17:14:04.100 INFO:tasks.workunit.client.1.smithi115.stdout:4/195: creat d7/dc/d1a/f49 x:0 0 0 2021-11-03T17:14:04.100 INFO:tasks.workunit.client.1.smithi115.stdout:7/117: rmdir d3/da 39 2021-11-03T17:14:04.103 INFO:tasks.workunit.client.1.smithi115.stdout:2/110: creat d4/d6/d13/f21 x:0 0 0 2021-11-03T17:14:04.103 INFO:tasks.workunit.client.1.smithi115.stdout:2/111: chown d4/d6/f1a 6798567 1 2021-11-03T17:14:04.103 INFO:tasks.workunit.client.1.smithi115.stdout:2/112: mknod d4/d6/c22 1 2021-11-03T17:14:04.104 INFO:tasks.workunit.client.1.smithi115.stdout:8/206: read d4/d6/f2a [188427,83648] 0 2021-11-03T17:14:04.105 INFO:tasks.workunit.client.1.smithi115.stdout:8/207: write d4/d6/d18/f2d [1142260,85509] 0 2021-11-03T17:14:04.108 INFO:tasks.workunit.client.1.smithi115.stdout:6/135: creat d0/f2a x:0 0 0 2021-11-03T17:14:04.108 INFO:tasks.workunit.client.1.smithi115.stdout:6/136: dread - d0/f2a zero size 2021-11-03T17:14:04.109 INFO:tasks.workunit.client.1.smithi115.stdout:9/92: write d1/d5/f17 [1221198,23675] 0 2021-11-03T17:14:04.110 INFO:tasks.workunit.client.1.smithi115.stdout:4/196: symlink d7/dc/d1a/d21/d32/l4a 0 2021-11-03T17:14:04.111 INFO:tasks.workunit.client.1.smithi115.stdout:7/118: creat d3/da/d18/f22 x:0 0 0 2021-11-03T17:14:04.111 INFO:tasks.workunit.client.1.smithi115.stdout:8/208: rmdir d4/d6/d18/d30 39 2021-11-03T17:14:04.113 INFO:tasks.workunit.client.1.smithi115.stdout:3/146: mkdir d13/d2c 0 2021-11-03T17:14:04.113 INFO:tasks.workunit.client.1.smithi115.stdout:3/147: truncate d13/f27 68529 0 2021-11-03T17:14:04.113 INFO:tasks.workunit.client.1.smithi115.stdout:3/148: write d13/f25 [150806,106419] 0 2021-11-03T17:14:04.114 INFO:tasks.workunit.client.1.smithi115.stdout:0/154: dwrite d9/d15/d1a/d1d/f24 [0,4194304] 0 2021-11-03T17:14:04.114 INFO:tasks.workunit.client.1.smithi115.stdout:6/137: creat d0/d1/d10/d13/d1c/d1e/f2b x:0 0 0 2021-11-03T17:14:04.115 INFO:tasks.workunit.client.1.smithi115.stdout:9/93: mknod d1/d18/c1b 1 2021-11-03T17:14:04.117 INFO:tasks.workunit.client.1.smithi115.stdout:2/113: rename d4/d6/d13/d1e/l19 to d4/l23 0 2021-11-03T17:14:04.118 INFO:tasks.workunit.client.1.smithi115.stdout:0/155: rename d9 to d9/d37/d3b 22 2021-11-03T17:14:04.118 INFO:tasks.workunit.client.1.smithi115.stdout:0/156: mknod d9/d37/c3c 1 2021-11-03T17:14:04.118 INFO:tasks.workunit.client.1.smithi115.stdout:2/114: mknod d4/d6/d13/d1e/c24 1 2021-11-03T17:14:04.120 INFO:tasks.workunit.client.1.smithi115.stdout:3/149: creat d13/d20/f2d x:0 0 0 2021-11-03T17:14:04.121 INFO:tasks.workunit.client.1.smithi115.stdout:6/138: symlink d0/d1/d10/l2c 0 2021-11-03T17:14:04.121 INFO:tasks.workunit.client.1.smithi115.stdout:6/139: mknod d0/d1/d2/c2d 1 2021-11-03T17:14:04.121 INFO:tasks.workunit.client.1.smithi115.stdout:6/140: fdatasync d0/d1/d10/d13/f25 0 2021-11-03T17:14:04.121 INFO:tasks.workunit.client.1.smithi115.stdout:6/141: chown d0/d1 473 1 2021-11-03T17:14:04.124 INFO:tasks.workunit.client.1.smithi115.stdout:7/119: dread d3/f1e [0,4194304] 0 2021-11-03T17:14:04.124 INFO:tasks.workunit.client.1.smithi115.stdout:9/94: link d1/d5/f17 d1/d18/f1c 0 2021-11-03T17:14:04.125 INFO:tasks.workunit.client.1.smithi115.stdout:9/95: readlink d1/d5/l11 0 2021-11-03T17:14:04.126 INFO:tasks.workunit.client.1.smithi115.stdout:2/115: creat d4/d6/d13/f25 x:0 0 0 2021-11-03T17:14:04.127 INFO:tasks.workunit.client.1.smithi115.stdout:3/150: write d13/f19 [4875556,7200] 0 2021-11-03T17:14:04.129 INFO:tasks.workunit.client.1.smithi115.stdout:4/197: dwrite d7/dc/d1a/d21/d32/d3c/f3e [0,4194304] 0 2021-11-03T17:14:04.130 INFO:tasks.workunit.client.1.smithi115.stdout:7/120: creat d3/da/d18/f23 x:0 0 0 2021-11-03T17:14:04.131 INFO:tasks.workunit.client.1.smithi115.stdout:9/96: creat d1/f1d x:0 0 0 2021-11-03T17:14:04.131 INFO:tasks.workunit.client.1.smithi115.stdout:2/116: dread d4/d6/f12 [0,4194304] 0 2021-11-03T17:14:04.131 INFO:tasks.workunit.client.1.smithi115.stdout:2/117: chown d4/d6/f1c 10761 1 2021-11-03T17:14:04.132 INFO:tasks.workunit.client.1.smithi115.stdout:3/151: truncate f12 2550138 0 2021-11-03T17:14:04.132 INFO:tasks.workunit.client.1.smithi115.stdout:3/152: write d13/f27 [1090081,109954] 0 2021-11-03T17:14:04.132 INFO:tasks.workunit.client.1.smithi115.stdout:3/153: chown d13/f23 48 1 2021-11-03T17:14:04.132 INFO:tasks.workunit.client.1.smithi115.stdout:3/154: write d13/f26 [321181,61566] 0 2021-11-03T17:14:04.133 INFO:tasks.workunit.client.1.smithi115.stdout:7/121: link d3/ld d3/da/l24 0 2021-11-03T17:14:04.134 INFO:tasks.workunit.client.1.smithi115.stdout:7/122: chown d3/da/f15 448 1 2021-11-03T17:14:04.134 INFO:tasks.workunit.client.1.smithi115.stdout:3/155: mkdir d13/d20/d2e 0 2021-11-03T17:14:04.136 INFO:tasks.workunit.client.1.smithi115.stdout:7/123: truncate d3/f14 938204 0 2021-11-03T17:14:04.136 INFO:tasks.workunit.client.1.smithi115.stdout:7/124: chown d3/da/l24 28886 1 2021-11-03T17:14:04.136 INFO:tasks.workunit.client.1.smithi115.stdout:7/125: fsync d3/f13 0 2021-11-03T17:14:04.136 INFO:tasks.workunit.client.1.smithi115.stdout:7/126: truncate d3/da/f15 693844 0 2021-11-03T17:14:04.137 INFO:tasks.workunit.client.1.smithi115.stdout:3/156: rename d13/f16 to d13/d20/d2e/f2f 0 2021-11-03T17:14:04.137 INFO:tasks.workunit.client.1.smithi115.stdout:3/157: rename d13/d2c to d13/d2c/d30 22 2021-11-03T17:14:04.138 INFO:tasks.workunit.client.1.smithi115.stdout:0/157: dwrite d9/f25 [0,4194304] 0 2021-11-03T17:14:04.139 INFO:tasks.workunit.client.1.smithi115.stdout:3/158: mknod d13/c31 1 2021-11-03T17:14:04.140 INFO:tasks.workunit.client.1.smithi115.stdout:4/198: dread d7/dc/d1a/d21/f22 [0,4194304] 0 2021-11-03T17:14:04.141 INFO:tasks.workunit.client.1.smithi115.stdout:2/118: dread d4/f10 [0,4194304] 0 2021-11-03T17:14:04.142 INFO:tasks.workunit.client.1.smithi115.stdout:0/158: dread d9/f23 [0,4194304] 0 2021-11-03T17:14:04.144 INFO:tasks.workunit.client.1.smithi115.stdout:6/142: getdents d0/d1/d10/d13/d1c/d1e 0 2021-11-03T17:14:04.144 INFO:tasks.workunit.client.1.smithi115.stdout:6/143: chown d0/d1/d2/ff 639550 1 2021-11-03T17:14:04.145 INFO:tasks.workunit.client.1.smithi115.stdout:7/127: rename d3/da/d18/f1c to d3/f25 0 2021-11-03T17:14:04.147 INFO:tasks.workunit.client.1.smithi115.stdout:7/128: write d3/da/f15 [25799,51251] 0 2021-11-03T17:14:04.147 INFO:tasks.workunit.client.1.smithi115.stdout:7/129: write d3/da/f1b [4938894,109168] 0 2021-11-03T17:14:04.149 INFO:tasks.workunit.client.1.smithi115.stdout:3/159: readlink d13/l17 0 2021-11-03T17:14:04.149 INFO:tasks.workunit.client.1.smithi115.stdout:3/160: fdatasync d13/d20/d2e/f2f 0 2021-11-03T17:14:04.149 INFO:tasks.workunit.client.1.smithi115.stdout:3/161: fsync d13/d20/f2d 0 2021-11-03T17:14:04.151 INFO:tasks.workunit.client.1.smithi115.stdout:2/119: getdents d4/d6/d13 0 2021-11-03T17:14:04.151 INFO:tasks.workunit.client.1.smithi115.stdout:2/120: dread - d4/d6/f1a zero size 2021-11-03T17:14:04.153 INFO:tasks.workunit.client.1.smithi115.stdout:0/159: creat d9/d15/d1a/f3d x:0 0 0 2021-11-03T17:14:04.154 INFO:tasks.workunit.client.1.smithi115.stdout:0/160: dread d9/f23 [0,4194304] 0 2021-11-03T17:14:04.155 INFO:tasks.workunit.client.1.smithi115.stdout:0/161: write d9/d15/f16 [814884,9467] 0 2021-11-03T17:14:04.155 INFO:tasks.workunit.client.1.smithi115.stdout:3/162: creat d13/d20/d2e/f32 x:0 0 0 2021-11-03T17:14:04.156 INFO:tasks.workunit.client.1.smithi115.stdout:6/144: mkdir d0/d1/d2e 0 2021-11-03T17:14:04.157 INFO:tasks.workunit.client.1.smithi115.stdout:6/145: readlink d0/d1/d10/l2c 0 2021-11-03T17:14:04.157 INFO:tasks.workunit.client.1.smithi115.stdout:6/146: dread - d0/d1/d10/f28 zero size 2021-11-03T17:14:04.157 INFO:tasks.workunit.client.1.smithi115.stdout:0/162: rename d9/db/l30 to d9/l3e 0 2021-11-03T17:14:04.158 INFO:tasks.workunit.client.1.smithi115.stdout:0/163: readlink d9/d15/d1a/l27 0 2021-11-03T17:14:04.158 INFO:tasks.workunit.client.1.smithi115.stdout:3/163: creat d13/f33 x:0 0 0 2021-11-03T17:14:04.159 INFO:tasks.workunit.client.1.smithi115.stdout:3/164: mknod d13/d2c/c34 1 2021-11-03T17:14:04.159 INFO:tasks.workunit.client.1.smithi115.stdout:2/121: mkdir d4/d26 0 2021-11-03T17:14:04.159 INFO:tasks.workunit.client.1.smithi115.stdout:4/199: dwrite d7/dc/d1a/f49 [0,4194304] 0 2021-11-03T17:14:04.160 INFO:tasks.workunit.client.1.smithi115.stdout:6/147: rename d0/d1/d2/f29 to d0/d1/d10/d13/d1c/f2f 0 2021-11-03T17:14:04.162 INFO:tasks.workunit.client.1.smithi115.stdout:0/164: unlink d9/d15/d1a/l2d 0 2021-11-03T17:14:04.162 INFO:tasks.workunit.client.1.smithi115.stdout:0/165: mknod d9/db/df/c3f 1 2021-11-03T17:14:04.162 INFO:tasks.workunit.client.1.smithi115.stdout:0/166: mknod d9/d15/d1a/c40 1 2021-11-03T17:14:04.162 INFO:tasks.workunit.client.1.smithi115.stdout:0/167: read f8 [1527089,96683] 0 2021-11-03T17:14:04.163 INFO:tasks.workunit.client.1.smithi115.stdout:0/168: write f0 [2659616,27990] 0 2021-11-03T17:14:04.168 INFO:tasks.workunit.client.1.smithi115.stdout:0/169: dread f7 [0,4194304] 0 2021-11-03T17:14:04.168 INFO:tasks.workunit.client.1.smithi115.stdout:0/170: chown d9/db/df 23 1 2021-11-03T17:14:04.169 INFO:tasks.workunit.client.1.smithi115.stdout:0/171: mknod d9/d15/d1a/c41 1 2021-11-03T17:14:04.170 INFO:tasks.workunit.client.1.smithi115.stdout:2/122: unlink d4/d6/f1a 0 2021-11-03T17:14:04.171 INFO:tasks.workunit.client.1.smithi115.stdout:2/123: chown d4/d6/l16 6 1 2021-11-03T17:14:04.173 INFO:tasks.workunit.client.1.smithi115.stdout:3/165: mkdir d13/d35 0 2021-11-03T17:14:04.173 INFO:tasks.workunit.client.1.smithi115.stdout:2/124: mknod d4/c27 1 2021-11-03T17:14:04.174 INFO:tasks.workunit.client.1.smithi115.stdout:7/130: dwrite d3/f13 [0,4194304] 0 2021-11-03T17:14:04.174 INFO:tasks.workunit.client.1.smithi115.stdout:7/131: fdatasync d3/f25 0 2021-11-03T17:14:04.175 INFO:tasks.workunit.client.1.smithi115.stdout:7/132: read d3/f1e [195557,121937] 0 2021-11-03T17:14:04.176 INFO:tasks.workunit.client.1.smithi115.stdout:4/200: link d7/dc/l15 d7/dc/d1a/d21/d32/l4b 0 2021-11-03T17:14:04.176 INFO:tasks.workunit.client.1.smithi115.stdout:4/201: fdatasync d7/dc/d1a/d21/d32/f46 0 2021-11-03T17:14:04.176 INFO:tasks.workunit.client.1.smithi115.stdout:0/172: dread d9/d15/d1a/d1d/f24 [0,4194304] 0 2021-11-03T17:14:04.176 INFO:tasks.workunit.client.1.smithi115.stdout:0/173: dread - d9/d15/f3a zero size 2021-11-03T17:14:04.177 INFO:tasks.workunit.client.1.smithi115.stdout:7/133: write d3/da/f1b [2678652,79494] 0 2021-11-03T17:14:04.177 INFO:tasks.workunit.client.1.smithi115.stdout:6/148: symlink d0/d1/l30 0 2021-11-03T17:14:04.178 INFO:tasks.workunit.client.1.smithi115.stdout:6/149: truncate d0/d1/d10/d13/f25 997606 0 2021-11-03T17:14:04.178 INFO:tasks.workunit.client.1.smithi115.stdout:6/150: chown d0/d1/d10/l2c 50218333 1 2021-11-03T17:14:04.178 INFO:tasks.workunit.client.1.smithi115.stdout:3/166: creat d13/d35/f36 x:0 0 0 2021-11-03T17:14:04.179 INFO:tasks.workunit.client.1.smithi115.stdout:2/125: mkdir d4/d26/d28 0 2021-11-03T17:14:04.179 INFO:tasks.workunit.client.1.smithi115.stdout:2/126: mknod d4/d6/c29 1 2021-11-03T17:14:04.180 INFO:tasks.workunit.client.1.smithi115.stdout:2/127: write d4/d6/d13/d1e/f17 [972275,95556] 0 2021-11-03T17:14:04.181 INFO:tasks.workunit.client.1.smithi115.stdout:2/128: dread d4/fa [0,4194304] 0 2021-11-03T17:14:04.182 INFO:tasks.workunit.client.1.smithi115.stdout:2/129: write d4/d6/d13/d1e/f14 [246567,110517] 0 2021-11-03T17:14:04.183 INFO:tasks.workunit.client.1.smithi115.stdout:4/202: symlink d7/dc/d36/l4c 0 2021-11-03T17:14:04.183 INFO:tasks.workunit.client.1.smithi115.stdout:4/203: mknod d7/d2b/c4d 1 2021-11-03T17:14:04.183 INFO:tasks.workunit.client.1.smithi115.stdout:4/204: fsync d7/dc/d1a/d21/d32/d3c/f3e 0 2021-11-03T17:14:04.184 INFO:tasks.workunit.client.1.smithi115.stdout:6/151: dread d0/d1/d2/fd [0,4194304] 0 2021-11-03T17:14:04.186 INFO:tasks.workunit.client.1.smithi115.stdout:7/134: mkdir d3/da/d26 0 2021-11-03T17:14:04.190 INFO:tasks.workunit.client.1.smithi115.stdout:6/152: creat d0/d1/d2e/f31 x:0 0 0 2021-11-03T17:14:04.191 INFO:tasks.workunit.client.1.smithi115.stdout:6/153: read d0/d1/d2/fc [253386,25363] 0 2021-11-03T17:14:04.191 INFO:tasks.workunit.client.1.smithi115.stdout:6/154: dread - d0/f2a zero size 2021-11-03T17:14:04.192 INFO:tasks.workunit.client.1.smithi115.stdout:3/167: unlink d13/f1b 0 2021-11-03T17:14:04.192 INFO:tasks.workunit.client.1.smithi115.stdout:4/205: truncate d7/dc/d1a/f49 521889 0 2021-11-03T17:14:04.193 INFO:tasks.workunit.client.1.smithi115.stdout:3/168: read - d13/f22 zero size 2021-11-03T17:14:04.194 INFO:tasks.workunit.client.1.smithi115.stdout:3/169: creat d13/d20/d2e/f37 x:0 0 0 2021-11-03T17:14:04.195 INFO:tasks.workunit.client.1.smithi115.stdout:3/170: fsync ff 0 2021-11-03T17:14:04.195 INFO:tasks.workunit.client.1.smithi115.stdout:3/171: write d13/f2b [569804,104297] 0 2021-11-03T17:14:04.195 INFO:tasks.workunit.client.1.smithi115.stdout:6/155: write d0/d1/d2/ff [259669,104360] 0 2021-11-03T17:14:04.198 INFO:tasks.workunit.client.1.smithi115.stdout:3/172: write d13/f1c [284050,130213] 0 2021-11-03T17:14:04.198 INFO:tasks.workunit.client.1.smithi115.stdout:3/173: truncate d13/f1e 1151386 0 2021-11-03T17:14:04.201 INFO:tasks.workunit.client.1.smithi115.stdout:0/174: dwrite d9/d15/d1a/f3d [0,4194304] 0 2021-11-03T17:14:04.201 INFO:tasks.workunit.client.1.smithi115.stdout:0/175: truncate d9/d15/f33 424760 0 2021-11-03T17:14:04.202 INFO:tasks.workunit.client.1.smithi115.stdout:7/135: dread d3/f13 [0,4194304] 0 2021-11-03T17:14:04.202 INFO:tasks.workunit.client.1.smithi115.stdout:7/136: fsync d3/f1e 0 2021-11-03T17:14:04.210 INFO:tasks.workunit.client.1.smithi115.stdout:0/176: rmdir d9/d15/d1f 39 2021-11-03T17:14:04.210 INFO:tasks.workunit.client.1.smithi115.stdout:0/177: fdatasync d9/d15/f33 0 2021-11-03T17:14:04.210 INFO:tasks.workunit.client.1.smithi115.stdout:7/137: stat d3/da/l24 0 2021-11-03T17:14:04.211 INFO:tasks.workunit.client.1.smithi115.stdout:7/138: chown d3/l6 535796 1 2021-11-03T17:14:04.211 INFO:tasks.workunit.client.1.smithi115.stdout:7/139: write d3/f9 [12851498,102361] 0 2021-11-03T17:14:04.211 INFO:tasks.workunit.client.1.smithi115.stdout:7/140: truncate d3/da/f1b 5307276 0 2021-11-03T17:14:04.211 INFO:tasks.workunit.client.1.smithi115.stdout:0/178: creat d9/db/f42 x:0 0 0 2021-11-03T17:14:04.211 INFO:tasks.workunit.client.1.smithi115.stdout:7/141: rmdir d3/da/d18 39 2021-11-03T17:14:04.212 INFO:tasks.workunit.client.1.smithi115.stdout:3/174: dread d13/f19 [0,4194304] 0 2021-11-03T17:14:04.212 INFO:tasks.workunit.client.1.smithi115.stdout:3/175: stat d13/d20/d2e/f32 0 2021-11-03T17:14:04.213 INFO:tasks.workunit.client.1.smithi115.stdout:2/130: dwrite d4/f10 [0,4194304] 0 2021-11-03T17:14:04.213 INFO:tasks.workunit.client.1.smithi115.stdout:2/131: chown d4/d6/d13/d1e/f17 11719814 1 2021-11-03T17:14:04.213 INFO:tasks.workunit.client.1.smithi115.stdout:0/179: rename f7 to d9/f43 0 2021-11-03T17:14:04.214 INFO:tasks.workunit.client.1.smithi115.stdout:0/180: fsync d9/d15/d1a/d1d/f22 0 2021-11-03T17:14:04.214 INFO:tasks.workunit.client.1.smithi115.stdout:0/181: fsync d9/f23 0 2021-11-03T17:14:04.214 INFO:tasks.workunit.client.1.smithi115.stdout:3/176: dread - d13/f21 zero size 2021-11-03T17:14:04.214 INFO:tasks.workunit.client.1.smithi115.stdout:4/206: dwrite d7/dc/d1a/d21/d32/f3a [4194304,4194304] 0 2021-11-03T17:14:04.215 INFO:tasks.workunit.client.1.smithi115.stdout:4/207: chown d7 130477545 1 2021-11-03T17:14:04.220 INFO:tasks.workunit.client.1.smithi115.stdout:0/182: creat d9/db/df/f44 x:0 0 0 2021-11-03T17:14:04.221 INFO:tasks.workunit.client.1.smithi115.stdout:0/183: readlink d9/db/ld 0 2021-11-03T17:14:04.225 INFO:tasks.workunit.client.1.smithi115.stdout:7/142: rename d3/da/d18/f23 to d3/f27 0 2021-11-03T17:14:04.225 INFO:tasks.workunit.client.1.smithi115.stdout:3/177: dwrite d13/f21 [0,4194304] 0 2021-11-03T17:14:04.229 INFO:tasks.workunit.client.1.smithi115.stdout:4/208: mkdir d7/dc/d1a/d4e 0 2021-11-03T17:14:04.235 INFO:tasks.workunit.client.1.smithi115.stdout:3/178: dread d13/d20/d2e/f2f [0,4194304] 0 2021-11-03T17:14:04.239 INFO:tasks.workunit.client.1.smithi115.stdout:2/132: rename d4/f10 to d4/d6/f2a 0 2021-11-03T17:14:04.243 INFO:tasks.workunit.client.1.smithi115.stdout:7/143: write d3/f14 [1395606,77930] 0 2021-11-03T17:14:04.245 INFO:tasks.workunit.client.1.smithi115.stdout:4/209: creat d7/dc/d1a/f4f x:0 0 0 2021-11-03T17:14:04.246 INFO:tasks.workunit.client.1.smithi115.stdout:4/210: truncate d7/dc/d1a/f1c 327571 0 2021-11-03T17:14:04.246 INFO:tasks.workunit.client.1.smithi115.stdout:4/211: mknod d7/dc/d36/c50 1 2021-11-03T17:14:04.246 INFO:tasks.workunit.client.1.smithi115.stdout:3/179: readlink d13/l18 0 2021-11-03T17:14:04.247 INFO:tasks.workunit.client.1.smithi115.stdout:2/133: unlink d4/fe 0 2021-11-03T17:14:04.247 INFO:tasks.workunit.client.1.smithi115.stdout:7/144: getdents d3/da/d18 0 2021-11-03T17:14:04.249 INFO:tasks.workunit.client.1.smithi115.stdout:3/180: symlink d13/d20/d2e/l38 0 2021-11-03T17:14:04.250 INFO:tasks.workunit.client.1.smithi115.stdout:7/145: stat d3/da/l1d 0 2021-11-03T17:14:04.251 INFO:tasks.workunit.client.1.smithi115.stdout:7/146: read d3/f1e [346282,72615] 0 2021-11-03T17:14:04.251 INFO:tasks.workunit.client.1.smithi115.stdout:3/181: creat d13/f39 x:0 0 0 2021-11-03T17:14:04.251 INFO:tasks.workunit.client.1.smithi115.stdout:3/182: readlink d13/d20/d2e/l38 0 2021-11-03T17:14:04.251 INFO:tasks.workunit.client.1.smithi115.stdout:3/183: truncate f12 3121326 0 2021-11-03T17:14:04.252 INFO:tasks.workunit.client.1.smithi115.stdout:4/212: rmdir d7/d2b 0 2021-11-03T17:14:04.252 INFO:tasks.workunit.client.1.smithi115.stdout:4/213: write d7/dc/f12 [1378795,676] 0 2021-11-03T17:14:04.253 INFO:tasks.workunit.client.1.smithi115.stdout:7/147: rename d3/l6 to d3/da/d18/l28 0 2021-11-03T17:14:04.253 INFO:tasks.workunit.client.1.smithi115.stdout:7/148: fdatasync d3/da/f1b 0 2021-11-03T17:14:04.254 INFO:tasks.workunit.client.1.smithi115.stdout:3/184: creat d13/d35/f3a x:0 0 0 2021-11-03T17:14:04.254 INFO:tasks.workunit.client.1.smithi115.stdout:4/214: rename d7/dc/d1a/f4f to d7/dc/d1a/d21/d47/f51 0 2021-11-03T17:14:04.255 INFO:tasks.workunit.client.1.smithi115.stdout:4/215: mknod d7/dc/d1a/d21/d37/c52 1 2021-11-03T17:14:04.255 INFO:tasks.workunit.client.1.smithi115.stdout:4/216: write d7/dc/d1a/d21/d47/f51 [870021,127703] 0 2021-11-03T17:14:04.255 INFO:tasks.workunit.client.1.smithi115.stdout:7/149: link d3/da/l24 d3/da/d26/l29 0 2021-11-03T17:14:04.257 INFO:tasks.workunit.client.1.smithi115.stdout:4/217: rmdir d7/dc/d1a/d21/d37/d44 0 2021-11-03T17:14:04.258 INFO:tasks.workunit.client.1.smithi115.stdout:3/185: mkdir d13/d3b 0 2021-11-03T17:14:04.260 INFO:tasks.workunit.client.1.smithi115.stdout:4/218: symlink d7/dc/d1a/d21/d47/l53 0 2021-11-03T17:14:04.260 INFO:tasks.workunit.client.1.smithi115.stdout:4/219: mknod d7/dc/d1a/d21/d47/c54 1 2021-11-03T17:14:04.270 INFO:tasks.workunit.client.1.smithi115.stdout:7/150: dwrite d3/f1e [0,4194304] 0 2021-11-03T17:14:04.270 INFO:tasks.workunit.client.1.smithi115.stdout:7/151: stat d3/da/f15 0 2021-11-03T17:14:04.270 INFO:tasks.workunit.client.1.smithi115.stdout:7/152: chown d3/f13 5101 1 2021-11-03T17:14:04.272 INFO:tasks.workunit.client.1.smithi115.stdout:7/153: rename d3/l20 to d3/da/d18/l2a 0 2021-11-03T17:14:04.287 INFO:tasks.workunit.client.1.smithi115.stdout:3/186: dwrite d13/d20/d2e/f37 [0,4194304] 0 2021-11-03T17:14:04.298 INFO:tasks.workunit.client.1.smithi115.stdout:3/187: mkdir d13/d3b/d3c 0 2021-11-03T17:14:04.333 INFO:tasks.workunit.client.1.smithi115.stdout:0/184: dwrite d9/f25 [0,4194304] 0 2021-11-03T17:14:04.333 INFO:tasks.workunit.client.1.smithi115.stdout:0/185: fsync d9/f25 0 2021-11-03T17:14:04.335 INFO:tasks.workunit.client.1.smithi115.stdout:0/186: symlink d9/d15/l45 0 2021-11-03T17:14:04.336 INFO:tasks.workunit.client.1.smithi115.stdout:2/134: dwrite d4/d6/d13/d1e/f14 [0,4194304] 0 2021-11-03T17:14:04.337 INFO:tasks.workunit.client.1.smithi115.stdout:2/135: chown d4/fa 5 1 2021-11-03T17:14:04.337 INFO:tasks.workunit.client.1.smithi115.stdout:2/136: write d4/d6/f12 [1274954,116345] 0 2021-11-03T17:14:04.362 INFO:tasks.workunit.client.1.smithi115.stdout:4/220: dwrite d7/dc/d1a/d21/d47/f51 [0,4194304] 0 2021-11-03T17:14:04.362 INFO:tasks.workunit.client.1.smithi115.stdout:4/221: chown d7/dc/d1a/d21/d32 15607313 1 2021-11-03T17:14:04.362 INFO:tasks.workunit.client.1.smithi115.stdout:0/187: dwrite d9/f35 [0,4194304] 0 2021-11-03T17:14:04.487 INFO:tasks.workunit.client.1.smithi115.stdout:2/137: dwrite d4/fa [0,4194304] 0 2021-11-03T17:14:04.487 INFO:tasks.workunit.client.1.smithi115.stdout:4/222: rmdir d7/dc/d1a/d4e 0 2021-11-03T17:14:04.487 INFO:tasks.workunit.client.1.smithi115.stdout:4/223: fdatasync d7/dc/d1a/f49 0 2021-11-03T17:14:04.488 INFO:tasks.workunit.client.1.smithi115.stdout:0/188: mkdir d9/d15/d2c/d46 0 2021-11-03T17:14:04.488 INFO:tasks.workunit.client.1.smithi115.stdout:0/189: chown d9/db/f20 3 1 2021-11-03T17:14:04.488 INFO:tasks.workunit.client.1.smithi115.stdout:2/138: creat d4/d26/f2b x:0 0 0 2021-11-03T17:14:04.489 INFO:tasks.workunit.client.1.smithi115.stdout:4/224: symlink d7/dc/d1a/d21/l55 0 2021-11-03T17:14:04.489 INFO:tasks.workunit.client.1.smithi115.stdout:4/225: getdents d7/dc/d1a/d21/d32 0 2021-11-03T17:14:04.489 INFO:tasks.workunit.client.1.smithi115.stdout:4/226: rename d7/dc/d36 to d7/dc/d1a/d21/d47/d56 0 2021-11-03T17:14:04.489 INFO:tasks.workunit.client.1.smithi115.stdout:0/190: dread d9/f25 [0,4194304] 0 2021-11-03T17:14:04.489 INFO:tasks.workunit.client.1.smithi115.stdout:0/191: link d9/d15/f33 d9/db/f47 0 2021-11-03T17:14:04.490 INFO:tasks.workunit.client.1.smithi115.stdout:0/192: creat d9/db/df/f48 x:0 0 0 2021-11-03T17:14:04.490 INFO:tasks.workunit.client.1.smithi115.stdout:0/193: fsync d9/f35 0 2021-11-03T17:14:04.490 INFO:tasks.workunit.client.1.smithi115.stdout:0/194: dread - d9/db/df/f48 zero size 2021-11-03T17:14:04.490 INFO:tasks.workunit.client.1.smithi115.stdout:0/195: chown d9/db/f42 108062410 1 2021-11-03T17:14:04.491 INFO:tasks.workunit.client.1.smithi115.stdout:2/139: dwrite d4/d6/d13/d1e/f1b [0,4194304] 0 2021-11-03T17:14:04.491 INFO:tasks.workunit.client.1.smithi115.stdout:4/227: dwrite d7/dc/d1a/d21/d32/d3c/f3e [0,4194304] 0 2021-11-03T17:14:04.491 INFO:tasks.workunit.client.1.smithi115.stdout:4/228: chown d7/dc/d1a/d21/d37 0 1 2021-11-03T17:14:04.491 INFO:tasks.workunit.client.1.smithi115.stdout:4/229: mknod d7/dc/d1a/d21/c57 1 2021-11-03T17:14:04.492 INFO:tasks.workunit.client.1.smithi115.stdout:4/230: fsync d7/dc/f48 0 2021-11-03T17:14:04.492 INFO:tasks.workunit.client.1.smithi115.stdout:4/231: write d7/dc/d1a/d21/d47/f51 [2490432,112625] 0 2021-11-03T17:14:04.492 INFO:tasks.workunit.client.1.smithi115.stdout:4/232: rename d7/f30 to d7/dc/d1a/d21/d47/d56/d42/f58 0 2021-11-03T17:14:04.492 INFO:tasks.workunit.client.1.smithi115.stdout:4/233: symlink d7/dc/d1a/d21/d37/l59 0 2021-11-03T17:14:04.493 INFO:tasks.workunit.client.1.smithi115.stdout:4/234: mknod d7/dc/d1a/d21/d37/c5a 1 2021-11-03T17:14:04.493 INFO:tasks.workunit.client.1.smithi115.stdout:4/235: write d7/dc/d1a/d21/f22 [3156155,6171] 0 2021-11-03T17:14:04.493 INFO:tasks.workunit.client.1.smithi115.stdout:4/236: mkdir d7/dc/d1a/d21/d32/d5b 0 2021-11-03T17:14:04.494 INFO:tasks.workunit.client.1.smithi115.stdout:2/140: dwrite d4/d6/f12 [0,4194304] 0 2021-11-03T17:14:04.494 INFO:tasks.workunit.client.1.smithi115.stdout:2/141: readlink d4/d6/d13/d1e/lf 0 2021-11-03T17:14:04.494 INFO:tasks.workunit.client.1.smithi115.stdout:2/142: unlink d4/d6/d13/d1e/f1b 0 2021-11-03T17:14:04.494 INFO:tasks.workunit.client.1.smithi115.stdout:2/143: symlink d4/l2c 0 2021-11-03T17:14:04.495 INFO:tasks.workunit.client.1.smithi115.stdout:2/144: creat d4/d6/d13/d1e/f2d x:0 0 0 2021-11-03T17:14:04.495 INFO:tasks.workunit.client.1.smithi115.stdout:2/145: write d4/d6/f12 [144237,47649] 0 2021-11-03T17:14:04.495 INFO:tasks.workunit.client.1.smithi115.stdout:4/237: dwrite d7/dc/d1a/d21/d32/f3a [4194304,4194304] 0 2021-11-03T17:14:04.495 INFO:tasks.workunit.client.1.smithi115.stdout:4/238: mknod d7/dc/c5c 1 2021-11-03T17:14:04.495 INFO:tasks.workunit.client.1.smithi115.stdout:2/146: rename d4/d6/d13/d1e to d4/d6/d2e 0 2021-11-03T17:14:04.496 INFO:tasks.workunit.client.1.smithi115.stdout:2/147: chown d4/d6/d2e 2893 1 2021-11-03T17:14:04.496 INFO:tasks.workunit.client.1.smithi115.stdout:4/239: dwrite d7/dc/d1a/d21/d32/d3c/f3e [0,4194304] 0 2021-11-03T17:14:04.496 INFO:tasks.workunit.client.1.smithi115.stdout:2/148: dread d4/d6/d2e/f14 [0,4194304] 0 2021-11-03T17:14:04.497 INFO:tasks.workunit.client.1.smithi115.stdout:2/149: fsync d4/d26/f2b 0 2021-11-03T17:14:04.497 INFO:tasks.workunit.client.1.smithi115.stdout:4/240: rename d7/dc/d1a/d21/d32/d5b to d7/dc/d1a/d21/d37/d5d 0 2021-11-03T17:14:04.497 INFO:tasks.workunit.client.1.smithi115.stdout:2/150: creat d4/d6/d13/f2f x:0 0 0 2021-11-03T17:14:04.497 INFO:tasks.workunit.client.1.smithi115.stdout:4/241: symlink d7/dc/d1a/d21/l5e 0 2021-11-03T17:14:04.498 INFO:tasks.workunit.client.1.smithi115.stdout:2/151: symlink d4/d26/d28/l30 0 2021-11-03T17:14:04.498 INFO:tasks.workunit.client.1.smithi115.stdout:4/242: mkdir d7/dc/d1a/d21/d47/d56/d42/d5f 0 2021-11-03T17:14:04.498 INFO:tasks.workunit.client.1.smithi115.stdout:4/243: creat d7/dc/f60 x:0 0 0 2021-11-03T17:14:04.498 INFO:tasks.workunit.client.1.smithi115.stdout:4/244: rmdir d7/dc/d1a/d21/d3f 0 2021-11-03T17:14:04.498 INFO:tasks.workunit.client.1.smithi115.stdout:4/245: rmdir d7/dc/d1a/d21 39 2021-11-03T17:14:04.499 INFO:tasks.workunit.client.1.smithi115.stdout:4/246: chown d7/dc/d1a/d21/d32/d3c 3491973 1 2021-11-03T17:14:04.499 INFO:tasks.workunit.client.1.smithi115.stdout:4/247: read d7/dc/d1a/d21/f25 [25765,34167] 0 2021-11-03T17:14:04.499 INFO:tasks.workunit.client.1.smithi115.stdout:2/152: dwrite d4/d6/d2e/f14 [0,4194304] 0 2021-11-03T17:14:04.499 INFO:tasks.workunit.client.1.smithi115.stdout:2/153: chown d4/d6/d2e 477 1 2021-11-03T17:14:04.500 INFO:tasks.workunit.client.1.smithi115.stdout:4/248: dwrite d7/dc/f60 [0,4194304] 0 2021-11-03T17:14:04.500 INFO:tasks.workunit.client.1.smithi115.stdout:4/249: truncate d7/dc/d1a/d21/d47/d56/d42/f58 973368 0 2021-11-03T17:14:04.500 INFO:tasks.workunit.client.1.smithi115.stdout:4/250: readlink l4 0 2021-11-03T17:14:04.500 INFO:tasks.workunit.client.1.smithi115.stdout:4/251: write d7/dc/d1a/f2e [2776712,53399] 0 2021-11-03T17:14:04.500 INFO:tasks.workunit.client.1.smithi115.stdout:4/252: symlink d7/dc/d1a/d21/d37/d5d/l61 0 2021-11-03T17:14:04.501 INFO:tasks.workunit.client.1.smithi115.stdout:2/154: dwrite d4/d6/d13/f25 [0,4194304] 0 2021-11-03T17:14:04.501 INFO:tasks.workunit.client.1.smithi115.stdout:2/155: dread d4/fa [0,4194304] 0 2021-11-03T17:14:04.501 INFO:tasks.workunit.client.1.smithi115.stdout:2/156: symlink d4/d26/d28/l31 0 2021-11-03T17:14:04.501 INFO:tasks.workunit.client.1.smithi115.stdout:2/157: link d4/d26/d28/l31 d4/d6/d13/l32 0 2021-11-03T17:14:04.502 INFO:tasks.workunit.client.1.smithi115.stdout:4/253: dwrite d7/dc/d1a/d21/d47/f51 [0,4194304] 0 2021-11-03T17:14:04.502 INFO:tasks.workunit.client.1.smithi115.stdout:4/254: stat d7/dc 0 2021-11-03T17:14:04.502 INFO:tasks.workunit.client.1.smithi115.stdout:4/255: symlink d7/dc/d1a/d21/l62 0 2021-11-03T17:14:04.503 INFO:tasks.workunit.client.1.smithi115.stdout:4/256: rename d7/dc/d1a/d21/d37/d5d to d7/dc/d1a/d21/d47/d56/d63 0 2021-11-03T17:14:04.503 INFO:tasks.workunit.client.1.smithi115.stdout:4/257: creat d7/dc/d1a/d21/d47/d56/d42/d5f/f64 x:0 0 0 2021-11-03T17:14:04.503 INFO:tasks.workunit.client.1.smithi115.stdout:4/258: symlink d7/dc/d1a/d21/d37/l65 0 2021-11-03T17:14:04.509 INFO:tasks.workunit.client.1.smithi115.stdout:2/158: dwrite d4/d6/d2e/f14 [4194304,4194304] 0 2021-11-03T17:14:04.509 INFO:tasks.workunit.client.1.smithi115.stdout:2/159: write d4/d6/f2a [5164352,18108] 0 2021-11-03T17:14:04.510 INFO:tasks.workunit.client.1.smithi115.stdout:2/160: symlink d4/d26/d28/l33 2 2021-11-03T17:14:04.510 INFO:tasks.workunit.client.1.smithi115.stdout:4/259: dwrite d7/dc/d1a/d21/d32/f3a [0,4194304] 0 2021-11-03T17:14:04.512 INFO:tasks.workunit.client.1.smithi115.stdout:2/161: unlink d4/d6/d2e/f2d 0 2021-11-03T17:14:04.518 INFO:tasks.workunit.client.1.smithi115.stdout:4/260: rmdir d7/dc/d1a/d21/d47/d56/d42/d5f 39 2021-11-03T17:14:04.520 INFO:tasks.workunit.client.1.smithi115.stdout:2/162: dread d4/d6/f12 [0,4194304] 0 2021-11-03T17:14:04.520 INFO:tasks.workunit.client.1.smithi115.stdout:2/163: dread - d4/d6/d13/f21 zero size 2021-11-03T17:14:04.520 INFO:tasks.workunit.client.1.smithi115.stdout:4/261: dread d7/dc/d1a/f49 [0,4194304] 0 2021-11-03T17:14:04.521 INFO:tasks.workunit.client.1.smithi115.stdout:2/164: getdents d4/d6/d13 0 2021-11-03T17:14:04.522 INFO:tasks.workunit.client.1.smithi115.stdout:4/262: creat d7/dc/d1a/d21/d32/d3c/f66 x:0 0 0 2021-11-03T17:14:04.523 INFO:tasks.workunit.client.1.smithi115.stdout:2/165: mkdir d4/d6/d2e/d34 0 2021-11-03T17:14:04.523 INFO:tasks.workunit.client.1.smithi115.stdout:2/166: fsync d4/fa 0 2021-11-03T17:14:04.524 INFO:tasks.workunit.client.1.smithi115.stdout:2/167: fdatasync d4/d6/d13/f25 0 2021-11-03T17:14:04.526 INFO:tasks.workunit.client.1.smithi115.stdout:4/263: link d7/dc/f11 d7/dc/d1a/d21/d47/d56/d63/f67 0 2021-11-03T17:14:04.527 INFO:tasks.workunit.client.1.smithi115.stdout:2/168: rmdir d4/d26 39 2021-11-03T17:14:04.527 INFO:tasks.workunit.client.1.smithi115.stdout:2/169: chown d4/d6/l16 207 1 2021-11-03T17:14:04.528 INFO:tasks.workunit.client.0.smithi027.stdout:1/186: sync 2021-11-03T17:14:04.528 INFO:tasks.workunit.client.0.smithi027.stdout:3/100: sync 2021-11-03T17:14:04.528 INFO:tasks.workunit.client.0.smithi027.stdout:9/139: sync 2021-11-03T17:14:04.529 INFO:tasks.workunit.client.0.smithi027.stdout:3/101: readlink d1/db/le 0 2021-11-03T17:14:04.529 INFO:tasks.workunit.client.0.smithi027.stdout:9/140: dread - db/d1b/d1c/d1d/f21 zero size 2021-11-03T17:14:04.529 INFO:tasks.workunit.client.0.smithi027.stdout:9/141: dread - db/d1b/d1c/d1d/f1e zero size 2021-11-03T17:14:04.529 INFO:tasks.workunit.client.0.smithi027.stdout:9/142: mknod db/d1b/d23/d2c/c36 1 2021-11-03T17:14:04.530 INFO:tasks.workunit.client.0.smithi027.stdout:9/143: chown db/d1b/d2b 7698 1 2021-11-03T17:14:04.530 INFO:tasks.workunit.client.0.smithi027.stdout:9/144: dread - db/f1f zero size 2021-11-03T17:14:04.530 INFO:tasks.workunit.client.0.smithi027.stdout:0/151: sync 2021-11-03T17:14:04.530 INFO:tasks.workunit.client.0.smithi027.stdout:0/152: mknod d9/c32 1 2021-11-03T17:14:04.531 INFO:tasks.workunit.client.0.smithi027.stdout:0/153: getdents d9/da/d31 0 2021-11-03T17:14:04.531 INFO:tasks.workunit.client.0.smithi027.stdout:3/102: unlink d1/db/le 0 2021-11-03T17:14:04.532 INFO:tasks.workunit.client.1.smithi115.stdout:4/264: rename d7/dc/f48 to d7/dc/d1a/d21/d47/f68 0 2021-11-03T17:14:04.532 INFO:tasks.workunit.client.1.smithi115.stdout:4/265: creat d7/dc/d1a/d21/d47/f69 x:0 0 0 2021-11-03T17:14:04.533 INFO:tasks.workunit.client.0.smithi027.stdout:9/145: creat db/d1b/d1c/d1d/f37 x:0 0 0 2021-11-03T17:14:04.534 INFO:tasks.workunit.client.1.smithi115.stdout:4/266: rename d7/dc/d1a/d21/d32 to d7/dc/d6a 0 2021-11-03T17:14:04.535 INFO:tasks.workunit.client.1.smithi115.stdout:4/267: fdatasync d7/dc/d1a/f1c 0 2021-11-03T17:14:04.535 INFO:tasks.workunit.client.0.smithi027.stdout:0/154: mkdir d9/da/d30/d33 0 2021-11-03T17:14:04.537 INFO:tasks.workunit.client.0.smithi027.stdout:9/146: rename db/f35 to db/d1b/f38 0 2021-11-03T17:14:04.539 INFO:tasks.workunit.client.0.smithi027.stdout:0/155: creat d9/da/d31/f34 x:0 0 0 2021-11-03T17:14:04.541 INFO:tasks.workunit.client.0.smithi027.stdout:9/147: creat db/d1b/d1c/f39 x:0 0 0 2021-11-03T17:14:04.542 INFO:tasks.workunit.client.0.smithi027.stdout:0/156: creat d9/f35 x:0 0 0 2021-11-03T17:14:04.545 INFO:tasks.workunit.client.0.smithi027.stdout:0/157: rename d9/f35 to d9/da/d31/f36 0 2021-11-03T17:14:04.545 INFO:tasks.workunit.client.0.smithi027.stdout:0/158: read d9/f1f [329335,103760] 0 2021-11-03T17:14:04.547 INFO:tasks.workunit.client.0.smithi027.stdout:0/159: creat d9/da/f37 x:0 0 0 2021-11-03T17:14:04.549 INFO:tasks.workunit.client.0.smithi027.stdout:1/187: dwrite d6/de/f1c [4194304,4194304] 0 2021-11-03T17:14:04.550 INFO:tasks.workunit.client.0.smithi027.stdout:3/103: dwrite d1/db/f1e [0,4194304] 0 2021-11-03T17:14:04.550 INFO:tasks.workunit.client.0.smithi027.stdout:3/104: chown d1/fa 595190802 1 2021-11-03T17:14:04.550 INFO:tasks.workunit.client.0.smithi027.stdout:3/105: fdatasync d1/f9 0 2021-11-03T17:14:04.551 INFO:tasks.workunit.client.0.smithi027.stdout:3/106: write d1/f9 [2427181,23489] 0 2021-11-03T17:14:04.551 INFO:tasks.workunit.client.0.smithi027.stdout:1/188: symlink d6/l3e 0 2021-11-03T17:14:04.556 INFO:tasks.workunit.client.0.smithi027.stdout:1/189: mkdir d6/d11/d14/d1f/d25/d28/d32/d3f 0 2021-11-03T17:14:04.556 INFO:tasks.workunit.client.0.smithi027.stdout:1/190: mknod d6/d11/d14/d1f/d25/d28/d32/d3f/c40 1 2021-11-03T17:14:04.557 INFO:tasks.workunit.client.0.smithi027.stdout:1/191: fsync d6/fa 0 2021-11-03T17:14:04.558 INFO:tasks.workunit.client.0.smithi027.stdout:1/192: mkdir d6/d11/d14/d1f/d25/d28/d41 0 2021-11-03T17:14:04.558 INFO:tasks.workunit.client.0.smithi027.stdout:9/148: dwrite db/d1b/f38 [0,4194304] 0 2021-11-03T17:14:04.567 INFO:tasks.workunit.client.0.smithi027.stdout:9/149: symlink db/d1b/d1c/l3a 0 2021-11-03T17:14:04.567 INFO:tasks.workunit.client.0.smithi027.stdout:9/150: write db/f14 [1507945,70781] 0 2021-11-03T17:14:04.567 INFO:tasks.workunit.client.0.smithi027.stdout:9/151: write db/d1b/d2b/f2a [4616853,79231] 0 2021-11-03T17:14:04.567 INFO:tasks.workunit.client.0.smithi027.stdout:9/152: mknod db/d1b/d23/c3b 1 2021-11-03T17:14:04.568 INFO:tasks.workunit.client.0.smithi027.stdout:0/160: dwrite d9/d12/f1e [4194304,4194304] 0 2021-11-03T17:14:04.572 INFO:tasks.workunit.client.0.smithi027.stdout:1/193: dwrite d6/de/d13/f38 [0,4194304] 0 2021-11-03T17:14:04.572 INFO:tasks.workunit.client.0.smithi027.stdout:0/161: dread d9/f2a [0,4194304] 0 2021-11-03T17:14:04.577 INFO:tasks.workunit.client.0.smithi027.stdout:1/194: getdents d6/d11/d14/d1f 0 2021-11-03T17:14:04.582 INFO:tasks.workunit.client.0.smithi027.stdout:9/153: truncate db/d1b/d2b/f2a 3063227 0 2021-11-03T17:14:04.582 INFO:tasks.workunit.client.0.smithi027.stdout:9/154: dread db/d1b/f20 [0,4194304] 0 2021-11-03T17:14:04.585 INFO:tasks.workunit.client.0.smithi027.stdout:9/155: creat db/d1b/d23/f3c x:0 0 0 2021-11-03T17:14:04.585 INFO:tasks.workunit.client.0.smithi027.stdout:1/195: dread d6/de/d13/f38 [0,4194304] 0 2021-11-03T17:14:04.587 INFO:tasks.workunit.client.0.smithi027.stdout:9/156: rmdir db/d1b/d1c/d1d/d2f 0 2021-11-03T17:14:04.588 INFO:tasks.workunit.client.1.smithi115.stdout:4/268: symlink d7/l6b 0 2021-11-03T17:14:04.589 INFO:tasks.workunit.client.1.smithi115.stdout:4/269: mknod d7/dc/d6a/d3c/c6c 1 2021-11-03T17:14:04.589 INFO:tasks.workunit.client.1.smithi115.stdout:4/270: write d7/dc/d1a/d21/d47/f51 [3993707,15119] 0 2021-11-03T17:14:04.590 INFO:tasks.workunit.client.0.smithi027.stdout:1/196: creat d6/de/d13/d15/f42 x:0 0 0 2021-11-03T17:14:04.591 INFO:tasks.workunit.client.0.smithi027.stdout:1/197: dread d6/de/d13/d15/f2f [0,4194304] 0 2021-11-03T17:14:04.591 INFO:tasks.workunit.client.0.smithi027.stdout:1/198: chown d6/l3b 27931 1 2021-11-03T17:14:04.591 INFO:tasks.workunit.client.0.smithi027.stdout:1/199: dread d6/de/d13/d15/f16 [0,4194304] 0 2021-11-03T17:14:04.592 INFO:tasks.workunit.client.1.smithi115.stdout:4/271: write d7/dc/d1a/f1c [147833,100937] 0 2021-11-03T17:14:04.593 INFO:tasks.workunit.client.1.smithi115.stdout:4/272: truncate d7/dc/d6a/d3c/f3e 4859104 0 2021-11-03T17:14:04.593 INFO:tasks.workunit.client.1.smithi115.stdout:4/273: chown d7/dc/d1a/d21/d47/d56/l4c 394 1 2021-11-03T17:14:04.593 INFO:tasks.workunit.client.1.smithi115.stdout:4/274: symlink d7/dc/d6a/l6d 0 2021-11-03T17:14:04.594 INFO:tasks.workunit.client.1.smithi115.stdout:4/275: creat d7/dc/d1a/d21/d47/d56/d63/f6e x:0 0 0 2021-11-03T17:14:04.595 INFO:tasks.workunit.client.1.smithi115.stdout:4/276: unlink d7/dc/d1a/d21/d47/f68 0 2021-11-03T17:14:04.596 INFO:tasks.workunit.client.1.smithi115.stdout:4/277: link d7/dc/d1a/f2e d7/dc/d6a/f6f 0 2021-11-03T17:14:04.597 INFO:tasks.workunit.client.1.smithi115.stdout:4/278: rename d7 to d7/dc/d6a/d70 22 2021-11-03T17:14:04.601 INFO:tasks.workunit.client.0.smithi027.stdout:9/157: dread db/d1b/d2b/f2a [0,4194304] 0 2021-11-03T17:14:04.602 INFO:tasks.workunit.client.0.smithi027.stdout:9/158: creat db/d1b/d1c/d1d/f3d x:0 0 0 2021-11-03T17:14:04.605 INFO:tasks.workunit.client.0.smithi027.stdout:9/159: link db/f17 db/d1b/d1c/f3e 0 2021-11-03T17:14:04.605 INFO:tasks.workunit.client.0.smithi027.stdout:9/160: dread - db/d1b/d1c/d1d/f3d zero size 2021-11-03T17:14:04.607 INFO:tasks.workunit.client.0.smithi027.stdout:9/161: creat db/d1b/d2b/f3f x:0 0 0 2021-11-03T17:14:04.609 INFO:tasks.workunit.client.0.smithi027.stdout:9/162: getdents db 0 2021-11-03T17:14:04.611 INFO:tasks.workunit.client.0.smithi027.stdout:9/163: mkdir db/d1b/d1c/d1d/d40 0 2021-11-03T17:14:04.612 INFO:tasks.workunit.client.0.smithi027.stdout:9/164: write db/d1b/d1c/f25 [260664,127184] 0 2021-11-03T17:14:04.613 INFO:tasks.workunit.client.0.smithi027.stdout:9/165: mkdir db/d1b/d2b/d41 0 2021-11-03T17:14:04.615 INFO:tasks.workunit.client.0.smithi027.stdout:9/166: rename db/l2e to db/d1b/d2b/d41/l42 0 2021-11-03T17:14:04.617 INFO:tasks.workunit.client.0.smithi027.stdout:9/167: creat db/d1b/d23/d2c/f43 x:0 0 0 2021-11-03T17:14:04.622 INFO:tasks.workunit.client.1.smithi115.stdout:4/279: dwrite d7/dc/d1a/f1c [0,4194304] 0 2021-11-03T17:14:04.623 INFO:tasks.workunit.client.1.smithi115.stdout:4/280: rmdir d7/dc/d1a/d21 39 2021-11-03T17:14:04.625 INFO:tasks.workunit.client.0.smithi027.stdout:9/168: dread db/d1b/d1c/f25 [0,4194304] 0 2021-11-03T17:14:04.625 INFO:tasks.workunit.client.0.smithi027.stdout:9/169: chown db/d1b/f20 862601603 1 2021-11-03T17:14:04.625 INFO:tasks.workunit.client.0.smithi027.stdout:9/170: write db/d1b/d1c/d1d/f1e [380380,121141] 0 2021-11-03T17:14:04.644 INFO:tasks.workunit.client.0.smithi027.stdout:4/173: sync 2021-11-03T17:14:04.644 INFO:tasks.workunit.client.0.smithi027.stdout:2/200: sync 2021-11-03T17:14:04.645 INFO:tasks.workunit.client.0.smithi027.stdout:7/169: sync 2021-11-03T17:14:04.645 INFO:tasks.workunit.client.0.smithi027.stdout:8/191: sync 2021-11-03T17:14:04.645 INFO:tasks.workunit.client.0.smithi027.stdout:6/106: sync 2021-11-03T17:14:04.646 INFO:tasks.workunit.client.0.smithi027.stdout:5/156: sync 2021-11-03T17:14:04.646 INFO:tasks.workunit.client.0.smithi027.stdout:2/201: readlink d0/l31 0 2021-11-03T17:14:04.646 INFO:tasks.workunit.client.0.smithi027.stdout:6/107: write fa [2465327,104181] 0 2021-11-03T17:14:04.647 INFO:tasks.workunit.client.1.smithi115.stdout:4/281: dwrite d7/dc/d1a/d21/d47/d56/d63/f6e [0,4194304] 0 2021-11-03T17:14:04.647 INFO:tasks.workunit.client.0.smithi027.stdout:7/170: link d0/d2/l9 d0/l34 0 2021-11-03T17:14:04.648 INFO:tasks.workunit.client.1.smithi115.stdout:4/282: getdents d7/dc/d1a/d21 0 2021-11-03T17:14:04.649 INFO:tasks.workunit.client.1.smithi115.stdout:4/283: dread - d7/dc/d6a/d3c/f66 zero size 2021-11-03T17:14:04.649 INFO:tasks.workunit.client.1.smithi115.stdout:4/284: mknod d7/dc/d1a/d21/d47/d56/d63/c71 1 2021-11-03T17:14:04.649 INFO:tasks.workunit.client.1.smithi115.stdout:4/285: rmdir d7/dc/d1a/d21/d47/d56/d42/d5f 39 2021-11-03T17:14:04.653 INFO:tasks.workunit.client.0.smithi027.stdout:6/108: dread f7 [0,4194304] 0 2021-11-03T17:14:04.653 INFO:tasks.workunit.client.0.smithi027.stdout:6/109: write dc/ff [541061,56371] 0 2021-11-03T17:14:04.654 INFO:tasks.workunit.client.0.smithi027.stdout:5/157: rename d2/d4/df/d2d to d2/d4/df/d39 0 2021-11-03T17:14:04.655 INFO:tasks.workunit.client.0.smithi027.stdout:5/158: write d2/d4/df/f15 [4072786,79972] 0 2021-11-03T17:14:04.656 INFO:tasks.workunit.client.0.smithi027.stdout:7/171: truncate d0/d2/d10/f11 2829865 0 2021-11-03T17:14:04.659 INFO:tasks.workunit.client.0.smithi027.stdout:6/110: creat dc/f14 x:0 0 0 2021-11-03T17:14:04.661 INFO:tasks.workunit.client.0.smithi027.stdout:5/159: creat d2/d4/df/d20/f3a x:0 0 0 2021-11-03T17:14:04.663 INFO:tasks.workunit.client.0.smithi027.stdout:6/111: creat dc/f15 x:0 0 0 2021-11-03T17:14:04.664 INFO:tasks.workunit.client.0.smithi027.stdout:8/192: dread f6 [4194304,4194304] 0 2021-11-03T17:14:04.664 INFO:tasks.workunit.client.0.smithi027.stdout:8/193: write d7/f8 [8695648,96494] 0 2021-11-03T17:14:04.664 INFO:tasks.workunit.client.0.smithi027.stdout:8/194: readlink d7/l11 0 2021-11-03T17:14:04.665 INFO:tasks.workunit.client.0.smithi027.stdout:9/171: dwrite db/d1b/f38 [0,4194304] 0 2021-11-03T17:14:04.665 INFO:tasks.workunit.client.0.smithi027.stdout:9/172: readlink db/l11 0 2021-11-03T17:14:04.665 INFO:tasks.workunit.client.0.smithi027.stdout:9/173: write db/d1b/d1c/f39 [511939,27863] 0 2021-11-03T17:14:04.666 INFO:tasks.workunit.client.0.smithi027.stdout:9/174: mknod db/d1b/d1c/d1d/c44 1 2021-11-03T17:14:04.666 INFO:tasks.workunit.client.0.smithi027.stdout:5/160: write d2/f7 [3512133,94883] 0 2021-11-03T17:14:04.666 INFO:tasks.workunit.client.0.smithi027.stdout:5/161: read - d2/d4/f34 zero size 2021-11-03T17:14:04.667 INFO:tasks.workunit.client.0.smithi027.stdout:7/172: mkdir d0/d2/d10/d35 0 2021-11-03T17:14:04.667 INFO:tasks.workunit.client.0.smithi027.stdout:7/173: stat d0/d2d/d2f 0 2021-11-03T17:14:04.678 INFO:tasks.workunit.client.0.smithi027.stdout:6/112: mkdir dc/d16 0 2021-11-03T17:14:04.679 INFO:tasks.workunit.client.0.smithi027.stdout:4/174: dwrite d6/f1f [0,4194304] 0 2021-11-03T17:14:04.680 INFO:tasks.workunit.client.0.smithi027.stdout:4/175: write d6/d9/d28/f3b [58944,20832] 0 2021-11-03T17:14:04.680 INFO:tasks.workunit.client.0.smithi027.stdout:8/195: symlink d7/d2d/d38/l3a 0 2021-11-03T17:14:04.681 INFO:tasks.workunit.client.1.smithi115.stdout:4/286: dwrite d7/dc/d1a/d21/d47/d56/d63/f6e [0,4194304] 0 2021-11-03T17:14:04.682 INFO:tasks.workunit.client.1.smithi115.stdout:4/287: chown d7/dc/d1a/d21/l29 8 1 2021-11-03T17:14:04.682 INFO:tasks.workunit.client.1.smithi115.stdout:4/288: truncate d7/dc/d6a/f46 814055 0 2021-11-03T17:14:04.682 INFO:tasks.workunit.client.1.smithi115.stdout:4/289: mkdir d7/dc/d6a/d72 0 2021-11-03T17:14:04.683 INFO:tasks.workunit.client.0.smithi027.stdout:2/202: dwrite d0/d1b/d29/f2f [0,4194304] 0 2021-11-03T17:14:04.683 INFO:tasks.workunit.client.0.smithi027.stdout:4/176: read d6/f8 [1754197,96350] 0 2021-11-03T17:14:04.684 INFO:tasks.workunit.client.0.smithi027.stdout:4/177: write d6/d9/d28/f3b [723421,97056] 0 2021-11-03T17:14:04.684 INFO:tasks.workunit.client.0.smithi027.stdout:4/178: read - d6/db/d21/f2c zero size 2021-11-03T17:14:04.684 INFO:tasks.workunit.client.0.smithi027.stdout:2/203: write d0/d5/f4 [8246348,130037] 0 2021-11-03T17:14:04.684 INFO:tasks.workunit.client.0.smithi027.stdout:2/204: fdatasync d0/d2a/f30 0 2021-11-03T17:14:04.685 INFO:tasks.workunit.client.1.smithi115.stdout:4/290: symlink d7/dc/d1a/d21/d47/d56/d42/d5f/l73 0 2021-11-03T17:14:04.685 INFO:tasks.workunit.client.1.smithi115.stdout:4/291: creat d7/dc/d1a/d21/d47/f74 x:0 0 0 2021-11-03T17:14:04.686 INFO:tasks.workunit.client.0.smithi027.stdout:9/175: link db/d1b/d1c/d1d/f21 db/d1b/f45 0 2021-11-03T17:14:04.686 INFO:tasks.workunit.client.0.smithi027.stdout:9/176: fdatasync db/d1b/d23/d2c/f43 0 2021-11-03T17:14:04.688 INFO:tasks.workunit.client.0.smithi027.stdout:5/162: symlink d2/d12/l3b 0 2021-11-03T17:14:04.691 INFO:tasks.workunit.client.0.smithi027.stdout:7/174: mkdir d0/d2/d10/d1e/d20/d31/d36 0 2021-11-03T17:14:04.696 INFO:tasks.workunit.client.0.smithi027.stdout:8/196: unlink d7/d10/f2b 0 2021-11-03T17:14:04.697 INFO:tasks.workunit.client.0.smithi027.stdout:8/197: mknod d7/d2d/d38/c3b 1 2021-11-03T17:14:04.697 INFO:tasks.workunit.client.0.smithi027.stdout:8/198: dread - d7/d10/d18/f33 zero size 2021-11-03T17:14:04.698 INFO:tasks.workunit.client.1.smithi115.stdout:4/292: dwrite d7/dc/d6a/d3c/f3e [0,4194304] 0 2021-11-03T17:14:04.698 INFO:tasks.workunit.client.1.smithi115.stdout:4/293: read d7/dc/d1a/f38 [3672750,55306] 0 2021-11-03T17:14:04.698 INFO:tasks.workunit.client.0.smithi027.stdout:8/199: write d7/d10/d18/d1a/f1f [1686250,51519] 0 2021-11-03T17:14:04.699 INFO:tasks.workunit.client.0.smithi027.stdout:8/200: dread - d7/f32 zero size 2021-11-03T17:14:04.699 INFO:tasks.workunit.client.0.smithi027.stdout:6/113: getdents dc 0 2021-11-03T17:14:04.700 INFO:tasks.workunit.client.1.smithi115.stdout:4/294: getdents d7/dc/d1a/d21/d47/d56/d42 0 2021-11-03T17:14:04.700 INFO:tasks.workunit.client.1.smithi115.stdout:4/295: stat d7/l9 0 2021-11-03T17:14:04.700 INFO:tasks.workunit.client.1.smithi115.stdout:4/296: stat d7/dc/d6a/d3c/f66 0 2021-11-03T17:14:04.701 INFO:tasks.workunit.client.1.smithi115.stdout:4/297: rmdir d7/dc/d6a 39 2021-11-03T17:14:04.701 INFO:tasks.workunit.client.1.smithi115.stdout:4/298: mknod d7/dc/d1a/d21/c75 1 2021-11-03T17:14:04.703 INFO:tasks.workunit.client.0.smithi027.stdout:6/114: dread dc/ff [0,4194304] 0 2021-11-03T17:14:04.703 INFO:tasks.workunit.client.0.smithi027.stdout:6/115: getdents dc 0 2021-11-03T17:14:04.703 INFO:tasks.workunit.client.0.smithi027.stdout:6/116: mknod dc/d16/c17 1 2021-11-03T17:14:04.707 INFO:tasks.workunit.client.0.smithi027.stdout:4/179: mkdir d6/db/d21/d2b/d2d/d3e 0 2021-11-03T17:14:04.708 INFO:tasks.workunit.client.0.smithi027.stdout:4/180: chown d6/db/l29 1 1 2021-11-03T17:14:04.708 INFO:tasks.workunit.client.0.smithi027.stdout:4/181: read f4 [3090713,31864] 0 2021-11-03T17:14:04.709 INFO:tasks.workunit.client.0.smithi027.stdout:4/182: dread - d6/d9/d28/f3c zero size 2021-11-03T17:14:04.709 INFO:tasks.workunit.client.0.smithi027.stdout:2/205: mkdir d0/d1b/d43 0 2021-11-03T17:14:04.713 INFO:tasks.workunit.client.0.smithi027.stdout:5/163: rename d2/d4/df/d39 to d2/d2c/d3c 0 2021-11-03T17:14:04.713 INFO:tasks.workunit.client.0.smithi027.stdout:5/164: mknod d2/d1f/c3d 1 2021-11-03T17:14:04.713 INFO:tasks.workunit.client.0.smithi027.stdout:5/165: write d2/d4/f34 [88922,37121] 0 2021-11-03T17:14:04.714 INFO:tasks.workunit.client.0.smithi027.stdout:9/177: symlink db/d1b/d1c/d1d/d40/l46 0 2021-11-03T17:14:04.714 INFO:tasks.workunit.client.0.smithi027.stdout:5/166: fsync d2/f7 0 2021-11-03T17:14:04.715 INFO:tasks.workunit.client.0.smithi027.stdout:7/175: truncate d0/f25 731564 0 2021-11-03T17:14:04.715 INFO:tasks.workunit.client.0.smithi027.stdout:7/176: chown d0/lc 32178 1 2021-11-03T17:14:04.715 INFO:tasks.workunit.client.0.smithi027.stdout:7/177: chown d0/d2/l9 1520023590 1 2021-11-03T17:14:04.716 INFO:tasks.workunit.client.0.smithi027.stdout:8/201: creat d7/d10/d18/d1a/f3c x:0 0 0 2021-11-03T17:14:04.717 INFO:tasks.workunit.client.0.smithi027.stdout:6/117: rename dc/fe to dc/f18 0 2021-11-03T17:14:04.723 INFO:tasks.workunit.client.0.smithi027.stdout:9/178: symlink db/d1b/d1c/d1d/l47 0 2021-11-03T17:14:04.723 INFO:tasks.workunit.client.0.smithi027.stdout:9/179: write db/d1b/d23/f3c [715559,85307] 0 2021-11-03T17:14:04.723 INFO:tasks.workunit.client.0.smithi027.stdout:9/180: write db/d1b/d23/d2c/f43 [576408,128027] 0 2021-11-03T17:14:04.724 INFO:tasks.workunit.client.0.smithi027.stdout:8/202: creat d7/d2d/d38/f3d x:0 0 0 2021-11-03T17:14:04.727 INFO:tasks.workunit.client.0.smithi027.stdout:6/118: creat dc/f19 x:0 0 0 2021-11-03T17:14:04.731 INFO:tasks.workunit.client.0.smithi027.stdout:2/206: dwrite d0/d1b/d29/f39 [4194304,4194304] 0 2021-11-03T17:14:04.737 INFO:tasks.workunit.client.0.smithi027.stdout:8/203: rename d7/f35 to d7/d10/d18/d1a/f3e 0 2021-11-03T17:14:04.739 INFO:tasks.workunit.client.0.smithi027.stdout:2/207: rmdir d0/d1b/d1c 39 2021-11-03T17:14:04.743 INFO:tasks.workunit.client.0.smithi027.stdout:8/204: write d7/d10/d18/d1a/f2e [1596918,30232] 0 2021-11-03T17:14:04.745 INFO:tasks.workunit.client.0.smithi027.stdout:2/208: creat d0/d2a/f44 x:0 0 0 2021-11-03T17:14:04.747 INFO:tasks.workunit.client.0.smithi027.stdout:2/209: mkdir d0/d45 0 2021-11-03T17:14:04.749 INFO:tasks.workunit.client.0.smithi027.stdout:2/210: creat d0/f46 x:0 0 0 2021-11-03T17:14:04.751 INFO:tasks.workunit.client.0.smithi027.stdout:2/211: rename d0/d1b/l40 to d0/d1b/d29/l47 0 2021-11-03T17:14:04.751 INFO:tasks.workunit.client.0.smithi027.stdout:7/178: dwrite d0/d2/d10/f19 [0,4194304] 0 2021-11-03T17:14:04.752 INFO:tasks.workunit.client.0.smithi027.stdout:5/167: dwrite d2/d4/df/f2f [0,4194304] 0 2021-11-03T17:14:04.753 INFO:tasks.workunit.client.0.smithi027.stdout:6/119: dwrite dc/f14 [0,4194304] 0 2021-11-03T17:14:04.753 INFO:tasks.workunit.client.0.smithi027.stdout:6/120: chown dc/f18 195965207 1 2021-11-03T17:14:04.753 INFO:tasks.workunit.client.0.smithi027.stdout:2/212: creat d0/d1b/d29/f48 x:0 0 0 2021-11-03T17:14:04.753 INFO:tasks.workunit.client.0.smithi027.stdout:7/179: dread d0/d2/d10/d1e/d20/f26 [0,4194304] 0 2021-11-03T17:14:04.756 INFO:tasks.workunit.client.0.smithi027.stdout:9/181: rmdir db/d1b/d1c/d1d 39 2021-11-03T17:14:04.756 INFO:tasks.workunit.client.0.smithi027.stdout:9/182: chown db/d1b/d23/d2c/f43 213010 1 2021-11-03T17:14:04.759 INFO:tasks.workunit.client.0.smithi027.stdout:6/121: fsync dc/f19 0 2021-11-03T17:14:04.760 INFO:tasks.workunit.client.0.smithi027.stdout:6/122: chown dc/f15 760640 1 2021-11-03T17:14:04.764 INFO:tasks.workunit.client.0.smithi027.stdout:8/205: dwrite d7/f1d [0,4194304] 0 2021-11-03T17:14:04.764 INFO:tasks.workunit.client.0.smithi027.stdout:8/206: chown d7/d10/d18/f33 7771154 1 2021-11-03T17:14:04.771 INFO:tasks.workunit.client.0.smithi027.stdout:5/168: getdents d2/d12 0 2021-11-03T17:14:04.773 INFO:tasks.workunit.client.0.smithi027.stdout:9/183: rename db/d1b/d23/f3c to db/d1b/d1c/d1d/d40/f48 0 2021-11-03T17:14:04.776 INFO:tasks.workunit.client.0.smithi027.stdout:6/123: mkdir dc/d1a 0 2021-11-03T17:14:04.778 INFO:tasks.workunit.client.0.smithi027.stdout:8/207: mkdir d7/d10/d3f 0 2021-11-03T17:14:04.778 INFO:tasks.workunit.client.0.smithi027.stdout:8/208: write d7/d2d/d38/f3d [757840,65496] 0 2021-11-03T17:14:04.779 INFO:tasks.workunit.client.0.smithi027.stdout:8/209: stat d7/d10/d21/f25 0 2021-11-03T17:14:04.780 INFO:tasks.workunit.client.0.smithi027.stdout:5/169: link d2/d4/f5 d2/f3e 0 2021-11-03T17:14:04.780 INFO:tasks.workunit.client.0.smithi027.stdout:5/170: fdatasync d2/f3e 0 2021-11-03T17:14:04.781 INFO:tasks.workunit.client.0.smithi027.stdout:5/171: dread - d2/d12/f2a zero size 2021-11-03T17:14:04.781 INFO:tasks.workunit.client.0.smithi027.stdout:2/213: truncate d0/d1b/d29/f39 5272104 0 2021-11-03T17:14:04.782 INFO:tasks.workunit.client.0.smithi027.stdout:6/124: rmdir dc/d1a 0 2021-11-03T17:14:04.784 INFO:tasks.workunit.client.0.smithi027.stdout:5/172: symlink d2/d4/df/l3f 0 2021-11-03T17:14:04.784 INFO:tasks.workunit.client.0.smithi027.stdout:5/173: write d2/f1e [1039761,55931] 0 2021-11-03T17:14:04.785 INFO:tasks.workunit.client.0.smithi027.stdout:5/174: fdatasync d2/d4/df/f2f 0 2021-11-03T17:14:04.786 INFO:tasks.workunit.client.0.smithi027.stdout:5/175: write d2/d4/df/f30 [691129,10728] 0 2021-11-03T17:14:04.786 INFO:tasks.workunit.client.0.smithi027.stdout:2/214: truncate d0/d5/f1e 346657 0 2021-11-03T17:14:04.786 INFO:tasks.workunit.client.0.smithi027.stdout:2/215: dread - d0/f46 zero size 2021-11-03T17:14:04.786 INFO:tasks.workunit.client.0.smithi027.stdout:2/216: chown d0/f41 1093805040 1 2021-11-03T17:14:04.787 INFO:tasks.workunit.client.0.smithi027.stdout:2/217: creat d0/d1b/d29/f49 x:0 0 0 2021-11-03T17:14:04.787 INFO:tasks.workunit.client.0.smithi027.stdout:5/176: write d2/d4/f5 [2454169,90332] 0 2021-11-03T17:14:04.816 INFO:tasks.workunit.client.0.smithi027.stdout:6/125: dwrite dc/f11 [0,4194304] 0 2021-11-03T17:14:04.816 INFO:tasks.workunit.client.0.smithi027.stdout:6/126: stat dc 0 2021-11-03T17:14:04.817 INFO:tasks.workunit.client.0.smithi027.stdout:8/210: dwrite d7/d2d/d38/f3d [0,4194304] 0 2021-11-03T17:14:04.817 INFO:tasks.workunit.client.0.smithi027.stdout:8/211: fsync d7/d10/d18/d1a/f1f 0 2021-11-03T17:14:04.818 INFO:tasks.workunit.client.0.smithi027.stdout:2/218: dwrite d0/d1b/d1c/f33 [0,4194304] 0 2021-11-03T17:14:04.823 INFO:tasks.workunit.client.0.smithi027.stdout:6/127: unlink dc/f11 0 2021-11-03T17:14:04.823 INFO:tasks.workunit.client.0.smithi027.stdout:6/128: dread - dc/f10 zero size 2021-11-03T17:14:04.826 INFO:tasks.workunit.client.0.smithi027.stdout:8/212: mkdir d7/d2d/d40 0 2021-11-03T17:14:04.829 INFO:tasks.workunit.client.0.smithi027.stdout:6/129: mkdir dc/d16/d1b 0 2021-11-03T17:14:04.831 INFO:tasks.workunit.client.0.smithi027.stdout:8/213: mkdir d7/d10/d21/d41 0 2021-11-03T17:14:04.846 INFO:tasks.workunit.client.0.smithi027.stdout:6/130: dwrite dc/f10 [0,4194304] 0 2021-11-03T17:14:04.846 INFO:tasks.workunit.client.0.smithi027.stdout:6/131: chown dc/f19 14935 1 2021-11-03T17:14:04.848 INFO:tasks.workunit.client.0.smithi027.stdout:6/132: symlink dc/d16/l1c 0 2021-11-03T17:14:04.850 INFO:tasks.workunit.client.0.smithi027.stdout:6/133: symlink dc/l1d 0 2021-11-03T17:14:04.851 INFO:tasks.workunit.client.0.smithi027.stdout:6/134: rmdir dc/d16 39 2021-11-03T17:14:04.852 INFO:tasks.workunit.client.0.smithi027.stdout:6/135: creat dc/f1e x:0 0 0 2021-11-03T17:14:04.852 INFO:tasks.workunit.client.0.smithi027.stdout:6/136: creat dc/f1f x:0 0 0 2021-11-03T17:14:04.904 INFO:tasks.workunit.client.0.smithi027.stdout:2/219: dwrite d0/d5/f4 [4194304,4194304] 0 2021-11-03T17:14:04.904 INFO:tasks.workunit.client.0.smithi027.stdout:2/220: fsync d0/f41 0 2021-11-03T17:14:04.904 INFO:tasks.workunit.client.0.smithi027.stdout:2/221: truncate d0/d2a/f30 737194 0 2021-11-03T17:14:04.907 INFO:tasks.workunit.client.0.smithi027.stdout:6/137: getdents dc 0 2021-11-03T17:14:04.909 INFO:tasks.workunit.client.0.smithi027.stdout:6/138: symlink dc/d16/d1b/l20 0 2021-11-03T17:14:04.909 INFO:tasks.workunit.client.0.smithi027.stdout:6/139: read fa [20320,123022] 0 2021-11-03T17:14:04.910 INFO:tasks.workunit.client.0.smithi027.stdout:6/140: stat dc/f1f 0 2021-11-03T17:14:04.916 INFO:tasks.workunit.client.0.smithi027.stdout:2/222: dread d0/d2a/f23 [0,4194304] 0 2021-11-03T17:14:04.916 INFO:tasks.workunit.client.0.smithi027.stdout:2/223: fdatasync d0/d2a/f30 0 2021-11-03T17:14:04.916 INFO:tasks.workunit.client.0.smithi027.stdout:2/224: mknod d0/d45/c4a 1 2021-11-03T17:14:04.917 INFO:tasks.workunit.client.0.smithi027.stdout:2/225: chown d0/d1b/d29/l47 11178374 1 2021-11-03T17:14:04.917 INFO:tasks.workunit.client.0.smithi027.stdout:6/141: dread dc/f10 [0,4194304] 0 2021-11-03T17:14:04.918 INFO:tasks.workunit.client.0.smithi027.stdout:2/226: creat d0/d1b/d29/f4b x:0 0 0 2021-11-03T17:14:04.919 INFO:tasks.workunit.client.0.smithi027.stdout:6/142: unlink dc/l1d 0 2021-11-03T17:14:04.920 INFO:tasks.workunit.client.0.smithi027.stdout:6/143: chown dc/d16/d1b 5286 1 2021-11-03T17:14:04.920 INFO:tasks.workunit.client.0.smithi027.stdout:6/144: fdatasync dc/f15 0 2021-11-03T17:14:04.920 INFO:tasks.workunit.client.0.smithi027.stdout:6/145: write dc/ff [893537,81578] 0 2021-11-03T17:14:04.920 INFO:tasks.workunit.client.0.smithi027.stdout:6/146: write dc/f1e [702262,33023] 0 2021-11-03T17:14:04.921 INFO:tasks.workunit.client.0.smithi027.stdout:6/147: dread - dc/f19 zero size 2021-11-03T17:14:04.922 INFO:tasks.workunit.client.0.smithi027.stdout:2/227: dread d0/d5/f8 [0,4194304] 0 2021-11-03T17:14:04.922 INFO:tasks.workunit.client.0.smithi027.stdout:2/228: write d0/d1b/d1c/f24 [1291586,233] 0 2021-11-03T17:14:04.930 INFO:tasks.workunit.client.0.smithi027.stdout:2/229: dread d0/d1b/d1c/f33 [0,4194304] 0 2021-11-03T17:14:04.931 INFO:tasks.workunit.client.0.smithi027.stdout:2/230: getdents d0/d1b 0 2021-11-03T17:14:04.933 INFO:tasks.workunit.client.0.smithi027.stdout:2/231: symlink d0/d1b/l4c 0 2021-11-03T17:14:04.934 INFO:tasks.workunit.client.0.smithi027.stdout:2/232: symlink d0/d2a/l4d 0 2021-11-03T17:14:04.936 INFO:tasks.workunit.client.0.smithi027.stdout:6/148: dwrite dc/f1f [0,4194304] 0 2021-11-03T17:14:04.936 INFO:tasks.workunit.client.0.smithi027.stdout:6/149: dread - dc/f19 zero size 2021-11-03T17:14:04.957 INFO:tasks.workunit.client.0.smithi027.stdout:2/233: dwrite d0/d1b/d29/f49 [0,4194304] 0 2021-11-03T17:14:04.957 INFO:tasks.workunit.client.0.smithi027.stdout:6/150: dwrite dc/f18 [0,4194304] 0 2021-11-03T17:14:04.960 INFO:tasks.workunit.client.0.smithi027.stdout:2/234: creat d0/d1b/d29/f4e x:0 0 0 2021-11-03T17:14:04.960 INFO:tasks.workunit.client.0.smithi027.stdout:2/235: chown d0/d2a/l3f 34773377 1 2021-11-03T17:14:04.962 INFO:tasks.workunit.client.0.smithi027.stdout:6/151: rmdir dc/d16 39 2021-11-03T17:14:04.962 INFO:tasks.workunit.client.0.smithi027.stdout:6/152: chown f7 30964 1 2021-11-03T17:14:04.969 INFO:tasks.workunit.client.0.smithi027.stdout:6/153: dread dc/f1f [0,4194304] 0 2021-11-03T17:14:04.969 INFO:tasks.workunit.client.0.smithi027.stdout:6/154: write dc/f15 [926779,7476] 0 2021-11-03T17:14:04.969 INFO:tasks.workunit.client.0.smithi027.stdout:6/155: write dc/f1e [809890,101802] 0 2021-11-03T17:14:04.970 INFO:tasks.workunit.client.0.smithi027.stdout:6/156: getdents dc 0 2021-11-03T17:14:04.972 INFO:tasks.workunit.client.0.smithi027.stdout:6/157: link dc/l12 dc/d16/d1b/l21 0 2021-11-03T17:14:04.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:04 smithi115 conmon[34670]: cluster 2021-11-03T17:14:04.603541+0000 mgr.smithi027.ufhhvb (mgr.14162) 299 : cluster 2021-11-03T17:14:04.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:04 smithi115 conmon[34670]: [DBG] pgmap v249: 65 pgs: 65 active+clean; 745 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 20 MiB/s rd, 44 MiB/s wr, 86 op/s 2021-11-03T17:14:05.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:04 smithi027 conmon[32322]: cluster 2021 2021-11-03T17:14:05.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:04 smithi027 conmon[32322]: -11-03T17:14:04.603541+0000 mgr.smithi027.ufhhvb (mgr.14162) 299 : cluster [DBG] pgmap v249: 65 pgs: 65 active+clean; 745 MiB data, 2.4 GiB used, 534 GiB / 536 GiB avail; 20 MiB/s rd, 44 MiB/s wr, 86 op/s 2021-11-03T17:14:05.327 INFO:tasks.workunit.client.0.smithi027.stdout:4/183: sync 2021-11-03T17:14:05.328 INFO:tasks.workunit.client.0.smithi027.stdout:7/180: sync 2021-11-03T17:14:05.328 INFO:tasks.workunit.client.0.smithi027.stdout:1/200: sync 2021-11-03T17:14:05.328 INFO:tasks.workunit.client.0.smithi027.stdout:0/162: sync 2021-11-03T17:14:05.329 INFO:tasks.workunit.client.0.smithi027.stdout:3/107: sync 2021-11-03T17:14:05.329 INFO:tasks.workunit.client.0.smithi027.stdout:3/108: fsync d1/f14 0 2021-11-03T17:14:05.330 INFO:tasks.workunit.client.0.smithi027.stdout:6/158: sync 2021-11-03T17:14:05.330 INFO:tasks.workunit.client.0.smithi027.stdout:5/177: sync 2021-11-03T17:14:05.331 INFO:tasks.workunit.client.0.smithi027.stdout:2/236: sync 2021-11-03T17:14:05.331 INFO:tasks.workunit.client.0.smithi027.stdout:9/184: sync 2021-11-03T17:14:05.331 INFO:tasks.workunit.client.0.smithi027.stdout:3/109: write d1/f17 [3704650,126309] 0 2021-11-03T17:14:05.331 INFO:tasks.workunit.client.0.smithi027.stdout:0/163: write d9/d12/f1e [392696,112205] 0 2021-11-03T17:14:05.331 INFO:tasks.workunit.client.0.smithi027.stdout:1/201: rmdir d6 39 2021-11-03T17:14:05.332 INFO:tasks.workunit.client.0.smithi027.stdout:0/164: write d9/f2a [1568524,118911] 0 2021-11-03T17:14:05.332 INFO:tasks.workunit.client.0.smithi027.stdout:5/178: write d2/d4/df/f2f [2337162,115028] 0 2021-11-03T17:14:05.332 INFO:tasks.workunit.client.0.smithi027.stdout:8/214: sync 2021-11-03T17:14:05.333 INFO:tasks.workunit.client.0.smithi027.stdout:7/181: link d0/le d0/d2/d10/d35/l37 0 2021-11-03T17:14:05.333 INFO:tasks.workunit.client.0.smithi027.stdout:7/182: dread - d0/d2/d1c/f27 zero size 2021-11-03T17:14:05.334 INFO:tasks.workunit.client.0.smithi027.stdout:2/237: mknod d0/d1b/d1c/c4f 1 2021-11-03T17:14:05.334 INFO:tasks.workunit.client.0.smithi027.stdout:9/185: creat db/d1b/d1c/d1d/f49 x:0 0 0 2021-11-03T17:14:05.334 INFO:tasks.workunit.client.0.smithi027.stdout:9/186: fsync db/d1b/d1c/f3e 0 2021-11-03T17:14:05.335 INFO:tasks.workunit.client.0.smithi027.stdout:1/202: mkdir d6/de/d13/d15/d43 0 2021-11-03T17:14:05.335 INFO:tasks.workunit.client.0.smithi027.stdout:0/165: creat d9/da/f38 x:0 0 0 2021-11-03T17:14:05.337 INFO:tasks.workunit.client.0.smithi027.stdout:8/215: symlink d7/d10/d21/d41/l42 0 2021-11-03T17:14:05.337 INFO:tasks.workunit.client.0.smithi027.stdout:8/216: read - d7/d10/d18/f33 zero size 2021-11-03T17:14:05.338 INFO:tasks.workunit.client.0.smithi027.stdout:7/183: symlink d0/d2/d10/d35/l38 0 2021-11-03T17:14:05.340 INFO:tasks.workunit.client.0.smithi027.stdout:2/238: unlink d0/l19 0 2021-11-03T17:14:05.341 INFO:tasks.workunit.client.0.smithi027.stdout:9/187: creat db/d1b/d2b/f4a x:0 0 0 2021-11-03T17:14:05.342 INFO:tasks.workunit.client.0.smithi027.stdout:1/203: mkdir d6/de/d44 0 2021-11-03T17:14:05.343 INFO:tasks.workunit.client.0.smithi027.stdout:7/184: write d0/d2/fd [3039744,80639] 0 2021-11-03T17:14:05.345 INFO:tasks.workunit.client.0.smithi027.stdout:3/110: dread d1/fa [0,4194304] 0 2021-11-03T17:14:05.345 INFO:tasks.workunit.client.0.smithi027.stdout:6/159: dwrite dc/ff [0,4194304] 0 2021-11-03T17:14:05.346 INFO:tasks.workunit.client.0.smithi027.stdout:5/179: dread d2/f18 [0,4194304] 0 2021-11-03T17:14:05.346 INFO:tasks.workunit.client.0.smithi027.stdout:2/239: unlink d0/d5/d2c/l3a 0 2021-11-03T17:14:05.347 INFO:tasks.workunit.client.0.smithi027.stdout:8/217: write d7/d2d/d38/f3d [1468309,25142] 0 2021-11-03T17:14:05.347 INFO:tasks.workunit.client.0.smithi027.stdout:8/218: fsync f6 0 2021-11-03T17:14:05.349 INFO:tasks.workunit.client.0.smithi027.stdout:9/188: unlink db/d1b/l29 0 2021-11-03T17:14:05.350 INFO:tasks.workunit.client.0.smithi027.stdout:1/204: symlink d6/de/l45 0 2021-11-03T17:14:05.352 INFO:tasks.workunit.client.0.smithi027.stdout:0/166: link d9/f2a d9/da/f39 0 2021-11-03T17:14:05.354 INFO:tasks.workunit.client.0.smithi027.stdout:4/184: dwrite d6/db/f34 [0,4194304] 0 2021-11-03T17:14:05.355 INFO:tasks.workunit.client.0.smithi027.stdout:7/185: unlink d0/d2/l7 0 2021-11-03T17:14:05.356 INFO:tasks.workunit.client.0.smithi027.stdout:3/111: symlink d1/db/l20 0 2021-11-03T17:14:05.356 INFO:tasks.workunit.client.0.smithi027.stdout:3/112: truncate d1/f1a 671821 0 2021-11-03T17:14:05.357 INFO:tasks.workunit.client.0.smithi027.stdout:1/205: write d6/de/f2c [2357030,103705] 0 2021-11-03T17:14:05.358 INFO:tasks.workunit.client.0.smithi027.stdout:3/113: write d1/f6 [1753864,55931] 0 2021-11-03T17:14:05.359 INFO:tasks.workunit.client.0.smithi027.stdout:3/114: write d1/f6 [424363,38471] 0 2021-11-03T17:14:05.359 INFO:tasks.workunit.client.0.smithi027.stdout:3/115: fsync d1/f1a 0 2021-11-03T17:14:05.359 INFO:tasks.workunit.client.0.smithi027.stdout:3/116: fsync d1/fa 0 2021-11-03T17:14:05.360 INFO:tasks.workunit.client.0.smithi027.stdout:5/180: rename d2/d4/df/f2f to d2/d2c/d3c/f40 0 2021-11-03T17:14:05.361 INFO:tasks.workunit.client.0.smithi027.stdout:5/181: write d2/d4/f34 [1056106,17948] 0 2021-11-03T17:14:05.361 INFO:tasks.workunit.client.0.smithi027.stdout:2/240: symlink d0/d1b/d29/l50 0 2021-11-03T17:14:05.361 INFO:tasks.workunit.client.0.smithi027.stdout:2/241: mknod d0/d45/c51 1 2021-11-03T17:14:05.362 INFO:tasks.workunit.client.0.smithi027.stdout:9/189: creat db/d1b/d23/f4b x:0 0 0 2021-11-03T17:14:05.362 INFO:tasks.workunit.client.0.smithi027.stdout:3/117: dread d1/f1d [0,4194304] 0 2021-11-03T17:14:05.367 INFO:tasks.workunit.client.0.smithi027.stdout:0/167: dwrite d9/d12/f17 [0,4194304] 0 2021-11-03T17:14:05.368 INFO:tasks.workunit.client.0.smithi027.stdout:7/186: creat d0/d2d/d2f/f39 x:0 0 0 2021-11-03T17:14:05.368 INFO:tasks.workunit.client.0.smithi027.stdout:7/187: write d0/d2d/d2f/f39 [1005588,25060] 0 2021-11-03T17:14:05.390 INFO:tasks.workunit.client.0.smithi027.stdout:1/206: truncate d6/de/d13/f38 1319866 0 2021-11-03T17:14:05.390 INFO:tasks.workunit.client.0.smithi027.stdout:1/207: mknod d6/d11/c46 1 2021-11-03T17:14:05.390 INFO:tasks.workunit.client.0.smithi027.stdout:1/208: write d6/f29 [617395,120108] 0 2021-11-03T17:14:05.391 INFO:tasks.workunit.client.0.smithi027.stdout:1/209: write d6/de/d13/d15/f42 [337131,105354] 0 2021-11-03T17:14:05.392 INFO:tasks.workunit.client.0.smithi027.stdout:7/188: dwrite d0/f13 [4194304,4194304] 0 2021-11-03T17:14:05.393 INFO:tasks.workunit.client.0.smithi027.stdout:8/219: rename d7/f32 to d7/d10/d18/d1a/f43 0 2021-11-03T17:14:05.394 INFO:tasks.workunit.client.0.smithi027.stdout:5/182: creat d2/d2c/d3c/f41 x:0 0 0 2021-11-03T17:14:05.395 INFO:tasks.workunit.client.0.smithi027.stdout:2/242: rmdir d0/d1b 39 2021-11-03T17:14:05.395 INFO:tasks.workunit.client.0.smithi027.stdout:9/190: dwrite db/d1b/d2b/f2a [0,4194304] 0 2021-11-03T17:14:05.395 INFO:tasks.workunit.client.0.smithi027.stdout:9/191: fsync db/d1b/d23/d2c/f43 0 2021-11-03T17:14:05.396 INFO:tasks.workunit.client.0.smithi027.stdout:9/192: dread - db/d1b/d2b/f4a zero size 2021-11-03T17:14:05.396 INFO:tasks.workunit.client.0.smithi027.stdout:3/118: dwrite d1/f13 [0,4194304] 0 2021-11-03T17:14:05.397 INFO:tasks.workunit.client.0.smithi027.stdout:9/193: read db/d1b/d2b/f2a [361347,60576] 0 2021-11-03T17:14:05.408 INFO:tasks.workunit.client.0.smithi027.stdout:0/168: link d9/d12/f28 d9/da/d30/f3a 0 2021-11-03T17:14:05.411 INFO:tasks.workunit.client.0.smithi027.stdout:1/210: creat d6/d11/d14/d1f/d25/f47 x:0 0 0 2021-11-03T17:14:05.414 INFO:tasks.workunit.client.0.smithi027.stdout:4/185: rename d6/d9/d16/f3d to d6/db/d21/d2b/f3f 0 2021-11-03T17:14:05.419 INFO:tasks.workunit.client.0.smithi027.stdout:8/220: link d7/d10/d18/d1a/f3e d7/d2d/d38/f44 0 2021-11-03T17:14:05.421 INFO:tasks.workunit.client.0.smithi027.stdout:5/183: getdents d2/d4/df 0 2021-11-03T17:14:05.422 INFO:tasks.workunit.client.0.smithi027.stdout:2/243: link d0/l2d d0/d5/d2c/l52 0 2021-11-03T17:14:05.423 INFO:tasks.workunit.client.0.smithi027.stdout:2/244: chown d0/d5/l17 50072 1 2021-11-03T17:14:05.424 INFO:tasks.workunit.client.0.smithi027.stdout:3/119: getdents d1 0 2021-11-03T17:14:05.425 INFO:tasks.workunit.client.0.smithi027.stdout:9/194: creat db/d1b/d23/d2c/f4c x:0 0 0 2021-11-03T17:14:05.426 INFO:tasks.workunit.client.0.smithi027.stdout:9/195: chown db/d1b/d1c/d1d/f1e 19816364 1 2021-11-03T17:14:05.426 INFO:tasks.workunit.client.0.smithi027.stdout:9/196: chown db/d1b/d1c/d1d/f49 310 1 2021-11-03T17:14:05.426 INFO:tasks.workunit.client.0.smithi027.stdout:9/197: chown db/d1b/d1c/d1d 21 1 2021-11-03T17:14:05.440 INFO:tasks.workunit.client.0.smithi027.stdout:0/169: creat d9/d12/f3b x:0 0 0 2021-11-03T17:14:05.441 INFO:tasks.workunit.client.0.smithi027.stdout:1/211: link d6/f29 d6/de/d13/d15/d43/f48 0 2021-11-03T17:14:05.443 INFO:tasks.workunit.client.0.smithi027.stdout:1/212: write f2 [2726588,25784] 0 2021-11-03T17:14:05.443 INFO:tasks.workunit.client.0.smithi027.stdout:1/213: write d6/d11/d14/d1f/d25/f2d [1056847,9816] 0 2021-11-03T17:14:05.443 INFO:tasks.workunit.client.0.smithi027.stdout:7/189: mkdir d0/d2/d10/d1e/d20/d31/d36/d3a 0 2021-11-03T17:14:05.443 INFO:tasks.workunit.client.0.smithi027.stdout:7/190: fsync d0/d2/d10/f11 0 2021-11-03T17:14:05.444 INFO:tasks.workunit.client.0.smithi027.stdout:1/214: write d6/de/f19 [2353760,30382] 0 2021-11-03T17:14:05.445 INFO:tasks.workunit.client.0.smithi027.stdout:1/215: dread - d6/d11/d14/f35 zero size 2021-11-03T17:14:05.445 INFO:tasks.workunit.client.0.smithi027.stdout:1/216: dread - d6/d11/d14/f35 zero size 2021-11-03T17:14:05.445 INFO:tasks.workunit.client.0.smithi027.stdout:4/186: mkdir d6/d9/d28/d40 0 2021-11-03T17:14:05.446 INFO:tasks.workunit.client.0.smithi027.stdout:8/221: truncate d7/d10/f12 741082 0 2021-11-03T17:14:05.447 INFO:tasks.workunit.client.0.smithi027.stdout:5/184: link d2/d12/f2a d2/f42 0 2021-11-03T17:14:05.449 INFO:tasks.workunit.client.0.smithi027.stdout:2/245: rename d0/d1b/d29/f49 to d0/d5/f53 0 2021-11-03T17:14:05.451 INFO:tasks.workunit.client.0.smithi027.stdout:0/170: creat d9/da/d30/f3c x:0 0 0 2021-11-03T17:14:05.451 INFO:tasks.workunit.client.0.smithi027.stdout:9/198: dwrite db/d1b/d2b/f2a [0,4194304] 0 2021-11-03T17:14:05.452 INFO:tasks.workunit.client.0.smithi027.stdout:3/120: dwrite d1/f6 [0,4194304] 0 2021-11-03T17:14:05.452 INFO:tasks.workunit.client.0.smithi027.stdout:3/121: rename d1/db to d1/db/d21 22 2021-11-03T17:14:05.452 INFO:tasks.workunit.client.0.smithi027.stdout:7/191: symlink d0/d2/d1f/l3b 0 2021-11-03T17:14:05.453 INFO:tasks.workunit.client.0.smithi027.stdout:1/217: mkdir d6/d11/d14/d1f/d37/d49 0 2021-11-03T17:14:05.453 INFO:tasks.workunit.client.0.smithi027.stdout:1/218: write d6/d11/d14/d1f/d25/f47 [523592,125744] 0 2021-11-03T17:14:05.454 INFO:tasks.workunit.client.0.smithi027.stdout:1/219: read d6/d11/d14/d1f/f24 [777108,9486] 0 2021-11-03T17:14:05.455 INFO:tasks.workunit.client.0.smithi027.stdout:1/220: chown d6/d11/d14/d1f/d37/d49 814976835 1 2021-11-03T17:14:05.455 INFO:tasks.workunit.client.0.smithi027.stdout:1/221: mknod d6/d11/d14/d1f/c4a 1 2021-11-03T17:14:05.455 INFO:tasks.workunit.client.0.smithi027.stdout:4/187: mkdir d6/db/d21/d41 0 2021-11-03T17:14:05.455 INFO:tasks.workunit.client.0.smithi027.stdout:0/171: dread d9/da/f39 [0,4194304] 0 2021-11-03T17:14:05.456 INFO:tasks.workunit.client.0.smithi027.stdout:0/172: chown d9/d12/f3b 43009 1 2021-11-03T17:14:05.457 INFO:tasks.workunit.client.0.smithi027.stdout:8/222: creat d7/d10/d21/f45 x:0 0 0 2021-11-03T17:14:05.457 INFO:tasks.workunit.client.0.smithi027.stdout:4/188: dread d6/db/f35 [0,4194304] 0 2021-11-03T17:14:05.457 INFO:tasks.workunit.client.0.smithi027.stdout:0/173: dread d9/f13 [0,4194304] 0 2021-11-03T17:14:05.457 INFO:tasks.workunit.client.0.smithi027.stdout:0/174: mknod d9/da/d31/c3d 1 2021-11-03T17:14:05.458 INFO:tasks.workunit.client.0.smithi027.stdout:5/185: rmdir d2/d1f 39 2021-11-03T17:14:05.458 INFO:tasks.workunit.client.0.smithi027.stdout:2/246: mkdir d0/d1b/d29/d54 0 2021-11-03T17:14:05.460 INFO:tasks.workunit.client.0.smithi027.stdout:9/199: unlink db/d1b/d23/d2c/f43 0 2021-11-03T17:14:05.465 INFO:tasks.workunit.client.0.smithi027.stdout:3/122: rename d1/f1a to d1/f22 0 2021-11-03T17:14:05.465 INFO:tasks.workunit.client.0.smithi027.stdout:3/123: stat d1/f14 0 2021-11-03T17:14:05.467 INFO:tasks.workunit.client.0.smithi027.stdout:7/192: write d0/d2d/d2f/f39 [184917,24270] 0 2021-11-03T17:14:05.467 INFO:tasks.workunit.client.0.smithi027.stdout:7/193: write d0/d2/d10/f1b [1057342,85967] 0 2021-11-03T17:14:05.469 INFO:tasks.workunit.client.0.smithi027.stdout:1/222: creat d6/de/d13/d15/f4b x:0 0 0 2021-11-03T17:14:05.473 INFO:tasks.workunit.client.0.smithi027.stdout:8/223: link l4 d7/d10/d3f/l46 0 2021-11-03T17:14:05.475 INFO:tasks.workunit.client.0.smithi027.stdout:4/189: unlink d6/d9/d28/f3c 0 2021-11-03T17:14:05.475 INFO:tasks.workunit.client.0.smithi027.stdout:4/190: truncate d6/d9/f11 5732575 0 2021-11-03T17:14:05.476 INFO:tasks.workunit.client.0.smithi027.stdout:4/191: write d6/d9/d16/d17/f31 [4219879,1809] 0 2021-11-03T17:14:05.476 INFO:tasks.workunit.client.0.smithi027.stdout:4/192: fdatasync d6/db/fc 0 2021-11-03T17:14:05.476 INFO:tasks.workunit.client.0.smithi027.stdout:4/193: chown d6/db/d21/f2c 21 1 2021-11-03T17:14:05.477 INFO:tasks.workunit.client.0.smithi027.stdout:2/247: dwrite d0/d1b/d1c/f33 [0,4194304] 0 2021-11-03T17:14:05.477 INFO:tasks.workunit.client.0.smithi027.stdout:2/248: fsync d0/d1b/d29/f2f 0 2021-11-03T17:14:05.478 INFO:tasks.workunit.client.0.smithi027.stdout:2/249: stat d0/d1b/d29/f2b 0 2021-11-03T17:14:05.483 INFO:tasks.workunit.client.0.smithi027.stdout:0/175: rmdir d9/da/d30/d33 0 2021-11-03T17:14:05.484 INFO:tasks.workunit.client.0.smithi027.stdout:5/186: creat d2/d4/f43 x:0 0 0 2021-11-03T17:14:05.485 INFO:tasks.workunit.client.0.smithi027.stdout:5/187: dread - d2/d4/df/d20/f3a zero size 2021-11-03T17:14:05.486 INFO:tasks.workunit.client.0.smithi027.stdout:7/194: creat d0/d2/d1c/f3c x:0 0 0 2021-11-03T17:14:05.487 INFO:tasks.workunit.client.0.smithi027.stdout:7/195: mknod d0/d2/d1c/c3d 1 2021-11-03T17:14:05.489 INFO:tasks.workunit.client.0.smithi027.stdout:9/200: dwrite db/d1b/d2b/f3f [0,4194304] 0 2021-11-03T17:14:05.489 INFO:tasks.workunit.client.0.smithi027.stdout:1/223: unlink d6/d11/d14/d1f/f24 0 2021-11-03T17:14:05.489 INFO:tasks.workunit.client.0.smithi027.stdout:1/224: fsync d6/de/ff 0 2021-11-03T17:14:05.490 INFO:tasks.workunit.client.0.smithi027.stdout:1/225: write d6/d11/d14/d1f/f22 [1768760,16264] 0 2021-11-03T17:14:05.490 INFO:tasks.workunit.client.0.smithi027.stdout:7/196: fdatasync d0/d2d/d2f/f39 0 2021-11-03T17:14:05.491 INFO:tasks.workunit.client.0.smithi027.stdout:8/224: rmdir d7/d10/d18 39 2021-11-03T17:14:05.492 INFO:tasks.workunit.client.0.smithi027.stdout:4/194: creat d6/d9/d16/d36/f42 x:0 0 0 2021-11-03T17:14:05.492 INFO:tasks.workunit.client.0.smithi027.stdout:3/124: dwrite d1/fa [0,4194304] 0 2021-11-03T17:14:05.493 INFO:tasks.workunit.client.0.smithi027.stdout:0/176: creat d9/da/d30/f3e x:0 0 0 2021-11-03T17:14:05.494 INFO:tasks.workunit.client.0.smithi027.stdout:5/188: creat d2/d12/f44 x:0 0 0 2021-11-03T17:14:05.499 INFO:tasks.workunit.client.0.smithi027.stdout:1/226: dread d6/fa [0,4194304] 0 2021-11-03T17:14:05.499 INFO:tasks.workunit.client.0.smithi027.stdout:1/227: fsync d6/de/f19 0 2021-11-03T17:14:05.500 INFO:tasks.workunit.client.0.smithi027.stdout:1/228: stat d6/d11/d14/d1f/d37/l3d 0 2021-11-03T17:14:05.500 INFO:tasks.workunit.client.0.smithi027.stdout:9/201: creat db/d1b/d2b/d41/f4d x:0 0 0 2021-11-03T17:14:05.500 INFO:tasks.workunit.client.0.smithi027.stdout:4/195: write d6/db/d21/f25 [1627953,9464] 0 2021-11-03T17:14:05.514 INFO:tasks.workunit.client.0.smithi027.stdout:7/197: symlink d0/d2/d10/d1e/l3e 0 2021-11-03T17:14:05.514 INFO:tasks.workunit.client.0.smithi027.stdout:7/198: chown d0/d2/d10/d35/l37 576831 1 2021-11-03T17:14:05.515 INFO:tasks.workunit.client.0.smithi027.stdout:7/199: mknod d0/d2/d10/d1e/c3f 1 2021-11-03T17:14:05.515 INFO:tasks.workunit.client.0.smithi027.stdout:8/225: dwrite d7/d2d/d38/f3d [0,4194304] 0 2021-11-03T17:14:05.516 INFO:tasks.workunit.client.0.smithi027.stdout:3/125: creat d1/f23 x:0 0 0 2021-11-03T17:14:05.517 INFO:tasks.workunit.client.0.smithi027.stdout:0/177: dwrite d9/da/d30/f3a [0,4194304] 0 2021-11-03T17:14:05.518 INFO:tasks.workunit.client.0.smithi027.stdout:5/189: mkdir d2/d45 0 2021-11-03T17:14:05.518 INFO:tasks.workunit.client.0.smithi027.stdout:5/190: truncate d2/d2c/d3c/f41 1014973 0 2021-11-03T17:14:05.519 INFO:tasks.workunit.client.0.smithi027.stdout:1/229: mkdir d6/d11/d14/d1f/d25/d28/d4c 0 2021-11-03T17:14:05.521 INFO:tasks.workunit.client.0.smithi027.stdout:5/191: dread d2/f1e [0,4194304] 0 2021-11-03T17:14:05.521 INFO:tasks.workunit.client.0.smithi027.stdout:5/192: mknod d2/d2c/c46 1 2021-11-03T17:14:05.521 INFO:tasks.workunit.client.0.smithi027.stdout:9/202: getdents db/d1b/d1c/d1d 0 2021-11-03T17:14:05.522 INFO:tasks.workunit.client.0.smithi027.stdout:9/203: read - db/d1b/d23/d2c/f4c zero size 2021-11-03T17:14:05.523 INFO:tasks.workunit.client.0.smithi027.stdout:4/196: rename d6/d9/l10 to d6/db/d21/d2b/d2d/d3e/l43 0 2021-11-03T17:14:05.524 INFO:tasks.workunit.client.0.smithi027.stdout:9/204: dread db/f13 [0,4194304] 0 2021-11-03T17:14:05.524 INFO:tasks.workunit.client.0.smithi027.stdout:7/200: dread d0/d2/d10/f19 [0,4194304] 0 2021-11-03T17:14:05.524 INFO:tasks.workunit.client.0.smithi027.stdout:4/197: write d6/d9/d16/f26 [979267,115682] 0 2021-11-03T17:14:05.529 INFO:tasks.workunit.client.0.smithi027.stdout:8/226: creat d7/d10/d3f/f47 x:0 0 0 2021-11-03T17:14:05.530 INFO:tasks.workunit.client.0.smithi027.stdout:3/126: symlink d1/db/l24 0 2021-11-03T17:14:05.535 INFO:tasks.workunit.client.0.smithi027.stdout:0/178: creat d9/da/d30/f3f x:0 0 0 2021-11-03T17:14:05.535 INFO:tasks.workunit.client.0.smithi027.stdout:0/179: dread - d9/da/d30/f3c zero size 2021-11-03T17:14:05.538 INFO:tasks.workunit.client.0.smithi027.stdout:3/127: dread d1/db/f1e [0,4194304] 0 2021-11-03T17:14:05.538 INFO:tasks.workunit.client.0.smithi027.stdout:3/128: read d1/db/f12 [213261,89134] 0 2021-11-03T17:14:05.542 INFO:tasks.workunit.client.0.smithi027.stdout:5/193: creat d2/d2c/f47 x:0 0 0 2021-11-03T17:14:05.542 INFO:tasks.workunit.client.0.smithi027.stdout:5/194: truncate d2/d2c/d3c/f41 1982984 0 2021-11-03T17:14:05.542 INFO:tasks.workunit.client.0.smithi027.stdout:5/195: truncate d2/d2c/f47 767919 0 2021-11-03T17:14:05.543 INFO:tasks.workunit.client.0.smithi027.stdout:5/196: stat d2/d2c 0 2021-11-03T17:14:05.544 INFO:tasks.workunit.client.0.smithi027.stdout:9/205: symlink db/d1b/d2b/l4e 0 2021-11-03T17:14:05.544 INFO:tasks.workunit.client.0.smithi027.stdout:9/206: write db/d1b/d1c/d1d/f21 [15128,127473] 0 2021-11-03T17:14:05.545 INFO:tasks.workunit.client.0.smithi027.stdout:7/201: unlink d0/d2/d10/d1e/l22 0 2021-11-03T17:14:05.548 INFO:tasks.workunit.client.0.smithi027.stdout:4/198: rename d6/db/d21/d2b/d3a to d6/db/d21/d41/d44 0 2021-11-03T17:14:05.549 INFO:tasks.workunit.client.0.smithi027.stdout:8/227: creat d7/d10/d18/f48 x:0 0 0 2021-11-03T17:14:05.550 INFO:tasks.workunit.client.0.smithi027.stdout:8/228: fdatasync d7/d10/d18/d1a/f1f 0 2021-11-03T17:14:05.550 INFO:tasks.workunit.client.0.smithi027.stdout:8/229: chown f6 3361653 1 2021-11-03T17:14:05.552 INFO:tasks.workunit.client.0.smithi027.stdout:3/129: unlink d1/f7 0 2021-11-03T17:14:05.553 INFO:tasks.workunit.client.0.smithi027.stdout:0/180: dwrite d9/da/f39 [0,4194304] 0 2021-11-03T17:14:05.553 INFO:tasks.workunit.client.0.smithi027.stdout:0/181: chown d9/f1a 1 1 2021-11-03T17:14:05.554 INFO:tasks.workunit.client.0.smithi027.stdout:1/230: symlink d6/d11/d14/d1f/l4d 0 2021-11-03T17:14:05.554 INFO:tasks.workunit.client.0.smithi027.stdout:0/182: write d9/f13 [408537,17252] 0 2021-11-03T17:14:05.555 INFO:tasks.workunit.client.0.smithi027.stdout:0/183: write d9/d12/f28 [4788777,51487] 0 2021-11-03T17:14:05.555 INFO:tasks.workunit.client.0.smithi027.stdout:0/184: chown d9/da/fe 1933417 1 2021-11-03T17:14:05.555 INFO:tasks.workunit.client.0.smithi027.stdout:0/185: mknod d9/da/d31/c40 1 2021-11-03T17:14:05.555 INFO:tasks.workunit.client.0.smithi027.stdout:0/186: readlink d9/da/l26 0 2021-11-03T17:14:05.556 INFO:tasks.workunit.client.0.smithi027.stdout:0/187: dread - d9/da/d30/f3f zero size 2021-11-03T17:14:05.556 INFO:tasks.workunit.client.0.smithi027.stdout:5/197: mkdir d2/d2c/d3c/d48 0 2021-11-03T17:14:05.557 INFO:tasks.workunit.client.0.smithi027.stdout:5/198: mknod d2/d2c/d3c/c49 1 2021-11-03T17:14:05.557 INFO:tasks.workunit.client.0.smithi027.stdout:5/199: stat d2/f1e 0 2021-11-03T17:14:05.558 INFO:tasks.workunit.client.0.smithi027.stdout:9/207: rename db/d1b/d1c/d1d/f3d to db/d1b/d1c/d1d/f4f 0 2021-11-03T17:14:05.558 INFO:tasks.workunit.client.0.smithi027.stdout:7/202: symlink d0/d2/d1f/l40 0 2021-11-03T17:14:05.560 INFO:tasks.workunit.client.0.smithi027.stdout:8/230: creat d7/d10/d18/f49 x:0 0 0 2021-11-03T17:14:05.561 INFO:tasks.workunit.client.0.smithi027.stdout:8/231: write f6 [4822567,113490] 0 2021-11-03T17:14:05.565 INFO:tasks.workunit.client.0.smithi027.stdout:0/188: mknod d9/da/c41 1 2021-11-03T17:14:05.565 INFO:tasks.workunit.client.0.smithi027.stdout:0/189: truncate d9/f2b 612705 0 2021-11-03T17:14:05.565 INFO:tasks.workunit.client.0.smithi027.stdout:9/208: dread db/f17 [0,4194304] 0 2021-11-03T17:14:05.566 INFO:tasks.workunit.client.0.smithi027.stdout:9/209: truncate db/d1b/d1c/d1d/f21 196859 0 2021-11-03T17:14:05.566 INFO:tasks.workunit.client.0.smithi027.stdout:5/200: link d2/d4/f34 d2/d4/df/f4a 0 2021-11-03T17:14:05.568 INFO:tasks.workunit.client.0.smithi027.stdout:4/199: dwrite d6/d9/f11 [0,4194304] 0 2021-11-03T17:14:05.569 INFO:tasks.workunit.client.0.smithi027.stdout:8/232: symlink d7/d2d/l4a 0 2021-11-03T17:14:05.570 INFO:tasks.workunit.client.0.smithi027.stdout:8/233: dread - d7/d10/d18/f48 zero size 2021-11-03T17:14:05.571 INFO:tasks.workunit.client.0.smithi027.stdout:7/203: write d0/f25 [279135,89253] 0 2021-11-03T17:14:05.572 INFO:tasks.workunit.client.0.smithi027.stdout:7/204: write d0/d2/d10/d1e/d20/d31/f32 [372072,51349] 0 2021-11-03T17:14:05.572 INFO:tasks.workunit.client.0.smithi027.stdout:7/205: mknod d0/d2d/d2f/c41 1 2021-11-03T17:14:05.573 INFO:tasks.workunit.client.0.smithi027.stdout:7/206: read d0/d2/fd [1015687,70800] 0 2021-11-03T17:14:05.573 INFO:tasks.workunit.client.0.smithi027.stdout:7/207: write d0/d2/d1c/f27 [988594,61543] 0 2021-11-03T17:14:05.573 INFO:tasks.workunit.client.0.smithi027.stdout:7/208: mknod d0/d2d/c42 1 2021-11-03T17:14:05.582 INFO:tasks.workunit.client.0.smithi027.stdout:3/130: getdents d1 0 2021-11-03T17:14:05.582 INFO:tasks.workunit.client.0.smithi027.stdout:3/131: chown d1/f23 55246769 1 2021-11-03T17:14:05.588 INFO:tasks.workunit.client.0.smithi027.stdout:8/234: dwrite d7/d2d/d38/f3d [0,4194304] 0 2021-11-03T17:14:05.597 INFO:tasks.workunit.client.0.smithi027.stdout:7/209: dwrite d0/d2/d1c/f27 [0,4194304] 0 2021-11-03T17:14:05.602 INFO:tasks.workunit.client.0.smithi027.stdout:7/210: dread d0/d2/f5 [0,4194304] 0 2021-11-03T17:14:05.608 INFO:tasks.workunit.client.0.smithi027.stdout:1/231: mkdir d6/de/d13/d15/d4e 0 2021-11-03T17:14:05.608 INFO:tasks.workunit.client.0.smithi027.stdout:1/232: chown d6/de/d13/d15/d43/f48 43541 1 2021-11-03T17:14:05.608 INFO:tasks.workunit.client.0.smithi027.stdout:1/233: chown d6/d11/f27 109 1 2021-11-03T17:14:05.609 INFO:tasks.workunit.client.0.smithi027.stdout:0/190: creat d9/f42 x:0 0 0 2021-11-03T17:14:05.616 INFO:tasks.workunit.client.0.smithi027.stdout:9/210: symlink db/d1b/d1c/l50 0 2021-11-03T17:14:05.618 INFO:tasks.workunit.client.0.smithi027.stdout:5/201: unlink d2/d12/f2a 0 2021-11-03T17:14:05.620 INFO:tasks.workunit.client.0.smithi027.stdout:4/200: creat d6/db/d21/d2b/f45 x:0 0 0 2021-11-03T17:14:05.624 INFO:tasks.workunit.client.0.smithi027.stdout:8/235: rmdir d7/d10/d18/d1a 39 2021-11-03T17:14:05.624 INFO:tasks.workunit.client.0.smithi027.stdout:9/211: dread db/f16 [0,4194304] 0 2021-11-03T17:14:05.627 INFO:tasks.workunit.client.0.smithi027.stdout:8/236: dread d7/f1b [0,4194304] 0 2021-11-03T17:14:05.628 INFO:tasks.workunit.client.0.smithi027.stdout:8/237: chown d7/f36 789 1 2021-11-03T17:14:05.629 INFO:tasks.workunit.client.0.smithi027.stdout:1/234: creat d6/d11/d14/d1f/d25/d28/d4c/f4f x:0 0 0 2021-11-03T17:14:05.629 INFO:tasks.workunit.client.0.smithi027.stdout:1/235: write d6/de/d13/d15/f42 [531985,117059] 0 2021-11-03T17:14:05.632 INFO:tasks.workunit.client.0.smithi027.stdout:0/191: rename d9/da/d30/f3a to d9/da/d30/f43 0 2021-11-03T17:14:05.636 INFO:tasks.workunit.client.0.smithi027.stdout:5/202: symlink d2/d2c/l4b 0 2021-11-03T17:14:05.638 INFO:tasks.workunit.client.0.smithi027.stdout:4/201: write d6/d9/d16/f32 [6169297,57499] 0 2021-11-03T17:14:05.640 INFO:tasks.workunit.client.0.smithi027.stdout:8/238: truncate d7/f8 532975 0 2021-11-03T17:14:05.645 INFO:tasks.workunit.client.0.smithi027.stdout:4/202: unlink d6/l22 0 2021-11-03T17:14:05.648 INFO:tasks.workunit.client.0.smithi027.stdout:4/203: dread d6/db/d21/f38 [0,4194304] 0 2021-11-03T17:14:05.650 INFO:tasks.workunit.client.0.smithi027.stdout:4/204: unlink d6/d9/d28/l30 0 2021-11-03T17:14:05.658 INFO:tasks.workunit.client.0.smithi027.stdout:0/192: dwrite d9/f1f [0,4194304] 0 2021-11-03T17:14:05.659 INFO:tasks.workunit.client.0.smithi027.stdout:5/203: dwrite d2/f3 [0,4194304] 0 2021-11-03T17:14:05.664 INFO:tasks.workunit.client.0.smithi027.stdout:4/205: dwrite d6/db/d21/f2c [0,4194304] 0 2021-11-03T17:14:05.667 INFO:tasks.workunit.client.0.smithi027.stdout:0/193: rmdir d9 39 2021-11-03T17:14:05.667 INFO:tasks.workunit.client.0.smithi027.stdout:0/194: chown d9/da/d30/f3f 11588068 1 2021-11-03T17:14:05.667 INFO:tasks.workunit.client.0.smithi027.stdout:0/195: fsync d9/da/f39 0 2021-11-03T17:14:05.669 INFO:tasks.workunit.client.0.smithi027.stdout:8/239: dread f6 [0,4194304] 0 2021-11-03T17:14:05.672 INFO:tasks.workunit.client.0.smithi027.stdout:8/240: write d7/d10/f16 [4022989,60857] 0 2021-11-03T17:14:05.673 INFO:tasks.workunit.client.0.smithi027.stdout:5/204: fdatasync d2/d4/df/f15 0 2021-11-03T17:14:05.677 INFO:tasks.workunit.client.0.smithi027.stdout:4/206: rename d6/f1f to d6/d24/f46 0 2021-11-03T17:14:05.679 INFO:tasks.workunit.client.0.smithi027.stdout:0/196: creat d9/da/d31/f44 x:0 0 0 2021-11-03T17:14:05.680 INFO:tasks.workunit.client.0.smithi027.stdout:8/241: symlink d7/d10/d18/d1a/l4b 0 2021-11-03T17:14:05.681 INFO:tasks.workunit.client.0.smithi027.stdout:8/242: write d7/d10/d18/d1a/f2e [905924,130217] 0 2021-11-03T17:14:05.681 INFO:tasks.workunit.client.0.smithi027.stdout:4/207: rmdir d6/db/d21/d41 39 2021-11-03T17:14:05.684 INFO:tasks.workunit.client.0.smithi027.stdout:0/197: unlink d9/f2e 0 2021-11-03T17:14:05.685 INFO:tasks.workunit.client.0.smithi027.stdout:5/205: dread d2/f18 [0,4194304] 0 2021-11-03T17:14:05.687 INFO:tasks.workunit.client.0.smithi027.stdout:5/206: symlink d2/d1f/l4c 0 2021-11-03T17:14:05.688 INFO:tasks.workunit.client.0.smithi027.stdout:4/208: write d6/db/f34 [2046649,94528] 0 2021-11-03T17:14:05.692 INFO:tasks.workunit.client.0.smithi027.stdout:4/209: rename d6/f8 to d6/db/d21/d2b/d2d/d3e/f47 0 2021-11-03T17:14:05.694 INFO:tasks.workunit.client.0.smithi027.stdout:4/210: creat d6/f48 x:0 0 0 2021-11-03T17:14:05.695 INFO:tasks.workunit.client.0.smithi027.stdout:4/211: symlink d6/db/d21/d2b/l49 0 2021-11-03T17:14:05.695 INFO:tasks.workunit.client.0.smithi027.stdout:4/212: chown d6/d9/d16 28513 1 2021-11-03T17:14:05.696 INFO:tasks.workunit.client.0.smithi027.stdout:4/213: creat d6/d9/d16/d36/f4a x:0 0 0 2021-11-03T17:14:05.697 INFO:tasks.workunit.client.0.smithi027.stdout:5/207: dread d2/f3e [0,4194304] 0 2021-11-03T17:14:05.698 INFO:tasks.workunit.client.0.smithi027.stdout:4/214: rmdir d6/d24 39 2021-11-03T17:14:05.698 INFO:tasks.workunit.client.0.smithi027.stdout:4/215: stat d6/db/fc 0 2021-11-03T17:14:05.700 INFO:tasks.workunit.client.0.smithi027.stdout:5/208: symlink d2/d4/df/d20/d25/l4d 0 2021-11-03T17:14:05.700 INFO:tasks.workunit.client.0.smithi027.stdout:4/216: getdents d6/db/d21/d41 0 2021-11-03T17:14:05.701 INFO:tasks.workunit.client.0.smithi027.stdout:5/209: dread - d2/d12/f44 zero size 2021-11-03T17:14:05.701 INFO:tasks.workunit.client.0.smithi027.stdout:5/210: fsync d2/d1f/f38 0 2021-11-03T17:14:05.702 INFO:tasks.workunit.client.0.smithi027.stdout:5/211: chown d2/d4/f13 9755 1 2021-11-03T17:14:05.702 INFO:tasks.workunit.client.0.smithi027.stdout:5/212: write d2/d4/df/f30 [1690977,66714] 0 2021-11-03T17:14:05.703 INFO:tasks.workunit.client.0.smithi027.stdout:4/217: rename d6/db/fc to d6/d9/d28/f4b 0 2021-11-03T17:14:05.703 INFO:tasks.workunit.client.0.smithi027.stdout:4/218: dread - d6/d9/d16/d36/f42 zero size 2021-11-03T17:14:05.706 INFO:tasks.workunit.client.0.smithi027.stdout:8/243: dwrite d7/f1d [0,4194304] 0 2021-11-03T17:14:05.711 INFO:tasks.workunit.client.0.smithi027.stdout:5/213: dread d2/d4/df/f15 [0,4194304] 0 2021-11-03T17:14:05.712 INFO:tasks.workunit.client.0.smithi027.stdout:0/198: dwrite d9/f42 [0,4194304] 0 2021-11-03T17:14:05.714 INFO:tasks.workunit.client.0.smithi027.stdout:4/219: write d6/d24/f46 [1847601,81264] 0 2021-11-03T17:14:05.727 INFO:tasks.workunit.client.0.smithi027.stdout:8/244: link d7/d10/d18/d1a/f26 d7/d10/d3f/f4c 0 2021-11-03T17:14:05.728 INFO:tasks.workunit.client.0.smithi027.stdout:8/245: mknod d7/d10/d21/c4d 1 2021-11-03T17:14:05.730 INFO:tasks.workunit.client.0.smithi027.stdout:5/214: mkdir d2/d4/d4e 0 2021-11-03T17:14:05.735 INFO:tasks.workunit.client.0.smithi027.stdout:0/199: symlink d9/d12/l45 0 2021-11-03T17:14:05.736 INFO:tasks.workunit.client.0.smithi027.stdout:0/200: read d9/f42 [3238174,99480] 0 2021-11-03T17:14:05.737 INFO:tasks.workunit.client.0.smithi027.stdout:4/220: unlink d6/d9/d28/f3b 0 2021-11-03T17:14:05.738 INFO:tasks.workunit.client.0.smithi027.stdout:8/246: creat d7/d10/d3f/f4e x:0 0 0 2021-11-03T17:14:05.743 INFO:tasks.workunit.client.0.smithi027.stdout:5/215: creat d2/d4/d4e/f4f x:0 0 0 2021-11-03T17:14:05.747 INFO:tasks.workunit.client.0.smithi027.stdout:4/221: rename d6/db/f34 to d6/d9/d16/d36/f4c 0 2021-11-03T17:14:05.752 INFO:tasks.workunit.client.0.smithi027.stdout:0/201: dwrite d9/da/d31/f44 [0,4194304] 0 2021-11-03T17:14:05.752 INFO:tasks.workunit.client.0.smithi027.stdout:0/202: chown d9/d12/f28 0 1 2021-11-03T17:14:05.753 INFO:tasks.workunit.client.0.smithi027.stdout:8/247: mkdir d7/d10/d4f 0 2021-11-03T17:14:05.755 INFO:tasks.workunit.client.0.smithi027.stdout:8/248: write d7/d2d/d38/f3d [3156422,14127] 0 2021-11-03T17:14:05.755 INFO:tasks.workunit.client.0.smithi027.stdout:8/249: chown d7/d10/d3f/l46 13 1 2021-11-03T17:14:05.755 INFO:tasks.workunit.client.0.smithi027.stdout:0/203: dread d9/da/fe [0,4194304] 0 2021-11-03T17:14:05.756 INFO:tasks.workunit.client.0.smithi027.stdout:0/204: truncate d9/d12/f3b 731737 0 2021-11-03T17:14:05.756 INFO:tasks.workunit.client.0.smithi027.stdout:0/205: fdatasync d9/f2a 0 2021-11-03T17:14:05.758 INFO:tasks.workunit.client.0.smithi027.stdout:0/206: dread d9/f13 [0,4194304] 0 2021-11-03T17:14:05.759 INFO:tasks.workunit.client.0.smithi027.stdout:0/207: chown d9/d12 600563573 1 2021-11-03T17:14:05.759 INFO:tasks.workunit.client.0.smithi027.stdout:0/208: dread - d9/da/f38 zero size 2021-11-03T17:14:05.759 INFO:tasks.workunit.client.0.smithi027.stdout:0/209: stat d9/da/d31 0 2021-11-03T17:14:05.760 INFO:tasks.workunit.client.0.smithi027.stdout:4/222: dread d6/d9/d16/f32 [0,4194304] 0 2021-11-03T17:14:05.761 INFO:tasks.workunit.client.0.smithi027.stdout:5/216: write d2/f42 [545024,90425] 0 2021-11-03T17:14:05.764 INFO:tasks.workunit.client.0.smithi027.stdout:8/250: symlink d7/l50 0 2021-11-03T17:14:05.765 INFO:tasks.workunit.client.0.smithi027.stdout:8/251: fdatasync d7/d10/d21/f45 0 2021-11-03T17:14:05.768 INFO:tasks.workunit.client.0.smithi027.stdout:8/252: symlink d7/d10/d21/l51 0 2021-11-03T17:14:05.771 INFO:tasks.workunit.client.0.smithi027.stdout:5/217: link d2/d4/df/d20/d25/l4d d2/d45/l50 0 2021-11-03T17:14:05.771 INFO:tasks.workunit.client.0.smithi027.stdout:5/218: chown d2/d4/df/f2e 1 1 2021-11-03T17:14:05.772 INFO:tasks.workunit.client.0.smithi027.stdout:8/253: symlink d7/d10/d3f/l52 0 2021-11-03T17:14:05.773 INFO:tasks.workunit.client.0.smithi027.stdout:8/254: rmdir d7/d10/d21/d41 39 2021-11-03T17:14:05.775 INFO:tasks.workunit.client.0.smithi027.stdout:8/255: mkdir d7/d53 0 2021-11-03T17:14:05.775 INFO:tasks.workunit.client.0.smithi027.stdout:8/256: write d7/d10/d18/d1a/f26 [236139,29095] 0 2021-11-03T17:14:05.776 INFO:tasks.workunit.client.0.smithi027.stdout:8/257: symlink d7/d10/d18/d1a/l54 0 2021-11-03T17:14:05.776 INFO:tasks.workunit.client.0.smithi027.stdout:5/219: write d2/d4/df/d20/d25/f27 [2328289,15160] 0 2021-11-03T17:14:05.777 INFO:tasks.workunit.client.0.smithi027.stdout:8/258: write d7/f1b [4313957,29249] 0 2021-11-03T17:14:05.778 INFO:tasks.workunit.client.0.smithi027.stdout:5/220: symlink d2/d2c/l51 0 2021-11-03T17:14:05.779 INFO:tasks.workunit.client.0.smithi027.stdout:5/221: creat d2/f52 x:0 0 0 2021-11-03T17:14:05.780 INFO:tasks.workunit.client.0.smithi027.stdout:4/223: dwrite d6/db/d21/f37 [0,4194304] 0 2021-11-03T17:14:05.781 INFO:tasks.workunit.client.0.smithi027.stdout:4/224: chown d6/f48 62013 1 2021-11-03T17:14:05.781 INFO:tasks.workunit.client.0.smithi027.stdout:4/225: fdatasync d6/db/d21/f25 0 2021-11-03T17:14:05.782 INFO:tasks.workunit.client.0.smithi027.stdout:5/222: symlink d2/d2c/d3c/d48/l53 0 2021-11-03T17:14:05.782 INFO:tasks.workunit.client.0.smithi027.stdout:5/223: chown d2/f52 31959624 1 2021-11-03T17:14:05.782 INFO:tasks.workunit.client.0.smithi027.stdout:0/210: dwrite d9/f42 [0,4194304] 0 2021-11-03T17:14:05.786 INFO:tasks.workunit.client.0.smithi027.stdout:0/211: getdents d9/da/d31 0 2021-11-03T17:14:05.788 INFO:tasks.workunit.client.0.smithi027.stdout:8/259: dread d7/d2d/d38/f44 [0,4194304] 0 2021-11-03T17:14:05.789 INFO:tasks.workunit.client.0.smithi027.stdout:5/224: truncate d2/f7 170038 0 2021-11-03T17:14:05.791 INFO:tasks.workunit.client.0.smithi027.stdout:8/260: rename d7/d2d/d40 to d7/d10/d3f/d55 0 2021-11-03T17:14:05.791 INFO:tasks.workunit.client.0.smithi027.stdout:8/261: fsync d7/f8 0 2021-11-03T17:14:05.791 INFO:tasks.workunit.client.0.smithi027.stdout:8/262: stat d7/d10/d18/d1a 0 2021-11-03T17:14:05.792 INFO:tasks.workunit.client.0.smithi027.stdout:5/225: symlink d2/d2c/d3c/l54 0 2021-11-03T17:14:05.792 INFO:tasks.workunit.client.0.smithi027.stdout:5/226: dread - d2/d2c/d3c/f35 zero size 2021-11-03T17:14:05.793 INFO:tasks.workunit.client.0.smithi027.stdout:5/227: write d2/d4/df/d20/f3a [650979,38849] 0 2021-11-03T17:14:05.793 INFO:tasks.workunit.client.0.smithi027.stdout:5/228: mknod d2/d4/df/c55 1 2021-11-03T17:14:05.793 INFO:tasks.workunit.client.0.smithi027.stdout:5/229: write d2/f52 [950510,8119] 0 2021-11-03T17:14:05.793 INFO:tasks.workunit.client.0.smithi027.stdout:5/230: readlink d2/d2c/l51 0 2021-11-03T17:14:05.794 INFO:tasks.workunit.client.0.smithi027.stdout:5/231: creat d2/d45/f56 x:0 0 0 2021-11-03T17:14:05.798 INFO:tasks.workunit.client.0.smithi027.stdout:0/212: dread d9/f42 [0,4194304] 0 2021-11-03T17:14:05.798 INFO:tasks.workunit.client.0.smithi027.stdout:0/213: write d9/da/f2d [1029889,102038] 0 2021-11-03T17:14:05.799 INFO:tasks.workunit.client.0.smithi027.stdout:4/226: dwrite d6/db/d21/f38 [0,4194304] 0 2021-11-03T17:14:05.800 INFO:tasks.workunit.client.0.smithi027.stdout:8/263: write d7/d10/d18/d1a/f2e [3432135,130826] 0 2021-11-03T17:14:05.800 INFO:tasks.workunit.client.0.smithi027.stdout:8/264: mknod d7/d10/d3f/d55/c56 1 2021-11-03T17:14:05.801 INFO:tasks.workunit.client.0.smithi027.stdout:8/265: chown d7/d10/d21/f25 10234 1 2021-11-03T17:14:05.801 INFO:tasks.workunit.client.0.smithi027.stdout:5/232: dread d2/d4/df/f15 [0,4194304] 0 2021-11-03T17:14:05.801 INFO:tasks.workunit.client.0.smithi027.stdout:8/266: mknod d7/d10/d21/d41/c57 1 2021-11-03T17:14:05.801 INFO:tasks.workunit.client.0.smithi027.stdout:8/267: write d7/d10/d18/d1a/f2e [4710801,82548] 0 2021-11-03T17:14:05.801 INFO:tasks.workunit.client.0.smithi027.stdout:4/227: write d6/db/f35 [275128,35140] 0 2021-11-03T17:14:05.802 INFO:tasks.workunit.client.0.smithi027.stdout:4/228: chown d6/db/d21/d2b/d2d 322489 1 2021-11-03T17:14:05.802 INFO:tasks.workunit.client.0.smithi027.stdout:5/233: dread d2/d4/f5 [0,4194304] 0 2021-11-03T17:14:05.803 INFO:tasks.workunit.client.0.smithi027.stdout:5/234: readlink d2/d2c/d3c/l54 0 2021-11-03T17:14:05.804 INFO:tasks.workunit.client.0.smithi027.stdout:0/214: mknod d9/da/c46 1 2021-11-03T17:14:05.808 INFO:tasks.workunit.client.0.smithi027.stdout:8/268: link d7/d2d/l29 d7/d2d/l58 0 2021-11-03T17:14:05.810 INFO:tasks.workunit.client.0.smithi027.stdout:4/229: symlink d6/db/l4d 0 2021-11-03T17:14:05.810 INFO:tasks.workunit.client.0.smithi027.stdout:4/230: fdatasync d6/d9/d16/f32 0 2021-11-03T17:14:05.811 INFO:tasks.workunit.client.0.smithi027.stdout:8/269: dread d7/d2d/d38/f44 [0,4194304] 0 2021-11-03T17:14:05.813 INFO:tasks.workunit.client.0.smithi027.stdout:5/235: rmdir d2/d4 39 2021-11-03T17:14:05.814 INFO:tasks.workunit.client.0.smithi027.stdout:0/215: truncate d9/f2b 1022445 0 2021-11-03T17:14:05.816 INFO:tasks.workunit.client.0.smithi027.stdout:5/236: symlink d2/d4/df/l57 0 2021-11-03T17:14:05.816 INFO:tasks.workunit.client.0.smithi027.stdout:5/237: fdatasync d2/d4/f13 0 2021-11-03T17:14:05.817 INFO:tasks.workunit.client.0.smithi027.stdout:0/216: fsync d9/da/f2d 0 2021-11-03T17:14:05.826 INFO:tasks.workunit.client.0.smithi027.stdout:5/238: mkdir d2/d2c/d3c/d58 0 2021-11-03T17:14:05.826 INFO:tasks.workunit.client.0.smithi027.stdout:5/239: chown d2/d2c/f47 12 1 2021-11-03T17:14:05.830 INFO:tasks.workunit.client.0.smithi027.stdout:0/217: rename d9/da/d30/f3f to d9/f47 0 2021-11-03T17:14:05.830 INFO:tasks.workunit.client.0.smithi027.stdout:0/218: write d9/da/d31/f36 [1000428,10905] 0 2021-11-03T17:14:05.830 INFO:tasks.workunit.client.0.smithi027.stdout:0/219: fdatasync d9/da/f38 0 2021-11-03T17:14:05.832 INFO:tasks.workunit.client.0.smithi027.stdout:0/220: creat d9/da/d31/f48 x:0 0 0 2021-11-03T17:14:05.832 INFO:tasks.workunit.client.0.smithi027.stdout:0/221: fdatasync d9/f47 0 2021-11-03T17:14:05.834 INFO:tasks.workunit.client.0.smithi027.stdout:0/222: creat d9/f49 x:0 0 0 2021-11-03T17:14:05.836 INFO:tasks.workunit.client.0.smithi027.stdout:0/223: symlink d9/l4a 0 2021-11-03T17:14:05.840 INFO:tasks.workunit.client.0.smithi027.stdout:0/224: dread d9/da/f10 [4194304,4194304] 0 2021-11-03T17:14:05.844 INFO:tasks.workunit.client.0.smithi027.stdout:0/225: creat d9/f4b x:0 0 0 2021-11-03T17:14:05.848 INFO:tasks.workunit.client.0.smithi027.stdout:5/240: dwrite d2/d4/f34 [0,4194304] 0 2021-11-03T17:14:05.862 INFO:tasks.workunit.client.0.smithi027.stdout:0/226: dwrite d9/d12/f17 [0,4194304] 0 2021-11-03T17:14:05.864 INFO:tasks.workunit.client.0.smithi027.stdout:0/227: rename d9/d12/l1c to d9/d12/l4c 0 2021-11-03T17:14:05.868 INFO:tasks.workunit.client.0.smithi027.stdout:5/241: dwrite d2/d4/f43 [0,4194304] 0 2021-11-03T17:14:05.869 INFO:tasks.workunit.client.0.smithi027.stdout:5/242: fsync d2/d45/f56 0 2021-11-03T17:14:05.871 INFO:tasks.workunit.client.0.smithi027.stdout:5/243: symlink d2/d2c/d3c/d48/l59 0 2021-11-03T17:14:05.872 INFO:tasks.workunit.client.0.smithi027.stdout:5/244: rename d2/d4/df/l3f to d2/d4/df/d20/l5a 0 2021-11-03T17:14:05.885 INFO:tasks.workunit.client.0.smithi027.stdout:0/228: dwrite d9/da/d31/f36 [0,4194304] 0 2021-11-03T17:14:05.889 INFO:tasks.workunit.client.0.smithi027.stdout:5/245: dwrite d2/d4/f13 [0,4194304] 0 2021-11-03T17:14:05.889 INFO:tasks.workunit.client.0.smithi027.stdout:5/246: chown d2/d4/df/d20/d25/l4d 449249 1 2021-11-03T17:14:05.889 INFO:tasks.workunit.client.0.smithi027.stdout:0/229: creat d9/f4d x:0 0 0 2021-11-03T17:14:05.890 INFO:tasks.workunit.client.0.smithi027.stdout:0/230: chown d9/f47 73 1 2021-11-03T17:14:05.890 INFO:tasks.workunit.client.0.smithi027.stdout:0/231: mknod d9/c4e 1 2021-11-03T17:14:05.906 INFO:tasks.workunit.client.0.smithi027.stdout:0/232: dwrite d9/d12/f1d [0,4194304] 0 2021-11-03T17:14:05.908 INFO:tasks.workunit.client.0.smithi027.stdout:0/233: dread d9/da/fe [0,4194304] 0 2021-11-03T17:14:05.909 INFO:tasks.workunit.client.0.smithi027.stdout:0/234: creat d9/f4f x:0 0 0 2021-11-03T17:14:05.910 INFO:tasks.workunit.client.0.smithi027.stdout:0/235: write d9/da/d31/f48 [422970,57230] 0 2021-11-03T17:14:05.910 INFO:tasks.workunit.client.0.smithi027.stdout:0/236: stat d9/da/d30/f3c 0 2021-11-03T17:14:05.911 INFO:tasks.workunit.client.0.smithi027.stdout:0/237: symlink d9/da/d31/l50 0 2021-11-03T17:14:05.911 INFO:tasks.workunit.client.0.smithi027.stdout:0/238: rename d9/da/d30 to d9/da/d30/d51 22 2021-11-03T17:14:05.912 INFO:tasks.workunit.client.0.smithi027.stdout:0/239: getdents d9/da 0 2021-11-03T17:14:05.913 INFO:tasks.workunit.client.0.smithi027.stdout:5/247: dwrite d2/d4/f34 [0,4194304] 0 2021-11-03T17:14:05.914 INFO:tasks.workunit.client.0.smithi027.stdout:5/248: rmdir d2/d4/df/d20 39 2021-11-03T17:14:05.915 INFO:tasks.workunit.client.0.smithi027.stdout:5/249: mknod d2/d2c/c5b 1 2021-11-03T17:14:05.916 INFO:tasks.workunit.client.0.smithi027.stdout:5/250: truncate d2/f36 672599 0 2021-11-03T17:14:05.916 INFO:tasks.workunit.client.0.smithi027.stdout:5/251: read - d2/d45/f56 zero size 2021-11-03T17:14:05.917 INFO:tasks.workunit.client.0.smithi027.stdout:5/252: creat d2/d12/f5c x:0 0 0 2021-11-03T17:14:05.917 INFO:tasks.workunit.client.0.smithi027.stdout:5/253: truncate d2/f42 707616 0 2021-11-03T17:14:05.918 INFO:tasks.workunit.client.0.smithi027.stdout:5/254: symlink d2/d2c/d3c/l5d 0 2021-11-03T17:14:05.918 INFO:tasks.workunit.client.0.smithi027.stdout:5/255: fdatasync d2/d12/f32 0 2021-11-03T17:14:05.919 INFO:tasks.workunit.client.0.smithi027.stdout:5/256: truncate d2/d4/df/d20/f3a 325929 0 2021-11-03T17:14:05.928 INFO:tasks.workunit.client.0.smithi027.stdout:0/240: dwrite d9/da/f2d [0,4194304] 0 2021-11-03T17:14:05.931 INFO:tasks.workunit.client.0.smithi027.stdout:5/257: dread d2/d4/f13 [0,4194304] 0 2021-11-03T17:14:05.934 INFO:tasks.workunit.client.0.smithi027.stdout:0/241: truncate d9/da/d30/f43 545204 0 2021-11-03T17:14:05.936 INFO:tasks.workunit.client.0.smithi027.stdout:5/258: truncate d2/d4/f5 2037938 0 2021-11-03T17:14:05.936 INFO:tasks.workunit.client.0.smithi027.stdout:5/259: fsync d2/d4/df/f11 0 2021-11-03T17:14:05.938 INFO:tasks.workunit.client.0.smithi027.stdout:0/242: rename d9/d12/l4c to d9/l52 0 2021-11-03T17:14:05.938 INFO:tasks.workunit.client.0.smithi027.stdout:0/243: fdatasync d9/f4b 0 2021-11-03T17:14:05.940 INFO:tasks.workunit.client.0.smithi027.stdout:5/260: getdents d2/d4/df/d20/d25 0 2021-11-03T17:14:05.946 INFO:tasks.workunit.client.0.smithi027.stdout:5/261: write d2/f3 [268679,38816] 0 2021-11-03T17:14:05.947 INFO:tasks.workunit.client.0.smithi027.stdout:5/262: dread d2/d2c/d3c/f41 [0,4194304] 0 2021-11-03T17:14:05.947 INFO:tasks.workunit.client.0.smithi027.stdout:5/263: read d2/f36 [366737,87955] 0 2021-11-03T17:14:05.948 INFO:tasks.workunit.client.0.smithi027.stdout:0/244: write d9/f2b [1140665,8691] 0 2021-11-03T17:14:05.949 INFO:tasks.workunit.client.0.smithi027.stdout:0/245: write d9/f2a [3091574,109758] 0 2021-11-03T17:14:05.951 INFO:tasks.workunit.client.0.smithi027.stdout:0/246: write d9/d12/f1d [2532354,21183] 0 2021-11-03T17:14:05.954 INFO:tasks.workunit.client.0.smithi027.stdout:5/264: dread d2/d4/fe [0,4194304] 0 2021-11-03T17:14:05.973 INFO:tasks.workunit.client.0.smithi027.stdout:0/247: dwrite d9/da/d31/f34 [0,4194304] 0 2021-11-03T17:14:05.973 INFO:tasks.workunit.client.0.smithi027.stdout:0/248: mknod d9/da/c53 1 2021-11-03T17:14:05.973 INFO:tasks.workunit.client.0.smithi027.stdout:0/249: truncate d9/da/f16 500495 0 2021-11-03T17:14:05.976 INFO:tasks.workunit.client.0.smithi027.stdout:5/265: dwrite d2/f18 [0,4194304] 0 2021-11-03T17:14:05.976 INFO:tasks.workunit.client.0.smithi027.stdout:5/266: chown d2/f52 24744 1 2021-11-03T17:14:05.976 INFO:tasks.workunit.client.0.smithi027.stdout:5/267: fsync d2/f3e 0 2021-11-03T17:14:05.978 INFO:tasks.workunit.client.0.smithi027.stdout:0/250: creat d9/da/d30/f54 x:0 0 0 2021-11-03T17:14:05.980 INFO:tasks.workunit.client.0.smithi027.stdout:0/251: fdatasync d9/f2a 0 2021-11-03T17:14:05.982 INFO:tasks.workunit.client.0.smithi027.stdout:5/268: write d2/f1e [1051040,64794] 0 2021-11-03T17:14:05.982 INFO:tasks.workunit.client.0.smithi027.stdout:5/269: write d2/d2c/f47 [1292921,125995] 0 2021-11-03T17:14:05.983 INFO:tasks.workunit.client.0.smithi027.stdout:5/270: truncate d2/d2c/f47 1749501 0 2021-11-03T17:14:05.983 INFO:tasks.workunit.client.0.smithi027.stdout:5/271: chown d2/d1f/f38 198433995 1 2021-11-03T17:14:05.985 INFO:tasks.workunit.client.0.smithi027.stdout:5/272: creat d2/d2c/d3c/d58/f5e x:0 0 0 2021-11-03T17:14:05.985 INFO:tasks.workunit.client.0.smithi027.stdout:5/273: readlink d2/d4/df/l57 0 2021-11-03T17:14:05.987 INFO:tasks.workunit.client.0.smithi027.stdout:5/274: write d2/d4/f43 [1561890,83738] 0 2021-11-03T17:14:05.987 INFO:tasks.workunit.client.0.smithi027.stdout:5/275: dread - d2/d4/d4e/f4f zero size 2021-11-03T17:14:05.989 INFO:tasks.workunit.client.0.smithi027.stdout:5/276: rename d2/d4/df/d20/d25/l2b to d2/d4/l5f 0 2021-11-03T17:14:05.989 INFO:tasks.workunit.client.0.smithi027.stdout:5/277: mknod d2/d2c/d3c/c60 1 2021-11-03T17:14:05.990 INFO:tasks.workunit.client.0.smithi027.stdout:5/278: mkdir d2/d4/d61 0 2021-11-03T17:14:05.991 INFO:tasks.workunit.client.0.smithi027.stdout:5/279: mknod d2/c62 1 2021-11-03T17:14:05.996 INFO:tasks.workunit.client.0.smithi027.stdout:0/252: dwrite d9/f1a [0,4194304] 0 2021-11-03T17:14:05.998 INFO:tasks.workunit.client.0.smithi027.stdout:0/253: symlink d9/d12/l55 0 2021-11-03T17:14:05.999 INFO:tasks.workunit.client.0.smithi027.stdout:0/254: mkdir d9/d12/d56 0 2021-11-03T17:14:06.027 INFO:tasks.workunit.client.0.smithi027.stdout:5/280: dwrite d2/d4/f43 [0,4194304] 0 2021-11-03T17:14:06.027 INFO:tasks.workunit.client.0.smithi027.stdout:0/255: dwrite d9/f4d [0,4194304] 0 2021-11-03T17:14:06.034 INFO:tasks.workunit.client.0.smithi027.stdout:5/281: symlink d2/d45/l63 0 2021-11-03T17:14:06.037 INFO:tasks.workunit.client.0.smithi027.stdout:0/256: rmdir d9/da/d31 39 2021-11-03T17:14:06.037 INFO:tasks.workunit.client.0.smithi027.stdout:0/257: getdents d9/da 0 2021-11-03T17:14:06.038 INFO:tasks.workunit.client.0.smithi027.stdout:0/258: write d9/da/d30/f3c [556657,59714] 0 2021-11-03T17:14:06.038 INFO:tasks.workunit.client.0.smithi027.stdout:0/259: mknod d9/da/d30/c57 1 2021-11-03T17:14:06.039 INFO:tasks.workunit.client.0.smithi027.stdout:5/282: link d2/d4/df/l57 d2/d4/df/d20/d25/l64 0 2021-11-03T17:14:06.041 INFO:tasks.workunit.client.0.smithi027.stdout:5/283: read d2/d4/df/d20/d25/f27 [48259,60472] 0 2021-11-03T17:14:06.041 INFO:tasks.workunit.client.0.smithi027.stdout:5/284: readlink d2/d4/df/d20/l5a 0 2021-11-03T17:14:06.052 INFO:tasks.workunit.client.0.smithi027.stdout:0/260: dwrite d9/da/f16 [0,4194304] 0 2021-11-03T17:14:06.053 INFO:tasks.workunit.client.0.smithi027.stdout:0/261: write d9/f42 [274935,82060] 0 2021-11-03T17:14:06.062 INFO:tasks.workunit.client.0.smithi027.stdout:5/285: dwrite d2/f18 [4194304,4194304] 0 2021-11-03T17:14:06.063 INFO:tasks.workunit.client.0.smithi027.stdout:5/286: chown d2/d45/f56 56294879 1 2021-11-03T17:14:06.067 INFO:tasks.workunit.client.0.smithi027.stdout:5/287: dread d2/d4/df/f30 [0,4194304] 0 2021-11-03T17:14:06.268 INFO:tasks.workunit.client.1.smithi115.stdout:0/196: sync 2021-11-03T17:14:06.268 INFO:tasks.workunit.client.1.smithi115.stdout:5/215: sync 2021-11-03T17:14:06.269 INFO:tasks.workunit.client.1.smithi115.stdout:0/197: write d9/f25 [298575,123697] 0 2021-11-03T17:14:06.275 INFO:tasks.workunit.client.1.smithi115.stdout:5/216: dread fe [0,4194304] 0 2021-11-03T17:14:06.276 INFO:tasks.workunit.client.1.smithi115.stdout:5/217: truncate d1a/f3c 729538 0 2021-11-03T17:14:06.276 INFO:tasks.workunit.client.1.smithi115.stdout:5/218: read d1a/d28/d34/f2a [678047,43824] 0 2021-11-03T17:14:06.277 INFO:tasks.workunit.client.1.smithi115.stdout:5/219: creat d1a/d3d/f47 x:0 0 0 2021-11-03T17:14:06.279 INFO:tasks.workunit.client.1.smithi115.stdout:5/220: mkdir d1a/d28/d44/d48 0 2021-11-03T17:14:06.279 INFO:tasks.workunit.client.1.smithi115.stdout:5/221: mknod d1a/d3d/c49 1 2021-11-03T17:14:06.281 INFO:tasks.workunit.client.1.smithi115.stdout:5/222: mkdir d1a/d28/d4a 0 2021-11-03T17:14:06.282 INFO:tasks.workunit.client.1.smithi115.stdout:5/223: truncate d1a/f1e 1357141 0 2021-11-03T17:14:06.284 INFO:tasks.workunit.client.1.smithi115.stdout:5/224: rename d1a/d28/d34/l39 to d1a/d28/l4b 0 2021-11-03T17:14:06.284 INFO:tasks.workunit.client.1.smithi115.stdout:5/225: mkdir d1a/d28/d4c 0 2021-11-03T17:14:06.290 INFO:tasks.workunit.client.1.smithi115.stdout:5/226: unlink d1a/f1e 0 2021-11-03T17:14:06.292 INFO:tasks.workunit.client.1.smithi115.stdout:4/299: sync 2021-11-03T17:14:06.292 INFO:tasks.workunit.client.1.smithi115.stdout:8/209: sync 2021-11-03T17:14:06.292 INFO:tasks.workunit.client.1.smithi115.stdout:6/156: sync 2021-11-03T17:14:06.293 INFO:tasks.workunit.client.1.smithi115.stdout:9/97: sync 2021-11-03T17:14:06.293 INFO:tasks.workunit.client.1.smithi115.stdout:1/199: sync 2021-11-03T17:14:06.293 INFO:tasks.workunit.client.1.smithi115.stdout:8/210: write d4/f27 [541108,116038] 0 2021-11-03T17:14:06.293 INFO:tasks.workunit.client.1.smithi115.stdout:1/200: write d7/dc/dd/d24/f28 [926234,84107] 0 2021-11-03T17:14:06.293 INFO:tasks.workunit.client.1.smithi115.stdout:1/201: chown d7/dc/d1a/l25 1787161523 1 2021-11-03T17:14:06.294 INFO:tasks.workunit.client.1.smithi115.stdout:1/202: stat d7/dc/d2a/d2d/f3c 0 2021-11-03T17:14:06.294 INFO:tasks.workunit.client.1.smithi115.stdout:7/154: sync 2021-11-03T17:14:06.294 INFO:tasks.workunit.client.1.smithi115.stdout:3/188: sync 2021-11-03T17:14:06.295 INFO:tasks.workunit.client.1.smithi115.stdout:5/227: link d1a/d28/l43 d1a/d28/l4d 0 2021-11-03T17:14:06.295 INFO:tasks.workunit.client.1.smithi115.stdout:5/228: chown d1a/d28/l43 0 1 2021-11-03T17:14:06.295 INFO:tasks.workunit.client.1.smithi115.stdout:5/229: truncate d1a/d3d/f42 722533 0 2021-11-03T17:14:06.296 INFO:tasks.workunit.client.1.smithi115.stdout:4/300: truncate d7/dc/d1a/f2e 1596890 0 2021-11-03T17:14:06.296 INFO:tasks.workunit.client.1.smithi115.stdout:4/301: chown d7/dc/d1a/d21/d47/d56/d42/d5f/f64 201 1 2021-11-03T17:14:06.296 INFO:tasks.workunit.client.1.smithi115.stdout:9/98: truncate d1/d5/f8 854392 0 2021-11-03T17:14:06.297 INFO:tasks.workunit.client.1.smithi115.stdout:7/155: write d3/f1e [3133876,33861] 0 2021-11-03T17:14:06.299 INFO:tasks.workunit.client.1.smithi115.stdout:4/302: read d7/dc/f12 [362294,50429] 0 2021-11-03T17:14:06.299 INFO:tasks.workunit.client.1.smithi115.stdout:0/198: dwrite d9/f25 [4194304,4194304] 0 2021-11-03T17:14:06.299 INFO:tasks.workunit.client.1.smithi115.stdout:0/199: chown d9/db/f20 713762899 1 2021-11-03T17:14:06.299 INFO:tasks.workunit.client.1.smithi115.stdout:0/200: chown d9/d15/d1a/d1d/f22 165 1 2021-11-03T17:14:06.300 INFO:tasks.workunit.client.1.smithi115.stdout:6/157: creat d0/d1/d10/d13/d1c/d1e/f32 x:0 0 0 2021-11-03T17:14:06.304 INFO:tasks.workunit.client.1.smithi115.stdout:7/156: getdents d3/da/d18 0 2021-11-03T17:14:06.307 INFO:tasks.workunit.client.1.smithi115.stdout:0/201: link f8 d9/db/df/f49 0 2021-11-03T17:14:06.307 INFO:tasks.workunit.client.1.smithi115.stdout:0/202: write d9/db/df/f49 [4548508,105672] 0 2021-11-03T17:14:06.309 INFO:tasks.workunit.client.1.smithi115.stdout:9/99: dread d1/d18/f1c [0,4194304] 0 2021-11-03T17:14:06.309 INFO:tasks.workunit.client.1.smithi115.stdout:4/303: dread d7/dc/d1a/d21/d47/d56/d63/f6e [0,4194304] 0 2021-11-03T17:14:06.310 INFO:tasks.workunit.client.1.smithi115.stdout:4/304: fdatasync d7/dc/d6a/f46 0 2021-11-03T17:14:06.310 INFO:tasks.workunit.client.1.smithi115.stdout:6/158: rename d0/d1/d10/l2c to d0/d1/d10/d13/l33 0 2021-11-03T17:14:06.310 INFO:tasks.workunit.client.1.smithi115.stdout:8/211: symlink d4/d6/d3f/d43/l46 0 2021-11-03T17:14:06.311 INFO:tasks.workunit.client.1.smithi115.stdout:7/157: creat d3/da/f2b x:0 0 0 2021-11-03T17:14:06.312 INFO:tasks.workunit.client.1.smithi115.stdout:7/158: write d3/f25 [365479,91231] 0 2021-11-03T17:14:06.312 INFO:tasks.workunit.client.1.smithi115.stdout:7/159: write d3/f25 [975132,65084] 0 2021-11-03T17:14:06.313 INFO:tasks.workunit.client.1.smithi115.stdout:3/189: dwrite d13/d20/f2d [0,4194304] 0 2021-11-03T17:14:06.313 INFO:tasks.workunit.client.1.smithi115.stdout:3/190: dread - d13/d35/f36 zero size 2021-11-03T17:14:06.314 INFO:tasks.workunit.client.1.smithi115.stdout:9/100: write d1/f9 [366354,36341] 0 2021-11-03T17:14:06.314 INFO:tasks.workunit.client.1.smithi115.stdout:4/305: read d7/dc/d6a/f3a [2635969,101962] 0 2021-11-03T17:14:06.315 INFO:tasks.workunit.client.1.smithi115.stdout:6/159: creat d0/d1/d10/d13/d1c/f34 x:0 0 0 2021-11-03T17:14:06.315 INFO:tasks.workunit.client.1.smithi115.stdout:1/203: dwrite d7/dc/d1a/f3d [0,4194304] 0 2021-11-03T17:14:06.316 INFO:tasks.workunit.client.1.smithi115.stdout:5/230: getdents d1a/d28 0 2021-11-03T17:14:06.320 INFO:tasks.workunit.client.1.smithi115.stdout:8/212: mkdir d4/d6/d18/d44/d47 0 2021-11-03T17:14:06.321 INFO:tasks.workunit.client.1.smithi115.stdout:7/160: creat d3/da/d18/f2c x:0 0 0 2021-11-03T17:14:06.322 INFO:tasks.workunit.client.1.smithi115.stdout:7/161: write d3/da/d18/f22 [943859,35443] 0 2021-11-03T17:14:06.328 INFO:tasks.workunit.client.1.smithi115.stdout:3/191: rename d13/f2b to d13/d35/f3d 0 2021-11-03T17:14:06.328 INFO:tasks.workunit.client.1.smithi115.stdout:3/192: mknod d13/d3b/d3c/c3e 1 2021-11-03T17:14:06.330 INFO:tasks.workunit.client.1.smithi115.stdout:9/101: dread d1/f13 [0,4194304] 0 2021-11-03T17:14:06.330 INFO:tasks.workunit.client.1.smithi115.stdout:9/102: truncate d1/f1d 160970 0 2021-11-03T17:14:06.330 INFO:tasks.workunit.client.1.smithi115.stdout:9/103: fsync d1/f13 0 2021-11-03T17:14:06.330 INFO:tasks.workunit.client.1.smithi115.stdout:9/104: fdatasync d1/d5/f12 0 2021-11-03T17:14:06.334 INFO:tasks.workunit.client.1.smithi115.stdout:4/306: creat d7/dc/d6a/d72/f76 x:0 0 0 2021-11-03T17:14:06.335 INFO:tasks.workunit.client.1.smithi115.stdout:4/307: mknod d7/dc/d6a/d3c/c77 1 2021-11-03T17:14:06.337 INFO:tasks.workunit.client.1.smithi115.stdout:6/160: rmdir d0/d1/d10/d13/d1c/d1e 39 2021-11-03T17:14:06.339 INFO:tasks.workunit.client.1.smithi115.stdout:5/231: symlink d1a/d28/d34/l4e 0 2021-11-03T17:14:06.339 INFO:tasks.workunit.client.1.smithi115.stdout:5/232: chown d1a/d28/d34/l4e 0 1 2021-11-03T17:14:06.339 INFO:tasks.workunit.client.1.smithi115.stdout:7/162: symlink d3/da/d18/l2d 0 2021-11-03T17:14:06.341 INFO:tasks.workunit.client.1.smithi115.stdout:9/105: creat d1/d18/f1e x:0 0 0 2021-11-03T17:14:06.342 INFO:tasks.workunit.client.1.smithi115.stdout:4/308: symlink d7/dc/d1a/d21/d47/d56/l78 0 2021-11-03T17:14:06.342 INFO:tasks.workunit.client.1.smithi115.stdout:4/309: chown d7/dc/l34 47404 1 2021-11-03T17:14:06.343 INFO:tasks.workunit.client.1.smithi115.stdout:6/161: creat d0/d1/d2/f35 x:0 0 0 2021-11-03T17:14:06.343 INFO:tasks.workunit.client.1.smithi115.stdout:6/162: write d0/d1/d10/d13/d1c/f2f [580798,30572] 0 2021-11-03T17:14:06.344 INFO:tasks.workunit.client.1.smithi115.stdout:5/233: mkdir d1a/d28/d44/d4f 0 2021-11-03T17:14:06.344 INFO:tasks.workunit.client.1.smithi115.stdout:5/234: mknod d1a/d28/d44/c50 1 2021-11-03T17:14:06.345 INFO:tasks.workunit.client.1.smithi115.stdout:5/235: mkdir d1a/d51 0 2021-11-03T17:14:06.348 INFO:tasks.workunit.client.1.smithi115.stdout:4/310: write d7/dc/d1a/d21/d47/d56/d63/f67 [6969753,81065] 0 2021-11-03T17:14:06.349 INFO:tasks.workunit.client.1.smithi115.stdout:4/311: write d7/dc/d1a/d21/d47/d56/d42/d5f/f64 [902527,80915] 0 2021-11-03T17:14:06.349 INFO:tasks.workunit.client.1.smithi115.stdout:4/312: write d7/dc/d6a/d72/f76 [253567,86809] 0 2021-11-03T17:14:06.349 INFO:tasks.workunit.client.1.smithi115.stdout:4/313: dread - d7/dc/d1a/d21/d47/f69 zero size 2021-11-03T17:14:06.351 INFO:tasks.workunit.client.1.smithi115.stdout:5/236: dread d1a/d28/f29 [0,4194304] 0 2021-11-03T17:14:06.353 INFO:tasks.workunit.client.1.smithi115.stdout:3/193: dwrite d13/d20/d2e/f2f [0,4194304] 0 2021-11-03T17:14:06.353 INFO:tasks.workunit.client.1.smithi115.stdout:3/194: write d13/f1f [759106,78210] 0 2021-11-03T17:14:06.353 INFO:tasks.workunit.client.1.smithi115.stdout:3/195: mknod d13/d2a/c3f 1 2021-11-03T17:14:06.354 INFO:tasks.workunit.client.1.smithi115.stdout:3/196: getdents d13/d2c 0 2021-11-03T17:14:06.354 INFO:tasks.workunit.client.1.smithi115.stdout:0/203: dwrite f8 [0,4194304] 0 2021-11-03T17:14:06.354 INFO:tasks.workunit.client.1.smithi115.stdout:0/204: chown l5 18619 1 2021-11-03T17:14:06.354 INFO:tasks.workunit.client.1.smithi115.stdout:5/237: dread d1a/d3d/f42 [0,4194304] 0 2021-11-03T17:14:06.355 INFO:tasks.workunit.client.1.smithi115.stdout:1/204: dwrite d7/dc/dd/d24/f2b [0,4194304] 0 2021-11-03T17:14:06.355 INFO:tasks.workunit.client.1.smithi115.stdout:8/213: dwrite d4/d6/d18/f3d [4194304,4194304] 0 2021-11-03T17:14:06.356 INFO:tasks.workunit.client.1.smithi115.stdout:0/205: creat d9/db/f4a x:0 0 0 2021-11-03T17:14:06.358 INFO:tasks.workunit.client.1.smithi115.stdout:5/238: mkdir d1a/d23/d52 0 2021-11-03T17:14:06.359 INFO:tasks.workunit.client.1.smithi115.stdout:9/106: dread d1/d5/f6 [0,4194304] 0 2021-11-03T17:14:06.361 INFO:tasks.workunit.client.1.smithi115.stdout:1/205: symlink d7/dc/dd/l3f 0 2021-11-03T17:14:06.363 INFO:tasks.workunit.client.1.smithi115.stdout:0/206: link d9/f43 d9/d15/d1a/d1d/f4b 0 2021-11-03T17:14:06.363 INFO:tasks.workunit.client.1.smithi115.stdout:9/107: truncate d1/d5/f8 412436 0 2021-11-03T17:14:06.364 INFO:tasks.workunit.client.1.smithi115.stdout:1/206: rmdir d7/dc/dd/d24 39 2021-11-03T17:14:06.365 INFO:tasks.workunit.client.1.smithi115.stdout:1/207: stat d7/dc/l1c 0 2021-11-03T17:14:06.366 INFO:tasks.workunit.client.1.smithi115.stdout:7/163: dwrite d3/da/d18/f2c [0,4194304] 0 2021-11-03T17:14:06.366 INFO:tasks.workunit.client.1.smithi115.stdout:0/207: creat d9/d15/d2c/d34/f4c x:0 0 0 2021-11-03T17:14:06.367 INFO:tasks.workunit.client.1.smithi115.stdout:0/208: write d9/d15/f1e [1377542,92461] 0 2021-11-03T17:14:06.369 INFO:tasks.workunit.client.1.smithi115.stdout:1/208: rename d7/f14 to d7/dc/f40 0 2021-11-03T17:14:06.369 INFO:tasks.workunit.client.1.smithi115.stdout:1/209: readlink d7/dc/l19 0 2021-11-03T17:14:06.369 INFO:tasks.workunit.client.1.smithi115.stdout:9/108: write d1/d5/f17 [2788255,11836] 0 2021-11-03T17:14:06.374 INFO:tasks.workunit.client.1.smithi115.stdout:4/314: truncate d7/dc/f11 7428273 0 2021-11-03T17:14:06.376 INFO:tasks.workunit.client.1.smithi115.stdout:0/209: link d9/d15/d1a/d1d/f4b d9/d15/d1a/d1d/f4d 0 2021-11-03T17:14:06.380 INFO:tasks.workunit.client.1.smithi115.stdout:4/315: dread d7/dc/d6a/d3c/f3e [0,4194304] 0 2021-11-03T17:14:06.381 INFO:tasks.workunit.client.1.smithi115.stdout:7/164: rename d3/da/l1f to d3/da/d18/l2e 0 2021-11-03T17:14:06.381 INFO:tasks.workunit.client.1.smithi115.stdout:7/165: chown d3/f9 431523 1 2021-11-03T17:14:06.382 INFO:tasks.workunit.client.1.smithi115.stdout:1/210: symlink d7/dc/dd/d24/d26/l41 0 2021-11-03T17:14:06.386 INFO:tasks.workunit.client.1.smithi115.stdout:4/316: rename d7/dc/d1a/d21/l55 to d7/dc/d1a/d21/d47/d56/l79 0 2021-11-03T17:14:06.386 INFO:tasks.workunit.client.1.smithi115.stdout:4/317: chown d7/dc/d1a/d21/d37/l59 235931 1 2021-11-03T17:14:06.386 INFO:tasks.workunit.client.1.smithi115.stdout:7/166: rmdir d3/da/d26 39 2021-11-03T17:14:06.387 INFO:tasks.workunit.client.1.smithi115.stdout:1/211: mkdir d7/dc/dd/d42 0 2021-11-03T17:14:06.387 INFO:tasks.workunit.client.1.smithi115.stdout:1/212: dread - d7/dc/d2a/d2d/d32/f3a zero size 2021-11-03T17:14:06.392 INFO:tasks.workunit.client.1.smithi115.stdout:1/213: write d7/dc/dd/d24/f28 [3837981,3318] 0 2021-11-03T17:14:06.392 INFO:tasks.workunit.client.1.smithi115.stdout:5/239: dread d1a/f1d [0,4194304] 0 2021-11-03T17:14:06.393 INFO:tasks.workunit.client.1.smithi115.stdout:5/240: mkdir d1a/d23/d53 0 2021-11-03T17:14:06.396 INFO:tasks.workunit.client.1.smithi115.stdout:5/241: write d1a/d28/d34/f27 [410452,25860] 0 2021-11-03T17:14:06.396 INFO:tasks.workunit.client.1.smithi115.stdout:5/242: stat d1a/f1d 0 2021-11-03T17:14:06.397 INFO:tasks.workunit.client.1.smithi115.stdout:6/163: dwrite d0/d1/d2/ff [0,4194304] 0 2021-11-03T17:14:06.397 INFO:tasks.workunit.client.1.smithi115.stdout:3/197: dwrite d13/f23 [0,4194304] 0 2021-11-03T17:14:06.398 INFO:tasks.workunit.client.1.smithi115.stdout:8/214: dwrite d4/d6/d18/d30/f34 [4194304,4194304] 0 2021-11-03T17:14:06.404 INFO:tasks.workunit.client.1.smithi115.stdout:0/210: dwrite d9/db/df/f48 [0,4194304] 0 2021-11-03T17:14:06.404 INFO:tasks.workunit.client.1.smithi115.stdout:0/211: fsync d9/d15/d1a/f2e 0 2021-11-03T17:14:06.405 INFO:tasks.workunit.client.1.smithi115.stdout:0/212: stat d9/d15/d1a/d1d/f24 0 2021-11-03T17:14:06.405 INFO:tasks.workunit.client.1.smithi115.stdout:0/213: mknod d9/d15/d2c/d34/c4e 1 2021-11-03T17:14:06.405 INFO:tasks.workunit.client.1.smithi115.stdout:1/214: dread d7/dc/d1a/f3d [0,4194304] 0 2021-11-03T17:14:06.405 INFO:tasks.workunit.client.1.smithi115.stdout:1/215: read - d7/dc/d2a/f31 zero size 2021-11-03T17:14:06.406 INFO:tasks.workunit.client.1.smithi115.stdout:1/216: dread - d7/dc/d1a/f35 zero size 2021-11-03T17:14:06.406 INFO:tasks.workunit.client.1.smithi115.stdout:3/198: mkdir d13/d20/d40 0 2021-11-03T17:14:06.409 INFO:tasks.workunit.client.1.smithi115.stdout:8/215: creat d4/d6/d18/d44/f48 x:0 0 0 2021-11-03T17:14:06.409 INFO:tasks.workunit.client.1.smithi115.stdout:8/216: chown d4/d6/d18/d30/f36 246222 1 2021-11-03T17:14:06.410 INFO:tasks.workunit.client.1.smithi115.stdout:9/109: truncate d1/d5/f6 2089952 0 2021-11-03T17:14:06.411 INFO:tasks.workunit.client.1.smithi115.stdout:4/318: rmdir d7/dc 39 2021-11-03T17:14:06.412 INFO:tasks.workunit.client.1.smithi115.stdout:4/319: write d7/dc/d1a/f2e [976803,120592] 0 2021-11-03T17:14:06.413 INFO:tasks.workunit.client.1.smithi115.stdout:7/167: truncate d3/f25 474764 0 2021-11-03T17:14:06.414 INFO:tasks.workunit.client.1.smithi115.stdout:9/110: read d1/d5/f8 [401884,30889] 0 2021-11-03T17:14:06.414 INFO:tasks.workunit.client.1.smithi115.stdout:5/243: dread d1a/d28/d34/f27 [0,4194304] 0 2021-11-03T17:14:06.415 INFO:tasks.workunit.client.1.smithi115.stdout:6/164: dwrite d0/d1/fb [4194304,4194304] 0 2021-11-03T17:14:06.415 INFO:tasks.workunit.client.1.smithi115.stdout:6/165: fsync d0/d1/f8 0 2021-11-03T17:14:06.416 INFO:tasks.workunit.client.1.smithi115.stdout:9/111: write d1/f9 [3982716,65218] 0 2021-11-03T17:14:06.418 INFO:tasks.workunit.client.1.smithi115.stdout:9/112: mkdir d1/d5/d1f 0 2021-11-03T17:14:06.428 INFO:tasks.workunit.client.1.smithi115.stdout:4/320: write d7/dc/f11 [3224030,80371] 0 2021-11-03T17:14:06.430 INFO:tasks.workunit.client.1.smithi115.stdout:4/321: unlink d7/dc/l15 0 2021-11-03T17:14:06.431 INFO:tasks.workunit.client.1.smithi115.stdout:4/322: truncate d7/dc/d6a/f3a 5716324 0 2021-11-03T17:14:06.433 INFO:tasks.workunit.client.1.smithi115.stdout:1/217: dread d7/dc/dd/d24/f28 [0,4194304] 0 2021-11-03T17:14:06.433 INFO:tasks.workunit.client.1.smithi115.stdout:7/168: dread d3/da/f15 [0,4194304] 0 2021-11-03T17:14:06.437 INFO:tasks.workunit.client.1.smithi115.stdout:1/218: rename d7/dc/d2a/d2d to d7/dc/dd/d24/d43 0 2021-11-03T17:14:06.439 INFO:tasks.workunit.client.1.smithi115.stdout:7/169: creat d3/da/d26/f2f x:0 0 0 2021-11-03T17:14:06.440 INFO:tasks.workunit.client.1.smithi115.stdout:0/214: dwrite d9/f25 [4194304,4194304] 0 2021-11-03T17:14:06.440 INFO:tasks.workunit.client.1.smithi115.stdout:3/199: dwrite f12 [0,4194304] 0 2021-11-03T17:14:06.441 INFO:tasks.workunit.client.1.smithi115.stdout:0/215: creat d9/db/f4f x:0 0 0 2021-11-03T17:14:06.444 INFO:tasks.workunit.client.1.smithi115.stdout:3/200: write d13/d20/d2e/f2f [43927,8695] 0 2021-11-03T17:14:06.444 INFO:tasks.workunit.client.1.smithi115.stdout:3/201: write d13/f22 [150563,5775] 0 2021-11-03T17:14:06.445 INFO:tasks.workunit.client.1.smithi115.stdout:0/216: rename d9/d15/d2c/d34/f4c to d9/d15/d1a/f50 0 2021-11-03T17:14:06.447 INFO:tasks.workunit.client.1.smithi115.stdout:3/202: rename d13/f39 to d13/d20/f41 0 2021-11-03T17:14:06.448 INFO:tasks.workunit.client.1.smithi115.stdout:8/217: dread d4/f27 [0,4194304] 0 2021-11-03T17:14:06.448 INFO:tasks.workunit.client.1.smithi115.stdout:0/217: getdents d9/db/df 0 2021-11-03T17:14:06.449 INFO:tasks.workunit.client.1.smithi115.stdout:3/203: creat d13/d3b/d3c/f42 x:0 0 0 2021-11-03T17:14:06.452 INFO:tasks.workunit.client.1.smithi115.stdout:8/218: symlink d4/l49 0 2021-11-03T17:14:06.453 INFO:tasks.workunit.client.1.smithi115.stdout:1/219: dread d7/dc/f2f [0,4194304] 0 2021-11-03T17:14:06.455 INFO:tasks.workunit.client.1.smithi115.stdout:3/204: creat d13/d3b/f43 x:0 0 0 2021-11-03T17:14:06.455 INFO:tasks.workunit.client.1.smithi115.stdout:3/205: chown d13/d3b/f43 9936 1 2021-11-03T17:14:06.455 INFO:tasks.workunit.client.1.smithi115.stdout:0/218: dread d9/d15/d1a/d1d/f4d [0,4194304] 0 2021-11-03T17:14:06.457 INFO:tasks.workunit.client.1.smithi115.stdout:5/244: dwrite d1a/d3d/f47 [0,4194304] 0 2021-11-03T17:14:06.457 INFO:tasks.workunit.client.1.smithi115.stdout:5/245: dread - d1a/d28/d34/f45 zero size 2021-11-03T17:14:06.459 INFO:tasks.workunit.client.1.smithi115.stdout:3/206: symlink d13/d35/l44 0 2021-11-03T17:14:06.459 INFO:tasks.workunit.client.1.smithi115.stdout:3/207: chown d13/d20/d2e/f32 208 1 2021-11-03T17:14:06.459 INFO:tasks.workunit.client.1.smithi115.stdout:3/208: truncate d13/d35/f3a 759104 0 2021-11-03T17:14:06.460 INFO:tasks.workunit.client.1.smithi115.stdout:0/219: symlink d9/db/df/l51 0 2021-11-03T17:14:06.466 INFO:tasks.workunit.client.1.smithi115.stdout:3/209: rename d13/f23 to d13/d20/f45 0 2021-11-03T17:14:06.466 INFO:tasks.workunit.client.1.smithi115.stdout:3/210: mknod d13/d35/c46 1 2021-11-03T17:14:06.469 INFO:tasks.workunit.client.1.smithi115.stdout:8/219: dread d4/d6/f2b [0,4194304] 0 2021-11-03T17:14:06.469 INFO:tasks.workunit.client.1.smithi115.stdout:6/166: dwrite d0/d1/d10/d13/d1c/d1e/f20 [0,4194304] 0 2021-11-03T17:14:06.471 INFO:tasks.workunit.client.1.smithi115.stdout:8/220: link d4/d6/l41 d4/d6/d3f/d43/l4a 0 2021-11-03T17:14:06.473 INFO:tasks.workunit.client.1.smithi115.stdout:4/323: dwrite d7/dc/d6a/d3c/f66 [0,4194304] 0 2021-11-03T17:14:06.475 INFO:tasks.workunit.client.1.smithi115.stdout:6/167: link d0/d1/d2/f1f d0/d1/d10/d13/d1c/f36 0 2021-11-03T17:14:06.476 INFO:tasks.workunit.client.1.smithi115.stdout:6/168: dread - d0/d1/f19 zero size 2021-11-03T17:14:06.479 INFO:tasks.workunit.client.1.smithi115.stdout:4/324: mkdir d7/d7a 0 2021-11-03T17:14:06.479 INFO:tasks.workunit.client.1.smithi115.stdout:4/325: write d7/dc/d1a/d21/d47/f69 [433451,56072] 0 2021-11-03T17:14:06.480 INFO:tasks.workunit.client.1.smithi115.stdout:4/326: mkdir d7/dc/d1a/d21/d47/d56/d63/d7b 0 2021-11-03T17:14:06.482 INFO:tasks.workunit.client.1.smithi115.stdout:1/220: dwrite d7/dc/dd/d24/d43/f3c [0,4194304] 0 2021-11-03T17:14:06.482 INFO:tasks.workunit.client.1.smithi115.stdout:1/221: dread - d7/dc/d2a/f31 zero size 2021-11-03T17:14:06.483 INFO:tasks.workunit.client.1.smithi115.stdout:4/327: write d7/dc/d1a/f1c [477281,114348] 0 2021-11-03T17:14:06.488 INFO:tasks.workunit.client.1.smithi115.stdout:1/222: creat d7/dc/dd/d42/f44 x:0 0 0 2021-11-03T17:14:06.491 INFO:tasks.workunit.client.1.smithi115.stdout:1/223: truncate d7/dc/f15 2836180 0 2021-11-03T17:14:06.491 INFO:tasks.workunit.client.1.smithi115.stdout:1/224: mknod d7/dc/dd/c45 1 2021-11-03T17:14:06.494 INFO:tasks.workunit.client.1.smithi115.stdout:8/221: dwrite d4/d6/d18/f3d [4194304,4194304] 0 2021-11-03T17:14:06.494 INFO:tasks.workunit.client.1.smithi115.stdout:5/246: dwrite d1a/f1d [0,4194304] 0 2021-11-03T17:14:06.495 INFO:tasks.workunit.client.1.smithi115.stdout:3/211: dwrite f10 [0,4194304] 0 2021-11-03T17:14:06.496 INFO:tasks.workunit.client.1.smithi115.stdout:3/212: chown d13/d20/f2d 19947691 1 2021-11-03T17:14:06.496 INFO:tasks.workunit.client.1.smithi115.stdout:3/213: chown l1 3 1 2021-11-03T17:14:06.496 INFO:tasks.workunit.client.1.smithi115.stdout:4/328: dread d7/dc/d6a/f3a [4194304,4194304] 0 2021-11-03T17:14:06.496 INFO:tasks.workunit.client.1.smithi115.stdout:4/329: write d7/dc/d1a/d21/d47/f74 [312982,20001] 0 2021-11-03T17:14:06.497 INFO:tasks.workunit.client.1.smithi115.stdout:2/170: sync 2021-11-03T17:14:06.497 INFO:tasks.workunit.client.1.smithi115.stdout:2/171: chown d4/d6/d13/f25 561491127 1 2021-11-03T17:14:06.498 INFO:tasks.workunit.client.1.smithi115.stdout:5/247: getdents d1a/d23 0 2021-11-03T17:14:06.498 INFO:tasks.workunit.client.1.smithi115.stdout:5/248: rename d1a to d1a/d3d/d54 22 2021-11-03T17:14:06.499 INFO:tasks.workunit.client.1.smithi115.stdout:3/214: symlink d13/d2a/l47 0 2021-11-03T17:14:06.499 INFO:tasks.workunit.client.1.smithi115.stdout:4/330: truncate d7/dc/d6a/d3c/f66 5194407 0 2021-11-03T17:14:06.500 INFO:tasks.workunit.client.1.smithi115.stdout:4/331: stat d7/dc/l33 0 2021-11-03T17:14:06.500 INFO:tasks.workunit.client.1.smithi115.stdout:4/332: write d7/dc/d1a/f38 [520798,59454] 0 2021-11-03T17:14:06.500 INFO:tasks.workunit.client.1.smithi115.stdout:8/222: symlink d4/d6/d3f/l4b 0 2021-11-03T17:14:06.504 INFO:tasks.workunit.client.1.smithi115.stdout:2/172: creat d4/f35 x:0 0 0 2021-11-03T17:14:06.505 INFO:tasks.workunit.client.1.smithi115.stdout:5/249: unlink fe 0 2021-11-03T17:14:06.505 INFO:tasks.workunit.client.1.smithi115.stdout:5/250: chown d1a/d28/d34/f27 0 1 2021-11-03T17:14:06.511 INFO:tasks.workunit.client.1.smithi115.stdout:1/225: dwrite d7/dc/dd/d24/f28 [0,4194304] 0 2021-11-03T17:14:06.511 INFO:tasks.workunit.client.1.smithi115.stdout:1/226: chown d7/dc/f2f 77486830 1 2021-11-03T17:14:06.513 INFO:tasks.workunit.client.1.smithi115.stdout:3/215: creat d13/f48 x:0 0 0 2021-11-03T17:14:06.514 INFO:tasks.workunit.client.1.smithi115.stdout:4/333: symlink d7/dc/d1a/d21/l7c 0 2021-11-03T17:14:06.516 INFO:tasks.workunit.client.1.smithi115.stdout:2/173: creat d4/d6/d13/f36 x:0 0 0 2021-11-03T17:14:06.516 INFO:tasks.workunit.client.1.smithi115.stdout:2/174: getdents d4/d6/d2e/d34 0 2021-11-03T17:14:06.516 INFO:tasks.workunit.client.1.smithi115.stdout:2/175: write d4/d6/d13/f25 [4411691,23028] 0 2021-11-03T17:14:06.519 INFO:tasks.workunit.client.1.smithi115.stdout:4/334: write d7/dc/d6a/f3a [2563586,27981] 0 2021-11-03T17:14:06.523 INFO:tasks.workunit.client.1.smithi115.stdout:5/251: dwrite f9 [0,4194304] 0 2021-11-03T17:14:06.526 INFO:tasks.workunit.client.1.smithi115.stdout:2/176: dread d4/d6/f1c [0,4194304] 0 2021-11-03T17:14:06.526 INFO:tasks.workunit.client.1.smithi115.stdout:3/216: truncate d13/f19 3024341 0 2021-11-03T17:14:06.528 INFO:tasks.workunit.client.1.smithi115.stdout:5/252: symlink d1a/d23/d53/l55 0 2021-11-03T17:14:06.529 INFO:tasks.workunit.client.1.smithi115.stdout:1/227: dwrite d7/dc/dd/d42/f44 [0,4194304] 0 2021-11-03T17:14:06.533 INFO:tasks.workunit.client.1.smithi115.stdout:2/177: fsync d4/d6/d13/f36 0 2021-11-03T17:14:06.533 INFO:tasks.workunit.client.1.smithi115.stdout:2/178: write d4/d6/f1c [5080213,66239] 0 2021-11-03T17:14:06.534 INFO:tasks.workunit.client.1.smithi115.stdout:2/179: chown d4/f35 0 1 2021-11-03T17:14:06.534 INFO:tasks.workunit.client.1.smithi115.stdout:2/180: mknod d4/d6/d2e/d34/c37 1 2021-11-03T17:14:06.534 INFO:tasks.workunit.client.1.smithi115.stdout:2/181: write d4/f35 [638960,16026] 0 2021-11-03T17:14:06.535 INFO:tasks.workunit.client.1.smithi115.stdout:1/228: truncate d7/dc/dd/d24/f2b 2215563 0 2021-11-03T17:14:06.535 INFO:tasks.workunit.client.1.smithi115.stdout:1/229: chown d7/f9 41236 1 2021-11-03T17:14:06.536 INFO:tasks.workunit.client.1.smithi115.stdout:5/253: truncate d1a/f1d 2256720 0 2021-11-03T17:14:06.537 INFO:tasks.workunit.client.1.smithi115.stdout:5/254: stat d1a/d28/f31 0 2021-11-03T17:14:06.537 INFO:tasks.workunit.client.1.smithi115.stdout:3/217: dread f12 [0,4194304] 0 2021-11-03T17:14:06.537 INFO:tasks.workunit.client.1.smithi115.stdout:3/218: chown d13/d20/d2e/f37 31 1 2021-11-03T17:14:06.537 INFO:tasks.workunit.client.1.smithi115.stdout:3/219: fsync d13/f33 0 2021-11-03T17:14:06.538 INFO:tasks.workunit.client.1.smithi115.stdout:3/220: fsync d13/d20/f29 0 2021-11-03T17:14:06.538 INFO:tasks.workunit.client.1.smithi115.stdout:3/221: readlink d13/l17 0 2021-11-03T17:14:06.539 INFO:tasks.workunit.client.1.smithi115.stdout:2/182: rmdir d4/d6/d2e 39 2021-11-03T17:14:06.539 INFO:tasks.workunit.client.1.smithi115.stdout:2/183: chown d4/d6/d2e/f14 111 1 2021-11-03T17:14:06.541 INFO:tasks.workunit.client.1.smithi115.stdout:1/230: unlink d7/dc/l19 0 2021-11-03T17:14:06.544 INFO:tasks.workunit.client.1.smithi115.stdout:5/255: truncate d1a/d28/d34/f2a 1941303 0 2021-11-03T17:14:06.545 INFO:tasks.workunit.client.1.smithi115.stdout:3/222: rmdir d13/d3b 39 2021-11-03T17:14:06.545 INFO:tasks.workunit.client.1.smithi115.stdout:3/223: chown d13/d2a/l47 1 1 2021-11-03T17:14:06.545 INFO:tasks.workunit.client.1.smithi115.stdout:3/224: mknod d13/d20/d40/c49 1 2021-11-03T17:14:06.546 INFO:tasks.workunit.client.1.smithi115.stdout:4/335: dwrite d7/dc/f17 [0,4194304] 0 2021-11-03T17:14:06.546 INFO:tasks.workunit.client.1.smithi115.stdout:4/336: write d7/dc/d1a/d21/f25 [997157,67111] 0 2021-11-03T17:14:06.546 INFO:tasks.workunit.client.1.smithi115.stdout:2/184: link d4/d6/f12 d4/d6/d13/f38 0 2021-11-03T17:14:06.548 INFO:tasks.workunit.client.1.smithi115.stdout:3/225: creat d13/d3b/f4a x:0 0 0 2021-11-03T17:14:06.548 INFO:tasks.workunit.client.1.smithi115.stdout:3/226: chown d13/l17 0 1 2021-11-03T17:14:06.550 INFO:tasks.workunit.client.1.smithi115.stdout:4/337: creat d7/dc/d1a/d21/d47/f7d x:0 0 0 2021-11-03T17:14:06.551 INFO:tasks.workunit.client.1.smithi115.stdout:2/185: getdents d4/d6/d2e 0 2021-11-03T17:14:06.553 INFO:tasks.workunit.client.1.smithi115.stdout:3/227: getdents d13 0 2021-11-03T17:14:06.554 INFO:tasks.workunit.client.1.smithi115.stdout:4/338: creat d7/dc/d1a/d21/d47/d56/d63/d7b/f7e x:0 0 0 2021-11-03T17:14:06.555 INFO:tasks.workunit.client.1.smithi115.stdout:4/339: write d7/dc/d1a/d21/d47/d56/d63/f6e [3174082,81941] 0 2021-11-03T17:14:06.555 INFO:tasks.workunit.client.1.smithi115.stdout:2/186: creat d4/d6/d2e/f39 x:0 0 0 2021-11-03T17:14:06.559 INFO:tasks.workunit.client.1.smithi115.stdout:3/228: symlink d13/d3b/d3c/l4b 0 2021-11-03T17:14:06.561 INFO:tasks.workunit.client.1.smithi115.stdout:4/340: symlink d7/dc/d6a/d3c/l7f 0 2021-11-03T17:14:06.562 INFO:tasks.workunit.client.1.smithi115.stdout:4/341: truncate d7/dc/f12 1335267 0 2021-11-03T17:14:06.562 INFO:tasks.workunit.client.1.smithi115.stdout:4/342: stat d7/dc/d1a/d21/f25 0 2021-11-03T17:14:06.563 INFO:tasks.workunit.client.1.smithi115.stdout:4/343: write d7/dc/d1a/f38 [1541693,45531] 0 2021-11-03T17:14:06.568 INFO:tasks.workunit.client.1.smithi115.stdout:3/229: dread d13/f21 [0,4194304] 0 2021-11-03T17:14:06.569 INFO:tasks.workunit.client.1.smithi115.stdout:3/230: creat d13/f4c x:0 0 0 2021-11-03T17:14:06.570 INFO:tasks.workunit.client.1.smithi115.stdout:3/231: creat d13/d3b/d3c/f4d x:0 0 0 2021-11-03T17:14:06.570 INFO:tasks.workunit.client.1.smithi115.stdout:3/232: write d13/f33 [344216,79912] 0 2021-11-03T17:14:06.571 INFO:tasks.workunit.client.1.smithi115.stdout:3/233: getdents d13/d3b 0 2021-11-03T17:14:06.572 INFO:tasks.workunit.client.1.smithi115.stdout:3/234: creat d13/d2a/f4e x:0 0 0 2021-11-03T17:14:06.573 INFO:tasks.workunit.client.1.smithi115.stdout:3/235: creat d13/d20/d40/f4f x:0 0 0 2021-11-03T17:14:06.594 INFO:tasks.workunit.client.1.smithi115.stdout:3/236: dwrite d13/d35/f3a [0,4194304] 0 2021-11-03T17:14:06.610 INFO:tasks.workunit.client.1.smithi115.stdout:4/344: dwrite d7/dc/d1a/f1c [4194304,4194304] 0 2021-11-03T17:14:06.611 INFO:tasks.workunit.client.1.smithi115.stdout:5/256: dwrite d1a/d28/f2c [0,4194304] 0 2021-11-03T17:14:06.613 INFO:tasks.workunit.client.1.smithi115.stdout:4/345: link d7/dc/d1a/d21/l7c d7/dc/d1a/l80 0 2021-11-03T17:14:06.613 INFO:tasks.workunit.client.1.smithi115.stdout:4/346: mknod d7/dc/d1a/d21/c81 1 2021-11-03T17:14:06.613 INFO:tasks.workunit.client.1.smithi115.stdout:4/347: mknod d7/dc/d1a/d21/d47/d56/d42/c82 1 2021-11-03T17:14:06.616 INFO:tasks.workunit.client.1.smithi115.stdout:5/257: creat d1a/d28/d4a/f56 x:0 0 0 2021-11-03T17:14:06.617 INFO:tasks.workunit.client.1.smithi115.stdout:4/348: rename d7/dc/d1a/d21/d47/d56/d42/d5f to d7/d7a/d83 0 2021-11-03T17:14:06.619 INFO:tasks.workunit.client.1.smithi115.stdout:5/258: mkdir d1a/d28/d44/d4f/d57 0 2021-11-03T17:14:06.620 INFO:tasks.workunit.client.1.smithi115.stdout:5/259: mkdir d1a/d3d/d58 0 2021-11-03T17:14:06.629 INFO:tasks.workunit.client.1.smithi115.stdout:3/237: dwrite d13/f1e [0,4194304] 0 2021-11-03T17:14:06.632 INFO:tasks.workunit.client.1.smithi115.stdout:3/238: dread d13/d35/f3d [0,4194304] 0 2021-11-03T17:14:06.633 INFO:tasks.workunit.client.1.smithi115.stdout:3/239: getdents d13/d20/d40 0 2021-11-03T17:14:06.634 INFO:tasks.workunit.client.1.smithi115.stdout:3/240: dread - d13/f48 zero size 2021-11-03T17:14:06.634 INFO:tasks.workunit.client.1.smithi115.stdout:3/241: truncate d13/d20/f2d 2860430 0 2021-11-03T17:14:06.638 INFO:tasks.workunit.client.1.smithi115.stdout:4/349: dwrite d7/dc/d1a/f1c [4194304,4194304] 0 2021-11-03T17:14:06.639 INFO:tasks.workunit.client.1.smithi115.stdout:4/350: mknod d7/dc/d6a/c84 1 2021-11-03T17:14:06.640 INFO:tasks.workunit.client.1.smithi115.stdout:4/351: creat d7/dc/d1a/d21/d47/d56/d63/f85 x:0 0 0 2021-11-03T17:14:06.641 INFO:tasks.workunit.client.1.smithi115.stdout:4/352: truncate d7/dc/d6a/d3c/f66 5192917 0 2021-11-03T17:14:06.642 INFO:tasks.workunit.client.1.smithi115.stdout:4/353: mkdir d7/d7a/d83/d86 0 2021-11-03T17:14:06.644 INFO:tasks.workunit.client.1.smithi115.stdout:4/354: creat d7/dc/d1a/d21/d47/d56/d63/f87 x:0 0 0 2021-11-03T17:14:06.650 INFO:tasks.workunit.client.1.smithi115.stdout:5/260: dwrite d1a/d28/f31 [0,4194304] 0 2021-11-03T17:14:06.650 INFO:tasks.workunit.client.1.smithi115.stdout:5/261: readlink d1a/d28/l4d 0 2021-11-03T17:14:06.651 INFO:tasks.workunit.client.1.smithi115.stdout:5/262: readlink l13 0 2021-11-03T17:14:06.652 INFO:tasks.workunit.client.1.smithi115.stdout:4/355: dread d7/dc/d1a/d21/d47/f51 [0,4194304] 0 2021-11-03T17:14:06.654 INFO:tasks.workunit.client.1.smithi115.stdout:4/356: mkdir d7/d7a/d88 0 2021-11-03T17:14:06.654 INFO:tasks.workunit.client.1.smithi115.stdout:4/357: dread - d7/dc/d1a/d21/d47/d56/d63/d7b/f7e zero size 2021-11-03T17:14:06.656 INFO:tasks.workunit.client.1.smithi115.stdout:4/358: fsync d7/dc/d6a/d72/f76 0 2021-11-03T17:14:06.656 INFO:tasks.workunit.client.1.smithi115.stdout:4/359: read d7/dc/d1a/d21/f25 [157252,123582] 0 2021-11-03T17:14:06.656 INFO:tasks.workunit.client.1.smithi115.stdout:4/360: readlink d7/dc/d1a/d21/l62 0 2021-11-03T17:14:06.657 INFO:tasks.workunit.client.1.smithi115.stdout:4/361: mknod d7/dc/d1a/d21/d37/c89 1 2021-11-03T17:14:06.657 INFO:tasks.workunit.client.1.smithi115.stdout:5/263: dread d1a/d3d/f3b [0,4194304] 0 2021-11-03T17:14:06.657 INFO:tasks.workunit.client.1.smithi115.stdout:3/242: dwrite d13/f19 [0,4194304] 0 2021-11-03T17:14:06.657 INFO:tasks.workunit.client.1.smithi115.stdout:5/264: write d1a/d28/d34/f33 [261503,124413] 0 2021-11-03T17:14:06.659 INFO:tasks.workunit.client.1.smithi115.stdout:3/243: rename d13/d3b/d3c to d13/d50 0 2021-11-03T17:14:06.660 INFO:tasks.workunit.client.1.smithi115.stdout:5/265: symlink d1a/d28/l59 0 2021-11-03T17:14:06.661 INFO:tasks.workunit.client.1.smithi115.stdout:5/266: rename d1a to d1a/d23/d53/d5a 22 2021-11-03T17:14:06.662 INFO:tasks.workunit.client.1.smithi115.stdout:3/244: write d13/d20/f2d [1578481,28975] 0 2021-11-03T17:14:06.662 INFO:tasks.workunit.client.1.smithi115.stdout:5/267: mkdir d1a/d28/d44/d5b 0 2021-11-03T17:14:06.663 INFO:tasks.workunit.client.1.smithi115.stdout:5/268: write f0 [399328,89259] 0 2021-11-03T17:14:06.663 INFO:tasks.workunit.client.1.smithi115.stdout:5/269: fdatasync f9 0 2021-11-03T17:14:06.664 INFO:tasks.workunit.client.1.smithi115.stdout:3/245: rename d13/d20 to d13/d51 0 2021-11-03T17:14:06.667 INFO:tasks.workunit.client.1.smithi115.stdout:5/270: rmdir d1a/d23/d52 0 2021-11-03T17:14:06.668 INFO:tasks.workunit.client.1.smithi115.stdout:3/246: creat d13/d51/d2e/f52 x:0 0 0 2021-11-03T17:14:06.670 INFO:tasks.workunit.client.1.smithi115.stdout:5/271: creat d1a/d23/f5c x:0 0 0 2021-11-03T17:14:06.670 INFO:tasks.workunit.client.1.smithi115.stdout:5/272: write fd [4265090,72352] 0 2021-11-03T17:14:06.671 INFO:tasks.workunit.client.1.smithi115.stdout:3/247: mkdir d13/d2a/d53 0 2021-11-03T17:14:06.672 INFO:tasks.workunit.client.1.smithi115.stdout:5/273: unlink l13 0 2021-11-03T17:14:06.672 INFO:tasks.workunit.client.1.smithi115.stdout:5/274: mknod d1a/d28/c5d 1 2021-11-03T17:14:06.674 INFO:tasks.workunit.client.1.smithi115.stdout:5/275: symlink d1a/d28/d44/d4f/l5e 0 2021-11-03T17:14:06.674 INFO:tasks.workunit.client.1.smithi115.stdout:5/276: mknod d1a/d23/d53/c5f 1 2021-11-03T17:14:06.675 INFO:tasks.workunit.client.1.smithi115.stdout:5/277: write d1a/d28/d34/f27 [2938203,38903] 0 2021-11-03T17:14:06.676 INFO:tasks.workunit.client.1.smithi115.stdout:4/362: dwrite d7/dc/d6a/d3c/f66 [4194304,4194304] 0 2021-11-03T17:14:06.677 INFO:tasks.workunit.client.1.smithi115.stdout:4/363: symlink d7/d7a/l8a 0 2021-11-03T17:14:06.678 INFO:tasks.workunit.client.1.smithi115.stdout:4/364: mknod d7/dc/d6a/d72/c8b 1 2021-11-03T17:14:06.678 INFO:tasks.workunit.client.1.smithi115.stdout:4/365: fdatasync d7/dc/d1a/d21/f22 0 2021-11-03T17:14:06.678 INFO:tasks.workunit.client.1.smithi115.stdout:4/366: read d7/dc/d1a/f2e [1495281,49833] 0 2021-11-03T17:14:06.690 INFO:tasks.workunit.client.1.smithi115.stdout:5/278: dread d1a/d28/f31 [0,4194304] 0 2021-11-03T17:14:06.690 INFO:tasks.workunit.client.1.smithi115.stdout:5/279: rename d1a/d23/d53 to d1a/d23/d53/d60 22 2021-11-03T17:14:06.690 INFO:tasks.workunit.client.1.smithi115.stdout:5/280: chown d1a/d28/d34/f33 2017831 1 2021-11-03T17:14:06.691 INFO:tasks.workunit.client.1.smithi115.stdout:5/281: unlink d1a/f3c 0 2021-11-03T17:14:06.694 INFO:tasks.workunit.client.1.smithi115.stdout:5/282: write f9 [4126037,16518] 0 2021-11-03T17:14:06.694 INFO:tasks.workunit.client.1.smithi115.stdout:5/283: fdatasync d1a/d28/d4a/f56 0 2021-11-03T17:14:06.695 INFO:tasks.workunit.client.1.smithi115.stdout:3/248: dwrite d13/d51/d2e/f52 [0,4194304] 0 2021-11-03T17:14:06.695 INFO:tasks.workunit.client.1.smithi115.stdout:3/249: write d13/f4c [3745,66993] 0 2021-11-03T17:14:06.695 INFO:tasks.workunit.client.1.smithi115.stdout:5/284: getdents d1a/d23/d53 0 2021-11-03T17:14:06.696 INFO:tasks.workunit.client.1.smithi115.stdout:4/367: dwrite d7/dc/d6a/d72/f76 [0,4194304] 0 2021-11-03T17:14:06.698 INFO:tasks.workunit.client.1.smithi115.stdout:3/250: mkdir d13/d35/d54 0 2021-11-03T17:14:06.700 INFO:tasks.workunit.client.1.smithi115.stdout:5/285: rename f9 to d1a/d23/f61 0 2021-11-03T17:14:06.703 INFO:tasks.workunit.client.1.smithi115.stdout:5/286: dread d1a/d28/d34/f33 [0,4194304] 0 2021-11-03T17:14:06.703 INFO:tasks.workunit.client.1.smithi115.stdout:3/251: dread d13/f1f [0,4194304] 0 2021-11-03T17:14:06.704 INFO:tasks.workunit.client.1.smithi115.stdout:3/252: chown d13/l18 0 1 2021-11-03T17:14:06.704 INFO:tasks.workunit.client.1.smithi115.stdout:3/253: truncate d13/d3b/f43 98921 0 2021-11-03T17:14:06.711 INFO:tasks.workunit.client.1.smithi115.stdout:4/368: dwrite d7/dc/d1a/d21/d47/d56/d63/f6e [0,4194304] 0 2021-11-03T17:14:06.716 INFO:tasks.workunit.client.1.smithi115.stdout:3/254: creat d13/d51/f55 x:0 0 0 2021-11-03T17:14:06.719 INFO:tasks.workunit.client.1.smithi115.stdout:5/287: truncate fd 3424245 0 2021-11-03T17:14:06.720 INFO:tasks.workunit.client.1.smithi115.stdout:5/288: write d1a/d3d/f3b [330637,1794] 0 2021-11-03T17:14:06.721 INFO:tasks.workunit.client.1.smithi115.stdout:4/369: dread d7/dc/f11 [0,4194304] 0 2021-11-03T17:14:06.721 INFO:tasks.workunit.client.1.smithi115.stdout:4/370: stat d7/d7a/d83/f64 0 2021-11-03T17:14:06.726 INFO:tasks.workunit.client.1.smithi115.stdout:4/371: dread d7/dc/d1a/d21/d47/f51 [0,4194304] 0 2021-11-03T17:14:06.727 INFO:tasks.workunit.client.1.smithi115.stdout:4/372: write d7/dc/d6a/f6f [1026630,42556] 0 2021-11-03T17:14:06.739 INFO:tasks.workunit.client.1.smithi115.stdout:4/373: write d7/dc/d1a/d21/d47/d56/d63/f67 [951009,13760] 0 2021-11-03T17:14:06.751 INFO:tasks.workunit.client.1.smithi115.stdout:5/289: dwrite fd [0,4194304] 0 2021-11-03T17:14:06.752 INFO:tasks.workunit.client.1.smithi115.stdout:3/255: dwrite d13/f4c [0,4194304] 0 2021-11-03T17:14:06.752 INFO:tasks.workunit.client.1.smithi115.stdout:5/290: creat d1a/d3d/f62 x:0 0 0 2021-11-03T17:14:06.761 INFO:tasks.workunit.client.1.smithi115.stdout:4/374: dwrite d7/dc/d1a/d21/d47/f69 [0,4194304] 0 2021-11-03T17:14:06.763 INFO:tasks.workunit.client.1.smithi115.stdout:5/291: dread f0 [0,4194304] 0 2021-11-03T17:14:06.764 INFO:tasks.workunit.client.1.smithi115.stdout:5/292: mkdir d1a/d28/d4c/d63 0 2021-11-03T17:14:06.768 INFO:tasks.workunit.client.1.smithi115.stdout:5/293: dread d1a/d28/f31 [0,4194304] 0 2021-11-03T17:14:06.769 INFO:tasks.workunit.client.1.smithi115.stdout:4/375: dread d7/dc/d1a/f1c [4194304,4194304] 0 2021-11-03T17:14:06.770 INFO:tasks.workunit.client.1.smithi115.stdout:3/256: dwrite d13/d51/f41 [0,4194304] 0 2021-11-03T17:14:06.770 INFO:tasks.workunit.client.1.smithi115.stdout:5/294: rename d1a/d28/d34/f45 to d1a/d3d/f64 0 2021-11-03T17:14:06.772 INFO:tasks.workunit.client.1.smithi115.stdout:3/257: creat d13/f56 x:0 0 0 2021-11-03T17:14:06.772 INFO:tasks.workunit.client.1.smithi115.stdout:4/376: truncate d7/dc/d1a/d21/d47/f51 824817 0 2021-11-03T17:14:06.773 INFO:tasks.workunit.client.1.smithi115.stdout:5/295: mkdir d1a/d28/d4a/d65 0 2021-11-03T17:14:06.774 INFO:tasks.workunit.client.1.smithi115.stdout:3/258: symlink d13/d51/l57 0 2021-11-03T17:14:06.775 INFO:tasks.workunit.client.1.smithi115.stdout:3/259: write d13/f1d [8660924,97807] 0 2021-11-03T17:14:06.775 INFO:tasks.workunit.client.1.smithi115.stdout:5/296: write f11 [3697057,95726] 0 2021-11-03T17:14:06.777 INFO:tasks.workunit.client.1.smithi115.stdout:4/377: creat d7/d7a/d83/f8c x:0 0 0 2021-11-03T17:14:06.777 INFO:tasks.workunit.client.1.smithi115.stdout:4/378: fsync d7/d7a/d83/f64 0 2021-11-03T17:14:06.777 INFO:tasks.workunit.client.1.smithi115.stdout:4/379: fdatasync d7/dc/f12 0 2021-11-03T17:14:06.777 INFO:tasks.workunit.client.1.smithi115.stdout:5/297: write d1a/d28/d34/f27 [2515390,29948] 0 2021-11-03T17:14:06.778 INFO:tasks.workunit.client.1.smithi115.stdout:3/260: creat d13/d3b/f58 x:0 0 0 2021-11-03T17:14:06.778 INFO:tasks.workunit.client.1.smithi115.stdout:3/261: fdatasync d13/d51/d2e/f2f 0 2021-11-03T17:14:06.778 INFO:tasks.workunit.client.1.smithi115.stdout:3/262: chown d13/d2c 152153169 1 2021-11-03T17:14:06.779 INFO:tasks.workunit.client.1.smithi115.stdout:4/380: creat d7/dc/d1a/d21/d47/d56/f8d x:0 0 0 2021-11-03T17:14:06.780 INFO:tasks.workunit.client.1.smithi115.stdout:5/298: creat d1a/d28/d34/f66 x:0 0 0 2021-11-03T17:14:06.783 INFO:tasks.workunit.client.1.smithi115.stdout:3/263: creat d13/d51/d2e/f59 x:0 0 0 2021-11-03T17:14:06.785 INFO:tasks.workunit.client.1.smithi115.stdout:4/381: rmdir d7/d7a/d88 0 2021-11-03T17:14:06.785 INFO:tasks.workunit.client.1.smithi115.stdout:4/382: stat d7/dc/f60 0 2021-11-03T17:14:06.804 INFO:tasks.workunit.client.1.smithi115.stdout:3/264: dwrite d13/d35/f36 [0,4194304] 0 2021-11-03T17:14:06.805 INFO:tasks.workunit.client.1.smithi115.stdout:4/383: dwrite d7/dc/d6a/f6f [0,4194304] 0 2021-11-03T17:14:06.808 INFO:tasks.workunit.client.1.smithi115.stdout:3/265: link d13/d51/d2e/f59 d13/d51/d40/f5a 0 2021-11-03T17:14:06.809 INFO:tasks.workunit.client.1.smithi115.stdout:3/266: chown d13/l18 3939974 1 2021-11-03T17:14:06.809 INFO:tasks.workunit.client.1.smithi115.stdout:4/384: rmdir d7/dc/d6a 39 2021-11-03T17:14:06.810 INFO:tasks.workunit.client.1.smithi115.stdout:4/385: truncate d7/d7a/d83/f8c 697152 0 2021-11-03T17:14:06.811 INFO:tasks.workunit.client.1.smithi115.stdout:4/386: mkdir d7/dc/d1a/d21/d37/d8e 0 2021-11-03T17:14:06.817 INFO:tasks.workunit.client.1.smithi115.stdout:4/387: truncate d7/dc/d6a/f6f 888931 0 2021-11-03T17:14:06.833 INFO:tasks.workunit.client.1.smithi115.stdout:4/388: dwrite d7/dc/f60 [4194304,4194304] 0 2021-11-03T17:14:06.834 INFO:tasks.workunit.client.1.smithi115.stdout:4/389: rename d7/d7a/l8a to d7/d7a/l8f 0 2021-11-03T17:14:06.835 INFO:tasks.workunit.client.1.smithi115.stdout:4/390: symlink d7/dc/l90 0 2021-11-03T17:14:06.836 INFO:tasks.workunit.client.1.smithi115.stdout:4/391: write d7/dc/d1a/d21/f25 [520579,86171] 0 2021-11-03T17:14:06.852 INFO:tasks.workunit.client.1.smithi115.stdout:4/392: dwrite d7/dc/d1a/d21/d47/d56/f8d [0,4194304] 0 2021-11-03T17:14:06.855 INFO:tasks.workunit.client.1.smithi115.stdout:4/393: rmdir d7/dc/d1a/d21/d37/d8e 0 2021-11-03T17:14:06.855 INFO:tasks.workunit.client.1.smithi115.stdout:4/394: read d7/d7a/d83/f64 [421479,65315] 0 2021-11-03T17:14:06.874 INFO:tasks.workunit.client.1.smithi115.stdout:4/395: dwrite d7/dc/f11 [4194304,4194304] 0 2021-11-03T17:14:06.875 INFO:tasks.workunit.client.1.smithi115.stdout:4/396: creat d7/d7a/f91 x:0 0 0 2021-11-03T17:14:06.875 INFO:tasks.workunit.client.1.smithi115.stdout:4/397: chown d7/dc/d1a/d21/d37 316 1 2021-11-03T17:14:06.876 INFO:tasks.workunit.client.1.smithi115.stdout:4/398: mknod d7/dc/c92 1 2021-11-03T17:14:06.876 INFO:tasks.workunit.client.1.smithi115.stdout:4/399: creat d7/dc/d1a/d21/d47/d56/f93 x:0 0 0 2021-11-03T17:14:06.877 INFO:tasks.workunit.client.1.smithi115.stdout:4/400: readlink d7/lb 0 2021-11-03T17:14:06.877 INFO:tasks.workunit.client.1.smithi115.stdout:4/401: fdatasync d7/dc/d1a/d21/d47/f74 0 2021-11-03T17:14:06.882 INFO:tasks.workunit.client.1.smithi115.stdout:4/402: dread d7/dc/f17 [0,4194304] 0 2021-11-03T17:14:06.890 INFO:tasks.workunit.client.1.smithi115.stdout:4/403: dread d7/dc/f60 [4194304,4194304] 0 2021-11-03T17:14:06.891 INFO:tasks.workunit.client.1.smithi115.stdout:4/404: write d7/dc/d1a/f1c [7325077,53196] 0 2021-11-03T17:14:06.892 INFO:tasks.workunit.client.1.smithi115.stdout:4/405: rmdir d7/d7a/d83/d86 0 2021-11-03T17:14:06.893 INFO:tasks.workunit.client.1.smithi115.stdout:4/406: write d7/dc/f11 [998513,8799] 0 2021-11-03T17:14:06.893 INFO:tasks.workunit.client.1.smithi115.stdout:4/407: readlink d7/d7a/d83/l73 0 2021-11-03T17:14:06.963 INFO:tasks.workunit.client.0.smithi027.stdout:6/160: sync 2021-11-03T17:14:06.964 INFO:tasks.workunit.client.0.smithi027.stdout:6/161: fsync dc/f10 0 2021-11-03T17:14:06.964 INFO:tasks.workunit.client.0.smithi027.stdout:6/162: fdatasync dc/f10 0 2021-11-03T17:14:06.964 INFO:tasks.workunit.client.0.smithi027.stdout:6/163: stat dc/d16/d1b/l20 0 2021-11-03T17:14:06.965 INFO:tasks.workunit.client.0.smithi027.stdout:6/164: readlink lb 0 2021-11-03T17:14:06.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:06 smithi115 conmon[34670]: cluster 2021-11-03T17:14:06.604054+0000 mgr.smithi027.ufhhvb (mgr 2021-11-03T17:14:06.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:06 smithi115 conmon[34670]: .14162) 300 : cluster [DBG] pgmap v250: 65 pgs: 65 active+clean; 891 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 33 MiB/s rd, 70 MiB/s wr, 146 op/s 2021-11-03T17:14:07.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:06 smithi027 conmon[32322]: cluster 2021-11-03T17:14:06.604054+0000 mgr.smithi027.ufhhvb (mgr. 2021-11-03T17:14:07.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:06 smithi027 conmon[32322]: 14162) 300 : cluster [DBG] pgmap v250: 65 pgs: 65 active+clean; 891 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 33 MiB/s rd, 70 MiB/s wr, 146 op/s 2021-11-03T17:14:07.052 INFO:tasks.workunit.client.0.smithi027.stdout:2/250: sync 2021-11-03T17:14:07.052 INFO:tasks.workunit.client.0.smithi027.stdout:7/211: sync 2021-11-03T17:14:07.053 INFO:tasks.workunit.client.0.smithi027.stdout:3/132: sync 2021-11-03T17:14:07.053 INFO:tasks.workunit.client.0.smithi027.stdout:3/133: getdents d1 0 2021-11-03T17:14:07.055 INFO:tasks.workunit.client.0.smithi027.stdout:7/212: creat d0/d2/d10/d1e/d20/f43 x:0 0 0 2021-11-03T17:14:07.056 INFO:tasks.workunit.client.0.smithi027.stdout:7/213: readlink d0/d2/d10/l30 0 2021-11-03T17:14:07.057 INFO:tasks.workunit.client.0.smithi027.stdout:2/251: creat d0/d1b/d43/f55 x:0 0 0 2021-11-03T17:14:07.076 INFO:tasks.workunit.client.0.smithi027.stdout:2/252: dread d0/d2a/f21 [0,4194304] 0 2021-11-03T17:14:07.077 INFO:tasks.workunit.client.0.smithi027.stdout:2/253: truncate d0/d5/f1e 1321133 0 2021-11-03T17:14:07.077 INFO:tasks.workunit.client.0.smithi027.stdout:2/254: dread - d0/f46 zero size 2021-11-03T17:14:07.077 INFO:tasks.workunit.client.0.smithi027.stdout:3/134: dwrite d1/db/f1e [4194304,4194304] 0 2021-11-03T17:14:07.078 INFO:tasks.workunit.client.0.smithi027.stdout:7/214: dwrite d0/d2/d10/f1b [0,4194304] 0 2021-11-03T17:14:07.078 INFO:tasks.workunit.client.0.smithi027.stdout:7/215: fdatasync d0/d2/d10/d1e/d20/f43 0 2021-11-03T17:14:07.078 INFO:tasks.workunit.client.0.smithi027.stdout:2/255: getdents d0/d5 0 2021-11-03T17:14:07.079 INFO:tasks.workunit.client.0.smithi027.stdout:7/216: read d0/d2/d1c/f27 [2254134,5444] 0 2021-11-03T17:14:07.079 INFO:tasks.workunit.client.0.smithi027.stdout:7/217: readlink d0/l34 0 2021-11-03T17:14:07.081 INFO:tasks.workunit.client.0.smithi027.stdout:7/218: symlink d0/l44 0 2021-11-03T17:14:07.082 INFO:tasks.workunit.client.0.smithi027.stdout:7/219: truncate d0/d2/d10/f19 2485716 0 2021-11-03T17:14:07.083 INFO:tasks.workunit.client.0.smithi027.stdout:7/220: truncate d0/f18 400106 0 2021-11-03T17:14:07.083 INFO:tasks.workunit.client.0.smithi027.stdout:7/221: mknod d0/d2/d10/d1e/d20/d31/c45 1 2021-11-03T17:14:07.084 INFO:tasks.workunit.client.0.smithi027.stdout:7/222: write d0/f18 [904059,43720] 0 2021-11-03T17:14:07.085 INFO:tasks.workunit.client.0.smithi027.stdout:2/256: dread d0/d5/f53 [0,4194304] 0 2021-11-03T17:14:07.085 INFO:tasks.workunit.client.0.smithi027.stdout:2/257: chown d0/d5/l1a 55797637 1 2021-11-03T17:14:07.086 INFO:tasks.workunit.client.0.smithi027.stdout:2/258: write d0/d5/f13 [7872477,46112] 0 2021-11-03T17:14:07.086 INFO:tasks.workunit.client.0.smithi027.stdout:2/259: fdatasync d0/d1b/d29/f2b 0 2021-11-03T17:14:07.091 INFO:tasks.workunit.client.0.smithi027.stdout:3/135: dwrite d1/f17 [0,4194304] 0 2021-11-03T17:14:07.101 INFO:tasks.workunit.client.0.smithi027.stdout:2/260: dread d0/d5/f3 [0,4194304] 0 2021-11-03T17:14:07.101 INFO:tasks.workunit.client.0.smithi027.stdout:2/261: mknod d0/d1b/d29/c56 1 2021-11-03T17:14:07.103 INFO:tasks.workunit.client.0.smithi027.stdout:2/262: creat d0/d1b/d1c/f57 x:0 0 0 2021-11-03T17:14:07.104 INFO:tasks.workunit.client.0.smithi027.stdout:2/263: write d0/d2a/f23 [4149969,26672] 0 2021-11-03T17:14:07.104 INFO:tasks.workunit.client.0.smithi027.stdout:2/264: getdents d0/d1b/d29/d54 0 2021-11-03T17:14:07.105 INFO:tasks.workunit.client.0.smithi027.stdout:2/265: unlink d0/d5/f8 0 2021-11-03T17:14:07.106 INFO:tasks.workunit.client.0.smithi027.stdout:2/266: mkdir d0/d1b/d29/d58 0 2021-11-03T17:14:07.107 INFO:tasks.workunit.client.0.smithi027.stdout:2/267: mknod d0/c59 1 2021-11-03T17:14:07.108 INFO:tasks.workunit.client.0.smithi027.stdout:2/268: creat d0/f5a x:0 0 0 2021-11-03T17:14:07.108 INFO:tasks.workunit.client.0.smithi027.stdout:2/269: write d0/f10 [4419054,64560] 0 2021-11-03T17:14:07.113 INFO:tasks.workunit.client.0.smithi027.stdout:2/270: dread d0/d1b/d1c/f26 [0,4194304] 0 2021-11-03T17:14:07.114 INFO:tasks.workunit.client.0.smithi027.stdout:2/271: mkdir d0/d1b/d29/d58/d5b 0 2021-11-03T17:14:07.116 INFO:tasks.workunit.client.0.smithi027.stdout:2/272: creat d0/d1b/d29/d58/f5c x:0 0 0 2021-11-03T17:14:07.116 INFO:tasks.workunit.client.0.smithi027.stdout:2/273: write d0/d1b/d29/f4e [757631,2956] 0 2021-11-03T17:14:07.116 INFO:tasks.workunit.client.0.smithi027.stdout:2/274: dread - d0/d1b/d1c/f57 zero size 2021-11-03T17:14:07.116 INFO:tasks.workunit.client.0.smithi027.stdout:2/275: fdatasync d0/d1b/d29/f2b 0 2021-11-03T17:14:07.219 INFO:tasks.workunit.client.0.smithi027.stdout:8/270: sync 2021-11-03T17:14:07.219 INFO:tasks.workunit.client.0.smithi027.stdout:9/212: sync 2021-11-03T17:14:07.220 INFO:tasks.workunit.client.0.smithi027.stdout:9/213: truncate db/d1b/d23/f4b 1027392 0 2021-11-03T17:14:07.220 INFO:tasks.workunit.client.0.smithi027.stdout:9/214: dread - db/d1b/d1c/d1d/f49 zero size 2021-11-03T17:14:07.221 INFO:tasks.workunit.client.0.smithi027.stdout:9/215: rmdir db/d1b/d23/d2c 39 2021-11-03T17:14:07.225 INFO:tasks.workunit.client.0.smithi027.stdout:9/216: dread db/d1b/f45 [0,4194304] 0 2021-11-03T17:14:07.228 INFO:tasks.workunit.client.0.smithi027.stdout:9/217: dread db/f17 [0,4194304] 0 2021-11-03T17:14:07.232 INFO:tasks.workunit.client.0.smithi027.stdout:9/218: dread db/d1b/d1c/d1d/d40/f48 [0,4194304] 0 2021-11-03T17:14:07.232 INFO:tasks.workunit.client.0.smithi027.stdout:9/219: chown db/l15 46248 1 2021-11-03T17:14:07.232 INFO:tasks.workunit.client.0.smithi027.stdout:9/220: write db/fd [5730921,800] 0 2021-11-03T17:14:07.233 INFO:tasks.workunit.client.0.smithi027.stdout:9/221: truncate db/d1b/d2b/f2a 5105413 0 2021-11-03T17:14:07.234 INFO:tasks.workunit.client.0.smithi027.stdout:9/222: getdents db/d1b/d23 0 2021-11-03T17:14:07.234 INFO:tasks.workunit.client.0.smithi027.stdout:9/223: write db/d1b/d2b/f3f [4312880,126407] 0 2021-11-03T17:14:07.234 INFO:tasks.workunit.client.0.smithi027.stdout:9/224: read db/d1b/d1c/d1d/d40/f48 [467594,11879] 0 2021-11-03T17:14:07.235 INFO:tasks.workunit.client.0.smithi027.stdout:8/271: dwrite d7/d10/f15 [0,4194304] 0 2021-11-03T17:14:07.235 INFO:tasks.workunit.client.0.smithi027.stdout:9/225: creat db/d1b/f51 x:0 0 0 2021-11-03T17:14:07.237 INFO:tasks.workunit.client.0.smithi027.stdout:8/272: symlink d7/d10/d3f/l59 0 2021-11-03T17:14:07.237 INFO:tasks.workunit.client.0.smithi027.stdout:8/273: chown d7/d10/d18 133625467 1 2021-11-03T17:14:07.259 INFO:tasks.workunit.client.0.smithi027.stdout:8/274: dwrite d7/d10/d18/f33 [0,4194304] 0 2021-11-03T17:14:07.259 INFO:tasks.workunit.client.0.smithi027.stdout:8/275: readlink d7/l50 0 2021-11-03T17:14:07.259 INFO:tasks.workunit.client.0.smithi027.stdout:8/276: chown d7/d10/d18/d1a/l30 3 1 2021-11-03T17:14:07.260 INFO:tasks.workunit.client.0.smithi027.stdout:8/277: fsync d7/d10/d18/f33 0 2021-11-03T17:14:07.260 INFO:tasks.workunit.client.0.smithi027.stdout:8/278: dread - d7/f36 zero size 2021-11-03T17:14:07.260 INFO:tasks.workunit.client.0.smithi027.stdout:9/226: dwrite db/f16 [0,4194304] 0 2021-11-03T17:14:07.261 INFO:tasks.workunit.client.0.smithi027.stdout:9/227: mknod db/d1b/d1c/d1d/d40/c52 1 2021-11-03T17:14:07.261 INFO:tasks.workunit.client.0.smithi027.stdout:9/228: write db/f14 [1298801,71623] 0 2021-11-03T17:14:07.263 INFO:tasks.workunit.client.0.smithi027.stdout:9/229: creat db/d1b/d1c/d1d/d40/f53 x:0 0 0 2021-11-03T17:14:07.263 INFO:tasks.workunit.client.0.smithi027.stdout:9/230: fdatasync db/d1b/d1c/f39 0 2021-11-03T17:14:07.265 INFO:tasks.workunit.client.0.smithi027.stdout:9/231: read db/d1b/f20 [325936,61172] 0 2021-11-03T17:14:07.265 INFO:tasks.workunit.client.0.smithi027.stdout:9/232: mknod db/d1b/c54 1 2021-11-03T17:14:07.266 INFO:tasks.workunit.client.0.smithi027.stdout:9/233: creat db/d1b/d2b/d41/f55 x:0 0 0 2021-11-03T17:14:07.267 INFO:tasks.workunit.client.0.smithi027.stdout:9/234: symlink db/d1b/d1c/d1d/l56 0 2021-11-03T17:14:07.268 INFO:tasks.workunit.client.0.smithi027.stdout:9/235: truncate db/d1b/f38 3065771 0 2021-11-03T17:14:07.269 INFO:tasks.workunit.client.0.smithi027.stdout:9/236: dread - db/d1b/d2b/f4a zero size 2021-11-03T17:14:07.269 INFO:tasks.workunit.client.0.smithi027.stdout:9/237: mknod db/d1b/d2b/c57 1 2021-11-03T17:14:07.270 INFO:tasks.workunit.client.0.smithi027.stdout:9/238: symlink db/d1b/d2b/d41/l58 0 2021-11-03T17:14:07.270 INFO:tasks.workunit.client.0.smithi027.stdout:9/239: write db/d1b/d23/f4b [1744652,124905] 0 2021-11-03T17:14:07.273 INFO:tasks.workunit.client.0.smithi027.stdout:8/279: dwrite d7/d10/d18/f49 [0,4194304] 0 2021-11-03T17:14:07.274 INFO:tasks.workunit.client.0.smithi027.stdout:8/280: read d7/d10/d18/d1a/f26 [14243,54551] 0 2021-11-03T17:14:07.279 INFO:tasks.workunit.client.0.smithi027.stdout:8/281: link d7/l2f d7/l5a 0 2021-11-03T17:14:07.281 INFO:tasks.workunit.client.0.smithi027.stdout:8/282: rename d7/d10/d3f/l46 to d7/d10/d21/d41/l5b 0 2021-11-03T17:14:07.282 INFO:tasks.workunit.client.0.smithi027.stdout:8/283: stat d7/d10/d18/d1a/f26 0 2021-11-03T17:14:07.282 INFO:tasks.workunit.client.0.smithi027.stdout:8/284: symlink d7/l5c 0 2021-11-03T17:14:07.285 INFO:tasks.workunit.client.0.smithi027.stdout:9/240: dwrite db/d1b/d2b/f4a [0,4194304] 0 2021-11-03T17:14:07.286 INFO:tasks.workunit.client.0.smithi027.stdout:9/241: read db/d1b/d2b/f2a [2445954,45676] 0 2021-11-03T17:14:07.289 INFO:tasks.workunit.client.0.smithi027.stdout:9/242: symlink db/d1b/d1c/l59 0 2021-11-03T17:14:07.289 INFO:tasks.workunit.client.0.smithi027.stdout:9/243: fdatasync db/d1b/d23/d2c/f4c 0 2021-11-03T17:14:07.294 INFO:tasks.workunit.client.0.smithi027.stdout:1/236: sync 2021-11-03T17:14:07.294 INFO:tasks.workunit.client.0.smithi027.stdout:4/231: sync 2021-11-03T17:14:07.294 INFO:tasks.workunit.client.0.smithi027.stdout:5/288: sync 2021-11-03T17:14:07.295 INFO:tasks.workunit.client.0.smithi027.stdout:4/232: chown d6/d24 511168 1 2021-11-03T17:14:07.295 INFO:tasks.workunit.client.0.smithi027.stdout:4/233: fsync d6/d9/d16/f19 0 2021-11-03T17:14:07.295 INFO:tasks.workunit.client.0.smithi027.stdout:4/234: fsync d6/d9/d16/d36/f4c 0 2021-11-03T17:14:07.295 INFO:tasks.workunit.client.0.smithi027.stdout:4/235: mknod d6/db/d21/c4e 1 2021-11-03T17:14:07.296 INFO:tasks.workunit.client.0.smithi027.stdout:4/236: dread - d6/d9/f1b zero size 2021-11-03T17:14:07.298 INFO:tasks.workunit.client.0.smithi027.stdout:4/237: read d6/db/d21/f37 [177595,35559] 0 2021-11-03T17:14:07.298 INFO:tasks.workunit.client.0.smithi027.stdout:4/238: mknod d6/db/d21/d41/c4f 1 2021-11-03T17:14:07.298 INFO:tasks.workunit.client.0.smithi027.stdout:4/239: readlink d6/db/l4d 0 2021-11-03T17:14:07.298 INFO:tasks.workunit.client.0.smithi027.stdout:4/240: fdatasync d6/d9/d16/d17/f31 0 2021-11-03T17:14:07.299 INFO:tasks.workunit.client.0.smithi027.stdout:5/289: rename d2/d45 to d2/d2c/d3c/d48/d65 0 2021-11-03T17:14:07.299 INFO:tasks.workunit.client.0.smithi027.stdout:5/290: fdatasync d2/d4/df/f11 0 2021-11-03T17:14:07.299 INFO:tasks.workunit.client.0.smithi027.stdout:8/285: dwrite d7/d10/d18/d1a/f3c [0,4194304] 0 2021-11-03T17:14:07.300 INFO:tasks.workunit.client.0.smithi027.stdout:9/244: dread db/d1b/d2b/f4a [0,4194304] 0 2021-11-03T17:14:07.301 INFO:tasks.workunit.client.0.smithi027.stdout:1/237: creat d6/d11/d14/d1f/d37/d49/f50 x:0 0 0 2021-11-03T17:14:07.303 INFO:tasks.workunit.client.0.smithi027.stdout:5/291: creat d2/d4/f66 x:0 0 0 2021-11-03T17:14:07.304 INFO:tasks.workunit.client.0.smithi027.stdout:5/292: dread - d2/d4/d4e/f4f zero size 2021-11-03T17:14:07.308 INFO:tasks.workunit.client.0.smithi027.stdout:1/238: creat d6/de/d13/d15/f51 x:0 0 0 2021-11-03T17:14:07.311 INFO:tasks.workunit.client.0.smithi027.stdout:5/293: rename d2/d4/df to d2/d4/d4e/d67 0 2021-11-03T17:14:07.311 INFO:tasks.workunit.client.0.smithi027.stdout:5/294: write d2/f1e [757027,34208] 0 2021-11-03T17:14:07.326 INFO:tasks.workunit.client.0.smithi027.stdout:4/241: dwrite d6/db/d21/d2b/d2d/d3e/f47 [0,4194304] 0 2021-11-03T17:14:07.329 INFO:tasks.workunit.client.0.smithi027.stdout:9/245: dwrite db/f14 [0,4194304] 0 2021-11-03T17:14:07.330 INFO:tasks.workunit.client.0.smithi027.stdout:5/295: dwrite d2/d4/d4e/d67/f11 [0,4194304] 0 2021-11-03T17:14:07.331 INFO:tasks.workunit.client.0.smithi027.stdout:5/296: mknod d2/d4/d4e/d67/d20/d25/c68 1 2021-11-03T17:14:07.331 INFO:tasks.workunit.client.0.smithi027.stdout:5/297: write d2/f42 [956658,29179] 0 2021-11-03T17:14:07.331 INFO:tasks.workunit.client.0.smithi027.stdout:8/286: truncate d7/d10/d18/d1a/f3c 3782509 0 2021-11-03T17:14:07.363 INFO:tasks.workunit.client.0.smithi027.stdout:5/298: dwrite d2/d4/d4e/d67/f4a [4194304,4194304] 0 2021-11-03T17:14:07.363 INFO:tasks.workunit.client.0.smithi027.stdout:5/299: dread - d2/d4/d4e/d67/f2e zero size 2021-11-03T17:14:07.364 INFO:tasks.workunit.client.0.smithi027.stdout:8/287: dwrite d7/d2d/d38/f3d [0,4194304] 0 2021-11-03T17:14:07.364 INFO:tasks.workunit.client.0.smithi027.stdout:8/288: fsync d7/f1b 0 2021-11-03T17:14:07.365 INFO:tasks.workunit.client.0.smithi027.stdout:9/246: dwrite db/f14 [4194304,4194304] 0 2021-11-03T17:14:07.365 INFO:tasks.workunit.client.0.smithi027.stdout:9/247: mknod db/d1b/d2b/d41/c5a 1 2021-11-03T17:14:07.365 INFO:tasks.workunit.client.0.smithi027.stdout:5/300: getdents d2/d2c 0 2021-11-03T17:14:07.367 INFO:tasks.workunit.client.0.smithi027.stdout:9/248: getdents db/d1b/d23/d2c 0 2021-11-03T17:14:07.374 INFO:tasks.workunit.client.0.smithi027.stdout:5/301: write d2/d2c/d3c/f41 [1845973,121811] 0 2021-11-03T17:14:07.377 INFO:tasks.workunit.client.0.smithi027.stdout:5/302: unlink d2/d2c/d3c/f40 0 2021-11-03T17:14:07.380 INFO:tasks.workunit.client.0.smithi027.stdout:5/303: rename d2/d4/l10 to d2/d2c/l69 0 2021-11-03T17:14:07.380 INFO:tasks.workunit.client.0.smithi027.stdout:5/304: chown d2/d2c/d3c/l54 0 1 2021-11-03T17:14:07.382 INFO:tasks.workunit.client.0.smithi027.stdout:8/289: dwrite d7/d2d/d38/f3d [0,4194304] 0 2021-11-03T17:14:07.383 INFO:tasks.workunit.client.0.smithi027.stdout:8/290: symlink d7/d53/l5d 0 2021-11-03T17:14:07.385 INFO:tasks.workunit.client.0.smithi027.stdout:8/291: symlink d7/d10/d4f/l5e 0 2021-11-03T17:14:07.388 INFO:tasks.workunit.client.0.smithi027.stdout:8/292: rename d7/d10/d18/d1a/l30 to d7/d10/d21/d41/l5f 0 2021-11-03T17:14:07.392 INFO:tasks.workunit.client.0.smithi027.stdout:9/249: dwrite db/d1b/f38 [0,4194304] 0 2021-11-03T17:14:07.392 INFO:tasks.workunit.client.0.smithi027.stdout:9/250: chown db/d1b/d2b/f2a 27 1 2021-11-03T17:14:07.398 INFO:tasks.workunit.client.0.smithi027.stdout:8/293: getdents d7/d10/d18/d1a 0 2021-11-03T17:14:07.399 INFO:tasks.workunit.client.0.smithi027.stdout:8/294: dread - d7/d10/d18/f48 zero size 2021-11-03T17:14:07.400 INFO:tasks.workunit.client.0.smithi027.stdout:8/295: truncate d7/d10/d18/d1a/f3e 1652326 0 2021-11-03T17:14:07.400 INFO:tasks.workunit.client.0.smithi027.stdout:5/305: dwrite d2/f36 [0,4194304] 0 2021-11-03T17:14:07.402 INFO:tasks.workunit.client.0.smithi027.stdout:5/306: read d2/d4/f34 [4194413,43475] 0 2021-11-03T17:14:07.424 INFO:tasks.workunit.client.0.smithi027.stdout:9/251: dwrite db/d1b/d1c/d1d/f37 [0,4194304] 0 2021-11-03T17:14:07.424 INFO:tasks.workunit.client.0.smithi027.stdout:5/307: dwrite d2/d4/fe [4194304,4194304] 0 2021-11-03T17:14:07.425 INFO:tasks.workunit.client.0.smithi027.stdout:8/296: dwrite d7/d10/d18/d1a/f2e [4194304,4194304] 0 2021-11-03T17:14:07.425 INFO:tasks.workunit.client.0.smithi027.stdout:8/297: chown d7/l2f 46815 1 2021-11-03T17:14:07.425 INFO:tasks.workunit.client.0.smithi027.stdout:8/298: fsync d7/d10/d18/d1a/f3e 0 2021-11-03T17:14:07.426 INFO:tasks.workunit.client.0.smithi027.stdout:9/252: dread db/f13 [0,4194304] 0 2021-11-03T17:14:07.426 INFO:tasks.workunit.client.0.smithi027.stdout:9/253: write db/d1b/d1c/d1d/f49 [573824,101293] 0 2021-11-03T17:14:07.427 INFO:tasks.workunit.client.0.smithi027.stdout:5/308: dread d2/d4/d4e/d67/d20/f3a [0,4194304] 0 2021-11-03T17:14:07.434 INFO:tasks.workunit.client.0.smithi027.stdout:5/309: mkdir d2/d4/d4e/d67/d20/d25/d6a 0 2021-11-03T17:14:07.436 INFO:tasks.workunit.client.0.smithi027.stdout:5/310: creat d2/d4/f6b x:0 0 0 2021-11-03T17:14:07.436 INFO:tasks.workunit.client.0.smithi027.stdout:5/311: chown d2/d2c/d3c/d48 1013956 1 2021-11-03T17:14:07.436 INFO:tasks.workunit.client.0.smithi027.stdout:5/312: mknod d2/d1f/c6c 1 2021-11-03T17:14:07.436 INFO:tasks.workunit.client.0.smithi027.stdout:5/313: write d2/f42 [1985684,110730] 0 2021-11-03T17:14:07.438 INFO:tasks.workunit.client.0.smithi027.stdout:5/314: rename d2/d4/d4e/d67/l57 to d2/d1f/l6d 0 2021-11-03T17:14:07.439 INFO:tasks.workunit.client.0.smithi027.stdout:5/315: creat d2/d2c/d3c/d58/f6e x:0 0 0 2021-11-03T17:14:07.444 INFO:tasks.workunit.client.0.smithi027.stdout:8/299: dwrite d7/f8 [0,4194304] 0 2021-11-03T17:14:07.446 INFO:tasks.workunit.client.0.smithi027.stdout:8/300: creat d7/d10/d18/d1a/f60 x:0 0 0 2021-11-03T17:14:07.448 INFO:tasks.workunit.client.0.smithi027.stdout:9/254: dwrite db/d1b/d1c/d1d/f37 [4194304,4194304] 0 2021-11-03T17:14:07.449 INFO:tasks.workunit.client.0.smithi027.stdout:9/255: write db/d1b/d1c/d1d/f21 [411731,33101] 0 2021-11-03T17:14:07.450 INFO:tasks.workunit.client.0.smithi027.stdout:9/256: creat db/f5b x:0 0 0 2021-11-03T17:14:07.452 INFO:tasks.workunit.client.0.smithi027.stdout:9/257: symlink db/d1b/d23/l5c 0 2021-11-03T17:14:07.452 INFO:tasks.workunit.client.0.smithi027.stdout:9/258: mknod db/d1b/c5d 1 2021-11-03T17:14:07.468 INFO:tasks.workunit.client.0.smithi027.stdout:9/259: dwrite db/d1b/d1c/d1d/d40/f48 [0,4194304] 0 2021-11-03T17:14:07.470 INFO:tasks.workunit.client.0.smithi027.stdout:9/260: dread db/d1b/d23/f4b [0,4194304] 0 2021-11-03T17:14:07.471 INFO:tasks.workunit.client.0.smithi027.stdout:9/261: mkdir db/d1b/d2b/d5e 0 2021-11-03T17:14:07.472 INFO:tasks.workunit.client.0.smithi027.stdout:9/262: write db/d1b/d1c/f3e [1140509,268] 0 2021-11-03T17:14:07.472 INFO:tasks.workunit.client.0.smithi027.stdout:9/263: chown db/l1a 1239 1 2021-11-03T17:14:07.473 INFO:tasks.workunit.client.0.smithi027.stdout:9/264: mknod db/d1b/d2b/d5e/c5f 1 2021-11-03T17:14:07.485 INFO:tasks.workunit.client.0.smithi027.stdout:9/265: dwrite db/f5b [0,4194304] 0 2021-11-03T17:14:07.492 INFO:tasks.workunit.client.0.smithi027.stdout:9/266: getdents db/d1b/d1c/d1d 0 2021-11-03T17:14:07.496 INFO:tasks.workunit.client.0.smithi027.stdout:9/267: mkdir db/d1b/d1c/d1d/d40/d60 0 2021-11-03T17:14:07.499 INFO:tasks.workunit.client.0.smithi027.stdout:9/268: rename db/d1b/d1c/l59 to db/d1b/l61 0 2021-11-03T17:14:07.499 INFO:tasks.workunit.client.0.smithi027.stdout:9/269: mknod db/d1b/d1c/d1d/c62 1 2021-11-03T17:14:07.500 INFO:tasks.workunit.client.0.smithi027.stdout:9/270: truncate db/d1b/d1c/d1d/f4f 411777 0 2021-11-03T17:14:07.500 INFO:tasks.workunit.client.0.smithi027.stdout:9/271: write db/d1b/f38 [3440740,72066] 0 2021-11-03T17:14:07.501 INFO:tasks.workunit.client.0.smithi027.stdout:9/272: creat db/d1b/d2b/d41/f63 x:0 0 0 2021-11-03T17:14:07.501 INFO:tasks.workunit.client.0.smithi027.stdout:9/273: chown db/l1a 23914055 1 2021-11-03T17:14:07.502 INFO:tasks.workunit.client.0.smithi027.stdout:9/274: creat db/d1b/f64 x:0 0 0 2021-11-03T17:14:07.503 INFO:tasks.workunit.client.0.smithi027.stdout:9/275: write db/f17 [3090675,24709] 0 2021-11-03T17:14:07.504 INFO:tasks.workunit.client.0.smithi027.stdout:9/276: creat db/d1b/d1c/f65 x:0 0 0 2021-11-03T17:14:07.505 INFO:tasks.workunit.client.0.smithi027.stdout:9/277: creat db/f66 x:0 0 0 2021-11-03T17:14:07.507 INFO:tasks.workunit.client.0.smithi027.stdout:9/278: link db/d1b/d2b/d41/l42 db/d1b/d1c/l67 0 2021-11-03T17:14:07.507 INFO:tasks.workunit.client.0.smithi027.stdout:9/279: chown db/d1b/d2b/f2a 1122521 1 2021-11-03T17:14:07.517 INFO:tasks.workunit.client.0.smithi027.stdout:9/280: rmdir db/d1b/d2b/d41 39 2021-11-03T17:14:07.517 INFO:tasks.workunit.client.0.smithi027.stdout:9/281: chown db/f1f 414121127 1 2021-11-03T17:14:07.517 INFO:tasks.workunit.client.0.smithi027.stdout:9/282: mknod db/d1b/d1c/d1d/c68 1 2021-11-03T17:14:07.517 INFO:tasks.workunit.client.0.smithi027.stdout:9/283: fdatasync db/d1b/d1c/d1d/d40/f53 0 2021-11-03T17:14:07.518 INFO:tasks.workunit.client.0.smithi027.stdout:9/284: fdatasync db/d1b/d1c/d1d/f49 0 2021-11-03T17:14:07.518 INFO:tasks.workunit.client.0.smithi027.stdout:9/285: write db/d1b/d23/d2c/f4c [467013,79117] 0 2021-11-03T17:14:07.518 INFO:tasks.workunit.client.0.smithi027.stdout:9/286: write db/d1b/d1c/d1d/d40/f53 [930560,40090] 0 2021-11-03T17:14:07.532 INFO:tasks.workunit.client.0.smithi027.stdout:9/287: dwrite db/d1b/d1c/d1d/f21 [0,4194304] 0 2021-11-03T17:14:07.542 INFO:tasks.workunit.client.0.smithi027.stdout:9/288: dread db/d1b/d1c/d1d/f21 [0,4194304] 0 2021-11-03T17:14:07.542 INFO:tasks.workunit.client.0.smithi027.stdout:9/289: mknod db/d1b/d1c/d1d/d40/c69 1 2021-11-03T17:14:07.543 INFO:tasks.workunit.client.0.smithi027.stdout:9/290: stat db/d1b/d1c/d1d/d40/d60 0 2021-11-03T17:14:07.557 INFO:tasks.workunit.client.0.smithi027.stdout:9/291: dwrite db/d1b/d1c/f65 [0,4194304] 0 2021-11-03T17:14:07.560 INFO:tasks.workunit.client.0.smithi027.stdout:9/292: creat db/d1b/d23/d2c/f6a x:0 0 0 2021-11-03T17:14:07.563 INFO:tasks.workunit.client.0.smithi027.stdout:9/293: link db/f17 db/d1b/d2b/d5e/f6b 0 2021-11-03T17:14:07.563 INFO:tasks.workunit.client.0.smithi027.stdout:9/294: chown db/d1b/d1c/f25 177 1 2021-11-03T17:14:07.567 INFO:tasks.workunit.client.0.smithi027.stdout:9/295: rmdir db/d1b/d2b 39 2021-11-03T17:14:07.567 INFO:tasks.workunit.client.0.smithi027.stdout:9/296: getdents db/d1b/d1c/d1d/d40/d60 0 2021-11-03T17:14:07.567 INFO:tasks.workunit.client.0.smithi027.stdout:9/297: stat db/fd 0 2021-11-03T17:14:07.568 INFO:tasks.workunit.client.0.smithi027.stdout:9/298: mkdir db/d1b/d2b/d41/d6c 0 2021-11-03T17:14:07.574 INFO:tasks.workunit.client.0.smithi027.stdout:9/299: dread db/f14 [0,4194304] 0 2021-11-03T17:14:07.576 INFO:tasks.workunit.client.0.smithi027.stdout:9/300: rename db/d1b/d2b/d41/f63 to db/d1b/d1c/d1d/f6d 0 2021-11-03T17:14:07.578 INFO:tasks.workunit.client.0.smithi027.stdout:9/301: truncate db/d1b/f20 472872 0 2021-11-03T17:14:07.579 INFO:tasks.workunit.client.0.smithi027.stdout:9/302: symlink db/d1b/d2b/l6e 0 2021-11-03T17:14:07.579 INFO:tasks.workunit.client.0.smithi027.stdout:9/303: mknod db/d1b/d1c/d1d/c6f 1 2021-11-03T17:14:07.580 INFO:tasks.workunit.client.0.smithi027.stdout:9/304: creat db/d1b/d1c/d1d/d40/f70 x:0 0 0 2021-11-03T17:14:07.580 INFO:tasks.workunit.client.0.smithi027.stdout:9/305: write db/f16 [5039719,86176] 0 2021-11-03T17:14:07.583 INFO:tasks.workunit.client.0.smithi027.stdout:9/306: getdents db/d1b/d2b/d5e 0 2021-11-03T17:14:07.585 INFO:tasks.workunit.client.0.smithi027.stdout:9/307: mkdir db/d1b/d1c/d1d/d40/d60/d71 0 2021-11-03T17:14:07.585 INFO:tasks.workunit.client.0.smithi027.stdout:9/308: getdents db/d1b/d1c/d1d/d40/d60/d71 0 2021-11-03T17:14:07.586 INFO:tasks.workunit.client.0.smithi027.stdout:9/309: symlink db/d1b/d1c/d1d/l72 0 2021-11-03T17:14:07.588 INFO:tasks.workunit.client.0.smithi027.stdout:9/310: creat db/d1b/d23/f73 x:0 0 0 2021-11-03T17:14:07.588 INFO:tasks.workunit.client.0.smithi027.stdout:9/311: write db/f14 [58501,77514] 0 2021-11-03T17:14:07.588 INFO:tasks.workunit.client.0.smithi027.stdout:9/312: write db/d1b/d1c/d1d/d40/f70 [101056,125432] 0 2021-11-03T17:14:07.589 INFO:tasks.workunit.client.0.smithi027.stdout:9/313: stat db/d1b/d2b/l6e 0 2021-11-03T17:14:07.590 INFO:tasks.workunit.client.0.smithi027.stdout:9/314: creat db/d1b/d2b/d41/f74 x:0 0 0 2021-11-03T17:14:07.590 INFO:tasks.workunit.client.0.smithi027.stdout:9/315: chown db/d1b/d2b/f2a 12285 1 2021-11-03T17:14:07.591 INFO:tasks.workunit.client.0.smithi027.stdout:9/316: creat db/f75 x:0 0 0 2021-11-03T17:14:07.600 INFO:tasks.workunit.client.0.smithi027.stdout:9/317: dread db/d1b/d1c/d1d/f37 [4194304,4194304] 0 2021-11-03T17:14:07.601 INFO:tasks.workunit.client.0.smithi027.stdout:9/318: dread db/d1b/d1c/d1d/f4f [0,4194304] 0 2021-11-03T17:14:07.603 INFO:tasks.workunit.client.0.smithi027.stdout:9/319: rename db/d1b/d2b/l4e to db/d1b/d2b/d5e/l76 0 2021-11-03T17:14:07.605 INFO:tasks.workunit.client.0.smithi027.stdout:9/320: link db/d1b/d2b/l6e db/l77 0 2021-11-03T17:14:07.606 INFO:tasks.workunit.client.0.smithi027.stdout:9/321: creat db/d1b/d1c/d1d/f78 x:0 0 0 2021-11-03T17:14:07.606 INFO:tasks.workunit.client.0.smithi027.stdout:9/322: chown db/d1b/d1c/d1d/l72 1324 1 2021-11-03T17:14:07.607 INFO:tasks.workunit.client.0.smithi027.stdout:9/323: write db/f14 [1517011,8729] 0 2021-11-03T17:14:07.609 INFO:tasks.workunit.client.0.smithi027.stdout:9/324: rename db/d1b/d1c/l67 to db/d1b/d1c/l79 0 2021-11-03T17:14:07.628 INFO:tasks.workunit.client.0.smithi027.stdout:9/325: dwrite db/d1b/d1c/d1d/d40/f53 [0,4194304] 0 2021-11-03T17:14:07.629 INFO:tasks.workunit.client.0.smithi027.stdout:9/326: stat db/fd 0 2021-11-03T17:14:07.644 INFO:tasks.workunit.client.0.smithi027.stdout:9/327: dwrite db/d1b/d1c/d1d/f4f [0,4194304] 0 2021-11-03T17:14:07.648 INFO:tasks.workunit.client.0.smithi027.stdout:9/328: rename db/d1b/d2b/d41/f55 to db/d1b/d23/f7a 0 2021-11-03T17:14:07.648 INFO:tasks.workunit.client.0.smithi027.stdout:9/329: chown db/d1b/d1c/d1d/f1e 1504 1 2021-11-03T17:14:07.648 INFO:tasks.workunit.client.0.smithi027.stdout:9/330: chown db/d1b/d23/f73 56 1 2021-11-03T17:14:07.666 INFO:tasks.workunit.client.0.smithi027.stdout:9/331: dwrite db/d1b/d23/f73 [0,4194304] 0 2021-11-03T17:14:07.666 INFO:tasks.workunit.client.0.smithi027.stdout:9/332: mknod db/d1b/d23/d2c/c7b 1 2021-11-03T17:14:07.666 INFO:tasks.workunit.client.0.smithi027.stdout:9/333: dread - db/d1b/d2b/d41/f74 zero size 2021-11-03T17:14:07.666 INFO:tasks.workunit.client.0.smithi027.stdout:9/334: truncate db/d1b/d1c/d1d/f21 4799648 0 2021-11-03T17:14:07.669 INFO:tasks.workunit.client.0.smithi027.stdout:9/335: rename db/d1b/d1c/f39 to db/d1b/d1c/d1d/d40/d60/f7c 0 2021-11-03T17:14:07.677 INFO:tasks.workunit.client.0.smithi027.stdout:0/262: sync 2021-11-03T17:14:07.679 INFO:tasks.workunit.client.0.smithi027.stdout:0/263: symlink d9/da/d31/l58 0 2021-11-03T17:14:07.681 INFO:tasks.workunit.client.0.smithi027.stdout:0/264: symlink d9/da/d30/l59 0 2021-11-03T17:14:07.681 INFO:tasks.workunit.client.0.smithi027.stdout:0/265: stat d9/da/f39 0 2021-11-03T17:14:07.681 INFO:tasks.workunit.client.0.smithi027.stdout:0/266: mknod d9/d12/d56/c5a 1 2021-11-03T17:14:07.681 INFO:tasks.workunit.client.0.smithi027.stdout:0/267: readlink d9/l52 0 2021-11-03T17:14:07.682 INFO:tasks.workunit.client.0.smithi027.stdout:0/268: write d9/d12/f3b [942903,66534] 0 2021-11-03T17:14:07.682 INFO:tasks.workunit.client.0.smithi027.stdout:0/269: chown d9/d12 36918 1 2021-11-03T17:14:07.686 INFO:tasks.workunit.client.0.smithi027.stdout:9/336: dwrite db/d1b/d23/f73 [0,4194304] 0 2021-11-03T17:14:07.686 INFO:tasks.workunit.client.0.smithi027.stdout:0/270: dread d9/da/d30/f43 [0,4194304] 0 2021-11-03T17:14:07.687 INFO:tasks.workunit.client.0.smithi027.stdout:0/271: mknod d9/c5b 1 2021-11-03T17:14:07.687 INFO:tasks.workunit.client.0.smithi027.stdout:0/272: dread - d9/f4f zero size 2021-11-03T17:14:07.687 INFO:tasks.workunit.client.0.smithi027.stdout:0/273: read - d9/f47 zero size 2021-11-03T17:14:07.687 INFO:tasks.workunit.client.0.smithi027.stdout:0/274: fdatasync d9/f4d 0 2021-11-03T17:14:07.689 INFO:tasks.workunit.client.0.smithi027.stdout:0/275: creat d9/da/d31/f5c x:0 0 0 2021-11-03T17:14:07.692 INFO:tasks.workunit.client.0.smithi027.stdout:0/276: dread d9/d12/f28 [0,4194304] 0 2021-11-03T17:14:07.693 INFO:tasks.workunit.client.0.smithi027.stdout:0/277: getdents d9/d12 0 2021-11-03T17:14:07.693 INFO:tasks.workunit.client.0.smithi027.stdout:0/278: read - d9/da/d30/f3e zero size 2021-11-03T17:14:07.694 INFO:tasks.workunit.client.0.smithi027.stdout:0/279: creat d9/da/f5d x:0 0 0 2021-11-03T17:14:07.695 INFO:tasks.workunit.client.0.smithi027.stdout:0/280: creat d9/da/d30/f5e x:0 0 0 2021-11-03T17:14:07.695 INFO:tasks.workunit.client.0.smithi027.stdout:0/281: chown d9/da/l29 958209 1 2021-11-03T17:14:07.703 INFO:tasks.workunit.client.0.smithi027.stdout:9/337: dwrite db/f5b [4194304,4194304] 0 2021-11-03T17:14:07.705 INFO:tasks.workunit.client.0.smithi027.stdout:9/338: creat db/d1b/d23/d2c/f7d x:0 0 0 2021-11-03T17:14:07.705 INFO:tasks.workunit.client.0.smithi027.stdout:9/339: fsync db/d1b/d23/d2c/f6a 0 2021-11-03T17:14:07.717 INFO:tasks.workunit.client.0.smithi027.stdout:9/340: rmdir db/d1b/d23/d2c 39 2021-11-03T17:14:07.720 INFO:tasks.workunit.client.0.smithi027.stdout:0/282: dwrite d9/d12/f3b [0,4194304] 0 2021-11-03T17:14:07.756 INFO:tasks.workunit.client.0.smithi027.stdout:0/283: dread d9/da/d31/f36 [0,4194304] 0 2021-11-03T17:14:07.758 INFO:tasks.workunit.client.0.smithi027.stdout:0/284: mkdir d9/d12/d56/d5f 0 2021-11-03T17:14:07.759 INFO:tasks.workunit.client.0.smithi027.stdout:0/285: rmdir d9/d12 39 2021-11-03T17:14:07.771 INFO:tasks.workunit.client.0.smithi027.stdout:0/286: dwrite d9/da/d30/f54 [0,4194304] 0 2021-11-03T17:14:07.773 INFO:tasks.workunit.client.0.smithi027.stdout:0/287: truncate d9/da/f16 3176830 0 2021-11-03T17:14:07.774 INFO:tasks.workunit.client.0.smithi027.stdout:0/288: fsync d9/da/d31/f48 0 2021-11-03T17:14:07.780 INFO:tasks.workunit.client.0.smithi027.stdout:0/289: write d9/da/f16 [1773364,11984] 0 2021-11-03T17:14:07.780 INFO:tasks.workunit.client.0.smithi027.stdout:0/290: dread - d9/da/d31/f5c zero size 2021-11-03T17:14:07.780 INFO:tasks.workunit.client.0.smithi027.stdout:0/291: readlink d9/l52 0 2021-11-03T17:14:07.791 INFO:tasks.workunit.client.0.smithi027.stdout:0/292: dread d9/da/fd [0,4194304] 0 2021-11-03T17:14:07.791 INFO:tasks.workunit.client.0.smithi027.stdout:0/293: write d9/da/d30/f3e [117305,124148] 0 2021-11-03T17:14:07.791 INFO:tasks.workunit.client.0.smithi027.stdout:0/294: mknod d9/da/d31/c60 1 2021-11-03T17:14:07.792 INFO:tasks.workunit.client.0.smithi027.stdout:0/295: unlink d9/da/f5d 0 2021-11-03T17:14:07.794 INFO:tasks.workunit.client.0.smithi027.stdout:0/296: symlink d9/da/d31/l61 0 2021-11-03T17:14:07.794 INFO:tasks.workunit.client.0.smithi027.stdout:0/297: chown d9/da/l29 47 1 2021-11-03T17:14:07.795 INFO:tasks.workunit.client.0.smithi027.stdout:0/298: mkdir d9/da/d30/d62 0 2021-11-03T17:14:07.795 INFO:tasks.workunit.client.0.smithi027.stdout:0/299: stat d9 0 2021-11-03T17:14:07.797 INFO:tasks.workunit.client.0.smithi027.stdout:0/300: dread d9/da/d30/f3c [0,4194304] 0 2021-11-03T17:14:07.799 INFO:tasks.workunit.client.0.smithi027.stdout:0/301: rmdir d9/da/d30/d62 0 2021-11-03T17:14:07.799 INFO:tasks.workunit.client.0.smithi027.stdout:0/302: mknod d9/d12/d56/c63 1 2021-11-03T17:14:07.800 INFO:tasks.workunit.client.0.smithi027.stdout:0/303: mkdir d9/da/d64 0 2021-11-03T17:14:07.800 INFO:tasks.workunit.client.0.smithi027.stdout:0/304: dread - d9/da/d31/f5c zero size 2021-11-03T17:14:07.800 INFO:tasks.workunit.client.0.smithi027.stdout:0/305: chown d9/da/fe 3809 1 2021-11-03T17:14:07.801 INFO:tasks.workunit.client.0.smithi027.stdout:0/306: write d9/f4d [2020061,54172] 0 2021-11-03T17:14:07.802 INFO:tasks.workunit.client.0.smithi027.stdout:0/307: chown d9/d12/l45 2 1 2021-11-03T17:14:07.803 INFO:tasks.workunit.client.0.smithi027.stdout:0/308: read d9/da/d30/f3c [181516,102935] 0 2021-11-03T17:14:07.803 INFO:tasks.workunit.client.0.smithi027.stdout:0/309: chown d9/l52 342 1 2021-11-03T17:14:07.803 INFO:tasks.workunit.client.0.smithi027.stdout:0/310: symlink d9/d12/l65 0 2021-11-03T17:14:07.818 INFO:tasks.workunit.client.0.smithi027.stdout:0/311: dwrite d9/d12/f1d [0,4194304] 0 2021-11-03T17:14:07.820 INFO:tasks.workunit.client.0.smithi027.stdout:0/312: creat d9/da/d64/f66 x:0 0 0 2021-11-03T17:14:07.820 INFO:tasks.workunit.client.0.smithi027.stdout:0/313: mknod d9/da/d30/c67 1 2021-11-03T17:14:07.832 INFO:tasks.workunit.client.0.smithi027.stdout:0/314: dwrite d9/f47 [0,4194304] 0 2021-11-03T17:14:07.835 INFO:tasks.workunit.client.0.smithi027.stdout:0/315: creat d9/da/d64/f68 x:0 0 0 2021-11-03T17:14:07.835 INFO:tasks.workunit.client.0.smithi027.stdout:0/316: stat d9/da/fd 0 2021-11-03T17:14:07.836 INFO:tasks.workunit.client.0.smithi027.stdout:0/317: write d9/da/d31/f34 [3175534,126047] 0 2021-11-03T17:14:07.841 INFO:tasks.workunit.client.0.smithi027.stdout:0/318: symlink d9/d12/l69 0 2021-11-03T17:14:07.841 INFO:tasks.workunit.client.0.smithi027.stdout:0/319: readlink d9/l21 0 2021-11-03T17:14:07.842 INFO:tasks.workunit.client.0.smithi027.stdout:0/320: stat d9/da/d31/f48 0 2021-11-03T17:14:07.842 INFO:tasks.workunit.client.0.smithi027.stdout:0/321: chown d9/f1f 0 1 2021-11-03T17:14:07.843 INFO:tasks.workunit.client.0.smithi027.stdout:0/322: creat d9/d12/d56/f6a x:0 0 0 2021-11-03T17:14:07.843 INFO:tasks.workunit.client.0.smithi027.stdout:0/323: truncate d9/da/d64/f66 586262 0 2021-11-03T17:14:07.843 INFO:tasks.workunit.client.0.smithi027.stdout:0/324: chown d9/da/d31/f36 0 1 2021-11-03T17:14:07.844 INFO:tasks.workunit.client.0.smithi027.stdout:0/325: symlink d9/da/d64/l6b 0 2021-11-03T17:14:07.845 INFO:tasks.workunit.client.0.smithi027.stdout:0/326: link d9/da/d31/f48 d9/f6c 0 2021-11-03T17:14:07.846 INFO:tasks.workunit.client.0.smithi027.stdout:0/327: mkdir d9/d12/d56/d6d 0 2021-11-03T17:14:07.876 INFO:tasks.workunit.client.0.smithi027.stdout:0/328: dwrite d9/da/d30/f5e [0,4194304] 0 2021-11-03T17:14:07.879 INFO:tasks.workunit.client.0.smithi027.stdout:0/329: dread d9/f24 [0,4194304] 0 2021-11-03T17:14:07.879 INFO:tasks.workunit.client.0.smithi027.stdout:0/330: stat d9/d12/d56/d5f 0 2021-11-03T17:14:07.879 INFO:tasks.workunit.client.0.smithi027.stdout:0/331: write d9/da/d64/f68 [25266,74301] 0 2021-11-03T17:14:07.880 INFO:tasks.workunit.client.0.smithi027.stdout:0/332: truncate d9/da/fd 835424 0 2021-11-03T17:14:07.880 INFO:tasks.workunit.client.0.smithi027.stdout:0/333: write d9/f6c [1370720,99128] 0 2021-11-03T17:14:07.888 INFO:tasks.workunit.client.0.smithi027.stdout:0/334: dread d9/da/d31/f44 [0,4194304] 0 2021-11-03T17:14:07.888 INFO:tasks.workunit.client.0.smithi027.stdout:0/335: readlink d9/l52 0 2021-11-03T17:14:07.890 INFO:tasks.workunit.client.0.smithi027.stdout:0/336: creat d9/d12/d56/d6d/f6e x:0 0 0 2021-11-03T17:14:07.891 INFO:tasks.workunit.client.0.smithi027.stdout:0/337: mkdir d9/d12/d56/d6f 0 2021-11-03T17:14:07.896 INFO:tasks.workunit.client.0.smithi027.stdout:0/338: rename d9/d12/d56 to d9/da/d64/d70 0 2021-11-03T17:14:07.897 INFO:tasks.workunit.client.0.smithi027.stdout:0/339: read d9/f1f [2915458,120104] 0 2021-11-03T17:14:07.898 INFO:tasks.workunit.client.0.smithi027.stdout:0/340: stat d9/d12/f3b 0 2021-11-03T17:14:07.898 INFO:tasks.workunit.client.0.smithi027.stdout:0/341: stat d9/da/d64 0 2021-11-03T17:14:07.898 INFO:tasks.workunit.client.0.smithi027.stdout:0/342: mknod d9/da/d64/d70/c71 1 2021-11-03T17:14:07.899 INFO:tasks.workunit.client.0.smithi027.stdout:0/343: readlink l2 0 2021-11-03T17:14:07.900 INFO:tasks.workunit.client.0.smithi027.stdout:0/344: dread d9/da/d31/f44 [0,4194304] 0 2021-11-03T17:14:07.906 INFO:tasks.workunit.client.0.smithi027.stdout:0/345: rmdir d9/da 39 2021-11-03T17:14:07.906 INFO:tasks.workunit.client.0.smithi027.stdout:0/346: chown d9/da/f38 0 1 2021-11-03T17:14:07.906 INFO:tasks.workunit.client.0.smithi027.stdout:0/347: write d9/da/d30/f3e [972982,20388] 0 2021-11-03T17:14:07.908 INFO:tasks.workunit.client.0.smithi027.stdout:0/348: rename d9/d12/f1e to d9/f72 0 2021-11-03T17:14:07.908 INFO:tasks.workunit.client.0.smithi027.stdout:0/349: mknod d9/da/d64/d70/d6d/c73 1 2021-11-03T17:14:07.911 INFO:tasks.workunit.client.0.smithi027.stdout:0/350: getdents d9/d12 0 2021-11-03T17:14:07.913 INFO:tasks.workunit.client.0.smithi027.stdout:0/351: rename d9/da/d64/f66 to d9/da/d64/d70/d5f/f74 0 2021-11-03T17:14:07.914 INFO:tasks.workunit.client.0.smithi027.stdout:0/352: write d9/da/d30/f3c [476410,14696] 0 2021-11-03T17:14:07.915 INFO:tasks.workunit.client.0.smithi027.stdout:0/353: creat d9/d12/f75 x:0 0 0 2021-11-03T17:14:07.917 INFO:tasks.workunit.client.0.smithi027.stdout:0/354: creat d9/da/d64/f76 x:0 0 0 2021-11-03T17:14:07.917 INFO:tasks.workunit.client.0.smithi027.stdout:0/355: chown d9/f72 494 1 2021-11-03T17:14:07.917 INFO:tasks.workunit.client.0.smithi027.stdout:0/356: fdatasync d9/f49 0 2021-11-03T17:14:07.918 INFO:tasks.workunit.client.0.smithi027.stdout:0/357: truncate d9/da/d30/f5e 153642 0 2021-11-03T17:14:07.918 INFO:tasks.workunit.client.0.smithi027.stdout:0/358: chown d9/da/d31/f48 4097729 1 2021-11-03T17:14:07.919 INFO:tasks.workunit.client.0.smithi027.stdout:0/359: readlink l7 0 2021-11-03T17:14:07.919 INFO:tasks.workunit.client.0.smithi027.stdout:0/360: read d9/da/f16 [1980891,28965] 0 2021-11-03T17:14:07.920 INFO:tasks.workunit.client.0.smithi027.stdout:0/361: mknod d9/da/d31/c77 1 2021-11-03T17:14:07.920 INFO:tasks.workunit.client.0.smithi027.stdout:0/362: creat d9/da/f78 x:0 0 0 2021-11-03T17:14:07.923 INFO:tasks.workunit.client.0.smithi027.stdout:0/363: mkdir d9/da/d64/d70/d79 0 2021-11-03T17:14:07.925 INFO:tasks.workunit.client.0.smithi027.stdout:0/364: mkdir d9/da/d64/d70/d6d/d7a 0 2021-11-03T17:14:07.938 INFO:tasks.workunit.client.0.smithi027.stdout:0/365: dwrite d9/da/d30/f5e [0,4194304] 0 2021-11-03T17:14:07.940 INFO:tasks.workunit.client.0.smithi027.stdout:0/366: rmdir d9/da 39 2021-11-03T17:14:07.940 INFO:tasks.workunit.client.0.smithi027.stdout:0/367: chown d9/da/d31/l50 5977166 1 2021-11-03T17:14:07.940 INFO:tasks.workunit.client.0.smithi027.stdout:0/368: mknod d9/da/d64/d70/d5f/c7b 1 2021-11-03T17:14:07.940 INFO:tasks.workunit.client.0.smithi027.stdout:0/369: mknod d9/da/d64/d70/d79/c7c 1 2021-11-03T17:14:07.942 INFO:tasks.workunit.client.0.smithi027.stdout:0/370: link d9/da/d64/f76 d9/da/d64/d70/d6d/f7d 0 2021-11-03T17:14:07.943 INFO:tasks.workunit.client.0.smithi027.stdout:0/371: symlink d9/da/d64/d70/l7e 0 2021-11-03T17:14:07.944 INFO:tasks.workunit.client.0.smithi027.stdout:0/372: symlink d9/da/d64/d70/d79/l7f 0 2021-11-03T17:14:07.946 INFO:tasks.workunit.client.0.smithi027.stdout:0/373: rename d9/da/f2d to d9/da/d64/d70/d6f/f80 0 2021-11-03T17:14:07.948 INFO:tasks.workunit.client.0.smithi027.stdout:0/374: unlink d9/da/d64/d70/l7e 0 2021-11-03T17:14:07.948 INFO:tasks.workunit.client.0.smithi027.stdout:0/375: chown d9/l52 829165483 1 2021-11-03T17:14:07.948 INFO:tasks.workunit.client.0.smithi027.stdout:0/376: rmdir d9/da/d64/d70/d6d 39 2021-11-03T17:14:07.950 INFO:tasks.workunit.client.0.smithi027.stdout:0/377: mkdir d9/d81 0 2021-11-03T17:14:07.951 INFO:tasks.workunit.client.0.smithi027.stdout:0/378: rmdir d9/da/d64/d70/d6d 39 2021-11-03T17:14:07.951 INFO:tasks.workunit.client.0.smithi027.stdout:0/379: readlink d9/da/d30/l59 0 2021-11-03T17:14:07.951 INFO:tasks.workunit.client.0.smithi027.stdout:0/380: fdatasync d9/f4f 0 2021-11-03T17:14:07.951 INFO:tasks.workunit.client.0.smithi027.stdout:0/381: truncate d9/da/d30/f54 4745711 0 2021-11-03T17:14:08.126 INFO:tasks.workunit.client.1.smithi115.stdout:6/169: sync 2021-11-03T17:14:08.127 INFO:tasks.workunit.client.1.smithi115.stdout:0/220: sync 2021-11-03T17:14:08.127 INFO:tasks.workunit.client.1.smithi115.stdout:9/113: sync 2021-11-03T17:14:08.127 INFO:tasks.workunit.client.1.smithi115.stdout:7/170: sync 2021-11-03T17:14:08.128 INFO:tasks.workunit.client.1.smithi115.stdout:6/170: stat d0/f15 0 2021-11-03T17:14:08.128 INFO:tasks.workunit.client.1.smithi115.stdout:6/171: dread - d0/f2a zero size 2021-11-03T17:14:08.128 INFO:tasks.workunit.client.1.smithi115.stdout:0/221: truncate d9/db/f4f 877656 0 2021-11-03T17:14:08.129 INFO:tasks.workunit.client.1.smithi115.stdout:7/171: mkdir d3/da/d26/d30 0 2021-11-03T17:14:08.129 INFO:tasks.workunit.client.1.smithi115.stdout:7/172: mknod d3/da/d18/c31 1 2021-11-03T17:14:08.130 INFO:tasks.workunit.client.1.smithi115.stdout:7/173: mknod d3/da/d18/c32 1 2021-11-03T17:14:08.130 INFO:tasks.workunit.client.1.smithi115.stdout:9/114: creat d1/d5/f20 x:0 0 0 2021-11-03T17:14:08.130 INFO:tasks.workunit.client.1.smithi115.stdout:6/172: getdents d0/d1/d10/d13/d1c 0 2021-11-03T17:14:08.186 INFO:tasks.workunit.client.1.smithi115.stdout:0/222: symlink d9/l52 0 2021-11-03T17:14:08.186 INFO:tasks.workunit.client.1.smithi115.stdout:7/174: mkdir d3/da/d26/d33 0 2021-11-03T17:14:08.193 INFO:tasks.workunit.client.1.smithi115.stdout:7/175: dread d3/da/f1b [4194304,4194304] 0 2021-11-03T17:14:08.198 INFO:tasks.workunit.client.1.smithi115.stdout:6/173: dwrite d0/d1/f5 [0,4194304] 0 2021-11-03T17:14:08.198 INFO:tasks.workunit.client.1.smithi115.stdout:6/174: write d0/d1/d2/f1f [8997934,30253] 0 2021-11-03T17:14:08.217 INFO:tasks.workunit.client.1.smithi115.stdout:6/175: dread d0/d1/d10/d13/d1c/f36 [4194304,4194304] 0 2021-11-03T17:14:08.218 INFO:tasks.workunit.client.1.smithi115.stdout:0/223: dwrite d9/d15/d1a/d1d/f22 [0,4194304] 0 2021-11-03T17:14:08.219 INFO:tasks.workunit.client.1.smithi115.stdout:9/115: dwrite d1/f7 [0,4194304] 0 2021-11-03T17:14:08.219 INFO:tasks.workunit.client.1.smithi115.stdout:9/116: write d1/d18/f1e [581282,5908] 0 2021-11-03T17:14:08.219 INFO:tasks.workunit.client.1.smithi115.stdout:0/224: read d9/db/f20 [2436012,1436] 0 2021-11-03T17:14:08.219 INFO:tasks.workunit.client.1.smithi115.stdout:6/176: truncate d0/f17 861826 0 2021-11-03T17:14:08.220 INFO:tasks.workunit.client.1.smithi115.stdout:7/176: dwrite d3/da/f15 [0,4194304] 0 2021-11-03T17:14:08.222 INFO:tasks.workunit.client.1.smithi115.stdout:9/117: readlink d1/ld 0 2021-11-03T17:14:08.225 INFO:tasks.workunit.client.1.smithi115.stdout:6/177: unlink d0/d1/d10/d13/d1c/d1e/f32 0 2021-11-03T17:14:08.228 INFO:tasks.workunit.client.1.smithi115.stdout:7/177: dread d3/f1e [0,4194304] 0 2021-11-03T17:14:08.229 INFO:tasks.workunit.client.1.smithi115.stdout:9/118: rmdir d1/d5 39 2021-11-03T17:14:08.230 INFO:tasks.workunit.client.1.smithi115.stdout:9/119: write d1/d5/f8 [336446,15433] 0 2021-11-03T17:14:08.230 INFO:tasks.workunit.client.1.smithi115.stdout:7/178: creat d3/f34 x:0 0 0 2021-11-03T17:14:08.240 INFO:tasks.workunit.client.1.smithi115.stdout:0/225: dwrite d9/db/df/f44 [0,4194304] 0 2021-11-03T17:14:08.240 INFO:tasks.workunit.client.1.smithi115.stdout:0/226: chown d9/d15/d1a/f2e 3 1 2021-11-03T17:14:08.240 INFO:tasks.workunit.client.1.smithi115.stdout:0/227: dread d9/d15/d1a/d1d/f4d [0,4194304] 0 2021-11-03T17:14:08.241 INFO:tasks.workunit.client.1.smithi115.stdout:0/228: rename d9/d15/f3a to d9/d15/f53 0 2021-11-03T17:14:08.241 INFO:tasks.workunit.client.1.smithi115.stdout:0/229: readlink d9/d15/d1a/l27 0 2021-11-03T17:14:08.241 INFO:tasks.workunit.client.1.smithi115.stdout:0/230: dread - d9/d15/d1a/f2e zero size 2021-11-03T17:14:08.250 INFO:tasks.workunit.client.1.smithi115.stdout:6/178: dwrite d0/d1/d10/f27 [0,4194304] 0 2021-11-03T17:14:08.254 INFO:tasks.workunit.client.1.smithi115.stdout:9/120: dwrite d1/d5/ff [0,4194304] 0 2021-11-03T17:14:08.258 INFO:tasks.workunit.client.1.smithi115.stdout:9/121: truncate d1/d5/f8 304991 0 2021-11-03T17:14:08.258 INFO:tasks.workunit.client.1.smithi115.stdout:9/122: truncate d1/d5/f20 779084 0 2021-11-03T17:14:08.260 INFO:tasks.workunit.client.1.smithi115.stdout:0/231: dread d9/f25 [4194304,4194304] 0 2021-11-03T17:14:08.260 INFO:tasks.workunit.client.1.smithi115.stdout:9/123: creat d1/d5/f21 x:0 0 0 2021-11-03T17:14:08.262 INFO:tasks.workunit.client.1.smithi115.stdout:0/232: unlink d9/db/df/l51 0 2021-11-03T17:14:08.262 INFO:tasks.workunit.client.1.smithi115.stdout:7/179: dwrite d3/da/d18/f22 [0,4194304] 0 2021-11-03T17:14:08.274 INFO:tasks.workunit.client.1.smithi115.stdout:6/179: dwrite d0/d1/fe [4194304,4194304] 0 2021-11-03T17:14:08.276 INFO:tasks.workunit.client.1.smithi115.stdout:9/124: dread d1/f7 [0,4194304] 0 2021-11-03T17:14:08.277 INFO:tasks.workunit.client.1.smithi115.stdout:6/180: write d0/d1/d2/f1f [1835487,77087] 0 2021-11-03T17:14:08.278 INFO:tasks.workunit.client.1.smithi115.stdout:0/233: dread d9/f35 [0,4194304] 0 2021-11-03T17:14:08.279 INFO:tasks.workunit.client.1.smithi115.stdout:0/234: symlink d9/d15/d2c/d46/l54 0 2021-11-03T17:14:08.281 INFO:tasks.workunit.client.1.smithi115.stdout:0/235: link d9/d15/l45 d9/d15/d1a/l55 0 2021-11-03T17:14:08.281 INFO:tasks.workunit.client.1.smithi115.stdout:0/236: readlink d9/db/ld 0 2021-11-03T17:14:08.281 INFO:tasks.workunit.client.1.smithi115.stdout:6/181: write d0/d1/d10/d13/d1c/d1e/f20 [38161,25238] 0 2021-11-03T17:14:08.282 INFO:tasks.workunit.client.1.smithi115.stdout:0/237: rename d9/d15/d2c/d34/l38 to d9/d15/l56 0 2021-11-03T17:14:08.282 INFO:tasks.workunit.client.1.smithi115.stdout:6/182: rename d0 to d0/d1/d10/d13/d1c/d1e/d37 22 2021-11-03T17:14:08.284 INFO:tasks.workunit.client.1.smithi115.stdout:6/183: rename d0/d1/d10/f16 to d0/d1/d10/d13/d1c/f38 0 2021-11-03T17:14:08.285 INFO:tasks.workunit.client.1.smithi115.stdout:6/184: truncate d0/f15 6376492 0 2021-11-03T17:14:08.285 INFO:tasks.workunit.client.1.smithi115.stdout:9/125: dread d1/d5/f6 [0,4194304] 0 2021-11-03T17:14:08.292 INFO:tasks.workunit.client.1.smithi115.stdout:7/180: dwrite d3/f25 [0,4194304] 0 2021-11-03T17:14:08.296 INFO:tasks.workunit.client.1.smithi115.stdout:0/238: dread d9/db/df/f48 [0,4194304] 0 2021-11-03T17:14:08.296 INFO:tasks.workunit.client.1.smithi115.stdout:0/239: read - d9/db/f42 zero size 2021-11-03T17:14:08.297 INFO:tasks.workunit.client.1.smithi115.stdout:9/126: dread d1/d5/ff [0,4194304] 0 2021-11-03T17:14:08.298 INFO:tasks.workunit.client.1.smithi115.stdout:0/240: truncate d9/d15/d1a/d1d/f4b 884216 0 2021-11-03T17:14:08.299 INFO:tasks.workunit.client.1.smithi115.stdout:7/181: dread d3/f13 [0,4194304] 0 2021-11-03T17:14:08.299 INFO:tasks.workunit.client.1.smithi115.stdout:9/127: creat d1/d18/f22 x:0 0 0 2021-11-03T17:14:08.300 INFO:tasks.workunit.client.1.smithi115.stdout:9/128: read d1/f7 [1506441,57133] 0 2021-11-03T17:14:08.301 INFO:tasks.workunit.client.1.smithi115.stdout:0/241: rename d9/db/f20 to d9/d15/d1a/d1d/f57 0 2021-11-03T17:14:08.304 INFO:tasks.workunit.client.1.smithi115.stdout:7/182: rmdir d3/da/d26/d30 0 2021-11-03T17:14:08.306 INFO:tasks.workunit.client.1.smithi115.stdout:9/129: creat d1/d5/d1f/f23 x:0 0 0 2021-11-03T17:14:08.308 INFO:tasks.workunit.client.1.smithi115.stdout:8/223: sync 2021-11-03T17:14:08.309 INFO:tasks.workunit.client.1.smithi115.stdout:4/408: sync 2021-11-03T17:14:08.309 INFO:tasks.workunit.client.1.smithi115.stdout:3/267: sync 2021-11-03T17:14:08.309 INFO:tasks.workunit.client.1.smithi115.stdout:1/231: sync 2021-11-03T17:14:08.309 INFO:tasks.workunit.client.1.smithi115.stdout:5/299: sync 2021-11-03T17:14:08.310 INFO:tasks.workunit.client.1.smithi115.stdout:2/187: sync 2021-11-03T17:14:08.311 INFO:tasks.workunit.client.1.smithi115.stdout:9/130: unlink d1/f9 0 2021-11-03T17:14:08.311 INFO:tasks.workunit.client.1.smithi115.stdout:9/131: write d1/d5/d1f/f23 [675403,73090] 0 2021-11-03T17:14:08.311 INFO:tasks.workunit.client.1.smithi115.stdout:9/132: stat d1/ld 0 2021-11-03T17:14:08.312 INFO:tasks.workunit.client.1.smithi115.stdout:3/268: truncate f9 4050066 0 2021-11-03T17:14:08.313 INFO:tasks.workunit.client.1.smithi115.stdout:6/185: dwrite d0/d1/d2/f1f [4194304,4194304] 0 2021-11-03T17:14:08.314 INFO:tasks.workunit.client.1.smithi115.stdout:5/300: creat d1a/d28/d44/d4f/d57/f67 x:0 0 0 2021-11-03T17:14:08.314 INFO:tasks.workunit.client.1.smithi115.stdout:4/409: dread d7/dc/d6a/f46 [0,4194304] 0 2021-11-03T17:14:08.314 INFO:tasks.workunit.client.1.smithi115.stdout:4/410: mknod d7/dc/d1a/d21/d47/d56/d63/d7b/c94 1 2021-11-03T17:14:08.315 INFO:tasks.workunit.client.1.smithi115.stdout:4/411: truncate d7/dc/d1a/f2e 1935949 0 2021-11-03T17:14:08.318 INFO:tasks.workunit.client.1.smithi115.stdout:0/242: rename d9/d15/d1a/d1d/f57 to d9/db/f58 0 2021-11-03T17:14:08.319 INFO:tasks.workunit.client.1.smithi115.stdout:0/243: chown l5 995 1 2021-11-03T17:14:08.319 INFO:tasks.workunit.client.1.smithi115.stdout:0/244: mknod d9/d15/d1f/c59 1 2021-11-03T17:14:08.319 INFO:tasks.workunit.client.1.smithi115.stdout:0/245: chown d9/d15/d1f 110950 1 2021-11-03T17:14:08.320 INFO:tasks.workunit.client.1.smithi115.stdout:9/133: creat d1/d5/d1f/f24 x:0 0 0 2021-11-03T17:14:08.320 INFO:tasks.workunit.client.1.smithi115.stdout:3/269: write d13/f1d [8240648,4989] 0 2021-11-03T17:14:08.321 INFO:tasks.workunit.client.1.smithi115.stdout:9/134: write d1/d18/f1c [629390,57075] 0 2021-11-03T17:14:08.326 INFO:tasks.workunit.client.1.smithi115.stdout:5/301: mkdir d1a/d23/d53/d68 0 2021-11-03T17:14:08.327 INFO:tasks.workunit.client.1.smithi115.stdout:2/188: link d4/d6/d13/f25 d4/f3a 0 2021-11-03T17:14:08.330 INFO:tasks.workunit.client.1.smithi115.stdout:6/186: rename d0/d1/d10/f27 to d0/d1/d10/d13/f39 0 2021-11-03T17:14:08.330 INFO:tasks.workunit.client.1.smithi115.stdout:3/270: getdents d13/d51 0 2021-11-03T17:14:08.330 INFO:tasks.workunit.client.1.smithi115.stdout:3/271: chown d13/f4c 159 1 2021-11-03T17:14:08.331 INFO:tasks.workunit.client.1.smithi115.stdout:7/183: dwrite d3/f34 [0,4194304] 0 2021-11-03T17:14:08.331 INFO:tasks.workunit.client.1.smithi115.stdout:7/184: dread - d3/f27 zero size 2021-11-03T17:14:08.331 INFO:tasks.workunit.client.1.smithi115.stdout:9/135: creat d1/f25 x:0 0 0 2021-11-03T17:14:08.333 INFO:tasks.workunit.client.1.smithi115.stdout:5/302: symlink d1a/l69 0 2021-11-03T17:14:08.334 INFO:tasks.workunit.client.1.smithi115.stdout:2/189: getdents d4/d26/d28 0 2021-11-03T17:14:08.336 INFO:tasks.workunit.client.1.smithi115.stdout:8/224: dwrite d4/d6/d18/d30/f34 [4194304,4194304] 0 2021-11-03T17:14:08.336 INFO:tasks.workunit.client.1.smithi115.stdout:3/272: mkdir d13/d51/d2e/d5b 0 2021-11-03T17:14:08.336 INFO:tasks.workunit.client.1.smithi115.stdout:3/273: stat d13/d51/d2e/l38 0 2021-11-03T17:14:08.338 INFO:tasks.workunit.client.1.smithi115.stdout:7/185: mkdir d3/da/d26/d35 0 2021-11-03T17:14:08.339 INFO:tasks.workunit.client.1.smithi115.stdout:9/136: creat d1/d5/d1f/f26 x:0 0 0 2021-11-03T17:14:08.340 INFO:tasks.workunit.client.1.smithi115.stdout:5/303: symlink d1a/d23/d53/l6a 0 2021-11-03T17:14:08.340 INFO:tasks.workunit.client.1.smithi115.stdout:7/186: write d3/da/f1b [4127060,16399] 0 2021-11-03T17:14:08.340 INFO:tasks.workunit.client.1.smithi115.stdout:5/304: write f0 [166463,8365] 0 2021-11-03T17:14:08.341 INFO:tasks.workunit.client.1.smithi115.stdout:5/305: truncate d1a/d28/d34/f66 840930 0 2021-11-03T17:14:08.341 INFO:tasks.workunit.client.1.smithi115.stdout:0/246: dwrite d9/f43 [0,4194304] 0 2021-11-03T17:14:08.342 INFO:tasks.workunit.client.1.smithi115.stdout:2/190: creat d4/d26/d28/f3b x:0 0 0 2021-11-03T17:14:08.342 INFO:tasks.workunit.client.1.smithi115.stdout:2/191: dread - d4/d26/d28/f3b zero size 2021-11-03T17:14:08.343 INFO:tasks.workunit.client.1.smithi115.stdout:4/412: rename d7/dc/d1a/f2e to d7/dc/f95 0 2021-11-03T17:14:08.344 INFO:tasks.workunit.client.1.smithi115.stdout:4/413: read d7/d7a/d83/f64 [161936,22311] 0 2021-11-03T17:14:08.345 INFO:tasks.workunit.client.1.smithi115.stdout:8/225: mkdir d4/d6/d18/d30/d33/d4c 0 2021-11-03T17:14:08.346 INFO:tasks.workunit.client.1.smithi115.stdout:9/137: mkdir d1/d5/d1f/d27 0 2021-11-03T17:14:08.347 INFO:tasks.workunit.client.1.smithi115.stdout:7/187: creat d3/da/f36 x:0 0 0 2021-11-03T17:14:08.349 INFO:tasks.workunit.client.1.smithi115.stdout:1/232: dwrite d7/f9 [0,4194304] 0 2021-11-03T17:14:08.349 INFO:tasks.workunit.client.1.smithi115.stdout:1/233: dread - d7/dc/d2a/f31 zero size 2021-11-03T17:14:08.350 INFO:tasks.workunit.client.1.smithi115.stdout:2/192: dread d4/d6/d13/f38 [0,4194304] 0 2021-11-03T17:14:08.350 INFO:tasks.workunit.client.1.smithi115.stdout:2/193: chown d4/d26/d28/f3b 8811978 1 2021-11-03T17:14:08.350 INFO:tasks.workunit.client.1.smithi115.stdout:2/194: stat d4/d6/d13/f25 0 2021-11-03T17:14:08.351 INFO:tasks.workunit.client.1.smithi115.stdout:5/306: creat d1a/d28/d4a/d65/f6b x:0 0 0 2021-11-03T17:14:08.355 INFO:tasks.workunit.client.1.smithi115.stdout:3/274: rename d13/l18 to d13/d2a/l5c 0 2021-11-03T17:14:08.358 INFO:tasks.workunit.client.1.smithi115.stdout:4/414: mkdir d7/dc/d1a/d21/d47/d56/d63/d7b/d96 0 2021-11-03T17:14:08.361 INFO:tasks.workunit.client.1.smithi115.stdout:9/138: unlink d1/d5/f8 0 2021-11-03T17:14:08.363 INFO:tasks.workunit.client.1.smithi115.stdout:7/188: symlink d3/da/d26/l37 0 2021-11-03T17:14:08.363 INFO:tasks.workunit.client.1.smithi115.stdout:7/189: chown d3/f13 506 1 2021-11-03T17:14:08.364 INFO:tasks.workunit.client.1.smithi115.stdout:6/187: dwrite d0/f6 [8388608,4194304] 0 2021-11-03T17:14:08.364 INFO:tasks.workunit.client.1.smithi115.stdout:6/188: fdatasync d0/f2a 0 2021-11-03T17:14:08.367 INFO:tasks.workunit.client.1.smithi115.stdout:0/247: dwrite d9/db/f58 [0,4194304] 0 2021-11-03T17:14:08.369 INFO:tasks.workunit.client.1.smithi115.stdout:2/195: dwrite d4/d6/d13/f38 [0,4194304] 0 2021-11-03T17:14:08.369 INFO:tasks.workunit.client.1.smithi115.stdout:5/307: dread d1a/d3d/f3b [0,4194304] 0 2021-11-03T17:14:08.369 INFO:tasks.workunit.client.1.smithi115.stdout:5/308: chown f11 84838 1 2021-11-03T17:14:08.371 INFO:tasks.workunit.client.1.smithi115.stdout:1/234: link d7/dc/dd/d24/d43/d32/f3a d7/dc/dd/f46 0 2021-11-03T17:14:08.373 INFO:tasks.workunit.client.1.smithi115.stdout:3/275: creat d13/d2a/f5d x:0 0 0 2021-11-03T17:14:08.376 INFO:tasks.workunit.client.1.smithi115.stdout:8/226: unlink d4/d6/f16 0 2021-11-03T17:14:08.376 INFO:tasks.workunit.client.1.smithi115.stdout:8/227: dread - d4/f26 zero size 2021-11-03T17:14:08.376 INFO:tasks.workunit.client.1.smithi115.stdout:8/228: chown d4/l15 1000 0 2021-11-03T17:14:08.377 INFO:tasks.workunit.client.1.smithi115.stdout:8/229: chown d4/d6/d18/f23 52666887 1 2021-11-03T17:14:08.378 INFO:tasks.workunit.client.1.smithi115.stdout:9/139: creat d1/d1a/f28 x:0 0 0 2021-11-03T17:14:08.378 INFO:tasks.workunit.client.1.smithi115.stdout:9/140: dread - d1/d5/f21 zero size 2021-11-03T17:14:08.378 INFO:tasks.workunit.client.1.smithi115.stdout:1/235: dread d7/dc/dd/d24/f28 [0,4194304] 0 2021-11-03T17:14:08.378 INFO:tasks.workunit.client.1.smithi115.stdout:7/190: symlink d3/da/d26/d33/l38 0 2021-11-03T17:14:08.379 INFO:tasks.workunit.client.1.smithi115.stdout:7/191: mknod d3/da/d18/c39 1 2021-11-03T17:14:08.379 INFO:tasks.workunit.client.1.smithi115.stdout:6/189: rmdir d0/d1/d10/d13/d1c/d1e 39 2021-11-03T17:14:08.380 INFO:tasks.workunit.client.1.smithi115.stdout:0/248: creat d9/d15/d2c/f5a x:0 0 0 2021-11-03T17:14:08.384 INFO:tasks.workunit.client.1.smithi115.stdout:5/309: creat d1a/d23/d53/d68/f6c x:0 0 0 2021-11-03T17:14:08.384 INFO:tasks.workunit.client.1.smithi115.stdout:3/276: symlink d13/d35/d54/l5e 0 2021-11-03T17:14:08.384 INFO:tasks.workunit.client.1.smithi115.stdout:3/277: mknod d13/d2a/d53/c5f 1 2021-11-03T17:14:08.385 INFO:tasks.workunit.client.1.smithi115.stdout:5/310: write d1a/d28/f31 [1347548,87069] 0 2021-11-03T17:14:08.385 INFO:tasks.workunit.client.1.smithi115.stdout:5/311: write d1a/d28/f2c [4451022,19166] 0 2021-11-03T17:14:08.385 INFO:tasks.workunit.client.1.smithi115.stdout:5/312: truncate d1a/d28/d44/d4f/d57/f67 368963 0 2021-11-03T17:14:08.386 INFO:tasks.workunit.client.1.smithi115.stdout:4/415: symlink d7/dc/l97 0 2021-11-03T17:14:08.389 INFO:tasks.workunit.client.1.smithi115.stdout:8/230: symlink d4/d6/d18/l4d 0 2021-11-03T17:14:08.389 INFO:tasks.workunit.client.1.smithi115.stdout:8/231: mknod d4/d6/d18/d30/c4e 1 2021-11-03T17:14:08.395 INFO:tasks.workunit.client.1.smithi115.stdout:0/249: creat d9/d15/d2c/d46/f5b x:0 0 0 2021-11-03T17:14:08.398 INFO:tasks.workunit.client.1.smithi115.stdout:3/278: truncate d13/d51/d2e/f37 3116185 0 2021-11-03T17:14:08.399 INFO:tasks.workunit.client.1.smithi115.stdout:5/313: symlink d1a/d3d/l6d 0 2021-11-03T17:14:08.399 INFO:tasks.workunit.client.1.smithi115.stdout:5/314: write f0 [1099604,122461] 0 2021-11-03T17:14:08.399 INFO:tasks.workunit.client.1.smithi115.stdout:5/315: chown d1a/d28/d34/f27 6 1 2021-11-03T17:14:08.400 INFO:tasks.workunit.client.1.smithi115.stdout:4/416: truncate d7/dc/f11 2741300 0 2021-11-03T17:14:08.400 INFO:tasks.workunit.client.1.smithi115.stdout:0/250: write d9/db/df/f49 [155248,7664] 0 2021-11-03T17:14:08.400 INFO:tasks.workunit.client.1.smithi115.stdout:0/251: fdatasync d9/d15/d1a/f2e 0 2021-11-03T17:14:08.401 INFO:tasks.workunit.client.1.smithi115.stdout:9/141: symlink d1/l29 0 2021-11-03T17:14:08.401 INFO:tasks.workunit.client.1.smithi115.stdout:9/142: fdatasync d1/d1a/f28 0 2021-11-03T17:14:08.403 INFO:tasks.workunit.client.1.smithi115.stdout:3/279: rename d13/d51/d2e/d5b to d13/d2a/d60 0 2021-11-03T17:14:08.404 INFO:tasks.workunit.client.1.smithi115.stdout:4/417: rmdir d7/d7a 39 2021-11-03T17:14:08.405 INFO:tasks.workunit.client.1.smithi115.stdout:8/232: dread d4/d6/d18/f3d [0,4194304] 0 2021-11-03T17:14:08.405 INFO:tasks.workunit.client.1.smithi115.stdout:9/143: creat d1/d5/d1f/f2a x:0 0 0 2021-11-03T17:14:08.407 INFO:tasks.workunit.client.1.smithi115.stdout:4/418: rename d7/dc/d6a/d3c/f66 to d7/dc/d1a/d21/d47/d56/f98 0 2021-11-03T17:14:08.408 INFO:tasks.workunit.client.1.smithi115.stdout:4/419: write d7/dc/d6a/f3a [5283084,83885] 0 2021-11-03T17:14:08.408 INFO:tasks.workunit.client.1.smithi115.stdout:8/233: symlink d4/d6/d18/d44/d47/l4f 0 2021-11-03T17:14:08.409 INFO:tasks.workunit.client.1.smithi115.stdout:8/234: write d4/d6/f8 [8504971,73828] 0 2021-11-03T17:14:08.409 INFO:tasks.workunit.client.1.smithi115.stdout:8/235: fdatasync d4/d6/d18/f3d 0 2021-11-03T17:14:08.410 INFO:tasks.workunit.client.1.smithi115.stdout:9/144: mkdir d1/d18/d2b 0 2021-11-03T17:14:08.417 INFO:tasks.workunit.client.1.smithi115.stdout:4/420: dread d7/dc/d1a/d21/f22 [0,4194304] 0 2021-11-03T17:14:08.418 INFO:tasks.workunit.client.1.smithi115.stdout:1/236: dwrite f2 [0,4194304] 0 2021-11-03T17:14:08.419 INFO:tasks.workunit.client.1.smithi115.stdout:1/237: stat d7/dc/dd/l3f 0 2021-11-03T17:14:08.420 INFO:tasks.workunit.client.1.smithi115.stdout:1/238: creat d7/dc/dd/d42/f47 x:0 0 0 2021-11-03T17:14:08.421 INFO:tasks.workunit.client.1.smithi115.stdout:7/192: dwrite d3/da/d18/f2c [0,4194304] 0 2021-11-03T17:14:08.422 INFO:tasks.workunit.client.1.smithi115.stdout:6/190: dwrite d0/d1/f5 [4194304,4194304] 0 2021-11-03T17:14:08.422 INFO:tasks.workunit.client.1.smithi115.stdout:6/191: chown d0/d1/d10/f28 648385846 1 2021-11-03T17:14:08.422 INFO:tasks.workunit.client.1.smithi115.stdout:1/239: creat d7/dc/dd/d3e/f48 x:0 0 0 2021-11-03T17:14:08.422 INFO:tasks.workunit.client.1.smithi115.stdout:6/192: chown d0/d1/d10/d13/f39 65247 1 2021-11-03T17:14:08.423 INFO:tasks.workunit.client.1.smithi115.stdout:2/196: dwrite d4/d6/d2e/f14 [8388608,4194304] 0 2021-11-03T17:14:08.423 INFO:tasks.workunit.client.1.smithi115.stdout:9/145: fsync d1/d5/fc 0 2021-11-03T17:14:08.424 INFO:tasks.workunit.client.1.smithi115.stdout:2/197: read d4/fa [1966059,89399] 0 2021-11-03T17:14:08.424 INFO:tasks.workunit.client.1.smithi115.stdout:7/193: symlink d3/da/d26/d33/l3a 0 2021-11-03T17:14:08.425 INFO:tasks.workunit.client.1.smithi115.stdout:7/194: chown d3/da/f36 3524470 1 2021-11-03T17:14:08.425 INFO:tasks.workunit.client.1.smithi115.stdout:7/195: truncate d3/f27 529485 0 2021-11-03T17:14:08.429 INFO:tasks.workunit.client.1.smithi115.stdout:1/240: mkdir d7/dc/dd/d24/d43/d32/d49 0 2021-11-03T17:14:08.429 INFO:tasks.workunit.client.1.smithi115.stdout:1/241: fdatasync d7/dc/dd/d24/d43/f3c 0 2021-11-03T17:14:08.432 INFO:tasks.workunit.client.1.smithi115.stdout:3/280: dread d13/d51/f41 [0,4194304] 0 2021-11-03T17:14:08.433 INFO:tasks.workunit.client.1.smithi115.stdout:3/281: chown d13/f21 63248 1 2021-11-03T17:14:08.433 INFO:tasks.workunit.client.1.smithi115.stdout:5/316: truncate d1a/d28/d34/f66 687606 0 2021-11-03T17:14:08.434 INFO:tasks.workunit.client.1.smithi115.stdout:4/421: getdents d7/dc/d1a/d21/d47/d56 0 2021-11-03T17:14:08.438 INFO:tasks.workunit.client.1.smithi115.stdout:9/146: symlink d1/d5/l2c 0 2021-11-03T17:14:08.443 INFO:tasks.workunit.client.1.smithi115.stdout:2/198: creat d4/d6/d2e/f3c x:0 0 0 2021-11-03T17:14:08.445 INFO:tasks.workunit.client.1.smithi115.stdout:6/193: dwrite d0/d1/f8 [0,4194304] 0 2021-11-03T17:14:08.445 INFO:tasks.workunit.client.1.smithi115.stdout:6/194: dread - d0/d1/f19 zero size 2021-11-03T17:14:08.445 INFO:tasks.workunit.client.1.smithi115.stdout:7/196: getdents d3/da 0 2021-11-03T17:14:08.445 INFO:tasks.workunit.client.1.smithi115.stdout:7/197: chown d3/da/d18/l2d 435 1 2021-11-03T17:14:08.446 INFO:tasks.workunit.client.1.smithi115.stdout:0/252: dread d9/db/df/f49 [0,4194304] 0 2021-11-03T17:14:08.447 INFO:tasks.workunit.client.1.smithi115.stdout:6/195: write d0/d1/d2/f12 [3221089,65217] 0 2021-11-03T17:14:08.447 INFO:tasks.workunit.client.1.smithi115.stdout:6/196: chown d0/f22 1107051 1 2021-11-03T17:14:08.447 INFO:tasks.workunit.client.1.smithi115.stdout:1/242: creat d7/dc/d1a/f4a x:0 0 0 2021-11-03T17:14:08.448 INFO:tasks.workunit.client.1.smithi115.stdout:1/243: readlink d7/l1d 0 2021-11-03T17:14:08.448 INFO:tasks.workunit.client.1.smithi115.stdout:1/244: fdatasync d7/dc/dd/d24/f2b 0 2021-11-03T17:14:08.449 INFO:tasks.workunit.client.1.smithi115.stdout:0/253: dread d9/d15/f1e [0,4194304] 0 2021-11-03T17:14:08.449 INFO:tasks.workunit.client.1.smithi115.stdout:7/198: dread d3/f14 [0,4194304] 0 2021-11-03T17:14:08.452 INFO:tasks.workunit.client.1.smithi115.stdout:3/282: symlink d13/d51/d2e/l61 0 2021-11-03T17:14:08.456 INFO:tasks.workunit.client.1.smithi115.stdout:5/317: link d1a/d3d/f62 d1a/d28/d4c/f6e 0 2021-11-03T17:14:08.456 INFO:tasks.workunit.client.1.smithi115.stdout:1/245: dread d7/dc/dd/d24/d26/f3b [0,4194304] 0 2021-11-03T17:14:08.457 INFO:tasks.workunit.client.1.smithi115.stdout:4/422: symlink d7/l99 0 2021-11-03T17:14:08.458 INFO:tasks.workunit.client.1.smithi115.stdout:9/147: stat d1/l4 0 2021-11-03T17:14:08.458 INFO:tasks.workunit.client.1.smithi115.stdout:9/148: chown d1/f25 12708941 1 2021-11-03T17:14:08.459 INFO:tasks.workunit.client.1.smithi115.stdout:2/199: creat d4/d26/f3d x:0 0 0 2021-11-03T17:14:08.465 INFO:tasks.workunit.client.1.smithi115.stdout:3/283: dread d13/d35/f3a [0,4194304] 0 2021-11-03T17:14:08.467 INFO:tasks.workunit.client.1.smithi115.stdout:1/246: rmdir d7/dc/d1a 39 2021-11-03T17:14:08.467 INFO:tasks.workunit.client.1.smithi115.stdout:1/247: truncate d7/dc/dd/d24/d43/d32/f3a 753575 0 2021-11-03T17:14:08.468 INFO:tasks.workunit.client.1.smithi115.stdout:6/197: symlink d0/d1/l3a 0 2021-11-03T17:14:08.468 INFO:tasks.workunit.client.1.smithi115.stdout:6/198: truncate d0/f17 1069255 0 2021-11-03T17:14:08.468 INFO:tasks.workunit.client.1.smithi115.stdout:6/199: chown d0/d1/d10/d13/d1c/f2f 64429 1 2021-11-03T17:14:08.469 INFO:tasks.workunit.client.1.smithi115.stdout:5/318: link l14 d1a/d28/d4c/d63/l6f 0 2021-11-03T17:14:08.469 INFO:tasks.workunit.client.1.smithi115.stdout:5/319: fdatasync d1a/d28/d34/f66 0 2021-11-03T17:14:08.469 INFO:tasks.workunit.client.1.smithi115.stdout:5/320: truncate d1a/d28/d34/f33 1193714 0 2021-11-03T17:14:08.469 INFO:tasks.workunit.client.1.smithi115.stdout:5/321: write d1a/d28/d34/f33 [299248,80285] 0 2021-11-03T17:14:08.470 INFO:tasks.workunit.client.1.smithi115.stdout:9/149: rename d1/d5/d1f to d1/d2d 0 2021-11-03T17:14:08.472 INFO:tasks.workunit.client.1.smithi115.stdout:5/322: rename d1a/l22 to d1a/d28/d44/d4f/d57/l70 0 2021-11-03T17:14:08.472 INFO:tasks.workunit.client.1.smithi115.stdout:5/323: read - d1a/d23/f5c zero size 2021-11-03T17:14:08.473 INFO:tasks.workunit.client.1.smithi115.stdout:5/324: read - d1a/d3d/f62 zero size 2021-11-03T17:14:08.473 INFO:tasks.workunit.client.1.smithi115.stdout:9/150: unlink d1/d2d/f24 0 2021-11-03T17:14:08.477 INFO:tasks.workunit.client.1.smithi115.stdout:5/325: rename d1a/d28/d4c/d63/l6f to d1a/d23/l71 0 2021-11-03T17:14:08.477 INFO:tasks.workunit.client.1.smithi115.stdout:5/326: dread - d1a/d23/d53/d68/f6c zero size 2021-11-03T17:14:08.478 INFO:tasks.workunit.client.1.smithi115.stdout:5/327: write f0 [2075237,120224] 0 2021-11-03T17:14:08.478 INFO:tasks.workunit.client.1.smithi115.stdout:5/328: fsync d1a/d3d/f42 0 2021-11-03T17:14:08.478 INFO:tasks.workunit.client.1.smithi115.stdout:5/329: truncate d1a/d28/d4c/f6e 354073 0 2021-11-03T17:14:08.478 INFO:tasks.workunit.client.1.smithi115.stdout:5/330: mknod d1a/d28/d34/c72 1 2021-11-03T17:14:08.480 INFO:tasks.workunit.client.1.smithi115.stdout:9/151: symlink d1/d5/l2e 0 2021-11-03T17:14:08.480 INFO:tasks.workunit.client.1.smithi115.stdout:9/152: chown d1/f25 12683 1 2021-11-03T17:14:08.480 INFO:tasks.workunit.client.1.smithi115.stdout:9/153: write d1/d1a/f28 [935175,76831] 0 2021-11-03T17:14:08.480 INFO:tasks.workunit.client.1.smithi115.stdout:9/154: fdatasync d1/f25 0 2021-11-03T17:14:08.481 INFO:tasks.workunit.client.1.smithi115.stdout:7/199: dwrite d3/f1e [4194304,4194304] 0 2021-11-03T17:14:08.481 INFO:tasks.workunit.client.1.smithi115.stdout:5/331: truncate d1a/d3d/f47 2106225 0 2021-11-03T17:14:08.481 INFO:tasks.workunit.client.1.smithi115.stdout:5/332: write f0 [2489707,119450] 0 2021-11-03T17:14:08.481 INFO:tasks.workunit.client.1.smithi115.stdout:5/333: write d1a/f1f [5048507,5176] 0 2021-11-03T17:14:08.482 INFO:tasks.workunit.client.1.smithi115.stdout:3/284: dread d13/f21 [0,4194304] 0 2021-11-03T17:14:08.482 INFO:tasks.workunit.client.1.smithi115.stdout:7/200: link d3/l10 d3/da/d26/d33/l3b 0 2021-11-03T17:14:08.482 INFO:tasks.workunit.client.1.smithi115.stdout:7/201: chown d3/f25 130759999 1 2021-11-03T17:14:08.483 INFO:tasks.workunit.client.1.smithi115.stdout:7/202: mknod d3/da/d18/c3c 1 2021-11-03T17:14:08.483 INFO:tasks.workunit.client.1.smithi115.stdout:5/334: rmdir d1a/d23/d53/d68 39 2021-11-03T17:14:08.484 INFO:tasks.workunit.client.1.smithi115.stdout:5/335: read d1a/d3d/f42 [378529,37615] 0 2021-11-03T17:14:08.485 INFO:tasks.workunit.client.1.smithi115.stdout:7/203: creat d3/da/d26/d35/f3d x:0 0 0 2021-11-03T17:14:08.487 INFO:tasks.workunit.client.1.smithi115.stdout:0/254: dwrite d9/db/f58 [0,4194304] 0 2021-11-03T17:14:08.487 INFO:tasks.workunit.client.1.smithi115.stdout:0/255: chown d9/d15/d1a/d1d/f4b 483982 1 2021-11-03T17:14:08.487 INFO:tasks.workunit.client.1.smithi115.stdout:4/423: dwrite d7/dc/d1a/d21/f25 [0,4194304] 0 2021-11-03T17:14:08.488 INFO:tasks.workunit.client.1.smithi115.stdout:7/204: getdents d3/da/d26 0 2021-11-03T17:14:08.493 INFO:tasks.workunit.client.1.smithi115.stdout:2/200: truncate d4/d6/d13/f38 1837747 0 2021-11-03T17:14:08.493 INFO:tasks.workunit.client.1.smithi115.stdout:9/155: dread d1/d1a/f28 [0,4194304] 0 2021-11-03T17:14:08.498 INFO:tasks.workunit.client.1.smithi115.stdout:1/248: dwrite d7/dc/f15 [0,4194304] 0 2021-11-03T17:14:08.499 INFO:tasks.workunit.client.1.smithi115.stdout:4/424: dread d7/dc/d6a/d72/f76 [0,4194304] 0 2021-11-03T17:14:08.499 INFO:tasks.workunit.client.1.smithi115.stdout:1/249: truncate d7/dc/dd/f46 825006 0 2021-11-03T17:14:08.499 INFO:tasks.workunit.client.1.smithi115.stdout:4/425: write d7/dc/d1a/d21/d47/f51 [1497459,107543] 0 2021-11-03T17:14:08.501 INFO:tasks.workunit.client.1.smithi115.stdout:7/205: creat d3/da/d26/d35/f3e x:0 0 0 2021-11-03T17:14:08.507 INFO:tasks.workunit.client.1.smithi115.stdout:0/256: dwrite d9/d15/f33 [0,4194304] 0 2021-11-03T17:14:08.513 INFO:tasks.workunit.client.1.smithi115.stdout:5/336: dread d1a/d3d/f47 [0,4194304] 0 2021-11-03T17:14:08.513 INFO:tasks.workunit.client.1.smithi115.stdout:5/337: chown d1a 10 1 2021-11-03T17:14:08.513 INFO:tasks.workunit.client.1.smithi115.stdout:5/338: truncate d1a/d28/d4a/d65/f6b 736786 0 2021-11-03T17:14:08.514 INFO:tasks.workunit.client.1.smithi115.stdout:4/426: dwrite d7/dc/d1a/d21/d47/f74 [0,4194304] 0 2021-11-03T17:14:08.517 INFO:tasks.workunit.client.1.smithi115.stdout:2/201: creat d4/d26/f3e x:0 0 0 2021-11-03T17:14:08.518 INFO:tasks.workunit.client.1.smithi115.stdout:2/202: write d4/d26/f3e [142500,130153] 0 2021-11-03T17:14:08.518 INFO:tasks.workunit.client.1.smithi115.stdout:2/203: write d4/d6/f2a [5328697,115224] 0 2021-11-03T17:14:08.518 INFO:tasks.workunit.client.1.smithi115.stdout:2/204: dread - d4/d6/d2e/f39 zero size 2021-11-03T17:14:08.518 INFO:tasks.workunit.client.1.smithi115.stdout:9/156: mkdir d1/d2d/d27/d2f 0 2021-11-03T17:14:08.519 INFO:tasks.workunit.client.1.smithi115.stdout:1/250: dread - d7/dc/d1a/f35 zero size 2021-11-03T17:14:08.519 INFO:tasks.workunit.client.1.smithi115.stdout:7/206: getdents d3/da/d26/d33 0 2021-11-03T17:14:08.525 INFO:tasks.workunit.client.1.smithi115.stdout:0/257: symlink d9/d15/d2c/l5c 0 2021-11-03T17:14:08.526 INFO:tasks.workunit.client.1.smithi115.stdout:5/339: symlink d1a/d28/d44/d4f/l73 0 2021-11-03T17:14:08.529 INFO:tasks.workunit.client.1.smithi115.stdout:4/427: rename d7/dc/d1a/d21/d47/f51 to d7/dc/d1a/d21/d37/f9a 0 2021-11-03T17:14:08.532 INFO:tasks.workunit.client.1.smithi115.stdout:7/207: unlink d3/da/d18/f2c 0 2021-11-03T17:14:08.533 INFO:tasks.workunit.client.1.smithi115.stdout:0/258: truncate d9/d15/d1a/d1d/f4b 2857806 0 2021-11-03T17:14:08.533 INFO:tasks.workunit.client.1.smithi115.stdout:0/259: chown d9/l3e 16315 1 2021-11-03T17:14:08.534 INFO:tasks.workunit.client.1.smithi115.stdout:5/340: creat d1a/d28/d44/f74 x:0 0 0 2021-11-03T17:14:08.536 INFO:tasks.workunit.client.1.smithi115.stdout:7/208: creat d3/da/d26/d35/f3f x:0 0 0 2021-11-03T17:14:08.537 INFO:tasks.workunit.client.1.smithi115.stdout:2/205: dwrite d4/d26/f3e [0,4194304] 0 2021-11-03T17:14:08.539 INFO:tasks.workunit.client.1.smithi115.stdout:9/157: rename d1/l16 to d1/d18/d2b/l30 0 2021-11-03T17:14:08.545 INFO:tasks.workunit.client.1.smithi115.stdout:0/260: dread f0 [0,4194304] 0 2021-11-03T17:14:08.547 INFO:tasks.workunit.client.1.smithi115.stdout:1/251: rename d7/dc/dd/d24/d43/d32 to d7/dc/dd/d24/d4b 0 2021-11-03T17:14:08.547 INFO:tasks.workunit.client.1.smithi115.stdout:1/252: mknod d7/dc/dd/d24/c4c 1 2021-11-03T17:14:08.548 INFO:tasks.workunit.client.1.smithi115.stdout:0/261: creat d9/db/f5d x:0 0 0 2021-11-03T17:14:08.548 INFO:tasks.workunit.client.1.smithi115.stdout:0/262: dread - d9/d15/d1a/d1d/f29 zero size 2021-11-03T17:14:08.548 INFO:tasks.workunit.client.1.smithi115.stdout:0/263: mknod d9/d15/d1a/c5e 1 2021-11-03T17:14:08.548 INFO:tasks.workunit.client.1.smithi115.stdout:0/264: mknod d9/db/df/c5f 1 2021-11-03T17:14:08.549 INFO:tasks.workunit.client.1.smithi115.stdout:0/265: fsync d9/db/df/f48 0 2021-11-03T17:14:08.549 INFO:tasks.workunit.client.1.smithi115.stdout:5/341: rename d1a/d23/d53 to d1a/d28/d4c/d75 0 2021-11-03T17:14:08.557 INFO:tasks.workunit.client.1.smithi115.stdout:2/206: dread d4/d6/f1c [0,4194304] 0 2021-11-03T17:14:08.558 INFO:tasks.workunit.client.1.smithi115.stdout:7/209: dwrite d3/da/d26/d35/f3e [0,4194304] 0 2021-11-03T17:14:08.577 INFO:tasks.workunit.client.1.smithi115.stdout:0/266: dwrite d9/d15/d1a/d1d/f29 [0,4194304] 0 2021-11-03T17:14:08.578 INFO:tasks.workunit.client.1.smithi115.stdout:5/342: dwrite d1a/d3d/f42 [0,4194304] 0 2021-11-03T17:14:08.578 INFO:tasks.workunit.client.1.smithi115.stdout:0/267: truncate d9/d15/d1a/f50 802514 0 2021-11-03T17:14:08.578 INFO:tasks.workunit.client.1.smithi115.stdout:2/207: dwrite d4/d6/d13/f36 [0,4194304] 0 2021-11-03T17:14:08.583 INFO:tasks.workunit.client.1.smithi115.stdout:0/268: dread d9/db/df/f49 [0,4194304] 0 2021-11-03T17:14:08.584 INFO:tasks.workunit.client.1.smithi115.stdout:5/343: mkdir d1a/d28/d76 0 2021-11-03T17:14:08.586 INFO:tasks.workunit.client.1.smithi115.stdout:0/269: creat d9/d15/d1f/f60 x:0 0 0 2021-11-03T17:14:08.589 INFO:tasks.workunit.client.1.smithi115.stdout:5/344: unlink d1a/d28/d34/f27 0 2021-11-03T17:14:08.589 INFO:tasks.workunit.client.0.smithi027.stdout:6/165: sync 2021-11-03T17:14:08.590 INFO:tasks.workunit.client.1.smithi115.stdout:2/208: creat d4/d6/d2e/d34/f3f x:0 0 0 2021-11-03T17:14:08.594 INFO:tasks.workunit.client.1.smithi115.stdout:2/209: dread d4/d6/f1c [0,4194304] 0 2021-11-03T17:14:08.594 INFO:tasks.workunit.client.1.smithi115.stdout:2/210: mknod d4/c40 1 2021-11-03T17:14:08.596 INFO:tasks.workunit.client.1.smithi115.stdout:2/211: rename d4/d6/f1c to d4/d26/d28/f41 0 2021-11-03T17:14:08.605 INFO:tasks.workunit.client.1.smithi115.stdout:2/212: dread d4/f3a [0,4194304] 0 2021-11-03T17:14:08.625 INFO:tasks.workunit.client.1.smithi115.stdout:0/270: dwrite d9/d15/d1a/d1d/f22 [0,4194304] 0 2021-11-03T17:14:08.626 INFO:tasks.workunit.client.0.smithi027.stdout:6/166: dwrite dc/f1f [0,4194304] 0 2021-11-03T17:14:08.626 INFO:tasks.workunit.client.1.smithi115.stdout:0/271: truncate d9/f43 1472659 0 2021-11-03T17:14:08.627 INFO:tasks.workunit.client.1.smithi115.stdout:0/272: chown d9/d15/d1a/d1d 0 1 2021-11-03T17:14:08.627 INFO:tasks.workunit.client.1.smithi115.stdout:2/213: dwrite d4/d26/d28/f41 [4194304,4194304] 0 2021-11-03T17:14:08.628 INFO:tasks.workunit.client.1.smithi115.stdout:5/345: dwrite d1a/d3d/f64 [0,4194304] 0 2021-11-03T17:14:08.628 INFO:tasks.workunit.client.1.smithi115.stdout:0/273: link l5 d9/d15/d1a/d1d/l61 0 2021-11-03T17:14:08.628 INFO:tasks.workunit.client.1.smithi115.stdout:0/274: chown d9/db/df/f49 121874329 1 2021-11-03T17:14:08.629 INFO:tasks.workunit.client.1.smithi115.stdout:0/275: readlink d9/db/ld 0 2021-11-03T17:14:08.629 INFO:tasks.workunit.client.0.smithi027.stdout:6/167: rename dc/l12 to dc/d16/d1b/l22 0 2021-11-03T17:14:08.630 INFO:tasks.workunit.client.1.smithi115.stdout:2/214: getdents d4 0 2021-11-03T17:14:08.631 INFO:tasks.workunit.client.0.smithi027.stdout:6/168: creat dc/f23 x:0 0 0 2021-11-03T17:14:08.632 INFO:tasks.workunit.client.0.smithi027.stdout:6/169: write dc/f19 [832328,91840] 0 2021-11-03T17:14:08.632 INFO:tasks.workunit.client.1.smithi115.stdout:5/346: unlink f0 0 2021-11-03T17:14:08.633 INFO:tasks.workunit.client.0.smithi027.stdout:6/170: mkdir dc/d16/d1b/d24 0 2021-11-03T17:14:08.633 INFO:tasks.workunit.client.0.smithi027.stdout:6/171: mknod dc/d16/c25 1 2021-11-03T17:14:08.633 INFO:tasks.workunit.client.0.smithi027.stdout:6/172: mknod dc/d16/c26 1 2021-11-03T17:14:08.634 INFO:tasks.workunit.client.0.smithi027.stdout:6/173: mknod dc/d16/c27 1 2021-11-03T17:14:08.634 INFO:tasks.workunit.client.1.smithi115.stdout:0/276: symlink d9/l62 0 2021-11-03T17:14:08.635 INFO:tasks.workunit.client.1.smithi115.stdout:0/277: truncate d9/db/f5d 520511 0 2021-11-03T17:14:08.635 INFO:tasks.workunit.client.0.smithi027.stdout:6/174: creat dc/d16/d1b/f28 x:0 0 0 2021-11-03T17:14:08.637 INFO:tasks.workunit.client.0.smithi027.stdout:6/175: rename dc/d16/l1c to dc/l29 0 2021-11-03T17:14:08.637 INFO:tasks.workunit.client.1.smithi115.stdout:2/215: mkdir d4/d6/d2e/d34/d42 0 2021-11-03T17:14:08.638 INFO:tasks.workunit.client.1.smithi115.stdout:2/216: chown d4/d6/d2e/f14 97 1 2021-11-03T17:14:08.638 INFO:tasks.workunit.client.1.smithi115.stdout:0/278: read d9/d15/d1a/d1d/f4b [647551,55150] 0 2021-11-03T17:14:08.638 INFO:tasks.workunit.client.1.smithi115.stdout:0/279: write d9/db/f4a [409154,58335] 0 2021-11-03T17:14:08.639 INFO:tasks.workunit.client.0.smithi027.stdout:6/176: creat dc/d16/d1b/d24/f2a x:0 0 0 2021-11-03T17:14:08.640 INFO:tasks.workunit.client.1.smithi115.stdout:0/280: read d9/d15/f16 [492940,35850] 0 2021-11-03T17:14:08.640 INFO:tasks.workunit.client.1.smithi115.stdout:5/347: truncate d1a/f1d 1703381 0 2021-11-03T17:14:08.640 INFO:tasks.workunit.client.1.smithi115.stdout:5/348: dread d1a/d3d/f62 [0,4194304] 0 2021-11-03T17:14:08.641 INFO:tasks.workunit.client.1.smithi115.stdout:2/217: rename d4/d26 to d4/d6/d2e/d34/d43 0 2021-11-03T17:14:08.643 INFO:tasks.workunit.client.1.smithi115.stdout:0/281: rmdir d9/d15/d2c 39 2021-11-03T17:14:08.644 INFO:tasks.workunit.client.1.smithi115.stdout:0/282: truncate d9/d15/d2c/d46/f5b 397970 0 2021-11-03T17:14:08.644 INFO:tasks.workunit.client.1.smithi115.stdout:0/283: chown d9/d15/d2c/d46/l54 45 1 2021-11-03T17:14:08.647 INFO:tasks.workunit.client.1.smithi115.stdout:5/349: rmdir d1a/d28/d44/d5b 0 2021-11-03T17:14:08.649 INFO:tasks.workunit.client.0.smithi027.stdout:6/177: dread dc/f14 [0,4194304] 0 2021-11-03T17:14:08.650 INFO:tasks.workunit.client.1.smithi115.stdout:2/218: truncate d4/f3a 2383175 0 2021-11-03T17:14:08.653 INFO:tasks.workunit.client.1.smithi115.stdout:0/284: dread d9/db/df/f44 [0,4194304] 0 2021-11-03T17:14:08.656 INFO:tasks.workunit.client.1.smithi115.stdout:5/350: creat d1a/d28/d44/f77 x:0 0 0 2021-11-03T17:14:08.660 INFO:tasks.workunit.client.1.smithi115.stdout:2/219: mkdir d4/d44 0 2021-11-03T17:14:08.660 INFO:tasks.workunit.client.1.smithi115.stdout:5/351: read d1a/d28/f29 [82716,68210] 0 2021-11-03T17:14:08.664 INFO:tasks.workunit.client.1.smithi115.stdout:2/220: creat d4/d6/f45 x:0 0 0 2021-11-03T17:14:08.664 INFO:tasks.workunit.client.1.smithi115.stdout:2/221: truncate d4/d6/d13/f38 2224572 0 2021-11-03T17:14:08.665 INFO:tasks.workunit.client.1.smithi115.stdout:2/222: creat d4/d6/d2e/f46 x:0 0 0 2021-11-03T17:14:08.674 INFO:tasks.workunit.client.0.smithi027.stdout:7/223: sync 2021-11-03T17:14:08.674 INFO:tasks.workunit.client.0.smithi027.stdout:3/136: sync 2021-11-03T17:14:08.675 INFO:tasks.workunit.client.0.smithi027.stdout:7/224: getdents d0/d2/d1c 0 2021-11-03T17:14:08.676 INFO:tasks.workunit.client.1.smithi115.stdout:0/285: dwrite d9/f23 [0,4194304] 0 2021-11-03T17:14:08.677 INFO:tasks.workunit.client.0.smithi027.stdout:3/137: creat d1/f25 x:0 0 0 2021-11-03T17:14:08.678 INFO:tasks.workunit.client.1.smithi115.stdout:5/352: dwrite d1a/d28/d4a/f56 [0,4194304] 0 2021-11-03T17:14:08.679 INFO:tasks.workunit.client.0.smithi027.stdout:6/178: dwrite dc/f1e [0,4194304] 0 2021-11-03T17:14:08.679 INFO:tasks.workunit.client.0.smithi027.stdout:6/179: mknod dc/d16/d1b/d24/c2b 1 2021-11-03T17:14:08.679 INFO:tasks.workunit.client.0.smithi027.stdout:6/180: read fa [486404,111228] 0 2021-11-03T17:14:08.679 INFO:tasks.workunit.client.0.smithi027.stdout:7/225: creat d0/d2/d1f/f46 x:0 0 0 2021-11-03T17:14:08.680 INFO:tasks.workunit.client.0.smithi027.stdout:7/226: fdatasync d0/d2/d10/d1e/d20/f43 0 2021-11-03T17:14:08.680 INFO:tasks.workunit.client.0.smithi027.stdout:3/138: mkdir d1/d26 0 2021-11-03T17:14:08.685 INFO:tasks.workunit.client.1.smithi115.stdout:0/286: dread d9/db/f47 [0,4194304] 0 2021-11-03T17:14:08.686 INFO:tasks.workunit.client.1.smithi115.stdout:0/287: chown d9/d15/d1f/f60 61 1 2021-11-03T17:14:08.686 INFO:tasks.workunit.client.1.smithi115.stdout:0/288: chown d9/db/f4f 4882521 1 2021-11-03T17:14:08.686 INFO:tasks.workunit.client.1.smithi115.stdout:0/289: fdatasync d9/db/df/f49 0 2021-11-03T17:14:08.687 INFO:tasks.workunit.client.1.smithi115.stdout:5/353: dread d1a/d3d/f47 [0,4194304] 0 2021-11-03T17:14:08.687 INFO:tasks.workunit.client.1.smithi115.stdout:5/354: chown fd 7173 1 2021-11-03T17:14:08.687 INFO:tasks.workunit.client.1.smithi115.stdout:5/355: chown d1a/d51 370 1 2021-11-03T17:14:08.688 INFO:tasks.workunit.client.1.smithi115.stdout:0/290: mkdir d9/d15/d2c/d46/d63 0 2021-11-03T17:14:08.688 INFO:tasks.workunit.client.1.smithi115.stdout:0/291: readlink d9/d15/d1a/l55 0 2021-11-03T17:14:08.693 INFO:tasks.workunit.client.1.smithi115.stdout:0/292: dread d9/db/df/f48 [0,4194304] 0 2021-11-03T17:14:08.695 INFO:tasks.workunit.client.1.smithi115.stdout:0/293: creat d9/db/f64 x:0 0 0 2021-11-03T17:14:08.695 INFO:tasks.workunit.client.1.smithi115.stdout:0/294: fsync d9/f23 0 2021-11-03T17:14:08.696 INFO:tasks.workunit.client.1.smithi115.stdout:0/295: symlink d9/d15/d2c/d46/l65 0 2021-11-03T17:14:08.696 INFO:tasks.workunit.client.1.smithi115.stdout:0/296: chown d9/db 132 1 2021-11-03T17:14:08.709 INFO:tasks.workunit.client.0.smithi027.stdout:6/181: dwrite dc/f19 [0,4194304] 0 2021-11-03T17:14:08.711 INFO:tasks.workunit.client.1.smithi115.stdout:5/356: dwrite d1a/d28/d4c/f6e [0,4194304] 0 2021-11-03T17:14:08.711 INFO:tasks.workunit.client.1.smithi115.stdout:0/297: dwrite d9/db/f47 [4194304,4194304] 0 2021-11-03T17:14:08.731 INFO:tasks.workunit.client.1.smithi115.stdout:0/298: dwrite d9/db/f4a [0,4194304] 0 2021-11-03T17:14:08.731 INFO:tasks.workunit.client.1.smithi115.stdout:0/299: chown d9/d15/d1f 122 1 2021-11-03T17:14:08.744 INFO:tasks.workunit.client.1.smithi115.stdout:0/300: dwrite d9/d15/d2c/f5a [0,4194304] 0 2021-11-03T17:14:08.746 INFO:tasks.workunit.client.1.smithi115.stdout:0/301: creat d9/f66 x:0 0 0 2021-11-03T17:14:08.747 INFO:tasks.workunit.client.0.smithi027.stdout:3/139: rmdir d1/d26 0 2021-11-03T17:14:08.747 INFO:tasks.workunit.client.1.smithi115.stdout:0/302: creat d9/d15/f67 x:0 0 0 2021-11-03T17:14:08.748 INFO:tasks.workunit.client.1.smithi115.stdout:0/303: chown d9/d15/d1f/f39 89745749 1 2021-11-03T17:14:08.749 INFO:tasks.workunit.client.1.smithi115.stdout:0/304: rename d9/d15/f33 to d9/d15/d1a/f68 0 2021-11-03T17:14:08.750 INFO:tasks.workunit.client.1.smithi115.stdout:0/305: fdatasync d9/d15/d1a/d1d/f4d 0 2021-11-03T17:14:08.750 INFO:tasks.workunit.client.1.smithi115.stdout:0/306: mknod d9/d37/c69 1 2021-11-03T17:14:08.750 INFO:tasks.workunit.client.0.smithi027.stdout:6/182: creat dc/d16/f2c x:0 0 0 2021-11-03T17:14:08.753 INFO:tasks.workunit.client.0.smithi027.stdout:6/183: mkdir dc/d16/d1b/d2d 0 2021-11-03T17:14:08.756 INFO:tasks.workunit.client.1.smithi115.stdout:0/307: dread d9/d15/d1a/d1d/f29 [0,4194304] 0 2021-11-03T17:14:08.759 INFO:tasks.workunit.client.0.smithi027.stdout:6/184: dread dc/f18 [0,4194304] 0 2021-11-03T17:14:08.760 INFO:tasks.workunit.client.0.smithi027.stdout:6/185: write dc/f23 [787532,65160] 0 2021-11-03T17:14:08.761 INFO:tasks.workunit.client.0.smithi027.stdout:6/186: symlink dc/d16/d1b/l2e 0 2021-11-03T17:14:08.763 INFO:tasks.workunit.client.0.smithi027.stdout:6/187: rename dc/f23 to dc/d16/d1b/f2f 0 2021-11-03T17:14:08.764 INFO:tasks.workunit.client.0.smithi027.stdout:6/188: mkdir dc/d16/d30 0 2021-11-03T17:14:08.766 INFO:tasks.workunit.client.0.smithi027.stdout:6/189: symlink dc/d16/l31 0 2021-11-03T17:14:08.766 INFO:tasks.workunit.client.0.smithi027.stdout:6/190: write dc/d16/d1b/d24/f2a [681289,103722] 0 2021-11-03T17:14:08.767 INFO:tasks.workunit.client.0.smithi027.stdout:6/191: truncate dc/f10 4692469 0 2021-11-03T17:14:08.767 INFO:tasks.workunit.client.0.smithi027.stdout:6/192: creat dc/d16/d30/f32 x:0 0 0 2021-11-03T17:14:08.785 INFO:tasks.workunit.client.1.smithi115.stdout:0/308: dwrite d9/d15/d1a/d1d/f4d [0,4194304] 0 2021-11-03T17:14:08.786 INFO:tasks.workunit.client.1.smithi115.stdout:0/309: link d9/d15/d2c/d46/l65 d9/d37/l6a 0 2021-11-03T17:14:08.790 INFO:tasks.workunit.client.0.smithi027.stdout:6/193: dwrite dc/d16/d30/f32 [0,4194304] 0 2021-11-03T17:14:08.790 INFO:tasks.workunit.client.0.smithi027.stdout:6/194: chown dc/d16/d1b/l22 1049449 1 2021-11-03T17:14:08.790 INFO:tasks.workunit.client.0.smithi027.stdout:6/195: fsync dc/f15 0 2021-11-03T17:14:08.792 INFO:tasks.workunit.client.0.smithi027.stdout:6/196: symlink dc/d16/d1b/d2d/l33 0 2021-11-03T17:14:08.792 INFO:tasks.workunit.client.0.smithi027.stdout:6/197: read - dc/d16/f2c zero size 2021-11-03T17:14:08.793 INFO:tasks.workunit.client.0.smithi027.stdout:6/198: stat dc/d16/d30 0 2021-11-03T17:14:08.793 INFO:tasks.workunit.client.0.smithi027.stdout:6/199: chown dc/d16/d1b/d24/f2a 0 1 2021-11-03T17:14:08.793 INFO:tasks.workunit.client.0.smithi027.stdout:6/200: readlink dc/d16/d1b/d2d/l33 0 2021-11-03T17:14:08.793 INFO:tasks.workunit.client.0.smithi027.stdout:6/201: readlink dc/d16/d1b/l22 0 2021-11-03T17:14:08.795 INFO:tasks.workunit.client.1.smithi115.stdout:0/310: dread d9/d15/d2c/f5a [0,4194304] 0 2021-11-03T17:14:08.795 INFO:tasks.workunit.client.1.smithi115.stdout:0/311: write d9/d15/d1a/d1d/f4d [929839,70122] 0 2021-11-03T17:14:08.795 INFO:tasks.workunit.client.1.smithi115.stdout:0/312: dread - d9/d15/d1f/f39 zero size 2021-11-03T17:14:08.795 INFO:tasks.workunit.client.0.smithi027.stdout:6/202: mkdir dc/d16/d1b/d2d/d34 0 2021-11-03T17:14:08.796 INFO:tasks.workunit.client.0.smithi027.stdout:6/203: dread - dc/d16/f2c zero size 2021-11-03T17:14:08.796 INFO:tasks.workunit.client.0.smithi027.stdout:6/204: chown dc/d16/d1b 3 1 2021-11-03T17:14:08.796 INFO:tasks.workunit.client.0.smithi027.stdout:6/205: truncate dc/f18 2741065 0 2021-11-03T17:14:08.800 INFO:tasks.workunit.client.1.smithi115.stdout:0/313: dread d9/db/f4a [0,4194304] 0 2021-11-03T17:14:08.801 INFO:tasks.workunit.client.1.smithi115.stdout:0/314: getdents d9/d15/d1f 0 2021-11-03T17:14:08.802 INFO:tasks.workunit.client.1.smithi115.stdout:0/315: getdents d9/db 0 2021-11-03T17:14:08.803 INFO:tasks.workunit.client.1.smithi115.stdout:0/316: truncate d9/db/f47 2823557 0 2021-11-03T17:14:08.804 INFO:tasks.workunit.client.0.smithi027.stdout:6/206: unlink f0 0 2021-11-03T17:14:08.804 INFO:tasks.workunit.client.0.smithi027.stdout:6/207: write dc/f1f [2107278,23152] 0 2021-11-03T17:14:08.805 INFO:tasks.workunit.client.0.smithi027.stdout:6/208: chown dc/d16/d1b/l20 62838 1 2021-11-03T17:14:08.807 INFO:tasks.workunit.client.0.smithi027.stdout:6/209: link dc/d16/d1b/d24/f2a dc/d16/d1b/d2d/f35 0 2021-11-03T17:14:08.807 INFO:tasks.workunit.client.1.smithi115.stdout:0/317: rmdir d9 39 2021-11-03T17:14:08.808 INFO:tasks.workunit.client.1.smithi115.stdout:0/318: write d9/db/df/f44 [1224613,48484] 0 2021-11-03T17:14:08.808 INFO:tasks.workunit.client.1.smithi115.stdout:0/319: write d9/d15/d1a/d1d/f29 [3883231,64631] 0 2021-11-03T17:14:08.809 INFO:tasks.workunit.client.1.smithi115.stdout:0/320: chown d9/db/f42 7941285 1 2021-11-03T17:14:08.809 INFO:tasks.workunit.client.1.smithi115.stdout:0/321: mknod d9/d15/d1f/c6b 1 2021-11-03T17:14:08.809 INFO:tasks.workunit.client.1.smithi115.stdout:0/322: chown d9/db/f4a 62231734 1 2021-11-03T17:14:08.809 INFO:tasks.workunit.client.1.smithi115.stdout:0/323: readlink d9/l62 0 2021-11-03T17:14:08.810 INFO:tasks.workunit.client.1.smithi115.stdout:0/324: read - d9/d15/f53 zero size 2021-11-03T17:14:08.810 INFO:tasks.workunit.client.1.smithi115.stdout:0/325: fsync d9/d15/d1a/f2e 0 2021-11-03T17:14:08.810 INFO:tasks.workunit.client.0.smithi027.stdout:6/210: mkdir dc/d16/d1b/d2d/d34/d36 0 2021-11-03T17:14:08.817 INFO:tasks.workunit.client.0.smithi027.stdout:6/211: dread dc/d16/d30/f32 [0,4194304] 0 2021-11-03T17:14:08.818 INFO:tasks.workunit.client.0.smithi027.stdout:6/212: mkdir dc/d37 0 2021-11-03T17:14:08.823 INFO:tasks.workunit.client.1.smithi115.stdout:0/326: dwrite d9/db/df/f48 [0,4194304] 0 2021-11-03T17:14:08.823 INFO:tasks.workunit.client.1.smithi115.stdout:0/327: chown d9/db 31 1 2021-11-03T17:14:08.825 INFO:tasks.workunit.client.1.smithi115.stdout:0/328: getdents d9/d15 0 2021-11-03T17:14:08.825 INFO:tasks.workunit.client.1.smithi115.stdout:0/329: getdents d9/db 0 2021-11-03T17:14:08.827 INFO:tasks.workunit.client.1.smithi115.stdout:0/330: unlink l5 0 2021-11-03T17:14:08.827 INFO:tasks.workunit.client.1.smithi115.stdout:0/331: chown d9/db/f42 993 1 2021-11-03T17:14:08.827 INFO:tasks.workunit.client.1.smithi115.stdout:0/332: symlink d9/d15/d1a/l6c 0 2021-11-03T17:14:08.828 INFO:tasks.workunit.client.1.smithi115.stdout:0/333: write d9/db/df/f44 [2996225,36006] 0 2021-11-03T17:14:08.830 INFO:tasks.workunit.client.1.smithi115.stdout:0/334: unlink d9/db/f5d 0 2021-11-03T17:14:08.831 INFO:tasks.workunit.client.1.smithi115.stdout:0/335: creat d9/d15/d2c/d46/d63/f6d x:0 0 0 2021-11-03T17:14:08.831 INFO:tasks.workunit.client.1.smithi115.stdout:0/336: truncate d9/d15/d1a/f68 3243357 0 2021-11-03T17:14:08.835 INFO:tasks.workunit.client.0.smithi027.stdout:6/213: dwrite dc/d16/f2c [0,4194304] 0 2021-11-03T17:14:08.837 INFO:tasks.workunit.client.0.smithi027.stdout:6/214: creat dc/d16/d30/f38 x:0 0 0 2021-11-03T17:14:08.840 INFO:tasks.workunit.client.0.smithi027.stdout:6/215: unlink dc/d16/d1b/f2f 0 2021-11-03T17:14:08.841 INFO:tasks.workunit.client.1.smithi115.stdout:0/337: dread d9/d15/d1a/d1d/f22 [0,4194304] 0 2021-11-03T17:14:08.841 INFO:tasks.workunit.client.1.smithi115.stdout:0/338: getdents d9/d15 0 2021-11-03T17:14:08.844 INFO:tasks.workunit.client.0.smithi027.stdout:6/216: truncate fa 1612021 0 2021-11-03T17:14:08.851 INFO:tasks.workunit.client.0.smithi027.stdout:6/217: dread dc/f1e [0,4194304] 0 2021-11-03T17:14:08.852 INFO:tasks.workunit.client.0.smithi027.stdout:6/218: read f7 [1294,17209] 0 2021-11-03T17:14:08.852 INFO:tasks.workunit.client.0.smithi027.stdout:6/219: stat dc/d16/d1b/d2d/d34 0 2021-11-03T17:14:08.853 INFO:tasks.workunit.client.0.smithi027.stdout:6/220: write dc/f10 [3340407,28604] 0 2021-11-03T17:14:08.853 INFO:tasks.workunit.client.0.smithi027.stdout:6/221: write dc/f18 [3406570,37699] 0 2021-11-03T17:14:08.854 INFO:tasks.workunit.client.0.smithi027.stdout:6/222: write dc/d16/d1b/d24/f2a [456210,131026] 0 2021-11-03T17:14:08.855 INFO:tasks.workunit.client.0.smithi027.stdout:6/223: write f7 [2736831,101683] 0 2021-11-03T17:14:08.857 INFO:tasks.workunit.client.0.smithi027.stdout:6/224: creat dc/d16/d1b/d2d/d34/f39 x:0 0 0 2021-11-03T17:14:08.857 INFO:tasks.workunit.client.0.smithi027.stdout:6/225: readlink dc/d16/l31 0 2021-11-03T17:14:08.857 INFO:tasks.workunit.client.0.smithi027.stdout:6/226: chown dc/f14 4074799 1 2021-11-03T17:14:08.857 INFO:tasks.workunit.client.0.smithi027.stdout:6/227: mknod dc/d16/d1b/d24/c3a 1 2021-11-03T17:14:08.858 INFO:tasks.workunit.client.0.smithi027.stdout:6/228: rename dc to dc/d3b 22 2021-11-03T17:14:08.858 INFO:tasks.workunit.client.0.smithi027.stdout:6/229: mkdir dc/d16/d1b/d3c 0 2021-11-03T17:14:08.859 INFO:tasks.workunit.client.0.smithi027.stdout:6/230: symlink dc/d16/l3d 0 2021-11-03T17:14:08.874 INFO:tasks.workunit.client.0.smithi027.stdout:6/231: dwrite dc/d16/d1b/d24/f2a [0,4194304] 0 2021-11-03T17:14:08.890 INFO:tasks.workunit.client.0.smithi027.stdout:2/276: sync 2021-11-03T17:14:08.890 INFO:tasks.workunit.client.0.smithi027.stdout:6/232: dwrite dc/d16/d30/f32 [0,4194304] 0 2021-11-03T17:14:08.890 INFO:tasks.workunit.client.0.smithi027.stdout:6/233: readlink dc/d16/d1b/l22 0 2021-11-03T17:14:08.890 INFO:tasks.workunit.client.0.smithi027.stdout:6/234: fdatasync dc/d16/d1b/d24/f2a 0 2021-11-03T17:14:08.891 INFO:tasks.workunit.client.0.smithi027.stdout:6/235: write dc/ff [4549971,51578] 0 2021-11-03T17:14:08.891 INFO:tasks.workunit.client.0.smithi027.stdout:2/277: mkdir d0/d2a/d5d 0 2021-11-03T17:14:08.892 INFO:tasks.workunit.client.0.smithi027.stdout:6/236: rmdir dc/d16/d1b/d2d/d34/d36 0 2021-11-03T17:14:08.894 INFO:tasks.workunit.client.0.smithi027.stdout:2/278: unlink d0/d5/f1e 0 2021-11-03T17:14:08.896 INFO:tasks.workunit.client.0.smithi027.stdout:6/237: link dc/d16/l3d dc/d16/d1b/d3c/l3e 0 2021-11-03T17:14:08.898 INFO:tasks.workunit.client.0.smithi027.stdout:6/238: rename lb to dc/d16/d1b/d3c/l3f 0 2021-11-03T17:14:08.899 INFO:tasks.workunit.client.0.smithi027.stdout:6/239: creat dc/d16/f40 x:0 0 0 2021-11-03T17:14:08.900 INFO:tasks.workunit.client.0.smithi027.stdout:6/240: stat dc/d16 0 2021-11-03T17:14:08.900 INFO:tasks.workunit.client.0.smithi027.stdout:6/241: readlink dc/d16/d1b/l2e 0 2021-11-03T17:14:08.900 INFO:tasks.workunit.client.0.smithi027.stdout:6/242: chown dc/d16/d1b/l22 1 1 2021-11-03T17:14:08.902 INFO:tasks.workunit.client.0.smithi027.stdout:6/243: dread dc/f18 [0,4194304] 0 2021-11-03T17:14:08.927 INFO:tasks.workunit.client.0.smithi027.stdout:6/244: rename dc/l13 to dc/d16/d1b/d2d/l41 0 2021-11-03T17:14:08.940 INFO:tasks.workunit.client.0.smithi027.stdout:6/245: dwrite dc/d16/d1b/f28 [0,4194304] 0 2021-11-03T17:14:08.942 INFO:tasks.workunit.client.0.smithi027.stdout:6/246: symlink dc/d16/d1b/l42 0 2021-11-03T17:14:08.944 INFO:tasks.workunit.client.0.smithi027.stdout:6/247: unlink dc/f19 0 2021-11-03T17:14:08.945 INFO:tasks.workunit.client.0.smithi027.stdout:6/248: mknod dc/d16/d1b/d3c/c43 1 2021-11-03T17:14:08.945 INFO:tasks.workunit.client.0.smithi027.stdout:6/249: chown dc/f14 50302 1 2021-11-03T17:14:08.951 INFO:tasks.workunit.client.0.smithi027.stdout:6/250: dread dc/f1f [0,4194304] 0 2021-11-03T17:14:08.955 INFO:tasks.workunit.client.0.smithi027.stdout:6/251: unlink dc/d16/d1b/d2d/l33 0 2021-11-03T17:14:08.955 INFO:tasks.workunit.client.0.smithi027.stdout:6/252: chown dc/d16/d1b/d24/f2a 956244150 1 2021-11-03T17:14:08.955 INFO:tasks.workunit.client.0.smithi027.stdout:6/253: chown dc/l29 1646260567 1 2021-11-03T17:14:08.956 INFO:tasks.workunit.client.0.smithi027.stdout:6/254: chown dc/f1e 466061556 1 2021-11-03T17:14:08.969 INFO:tasks.workunit.client.0.smithi027.stdout:6/255: dwrite dc/f18 [0,4194304] 0 2021-11-03T17:14:08.969 INFO:tasks.workunit.client.0.smithi027.stdout:6/256: write dc/d16/d30/f38 [935784,116963] 0 2021-11-03T17:14:08.971 INFO:tasks.workunit.client.0.smithi027.stdout:6/257: link dc/f15 dc/d16/d1b/d3c/f44 0 2021-11-03T17:14:08.972 INFO:tasks.workunit.client.0.smithi027.stdout:6/258: chown dc/d16/d1b/l22 252 1 2021-11-03T17:14:08.973 INFO:tasks.workunit.client.0.smithi027.stdout:6/259: creat dc/d16/d1b/d24/f45 x:0 0 0 2021-11-03T17:14:08.981 INFO:tasks.workunit.client.0.smithi027.stdout:6/260: dread f7 [0,4194304] 0 2021-11-03T17:14:08.984 INFO:tasks.workunit.client.0.smithi027.stdout:6/261: dread dc/d16/d1b/d3c/f44 [0,4194304] 0 2021-11-03T17:14:08.985 INFO:tasks.workunit.client.0.smithi027.stdout:6/262: creat dc/d16/d1b/d24/f46 x:0 0 0 2021-11-03T17:14:08.986 INFO:tasks.workunit.client.0.smithi027.stdout:6/263: rename dc/d16/d1b/d24/f45 to dc/d37/f47 0 2021-11-03T17:14:08.988 INFO:tasks.workunit.client.0.smithi027.stdout:6/264: mkdir dc/d16/d1b/d3c/d48 0 2021-11-03T17:14:09.032 INFO:tasks.workunit.client.0.smithi027.stdout:6/265: symlink dc/d16/d1b/d2d/d34/l49 0 2021-11-03T17:14:09.032 INFO:tasks.workunit.client.0.smithi027.stdout:6/266: getdents dc/d16/d1b/d2d 0 2021-11-03T17:14:09.034 INFO:tasks.workunit.client.0.smithi027.stdout:6/267: symlink dc/d16/d1b/d2d/d34/l4a 0 2021-11-03T17:14:09.191 INFO:tasks.workunit.client.0.smithi027.stdout:8/301: sync 2021-11-03T17:14:09.191 INFO:tasks.workunit.client.0.smithi027.stdout:5/316: sync 2021-11-03T17:14:09.192 INFO:tasks.workunit.client.0.smithi027.stdout:1/239: sync 2021-11-03T17:14:09.192 INFO:tasks.workunit.client.0.smithi027.stdout:4/242: sync 2021-11-03T17:14:09.192 INFO:tasks.workunit.client.0.smithi027.stdout:1/240: chown d6/d11 2862 1 2021-11-03T17:14:09.192 INFO:tasks.workunit.client.0.smithi027.stdout:1/241: fsync f2 0 2021-11-03T17:14:09.193 INFO:tasks.workunit.client.0.smithi027.stdout:8/302: write d7/d10/f16 [523937,93128] 0 2021-11-03T17:14:09.193 INFO:tasks.workunit.client.0.smithi027.stdout:5/317: creat d2/d1f/f6f x:0 0 0 2021-11-03T17:14:09.197 INFO:tasks.workunit.client.0.smithi027.stdout:8/303: rmdir d7/d10 39 2021-11-03T17:14:09.201 INFO:tasks.workunit.client.0.smithi027.stdout:5/318: rename d2/d1f/l4c to d2/d4/d61/l70 0 2021-11-03T17:14:09.201 INFO:tasks.workunit.client.0.smithi027.stdout:5/319: dread - d2/d2c/d3c/d58/f5e zero size 2021-11-03T17:14:09.202 INFO:tasks.workunit.client.0.smithi027.stdout:5/320: stat d2/f42 0 2021-11-03T17:14:09.205 INFO:tasks.workunit.client.0.smithi027.stdout:5/321: creat d2/d4/d4e/d67/d20/d25/f71 x:0 0 0 2021-11-03T17:14:09.206 INFO:tasks.workunit.client.0.smithi027.stdout:5/322: readlink d2/d2c/d3c/l54 0 2021-11-03T17:14:09.206 INFO:tasks.workunit.client.0.smithi027.stdout:5/323: truncate d2/d2c/d3c/d58/f5e 352859 0 2021-11-03T17:14:09.206 INFO:tasks.workunit.client.0.smithi027.stdout:5/324: mknod d2/d4/d4e/d67/d20/d25/c72 1 2021-11-03T17:14:09.206 INFO:tasks.workunit.client.0.smithi027.stdout:5/325: dread - d2/d12/f32 zero size 2021-11-03T17:14:09.207 INFO:tasks.workunit.client.0.smithi027.stdout:5/326: write d2/d2c/d3c/f41 [2880853,24341] 0 2021-11-03T17:14:09.208 INFO:tasks.workunit.client.0.smithi027.stdout:5/327: link d2/d4/d4e/d67/f15 d2/d4/d4e/d67/d20/f73 0 2021-11-03T17:14:09.212 INFO:tasks.workunit.client.0.smithi027.stdout:4/243: dwrite d6/db/d21/f25 [4194304,4194304] 0 2021-11-03T17:14:09.213 INFO:tasks.workunit.client.0.smithi027.stdout:1/242: dwrite d6/d11/d14/d1f/d25/f47 [0,4194304] 0 2021-11-03T17:14:09.213 INFO:tasks.workunit.client.0.smithi027.stdout:1/243: fdatasync d6/d11/d14/d1f/f2b 0 2021-11-03T17:14:09.214 INFO:tasks.workunit.client.0.smithi027.stdout:1/244: chown d6/d11/d14/d1f/d37/d49 15 1 2021-11-03T17:14:09.214 INFO:tasks.workunit.client.0.smithi027.stdout:1/245: write d6/de/d13/d15/f51 [465566,19801] 0 2021-11-03T17:14:09.214 INFO:tasks.workunit.client.0.smithi027.stdout:1/246: chown l3 0 1 2021-11-03T17:14:09.215 INFO:tasks.workunit.client.0.smithi027.stdout:8/304: truncate d7/d10/f16 624614 0 2021-11-03T17:14:09.215 INFO:tasks.workunit.client.0.smithi027.stdout:8/305: truncate d7/d10/d18/d1a/f60 187506 0 2021-11-03T17:14:09.215 INFO:tasks.workunit.client.0.smithi027.stdout:8/306: stat d7/d10/d3f/f4c 0 2021-11-03T17:14:09.216 INFO:tasks.workunit.client.0.smithi027.stdout:1/247: write d6/de/f2c [3185167,123102] 0 2021-11-03T17:14:09.220 INFO:tasks.workunit.client.0.smithi027.stdout:4/244: rmdir d6/db 39 2021-11-03T17:14:09.220 INFO:tasks.workunit.client.0.smithi027.stdout:4/245: dread - d6/d9/d16/d36/f4a zero size 2021-11-03T17:14:09.221 INFO:tasks.workunit.client.0.smithi027.stdout:5/328: dread d2/d4/f34 [4194304,4194304] 0 2021-11-03T17:14:09.221 INFO:tasks.workunit.client.0.smithi027.stdout:4/246: write d6/d9/d16/f32 [4754297,102286] 0 2021-11-03T17:14:09.230 INFO:tasks.workunit.client.0.smithi027.stdout:1/248: rename d6/d11/d14/d1f/d25/l2e to d6/d11/d14/d1f/d25/d28/d4c/l52 0 2021-11-03T17:14:09.231 INFO:tasks.workunit.client.0.smithi027.stdout:8/307: dwrite d7/d10/d3f/f4e [0,4194304] 0 2021-11-03T17:14:09.232 INFO:tasks.workunit.client.0.smithi027.stdout:8/308: chown d7/d10/d18/l19 246 1 2021-11-03T17:14:09.232 INFO:tasks.workunit.client.0.smithi027.stdout:8/309: dread - d7/d10/d21/f45 zero size 2021-11-03T17:14:09.232 INFO:tasks.workunit.client.0.smithi027.stdout:8/310: fdatasync d7/f1b 0 2021-11-03T17:14:09.233 INFO:tasks.workunit.client.0.smithi027.stdout:5/329: link d2/d4/f66 d2/d2c/d3c/d58/f74 0 2021-11-03T17:14:09.235 INFO:tasks.workunit.client.0.smithi027.stdout:1/249: creat d6/f53 x:0 0 0 2021-11-03T17:14:09.236 INFO:tasks.workunit.client.0.smithi027.stdout:1/250: dread d6/de/d13/f38 [0,4194304] 0 2021-11-03T17:14:09.245 INFO:tasks.workunit.client.0.smithi027.stdout:4/247: dwrite d6/d9/d16/f26 [0,4194304] 0 2021-11-03T17:14:09.246 INFO:tasks.workunit.client.0.smithi027.stdout:5/330: truncate d2/d4/d4e/d67/f15 2780048 0 2021-11-03T17:14:09.247 INFO:tasks.workunit.client.0.smithi027.stdout:1/251: creat d6/d11/d14/d1f/d31/f54 x:0 0 0 2021-11-03T17:14:09.248 INFO:tasks.workunit.client.0.smithi027.stdout:4/248: symlink d6/db/d21/d41/l50 0 2021-11-03T17:14:09.251 INFO:tasks.workunit.client.0.smithi027.stdout:1/252: rename d6/d11/d14/d1f/f2b to d6/d11/d14/d1f/d25/d28/f55 0 2021-11-03T17:14:09.252 INFO:tasks.workunit.client.0.smithi027.stdout:8/311: dwrite d7/d10/d18/d1a/f60 [0,4194304] 0 2021-11-03T17:14:09.252 INFO:tasks.workunit.client.0.smithi027.stdout:4/249: creat d6/d9/d16/d36/f51 x:0 0 0 2021-11-03T17:14:09.253 INFO:tasks.workunit.client.0.smithi027.stdout:5/331: dread d2/d4/f5 [0,4194304] 0 2021-11-03T17:14:09.253 INFO:tasks.workunit.client.0.smithi027.stdout:5/332: dread d2/d4/d4e/d67/d20/f3a [0,4194304] 0 2021-11-03T17:14:09.254 INFO:tasks.workunit.client.0.smithi027.stdout:5/333: chown d2/d2c/d3c/l5d 143449667 1 2021-11-03T17:14:09.258 INFO:tasks.workunit.client.0.smithi027.stdout:5/334: write d2/d4/f34 [942043,79407] 0 2021-11-03T17:14:09.258 INFO:tasks.workunit.client.0.smithi027.stdout:5/335: dread - d2/d4/d4e/d67/f2e zero size 2021-11-03T17:14:09.260 INFO:tasks.workunit.client.0.smithi027.stdout:5/336: dread d2/f7 [0,4194304] 0 2021-11-03T17:14:09.260 INFO:tasks.workunit.client.0.smithi027.stdout:4/250: symlink d6/db/d21/d41/l52 0 2021-11-03T17:14:09.261 INFO:tasks.workunit.client.0.smithi027.stdout:8/312: dread d7/f1d [0,4194304] 0 2021-11-03T17:14:09.261 INFO:tasks.workunit.client.0.smithi027.stdout:5/337: write d2/d4/d4e/d67/f11 [3423419,96274] 0 2021-11-03T17:14:09.263 INFO:tasks.workunit.client.0.smithi027.stdout:1/253: rename d6/l9 to d6/de/d13/d15/d4e/l56 0 2021-11-03T17:14:09.263 INFO:tasks.workunit.client.0.smithi027.stdout:1/254: mknod d6/de/d13/d15/d4e/c57 1 2021-11-03T17:14:09.266 INFO:tasks.workunit.client.0.smithi027.stdout:4/251: creat d6/d9/d28/d40/f53 x:0 0 0 2021-11-03T17:14:09.271 INFO:tasks.workunit.client.0.smithi027.stdout:8/313: creat d7/d10/d18/f61 x:0 0 0 2021-11-03T17:14:09.273 INFO:tasks.workunit.client.0.smithi027.stdout:8/314: dread d7/d10/d3f/f4c [0,4194304] 0 2021-11-03T17:14:09.273 INFO:tasks.workunit.client.0.smithi027.stdout:8/315: fdatasync d7/f1b 0 2021-11-03T17:14:09.276 INFO:tasks.workunit.client.0.smithi027.stdout:8/316: dread d7/d10/d18/d1a/f3e [0,4194304] 0 2021-11-03T17:14:09.276 INFO:tasks.workunit.client.0.smithi027.stdout:8/317: fdatasync d7/d10/d3f/f4e 0 2021-11-03T17:14:09.279 INFO:tasks.workunit.client.0.smithi027.stdout:5/338: rename d2/d4/d4e/d67/f15 to d2/d4/d4e/d67/d20/d25/d6a/f75 0 2021-11-03T17:14:09.279 INFO:tasks.workunit.client.0.smithi027.stdout:5/339: dread - d2/d4/d4e/f4f zero size 2021-11-03T17:14:09.284 INFO:tasks.workunit.client.0.smithi027.stdout:5/340: write d2/d4/f43 [2452361,62919] 0 2021-11-03T17:14:09.297 INFO:tasks.workunit.client.0.smithi027.stdout:1/255: dwrite d6/f29 [0,4194304] 0 2021-11-03T17:14:09.298 INFO:tasks.workunit.client.0.smithi027.stdout:4/252: dwrite d6/d9/d16/d36/f4a [0,4194304] 0 2021-11-03T17:14:09.298 INFO:tasks.workunit.client.0.smithi027.stdout:5/341: dread d2/d4/d4e/d67/d20/d25/f27 [0,4194304] 0 2021-11-03T17:14:09.298 INFO:tasks.workunit.client.0.smithi027.stdout:5/342: readlink d2/d2c/d3c/l5d 0 2021-11-03T17:14:09.298 INFO:tasks.workunit.client.0.smithi027.stdout:8/318: dwrite d7/d10/d3f/f4c [0,4194304] 0 2021-11-03T17:14:09.311 INFO:tasks.workunit.client.0.smithi027.stdout:4/253: rmdir d6/db/d21/d2b 39 2021-11-03T17:14:09.312 INFO:tasks.workunit.client.0.smithi027.stdout:1/256: getdents d6/de/d13/d15/d43 0 2021-11-03T17:14:09.313 INFO:tasks.workunit.client.0.smithi027.stdout:5/343: rmdir d2/d4/d4e/d67/d20/d25/d6a 39 2021-11-03T17:14:09.313 INFO:tasks.workunit.client.0.smithi027.stdout:5/344: mknod d2/d2c/c76 1 2021-11-03T17:14:09.316 INFO:tasks.workunit.client.0.smithi027.stdout:8/319: link d7/d10/d18/d1a/f1f d7/f62 0 2021-11-03T17:14:09.319 INFO:tasks.workunit.client.0.smithi027.stdout:4/254: rename d6/l18 to d6/db/l54 0 2021-11-03T17:14:09.319 INFO:tasks.workunit.client.0.smithi027.stdout:4/255: write d6/d9/d16/d36/f51 [634342,113821] 0 2021-11-03T17:14:09.319 INFO:tasks.workunit.client.0.smithi027.stdout:4/256: dread - d6/f48 zero size 2021-11-03T17:14:09.319 INFO:tasks.workunit.client.0.smithi027.stdout:8/320: write d7/d2d/d38/f3d [2196415,13184] 0 2021-11-03T17:14:09.321 INFO:tasks.workunit.client.0.smithi027.stdout:1/257: unlink l3 0 2021-11-03T17:14:09.321 INFO:tasks.workunit.client.0.smithi027.stdout:5/345: getdents d2/d4/d4e 0 2021-11-03T17:14:09.324 INFO:tasks.workunit.client.0.smithi027.stdout:4/257: creat d6/db/d21/d41/d44/f55 x:0 0 0 2021-11-03T17:14:09.324 INFO:tasks.workunit.client.0.smithi027.stdout:4/258: fsync d6/db/d21/d2b/f45 0 2021-11-03T17:14:09.324 INFO:tasks.workunit.client.0.smithi027.stdout:4/259: truncate d6/d9/d16/d36/f42 648728 0 2021-11-03T17:14:09.326 INFO:tasks.workunit.client.0.smithi027.stdout:8/321: creat d7/d10/d21/f63 x:0 0 0 2021-11-03T17:14:09.328 INFO:tasks.workunit.client.0.smithi027.stdout:1/258: unlink d6/de/d13/d15/f51 0 2021-11-03T17:14:09.328 INFO:tasks.workunit.client.0.smithi027.stdout:1/259: chown d6/d11/d14/d1f/d25/d28/d4c/f4f 209404 1 2021-11-03T17:14:09.333 INFO:tasks.workunit.client.0.smithi027.stdout:4/260: link d6/d9/f1b d6/d9/d16/d17/f56 0 2021-11-03T17:14:09.337 INFO:tasks.workunit.client.0.smithi027.stdout:4/261: write d6/d9/d16/f19 [176116,99931] 0 2021-11-03T17:14:09.339 INFO:tasks.workunit.client.0.smithi027.stdout:4/262: unlink d6/d9/d16/d36/f51 0 2021-11-03T17:14:09.341 INFO:tasks.workunit.client.0.smithi027.stdout:4/263: write d6/d9/d16/d17/f31 [2435509,96237] 0 2021-11-03T17:14:09.342 INFO:tasks.workunit.client.0.smithi027.stdout:4/264: mkdir d6/d9/d16/d36/d57 0 2021-11-03T17:14:09.344 INFO:tasks.workunit.client.0.smithi027.stdout:4/265: write d6/db/d21/f25 [3490742,66524] 0 2021-11-03T17:14:09.345 INFO:tasks.workunit.client.0.smithi027.stdout:8/322: dread d7/d10/d18/f49 [0,4194304] 0 2021-11-03T17:14:09.346 INFO:tasks.workunit.client.0.smithi027.stdout:8/323: creat d7/d10/d3f/d55/f64 x:0 0 0 2021-11-03T17:14:09.347 INFO:tasks.workunit.client.0.smithi027.stdout:1/260: dwrite d6/d11/d14/f35 [0,4194304] 0 2021-11-03T17:14:09.351 INFO:tasks.workunit.client.0.smithi027.stdout:1/261: mkdir d6/d11/d14/d1f/d31/d58 0 2021-11-03T17:14:09.354 INFO:tasks.workunit.client.0.smithi027.stdout:1/262: creat d6/d11/d14/d1f/f59 x:0 0 0 2021-11-03T17:14:09.366 INFO:tasks.workunit.client.0.smithi027.stdout:4/266: dwrite d6/db/d21/f2c [0,4194304] 0 2021-11-03T17:14:09.369 INFO:tasks.workunit.client.0.smithi027.stdout:4/267: mkdir d6/db/d21/d2b/d2d/d3e/d58 0 2021-11-03T17:14:09.369 INFO:tasks.workunit.client.0.smithi027.stdout:4/268: write d6/d9/d28/f4b [1984286,97576] 0 2021-11-03T17:14:09.369 INFO:tasks.workunit.client.0.smithi027.stdout:8/324: dwrite d7/d10/d18/f48 [0,4194304] 0 2021-11-03T17:14:09.370 INFO:tasks.workunit.client.0.smithi027.stdout:8/325: truncate d7/d10/d21/f45 477421 0 2021-11-03T17:14:09.370 INFO:tasks.workunit.client.0.smithi027.stdout:4/269: getdents d6/db/d21/d41/d44 0 2021-11-03T17:14:09.372 INFO:tasks.workunit.client.0.smithi027.stdout:8/326: write d7/f62 [1058864,95062] 0 2021-11-03T17:14:09.372 INFO:tasks.workunit.client.0.smithi027.stdout:1/263: dwrite d6/d11/d14/d1f/d31/f54 [0,4194304] 0 2021-11-03T17:14:09.373 INFO:tasks.workunit.client.0.smithi027.stdout:8/327: truncate d7/d10/d3f/f4c 3895018 0 2021-11-03T17:14:09.377 INFO:tasks.workunit.client.0.smithi027.stdout:1/264: symlink d6/de/d13/d15/d43/l5a 0 2021-11-03T17:14:09.377 INFO:tasks.workunit.client.0.smithi027.stdout:1/265: stat d6/d11/d14/d1f/d25/d28/d4c 0 2021-11-03T17:14:09.378 INFO:tasks.workunit.client.0.smithi027.stdout:1/266: write d6/de/f1a [2140467,47621] 0 2021-11-03T17:14:09.382 INFO:tasks.workunit.client.0.smithi027.stdout:4/270: dread d6/db/d21/f37 [0,4194304] 0 2021-11-03T17:14:09.390 INFO:tasks.workunit.client.0.smithi027.stdout:8/328: dwrite d7/d10/d18/d1a/f3e [0,4194304] 0 2021-11-03T17:14:09.390 INFO:tasks.workunit.client.0.smithi027.stdout:8/329: chown d7/d53/l5d 2 1 2021-11-03T17:14:09.390 INFO:tasks.workunit.client.0.smithi027.stdout:8/330: stat d7/d10/d18/d1a/f3e 0 2021-11-03T17:14:09.390 INFO:tasks.workunit.client.0.smithi027.stdout:8/331: fsync d7/d10/d18/f49 0 2021-11-03T17:14:09.391 INFO:tasks.workunit.client.0.smithi027.stdout:8/332: write d7/d10/d3f/f47 [222798,114423] 0 2021-11-03T17:14:09.391 INFO:tasks.workunit.client.0.smithi027.stdout:8/333: mknod d7/d10/d21/d41/c65 1 2021-11-03T17:14:09.391 INFO:tasks.workunit.client.0.smithi027.stdout:8/334: read d7/d10/d3f/f47 [64152,111318] 0 2021-11-03T17:14:09.394 INFO:tasks.workunit.client.0.smithi027.stdout:1/267: dread d6/d11/d14/f17 [0,4194304] 0 2021-11-03T17:14:09.396 INFO:tasks.workunit.client.0.smithi027.stdout:1/268: creat d6/d11/d14/d1f/d37/d49/f5b x:0 0 0 2021-11-03T17:14:09.401 INFO:tasks.workunit.client.0.smithi027.stdout:4/271: dwrite d6/db/d21/d2b/f45 [0,4194304] 0 2021-11-03T17:14:09.410 INFO:tasks.workunit.client.0.smithi027.stdout:4/272: symlink d6/l59 0 2021-11-03T17:14:09.411 INFO:tasks.workunit.client.0.smithi027.stdout:4/273: chown d6/d9/d16/f19 609 1 2021-11-03T17:14:09.411 INFO:tasks.workunit.client.0.smithi027.stdout:4/274: chown d6/d9/d16/f19 0 1 2021-11-03T17:14:09.412 INFO:tasks.workunit.client.0.smithi027.stdout:4/275: write d6/db/d21/f25 [6844207,103113] 0 2021-11-03T17:14:09.414 INFO:tasks.workunit.client.0.smithi027.stdout:8/335: dread d7/d10/d18/d1a/f1f [0,4194304] 0 2021-11-03T17:14:09.416 INFO:tasks.workunit.client.0.smithi027.stdout:1/269: dwrite d6/d11/f27 [0,4194304] 0 2021-11-03T17:14:09.417 INFO:tasks.workunit.client.0.smithi027.stdout:8/336: mkdir d7/d10/d21/d66 0 2021-11-03T17:14:09.418 INFO:tasks.workunit.client.0.smithi027.stdout:8/337: fsync d7/f8 0 2021-11-03T17:14:09.418 INFO:tasks.workunit.client.0.smithi027.stdout:8/338: chown d7/d10/d21/d66 9824 1 2021-11-03T17:14:09.418 INFO:tasks.workunit.client.0.smithi027.stdout:8/339: chown d7/l50 231 1 2021-11-03T17:14:09.419 INFO:tasks.workunit.client.0.smithi027.stdout:1/270: mkdir d6/d11/d14/d1f/d25/d28/d5c 0 2021-11-03T17:14:09.421 INFO:tasks.workunit.client.0.smithi027.stdout:8/340: mkdir d7/d53/d67 0 2021-11-03T17:14:09.426 INFO:tasks.workunit.client.0.smithi027.stdout:8/341: rename d7/d10/d21/d66 to d7/d53/d67/d68 0 2021-11-03T17:14:09.426 INFO:tasks.workunit.client.0.smithi027.stdout:8/342: chown d7/d10/d4f/l5e 11 1 2021-11-03T17:14:09.426 INFO:tasks.workunit.client.0.smithi027.stdout:8/343: stat d7/d53/d67/d68 0 2021-11-03T17:14:09.427 INFO:tasks.workunit.client.0.smithi027.stdout:8/344: creat d7/d10/d3f/d55/f69 x:0 0 0 2021-11-03T17:14:09.428 INFO:tasks.workunit.client.0.smithi027.stdout:8/345: symlink d7/l6a 0 2021-11-03T17:14:09.429 INFO:tasks.workunit.client.0.smithi027.stdout:8/346: stat d7/l2c 0 2021-11-03T17:14:09.429 INFO:tasks.workunit.client.0.smithi027.stdout:8/347: fsync d7/f1b 0 2021-11-03T17:14:09.430 INFO:tasks.workunit.client.0.smithi027.stdout:4/276: dwrite d6/d24/f46 [0,4194304] 0 2021-11-03T17:14:09.430 INFO:tasks.workunit.client.0.smithi027.stdout:4/277: dread - d6/db/d21/d2b/f3f zero size 2021-11-03T17:14:09.431 INFO:tasks.workunit.client.0.smithi027.stdout:1/271: dread d6/fd [0,4194304] 0 2021-11-03T17:14:09.431 INFO:tasks.workunit.client.0.smithi027.stdout:8/348: rename d7/d10/d18/d1a/f43 to d7/d53/d67/f6b 0 2021-11-03T17:14:09.432 INFO:tasks.workunit.client.0.smithi027.stdout:4/278: mkdir d6/d9/d16/d36/d57/d5a 0 2021-11-03T17:14:09.433 INFO:tasks.workunit.client.0.smithi027.stdout:1/272: creat d6/de/d13/d15/d43/f5d x:0 0 0 2021-11-03T17:14:09.433 INFO:tasks.workunit.client.0.smithi027.stdout:9/341: sync 2021-11-03T17:14:09.435 INFO:tasks.workunit.client.0.smithi027.stdout:4/279: rename d6/db/l15 to d6/l5b 0 2021-11-03T17:14:09.436 INFO:tasks.workunit.client.0.smithi027.stdout:1/273: fdatasync d6/fa 0 2021-11-03T17:14:09.439 INFO:tasks.workunit.client.0.smithi027.stdout:9/342: dread db/d1b/f45 [0,4194304] 0 2021-11-03T17:14:09.439 INFO:tasks.workunit.client.0.smithi027.stdout:9/343: truncate db/d1b/d23/d2c/f4c 730731 0 2021-11-03T17:14:09.439 INFO:tasks.workunit.client.0.smithi027.stdout:9/344: dread - db/d1b/d2b/d41/f74 zero size 2021-11-03T17:14:09.440 INFO:tasks.workunit.client.0.smithi027.stdout:9/345: mknod db/d1b/d2b/d41/c7e 1 2021-11-03T17:14:09.441 INFO:tasks.workunit.client.0.smithi027.stdout:8/349: getdents d7 0 2021-11-03T17:14:09.442 INFO:tasks.workunit.client.0.smithi027.stdout:8/350: read d7/d10/d18/d1a/f3c [143446,30946] 0 2021-11-03T17:14:09.448 INFO:tasks.workunit.client.0.smithi027.stdout:4/280: unlink d6/d9/f11 0 2021-11-03T17:14:09.449 INFO:tasks.workunit.client.0.smithi027.stdout:9/346: getdents db/d1b/d1c/d1d/d40 0 2021-11-03T17:14:09.449 INFO:tasks.workunit.client.0.smithi027.stdout:9/347: readlink db/d1b/d2b/d41/l58 0 2021-11-03T17:14:09.451 INFO:tasks.workunit.client.0.smithi027.stdout:8/351: rename d7/d10/d18/d1a/f26 to d7/d53/f6c 0 2021-11-03T17:14:09.452 INFO:tasks.workunit.client.0.smithi027.stdout:8/352: chown l4 392631 1 2021-11-03T17:14:09.452 INFO:tasks.workunit.client.0.smithi027.stdout:4/281: truncate d6/d9/d16/f32 6609242 0 2021-11-03T17:14:09.454 INFO:tasks.workunit.client.0.smithi027.stdout:9/348: link db/d1b/d1c/l50 db/d1b/d2b/l7f 0 2021-11-03T17:14:09.454 INFO:tasks.workunit.client.0.smithi027.stdout:1/274: dwrite d6/d11/d14/d1f/d25/d28/f55 [0,4194304] 0 2021-11-03T17:14:09.455 INFO:tasks.workunit.client.0.smithi027.stdout:1/275: dread - d6/d11/d14/d1f/d25/d28/d4c/f4f zero size 2021-11-03T17:14:09.456 INFO:tasks.workunit.client.0.smithi027.stdout:8/353: rename d7/l2f to d7/l6d 0 2021-11-03T17:14:09.458 INFO:tasks.workunit.client.0.smithi027.stdout:4/282: creat d6/d9/d16/d36/d57/d5a/f5c x:0 0 0 2021-11-03T17:14:09.462 INFO:tasks.workunit.client.0.smithi027.stdout:9/349: symlink db/d1b/d23/l80 0 2021-11-03T17:14:09.462 INFO:tasks.workunit.client.0.smithi027.stdout:9/350: truncate db/d1b/d23/f7a 208807 0 2021-11-03T17:14:09.462 INFO:tasks.workunit.client.0.smithi027.stdout:9/351: chown db/d1b/d1c/l50 42 1 2021-11-03T17:14:09.463 INFO:tasks.workunit.client.0.smithi027.stdout:1/276: mkdir d6/d11/d14/d1f/d25/d5e 0 2021-11-03T17:14:09.466 INFO:tasks.workunit.client.0.smithi027.stdout:8/354: creat d7/d2d/d38/f6e x:0 0 0 2021-11-03T17:14:09.467 INFO:tasks.workunit.client.0.smithi027.stdout:8/355: truncate d7/d10/d18/f48 4304278 0 2021-11-03T17:14:09.470 INFO:tasks.workunit.client.0.smithi027.stdout:4/283: dread d6/d9/d16/f26 [0,4194304] 0 2021-11-03T17:14:09.471 INFO:tasks.workunit.client.0.smithi027.stdout:4/284: dread - d6/f48 zero size 2021-11-03T17:14:09.472 INFO:tasks.workunit.client.0.smithi027.stdout:9/352: rename db/l77 to db/d1b/d2b/d41/d6c/l81 0 2021-11-03T17:14:09.474 INFO:tasks.workunit.client.0.smithi027.stdout:8/356: rmdir d7/d10/d18/d1a 39 2021-11-03T17:14:09.474 INFO:tasks.workunit.client.0.smithi027.stdout:8/357: mknod d7/d10/c6f 1 2021-11-03T17:14:09.474 INFO:tasks.workunit.client.0.smithi027.stdout:8/358: dread - d7/f36 zero size 2021-11-03T17:14:09.475 INFO:tasks.workunit.client.0.smithi027.stdout:8/359: fdatasync d7/d10/d3f/f4c 0 2021-11-03T17:14:09.477 INFO:tasks.workunit.client.0.smithi027.stdout:1/277: dwrite d6/d11/f27 [0,4194304] 0 2021-11-03T17:14:09.478 INFO:tasks.workunit.client.0.smithi027.stdout:1/278: chown d6/de/d44 87329 1 2021-11-03T17:14:09.479 INFO:tasks.workunit.client.0.smithi027.stdout:1/279: read d6/de/f1a [3052151,111566] 0 2021-11-03T17:14:09.481 INFO:tasks.workunit.client.0.smithi027.stdout:1/280: unlink d6/fd 0 2021-11-03T17:14:09.489 INFO:tasks.workunit.client.0.smithi027.stdout:8/360: dwrite d7/d10/d18/d1a/f3e [0,4194304] 0 2021-11-03T17:14:09.497 INFO:tasks.workunit.client.0.smithi027.stdout:1/281: dwrite d6/de/d13/d15/f42 [0,4194304] 0 2021-11-03T17:14:09.506 INFO:tasks.workunit.client.0.smithi027.stdout:1/282: truncate d6/d11/f27 3627645 0 2021-11-03T17:14:09.508 INFO:tasks.workunit.client.0.smithi027.stdout:1/283: creat d6/d11/d14/d1f/d25/d28/d41/f5f x:0 0 0 2021-11-03T17:14:09.508 INFO:tasks.workunit.client.0.smithi027.stdout:1/284: dread - d6/de/d13/d15/d43/f5d zero size 2021-11-03T17:14:09.509 INFO:tasks.workunit.client.0.smithi027.stdout:1/285: write d6/fa [1126807,60863] 0 2021-11-03T17:14:09.510 INFO:tasks.workunit.client.0.smithi027.stdout:1/286: chown d6/d11/d14/d1f/d31/d58 41879109 1 2021-11-03T17:14:09.510 INFO:tasks.workunit.client.0.smithi027.stdout:1/287: mknod d6/de/d13/d15/d4e/c60 1 2021-11-03T17:14:09.511 INFO:tasks.workunit.client.0.smithi027.stdout:1/288: symlink d6/d11/d14/d1f/l61 0 2021-11-03T17:14:09.511 INFO:tasks.workunit.client.0.smithi027.stdout:1/289: chown d6/fa 16206495 1 2021-11-03T17:14:09.511 INFO:tasks.workunit.client.0.smithi027.stdout:1/290: mknod d6/d11/d14/d1f/d25/d28/c62 1 2021-11-03T17:14:09.511 INFO:tasks.workunit.client.0.smithi027.stdout:1/291: chown d6/d11/d14/d1f/d25/d28/d32 211246940 1 2021-11-03T17:14:09.512 INFO:tasks.workunit.client.0.smithi027.stdout:1/292: link d6/d11/d14/d1f/d31/f34 d6/d11/d14/d1f/f63 0 2021-11-03T17:14:09.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:09 smithi027 conmon[32322]: cephadm 2021-11-03T17:14:08.225995+0000 mgr.smithi027.ufhhvb (mgr.14162) 2021-11-03T17:14:09.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:09 smithi027 conmon[32322]: 301 : cephadm [INF] Upgrade: Updating mgr.smithi115.tcyodz 2021-11-03T17:14:09.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:09 smithi027 conmon[32322]: audit 2021-11-03T17:14:08.232164+0000 mon.smithi027 (mon.0) 567 2021-11-03T17:14:09.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:09 smithi027 conmon[32322]: : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:14:09.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:09 smithi027 conmon[32322]: audit 2021-11-03T17:14:09.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:09 smithi027 conmon[32322]: 2021-11-03T17:14: 2021-11-03T17:14:09.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:09 smithi027 conmon[32322]: 08.233440+0000 mon.smithi027 (mon 2021-11-03T17:14:09.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:09 smithi027 conmon[32322]: .0) 568 : audit [INF] 2021-11-03T17:14:09.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:09 smithi027 conmon[32322]: from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi115.tcyodz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2021-11-03T17:14:09.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:09 smithi027 conmon[32322]: audit 2021- 2021-11-03T17:14:09.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:09 smithi027 conmon[32322]: 11-03T17:14:08.234450+0000 2021-11-03T17:14:09.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:09 smithi027 conmon[32322]: mon.smithi027 (mon.0) 569 : audit 2021-11-03T17:14:09.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:09 smithi027 conmon[32322]: [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mgr services"}]: dispatch 2021-11-03T17:14:09.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:09 smithi027 conmon[32322]: audit 2021-11-03T17:14:08.235365+0000 mon.smithi027 (mon.0) 570 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:14:09.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:09 smithi027 conmon[32322]: cephadm 2021-11-03T17:14:08.236215+0000 mgr.smithi027.ufhhvb (mgr.14162) 302 : cephadm [INF] Deploying daemon mgr.smithi115.tcyodz on smithi115 2021-11-03T17:14:09.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:09 smithi027 conmon[32322]: cluster 2021-11-03T17:14:08.604529+0000 mgr.smithi027.ufhhvb (mgr.14162) 303 : cluster [DBG] pgmap v251: 65 pgs: 65 active+clean; 1.0 GiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 40 MiB/s rd, 91 MiB/s wr, 150 op/s 2021-11-03T17:14:09.524 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:09 smithi115 conmon[34670]: cephadm 2021-11-03T17:14:08.225995+0000 mgr.smithi027.ufhhvb (mgr.14162) 301 : cephadm [INF] Upgrade: Updating mgr.smithi115.tcyodz 2021-11-03T17:14:09.525 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:09 smithi115 conmon[34670]: audit 2021-11-03T17:14:08.232164+0000 mon.smithi027 (mon.0) 567 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:14:09.525 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:09 smithi115 conmon[34670]: audit 2021-11-03T17:14:08.233440+0000 mon.smithi027 (mon.0) 568 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi115.tcyodz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2021-11-03T17:14:09.525 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:09 smithi115 conmon[34670]: audit 2021-11-03T17:14:08.234450+0000 mon.smithi027 (mon.0) 569 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mgr services"}]: dispatch 2021-11-03T17:14:09.525 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:09 smithi115 conmon[34670]: audit 2021-11-03T17:14:08.235365+0000 mon.smithi027 (mon.0) 570 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:14:09.525 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:09 smithi115 conmon[34670]: cephadm 2021-11-03T17:14:08.236215+0000 mgr.smithi027.ufhhvb (mgr.14162) 302 : cephadm [INF] Deploying daemon mgr.smithi115.tcyodz on smithi115 2021-11-03T17:14:09.526 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:09 smithi115 conmon[34670]: cluster 2021-11-03T17:14:08.604529+0000 mgr.smithi027.ufhhvb (mgr.14162) 303 : cluster [DBG] pgmap v251: 65 pgs: 65 active+clean; 1.0 GiB data, 3.1 GiB used, 533 GiB / 536 GiB avail; 40 MiB/s rd, 91 MiB/s wr, 150 op/s 2021-11-03T17:14:09.530 INFO:tasks.workunit.client.0.smithi027.stdout:1/293: dwrite d6/de/d13/d15/f42 [0,4194304] 0 2021-11-03T17:14:09.530 INFO:tasks.workunit.client.0.smithi027.stdout:1/294: mknod d6/de/d13/c64 1 2021-11-03T17:14:09.531 INFO:tasks.workunit.client.0.smithi027.stdout:1/295: rmdir d6/d11/d14 39 2021-11-03T17:14:09.531 INFO:tasks.workunit.client.0.smithi027.stdout:1/296: mknod d6/d11/d14/d1f/d31/c65 1 2021-11-03T17:14:09.532 INFO:tasks.workunit.client.0.smithi027.stdout:1/297: creat d6/de/d13/f66 x:0 0 0 2021-11-03T17:14:09.533 INFO:tasks.workunit.client.0.smithi027.stdout:1/298: write d6/de/d13/d15/f2f [463509,36997] 0 2021-11-03T17:14:09.542 INFO:tasks.workunit.client.0.smithi027.stdout:1/299: dread d6/d11/d14/d1f/f22 [0,4194304] 0 2021-11-03T17:14:09.544 INFO:tasks.workunit.client.0.smithi027.stdout:1/300: symlink d6/d11/d14/d1f/d25/d28/l67 0 2021-11-03T17:14:09.544 INFO:tasks.workunit.client.0.smithi027.stdout:1/301: write d6/d11/d14/d1f/f59 [621328,45485] 0 2021-11-03T17:14:09.550 INFO:tasks.workunit.client.0.smithi027.stdout:1/302: dread d6/de/d13/d15/d43/f48 [0,4194304] 0 2021-11-03T17:14:09.552 INFO:tasks.workunit.client.0.smithi027.stdout:1/303: mkdir d6/d11/d14/d1f/d25/d28/d5c/d68 0 2021-11-03T17:14:09.552 INFO:tasks.workunit.client.0.smithi027.stdout:1/304: mknod d6/d11/d14/d1f/d25/d28/d32/c69 1 2021-11-03T17:14:09.559 INFO:tasks.workunit.client.0.smithi027.stdout:1/305: rename d6/f23 to d6/f6a 0 2021-11-03T17:14:09.559 INFO:tasks.workunit.client.0.smithi027.stdout:1/306: fdatasync d6/fa 0 2021-11-03T17:14:09.560 INFO:tasks.workunit.client.0.smithi027.stdout:1/307: creat d6/de/d13/f6b x:0 0 0 2021-11-03T17:14:09.562 INFO:tasks.workunit.client.0.smithi027.stdout:1/308: creat d6/d11/d14/d1f/d25/d28/d5c/d68/f6c x:0 0 0 2021-11-03T17:14:09.564 INFO:tasks.workunit.client.0.smithi027.stdout:1/309: truncate f2 6152087 0 2021-11-03T17:14:09.564 INFO:tasks.workunit.client.0.smithi027.stdout:1/310: truncate d6/d11/d14/d1f/d37/d49/f50 954379 0 2021-11-03T17:14:09.566 INFO:tasks.workunit.client.0.smithi027.stdout:1/311: creat d6/de/d44/f6d x:0 0 0 2021-11-03T17:14:09.575 INFO:tasks.workunit.client.0.smithi027.stdout:1/312: dread d6/d11/d14/d1f/d31/f54 [0,4194304] 0 2021-11-03T17:14:09.575 INFO:tasks.workunit.client.0.smithi027.stdout:1/313: dread - d6/d11/d14/d1f/f63 zero size 2021-11-03T17:14:09.593 INFO:tasks.workunit.client.0.smithi027.stdout:1/314: dwrite d6/de/d13/f6b [0,4194304] 0 2021-11-03T17:14:09.595 INFO:tasks.workunit.client.0.smithi027.stdout:1/315: symlink d6/de/d13/d15/l6e 0 2021-11-03T17:14:09.596 INFO:tasks.workunit.client.0.smithi027.stdout:1/316: creat d6/d11/d14/d1f/d25/d28/d41/f6f x:0 0 0 2021-11-03T17:14:09.598 INFO:tasks.workunit.client.0.smithi027.stdout:1/317: dread d6/de/d13/f38 [0,4194304] 0 2021-11-03T17:14:09.615 INFO:tasks.workunit.client.0.smithi027.stdout:1/318: dwrite d6/fa [0,4194304] 0 2021-11-03T17:14:09.617 INFO:tasks.workunit.client.0.smithi027.stdout:1/319: creat d6/d11/d14/d1f/d31/d58/f70 x:0 0 0 2021-11-03T17:14:09.634 INFO:tasks.workunit.client.0.smithi027.stdout:1/320: dwrite d6/de/f19 [4194304,4194304] 0 2021-11-03T17:14:09.634 INFO:tasks.workunit.client.0.smithi027.stdout:1/321: mknod d6/d11/d14/d1f/d37/c71 1 2021-11-03T17:14:09.635 INFO:tasks.workunit.client.0.smithi027.stdout:1/322: mkdir d6/d11/d14/d1f/d25/d5e/d72 0 2021-11-03T17:14:09.638 INFO:tasks.workunit.client.0.smithi027.stdout:1/323: link d6/de/d13/d15/f4b d6/d11/d14/d1f/d25/d28/d5c/f73 0 2021-11-03T17:14:09.639 INFO:tasks.workunit.client.0.smithi027.stdout:0/382: sync 2021-11-03T17:14:09.639 INFO:tasks.workunit.client.0.smithi027.stdout:0/383: truncate d9/f13 1486275 0 2021-11-03T17:14:09.639 INFO:tasks.workunit.client.0.smithi027.stdout:1/324: truncate d6/de/d13/d15/f2f 131397 0 2021-11-03T17:14:09.640 INFO:tasks.workunit.client.0.smithi027.stdout:1/325: creat d6/d11/d14/d1f/d25/f74 x:0 0 0 2021-11-03T17:14:09.640 INFO:tasks.workunit.client.0.smithi027.stdout:1/326: mknod d6/d11/c75 1 2021-11-03T17:14:09.649 INFO:tasks.workunit.client.0.smithi027.stdout:1/327: link d6/de/d13/d15/f16 d6/d11/d14/f76 0 2021-11-03T17:14:09.649 INFO:tasks.workunit.client.0.smithi027.stdout:1/328: mknod d6/d11/d14/d1f/d25/d28/d4c/c77 1 2021-11-03T17:14:09.656 INFO:tasks.workunit.client.0.smithi027.stdout:1/329: creat d6/d11/d14/d1f/d25/d28/d5c/f78 x:0 0 0 2021-11-03T17:14:09.666 INFO:tasks.workunit.client.0.smithi027.stdout:1/330: rename d6/d11/d14/d1f to d6/de/d44/d79 0 2021-11-03T17:14:09.666 INFO:tasks.workunit.client.0.smithi027.stdout:1/331: truncate d6/de/d44/d79/d31/f34 656475 0 2021-11-03T17:14:09.667 INFO:tasks.workunit.client.0.smithi027.stdout:1/332: mkdir d6/de/d44/d79/d25/d28/d5c/d7a 0 2021-11-03T17:14:09.668 INFO:tasks.workunit.client.0.smithi027.stdout:1/333: mknod d6/de/d44/d79/d25/d28/d5c/d7a/c7b 1 2021-11-03T17:14:10.077 INFO:tasks.workunit.client.0.smithi027.stdout:3/140: sync 2021-11-03T17:14:10.077 INFO:tasks.workunit.client.0.smithi027.stdout:2/279: sync 2021-11-03T17:14:10.077 INFO:tasks.workunit.client.0.smithi027.stdout:7/227: sync 2021-11-03T17:14:10.078 INFO:tasks.workunit.client.0.smithi027.stdout:7/228: truncate d0/d2/d1c/f27 4381813 0 2021-11-03T17:14:10.078 INFO:tasks.workunit.client.0.smithi027.stdout:7/229: truncate d0/d2/d10/f11 3697787 0 2021-11-03T17:14:10.078 INFO:tasks.workunit.client.0.smithi027.stdout:3/141: symlink d1/l27 0 2021-11-03T17:14:10.079 INFO:tasks.workunit.client.0.smithi027.stdout:3/142: symlink d1/db/l28 0 2021-11-03T17:14:10.082 INFO:tasks.workunit.client.0.smithi027.stdout:3/143: dread d1/f1d [0,4194304] 0 2021-11-03T17:14:10.086 INFO:tasks.workunit.client.0.smithi027.stdout:7/230: dread d0/f13 [4194304,4194304] 0 2021-11-03T17:14:10.086 INFO:tasks.workunit.client.0.smithi027.stdout:7/231: mknod d0/d2/d10/d1e/d20/d31/d36/d3a/c47 1 2021-11-03T17:14:10.087 INFO:tasks.workunit.client.0.smithi027.stdout:7/232: write d0/d2/d10/f11 [1416929,106258] 0 2021-11-03T17:14:10.095 INFO:tasks.workunit.client.0.smithi027.stdout:7/233: dread d0/d2/d1c/f27 [0,4194304] 0 2021-11-03T17:14:10.096 INFO:tasks.workunit.client.0.smithi027.stdout:2/280: dwrite d0/d1b/d29/f2b [0,4194304] 0 2021-11-03T17:14:10.096 INFO:tasks.workunit.client.0.smithi027.stdout:2/281: write d0/d2a/f21 [202550,20528] 0 2021-11-03T17:14:10.097 INFO:tasks.workunit.client.0.smithi027.stdout:2/282: chown d0/d1b/l4c 269541 1 2021-11-03T17:14:10.099 INFO:tasks.workunit.client.0.smithi027.stdout:7/234: link d0/d2/f5 d0/d2/d10/d1e/f48 0 2021-11-03T17:14:10.099 INFO:tasks.workunit.client.0.smithi027.stdout:7/235: stat d0/d2/d10 0 2021-11-03T17:14:10.112 INFO:tasks.workunit.client.0.smithi027.stdout:7/236: creat d0/d2/d10/d35/f49 x:0 0 0 2021-11-03T17:14:10.113 INFO:tasks.workunit.client.0.smithi027.stdout:7/237: mknod d0/d2/d10/d35/c4a 1 2021-11-03T17:14:10.114 INFO:tasks.workunit.client.0.smithi027.stdout:7/238: write d0/f16 [1311032,52215] 0 2021-11-03T17:14:10.114 INFO:tasks.workunit.client.0.smithi027.stdout:7/239: write d0/d2/d1f/f46 [554688,31621] 0 2021-11-03T17:14:10.114 INFO:tasks.workunit.client.0.smithi027.stdout:7/240: readlink d0/lb 0 2021-11-03T17:14:10.114 INFO:tasks.workunit.client.0.smithi027.stdout:7/241: mknod d0/d2/d1f/c4b 1 2021-11-03T17:14:10.115 INFO:tasks.workunit.client.0.smithi027.stdout:2/283: dwrite d0/d1b/d29/f4b [0,4194304] 0 2021-11-03T17:14:10.115 INFO:tasks.workunit.client.0.smithi027.stdout:2/284: read d0/d2a/f21 [124398,38621] 0 2021-11-03T17:14:10.117 INFO:tasks.workunit.client.0.smithi027.stdout:7/242: creat d0/f4c x:0 0 0 2021-11-03T17:14:10.117 INFO:tasks.workunit.client.0.smithi027.stdout:7/243: chown d0/d2d/d2f/f39 60887275 1 2021-11-03T17:14:10.119 INFO:tasks.workunit.client.0.smithi027.stdout:2/285: creat d0/d1b/d1c/f5e x:0 0 0 2021-11-03T17:14:10.120 INFO:tasks.workunit.client.0.smithi027.stdout:2/286: chown d0/f10 5069 1 2021-11-03T17:14:10.120 INFO:tasks.workunit.client.0.smithi027.stdout:7/244: mkdir d0/d2/d1c/d4d 0 2021-11-03T17:14:10.122 INFO:tasks.workunit.client.0.smithi027.stdout:2/287: truncate d0/d5/f13 7014317 0 2021-11-03T17:14:10.133 INFO:tasks.workunit.client.0.smithi027.stdout:2/288: dread d0/d1b/d29/f4b [0,4194304] 0 2021-11-03T17:14:10.135 INFO:tasks.workunit.client.0.smithi027.stdout:2/289: rename d0/d1b/d29/l47 to d0/d1b/d29/d58/d5b/l5f 0 2021-11-03T17:14:10.137 INFO:tasks.workunit.client.0.smithi027.stdout:2/290: symlink d0/d1b/l60 0 2021-11-03T17:14:10.137 INFO:tasks.workunit.client.0.smithi027.stdout:2/291: chown d0/d1b/d29/f2f 440279808 1 2021-11-03T17:14:10.138 INFO:tasks.workunit.client.0.smithi027.stdout:2/292: write d0/f41 [703086,64405] 0 2021-11-03T17:14:10.138 INFO:tasks.workunit.client.0.smithi027.stdout:2/293: mkdir d0/d2a/d61 0 2021-11-03T17:14:10.139 INFO:tasks.workunit.client.0.smithi027.stdout:2/294: write d0/d2a/f44 [259003,21209] 0 2021-11-03T17:14:10.155 INFO:tasks.workunit.client.0.smithi027.stdout:2/295: dwrite d0/d1b/d1c/f26 [4194304,4194304] 0 2021-11-03T17:14:10.155 INFO:tasks.workunit.client.0.smithi027.stdout:2/296: fsync d0/d1b/d1c/f24 0 2021-11-03T17:14:10.159 INFO:tasks.workunit.client.0.smithi027.stdout:2/297: rmdir d0/d1b/d29/d58 39 2021-11-03T17:14:10.162 INFO:tasks.workunit.client.0.smithi027.stdout:2/298: creat d0/d1b/d29/d58/d5b/f62 x:0 0 0 2021-11-03T17:14:10.163 INFO:tasks.workunit.client.0.smithi027.stdout:2/299: dread - d0/f5a zero size 2021-11-03T17:14:10.163 INFO:tasks.workunit.client.0.smithi027.stdout:2/300: chown d0/l31 1424 1 2021-11-03T17:14:10.181 INFO:tasks.workunit.client.1.smithi115.stdout:2/223: sync 2021-11-03T17:14:10.181 INFO:tasks.workunit.client.1.smithi115.stdout:8/236: sync 2021-11-03T17:14:10.181 INFO:tasks.workunit.client.1.smithi115.stdout:6/200: sync 2021-11-03T17:14:10.182 INFO:tasks.workunit.client.1.smithi115.stdout:3/285: sync 2021-11-03T17:14:10.182 INFO:tasks.workunit.client.1.smithi115.stdout:8/237: readlink d4/l10 0 2021-11-03T17:14:10.182 INFO:tasks.workunit.client.1.smithi115.stdout:5/357: sync 2021-11-03T17:14:10.183 INFO:tasks.workunit.client.1.smithi115.stdout:5/358: write d1a/d3d/f47 [2850816,67476] 0 2021-11-03T17:14:10.183 INFO:tasks.workunit.client.1.smithi115.stdout:5/359: stat d1a/d28/d4c/d75/d68 0 2021-11-03T17:14:10.183 INFO:tasks.workunit.client.1.smithi115.stdout:5/360: truncate d1a/d3d/f47 3754796 0 2021-11-03T17:14:10.184 INFO:tasks.workunit.client.0.smithi027.stdout:2/301: dwrite d0/d1b/d29/f4e [0,4194304] 0 2021-11-03T17:14:10.184 INFO:tasks.workunit.client.1.smithi115.stdout:3/286: rmdir d13 39 2021-11-03T17:14:10.185 INFO:tasks.workunit.client.1.smithi115.stdout:6/201: link d0/f17 d0/d1/d10/f3b 0 2021-11-03T17:14:10.185 INFO:tasks.workunit.client.1.smithi115.stdout:2/224: getdents d4/d6/d2e/d34/d43 0 2021-11-03T17:14:10.187 INFO:tasks.workunit.client.0.smithi027.stdout:2/302: dread d0/d1b/d1c/f26 [0,4194304] 0 2021-11-03T17:14:10.189 INFO:tasks.workunit.client.0.smithi027.stdout:2/303: write d0/d1b/d29/f2b [3927299,56132] 0 2021-11-03T17:14:10.189 INFO:tasks.workunit.client.0.smithi027.stdout:2/304: chown d0/d1b/d29/d58/f5c 10823 1 2021-11-03T17:14:10.189 INFO:tasks.workunit.client.0.smithi027.stdout:2/305: stat d0/d2a/d61 0 2021-11-03T17:14:10.191 INFO:tasks.workunit.client.0.smithi027.stdout:2/306: unlink d0/d5/d2c/l3e 0 2021-11-03T17:14:10.194 INFO:tasks.workunit.client.0.smithi027.stdout:2/307: unlink d0/d2a/l3f 0 2021-11-03T17:14:10.196 INFO:tasks.workunit.client.0.smithi027.stdout:2/308: dread d0/d2a/f30 [0,4194304] 0 2021-11-03T17:14:10.198 INFO:tasks.workunit.client.0.smithi027.stdout:2/309: symlink d0/d2a/d61/l63 0 2021-11-03T17:14:10.201 INFO:tasks.workunit.client.0.smithi027.stdout:2/310: rename d0/d1b/d43 to d0/d1b/d29/d54/d64 0 2021-11-03T17:14:10.203 INFO:tasks.workunit.client.0.smithi027.stdout:2/311: unlink d0/f5a 0 2021-11-03T17:14:10.203 INFO:tasks.workunit.client.0.smithi027.stdout:2/312: stat d0/d1b/d29/d54/d64 0 2021-11-03T17:14:10.204 INFO:tasks.workunit.client.0.smithi027.stdout:2/313: truncate d0/d5/d2c/f3d 93532 0 2021-11-03T17:14:10.204 INFO:tasks.workunit.client.0.smithi027.stdout:2/314: dread - d0/d1b/d29/f48 zero size 2021-11-03T17:14:10.205 INFO:tasks.workunit.client.1.smithi115.stdout:2/225: dwrite d4/d6/d2e/d34/d43/f3d [0,4194304] 0 2021-11-03T17:14:10.205 INFO:tasks.workunit.client.0.smithi027.stdout:2/315: creat d0/d2a/d5d/f65 x:0 0 0 2021-11-03T17:14:10.227 INFO:tasks.workunit.client.0.smithi027.stdout:2/316: write d0/d2a/f21 [4529696,5681] 0 2021-11-03T17:14:10.227 INFO:tasks.workunit.client.0.smithi027.stdout:2/317: dread - d0/d2a/d5d/f65 zero size 2021-11-03T17:14:10.228 INFO:tasks.workunit.client.0.smithi027.stdout:2/318: stat d0/d1b/d29/f4b 0 2021-11-03T17:14:10.228 INFO:tasks.workunit.client.1.smithi115.stdout:8/238: rename d4/l15 to d4/d6/d18/d44/l50 0 2021-11-03T17:14:10.228 INFO:tasks.workunit.client.1.smithi115.stdout:8/239: fsync d4/fa 0 2021-11-03T17:14:10.229 INFO:tasks.workunit.client.1.smithi115.stdout:5/361: mkdir d1a/d28/d78 0 2021-11-03T17:14:10.230 INFO:tasks.workunit.client.1.smithi115.stdout:3/287: getdents d13/d3b 0 2021-11-03T17:14:10.230 INFO:tasks.workunit.client.1.smithi115.stdout:3/288: chown d13/d35 76 1 2021-11-03T17:14:10.231 INFO:tasks.workunit.client.1.smithi115.stdout:6/202: symlink d0/d1/d10/d13/d1c/d1e/l3c 0 2021-11-03T17:14:10.238 INFO:tasks.workunit.client.1.smithi115.stdout:2/226: symlink d4/d44/l47 0 2021-11-03T17:14:10.238 INFO:tasks.workunit.client.1.smithi115.stdout:5/362: symlink d1a/d51/l79 0 2021-11-03T17:14:10.239 INFO:tasks.workunit.client.1.smithi115.stdout:3/289: symlink d13/l62 0 2021-11-03T17:14:10.239 INFO:tasks.workunit.client.1.smithi115.stdout:3/290: mknod d13/d2a/d53/c63 1 2021-11-03T17:14:10.240 INFO:tasks.workunit.client.1.smithi115.stdout:2/227: creat d4/d6/f48 x:0 0 0 2021-11-03T17:14:10.240 INFO:tasks.workunit.client.1.smithi115.stdout:2/228: mknod d4/d6/d2e/d34/c49 1 2021-11-03T17:14:10.240 INFO:tasks.workunit.client.1.smithi115.stdout:5/363: symlink d1a/d28/d34/l7a 0 2021-11-03T17:14:10.244 INFO:tasks.workunit.client.1.smithi115.stdout:5/364: rename d1a/d28/d44 to d1a/d28/d7b 0 2021-11-03T17:14:10.251 INFO:tasks.workunit.client.1.smithi115.stdout:6/203: dwrite d0/d1/d2/fc [0,4194304] 0 2021-11-03T17:14:10.257 INFO:tasks.workunit.client.1.smithi115.stdout:8/240: dwrite d4/f27 [0,4194304] 0 2021-11-03T17:14:10.257 INFO:tasks.workunit.client.1.smithi115.stdout:8/241: mknod d4/d6/d18/d30/d33/d4c/c51 1 2021-11-03T17:14:10.258 INFO:tasks.workunit.client.1.smithi115.stdout:8/242: mknod d4/d6/d3f/d43/c52 1 2021-11-03T17:14:10.259 INFO:tasks.workunit.client.1.smithi115.stdout:8/243: truncate d4/d6/d18/d30/f34 7509590 0 2021-11-03T17:14:10.261 INFO:tasks.workunit.client.1.smithi115.stdout:3/291: dwrite d13/d51/f2d [0,4194304] 0 2021-11-03T17:14:10.261 INFO:tasks.workunit.client.1.smithi115.stdout:3/292: mknod d13/d51/c64 1 2021-11-03T17:14:10.262 INFO:tasks.workunit.client.1.smithi115.stdout:8/244: rename d4/d6/d18/l25 to d4/d6/d18/d30/d33/d4c/l53 0 2021-11-03T17:14:10.263 INFO:tasks.workunit.client.1.smithi115.stdout:3/293: getdents d13/d51/d40 0 2021-11-03T17:14:10.265 INFO:tasks.workunit.client.1.smithi115.stdout:3/294: creat d13/d2c/f65 x:0 0 0 2021-11-03T17:14:10.265 INFO:tasks.workunit.client.1.smithi115.stdout:3/295: mknod d13/d2a/c66 1 2021-11-03T17:14:10.265 INFO:tasks.workunit.client.1.smithi115.stdout:3/296: write d13/d2c/f65 [222678,66500] 0 2021-11-03T17:14:10.266 INFO:tasks.workunit.client.1.smithi115.stdout:5/365: rmdir d1a/d28/d7b 39 2021-11-03T17:14:10.267 INFO:tasks.workunit.client.1.smithi115.stdout:8/245: write d4/d6/f2b [7457686,87767] 0 2021-11-03T17:14:10.267 INFO:tasks.workunit.client.1.smithi115.stdout:8/246: chown d4/d6/l41 4634 1 2021-11-03T17:14:10.267 INFO:tasks.workunit.client.1.smithi115.stdout:8/247: write d4/fa [4792458,80097] 0 2021-11-03T17:14:10.268 INFO:tasks.workunit.client.1.smithi115.stdout:8/248: unlink d4/d6/d18/f2d 0 2021-11-03T17:14:10.269 INFO:tasks.workunit.client.1.smithi115.stdout:5/366: creat d1a/d3d/d58/f7c x:0 0 0 2021-11-03T17:14:10.270 INFO:tasks.workunit.client.1.smithi115.stdout:8/249: creat d4/d6/d18/d30/d33/d4c/f54 x:0 0 0 2021-11-03T17:14:10.272 INFO:tasks.workunit.client.1.smithi115.stdout:8/250: mknod d4/d6/d3f/c55 1 2021-11-03T17:14:10.272 INFO:tasks.workunit.client.1.smithi115.stdout:2/229: dwrite d4/f3a [0,4194304] 0 2021-11-03T17:14:10.274 INFO:tasks.workunit.client.1.smithi115.stdout:8/251: link d4/d6/d18/d44/l50 d4/d6/d18/d30/l56 0 2021-11-03T17:14:10.274 INFO:tasks.workunit.client.1.smithi115.stdout:6/204: dwrite d0/d1/d2/f1f [8388608,4194304] 0 2021-11-03T17:14:10.275 INFO:tasks.workunit.client.1.smithi115.stdout:8/252: truncate d4/d6/d18/d44/f48 471998 0 2021-11-03T17:14:10.275 INFO:tasks.workunit.client.1.smithi115.stdout:6/205: truncate d0/d1/d10/d13/d1c/d1e/f2b 151030 0 2021-11-03T17:14:10.275 INFO:tasks.workunit.client.1.smithi115.stdout:6/206: mknod d0/d1/d2e/c3d 1 2021-11-03T17:14:10.320 INFO:tasks.workunit.client.1.smithi115.stdout:5/367: dwrite d1a/d3d/f64 [0,4194304] 0 2021-11-03T17:14:10.321 INFO:tasks.workunit.client.1.smithi115.stdout:2/230: dwrite d4/d6/d2e/d34/d43/f2b [0,4194304] 0 2021-11-03T17:14:10.321 INFO:tasks.workunit.client.1.smithi115.stdout:3/297: dwrite d13/d2c/f65 [0,4194304] 0 2021-11-03T17:14:10.322 INFO:tasks.workunit.client.1.smithi115.stdout:2/231: creat d4/d44/f4a x:0 0 0 2021-11-03T17:14:10.323 INFO:tasks.workunit.client.1.smithi115.stdout:3/298: getdents d13/d51/d2e 0 2021-11-03T17:14:10.325 INFO:tasks.workunit.client.1.smithi115.stdout:8/253: dwrite d4/d6/f1d [0,4194304] 0 2021-11-03T17:14:10.325 INFO:tasks.workunit.client.1.smithi115.stdout:8/254: chown d4/d6/d18/d44 13978 1 2021-11-03T17:14:10.329 INFO:tasks.workunit.client.1.smithi115.stdout:2/232: symlink d4/l4b 0 2021-11-03T17:14:10.330 INFO:tasks.workunit.client.1.smithi115.stdout:3/299: getdents d13 0 2021-11-03T17:14:10.334 INFO:tasks.workunit.client.1.smithi115.stdout:5/368: dread d1a/d28/d4c/f6e [0,4194304] 0 2021-11-03T17:14:10.334 INFO:tasks.workunit.client.1.smithi115.stdout:2/233: mknod d4/d6/d2e/d34/d43/d28/c4c 1 2021-11-03T17:14:10.335 INFO:tasks.workunit.client.1.smithi115.stdout:3/300: mkdir d13/d67 0 2021-11-03T17:14:10.336 INFO:tasks.workunit.client.1.smithi115.stdout:8/255: rename d4/d6/d3f/d43/l46 to d4/d6/d3f/l57 0 2021-11-03T17:14:10.336 INFO:tasks.workunit.client.1.smithi115.stdout:8/256: chown d4/d6/d18/d30/d33/d4c/l53 197 1 2021-11-03T17:14:10.337 INFO:tasks.workunit.client.1.smithi115.stdout:5/369: mkdir d1a/d7d 0 2021-11-03T17:14:10.339 INFO:tasks.workunit.client.1.smithi115.stdout:2/234: creat d4/d6/d2e/d34/d43/f4d x:0 0 0 2021-11-03T17:14:10.339 INFO:tasks.workunit.client.1.smithi115.stdout:2/235: fsync d4/d6/d2e/d34/d43/f3e 0 2021-11-03T17:14:10.341 INFO:tasks.workunit.client.1.smithi115.stdout:8/257: rename d4/d6/d18/d44/l50 to d4/d6/d18/d30/d33/l58 0 2021-11-03T17:14:10.343 INFO:tasks.workunit.client.1.smithi115.stdout:2/236: creat d4/d6/d13/f4e x:0 0 0 2021-11-03T17:14:10.343 INFO:tasks.workunit.client.1.smithi115.stdout:2/237: chown d4/d6/f48 12206466 1 2021-11-03T17:14:10.344 INFO:tasks.workunit.client.1.smithi115.stdout:3/301: mkdir d13/d2a/d53/d68 0 2021-11-03T17:14:10.344 INFO:tasks.workunit.client.0.smithi027.stdout:6/268: sync 2021-11-03T17:14:10.345 INFO:tasks.workunit.client.0.smithi027.stdout:6/269: mknod dc/d37/c4b 1 2021-11-03T17:14:10.345 INFO:tasks.workunit.client.1.smithi115.stdout:5/370: link d1a/d3d/l6d d1a/d3d/d58/l7e 0 2021-11-03T17:14:10.346 INFO:tasks.workunit.client.1.smithi115.stdout:2/238: symlink d4/l4f 0 2021-11-03T17:14:10.347 INFO:tasks.workunit.client.1.smithi115.stdout:3/302: chown d13/f1d 12 1 2021-11-03T17:14:10.348 INFO:tasks.workunit.client.1.smithi115.stdout:5/371: truncate d1a/d28/d34/f2a 1676756 0 2021-11-03T17:14:10.348 INFO:tasks.workunit.client.1.smithi115.stdout:5/372: truncate d1a/d28/d34/f66 935577 0 2021-11-03T17:14:10.349 INFO:tasks.workunit.client.1.smithi115.stdout:5/373: chown d1a/d28/d4c 113452 1 2021-11-03T17:14:10.349 INFO:tasks.workunit.client.1.smithi115.stdout:2/239: chown d4/d6/d2e/d34/d43/d28/l31 520901339 1 2021-11-03T17:14:10.350 INFO:tasks.workunit.client.1.smithi115.stdout:8/258: dread d4/d6/f1d [0,4194304] 0 2021-11-03T17:14:10.351 INFO:tasks.workunit.client.1.smithi115.stdout:5/374: mkdir d1a/d28/d7b/d48/d7f 0 2021-11-03T17:14:10.353 INFO:tasks.workunit.client.1.smithi115.stdout:8/259: creat d4/d6/d18/d30/d33/f59 x:0 0 0 2021-11-03T17:14:10.354 INFO:tasks.workunit.client.1.smithi115.stdout:5/375: rmdir d1a/d28/d7b/d48 39 2021-11-03T17:14:10.355 INFO:tasks.workunit.client.1.smithi115.stdout:8/260: symlink d4/d6/d18/d30/l5a 0 2021-11-03T17:14:10.356 INFO:tasks.workunit.client.1.smithi115.stdout:5/376: symlink d1a/d3d/d58/l80 0 2021-11-03T17:14:10.357 INFO:tasks.workunit.client.1.smithi115.stdout:8/261: creat d4/d6/d3f/d43/f5b x:0 0 0 2021-11-03T17:14:10.358 INFO:tasks.workunit.client.1.smithi115.stdout:8/262: write d4/d6/f2c [80623,72551] 0 2021-11-03T17:14:10.358 INFO:tasks.workunit.client.1.smithi115.stdout:8/263: dread - d4/d6/d18/d30/d33/f59 zero size 2021-11-03T17:14:10.359 INFO:tasks.workunit.client.1.smithi115.stdout:8/264: creat d4/d6/d18/d30/d33/f5c x:0 0 0 2021-11-03T17:14:10.360 INFO:tasks.workunit.client.1.smithi115.stdout:8/265: write d4/d6/d18/d30/f35 [237830,5832] 0 2021-11-03T17:14:10.360 INFO:tasks.workunit.client.1.smithi115.stdout:8/266: mknod d4/c5d 1 2021-11-03T17:14:10.362 INFO:tasks.workunit.client.1.smithi115.stdout:8/267: link d4/f27 d4/d6/d3f/d43/f5e 0 2021-11-03T17:14:10.363 INFO:tasks.workunit.client.1.smithi115.stdout:8/268: fsync d4/d6/d3f/f45 0 2021-11-03T17:14:10.364 INFO:tasks.workunit.client.1.smithi115.stdout:2/240: getdents d4/d6/d13 0 2021-11-03T17:14:10.366 INFO:tasks.workunit.client.1.smithi115.stdout:4/428: sync 2021-11-03T17:14:10.366 INFO:tasks.workunit.client.1.smithi115.stdout:0/339: sync 2021-11-03T17:14:10.366 INFO:tasks.workunit.client.1.smithi115.stdout:7/210: sync 2021-11-03T17:14:10.366 INFO:tasks.workunit.client.1.smithi115.stdout:1/253: sync 2021-11-03T17:14:10.367 INFO:tasks.workunit.client.1.smithi115.stdout:9/158: sync 2021-11-03T17:14:10.367 INFO:tasks.workunit.client.1.smithi115.stdout:9/159: write d1/d5/ff [5091887,45018] 0 2021-11-03T17:14:10.371 INFO:tasks.workunit.client.1.smithi115.stdout:3/303: dwrite d13/d51/d2e/f59 [0,4194304] 0 2021-11-03T17:14:10.371 INFO:tasks.workunit.client.1.smithi115.stdout:3/304: chown d13/d51/d40 421830954 1 2021-11-03T17:14:10.372 INFO:tasks.workunit.client.1.smithi115.stdout:4/429: link d7/dc/d6a/f3a d7/dc/d1a/d21/d47/d56/d63/f9b 0 2021-11-03T17:14:10.373 INFO:tasks.workunit.client.1.smithi115.stdout:0/340: mkdir d9/d15/d2c/d6e 0 2021-11-03T17:14:10.373 INFO:tasks.workunit.client.1.smithi115.stdout:0/341: getdents d9/d15 0 2021-11-03T17:14:10.373 INFO:tasks.workunit.client.1.smithi115.stdout:0/342: truncate d9/d15/f16 1271214 0 2021-11-03T17:14:10.374 INFO:tasks.workunit.client.1.smithi115.stdout:0/343: chown d9/d15/l45 0 1 2021-11-03T17:14:10.375 INFO:tasks.workunit.client.1.smithi115.stdout:7/211: creat d3/da/d18/f40 x:0 0 0 2021-11-03T17:14:10.375 INFO:tasks.workunit.client.1.smithi115.stdout:7/212: mknod d3/da/d26/c41 1 2021-11-03T17:14:10.376 INFO:tasks.workunit.client.1.smithi115.stdout:4/430: write d7/dc/d1a/d21/d47/f69 [3331063,98759] 0 2021-11-03T17:14:10.377 INFO:tasks.workunit.client.1.smithi115.stdout:1/254: mkdir d7/dc/dd/d24/d4d 0 2021-11-03T17:14:10.379 INFO:tasks.workunit.client.1.smithi115.stdout:8/269: link d4/d6/f3c d4/d6/d18/d30/d33/f5f 0 2021-11-03T17:14:10.379 INFO:tasks.workunit.client.1.smithi115.stdout:8/270: mknod d4/d6/d18/d44/d47/c60 1 2021-11-03T17:14:10.384 INFO:tasks.workunit.client.1.smithi115.stdout:9/160: rename d1/ld to d1/d5/l31 0 2021-11-03T17:14:10.385 INFO:tasks.workunit.client.1.smithi115.stdout:0/344: symlink d9/d15/d1a/l6f 0 2021-11-03T17:14:10.385 INFO:tasks.workunit.client.1.smithi115.stdout:4/431: rmdir d7 39 2021-11-03T17:14:10.385 INFO:tasks.workunit.client.1.smithi115.stdout:9/161: fdatasync d1/d5/f21 0 2021-11-03T17:14:10.385 INFO:tasks.workunit.client.1.smithi115.stdout:4/432: dread - d7/dc/d1a/d21/d47/d56/f93 zero size 2021-11-03T17:14:10.386 INFO:tasks.workunit.client.1.smithi115.stdout:4/433: dread - d7/d7a/f91 zero size 2021-11-03T17:14:10.386 INFO:tasks.workunit.client.1.smithi115.stdout:4/434: mknod d7/dc/d6a/d72/c9c 1 2021-11-03T17:14:10.387 INFO:tasks.workunit.client.1.smithi115.stdout:1/255: symlink d7/dc/dd/d42/l4e 0 2021-11-03T17:14:10.387 INFO:tasks.workunit.client.1.smithi115.stdout:8/271: creat d4/d6/d3f/f61 x:0 0 0 2021-11-03T17:14:10.387 INFO:tasks.workunit.client.1.smithi115.stdout:0/345: mkdir d9/d15/d2c/d46/d63/d70 0 2021-11-03T17:14:10.388 INFO:tasks.workunit.client.1.smithi115.stdout:1/256: dread d7/dc/dd/d24/f2b [0,4194304] 0 2021-11-03T17:14:10.388 INFO:tasks.workunit.client.1.smithi115.stdout:1/257: stat d7/dc/dd/d3e 0 2021-11-03T17:14:10.389 INFO:tasks.workunit.client.1.smithi115.stdout:4/435: rename d7/dc/d6a/d3c/l7f to d7/dc/d1a/d21/d47/d56/d63/d7b/d96/l9d 0 2021-11-03T17:14:10.390 INFO:tasks.workunit.client.1.smithi115.stdout:9/162: mkdir d1/d32 0 2021-11-03T17:14:10.390 INFO:tasks.workunit.client.1.smithi115.stdout:9/163: getdents d1/d32 0 2021-11-03T17:14:10.390 INFO:tasks.workunit.client.1.smithi115.stdout:9/164: dread - d1/d2d/f2a zero size 2021-11-03T17:14:10.391 INFO:tasks.workunit.client.1.smithi115.stdout:9/165: mknod d1/d2d/d27/c33 1 2021-11-03T17:14:10.391 INFO:tasks.workunit.client.1.smithi115.stdout:0/346: stat d9/d15/d1a/d1d/f4b 0 2021-11-03T17:14:10.391 INFO:tasks.workunit.client.1.smithi115.stdout:0/347: fsync d9/db/f1b 0 2021-11-03T17:14:10.392 INFO:tasks.workunit.client.1.smithi115.stdout:9/166: mkdir d1/d2d/d27/d34 0 2021-11-03T17:14:10.392 INFO:tasks.workunit.client.1.smithi115.stdout:4/436: write d7/dc/d1a/d21/d47/d56/d63/f67 [1939259,57179] 0 2021-11-03T17:14:10.393 INFO:tasks.workunit.client.1.smithi115.stdout:0/348: rmdir d9/d15 39 2021-11-03T17:14:10.393 INFO:tasks.workunit.client.1.smithi115.stdout:8/272: write d4/d6/d3f/d43/f5e [14018,21890] 0 2021-11-03T17:14:10.393 INFO:tasks.workunit.client.0.smithi027.stdout:5/346: sync 2021-11-03T17:14:10.394 INFO:tasks.workunit.client.1.smithi115.stdout:9/167: unlink d1/d5/f6 0 2021-11-03T17:14:10.394 INFO:tasks.workunit.client.1.smithi115.stdout:0/349: mkdir d9/d15/d2c/d6e/d71 0 2021-11-03T17:14:10.395 INFO:tasks.workunit.client.1.smithi115.stdout:0/350: write d9/d15/f16 [259655,57982] 0 2021-11-03T17:14:10.396 INFO:tasks.workunit.client.0.smithi027.stdout:5/347: read d2/d4/d4e/d67/d20/f3a [95753,113959] 0 2021-11-03T17:14:10.396 INFO:tasks.workunit.client.0.smithi027.stdout:5/348: write d2/d2c/d3c/d58/f6e [599281,128589] 0 2021-11-03T17:14:10.397 INFO:tasks.workunit.client.1.smithi115.stdout:9/168: dread d1/d18/f1e [0,4194304] 0 2021-11-03T17:14:10.401 INFO:tasks.workunit.client.0.smithi027.stdout:5/349: unlink d2/d2c/l4b 0 2021-11-03T17:14:10.403 INFO:tasks.workunit.client.1.smithi115.stdout:2/241: dwrite d4/d6/d2e/f39 [0,4194304] 0 2021-11-03T17:14:10.405 INFO:tasks.workunit.client.1.smithi115.stdout:2/242: creat d4/d6/f50 x:0 0 0 2021-11-03T17:14:10.405 INFO:tasks.workunit.client.1.smithi115.stdout:5/377: dwrite f11 [0,4194304] 0 2021-11-03T17:14:10.406 INFO:tasks.workunit.client.0.smithi027.stdout:5/350: write d2/d2c/d3c/f41 [716939,101586] 0 2021-11-03T17:14:10.406 INFO:tasks.workunit.client.1.smithi115.stdout:3/305: dwrite d13/d51/f2d [0,4194304] 0 2021-11-03T17:14:10.407 INFO:tasks.workunit.client.1.smithi115.stdout:0/351: dread d9/db/df/f48 [0,4194304] 0 2021-11-03T17:14:10.407 INFO:tasks.workunit.client.1.smithi115.stdout:0/352: chown d9/d15/f53 49793 1 2021-11-03T17:14:10.408 INFO:tasks.workunit.client.1.smithi115.stdout:0/353: rmdir d9/d37 39 2021-11-03T17:14:10.408 INFO:tasks.workunit.client.1.smithi115.stdout:0/354: chown d9/db 125 1 2021-11-03T17:14:10.409 INFO:tasks.workunit.client.1.smithi115.stdout:7/213: dwrite d3/f14 [0,4194304] 0 2021-11-03T17:14:10.409 INFO:tasks.workunit.client.1.smithi115.stdout:7/214: write d3/f27 [1465272,79064] 0 2021-11-03T17:14:10.410 INFO:tasks.workunit.client.0.smithi027.stdout:5/351: rename d2/d2c/l51 to d2/d4/d4e/l77 0 2021-11-03T17:14:10.411 INFO:tasks.workunit.client.1.smithi115.stdout:0/355: creat d9/d15/d2c/d6e/f72 x:0 0 0 2021-11-03T17:14:10.411 INFO:tasks.workunit.client.1.smithi115.stdout:0/356: mknod d9/d37/c73 1 2021-11-03T17:14:10.411 INFO:tasks.workunit.client.1.smithi115.stdout:5/378: read d1a/d28/f31 [1142372,129401] 0 2021-11-03T17:14:10.412 INFO:tasks.workunit.client.1.smithi115.stdout:2/243: write d4/d6/f12 [877846,75384] 0 2021-11-03T17:14:10.412 INFO:tasks.workunit.client.1.smithi115.stdout:2/244: write d4/d6/d13/f36 [4534924,93467] 0 2021-11-03T17:14:10.412 INFO:tasks.workunit.client.1.smithi115.stdout:7/215: link d3/da/d26/d35/f3e d3/da/f42 0 2021-11-03T17:14:10.414 INFO:tasks.workunit.client.1.smithi115.stdout:0/357: rename d9/db/f64 to d9/db/df/f74 0 2021-11-03T17:14:10.416 INFO:tasks.workunit.client.0.smithi027.stdout:5/352: dread d2/d4/d4e/d67/d20/d25/f27 [0,4194304] 0 2021-11-03T17:14:10.419 INFO:tasks.workunit.client.0.smithi027.stdout:5/353: rmdir d2/d2c/d3c 39 2021-11-03T17:14:10.420 INFO:tasks.workunit.client.1.smithi115.stdout:2/245: symlink d4/d6/d2e/d34/d42/l51 0 2021-11-03T17:14:10.421 INFO:tasks.workunit.client.1.smithi115.stdout:5/379: write d1a/d3d/f62 [2262277,79289] 0 2021-11-03T17:14:10.421 INFO:tasks.workunit.client.0.smithi027.stdout:5/354: creat d2/d4/d4e/d67/f78 x:0 0 0 2021-11-03T17:14:10.424 INFO:tasks.workunit.client.1.smithi115.stdout:1/258: dwrite d7/dc/dd/d24/f2b [0,4194304] 0 2021-11-03T17:14:10.429 INFO:tasks.workunit.client.1.smithi115.stdout:0/358: creat d9/db/f75 x:0 0 0 2021-11-03T17:14:10.430 INFO:tasks.workunit.client.1.smithi115.stdout:8/273: dwrite d4/d6/f8 [0,4194304] 0 2021-11-03T17:14:10.430 INFO:tasks.workunit.client.1.smithi115.stdout:0/359: write d9/f35 [17370,47720] 0 2021-11-03T17:14:10.431 INFO:tasks.workunit.client.1.smithi115.stdout:0/360: write d9/db/f75 [108925,109804] 0 2021-11-03T17:14:10.439 INFO:tasks.workunit.client.1.smithi115.stdout:5/380: rename d1a/d28/d76 to d1a/d28/d7b/d4f/d81 0 2021-11-03T17:14:10.443 INFO:tasks.workunit.client.1.smithi115.stdout:0/361: symlink d9/d37/l76 0 2021-11-03T17:14:10.445 INFO:tasks.workunit.client.1.smithi115.stdout:9/169: dwrite d1/f13 [0,4194304] 0 2021-11-03T17:14:10.446 INFO:tasks.workunit.client.1.smithi115.stdout:5/381: symlink d1a/d7d/l82 0 2021-11-03T17:14:10.447 INFO:tasks.workunit.client.1.smithi115.stdout:3/306: dwrite d13/d51/d2e/f32 [0,4194304] 0 2021-11-03T17:14:10.448 INFO:tasks.workunit.client.0.smithi027.stdout:5/355: dwrite d2/d12/f44 [0,4194304] 0 2021-11-03T17:14:10.448 INFO:tasks.workunit.client.1.smithi115.stdout:2/246: dwrite d4/d6/d2e/d34/d43/d28/f41 [4194304,4194304] 0 2021-11-03T17:14:10.449 INFO:tasks.workunit.client.1.smithi115.stdout:7/216: dwrite d3/f1e [4194304,4194304] 0 2021-11-03T17:14:10.449 INFO:tasks.workunit.client.1.smithi115.stdout:7/217: readlink d3/da/d18/l2d 0 2021-11-03T17:14:10.450 INFO:tasks.workunit.client.1.smithi115.stdout:0/362: creat d9/d15/d1a/f77 x:0 0 0 2021-11-03T17:14:10.450 INFO:tasks.workunit.client.1.smithi115.stdout:0/363: chown d9/db/df/f48 1 1 2021-11-03T17:14:10.450 INFO:tasks.workunit.client.1.smithi115.stdout:0/364: chown d9/d15/d1a/l6f 6089 1 2021-11-03T17:14:10.451 INFO:tasks.workunit.client.0.smithi027.stdout:5/356: creat d2/d12/f79 x:0 0 0 2021-11-03T17:14:10.451 INFO:tasks.workunit.client.0.smithi027.stdout:5/357: mknod d2/d4/d4e/d67/c7a 1 2021-11-03T17:14:10.452 INFO:tasks.workunit.client.1.smithi115.stdout:5/382: mkdir d1a/d28/d4c/d75/d83 0 2021-11-03T17:14:10.452 INFO:tasks.workunit.client.1.smithi115.stdout:5/383: mknod d1a/d28/d4a/c84 1 2021-11-03T17:14:10.453 INFO:tasks.workunit.client.0.smithi027.stdout:5/358: unlink d2/d12/f32 0 2021-11-03T17:14:10.453 INFO:tasks.workunit.client.0.smithi027.stdout:5/359: chown d2/d4/d4e/d67/f30 213424 1 2021-11-03T17:14:10.453 INFO:tasks.workunit.client.0.smithi027.stdout:5/360: mknod d2/d1f/c7b 1 2021-11-03T17:14:10.454 INFO:tasks.workunit.client.1.smithi115.stdout:3/307: stat d13/l17 0 2021-11-03T17:14:10.455 INFO:tasks.workunit.client.1.smithi115.stdout:2/247: creat d4/d6/d2e/d34/f52 x:0 0 0 2021-11-03T17:14:10.456 INFO:tasks.workunit.client.1.smithi115.stdout:7/218: symlink d3/l43 0 2021-11-03T17:14:10.456 INFO:tasks.workunit.client.1.smithi115.stdout:7/219: chown d3 18927934 1 2021-11-03T17:14:10.457 INFO:tasks.workunit.client.1.smithi115.stdout:7/220: readlink d3/da/d26/l37 0 2021-11-03T17:14:10.457 INFO:tasks.workunit.client.1.smithi115.stdout:7/221: mknod d3/da/c44 1 2021-11-03T17:14:10.459 INFO:tasks.workunit.client.1.smithi115.stdout:0/365: rename d9/d15/d2c/d34 to d9/d15/d78 0 2021-11-03T17:14:10.459 INFO:tasks.workunit.client.1.smithi115.stdout:0/366: chown d9/d15/l56 7 1 2021-11-03T17:14:10.459 INFO:tasks.workunit.client.1.smithi115.stdout:0/367: fdatasync d9/db/f4a 0 2021-11-03T17:14:10.459 INFO:tasks.workunit.client.1.smithi115.stdout:0/368: fdatasync d9/d15/d2c/f5a 0 2021-11-03T17:14:10.460 INFO:tasks.workunit.client.1.smithi115.stdout:0/369: chown d9/db/f4a 18928 1 2021-11-03T17:14:10.460 INFO:tasks.workunit.client.0.smithi027.stdout:5/361: dread d2/d4/f34 [4194304,4194304] 0 2021-11-03T17:14:10.461 INFO:tasks.workunit.client.0.smithi027.stdout:5/362: fsync d2/d4/d4e/d67/f4a 0 2021-11-03T17:14:10.461 INFO:tasks.workunit.client.1.smithi115.stdout:5/384: mkdir d1a/d28/d7b/d4f/d81/d85 0 2021-11-03T17:14:10.461 INFO:tasks.workunit.client.1.smithi115.stdout:2/248: symlink d4/d44/l53 0 2021-11-03T17:14:10.462 INFO:tasks.workunit.client.1.smithi115.stdout:5/385: chown d1a/d3d/l46 215 1 2021-11-03T17:14:10.462 INFO:tasks.workunit.client.1.smithi115.stdout:2/249: chown d4/d6/d2e/f17 13 1 2021-11-03T17:14:10.463 INFO:tasks.workunit.client.0.smithi027.stdout:5/363: link d2/d4/f66 d2/d1f/f7c 0 2021-11-03T17:14:10.463 INFO:tasks.workunit.client.0.smithi027.stdout:5/364: write d2/f1e [1435239,70050] 0 2021-11-03T17:14:10.464 INFO:tasks.workunit.client.0.smithi027.stdout:5/365: readlink d2/d4/d4e/d67/d20/l5a 0 2021-11-03T17:14:10.464 INFO:tasks.workunit.client.0.smithi027.stdout:5/366: chown d2/d2c/f47 22937298 1 2021-11-03T17:14:10.464 INFO:tasks.workunit.client.0.smithi027.stdout:5/367: mkdir d2/d12/d7d 0 2021-11-03T17:14:10.465 INFO:tasks.workunit.client.0.smithi027.stdout:5/368: write d2/d12/f79 [527035,63507] 0 2021-11-03T17:14:10.465 INFO:tasks.workunit.client.1.smithi115.stdout:1/259: dwrite d7/dc/d1a/f3d [0,4194304] 0 2021-11-03T17:14:10.466 INFO:tasks.workunit.client.1.smithi115.stdout:2/250: creat d4/d6/f54 x:0 0 0 2021-11-03T17:14:10.468 INFO:tasks.workunit.client.1.smithi115.stdout:1/260: creat d7/dc/dd/d24/d26/f4f x:0 0 0 2021-11-03T17:14:10.468 INFO:tasks.workunit.client.1.smithi115.stdout:7/222: write d3/f9 [990431,10312] 0 2021-11-03T17:14:10.470 INFO:tasks.workunit.client.1.smithi115.stdout:7/223: dread d3/da/f1b [4194304,4194304] 0 2021-11-03T17:14:10.471 INFO:tasks.workunit.client.1.smithi115.stdout:9/170: write d1/d5/f17 [1164170,11502] 0 2021-11-03T17:14:10.483 INFO:tasks.workunit.client.1.smithi115.stdout:7/224: dread d3/f1e [0,4194304] 0 2021-11-03T17:14:10.491 INFO:tasks.workunit.client.1.smithi115.stdout:5/386: dwrite d1a/d28/d7b/f77 [0,4194304] 0 2021-11-03T17:14:10.491 INFO:tasks.workunit.client.1.smithi115.stdout:2/251: dwrite d4/f35 [0,4194304] 0 2021-11-03T17:14:10.492 INFO:tasks.workunit.client.1.smithi115.stdout:3/308: dwrite d13/d51/d2e/f59 [0,4194304] 0 2021-11-03T17:14:10.498 INFO:tasks.workunit.client.0.smithi027.stdout:5/369: dwrite d2/d2c/d3c/d48/d65/f56 [0,4194304] 0 2021-11-03T17:14:10.499 INFO:tasks.workunit.client.1.smithi115.stdout:7/225: dwrite d3/da/f42 [4194304,4194304] 0 2021-11-03T17:14:10.499 INFO:tasks.workunit.client.1.smithi115.stdout:7/226: stat d3/da/f15 0 2021-11-03T17:14:10.501 INFO:tasks.workunit.client.0.smithi027.stdout:5/370: write d2/f18 [4488633,86155] 0 2021-11-03T17:14:10.501 INFO:tasks.workunit.client.0.smithi027.stdout:5/371: mknod d2/d4/d61/c7e 1 2021-11-03T17:14:10.502 INFO:tasks.workunit.client.0.smithi027.stdout:5/372: creat d2/d4/d4e/d67/f7f x:0 0 0 2021-11-03T17:14:10.502 INFO:tasks.workunit.client.0.smithi027.stdout:5/373: fdatasync d2/d4/f43 0 2021-11-03T17:14:10.507 INFO:tasks.workunit.client.0.smithi027.stdout:5/374: unlink d2/d1f/f38 0 2021-11-03T17:14:10.507 INFO:tasks.workunit.client.0.smithi027.stdout:5/375: read d2/d12/f79 [84594,69358] 0 2021-11-03T17:14:10.508 INFO:tasks.workunit.client.0.smithi027.stdout:5/376: truncate d2/d4/d4e/d67/f2e 135264 0 2021-11-03T17:14:10.509 INFO:tasks.workunit.client.0.smithi027.stdout:5/377: getdents d2/d2c/d3c 0 2021-11-03T17:14:10.509 INFO:tasks.workunit.client.0.smithi027.stdout:5/378: write d2/d4/d4e/f4f [444716,13229] 0 2021-11-03T17:14:10.509 INFO:tasks.workunit.client.0.smithi027.stdout:5/379: dread - d2/d2c/d3c/f35 zero size 2021-11-03T17:14:10.509 INFO:tasks.workunit.client.0.smithi027.stdout:5/380: chown d2/f3 26148 1 2021-11-03T17:14:10.510 INFO:tasks.workunit.client.0.smithi027.stdout:5/381: mkdir d2/d2c/d3c/d48/d65/d80 0 2021-11-03T17:14:10.513 INFO:tasks.workunit.client.0.smithi027.stdout:5/382: dread d2/d4/d4e/d67/f30 [0,4194304] 0 2021-11-03T17:14:10.513 INFO:tasks.workunit.client.0.smithi027.stdout:5/383: truncate d2/f3 4288255 0 2021-11-03T17:14:10.515 INFO:tasks.workunit.client.0.smithi027.stdout:5/384: link d2/d2c/l69 d2/l81 0 2021-11-03T17:14:10.515 INFO:tasks.workunit.client.0.smithi027.stdout:5/385: chown d2/d2c/d3c/d48/d65/d80 197 1 2021-11-03T17:14:10.517 INFO:tasks.workunit.client.0.smithi027.stdout:5/386: rename d2/f36 to d2/d4/d4e/d67/f82 0 2021-11-03T17:14:10.517 INFO:tasks.workunit.client.0.smithi027.stdout:5/387: chown d2/d4/d4e/d67/d20/f73 45825649 1 2021-11-03T17:14:10.518 INFO:tasks.workunit.client.0.smithi027.stdout:5/388: creat d2/d2c/d3c/d48/d65/d80/f83 x:0 0 0 2021-11-03T17:14:10.519 INFO:tasks.workunit.client.0.smithi027.stdout:5/389: dread - d2/d4/d4e/d67/d20/d25/f71 zero size 2021-11-03T17:14:10.536 INFO:tasks.workunit.client.1.smithi115.stdout:5/387: creat d1a/d28/d7b/d48/d7f/f86 x:0 0 0 2021-11-03T17:14:10.537 INFO:tasks.workunit.client.1.smithi115.stdout:2/252: symlink d4/d6/d2e/d34/l55 0 2021-11-03T17:14:10.539 INFO:tasks.workunit.client.1.smithi115.stdout:3/309: truncate d13/d2c/f65 5196422 0 2021-11-03T17:14:10.541 INFO:tasks.workunit.client.0.smithi027.stdout:5/390: dwrite d2/f18 [0,4194304] 0 2021-11-03T17:14:10.542 INFO:tasks.workunit.client.1.smithi115.stdout:7/227: creat d3/da/d26/f45 x:0 0 0 2021-11-03T17:14:10.543 INFO:tasks.workunit.client.0.smithi027.stdout:5/391: creat d2/d2c/d3c/f84 x:0 0 0 2021-11-03T17:14:10.545 INFO:tasks.workunit.client.0.smithi027.stdout:5/392: truncate d2/d4/d4e/d67/f11 1653407 0 2021-11-03T17:14:10.545 INFO:tasks.workunit.client.0.smithi027.stdout:5/393: chown d2/d4/d4e/d67/f78 12236615 1 2021-11-03T17:14:10.546 INFO:tasks.workunit.client.0.smithi027.stdout:5/394: truncate d2/f42 978239 0 2021-11-03T17:14:10.547 INFO:tasks.workunit.client.0.smithi027.stdout:5/395: mknod d2/d4/d4e/d67/d20/d25/c85 1 2021-11-03T17:14:10.549 INFO:tasks.workunit.client.0.smithi027.stdout:5/396: unlink d2/d4/d4e/d67/d20/d25/f27 0 2021-11-03T17:14:10.550 INFO:tasks.workunit.client.0.smithi027.stdout:5/397: chown d2/d4/f13 7147247 1 2021-11-03T17:14:10.550 INFO:tasks.workunit.client.0.smithi027.stdout:5/398: dread - d2/d4/d4e/d67/f7f zero size 2021-11-03T17:14:10.550 INFO:tasks.workunit.client.0.smithi027.stdout:5/399: fsync d2/f18 0 2021-11-03T17:14:10.551 INFO:tasks.workunit.client.0.smithi027.stdout:5/400: mknod d2/d2c/d3c/d58/c86 1 2021-11-03T17:14:10.551 INFO:tasks.workunit.client.0.smithi027.stdout:5/401: fsync d2/d4/d4e/d67/d20/f73 0 2021-11-03T17:14:10.551 INFO:tasks.workunit.client.0.smithi027.stdout:5/402: write d2/d4/f66 [356214,122431] 0 2021-11-03T17:14:10.552 INFO:tasks.workunit.client.1.smithi115.stdout:1/261: symlink d7/dc/d1a/l50 0 2021-11-03T17:14:10.555 INFO:tasks.workunit.client.1.smithi115.stdout:2/253: dwrite d4/d6/d13/f2f [0,4194304] 0 2021-11-03T17:14:10.556 INFO:tasks.workunit.client.1.smithi115.stdout:5/388: rename d1a/d28/d34/l37 to d1a/d28/d4c/d63/l87 0 2021-11-03T17:14:10.558 INFO:tasks.workunit.client.1.smithi115.stdout:3/310: rmdir d13/d2a/d53/d68 0 2021-11-03T17:14:10.562 INFO:tasks.workunit.client.1.smithi115.stdout:7/228: unlink d3/da/f36 0 2021-11-03T17:14:10.564 INFO:tasks.workunit.client.1.smithi115.stdout:2/254: mkdir d4/d6/d56 0 2021-11-03T17:14:10.566 INFO:tasks.workunit.client.1.smithi115.stdout:3/311: stat d13/f56 0 2021-11-03T17:14:10.566 INFO:tasks.workunit.client.1.smithi115.stdout:3/312: stat d13/d2c 0 2021-11-03T17:14:10.567 INFO:tasks.workunit.client.1.smithi115.stdout:7/229: getdents d3/da/d18 0 2021-11-03T17:14:10.571 INFO:tasks.workunit.client.1.smithi115.stdout:7/230: rename d3/da/d26/l37 to d3/da/l46 0 2021-11-03T17:14:10.572 INFO:tasks.workunit.client.1.smithi115.stdout:3/313: creat d13/d35/f69 x:0 0 0 2021-11-03T17:14:10.573 INFO:tasks.workunit.client.1.smithi115.stdout:7/231: rename d3/da/f15 to d3/f47 0 2021-11-03T17:14:10.574 INFO:tasks.workunit.client.1.smithi115.stdout:7/232: mknod d3/c48 1 2021-11-03T17:14:10.574 INFO:tasks.workunit.client.1.smithi115.stdout:3/314: symlink d13/d35/l6a 0 2021-11-03T17:14:10.575 INFO:tasks.workunit.client.1.smithi115.stdout:3/315: link d13/d35/d54/l5e d13/d51/d40/l6b 0 2021-11-03T17:14:10.576 INFO:tasks.workunit.client.1.smithi115.stdout:7/233: write d3/f1e [7637204,129561] 0 2021-11-03T17:14:10.576 INFO:tasks.workunit.client.1.smithi115.stdout:7/234: truncate d3/da/d26/f45 425077 0 2021-11-03T17:14:10.578 INFO:tasks.workunit.client.1.smithi115.stdout:3/316: link d13/d3b/f58 d13/d67/f6c 0 2021-11-03T17:14:10.580 INFO:tasks.workunit.client.1.smithi115.stdout:3/317: symlink d13/d50/l6d 0 2021-11-03T17:14:10.586 INFO:tasks.workunit.client.1.smithi115.stdout:1/262: dwrite d7/dc/dd/d24/d4b/f3a [0,4194304] 0 2021-11-03T17:14:10.587 INFO:tasks.workunit.client.1.smithi115.stdout:1/263: write d7/dc/d2a/f33 [471917,90557] 0 2021-11-03T17:14:10.587 INFO:tasks.workunit.client.1.smithi115.stdout:1/264: chown d7/dc/dd/f46 1 1 2021-11-03T17:14:10.587 INFO:tasks.workunit.client.1.smithi115.stdout:1/265: stat d7/dc/dd/d24/l37 0 2021-11-03T17:14:10.588 INFO:tasks.workunit.client.1.smithi115.stdout:1/266: fdatasync d7/dc/f40 0 2021-11-03T17:14:10.590 INFO:tasks.workunit.client.1.smithi115.stdout:1/267: rmdir d7 39 2021-11-03T17:14:10.590 INFO:tasks.workunit.client.1.smithi115.stdout:1/268: mknod d7/dc/dd/d24/c51 1 2021-11-03T17:14:10.592 INFO:tasks.workunit.client.1.smithi115.stdout:1/269: creat d7/dc/d2a/f52 x:0 0 0 2021-11-03T17:14:10.592 INFO:tasks.workunit.client.1.smithi115.stdout:3/318: dread d13/d51/d2e/f32 [0,4194304] 0 2021-11-03T17:14:10.593 INFO:tasks.workunit.client.1.smithi115.stdout:1/270: write d7/dc/dd/d42/f47 [325434,87462] 0 2021-11-03T17:14:10.593 INFO:tasks.workunit.client.1.smithi115.stdout:1/271: getdents d7/dc/dd/d24/d4d 0 2021-11-03T17:14:10.594 INFO:tasks.workunit.client.1.smithi115.stdout:5/389: dwrite d1a/d28/d34/f2a [0,4194304] 0 2021-11-03T17:14:10.594 INFO:tasks.workunit.client.1.smithi115.stdout:5/390: fdatasync d1a/d28/d7b/f77 0 2021-11-03T17:14:10.594 INFO:tasks.workunit.client.1.smithi115.stdout:5/391: write d1a/d28/d7b/f74 [226502,77556] 0 2021-11-03T17:14:10.594 INFO:tasks.workunit.client.1.smithi115.stdout:3/319: symlink d13/d51/d40/l6e 0 2021-11-03T17:14:10.595 INFO:tasks.workunit.client.1.smithi115.stdout:5/392: creat d1a/d28/d4c/d75/d83/f88 x:0 0 0 2021-11-03T17:14:10.596 INFO:tasks.workunit.client.1.smithi115.stdout:5/393: fdatasync fd 0 2021-11-03T17:14:10.599 INFO:tasks.workunit.client.1.smithi115.stdout:2/255: dwrite d4/d6/d2e/f3c [0,4194304] 0 2021-11-03T17:14:10.599 INFO:tasks.workunit.client.1.smithi115.stdout:2/256: dread - d4/d6/f50 zero size 2021-11-03T17:14:10.599 INFO:tasks.workunit.client.1.smithi115.stdout:3/320: read d13/f19 [232105,29129] 0 2021-11-03T17:14:10.600 INFO:tasks.workunit.client.0.smithi027.stdout:5/403: dwrite d2/d4/d4e/d67/f4a [0,4194304] 0 2021-11-03T17:14:10.600 INFO:tasks.workunit.client.1.smithi115.stdout:7/235: dwrite d3/f25 [0,4194304] 0 2021-11-03T17:14:10.602 INFO:tasks.workunit.client.1.smithi115.stdout:5/394: dread d1a/d28/d4c/f6e [0,4194304] 0 2021-11-03T17:14:10.610 INFO:tasks.workunit.client.1.smithi115.stdout:1/272: dwrite d7/dc/f15 [0,4194304] 0 2021-11-03T17:14:10.610 INFO:tasks.workunit.client.1.smithi115.stdout:1/273: truncate d7/dc/d1a/f3d 4284479 0 2021-11-03T17:14:10.610 INFO:tasks.workunit.client.1.smithi115.stdout:1/274: chown d7/dc/dd/d24/d26/l41 239297171 1 2021-11-03T17:14:10.611 INFO:tasks.workunit.client.0.smithi027.stdout:5/404: dread d2/d4/d4e/d67/f82 [0,4194304] 0 2021-11-03T17:14:10.612 INFO:tasks.workunit.client.0.smithi027.stdout:5/405: rmdir d2/d2c/d3c/d58 39 2021-11-03T17:14:10.613 INFO:tasks.workunit.client.0.smithi027.stdout:5/406: symlink d2/d2c/d3c/d48/l87 0 2021-11-03T17:14:10.614 INFO:tasks.workunit.client.1.smithi115.stdout:1/275: write d7/dc/d1a/f3d [768594,83810] 0 2021-11-03T17:14:10.614 INFO:tasks.workunit.client.1.smithi115.stdout:3/321: dwrite d13/d50/f4d [0,4194304] 0 2021-11-03T17:14:10.616 INFO:tasks.workunit.client.0.smithi027.stdout:5/407: unlink d2/d2c/d3c/l5d 0 2021-11-03T17:14:10.618 INFO:tasks.workunit.client.0.smithi027.stdout:5/408: dread d2/d4/d4e/d67/f2e [0,4194304] 0 2021-11-03T17:14:10.618 INFO:tasks.workunit.client.0.smithi027.stdout:5/409: write d2/d12/f5c [438722,33373] 0 2021-11-03T17:14:10.618 INFO:tasks.workunit.client.0.smithi027.stdout:5/410: mknod d2/d2c/d3c/d48/d65/c88 1 2021-11-03T17:14:10.619 INFO:tasks.workunit.client.0.smithi027.stdout:5/411: creat d2/d4/d61/f89 x:0 0 0 2021-11-03T17:14:10.620 INFO:tasks.workunit.client.0.smithi027.stdout:5/412: fsync d2/d4/d4e/d67/f11 0 2021-11-03T17:14:10.621 INFO:tasks.workunit.client.0.smithi027.stdout:5/413: symlink d2/d2c/l8a 0 2021-11-03T17:14:10.621 INFO:tasks.workunit.client.0.smithi027.stdout:5/414: chown d2/d2c/d3c/d48/l53 24498 1 2021-11-03T17:14:10.629 INFO:tasks.workunit.client.0.smithi027.stdout:9/353: sync 2021-11-03T17:14:10.629 INFO:tasks.workunit.client.0.smithi027.stdout:4/285: sync 2021-11-03T17:14:10.630 INFO:tasks.workunit.client.0.smithi027.stdout:8/361: sync 2021-11-03T17:14:10.630 INFO:tasks.workunit.client.0.smithi027.stdout:8/362: rename d7 to d7/d53/d70 22 2021-11-03T17:14:10.630 INFO:tasks.workunit.client.0.smithi027.stdout:4/286: mknod d6/db/d21/d2b/c5d 1 2021-11-03T17:14:10.630 INFO:tasks.workunit.client.0.smithi027.stdout:8/363: fdatasync d7/d10/f15 0 2021-11-03T17:14:10.633 INFO:tasks.workunit.client.0.smithi027.stdout:5/415: symlink d2/d2c/l8b 0 2021-11-03T17:14:10.633 INFO:tasks.workunit.client.0.smithi027.stdout:5/416: mknod d2/d2c/d3c/c8c 1 2021-11-03T17:14:10.633 INFO:tasks.workunit.client.0.smithi027.stdout:5/417: fsync d2/d4/fe 0 2021-11-03T17:14:10.633 INFO:tasks.workunit.client.0.smithi027.stdout:5/418: mknod d2/d2c/d3c/d48/d65/c8d 1 2021-11-03T17:14:10.634 INFO:tasks.workunit.client.0.smithi027.stdout:5/419: chown d2/f7 73 1 2021-11-03T17:14:10.634 INFO:tasks.workunit.client.0.smithi027.stdout:5/420: write d2/d4/d4e/d67/d20/f3a [729919,103020] 0 2021-11-03T17:14:10.635 INFO:tasks.workunit.client.0.smithi027.stdout:9/354: truncate db/d1b/d1c/d1d/f4f 2021783 0 2021-11-03T17:14:10.636 INFO:tasks.workunit.client.1.smithi115.stdout:2/257: mkdir d4/d6/d2e/d34/d42/d57 0 2021-11-03T17:14:10.636 INFO:tasks.workunit.client.1.smithi115.stdout:2/258: dread - d4/d6/d13/f21 zero size 2021-11-03T17:14:10.636 INFO:tasks.workunit.client.1.smithi115.stdout:2/259: truncate d4/d6/d2e/f46 700966 0 2021-11-03T17:14:10.636 INFO:tasks.workunit.client.1.smithi115.stdout:2/260: fsync d4/d6/d13/f4e 0 2021-11-03T17:14:10.637 INFO:tasks.workunit.client.1.smithi115.stdout:1/276: dwrite d7/dc/f16 [0,4194304] 0 2021-11-03T17:14:10.637 INFO:tasks.workunit.client.1.smithi115.stdout:7/236: getdents d3 0 2021-11-03T17:14:10.638 INFO:tasks.workunit.client.0.smithi027.stdout:8/364: creat d7/d10/f71 x:0 0 0 2021-11-03T17:14:10.638 INFO:tasks.workunit.client.0.smithi027.stdout:8/365: readlink d7/d10/d21/d41/l42 0 2021-11-03T17:14:10.653 INFO:tasks.workunit.client.0.smithi027.stdout:9/355: rename db/d1b/d2b/d5e/f6b to db/d1b/d2b/d41/f82 0 2021-11-03T17:14:10.654 INFO:tasks.workunit.client.0.smithi027.stdout:9/356: write db/d1b/d1c/d1d/d40/f70 [727461,7783] 0 2021-11-03T17:14:10.654 INFO:tasks.workunit.client.0.smithi027.stdout:9/357: mknod db/d1b/d2b/d41/d6c/c83 1 2021-11-03T17:14:10.654 INFO:tasks.workunit.client.0.smithi027.stdout:9/358: chown db/d1b/d2b/d41/f82 1 1 2021-11-03T17:14:10.655 INFO:tasks.workunit.client.0.smithi027.stdout:8/366: truncate d7/d10/d18/f33 363367 0 2021-11-03T17:14:10.656 INFO:tasks.workunit.client.0.smithi027.stdout:8/367: truncate d7/d10/d21/f45 341541 0 2021-11-03T17:14:10.662 INFO:tasks.workunit.client.0.smithi027.stdout:5/421: dwrite d2/d2c/d3c/f41 [0,4194304] 0 2021-11-03T17:14:10.663 INFO:tasks.workunit.client.0.smithi027.stdout:5/422: stat d2/d4/f34 0 2021-11-03T17:14:10.663 INFO:tasks.workunit.client.0.smithi027.stdout:5/423: fdatasync d2/d2c/d3c/f41 0 2021-11-03T17:14:10.663 INFO:tasks.workunit.client.1.smithi115.stdout:7/237: dwrite d3/da/d26/d35/f3d [0,4194304] 0 2021-11-03T17:14:10.669 INFO:tasks.workunit.client.0.smithi027.stdout:9/359: dread db/d1b/d23/f73 [0,4194304] 0 2021-11-03T17:14:10.669 INFO:tasks.workunit.client.0.smithi027.stdout:9/360: mknod db/d1b/d2b/d41/c84 1 2021-11-03T17:14:10.674 INFO:tasks.workunit.client.0.smithi027.stdout:5/424: mkdir d2/d4/d4e/d67/d8e 0 2021-11-03T17:14:10.676 INFO:tasks.workunit.client.0.smithi027.stdout:5/425: dread d2/f3e [0,4194304] 0 2021-11-03T17:14:10.682 INFO:tasks.workunit.client.0.smithi027.stdout:9/361: dread db/d1b/d1c/f65 [0,4194304] 0 2021-11-03T17:14:10.682 INFO:tasks.workunit.client.0.smithi027.stdout:9/362: dread - db/d1b/d1c/d1d/f6d zero size 2021-11-03T17:14:10.683 INFO:tasks.workunit.client.1.smithi115.stdout:3/322: creat d13/d35/d54/f6f x:0 0 0 2021-11-03T17:14:10.684 INFO:tasks.workunit.client.0.smithi027.stdout:9/363: symlink db/d1b/d1c/d1d/l85 0 2021-11-03T17:14:10.685 INFO:tasks.workunit.client.0.smithi027.stdout:9/364: creat db/d1b/d2b/f86 x:0 0 0 2021-11-03T17:14:10.686 INFO:tasks.workunit.client.1.smithi115.stdout:2/261: creat d4/d6/d13/f58 x:0 0 0 2021-11-03T17:14:10.687 INFO:tasks.workunit.client.1.smithi115.stdout:2/262: fdatasync d4/d6/d13/f38 0 2021-11-03T17:14:10.693 INFO:tasks.workunit.client.1.smithi115.stdout:1/277: rmdir d7/dc/dd/d24/d4b 39 2021-11-03T17:14:10.695 INFO:tasks.workunit.client.1.smithi115.stdout:5/395: rename d1a/d28/d7b/d4f/d57 to d1a/d3d/d89 0 2021-11-03T17:14:10.695 INFO:tasks.workunit.client.1.smithi115.stdout:7/238: rename d3 to d3/d49 22 2021-11-03T17:14:10.697 INFO:tasks.workunit.client.1.smithi115.stdout:3/323: rmdir d13/d35/d54 39 2021-11-03T17:14:10.699 INFO:tasks.workunit.client.1.smithi115.stdout:2/263: symlink d4/d6/d2e/d34/d42/d57/l59 0 2021-11-03T17:14:10.699 INFO:tasks.workunit.client.1.smithi115.stdout:5/396: symlink d1a/d28/d4c/d63/l8a 0 2021-11-03T17:14:10.700 INFO:tasks.workunit.client.1.smithi115.stdout:5/397: read d1a/d3d/d89/f67 [250313,19238] 0 2021-11-03T17:14:10.700 INFO:tasks.workunit.client.1.smithi115.stdout:5/398: readlink d1a/d28/l43 0 2021-11-03T17:14:10.701 INFO:tasks.workunit.client.1.smithi115.stdout:7/239: rename d3/da/l46 to d3/da/d26/d35/l4a 0 2021-11-03T17:14:10.701 INFO:tasks.workunit.client.1.smithi115.stdout:7/240: readlink d3/da/d26/d33/l3a 0 2021-11-03T17:14:10.701 INFO:tasks.workunit.client.1.smithi115.stdout:3/324: getdents d13/d51/d40 0 2021-11-03T17:14:10.702 INFO:tasks.workunit.client.1.smithi115.stdout:2/264: creat d4/f5a x:0 0 0 2021-11-03T17:14:10.704 INFO:tasks.workunit.client.1.smithi115.stdout:5/399: creat d1a/d28/d4a/f8b x:0 0 0 2021-11-03T17:14:10.704 INFO:tasks.workunit.client.1.smithi115.stdout:7/241: write d3/da/f1b [1084817,107673] 0 2021-11-03T17:14:10.705 INFO:tasks.workunit.client.1.smithi115.stdout:7/242: truncate d3/da/d26/d35/f3f 95829 0 2021-11-03T17:14:10.705 INFO:tasks.workunit.client.1.smithi115.stdout:1/278: rename d7/dc/d2a/f52 to d7/f53 0 2021-11-03T17:14:10.706 INFO:tasks.workunit.client.1.smithi115.stdout:3/325: mkdir d13/d2a/d53/d70 0 2021-11-03T17:14:10.706 INFO:tasks.workunit.client.1.smithi115.stdout:3/326: readlink l7 0 2021-11-03T17:14:10.707 INFO:tasks.workunit.client.0.smithi027.stdout:5/426: dwrite d2/d4/f34 [0,4194304] 0 2021-11-03T17:14:10.708 INFO:tasks.workunit.client.0.smithi027.stdout:5/427: stat d2/d4/d4e/d67/f82 0 2021-11-03T17:14:10.708 INFO:tasks.workunit.client.1.smithi115.stdout:7/243: write d3/da/d26/d35/f3e [796700,51086] 0 2021-11-03T17:14:10.708 INFO:tasks.workunit.client.1.smithi115.stdout:2/265: getdents d4/d6/d2e/d34/d42 0 2021-11-03T17:14:10.709 INFO:tasks.workunit.client.1.smithi115.stdout:2/266: chown d4/d6/d2e/d34/d43/d28/f3b 752351 1 2021-11-03T17:14:10.709 INFO:tasks.workunit.client.1.smithi115.stdout:5/400: symlink d1a/d28/l8c 0 2021-11-03T17:14:10.709 INFO:tasks.workunit.client.1.smithi115.stdout:1/279: readlink d7/dc/d1a/l21 0 2021-11-03T17:14:10.710 INFO:tasks.workunit.client.0.smithi027.stdout:9/365: dwrite db/d1b/d2b/f2a [0,4194304] 0 2021-11-03T17:14:10.711 INFO:tasks.workunit.client.0.smithi027.stdout:5/428: unlink d2/l81 0 2021-11-03T17:14:10.711 INFO:tasks.workunit.client.0.smithi027.stdout:5/429: stat d2/d1f/f7c 0 2021-11-03T17:14:10.711 INFO:tasks.workunit.client.0.smithi027.stdout:9/366: write db/d1b/d2b/d41/f74 [322745,33200] 0 2021-11-03T17:14:10.711 INFO:tasks.workunit.client.0.smithi027.stdout:9/367: chown db/l1a 416177 1 2021-11-03T17:14:10.712 INFO:tasks.workunit.client.1.smithi115.stdout:7/244: symlink d3/l4b 0 2021-11-03T17:14:10.713 INFO:tasks.workunit.client.1.smithi115.stdout:2/267: mknod d4/d6/d2e/c5b 1 2021-11-03T17:14:10.713 INFO:tasks.workunit.client.0.smithi027.stdout:5/430: rename d2/d4/d4e/d67/d20/d25/d6a/f75 to d2/d2c/d3c/d48/d65/f8f 0 2021-11-03T17:14:10.713 INFO:tasks.workunit.client.0.smithi027.stdout:9/368: fsync db/d1b/d2b/f86 0 2021-11-03T17:14:10.714 INFO:tasks.workunit.client.1.smithi115.stdout:5/401: unlink d1a/d7d/l82 0 2021-11-03T17:14:10.715 INFO:tasks.workunit.client.0.smithi027.stdout:9/369: rmdir db/d1b/d2b/d41/d6c 39 2021-11-03T17:14:10.715 INFO:tasks.workunit.client.0.smithi027.stdout:9/370: write db/d1b/d23/d2c/f6a [460145,39745] 0 2021-11-03T17:14:10.715 INFO:tasks.workunit.client.0.smithi027.stdout:9/371: write db/d1b/d23/d2c/f4c [1671158,45792] 0 2021-11-03T17:14:10.716 INFO:tasks.workunit.client.1.smithi115.stdout:1/280: mkdir d7/dc/dd/d24/d4b/d49/d54 0 2021-11-03T17:14:10.716 INFO:tasks.workunit.client.1.smithi115.stdout:7/245: mkdir d3/d4c 0 2021-11-03T17:14:10.718 INFO:tasks.workunit.client.1.smithi115.stdout:1/281: unlink d7/dc/d2a/f31 0 2021-11-03T17:14:10.718 INFO:tasks.workunit.client.1.smithi115.stdout:1/282: chown d7/dc/d1a/f4a 12902814 1 2021-11-03T17:14:10.718 INFO:tasks.workunit.client.1.smithi115.stdout:2/268: symlink d4/d6/d2e/d34/d43/d28/l5c 0 2021-11-03T17:14:10.721 INFO:tasks.workunit.client.1.smithi115.stdout:2/269: write d4/f3a [2037525,82795] 0 2021-11-03T17:14:10.722 INFO:tasks.workunit.client.1.smithi115.stdout:7/246: write d3/f27 [187633,53437] 0 2021-11-03T17:14:10.722 INFO:tasks.workunit.client.1.smithi115.stdout:1/283: write d7/dc/d1a/f3d [3662137,107634] 0 2021-11-03T17:14:10.725 INFO:tasks.workunit.client.1.smithi115.stdout:2/270: write d4/d6/d13/f25 [225180,52117] 0 2021-11-03T17:14:10.726 INFO:tasks.workunit.client.1.smithi115.stdout:2/271: write d4/d6/f12 [3233269,54609] 0 2021-11-03T17:14:10.726 INFO:tasks.workunit.client.1.smithi115.stdout:2/272: write d4/d6/d2e/f46 [1496589,93776] 0 2021-11-03T17:14:10.727 INFO:tasks.workunit.client.1.smithi115.stdout:2/273: rename d4/fa to d4/d6/d56/f5d 0 2021-11-03T17:14:10.727 INFO:tasks.workunit.client.1.smithi115.stdout:2/274: chown d4/d6/d13/f21 365 1 2021-11-03T17:14:10.728 INFO:tasks.workunit.client.1.smithi115.stdout:2/275: symlink d4/d6/d2e/l5e 0 2021-11-03T17:14:10.732 INFO:tasks.workunit.client.1.smithi115.stdout:3/327: dwrite d13/d51/d40/f4f [0,4194304] 0 2021-11-03T17:14:10.732 INFO:tasks.workunit.client.1.smithi115.stdout:3/328: truncate d13/d3b/f58 1007901 0 2021-11-03T17:14:10.735 INFO:tasks.workunit.client.1.smithi115.stdout:3/329: creat d13/d2a/f71 x:0 0 0 2021-11-03T17:14:10.735 INFO:tasks.workunit.client.1.smithi115.stdout:7/247: dread d3/da/d18/f22 [0,4194304] 0 2021-11-03T17:14:10.736 INFO:tasks.workunit.client.1.smithi115.stdout:7/248: creat d3/d4c/f4d x:0 0 0 2021-11-03T17:14:10.737 INFO:tasks.workunit.client.1.smithi115.stdout:7/249: chown d3/da/d26/d35/f3d 6255 1 2021-11-03T17:14:10.739 INFO:tasks.workunit.client.1.smithi115.stdout:7/250: rename d3/l43 to d3/da/d26/l4e 0 2021-11-03T17:14:10.739 INFO:tasks.workunit.client.1.smithi115.stdout:3/330: write f10 [694796,53231] 0 2021-11-03T17:14:10.742 INFO:tasks.workunit.client.1.smithi115.stdout:7/251: getdents d3/da 0 2021-11-03T17:14:10.742 INFO:tasks.workunit.client.1.smithi115.stdout:7/252: write d3/da/f42 [8620279,86244] 0 2021-11-03T17:14:10.742 INFO:tasks.workunit.client.1.smithi115.stdout:5/402: dwrite d1a/d3d/f64 [0,4194304] 0 2021-11-03T17:14:10.743 INFO:tasks.workunit.client.0.smithi027.stdout:5/431: dwrite d2/d2c/d3c/d58/f6e [0,4194304] 0 2021-11-03T17:14:10.743 INFO:tasks.workunit.client.0.smithi027.stdout:5/432: mknod d2/d2c/d3c/c90 1 2021-11-03T17:14:10.745 INFO:tasks.workunit.client.0.smithi027.stdout:5/433: truncate d2/f52 22392 0 2021-11-03T17:14:10.746 INFO:tasks.workunit.client.0.smithi027.stdout:5/434: dread d2/d4/d4e/d67/f2e [0,4194304] 0 2021-11-03T17:14:10.746 INFO:tasks.workunit.client.0.smithi027.stdout:5/435: mknod d2/d2c/d3c/d48/c91 1 2021-11-03T17:14:10.749 INFO:tasks.workunit.client.1.smithi115.stdout:3/331: mkdir d13/d2a/d53/d72 0 2021-11-03T17:14:10.750 INFO:tasks.workunit.client.0.smithi027.stdout:5/436: dread d2/d4/d4e/f4f [0,4194304] 0 2021-11-03T17:14:10.751 INFO:tasks.workunit.client.1.smithi115.stdout:5/403: dread d1a/d28/d7b/f77 [0,4194304] 0 2021-11-03T17:14:10.751 INFO:tasks.workunit.client.1.smithi115.stdout:5/404: fdatasync d1a/d28/d4a/d65/f6b 0 2021-11-03T17:14:10.751 INFO:tasks.workunit.client.0.smithi027.stdout:5/437: symlink d2/d4/d4e/d67/d20/l92 0 2021-11-03T17:14:10.753 INFO:tasks.workunit.client.0.smithi027.stdout:5/438: mkdir d2/d12/d7d/d93 0 2021-11-03T17:14:10.754 INFO:tasks.workunit.client.1.smithi115.stdout:1/284: dwrite d7/dc/dd/d42/f47 [0,4194304] 0 2021-11-03T17:14:10.755 INFO:tasks.workunit.client.1.smithi115.stdout:1/285: fdatasync d7/f9 0 2021-11-03T17:14:10.756 INFO:tasks.workunit.client.0.smithi027.stdout:5/439: write d2/d2c/d3c/d58/f6e [3171671,12517] 0 2021-11-03T17:14:10.757 INFO:tasks.workunit.client.0.smithi027.stdout:5/440: write d2/d4/d4e/f4f [940312,4670] 0 2021-11-03T17:14:10.758 INFO:tasks.workunit.client.1.smithi115.stdout:7/253: getdents d3 0 2021-11-03T17:14:10.758 INFO:tasks.workunit.client.1.smithi115.stdout:7/254: readlink d3/da/d18/l28 0 2021-11-03T17:14:10.763 INFO:tasks.workunit.client.0.smithi027.stdout:5/441: rename d2/d12/d7d/d93 to d2/d2c/d3c/d94 0 2021-11-03T17:14:10.763 INFO:tasks.workunit.client.1.smithi115.stdout:1/286: rename d7/dc/dd/d24/d4b/d49/d54 to d7/dc/dd/d24/d4b/d55 0 2021-11-03T17:14:10.765 INFO:tasks.workunit.client.0.smithi027.stdout:5/442: symlink d2/d12/d7d/l95 0 2021-11-03T17:14:10.765 INFO:tasks.workunit.client.0.smithi027.stdout:5/443: fdatasync d2/d4/d4e/d67/d20/d25/f71 0 2021-11-03T17:14:10.768 INFO:tasks.workunit.client.0.smithi027.stdout:5/444: creat d2/d2c/d3c/d94/f96 x:0 0 0 2021-11-03T17:14:10.771 INFO:tasks.workunit.client.0.smithi027.stdout:5/445: dread d2/d1f/f7c [0,4194304] 0 2021-11-03T17:14:10.772 INFO:tasks.workunit.client.0.smithi027.stdout:5/446: read d2/d2c/d3c/d58/f74 [280542,49523] 0 2021-11-03T17:14:10.773 INFO:tasks.workunit.client.0.smithi027.stdout:5/447: symlink d2/d2c/d3c/d48/d65/l97 0 2021-11-03T17:14:10.774 INFO:tasks.workunit.client.1.smithi115.stdout:1/287: truncate d7/dc/dd/d42/f47 787415 0 2021-11-03T17:14:10.775 INFO:tasks.workunit.client.0.smithi027.stdout:5/448: rename d2/d4/fe to d2/d2c/d3c/f98 0 2021-11-03T17:14:10.776 INFO:tasks.workunit.client.1.smithi115.stdout:1/288: symlink d7/dc/dd/d3e/l56 0 2021-11-03T17:14:10.776 INFO:tasks.workunit.client.1.smithi115.stdout:1/289: mknod d7/dc/dd/d42/c57 1 2021-11-03T17:14:10.777 INFO:tasks.workunit.client.1.smithi115.stdout:1/290: rmdir d7/dc/dd/d24/d26 39 2021-11-03T17:14:10.778 INFO:tasks.workunit.client.1.smithi115.stdout:5/405: dwrite d1a/d3d/f42 [0,4194304] 0 2021-11-03T17:14:10.779 INFO:tasks.workunit.client.1.smithi115.stdout:1/291: mkdir d7/dc/dd/d3e/d58 0 2021-11-03T17:14:10.779 INFO:tasks.workunit.client.1.smithi115.stdout:1/292: fdatasync d7/dc/f2f 0 2021-11-03T17:14:10.780 INFO:tasks.workunit.client.1.smithi115.stdout:5/406: write d1a/d28/d4a/f56 [2739885,22780] 0 2021-11-03T17:14:10.781 INFO:tasks.workunit.client.1.smithi115.stdout:7/255: dwrite d3/da/d18/f40 [0,4194304] 0 2021-11-03T17:14:10.781 INFO:tasks.workunit.client.1.smithi115.stdout:7/256: mknod d3/da/d18/c4f 1 2021-11-03T17:14:10.781 INFO:tasks.workunit.client.1.smithi115.stdout:7/257: fdatasync d3/f1e 0 2021-11-03T17:14:10.781 INFO:tasks.workunit.client.1.smithi115.stdout:7/258: mknod d3/da/d26/d33/c50 1 2021-11-03T17:14:10.782 INFO:tasks.workunit.client.1.smithi115.stdout:1/293: write d7/dc/dd/d42/f44 [1698090,97313] 0 2021-11-03T17:14:10.785 INFO:tasks.workunit.client.1.smithi115.stdout:1/294: rename d7/dc/dd/d24/d4b/d49 to d7/d38/d59 0 2021-11-03T17:14:10.785 INFO:tasks.workunit.client.1.smithi115.stdout:1/295: chown d7/dc/d1a/l21 113 1 2021-11-03T17:14:10.785 INFO:tasks.workunit.client.1.smithi115.stdout:1/296: write d7/dc/d1a/f3d [4551659,119841] 0 2021-11-03T17:14:10.787 INFO:tasks.workunit.client.0.smithi027.stdout:5/449: rename d2/d2c/d3c/d94 to d2/d4/d4e/d67/d8e/d99 0 2021-11-03T17:14:10.789 INFO:tasks.workunit.client.1.smithi115.stdout:7/259: dread d3/da/d26/d35/f3d [0,4194304] 0 2021-11-03T17:14:10.789 INFO:tasks.workunit.client.0.smithi027.stdout:5/450: unlink d2/d2c/d3c/d48/d65/l63 0 2021-11-03T17:14:10.790 INFO:tasks.workunit.client.0.smithi027.stdout:5/451: truncate d2/d2c/d3c/f84 953429 0 2021-11-03T17:14:10.790 INFO:tasks.workunit.client.0.smithi027.stdout:5/452: dread d2/f52 [0,4194304] 0 2021-11-03T17:14:10.790 INFO:tasks.workunit.client.0.smithi027.stdout:5/453: fdatasync d2/d12/f5c 0 2021-11-03T17:14:10.791 INFO:tasks.workunit.client.1.smithi115.stdout:1/297: write d7/dc/f16 [4092898,93733] 0 2021-11-03T17:14:10.791 INFO:tasks.workunit.client.1.smithi115.stdout:1/298: write d7/dc/ff [1511653,23331] 0 2021-11-03T17:14:10.792 INFO:tasks.workunit.client.1.smithi115.stdout:1/299: stat d7/dc/f15 0 2021-11-03T17:14:10.792 INFO:tasks.workunit.client.1.smithi115.stdout:7/260: rename d3/f34 to d3/d4c/f51 0 2021-11-03T17:14:10.792 INFO:tasks.workunit.client.1.smithi115.stdout:7/261: readlink d3/l7 0 2021-11-03T17:14:10.792 INFO:tasks.workunit.client.1.smithi115.stdout:7/262: mkdir d3/da/d26/d52 0 2021-11-03T17:14:10.793 INFO:tasks.workunit.client.1.smithi115.stdout:7/263: mkdir d3/da/d26/d35/d53 0 2021-11-03T17:14:10.793 INFO:tasks.workunit.client.1.smithi115.stdout:7/264: unlink d3/da/d26/d33/l3a 0 2021-11-03T17:14:10.794 INFO:tasks.workunit.client.0.smithi027.stdout:5/454: dread d2/d4/d4e/d67/f82 [0,4194304] 0 2021-11-03T17:14:10.794 INFO:tasks.workunit.client.0.smithi027.stdout:5/455: truncate d2/d4/d4e/d67/f4a 986506 0 2021-11-03T17:14:10.796 INFO:tasks.workunit.client.0.smithi027.stdout:5/456: truncate d2/f3 749714 0 2021-11-03T17:14:10.796 INFO:tasks.workunit.client.0.smithi027.stdout:5/457: chown d2/d4/f34 42124390 1 2021-11-03T17:14:10.798 INFO:tasks.workunit.client.0.smithi027.stdout:5/458: creat d2/d12/f9a x:0 0 0 2021-11-03T17:14:10.804 INFO:tasks.workunit.client.1.smithi115.stdout:1/300: dread d7/dc/dd/d42/f44 [0,4194304] 0 2021-11-03T17:14:10.804 INFO:tasks.workunit.client.1.smithi115.stdout:1/301: getdents d7/dc/dd/d24 0 2021-11-03T17:14:10.897 INFO:tasks.workunit.client.0.smithi027.stdout:0/384: sync 2021-11-03T17:14:10.898 INFO:tasks.workunit.client.0.smithi027.stdout:0/385: write d9/f4f [453531,1620] 0 2021-11-03T17:14:10.898 INFO:tasks.workunit.client.0.smithi027.stdout:0/386: write d9/da/d64/f76 [1009646,49414] 0 2021-11-03T17:14:10.899 INFO:tasks.workunit.client.0.smithi027.stdout:0/387: truncate d9/da/f10 2477555 0 2021-11-03T17:14:10.899 INFO:tasks.workunit.client.0.smithi027.stdout:0/388: write d9/da/f37 [391491,22238] 0 2021-11-03T17:14:10.899 INFO:tasks.workunit.client.0.smithi027.stdout:0/389: read d9/da/f37 [75719,66791] 0 2021-11-03T17:14:10.900 INFO:tasks.workunit.client.0.smithi027.stdout:0/390: chown d9/f4d 16342 1 2021-11-03T17:14:10.900 INFO:tasks.workunit.client.0.smithi027.stdout:0/391: symlink d9/da/d64/d70/d6d/l82 0 2021-11-03T17:14:10.901 INFO:tasks.workunit.client.0.smithi027.stdout:0/392: symlink d9/da/d64/d70/l83 0 2021-11-03T17:14:10.912 INFO:tasks.workunit.client.0.smithi027.stdout:0/393: read d9/f1f [2010261,58872] 0 2021-11-03T17:14:10.923 INFO:tasks.workunit.client.1.smithi115.stdout:7/265: dwrite d3/da/d26/d35/f3e [0,4194304] 0 2021-11-03T17:14:10.923 INFO:tasks.workunit.client.1.smithi115.stdout:1/302: dwrite d7/dc/dd/d42/f47 [0,4194304] 0 2021-11-03T17:14:10.925 INFO:tasks.workunit.client.1.smithi115.stdout:7/266: rename d3/da/d26/d35/l4a to d3/da/d26/d52/l54 0 2021-11-03T17:14:10.925 INFO:tasks.workunit.client.1.smithi115.stdout:1/303: write d7/dc/f15 [2555959,75003] 0 2021-11-03T17:14:10.936 INFO:tasks.workunit.client.0.smithi027.stdout:5/459: dwrite d2/d4/f34 [0,4194304] 0 2021-11-03T17:14:10.936 INFO:tasks.workunit.client.0.smithi027.stdout:5/460: stat d2/d4/d4e/d67/d20/d25/d6a 0 2021-11-03T17:14:10.936 INFO:tasks.workunit.client.0.smithi027.stdout:5/461: stat d2/d4/d4e/d67/f82 0 2021-11-03T17:14:10.938 INFO:tasks.workunit.client.0.smithi027.stdout:5/462: rmdir d2/d2c/d3c 39 2021-11-03T17:14:10.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:10 smithi115 conmon[34670]: cluster 2021-11-03T17:14:10.605109+0000 mgr.smithi027.ufhhvb (mgr.14162) 304 : cluster [DBG] pgmap v252: 65 pgs: 65 active+clean; 1.3 GiB data, 3.9 GiB used, 533 GiB / 536 GiB avail; 62 MiB/s rd, 126 MiB/s wr, 219 op/s 2021-11-03T17:14:10.988 INFO:tasks.workunit.client.1.smithi115.stdout:1/304: write d7/dc/dd/d24/d43/f3c [4039278,44112] 0 2021-11-03T17:14:10.989 INFO:tasks.workunit.client.0.smithi027.stdout:1/334: sync 2021-11-03T17:14:10.990 INFO:tasks.workunit.client.1.smithi115.stdout:1/305: creat d7/dc/dd/f5a x:0 0 0 2021-11-03T17:14:10.992 INFO:tasks.workunit.client.1.smithi115.stdout:1/306: symlink d7/dc/d2a/l5b 0 2021-11-03T17:14:10.993 INFO:tasks.workunit.client.0.smithi027.stdout:1/335: creat d6/f7c x:0 0 0 2021-11-03T17:14:10.994 INFO:tasks.workunit.client.1.smithi115.stdout:1/307: creat d7/d38/f5c x:0 0 0 2021-11-03T17:14:10.996 INFO:tasks.workunit.client.1.smithi115.stdout:7/267: dwrite d3/da/d26/d35/f3f [0,4194304] 0 2021-11-03T17:14:10.998 INFO:tasks.workunit.client.0.smithi027.stdout:1/336: mkdir d6/de/d44/d79/d25/d28/d4c/d7d 0 2021-11-03T17:14:10.999 INFO:tasks.workunit.client.1.smithi115.stdout:7/268: rename d3/da/d26/d33/l3b to d3/da/d26/d33/l55 0 2021-11-03T17:14:11.000 INFO:tasks.workunit.client.0.smithi027.stdout:1/337: creat d6/de/d44/d79/d25/d28/d4c/d7d/f7e x:0 0 0 2021-11-03T17:14:11.001 INFO:tasks.workunit.client.1.smithi115.stdout:7/269: creat d3/da/d26/d52/f56 x:0 0 0 2021-11-03T17:14:11.002 INFO:tasks.workunit.client.1.smithi115.stdout:7/270: creat d3/da/d26/d35/f57 x:0 0 0 2021-11-03T17:14:11.004 INFO:tasks.workunit.client.0.smithi027.stdout:1/338: creat d6/de/d44/d79/d31/f7f x:0 0 0 2021-11-03T17:14:11.004 INFO:tasks.workunit.client.0.smithi027.stdout:1/339: readlink d6/de/d44/d79/d25/d28/l67 0 2021-11-03T17:14:11.004 INFO:tasks.workunit.client.0.smithi027.stdout:5/463: dwrite d2/d2c/d3c/d58/f5e [0,4194304] 0 2021-11-03T17:14:11.007 INFO:tasks.workunit.client.0.smithi027.stdout:5/464: mkdir d2/d2c/d9b 0 2021-11-03T17:14:11.008 INFO:tasks.workunit.client.0.smithi027.stdout:5/465: symlink d2/d2c/d3c/l9c 0 2021-11-03T17:14:11.010 INFO:tasks.workunit.client.1.smithi115.stdout:7/271: unlink d3/d4c/f51 0 2021-11-03T17:14:11.012 INFO:tasks.workunit.client.1.smithi115.stdout:7/272: creat d3/da/d26/d35/d53/f58 x:0 0 0 2021-11-03T17:14:11.012 INFO:tasks.workunit.client.1.smithi115.stdout:7/273: write d3/f1e [3444909,42713] 0 2021-11-03T17:14:11.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:10 smithi027 conmon[32322]: cluster 2021-11-03T17:14:10.605109+0000 mgr.smithi027.ufhhvb (mgr.14162 2021-11-03T17:14:11.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:10 smithi027 conmon[32322]: ) 304 : cluster [DBG] pgmap v252: 65 pgs: 65 active+clean; 1.3 GiB data, 3.9 GiB used, 533 GiB / 536 GiB avail; 62 MiB/s rd, 126 MiB/s wr, 219 op/s 2021-11-03T17:14:11.024 INFO:tasks.workunit.client.0.smithi027.stdout:1/340: dread d6/d11/d14/f35 [0,4194304] 0 2021-11-03T17:14:11.025 INFO:tasks.workunit.client.0.smithi027.stdout:1/341: write d6/de/d44/d79/f59 [1021064,116186] 0 2021-11-03T17:14:11.028 INFO:tasks.workunit.client.0.smithi027.stdout:1/342: rename d6/de/ff to d6/de/d44/d79/d37/f80 0 2021-11-03T17:14:11.031 INFO:tasks.workunit.client.0.smithi027.stdout:1/343: rename d6/de/d44/d79/d25/d28/d4c to d6/de/d44/d79/d25/d28/d5c/d81 0 2021-11-03T17:14:11.032 INFO:tasks.workunit.client.0.smithi027.stdout:1/344: dread d6/de/d44/d79/f63 [0,4194304] 0 2021-11-03T17:14:11.032 INFO:tasks.workunit.client.0.smithi027.stdout:1/345: write d6/d11/d14/f17 [271594,40821] 0 2021-11-03T17:14:11.033 INFO:tasks.workunit.client.0.smithi027.stdout:1/346: creat d6/de/d44/f82 x:0 0 0 2021-11-03T17:14:11.034 INFO:tasks.workunit.client.0.smithi027.stdout:1/347: dread d6/de/d44/d79/d37/d49/f50 [0,4194304] 0 2021-11-03T17:14:11.035 INFO:tasks.workunit.client.0.smithi027.stdout:1/348: write d6/d11/d14/f76 [40660,57584] 0 2021-11-03T17:14:11.035 INFO:tasks.workunit.client.0.smithi027.stdout:5/466: dwrite d2/d4/d4e/d67/d20/f3a [0,4194304] 0 2021-11-03T17:14:11.050 INFO:tasks.workunit.client.0.smithi027.stdout:5/467: dread d2/f18 [4194304,4194304] 0 2021-11-03T17:14:11.052 INFO:tasks.workunit.client.0.smithi027.stdout:1/349: dwrite d6/de/d44/d79/d25/d28/d41/f6f [0,4194304] 0 2021-11-03T17:14:11.052 INFO:tasks.workunit.client.0.smithi027.stdout:1/350: dread - d6/de/d44/d79/d37/d49/f5b zero size 2021-11-03T17:14:11.053 INFO:tasks.workunit.client.0.smithi027.stdout:5/468: unlink d2/d4/d4e/d67/f30 0 2021-11-03T17:14:11.056 INFO:tasks.workunit.client.0.smithi027.stdout:5/469: mkdir d2/d2c/d9b/d9d 0 2021-11-03T17:14:11.057 INFO:tasks.workunit.client.0.smithi027.stdout:5/470: stat d2/d12/f79 0 2021-11-03T17:14:11.057 INFO:tasks.workunit.client.0.smithi027.stdout:5/471: fsync d2/d4/d4e/d67/d20/f73 0 2021-11-03T17:14:11.057 INFO:tasks.workunit.client.0.smithi027.stdout:5/472: write d2/d4/d4e/d67/d20/f3a [860191,11449] 0 2021-11-03T17:14:11.058 INFO:tasks.workunit.client.0.smithi027.stdout:5/473: mkdir d2/d2c/d3c/d48/d65/d80/d9e 0 2021-11-03T17:14:11.061 INFO:tasks.workunit.client.0.smithi027.stdout:5/474: creat d2/d2c/d3c/f9f x:0 0 0 2021-11-03T17:14:11.064 INFO:tasks.workunit.client.0.smithi027.stdout:5/475: creat d2/d2c/d3c/d58/fa0 x:0 0 0 2021-11-03T17:14:11.064 INFO:tasks.workunit.client.0.smithi027.stdout:5/476: mknod d2/d12/ca1 1 2021-11-03T17:14:11.064 INFO:tasks.workunit.client.0.smithi027.stdout:5/477: dread - d2/d4/f6b zero size 2021-11-03T17:14:11.064 INFO:tasks.workunit.client.0.smithi027.stdout:5/478: mknod d2/d12/d7d/ca2 1 2021-11-03T17:14:11.065 INFO:tasks.workunit.client.0.smithi027.stdout:5/479: stat d2/d12/f44 0 2021-11-03T17:14:11.065 INFO:tasks.workunit.client.0.smithi027.stdout:5/480: chown d2/d4/f43 47580 1 2021-11-03T17:14:11.065 INFO:tasks.workunit.client.0.smithi027.stdout:5/481: write d2/d4/d61/f89 [554454,75062] 0 2021-11-03T17:14:11.066 INFO:tasks.workunit.client.0.smithi027.stdout:5/482: link d2/d4/d4e/d67/f82 d2/d2c/d3c/d48/d65/d80/fa3 0 2021-11-03T17:14:11.066 INFO:tasks.workunit.client.0.smithi027.stdout:1/351: dwrite f2 [0,4194304] 0 2021-11-03T17:14:11.069 INFO:tasks.workunit.client.0.smithi027.stdout:5/483: getdents d2/d2c/d9b 0 2021-11-03T17:14:11.072 INFO:tasks.workunit.client.0.smithi027.stdout:1/352: dread d6/de/d13/f38 [0,4194304] 0 2021-11-03T17:14:11.073 INFO:tasks.workunit.client.0.smithi027.stdout:5/484: getdents d2/d12 0 2021-11-03T17:14:11.074 INFO:tasks.workunit.client.0.smithi027.stdout:1/353: symlink d6/de/l83 0 2021-11-03T17:14:11.076 INFO:tasks.workunit.client.0.smithi027.stdout:5/485: symlink d2/d1f/la4 0 2021-11-03T17:14:11.076 INFO:tasks.workunit.client.0.smithi027.stdout:5/486: write d2/d4/d4e/d67/f11 [1820335,27392] 0 2021-11-03T17:14:11.076 INFO:tasks.workunit.client.0.smithi027.stdout:5/487: stat d2/d2c/d3c/d48/d65/l97 0 2021-11-03T17:14:11.077 INFO:tasks.workunit.client.0.smithi027.stdout:5/488: write d2/d4/d4e/d67/f2e [435195,40222] 0 2021-11-03T17:14:11.077 INFO:tasks.workunit.client.0.smithi027.stdout:5/489: chown d2/d1f 1 1 2021-11-03T17:14:11.077 INFO:tasks.workunit.client.0.smithi027.stdout:5/490: chown d2/d2c/d3c/d58/fa0 38220734 1 2021-11-03T17:14:11.077 INFO:tasks.workunit.client.0.smithi027.stdout:5/491: mknod d2/d1f/ca5 1 2021-11-03T17:14:11.078 INFO:tasks.workunit.client.0.smithi027.stdout:1/354: creat d6/de/d44/d79/d25/d28/d32/f84 x:0 0 0 2021-11-03T17:14:11.079 INFO:tasks.workunit.client.0.smithi027.stdout:5/492: rmdir d2/d2c/d9b 39 2021-11-03T17:14:11.079 INFO:tasks.workunit.client.0.smithi027.stdout:5/493: write d2/d4/d4e/d67/d8e/d99/f96 [28836,123039] 0 2021-11-03T17:14:11.079 INFO:tasks.workunit.client.0.smithi027.stdout:5/494: mknod d2/d4/d4e/d67/d20/d25/ca6 1 2021-11-03T17:14:11.079 INFO:tasks.workunit.client.0.smithi027.stdout:5/495: truncate d2/d4/f66 793280 0 2021-11-03T17:14:11.080 INFO:tasks.workunit.client.0.smithi027.stdout:5/496: write d2/d4/f13 [2564174,115351] 0 2021-11-03T17:14:11.080 INFO:tasks.workunit.client.0.smithi027.stdout:5/497: readlink d2/d2c/d3c/d48/d65/l97 0 2021-11-03T17:14:11.080 INFO:tasks.workunit.client.0.smithi027.stdout:1/355: creat d6/de/d44/d79/d25/d28/d5c/d81/f85 x:0 0 0 2021-11-03T17:14:11.081 INFO:tasks.workunit.client.0.smithi027.stdout:5/498: getdents d2/d2c/d3c 0 2021-11-03T17:14:11.082 INFO:tasks.workunit.client.0.smithi027.stdout:5/499: dread d2/f52 [0,4194304] 0 2021-11-03T17:14:11.082 INFO:tasks.workunit.client.0.smithi027.stdout:1/356: creat d6/de/d44/d79/d25/d28/d5c/d81/f86 x:0 0 0 2021-11-03T17:14:11.083 INFO:tasks.workunit.client.0.smithi027.stdout:5/500: getdents d2/d4/d4e/d67/d20 0 2021-11-03T17:14:11.086 INFO:tasks.workunit.client.0.smithi027.stdout:1/357: rename d6/de/d44/d79/l61 to d6/de/d44/d79/l87 0 2021-11-03T17:14:11.087 INFO:tasks.workunit.client.0.smithi027.stdout:1/358: dread - d6/de/d44/d79/d25/d28/d5c/f73 zero size 2021-11-03T17:14:11.087 INFO:tasks.workunit.client.0.smithi027.stdout:5/501: rmdir d2/d4/d4e 39 2021-11-03T17:14:11.088 INFO:tasks.workunit.client.0.smithi027.stdout:5/502: mkdir d2/d12/d7d/da7 0 2021-11-03T17:14:11.089 INFO:tasks.workunit.client.0.smithi027.stdout:5/503: mkdir d2/d4/d4e/d67/d20/d25/da8 0 2021-11-03T17:14:11.103 INFO:tasks.workunit.client.0.smithi027.stdout:1/359: dwrite d6/de/d44/f82 [0,4194304] 0 2021-11-03T17:14:11.107 INFO:tasks.workunit.client.0.smithi027.stdout:1/360: rename d6/f6a to d6/d11/d14/f88 0 2021-11-03T17:14:11.108 INFO:tasks.workunit.client.0.smithi027.stdout:1/361: getdents d6/de/d44/d79/d25/d28/d5c/d68 0 2021-11-03T17:14:11.110 INFO:tasks.workunit.client.0.smithi027.stdout:1/362: creat d6/d11/f89 x:0 0 0 2021-11-03T17:14:11.112 INFO:tasks.workunit.client.0.smithi027.stdout:1/363: link d6/de/d13/d15/f2f d6/de/d44/d79/d31/f8a 0 2021-11-03T17:14:11.121 INFO:tasks.workunit.client.0.smithi027.stdout:5/504: dwrite d2/d4/d4e/d67/d20/f3a [0,4194304] 0 2021-11-03T17:14:11.124 INFO:tasks.workunit.client.0.smithi027.stdout:5/505: rmdir d2/d2c/d3c/d48/d65/d80 39 2021-11-03T17:14:11.131 INFO:tasks.workunit.client.0.smithi027.stdout:5/506: dread d2/d12/f44 [0,4194304] 0 2021-11-03T17:14:11.132 INFO:tasks.workunit.client.0.smithi027.stdout:5/507: write d2/d2c/d3c/d48/d65/f56 [4000336,110071] 0 2021-11-03T17:14:11.133 INFO:tasks.workunit.client.0.smithi027.stdout:5/508: fdatasync d2/d2c/d3c/d58/f6e 0 2021-11-03T17:14:11.134 INFO:tasks.workunit.client.0.smithi027.stdout:5/509: mkdir d2/d4/d4e/d67/d20/d25/da9 0 2021-11-03T17:14:11.135 INFO:tasks.workunit.client.0.smithi027.stdout:5/510: write d2/d4/d4e/d67/f82 [1326131,12012] 0 2021-11-03T17:14:11.135 INFO:tasks.workunit.client.0.smithi027.stdout:5/511: read - d2/d4/d4e/d67/d20/d25/f71 zero size 2021-11-03T17:14:11.136 INFO:tasks.workunit.client.0.smithi027.stdout:5/512: stat d2/d4/f5 0 2021-11-03T17:14:11.136 INFO:tasks.workunit.client.0.smithi027.stdout:5/513: write d2/d4/f13 [4222681,120071] 0 2021-11-03T17:14:11.149 INFO:tasks.workunit.client.0.smithi027.stdout:5/514: dwrite d2/d12/f44 [0,4194304] 0 2021-11-03T17:14:11.151 INFO:tasks.workunit.client.0.smithi027.stdout:5/515: creat d2/d2c/d9b/faa x:0 0 0 2021-11-03T17:14:11.167 INFO:tasks.workunit.client.0.smithi027.stdout:5/516: dwrite d2/d4/d4e/d67/f7f [0,4194304] 0 2021-11-03T17:14:11.168 INFO:tasks.workunit.client.0.smithi027.stdout:5/517: fsync d2/d2c/d3c/f98 0 2021-11-03T17:14:11.181 INFO:tasks.workunit.client.0.smithi027.stdout:5/518: dwrite d2/f3e [0,4194304] 0 2021-11-03T17:14:11.182 INFO:tasks.workunit.client.0.smithi027.stdout:5/519: write d2/f3e [3654874,87386] 0 2021-11-03T17:14:11.183 INFO:tasks.workunit.client.0.smithi027.stdout:5/520: chown d2/d12 129774592 1 2021-11-03T17:14:11.184 INFO:tasks.workunit.client.0.smithi027.stdout:5/521: creat d2/d4/d4e/d67/d20/d25/da8/fab x:0 0 0 2021-11-03T17:14:11.185 INFO:tasks.workunit.client.0.smithi027.stdout:5/522: creat d2/d12/d7d/da7/fac x:0 0 0 2021-11-03T17:14:11.201 INFO:tasks.workunit.client.0.smithi027.stdout:5/523: dwrite d2/d2c/d3c/d58/f74 [0,4194304] 0 2021-11-03T17:14:11.216 INFO:tasks.workunit.client.0.smithi027.stdout:5/524: dwrite d2/f1e [0,4194304] 0 2021-11-03T17:14:11.225 INFO:tasks.workunit.client.0.smithi027.stdout:5/525: dread d2/d2c/d3c/d48/d65/f56 [0,4194304] 0 2021-11-03T17:14:11.230 INFO:tasks.workunit.client.0.smithi027.stdout:5/526: rename d2/d4/d4e/d67/d20/d25 to d2/d12/dad 0 2021-11-03T17:14:11.230 INFO:tasks.workunit.client.0.smithi027.stdout:5/527: readlink d2/d4/d61/l70 0 2021-11-03T17:14:11.232 INFO:tasks.workunit.client.0.smithi027.stdout:5/528: symlink d2/d4/lae 0 2021-11-03T17:14:11.234 INFO:tasks.workunit.client.0.smithi027.stdout:5/529: symlink d2/d2c/d3c/d48/laf 0 2021-11-03T17:14:11.235 INFO:tasks.workunit.client.0.smithi027.stdout:5/530: mknod d2/d4/d61/cb0 1 2021-11-03T17:14:11.235 INFO:tasks.workunit.client.0.smithi027.stdout:5/531: chown d2/d4/f43 32 1 2021-11-03T17:14:11.239 INFO:tasks.workunit.client.0.smithi027.stdout:5/532: symlink d2/d2c/d3c/d48/d65/d80/lb1 0 2021-11-03T17:14:11.239 INFO:tasks.workunit.client.0.smithi027.stdout:5/533: read - d2/d2c/d3c/f9f zero size 2021-11-03T17:14:11.240 INFO:tasks.workunit.client.0.smithi027.stdout:5/534: truncate d2/d4/f43 711261 0 2021-11-03T17:14:11.240 INFO:tasks.workunit.client.0.smithi027.stdout:5/535: write d2/d12/dad/f71 [519577,28977] 0 2021-11-03T17:14:11.240 INFO:tasks.workunit.client.0.smithi027.stdout:5/536: mknod d2/d2c/d9b/cb2 1 2021-11-03T17:14:11.241 INFO:tasks.workunit.client.0.smithi027.stdout:5/537: mkdir d2/d2c/d9b/db3 0 2021-11-03T17:14:11.241 INFO:tasks.workunit.client.0.smithi027.stdout:5/538: write d2/d2c/d3c/d48/d65/d80/f83 [386714,76178] 0 2021-11-03T17:14:11.242 INFO:tasks.workunit.client.0.smithi027.stdout:5/539: chown d2/d12/dad/da9 7 1 2021-11-03T17:14:11.243 INFO:tasks.workunit.client.0.smithi027.stdout:5/540: creat d2/d2c/d9b/db3/fb4 x:0 0 0 2021-11-03T17:14:11.269 INFO:tasks.workunit.client.0.smithi027.stdout:5/541: dwrite d2/d4/f43 [0,4194304] 0 2021-11-03T17:14:11.269 INFO:tasks.workunit.client.0.smithi027.stdout:5/542: fdatasync d2/d4/f34 0 2021-11-03T17:14:11.279 INFO:tasks.workunit.client.0.smithi027.stdout:5/543: dread d2/d4/d4e/d67/f4a [0,4194304] 0 2021-11-03T17:14:11.279 INFO:tasks.workunit.client.0.smithi027.stdout:5/544: fdatasync d2/d2c/d3c/f35 0 2021-11-03T17:14:11.280 INFO:tasks.workunit.client.0.smithi027.stdout:5/545: dread d2/f7 [0,4194304] 0 2021-11-03T17:14:11.280 INFO:tasks.workunit.client.0.smithi027.stdout:5/546: stat d2/d2c/d3c/d48/l87 0 2021-11-03T17:14:11.281 INFO:tasks.workunit.client.0.smithi027.stdout:5/547: creat d2/d2c/d9b/fb5 x:0 0 0 2021-11-03T17:14:11.282 INFO:tasks.workunit.client.0.smithi027.stdout:5/548: symlink d2/d2c/d9b/d9d/lb6 0 2021-11-03T17:14:11.284 INFO:tasks.workunit.client.0.smithi027.stdout:5/549: unlink d2/d12/f79 0 2021-11-03T17:14:11.284 INFO:tasks.workunit.client.0.smithi027.stdout:5/550: readlink d2/d2c/d3c/d48/d65/d80/lb1 0 2021-11-03T17:14:11.285 INFO:tasks.workunit.client.0.smithi027.stdout:5/551: creat d2/d4/d4e/d67/fb7 x:0 0 0 2021-11-03T17:14:11.286 INFO:tasks.workunit.client.0.smithi027.stdout:5/552: creat d2/d12/dad/fb8 x:0 0 0 2021-11-03T17:14:11.287 INFO:tasks.workunit.client.0.smithi027.stdout:5/553: creat d2/d4/d4e/d67/d20/fb9 x:0 0 0 2021-11-03T17:14:11.307 INFO:tasks.workunit.client.0.smithi027.stdout:5/554: dwrite d2/d12/dad/f71 [0,4194304] 0 2021-11-03T17:14:11.309 INFO:tasks.workunit.client.0.smithi027.stdout:5/555: creat d2/d2c/d3c/fba x:0 0 0 2021-11-03T17:14:11.309 INFO:tasks.workunit.client.0.smithi027.stdout:5/556: chown d2/d4/d4e/d67/f7f 27855082 1 2021-11-03T17:14:11.310 INFO:tasks.workunit.client.0.smithi027.stdout:5/557: chown d2/d12/dad/l4d 65535378 1 2021-11-03T17:14:11.312 INFO:tasks.workunit.client.0.smithi027.stdout:5/558: fsync d2/d2c/d3c/d48/d65/d80/f83 0 2021-11-03T17:14:11.312 INFO:tasks.workunit.client.0.smithi027.stdout:5/559: write d2/d2c/d3c/fba [678851,82544] 0 2021-11-03T17:14:11.317 INFO:tasks.workunit.client.0.smithi027.stdout:5/560: dread d2/d2c/d3c/fba [0,4194304] 0 2021-11-03T17:14:11.317 INFO:tasks.workunit.client.0.smithi027.stdout:5/561: chown d2/d4/d4e/d67/d20/f3a 954445 1 2021-11-03T17:14:11.317 INFO:tasks.workunit.client.0.smithi027.stdout:5/562: chown d2/d2c/d3c/d58 0 1 2021-11-03T17:14:11.318 INFO:tasks.workunit.client.0.smithi027.stdout:5/563: truncate d2/d4/d4e/d67/d20/fb9 483330 0 2021-11-03T17:14:11.318 INFO:tasks.workunit.client.0.smithi027.stdout:5/564: truncate d2/d12/dad/da8/fab 643750 0 2021-11-03T17:14:11.318 INFO:tasks.workunit.client.0.smithi027.stdout:5/565: link d2/d4/l22 d2/d12/d7d/lbb 0 2021-11-03T17:14:11.319 INFO:tasks.workunit.client.0.smithi027.stdout:5/566: read d2/f1e [660093,55796] 0 2021-11-03T17:14:11.320 INFO:tasks.workunit.client.0.smithi027.stdout:5/567: write d2/d4/f66 [2351277,82027] 0 2021-11-03T17:14:11.321 INFO:tasks.workunit.client.0.smithi027.stdout:5/568: truncate d2/d12/f5c 204648 0 2021-11-03T17:14:11.337 INFO:tasks.workunit.client.0.smithi027.stdout:5/569: dwrite d2/d4/d4e/d67/d20/f3a [4194304,4194304] 0 2021-11-03T17:14:11.339 INFO:tasks.workunit.client.0.smithi027.stdout:5/570: creat d2/d1f/fbc x:0 0 0 2021-11-03T17:14:11.340 INFO:tasks.workunit.client.0.smithi027.stdout:5/571: dread d2/d4/d4e/d67/d20/fb9 [0,4194304] 0 2021-11-03T17:14:11.341 INFO:tasks.workunit.client.0.smithi027.stdout:5/572: write d2/d4/d4e/d67/f7f [512192,40360] 0 2021-11-03T17:14:11.343 INFO:tasks.workunit.client.0.smithi027.stdout:5/573: getdents d2/d4/d4e/d67/d20 0 2021-11-03T17:14:11.343 INFO:tasks.workunit.client.0.smithi027.stdout:5/574: fdatasync d2/d2c/d3c/d48/d65/d80/fa3 0 2021-11-03T17:14:11.344 INFO:tasks.workunit.client.0.smithi027.stdout:5/575: rmdir d2/d12/dad 39 2021-11-03T17:14:11.346 INFO:tasks.workunit.client.0.smithi027.stdout:5/576: creat d2/d2c/d3c/d48/d65/d80/d9e/fbd x:0 0 0 2021-11-03T17:14:11.347 INFO:tasks.workunit.client.0.smithi027.stdout:5/577: fsync d2/d4/d4e/d67/f11 0 2021-11-03T17:14:11.347 INFO:tasks.workunit.client.0.smithi027.stdout:5/578: chown d2/d12/d7d/lbb 716700 1 2021-11-03T17:14:11.348 INFO:tasks.workunit.client.0.smithi027.stdout:5/579: link d2/d4/d4e/d67/f78 d2/d2c/d9b/d9d/fbe 0 2021-11-03T17:14:11.349 INFO:tasks.workunit.client.0.smithi027.stdout:5/580: fdatasync d2/d2c/d3c/d48/d65/d80/fa3 0 2021-11-03T17:14:11.350 INFO:tasks.workunit.client.0.smithi027.stdout:5/581: rename d2/d2c/d9b/fb5 to d2/d1f/fbf 0 2021-11-03T17:14:11.351 INFO:tasks.workunit.client.0.smithi027.stdout:5/582: write d2/d4/f43 [3427585,12178] 0 2021-11-03T17:14:11.352 INFO:tasks.workunit.client.0.smithi027.stdout:5/583: creat d2/d12/fc0 x:0 0 0 2021-11-03T17:14:11.367 INFO:tasks.workunit.client.0.smithi027.stdout:5/584: dwrite d2/d2c/d3c/d48/d65/f56 [0,4194304] 0 2021-11-03T17:14:11.368 INFO:tasks.workunit.client.0.smithi027.stdout:5/585: mknod d2/d4/d61/cc1 1 2021-11-03T17:14:11.368 INFO:tasks.workunit.client.0.smithi027.stdout:5/586: rmdir d2/d12/d7d 39 2021-11-03T17:14:11.369 INFO:tasks.workunit.client.0.smithi027.stdout:5/587: symlink d2/d4/d4e/d67/d8e/lc2 0 2021-11-03T17:14:11.371 INFO:tasks.workunit.client.0.smithi027.stdout:5/588: truncate d2/d4/d4e/d67/f2e 430522 0 2021-11-03T17:14:11.372 INFO:tasks.workunit.client.0.smithi027.stdout:5/589: symlink d2/d2c/d3c/d58/lc3 0 2021-11-03T17:14:11.373 INFO:tasks.workunit.client.0.smithi027.stdout:5/590: creat d2/d12/dad/da8/fc4 x:0 0 0 2021-11-03T17:14:11.402 INFO:tasks.workunit.client.0.smithi027.stdout:5/591: dwrite d2/f42 [0,4194304] 0 2021-11-03T17:14:11.419 INFO:tasks.workunit.client.0.smithi027.stdout:5/592: dwrite d2/d4/d61/f89 [0,4194304] 0 2021-11-03T17:14:11.431 INFO:tasks.workunit.client.0.smithi027.stdout:5/593: dwrite d2/d2c/d3c/d58/fa0 [0,4194304] 0 2021-11-03T17:14:11.433 INFO:tasks.workunit.client.0.smithi027.stdout:5/594: read d2/d4/f5 [1924121,99128] 0 2021-11-03T17:14:11.434 INFO:tasks.workunit.client.0.smithi027.stdout:5/595: write d2/d12/f9a [1020150,88496] 0 2021-11-03T17:14:11.441 INFO:tasks.workunit.client.0.smithi027.stdout:5/596: dread d2/d2c/d3c/f41 [0,4194304] 0 2021-11-03T17:14:11.442 INFO:tasks.workunit.client.0.smithi027.stdout:5/597: write d2/d4/f5 [464314,84626] 0 2021-11-03T17:14:11.442 INFO:tasks.workunit.client.0.smithi027.stdout:5/598: fdatasync d2/d1f/fbc 0 2021-11-03T17:14:11.443 INFO:tasks.workunit.client.0.smithi027.stdout:5/599: write d2/d4/f34 [1078786,80142] 0 2021-11-03T17:14:11.464 INFO:tasks.workunit.client.0.smithi027.stdout:5/600: dwrite d2/d4/f5 [0,4194304] 0 2021-11-03T17:14:11.464 INFO:tasks.workunit.client.0.smithi027.stdout:5/601: write d2/d12/d7d/da7/fac [26747,101234] 0 2021-11-03T17:14:11.464 INFO:tasks.workunit.client.0.smithi027.stdout:5/602: dread - d2/d2c/d9b/faa zero size 2021-11-03T17:14:11.464 INFO:tasks.workunit.client.0.smithi027.stdout:5/603: dread - d2/d1f/fbc zero size 2021-11-03T17:14:11.465 INFO:tasks.workunit.client.0.smithi027.stdout:5/604: chown d2/d12/dad/fb8 360557 1 2021-11-03T17:14:11.465 INFO:tasks.workunit.client.0.smithi027.stdout:5/605: creat d2/d12/dad/d6a/fc5 x:0 0 0 2021-11-03T17:14:11.481 INFO:tasks.workunit.client.0.smithi027.stdout:5/606: dwrite d2/d4/f5 [4194304,4194304] 0 2021-11-03T17:14:11.483 INFO:tasks.workunit.client.0.smithi027.stdout:5/607: symlink d2/d2c/d9b/d9d/lc6 0 2021-11-03T17:14:11.483 INFO:tasks.workunit.client.0.smithi027.stdout:5/608: readlink d2/d2c/d3c/d48/d65/l50 0 2021-11-03T17:14:11.485 INFO:tasks.workunit.client.0.smithi027.stdout:5/609: dread d2/d4/d4e/d67/f2e [0,4194304] 0 2021-11-03T17:14:11.485 INFO:tasks.workunit.client.0.smithi027.stdout:5/610: stat d2/d2c/d3c/l9c 0 2021-11-03T17:14:11.485 INFO:tasks.workunit.client.0.smithi027.stdout:5/611: stat d2/d2c 0 2021-11-03T17:14:11.761 INFO:tasks.workunit.client.0.smithi027.stdout:7/245: sync 2021-11-03T17:14:11.761 INFO:tasks.workunit.client.0.smithi027.stdout:7/246: mknod d0/d2/d10/d1e/c4e 1 2021-11-03T17:14:11.763 INFO:tasks.workunit.client.0.smithi027.stdout:7/247: mkdir d0/d2/d4f 0 2021-11-03T17:14:11.763 INFO:tasks.workunit.client.0.smithi027.stdout:7/248: mknod d0/d2/d10/d1e/d20/d31/d36/d3a/c50 1 2021-11-03T17:14:11.763 INFO:tasks.workunit.client.0.smithi027.stdout:7/249: write d0/d2/d10/d1e/d20/f43 [295961,76969] 0 2021-11-03T17:14:11.778 INFO:tasks.workunit.client.0.smithi027.stdout:7/250: dwrite d0/d2/f5 [4194304,4194304] 0 2021-11-03T17:14:11.778 INFO:tasks.workunit.client.0.smithi027.stdout:7/251: read - d0/d2/d10/d1e/f24 zero size 2021-11-03T17:14:11.779 INFO:tasks.workunit.client.0.smithi027.stdout:7/252: creat d0/f51 x:0 0 0 2021-11-03T17:14:11.780 INFO:tasks.workunit.client.0.smithi027.stdout:7/253: truncate d0/f18 570246 0 2021-11-03T17:14:11.781 INFO:tasks.workunit.client.0.smithi027.stdout:7/254: chown d0/l1d 1212266 1 2021-11-03T17:14:11.800 INFO:tasks.workunit.client.0.smithi027.stdout:7/255: dwrite d0/d2/d1c/f3c [0,4194304] 0 2021-11-03T17:14:11.800 INFO:tasks.workunit.client.0.smithi027.stdout:7/256: dread d0/d2/d1c/f2b [0,4194304] 0 2021-11-03T17:14:11.800 INFO:tasks.workunit.client.0.smithi027.stdout:7/257: mknod d0/d2/d1c/c52 1 2021-11-03T17:14:11.801 INFO:tasks.workunit.client.0.smithi027.stdout:7/258: mkdir d0/d2d/d53 0 2021-11-03T17:14:11.801 INFO:tasks.workunit.client.0.smithi027.stdout:7/259: write d0/d2/f5 [8350439,44587] 0 2021-11-03T17:14:11.805 INFO:tasks.workunit.client.0.smithi027.stdout:7/260: creat d0/d2/d1f/f54 x:0 0 0 2021-11-03T17:14:11.805 INFO:tasks.workunit.client.0.smithi027.stdout:7/261: symlink d0/l55 0 2021-11-03T17:14:11.806 INFO:tasks.workunit.client.0.smithi027.stdout:7/262: symlink d0/d2/d10/d35/l56 0 2021-11-03T17:14:11.895 INFO:tasks.workunit.client.0.smithi027.stdout:3/144: sync 2021-11-03T17:14:11.907 INFO:tasks.workunit.client.0.smithi027.stdout:3/145: dwrite d1/f23 [0,4194304] 0 2021-11-03T17:14:11.921 INFO:tasks.workunit.client.0.smithi027.stdout:3/146: dwrite d1/f22 [0,4194304] 0 2021-11-03T17:14:11.921 INFO:tasks.workunit.client.0.smithi027.stdout:3/147: write d1/fa [4482790,23601] 0 2021-11-03T17:14:11.921 INFO:tasks.workunit.client.0.smithi027.stdout:3/148: stat d1/f18 0 2021-11-03T17:14:11.923 INFO:tasks.workunit.client.0.smithi027.stdout:3/149: creat d1/f29 x:0 0 0 2021-11-03T17:14:11.924 INFO:tasks.workunit.client.0.smithi027.stdout:3/150: creat d1/f2a x:0 0 0 2021-11-03T17:14:11.925 INFO:tasks.workunit.client.0.smithi027.stdout:3/151: write d1/db/f12 [849736,109897] 0 2021-11-03T17:14:11.926 INFO:tasks.workunit.client.0.smithi027.stdout:3/152: symlink d1/l2b 0 2021-11-03T17:14:11.928 INFO:tasks.workunit.client.0.smithi027.stdout:3/153: dread d1/f1d [0,4194304] 0 2021-11-03T17:14:11.928 INFO:tasks.workunit.client.0.smithi027.stdout:3/154: truncate d1/f29 838185 0 2021-11-03T17:14:11.931 INFO:tasks.workunit.client.0.smithi027.stdout:3/155: rename d1/db/l28 to d1/l2c 0 2021-11-03T17:14:11.944 INFO:tasks.workunit.client.0.smithi027.stdout:3/156: dwrite d1/f23 [0,4194304] 0 2021-11-03T17:14:11.946 INFO:tasks.workunit.client.0.smithi027.stdout:3/157: link d1/db/l20 d1/db/l2d 0 2021-11-03T17:14:11.948 INFO:tasks.workunit.client.0.smithi027.stdout:3/158: rename d1/f6 to d1/f2e 0 2021-11-03T17:14:11.949 INFO:tasks.workunit.client.0.smithi027.stdout:3/159: read d1/f23 [3102388,76923] 0 2021-11-03T17:14:11.950 INFO:tasks.workunit.client.0.smithi027.stdout:3/160: rename d1/f23 to d1/f2f 0 2021-11-03T17:14:11.952 INFO:tasks.workunit.client.0.smithi027.stdout:3/161: rename d1/f29 to d1/f30 0 2021-11-03T17:14:11.953 INFO:tasks.workunit.client.0.smithi027.stdout:3/162: write d1/f14 [549056,15038] 0 2021-11-03T17:14:11.953 INFO:tasks.workunit.client.0.smithi027.stdout:3/163: fdatasync d1/f9 0 2021-11-03T17:14:11.954 INFO:tasks.workunit.client.0.smithi027.stdout:3/164: write d1/f2f [233577,40903] 0 2021-11-03T17:14:11.960 INFO:tasks.workunit.client.0.smithi027.stdout:3/165: dread d1/f22 [0,4194304] 0 2021-11-03T17:14:11.969 INFO:tasks.workunit.client.0.smithi027.stdout:3/166: dread d1/db/f1c [0,4194304] 0 2021-11-03T17:14:11.969 INFO:tasks.workunit.client.0.smithi027.stdout:2/319: sync 2021-11-03T17:14:11.970 INFO:tasks.workunit.client.0.smithi027.stdout:2/320: write d0/d1b/d1c/f5e [771147,71937] 0 2021-11-03T17:14:11.970 INFO:tasks.workunit.client.0.smithi027.stdout:2/321: fsync d0/d1b/d1c/f26 0 2021-11-03T17:14:11.975 INFO:tasks.workunit.client.0.smithi027.stdout:3/167: getdents d1/db 0 2021-11-03T17:14:11.991 INFO:tasks.workunit.client.0.smithi027.stdout:2/322: link d0/d5/f3 d0/d2a/f66 0 2021-11-03T17:14:11.995 INFO:tasks.workunit.client.0.smithi027.stdout:2/323: dread d0/d1b/d29/f4b [0,4194304] 0 2021-11-03T17:14:11.996 INFO:tasks.workunit.client.0.smithi027.stdout:3/168: dwrite d1/f1d [0,4194304] 0 2021-11-03T17:14:11.999 INFO:tasks.workunit.client.0.smithi027.stdout:3/169: rename d1/db/l20 to d1/l31 0 2021-11-03T17:14:12.000 INFO:tasks.workunit.client.0.smithi027.stdout:3/170: write d1/f9 [110520,72934] 0 2021-11-03T17:14:12.001 INFO:tasks.workunit.client.0.smithi027.stdout:3/171: stat d1/l2b 0 2021-11-03T17:14:12.001 INFO:tasks.workunit.client.0.smithi027.stdout:3/172: chown d1/l2b 6483849 1 2021-11-03T17:14:12.002 INFO:tasks.workunit.client.0.smithi027.stdout:2/324: mkdir d0/d1b/d29/d54/d67 0 2021-11-03T17:14:12.003 INFO:tasks.workunit.client.0.smithi027.stdout:2/325: dread - d0/d1b/d1c/f57 zero size 2021-11-03T17:14:12.004 INFO:tasks.workunit.client.0.smithi027.stdout:3/173: creat d1/db/f32 x:0 0 0 2021-11-03T17:14:12.004 INFO:tasks.workunit.client.0.smithi027.stdout:3/174: write d1/f1d [4478885,115634] 0 2021-11-03T17:14:12.004 INFO:tasks.workunit.client.0.smithi027.stdout:3/175: write d1/f17 [3217723,94729] 0 2021-11-03T17:14:12.006 INFO:tasks.workunit.client.0.smithi027.stdout:3/176: symlink d1/l33 0 2021-11-03T17:14:12.009 INFO:tasks.workunit.client.0.smithi027.stdout:3/177: link d1/f2a d1/f34 0 2021-11-03T17:14:12.009 INFO:tasks.workunit.client.0.smithi027.stdout:3/178: chown d1/f13 1 1 2021-11-03T17:14:12.009 INFO:tasks.workunit.client.0.smithi027.stdout:3/179: chown d1/l2b 12907526 1 2021-11-03T17:14:12.010 INFO:tasks.workunit.client.0.smithi027.stdout:3/180: creat d1/db/f35 x:0 0 0 2021-11-03T17:14:12.010 INFO:tasks.workunit.client.0.smithi027.stdout:3/181: fdatasync d1/db/f32 0 2021-11-03T17:14:12.010 INFO:tasks.workunit.client.0.smithi027.stdout:3/182: symlink d1/l36 0 2021-11-03T17:14:12.012 INFO:tasks.workunit.client.0.smithi027.stdout:3/183: rename d1/l27 to d1/l37 0 2021-11-03T17:14:12.014 INFO:tasks.workunit.client.0.smithi027.stdout:3/184: link d1/db/f35 d1/f38 0 2021-11-03T17:14:12.015 INFO:tasks.workunit.client.0.smithi027.stdout:3/185: symlink d1/db/l39 0 2021-11-03T17:14:12.016 INFO:tasks.workunit.client.0.smithi027.stdout:3/186: symlink d1/l3a 0 2021-11-03T17:14:12.023 INFO:tasks.workunit.client.0.smithi027.stdout:2/326: dwrite d0/d1b/d1c/f57 [0,4194304] 0 2021-11-03T17:14:12.031 INFO:tasks.workunit.client.0.smithi027.stdout:3/187: dwrite d1/f22 [4194304,4194304] 0 2021-11-03T17:14:12.031 INFO:tasks.workunit.client.0.smithi027.stdout:3/188: chown d1/db/l39 37396244 1 2021-11-03T17:14:12.032 INFO:tasks.workunit.client.0.smithi027.stdout:3/189: write d1/f30 [159582,34165] 0 2021-11-03T17:14:12.032 INFO:tasks.workunit.client.0.smithi027.stdout:3/190: readlink d1/l36 0 2021-11-03T17:14:12.039 INFO:tasks.workunit.client.0.smithi027.stdout:2/327: write d0/d5/d2c/f3d [785842,100378] 0 2021-11-03T17:14:12.040 INFO:tasks.workunit.client.0.smithi027.stdout:3/191: creat d1/db/f3b x:0 0 0 2021-11-03T17:14:12.043 INFO:tasks.workunit.client.0.smithi027.stdout:2/328: link d0/d5/l42 d0/d2a/l68 0 2021-11-03T17:14:12.046 INFO:tasks.workunit.client.0.smithi027.stdout:3/192: rename d1/f25 to d1/db/f3c 0 2021-11-03T17:14:12.071 INFO:tasks.workunit.client.0.smithi027.stdout:2/329: dwrite d0/d1b/d29/d54/d64/f55 [0,4194304] 0 2021-11-03T17:14:12.071 INFO:tasks.workunit.client.0.smithi027.stdout:2/330: fdatasync d0/d1b/d29/f4b 0 2021-11-03T17:14:12.072 INFO:tasks.workunit.client.0.smithi027.stdout:2/331: mknod d0/d1b/d29/d54/d67/c69 1 2021-11-03T17:14:12.072 INFO:tasks.workunit.client.0.smithi027.stdout:2/332: dread - d0/f46 zero size 2021-11-03T17:14:12.072 INFO:tasks.workunit.client.0.smithi027.stdout:3/193: dwrite d1/f18 [0,4194304] 0 2021-11-03T17:14:12.074 INFO:tasks.workunit.client.0.smithi027.stdout:2/333: mkdir d0/d1b/d29/d54/d67/d6a 0 2021-11-03T17:14:12.091 INFO:tasks.workunit.client.0.smithi027.stdout:3/194: write d1/fa [1739424,119795] 0 2021-11-03T17:14:12.091 INFO:tasks.workunit.client.0.smithi027.stdout:3/195: read - d1/db/f35 zero size 2021-11-03T17:14:12.092 INFO:tasks.workunit.client.0.smithi027.stdout:3/196: creat d1/db/f3d x:0 0 0 2021-11-03T17:14:12.096 INFO:tasks.workunit.client.0.smithi027.stdout:2/334: dwrite d0/d1b/d29/f2b [0,4194304] 0 2021-11-03T17:14:12.097 INFO:tasks.workunit.client.0.smithi027.stdout:2/335: fdatasync d0/d2a/f23 0 2021-11-03T17:14:12.097 INFO:tasks.workunit.client.0.smithi027.stdout:2/336: truncate d0/d1b/d29/f4b 4843208 0 2021-11-03T17:14:12.098 INFO:tasks.workunit.client.0.smithi027.stdout:2/337: write d0/d1b/d29/f4e [276833,17174] 0 2021-11-03T17:14:12.100 INFO:tasks.workunit.client.0.smithi027.stdout:2/338: mkdir d0/d5/d6b 0 2021-11-03T17:14:12.102 INFO:tasks.workunit.client.0.smithi027.stdout:2/339: rename d0/d1b/d1c/f5e to d0/d1b/d29/d54/d67/f6c 0 2021-11-03T17:14:12.103 INFO:tasks.workunit.client.0.smithi027.stdout:2/340: getdents d0/d1b 0 2021-11-03T17:14:12.105 INFO:tasks.workunit.client.0.smithi027.stdout:2/341: creat d0/d5/f6d x:0 0 0 2021-11-03T17:14:12.105 INFO:tasks.workunit.client.0.smithi027.stdout:2/342: chown d0/d1b/d29/d58/d5b/f62 1709316504 1 2021-11-03T17:14:12.106 INFO:tasks.workunit.client.0.smithi027.stdout:2/343: chown d0/d1b/l60 16 1 2021-11-03T17:14:12.106 INFO:tasks.workunit.client.0.smithi027.stdout:2/344: unlink d0/d5/l42 0 2021-11-03T17:14:12.109 INFO:tasks.workunit.client.0.smithi027.stdout:2/345: rename d0/d1b/d29/d54/d64/f55 to d0/d1b/f6e 0 2021-11-03T17:14:12.109 INFO:tasks.workunit.client.0.smithi027.stdout:3/197: dwrite d1/f38 [0,4194304] 0 2021-11-03T17:14:12.110 INFO:tasks.workunit.client.0.smithi027.stdout:3/198: stat d1/fa 0 2021-11-03T17:14:12.121 INFO:tasks.workunit.client.0.smithi027.stdout:3/199: truncate d1/f17 1903138 0 2021-11-03T17:14:12.134 INFO:tasks.workunit.client.0.smithi027.stdout:2/346: dread d0/d5/f3 [0,4194304] 0 2021-11-03T17:14:12.134 INFO:tasks.workunit.client.0.smithi027.stdout:2/347: write d0/d1b/d29/d58/f5c [414603,38899] 0 2021-11-03T17:14:12.135 INFO:tasks.workunit.client.0.smithi027.stdout:2/348: readlink d0/d2a/l22 0 2021-11-03T17:14:12.136 INFO:tasks.workunit.client.0.smithi027.stdout:2/349: symlink d0/d1b/d29/d58/l6f 0 2021-11-03T17:14:12.137 INFO:tasks.workunit.client.0.smithi027.stdout:2/350: write d0/d1b/f27 [332995,100041] 0 2021-11-03T17:14:12.138 INFO:tasks.workunit.client.0.smithi027.stdout:3/200: dwrite d1/db/f3d [0,4194304] 0 2021-11-03T17:14:12.138 INFO:tasks.workunit.client.0.smithi027.stdout:3/201: fsync d1/db/f3c 0 2021-11-03T17:14:12.139 INFO:tasks.workunit.client.0.smithi027.stdout:2/351: mkdir d0/d2a/d5d/d70 0 2021-11-03T17:14:12.142 INFO:tasks.workunit.client.0.smithi027.stdout:3/202: getdents d1 0 2021-11-03T17:14:12.142 INFO:tasks.workunit.client.0.smithi027.stdout:3/203: fdatasync d1/f38 0 2021-11-03T17:14:12.143 INFO:tasks.workunit.client.0.smithi027.stdout:2/352: symlink d0/d1b/d1c/l71 0 2021-11-03T17:14:12.146 INFO:tasks.workunit.client.0.smithi027.stdout:3/204: creat d1/db/f3e x:0 0 0 2021-11-03T17:14:12.153 INFO:tasks.workunit.client.0.smithi027.stdout:3/205: truncate d1/f22 4417868 0 2021-11-03T17:14:12.155 INFO:tasks.workunit.client.0.smithi027.stdout:3/206: creat d1/db/f3f x:0 0 0 2021-11-03T17:14:12.162 INFO:tasks.workunit.client.0.smithi027.stdout:2/353: dwrite d0/f10 [0,4194304] 0 2021-11-03T17:14:12.164 INFO:tasks.workunit.client.0.smithi027.stdout:2/354: symlink d0/d1b/d29/d54/l72 0 2021-11-03T17:14:12.169 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.174+0000 7f77b6648700 1 -- 172.21.15.27:0/1170726744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77b00f9850 msgr2=0x7f77b00fbc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:14:12.169 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.174+0000 7f77b6648700 1 --2- 172.21.15.27:0/1170726744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77b00f9850 0x7f77b00fbc80 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f77a4004660 tx=0x7f77a4009b30).stop 2021-11-03T17:14:12.170 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.174+0000 7f77b6648700 1 -- 172.21.15.27:0/1170726744 shutdown_connections 2021-11-03T17:14:12.170 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.174+0000 7f77b6648700 1 --2- 172.21.15.27:0/1170726744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77b00f9850 0x7f77b00fbc80 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:12.170 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.174+0000 7f77b6648700 1 --2- 172.21.15.27:0/1170726744 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77b0060250 0x7f77b00f9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:14:12.170 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.174+0000 7f77b6648700 1 -- 172.21.15.27:0/1170726744 >> 172.21.15.27:0/1170726744 conn(0x7f77b00f0d40 msgr2=0x7f77b00f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:14:12.171 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.174+0000 7f77b6648700 1 -- 172.21.15.27:0/1170726744 shutdown_connections 2021-11-03T17:14:12.171 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.174+0000 7f77b6648700 1 -- 172.21.15.27:0/1170726744 wait complete. 2021-11-03T17:14:12.173 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.175+0000 7f77b6648700 1 Processor -- start 2021-11-03T17:14:12.174 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.175+0000 7f77b6648700 1 -- start start 2021-11-03T17:14:12.174 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.175+0000 7f77b6648700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77b0060250 0x7f77b0106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:14:12.174 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.175+0000 7f77b6648700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77b00f9850 0x7f77b0107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:14:12.174 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.175+0000 7f77b6648700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b0107aa0 con 0x7f77b0060250 2021-11-03T17:14:12.175 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.175+0000 7f77b6648700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b0108350 con 0x7f77b00f9850 2021-11-03T17:14:12.175 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.175+0000 7f77affff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77b0060250 0x7f77b0106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:14:12.175 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.175+0000 7f77affff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77b0060250 0x7f77b0106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55308/0 (socket says 172.21.15.27:55308) 2021-11-03T17:14:12.175 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.175+0000 7f77affff700 1 -- 172.21.15.27:0/147811846 learned_addr learned my addr 172.21.15.27:0/147811846 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:14:12.176 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.176+0000 7f77affff700 1 -- 172.21.15.27:0/147811846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77b00f9850 msgr2=0x7f77b0107460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:14:12.176 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.176+0000 7f77affff700 1 --2- 172.21.15.27:0/147811846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77b00f9850 0x7f77b0107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:14:12.176 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.176+0000 7f77affff700 1 -- 172.21.15.27:0/147811846 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a4005040 con 0x7f77b0060250 2021-11-03T17:14:12.176 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.176+0000 7f77affff700 1 --2- 172.21.15.27:0/147811846 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77b0060250 0x7f77b0106f70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f77a000b410 tx=0x7f77a0006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:14:12.177 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.176+0000 7f77acff9700 1 -- 172.21.15.27:0/147811846 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77a00125f0 con 0x7f77b0060250 2021-11-03T17:14:12.177 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.176+0000 7f77b6648700 1 -- 172.21.15.27:0/147811846 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77b0108630 con 0x7f77b0060250 2021-11-03T17:14:12.177 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.176+0000 7f77acff9700 1 -- 172.21.15.27:0/147811846 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f77a0012c00 con 0x7f77b0060250 2021-11-03T17:14:12.177 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.176+0000 7f77acff9700 1 -- 172.21.15.27:0/147811846 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77a0002ae0 con 0x7f77b0060250 2021-11-03T17:14:12.178 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.176+0000 7f77b6648700 1 -- 172.21.15.27:0/147811846 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77b0108bf0 con 0x7f77b0060250 2021-11-03T17:14:12.178 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.177+0000 7f77b6648700 1 -- 172.21.15.27:0/147811846 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77b00f49b0 con 0x7f77b0060250 2021-11-03T17:14:12.178 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.178+0000 7f77acff9700 1 -- 172.21.15.27:0/147811846 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f77a000f030 con 0x7f77b0060250 2021-11-03T17:14:12.178 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.179+0000 7f77acff9700 1 --2- 172.21.15.27:0/147811846 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f779805c2b0 0x7f779805e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:14:12.179 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.179+0000 7f77acff9700 1 -- 172.21.15.27:0/147811846 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f77a00766d0 con 0x7f77b0060250 2021-11-03T17:14:12.179 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.179+0000 7f77af7fe700 1 --2- 172.21.15.27:0/147811846 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f779805c2b0 0x7f779805e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:14:12.180 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.179+0000 7f77af7fe700 1 --2- 172.21.15.27:0/147811846 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f779805c2b0 0x7f779805e770 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f77a4004e30 tx=0x7f77a4009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:14:12.180 INFO:tasks.workunit.client.0.smithi027.stdout:2/355: read d0/d2a/f30 [472168,116347] 0 2021-11-03T17:14:12.181 INFO:tasks.workunit.client.0.smithi027.stdout:2/356: chown d0/d5/d2c 810604 1 2021-11-03T17:14:12.181 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.182+0000 7f77acff9700 1 -- 172.21.15.27:0/147811846 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f77a00494a0 con 0x7f77b0060250 2021-11-03T17:14:12.181 INFO:tasks.workunit.client.0.smithi027.stdout:3/207: dwrite d1/f2e [0,4194304] 0 2021-11-03T17:14:12.182 INFO:tasks.workunit.client.0.smithi027.stdout:3/208: mkdir d1/d40 0 2021-11-03T17:14:12.183 INFO:tasks.workunit.client.0.smithi027.stdout:3/209: chown d1/fa 430 1 2021-11-03T17:14:12.183 INFO:tasks.workunit.client.0.smithi027.stdout:3/210: write d1/db/f12 [1896096,3320] 0 2021-11-03T17:14:12.183 INFO:tasks.workunit.client.0.smithi027.stdout:3/211: write d1/fa [5165659,57467] 0 2021-11-03T17:14:12.184 INFO:tasks.workunit.client.0.smithi027.stdout:3/212: rmdir d1 39 2021-11-03T17:14:12.186 INFO:tasks.workunit.client.0.smithi027.stdout:3/213: rename d1/db/f3b to d1/f41 0 2021-11-03T17:14:12.186 INFO:tasks.workunit.client.0.smithi027.stdout:3/214: readlink d1/l33 0 2021-11-03T17:14:12.195 INFO:tasks.workunit.client.0.smithi027.stdout:2/357: dwrite d0/d1b/d29/f4b [0,4194304] 0 2021-11-03T17:14:12.195 INFO:tasks.workunit.client.0.smithi027.stdout:2/358: dread - d0/d5/f6d zero size 2021-11-03T17:14:12.196 INFO:tasks.workunit.client.0.smithi027.stdout:2/359: dread - d0/d1b/d29/d58/d5b/f62 zero size 2021-11-03T17:14:12.214 INFO:tasks.workunit.client.0.smithi027.stdout:3/215: dwrite d1/fa [4194304,4194304] 0 2021-11-03T17:14:12.216 INFO:tasks.workunit.client.0.smithi027.stdout:2/360: dwrite d0/d2a/f44 [0,4194304] 0 2021-11-03T17:14:12.219 INFO:tasks.workunit.client.0.smithi027.stdout:2/361: symlink d0/d1b/d29/d54/d67/d6a/l73 0 2021-11-03T17:14:12.221 INFO:tasks.workunit.client.0.smithi027.stdout:2/362: getdents d0/d2a/d5d 0 2021-11-03T17:14:12.269 INFO:tasks.workunit.client.0.smithi027.stdout:2/363: dread d0/d1b/f27 [0,4194304] 0 2021-11-03T17:14:12.269 INFO:tasks.workunit.client.0.smithi027.stdout:2/364: rename d0 to d0/d5/d2c/d74 22 2021-11-03T17:14:12.273 INFO:tasks.workunit.client.0.smithi027.stdout:3/216: dwrite d1/f1d [0,4194304] 0 2021-11-03T17:14:12.274 INFO:tasks.workunit.client.0.smithi027.stdout:2/365: fdatasync d0/d1b/d29/f4e 0 2021-11-03T17:14:12.276 INFO:tasks.workunit.client.0.smithi027.stdout:2/366: rename d0/d1b/d1c/f24 to d0/d2a/d5d/f75 0 2021-11-03T17:14:12.276 INFO:tasks.workunit.client.0.smithi027.stdout:2/367: write d0/d2a/d5d/f65 [742735,45598] 0 2021-11-03T17:14:12.277 INFO:tasks.workunit.client.0.smithi027.stdout:2/368: chown d0/l28 1999796 1 2021-11-03T17:14:12.277 INFO:tasks.workunit.client.0.smithi027.stdout:3/217: dread d1/f22 [0,4194304] 0 2021-11-03T17:14:12.278 INFO:tasks.workunit.client.0.smithi027.stdout:3/218: dread - d1/f2a zero size 2021-11-03T17:14:12.300 INFO:tasks.workunit.client.0.smithi027.stdout:3/219: dwrite d1/f2e [0,4194304] 0 2021-11-03T17:14:12.300 INFO:tasks.workunit.client.0.smithi027.stdout:2/369: dread d0/d1b/d29/f4b [0,4194304] 0 2021-11-03T17:14:12.303 INFO:tasks.workunit.client.0.smithi027.stdout:2/370: unlink d0/d1b/d29/f4e 0 2021-11-03T17:14:12.304 INFO:tasks.workunit.client.0.smithi027.stdout:3/220: symlink d1/d40/l42 0 2021-11-03T17:14:12.305 INFO:tasks.workunit.client.0.smithi027.stdout:3/221: truncate d1/db/f32 518062 0 2021-11-03T17:14:12.305 INFO:tasks.workunit.client.0.smithi027.stdout:3/222: stat d1/f22 0 2021-11-03T17:14:12.305 INFO:tasks.workunit.client.0.smithi027.stdout:3/223: chown d1/db 3838454 1 2021-11-03T17:14:12.306 INFO:tasks.workunit.client.0.smithi027.stdout:3/224: truncate d1/f18 252512 0 2021-11-03T17:14:12.306 INFO:tasks.workunit.client.0.smithi027.stdout:3/225: readlink d1/l2b 0 2021-11-03T17:14:12.306 INFO:tasks.workunit.client.0.smithi027.stdout:3/226: symlink d1/l43 0 2021-11-03T17:14:12.307 INFO:tasks.workunit.client.0.smithi027.stdout:3/227: creat d1/d40/f44 x:0 0 0 2021-11-03T17:14:12.308 INFO:tasks.workunit.client.0.smithi027.stdout:3/228: mknod d1/d40/c45 1 2021-11-03T17:14:12.308 INFO:tasks.workunit.client.0.smithi027.stdout:3/229: creat d1/d40/f46 x:0 0 0 2021-11-03T17:14:12.309 INFO:tasks.workunit.client.0.smithi027.stdout:6/270: sync 2021-11-03T17:14:12.312 INFO:tasks.workunit.client.0.smithi027.stdout:2/371: dread d0/d1b/d29/f2b [0,4194304] 0 2021-11-03T17:14:12.314 INFO:tasks.workunit.client.0.smithi027.stdout:3/230: link d1/l4 d1/l47 0 2021-11-03T17:14:12.315 INFO:tasks.workunit.client.0.smithi027.stdout:3/231: dread d1/db/f32 [0,4194304] 0 2021-11-03T17:14:12.315 INFO:tasks.workunit.client.0.smithi027.stdout:3/232: chown d1/db/l24 4634 1 2021-11-03T17:14:12.315 INFO:tasks.workunit.client.0.smithi027.stdout:3/233: dread - d1/f34 zero size 2021-11-03T17:14:12.317 INFO:tasks.workunit.client.0.smithi027.stdout:6/271: creat dc/d16/d1b/d2d/d34/f4c x:0 0 0 2021-11-03T17:14:12.317 INFO:tasks.workunit.client.0.smithi027.stdout:6/272: fdatasync dc/d16/d1b/d2d/d34/f4c 0 2021-11-03T17:14:12.318 INFO:tasks.workunit.client.0.smithi027.stdout:2/372: readlink d0/d2a/d61/l63 0 2021-11-03T17:14:12.319 INFO:tasks.workunit.client.0.smithi027.stdout:3/234: creat d1/d40/f48 x:0 0 0 2021-11-03T17:14:12.321 INFO:tasks.workunit.client.0.smithi027.stdout:6/273: unlink dc/d16/d1b/d2d/d34/f39 0 2021-11-03T17:14:12.325 INFO:tasks.workunit.client.0.smithi027.stdout:3/235: rename d1/f2e to d1/d40/f49 0 2021-11-03T17:14:12.327 INFO:tasks.workunit.client.0.smithi027.stdout:2/373: dread d0/d1b/d29/f2f [0,4194304] 0 2021-11-03T17:14:12.327 INFO:tasks.workunit.client.0.smithi027.stdout:2/374: dread - d0/f46 zero size 2021-11-03T17:14:12.328 INFO:tasks.workunit.client.1.smithi115.stdout:6/207: sync 2021-11-03T17:14:12.328 INFO:tasks.workunit.client.1.smithi115.stdout:4/437: sync 2021-11-03T17:14:12.329 INFO:tasks.workunit.client.0.smithi027.stdout:6/274: link dc/d16/d1b/d24/f46 dc/f4d 0 2021-11-03T17:14:12.329 INFO:tasks.workunit.client.0.smithi027.stdout:6/275: truncate dc/d16/f40 838648 0 2021-11-03T17:14:12.330 INFO:tasks.workunit.client.0.smithi027.stdout:3/236: creat d1/d40/f4a x:0 0 0 2021-11-03T17:14:12.330 INFO:tasks.workunit.client.0.smithi027.stdout:3/237: dread - d1/d40/f44 zero size 2021-11-03T17:14:12.331 INFO:tasks.workunit.client.0.smithi027.stdout:3/238: chown d1 3435627 1 2021-11-03T17:14:12.332 INFO:tasks.workunit.client.1.smithi115.stdout:6/208: unlink d0/d1/d10/d13/d1c/d1e/f2b 0 2021-11-03T17:14:12.332 INFO:tasks.workunit.client.1.smithi115.stdout:6/209: mknod d0/d1/d10/c3e 1 2021-11-03T17:14:12.333 INFO:tasks.workunit.client.1.smithi115.stdout:6/210: mkdir d0/d1/d2/d3f 0 2021-11-03T17:14:12.335 INFO:tasks.workunit.client.1.smithi115.stdout:6/211: creat d0/f40 x:0 0 0 2021-11-03T17:14:12.335 INFO:tasks.workunit.client.1.smithi115.stdout:6/212: chown d0/f6 45368 1 2021-11-03T17:14:12.336 INFO:tasks.workunit.client.0.smithi027.stdout:6/276: write dc/f10 [3606959,98953] 0 2021-11-03T17:14:12.336 INFO:tasks.workunit.client.0.smithi027.stdout:6/277: fdatasync dc/d16/d1b/d2d/f35 0 2021-11-03T17:14:12.337 INFO:tasks.workunit.client.1.smithi115.stdout:6/213: unlink d0/f17 0 2021-11-03T17:14:12.337 INFO:tasks.workunit.client.0.smithi027.stdout:2/375: symlink d0/d45/l76 0 2021-11-03T17:14:12.337 INFO:tasks.workunit.client.0.smithi027.stdout:2/376: write d0/d5/d2c/f3d [1556863,99612] 0 2021-11-03T17:14:12.338 INFO:tasks.workunit.client.0.smithi027.stdout:2/377: readlink d0/l28 0 2021-11-03T17:14:12.338 INFO:tasks.workunit.client.1.smithi115.stdout:6/214: getdents d0/d1 0 2021-11-03T17:14:12.339 INFO:tasks.workunit.client.1.smithi115.stdout:6/215: rename d0/f15 to d0/f41 0 2021-11-03T17:14:12.340 INFO:tasks.workunit.client.1.smithi115.stdout:6/216: symlink d0/d1/d10/d13/d1c/d1e/l42 0 2021-11-03T17:14:12.341 INFO:tasks.workunit.client.0.smithi027.stdout:2/378: getdents d0/d1b/d1c 0 2021-11-03T17:14:12.342 INFO:tasks.workunit.client.0.smithi027.stdout:2/379: stat d0/d5/f4 0 2021-11-03T17:14:12.342 INFO:tasks.workunit.client.1.smithi115.stdout:4/438: dread d7/dc/d1a/d21/d47/d56/d63/f9b [0,4194304] 0 2021-11-03T17:14:12.344 INFO:tasks.workunit.client.1.smithi115.stdout:4/439: mkdir d7/dc/d1a/d21/d47/d56/d63/d9e 0 2021-11-03T17:14:12.345 INFO:tasks.workunit.client.0.smithi027.stdout:2/380: symlink d0/d1b/d29/d54/d67/d6a/l77 0 2021-11-03T17:14:12.350 INFO:tasks.workunit.client.0.smithi027.stdout:2/381: rmdir d0/d5/d2c 39 2021-11-03T17:14:12.354 INFO:tasks.workunit.client.0.smithi027.stdout:6/278: dread dc/d16/f2c [0,4194304] 0 2021-11-03T17:14:12.355 INFO:tasks.workunit.client.0.smithi027.stdout:2/382: link d0/f3b d0/d5/d2c/f78 0 2021-11-03T17:14:12.355 INFO:tasks.workunit.client.1.smithi115.stdout:6/217: dread d0/d1/d10/d13/d1c/f38 [0,4194304] 0 2021-11-03T17:14:12.356 INFO:tasks.workunit.client.1.smithi115.stdout:6/218: symlink d0/d1/d10/d13/d1c/d1e/l43 0 2021-11-03T17:14:12.356 INFO:tasks.workunit.client.1.smithi115.stdout:6/219: mknod d0/d1/d10/d13/c44 1 2021-11-03T17:14:12.357 INFO:tasks.workunit.client.0.smithi027.stdout:6/279: truncate dc/ff 2146738 0 2021-11-03T17:14:12.357 INFO:tasks.workunit.client.0.smithi027.stdout:6/280: stat dc/d16/d30/f32 0 2021-11-03T17:14:12.358 INFO:tasks.workunit.client.1.smithi115.stdout:6/220: rename d0/d1/d10/d13/d1c/d1e/l42 to d0/d1/l45 0 2021-11-03T17:14:12.358 INFO:tasks.workunit.client.1.smithi115.stdout:6/221: dread - d0/d1/f19 zero size 2021-11-03T17:14:12.359 INFO:tasks.workunit.client.1.smithi115.stdout:6/222: symlink d0/d1/d10/d13/d1c/l46 0 2021-11-03T17:14:12.359 INFO:tasks.workunit.client.1.smithi115.stdout:6/223: stat d0 0 2021-11-03T17:14:12.361 INFO:tasks.workunit.client.1.smithi115.stdout:6/224: unlink d0/d1/f5 0 2021-11-03T17:14:12.362 INFO:tasks.workunit.client.1.smithi115.stdout:6/225: creat d0/d1/d10/d13/d1c/d1e/f47 x:0 0 0 2021-11-03T17:14:12.363 INFO:tasks.workunit.client.0.smithi027.stdout:6/281: fdatasync dc/f10 0 2021-11-03T17:14:12.363 INFO:tasks.workunit.client.1.smithi115.stdout:4/440: dwrite d7/dc/d1a/d21/f22 [0,4194304] 0 2021-11-03T17:14:12.364 INFO:tasks.workunit.client.1.smithi115.stdout:4/441: chown d7/dc/d6a/f6f 1881 1 2021-11-03T17:14:12.371 INFO:tasks.workunit.client.0.smithi027.stdout:6/282: creat dc/d16/d1b/d2d/d34/f4e x:0 0 0 2021-11-03T17:14:12.374 INFO:tasks.workunit.client.0.smithi027.stdout:6/283: rename dc/d16/d1b/l22 to dc/d16/d1b/d3c/d48/l4f 0 2021-11-03T17:14:12.379 INFO:tasks.workunit.client.0.smithi027.stdout:6/284: getdents dc/d16/d1b 0 2021-11-03T17:14:12.382 INFO:tasks.workunit.client.0.smithi027.stdout:6/285: truncate dc/d16/d1b/f28 3588136 0 2021-11-03T17:14:12.383 INFO:tasks.workunit.client.1.smithi115.stdout:4/442: dwrite d7/dc/d6a/f46 [0,4194304] 0 2021-11-03T17:14:12.383 INFO:tasks.workunit.client.1.smithi115.stdout:4/443: chown d7/dc/d1a/d21/d47/d56/d63/f9b 124 1 2021-11-03T17:14:12.383 INFO:tasks.workunit.client.1.smithi115.stdout:4/444: fsync d7/dc/d1a/d21/d47/d56/f93 0 2021-11-03T17:14:12.384 INFO:tasks.workunit.client.0.smithi027.stdout:6/286: symlink dc/d16/d1b/d2d/l50 0 2021-11-03T17:14:12.384 INFO:tasks.workunit.client.0.smithi027.stdout:6/287: chown dc/d16/d1b/d3c/f44 2153282 1 2021-11-03T17:14:12.385 INFO:tasks.workunit.client.0.smithi027.stdout:6/288: mkdir dc/d16/d1b/d3c/d51 0 2021-11-03T17:14:12.385 INFO:tasks.workunit.client.0.smithi027.stdout:6/289: chown dc/d16/d30/f32 12 1 2021-11-03T17:14:12.385 INFO:tasks.workunit.client.0.smithi027.stdout:6/290: chown dc/d16/d1b/d24/f2a 516570 1 2021-11-03T17:14:12.386 INFO:tasks.workunit.client.0.smithi027.stdout:6/291: write dc/f18 [1433515,128483] 0 2021-11-03T17:14:12.386 INFO:tasks.workunit.client.0.smithi027.stdout:6/292: write dc/d37/f47 [1041222,7577] 0 2021-11-03T17:14:12.387 INFO:tasks.workunit.client.0.smithi027.stdout:6/293: write dc/f1e [3171613,35266] 0 2021-11-03T17:14:12.389 INFO:tasks.workunit.client.0.smithi027.stdout:6/294: mkdir dc/d16/d1b/d2d/d52 0 2021-11-03T17:14:12.389 INFO:tasks.workunit.client.0.smithi027.stdout:6/295: readlink dc/d16/l3d 0 2021-11-03T17:14:12.392 INFO:tasks.workunit.client.1.smithi115.stdout:4/445: rmdir d7/dc/d1a/d21/d47/d56/d42 39 2021-11-03T17:14:12.415 INFO:tasks.workunit.client.1.smithi115.stdout:4/446: dwrite d7/dc/d1a/f49 [0,4194304] 0 2021-11-03T17:14:12.415 INFO:tasks.workunit.client.1.smithi115.stdout:4/447: write d7/dc/d6a/d3c/f3e [2183587,129858] 0 2021-11-03T17:14:12.416 INFO:tasks.workunit.client.0.smithi027.stdout:6/296: dwrite dc/d16/d1b/d24/f46 [0,4194304] 0 2021-11-03T17:14:12.416 INFO:tasks.workunit.client.0.smithi027.stdout:6/297: truncate dc/d16/d1b/d2d/d34/f4c 211990 0 2021-11-03T17:14:12.416 INFO:tasks.workunit.client.0.smithi027.stdout:6/298: fdatasync dc/d16/d1b/d24/f46 0 2021-11-03T17:14:12.417 INFO:tasks.workunit.client.0.smithi027.stdout:6/299: write dc/f1e [4945757,53184] 0 2021-11-03T17:14:12.417 INFO:tasks.workunit.client.0.smithi027.stdout:6/300: mknod dc/d16/d1b/d3c/c53 1 2021-11-03T17:14:12.418 INFO:tasks.workunit.client.1.smithi115.stdout:4/448: read d7/dc/d1a/d21/d47/d56/f98 [2468795,13385] 0 2021-11-03T17:14:12.418 INFO:tasks.workunit.client.1.smithi115.stdout:4/449: mknod d7/dc/d6a/d72/c9f 1 2021-11-03T17:14:12.418 INFO:tasks.workunit.client.1.smithi115.stdout:4/450: chown d7/dc 53 1 2021-11-03T17:14:12.419 INFO:tasks.workunit.client.1.smithi115.stdout:4/451: creat d7/fa0 x:0 0 0 2021-11-03T17:14:12.430 INFO:tasks.workunit.client.0.smithi027.stdout:6/301: stat dc/d16/d1b/l21 0 2021-11-03T17:14:12.434 INFO:tasks.workunit.client.0.smithi027.stdout:6/302: write dc/f1f [3213331,44653] 0 2021-11-03T17:14:12.434 INFO:tasks.workunit.client.0.smithi027.stdout:6/303: mknod dc/c54 1 2021-11-03T17:14:12.434 INFO:tasks.workunit.client.0.smithi027.stdout:6/304: dread - dc/d16/d1b/d2d/d34/f4e zero size 2021-11-03T17:14:12.434 INFO:tasks.workunit.client.0.smithi027.stdout:6/305: stat dc/f1e 0 2021-11-03T17:14:12.435 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.441+0000 7f77b6648700 1 -- 172.21.15.27:0/147811846 --> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f77b00ff080 con 0x7f779805c2b0 2021-11-03T17:14:12.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.442+0000 7f77acff9700 1 -- 172.21.15.27:0/147811846 <== mgr.14162 v2:172.21.15.27:6800/3802716887 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+274 (secure 0 0 0) 0x7f77b00ff080 con 0x7f779805c2b0 2021-11-03T17:14:12.438 INFO:tasks.workunit.client.0.smithi027.stdout:6/306: dread dc/f10 [0,4194304] 0 2021-11-03T17:14:12.439 INFO:tasks.workunit.client.0.smithi027.stdout:6/307: chown dc/f10 21 1 2021-11-03T17:14:12.439 INFO:tasks.workunit.client.0.smithi027.stdout:6/308: fsync dc/d16/d30/f32 0 2021-11-03T17:14:12.439 INFO:tasks.workunit.client.0.smithi027.stdout:6/309: mknod dc/d16/c55 1 2021-11-03T17:14:12.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.444+0000 7f77b6648700 1 -- 172.21.15.27:0/147811846 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f779805c2b0 msgr2=0x7f779805e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:14:12.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.444+0000 7f77b6648700 1 --2- 172.21.15.27:0/147811846 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f779805c2b0 0x7f779805e770 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f77a4004e30 tx=0x7f77a4009b30).stop 2021-11-03T17:14:12.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.444+0000 7f77b6648700 1 -- 172.21.15.27:0/147811846 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77b0060250 msgr2=0x7f77b0106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:14:12.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.444+0000 7f77b6648700 1 --2- 172.21.15.27:0/147811846 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77b0060250 0x7f77b0106f70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f77a000b410 tx=0x7f77a0006ef0).stop 2021-11-03T17:14:12.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.444+0000 7f77b6648700 1 -- 172.21.15.27:0/147811846 shutdown_connections 2021-11-03T17:14:12.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.444+0000 7f77b6648700 1 --2- 172.21.15.27:0/147811846 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f779805c2b0 0x7f779805e770 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:12.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.444+0000 7f77b6648700 1 --2- 172.21.15.27:0/147811846 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77b0060250 0x7f77b0106f70 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:12.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.444+0000 7f77b6648700 1 --2- 172.21.15.27:0/147811846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77b00f9850 0x7f77b0107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:14:12.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.444+0000 7f77b6648700 1 -- 172.21.15.27:0/147811846 >> 172.21.15.27:0/147811846 conn(0x7f77b00f0d40 msgr2=0x7f77b00f3130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:14:12.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.444+0000 7f77b6648700 1 -- 172.21.15.27:0/147811846 shutdown_connections 2021-11-03T17:14:12.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.444+0000 7f77b6648700 1 -- 172.21.15.27:0/147811846 wait complete. 2021-11-03T17:14:12.443 INFO:tasks.workunit.client.0.smithi027.stdout:6/310: dread dc/ff [0,4194304] 0 2021-11-03T17:14:12.444 INFO:tasks.workunit.client.1.smithi115.stdout:4/452: dwrite d7/dc/f17 [4194304,4194304] 0 2021-11-03T17:14:12.444 INFO:tasks.workunit.client.1.smithi115.stdout:4/453: creat d7/dc/d1a/d21/fa1 x:0 0 0 2021-11-03T17:14:12.444 INFO:tasks.workunit.client.1.smithi115.stdout:4/454: fdatasync d7/dc/d6a/f3a 0 2021-11-03T17:14:12.445 INFO:tasks.workunit.client.1.smithi115.stdout:4/455: creat d7/dc/d1a/fa2 x:0 0 0 2021-11-03T17:14:12.446 INFO:tasks.workunit.client.0.smithi027.stdout:6/311: creat dc/d16/d1b/d3c/d51/f56 x:0 0 0 2021-11-03T17:14:12.446 INFO:tasks.workunit.client.0.smithi027.stdout:6/312: mkdir dc/d16/d1b/d3c/d48/d57 0 2021-11-03T17:14:12.446 INFO:tasks.workunit.client.1.smithi115.stdout:4/456: rmdir d7/dc/d1a/d21/d47/d56 39 2021-11-03T17:14:12.447 INFO:tasks.workunit.client.1.smithi115.stdout:4/457: dread - d7/dc/d1a/d21/d47/d56/d63/f85 zero size 2021-11-03T17:14:12.447 INFO:tasks.workunit.client.1.smithi115.stdout:4/458: chown d7/dc/d1a/d21/d47/d56/l78 5699 1 2021-11-03T17:14:12.447 INFO:tasks.workunit.client.1.smithi115.stdout:4/459: write d7/dc/d1a/d21/d47/d56/f93 [74555,30621] 0 2021-11-03T17:14:12.448 INFO:tasks.workunit.client.0.smithi027.stdout:6/313: rmdir dc/d16/d1b/d2d/d52 0 2021-11-03T17:14:12.448 INFO:tasks.workunit.client.1.smithi115.stdout:4/460: creat d7/d7a/fa3 x:0 0 0 2021-11-03T17:14:12.448 INFO:tasks.workunit.client.1.smithi115.stdout:4/461: creat d7/dc/fa4 x:0 0 0 2021-11-03T17:14:12.449 INFO:tasks.workunit.client.1.smithi115.stdout:4/462: creat d7/dc/d1a/d21/d47/d56/d42/fa5 x:0 0 0 2021-11-03T17:14:12.450 INFO:tasks.workunit.client.1.smithi115.stdout:4/463: write d7/dc/d1a/f38 [2133844,66640] 0 2021-11-03T17:14:12.450 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:14:12.459 INFO:tasks.workunit.client.1.smithi115.stdout:8/274: sync 2021-11-03T17:14:12.459 INFO:tasks.workunit.client.1.smithi115.stdout:9/171: sync 2021-11-03T17:14:12.459 INFO:tasks.workunit.client.1.smithi115.stdout:0/370: sync 2021-11-03T17:14:12.459 INFO:tasks.workunit.client.1.smithi115.stdout:3/332: sync 2021-11-03T17:14:12.460 INFO:tasks.workunit.client.1.smithi115.stdout:3/333: read f12 [2263307,74436] 0 2021-11-03T17:14:12.461 INFO:tasks.workunit.client.1.smithi115.stdout:3/334: fsync d13/d51/f29 0 2021-11-03T17:14:12.461 INFO:tasks.workunit.client.1.smithi115.stdout:9/172: creat d1/d18/d2b/f35 x:0 0 0 2021-11-03T17:14:12.462 INFO:tasks.workunit.client.1.smithi115.stdout:8/275: link d4/d6/d18/d44/f48 d4/d6/d18/d44/f62 0 2021-11-03T17:14:12.462 INFO:tasks.workunit.client.1.smithi115.stdout:8/276: write d4/d6/d18/d30/f35 [2710613,27717] 0 2021-11-03T17:14:12.463 INFO:tasks.workunit.client.1.smithi115.stdout:3/335: symlink d13/d2a/d53/d70/l73 0 2021-11-03T17:14:12.464 INFO:tasks.workunit.client.1.smithi115.stdout:9/173: creat d1/d2d/f36 x:0 0 0 2021-11-03T17:14:12.465 INFO:tasks.workunit.client.1.smithi115.stdout:8/277: unlink d4/d6/d3f/d43/f5e 0 2021-11-03T17:14:12.467 INFO:tasks.workunit.client.1.smithi115.stdout:9/174: symlink d1/d5/l37 0 2021-11-03T17:14:12.467 INFO:tasks.workunit.client.1.smithi115.stdout:9/175: truncate d1/d2d/f26 428008 0 2021-11-03T17:14:12.468 INFO:tasks.workunit.client.1.smithi115.stdout:9/176: getdents d1/d1a 0 2021-11-03T17:14:12.473 INFO:tasks.workunit.client.1.smithi115.stdout:9/177: rename d1/d5/l2c to d1/d1a/l38 0 2021-11-03T17:14:12.475 INFO:tasks.workunit.client.1.smithi115.stdout:9/178: mkdir d1/d39 0 2021-11-03T17:14:12.476 INFO:tasks.workunit.client.1.smithi115.stdout:9/179: stat d1/d5/l2e 0 2021-11-03T17:14:12.477 INFO:tasks.workunit.client.0.smithi027.stdout:6/314: dwrite dc/d16/d1b/d3c/d51/f56 [0,4194304] 0 2021-11-03T17:14:12.477 INFO:tasks.workunit.client.1.smithi115.stdout:9/180: mkdir d1/d32/d3a 0 2021-11-03T17:14:12.480 INFO:tasks.workunit.client.0.smithi027.stdout:6/315: creat dc/d16/d1b/d3c/d51/f58 x:0 0 0 2021-11-03T17:14:12.483 INFO:tasks.workunit.client.0.smithi027.stdout:6/316: dread dc/ff [0,4194304] 0 2021-11-03T17:14:12.484 INFO:tasks.workunit.client.1.smithi115.stdout:9/181: dread d1/d5/f17 [0,4194304] 0 2021-11-03T17:14:12.485 INFO:tasks.workunit.client.1.smithi115.stdout:9/182: creat d1/d32/d3a/f3b x:0 0 0 2021-11-03T17:14:12.485 INFO:tasks.workunit.client.1.smithi115.stdout:9/183: mknod d1/d32/c3c 1 2021-11-03T17:14:12.486 INFO:tasks.workunit.client.1.smithi115.stdout:9/184: creat d1/d2d/d27/f3d x:0 0 0 2021-11-03T17:14:12.487 INFO:tasks.workunit.client.1.smithi115.stdout:9/185: unlink d1/d2d/f36 0 2021-11-03T17:14:12.488 INFO:tasks.workunit.client.1.smithi115.stdout:9/186: fdatasync d1/d5/fc 0 2021-11-03T17:14:12.488 INFO:tasks.workunit.client.1.smithi115.stdout:9/187: mknod d1/d1a/c3e 1 2021-11-03T17:14:12.489 INFO:tasks.workunit.client.1.smithi115.stdout:4/464: dwrite d7/dc/f12 [0,4194304] 0 2021-11-03T17:14:12.489 INFO:tasks.workunit.client.1.smithi115.stdout:0/371: dwrite d9/db/f4f [0,4194304] 0 2021-11-03T17:14:12.490 INFO:tasks.workunit.client.1.smithi115.stdout:4/465: read d7/dc/f12 [3252979,22205] 0 2021-11-03T17:14:12.490 INFO:tasks.workunit.client.1.smithi115.stdout:9/188: symlink d1/d5/l3f 0 2021-11-03T17:14:12.493 INFO:tasks.workunit.client.1.smithi115.stdout:0/372: creat d9/db/df/f79 x:0 0 0 2021-11-03T17:14:12.497 INFO:tasks.workunit.client.1.smithi115.stdout:0/373: write d9/d15/d1a/f3d [4192147,94000] 0 2021-11-03T17:14:12.497 INFO:tasks.workunit.client.1.smithi115.stdout:8/278: dwrite d4/d6/f2b [0,4194304] 0 2021-11-03T17:14:12.498 INFO:tasks.workunit.client.1.smithi115.stdout:9/189: dread d1/d5/f17 [0,4194304] 0 2021-11-03T17:14:12.498 INFO:tasks.workunit.client.1.smithi115.stdout:9/190: dread - d1/d2d/f2a zero size 2021-11-03T17:14:12.498 INFO:tasks.workunit.client.1.smithi115.stdout:3/336: dwrite d13/f27 [0,4194304] 0 2021-11-03T17:14:12.500 INFO:tasks.workunit.client.1.smithi115.stdout:8/279: truncate d4/d6/d18/d30/f31 2912515 0 2021-11-03T17:14:12.501 INFO:tasks.workunit.client.1.smithi115.stdout:9/191: symlink d1/d2d/d27/l40 0 2021-11-03T17:14:12.502 INFO:tasks.workunit.client.1.smithi115.stdout:3/337: truncate d13/d67/f6c 446302 0 2021-11-03T17:14:12.503 INFO:tasks.workunit.client.1.smithi115.stdout:0/374: truncate d9/d15/d1a/d1d/f22 2918184 0 2021-11-03T17:14:12.505 INFO:tasks.workunit.client.1.smithi115.stdout:8/280: rename d4/f27 to d4/d6/d18/f63 0 2021-11-03T17:14:12.506 INFO:tasks.workunit.client.1.smithi115.stdout:9/192: getdents d1/d2d 0 2021-11-03T17:14:12.507 INFO:tasks.workunit.client.0.smithi027.stdout:6/317: dwrite dc/d16/f40 [0,4194304] 0 2021-11-03T17:14:12.508 INFO:tasks.workunit.client.1.smithi115.stdout:0/375: symlink d9/d15/d2c/d46/d63/d70/l7a 0 2021-11-03T17:14:12.509 INFO:tasks.workunit.client.1.smithi115.stdout:3/338: rename d13/d51/d2e/l38 to d13/d35/l74 0 2021-11-03T17:14:12.510 INFO:tasks.workunit.client.1.smithi115.stdout:3/339: mknod d13/d51/c75 1 2021-11-03T17:14:12.510 INFO:tasks.workunit.client.1.smithi115.stdout:3/340: fdatasync d13/d51/d40/f4f 0 2021-11-03T17:14:12.512 INFO:tasks.workunit.client.0.smithi027.stdout:6/318: symlink dc/d16/d1b/d3c/d48/d57/l59 0 2021-11-03T17:14:12.512 INFO:tasks.workunit.client.0.smithi027.stdout:6/319: chown dc/d16/d1b/l2e 6 1 2021-11-03T17:14:12.513 INFO:tasks.workunit.client.1.smithi115.stdout:9/193: mkdir d1/d2d/d27/d2f/d41 0 2021-11-03T17:14:12.513 INFO:tasks.workunit.client.1.smithi115.stdout:4/466: dwrite d7/dc/d1a/d21/d47/d56/f98 [4194304,4194304] 0 2021-11-03T17:14:12.513 INFO:tasks.workunit.client.1.smithi115.stdout:4/467: readlink d7/dc/d1a/d21/d37/l65 0 2021-11-03T17:14:12.517 INFO:tasks.workunit.client.1.smithi115.stdout:0/376: rename d9/f43 to d9/d15/d2c/d46/d63/f7b 0 2021-11-03T17:14:12.521 INFO:tasks.workunit.client.1.smithi115.stdout:3/341: unlink d13/d35/l6a 0 2021-11-03T17:14:12.522 INFO:tasks.workunit.client.1.smithi115.stdout:8/281: write d4/d6/d18/d30/f31 [662845,67552] 0 2021-11-03T17:14:12.522 INFO:tasks.workunit.client.1.smithi115.stdout:4/468: creat d7/dc/d1a/d21/d47/d56/d63/fa6 x:0 0 0 2021-11-03T17:14:12.523 INFO:tasks.workunit.client.1.smithi115.stdout:0/377: creat d9/db/df/f7c x:0 0 0 2021-11-03T17:14:12.525 INFO:tasks.workunit.client.1.smithi115.stdout:3/342: rename d13/f1d to d13/d3b/f76 0 2021-11-03T17:14:12.526 INFO:tasks.workunit.client.1.smithi115.stdout:4/469: creat d7/dc/d1a/d21/d47/d56/d63/d7b/d96/fa7 x:0 0 0 2021-11-03T17:14:12.527 INFO:tasks.workunit.client.1.smithi115.stdout:4/470: dread - d7/dc/d1a/d21/fa1 zero size 2021-11-03T17:14:12.527 INFO:tasks.workunit.client.1.smithi115.stdout:0/378: rmdir d9/d15/d2c/d46 39 2021-11-03T17:14:12.529 INFO:tasks.workunit.client.1.smithi115.stdout:4/471: creat d7/dc/d6a/d3c/fa8 x:0 0 0 2021-11-03T17:14:12.531 INFO:tasks.workunit.client.1.smithi115.stdout:4/472: creat d7/dc/d1a/d21/d47/d56/d63/d9e/fa9 x:0 0 0 2021-11-03T17:14:12.539 INFO:tasks.workunit.client.0.smithi027.stdout:6/320: dwrite dc/d16/f2c [0,4194304] 0 2021-11-03T17:14:12.540 INFO:tasks.workunit.client.1.smithi115.stdout:9/194: dwrite d1/f25 [0,4194304] 0 2021-11-03T17:14:12.546 INFO:tasks.workunit.client.1.smithi115.stdout:3/343: write d13/d67/f6c [881638,11843] 0 2021-11-03T17:14:12.546 INFO:tasks.workunit.client.1.smithi115.stdout:3/344: fsync d13/d2a/f5d 0 2021-11-03T17:14:12.550 INFO:tasks.workunit.client.1.smithi115.stdout:9/195: getdents d1/d1a 0 2021-11-03T17:14:12.550 INFO:tasks.workunit.client.1.smithi115.stdout:9/196: truncate d1/d18/f22 914180 0 2021-11-03T17:14:12.550 INFO:tasks.workunit.client.1.smithi115.stdout:9/197: truncate d1/d18/f1e 763538 0 2021-11-03T17:14:12.551 INFO:tasks.workunit.client.1.smithi115.stdout:3/345: rename f9 to d13/d2a/d60/f77 0 2021-11-03T17:14:12.553 INFO:tasks.workunit.client.1.smithi115.stdout:9/198: rename d1/d5 to d1/d18/d2b/d42 0 2021-11-03T17:14:12.555 INFO:tasks.workunit.client.1.smithi115.stdout:9/199: rmdir d1/d32/d3a 39 2021-11-03T17:14:12.555 INFO:tasks.workunit.client.1.smithi115.stdout:9/200: chown d1/d2d/d27/l40 11 1 2021-11-03T17:14:12.558 INFO:tasks.workunit.client.1.smithi115.stdout:0/379: dwrite d9/d15/d1a/d1d/f22 [0,4194304] 0 2021-11-03T17:14:12.559 INFO:tasks.workunit.client.1.smithi115.stdout:9/201: dread d1/d18/d2b/d42/fc [0,4194304] 0 2021-11-03T17:14:12.559 INFO:tasks.workunit.client.1.smithi115.stdout:9/202: mknod d1/d2d/c43 1 2021-11-03T17:14:12.560 INFO:tasks.workunit.client.0.smithi027.stdout:6/321: dwrite dc/f15 [0,4194304] 0 2021-11-03T17:14:12.560 INFO:tasks.workunit.client.1.smithi115.stdout:4/473: dwrite d7/dc/f11 [0,4194304] 0 2021-11-03T17:14:12.560 INFO:tasks.workunit.client.1.smithi115.stdout:4/474: truncate d7/dc/d6a/d3c/f3e 2431624 0 2021-11-03T17:14:12.561 INFO:tasks.workunit.client.1.smithi115.stdout:9/203: write d1/f1d [41286,22463] 0 2021-11-03T17:14:12.561 INFO:tasks.workunit.client.1.smithi115.stdout:0/380: unlink d9/d15/d1a/f77 0 2021-11-03T17:14:12.561 INFO:tasks.workunit.client.1.smithi115.stdout:0/381: fsync d9/db/f58 0 2021-11-03T17:14:12.574 INFO:tasks.workunit.client.1.smithi115.stdout:3/346: write d13/d2a/d60/f77 [3234910,9219] 0 2021-11-03T17:14:12.579 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.584+0000 7fcb0d714700 1 -- 172.21.15.27:0/2110231018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb080f9690 msgr2=0x7fcb080fbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:14:12.579 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.584+0000 7fcb0d714700 1 --2- 172.21.15.27:0/2110231018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb080f9690 0x7fcb080fbae0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7fcafc004660 tx=0x7fcafc009b30).stop 2021-11-03T17:14:12.579 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.584+0000 7fcb0d714700 1 -- 172.21.15.27:0/2110231018 shutdown_connections 2021-11-03T17:14:12.579 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.584+0000 7fcb0d714700 1 --2- 172.21.15.27:0/2110231018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb080f9690 0x7fcb080fbae0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:12.580 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.584+0000 7fcb0d714700 1 --2- 172.21.15.27:0/2110231018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb08060310 0x7fcb080f9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:12.580 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.584+0000 7fcb0d714700 1 -- 172.21.15.27:0/2110231018 >> 172.21.15.27:0/2110231018 conn(0x7fcb080f0b90 msgr2=0x7fcb080f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:14:12.580 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.584+0000 7fcb0d714700 1 -- 172.21.15.27:0/2110231018 shutdown_connections 2021-11-03T17:14:12.580 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.584+0000 7fcb0d714700 1 -- 172.21.15.27:0/2110231018 wait complete. 2021-11-03T17:14:12.581 INFO:tasks.workunit.client.1.smithi115.stdout:3/347: dread d13/d51/d2e/f32 [0,4194304] 0 2021-11-03T17:14:12.584 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.584+0000 7fcb0d714700 1 Processor -- start 2021-11-03T17:14:12.584 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.585+0000 7fcb0d714700 1 -- start start 2021-11-03T17:14:12.585 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.585+0000 7fcb0d714700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb08060310 0x7fcb080f6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:14:12.585 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.585+0000 7fcb0d714700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb080f9690 0x7fcb080f5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:14:12.585 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.585+0000 7fcb0d714700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb080f5570 con 0x7fcb08060310 2021-11-03T17:14:12.585 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.585+0000 7fcb0d714700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb080f56b0 con 0x7fcb080f9690 2021-11-03T17:14:12.585 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.585+0000 7fcb07fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb08060310 0x7fcb080f6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:14:12.586 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.585+0000 7fcb07fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb08060310 0x7fcb080f6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55318/0 (socket says 172.21.15.27:55318) 2021-11-03T17:14:12.586 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.585+0000 7fcb07fff700 1 -- 172.21.15.27:0/3800661622 learned_addr learned my addr 172.21.15.27:0/3800661622 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:14:12.586 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.585+0000 7fcb077fe700 1 --2- 172.21.15.27:0/3800661622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb080f9690 0x7fcb080f5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:14:12.586 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.585+0000 7fcb07fff700 1 -- 172.21.15.27:0/3800661622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb080f9690 msgr2=0x7fcb080f5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:14:12.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.585+0000 7fcb07fff700 1 --2- 172.21.15.27:0/3800661622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb080f9690 0x7fcb080f5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:12.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.585+0000 7fcb07fff700 1 -- 172.21.15.27:0/3800661622 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcafc005040 con 0x7fcb08060310 2021-11-03T17:14:12.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.585+0000 7fcb07fff700 1 --2- 172.21.15.27:0/3800661622 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb08060310 0x7fcb080f6b90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7fcaf80026d0 tx=0x7fcaf8009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:14:12.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.586+0000 7fcb04ff9700 1 -- 172.21.15.27:0/3800661622 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcaf80079c0 con 0x7fcb08060310 2021-11-03T17:14:12.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.586+0000 7fcb0d714700 1 -- 172.21.15.27:0/3800661622 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb080f5990 con 0x7fcb08060310 2021-11-03T17:14:12.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.586+0000 7fcb04ff9700 1 -- 172.21.15.27:0/3800661622 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fcaf800f040 con 0x7fcb08060310 2021-11-03T17:14:12.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.586+0000 7fcb04ff9700 1 -- 172.21.15.27:0/3800661622 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcaf80148b0 con 0x7fcb08060310 2021-11-03T17:14:12.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.586+0000 7fcb0d714700 1 -- 172.21.15.27:0/3800661622 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb0810ed80 con 0x7fcb08060310 2021-11-03T17:14:12.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.587+0000 7fcb0d714700 1 -- 172.21.15.27:0/3800661622 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb08045ad0 con 0x7fcb08060310 2021-11-03T17:14:12.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.587+0000 7fcb04ff9700 1 -- 172.21.15.27:0/3800661622 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7fcaf8007b20 con 0x7fcb08060310 2021-11-03T17:14:12.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.587+0000 7fcb04ff9700 1 --2- 172.21.15.27:0/3800661622 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fcaf005c300 0x7fcaf005e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:14:12.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.587+0000 7fcb04ff9700 1 -- 172.21.15.27:0/3800661622 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5349+0+0 (secure 0 0 0) 0x7fcaf8075ec0 con 0x7fcb08060310 2021-11-03T17:14:12.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.588+0000 7fcb077fe700 1 --2- 172.21.15.27:0/3800661622 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fcaf005c300 0x7fcaf005e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:14:12.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.588+0000 7fcb077fe700 1 --2- 172.21.15.27:0/3800661622 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fcaf005c300 0x7fcaf005e7c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fcafc004630 tx=0x7fcafc009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:14:12.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.592+0000 7fcb04ff9700 1 -- 172.21.15.27:0/3800661622 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7fcaf8048ba0 con 0x7fcb08060310 2021-11-03T17:14:12.591 INFO:tasks.workunit.client.1.smithi115.stdout:3/348: rename l8 to d13/d2a/d60/l78 0 2021-11-03T17:14:12.591 INFO:tasks.workunit.client.1.smithi115.stdout:3/349: stat d13/d35/f3a 0 2021-11-03T17:14:12.600 INFO:tasks.workunit.client.1.smithi115.stdout:4/475: dwrite d7/dc/d1a/f1c [4194304,4194304] 0 2021-11-03T17:14:12.601 INFO:tasks.workunit.client.1.smithi115.stdout:4/476: mknod d7/caa 1 2021-11-03T17:14:12.601 INFO:tasks.workunit.client.1.smithi115.stdout:4/477: truncate d7/dc/d6a/d72/f76 2837382 0 2021-11-03T17:14:12.602 INFO:tasks.workunit.client.1.smithi115.stdout:4/478: mkdir d7/dc/d6a/dab 0 2021-11-03T17:14:12.603 INFO:tasks.workunit.client.1.smithi115.stdout:4/479: read d7/dc/f11 [3544277,8337] 0 2021-11-03T17:14:12.610 INFO:tasks.workunit.client.1.smithi115.stdout:9/204: dwrite d1/d18/d2b/d42/f12 [0,4194304] 0 2021-11-03T17:14:12.611 INFO:tasks.workunit.client.0.smithi027.stdout:6/322: dwrite dc/d16/d1b/d2d/d34/f4c [0,4194304] 0 2021-11-03T17:14:12.611 INFO:tasks.workunit.client.0.smithi027.stdout:6/323: readlink dc/d16/d1b/d3c/l3f 0 2021-11-03T17:14:12.611 INFO:tasks.workunit.client.0.smithi027.stdout:6/324: fsync dc/d16/d1b/d3c/d51/f56 0 2021-11-03T17:14:12.612 INFO:tasks.workunit.client.1.smithi115.stdout:3/350: dwrite d13/f33 [0,4194304] 0 2021-11-03T17:14:12.612 INFO:tasks.workunit.client.0.smithi027.stdout:6/325: write dc/d16/d1b/d24/f2a [112286,68530] 0 2021-11-03T17:14:12.615 INFO:tasks.workunit.client.0.smithi027.stdout:6/326: creat dc/d16/d30/f5a x:0 0 0 2021-11-03T17:14:12.615 INFO:tasks.workunit.client.0.smithi027.stdout:6/327: chown dc/d37/f47 116933 1 2021-11-03T17:14:12.618 INFO:tasks.workunit.client.1.smithi115.stdout:3/351: dread d13/f26 [0,4194304] 0 2021-11-03T17:14:12.619 INFO:tasks.workunit.client.1.smithi115.stdout:3/352: mknod d13/d51/c79 1 2021-11-03T17:14:12.622 INFO:tasks.workunit.client.1.smithi115.stdout:9/205: dread d1/d18/d2b/d42/f12 [0,4194304] 0 2021-11-03T17:14:12.625 INFO:tasks.workunit.client.1.smithi115.stdout:3/353: creat d13/d3b/f7a x:0 0 0 2021-11-03T17:14:12.626 INFO:tasks.workunit.client.1.smithi115.stdout:9/206: mknod d1/c44 1 2021-11-03T17:14:12.627 INFO:tasks.workunit.client.1.smithi115.stdout:3/354: symlink d13/d51/l7b 0 2021-11-03T17:14:12.629 INFO:tasks.workunit.client.1.smithi115.stdout:9/207: write d1/d1a/f28 [354990,68745] 0 2021-11-03T17:14:12.629 INFO:tasks.workunit.client.1.smithi115.stdout:9/208: write d1/d18/d2b/d42/f21 [877836,45940] 0 2021-11-03T17:14:12.630 INFO:tasks.workunit.client.1.smithi115.stdout:9/209: write d1/f1d [1130483,126883] 0 2021-11-03T17:14:12.632 INFO:tasks.workunit.client.1.smithi115.stdout:9/210: truncate d1/d18/d2b/d42/f12 1456742 0 2021-11-03T17:14:12.632 INFO:tasks.workunit.client.1.smithi115.stdout:9/211: write d1/d18/d2b/d42/f21 [1863099,24916] 0 2021-11-03T17:14:12.635 INFO:tasks.workunit.client.1.smithi115.stdout:9/212: mkdir d1/d18/d2b/d45 0 2021-11-03T17:14:12.636 INFO:tasks.workunit.client.1.smithi115.stdout:9/213: mkdir d1/d2d/d46 0 2021-11-03T17:14:12.638 INFO:tasks.workunit.client.1.smithi115.stdout:4/480: dread d7/dc/d1a/d21/d47/d56/d63/f67 [0,4194304] 0 2021-11-03T17:14:12.639 INFO:tasks.workunit.client.1.smithi115.stdout:9/214: write d1/d18/f1c [911994,118539] 0 2021-11-03T17:14:12.640 INFO:tasks.workunit.client.1.smithi115.stdout:4/481: truncate d7/dc/d6a/d72/f76 512185 0 2021-11-03T17:14:12.641 INFO:tasks.workunit.client.0.smithi027.stdout:6/328: dwrite dc/d16/d1b/d3c/d51/f56 [0,4194304] 0 2021-11-03T17:14:12.641 INFO:tasks.workunit.client.0.smithi027.stdout:6/329: stat dc/f10 0 2021-11-03T17:14:12.642 INFO:tasks.workunit.client.1.smithi115.stdout:9/215: rename d1/d2d/f23 to d1/d2d/d27/f47 0 2021-11-03T17:14:12.656 INFO:tasks.workunit.client.1.smithi115.stdout:3/355: dwrite d13/f1c [0,4194304] 0 2021-11-03T17:14:12.658 INFO:tasks.workunit.client.0.smithi027.stdout:6/330: dwrite dc/d16/d30/f32 [0,4194304] 0 2021-11-03T17:14:12.660 INFO:tasks.workunit.client.0.smithi027.stdout:6/331: unlink dc/f15 0 2021-11-03T17:14:12.661 INFO:tasks.workunit.client.0.smithi027.stdout:6/332: creat dc/d16/d1b/d3c/d51/f5b x:0 0 0 2021-11-03T17:14:12.670 INFO:tasks.workunit.client.1.smithi115.stdout:4/482: dwrite d7/dc/d1a/d21/d47/d56/d63/f87 [0,4194304] 0 2021-11-03T17:14:12.670 INFO:tasks.workunit.client.1.smithi115.stdout:4/483: chown d7/dc/d1a/d21/d47/d56/d63/d7b/f7e 25 1 2021-11-03T17:14:12.671 INFO:tasks.workunit.client.0.smithi027.stdout:8/368: sync 2021-11-03T17:14:12.672 INFO:tasks.workunit.client.0.smithi027.stdout:4/287: sync 2021-11-03T17:14:12.673 INFO:tasks.workunit.client.1.smithi115.stdout:4/484: unlink d7/dc/d1a/d21/d47/l53 0 2021-11-03T17:14:12.682 INFO:tasks.workunit.client.1.smithi115.stdout:3/356: dread d13/f4c [0,4194304] 0 2021-11-03T17:14:12.682 INFO:tasks.workunit.client.1.smithi115.stdout:9/216: dwrite d1/d18/d2b/d42/fc [0,4194304] 0 2021-11-03T17:14:12.682 INFO:tasks.workunit.client.1.smithi115.stdout:9/217: stat d1/d2d/f2a 0 2021-11-03T17:14:12.683 INFO:tasks.workunit.client.0.smithi027.stdout:4/288: dread d6/d9/d16/d36/f4c [0,4194304] 0 2021-11-03T17:14:12.684 INFO:tasks.workunit.client.0.smithi027.stdout:4/289: getdents d6/db/d21/d2b/d2d/d3e 0 2021-11-03T17:14:12.684 INFO:tasks.workunit.client.0.smithi027.stdout:4/290: rmdir d6/d24 39 2021-11-03T17:14:12.687 INFO:tasks.workunit.client.0.smithi027.stdout:4/291: symlink d6/db/l5e 0 2021-11-03T17:14:12.687 INFO:tasks.workunit.client.0.smithi027.stdout:4/292: readlink d6/db/l5e 0 2021-11-03T17:14:12.696 INFO:tasks.workunit.client.0.smithi027.stdout:8/369: dwrite d7/d10/d18/f49 [4194304,4194304] 0 2021-11-03T17:14:12.697 INFO:tasks.workunit.client.0.smithi027.stdout:6/333: dwrite dc/d16/d1b/d3c/f44 [4194304,4194304] 0 2021-11-03T17:14:12.701 INFO:tasks.workunit.client.0.smithi027.stdout:6/334: creat dc/d16/d1b/d3c/d48/f5c x:0 0 0 2021-11-03T17:14:12.702 INFO:tasks.workunit.client.0.smithi027.stdout:6/335: read - dc/d16/d1b/d3c/d51/f58 zero size 2021-11-03T17:14:12.702 INFO:tasks.workunit.client.0.smithi027.stdout:6/336: stat dc/f10 0 2021-11-03T17:14:12.702 INFO:tasks.workunit.client.0.smithi027.stdout:6/337: readlink dc/d16/d1b/d3c/l3f 0 2021-11-03T17:14:12.702 INFO:tasks.workunit.client.0.smithi027.stdout:6/338: mknod dc/d16/d1b/d3c/d51/c5d 1 2021-11-03T17:14:12.703 INFO:tasks.workunit.client.0.smithi027.stdout:6/339: mknod dc/d16/d1b/d3c/d48/c5e 1 2021-11-03T17:14:12.703 INFO:tasks.workunit.client.1.smithi115.stdout:9/218: dread d1/d18/d2b/d42/f17 [0,4194304] 0 2021-11-03T17:14:12.704 INFO:tasks.workunit.client.0.smithi027.stdout:6/340: rmdir dc/d16/d1b/d24 39 2021-11-03T17:14:12.704 INFO:tasks.workunit.client.0.smithi027.stdout:6/341: mkdir dc/d16/d1b/d3c/d51/d5f 0 2021-11-03T17:14:12.704 INFO:tasks.workunit.client.0.smithi027.stdout:6/342: readlink dc/d16/l31 0 2021-11-03T17:14:12.705 INFO:tasks.workunit.client.1.smithi115.stdout:9/219: rename d1/d2d to d1/d1a/d48 0 2021-11-03T17:14:12.705 INFO:tasks.workunit.client.1.smithi115.stdout:9/220: fsync d1/d18/d2b/d42/f12 0 2021-11-03T17:14:12.706 INFO:tasks.workunit.client.1.smithi115.stdout:9/221: mkdir d1/d1a/d48/d27/d2f/d49 0 2021-11-03T17:14:12.707 INFO:tasks.workunit.client.0.smithi027.stdout:6/343: unlink dc/d16/d1b/d3c/d51/f56 0 2021-11-03T17:14:12.709 INFO:tasks.workunit.client.1.smithi115.stdout:9/222: mkdir d1/d18/d4a 0 2021-11-03T17:14:12.709 INFO:tasks.workunit.client.1.smithi115.stdout:9/223: dread - d1/d1a/d48/d27/f3d zero size 2021-11-03T17:14:12.709 INFO:tasks.workunit.client.1.smithi115.stdout:9/224: mknod d1/d1a/d48/d27/d2f/d49/c4b 1 2021-11-03T17:14:12.709 INFO:tasks.workunit.client.1.smithi115.stdout:9/225: truncate d1/d1a/d48/d27/f47 1371476 0 2021-11-03T17:14:12.710 INFO:tasks.workunit.client.1.smithi115.stdout:9/226: chown d1/d18/d4a 976 1 2021-11-03T17:14:12.711 INFO:tasks.workunit.client.1.smithi115.stdout:9/227: rmdir d1/d1a/d48/d46 0 2021-11-03T17:14:12.712 INFO:tasks.workunit.client.1.smithi115.stdout:9/228: fdatasync d1/d18/d2b/d42/f12 0 2021-11-03T17:14:12.712 INFO:tasks.workunit.client.1.smithi115.stdout:9/229: creat d1/d1a/d48/d27/f4c x:0 0 0 2021-11-03T17:14:12.712 INFO:tasks.workunit.client.1.smithi115.stdout:3/357: dwrite d13/f33 [4194304,4194304] 0 2021-11-03T17:14:12.712 INFO:tasks.workunit.client.0.smithi027.stdout:6/344: creat dc/d16/d1b/d3c/d48/d57/f60 x:0 0 0 2021-11-03T17:14:12.713 INFO:tasks.workunit.client.0.smithi027.stdout:8/370: dread d7/d10/f15 [0,4194304] 0 2021-11-03T17:14:12.713 INFO:tasks.workunit.client.0.smithi027.stdout:4/293: dwrite d6/d9/d16/f32 [4194304,4194304] 0 2021-11-03T17:14:12.713 INFO:tasks.workunit.client.0.smithi027.stdout:6/345: creat dc/d16/d30/f61 x:0 0 0 2021-11-03T17:14:12.713 INFO:tasks.workunit.client.0.smithi027.stdout:6/346: stat dc/d16/d1b/d3c/d51/f5b 0 2021-11-03T17:14:12.714 INFO:tasks.workunit.client.0.smithi027.stdout:6/347: truncate dc/f1e 5943631 0 2021-11-03T17:14:12.714 INFO:tasks.workunit.client.0.smithi027.stdout:6/348: write dc/d16/d1b/d2d/d34/f4c [5001910,76233] 0 2021-11-03T17:14:12.715 INFO:tasks.workunit.client.1.smithi115.stdout:3/358: chown d13/f48 1 1 2021-11-03T17:14:12.715 INFO:tasks.workunit.client.1.smithi115.stdout:3/359: creat d13/f7c x:0 0 0 2021-11-03T17:14:12.715 INFO:tasks.workunit.client.1.smithi115.stdout:3/360: chown d13/d2a/d53/d72 1779188 1 2021-11-03T17:14:12.716 INFO:tasks.workunit.client.1.smithi115.stdout:3/361: readlink d13/d50/l6d 0 2021-11-03T17:14:12.716 INFO:tasks.workunit.client.0.smithi027.stdout:4/294: write d6/d24/f46 [881143,41586] 0 2021-11-03T17:14:12.720 INFO:tasks.workunit.client.0.smithi027.stdout:4/295: unlink d6/db/d21/f25 0 2021-11-03T17:14:12.720 INFO:tasks.workunit.client.0.smithi027.stdout:4/296: mknod d6/db/c5f 1 2021-11-03T17:14:12.722 INFO:tasks.workunit.client.1.smithi115.stdout:9/230: getdents d1/d18 0 2021-11-03T17:14:12.722 INFO:tasks.workunit.client.1.smithi115.stdout:9/231: chown d1/d18/d2b/d42/l3f 3121 1 2021-11-03T17:14:12.723 INFO:tasks.workunit.client.1.smithi115.stdout:3/362: dread d13/d51/d40/f4f [0,4194304] 0 2021-11-03T17:14:12.724 INFO:tasks.workunit.client.0.smithi027.stdout:6/349: dread dc/d16/f2c [0,4194304] 0 2021-11-03T17:14:12.726 INFO:tasks.workunit.client.1.smithi115.stdout:3/363: rmdir d13/d2a 39 2021-11-03T17:14:12.727 INFO:tasks.workunit.client.1.smithi115.stdout:9/232: rename d1/d18/d2b/d42/l31 to d1/l4d 0 2021-11-03T17:14:12.728 INFO:tasks.workunit.client.1.smithi115.stdout:9/233: write d1/d18/d2b/d42/f21 [777664,98833] 0 2021-11-03T17:14:12.729 INFO:tasks.workunit.client.1.smithi115.stdout:3/364: symlink d13/l7d 0 2021-11-03T17:14:12.730 INFO:tasks.workunit.client.1.smithi115.stdout:9/234: symlink d1/d1a/d48/d27/d34/l4e 0 2021-11-03T17:14:12.731 INFO:tasks.workunit.client.1.smithi115.stdout:3/365: creat d13/d67/f7e x:0 0 0 2021-11-03T17:14:12.732 INFO:tasks.workunit.client.1.smithi115.stdout:3/366: stat d13/f1e 0 2021-11-03T17:14:12.733 INFO:tasks.workunit.client.1.smithi115.stdout:3/367: chown ff 9205374 1 2021-11-03T17:14:12.735 INFO:tasks.workunit.client.1.smithi115.stdout:9/235: read d1/f25 [2727154,19915] 0 2021-11-03T17:14:12.735 INFO:tasks.workunit.client.1.smithi115.stdout:9/236: write d1/d1a/d48/f2a [908150,125460] 0 2021-11-03T17:14:12.736 INFO:tasks.workunit.client.1.smithi115.stdout:3/368: rename d13/l62 to d13/l7f 0 2021-11-03T17:14:12.736 INFO:tasks.workunit.client.1.smithi115.stdout:3/369: chown d13/d2a/d53/d70 279185005 1 2021-11-03T17:14:12.737 INFO:tasks.workunit.client.1.smithi115.stdout:9/237: write d1/d18/d2b/d42/f20 [1763515,27902] 0 2021-11-03T17:14:12.737 INFO:tasks.workunit.client.1.smithi115.stdout:9/238: mknod d1/d18/d2b/d45/c4f 1 2021-11-03T17:14:12.737 INFO:tasks.workunit.client.1.smithi115.stdout:3/370: symlink d13/d51/d40/l80 0 2021-11-03T17:14:12.738 INFO:tasks.workunit.client.1.smithi115.stdout:3/371: chown d13/d2a 169 1 2021-11-03T17:14:12.739 INFO:tasks.workunit.client.1.smithi115.stdout:9/239: rename d1/d1a/d48/d27/d2f/d49 to d1/d1a/d48/d27/d34/d50 0 2021-11-03T17:14:12.740 INFO:tasks.workunit.client.1.smithi115.stdout:9/240: chown d1/d1a/d48/d27/f3d 25 1 2021-11-03T17:14:12.740 INFO:tasks.workunit.client.1.smithi115.stdout:9/241: chown d1/d18/d2b/l30 8063521 1 2021-11-03T17:14:12.740 INFO:tasks.workunit.client.0.smithi027.stdout:6/350: dread dc/d16/d30/f32 [0,4194304] 0 2021-11-03T17:14:12.741 INFO:tasks.workunit.client.1.smithi115.stdout:9/242: rename d1/d18/d2b/d42/l37 to d1/d1a/d48/d27/d2f/l51 0 2021-11-03T17:14:12.741 INFO:tasks.workunit.client.1.smithi115.stdout:9/243: chown d1/d39 95 1 2021-11-03T17:14:12.743 INFO:tasks.workunit.client.0.smithi027.stdout:4/297: dwrite d6/db/d21/d2b/f45 [0,4194304] 0 2021-11-03T17:14:12.743 INFO:tasks.workunit.client.0.smithi027.stdout:4/298: chown d6/db/d21/d41/l52 1 1 2021-11-03T17:14:12.743 INFO:tasks.workunit.client.0.smithi027.stdout:4/299: fsync d6/d9/d16/d36/f42 0 2021-11-03T17:14:12.750 INFO:tasks.workunit.client.0.smithi027.stdout:6/351: unlink dc/d37/f47 0 2021-11-03T17:14:12.751 INFO:tasks.workunit.client.1.smithi115.stdout:9/244: dread d1/d18/d2b/d42/f21 [0,4194304] 0 2021-11-03T17:14:12.752 INFO:tasks.workunit.client.0.smithi027.stdout:4/300: creat d6/db/d21/d2b/d2d/d3e/d58/f60 x:0 0 0 2021-11-03T17:14:12.753 INFO:tasks.workunit.client.1.smithi115.stdout:9/245: rename d1/d18/f1e to d1/d39/f52 0 2021-11-03T17:14:12.753 INFO:tasks.workunit.client.1.smithi115.stdout:9/246: write d1/d1a/d48/f26 [721288,21456] 0 2021-11-03T17:14:12.754 INFO:tasks.workunit.client.0.smithi027.stdout:4/301: creat d6/db/d21/d2b/d2d/d3e/d58/f61 x:0 0 0 2021-11-03T17:14:12.754 INFO:tasks.workunit.client.0.smithi027.stdout:4/302: mknod d6/c62 1 2021-11-03T17:14:12.755 INFO:tasks.workunit.client.1.smithi115.stdout:9/247: dread d1/d39/f52 [0,4194304] 0 2021-11-03T17:14:12.756 INFO:tasks.workunit.client.0.smithi027.stdout:4/303: mkdir d6/d9/d16/d36/d57/d5a/d63 0 2021-11-03T17:14:12.756 INFO:tasks.workunit.client.0.smithi027.stdout:4/304: mknod d6/db/d21/d2b/d2d/d3e/d58/c64 1 2021-11-03T17:14:12.757 INFO:tasks.workunit.client.1.smithi115.stdout:3/372: dwrite d13/f56 [0,4194304] 0 2021-11-03T17:14:12.759 INFO:tasks.workunit.client.0.smithi027.stdout:6/352: dread dc/f10 [0,4194304] 0 2021-11-03T17:14:12.759 INFO:tasks.workunit.client.0.smithi027.stdout:4/305: dread d6/d9/d16/f19 [0,4194304] 0 2021-11-03T17:14:12.760 INFO:tasks.workunit.client.0.smithi027.stdout:4/306: stat d6/db/d21/d2b/d2d/d3e/d58/f60 0 2021-11-03T17:14:12.761 INFO:tasks.workunit.client.0.smithi027.stdout:4/307: symlink d6/db/d21/d2b/d2d/d3e/l65 0 2021-11-03T17:14:12.762 INFO:tasks.workunit.client.1.smithi115.stdout:3/373: fsync d13/d67/f7e 0 2021-11-03T17:14:12.763 INFO:tasks.workunit.client.0.smithi027.stdout:4/308: creat d6/d9/d16/d36/d57/d5a/f66 x:0 0 0 2021-11-03T17:14:12.766 INFO:tasks.workunit.client.1.smithi115.stdout:3/374: rename l1 to d13/d50/l81 0 2021-11-03T17:14:12.767 INFO:tasks.workunit.client.0.smithi027.stdout:4/309: dread d6/d9/d28/f4b [0,4194304] 0 2021-11-03T17:14:12.773 INFO:tasks.workunit.client.0.smithi027.stdout:4/310: write f4 [3605012,111972] 0 2021-11-03T17:14:12.773 INFO:tasks.workunit.client.0.smithi027.stdout:4/311: write d6/d9/d16/d36/d57/d5a/f5c [371498,71395] 0 2021-11-03T17:14:12.777 INFO:tasks.workunit.client.0.smithi027.stdout:4/312: write f4 [670583,50923] 0 2021-11-03T17:14:12.779 INFO:tasks.workunit.client.1.smithi115.stdout:9/248: dwrite d1/d18/d2b/d42/f17 [0,4194304] 0 2021-11-03T17:14:12.780 INFO:tasks.workunit.client.0.smithi027.stdout:4/313: link d6/d9/d16/f19 d6/d9/d16/d36/d57/d5a/f67 0 2021-11-03T17:14:12.781 INFO:tasks.workunit.client.0.smithi027.stdout:4/314: write d6/db/f35 [1465148,95790] 0 2021-11-03T17:14:12.781 INFO:tasks.workunit.client.0.smithi027.stdout:4/315: readlink d6/db/d21/d2b/l49 0 2021-11-03T17:14:12.782 INFO:tasks.workunit.client.0.smithi027.stdout:4/316: write d6/db/d21/d2b/d2d/d3e/d58/f61 [633030,58195] 0 2021-11-03T17:14:12.783 INFO:tasks.workunit.client.1.smithi115.stdout:9/249: mkdir d1/d1a/d48/d27/d53 0 2021-11-03T17:14:12.784 INFO:tasks.workunit.client.1.smithi115.stdout:9/250: rename d1/d32 to d1/d18/d4a/d54 0 2021-11-03T17:14:12.785 INFO:tasks.workunit.client.1.smithi115.stdout:9/251: unlink d1/f7 0 2021-11-03T17:14:12.787 INFO:tasks.workunit.client.1.smithi115.stdout:9/252: creat d1/d1a/d48/d27/f55 x:0 0 0 2021-11-03T17:14:12.787 INFO:tasks.workunit.client.1.smithi115.stdout:9/253: truncate d1/f1d 920615 0 2021-11-03T17:14:12.789 INFO:tasks.workunit.client.1.smithi115.stdout:9/254: symlink d1/d18/d2b/d45/l56 0 2021-11-03T17:14:12.789 INFO:tasks.workunit.client.1.smithi115.stdout:9/255: stat d1/d18 0 2021-11-03T17:14:12.790 INFO:tasks.workunit.client.1.smithi115.stdout:9/256: chown d1/d1a/d48/d27/d53 496830145 1 2021-11-03T17:14:12.790 INFO:tasks.workunit.client.1.smithi115.stdout:9/257: mknod d1/d18/c57 1 2021-11-03T17:14:12.792 INFO:tasks.workunit.client.0.smithi027.stdout:6/353: dwrite dc/d16/d1b/d3c/f44 [4194304,4194304] 0 2021-11-03T17:14:12.793 INFO:tasks.workunit.client.1.smithi115.stdout:9/258: mkdir d1/d18/d4a/d54/d3a/d58 0 2021-11-03T17:14:12.793 INFO:tasks.workunit.client.1.smithi115.stdout:9/259: creat d1/d1a/d48/d27/d2f/f59 x:0 0 0 2021-11-03T17:14:12.794 INFO:tasks.workunit.client.1.smithi115.stdout:3/375: dwrite d13/d35/f3d [0,4194304] 0 2021-11-03T17:14:12.794 INFO:tasks.workunit.client.1.smithi115.stdout:3/376: mknod d13/d2a/d60/c82 1 2021-11-03T17:14:12.794 INFO:tasks.workunit.client.1.smithi115.stdout:9/260: write d1/d18/d2b/d42/ff [503585,49881] 0 2021-11-03T17:14:12.795 INFO:tasks.workunit.client.1.smithi115.stdout:9/261: mknod d1/d18/d4a/d54/d3a/d58/c5a 1 2021-11-03T17:14:12.798 INFO:tasks.workunit.client.0.smithi027.stdout:4/317: dread d6/d24/f46 [0,4194304] 0 2021-11-03T17:14:12.798 INFO:tasks.workunit.client.0.smithi027.stdout:4/318: chown d6/db/l4d 21572133 1 2021-11-03T17:14:12.798 INFO:tasks.workunit.client.0.smithi027.stdout:4/319: truncate d6/d9/d16/d36/d57/d5a/f5c 522646 0 2021-11-03T17:14:12.799 INFO:tasks.workunit.client.0.smithi027.stdout:4/320: mknod d6/d9/d16/d36/d57/d5a/c68 1 2021-11-03T17:14:12.799 INFO:tasks.workunit.client.0.smithi027.stdout:4/321: creat d6/db/d21/f69 x:0 0 0 2021-11-03T17:14:12.802 INFO:tasks.workunit.client.1.smithi115.stdout:3/377: truncate d13/d51/f2d 1917304 0 2021-11-03T17:14:12.803 INFO:tasks.workunit.client.1.smithi115.stdout:3/378: mknod d13/d51/d2e/c83 1 2021-11-03T17:14:12.803 INFO:tasks.workunit.client.1.smithi115.stdout:3/379: fdatasync d13/d50/f4d 0 2021-11-03T17:14:12.804 INFO:tasks.workunit.client.0.smithi027.stdout:4/322: creat d6/d9/d16/d36/f6a x:0 0 0 2021-11-03T17:14:12.804 INFO:tasks.workunit.client.1.smithi115.stdout:3/380: rmdir d13/d2a/d60 39 2021-11-03T17:14:12.805 INFO:tasks.workunit.client.1.smithi115.stdout:3/381: creat d13/d2a/d60/f84 x:0 0 0 2021-11-03T17:14:12.806 INFO:tasks.workunit.client.0.smithi027.stdout:4/323: rename d6/d9/d16/d36/d57/d5a/f66 to d6/db/d21/d2b/d2d/d3e/f6b 0 2021-11-03T17:14:12.809 INFO:tasks.workunit.client.1.smithi115.stdout:5/407: sync 2021-11-03T17:14:12.809 INFO:tasks.workunit.client.1.smithi115.stdout:7/274: sync 2021-11-03T17:14:12.810 INFO:tasks.workunit.client.1.smithi115.stdout:2/276: sync 2021-11-03T17:14:12.810 INFO:tasks.workunit.client.1.smithi115.stdout:1/308: sync 2021-11-03T17:14:12.810 INFO:tasks.workunit.client.1.smithi115.stdout:2/277: write d4/d6/d2e/d34/d43/f4d [682515,5625] 0 2021-11-03T17:14:12.811 INFO:tasks.workunit.client.0.smithi027.stdout:4/324: creat d6/db/d21/d2b/d2d/d3e/f6c x:0 0 0 2021-11-03T17:14:12.811 INFO:tasks.workunit.client.1.smithi115.stdout:7/275: rename d3/da/d26/f45 to d3/da/d26/d33/f59 0 2021-11-03T17:14:12.812 INFO:tasks.workunit.client.1.smithi115.stdout:1/309: truncate d7/dc/f16 699304 0 2021-11-03T17:14:12.812 INFO:tasks.workunit.client.0.smithi027.stdout:6/354: truncate dc/f1e 5317478 0 2021-11-03T17:14:12.813 INFO:tasks.workunit.client.0.smithi027.stdout:6/355: dread - dc/d16/d1b/d2d/d34/f4e zero size 2021-11-03T17:14:12.814 INFO:tasks.workunit.client.0.smithi027.stdout:6/356: getdents dc/d16/d1b/d24 0 2021-11-03T17:14:12.815 INFO:tasks.workunit.client.1.smithi115.stdout:5/408: mkdir d1a/d28/d78/d8d 0 2021-11-03T17:14:12.815 INFO:tasks.workunit.client.1.smithi115.stdout:9/262: dwrite d1/d1a/d48/d27/f3d [0,4194304] 0 2021-11-03T17:14:12.818 INFO:tasks.workunit.client.1.smithi115.stdout:2/278: rename d4/d6/d2e/d34/d43/f3e to d4/d6/d2e/d34/d42/d57/f5f 0 2021-11-03T17:14:12.819 INFO:tasks.workunit.client.1.smithi115.stdout:2/279: write d4/d6/f12 [4150751,23183] 0 2021-11-03T17:14:12.821 INFO:tasks.workunit.client.1.smithi115.stdout:1/310: link d7/dc/dd/f5a d7/dc/dd/d24/d4b/d55/f5d 0 2021-11-03T17:14:12.821 INFO:tasks.workunit.client.1.smithi115.stdout:1/311: mknod d7/dc/dd/d42/c5e 1 2021-11-03T17:14:12.822 INFO:tasks.workunit.client.1.smithi115.stdout:1/312: mknod d7/dc/dd/d24/d26/c5f 1 2021-11-03T17:14:12.824 INFO:tasks.workunit.client.0.smithi027.stdout:6/357: dread f7 [0,4194304] 0 2021-11-03T17:14:12.825 INFO:tasks.workunit.client.1.smithi115.stdout:9/263: creat d1/d18/d2b/f5b x:0 0 0 2021-11-03T17:14:12.825 INFO:tasks.workunit.client.1.smithi115.stdout:9/264: dread - d1/d1a/d48/d27/d2f/f59 zero size 2021-11-03T17:14:12.826 INFO:tasks.workunit.client.0.smithi027.stdout:6/358: rmdir dc/d16/d1b/d2d/d34 39 2021-11-03T17:14:12.827 INFO:tasks.workunit.client.1.smithi115.stdout:7/276: rename d3/da/d18/l2e to d3/da/d26/d35/d53/l5a 0 2021-11-03T17:14:12.828 INFO:tasks.workunit.client.0.smithi027.stdout:6/359: truncate dc/d16/d1b/d2d/d34/f4c 2707643 0 2021-11-03T17:14:12.829 INFO:tasks.workunit.client.1.smithi115.stdout:3/382: dwrite d13/d67/f6c [0,4194304] 0 2021-11-03T17:14:12.830 INFO:tasks.workunit.client.1.smithi115.stdout:1/313: mkdir d7/dc/dd/d24/d4d/d60 0 2021-11-03T17:14:12.830 INFO:tasks.workunit.client.1.smithi115.stdout:2/280: creat d4/d6/d2e/d34/d42/d57/f60 x:0 0 0 2021-11-03T17:14:12.830 INFO:tasks.workunit.client.1.smithi115.stdout:1/314: dread - d7/d38/f5c zero size 2021-11-03T17:14:12.831 INFO:tasks.workunit.client.1.smithi115.stdout:9/265: creat d1/d39/f5c x:0 0 0 2021-11-03T17:14:12.831 INFO:tasks.workunit.client.1.smithi115.stdout:9/266: readlink d1/l29 0 2021-11-03T17:14:12.832 INFO:tasks.workunit.client.0.smithi027.stdout:6/360: write dc/d16/f2c [2237614,54223] 0 2021-11-03T17:14:12.832 INFO:tasks.workunit.client.0.smithi027.stdout:6/361: rmdir dc/d16/d1b/d3c 39 2021-11-03T17:14:12.833 INFO:tasks.workunit.client.0.smithi027.stdout:6/362: chown dc/d16/d1b/l42 2365 1 2021-11-03T17:14:12.833 INFO:tasks.workunit.client.0.smithi027.stdout:6/363: fsync dc/d16/d1b/d2d/f35 0 2021-11-03T17:14:12.833 INFO:tasks.workunit.client.0.smithi027.stdout:6/364: chown dc/d16/d1b/d2d 0 1 2021-11-03T17:14:12.834 INFO:tasks.workunit.client.0.smithi027.stdout:4/325: dwrite d6/db/d21/f37 [0,4194304] 0 2021-11-03T17:14:12.834 INFO:tasks.workunit.client.1.smithi115.stdout:9/267: dread d1/d18/f22 [0,4194304] 0 2021-11-03T17:14:12.835 INFO:tasks.workunit.client.1.smithi115.stdout:5/409: rename d1a/d28/d34/l7a to d1a/d28/d4a/d65/l8e 0 2021-11-03T17:14:12.835 INFO:tasks.workunit.client.0.smithi027.stdout:4/326: mkdir d6/d24/d6d 0 2021-11-03T17:14:12.836 INFO:tasks.workunit.client.1.smithi115.stdout:2/281: link d4/d6/d2e/d34/d42/d57/l59 d4/d44/l61 0 2021-11-03T17:14:12.837 INFO:tasks.workunit.client.1.smithi115.stdout:3/383: dread d13/f1e [0,4194304] 0 2021-11-03T17:14:12.837 INFO:tasks.workunit.client.1.smithi115.stdout:2/282: read - d4/d6/d13/f21 zero size 2021-11-03T17:14:12.838 INFO:tasks.workunit.client.0.smithi027.stdout:4/327: creat d6/d9/d16/d17/f6e x:0 0 0 2021-11-03T17:14:12.838 INFO:tasks.workunit.client.0.smithi027.stdout:4/328: truncate d6/db/d21/d2b/d2d/d3e/f6c 434943 0 2021-11-03T17:14:12.840 INFO:tasks.workunit.client.1.smithi115.stdout:1/315: rename d7/dc/dd/d42/f44 to d7/dc/dd/d42/f61 0 2021-11-03T17:14:12.841 INFO:tasks.workunit.client.1.smithi115.stdout:9/268: symlink d1/d1a/d48/d27/d2f/l5d 0 2021-11-03T17:14:12.841 INFO:tasks.workunit.client.1.smithi115.stdout:9/269: write d1/d18/d2b/f5b [1001581,79514] 0 2021-11-03T17:14:12.842 INFO:tasks.workunit.client.1.smithi115.stdout:2/283: creat d4/f62 x:0 0 0 2021-11-03T17:14:12.843 INFO:tasks.workunit.client.1.smithi115.stdout:1/316: read d7/dc/d2a/f33 [624700,85209] 0 2021-11-03T17:14:12.844 INFO:tasks.workunit.client.1.smithi115.stdout:9/270: link d1/d18/d2b/d42/l3f d1/d1a/d48/d27/d2f/d41/l5e 0 2021-11-03T17:14:12.845 INFO:tasks.workunit.client.1.smithi115.stdout:5/410: dread d1a/d23/f61 [0,4194304] 0 2021-11-03T17:14:12.845 INFO:tasks.workunit.client.1.smithi115.stdout:5/411: fdatasync d1a/d28/d7b/f77 0 2021-11-03T17:14:12.846 INFO:tasks.workunit.client.1.smithi115.stdout:1/317: creat d7/dc/dd/d24/d4d/d60/f62 x:0 0 0 2021-11-03T17:14:12.847 INFO:tasks.workunit.client.1.smithi115.stdout:1/318: readlink d7/dc/dd/d3e/l56 0 2021-11-03T17:14:12.848 INFO:tasks.workunit.client.0.smithi027.stdout:6/365: dread dc/f14 [0,4194304] 0 2021-11-03T17:14:12.848 INFO:tasks.workunit.client.1.smithi115.stdout:9/271: symlink d1/d1a/l5f 0 2021-11-03T17:14:12.848 INFO:tasks.workunit.client.1.smithi115.stdout:9/272: dread - d1/d18/d4a/d54/d3a/f3b zero size 2021-11-03T17:14:12.849 INFO:tasks.workunit.client.0.smithi027.stdout:6/366: mkdir dc/d16/d62 0 2021-11-03T17:14:12.852 INFO:tasks.workunit.client.1.smithi115.stdout:5/412: rmdir d1a/d28/d78/d8d 0 2021-11-03T17:14:12.852 INFO:tasks.workunit.client.1.smithi115.stdout:5/413: chown d1a/d28/d4a/d65/l8e 482837 1 2021-11-03T17:14:12.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.858+0000 7fcb0d714700 1 -- 172.21.15.27:0/3800661622 --> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb080fee60 con 0x7fcaf005c300 2021-11-03T17:14:12.854 INFO:tasks.workunit.client.1.smithi115.stdout:9/273: creat d1/d1a/d48/d27/d2f/d41/f60 x:0 0 0 2021-11-03T17:14:12.857 INFO:tasks.workunit.client.1.smithi115.stdout:5/414: creat d1a/d3d/d89/f8f x:0 0 0 2021-11-03T17:14:12.857 INFO:tasks.workunit.client.1.smithi115.stdout:5/415: read - d1a/d28/d4c/d75/d83/f88 zero size 2021-11-03T17:14:12.858 INFO:tasks.workunit.client.1.smithi115.stdout:9/274: symlink d1/d18/l61 0 2021-11-03T17:14:12.858 INFO:tasks.workunit.client.1.smithi115.stdout:3/384: dwrite d13/d51/d2e/f2f [0,4194304] 0 2021-11-03T17:14:12.859 INFO:tasks.workunit.client.1.smithi115.stdout:9/275: creat d1/d1a/d48/f62 x:0 0 0 2021-11-03T17:14:12.859 INFO:tasks.workunit.client.1.smithi115.stdout:3/385: rmdir d13/d35/d54 39 2021-11-03T17:14:12.859 INFO:tasks.workunit.client.1.smithi115.stdout:3/386: mknod d13/d35/c85 1 2021-11-03T17:14:12.861 INFO:tasks.workunit.client.1.smithi115.stdout:3/387: link ld d13/d2a/d53/l86 0 2021-11-03T17:14:12.862 INFO:tasks.workunit.client.1.smithi115.stdout:3/388: chown d13/d51/f29 2020087 1 2021-11-03T17:14:12.862 INFO:tasks.workunit.client.1.smithi115.stdout:9/276: mkdir d1/d18/d2b/d42/d63 0 2021-11-03T17:14:12.863 INFO:tasks.workunit.client.1.smithi115.stdout:9/277: truncate d1/d18/d2b/f35 588543 0 2021-11-03T17:14:12.863 INFO:tasks.workunit.client.1.smithi115.stdout:9/278: fdatasync d1/d1a/f28 0 2021-11-03T17:14:12.863 INFO:tasks.workunit.client.1.smithi115.stdout:9/279: mknod d1/d18/d2b/c64 1 2021-11-03T17:14:12.864 INFO:tasks.workunit.client.0.smithi027.stdout:9/372: sync 2021-11-03T17:14:12.864 INFO:tasks.workunit.client.0.smithi027.stdout:9/373: mknod db/d1b/d23/d2c/c87 1 2021-11-03T17:14:12.865 INFO:tasks.workunit.client.1.smithi115.stdout:3/389: getdents d13/d2a 0 2021-11-03T17:14:12.867 INFO:tasks.workunit.client.1.smithi115.stdout:2/284: rmdir d4/d44 39 2021-11-03T17:14:12.867 INFO:tasks.workunit.client.1.smithi115.stdout:2/285: write d4/d6/d56/f5d [8896785,20821] 0 2021-11-03T17:14:12.868 INFO:tasks.workunit.client.1.smithi115.stdout:9/280: mkdir d1/d18/d65 0 2021-11-03T17:14:12.868 INFO:tasks.workunit.client.1.smithi115.stdout:9/281: fsync d1/d18/d2b/d42/f21 0 2021-11-03T17:14:12.869 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.875+0000 7fcb04ff9700 1 -- 172.21.15.27:0/3800661622 <== mgr.14162 v2:172.21.15.27:6800/3802716887 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2662 (secure 0 0 0) 0x7fcb080fee60 con 0x7fcaf005c300 2021-11-03T17:14:12.870 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE VERSION IMAGE ID CONTAINER ID 2021-11-03T17:14:12.870 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (5m) 114s ago 7m 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:14:12.870 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (6m) 114s ago 6m 16.2.4 8d91d370c2b8 d494472a47c0 2021-11-03T17:14:12.871 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (6m) 118s ago 6m 16.2.4 8d91d370c2b8 f84fd4042530 2021-11-03T17:14:12.871 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (5m) 114s ago 6m 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:14:12.871 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (2m) 114s ago 2m 16.2.4 8d91d370c2b8 509a42d9cc82 2021-11-03T17:14:12.871 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (2m) 114s ago 2m 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:14:12.871 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (2m) 118s ago 2m 16.2.4 8d91d370c2b8 bfd990d6415e 2021-11-03T17:14:12.872 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (2m) 118s ago 2m 16.2.4 8d91d370c2b8 f8e6ab2a5ed4 2021-11-03T17:14:12.872 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:9283 running (8m) 114s ago 8m 16.2.4 8d91d370c2b8 999298a5769c 2021-11-03T17:14:12.872 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8443,9283 running (6m) 118s ago 6m 16.2.4 8d91d370c2b8 722d63932db9 2021-11-03T17:14:12.872 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (8m) 114s ago 8m 16.2.4 8d91d370c2b8 060d5938214f 2021-11-03T17:14:12.873 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (6m) 118s ago 6m 16.2.4 8d91d370c2b8 319323f09834 2021-11-03T17:14:12.873 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (6m) 114s ago 6m 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:14:12.873 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (6m) 118s ago 5m 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:14:12.873 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (5m) 114s ago 5m 16.2.4 8d91d370c2b8 17f79a815a0f 2021-11-03T17:14:12.873 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (5m) 114s ago 5m 16.2.4 8d91d370c2b8 422be0cac0db 2021-11-03T17:14:12.874 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (4m) 114s ago 4m 16.2.4 8d91d370c2b8 c211ef5551c9 2021-11-03T17:14:12.874 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (4m) 118s ago 4m 16.2.4 8d91d370c2b8 b1486ec1b619 2021-11-03T17:14:12.874 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (4m) 118s ago 4m 16.2.4 8d91d370c2b8 8b91230c69e5 2021-11-03T17:14:12.874 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (3m) 118s ago 3m 16.2.4 8d91d370c2b8 4a54c4d6b6f5 2021-11-03T17:14:12.874 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (5m) 114s ago 6m 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:14:12.876 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.877+0000 7fcb0d714700 1 -- 172.21.15.27:0/3800661622 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fcaf005c300 msgr2=0x7fcaf005e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:14:12.876 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.877+0000 7fcb0d714700 1 --2- 172.21.15.27:0/3800661622 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fcaf005c300 0x7fcaf005e7c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fcafc004630 tx=0x7fcafc009b30).stop 2021-11-03T17:14:12.877 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.877+0000 7fcb0d714700 1 -- 172.21.15.27:0/3800661622 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb08060310 msgr2=0x7fcb080f6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:14:12.877 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.877+0000 7fcb0d714700 1 --2- 172.21.15.27:0/3800661622 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb08060310 0x7fcb080f6b90 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7fcaf80026d0 tx=0x7fcaf8009de0).stop 2021-11-03T17:14:12.877 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.877+0000 7fcb0d714700 1 -- 172.21.15.27:0/3800661622 shutdown_connections 2021-11-03T17:14:12.877 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.877+0000 7fcb0d714700 1 --2- 172.21.15.27:0/3800661622 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7fcaf005c300 0x7fcaf005e7c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:12.877 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.877+0000 7fcb0d714700 1 --2- 172.21.15.27:0/3800661622 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb08060310 0x7fcb080f6b90 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:12.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.877+0000 7fcb0d714700 1 --2- 172.21.15.27:0/3800661622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb080f9690 0x7fcb080f5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:12.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.877+0000 7fcb0d714700 1 -- 172.21.15.27:0/3800661622 >> 172.21.15.27:0/3800661622 conn(0x7fcb080f0b90 msgr2=0x7fcb080f2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:14:12.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.878+0000 7fcb0d714700 1 -- 172.21.15.27:0/3800661622 shutdown_connections 2021-11-03T17:14:12.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:12.878+0000 7fcb0d714700 1 -- 172.21.15.27:0/3800661622 wait complete. 2021-11-03T17:14:12.880 INFO:tasks.workunit.client.0.smithi027.stdout:9/374: dread db/f17 [0,4194304] 0 2021-11-03T17:14:12.880 INFO:tasks.workunit.client.0.smithi027.stdout:9/375: stat db/d1b/d1c/d1d/l72 0 2021-11-03T17:14:12.880 INFO:tasks.workunit.client.0.smithi027.stdout:9/376: readlink db/l1a 0 2021-11-03T17:14:12.880 INFO:tasks.workunit.client.0.smithi027.stdout:6/367: dwrite dc/f18 [0,4194304] 0 2021-11-03T17:14:12.882 INFO:tasks.workunit.client.0.smithi027.stdout:6/368: symlink dc/d16/d1b/d3c/d51/d5f/l63 0 2021-11-03T17:14:12.882 INFO:tasks.workunit.client.0.smithi027.stdout:6/369: creat dc/f64 x:0 0 0 2021-11-03T17:14:12.883 INFO:tasks.workunit.client.1.smithi115.stdout:2/286: rmdir d4/d6/d2e/d34 39 2021-11-03T17:14:12.883 INFO:tasks.workunit.client.1.smithi115.stdout:1/319: dwrite d7/dc/dd/f46 [0,4194304] 0 2021-11-03T17:14:12.883 INFO:tasks.workunit.client.1.smithi115.stdout:9/282: truncate d1/f13 3730951 0 2021-11-03T17:14:12.884 INFO:tasks.workunit.client.1.smithi115.stdout:2/287: mkdir d4/d6/d2e/d34/d43/d63 0 2021-11-03T17:14:12.884 INFO:tasks.workunit.client.1.smithi115.stdout:1/320: symlink d7/dc/dd/d24/l63 0 2021-11-03T17:14:12.884 INFO:tasks.workunit.client.1.smithi115.stdout:1/321: write d7/f53 [788965,27794] 0 2021-11-03T17:14:12.884 INFO:tasks.workunit.client.1.smithi115.stdout:1/322: fdatasync d7/dc/dd/d24/d26/f3b 0 2021-11-03T17:14:12.884 INFO:tasks.workunit.client.1.smithi115.stdout:9/283: readlink d1/d18/d2b/d42/l11 0 2021-11-03T17:14:12.885 INFO:tasks.workunit.client.1.smithi115.stdout:9/284: write d1/d18/d2b/d42/ff [4925362,130755] 0 2021-11-03T17:14:12.885 INFO:tasks.workunit.client.1.smithi115.stdout:2/288: unlink d4/d6/d13/l32 0 2021-11-03T17:14:12.885 INFO:tasks.workunit.client.1.smithi115.stdout:5/416: dwrite d1a/d3d/f62 [4194304,4194304] 0 2021-11-03T17:14:12.885 INFO:tasks.workunit.client.1.smithi115.stdout:5/417: write d1a/f1d [2571639,61386] 0 2021-11-03T17:14:12.886 INFO:tasks.workunit.client.1.smithi115.stdout:1/323: truncate d7/dc/dd/d24/f28 521454 0 2021-11-03T17:14:12.886 INFO:tasks.workunit.client.1.smithi115.stdout:1/324: write d7/dc/dd/d24/d4d/d60/f62 [520590,111047] 0 2021-11-03T17:14:12.886 INFO:tasks.workunit.client.1.smithi115.stdout:1/325: chown f2 15 1 2021-11-03T17:14:12.886 INFO:tasks.workunit.client.1.smithi115.stdout:2/289: symlink d4/l64 0 2021-11-03T17:14:12.886 INFO:tasks.workunit.client.1.smithi115.stdout:2/290: dread - d4/d6/d2e/d34/f3f zero size 2021-11-03T17:14:12.887 INFO:tasks.workunit.client.1.smithi115.stdout:3/390: dread d13/f27 [0,4194304] 0 2021-11-03T17:14:12.887 INFO:tasks.workunit.client.1.smithi115.stdout:1/326: read d7/dc/dd/d24/d43/f3c [752982,72632] 0 2021-11-03T17:14:12.887 INFO:tasks.workunit.client.1.smithi115.stdout:1/327: chown f2 740 1 2021-11-03T17:14:12.887 INFO:tasks.workunit.client.1.smithi115.stdout:5/418: symlink d1a/d28/d7b/l90 0 2021-11-03T17:14:12.888 INFO:tasks.workunit.client.1.smithi115.stdout:5/419: dread - d1a/d3d/d58/f7c zero size 2021-11-03T17:14:12.888 INFO:tasks.workunit.client.1.smithi115.stdout:9/285: mkdir d1/d66 0 2021-11-03T17:14:12.888 INFO:tasks.workunit.client.1.smithi115.stdout:2/291: getdents d4/d44 0 2021-11-03T17:14:12.889 INFO:tasks.workunit.client.1.smithi115.stdout:5/420: truncate d1a/d3d/f47 3622105 0 2021-11-03T17:14:12.889 INFO:tasks.workunit.client.1.smithi115.stdout:5/421: fdatasync d1a/d3d/f62 0 2021-11-03T17:14:12.889 INFO:tasks.workunit.client.1.smithi115.stdout:5/422: chown d1a/d23/f5c 71521102 1 2021-11-03T17:14:12.890 INFO:tasks.workunit.client.1.smithi115.stdout:5/423: creat d1a/d28/f91 x:0 0 0 2021-11-03T17:14:12.890 INFO:tasks.workunit.client.1.smithi115.stdout:5/424: chown d1a/d28/l2e 21400769 1 2021-11-03T17:14:12.890 INFO:tasks.workunit.client.1.smithi115.stdout:5/425: truncate d1a/d28/d4a/d65/f6b 1576777 0 2021-11-03T17:14:12.890 INFO:tasks.workunit.client.1.smithi115.stdout:5/426: fdatasync d1a/d28/d4c/d75/d83/f88 0 2021-11-03T17:14:12.891 INFO:tasks.workunit.client.1.smithi115.stdout:5/427: mknod d1a/d28/d34/c92 1 2021-11-03T17:14:12.891 INFO:tasks.workunit.client.0.smithi027.stdout:9/377: dwrite db/d1b/f51 [0,4194304] 0 2021-11-03T17:14:12.891 INFO:tasks.workunit.client.0.smithi027.stdout:9/378: mknod db/d1b/d2b/d41/c88 1 2021-11-03T17:14:12.893 INFO:tasks.workunit.client.1.smithi115.stdout:9/286: symlink d1/l67 0 2021-11-03T17:14:12.904 INFO:tasks.workunit.client.1.smithi115.stdout:9/287: symlink d1/d1a/d48/d27/d34/d50/l68 0 2021-11-03T17:14:12.906 INFO:tasks.workunit.client.1.smithi115.stdout:1/328: rename d7/dc/dd/d24/f28 to d7/dc/dd/d3e/d58/f64 0 2021-11-03T17:14:12.906 INFO:tasks.workunit.client.1.smithi115.stdout:1/329: read d7/f53 [3325,80429] 0 2021-11-03T17:14:12.907 INFO:tasks.workunit.client.1.smithi115.stdout:1/330: truncate d7/dc/dd/d24/d4d/d60/f62 760321 0 2021-11-03T17:14:12.919 INFO:tasks.workunit.client.1.smithi115.stdout:1/331: dread d7/dc/ff [0,4194304] 0 2021-11-03T17:14:12.920 INFO:tasks.workunit.client.1.smithi115.stdout:1/332: creat d7/dc/dd/f65 x:0 0 0 2021-11-03T17:14:12.921 INFO:tasks.workunit.client.1.smithi115.stdout:1/333: getdents d7/dc/d1a 0 2021-11-03T17:14:12.924 INFO:tasks.workunit.client.1.smithi115.stdout:9/288: dwrite d1/d1a/d48/d27/f47 [0,4194304] 0 2021-11-03T17:14:12.924 INFO:tasks.workunit.client.1.smithi115.stdout:9/289: truncate d1/d18/d2b/d42/f21 1903377 0 2021-11-03T17:14:12.924 INFO:tasks.workunit.client.1.smithi115.stdout:9/290: write d1/d18/d2b/d42/f12 [2379293,15657] 0 2021-11-03T17:14:12.925 INFO:tasks.workunit.client.1.smithi115.stdout:3/391: dwrite d13/f21 [0,4194304] 0 2021-11-03T17:14:12.927 INFO:tasks.workunit.client.0.smithi027.stdout:6/370: dwrite dc/d16/d1b/d3c/f44 [4194304,4194304] 0 2021-11-03T17:14:12.927 INFO:tasks.workunit.client.1.smithi115.stdout:9/291: rename d1/d1a/d48/d27/d2f/l51 to d1/d18/d2b/d42/d63/l69 0 2021-11-03T17:14:12.928 INFO:tasks.workunit.client.1.smithi115.stdout:2/292: dwrite d4/d6/d2e/d34/d43/d28/f41 [8388608,4194304] 0 2021-11-03T17:14:12.928 INFO:tasks.workunit.client.1.smithi115.stdout:2/293: chown d4/d6/d13/f4e 842890 1 2021-11-03T17:14:12.928 INFO:tasks.workunit.client.1.smithi115.stdout:1/334: write d7/dc/d1a/f3d [93496,89169] 0 2021-11-03T17:14:12.929 INFO:tasks.workunit.client.1.smithi115.stdout:9/292: write d1/d18/f22 [442253,25021] 0 2021-11-03T17:14:12.929 INFO:tasks.workunit.client.0.smithi027.stdout:9/379: dwrite db/d1b/d23/f73 [0,4194304] 0 2021-11-03T17:14:12.930 INFO:tasks.workunit.client.0.smithi027.stdout:6/371: creat dc/d16/d30/f65 x:0 0 0 2021-11-03T17:14:12.931 INFO:tasks.workunit.client.1.smithi115.stdout:1/335: creat d7/dc/dd/d24/d4b/d55/f66 x:0 0 0 2021-11-03T17:14:12.931 INFO:tasks.workunit.client.1.smithi115.stdout:1/336: stat d7/dc/d1a/f35 0 2021-11-03T17:14:12.931 INFO:tasks.workunit.client.1.smithi115.stdout:1/337: write d7/d38/f5c [40832,33127] 0 2021-11-03T17:14:12.931 INFO:tasks.workunit.client.1.smithi115.stdout:1/338: chown d7/d38 3 1 2021-11-03T17:14:12.931 INFO:tasks.workunit.client.1.smithi115.stdout:9/293: dread d1/d18/d2b/f35 [0,4194304] 0 2021-11-03T17:14:12.933 INFO:tasks.workunit.client.1.smithi115.stdout:5/428: dread d1a/d3d/f47 [0,4194304] 0 2021-11-03T17:14:12.933 INFO:tasks.workunit.client.1.smithi115.stdout:5/429: stat d1a/d23/f5c 0 2021-11-03T17:14:12.934 INFO:tasks.workunit.client.1.smithi115.stdout:5/430: stat d1a/d28/d7b/d48/d7f/f86 0 2021-11-03T17:14:12.934 INFO:tasks.workunit.client.1.smithi115.stdout:5/431: chown d1a/d28/f31 113545 1 2021-11-03T17:14:12.934 INFO:tasks.workunit.client.1.smithi115.stdout:1/339: rename d7/dc/l1c to d7/dc/l67 0 2021-11-03T17:14:12.934 INFO:tasks.workunit.client.1.smithi115.stdout:1/340: read d7/d38/f5c [67827,99461] 0 2021-11-03T17:14:12.935 INFO:tasks.workunit.client.0.smithi027.stdout:6/372: read dc/d16/d30/f38 [699663,108467] 0 2021-11-03T17:14:12.935 INFO:tasks.workunit.client.0.smithi027.stdout:6/373: mknod dc/d16/d1b/d24/c66 1 2021-11-03T17:14:12.935 INFO:tasks.workunit.client.0.smithi027.stdout:9/380: creat db/d1b/d23/d2c/f89 x:0 0 0 2021-11-03T17:14:12.935 INFO:tasks.workunit.client.0.smithi027.stdout:9/381: chown db/d1b/d1c/d1d/d40/f70 99099 1 2021-11-03T17:14:12.936 INFO:tasks.workunit.client.0.smithi027.stdout:9/382: write db/f17 [5020727,34985] 0 2021-11-03T17:14:12.936 INFO:tasks.workunit.client.0.smithi027.stdout:9/383: write db/d1b/d2b/d41/f74 [686995,37768] 0 2021-11-03T17:14:12.936 INFO:tasks.workunit.client.0.smithi027.stdout:9/384: mknod db/d1b/d23/d2c/c8a 1 2021-11-03T17:14:12.937 INFO:tasks.workunit.client.0.smithi027.stdout:6/374: symlink dc/d16/d30/l67 0 2021-11-03T17:14:12.937 INFO:tasks.workunit.client.0.smithi027.stdout:6/375: write dc/f64 [90143,116349] 0 2021-11-03T17:14:12.938 INFO:tasks.workunit.client.1.smithi115.stdout:9/294: mkdir d1/d1a/d48/d27/d2f/d6a 0 2021-11-03T17:14:12.938 INFO:tasks.workunit.client.1.smithi115.stdout:1/341: creat d7/d38/d59/f68 x:0 0 0 2021-11-03T17:14:12.938 INFO:tasks.workunit.client.0.smithi027.stdout:9/385: symlink db/d1b/d2b/d41/l8b 0 2021-11-03T17:14:12.939 INFO:tasks.workunit.client.0.smithi027.stdout:6/376: creat dc/d16/d1b/d2d/f68 x:0 0 0 2021-11-03T17:14:12.940 INFO:tasks.workunit.client.0.smithi027.stdout:6/377: stat f7 0 2021-11-03T17:14:12.940 INFO:tasks.workunit.client.0.smithi027.stdout:6/378: chown dc/d16/d1b/f28 50 1 2021-11-03T17:14:12.942 INFO:tasks.workunit.client.0.smithi027.stdout:6/379: symlink dc/d16/d1b/d2d/l69 0 2021-11-03T17:14:12.942 INFO:tasks.workunit.client.0.smithi027.stdout:6/380: stat f7 0 2021-11-03T17:14:12.945 INFO:tasks.workunit.client.0.smithi027.stdout:0/394: sync 2021-11-03T17:14:12.946 INFO:tasks.workunit.client.1.smithi115.stdout:3/392: dwrite d13/d35/f3d [0,4194304] 0 2021-11-03T17:14:12.951 INFO:tasks.workunit.client.0.smithi027.stdout:6/381: dread dc/d16/d30/f38 [0,4194304] 0 2021-11-03T17:14:12.951 INFO:tasks.workunit.client.0.smithi027.stdout:6/382: dread - dc/d16/d1b/d3c/d51/f5b zero size 2021-11-03T17:14:12.951 INFO:tasks.workunit.client.0.smithi027.stdout:6/383: write dc/d16/f40 [4413374,3018] 0 2021-11-03T17:14:12.953 INFO:tasks.workunit.client.1.smithi115.stdout:2/294: dwrite d4/d6/f54 [0,4194304] 0 2021-11-03T17:14:12.955 INFO:tasks.workunit.client.1.smithi115.stdout:2/295: creat d4/d6/d2e/f65 x:0 0 0 2021-11-03T17:14:12.955 INFO:tasks.workunit.client.1.smithi115.stdout:2/296: fdatasync d4/d6/f2a 0 2021-11-03T17:14:12.956 INFO:tasks.workunit.client.1.smithi115.stdout:2/297: chown d4/d6/f45 6 1 2021-11-03T17:14:12.962 INFO:tasks.workunit.client.0.smithi027.stdout:9/386: dwrite db/d1b/d23/d2c/f6a [0,4194304] 0 2021-11-03T17:14:12.964 INFO:tasks.workunit.client.0.smithi027.stdout:0/395: dwrite d9/da/d31/f44 [0,4194304] 0 2021-11-03T17:14:12.964 INFO:tasks.workunit.client.0.smithi027.stdout:0/396: getdents d9/d81 0 2021-11-03T17:14:12.964 INFO:tasks.workunit.client.0.smithi027.stdout:0/397: chown d9/f1f 200 1 2021-11-03T17:14:12.964 INFO:tasks.workunit.client.0.smithi027.stdout:0/398: getdents d9/da/d64/d70/d6d/d7a 0 2021-11-03T17:14:12.967 INFO:tasks.workunit.client.0.smithi027.stdout:9/387: dread db/d1b/d23/d2c/f4c [0,4194304] 0 2021-11-03T17:14:12.968 INFO:tasks.workunit.client.0.smithi027.stdout:9/388: write db/d1b/d2b/d41/f74 [1516016,20762] 0 2021-11-03T17:14:12.969 INFO:tasks.workunit.client.0.smithi027.stdout:0/399: creat d9/da/d64/d70/d5f/f84 x:0 0 0 2021-11-03T17:14:12.971 INFO:tasks.workunit.client.0.smithi027.stdout:9/389: mkdir db/d1b/d8c 0 2021-11-03T17:14:12.972 INFO:tasks.workunit.client.1.smithi115.stdout:2/298: dwrite d4/d6/f48 [0,4194304] 0 2021-11-03T17:14:12.973 INFO:tasks.workunit.client.1.smithi115.stdout:2/299: chown d4 484 1 2021-11-03T17:14:12.974 INFO:tasks.workunit.client.0.smithi027.stdout:0/400: symlink d9/da/d31/l85 0 2021-11-03T17:14:12.974 INFO:tasks.workunit.client.0.smithi027.stdout:0/401: mknod d9/da/c86 1 2021-11-03T17:14:12.975 INFO:tasks.workunit.client.0.smithi027.stdout:0/402: readlink d9/da/d30/l59 0 2021-11-03T17:14:12.975 INFO:tasks.workunit.client.0.smithi027.stdout:9/390: symlink db/d1b/d8c/l8d 0 2021-11-03T17:14:12.976 INFO:tasks.workunit.client.1.smithi115.stdout:2/300: truncate d4/d6/d2e/d34/d43/f2b 3756881 0 2021-11-03T17:14:12.976 INFO:tasks.workunit.client.1.smithi115.stdout:2/301: mknod d4/d6/d56/c66 1 2021-11-03T17:14:12.976 INFO:tasks.workunit.client.1.smithi115.stdout:2/302: chown d4/f5a 987415 1 2021-11-03T17:14:12.977 INFO:tasks.workunit.client.1.smithi115.stdout:2/303: readlink d4/d6/d2e/d34/l55 0 2021-11-03T17:14:12.977 INFO:tasks.workunit.client.1.smithi115.stdout:2/304: mknod d4/d6/d2e/d34/d42/d57/c67 1 2021-11-03T17:14:12.977 INFO:tasks.workunit.client.1.smithi115.stdout:2/305: chown d4/d6/d13/f21 121 1 2021-11-03T17:14:12.977 INFO:tasks.workunit.client.1.smithi115.stdout:2/306: rmdir d4/d44 39 2021-11-03T17:14:12.978 INFO:tasks.workunit.client.1.smithi115.stdout:2/307: rename d4/d6/f54 to d4/f68 0 2021-11-03T17:14:12.978 INFO:tasks.workunit.client.1.smithi115.stdout:2/308: fsync d4/d6/d2e/f3c 0 2021-11-03T17:14:12.979 INFO:tasks.workunit.client.1.smithi115.stdout:2/309: mkdir d4/d6/d2e/d34/d43/d63/d69 0 2021-11-03T17:14:12.979 INFO:tasks.workunit.client.0.smithi027.stdout:9/391: mkdir db/d1b/d8e 0 2021-11-03T17:14:12.979 INFO:tasks.workunit.client.0.smithi027.stdout:9/392: chown db/d1b/d2b/f2a 0 1 2021-11-03T17:14:12.980 INFO:tasks.workunit.client.0.smithi027.stdout:9/393: truncate db/f5b 2312996 0 2021-11-03T17:14:12.981 INFO:tasks.workunit.client.0.smithi027.stdout:9/394: rename db/d1b/d1c/f3e to db/d1b/d1c/d1d/f8f 0 2021-11-03T17:14:12.982 INFO:tasks.workunit.client.0.smithi027.stdout:9/395: mknod db/d1b/d8e/c90 1 2021-11-03T17:14:12.982 INFO:tasks.workunit.client.0.smithi027.stdout:9/396: fsync db/d1b/d23/d2c/f4c 0 2021-11-03T17:14:12.984 INFO:tasks.workunit.client.0.smithi027.stdout:9/397: link db/d1b/d1c/d1d/d40/f53 db/d1b/d2b/f91 0 2021-11-03T17:14:12.986 INFO:tasks.workunit.client.0.smithi027.stdout:0/403: read d9/da/d31/f48 [942288,32011] 0 2021-11-03T17:14:12.988 INFO:tasks.workunit.client.0.smithi027.stdout:0/404: truncate d9/da/fe 1267723 0 2021-11-03T17:14:12.988 INFO:tasks.workunit.client.0.smithi027.stdout:0/405: stat d9/da/l11 0 2021-11-03T17:14:12.991 INFO:tasks.workunit.client.0.smithi027.stdout:0/406: mkdir d9/da/d87 0 2021-11-03T17:14:12.992 INFO:tasks.workunit.client.1.smithi115.stdout:9/295: write d1/d18/d2b/d42/f12 [234395,50717] 0 2021-11-03T17:14:12.992 INFO:tasks.workunit.client.1.smithi115.stdout:9/296: write d1/d1a/d48/f2a [1848801,52089] 0 2021-11-03T17:14:12.993 INFO:tasks.workunit.client.0.smithi027.stdout:9/398: rmdir db/d1b/d1c/d1d/d40/d60/d71 0 2021-11-03T17:14:12.994 INFO:tasks.workunit.client.0.smithi027.stdout:9/399: mknod db/d1b/d8e/c92 1 2021-11-03T17:14:12.997 INFO:tasks.workunit.client.0.smithi027.stdout:9/400: rename db/d1b/d1c/d1d/d40/d60/f7c to db/d1b/d2b/d41/d6c/f93 0 2021-11-03T17:14:12.998 INFO:tasks.workunit.client.0.smithi027.stdout:0/407: dread d9/da/f16 [0,4194304] 0 2021-11-03T17:14:12.998 INFO:tasks.workunit.client.0.smithi027.stdout:6/384: write dc/d16/d1b/d24/f46 [1540882,119137] 0 2021-11-03T17:14:12.999 INFO:tasks.workunit.client.0.smithi027.stdout:9/401: symlink db/d1b/d2b/d41/d6c/l94 0 2021-11-03T17:14:13.000 INFO:tasks.workunit.client.1.smithi115.stdout:2/310: dread d4/d6/d2e/f39 [0,4194304] 0 2021-11-03T17:14:13.000 INFO:tasks.workunit.client.1.smithi115.stdout:2/311: read - d4/d6/d13/f21 zero size 2021-11-03T17:14:13.000 INFO:tasks.workunit.client.1.smithi115.stdout:2/312: readlink d4/l4b 0 2021-11-03T17:14:13.001 INFO:tasks.workunit.client.1.smithi115.stdout:2/313: getdents d4/d6/d2e/d34/d43/d63/d69 0 2021-11-03T17:14:13.002 INFO:tasks.workunit.client.0.smithi027.stdout:0/408: rmdir d9/da/d87 0 2021-11-03T17:14:13.007 INFO:tasks.workunit.client.0.smithi027.stdout:6/385: mkdir dc/d16/d1b/d24/d6a 0 2021-11-03T17:14:13.008 INFO:tasks.workunit.client.0.smithi027.stdout:0/409: write d9/f2a [2244029,96402] 0 2021-11-03T17:14:13.010 INFO:tasks.workunit.client.0.smithi027.stdout:6/386: dread fa [0,4194304] 0 2021-11-03T17:14:13.011 INFO:tasks.workunit.client.1.smithi115.stdout:1/342: dwrite d7/dc/dd/f46 [0,4194304] 0 2021-11-03T17:14:13.012 INFO:tasks.workunit.client.1.smithi115.stdout:1/343: truncate d7/dc/d1a/f35 857541 0 2021-11-03T17:14:13.013 INFO:tasks.workunit.client.0.smithi027.stdout:9/402: write db/d1b/d23/f73 [3261449,57264] 0 2021-11-03T17:14:13.013 INFO:tasks.workunit.client.0.smithi027.stdout:6/387: link dc/l29 dc/d16/d1b/d3c/l6b 0 2021-11-03T17:14:13.013 INFO:tasks.workunit.client.1.smithi115.stdout:3/393: dwrite d13/d51/f41 [4194304,4194304] 0 2021-11-03T17:14:13.014 INFO:tasks.workunit.client.1.smithi115.stdout:2/314: symlink d4/d6/d2e/d34/d43/l6a 0 2021-11-03T17:14:13.014 INFO:tasks.workunit.client.1.smithi115.stdout:1/344: creat d7/dc/dd/d24/d4d/f69 x:0 0 0 2021-11-03T17:14:13.015 INFO:tasks.workunit.client.0.smithi027.stdout:6/388: readlink dc/d16/d1b/d3c/l3e 0 2021-11-03T17:14:13.015 INFO:tasks.workunit.client.1.smithi115.stdout:3/394: creat d13/d67/f87 x:0 0 0 2021-11-03T17:14:13.015 INFO:tasks.workunit.client.1.smithi115.stdout:3/395: fdatasync d13/d51/f29 0 2021-11-03T17:14:13.015 INFO:tasks.workunit.client.1.smithi115.stdout:3/396: dread - d13/d2a/f5d zero size 2021-11-03T17:14:13.016 INFO:tasks.workunit.client.1.smithi115.stdout:2/315: creat d4/d6/d13/f6b x:0 0 0 2021-11-03T17:14:13.017 INFO:tasks.workunit.client.1.smithi115.stdout:1/345: symlink d7/dc/dd/d24/d4b/d55/l6a 0 2021-11-03T17:14:13.017 INFO:tasks.workunit.client.1.smithi115.stdout:2/316: chown d4/d6/d2e/d34/d43/d28/f41 55925 1 2021-11-03T17:14:13.017 INFO:tasks.workunit.client.1.smithi115.stdout:2/317: read - d4/d6/d2e/d34/d42/d57/f60 zero size 2021-11-03T17:14:13.018 INFO:tasks.workunit.client.0.smithi027.stdout:6/389: rename dc/f1e to dc/d16/d1b/d3c/f6c 0 2021-11-03T17:14:13.019 INFO:tasks.workunit.client.1.smithi115.stdout:1/346: creat d7/dc/d1a/f6b x:0 0 0 2021-11-03T17:14:13.020 INFO:tasks.workunit.client.1.smithi115.stdout:1/347: creat d7/dc/dd/d24/d4b/d55/f6c x:0 0 0 2021-11-03T17:14:13.023 INFO:tasks.workunit.client.1.smithi115.stdout:9/297: dwrite d1/d1a/d48/d27/d2f/d41/f60 [0,4194304] 0 2021-11-03T17:14:13.023 INFO:tasks.workunit.client.1.smithi115.stdout:9/298: fsync d1/d18/d2b/d42/fc 0 2021-11-03T17:14:13.023 INFO:tasks.workunit.client.1.smithi115.stdout:9/299: mknod d1/d1a/d48/d27/d34/d50/c6b 1 2021-11-03T17:14:13.024 INFO:tasks.workunit.client.1.smithi115.stdout:9/300: truncate d1/d1a/d48/f62 178639 0 2021-11-03T17:14:13.027 INFO:tasks.workunit.client.0.smithi027.stdout:6/390: dread dc/d16/d1b/d3c/f6c [0,4194304] 0 2021-11-03T17:14:13.028 INFO:tasks.workunit.client.1.smithi115.stdout:2/318: rename d4/d6/d2e to d4/d6/d56/d6c 0 2021-11-03T17:14:13.029 INFO:tasks.workunit.client.0.smithi027.stdout:6/391: symlink dc/d16/l6d 0 2021-11-03T17:14:13.031 INFO:tasks.workunit.client.0.smithi027.stdout:6/392: rmdir dc/d16/d1b/d24/d6a 0 2021-11-03T17:14:13.031 INFO:tasks.workunit.client.0.smithi027.stdout:6/393: fsync dc/d16/d30/f38 0 2021-11-03T17:14:13.032 INFO:tasks.workunit.client.1.smithi115.stdout:1/348: rename d7/dc/dd/f65 to d7/dc/dd/d24/d4d/d60/f6d 0 2021-11-03T17:14:13.034 INFO:tasks.workunit.client.1.smithi115.stdout:9/301: rename d1/d18/d2b/f5b to d1/d66/f6c 0 2021-11-03T17:14:13.036 INFO:tasks.workunit.client.1.smithi115.stdout:3/397: dwrite d13/d35/d54/f6f [0,4194304] 0 2021-11-03T17:14:13.036 INFO:tasks.workunit.client.1.smithi115.stdout:3/398: dread - d13/d51/f55 zero size 2021-11-03T17:14:13.038 INFO:tasks.workunit.client.0.smithi027.stdout:9/403: dwrite db/d1b/d23/d2c/f7d [0,4194304] 0 2021-11-03T17:14:13.039 INFO:tasks.workunit.client.0.smithi027.stdout:9/404: fdatasync db/d1b/d23/d2c/f7d 0 2021-11-03T17:14:13.042 INFO:tasks.workunit.client.0.smithi027.stdout:9/405: mkdir db/d1b/d2b/d41/d95 0 2021-11-03T17:14:13.045 INFO:tasks.workunit.client.0.smithi027.stdout:9/406: symlink db/d1b/d2b/d41/l96 0 2021-11-03T17:14:13.045 INFO:tasks.workunit.client.0.smithi027.stdout:9/407: chown db/d1b/d8c/l8d 32141 1 2021-11-03T17:14:13.048 INFO:tasks.workunit.client.1.smithi115.stdout:3/399: dread d13/d51/d40/f5a [0,4194304] 0 2021-11-03T17:14:13.048 INFO:tasks.workunit.client.1.smithi115.stdout:3/400: stat d13/d51/d40/l6b 0 2021-11-03T17:14:13.048 INFO:tasks.workunit.client.1.smithi115.stdout:3/401: fdatasync d13/d35/f69 0 2021-11-03T17:14:13.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.057+0000 7f08498d5700 1 -- 172.21.15.27:0/3275190924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08440f7480 msgr2=0x7f08440f9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:14:13.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.057+0000 7f08498d5700 1 --2- 172.21.15.27:0/3275190924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08440f7480 0x7f08440f9910 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f0830004660 tx=0x7f0830009b30).stop 2021-11-03T17:14:13.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.057+0000 7f08498d5700 1 -- 172.21.15.27:0/3275190924 shutdown_connections 2021-11-03T17:14:13.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.057+0000 7f08498d5700 1 --2- 172.21.15.27:0/3275190924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08440f7480 0x7f08440f9910 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:13.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.057+0000 7f08498d5700 1 --2- 172.21.15.27:0/3275190924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0844060310 0x7f08440f6f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:13.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.057+0000 7f08498d5700 1 -- 172.21.15.27:0/3275190924 >> 172.21.15.27:0/3275190924 conn(0x7f08440f0b90 msgr2=0x7f08440f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:14:13.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.057+0000 7f08498d5700 1 -- 172.21.15.27:0/3275190924 shutdown_connections 2021-11-03T17:14:13.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.057+0000 7f08498d5700 1 -- 172.21.15.27:0/3275190924 wait complete. 2021-11-03T17:14:13.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.057+0000 7f08498d5700 1 Processor -- start 2021-11-03T17:14:13.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.057+0000 7f08498d5700 1 -- start start 2021-11-03T17:14:13.055 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.058+0000 7f08498d5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0844060310 0x7f0844104b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:14:13.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.058+0000 7f08498d5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08440f7480 0x7f0844105000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:14:13.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.058+0000 7f08498d5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08441055b0 con 0x7f0844060310 2021-11-03T17:14:13.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.058+0000 7f08498d5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08441056f0 con 0x7f08440f7480 2021-11-03T17:14:13.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.058+0000 7f08488d3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0844060310 0x7f0844104b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:14:13.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.058+0000 7f08488d3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0844060310 0x7f0844104b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55328/0 (socket says 172.21.15.27:55328) 2021-11-03T17:14:13.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.058+0000 7f08488d3700 1 -- 172.21.15.27:0/816658946 learned_addr learned my addr 172.21.15.27:0/816658946 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:14:13.059 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.058+0000 7f0843fff700 1 --2- 172.21.15.27:0/816658946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08440f7480 0x7f0844105000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:14:13.059 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.059+0000 7f08488d3700 1 -- 172.21.15.27:0/816658946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08440f7480 msgr2=0x7f0844105000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:14:13.059 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.059+0000 7f08488d3700 1 --2- 172.21.15.27:0/816658946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08440f7480 0x7f0844105000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:13.060 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.059+0000 7f08488d3700 1 -- 172.21.15.27:0/816658946 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0830005040 con 0x7f0844060310 2021-11-03T17:14:13.060 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.059+0000 7f08488d3700 1 --2- 172.21.15.27:0/816658946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0844060310 0x7f0844104b10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f08380041d0 tx=0x7f083801adf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:14:13.061 INFO:tasks.workunit.client.0.smithi027.stdout:6/394: dwrite dc/d16/d1b/d24/f2a [0,4194304] 0 2021-11-03T17:14:13.061 INFO:tasks.workunit.client.1.smithi115.stdout:2/319: dwrite d4/d6/f12 [0,4194304] 0 2021-11-03T17:14:13.062 INFO:tasks.workunit.client.1.smithi115.stdout:2/320: fdatasync d4/d6/d13/f38 0 2021-11-03T17:14:13.063 INFO:tasks.workunit.client.0.smithi027.stdout:6/395: rmdir dc/d16/d1b 39 2021-11-03T17:14:13.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.076+0000 7f08417fa700 1 -- 172.21.15.27:0/816658946 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08380189d0 con 0x7f0844060310 2021-11-03T17:14:13.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.076+0000 7f08417fa700 1 -- 172.21.15.27:0/816658946 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f0838020040 con 0x7f0844060310 2021-11-03T17:14:13.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.076+0000 7f08498d5700 1 -- 172.21.15.27:0/816658946 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0844106090 con 0x7f0844060310 2021-11-03T17:14:13.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.076+0000 7f08417fa700 1 -- 172.21.15.27:0/816658946 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0838025800 con 0x7f0844060310 2021-11-03T17:14:13.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.076+0000 7f08498d5700 1 -- 172.21.15.27:0/816658946 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0844106650 con 0x7f0844060310 2021-11-03T17:14:13.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.078+0000 7f08417fa700 1 -- 172.21.15.27:0/816658946 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f083802c060 con 0x7f0844060310 2021-11-03T17:14:13.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.078+0000 7f08417fa700 1 --2- 172.21.15.27:0/816658946 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f083405c300 0x7f083405e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:14:13.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.079+0000 7f08417fa700 1 -- 172.21.15.27:0/816658946 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f0838086cf0 con 0x7f0844060310 2021-11-03T17:14:13.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.079+0000 7f08498d5700 1 -- 172.21.15.27:0/816658946 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0844059100 con 0x7f0844060310 2021-11-03T17:14:13.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.079+0000 7f0843fff700 1 --2- 172.21.15.27:0/816658946 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f083405c300 0x7f083405e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:14:13.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.079+0000 7f0843fff700 1 --2- 172.21.15.27:0/816658946 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f083405c300 0x7f083405e7c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f0830004630 tx=0x7f0830009b30).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:14:13.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.083+0000 7f08417fa700 1 -- 172.21.15.27:0/816658946 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f0838059a50 con 0x7f0844060310 2021-11-03T17:14:13.084 INFO:tasks.workunit.client.1.smithi115.stdout:9/302: dwrite d1/d39/f52 [0,4194304] 0 2021-11-03T17:14:13.084 INFO:tasks.workunit.client.1.smithi115.stdout:1/349: dwrite d7/dc/dd/d24/d26/f4f [0,4194304] 0 2021-11-03T17:14:13.085 INFO:tasks.workunit.client.1.smithi115.stdout:3/402: dwrite d13/d2a/f5d [0,4194304] 0 2021-11-03T17:14:13.086 INFO:tasks.workunit.client.1.smithi115.stdout:3/403: write d13/d51/f55 [59976,9939] 0 2021-11-03T17:14:13.086 INFO:tasks.workunit.client.1.smithi115.stdout:3/404: readlink d13/d51/d40/l6b 0 2021-11-03T17:14:13.087 INFO:tasks.workunit.client.0.smithi027.stdout:0/410: fsync d9/da/d64/d70/d5f/f84 0 2021-11-03T17:14:13.087 INFO:tasks.workunit.client.1.smithi115.stdout:9/303: link d1/d1a/d48/d27/f3d d1/d18/d4a/d54/d3a/f6d 0 2021-11-03T17:14:13.088 INFO:tasks.workunit.client.1.smithi115.stdout:3/405: write d13/d3b/f58 [3818577,25097] 0 2021-11-03T17:14:13.089 INFO:tasks.workunit.client.1.smithi115.stdout:1/350: symlink d7/dc/dd/d42/l6e 0 2021-11-03T17:14:13.089 INFO:tasks.workunit.client.1.smithi115.stdout:1/351: write d7/dc/d1a/f6b [104103,103915] 0 2021-11-03T17:14:13.091 INFO:tasks.workunit.client.1.smithi115.stdout:2/321: dwrite d4/d6/f48 [0,4194304] 0 2021-11-03T17:14:13.093 INFO:tasks.workunit.client.1.smithi115.stdout:9/304: rename d1/d39 to d1/d18/d2b/d42/d6e 0 2021-11-03T17:14:13.093 INFO:tasks.workunit.client.1.smithi115.stdout:9/305: fsync d1/d18/d2b/d42/f21 0 2021-11-03T17:14:13.093 INFO:tasks.workunit.client.1.smithi115.stdout:9/306: mknod d1/d18/d4a/d54/c6f 1 2021-11-03T17:14:13.094 INFO:tasks.workunit.client.1.smithi115.stdout:9/307: fsync d1/d18/f22 0 2021-11-03T17:14:13.094 INFO:tasks.workunit.client.1.smithi115.stdout:9/308: write d1/d1a/f28 [1651922,70504] 0 2021-11-03T17:14:13.095 INFO:tasks.workunit.client.1.smithi115.stdout:9/309: read d1/d1a/d48/f2a [78393,41409] 0 2021-11-03T17:14:13.095 INFO:tasks.workunit.client.1.smithi115.stdout:2/322: dread d4/d6/d56/d6c/f46 [0,4194304] 0 2021-11-03T17:14:13.097 INFO:tasks.workunit.client.1.smithi115.stdout:3/406: rename d13/d2a/f5d to d13/d51/d40/f88 0 2021-11-03T17:14:13.098 INFO:tasks.workunit.client.0.smithi027.stdout:6/396: dwrite dc/d16/d1b/d3c/d48/f5c [0,4194304] 0 2021-11-03T17:14:13.102 INFO:tasks.workunit.client.1.smithi115.stdout:1/352: truncate d7/dc/d1a/f35 113569 0 2021-11-03T17:14:13.103 INFO:tasks.workunit.client.1.smithi115.stdout:3/407: truncate d13/d51/d2e/f59 3587637 0 2021-11-03T17:14:13.105 INFO:tasks.workunit.client.1.smithi115.stdout:3/408: read d13/d35/f36 [128271,76476] 0 2021-11-03T17:14:13.108 INFO:tasks.workunit.client.1.smithi115.stdout:1/353: rename d7/f9 to d7/dc/dd/d24/d43/f6f 0 2021-11-03T17:14:13.110 INFO:tasks.workunit.client.0.smithi027.stdout:0/411: symlink d9/da/d31/l88 0 2021-11-03T17:14:13.111 INFO:tasks.workunit.client.1.smithi115.stdout:3/409: link d13/d67/f7e d13/d3b/f89 0 2021-11-03T17:14:13.111 INFO:tasks.workunit.client.1.smithi115.stdout:3/410: write d13/d67/f87 [735727,82615] 0 2021-11-03T17:14:13.114 INFO:tasks.workunit.client.1.smithi115.stdout:3/411: mkdir d13/d2a/d8a 0 2021-11-03T17:14:13.117 INFO:tasks.workunit.client.0.smithi027.stdout:6/397: dwrite dc/f4d [4194304,4194304] 0 2021-11-03T17:14:13.118 INFO:tasks.workunit.client.1.smithi115.stdout:1/354: dread d7/dc/dd/d42/f61 [0,4194304] 0 2021-11-03T17:14:13.119 INFO:tasks.workunit.client.0.smithi027.stdout:6/398: truncate dc/f18 3585086 0 2021-11-03T17:14:13.120 INFO:tasks.workunit.client.1.smithi115.stdout:1/355: creat d7/dc/d1a/f70 x:0 0 0 2021-11-03T17:14:13.121 INFO:tasks.workunit.client.1.smithi115.stdout:1/356: readlink d7/dc/d1a/l21 0 2021-11-03T17:14:13.121 INFO:tasks.workunit.client.0.smithi027.stdout:6/399: creat dc/d16/d1b/d3c/d48/f6e x:0 0 0 2021-11-03T17:14:13.122 INFO:tasks.workunit.client.1.smithi115.stdout:1/357: symlink d7/dc/dd/d24/d4b/l71 0 2021-11-03T17:14:13.124 INFO:tasks.workunit.client.1.smithi115.stdout:3/412: dread d13/d51/d2e/f2f [0,4194304] 0 2021-11-03T17:14:13.125 INFO:tasks.workunit.client.1.smithi115.stdout:3/413: symlink d13/d50/l8b 0 2021-11-03T17:14:13.126 INFO:tasks.workunit.client.0.smithi027.stdout:6/400: dread dc/d16/d1b/d3c/f6c [4194304,4194304] 0 2021-11-03T17:14:13.127 INFO:tasks.workunit.client.1.smithi115.stdout:9/310: dwrite d1/d1a/f28 [0,4194304] 0 2021-11-03T17:14:13.127 INFO:tasks.workunit.client.1.smithi115.stdout:9/311: truncate d1/d1a/d48/d27/d2f/f59 631729 0 2021-11-03T17:14:13.127 INFO:tasks.workunit.client.1.smithi115.stdout:3/414: truncate d13/d51/d2e/f2f 1976333 0 2021-11-03T17:14:13.129 INFO:tasks.workunit.client.0.smithi027.stdout:6/401: getdents dc/d16 0 2021-11-03T17:14:13.134 INFO:tasks.workunit.client.0.smithi027.stdout:6/402: rename dc/d16/d1b/d2d/f68 to dc/d16/d1b/d3c/d48/f6f 0 2021-11-03T17:14:13.134 INFO:tasks.workunit.client.0.smithi027.stdout:6/403: read - dc/d16/d1b/d3c/d48/d57/f60 zero size 2021-11-03T17:14:13.135 INFO:tasks.workunit.client.1.smithi115.stdout:3/415: creat d13/d2a/d53/f8c x:0 0 0 2021-11-03T17:14:13.135 INFO:tasks.workunit.client.1.smithi115.stdout:9/312: dread d1/d18/d2b/d42/f17 [0,4194304] 0 2021-11-03T17:14:13.135 INFO:tasks.workunit.client.1.smithi115.stdout:9/313: truncate d1/d18/d2b/d42/f20 2249057 0 2021-11-03T17:14:13.136 INFO:tasks.workunit.client.1.smithi115.stdout:9/314: write d1/d18/d2b/d42/f20 [1237527,100385] 0 2021-11-03T17:14:13.143 INFO:tasks.workunit.client.0.smithi027.stdout:6/404: dread dc/d16/d30/f32 [0,4194304] 0 2021-11-03T17:14:13.143 INFO:tasks.workunit.client.0.smithi027.stdout:6/405: write dc/d16/d30/f38 [958215,73768] 0 2021-11-03T17:14:13.146 INFO:tasks.workunit.client.0.smithi027.stdout:6/406: rename dc/ff to dc/d16/d1b/d3c/d51/f70 0 2021-11-03T17:14:13.146 INFO:tasks.workunit.client.0.smithi027.stdout:6/407: mknod dc/d16/c71 1 2021-11-03T17:14:13.146 INFO:tasks.workunit.client.0.smithi027.stdout:6/408: write dc/d16/d1b/d3c/f44 [9252251,64995] 0 2021-11-03T17:14:13.146 INFO:tasks.workunit.client.0.smithi027.stdout:6/409: mknod dc/d16/c72 1 2021-11-03T17:14:13.147 INFO:tasks.workunit.client.0.smithi027.stdout:6/410: truncate dc/d16/d1b/d2d/f35 532645 0 2021-11-03T17:14:13.148 INFO:tasks.workunit.client.0.smithi027.stdout:6/411: write dc/d16/d1b/d3c/d48/f5c [3142698,90767] 0 2021-11-03T17:14:13.148 INFO:tasks.workunit.client.1.smithi115.stdout:9/315: dread d1/d1a/d48/d27/f3d [0,4194304] 0 2021-11-03T17:14:13.149 INFO:tasks.workunit.client.1.smithi115.stdout:9/316: read d1/d18/d2b/d42/d6e/f52 [369097,21358] 0 2021-11-03T17:14:13.149 INFO:tasks.workunit.client.1.smithi115.stdout:9/317: mknod d1/d66/c70 1 2021-11-03T17:14:13.150 INFO:tasks.workunit.client.1.smithi115.stdout:9/318: mknod d1/d18/d4a/d54/d3a/c71 1 2021-11-03T17:14:13.157 INFO:tasks.workunit.client.1.smithi115.stdout:3/416: dwrite d13/d67/f87 [0,4194304] 0 2021-11-03T17:14:13.158 INFO:tasks.workunit.client.1.smithi115.stdout:9/319: dread d1/f25 [0,4194304] 0 2021-11-03T17:14:13.158 INFO:tasks.workunit.client.1.smithi115.stdout:3/417: write d13/d35/f36 [4092697,11202] 0 2021-11-03T17:14:13.159 INFO:tasks.workunit.client.1.smithi115.stdout:9/320: dread d1/d18/f22 [0,4194304] 0 2021-11-03T17:14:13.160 INFO:tasks.workunit.client.1.smithi115.stdout:3/418: write d13/f1c [2138615,59652] 0 2021-11-03T17:14:13.160 INFO:tasks.workunit.client.0.smithi027.stdout:6/412: rmdir dc/d16/d1b/d3c/d51 39 2021-11-03T17:14:13.161 INFO:tasks.workunit.client.0.smithi027.stdout:6/413: fdatasync dc/d16/d1b/d3c/d48/f6f 0 2021-11-03T17:14:13.161 INFO:tasks.workunit.client.1.smithi115.stdout:9/321: write d1/d18/f1c [625440,76338] 0 2021-11-03T17:14:13.162 INFO:tasks.workunit.client.1.smithi115.stdout:9/322: stat d1/d1a/d48/f26 0 2021-11-03T17:14:13.163 INFO:tasks.workunit.client.1.smithi115.stdout:9/323: creat d1/f72 x:0 0 0 2021-11-03T17:14:13.166 INFO:tasks.workunit.client.1.smithi115.stdout:9/324: fdatasync d1/d18/d2b/d42/f17 0 2021-11-03T17:14:13.167 INFO:tasks.workunit.client.0.smithi027.stdout:6/414: dread f7 [0,4194304] 0 2021-11-03T17:14:13.167 INFO:tasks.workunit.client.0.smithi027.stdout:6/415: write dc/f10 [2892568,66130] 0 2021-11-03T17:14:13.169 INFO:tasks.workunit.client.0.smithi027.stdout:6/416: creat dc/d16/d1b/d3c/d51/f73 x:0 0 0 2021-11-03T17:14:13.171 INFO:tasks.workunit.client.0.smithi027.stdout:6/417: symlink dc/d16/d1b/d24/l74 0 2021-11-03T17:14:13.172 INFO:tasks.workunit.client.0.smithi027.stdout:6/418: write dc/d16/d1b/d3c/d51/f5b [59299,65209] 0 2021-11-03T17:14:13.172 INFO:tasks.workunit.client.0.smithi027.stdout:6/419: truncate dc/d16/d1b/d3c/d51/f70 2930688 0 2021-11-03T17:14:13.173 INFO:tasks.workunit.client.1.smithi115.stdout:9/325: truncate d1/d18/f1c 3826883 0 2021-11-03T17:14:13.173 INFO:tasks.workunit.client.1.smithi115.stdout:9/326: readlink d1/d18/d2b/d42/l3f 0 2021-11-03T17:14:13.180 INFO:tasks.workunit.client.0.smithi027.stdout:6/420: dread dc/d16/f2c [0,4194304] 0 2021-11-03T17:14:13.181 INFO:tasks.workunit.client.1.smithi115.stdout:9/327: dread d1/d18/d2b/d42/f17 [0,4194304] 0 2021-11-03T17:14:13.181 INFO:tasks.workunit.client.0.smithi027.stdout:6/421: write f7 [3269834,83779] 0 2021-11-03T17:14:13.182 INFO:tasks.workunit.client.1.smithi115.stdout:9/328: read d1/d1a/d48/d27/d2f/d41/f60 [2252471,24255] 0 2021-11-03T17:14:13.182 INFO:tasks.workunit.client.1.smithi115.stdout:9/329: stat d1/d1a/d48/d27/f47 0 2021-11-03T17:14:13.183 INFO:tasks.workunit.client.1.smithi115.stdout:9/330: creat d1/d1a/d48/f73 x:0 0 0 2021-11-03T17:14:13.183 INFO:tasks.workunit.client.1.smithi115.stdout:9/331: getdents d1/d18/d2b/d42/d63 0 2021-11-03T17:14:13.184 INFO:tasks.workunit.client.1.smithi115.stdout:9/332: chown d1/d1a/d48/d27/f3d 0 1 2021-11-03T17:14:13.184 INFO:tasks.workunit.client.1.smithi115.stdout:9/333: mkdir d1/d18/d4a/d74 0 2021-11-03T17:14:13.184 INFO:tasks.workunit.client.1.smithi115.stdout:9/334: readlink d1/l67 0 2021-11-03T17:14:13.209 INFO:tasks.workunit.client.0.smithi027.stdout:6/422: dwrite dc/d16/d1b/d3c/d51/f70 [0,4194304] 0 2021-11-03T17:14:13.210 INFO:tasks.workunit.client.1.smithi115.stdout:9/335: dwrite d1/d18/d2b/d42/f12 [0,4194304] 0 2021-11-03T17:14:13.210 INFO:tasks.workunit.client.1.smithi115.stdout:9/336: write d1/f1d [752064,7569] 0 2021-11-03T17:14:13.211 INFO:tasks.workunit.client.1.smithi115.stdout:9/337: creat d1/f75 x:0 0 0 2021-11-03T17:14:13.211 INFO:tasks.workunit.client.0.smithi027.stdout:6/423: creat dc/d16/d1b/f75 x:0 0 0 2021-11-03T17:14:13.212 INFO:tasks.workunit.client.1.smithi115.stdout:9/338: rename d1/d1a/d48/f62 to d1/d1a/f76 0 2021-11-03T17:14:13.213 INFO:tasks.workunit.client.1.smithi115.stdout:9/339: creat d1/d1a/d48/d27/d53/f77 x:0 0 0 2021-11-03T17:14:13.213 INFO:tasks.workunit.client.1.smithi115.stdout:9/340: chown d1/d1a/d48/d27/d34 0 1 2021-11-03T17:14:13.214 INFO:tasks.workunit.client.1.smithi115.stdout:9/341: symlink d1/d1a/d48/d27/d2f/d41/l78 0 2021-11-03T17:14:13.215 INFO:tasks.workunit.client.1.smithi115.stdout:9/342: write d1/d18/d2b/d42/f12 [777979,128696] 0 2021-11-03T17:14:13.216 INFO:tasks.workunit.client.1.smithi115.stdout:9/343: truncate d1/d1a/f28 4954712 0 2021-11-03T17:14:13.216 INFO:tasks.workunit.client.1.smithi115.stdout:9/344: mkdir d1/d1a/d48/d27/d34/d50/d79 0 2021-11-03T17:14:13.217 INFO:tasks.workunit.client.1.smithi115.stdout:9/345: creat d1/d1a/d48/d27/f7a x:0 0 0 2021-11-03T17:14:13.217 INFO:tasks.workunit.client.1.smithi115.stdout:9/346: readlink d1/d1a/l5f 0 2021-11-03T17:14:13.222 INFO:tasks.workunit.client.1.smithi115.stdout:9/347: rmdir d1/d1a/d48/d27/d53 39 2021-11-03T17:14:13.223 INFO:tasks.workunit.client.1.smithi115.stdout:9/348: chown d1/d18/d4a/d54 0 1 2021-11-03T17:14:13.224 INFO:tasks.workunit.client.1.smithi115.stdout:9/349: mkdir d1/d1a/d48/d27/d2f/d7b 0 2021-11-03T17:14:13.224 INFO:tasks.workunit.client.1.smithi115.stdout:9/350: creat d1/d18/f7c x:0 0 0 2021-11-03T17:14:13.225 INFO:tasks.workunit.client.1.smithi115.stdout:9/351: creat d1/d1a/d48/d27/d2f/d41/f7d x:0 0 0 2021-11-03T17:14:13.226 INFO:tasks.workunit.client.1.smithi115.stdout:9/352: creat d1/d18/d4a/d74/f7e x:0 0 0 2021-11-03T17:14:13.227 INFO:tasks.workunit.client.1.smithi115.stdout:9/353: dread - d1/d18/d4a/d54/d3a/f3b zero size 2021-11-03T17:14:13.227 INFO:tasks.workunit.client.1.smithi115.stdout:9/354: read - d1/d18/f7c zero size 2021-11-03T17:14:13.227 INFO:tasks.workunit.client.1.smithi115.stdout:9/355: chown d1/d1a/f76 296 1 2021-11-03T17:14:13.228 INFO:tasks.workunit.client.1.smithi115.stdout:9/356: write d1/d1a/d48/d27/d2f/d41/f60 [1535027,108185] 0 2021-11-03T17:14:13.229 INFO:tasks.workunit.client.1.smithi115.stdout:9/357: link d1/d1a/d48/f73 d1/d1a/d48/d27/d53/f7f 0 2021-11-03T17:14:13.229 INFO:tasks.workunit.client.1.smithi115.stdout:9/358: fsync d1/d18/d4a/d54/d3a/f6d 0 2021-11-03T17:14:13.229 INFO:tasks.workunit.client.1.smithi115.stdout:9/359: readlink d1/d1a/d48/d27/d34/l4e 0 2021-11-03T17:14:13.230 INFO:tasks.workunit.client.1.smithi115.stdout:9/360: mkdir d1/d1a/d48/d27/d80 0 2021-11-03T17:14:13.230 INFO:tasks.workunit.client.1.smithi115.stdout:9/361: mknod d1/d18/d2b/d42/c81 1 2021-11-03T17:14:13.231 INFO:tasks.workunit.client.1.smithi115.stdout:9/362: link d1/d18/f22 d1/d1a/d48/d27/d53/f82 0 2021-11-03T17:14:13.244 INFO:tasks.workunit.client.1.smithi115.stdout:9/363: dwrite d1/d18/d2b/d42/f17 [4194304,4194304] 0 2021-11-03T17:14:13.244 INFO:tasks.workunit.client.1.smithi115.stdout:9/364: write d1/d18/d4a/d74/f7e [801651,112759] 0 2021-11-03T17:14:13.245 INFO:tasks.workunit.client.1.smithi115.stdout:9/365: write d1/d18/d2b/d42/f12 [405027,75486] 0 2021-11-03T17:14:13.245 INFO:tasks.workunit.client.1.smithi115.stdout:9/366: dread - d1/d1a/d48/d27/d53/f7f zero size 2021-11-03T17:14:13.245 INFO:tasks.workunit.client.1.smithi115.stdout:9/367: write d1/d18/d4a/d74/f7e [531496,112031] 0 2021-11-03T17:14:13.247 INFO:tasks.workunit.client.1.smithi115.stdout:9/368: mkdir d1/d1a/d48/d27/d53/d83 0 2021-11-03T17:14:13.248 INFO:tasks.workunit.client.1.smithi115.stdout:9/369: rename d1/d66/f6c to d1/d18/d4a/d74/f84 0 2021-11-03T17:14:13.261 INFO:tasks.workunit.client.1.smithi115.stdout:9/370: dwrite d1/d1a/d48/d27/d2f/f59 [0,4194304] 0 2021-11-03T17:14:13.264 INFO:tasks.workunit.client.1.smithi115.stdout:9/371: dread d1/d18/d2b/d42/f21 [0,4194304] 0 2021-11-03T17:14:13.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:13 smithi027 conmon[32322]: audit 2021-11-03T17: 2021-11-03T17:14:13.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:13 smithi027 conmon[32322]: 14:12.442509+0000 mgr.smithi027.ufhhvb (mgr.14162) 305 : audit [DBG] from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:14:13.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:13 smithi027 conmon[32322]: cluster 2021-11-03T 2021-11-03T17:14:13.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:13 smithi027 conmon[32322]: 17:14:12.605623+0000 mgr.smithi027.ufhhvb (mgr.14162) 306 2021-11-03T17:14:13.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:13 smithi027 conmon[32322]: : cluster [DBG] pgmap v253: 65 pgs: 65 active+clean; 1.4 GiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 65 MiB/s rd, 130 MiB/s wr, 204 op/s 2021-11-03T17:14:13.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.345+0000 7f08498d5700 1 -- 172.21.15.27:0/816658946 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f08440008d0 con 0x7f0844060310 2021-11-03T17:14:13.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.345+0000 7f08417fa700 1 -- 172.21.15.27:0/816658946 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+573 (secure 0 0 0) 0x7f083805d070 con 0x7f0844060310 2021-11-03T17:14:13.341 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:14:13.341 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:14:13.341 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 2 2021-11-03T17:14:13.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:14:13.342 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:14:13.342 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 2 2021-11-03T17:14:13.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:14:13.342 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:14:13.343 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 6 2021-11-03T17:14:13.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:14:13.343 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:14:13.343 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:14:13.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:14:13.344 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:14:13.344 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 14 2021-11-03T17:14:13.344 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:14:13.344 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:14:13.345 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.347+0000 7f08498d5700 1 -- 172.21.15.27:0/816658946 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f083405c300 msgr2=0x7f083405e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:14:13.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.347+0000 7f08498d5700 1 --2- 172.21.15.27:0/816658946 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f083405c300 0x7f083405e7c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f0830004630 tx=0x7f0830009b30).stop 2021-11-03T17:14:13.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.347+0000 7f08498d5700 1 -- 172.21.15.27:0/816658946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0844060310 msgr2=0x7f0844104b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:14:13.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.347+0000 7f08498d5700 1 --2- 172.21.15.27:0/816658946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0844060310 0x7f0844104b10 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f08380041d0 tx=0x7f083801adf0).stop 2021-11-03T17:14:13.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.347+0000 7f08498d5700 1 -- 172.21.15.27:0/816658946 shutdown_connections 2021-11-03T17:14:13.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.347+0000 7f08498d5700 1 --2- 172.21.15.27:0/816658946 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f083405c300 0x7f083405e7c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:13.347 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.347+0000 7f08498d5700 1 --2- 172.21.15.27:0/816658946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0844060310 0x7f0844104b10 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:13.347 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.347+0000 7f08498d5700 1 --2- 172.21.15.27:0/816658946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08440f7480 0x7f0844105000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:13.347 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.347+0000 7f08498d5700 1 -- 172.21.15.27:0/816658946 >> 172.21.15.27:0/816658946 conn(0x7f08440f0b90 msgr2=0x7f08440f2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:14:13.347 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.347+0000 7f08498d5700 1 -- 172.21.15.27:0/816658946 shutdown_connections 2021-11-03T17:14:13.348 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.347+0000 7f08498d5700 1 -- 172.21.15.27:0/816658946 wait complete. 2021-11-03T17:14:13.374 INFO:tasks.workunit.client.0.smithi027.stdout:1/364: sync 2021-11-03T17:14:13.374 INFO:tasks.workunit.client.0.smithi027.stdout:1/365: chown d6/fa 26972 1 2021-11-03T17:14:13.376 INFO:tasks.workunit.client.0.smithi027.stdout:1/366: mkdir d6/de/d44/d79/d25/d28/d5c/d68/d8b 0 2021-11-03T17:14:13.377 INFO:tasks.workunit.client.0.smithi027.stdout:1/367: write d6/d11/f27 [4301617,22772] 0 2021-11-03T17:14:13.378 INFO:tasks.workunit.client.0.smithi027.stdout:1/368: unlink d6/de/d44/d79/d25/d28/d41/f5f 0 2021-11-03T17:14:13.379 INFO:tasks.workunit.client.0.smithi027.stdout:1/369: symlink d6/de/d44/d79/d25/d28/d5c/d81/l8c 0 2021-11-03T17:14:13.379 INFO:tasks.workunit.client.0.smithi027.stdout:1/370: dread - d6/de/d44/d79/d25/d28/d5c/d68/f6c zero size 2021-11-03T17:14:13.417 INFO:tasks.workunit.client.0.smithi027.stdout:1/371: mkdir d6/de/d44/d79/d37/d49/d8d 0 2021-11-03T17:14:13.417 INFO:tasks.workunit.client.0.smithi027.stdout:1/372: creat d6/de/d44/d79/d25/d28/d32/f8e x:0 0 0 2021-11-03T17:14:13.417 INFO:tasks.workunit.client.0.smithi027.stdout:1/373: chown d6/de/d13/d15/d4e 6989 1 2021-11-03T17:14:13.418 INFO:tasks.workunit.client.0.smithi027.stdout:1/374: truncate d6/de/f19 6826730 0 2021-11-03T17:14:13.425 INFO:tasks.workunit.client.0.smithi027.stdout:1/375: read d6/de/f19 [456002,69115] 0 2021-11-03T17:14:13.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.449+0000 7f10eec5b700 1 -- 172.21.15.27:0/2451072672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10e80f66e0 msgr2=0x7f10e80f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:14:13.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.449+0000 7f10eec5b700 1 --2- 172.21.15.27:0/2451072672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10e80f66e0 0x7f10e80f6b40 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f10dc004660 tx=0x7f10dc009b30).stop 2021-11-03T17:14:13.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.449+0000 7f10eec5b700 1 -- 172.21.15.27:0/2451072672 shutdown_connections 2021-11-03T17:14:13.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.449+0000 7f10eec5b700 1 --2- 172.21.15.27:0/2451072672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10e80f66e0 0x7f10e80f6b40 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:13.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.449+0000 7f10eec5b700 1 --2- 172.21.15.27:0/2451072672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10e80f5440 0x7f10e80f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:14:13.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.449+0000 7f10eec5b700 1 -- 172.21.15.27:0/2451072672 >> 172.21.15.27:0/2451072672 conn(0x7f10e80f0ba0 msgr2=0x7f10e80f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:14:13.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.449+0000 7f10eec5b700 1 -- 172.21.15.27:0/2451072672 shutdown_connections 2021-11-03T17:14:13.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.449+0000 7f10eec5b700 1 -- 172.21.15.27:0/2451072672 wait complete. 2021-11-03T17:14:13.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.450+0000 7f10eec5b700 1 Processor -- start 2021-11-03T17:14:13.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.450+0000 7f10eec5b700 1 -- start start 2021-11-03T17:14:13.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.450+0000 7f10eec5b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10e80f5440 0x7f10e8109aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:14:13.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.450+0000 7f10eec5b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10e80f66e0 0x7f10e810bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:14:13.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.450+0000 7f10eec5b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e8060450 con 0x7f10e80f66e0 2021-11-03T17:14:13.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.450+0000 7f10eec5b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e8060590 con 0x7f10e80f5440 2021-11-03T17:14:13.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.450+0000 7f10ed458700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10e80f66e0 0x7f10e810bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:14:13.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.450+0000 7f10ed458700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10e80f66e0 0x7f10e810bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55340/0 (socket says 172.21.15.27:55340) 2021-11-03T17:14:13.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.450+0000 7f10ed458700 1 -- 172.21.15.27:0/3781211354 learned_addr learned my addr 172.21.15.27:0/3781211354 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:14:13.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.450+0000 7f10edc59700 1 --2- 172.21.15.27:0/3781211354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10e80f5440 0x7f10e8109aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:14:13.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.451+0000 7f10ed458700 1 -- 172.21.15.27:0/3781211354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10e80f5440 msgr2=0x7f10e8109aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:14:13.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.451+0000 7f10ed458700 1 --2- 172.21.15.27:0/3781211354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10e80f5440 0x7f10e8109aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:13.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.451+0000 7f10ed458700 1 -- 172.21.15.27:0/3781211354 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10dc005040 con 0x7f10e80f66e0 2021-11-03T17:14:13.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.451+0000 7f10ed458700 1 --2- 172.21.15.27:0/3781211354 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10e80f66e0 0x7f10e810bfa0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f10dc009990 tx=0x7f10dc009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:14:13.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.451+0000 7f10e67fc700 1 -- 172.21.15.27:0/3781211354 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10dc018070 con 0x7f10e80f66e0 2021-11-03T17:14:13.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.451+0000 7f10e67fc700 1 -- 172.21.15.27:0/3781211354 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f10dc007ba0 con 0x7f10e80f66e0 2021-11-03T17:14:13.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.451+0000 7f10e67fc700 1 -- 172.21.15.27:0/3781211354 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10dc01c620 con 0x7f10e80f66e0 2021-11-03T17:14:13.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.451+0000 7f10eec5b700 1 -- 172.21.15.27:0/3781211354 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10e810c550 con 0x7f10e80f66e0 2021-11-03T17:14:13.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.451+0000 7f10eec5b700 1 -- 172.21.15.27:0/3781211354 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10e810cab0 con 0x7f10e80f66e0 2021-11-03T17:14:13.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.452+0000 7f10e67fc700 1 -- 172.21.15.27:0/3781211354 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 18) v1 ==== 72193+0+0 (secure 0 0 0) 0x7f10dc01c890 con 0x7f10e80f66e0 2021-11-03T17:14:13.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.452+0000 7f10eec5b700 1 -- 172.21.15.27:0/3781211354 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10e8045ad0 con 0x7f10e80f66e0 2021-11-03T17:14:13.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.453+0000 7f10e67fc700 1 --2- 172.21.15.27:0/3781211354 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f10d005c300 0x7f10d005e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:14:13.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.453+0000 7f10e67fc700 1 -- 172.21.15.27:0/3781211354 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 5349+0+0 (secure 0 0 0) 0x7f10dc077c70 con 0x7f10e80f66e0 2021-11-03T17:14:13.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.453+0000 7f10edc59700 1 --2- 172.21.15.27:0/3781211354 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f10d005c300 0x7f10d005e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:14:13.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.453+0000 7f10edc59700 1 --2- 172.21.15.27:0/3781211354 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f10d005c300 0x7f10d005e7c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f10e8060040 tx=0x7f10d8009220).ready entity=mgr.14162 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:14:13.455 INFO:tasks.workunit.client.0.smithi027.stdout:1/376: link d6/d11/d14/f17 d6/de/d44/d79/d37/f8f 0 2021-11-03T17:14:13.455 INFO:tasks.workunit.client.0.smithi027.stdout:1/377: symlink d6/de/d13/d15/l90 0 2021-11-03T17:14:13.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.459+0000 7f10e67fc700 1 -- 172.21.15.27:0/3781211354 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+152212 (secure 0 0 0) 0x7f10dc04a9d0 con 0x7f10e80f66e0 2021-11-03T17:14:13.467 INFO:tasks.workunit.client.0.smithi027.stdout:1/378: dwrite d6/f18 [4194304,4194304] 0 2021-11-03T17:14:13.471 INFO:tasks.workunit.client.0.smithi027.stdout:1/379: rename d6/f53 to d6/de/d13/d15/d4e/f91 0 2021-11-03T17:14:13.473 INFO:tasks.workunit.client.0.smithi027.stdout:1/380: creat d6/de/f92 x:0 0 0 2021-11-03T17:14:13.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:13 smithi115 conmon[34670]: audit 2021-11-03T17:14:12.442509+0000 mgr.smithi027.ufhhvb (mgr.14162) 305 : audit [DBG] from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:14:13.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:13 smithi115 conmon[34670]: cluster 2021-11-03T17:14:12.605623+0000 mgr.smithi027.ufhhvb (mgr.14162) 306 : cluster [DBG] pgmap v253: 65 pgs: 65 active+clean; 1.4 GiB data, 4.3 GiB used, 532 GiB / 536 GiB avail; 65 MiB/s rd, 130 MiB/s wr, 204 op/s 2021-11-03T17:14:13.486 INFO:tasks.workunit.client.0.smithi027.stdout:1/381: dwrite d6/de/d44/d79/d25/d28/d32/f8e [0,4194304] 0 2021-11-03T17:14:13.487 INFO:tasks.workunit.client.0.smithi027.stdout:1/382: mknod d6/d11/d14/c93 1 2021-11-03T17:14:13.487 INFO:tasks.workunit.client.0.smithi027.stdout:1/383: mknod d6/de/d44/d79/d25/d28/d5c/d81/d7d/c94 1 2021-11-03T17:14:13.489 INFO:tasks.workunit.client.0.smithi027.stdout:1/384: rmdir d6/d11/d14 39 2021-11-03T17:14:13.490 INFO:tasks.workunit.client.0.smithi027.stdout:1/385: chown d6/de/d44/d79/d25/d28/d5c/d68/d8b 1 1 2021-11-03T17:14:13.491 INFO:tasks.workunit.client.0.smithi027.stdout:1/386: creat d6/d11/f95 x:0 0 0 2021-11-03T17:14:13.495 INFO:tasks.workunit.client.0.smithi027.stdout:1/387: symlink d6/de/d44/d79/d25/d5e/d72/l96 0 2021-11-03T17:14:13.497 INFO:tasks.workunit.client.0.smithi027.stdout:1/388: symlink d6/de/d44/d79/d37/l97 0 2021-11-03T17:14:13.498 INFO:tasks.workunit.client.0.smithi027.stdout:1/389: write d6/d11/d14/f17 [2457741,46821] 0 2021-11-03T17:14:13.499 INFO:tasks.workunit.client.0.smithi027.stdout:1/390: rmdir d6/de/d44/d79/d25/d5e 39 2021-11-03T17:14:13.500 INFO:tasks.workunit.client.0.smithi027.stdout:1/391: mknod d6/d11/d14/c98 1 2021-11-03T17:14:13.502 INFO:tasks.workunit.client.0.smithi027.stdout:1/392: creat d6/de/d44/d79/f99 x:0 0 0 2021-11-03T17:14:13.502 INFO:tasks.workunit.client.0.smithi027.stdout:1/393: mknod d6/de/d44/d79/d25/d28/d5c/d81/d7d/c9a 1 2021-11-03T17:14:13.504 INFO:tasks.workunit.client.0.smithi027.stdout:1/394: mkdir d6/de/d44/d9b 0 2021-11-03T17:14:13.526 INFO:tasks.workunit.client.0.smithi027.stdout:1/395: dwrite d6/de/d44/d79/d25/d28/d5c/d68/f6c [0,4194304] 0 2021-11-03T17:14:13.526 INFO:tasks.workunit.client.0.smithi027.stdout:1/396: write d6/de/d44/d79/d25/d28/d5c/f73 [826526,27522] 0 2021-11-03T17:14:13.526 INFO:tasks.workunit.client.0.smithi027.stdout:1/397: mknod d6/de/d44/d79/d37/c9c 1 2021-11-03T17:14:13.540 INFO:tasks.workunit.client.0.smithi027.stdout:1/398: dwrite d6/de/d44/d79/d25/f47 [0,4194304] 0 2021-11-03T17:14:13.541 INFO:tasks.workunit.client.0.smithi027.stdout:1/399: dread - d6/de/d44/d79/d31/f7f zero size 2021-11-03T17:14:13.542 INFO:tasks.workunit.client.0.smithi027.stdout:1/400: creat d6/de/d44/d79/f9d x:0 0 0 2021-11-03T17:14:13.542 INFO:tasks.workunit.client.0.smithi027.stdout:1/401: write d6/f7c [807923,48331] 0 2021-11-03T17:14:13.558 INFO:tasks.workunit.client.0.smithi027.stdout:1/402: dwrite d6/de/d44/d79/d25/d28/d32/f84 [0,4194304] 0 2021-11-03T17:14:13.567 INFO:tasks.workunit.client.0.smithi027.stdout:1/403: dread d6/de/d44/d79/d25/d28/d5c/d68/f6c [0,4194304] 0 2021-11-03T17:14:13.570 INFO:tasks.workunit.client.0.smithi027.stdout:1/404: unlink d6/de/d44/d79/l4d 0 2021-11-03T17:14:13.573 INFO:tasks.workunit.client.0.smithi027.stdout:1/405: link d6/de/d44/d79/l87 d6/de/d44/d79/d25/d28/d41/l9e 0 2021-11-03T17:14:13.573 INFO:tasks.workunit.client.0.smithi027.stdout:1/406: chown d6/de/d13/f38 211 1 2021-11-03T17:14:13.575 INFO:tasks.workunit.client.0.smithi027.stdout:1/407: rmdir d6/de/d44/d79/d37/d49/d8d 0 2021-11-03T17:14:13.576 INFO:tasks.workunit.client.0.smithi027.stdout:1/408: symlink d6/de/d44/d79/d25/d5e/l9f 0 2021-11-03T17:14:13.579 INFO:tasks.workunit.client.0.smithi027.stdout:1/409: unlink d6/de/d44/d79/d37/l97 0 2021-11-03T17:14:13.580 INFO:tasks.workunit.client.0.smithi027.stdout:1/410: write d6/de/d13/d15/f2f [381419,74537] 0 2021-11-03T17:14:13.580 INFO:tasks.workunit.client.0.smithi027.stdout:1/411: chown d6/de/d44/d79/d25/d5e 28 1 2021-11-03T17:14:13.581 INFO:tasks.workunit.client.0.smithi027.stdout:1/412: creat d6/de/d44/d79/d31/fa0 x:0 0 0 2021-11-03T17:14:13.596 INFO:tasks.workunit.client.0.smithi027.stdout:1/413: dwrite d6/de/f19 [0,4194304] 0 2021-11-03T17:14:13.612 INFO:tasks.workunit.client.0.smithi027.stdout:1/414: dwrite d6/de/d44/d79/d25/d28/d32/f8e [0,4194304] 0 2021-11-03T17:14:13.627 INFO:tasks.workunit.client.0.smithi027.stdout:1/415: dwrite d6/d11/d14/f35 [4194304,4194304] 0 2021-11-03T17:14:13.627 INFO:tasks.workunit.client.0.smithi027.stdout:1/416: chown d6/de/d44/d79/d25/d28/d5c/d81/d7d/f7e 90532 1 2021-11-03T17:14:13.628 INFO:tasks.workunit.client.0.smithi027.stdout:1/417: chown f2 752 1 2021-11-03T17:14:13.630 INFO:tasks.workunit.client.0.smithi027.stdout:1/418: symlink d6/de/d44/d79/d37/d49/la1 0 2021-11-03T17:14:13.646 INFO:tasks.workunit.client.0.smithi027.stdout:1/419: dread d6/de/f1c [0,4194304] 0 2021-11-03T17:14:13.652 INFO:tasks.workunit.client.0.smithi027.stdout:1/420: dread d6/de/d44/d79/d31/f8a [0,4194304] 0 2021-11-03T17:14:13.653 INFO:tasks.workunit.client.0.smithi027.stdout:1/421: mknod d6/de/d44/d9b/ca2 1 2021-11-03T17:14:13.653 INFO:tasks.workunit.client.0.smithi027.stdout:1/422: creat d6/fa3 x:0 0 0 2021-11-03T17:14:13.675 INFO:tasks.workunit.client.0.smithi027.stdout:1/423: dwrite d6/de/d44/d79/d25/d28/d5c/f73 [0,4194304] 0 2021-11-03T17:14:13.686 INFO:tasks.workunit.client.0.smithi027.stdout:1/424: dwrite d6/de/f92 [0,4194304] 0 2021-11-03T17:14:13.690 INFO:tasks.workunit.client.0.smithi027.stdout:1/425: symlink d6/de/d44/d79/d25/d28/la4 0 2021-11-03T17:14:13.690 INFO:tasks.workunit.client.0.smithi027.stdout:1/426: dread - d6/de/d44/d79/d25/d28/d5c/d81/f4f zero size 2021-11-03T17:14:13.690 INFO:tasks.workunit.client.0.smithi027.stdout:1/427: write d6/de/d44/d79/d25/d28/d5c/d81/d7d/f7e [292937,111917] 0 2021-11-03T17:14:13.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.714+0000 7f10eec5b700 1 -- 172.21.15.27:0/3781211354 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f10e8027a00 con 0x7f10e80f66e0 2021-11-03T17:14:13.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.715+0000 7f10e67fc700 1 -- 172.21.15.27:0/3781211354 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 15 v15) v1 ==== 76+0+1662 (secure 0 0 0) 0x7f10dc04dff0 con 0x7f10e80f66e0 2021-11-03T17:14:13.710 INFO:teuthology.orchestra.run.smithi027.stdout:e15 2021-11-03T17:14:13.710 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:14:13.711 INFO:teuthology.orchestra.run.smithi027.stdout:compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:14:13.711 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:14:13.711 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:14:13.711 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:14:13.712 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:14:13.712 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 15 2021-11-03T17:14:13.712 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 2021-11-03T17:14:13.712 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:14:13.713 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:13:59.690012+0000 2021-11-03T17:14:13.713 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:14:13.713 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:14:13.713 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:14:13.713 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:14:13.714 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:14:13.714 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:14:13.714 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:14:13.714 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 0 2021-11-03T17:14:13.715 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:14:13.715 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 2 2021-11-03T17:14:13.715 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:14:13.715 INFO:teuthology.orchestra.run.smithi027.stdout:up {0=24273,1=14436} 2021-11-03T17:14:13.716 INFO:teuthology.orchestra.run.smithi027.stdout:failed 2021-11-03T17:14:13.716 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:14:13.716 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:14:13.716 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:14:13.716 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:14:13.717 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:14:13.717 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:14:13.717 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:14:13.717 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{0:24273} state up:active seq 3 export targets 1 join_fscid=1 addr [v2:172.21.15.115:6824/1621245036,v1:172.21.15.115:6825/1621245036]] 2021-11-03T17:14:13.717 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 3 export targets 0 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060]] 2021-11-03T17:14:13.718 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:14:13.718 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:14:13.718 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:14:13.718 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:14:13.719 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14448} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6828/1337044739,v1:172.21.15.27:6829/1337044739]] 2021-11-03T17:14:13.719 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:24289} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.115:6826/3367380088,v1:172.21.15.115:6827/3367380088]] 2021-11-03T17:14:13.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.716+0000 7f10eec5b700 1 -- 172.21.15.27:0/3781211354 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f10d005c300 msgr2=0x7f10d005e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:14:13.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.716+0000 7f10eec5b700 1 --2- 172.21.15.27:0/3781211354 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f10d005c300 0x7f10d005e7c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f10e8060040 tx=0x7f10d8009220).stop 2021-11-03T17:14:13.721 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.716+0000 7f10eec5b700 1 -- 172.21.15.27:0/3781211354 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10e80f66e0 msgr2=0x7f10e810bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:14:13.721 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.716+0000 7f10eec5b700 1 --2- 172.21.15.27:0/3781211354 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10e80f66e0 0x7f10e810bfa0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f10dc009990 tx=0x7f10dc009b30).stop 2021-11-03T17:14:13.721 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.716+0000 7f10eec5b700 1 -- 172.21.15.27:0/3781211354 shutdown_connections 2021-11-03T17:14:13.721 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.716+0000 7f10eec5b700 1 --2- 172.21.15.27:0/3781211354 >> [v2:172.21.15.27:6800/3802716887,v1:172.21.15.27:6801/3802716887] conn(0x7f10d005c300 0x7f10d005e7c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:13.722 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.716+0000 7f10eec5b700 1 --2- 172.21.15.27:0/3781211354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f10e80f5440 0x7f10e8109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:13.722 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.716+0000 7f10eec5b700 1 --2- 172.21.15.27:0/3781211354 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f10e80f66e0 0x7f10e810bfa0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:13.722 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.716+0000 7f10eec5b700 1 -- 172.21.15.27:0/3781211354 >> 172.21.15.27:0/3781211354 conn(0x7f10e80f0ba0 msgr2=0x7f10e80fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:14:13.722 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.716+0000 7f10eec5b700 1 -- 172.21.15.27:0/3781211354 shutdown_connections 2021-11-03T17:14:13.723 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:13.716+0000 7f10eec5b700 1 -- 172.21.15.27:0/3781211354 wait complete. 2021-11-03T17:14:13.723 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 15 2021-11-03T17:14:13.724 INFO:tasks.workunit.client.0.smithi027.stdout:1/428: dwrite d6/f29 [4194304,4194304] 0 2021-11-03T17:14:13.724 INFO:tasks.workunit.client.0.smithi027.stdout:1/429: symlink d6/de/d13/d15/la5 0 2021-11-03T17:14:13.725 INFO:tasks.workunit.client.0.smithi027.stdout:1/430: unlink d6/de/d44/d79/d25/d28/d5c/d81/f4f 0 2021-11-03T17:14:13.725 INFO:tasks.workunit.client.0.smithi027.stdout:1/431: symlink d6/de/d44/d79/d25/d28/d5c/la6 0 2021-11-03T17:14:13.725 INFO:tasks.workunit.client.0.smithi027.stdout:1/432: rename d6/de/d44/d79/d25/d28/d32/d3f to d6/da7 0 2021-11-03T17:14:13.725 INFO:tasks.workunit.client.0.smithi027.stdout:1/433: chown d6/de/d44/d79/f59 281981593 1 2021-11-03T17:14:13.726 INFO:tasks.workunit.client.0.smithi027.stdout:1/434: symlink d6/de/d44/d79/d31/d58/la8 0 2021-11-03T17:14:13.925 INFO:tasks.workunit.client.0.smithi027.stdout:5/612: sync 2021-11-03T17:14:13.927 INFO:tasks.workunit.client.0.smithi027.stdout:5/613: creat d2/d2c/d3c/d58/fc7 x:0 0 0 2021-11-03T17:14:13.927 INFO:tasks.workunit.client.0.smithi027.stdout:5/614: readlink d2/d4/l5f 0 2021-11-03T17:14:13.927 INFO:tasks.workunit.client.0.smithi027.stdout:5/615: fdatasync d2/d12/f44 0 2021-11-03T17:14:13.928 INFO:tasks.workunit.client.0.smithi027.stdout:5/616: chown d2/d4/d61/f89 2095689586 1 2021-11-03T17:14:13.950 INFO:tasks.workunit.client.0.smithi027.stdout:5/617: dwrite d2/d4/d4e/d67/d20/f3a [0,4194304] 0 2021-11-03T17:14:13.950 INFO:tasks.workunit.client.0.smithi027.stdout:5/618: write d2/d4/d4e/d67/d20/fb9 [1469765,44409] 0 2021-11-03T17:14:13.952 INFO:tasks.workunit.client.0.smithi027.stdout:5/619: mkdir d2/d2c/d3c/dc8 0 2021-11-03T17:14:13.952 INFO:tasks.workunit.client.0.smithi027.stdout:5/620: mknod d2/d4/cc9 1 2021-11-03T17:14:13.969 INFO:tasks.workunit.client.0.smithi027.stdout:5/621: dwrite d2/d4/d4e/d67/f7f [0,4194304] 0 2021-11-03T17:14:13.985 INFO:tasks.workunit.client.0.smithi027.stdout:5/622: dwrite d2/d4/f6b [0,4194304] 0 2021-11-03T17:14:13.985 INFO:tasks.workunit.client.0.smithi027.stdout:5/623: chown d2/d12/dad/d6a/fc5 0 1 2021-11-03T17:14:13.986 INFO:tasks.workunit.client.0.smithi027.stdout:5/624: chown d2/d2c/d3c/d48/d65/l50 1604 1 2021-11-03T17:14:13.987 INFO:tasks.workunit.client.0.smithi027.stdout:5/625: creat d2/d2c/d3c/d58/fca x:0 0 0 2021-11-03T17:14:13.988 INFO:tasks.workunit.client.0.smithi027.stdout:5/626: truncate d2/d2c/d3c/f98 3362104 0 2021-11-03T17:14:13.990 INFO:tasks.workunit.client.0.smithi027.stdout:5/627: creat d2/d12/dad/da8/fcb x:0 0 0 2021-11-03T17:14:13.991 INFO:tasks.workunit.client.0.smithi027.stdout:5/628: write d2/d2c/d3c/f35 [770353,59382] 0 2021-11-03T17:14:14.000 INFO:tasks.workunit.client.0.smithi027.stdout:5/629: dread d2/f18 [0,4194304] 0 2021-11-03T17:14:14.002 INFO:tasks.workunit.client.0.smithi027.stdout:5/630: link d2/d2c/d9b/d9d/fbe d2/d4/fcc 0 2021-11-03T17:14:14.003 INFO:tasks.workunit.client.0.smithi027.stdout:5/631: creat d2/d2c/d9b/fcd x:0 0 0 2021-11-03T17:14:14.004 INFO:tasks.workunit.client.0.smithi027.stdout:5/632: symlink d2/d2c/lce 0 2021-11-03T17:14:14.005 INFO:tasks.workunit.client.0.smithi027.stdout:5/633: mknod d2/d2c/d3c/d48/d65/ccf 1 2021-11-03T17:14:14.005 INFO:tasks.workunit.client.0.smithi027.stdout:5/634: truncate d2/d2c/d3c/f98 3353554 0 2021-11-03T17:14:14.012 INFO:tasks.workunit.client.0.smithi027.stdout:5/635: dread d2/d2c/d3c/d48/d65/f8f [0,4194304] 0 2021-11-03T17:14:14.036 INFO:tasks.workunit.client.0.smithi027.stdout:5/636: dwrite d2/f42 [0,4194304] 0 2021-11-03T17:14:14.037 INFO:tasks.workunit.client.0.smithi027.stdout:5/637: symlink d2/d12/d7d/ld0 0 2021-11-03T17:14:14.038 INFO:tasks.workunit.client.0.smithi027.stdout:5/638: symlink d2/d4/d4e/d67/d8e/d99/ld1 0 2021-11-03T17:14:14.051 INFO:tasks.workunit.client.0.smithi027.stdout:5/639: dwrite d2/d2c/d9b/fcd [0,4194304] 0 2021-11-03T17:14:14.051 INFO:tasks.workunit.client.0.smithi027.stdout:5/640: chown d2/d1f 3 1 2021-11-03T17:14:14.052 INFO:tasks.workunit.client.0.smithi027.stdout:5/641: fsync d2/d2c/d9b/fcd 0 2021-11-03T17:14:14.054 INFO:tasks.workunit.client.0.smithi027.stdout:5/642: mkdir d2/d2c/d3c/d48/d65/d80/dd2 0 2021-11-03T17:14:14.054 INFO:tasks.workunit.client.0.smithi027.stdout:5/643: read d2/d12/dad/da8/fab [247649,109028] 0 2021-11-03T17:14:14.056 INFO:tasks.workunit.client.0.smithi027.stdout:5/644: dread d2/f7 [0,4194304] 0 2021-11-03T17:14:14.056 INFO:tasks.workunit.client.0.smithi027.stdout:5/645: write d2/d12/dad/d6a/fc5 [36927,65989] 0 2021-11-03T17:14:14.057 INFO:tasks.workunit.client.0.smithi027.stdout:5/646: write d2/f18 [4939787,75786] 0 2021-11-03T17:14:14.071 INFO:tasks.workunit.client.0.smithi027.stdout:7/263: sync 2021-11-03T17:14:14.072 INFO:tasks.workunit.client.0.smithi027.stdout:7/264: mknod d0/d2/d10/d1e/d20/d31/d36/c57 1 2021-11-03T17:14:14.072 INFO:tasks.workunit.client.0.smithi027.stdout:7/265: mkdir d0/d2d/d58 0 2021-11-03T17:14:14.254 INFO:tasks.workunit.client.1.smithi115.stdout:6/226: sync 2021-11-03T17:14:14.255 INFO:tasks.workunit.client.1.smithi115.stdout:8/282: sync 2021-11-03T17:14:14.255 INFO:tasks.workunit.client.1.smithi115.stdout:8/283: mknod d4/d6/d18/d30/c64 1 2021-11-03T17:14:14.256 INFO:tasks.workunit.client.1.smithi115.stdout:8/284: creat d4/d6/d18/d30/d33/d4c/f65 x:0 0 0 2021-11-03T17:14:14.256 INFO:tasks.workunit.client.1.smithi115.stdout:6/227: fdatasync d0/d1/d10/d13/d1c/d1e/f20 0 2021-11-03T17:14:14.257 INFO:tasks.workunit.client.1.smithi115.stdout:6/228: write d0/f22 [381501,98120] 0 2021-11-03T17:14:14.257 INFO:tasks.workunit.client.1.smithi115.stdout:6/229: dread - d0/d1/d10/f28 zero size 2021-11-03T17:14:14.257 INFO:tasks.workunit.client.1.smithi115.stdout:6/230: mknod d0/d1/d2e/c48 1 2021-11-03T17:14:14.265 INFO:tasks.workunit.client.1.smithi115.stdout:6/231: dread d0/f22 [0,4194304] 0 2021-11-03T17:14:14.272 INFO:tasks.workunit.client.1.smithi115.stdout:8/285: dwrite d4/f28 [0,4194304] 0 2021-11-03T17:14:14.287 INFO:tasks.workunit.client.1.smithi115.stdout:6/232: creat d0/d1/d2/d3f/f49 x:0 0 0 2021-11-03T17:14:14.287 INFO:tasks.workunit.client.1.smithi115.stdout:6/233: dread - d0/d1/d10/f28 zero size 2021-11-03T17:14:14.288 INFO:tasks.workunit.client.1.smithi115.stdout:6/234: read - d0/d1/d2/f35 zero size 2021-11-03T17:14:14.288 INFO:tasks.workunit.client.1.smithi115.stdout:8/286: creat d4/d6/d3f/f66 x:0 0 0 2021-11-03T17:14:14.289 INFO:tasks.workunit.client.1.smithi115.stdout:6/235: creat d0/d1/d2/f4a x:0 0 0 2021-11-03T17:14:14.289 INFO:tasks.workunit.client.1.smithi115.stdout:8/287: write d4/d6/d18/f3d [1650428,28364] 0 2021-11-03T17:14:14.290 INFO:tasks.workunit.client.1.smithi115.stdout:8/288: read - d4/d6/f42 zero size 2021-11-03T17:14:14.291 INFO:tasks.workunit.client.1.smithi115.stdout:6/236: link d0/d1/d10/d13/d1c/d1e/f47 d0/d1/d2/d3f/f4b 0 2021-11-03T17:14:14.292 INFO:tasks.workunit.client.1.smithi115.stdout:8/289: mknod d4/c67 1 2021-11-03T17:14:14.292 INFO:tasks.workunit.client.1.smithi115.stdout:8/290: chown d4/f28 328213420 1 2021-11-03T17:14:14.292 INFO:tasks.workunit.client.1.smithi115.stdout:8/291: truncate d4/d6/d3f/f61 330187 0 2021-11-03T17:14:14.298 INFO:tasks.workunit.client.1.smithi115.stdout:8/292: dread d4/d6/d3f/f45 [0,4194304] 0 2021-11-03T17:14:14.300 INFO:tasks.workunit.client.1.smithi115.stdout:8/293: symlink d4/d6/d18/d30/d33/l68 0 2021-11-03T17:14:14.310 INFO:tasks.workunit.client.1.smithi115.stdout:6/237: dwrite d0/f40 [0,4194304] 0 2021-11-03T17:14:14.320 INFO:tasks.workunit.client.1.smithi115.stdout:6/238: dread d0/d1/d10/d13/f39 [0,4194304] 0 2021-11-03T17:14:14.320 INFO:tasks.workunit.client.1.smithi115.stdout:6/239: stat d0/d1/d10/d13/d1c/d1e/l3c 0 2021-11-03T17:14:14.320 INFO:tasks.workunit.client.1.smithi115.stdout:6/240: fsync d0/d1/d10/d13/d1c/d1e/f20 0 2021-11-03T17:14:14.321 INFO:tasks.workunit.client.1.smithi115.stdout:6/241: chown d0/d1/d10 11 1 2021-11-03T17:14:14.321 INFO:tasks.workunit.client.1.smithi115.stdout:6/242: mkdir d0/d1/d2e/d4c 0 2021-11-03T17:14:14.322 INFO:tasks.workunit.client.1.smithi115.stdout:6/243: creat d0/f4d x:0 0 0 2021-11-03T17:14:14.322 INFO:tasks.workunit.client.1.smithi115.stdout:6/244: truncate d0/d1/d2/f1f 13412349 0 2021-11-03T17:14:14.333 INFO:tasks.workunit.client.1.smithi115.stdout:6/245: dread d0/f41 [0,4194304] 0 2021-11-03T17:14:14.334 INFO:tasks.workunit.client.1.smithi115.stdout:6/246: creat d0/d1/d2e/d4c/f4e x:0 0 0 2021-11-03T17:14:14.352 INFO:tasks.workunit.client.1.smithi115.stdout:6/247: rmdir d0/d1/d2e 39 2021-11-03T17:14:14.354 INFO:tasks.workunit.client.1.smithi115.stdout:6/248: unlink d0/d1/d10/d13/d1c/d1e/f20 0 2021-11-03T17:14:14.354 INFO:tasks.workunit.client.1.smithi115.stdout:6/249: stat d0/f4d 0 2021-11-03T17:14:14.354 INFO:tasks.workunit.client.1.smithi115.stdout:6/250: chown d0/d1/d10/d13/l33 42664 1 2021-11-03T17:14:14.355 INFO:tasks.workunit.client.1.smithi115.stdout:6/251: creat d0/f4f x:0 0 0 2021-11-03T17:14:14.355 INFO:tasks.workunit.client.1.smithi115.stdout:6/252: chown d0/f22 147282 1 2021-11-03T17:14:14.356 INFO:tasks.workunit.client.1.smithi115.stdout:6/253: chown d0/f6 3 1 2021-11-03T17:14:14.416 INFO:tasks.workunit.client.1.smithi115.stdout:6/254: dwrite d0/d1/d2/ff [0,4194304] 0 2021-11-03T17:14:14.416 INFO:tasks.workunit.client.1.smithi115.stdout:6/255: fsync d0/d1/f19 0 2021-11-03T17:14:14.416 INFO:tasks.workunit.client.1.smithi115.stdout:6/256: chown d0/d1/d10/d13/d1c/d1e/f47 22 1 2021-11-03T17:14:14.416 INFO:tasks.workunit.client.1.smithi115.stdout:6/257: readlink d0/d1/l45 0 2021-11-03T17:14:14.417 INFO:tasks.workunit.client.1.smithi115.stdout:6/258: rename d0/d1/d2/fc to d0/d1/f50 0 2021-11-03T17:14:14.418 INFO:tasks.workunit.client.1.smithi115.stdout:6/259: link d0/f4f d0/d1/d2/d3f/f51 0 2021-11-03T17:14:14.419 INFO:tasks.workunit.client.1.smithi115.stdout:6/260: chown d0/d1/d2/f12 1273 1 2021-11-03T17:14:14.439 INFO:tasks.workunit.client.1.smithi115.stdout:6/261: dwrite d0/d1/fe [0,4194304] 0 2021-11-03T17:14:14.440 INFO:tasks.workunit.client.1.smithi115.stdout:6/262: mknod d0/d1/d10/d13/d1c/c52 1 2021-11-03T17:14:14.440 INFO:tasks.workunit.client.1.smithi115.stdout:6/263: creat d0/d1/d10/f53 x:0 0 0 2021-11-03T17:14:14.445 INFO:tasks.workunit.client.1.smithi115.stdout:6/264: getdents d0/d1/d10/d13/d1c/d1e 0 2021-11-03T17:14:14.445 INFO:tasks.workunit.client.1.smithi115.stdout:6/265: fdatasync d0/d1/f50 0 2021-11-03T17:14:14.445 INFO:tasks.workunit.client.1.smithi115.stdout:6/266: write d0/d1/d10/f28 [180890,51445] 0 2021-11-03T17:14:14.447 INFO:tasks.workunit.client.1.smithi115.stdout:6/267: creat d0/d1/d2/d3f/f54 x:0 0 0 2021-11-03T17:14:14.447 INFO:tasks.workunit.client.1.smithi115.stdout:6/268: truncate d0/d1/d10/d13/f25 1014489 0 2021-11-03T17:14:14.447 INFO:tasks.workunit.client.1.smithi115.stdout:6/269: dread - d0/d1/d10/d13/d1c/f21 zero size 2021-11-03T17:14:14.469 INFO:tasks.workunit.client.1.smithi115.stdout:6/270: dwrite d0/f41 [0,4194304] 0 2021-11-03T17:14:14.470 INFO:tasks.workunit.client.1.smithi115.stdout:6/271: unlink d0/d1/l45 0 2021-11-03T17:14:14.471 INFO:tasks.workunit.client.1.smithi115.stdout:6/272: mknod d0/d1/d2/c55 1 2021-11-03T17:14:14.471 INFO:tasks.workunit.client.1.smithi115.stdout:6/273: mknod d0/d1/d10/d13/d1c/c56 1 2021-11-03T17:14:14.471 INFO:tasks.workunit.client.1.smithi115.stdout:6/274: dread - d0/d1/d10/d13/d1c/f21 zero size 2021-11-03T17:14:14.472 INFO:tasks.workunit.client.1.smithi115.stdout:6/275: symlink d0/d1/d10/d13/d1c/l57 0 2021-11-03T17:14:14.472 INFO:tasks.workunit.client.1.smithi115.stdout:6/276: creat d0/d1/d2/f58 x:0 0 0 2021-11-03T17:14:14.473 INFO:tasks.workunit.client.1.smithi115.stdout:6/277: truncate d0/d1/f8 3977985 0 2021-11-03T17:14:14.473 INFO:tasks.workunit.client.1.smithi115.stdout:6/278: write d0/d1/d2/d3f/f49 [167665,75123] 0 2021-11-03T17:14:14.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:14 smithi115 conmon[34670]: audit 2021-11-03T17:14:12.860638+0000 mgr.smithi027.ufhhvb (mgr.14162) 307 : audit [DBG] 2021-11-03T17:14:14.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:14 smithi115 conmon[34670]: from='client.14560 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:14:14.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:14 smithi115 conmon[34670]: audit 2021-11-03T17:14:13.346003+0000 mon.smithi027 (mon.0) 571 : audit [DBG] from='client.? 172.21.15.27:0/816658946' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:14:14.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:14 smithi115 conmon[34670]: audit 2021-11-03T17:14:13.715404+0000 mon.smithi027 (mon.0) 572 : audit [DBG] from='client.? 172.21.15.27:0/3781211354' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:14:14.490 INFO:tasks.workunit.client.1.smithi115.stdout:6/279: dwrite d0/d1/d10/d13/d1c/f38 [0,4194304] 0 2021-11-03T17:14:14.495 INFO:tasks.workunit.client.1.smithi115.stdout:6/280: mkdir d0/d1/d10/d13/d59 0 2021-11-03T17:14:14.497 INFO:tasks.workunit.client.1.smithi115.stdout:6/281: symlink d0/l5a 0 2021-11-03T17:14:14.499 INFO:tasks.workunit.client.1.smithi115.stdout:6/282: dread d0/d1/d10/d13/d1c/f2f [0,4194304] 0 2021-11-03T17:14:14.499 INFO:tasks.workunit.client.1.smithi115.stdout:6/283: dread - d0/d1/d2/d3f/f51 zero size 2021-11-03T17:14:14.500 INFO:tasks.workunit.client.1.smithi115.stdout:6/284: mknod d0/d1/d2e/c5b 1 2021-11-03T17:14:14.500 INFO:tasks.workunit.client.1.smithi115.stdout:6/285: link d0/l5a d0/d1/d10/d13/d1c/l5c 0 2021-11-03T17:14:14.504 INFO:tasks.workunit.client.1.smithi115.stdout:0/382: sync 2021-11-03T17:14:14.504 INFO:tasks.workunit.client.1.smithi115.stdout:4/485: sync 2021-11-03T17:14:14.505 INFO:tasks.workunit.client.1.smithi115.stdout:4/486: rmdir d7/dc/d6a/d3c 39 2021-11-03T17:14:14.509 INFO:tasks.workunit.client.1.smithi115.stdout:4/487: rename d7/dc/d1a/d21/d47/f7d to d7/dc/d6a/fac 0 2021-11-03T17:14:14.513 INFO:tasks.workunit.client.1.smithi115.stdout:4/488: rename d7/dc/d1a/d21/d47/d56/l78 to d7/dc/lad 0 2021-11-03T17:14:14.515 INFO:tasks.workunit.client.1.smithi115.stdout:4/489: unlink d7/dc/d1a/d21/d47/d56/d63/d7b/f7e 0 2021-11-03T17:14:14.515 INFO:tasks.workunit.client.1.smithi115.stdout:4/490: chown d7/d7a/d83 5 1 2021-11-03T17:14:14.515 INFO:tasks.workunit.client.1.smithi115.stdout:4/491: symlink d7/dc/d1a/d21/d47/d56/lae 0 2021-11-03T17:14:14.516 INFO:tasks.workunit.client.1.smithi115.stdout:4/492: mkdir d7/dc/d1a/d21/d47/d56/d42/daf 0 2021-11-03T17:14:14.517 INFO:tasks.workunit.client.1.smithi115.stdout:4/493: truncate d7/dc/f95 971736 0 2021-11-03T17:14:14.517 INFO:tasks.workunit.client.1.smithi115.stdout:4/494: readlink d7/dc/l33 0 2021-11-03T17:14:14.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:14 smithi027 conmon[32322]: audit 2021-11-03T17:14:12.860638+0000 mgr.smithi027.ufhhvb (mgr 2021-11-03T17:14:14.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:14 smithi027 conmon[32322]: .14162) 307 : audit [DBG] from='client.14560 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:14:14.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:14 smithi027 conmon[32322]: audit 2021-11-03T17:14:13.346003+0000 mon.smithi027 (mon.0) 571 : audit [DBG] from='client.? 172.21.15.27:0/816658946' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:14:14.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:14 smithi027 conmon[32322]: audit 2021-11-03T17:14:13.715404+0000 mon.smithi027 (mon.0) 572 : audit [DBG] from='client.? 172.21.15.27:0/3781211354' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:14:14.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:14 smithi027 conmon[32322]: 2021-11-03T17:14:14.533 INFO:tasks.workunit.client.1.smithi115.stdout:4/495: write d7/dc/f95 [755833,71443] 0 2021-11-03T17:14:14.534 INFO:tasks.workunit.client.0.smithi027.stdout:3/239: sync 2021-11-03T17:14:14.534 INFO:tasks.workunit.client.0.smithi027.stdout:2/383: sync 2021-11-03T17:14:14.534 INFO:tasks.workunit.client.1.smithi115.stdout:4/496: getdents d7/d7a/d83 0 2021-11-03T17:14:14.535 INFO:tasks.workunit.client.1.smithi115.stdout:6/286: dwrite d0/d1/d2/d3f/f49 [0,4194304] 0 2021-11-03T17:14:14.535 INFO:tasks.workunit.client.0.smithi027.stdout:3/240: getdents d1/db 0 2021-11-03T17:14:14.536 INFO:tasks.workunit.client.1.smithi115.stdout:0/383: dwrite d9/db/f58 [0,4194304] 0 2021-11-03T17:14:14.536 INFO:tasks.workunit.client.1.smithi115.stdout:6/287: mkdir d0/d1/d2/d5d 0 2021-11-03T17:14:14.537 INFO:tasks.workunit.client.1.smithi115.stdout:0/384: write d9/d15/d1a/d1d/f4b [2996347,121549] 0 2021-11-03T17:14:14.538 INFO:tasks.workunit.client.0.smithi027.stdout:3/241: creat d1/f4b x:0 0 0 2021-11-03T17:14:14.538 INFO:tasks.workunit.client.0.smithi027.stdout:3/242: chown d1/db 1625406 1 2021-11-03T17:14:14.538 INFO:tasks.workunit.client.1.smithi115.stdout:4/497: rename d7/dc/d1a/d21/f25 to d7/dc/fb0 0 2021-11-03T17:14:14.539 INFO:tasks.workunit.client.0.smithi027.stdout:2/384: mkdir d0/d2a/d79 0 2021-11-03T17:14:14.539 INFO:tasks.workunit.client.1.smithi115.stdout:6/288: symlink d0/d1/d10/d13/d1c/d1e/l5e 0 2021-11-03T17:14:14.540 INFO:tasks.workunit.client.1.smithi115.stdout:0/385: dread d9/d15/f1e [0,4194304] 0 2021-11-03T17:14:14.540 INFO:tasks.workunit.client.0.smithi027.stdout:3/243: truncate d1/db/f1e 2089886 0 2021-11-03T17:14:14.541 INFO:tasks.workunit.client.0.smithi027.stdout:2/385: write d0/d1b/d29/f2b [2306379,102447] 0 2021-11-03T17:14:14.541 INFO:tasks.workunit.client.0.smithi027.stdout:2/386: mknod d0/d1b/d29/c7a 1 2021-11-03T17:14:14.541 INFO:tasks.workunit.client.0.smithi027.stdout:2/387: chown d0/f3b 28047 1 2021-11-03T17:14:14.541 INFO:tasks.workunit.client.0.smithi027.stdout:2/388: chown d0/d5/d2c/l52 119666 1 2021-11-03T17:14:14.542 INFO:tasks.workunit.client.0.smithi027.stdout:3/244: rename d1/f9 to d1/f4c 0 2021-11-03T17:14:14.542 INFO:tasks.workunit.client.0.smithi027.stdout:2/389: write d0/d2a/f23 [7143972,83151] 0 2021-11-03T17:14:14.543 INFO:tasks.workunit.client.1.smithi115.stdout:0/386: write d9/db/f58 [138485,50563] 0 2021-11-03T17:14:14.543 INFO:tasks.workunit.client.0.smithi027.stdout:3/245: symlink d1/db/l4d 0 2021-11-03T17:14:14.545 INFO:tasks.workunit.client.0.smithi027.stdout:2/390: creat d0/d45/f7b x:0 0 0 2021-11-03T17:14:14.546 INFO:tasks.workunit.client.0.smithi027.stdout:3/246: creat d1/f4e x:0 0 0 2021-11-03T17:14:14.546 INFO:tasks.workunit.client.1.smithi115.stdout:0/387: symlink d9/d15/d1a/d1d/l7d 0 2021-11-03T17:14:14.547 INFO:tasks.workunit.client.1.smithi115.stdout:0/388: write d9/db/df/f48 [3670502,1931] 0 2021-11-03T17:14:14.548 INFO:tasks.workunit.client.1.smithi115.stdout:0/389: creat d9/db/f7e x:0 0 0 2021-11-03T17:14:14.548 INFO:tasks.workunit.client.0.smithi027.stdout:3/247: symlink d1/l4f 0 2021-11-03T17:14:14.548 INFO:tasks.workunit.client.0.smithi027.stdout:3/248: stat d1/f4b 0 2021-11-03T17:14:14.549 INFO:tasks.workunit.client.0.smithi027.stdout:2/391: mkdir d0/d7c 0 2021-11-03T17:14:14.549 INFO:tasks.workunit.client.0.smithi027.stdout:2/392: fsync d0/d5/f6d 0 2021-11-03T17:14:14.550 INFO:tasks.workunit.client.0.smithi027.stdout:3/249: link d1/f30 d1/f50 0 2021-11-03T17:14:14.550 INFO:tasks.workunit.client.0.smithi027.stdout:3/250: chown d1/db/f3e 72 1 2021-11-03T17:14:14.551 INFO:tasks.workunit.client.0.smithi027.stdout:3/251: rmdir d1/db 39 2021-11-03T17:14:14.551 INFO:tasks.workunit.client.0.smithi027.stdout:3/252: chown d1/db/f3c 254489 1 2021-11-03T17:14:14.551 INFO:tasks.workunit.client.0.smithi027.stdout:3/253: write d1/f1d [4574772,21678] 0 2021-11-03T17:14:14.552 INFO:tasks.workunit.client.1.smithi115.stdout:4/498: dread d7/dc/d6a/f46 [0,4194304] 0 2021-11-03T17:14:14.552 INFO:tasks.workunit.client.1.smithi115.stdout:4/499: readlink d7/dc/l97 0 2021-11-03T17:14:14.553 INFO:tasks.workunit.client.1.smithi115.stdout:4/500: chown d7/l99 1 1 2021-11-03T17:14:14.553 INFO:tasks.workunit.client.1.smithi115.stdout:4/501: write d7/dc/d6a/fac [843338,49017] 0 2021-11-03T17:14:14.553 INFO:tasks.workunit.client.1.smithi115.stdout:4/502: unlink d7/dc/d6a/l41 0 2021-11-03T17:14:14.553 INFO:tasks.workunit.client.1.smithi115.stdout:4/503: truncate d7/dc/d1a/d21/d47/d56/d63/d7b/d96/fa7 781617 0 2021-11-03T17:14:14.554 INFO:tasks.workunit.client.1.smithi115.stdout:4/504: chown d7/lb 186 1 2021-11-03T17:14:14.554 INFO:tasks.workunit.client.1.smithi115.stdout:4/505: fdatasync d7/dc/d1a/d21/d47/d56/d42/fa5 0 2021-11-03T17:14:14.554 INFO:tasks.workunit.client.1.smithi115.stdout:4/506: truncate d7/dc/d1a/d21/d47/f74 1365224 0 2021-11-03T17:14:14.554 INFO:tasks.workunit.client.1.smithi115.stdout:0/390: dread d9/d15/f16 [0,4194304] 0 2021-11-03T17:14:14.555 INFO:tasks.workunit.client.1.smithi115.stdout:0/391: write d9/d15/f67 [336713,126050] 0 2021-11-03T17:14:14.555 INFO:tasks.workunit.client.1.smithi115.stdout:6/289: dwrite d0/d1/d2/d3f/f4b [0,4194304] 0 2021-11-03T17:14:14.556 INFO:tasks.workunit.client.1.smithi115.stdout:6/290: read - d0/d1/d2/f4a zero size 2021-11-03T17:14:14.556 INFO:tasks.workunit.client.1.smithi115.stdout:0/392: getdents d9 0 2021-11-03T17:14:14.556 INFO:tasks.workunit.client.1.smithi115.stdout:0/393: chown d9/db/f42 9683500 1 2021-11-03T17:14:14.558 INFO:tasks.workunit.client.1.smithi115.stdout:6/291: getdents d0/d1 0 2021-11-03T17:14:14.559 INFO:tasks.workunit.client.1.smithi115.stdout:6/292: rmdir d0/d1 39 2021-11-03T17:14:14.566 INFO:tasks.workunit.client.1.smithi115.stdout:0/394: dread d9/db/f4f [0,4194304] 0 2021-11-03T17:14:14.568 INFO:tasks.workunit.client.1.smithi115.stdout:0/395: rename f0 to d9/d15/d2c/f7f 0 2021-11-03T17:14:14.574 INFO:tasks.workunit.client.1.smithi115.stdout:6/293: dread d0/d1/d10/d13/d1c/f36 [8388608,4194304] 0 2021-11-03T17:14:14.577 INFO:tasks.workunit.client.1.smithi115.stdout:4/507: rename d7/dc/d1a/d21/d47 to d7/dc/d1a/d21/db1 0 2021-11-03T17:14:14.577 INFO:tasks.workunit.client.1.smithi115.stdout:4/508: chown d7/d7a/fa3 116 1 2021-11-03T17:14:14.580 INFO:tasks.workunit.client.0.smithi027.stdout:2/393: dwrite d0/d2a/d5d/f75 [0,4194304] 0 2021-11-03T17:14:14.580 INFO:tasks.workunit.client.0.smithi027.stdout:2/394: mknod d0/d2a/d61/c7d 1 2021-11-03T17:14:14.580 INFO:tasks.workunit.client.0.smithi027.stdout:2/395: chown d0/d1b/d29/d54/d67/d6a/l77 2 1 2021-11-03T17:14:14.580 INFO:tasks.workunit.client.0.smithi027.stdout:2/396: write d0/d5/f13 [7227231,20562] 0 2021-11-03T17:14:14.581 INFO:tasks.workunit.client.1.smithi115.stdout:4/509: creat d7/dc/d1a/d21/db1/d56/d63/d7b/fb2 x:0 0 0 2021-11-03T17:14:14.592 INFO:tasks.workunit.client.0.smithi027.stdout:3/254: dwrite d1/f34 [0,4194304] 0 2021-11-03T17:14:14.593 INFO:tasks.workunit.client.1.smithi115.stdout:0/396: dwrite d9/db/df/f7c [0,4194304] 0 2021-11-03T17:14:14.598 INFO:tasks.workunit.client.0.smithi027.stdout:3/255: write d1/f4c [3340572,75833] 0 2021-11-03T17:14:14.599 INFO:tasks.workunit.client.1.smithi115.stdout:0/397: write d9/db/df/f44 [1976944,26629] 0 2021-11-03T17:14:14.600 INFO:tasks.workunit.client.0.smithi027.stdout:3/256: symlink d1/db/l51 0 2021-11-03T17:14:14.606 INFO:tasks.workunit.client.0.smithi027.stdout:2/397: dwrite d0/d1b/d29/f39 [0,4194304] 0 2021-11-03T17:14:14.607 INFO:tasks.workunit.client.1.smithi115.stdout:4/510: dwrite d7/fa0 [0,4194304] 0 2021-11-03T17:14:14.608 INFO:tasks.workunit.client.0.smithi027.stdout:2/398: creat d0/d1b/d1c/f7e x:0 0 0 2021-11-03T17:14:14.611 INFO:tasks.workunit.client.1.smithi115.stdout:4/511: mkdir d7/dc/d1a/d21/db1/d56/db3 0 2021-11-03T17:14:14.611 INFO:tasks.workunit.client.1.smithi115.stdout:4/512: truncate d7/d7a/fa3 26337 0 2021-11-03T17:14:14.616 INFO:tasks.workunit.client.0.smithi027.stdout:2/399: creat d0/d1b/d29/d54/f7f x:0 0 0 2021-11-03T17:14:14.617 INFO:tasks.workunit.client.0.smithi027.stdout:3/257: dread d1/db/f35 [0,4194304] 0 2021-11-03T17:14:14.619 INFO:tasks.workunit.client.0.smithi027.stdout:3/258: write d1/db/f35 [3138597,40864] 0 2021-11-03T17:14:14.620 INFO:tasks.workunit.client.0.smithi027.stdout:3/259: mkdir d1/db/d52 0 2021-11-03T17:14:14.622 INFO:tasks.workunit.client.0.smithi027.stdout:3/260: unlink d1/db/f1e 0 2021-11-03T17:14:14.622 INFO:tasks.workunit.client.0.smithi027.stdout:3/261: read d1/f2a [2657274,49170] 0 2021-11-03T17:14:14.625 INFO:tasks.workunit.client.0.smithi027.stdout:3/262: symlink d1/db/l53 0 2021-11-03T17:14:14.625 INFO:tasks.workunit.client.0.smithi027.stdout:3/263: write d1/db/f32 [1085459,109377] 0 2021-11-03T17:14:14.635 INFO:tasks.workunit.client.1.smithi115.stdout:4/513: read d7/d7a/d83/f8c [415394,20404] 0 2021-11-03T17:14:14.636 INFO:tasks.workunit.client.1.smithi115.stdout:4/514: mknod d7/dc/d6a/d72/cb4 1 2021-11-03T17:14:14.637 INFO:tasks.workunit.client.0.smithi027.stdout:3/264: fdatasync d1/d40/f49 0 2021-11-03T17:14:14.638 INFO:tasks.workunit.client.1.smithi115.stdout:4/515: dread d7/dc/f95 [0,4194304] 0 2021-11-03T17:14:14.638 INFO:tasks.workunit.client.1.smithi115.stdout:4/516: creat d7/fb5 x:0 0 0 2021-11-03T17:14:14.640 INFO:tasks.workunit.client.1.smithi115.stdout:4/517: rename d7/dc/d1a/d21/d37 to d7/dc/db6 0 2021-11-03T17:14:14.641 INFO:tasks.workunit.client.1.smithi115.stdout:4/518: getdents d7/dc/d6a 0 2021-11-03T17:14:14.642 INFO:tasks.workunit.client.1.smithi115.stdout:4/519: chown d7/dc/d6a/l4a 7032 1 2021-11-03T17:14:14.643 INFO:tasks.workunit.client.1.smithi115.stdout:4/520: truncate d7/dc/d1a/d21/db1/d56/d63/d7b/d96/fa7 742007 0 2021-11-03T17:14:14.643 INFO:tasks.workunit.client.1.smithi115.stdout:4/521: chown d7/dc/d6a/fac 60187499 1 2021-11-03T17:14:14.644 INFO:tasks.workunit.client.1.smithi115.stdout:4/522: mknod d7/dc/d1a/d21/db1/d56/d42/cb7 1 2021-11-03T17:14:14.646 INFO:tasks.workunit.client.1.smithi115.stdout:0/398: dwrite d9/d15/d1a/d1d/f4d [0,4194304] 0 2021-11-03T17:14:14.647 INFO:tasks.workunit.client.1.smithi115.stdout:4/523: link d7/d7a/d83/f64 d7/dc/d1a/d21/db1/d56/d63/d7b/fb8 0 2021-11-03T17:14:14.649 INFO:tasks.workunit.client.1.smithi115.stdout:4/524: symlink d7/dc/d1a/d21/db1/d56/db3/lb9 0 2021-11-03T17:14:14.654 INFO:tasks.workunit.client.1.smithi115.stdout:4/525: write d7/dc/d1a/d21/db1/d56/f98 [6525678,113330] 0 2021-11-03T17:14:14.654 INFO:tasks.workunit.client.1.smithi115.stdout:4/526: truncate d7/dc/d1a/d21/db1/d56/d63/f85 756151 0 2021-11-03T17:14:14.655 INFO:tasks.workunit.client.1.smithi115.stdout:4/527: unlink d7/dc/d1a/d21/db1/d56/d63/d9e/fa9 0 2021-11-03T17:14:14.658 INFO:tasks.workunit.client.0.smithi027.stdout:3/265: dwrite d1/db/f12 [0,4194304] 0 2021-11-03T17:14:14.672 INFO:tasks.workunit.client.1.smithi115.stdout:4/528: dwrite d7/dc/fa4 [0,4194304] 0 2021-11-03T17:14:14.673 INFO:tasks.workunit.client.1.smithi115.stdout:4/529: creat d7/fba x:0 0 0 2021-11-03T17:14:14.675 INFO:tasks.workunit.client.1.smithi115.stdout:4/530: rename d7/l6b to d7/dc/d6a/dab/lbb 0 2021-11-03T17:14:14.675 INFO:tasks.workunit.client.1.smithi115.stdout:4/531: write d7/dc/d6a/d72/f76 [1428037,108820] 0 2021-11-03T17:14:14.675 INFO:tasks.workunit.client.1.smithi115.stdout:4/532: readlink d7/dc/d6a/l4a 0 2021-11-03T17:14:14.676 INFO:tasks.workunit.client.1.smithi115.stdout:4/533: mknod d7/dc/db6/cbc 1 2021-11-03T17:14:14.676 INFO:tasks.workunit.client.1.smithi115.stdout:4/534: truncate d7/dc/d1a/d21/db1/d56/d42/f58 304019 0 2021-11-03T17:14:14.677 INFO:tasks.workunit.client.1.smithi115.stdout:4/535: chown d7/dc/d1a/d21/db1/d56/d63/f9b 3437 1 2021-11-03T17:14:14.679 INFO:tasks.workunit.client.0.smithi027.stdout:3/266: dread d1/f2a [0,4194304] 0 2021-11-03T17:14:14.679 INFO:tasks.workunit.client.0.smithi027.stdout:3/267: chown d1/l33 0 1 2021-11-03T17:14:14.679 INFO:tasks.workunit.client.0.smithi027.stdout:3/268: fdatasync d1/d40/f4a 0 2021-11-03T17:14:14.685 INFO:tasks.workunit.client.0.smithi027.stdout:3/269: dread d1/f22 [0,4194304] 0 2021-11-03T17:14:14.687 INFO:tasks.workunit.client.0.smithi027.stdout:3/270: symlink d1/db/l54 0 2021-11-03T17:14:14.758 INFO:tasks.workunit.client.1.smithi115.stdout:4/536: dread d7/dc/d1a/d21/f22 [0,4194304] 0 2021-11-03T17:14:14.758 INFO:tasks.workunit.client.1.smithi115.stdout:4/537: fsync d7/dc/d1a/d21/db1/f74 0 2021-11-03T17:14:14.759 INFO:tasks.workunit.client.0.smithi027.stdout:3/271: getdents d1/db 0 2021-11-03T17:14:14.760 INFO:tasks.workunit.client.1.smithi115.stdout:4/538: creat d7/dc/d6a/d3c/fbd x:0 0 0 2021-11-03T17:14:14.760 INFO:tasks.workunit.client.1.smithi115.stdout:4/539: write d7/dc/f11 [4279156,121650] 0 2021-11-03T17:14:14.766 INFO:tasks.workunit.client.1.smithi115.stdout:4/540: rename d7/dc/d1a/d21/db1/d56/f98 to d7/dc/d1a/d21/fbe 0 2021-11-03T17:14:14.766 INFO:tasks.workunit.client.1.smithi115.stdout:4/541: dread - d7/dc/d1a/d21/db1/d56/d63/fa6 zero size 2021-11-03T17:14:14.769 INFO:tasks.workunit.client.1.smithi115.stdout:4/542: mkdir d7/dc/d6a/d3c/dbf 0 2021-11-03T17:14:14.776 INFO:tasks.workunit.client.0.smithi027.stdout:3/272: dwrite d1/f50 [0,4194304] 0 2021-11-03T17:14:14.779 INFO:tasks.workunit.client.0.smithi027.stdout:3/273: truncate d1/db/f12 3625549 0 2021-11-03T17:14:14.779 INFO:tasks.workunit.client.0.smithi027.stdout:3/274: chown d1/db/f35 76 1 2021-11-03T17:14:14.779 INFO:tasks.workunit.client.0.smithi027.stdout:3/275: fsync d1/f22 0 2021-11-03T17:14:14.781 INFO:tasks.workunit.client.0.smithi027.stdout:3/276: link d1/db/f32 d1/f55 0 2021-11-03T17:14:14.784 INFO:tasks.workunit.client.0.smithi027.stdout:3/277: truncate d1/f30 2857691 0 2021-11-03T17:14:14.788 INFO:tasks.workunit.client.0.smithi027.stdout:3/278: rename d1/db/l24 to d1/db/d52/l56 0 2021-11-03T17:14:14.790 INFO:tasks.workunit.client.0.smithi027.stdout:3/279: mkdir d1/db/d52/d57 0 2021-11-03T17:14:14.792 INFO:tasks.workunit.client.0.smithi027.stdout:3/280: creat d1/d40/f58 x:0 0 0 2021-11-03T17:14:14.792 INFO:tasks.workunit.client.0.smithi027.stdout:3/281: fdatasync d1/f2a 0 2021-11-03T17:14:14.793 INFO:tasks.workunit.client.0.smithi027.stdout:3/282: creat d1/d40/f59 x:0 0 0 2021-11-03T17:14:14.793 INFO:tasks.workunit.client.0.smithi027.stdout:3/283: truncate d1/f4b 870207 0 2021-11-03T17:14:14.798 INFO:tasks.workunit.client.0.smithi027.stdout:3/284: dread d1/f2a [0,4194304] 0 2021-11-03T17:14:14.799 INFO:tasks.workunit.client.0.smithi027.stdout:3/285: write d1/db/f1c [1642689,25175] 0 2021-11-03T17:14:14.799 INFO:tasks.workunit.client.0.smithi027.stdout:3/286: readlink d1/db/l39 0 2021-11-03T17:14:14.799 INFO:tasks.workunit.client.0.smithi027.stdout:3/287: write d1/db/f3c [978654,75132] 0 2021-11-03T17:14:14.800 INFO:tasks.workunit.client.0.smithi027.stdout:3/288: write d1/d40/f58 [534603,86226] 0 2021-11-03T17:14:14.800 INFO:tasks.workunit.client.0.smithi027.stdout:3/289: creat d1/d40/f5a x:0 0 0 2021-11-03T17:14:14.801 INFO:tasks.workunit.client.0.smithi027.stdout:3/290: creat d1/db/f5b x:0 0 0 2021-11-03T17:14:14.801 INFO:tasks.workunit.client.0.smithi027.stdout:3/291: chown d1/f34 14315009 1 2021-11-03T17:14:14.802 INFO:tasks.workunit.client.0.smithi027.stdout:3/292: creat d1/d40/f5c x:0 0 0 2021-11-03T17:14:14.802 INFO:tasks.workunit.client.0.smithi027.stdout:3/293: chown d1/f1d 1880456728 1 2021-11-03T17:14:14.804 INFO:tasks.workunit.client.0.smithi027.stdout:3/294: link d1/d40/f49 d1/db/d52/f5d 0 2021-11-03T17:14:14.806 INFO:tasks.workunit.client.0.smithi027.stdout:3/295: rename d1/db/f3d to d1/f5e 0 2021-11-03T17:14:14.820 INFO:tasks.workunit.client.0.smithi027.stdout:3/296: dwrite d1/f2f [0,4194304] 0 2021-11-03T17:14:14.824 INFO:tasks.workunit.client.0.smithi027.stdout:3/297: symlink d1/l5f 0 2021-11-03T17:14:14.825 INFO:tasks.workunit.client.0.smithi027.stdout:3/298: creat d1/db/f60 x:0 0 0 2021-11-03T17:14:14.826 INFO:tasks.workunit.client.0.smithi027.stdout:3/299: mknod d1/d40/c61 1 2021-11-03T17:14:14.826 INFO:tasks.workunit.client.0.smithi027.stdout:3/300: fdatasync d1/d40/f46 0 2021-11-03T17:14:14.826 INFO:tasks.workunit.client.0.smithi027.stdout:3/301: dread - d1/f41 zero size 2021-11-03T17:14:14.828 INFO:tasks.workunit.client.0.smithi027.stdout:3/302: fsync d1/f4c 0 2021-11-03T17:14:14.832 INFO:tasks.workunit.client.0.smithi027.stdout:3/303: truncate d1/db/d52/f5d 80850 0 2021-11-03T17:14:14.834 INFO:tasks.workunit.client.0.smithi027.stdout:3/304: creat d1/db/d52/d57/f62 x:0 0 0 2021-11-03T17:14:14.834 INFO:tasks.workunit.client.0.smithi027.stdout:3/305: dread - d1/db/f3e zero size 2021-11-03T17:14:14.834 INFO:tasks.workunit.client.0.smithi027.stdout:3/306: fdatasync d1/d40/f5c 0 2021-11-03T17:14:14.834 INFO:tasks.workunit.client.0.smithi027.stdout:3/307: write d1/f4e [405237,1723] 0 2021-11-03T17:14:14.835 INFO:tasks.workunit.client.0.smithi027.stdout:3/308: dread - d1/d40/f48 zero size 2021-11-03T17:14:14.835 INFO:tasks.workunit.client.0.smithi027.stdout:3/309: stat d1/f14 0 2021-11-03T17:14:14.961 INFO:tasks.workunit.client.0.smithi027.stdout:8/371: sync 2021-11-03T17:14:14.974 INFO:tasks.workunit.client.0.smithi027.stdout:8/372: dwrite d7/d10/d18/f61 [0,4194304] 0 2021-11-03T17:14:14.974 INFO:tasks.workunit.client.0.smithi027.stdout:8/373: write d7/d10/f71 [383528,93505] 0 2021-11-03T17:14:14.978 INFO:tasks.workunit.client.0.smithi027.stdout:8/374: rename d7/d10/d3f/f4c to d7/d10/d21/f72 0 2021-11-03T17:14:14.982 INFO:tasks.workunit.client.0.smithi027.stdout:8/375: link d7/d10/d3f/f4e d7/d10/d3f/f73 0 2021-11-03T17:14:14.983 INFO:tasks.workunit.client.0.smithi027.stdout:8/376: rmdir d7/d10 39 2021-11-03T17:14:14.984 INFO:tasks.workunit.client.0.smithi027.stdout:8/377: rename d7/d10/d3f/d55/f69 to d7/d10/d21/f74 0 2021-11-03T17:14:14.985 INFO:tasks.workunit.client.0.smithi027.stdout:8/378: mkdir d7/d75 0 2021-11-03T17:14:14.986 INFO:tasks.workunit.client.0.smithi027.stdout:8/379: rmdir d7/d53 39 2021-11-03T17:14:14.988 INFO:tasks.workunit.client.0.smithi027.stdout:8/380: write d7/d2d/d38/f3d [74623,120409] 0 2021-11-03T17:14:15.004 INFO:tasks.workunit.client.0.smithi027.stdout:8/381: dwrite d7/d10/d3f/d55/f64 [0,4194304] 0 2021-11-03T17:14:15.010 INFO:tasks.workunit.client.0.smithi027.stdout:4/329: sync 2021-11-03T17:14:15.010 INFO:tasks.workunit.client.0.smithi027.stdout:9/408: sync 2021-11-03T17:14:15.011 INFO:tasks.workunit.client.0.smithi027.stdout:9/409: chown db/d1b/d1c/d1d/f6d 359034 1 2021-11-03T17:14:15.011 INFO:tasks.workunit.client.0.smithi027.stdout:9/410: readlink db/d1b/d23/d2c/l2d 0 2021-11-03T17:14:15.013 INFO:tasks.workunit.client.0.smithi027.stdout:8/382: rename d7/d10/d21/f74 to d7/d10/d3f/f76 0 2021-11-03T17:14:15.013 INFO:tasks.workunit.client.0.smithi027.stdout:8/383: truncate d7/d10/d21/f39 103546 0 2021-11-03T17:14:15.014 INFO:tasks.workunit.client.0.smithi027.stdout:8/384: mknod d7/d10/c77 1 2021-11-03T17:14:15.016 INFO:tasks.workunit.client.0.smithi027.stdout:4/330: symlink d6/db/d21/d2b/d2d/l6f 0 2021-11-03T17:14:15.017 INFO:tasks.workunit.client.0.smithi027.stdout:9/411: rmdir db/d1b 39 2021-11-03T17:14:15.019 INFO:tasks.workunit.client.0.smithi027.stdout:8/385: rename d7/d10/d21/f72 to d7/d10/d3f/f78 0 2021-11-03T17:14:15.042 INFO:tasks.workunit.client.0.smithi027.stdout:9/412: dwrite db/d1b/d23/d2c/f89 [0,4194304] 0 2021-11-03T17:14:15.042 INFO:tasks.workunit.client.0.smithi027.stdout:8/386: dwrite d7/d10/d18/d1a/f3c [0,4194304] 0 2021-11-03T17:14:15.042 INFO:tasks.workunit.client.0.smithi027.stdout:9/413: dread - db/f1f zero size 2021-11-03T17:14:15.043 INFO:tasks.workunit.client.0.smithi027.stdout:4/331: truncate d6/db/d21/d2b/d2d/d3e/d58/f61 587513 0 2021-11-03T17:14:15.043 INFO:tasks.workunit.client.0.smithi027.stdout:8/387: read d7/d53/f6c [950031,125128] 0 2021-11-03T17:14:15.057 INFO:tasks.workunit.client.0.smithi027.stdout:9/414: mkdir db/d1b/d2b/d41/d97 0 2021-11-03T17:14:15.057 INFO:tasks.workunit.client.0.smithi027.stdout:9/415: mknod db/d1b/d1c/d1d/c98 1 2021-11-03T17:14:15.061 INFO:tasks.workunit.client.0.smithi027.stdout:4/332: rename d6/db/d21/d2b/d2d to d6/d9/d16/d36/d70 0 2021-11-03T17:14:15.062 INFO:tasks.workunit.client.0.smithi027.stdout:4/333: readlink d6/l5b 0 2021-11-03T17:14:15.066 INFO:tasks.workunit.client.0.smithi027.stdout:8/388: rename d7/l2c to d7/d2d/l79 0 2021-11-03T17:14:15.066 INFO:tasks.workunit.client.0.smithi027.stdout:8/389: mknod d7/d53/d67/d68/c7a 1 2021-11-03T17:14:15.067 INFO:tasks.workunit.client.0.smithi027.stdout:8/390: mknod d7/d2d/d38/c7b 1 2021-11-03T17:14:15.067 INFO:tasks.workunit.client.0.smithi027.stdout:8/391: mknod d7/d53/d67/d68/c7c 1 2021-11-03T17:14:15.070 INFO:tasks.workunit.client.0.smithi027.stdout:4/334: creat d6/d9/d16/d36/d70/f71 x:0 0 0 2021-11-03T17:14:15.072 INFO:tasks.workunit.client.0.smithi027.stdout:9/416: rename l7 to db/d1b/d1c/d1d/d40/l99 0 2021-11-03T17:14:15.074 INFO:tasks.workunit.client.0.smithi027.stdout:9/417: creat db/d1b/d1c/d1d/d40/d60/f9a x:0 0 0 2021-11-03T17:14:15.075 INFO:tasks.workunit.client.0.smithi027.stdout:9/418: mknod db/d1b/d2b/d5e/c9b 1 2021-11-03T17:14:15.076 INFO:tasks.workunit.client.0.smithi027.stdout:9/419: mkdir db/d1b/d2b/d9c 0 2021-11-03T17:14:15.078 INFO:tasks.workunit.client.0.smithi027.stdout:9/420: rename db/d1b/d1c/d1d/f6d to db/d1b/d2b/d41/d97/f9d 0 2021-11-03T17:14:15.080 INFO:tasks.workunit.client.0.smithi027.stdout:9/421: getdents db/d1b/d23/d2c 0 2021-11-03T17:14:15.080 INFO:tasks.workunit.client.0.smithi027.stdout:9/422: chown db/d1b/d23/f7a 447565 1 2021-11-03T17:14:15.083 INFO:tasks.workunit.client.0.smithi027.stdout:9/423: link db/d1b/d23/f7a db/d1b/d1c/d1d/d40/f9e 0 2021-11-03T17:14:15.083 INFO:tasks.workunit.client.0.smithi027.stdout:9/424: dread - db/f75 zero size 2021-11-03T17:14:15.085 INFO:tasks.workunit.client.0.smithi027.stdout:9/425: unlink db/d1b/d2b/d41/d6c/l94 0 2021-11-03T17:14:15.093 INFO:tasks.workunit.client.0.smithi027.stdout:8/392: dwrite f6 [0,4194304] 0 2021-11-03T17:14:15.095 INFO:tasks.workunit.client.0.smithi027.stdout:8/393: creat d7/f7d x:0 0 0 2021-11-03T17:14:15.100 INFO:tasks.workunit.client.0.smithi027.stdout:4/335: dwrite d6/db/d21/f69 [0,4194304] 0 2021-11-03T17:14:15.100 INFO:tasks.workunit.client.0.smithi027.stdout:4/336: mknod d6/d9/d16/d36/c72 1 2021-11-03T17:14:15.100 INFO:tasks.workunit.client.0.smithi027.stdout:4/337: stat d6/l1c 0 2021-11-03T17:14:15.100 INFO:tasks.workunit.client.0.smithi027.stdout:4/338: dread - d6/d9/d16/d36/f6a zero size 2021-11-03T17:14:15.111 INFO:tasks.workunit.client.0.smithi027.stdout:8/394: dwrite d7/d10/d18/f33 [0,4194304] 0 2021-11-03T17:14:15.111 INFO:tasks.workunit.client.0.smithi027.stdout:8/395: write d7/d10/d18/d1a/f3c [4645998,128947] 0 2021-11-03T17:14:15.112 INFO:tasks.workunit.client.0.smithi027.stdout:4/339: dread d6/db/d21/f38 [0,4194304] 0 2021-11-03T17:14:15.112 INFO:tasks.workunit.client.0.smithi027.stdout:4/340: read d6/db/d21/f37 [802695,10553] 0 2021-11-03T17:14:15.112 INFO:tasks.workunit.client.0.smithi027.stdout:4/341: fsync d6/db/d21/f2c 0 2021-11-03T17:14:15.115 INFO:tasks.workunit.client.0.smithi027.stdout:9/426: unlink db/d1b/d2b/d41/d97/f9d 0 2021-11-03T17:14:15.115 INFO:tasks.workunit.client.0.smithi027.stdout:9/427: dread - db/d1b/d1c/d1d/d40/d60/f9a zero size 2021-11-03T17:14:15.115 INFO:tasks.workunit.client.0.smithi027.stdout:9/428: write db/d1b/d1c/d1d/d40/f9e [621878,79356] 0 2021-11-03T17:14:15.118 INFO:tasks.workunit.client.0.smithi027.stdout:8/396: dread d7/f62 [0,4194304] 0 2021-11-03T17:14:15.121 INFO:tasks.workunit.client.0.smithi027.stdout:9/429: mkdir db/d1b/d2b/d41/d95/d9f 0 2021-11-03T17:14:15.121 INFO:tasks.workunit.client.0.smithi027.stdout:9/430: fsync db/fd 0 2021-11-03T17:14:15.122 INFO:tasks.workunit.client.0.smithi027.stdout:9/431: chown db/d1b/d2b/d41/d6c 90258 1 2021-11-03T17:14:15.122 INFO:tasks.workunit.client.0.smithi027.stdout:9/432: chown db/d1b/d2b/d41/l8b 1960 1 2021-11-03T17:14:15.122 INFO:tasks.workunit.client.0.smithi027.stdout:9/433: chown db/d1b/d8c/l8d 60071939 1 2021-11-03T17:14:15.123 INFO:tasks.workunit.client.0.smithi027.stdout:4/342: rename d6/d9/d16/d17/f31 to d6/d9/f73 0 2021-11-03T17:14:15.124 INFO:tasks.workunit.client.0.smithi027.stdout:8/397: link d7/f36 d7/d53/d67/d68/f7e 0 2021-11-03T17:14:15.125 INFO:tasks.workunit.client.0.smithi027.stdout:8/398: chown d7/d2d/l58 185 1 2021-11-03T17:14:15.125 INFO:tasks.workunit.client.0.smithi027.stdout:8/399: write d7/d10/d3f/f76 [771170,93782] 0 2021-11-03T17:14:15.126 INFO:tasks.workunit.client.0.smithi027.stdout:9/434: symlink db/d1b/d2b/d5e/la0 0 2021-11-03T17:14:15.127 INFO:tasks.workunit.client.0.smithi027.stdout:4/343: rmdir d6/d9/d28/d40 39 2021-11-03T17:14:15.127 INFO:tasks.workunit.client.0.smithi027.stdout:4/344: fsync d6/d9/d16/d36/d57/d5a/f5c 0 2021-11-03T17:14:15.130 INFO:tasks.workunit.client.0.smithi027.stdout:8/400: mkdir d7/d10/d21/d7f 0 2021-11-03T17:14:15.130 INFO:tasks.workunit.client.0.smithi027.stdout:9/435: getdents db/d1b/d23 0 2021-11-03T17:14:15.132 INFO:tasks.workunit.client.0.smithi027.stdout:4/345: symlink d6/d9/d16/d36/d70/d3e/d58/l74 0 2021-11-03T17:14:15.138 INFO:tasks.workunit.client.0.smithi027.stdout:8/401: dread d7/d10/d18/f49 [0,4194304] 0 2021-11-03T17:14:15.139 INFO:tasks.workunit.client.0.smithi027.stdout:9/436: creat db/d1b/d1c/d1d/fa1 x:0 0 0 2021-11-03T17:14:15.140 INFO:tasks.workunit.client.0.smithi027.stdout:4/346: creat d6/d9/d16/d36/d57/f75 x:0 0 0 2021-11-03T17:14:15.141 INFO:tasks.workunit.client.0.smithi027.stdout:8/402: symlink d7/d10/d21/d7f/l80 0 2021-11-03T17:14:15.141 INFO:tasks.workunit.client.0.smithi027.stdout:4/347: getdents d6/d9 0 2021-11-03T17:14:15.143 INFO:tasks.workunit.client.0.smithi027.stdout:4/348: creat d6/d9/d28/d40/f76 x:0 0 0 2021-11-03T17:14:15.143 INFO:tasks.workunit.client.0.smithi027.stdout:4/349: chown d6/d9/f1b 17698 1 2021-11-03T17:14:15.146 INFO:tasks.workunit.client.0.smithi027.stdout:4/350: creat d6/db/d21/d41/f77 x:0 0 0 2021-11-03T17:14:15.146 INFO:tasks.workunit.client.0.smithi027.stdout:4/351: stat d6/l59 0 2021-11-03T17:14:15.146 INFO:tasks.workunit.client.0.smithi027.stdout:4/352: mknod d6/db/c78 1 2021-11-03T17:14:15.146 INFO:tasks.workunit.client.0.smithi027.stdout:4/353: read - d6/d9/d16/d36/d70/d3e/f6b zero size 2021-11-03T17:14:15.147 INFO:tasks.workunit.client.0.smithi027.stdout:4/354: chown d6/d9/d16/d36/d70 92992711 1 2021-11-03T17:14:15.148 INFO:tasks.workunit.client.0.smithi027.stdout:4/355: truncate d6/db/d21/d2b/f45 4012177 0 2021-11-03T17:14:15.149 INFO:tasks.workunit.client.0.smithi027.stdout:9/437: dread db/d1b/d2b/f2a [0,4194304] 0 2021-11-03T17:14:15.149 INFO:tasks.workunit.client.0.smithi027.stdout:9/438: chown db/d1b/d2b/f4a 1528280839 1 2021-11-03T17:14:15.149 INFO:tasks.workunit.client.0.smithi027.stdout:9/439: mknod db/d1b/d2b/d41/d95/ca2 1 2021-11-03T17:14:15.149 INFO:tasks.workunit.client.0.smithi027.stdout:4/356: mkdir d6/db/d21/d41/d44/d79 0 2021-11-03T17:14:15.149 INFO:tasks.workunit.client.0.smithi027.stdout:4/357: stat d6/l59 0 2021-11-03T17:14:15.150 INFO:tasks.workunit.client.0.smithi027.stdout:9/440: write db/d1b/f20 [436771,34589] 0 2021-11-03T17:14:15.150 INFO:tasks.workunit.client.0.smithi027.stdout:4/358: read d6/d9/d28/f4b [1140289,116341] 0 2021-11-03T17:14:15.150 INFO:tasks.workunit.client.0.smithi027.stdout:4/359: symlink d6/d9/d16/d36/d57/l7a 0 2021-11-03T17:14:15.151 INFO:tasks.workunit.client.0.smithi027.stdout:4/360: mknod d6/db/c7b 1 2021-11-03T17:14:15.151 INFO:tasks.workunit.client.0.smithi027.stdout:4/361: chown d6/f48 6 1 2021-11-03T17:14:15.151 INFO:tasks.workunit.client.0.smithi027.stdout:4/362: chown d6/d9/d16/d17/f6e 21006517 1 2021-11-03T17:14:15.151 INFO:tasks.workunit.client.0.smithi027.stdout:4/363: write d6/d9/d28/f4b [2489247,82684] 0 2021-11-03T17:14:15.156 INFO:tasks.workunit.client.0.smithi027.stdout:4/364: link d6/d24/l33 d6/d24/l7c 0 2021-11-03T17:14:15.160 INFO:tasks.workunit.client.0.smithi027.stdout:4/365: getdents d6 0 2021-11-03T17:14:15.162 INFO:tasks.workunit.client.0.smithi027.stdout:4/366: symlink d6/d24/d6d/l7d 0 2021-11-03T17:14:15.164 INFO:tasks.workunit.client.0.smithi027.stdout:4/367: creat d6/db/f7e x:0 0 0 2021-11-03T17:14:15.167 INFO:tasks.workunit.client.0.smithi027.stdout:4/368: link d6/d9/f1b d6/d9/d16/f7f 0 2021-11-03T17:14:15.168 INFO:tasks.workunit.client.0.smithi027.stdout:4/369: write d6/d9/d16/d36/d57/d5a/f5c [1374581,22524] 0 2021-11-03T17:14:15.170 INFO:tasks.workunit.client.0.smithi027.stdout:9/441: dwrite db/d1b/d2b/f86 [0,4194304] 0 2021-11-03T17:14:15.170 INFO:tasks.workunit.client.0.smithi027.stdout:8/403: dwrite d7/d10/d18/d1a/f3c [0,4194304] 0 2021-11-03T17:14:15.173 INFO:tasks.workunit.client.0.smithi027.stdout:9/442: getdents db/d1b/d2b 0 2021-11-03T17:14:15.175 INFO:tasks.workunit.client.0.smithi027.stdout:8/404: symlink d7/d53/d67/l81 0 2021-11-03T17:14:15.175 INFO:tasks.workunit.client.0.smithi027.stdout:8/405: write d7/d10/d21/f63 [413714,28228] 0 2021-11-03T17:14:15.175 INFO:tasks.workunit.client.0.smithi027.stdout:8/406: chown d7/d53/d67 4 1 2021-11-03T17:14:15.179 INFO:tasks.workunit.client.0.smithi027.stdout:9/443: rmdir db/d1b/d2b/d41/d95 39 2021-11-03T17:14:15.179 INFO:tasks.workunit.client.0.smithi027.stdout:9/444: rename db/d1b/d1c/d1d to db/d1b/d1c/d1d/d40/d60/da3 22 2021-11-03T17:14:15.192 INFO:tasks.workunit.client.0.smithi027.stdout:0/412: sync 2021-11-03T17:14:15.192 INFO:tasks.workunit.client.0.smithi027.stdout:9/445: dread db/d1b/f38 [0,4194304] 0 2021-11-03T17:14:15.193 INFO:tasks.workunit.client.0.smithi027.stdout:9/446: write db/d1b/d2b/d41/f4d [595858,120831] 0 2021-11-03T17:14:15.193 INFO:tasks.workunit.client.0.smithi027.stdout:9/447: write db/f75 [874609,15580] 0 2021-11-03T17:14:15.194 INFO:tasks.workunit.client.0.smithi027.stdout:4/370: dwrite d6/db/d21/f2c [4194304,4194304] 0 2021-11-03T17:14:15.194 INFO:tasks.workunit.client.0.smithi027.stdout:0/413: read d9/f2a [3866271,101666] 0 2021-11-03T17:14:15.194 INFO:tasks.workunit.client.0.smithi027.stdout:0/414: fsync d9/da/fe 0 2021-11-03T17:14:15.195 INFO:tasks.workunit.client.0.smithi027.stdout:8/407: dwrite d7/d2d/d38/f44 [0,4194304] 0 2021-11-03T17:14:15.195 INFO:tasks.workunit.client.0.smithi027.stdout:8/408: write d7/d53/d67/f6b [893789,118665] 0 2021-11-03T17:14:15.199 INFO:tasks.workunit.client.0.smithi027.stdout:4/371: link d6/l1c d6/d24/l80 0 2021-11-03T17:14:15.200 INFO:tasks.workunit.client.0.smithi027.stdout:4/372: fdatasync d6/d9/d16/d36/d70/f71 0 2021-11-03T17:14:15.200 INFO:tasks.workunit.client.0.smithi027.stdout:4/373: dread - d6/db/d21/d41/f77 zero size 2021-11-03T17:14:15.200 INFO:tasks.workunit.client.0.smithi027.stdout:0/415: symlink d9/da/d64/d70/d6d/l89 0 2021-11-03T17:14:15.205 INFO:tasks.workunit.client.0.smithi027.stdout:4/374: write f4 [2203549,66418] 0 2021-11-03T17:14:15.207 INFO:tasks.workunit.client.0.smithi027.stdout:4/375: mkdir d6/d9/d28/d40/d81 0 2021-11-03T17:14:15.211 INFO:tasks.workunit.client.0.smithi027.stdout:0/416: dread d9/da/d30/f54 [0,4194304] 0 2021-11-03T17:14:15.211 INFO:tasks.workunit.client.0.smithi027.stdout:4/376: rmdir d6/d9/d16/d36 39 2021-11-03T17:14:15.212 INFO:tasks.workunit.client.0.smithi027.stdout:4/377: readlink d6/db/l4d 0 2021-11-03T17:14:15.212 INFO:tasks.workunit.client.0.smithi027.stdout:4/378: mknod d6/d9/c82 1 2021-11-03T17:14:15.213 INFO:tasks.workunit.client.0.smithi027.stdout:0/417: creat d9/d81/f8a x:0 0 0 2021-11-03T17:14:15.217 INFO:tasks.workunit.client.0.smithi027.stdout:9/448: dwrite db/d1b/d23/d2c/f4c [0,4194304] 0 2021-11-03T17:14:15.217 INFO:tasks.workunit.client.0.smithi027.stdout:4/379: rename d6/d9/d16/d36/f4a to d6/d9/d28/d40/f83 0 2021-11-03T17:14:15.218 INFO:tasks.workunit.client.0.smithi027.stdout:4/380: write d6/d9/d16/d36/f42 [1640236,119101] 0 2021-11-03T17:14:15.219 INFO:tasks.workunit.client.0.smithi027.stdout:0/418: creat d9/da/f8b x:0 0 0 2021-11-03T17:14:15.219 INFO:tasks.workunit.client.0.smithi027.stdout:0/419: mknod d9/da/d31/c8c 1 2021-11-03T17:14:15.219 INFO:tasks.workunit.client.0.smithi027.stdout:8/409: dwrite d7/f7d [0,4194304] 0 2021-11-03T17:14:15.222 INFO:tasks.workunit.client.0.smithi027.stdout:9/449: creat db/d1b/d1c/d1d/d40/d60/fa4 x:0 0 0 2021-11-03T17:14:15.224 INFO:tasks.workunit.client.0.smithi027.stdout:4/381: mkdir d6/d9/d16/d36/d70/d3e/d58/d84 0 2021-11-03T17:14:15.224 INFO:tasks.workunit.client.0.smithi027.stdout:4/382: fsync d6/d24/f46 0 2021-11-03T17:14:15.228 INFO:tasks.workunit.client.0.smithi027.stdout:0/420: rmdir d9 39 2021-11-03T17:14:15.231 INFO:tasks.workunit.client.0.smithi027.stdout:4/383: dread d6/d9/d16/d36/f42 [0,4194304] 0 2021-11-03T17:14:15.233 INFO:tasks.workunit.client.0.smithi027.stdout:8/410: creat d7/d10/d18/f82 x:0 0 0 2021-11-03T17:14:15.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:15 smithi115 conmon[34670]: cluster 2021-11-03T17:14:14.605984+0000 mgr.smithi027.ufhhvb (mgr. 2021-11-03T17:14:15.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:15 smithi115 conmon[34670]: 14162) 308 : cluster [DBG] pgmap v254: 65 pgs: 65 active+clean; 1.5 GiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 63 MiB/s rd, 127 MiB/s wr, 194 op/s 2021-11-03T17:14:15.235 INFO:tasks.workunit.client.0.smithi027.stdout:9/450: rename db/d1b/d1c/d1d/f21 to db/d1b/d8e/fa5 0 2021-11-03T17:14:15.235 INFO:tasks.workunit.client.0.smithi027.stdout:9/451: chown db/d1b/d23/f73 150 1 2021-11-03T17:14:15.235 INFO:tasks.workunit.client.0.smithi027.stdout:9/452: fdatasync db/d1b/d1c/d1d/d40/d60/f9a 0 2021-11-03T17:14:15.236 INFO:tasks.workunit.client.0.smithi027.stdout:4/384: symlink d6/d9/d16/d36/d70/d3e/l85 0 2021-11-03T17:14:15.237 INFO:tasks.workunit.client.0.smithi027.stdout:9/453: write db/f16 [815029,21916] 0 2021-11-03T17:14:15.237 INFO:tasks.workunit.client.0.smithi027.stdout:9/454: stat db/d1b/d8c 0 2021-11-03T17:14:15.239 INFO:tasks.workunit.client.0.smithi027.stdout:8/411: rename d7/f8 to d7/d10/d21/d41/f83 0 2021-11-03T17:14:15.239 INFO:tasks.workunit.client.0.smithi027.stdout:9/455: creat db/d1b/d1c/d1d/d40/d60/fa6 x:0 0 0 2021-11-03T17:14:15.243 INFO:tasks.workunit.client.0.smithi027.stdout:8/412: rename d7/d75 to d7/d10/d18/d84 0 2021-11-03T17:14:15.243 INFO:tasks.workunit.client.0.smithi027.stdout:0/421: dwrite d9/da/d64/d70/d5f/f74 [0,4194304] 0 2021-11-03T17:14:15.244 INFO:tasks.workunit.client.0.smithi027.stdout:9/456: creat db/d1b/d1c/d1d/d40/fa7 x:0 0 0 2021-11-03T17:14:15.246 INFO:tasks.workunit.client.0.smithi027.stdout:8/413: creat d7/d10/d4f/f85 x:0 0 0 2021-11-03T17:14:15.250 INFO:tasks.workunit.client.0.smithi027.stdout:0/422: rename d9/da/d30/f3e to d9/da/f8d 0 2021-11-03T17:14:15.250 INFO:tasks.workunit.client.0.smithi027.stdout:0/423: write d9/da/d64/d70/d6d/f6e [145986,36442] 0 2021-11-03T17:14:15.250 INFO:tasks.workunit.client.0.smithi027.stdout:0/424: mknod d9/da/d64/d70/c8e 1 2021-11-03T17:14:15.252 INFO:tasks.workunit.client.0.smithi027.stdout:9/457: creat db/fa8 x:0 0 0 2021-11-03T17:14:15.252 INFO:tasks.workunit.client.0.smithi027.stdout:8/414: creat d7/d10/d3f/d55/f86 x:0 0 0 2021-11-03T17:14:15.255 INFO:tasks.workunit.client.0.smithi027.stdout:9/458: creat db/d1b/d1c/d1d/fa9 x:0 0 0 2021-11-03T17:14:15.256 INFO:tasks.workunit.client.0.smithi027.stdout:4/385: dwrite d6/d9/d16/d17/f56 [0,4194304] 0 2021-11-03T17:14:15.256 INFO:tasks.workunit.client.0.smithi027.stdout:4/386: dread - d6/d9/d16/d36/d70/d3e/d58/f60 zero size 2021-11-03T17:14:15.256 INFO:tasks.workunit.client.0.smithi027.stdout:8/415: creat d7/f87 x:0 0 0 2021-11-03T17:14:15.257 INFO:tasks.workunit.client.0.smithi027.stdout:4/387: write d6/d9/d16/f32 [7771423,14744] 0 2021-11-03T17:14:15.263 INFO:tasks.workunit.client.0.smithi027.stdout:9/459: mkdir db/d1b/d2b/d5e/daa 0 2021-11-03T17:14:15.268 INFO:tasks.workunit.client.0.smithi027.stdout:6/424: sync 2021-11-03T17:14:15.270 INFO:tasks.workunit.client.0.smithi027.stdout:8/416: creat d7/d10/d3f/f88 x:0 0 0 2021-11-03T17:14:15.277 INFO:tasks.workunit.client.0.smithi027.stdout:4/388: rename d6/d9/d28/f4b to d6/d9/d16/d36/d70/d3e/d58/d84/f86 0 2021-11-03T17:14:15.277 INFO:tasks.workunit.client.0.smithi027.stdout:4/389: chown d6/d9/d16/d17/f56 110706712 1 2021-11-03T17:14:15.278 INFO:tasks.workunit.client.0.smithi027.stdout:4/390: write d6/d9/d16/d36/d70/f71 [508967,127413] 0 2021-11-03T17:14:15.287 INFO:tasks.workunit.client.0.smithi027.stdout:9/460: creat db/d1b/d2b/d9c/fab x:0 0 0 2021-11-03T17:14:15.288 INFO:tasks.workunit.client.0.smithi027.stdout:6/425: getdents dc/d16/d1b/d3c 0 2021-11-03T17:14:15.289 INFO:tasks.workunit.client.0.smithi027.stdout:9/461: dread db/d1b/d2b/d41/f74 [0,4194304] 0 2021-11-03T17:14:15.290 INFO:tasks.workunit.client.0.smithi027.stdout:6/426: write dc/d16/d1b/d3c/d51/f70 [1541776,63361] 0 2021-11-03T17:14:15.290 INFO:tasks.workunit.client.0.smithi027.stdout:6/427: fsync dc/f64 0 2021-11-03T17:14:15.290 INFO:tasks.workunit.client.0.smithi027.stdout:8/417: creat d7/d10/f89 x:0 0 0 2021-11-03T17:14:15.291 INFO:tasks.workunit.client.0.smithi027.stdout:4/391: creat d6/d9/d28/f87 x:0 0 0 2021-11-03T17:14:15.292 INFO:tasks.workunit.client.0.smithi027.stdout:9/462: rmdir db/d1b 39 2021-11-03T17:14:15.292 INFO:tasks.workunit.client.0.smithi027.stdout:9/463: truncate db/d1b/d23/f7a 1350719 0 2021-11-03T17:14:15.296 INFO:tasks.workunit.client.0.smithi027.stdout:6/428: rename dc/d16/d1b/d2d/d34/l49 to dc/d16/d62/l76 0 2021-11-03T17:14:15.297 INFO:tasks.workunit.client.0.smithi027.stdout:6/429: mknod dc/d16/d62/c77 1 2021-11-03T17:14:15.298 INFO:tasks.workunit.client.0.smithi027.stdout:9/464: creat db/d1b/d8c/fac x:0 0 0 2021-11-03T17:14:15.299 INFO:tasks.workunit.client.0.smithi027.stdout:9/465: write db/d1b/d23/d2c/f4c [1062352,34593] 0 2021-11-03T17:14:15.300 INFO:tasks.workunit.client.0.smithi027.stdout:6/430: creat dc/d16/d1b/d3c/d51/f78 x:0 0 0 2021-11-03T17:14:15.303 INFO:tasks.workunit.client.0.smithi027.stdout:9/466: mkdir db/d1b/d2b/d41/dad 0 2021-11-03T17:14:15.304 INFO:tasks.workunit.client.0.smithi027.stdout:6/431: mkdir dc/d16/d1b/d3c/d51/d5f/d79 0 2021-11-03T17:14:15.305 INFO:tasks.workunit.client.0.smithi027.stdout:6/432: write dc/d16/d1b/d3c/d48/f6f [108733,16513] 0 2021-11-03T17:14:15.306 INFO:tasks.workunit.client.0.smithi027.stdout:9/467: rename db/d1b/d23/l5c to db/d1b/d1c/d1d/d40/lae 0 2021-11-03T17:14:15.315 INFO:tasks.workunit.client.0.smithi027.stdout:6/433: unlink dc/d16/d30/f65 0 2021-11-03T17:14:15.316 INFO:tasks.workunit.client.0.smithi027.stdout:9/468: symlink db/d1b/d2b/d41/dad/laf 0 2021-11-03T17:14:15.317 INFO:tasks.workunit.client.0.smithi027.stdout:8/418: dwrite d7/d10/d21/f25 [0,4194304] 0 2021-11-03T17:14:15.318 INFO:tasks.workunit.client.0.smithi027.stdout:8/419: mknod d7/d10/d21/d7f/c8a 1 2021-11-03T17:14:15.318 INFO:tasks.workunit.client.0.smithi027.stdout:8/420: mknod d7/d10/d18/d84/c8b 1 2021-11-03T17:14:15.318 INFO:tasks.workunit.client.0.smithi027.stdout:8/421: write d7/d10/d3f/f88 [980129,79539] 0 2021-11-03T17:14:15.318 INFO:tasks.workunit.client.0.smithi027.stdout:8/422: readlink d7/d10/d21/l51 0 2021-11-03T17:14:15.319 INFO:tasks.workunit.client.0.smithi027.stdout:8/423: chown l4 449988 1 2021-11-03T17:14:15.319 INFO:tasks.workunit.client.0.smithi027.stdout:8/424: dread - d7/d10/d18/f82 zero size 2021-11-03T17:14:15.319 INFO:tasks.workunit.client.0.smithi027.stdout:8/425: chown d7/d10/d21/l51 0 1 2021-11-03T17:14:15.319 INFO:tasks.workunit.client.0.smithi027.stdout:8/426: write d7/d10/d18/d1a/f60 [4707833,58207] 0 2021-11-03T17:14:15.320 INFO:tasks.workunit.client.0.smithi027.stdout:8/427: chown d7/d2d/l29 58 1 2021-11-03T17:14:15.320 INFO:tasks.workunit.client.0.smithi027.stdout:8/428: write d7/d53/f6c [4667679,48384] 0 2021-11-03T17:14:15.320 INFO:tasks.workunit.client.0.smithi027.stdout:6/434: dread dc/d16/d1b/d3c/d51/f5b [0,4194304] 0 2021-11-03T17:14:15.320 INFO:tasks.workunit.client.0.smithi027.stdout:4/392: dwrite d6/d9/d16/d17/f6e [0,4194304] 0 2021-11-03T17:14:15.322 INFO:tasks.workunit.client.0.smithi027.stdout:9/469: link db/f1f db/d1b/fb0 0 2021-11-03T17:14:15.323 INFO:tasks.workunit.client.0.smithi027.stdout:9/470: dread db/d1b/f20 [0,4194304] 0 2021-11-03T17:14:15.328 INFO:tasks.workunit.client.0.smithi027.stdout:8/429: creat d7/d10/d21/f8c x:0 0 0 2021-11-03T17:14:15.328 INFO:tasks.workunit.client.0.smithi027.stdout:8/430: chown d7/d10/d18 14631 1 2021-11-03T17:14:15.336 INFO:tasks.workunit.client.0.smithi027.stdout:4/393: dwrite d6/d9/f1b [0,4194304] 0 2021-11-03T17:14:15.343 INFO:tasks.workunit.client.0.smithi027.stdout:9/471: unlink db/d1b/d2b/f2a 0 2021-11-03T17:14:15.351 INFO:tasks.workunit.client.0.smithi027.stdout:9/472: rename db/lf to db/d1b/d2b/d41/d97/lb1 0 2021-11-03T17:14:15.351 INFO:tasks.workunit.client.0.smithi027.stdout:9/473: fdatasync db/d1b/d1c/d1d/f49 0 2021-11-03T17:14:15.351 INFO:tasks.workunit.client.0.smithi027.stdout:9/474: chown db/d1b/d23/d2c/l2d 0 1 2021-11-03T17:14:15.352 INFO:tasks.workunit.client.0.smithi027.stdout:9/475: write db/d1b/f64 [941260,81438] 0 2021-11-03T17:14:15.352 INFO:tasks.workunit.client.0.smithi027.stdout:9/476: creat db/d1b/fb2 x:0 0 0 2021-11-03T17:14:15.356 INFO:tasks.workunit.client.0.smithi027.stdout:8/431: write d7/d10/f12 [286709,50351] 0 2021-11-03T17:14:15.357 INFO:tasks.workunit.client.0.smithi027.stdout:8/432: dread - d7/d10/d4f/f85 zero size 2021-11-03T17:14:15.357 INFO:tasks.workunit.client.0.smithi027.stdout:4/394: dwrite d6/d9/d16/f26 [0,4194304] 0 2021-11-03T17:14:15.359 INFO:tasks.workunit.client.0.smithi027.stdout:8/433: rename d7/d10/d21/d41/f83 to d7/d10/d21/f8d 0 2021-11-03T17:14:15.359 INFO:tasks.workunit.client.0.smithi027.stdout:8/434: mknod d7/d2d/d38/c8e 1 2021-11-03T17:14:15.364 INFO:tasks.workunit.client.0.smithi027.stdout:9/477: dread db/d1b/d23/d2c/f89 [0,4194304] 0 2021-11-03T17:14:15.364 INFO:tasks.workunit.client.0.smithi027.stdout:9/478: truncate db/d1b/d1c/d1d/d40/fa7 97174 0 2021-11-03T17:14:15.368 INFO:tasks.workunit.client.0.smithi027.stdout:8/435: dread d7/d10/d3f/f78 [0,4194304] 0 2021-11-03T17:14:15.368 INFO:tasks.workunit.client.0.smithi027.stdout:8/436: mknod d7/d10/d3f/d55/c8f 1 2021-11-03T17:14:15.371 INFO:tasks.workunit.client.0.smithi027.stdout:4/395: dwrite d6/d9/d28/f87 [0,4194304] 0 2021-11-03T17:14:15.374 INFO:tasks.workunit.client.0.smithi027.stdout:4/396: link d6/d9/d28/f87 d6/db/d21/d41/f88 0 2021-11-03T17:14:15.376 INFO:tasks.workunit.client.0.smithi027.stdout:4/397: symlink d6/d9/d16/d36/d70/d3e/d58/l89 0 2021-11-03T17:14:15.377 INFO:tasks.workunit.client.0.smithi027.stdout:4/398: creat d6/db/f8a x:0 0 0 2021-11-03T17:14:15.378 INFO:tasks.workunit.client.0.smithi027.stdout:4/399: creat d6/d9/d28/d40/f8b x:0 0 0 2021-11-03T17:14:15.390 INFO:tasks.workunit.client.0.smithi027.stdout:9/479: dwrite db/d1b/d2b/f3f [4194304,4194304] 0 2021-11-03T17:14:15.391 INFO:tasks.workunit.client.0.smithi027.stdout:9/480: write db/d1b/d1c/d1d/d40/d60/fa4 [746386,22615] 0 2021-11-03T17:14:15.392 INFO:tasks.workunit.client.0.smithi027.stdout:9/481: truncate db/d1b/d23/d2c/f89 2096317 0 2021-11-03T17:14:15.392 INFO:tasks.workunit.client.0.smithi027.stdout:8/437: dwrite d7/d10/d4f/f85 [0,4194304] 0 2021-11-03T17:14:15.392 INFO:tasks.workunit.client.0.smithi027.stdout:9/482: getdents db/d1b/d1c/d1d/d40/d60 0 2021-11-03T17:14:15.396 INFO:tasks.workunit.client.0.smithi027.stdout:4/400: dwrite d6/f48 [0,4194304] 0 2021-11-03T17:14:15.396 INFO:tasks.workunit.client.0.smithi027.stdout:8/438: symlink d7/d53/d67/d68/l90 0 2021-11-03T17:14:15.398 INFO:tasks.workunit.client.0.smithi027.stdout:9/483: unlink db/d1b/d2b/f4a 0 2021-11-03T17:14:15.406 INFO:tasks.workunit.client.0.smithi027.stdout:4/401: mkdir d6/d8c 0 2021-11-03T17:14:15.407 INFO:tasks.workunit.client.0.smithi027.stdout:4/402: write d6/d9/d16/d36/d70/d3e/d58/f61 [1175827,120578] 0 2021-11-03T17:14:15.407 INFO:tasks.workunit.client.0.smithi027.stdout:8/439: mkdir d7/d10/d3f/d91 0 2021-11-03T17:14:15.410 INFO:tasks.workunit.client.0.smithi027.stdout:9/484: link db/d1b/d1c/d1d/d40/l46 db/d1b/d1c/d1d/d40/d60/lb3 0 2021-11-03T17:14:15.417 INFO:tasks.workunit.client.0.smithi027.stdout:8/440: read d7/d10/d21/f39 [17670,34882] 0 2021-11-03T17:14:15.421 INFO:tasks.workunit.client.0.smithi027.stdout:8/441: unlink d7/d10/d3f/f76 0 2021-11-03T17:14:15.434 INFO:tasks.workunit.client.0.smithi027.stdout:4/403: dwrite d6/d9/d28/d40/f83 [0,4194304] 0 2021-11-03T17:14:15.435 INFO:tasks.workunit.client.0.smithi027.stdout:9/485: dwrite db/d1b/d1c/f65 [0,4194304] 0 2021-11-03T17:14:15.442 INFO:tasks.workunit.client.0.smithi027.stdout:8/442: dwrite d7/d10/d21/f63 [0,4194304] 0 2021-11-03T17:14:15.445 INFO:tasks.workunit.client.0.smithi027.stdout:8/443: creat d7/d10/d4f/f92 x:0 0 0 2021-11-03T17:14:15.445 INFO:tasks.workunit.client.0.smithi027.stdout:8/444: chown d7/d53/d67 31514 1 2021-11-03T17:14:15.446 INFO:tasks.workunit.client.0.smithi027.stdout:8/445: fdatasync d7/d10/d18/f33 0 2021-11-03T17:14:15.447 INFO:tasks.workunit.client.0.smithi027.stdout:8/446: link d7/d10/d18/d1a/f2e d7/d2d/d38/f93 0 2021-11-03T17:14:15.447 INFO:tasks.workunit.client.0.smithi027.stdout:8/447: stat d7/d10/f71 0 2021-11-03T17:14:15.448 INFO:tasks.workunit.client.0.smithi027.stdout:9/486: dwrite db/f75 [0,4194304] 0 2021-11-03T17:14:15.448 INFO:tasks.workunit.client.0.smithi027.stdout:9/487: mknod db/d1b/d1c/d1d/d40/d60/cb4 1 2021-11-03T17:14:15.449 INFO:tasks.workunit.client.0.smithi027.stdout:9/488: mknod db/d1b/d2b/d41/d97/cb5 1 2021-11-03T17:14:15.449 INFO:tasks.workunit.client.0.smithi027.stdout:8/448: mkdir d7/d53/d94 0 2021-11-03T17:14:15.453 INFO:tasks.workunit.client.0.smithi027.stdout:8/449: rename d7/d53/d67/d68 to d7/d10/d18/d1a/d95 0 2021-11-03T17:14:15.469 INFO:tasks.workunit.client.0.smithi027.stdout:8/450: dwrite d7/d10/d3f/f78 [0,4194304] 0 2021-11-03T17:14:15.483 INFO:tasks.workunit.client.0.smithi027.stdout:8/451: dwrite d7/d10/d3f/f47 [0,4194304] 0 2021-11-03T17:14:15.483 INFO:tasks.workunit.client.0.smithi027.stdout:8/452: mknod d7/d10/d3f/d91/c96 1 2021-11-03T17:14:15.483 INFO:tasks.workunit.client.0.smithi027.stdout:8/453: mknod d7/d10/d18/d84/c97 1 2021-11-03T17:14:15.486 INFO:tasks.workunit.client.0.smithi027.stdout:8/454: unlink d7/d10/d3f/f4e 0 2021-11-03T17:14:15.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:15 smithi027 conmon[32322]: cluster 2021-11-03T17:14:14 2021-11-03T17:14:15.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:15 smithi027 conmon[32322]: .605984+0000 mgr.smithi027.ufhhvb (mgr.14162) 308 : cluster [DBG] pgmap v254: 65 pgs: 65 active+clean; 1.5 GiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 63 MiB/s rd, 127 MiB/s wr, 194 op/s 2021-11-03T17:14:15.739 INFO:tasks.workunit.client.0.smithi027.stdout:1/435: sync 2021-11-03T17:14:15.740 INFO:tasks.workunit.client.0.smithi027.stdout:1/436: mknod d6/de/ca9 1 2021-11-03T17:14:15.743 INFO:tasks.workunit.client.0.smithi027.stdout:1/437: symlink d6/de/d44/d79/d25/d28/d5c/laa 0 2021-11-03T17:14:15.743 INFO:tasks.workunit.client.0.smithi027.stdout:1/438: fdatasync d6/d11/f89 0 2021-11-03T17:14:15.744 INFO:tasks.workunit.client.0.smithi027.stdout:1/439: chown d6/de/f92 15976 1 2021-11-03T17:14:15.745 INFO:tasks.workunit.client.0.smithi027.stdout:1/440: rmdir d6/de/d44/d79/d25/d28/d41 39 2021-11-03T17:14:15.770 INFO:tasks.workunit.client.0.smithi027.stdout:1/441: dwrite d6/de/d13/d15/f42 [0,4194304] 0 2021-11-03T17:14:15.771 INFO:tasks.workunit.client.0.smithi027.stdout:1/442: write d6/de/d44/d79/d25/d28/f3c [267261,3356] 0 2021-11-03T17:14:15.772 INFO:tasks.workunit.client.0.smithi027.stdout:1/443: mkdir d6/de/d44/d79/dab 0 2021-11-03T17:14:15.787 INFO:tasks.workunit.client.0.smithi027.stdout:1/444: dwrite d6/de/d44/d79/d31/fa0 [0,4194304] 0 2021-11-03T17:14:15.788 INFO:tasks.workunit.client.0.smithi027.stdout:1/445: dread - d6/d11/f89 zero size 2021-11-03T17:14:15.788 INFO:tasks.workunit.client.0.smithi027.stdout:1/446: dread - d6/de/d44/d79/d25/d28/d5c/f78 zero size 2021-11-03T17:14:15.788 INFO:tasks.workunit.client.0.smithi027.stdout:1/447: chown d6/de/d44/d79/d25/d5e 4205545 1 2021-11-03T17:14:15.795 INFO:tasks.workunit.client.0.smithi027.stdout:1/448: rename d6/de/d44 to d6/d11/dac 0 2021-11-03T17:14:15.797 INFO:tasks.workunit.client.0.smithi027.stdout:1/449: symlink d6/d11/dac/d79/d25/d28/d5c/d7a/lad 0 2021-11-03T17:14:15.798 INFO:tasks.workunit.client.0.smithi027.stdout:1/450: creat d6/d11/dac/d79/d25/d28/d32/fae x:0 0 0 2021-11-03T17:14:15.800 INFO:tasks.workunit.client.0.smithi027.stdout:1/451: creat d6/de/faf x:0 0 0 2021-11-03T17:14:15.800 INFO:tasks.workunit.client.0.smithi027.stdout:1/452: write d6/d11/dac/d79/d25/d28/d5c/d81/d7d/f7e [556146,69591] 0 2021-11-03T17:14:15.801 INFO:tasks.workunit.client.0.smithi027.stdout:1/453: mkdir d6/d11/dac/d79/db0 0 2021-11-03T17:14:15.824 INFO:tasks.workunit.client.0.smithi027.stdout:1/454: symlink d6/de/d13/d15/d4e/lb1 0 2021-11-03T17:14:15.824 INFO:tasks.workunit.client.0.smithi027.stdout:1/455: write d6/d11/dac/d79/d25/d28/d5c/d81/d7d/f7e [1312175,13482] 0 2021-11-03T17:14:15.824 INFO:tasks.workunit.client.0.smithi027.stdout:1/456: truncate d6/d11/dac/d79/d25/d28/d32/fae 880844 0 2021-11-03T17:14:15.825 INFO:tasks.workunit.client.0.smithi027.stdout:1/457: readlink d6/l3b 0 2021-11-03T17:14:15.836 INFO:tasks.workunit.client.0.smithi027.stdout:1/458: dread d6/d11/dac/d79/d25/d28/d32/f84 [0,4194304] 0 2021-11-03T17:14:15.854 INFO:tasks.workunit.client.0.smithi027.stdout:1/459: dwrite d6/fa [0,4194304] 0 2021-11-03T17:14:15.865 INFO:tasks.workunit.client.0.smithi027.stdout:1/460: dread d6/de/d13/d15/d43/f48 [0,4194304] 0 2021-11-03T17:14:15.865 INFO:tasks.workunit.client.0.smithi027.stdout:1/461: write d6/d11/dac/d79/f99 [611657,114581] 0 2021-11-03T17:14:15.866 INFO:tasks.workunit.client.0.smithi027.stdout:1/462: symlink d6/de/d13/lb2 0 2021-11-03T17:14:15.868 INFO:tasks.workunit.client.0.smithi027.stdout:1/463: rename d6/de/f1c to d6/d11/dac/d79/d25/d28/d5c/d68/fb3 0 2021-11-03T17:14:15.870 INFO:tasks.workunit.client.0.smithi027.stdout:1/464: mkdir d6/d11/dac/d79/dab/db4 0 2021-11-03T17:14:15.872 INFO:tasks.workunit.client.0.smithi027.stdout:1/465: link d6/de/d13/d15/d43/l5a d6/de/lb5 0 2021-11-03T17:14:15.874 INFO:tasks.workunit.client.0.smithi027.stdout:1/466: rename d6/d11/dac/d79/d25/d28/d5c/laa to d6/lb6 0 2021-11-03T17:14:15.875 INFO:tasks.workunit.client.1.smithi115.stdout:3/419: sync 2021-11-03T17:14:15.876 INFO:tasks.workunit.client.1.smithi115.stdout:5/432: sync 2021-11-03T17:14:15.876 INFO:tasks.workunit.client.1.smithi115.stdout:7/277: sync 2021-11-03T17:14:15.876 INFO:tasks.workunit.client.1.smithi115.stdout:1/358: sync 2021-11-03T17:14:15.877 INFO:tasks.workunit.client.1.smithi115.stdout:0/399: sync 2021-11-03T17:14:15.877 INFO:tasks.workunit.client.1.smithi115.stdout:3/420: chown d13/f26 5057161 1 2021-11-03T17:14:15.877 INFO:tasks.workunit.client.1.smithi115.stdout:7/278: chown d3/da/d26/d33/f59 1 1 2021-11-03T17:14:15.877 INFO:tasks.workunit.client.1.smithi115.stdout:5/433: mknod d1a/d3d/d89/c93 1 2021-11-03T17:14:15.878 INFO:tasks.workunit.client.1.smithi115.stdout:1/359: chown d7/dc/dd/d24/d43/f3c 59119 1 2021-11-03T17:14:15.878 INFO:tasks.workunit.client.1.smithi115.stdout:5/434: stat d1a/d28/d4c/d75 0 2021-11-03T17:14:15.878 INFO:tasks.workunit.client.1.smithi115.stdout:5/435: chown d1a/d28/l43 67 1 2021-11-03T17:14:15.878 INFO:tasks.workunit.client.1.smithi115.stdout:9/372: sync 2021-11-03T17:14:15.879 INFO:tasks.workunit.client.1.smithi115.stdout:2/323: sync 2021-11-03T17:14:15.879 INFO:tasks.workunit.client.1.smithi115.stdout:2/324: fdatasync d4/f35 0 2021-11-03T17:14:15.883 INFO:tasks.workunit.client.1.smithi115.stdout:7/279: rmdir d3 39 2021-11-03T17:14:15.883 INFO:tasks.workunit.client.1.smithi115.stdout:7/280: getdents d3/da 0 2021-11-03T17:14:15.883 INFO:tasks.workunit.client.1.smithi115.stdout:5/436: creat d1a/d28/d7b/d48/d7f/f94 x:0 0 0 2021-11-03T17:14:15.884 INFO:tasks.workunit.client.1.smithi115.stdout:0/400: mkdir d9/d15/d78/d80 0 2021-11-03T17:14:15.885 INFO:tasks.workunit.client.1.smithi115.stdout:9/373: creat d1/d1a/d48/d27/d2f/d7b/f85 x:0 0 0 2021-11-03T17:14:15.887 INFO:tasks.workunit.client.1.smithi115.stdout:2/325: rename d4/d6/d56/d6c/d34/d42/d57 to d4/d6/d13/d6d 0 2021-11-03T17:14:15.888 INFO:tasks.workunit.client.1.smithi115.stdout:7/281: unlink d3/da/f2b 0 2021-11-03T17:14:15.891 INFO:tasks.workunit.client.1.smithi115.stdout:3/421: symlink d13/d2a/l8d 0 2021-11-03T17:14:15.892 INFO:tasks.workunit.client.1.smithi115.stdout:0/401: mkdir d9/d15/d1f/d81 0 2021-11-03T17:14:15.892 INFO:tasks.workunit.client.1.smithi115.stdout:5/437: dread d1a/d28/d34/f2a [0,4194304] 0 2021-11-03T17:14:15.893 INFO:tasks.workunit.client.1.smithi115.stdout:2/326: getdents d4/d6 0 2021-11-03T17:14:15.893 INFO:tasks.workunit.client.1.smithi115.stdout:2/327: chown d4/d6/d13/d6d/f60 7449 1 2021-11-03T17:14:15.893 INFO:tasks.workunit.client.1.smithi115.stdout:2/328: mknod d4/d6/d56/d6c/d34/c6e 1 2021-11-03T17:14:15.894 INFO:tasks.workunit.client.1.smithi115.stdout:0/402: mkdir d9/d15/d78/d80/d82 0 2021-11-03T17:14:15.895 INFO:tasks.workunit.client.1.smithi115.stdout:5/438: getdents d1a/d3d/d58 0 2021-11-03T17:14:15.896 INFO:tasks.workunit.client.1.smithi115.stdout:2/329: creat d4/d6/d56/d6c/f6f x:0 0 0 2021-11-03T17:14:15.897 INFO:tasks.workunit.client.1.smithi115.stdout:0/403: dread d9/d15/f1e [0,4194304] 0 2021-11-03T17:14:15.898 INFO:tasks.workunit.client.1.smithi115.stdout:2/330: creat d4/d6/d56/d6c/d34/f70 x:0 0 0 2021-11-03T17:14:15.898 INFO:tasks.workunit.client.1.smithi115.stdout:2/331: chown d4/d6/d56/d6c/d34/d43/d28/l31 419276 1 2021-11-03T17:14:15.898 INFO:tasks.workunit.client.1.smithi115.stdout:2/332: mknod d4/d6/d56/d6c/d34/d42/c71 1 2021-11-03T17:14:15.899 INFO:tasks.workunit.client.1.smithi115.stdout:5/439: dread d1a/d28/d34/f2a [0,4194304] 0 2021-11-03T17:14:15.900 INFO:tasks.workunit.client.1.smithi115.stdout:5/440: dread - d1a/d28/d4c/d75/d68/f6c zero size 2021-11-03T17:14:15.900 INFO:tasks.workunit.client.1.smithi115.stdout:5/441: fdatasync d1a/d28/d34/f2a 0 2021-11-03T17:14:15.900 INFO:tasks.workunit.client.1.smithi115.stdout:5/442: readlink d1a/d28/d7b/l90 0 2021-11-03T17:14:15.900 INFO:tasks.workunit.client.1.smithi115.stdout:5/443: rename d1a/d28/d4c to d1a/d28/d4c/d75/d95 22 2021-11-03T17:14:15.901 INFO:tasks.workunit.client.1.smithi115.stdout:5/444: write d1a/d28/d4c/d75/d83/f88 [440096,15977] 0 2021-11-03T17:14:15.901 INFO:tasks.workunit.client.1.smithi115.stdout:7/282: dread d3/f14 [0,4194304] 0 2021-11-03T17:14:15.902 INFO:tasks.workunit.client.1.smithi115.stdout:5/445: link d1a/d3d/d58/l80 d1a/d3d/l96 0 2021-11-03T17:14:15.907 INFO:tasks.workunit.client.1.smithi115.stdout:3/422: dread d13/d35/f36 [0,4194304] 0 2021-11-03T17:14:15.911 INFO:tasks.workunit.client.1.smithi115.stdout:0/404: write d9/d15/d1a/d1d/f4b [366583,128006] 0 2021-11-03T17:14:15.911 INFO:tasks.workunit.client.0.smithi027.stdout:1/467: dwrite d6/de/d13/f6b [0,4194304] 0 2021-11-03T17:14:15.914 INFO:tasks.workunit.client.1.smithi115.stdout:7/283: dread d3/f25 [0,4194304] 0 2021-11-03T17:14:15.914 INFO:tasks.workunit.client.1.smithi115.stdout:0/405: rmdir d9/d15/d1a 39 2021-11-03T17:14:15.914 INFO:tasks.workunit.client.1.smithi115.stdout:0/406: readlink d9/le 0 2021-11-03T17:14:15.914 INFO:tasks.workunit.client.1.smithi115.stdout:0/407: readlink d9/d15/d2c/d46/l65 0 2021-11-03T17:14:15.915 INFO:tasks.workunit.client.0.smithi027.stdout:1/468: link d6/d11/dac/d79/d31/fa0 d6/d11/dac/d79/d31/fb7 0 2021-11-03T17:14:15.916 INFO:tasks.workunit.client.1.smithi115.stdout:7/284: truncate d3/f47 3521063 0 2021-11-03T17:14:15.916 INFO:tasks.workunit.client.1.smithi115.stdout:7/285: chown d3/d4c/f4d 118 1 2021-11-03T17:14:15.917 INFO:tasks.workunit.client.1.smithi115.stdout:0/408: mkdir d9/db/d83 0 2021-11-03T17:14:15.917 INFO:tasks.workunit.client.0.smithi027.stdout:1/469: link d6/d11/dac/d79/d25/d28/l39 d6/d11/dac/d79/dab/lb8 0 2021-11-03T17:14:15.918 INFO:tasks.workunit.client.1.smithi115.stdout:7/286: getdents d3/da/d26/d52 0 2021-11-03T17:14:15.919 INFO:tasks.workunit.client.1.smithi115.stdout:0/409: link d9/d15/d2c/f7f d9/d15/d1f/f84 0 2021-11-03T17:14:15.920 INFO:tasks.workunit.client.0.smithi027.stdout:1/470: link d6/d11/dac/d79/d37/d49/la1 d6/d11/dac/d79/d37/d49/lb9 0 2021-11-03T17:14:15.921 INFO:tasks.workunit.client.1.smithi115.stdout:7/287: link d3/d4c/f4d d3/d4c/f5b 0 2021-11-03T17:14:15.921 INFO:tasks.workunit.client.1.smithi115.stdout:7/288: mknod d3/da/d26/d35/d53/c5c 1 2021-11-03T17:14:15.921 INFO:tasks.workunit.client.1.smithi115.stdout:7/289: mknod d3/da/c5d 1 2021-11-03T17:14:15.921 INFO:tasks.workunit.client.1.smithi115.stdout:7/290: readlink d3/da/l1d 0 2021-11-03T17:14:15.922 INFO:tasks.workunit.client.1.smithi115.stdout:7/291: write d3/d4c/f4d [188786,76130] 0 2021-11-03T17:14:15.925 INFO:tasks.workunit.client.1.smithi115.stdout:1/360: dwrite d7/dc/dd/d42/f47 [0,4194304] 0 2021-11-03T17:14:15.925 INFO:tasks.workunit.client.1.smithi115.stdout:1/361: chown d7/dc/d1a/l22 4819421 1 2021-11-03T17:14:15.925 INFO:tasks.workunit.client.1.smithi115.stdout:1/362: truncate d7/dc/dd/d24/d4d/d60/f6d 80480 0 2021-11-03T17:14:15.925 INFO:tasks.workunit.client.1.smithi115.stdout:2/333: dwrite d4/d6/f2a [0,4194304] 0 2021-11-03T17:14:15.926 INFO:tasks.workunit.client.1.smithi115.stdout:1/363: truncate d7/dc/f2f 158344 0 2021-11-03T17:14:15.928 INFO:tasks.workunit.client.1.smithi115.stdout:1/364: read d7/f8 [3138794,1715] 0 2021-11-03T17:14:15.932 INFO:tasks.workunit.client.1.smithi115.stdout:0/410: dread d9/d15/d1a/f3d [0,4194304] 0 2021-11-03T17:14:15.933 INFO:tasks.workunit.client.0.smithi027.stdout:1/471: dread d6/de/f1a [0,4194304] 0 2021-11-03T17:14:15.933 INFO:tasks.workunit.client.0.smithi027.stdout:1/472: readlink d6/de/d13/d15/l90 0 2021-11-03T17:14:15.937 INFO:tasks.workunit.client.1.smithi115.stdout:7/292: dread d3/f25 [0,4194304] 0 2021-11-03T17:14:15.937 INFO:tasks.workunit.client.1.smithi115.stdout:7/293: dread - d3/da/d26/f2f zero size 2021-11-03T17:14:15.937 INFO:tasks.workunit.client.0.smithi027.stdout:1/473: link d6/d11/dac/d79/d31/d58/la8 d6/de/d13/d15/d43/lba 0 2021-11-03T17:14:15.941 INFO:tasks.workunit.client.1.smithi115.stdout:3/423: dwrite d13/d51/d40/f5a [0,4194304] 0 2021-11-03T17:14:15.941 INFO:tasks.workunit.client.1.smithi115.stdout:3/424: chown d13/d3b/f89 3713320 1 2021-11-03T17:14:15.942 INFO:tasks.workunit.client.1.smithi115.stdout:8/294: sync 2021-11-03T17:14:15.942 INFO:tasks.workunit.client.1.smithi115.stdout:8/295: write d4/d6/f3c [351762,114724] 0 2021-11-03T17:14:15.942 INFO:tasks.workunit.client.1.smithi115.stdout:6/294: sync 2021-11-03T17:14:15.944 INFO:tasks.workunit.client.1.smithi115.stdout:0/411: write d9/d15/d1a/f50 [615002,32561] 0 2021-11-03T17:14:15.947 INFO:tasks.workunit.client.1.smithi115.stdout:5/446: dwrite d1a/d3d/f3b [0,4194304] 0 2021-11-03T17:14:15.949 INFO:tasks.workunit.client.1.smithi115.stdout:3/425: creat d13/d50/f8e x:0 0 0 2021-11-03T17:14:15.949 INFO:tasks.workunit.client.1.smithi115.stdout:5/447: write d1a/d28/d4a/d65/f6b [165769,92125] 0 2021-11-03T17:14:15.949 INFO:tasks.workunit.client.1.smithi115.stdout:5/448: write d1a/d28/d7b/f74 [727024,59502] 0 2021-11-03T17:14:15.950 INFO:tasks.workunit.client.1.smithi115.stdout:5/449: chown d1a 1 1 2021-11-03T17:14:15.950 INFO:tasks.workunit.client.1.smithi115.stdout:5/450: readlink d1a/d28/d4c/d63/l8a 0 2021-11-03T17:14:15.951 INFO:tasks.workunit.client.1.smithi115.stdout:6/295: write d0/d1/d2/f1f [3016876,91861] 0 2021-11-03T17:14:15.951 INFO:tasks.workunit.client.1.smithi115.stdout:8/296: symlink d4/d6/d18/d30/d33/d4c/l69 0 2021-11-03T17:14:15.951 INFO:tasks.workunit.client.1.smithi115.stdout:8/297: readlink d4/d6/d3f/d43/l4a 0 2021-11-03T17:14:15.953 INFO:tasks.workunit.client.1.smithi115.stdout:0/412: mkdir d9/d15/d2c/d85 0 2021-11-03T17:14:15.953 INFO:tasks.workunit.client.1.smithi115.stdout:1/365: dwrite d7/dc/dd/d24/d4b/d55/f66 [0,4194304] 0 2021-11-03T17:14:15.957 INFO:tasks.workunit.client.1.smithi115.stdout:7/294: link d3/f47 d3/da/d26/d33/f5e 0 2021-11-03T17:14:15.957 INFO:tasks.workunit.client.1.smithi115.stdout:3/426: truncate d13/f25 975864 0 2021-11-03T17:14:15.957 INFO:tasks.workunit.client.1.smithi115.stdout:2/334: dwrite d4/d6/f45 [0,4194304] 0 2021-11-03T17:14:15.958 INFO:tasks.workunit.client.1.smithi115.stdout:2/335: write d4/d6/d13/f6b [817822,85846] 0 2021-11-03T17:14:15.961 INFO:tasks.workunit.client.1.smithi115.stdout:5/451: link d1a/d3d/l96 d1a/d28/d4c/l97 0 2021-11-03T17:14:15.961 INFO:tasks.workunit.client.1.smithi115.stdout:7/295: write d3/da/d18/f40 [3177628,70471] 0 2021-11-03T17:14:15.961 INFO:tasks.workunit.client.1.smithi115.stdout:7/296: chown d3/da/d26/d52/l54 3 1 2021-11-03T17:14:15.962 INFO:tasks.workunit.client.1.smithi115.stdout:6/296: creat d0/d1/d10/f5f x:0 0 0 2021-11-03T17:14:15.963 INFO:tasks.workunit.client.1.smithi115.stdout:8/298: rename d4/d6/d18/d30/l56 to d4/d6/d18/d30/l6a 0 2021-11-03T17:14:15.963 INFO:tasks.workunit.client.1.smithi115.stdout:8/299: fdatasync d4/d6/d18/d30/d33/d4c/f54 0 2021-11-03T17:14:15.971 INFO:tasks.workunit.client.1.smithi115.stdout:0/413: rename d9/d15/d2c/d46/l65 to d9/d15/d2c/d46/d63/d70/l86 0 2021-11-03T17:14:15.978 INFO:tasks.workunit.client.1.smithi115.stdout:3/427: stat d13/l17 0 2021-11-03T17:14:15.978 INFO:tasks.workunit.client.1.smithi115.stdout:3/428: dread - d13/d2a/d53/f8c zero size 2021-11-03T17:14:15.979 INFO:tasks.workunit.client.1.smithi115.stdout:3/429: chown d13/d51/d40/l6b 1755 1 2021-11-03T17:14:15.979 INFO:tasks.workunit.client.1.smithi115.stdout:3/430: fsync d13/d51/f2d 0 2021-11-03T17:14:15.980 INFO:tasks.workunit.client.1.smithi115.stdout:2/336: mkdir d4/d6/d56/d6c/d34/d42/d72 0 2021-11-03T17:14:15.980 INFO:tasks.workunit.client.1.smithi115.stdout:2/337: write d4/d6/d13/f21 [754237,105603] 0 2021-11-03T17:14:15.982 INFO:tasks.workunit.client.1.smithi115.stdout:6/297: dread d0/d1/d2/d3f/f49 [0,4194304] 0 2021-11-03T17:14:15.984 INFO:tasks.workunit.client.1.smithi115.stdout:8/300: creat d4/d6/f6b x:0 0 0 2021-11-03T17:14:15.984 INFO:tasks.workunit.client.1.smithi115.stdout:1/366: rename d7/d38/d59/f68 to d7/dc/dd/d24/d4b/f72 0 2021-11-03T17:14:15.990 INFO:tasks.workunit.client.1.smithi115.stdout:5/452: dwrite d1a/d28/d4c/d75/d83/f88 [0,4194304] 0 2021-11-03T17:14:15.990 INFO:tasks.workunit.client.1.smithi115.stdout:5/453: dread - d1a/d3d/d89/f8f zero size 2021-11-03T17:14:15.991 INFO:tasks.workunit.client.1.smithi115.stdout:5/454: creat d1a/d28/d4c/d75/d83/f98 x:0 0 0 2021-11-03T17:14:15.991 INFO:tasks.workunit.client.1.smithi115.stdout:6/298: write d0/f41 [5020718,3417] 0 2021-11-03T17:14:15.992 INFO:tasks.workunit.client.1.smithi115.stdout:6/299: readlink d0/d1/l3a 0 2021-11-03T17:14:15.992 INFO:tasks.workunit.client.1.smithi115.stdout:5/455: symlink d1a/d28/d4c/d63/l99 0 2021-11-03T17:14:15.994 INFO:tasks.workunit.client.1.smithi115.stdout:6/300: creat d0/d1/d2e/f60 x:0 0 0 2021-11-03T17:14:15.994 INFO:tasks.workunit.client.1.smithi115.stdout:6/301: chown d0/d1/d10/d13/d1c/d1e/l5e 16 1 2021-11-03T17:14:15.995 INFO:tasks.workunit.client.1.smithi115.stdout:5/456: symlink d1a/d51/l9a 0 2021-11-03T17:14:15.996 INFO:tasks.workunit.client.1.smithi115.stdout:5/457: getdents d1a/d3d/d58 0 2021-11-03T17:14:15.998 INFO:tasks.workunit.client.1.smithi115.stdout:6/302: mkdir d0/d1/d2/d5d/d61 0 2021-11-03T17:14:16.003 INFO:tasks.workunit.client.1.smithi115.stdout:7/297: rmdir d3/d4c 39 2021-11-03T17:14:16.004 INFO:tasks.workunit.client.1.smithi115.stdout:7/298: fsync d3/da/f1b 0 2021-11-03T17:14:16.004 INFO:tasks.workunit.client.1.smithi115.stdout:6/303: getdents d0/d1/d2 0 2021-11-03T17:14:16.009 INFO:tasks.workunit.client.1.smithi115.stdout:7/299: write d3/f25 [2842132,10300] 0 2021-11-03T17:14:16.012 INFO:tasks.workunit.client.1.smithi115.stdout:6/304: write d0/d1/d2/f1f [1696307,60168] 0 2021-11-03T17:14:16.012 INFO:tasks.workunit.client.1.smithi115.stdout:6/305: fsync d0/d1/d2/ff 0 2021-11-03T17:14:16.013 INFO:tasks.workunit.client.1.smithi115.stdout:0/414: dwrite f8 [0,4194304] 0 2021-11-03T17:14:16.013 INFO:tasks.workunit.client.1.smithi115.stdout:0/415: readlink d9/d37/l6a 0 2021-11-03T17:14:16.017 INFO:tasks.workunit.client.1.smithi115.stdout:6/306: symlink d0/d1/d10/d13/l62 0 2021-11-03T17:14:16.017 INFO:tasks.workunit.client.1.smithi115.stdout:6/307: fdatasync d0/f40 0 2021-11-03T17:14:16.017 INFO:tasks.workunit.client.1.smithi115.stdout:6/308: mknod d0/d1/d10/d13/d1c/c63 1 2021-11-03T17:14:16.021 INFO:tasks.workunit.client.1.smithi115.stdout:7/300: dread d3/da/d26/d35/f3f [0,4194304] 0 2021-11-03T17:14:16.022 INFO:tasks.workunit.client.1.smithi115.stdout:6/309: write d0/d1/d2/fd [1784264,17033] 0 2021-11-03T17:14:16.022 INFO:tasks.workunit.client.1.smithi115.stdout:0/416: creat d9/d15/d78/d80/f87 x:0 0 0 2021-11-03T17:14:16.024 INFO:tasks.workunit.client.1.smithi115.stdout:6/310: rmdir d0/d1/d2/d3f 39 2021-11-03T17:14:16.025 INFO:tasks.workunit.client.1.smithi115.stdout:0/417: symlink d9/d15/d2c/d85/l88 0 2021-11-03T17:14:16.027 INFO:tasks.workunit.client.1.smithi115.stdout:7/301: read d3/da/f42 [5328144,36013] 0 2021-11-03T17:14:16.028 INFO:tasks.workunit.client.1.smithi115.stdout:6/311: rename d0/d1/d10/f28 to d0/d1/d10/d13/d59/f64 0 2021-11-03T17:14:16.028 INFO:tasks.workunit.client.1.smithi115.stdout:2/338: dwrite d4/f68 [0,4194304] 0 2021-11-03T17:14:16.029 INFO:tasks.workunit.client.1.smithi115.stdout:8/301: dwrite d4/d6/d18/d30/d33/f59 [0,4194304] 0 2021-11-03T17:14:16.029 INFO:tasks.workunit.client.1.smithi115.stdout:1/367: dwrite d7/dc/dd/d24/d26/f4f [0,4194304] 0 2021-11-03T17:14:16.030 INFO:tasks.workunit.client.1.smithi115.stdout:5/458: dwrite d1a/d28/d34/f33 [0,4194304] 0 2021-11-03T17:14:16.030 INFO:tasks.workunit.client.1.smithi115.stdout:3/431: dwrite d13/d51/f55 [0,4194304] 0 2021-11-03T17:14:16.031 INFO:tasks.workunit.client.1.smithi115.stdout:7/302: rmdir d3/da/d26 39 2021-11-03T17:14:16.031 INFO:tasks.workunit.client.1.smithi115.stdout:7/303: fsync d3/da/d26/d35/f3e 0 2021-11-03T17:14:16.031 INFO:tasks.workunit.client.1.smithi115.stdout:1/368: read d7/dc/dd/d24/d4d/d60/f62 [494597,31837] 0 2021-11-03T17:14:16.037 INFO:tasks.workunit.client.1.smithi115.stdout:1/369: dread d7/dc/d1a/f6b [0,4194304] 0 2021-11-03T17:14:16.037 INFO:tasks.workunit.client.1.smithi115.stdout:0/418: rename d9/d15/d1f to d9/db/d89 0 2021-11-03T17:14:16.037 INFO:tasks.workunit.client.1.smithi115.stdout:0/419: chown d9/d15/d1a/d1d/f4d 2784 1 2021-11-03T17:14:16.038 INFO:tasks.workunit.client.1.smithi115.stdout:0/420: fdatasync d9/db/f4f 0 2021-11-03T17:14:16.045 INFO:tasks.workunit.client.1.smithi115.stdout:6/312: dread d0/f40 [0,4194304] 0 2021-11-03T17:14:16.045 INFO:tasks.workunit.client.1.smithi115.stdout:5/459: truncate d1a/f1d 2054181 0 2021-11-03T17:14:16.045 INFO:tasks.workunit.client.1.smithi115.stdout:5/460: stat d1a/d7d 0 2021-11-03T17:14:16.046 INFO:tasks.workunit.client.1.smithi115.stdout:0/421: unlink d9/d15/d1a/f50 0 2021-11-03T17:14:16.046 INFO:tasks.workunit.client.1.smithi115.stdout:0/422: chown d9/d15/d1a/d1d 652593088 1 2021-11-03T17:14:16.046 INFO:tasks.workunit.client.1.smithi115.stdout:0/423: fdatasync d9/d15/d2c/d46/d63/f7b 0 2021-11-03T17:14:16.046 INFO:tasks.workunit.client.1.smithi115.stdout:7/304: creat d3/da/d26/d35/f5f x:0 0 0 2021-11-03T17:14:16.047 INFO:tasks.workunit.client.1.smithi115.stdout:3/432: dread fc [0,4194304] 0 2021-11-03T17:14:16.047 INFO:tasks.workunit.client.1.smithi115.stdout:2/339: rename d4/l4b to d4/d6/d56/l73 0 2021-11-03T17:14:16.048 INFO:tasks.workunit.client.1.smithi115.stdout:3/433: dread d13/f26 [0,4194304] 0 2021-11-03T17:14:16.049 INFO:tasks.workunit.client.1.smithi115.stdout:8/302: rename d4/fd to d4/f6c 0 2021-11-03T17:14:16.049 INFO:tasks.workunit.client.1.smithi115.stdout:8/303: mknod d4/d6/d18/c6d 1 2021-11-03T17:14:16.049 INFO:tasks.workunit.client.1.smithi115.stdout:6/313: rmdir d0/d1/d10/d13/d1c 39 2021-11-03T17:14:16.051 INFO:tasks.workunit.client.1.smithi115.stdout:0/424: symlink d9/d15/d2c/d6e/l8a 0 2021-11-03T17:14:16.051 INFO:tasks.workunit.client.1.smithi115.stdout:0/425: getdents d9/db/d83 0 2021-11-03T17:14:16.052 INFO:tasks.workunit.client.1.smithi115.stdout:2/340: link d4/d44/l47 d4/d44/l74 0 2021-11-03T17:14:16.056 INFO:tasks.workunit.client.1.smithi115.stdout:8/304: readlink d4/d6/l3e 0 2021-11-03T17:14:16.056 INFO:tasks.workunit.client.1.smithi115.stdout:0/426: dread d9/d15/f1e [0,4194304] 0 2021-11-03T17:14:16.058 INFO:tasks.workunit.client.1.smithi115.stdout:8/305: dread d4/d6/f3c [0,4194304] 0 2021-11-03T17:14:16.058 INFO:tasks.workunit.client.1.smithi115.stdout:8/306: truncate d4/d6/f3c 1424734 0 2021-11-03T17:14:16.059 INFO:tasks.workunit.client.1.smithi115.stdout:6/314: mknod d0/c65 1 2021-11-03T17:14:16.060 INFO:tasks.workunit.client.1.smithi115.stdout:6/315: mknod d0/d1/d2e/c66 1 2021-11-03T17:14:16.060 INFO:tasks.workunit.client.1.smithi115.stdout:3/434: rename d13/d2a/d60/l78 to d13/d50/l8f 0 2021-11-03T17:14:16.067 INFO:tasks.workunit.client.1.smithi115.stdout:8/307: write d4/d6/f1d [258206,16501] 0 2021-11-03T17:14:16.068 INFO:tasks.workunit.client.1.smithi115.stdout:2/341: mkdir d4/d6/d13/d75 0 2021-11-03T17:14:16.068 INFO:tasks.workunit.client.1.smithi115.stdout:0/427: symlink d9/l8b 0 2021-11-03T17:14:16.069 INFO:tasks.workunit.client.1.smithi115.stdout:8/308: mkdir d4/d6/d18/d44/d6e 0 2021-11-03T17:14:16.070 INFO:tasks.workunit.client.1.smithi115.stdout:8/309: mknod d4/d6/d18/d30/d33/d4c/c6f 1 2021-11-03T17:14:16.070 INFO:tasks.workunit.client.1.smithi115.stdout:2/342: rmdir d4/d6/d56/d6c/d34/d42 39 2021-11-03T17:14:16.071 INFO:tasks.workunit.client.1.smithi115.stdout:2/343: fdatasync d4/d6/d13/f36 0 2021-11-03T17:14:16.072 INFO:tasks.workunit.client.1.smithi115.stdout:0/428: rename d9/db/df to d9/d15/d2c/d46/d8c 0 2021-11-03T17:14:16.074 INFO:tasks.workunit.client.1.smithi115.stdout:3/435: symlink d13/l90 0 2021-11-03T17:14:16.075 INFO:tasks.workunit.client.1.smithi115.stdout:3/436: dread - d13/d50/f42 zero size 2021-11-03T17:14:16.075 INFO:tasks.workunit.client.1.smithi115.stdout:3/437: stat d13/d67 0 2021-11-03T17:14:16.075 INFO:tasks.workunit.client.1.smithi115.stdout:8/310: symlink d4/d6/d18/d30/d33/l70 0 2021-11-03T17:14:16.076 INFO:tasks.workunit.client.1.smithi115.stdout:5/461: dwrite d1a/d3d/f64 [0,4194304] 0 2021-11-03T17:14:16.079 INFO:tasks.workunit.client.1.smithi115.stdout:0/429: unlink d9/db/d89/f60 0 2021-11-03T17:14:16.081 INFO:tasks.workunit.client.1.smithi115.stdout:3/438: write d13/f21 [1482227,130789] 0 2021-11-03T17:14:16.081 INFO:tasks.workunit.client.1.smithi115.stdout:2/344: mkdir d4/d6/d56/d6c/d34/d43/d28/d76 0 2021-11-03T17:14:16.083 INFO:tasks.workunit.client.1.smithi115.stdout:5/462: dread d1a/d28/d34/f2a [0,4194304] 0 2021-11-03T17:14:16.085 INFO:tasks.workunit.client.1.smithi115.stdout:3/439: dread d13/d51/d2e/f2f [0,4194304] 0 2021-11-03T17:14:16.086 INFO:tasks.workunit.client.1.smithi115.stdout:3/440: write d13/d3b/f89 [945461,75588] 0 2021-11-03T17:14:16.087 INFO:tasks.workunit.client.1.smithi115.stdout:2/345: truncate d4/d6/f12 3005049 0 2021-11-03T17:14:16.090 INFO:tasks.workunit.client.1.smithi115.stdout:5/463: symlink d1a/d28/d7b/d4f/l9b 0 2021-11-03T17:14:16.090 INFO:tasks.workunit.client.1.smithi115.stdout:3/441: creat d13/d2a/d53/d72/f91 x:0 0 0 2021-11-03T17:14:16.092 INFO:tasks.workunit.client.1.smithi115.stdout:6/316: dwrite d0/d1/d10/d13/d1c/f21 [0,4194304] 0 2021-11-03T17:14:16.092 INFO:tasks.workunit.client.1.smithi115.stdout:6/317: write d0/d1/d2/d3f/f51 [919757,52946] 0 2021-11-03T17:14:16.092 INFO:tasks.workunit.client.1.smithi115.stdout:2/346: mkdir d4/d6/d56/d6c/d34/d43/d63/d77 0 2021-11-03T17:14:16.093 INFO:tasks.workunit.client.1.smithi115.stdout:2/347: chown d4/f3a 19054546 1 2021-11-03T17:14:16.094 INFO:tasks.workunit.client.1.smithi115.stdout:5/464: link d1a/d28/d34/f32 d1a/d28/d4c/d63/f9c 0 2021-11-03T17:14:16.094 INFO:tasks.workunit.client.1.smithi115.stdout:5/465: mknod d1a/d28/d7b/d48/d7f/c9d 1 2021-11-03T17:14:16.094 INFO:tasks.workunit.client.1.smithi115.stdout:5/466: stat d1a/d28/d34/f32 0 2021-11-03T17:14:16.095 INFO:tasks.workunit.client.1.smithi115.stdout:5/467: write d1a/d28/f91 [512076,11287] 0 2021-11-03T17:14:16.095 INFO:tasks.workunit.client.1.smithi115.stdout:6/318: write d0/d1/d2/fd [4062840,32903] 0 2021-11-03T17:14:16.095 INFO:tasks.workunit.client.1.smithi115.stdout:7/305: truncate d3/f25 1531679 0 2021-11-03T17:14:16.096 INFO:tasks.workunit.client.1.smithi115.stdout:2/348: read d4/d6/d56/d6c/f14 [11896425,18737] 0 2021-11-03T17:14:16.100 INFO:tasks.workunit.client.1.smithi115.stdout:3/442: mkdir d13/d2a/d53/d70/d92 0 2021-11-03T17:14:16.104 INFO:tasks.workunit.client.1.smithi115.stdout:3/443: dread d13/f27 [0,4194304] 0 2021-11-03T17:14:16.105 INFO:tasks.workunit.client.1.smithi115.stdout:6/319: creat d0/d1/d10/d13/d59/f67 x:0 0 0 2021-11-03T17:14:16.105 INFO:tasks.workunit.client.1.smithi115.stdout:2/349: creat d4/f78 x:0 0 0 2021-11-03T17:14:16.105 INFO:tasks.workunit.client.1.smithi115.stdout:2/350: readlink d4/d6/d56/d6c/d34/d43/d28/l5c 0 2021-11-03T17:14:16.106 INFO:tasks.workunit.client.1.smithi115.stdout:3/444: truncate d13/f26 285810 0 2021-11-03T17:14:16.108 INFO:tasks.workunit.client.1.smithi115.stdout:6/320: link d0/d1/d10/d13/l33 d0/d1/d2e/l68 0 2021-11-03T17:14:16.110 INFO:tasks.workunit.client.1.smithi115.stdout:2/351: mkdir d4/d6/d56/d6c/d34/d43/d63/d77/d79 0 2021-11-03T17:14:16.110 INFO:tasks.workunit.client.1.smithi115.stdout:2/352: write d4/d6/d13/f2f [4544757,21208] 0 2021-11-03T17:14:16.113 INFO:tasks.workunit.client.1.smithi115.stdout:3/445: creat d13/d35/f93 x:0 0 0 2021-11-03T17:14:16.114 INFO:tasks.workunit.client.1.smithi115.stdout:6/321: fdatasync d0/d1/d10/f3b 0 2021-11-03T17:14:16.116 INFO:tasks.workunit.client.1.smithi115.stdout:7/306: dread d3/da/f42 [0,4194304] 0 2021-11-03T17:14:16.116 INFO:tasks.workunit.client.1.smithi115.stdout:2/353: creat d4/d6/d13/d75/f7a x:0 0 0 2021-11-03T17:14:16.116 INFO:tasks.workunit.client.1.smithi115.stdout:2/354: readlink d4/d44/l61 0 2021-11-03T17:14:16.117 INFO:tasks.workunit.client.1.smithi115.stdout:2/355: write d4/d6/d56/d6c/f17 [1857787,119744] 0 2021-11-03T17:14:16.117 INFO:tasks.workunit.client.1.smithi115.stdout:2/356: truncate d4/d6/d56/d6c/f6f 850753 0 2021-11-03T17:14:16.117 INFO:tasks.workunit.client.1.smithi115.stdout:0/430: dwrite d9/db/f4f [0,4194304] 0 2021-11-03T17:14:16.120 INFO:tasks.workunit.client.1.smithi115.stdout:8/311: dwrite d4/d6/d3f/f45 [0,4194304] 0 2021-11-03T17:14:16.121 INFO:tasks.workunit.client.1.smithi115.stdout:0/431: read d9/d15/d2c/d46/d8c/f48 [3905904,23599] 0 2021-11-03T17:14:16.121 INFO:tasks.workunit.client.1.smithi115.stdout:0/432: chown d9/d15/d2c/d46/d63 2 1 2021-11-03T17:14:16.121 INFO:tasks.workunit.client.1.smithi115.stdout:0/433: stat d9/d15/d2c/d46/d63/f6d 0 2021-11-03T17:14:16.121 INFO:tasks.workunit.client.1.smithi115.stdout:0/434: write d9/d15/f67 [798363,128397] 0 2021-11-03T17:14:16.122 INFO:tasks.workunit.client.1.smithi115.stdout:6/322: symlink d0/d1/d10/d13/d1c/d1e/l69 0 2021-11-03T17:14:16.122 INFO:tasks.workunit.client.1.smithi115.stdout:2/357: rename d4/d44/f4a to d4/d6/d13/d75/f7b 0 2021-11-03T17:14:16.124 INFO:tasks.workunit.client.1.smithi115.stdout:5/468: dwrite d1a/d28/d4c/d75/d68/f6c [0,4194304] 0 2021-11-03T17:14:16.125 INFO:tasks.workunit.client.1.smithi115.stdout:5/469: mknod d1a/d3d/d89/c9e 1 2021-11-03T17:14:16.125 INFO:tasks.workunit.client.1.smithi115.stdout:5/470: write d1a/d28/d4c/d63/f9c [4626505,1540] 0 2021-11-03T17:14:16.131 INFO:tasks.workunit.client.1.smithi115.stdout:8/312: symlink d4/d6/d18/d30/d33/l71 0 2021-11-03T17:14:16.132 INFO:tasks.workunit.client.1.smithi115.stdout:6/323: dread d0/d1/d10/d13/d1c/d1e/f47 [0,4194304] 0 2021-11-03T17:14:16.133 INFO:tasks.workunit.client.1.smithi115.stdout:0/435: creat d9/d15/d2c/d46/d63/f8d x:0 0 0 2021-11-03T17:14:16.133 INFO:tasks.workunit.client.1.smithi115.stdout:0/436: truncate d9/d15/f67 1927770 0 2021-11-03T17:14:16.133 INFO:tasks.workunit.client.1.smithi115.stdout:0/437: stat d9/d15/d2c/d46/d63/f7b 0 2021-11-03T17:14:16.133 INFO:tasks.workunit.client.1.smithi115.stdout:0/438: fdatasync d9/db/f7e 0 2021-11-03T17:14:16.134 INFO:tasks.workunit.client.1.smithi115.stdout:6/324: read d0/f4f [488200,39106] 0 2021-11-03T17:14:16.135 INFO:tasks.workunit.client.1.smithi115.stdout:3/446: dwrite d13/d35/f36 [4194304,4194304] 0 2021-11-03T17:14:16.136 INFO:tasks.workunit.client.1.smithi115.stdout:5/471: creat d1a/d28/d7b/f9f x:0 0 0 2021-11-03T17:14:16.142 INFO:tasks.workunit.client.1.smithi115.stdout:0/439: creat d9/db/f8e x:0 0 0 2021-11-03T17:14:16.142 INFO:tasks.workunit.client.1.smithi115.stdout:0/440: write d9/db/f42 [456842,103446] 0 2021-11-03T17:14:16.142 INFO:tasks.workunit.client.1.smithi115.stdout:2/358: symlink d4/d6/d56/d6c/d34/d43/d28/l7c 0 2021-11-03T17:14:16.143 INFO:tasks.workunit.client.1.smithi115.stdout:6/325: mkdir d0/d1/d6a 0 2021-11-03T17:14:16.143 INFO:tasks.workunit.client.1.smithi115.stdout:0/441: write d9/d15/d2c/d46/d8c/f49 [131437,122331] 0 2021-11-03T17:14:16.144 INFO:tasks.workunit.client.1.smithi115.stdout:3/447: creat d13/d2a/d8a/f94 x:0 0 0 2021-11-03T17:14:16.145 INFO:tasks.workunit.client.1.smithi115.stdout:3/448: dread - d13/d2a/d8a/f94 zero size 2021-11-03T17:14:16.146 INFO:tasks.workunit.client.1.smithi115.stdout:5/472: unlink d1a/d28/d7b/l36 0 2021-11-03T17:14:16.147 INFO:tasks.workunit.client.1.smithi115.stdout:8/313: dwrite d4/d6/d18/d30/d33/f5c [0,4194304] 0 2021-11-03T17:14:16.151 INFO:tasks.workunit.client.1.smithi115.stdout:2/359: rmdir d4/d6/d56/d6c/d34/d43/d28/d76 0 2021-11-03T17:14:16.152 INFO:tasks.workunit.client.1.smithi115.stdout:6/326: truncate d0/d1/d10/d13/d1c/d1e/f47 2276148 0 2021-11-03T17:14:16.155 INFO:tasks.workunit.client.1.smithi115.stdout:0/442: unlink d9/d37/l76 0 2021-11-03T17:14:16.155 INFO:tasks.workunit.client.1.smithi115.stdout:0/443: mknod d9/d15/d2c/d6e/d71/c8f 1 2021-11-03T17:14:16.156 INFO:tasks.workunit.client.1.smithi115.stdout:3/449: creat d13/d2a/d60/f95 x:0 0 0 2021-11-03T17:14:16.157 INFO:tasks.workunit.client.1.smithi115.stdout:5/473: mkdir d1a/d51/da0 0 2021-11-03T17:14:16.158 INFO:tasks.workunit.client.1.smithi115.stdout:5/474: readlink d1a/d28/d7b/d4f/l9b 0 2021-11-03T17:14:16.158 INFO:tasks.workunit.client.1.smithi115.stdout:5/475: truncate d1a/d28/d4c/d75/d83/f98 602792 0 2021-11-03T17:14:16.163 INFO:tasks.workunit.client.1.smithi115.stdout:8/314: symlink d4/d6/d18/l72 0 2021-11-03T17:14:16.163 INFO:tasks.workunit.client.1.smithi115.stdout:2/360: symlink d4/d6/d13/d75/l7d 0 2021-11-03T17:14:16.163 INFO:tasks.workunit.client.1.smithi115.stdout:2/361: write d4/f62 [83420,41046] 0 2021-11-03T17:14:16.164 INFO:tasks.workunit.client.1.smithi115.stdout:3/450: mkdir d13/d2a/d53/d70/d96 0 2021-11-03T17:14:16.166 INFO:tasks.workunit.client.1.smithi115.stdout:8/315: write d4/d6/d18/d30/d33/f5c [3974553,104537] 0 2021-11-03T17:14:16.166 INFO:tasks.workunit.client.1.smithi115.stdout:8/316: dread - d4/d6/d3f/d43/f5b zero size 2021-11-03T17:14:16.167 INFO:tasks.workunit.client.1.smithi115.stdout:2/362: link d4/d6/d56/d6c/d34/d43/d28/f3b d4/d6/f7e 0 2021-11-03T17:14:16.169 INFO:tasks.workunit.client.1.smithi115.stdout:8/317: symlink d4/d6/d18/d30/d33/l73 0 2021-11-03T17:14:16.170 INFO:tasks.workunit.client.1.smithi115.stdout:2/363: mkdir d4/d6/d13/d6d/d7f 0 2021-11-03T17:14:16.185 INFO:tasks.workunit.client.1.smithi115.stdout:5/476: dwrite d1a/d28/f91 [0,4194304] 0 2021-11-03T17:14:16.185 INFO:tasks.workunit.client.1.smithi115.stdout:5/477: chown d1a/l2f 21418 1 2021-11-03T17:14:16.186 INFO:tasks.workunit.client.1.smithi115.stdout:3/451: rmdir d13/d2a/d60 39 2021-11-03T17:14:16.188 INFO:tasks.workunit.client.1.smithi115.stdout:6/327: dwrite d0/d1/d2/f58 [0,4194304] 0 2021-11-03T17:14:16.190 INFO:tasks.workunit.client.1.smithi115.stdout:5/478: mkdir d1a/d28/d4c/d75/d83/da1 0 2021-11-03T17:14:16.191 INFO:tasks.workunit.client.1.smithi115.stdout:5/479: mknod d1a/d28/d78/ca2 1 2021-11-03T17:14:16.195 INFO:tasks.workunit.client.1.smithi115.stdout:6/328: creat d0/d1/d2/d5d/f6b x:0 0 0 2021-11-03T17:14:16.195 INFO:tasks.workunit.client.1.smithi115.stdout:6/329: write d0/d1/d2/f35 [248255,80374] 0 2021-11-03T17:14:16.195 INFO:tasks.workunit.client.1.smithi115.stdout:6/330: chown d0/d1/d10/d13/d1c/d1e/l43 398 1 2021-11-03T17:14:16.198 INFO:tasks.workunit.client.1.smithi115.stdout:3/452: dread d13/d51/f41 [4194304,4194304] 0 2021-11-03T17:14:16.198 INFO:tasks.workunit.client.1.smithi115.stdout:3/453: write d13/d2a/d60/f95 [429366,69521] 0 2021-11-03T17:14:16.199 INFO:tasks.workunit.client.1.smithi115.stdout:3/454: chown d13/d51/d40/l6e 408 1 2021-11-03T17:14:16.201 INFO:tasks.workunit.client.1.smithi115.stdout:6/331: dread d0/d1/d10/d13/d1c/f2f [0,4194304] 0 2021-11-03T17:14:16.201 INFO:tasks.workunit.client.1.smithi115.stdout:6/332: write d0/d1/f19 [136108,83309] 0 2021-11-03T17:14:16.201 INFO:tasks.workunit.client.1.smithi115.stdout:6/333: readlink d0/d1/d10/d13/d1c/l57 0 2021-11-03T17:14:16.202 INFO:tasks.workunit.client.1.smithi115.stdout:2/364: dwrite d4/d6/d56/f5d [4194304,4194304] 0 2021-11-03T17:14:16.202 INFO:tasks.workunit.client.1.smithi115.stdout:6/334: symlink d0/d1/d2/d5d/l6c 0 2021-11-03T17:14:16.203 INFO:tasks.workunit.client.1.smithi115.stdout:6/335: chown d0/d1/d10/d13/d1c/f34 253 1 2021-11-03T17:14:16.203 INFO:tasks.workunit.client.1.smithi115.stdout:2/365: rmdir d4/d6/d13/d6d 39 2021-11-03T17:14:16.204 INFO:tasks.workunit.client.1.smithi115.stdout:2/366: chown d4/d6/d56/d6c/l5e 7 1 2021-11-03T17:14:16.249 INFO:tasks.workunit.client.1.smithi115.stdout:6/336: dwrite d0/d1/f19 [0,4194304] 0 2021-11-03T17:14:16.249 INFO:tasks.workunit.client.1.smithi115.stdout:6/337: mknod d0/d1/d2/d5d/d61/c6d 1 2021-11-03T17:14:16.250 INFO:tasks.workunit.client.1.smithi115.stdout:2/367: dwrite d4/d6/d56/d6c/f3c [4194304,4194304] 0 2021-11-03T17:14:16.250 INFO:tasks.workunit.client.1.smithi115.stdout:6/338: creat d0/d1/d6a/f6e x:0 0 0 2021-11-03T17:14:16.252 INFO:tasks.workunit.client.1.smithi115.stdout:2/368: mknod d4/d6/d56/d6c/d34/d43/d28/c80 1 2021-11-03T17:14:16.252 INFO:tasks.workunit.client.1.smithi115.stdout:6/339: symlink d0/d1/d2/d5d/d61/l6f 0 2021-11-03T17:14:16.253 INFO:tasks.workunit.client.1.smithi115.stdout:2/369: unlink d4/d6/d56/d6c/d34/d43/d28/l31 0 2021-11-03T17:14:16.262 INFO:tasks.workunit.client.1.smithi115.stdout:6/340: write d0/d1/d2/f1f [218838,80207] 0 2021-11-03T17:14:16.263 INFO:tasks.workunit.client.1.smithi115.stdout:6/341: mknod d0/d1/d2/d5d/c70 1 2021-11-03T17:14:16.263 INFO:tasks.workunit.client.1.smithi115.stdout:5/480: write f11 [3531813,25666] 0 2021-11-03T17:14:16.264 INFO:tasks.workunit.client.1.smithi115.stdout:0/444: dwrite d9/d15/d2c/d46/d8c/f79 [0,4194304] 0 2021-11-03T17:14:16.264 INFO:tasks.workunit.client.1.smithi115.stdout:0/445: dread - d9/d15/d2c/d46/d8c/f74 zero size 2021-11-03T17:14:16.265 INFO:tasks.workunit.client.1.smithi115.stdout:5/481: rename d1a/d28/d34/l4e to d1a/d23/la3 0 2021-11-03T17:14:16.265 INFO:tasks.workunit.client.1.smithi115.stdout:5/482: dread - d1a/d28/d7b/d48/d7f/f86 zero size 2021-11-03T17:14:16.266 INFO:tasks.workunit.client.1.smithi115.stdout:5/483: fsync d1a/d28/d4c/d63/f9c 0 2021-11-03T17:14:16.266 INFO:tasks.workunit.client.1.smithi115.stdout:6/342: chown d0/d1/d10/l26 653872 1 2021-11-03T17:14:16.266 INFO:tasks.workunit.client.1.smithi115.stdout:5/484: truncate d1a/d28/d7b/d48/d7f/f86 182974 0 2021-11-03T17:14:16.266 INFO:tasks.workunit.client.1.smithi115.stdout:6/343: fsync d0/d1/d6a/f6e 0 2021-11-03T17:14:16.267 INFO:tasks.workunit.client.1.smithi115.stdout:5/485: fdatasync d1a/d3d/d89/f8f 0 2021-11-03T17:14:16.267 INFO:tasks.workunit.client.1.smithi115.stdout:6/344: readlink d0/d1/l30 0 2021-11-03T17:14:16.267 INFO:tasks.workunit.client.1.smithi115.stdout:8/318: dwrite d4/d6/d18/d30/f3a [4194304,4194304] 0 2021-11-03T17:14:16.269 INFO:tasks.workunit.client.1.smithi115.stdout:2/370: dwrite d4/d6/d13/f58 [0,4194304] 0 2021-11-03T17:14:16.271 INFO:tasks.workunit.client.1.smithi115.stdout:6/345: unlink d0/d1/d10/d13/d1c/f34 0 2021-11-03T17:14:16.271 INFO:tasks.workunit.client.1.smithi115.stdout:5/486: symlink d1a/d28/d4c/d75/d83/la4 0 2021-11-03T17:14:16.274 INFO:tasks.workunit.client.1.smithi115.stdout:5/487: read d1a/d3d/f42 [2396370,99028] 0 2021-11-03T17:14:16.274 INFO:tasks.workunit.client.1.smithi115.stdout:8/319: rename d4/d6/l41 to d4/d6/d18/d30/l74 0 2021-11-03T17:14:16.275 INFO:tasks.workunit.client.1.smithi115.stdout:6/346: symlink d0/d1/d2/d3f/l71 0 2021-11-03T17:14:16.275 INFO:tasks.workunit.client.1.smithi115.stdout:6/347: write d0/d1/d10/d13/d59/f67 [287098,26996] 0 2021-11-03T17:14:16.275 INFO:tasks.workunit.client.1.smithi115.stdout:6/348: readlink d0/d1/d10/d13/d1c/l57 0 2021-11-03T17:14:16.276 INFO:tasks.workunit.client.1.smithi115.stdout:8/320: creat d4/d6/d18/d30/d33/f75 x:0 0 0 2021-11-03T17:14:16.278 INFO:tasks.workunit.client.1.smithi115.stdout:8/321: rmdir d4/d6/d18/d44 39 2021-11-03T17:14:16.278 INFO:tasks.workunit.client.1.smithi115.stdout:8/322: chown d4/d6/d18/d30/d33/l73 72 1 2021-11-03T17:14:16.279 INFO:tasks.workunit.client.1.smithi115.stdout:8/323: chown d4/d6/f1d 349232742 1 2021-11-03T17:14:16.280 INFO:tasks.workunit.client.1.smithi115.stdout:5/488: dread d1a/d23/f61 [0,4194304] 0 2021-11-03T17:14:16.281 INFO:tasks.workunit.client.1.smithi115.stdout:8/324: link d4/d6/d18/d44/f48 d4/d6/d18/d30/d33/d4c/f76 0 2021-11-03T17:14:16.282 INFO:tasks.workunit.client.1.smithi115.stdout:8/325: write d4/d6/d18/d44/f62 [758023,61810] 0 2021-11-03T17:14:16.282 INFO:tasks.workunit.client.1.smithi115.stdout:8/326: write d4/d6/d3f/f66 [492356,47486] 0 2021-11-03T17:14:16.283 INFO:tasks.workunit.client.1.smithi115.stdout:5/489: rename d1a/d51 to d1a/d28/d4c/d75/d68/da5 0 2021-11-03T17:14:16.289 INFO:tasks.workunit.client.1.smithi115.stdout:5/490: write d1a/d3d/f47 [2575409,60105] 0 2021-11-03T17:14:16.290 INFO:tasks.workunit.client.1.smithi115.stdout:0/446: dwrite d9/d15/f67 [0,4194304] 0 2021-11-03T17:14:16.290 INFO:tasks.workunit.client.1.smithi115.stdout:0/447: chown d9/d15/d1a/d1d/l7d 3 1 2021-11-03T17:14:16.292 INFO:tasks.workunit.client.1.smithi115.stdout:5/491: unlink d1a/d3d/f64 0 2021-11-03T17:14:16.294 INFO:tasks.workunit.client.1.smithi115.stdout:0/448: write d9/d15/f67 [2074573,5089] 0 2021-11-03T17:14:16.296 INFO:tasks.workunit.client.1.smithi115.stdout:5/492: dread d1a/d28/d7b/f74 [0,4194304] 0 2021-11-03T17:14:16.305 INFO:tasks.workunit.client.1.smithi115.stdout:8/327: dwrite d4/d6/d18/d30/f35 [0,4194304] 0 2021-11-03T17:14:16.307 INFO:tasks.workunit.client.1.smithi115.stdout:8/328: write d4/d6/d18/d30/f31 [75972,128235] 0 2021-11-03T17:14:16.307 INFO:tasks.workunit.client.1.smithi115.stdout:2/371: dwrite d4/d6/f50 [0,4194304] 0 2021-11-03T17:14:16.308 INFO:tasks.workunit.client.1.smithi115.stdout:0/449: dread d9/d15/d2c/d46/d8c/f79 [0,4194304] 0 2021-11-03T17:14:16.308 INFO:tasks.workunit.client.1.smithi115.stdout:0/450: readlink d9/l52 0 2021-11-03T17:14:16.308 INFO:tasks.workunit.client.1.smithi115.stdout:2/372: readlink d4/d6/d56/d6c/d34/d43/l6a 0 2021-11-03T17:14:16.310 INFO:tasks.workunit.client.1.smithi115.stdout:8/329: creat d4/d6/d18/f77 x:0 0 0 2021-11-03T17:14:16.313 INFO:tasks.workunit.client.1.smithi115.stdout:5/493: truncate d1a/d3d/f47 257829 0 2021-11-03T17:14:16.316 INFO:tasks.workunit.client.1.smithi115.stdout:0/451: mkdir d9/d15/d90 0 2021-11-03T17:14:16.316 INFO:tasks.workunit.client.1.smithi115.stdout:0/452: fsync d9/d15/f1e 0 2021-11-03T17:14:16.317 INFO:tasks.workunit.client.1.smithi115.stdout:8/330: symlink d4/d6/d18/l78 0 2021-11-03T17:14:16.318 INFO:tasks.workunit.client.1.smithi115.stdout:8/331: mkdir d4/d6/d18/d30/d79 0 2021-11-03T17:14:16.318 INFO:tasks.workunit.client.1.smithi115.stdout:8/332: read - d4/d6/d18/d30/d33/f75 zero size 2021-11-03T17:14:16.319 INFO:tasks.workunit.client.1.smithi115.stdout:8/333: fsync d4/d6/d18/d30/d33/f5c 0 2021-11-03T17:14:16.322 INFO:tasks.workunit.client.1.smithi115.stdout:2/373: dread d4/d6/f48 [0,4194304] 0 2021-11-03T17:14:16.347 INFO:tasks.workunit.client.1.smithi115.stdout:5/494: dwrite f11 [0,4194304] 0 2021-11-03T17:14:16.347 INFO:tasks.workunit.client.1.smithi115.stdout:5/495: write d1a/d28/d4c/f6e [8857813,57548] 0 2021-11-03T17:14:16.348 INFO:tasks.workunit.client.1.smithi115.stdout:0/453: dwrite d9/db/f42 [0,4194304] 0 2021-11-03T17:14:16.348 INFO:tasks.workunit.client.1.smithi115.stdout:5/496: stat d1a/d28/d7b/l41 0 2021-11-03T17:14:16.352 INFO:tasks.workunit.client.1.smithi115.stdout:2/374: dwrite d4/d6/d13/d75/f7a [0,4194304] 0 2021-11-03T17:14:16.353 INFO:tasks.workunit.client.1.smithi115.stdout:0/454: creat d9/d15/d78/d80/d82/f91 x:0 0 0 2021-11-03T17:14:16.355 INFO:tasks.workunit.client.1.smithi115.stdout:8/334: dwrite d4/d6/d18/d30/f35 [0,4194304] 0 2021-11-03T17:14:16.357 INFO:tasks.workunit.client.1.smithi115.stdout:0/455: rename d9/d15/d2c/f7f to d9/d15/d1a/f92 0 2021-11-03T17:14:16.360 INFO:tasks.workunit.client.1.smithi115.stdout:2/375: dread d4/d6/d56/d6c/f17 [0,4194304] 0 2021-11-03T17:14:16.362 INFO:tasks.workunit.client.1.smithi115.stdout:2/376: unlink d4/d6/d56/d6c/d34/d43/f3d 0 2021-11-03T17:14:16.365 INFO:tasks.workunit.client.1.smithi115.stdout:2/377: link d4/d6/d56/d6c/d34/d43/f4d d4/d6/d56/d6c/d34/d43/d63/d77/f81 0 2021-11-03T17:14:16.366 INFO:tasks.workunit.client.1.smithi115.stdout:5/497: dwrite d1a/d28/d34/f2a [0,4194304] 0 2021-11-03T17:14:16.368 INFO:tasks.workunit.client.1.smithi115.stdout:4/543: sync 2021-11-03T17:14:16.380 INFO:tasks.workunit.client.1.smithi115.stdout:4/544: dread d7/dc/fb0 [0,4194304] 0 2021-11-03T17:14:16.381 INFO:tasks.workunit.client.1.smithi115.stdout:4/545: symlink d7/dc/lc0 0 2021-11-03T17:14:16.381 INFO:tasks.workunit.client.1.smithi115.stdout:4/546: stat d7/dc/d1a/d21/db1/d56/f8d 0 2021-11-03T17:14:16.382 INFO:tasks.workunit.client.1.smithi115.stdout:0/456: dwrite d9/d15/d1a/f68 [0,4194304] 0 2021-11-03T17:14:16.382 INFO:tasks.workunit.client.1.smithi115.stdout:0/457: mknod d9/d15/d2c/c93 1 2021-11-03T17:14:16.385 INFO:tasks.workunit.client.1.smithi115.stdout:2/378: dwrite d4/d6/d56/d6c/d34/d43/f2b [0,4194304] 0 2021-11-03T17:14:16.385 INFO:tasks.workunit.client.1.smithi115.stdout:8/335: dwrite d4/d6/d18/f63 [0,4194304] 0 2021-11-03T17:14:16.385 INFO:tasks.workunit.client.1.smithi115.stdout:8/336: mknod d4/d6/d18/d30/d79/c7a 1 2021-11-03T17:14:16.387 INFO:tasks.workunit.client.1.smithi115.stdout:8/337: dread d4/d6/d18/d44/f62 [0,4194304] 0 2021-11-03T17:14:16.387 INFO:tasks.workunit.client.1.smithi115.stdout:8/338: write d4/d6/d18/f3d [189262,14739] 0 2021-11-03T17:14:16.388 INFO:tasks.workunit.client.1.smithi115.stdout:8/339: read d4/d6/d3f/f61 [148220,88469] 0 2021-11-03T17:14:16.388 INFO:tasks.workunit.client.1.smithi115.stdout:8/340: write d4/d6/f3c [693498,60598] 0 2021-11-03T17:14:16.389 INFO:tasks.workunit.client.1.smithi115.stdout:2/379: mkdir d4/d6/d56/d6c/d34/d43/d63/d77/d79/d82 0 2021-11-03T17:14:16.393 INFO:tasks.workunit.client.1.smithi115.stdout:5/498: dread d1a/d3d/f62 [4194304,4194304] 0 2021-11-03T17:14:16.393 INFO:tasks.workunit.client.1.smithi115.stdout:8/341: write d4/d6/d18/d30/d33/f5c [707636,80924] 0 2021-11-03T17:14:16.394 INFO:tasks.workunit.client.1.smithi115.stdout:8/342: readlink d4/d6/d3f/l57 0 2021-11-03T17:14:16.394 INFO:tasks.workunit.client.1.smithi115.stdout:5/499: creat d1a/d28/d7b/d4f/d81/fa6 x:0 0 0 2021-11-03T17:14:16.399 INFO:tasks.workunit.client.1.smithi115.stdout:4/547: dread d7/dc/fa4 [0,4194304] 0 2021-11-03T17:14:16.405 INFO:tasks.workunit.client.1.smithi115.stdout:8/343: rename d4/d6/d18/d30/d33/d4c to d4/d6/d18/d30/d79/d7b 0 2021-11-03T17:14:16.405 INFO:tasks.workunit.client.1.smithi115.stdout:8/344: dread - d4/d6/d18/d30/d79/d7b/f65 zero size 2021-11-03T17:14:16.415 INFO:tasks.workunit.client.1.smithi115.stdout:8/345: dread d4/d6/d18/d30/f3a [0,4194304] 0 2021-11-03T17:14:16.423 INFO:tasks.workunit.client.1.smithi115.stdout:2/380: dwrite d4/d6/d56/d6c/d34/f70 [0,4194304] 0 2021-11-03T17:14:16.423 INFO:tasks.workunit.client.1.smithi115.stdout:2/381: readlink d4/d6/d13/d6d/l59 0 2021-11-03T17:14:16.423 INFO:tasks.workunit.client.1.smithi115.stdout:2/382: mknod d4/d6/c83 1 2021-11-03T17:14:16.423 INFO:tasks.workunit.client.1.smithi115.stdout:0/458: dwrite d9/d15/d1a/d1d/f22 [0,4194304] 0 2021-11-03T17:14:16.424 INFO:tasks.workunit.client.1.smithi115.stdout:5/500: dwrite d1a/d28/d4c/d75/d68/f6c [0,4194304] 0 2021-11-03T17:14:16.424 INFO:tasks.workunit.client.1.smithi115.stdout:5/501: write d1a/d28/d7b/d4f/d81/fa6 [741119,29798] 0 2021-11-03T17:14:16.425 INFO:tasks.workunit.client.1.smithi115.stdout:5/502: write d1a/d28/d4c/d75/d83/f88 [1237970,123693] 0 2021-11-03T17:14:16.425 INFO:tasks.workunit.client.1.smithi115.stdout:0/459: write d9/f25 [4966229,91040] 0 2021-11-03T17:14:16.425 INFO:tasks.workunit.client.1.smithi115.stdout:0/460: truncate d9/d15/d2c/d46/d63/f8d 869831 0 2021-11-03T17:14:16.426 INFO:tasks.workunit.client.1.smithi115.stdout:4/548: dwrite d7/dc/d6a/f3a [0,4194304] 0 2021-11-03T17:14:16.426 INFO:tasks.workunit.client.1.smithi115.stdout:2/383: mkdir d4/d6/d56/d6c/d34/d84 0 2021-11-03T17:14:16.428 INFO:tasks.workunit.client.1.smithi115.stdout:5/503: unlink d1a/l69 0 2021-11-03T17:14:16.429 INFO:tasks.workunit.client.1.smithi115.stdout:4/549: rmdir d7/dc/d1a/d21/db1/d56 39 2021-11-03T17:14:16.429 INFO:tasks.workunit.client.1.smithi115.stdout:4/550: stat d7/dc/d1a/d21/db1/d56/d63/f87 0 2021-11-03T17:14:16.429 INFO:tasks.workunit.client.1.smithi115.stdout:0/461: symlink d9/d15/d78/d80/l94 0 2021-11-03T17:14:16.430 INFO:tasks.workunit.client.1.smithi115.stdout:2/384: mknod d4/d6/d56/d6c/d34/d43/d28/c85 1 2021-11-03T17:14:16.431 INFO:tasks.workunit.client.1.smithi115.stdout:5/504: creat d1a/d28/d4c/d75/d68/da5/da0/fa7 x:0 0 0 2021-11-03T17:14:16.431 INFO:tasks.workunit.client.1.smithi115.stdout:5/505: stat d1a/d3d/f42 0 2021-11-03T17:14:16.434 INFO:tasks.workunit.client.1.smithi115.stdout:8/346: dread d4/d6/d18/d30/f31 [0,4194304] 0 2021-11-03T17:14:16.434 INFO:tasks.workunit.client.1.smithi115.stdout:8/347: fsync d4/d6/d18/d30/f31 0 2021-11-03T17:14:16.434 INFO:tasks.workunit.client.1.smithi115.stdout:4/551: creat d7/dc/d1a/d21/db1/d56/d63/d7b/fc1 x:0 0 0 2021-11-03T17:14:16.435 INFO:tasks.workunit.client.1.smithi115.stdout:2/385: link d4/d44/l47 d4/d6/d56/l86 0 2021-11-03T17:14:16.439 INFO:tasks.workunit.client.1.smithi115.stdout:4/552: unlink d7/dc/d1a/d21/db1/d56/d63/f85 0 2021-11-03T17:14:16.439 INFO:tasks.workunit.client.1.smithi115.stdout:4/553: dread - d7/dc/d1a/d21/db1/d56/d63/fa6 zero size 2021-11-03T17:14:16.440 INFO:tasks.workunit.client.0.smithi027.stdout:5/647: sync 2021-11-03T17:14:16.440 INFO:tasks.workunit.client.0.smithi027.stdout:7/266: sync 2021-11-03T17:14:16.441 INFO:tasks.workunit.client.1.smithi115.stdout:2/386: symlink d4/d6/d56/d6c/d34/d43/d63/d77/d79/l87 0 2021-11-03T17:14:16.441 INFO:tasks.workunit.client.1.smithi115.stdout:2/387: readlink d4/d6/d56/d6c/d34/d42/l51 0 2021-11-03T17:14:16.442 INFO:tasks.workunit.client.1.smithi115.stdout:4/554: symlink d7/dc/d1a/d21/db1/d56/d63/lc2 0 2021-11-03T17:14:16.443 INFO:tasks.workunit.client.0.smithi027.stdout:7/267: mkdir d0/d2/d59 0 2021-11-03T17:14:16.443 INFO:tasks.workunit.client.1.smithi115.stdout:2/388: symlink d4/d6/d13/l88 0 2021-11-03T17:14:16.444 INFO:tasks.workunit.client.0.smithi027.stdout:7/268: symlink d0/d2/d4f/l5a 0 2021-11-03T17:14:16.446 INFO:tasks.workunit.client.1.smithi115.stdout:4/555: unlink d7/dc/d1a/d21/db1/d56/lae 0 2021-11-03T17:14:16.447 INFO:tasks.workunit.client.0.smithi027.stdout:7/269: rename d0/f51 to d0/d2/d10/d1e/d20/f5b 0 2021-11-03T17:14:16.447 INFO:tasks.workunit.client.1.smithi115.stdout:2/389: truncate d4/d6/d56/d6c/d34/d43/f4d 34277 0 2021-11-03T17:14:16.448 INFO:tasks.workunit.client.1.smithi115.stdout:2/390: chown d4/d6/d13/f38 1128690 1 2021-11-03T17:14:16.448 INFO:tasks.workunit.client.0.smithi027.stdout:7/270: creat d0/d2/d10/d1e/f5c x:0 0 0 2021-11-03T17:14:16.449 INFO:tasks.workunit.client.1.smithi115.stdout:4/556: creat d7/d7a/fc3 x:0 0 0 2021-11-03T17:14:16.449 INFO:tasks.workunit.client.1.smithi115.stdout:2/391: creat d4/d6/d56/d6c/d34/f89 x:0 0 0 2021-11-03T17:14:16.451 INFO:tasks.workunit.client.1.smithi115.stdout:4/557: truncate d7/dc/d6a/f46 280608 0 2021-11-03T17:14:16.451 INFO:tasks.workunit.client.1.smithi115.stdout:2/392: rmdir d4/d6/d56/d6c/d34/d42/d72 0 2021-11-03T17:14:16.453 INFO:tasks.workunit.client.1.smithi115.stdout:4/558: mkdir d7/dc/d6a/d3c/dbf/dc4 0 2021-11-03T17:14:16.454 INFO:tasks.workunit.client.1.smithi115.stdout:2/393: unlink d4/d6/d56/d6c/d34/f3f 0 2021-11-03T17:14:16.456 INFO:tasks.workunit.client.1.smithi115.stdout:2/394: rename d4/d6/d56/d6c/d34/d43/d28/l7c to d4/d6/d13/d75/l8a 0 2021-11-03T17:14:16.456 INFO:tasks.workunit.client.1.smithi115.stdout:2/395: write d4/d6/f48 [5118646,95766] 0 2021-11-03T17:14:16.457 INFO:tasks.workunit.client.1.smithi115.stdout:8/348: dwrite d4/d6/d18/d30/f35 [4194304,4194304] 0 2021-11-03T17:14:16.457 INFO:tasks.workunit.client.1.smithi115.stdout:8/349: write d4/d6/f6b [865885,73569] 0 2021-11-03T17:14:16.457 INFO:tasks.workunit.client.1.smithi115.stdout:0/462: dwrite d9/d15/d2c/d46/d8c/f48 [0,4194304] 0 2021-11-03T17:14:16.462 INFO:tasks.workunit.client.1.smithi115.stdout:2/396: rename d4/d6/f50 to d4/d6/d13/d6d/f8b 0 2021-11-03T17:14:16.464 INFO:tasks.workunit.client.1.smithi115.stdout:2/397: dread d4/f62 [0,4194304] 0 2021-11-03T17:14:16.464 INFO:tasks.workunit.client.1.smithi115.stdout:2/398: chown d4/d44 7 1 2021-11-03T17:14:16.468 INFO:tasks.workunit.client.1.smithi115.stdout:8/350: creat d4/d6/d18/d44/d47/f7c x:0 0 0 2021-11-03T17:14:16.470 INFO:tasks.workunit.client.0.smithi027.stdout:7/271: truncate d0/d2d/d2f/f39 400280 0 2021-11-03T17:14:16.470 INFO:tasks.workunit.client.0.smithi027.stdout:7/272: fdatasync d0/d2/d1f/f46 0 2021-11-03T17:14:16.472 INFO:tasks.workunit.client.0.smithi027.stdout:7/273: rmdir d0/d2 39 2021-11-03T17:14:16.473 INFO:tasks.workunit.client.0.smithi027.stdout:7/274: chown d0/f18 648 1 2021-11-03T17:14:16.476 INFO:tasks.workunit.client.1.smithi115.stdout:0/463: getdents d9/d37 0 2021-11-03T17:14:16.479 INFO:tasks.workunit.client.1.smithi115.stdout:0/464: unlink d9/d15/d2c/d46/d8c/f44 0 2021-11-03T17:14:16.483 INFO:tasks.workunit.client.1.smithi115.stdout:0/465: symlink d9/d15/d2c/d46/d63/l95 0 2021-11-03T17:14:16.483 INFO:tasks.workunit.client.1.smithi115.stdout:0/466: chown d9/d15/d78/d80/f87 0 1 2021-11-03T17:14:16.488 INFO:tasks.workunit.client.1.smithi115.stdout:8/351: dread d4/d6/d18/d30/f35 [4194304,4194304] 0 2021-11-03T17:14:16.488 INFO:tasks.workunit.client.1.smithi115.stdout:8/352: chown d4/f6c 101 1 2021-11-03T17:14:16.489 INFO:tasks.workunit.client.1.smithi115.stdout:8/353: chown d4/d6/d18/l78 252 1 2021-11-03T17:14:16.489 INFO:tasks.workunit.client.1.smithi115.stdout:8/354: write d4/d6/d3f/d43/f5b [147268,112722] 0 2021-11-03T17:14:16.490 INFO:tasks.workunit.client.1.smithi115.stdout:2/399: dwrite d4/d6/f45 [0,4194304] 0 2021-11-03T17:14:16.492 INFO:tasks.workunit.client.1.smithi115.stdout:4/559: dwrite d7/dc/d6a/f46 [0,4194304] 0 2021-11-03T17:14:16.494 INFO:tasks.workunit.client.1.smithi115.stdout:4/560: mkdir d7/dc/d6a/d3c/dc5 0 2021-11-03T17:14:16.501 INFO:tasks.workunit.client.1.smithi115.stdout:0/467: dread d9/d15/d1a/d1d/f22 [0,4194304] 0 2021-11-03T17:14:16.501 INFO:tasks.workunit.client.1.smithi115.stdout:0/468: fsync d9/d15/d1a/f92 0 2021-11-03T17:14:16.503 INFO:tasks.workunit.client.1.smithi115.stdout:0/469: chown d9/d15/d1a/d1d/l61 43137 1 2021-11-03T17:14:16.510 INFO:tasks.workunit.client.0.smithi027.stdout:7/275: write d0/f13 [4935750,104361] 0 2021-11-03T17:14:16.517 INFO:tasks.workunit.client.1.smithi115.stdout:8/355: dwrite d4/f28 [0,4194304] 0 2021-11-03T17:14:16.517 INFO:tasks.workunit.client.1.smithi115.stdout:8/356: read - d4/d6/d18/f77 zero size 2021-11-03T17:14:16.518 INFO:tasks.workunit.client.1.smithi115.stdout:8/357: mkdir d4/d7d 0 2021-11-03T17:14:16.519 INFO:tasks.workunit.client.1.smithi115.stdout:8/358: write d4/f6c [127726,127372] 0 2021-11-03T17:14:16.520 INFO:tasks.workunit.client.1.smithi115.stdout:8/359: link d4/d6/d18/d44/f48 d4/d6/d18/d30/d33/f7e 0 2021-11-03T17:14:16.521 INFO:tasks.workunit.client.1.smithi115.stdout:8/360: mkdir d4/d6/d18/d30/d79/d7f 0 2021-11-03T17:14:16.526 INFO:tasks.workunit.client.1.smithi115.stdout:4/561: dwrite d7/dc/db6/f9a [0,4194304] 0 2021-11-03T17:14:16.530 INFO:tasks.workunit.client.1.smithi115.stdout:4/562: mkdir d7/dc/d6a/d3c/dbf/dc4/dc6 0 2021-11-03T17:14:16.531 INFO:tasks.workunit.client.1.smithi115.stdout:2/400: dwrite d4/d6/d56/d6c/f6f [0,4194304] 0 2021-11-03T17:14:16.531 INFO:tasks.workunit.client.1.smithi115.stdout:4/563: stat d7/l26 0 2021-11-03T17:14:16.532 INFO:tasks.workunit.client.1.smithi115.stdout:0/470: dwrite d9/d15/d2c/d46/f5b [0,4194304] 0 2021-11-03T17:14:16.536 INFO:tasks.workunit.client.1.smithi115.stdout:4/564: read d7/dc/d1a/d21/db1/d56/d42/f58 [274342,68121] 0 2021-11-03T17:14:16.536 INFO:tasks.workunit.client.1.smithi115.stdout:4/565: write d7/d7a/fc3 [467142,14970] 0 2021-11-03T17:14:16.538 INFO:tasks.workunit.client.1.smithi115.stdout:2/401: mkdir d4/d6/d13/d75/d8c 0 2021-11-03T17:14:16.539 INFO:tasks.workunit.client.1.smithi115.stdout:0/471: dread d9/d15/d1a/d1d/f22 [0,4194304] 0 2021-11-03T17:14:16.545 INFO:tasks.workunit.client.1.smithi115.stdout:4/566: rmdir d7/dc/d1a/d21/db1/d56/d42 39 2021-11-03T17:14:16.549 INFO:tasks.workunit.client.1.smithi115.stdout:2/402: mkdir d4/d6/d56/d6c/d34/d42/d8d 0 2021-11-03T17:14:16.549 INFO:tasks.workunit.client.1.smithi115.stdout:8/361: dwrite d4/f26 [0,4194304] 0 2021-11-03T17:14:16.549 INFO:tasks.workunit.client.1.smithi115.stdout:8/362: chown d4/d6/d18/d30/d79/d7b/l53 47 1 2021-11-03T17:14:16.549 INFO:tasks.workunit.client.1.smithi115.stdout:8/363: chown d4/d6/d18/d30/d79/d7b/f54 3013399 1 2021-11-03T17:14:16.550 INFO:tasks.workunit.client.1.smithi115.stdout:0/472: creat d9/d15/d2c/d46/d63/f96 x:0 0 0 2021-11-03T17:14:16.550 INFO:tasks.workunit.client.1.smithi115.stdout:0/473: chown d9/d15/d2c/d46/d63/d70/l86 112 1 2021-11-03T17:14:16.552 INFO:tasks.workunit.client.0.smithi027.stdout:7/276: dwrite d0/f13 [8388608,4194304] 0 2021-11-03T17:14:16.553 INFO:tasks.workunit.client.1.smithi115.stdout:8/364: getdents d4/d6/d18 0 2021-11-03T17:14:16.555 INFO:tasks.workunit.client.0.smithi027.stdout:7/277: link d0/l55 d0/d2/d1c/d4d/l5d 0 2021-11-03T17:14:16.556 INFO:tasks.workunit.client.1.smithi115.stdout:0/474: rename d9/d15/d2c/d46/d63/f8d to d9/d15/d78/d80/f97 0 2021-11-03T17:14:16.557 INFO:tasks.workunit.client.0.smithi027.stdout:7/278: unlink d0/d2/l9 0 2021-11-03T17:14:16.557 INFO:tasks.workunit.client.0.smithi027.stdout:7/279: chown d0/d2/d10/d35/l38 8171689 1 2021-11-03T17:14:16.559 INFO:tasks.workunit.client.0.smithi027.stdout:7/280: rename d0/d2/d10/d1e/d20/d31 to d0/d2/d5e 0 2021-11-03T17:14:16.559 INFO:tasks.workunit.client.1.smithi115.stdout:0/475: creat d9/f98 x:0 0 0 2021-11-03T17:14:16.559 INFO:tasks.workunit.client.1.smithi115.stdout:0/476: readlink d9/l52 0 2021-11-03T17:14:16.560 INFO:tasks.workunit.client.1.smithi115.stdout:0/477: mknod d9/d15/d78/c99 1 2021-11-03T17:14:16.561 INFO:tasks.workunit.client.1.smithi115.stdout:0/478: rename d9/d15/d1a/d1d/f22 to d9/d15/d78/d80/f9a 0 2021-11-03T17:14:16.561 INFO:tasks.workunit.client.1.smithi115.stdout:0/479: readlink d9/d15/l45 0 2021-11-03T17:14:16.573 INFO:tasks.workunit.client.1.smithi115.stdout:8/365: dwrite d4/d6/d18/d30/d79/d7b/f54 [0,4194304] 0 2021-11-03T17:14:16.574 INFO:tasks.workunit.client.1.smithi115.stdout:8/366: mknod d4/d6/d18/d30/d33/c80 1 2021-11-03T17:14:16.576 INFO:tasks.workunit.client.1.smithi115.stdout:0/480: unlink d9/d15/d2c/d46/d63/d70/l7a 0 2021-11-03T17:14:16.586 INFO:tasks.workunit.client.0.smithi027.stdout:7/281: dwrite d0/d2/d10/d1e/d20/f43 [0,4194304] 0 2021-11-03T17:14:16.586 INFO:tasks.workunit.client.0.smithi027.stdout:7/282: write d0/d2/d10/d1e/d20/f5b [909540,36137] 0 2021-11-03T17:14:16.587 INFO:tasks.workunit.client.1.smithi115.stdout:0/481: dread d9/db/f42 [0,4194304] 0 2021-11-03T17:14:16.589 INFO:tasks.workunit.client.1.smithi115.stdout:0/482: creat d9/d15/d90/f9b x:0 0 0 2021-11-03T17:14:16.589 INFO:tasks.workunit.client.0.smithi027.stdout:7/283: creat d0/d2/d10/d1e/d20/f5f x:0 0 0 2021-11-03T17:14:16.590 INFO:tasks.workunit.client.0.smithi027.stdout:7/284: dread - d0/d2/d10/d1e/f5c zero size 2021-11-03T17:14:16.592 INFO:tasks.workunit.client.1.smithi115.stdout:8/367: dwrite d4/d6/d18/d30/d33/f5f [0,4194304] 0 2021-11-03T17:14:16.592 INFO:tasks.workunit.client.1.smithi115.stdout:8/368: write d4/d6/d18/d44/d47/f7c [64615,82983] 0 2021-11-03T17:14:16.595 INFO:tasks.workunit.client.1.smithi115.stdout:0/483: creat d9/db/d89/f9c x:0 0 0 2021-11-03T17:14:16.597 INFO:tasks.workunit.client.1.smithi115.stdout:8/369: dread d4/fa [4194304,4194304] 0 2021-11-03T17:14:16.598 INFO:tasks.workunit.client.1.smithi115.stdout:0/484: rename d9/d15/d2c/d46/d63/f6d to d9/db/d83/f9d 0 2021-11-03T17:14:16.605 INFO:tasks.workunit.client.0.smithi027.stdout:7/285: dwrite d0/f16 [0,4194304] 0 2021-11-03T17:14:16.606 INFO:tasks.workunit.client.1.smithi115.stdout:8/370: dread d4/d6/d18/d30/d33/f5f [0,4194304] 0 2021-11-03T17:14:16.606 INFO:tasks.workunit.client.1.smithi115.stdout:8/371: truncate d4/d6/d18/d30/d79/d7b/f65 975265 0 2021-11-03T17:14:16.608 INFO:tasks.workunit.client.0.smithi027.stdout:7/286: symlink d0/d2/d4f/l60 0 2021-11-03T17:14:16.610 INFO:tasks.workunit.client.0.smithi027.stdout:7/287: rename d0/d2/d1c/f3c to d0/d2/d1c/f61 0 2021-11-03T17:14:16.613 INFO:tasks.workunit.client.0.smithi027.stdout:7/288: symlink d0/d2/d10/d1e/l62 0 2021-11-03T17:14:16.614 INFO:tasks.workunit.client.1.smithi115.stdout:0/485: dwrite d9/d15/d2c/d46/d8c/f7c [4194304,4194304] 0 2021-11-03T17:14:16.615 INFO:tasks.workunit.client.0.smithi027.stdout:7/289: creat d0/d2/d5e/d36/f63 x:0 0 0 2021-11-03T17:14:16.616 INFO:tasks.workunit.client.0.smithi027.stdout:7/290: chown d0/d2/d10/d35/l56 6 1 2021-11-03T17:14:16.616 INFO:tasks.workunit.client.0.smithi027.stdout:7/291: creat d0/d2/d5e/d36/f64 x:0 0 0 2021-11-03T17:14:16.617 INFO:tasks.workunit.client.1.smithi115.stdout:0/486: creat d9/d15/d1a/f9e x:0 0 0 2021-11-03T17:14:16.617 INFO:tasks.workunit.client.1.smithi115.stdout:0/487: write d9/db/f8e [1008610,18194] 0 2021-11-03T17:14:16.618 INFO:tasks.workunit.client.0.smithi027.stdout:7/292: creat d0/d2d/d53/f65 x:0 0 0 2021-11-03T17:14:16.618 INFO:tasks.workunit.client.1.smithi115.stdout:0/488: link d9/d15/d78/d80/f97 d9/d15/d2c/d6e/f9f 0 2021-11-03T17:14:16.620 INFO:tasks.workunit.client.0.smithi027.stdout:7/293: mkdir d0/d2d/d66 0 2021-11-03T17:14:16.620 INFO:tasks.workunit.client.0.smithi027.stdout:7/294: getdents d0/d2/d10 0 2021-11-03T17:14:16.624 INFO:tasks.workunit.client.1.smithi115.stdout:8/372: dwrite d4/d6/d18/f63 [0,4194304] 0 2021-11-03T17:14:16.627 INFO:tasks.workunit.client.1.smithi115.stdout:8/373: dread d4/d6/d3f/d43/f5b [0,4194304] 0 2021-11-03T17:14:16.628 INFO:tasks.workunit.client.1.smithi115.stdout:8/374: rename d4/d6/d18/d30/l6a to d4/d7d/l81 0 2021-11-03T17:14:16.628 INFO:tasks.workunit.client.1.smithi115.stdout:8/375: dread - d4/d6/d18/f77 zero size 2021-11-03T17:14:16.630 INFO:tasks.workunit.client.1.smithi115.stdout:8/376: unlink d4/d6/d3f/f45 0 2021-11-03T17:14:16.630 INFO:tasks.workunit.client.1.smithi115.stdout:8/377: chown d4/d6/d18/d30/d79/d7b/f65 24260909 1 2021-11-03T17:14:16.630 INFO:tasks.workunit.client.1.smithi115.stdout:8/378: write d4/d6/d18/d44/f48 [1856488,119976] 0 2021-11-03T17:14:16.631 INFO:tasks.workunit.client.1.smithi115.stdout:8/379: symlink d4/d6/l82 0 2021-11-03T17:14:16.631 INFO:tasks.workunit.client.1.smithi115.stdout:8/380: chown d4/d6/d18/d30/d33/f59 109 1 2021-11-03T17:14:16.631 INFO:tasks.workunit.client.1.smithi115.stdout:8/381: mknod d4/d6/d18/d44/d6e/c83 1 2021-11-03T17:14:16.632 INFO:tasks.workunit.client.1.smithi115.stdout:8/382: creat d4/d6/d18/d30/d79/d7b/f84 x:0 0 0 2021-11-03T17:14:16.633 INFO:tasks.workunit.client.1.smithi115.stdout:8/383: rmdir d4/d6/d3f/d43 39 2021-11-03T17:14:16.634 INFO:tasks.workunit.client.1.smithi115.stdout:8/384: creat d4/d6/d18/f85 x:0 0 0 2021-11-03T17:14:16.635 INFO:tasks.workunit.client.1.smithi115.stdout:8/385: fsync d4/d6/f2b 0 2021-11-03T17:14:16.635 INFO:tasks.workunit.client.1.smithi115.stdout:8/386: write d4/fa [4385922,73186] 0 2021-11-03T17:14:16.636 INFO:tasks.workunit.client.1.smithi115.stdout:8/387: chown d4/d6/d18/d30/d33/l73 9654030 1 2021-11-03T17:14:16.636 INFO:tasks.workunit.client.0.smithi027.stdout:7/295: dwrite d0/d2/d10/f11 [0,4194304] 0 2021-11-03T17:14:16.637 INFO:tasks.workunit.client.1.smithi115.stdout:8/388: truncate d4/d6/d18/d30/d33/f5c 2226146 0 2021-11-03T17:14:16.638 INFO:tasks.workunit.client.0.smithi027.stdout:7/296: rmdir d0/d2/d10 39 2021-11-03T17:14:16.648 INFO:tasks.workunit.client.1.smithi115.stdout:8/389: dread d4/f26 [0,4194304] 0 2021-11-03T17:14:16.649 INFO:tasks.workunit.client.1.smithi115.stdout:8/390: read d4/d6/d18/f23 [1966598,93514] 0 2021-11-03T17:14:16.654 INFO:tasks.workunit.client.1.smithi115.stdout:8/391: dread d4/f6c [0,4194304] 0 2021-11-03T17:14:16.654 INFO:tasks.workunit.client.1.smithi115.stdout:8/392: truncate d4/d6/d18/d30/f31 3014497 0 2021-11-03T17:14:16.655 INFO:tasks.workunit.client.1.smithi115.stdout:8/393: symlink d4/d6/d18/d30/d79/d7f/l86 0 2021-11-03T17:14:16.655 INFO:tasks.workunit.client.1.smithi115.stdout:8/394: write d4/d6/d18/d30/d79/d7b/f84 [761922,78936] 0 2021-11-03T17:14:16.656 INFO:tasks.workunit.client.1.smithi115.stdout:8/395: creat d4/d6/d18/d30/d33/f87 x:0 0 0 2021-11-03T17:14:16.657 INFO:tasks.workunit.client.0.smithi027.stdout:7/297: dwrite d0/d2d/d53/f65 [0,4194304] 0 2021-11-03T17:14:16.658 INFO:tasks.workunit.client.1.smithi115.stdout:8/396: dread d4/d6/f6b [0,4194304] 0 2021-11-03T17:14:16.659 INFO:tasks.workunit.client.1.smithi115.stdout:8/397: chown d4/d6/l2e 0 1 2021-11-03T17:14:16.660 INFO:tasks.workunit.client.0.smithi027.stdout:7/298: symlink d0/d2/d10/d1e/d20/l67 0 2021-11-03T17:14:16.660 INFO:tasks.workunit.client.0.smithi027.stdout:7/299: stat d0/d2/d10/f1b 0 2021-11-03T17:14:16.660 INFO:tasks.workunit.client.0.smithi027.stdout:7/300: stat d0 0 2021-11-03T17:14:16.661 INFO:tasks.workunit.client.1.smithi115.stdout:8/398: read d4/d6/d18/d30/d33/f5c [855726,63384] 0 2021-11-03T17:14:16.661 INFO:tasks.workunit.client.0.smithi027.stdout:7/301: symlink d0/d2d/d58/l68 0 2021-11-03T17:14:16.670 INFO:tasks.workunit.client.0.smithi027.stdout:7/302: rmdir d0/d2/d10/d1e 39 2021-11-03T17:14:16.672 INFO:tasks.workunit.client.0.smithi027.stdout:7/303: getdents d0/d2 0 2021-11-03T17:14:16.674 INFO:tasks.workunit.client.0.smithi027.stdout:7/304: link d0/d2/d1c/f27 d0/d2d/d58/f69 0 2021-11-03T17:14:16.674 INFO:tasks.workunit.client.0.smithi027.stdout:7/305: write d0/f4c [178609,103911] 0 2021-11-03T17:14:16.674 INFO:tasks.workunit.client.0.smithi027.stdout:7/306: mknod d0/d2/d1c/d4d/c6a 1 2021-11-03T17:14:16.675 INFO:tasks.workunit.client.0.smithi027.stdout:7/307: truncate d0/d2/d10/d1e/f48 202633 0 2021-11-03T17:14:16.676 INFO:tasks.workunit.client.0.smithi027.stdout:7/308: symlink d0/d2/d5e/l6b 0 2021-11-03T17:14:16.677 INFO:tasks.workunit.client.1.smithi115.stdout:8/399: dwrite d4/d6/d18/d30/d79/d7b/f76 [0,4194304] 0 2021-11-03T17:14:16.679 INFO:tasks.workunit.client.0.smithi027.stdout:7/309: dread d0/d2/d1c/f27 [0,4194304] 0 2021-11-03T17:14:16.680 INFO:tasks.workunit.client.0.smithi027.stdout:7/310: fsync d0/d2/f5 0 2021-11-03T17:14:16.683 INFO:tasks.workunit.client.0.smithi027.stdout:7/311: link d0/d2/d10/d1e/f5c d0/d2/f6c 0 2021-11-03T17:14:16.683 INFO:tasks.workunit.client.0.smithi027.stdout:7/312: write d0/f25 [4225512,25191] 0 2021-11-03T17:14:16.684 INFO:tasks.workunit.client.0.smithi027.stdout:7/313: read - d0/d2/d1f/f54 zero size 2021-11-03T17:14:16.685 INFO:tasks.workunit.client.0.smithi027.stdout:7/314: creat d0/d2d/d58/f6d x:0 0 0 2021-11-03T17:14:16.702 INFO:tasks.workunit.client.0.smithi027.stdout:7/315: truncate d0/f13 5680959 0 2021-11-03T17:14:16.702 INFO:tasks.workunit.client.0.smithi027.stdout:7/316: write d0/d2/d1f/f54 [278020,9219] 0 2021-11-03T17:14:16.702 INFO:tasks.workunit.client.0.smithi027.stdout:7/317: write d0/d2/d10/d1e/f5c [370420,49411] 0 2021-11-03T17:14:16.713 INFO:tasks.workunit.client.1.smithi115.stdout:8/400: dwrite d4/fa [0,4194304] 0 2021-11-03T17:14:16.713 INFO:tasks.workunit.client.1.smithi115.stdout:8/401: chown d4/d6/d18/f23 0 1 2021-11-03T17:14:16.723 INFO:tasks.workunit.client.1.smithi115.stdout:8/402: dread d4/d6/d18/d30/d79/d7b/f54 [0,4194304] 0 2021-11-03T17:14:16.724 INFO:tasks.workunit.client.1.smithi115.stdout:8/403: mkdir d4/d6/d3f/d43/d88 0 2021-11-03T17:14:16.725 INFO:tasks.workunit.client.0.smithi027.stdout:7/318: dwrite d0/f4c [0,4194304] 0 2021-11-03T17:14:16.726 INFO:tasks.workunit.client.0.smithi027.stdout:7/319: write d0/d2/d10/d1e/f5c [825994,104160] 0 2021-11-03T17:14:16.728 INFO:tasks.workunit.client.1.smithi115.stdout:8/404: read d4/d6/d18/d30/d33/f7e [227839,121439] 0 2021-11-03T17:14:16.728 INFO:tasks.workunit.client.1.smithi115.stdout:8/405: creat d4/d6/d18/d30/d79/d7f/f89 x:0 0 0 2021-11-03T17:14:16.728 INFO:tasks.workunit.client.1.smithi115.stdout:8/406: getdents d4/d6/d3f/d43/d88 0 2021-11-03T17:14:16.729 INFO:tasks.workunit.client.1.smithi115.stdout:8/407: readlink d4/d6/l24 0 2021-11-03T17:14:16.730 INFO:tasks.workunit.client.0.smithi027.stdout:7/320: getdents d0/d2d/d58 0 2021-11-03T17:14:16.731 INFO:tasks.workunit.client.1.smithi115.stdout:8/408: read d4/d6/d18/d44/f62 [4076329,64439] 0 2021-11-03T17:14:16.731 INFO:tasks.workunit.client.1.smithi115.stdout:8/409: creat d4/d6/f8a x:0 0 0 2021-11-03T17:14:16.742 INFO:tasks.workunit.client.0.smithi027.stdout:7/321: dread d0/d2/d10/d1e/d20/f43 [0,4194304] 0 2021-11-03T17:14:16.744 INFO:tasks.workunit.client.0.smithi027.stdout:7/322: mkdir d0/d2/d1c/d4d/d6e 0 2021-11-03T17:14:16.745 INFO:tasks.workunit.client.1.smithi115.stdout:8/410: dwrite d4/fa [0,4194304] 0 2021-11-03T17:14:16.746 INFO:tasks.workunit.client.0.smithi027.stdout:7/323: symlink d0/d2/d5e/d36/d3a/l6f 0 2021-11-03T17:14:16.747 INFO:tasks.workunit.client.0.smithi027.stdout:7/324: write d0/d2/d10/f1b [1052932,30996] 0 2021-11-03T17:14:16.761 INFO:tasks.workunit.client.1.smithi115.stdout:8/411: dwrite d4/d6/d18/d30/d79/d7f/f89 [0,4194304] 0 2021-11-03T17:14:16.762 INFO:tasks.workunit.client.1.smithi115.stdout:8/412: stat d4/d6/d18/d30/d79/d7b/f76 0 2021-11-03T17:14:16.764 INFO:tasks.workunit.client.1.smithi115.stdout:8/413: creat d4/d6/d3f/d43/d88/f8b x:0 0 0 2021-11-03T17:14:16.764 INFO:tasks.workunit.client.1.smithi115.stdout:8/414: truncate d4/d6/f42 578706 0 2021-11-03T17:14:16.765 INFO:tasks.workunit.client.1.smithi115.stdout:8/415: creat d4/d6/d18/d44/f8c x:0 0 0 2021-11-03T17:14:16.767 INFO:tasks.workunit.client.1.smithi115.stdout:8/416: read d4/d6/d18/d30/d79/d7b/f76 [545032,105279] 0 2021-11-03T17:14:16.767 INFO:tasks.workunit.client.1.smithi115.stdout:8/417: chown d4/d6/f6b 49139304 1 2021-11-03T17:14:16.789 INFO:tasks.workunit.client.1.smithi115.stdout:8/418: dwrite d4/d6/d18/d30/f31 [0,4194304] 0 2021-11-03T17:14:16.790 INFO:tasks.workunit.client.1.smithi115.stdout:8/419: symlink d4/d6/d18/l8d 0 2021-11-03T17:14:16.804 INFO:tasks.workunit.client.1.smithi115.stdout:8/420: dwrite d4/d6/d18/d44/d47/f7c [0,4194304] 0 2021-11-03T17:14:16.812 INFO:tasks.workunit.client.1.smithi115.stdout:8/421: rmdir d4/d6 39 2021-11-03T17:14:16.817 INFO:tasks.workunit.client.1.smithi115.stdout:8/422: truncate d4/d6/d18/d44/f48 294516 0 2021-11-03T17:14:16.817 INFO:tasks.workunit.client.1.smithi115.stdout:8/423: fdatasync d4/d6/d18/d44/f8c 0 2021-11-03T17:14:16.818 INFO:tasks.workunit.client.1.smithi115.stdout:8/424: fdatasync d4/d6/d18/d30/f31 0 2021-11-03T17:14:16.818 INFO:tasks.workunit.client.1.smithi115.stdout:8/425: symlink d4/d6/d3f/d43/l8e 0 2021-11-03T17:14:16.833 INFO:tasks.workunit.client.1.smithi115.stdout:8/426: dwrite d4/d6/f2a [0,4194304] 0 2021-11-03T17:14:16.833 INFO:tasks.workunit.client.1.smithi115.stdout:8/427: dread - d4/d6/d18/d44/f8c zero size 2021-11-03T17:14:16.849 INFO:tasks.workunit.client.1.smithi115.stdout:8/428: dwrite d4/d6/f3c [4194304,4194304] 0 2021-11-03T17:14:16.849 INFO:tasks.workunit.client.1.smithi115.stdout:8/429: fdatasync d4/d6/d18/f3d 0 2021-11-03T17:14:16.850 INFO:tasks.workunit.client.1.smithi115.stdout:8/430: write d4/d6/d3f/f66 [102516,99692] 0 2021-11-03T17:14:16.851 INFO:tasks.workunit.client.1.smithi115.stdout:8/431: creat d4/d6/d3f/d43/d88/f8f x:0 0 0 2021-11-03T17:14:16.852 INFO:tasks.workunit.client.1.smithi115.stdout:8/432: unlink d4/d6/d18/d30/d79/d7b/f76 0 2021-11-03T17:14:16.854 INFO:tasks.workunit.client.1.smithi115.stdout:8/433: rename d4/d6/d3f/f66 to d4/d6/d18/f90 0 2021-11-03T17:14:16.855 INFO:tasks.workunit.client.1.smithi115.stdout:8/434: symlink d4/l91 0 2021-11-03T17:14:16.860 INFO:tasks.workunit.client.1.smithi115.stdout:8/435: dread d4/d6/d18/f90 [0,4194304] 0 2021-11-03T17:14:16.874 INFO:tasks.workunit.client.1.smithi115.stdout:8/436: dwrite d4/fa [0,4194304] 0 2021-11-03T17:14:16.875 INFO:tasks.workunit.client.1.smithi115.stdout:8/437: creat d4/d6/d3f/d43/f92 x:0 0 0 2021-11-03T17:14:16.877 INFO:tasks.workunit.client.1.smithi115.stdout:8/438: unlink d4/d6/d18/d30/l5a 0 2021-11-03T17:14:16.877 INFO:tasks.workunit.client.1.smithi115.stdout:8/439: mkdir d4/d6/d18/d30/d33/d93 0 2021-11-03T17:14:16.895 INFO:tasks.workunit.client.1.smithi115.stdout:8/440: dwrite d4/d6/d18/d30/d79/d7b/f84 [0,4194304] 0 2021-11-03T17:14:16.896 INFO:tasks.workunit.client.1.smithi115.stdout:8/441: mknod d4/d6/d18/d30/d79/c94 1 2021-11-03T17:14:16.904 INFO:tasks.workunit.client.1.smithi115.stdout:8/442: dread d4/d6/d18/d30/d33/f5f [4194304,4194304] 0 2021-11-03T17:14:16.905 INFO:tasks.workunit.client.1.smithi115.stdout:8/443: link d4/fa d4/d6/d18/d30/d33/f95 0 2021-11-03T17:14:16.905 INFO:tasks.workunit.client.1.smithi115.stdout:8/444: stat d4/d6/d18/d30/d79 0 2021-11-03T17:14:16.906 INFO:tasks.workunit.client.1.smithi115.stdout:8/445: rename d4/d6/d3f/d43/l8e to d4/d7d/l96 0 2021-11-03T17:14:16.907 INFO:tasks.workunit.client.1.smithi115.stdout:8/446: symlink d4/d6/d18/d30/d79/l97 0 2021-11-03T17:14:16.908 INFO:tasks.workunit.client.1.smithi115.stdout:8/447: symlink d4/d6/l98 0 2021-11-03T17:14:16.909 INFO:tasks.workunit.client.1.smithi115.stdout:8/448: creat d4/d6/d18/d44/f99 x:0 0 0 2021-11-03T17:14:16.910 INFO:tasks.workunit.client.1.smithi115.stdout:8/449: readlink d4/d6/d18/d30/d33/l68 0 2021-11-03T17:14:16.915 INFO:tasks.workunit.client.1.smithi115.stdout:8/450: rename d4/d6/d18/d30/d79/d7b to d4/d6/d3f/d9a 0 2021-11-03T17:14:16.917 INFO:tasks.workunit.client.1.smithi115.stdout:8/451: symlink d4/d6/d18/d30/d33/d93/l9b 0 2021-11-03T17:14:16.933 INFO:tasks.workunit.client.1.smithi115.stdout:8/452: dwrite d4/d6/d3f/d43/d88/f8f [0,4194304] 0 2021-11-03T17:14:16.934 INFO:tasks.workunit.client.1.smithi115.stdout:8/453: mknod d4/d6/d3f/c9c 1 2021-11-03T17:14:16.934 INFO:tasks.workunit.client.1.smithi115.stdout:8/454: readlink d4/l10 0 2021-11-03T17:14:16.935 INFO:tasks.workunit.client.0.smithi027.stdout:2/400: sync 2021-11-03T17:14:16.936 INFO:tasks.workunit.client.0.smithi027.stdout:2/401: fdatasync d0/d1b/d29/f2f 0 2021-11-03T17:14:16.936 INFO:tasks.workunit.client.1.smithi115.stdout:8/455: unlink d4/d6/d3f/d43/l4a 0 2021-11-03T17:14:16.937 INFO:tasks.workunit.client.1.smithi115.stdout:8/456: mkdir d4/d6/d18/d9d 0 2021-11-03T17:14:16.938 INFO:tasks.workunit.client.1.smithi115.stdout:8/457: dread - d4/d6/d18/d30/d33/f87 zero size 2021-11-03T17:14:16.938 INFO:tasks.workunit.client.1.smithi115.stdout:8/458: chown d4/d6/d18/d44/d47 344 1 2021-11-03T17:14:16.939 INFO:tasks.workunit.client.1.smithi115.stdout:8/459: rename d4/d6/d18/d30/d33/l68 to d4/d6/d18/l9e 0 2021-11-03T17:14:16.948 INFO:tasks.workunit.client.0.smithi027.stdout:2/402: dread d0/d1b/d29/f39 [0,4194304] 0 2021-11-03T17:14:16.948 INFO:tasks.workunit.client.0.smithi027.stdout:2/403: chown d0/d1b/d29/d54/f7f 9940 1 2021-11-03T17:14:16.964 INFO:tasks.workunit.client.1.smithi115.stdout:8/460: dwrite d4/d6/d18/f23 [8388608,4194304] 0 2021-11-03T17:14:16.965 INFO:tasks.workunit.client.1.smithi115.stdout:8/461: creat d4/d6/d18/d44/d6e/f9f x:0 0 0 2021-11-03T17:14:16.969 INFO:tasks.workunit.client.0.smithi027.stdout:2/404: dwrite d0/d2a/f21 [0,4194304] 0 2021-11-03T17:14:16.970 INFO:tasks.workunit.client.0.smithi027.stdout:2/405: chown d0/d1b/d29/f39 0 1 2021-11-03T17:14:16.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:16 smithi115 conmon[34670]: audit 2021-11-03T17:14:15.695610+0000 mon.smithi027 (mon.0) 573 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:14:16.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:16 smithi115 conmon[34670]: audit 2021-11-03T17:14:15.696292+0000 mon.smithi027 (mon.0) 574 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:14:16.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:16 smithi115 conmon[34670]: cluster 2021-11-03T17:14:16.606423+0000 mgr.smithi027.ufhhvb (mgr.14162) 309 : cluster [DBG] pgmap v255: 65 pgs: 65 active+clean; 1.8 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 84 MiB/s rd, 174 MiB/s wr, 284 op/s 2021-11-03T17:14:16.987 INFO:tasks.workunit.client.0.smithi027.stdout:2/406: dwrite d0/d2a/f21 [0,4194304] 0 2021-11-03T17:14:17.002 INFO:tasks.workunit.client.0.smithi027.stdout:3/310: sync 2021-11-03T17:14:17.011 INFO:tasks.workunit.client.0.smithi027.stdout:2/407: dwrite d0/d1b/d29/d54/f7f [0,4194304] 0 2021-11-03T17:14:17.015 INFO:tasks.workunit.client.0.smithi027.stdout:2/408: rmdir d0/d2a/d79 0 2021-11-03T17:14:17.015 INFO:tasks.workunit.client.0.smithi027.stdout:2/409: fsync d0/d2a/f21 0 2021-11-03T17:14:17.018 INFO:tasks.workunit.client.0.smithi027.stdout:2/410: rename d0/f41 to d0/d1b/d29/d58/d5b/f80 0 2021-11-03T17:14:17.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:16 smithi027 conmon[32322]: audit 2021-11-03T17:14:15.695610 2021-11-03T17:14:17.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:16 smithi027 conmon[32322]: +0000 mon.smithi027 (mon.0) 573 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:14:17.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:16 smithi027 conmon[32322]: audit 2021-11-03T17:14:15.696292+0000 mon.smithi027 (mon.0) 574 : audit [DBG] 2021-11-03T17:14:17.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:16 smithi027 conmon[32322]: from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:14:17.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:16 smithi027 conmon[32322]: cluster 2021-11-03T17:14:17.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:16 smithi027 conmon[32322]: 2021-11-03T17:14:16.606423+0000 mgr.smithi027.ufhhvb (mgr.14162 2021-11-03T17:14:17.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:16 smithi027 conmon[32322]: ) 309 : cluster [DBG] pgmap v255: 65 pgs: 65 active+clean; 1.8 GiB data, 5.4 GiB used, 531 GiB / 536 GiB avail; 84 MiB/s rd, 174 MiB/s wr, 284 op/s 2021-11-03T17:14:17.020 INFO:tasks.workunit.client.0.smithi027.stdout:2/411: mknod d0/c81 1 2021-11-03T17:14:17.021 INFO:tasks.workunit.client.0.smithi027.stdout:2/412: dread d0/d1b/d29/d58/d5b/f80 [0,4194304] 0 2021-11-03T17:14:17.042 INFO:tasks.workunit.client.0.smithi027.stdout:2/413: dwrite d0/d1b/d1c/f7e [0,4194304] 0 2021-11-03T17:14:17.044 INFO:tasks.workunit.client.0.smithi027.stdout:2/414: readlink d0/d2a/l68 0 2021-11-03T17:14:17.047 INFO:tasks.workunit.client.0.smithi027.stdout:2/415: dread d0/d5/d2c/f3d [0,4194304] 0 2021-11-03T17:14:17.047 INFO:tasks.workunit.client.0.smithi027.stdout:2/416: truncate d0/d2a/d5d/f75 4356964 0 2021-11-03T17:14:17.048 INFO:tasks.workunit.client.0.smithi027.stdout:2/417: creat d0/d1b/f82 x:0 0 0 2021-11-03T17:14:17.219 INFO:tasks.workunit.client.0.smithi027.stdout:6/435: sync 2021-11-03T17:14:17.219 INFO:tasks.workunit.client.0.smithi027.stdout:6/436: chown dc/d16/d1b/d2d/l69 237 1 2021-11-03T17:14:17.220 INFO:tasks.workunit.client.0.smithi027.stdout:6/437: creat dc/d16/f7a x:0 0 0 2021-11-03T17:14:17.222 INFO:tasks.workunit.client.0.smithi027.stdout:6/438: rename f7 to dc/f7b 0 2021-11-03T17:14:17.222 INFO:tasks.workunit.client.0.smithi027.stdout:6/439: truncate dc/d16/d1b/d3c/d48/f5c 5211100 0 2021-11-03T17:14:17.223 INFO:tasks.workunit.client.0.smithi027.stdout:6/440: symlink dc/d16/d1b/d24/l7c 0 2021-11-03T17:14:17.224 INFO:tasks.workunit.client.0.smithi027.stdout:6/441: write dc/f7b [335104,117392] 0 2021-11-03T17:14:17.225 INFO:tasks.workunit.client.0.smithi027.stdout:6/442: chown dc/d16/d1b/d24/l74 151514 1 2021-11-03T17:14:17.225 INFO:tasks.workunit.client.0.smithi027.stdout:6/443: write dc/d16/d1b/f28 [1839982,113303] 0 2021-11-03T17:14:17.227 INFO:tasks.workunit.client.0.smithi027.stdout:6/444: symlink dc/d16/d62/l7d 0 2021-11-03T17:14:17.229 INFO:tasks.workunit.client.0.smithi027.stdout:6/445: dread dc/d16/d1b/d24/f2a [0,4194304] 0 2021-11-03T17:14:17.230 INFO:tasks.workunit.client.0.smithi027.stdout:6/446: mkdir dc/d16/d1b/d24/d7e 0 2021-11-03T17:14:17.231 INFO:tasks.workunit.client.0.smithi027.stdout:6/447: symlink dc/d16/d1b/d2d/l7f 0 2021-11-03T17:14:17.232 INFO:tasks.workunit.client.0.smithi027.stdout:6/448: read dc/f7b [940837,124329] 0 2021-11-03T17:14:17.232 INFO:tasks.workunit.client.0.smithi027.stdout:6/449: chown dc/d16/d1b/d2d/d34/f4e 66216 1 2021-11-03T17:14:17.233 INFO:tasks.workunit.client.0.smithi027.stdout:6/450: rename dc/d16 to dc/d16/d1b/d3c/d51/d5f/d80 22 2021-11-03T17:14:17.234 INFO:tasks.workunit.client.0.smithi027.stdout:6/451: rename dc/d16/d1b/d3c/d48/f6f to dc/d16/d1b/d3c/d48/f81 0 2021-11-03T17:14:17.234 INFO:tasks.workunit.client.0.smithi027.stdout:6/452: chown dc/d16/d1b/d24/l74 4784 1 2021-11-03T17:14:17.235 INFO:tasks.workunit.client.0.smithi027.stdout:6/453: write dc/f7b [4529969,92136] 0 2021-11-03T17:14:17.238 INFO:tasks.workunit.client.0.smithi027.stdout:6/454: dread dc/f10 [0,4194304] 0 2021-11-03T17:14:17.248 INFO:tasks.workunit.client.0.smithi027.stdout:6/455: rename dc/d16/d1b/d3c/d48 to dc/d82 0 2021-11-03T17:14:17.268 INFO:tasks.workunit.client.0.smithi027.stdout:6/456: dwrite dc/f64 [0,4194304] 0 2021-11-03T17:14:17.270 INFO:tasks.workunit.client.0.smithi027.stdout:6/457: symlink dc/l83 0 2021-11-03T17:14:17.270 INFO:tasks.workunit.client.0.smithi027.stdout:6/458: getdents dc/d16/d1b/d2d 0 2021-11-03T17:14:17.271 INFO:tasks.workunit.client.0.smithi027.stdout:6/459: read dc/f4d [8112374,82576] 0 2021-11-03T17:14:17.326 INFO:tasks.workunit.client.0.smithi027.stdout:4/404: sync 2021-11-03T17:14:17.326 INFO:tasks.workunit.client.0.smithi027.stdout:0/425: sync 2021-11-03T17:14:17.327 INFO:tasks.workunit.client.0.smithi027.stdout:9/489: sync 2021-11-03T17:14:17.330 INFO:tasks.workunit.client.0.smithi027.stdout:0/426: rename d9/da/d64/d70/d79 to d9/d12/d8f 0 2021-11-03T17:14:17.331 INFO:tasks.workunit.client.0.smithi027.stdout:0/427: symlink d9/da/l90 0 2021-11-03T17:14:17.331 INFO:tasks.workunit.client.0.smithi027.stdout:0/428: readlink d9/da/d64/d70/d6d/l82 0 2021-11-03T17:14:17.332 INFO:tasks.workunit.client.0.smithi027.stdout:0/429: mknod d9/c91 1 2021-11-03T17:14:17.334 INFO:tasks.workunit.client.0.smithi027.stdout:9/490: dread db/f16 [0,4194304] 0 2021-11-03T17:14:17.335 INFO:tasks.workunit.client.0.smithi027.stdout:9/491: write db/d1b/d1c/d1d/d40/f53 [2115647,57648] 0 2021-11-03T17:14:17.354 INFO:tasks.workunit.client.0.smithi027.stdout:0/430: dwrite d9/da/d30/f3c [0,4194304] 0 2021-11-03T17:14:17.354 INFO:tasks.workunit.client.0.smithi027.stdout:9/492: dwrite db/d1b/fb0 [0,4194304] 0 2021-11-03T17:14:17.354 INFO:tasks.workunit.client.0.smithi027.stdout:9/493: truncate db/d1b/f38 4277789 0 2021-11-03T17:14:17.358 INFO:tasks.workunit.client.0.smithi027.stdout:0/431: mkdir d9/da/d30/d92 0 2021-11-03T17:14:17.359 INFO:tasks.workunit.client.0.smithi027.stdout:0/432: mkdir d9/da/d30/d93 0 2021-11-03T17:14:17.362 INFO:tasks.workunit.client.0.smithi027.stdout:0/433: link d9/l2f d9/da/d64/d70/d6d/d7a/l94 0 2021-11-03T17:14:17.362 INFO:tasks.workunit.client.0.smithi027.stdout:0/434: getdents d9/da/d30/d93 0 2021-11-03T17:14:17.362 INFO:tasks.workunit.client.0.smithi027.stdout:0/435: stat d9/da/d64/d70/d6d 0 2021-11-03T17:14:17.363 INFO:tasks.workunit.client.0.smithi027.stdout:0/436: write d9/da/d31/f36 [3843033,129965] 0 2021-11-03T17:14:17.370 INFO:tasks.workunit.client.0.smithi027.stdout:0/437: dread d9/f72 [4194304,4194304] 0 2021-11-03T17:14:17.370 INFO:tasks.workunit.client.0.smithi027.stdout:0/438: fsync d9/da/f37 0 2021-11-03T17:14:17.371 INFO:tasks.workunit.client.0.smithi027.stdout:0/439: chown d9/da/d31/l58 11044 1 2021-11-03T17:14:17.372 INFO:tasks.workunit.client.0.smithi027.stdout:0/440: creat d9/f95 x:0 0 0 2021-11-03T17:14:17.373 INFO:tasks.workunit.client.0.smithi027.stdout:0/441: creat d9/da/f96 x:0 0 0 2021-11-03T17:14:17.374 INFO:tasks.workunit.client.0.smithi027.stdout:0/442: chown d9/da/d64/d70/d6d/f6e 0 1 2021-11-03T17:14:17.374 INFO:tasks.workunit.client.0.smithi027.stdout:0/443: fsync d9/f4f 0 2021-11-03T17:14:17.374 INFO:tasks.workunit.client.0.smithi027.stdout:0/444: truncate d9/f95 66988 0 2021-11-03T17:14:17.375 INFO:tasks.workunit.client.0.smithi027.stdout:0/445: getdents d9/d81 0 2021-11-03T17:14:17.393 INFO:tasks.workunit.client.0.smithi027.stdout:0/446: dwrite d9/f24 [0,4194304] 0 2021-11-03T17:14:17.412 INFO:tasks.workunit.client.0.smithi027.stdout:0/447: dwrite d9/da/f78 [0,4194304] 0 2021-11-03T17:14:17.412 INFO:tasks.workunit.client.0.smithi027.stdout:0/448: mknod d9/da/d31/c97 1 2021-11-03T17:14:17.412 INFO:tasks.workunit.client.0.smithi027.stdout:0/449: mknod d9/da/d31/c98 1 2021-11-03T17:14:17.415 INFO:tasks.workunit.client.0.smithi027.stdout:0/450: mkdir d9/da/d30/d93/d99 0 2021-11-03T17:14:17.417 INFO:tasks.workunit.client.0.smithi027.stdout:0/451: truncate d9/da/d30/f54 726967 0 2021-11-03T17:14:17.417 INFO:tasks.workunit.client.0.smithi027.stdout:0/452: mknod d9/da/d64/c9a 1 2021-11-03T17:14:17.418 INFO:tasks.workunit.client.0.smithi027.stdout:0/453: creat d9/da/f9b x:0 0 0 2021-11-03T17:14:17.418 INFO:tasks.workunit.client.0.smithi027.stdout:0/454: fsync d9/f49 0 2021-11-03T17:14:17.419 INFO:tasks.workunit.client.0.smithi027.stdout:0/455: mknod d9/da/d31/c9c 1 2021-11-03T17:14:17.421 INFO:tasks.workunit.client.0.smithi027.stdout:0/456: dread d9/f72 [4194304,4194304] 0 2021-11-03T17:14:17.424 INFO:tasks.workunit.client.0.smithi027.stdout:0/457: rmdir d9/da/d30/d93/d99 0 2021-11-03T17:14:17.430 INFO:tasks.workunit.client.0.smithi027.stdout:0/458: dread d9/f6c [0,4194304] 0 2021-11-03T17:14:17.431 INFO:tasks.workunit.client.0.smithi027.stdout:0/459: truncate d9/da/f16 578777 0 2021-11-03T17:14:17.433 INFO:tasks.workunit.client.0.smithi027.stdout:0/460: creat d9/da/f9d x:0 0 0 2021-11-03T17:14:17.435 INFO:tasks.workunit.client.0.smithi027.stdout:0/461: symlink d9/da/d64/d70/d5f/l9e 0 2021-11-03T17:14:17.436 INFO:tasks.workunit.client.0.smithi027.stdout:0/462: write d9/da/f37 [1213747,120636] 0 2021-11-03T17:14:17.436 INFO:tasks.workunit.client.0.smithi027.stdout:0/463: truncate d9/da/d64/d70/f6a 200563 0 2021-11-03T17:14:17.436 INFO:tasks.workunit.client.0.smithi027.stdout:0/464: truncate d9/da/f9b 123712 0 2021-11-03T17:14:17.437 INFO:tasks.workunit.client.0.smithi027.stdout:0/465: dread - d9/da/f38 zero size 2021-11-03T17:14:17.437 INFO:tasks.workunit.client.0.smithi027.stdout:0/466: truncate d9/da/d64/f76 1785158 0 2021-11-03T17:14:17.437 INFO:tasks.workunit.client.0.smithi027.stdout:0/467: rmdir d9/da/d31 39 2021-11-03T17:14:17.446 INFO:tasks.workunit.client.0.smithi027.stdout:0/468: dread d9/d12/f1d [0,4194304] 0 2021-11-03T17:14:17.447 INFO:tasks.workunit.client.0.smithi027.stdout:0/469: mkdir d9/da/d64/d70/d6d/d9f 0 2021-11-03T17:14:17.449 INFO:tasks.workunit.client.0.smithi027.stdout:0/470: creat d9/da/d30/d92/fa0 x:0 0 0 2021-11-03T17:14:17.471 INFO:tasks.workunit.client.0.smithi027.stdout:0/471: dwrite d9/da/d31/f5c [0,4194304] 0 2021-11-03T17:14:17.472 INFO:tasks.workunit.client.0.smithi027.stdout:0/472: write d9/da/d30/f43 [601816,81431] 0 2021-11-03T17:14:17.488 INFO:tasks.workunit.client.0.smithi027.stdout:0/473: dwrite d9/f2b [0,4194304] 0 2021-11-03T17:14:17.492 INFO:tasks.workunit.client.0.smithi027.stdout:0/474: creat d9/da/d64/d70/d6f/fa1 x:0 0 0 2021-11-03T17:14:17.509 INFO:tasks.workunit.client.0.smithi027.stdout:0/475: dwrite d9/da/f96 [0,4194304] 0 2021-11-03T17:14:17.512 INFO:tasks.workunit.client.0.smithi027.stdout:0/476: symlink d9/da/la2 0 2021-11-03T17:14:17.530 INFO:tasks.workunit.client.0.smithi027.stdout:0/477: dwrite d9/da/f8b [0,4194304] 0 2021-11-03T17:14:17.530 INFO:tasks.workunit.client.0.smithi027.stdout:0/478: write d9/d12/f75 [462709,56661] 0 2021-11-03T17:14:17.536 INFO:tasks.workunit.client.0.smithi027.stdout:8/455: sync 2021-11-03T17:14:17.536 INFO:tasks.workunit.client.0.smithi027.stdout:8/456: mknod d7/d10/d18/d1a/d95/c98 1 2021-11-03T17:14:17.536 INFO:tasks.workunit.client.0.smithi027.stdout:8/457: write d7/d10/d18/f33 [5037549,11134] 0 2021-11-03T17:14:17.537 INFO:tasks.workunit.client.0.smithi027.stdout:0/479: dread d9/da/d30/f5e [0,4194304] 0 2021-11-03T17:14:17.537 INFO:tasks.workunit.client.0.smithi027.stdout:0/480: chown d9/da/d31/f44 305087836 1 2021-11-03T17:14:17.540 INFO:tasks.workunit.client.0.smithi027.stdout:0/481: rename d9/da/fe to d9/da/d64/d70/d6f/fa3 0 2021-11-03T17:14:17.540 INFO:tasks.workunit.client.0.smithi027.stdout:0/482: readlink d9/da/l90 0 2021-11-03T17:14:17.540 INFO:tasks.workunit.client.0.smithi027.stdout:0/483: write d9/da/d30/f5e [1962639,12063] 0 2021-11-03T17:14:17.542 INFO:tasks.workunit.client.0.smithi027.stdout:0/484: creat d9/da/d64/d70/d6d/fa4 x:0 0 0 2021-11-03T17:14:17.542 INFO:tasks.workunit.client.0.smithi027.stdout:8/458: dread d7/d10/d18/d1a/f1f [0,4194304] 0 2021-11-03T17:14:17.545 INFO:tasks.workunit.client.0.smithi027.stdout:8/459: creat d7/d10/d21/f99 x:0 0 0 2021-11-03T17:14:17.547 INFO:tasks.workunit.client.0.smithi027.stdout:8/460: unlink d7/d10/f15 0 2021-11-03T17:14:17.549 INFO:tasks.workunit.client.0.smithi027.stdout:8/461: read d7/d10/d3f/d55/f64 [3268994,2950] 0 2021-11-03T17:14:17.549 INFO:tasks.workunit.client.0.smithi027.stdout:8/462: truncate d7/d10/d21/f45 378558 0 2021-11-03T17:14:17.550 INFO:tasks.workunit.client.0.smithi027.stdout:8/463: chown d7/d10/d18/d1a/f60 98923 1 2021-11-03T17:14:17.550 INFO:tasks.workunit.client.0.smithi027.stdout:8/464: write d7/d10/d4f/f92 [617816,54506] 0 2021-11-03T17:14:17.551 INFO:tasks.workunit.client.0.smithi027.stdout:8/465: creat d7/d10/d18/d1a/d95/f9a x:0 0 0 2021-11-03T17:14:17.560 INFO:tasks.workunit.client.0.smithi027.stdout:0/485: dwrite d9/da/f78 [0,4194304] 0 2021-11-03T17:14:17.567 INFO:tasks.workunit.client.0.smithi027.stdout:0/486: link d9/da/l11 d9/d81/la5 0 2021-11-03T17:14:17.569 INFO:tasks.workunit.client.0.smithi027.stdout:8/466: rename d7/d10/d18/d1a to d7/d10/d3f/d9b 0 2021-11-03T17:14:17.569 INFO:tasks.workunit.client.0.smithi027.stdout:8/467: write d7/d10/d3f/d9b/f3e [3255562,641] 0 2021-11-03T17:14:17.570 INFO:tasks.workunit.client.0.smithi027.stdout:0/487: creat d9/d12/fa6 x:0 0 0 2021-11-03T17:14:17.570 INFO:tasks.workunit.client.0.smithi027.stdout:0/488: truncate d9/da/f16 1170780 0 2021-11-03T17:14:17.570 INFO:tasks.workunit.client.0.smithi027.stdout:0/489: write d9/d12/fa6 [938714,89814] 0 2021-11-03T17:14:17.571 INFO:tasks.workunit.client.0.smithi027.stdout:0/490: truncate d9/da/d30/f3c 2391760 0 2021-11-03T17:14:17.572 INFO:tasks.workunit.client.0.smithi027.stdout:0/491: fdatasync d9/f23 0 2021-11-03T17:14:17.572 INFO:tasks.workunit.client.1.smithi115.stdout:9/374: sync 2021-11-03T17:14:17.573 INFO:tasks.workunit.client.0.smithi027.stdout:8/468: link d7/d10/d3f/d9b/f2e d7/d10/d3f/d91/f9c 0 2021-11-03T17:14:17.573 INFO:tasks.workunit.client.0.smithi027.stdout:8/469: chown d7/d10/d3f/d9b/d95 160782721 1 2021-11-03T17:14:17.574 INFO:tasks.workunit.client.1.smithi115.stdout:9/375: truncate d1/d18/d2b/d42/f21 1421179 0 2021-11-03T17:14:17.574 INFO:tasks.workunit.client.1.smithi115.stdout:9/376: write d1/d18/d4a/d54/d3a/f3b [408431,115698] 0 2021-11-03T17:14:17.579 INFO:tasks.workunit.client.0.smithi027.stdout:0/492: rename d9/d81 to d9/da/d30/da7 0 2021-11-03T17:14:17.580 INFO:tasks.workunit.client.0.smithi027.stdout:0/493: fdatasync d9/da/f9b 0 2021-11-03T17:14:17.585 INFO:tasks.workunit.client.0.smithi027.stdout:8/470: rename d7/d53/d94 to d7/d10/d4f/d9d 0 2021-11-03T17:14:17.587 INFO:tasks.workunit.client.0.smithi027.stdout:0/494: creat d9/da/d64/fa8 x:0 0 0 2021-11-03T17:14:17.589 INFO:tasks.workunit.client.0.smithi027.stdout:0/495: creat d9/da/d64/d70/d6d/d7a/fa9 x:0 0 0 2021-11-03T17:14:17.589 INFO:tasks.workunit.client.0.smithi027.stdout:0/496: read - d9/da/d64/fa8 zero size 2021-11-03T17:14:17.590 INFO:tasks.workunit.client.0.smithi027.stdout:0/497: creat d9/da/d64/d70/d6d/d7a/faa x:0 0 0 2021-11-03T17:14:17.591 INFO:tasks.workunit.client.0.smithi027.stdout:0/498: truncate d9/da/d31/f44 226423 0 2021-11-03T17:14:17.591 INFO:tasks.workunit.client.0.smithi027.stdout:0/499: write d9/da/d64/d70/d6d/fa4 [863109,37935] 0 2021-11-03T17:14:17.592 INFO:tasks.workunit.client.0.smithi027.stdout:0/500: creat d9/da/d30/da7/fab x:0 0 0 2021-11-03T17:14:17.595 INFO:tasks.workunit.client.0.smithi027.stdout:0/501: dread d9/d12/f1d [0,4194304] 0 2021-11-03T17:14:17.597 INFO:tasks.workunit.client.0.smithi027.stdout:0/502: getdents d9/da 0 2021-11-03T17:14:17.599 INFO:tasks.workunit.client.0.smithi027.stdout:0/503: mkdir d9/da/d30/dac 0 2021-11-03T17:14:17.599 INFO:tasks.workunit.client.0.smithi027.stdout:0/504: fsync d9/f4f 0 2021-11-03T17:14:17.600 INFO:tasks.workunit.client.0.smithi027.stdout:0/505: creat d9/da/d64/d70/d5f/fad x:0 0 0 2021-11-03T17:14:17.600 INFO:tasks.workunit.client.0.smithi027.stdout:0/506: chown d9/da/d30/d92 91 1 2021-11-03T17:14:17.601 INFO:tasks.workunit.client.0.smithi027.stdout:0/507: truncate d9/da/f10 832486 0 2021-11-03T17:14:17.601 INFO:tasks.workunit.client.0.smithi027.stdout:0/508: write d9/da/d64/d70/d5f/fad [554477,50120] 0 2021-11-03T17:14:17.602 INFO:tasks.workunit.client.0.smithi027.stdout:0/509: creat d9/da/d31/fae x:0 0 0 2021-11-03T17:14:17.604 INFO:tasks.workunit.client.0.smithi027.stdout:0/510: rename d9/da/d30/l59 to d9/da/d64/d70/d6d/d9f/laf 0 2021-11-03T17:14:17.605 INFO:tasks.workunit.client.0.smithi027.stdout:0/511: read d9/d12/f17 [505590,96019] 0 2021-11-03T17:14:17.605 INFO:tasks.workunit.client.0.smithi027.stdout:0/512: fdatasync d9/f47 0 2021-11-03T17:14:17.605 INFO:tasks.workunit.client.0.smithi027.stdout:0/513: readlink d9/da/la2 0 2021-11-03T17:14:17.613 INFO:tasks.workunit.client.0.smithi027.stdout:0/514: write d9/f1f [112154,77593] 0 2021-11-03T17:14:17.615 INFO:tasks.workunit.client.0.smithi027.stdout:0/515: read d9/da/f8b [3014330,90690] 0 2021-11-03T17:14:17.615 INFO:tasks.workunit.client.0.smithi027.stdout:0/516: readlink d9/da/l11 0 2021-11-03T17:14:17.616 INFO:tasks.workunit.client.0.smithi027.stdout:0/517: dread - d9/da/d64/fa8 zero size 2021-11-03T17:14:17.616 INFO:tasks.workunit.client.0.smithi027.stdout:0/518: chown d9/da/d64/d70/l83 30 1 2021-11-03T17:14:17.617 INFO:tasks.workunit.client.1.smithi115.stdout:9/377: dwrite d1/d18/f22 [0,4194304] 0 2021-11-03T17:14:17.628 INFO:tasks.workunit.client.0.smithi027.stdout:8/471: dwrite d7/d10/d3f/d55/f64 [0,4194304] 0 2021-11-03T17:14:17.629 INFO:tasks.workunit.client.0.smithi027.stdout:8/472: symlink d7/d10/d18/d84/l9e 0 2021-11-03T17:14:17.631 INFO:tasks.workunit.client.0.smithi027.stdout:8/473: creat d7/d10/d3f/d55/f9f x:0 0 0 2021-11-03T17:14:17.631 INFO:tasks.workunit.client.0.smithi027.stdout:8/474: write d7/d53/d67/f6b [1127885,73936] 0 2021-11-03T17:14:17.638 INFO:tasks.workunit.client.1.smithi115.stdout:9/378: dwrite d1/d1a/d48/f2a [0,4194304] 0 2021-11-03T17:14:17.639 INFO:tasks.workunit.client.0.smithi027.stdout:0/519: dwrite d9/da/d64/d70/d6f/fa3 [0,4194304] 0 2021-11-03T17:14:17.640 INFO:tasks.workunit.client.0.smithi027.stdout:8/475: fsync d7/d10/d4f/f92 0 2021-11-03T17:14:17.640 INFO:tasks.workunit.client.0.smithi027.stdout:8/476: fsync d7/d10/d3f/d9b/d95/f7e 0 2021-11-03T17:14:17.641 INFO:tasks.workunit.client.1.smithi115.stdout:9/379: write d1/d18/d4a/d54/d3a/f6d [1227822,109765] 0 2021-11-03T17:14:17.642 INFO:tasks.workunit.client.0.smithi027.stdout:0/520: symlink d9/da/d64/d70/d6d/lb0 0 2021-11-03T17:14:17.644 INFO:tasks.workunit.client.0.smithi027.stdout:8/477: symlink d7/d53/d67/la0 0 2021-11-03T17:14:17.645 INFO:tasks.workunit.client.0.smithi027.stdout:8/478: readlink l4 0 2021-11-03T17:14:17.647 INFO:tasks.workunit.client.0.smithi027.stdout:0/521: unlink d9/da/d64/fa8 0 2021-11-03T17:14:17.648 INFO:tasks.workunit.client.0.smithi027.stdout:8/479: creat d7/d10/fa1 x:0 0 0 2021-11-03T17:14:17.662 INFO:tasks.workunit.client.0.smithi027.stdout:0/522: creat d9/da/d31/fb1 x:0 0 0 2021-11-03T17:14:17.663 INFO:tasks.workunit.client.0.smithi027.stdout:0/523: stat d9/da/f16 0 2021-11-03T17:14:17.663 INFO:tasks.workunit.client.0.smithi027.stdout:8/480: chown d7/d10/d3f/f78 76163859 1 2021-11-03T17:14:17.665 INFO:tasks.workunit.client.0.smithi027.stdout:0/524: creat d9/da/d30/d93/fb2 x:0 0 0 2021-11-03T17:14:17.666 INFO:tasks.workunit.client.1.smithi115.stdout:9/380: dread d1/d1a/d48/d27/f47 [0,4194304] 0 2021-11-03T17:14:17.667 INFO:tasks.workunit.client.1.smithi115.stdout:9/381: stat d1/d18/d4a/d74/f84 0 2021-11-03T17:14:17.667 INFO:tasks.workunit.client.1.smithi115.stdout:9/382: dread - d1/f72 zero size 2021-11-03T17:14:17.668 INFO:tasks.workunit.client.1.smithi115.stdout:9/383: symlink d1/d18/d2b/d42/d63/l86 0 2021-11-03T17:14:17.668 INFO:tasks.workunit.client.1.smithi115.stdout:9/384: creat d1/d18/d4a/f87 x:0 0 0 2021-11-03T17:14:17.669 INFO:tasks.workunit.client.1.smithi115.stdout:9/385: dread - d1/d1a/d48/d27/d53/f77 zero size 2021-11-03T17:14:17.669 INFO:tasks.workunit.client.1.smithi115.stdout:9/386: mkdir d1/d1a/d48/d27/d53/d83/d88 0 2021-11-03T17:14:17.670 INFO:tasks.workunit.client.1.smithi115.stdout:9/387: chown d1/d1a/d48/d27/d53/f7f 31467671 1 2021-11-03T17:14:17.670 INFO:tasks.workunit.client.1.smithi115.stdout:9/388: dread - d1/d18/f7c zero size 2021-11-03T17:14:17.671 INFO:tasks.workunit.client.1.smithi115.stdout:9/389: rename d1/d18/f7c to d1/d1a/d48/d27/f89 0 2021-11-03T17:14:17.672 INFO:tasks.workunit.client.1.smithi115.stdout:9/390: mkdir d1/d8a 0 2021-11-03T17:14:17.672 INFO:tasks.workunit.client.1.smithi115.stdout:9/391: truncate d1/d18/d4a/f87 740247 0 2021-11-03T17:14:17.673 INFO:tasks.workunit.client.1.smithi115.stdout:9/392: chown d1/l29 623849957 1 2021-11-03T17:14:17.674 INFO:tasks.workunit.client.0.smithi027.stdout:8/481: rename d7/f7d to d7/d10/d21/d41/fa2 0 2021-11-03T17:14:17.676 INFO:tasks.workunit.client.0.smithi027.stdout:0/525: symlink d9/lb3 0 2021-11-03T17:14:17.676 INFO:tasks.workunit.client.0.smithi027.stdout:0/526: readlink d9/da/d64/d70/d6d/d7a/l94 0 2021-11-03T17:14:17.677 INFO:tasks.workunit.client.0.smithi027.stdout:0/527: creat d9/fb4 x:0 0 0 2021-11-03T17:14:17.677 INFO:tasks.workunit.client.0.smithi027.stdout:0/528: mknod d9/da/d64/d70/cb5 1 2021-11-03T17:14:17.678 INFO:tasks.workunit.client.0.smithi027.stdout:0/529: truncate d9/da/d30/f54 1258569 0 2021-11-03T17:14:17.678 INFO:tasks.workunit.client.0.smithi027.stdout:0/530: creat d9/da/d64/d70/fb6 x:0 0 0 2021-11-03T17:14:17.696 INFO:tasks.workunit.client.1.smithi115.stdout:9/393: dwrite d1/d1a/d48/d27/f89 [0,4194304] 0 2021-11-03T17:14:17.697 INFO:tasks.workunit.client.0.smithi027.stdout:8/482: dwrite d7/d10/d3f/d9b/f60 [0,4194304] 0 2021-11-03T17:14:17.697 INFO:tasks.workunit.client.0.smithi027.stdout:8/483: stat d7/d10/d3f/d9b/f1f 0 2021-11-03T17:14:17.697 INFO:tasks.workunit.client.0.smithi027.stdout:8/484: write d7/d10/d3f/d9b/f3c [314135,95615] 0 2021-11-03T17:14:17.698 INFO:tasks.workunit.client.1.smithi115.stdout:5/506: sync 2021-11-03T17:14:17.699 INFO:tasks.workunit.client.0.smithi027.stdout:8/485: creat d7/d10/d21/d41/fa3 x:0 0 0 2021-11-03T17:14:17.702 INFO:tasks.workunit.client.1.smithi115.stdout:5/507: link d1a/d28/d4a/f8b d1a/d28/d4c/d75/d68/da5/fa8 0 2021-11-03T17:14:17.704 INFO:tasks.workunit.client.1.smithi115.stdout:5/508: creat d1a/d28/d4c/d75/d68/da5/fa9 x:0 0 0 2021-11-03T17:14:17.707 INFO:tasks.workunit.client.1.smithi115.stdout:5/509: rmdir d1a/d28/d7b/d4f/d81/d85 0 2021-11-03T17:14:17.713 INFO:tasks.workunit.client.0.smithi027.stdout:8/486: dread d7/d10/d3f/d9b/f60 [0,4194304] 0 2021-11-03T17:14:17.718 INFO:tasks.workunit.client.0.smithi027.stdout:0/531: dwrite d9/da/d64/d70/d6d/fa4 [0,4194304] 0 2021-11-03T17:14:17.719 INFO:tasks.workunit.client.1.smithi115.stdout:9/394: dwrite d1/d1a/d48/d27/d53/f7f [0,4194304] 0 2021-11-03T17:14:17.726 INFO:tasks.workunit.client.1.smithi115.stdout:9/395: write d1/d1a/d48/d27/d2f/d41/f60 [1232235,105874] 0 2021-11-03T17:14:17.729 INFO:tasks.workunit.client.1.smithi115.stdout:9/396: mkdir d1/d18/d4a/d74/d8b 0 2021-11-03T17:14:17.732 INFO:tasks.workunit.client.1.smithi115.stdout:9/397: rename d1/d1a/d48/d27/f55 to d1/d1a/d48/d27/d2f/d41/f8c 0 2021-11-03T17:14:17.742 INFO:tasks.workunit.client.1.smithi115.stdout:9/398: dread d1/f13 [0,4194304] 0 2021-11-03T17:14:17.743 INFO:tasks.workunit.client.1.smithi115.stdout:9/399: chown d1/d18/d2b/d42/l2e 6 1 2021-11-03T17:14:17.743 INFO:tasks.workunit.client.1.smithi115.stdout:9/400: truncate d1/d18/d2b/f35 1118003 0 2021-11-03T17:14:17.743 INFO:tasks.workunit.client.1.smithi115.stdout:5/510: dwrite d1a/d28/d4c/d75/d83/f98 [0,4194304] 0 2021-11-03T17:14:17.744 INFO:tasks.workunit.client.1.smithi115.stdout:9/401: creat d1/d1a/d48/f8d x:0 0 0 2021-11-03T17:14:17.744 INFO:tasks.workunit.client.1.smithi115.stdout:9/402: mknod d1/d18/d4a/c8e 1 2021-11-03T17:14:17.745 INFO:tasks.workunit.client.0.smithi027.stdout:8/487: dwrite d7/d10/fa1 [0,4194304] 0 2021-11-03T17:14:17.745 INFO:tasks.workunit.client.0.smithi027.stdout:0/532: dwrite d9/d12/f17 [0,4194304] 0 2021-11-03T17:14:17.746 INFO:tasks.workunit.client.0.smithi027.stdout:0/533: fdatasync d9/da/d64/f76 0 2021-11-03T17:14:17.746 INFO:tasks.workunit.client.0.smithi027.stdout:0/534: truncate d9/da/d31/fb1 824718 0 2021-11-03T17:14:17.746 INFO:tasks.workunit.client.0.smithi027.stdout:0/535: chown d9/da/d64/d70 0 1 2021-11-03T17:14:17.746 INFO:tasks.workunit.client.0.smithi027.stdout:0/536: chown d9/da/d30/f43 17589 1 2021-11-03T17:14:17.747 INFO:tasks.workunit.client.0.smithi027.stdout:8/488: truncate d7/d10/d21/f25 1656119 0 2021-11-03T17:14:17.748 INFO:tasks.workunit.client.1.smithi115.stdout:9/403: link d1/d18/d4a/f87 d1/d1a/d48/d27/d2f/f8f 0 2021-11-03T17:14:17.749 INFO:tasks.workunit.client.1.smithi115.stdout:5/511: rmdir d1a/d7d 0 2021-11-03T17:14:17.751 INFO:tasks.workunit.client.1.smithi115.stdout:9/404: creat d1/d18/d4a/d54/d3a/f90 x:0 0 0 2021-11-03T17:14:17.753 INFO:tasks.workunit.client.0.smithi027.stdout:0/537: creat d9/fb7 x:0 0 0 2021-11-03T17:14:17.754 INFO:tasks.workunit.client.1.smithi115.stdout:5/512: mkdir d1a/d28/daa 0 2021-11-03T17:14:17.755 INFO:tasks.workunit.client.1.smithi115.stdout:9/405: creat d1/d18/d2b/d42/d63/f91 x:0 0 0 2021-11-03T17:14:17.755 INFO:tasks.workunit.client.1.smithi115.stdout:9/406: readlink d1/d18/l61 0 2021-11-03T17:14:17.756 INFO:tasks.workunit.client.1.smithi115.stdout:9/407: mknod d1/d18/d2b/d42/d63/c92 1 2021-11-03T17:14:17.756 INFO:tasks.workunit.client.1.smithi115.stdout:9/408: fdatasync d1/f25 0 2021-11-03T17:14:17.757 INFO:tasks.workunit.client.0.smithi027.stdout:8/489: dread d7/d10/d4f/f85 [0,4194304] 0 2021-11-03T17:14:17.757 INFO:tasks.workunit.client.0.smithi027.stdout:0/538: write d9/da/f96 [469969,91655] 0 2021-11-03T17:14:17.758 INFO:tasks.workunit.client.1.smithi115.stdout:5/513: getdents d1a/d23 0 2021-11-03T17:14:17.758 INFO:tasks.workunit.client.1.smithi115.stdout:5/514: write d1a/d3d/f62 [2010924,100211] 0 2021-11-03T17:14:17.759 INFO:tasks.workunit.client.1.smithi115.stdout:9/409: creat d1/d1a/d48/d27/d34/f93 x:0 0 0 2021-11-03T17:14:17.759 INFO:tasks.workunit.client.1.smithi115.stdout:9/410: read d1/d18/d2b/d42/f12 [1927260,64116] 0 2021-11-03T17:14:17.759 INFO:tasks.workunit.client.1.smithi115.stdout:5/515: mkdir d1a/d3d/d58/dab 0 2021-11-03T17:14:17.761 INFO:tasks.workunit.client.1.smithi115.stdout:9/411: creat d1/d8a/f94 x:0 0 0 2021-11-03T17:14:17.761 INFO:tasks.workunit.client.1.smithi115.stdout:9/412: chown d1/d18/d2b/d42/d63/l86 40584 1 2021-11-03T17:14:17.761 INFO:tasks.workunit.client.1.smithi115.stdout:5/516: getdents d1a/d28/d4c/d75/d68/da5/da0 0 2021-11-03T17:14:17.762 INFO:tasks.workunit.client.0.smithi027.stdout:8/490: creat d7/d53/fa4 x:0 0 0 2021-11-03T17:14:17.762 INFO:tasks.workunit.client.0.smithi027.stdout:8/491: mkdir d7/d10/d18/d84/da5 0 2021-11-03T17:14:17.762 INFO:tasks.workunit.client.0.smithi027.stdout:8/492: truncate d7/d10/d3f/d9b/f2e 8013113 0 2021-11-03T17:14:17.762 INFO:tasks.workunit.client.0.smithi027.stdout:8/493: chown d7/d2d/l79 2807 1 2021-11-03T17:14:17.763 INFO:tasks.workunit.client.0.smithi027.stdout:8/494: creat d7/d10/d3f/d55/fa6 x:0 0 0 2021-11-03T17:14:17.763 INFO:tasks.workunit.client.1.smithi115.stdout:5/517: unlink d1a/d28/l43 0 2021-11-03T17:14:17.764 INFO:tasks.workunit.client.1.smithi115.stdout:5/518: chown d1a/d28/d7b/d48/d7f 1255593 1 2021-11-03T17:14:17.767 INFO:tasks.workunit.client.0.smithi027.stdout:0/539: dread d9/d12/f28 [0,4194304] 0 2021-11-03T17:14:17.768 INFO:tasks.workunit.client.0.smithi027.stdout:0/540: mknod d9/da/d64/cb8 1 2021-11-03T17:14:17.770 INFO:tasks.workunit.client.0.smithi027.stdout:0/541: rename d9/da/f37 to d9/da/d64/d70/d6d/d9f/fb9 0 2021-11-03T17:14:17.770 INFO:tasks.workunit.client.0.smithi027.stdout:0/542: chown d9/da/f38 136790609 1 2021-11-03T17:14:17.771 INFO:tasks.workunit.client.0.smithi027.stdout:0/543: mkdir d9/da/d30/dac/dba 0 2021-11-03T17:14:17.771 INFO:tasks.workunit.client.0.smithi027.stdout:0/544: chown d9/da/d30/d92 93 1 2021-11-03T17:14:17.772 INFO:tasks.workunit.client.0.smithi027.stdout:0/545: creat d9/da/d64/fbb x:0 0 0 2021-11-03T17:14:17.772 INFO:tasks.workunit.client.0.smithi027.stdout:0/546: chown d9/da/d30/da7 130355 1 2021-11-03T17:14:17.773 INFO:tasks.workunit.client.0.smithi027.stdout:0/547: truncate d9/da/d64/d70/d6d/f6e 786518 0 2021-11-03T17:14:17.773 INFO:tasks.workunit.client.0.smithi027.stdout:0/548: readlink d9/da/d64/l6b 0 2021-11-03T17:14:17.774 INFO:tasks.workunit.client.0.smithi027.stdout:8/495: dread d7/d10/d3f/d9b/f3c [0,4194304] 0 2021-11-03T17:14:17.774 INFO:tasks.workunit.client.0.smithi027.stdout:8/496: write d7/d10/fa1 [5061146,49192] 0 2021-11-03T17:14:17.774 INFO:tasks.workunit.client.0.smithi027.stdout:8/497: dread - d7/d10/d18/f82 zero size 2021-11-03T17:14:17.779 INFO:tasks.workunit.client.1.smithi115.stdout:9/413: dwrite d1/d1a/d48/f8d [0,4194304] 0 2021-11-03T17:14:17.779 INFO:tasks.workunit.client.1.smithi115.stdout:9/414: chown d1/d18/d2b/d42/ff 10925 1 2021-11-03T17:14:17.781 INFO:tasks.workunit.client.1.smithi115.stdout:9/415: creat d1/d1a/d48/d27/d53/f95 x:0 0 0 2021-11-03T17:14:17.782 INFO:tasks.workunit.client.1.smithi115.stdout:9/416: dread - d1/d1a/d48/d27/d53/f95 zero size 2021-11-03T17:14:17.783 INFO:tasks.workunit.client.1.smithi115.stdout:9/417: symlink d1/d8a/l96 0 2021-11-03T17:14:17.783 INFO:tasks.workunit.client.1.smithi115.stdout:9/418: write d1/d1a/f28 [4221914,17933] 0 2021-11-03T17:14:17.785 INFO:tasks.workunit.client.1.smithi115.stdout:9/419: symlink d1/d1a/d48/d27/d53/l97 0 2021-11-03T17:14:17.786 INFO:tasks.workunit.client.1.smithi115.stdout:9/420: readlink d1/l67 0 2021-11-03T17:14:17.786 INFO:tasks.workunit.client.1.smithi115.stdout:9/421: mknod d1/d1a/d48/d27/d2f/d7b/c98 1 2021-11-03T17:14:17.786 INFO:tasks.workunit.client.1.smithi115.stdout:9/422: readlink d1/d1a/d48/d27/d2f/d41/l5e 0 2021-11-03T17:14:17.787 INFO:tasks.workunit.client.1.smithi115.stdout:9/423: creat d1/d1a/d48/d27/d34/f99 x:0 0 0 2021-11-03T17:14:17.787 INFO:tasks.workunit.client.1.smithi115.stdout:9/424: readlink d1/l67 0 2021-11-03T17:14:17.788 INFO:tasks.workunit.client.1.smithi115.stdout:9/425: dread - d1/d18/d4a/d54/d3a/f90 zero size 2021-11-03T17:14:17.788 INFO:tasks.workunit.client.1.smithi115.stdout:9/426: mknod d1/d18/d4a/c9a 1 2021-11-03T17:14:17.788 INFO:tasks.workunit.client.0.smithi027.stdout:0/549: truncate d9/d12/f17 3977050 0 2021-11-03T17:14:17.789 INFO:tasks.workunit.client.0.smithi027.stdout:0/550: stat d9/da/d64/d70/d6d/d9f 0 2021-11-03T17:14:17.791 INFO:tasks.workunit.client.0.smithi027.stdout:0/551: rename d9/da/f9b to d9/da/d30/dac/dba/fbc 0 2021-11-03T17:14:17.796 INFO:tasks.workunit.client.1.smithi115.stdout:5/519: dwrite d1a/d28/d4c/f6e [0,4194304] 0 2021-11-03T17:14:17.797 INFO:tasks.workunit.client.1.smithi115.stdout:9/427: dread d1/d1a/d48/f2a [0,4194304] 0 2021-11-03T17:14:17.798 INFO:tasks.workunit.client.1.smithi115.stdout:9/428: mknod d1/d18/d4a/d54/c9b 1 2021-11-03T17:14:17.798 INFO:tasks.workunit.client.1.smithi115.stdout:9/429: write d1/d18/d4a/d74/f7e [1610126,68355] 0 2021-11-03T17:14:17.798 INFO:tasks.workunit.client.1.smithi115.stdout:9/430: readlink d1/l67 0 2021-11-03T17:14:17.799 INFO:tasks.workunit.client.1.smithi115.stdout:9/431: chown d1/d1a/d48/d27/d34 37 1 2021-11-03T17:14:17.799 INFO:tasks.workunit.client.0.smithi027.stdout:0/552: rename d9/f2a to d9/d12/d8f/fbd 0 2021-11-03T17:14:17.803 INFO:tasks.workunit.client.0.smithi027.stdout:0/553: rename d9/da/d64/d70/fb6 to d9/da/d30/dac/fbe 0 2021-11-03T17:14:17.803 INFO:tasks.workunit.client.0.smithi027.stdout:0/554: fdatasync d9/f42 0 2021-11-03T17:14:17.804 INFO:tasks.workunit.client.0.smithi027.stdout:0/555: creat d9/fbf x:0 0 0 2021-11-03T17:14:17.809 INFO:tasks.workunit.client.0.smithi027.stdout:0/556: symlink d9/da/lc0 0 2021-11-03T17:14:17.813 INFO:tasks.workunit.client.0.smithi027.stdout:0/557: write d9/da/d31/f36 [1328004,81665] 0 2021-11-03T17:14:17.813 INFO:tasks.workunit.client.0.smithi027.stdout:0/558: truncate d9/da/d30/f43 692915 0 2021-11-03T17:14:17.814 INFO:tasks.workunit.client.1.smithi115.stdout:5/520: dwrite d1a/d28/d4c/d75/d83/f88 [0,4194304] 0 2021-11-03T17:14:17.815 INFO:tasks.workunit.client.0.smithi027.stdout:0/559: rename d9/da/f8d to d9/fc1 0 2021-11-03T17:14:17.821 INFO:tasks.workunit.client.0.smithi027.stdout:0/560: rename d9/l4a to d9/d12/d8f/lc2 0 2021-11-03T17:14:17.821 INFO:tasks.workunit.client.0.smithi027.stdout:0/561: chown d9/da 1014813 1 2021-11-03T17:14:17.821 INFO:tasks.workunit.client.0.smithi027.stdout:0/562: dread - d9/da/d30/da7/f8a zero size 2021-11-03T17:14:17.832 INFO:tasks.workunit.client.1.smithi115.stdout:5/521: dwrite d1a/d28/d7b/f74 [0,4194304] 0 2021-11-03T17:14:17.835 INFO:tasks.workunit.client.1.smithi115.stdout:5/522: link l14 d1a/d28/d4c/lac 0 2021-11-03T17:14:17.838 INFO:tasks.workunit.client.1.smithi115.stdout:5/523: creat d1a/d28/d78/fad x:0 0 0 2021-11-03T17:14:17.847 INFO:tasks.workunit.client.1.smithi115.stdout:5/524: dread d1a/d28/f2c [0,4194304] 0 2021-11-03T17:14:17.866 INFO:tasks.workunit.client.1.smithi115.stdout:5/525: dwrite d1a/d28/d7b/d48/d7f/f86 [0,4194304] 0 2021-11-03T17:14:17.874 INFO:tasks.workunit.client.1.smithi115.stdout:5/526: dread d1a/d28/d4c/d75/d83/f88 [0,4194304] 0 2021-11-03T17:14:17.878 INFO:tasks.workunit.client.0.smithi027.stdout:1/474: sync 2021-11-03T17:14:17.879 INFO:tasks.workunit.client.0.smithi027.stdout:1/475: fsync d6/d11/dac/d79/d31/f34 0 2021-11-03T17:14:17.879 INFO:tasks.workunit.client.0.smithi027.stdout:1/476: fdatasync d6/de/d13/d15/f2f 0 2021-11-03T17:14:17.881 INFO:tasks.workunit.client.0.smithi027.stdout:1/477: creat d6/d11/dac/d79/fbb x:0 0 0 2021-11-03T17:14:17.881 INFO:tasks.workunit.client.0.smithi027.stdout:1/478: chown d6/d11/dac/d79/f59 1260 1 2021-11-03T17:14:17.892 INFO:tasks.workunit.client.1.smithi115.stdout:5/527: dwrite d1a/d28/d34/f66 [0,4194304] 0 2021-11-03T17:14:17.895 INFO:tasks.workunit.client.1.smithi115.stdout:5/528: mkdir d1a/d28/d4c/d75/dae 0 2021-11-03T17:14:17.896 INFO:tasks.workunit.client.0.smithi027.stdout:1/479: dread d6/de/d13/d15/f42 [0,4194304] 0 2021-11-03T17:14:17.917 INFO:tasks.workunit.client.1.smithi115.stdout:1/370: sync 2021-11-03T17:14:17.920 INFO:tasks.workunit.client.1.smithi115.stdout:1/371: mkdir d7/dc/d2a/d73 0 2021-11-03T17:14:17.920 INFO:tasks.workunit.client.1.smithi115.stdout:1/372: chown d7/dc/d1a/f3d 2021548877 1 2021-11-03T17:14:17.922 INFO:tasks.workunit.client.1.smithi115.stdout:1/373: rmdir d7/dc/dd/d24 39 2021-11-03T17:14:17.922 INFO:tasks.workunit.client.1.smithi115.stdout:7/307: sync 2021-11-03T17:14:17.922 INFO:tasks.workunit.client.1.smithi115.stdout:6/349: sync 2021-11-03T17:14:17.923 INFO:tasks.workunit.client.1.smithi115.stdout:7/308: mknod d3/d4c/c60 1 2021-11-03T17:14:17.923 INFO:tasks.workunit.client.1.smithi115.stdout:6/350: fdatasync d0/d1/d2/f58 0 2021-11-03T17:14:17.923 INFO:tasks.workunit.client.1.smithi115.stdout:3/455: sync 2021-11-03T17:14:17.924 INFO:tasks.workunit.client.1.smithi115.stdout:3/456: truncate d13/d51/d2e/f52 4212734 0 2021-11-03T17:14:17.924 INFO:tasks.workunit.client.1.smithi115.stdout:1/374: getdents d7/dc/dd/d24/d26 0 2021-11-03T17:14:17.924 INFO:tasks.workunit.client.1.smithi115.stdout:1/375: chown d7/d38 419 1 2021-11-03T17:14:17.926 INFO:tasks.workunit.client.0.smithi027.stdout:1/480: dwrite d6/de/d13/d15/d43/f48 [0,4194304] 0 2021-11-03T17:14:17.926 INFO:tasks.workunit.client.0.smithi027.stdout:1/481: mknod d6/d11/dac/d79/d25/d28/d32/cbc 1 2021-11-03T17:14:17.926 INFO:tasks.workunit.client.0.smithi027.stdout:1/482: mkdir d6/de/d13/d15/d4e/dbd 0 2021-11-03T17:14:17.926 INFO:tasks.workunit.client.1.smithi115.stdout:5/529: dwrite d1a/d3d/d89/f67 [0,4194304] 0 2021-11-03T17:14:17.927 INFO:tasks.workunit.client.1.smithi115.stdout:5/530: chown d1a/d28/d4c/d75/d68/f6c 1 1 2021-11-03T17:14:17.927 INFO:tasks.workunit.client.1.smithi115.stdout:6/351: creat d0/d1/d10/d13/f72 x:0 0 0 2021-11-03T17:14:17.927 INFO:tasks.workunit.client.1.smithi115.stdout:6/352: write d0/d1/d6a/f6e [107374,83772] 0 2021-11-03T17:14:17.928 INFO:tasks.workunit.client.1.smithi115.stdout:3/457: symlink d13/d2c/l97 0 2021-11-03T17:14:17.930 INFO:tasks.workunit.client.1.smithi115.stdout:3/458: write d13/d35/f3a [4031076,52766] 0 2021-11-03T17:14:17.930 INFO:tasks.workunit.client.1.smithi115.stdout:1/376: creat d7/dc/dd/d24/d43/f74 x:0 0 0 2021-11-03T17:14:17.931 INFO:tasks.workunit.client.1.smithi115.stdout:1/377: dread - d7/dc/dd/d24/d43/f74 zero size 2021-11-03T17:14:17.935 INFO:tasks.workunit.client.1.smithi115.stdout:6/353: unlink d0/d1/d6a/f6e 0 2021-11-03T17:14:17.935 INFO:tasks.workunit.client.1.smithi115.stdout:6/354: mknod d0/d1/d10/d13/d1c/c73 1 2021-11-03T17:14:17.936 INFO:tasks.workunit.client.1.smithi115.stdout:3/459: link d13/d50/f42 d13/d3b/f98 0 2021-11-03T17:14:17.936 INFO:tasks.workunit.client.1.smithi115.stdout:3/460: stat d13/d35/l44 0 2021-11-03T17:14:17.936 INFO:tasks.workunit.client.1.smithi115.stdout:1/378: creat d7/dc/dd/d42/f75 x:0 0 0 2021-11-03T17:14:17.937 INFO:tasks.workunit.client.1.smithi115.stdout:1/379: mknod d7/d38/d59/c76 1 2021-11-03T17:14:17.939 INFO:tasks.workunit.client.1.smithi115.stdout:3/461: mkdir d13/d2a/d8a/d99 0 2021-11-03T17:14:17.941 INFO:tasks.workunit.client.1.smithi115.stdout:6/355: mkdir d0/d74 0 2021-11-03T17:14:17.941 INFO:tasks.workunit.client.1.smithi115.stdout:6/356: write d0/d1/d2/f4a [194234,55516] 0 2021-11-03T17:14:17.943 INFO:tasks.workunit.client.1.smithi115.stdout:3/462: rename d13/d51/f2d to d13/d2a/f9a 0 2021-11-03T17:14:17.945 INFO:tasks.workunit.client.1.smithi115.stdout:3/463: symlink d13/d35/d54/l9b 0 2021-11-03T17:14:17.946 INFO:tasks.workunit.client.1.smithi115.stdout:7/309: dwrite d3/f14 [0,4194304] 0 2021-11-03T17:14:17.946 INFO:tasks.workunit.client.1.smithi115.stdout:5/531: dwrite d1a/d28/d7b/f77 [0,4194304] 0 2021-11-03T17:14:17.946 INFO:tasks.workunit.client.1.smithi115.stdout:5/532: chown d1a/d28/d78/fad 672 1 2021-11-03T17:14:17.948 INFO:tasks.workunit.client.1.smithi115.stdout:6/357: rename d0/d1/d2e/l68 to d0/d1/d10/d13/d1c/d1e/l75 0 2021-11-03T17:14:17.950 INFO:tasks.workunit.client.1.smithi115.stdout:3/464: symlink d13/d35/l9c 0 2021-11-03T17:14:17.954 INFO:tasks.workunit.client.1.smithi115.stdout:5/533: mkdir d1a/d3d/d58/dab/daf 0 2021-11-03T17:14:17.958 INFO:tasks.workunit.client.1.smithi115.stdout:5/534: symlink d1a/d28/d7b/lb0 0 2021-11-03T17:14:17.958 INFO:tasks.workunit.client.1.smithi115.stdout:3/465: dread d13/d2a/f9a [0,4194304] 0 2021-11-03T17:14:17.963 INFO:tasks.workunit.client.1.smithi115.stdout:3/466: creat d13/d51/d40/f9d x:0 0 0 2021-11-03T17:14:17.963 INFO:tasks.workunit.client.1.smithi115.stdout:3/467: fdatasync d13/d2a/d53/f8c 0 2021-11-03T17:14:17.963 INFO:tasks.workunit.client.1.smithi115.stdout:3/468: write d13/d51/d2e/f37 [4043686,88888] 0 2021-11-03T17:14:17.964 INFO:tasks.workunit.client.1.smithi115.stdout:7/310: dwrite d3/da/d26/d35/f5f [0,4194304] 0 2021-11-03T17:14:17.964 INFO:tasks.workunit.client.1.smithi115.stdout:1/380: dwrite d7/dc/dd/d24/d4b/f72 [0,4194304] 0 2021-11-03T17:14:17.968 INFO:tasks.workunit.client.1.smithi115.stdout:7/311: unlink d3/da/d26/d35/f3e 0 2021-11-03T17:14:17.972 INFO:tasks.workunit.client.1.smithi115.stdout:1/381: link d7/dc/d1a/f6b d7/f77 0 2021-11-03T17:14:17.973 INFO:tasks.workunit.client.1.smithi115.stdout:7/312: symlink d3/da/l61 0 2021-11-03T17:14:17.974 INFO:tasks.workunit.client.1.smithi115.stdout:7/313: truncate d3/d4c/f5b 395844 0 2021-11-03T17:14:17.974 INFO:tasks.workunit.client.1.smithi115.stdout:7/314: stat d3/da/d26/d35/f3d 0 2021-11-03T17:14:17.974 INFO:tasks.workunit.client.1.smithi115.stdout:1/382: mkdir d7/dc/dd/d24/d43/d78 0 2021-11-03T17:14:17.975 INFO:tasks.workunit.client.1.smithi115.stdout:5/535: write d1a/d28/d4c/f6e [388606,44087] 0 2021-11-03T17:14:17.975 INFO:tasks.workunit.client.1.smithi115.stdout:7/315: creat d3/da/f62 x:0 0 0 2021-11-03T17:14:17.978 INFO:tasks.workunit.client.1.smithi115.stdout:1/383: rename d7/dc/d1a/f70 to d7/dc/dd/d24/d4d/f79 0 2021-11-03T17:14:17.979 INFO:tasks.workunit.client.1.smithi115.stdout:5/536: truncate d1a/d28/d7b/d4f/d81/fa6 470009 0 2021-11-03T17:14:17.980 INFO:tasks.workunit.client.1.smithi115.stdout:7/316: getdents d3/d4c 0 2021-11-03T17:14:17.981 INFO:tasks.workunit.client.1.smithi115.stdout:1/384: write d7/dc/dd/d24/d4b/d55/f66 [1616208,30490] 0 2021-11-03T17:14:17.981 INFO:tasks.workunit.client.1.smithi115.stdout:1/385: chown d7/dc/ff 128327 1 2021-11-03T17:14:17.982 INFO:tasks.workunit.client.1.smithi115.stdout:5/537: creat d1a/d28/d7b/d48/fb1 x:0 0 0 2021-11-03T17:14:17.982 INFO:tasks.workunit.client.1.smithi115.stdout:6/358: dwrite d0/f2a [0,4194304] 0 2021-11-03T17:14:17.983 INFO:tasks.workunit.client.1.smithi115.stdout:6/359: read - d0/d1/d2e/f31 zero size 2021-11-03T17:14:17.983 INFO:tasks.workunit.client.1.smithi115.stdout:6/360: chown d0 8370 1 2021-11-03T17:14:17.983 INFO:tasks.workunit.client.1.smithi115.stdout:3/469: dwrite d13/f27 [0,4194304] 0 2021-11-03T17:14:17.983 INFO:tasks.workunit.client.1.smithi115.stdout:3/470: readlink d13/d50/l4b 0 2021-11-03T17:14:17.984 INFO:tasks.workunit.client.1.smithi115.stdout:7/317: symlink d3/d4c/l63 0 2021-11-03T17:14:17.985 INFO:tasks.workunit.client.1.smithi115.stdout:1/386: rename d7/dc/dd/d42/f61 to d7/dc/dd/d42/f7a 0 2021-11-03T17:14:17.985 INFO:tasks.workunit.client.1.smithi115.stdout:1/387: truncate d7/dc/dd/d24/d4d/d60/f6d 209870 0 2021-11-03T17:14:17.994 INFO:tasks.workunit.client.1.smithi115.stdout:3/471: write d13/f1f [1351248,82503] 0 2021-11-03T17:14:17.995 INFO:tasks.workunit.client.1.smithi115.stdout:7/318: creat d3/da/f64 x:0 0 0 2021-11-03T17:14:17.996 INFO:tasks.workunit.client.1.smithi115.stdout:5/538: dread d1a/d28/f31 [0,4194304] 0 2021-11-03T17:14:17.997 INFO:tasks.workunit.client.1.smithi115.stdout:7/319: write d3/f25 [1271469,117316] 0 2021-11-03T17:14:17.999 INFO:tasks.workunit.client.1.smithi115.stdout:1/388: creat d7/dc/dd/d24/d4d/f7b x:0 0 0 2021-11-03T17:14:18.000 INFO:tasks.workunit.client.1.smithi115.stdout:6/361: symlink d0/l76 0 2021-11-03T17:14:18.000 INFO:tasks.workunit.client.1.smithi115.stdout:6/362: mknod d0/d1/d2/d3f/c77 1 2021-11-03T17:14:18.000 INFO:tasks.workunit.client.1.smithi115.stdout:6/363: write d0/d1/d2e/f60 [124397,110475] 0 2021-11-03T17:14:18.000 INFO:tasks.workunit.client.1.smithi115.stdout:6/364: truncate d0/d1/d2e/f60 693104 0 2021-11-03T17:14:18.001 INFO:tasks.workunit.client.1.smithi115.stdout:6/365: read - d0/d1/d10/f53 zero size 2021-11-03T17:14:18.002 INFO:tasks.workunit.client.1.smithi115.stdout:5/539: truncate d1a/d28/d34/f2a 522641 0 2021-11-03T17:14:18.004 INFO:tasks.workunit.client.1.smithi115.stdout:7/320: rename d3/da/f42 to d3/da/d26/f65 0 2021-11-03T17:14:18.005 INFO:tasks.workunit.client.1.smithi115.stdout:7/321: fdatasync d3/da/d26/d35/f3d 0 2021-11-03T17:14:18.009 INFO:tasks.workunit.client.1.smithi115.stdout:1/389: rmdir d7 39 2021-11-03T17:14:18.011 INFO:tasks.workunit.client.1.smithi115.stdout:7/322: creat d3/da/d26/d52/f66 x:0 0 0 2021-11-03T17:14:18.012 INFO:tasks.workunit.client.1.smithi115.stdout:1/390: creat d7/f7c x:0 0 0 2021-11-03T17:14:18.012 INFO:tasks.workunit.client.1.smithi115.stdout:1/391: fdatasync d7/d38/f5c 0 2021-11-03T17:14:18.013 INFO:tasks.workunit.client.1.smithi115.stdout:1/392: mknod d7/dc/dd/d3e/c7d 1 2021-11-03T17:14:18.017 INFO:tasks.workunit.client.1.smithi115.stdout:1/393: creat d7/dc/dd/d24/d43/f7e x:0 0 0 2021-11-03T17:14:18.024 INFO:tasks.workunit.client.1.smithi115.stdout:3/472: dwrite d13/d50/f8e [0,4194304] 0 2021-11-03T17:14:18.026 INFO:tasks.workunit.client.1.smithi115.stdout:5/540: dwrite d1a/d3d/f3b [0,4194304] 0 2021-11-03T17:14:18.026 INFO:tasks.workunit.client.1.smithi115.stdout:5/541: fsync d1a/d28/d4c/d75/d68/da5/fa9 0 2021-11-03T17:14:18.026 INFO:tasks.workunit.client.1.smithi115.stdout:6/366: dwrite d0/d1/d10/d13/d59/f67 [0,4194304] 0 2021-11-03T17:14:18.029 INFO:tasks.workunit.client.1.smithi115.stdout:5/542: write d1a/d28/f31 [14585,27109] 0 2021-11-03T17:14:18.031 INFO:tasks.workunit.client.1.smithi115.stdout:6/367: symlink d0/d1/d2/d5d/d61/l78 0 2021-11-03T17:14:18.034 INFO:tasks.workunit.client.1.smithi115.stdout:6/368: rename d0/d1/d10/d13/f25 to d0/f79 0 2021-11-03T17:14:18.035 INFO:tasks.workunit.client.1.smithi115.stdout:3/473: dread d13/d51/d2e/f32 [0,4194304] 0 2021-11-03T17:14:18.038 INFO:tasks.workunit.client.1.smithi115.stdout:6/369: rename d0/l76 to d0/d1/d10/d13/d59/l7a 0 2021-11-03T17:14:18.040 INFO:tasks.workunit.client.1.smithi115.stdout:1/394: dwrite d7/dc/dd/d24/d26/f3b [0,4194304] 0 2021-11-03T17:14:18.047 INFO:tasks.workunit.client.1.smithi115.stdout:6/370: dread d0/d1/d2/f1f [4194304,4194304] 0 2021-11-03T17:14:18.048 INFO:tasks.workunit.client.1.smithi115.stdout:6/371: write d0/d1/f19 [1418776,37762] 0 2021-11-03T17:14:18.067 INFO:tasks.workunit.client.1.smithi115.stdout:1/395: dread d7/dc/dd/d24/d4b/f72 [0,4194304] 0 2021-11-03T17:14:18.069 INFO:tasks.workunit.client.1.smithi115.stdout:1/396: getdents d7/dc/d2a 0 2021-11-03T17:14:18.070 INFO:tasks.workunit.client.1.smithi115.stdout:1/397: mkdir d7/dc/d2a/d73/d7f 0 2021-11-03T17:14:18.071 INFO:tasks.workunit.client.1.smithi115.stdout:1/398: mknod d7/dc/d1a/c80 1 2021-11-03T17:14:18.073 INFO:tasks.workunit.client.1.smithi115.stdout:7/323: dwrite d3/f25 [0,4194304] 0 2021-11-03T17:14:18.073 INFO:tasks.workunit.client.1.smithi115.stdout:7/324: readlink d3/l7 0 2021-11-03T17:14:18.074 INFO:tasks.workunit.client.1.smithi115.stdout:1/399: dread d7/dc/f2f [0,4194304] 0 2021-11-03T17:14:18.074 INFO:tasks.workunit.client.1.smithi115.stdout:5/543: dwrite d1a/d3d/f42 [0,4194304] 0 2021-11-03T17:14:18.075 INFO:tasks.workunit.client.1.smithi115.stdout:5/544: truncate d1a/d28/d4c/d75/d68/da5/fa9 268001 0 2021-11-03T17:14:18.075 INFO:tasks.workunit.client.1.smithi115.stdout:5/545: chown d1a/d3d/l46 98702271 1 2021-11-03T17:14:18.075 INFO:tasks.workunit.client.1.smithi115.stdout:7/325: creat d3/da/d18/f67 x:0 0 0 2021-11-03T17:14:18.078 INFO:tasks.workunit.client.1.smithi115.stdout:1/400: truncate d7/dc/f16 667687 0 2021-11-03T17:14:18.078 INFO:tasks.workunit.client.1.smithi115.stdout:5/546: creat d1a/d3d/d58/fb2 x:0 0 0 2021-11-03T17:14:18.079 INFO:tasks.workunit.client.1.smithi115.stdout:5/547: truncate d1a/d3d/d89/f8f 970335 0 2021-11-03T17:14:18.079 INFO:tasks.workunit.client.1.smithi115.stdout:5/548: chown l10 1 1 2021-11-03T17:14:18.080 INFO:tasks.workunit.client.1.smithi115.stdout:7/326: mknod d3/da/d26/d35/d53/c68 1 2021-11-03T17:14:18.080 INFO:tasks.workunit.client.1.smithi115.stdout:7/327: chown d3/f1e 15 1 2021-11-03T17:14:18.080 INFO:tasks.workunit.client.1.smithi115.stdout:7/328: write d3/da/d18/f67 [363320,29722] 0 2021-11-03T17:14:18.081 INFO:tasks.workunit.client.1.smithi115.stdout:1/401: creat d7/dc/dd/d24/d4d/d60/f81 x:0 0 0 2021-11-03T17:14:18.081 INFO:tasks.workunit.client.1.smithi115.stdout:3/474: dwrite d13/d51/f45 [4194304,4194304] 0 2021-11-03T17:14:18.081 INFO:tasks.workunit.client.1.smithi115.stdout:6/372: dwrite d0/f40 [0,4194304] 0 2021-11-03T17:14:18.081 INFO:tasks.workunit.client.1.smithi115.stdout:6/373: chown d0/d1/l3a 25093431 1 2021-11-03T17:14:18.081 INFO:tasks.workunit.client.1.smithi115.stdout:7/329: creat d3/da/f69 x:0 0 0 2021-11-03T17:14:18.082 INFO:tasks.workunit.client.1.smithi115.stdout:3/475: truncate d13/f1e 1181550 0 2021-11-03T17:14:18.083 INFO:tasks.workunit.client.1.smithi115.stdout:3/476: write d13/d51/f41 [4842361,77328] 0 2021-11-03T17:14:18.083 INFO:tasks.workunit.client.1.smithi115.stdout:7/330: rename d3/da/d26/f65 to d3/f6a 0 2021-11-03T17:14:18.084 INFO:tasks.workunit.client.1.smithi115.stdout:7/331: getdents d3/d4c 0 2021-11-03T17:14:18.089 INFO:tasks.workunit.client.1.smithi115.stdout:1/402: dread d7/dc/dd/d24/d43/f6f [0,4194304] 0 2021-11-03T17:14:18.089 INFO:tasks.workunit.client.1.smithi115.stdout:1/403: read - d7/dc/dd/d24/d43/f7e zero size 2021-11-03T17:14:18.091 INFO:tasks.workunit.client.1.smithi115.stdout:1/404: symlink d7/dc/d1a/l82 0 2021-11-03T17:14:18.093 INFO:tasks.workunit.client.1.smithi115.stdout:7/332: dread d3/da/d26/d35/f3f [0,4194304] 0 2021-11-03T17:14:18.107 INFO:tasks.workunit.client.1.smithi115.stdout:5/549: dwrite d1a/d28/d34/f33 [4194304,4194304] 0 2021-11-03T17:14:18.121 INFO:tasks.workunit.client.1.smithi115.stdout:1/405: dwrite d7/dc/dd/f5a [0,4194304] 0 2021-11-03T17:14:18.122 INFO:tasks.workunit.client.1.smithi115.stdout:3/477: dwrite d13/d35/f36 [0,4194304] 0 2021-11-03T17:14:18.123 INFO:tasks.workunit.client.1.smithi115.stdout:6/374: dwrite d0/d1/d10/d13/f72 [0,4194304] 0 2021-11-03T17:14:18.124 INFO:tasks.workunit.client.1.smithi115.stdout:7/333: dwrite d3/da/d26/d35/d53/f58 [0,4194304] 0 2021-11-03T17:14:18.125 INFO:tasks.workunit.client.1.smithi115.stdout:1/406: mkdir d7/d38/d59/d83 0 2021-11-03T17:14:18.128 INFO:tasks.workunit.client.1.smithi115.stdout:6/375: dread d0/d1/d2/d3f/f4b [0,4194304] 0 2021-11-03T17:14:18.128 INFO:tasks.workunit.client.1.smithi115.stdout:6/376: chown d0/d1/d2/f58 137969 1 2021-11-03T17:14:18.132 INFO:tasks.workunit.client.1.smithi115.stdout:6/377: dread d0/d1/d10/f3b [0,4194304] 0 2021-11-03T17:14:18.136 INFO:tasks.workunit.client.1.smithi115.stdout:5/550: dwrite d1a/d3d/d89/f67 [0,4194304] 0 2021-11-03T17:14:18.136 INFO:tasks.workunit.client.1.smithi115.stdout:5/551: fdatasync d1a/f1f 0 2021-11-03T17:14:18.136 INFO:tasks.workunit.client.1.smithi115.stdout:5/552: dread - d1a/d3d/d58/f7c zero size 2021-11-03T17:14:18.136 INFO:tasks.workunit.client.1.smithi115.stdout:5/553: mknod d1a/d28/d78/cb3 1 2021-11-03T17:14:18.137 INFO:tasks.workunit.client.1.smithi115.stdout:5/554: readlink l14 0 2021-11-03T17:14:18.145 INFO:tasks.workunit.client.1.smithi115.stdout:3/478: dwrite d13/d3b/f58 [0,4194304] 0 2021-11-03T17:14:18.147 INFO:tasks.workunit.client.1.smithi115.stdout:1/407: mkdir d7/d38/d84 0 2021-11-03T17:14:18.149 INFO:tasks.workunit.client.1.smithi115.stdout:6/378: truncate d0/d1/d10/d13/d1c/f21 2736124 0 2021-11-03T17:14:18.150 INFO:tasks.workunit.client.1.smithi115.stdout:7/334: link d3/da/l61 d3/da/d26/d33/l6b 0 2021-11-03T17:14:18.150 INFO:tasks.workunit.client.1.smithi115.stdout:7/335: chown d3/da/d26/d33/l38 27918055 1 2021-11-03T17:14:18.151 INFO:tasks.workunit.client.1.smithi115.stdout:5/555: mkdir d1a/d23/db4 0 2021-11-03T17:14:18.152 INFO:tasks.workunit.client.1.smithi115.stdout:3/479: mkdir d13/d50/d9e 0 2021-11-03T17:14:18.152 INFO:tasks.workunit.client.1.smithi115.stdout:5/556: read d1a/d28/d4c/d63/f9c [4059078,23763] 0 2021-11-03T17:14:18.153 INFO:tasks.workunit.client.1.smithi115.stdout:1/408: creat d7/dc/dd/d24/d26/f85 x:0 0 0 2021-11-03T17:14:18.154 INFO:tasks.workunit.client.1.smithi115.stdout:1/409: stat d7/dc/dd/d24/d4d/f79 0 2021-11-03T17:14:18.154 INFO:tasks.workunit.client.1.smithi115.stdout:6/379: symlink d0/d1/d10/d13/d1c/d1e/l7b 0 2021-11-03T17:14:18.156 INFO:tasks.workunit.client.1.smithi115.stdout:7/336: symlink d3/da/d26/d35/d53/l6c 0 2021-11-03T17:14:18.156 INFO:tasks.workunit.client.1.smithi115.stdout:7/337: chown d3/da/d26/d35/d53/l6c 987517259 1 2021-11-03T17:14:18.156 INFO:tasks.workunit.client.1.smithi115.stdout:7/338: chown d3/da/f69 905302 1 2021-11-03T17:14:18.157 INFO:tasks.workunit.client.1.smithi115.stdout:3/480: symlink d13/d2a/l9f 0 2021-11-03T17:14:18.157 INFO:tasks.workunit.client.0.smithi027.stdout:5/648: sync 2021-11-03T17:14:18.158 INFO:tasks.workunit.client.0.smithi027.stdout:5/649: mknod d2/d12/dad/da9/cd3 1 2021-11-03T17:14:18.162 INFO:tasks.workunit.client.0.smithi027.stdout:5/650: dread d2/d4/d4e/d67/d20/f73 [0,4194304] 0 2021-11-03T17:14:18.162 INFO:tasks.workunit.client.0.smithi027.stdout:5/651: chown d2/d2c/d3c/d58/f5e 27 1 2021-11-03T17:14:18.162 INFO:tasks.workunit.client.0.smithi027.stdout:5/652: chown d2/d4/d4e/l77 2 1 2021-11-03T17:14:18.163 INFO:tasks.workunit.client.0.smithi027.stdout:5/653: mknod d2/d12/d7d/da7/cd4 1 2021-11-03T17:14:18.164 INFO:tasks.workunit.client.0.smithi027.stdout:5/654: truncate d2/d4/d4e/d67/f7f 1911385 0 2021-11-03T17:14:18.166 INFO:tasks.workunit.client.0.smithi027.stdout:5/655: write d2/d2c/d9b/fcd [3309410,129867] 0 2021-11-03T17:14:18.168 INFO:tasks.workunit.client.1.smithi115.stdout:6/380: mkdir d0/d1/d6a/d7c 0 2021-11-03T17:14:18.168 INFO:tasks.workunit.client.0.smithi027.stdout:5/656: rename d2/d12/f44 to d2/d4/fd5 0 2021-11-03T17:14:18.170 INFO:tasks.workunit.client.1.smithi115.stdout:5/557: rename d1a/d3d/d89 to d1a/d28/d4a/d65/db5 0 2021-11-03T17:14:18.170 INFO:tasks.workunit.client.0.smithi027.stdout:5/657: mkdir d2/d2c/d3c/d48/d65/dd6 0 2021-11-03T17:14:18.171 INFO:tasks.workunit.client.1.smithi115.stdout:5/558: write d1a/d3d/f42 [902817,4391] 0 2021-11-03T17:14:18.173 INFO:tasks.workunit.client.0.smithi027.stdout:5/658: symlink d2/d2c/d3c/d48/d65/d80/ld7 0 2021-11-03T17:14:18.174 INFO:tasks.workunit.client.0.smithi027.stdout:5/659: getdents d2/d4/d61 0 2021-11-03T17:14:18.175 INFO:tasks.workunit.client.0.smithi027.stdout:5/660: read d2/d4/d4e/d67/d20/f73 [2583081,38847] 0 2021-11-03T17:14:18.186 INFO:tasks.workunit.client.1.smithi115.stdout:5/559: dread d1a/d28/f91 [0,4194304] 0 2021-11-03T17:14:18.187 INFO:tasks.workunit.client.1.smithi115.stdout:5/560: truncate f11 712412 0 2021-11-03T17:14:18.187 INFO:tasks.workunit.client.1.smithi115.stdout:5/561: fdatasync d1a/d28/d7b/d4f/d81/fa6 0 2021-11-03T17:14:18.188 INFO:tasks.workunit.client.1.smithi115.stdout:5/562: write d1a/d28/d4c/d75/d68/da5/fa9 [568240,16687] 0 2021-11-03T17:14:18.188 INFO:tasks.workunit.client.1.smithi115.stdout:5/563: creat d1a/d28/d7b/d48/d7f/fb6 x:0 0 0 2021-11-03T17:14:18.190 INFO:tasks.workunit.client.1.smithi115.stdout:0/489: sync 2021-11-03T17:14:18.191 INFO:tasks.workunit.client.1.smithi115.stdout:2/403: sync 2021-11-03T17:14:18.191 INFO:tasks.workunit.client.1.smithi115.stdout:4/567: sync 2021-11-03T17:14:18.191 INFO:tasks.workunit.client.1.smithi115.stdout:4/568: fdatasync d7/dc/f13 0 2021-11-03T17:14:18.196 INFO:tasks.workunit.client.1.smithi115.stdout:4/569: symlink d7/dc/d1a/d21/db1/d56/d42/daf/lc7 0 2021-11-03T17:14:18.196 INFO:tasks.workunit.client.1.smithi115.stdout:4/570: stat d7/lb 0 2021-11-03T17:14:18.203 INFO:tasks.workunit.client.1.smithi115.stdout:4/571: dread d7/d7a/d83/f64 [0,4194304] 0 2021-11-03T17:14:18.204 INFO:tasks.workunit.client.1.smithi115.stdout:4/572: dread - d7/dc/d6a/d3c/fa8 zero size 2021-11-03T17:14:18.205 INFO:tasks.workunit.client.1.smithi115.stdout:2/404: dread d4/d6/d56/d6c/f39 [0,4194304] 0 2021-11-03T17:14:18.206 INFO:tasks.workunit.client.1.smithi115.stdout:1/410: dwrite d7/dc/f2f [0,4194304] 0 2021-11-03T17:14:18.207 INFO:tasks.workunit.client.1.smithi115.stdout:0/490: dread f8 [0,4194304] 0 2021-11-03T17:14:18.207 INFO:tasks.workunit.client.1.smithi115.stdout:0/491: write d9/d15/d78/d80/d82/f91 [468688,68014] 0 2021-11-03T17:14:18.207 INFO:tasks.workunit.client.1.smithi115.stdout:0/492: mknod d9/d37/ca0 1 2021-11-03T17:14:18.207 INFO:tasks.workunit.client.1.smithi115.stdout:0/493: write d9/d15/d78/d80/d82/f91 [1226665,74877] 0 2021-11-03T17:14:18.208 INFO:tasks.workunit.client.1.smithi115.stdout:2/405: creat d4/d6/d13/d6d/d7f/f8e x:0 0 0 2021-11-03T17:14:18.209 INFO:tasks.workunit.client.1.smithi115.stdout:1/411: getdents d7/d38 0 2021-11-03T17:14:18.209 INFO:tasks.workunit.client.1.smithi115.stdout:1/412: write d7/dc/d1a/f35 [852972,47345] 0 2021-11-03T17:14:18.210 INFO:tasks.workunit.client.1.smithi115.stdout:1/413: dread - d7/dc/dd/d24/d4d/f69 zero size 2021-11-03T17:14:18.211 INFO:tasks.workunit.client.1.smithi115.stdout:0/494: creat d9/db/d89/fa1 x:0 0 0 2021-11-03T17:14:18.211 INFO:tasks.workunit.client.1.smithi115.stdout:0/495: truncate d9/d15/d78/d80/f87 563700 0 2021-11-03T17:14:18.212 INFO:tasks.workunit.client.0.smithi027.stdout:5/661: dwrite d2/d4/d4e/d67/f82 [0,4194304] 0 2021-11-03T17:14:18.212 INFO:tasks.workunit.client.0.smithi027.stdout:5/662: dread - d2/d2c/d9b/d9d/fbe zero size 2021-11-03T17:14:18.212 INFO:tasks.workunit.client.1.smithi115.stdout:2/406: creat d4/d6/d56/d6c/d34/d42/f8f x:0 0 0 2021-11-03T17:14:18.213 INFO:tasks.workunit.client.1.smithi115.stdout:0/496: symlink d9/d15/d2c/d46/d63/la2 0 2021-11-03T17:14:18.214 INFO:tasks.workunit.client.0.smithi027.stdout:5/663: write d2/d4/f43 [2898139,6575] 0 2021-11-03T17:14:18.215 INFO:tasks.workunit.client.1.smithi115.stdout:7/339: dwrite d3/f47 [0,4194304] 0 2021-11-03T17:14:18.216 INFO:tasks.workunit.client.1.smithi115.stdout:5/564: dwrite d1a/d3d/f3b [0,4194304] 0 2021-11-03T17:14:18.216 INFO:tasks.workunit.client.1.smithi115.stdout:7/340: creat d3/da/d18/f6d x:0 0 0 2021-11-03T17:14:18.217 INFO:tasks.workunit.client.1.smithi115.stdout:7/341: chown d3/da/d18/l2d 783860 1 2021-11-03T17:14:18.217 INFO:tasks.workunit.client.1.smithi115.stdout:7/342: write d3/d4c/f5b [266005,119787] 0 2021-11-03T17:14:18.217 INFO:tasks.workunit.client.1.smithi115.stdout:7/343: write d3/da/d26/d33/f59 [1429774,31808] 0 2021-11-03T17:14:18.218 INFO:tasks.workunit.client.0.smithi027.stdout:5/664: rename d2/d4/f5 to d2/d2c/d3c/d48/d65/d80/fd8 0 2021-11-03T17:14:18.219 INFO:tasks.workunit.client.1.smithi115.stdout:5/565: unlink d1a/d28/d34/f32 0 2021-11-03T17:14:18.221 INFO:tasks.workunit.client.0.smithi027.stdout:5/665: mkdir d2/d12/dad/da8/dd9 0 2021-11-03T17:14:18.222 INFO:tasks.workunit.client.1.smithi115.stdout:0/497: write d9/d15/d78/d80/f9a [3482590,61720] 0 2021-11-03T17:14:18.225 INFO:tasks.workunit.client.1.smithi115.stdout:7/344: chown d3/da/d26/d52/f66 272495877 1 2021-11-03T17:14:18.227 INFO:tasks.workunit.client.1.smithi115.stdout:5/566: getdents d1a/d28/d7b/d48/d7f 0 2021-11-03T17:14:18.229 INFO:tasks.workunit.client.1.smithi115.stdout:7/345: write d3/da/d18/f22 [2778383,111162] 0 2021-11-03T17:14:18.229 INFO:tasks.workunit.client.1.smithi115.stdout:0/498: unlink d9/d15/d2c/d46/l54 0 2021-11-03T17:14:18.230 INFO:tasks.workunit.client.1.smithi115.stdout:5/567: truncate d1a/d28/d34/f66 212540 0 2021-11-03T17:14:18.231 INFO:tasks.workunit.client.1.smithi115.stdout:7/346: getdents d3/da/d26/d33 0 2021-11-03T17:14:18.232 INFO:tasks.workunit.client.1.smithi115.stdout:7/347: mknod d3/da/d26/d35/c6e 1 2021-11-03T17:14:18.236 INFO:tasks.workunit.client.1.smithi115.stdout:5/568: unlink d1a/d28/d7b/d4f/l9b 0 2021-11-03T17:14:18.237 INFO:tasks.workunit.client.1.smithi115.stdout:5/569: getdents d1a/d28/d4a/d65 0 2021-11-03T17:14:18.238 INFO:tasks.workunit.client.1.smithi115.stdout:5/570: write d1a/d3d/f47 [686690,130744] 0 2021-11-03T17:14:18.244 INFO:tasks.workunit.client.1.smithi115.stdout:1/414: dwrite d7/dc/dd/d24/d43/f6f [0,4194304] 0 2021-11-03T17:14:18.245 INFO:tasks.workunit.client.1.smithi115.stdout:0/499: dwrite d9/db/d89/f9c [0,4194304] 0 2021-11-03T17:14:18.245 INFO:tasks.workunit.client.1.smithi115.stdout:0/500: chown d9/d15/d2c/d46/d63 3 1 2021-11-03T17:14:18.247 INFO:tasks.workunit.client.1.smithi115.stdout:0/501: rename d9/d15/d2c/f5a to d9/d15/d2c/d6e/fa3 0 2021-11-03T17:14:18.248 INFO:tasks.workunit.client.1.smithi115.stdout:0/502: truncate d9/db/d83/f9d 311711 0 2021-11-03T17:14:18.248 INFO:tasks.workunit.client.1.smithi115.stdout:0/503: dread - d9/db/d89/f39 zero size 2021-11-03T17:14:18.248 INFO:tasks.workunit.client.1.smithi115.stdout:0/504: readlink d9/d15/d1a/l6c 0 2021-11-03T17:14:18.249 INFO:tasks.workunit.client.1.smithi115.stdout:0/505: creat d9/d15/d1a/fa4 x:0 0 0 2021-11-03T17:14:18.249 INFO:tasks.workunit.client.1.smithi115.stdout:0/506: chown d9/d15/d2c/d46/d63 129 1 2021-11-03T17:14:18.255 INFO:tasks.workunit.client.1.smithi115.stdout:0/507: dread d9/d15/f67 [0,4194304] 0 2021-11-03T17:14:18.256 INFO:tasks.workunit.client.1.smithi115.stdout:0/508: fsync d9/d15/d2c/d46/d8c/f7c 0 2021-11-03T17:14:18.256 INFO:tasks.workunit.client.1.smithi115.stdout:0/509: chown d9/d15/d78 406391 1 2021-11-03T17:14:18.256 INFO:tasks.workunit.client.1.smithi115.stdout:0/510: write d9/d15/d1a/f9e [386587,41450] 0 2021-11-03T17:14:18.257 INFO:tasks.workunit.client.1.smithi115.stdout:0/511: creat d9/d15/d2c/d6e/d71/fa5 x:0 0 0 2021-11-03T17:14:18.257 INFO:tasks.workunit.client.1.smithi115.stdout:0/512: truncate d9/d15/d2c/d46/d63/f7b 2261 0 2021-11-03T17:14:18.259 INFO:tasks.workunit.client.1.smithi115.stdout:0/513: rename d9/d15/d78/d80/f87 to d9/d15/d2c/d46/d63/fa6 0 2021-11-03T17:14:18.259 INFO:tasks.workunit.client.1.smithi115.stdout:0/514: mkdir d9/d15/d1a/da7 0 2021-11-03T17:14:18.260 INFO:tasks.workunit.client.1.smithi115.stdout:0/515: stat f8 0 2021-11-03T17:14:18.278 INFO:tasks.workunit.client.0.smithi027.stdout:5/666: rmdir d2/d12/dad/da8 39 2021-11-03T17:14:18.282 INFO:tasks.workunit.client.1.smithi115.stdout:2/407: dwrite d4/d6/d56/d6c/f6f [0,4194304] 0 2021-11-03T17:14:18.282 INFO:tasks.workunit.client.1.smithi115.stdout:2/408: chown d4/d6/d56/d6c/d34/d43/d63/d69 507292 1 2021-11-03T17:14:18.285 INFO:tasks.workunit.client.1.smithi115.stdout:1/415: write d7/dc/d1a/f3d [1023977,57079] 0 2021-11-03T17:14:18.308 INFO:tasks.workunit.client.1.smithi115.stdout:7/348: fdatasync d3/da/d18/f22 0 2021-11-03T17:14:18.313 INFO:tasks.workunit.client.1.smithi115.stdout:5/571: dwrite d1a/d28/f91 [0,4194304] 0 2021-11-03T17:14:18.314 INFO:tasks.workunit.client.1.smithi115.stdout:1/416: dwrite d7/dc/dd/d24/d43/f3c [4194304,4194304] 0 2021-11-03T17:14:18.314 INFO:tasks.workunit.client.1.smithi115.stdout:1/417: truncate d7/dc/f2f 5155893 0 2021-11-03T17:14:18.319 INFO:tasks.workunit.client.1.smithi115.stdout:5/572: rename fd to d1a/d28/d7b/d4f/d81/fb7 0 2021-11-03T17:14:18.322 INFO:tasks.workunit.client.1.smithi115.stdout:1/418: creat d7/dc/dd/d24/d4b/d55/f86 x:0 0 0 2021-11-03T17:14:18.323 INFO:tasks.workunit.client.1.smithi115.stdout:5/573: truncate d1a/d28/d34/f33 7055845 0 2021-11-03T17:14:18.324 INFO:tasks.workunit.client.1.smithi115.stdout:1/419: link d7/dc/d1a/f3d d7/dc/f87 0 2021-11-03T17:14:18.324 INFO:tasks.workunit.client.1.smithi115.stdout:1/420: fsync d7/dc/dd/d24/d4b/f72 0 2021-11-03T17:14:18.324 INFO:tasks.workunit.client.1.smithi115.stdout:1/421: write d7/dc/dd/d24/d4d/f7b [191814,33833] 0 2021-11-03T17:14:18.324 INFO:tasks.workunit.client.1.smithi115.stdout:1/422: fdatasync d7/dc/d1a/f35 0 2021-11-03T17:14:18.325 INFO:tasks.workunit.client.1.smithi115.stdout:5/574: rmdir d1a/d28/d4a/d65 39 2021-11-03T17:14:18.325 INFO:tasks.workunit.client.1.smithi115.stdout:5/575: write d1a/d23/f61 [5060952,114245] 0 2021-11-03T17:14:18.335 INFO:tasks.workunit.client.1.smithi115.stdout:5/576: dread d1a/d28/d7b/d4f/d81/fb7 [0,4194304] 0 2021-11-03T17:14:18.337 INFO:tasks.workunit.client.1.smithi115.stdout:5/577: rmdir d1a/d28/d7b/d4f/d81 39 2021-11-03T17:14:18.347 INFO:tasks.workunit.client.1.smithi115.stdout:0/516: dwrite d9/d15/d78/d80/f97 [0,4194304] 0 2021-11-03T17:14:18.351 INFO:tasks.workunit.client.1.smithi115.stdout:0/517: read d9/d15/d1a/f92 [2019104,102312] 0 2021-11-03T17:14:18.355 INFO:tasks.workunit.client.1.smithi115.stdout:0/518: write d9/db/d89/f9c [2541155,100018] 0 2021-11-03T17:14:18.357 INFO:tasks.workunit.client.1.smithi115.stdout:2/409: dwrite d4/d6/d56/d6c/d34/d43/f4d [0,4194304] 0 2021-11-03T17:14:18.357 INFO:tasks.workunit.client.1.smithi115.stdout:5/578: dwrite d1a/d28/d4c/d75/d83/f88 [0,4194304] 0 2021-11-03T17:14:18.358 INFO:tasks.workunit.client.1.smithi115.stdout:5/579: stat d1a/d28/d4c/d75/d68/da5 0 2021-11-03T17:14:18.358 INFO:tasks.workunit.client.1.smithi115.stdout:7/349: dwrite d3/f14 [4194304,4194304] 0 2021-11-03T17:14:18.359 INFO:tasks.workunit.client.0.smithi027.stdout:5/667: dwrite d2/d12/d7d/da7/fac [0,4194304] 0 2021-11-03T17:14:18.359 INFO:tasks.workunit.client.0.smithi027.stdout:5/668: mknod d2/d4/d4e/d67/d8e/cda 1 2021-11-03T17:14:18.361 INFO:tasks.workunit.client.1.smithi115.stdout:2/410: symlink d4/d6/d56/d6c/d34/d43/d63/d77/d79/d82/l90 0 2021-11-03T17:14:18.362 INFO:tasks.workunit.client.1.smithi115.stdout:2/411: mknod d4/d6/d56/d6c/d34/d43/d63/d77/d79/d82/c91 1 2021-11-03T17:14:18.362 INFO:tasks.workunit.client.1.smithi115.stdout:2/412: mknod d4/d6/d56/d6c/d34/d43/d63/d77/d79/d82/c92 1 2021-11-03T17:14:18.362 INFO:tasks.workunit.client.0.smithi027.stdout:5/669: creat d2/d2c/d3c/d48/d65/d80/fdb x:0 0 0 2021-11-03T17:14:18.363 INFO:tasks.workunit.client.1.smithi115.stdout:5/580: truncate d1a/d28/d4c/d75/d83/f98 3004623 0 2021-11-03T17:14:18.364 INFO:tasks.workunit.client.1.smithi115.stdout:7/350: symlink d3/da/d26/d35/d53/l6f 0 2021-11-03T17:14:18.365 INFO:tasks.workunit.client.0.smithi027.stdout:5/670: dread d2/d2c/d3c/d48/d65/f8f [0,4194304] 0 2021-11-03T17:14:18.365 INFO:tasks.workunit.client.0.smithi027.stdout:5/671: chown d2/d4/d4e/f4f 220785241 1 2021-11-03T17:14:18.366 INFO:tasks.workunit.client.1.smithi115.stdout:5/581: creat d1a/d28/d78/fb8 x:0 0 0 2021-11-03T17:14:18.367 INFO:tasks.workunit.client.1.smithi115.stdout:7/351: write d3/f47 [3022405,40995] 0 2021-11-03T17:14:18.367 INFO:tasks.workunit.client.1.smithi115.stdout:7/352: mknod d3/da/d26/d35/d53/c70 1 2021-11-03T17:14:18.368 INFO:tasks.workunit.client.1.smithi115.stdout:7/353: chown d3/f47 56 1 2021-11-03T17:14:18.368 INFO:tasks.workunit.client.1.smithi115.stdout:7/354: mknod d3/da/d26/d52/c71 1 2021-11-03T17:14:18.368 INFO:tasks.workunit.client.1.smithi115.stdout:7/355: readlink d3/l10 0 2021-11-03T17:14:18.368 INFO:tasks.workunit.client.1.smithi115.stdout:7/356: truncate d3/da/d26/f2f 926896 0 2021-11-03T17:14:18.369 INFO:tasks.workunit.client.1.smithi115.stdout:7/357: unlink d3/da/l24 0 2021-11-03T17:14:18.371 INFO:tasks.workunit.client.1.smithi115.stdout:7/358: creat d3/d4c/f72 x:0 0 0 2021-11-03T17:14:18.371 INFO:tasks.workunit.client.1.smithi115.stdout:7/359: dread - d3/da/d26/d52/f56 zero size 2021-11-03T17:14:18.375 INFO:tasks.workunit.client.1.smithi115.stdout:0/519: write d9/db/d83/f9d [980489,90491] 0 2021-11-03T17:14:18.377 INFO:tasks.workunit.client.1.smithi115.stdout:0/520: creat d9/fa8 x:0 0 0 2021-11-03T17:14:18.378 INFO:tasks.workunit.client.1.smithi115.stdout:0/521: creat d9/d15/d2c/d6e/d71/fa9 x:0 0 0 2021-11-03T17:14:18.380 INFO:tasks.workunit.client.1.smithi115.stdout:0/522: link d9/l3e d9/d15/d1a/laa 0 2021-11-03T17:14:18.381 INFO:tasks.workunit.client.1.smithi115.stdout:0/523: write d9/d15/f67 [4052944,6068] 0 2021-11-03T17:14:18.382 INFO:tasks.workunit.client.1.smithi115.stdout:0/524: mknod d9/d15/d1a/cab 1 2021-11-03T17:14:18.382 INFO:tasks.workunit.client.1.smithi115.stdout:7/360: rename d3/da/d26 to d3/da/d18/d73 0 2021-11-03T17:14:18.382 INFO:tasks.workunit.client.1.smithi115.stdout:7/361: fsync d3/da/d18/f6d 0 2021-11-03T17:14:18.383 INFO:tasks.workunit.client.1.smithi115.stdout:0/525: mkdir d9/db/d83/dac 0 2021-11-03T17:14:18.385 INFO:tasks.workunit.client.1.smithi115.stdout:0/526: write d9/d15/d2c/d46/d8c/f7c [6495674,32184] 0 2021-11-03T17:14:18.385 INFO:tasks.workunit.client.1.smithi115.stdout:0/527: write d9/db/f75 [877803,108291] 0 2021-11-03T17:14:18.386 INFO:tasks.workunit.client.1.smithi115.stdout:0/528: fdatasync d9/d15/d2c/d46/d63/f7b 0 2021-11-03T17:14:18.386 INFO:tasks.workunit.client.1.smithi115.stdout:7/362: getdents d3 0 2021-11-03T17:14:18.387 INFO:tasks.workunit.client.1.smithi115.stdout:2/413: dwrite d4/f62 [0,4194304] 0 2021-11-03T17:14:18.387 INFO:tasks.workunit.client.1.smithi115.stdout:0/529: unlink d9/db/f1b 0 2021-11-03T17:14:18.388 INFO:tasks.workunit.client.0.smithi027.stdout:5/672: dwrite d2/d12/dad/da8/fc4 [0,4194304] 0 2021-11-03T17:14:18.389 INFO:tasks.workunit.client.0.smithi027.stdout:5/673: write d2/d4/d61/f89 [430706,125172] 0 2021-11-03T17:14:18.389 INFO:tasks.workunit.client.0.smithi027.stdout:5/674: chown d2/d2c/d9b/faa 233456 1 2021-11-03T17:14:18.390 INFO:tasks.workunit.client.0.smithi027.stdout:5/675: mknod d2/d2c/d3c/dc8/cdc 1 2021-11-03T17:14:18.390 INFO:tasks.workunit.client.1.smithi115.stdout:2/414: rename d4/d44/l61 to d4/d6/d56/d6c/d34/d43/l93 0 2021-11-03T17:14:18.391 INFO:tasks.workunit.client.1.smithi115.stdout:2/415: truncate d4/d6/d56/d6c/d34/f89 722987 0 2021-11-03T17:14:18.392 INFO:tasks.workunit.client.1.smithi115.stdout:0/530: symlink d9/d15/d2c/d46/d8c/lad 0 2021-11-03T17:14:18.392 INFO:tasks.workunit.client.1.smithi115.stdout:0/531: readlink d9/d15/d78/d80/l94 0 2021-11-03T17:14:18.393 INFO:tasks.workunit.client.1.smithi115.stdout:2/416: symlink d4/d6/d13/l94 0 2021-11-03T17:14:18.393 INFO:tasks.workunit.client.1.smithi115.stdout:2/417: mknod d4/d6/d56/d6c/d34/d43/d63/d77/d79/c95 1 2021-11-03T17:14:18.394 INFO:tasks.workunit.client.0.smithi027.stdout:5/676: dread d2/f3 [0,4194304] 0 2021-11-03T17:14:18.394 INFO:tasks.workunit.client.1.smithi115.stdout:0/532: mkdir d9/d15/d2c/d46/d63/dae 0 2021-11-03T17:14:18.395 INFO:tasks.workunit.client.1.smithi115.stdout:0/533: dread - d9/f66 zero size 2021-11-03T17:14:18.395 INFO:tasks.workunit.client.1.smithi115.stdout:0/534: chown d9/db/d89 1793 1 2021-11-03T17:14:18.395 INFO:tasks.workunit.client.1.smithi115.stdout:0/535: mkdir d9/d15/d1a/da7/daf 0 2021-11-03T17:14:18.399 INFO:tasks.workunit.client.1.smithi115.stdout:0/536: unlink d9/db/d89/f39 0 2021-11-03T17:14:18.399 INFO:tasks.workunit.client.1.smithi115.stdout:0/537: readlink d9/d15/d2c/d46/d8c/lad 0 2021-11-03T17:14:18.400 INFO:tasks.workunit.client.1.smithi115.stdout:0/538: link d9/d15/l45 d9/d15/d78/lb0 0 2021-11-03T17:14:18.401 INFO:tasks.workunit.client.1.smithi115.stdout:0/539: creat d9/db/d89/fb1 x:0 0 0 2021-11-03T17:14:18.402 INFO:tasks.workunit.client.1.smithi115.stdout:7/363: dread d3/f47 [0,4194304] 0 2021-11-03T17:14:18.404 INFO:tasks.workunit.client.1.smithi115.stdout:0/540: rmdir d9/db/d83/dac 0 2021-11-03T17:14:18.405 INFO:tasks.workunit.client.1.smithi115.stdout:7/364: link d3/da/d18/f22 d3/da/d18/d73/d33/f74 0 2021-11-03T17:14:18.406 INFO:tasks.workunit.client.1.smithi115.stdout:0/541: symlink d9/d15/d2c/d46/d8c/lb2 0 2021-11-03T17:14:18.407 INFO:tasks.workunit.client.1.smithi115.stdout:2/418: dread d4/d6/d13/d6d/f5f [0,4194304] 0 2021-11-03T17:14:18.408 INFO:tasks.workunit.client.1.smithi115.stdout:7/365: creat d3/da/d18/d73/d35/f75 x:0 0 0 2021-11-03T17:14:18.408 INFO:tasks.workunit.client.1.smithi115.stdout:7/366: symlink d3/da/d18/d73/d33/l76 0 2021-11-03T17:14:18.409 INFO:tasks.workunit.client.1.smithi115.stdout:7/367: write d3/da/d18/f6d [515265,98020] 0 2021-11-03T17:14:18.409 INFO:tasks.workunit.client.1.smithi115.stdout:7/368: stat d3/da/d18/d73/l4e 0 2021-11-03T17:14:18.412 INFO:tasks.workunit.client.0.smithi027.stdout:5/677: dwrite d2/d4/d4e/d67/f11 [0,4194304] 0 2021-11-03T17:14:18.415 INFO:tasks.workunit.client.1.smithi115.stdout:2/419: dread d4/d6/d56/d6c/f17 [0,4194304] 0 2021-11-03T17:14:18.417 INFO:tasks.workunit.client.0.smithi027.stdout:5/678: rename d2/d12/dad/fb8 to d2/d2c/d3c/fdd 0 2021-11-03T17:14:18.417 INFO:tasks.workunit.client.0.smithi027.stdout:5/679: truncate d2/d2c/d3c/d58/fca 175690 0 2021-11-03T17:14:18.446 INFO:tasks.workunit.client.1.smithi115.stdout:2/420: dwrite d4/d6/d56/d6c/d34/f52 [0,4194304] 0 2021-11-03T17:14:18.447 INFO:tasks.workunit.client.0.smithi027.stdout:5/680: dwrite d2/d4/d4e/d67/d20/fb9 [0,4194304] 0 2021-11-03T17:14:18.447 INFO:tasks.workunit.client.0.smithi027.stdout:5/681: chown d2/f7 5595753 1 2021-11-03T17:14:18.447 INFO:tasks.workunit.client.0.smithi027.stdout:5/682: chown d2/d2c/d3c/d48/laf 167498570 1 2021-11-03T17:14:18.448 INFO:tasks.workunit.client.1.smithi115.stdout:0/542: dwrite d9/db/f58 [0,4194304] 0 2021-11-03T17:14:18.448 INFO:tasks.workunit.client.1.smithi115.stdout:7/369: dwrite d3/da/f64 [0,4194304] 0 2021-11-03T17:14:18.449 INFO:tasks.workunit.client.0.smithi027.stdout:5/683: mkdir d2/d12/dad/da8/dd9/dde 0 2021-11-03T17:14:18.450 INFO:tasks.workunit.client.1.smithi115.stdout:7/370: write d3/da/d18/d73/d35/d53/f58 [1195092,121225] 0 2021-11-03T17:14:18.450 INFO:tasks.workunit.client.1.smithi115.stdout:0/543: link l4 d9/d15/d2c/d46/d63/lb3 0 2021-11-03T17:14:18.451 INFO:tasks.workunit.client.1.smithi115.stdout:7/371: write d3/f27 [448677,1055] 0 2021-11-03T17:14:18.451 INFO:tasks.workunit.client.1.smithi115.stdout:7/372: write d3/da/d18/d73/f2f [1420941,125724] 0 2021-11-03T17:14:18.451 INFO:tasks.workunit.client.1.smithi115.stdout:0/544: creat d9/d15/d1a/d1d/fb4 x:0 0 0 2021-11-03T17:14:18.452 INFO:tasks.workunit.client.1.smithi115.stdout:7/373: truncate d3/da/d18/d73/d35/f3f 854844 0 2021-11-03T17:14:18.460 INFO:tasks.workunit.client.1.smithi115.stdout:2/421: dread d4/d6/d56/d6c/f3c [4194304,4194304] 0 2021-11-03T17:14:18.464 INFO:tasks.workunit.client.1.smithi115.stdout:2/422: symlink d4/d6/d13/d6d/d7f/l96 0 2021-11-03T17:14:18.464 INFO:tasks.workunit.client.1.smithi115.stdout:0/545: dread d9/d15/d2c/d46/d8c/f49 [0,4194304] 0 2021-11-03T17:14:18.465 INFO:tasks.workunit.client.1.smithi115.stdout:0/546: dread - d9/d15/d2c/d6e/d71/fa5 zero size 2021-11-03T17:14:18.469 INFO:tasks.workunit.client.1.smithi115.stdout:2/423: truncate d4/d6/d56/d6c/f46 1466498 0 2021-11-03T17:14:18.470 INFO:tasks.workunit.client.1.smithi115.stdout:0/547: creat d9/d15/d78/d80/d82/fb5 x:0 0 0 2021-11-03T17:14:18.471 INFO:tasks.workunit.client.0.smithi027.stdout:5/684: dwrite d2/d2c/d3c/f9f [0,4194304] 0 2021-11-03T17:14:18.471 INFO:tasks.workunit.client.0.smithi027.stdout:5/685: write d2/d4/d4e/d67/d20/f3a [8425012,64813] 0 2021-11-03T17:14:18.473 INFO:tasks.workunit.client.1.smithi115.stdout:2/424: link d4/d6/d13/d6d/f8b d4/d6/d56/d6c/d34/d43/d63/f97 0 2021-11-03T17:14:18.473 INFO:tasks.workunit.client.1.smithi115.stdout:0/548: write d9/d15/d2c/d46/d8c/f79 [3598635,124585] 0 2021-11-03T17:14:18.474 INFO:tasks.workunit.client.0.smithi027.stdout:5/686: getdents d2/d2c/d9b 0 2021-11-03T17:14:18.475 INFO:tasks.workunit.client.0.smithi027.stdout:5/687: chown d2/d2c/d3c/fdd 4 1 2021-11-03T17:14:18.475 INFO:tasks.workunit.client.1.smithi115.stdout:2/425: unlink d4/d6/d56/d6c/f65 0 2021-11-03T17:14:18.475 INFO:tasks.workunit.client.1.smithi115.stdout:2/426: truncate d4/d6/d56/d6c/d34/d43/d28/f3b 33491 0 2021-11-03T17:14:18.479 INFO:tasks.workunit.client.1.smithi115.stdout:0/549: getdents d9/d15/d1a/da7 0 2021-11-03T17:14:18.480 INFO:tasks.workunit.client.1.smithi115.stdout:0/550: dread - d9/d15/d1a/fa4 zero size 2021-11-03T17:14:18.480 INFO:tasks.workunit.client.1.smithi115.stdout:2/427: mkdir d4/d44/d98 0 2021-11-03T17:14:18.483 INFO:tasks.workunit.client.1.smithi115.stdout:0/551: mkdir d9/d15/db6 0 2021-11-03T17:14:18.483 INFO:tasks.workunit.client.1.smithi115.stdout:0/552: write d9/db/f58 [4720785,1787] 0 2021-11-03T17:14:18.483 INFO:tasks.workunit.client.1.smithi115.stdout:2/428: creat d4/d6/d56/d6c/d34/d43/d63/f99 x:0 0 0 2021-11-03T17:14:18.484 INFO:tasks.workunit.client.1.smithi115.stdout:2/429: chown d4/f78 297950 1 2021-11-03T17:14:18.484 INFO:tasks.workunit.client.1.smithi115.stdout:0/553: creat d9/d15/d1a/d1d/fb7 x:0 0 0 2021-11-03T17:14:18.485 INFO:tasks.workunit.client.1.smithi115.stdout:2/430: read d4/d6/d56/d6c/d34/f70 [1612952,101258] 0 2021-11-03T17:14:18.486 INFO:tasks.workunit.client.1.smithi115.stdout:2/431: rename d4/d6/f12 to d4/d6/d56/f9a 0 2021-11-03T17:14:18.494 INFO:tasks.workunit.client.0.smithi027.stdout:5/688: dwrite d2/d2c/d3c/d58/fc7 [0,4194304] 0 2021-11-03T17:14:18.495 INFO:tasks.workunit.client.0.smithi027.stdout:5/689: mknod d2/d2c/d3c/d58/cdf 1 2021-11-03T17:14:18.495 INFO:tasks.workunit.client.0.smithi027.stdout:5/690: mknod d2/d12/dad/da8/dd9/dde/ce0 1 2021-11-03T17:14:18.496 INFO:tasks.workunit.client.1.smithi115.stdout:0/554: dread d9/db/f47 [0,4194304] 0 2021-11-03T17:14:18.497 INFO:tasks.workunit.client.1.smithi115.stdout:0/555: symlink d9/d15/lb8 0 2021-11-03T17:14:18.497 INFO:tasks.workunit.client.1.smithi115.stdout:0/556: chown d9/d15/d1a/d1d/fb7 273754 1 2021-11-03T17:14:18.497 INFO:tasks.workunit.client.0.smithi027.stdout:5/691: mkdir d2/d2c/d9b/de1 0 2021-11-03T17:14:18.499 INFO:tasks.workunit.client.0.smithi027.stdout:5/692: creat d2/d4/d4e/d67/fe2 x:0 0 0 2021-11-03T17:14:18.499 INFO:tasks.workunit.client.0.smithi027.stdout:5/693: dread - d2/d2c/d9b/d9d/fbe zero size 2021-11-03T17:14:18.499 INFO:tasks.workunit.client.0.smithi027.stdout:5/694: dread - d2/d1f/fbf zero size 2021-11-03T17:14:18.500 INFO:tasks.workunit.client.0.smithi027.stdout:5/695: mknod d2/d2c/d9b/ce3 1 2021-11-03T17:14:18.500 INFO:tasks.workunit.client.0.smithi027.stdout:7/325: sync 2021-11-03T17:14:18.502 INFO:tasks.workunit.client.0.smithi027.stdout:7/326: truncate d0/d2d/d53/f65 3095449 0 2021-11-03T17:14:18.502 INFO:tasks.workunit.client.1.smithi115.stdout:2/432: dwrite d4/f35 [0,4194304] 0 2021-11-03T17:14:18.503 INFO:tasks.workunit.client.0.smithi027.stdout:7/327: creat d0/d2d/d66/f70 x:0 0 0 2021-11-03T17:14:18.529 INFO:tasks.workunit.client.1.smithi115.stdout:0/557: dwrite d9/d15/d2c/d6e/f9f [0,4194304] 0 2021-11-03T17:14:18.529 INFO:tasks.workunit.client.1.smithi115.stdout:0/558: getdents d9/d15/db6 0 2021-11-03T17:14:18.529 INFO:tasks.workunit.client.1.smithi115.stdout:0/559: mknod d9/d15/d1a/da7/cb9 1 2021-11-03T17:14:18.529 INFO:tasks.workunit.client.1.smithi115.stdout:0/560: fdatasync d9/d15/d2c/d6e/fa3 0 2021-11-03T17:14:18.530 INFO:tasks.workunit.client.0.smithi027.stdout:5/696: getdents d2/d2c/d9b 0 2021-11-03T17:14:18.531 INFO:tasks.workunit.client.1.smithi115.stdout:0/561: rename d9/d15/d2c/d46/f5b to d9/d15/db6/fba 0 2021-11-03T17:14:18.531 INFO:tasks.workunit.client.1.smithi115.stdout:0/562: stat d9/d15/d78/d80/d82 0 2021-11-03T17:14:18.531 INFO:tasks.workunit.client.1.smithi115.stdout:0/563: mknod d9/d15/d2c/d46/d8c/cbb 1 2021-11-03T17:14:18.533 INFO:tasks.workunit.client.1.smithi115.stdout:2/433: dwrite d4/d6/d56/d6c/d34/f52 [0,4194304] 0 2021-11-03T17:14:18.533 INFO:tasks.workunit.client.1.smithi115.stdout:0/564: unlink d9/db/d83/f9d 0 2021-11-03T17:14:18.534 INFO:tasks.workunit.client.0.smithi027.stdout:7/328: dwrite d0/d2/d5e/f32 [0,4194304] 0 2021-11-03T17:14:18.534 INFO:tasks.workunit.client.0.smithi027.stdout:7/329: chown d0/d2/d10/d1e/d20 3 1 2021-11-03T17:14:18.537 INFO:tasks.workunit.client.1.smithi115.stdout:0/565: rmdir d9/d15/d2c/d46/d8c 39 2021-11-03T17:14:18.538 INFO:tasks.workunit.client.0.smithi027.stdout:7/330: write d0/f25 [553744,83080] 0 2021-11-03T17:14:18.539 INFO:tasks.workunit.client.1.smithi115.stdout:0/566: symlink d9/d15/db6/lbc 0 2021-11-03T17:14:18.540 INFO:tasks.workunit.client.1.smithi115.stdout:0/567: symlink d9/db/d89/d81/lbd 0 2021-11-03T17:14:18.541 INFO:tasks.workunit.client.1.smithi115.stdout:0/568: symlink d9/d15/d78/d80/lbe 0 2021-11-03T17:14:18.545 INFO:tasks.workunit.client.0.smithi027.stdout:5/697: dwrite d2/d1f/f6f [0,4194304] 0 2021-11-03T17:14:18.550 INFO:tasks.workunit.client.1.smithi115.stdout:2/434: rmdir d4/d6/d56/d6c/d34 39 2021-11-03T17:14:18.550 INFO:tasks.workunit.client.1.smithi115.stdout:2/435: rename d4/d6/d56/d6c to d4/d6/d56/d6c/d34/d43/d9b 22 2021-11-03T17:14:18.554 INFO:tasks.workunit.client.0.smithi027.stdout:7/331: dwrite d0/d2/d10/f19 [0,4194304] 0 2021-11-03T17:14:18.555 INFO:tasks.workunit.client.1.smithi115.stdout:2/436: unlink d4/d6/d56/d6c/d34/d43/d63/f97 0 2021-11-03T17:14:18.557 INFO:tasks.workunit.client.1.smithi115.stdout:2/437: mkdir d4/d6/d56/d6c/d34/d43/d63/d77/d79/d9c 0 2021-11-03T17:14:18.557 INFO:tasks.workunit.client.1.smithi115.stdout:2/438: stat d4/d6/f7e 0 2021-11-03T17:14:18.557 INFO:tasks.workunit.client.0.smithi027.stdout:5/698: dread d2/d2c/d3c/d58/f6e [0,4194304] 0 2021-11-03T17:14:18.558 INFO:tasks.workunit.client.1.smithi115.stdout:2/439: read d4/d6/d56/d6c/d34/d43/f4d [1779792,129880] 0 2021-11-03T17:14:18.558 INFO:tasks.workunit.client.1.smithi115.stdout:2/440: symlink d4/d6/d56/d6c/d34/d43/d63/d77/d79/d82/l9d 2 2021-11-03T17:14:18.559 INFO:tasks.workunit.client.0.smithi027.stdout:5/699: write d2/d2c/d3c/d48/d65/d80/fd8 [6319067,44197] 0 2021-11-03T17:14:18.560 INFO:tasks.workunit.client.0.smithi027.stdout:5/700: chown d2/d2c/d3c/d58/fca 25911 1 2021-11-03T17:14:18.560 INFO:tasks.workunit.client.0.smithi027.stdout:5/701: write d2/d2c/d3c/d58/f74 [1862743,35563] 0 2021-11-03T17:14:18.561 INFO:tasks.workunit.client.0.smithi027.stdout:5/702: chown d2/d4/f66 0 1 2021-11-03T17:14:18.562 INFO:tasks.workunit.client.1.smithi115.stdout:2/441: creat d4/d6/f9e x:0 0 0 2021-11-03T17:14:18.565 INFO:tasks.workunit.client.1.smithi115.stdout:2/442: mkdir d4/d6/d56/d6c/d34/d43/d63/d77/d79/d9c/d9f 0 2021-11-03T17:14:18.565 INFO:tasks.workunit.client.1.smithi115.stdout:2/443: mknod d4/d6/d56/d6c/ca0 1 2021-11-03T17:14:18.568 INFO:tasks.workunit.client.0.smithi027.stdout:5/703: dread d2/d12/f9a [0,4194304] 0 2021-11-03T17:14:18.578 INFO:tasks.workunit.client.0.smithi027.stdout:7/332: dwrite d0/d2/d10/f19 [4194304,4194304] 0 2021-11-03T17:14:18.578 INFO:tasks.workunit.client.1.smithi115.stdout:2/444: dread d4/d6/d13/f2f [0,4194304] 0 2021-11-03T17:14:18.579 INFO:tasks.workunit.client.0.smithi027.stdout:7/333: symlink d0/d2/d5e/d36/l71 0 2021-11-03T17:14:18.581 INFO:tasks.workunit.client.0.smithi027.stdout:7/334: unlink d0/d2/d10/f19 0 2021-11-03T17:14:18.588 INFO:tasks.workunit.client.0.smithi027.stdout:5/704: dwrite d2/d2c/d3c/d48/d65/d80/d9e/fbd [0,4194304] 0 2021-11-03T17:14:18.591 INFO:tasks.workunit.client.0.smithi027.stdout:7/335: write d0/d2d/d2f/f39 [596354,25182] 0 2021-11-03T17:14:18.593 INFO:tasks.workunit.client.0.smithi027.stdout:5/705: creat d2/d12/dad/da9/fe4 x:0 0 0 2021-11-03T17:14:18.593 INFO:tasks.workunit.client.0.smithi027.stdout:5/706: mknod d2/d2c/d3c/d58/ce5 1 2021-11-03T17:14:18.594 INFO:tasks.workunit.client.0.smithi027.stdout:5/707: fsync d2/d12/fc0 0 2021-11-03T17:14:18.594 INFO:tasks.workunit.client.0.smithi027.stdout:5/708: mknod d2/d12/dad/da9/ce6 1 2021-11-03T17:14:18.594 INFO:tasks.workunit.client.0.smithi027.stdout:5/709: truncate d2/d12/dad/da8/fcb 9563 0 2021-11-03T17:14:18.595 INFO:tasks.workunit.client.0.smithi027.stdout:5/710: write d2/d12/dad/da8/fab [423864,55019] 0 2021-11-03T17:14:18.595 INFO:tasks.workunit.client.0.smithi027.stdout:7/336: link d0/d2/d1c/d4d/l5d d0/d2/d10/l72 0 2021-11-03T17:14:18.596 INFO:tasks.workunit.client.0.smithi027.stdout:5/711: creat d2/d2c/d3c/dc8/fe7 x:0 0 0 2021-11-03T17:14:18.596 INFO:tasks.workunit.client.0.smithi027.stdout:5/712: mknod d2/d2c/d3c/d48/d65/d80/dd2/ce8 1 2021-11-03T17:14:18.597 INFO:tasks.workunit.client.0.smithi027.stdout:7/337: creat d0/d2/d59/f73 x:0 0 0 2021-11-03T17:14:18.599 INFO:tasks.workunit.client.0.smithi027.stdout:5/713: creat d2/d1f/fe9 x:0 0 0 2021-11-03T17:14:18.602 INFO:tasks.workunit.client.1.smithi115.stdout:2/445: dwrite d4/d6/d56/d6c/f46 [0,4194304] 0 2021-11-03T17:14:18.605 INFO:tasks.workunit.client.1.smithi115.stdout:2/446: symlink d4/d6/d56/d6c/d34/d42/la1 0 2021-11-03T17:14:18.605 INFO:tasks.workunit.client.1.smithi115.stdout:2/447: chown d4/d6/d13/f4e 3424 1 2021-11-03T17:14:18.612 INFO:tasks.workunit.client.1.smithi115.stdout:2/448: dread d4/f62 [0,4194304] 0 2021-11-03T17:14:18.612 INFO:tasks.workunit.client.1.smithi115.stdout:2/449: write d4/d6/d56/d6c/f3c [5494636,43996] 0 2021-11-03T17:14:18.613 INFO:tasks.workunit.client.1.smithi115.stdout:2/450: fdatasync d4/f78 0 2021-11-03T17:14:18.613 INFO:tasks.workunit.client.0.smithi027.stdout:7/338: dwrite d0/d2/d10/d1e/d20/f26 [0,4194304] 0 2021-11-03T17:14:18.614 INFO:tasks.workunit.client.0.smithi027.stdout:7/339: write d0/d2/d10/f1b [816161,114105] 0 2021-11-03T17:14:18.621 INFO:tasks.workunit.client.0.smithi027.stdout:5/714: fsync d2/d1f/fe9 0 2021-11-03T17:14:18.630 INFO:tasks.workunit.client.1.smithi115.stdout:2/451: dwrite d4/d6/d13/f36 [4194304,4194304] 0 2021-11-03T17:14:18.632 INFO:tasks.workunit.client.0.smithi027.stdout:5/715: truncate d2/d12/dad/da8/fab 411285 0 2021-11-03T17:14:18.636 INFO:tasks.workunit.client.0.smithi027.stdout:7/340: dwrite d0/d2/d1f/f54 [0,4194304] 0 2021-11-03T17:14:18.656 INFO:tasks.workunit.client.0.smithi027.stdout:5/716: dwrite d2/d4/d4e/d67/f11 [0,4194304] 0 2021-11-03T17:14:18.658 INFO:tasks.workunit.client.1.smithi115.stdout:2/452: dwrite d4/d6/f45 [0,4194304] 0 2021-11-03T17:14:18.660 INFO:tasks.workunit.client.1.smithi115.stdout:8/462: sync 2021-11-03T17:14:18.663 INFO:tasks.workunit.client.1.smithi115.stdout:8/463: link d4/d6/d18/d44/f48 d4/d7d/fa0 0 2021-11-03T17:14:18.664 INFO:tasks.workunit.client.1.smithi115.stdout:8/464: dread d4/d6/f2c [0,4194304] 0 2021-11-03T17:14:18.665 INFO:tasks.workunit.client.0.smithi027.stdout:5/717: truncate d2/d2c/d3c/d58/f74 2956696 0 2021-11-03T17:14:18.666 INFO:tasks.workunit.client.1.smithi115.stdout:8/465: creat d4/d6/d18/d30/d79/d7f/fa1 x:0 0 0 2021-11-03T17:14:18.668 INFO:tasks.workunit.client.0.smithi027.stdout:5/718: symlink d2/d12/d7d/da7/lea 0 2021-11-03T17:14:18.674 INFO:tasks.workunit.client.0.smithi027.stdout:5/719: write d2/d4/fd5 [1164885,38494] 0 2021-11-03T17:14:18.675 INFO:tasks.workunit.client.0.smithi027.stdout:5/720: symlink d2/d2c/d9b/de1/leb 0 2021-11-03T17:14:18.681 INFO:tasks.workunit.client.0.smithi027.stdout:5/721: write d2/d2c/d3c/f41 [2714978,1063] 0 2021-11-03T17:14:18.681 INFO:tasks.workunit.client.0.smithi027.stdout:5/722: mknod d2/cec 1 2021-11-03T17:14:18.682 INFO:tasks.workunit.client.0.smithi027.stdout:5/723: truncate d2/d12/dad/da8/fc4 166441 0 2021-11-03T17:14:18.682 INFO:tasks.workunit.client.0.smithi027.stdout:5/724: chown d2/d12 78 1 2021-11-03T17:14:18.683 INFO:tasks.workunit.client.1.smithi115.stdout:2/453: dwrite d4/d6/d56/d6c/d34/d43/d63/f99 [0,4194304] 0 2021-11-03T17:14:18.686 INFO:tasks.workunit.client.1.smithi115.stdout:8/466: dwrite d4/d6/d18/d30/f3a [4194304,4194304] 0 2021-11-03T17:14:18.686 INFO:tasks.workunit.client.1.smithi115.stdout:8/467: chown d4/d6/d18/l9e 95 1 2021-11-03T17:14:18.687 INFO:tasks.workunit.client.1.smithi115.stdout:8/468: write d4/d6/d18/f63 [431507,87854] 0 2021-11-03T17:14:18.687 INFO:tasks.workunit.client.1.smithi115.stdout:8/469: chown d4/d6/d18/f3d 56880 1 2021-11-03T17:14:18.689 INFO:tasks.workunit.client.1.smithi115.stdout:8/470: symlink d4/d6/d3f/d43/la2 0 2021-11-03T17:14:18.689 INFO:tasks.workunit.client.0.smithi027.stdout:5/725: dread d2/d2c/d3c/d58/fa0 [0,4194304] 0 2021-11-03T17:14:18.698 INFO:tasks.workunit.client.1.smithi115.stdout:8/471: dread d4/d6/d18/d30/f3a [0,4194304] 0 2021-11-03T17:14:18.700 INFO:tasks.workunit.client.1.smithi115.stdout:8/472: creat d4/d6/d18/d44/fa3 x:0 0 0 2021-11-03T17:14:18.700 INFO:tasks.workunit.client.1.smithi115.stdout:8/473: stat d4/d6/d18/d30/d79/d7f 0 2021-11-03T17:14:18.700 INFO:tasks.workunit.client.1.smithi115.stdout:8/474: write d4/d6/f8 [4300111,32754] 0 2021-11-03T17:14:18.707 INFO:tasks.workunit.client.1.smithi115.stdout:8/475: dread d4/d6/d3f/d9a/f84 [0,4194304] 0 2021-11-03T17:14:18.708 INFO:tasks.workunit.client.0.smithi027.stdout:5/726: dwrite d2/d4/d4e/d67/d20/fb9 [4194304,4194304] 0 2021-11-03T17:14:18.731 INFO:tasks.workunit.client.1.smithi115.stdout:8/476: dwrite d4/d6/f6b [0,4194304] 0 2021-11-03T17:14:18.758 INFO:tasks.workunit.client.1.smithi115.stdout:8/477: dwrite d4/d6/d18/f63 [0,4194304] 0 2021-11-03T17:14:18.759 INFO:tasks.workunit.client.1.smithi115.stdout:8/478: write d4/d6/f42 [1269110,62011] 0 2021-11-03T17:14:18.760 INFO:tasks.workunit.client.1.smithi115.stdout:8/479: mkdir d4/d6/d3f/da4 0 2021-11-03T17:14:18.762 INFO:tasks.workunit.client.1.smithi115.stdout:8/480: getdents d4/d6/d3f/d43/d88 0 2021-11-03T17:14:18.907 INFO:tasks.workunit.client.0.smithi027.stdout:3/311: sync 2021-11-03T17:14:18.908 INFO:tasks.workunit.client.0.smithi027.stdout:3/312: write d1/f55 [175462,96936] 0 2021-11-03T17:14:18.908 INFO:tasks.workunit.client.0.smithi027.stdout:3/313: stat d1/d40/f4a 0 2021-11-03T17:14:18.910 INFO:tasks.workunit.client.0.smithi027.stdout:3/314: symlink d1/d40/l63 0 2021-11-03T17:14:18.911 INFO:tasks.workunit.client.0.smithi027.stdout:3/315: getdents d1/db/d52 0 2021-11-03T17:14:18.921 INFO:tasks.workunit.client.0.smithi027.stdout:3/316: dread d1/fa [0,4194304] 0 2021-11-03T17:14:18.921 INFO:tasks.workunit.client.0.smithi027.stdout:3/317: chown d1/l4 330207159 1 2021-11-03T17:14:18.921 INFO:tasks.workunit.client.0.smithi027.stdout:3/318: fsync d1/d40/f46 0 2021-11-03T17:14:18.924 INFO:tasks.workunit.client.0.smithi027.stdout:3/319: link d1/f5e d1/db/d52/f64 0 2021-11-03T17:14:18.926 INFO:tasks.workunit.client.0.smithi027.stdout:3/320: creat d1/db/d52/f65 x:0 0 0 2021-11-03T17:14:18.927 INFO:tasks.workunit.client.0.smithi027.stdout:3/321: mknod d1/db/d52/d57/c66 1 2021-11-03T17:14:18.927 INFO:tasks.workunit.client.0.smithi027.stdout:3/322: write d1/f50 [3826149,110178] 0 2021-11-03T17:14:18.934 INFO:tasks.workunit.client.0.smithi027.stdout:3/323: dread d1/db/f32 [0,4194304] 0 2021-11-03T17:14:18.934 INFO:tasks.workunit.client.0.smithi027.stdout:3/324: mknod d1/db/c67 1 2021-11-03T17:14:18.934 INFO:tasks.workunit.client.0.smithi027.stdout:3/325: truncate d1/d40/f58 1496182 0 2021-11-03T17:14:18.935 INFO:tasks.workunit.client.0.smithi027.stdout:3/326: write d1/db/d52/f5d [92516,49798] 0 2021-11-03T17:14:18.941 INFO:tasks.workunit.client.0.smithi027.stdout:3/327: rmdir d1/db 39 2021-11-03T17:14:18.950 INFO:tasks.workunit.client.0.smithi027.stdout:3/328: dread d1/db/f12 [0,4194304] 0 2021-11-03T17:14:18.951 INFO:tasks.workunit.client.0.smithi027.stdout:3/329: rmdir d1/db/d52 39 2021-11-03T17:14:18.951 INFO:tasks.workunit.client.0.smithi027.stdout:3/330: stat d1/f30 0 2021-11-03T17:14:18.953 INFO:tasks.workunit.client.0.smithi027.stdout:3/331: fdatasync d1/db/d52/f5d 0 2021-11-03T17:14:18.954 INFO:tasks.workunit.client.0.smithi027.stdout:3/332: dread d1/f4b [0,4194304] 0 2021-11-03T17:14:18.955 INFO:tasks.workunit.client.0.smithi027.stdout:3/333: truncate d1/db/f32 1316884 0 2021-11-03T17:14:18.955 INFO:tasks.workunit.client.0.smithi027.stdout:3/334: chown d1/d40/f4a 0 1 2021-11-03T17:14:18.955 INFO:tasks.workunit.client.0.smithi027.stdout:3/335: dread - d1/db/f5b zero size 2021-11-03T17:14:18.955 INFO:tasks.workunit.client.0.smithi027.stdout:3/336: dread - d1/db/f3f zero size 2021-11-03T17:14:18.956 INFO:tasks.workunit.client.0.smithi027.stdout:3/337: mknod d1/db/d52/d57/c68 1 2021-11-03T17:14:18.962 INFO:tasks.workunit.client.0.smithi027.stdout:3/338: dread d1/f30 [0,4194304] 0 2021-11-03T17:14:18.964 INFO:tasks.workunit.client.0.smithi027.stdout:3/339: symlink d1/d40/l69 0 2021-11-03T17:14:18.965 INFO:tasks.workunit.client.0.smithi027.stdout:3/340: mkdir d1/db/d52/d6a 0 2021-11-03T17:14:18.967 INFO:tasks.workunit.client.0.smithi027.stdout:3/341: symlink d1/d40/l6b 0 2021-11-03T17:14:18.967 INFO:tasks.workunit.client.0.smithi027.stdout:3/342: mknod d1/db/d52/d6a/c6c 1 2021-11-03T17:14:18.968 INFO:tasks.workunit.client.0.smithi027.stdout:3/343: creat d1/d40/f6d x:0 0 0 2021-11-03T17:14:18.968 INFO:tasks.workunit.client.0.smithi027.stdout:3/344: truncate d1/f55 2112365 0 2021-11-03T17:14:18.970 INFO:tasks.workunit.client.0.smithi027.stdout:3/345: creat d1/db/d52/d6a/f6e x:0 0 0 2021-11-03T17:14:18.971 INFO:tasks.workunit.client.0.smithi027.stdout:3/346: mkdir d1/d6f 0 2021-11-03T17:14:18.972 INFO:tasks.workunit.client.0.smithi027.stdout:3/347: write d1/d40/f59 [405857,62992] 0 2021-11-03T17:14:18.972 INFO:tasks.workunit.client.0.smithi027.stdout:3/348: fdatasync d1/f30 0 2021-11-03T17:14:18.977 INFO:tasks.workunit.client.0.smithi027.stdout:2/418: sync 2021-11-03T17:14:18.978 INFO:tasks.workunit.client.0.smithi027.stdout:2/419: write d0/d45/f7b [706832,106310] 0 2021-11-03T17:14:18.978 INFO:tasks.workunit.client.0.smithi027.stdout:2/420: write d0/d1b/f27 [417072,123091] 0 2021-11-03T17:14:18.980 INFO:tasks.workunit.client.0.smithi027.stdout:2/421: truncate d0/f10 2974921 0 2021-11-03T17:14:18.982 INFO:tasks.workunit.client.0.smithi027.stdout:2/422: creat d0/d5/f83 x:0 0 0 2021-11-03T17:14:18.982 INFO:tasks.workunit.client.0.smithi027.stdout:3/349: dread d1/f5e [0,4194304] 0 2021-11-03T17:14:18.982 INFO:tasks.workunit.client.0.smithi027.stdout:3/350: fdatasync d1/db/f5b 0 2021-11-03T17:14:18.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:18 smithi115 conmon[34670]: cluster 2021-11-03T17:14:18.606823+0000 mgr.smithi027.ufhhvb (mgr.14162) 310 : 2021-11-03T17:14:18.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:18 smithi115 conmon[34670]: cluster [DBG] pgmap v256: 65 pgs: 65 active+clean; 1.9 GiB data, 5.7 GiB used, 531 GiB / 536 GiB avail; 80 MiB/s rd, 162 MiB/s wr, 240 op/s 2021-11-03T17:14:18.985 INFO:tasks.workunit.client.0.smithi027.stdout:3/351: unlink d1/f4b 0 2021-11-03T17:14:18.985 INFO:tasks.workunit.client.0.smithi027.stdout:3/352: write d1/d40/f46 [85712,47317] 0 2021-11-03T17:14:18.988 INFO:tasks.workunit.client.0.smithi027.stdout:2/423: dread d0/d45/f7b [0,4194304] 0 2021-11-03T17:14:18.996 INFO:tasks.workunit.client.0.smithi027.stdout:3/353: getdents d1 0 2021-11-03T17:14:18.997 INFO:tasks.workunit.client.0.smithi027.stdout:3/354: creat d1/d40/f70 x:0 0 0 2021-11-03T17:14:18.997 INFO:tasks.workunit.client.0.smithi027.stdout:2/424: dread d0/d2a/f66 [4194304,4194304] 0 2021-11-03T17:14:18.999 INFO:tasks.workunit.client.0.smithi027.stdout:3/355: creat d1/db/d52/d57/f71 x:0 0 0 2021-11-03T17:14:18.999 INFO:tasks.workunit.client.0.smithi027.stdout:3/356: chown d1/d40/f49 0 1 2021-11-03T17:14:19.000 INFO:tasks.workunit.client.0.smithi027.stdout:2/425: symlink d0/d2a/d5d/l84 0 2021-11-03T17:14:19.002 INFO:tasks.workunit.client.0.smithi027.stdout:3/357: rename d1/db/l54 to d1/db/d52/d6a/l72 0 2021-11-03T17:14:19.002 INFO:tasks.workunit.client.0.smithi027.stdout:3/358: write d1/f2a [4227724,2793] 0 2021-11-03T17:14:19.016 INFO:tasks.workunit.client.0.smithi027.stdout:2/426: dwrite d0/d5/f6d [0,4194304] 0 2021-11-03T17:14:19.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:18 smithi027 conmon[32322]: cluster 2021-11-03T17:14:18.606823+0000 mgr.smithi027.ufhhvb ( 2021-11-03T17:14:19.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:18 smithi027 conmon[32322]: mgr.14162) 310 : cluster [DBG] pgmap v256: 65 pgs: 65 active+clean; 1.9 GiB data, 5.7 GiB used, 531 GiB / 536 GiB avail; 80 MiB/s rd, 162 MiB/s wr, 240 op/s 2021-11-03T17:14:19.025 INFO:tasks.workunit.client.0.smithi027.stdout:3/359: fsync d1/db/d52/d57/f71 0 2021-11-03T17:14:19.026 INFO:tasks.workunit.client.0.smithi027.stdout:3/360: truncate d1/db/d52/d57/f71 418393 0 2021-11-03T17:14:19.026 INFO:tasks.workunit.client.0.smithi027.stdout:2/427: dread d0/d2a/f44 [0,4194304] 0 2021-11-03T17:14:19.027 INFO:tasks.workunit.client.0.smithi027.stdout:2/428: write d0/d1b/d1c/f57 [2855808,36186] 0 2021-11-03T17:14:19.031 INFO:tasks.workunit.client.0.smithi027.stdout:2/429: truncate d0/d1b/d1c/f33 1940367 0 2021-11-03T17:14:19.055 INFO:tasks.workunit.client.0.smithi027.stdout:2/430: dwrite d0/d5/d2c/f3d [0,4194304] 0 2021-11-03T17:14:19.058 INFO:tasks.workunit.client.0.smithi027.stdout:3/361: dwrite d1/f34 [0,4194304] 0 2021-11-03T17:14:19.058 INFO:tasks.workunit.client.0.smithi027.stdout:3/362: fsync d1/db/d52/f5d 0 2021-11-03T17:14:19.059 INFO:tasks.workunit.client.0.smithi027.stdout:3/363: mknod d1/d6f/c73 1 2021-11-03T17:14:19.059 INFO:tasks.workunit.client.0.smithi027.stdout:2/431: symlink d0/d1b/d29/d58/d5b/l85 0 2021-11-03T17:14:19.079 INFO:tasks.workunit.client.0.smithi027.stdout:2/432: dwrite d0/d2a/f44 [4194304,4194304] 0 2021-11-03T17:14:19.079 INFO:tasks.workunit.client.0.smithi027.stdout:3/364: dwrite d1/d40/f4a [0,4194304] 0 2021-11-03T17:14:19.081 INFO:tasks.workunit.client.0.smithi027.stdout:2/433: getdents d0/d2a 0 2021-11-03T17:14:19.085 INFO:tasks.workunit.client.0.smithi027.stdout:2/434: getdents d0/d1b/d29/d58 0 2021-11-03T17:14:19.088 INFO:tasks.workunit.client.0.smithi027.stdout:2/435: rename d0/d5/l32 to d0/d2a/d5d/l86 0 2021-11-03T17:14:19.099 INFO:tasks.workunit.client.0.smithi027.stdout:3/365: dwrite d1/f22 [4194304,4194304] 0 2021-11-03T17:14:19.099 INFO:tasks.workunit.client.0.smithi027.stdout:3/366: read d1/f4c [92420,59393] 0 2021-11-03T17:14:19.099 INFO:tasks.workunit.client.0.smithi027.stdout:3/367: dread - d1/d40/f6d zero size 2021-11-03T17:14:19.100 INFO:tasks.workunit.client.0.smithi027.stdout:3/368: truncate d1/f50 3849109 0 2021-11-03T17:14:19.101 INFO:tasks.workunit.client.0.smithi027.stdout:3/369: rmdir d1/d40 39 2021-11-03T17:14:19.104 INFO:tasks.workunit.client.0.smithi027.stdout:3/370: fsync d1/f50 0 2021-11-03T17:14:19.105 INFO:tasks.workunit.client.0.smithi027.stdout:3/371: rename d1/d40/f58 to d1/f74 0 2021-11-03T17:14:19.112 INFO:tasks.workunit.client.0.smithi027.stdout:2/436: dwrite d0/d1b/d1c/f7e [0,4194304] 0 2021-11-03T17:14:19.113 INFO:tasks.workunit.client.0.smithi027.stdout:2/437: readlink d0/l28 0 2021-11-03T17:14:19.120 INFO:tasks.workunit.client.0.smithi027.stdout:2/438: dread d0/d1b/f6e [0,4194304] 0 2021-11-03T17:14:19.120 INFO:tasks.workunit.client.0.smithi027.stdout:2/439: fdatasync d0/f10 0 2021-11-03T17:14:19.217 INFO:tasks.workunit.client.0.smithi027.stdout:6/460: sync 2021-11-03T17:14:19.237 INFO:tasks.workunit.client.0.smithi027.stdout:6/461: dwrite dc/f10 [0,4194304] 0 2021-11-03T17:14:19.237 INFO:tasks.workunit.client.0.smithi027.stdout:6/462: chown dc/d16/d1b/d2d 13999 1 2021-11-03T17:14:19.238 INFO:tasks.workunit.client.0.smithi027.stdout:6/463: mknod dc/c84 1 2021-11-03T17:14:19.238 INFO:tasks.workunit.client.0.smithi027.stdout:6/464: dread - dc/d16/d1b/d2d/d34/f4e zero size 2021-11-03T17:14:19.238 INFO:tasks.workunit.client.0.smithi027.stdout:6/465: readlink dc/d16/l31 0 2021-11-03T17:14:19.238 INFO:tasks.workunit.client.0.smithi027.stdout:6/466: chown dc/f1f 774430 1 2021-11-03T17:14:19.239 INFO:tasks.workunit.client.0.smithi027.stdout:6/467: dread - dc/d16/d1b/d3c/d51/f73 zero size 2021-11-03T17:14:19.239 INFO:tasks.workunit.client.0.smithi027.stdout:6/468: link dc/f18 dc/d16/d1b/d24/f85 0 2021-11-03T17:14:19.240 INFO:tasks.workunit.client.0.smithi027.stdout:6/469: symlink dc/d16/d1b/d2d/l86 0 2021-11-03T17:14:19.241 INFO:tasks.workunit.client.0.smithi027.stdout:6/470: write dc/d16/f2c [4400383,25793] 0 2021-11-03T17:14:19.242 INFO:tasks.workunit.client.0.smithi027.stdout:6/471: rmdir dc/d16/d30 39 2021-11-03T17:14:19.242 INFO:tasks.workunit.client.0.smithi027.stdout:6/472: read - dc/d16/d1b/d2d/d34/f4e zero size 2021-11-03T17:14:19.244 INFO:tasks.workunit.client.0.smithi027.stdout:6/473: mkdir dc/d82/d87 0 2021-11-03T17:14:19.244 INFO:tasks.workunit.client.0.smithi027.stdout:6/474: truncate dc/d16/d30/f5a 568092 0 2021-11-03T17:14:19.245 INFO:tasks.workunit.client.0.smithi027.stdout:6/475: creat dc/f88 x:0 0 0 2021-11-03T17:14:19.246 INFO:tasks.workunit.client.0.smithi027.stdout:6/476: truncate dc/d16/d30/f32 1856547 0 2021-11-03T17:14:19.249 INFO:tasks.workunit.client.0.smithi027.stdout:6/477: getdents dc/d16/d62 0 2021-11-03T17:14:19.251 INFO:tasks.workunit.client.0.smithi027.stdout:9/494: sync 2021-11-03T17:14:19.280 INFO:tasks.workunit.client.0.smithi027.stdout:9/495: dwrite db/d1b/d2b/d41/f82 [4194304,4194304] 0 2021-11-03T17:14:19.280 INFO:tasks.workunit.client.0.smithi027.stdout:6/478: dwrite dc/d16/d1b/d2d/d34/f4c [0,4194304] 0 2021-11-03T17:14:19.282 INFO:tasks.workunit.client.0.smithi027.stdout:9/496: write db/d1b/d2b/f3f [2821433,3272] 0 2021-11-03T17:14:19.348 INFO:tasks.workunit.client.0.smithi027.stdout:6/479: mkdir dc/d16/d1b/d2d/d34/d89 0 2021-11-03T17:14:19.350 INFO:tasks.workunit.client.0.smithi027.stdout:9/497: creat db/d1b/d1c/d1d/fb6 x:0 0 0 2021-11-03T17:14:19.354 INFO:tasks.workunit.client.0.smithi027.stdout:9/498: unlink db/d1b/d2b/d41/l8b 0 2021-11-03T17:14:19.361 INFO:tasks.workunit.client.0.smithi027.stdout:9/499: fdatasync db/d1b/d2b/f3f 0 2021-11-03T17:14:19.363 INFO:tasks.workunit.client.0.smithi027.stdout:9/500: getdents db/d1b/d1c/d1d/d40/d60 0 2021-11-03T17:14:19.363 INFO:tasks.workunit.client.0.smithi027.stdout:9/501: write db/d1b/f51 [4883735,75541] 0 2021-11-03T17:14:19.363 INFO:tasks.workunit.client.0.smithi027.stdout:9/502: fdatasync db/d1b/d1c/d1d/d40/f70 0 2021-11-03T17:14:19.364 INFO:tasks.workunit.client.0.smithi027.stdout:9/503: chown db/d1b/d1c/d1d/f49 6 1 2021-11-03T17:14:19.364 INFO:tasks.workunit.client.0.smithi027.stdout:9/504: dread - db/d1b/d1c/d1d/fa9 zero size 2021-11-03T17:14:19.364 INFO:tasks.workunit.client.0.smithi027.stdout:9/505: read - db/d1b/d1c/d1d/fa9 zero size 2021-11-03T17:14:19.364 INFO:tasks.workunit.client.0.smithi027.stdout:6/480: dwrite dc/d82/f81 [0,4194304] 0 2021-11-03T17:14:19.387 INFO:tasks.workunit.client.0.smithi027.stdout:9/506: dwrite db/d1b/d2b/f91 [0,4194304] 0 2021-11-03T17:14:19.387 INFO:tasks.workunit.client.0.smithi027.stdout:9/507: chown db/d1b/d8c/l8d 15153293 1 2021-11-03T17:14:19.387 INFO:tasks.workunit.client.0.smithi027.stdout:6/481: dwrite dc/d16/d1b/d24/f2a [0,4194304] 0 2021-11-03T17:14:19.388 INFO:tasks.workunit.client.0.smithi027.stdout:9/508: dread db/d1b/d1c/d1d/f1e [0,4194304] 0 2021-11-03T17:14:19.388 INFO:tasks.workunit.client.0.smithi027.stdout:9/509: fdatasync db/d1b/f64 0 2021-11-03T17:14:19.388 INFO:tasks.workunit.client.0.smithi027.stdout:9/510: readlink db/d1b/d1c/d1d/l85 0 2021-11-03T17:14:19.389 INFO:tasks.workunit.client.0.smithi027.stdout:6/482: creat dc/d16/d1b/d2d/d34/f8a x:0 0 0 2021-11-03T17:14:19.391 INFO:tasks.workunit.client.0.smithi027.stdout:9/511: rmdir db/d1b/d2b 39 2021-11-03T17:14:19.393 INFO:tasks.workunit.client.0.smithi027.stdout:6/483: truncate dc/d16/f2c 3031211 0 2021-11-03T17:14:19.397 INFO:tasks.workunit.client.0.smithi027.stdout:6/484: rmdir dc/d16 39 2021-11-03T17:14:19.400 INFO:tasks.workunit.client.0.smithi027.stdout:9/512: dread db/d1b/d1c/d1d/d40/f53 [0,4194304] 0 2021-11-03T17:14:19.400 INFO:tasks.workunit.client.0.smithi027.stdout:6/485: rename fa to dc/d82/d87/f8b 0 2021-11-03T17:14:19.402 INFO:tasks.workunit.client.0.smithi027.stdout:9/513: creat db/d1b/d2b/d5e/daa/fb7 x:0 0 0 2021-11-03T17:14:19.402 INFO:tasks.workunit.client.0.smithi027.stdout:9/514: stat db/d1b/d2b/f86 0 2021-11-03T17:14:19.402 INFO:tasks.workunit.client.0.smithi027.stdout:9/515: read - db/d1b/d2b/d9c/fab zero size 2021-11-03T17:14:19.403 INFO:tasks.workunit.client.0.smithi027.stdout:9/516: readlink db/d1b/d1c/l50 0 2021-11-03T17:14:19.403 INFO:tasks.workunit.client.0.smithi027.stdout:6/486: rmdir dc/d16/d1b/d2d/d34 39 2021-11-03T17:14:19.404 INFO:tasks.workunit.client.0.smithi027.stdout:6/487: write dc/d16/d1b/d2d/d34/f8a [717052,5233] 0 2021-11-03T17:14:19.404 INFO:tasks.workunit.client.0.smithi027.stdout:6/488: fsync dc/d16/d30/f5a 0 2021-11-03T17:14:19.405 INFO:tasks.workunit.client.0.smithi027.stdout:9/517: unlink db/d1b/d1c/d1d/f4f 0 2021-11-03T17:14:19.409 INFO:tasks.workunit.client.0.smithi027.stdout:9/518: rename db/d1b/d23/d2c/f7d to db/d1b/d2b/d41/fb8 0 2021-11-03T17:14:19.411 INFO:tasks.workunit.client.0.smithi027.stdout:6/489: dread dc/d82/f81 [0,4194304] 0 2021-11-03T17:14:19.411 INFO:tasks.workunit.client.0.smithi027.stdout:6/490: truncate dc/d16/d1b/d2d/d34/f4e 826091 0 2021-11-03T17:14:19.411 INFO:tasks.workunit.client.0.smithi027.stdout:6/491: dread - dc/d82/f6e zero size 2021-11-03T17:14:19.412 INFO:tasks.workunit.client.0.smithi027.stdout:9/519: write db/f14 [6603489,97197] 0 2021-11-03T17:14:19.412 INFO:tasks.workunit.client.0.smithi027.stdout:9/520: write db/d1b/d23/d2c/f89 [1777818,63712] 0 2021-11-03T17:14:19.416 INFO:tasks.workunit.client.0.smithi027.stdout:9/521: creat db/d1b/d2b/d41/fb9 x:0 0 0 2021-11-03T17:14:19.416 INFO:tasks.workunit.client.0.smithi027.stdout:9/522: truncate db/d1b/d23/d2c/f89 3052746 0 2021-11-03T17:14:19.416 INFO:tasks.workunit.client.0.smithi027.stdout:9/523: fdatasync db/f66 0 2021-11-03T17:14:19.433 INFO:tasks.workunit.client.0.smithi027.stdout:6/492: dwrite dc/f88 [0,4194304] 0 2021-11-03T17:14:19.433 INFO:tasks.workunit.client.0.smithi027.stdout:6/493: fsync dc/d16/d1b/d2d/f35 0 2021-11-03T17:14:19.434 INFO:tasks.workunit.client.0.smithi027.stdout:9/524: dwrite db/d1b/d1c/d1d/f8f [0,4194304] 0 2021-11-03T17:14:19.436 INFO:tasks.workunit.client.0.smithi027.stdout:6/494: symlink dc/l8c 0 2021-11-03T17:14:19.436 INFO:tasks.workunit.client.0.smithi027.stdout:6/495: dread - dc/d16/f7a zero size 2021-11-03T17:14:19.436 INFO:tasks.workunit.client.0.smithi027.stdout:6/496: readlink dc/d16/d1b/d3c/l3e 0 2021-11-03T17:14:19.438 INFO:tasks.workunit.client.0.smithi027.stdout:6/497: write dc/f88 [707569,81153] 0 2021-11-03T17:14:19.439 INFO:tasks.workunit.client.0.smithi027.stdout:6/498: fsync dc/d16/d1b/d2d/d34/f4e 0 2021-11-03T17:14:19.442 INFO:tasks.workunit.client.0.smithi027.stdout:6/499: symlink dc/d82/d57/l8d 0 2021-11-03T17:14:19.443 INFO:tasks.workunit.client.0.smithi027.stdout:6/500: mkdir dc/d16/d1b/d24/d8e 0 2021-11-03T17:14:19.445 INFO:tasks.workunit.client.0.smithi027.stdout:6/501: rename dc/d16/d1b/d24/l7c to dc/d16/l8f 0 2021-11-03T17:14:19.448 INFO:tasks.workunit.client.0.smithi027.stdout:9/525: dwrite db/d1b/d23/f7a [0,4194304] 0 2021-11-03T17:14:19.448 INFO:tasks.workunit.client.0.smithi027.stdout:9/526: truncate db/d1b/d2b/d9c/fab 608569 0 2021-11-03T17:14:19.449 INFO:tasks.workunit.client.0.smithi027.stdout:6/502: write dc/d16/d1b/d24/f2a [3442577,71369] 0 2021-11-03T17:14:19.451 INFO:tasks.workunit.client.0.smithi027.stdout:9/527: truncate db/d1b/d23/f4b 815457 0 2021-11-03T17:14:19.454 INFO:tasks.workunit.client.0.smithi027.stdout:6/503: rename dc/d16/d1b/d2d/d34/f4c to dc/d16/d1b/d2d/d34/f90 0 2021-11-03T17:14:19.456 INFO:tasks.workunit.client.0.smithi027.stdout:6/504: write dc/d16/d1b/d24/f85 [842705,53734] 0 2021-11-03T17:14:19.457 INFO:tasks.workunit.client.0.smithi027.stdout:6/505: write dc/d16/d1b/d2d/d34/f8a [173990,18534] 0 2021-11-03T17:14:19.475 INFO:tasks.workunit.client.0.smithi027.stdout:6/506: dwrite dc/d82/f6e [0,4194304] 0 2021-11-03T17:14:19.477 INFO:tasks.workunit.client.0.smithi027.stdout:6/507: rmdir dc/d16/d62 39 2021-11-03T17:14:19.479 INFO:tasks.workunit.client.0.smithi027.stdout:6/508: rename dc/d82/f6e to dc/d82/d57/f91 0 2021-11-03T17:14:19.494 INFO:tasks.workunit.client.0.smithi027.stdout:6/509: dwrite dc/d16/d1b/d3c/d51/f78 [0,4194304] 0 2021-11-03T17:14:19.502 INFO:tasks.workunit.client.0.smithi027.stdout:6/510: dread dc/d16/d1b/d3c/d51/f78 [0,4194304] 0 2021-11-03T17:14:19.503 INFO:tasks.workunit.client.0.smithi027.stdout:6/511: mknod dc/d16/c92 1 2021-11-03T17:14:19.518 INFO:tasks.workunit.client.0.smithi027.stdout:6/512: rmdir dc/d16/d1b/d24/d8e 0 2021-11-03T17:14:19.520 INFO:tasks.workunit.client.0.smithi027.stdout:6/513: rename dc/d16/d1b/d2d/d34/f90 to dc/d16/d1b/d3c/d51/d5f/d79/f93 0 2021-11-03T17:14:19.520 INFO:tasks.workunit.client.0.smithi027.stdout:6/514: rename dc/d16/d1b to dc/d16/d1b/d3c/d51/d5f/d79/d94 22 2021-11-03T17:14:19.521 INFO:tasks.workunit.client.0.smithi027.stdout:6/515: read dc/d16/f40 [2193214,31448] 0 2021-11-03T17:14:19.628 INFO:tasks.workunit.client.0.smithi027.stdout:4/405: sync 2021-11-03T17:14:19.628 INFO:tasks.workunit.client.0.smithi027.stdout:0/563: sync 2021-11-03T17:14:19.628 INFO:tasks.workunit.client.0.smithi027.stdout:4/406: dread - d6/d9/d28/d40/f53 zero size 2021-11-03T17:14:19.629 INFO:tasks.workunit.client.0.smithi027.stdout:4/407: dread - d6/d9/d16/d36/d70/d3e/f6b zero size 2021-11-03T17:14:19.629 INFO:tasks.workunit.client.0.smithi027.stdout:4/408: dread - d6/db/f8a zero size 2021-11-03T17:14:19.646 INFO:tasks.workunit.client.0.smithi027.stdout:0/564: dwrite d9/f1f [0,4194304] 0 2021-11-03T17:14:19.647 INFO:tasks.workunit.client.0.smithi027.stdout:0/565: readlink d9/d12/d8f/l7f 0 2021-11-03T17:14:19.647 INFO:tasks.workunit.client.0.smithi027.stdout:0/566: write d9/da/d64/fbb [784662,98163] 0 2021-11-03T17:14:19.647 INFO:tasks.workunit.client.0.smithi027.stdout:0/567: chown d9/d12/f17 40 1 2021-11-03T17:14:19.649 INFO:tasks.workunit.client.0.smithi027.stdout:0/568: creat d9/da/d64/d70/d6d/d9f/fc3 x:0 0 0 2021-11-03T17:14:19.651 INFO:tasks.workunit.client.0.smithi027.stdout:0/569: creat d9/da/d30/da7/fc4 x:0 0 0 2021-11-03T17:14:19.653 INFO:tasks.workunit.client.0.smithi027.stdout:0/570: rename d9/da/d31/f44 to d9/da/d64/d70/fc5 0 2021-11-03T17:14:19.656 INFO:tasks.workunit.client.0.smithi027.stdout:0/571: rename d9/l14 to d9/da/d30/lc6 0 2021-11-03T17:14:19.663 INFO:tasks.workunit.client.0.smithi027.stdout:8/498: sync 2021-11-03T17:14:19.663 INFO:tasks.workunit.client.0.smithi027.stdout:8/499: write d7/d10/d18/f82 [316677,31188] 0 2021-11-03T17:14:19.664 INFO:tasks.workunit.client.0.smithi027.stdout:8/500: write d7/d10/d18/f61 [3779381,69728] 0 2021-11-03T17:14:19.682 INFO:tasks.workunit.client.0.smithi027.stdout:8/501: dwrite d7/d10/d4f/f85 [0,4194304] 0 2021-11-03T17:14:19.682 INFO:tasks.workunit.client.0.smithi027.stdout:8/502: write d7/d53/d67/f6b [1598801,62694] 0 2021-11-03T17:14:19.687 INFO:tasks.workunit.client.0.smithi027.stdout:8/503: write d7/d10/d3f/d91/f9c [6821278,6796] 0 2021-11-03T17:14:19.707 INFO:tasks.workunit.client.0.smithi027.stdout:8/504: dwrite d7/d10/d18/f82 [0,4194304] 0 2021-11-03T17:14:19.709 INFO:tasks.workunit.client.0.smithi027.stdout:8/505: dread d7/d10/f71 [0,4194304] 0 2021-11-03T17:14:19.710 INFO:tasks.workunit.client.0.smithi027.stdout:8/506: truncate d7/d10/d21/f8c 35344 0 2021-11-03T17:14:19.711 INFO:tasks.workunit.client.0.smithi027.stdout:8/507: link d7/d10/d3f/d55/f9f d7/d10/d21/d41/fa7 0 2021-11-03T17:14:19.719 INFO:tasks.workunit.client.1.smithi115.stdout:9/432: sync 2021-11-03T17:14:19.723 INFO:tasks.workunit.client.1.smithi115.stdout:9/433: link d1/d18/d4a/d54/d3a/f6d d1/d1a/d48/d27/d34/f9c 0 2021-11-03T17:14:19.723 INFO:tasks.workunit.client.1.smithi115.stdout:9/434: readlink d1/l29 0 2021-11-03T17:14:19.725 INFO:tasks.workunit.client.1.smithi115.stdout:9/435: creat d1/d1a/d48/d27/d2f/d41/f9d x:0 0 0 2021-11-03T17:14:19.725 INFO:tasks.workunit.client.1.smithi115.stdout:9/436: write d1/d18/d4a/d74/f7e [753837,109434] 0 2021-11-03T17:14:19.727 INFO:tasks.workunit.client.0.smithi027.stdout:8/508: dwrite d7/f1b [0,4194304] 0 2021-11-03T17:14:19.727 INFO:tasks.workunit.client.0.smithi027.stdout:8/509: dread - d7/d10/d21/d41/fa3 zero size 2021-11-03T17:14:19.728 INFO:tasks.workunit.client.1.smithi115.stdout:9/437: truncate d1/d1a/d48/d27/d2f/f59 3174230 0 2021-11-03T17:14:19.728 INFO:tasks.workunit.client.1.smithi115.stdout:9/438: truncate d1/d1a/d48/d27/d53/f82 4834474 0 2021-11-03T17:14:19.729 INFO:tasks.workunit.client.1.smithi115.stdout:9/439: rmdir d1/d1a/d48/d27/d2f/d7b 39 2021-11-03T17:14:19.732 INFO:tasks.workunit.client.0.smithi027.stdout:8/510: mkdir d7/d2d/da8 0 2021-11-03T17:14:19.733 INFO:tasks.workunit.client.0.smithi027.stdout:8/511: mkdir d7/d2d/da9 0 2021-11-03T17:14:19.734 INFO:tasks.workunit.client.0.smithi027.stdout:8/512: dread d7/d10/d21/f39 [0,4194304] 0 2021-11-03T17:14:19.734 INFO:tasks.workunit.client.0.smithi027.stdout:8/513: stat d7/d10/d18/f48 0 2021-11-03T17:14:19.735 INFO:tasks.workunit.client.0.smithi027.stdout:8/514: write d7/d10/d18/f48 [631407,40407] 0 2021-11-03T17:14:19.738 INFO:tasks.workunit.client.0.smithi027.stdout:8/515: creat d7/d10/faa x:0 0 0 2021-11-03T17:14:19.739 INFO:tasks.workunit.client.0.smithi027.stdout:8/516: write d7/d53/f6c [2517501,45504] 0 2021-11-03T17:14:19.740 INFO:tasks.workunit.client.1.smithi115.stdout:9/440: dread d1/d18/d2b/d42/f12 [0,4194304] 0 2021-11-03T17:14:19.744 INFO:tasks.workunit.client.1.smithi115.stdout:9/441: rename d1/d18/d4a/d54/d3a to d1/d1a/d48/d27/d53/d83/d88/d9e 0 2021-11-03T17:14:19.744 INFO:tasks.workunit.client.1.smithi115.stdout:9/442: getdents d1/d18/d65 0 2021-11-03T17:14:19.745 INFO:tasks.workunit.client.1.smithi115.stdout:9/443: read - d1/d1a/d48/d27/f4c zero size 2021-11-03T17:14:19.746 INFO:tasks.workunit.client.1.smithi115.stdout:9/444: link d1/d18/d2b/d42/f12 d1/d18/d2b/d42/d63/f9f 0 2021-11-03T17:14:19.747 INFO:tasks.workunit.client.0.smithi027.stdout:8/517: dread d7/d53/f6c [4194304,4194304] 0 2021-11-03T17:14:19.747 INFO:tasks.workunit.client.0.smithi027.stdout:8/518: symlink d7/d2d/da9/lab 0 2021-11-03T17:14:19.752 INFO:tasks.workunit.client.0.smithi027.stdout:8/519: dread d7/d10/fa1 [0,4194304] 0 2021-11-03T17:14:19.753 INFO:tasks.workunit.client.0.smithi027.stdout:8/520: write d7/d10/d3f/d55/f86 [748909,82339] 0 2021-11-03T17:14:19.753 INFO:tasks.workunit.client.0.smithi027.stdout:8/521: mknod d7/d10/d3f/d9b/d95/cac 1 2021-11-03T17:14:19.753 INFO:tasks.workunit.client.0.smithi027.stdout:8/522: chown d7/d2d/d38/f3d 1511 1 2021-11-03T17:14:19.754 INFO:tasks.workunit.client.0.smithi027.stdout:8/523: symlink d7/d2d/da8/lad 0 2021-11-03T17:14:19.761 INFO:tasks.workunit.client.1.smithi115.stdout:9/445: dwrite d1/d1a/d48/f26 [0,4194304] 0 2021-11-03T17:14:19.761 INFO:tasks.workunit.client.1.smithi115.stdout:9/446: chown d1/d1a/d48/d27/d53/d83/d88 52871 1 2021-11-03T17:14:19.764 INFO:tasks.workunit.client.1.smithi115.stdout:9/447: symlink d1/d18/d2b/d42/la0 0 2021-11-03T17:14:19.764 INFO:tasks.workunit.client.1.smithi115.stdout:9/448: write d1/f75 [330165,72991] 0 2021-11-03T17:14:19.764 INFO:tasks.workunit.client.1.smithi115.stdout:9/449: mknod d1/d1a/d48/d27/d2f/d6a/ca1 1 2021-11-03T17:14:19.765 INFO:tasks.workunit.client.1.smithi115.stdout:9/450: chown d1/d1a/d48/d27/d53/d83/d88 135675 1 2021-11-03T17:14:19.765 INFO:tasks.workunit.client.1.smithi115.stdout:9/451: getdents d1/d1a 0 2021-11-03T17:14:19.767 INFO:tasks.workunit.client.1.smithi115.stdout:9/452: link d1/d18/d2b/d42/d63/l86 d1/d18/d2b/d42/d6e/la2 0 2021-11-03T17:14:19.768 INFO:tasks.workunit.client.0.smithi027.stdout:8/524: dread d7/d2d/d38/f3d [0,4194304] 0 2021-11-03T17:14:19.769 INFO:tasks.workunit.client.0.smithi027.stdout:8/525: readlink d7/d10/d3f/l59 0 2021-11-03T17:14:19.769 INFO:tasks.workunit.client.0.smithi027.stdout:8/526: mknod d7/d2d/d38/cae 1 2021-11-03T17:14:19.771 INFO:tasks.workunit.client.0.smithi027.stdout:8/527: truncate d7/d10/d3f/f73 1370815 0 2021-11-03T17:14:19.771 INFO:tasks.workunit.client.0.smithi027.stdout:8/528: fdatasync d7/d10/f16 0 2021-11-03T17:14:19.772 INFO:tasks.workunit.client.1.smithi115.stdout:9/453: rename d1/l29 to d1/d18/la3 0 2021-11-03T17:14:19.772 INFO:tasks.workunit.client.0.smithi027.stdout:8/529: write d7/f1d [3727101,83315] 0 2021-11-03T17:14:19.773 INFO:tasks.workunit.client.0.smithi027.stdout:8/530: chown d7/l6d 2673 1 2021-11-03T17:14:19.774 INFO:tasks.workunit.client.0.smithi027.stdout:8/531: fdatasync d7/f1d 0 2021-11-03T17:14:19.775 INFO:tasks.workunit.client.0.smithi027.stdout:8/532: chown d7/d10/d4f/d9d 36120348 1 2021-11-03T17:14:19.776 INFO:tasks.workunit.client.1.smithi115.stdout:9/454: rename d1/d18/d2b/d42/fc to d1/d1a/d48/d27/d53/d83/d88/d9e/d58/fa4 0 2021-11-03T17:14:19.776 INFO:tasks.workunit.client.0.smithi027.stdout:8/533: symlink d7/d53/laf 0 2021-11-03T17:14:19.777 INFO:tasks.workunit.client.0.smithi027.stdout:8/534: write d7/d10/d3f/d9b/f2e [8530161,100614] 0 2021-11-03T17:14:19.777 INFO:tasks.workunit.client.0.smithi027.stdout:8/535: write d7/d10/d21/f63 [2938175,90749] 0 2021-11-03T17:14:19.797 INFO:tasks.workunit.client.0.smithi027.stdout:8/536: dwrite d7/d10/d3f/d9b/d95/f9a [0,4194304] 0 2021-11-03T17:14:19.798 INFO:tasks.workunit.client.1.smithi115.stdout:9/455: dwrite d1/d1a/d48/d27/d2f/d41/f8c [0,4194304] 0 2021-11-03T17:14:19.799 INFO:tasks.workunit.client.1.smithi115.stdout:9/456: read d1/d1a/d48/d27/d2f/f8f [608287,49648] 0 2021-11-03T17:14:19.799 INFO:tasks.workunit.client.1.smithi115.stdout:9/457: chown d1/d1a/d48/d27/d34 52133 1 2021-11-03T17:14:19.799 INFO:tasks.workunit.client.1.smithi115.stdout:9/458: dread - d1/d18/d2b/d42/d6e/f5c zero size 2021-11-03T17:14:19.799 INFO:tasks.workunit.client.1.smithi115.stdout:9/459: chown d1/d1a/d48/d27/d53/d83/d88/d9e/d58 38 1 2021-11-03T17:14:19.799 INFO:tasks.workunit.client.1.smithi115.stdout:9/460: read - d1/d18/d2b/d42/d6e/f5c zero size 2021-11-03T17:14:19.800 INFO:tasks.workunit.client.1.smithi115.stdout:9/461: mknod d1/d18/d2b/d45/ca5 1 2021-11-03T17:14:19.800 INFO:tasks.workunit.client.1.smithi115.stdout:9/462: readlink d1/d18/d2b/l30 0 2021-11-03T17:14:19.802 INFO:tasks.workunit.client.1.smithi115.stdout:9/463: creat d1/d18/d2b/d42/d6e/fa6 x:0 0 0 2021-11-03T17:14:19.802 INFO:tasks.workunit.client.1.smithi115.stdout:9/464: read d1/d1a/d48/d27/d2f/f8f [313818,73302] 0 2021-11-03T17:14:19.803 INFO:tasks.workunit.client.1.smithi115.stdout:9/465: chown d1/d1a/d48/d27/d34/d50/d79 716 1 2021-11-03T17:14:19.803 INFO:tasks.workunit.client.0.smithi027.stdout:8/537: write d7/d10/d3f/f78 [1570784,93582] 0 2021-11-03T17:14:19.803 INFO:tasks.workunit.client.0.smithi027.stdout:8/538: mknod d7/d10/d4f/cb0 1 2021-11-03T17:14:19.804 INFO:tasks.workunit.client.0.smithi027.stdout:8/539: rename d7/d10/d18/d84 to d7/d2d/d38/db1 0 2021-11-03T17:14:19.804 INFO:tasks.workunit.client.0.smithi027.stdout:8/540: creat d7/fb2 x:0 0 0 2021-11-03T17:14:19.804 INFO:tasks.workunit.client.0.smithi027.stdout:8/541: chown d7/d10/d21/d41/l5b 13874375 1 2021-11-03T17:14:19.805 INFO:tasks.workunit.client.0.smithi027.stdout:8/542: dread d7/d10/f16 [0,4194304] 0 2021-11-03T17:14:19.805 INFO:tasks.workunit.client.0.smithi027.stdout:8/543: fsync d7/d10/d21/f45 0 2021-11-03T17:14:19.805 INFO:tasks.workunit.client.1.smithi115.stdout:9/466: unlink d1/d18/d4a/f87 0 2021-11-03T17:14:19.806 INFO:tasks.workunit.client.1.smithi115.stdout:9/467: stat d1/d1a/d48/d27 0 2021-11-03T17:14:19.806 INFO:tasks.workunit.client.1.smithi115.stdout:9/468: mkdir d1/d1a/d48/d27/d80/da7 0 2021-11-03T17:14:19.809 INFO:tasks.workunit.client.0.smithi027.stdout:8/544: truncate d7/d10/d4f/f85 363610 0 2021-11-03T17:14:19.820 INFO:tasks.workunit.client.1.smithi115.stdout:9/469: truncate d1/f75 109869 0 2021-11-03T17:14:19.820 INFO:tasks.workunit.client.1.smithi115.stdout:9/470: write d1/d1a/d48/d27/d53/f77 [675659,4121] 0 2021-11-03T17:14:19.827 INFO:tasks.workunit.client.1.smithi115.stdout:9/471: symlink d1/d1a/d48/d27/d53/d83/d88/d9e/la8 0 2021-11-03T17:14:19.831 INFO:tasks.workunit.client.0.smithi027.stdout:8/545: dwrite d7/d10/d18/f82 [0,4194304] 0 2021-11-03T17:14:19.831 INFO:tasks.workunit.client.0.smithi027.stdout:8/546: write d7/d10/d3f/d55/f86 [1673045,127296] 0 2021-11-03T17:14:19.860 INFO:tasks.workunit.client.0.smithi027.stdout:8/547: dwrite d7/f1d [0,4194304] 0 2021-11-03T17:14:19.860 INFO:tasks.workunit.client.1.smithi115.stdout:9/472: dwrite d1/d1a/f76 [0,4194304] 0 2021-11-03T17:14:19.865 INFO:tasks.workunit.client.1.smithi115.stdout:9/473: rename d1/d1a/d48/d27/d34/d50 to d1/d1a/d48/d27/da9 0 2021-11-03T17:14:19.871 INFO:tasks.workunit.client.1.smithi115.stdout:9/474: dread d1/d1a/d48/d27/d53/f77 [0,4194304] 0 2021-11-03T17:14:19.871 INFO:tasks.workunit.client.0.smithi027.stdout:8/548: dread d7/d53/f6c [0,4194304] 0 2021-11-03T17:14:19.872 INFO:tasks.workunit.client.1.smithi115.stdout:9/475: creat d1/d18/d4a/d74/d8b/faa x:0 0 0 2021-11-03T17:14:19.873 INFO:tasks.workunit.client.1.smithi115.stdout:9/476: write d1/d1a/d48/f73 [481890,52160] 0 2021-11-03T17:14:19.873 INFO:tasks.workunit.client.1.smithi115.stdout:9/477: dread - d1/d1a/d48/d27/d34/f93 zero size 2021-11-03T17:14:19.873 INFO:tasks.workunit.client.1.smithi115.stdout:9/478: chown d1/d1a/d48/d27/d53/f7f 54449 1 2021-11-03T17:14:19.874 INFO:tasks.workunit.client.0.smithi027.stdout:8/549: link d7/l5a d7/d10/d4f/lb3 0 2021-11-03T17:14:19.874 INFO:tasks.workunit.client.0.smithi027.stdout:8/550: write d7/d10/d3f/f78 [4823264,78157] 0 2021-11-03T17:14:19.875 INFO:tasks.workunit.client.0.smithi027.stdout:8/551: getdents d7/d10/d21 0 2021-11-03T17:14:19.876 INFO:tasks.workunit.client.0.smithi027.stdout:8/552: truncate d7/d2d/d38/f3d 2537457 0 2021-11-03T17:14:19.878 INFO:tasks.workunit.client.0.smithi027.stdout:8/553: link f6 d7/d10/d4f/d9d/fb4 0 2021-11-03T17:14:19.889 INFO:tasks.workunit.client.0.smithi027.stdout:8/554: dread d7/d10/d18/f82 [0,4194304] 0 2021-11-03T17:14:19.895 INFO:tasks.workunit.client.1.smithi115.stdout:9/479: dwrite d1/d1a/d48/d27/d2f/f59 [0,4194304] 0 2021-11-03T17:14:19.902 INFO:tasks.workunit.client.0.smithi027.stdout:1/483: sync 2021-11-03T17:14:19.903 INFO:tasks.workunit.client.0.smithi027.stdout:1/484: getdents d6/d11/dac/d79/d31/d58 0 2021-11-03T17:14:19.905 INFO:tasks.workunit.client.0.smithi027.stdout:1/485: link d6/d11/dac/d79/d25/d28/d5c/d81/f86 d6/d11/dac/d79/d25/d28/d5c/d7a/fbe 0 2021-11-03T17:14:19.906 INFO:tasks.workunit.client.0.smithi027.stdout:1/486: read d6/de/d13/d15/f2f [375817,49689] 0 2021-11-03T17:14:19.907 INFO:tasks.workunit.client.0.smithi027.stdout:1/487: rmdir d6/d11/dac/d79/d31 39 2021-11-03T17:14:19.907 INFO:tasks.workunit.client.0.smithi027.stdout:1/488: chown d6/d11/dac/d79/d25/d28/d5c/d68 1154672951 1 2021-11-03T17:14:19.908 INFO:tasks.workunit.client.0.smithi027.stdout:1/489: mkdir d6/d11/dac/d79/db0/dbf 0 2021-11-03T17:14:19.908 INFO:tasks.workunit.client.0.smithi027.stdout:1/490: fdatasync d6/fa 0 2021-11-03T17:14:19.908 INFO:tasks.workunit.client.0.smithi027.stdout:1/491: chown d6/de/d13/f66 391 1 2021-11-03T17:14:19.910 INFO:tasks.workunit.client.0.smithi027.stdout:8/555: dwrite d7/d10/d21/f8c [0,4194304] 0 2021-11-03T17:14:19.914 INFO:tasks.workunit.client.0.smithi027.stdout:8/556: link d7/d10/d4f/l5e d7/d10/d3f/d55/lb5 0 2021-11-03T17:14:19.940 INFO:tasks.workunit.client.0.smithi027.stdout:8/557: creat d7/d2d/d38/db1/fb6 x:0 0 0 2021-11-03T17:14:19.944 INFO:tasks.workunit.client.0.smithi027.stdout:8/558: mkdir d7/d10/d3f/d55/db7 0 2021-11-03T17:14:19.945 INFO:tasks.workunit.client.0.smithi027.stdout:8/559: creat d7/fb8 x:0 0 0 2021-11-03T17:14:19.946 INFO:tasks.workunit.client.0.smithi027.stdout:8/560: mknod d7/d2d/da9/cb9 1 2021-11-03T17:14:19.968 INFO:tasks.workunit.client.0.smithi027.stdout:8/561: symlink d7/d53/d67/lba 0 2021-11-03T17:14:19.973 INFO:tasks.workunit.client.0.smithi027.stdout:8/562: rename d7/d10/d18/f33 to d7/d2d/da9/fbb 0 2021-11-03T17:14:19.981 INFO:tasks.workunit.client.0.smithi027.stdout:8/563: dread d7/d2d/d38/f93 [0,4194304] 0 2021-11-03T17:14:19.981 INFO:tasks.workunit.client.0.smithi027.stdout:8/564: getdents d7/d10/d3f/d9b/d95 0 2021-11-03T17:14:19.983 INFO:tasks.workunit.client.0.smithi027.stdout:8/565: creat d7/d10/d3f/d55/db7/fbc x:0 0 0 2021-11-03T17:14:19.984 INFO:tasks.workunit.client.0.smithi027.stdout:8/566: mkdir d7/d10/dbd 0 2021-11-03T17:14:19.988 INFO:tasks.workunit.client.0.smithi027.stdout:8/567: write d7/f1b [1982437,73850] 0 2021-11-03T17:14:19.988 INFO:tasks.workunit.client.0.smithi027.stdout:8/568: symlink d7/d10/d3f/d55/db7/lbe 0 2021-11-03T17:14:19.991 INFO:tasks.workunit.client.0.smithi027.stdout:8/569: dread d7/d10/d3f/d9b/f60 [0,4194304] 0 2021-11-03T17:14:19.992 INFO:tasks.workunit.client.0.smithi027.stdout:8/570: symlink d7/d10/d3f/d9b/lbf 0 2021-11-03T17:14:19.992 INFO:tasks.workunit.client.0.smithi027.stdout:8/571: getdents d7/d10/d3f/d9b/d95 0 2021-11-03T17:14:19.993 INFO:tasks.workunit.client.0.smithi027.stdout:8/572: dread d7/d10/d3f/d9b/f60 [4194304,4194304] 0 2021-11-03T17:14:19.994 INFO:tasks.workunit.client.0.smithi027.stdout:8/573: mknod d7/d53/d67/cc0 1 2021-11-03T17:14:19.994 INFO:tasks.workunit.client.0.smithi027.stdout:8/574: chown d7/d10/f16 332 1 2021-11-03T17:14:19.995 INFO:tasks.workunit.client.0.smithi027.stdout:8/575: write d7/d53/d67/f6b [426608,102732] 0 2021-11-03T17:14:19.995 INFO:tasks.workunit.client.0.smithi027.stdout:8/576: truncate d7/d53/d67/f6b 2399837 0 2021-11-03T17:14:19.995 INFO:tasks.workunit.client.0.smithi027.stdout:8/577: fdatasync d7/d2d/d38/f3d 0 2021-11-03T17:14:19.995 INFO:tasks.workunit.client.0.smithi027.stdout:8/578: truncate d7/d10/d21/d41/fa7 953852 0 2021-11-03T17:14:20.000 INFO:tasks.workunit.client.0.smithi027.stdout:8/579: dread d7/d2d/d38/f44 [0,4194304] 0 2021-11-03T17:14:20.001 INFO:tasks.workunit.client.0.smithi027.stdout:8/580: fsync d7/d10/d21/f8c 0 2021-11-03T17:14:20.012 INFO:tasks.workunit.client.1.smithi115.stdout:1/423: sync 2021-11-03T17:14:20.014 INFO:tasks.workunit.client.1.smithi115.stdout:1/424: creat d7/dc/dd/d3e/d58/f88 x:0 0 0 2021-11-03T17:14:20.014 INFO:tasks.workunit.client.1.smithi115.stdout:1/425: symlink d7/dc/dd/d24/d4d/l89 0 2021-11-03T17:14:20.015 INFO:tasks.workunit.client.1.smithi115.stdout:1/426: creat d7/f8a x:0 0 0 2021-11-03T17:14:20.016 INFO:tasks.workunit.client.1.smithi115.stdout:1/427: creat d7/f8b x:0 0 0 2021-11-03T17:14:20.017 INFO:tasks.workunit.client.0.smithi027.stdout:8/581: dwrite d7/d10/d21/d41/fa7 [0,4194304] 0 2021-11-03T17:14:20.017 INFO:tasks.workunit.client.0.smithi027.stdout:8/582: chown d7/d10/d3f/d9b/l4b 3389 1 2021-11-03T17:14:20.017 INFO:tasks.workunit.client.0.smithi027.stdout:8/583: chown d7/d10/d3f/d9b/l4b 21604912 1 2021-11-03T17:14:20.018 INFO:tasks.workunit.client.1.smithi115.stdout:1/428: write d7/dc/dd/f5a [176382,28814] 0 2021-11-03T17:14:20.019 INFO:tasks.workunit.client.1.smithi115.stdout:1/429: write f2 [5238956,120017] 0 2021-11-03T17:14:20.019 INFO:tasks.workunit.client.1.smithi115.stdout:1/430: stat d7/dc/l67 0 2021-11-03T17:14:20.019 INFO:tasks.workunit.client.1.smithi115.stdout:1/431: chown d7/dc/ff 1890 1 2021-11-03T17:14:20.020 INFO:tasks.workunit.client.1.smithi115.stdout:1/432: rename d7/dc/f40 to d7/d38/d59/f8c 0 2021-11-03T17:14:20.023 INFO:tasks.workunit.client.0.smithi027.stdout:8/584: dread d7/d10/d3f/f73 [0,4194304] 0 2021-11-03T17:14:20.023 INFO:tasks.workunit.client.0.smithi027.stdout:8/585: chown d7/d10/d3f/d55/f64 16117163 1 2021-11-03T17:14:20.025 INFO:tasks.workunit.client.0.smithi027.stdout:8/586: write d7/d10/d3f/d9b/f1f [1666523,95330] 0 2021-11-03T17:14:20.025 INFO:tasks.workunit.client.0.smithi027.stdout:8/587: stat d7/d10/d4f/d9d/fb4 0 2021-11-03T17:14:20.025 INFO:tasks.workunit.client.0.smithi027.stdout:8/588: chown d7/d10/d3f/d91 2 1 2021-11-03T17:14:20.026 INFO:tasks.workunit.client.0.smithi027.stdout:8/589: fsync d7/d10/d3f/f88 0 2021-11-03T17:14:20.026 INFO:tasks.workunit.client.0.smithi027.stdout:8/590: truncate d7/d10/d3f/d9b/f2e 7656659 0 2021-11-03T17:14:20.027 INFO:tasks.workunit.client.0.smithi027.stdout:8/591: chown l4 108 1 2021-11-03T17:14:20.031 INFO:tasks.workunit.client.0.smithi027.stdout:8/592: rmdir d7/d10 39 2021-11-03T17:14:20.034 INFO:tasks.workunit.client.0.smithi027.stdout:8/593: getdents d7/d10/d3f/d55/db7 0 2021-11-03T17:14:20.035 INFO:tasks.workunit.client.0.smithi027.stdout:8/594: rename d7/d10/f71 to d7/d10/dbd/fc1 0 2021-11-03T17:14:20.035 INFO:tasks.workunit.client.0.smithi027.stdout:8/595: chown d7/d10/d3f/d55/f9f 17 1 2021-11-03T17:14:20.036 INFO:tasks.workunit.client.0.smithi027.stdout:8/596: truncate d7/d53/f6c 1434433 0 2021-11-03T17:14:20.037 INFO:tasks.workunit.client.0.smithi027.stdout:8/597: stat d7/d10/dbd/fc1 0 2021-11-03T17:14:20.038 INFO:tasks.workunit.client.0.smithi027.stdout:8/598: creat d7/d10/d21/d41/fc2 x:0 0 0 2021-11-03T17:14:20.038 INFO:tasks.workunit.client.0.smithi027.stdout:8/599: chown d7/d53/d67 31 1 2021-11-03T17:14:20.040 INFO:tasks.workunit.client.0.smithi027.stdout:8/600: fsync d7/d10/d3f/f78 0 2021-11-03T17:14:20.041 INFO:tasks.workunit.client.0.smithi027.stdout:8/601: stat d7/d53/d67/f6b 0 2021-11-03T17:14:20.042 INFO:tasks.workunit.client.0.smithi027.stdout:8/602: mkdir d7/d2d/d38/dc3 0 2021-11-03T17:14:20.044 INFO:tasks.workunit.client.0.smithi027.stdout:8/603: link d7/d10/d3f/d9b/f60 d7/d2d/d38/fc4 0 2021-11-03T17:14:20.046 INFO:tasks.workunit.client.0.smithi027.stdout:8/604: creat d7/d10/d3f/d55/fc5 x:0 0 0 2021-11-03T17:14:20.059 INFO:tasks.workunit.client.0.smithi027.stdout:8/605: dwrite d7/d53/f6c [0,4194304] 0 2021-11-03T17:14:20.060 INFO:tasks.workunit.client.0.smithi027.stdout:8/606: readlink d7/d10/d3f/d9b/lbf 0 2021-11-03T17:14:20.060 INFO:tasks.workunit.client.0.smithi027.stdout:8/607: truncate d7/d10/d3f/d55/db7/fbc 348887 0 2021-11-03T17:14:20.073 INFO:tasks.workunit.client.0.smithi027.stdout:8/608: dwrite d7/d53/fa4 [0,4194304] 0 2021-11-03T17:14:20.073 INFO:tasks.workunit.client.0.smithi027.stdout:8/609: write d7/d10/d3f/d9b/f3c [4708431,89844] 0 2021-11-03T17:14:20.078 INFO:tasks.workunit.client.0.smithi027.stdout:8/610: write d7/d10/d21/f39 [632261,5347] 0 2021-11-03T17:14:20.078 INFO:tasks.workunit.client.0.smithi027.stdout:8/611: write d7/d10/faa [627944,9459] 0 2021-11-03T17:14:20.079 INFO:tasks.workunit.client.0.smithi027.stdout:8/612: mknod d7/d2d/da9/cc6 1 2021-11-03T17:14:20.079 INFO:tasks.workunit.client.0.smithi027.stdout:8/613: creat d7/d53/d67/fc7 x:0 0 0 2021-11-03T17:14:20.081 INFO:tasks.workunit.client.0.smithi027.stdout:8/614: rename d7/d2d/d38/f93 to d7/fc8 0 2021-11-03T17:14:20.082 INFO:tasks.workunit.client.0.smithi027.stdout:8/615: dread - d7/fb8 zero size 2021-11-03T17:14:20.083 INFO:tasks.workunit.client.0.smithi027.stdout:8/616: fdatasync d7/d10/d21/f39 0 2021-11-03T17:14:20.084 INFO:tasks.workunit.client.0.smithi027.stdout:8/617: write d7/d2d/d38/fc4 [262580,61977] 0 2021-11-03T17:14:20.085 INFO:tasks.workunit.client.0.smithi027.stdout:8/618: write d7/d10/d4f/d9d/fb4 [3575249,78248] 0 2021-11-03T17:14:20.085 INFO:tasks.workunit.client.0.smithi027.stdout:8/619: chown d7/d10/d21/f63 9 1 2021-11-03T17:14:20.087 INFO:tasks.workunit.client.0.smithi027.stdout:8/620: mkdir d7/d10/d3f/d55/db7/dc9 0 2021-11-03T17:14:20.087 INFO:tasks.workunit.client.0.smithi027.stdout:8/621: write d7/d10/d21/f8d [4155165,53822] 0 2021-11-03T17:14:20.089 INFO:tasks.workunit.client.1.smithi115.stdout:1/433: creat d7/dc/dd/d42/f8d x:0 0 0 2021-11-03T17:14:20.090 INFO:tasks.workunit.client.0.smithi027.stdout:8/622: creat d7/d10/fca x:0 0 0 2021-11-03T17:14:20.090 INFO:tasks.workunit.client.1.smithi115.stdout:1/434: read d7/dc/dd/d24/d4d/f7b [59861,46851] 0 2021-11-03T17:14:20.091 INFO:tasks.workunit.client.0.smithi027.stdout:8/623: creat d7/d10/fcb x:0 0 0 2021-11-03T17:14:20.091 INFO:tasks.workunit.client.0.smithi027.stdout:8/624: dread - d7/fb2 zero size 2021-11-03T17:14:20.091 INFO:tasks.workunit.client.0.smithi027.stdout:8/625: write d7/d10/d3f/d9b/d95/f7e [540691,7398] 0 2021-11-03T17:14:20.092 INFO:tasks.workunit.client.1.smithi115.stdout:1/435: link d7/dc/dd/d24/d4b/f3a d7/dc/dd/d24/d4b/f8e 0 2021-11-03T17:14:20.092 INFO:tasks.workunit.client.1.smithi115.stdout:1/436: read d7/dc/f2f [2306295,128317] 0 2021-11-03T17:14:20.093 INFO:tasks.workunit.client.1.smithi115.stdout:1/437: write d7/dc/dd/d24/d26/f3b [5647840,103902] 0 2021-11-03T17:14:20.093 INFO:tasks.workunit.client.1.smithi115.stdout:1/438: fdatasync d7/dc/dd/d24/f2b 0 2021-11-03T17:14:20.094 INFO:tasks.workunit.client.0.smithi027.stdout:8/626: fdatasync d7/d10/d3f/d9b/f60 0 2021-11-03T17:14:20.094 INFO:tasks.workunit.client.1.smithi115.stdout:1/439: mkdir d7/dc/dd/d24/d4d/d60/d8f 0 2021-11-03T17:14:20.096 INFO:tasks.workunit.client.0.smithi027.stdout:8/627: rename d7/d10/d3f/d91 to d7/dcc 0 2021-11-03T17:14:20.119 INFO:tasks.workunit.client.0.smithi027.stdout:8/628: dwrite d7/d10/d18/f82 [0,4194304] 0 2021-11-03T17:14:20.121 INFO:tasks.workunit.client.0.smithi027.stdout:8/629: creat d7/d10/d21/fcd x:0 0 0 2021-11-03T17:14:20.123 INFO:tasks.workunit.client.0.smithi027.stdout:8/630: creat d7/d53/fce x:0 0 0 2021-11-03T17:14:20.123 INFO:tasks.workunit.client.0.smithi027.stdout:8/631: truncate d7/f1b 5307740 0 2021-11-03T17:14:20.126 INFO:tasks.workunit.client.0.smithi027.stdout:8/632: rename d7/d10/d4f to d7/dcc/dcf 0 2021-11-03T17:14:20.127 INFO:tasks.workunit.client.0.smithi027.stdout:8/633: creat d7/d10/d3f/d55/db7/dc9/fd0 x:0 0 0 2021-11-03T17:14:20.128 INFO:tasks.workunit.client.1.smithi115.stdout:1/440: dwrite d7/dc/f87 [0,4194304] 0 2021-11-03T17:14:20.151 INFO:tasks.workunit.client.0.smithi027.stdout:8/634: dwrite d7/d10/d3f/d9b/d95/f7e [0,4194304] 0 2021-11-03T17:14:20.153 INFO:tasks.workunit.client.1.smithi115.stdout:1/441: dwrite d7/dc/d2a/f33 [0,4194304] 0 2021-11-03T17:14:20.154 INFO:tasks.workunit.client.0.smithi027.stdout:8/635: creat d7/d10/d21/d7f/fd1 x:0 0 0 2021-11-03T17:14:20.155 INFO:tasks.workunit.client.0.smithi027.stdout:8/636: read d7/f36 [483658,70630] 0 2021-11-03T17:14:20.156 INFO:tasks.workunit.client.0.smithi027.stdout:8/637: creat d7/d10/dbd/fd2 x:0 0 0 2021-11-03T17:14:20.158 INFO:tasks.workunit.client.1.smithi115.stdout:1/442: rename d7/dc/dd/d24/d4b/d55/f5d to d7/f90 0 2021-11-03T17:14:20.159 INFO:tasks.workunit.client.0.smithi027.stdout:8/638: fdatasync d7/f1b 0 2021-11-03T17:14:20.159 INFO:tasks.workunit.client.0.smithi027.stdout:8/639: dread - d7/d53/d67/fc7 zero size 2021-11-03T17:14:20.161 INFO:tasks.workunit.client.1.smithi115.stdout:1/443: creat d7/dc/dd/d24/d4d/d60/f91 x:0 0 0 2021-11-03T17:14:20.161 INFO:tasks.workunit.client.1.smithi115.stdout:1/444: readlink d7/dc/dd/d24/d26/l29 0 2021-11-03T17:14:20.162 INFO:tasks.workunit.client.1.smithi115.stdout:1/445: write d7/dc/dd/d3e/d58/f64 [1326132,104123] 0 2021-11-03T17:14:20.169 INFO:tasks.workunit.client.0.smithi027.stdout:8/640: rmdir d7/d10 39 2021-11-03T17:14:20.169 INFO:tasks.workunit.client.0.smithi027.stdout:8/641: write d7/d10/f12 [1205804,93635] 0 2021-11-03T17:14:20.171 INFO:tasks.workunit.client.0.smithi027.stdout:8/642: link d7/d10/d18/f82 d7/d10/d3f/d9b/fd3 0 2021-11-03T17:14:20.189 INFO:tasks.workunit.client.0.smithi027.stdout:8/643: dwrite d7/f1b [4194304,4194304] 0 2021-11-03T17:14:20.209 INFO:tasks.workunit.client.1.smithi115.stdout:0/569: sync 2021-11-03T17:14:20.209 INFO:tasks.workunit.client.0.smithi027.stdout:8/644: dwrite d7/d53/fce [0,4194304] 0 2021-11-03T17:14:20.212 INFO:tasks.workunit.client.0.smithi027.stdout:8/645: unlink d7/d10/d21/f99 0 2021-11-03T17:14:20.215 INFO:tasks.workunit.client.0.smithi027.stdout:8/646: link d7/f87 d7/d10/d3f/d9b/fd4 0 2021-11-03T17:14:20.215 INFO:tasks.workunit.client.0.smithi027.stdout:8/647: dread - d7/d10/dbd/fd2 zero size 2021-11-03T17:14:20.215 INFO:tasks.workunit.client.0.smithi027.stdout:8/648: write d7/d10/d21/f39 [1598133,116624] 0 2021-11-03T17:14:20.216 INFO:tasks.workunit.client.0.smithi027.stdout:8/649: creat d7/d10/d3f/d9b/d95/fd5 x:0 0 0 2021-11-03T17:14:20.231 INFO:tasks.workunit.client.0.smithi027.stdout:8/650: dwrite d7/d53/fa4 [0,4194304] 0 2021-11-03T17:14:20.231 INFO:tasks.workunit.client.0.smithi027.stdout:8/651: readlink d7/d10/d3f/l52 0 2021-11-03T17:14:20.239 INFO:tasks.workunit.client.1.smithi115.stdout:4/573: sync 2021-11-03T17:14:20.239 INFO:tasks.workunit.client.1.smithi115.stdout:6/381: sync 2021-11-03T17:14:20.239 INFO:tasks.workunit.client.1.smithi115.stdout:3/481: sync 2021-11-03T17:14:20.240 INFO:tasks.workunit.client.1.smithi115.stdout:4/574: chown d7/d7a/d83/f64 161 1 2021-11-03T17:14:20.240 INFO:tasks.workunit.client.1.smithi115.stdout:5/582: sync 2021-11-03T17:14:20.240 INFO:tasks.workunit.client.1.smithi115.stdout:7/374: sync 2021-11-03T17:14:20.240 INFO:tasks.workunit.client.1.smithi115.stdout:7/375: stat d3/da/d18 0 2021-11-03T17:14:20.242 INFO:tasks.workunit.client.1.smithi115.stdout:4/575: creat d7/dc/fc8 x:0 0 0 2021-11-03T17:14:20.243 INFO:tasks.workunit.client.1.smithi115.stdout:6/382: dread d0/d1/d10/f3b [0,4194304] 0 2021-11-03T17:14:20.243 INFO:tasks.workunit.client.1.smithi115.stdout:6/383: fdatasync d0/d1/d2/f58 0 2021-11-03T17:14:20.244 INFO:tasks.workunit.client.1.smithi115.stdout:4/576: dread d7/dc/d1a/d21/db1/d56/d63/d7b/d96/fa7 [0,4194304] 0 2021-11-03T17:14:20.245 INFO:tasks.workunit.client.1.smithi115.stdout:5/583: creat d1a/d3d/d58/fb9 x:0 0 0 2021-11-03T17:14:20.247 INFO:tasks.workunit.client.1.smithi115.stdout:4/577: write d7/fa0 [1532971,79903] 0 2021-11-03T17:14:20.253 INFO:tasks.workunit.client.1.smithi115.stdout:6/384: dread d0/d1/d10/d13/d59/f67 [0,4194304] 0 2021-11-03T17:14:20.255 INFO:tasks.workunit.client.0.smithi027.stdout:8/652: dwrite d7/dcc/dcf/f85 [0,4194304] 0 2021-11-03T17:14:20.256 INFO:tasks.workunit.client.0.smithi027.stdout:8/653: stat d7/d2d/l79 0 2021-11-03T17:14:20.257 INFO:tasks.workunit.client.1.smithi115.stdout:6/385: unlink d0/d1/d2e/f60 0 2021-11-03T17:14:20.258 INFO:tasks.workunit.client.0.smithi027.stdout:8/654: symlink d7/d53/d67/ld6 0 2021-11-03T17:14:20.258 INFO:tasks.workunit.client.0.smithi027.stdout:8/655: fdatasync d7/d10/d18/f48 0 2021-11-03T17:14:20.259 INFO:tasks.workunit.client.1.smithi115.stdout:6/386: write d0/d1/d2/f12 [1258712,106469] 0 2021-11-03T17:14:20.261 INFO:tasks.workunit.client.0.smithi027.stdout:8/656: rename d7/d10/d3f/d55/f64 to d7/d10/fd7 0 2021-11-03T17:14:20.261 INFO:tasks.workunit.client.0.smithi027.stdout:8/657: write d7/fc8 [930983,80056] 0 2021-11-03T17:14:20.262 INFO:tasks.workunit.client.0.smithi027.stdout:8/658: rmdir d7/d10 39 2021-11-03T17:14:20.262 INFO:tasks.workunit.client.0.smithi027.stdout:8/659: write d7/d2d/d38/f3d [3317015,23891] 0 2021-11-03T17:14:20.263 INFO:tasks.workunit.client.1.smithi115.stdout:7/376: dwrite d3/da/f64 [4194304,4194304] 0 2021-11-03T17:14:20.263 INFO:tasks.workunit.client.1.smithi115.stdout:7/377: truncate d3/da/f69 460888 0 2021-11-03T17:14:20.264 INFO:tasks.workunit.client.0.smithi027.stdout:8/660: rename d7/d2d/da8/lad to d7/d2d/d38/db1/ld8 0 2021-11-03T17:14:20.266 INFO:tasks.workunit.client.0.smithi027.stdout:8/661: mkdir d7/d10/d21/d7f/dd9 0 2021-11-03T17:14:20.267 INFO:tasks.workunit.client.1.smithi115.stdout:7/378: rename d3/l7 to d3/da/d18/d73/d33/l77 0 2021-11-03T17:14:20.270 INFO:tasks.workunit.client.1.smithi115.stdout:4/578: dwrite d7/dc/d6a/fac [0,4194304] 0 2021-11-03T17:14:20.270 INFO:tasks.workunit.client.1.smithi115.stdout:3/482: dwrite fc [4194304,4194304] 0 2021-11-03T17:14:20.271 INFO:tasks.workunit.client.0.smithi027.stdout:8/662: write d7/d10/d3f/d55/f86 [1488599,102002] 0 2021-11-03T17:14:20.273 INFO:tasks.workunit.client.0.smithi027.stdout:8/663: link d7/d10/d3f/d55/fa6 d7/d10/d21/fda 0 2021-11-03T17:14:20.277 INFO:tasks.workunit.client.1.smithi115.stdout:3/483: creat d13/d35/d54/fa0 x:0 0 0 2021-11-03T17:14:20.278 INFO:tasks.workunit.client.0.smithi027.stdout:8/664: write d7/d10/fa1 [4302989,44713] 0 2021-11-03T17:14:20.280 INFO:tasks.workunit.client.0.smithi027.stdout:8/665: mkdir d7/d10/d3f/d55/ddb 0 2021-11-03T17:14:20.281 INFO:tasks.workunit.client.0.smithi027.stdout:8/666: creat d7/d2d/d38/dc3/fdc x:0 0 0 2021-11-03T17:14:20.281 INFO:tasks.workunit.client.0.smithi027.stdout:8/667: stat d7/d10/d21/f39 0 2021-11-03T17:14:20.283 INFO:tasks.workunit.client.1.smithi115.stdout:6/387: dwrite d0/d1/d10/d13/f39 [0,4194304] 0 2021-11-03T17:14:20.283 INFO:tasks.workunit.client.1.smithi115.stdout:6/388: write d0/f4d [183389,67126] 0 2021-11-03T17:14:20.284 INFO:tasks.workunit.client.0.smithi027.stdout:8/668: write d7/f62 [1195602,11121] 0 2021-11-03T17:14:20.284 INFO:tasks.workunit.client.0.smithi027.stdout:8/669: dread - d7/fb2 zero size 2021-11-03T17:14:20.285 INFO:tasks.workunit.client.0.smithi027.stdout:8/670: stat d7/d10/d3f/d55/db7 0 2021-11-03T17:14:20.285 INFO:tasks.workunit.client.0.smithi027.stdout:8/671: chown d7/d10/d3f/d9b/f3e 45982 1 2021-11-03T17:14:20.285 INFO:tasks.workunit.client.1.smithi115.stdout:6/389: symlink d0/d1/d2/l7d 0 2021-11-03T17:14:20.294 INFO:tasks.workunit.client.1.smithi115.stdout:4/579: dwrite d7/dc/d1a/f38 [4194304,4194304] 0 2021-11-03T17:14:20.295 INFO:tasks.workunit.client.1.smithi115.stdout:3/484: dread d13/d67/f6c [0,4194304] 0 2021-11-03T17:14:20.295 INFO:tasks.workunit.client.1.smithi115.stdout:3/485: fsync d13/d35/f3a 0 2021-11-03T17:14:20.295 INFO:tasks.workunit.client.1.smithi115.stdout:4/580: dread - d7/dc/d1a/d21/fa1 zero size 2021-11-03T17:14:20.295 INFO:tasks.workunit.client.1.smithi115.stdout:4/581: chown d7/dc/d1a/d21/db1/d56/f93 3130 1 2021-11-03T17:14:20.296 INFO:tasks.workunit.client.1.smithi115.stdout:4/582: creat d7/dc/db6/fc9 x:0 0 0 2021-11-03T17:14:20.305 INFO:tasks.workunit.client.1.smithi115.stdout:3/486: dread d13/d50/f4d [0,4194304] 0 2021-11-03T17:14:20.307 INFO:tasks.workunit.client.1.smithi115.stdout:3/487: unlink d13/d35/d54/l9b 0 2021-11-03T17:14:20.308 INFO:tasks.workunit.client.1.smithi115.stdout:3/488: creat d13/d3b/fa1 x:0 0 0 2021-11-03T17:14:20.308 INFO:tasks.workunit.client.1.smithi115.stdout:3/489: dread - d13/d50/f42 zero size 2021-11-03T17:14:20.312 INFO:tasks.workunit.client.1.smithi115.stdout:4/583: dread d7/dc/f17 [0,4194304] 0 2021-11-03T17:14:20.313 INFO:tasks.workunit.client.1.smithi115.stdout:4/584: truncate d7/dc/fc8 178005 0 2021-11-03T17:14:20.313 INFO:tasks.workunit.client.1.smithi115.stdout:6/390: dwrite d0/f6 [0,4194304] 0 2021-11-03T17:14:20.314 INFO:tasks.workunit.client.1.smithi115.stdout:6/391: truncate d0/f40 758887 0 2021-11-03T17:14:20.316 INFO:tasks.workunit.client.1.smithi115.stdout:6/392: creat d0/f7e x:0 0 0 2021-11-03T17:14:20.316 INFO:tasks.workunit.client.1.smithi115.stdout:6/393: write d0/d1/d10/d13/d59/f64 [1264598,92070] 0 2021-11-03T17:14:20.317 INFO:tasks.workunit.client.1.smithi115.stdout:6/394: truncate d0/d1/d2/f4a 837794 0 2021-11-03T17:14:20.318 INFO:tasks.workunit.client.1.smithi115.stdout:6/395: mkdir d0/d7f 0 2021-11-03T17:14:20.320 INFO:tasks.workunit.client.1.smithi115.stdout:6/396: unlink d0/d1/d10/d13/d1c/d1e/l7b 0 2021-11-03T17:14:20.321 INFO:tasks.workunit.client.1.smithi115.stdout:6/397: creat d0/d1/d10/d13/d59/f80 x:0 0 0 2021-11-03T17:14:20.335 INFO:tasks.workunit.client.1.smithi115.stdout:4/585: dwrite d7/dc/f13 [0,4194304] 0 2021-11-03T17:14:20.335 INFO:tasks.workunit.client.1.smithi115.stdout:4/586: fdatasync d7/d7a/f91 0 2021-11-03T17:14:20.336 INFO:tasks.workunit.client.1.smithi115.stdout:4/587: chown d7/fb5 234377 1 2021-11-03T17:14:20.336 INFO:tasks.workunit.client.1.smithi115.stdout:4/588: chown d7/dc/f13 11 1 2021-11-03T17:14:20.336 INFO:tasks.workunit.client.1.smithi115.stdout:4/589: mknod d7/dc/d1a/d21/db1/d56/db3/cca 1 2021-11-03T17:14:20.337 INFO:tasks.workunit.client.1.smithi115.stdout:6/398: dwrite d0/d1/d10/d13/d1c/f2f [0,4194304] 0 2021-11-03T17:14:20.338 INFO:tasks.workunit.client.1.smithi115.stdout:4/590: symlink d7/dc/d6a/dab/lcb 0 2021-11-03T17:14:20.338 INFO:tasks.workunit.client.1.smithi115.stdout:4/591: write d7/dc/d1a/d21/db1/d56/d42/fa5 [650585,84970] 0 2021-11-03T17:14:20.338 INFO:tasks.workunit.client.1.smithi115.stdout:4/592: chown d7/dc/l35 36385 1 2021-11-03T17:14:20.341 INFO:tasks.workunit.client.1.smithi115.stdout:6/399: symlink d0/d1/d2/d3f/l81 0 2021-11-03T17:14:20.341 INFO:tasks.workunit.client.1.smithi115.stdout:6/400: mknod d0/d1/d2e/c82 1 2021-11-03T17:14:20.342 INFO:tasks.workunit.client.1.smithi115.stdout:4/593: mkdir d7/dc/d6a/d3c/dc5/dcc 0 2021-11-03T17:14:20.343 INFO:tasks.workunit.client.1.smithi115.stdout:6/401: creat d0/d1/d2e/f83 x:0 0 0 2021-11-03T17:14:20.344 INFO:tasks.workunit.client.1.smithi115.stdout:6/402: getdents d0/d1 0 2021-11-03T17:14:20.344 INFO:tasks.workunit.client.1.smithi115.stdout:6/403: readlink d0/d1/d10/d13/d59/l7a 0 2021-11-03T17:14:20.344 INFO:tasks.workunit.client.1.smithi115.stdout:6/404: truncate d0/d1/d2/f35 946142 0 2021-11-03T17:14:20.345 INFO:tasks.workunit.client.1.smithi115.stdout:6/405: symlink d0/d1/d2e/d4c/l84 0 2021-11-03T17:14:20.346 INFO:tasks.workunit.client.1.smithi115.stdout:6/406: truncate d0/d1/d10/d13/d1c/f36 12382182 0 2021-11-03T17:14:20.346 INFO:tasks.workunit.client.1.smithi115.stdout:6/407: truncate d0/f4d 520095 0 2021-11-03T17:14:20.349 INFO:tasks.workunit.client.1.smithi115.stdout:6/408: rename d0/d1/d6a/d7c to d0/d85 0 2021-11-03T17:14:20.350 INFO:tasks.workunit.client.1.smithi115.stdout:4/594: dread d7/dc/f95 [0,4194304] 0 2021-11-03T17:14:20.351 INFO:tasks.workunit.client.1.smithi115.stdout:3/490: dwrite d13/d51/d2e/f52 [0,4194304] 0 2021-11-03T17:14:20.351 INFO:tasks.workunit.client.1.smithi115.stdout:6/409: creat d0/d1/d2/f86 x:0 0 0 2021-11-03T17:14:20.354 INFO:tasks.workunit.client.1.smithi115.stdout:3/491: link d13/d35/f93 d13/d51/d2e/fa2 0 2021-11-03T17:14:20.357 INFO:tasks.workunit.client.1.smithi115.stdout:3/492: rmdir d13/d51 39 2021-11-03T17:14:20.363 INFO:tasks.workunit.client.1.smithi115.stdout:6/410: dread d0/d1/d2/ff [0,4194304] 0 2021-11-03T17:14:20.373 INFO:tasks.workunit.client.1.smithi115.stdout:4/595: dread d7/dc/d1a/f49 [0,4194304] 0 2021-11-03T17:14:20.374 INFO:tasks.workunit.client.1.smithi115.stdout:3/493: dwrite d13/d2a/f9a [0,4194304] 0 2021-11-03T17:14:20.378 INFO:tasks.workunit.client.1.smithi115.stdout:6/411: dread d0/d1/d10/d13/d1c/f21 [0,4194304] 0 2021-11-03T17:14:20.378 INFO:tasks.workunit.client.1.smithi115.stdout:6/412: chown d0/d1/d10/d13/d59/f64 30548 1 2021-11-03T17:14:20.378 INFO:tasks.workunit.client.1.smithi115.stdout:3/494: unlink d13/d51/d2e/f52 0 2021-11-03T17:14:20.380 INFO:tasks.workunit.client.1.smithi115.stdout:3/495: creat d13/d35/fa3 x:0 0 0 2021-11-03T17:14:20.382 INFO:tasks.workunit.client.1.smithi115.stdout:3/496: mkdir d13/da4 0 2021-11-03T17:14:20.382 INFO:tasks.workunit.client.1.smithi115.stdout:6/413: dread d0/f4f [0,4194304] 0 2021-11-03T17:14:20.383 INFO:tasks.workunit.client.1.smithi115.stdout:3/497: symlink d13/d2a/d53/d70/la5 0 2021-11-03T17:14:20.384 INFO:tasks.workunit.client.1.smithi115.stdout:3/498: creat d13/d2a/d8a/fa6 x:0 0 0 2021-11-03T17:14:20.384 INFO:tasks.workunit.client.1.smithi115.stdout:3/499: write d13/d35/f69 [406078,103645] 0 2021-11-03T17:14:20.385 INFO:tasks.workunit.client.1.smithi115.stdout:3/500: truncate d13/d51/d2e/fa2 636852 0 2021-11-03T17:14:20.390 INFO:tasks.workunit.client.1.smithi115.stdout:3/501: link d13/f48 d13/d2a/d8a/fa7 0 2021-11-03T17:14:20.397 INFO:tasks.workunit.client.1.smithi115.stdout:4/596: dwrite d7/dc/d1a/d21/db1/d56/d63/f87 [0,4194304] 0 2021-11-03T17:14:20.397 INFO:tasks.workunit.client.1.smithi115.stdout:4/597: mknod d7/dc/d1a/d21/db1/d56/d63/d7b/d96/ccd 1 2021-11-03T17:14:20.398 INFO:tasks.workunit.client.1.smithi115.stdout:4/598: read d7/dc/d1a/d21/db1/d56/d63/f9b [1125425,675] 0 2021-11-03T17:14:20.399 INFO:tasks.workunit.client.1.smithi115.stdout:4/599: fdatasync d7/dc/d1a/d21/db1/d56/d63/f9b 0 2021-11-03T17:14:20.399 INFO:tasks.workunit.client.1.smithi115.stdout:4/600: rename d7/dc/d6a/l4b to d7/dc/d6a/dab/lce 0 2021-11-03T17:14:20.402 INFO:tasks.workunit.client.1.smithi115.stdout:4/601: write d7/dc/db6/f9a [3482147,18022] 0 2021-11-03T17:14:20.402 INFO:tasks.workunit.client.1.smithi115.stdout:4/602: chown d7/d7a 968426 1 2021-11-03T17:14:20.402 INFO:tasks.workunit.client.1.smithi115.stdout:4/603: readlink d7/dc/d1a/d21/l31 0 2021-11-03T17:14:20.403 INFO:tasks.workunit.client.1.smithi115.stdout:4/604: rmdir d7/dc/d1a/d21/db1/d56/db3 39 2021-11-03T17:14:20.403 INFO:tasks.workunit.client.1.smithi115.stdout:4/605: fdatasync d7/dc/f12 0 2021-11-03T17:14:20.404 INFO:tasks.workunit.client.1.smithi115.stdout:6/414: dwrite d0/d1/d2/f86 [0,4194304] 0 2021-11-03T17:14:20.404 INFO:tasks.workunit.client.1.smithi115.stdout:6/415: truncate d0/d1/d10/f53 196403 0 2021-11-03T17:14:20.405 INFO:tasks.workunit.client.1.smithi115.stdout:6/416: truncate d0/d1/d10/d13/d1c/f2f 5007360 0 2021-11-03T17:14:20.409 INFO:tasks.workunit.client.1.smithi115.stdout:3/502: dwrite d13/f4c [4194304,4194304] 0 2021-11-03T17:14:20.409 INFO:tasks.workunit.client.1.smithi115.stdout:3/503: fdatasync d13/d67/f7e 0 2021-11-03T17:14:20.409 INFO:tasks.workunit.client.1.smithi115.stdout:3/504: write d13/d2a/d60/f84 [45362,40454] 0 2021-11-03T17:14:20.412 INFO:tasks.workunit.client.1.smithi115.stdout:4/606: fdatasync d7/dc/db6/f9a 0 2021-11-03T17:14:20.413 INFO:tasks.workunit.client.1.smithi115.stdout:4/607: readlink d7/dc/d1a/d21/l29 0 2021-11-03T17:14:20.414 INFO:tasks.workunit.client.1.smithi115.stdout:4/608: rename d7/dc/l90 to d7/dc/d1a/d21/db1/d56/d63/d7b/lcf 0 2021-11-03T17:14:20.414 INFO:tasks.workunit.client.1.smithi115.stdout:4/609: creat d7/d7a/d83/fd0 x:0 0 0 2021-11-03T17:14:20.416 INFO:tasks.workunit.client.1.smithi115.stdout:4/610: mkdir d7/dd1 0 2021-11-03T17:14:20.419 INFO:tasks.workunit.client.1.smithi115.stdout:3/505: dread f10 [0,4194304] 0 2021-11-03T17:14:20.419 INFO:tasks.workunit.client.1.smithi115.stdout:3/506: mknod d13/d2a/d53/d70/ca8 1 2021-11-03T17:14:20.419 INFO:tasks.workunit.client.1.smithi115.stdout:3/507: write d13/d35/d54/fa0 [177902,77359] 0 2021-11-03T17:14:20.423 INFO:tasks.workunit.client.1.smithi115.stdout:4/611: rename d7/dc/d1a/l80 to d7/dc/ld2 0 2021-11-03T17:14:20.423 INFO:tasks.workunit.client.1.smithi115.stdout:3/508: write d13/d35/f3a [1404204,80762] 0 2021-11-03T17:14:20.425 INFO:tasks.workunit.client.1.smithi115.stdout:6/417: dwrite d0/d1/d2e/d4c/f4e [0,4194304] 0 2021-11-03T17:14:20.426 INFO:tasks.workunit.client.1.smithi115.stdout:3/509: rename d13/d2a/f9a to d13/fa9 0 2021-11-03T17:14:20.430 INFO:tasks.workunit.client.1.smithi115.stdout:6/418: rmdir d0/d1/d10/d13 39 2021-11-03T17:14:20.431 INFO:tasks.workunit.client.1.smithi115.stdout:3/510: creat d13/d2a/d53/d70/d92/faa x:0 0 0 2021-11-03T17:14:20.438 INFO:tasks.workunit.client.1.smithi115.stdout:3/511: link d13/d51/d2e/f37 d13/d35/d54/fab 0 2021-11-03T17:14:20.439 INFO:tasks.workunit.client.1.smithi115.stdout:3/512: chown d13/d51/d40/l6e 684318 1 2021-11-03T17:14:20.440 INFO:tasks.workunit.client.1.smithi115.stdout:3/513: symlink d13/d3b/lac 0 2021-11-03T17:14:20.442 INFO:tasks.workunit.client.1.smithi115.stdout:3/514: read d13/f22 [101844,48680] 0 2021-11-03T17:14:20.445 INFO:tasks.workunit.client.1.smithi115.stdout:3/515: mkdir d13/d35/d54/dad 0 2021-11-03T17:14:20.447 INFO:tasks.workunit.client.1.smithi115.stdout:3/516: unlink d13/d3b/f89 0 2021-11-03T17:14:20.448 INFO:tasks.workunit.client.1.smithi115.stdout:3/517: symlink d13/lae 0 2021-11-03T17:14:20.453 INFO:tasks.workunit.client.1.smithi115.stdout:6/419: dwrite d0/f22 [0,4194304] 0 2021-11-03T17:14:20.456 INFO:tasks.workunit.client.1.smithi115.stdout:6/420: mkdir d0/d1/d10/d13/d87 0 2021-11-03T17:14:20.456 INFO:tasks.workunit.client.1.smithi115.stdout:6/421: chown d0/d1/l3a 160 1 2021-11-03T17:14:20.457 INFO:tasks.workunit.client.1.smithi115.stdout:6/422: read d0/d1/d2/d3f/f4b [247129,16303] 0 2021-11-03T17:14:20.457 INFO:tasks.workunit.client.1.smithi115.stdout:6/423: mkdir d0/d1/d6a/d88 0 2021-11-03T17:14:20.458 INFO:tasks.workunit.client.1.smithi115.stdout:6/424: write d0/d1/d2e/f31 [474892,70739] 0 2021-11-03T17:14:20.459 INFO:tasks.workunit.client.1.smithi115.stdout:6/425: mkdir d0/d1/d10/d13/d87/d89 0 2021-11-03T17:14:20.462 INFO:tasks.workunit.client.1.smithi115.stdout:6/426: write d0/d1/f50 [158627,32660] 0 2021-11-03T17:14:20.463 INFO:tasks.workunit.client.1.smithi115.stdout:3/518: dwrite d13/d50/f4d [0,4194304] 0 2021-11-03T17:14:20.469 INFO:tasks.workunit.client.1.smithi115.stdout:6/427: rmdir d0/d1/d6a 39 2021-11-03T17:14:20.471 INFO:tasks.workunit.client.1.smithi115.stdout:6/428: rename d0/d1/d2/d3f/f54 to d0/d1/d2/d5d/d61/f8a 0 2021-11-03T17:14:20.473 INFO:tasks.workunit.client.1.smithi115.stdout:3/519: dread d13/f1c [0,4194304] 0 2021-11-03T17:14:20.473 INFO:tasks.workunit.client.1.smithi115.stdout:6/429: dread d0/f4d [0,4194304] 0 2021-11-03T17:14:20.474 INFO:tasks.workunit.client.1.smithi115.stdout:6/430: mknod d0/d1/d2/d5d/c8b 1 2021-11-03T17:14:20.474 INFO:tasks.workunit.client.1.smithi115.stdout:6/431: write d0/d1/d2/f35 [186356,119335] 0 2021-11-03T17:14:20.475 INFO:tasks.workunit.client.1.smithi115.stdout:3/520: rename d13/d3b/fa1 to d13/d2a/d53/d70/d96/faf 0 2021-11-03T17:14:20.477 INFO:tasks.workunit.client.1.smithi115.stdout:6/432: mkdir d0/d1/d2/d8c 0 2021-11-03T17:14:20.477 INFO:tasks.workunit.client.1.smithi115.stdout:6/433: mknod d0/d1/d6a/d88/c8d 1 2021-11-03T17:14:20.478 INFO:tasks.workunit.client.1.smithi115.stdout:6/434: read d0/d1/d10/d13/f39 [1753157,55867] 0 2021-11-03T17:14:20.479 INFO:tasks.workunit.client.1.smithi115.stdout:6/435: rename d0/d1/d10/f5f to d0/d1/d2e/f8e 0 2021-11-03T17:14:20.495 INFO:tasks.workunit.client.1.smithi115.stdout:3/521: dwrite ff [0,4194304] 0 2021-11-03T17:14:20.496 INFO:tasks.workunit.client.1.smithi115.stdout:3/522: write f10 [4575253,29249] 0 2021-11-03T17:14:20.496 INFO:tasks.workunit.client.1.smithi115.stdout:3/523: write d13/d2a/d60/f95 [729740,47188] 0 2021-11-03T17:14:20.496 INFO:tasks.workunit.client.1.smithi115.stdout:3/524: write d13/d35/d54/fab [115745,82013] 0 2021-11-03T17:14:20.498 INFO:tasks.workunit.client.1.smithi115.stdout:3/525: rename d13/d2a/d53/d70/l73 to d13/d2a/d8a/lb0 0 2021-11-03T17:14:20.498 INFO:tasks.workunit.client.1.smithi115.stdout:3/526: write d13/d50/f42 [523728,68650] 0 2021-11-03T17:14:20.498 INFO:tasks.workunit.client.1.smithi115.stdout:3/527: truncate d13/d2a/f71 861090 0 2021-11-03T17:14:20.499 INFO:tasks.workunit.client.1.smithi115.stdout:3/528: mkdir d13/d3b/db1 0 2021-11-03T17:14:20.499 INFO:tasks.workunit.client.1.smithi115.stdout:3/529: mkdir d13/da4/db2 0 2021-11-03T17:14:20.500 INFO:tasks.workunit.client.1.smithi115.stdout:3/530: read - d13/d2a/d53/d70/d96/faf zero size 2021-11-03T17:14:20.500 INFO:tasks.workunit.client.1.smithi115.stdout:3/531: creat d13/d50/fb3 x:0 0 0 2021-11-03T17:14:20.501 INFO:tasks.workunit.client.1.smithi115.stdout:3/532: mkdir d13/d51/d2e/db4 0 2021-11-03T17:14:20.501 INFO:tasks.workunit.client.1.smithi115.stdout:3/533: creat d13/d35/d54/dad/fb5 x:0 0 0 2021-11-03T17:14:20.503 INFO:tasks.workunit.client.1.smithi115.stdout:3/534: dread d13/d35/d54/fa0 [0,4194304] 0 2021-11-03T17:14:20.504 INFO:tasks.workunit.client.1.smithi115.stdout:3/535: write d13/d50/f4d [3443090,44427] 0 2021-11-03T17:14:20.505 INFO:tasks.workunit.client.1.smithi115.stdout:3/536: truncate d13/d35/f36 9290468 0 2021-11-03T17:14:20.505 INFO:tasks.workunit.client.1.smithi115.stdout:3/537: stat d13/f27 0 2021-11-03T17:14:20.505 INFO:tasks.workunit.client.1.smithi115.stdout:3/538: fsync d13/d51/f45 0 2021-11-03T17:14:20.506 INFO:tasks.workunit.client.1.smithi115.stdout:3/539: truncate d13/d3b/f98 1627799 0 2021-11-03T17:14:20.507 INFO:tasks.workunit.client.1.smithi115.stdout:3/540: fsync d13/d51/d2e/f37 0 2021-11-03T17:14:20.509 INFO:tasks.workunit.client.1.smithi115.stdout:3/541: symlink d13/d50/d9e/lb6 0 2021-11-03T17:14:20.509 INFO:tasks.workunit.client.1.smithi115.stdout:3/542: write d13/d2a/d60/f95 [1791772,37719] 0 2021-11-03T17:14:20.509 INFO:tasks.workunit.client.1.smithi115.stdout:3/543: write d13/d35/d54/f6f [4971269,89055] 0 2021-11-03T17:14:20.525 INFO:tasks.workunit.client.1.smithi115.stdout:3/544: dwrite d13/f1f [0,4194304] 0 2021-11-03T17:14:20.526 INFO:tasks.workunit.client.1.smithi115.stdout:3/545: write d13/d51/d40/f9d [100337,4436] 0 2021-11-03T17:14:20.528 INFO:tasks.workunit.client.1.smithi115.stdout:3/546: mkdir d13/d2a/d53/d70/d92/db7 0 2021-11-03T17:14:20.530 INFO:tasks.workunit.client.1.smithi115.stdout:3/547: creat d13/d2a/d8a/d99/fb8 x:0 0 0 2021-11-03T17:14:20.532 INFO:tasks.workunit.client.1.smithi115.stdout:3/548: creat d13/da4/fb9 x:0 0 0 2021-11-03T17:14:20.533 INFO:tasks.workunit.client.1.smithi115.stdout:3/549: mknod d13/d50/d9e/cba 1 2021-11-03T17:14:20.535 INFO:tasks.workunit.client.1.smithi115.stdout:3/550: rename d13/d2a/d53/l86 to d13/d2a/d8a/d99/lbb 0 2021-11-03T17:14:20.556 INFO:tasks.workunit.client.1.smithi115.stdout:3/551: dwrite d13/d35/f3a [0,4194304] 0 2021-11-03T17:14:20.558 INFO:tasks.workunit.client.1.smithi115.stdout:3/552: mkdir d13/d3b/db1/dbc 0 2021-11-03T17:14:20.558 INFO:tasks.workunit.client.1.smithi115.stdout:3/553: readlink d13/d2c/l97 0 2021-11-03T17:14:20.564 INFO:tasks.workunit.client.1.smithi115.stdout:8/481: sync 2021-11-03T17:14:20.564 INFO:tasks.workunit.client.1.smithi115.stdout:2/454: sync 2021-11-03T17:14:20.583 INFO:tasks.workunit.client.1.smithi115.stdout:3/554: dwrite d13/d67/f87 [0,4194304] 0 2021-11-03T17:14:20.583 INFO:tasks.workunit.client.1.smithi115.stdout:2/455: dwrite d4/d6/d56/f5d [12582912,4194304] 0 2021-11-03T17:14:20.589 INFO:tasks.workunit.client.1.smithi115.stdout:3/555: link d13/d2c/f65 d13/d2a/d53/d72/fbd 0 2021-11-03T17:14:20.589 INFO:tasks.workunit.client.1.smithi115.stdout:3/556: mknod d13/d2a/d8a/d99/cbe 1 2021-11-03T17:14:20.591 INFO:tasks.workunit.client.1.smithi115.stdout:3/557: unlink d13/d51/d2e/f32 0 2021-11-03T17:14:20.591 INFO:tasks.workunit.client.1.smithi115.stdout:3/558: readlink d13/d2a/l9f 0 2021-11-03T17:14:20.594 INFO:tasks.workunit.client.1.smithi115.stdout:3/559: write d13/f4c [6030741,129430] 0 2021-11-03T17:14:20.594 INFO:tasks.workunit.client.1.smithi115.stdout:3/560: chown d13/d35/f3a 5227574 1 2021-11-03T17:14:20.594 INFO:tasks.workunit.client.1.smithi115.stdout:3/561: chown d13/d3b/f58 71 1 2021-11-03T17:14:20.595 INFO:tasks.workunit.client.1.smithi115.stdout:3/562: mkdir d13/d51/d2e/dbf 0 2021-11-03T17:14:20.596 INFO:tasks.workunit.client.1.smithi115.stdout:3/563: fdatasync d13/f26 0 2021-11-03T17:14:20.597 INFO:tasks.workunit.client.1.smithi115.stdout:3/564: mkdir d13/d2c/dc0 0 2021-11-03T17:14:20.598 INFO:tasks.workunit.client.1.smithi115.stdout:3/565: fdatasync d13/d3b/f98 0 2021-11-03T17:14:20.598 INFO:tasks.workunit.client.1.smithi115.stdout:3/566: chown d13/d35 50 1 2021-11-03T17:14:20.599 INFO:tasks.workunit.client.1.smithi115.stdout:3/567: creat d13/d2a/d53/d70/d92/fc1 x:0 0 0 2021-11-03T17:14:20.602 INFO:tasks.workunit.client.1.smithi115.stdout:2/456: dwrite d4/d6/d56/d6c/d34/f52 [0,4194304] 0 2021-11-03T17:14:20.604 INFO:tasks.workunit.client.1.smithi115.stdout:2/457: truncate d4/d6/d56/d6c/f3c 7080646 0 2021-11-03T17:14:20.605 INFO:tasks.workunit.client.1.smithi115.stdout:2/458: rmdir d4/d44/d98 0 2021-11-03T17:14:20.606 INFO:tasks.workunit.client.1.smithi115.stdout:2/459: mknod d4/ca2 1 2021-11-03T17:14:20.609 INFO:tasks.workunit.client.1.smithi115.stdout:3/568: write d13/d35/f36 [4385511,11561] 0 2021-11-03T17:14:20.612 INFO:tasks.workunit.client.1.smithi115.stdout:2/460: getdents d4/d44 0 2021-11-03T17:14:20.635 INFO:tasks.workunit.client.1.smithi115.stdout:3/569: dwrite d13/d35/f3a [0,4194304] 0 2021-11-03T17:14:20.635 INFO:tasks.workunit.client.1.smithi115.stdout:3/570: chown d13/f22 269 1 2021-11-03T17:14:20.636 INFO:tasks.workunit.client.1.smithi115.stdout:3/571: readlink d13/d2a/l8d 0 2021-11-03T17:14:20.643 INFO:tasks.workunit.client.1.smithi115.stdout:2/461: dread d4/d6/d56/d6c/d34/d43/d28/f3b [0,4194304] 0 2021-11-03T17:14:20.646 INFO:tasks.workunit.client.1.smithi115.stdout:2/462: write d4/d6/d56/f9a [2564564,95701] 0 2021-11-03T17:14:20.646 INFO:tasks.workunit.client.1.smithi115.stdout:2/463: write d4/d6/d56/d6c/d34/d43/d63/f99 [4392625,20089] 0 2021-11-03T17:14:20.655 INFO:tasks.workunit.client.1.smithi115.stdout:3/572: mkdir d13/d2a/d53/d70/d92/dc2 0 2021-11-03T17:14:20.656 INFO:tasks.workunit.client.1.smithi115.stdout:3/573: readlink d13/d51/d40/l6b 0 2021-11-03T17:14:20.662 INFO:tasks.workunit.client.1.smithi115.stdout:2/464: dread d4/d6/d56/f9a [0,4194304] 0 2021-11-03T17:14:20.662 INFO:tasks.workunit.client.1.smithi115.stdout:2/465: stat d4/d6/d56/d6c/d34/f52 0 2021-11-03T17:14:20.663 INFO:tasks.workunit.client.1.smithi115.stdout:2/466: write d4/f5a [284219,2358] 0 2021-11-03T17:14:20.663 INFO:tasks.workunit.client.1.smithi115.stdout:2/467: read d4/d6/d13/f36 [6622652,4712] 0 2021-11-03T17:14:20.689 INFO:tasks.workunit.client.1.smithi115.stdout:3/574: link d13/d2a/d8a/fa6 d13/d35/fc3 0 2021-11-03T17:14:20.690 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:20 smithi115 conmon[34670]: cluster 2021-11-03T17:14:20.607224+0000 mgr.smithi027.ufhhvb 2021-11-03T17:14:20.691 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:20 smithi115 conmon[34670]: (mgr.14162) 311 : cluster [DBG] pgmap v257: 65 pgs: 65 active+clean; 2.1 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 90 MiB/s rd, 181 MiB/s wr, 297 op/s 2021-11-03T17:14:20.693 INFO:tasks.workunit.client.1.smithi115.stdout:2/468: link d4/d6/d13/f58 d4/d6/d56/d6c/d34/d43/d63/d77/d79/d82/fa3 0 2021-11-03T17:14:20.693 INFO:tasks.workunit.client.1.smithi115.stdout:2/469: mknod d4/d6/d56/d6c/d34/d43/d63/d77/d79/d82/ca4 1 2021-11-03T17:14:20.699 INFO:tasks.workunit.client.0.smithi027.stdout:5/727: sync 2021-11-03T17:14:20.699 INFO:tasks.workunit.client.0.smithi027.stdout:7/341: sync 2021-11-03T17:14:20.700 INFO:tasks.workunit.client.0.smithi027.stdout:5/728: write d2/d4/fcc [82005,47626] 0 2021-11-03T17:14:20.703 INFO:tasks.workunit.client.1.smithi115.stdout:2/470: truncate d4/d6/d13/d6d/f8b 3504561 0 2021-11-03T17:14:20.704 INFO:tasks.workunit.client.0.smithi027.stdout:5/729: rename d2/d2c/d3c/d48/d65/l50 to d2/d4/d4e/led 0 2021-11-03T17:14:20.705 INFO:tasks.workunit.client.0.smithi027.stdout:5/730: write d2/d2c/d3c/d48/d65/d80/fa3 [435383,90519] 0 2021-11-03T17:14:20.706 INFO:tasks.workunit.client.1.smithi115.stdout:2/471: creat d4/d6/d13/d6d/d7f/fa5 x:0 0 0 2021-11-03T17:14:20.710 INFO:tasks.workunit.client.1.smithi115.stdout:3/575: unlink d13/l28 0 2021-11-03T17:14:20.711 INFO:tasks.workunit.client.1.smithi115.stdout:3/576: write d13/d51/d40/f88 [4490783,128594] 0 2021-11-03T17:14:20.713 INFO:tasks.workunit.client.1.smithi115.stdout:3/577: symlink d13/d2a/d53/d70/d96/lc4 0 2021-11-03T17:14:20.716 INFO:tasks.workunit.client.1.smithi115.stdout:3/578: creat d13/d2a/d53/d70/d92/db7/fc5 x:0 0 0 2021-11-03T17:14:20.719 INFO:tasks.workunit.client.1.smithi115.stdout:3/579: rename d13/d35/fa3 to d13/d2a/d8a/d99/fc6 0 2021-11-03T17:14:20.719 INFO:tasks.workunit.client.1.smithi115.stdout:3/580: chown d13/f4c 0 1 2021-11-03T17:14:20.738 INFO:tasks.workunit.client.0.smithi027.stdout:5/731: dwrite d2/d1f/fbf [0,4194304] 0 2021-11-03T17:14:20.739 INFO:tasks.workunit.client.1.smithi115.stdout:2/472: dwrite d4/f68 [0,4194304] 0 2021-11-03T17:14:20.740 INFO:tasks.workunit.client.1.smithi115.stdout:3/581: dwrite d13/d51/d40/f5a [0,4194304] 0 2021-11-03T17:14:20.741 INFO:tasks.workunit.client.0.smithi027.stdout:5/732: creat d2/d2c/d9b/db3/fee x:0 0 0 2021-11-03T17:14:20.742 INFO:tasks.workunit.client.1.smithi115.stdout:2/473: creat d4/d6/d56/d6c/d34/d43/d63/d77/fa6 x:0 0 0 2021-11-03T17:14:20.742 INFO:tasks.workunit.client.1.smithi115.stdout:2/474: chown d4/l4f 3713 1 2021-11-03T17:14:20.743 INFO:tasks.workunit.client.1.smithi115.stdout:2/475: dread - d4/d6/d13/d6d/f60 zero size 2021-11-03T17:14:20.743 INFO:tasks.workunit.client.1.smithi115.stdout:2/476: write d4/d6/d56/d6c/d34/d43/d63/d77/fa6 [933797,31860] 0 2021-11-03T17:14:20.743 INFO:tasks.workunit.client.1.smithi115.stdout:2/477: fdatasync d4/d6/d56/d6c/f14 0 2021-11-03T17:14:20.744 INFO:tasks.workunit.client.0.smithi027.stdout:5/733: creat d2/d2c/d3c/d58/fef x:0 0 0 2021-11-03T17:14:20.745 INFO:tasks.workunit.client.1.smithi115.stdout:3/582: symlink d13/d51/d2e/db4/lc7 0 2021-11-03T17:14:20.746 INFO:tasks.workunit.client.1.smithi115.stdout:2/478: creat d4/d6/d56/d6c/d34/d43/d63/d69/fa7 x:0 0 0 2021-11-03T17:14:20.747 INFO:tasks.workunit.client.1.smithi115.stdout:3/583: rename d13/d35/fc3 to d13/d2a/d8a/fc8 0 2021-11-03T17:14:20.750 INFO:tasks.workunit.client.1.smithi115.stdout:2/479: link d4/l2c d4/d6/d56/d6c/d34/d43/d63/d77/d79/d82/la8 0 2021-11-03T17:14:20.769 INFO:tasks.workunit.client.1.smithi115.stdout:3/584: dwrite d13/d51/d2e/f59 [4194304,4194304] 0 2021-11-03T17:14:20.771 INFO:tasks.workunit.client.1.smithi115.stdout:2/480: dwrite d4/d6/d56/d6c/f46 [0,4194304] 0 2021-11-03T17:14:20.780 INFO:tasks.workunit.client.1.smithi115.stdout:2/481: dread d4/d6/d56/d6c/f6f [0,4194304] 0 2021-11-03T17:14:20.782 INFO:tasks.workunit.client.1.smithi115.stdout:2/482: mkdir d4/d6/d56/d6c/d34/d42/d8d/da9 0 2021-11-03T17:14:20.782 INFO:tasks.workunit.client.1.smithi115.stdout:2/483: stat d4/f5a 0 2021-11-03T17:14:20.783 INFO:tasks.workunit.client.1.smithi115.stdout:2/484: rename d4/d6/d56/d6c/d34/d43/l6a to d4/d6/d56/d6c/d34/d43/d63/d69/laa 0 2021-11-03T17:14:20.784 INFO:tasks.workunit.client.1.smithi115.stdout:2/485: write d4/d6/f2a [4910574,115121] 0 2021-11-03T17:14:20.799 INFO:tasks.workunit.client.1.smithi115.stdout:2/486: dwrite d4/f35 [0,4194304] 0 2021-11-03T17:14:20.799 INFO:tasks.workunit.client.1.smithi115.stdout:2/487: chown d4/d6/d56/d6c/d34/d43/d63/d77/fa6 1 1 2021-11-03T17:14:20.799 INFO:tasks.workunit.client.1.smithi115.stdout:2/488: readlink d4/d6/d56/d6c/d34/d43/d63/d77/d79/l87 0 2021-11-03T17:14:20.814 INFO:tasks.workunit.client.1.smithi115.stdout:2/489: dwrite d4/d6/d13/d6d/d7f/fa5 [0,4194304] 0 2021-11-03T17:14:20.814 INFO:tasks.workunit.client.1.smithi115.stdout:2/490: chown d4/d6/d13/d75/l7d 9775679 1 2021-11-03T17:14:20.814 INFO:tasks.workunit.client.1.smithi115.stdout:2/491: fdatasync d4/d6/d56/d6c/d34/d43/f4d 0 2021-11-03T17:14:20.814 INFO:tasks.workunit.client.1.smithi115.stdout:2/492: dread - d4/d6/d56/d6c/d34/d43/d63/d69/fa7 zero size 2021-11-03T17:14:20.821 INFO:tasks.workunit.client.1.smithi115.stdout:2/493: dread d4/d6/d13/f58 [0,4194304] 0 2021-11-03T17:14:20.822 INFO:tasks.workunit.client.1.smithi115.stdout:2/494: write d4/d6/d56/d6c/f6f [4874953,68865] 0 2021-11-03T17:14:20.835 INFO:tasks.workunit.client.1.smithi115.stdout:2/495: dwrite d4/d6/d56/d6c/d34/d43/d63/d77/d79/d82/fa3 [0,4194304] 0 2021-11-03T17:14:20.835 INFO:tasks.workunit.client.1.smithi115.stdout:2/496: readlink d4/d6/d13/d75/l8a 0 2021-11-03T17:14:20.838 INFO:tasks.workunit.client.1.smithi115.stdout:2/497: symlink d4/d6/d56/lab 0 2021-11-03T17:14:20.841 INFO:tasks.workunit.client.1.smithi115.stdout:2/498: rename d4/d6/d13/d75/d8c to d4/d6/d56/d6c/d34/d43/d63/d77/d79/dac 0 2021-11-03T17:14:20.843 INFO:tasks.workunit.client.1.smithi115.stdout:2/499: rmdir d4/d6/d56/d6c/d34/d43/d63/d69 39 2021-11-03T17:14:20.844 INFO:tasks.workunit.client.1.smithi115.stdout:2/500: getdents d4/d6 0 2021-11-03T17:14:20.847 INFO:tasks.workunit.client.1.smithi115.stdout:2/501: write d4/d6/f48 [1862791,60668] 0 2021-11-03T17:14:20.851 INFO:tasks.workunit.client.1.smithi115.stdout:2/502: rename d4/d6/d13/d6d to d4/d6/dad 0 2021-11-03T17:14:20.853 INFO:tasks.workunit.client.1.smithi115.stdout:2/503: link d4/d6/d56/d6c/d34/d42/la1 d4/d6/d56/d6c/d34/d43/lae 0 2021-11-03T17:14:20.855 INFO:tasks.workunit.client.1.smithi115.stdout:2/504: rename d4/d6/d13/d75/l8a to d4/laf 0 2021-11-03T17:14:20.856 INFO:tasks.workunit.client.1.smithi115.stdout:2/505: write d4/f62 [2355965,129467] 0 2021-11-03T17:14:20.856 INFO:tasks.workunit.client.1.smithi115.stdout:2/506: mknod d4/d6/d56/d6c/d34/d43/cb0 1 2021-11-03T17:14:20.859 INFO:tasks.workunit.client.0.smithi027.stdout:2/440: sync 2021-11-03T17:14:20.862 INFO:tasks.workunit.client.1.smithi115.stdout:2/507: dread d4/d6/dad/f5f [0,4194304] 0 2021-11-03T17:14:20.862 INFO:tasks.workunit.client.1.smithi115.stdout:2/508: getdents d4/d6/d56/d6c/d34/d43/d63/d77/d79/d9c/d9f 0 2021-11-03T17:14:20.866 INFO:tasks.workunit.client.0.smithi027.stdout:2/441: dread d0/d1b/d1c/f7e [0,4194304] 0 2021-11-03T17:14:20.866 INFO:tasks.workunit.client.0.smithi027.stdout:2/442: write d0/d1b/d29/d58/f5c [338790,91815] 0 2021-11-03T17:14:20.868 INFO:tasks.workunit.client.1.smithi115.stdout:2/509: write d4/d6/f7e [776451,5959] 0 2021-11-03T17:14:20.868 INFO:tasks.workunit.client.0.smithi027.stdout:2/443: rmdir d0/d5/d6b 0 2021-11-03T17:14:20.870 INFO:tasks.workunit.client.1.smithi115.stdout:2/510: dread d4/d6/d13/f21 [0,4194304] 0 2021-11-03T17:14:20.871 INFO:tasks.workunit.client.1.smithi115.stdout:2/511: creat d4/d6/d56/d6c/d34/d43/d63/d77/fb1 x:0 0 0 2021-11-03T17:14:20.873 INFO:tasks.workunit.client.1.smithi115.stdout:2/512: creat d4/d6/d13/fb2 x:0 0 0 2021-11-03T17:14:20.874 INFO:tasks.workunit.client.1.smithi115.stdout:2/513: symlink d4/d6/d56/d6c/d34/d43/d63/d77/d79/dac/lb3 0 2021-11-03T17:14:20.874 INFO:tasks.workunit.client.1.smithi115.stdout:2/514: stat d4/d6/dad/f5f 0 2021-11-03T17:14:20.875 INFO:tasks.workunit.client.0.smithi027.stdout:2/444: dread d0/d5/f13 [4194304,4194304] 0 2021-11-03T17:14:20.875 INFO:tasks.workunit.client.0.smithi027.stdout:2/445: chown d0/d2a/f30 28031697 1 2021-11-03T17:14:20.876 INFO:tasks.workunit.client.1.smithi115.stdout:2/515: truncate d4/f35 1586593 0 2021-11-03T17:14:20.876 INFO:tasks.workunit.client.1.smithi115.stdout:2/516: dread - d4/d6/d13/fb2 zero size 2021-11-03T17:14:20.877 INFO:tasks.workunit.client.1.smithi115.stdout:2/517: write d4/d6/dad/f8b [1137194,7954] 0 2021-11-03T17:14:20.877 INFO:tasks.workunit.client.1.smithi115.stdout:2/518: chown d4/d6/d13/d75 15 1 2021-11-03T17:14:20.877 INFO:tasks.workunit.client.1.smithi115.stdout:2/519: write d4/f5a [1273870,73496] 0 2021-11-03T17:14:20.878 INFO:tasks.workunit.client.0.smithi027.stdout:2/446: unlink d0/d1b/d29/f39 0 2021-11-03T17:14:20.878 INFO:tasks.workunit.client.0.smithi027.stdout:2/447: readlink d0/d5/l1a 0 2021-11-03T17:14:20.906 INFO:tasks.workunit.client.1.smithi115.stdout:2/520: dwrite d4/d6/f7e [0,4194304] 0 2021-11-03T17:14:20.906 INFO:tasks.workunit.client.1.smithi115.stdout:2/521: readlink d4/d44/l53 0 2021-11-03T17:14:20.907 INFO:tasks.workunit.client.0.smithi027.stdout:2/448: dwrite d0/d1b/d29/f4b [0,4194304] 0 2021-11-03T17:14:20.907 INFO:tasks.workunit.client.0.smithi027.stdout:2/449: chown d0/d1b/d29/l50 88173059 1 2021-11-03T17:14:20.907 INFO:tasks.workunit.client.1.smithi115.stdout:2/522: write d4/d6/d56/d6c/d34/d43/d28/f41 [1306360,109623] 0 2021-11-03T17:14:20.907 INFO:tasks.workunit.client.1.smithi115.stdout:2/523: fdatasync d4/d6/d56/d6c/d34/d43/d63/d69/fa7 0 2021-11-03T17:14:20.908 INFO:tasks.workunit.client.1.smithi115.stdout:2/524: mkdir d4/d6/d56/d6c/d34/d43/d63/d77/d79/dac/db4 0 2021-11-03T17:14:20.909 INFO:tasks.workunit.client.1.smithi115.stdout:2/525: read d4/f35 [783506,1363] 0 2021-11-03T17:14:20.909 INFO:tasks.workunit.client.1.smithi115.stdout:2/526: write d4/d6/d56/d6c/f14 [13513437,52779] 0 2021-11-03T17:14:20.910 INFO:tasks.workunit.client.1.smithi115.stdout:2/527: creat d4/d6/d56/d6c/d34/d84/fb5 x:0 0 0 2021-11-03T17:14:20.911 INFO:tasks.workunit.client.1.smithi115.stdout:2/528: write d4/d6/d56/d6c/d34/d43/d63/d77/d79/d82/fa3 [1560259,35784] 0 2021-11-03T17:14:20.912 INFO:tasks.workunit.client.1.smithi115.stdout:2/529: fsync d4/d6/d13/f38 0 2021-11-03T17:14:20.917 INFO:tasks.workunit.client.0.smithi027.stdout:2/450: dread d0/d2a/d5d/f75 [0,4194304] 0 2021-11-03T17:14:20.918 INFO:tasks.workunit.client.0.smithi027.stdout:2/451: read d0/d1b/f27 [147997,30094] 0 2021-11-03T17:14:20.920 INFO:tasks.workunit.client.0.smithi027.stdout:2/452: mkdir d0/d1b/d87 0 2021-11-03T17:14:20.922 INFO:tasks.workunit.client.0.smithi027.stdout:2/453: mkdir d0/d2a/d88 0 2021-11-03T17:14:20.923 INFO:tasks.workunit.client.0.smithi027.stdout:2/454: mkdir d0/d7c/d89 0 2021-11-03T17:14:20.924 INFO:tasks.workunit.client.0.smithi027.stdout:2/455: mknod d0/d2a/d5d/d70/c8a 1 2021-11-03T17:14:20.925 INFO:tasks.workunit.client.0.smithi027.stdout:2/456: truncate d0/d2a/f30 596929 0 2021-11-03T17:14:20.929 INFO:tasks.workunit.client.0.smithi027.stdout:2/457: write d0/d2a/f44 [5129345,14244] 0 2021-11-03T17:14:20.929 INFO:tasks.workunit.client.0.smithi027.stdout:2/458: truncate d0/d1b/f82 812466 0 2021-11-03T17:14:20.929 INFO:tasks.workunit.client.0.smithi027.stdout:2/459: fdatasync d0/d1b/f82 0 2021-11-03T17:14:20.930 INFO:tasks.workunit.client.0.smithi027.stdout:2/460: write d0/f46 [472837,34242] 0 2021-11-03T17:14:20.930 INFO:tasks.workunit.client.0.smithi027.stdout:2/461: getdents d0/d2a/d5d/d70 0 2021-11-03T17:14:20.930 INFO:tasks.workunit.client.0.smithi027.stdout:2/462: chown d0/d5/f83 52711357 1 2021-11-03T17:14:20.933 INFO:tasks.workunit.client.1.smithi115.stdout:2/530: dwrite d4/d6/d56/d6c/d34/d43/d63/d77/f81 [0,4194304] 0 2021-11-03T17:14:20.942 INFO:tasks.workunit.client.1.smithi115.stdout:2/531: write d4/d6/d56/d6c/d34/f70 [3499841,117642] 0 2021-11-03T17:14:20.942 INFO:tasks.workunit.client.1.smithi115.stdout:2/532: chown d4/d6/f45 0 1 2021-11-03T17:14:20.942 INFO:tasks.workunit.client.1.smithi115.stdout:2/533: fdatasync d4/d6/d56/d6c/f6f 0 2021-11-03T17:14:20.942 INFO:tasks.workunit.client.1.smithi115.stdout:2/534: fsync d4/f62 0 2021-11-03T17:14:20.943 INFO:tasks.workunit.client.1.smithi115.stdout:2/535: truncate d4/f78 674749 0 2021-11-03T17:14:20.943 INFO:tasks.workunit.client.1.smithi115.stdout:2/536: mknod d4/d6/d56/d6c/d34/d43/d63/d77/d79/d9c/d9f/cb6 1 2021-11-03T17:14:20.943 INFO:tasks.workunit.client.1.smithi115.stdout:2/537: chown d4/f78 1 1 2021-11-03T17:14:20.945 INFO:tasks.workunit.client.1.smithi115.stdout:2/538: write d4/d6/d56/d6c/d34/d43/d28/f3b [2832097,54656] 0 2021-11-03T17:14:20.949 INFO:tasks.workunit.client.1.smithi115.stdout:2/539: unlink d4/d6/d56/f9a 0 2021-11-03T17:14:20.953 INFO:tasks.workunit.client.0.smithi027.stdout:3/372: sync 2021-11-03T17:14:20.954 INFO:tasks.workunit.client.0.smithi027.stdout:3/373: dread - d1/f41 zero size 2021-11-03T17:14:20.954 INFO:tasks.workunit.client.0.smithi027.stdout:3/374: write d1/f22 [8988864,108038] 0 2021-11-03T17:14:20.955 INFO:tasks.workunit.client.1.smithi115.stdout:2/540: fdatasync d4/d6/d56/d6c/d34/d43/d28/f41 0 2021-11-03T17:14:20.956 INFO:tasks.workunit.client.0.smithi027.stdout:2/463: dwrite d0/d5/f4 [0,4194304] 0 2021-11-03T17:14:20.956 INFO:tasks.workunit.client.0.smithi027.stdout:3/375: symlink d1/d6f/l75 0 2021-11-03T17:14:20.959 INFO:tasks.workunit.client.0.smithi027.stdout:3/376: rename d1/d40/f49 to d1/d40/f76 0 2021-11-03T17:14:20.960 INFO:tasks.workunit.client.1.smithi115.stdout:2/541: mknod d4/d6/d56/d6c/cb7 1 2021-11-03T17:14:20.960 INFO:tasks.workunit.client.1.smithi115.stdout:2/542: readlink d4/d6/d56/d6c/d34/d43/d63/d77/d79/dac/lb3 0 2021-11-03T17:14:20.961 INFO:tasks.workunit.client.0.smithi027.stdout:2/464: truncate d0/d1b/d1c/f33 484663 0 2021-11-03T17:14:20.961 INFO:tasks.workunit.client.0.smithi027.stdout:3/377: symlink d1/db/d52/d6a/l77 0 2021-11-03T17:14:20.963 INFO:tasks.workunit.client.0.smithi027.stdout:2/465: creat d0/d1b/d29/d58/d5b/f8b x:0 0 0 2021-11-03T17:14:20.963 INFO:tasks.workunit.client.0.smithi027.stdout:3/378: write d1/db/f3c [969060,62641] 0 2021-11-03T17:14:20.963 INFO:tasks.workunit.client.0.smithi027.stdout:3/379: mknod d1/db/c78 1 2021-11-03T17:14:20.966 INFO:tasks.workunit.client.0.smithi027.stdout:2/466: link d0/d5/f53 d0/d1b/d87/f8c 0 2021-11-03T17:14:20.967 INFO:tasks.workunit.client.0.smithi027.stdout:3/380: truncate d1/f38 3707405 0 2021-11-03T17:14:20.968 INFO:tasks.workunit.client.0.smithi027.stdout:3/381: dread - d1/db/f5b zero size 2021-11-03T17:14:20.968 INFO:tasks.workunit.client.0.smithi027.stdout:3/382: chown d1/l2c 6251 1 2021-11-03T17:14:20.981 INFO:tasks.workunit.client.1.smithi115.stdout:2/543: dwrite d4/d6/d56/d6c/d34/d42/f8f [0,4194304] 0 2021-11-03T17:14:20.983 INFO:tasks.workunit.client.0.smithi027.stdout:3/383: dwrite d1/f17 [0,4194304] 0 2021-11-03T17:14:20.983 INFO:tasks.workunit.client.0.smithi027.stdout:3/384: readlink d1/l3a 0 2021-11-03T17:14:20.984 INFO:tasks.workunit.client.1.smithi115.stdout:2/544: write d4/d6/f45 [1868226,102248] 0 2021-11-03T17:14:20.984 INFO:tasks.workunit.client.1.smithi115.stdout:2/545: mknod d4/d6/d56/d6c/d34/d42/d8d/cb8 1 2021-11-03T17:14:20.988 INFO:tasks.workunit.client.1.smithi115.stdout:2/546: unlink d4/d6/d56/d6c/d34/d43/f4d 0 2021-11-03T17:14:20.989 INFO:tasks.workunit.client.1.smithi115.stdout:2/547: write d4/d6/f9e [941503,97479] 0 2021-11-03T17:14:21.014 INFO:tasks.workunit.client.0.smithi027.stdout:2/467: dwrite d0/d2a/f30 [0,4194304] 0 2021-11-03T17:14:21.015 INFO:tasks.workunit.client.0.smithi027.stdout:2/468: mknod d0/d2a/c8d 1 2021-11-03T17:14:21.015 INFO:tasks.workunit.client.0.smithi027.stdout:2/469: chown d0/f10 478372 1 2021-11-03T17:14:21.016 INFO:tasks.workunit.client.0.smithi027.stdout:3/385: dwrite d1/db/d52/f65 [0,4194304] 0 2021-11-03T17:14:21.017 INFO:tasks.workunit.client.1.smithi115.stdout:2/548: dwrite d4/d6/d56/d6c/f39 [0,4194304] 0 2021-11-03T17:14:21.017 INFO:tasks.workunit.client.0.smithi027.stdout:3/386: write d1/db/f12 [241320,84280] 0 2021-11-03T17:14:21.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:20 smithi027 conmon[32322]: cluster 2021-11-03T17:14:20.607224 2021-11-03T17:14:21.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:20 smithi027 conmon[32322]: +0000 mgr.smithi027.ufhhvb (mgr.14162) 311 : cluster [DBG] pgmap v257: 65 pgs: 65 active+clean; 2.1 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 90 MiB/s rd, 181 MiB/s wr, 297 op/s 2021-11-03T17:14:21.019 INFO:tasks.workunit.client.0.smithi027.stdout:2/470: getdents d0/d1b/d29 0 2021-11-03T17:14:21.019 INFO:tasks.workunit.client.0.smithi027.stdout:2/471: read d0/d2a/f30 [3641470,30736] 0 2021-11-03T17:14:21.021 INFO:tasks.workunit.client.1.smithi115.stdout:2/549: unlink d4/d6/f9e 0 2021-11-03T17:14:21.022 INFO:tasks.workunit.client.1.smithi115.stdout:2/550: symlink d4/d6/d56/d6c/d34/d43/d63/d77/d79/d9c/lb9 0 2021-11-03T17:14:21.026 INFO:tasks.workunit.client.0.smithi027.stdout:3/387: creat d1/d6f/f79 x:0 0 0 2021-11-03T17:14:21.027 INFO:tasks.workunit.client.0.smithi027.stdout:3/388: dread - d1/d40/f5c zero size 2021-11-03T17:14:21.027 INFO:tasks.workunit.client.0.smithi027.stdout:3/389: read d1/d40/f46 [99097,58093] 0 2021-11-03T17:14:21.027 INFO:tasks.workunit.client.0.smithi027.stdout:3/390: chown d1/d40/f4a 166483248 1 2021-11-03T17:14:21.036 INFO:tasks.workunit.client.0.smithi027.stdout:2/472: symlink d0/l8e 0 2021-11-03T17:14:21.037 INFO:tasks.workunit.client.0.smithi027.stdout:2/473: stat d0/f10 0 2021-11-03T17:14:21.037 INFO:tasks.workunit.client.0.smithi027.stdout:2/474: fsync d0/f3b 0 2021-11-03T17:14:21.037 INFO:tasks.workunit.client.0.smithi027.stdout:2/475: chown d0/d7c 11752 1 2021-11-03T17:14:21.038 INFO:tasks.workunit.client.0.smithi027.stdout:2/476: write d0/d1b/d1c/f7e [3586582,5498] 0 2021-11-03T17:14:21.038 INFO:tasks.workunit.client.0.smithi027.stdout:3/391: symlink d1/d40/l7a 0 2021-11-03T17:14:21.045 INFO:tasks.workunit.client.1.smithi115.stdout:2/551: dwrite d4/d6/d56/d6c/f17 [0,4194304] 0 2021-11-03T17:14:21.046 INFO:tasks.workunit.client.1.smithi115.stdout:2/552: mknod d4/d6/d56/cba 1 2021-11-03T17:14:21.046 INFO:tasks.workunit.client.0.smithi027.stdout:2/477: dread d0/d1b/d29/d58/f5c [0,4194304] 0 2021-11-03T17:14:21.047 INFO:tasks.workunit.client.1.smithi115.stdout:2/553: symlink d4/d6/d56/d6c/d34/d84/lbb 0 2021-11-03T17:14:21.057 INFO:tasks.workunit.client.0.smithi027.stdout:3/392: dwrite d1/db/d52/d57/f71 [0,4194304] 0 2021-11-03T17:14:21.057 INFO:tasks.workunit.client.0.smithi027.stdout:3/393: write d1/db/d52/f5d [97804,19907] 0 2021-11-03T17:14:21.058 INFO:tasks.workunit.client.0.smithi027.stdout:3/394: truncate d1/db/d52/d57/f62 870807 0 2021-11-03T17:14:21.058 INFO:tasks.workunit.client.0.smithi027.stdout:3/395: chown d1/f4c 2744 1 2021-11-03T17:14:21.058 INFO:tasks.workunit.client.0.smithi027.stdout:3/396: mknod d1/db/c7b 1 2021-11-03T17:14:21.060 INFO:tasks.workunit.client.0.smithi027.stdout:3/397: getdents d1/d40 0 2021-11-03T17:14:21.062 INFO:tasks.workunit.client.0.smithi027.stdout:3/398: mkdir d1/db/d52/d6a/d7c 0 2021-11-03T17:14:21.070 INFO:tasks.workunit.client.1.smithi115.stdout:2/554: dwrite d4/f78 [0,4194304] 0 2021-11-03T17:14:21.071 INFO:tasks.workunit.client.0.smithi027.stdout:2/478: dwrite d0/d1b/d29/f4b [0,4194304] 0 2021-11-03T17:14:21.071 INFO:tasks.workunit.client.0.smithi027.stdout:2/479: chown d0/f46 22304450 1 2021-11-03T17:14:21.071 INFO:tasks.workunit.client.0.smithi027.stdout:2/480: write d0/d45/f7b [1302676,118909] 0 2021-11-03T17:14:21.080 INFO:tasks.workunit.client.0.smithi027.stdout:3/399: dwrite d1/d40/f44 [0,4194304] 0 2021-11-03T17:14:21.085 INFO:tasks.workunit.client.0.smithi027.stdout:3/400: creat d1/db/f7d x:0 0 0 2021-11-03T17:14:21.087 INFO:tasks.workunit.client.0.smithi027.stdout:3/401: unlink d1/d6f/f79 0 2021-11-03T17:14:21.088 INFO:tasks.workunit.client.0.smithi027.stdout:3/402: symlink d1/db/l7e 0 2021-11-03T17:14:21.089 INFO:tasks.workunit.client.0.smithi027.stdout:3/403: fsync d1/d40/f4a 0 2021-11-03T17:14:21.089 INFO:tasks.workunit.client.0.smithi027.stdout:3/404: fsync d1/db/d52/d57/f71 0 2021-11-03T17:14:21.089 INFO:tasks.workunit.client.1.smithi115.stdout:2/555: dwrite d4/f62 [0,4194304] 0 2021-11-03T17:14:21.090 INFO:tasks.workunit.client.0.smithi027.stdout:3/405: unlink d1/l33 0 2021-11-03T17:14:21.091 INFO:tasks.workunit.client.1.smithi115.stdout:2/556: rename d4/d44/l47 to d4/d6/d13/lbc 0 2021-11-03T17:14:21.092 INFO:tasks.workunit.client.1.smithi115.stdout:2/557: unlink d4/d6/d56/d6c/d34/f89 0 2021-11-03T17:14:21.094 INFO:tasks.workunit.client.1.smithi115.stdout:2/558: rename d4/d6/d56/d6c/f17 to d4/d6/fbd 0 2021-11-03T17:14:21.095 INFO:tasks.workunit.client.0.smithi027.stdout:2/481: dwrite d0/d45/f7b [0,4194304] 0 2021-11-03T17:14:21.098 INFO:tasks.workunit.client.0.smithi027.stdout:2/482: creat d0/d2a/d5d/f8f x:0 0 0 2021-11-03T17:14:21.098 INFO:tasks.workunit.client.0.smithi027.stdout:2/483: chown d0/d2a 91281771 1 2021-11-03T17:14:21.099 INFO:tasks.workunit.client.0.smithi027.stdout:2/484: unlink d0/d5/f13 0 2021-11-03T17:14:21.105 INFO:tasks.workunit.client.1.smithi115.stdout:2/559: fdatasync d4/d6/d56/d6c/f14 0 2021-11-03T17:14:21.106 INFO:tasks.workunit.client.0.smithi027.stdout:3/406: dwrite d1/f17 [0,4194304] 0 2021-11-03T17:14:21.107 INFO:tasks.workunit.client.1.smithi115.stdout:2/560: mkdir d4/d6/d56/d6c/d34/d43/d63/d77/d79/d9c/dbe 0 2021-11-03T17:14:21.116 INFO:tasks.workunit.client.1.smithi115.stdout:2/561: dread d4/d6/dad/f8b [0,4194304] 0 2021-11-03T17:14:21.119 INFO:tasks.workunit.client.1.smithi115.stdout:2/562: write d4/d6/d56/d6c/f3c [540306,86499] 0 2021-11-03T17:14:21.121 INFO:tasks.workunit.client.1.smithi115.stdout:2/563: rename d4/d6/d56/d6c/f3c to d4/d6/d56/d6c/d34/d43/d63/d69/fbf 0 2021-11-03T17:14:21.122 INFO:tasks.workunit.client.1.smithi115.stdout:2/564: creat d4/d6/dad/d7f/fc0 x:0 0 0 2021-11-03T17:14:21.123 INFO:tasks.workunit.client.1.smithi115.stdout:2/565: symlink d4/d6/d56/d6c/d34/d43/d63/d77/d79/dac/lc1 0 2021-11-03T17:14:21.127 INFO:tasks.workunit.client.0.smithi027.stdout:3/407: dwrite d1/d40/f59 [0,4194304] 0 2021-11-03T17:14:21.129 INFO:tasks.workunit.client.0.smithi027.stdout:2/485: dwrite d0/d5/f83 [0,4194304] 0 2021-11-03T17:14:21.129 INFO:tasks.workunit.client.0.smithi027.stdout:2/486: chown d0/d1b/d29/f2b 2860 1 2021-11-03T17:14:21.132 INFO:tasks.workunit.client.0.smithi027.stdout:3/408: write d1/fa [498417,75224] 0 2021-11-03T17:14:21.133 INFO:tasks.workunit.client.0.smithi027.stdout:2/487: creat d0/f90 x:0 0 0 2021-11-03T17:14:21.133 INFO:tasks.workunit.client.0.smithi027.stdout:2/488: write d0/d1b/d1c/f7e [5116057,77556] 0 2021-11-03T17:14:21.134 INFO:tasks.workunit.client.0.smithi027.stdout:2/489: fsync d0/d5/f4 0 2021-11-03T17:14:21.134 INFO:tasks.workunit.client.0.smithi027.stdout:2/490: fsync d0/f3b 0 2021-11-03T17:14:21.134 INFO:tasks.workunit.client.0.smithi027.stdout:2/491: stat d0/d1b/l60 0 2021-11-03T17:14:21.134 INFO:tasks.workunit.client.0.smithi027.stdout:2/492: read - d0/d1b/d29/d58/d5b/f8b zero size 2021-11-03T17:14:21.135 INFO:tasks.workunit.client.0.smithi027.stdout:3/409: rename d1/f74 to d1/db/d52/d6a/f7f 0 2021-11-03T17:14:21.136 INFO:tasks.workunit.client.0.smithi027.stdout:2/493: write d0/d2a/d5d/f75 [3805494,78920] 0 2021-11-03T17:14:21.139 INFO:tasks.workunit.client.0.smithi027.stdout:3/410: symlink d1/db/d52/l80 0 2021-11-03T17:14:21.139 INFO:tasks.workunit.client.0.smithi027.stdout:3/411: readlink d1/l1b 0 2021-11-03T17:14:21.139 INFO:tasks.workunit.client.0.smithi027.stdout:2/494: creat d0/f91 x:0 0 0 2021-11-03T17:14:21.141 INFO:tasks.workunit.client.0.smithi027.stdout:2/495: mkdir d0/d1b/d29/d92 0 2021-11-03T17:14:21.142 INFO:tasks.workunit.client.0.smithi027.stdout:2/496: write d0/d1b/d29/d54/f7f [4566797,47560] 0 2021-11-03T17:14:21.143 INFO:tasks.workunit.client.1.smithi115.stdout:2/566: dwrite d4/d6/d13/f4e [0,4194304] 0 2021-11-03T17:14:21.144 INFO:tasks.workunit.client.0.smithi027.stdout:2/497: stat d0/d1b/l36 0 2021-11-03T17:14:21.144 INFO:tasks.workunit.client.0.smithi027.stdout:2/498: symlink d0/d2a/l93 0 2021-11-03T17:14:21.145 INFO:tasks.workunit.client.0.smithi027.stdout:2/499: chown d0/d45/f7b 2035072 1 2021-11-03T17:14:21.145 INFO:tasks.workunit.client.1.smithi115.stdout:2/567: truncate d4/d6/d13/f2f 5511680 0 2021-11-03T17:14:21.147 INFO:tasks.workunit.client.0.smithi027.stdout:2/500: creat d0/d2a/f94 x:0 0 0 2021-11-03T17:14:21.150 INFO:tasks.workunit.client.0.smithi027.stdout:2/501: mkdir d0/d1b/d29/d95 0 2021-11-03T17:14:21.150 INFO:tasks.workunit.client.0.smithi027.stdout:2/502: write d0/d2a/f94 [268253,76775] 0 2021-11-03T17:14:21.151 INFO:tasks.workunit.client.0.smithi027.stdout:6/516: sync 2021-11-03T17:14:21.159 INFO:tasks.workunit.client.0.smithi027.stdout:3/412: dwrite d1/f18 [0,4194304] 0 2021-11-03T17:14:21.163 INFO:tasks.workunit.client.0.smithi027.stdout:3/413: dread d1/db/d52/f64 [0,4194304] 0 2021-11-03T17:14:21.168 INFO:tasks.workunit.client.0.smithi027.stdout:6/517: dread dc/d16/d1b/f28 [0,4194304] 0 2021-11-03T17:14:21.169 INFO:tasks.workunit.client.0.smithi027.stdout:2/503: dwrite d0/d5/d2c/f78 [0,4194304] 0 2021-11-03T17:14:21.176 INFO:tasks.workunit.client.0.smithi027.stdout:6/518: dread dc/d16/d1b/d24/f85 [0,4194304] 0 2021-11-03T17:14:21.179 INFO:tasks.workunit.client.0.smithi027.stdout:6/519: rmdir dc/d16/d1b/d24 39 2021-11-03T17:14:21.182 INFO:tasks.workunit.client.0.smithi027.stdout:2/504: read d0/d2a/f66 [2863931,80735] 0 2021-11-03T17:14:21.182 INFO:tasks.workunit.client.0.smithi027.stdout:2/505: chown d0/d1b/d29/d58/d5b/f62 914725 1 2021-11-03T17:14:21.183 INFO:tasks.workunit.client.0.smithi027.stdout:2/506: write d0/d2a/d5d/f65 [178175,9055] 0 2021-11-03T17:14:21.183 INFO:tasks.workunit.client.0.smithi027.stdout:2/507: fdatasync d0/d1b/d29/d58/d5b/f8b 0 2021-11-03T17:14:21.184 INFO:tasks.workunit.client.0.smithi027.stdout:2/508: chown d0/d5 23497184 1 2021-11-03T17:14:21.184 INFO:tasks.workunit.client.0.smithi027.stdout:2/509: mknod d0/d2a/d5d/d70/c96 1 2021-11-03T17:14:21.187 INFO:tasks.workunit.client.0.smithi027.stdout:3/414: dwrite d1/db/d52/f64 [0,4194304] 0 2021-11-03T17:14:21.188 INFO:tasks.workunit.client.0.smithi027.stdout:6/520: dread dc/d16/d1b/d3c/d51/f70 [0,4194304] 0 2021-11-03T17:14:21.190 INFO:tasks.workunit.client.0.smithi027.stdout:3/415: rename d1/f38 to d1/d6f/f81 0 2021-11-03T17:14:21.190 INFO:tasks.workunit.client.0.smithi027.stdout:3/416: mknod d1/db/d52/d57/c82 1 2021-11-03T17:14:21.190 INFO:tasks.workunit.client.0.smithi027.stdout:3/417: chown d1/d40/f5a 16695035 1 2021-11-03T17:14:21.190 INFO:tasks.workunit.client.0.smithi027.stdout:3/418: write d1/d40/f76 [319693,118225] 0 2021-11-03T17:14:21.191 INFO:tasks.workunit.client.0.smithi027.stdout:3/419: truncate d1/db/d52/f5d 1227734 0 2021-11-03T17:14:21.194 INFO:tasks.workunit.client.0.smithi027.stdout:2/510: dread d0/d5/f3 [0,4194304] 0 2021-11-03T17:14:21.194 INFO:tasks.workunit.client.0.smithi027.stdout:6/521: symlink dc/d16/d1b/d24/d7e/l95 0 2021-11-03T17:14:21.194 INFO:tasks.workunit.client.0.smithi027.stdout:6/522: write dc/d16/d1b/d2d/d34/f8a [939381,4041] 0 2021-11-03T17:14:21.199 INFO:tasks.workunit.client.0.smithi027.stdout:2/511: rename d0/d2a/f44 to d0/d1b/d29/d54/d67/d6a/f97 0 2021-11-03T17:14:21.201 INFO:tasks.workunit.client.0.smithi027.stdout:6/523: mkdir dc/d82/d57/d96 0 2021-11-03T17:14:21.206 INFO:tasks.workunit.client.0.smithi027.stdout:6/524: write dc/f4d [799531,6416] 0 2021-11-03T17:14:21.207 INFO:tasks.workunit.client.0.smithi027.stdout:3/420: dwrite d1/f18 [0,4194304] 0 2021-11-03T17:14:21.207 INFO:tasks.workunit.client.0.smithi027.stdout:3/421: chown d1/fa 358107299 1 2021-11-03T17:14:21.208 INFO:tasks.workunit.client.0.smithi027.stdout:6/525: creat dc/d82/d57/f97 x:0 0 0 2021-11-03T17:14:21.209 INFO:tasks.workunit.client.0.smithi027.stdout:6/526: stat dc/d16/d1b/d24/d7e/l95 0 2021-11-03T17:14:21.209 INFO:tasks.workunit.client.0.smithi027.stdout:6/527: mknod dc/d16/d1b/d2d/d34/d89/c98 1 2021-11-03T17:14:21.219 INFO:tasks.workunit.client.0.smithi027.stdout:6/528: dread dc/d16/d1b/d3c/f44 [4194304,4194304] 0 2021-11-03T17:14:21.219 INFO:tasks.workunit.client.0.smithi027.stdout:6/529: mknod dc/d16/d1b/d3c/d51/d5f/c99 1 2021-11-03T17:14:21.219 INFO:tasks.workunit.client.0.smithi027.stdout:6/530: dread - dc/d16/f7a zero size 2021-11-03T17:14:21.220 INFO:tasks.workunit.client.0.smithi027.stdout:6/531: mkdir dc/d16/d9a 0 2021-11-03T17:14:21.220 INFO:tasks.workunit.client.0.smithi027.stdout:6/532: write dc/d16/f7a [4276,23485] 0 2021-11-03T17:14:21.221 INFO:tasks.workunit.client.0.smithi027.stdout:3/422: dread d1/f1d [0,4194304] 0 2021-11-03T17:14:21.223 INFO:tasks.workunit.client.0.smithi027.stdout:3/423: getdents d1 0 2021-11-03T17:14:21.224 INFO:tasks.workunit.client.0.smithi027.stdout:3/424: creat d1/d6f/f83 x:0 0 0 2021-11-03T17:14:21.225 INFO:tasks.workunit.client.0.smithi027.stdout:2/512: dwrite d0/d1b/d1c/f33 [0,4194304] 0 2021-11-03T17:14:21.226 INFO:tasks.workunit.client.0.smithi027.stdout:3/425: symlink d1/db/l84 0 2021-11-03T17:14:21.230 INFO:tasks.workunit.client.0.smithi027.stdout:2/513: fsync d0/d1b/d29/d54/f7f 0 2021-11-03T17:14:21.234 INFO:tasks.workunit.client.0.smithi027.stdout:3/426: dread d1/db/d52/f5d [0,4194304] 0 2021-11-03T17:14:21.238 INFO:tasks.workunit.client.0.smithi027.stdout:3/427: link d1/d6f/f81 d1/d6f/f85 0 2021-11-03T17:14:21.245 INFO:tasks.workunit.client.0.smithi027.stdout:6/533: dwrite dc/d16/d1b/d24/f46 [4194304,4194304] 0 2021-11-03T17:14:21.250 INFO:tasks.workunit.client.0.smithi027.stdout:2/514: dwrite d0/d1b/d29/f2f [0,4194304] 0 2021-11-03T17:14:21.250 INFO:tasks.workunit.client.0.smithi027.stdout:2/515: getdents d0/d2a/d5d/d70 0 2021-11-03T17:14:21.256 INFO:tasks.workunit.client.0.smithi027.stdout:3/428: dwrite d1/f13 [4194304,4194304] 0 2021-11-03T17:14:21.256 INFO:tasks.workunit.client.0.smithi027.stdout:3/429: chown d1/db/d52/l56 5120634 1 2021-11-03T17:14:21.261 INFO:tasks.workunit.client.0.smithi027.stdout:3/430: read d1/fa [4696949,127323] 0 2021-11-03T17:14:21.262 INFO:tasks.workunit.client.0.smithi027.stdout:3/431: dread - d1/db/d52/d6a/f6e zero size 2021-11-03T17:14:21.263 INFO:tasks.workunit.client.0.smithi027.stdout:3/432: link d1/db/f12 d1/db/f86 0 2021-11-03T17:14:21.265 INFO:tasks.workunit.client.0.smithi027.stdout:3/433: rename d1/db/f12 to d1/d40/f87 0 2021-11-03T17:14:21.265 INFO:tasks.workunit.client.0.smithi027.stdout:3/434: readlink d1/db/l7e 0 2021-11-03T17:14:21.266 INFO:tasks.workunit.client.0.smithi027.stdout:3/435: truncate d1/db/f32 167303 0 2021-11-03T17:14:21.266 INFO:tasks.workunit.client.0.smithi027.stdout:3/436: stat d1/d40/f5a 0 2021-11-03T17:14:21.267 INFO:tasks.workunit.client.0.smithi027.stdout:3/437: mkdir d1/db/d52/d6a/d7c/d88 0 2021-11-03T17:14:21.269 INFO:tasks.workunit.client.0.smithi027.stdout:3/438: truncate d1/d40/f87 3436423 0 2021-11-03T17:14:21.269 INFO:tasks.workunit.client.0.smithi027.stdout:3/439: chown d1/d6f 12935 1 2021-11-03T17:14:21.271 INFO:tasks.workunit.client.0.smithi027.stdout:3/440: rename d1/db/d52/d6a/f6e to d1/db/d52/d6a/d7c/f89 0 2021-11-03T17:14:21.272 INFO:tasks.workunit.client.0.smithi027.stdout:3/441: unlink d1/d40/f70 0 2021-11-03T17:14:21.274 INFO:tasks.workunit.client.0.smithi027.stdout:3/442: creat d1/d40/f8a x:0 0 0 2021-11-03T17:14:21.274 INFO:tasks.workunit.client.0.smithi027.stdout:3/443: chown d1/d40/l63 287 1 2021-11-03T17:14:21.275 INFO:tasks.workunit.client.0.smithi027.stdout:3/444: mkdir d1/db/d52/d57/d8b 0 2021-11-03T17:14:21.276 INFO:tasks.workunit.client.0.smithi027.stdout:6/534: dwrite dc/f7b [0,4194304] 0 2021-11-03T17:14:21.276 INFO:tasks.workunit.client.0.smithi027.stdout:6/535: fsync dc/f88 0 2021-11-03T17:14:21.276 INFO:tasks.workunit.client.0.smithi027.stdout:6/536: write dc/d82/f5c [3182330,15087] 0 2021-11-03T17:14:21.277 INFO:tasks.workunit.client.0.smithi027.stdout:3/445: symlink d1/db/d52/d57/d8b/l8c 0 2021-11-03T17:14:21.277 INFO:tasks.workunit.client.0.smithi027.stdout:3/446: chown d1/f41 36555356 1 2021-11-03T17:14:21.277 INFO:tasks.workunit.client.0.smithi027.stdout:2/516: dwrite d0/d1b/d29/f2b [4194304,4194304] 0 2021-11-03T17:14:21.281 INFO:tasks.workunit.client.0.smithi027.stdout:3/447: link d1/d40/l42 d1/db/d52/d57/d8b/l8d 0 2021-11-03T17:14:21.281 INFO:tasks.workunit.client.0.smithi027.stdout:3/448: chown d1/f34 48067447 1 2021-11-03T17:14:21.281 INFO:tasks.workunit.client.0.smithi027.stdout:3/449: readlink d1/db/d52/d6a/l77 0 2021-11-03T17:14:21.282 INFO:tasks.workunit.client.0.smithi027.stdout:6/537: fsync dc/d16/d1b/d2d/d34/f8a 0 2021-11-03T17:14:21.282 INFO:tasks.workunit.client.0.smithi027.stdout:6/538: readlink dc/d16/d30/l67 0 2021-11-03T17:14:21.285 INFO:tasks.workunit.client.0.smithi027.stdout:2/517: dread d0/d2a/f23 [0,4194304] 0 2021-11-03T17:14:21.287 INFO:tasks.workunit.client.0.smithi027.stdout:2/518: read d0/f10 [2285531,17277] 0 2021-11-03T17:14:21.288 INFO:tasks.workunit.client.0.smithi027.stdout:3/450: rename d1/f22 to d1/d6f/f8e 0 2021-11-03T17:14:21.300 INFO:tasks.workunit.client.0.smithi027.stdout:2/519: dread d0/d5/f83 [0,4194304] 0 2021-11-03T17:14:21.302 INFO:tasks.workunit.client.0.smithi027.stdout:2/520: write d0/d1b/d87/f8c [1282731,91465] 0 2021-11-03T17:14:21.303 INFO:tasks.workunit.client.0.smithi027.stdout:3/451: dread d1/f5e [0,4194304] 0 2021-11-03T17:14:21.305 INFO:tasks.workunit.client.0.smithi027.stdout:2/521: creat d0/d1b/d29/d58/d5b/f98 x:0 0 0 2021-11-03T17:14:21.307 INFO:tasks.workunit.client.0.smithi027.stdout:3/452: link d1/d40/f46 d1/d40/f8f 0 2021-11-03T17:14:21.309 INFO:tasks.workunit.client.0.smithi027.stdout:3/453: creat d1/f90 x:0 0 0 2021-11-03T17:14:21.310 INFO:tasks.workunit.client.0.smithi027.stdout:3/454: mknod d1/db/d52/d6a/d7c/d88/c91 1 2021-11-03T17:14:21.312 INFO:tasks.workunit.client.0.smithi027.stdout:3/455: rename d1/d40/f5a to d1/db/d52/d6a/d7c/f92 0 2021-11-03T17:14:21.316 INFO:tasks.workunit.client.0.smithi027.stdout:3/456: rename d1/d6f/f8e to d1/db/f93 0 2021-11-03T17:14:21.317 INFO:tasks.workunit.client.0.smithi027.stdout:3/457: truncate d1/db/f93 7575622 0 2021-11-03T17:14:21.317 INFO:tasks.workunit.client.0.smithi027.stdout:3/458: readlink d1/d6f/l75 0 2021-11-03T17:14:21.322 INFO:tasks.workunit.client.0.smithi027.stdout:2/522: dwrite d0/d1b/d29/d58/f5c [0,4194304] 0 2021-11-03T17:14:21.325 INFO:tasks.workunit.client.0.smithi027.stdout:3/459: dread d1/d40/f87 [0,4194304] 0 2021-11-03T17:14:21.325 INFO:tasks.workunit.client.0.smithi027.stdout:3/460: chown d1/l2c 126473 1 2021-11-03T17:14:21.326 INFO:tasks.workunit.client.0.smithi027.stdout:2/523: rmdir d0/d7c/d89 0 2021-11-03T17:14:21.327 INFO:tasks.workunit.client.0.smithi027.stdout:2/524: read d0/d1b/d1c/f33 [2994952,1769] 0 2021-11-03T17:14:21.334 INFO:tasks.workunit.client.0.smithi027.stdout:3/461: unlink d1/db/d52/d6a/d7c/f92 0 2021-11-03T17:14:21.335 INFO:tasks.workunit.client.0.smithi027.stdout:3/462: dread - d1/d40/f5c zero size 2021-11-03T17:14:21.337 INFO:tasks.workunit.client.0.smithi027.stdout:2/525: link d0/d1b/d29/f4b d0/d1b/d1c/f99 0 2021-11-03T17:14:21.339 INFO:tasks.workunit.client.0.smithi027.stdout:2/526: creat d0/d45/f9a x:0 0 0 2021-11-03T17:14:21.342 INFO:tasks.workunit.client.0.smithi027.stdout:2/527: rmdir d0/d1b/d29/d95 0 2021-11-03T17:14:21.343 INFO:tasks.workunit.client.0.smithi027.stdout:2/528: fsync d0/f46 0 2021-11-03T17:14:21.351 INFO:tasks.workunit.client.0.smithi027.stdout:4/409: sync 2021-11-03T17:14:21.352 INFO:tasks.workunit.client.0.smithi027.stdout:9/528: sync 2021-11-03T17:14:21.353 INFO:tasks.workunit.client.0.smithi027.stdout:4/410: write d6/d9/d16/f19 [582695,54439] 0 2021-11-03T17:14:21.353 INFO:tasks.workunit.client.0.smithi027.stdout:4/411: truncate d6/db/d21/d41/f77 950511 0 2021-11-03T17:14:21.353 INFO:tasks.workunit.client.0.smithi027.stdout:4/412: mknod d6/d9/d28/d40/d81/c8d 1 2021-11-03T17:14:21.353 INFO:tasks.workunit.client.0.smithi027.stdout:4/413: read - d6/d9/d16/d36/f6a zero size 2021-11-03T17:14:21.354 INFO:tasks.workunit.client.0.smithi027.stdout:9/529: truncate db/f1f 480171 0 2021-11-03T17:14:21.354 INFO:tasks.workunit.client.0.smithi027.stdout:3/463: dwrite d1/d40/f48 [0,4194304] 0 2021-11-03T17:14:21.355 INFO:tasks.workunit.client.0.smithi027.stdout:9/530: getdents db/d1b/d1c/d1d 0 2021-11-03T17:14:21.357 INFO:tasks.workunit.client.0.smithi027.stdout:9/531: creat db/d1b/d2b/fba x:0 0 0 2021-11-03T17:14:21.358 INFO:tasks.workunit.client.0.smithi027.stdout:9/532: write db/d1b/f64 [282667,22069] 0 2021-11-03T17:14:21.359 INFO:tasks.workunit.client.0.smithi027.stdout:3/464: dread d1/f50 [0,4194304] 0 2021-11-03T17:14:21.359 INFO:tasks.workunit.client.0.smithi027.stdout:3/465: mknod d1/db/d52/d6a/c94 1 2021-11-03T17:14:21.360 INFO:tasks.workunit.client.0.smithi027.stdout:4/414: fdatasync d6/d9/d16/d36/d57/d5a/f67 0 2021-11-03T17:14:21.360 INFO:tasks.workunit.client.0.smithi027.stdout:2/529: dwrite d0/d1b/f82 [0,4194304] 0 2021-11-03T17:14:21.362 INFO:tasks.workunit.client.0.smithi027.stdout:9/533: dread db/d1b/d1c/f25 [0,4194304] 0 2021-11-03T17:14:21.363 INFO:tasks.workunit.client.0.smithi027.stdout:3/466: dread d1/d40/f87 [0,4194304] 0 2021-11-03T17:14:21.363 INFO:tasks.workunit.client.0.smithi027.stdout:3/467: chown d1/db/d52 29078543 1 2021-11-03T17:14:21.373 INFO:tasks.workunit.client.0.smithi027.stdout:9/534: dread db/d1b/d23/f73 [0,4194304] 0 2021-11-03T17:14:21.373 INFO:tasks.workunit.client.0.smithi027.stdout:9/535: read - db/d1b/d1c/d1d/f78 zero size 2021-11-03T17:14:21.375 INFO:tasks.workunit.client.0.smithi027.stdout:4/415: stat d6/l1c 0 2021-11-03T17:14:21.376 INFO:tasks.workunit.client.0.smithi027.stdout:4/416: dread - d6/d9/d16/d36/d70/d3e/f6b zero size 2021-11-03T17:14:21.376 INFO:tasks.workunit.client.0.smithi027.stdout:4/417: write d6/db/f8a [232445,124809] 0 2021-11-03T17:14:21.376 INFO:tasks.workunit.client.0.smithi027.stdout:4/418: chown d6/db/d21 7 1 2021-11-03T17:14:21.380 INFO:tasks.workunit.client.0.smithi027.stdout:3/468: dwrite d1/f17 [4194304,4194304] 0 2021-11-03T17:14:21.380 INFO:tasks.workunit.client.0.smithi027.stdout:2/530: creat d0/f9b x:0 0 0 2021-11-03T17:14:21.382 INFO:tasks.workunit.client.0.smithi027.stdout:9/536: mkdir db/d1b/d2b/d41/dad/dbb 0 2021-11-03T17:14:21.383 INFO:tasks.workunit.client.0.smithi027.stdout:9/537: mknod db/d1b/d23/cbc 1 2021-11-03T17:14:21.383 INFO:tasks.workunit.client.0.smithi027.stdout:9/538: chown db/d1b/d23/f73 53816825 1 2021-11-03T17:14:21.383 INFO:tasks.workunit.client.0.smithi027.stdout:9/539: chown db/d1b/d2b/fba 258607 1 2021-11-03T17:14:21.387 INFO:tasks.workunit.client.0.smithi027.stdout:4/419: creat d6/db/f8e x:0 0 0 2021-11-03T17:14:21.392 INFO:tasks.workunit.client.0.smithi027.stdout:3/469: creat d1/db/d52/d6a/f95 x:0 0 0 2021-11-03T17:14:21.394 INFO:tasks.workunit.client.0.smithi027.stdout:2/531: rmdir d0/d1b/d29/d54 39 2021-11-03T17:14:21.395 INFO:tasks.workunit.client.0.smithi027.stdout:2/532: mknod d0/d2a/d88/c9c 1 2021-11-03T17:14:21.397 INFO:tasks.workunit.client.0.smithi027.stdout:9/540: link db/d1b/d1c/d1d/d40/f70 db/d1b/d2b/d41/d97/fbd 0 2021-11-03T17:14:21.398 INFO:tasks.workunit.client.0.smithi027.stdout:9/541: getdents db/d1b/d2b/d41/d95 0 2021-11-03T17:14:21.399 INFO:tasks.workunit.client.0.smithi027.stdout:9/542: getdents db/d1b/d23/d2c 0 2021-11-03T17:14:21.399 INFO:tasks.workunit.client.0.smithi027.stdout:9/543: chown db/d1b/d1c/d1d/fa1 4755045 1 2021-11-03T17:14:21.400 INFO:tasks.workunit.client.0.smithi027.stdout:3/470: dread d1/db/d52/f64 [0,4194304] 0 2021-11-03T17:14:21.401 INFO:tasks.workunit.client.0.smithi027.stdout:9/544: rename db/d1b/d2b/d41/dad/laf to db/d1b/d1c/d1d/d40/lbe 0 2021-11-03T17:14:21.401 INFO:tasks.workunit.client.0.smithi027.stdout:9/545: fdatasync db/d1b/d23/d2c/f6a 0 2021-11-03T17:14:21.402 INFO:tasks.workunit.client.0.smithi027.stdout:3/471: rmdir d1/db/d52 39 2021-11-03T17:14:21.403 INFO:tasks.workunit.client.0.smithi027.stdout:9/546: creat db/d1b/d2b/d5e/daa/fbf x:0 0 0 2021-11-03T17:14:21.403 INFO:tasks.workunit.client.0.smithi027.stdout:9/547: write db/d1b/d1c/d1d/f1e [1234135,127801] 0 2021-11-03T17:14:21.405 INFO:tasks.workunit.client.0.smithi027.stdout:2/533: dread d0/d5/d2c/f3d [0,4194304] 0 2021-11-03T17:14:21.405 INFO:tasks.workunit.client.0.smithi027.stdout:3/472: rename d1/l4 to d1/l96 0 2021-11-03T17:14:21.406 INFO:tasks.workunit.client.0.smithi027.stdout:2/534: creat d0/d1b/d29/f9d x:0 0 0 2021-11-03T17:14:21.406 INFO:tasks.workunit.client.0.smithi027.stdout:2/535: mknod d0/d1b/d29/c9e 1 2021-11-03T17:14:21.408 INFO:tasks.workunit.client.0.smithi027.stdout:9/548: write db/d1b/f45 [3177318,56405] 0 2021-11-03T17:14:21.408 INFO:tasks.workunit.client.0.smithi027.stdout:9/549: mknod db/d1b/d1c/cc0 1 2021-11-03T17:14:21.408 INFO:tasks.workunit.client.0.smithi027.stdout:4/420: dwrite d6/d9/d28/d40/f8b [0,4194304] 0 2021-11-03T17:14:21.409 INFO:tasks.workunit.client.0.smithi027.stdout:4/421: stat d6/d9/d28/d40/d81 0 2021-11-03T17:14:21.410 INFO:tasks.workunit.client.0.smithi027.stdout:9/550: rename db/l11 to db/lc1 0 2021-11-03T17:14:21.410 INFO:tasks.workunit.client.0.smithi027.stdout:9/551: truncate db/d1b/d2b/d5e/daa/fb7 471822 0 2021-11-03T17:14:21.413 INFO:tasks.workunit.client.0.smithi027.stdout:4/422: mkdir d6/d9/d8f 0 2021-11-03T17:14:21.413 INFO:tasks.workunit.client.0.smithi027.stdout:4/423: readlink d6/db/d21/d41/l50 0 2021-11-03T17:14:21.421 INFO:tasks.workunit.client.0.smithi027.stdout:2/536: dwrite d0/f90 [0,4194304] 0 2021-11-03T17:14:21.421 INFO:tasks.workunit.client.0.smithi027.stdout:2/537: truncate d0/f90 4890122 0 2021-11-03T17:14:21.421 INFO:tasks.workunit.client.0.smithi027.stdout:4/424: symlink d6/d9/d16/d36/d57/d5a/d63/l90 0 2021-11-03T17:14:21.428 INFO:tasks.workunit.client.0.smithi027.stdout:9/552: rmdir db 39 2021-11-03T17:14:21.430 INFO:tasks.workunit.client.0.smithi027.stdout:2/538: getdents d0/d2a/d61 0 2021-11-03T17:14:21.433 INFO:tasks.workunit.client.0.smithi027.stdout:4/425: unlink d6/d9/d16/d36/d70/d3e/f47 0 2021-11-03T17:14:21.435 INFO:tasks.workunit.client.0.smithi027.stdout:2/539: creat d0/d7c/f9f x:0 0 0 2021-11-03T17:14:21.435 INFO:tasks.workunit.client.0.smithi027.stdout:2/540: mknod d0/d2a/d5d/d70/ca0 1 2021-11-03T17:14:21.436 INFO:tasks.workunit.client.0.smithi027.stdout:9/553: dread db/f14 [0,4194304] 0 2021-11-03T17:14:21.436 INFO:tasks.workunit.client.0.smithi027.stdout:2/541: write d0/d2a/f30 [2529194,39010] 0 2021-11-03T17:14:21.438 INFO:tasks.workunit.client.0.smithi027.stdout:9/554: write db/f14 [1913661,122336] 0 2021-11-03T17:14:21.439 INFO:tasks.workunit.client.0.smithi027.stdout:2/542: rename d0/d1b/d29/d54/l72 to d0/d1b/d1c/la1 0 2021-11-03T17:14:21.457 INFO:tasks.workunit.client.0.smithi027.stdout:4/426: dwrite f4 [0,4194304] 0 2021-11-03T17:14:21.457 INFO:tasks.workunit.client.0.smithi027.stdout:4/427: stat d6/d9/d16/d36/d70/l6f 0 2021-11-03T17:14:21.457 INFO:tasks.workunit.client.0.smithi027.stdout:4/428: write d6/d9/d16/f19 [1137307,119589] 0 2021-11-03T17:14:21.458 INFO:tasks.workunit.client.0.smithi027.stdout:4/429: mknod d6/c91 1 2021-11-03T17:14:21.458 INFO:tasks.workunit.client.0.smithi027.stdout:4/430: write d6/d9/d16/d36/d70/f71 [1359695,10536] 0 2021-11-03T17:14:21.458 INFO:tasks.workunit.client.0.smithi027.stdout:4/431: write d6/d9/d16/d36/d70/d3e/d58/f61 [758331,107471] 0 2021-11-03T17:14:21.459 INFO:tasks.workunit.client.0.smithi027.stdout:4/432: mkdir d6/d9/d16/d36/d70/d3e/d92 0 2021-11-03T17:14:21.461 INFO:tasks.workunit.client.0.smithi027.stdout:4/433: creat d6/db/d21/f93 x:0 0 0 2021-11-03T17:14:21.464 INFO:tasks.workunit.client.0.smithi027.stdout:9/555: dwrite db/d1b/d1c/d1d/d40/d60/f9a [0,4194304] 0 2021-11-03T17:14:21.465 INFO:tasks.workunit.client.0.smithi027.stdout:9/556: chown db/d1b/d2b/d41/d6c/f93 32153514 1 2021-11-03T17:14:21.466 INFO:tasks.workunit.client.0.smithi027.stdout:2/543: dwrite d0/d1b/d29/d58/f5c [4194304,4194304] 0 2021-11-03T17:14:21.467 INFO:tasks.workunit.client.0.smithi027.stdout:9/557: symlink db/d1b/d1c/d1d/lc2 0 2021-11-03T17:14:21.467 INFO:tasks.workunit.client.0.smithi027.stdout:9/558: fsync db/f13 0 2021-11-03T17:14:21.474 INFO:tasks.workunit.client.0.smithi027.stdout:2/544: dread d0/d1b/d1c/f33 [0,4194304] 0 2021-11-03T17:14:21.474 INFO:tasks.workunit.client.0.smithi027.stdout:2/545: mknod d0/d1b/ca2 1 2021-11-03T17:14:21.476 INFO:tasks.workunit.client.0.smithi027.stdout:9/559: rmdir db/d1b 39 2021-11-03T17:14:21.476 INFO:tasks.workunit.client.0.smithi027.stdout:9/560: mknod db/d1b/d1c/d1d/d40/d60/cc3 1 2021-11-03T17:14:21.477 INFO:tasks.workunit.client.0.smithi027.stdout:9/561: dread db/d1b/d1c/d1d/d40/d60/fa4 [0,4194304] 0 2021-11-03T17:14:21.481 INFO:tasks.workunit.client.0.smithi027.stdout:2/546: creat d0/d1b/d29/d54/d67/d6a/fa3 x:0 0 0 2021-11-03T17:14:21.486 INFO:tasks.workunit.client.0.smithi027.stdout:9/562: dread db/fd [0,4194304] 0 2021-11-03T17:14:21.490 INFO:tasks.workunit.client.0.smithi027.stdout:9/563: dread db/d1b/d23/d2c/f89 [0,4194304] 0 2021-11-03T17:14:21.492 INFO:tasks.workunit.client.0.smithi027.stdout:9/564: link db/d1b/d2b/d41/l58 db/lc4 0 2021-11-03T17:14:21.493 INFO:tasks.workunit.client.0.smithi027.stdout:9/565: mkdir db/d1b/d1c/dc5 0 2021-11-03T17:14:21.494 INFO:tasks.workunit.client.0.smithi027.stdout:9/566: creat db/d1b/d1c/d1d/d40/d60/fc6 x:0 0 0 2021-11-03T17:14:21.496 INFO:tasks.workunit.client.0.smithi027.stdout:9/567: mkdir db/d1b/d2b/dc7 0 2021-11-03T17:14:21.496 INFO:tasks.workunit.client.0.smithi027.stdout:9/568: dread - db/fa8 zero size 2021-11-03T17:14:21.497 INFO:tasks.workunit.client.0.smithi027.stdout:9/569: creat db/fc8 x:0 0 0 2021-11-03T17:14:21.498 INFO:tasks.workunit.client.0.smithi027.stdout:9/570: read db/fd [3390813,81527] 0 2021-11-03T17:14:21.498 INFO:tasks.workunit.client.0.smithi027.stdout:9/571: readlink db/d1b/d2b/d5e/l76 0 2021-11-03T17:14:21.499 INFO:tasks.workunit.client.0.smithi027.stdout:9/572: symlink db/d1b/d2b/d9c/lc9 0 2021-11-03T17:14:21.507 INFO:tasks.workunit.client.0.smithi027.stdout:9/573: dread db/d1b/d1c/d1d/d40/f48 [0,4194304] 0 2021-11-03T17:14:21.520 INFO:tasks.workunit.client.0.smithi027.stdout:9/574: dwrite db/d1b/d2b/f86 [0,4194304] 0 2021-11-03T17:14:21.520 INFO:tasks.workunit.client.0.smithi027.stdout:9/575: chown db/d1b/f38 3751 1 2021-11-03T17:14:21.537 INFO:tasks.workunit.client.0.smithi027.stdout:9/576: dwrite db/d1b/f64 [0,4194304] 0 2021-11-03T17:14:21.539 INFO:tasks.workunit.client.0.smithi027.stdout:9/577: stat db/l1a 0 2021-11-03T17:14:21.541 INFO:tasks.workunit.client.0.smithi027.stdout:9/578: creat db/d1b/d1c/fca x:0 0 0 2021-11-03T17:14:21.541 INFO:tasks.workunit.client.0.smithi027.stdout:9/579: mknod db/d1b/d2b/dc7/ccb 1 2021-11-03T17:14:21.542 INFO:tasks.workunit.client.0.smithi027.stdout:9/580: write db/d1b/d23/f73 [576940,119680] 0 2021-11-03T17:14:21.547 INFO:tasks.workunit.client.0.smithi027.stdout:9/581: truncate db/d1b/d2b/d41/f74 1037099 0 2021-11-03T17:14:21.549 INFO:tasks.workunit.client.0.smithi027.stdout:9/582: mkdir db/d1b/d1c/d1d/d40/dcc 0 2021-11-03T17:14:21.549 INFO:tasks.workunit.client.0.smithi027.stdout:9/583: dread - db/d1b/d1c/d1d/fb6 zero size 2021-11-03T17:14:21.549 INFO:tasks.workunit.client.0.smithi027.stdout:9/584: write db/d1b/d2b/d41/d6c/f93 [985002,103086] 0 2021-11-03T17:14:21.550 INFO:tasks.workunit.client.0.smithi027.stdout:9/585: chown db/f5b 1400 1 2021-11-03T17:14:21.550 INFO:tasks.workunit.client.0.smithi027.stdout:9/586: getdents db/d1b/d2b/d41/d95 0 2021-11-03T17:14:21.550 INFO:tasks.workunit.client.0.smithi027.stdout:9/587: rmdir db/d1b/d23 39 2021-11-03T17:14:21.703 INFO:tasks.workunit.client.0.smithi027.stdout:0/572: sync 2021-11-03T17:14:21.706 INFO:tasks.workunit.client.0.smithi027.stdout:0/573: mkdir d9/d12/d8f/dc7 0 2021-11-03T17:14:21.706 INFO:tasks.workunit.client.0.smithi027.stdout:0/574: read - d9/da/d64/d70/d5f/f84 zero size 2021-11-03T17:14:21.708 INFO:tasks.workunit.client.0.smithi027.stdout:0/575: mkdir d9/da/d30/d92/dc8 0 2021-11-03T17:14:21.708 INFO:tasks.workunit.client.0.smithi027.stdout:0/576: chown d9/da/d30/d92/dc8 3084 1 2021-11-03T17:14:21.709 INFO:tasks.workunit.client.0.smithi027.stdout:0/577: chown d9/da/d64/d70/f6a 267953 1 2021-11-03T17:14:21.731 INFO:tasks.workunit.client.0.smithi027.stdout:0/578: dwrite d9/da/d30/da7/f8a [0,4194304] 0 2021-11-03T17:14:21.731 INFO:tasks.workunit.client.0.smithi027.stdout:0/579: write d9/da/d64/d70/d6f/fa1 [387058,66154] 0 2021-11-03T17:14:21.735 INFO:tasks.workunit.client.0.smithi027.stdout:0/580: mkdir d9/da/d64/d70/d6d/d7a/dc9 0 2021-11-03T17:14:21.736 INFO:tasks.workunit.client.0.smithi027.stdout:0/581: write d9/f23 [901991,50805] 0 2021-11-03T17:14:21.736 INFO:tasks.workunit.client.0.smithi027.stdout:0/582: dread - d9/fb4 zero size 2021-11-03T17:14:21.736 INFO:tasks.workunit.client.0.smithi027.stdout:0/583: chown d9/da/d64/d70/fc5 0 1 2021-11-03T17:14:21.737 INFO:tasks.workunit.client.0.smithi027.stdout:0/584: unlink d9/d12/l69 0 2021-11-03T17:14:21.741 INFO:tasks.workunit.client.0.smithi027.stdout:0/585: unlink d9/da/d31/l61 0 2021-11-03T17:14:21.741 INFO:tasks.workunit.client.0.smithi027.stdout:0/586: write d9/da/d64/d70/d6d/d7a/fa9 [654424,44148] 0 2021-11-03T17:14:21.760 INFO:tasks.workunit.client.1.smithi115.stdout:9/480: sync 2021-11-03T17:14:21.761 INFO:tasks.workunit.client.1.smithi115.stdout:9/481: fdatasync d1/d1a/d48/d27/d34/f93 0 2021-11-03T17:14:21.761 INFO:tasks.workunit.client.0.smithi027.stdout:0/587: dwrite d9/f95 [0,4194304] 0 2021-11-03T17:14:21.763 INFO:tasks.workunit.client.0.smithi027.stdout:0/588: write d9/da/f39 [3707452,29463] 0 2021-11-03T17:14:21.767 INFO:tasks.workunit.client.0.smithi027.stdout:0/589: mkdir d9/da/d30/d92/dca 0 2021-11-03T17:14:21.768 INFO:tasks.workunit.client.0.smithi027.stdout:0/590: creat d9/da/d64/d70/fcb x:0 0 0 2021-11-03T17:14:21.773 INFO:tasks.workunit.client.1.smithi115.stdout:9/482: dread d1/d1a/d48/d27/f89 [0,4194304] 0 2021-11-03T17:14:21.773 INFO:tasks.workunit.client.0.smithi027.stdout:0/591: dread d9/d12/fa6 [0,4194304] 0 2021-11-03T17:14:21.774 INFO:tasks.workunit.client.0.smithi027.stdout:0/592: write d9/d12/fa6 [549336,38812] 0 2021-11-03T17:14:21.778 INFO:tasks.workunit.client.0.smithi027.stdout:0/593: fdatasync d9/da/d64/d70/d6f/fa1 0 2021-11-03T17:14:21.780 INFO:tasks.workunit.client.0.smithi027.stdout:0/594: mkdir d9/da/d64/dcc 0 2021-11-03T17:14:21.780 INFO:tasks.workunit.client.0.smithi027.stdout:0/595: write d9/d12/f75 [595418,46524] 0 2021-11-03T17:14:21.781 INFO:tasks.workunit.client.0.smithi027.stdout:0/596: write d9/da/f16 [727123,21775] 0 2021-11-03T17:14:21.781 INFO:tasks.workunit.client.0.smithi027.stdout:0/597: write d9/f4f [86521,70015] 0 2021-11-03T17:14:21.783 INFO:tasks.workunit.client.0.smithi027.stdout:0/598: rmdir d9/da/d30/d92/dc8 0 2021-11-03T17:14:21.788 INFO:tasks.workunit.client.1.smithi115.stdout:9/483: dwrite d1/d1a/d48/f26 [0,4194304] 0 2021-11-03T17:14:21.801 INFO:tasks.workunit.client.0.smithi027.stdout:0/599: dwrite d9/f1f [0,4194304] 0 2021-11-03T17:14:21.802 INFO:tasks.workunit.client.0.smithi027.stdout:0/600: fdatasync d9/f42 0 2021-11-03T17:14:21.802 INFO:tasks.workunit.client.0.smithi027.stdout:0/601: symlink d9/d12/lcd 0 2021-11-03T17:14:21.803 INFO:tasks.workunit.client.0.smithi027.stdout:0/602: creat d9/fce x:0 0 0 2021-11-03T17:14:21.804 INFO:tasks.workunit.client.0.smithi027.stdout:0/603: write d9/da/d31/f48 [2040448,111624] 0 2021-11-03T17:14:21.805 INFO:tasks.workunit.client.0.smithi027.stdout:0/604: creat d9/da/d30/dac/fcf x:0 0 0 2021-11-03T17:14:21.806 INFO:tasks.workunit.client.0.smithi027.stdout:0/605: creat d9/da/d30/dac/dba/fd0 x:0 0 0 2021-11-03T17:14:21.813 INFO:tasks.workunit.client.0.smithi027.stdout:0/606: dread d9/f47 [0,4194304] 0 2021-11-03T17:14:21.814 INFO:tasks.workunit.client.0.smithi027.stdout:0/607: read d9/f4d [1482047,93579] 0 2021-11-03T17:14:21.815 INFO:tasks.workunit.client.0.smithi027.stdout:0/608: write d9/da/d64/d70/d6d/f7d [575072,78828] 0 2021-11-03T17:14:21.817 INFO:tasks.workunit.client.0.smithi027.stdout:0/609: link d9/d12/l45 d9/da/d64/d70/ld1 0 2021-11-03T17:14:21.818 INFO:tasks.workunit.client.0.smithi027.stdout:0/610: creat d9/da/d30/d92/fd2 x:0 0 0 2021-11-03T17:14:21.819 INFO:tasks.workunit.client.0.smithi027.stdout:0/611: mkdir d9/dd3 0 2021-11-03T17:14:21.820 INFO:tasks.workunit.client.0.smithi027.stdout:0/612: creat d9/da/d30/dac/fd4 x:0 0 0 2021-11-03T17:14:21.820 INFO:tasks.workunit.client.0.smithi027.stdout:0/613: dread - d9/da/d30/dac/fd4 zero size 2021-11-03T17:14:21.821 INFO:tasks.workunit.client.0.smithi027.stdout:0/614: creat d9/da/d30/d92/dca/fd5 x:0 0 0 2021-11-03T17:14:21.824 INFO:tasks.workunit.client.0.smithi027.stdout:0/615: rename d9/f4f to d9/da/d64/d70/d5f/fd6 0 2021-11-03T17:14:21.824 INFO:tasks.workunit.client.0.smithi027.stdout:0/616: truncate d9/da/d30/da7/fab 248538 0 2021-11-03T17:14:21.824 INFO:tasks.workunit.client.0.smithi027.stdout:0/617: symlink d9/dd3/ld7 0 2021-11-03T17:14:21.842 INFO:tasks.workunit.client.0.smithi027.stdout:0/618: dwrite d9/da/d31/fae [0,4194304] 0 2021-11-03T17:14:21.844 INFO:tasks.workunit.client.0.smithi027.stdout:0/619: dread d9/fc1 [0,4194304] 0 2021-11-03T17:14:21.862 INFO:tasks.workunit.client.0.smithi027.stdout:0/620: dwrite d9/da/d30/d92/fa0 [0,4194304] 0 2021-11-03T17:14:21.862 INFO:tasks.workunit.client.0.smithi027.stdout:0/621: chown d9/da/d64/l6b 4447224 1 2021-11-03T17:14:21.864 INFO:tasks.workunit.client.1.smithi115.stdout:1/446: sync 2021-11-03T17:14:21.864 INFO:tasks.workunit.client.1.smithi115.stdout:1/447: chown d7/dc/dd/d24/d26/f85 0 1 2021-11-03T17:14:21.865 INFO:tasks.workunit.client.1.smithi115.stdout:1/448: mknod d7/dc/dd/d3e/d58/c92 1 2021-11-03T17:14:21.865 INFO:tasks.workunit.client.1.smithi115.stdout:1/449: creat d7/dc/dd/d24/d4b/d55/f93 x:0 0 0 2021-11-03T17:14:21.866 INFO:tasks.workunit.client.1.smithi115.stdout:1/450: read - d7/dc/dd/d42/f8d zero size 2021-11-03T17:14:21.867 INFO:tasks.workunit.client.0.smithi027.stdout:0/622: rename d9/da/d64/d70/f6a to d9/da/d30/fd8 0 2021-11-03T17:14:21.867 INFO:tasks.workunit.client.1.smithi115.stdout:1/451: symlink d7/dc/dd/d24/d43/l94 0 2021-11-03T17:14:21.868 INFO:tasks.workunit.client.1.smithi115.stdout:1/452: truncate d7/dc/f2f 2219006 0 2021-11-03T17:14:21.868 INFO:tasks.workunit.client.1.smithi115.stdout:1/453: readlink d7/dc/dd/d42/l6e 0 2021-11-03T17:14:21.868 INFO:tasks.workunit.client.1.smithi115.stdout:1/454: write d7/dc/dd/d24/d4d/f7b [609019,103464] 0 2021-11-03T17:14:21.869 INFO:tasks.workunit.client.1.smithi115.stdout:1/455: creat d7/dc/dd/f95 x:0 0 0 2021-11-03T17:14:21.869 INFO:tasks.workunit.client.1.smithi115.stdout:1/456: rename d7/dc/dd/d24/d26/l29 to d7/dc/d2a/d73/l96 0 2021-11-03T17:14:21.870 INFO:tasks.workunit.client.1.smithi115.stdout:1/457: creat d7/dc/dd/d24/d4b/d55/f97 x:0 0 0 2021-11-03T17:14:21.870 INFO:tasks.workunit.client.1.smithi115.stdout:1/458: readlink d7/dc/dd/d3e/l56 0 2021-11-03T17:14:21.870 INFO:tasks.workunit.client.1.smithi115.stdout:1/459: mknod d7/dc/dd/d24/c98 1 2021-11-03T17:14:21.870 INFO:tasks.workunit.client.1.smithi115.stdout:1/460: stat d7/dc/dd/d3e/d58 0 2021-11-03T17:14:21.876 INFO:tasks.workunit.client.0.smithi027.stdout:1/492: sync 2021-11-03T17:14:21.885 INFO:tasks.workunit.client.0.smithi027.stdout:0/623: dwrite d9/da/d64/d70/d6f/fa3 [0,4194304] 0 2021-11-03T17:14:21.887 INFO:tasks.workunit.client.0.smithi027.stdout:0/624: link d9/f1f d9/da/d64/d70/d6d/d7a/dc9/fd9 0 2021-11-03T17:14:21.888 INFO:tasks.workunit.client.0.smithi027.stdout:0/625: symlink d9/da/d64/d70/d6d/d7a/dc9/lda 0 2021-11-03T17:14:21.890 INFO:tasks.workunit.client.0.smithi027.stdout:0/626: rename d9/da/d31/f5c to d9/da/d64/d70/fdb 0 2021-11-03T17:14:21.891 INFO:tasks.workunit.client.0.smithi027.stdout:0/627: write d9/f72 [5461902,98040] 0 2021-11-03T17:14:21.891 INFO:tasks.workunit.client.0.smithi027.stdout:0/628: fsync d9/da/d30/f3c 0 2021-11-03T17:14:21.892 INFO:tasks.workunit.client.0.smithi027.stdout:1/493: dwrite f2 [4194304,4194304] 0 2021-11-03T17:14:21.893 INFO:tasks.workunit.client.0.smithi027.stdout:0/629: mkdir d9/da/d30/dac/dba/ddc 0 2021-11-03T17:14:21.893 INFO:tasks.workunit.client.0.smithi027.stdout:0/630: write d9/fb7 [264067,101905] 0 2021-11-03T17:14:21.894 INFO:tasks.workunit.client.0.smithi027.stdout:0/631: truncate d9/f42 185672 0 2021-11-03T17:14:21.900 INFO:tasks.workunit.client.1.smithi115.stdout:1/461: write d7/dc/dd/d24/d4d/d60/f6d [93147,88314] 0 2021-11-03T17:14:21.900 INFO:tasks.workunit.client.1.smithi115.stdout:1/462: creat d7/dc/dd/d24/d43/f99 x:0 0 0 2021-11-03T17:14:21.901 INFO:tasks.workunit.client.1.smithi115.stdout:1/463: rmdir d7/dc/dd/d24 39 2021-11-03T17:14:21.902 INFO:tasks.workunit.client.1.smithi115.stdout:1/464: creat d7/d38/d84/f9a x:0 0 0 2021-11-03T17:14:21.902 INFO:tasks.workunit.client.0.smithi027.stdout:1/494: dread d6/fa [0,4194304] 0 2021-11-03T17:14:21.903 INFO:tasks.workunit.client.1.smithi115.stdout:1/465: link d7/dc/dd/d24/d26/f85 d7/d38/d59/f9b 0 2021-11-03T17:14:21.904 INFO:tasks.workunit.client.1.smithi115.stdout:1/466: write d7/dc/dd/d24/d4b/f3a [965091,41059] 0 2021-11-03T17:14:21.904 INFO:tasks.workunit.client.1.smithi115.stdout:1/467: readlink d7/dc/dd/d24/d43/l94 0 2021-11-03T17:14:21.905 INFO:tasks.workunit.client.0.smithi027.stdout:1/495: rmdir d6/d11/dac/d79/d37 39 2021-11-03T17:14:21.906 INFO:tasks.workunit.client.0.smithi027.stdout:0/632: dread d9/da/d64/f76 [0,4194304] 0 2021-11-03T17:14:21.906 INFO:tasks.workunit.client.0.smithi027.stdout:0/633: chown d9/da/d64 39 1 2021-11-03T17:14:21.909 INFO:tasks.workunit.client.0.smithi027.stdout:1/496: creat d6/d11/dac/d79/d25/d28/d5c/fc0 x:0 0 0 2021-11-03T17:14:21.909 INFO:tasks.workunit.client.0.smithi027.stdout:1/497: write d6/d11/dac/d79/d25/d28/d5c/d81/d7d/f7e [1872398,67499] 0 2021-11-03T17:14:21.909 INFO:tasks.workunit.client.0.smithi027.stdout:1/498: mknod d6/d11/dac/d79/d31/cc1 1 2021-11-03T17:14:21.910 INFO:tasks.workunit.client.0.smithi027.stdout:0/634: fdatasync d9/f72 0 2021-11-03T17:14:21.911 INFO:tasks.workunit.client.0.smithi027.stdout:1/499: creat d6/d11/dac/d79/d25/d5e/d72/fc2 x:0 0 0 2021-11-03T17:14:21.911 INFO:tasks.workunit.client.0.smithi027.stdout:1/500: chown d6/d11/dac/d79/d25/f2d 905087 1 2021-11-03T17:14:21.913 INFO:tasks.workunit.client.0.smithi027.stdout:0/635: link d9/d12/f17 d9/d12/d8f/dc7/fdd 0 2021-11-03T17:14:21.913 INFO:tasks.workunit.client.0.smithi027.stdout:0/636: fsync d9/fc1 0 2021-11-03T17:14:21.913 INFO:tasks.workunit.client.0.smithi027.stdout:1/501: dread d6/d11/dac/d79/f59 [0,4194304] 0 2021-11-03T17:14:21.913 INFO:tasks.workunit.client.0.smithi027.stdout:1/502: truncate d6/fa3 1033542 0 2021-11-03T17:14:21.914 INFO:tasks.workunit.client.0.smithi027.stdout:0/637: read d9/d12/f1d [3040421,24471] 0 2021-11-03T17:14:21.915 INFO:tasks.workunit.client.1.smithi115.stdout:1/468: dread d7/dc/dd/d24/d4b/d55/f66 [0,4194304] 0 2021-11-03T17:14:21.915 INFO:tasks.workunit.client.1.smithi115.stdout:1/469: chown d7/dc/dd/d24/d26/f4f 59413 1 2021-11-03T17:14:21.915 INFO:tasks.workunit.client.1.smithi115.stdout:1/470: mknod d7/dc/dd/d3e/c9c 1 2021-11-03T17:14:21.916 INFO:tasks.workunit.client.1.smithi115.stdout:1/471: mkdir d7/d9d 0 2021-11-03T17:14:21.916 INFO:tasks.workunit.client.1.smithi115.stdout:1/472: write d7/d38/f5c [26565,56945] 0 2021-11-03T17:14:21.917 INFO:tasks.workunit.client.1.smithi115.stdout:1/473: symlink d7/d9d/l9e 0 2021-11-03T17:14:21.917 INFO:tasks.workunit.client.0.smithi027.stdout:0/638: dread d9/da/d64/f76 [0,4194304] 0 2021-11-03T17:14:21.918 INFO:tasks.workunit.client.1.smithi115.stdout:1/474: write d7/dc/d1a/f3d [3078805,106639] 0 2021-11-03T17:14:21.918 INFO:tasks.workunit.client.1.smithi115.stdout:1/475: truncate d7/dc/dd/d24/d4d/d60/f6d 986136 0 2021-11-03T17:14:21.918 INFO:tasks.workunit.client.0.smithi027.stdout:0/639: link d9/da/d64/d70/ld1 d9/da/d30/dac/dba/ddc/lde 0 2021-11-03T17:14:21.925 INFO:tasks.workunit.client.1.smithi115.stdout:1/476: rename d7/dc/d1a/l82 to d7/dc/dd/d24/d4b/l9f 0 2021-11-03T17:14:21.934 INFO:tasks.workunit.client.0.smithi027.stdout:1/503: dwrite d6/d11/dac/d79/d31/f7f [0,4194304] 0 2021-11-03T17:14:21.935 INFO:tasks.workunit.client.0.smithi027.stdout:1/504: mknod d6/de/cc3 1 2021-11-03T17:14:21.935 INFO:tasks.workunit.client.1.smithi115.stdout:1/477: dread f2 [0,4194304] 0 2021-11-03T17:14:21.940 INFO:tasks.workunit.client.1.smithi115.stdout:1/478: rename d7/dc/d2a to d7/dc/d1a/da0 0 2021-11-03T17:14:21.943 INFO:tasks.workunit.client.0.smithi027.stdout:1/505: unlink d6/d11/f95 0 2021-11-03T17:14:21.944 INFO:tasks.workunit.client.0.smithi027.stdout:1/506: write d6/d11/dac/d79/d37/d49/f50 [250395,104583] 0 2021-11-03T17:14:21.944 INFO:tasks.workunit.client.0.smithi027.stdout:1/507: chown d6/d11/dac/d79/d25/f2d 576387 1 2021-11-03T17:14:21.952 INFO:tasks.workunit.client.0.smithi027.stdout:1/508: dread d6/de/f92 [0,4194304] 0 2021-11-03T17:14:21.962 INFO:tasks.workunit.client.1.smithi115.stdout:1/479: dwrite d7/dc/d1a/f35 [0,4194304] 0 2021-11-03T17:14:21.967 INFO:tasks.workunit.client.0.smithi027.stdout:1/509: dwrite d6/d11/d14/f76 [0,4194304] 0 2021-11-03T17:14:21.967 INFO:tasks.workunit.client.0.smithi027.stdout:1/510: truncate d6/de/d13/f66 275485 0 2021-11-03T17:14:21.967 INFO:tasks.workunit.client.0.smithi027.stdout:1/511: fdatasync d6/d11/dac/d79/d25/d28/f3c 0 2021-11-03T17:14:21.968 INFO:tasks.workunit.client.0.smithi027.stdout:1/512: mknod d6/de/d13/d15/d43/cc4 1 2021-11-03T17:14:21.968 INFO:tasks.workunit.client.0.smithi027.stdout:1/513: dread d6/d11/dac/d79/d25/d28/d32/fae [0,4194304] 0 2021-11-03T17:14:21.969 INFO:tasks.workunit.client.0.smithi027.stdout:1/514: dread - d6/d11/dac/d79/d25/d28/d5c/fc0 zero size 2021-11-03T17:14:21.972 INFO:tasks.workunit.client.0.smithi027.stdout:1/515: rmdir d6/d11/dac/d9b 0 2021-11-03T17:14:21.974 INFO:tasks.workunit.client.0.smithi027.stdout:1/516: creat d6/d11/dac/d79/dab/db4/fc5 x:0 0 0 2021-11-03T17:14:21.976 INFO:tasks.workunit.client.0.smithi027.stdout:1/517: unlink d6/de/d13/d15/d43/lba 0 2021-11-03T17:14:21.977 INFO:tasks.workunit.client.0.smithi027.stdout:1/518: creat d6/d11/dac/d79/d25/d28/d5c/fc6 x:0 0 0 2021-11-03T17:14:21.980 INFO:tasks.workunit.client.1.smithi115.stdout:1/480: dwrite d7/dc/dd/d24/d4b/d55/f93 [0,4194304] 0 2021-11-03T17:14:21.984 INFO:tasks.workunit.client.1.smithi115.stdout:1/481: dread d7/dc/dd/d24/f2b [0,4194304] 0 2021-11-03T17:14:21.984 INFO:tasks.workunit.client.1.smithi115.stdout:1/482: write d7/dc/dd/d24/d4d/f69 [741426,84119] 0 2021-11-03T17:14:21.985 INFO:tasks.workunit.client.1.smithi115.stdout:1/483: getdents d7/dc 0 2021-11-03T17:14:21.988 INFO:tasks.workunit.client.1.smithi115.stdout:1/484: rename d7/dc/dd/f46 to d7/dc/dd/d3e/fa1 0 2021-11-03T17:14:21.995 INFO:tasks.workunit.client.0.smithi027.stdout:1/519: dwrite d6/d11/dac/d79/d31/f34 [0,4194304] 0 2021-11-03T17:14:21.995 INFO:tasks.workunit.client.0.smithi027.stdout:1/520: chown d6/fa 405916 1 2021-11-03T17:14:21.999 INFO:tasks.workunit.client.0.smithi027.stdout:1/521: creat d6/d11/dac/d79/d31/fc7 x:0 0 0 2021-11-03T17:14:22.000 INFO:tasks.workunit.client.0.smithi027.stdout:1/522: truncate d6/d11/dac/d79/d25/d28/f3c 173254 0 2021-11-03T17:14:22.000 INFO:tasks.workunit.client.0.smithi027.stdout:1/523: fsync d6/d11/dac/d79/d31/d58/f70 0 2021-11-03T17:14:22.001 INFO:tasks.workunit.client.0.smithi027.stdout:1/524: read d6/d11/dac/d79/d37/d49/f50 [372710,109306] 0 2021-11-03T17:14:22.004 INFO:tasks.workunit.client.0.smithi027.stdout:1/525: read d6/d11/dac/d79/f63 [3571842,57264] 0 2021-11-03T17:14:22.005 INFO:tasks.workunit.client.0.smithi027.stdout:1/526: truncate d6/f7c 181946 0 2021-11-03T17:14:22.007 INFO:tasks.workunit.client.0.smithi027.stdout:1/527: truncate d6/de/d13/d15/f4b 3837416 0 2021-11-03T17:14:22.007 INFO:tasks.workunit.client.0.smithi027.stdout:1/528: chown d6/de/d13/d15/d4e/l56 376430 1 2021-11-03T17:14:22.007 INFO:tasks.workunit.client.0.smithi027.stdout:1/529: chown d6/de/d13/d15/d4e/lb1 93 1 2021-11-03T17:14:22.007 INFO:tasks.workunit.client.0.smithi027.stdout:1/530: mknod d6/d11/dac/d79/d31/cc8 1 2021-11-03T17:14:22.008 INFO:tasks.workunit.client.0.smithi027.stdout:1/531: write d6/d11/dac/d79/fbb [1043976,13382] 0 2021-11-03T17:14:22.009 INFO:tasks.workunit.client.0.smithi027.stdout:1/532: write d6/fa [4080398,63631] 0 2021-11-03T17:14:22.009 INFO:tasks.workunit.client.1.smithi115.stdout:1/485: dwrite d7/dc/ff [0,4194304] 0 2021-11-03T17:14:22.009 INFO:tasks.workunit.client.1.smithi115.stdout:1/486: chown d7/f53 0 1 2021-11-03T17:14:22.019 INFO:tasks.workunit.client.1.smithi115.stdout:1/487: write d7/dc/f16 [1230002,61023] 0 2021-11-03T17:14:22.020 INFO:tasks.workunit.client.1.smithi115.stdout:1/488: link d7/d38/d84/f9a d7/dc/fa2 0 2021-11-03T17:14:22.020 INFO:tasks.workunit.client.1.smithi115.stdout:1/489: chown d7/f8a 31 1 2021-11-03T17:14:22.021 INFO:tasks.workunit.client.1.smithi115.stdout:1/490: fsync d7/dc/dd/d24/d26/f85 0 2021-11-03T17:14:22.021 INFO:tasks.workunit.client.1.smithi115.stdout:1/491: write d7/f8a [58391,32322] 0 2021-11-03T17:14:22.021 INFO:tasks.workunit.client.1.smithi115.stdout:1/492: write d7/dc/ff [4297357,118131] 0 2021-11-03T17:14:22.022 INFO:tasks.workunit.client.1.smithi115.stdout:1/493: link d7/dc/dd/d42/l4e d7/dc/dd/d24/d43/la3 0 2021-11-03T17:14:22.022 INFO:tasks.workunit.client.1.smithi115.stdout:1/494: mknod d7/dc/dd/d24/d43/d78/ca4 1 2021-11-03T17:14:22.024 INFO:tasks.workunit.client.0.smithi027.stdout:1/533: dwrite d6/d11/dac/d79/d31/f8a [4194304,4194304] 0 2021-11-03T17:14:22.029 INFO:tasks.workunit.client.0.smithi027.stdout:1/534: creat d6/d11/dac/d79/d25/d28/d5c/fc9 x:0 0 0 2021-11-03T17:14:22.029 INFO:tasks.workunit.client.0.smithi027.stdout:1/535: chown d6/d11/f89 1888 1 2021-11-03T17:14:22.029 INFO:tasks.workunit.client.0.smithi027.stdout:1/536: fsync d6/d11/dac/d79/d25/d28/d32/f8e 0 2021-11-03T17:14:22.038 INFO:tasks.workunit.client.0.smithi027.stdout:1/537: dread d6/d11/dac/d79/d25/f47 [0,4194304] 0 2021-11-03T17:14:22.041 INFO:tasks.workunit.client.0.smithi027.stdout:1/538: link d6/d11/dac/d79/d31/fa0 d6/d11/dac/d79/d25/d28/d5c/d81/d7d/fca 0 2021-11-03T17:14:22.043 INFO:tasks.workunit.client.0.smithi027.stdout:1/539: creat d6/d11/dac/d79/d37/d49/fcb x:0 0 0 2021-11-03T17:14:22.044 INFO:tasks.workunit.client.1.smithi115.stdout:1/495: dwrite d7/dc/dd/d24/d26/f4f [0,4194304] 0 2021-11-03T17:14:22.044 INFO:tasks.workunit.client.1.smithi115.stdout:1/496: truncate d7/dc/dd/d24/d4d/f69 1319637 0 2021-11-03T17:14:22.045 INFO:tasks.workunit.client.1.smithi115.stdout:1/497: mknod d7/dc/d1a/da0/ca5 1 2021-11-03T17:14:22.045 INFO:tasks.workunit.client.0.smithi027.stdout:1/540: rename d6/de/f2c to d6/d11/dac/d79/d25/d5e/fcc 0 2021-11-03T17:14:22.049 INFO:tasks.workunit.client.1.smithi115.stdout:5/584: sync 2021-11-03T17:14:22.050 INFO:tasks.workunit.client.1.smithi115.stdout:5/585: symlink d1a/d28/d34/lba 0 2021-11-03T17:14:22.051 INFO:tasks.workunit.client.1.smithi115.stdout:5/586: creat d1a/d3d/d58/dab/fbb x:0 0 0 2021-11-03T17:14:22.052 INFO:tasks.workunit.client.1.smithi115.stdout:5/587: fdatasync d1a/d3d/f62 0 2021-11-03T17:14:22.053 INFO:tasks.workunit.client.1.smithi115.stdout:5/588: symlink d1a/d28/d4c/d75/d68/da5/lbc 0 2021-11-03T17:14:22.061 INFO:tasks.workunit.client.1.smithi115.stdout:5/589: dread d1a/d28/f31 [0,4194304] 0 2021-11-03T17:14:22.061 INFO:tasks.workunit.client.1.smithi115.stdout:5/590: chown d1a/d28/d78/fad 169616 1 2021-11-03T17:14:22.063 INFO:tasks.workunit.client.0.smithi027.stdout:1/541: dwrite d6/d11/dac/d79/d25/d28/d5c/f78 [0,4194304] 0 2021-11-03T17:14:22.064 INFO:tasks.workunit.client.1.smithi115.stdout:5/591: unlink d1a/d28/d78/fb8 0 2021-11-03T17:14:22.066 INFO:tasks.workunit.client.1.smithi115.stdout:5/592: rename d1a/l2f to d1a/d28/d7b/lbd 0 2021-11-03T17:14:22.067 INFO:tasks.workunit.client.1.smithi115.stdout:5/593: stat d1a/d28 0 2021-11-03T17:14:22.067 INFO:tasks.workunit.client.0.smithi027.stdout:1/542: unlink d6/de/lb5 0 2021-11-03T17:14:22.068 INFO:tasks.workunit.client.0.smithi027.stdout:1/543: getdents d6/de/d13/d15/d4e/dbd 0 2021-11-03T17:14:22.070 INFO:tasks.workunit.client.0.smithi027.stdout:1/544: link d6/de/d13/d15/d43/f48 d6/d11/dac/d79/d31/d58/fcd 0 2021-11-03T17:14:22.080 INFO:tasks.workunit.client.1.smithi115.stdout:5/594: dread d1a/d28/f91 [0,4194304] 0 2021-11-03T17:14:22.082 INFO:tasks.workunit.client.1.smithi115.stdout:1/498: dwrite d7/f8a [0,4194304] 0 2021-11-03T17:14:22.083 INFO:tasks.workunit.client.1.smithi115.stdout:1/499: stat d7/dc/dd/d24/d4b/d55/f97 0 2021-11-03T17:14:22.083 INFO:tasks.workunit.client.1.smithi115.stdout:1/500: truncate d7/dc/dd/d24/d4d/f79 996457 0 2021-11-03T17:14:22.083 INFO:tasks.workunit.client.1.smithi115.stdout:1/501: fsync d7/dc/dd/d24/d43/f6f 0 2021-11-03T17:14:22.084 INFO:tasks.workunit.client.0.smithi027.stdout:1/545: dread d6/d11/dac/d79/d25/d28/d5c/d68/f6c [0,4194304] 0 2021-11-03T17:14:22.085 INFO:tasks.workunit.client.1.smithi115.stdout:1/502: rmdir d7/d38 39 2021-11-03T17:14:22.086 INFO:tasks.workunit.client.1.smithi115.stdout:1/503: write d7/dc/dd/d24/d4d/d60/f81 [442958,19453] 0 2021-11-03T17:14:22.086 INFO:tasks.workunit.client.1.smithi115.stdout:1/504: write d7/dc/dd/d24/d43/f74 [763747,1423] 0 2021-11-03T17:14:22.086 INFO:tasks.workunit.client.1.smithi115.stdout:1/505: mknod d7/d38/d59/d83/ca6 1 2021-11-03T17:14:22.087 INFO:tasks.workunit.client.1.smithi115.stdout:1/506: creat d7/dc/dd/d24/d4d/d60/d8f/fa7 x:0 0 0 2021-11-03T17:14:22.089 INFO:tasks.workunit.client.1.smithi115.stdout:1/507: creat d7/dc/d1a/da0/d73/d7f/fa8 x:0 0 0 2021-11-03T17:14:22.090 INFO:tasks.workunit.client.1.smithi115.stdout:5/595: dread d1a/d3d/f3b [0,4194304] 0 2021-11-03T17:14:22.093 INFO:tasks.workunit.client.1.smithi115.stdout:1/508: unlink d7/d38/d59/f8c 0 2021-11-03T17:14:22.093 INFO:tasks.workunit.client.1.smithi115.stdout:1/509: fdatasync d7/dc/d1a/da0/f33 0 2021-11-03T17:14:22.094 INFO:tasks.workunit.client.1.smithi115.stdout:1/510: creat d7/dc/d1a/da0/d73/fa9 x:0 0 0 2021-11-03T17:14:22.097 INFO:tasks.workunit.client.1.smithi115.stdout:1/511: dread d7/dc/dd/d24/d4d/f79 [0,4194304] 0 2021-11-03T17:14:22.098 INFO:tasks.workunit.client.1.smithi115.stdout:1/512: truncate d7/dc/dd/d24/d4d/f7b 171510 0 2021-11-03T17:14:22.098 INFO:tasks.workunit.client.1.smithi115.stdout:1/513: stat d7/dc/dd/d3e/d58 0 2021-11-03T17:14:22.099 INFO:tasks.workunit.client.1.smithi115.stdout:1/514: truncate d7/dc/d1a/f6b 232753 0 2021-11-03T17:14:22.101 INFO:tasks.workunit.client.1.smithi115.stdout:1/515: unlink d7/dc/dd/d24/d4d/d60/f62 0 2021-11-03T17:14:22.108 INFO:tasks.workunit.client.0.smithi027.stdout:1/546: dwrite d6/d11/dac/d79/d31/f7f [4194304,4194304] 0 2021-11-03T17:14:22.109 INFO:tasks.workunit.client.0.smithi027.stdout:1/547: stat d6/d11/dac/d79/d31/fb7 0 2021-11-03T17:14:22.109 INFO:tasks.workunit.client.0.smithi027.stdout:1/548: dread - d6/de/faf zero size 2021-11-03T17:14:22.113 INFO:tasks.workunit.client.1.smithi115.stdout:1/516: dread d7/dc/dd/d24/d26/f4f [0,4194304] 0 2021-11-03T17:14:22.116 INFO:tasks.workunit.client.1.smithi115.stdout:7/379: sync 2021-11-03T17:14:22.116 INFO:tasks.workunit.client.1.smithi115.stdout:1/517: dread d7/f53 [0,4194304] 0 2021-11-03T17:14:22.117 INFO:tasks.workunit.client.1.smithi115.stdout:4/612: sync 2021-11-03T17:14:22.117 INFO:tasks.workunit.client.1.smithi115.stdout:0/570: sync 2021-11-03T17:14:22.117 INFO:tasks.workunit.client.1.smithi115.stdout:0/571: dread - d9/d15/d78/d80/d82/fb5 zero size 2021-11-03T17:14:22.118 INFO:tasks.workunit.client.1.smithi115.stdout:1/518: symlink d7/dc/dd/d24/d4d/d60/d8f/laa 0 2021-11-03T17:14:22.119 INFO:tasks.workunit.client.1.smithi115.stdout:0/572: write d9/d15/d2c/d46/d8c/f49 [3176973,85948] 0 2021-11-03T17:14:22.122 INFO:tasks.workunit.client.1.smithi115.stdout:4/613: rename d7/dc/d1a/d21/db1/d56/d63/d7b/fc1 to d7/dc/d6a/fd3 0 2021-11-03T17:14:22.123 INFO:tasks.workunit.client.1.smithi115.stdout:5/596: dwrite d1a/d28/d4c/d75/d68/da5/da0/fa7 [0,4194304] 0 2021-11-03T17:14:22.123 INFO:tasks.workunit.client.1.smithi115.stdout:5/597: write d1a/d3d/f62 [8812355,62059] 0 2021-11-03T17:14:22.124 INFO:tasks.workunit.client.1.smithi115.stdout:5/598: stat d1a/d28/d7b/d48/d7f/f86 0 2021-11-03T17:14:22.124 INFO:tasks.workunit.client.0.smithi027.stdout:1/549: dread d6/d11/dac/d79/f63 [0,4194304] 0 2021-11-03T17:14:22.125 INFO:tasks.workunit.client.1.smithi115.stdout:0/573: symlink d9/d15/d2c/d85/lbf 0 2021-11-03T17:14:22.125 INFO:tasks.workunit.client.1.smithi115.stdout:4/614: creat d7/dc/fd4 x:0 0 0 2021-11-03T17:14:22.126 INFO:tasks.workunit.client.1.smithi115.stdout:7/380: dread d3/f47 [0,4194304] 0 2021-11-03T17:14:22.126 INFO:tasks.workunit.client.0.smithi027.stdout:1/550: rename d6/d11/dac/d79/d25/d28/d5c/la6 to d6/d11/dac/d79/d37/lce 0 2021-11-03T17:14:22.131 INFO:tasks.workunit.client.0.smithi027.stdout:1/551: dread d6/f7c [0,4194304] 0 2021-11-03T17:14:22.135 INFO:tasks.workunit.client.1.smithi115.stdout:0/574: link d9/d15/d2c/d46/d8c/lad d9/db/d89/d81/lc0 0 2021-11-03T17:14:22.140 INFO:tasks.workunit.client.0.smithi027.stdout:1/552: write d6/d11/dac/f82 [2258168,76083] 0 2021-11-03T17:14:22.142 INFO:tasks.workunit.client.1.smithi115.stdout:0/575: symlink d9/d15/d1a/da7/daf/lc1 0 2021-11-03T17:14:22.144 INFO:tasks.workunit.client.0.smithi027.stdout:1/553: read d6/d11/dac/d79/d25/d28/d32/f8e [2269452,26277] 0 2021-11-03T17:14:22.144 INFO:tasks.workunit.client.1.smithi115.stdout:0/576: creat d9/d15/d2c/d46/d8c/fc2 x:0 0 0 2021-11-03T17:14:22.146 INFO:tasks.workunit.client.1.smithi115.stdout:0/577: link d9/d15/d78/d80/lbe d9/lc3 0 2021-11-03T17:14:22.161 INFO:tasks.workunit.client.1.smithi115.stdout:0/578: dread d9/db/f58 [0,4194304] 0 2021-11-03T17:14:22.161 INFO:tasks.workunit.client.1.smithi115.stdout:0/579: stat d9/l8b 0 2021-11-03T17:14:22.162 INFO:tasks.workunit.client.1.smithi115.stdout:0/580: mkdir d9/d15/d78/d80/dc4 0 2021-11-03T17:14:22.162 INFO:tasks.workunit.client.1.smithi115.stdout:5/599: dwrite d1a/d28/d34/f66 [0,4194304] 0 2021-11-03T17:14:22.163 INFO:tasks.workunit.client.1.smithi115.stdout:4/615: dwrite d7/dc/d6a/d3c/fbd [0,4194304] 0 2021-11-03T17:14:22.163 INFO:tasks.workunit.client.1.smithi115.stdout:4/616: truncate d7/d7a/fa3 769631 0 2021-11-03T17:14:22.164 INFO:tasks.workunit.client.1.smithi115.stdout:5/600: mknod d1a/d3d/cbe 1 2021-11-03T17:14:22.166 INFO:tasks.workunit.client.1.smithi115.stdout:4/617: creat d7/dc/d1a/d21/fd5 x:0 0 0 2021-11-03T17:14:22.167 INFO:tasks.workunit.client.1.smithi115.stdout:5/601: write d1a/d28/d34/f66 [84890,125732] 0 2021-11-03T17:14:22.167 INFO:tasks.workunit.client.1.smithi115.stdout:5/602: write d1a/d28/d4a/f8b [903586,38396] 0 2021-11-03T17:14:22.167 INFO:tasks.workunit.client.1.smithi115.stdout:5/603: chown d1a/d3d/f42 0 1 2021-11-03T17:14:22.169 INFO:tasks.workunit.client.1.smithi115.stdout:7/381: dwrite d3/f27 [0,4194304] 0 2021-11-03T17:14:22.169 INFO:tasks.workunit.client.1.smithi115.stdout:5/604: read d1a/d28/d34/f66 [3358316,20155] 0 2021-11-03T17:14:22.171 INFO:tasks.workunit.client.1.smithi115.stdout:1/519: dwrite d7/dc/dd/d24/d4d/d60/f81 [0,4194304] 0 2021-11-03T17:14:22.172 INFO:tasks.workunit.client.1.smithi115.stdout:0/581: rename d9/d15/d1a/da7/daf to d9/d15/d2c/d46/d63/dc5 0 2021-11-03T17:14:22.172 INFO:tasks.workunit.client.1.smithi115.stdout:0/582: readlink d9/d37/l6a 0 2021-11-03T17:14:22.173 INFO:tasks.workunit.client.0.smithi027.stdout:1/554: dwrite d6/d11/dac/d79/d25/d28/f55 [4194304,4194304] 0 2021-11-03T17:14:22.173 INFO:tasks.workunit.client.0.smithi027.stdout:1/555: chown d6/de/d13/d15 1313589598 1 2021-11-03T17:14:22.173 INFO:tasks.workunit.client.0.smithi027.stdout:1/556: chown d6/d11/dac/d79/d37/d49/la1 0 1 2021-11-03T17:14:22.174 INFO:tasks.workunit.client.0.smithi027.stdout:1/557: dread - d6/d11/dac/d79/f9d zero size 2021-11-03T17:14:22.177 INFO:tasks.workunit.client.0.smithi027.stdout:1/558: rename d6/de/d13/d15/d4e/l56 to d6/d11/dac/d79/d25/d28/d41/lcf 0 2021-11-03T17:14:22.178 INFO:tasks.workunit.client.1.smithi115.stdout:5/605: symlink d1a/d28/d7b/d48/d7f/lbf 0 2021-11-03T17:14:22.181 INFO:tasks.workunit.client.1.smithi115.stdout:5/606: mkdir d1a/d3d/dc0 0 2021-11-03T17:14:22.183 INFO:tasks.workunit.client.1.smithi115.stdout:5/607: creat d1a/d28/d4c/d75/d68/da5/da0/fc1 x:0 0 0 2021-11-03T17:14:22.183 INFO:tasks.workunit.client.1.smithi115.stdout:1/520: dread d7/f90 [0,4194304] 0 2021-11-03T17:14:22.184 INFO:tasks.workunit.client.1.smithi115.stdout:5/608: symlink d1a/d3d/d58/dab/lc2 0 2021-11-03T17:14:22.184 INFO:tasks.workunit.client.1.smithi115.stdout:5/609: chown d1a/d28/d4c/d75/d83/f88 619737 1 2021-11-03T17:14:22.186 INFO:tasks.workunit.client.1.smithi115.stdout:1/521: rename d7/dc/d1a/da0 to d7/dc/dab 0 2021-11-03T17:14:22.186 INFO:tasks.workunit.client.1.smithi115.stdout:1/522: mknod d7/dc/dd/d24/d4d/d60/cac 1 2021-11-03T17:14:22.187 INFO:tasks.workunit.client.0.smithi027.stdout:1/559: dread d6/de/d13/f38 [0,4194304] 0 2021-11-03T17:14:22.188 INFO:tasks.workunit.client.1.smithi115.stdout:5/610: unlink d1a/d28/d7b/d4f/d81/fb7 0 2021-11-03T17:14:22.188 INFO:tasks.workunit.client.1.smithi115.stdout:5/611: fdatasync d1a/d28/d4c/d75/d68/da5/fa9 0 2021-11-03T17:14:22.189 INFO:tasks.workunit.client.1.smithi115.stdout:1/523: symlink d7/d38/d84/lad 0 2021-11-03T17:14:22.189 INFO:tasks.workunit.client.1.smithi115.stdout:1/524: chown d7/dc/dd/d3e/d58/f88 1 1 2021-11-03T17:14:22.189 INFO:tasks.workunit.client.1.smithi115.stdout:1/525: chown d7/dc 1190 1 2021-11-03T17:14:22.190 INFO:tasks.workunit.client.1.smithi115.stdout:4/618: dwrite d7/dc/d1a/d21/db1/d56/d63/d7b/fb2 [0,4194304] 0 2021-11-03T17:14:22.190 INFO:tasks.workunit.client.1.smithi115.stdout:5/612: creat d1a/d28/d4c/d75/d68/da5/fc3 x:0 0 0 2021-11-03T17:14:22.193 INFO:tasks.workunit.client.1.smithi115.stdout:1/526: read d7/dc/dd/d42/f7a [961399,22284] 0 2021-11-03T17:14:22.195 INFO:tasks.workunit.client.1.smithi115.stdout:5/613: mkdir d1a/d3d/dc4 0 2021-11-03T17:14:22.196 INFO:tasks.workunit.client.1.smithi115.stdout:1/527: truncate d7/f77 1093242 0 2021-11-03T17:14:22.197 INFO:tasks.workunit.client.1.smithi115.stdout:5/614: symlink d1a/d28/d4c/d75/d83/da1/lc5 0 2021-11-03T17:14:22.198 INFO:tasks.workunit.client.1.smithi115.stdout:1/528: creat d7/dc/dd/d24/d4d/d60/d8f/fae x:0 0 0 2021-11-03T17:14:22.201 INFO:tasks.workunit.client.1.smithi115.stdout:1/529: mkdir d7/dc/daf 0 2021-11-03T17:14:22.204 INFO:tasks.workunit.client.1.smithi115.stdout:7/382: dwrite d3/f14 [4194304,4194304] 0 2021-11-03T17:14:22.208 INFO:tasks.workunit.client.1.smithi115.stdout:4/619: dread d7/dc/d6a/d3c/fbd [0,4194304] 0 2021-11-03T17:14:22.208 INFO:tasks.workunit.client.1.smithi115.stdout:0/583: dwrite d9/d15/d2c/d46/d8c/f7c [4194304,4194304] 0 2021-11-03T17:14:22.212 INFO:tasks.workunit.client.1.smithi115.stdout:7/383: unlink d3/da/d18/f67 0 2021-11-03T17:14:22.212 INFO:tasks.workunit.client.1.smithi115.stdout:0/584: symlink d9/d15/d78/d80/dc4/lc6 0 2021-11-03T17:14:22.212 INFO:tasks.workunit.client.1.smithi115.stdout:0/585: dread - d9/db/f7e zero size 2021-11-03T17:14:22.214 INFO:tasks.workunit.client.1.smithi115.stdout:4/620: dread d7/dc/d1a/d21/db1/f74 [0,4194304] 0 2021-11-03T17:14:22.216 INFO:tasks.workunit.client.1.smithi115.stdout:4/621: rmdir d7/dc/d6a/d3c/dbf 39 2021-11-03T17:14:22.217 INFO:tasks.workunit.client.1.smithi115.stdout:0/586: read d9/d15/d2c/d46/d8c/f49 [1824018,651] 0 2021-11-03T17:14:22.218 INFO:tasks.workunit.client.1.smithi115.stdout:0/587: symlink d9/d15/d1a/d1d/lc7 0 2021-11-03T17:14:22.222 INFO:tasks.workunit.client.1.smithi115.stdout:0/588: truncate d9/d15/d78/d80/f9a 3052291 0 2021-11-03T17:14:22.224 INFO:tasks.workunit.client.1.smithi115.stdout:0/589: mkdir d9/d15/d2c/d46/d63/dc5/dc8 0 2021-11-03T17:14:22.224 INFO:tasks.workunit.client.0.smithi027.stdout:1/560: dwrite d6/de/d13/d15/d43/f48 [4194304,4194304] 0 2021-11-03T17:14:22.225 INFO:tasks.workunit.client.1.smithi115.stdout:0/590: creat d9/d15/d2c/d6e/fc9 x:0 0 0 2021-11-03T17:14:22.225 INFO:tasks.workunit.client.1.smithi115.stdout:0/591: write d9/d15/d2c/d46/d8c/f48 [4760341,99403] 0 2021-11-03T17:14:22.228 INFO:tasks.workunit.client.1.smithi115.stdout:1/530: dwrite d7/dc/dd/d24/d4b/d55/f6c [0,4194304] 0 2021-11-03T17:14:22.228 INFO:tasks.workunit.client.1.smithi115.stdout:1/531: write d7/dc/d1a/f3d [5248291,111631] 0 2021-11-03T17:14:22.228 INFO:tasks.workunit.client.1.smithi115.stdout:7/384: dwrite d3/da/d18/d73/d52/f56 [0,4194304] 0 2021-11-03T17:14:22.231 INFO:tasks.workunit.client.1.smithi115.stdout:1/532: creat d7/dc/dab/d73/fb0 x:0 0 0 2021-11-03T17:14:22.231 INFO:tasks.workunit.client.1.smithi115.stdout:1/533: chown d7/dc/dd/d24/d4d/l89 346095397 1 2021-11-03T17:14:22.232 INFO:tasks.workunit.client.0.smithi027.stdout:1/561: dread d6/d11/dac/d79/d37/d49/f50 [0,4194304] 0 2021-11-03T17:14:22.232 INFO:tasks.workunit.client.0.smithi027.stdout:1/562: dread - d6/d11/dac/d79/d31/d58/f70 zero size 2021-11-03T17:14:22.233 INFO:tasks.workunit.client.1.smithi115.stdout:7/385: mkdir d3/da/d78 0 2021-11-03T17:14:22.234 INFO:tasks.workunit.client.1.smithi115.stdout:7/386: symlink d3/d4c/l79 0 2021-11-03T17:14:22.235 INFO:tasks.workunit.client.0.smithi027.stdout:1/563: truncate d6/d11/f27 2037745 0 2021-11-03T17:14:22.237 INFO:tasks.workunit.client.1.smithi115.stdout:7/387: symlink d3/da/d18/d73/d52/l7a 0 2021-11-03T17:14:22.238 INFO:tasks.workunit.client.1.smithi115.stdout:7/388: chown d3/da/f64 547663 1 2021-11-03T17:14:22.261 INFO:tasks.workunit.client.1.smithi115.stdout:0/592: rmdir d9/d15/d78/d80 39 2021-11-03T17:14:22.262 INFO:tasks.workunit.client.1.smithi115.stdout:0/593: stat d9/d15/d1a/d1d 0 2021-11-03T17:14:22.262 INFO:tasks.workunit.client.1.smithi115.stdout:0/594: fdatasync d9/db/d89/f84 0 2021-11-03T17:14:22.262 INFO:tasks.workunit.client.1.smithi115.stdout:0/595: fsync d9/d15/f67 0 2021-11-03T17:14:22.273 INFO:tasks.workunit.client.1.smithi115.stdout:0/596: dread d9/d15/d1a/d1d/f24 [0,4194304] 0 2021-11-03T17:14:22.274 INFO:tasks.workunit.client.1.smithi115.stdout:0/597: rename d9/l8b to d9/d15/d1a/d1d/lca 0 2021-11-03T17:14:22.275 INFO:tasks.workunit.client.1.smithi115.stdout:0/598: mknod d9/d15/d78/ccb 1 2021-11-03T17:14:22.280 INFO:tasks.workunit.client.0.smithi027.stdout:1/564: dwrite d6/d11/d14/f76 [0,4194304] 0 2021-11-03T17:14:22.282 INFO:tasks.workunit.client.1.smithi115.stdout:1/534: dwrite d7/dc/dd/d24/d4b/d55/f93 [0,4194304] 0 2021-11-03T17:14:22.282 INFO:tasks.workunit.client.1.smithi115.stdout:4/622: dwrite d7/dc/d1a/d21/fa1 [0,4194304] 0 2021-11-03T17:14:22.283 INFO:tasks.workunit.client.1.smithi115.stdout:7/389: dwrite d3/da/f62 [0,4194304] 0 2021-11-03T17:14:22.283 INFO:tasks.workunit.client.1.smithi115.stdout:1/535: rename d7/dc/dd/d24/d4d/d60/f91 to d7/dc/dd/d24/d43/d78/fb1 0 2021-11-03T17:14:22.284 INFO:tasks.workunit.client.1.smithi115.stdout:1/536: readlink d7/dc/dd/d24/d4b/l71 0 2021-11-03T17:14:22.284 INFO:tasks.workunit.client.1.smithi115.stdout:1/537: chown d7/dc/dd/d24/d4b/d55/f93 59642 1 2021-11-03T17:14:22.285 INFO:tasks.workunit.client.1.smithi115.stdout:4/623: creat d7/d7a/fd6 x:0 0 0 2021-11-03T17:14:22.288 INFO:tasks.workunit.client.1.smithi115.stdout:1/538: mkdir d7/dc/dd/d24/d4d/d60/db2 0 2021-11-03T17:14:22.295 INFO:tasks.workunit.client.0.smithi027.stdout:1/565: dread d6/d11/d14/f88 [0,4194304] 0 2021-11-03T17:14:22.298 INFO:tasks.workunit.client.0.smithi027.stdout:1/566: unlink d6/f7c 0 2021-11-03T17:14:22.299 INFO:tasks.workunit.client.0.smithi027.stdout:1/567: truncate d6/d11/dac/d79/d31/f34 23397 0 2021-11-03T17:14:22.300 INFO:tasks.workunit.client.1.smithi115.stdout:0/599: dwrite d9/d15/f67 [0,4194304] 0 2021-11-03T17:14:22.304 INFO:tasks.workunit.client.1.smithi115.stdout:0/600: rename d9/d15/d2c to d9/d15/d1a/d1d/dcc 0 2021-11-03T17:14:22.312 INFO:tasks.workunit.client.1.smithi115.stdout:7/390: dwrite d3/da/f62 [0,4194304] 0 2021-11-03T17:14:22.314 INFO:tasks.workunit.client.1.smithi115.stdout:7/391: mkdir d3/d4c/d7b 0 2021-11-03T17:14:22.316 INFO:tasks.workunit.client.1.smithi115.stdout:1/539: write d7/dc/dd/d24/d4b/d55/f93 [3993668,83313] 0 2021-11-03T17:14:22.316 INFO:tasks.workunit.client.1.smithi115.stdout:4/624: dread d7/fa0 [0,4194304] 0 2021-11-03T17:14:22.317 INFO:tasks.workunit.client.1.smithi115.stdout:7/392: creat d3/da/d18/d73/d35/f7c x:0 0 0 2021-11-03T17:14:22.318 INFO:tasks.workunit.client.1.smithi115.stdout:7/393: symlink d3/da/l7d 0 2021-11-03T17:14:22.318 INFO:tasks.workunit.client.1.smithi115.stdout:4/625: dread d7/dc/d1a/d21/db1/d56/d42/f58 [0,4194304] 0 2021-11-03T17:14:22.319 INFO:tasks.workunit.client.1.smithi115.stdout:4/626: rmdir d7/dc/d6a 39 2021-11-03T17:14:22.319 INFO:tasks.workunit.client.1.smithi115.stdout:4/627: chown d7/dc/d1a/d21/db1/d56/d42/f58 928 1 2021-11-03T17:14:22.320 INFO:tasks.workunit.client.0.smithi027.stdout:1/568: dread d6/d11/dac/d79/d31/f7f [4194304,4194304] 0 2021-11-03T17:14:22.320 INFO:tasks.workunit.client.1.smithi115.stdout:4/628: write d7/dc/d1a/d21/f22 [2782427,86388] 0 2021-11-03T17:14:22.321 INFO:tasks.workunit.client.0.smithi027.stdout:1/569: creat d6/da7/fd0 x:0 0 0 2021-11-03T17:14:22.321 INFO:tasks.workunit.client.0.smithi027.stdout:1/570: chown d6/d11/dac/d79/d37/d49/lb9 30052531 1 2021-11-03T17:14:22.322 INFO:tasks.workunit.client.0.smithi027.stdout:1/571: write d6/d11/dac/d79/d25/f2d [1352471,51551] 0 2021-11-03T17:14:22.325 INFO:tasks.workunit.client.0.smithi027.stdout:1/572: write d6/d11/dac/f82 [1180103,67990] 0 2021-11-03T17:14:22.326 INFO:tasks.workunit.client.1.smithi115.stdout:7/394: dread d3/da/d18/d73/f2f [0,4194304] 0 2021-11-03T17:14:22.328 INFO:tasks.workunit.client.0.smithi027.stdout:1/573: rename d6/d11/dac/d79/db0/dbf to d6/d11/dac/d79/d25/d5e/dd1 0 2021-11-03T17:14:22.328 INFO:tasks.workunit.client.1.smithi115.stdout:4/629: dread d7/dc/d6a/d72/f76 [0,4194304] 0 2021-11-03T17:14:22.329 INFO:tasks.workunit.client.0.smithi027.stdout:1/574: creat d6/da7/fd2 x:0 0 0 2021-11-03T17:14:22.329 INFO:tasks.workunit.client.0.smithi027.stdout:1/575: fsync d6/d11/dac/d79/f9d 0 2021-11-03T17:14:22.330 INFO:tasks.workunit.client.1.smithi115.stdout:1/540: dread d7/dc/dd/d24/d43/f3c [4194304,4194304] 0 2021-11-03T17:14:22.331 INFO:tasks.workunit.client.1.smithi115.stdout:4/630: unlink d7/d7a/fd6 0 2021-11-03T17:14:22.331 INFO:tasks.workunit.client.1.smithi115.stdout:0/601: dwrite d9/db/f75 [0,4194304] 0 2021-11-03T17:14:22.331 INFO:tasks.workunit.client.1.smithi115.stdout:4/631: write d7/dc/d1a/d21/db1/d56/d63/fa6 [560302,69103] 0 2021-11-03T17:14:22.332 INFO:tasks.workunit.client.1.smithi115.stdout:1/541: write d7/dc/dd/d24/d43/f3c [2324359,99001] 0 2021-11-03T17:14:22.332 INFO:tasks.workunit.client.0.smithi027.stdout:1/576: rename d6/d11/dac/d79/d37/f80 to d6/d11/dac/fd3 0 2021-11-03T17:14:22.333 INFO:tasks.workunit.client.0.smithi027.stdout:1/577: truncate d6/d11/dac/d79/d37/d49/f50 1094944 0 2021-11-03T17:14:22.333 INFO:tasks.workunit.client.0.smithi027.stdout:1/578: getdents d6/d11/dac/d79/d37/d49 0 2021-11-03T17:14:22.334 INFO:tasks.workunit.client.1.smithi115.stdout:0/602: mkdir d9/db/d83/dcd 0 2021-11-03T17:14:22.334 INFO:tasks.workunit.client.1.smithi115.stdout:4/632: creat d7/dc/d1a/d21/db1/d56/d63/fd7 x:0 0 0 2021-11-03T17:14:22.336 INFO:tasks.workunit.client.0.smithi027.stdout:1/579: link d6/de/d13/f66 d6/d11/dac/d79/d25/d28/fd4 0 2021-11-03T17:14:22.338 INFO:tasks.workunit.client.0.smithi027.stdout:1/580: readlink d6/lb 0 2021-11-03T17:14:22.339 INFO:tasks.workunit.client.1.smithi115.stdout:1/542: unlink d7/dc/dd/d42/l4e 0 2021-11-03T17:14:22.342 INFO:tasks.workunit.client.1.smithi115.stdout:0/603: dread d9/db/d89/f9c [0,4194304] 0 2021-11-03T17:14:22.342 INFO:tasks.workunit.client.1.smithi115.stdout:1/543: rename d7/dc/dd/d24/d4b/d55/f86 to d7/fb3 0 2021-11-03T17:14:22.342 INFO:tasks.workunit.client.1.smithi115.stdout:0/604: read d9/db/d89/f9c [2964097,104377] 0 2021-11-03T17:14:22.351 INFO:tasks.workunit.client.1.smithi115.stdout:0/605: dread d9/db/d89/f84 [0,4194304] 0 2021-11-03T17:14:22.353 INFO:tasks.workunit.client.1.smithi115.stdout:0/606: mkdir d9/d15/dce 0 2021-11-03T17:14:22.358 INFO:tasks.workunit.client.1.smithi115.stdout:0/607: rename d9/d15/d1a/d1d/dcc/d46/d63/fa6 to d9/d15/d1a/d1d/dcc/fcf 0 2021-11-03T17:14:22.358 INFO:tasks.workunit.client.1.smithi115.stdout:0/608: write d9/d15/d1a/f3d [4403363,25615] 0 2021-11-03T17:14:22.361 INFO:tasks.workunit.client.1.smithi115.stdout:0/609: rename d9/d15/d1a/d1d/f4b to d9/d15/d78/fd0 0 2021-11-03T17:14:22.362 INFO:tasks.workunit.client.1.smithi115.stdout:0/610: creat d9/db/d83/fd1 x:0 0 0 2021-11-03T17:14:22.363 INFO:tasks.workunit.client.1.smithi115.stdout:0/611: write d9/d15/d1a/f92 [3254297,57475] 0 2021-11-03T17:14:22.364 INFO:tasks.workunit.client.1.smithi115.stdout:0/612: chown d9/d15/d1a/fa4 2537324 1 2021-11-03T17:14:22.377 INFO:tasks.workunit.client.0.smithi027.stdout:1/581: dwrite d6/de/d13/d15/f2f [4194304,4194304] 0 2021-11-03T17:14:22.378 INFO:tasks.workunit.client.1.smithi115.stdout:4/633: dwrite d7/dc/fc8 [0,4194304] 0 2021-11-03T17:14:22.378 INFO:tasks.workunit.client.1.smithi115.stdout:4/634: write d7/dc/fc8 [3092622,80283] 0 2021-11-03T17:14:22.379 INFO:tasks.workunit.client.1.smithi115.stdout:1/544: dwrite d7/dc/dd/d24/d26/f4f [0,4194304] 0 2021-11-03T17:14:22.379 INFO:tasks.workunit.client.1.smithi115.stdout:4/635: write d7/dc/d1a/d21/db1/d56/d63/f67 [2121878,94659] 0 2021-11-03T17:14:22.384 INFO:tasks.workunit.client.1.smithi115.stdout:4/636: mkdir d7/dc/dd8 0 2021-11-03T17:14:22.384 INFO:tasks.workunit.client.1.smithi115.stdout:4/637: chown d7/dc/d6a/d3c/f3e 18220 1 2021-11-03T17:14:22.385 INFO:tasks.workunit.client.1.smithi115.stdout:1/545: dread d7/dc/dd/f5a [0,4194304] 0 2021-11-03T17:14:22.385 INFO:tasks.workunit.client.0.smithi027.stdout:1/582: rmdir d6 39 2021-11-03T17:14:22.386 INFO:tasks.workunit.client.0.smithi027.stdout:1/583: truncate d6/d11/dac/d79/d25/d28/d5c/fc0 616406 0 2021-11-03T17:14:22.386 INFO:tasks.workunit.client.0.smithi027.stdout:1/584: getdents d6/d11/dac/d79/d37 0 2021-11-03T17:14:22.386 INFO:tasks.workunit.client.1.smithi115.stdout:4/638: rmdir d7/dc/dd8 0 2021-11-03T17:14:22.392 INFO:tasks.workunit.client.1.smithi115.stdout:1/546: dread d7/dc/d1a/f6b [0,4194304] 0 2021-11-03T17:14:22.393 INFO:tasks.workunit.client.1.smithi115.stdout:4/639: mkdir d7/dc/d6a/d3c/dc5/dcc/dd9 0 2021-11-03T17:14:22.393 INFO:tasks.workunit.client.1.smithi115.stdout:4/640: readlink d7/dc/l97 0 2021-11-03T17:14:22.399 INFO:tasks.workunit.client.0.smithi027.stdout:1/585: dwrite d6/d11/dac/d79/d25/d28/d5c/f78 [4194304,4194304] 0 2021-11-03T17:14:22.399 INFO:tasks.workunit.client.0.smithi027.stdout:1/586: dread - d6/d11/dac/d79/d25/f74 zero size 2021-11-03T17:14:22.400 INFO:tasks.workunit.client.1.smithi115.stdout:1/547: unlink d7/dc/dd/d42/f8d 0 2021-11-03T17:14:22.401 INFO:tasks.workunit.client.0.smithi027.stdout:1/587: dread d6/d11/dac/d79/f99 [0,4194304] 0 2021-11-03T17:14:22.401 INFO:tasks.workunit.client.1.smithi115.stdout:1/548: unlink d7/dc/dd/d24/l37 0 2021-11-03T17:14:22.403 INFO:tasks.workunit.client.1.smithi115.stdout:1/549: getdents d7/dc/d1a 0 2021-11-03T17:14:22.420 INFO:tasks.workunit.client.0.smithi027.stdout:1/588: dwrite d6/de/d13/d15/d4e/f91 [0,4194304] 0 2021-11-03T17:14:22.424 INFO:tasks.workunit.client.0.smithi027.stdout:1/589: dread d6/de/f92 [0,4194304] 0 2021-11-03T17:14:22.425 INFO:tasks.workunit.client.0.smithi027.stdout:1/590: getdents d6/de/d13/d15/d4e 0 2021-11-03T17:14:22.425 INFO:tasks.workunit.client.0.smithi027.stdout:1/591: dread - d6/d11/dac/d79/d25/d28/d5c/fc9 zero size 2021-11-03T17:14:22.425 INFO:tasks.workunit.client.0.smithi027.stdout:1/592: truncate d6/d11/dac/d79/d31/d58/fcd 8425182 0 2021-11-03T17:14:22.431 INFO:tasks.workunit.client.1.smithi115.stdout:1/550: dwrite d7/dc/dab/d73/d7f/fa8 [0,4194304] 0 2021-11-03T17:14:22.434 INFO:tasks.workunit.client.1.smithi115.stdout:1/551: mkdir d7/dc/dd/db4 0 2021-11-03T17:14:22.437 INFO:tasks.workunit.client.1.smithi115.stdout:1/552: unlink d7/dc/f2f 0 2021-11-03T17:14:22.437 INFO:tasks.workunit.client.1.smithi115.stdout:1/553: fdatasync d7/fb3 0 2021-11-03T17:14:22.437 INFO:tasks.workunit.client.1.smithi115.stdout:1/554: readlink d7/dc/dd/d24/d4b/d55/l6a 0 2021-11-03T17:14:22.438 INFO:tasks.workunit.client.1.smithi115.stdout:1/555: read d7/f90 [1274630,58025] 0 2021-11-03T17:14:22.438 INFO:tasks.workunit.client.1.smithi115.stdout:1/556: chown d7/dc/dd/d24/d26/f4f 1 1 2021-11-03T17:14:22.447 INFO:tasks.workunit.client.0.smithi027.stdout:1/593: dwrite d6/d11/dac/d79/d31/f54 [0,4194304] 0 2021-11-03T17:14:22.447 INFO:tasks.workunit.client.0.smithi027.stdout:1/594: stat d6/d11/d14/f88 0 2021-11-03T17:14:22.447 INFO:tasks.workunit.client.0.smithi027.stdout:1/595: dread - d6/da7/fd2 zero size 2021-11-03T17:14:22.447 INFO:tasks.workunit.client.0.smithi027.stdout:1/596: readlink d6/d11/dac/d79/l87 0 2021-11-03T17:14:22.449 INFO:tasks.workunit.client.0.smithi027.stdout:1/597: mkdir d6/d11/dac/d79/d25/d28/d5c/d81/dd5 0 2021-11-03T17:14:22.450 INFO:tasks.workunit.client.0.smithi027.stdout:1/598: chown d6/d11/dac/d79/d37/d49/la1 309396 1 2021-11-03T17:14:22.451 INFO:tasks.workunit.client.0.smithi027.stdout:1/599: getdents d6/d11/dac/d79/d31/d58 0 2021-11-03T17:14:22.451 INFO:tasks.workunit.client.0.smithi027.stdout:1/600: mknod d6/de/cd6 1 2021-11-03T17:14:22.453 INFO:tasks.workunit.client.1.smithi115.stdout:1/557: dwrite d7/dc/dd/d3e/fa1 [0,4194304] 0 2021-11-03T17:14:22.453 INFO:tasks.workunit.client.1.smithi115.stdout:1/558: write d7/dc/dd/d24/d43/f74 [1525372,10900] 0 2021-11-03T17:14:22.454 INFO:tasks.workunit.client.1.smithi115.stdout:1/559: stat d7/l39 0 2021-11-03T17:14:22.486 INFO:tasks.workunit.client.1.smithi115.stdout:1/560: dwrite d7/dc/dd/d24/f2b [0,4194304] 0 2021-11-03T17:14:22.487 INFO:tasks.workunit.client.1.smithi115.stdout:1/561: fdatasync d7/dc/dd/d24/d4d/d60/f6d 0 2021-11-03T17:14:22.487 INFO:tasks.workunit.client.1.smithi115.stdout:1/562: stat d7/dc/dd/d24/d4d/d60/d8f/laa 0 2021-11-03T17:14:22.487 INFO:tasks.workunit.client.1.smithi115.stdout:1/563: fsync d7/dc/dd/d42/f7a 0 2021-11-03T17:14:22.488 INFO:tasks.workunit.client.0.smithi027.stdout:8/672: sync 2021-11-03T17:14:22.489 INFO:tasks.workunit.client.0.smithi027.stdout:1/601: dwrite d6/d11/d14/f17 [0,4194304] 0 2021-11-03T17:14:22.489 INFO:tasks.workunit.client.1.smithi115.stdout:3/585: sync 2021-11-03T17:14:22.489 INFO:tasks.workunit.client.1.smithi115.stdout:8/482: sync 2021-11-03T17:14:22.489 INFO:tasks.workunit.client.1.smithi115.stdout:6/436: sync 2021-11-03T17:14:22.490 INFO:tasks.workunit.client.1.smithi115.stdout:3/586: mknod d13/d3b/db1/cc9 1 2021-11-03T17:14:22.490 INFO:tasks.workunit.client.1.smithi115.stdout:3/587: read - d13/f7c zero size 2021-11-03T17:14:22.491 INFO:tasks.workunit.client.0.smithi027.stdout:8/673: creat d7/d2d/d38/dc3/fdd x:0 0 0 2021-11-03T17:14:22.491 INFO:tasks.workunit.client.1.smithi115.stdout:3/588: rmdir d13/d3b 39 2021-11-03T17:14:22.491 INFO:tasks.workunit.client.1.smithi115.stdout:6/437: write d0/d1/d2/ff [3796801,46467] 0 2021-11-03T17:14:22.492 INFO:tasks.workunit.client.0.smithi027.stdout:1/602: rename d6/d11/dac/d79/d25/d28/d5c/d68 to d6/d11/dac/d79/d25/d28/d32/dd7 0 2021-11-03T17:14:22.493 INFO:tasks.workunit.client.1.smithi115.stdout:3/589: creat d13/d2a/d53/d70/d92/dc2/fca x:0 0 0 2021-11-03T17:14:22.493 INFO:tasks.workunit.client.1.smithi115.stdout:3/590: fsync d13/d2a/d60/f77 0 2021-11-03T17:14:22.495 INFO:tasks.workunit.client.0.smithi027.stdout:1/603: creat d6/d11/dac/d79/d25/d28/d41/fd8 x:0 0 0 2021-11-03T17:14:22.495 INFO:tasks.workunit.client.0.smithi027.stdout:1/604: write d6/d11/dac/d79/f9d [865524,14472] 0 2021-11-03T17:14:22.495 INFO:tasks.workunit.client.0.smithi027.stdout:1/605: symlink d6/de/ld9 0 2021-11-03T17:14:22.496 INFO:tasks.workunit.client.0.smithi027.stdout:1/606: creat d6/d11/dac/d79/d25/d28/d5c/d81/dd5/fda x:0 0 0 2021-11-03T17:14:22.497 INFO:tasks.workunit.client.1.smithi115.stdout:8/483: dread d4/f28 [0,4194304] 0 2021-11-03T17:14:22.498 INFO:tasks.workunit.client.1.smithi115.stdout:3/591: dread d13/d51/d2e/f2f [0,4194304] 0 2021-11-03T17:14:22.498 INFO:tasks.workunit.client.1.smithi115.stdout:1/564: write d7/dc/d1a/f6b [1006600,92751] 0 2021-11-03T17:14:22.499 INFO:tasks.workunit.client.1.smithi115.stdout:1/565: truncate d7/dc/dd/f95 507489 0 2021-11-03T17:14:22.499 INFO:tasks.workunit.client.0.smithi027.stdout:1/607: dread d6/d11/dac/d79/d31/f34 [0,4194304] 0 2021-11-03T17:14:22.502 INFO:tasks.workunit.client.1.smithi115.stdout:8/484: link d4/d6/f2c d4/d6/d18/fa5 0 2021-11-03T17:14:22.502 INFO:tasks.workunit.client.1.smithi115.stdout:3/592: mkdir d13/da4/dcb 0 2021-11-03T17:14:22.503 INFO:tasks.workunit.client.1.smithi115.stdout:1/566: symlink d7/dc/dd/d24/d4d/lb5 0 2021-11-03T17:14:22.503 INFO:tasks.workunit.client.1.smithi115.stdout:8/485: truncate d4/d6/d18/d44/d6e/f9f 465851 0 2021-11-03T17:14:22.503 INFO:tasks.workunit.client.1.smithi115.stdout:3/593: rmdir d13/d2a/d60 39 2021-11-03T17:14:22.505 INFO:tasks.workunit.client.1.smithi115.stdout:1/567: creat d7/dc/dab/d73/d7f/fb6 x:0 0 0 2021-11-03T17:14:22.506 INFO:tasks.workunit.client.1.smithi115.stdout:3/594: symlink d13/d2a/lcc 0 2021-11-03T17:14:22.506 INFO:tasks.workunit.client.1.smithi115.stdout:1/568: symlink d7/d38/d84/lb7 0 2021-11-03T17:14:22.510 INFO:tasks.workunit.client.1.smithi115.stdout:3/595: link d13/d51/d40/l80 d13/d2a/d53/d70/d96/lcd 0 2021-11-03T17:14:22.512 INFO:tasks.workunit.client.0.smithi027.stdout:8/674: dwrite d7/d53/d67/fc7 [0,4194304] 0 2021-11-03T17:14:22.513 INFO:tasks.workunit.client.1.smithi115.stdout:3/596: chown d13/l15 28063 1 2021-11-03T17:14:22.517 INFO:tasks.workunit.client.0.smithi027.stdout:8/675: symlink d7/d10/d3f/d55/db7/lde 0 2021-11-03T17:14:22.518 INFO:tasks.workunit.client.1.smithi115.stdout:3/597: read fc [5897284,38168] 0 2021-11-03T17:14:22.521 INFO:tasks.workunit.client.1.smithi115.stdout:3/598: symlink d13/d51/d2e/db4/lce 0 2021-11-03T17:14:22.521 INFO:tasks.workunit.client.1.smithi115.stdout:3/599: read - d13/d2a/f4e zero size 2021-11-03T17:14:22.545 INFO:tasks.workunit.client.0.smithi027.stdout:1/608: dwrite d6/d11/dac/d79/d25/d28/d41/f6f [0,4194304] 0 2021-11-03T17:14:22.545 INFO:tasks.workunit.client.1.smithi115.stdout:1/569: dwrite d7/d38/d84/f9a [0,4194304] 0 2021-11-03T17:14:22.546 INFO:tasks.workunit.client.0.smithi027.stdout:1/609: unlink d6/l3b 0 2021-11-03T17:14:22.547 INFO:tasks.workunit.client.0.smithi027.stdout:1/610: write d6/d11/dac/d79/f9d [848036,105014] 0 2021-11-03T17:14:22.548 INFO:tasks.workunit.client.1.smithi115.stdout:8/486: dwrite d4/d6/d18/fa5 [0,4194304] 0 2021-11-03T17:14:22.548 INFO:tasks.workunit.client.1.smithi115.stdout:3/600: dwrite d13/d51/d40/f4f [0,4194304] 0 2021-11-03T17:14:22.549 INFO:tasks.workunit.client.0.smithi027.stdout:8/676: dwrite d7/d2d/d38/f44 [0,4194304] 0 2021-11-03T17:14:22.549 INFO:tasks.workunit.client.0.smithi027.stdout:8/677: mknod d7/dcc/dcf/d9d/cdf 1 2021-11-03T17:14:22.549 INFO:tasks.workunit.client.0.smithi027.stdout:8/678: fsync d7/d2d/d38/f44 0 2021-11-03T17:14:22.550 INFO:tasks.workunit.client.0.smithi027.stdout:8/679: rmdir d7/d10/d3f/d55/db7 39 2021-11-03T17:14:22.551 INFO:tasks.workunit.client.0.smithi027.stdout:8/680: chown d7/d10/d3f/f88 1125423 1 2021-11-03T17:14:22.551 INFO:tasks.workunit.client.1.smithi115.stdout:8/487: mkdir d4/d6/d3f/da4/da6 0 2021-11-03T17:14:22.552 INFO:tasks.workunit.client.1.smithi115.stdout:8/488: fsync d4/d6/d3f/d43/f92 0 2021-11-03T17:14:22.552 INFO:tasks.workunit.client.1.smithi115.stdout:8/489: mknod d4/d6/d18/d9d/ca7 1 2021-11-03T17:14:22.553 INFO:tasks.workunit.client.1.smithi115.stdout:3/601: rmdir d13 39 2021-11-03T17:14:22.554 INFO:tasks.workunit.client.1.smithi115.stdout:8/490: write d4/d6/f2b [4002155,89522] 0 2021-11-03T17:14:22.555 INFO:tasks.workunit.client.1.smithi115.stdout:8/491: chown d4/d6/d18/l78 126 1 2021-11-03T17:14:22.555 INFO:tasks.workunit.client.1.smithi115.stdout:3/602: write d13/d50/f8e [3190687,111229] 0 2021-11-03T17:14:22.556 INFO:tasks.workunit.client.1.smithi115.stdout:3/603: fsync d13/d50/f42 0 2021-11-03T17:14:22.556 INFO:tasks.workunit.client.1.smithi115.stdout:3/604: readlink d13/d50/l6d 0 2021-11-03T17:14:22.556 INFO:tasks.workunit.client.1.smithi115.stdout:8/492: getdents d4/d6/d3f/d43 0 2021-11-03T17:14:22.557 INFO:tasks.workunit.client.1.smithi115.stdout:8/493: fsync d4/d6/d18/f77 0 2021-11-03T17:14:22.557 INFO:tasks.workunit.client.1.smithi115.stdout:8/494: write d4/d6/d18/d30/d33/f75 [423724,110943] 0 2021-11-03T17:14:22.557 INFO:tasks.workunit.client.1.smithi115.stdout:8/495: fdatasync d4/d6/d18/f77 0 2021-11-03T17:14:22.558 INFO:tasks.workunit.client.1.smithi115.stdout:8/496: creat d4/d6/d18/d44/d6e/fa8 x:0 0 0 2021-11-03T17:14:22.560 INFO:tasks.workunit.client.1.smithi115.stdout:8/497: link d4/d6/d18/d30/f3a d4/d6/fa9 0 2021-11-03T17:14:22.563 INFO:tasks.workunit.client.1.smithi115.stdout:8/498: write d4/d6/d18/d30/d33/f59 [2790910,121297] 0 2021-11-03T17:14:22.565 INFO:tasks.workunit.client.1.smithi115.stdout:8/499: link d4/l49 d4/laa 0 2021-11-03T17:14:22.566 INFO:tasks.workunit.client.1.smithi115.stdout:8/500: fdatasync d4/d6/d18/d30/d33/f87 0 2021-11-03T17:14:22.566 INFO:tasks.workunit.client.1.smithi115.stdout:8/501: write d4/d6/d18/d44/d6e/f9f [1417143,56401] 0 2021-11-03T17:14:22.567 INFO:tasks.workunit.client.0.smithi027.stdout:1/611: dwrite d6/de/f19 [0,4194304] 0 2021-11-03T17:14:22.567 INFO:tasks.workunit.client.0.smithi027.stdout:1/612: truncate d6/d11/f89 726442 0 2021-11-03T17:14:22.567 INFO:tasks.workunit.client.0.smithi027.stdout:1/613: chown d6/de/f1a 536205538 1 2021-11-03T17:14:22.567 INFO:tasks.workunit.client.0.smithi027.stdout:8/681: dread f6 [0,4194304] 0 2021-11-03T17:14:22.568 INFO:tasks.workunit.client.0.smithi027.stdout:8/682: write d7/d10/d21/d41/fc2 [723229,37325] 0 2021-11-03T17:14:22.568 INFO:tasks.workunit.client.1.smithi115.stdout:8/502: creat d4/d6/d18/d44/d47/fab x:0 0 0 2021-11-03T17:14:22.569 INFO:tasks.workunit.client.1.smithi115.stdout:1/570: dwrite d7/dc/dab/d73/fa9 [0,4194304] 0 2021-11-03T17:14:22.569 INFO:tasks.workunit.client.1.smithi115.stdout:1/571: stat d7/d38/d84/lad 0 2021-11-03T17:14:22.570 INFO:tasks.workunit.client.0.smithi027.stdout:1/614: creat d6/d11/dac/d79/d25/d28/d5c/d81/d7d/fdb x:0 0 0 2021-11-03T17:14:22.570 INFO:tasks.workunit.client.0.smithi027.stdout:1/615: truncate d6/d11/dac/d79/d25/d28/d5c/d81/f86 629176 0 2021-11-03T17:14:22.571 INFO:tasks.workunit.client.0.smithi027.stdout:8/683: creat d7/d2d/da8/fe0 x:0 0 0 2021-11-03T17:14:22.572 INFO:tasks.workunit.client.0.smithi027.stdout:8/684: dread - d7/d2d/d38/f6e zero size 2021-11-03T17:14:22.572 INFO:tasks.workunit.client.0.smithi027.stdout:1/616: getdents d6/d11/dac/d79/d25/d28/d5c/d7a 0 2021-11-03T17:14:22.572 INFO:tasks.workunit.client.0.smithi027.stdout:1/617: fdatasync d6/d11/dac/d79/d25/d28/d5c/d81/f86 0 2021-11-03T17:14:22.572 INFO:tasks.workunit.client.0.smithi027.stdout:8/685: dread d7/d10/d3f/d55/db7/fbc [0,4194304] 0 2021-11-03T17:14:22.572 INFO:tasks.workunit.client.0.smithi027.stdout:1/618: write d6/d11/dac/d79/f59 [1998972,29753] 0 2021-11-03T17:14:22.573 INFO:tasks.workunit.client.0.smithi027.stdout:8/686: rmdir d7/d10/d21/d41 39 2021-11-03T17:14:22.576 INFO:tasks.workunit.client.0.smithi027.stdout:1/619: link d6/d11/dac/d79/d31/d58/fcd d6/d11/dac/d79/d25/d28/d5c/d81/fdc 0 2021-11-03T17:14:22.577 INFO:tasks.workunit.client.1.smithi115.stdout:1/572: rename d7/dc/dd/d24/d26/f85 to d7/dc/dd/d24/d4d/d60/fb8 0 2021-11-03T17:14:22.577 INFO:tasks.workunit.client.1.smithi115.stdout:1/573: read d7/dc/dd/d24/d4d/f7b [45010,85365] 0 2021-11-03T17:14:22.579 INFO:tasks.workunit.client.0.smithi027.stdout:1/620: link d6/d11/dac/d79/d25/d28/d41/lcf d6/d11/dac/d79/d25/d28/d5c/d81/ldd 0 2021-11-03T17:14:22.579 INFO:tasks.workunit.client.1.smithi115.stdout:1/574: unlink d7/dc/dd/d24/d4d/f79 0 2021-11-03T17:14:22.580 INFO:tasks.workunit.client.0.smithi027.stdout:1/621: symlink d6/d11/dac/d79/dab/db4/lde 0 2021-11-03T17:14:22.581 INFO:tasks.workunit.client.1.smithi115.stdout:1/575: rename d7/dc/dd/d24/d4b/l9f to d7/dc/dd/d24/d4d/lb9 0 2021-11-03T17:14:22.581 INFO:tasks.workunit.client.1.smithi115.stdout:1/576: creat d7/dc/dd/fba x:0 0 0 2021-11-03T17:14:22.582 INFO:tasks.workunit.client.1.smithi115.stdout:1/577: rmdir d7/d9d 39 2021-11-03T17:14:22.583 INFO:tasks.workunit.client.1.smithi115.stdout:1/578: chown d7/dc/dd/d24/d4b/d55/f66 1 1 2021-11-03T17:14:22.583 INFO:tasks.workunit.client.1.smithi115.stdout:1/579: mknod d7/dc/dd/d42/cbb 1 2021-11-03T17:14:22.583 INFO:tasks.workunit.client.1.smithi115.stdout:3/605: dwrite d13/d51/f45 [4194304,4194304] 0 2021-11-03T17:14:22.583 INFO:tasks.workunit.client.1.smithi115.stdout:3/606: readlink d13/l15 0 2021-11-03T17:14:22.584 INFO:tasks.workunit.client.1.smithi115.stdout:3/607: link d13/d2a/d53/d70/d92/fc1 d13/d51/fcf 0 2021-11-03T17:14:22.587 INFO:tasks.workunit.client.1.smithi115.stdout:8/503: dwrite d4/d7d/fa0 [0,4194304] 0 2021-11-03T17:14:22.594 INFO:tasks.workunit.client.1.smithi115.stdout:8/504: write d4/d6/d18/d30/f35 [7568709,118960] 0 2021-11-03T17:14:22.594 INFO:tasks.workunit.client.1.smithi115.stdout:8/505: fsync d4/d6/d18/f85 0 2021-11-03T17:14:22.594 INFO:tasks.workunit.client.1.smithi115.stdout:8/506: mknod d4/d6/d18/d30/d33/d93/cac 1 2021-11-03T17:14:22.595 INFO:tasks.workunit.client.1.smithi115.stdout:1/580: dread d7/dc/dab/d73/fa9 [0,4194304] 0 2021-11-03T17:14:22.595 INFO:tasks.workunit.client.1.smithi115.stdout:1/581: chown d7/dc/dd/d42 56315 1 2021-11-03T17:14:22.596 INFO:tasks.workunit.client.0.smithi027.stdout:8/687: dwrite d7/d2d/d38/f6e [0,4194304] 0 2021-11-03T17:14:22.596 INFO:tasks.workunit.client.0.smithi027.stdout:1/622: dwrite f2 [0,4194304] 0 2021-11-03T17:14:22.596 INFO:tasks.workunit.client.0.smithi027.stdout:1/623: write d6/de/f19 [7716608,25439] 0 2021-11-03T17:14:22.596 INFO:tasks.workunit.client.0.smithi027.stdout:1/624: getdents d6/d11/dac/d79/d25/d28/d32/dd7/d8b 0 2021-11-03T17:14:22.597 INFO:tasks.workunit.client.1.smithi115.stdout:8/507: link d4/d6/d18/d30/d33/f5c d4/d6/d18/d30/d33/d93/fad 0 2021-11-03T17:14:22.597 INFO:tasks.workunit.client.1.smithi115.stdout:8/508: write d4/d6/d18/f85 [869944,111853] 0 2021-11-03T17:14:22.598 INFO:tasks.workunit.client.1.smithi115.stdout:8/509: write d4/d6/d18/d30/d33/f75 [475348,70923] 0 2021-11-03T17:14:22.598 INFO:tasks.workunit.client.1.smithi115.stdout:8/510: readlink d4/d6/d18/d44/d47/l4f 0 2021-11-03T17:14:22.598 INFO:tasks.workunit.client.1.smithi115.stdout:3/608: write d13/d35/f3a [2761054,92897] 0 2021-11-03T17:14:22.599 INFO:tasks.workunit.client.1.smithi115.stdout:1/582: creat d7/dc/dd/d24/d43/d78/fbc x:0 0 0 2021-11-03T17:14:22.600 INFO:tasks.workunit.client.1.smithi115.stdout:1/583: chown d7/dc/dd/d42/f47 247828021 1 2021-11-03T17:14:22.600 INFO:tasks.workunit.client.1.smithi115.stdout:3/609: creat d13/d2a/d8a/fd0 x:0 0 0 2021-11-03T17:14:22.601 INFO:tasks.workunit.client.0.smithi027.stdout:8/688: getdents d7/d2d/d38/db1 0 2021-11-03T17:14:22.601 INFO:tasks.workunit.client.0.smithi027.stdout:8/689: fdatasync d7/d10/d3f/d9b/f3e 0 2021-11-03T17:14:22.601 INFO:tasks.workunit.client.0.smithi027.stdout:8/690: write d7/fb2 [886610,36416] 0 2021-11-03T17:14:22.601 INFO:tasks.workunit.client.0.smithi027.stdout:1/625: mknod d6/cdf 1 2021-11-03T17:14:22.602 INFO:tasks.workunit.client.0.smithi027.stdout:1/626: stat d6/d11/dac/d79/d25/d28/la4 0 2021-11-03T17:14:22.602 INFO:tasks.workunit.client.0.smithi027.stdout:1/627: fsync d6/d11/dac/d79/d25/d28/d32/f8e 0 2021-11-03T17:14:22.602 INFO:tasks.workunit.client.0.smithi027.stdout:8/691: creat d7/d10/dbd/fe1 x:0 0 0 2021-11-03T17:14:22.602 INFO:tasks.workunit.client.0.smithi027.stdout:8/692: stat d7/d10/d21/d41/fa3 0 2021-11-03T17:14:22.603 INFO:tasks.workunit.client.0.smithi027.stdout:1/628: rename d6/de/d13/d15 to d6/d11/dac/d79/d37/de0 0 2021-11-03T17:14:22.603 INFO:tasks.workunit.client.1.smithi115.stdout:1/584: creat d7/dc/dd/db4/fbd x:0 0 0 2021-11-03T17:14:22.604 INFO:tasks.workunit.client.0.smithi027.stdout:1/629: write f2 [6190524,65532] 0 2021-11-03T17:14:22.610 INFO:tasks.workunit.client.1.smithi115.stdout:8/511: dread d4/d6/d18/f63 [0,4194304] 0 2021-11-03T17:14:22.610 INFO:tasks.workunit.client.1.smithi115.stdout:8/512: chown d4/d6/d18/d30/l74 42 1 2021-11-03T17:14:22.612 INFO:tasks.workunit.client.0.smithi027.stdout:8/693: rename d7/f1b to d7/d10/d21/d41/fe2 0 2021-11-03T17:14:22.612 INFO:tasks.workunit.client.1.smithi115.stdout:8/513: creat d4/d6/d18/d30/d79/d7f/fae x:0 0 0 2021-11-03T17:14:22.613 INFO:tasks.workunit.client.1.smithi115.stdout:8/514: write d4/d6/d18/d44/d6e/fa8 [475366,27070] 0 2021-11-03T17:14:22.613 INFO:tasks.workunit.client.1.smithi115.stdout:8/515: dread - d4/d6/f8a zero size 2021-11-03T17:14:22.613 INFO:tasks.workunit.client.0.smithi027.stdout:1/630: getdents d6/d11/dac/d79/d25/d28/d5c/d81/dd5 0 2021-11-03T17:14:22.614 INFO:tasks.workunit.client.0.smithi027.stdout:1/631: dread - d6/d11/dac/d79/d25/d28/d41/fd8 zero size 2021-11-03T17:14:22.614 INFO:tasks.workunit.client.0.smithi027.stdout:1/632: fsync d6/de/faf 0 2021-11-03T17:14:22.614 INFO:tasks.workunit.client.0.smithi027.stdout:1/633: write d6/d11/dac/d79/d31/d58/f70 [933124,87032] 0 2021-11-03T17:14:22.614 INFO:tasks.workunit.client.0.smithi027.stdout:1/634: mknod d6/d11/dac/d79/d25/ce1 1 2021-11-03T17:14:22.615 INFO:tasks.workunit.client.0.smithi027.stdout:8/694: mkdir d7/d10/dbd/de3 0 2021-11-03T17:14:22.617 INFO:tasks.workunit.client.0.smithi027.stdout:8/695: dread d7/d10/dbd/fc1 [0,4194304] 0 2021-11-03T17:14:22.617 INFO:tasks.workunit.client.0.smithi027.stdout:8/696: mknod d7/d10/d21/d41/ce4 1 2021-11-03T17:14:22.617 INFO:tasks.workunit.client.0.smithi027.stdout:1/635: mkdir d6/d11/de2 0 2021-11-03T17:14:22.618 INFO:tasks.workunit.client.0.smithi027.stdout:1/636: write d6/d11/dac/d79/d25/d28/d5c/d7a/fbe [1534417,43481] 0 2021-11-03T17:14:22.618 INFO:tasks.workunit.client.0.smithi027.stdout:1/637: mknod d6/d11/dac/d79/dab/ce3 1 2021-11-03T17:14:22.618 INFO:tasks.workunit.client.0.smithi027.stdout:8/697: link d7/d2d/da8/fe0 d7/d2d/d38/db1/fe5 0 2021-11-03T17:14:22.619 INFO:tasks.workunit.client.0.smithi027.stdout:1/638: rmdir d6/d11/dac/d79 39 2021-11-03T17:14:22.620 INFO:tasks.workunit.client.0.smithi027.stdout:8/698: getdents d7/d2d/da9 0 2021-11-03T17:14:22.620 INFO:tasks.workunit.client.0.smithi027.stdout:8/699: mknod d7/d10/d3f/d9b/ce6 1 2021-11-03T17:14:22.622 INFO:tasks.workunit.client.0.smithi027.stdout:8/700: rename d7/d10/d3f/d9b/d95/l90 to d7/d2d/d38/db1/le7 0 2021-11-03T17:14:22.625 INFO:tasks.workunit.client.1.smithi115.stdout:3/610: dwrite d13/d2a/d8a/fa7 [0,4194304] 0 2021-11-03T17:14:22.626 INFO:tasks.workunit.client.1.smithi115.stdout:1/585: dwrite d7/f8a [4194304,4194304] 0 2021-11-03T17:14:22.629 INFO:tasks.workunit.client.1.smithi115.stdout:3/611: getdents d13/d2a/d53 0 2021-11-03T17:14:22.631 INFO:tasks.workunit.client.1.smithi115.stdout:1/586: truncate d7/dc/f16 1125843 0 2021-11-03T17:14:22.634 INFO:tasks.workunit.client.0.smithi027.stdout:1/639: dread d6/d11/dac/d79/f22 [0,4194304] 0 2021-11-03T17:14:22.636 INFO:tasks.workunit.client.0.smithi027.stdout:1/640: getdents d6/d11/dac 0 2021-11-03T17:14:22.644 INFO:tasks.workunit.client.1.smithi115.stdout:3/612: dread d13/d35/d54/fab [0,4194304] 0 2021-11-03T17:14:22.644 INFO:tasks.workunit.client.1.smithi115.stdout:3/613: write d13/d51/f29 [228818,57492] 0 2021-11-03T17:14:22.648 INFO:tasks.workunit.client.1.smithi115.stdout:1/587: dread d7/f77 [0,4194304] 0 2021-11-03T17:14:22.649 INFO:tasks.workunit.client.1.smithi115.stdout:8/516: dwrite d4/d6/d18/d44/d6e/fa8 [0,4194304] 0 2021-11-03T17:14:22.650 INFO:tasks.workunit.client.1.smithi115.stdout:1/588: mkdir d7/dc/dd/d3e/dbe 0 2021-11-03T17:14:22.651 INFO:tasks.workunit.client.1.smithi115.stdout:1/589: chown d7/dc/dd/d24/d43/d78/fb1 6269591 1 2021-11-03T17:14:22.652 INFO:tasks.workunit.client.1.smithi115.stdout:8/517: creat d4/d6/d18/faf x:0 0 0 2021-11-03T17:14:22.653 INFO:tasks.workunit.client.1.smithi115.stdout:1/590: truncate d7/dc/dd/d24/f2b 4073623 0 2021-11-03T17:14:22.654 INFO:tasks.workunit.client.1.smithi115.stdout:8/518: getdents d4/d6/d18/d44/d6e 0 2021-11-03T17:14:22.654 INFO:tasks.workunit.client.1.smithi115.stdout:8/519: rename d4/d6/d18/d30/d79/d7f to d4/d6/d18/d30/d79/d7f/db0 22 2021-11-03T17:14:22.655 INFO:tasks.workunit.client.0.smithi027.stdout:1/641: dread d6/d11/dac/d79/d37/de0/f16 [4194304,4194304] 0 2021-11-03T17:14:22.656 INFO:tasks.workunit.client.1.smithi115.stdout:1/591: mkdir d7/dc/dd/d3e/d58/dbf 0 2021-11-03T17:14:22.656 INFO:tasks.workunit.client.0.smithi027.stdout:8/701: dwrite d7/d10/f16 [0,4194304] 0 2021-11-03T17:14:22.658 INFO:tasks.workunit.client.1.smithi115.stdout:1/592: symlink d7/lc0 0 2021-11-03T17:14:22.658 INFO:tasks.workunit.client.1.smithi115.stdout:1/593: write d7/dc/dd/fba [628795,47868] 0 2021-11-03T17:14:22.658 INFO:tasks.workunit.client.0.smithi027.stdout:1/642: rename d6/d11/dac/d79/d31 to d6/d11/dac/d79/d37/de0/de4 0 2021-11-03T17:14:22.659 INFO:tasks.workunit.client.0.smithi027.stdout:1/643: write d6/d11/dac/d79/d37/de0/d43/f48 [36448,127216] 0 2021-11-03T17:14:22.660 INFO:tasks.workunit.client.1.smithi115.stdout:1/594: rename d7/dc/dab/d73/l96 to d7/dc/dd/d24/d4d/d60/lc1 0 2021-11-03T17:14:22.661 INFO:tasks.workunit.client.1.smithi115.stdout:1/595: mkdir d7/dc/dd/d3e/d58/dc2 0 2021-11-03T17:14:22.661 INFO:tasks.workunit.client.1.smithi115.stdout:1/596: truncate d7/f90 4384700 0 2021-11-03T17:14:22.662 INFO:tasks.workunit.client.0.smithi027.stdout:1/644: mkdir d6/d11/dac/d79/db0/de5 0 2021-11-03T17:14:22.662 INFO:tasks.workunit.client.1.smithi115.stdout:1/597: truncate d7/dc/dd/d24/d4b/f72 4078702 0 2021-11-03T17:14:22.663 INFO:tasks.workunit.client.0.smithi027.stdout:1/645: symlink d6/d11/dac/d79/dab/db4/le6 0 2021-11-03T17:14:22.663 INFO:tasks.workunit.client.1.smithi115.stdout:1/598: mkdir d7/dc/dd/d24/d26/dc3 0 2021-11-03T17:14:22.664 INFO:tasks.workunit.client.0.smithi027.stdout:1/646: symlink d6/de/d13/le7 0 2021-11-03T17:14:22.664 INFO:tasks.workunit.client.0.smithi027.stdout:1/647: getdents d6/d11/dac/d79/d37/de0/d4e 0 2021-11-03T17:14:22.664 INFO:tasks.workunit.client.0.smithi027.stdout:1/648: chown d6/d11/dac/d79/f9d 14 1 2021-11-03T17:14:22.666 INFO:tasks.workunit.client.1.smithi115.stdout:3/614: dwrite d13/f7c [0,4194304] 0 2021-11-03T17:14:22.668 INFO:tasks.workunit.client.0.smithi027.stdout:8/702: dread d7/d10/d3f/d55/f9f [0,4194304] 0 2021-11-03T17:14:22.668 INFO:tasks.workunit.client.0.smithi027.stdout:8/703: write d7/d10/d21/f8d [5193276,4897] 0 2021-11-03T17:14:22.670 INFO:tasks.workunit.client.1.smithi115.stdout:3/615: symlink d13/d3b/db1/ld1 0 2021-11-03T17:14:22.671 INFO:tasks.workunit.client.1.smithi115.stdout:3/616: readlink d13/d51/d2e/db4/lc7 0 2021-11-03T17:14:22.671 INFO:tasks.workunit.client.1.smithi115.stdout:3/617: read - d13/d2a/d53/f8c zero size 2021-11-03T17:14:22.673 INFO:tasks.workunit.client.0.smithi027.stdout:8/704: read d7/d10/d21/d41/fa7 [833231,118033] 0 2021-11-03T17:14:22.673 INFO:tasks.workunit.client.0.smithi027.stdout:8/705: mknod d7/dcc/dcf/ce8 1 2021-11-03T17:14:22.675 INFO:tasks.workunit.client.0.smithi027.stdout:8/706: symlink d7/le9 0 2021-11-03T17:14:22.680 INFO:tasks.workunit.client.0.smithi027.stdout:8/707: dread d7/d10/d3f/d9b/f3c [4194304,4194304] 0 2021-11-03T17:14:22.681 INFO:tasks.workunit.client.1.smithi115.stdout:3/618: truncate d13/d51/d2e/fa2 351712 0 2021-11-03T17:14:22.683 INFO:tasks.workunit.client.1.smithi115.stdout:3/619: mkdir d13/d2a/d53/d72/dd2 0 2021-11-03T17:14:22.697 INFO:tasks.workunit.client.1.smithi115.stdout:3/620: dread d13/d35/f3a [0,4194304] 0 2021-11-03T17:14:22.698 INFO:tasks.workunit.client.1.smithi115.stdout:1/599: dwrite d7/dc/d1a/f3d [0,4194304] 0 2021-11-03T17:14:22.698 INFO:tasks.workunit.client.1.smithi115.stdout:1/600: chown d7/dc/dd/d24/d4b 1 1 2021-11-03T17:14:22.699 INFO:tasks.workunit.client.1.smithi115.stdout:8/520: dwrite d4/d6/d18/d44/d6e/fa8 [4194304,4194304] 0 2021-11-03T17:14:22.699 INFO:tasks.workunit.client.1.smithi115.stdout:8/521: mknod d4/d6/d18/d44/d47/cb1 1 2021-11-03T17:14:22.700 INFO:tasks.workunit.client.1.smithi115.stdout:3/621: mkdir d13/d2a/d60/dd3 0 2021-11-03T17:14:22.702 INFO:tasks.workunit.client.0.smithi027.stdout:1/649: dwrite d6/d11/dac/d79/f99 [0,4194304] 0 2021-11-03T17:14:22.703 INFO:tasks.workunit.client.0.smithi027.stdout:8/708: rmdir d7/d2d/d38 39 2021-11-03T17:14:22.704 INFO:tasks.workunit.client.1.smithi115.stdout:8/522: link d4/f26 d4/d6/d18/d30/d79/fb2 0 2021-11-03T17:14:22.704 INFO:tasks.workunit.client.1.smithi115.stdout:8/523: mknod d4/d6/d18/d30/d79/d7f/cb3 1 2021-11-03T17:14:22.704 INFO:tasks.workunit.client.1.smithi115.stdout:8/524: chown d4/d6/l24 1 1 2021-11-03T17:14:22.705 INFO:tasks.workunit.client.1.smithi115.stdout:3/622: creat d13/d2a/d8a/d99/fd4 x:0 0 0 2021-11-03T17:14:22.709 INFO:tasks.workunit.client.1.smithi115.stdout:8/525: symlink d4/d6/d18/d44/d6e/lb4 0 2021-11-03T17:14:22.711 INFO:tasks.workunit.client.0.smithi027.stdout:1/650: unlink d6/d11/dac/fd3 0 2021-11-03T17:14:22.712 INFO:tasks.workunit.client.1.smithi115.stdout:3/623: symlink d13/ld5 0 2021-11-03T17:14:22.712 INFO:tasks.workunit.client.1.smithi115.stdout:3/624: fsync d13/d2a/d60/f77 0 2021-11-03T17:14:22.713 INFO:tasks.workunit.client.0.smithi027.stdout:8/709: rmdir d7/dcc/dcf 39 2021-11-03T17:14:22.714 INFO:tasks.workunit.client.1.smithi115.stdout:3/625: write d13/d35/f3a [398584,36732] 0 2021-11-03T17:14:22.714 INFO:tasks.workunit.client.1.smithi115.stdout:3/626: readlink d13/l15 0 2021-11-03T17:14:22.714 INFO:tasks.workunit.client.1.smithi115.stdout:3/627: symlink d13/d2c/ld6 0 2021-11-03T17:14:22.715 INFO:tasks.workunit.client.1.smithi115.stdout:3/628: creat d13/d3b/db1/fd7 x:0 0 0 2021-11-03T17:14:22.716 INFO:tasks.workunit.client.0.smithi027.stdout:8/710: symlink d7/d2d/da9/lea 0 2021-11-03T17:14:22.716 INFO:tasks.workunit.client.0.smithi027.stdout:8/711: chown d7/d2d/da8/fe0 1723444 1 2021-11-03T17:14:22.716 INFO:tasks.workunit.client.0.smithi027.stdout:8/712: truncate d7/d10/d3f/d9b/f2e 3762147 0 2021-11-03T17:14:22.717 INFO:tasks.workunit.client.0.smithi027.stdout:1/651: dread d6/d11/dac/d79/d37/de0/f16 [4194304,4194304] 0 2021-11-03T17:14:22.717 INFO:tasks.workunit.client.0.smithi027.stdout:1/652: dread - d6/d11/dac/d79/d25/d28/d5c/d81/f85 zero size 2021-11-03T17:14:22.717 INFO:tasks.workunit.client.0.smithi027.stdout:8/713: creat d7/d2d/d38/feb x:0 0 0 2021-11-03T17:14:22.718 INFO:tasks.workunit.client.1.smithi115.stdout:1/601: dwrite d7/dc/f15 [4194304,4194304] 0 2021-11-03T17:14:22.719 INFO:tasks.workunit.client.0.smithi027.stdout:8/714: symlink d7/d10/d3f/d55/lec 0 2021-11-03T17:14:22.719 INFO:tasks.workunit.client.0.smithi027.stdout:8/715: read - d7/d10/d3f/d9b/fd4 zero size 2021-11-03T17:14:22.720 INFO:tasks.workunit.client.0.smithi027.stdout:8/716: getdents d7/d10/d18 0 2021-11-03T17:14:22.720 INFO:tasks.workunit.client.0.smithi027.stdout:8/717: fdatasync d7/fb8 0 2021-11-03T17:14:22.720 INFO:tasks.workunit.client.0.smithi027.stdout:8/718: dread - d7/d2d/d38/db1/fe5 zero size 2021-11-03T17:14:22.722 INFO:tasks.workunit.client.1.smithi115.stdout:1/602: write d7/dc/dd/f5a [2178322,19634] 0 2021-11-03T17:14:22.722 INFO:tasks.workunit.client.0.smithi027.stdout:8/719: creat d7/d10/fed x:0 0 0 2021-11-03T17:14:22.723 INFO:tasks.workunit.client.0.smithi027.stdout:8/720: readlink d7/d53/laf 0 2021-11-03T17:14:22.724 INFO:tasks.workunit.client.1.smithi115.stdout:1/603: truncate d7/dc/dab/f33 991051 0 2021-11-03T17:14:22.724 INFO:tasks.workunit.client.1.smithi115.stdout:1/604: mknod d7/dc/daf/cc4 1 2021-11-03T17:14:22.724 INFO:tasks.workunit.client.1.smithi115.stdout:1/605: chown d7/dc/dd/d24/f2b 172 1 2021-11-03T17:14:22.724 INFO:tasks.workunit.client.1.smithi115.stdout:1/606: mknod d7/dc/dd/d24/d26/cc5 1 2021-11-03T17:14:22.727 INFO:tasks.workunit.client.0.smithi027.stdout:8/721: link d7/d10/d3f/d9b/d95/f9a d7/d10/d3f/d9b/fee 0 2021-11-03T17:14:22.728 INFO:tasks.workunit.client.1.smithi115.stdout:3/629: dread d13/d3b/f76 [4194304,4194304] 0 2021-11-03T17:14:22.729 INFO:tasks.workunit.client.1.smithi115.stdout:1/607: write d7/dc/dd/fba [117353,113058] 0 2021-11-03T17:14:22.733 INFO:tasks.workunit.client.1.smithi115.stdout:1/608: symlink d7/dc/dd/d24/d4d/lc6 0 2021-11-03T17:14:22.733 INFO:tasks.workunit.client.1.smithi115.stdout:1/609: readlink d7/dc/d1a/l21 0 2021-11-03T17:14:22.734 INFO:tasks.workunit.client.1.smithi115.stdout:1/610: link d7/dc/dd/d24/d43/f74 d7/dc/dd/fc7 0 2021-11-03T17:14:22.734 INFO:tasks.workunit.client.1.smithi115.stdout:1/611: fsync d7/dc/dd/d24/d4d/d60/f6d 0 2021-11-03T17:14:22.736 INFO:tasks.workunit.client.1.smithi115.stdout:1/612: rename d7/dc/dd/d24/d43/la3 to d7/dc/dd/d3e/dbe/lc8 0 2021-11-03T17:14:22.737 INFO:tasks.workunit.client.1.smithi115.stdout:1/613: chown d7/dc/dd/f5a 3360 1 2021-11-03T17:14:22.737 INFO:tasks.workunit.client.1.smithi115.stdout:1/614: truncate d7/dc/dd/d24/d43/f7e 1032760 0 2021-11-03T17:14:22.739 INFO:tasks.workunit.client.1.smithi115.stdout:1/615: creat d7/dc/dd/d24/d43/fc9 x:0 0 0 2021-11-03T17:14:22.740 INFO:tasks.workunit.client.1.smithi115.stdout:3/630: dread f12 [0,4194304] 0 2021-11-03T17:14:22.741 INFO:tasks.workunit.client.1.smithi115.stdout:1/616: mkdir d7/d38/d84/dca 0 2021-11-03T17:14:22.741 INFO:tasks.workunit.client.0.smithi027.stdout:8/722: dread d7/d10/d3f/d9b/f2e [0,4194304] 0 2021-11-03T17:14:22.741 INFO:tasks.workunit.client.0.smithi027.stdout:8/723: chown d7/d10/d21/d41/fa7 0 1 2021-11-03T17:14:22.742 INFO:tasks.workunit.client.1.smithi115.stdout:3/631: creat d13/d67/fd8 x:0 0 0 2021-11-03T17:14:22.745 INFO:tasks.workunit.client.1.smithi115.stdout:1/617: unlink d7/f77 0 2021-11-03T17:14:22.746 INFO:tasks.workunit.client.1.smithi115.stdout:1/618: write d7/dc/dd/d24/d43/f74 [2507794,78340] 0 2021-11-03T17:14:22.746 INFO:tasks.workunit.client.1.smithi115.stdout:1/619: write d7/dc/f15 [8757277,5823] 0 2021-11-03T17:14:22.746 INFO:tasks.workunit.client.1.smithi115.stdout:1/620: truncate d7/dc/dd/d24/d4d/f7b 1191361 0 2021-11-03T17:14:22.747 INFO:tasks.workunit.client.1.smithi115.stdout:3/632: getdents d13/d2a/d8a 0 2021-11-03T17:14:22.747 INFO:tasks.workunit.client.1.smithi115.stdout:1/621: symlink d7/dc/dd/d3e/d58/dc2/lcb 0 2021-11-03T17:14:22.748 INFO:tasks.workunit.client.1.smithi115.stdout:1/622: symlink d7/dc/dd/d24/d4b/d55/lcc 0 2021-11-03T17:14:22.750 INFO:tasks.workunit.client.1.smithi115.stdout:1/623: link d7/dc/d1a/l25 d7/d9d/lcd 0 2021-11-03T17:14:22.753 INFO:tasks.workunit.client.1.smithi115.stdout:3/633: dread d13/d3b/f76 [0,4194304] 0 2021-11-03T17:14:22.759 INFO:tasks.workunit.client.1.smithi115.stdout:1/624: dread d7/dc/dab/d73/d7f/fa8 [0,4194304] 0 2021-11-03T17:14:22.761 INFO:tasks.workunit.client.1.smithi115.stdout:1/625: creat d7/dc/dd/d24/d4d/d60/d8f/fce x:0 0 0 2021-11-03T17:14:22.768 INFO:tasks.workunit.client.1.smithi115.stdout:1/626: dread d7/dc/f87 [0,4194304] 0 2021-11-03T17:14:22.769 INFO:tasks.workunit.client.1.smithi115.stdout:1/627: symlink d7/dc/dd/d42/lcf 0 2021-11-03T17:14:22.770 INFO:tasks.workunit.client.1.smithi115.stdout:1/628: link d7/dc/dd/d24/d43/f99 d7/d38/d84/fd0 0 2021-11-03T17:14:22.784 INFO:tasks.workunit.client.1.smithi115.stdout:1/629: dwrite d7/dc/dd/d24/d4b/d55/f6c [0,4194304] 0 2021-11-03T17:14:22.785 INFO:tasks.workunit.client.1.smithi115.stdout:1/630: write d7/dc/ff [2151391,7109] 0 2021-11-03T17:14:22.785 INFO:tasks.workunit.client.1.smithi115.stdout:1/631: chown d7/dc/dd/d24/f2b 993312715 1 2021-11-03T17:14:22.791 INFO:tasks.workunit.client.1.smithi115.stdout:1/632: rename d7/dc/dd/d24 to d7/dc/dd/d3e/dd1 0 2021-11-03T17:14:22.792 INFO:tasks.workunit.client.1.smithi115.stdout:1/633: mkdir d7/dc/dd/d3e/d58/dc2/dd2 0 2021-11-03T17:14:22.792 INFO:tasks.workunit.client.1.smithi115.stdout:1/634: chown d7/dc/dd/d3e/dd1/d43/f99 7 1 2021-11-03T17:14:22.793 INFO:tasks.workunit.client.1.smithi115.stdout:1/635: creat d7/dc/dd/d3e/dd1/d26/dc3/fd3 x:0 0 0 2021-11-03T17:14:22.794 INFO:tasks.workunit.client.1.smithi115.stdout:1/636: truncate d7/dc/dd/d3e/d58/f88 37933 0 2021-11-03T17:14:22.794 INFO:tasks.workunit.client.1.smithi115.stdout:1/637: chown d7/dc/dd/fc7 78167034 1 2021-11-03T17:14:22.795 INFO:tasks.workunit.client.1.smithi115.stdout:1/638: unlink d7/d38/f5c 0 2021-11-03T17:14:22.796 INFO:tasks.workunit.client.1.smithi115.stdout:1/639: dread d7/dc/dd/d3e/dd1/d4d/d60/f6d [0,4194304] 0 2021-11-03T17:14:22.798 INFO:tasks.workunit.client.1.smithi115.stdout:1/640: unlink d7/dc/dd/fba 0 2021-11-03T17:14:22.800 INFO:tasks.workunit.client.1.smithi115.stdout:1/641: creat d7/d38/fd4 x:0 0 0 2021-11-03T17:14:22.800 INFO:tasks.workunit.client.1.smithi115.stdout:1/642: mknod d7/dc/dd/db4/cd5 1 2021-11-03T17:14:22.800 INFO:tasks.workunit.client.1.smithi115.stdout:1/643: write d7/f8 [4842735,14852] 0 2021-11-03T17:14:22.801 INFO:tasks.workunit.client.1.smithi115.stdout:1/644: stat d7/dc/d1a/f35 0 2021-11-03T17:14:22.801 INFO:tasks.workunit.client.1.smithi115.stdout:1/645: read d7/dc/dd/d3e/fa1 [3583666,13303] 0 2021-11-03T17:14:22.801 INFO:tasks.workunit.client.1.smithi115.stdout:1/646: mknod d7/dc/dab/d73/cd6 1 2021-11-03T17:14:22.801 INFO:tasks.workunit.client.1.smithi115.stdout:1/647: stat d7/dc/dd/d3e/dd1/f2b 0 2021-11-03T17:14:22.802 INFO:tasks.workunit.client.1.smithi115.stdout:1/648: read - d7/d38/d59/f9b zero size 2021-11-03T17:14:22.802 INFO:tasks.workunit.client.1.smithi115.stdout:1/649: dread - d7/dc/dd/d3e/dd1/d4b/d55/f97 zero size 2021-11-03T17:14:22.802 INFO:tasks.workunit.client.1.smithi115.stdout:1/650: chown d7/f90 125777 1 2021-11-03T17:14:22.803 INFO:tasks.workunit.client.1.smithi115.stdout:1/651: unlink d7/dc/dd/d3e/dd1/d4b/d55/f93 0 2021-11-03T17:14:22.803 INFO:tasks.workunit.client.1.smithi115.stdout:1/652: creat d7/dc/d1a/fd7 x:0 0 0 2021-11-03T17:14:22.803 INFO:tasks.workunit.client.1.smithi115.stdout:1/653: link d7/dc/dd/d3e/dd1/d4d/f7b d7/dc/dd/d3e/d58/dbf/fd8 0 2021-11-03T17:14:22.804 INFO:tasks.workunit.client.1.smithi115.stdout:1/654: creat d7/d38/d84/dca/fd9 x:0 0 0 2021-11-03T17:14:22.805 INFO:tasks.workunit.client.1.smithi115.stdout:1/655: getdents d7/dc/dd/d3e/dd1/d4d 0 2021-11-03T17:14:22.805 INFO:tasks.workunit.client.1.smithi115.stdout:1/656: creat d7/d9d/fda x:0 0 0 2021-11-03T17:14:22.806 INFO:tasks.workunit.client.1.smithi115.stdout:1/657: fdatasync d7/d38/d59/f9b 0 2021-11-03T17:14:22.820 INFO:tasks.workunit.client.1.smithi115.stdout:1/658: rmdir d7/d38 39 2021-11-03T17:14:22.820 INFO:tasks.workunit.client.1.smithi115.stdout:1/659: mknod d7/dc/dd/d3e/dd1/d26/cdb 1 2021-11-03T17:14:22.821 INFO:tasks.workunit.client.1.smithi115.stdout:1/660: stat d7/dc/dd/d42/f47 0 2021-11-03T17:14:22.821 INFO:tasks.workunit.client.1.smithi115.stdout:1/661: truncate d7/dc/dab/d73/fa9 2822960 0 2021-11-03T17:14:22.835 INFO:tasks.workunit.client.1.smithi115.stdout:1/662: dwrite d7/dc/d1a/fd7 [0,4194304] 0 2021-11-03T17:14:22.835 INFO:tasks.workunit.client.1.smithi115.stdout:1/663: stat d7/dc/dd/d42/lcf 0 2021-11-03T17:14:22.837 INFO:tasks.workunit.client.1.smithi115.stdout:1/664: dread d7/dc/d1a/f6b [0,4194304] 0 2021-11-03T17:14:22.854 INFO:tasks.workunit.client.1.smithi115.stdout:1/665: dwrite d7/d38/d84/f9a [0,4194304] 0 2021-11-03T17:14:22.854 INFO:tasks.workunit.client.1.smithi115.stdout:1/666: write d7/dc/dd/d3e/dd1/d43/d78/fb1 [76391,115220] 0 2021-11-03T17:14:22.855 INFO:tasks.workunit.client.1.smithi115.stdout:1/667: chown d7/f7c 111415162 1 2021-11-03T17:14:22.855 INFO:tasks.workunit.client.1.smithi115.stdout:1/668: dread - d7/dc/dd/d3e/dd1/d4b/d55/f97 zero size 2021-11-03T17:14:22.936 INFO:tasks.workunit.client.1.smithi115.stdout:2/568: sync 2021-11-03T17:14:22.938 INFO:tasks.workunit.client.1.smithi115.stdout:2/569: mkdir d4/d6/dad/dc2 0 2021-11-03T17:14:22.939 INFO:tasks.workunit.client.1.smithi115.stdout:2/570: write d4/d6/d56/d6c/d34/d43/d63/d77/fa6 [1483990,70371] 0 2021-11-03T17:14:22.948 INFO:tasks.workunit.client.0.smithi027.stdout:5/734: sync 2021-11-03T17:14:22.948 INFO:tasks.workunit.client.0.smithi027.stdout:7/342: sync 2021-11-03T17:14:22.950 INFO:tasks.workunit.client.0.smithi027.stdout:5/735: write d2/d4/f34 [480835,50276] 0 2021-11-03T17:14:22.961 INFO:tasks.workunit.client.0.smithi027.stdout:5/736: dread d2/d2c/d3c/d58/fc7 [0,4194304] 0 2021-11-03T17:14:22.963 INFO:tasks.workunit.client.0.smithi027.stdout:5/737: symlink d2/d2c/d9b/db3/lf0 0 2021-11-03T17:14:22.965 INFO:tasks.workunit.client.0.smithi027.stdout:7/343: dwrite d0/d2/d1c/f2b [0,4194304] 0 2021-11-03T17:14:22.966 INFO:tasks.workunit.client.0.smithi027.stdout:5/738: unlink d2/d4/d4e/d67/fb7 0 2021-11-03T17:14:22.967 INFO:tasks.workunit.client.0.smithi027.stdout:7/344: write d0/f16 [1127621,38782] 0 2021-11-03T17:14:22.968 INFO:tasks.workunit.client.0.smithi027.stdout:7/345: stat d0/d2/fd 0 2021-11-03T17:14:22.969 INFO:tasks.workunit.client.0.smithi027.stdout:5/739: symlink d2/d2c/d3c/lf1 0 2021-11-03T17:14:22.969 INFO:tasks.workunit.client.0.smithi027.stdout:5/740: readlink d2/d4/l5f 0 2021-11-03T17:14:22.970 INFO:tasks.workunit.client.0.smithi027.stdout:5/741: write d2/d12/d7d/da7/fac [5091731,13793] 0 2021-11-03T17:14:22.970 INFO:tasks.workunit.client.0.smithi027.stdout:5/742: write d2/d12/dad/f71 [3910251,15264] 0 2021-11-03T17:14:22.971 INFO:tasks.workunit.client.0.smithi027.stdout:7/346: mkdir d0/d2/d10/d1e/d20/d74 0 2021-11-03T17:14:22.971 INFO:tasks.workunit.client.0.smithi027.stdout:7/347: read d0/d2/d5e/f32 [397486,86031] 0 2021-11-03T17:14:22.980 INFO:tasks.workunit.client.0.smithi027.stdout:5/743: dread d2/d2c/d3c/d48/d65/d80/d9e/fbd [0,4194304] 0 2021-11-03T17:14:22.980 INFO:tasks.workunit.client.0.smithi027.stdout:5/744: fdatasync d2/d12/f9a 0 2021-11-03T17:14:22.981 INFO:tasks.workunit.client.0.smithi027.stdout:5/745: dread d2/d12/f5c [0,4194304] 0 2021-11-03T17:14:22.982 INFO:tasks.workunit.client.0.smithi027.stdout:5/746: symlink d2/d2c/d9b/db3/lf2 0 2021-11-03T17:14:22.982 INFO:tasks.workunit.client.0.smithi027.stdout:5/747: chown d2/d4/d4e/d67/d8e 1 1 2021-11-03T17:14:22.983 INFO:tasks.workunit.client.0.smithi027.stdout:5/748: creat d2/d1f/ff3 x:0 0 0 2021-11-03T17:14:22.983 INFO:tasks.workunit.client.0.smithi027.stdout:5/749: fsync d2/d4/d4e/d67/f7f 0 2021-11-03T17:14:22.990 INFO:tasks.workunit.client.0.smithi027.stdout:7/348: dwrite d0/f18 [0,4194304] 0 2021-11-03T17:14:22.993 INFO:tasks.workunit.client.0.smithi027.stdout:7/349: mkdir d0/d2/d5e/d75 0 2021-11-03T17:14:22.994 INFO:tasks.workunit.client.0.smithi027.stdout:7/350: symlink d0/d2/d10/d1e/d20/l76 0 2021-11-03T17:14:22.994 INFO:tasks.workunit.client.0.smithi027.stdout:7/351: mknod d0/d2/d10/d1e/d20/c77 1 2021-11-03T17:14:23.000 INFO:tasks.workunit.client.0.smithi027.stdout:5/750: dwrite d2/d2c/d3c/f98 [0,4194304] 0 2021-11-03T17:14:23.001 INFO:tasks.workunit.client.0.smithi027.stdout:5/751: write d2/d2c/d9b/fcd [3277809,95938] 0 2021-11-03T17:14:23.004 INFO:tasks.workunit.client.0.smithi027.stdout:7/352: dread d0/d2/d10/d1e/d20/f26 [0,4194304] 0 2021-11-03T17:14:23.004 INFO:tasks.workunit.client.0.smithi027.stdout:7/353: fsync d0/d2/d5e/f32 0 2021-11-03T17:14:23.004 INFO:tasks.workunit.client.0.smithi027.stdout:7/354: mknod d0/d2/d5e/d36/c78 1 2021-11-03T17:14:23.005 INFO:tasks.workunit.client.0.smithi027.stdout:5/752: creat d2/d2c/d3c/d48/d65/d80/d9e/ff4 x:0 0 0 2021-11-03T17:14:23.005 INFO:tasks.workunit.client.0.smithi027.stdout:5/753: dread - d2/d2c/d3c/fdd zero size 2021-11-03T17:14:23.006 INFO:tasks.workunit.client.0.smithi027.stdout:5/754: read d2/d2c/d3c/d48/d65/d80/f83 [153236,48316] 0 2021-11-03T17:14:23.007 INFO:tasks.workunit.client.0.smithi027.stdout:7/355: unlink d0/d2/d10/d1e/f48 0 2021-11-03T17:14:23.009 INFO:tasks.workunit.client.0.smithi027.stdout:7/356: getdents d0/d2/d5e/d36 0 2021-11-03T17:14:23.013 INFO:tasks.workunit.client.0.smithi027.stdout:5/755: dread d2/d1f/f6f [0,4194304] 0 2021-11-03T17:14:23.016 INFO:tasks.workunit.client.0.smithi027.stdout:5/756: symlink d2/d12/dad/da8/lf5 0 2021-11-03T17:14:23.018 INFO:tasks.workunit.client.0.smithi027.stdout:7/357: dread d0/d2/d1c/f27 [0,4194304] 0 2021-11-03T17:14:23.019 INFO:tasks.workunit.client.0.smithi027.stdout:5/757: rename d2/d4/d4e/d67/f78 to d2/d2c/d3c/d48/d65/d80/d9e/ff6 0 2021-11-03T17:14:23.035 INFO:tasks.workunit.client.0.smithi027.stdout:5/758: dwrite d2/d12/dad/da9/fe4 [0,4194304] 0 2021-11-03T17:14:23.036 INFO:tasks.workunit.client.0.smithi027.stdout:5/759: stat d2/d2c/d3c/d58/fef 0 2021-11-03T17:14:23.039 INFO:tasks.workunit.client.0.smithi027.stdout:7/358: dwrite d0/d2/f5 [0,4194304] 0 2021-11-03T17:14:23.052 INFO:tasks.workunit.client.0.smithi027.stdout:7/359: truncate d0/d2/f5 1057359 0 2021-11-03T17:14:23.052 INFO:tasks.workunit.client.0.smithi027.stdout:7/360: fsync d0/d2/d5e/d36/f63 0 2021-11-03T17:14:23.053 INFO:tasks.workunit.client.0.smithi027.stdout:7/361: creat d0/d2/d1c/f79 x:0 0 0 2021-11-03T17:14:23.054 INFO:tasks.workunit.client.0.smithi027.stdout:7/362: write d0/d2/d10/d1e/d20/f5b [433120,73216] 0 2021-11-03T17:14:23.056 INFO:tasks.workunit.client.0.smithi027.stdout:7/363: creat d0/d2/d10/d35/f7a x:0 0 0 2021-11-03T17:14:23.057 INFO:tasks.workunit.client.0.smithi027.stdout:7/364: chown d0/d2/d10/l2e 11511658 1 2021-11-03T17:14:23.058 INFO:tasks.workunit.client.0.smithi027.stdout:7/365: creat d0/d2/d10/d1e/d20/d74/f7b x:0 0 0 2021-11-03T17:14:23.059 INFO:tasks.workunit.client.0.smithi027.stdout:7/366: rmdir d0/d2d/d58 39 2021-11-03T17:14:23.061 INFO:tasks.workunit.client.0.smithi027.stdout:7/367: creat d0/d2d/d2f/f7c x:0 0 0 2021-11-03T17:14:23.061 INFO:tasks.workunit.client.0.smithi027.stdout:7/368: mknod d0/d2/d10/d1e/d20/c7d 1 2021-11-03T17:14:23.062 INFO:tasks.workunit.client.0.smithi027.stdout:7/369: write d0/d2/d10/d1e/d20/d74/f7b [535848,50585] 0 2021-11-03T17:14:23.062 INFO:tasks.workunit.client.0.smithi027.stdout:7/370: symlink d0/d2/d10/d1e/d20/d74/l7e 0 2021-11-03T17:14:23.063 INFO:tasks.workunit.client.0.smithi027.stdout:5/760: dwrite d2/d4/d4e/d67/d20/f3a [0,4194304] 0 2021-11-03T17:14:23.064 INFO:tasks.workunit.client.0.smithi027.stdout:7/371: unlink d0/d2d/d2f/f7c 0 2021-11-03T17:14:23.066 INFO:tasks.workunit.client.0.smithi027.stdout:5/761: symlink d2/d12/dad/da8/dd9/dde/lf7 0 2021-11-03T17:14:23.068 INFO:tasks.workunit.client.0.smithi027.stdout:7/372: creat d0/d2d/d58/f7f x:0 0 0 2021-11-03T17:14:23.068 INFO:tasks.workunit.client.0.smithi027.stdout:7/373: chown d0/d2/d5e/d36/l71 34341 1 2021-11-03T17:14:23.069 INFO:tasks.workunit.client.0.smithi027.stdout:5/762: creat d2/d12/d7d/da7/ff8 x:0 0 0 2021-11-03T17:14:23.074 INFO:tasks.workunit.client.0.smithi027.stdout:5/763: dread d2/d4/d4e/d67/f2e [0,4194304] 0 2021-11-03T17:14:23.076 INFO:tasks.workunit.client.0.smithi027.stdout:5/764: creat d2/d2c/d3c/d48/d65/d80/dd2/ff9 x:0 0 0 2021-11-03T17:14:23.077 INFO:tasks.workunit.client.0.smithi027.stdout:5/765: fdatasync d2/d4/d4e/d67/f7f 0 2021-11-03T17:14:23.078 INFO:tasks.workunit.client.0.smithi027.stdout:7/374: dread d0/f16 [0,4194304] 0 2021-11-03T17:14:23.078 INFO:tasks.workunit.client.0.smithi027.stdout:7/375: stat d0/d2/d10/d1e/d20/f43 0 2021-11-03T17:14:23.078 INFO:tasks.workunit.client.0.smithi027.stdout:7/376: write d0/d2/d10/d1e/d20/f5b [1453389,104812] 0 2021-11-03T17:14:23.079 INFO:tasks.workunit.client.0.smithi027.stdout:5/766: rename d2/d12/d7d/l95 to d2/d4/d4e/d67/d8e/lfa 0 2021-11-03T17:14:23.081 INFO:tasks.workunit.client.0.smithi027.stdout:7/377: symlink d0/d2/d4f/l80 0 2021-11-03T17:14:23.081 INFO:tasks.workunit.client.0.smithi027.stdout:7/378: fsync d0/d2/f5 0 2021-11-03T17:14:23.082 INFO:tasks.workunit.client.0.smithi027.stdout:7/379: fdatasync d0/d2/d5e/f32 0 2021-11-03T17:14:23.082 INFO:tasks.workunit.client.0.smithi027.stdout:7/380: truncate d0/d2d/d58/f7f 714485 0 2021-11-03T17:14:23.083 INFO:tasks.workunit.client.0.smithi027.stdout:7/381: link d0/d2/d5e/l6b d0/d2/l81 0 2021-11-03T17:14:23.087 INFO:tasks.workunit.client.0.smithi027.stdout:7/382: mkdir d0/d2/d10/d82 0 2021-11-03T17:14:23.087 INFO:tasks.workunit.client.0.smithi027.stdout:7/383: mknod d0/d2/d10/d1e/d20/d74/c83 1 2021-11-03T17:14:23.095 INFO:tasks.workunit.client.0.smithi027.stdout:7/384: rename d0/d2/d4f to d0/d2/d1c/d4d/d6e/d84 0 2021-11-03T17:14:23.095 INFO:tasks.workunit.client.0.smithi027.stdout:7/385: mknod d0/d2/d10/d1e/c85 1 2021-11-03T17:14:23.113 INFO:tasks.workunit.client.0.smithi027.stdout:7/386: dwrite d0/d2/d1f/f46 [0,4194304] 0 2021-11-03T17:14:23.116 INFO:tasks.workunit.client.0.smithi027.stdout:7/387: rmdir d0/d2/d5e 39 2021-11-03T17:14:23.116 INFO:tasks.workunit.client.0.smithi027.stdout:7/388: mknod d0/d2/d1f/c86 1 2021-11-03T17:14:23.116 INFO:tasks.workunit.client.0.smithi027.stdout:7/389: chown d0/d2/d5e/f32 63 1 2021-11-03T17:14:23.117 INFO:tasks.workunit.client.0.smithi027.stdout:7/390: stat d0/d2/l1a 0 2021-11-03T17:14:23.117 INFO:tasks.workunit.client.0.smithi027.stdout:7/391: truncate d0/d2/d5e/d36/f64 1026829 0 2021-11-03T17:14:23.119 INFO:tasks.workunit.client.0.smithi027.stdout:7/392: symlink d0/d2/d1c/d4d/d6e/d84/l87 0 2021-11-03T17:14:23.120 INFO:tasks.workunit.client.0.smithi027.stdout:7/393: chown d0/l1 11 1 2021-11-03T17:14:23.120 INFO:tasks.workunit.client.0.smithi027.stdout:7/394: stat d0/d2/d10/d1e/l21 0 2021-11-03T17:14:23.120 INFO:tasks.workunit.client.0.smithi027.stdout:7/395: chown d0/d2/d10/l72 1429 1 2021-11-03T17:14:23.121 INFO:tasks.workunit.client.0.smithi027.stdout:7/396: dread d0/d2d/d2f/f39 [0,4194304] 0 2021-11-03T17:14:23.123 INFO:tasks.workunit.client.0.smithi027.stdout:7/397: creat d0/d2/d10/d1e/f88 x:0 0 0 2021-11-03T17:14:23.124 INFO:tasks.workunit.client.0.smithi027.stdout:7/398: write d0/f16 [2203751,50638] 0 2021-11-03T17:14:23.127 INFO:tasks.workunit.client.0.smithi027.stdout:7/399: rename d0/d2/d10/d35/f49 to d0/d2/d10/d35/f89 0 2021-11-03T17:14:23.127 INFO:tasks.workunit.client.0.smithi027.stdout:7/400: write d0/d2/d10/d1e/f88 [211948,15678] 0 2021-11-03T17:14:23.128 INFO:tasks.workunit.client.0.smithi027.stdout:7/401: write d0/f25 [218823,89544] 0 2021-11-03T17:14:23.129 INFO:tasks.workunit.client.0.smithi027.stdout:7/402: chown d0/d2/d10/d1e/f88 122 1 2021-11-03T17:14:23.129 INFO:tasks.workunit.client.0.smithi027.stdout:7/403: mknod d0/d2/d10/d35/c8a 1 2021-11-03T17:14:23.129 INFO:tasks.workunit.client.0.smithi027.stdout:7/404: chown d0/d2d/d53 123370 1 2021-11-03T17:14:23.139 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:22 smithi115 conmon[34670]: audit 2021-11-03T17:14:21.800466+0000 mon.smithi027 (mon.0) 575 : audit [INF] 2021-11-03T17:14:23.140 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:22 smithi115 conmon[34670]: from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:14:23.140 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:22 smithi115 conmon[34670]: audit 2021-11-03T17:14:22.177037+0000 mon.smithi027 (mon.0) 576 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:14:23.140 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:22 smithi115 conmon[34670]: audit 2021-11-03T17:14:22.186061+0000 mon.smithi027 (mon.0) 577 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:14:23.141 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:22 smithi115 conmon[34670]: cephadm 2021-11-03T17:14:22.187565+0000 mgr.smithi027.ufhhvb (mgr.14162) 312 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.smithi027.ufhhvb) 2021-11-03T17:14:23.141 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:22 smithi115 conmon[34670]: cephadm 2021-11-03T17:14:22.193476+0000 mgr.smithi027.ufhhvb (mgr.14162) 313 : cephadm [INF] Failing over to other MGR 2021-11-03T17:14:23.141 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:22 smithi115 conmon[34670]: audit 2021-11-03T17:14:22.195830+0000 mon.smithi027 (mon.0) 578 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mgr fail", "who": "smithi027.ufhhvb"}]: dispatch 2021-11-03T17:14:23.141 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:22 smithi115 conmon[34670]: cluster 2021-11-03T17:14:22.224511+0000 mon.smithi027 (mon.0) 579 : cluster [DBG] osdmap e45: 6 total, 6 up, 6 in 2021-11-03T17:14:23.141 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:22 smithi115 conmon[34670]: cluster 2021-11-03T17:14:22.607605+0000 mgr.smithi027.ufhhvb (mgr.14162) 314 : cluster [DBG] pgmap v259: 65 pgs: 65 active+clean; 2.3 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 81 MiB/s rd, 177 MiB/s wr, 275 op/s 2021-11-03T17:14:23.142 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:22 smithi115 conmon[34670]: cluster 2021-11-03T17:14:22.728351+0000 mon.smithi027 (mon.0) 580 : cluster [DBG] 2021-11-03T17:14:23.142 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:22 smithi115 conmon[34670]: Standby manager daemon smithi027.ufhhvb started 2021-11-03T17:14:23.146 INFO:tasks.workunit.client.0.smithi027.stdout:7/405: dwrite d0/d2/d1f/f46 [0,4194304] 0 2021-11-03T17:14:23.150 INFO:tasks.workunit.client.0.smithi027.stdout:7/406: unlink d0/d2/f6c 0 2021-11-03T17:14:23.150 INFO:tasks.workunit.client.0.smithi027.stdout:7/407: readlink d0/d2/d1f/l40 0 2021-11-03T17:14:23.152 INFO:tasks.workunit.client.0.smithi027.stdout:7/408: mkdir d0/d2/d1c/d4d/d6e/d8b 0 2021-11-03T17:14:23.152 INFO:tasks.workunit.client.0.smithi027.stdout:7/409: dread - d0/d2/d10/d1e/f24 zero size 2021-11-03T17:14:23.162 INFO:tasks.workunit.client.0.smithi027.stdout:7/410: truncate d0/d2/d10/f1b 3314991 0 2021-11-03T17:14:23.170 INFO:tasks.workunit.client.0.smithi027.stdout:7/411: unlink d0/d2/d10/d1e/d20/f5f 0 2021-11-03T17:14:23.171 INFO:tasks.workunit.client.0.smithi027.stdout:7/412: write d0/d2/d10/d35/f89 [802463,71312] 0 2021-11-03T17:14:23.171 INFO:tasks.workunit.client.0.smithi027.stdout:7/413: write d0/d2d/d66/f70 [439258,84913] 0 2021-11-03T17:14:23.171 INFO:tasks.workunit.client.0.smithi027.stdout:7/414: chown d0/d2/d10/d35/l38 7817590 1 2021-11-03T17:14:23.172 INFO:tasks.workunit.client.0.smithi027.stdout:7/415: write d0/d2/d10/d1e/d20/f5b [2508504,99770] 0 2021-11-03T17:14:23.173 INFO:tasks.workunit.client.0.smithi027.stdout:7/416: symlink d0/d2/d1c/l8c 0 2021-11-03T17:14:23.174 INFO:tasks.workunit.client.0.smithi027.stdout:7/417: mknod d0/d2/d1c/d4d/d6e/c8d 1 2021-11-03T17:14:23.175 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:22 smithi027 conmon[32322]: audit 2021-11-03T17:14:21.800466+0000 mon.smithi027 (mon.0) 575 2021-11-03T17:14:23.175 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:22 smithi027 conmon[32322]: : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:14:23.176 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:22 smithi027 conmon[32322]: audit 2021-11-03T17:14:22.177037+0000 mon.smithi027 (mon.0) 576 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:14:23.176 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:22 smithi027 conmon[32322]: audit 2021-11-03T17:14:22.186061+0000 mon.smithi027 (mon. 2021-11-03T17:14:23.177 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:22 smithi027 conmon[32322]: 0) 577 : audit [DBG] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:14:23.178 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:22 smithi027 conmon[32322]: cephadm 2021-11-03T17:14:22. 2021-11-03T17:14:23.178 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:22 smithi027 conmon[32322]: 187565+0000 mgr.smithi027.ufhhvb (mgr.14162) 312 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.smithi027.ufhhvb) 2021-11-03T17:14:23.179 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:22 smithi027 conmon[32322]: cephadm 2021-11 2021-11-03T17:14:23.179 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:22 smithi027 conmon[32322]: -03T17:14:22.193476+0000 mgr.smithi027.ufhhvb (mgr.14162) 313 : cephadm [INF] Failing over to other MGR 2021-11-03T17:14:23.180 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:22 smithi027 conmon[32322]: audit 2021-11-03T17:14:22. 2021-11-03T17:14:23.180 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:22 smithi027 conmon[32322]: 195830+0000 mon.smithi027 (mon.0) 578 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mgr fail", "who": "smithi027.ufhhvb"}]: dispatch 2021-11-03T17:14:23.180 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:22 smithi027 conmon[32322]: cluster 2021-11-03T17:14:22.224511 2021-11-03T17:14:23.180 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:22 smithi027 conmon[32322]: +0000 mon.smithi027 (mon.0) 579 : cluster [DBG] osdmap e45: 6 total, 6 up, 6 in 2021-11-03T17:14:23.180 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:22 smithi027 conmon[32322]: cluster 2021-11-03T17:14:22.607605+0000 2021-11-03T17:14:23.181 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:22 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14162) 314 : cluster [DBG] pgmap v259: 65 pgs: 65 active+clean; 2.3 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 81 MiB/s rd, 177 MiB/s wr, 275 op/s 2021-11-03T17:14:23.181 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:22 smithi027 conmon[32322]: cluster 2021-11-03T17:14:22.728351+0000 mon.smithi027 2021-11-03T17:14:23.181 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:22 smithi027 conmon[32322]: (mon.0) 580 : cluster [DBG] Standby manager daemon smithi027.ufhhvb started 2021-11-03T17:14:23.182 INFO:tasks.workunit.client.0.smithi027.stdout:7/418: rename d0/l44 to d0/d2/d1c/d4d/d6e/d8b/l8e 0 2021-11-03T17:14:23.182 INFO:tasks.workunit.client.0.smithi027.stdout:7/419: mknod d0/d2d/d2f/c8f 1 2021-11-03T17:14:23.183 INFO:tasks.workunit.client.0.smithi027.stdout:7/420: mknod d0/d2/d1c/d4d/c90 1 2021-11-03T17:14:23.193 INFO:tasks.workunit.client.0.smithi027.stdout:7/421: dread d0/d2/d1c/f61 [0,4194304] 0 2021-11-03T17:14:23.193 INFO:tasks.workunit.client.0.smithi027.stdout:7/422: mknod d0/d2/d10/d1e/d20/d74/c91 1 2021-11-03T17:14:23.194 INFO:tasks.workunit.client.0.smithi027.stdout:7/423: creat d0/d2/d1c/d4d/d6e/d84/f92 x:0 0 0 2021-11-03T17:14:23.194 INFO:tasks.workunit.client.0.smithi027.stdout:7/424: mknod d0/d2/d10/d1e/c93 1 2021-11-03T17:14:23.196 INFO:tasks.workunit.client.0.smithi027.stdout:7/425: link d0/l34 d0/d2/d1c/d4d/l94 0 2021-11-03T17:14:23.211 INFO:tasks.workunit.client.0.smithi027.stdout:7/426: dwrite d0/d2d/d58/f6d [0,4194304] 0 2021-11-03T17:14:23.213 INFO:tasks.workunit.client.0.smithi027.stdout:7/427: creat d0/d2/d10/f95 x:0 0 0 2021-11-03T17:14:23.215 INFO:tasks.workunit.client.0.smithi027.stdout:7/428: mkdir d0/d2/d10/d35/d96 0 2021-11-03T17:14:23.216 INFO:tasks.workunit.client.0.smithi027.stdout:7/429: dread d0/d2/d10/d1e/f88 [0,4194304] 0 2021-11-03T17:14:23.216 INFO:tasks.workunit.client.0.smithi027.stdout:7/430: chown d0/d2/d10/l72 87 1 2021-11-03T17:14:23.220 INFO:tasks.workunit.client.0.smithi027.stdout:7/431: creat d0/d2/d1c/f97 x:0 0 0 2021-11-03T17:14:23.221 INFO:tasks.workunit.client.0.smithi027.stdout:7/432: chown d0/d2 55454627 1 2021-11-03T17:14:23.222 INFO:tasks.workunit.client.0.smithi027.stdout:7/433: creat d0/d2d/d66/f98 x:0 0 0 2021-11-03T17:14:23.235 INFO:tasks.workunit.client.0.smithi027.stdout:7/434: dread d0/d2/d10/d35/f89 [0,4194304] 0 2021-11-03T17:14:23.248 INFO:tasks.workunit.client.0.smithi027.stdout:7/435: dread d0/d2/d10/f1b [0,4194304] 0 2021-11-03T17:14:23.249 INFO:tasks.workunit.client.0.smithi027.stdout:7/436: creat d0/d2/d10/d1e/f99 x:0 0 0 2021-11-03T17:14:23.737 INFO:tasks.workunit.client.1.smithi115.stdout:9/484: sync 2021-11-03T17:14:23.738 INFO:tasks.workunit.client.1.smithi115.stdout:9/485: mknod d1/d18/d2b/d42/cab 1 2021-11-03T17:14:23.744 INFO:tasks.workunit.client.1.smithi115.stdout:9/486: write d1/d1a/d48/d27/d34/f9c [1688115,93121] 0 2021-11-03T17:14:23.744 INFO:tasks.workunit.client.1.smithi115.stdout:9/487: chown d1/d1a/l5f 4048 1 2021-11-03T17:14:23.744 INFO:tasks.workunit.client.1.smithi115.stdout:9/488: fsync d1/d1a/d48/f2a 0 2021-11-03T17:14:23.744 INFO:tasks.workunit.client.1.smithi115.stdout:9/489: fdatasync d1/d1a/d48/f73 0 2021-11-03T17:14:23.745 INFO:tasks.workunit.client.0.smithi027.stdout:6/539: sync 2021-11-03T17:14:23.747 INFO:tasks.workunit.client.0.smithi027.stdout:6/540: mkdir dc/d82/d57/d9b 0 2021-11-03T17:14:23.750 INFO:tasks.workunit.client.1.smithi115.stdout:9/490: dread d1/d18/d4a/d74/f7e [0,4194304] 0 2021-11-03T17:14:23.750 INFO:tasks.workunit.client.1.smithi115.stdout:9/491: chown d1/d8a 58 1 2021-11-03T17:14:23.750 INFO:tasks.workunit.client.1.smithi115.stdout:9/492: mknod d1/d18/cac 1 2021-11-03T17:14:23.752 INFO:tasks.workunit.client.1.smithi115.stdout:9/493: creat d1/d1a/d48/d27/da9/d79/fad x:0 0 0 2021-11-03T17:14:23.752 INFO:tasks.workunit.client.1.smithi115.stdout:9/494: fsync d1/d18/d2b/d42/d6e/f5c 0 2021-11-03T17:14:23.753 INFO:tasks.workunit.client.1.smithi115.stdout:9/495: creat d1/fae x:0 0 0 2021-11-03T17:14:23.754 INFO:tasks.workunit.client.1.smithi115.stdout:9/496: creat d1/d18/d65/faf x:0 0 0 2021-11-03T17:14:23.754 INFO:tasks.workunit.client.1.smithi115.stdout:9/497: getdents d1/d1a/d48/d27/d53/d83/d88/d9e 0 2021-11-03T17:14:23.756 INFO:tasks.workunit.client.1.smithi115.stdout:9/498: symlink d1/d18/d2b/d45/lb0 0 2021-11-03T17:14:23.756 INFO:tasks.workunit.client.1.smithi115.stdout:9/499: symlink d1/d1a/d48/d27/da9/lb1 0 2021-11-03T17:14:23.767 INFO:tasks.workunit.client.0.smithi027.stdout:6/541: dwrite dc/d16/d1b/d24/f85 [0,4194304] 0 2021-11-03T17:14:23.769 INFO:tasks.workunit.client.0.smithi027.stdout:6/542: symlink dc/d16/d1b/l9c 0 2021-11-03T17:14:23.770 INFO:tasks.workunit.client.0.smithi027.stdout:6/543: creat dc/d16/d1b/d2d/d34/d89/f9d x:0 0 0 2021-11-03T17:14:23.771 INFO:tasks.workunit.client.0.smithi027.stdout:6/544: chown dc/d82/d57/l59 397206 1 2021-11-03T17:14:23.771 INFO:tasks.workunit.client.1.smithi115.stdout:9/500: dread d1/d1a/f28 [0,4194304] 0 2021-11-03T17:14:23.772 INFO:tasks.workunit.client.1.smithi115.stdout:9/501: symlink d1/d66/lb2 0 2021-11-03T17:14:23.773 INFO:tasks.workunit.client.1.smithi115.stdout:9/502: symlink d1/d1a/d48/d27/d53/d83/d88/d9e/d58/lb3 0 2021-11-03T17:14:23.774 INFO:tasks.workunit.client.1.smithi115.stdout:9/503: read d1/d18/d4a/d74/f7e [298423,38891] 0 2021-11-03T17:14:23.775 INFO:tasks.workunit.client.1.smithi115.stdout:9/504: mknod d1/d18/d2b/cb4 1 2021-11-03T17:14:23.775 INFO:tasks.workunit.client.1.smithi115.stdout:9/505: write d1/f72 [188008,93700] 0 2021-11-03T17:14:23.776 INFO:tasks.workunit.client.1.smithi115.stdout:9/506: rename d1/d1a/d48/d27/d34 to d1/d18/d2b/d42/d6e/db5 0 2021-11-03T17:14:23.777 INFO:tasks.workunit.client.1.smithi115.stdout:9/507: readlink d1/l67 0 2021-11-03T17:14:23.777 INFO:tasks.workunit.client.1.smithi115.stdout:9/508: symlink d1/d18/d4a/lb6 0 2021-11-03T17:14:23.782 INFO:tasks.workunit.client.1.smithi115.stdout:9/509: rename d1/d1a/d48/d27/d2f/d41/f7d to d1/d18/d2b/d42/fb7 0 2021-11-03T17:14:23.783 INFO:tasks.workunit.client.1.smithi115.stdout:9/510: symlink d1/d1a/d48/d27/d2f/lb8 0 2021-11-03T17:14:23.784 INFO:tasks.workunit.client.1.smithi115.stdout:9/511: mkdir d1/d18/d2b/db9 0 2021-11-03T17:14:23.785 INFO:tasks.workunit.client.1.smithi115.stdout:9/512: getdents d1/d1a/d48/d27/d2f/d6a 0 2021-11-03T17:14:23.787 INFO:tasks.workunit.client.1.smithi115.stdout:9/513: rename d1/d18/d2b/d42/d6e/db5/f9c to d1/d1a/d48/d27/da9/fba 0 2021-11-03T17:14:23.794 INFO:tasks.workunit.client.0.smithi027.stdout:6/545: dwrite dc/d16/d30/f61 [0,4194304] 0 2021-11-03T17:14:23.794 INFO:tasks.workunit.client.0.smithi027.stdout:6/546: dread - dc/d16/d1b/d3c/d51/f73 zero size 2021-11-03T17:14:23.794 INFO:tasks.workunit.client.0.smithi027.stdout:6/547: read dc/d16/d1b/d3c/d51/f5b [113693,94544] 0 2021-11-03T17:14:23.795 INFO:tasks.workunit.client.0.smithi027.stdout:6/548: write dc/f88 [68894,265] 0 2021-11-03T17:14:23.797 INFO:tasks.workunit.client.0.smithi027.stdout:6/549: creat dc/d16/d1b/d3c/f9e x:0 0 0 2021-11-03T17:14:23.805 INFO:tasks.workunit.client.1.smithi115.stdout:9/514: dwrite d1/d1a/d48/d27/d53/f77 [0,4194304] 0 2021-11-03T17:14:23.805 INFO:tasks.workunit.client.1.smithi115.stdout:9/515: readlink d1/d18/d2b/d42/la0 0 2021-11-03T17:14:23.809 INFO:tasks.workunit.client.1.smithi115.stdout:9/516: rename d1/d1a/d48/d27/d2f/d41/f60 to d1/d1a/d48/d27/d53/fbb 0 2021-11-03T17:14:23.809 INFO:tasks.workunit.client.1.smithi115.stdout:9/517: chown d1/d18/d2b/d42/f21 152401299 1 2021-11-03T17:14:23.810 INFO:tasks.workunit.client.1.smithi115.stdout:9/518: mkdir d1/d18/d4a/d74/d8b/dbc 0 2021-11-03T17:14:23.814 INFO:tasks.workunit.client.1.smithi115.stdout:9/519: rename d1/d18/d2b/d42/f17 to d1/d18/d2b/d42/fbd 0 2021-11-03T17:14:23.816 INFO:tasks.workunit.client.1.smithi115.stdout:9/520: write d1/d18/d2b/d42/fbd [1274789,115660] 0 2021-11-03T17:14:23.837 INFO:tasks.workunit.client.1.smithi115.stdout:9/521: dwrite d1/d18/d4a/d74/d8b/faa [0,4194304] 0 2021-11-03T17:14:23.839 INFO:tasks.workunit.client.1.smithi115.stdout:9/522: symlink d1/d18/d2b/d42/d6e/lbe 0 2021-11-03T17:14:23.840 INFO:tasks.workunit.client.1.smithi115.stdout:9/523: readlink d1/d18/d2b/d42/l11 0 2021-11-03T17:14:23.840 INFO:tasks.workunit.client.1.smithi115.stdout:9/524: write d1/d1a/d48/d27/d53/fbb [4476437,26668] 0 2021-11-03T17:14:23.840 INFO:tasks.workunit.client.1.smithi115.stdout:9/525: symlink d1/d18/d2b/db9/lbf 0 2021-11-03T17:14:23.843 INFO:tasks.workunit.client.1.smithi115.stdout:9/526: write d1/d1a/d48/d27/d2f/d7b/f85 [696643,78045] 0 2021-11-03T17:14:23.843 INFO:tasks.workunit.client.1.smithi115.stdout:9/527: dread - d1/d1a/d48/d27/d53/f95 zero size 2021-11-03T17:14:23.843 INFO:tasks.workunit.client.1.smithi115.stdout:9/528: chown d1/d1a/d48/d27/d53/f95 0 1 2021-11-03T17:14:23.844 INFO:tasks.workunit.client.1.smithi115.stdout:9/529: creat d1/d66/fc0 x:0 0 0 2021-11-03T17:14:23.846 INFO:tasks.workunit.client.1.smithi115.stdout:9/530: creat d1/d1a/d48/d27/d53/d83/d88/d9e/d58/fc1 x:0 0 0 2021-11-03T17:14:23.847 INFO:tasks.workunit.client.1.smithi115.stdout:9/531: fsync d1/d18/d4a/d74/d8b/faa 0 2021-11-03T17:14:23.847 INFO:tasks.workunit.client.1.smithi115.stdout:9/532: chown d1/d1a/d48/d27/d2f 0 1 2021-11-03T17:14:23.847 INFO:tasks.workunit.client.1.smithi115.stdout:9/533: mkdir d1/d18/d2b/d45/dc2 0 2021-11-03T17:14:23.848 INFO:tasks.workunit.client.1.smithi115.stdout:9/534: rmdir d1/d18/d4a/d54 0 2021-11-03T17:14:23.849 INFO:tasks.workunit.client.1.smithi115.stdout:9/535: write d1/d1a/d48/d27/d2f/d41/f8c [2856397,12997] 0 2021-11-03T17:14:23.901 INFO:tasks.workunit.client.1.smithi115.stdout:9/536: creat d1/d18/d2b/db9/fc3 x:0 0 0 2021-11-03T17:14:23.901 INFO:tasks.workunit.client.1.smithi115.stdout:9/537: read d1/d18/d2b/d42/f12 [3621809,79812] 0 2021-11-03T17:14:23.902 INFO:tasks.workunit.client.1.smithi115.stdout:9/538: creat d1/d18/d4a/fc4 x:0 0 0 2021-11-03T17:14:23.903 INFO:tasks.workunit.client.1.smithi115.stdout:9/539: rename d1/d1a/d48/d27/d2f/d41/l5e to d1/d1a/d48/d27/d80/lc5 0 2021-11-03T17:14:23.904 INFO:tasks.workunit.client.1.smithi115.stdout:9/540: getdents d1/d1a/d48/d27/d53 0 2021-11-03T17:14:23.951 INFO:tasks.workunit.client.1.smithi115.stdout:9/541: getdents d1/d18/d2b/d42/d6e 0 2021-11-03T17:14:23.967 INFO:tasks.workunit.client.1.smithi115.stdout:9/542: dwrite d1/d1a/d48/d27/d53/f82 [0,4194304] 0 2021-11-03T17:14:23.980 INFO:tasks.workunit.client.1.smithi115.stdout:9/543: dwrite d1/f1d [0,4194304] 0 2021-11-03T17:14:23.982 INFO:tasks.workunit.client.1.smithi115.stdout:9/544: getdents d1/d1a/d48/d27/d2f 0 2021-11-03T17:14:23.982 INFO:tasks.workunit.client.1.smithi115.stdout:9/545: chown d1/d1a/d48/d27/d2f/lb8 20247 1 2021-11-03T17:14:23.983 INFO:tasks.workunit.client.1.smithi115.stdout:9/546: creat d1/d1a/d48/d27/d2f/fc6 x:0 0 0 2021-11-03T17:14:23.984 INFO:tasks.workunit.client.1.smithi115.stdout:9/547: mkdir d1/d18/d4a/d74/dc7 0 2021-11-03T17:14:23.984 INFO:tasks.workunit.client.1.smithi115.stdout:9/548: truncate d1/d1a/d48/d27/d53/f95 541673 0 2021-11-03T17:14:23.984 INFO:tasks.workunit.client.1.smithi115.stdout:9/549: fdatasync d1/d1a/d48/d27/f89 0 2021-11-03T17:14:23.985 INFO:tasks.workunit.client.1.smithi115.stdout:9/550: creat d1/d18/d4a/fc8 x:0 0 0 2021-11-03T17:14:23.988 INFO:tasks.workunit.client.1.smithi115.stdout:9/551: symlink d1/d18/d2b/db9/lc9 0 2021-11-03T17:14:24.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: cluster 2021-11-03T17:14:23.039364+0000 mon.smithi027 (mon.0) 581 : cluster [INF] Active manager daemon smithi115.tcyodz restarted 2021-11-03T17:14:24.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: audit 2021-11-03T17:14:23.041203+0000 mon.smithi115 (mon.1) 17 : audit [DBG] from='mgr.? 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi115.tcyodz/crt"}]: dispatch 2021-11-03T17:14:24.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: audit 2021-11-03T17:14:23.041705+0000 mon.smithi115 (mon.1) 18 : audit [DBG] from='mgr.? 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-11-03T17:14:24.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: cluster 2021-11-03T17:14:23.043099+0000 mon.smithi027 (mon.0) 582 : cluster [INF] Activating manager daemon smithi115.tcyodz 2021-11-03T17:14:24.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: audit 2021-11-03T17:14:23.043898+0000 mon.smithi115 (mon.1) 19 : audit [DBG] from='mgr.? 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi115.tcyodz/key"}]: dispatch 2021-11-03T17:14:24.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: cluster 2021-11-03T17:14:23.049369+0000 mon.smithi027 (mon.0) 583 : cluster [DBG] osdmap e46: 6 total, 6 up, 6 in 2021-11-03T17:14:24.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: audit 2021-11-03T17:14:23.050508+0000 mon.smithi115 (mon.1) 20 : audit [DBG] from='mgr.? 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-11-03T17:14:24.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: audit 2021-11-03T17:14:23.182911+0000 mon.smithi115 (mon.1) 21 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2021-11-03T17:14:24.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: audit 2021-11-03T17:14:23.183026+0000 mon.smithi115 (mon.1) 22 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2021-11-03T17:14:24.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: audit 2021-11-03T17:14:23.183793+0000 mon.smithi027 (mon.0) 584 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "mgr fail", "who": "smithi027.ufhhvb"}]': finished 2021-11-03T17:14:24.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: cluster 2021-11-03T17:14:23.183856+0000 mon.smithi027 (mon.0) 585 : cluster [DBG] mgrmap e19: smithi115.tcyodz(active, starting, since 0.140967s), standbys: smithi027.ufhhvb 2021-11-03T17:14:24.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: audit 2021-11-03T17:14:23.183858 2021-11-03T17:14:24.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: +0000 mon.smithi115 (mon.1) 23 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.jdwohn"}]: dispatch 2021-11-03T17:14:24.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: audit 2021-11-03T17:14:23.184188+0000 mon.smithi115 (mon.1) 24 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.fksixo"}]: dispatch 2021-11-03T17:14:24.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: audit 2021-11-03T17:14:23.184512+0000 mon.smithi115 (mon.1) 25 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.zsfgnv"}]: dispatch 2021-11-03T17:14:24.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: audit 2021-11-03T17:14:23.184810+0000 mon.smithi115 (mon.1) 26 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.semywj"}]: dispatch 2021-11-03T17:14:24.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: audit 2021-11-03T17:14:23.185113+0000 mon.smithi115 (mon.1) 27 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "mgr metadata", "who": "smithi115.tcyodz", "id": "smithi115.tcyodz"}]: dispatch 2021-11-03T17:14:24.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: audit 2021-11-03T17:14:23.185354+0000 mon.smithi115 (mon.1) 28 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "mgr metadata", "who": "smithi027.ufhhvb", "id": "smithi027.ufhhvb"}]: dispatch 2021-11-03T17:14:24.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: audit 2021-11-03T17:14:23.185586+0000 mon.smithi115 (mon.1) 29 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-11-03T17:14:24.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: audit 2021-11-03T17:14:23.185863+0000 mon.smithi115 (mon.1) 30 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-11-03T17:14:24.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: audit 2021-11-03T17:14:23.186133+0000 mon.smithi115 (mon.1) 31 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-11-03T17:14:24.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: audit 2021-11-03T17:14:23.186376+0000 mon.smithi115 (mon.1) 32 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-11-03T17:14:24.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: audit 2021-11-03T17:14:23.186686+0000 mon.smithi115 (mon.1) 33 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-11-03T17:14:24.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: audit 2021-11-03T17:14:23.186953+0000 mon.smithi115 (mon.1) 34 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-11-03T17:14:24.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: audit 2021-11-03T17:14:23.187200+0000 mon.smithi115 (mon.1) 35 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "mds metadata"}]: dispatch 2021-11-03T17:14:24.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: audit 2021-11-03T17:14:23.187479+0000 mon.smithi115 (mon.1) 36 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "osd metadata"}]: dispatch 2021-11-03T17:14:24.026 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: audit 2021-11-03T17:14:23.187850+0000 mon.smithi115 (mon.1) 37 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "mon metadata"}]: dispatch 2021-11-03T17:14:24.026 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: cluster 2021-11-03T17:14:23.207816+0000 mon.smithi027 (mon.0) 586 : cluster [INF] Manager daemon smithi115.tcyodz is now available 2021-11-03T17:14:24.026 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: audit 2021-11-03T17:14:23.225263+0000 mon.smithi115 (mon.1) 38 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:14:24.026 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: audit 2021-11-03T17:14:23.228838+0000 mon.smithi115 (mon.1) 39 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:14:24.027 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: audit 2021-11-03T17:14:23.238610+0000 mon.smithi115 (mon.1) 40 : audit [INF] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "osd pool rename", "format": "json", "srcpool": "device_health_metrics", "destpool": ".mgr"}]: dispatch 2021-11-03T17:14:24.027 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: audit 2021-11-03T17:14:23.239321+0000 mon.smithi027 (mon.0) 587 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "osd pool rename", "format": "json", "srcpool": "device_health_metrics", "destpool": ".mgr"}]: dispatch 2021-11-03T17:14:24.027 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: audit 2021-11-03T17:14:23 2021-11-03T17:14:24.027 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: .247927+0000 mon.smithi027 (mon.0) 588 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' 2021-11-03T17:14:24.028 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: audit 2021-11-03T17:14:23.257853+0000 mon.smithi115 (mon.1) 41 : audit [INF] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi115.tcyodz/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:14:24.028 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: audit 2021-11-03T17:14:23.258254+0000 mon.smithi027 2021-11-03T17:14:24.028 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: (mon.0) 589 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi115.tcyodz/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:14:24.028 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: audit 2021-11- 2021-11-03T17:14:24.029 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: 03T17:14:23.267096+0000 mon.smithi115 ( 2021-11-03T17:14:24.029 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: mon.1) 42 : audit [INF] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi115.tcyodz/trash_purge_schedule"}]: dispatch 2021-11-03T17:14:24.029 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: audit 2021- 2021-11-03T17:14:24.030 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: 11-03T17:14:23.267553+0000 2021-11-03T17:14:24.030 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:23 smithi027 conmon[32322]: mon.smithi027 (mon.0) 590 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi115.tcyodz/trash_purge_schedule"}]: dispatch 2021-11-03T17:14:24.031 INFO:tasks.workunit.client.0.smithi027.stdout:3/473: sync 2021-11-03T17:14:24.031 INFO:tasks.workunit.client.0.smithi027.stdout:2/547: sync 2021-11-03T17:14:24.031 INFO:tasks.workunit.client.0.smithi027.stdout:3/474: write d1/db/f32 [932111,111509] 0 2021-11-03T17:14:24.032 INFO:tasks.workunit.client.0.smithi027.stdout:3/475: readlink d1/db/d52/l80 0 2021-11-03T17:14:24.032 INFO:tasks.workunit.client.0.smithi027.stdout:3/476: chown d1/db/f1c 491 1 2021-11-03T17:14:24.032 INFO:tasks.workunit.client.0.smithi027.stdout:2/548: unlink d0/d5/l1a 0 2021-11-03T17:14:24.033 INFO:tasks.workunit.client.0.smithi027.stdout:3/477: mkdir d1/d6f/d97 0 2021-11-03T17:14:24.033 INFO:tasks.workunit.client.0.smithi027.stdout:2/549: rename d0/d2a/d5d/l84 to d0/d1b/d29/d54/d67/d6a/la4 0 2021-11-03T17:14:24.034 INFO:tasks.workunit.client.0.smithi027.stdout:2/550: write d0/d1b/d1c/f7e [5216646,94905] 0 2021-11-03T17:14:24.034 INFO:tasks.workunit.client.0.smithi027.stdout:2/551: write d0/d7c/f9f [1004369,98641] 0 2021-11-03T17:14:24.035 INFO:tasks.workunit.client.0.smithi027.stdout:2/552: write d0/d1b/d29/d54/d67/d6a/f97 [6139122,71103] 0 2021-11-03T17:14:24.035 INFO:tasks.workunit.client.0.smithi027.stdout:2/553: truncate d0/d1b/f27 813051 0 2021-11-03T17:14:24.037 INFO:tasks.workunit.client.0.smithi027.stdout:2/554: unlink d0/d1b/d1c/l1d 0 2021-11-03T17:14:24.037 INFO:tasks.workunit.client.0.smithi027.stdout:2/555: mknod d0/d5/d2c/ca5 1 2021-11-03T17:14:24.039 INFO:tasks.workunit.client.0.smithi027.stdout:2/556: creat d0/d1b/d29/d58/d5b/fa6 x:0 0 0 2021-11-03T17:14:24.047 INFO:tasks.workunit.client.0.smithi027.stdout:2/557: fsync d0/d1b/d1c/f7e 0 2021-11-03T17:14:24.049 INFO:tasks.workunit.client.0.smithi027.stdout:3/478: dwrite d1/db/f32 [0,4194304] 0 2021-11-03T17:14:24.049 INFO:tasks.workunit.client.0.smithi027.stdout:3/479: stat d1/d40/f8f 0 2021-11-03T17:14:24.050 INFO:tasks.workunit.client.0.smithi027.stdout:3/480: write d1/db/d52/d6a/d7c/f89 [961024,103262] 0 2021-11-03T17:14:24.062 INFO:tasks.workunit.client.0.smithi027.stdout:3/481: dread d1/f18 [0,4194304] 0 2021-11-03T17:14:24.062 INFO:tasks.workunit.client.0.smithi027.stdout:3/482: getdents d1/d6f/d97 0 2021-11-03T17:14:24.066 INFO:tasks.workunit.client.0.smithi027.stdout:2/558: dwrite d0/d1b/d29/d54/d67/d6a/f97 [0,4194304] 0 2021-11-03T17:14:24.069 INFO:tasks.workunit.client.0.smithi027.stdout:2/559: dread d0/f46 [0,4194304] 0 2021-11-03T17:14:24.084 INFO:tasks.workunit.client.0.smithi027.stdout:2/560: dwrite d0/d1b/d29/d58/d5b/f62 [0,4194304] 0 2021-11-03T17:14:24.084 INFO:tasks.workunit.client.0.smithi027.stdout:2/561: fsync d0/d1b/d29/d54/f7f 0 2021-11-03T17:14:24.108 INFO:tasks.workunit.client.0.smithi027.stdout:3/483: mkdir d1/db/d52/d6a/d7c/d88/d98 0 2021-11-03T17:14:24.108 INFO:tasks.workunit.client.0.smithi027.stdout:3/484: chown d1/f90 422012 1 2021-11-03T17:14:24.113 INFO:tasks.workunit.client.0.smithi027.stdout:2/562: mkdir d0/da7 0 2021-11-03T17:14:24.123 INFO:tasks.workunit.client.0.smithi027.stdout:2/563: rmdir d0/d2a 39 2021-11-03T17:14:24.123 INFO:tasks.workunit.client.0.smithi027.stdout:2/564: mknod d0/d1b/d87/ca8 1 2021-11-03T17:14:24.129 INFO:tasks.workunit.client.0.smithi027.stdout:2/565: dread d0/f10 [0,4194304] 0 2021-11-03T17:14:24.133 INFO:tasks.workunit.client.0.smithi027.stdout:2/566: truncate d0/d1b/f82 3438531 0 2021-11-03T17:14:24.135 INFO:tasks.workunit.client.0.smithi027.stdout:2/567: write d0/d1b/d29/f9d [141656,64287] 0 2021-11-03T17:14:24.154 INFO:tasks.workunit.client.0.smithi027.stdout:2/568: dwrite d0/d5/f3 [4194304,4194304] 0 2021-11-03T17:14:24.155 INFO:tasks.workunit.client.0.smithi027.stdout:2/569: creat d0/fa9 x:0 0 0 2021-11-03T17:14:24.156 INFO:tasks.workunit.client.0.smithi027.stdout:2/570: write d0/d1b/d1c/f57 [3784025,66075] 0 2021-11-03T17:14:24.159 INFO:tasks.workunit.client.0.smithi027.stdout:2/571: rename d0/d2a to d0/d1b/d29/d54/d64/daa 0 2021-11-03T17:14:24.161 INFO:tasks.workunit.client.0.smithi027.stdout:2/572: dread d0/d1b/d29/d54/d64/daa/d5d/f65 [0,4194304] 0 2021-11-03T17:14:24.162 INFO:tasks.workunit.client.0.smithi027.stdout:2/573: rmdir d0/d1b/d29/d58/d5b 39 2021-11-03T17:14:24.165 INFO:tasks.workunit.client.0.smithi027.stdout:2/574: getdents d0/d5 0 2021-11-03T17:14:24.165 INFO:tasks.workunit.client.0.smithi027.stdout:2/575: mknod d0/d1b/d29/cab 1 2021-11-03T17:14:24.167 INFO:tasks.workunit.client.0.smithi027.stdout:2/576: getdents d0/d1b/d29/d54/d64/daa 0 2021-11-03T17:14:24.177 INFO:tasks.workunit.client.0.smithi027.stdout:2/577: dread d0/d5/f6d [0,4194304] 0 2021-11-03T17:14:24.178 INFO:tasks.workunit.client.0.smithi027.stdout:2/578: creat d0/d5/fac x:0 0 0 2021-11-03T17:14:24.179 INFO:tasks.workunit.client.0.smithi027.stdout:2/579: fsync d0/d1b/d87/f8c 0 2021-11-03T17:14:24.179 INFO:tasks.workunit.client.0.smithi027.stdout:2/580: getdents d0/da7 0 2021-11-03T17:14:24.186 INFO:tasks.workunit.client.0.smithi027.stdout:2/581: dread d0/d45/f7b [0,4194304] 0 2021-11-03T17:14:24.187 INFO:tasks.workunit.client.0.smithi027.stdout:2/582: write d0/f46 [1175085,33835] 0 2021-11-03T17:14:24.190 INFO:tasks.workunit.client.0.smithi027.stdout:2/583: rmdir d0/d1b/d29/d54/d64/daa/d5d/d70 0 2021-11-03T17:14:24.191 INFO:tasks.workunit.client.0.smithi027.stdout:2/584: chown d0/d5/la 7 1 2021-11-03T17:14:24.191 INFO:tasks.workunit.client.0.smithi027.stdout:2/585: truncate d0/d7c/f9f 2086995 0 2021-11-03T17:14:24.192 INFO:tasks.workunit.client.0.smithi027.stdout:2/586: getdents d0/d1b 0 2021-11-03T17:14:24.193 INFO:tasks.workunit.client.0.smithi027.stdout:2/587: chown d0/d1b/d1c 2888522 1 2021-11-03T17:14:24.207 INFO:tasks.workunit.client.1.smithi115.stdout:7/395: sync 2021-11-03T17:14:24.207 INFO:tasks.workunit.client.1.smithi115.stdout:5/615: sync 2021-11-03T17:14:24.208 INFO:tasks.workunit.client.1.smithi115.stdout:5/616: chown d1a/d28/d7b/f74 27395 1 2021-11-03T17:14:24.208 INFO:tasks.workunit.client.1.smithi115.stdout:7/396: fdatasync d3/da/d18/d73/d33/f59 0 2021-11-03T17:14:24.209 INFO:tasks.workunit.client.1.smithi115.stdout:7/397: chown d3/f27 477 1 2021-11-03T17:14:24.209 INFO:tasks.workunit.client.1.smithi115.stdout:5/617: fdatasync d1a/d28/f29 0 2021-11-03T17:14:24.210 INFO:tasks.workunit.client.0.smithi027.stdout:4/434: sync 2021-11-03T17:14:24.211 INFO:tasks.workunit.client.1.smithi115.stdout:7/398: creat d3/da/f7e x:0 0 0 2021-11-03T17:14:24.212 INFO:tasks.workunit.client.1.smithi115.stdout:7/399: fdatasync d3/f47 0 2021-11-03T17:14:24.212 INFO:tasks.workunit.client.1.smithi115.stdout:5/618: link d1a/d28/d4c/d75/d68/da5/da0/fa7 d1a/d28/d4c/d75/fc6 0 2021-11-03T17:14:24.213 INFO:tasks.workunit.client.1.smithi115.stdout:7/400: link d3/da/d18/f22 d3/d4c/f7f 0 2021-11-03T17:14:24.214 INFO:tasks.workunit.client.1.smithi115.stdout:7/401: mknod d3/da/d18/d73/d35/d53/c80 1 2021-11-03T17:14:24.214 INFO:tasks.workunit.client.1.smithi115.stdout:7/402: truncate d3/f27 3996871 0 2021-11-03T17:14:24.220 INFO:tasks.workunit.client.1.smithi115.stdout:7/403: rename d3/da/d18/d73/d33/l77 to d3/da/d18/d73/l81 0 2021-11-03T17:14:24.220 INFO:tasks.workunit.client.1.smithi115.stdout:7/404: mknod d3/da/c82 1 2021-11-03T17:14:24.225 INFO:tasks.workunit.client.1.smithi115.stdout:7/405: write d3/da/d18/d73/d52/f56 [1470053,94212] 0 2021-11-03T17:14:24.226 INFO:tasks.workunit.client.1.smithi115.stdout:7/406: getdents d3 0 2021-11-03T17:14:24.228 INFO:tasks.workunit.client.1.smithi115.stdout:7/407: creat d3/da/d18/d73/d52/f83 x:0 0 0 2021-11-03T17:14:24.228 INFO:tasks.workunit.client.1.smithi115.stdout:7/408: chown d3/da/d18/d73/d52/f56 6976163 1 2021-11-03T17:14:24.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: cluster 2021-11-03T17:14:23.039364+0000 mon.smithi027 (mon.0) 581 : cluster [INF] Active manager daemon smithi115.tcyodz restarted 2021-11-03T17:14:24.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: audit 2021-11-03T17:14:23.041203+0000 mon.smithi115 (mon.1) 17 : audit [DBG] from='mgr.? 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi115.tcyodz/crt"}]: dispatch 2021-11-03T17:14:24.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: audit 2021-11-03T17:14:23.041705+0000 mon.smithi115 (mon.1) 18 : audit [DBG] from='mgr.? 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-11-03T17:14:24.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: cluster 2021-11-03T17:14:23.043099+0000 mon.smithi027 (mon.0) 582 : cluster [INF] Activating manager daemon smithi115.tcyodz 2021-11-03T17:14:24.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: audit 2021-11-03T17:14:23.043898+0000 mon.smithi115 (mon.1) 19 : audit [DBG] from='mgr.? 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi115.tcyodz/key"}]: dispatch 2021-11-03T17:14:24.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: cluster 2021-11-03T17:14:23.049369+0000 mon.smithi027 (mon.0) 583 : cluster [DBG] osdmap e46: 6 total, 6 up, 6 in 2021-11-03T17:14:24.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: audit 2021-11-03T17:14:23.050508+0000 mon.smithi115 (mon.1) 20 : audit [DBG] from='mgr.? 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-11-03T17:14:24.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: audit 2021-11-03T17:14:23.182911+0000 mon.smithi115 (mon.1) 21 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2021-11-03T17:14:24.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: audit 2021-11-03T17:14:23.183026+0000 mon.smithi115 (mon.1) 22 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2021-11-03T17:14:24.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: audit 2021-11-03T17:14:23.183793+0000 mon.smithi027 (mon.0) 584 : audit [INF] from='mgr.14162 172.21.15.27:0/1544825072' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "mgr fail", "who": "smithi027.ufhhvb"}]': finished 2021-11-03T17:14:24.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: cluster 2021-11-03T17:14:23.183856+0000 mon.smithi027 (mon.0) 585 : cluster [DBG] mgrmap e19: smithi115.tcyodz(active, starting, since 0.140967s), standbys: smithi027.ufhhvb 2021-11-03T17:14:24.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: audit 2021-11-03T17:14:23.183858+0000 mon.smithi115 (mon.1) 23 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.jdwohn"}]: dispatch 2021-11-03T17:14:24.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: audit 2021-11-03T17:14:23.184188+0000 mon.smithi115 (mon.1) 24 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.fksixo"}]: dispatch 2021-11-03T17:14:24.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: audit 2021-11-03T17:14:23.184512+0000 mon.smithi115 (mon.1) 25 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.zsfgnv"}]: dispatch 2021-11-03T17:14:24.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: audit 2021-11-03T17:14:23.184810+0000 mon.smithi115 (mon.1) 26 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.semywj"}]: dispatch 2021-11-03T17:14:24.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: audit 2021-11-03T17:14:23.185113+0000 mon.smithi115 (mon.1) 27 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "mgr metadata", "who": "smithi115.tcyodz", "id": "smithi115.tcyodz"}]: dispatch 2021-11-03T17:14:24.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: audit 2021-11-03T17:14:23.185354+0000 mon.smithi115 (mon.1) 28 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "mgr metadata", "who": "smithi027.ufhhvb", "id": "smithi027.ufhhvb"}]: dispatch 2021-11-03T17:14:24.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: audit 2021-11-03T17:14:23.185586+0000 mon.smithi115 (mon.1) 29 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-11-03T17:14:24.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: audit 2021-11-03T17:14:23.185863+0000 mon.smithi115 (mon.1) 30 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-11-03T17:14:24.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: audit 2021-11-03T17:14:23.186133+0000 mon.smithi115 (mon.1) 31 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-11-03T17:14:24.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: audit 2021-11-03T17:14:23.186376+0000 mon.smithi115 (mon.1) 32 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-11-03T17:14:24.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: audit 2021-11-03T17:14:23.186686+0000 mon.smithi115 (mon.1) 33 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-11-03T17:14:24.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: audit 2021-11-03T17:14:23.186953+0000 mon.smithi115 (mon.1) 34 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-11-03T17:14:24.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: audit 2021-11-03T17:14:23.187200+0000 mon.smithi115 (mon.1) 35 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "mds metadata"}]: dispatch 2021-11-03T17:14:24.241 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: audit 2021-11-03T17:14:23.187479+0000 mon.smithi115 (mon.1) 36 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "osd metadata"}]: dispatch 2021-11-03T17:14:24.241 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: audit 2021-11-03T17:14:23.187850+0000 mon.smithi115 (mon.1) 37 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "mon metadata"}]: dispatch 2021-11-03T17:14:24.241 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: cluster 2021-11-03T17:14:23.207816+0000 mon.smithi027 (mon.0) 586 : cluster [INF] Manager daemon smithi115.tcyodz is now available 2021-11-03T17:14:24.241 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: audit 2021-11-03T17:14:23.225263+0000 mon.smithi115 (mon.1) 38 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:14:24.242 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: audit 2021-11-03T17:14:23.228838+0000 mon.smithi115 (mon.1) 39 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:14:24.242 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: audit 2021-11-03T17:14:23.238610+0000 mon.smithi115 (mon 2021-11-03T17:14:24.243 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: .1) 40 : audit [INF] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "osd pool rename", "format": "json", "srcpool": "device_health_metrics", "destpool": ".mgr"}]: dispatch 2021-11-03T17:14:24.243 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: audit 2021-11-03T17:14:23.239321+0000 mon.smithi027 (mon.0) 587 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "osd pool rename", "format": "json", "srcpool": "device_health_metrics", "destpool": ".mgr"}]: dispatch 2021-11-03T17:14:24.243 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: audit 2021-11-03T17:14:23.247927+0000 mon.smithi027 (mon.0) 588 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' 2021-11-03T17:14:24.244 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: audit 2021-11-03T17:14:23.257853+0000 mon.smithi115 (mon.1) 41 : audit [INF] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi115.tcyodz/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:14:24.244 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: audit 2021-11-03T17:14:23.258254+0000 2021-11-03T17:14:24.244 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: mon.smithi027 (mon.0) 589 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi115.tcyodz/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:14:24.245 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: audit 2021-11-03T17:14 2021-11-03T17:14:24.245 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: :23.267096+0000 mon.smithi115 (mon.1) 42 : audit [INF] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi115.tcyodz/trash_purge_schedule"}]: dispatch 2021-11-03T17:14:24.245 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: audit 2021-11-03T17:14 2021-11-03T17:14:24.246 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: :23.267553+0000 mon.smithi027 (mon.0) 590 : audit 2021-11-03T17:14:24.246 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:23 smithi115 conmon[34670]: [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi115.tcyodz/trash_purge_schedule"}]: dispatch 2021-11-03T17:14:24.248 INFO:tasks.workunit.client.0.smithi027.stdout:4/435: dwrite d6/db/d21/f37 [0,4194304] 0 2021-11-03T17:14:24.248 INFO:tasks.workunit.client.0.smithi027.stdout:4/436: write d6/d9/d16/f19 [1735257,54911] 0 2021-11-03T17:14:24.249 INFO:tasks.workunit.client.0.smithi027.stdout:4/437: getdents d6/d9/d8f 0 2021-11-03T17:14:24.249 INFO:tasks.workunit.client.1.smithi115.stdout:5/619: dwrite d1a/d3d/f62 [8388608,4194304] 0 2021-11-03T17:14:24.250 INFO:tasks.workunit.client.1.smithi115.stdout:5/620: write d1a/d28/d4a/d65/db5/f67 [3158610,93866] 0 2021-11-03T17:14:24.250 INFO:tasks.workunit.client.1.smithi115.stdout:5/621: readlink d1a/d28/l4d 0 2021-11-03T17:14:24.250 INFO:tasks.workunit.client.1.smithi115.stdout:5/622: creat d1a/d3d/fc7 x:0 0 0 2021-11-03T17:14:24.251 INFO:tasks.workunit.client.1.smithi115.stdout:5/623: truncate d1a/d28/d4c/d75/d68/da5/fa9 411368 0 2021-11-03T17:14:24.252 INFO:tasks.workunit.client.0.smithi027.stdout:4/438: getdents d6/d9 0 2021-11-03T17:14:24.252 INFO:tasks.workunit.client.0.smithi027.stdout:4/439: creat d6/db/d21/f94 x:0 0 0 2021-11-03T17:14:24.253 INFO:tasks.workunit.client.1.smithi115.stdout:7/409: dwrite d3/f6a [4194304,4194304] 0 2021-11-03T17:14:24.253 INFO:tasks.workunit.client.1.smithi115.stdout:5/624: link d1a/d28/d4c/d75/d68/da5/l79 d1a/d28/d4c/d63/lc8 0 2021-11-03T17:14:24.253 INFO:tasks.workunit.client.1.smithi115.stdout:7/410: truncate d3/f47 1786050 0 2021-11-03T17:14:24.253 INFO:tasks.workunit.client.1.smithi115.stdout:7/411: mknod d3/da/c84 1 2021-11-03T17:14:24.254 INFO:tasks.workunit.client.1.smithi115.stdout:7/412: fdatasync d3/da/d18/d73/d35/f7c 0 2021-11-03T17:14:24.254 INFO:tasks.workunit.client.1.smithi115.stdout:7/413: mknod d3/da/d18/d73/d52/c85 1 2021-11-03T17:14:24.254 INFO:tasks.workunit.client.1.smithi115.stdout:7/414: write d3/da/d18/d73/d35/f3f [1831877,25580] 0 2021-11-03T17:14:24.254 INFO:tasks.workunit.client.1.smithi115.stdout:7/415: readlink d3/da/d18/d73/d33/l6b 0 2021-11-03T17:14:24.255 INFO:tasks.workunit.client.1.smithi115.stdout:7/416: write d3/d4c/f7f [3040610,125619] 0 2021-11-03T17:14:24.255 INFO:tasks.workunit.client.1.smithi115.stdout:7/417: rename d3/da/d18/l2a to d3/d4c/l86 0 2021-11-03T17:14:24.256 INFO:tasks.workunit.client.1.smithi115.stdout:7/418: chown d3/da/d18/d73/d35/d53/l5a 0 1 2021-11-03T17:14:24.256 INFO:tasks.workunit.client.1.smithi115.stdout:7/419: fdatasync d3/da/d18/d73/d35/f75 0 2021-11-03T17:14:24.256 INFO:tasks.workunit.client.1.smithi115.stdout:7/420: fsync d3/f25 0 2021-11-03T17:14:24.257 INFO:tasks.workunit.client.1.smithi115.stdout:7/421: write d3/da/d18/d73/d33/f74 [2561639,56963] 0 2021-11-03T17:14:24.258 INFO:tasks.workunit.client.1.smithi115.stdout:5/625: dread d1a/d28/d4a/f56 [0,4194304] 0 2021-11-03T17:14:24.259 INFO:tasks.workunit.client.0.smithi027.stdout:4/440: dread d6/db/d21/f38 [0,4194304] 0 2021-11-03T17:14:24.260 INFO:tasks.workunit.client.1.smithi115.stdout:7/422: creat d3/da/d18/d73/d35/d53/f87 x:0 0 0 2021-11-03T17:14:24.260 INFO:tasks.workunit.client.0.smithi027.stdout:4/441: write d6/db/d21/d41/f88 [813856,105341] 0 2021-11-03T17:14:24.261 INFO:tasks.workunit.client.0.smithi027.stdout:4/442: chown d6/db/f8e 18219989 1 2021-11-03T17:14:24.261 INFO:tasks.workunit.client.1.smithi115.stdout:5/626: creat d1a/d28/d7b/d4f/fc9 x:0 0 0 2021-11-03T17:14:24.262 INFO:tasks.workunit.client.1.smithi115.stdout:5/627: creat d1a/d28/d7b/d48/fca x:0 0 0 2021-11-03T17:14:24.262 INFO:tasks.workunit.client.0.smithi027.stdout:4/443: creat d6/d9/d8f/f95 x:0 0 0 2021-11-03T17:14:24.264 INFO:tasks.workunit.client.0.smithi027.stdout:4/444: creat d6/d9/d28/f96 x:0 0 0 2021-11-03T17:14:24.265 INFO:tasks.workunit.client.1.smithi115.stdout:5/628: link d1a/d28/d4c/d75/d68/da5/fc3 d1a/d23/fcb 0 2021-11-03T17:14:24.266 INFO:tasks.workunit.client.1.smithi115.stdout:5/629: fdatasync d1a/d28/f91 0 2021-11-03T17:14:24.267 INFO:tasks.workunit.client.1.smithi115.stdout:5/630: rename d1a/l38 to d1a/d28/d4c/d75/dae/lcc 0 2021-11-03T17:14:24.270 INFO:tasks.workunit.client.1.smithi115.stdout:5/631: dread f11 [0,4194304] 0 2021-11-03T17:14:24.272 INFO:tasks.workunit.client.1.smithi115.stdout:7/423: truncate d3/da/d18/d73/d33/f74 787744 0 2021-11-03T17:14:24.272 INFO:tasks.workunit.client.1.smithi115.stdout:5/632: write d1a/d28/d7b/d48/d7f/f86 [605560,101288] 0 2021-11-03T17:14:24.273 INFO:tasks.workunit.client.1.smithi115.stdout:7/424: rmdir d3/da/d18/d73 39 2021-11-03T17:14:24.276 INFO:tasks.workunit.client.1.smithi115.stdout:7/425: unlink d3/da/d18/d73/d35/d53/l5a 0 2021-11-03T17:14:24.276 INFO:tasks.workunit.client.1.smithi115.stdout:7/426: truncate d3/da/d18/d73/d52/f66 901118 0 2021-11-03T17:14:24.278 INFO:tasks.workunit.client.1.smithi115.stdout:5/633: write d1a/d23/f61 [2393832,130880] 0 2021-11-03T17:14:24.289 INFO:tasks.workunit.client.0.smithi027.stdout:4/445: dwrite d6/d9/d16/d36/d70/d3e/f6c [0,4194304] 0 2021-11-03T17:14:24.290 INFO:tasks.workunit.client.1.smithi115.stdout:7/427: dread d3/f6a [0,4194304] 0 2021-11-03T17:14:24.290 INFO:tasks.workunit.client.1.smithi115.stdout:7/428: readlink d3/da/d18/d73/d33/l6b 0 2021-11-03T17:14:24.290 INFO:tasks.workunit.client.1.smithi115.stdout:7/429: readlink d3/da/d18/l28 0 2021-11-03T17:14:24.291 INFO:tasks.workunit.client.1.smithi115.stdout:7/430: write d3/da/d18/f22 [1495813,85448] 0 2021-11-03T17:14:24.293 INFO:tasks.workunit.client.1.smithi115.stdout:7/431: symlink d3/da/d18/d73/d52/l88 0 2021-11-03T17:14:24.295 INFO:tasks.workunit.client.1.smithi115.stdout:7/432: rename d3/f1e to d3/da/d18/d73/f89 0 2021-11-03T17:14:24.296 INFO:tasks.workunit.client.1.smithi115.stdout:7/433: symlink d3/l8a 0 2021-11-03T17:14:24.300 INFO:tasks.workunit.client.1.smithi115.stdout:7/434: dread d3/d4c/f5b [0,4194304] 0 2021-11-03T17:14:24.301 INFO:tasks.workunit.client.0.smithi027.stdout:9/588: sync 2021-11-03T17:14:24.305 INFO:tasks.workunit.client.1.smithi115.stdout:5/634: dwrite d1a/d28/d4c/d75/d83/f88 [0,4194304] 0 2021-11-03T17:14:24.307 INFO:tasks.workunit.client.1.smithi115.stdout:5/635: read d1a/d3d/f47 [435295,59192] 0 2021-11-03T17:14:24.319 INFO:tasks.workunit.client.1.smithi115.stdout:3/634: sync 2021-11-03T17:14:24.319 INFO:tasks.workunit.client.1.smithi115.stdout:4/641: sync 2021-11-03T17:14:24.319 INFO:tasks.workunit.client.1.smithi115.stdout:4/642: readlink d7/dc/l33 0 2021-11-03T17:14:24.320 INFO:tasks.workunit.client.1.smithi115.stdout:4/643: mknod d7/dc/d6a/d3c/dc5/dcc/cda 1 2021-11-03T17:14:24.320 INFO:tasks.workunit.client.1.smithi115.stdout:4/644: dread - d7/dc/d6a/d3c/fa8 zero size 2021-11-03T17:14:24.320 INFO:tasks.workunit.client.1.smithi115.stdout:6/438: sync 2021-11-03T17:14:24.320 INFO:tasks.workunit.client.1.smithi115.stdout:6/439: write d0/f4d [1194424,86481] 0 2021-11-03T17:14:24.321 INFO:tasks.workunit.client.1.smithi115.stdout:6/440: stat d0/d1/d2e/f8e 0 2021-11-03T17:14:24.321 INFO:tasks.workunit.client.1.smithi115.stdout:0/613: sync 2021-11-03T17:14:24.321 INFO:tasks.workunit.client.1.smithi115.stdout:0/614: dread - d9/d15/d1a/d1d/fb4 zero size 2021-11-03T17:14:24.321 INFO:tasks.workunit.client.1.smithi115.stdout:0/615: write d9/d15/d78/d80/d82/f91 [1257233,19334] 0 2021-11-03T17:14:24.321 INFO:tasks.workunit.client.1.smithi115.stdout:0/616: dread - d9/d15/d1a/d1d/dcc/d6e/d71/fa5 zero size 2021-11-03T17:14:24.330 INFO:tasks.workunit.client.1.smithi115.stdout:4/645: dread d7/dc/fb0 [0,4194304] 0 2021-11-03T17:14:24.331 INFO:tasks.workunit.client.0.smithi027.stdout:9/589: dread db/d1b/d2b/f3f [0,4194304] 0 2021-11-03T17:14:24.332 INFO:tasks.workunit.client.0.smithi027.stdout:9/590: fsync db/d1b/d1c/d1d/fb6 0 2021-11-03T17:14:24.333 INFO:tasks.workunit.client.1.smithi115.stdout:7/435: dwrite d3/da/d18/d73/d35/f5f [0,4194304] 0 2021-11-03T17:14:24.355 INFO:tasks.workunit.client.0.smithi027.stdout:4/446: mkdir d6/d9/d16/d36/d97 0 2021-11-03T17:14:24.355 INFO:tasks.workunit.client.0.smithi027.stdout:9/591: mkdir db/d1b/d8c/dcd 0 2021-11-03T17:14:24.366 INFO:tasks.workunit.client.1.smithi115.stdout:3/635: dwrite d13/d51/d40/f5a [4194304,4194304] 0 2021-11-03T17:14:24.375 INFO:tasks.workunit.client.1.smithi115.stdout:0/617: dwrite f8 [0,4194304] 0 2021-11-03T17:14:24.376 INFO:tasks.workunit.client.0.smithi027.stdout:4/447: dwrite f4 [0,4194304] 0 2021-11-03T17:14:24.377 INFO:tasks.workunit.client.0.smithi027.stdout:4/448: dread - d6/db/d21/f93 zero size 2021-11-03T17:14:24.399 INFO:tasks.workunit.client.1.smithi115.stdout:0/618: dwrite d9/d15/d1a/d1d/dcc/d46/d8c/f74 [0,4194304] 0 2021-11-03T17:14:24.409 INFO:tasks.workunit.client.0.smithi027.stdout:4/449: dwrite d6/db/d21/f37 [0,4194304] 0 2021-11-03T17:14:24.418 INFO:tasks.workunit.client.1.smithi115.stdout:0/619: dwrite d9/d15/d78/fd0 [0,4194304] 0 2021-11-03T17:14:24.418 INFO:tasks.workunit.client.1.smithi115.stdout:0/620: readlink d9/d15/d1a/d1d/lca 0 2021-11-03T17:14:24.425 INFO:tasks.workunit.client.1.smithi115.stdout:0/621: dread d9/f25 [0,4194304] 0 2021-11-03T17:14:24.427 INFO:tasks.workunit.client.1.smithi115.stdout:0/622: write d9/f23 [523277,96632] 0 2021-11-03T17:14:24.439 INFO:tasks.workunit.client.0.smithi027.stdout:4/450: unlink d6/db/d21/d41/d44/f55 0 2021-11-03T17:14:24.439 INFO:tasks.workunit.client.0.smithi027.stdout:4/451: fsync d6/d9/d16/d36/d70/d3e/d58/f60 0 2021-11-03T17:14:24.448 INFO:tasks.workunit.client.1.smithi115.stdout:0/623: dwrite f8 [0,4194304] 0 2021-11-03T17:14:24.451 INFO:tasks.workunit.client.0.smithi027.stdout:4/452: rename d6/db to d6/d9/d16/d36/d70/d3e/d92/d98 0 2021-11-03T17:14:24.459 INFO:tasks.workunit.client.0.smithi027.stdout:4/453: dread d6/d9/d16/d36/d70/d3e/d92/d98/d21/f2c [0,4194304] 0 2021-11-03T17:14:24.883 INFO:tasks.workunit.client.0.smithi027.stdout:0/640: sync 2021-11-03T17:14:24.883 INFO:tasks.workunit.client.0.smithi027.stdout:0/641: rename d9/da/d30/d92/dca to d9/da/d30/d92/dca/ddf 22 2021-11-03T17:14:24.884 INFO:tasks.workunit.client.0.smithi027.stdout:0/642: read d9/da/d64/d70/d6f/fa3 [1868743,110002] 0 2021-11-03T17:14:24.891 INFO:tasks.workunit.client.0.smithi027.stdout:0/643: creat d9/da/d30/d92/dca/fe0 x:0 0 0 2021-11-03T17:14:24.891 INFO:tasks.workunit.client.0.smithi027.stdout:0/644: chown d9/da/d30/d92/fd2 291 1 2021-11-03T17:14:24.891 INFO:tasks.workunit.client.0.smithi027.stdout:0/645: chown d9/da/d64/d70/ld1 14624186 1 2021-11-03T17:14:24.895 INFO:tasks.workunit.client.0.smithi027.stdout:0/646: creat d9/da/d30/dac/dba/ddc/fe1 x:0 0 0 2021-11-03T17:14:24.895 INFO:tasks.workunit.client.0.smithi027.stdout:0/647: chown d9/da/d30/da7/la5 201071246 1 2021-11-03T17:14:24.896 INFO:tasks.workunit.client.0.smithi027.stdout:0/648: stat d9/fbf 0 2021-11-03T17:14:24.896 INFO:tasks.workunit.client.0.smithi027.stdout:0/649: write d9/da/d31/f36 [201370,50838] 0 2021-11-03T17:14:24.899 INFO:tasks.workunit.client.0.smithi027.stdout:0/650: write d9/d12/f17 [3704744,54091] 0 2021-11-03T17:14:24.928 INFO:tasks.workunit.client.0.smithi027.stdout:0/651: dwrite d9/f4b [0,4194304] 0 2021-11-03T17:14:24.931 INFO:tasks.workunit.client.0.smithi027.stdout:0/652: rmdir d9/da/d30/d93 39 2021-11-03T17:14:24.932 INFO:tasks.workunit.client.0.smithi027.stdout:0/653: chown d9/da/d30/d92/dca 90 1 2021-11-03T17:14:24.932 INFO:tasks.workunit.client.0.smithi027.stdout:0/654: write d9/d12/fa6 [775798,25633] 0 2021-11-03T17:14:24.935 INFO:tasks.workunit.client.0.smithi027.stdout:0/655: creat d9/d12/d8f/fe2 x:0 0 0 2021-11-03T17:14:24.935 INFO:tasks.workunit.client.0.smithi027.stdout:0/656: chown d9/da/d64/d70/d6d/d7a/dc9/fd9 9481013 1 2021-11-03T17:14:24.937 INFO:tasks.workunit.client.0.smithi027.stdout:0/657: symlink d9/da/d30/d92/le3 0 2021-11-03T17:14:24.937 INFO:tasks.workunit.client.0.smithi027.stdout:0/658: write d9/da/d31/fb1 [1100921,66088] 0 2021-11-03T17:14:24.939 INFO:tasks.workunit.client.0.smithi027.stdout:0/659: symlink d9/da/d30/d92/dca/le4 0 2021-11-03T17:14:24.945 INFO:tasks.workunit.client.0.smithi027.stdout:0/660: rmdir d9/da/d30/d92 39 2021-11-03T17:14:24.949 INFO:tasks.workunit.client.0.smithi027.stdout:0/661: write d9/da/d30/fd8 [1230263,40298] 0 2021-11-03T17:14:24.951 INFO:tasks.workunit.client.0.smithi027.stdout:0/662: link d9/da/d64/d70/d6d/d9f/laf d9/le5 0 2021-11-03T17:14:24.952 INFO:tasks.workunit.client.0.smithi027.stdout:0/663: chown d9/da/lc0 0 1 2021-11-03T17:14:24.952 INFO:tasks.workunit.client.0.smithi027.stdout:0/664: truncate d9/da/d64/d70/d6d/d9f/fb9 166692 0 2021-11-03T17:14:24.954 INFO:tasks.workunit.client.0.smithi027.stdout:0/665: mkdir d9/da/d30/d92/dca/de6 0 2021-11-03T17:14:24.955 INFO:tasks.workunit.client.0.smithi027.stdout:0/666: write d9/da/d30/d92/dca/fd5 [360410,34624] 0 2021-11-03T17:14:24.955 INFO:tasks.workunit.client.0.smithi027.stdout:0/667: mknod d9/d12/d8f/dc7/ce7 1 2021-11-03T17:14:24.955 INFO:tasks.workunit.client.0.smithi027.stdout:0/668: write d9/da/d64/d70/d5f/f84 [835946,60753] 0 2021-11-03T17:14:24.956 INFO:tasks.workunit.client.0.smithi027.stdout:0/669: link d9/dd3/ld7 d9/d12/d8f/dc7/le8 0 2021-11-03T17:14:24.957 INFO:tasks.workunit.client.0.smithi027.stdout:0/670: creat d9/da/d64/d70/d6f/fe9 x:0 0 0 2021-11-03T17:14:24.957 INFO:tasks.workunit.client.0.smithi027.stdout:0/671: write d9/da/d64/d70/d6d/d7a/fa9 [1039283,78227] 0 2021-11-03T17:14:24.957 INFO:tasks.workunit.client.0.smithi027.stdout:0/672: truncate d9/da/d64/d70/d6f/fa1 655058 0 2021-11-03T17:14:24.958 INFO:tasks.workunit.client.0.smithi027.stdout:0/673: creat d9/da/d30/dac/fea x:0 0 0 2021-11-03T17:14:24.959 INFO:tasks.workunit.client.0.smithi027.stdout:0/674: truncate d9/da/d64/d70/fdb 5053385 0 2021-11-03T17:14:24.959 INFO:tasks.workunit.client.0.smithi027.stdout:0/675: creat d9/da/d30/dac/dba/feb x:0 0 0 2021-11-03T17:14:24.971 INFO:tasks.workunit.client.0.smithi027.stdout:0/676: dread d9/da/d31/f36 [0,4194304] 0 2021-11-03T17:14:24.973 INFO:tasks.workunit.client.0.smithi027.stdout:0/677: creat d9/da/d30/dac/fec x:0 0 0 2021-11-03T17:14:24.973 INFO:tasks.workunit.client.0.smithi027.stdout:0/678: write d9/da/d30/d92/dca/fd5 [559136,123358] 0 2021-11-03T17:14:24.973 INFO:tasks.workunit.client.0.smithi027.stdout:0/679: chown d9/d12/l65 4 1 2021-11-03T17:14:24.976 INFO:tasks.workunit.client.0.smithi027.stdout:0/680: write d9/da/f96 [314515,98841] 0 2021-11-03T17:14:24.993 INFO:tasks.workunit.client.0.smithi027.stdout:0/681: dwrite d9/da/d30/dac/fcf [0,4194304] 0 2021-11-03T17:14:24.993 INFO:tasks.workunit.client.0.smithi027.stdout:0/682: chown d9/d12/lcd 1 1 2021-11-03T17:14:24.996 INFO:tasks.workunit.client.0.smithi027.stdout:0/683: dread d9/da/f16 [0,4194304] 0 2021-11-03T17:14:25.006 INFO:tasks.workunit.client.0.smithi027.stdout:0/684: getdents d9/da/d64/d70/d6f 0 2021-11-03T17:14:25.022 INFO:tasks.workunit.client.0.smithi027.stdout:0/685: dwrite d9/da/d30/dac/dba/fbc [0,4194304] 0 2021-11-03T17:14:25.022 INFO:tasks.workunit.client.0.smithi027.stdout:0/686: mknod d9/d12/d8f/ced 1 2021-11-03T17:14:25.028 INFO:tasks.workunit.client.0.smithi027.stdout:0/687: dread d9/da/d31/fb1 [0,4194304] 0 2021-11-03T17:14:25.029 INFO:tasks.workunit.client.0.smithi027.stdout:0/688: stat d9/da/d64/d70/d6d/d7a/l94 0 2021-11-03T17:14:25.029 INFO:tasks.workunit.client.0.smithi027.stdout:0/689: symlink d9/d12/d8f/lee 0 2021-11-03T17:14:25.030 INFO:tasks.workunit.client.0.smithi027.stdout:0/690: fsync d9/da/d30/dac/fbe 0 2021-11-03T17:14:25.030 INFO:tasks.workunit.client.0.smithi027.stdout:0/691: mkdir d9/d12/def 0 2021-11-03T17:14:25.047 INFO:tasks.workunit.client.0.smithi027.stdout:0/692: dwrite d9/da/f39 [0,4194304] 0 2021-11-03T17:14:25.066 INFO:tasks.workunit.client.0.smithi027.stdout:0/693: dwrite d9/da/d30/dac/fcf [0,4194304] 0 2021-11-03T17:14:25.067 INFO:tasks.workunit.client.0.smithi027.stdout:0/694: creat d9/da/d64/d70/d6d/ff0 x:0 0 0 2021-11-03T17:14:25.068 INFO:tasks.workunit.client.0.smithi027.stdout:0/695: mkdir d9/da/d64/d70/d6d/d9f/df1 0 2021-11-03T17:14:25.074 INFO:tasks.workunit.client.0.smithi027.stdout:0/696: dread d9/da/d30/f3c [0,4194304] 0 2021-11-03T17:14:25.074 INFO:tasks.workunit.client.0.smithi027.stdout:0/697: write d9/da/d64/d70/d6f/fe9 [749373,89561] 0 2021-11-03T17:14:25.075 INFO:tasks.workunit.client.0.smithi027.stdout:0/698: mknod d9/da/d64/d70/cf2 1 2021-11-03T17:14:25.096 INFO:tasks.workunit.client.0.smithi027.stdout:0/699: dwrite d9/da/d64/d70/d6d/f6e [0,4194304] 0 2021-11-03T17:14:25.099 INFO:tasks.workunit.client.0.smithi027.stdout:0/700: unlink d9/da/d31/fb1 0 2021-11-03T17:14:25.125 INFO:tasks.workunit.client.0.smithi027.stdout:0/701: dwrite d9/da/d64/d70/d6f/fa1 [0,4194304] 0 2021-11-03T17:14:25.145 INFO:tasks.workunit.client.0.smithi027.stdout:0/702: dwrite d9/da/d64/f76 [0,4194304] 0 2021-11-03T17:14:25.146 INFO:tasks.workunit.client.0.smithi027.stdout:0/703: dread - d9/da/d30/d93/fb2 zero size 2021-11-03T17:14:25.149 INFO:tasks.workunit.client.0.smithi027.stdout:0/704: rename d9/d12/l65 to d9/da/d30/lf3 0 2021-11-03T17:14:25.151 INFO:tasks.workunit.client.0.smithi027.stdout:0/705: unlink d9/f49 0 2021-11-03T17:14:25.152 INFO:tasks.workunit.client.0.smithi027.stdout:0/706: write d9/fbf [569691,55262] 0 2021-11-03T17:14:25.155 INFO:tasks.workunit.client.0.smithi027.stdout:0/707: truncate d9/da/d64/f76 2669972 0 2021-11-03T17:14:25.155 INFO:tasks.workunit.client.0.smithi027.stdout:0/708: chown d9/da/d30/dac/fd4 30376 1 2021-11-03T17:14:25.161 INFO:tasks.workunit.client.0.smithi027.stdout:0/709: rename d9/da/d64/d70/d6d/d7a/l94 to d9/da/d30/d92/dca/lf4 0 2021-11-03T17:14:25.163 INFO:tasks.workunit.client.0.smithi027.stdout:0/710: dread d9/d12/f75 [0,4194304] 0 2021-11-03T17:14:25.163 INFO:tasks.workunit.client.0.smithi027.stdout:0/711: read d9/da/d64/d70/d5f/fd6 [56856,36874] 0 2021-11-03T17:14:25.165 INFO:tasks.workunit.client.0.smithi027.stdout:0/712: creat d9/da/d64/ff5 x:0 0 0 2021-11-03T17:14:25.167 INFO:tasks.workunit.client.0.smithi027.stdout:0/713: getdents d9/da/d30/d92 0 2021-11-03T17:14:25.168 INFO:tasks.workunit.client.0.smithi027.stdout:0/714: read d9/da/d64/d70/d6f/fe9 [707597,20896] 0 2021-11-03T17:14:25.168 INFO:tasks.workunit.client.0.smithi027.stdout:0/715: readlink d9/da/d31/l85 0 2021-11-03T17:14:25.169 INFO:tasks.workunit.client.0.smithi027.stdout:0/716: rename d9/da/d64/d70/d6d/d7a/faa to d9/da/d64/dcc/ff6 0 2021-11-03T17:14:25.171 INFO:tasks.workunit.client.0.smithi027.stdout:0/717: rename d9/lb3 to d9/lf7 0 2021-11-03T17:14:25.191 INFO:tasks.workunit.client.0.smithi027.stdout:0/718: dwrite d9/da/d30/d92/fd2 [0,4194304] 0 2021-11-03T17:14:25.193 INFO:tasks.workunit.client.0.smithi027.stdout:0/719: symlink d9/d12/lf8 0 2021-11-03T17:14:25.194 INFO:tasks.workunit.client.0.smithi027.stdout:0/720: mkdir d9/da/d30/dac/dba/df9 0 2021-11-03T17:14:25.201 INFO:tasks.workunit.client.0.smithi027.stdout:0/721: dread d9/da/d64/d70/d6f/f80 [0,4194304] 0 2021-11-03T17:14:25.203 INFO:tasks.workunit.client.0.smithi027.stdout:0/722: creat d9/ffa x:0 0 0 2021-11-03T17:14:25.204 INFO:tasks.workunit.client.0.smithi027.stdout:0/723: creat d9/d12/def/ffb x:0 0 0 2021-11-03T17:14:25.204 INFO:tasks.workunit.client.0.smithi027.stdout:0/724: readlink d9/da/d31/l58 0 2021-11-03T17:14:25.205 INFO:tasks.workunit.client.0.smithi027.stdout:0/725: chown d9/da/d30/d92/dca/lf4 27932781 1 2021-11-03T17:14:25.206 INFO:tasks.workunit.client.0.smithi027.stdout:0/726: getdents d9/d12/d8f 0 2021-11-03T17:14:25.206 INFO:tasks.workunit.client.0.smithi027.stdout:0/727: fsync d9/f13 0 2021-11-03T17:14:25.207 INFO:tasks.workunit.client.0.smithi027.stdout:0/728: creat d9/dd3/ffc x:0 0 0 2021-11-03T17:14:25.208 INFO:tasks.workunit.client.0.smithi027.stdout:0/729: fsync d9/f72 0 2021-11-03T17:14:25.208 INFO:tasks.workunit.client.0.smithi027.stdout:0/730: write d9/da/d30/d92/fa0 [3515263,92197] 0 2021-11-03T17:14:25.211 INFO:tasks.workunit.client.0.smithi027.stdout:0/731: mkdir d9/dfd 0 2021-11-03T17:14:25.211 INFO:tasks.workunit.client.0.smithi027.stdout:0/732: dread - d9/fce zero size 2021-11-03T17:14:25.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:25 smithi115 conmon[34670]: cephadm 2021-11-03T17:14:23.702135+0000 mgr.smithi115.tcyodz (mgr.24395) 1 : cephadm [INF] 2021-11-03T17:14:25.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:25 smithi115 conmon[34670]: Deploying cephadm binary to smithi115 2021-11-03T17:14:25.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:25 smithi115 conmon[34670]: cephadm 2021-11-03T17:14:23.718821+0000 mgr.smithi115.tcyodz (mgr.24395) 2 : cephadm [INF] Deploying cephadm binary to smithi027 2021-11-03T17:14:25.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:25 smithi115 conmon[34670]: cluster 2021-11-03T17:14:24. 2021-11-03T17:14:25.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:25 smithi115 conmon[34670]: 185673+0000 mon.smithi027 (mon.0) 591 : cluster [DBG] mgrmap e20: smithi115.tcyodz(active, since 1.14278s), standbys: smithi027.ufhhvb 2021-11-03T17:14:25.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:25 smithi115 conmon[34670]: cluster 2021-11-03T17:14:24.195200+0000 2021-11-03T17:14:25.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:25 smithi115 conmon[34670]: mgr.smithi115.tcyodz (mgr.24395) 3 : cluster [DBG] pgmap v3: 65 pgs: 65 active+clean; 2.6 GiB data, 7.7 GiB used, 529 GiB / 536 GiB avail 2021-11-03T17:14:25.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:25 smithi115 conmon[34670]: audit 2021-11-03T17:14:25.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:25 smithi115 conmon[34670]: 2021-11-03T17:14:24.220691+0000 mon.smithi027 (mon.0) 592 : audit 2021-11-03T17:14:25.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:25 smithi115 conmon[34670]: [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' cmd='[{"prefix": "osd pool rename", "format": "json", "srcpool": "device_health_metrics", "destpool": ".mgr"}]': finished 2021-11-03T17:14:25.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:25 smithi115 conmon[34670]: cluster 2021 2021-11-03T17:14:25.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:25 smithi115 conmon[34670]: -11-03T17:14:24.220860+0000 mon.smithi027 (mon.0) 593 : cluster [DBG] 2021-11-03T17:14:25.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:25 smithi115 conmon[34670]: osdmap e47: 6 total, 6 up, 6 in 2021-11-03T17:14:25.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:25 smithi115 conmon[34670]: audit 2021- 2021-11-03T17:14:25.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:25 smithi115 conmon[34670]: 11-03T17:14:24.227753+0000 mon.smithi115 (mon.1) 43 : audit [INF] 2021-11-03T17:14:25.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:25 smithi115 conmon[34670]: from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2021-11-03T17:14:25.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:25 smithi115 conmon[34670]: audit 2021- 2021-11-03T17:14:25.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:25 smithi115 conmon[34670]: 11-03T17:14:24.228421+0000 mon.smithi027 (mon.0) 594 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2021-11-03T17:14:25.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:25 smithi115 conmon[34670]: cephadm 2021-11-03T17: 2021-11-03T17:14:25.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:25 smithi115 conmon[34670]: 14:24.521786+0000 mgr.smithi115.tcyodz (mgr.24395) 4 : cephadm [INF] Starting cherrypy engine... 2021-11-03T17:14:25.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:25 smithi115 conmon[34670]: cephadm 2021-11-03T17:14 2021-11-03T17:14:25.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:25 smithi115 conmon[34670]: :24.521987+0000 mgr.smithi115.tcyodz (mgr.24395) 5 : cephadm [INF] [03/Nov/2021:17:14:24] ENGINE Bus STARTING 2021-11-03T17:14:25.254 INFO:tasks.workunit.client.0.smithi027.stdout:0/733: dwrite d9/da/d64/d70/d6f/fa1 [0,4194304] 0 2021-11-03T17:14:25.256 INFO:tasks.workunit.client.0.smithi027.stdout:0/734: rename d9/da/d64/d70/d6d/d9f/df1 to d9/da/d30/dfe 0 2021-11-03T17:14:25.258 INFO:tasks.workunit.client.0.smithi027.stdout:0/735: dread d9/d12/f75 [0,4194304] 0 2021-11-03T17:14:25.260 INFO:tasks.workunit.client.0.smithi027.stdout:0/736: link d9/da/d31/l88 d9/d12/def/lff 0 2021-11-03T17:14:25.260 INFO:tasks.workunit.client.0.smithi027.stdout:0/737: read d9/da/d64/d70/d6d/d7a/fa9 [1085587,82159] 0 2021-11-03T17:14:25.261 INFO:tasks.workunit.client.0.smithi027.stdout:0/738: symlink d9/da/d30/dac/l100 0 2021-11-03T17:14:25.263 INFO:tasks.workunit.client.0.smithi027.stdout:0/739: symlink d9/d12/d8f/dc7/l101 0 2021-11-03T17:14:25.270 INFO:tasks.workunit.client.0.smithi027.stdout:0/740: write d9/f4b [3512874,31854] 0 2021-11-03T17:14:25.270 INFO:tasks.workunit.client.0.smithi027.stdout:0/741: fdatasync d9/d12/f75 0 2021-11-03T17:14:25.277 INFO:tasks.workunit.client.0.smithi027.stdout:0/742: dread d9/f72 [4194304,4194304] 0 2021-11-03T17:14:25.286 INFO:tasks.workunit.client.0.smithi027.stdout:0/743: dread d9/da/f8b [0,4194304] 0 2021-11-03T17:14:25.286 INFO:tasks.workunit.client.0.smithi027.stdout:0/744: truncate d9/d12/def/ffb 528052 0 2021-11-03T17:14:25.290 INFO:tasks.workunit.client.0.smithi027.stdout:0/745: dread d9/da/d64/d70/fc5 [0,4194304] 0 2021-11-03T17:14:25.290 INFO:tasks.workunit.client.0.smithi027.stdout:0/746: write d9/fb4 [454981,7408] 0 2021-11-03T17:14:25.297 INFO:tasks.workunit.client.0.smithi027.stdout:0/747: creat d9/da/d64/d70/d6d/d7a/dc9/f102 x:0 0 0 2021-11-03T17:14:25.299 INFO:tasks.workunit.client.0.smithi027.stdout:0/748: creat d9/d12/def/f103 x:0 0 0 2021-11-03T17:14:25.299 INFO:tasks.workunit.client.0.smithi027.stdout:0/749: mknod d9/da/d64/d70/d6d/c104 1 2021-11-03T17:14:25.300 INFO:tasks.workunit.client.0.smithi027.stdout:0/750: symlink d9/da/d30/da7/l105 0 2021-11-03T17:14:25.302 INFO:tasks.workunit.client.0.smithi027.stdout:0/751: rmdir d9/da/d30/dac/dba/df9 0 2021-11-03T17:14:25.311 INFO:tasks.workunit.client.0.smithi027.stdout:0/752: truncate d9/da/d30/f5e 4058400 0 2021-11-03T17:14:25.312 INFO:tasks.workunit.client.0.smithi027.stdout:0/753: read d9/da/d30/d92/fa0 [418934,88841] 0 2021-11-03T17:14:25.318 INFO:tasks.workunit.client.0.smithi027.stdout:0/754: rmdir d9/da/d30/d92/dca/de6 0 2021-11-03T17:14:25.320 INFO:tasks.workunit.client.0.smithi027.stdout:0/755: dread d9/d12/f28 [0,4194304] 0 2021-11-03T17:14:25.321 INFO:tasks.workunit.client.0.smithi027.stdout:0/756: rmdir d9/da/d30/d93 39 2021-11-03T17:14:25.321 INFO:tasks.workunit.client.0.smithi027.stdout:0/757: mknod d9/da/d30/d93/c106 1 2021-11-03T17:14:25.330 INFO:tasks.workunit.client.0.smithi027.stdout:0/758: write d9/da/d64/d70/fdb [5964141,779] 0 2021-11-03T17:14:25.332 INFO:tasks.workunit.client.0.smithi027.stdout:0/759: write d9/f6c [1847324,25216] 0 2021-11-03T17:14:25.333 INFO:tasks.workunit.client.0.smithi027.stdout:0/760: rmdir d9/da/d30/d92 39 2021-11-03T17:14:25.334 INFO:tasks.workunit.client.0.smithi027.stdout:0/761: chown d9/da/f78 6914214 1 2021-11-03T17:14:25.334 INFO:tasks.workunit.client.0.smithi027.stdout:0/762: write d9/da/d64/d70/d5f/f74 [3855753,112984] 0 2021-11-03T17:14:25.337 INFO:tasks.workunit.client.0.smithi027.stdout:0/763: truncate d9/f13 2908852 0 2021-11-03T17:14:25.348 INFO:tasks.workunit.client.0.smithi027.stdout:0/764: dread d9/f1f [0,4194304] 0 2021-11-03T17:14:25.350 INFO:tasks.workunit.client.0.smithi027.stdout:0/765: dread d9/da/d30/f54 [0,4194304] 0 2021-11-03T17:14:25.351 INFO:tasks.workunit.client.0.smithi027.stdout:0/766: creat d9/da/d64/d70/d5f/f107 x:0 0 0 2021-11-03T17:14:25.352 INFO:tasks.workunit.client.0.smithi027.stdout:0/767: creat d9/dd3/f108 x:0 0 0 2021-11-03T17:14:25.352 INFO:tasks.workunit.client.0.smithi027.stdout:0/768: fsync d9/d12/f75 0 2021-11-03T17:14:25.353 INFO:tasks.workunit.client.0.smithi027.stdout:0/769: dread - d9/fce zero size 2021-11-03T17:14:25.354 INFO:tasks.workunit.client.0.smithi027.stdout:0/770: dread d9/da/d64/d70/d5f/fd6 [0,4194304] 0 2021-11-03T17:14:25.379 INFO:tasks.workunit.client.0.smithi027.stdout:0/771: dwrite d9/fc1 [0,4194304] 0 2021-11-03T17:14:25.382 INFO:tasks.workunit.client.0.smithi027.stdout:0/772: truncate d9/da/d64/d70/d6f/fe9 23876 0 2021-11-03T17:14:25.384 INFO:tasks.workunit.client.0.smithi027.stdout:0/773: creat d9/d12/f109 x:0 0 0 2021-11-03T17:14:25.384 INFO:tasks.workunit.client.0.smithi027.stdout:0/774: mknod d9/da/d30/dfe/c10a 1 2021-11-03T17:14:25.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:25 smithi027 conmon[32322]: cephadm 2021-11-03T17:14:23.702135+0000 mgr.smithi115.tcyodz (mgr.24395) 1 : 2021-11-03T17:14:25.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:25 smithi027 conmon[32322]: cephadm [INF] Deploying cephadm binary to smithi115 2021-11-03T17:14:25.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:25 smithi027 conmon[32322]: cephadm 2021-11-03T17:14:23.718821+0000 mgr.smithi115.tcyodz (mgr.24395) 2 : cephadm [INF] Deploying cephadm binary to smithi027 2021-11-03T17:14:25.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:25 smithi027 conmon[32322]: cluster 2021-11-03T17:14:24.185673+0000 mon.smithi027 (mon.0) 591 : cluster [DBG] mgrmap e20: smithi115.tcyodz(active, since 1.14278s), standbys: smithi027.ufhhvb 2021-11-03T17:14:25.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:25 smithi027 conmon[32322]: cluster 2021-11-03T17:14:24.195200+0000 mgr.smithi115.tcyodz (mgr.24395) 3 : cluster [DBG] pgmap v3: 65 pgs: 65 active+clean; 2.6 GiB data, 7.7 GiB used, 529 GiB / 536 GiB avail 2021-11-03T17:14:25.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:25 smithi027 conmon[32322]: 2021-11-03T17:14:25.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:25 smithi027 conmon[32322]: audit 2021-11-03T17:14:24.220691+0000 mon.smithi027 (mon.0) 592 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' cmd='[{"prefix": "osd pool rename", "format": "json", "srcpool": "device_health_metrics", "destpool": ".mgr"}]': finished 2021-11-03T17:14:25.524 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:25 smithi027 conmon[32322]: cluster 2021-11-03T17: 2021-11-03T17:14:25.524 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:25 smithi027 conmon[32322]: 14:24.220860+0000 mon.smithi027 (mon.0) 593 : cluster 2021-11-03T17:14:25.524 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:25 smithi027 conmon[32322]: [DBG] osdmap e47: 6 total, 6 up, 6 in 2021-11-03T17:14:25.524 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:25 smithi027 conmon[32322]: audit 2021-11-03T17:14:25.525 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:25 smithi027 conmon[32322]: 2021-11-03T17:14:24.227753+0000 mon.smithi115 2021-11-03T17:14:25.525 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:25 smithi027 conmon[32322]: (mon.1) 43 : audit [INF] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2021-11-03T17:14:25.525 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:25 smithi027 conmon[32322]: audit 2021-11- 2021-11-03T17:14:25.525 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:25 smithi027 conmon[32322]: 03T17:14:24.228421+0000 mon.smithi027 (mon. 2021-11-03T17:14:25.526 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:25 smithi027 conmon[32322]: 0) 594 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2021-11-03T17:14:25.526 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:25 smithi027 conmon[32322]: cephadm 2021-11-03 2021-11-03T17:14:25.526 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:25 smithi027 conmon[32322]: T17:14:24.521786+0000 mgr.smithi115.tcyodz (mgr.24395 2021-11-03T17:14:25.526 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:25 smithi027 conmon[32322]: ) 4 : cephadm [INF] Starting cherrypy engine... 2021-11-03T17:14:25.526 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:25 smithi027 conmon[32322]: cephadm 2021-11-03T17:14 2021-11-03T17:14:25.527 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:25 smithi027 conmon[32322]: :24.521987+0000 mgr.smithi115.tcyodz (mgr.24395) 5 : cephadm 2021-11-03T17:14:25.527 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:25 smithi027 conmon[32322]: [INF] [03/Nov/2021:17:14:24] ENGINE Bus STARTING 2021-11-03T17:14:25.678 INFO:tasks.workunit.client.0.smithi027.stdout:1/653: sync 2021-11-03T17:14:25.679 INFO:tasks.workunit.client.0.smithi027.stdout:8/724: sync 2021-11-03T17:14:25.680 INFO:tasks.workunit.client.0.smithi027.stdout:1/654: write d6/d11/dac/d79/d25/d28/d5c/f78 [6626252,95398] 0 2021-11-03T17:14:25.681 INFO:tasks.workunit.client.0.smithi027.stdout:1/655: chown d6/d11/dac/d79/d25/d28/d5c/d81/l8c 97045847 1 2021-11-03T17:14:25.681 INFO:tasks.workunit.client.0.smithi027.stdout:1/656: fdatasync d6/d11/dac/d79/d25/d28/d5c/d7a/fbe 0 2021-11-03T17:14:25.685 INFO:tasks.workunit.client.0.smithi027.stdout:1/657: rename d6/d11/dac/d79/d25/d28/d5c/fc9 to d6/d11/dac/d79/db0/de5/fe8 0 2021-11-03T17:14:25.685 INFO:tasks.workunit.client.0.smithi027.stdout:1/658: write d6/d11/dac/d79/d25/d28/d5c/d81/dd5/fda [1029635,69546] 0 2021-11-03T17:14:25.686 INFO:tasks.workunit.client.0.smithi027.stdout:1/659: creat d6/da7/fe9 x:0 0 0 2021-11-03T17:14:25.687 INFO:tasks.workunit.client.0.smithi027.stdout:1/660: chown d6/d11/dac/d79/d25/d28/f3c 6697 1 2021-11-03T17:14:25.688 INFO:tasks.workunit.client.0.smithi027.stdout:1/661: truncate d6/d11/dac/d79/d37/de0/de4/fa0 797796 0 2021-11-03T17:14:25.688 INFO:tasks.workunit.client.0.smithi027.stdout:1/662: chown d6/d11/dac/d79/d25/f47 26363810 1 2021-11-03T17:14:25.700 INFO:tasks.workunit.client.0.smithi027.stdout:8/725: dwrite d7/fb8 [0,4194304] 0 2021-11-03T17:14:25.708 INFO:tasks.workunit.client.0.smithi027.stdout:8/726: unlink d7/d10/d3f/d55/f86 0 2021-11-03T17:14:25.710 INFO:tasks.workunit.client.0.smithi027.stdout:1/663: dread d6/d11/dac/d79/d25/d28/d5c/d81/d7d/fca [0,4194304] 0 2021-11-03T17:14:25.712 INFO:tasks.workunit.client.0.smithi027.stdout:8/727: creat d7/d10/d21/d41/fef x:0 0 0 2021-11-03T17:14:25.718 INFO:tasks.workunit.client.0.smithi027.stdout:8/728: rename d7/d10/d18 to d7/d2d/d38/db1/df0 0 2021-11-03T17:14:25.721 INFO:tasks.workunit.client.0.smithi027.stdout:1/664: dread d6/de/d13/f6b [0,4194304] 0 2021-11-03T17:14:25.721 INFO:tasks.workunit.client.0.smithi027.stdout:1/665: read d6/d11/dac/d79/d25/d28/d5c/d81/f86 [1568734,58379] 0 2021-11-03T17:14:25.722 INFO:tasks.workunit.client.0.smithi027.stdout:1/666: truncate d6/d11/dac/d79/d25/d28/d5c/d81/f85 850765 0 2021-11-03T17:14:25.744 INFO:tasks.workunit.client.0.smithi027.stdout:1/667: dwrite d6/d11/dac/d79/d25/d5e/fcc [0,4194304] 0 2021-11-03T17:14:25.748 INFO:tasks.workunit.client.0.smithi027.stdout:8/729: dwrite d7/d10/d3f/d9b/d95/f9a [0,4194304] 0 2021-11-03T17:14:25.749 INFO:tasks.workunit.client.0.smithi027.stdout:8/730: mknod d7/d10/d21/d7f/dd9/cf1 1 2021-11-03T17:14:25.749 INFO:tasks.workunit.client.0.smithi027.stdout:8/731: chown d7/d10/d3f/l59 267827 1 2021-11-03T17:14:25.751 INFO:tasks.workunit.client.0.smithi027.stdout:8/732: link d7/d10/d3f/f78 d7/d10/d3f/d55/ff2 0 2021-11-03T17:14:25.753 INFO:tasks.workunit.client.0.smithi027.stdout:8/733: creat d7/d2d/ff3 x:0 0 0 2021-11-03T17:14:25.754 INFO:tasks.workunit.client.0.smithi027.stdout:8/734: chown d7/fb8 429606022 1 2021-11-03T17:14:25.765 INFO:tasks.workunit.client.0.smithi027.stdout:1/668: dwrite d6/d11/dac/d79/d37/de0/f2f [4194304,4194304] 0 2021-11-03T17:14:25.775 INFO:tasks.workunit.client.0.smithi027.stdout:8/735: dwrite d7/d10/faa [0,4194304] 0 2021-11-03T17:14:25.779 INFO:tasks.workunit.client.0.smithi027.stdout:8/736: rmdir d7/d53/d67 39 2021-11-03T17:14:25.780 INFO:tasks.workunit.client.0.smithi027.stdout:8/737: creat d7/ff4 x:0 0 0 2021-11-03T17:14:25.785 INFO:tasks.workunit.client.0.smithi027.stdout:8/738: dread d7/d10/d21/f25 [0,4194304] 0 2021-11-03T17:14:25.786 INFO:tasks.workunit.client.0.smithi027.stdout:8/739: chown d7/d10 1485782 1 2021-11-03T17:14:25.788 INFO:tasks.workunit.client.0.smithi027.stdout:8/740: getdents d7/d2d 0 2021-11-03T17:14:25.789 INFO:tasks.workunit.client.0.smithi027.stdout:8/741: rmdir d7/d10/d3f/d55/db7/dc9 39 2021-11-03T17:14:25.792 INFO:tasks.workunit.client.0.smithi027.stdout:1/669: dwrite d6/d11/dac/d79/f22 [4194304,4194304] 0 2021-11-03T17:14:25.795 INFO:tasks.workunit.client.0.smithi027.stdout:1/670: symlink d6/lea 0 2021-11-03T17:14:25.807 INFO:tasks.workunit.client.0.smithi027.stdout:8/742: dwrite d7/d10/d3f/d55/ff2 [0,4194304] 0 2021-11-03T17:14:25.808 INFO:tasks.workunit.client.0.smithi027.stdout:8/743: write d7/d10/d21/f45 [985443,13396] 0 2021-11-03T17:14:25.823 INFO:tasks.workunit.client.0.smithi027.stdout:1/671: dwrite d6/d11/dac/d79/d25/d28/d5c/d81/f86 [0,4194304] 0 2021-11-03T17:14:25.823 INFO:tasks.workunit.client.0.smithi027.stdout:1/672: read - d6/d11/dac/d79/d37/d49/f5b zero size 2021-11-03T17:14:25.828 INFO:tasks.workunit.client.0.smithi027.stdout:8/744: dwrite d7/d10/d3f/f88 [0,4194304] 0 2021-11-03T17:14:25.834 INFO:tasks.workunit.client.0.smithi027.stdout:8/745: symlink d7/d10/lf5 0 2021-11-03T17:14:25.834 INFO:tasks.workunit.client.0.smithi027.stdout:8/746: write d7/d10/f89 [178215,86935] 0 2021-11-03T17:14:25.834 INFO:tasks.workunit.client.0.smithi027.stdout:8/747: symlink d7/d10/d21/lf6 0 2021-11-03T17:14:25.835 INFO:tasks.workunit.client.0.smithi027.stdout:8/748: write d7/d10/fa1 [3579716,115917] 0 2021-11-03T17:14:25.835 INFO:tasks.workunit.client.0.smithi027.stdout:8/749: dread - d7/d2d/d38/db1/fe5 zero size 2021-11-03T17:14:25.836 INFO:tasks.workunit.client.0.smithi027.stdout:8/750: mknod d7/d10/d21/cf7 1 2021-11-03T17:14:25.837 INFO:tasks.workunit.client.0.smithi027.stdout:8/751: creat d7/d2d/d38/db1/da5/ff8 x:0 0 0 2021-11-03T17:14:25.838 INFO:tasks.workunit.client.0.smithi027.stdout:8/752: symlink d7/d2d/d38/db1/df0/lf9 0 2021-11-03T17:14:25.840 INFO:tasks.workunit.client.0.smithi027.stdout:8/753: rmdir d7/d10/dbd/de3 0 2021-11-03T17:14:25.840 INFO:tasks.workunit.client.0.smithi027.stdout:8/754: read - d7/ff4 zero size 2021-11-03T17:14:25.842 INFO:tasks.workunit.client.0.smithi027.stdout:8/755: link d7/d10/d3f/f73 d7/d10/d3f/ffa 0 2021-11-03T17:14:25.843 INFO:tasks.workunit.client.0.smithi027.stdout:8/756: mknod d7/d10/d3f/d55/cfb 1 2021-11-03T17:14:25.844 INFO:tasks.workunit.client.0.smithi027.stdout:8/757: mkdir d7/d10/d3f/d9b/dfc 0 2021-11-03T17:14:25.844 INFO:tasks.workunit.client.0.smithi027.stdout:8/758: chown d7/d10/d3f/d9b/d95 4 1 2021-11-03T17:14:25.844 INFO:tasks.workunit.client.0.smithi027.stdout:8/759: mknod d7/d10/d3f/d9b/cfd 1 2021-11-03T17:14:25.845 INFO:tasks.workunit.client.0.smithi027.stdout:8/760: mkdir d7/dcc/dfe 0 2021-11-03T17:14:25.846 INFO:tasks.workunit.client.0.smithi027.stdout:8/761: mkdir d7/dff 0 2021-11-03T17:14:25.847 INFO:tasks.workunit.client.0.smithi027.stdout:1/673: dwrite d6/d11/dac/d79/d25/d28/d41/fd8 [0,4194304] 0 2021-11-03T17:14:25.848 INFO:tasks.workunit.client.0.smithi027.stdout:1/674: write d6/d11/dac/d79/d25/d28/d5c/d81/dd5/fda [208210,20499] 0 2021-11-03T17:14:25.856 INFO:tasks.workunit.client.0.smithi027.stdout:8/762: dread d7/d2d/d38/db1/df0/f49 [0,4194304] 0 2021-11-03T17:14:25.856 INFO:tasks.workunit.client.0.smithi027.stdout:8/763: getdents d7/d10/d3f/d9b/dfc 0 2021-11-03T17:14:25.858 INFO:tasks.workunit.client.0.smithi027.stdout:8/764: creat d7/dcc/dfe/f100 x:0 0 0 2021-11-03T17:14:25.858 INFO:tasks.workunit.client.0.smithi027.stdout:1/675: dread d6/d11/dac/d79/d25/d28/d5c/f73 [0,4194304] 0 2021-11-03T17:14:25.859 INFO:tasks.workunit.client.0.smithi027.stdout:1/676: write d6/d11/dac/d79/d25/d28/d32/fae [328243,88141] 0 2021-11-03T17:14:25.869 INFO:tasks.workunit.client.0.smithi027.stdout:7/437: sync 2021-11-03T17:14:25.869 INFO:tasks.workunit.client.0.smithi027.stdout:5/767: sync 2021-11-03T17:14:25.869 INFO:tasks.workunit.client.0.smithi027.stdout:7/438: write d0/d2d/d58/f7f [1704920,106903] 0 2021-11-03T17:14:25.870 INFO:tasks.workunit.client.0.smithi027.stdout:7/439: stat d0/d2/d1c/d4d/d6e 0 2021-11-03T17:14:25.871 INFO:tasks.workunit.client.0.smithi027.stdout:7/440: truncate d0/d2d/d58/f69 1940382 0 2021-11-03T17:14:25.872 INFO:tasks.workunit.client.0.smithi027.stdout:7/441: getdents d0/d2/d5e/d36 0 2021-11-03T17:14:25.885 INFO:tasks.workunit.client.0.smithi027.stdout:8/765: dwrite d7/d2d/d38/f3d [0,4194304] 0 2021-11-03T17:14:25.885 INFO:tasks.workunit.client.0.smithi027.stdout:8/766: chown d7/d10/d3f/d9b/d95/fd5 866524987 1 2021-11-03T17:14:25.887 INFO:tasks.workunit.client.0.smithi027.stdout:8/767: unlink d7/d53/fa4 0 2021-11-03T17:14:25.890 INFO:tasks.workunit.client.0.smithi027.stdout:1/677: dwrite d6/d11/dac/d79/d25/d28/d5c/d81/dd5/fda [0,4194304] 0 2021-11-03T17:14:25.891 INFO:tasks.workunit.client.0.smithi027.stdout:1/678: chown d6/d11/dac/d79/d37/de0/de4/d58 2026 1 2021-11-03T17:14:25.895 INFO:tasks.workunit.client.0.smithi027.stdout:1/679: symlink d6/d11/dac/d79/d25/d5e/dd1/leb 0 2021-11-03T17:14:25.898 INFO:tasks.workunit.client.0.smithi027.stdout:1/680: rename d6/da7 to d6/d11/dac/d79/d25/d28/dec 0 2021-11-03T17:14:25.898 INFO:tasks.workunit.client.0.smithi027.stdout:1/681: chown d6/d11 11 1 2021-11-03T17:14:25.901 INFO:tasks.workunit.client.0.smithi027.stdout:5/768: dwrite d2/d4/d4e/d67/fe2 [0,4194304] 0 2021-11-03T17:14:25.902 INFO:tasks.workunit.client.0.smithi027.stdout:5/769: fdatasync d2/d12/dad/da8/fc4 0 2021-11-03T17:14:25.907 INFO:tasks.workunit.client.0.smithi027.stdout:5/770: unlink d2/d4/d4e/d67/f82 0 2021-11-03T17:14:25.907 INFO:tasks.workunit.client.0.smithi027.stdout:5/771: chown d2/d2c/d3c/l54 438744 1 2021-11-03T17:14:25.908 INFO:tasks.workunit.client.0.smithi027.stdout:7/442: dwrite d0/d2/d10/d1e/d20/f43 [0,4194304] 0 2021-11-03T17:14:25.908 INFO:tasks.workunit.client.0.smithi027.stdout:7/443: chown d0/d2/d10/d1e 8 1 2021-11-03T17:14:25.908 INFO:tasks.workunit.client.0.smithi027.stdout:7/444: dread - d0/d2/d59/f73 zero size 2021-11-03T17:14:25.913 INFO:tasks.workunit.client.0.smithi027.stdout:5/772: creat d2/d2c/d3c/d48/d65/d80/d9e/ffb x:0 0 0 2021-11-03T17:14:25.914 INFO:tasks.workunit.client.0.smithi027.stdout:8/768: dwrite d7/f87 [0,4194304] 0 2021-11-03T17:14:25.923 INFO:tasks.workunit.client.0.smithi027.stdout:7/445: getdents d0/d2d/d58 0 2021-11-03T17:14:25.932 INFO:tasks.workunit.client.0.smithi027.stdout:5/773: rename d2/d4/fd5 to d2/d2c/d3c/d48/ffc 0 2021-11-03T17:14:25.934 INFO:tasks.workunit.client.0.smithi027.stdout:8/769: creat d7/dcc/dcf/f101 x:0 0 0 2021-11-03T17:14:25.935 INFO:tasks.workunit.client.0.smithi027.stdout:7/446: symlink d0/d2/l9a 0 2021-11-03T17:14:25.942 INFO:tasks.workunit.client.0.smithi027.stdout:5/774: dread d2/d4/d61/f89 [0,4194304] 0 2021-11-03T17:14:25.953 INFO:tasks.workunit.client.0.smithi027.stdout:8/770: rmdir d7 39 2021-11-03T17:14:25.954 INFO:tasks.workunit.client.0.smithi027.stdout:7/447: dwrite d0/d2/d1c/f79 [0,4194304] 0 2021-11-03T17:14:25.954 INFO:tasks.workunit.client.0.smithi027.stdout:7/448: mknod d0/d2/d5e/d75/c9b 1 2021-11-03T17:14:25.957 INFO:tasks.workunit.client.0.smithi027.stdout:7/449: write d0/f4c [192919,59324] 0 2021-11-03T17:14:25.957 INFO:tasks.workunit.client.0.smithi027.stdout:7/450: dread - d0/d2/d10/d35/f7a zero size 2021-11-03T17:14:25.958 INFO:tasks.workunit.client.0.smithi027.stdout:5/775: fdatasync d2/d2c/d3c/d48/d65/d80/fa3 0 2021-11-03T17:14:25.959 INFO:tasks.workunit.client.0.smithi027.stdout:5/776: mknod d2/d2c/d3c/d48/d65/cfd 1 2021-11-03T17:14:25.978 INFO:tasks.workunit.client.0.smithi027.stdout:8/771: dwrite d7/d10/dbd/fd2 [0,4194304] 0 2021-11-03T17:14:25.982 INFO:tasks.workunit.client.0.smithi027.stdout:8/772: creat d7/d53/f102 x:0 0 0 2021-11-03T17:14:25.987 INFO:tasks.workunit.client.0.smithi027.stdout:8/773: dread d7/d10/d21/f39 [0,4194304] 0 2021-11-03T17:14:25.990 INFO:tasks.workunit.client.0.smithi027.stdout:8/774: rename d7/d2d/d38/db1 to d7/d10/d3f/d55/db7/dc9/d103 0 2021-11-03T17:14:25.991 INFO:tasks.workunit.client.0.smithi027.stdout:8/775: dread - d7/dcc/dfe/f100 zero size 2021-11-03T17:14:25.991 INFO:tasks.workunit.client.0.smithi027.stdout:8/776: truncate d7/d10/dbd/fc1 522611 0 2021-11-03T17:14:25.992 INFO:tasks.workunit.client.0.smithi027.stdout:5/777: dwrite d2/d2c/d3c/f84 [0,4194304] 0 2021-11-03T17:14:25.993 INFO:tasks.workunit.client.0.smithi027.stdout:7/451: dwrite d0/d2/d1f/f54 [0,4194304] 0 2021-11-03T17:14:26.003 INFO:tasks.workunit.client.0.smithi027.stdout:5/778: dread d2/d12/f5c [0,4194304] 0 2021-11-03T17:14:26.003 INFO:tasks.workunit.client.0.smithi027.stdout:5/779: stat d2/d4/f66 0 2021-11-03T17:14:26.004 INFO:tasks.workunit.client.0.smithi027.stdout:5/780: fdatasync d2/d12/d7d/da7/fac 0 2021-11-03T17:14:26.004 INFO:tasks.workunit.client.0.smithi027.stdout:5/781: dread - d2/d1f/fbc zero size 2021-11-03T17:14:26.004 INFO:tasks.workunit.client.0.smithi027.stdout:5/782: creat d2/ffe x:0 0 0 2021-11-03T17:14:26.016 INFO:tasks.workunit.client.0.smithi027.stdout:8/777: dwrite d7/d10/fa1 [0,4194304] 0 2021-11-03T17:14:26.017 INFO:tasks.workunit.client.0.smithi027.stdout:7/452: dwrite d0/f13 [4194304,4194304] 0 2021-11-03T17:14:26.019 INFO:tasks.workunit.client.0.smithi027.stdout:8/778: read d7/f62 [2768439,77430] 0 2021-11-03T17:14:26.019 INFO:tasks.workunit.client.0.smithi027.stdout:8/779: dread - d7/d10/d21/d41/fef zero size 2021-11-03T17:14:26.019 INFO:tasks.workunit.client.0.smithi027.stdout:5/783: dread d2/d4/f34 [0,4194304] 0 2021-11-03T17:14:26.019 INFO:tasks.workunit.client.0.smithi027.stdout:5/784: write d2/d4/d4e/f4f [1392760,28030] 0 2021-11-03T17:14:26.020 INFO:tasks.workunit.client.0.smithi027.stdout:7/453: write d0/d2/d10/d1e/d20/f26 [499438,47006] 0 2021-11-03T17:14:26.027 INFO:tasks.workunit.client.0.smithi027.stdout:8/780: unlink d7/d10/d21/d41/l5b 0 2021-11-03T17:14:26.031 INFO:tasks.workunit.client.0.smithi027.stdout:5/785: symlink d2/d12/dad/da9/lff 0 2021-11-03T17:14:26.038 INFO:tasks.workunit.client.0.smithi027.stdout:7/454: rmdir d0 39 2021-11-03T17:14:26.038 INFO:tasks.workunit.client.0.smithi027.stdout:8/781: rename d7/l5a to d7/d53/l104 0 2021-11-03T17:14:26.038 INFO:tasks.workunit.client.0.smithi027.stdout:7/455: chown d0/d2/d10/d1e 17448276 1 2021-11-03T17:14:26.038 INFO:tasks.workunit.client.0.smithi027.stdout:7/456: stat d0/d2/d1f/f46 0 2021-11-03T17:14:26.039 INFO:tasks.workunit.client.0.smithi027.stdout:8/782: readlink d7/l5c 0 2021-11-03T17:14:26.039 INFO:tasks.workunit.client.0.smithi027.stdout:5/786: chown d2/d2c/d3c/d48/d65/d80/ld7 241666224 1 2021-11-03T17:14:26.039 INFO:tasks.workunit.client.0.smithi027.stdout:5/787: dread - d2/d2c/d3c/fdd zero size 2021-11-03T17:14:26.040 INFO:tasks.workunit.client.0.smithi027.stdout:8/783: symlink d7/d2d/d38/l105 0 2021-11-03T17:14:26.042 INFO:tasks.workunit.client.0.smithi027.stdout:8/784: creat d7/d10/d3f/f106 x:0 0 0 2021-11-03T17:14:26.059 INFO:tasks.workunit.client.0.smithi027.stdout:5/788: dread d2/d4/d4e/d67/d20/f3a [0,4194304] 0 2021-11-03T17:14:26.065 INFO:tasks.workunit.client.0.smithi027.stdout:5/789: creat d2/d2c/d3c/d58/f100 x:0 0 0 2021-11-03T17:14:26.065 INFO:tasks.workunit.client.0.smithi027.stdout:5/790: fsync d2/d4/d4e/d67/d20/f73 0 2021-11-03T17:14:26.067 INFO:tasks.workunit.client.0.smithi027.stdout:5/791: write d2/f1e [3126297,26976] 0 2021-11-03T17:14:26.070 INFO:tasks.workunit.client.0.smithi027.stdout:5/792: write d2/d2c/d3c/d48/d65/f56 [2868808,60259] 0 2021-11-03T17:14:26.072 INFO:tasks.workunit.client.0.smithi027.stdout:7/457: dwrite d0/d2d/d2f/f39 [0,4194304] 0 2021-11-03T17:14:26.086 INFO:tasks.workunit.client.0.smithi027.stdout:8/785: dwrite d7/d10/d21/d41/fa3 [0,4194304] 0 2021-11-03T17:14:26.086 INFO:tasks.workunit.client.0.smithi027.stdout:8/786: write d7/dcc/f9c [4542364,100455] 0 2021-11-03T17:14:26.088 INFO:tasks.workunit.client.0.smithi027.stdout:5/793: rmdir d2/d12/dad/da9 39 2021-11-03T17:14:26.093 INFO:tasks.workunit.client.0.smithi027.stdout:5/794: write d2/f3e [4626847,84865] 0 2021-11-03T17:14:26.100 INFO:tasks.workunit.client.0.smithi027.stdout:5/795: creat d2/d12/d7d/f101 x:0 0 0 2021-11-03T17:14:26.100 INFO:tasks.workunit.client.0.smithi027.stdout:5/796: dread - d2/d12/d7d/da7/ff8 zero size 2021-11-03T17:14:26.100 INFO:tasks.workunit.client.0.smithi027.stdout:7/458: dwrite d0/d2/d10/f1b [0,4194304] 0 2021-11-03T17:14:26.102 INFO:tasks.workunit.client.0.smithi027.stdout:5/797: symlink d2/d12/dad/d6a/l102 0 2021-11-03T17:14:26.103 INFO:tasks.workunit.client.0.smithi027.stdout:5/798: write d2/d4/d4e/d67/d20/f73 [1357357,75525] 0 2021-11-03T17:14:26.104 INFO:tasks.workunit.client.0.smithi027.stdout:7/459: mkdir d0/d2/d5e/d36/d3a/d9c 0 2021-11-03T17:14:26.106 INFO:tasks.workunit.client.0.smithi027.stdout:5/799: truncate d2/d4/f6b 2594905 0 2021-11-03T17:14:26.109 INFO:tasks.workunit.client.0.smithi027.stdout:7/460: creat d0/d2/d10/d1e/d20/d74/f9d x:0 0 0 2021-11-03T17:14:26.112 INFO:tasks.workunit.client.0.smithi027.stdout:5/800: creat d2/d4/d4e/d67/d20/f103 x:0 0 0 2021-11-03T17:14:26.112 INFO:tasks.workunit.client.0.smithi027.stdout:5/801: write d2/d2c/d3c/d48/d65/d80/d9e/ff6 [396231,96444] 0 2021-11-03T17:14:26.112 INFO:tasks.workunit.client.0.smithi027.stdout:5/802: dread - d2/d1f/fe9 zero size 2021-11-03T17:14:26.114 INFO:tasks.workunit.client.0.smithi027.stdout:5/803: symlink d2/d2c/d3c/d48/d65/d80/dd2/l104 0 2021-11-03T17:14:26.116 INFO:tasks.workunit.client.0.smithi027.stdout:5/804: dread d2/d2c/d3c/d48/d65/d80/f83 [0,4194304] 0 2021-11-03T17:14:26.116 INFO:tasks.workunit.client.0.smithi027.stdout:5/805: chown d2/d4/d4e/d67/d20/fb9 19 1 2021-11-03T17:14:26.117 INFO:tasks.workunit.client.0.smithi027.stdout:5/806: rename d2/d12/dad/da8/dd9 to d2/d12/dad/da8/dd9/dde/d105 22 2021-11-03T17:14:26.117 INFO:tasks.workunit.client.0.smithi027.stdout:5/807: chown d2/d1f/f7c 8226 1 2021-11-03T17:14:26.118 INFO:tasks.workunit.client.0.smithi027.stdout:5/808: creat d2/d12/dad/d6a/f106 x:0 0 0 2021-11-03T17:14:26.119 INFO:tasks.workunit.client.0.smithi027.stdout:5/809: write d2/d1f/f7c [2751487,45097] 0 2021-11-03T17:14:26.128 INFO:tasks.workunit.client.0.smithi027.stdout:8/787: dwrite d7/d10/d3f/f73 [0,4194304] 0 2021-11-03T17:14:26.128 INFO:tasks.workunit.client.0.smithi027.stdout:8/788: fdatasync d7/d53/fce 0 2021-11-03T17:14:26.128 INFO:tasks.workunit.client.0.smithi027.stdout:8/789: stat d7/f36 0 2021-11-03T17:14:26.133 INFO:tasks.workunit.client.0.smithi027.stdout:5/810: dread d2/d4/f43 [0,4194304] 0 2021-11-03T17:14:26.141 INFO:tasks.workunit.client.0.smithi027.stdout:7/461: dwrite d0/d2/d10/d1e/f99 [0,4194304] 0 2021-11-03T17:14:26.147 INFO:tasks.workunit.client.0.smithi027.stdout:8/790: dwrite d7/d10/d3f/d55/db7/dc9/d103/fb6 [0,4194304] 0 2021-11-03T17:14:26.147 INFO:tasks.workunit.client.0.smithi027.stdout:8/791: dread - d7/d10/d3f/d9b/d95/fd5 zero size 2021-11-03T17:14:26.148 INFO:tasks.workunit.client.0.smithi027.stdout:8/792: read d7/fc8 [2900878,4466] 0 2021-11-03T17:14:26.149 INFO:tasks.workunit.client.0.smithi027.stdout:8/793: fdatasync d7/ff4 0 2021-11-03T17:14:26.149 INFO:tasks.workunit.client.0.smithi027.stdout:8/794: fsync d7/d10/d21/d7f/fd1 0 2021-11-03T17:14:26.149 INFO:tasks.workunit.client.0.smithi027.stdout:7/462: unlink d0/d2/d10/l30 0 2021-11-03T17:14:26.152 INFO:tasks.workunit.client.0.smithi027.stdout:7/463: dread d0/f16 [0,4194304] 0 2021-11-03T17:14:26.162 INFO:tasks.workunit.client.0.smithi027.stdout:8/795: mkdir d7/d10/d3f/d9b/d95/d107 0 2021-11-03T17:14:26.171 INFO:tasks.workunit.client.0.smithi027.stdout:7/464: creat d0/d2/d1f/f9e x:0 0 0 2021-11-03T17:14:26.172 INFO:tasks.workunit.client.0.smithi027.stdout:7/465: chown d0/d2/d1c/f2b 33 1 2021-11-03T17:14:26.174 INFO:tasks.workunit.client.0.smithi027.stdout:8/796: rmdir d7/d10/d3f/d55/ddb 0 2021-11-03T17:14:26.177 INFO:tasks.workunit.client.0.smithi027.stdout:8/797: creat d7/d2d/d38/f108 x:0 0 0 2021-11-03T17:14:26.183 INFO:tasks.workunit.client.0.smithi027.stdout:7/466: write d0/d2/d5e/d36/f64 [62354,6352] 0 2021-11-03T17:14:26.184 INFO:tasks.workunit.client.0.smithi027.stdout:7/467: readlink d0/d2/d1c/d4d/d6e/d84/l87 0 2021-11-03T17:14:26.184 INFO:tasks.workunit.client.0.smithi027.stdout:7/468: write d0/d2/d10/d1e/d20/d74/f7b [1265251,100939] 0 2021-11-03T17:14:26.184 INFO:tasks.workunit.client.0.smithi027.stdout:7/469: chown d0/d2/d1c/d4d/d6e/d84/l60 4752 1 2021-11-03T17:14:26.184 INFO:tasks.workunit.client.0.smithi027.stdout:7/470: write d0/d2/d10/d35/f89 [1834173,41001] 0 2021-11-03T17:14:26.184 INFO:tasks.workunit.client.0.smithi027.stdout:7/471: readlink d0/l1 0 2021-11-03T17:14:26.185 INFO:tasks.workunit.client.0.smithi027.stdout:7/472: rename d0/d2/d5e/d36/d3a/l6f to d0/d2/d1c/l9f 0 2021-11-03T17:14:26.187 INFO:tasks.workunit.client.0.smithi027.stdout:7/473: creat d0/d2d/d53/fa0 x:0 0 0 2021-11-03T17:14:26.188 INFO:tasks.workunit.client.0.smithi027.stdout:7/474: chown d0/f18 13 1 2021-11-03T17:14:26.189 INFO:tasks.workunit.client.0.smithi027.stdout:7/475: link d0/l1d d0/d2/d5e/la1 0 2021-11-03T17:14:26.194 INFO:tasks.workunit.client.0.smithi027.stdout:8/798: dwrite d7/d10/d3f/d55/db7/dc9/d103/df0/f49 [0,4194304] 0 2021-11-03T17:14:26.195 INFO:tasks.workunit.client.0.smithi027.stdout:8/799: write d7/d10/d3f/d55/db7/dc9/d103/da5/ff8 [263871,50710] 0 2021-11-03T17:14:26.195 INFO:tasks.workunit.client.0.smithi027.stdout:8/800: read - d7/d2d/ff3 zero size 2021-11-03T17:14:26.195 INFO:tasks.workunit.client.0.smithi027.stdout:8/801: write d7/d10/d3f/d9b/d95/f7e [4267428,128508] 0 2021-11-03T17:14:26.195 INFO:tasks.workunit.client.0.smithi027.stdout:8/802: write d7/d10/d3f/d55/fa6 [253392,55514] 0 2021-11-03T17:14:26.196 INFO:tasks.workunit.client.0.smithi027.stdout:8/803: rename l4 to d7/l109 0 2021-11-03T17:14:26.199 INFO:tasks.workunit.client.0.smithi027.stdout:8/804: unlink d7/d10/d3f/d55/db7/dc9/fd0 0 2021-11-03T17:14:26.216 INFO:tasks.workunit.client.0.smithi027.stdout:8/805: dwrite d7/d10/d21/d41/fa3 [4194304,4194304] 0 2021-11-03T17:14:26.219 INFO:tasks.workunit.client.0.smithi027.stdout:8/806: creat d7/d10/d3f/d55/db7/dc9/f10a x:0 0 0 2021-11-03T17:14:26.237 INFO:tasks.workunit.client.0.smithi027.stdout:8/807: dwrite d7/d2d/d38/dc3/fdd [0,4194304] 0 2021-11-03T17:14:26.237 INFO:tasks.workunit.client.0.smithi027.stdout:8/808: chown d7/d53/laf 27429 1 2021-11-03T17:14:26.239 INFO:tasks.workunit.client.0.smithi027.stdout:8/809: symlink d7/dcc/l10b 0 2021-11-03T17:14:26.240 INFO:tasks.workunit.client.0.smithi027.stdout:8/810: write d7/d10/d21/f45 [450210,15855] 0 2021-11-03T17:14:26.242 INFO:tasks.workunit.client.0.smithi027.stdout:8/811: creat d7/f10c x:0 0 0 2021-11-03T17:14:26.244 INFO:tasks.workunit.client.0.smithi027.stdout:8/812: write d7/d10/d3f/d9b/f3c [1710723,58390] 0 2021-11-03T17:14:26.244 INFO:tasks.workunit.client.0.smithi027.stdout:8/813: chown d7/d10/d3f/d55/db7/dc9/d103/df0 46 1 2021-11-03T17:14:26.249 INFO:tasks.workunit.client.0.smithi027.stdout:8/814: truncate d7/d2d/d38/dc3/fdd 1644119 0 2021-11-03T17:14:26.250 INFO:tasks.workunit.client.0.smithi027.stdout:8/815: rmdir d7/d53/d67 39 2021-11-03T17:14:26.252 INFO:tasks.workunit.client.0.smithi027.stdout:8/816: creat d7/d10/d21/d7f/f10d x:0 0 0 2021-11-03T17:14:26.255 INFO:tasks.workunit.client.0.smithi027.stdout:8/817: link d7/d10/d21/d41/fa2 d7/d53/f10e 0 2021-11-03T17:14:26.281 INFO:tasks.workunit.client.0.smithi027.stdout:8/818: dwrite d7/d10/d3f/d9b/fd4 [0,4194304] 0 2021-11-03T17:14:26.291 INFO:tasks.workunit.client.0.smithi027.stdout:8/819: dread d7/d10/dbd/fd2 [0,4194304] 0 2021-11-03T17:14:26.292 INFO:tasks.workunit.client.0.smithi027.stdout:8/820: chown d7/d10/dbd/fe1 217035563 1 2021-11-03T17:14:26.292 INFO:tasks.workunit.client.0.smithi027.stdout:8/821: chown d7/d2d/l79 29898545 1 2021-11-03T17:14:26.293 INFO:tasks.workunit.client.0.smithi027.stdout:8/822: read d7/d10/f12 [617313,109971] 0 2021-11-03T17:14:26.294 INFO:tasks.workunit.client.0.smithi027.stdout:8/823: stat d7/dcc/dcf/l5e 0 2021-11-03T17:14:26.294 INFO:tasks.workunit.client.0.smithi027.stdout:8/824: write d7/d53/f102 [660806,96744] 0 2021-11-03T17:14:26.295 INFO:tasks.workunit.client.0.smithi027.stdout:8/825: rename d7/d10/d3f/f106 to d7/d10/d3f/d55/f10f 0 2021-11-03T17:14:26.296 INFO:tasks.workunit.client.0.smithi027.stdout:8/826: mkdir d7/d2d/da9/d110 0 2021-11-03T17:14:26.302 INFO:tasks.workunit.client.0.smithi027.stdout:8/827: rename d7/d2d/da9 to d7/d111 0 2021-11-03T17:14:26.302 INFO:tasks.workunit.client.0.smithi027.stdout:8/828: mknod d7/d53/c112 1 2021-11-03T17:14:26.316 INFO:tasks.workunit.client.0.smithi027.stdout:8/829: dwrite d7/d53/d67/fc7 [4194304,4194304] 0 2021-11-03T17:14:26.316 INFO:tasks.workunit.client.0.smithi027.stdout:8/830: mknod d7/d53/c113 1 2021-11-03T17:14:26.319 INFO:tasks.workunit.client.0.smithi027.stdout:8/831: rename d7/dcc/l10b to d7/d111/l114 0 2021-11-03T17:14:26.320 INFO:tasks.workunit.client.0.smithi027.stdout:8/832: getdents d7/d111/d110 0 2021-11-03T17:14:26.321 INFO:tasks.workunit.client.0.smithi027.stdout:8/833: truncate d7/d53/d67/f6b 1049115 0 2021-11-03T17:14:26.327 INFO:tasks.workunit.client.0.smithi027.stdout:8/834: dread d7/d10/d3f/f88 [0,4194304] 0 2021-11-03T17:14:26.329 INFO:tasks.workunit.client.0.smithi027.stdout:8/835: rename d7/d10/d3f/d55/fa6 to d7/dcc/dfe/f115 0 2021-11-03T17:14:26.329 INFO:tasks.workunit.client.0.smithi027.stdout:8/836: dread - d7/d10/fcb zero size 2021-11-03T17:14:26.329 INFO:tasks.workunit.client.0.smithi027.stdout:8/837: mknod d7/d111/c116 1 2021-11-03T17:14:26.330 INFO:tasks.workunit.client.0.smithi027.stdout:8/838: fdatasync d7/d10/d3f/d55/f9f 0 2021-11-03T17:14:26.330 INFO:tasks.workunit.client.0.smithi027.stdout:8/839: write d7/d10/d3f/d9b/d95/f9a [3041007,101866] 0 2021-11-03T17:14:26.425 INFO:tasks.workunit.client.0.smithi027.stdout:8/840: dwrite d7/d2d/d38/f3d [0,4194304] 0 2021-11-03T17:14:26.429 INFO:tasks.workunit.client.0.smithi027.stdout:8/841: write d7/d10/d3f/f73 [2923095,57401] 0 2021-11-03T17:14:26.434 INFO:tasks.workunit.client.0.smithi027.stdout:8/842: rename d7/d10/d21 to d7/d10/d3f/d55/db7/dc9/d117 0 2021-11-03T17:14:26.436 INFO:tasks.workunit.client.0.smithi027.stdout:8/843: creat d7/d10/d3f/d55/db7/f118 x:0 0 0 2021-11-03T17:14:26.440 INFO:tasks.workunit.client.0.smithi027.stdout:8/844: creat d7/d10/d3f/f119 x:0 0 0 2021-11-03T17:14:26.440 INFO:tasks.workunit.client.0.smithi027.stdout:8/845: truncate d7/f10c 469318 0 2021-11-03T17:14:26.460 INFO:tasks.workunit.client.0.smithi027.stdout:8/846: dwrite d7/d2d/d38/f44 [0,4194304] 0 2021-11-03T17:14:26.461 INFO:tasks.workunit.client.0.smithi027.stdout:8/847: rmdir d7/dcc/dfe 39 2021-11-03T17:14:26.463 INFO:tasks.workunit.client.0.smithi027.stdout:8/848: unlink d7/d10/fd7 0 2021-11-03T17:14:26.465 INFO:tasks.workunit.client.0.smithi027.stdout:8/849: symlink d7/d10/d3f/d55/db7/dc9/d103/da5/l11a 0 2021-11-03T17:14:26.465 INFO:tasks.workunit.client.0.smithi027.stdout:8/850: chown d7/d2d/d38/f108 1546 1 2021-11-03T17:14:26.467 INFO:tasks.workunit.client.0.smithi027.stdout:8/851: rename d7/l6a to d7/d10/d3f/d55/db7/dc9/d103/l11b 0 2021-11-03T17:14:26.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:26 smithi115 conmon[34670]: cephadm 2021-11-03T17:14:24.638533+0000 mgr.smithi115.tcyodz (mgr.24395) 6 : cephadm [INF] [03/Nov/2021:17:14:24] ENGINE Serving on https://172.21.15.115:8499 2021-11-03T17:14:26.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:26 smithi115 conmon[34670]: cephadm 2021-11-03T17:14:24.638693+0000 mgr.smithi115.tcyodz (mgr.24395) 7 : cephadm [INF] [03/Nov/2021:17:14:24] ENGINE Bus STARTED 2021-11-03T17:14:26.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:26 smithi115 conmon[34670]: cephadm 2021-11-03T17:14:24.638762+0000 mgr.smithi115.tcyodz (mgr.24395) 8 : cephadm [INF] Cherrypy engine started. 2021-11-03T17:14:26.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:26 smithi115 conmon[34670]: cluster 2021-11-03T17:14:25.186656+0000 mgr.smithi115.tcyodz (mgr.24395) 9 : cluster [DBG] pgmap v5: 65 pgs: 65 active+clean; 2.6 GiB data, 7.7 GiB used, 529 GiB / 536 GiB avail 2021-11-03T17:14:26.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:26 smithi115 conmon[34670]: cluster 2021-11-03T17:14:25.224538+0000 mon.smithi027 (mon.0) 595 : cluster [DBG] mgrmap e21: smithi115.tcyodz(active, since 2s), standbys: smithi027.ufhhvb 2021-11-03T17:14:26.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:26 smithi115 conmon[34670]: audit 2021-11-03T17:14:25.229809+0000 mon.smithi027 (mon.0) 596 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2021-11-03T17:14:26.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:26 smithi115 conmon[34670]: cluster 2021-11-03T17:14:25.229909+0000 mon.smithi027 (mon.0) 597 : cluster [DBG] osdmap e48: 6 total, 6 up, 6 in 2021-11-03T17:14:26.486 INFO:tasks.workunit.client.0.smithi027.stdout:8/852: dwrite d7/d10/f89 [0,4194304] 0 2021-11-03T17:14:26.490 INFO:tasks.workunit.client.0.smithi027.stdout:8/853: mkdir d7/d10/d3f/d9b/d11c 0 2021-11-03T17:14:26.491 INFO:tasks.workunit.client.0.smithi027.stdout:8/854: mkdir d7/d111/d11d 0 2021-11-03T17:14:26.502 INFO:tasks.workunit.client.0.smithi027.stdout:8/855: dread d7/d10/d3f/d55/db7/dc9/d103/df0/f82 [0,4194304] 0 2021-11-03T17:14:26.503 INFO:tasks.workunit.client.0.smithi027.stdout:8/856: mknod d7/d10/d3f/d55/db7/dc9/d117/d7f/c11e 1 2021-11-03T17:14:26.503 INFO:tasks.workunit.client.0.smithi027.stdout:8/857: write d7/d10/d3f/f119 [436513,54139] 0 2021-11-03T17:14:26.504 INFO:tasks.workunit.client.0.smithi027.stdout:8/858: rmdir d7/d10/d3f/d55/db7/dc9/d117/d41 39 2021-11-03T17:14:26.504 INFO:tasks.workunit.client.0.smithi027.stdout:8/859: truncate d7/d10/d3f/d9b/f3c 5765381 0 2021-11-03T17:14:26.504 INFO:tasks.workunit.client.0.smithi027.stdout:8/860: fdatasync d7/f87 0 2021-11-03T17:14:26.505 INFO:tasks.workunit.client.0.smithi027.stdout:8/861: write d7/d10/d3f/d55/db7/dc9/d117/d41/fa7 [2588836,79954] 0 2021-11-03T17:14:26.506 INFO:tasks.workunit.client.0.smithi027.stdout:8/862: mkdir d7/dcc/dfe/d11f 0 2021-11-03T17:14:26.506 INFO:tasks.workunit.client.0.smithi027.stdout:8/863: chown d7/d2d/d38/f108 5 1 2021-11-03T17:14:26.506 INFO:tasks.workunit.client.0.smithi027.stdout:8/864: chown d7/d10/f16 0 1 2021-11-03T17:14:26.506 INFO:tasks.workunit.client.0.smithi027.stdout:8/865: stat d7/d10/d3f/d55/db7/dc9/d117/d41/fa3 0 2021-11-03T17:14:26.506 INFO:tasks.workunit.client.0.smithi027.stdout:8/866: fdatasync d7/d10/f89 0 2021-11-03T17:14:26.507 INFO:tasks.workunit.client.0.smithi027.stdout:8/867: write d7/d10/d3f/d55/db7/dc9/d117/f45 [81293,33433] 0 2021-11-03T17:14:26.508 INFO:tasks.workunit.client.0.smithi027.stdout:8/868: write d7/d10/d3f/d55/db7/dc9/d117/f63 [253701,8536] 0 2021-11-03T17:14:26.510 INFO:tasks.workunit.client.0.smithi027.stdout:8/869: unlink d7/d10/d3f/l52 0 2021-11-03T17:14:26.511 INFO:tasks.workunit.client.0.smithi027.stdout:8/870: write d7/d10/d3f/d55/db7/dc9/d117/d41/fa7 [3754351,41880] 0 2021-11-03T17:14:26.511 INFO:tasks.workunit.client.0.smithi027.stdout:8/871: read d7/d10/dbd/fc1 [82545,38023] 0 2021-11-03T17:14:26.514 INFO:tasks.workunit.client.0.smithi027.stdout:8/872: unlink d7/d10/d3f/d55/db7/dc9/d103/df0/f48 0 2021-11-03T17:14:26.515 INFO:tasks.workunit.client.0.smithi027.stdout:8/873: creat d7/d10/dbd/f120 x:0 0 0 2021-11-03T17:14:26.516 INFO:tasks.workunit.client.0.smithi027.stdout:8/874: symlink d7/dcc/dfe/d11f/l121 0 2021-11-03T17:14:26.518 INFO:tasks.workunit.client.0.smithi027.stdout:8/875: creat d7/d111/d11d/f122 x:0 0 0 2021-11-03T17:14:26.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:26 smithi027 conmon[32322]: cephadm 2021-11-03T17:14:24.638533+0000 mgr.smithi115.tcyodz ( 2021-11-03T17:14:26.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:26 smithi027 conmon[32322]: mgr.24395) 6 : cephadm [INF] [03/Nov/2021:17:14:24] ENGINE Serving on https://172.21.15.115:8499 2021-11-03T17:14:26.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:26 smithi027 conmon[32322]: cephadm 2021-11-03T17:14:24.638693+0000 mgr.smithi115.tcyodz (mgr.24395) 7 : cephadm [INF] [03/Nov/2021:17:14:24] ENGINE Bus STARTED 2021-11-03T17:14:26.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:26 smithi027 conmon[32322]: cephadm 2021-11-03T17:14:24.638762+0000 mgr.smithi115.tcyodz (mgr.24395) 8 : cephadm [INF] Cherrypy engine started. 2021-11-03T17:14:26.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:26 smithi027 conmon[32322]: cluster 2021-11-03T17:14:25.186656+0000 mgr.smithi115.tcyodz (mgr.24395) 9 : cluster [DBG] pgmap v5: 65 pgs: 65 active+clean; 2.6 GiB data, 7.7 GiB used, 529 GiB / 536 GiB avail 2021-11-03T17:14:26.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:26 smithi027 conmon[32322]: cluster 2021-11-03T17:14:25.224538+0000 mon.smithi027 (mon.0) 595 : cluster [DBG] mgrmap e21: smithi115.tcyodz(active, since 2s), standbys: smithi027.ufhhvb 2021-11-03T17:14:26.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:26 smithi027 conmon[32322]: audit 2021-11-03T17:14:25.229809+0000 mon.smithi027 (mon.0) 596 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2021-11-03T17:14:26.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:26 smithi027 conmon[32322]: cluster 2021-11-03T17:14:25.229909+0000 mon.smithi027 (mon.0) 597 : cluster [DBG] osdmap e48: 6 total, 6 up, 6 in 2021-11-03T17:14:26.520 INFO:tasks.workunit.client.0.smithi027.stdout:8/876: mkdir d7/dcc/dcf/d123 0 2021-11-03T17:14:26.521 INFO:tasks.workunit.client.0.smithi027.stdout:8/877: symlink d7/dcc/dcf/l124 0 2021-11-03T17:14:26.531 INFO:tasks.workunit.client.0.smithi027.stdout:8/878: getdents d7/d10/d3f 0 2021-11-03T17:14:26.532 INFO:tasks.workunit.client.0.smithi027.stdout:8/879: write d7/d10/d3f/d55/db7/dc9/d117/d41/fa2 [4099513,119965] 0 2021-11-03T17:14:26.532 INFO:tasks.workunit.client.0.smithi027.stdout:8/880: chown d7/d2d/d38/feb 430 1 2021-11-03T17:14:26.532 INFO:tasks.workunit.client.0.smithi027.stdout:8/881: read - d7/d10/dbd/f120 zero size 2021-11-03T17:14:26.533 INFO:tasks.workunit.client.0.smithi027.stdout:8/882: read - d7/dcc/dfe/f100 zero size 2021-11-03T17:14:26.533 INFO:tasks.workunit.client.0.smithi027.stdout:8/883: write d7/fb2 [283975,29199] 0 2021-11-03T17:14:26.534 INFO:tasks.workunit.client.0.smithi027.stdout:8/884: mkdir d7/d10/d3f/d9b/d95/d125 0 2021-11-03T17:14:26.536 INFO:tasks.workunit.client.0.smithi027.stdout:8/885: unlink d7/d2d/l29 0 2021-11-03T17:14:26.537 INFO:tasks.workunit.client.0.smithi027.stdout:8/886: symlink d7/d10/d3f/d55/db7/l126 0 2021-11-03T17:14:26.539 INFO:tasks.workunit.client.0.smithi027.stdout:8/887: write f6 [180736,3853] 0 2021-11-03T17:14:26.541 INFO:tasks.workunit.client.0.smithi027.stdout:8/888: unlink d7/d10/d3f/d9b/l54 0 2021-11-03T17:14:26.543 INFO:tasks.workunit.client.0.smithi027.stdout:8/889: link d7/d111/lab d7/d10/d3f/d55/l127 0 2021-11-03T17:14:26.543 INFO:tasks.workunit.client.0.smithi027.stdout:8/890: write d7/d10/dbd/fe1 [429969,77525] 0 2021-11-03T17:14:26.544 INFO:tasks.workunit.client.0.smithi027.stdout:8/891: write d7/d10/d3f/d9b/d95/fd5 [309306,73234] 0 2021-11-03T17:14:26.544 INFO:tasks.workunit.client.0.smithi027.stdout:8/892: rmdir d7/d10/d3f/d55/db7/dc9/d103 39 2021-11-03T17:14:26.562 INFO:tasks.workunit.client.0.smithi027.stdout:8/893: dwrite d7/d10/d3f/d55/db7/dc9/d117/d41/fc2 [0,4194304] 0 2021-11-03T17:14:26.563 INFO:tasks.workunit.client.0.smithi027.stdout:8/894: fdatasync d7/d10/d3f/d9b/f2e 0 2021-11-03T17:14:26.597 INFO:tasks.workunit.client.0.smithi027.stdout:8/895: truncate d7/d2d/d38/f44 1124463 0 2021-11-03T17:14:26.601 INFO:tasks.workunit.client.0.smithi027.stdout:8/896: creat d7/d10/d3f/d9b/f128 x:0 0 0 2021-11-03T17:14:26.604 INFO:tasks.workunit.client.0.smithi027.stdout:8/897: creat d7/d10/d3f/d55/db7/dc9/d117/d7f/f129 x:0 0 0 2021-11-03T17:14:26.606 INFO:tasks.workunit.client.0.smithi027.stdout:8/898: rename d7/d10/fca to d7/dff/f12a 0 2021-11-03T17:14:26.608 INFO:tasks.workunit.client.0.smithi027.stdout:8/899: chown d7/d2d/ff3 897 1 2021-11-03T17:14:26.618 INFO:tasks.workunit.client.0.smithi027.stdout:8/900: dread d7/d2d/d38/fc4 [0,4194304] 0 2021-11-03T17:14:26.620 INFO:tasks.workunit.client.0.smithi027.stdout:8/901: write d7/d53/d67/fc7 [7801142,43829] 0 2021-11-03T17:14:26.622 INFO:tasks.workunit.client.0.smithi027.stdout:8/902: link d7/d10/d3f/f119 d7/d53/f12b 0 2021-11-03T17:14:26.624 INFO:tasks.workunit.client.0.smithi027.stdout:8/903: truncate d7/d10/d3f/d55/f9f 3289385 0 2021-11-03T17:14:26.624 INFO:tasks.workunit.client.0.smithi027.stdout:8/904: mknod d7/d53/d67/c12c 1 2021-11-03T17:14:26.624 INFO:tasks.workunit.client.0.smithi027.stdout:8/905: fdatasync d7/d10/d3f/d9b/fd4 0 2021-11-03T17:14:26.624 INFO:tasks.workunit.client.0.smithi027.stdout:8/906: read - d7/dcc/dcf/f101 zero size 2021-11-03T17:14:26.625 INFO:tasks.workunit.client.0.smithi027.stdout:8/907: chown d7/d10/d3f/d9b/l4b 23 1 2021-11-03T17:14:26.625 INFO:tasks.workunit.client.0.smithi027.stdout:8/908: dread - d7/d10/d3f/d9b/f128 zero size 2021-11-03T17:14:26.625 INFO:tasks.workunit.client.0.smithi027.stdout:8/909: mkdir d7/d10/d3f/d12d 0 2021-11-03T17:14:26.627 INFO:tasks.workunit.client.0.smithi027.stdout:8/910: getdents d7/d2d/d38 0 2021-11-03T17:14:26.649 INFO:tasks.workunit.client.0.smithi027.stdout:8/911: dwrite d7/d10/d3f/d55/f10f [0,4194304] 0 2021-11-03T17:14:26.649 INFO:tasks.workunit.client.0.smithi027.stdout:8/912: truncate d7/d10/d3f/d55/db7/dc9/d103/fe5 913593 0 2021-11-03T17:14:26.652 INFO:tasks.workunit.client.0.smithi027.stdout:8/913: dread d7/d10/d3f/d55/db7/dc9/d117/f25 [0,4194304] 0 2021-11-03T17:14:26.657 INFO:tasks.workunit.client.0.smithi027.stdout:8/914: dread d7/d10/d3f/d9b/f60 [0,4194304] 0 2021-11-03T17:14:26.658 INFO:tasks.workunit.client.0.smithi027.stdout:8/915: truncate d7/d10/d3f/d55/db7/dc9/d117/f8d 1101417 0 2021-11-03T17:14:26.658 INFO:tasks.workunit.client.0.smithi027.stdout:8/916: mknod d7/d10/d3f/d55/db7/dc9/d117/d41/c12e 1 2021-11-03T17:14:26.837 INFO:tasks.workunit.client.0.smithi027.stdout:6/550: sync 2021-11-03T17:14:26.839 INFO:tasks.workunit.client.0.smithi027.stdout:6/551: symlink dc/d82/d87/l9f 0 2021-11-03T17:14:26.841 INFO:tasks.workunit.client.0.smithi027.stdout:6/552: unlink dc/d16/d62/l7d 0 2021-11-03T17:14:26.854 INFO:tasks.workunit.client.0.smithi027.stdout:6/553: dread dc/d16/d1b/d3c/d51/d5f/d79/f93 [0,4194304] 0 2021-11-03T17:14:26.857 INFO:tasks.workunit.client.0.smithi027.stdout:6/554: rename dc/d16/l8f to dc/d82/d57/d96/la0 0 2021-11-03T17:14:26.878 INFO:tasks.workunit.client.0.smithi027.stdout:6/555: dwrite dc/d82/f81 [0,4194304] 0 2021-11-03T17:14:26.879 INFO:tasks.workunit.client.0.smithi027.stdout:6/556: fdatasync dc/d16/f2c 0 2021-11-03T17:14:26.880 INFO:tasks.workunit.client.0.smithi027.stdout:6/557: mkdir dc/da1 0 2021-11-03T17:14:26.881 INFO:tasks.workunit.client.0.smithi027.stdout:6/558: dread dc/d16/d30/f5a [0,4194304] 0 2021-11-03T17:14:26.882 INFO:tasks.workunit.client.0.smithi027.stdout:6/559: symlink dc/d82/d57/d96/la2 0 2021-11-03T17:14:26.894 INFO:tasks.workunit.client.0.smithi027.stdout:4/454: sync 2021-11-03T17:14:26.895 INFO:tasks.workunit.client.0.smithi027.stdout:3/485: sync 2021-11-03T17:14:26.895 INFO:tasks.workunit.client.0.smithi027.stdout:2/588: sync 2021-11-03T17:14:26.895 INFO:tasks.workunit.client.0.smithi027.stdout:2/589: chown d0/d1b/d29/d58/d5b/f8b 1 1 2021-11-03T17:14:26.896 INFO:tasks.workunit.client.0.smithi027.stdout:2/590: read d0/d1b/d29/d54/d64/daa/f21 [5347310,127030] 0 2021-11-03T17:14:26.897 INFO:tasks.workunit.client.0.smithi027.stdout:4/455: truncate d6/d9/d28/f87 2713930 0 2021-11-03T17:14:26.898 INFO:tasks.workunit.client.0.smithi027.stdout:3/486: dread d1/d40/f46 [0,4194304] 0 2021-11-03T17:14:26.902 INFO:tasks.workunit.client.0.smithi027.stdout:3/487: mkdir d1/db/d52/d57/d99 0 2021-11-03T17:14:26.903 INFO:tasks.workunit.client.0.smithi027.stdout:6/560: dwrite dc/d16/d1b/d2d/d34/f4e [0,4194304] 0 2021-11-03T17:14:26.917 INFO:tasks.workunit.client.0.smithi027.stdout:6/561: dread dc/d16/d1b/d3c/d51/f78 [0,4194304] 0 2021-11-03T17:14:26.931 INFO:tasks.workunit.client.0.smithi027.stdout:3/488: dread d1/d40/f4a [0,4194304] 0 2021-11-03T17:14:26.931 INFO:tasks.workunit.client.0.smithi027.stdout:2/591: dwrite d0/d1b/d29/d54/d64/daa/f66 [0,4194304] 0 2021-11-03T17:14:26.931 INFO:tasks.workunit.client.0.smithi027.stdout:3/489: fdatasync d1/db/f1c 0 2021-11-03T17:14:26.932 INFO:tasks.workunit.client.0.smithi027.stdout:3/490: chown d1/d6f 30092 1 2021-11-03T17:14:26.932 INFO:tasks.workunit.client.0.smithi027.stdout:3/491: fsync d1/d40/f76 0 2021-11-03T17:14:26.932 INFO:tasks.workunit.client.0.smithi027.stdout:3/492: dread d1/d40/f46 [0,4194304] 0 2021-11-03T17:14:26.934 INFO:tasks.workunit.client.0.smithi027.stdout:4/456: dwrite d6/f48 [0,4194304] 0 2021-11-03T17:14:26.938 INFO:tasks.workunit.client.0.smithi027.stdout:4/457: mkdir d6/d9/d16/d36/d99 0 2021-11-03T17:14:26.943 INFO:tasks.workunit.client.0.smithi027.stdout:4/458: mkdir d6/d9/d16/d36/d70/d3e/d92/d98/d21/d9a 0 2021-11-03T17:14:26.954 INFO:tasks.workunit.client.0.smithi027.stdout:2/592: dwrite d0/d1b/d29/f48 [0,4194304] 0 2021-11-03T17:14:26.960 INFO:tasks.workunit.client.0.smithi027.stdout:2/593: unlink d0/d1b/d29/d54/d67/d6a/l77 0 2021-11-03T17:14:26.960 INFO:tasks.workunit.client.0.smithi027.stdout:2/594: fsync d0/d1b/d29/d54/d67/d6a/fa3 0 2021-11-03T17:14:26.965 INFO:tasks.workunit.client.0.smithi027.stdout:2/595: truncate d0/d1b/d1c/f26 1715798 0 2021-11-03T17:14:26.967 INFO:tasks.workunit.client.0.smithi027.stdout:2/596: mkdir d0/d1b/d1c/dad 0 2021-11-03T17:14:26.969 INFO:tasks.workunit.client.0.smithi027.stdout:2/597: creat d0/d1b/d29/d54/fae x:0 0 0 2021-11-03T17:14:26.969 INFO:tasks.workunit.client.0.smithi027.stdout:2/598: chown d0/d1b/d29/d54/d64/daa/d61/l63 23788 1 2021-11-03T17:14:26.969 INFO:tasks.workunit.client.0.smithi027.stdout:2/599: readlink d0/d1b/d1c/l71 0 2021-11-03T17:14:26.970 INFO:tasks.workunit.client.0.smithi027.stdout:2/600: stat d0/d1b/d1c/dad 0 2021-11-03T17:14:26.971 INFO:tasks.workunit.client.0.smithi027.stdout:2/601: write d0/d1b/d29/d54/d64/daa/f30 [1511267,81449] 0 2021-11-03T17:14:26.978 INFO:tasks.workunit.client.0.smithi027.stdout:2/602: getdents d0/d1b/d1c 0 2021-11-03T17:14:26.979 INFO:tasks.workunit.client.0.smithi027.stdout:2/603: fdatasync d0/d5/d2c/f3d 0 2021-11-03T17:14:26.979 INFO:tasks.workunit.client.0.smithi027.stdout:2/604: write d0/d1b/d29/f9d [232615,28902] 0 2021-11-03T17:14:26.983 INFO:tasks.workunit.client.0.smithi027.stdout:3/493: dwrite d1/db/f1c [0,4194304] 0 2021-11-03T17:14:26.995 INFO:tasks.workunit.client.0.smithi027.stdout:2/605: dwrite d0/d1b/d29/d58/d5b/f80 [0,4194304] 0 2021-11-03T17:14:26.996 INFO:tasks.workunit.client.0.smithi027.stdout:2/606: write d0/f46 [2169857,46990] 0 2021-11-03T17:14:26.996 INFO:tasks.workunit.client.0.smithi027.stdout:2/607: fdatasync d0/d1b/d29/d54/d64/daa/f30 0 2021-11-03T17:14:27.002 INFO:tasks.workunit.client.0.smithi027.stdout:2/608: creat d0/d5/faf x:0 0 0 2021-11-03T17:14:27.004 INFO:tasks.workunit.client.0.smithi027.stdout:2/609: symlink d0/d1b/d1c/dad/lb0 0 2021-11-03T17:14:27.004 INFO:tasks.workunit.client.0.smithi027.stdout:2/610: fdatasync d0/f91 0 2021-11-03T17:14:27.004 INFO:tasks.workunit.client.0.smithi027.stdout:2/611: read - d0/d5/fac zero size 2021-11-03T17:14:27.004 INFO:tasks.workunit.client.0.smithi027.stdout:2/612: chown d0/d5/d2c/f78 9582326 1 2021-11-03T17:14:27.005 INFO:tasks.workunit.client.0.smithi027.stdout:2/613: write d0/d1b/d1c/f57 [4031049,53679] 0 2021-11-03T17:14:27.006 INFO:tasks.workunit.client.0.smithi027.stdout:2/614: chown d0/l8e 1987 1 2021-11-03T17:14:27.006 INFO:tasks.workunit.client.0.smithi027.stdout:2/615: mknod d0/d1b/d29/d58/d5b/cb1 1 2021-11-03T17:14:27.006 INFO:tasks.workunit.client.0.smithi027.stdout:2/616: truncate d0/d1b/d29/d54/d64/daa/d5d/f8f 406509 0 2021-11-03T17:14:27.007 INFO:tasks.workunit.client.0.smithi027.stdout:2/617: write d0/f3b [3506456,55808] 0 2021-11-03T17:14:27.029 INFO:tasks.workunit.client.0.smithi027.stdout:2/618: dwrite d0/d1b/d29/d54/d67/d6a/f97 [4194304,4194304] 0 2021-11-03T17:14:27.034 INFO:tasks.workunit.client.0.smithi027.stdout:2/619: fsync d0/f3b 0 2021-11-03T17:14:27.042 INFO:tasks.workunit.client.0.smithi027.stdout:2/620: creat d0/d1b/d29/d58/fb2 x:0 0 0 2021-11-03T17:14:27.042 INFO:tasks.workunit.client.0.smithi027.stdout:2/621: dread - d0/fa9 zero size 2021-11-03T17:14:27.046 INFO:tasks.workunit.client.0.smithi027.stdout:2/622: unlink d0/d1b/d1c/f99 0 2021-11-03T17:14:27.049 INFO:tasks.workunit.client.0.smithi027.stdout:2/623: symlink d0/d1b/d29/d58/d5b/lb3 0 2021-11-03T17:14:27.051 INFO:tasks.workunit.client.0.smithi027.stdout:2/624: link d0/f91 d0/d1b/d29/d54/fb4 0 2021-11-03T17:14:27.052 INFO:tasks.workunit.client.0.smithi027.stdout:2/625: readlink d0/d1b/l60 0 2021-11-03T17:14:27.053 INFO:tasks.workunit.client.0.smithi027.stdout:2/626: rmdir d0/d5 39 2021-11-03T17:14:27.222 INFO:tasks.workunit.client.0.smithi027.stdout:9/592: sync 2021-11-03T17:14:27.230 INFO:tasks.workunit.client.0.smithi027.stdout:9/593: creat db/d1b/d2b/d41/d95/fce x:0 0 0 2021-11-03T17:14:27.231 INFO:tasks.workunit.client.0.smithi027.stdout:9/594: chown db/d1b/d2b/fba 14 1 2021-11-03T17:14:27.234 INFO:tasks.workunit.client.0.smithi027.stdout:9/595: truncate db/d1b/d2b/f91 364595 0 2021-11-03T17:14:27.235 INFO:tasks.workunit.client.0.smithi027.stdout:9/596: dread - db/d1b/d2b/fba zero size 2021-11-03T17:14:27.246 INFO:tasks.workunit.client.0.smithi027.stdout:9/597: dread db/f14 [4194304,4194304] 0 2021-11-03T17:14:27.247 INFO:tasks.workunit.client.0.smithi027.stdout:9/598: fsync db/d1b/d2b/d41/fb8 0 2021-11-03T17:14:27.247 INFO:tasks.workunit.client.0.smithi027.stdout:9/599: readlink db/d1b/d1c/d1d/l31 0 2021-11-03T17:14:27.247 INFO:tasks.workunit.client.0.smithi027.stdout:9/600: chown db/d1b/d1c/d1d/l47 529745789 1 2021-11-03T17:14:27.247 INFO:tasks.workunit.client.0.smithi027.stdout:9/601: chown db/d1b/f51 23062 1 2021-11-03T17:14:27.269 INFO:tasks.workunit.client.0.smithi027.stdout:9/602: dwrite db/d1b/d1c/d1d/f49 [0,4194304] 0 2021-11-03T17:14:27.273 INFO:tasks.workunit.client.0.smithi027.stdout:9/603: creat db/d1b/d2b/d41/d95/fcf x:0 0 0 2021-11-03T17:14:27.273 INFO:tasks.workunit.client.0.smithi027.stdout:9/604: fsync db/d1b/d23/d2c/f6a 0 2021-11-03T17:14:27.275 INFO:tasks.workunit.client.0.smithi027.stdout:9/605: creat db/d1b/d1c/fd0 x:0 0 0 2021-11-03T17:14:27.275 INFO:tasks.workunit.client.0.smithi027.stdout:9/606: fdatasync db/d1b/d2b/fba 0 2021-11-03T17:14:27.278 INFO:tasks.workunit.client.0.smithi027.stdout:9/607: symlink db/d1b/d2b/d41/dad/dbb/ld1 0 2021-11-03T17:14:27.278 INFO:tasks.workunit.client.0.smithi027.stdout:9/608: chown db/d1b/d2b/d5e 822 1 2021-11-03T17:14:27.282 INFO:tasks.workunit.client.0.smithi027.stdout:9/609: rmdir db/d1b 39 2021-11-03T17:14:27.283 INFO:tasks.workunit.client.0.smithi027.stdout:9/610: mknod db/d1b/d8c/cd2 1 2021-11-03T17:14:27.283 INFO:tasks.workunit.client.0.smithi027.stdout:9/611: dread - db/fa8 zero size 2021-11-03T17:14:27.283 INFO:tasks.workunit.client.0.smithi027.stdout:9/612: stat db/fd 0 2021-11-03T17:14:27.283 INFO:tasks.workunit.client.0.smithi027.stdout:9/613: fsync db/d1b/d1c/d1d/d40/fa7 0 2021-11-03T17:14:27.294 INFO:tasks.workunit.client.0.smithi027.stdout:9/614: dread db/d1b/d2b/f91 [0,4194304] 0 2021-11-03T17:14:27.294 INFO:tasks.workunit.client.0.smithi027.stdout:9/615: stat db/d1b/d23/f73 0 2021-11-03T17:14:27.297 INFO:tasks.workunit.client.0.smithi027.stdout:9/616: rename db/d1b/d23/d2c/f6a to db/d1b/d2b/d41/dad/fd3 0 2021-11-03T17:14:27.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:27 smithi115 conmon[34670]: audit 2021-11-03T17:14:26.478662+0000 2021-11-03T17:14:27.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:27 smithi115 conmon[34670]: mon.smithi027 (mon.0) 598 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' 2021-11-03T17:14:27.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:27 smithi027 conmon[32322]: audit 2021-11-03T17:14:26.478662+0000 mon.smithi027 (mon.0) 598 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' 2021-11-03T17:14:28.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:28 smithi115 conmon[34670]: cluster 2021-11-03T17:14:27.187014+0000 mgr.smithi115.tcyodz (mgr.24395) 2021-11-03T17:14:28.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:28 smithi115 conmon[34670]: 10 : cluster [DBG] pgmap v7: 65 pgs: 65 active+clean; 2.6 GiB data, 7.7 GiB used, 529 GiB / 536 GiB avail 2021-11-03T17:14:28.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:28 smithi115 conmon[34670]: cluster 2021-11-03T17:14:27.487841+0000 mon.smithi027 (mon.0) 599 : cluster [DBG] mgrmap e22: smithi115.tcyodz(active, since 4s), standbys: smithi027.ufhhvb 2021-11-03T17:14:28.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:28 smithi027 conmon[32322]: cluster 2021-11-03T17:14:27.187014+0000 mgr.smithi115.tcyodz (mgr.24395) 10 : cluster [DBG] pgmap v7: 65 pgs: 65 active+clean; 2.6 GiB data, 7.7 GiB used, 529 GiB / 536 GiB avail 2021-11-03T17:14:28.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:28 smithi027 conmon[32322]: cluster 2021-11-03T17:14:27.487841+0000 mon.smithi027 (mon.0) 599 : cluster [DBG] mgrmap e22: smithi115.tcyodz(active, since 4s), standbys: smithi027.ufhhvb 2021-11-03T17:14:29.653 INFO:tasks.workunit.client.0.smithi027.stdout:7/476: sync 2021-11-03T17:14:29.654 INFO:tasks.workunit.client.0.smithi027.stdout:7/477: chown d0/d2/d59/f73 1897 1 2021-11-03T17:14:29.654 INFO:tasks.workunit.client.0.smithi027.stdout:7/478: truncate d0/d2/d1c/f79 4832526 0 2021-11-03T17:14:29.673 INFO:tasks.workunit.client.0.smithi027.stdout:7/479: dwrite d0/d2/d10/d1e/f5c [0,4194304] 0 2021-11-03T17:14:29.679 INFO:tasks.workunit.client.0.smithi027.stdout:7/480: link d0/f25 d0/d2/d10/d35/d96/fa2 0 2021-11-03T17:14:29.679 INFO:tasks.workunit.client.0.smithi027.stdout:7/481: chown d0/d2/d10/d1e/d20/d74 1340660 1 2021-11-03T17:14:29.681 INFO:tasks.workunit.client.0.smithi027.stdout:7/482: creat d0/d2/fa3 x:0 0 0 2021-11-03T17:14:29.683 INFO:tasks.workunit.client.0.smithi027.stdout:7/483: truncate d0/d2d/d2f/f39 4092755 0 2021-11-03T17:14:29.686 INFO:tasks.workunit.client.0.smithi027.stdout:7/484: link d0/d2/d10/d35/l56 d0/d2/d5e/d36/d3a/la4 0 2021-11-03T17:14:29.700 INFO:tasks.workunit.client.0.smithi027.stdout:7/485: dread d0/d2/d10/d1e/d20/f43 [0,4194304] 0 2021-11-03T17:14:29.702 INFO:tasks.workunit.client.0.smithi027.stdout:7/486: rmdir d0/d2/d5e/d36/d3a/d9c 0 2021-11-03T17:14:29.702 INFO:tasks.workunit.client.0.smithi027.stdout:7/487: mknod d0/d2d/d58/ca5 1 2021-11-03T17:14:29.711 INFO:tasks.workunit.client.0.smithi027.stdout:6/562: sync 2021-11-03T17:14:29.711 INFO:tasks.workunit.client.0.smithi027.stdout:2/627: sync 2021-11-03T17:14:29.711 INFO:tasks.workunit.client.0.smithi027.stdout:5/811: sync 2021-11-03T17:14:29.711 INFO:tasks.workunit.client.0.smithi027.stdout:4/459: sync 2021-11-03T17:14:29.712 INFO:tasks.workunit.client.0.smithi027.stdout:9/617: sync 2021-11-03T17:14:29.712 INFO:tasks.workunit.client.0.smithi027.stdout:3/494: sync 2021-11-03T17:14:29.712 INFO:tasks.workunit.client.0.smithi027.stdout:0/775: sync 2021-11-03T17:14:29.712 INFO:tasks.workunit.client.0.smithi027.stdout:9/618: mknod db/d1b/d2b/cd4 1 2021-11-03T17:14:29.712 INFO:tasks.workunit.client.0.smithi027.stdout:5/812: mknod d2/d4/d4e/d67/d8e/c107 1 2021-11-03T17:14:29.713 INFO:tasks.workunit.client.0.smithi027.stdout:4/460: readlink d6/d9/d16/d36/d70/d3e/d92/d98/l54 0 2021-11-03T17:14:29.713 INFO:tasks.workunit.client.0.smithi027.stdout:9/619: chown db/d1b/d1c/d1d/d40/d60 1273899 1 2021-11-03T17:14:29.713 INFO:tasks.workunit.client.0.smithi027.stdout:9/620: stat db/d1b/d8e/fa5 0 2021-11-03T17:14:29.713 INFO:tasks.workunit.client.0.smithi027.stdout:1/682: sync 2021-11-03T17:14:29.713 INFO:tasks.workunit.client.0.smithi027.stdout:9/621: truncate db/d1b/d2b/d41/d6c/f93 1588466 0 2021-11-03T17:14:29.714 INFO:tasks.workunit.client.0.smithi027.stdout:1/683: chown d6/d11/dac/d79/d37/de0/d4e/f91 563761 1 2021-11-03T17:14:29.714 INFO:tasks.workunit.client.0.smithi027.stdout:1/684: readlink d6/d11/dac/d79/dab/db4/lde 0 2021-11-03T17:14:29.714 INFO:tasks.workunit.client.0.smithi027.stdout:8/917: sync 2021-11-03T17:14:29.715 INFO:tasks.workunit.client.0.smithi027.stdout:9/622: read db/d1b/d1c/d1d/f49 [987393,9723] 0 2021-11-03T17:14:29.715 INFO:tasks.workunit.client.0.smithi027.stdout:3/495: write d1/db/d52/f5d [29017,128775] 0 2021-11-03T17:14:29.716 INFO:tasks.workunit.client.0.smithi027.stdout:3/496: chown d1/l5f 3278553 1 2021-11-03T17:14:29.716 INFO:tasks.workunit.client.0.smithi027.stdout:0/776: dread d9/da/d64/f68 [0,4194304] 0 2021-11-03T17:14:29.716 INFO:tasks.workunit.client.0.smithi027.stdout:4/461: rmdir d6/d9/d16/d36/d70/d3e/d92/d98/d21/d41/d44/d79 0 2021-11-03T17:14:29.716 INFO:tasks.workunit.client.0.smithi027.stdout:4/462: readlink d6/d9/d16/d36/d70/l6f 0 2021-11-03T17:14:29.717 INFO:tasks.workunit.client.0.smithi027.stdout:5/813: creat d2/d12/dad/da9/f108 x:0 0 0 2021-11-03T17:14:29.717 INFO:tasks.workunit.client.0.smithi027.stdout:5/814: chown d2/d4/d4e/d67/f11 110879 1 2021-11-03T17:14:29.720 INFO:tasks.workunit.client.0.smithi027.stdout:1/685: link d6/d11/dac/d79/d37/d49/fcb d6/d11/de2/fed 0 2021-11-03T17:14:29.722 INFO:tasks.workunit.client.0.smithi027.stdout:8/918: getdents d7/d10/d3f/d55 0 2021-11-03T17:14:29.722 INFO:tasks.workunit.client.0.smithi027.stdout:8/919: mknod d7/dcc/c12f 1 2021-11-03T17:14:29.727 INFO:tasks.workunit.client.0.smithi027.stdout:9/623: link db/d1b/d1c/d1d/f8f db/d1b/d2b/d41/dad/dbb/fd5 0 2021-11-03T17:14:29.727 INFO:tasks.workunit.client.0.smithi027.stdout:2/628: dread d0/d5/f53 [0,4194304] 0 2021-11-03T17:14:29.729 INFO:tasks.workunit.client.0.smithi027.stdout:9/624: write db/d1b/f64 [3532089,51850] 0 2021-11-03T17:14:29.729 INFO:tasks.workunit.client.0.smithi027.stdout:9/625: read db/d1b/d1c/d1d/f1e [55092,38870] 0 2021-11-03T17:14:29.730 INFO:tasks.workunit.client.0.smithi027.stdout:6/563: dread dc/d16/d1b/d24/f85 [0,4194304] 0 2021-11-03T17:14:29.731 INFO:tasks.workunit.client.0.smithi027.stdout:4/463: dread d6/d9/d16/f26 [0,4194304] 0 2021-11-03T17:14:29.731 INFO:tasks.workunit.client.0.smithi027.stdout:4/464: dread - d6/d9/d16/d36/d70/d3e/d92/d98/d21/d2b/f3f zero size 2021-11-03T17:14:29.731 INFO:tasks.workunit.client.0.smithi027.stdout:0/777: creat d9/da/d64/d70/d6f/f10b x:0 0 0 2021-11-03T17:14:29.732 INFO:tasks.workunit.client.0.smithi027.stdout:5/815: creat d2/d12/dad/da9/f109 x:0 0 0 2021-11-03T17:14:29.736 INFO:tasks.workunit.client.0.smithi027.stdout:9/626: rmdir db/d1b/d2b/d41/d6c 39 2021-11-03T17:14:29.739 INFO:tasks.workunit.client.0.smithi027.stdout:6/564: creat dc/d16/d1b/d3c/d51/fa3 x:0 0 0 2021-11-03T17:14:29.739 INFO:tasks.workunit.client.0.smithi027.stdout:6/565: mknod dc/d82/ca4 1 2021-11-03T17:14:29.740 INFO:tasks.workunit.client.0.smithi027.stdout:5/816: truncate d2/d4/d4e/d67/f4a 2003820 0 2021-11-03T17:14:29.742 INFO:tasks.workunit.client.0.smithi027.stdout:8/920: creat d7/d2d/da8/f130 x:0 0 0 2021-11-03T17:14:29.744 INFO:tasks.workunit.client.0.smithi027.stdout:3/497: dwrite d1/d6f/f83 [0,4194304] 0 2021-11-03T17:14:29.744 INFO:tasks.workunit.client.0.smithi027.stdout:3/498: chown d1/f30 1629057 1 2021-11-03T17:14:29.747 INFO:tasks.workunit.client.0.smithi027.stdout:7/488: dwrite d0/d2/d10/d35/f89 [0,4194304] 0 2021-11-03T17:14:29.747 INFO:tasks.workunit.client.0.smithi027.stdout:7/489: chown d0 1 1 2021-11-03T17:14:29.751 INFO:tasks.workunit.client.0.smithi027.stdout:0/778: dread d9/da/d64/d70/d6d/f6e [0,4194304] 0 2021-11-03T17:14:29.751 INFO:tasks.workunit.client.0.smithi027.stdout:9/627: creat db/fd6 x:0 0 0 2021-11-03T17:14:29.751 INFO:tasks.workunit.client.0.smithi027.stdout:6/566: rename dc/d16/d1b/d3c/l3f to dc/d16/d1b/la5 0 2021-11-03T17:14:29.753 INFO:tasks.workunit.client.0.smithi027.stdout:3/499: creat d1/db/d52/f9a x:0 0 0 2021-11-03T17:14:29.754 INFO:tasks.workunit.client.0.smithi027.stdout:3/500: write d1/f41 [38554,114141] 0 2021-11-03T17:14:29.754 INFO:tasks.workunit.client.0.smithi027.stdout:8/921: rmdir d7/d10/d3f/d55/db7 39 2021-11-03T17:14:29.756 INFO:tasks.workunit.client.0.smithi027.stdout:7/490: creat d0/d2/d1f/fa6 x:0 0 0 2021-11-03T17:14:29.759 INFO:tasks.workunit.client.0.smithi027.stdout:0/779: getdents d9/da/d64/d70/d6d/d9f 0 2021-11-03T17:14:29.759 INFO:tasks.workunit.client.0.smithi027.stdout:9/628: rmdir db/d1b/d1c/d1d/d40 39 2021-11-03T17:14:29.760 INFO:tasks.workunit.client.0.smithi027.stdout:4/465: dwrite d6/d9/d16/d36/d70/d3e/d92/d98/d21/d2b/f3f [0,4194304] 0 2021-11-03T17:14:29.761 INFO:tasks.workunit.client.0.smithi027.stdout:6/567: symlink dc/d16/d1b/la6 0 2021-11-03T17:14:29.763 INFO:tasks.workunit.client.0.smithi027.stdout:5/817: dwrite d2/d12/dad/da9/f108 [0,4194304] 0 2021-11-03T17:14:29.769 INFO:tasks.workunit.client.0.smithi027.stdout:3/501: symlink d1/db/d52/d57/d99/l9b 0 2021-11-03T17:14:29.771 INFO:tasks.workunit.client.0.smithi027.stdout:8/922: rename d7/d10/d3f/d9b/d95/f7e to d7/dcc/f131 0 2021-11-03T17:14:29.781 INFO:tasks.workunit.client.0.smithi027.stdout:0/780: creat d9/dfd/f10c x:0 0 0 2021-11-03T17:14:29.781 INFO:tasks.workunit.client.0.smithi027.stdout:0/781: chown d9/da/d64/d70/d6d/l89 50278 1 2021-11-03T17:14:29.781 INFO:tasks.workunit.client.0.smithi027.stdout:3/502: dread d1/fa [0,4194304] 0 2021-11-03T17:14:29.785 INFO:tasks.workunit.client.0.smithi027.stdout:7/491: dwrite d0/d2/fd [0,4194304] 0 2021-11-03T17:14:29.785 INFO:tasks.workunit.client.0.smithi027.stdout:7/492: write d0/d2/d1c/f27 [2855477,11984] 0 2021-11-03T17:14:29.788 INFO:tasks.workunit.client.0.smithi027.stdout:9/629: creat db/d1b/d2b/d41/dad/fd7 x:0 0 0 2021-11-03T17:14:29.788 INFO:tasks.workunit.client.0.smithi027.stdout:9/630: chown db/d1b/d2b/d41/l42 75 1 2021-11-03T17:14:29.792 INFO:tasks.workunit.client.0.smithi027.stdout:7/493: write d0/d2d/d2f/f39 [1973323,124638] 0 2021-11-03T17:14:29.795 INFO:tasks.workunit.client.0.smithi027.stdout:4/466: dwrite d6/d9/d16/d36/d70/d3e/d92/d98/d21/f37 [0,4194304] 0 2021-11-03T17:14:29.796 INFO:tasks.workunit.client.0.smithi027.stdout:6/568: link dc/d16/d1b/l9c dc/d16/d1b/d2d/d34/d89/la7 0 2021-11-03T17:14:29.801 INFO:tasks.workunit.client.0.smithi027.stdout:8/923: rmdir d7/d10/d3f/d12d 0 2021-11-03T17:14:29.803 INFO:tasks.workunit.client.0.smithi027.stdout:6/569: write dc/f18 [4019629,69555] 0 2021-11-03T17:14:29.807 INFO:tasks.workunit.client.0.smithi027.stdout:9/631: creat db/d1b/d2b/fd8 x:0 0 0 2021-11-03T17:14:29.807 INFO:tasks.workunit.client.0.smithi027.stdout:9/632: dread - db/d1b/d1c/fd0 zero size 2021-11-03T17:14:29.808 INFO:tasks.workunit.client.0.smithi027.stdout:9/633: write db/fc8 [331108,44742] 0 2021-11-03T17:14:29.808 INFO:tasks.workunit.client.0.smithi027.stdout:9/634: mknod db/d1b/d1c/d1d/d40/d60/cd9 1 2021-11-03T17:14:29.809 INFO:tasks.workunit.client.0.smithi027.stdout:4/467: unlink d6/d9/d16/d36/d70/d3e/d92/d98/d21/d41/l50 0 2021-11-03T17:14:29.809 INFO:tasks.workunit.client.0.smithi027.stdout:4/468: fsync d6/d9/d16/d36/d70/f71 0 2021-11-03T17:14:29.809 INFO:tasks.workunit.client.0.smithi027.stdout:4/469: chown d6/d9/d16/d36/d70/d3e/d92/d98/d21/d2b 1654999166 1 2021-11-03T17:14:29.810 INFO:tasks.workunit.client.0.smithi027.stdout:4/470: read d6/d9/d28/f87 [1643848,3741] 0 2021-11-03T17:14:29.810 INFO:tasks.workunit.client.0.smithi027.stdout:8/924: rename d7/d53/d67/ld6 to d7/d111/d11d/l132 0 2021-11-03T17:14:29.811 INFO:tasks.workunit.client.0.smithi027.stdout:8/925: stat d7/dcc/dfe/d11f/l121 0 2021-11-03T17:14:29.814 INFO:tasks.workunit.client.0.smithi027.stdout:2/629: dwrite d0/d1b/d29/d54/d64/daa/d5d/f8f [0,4194304] 0 2021-11-03T17:14:29.815 INFO:tasks.workunit.client.0.smithi027.stdout:7/494: dwrite d0/d2d/d53/fa0 [0,4194304] 0 2021-11-03T17:14:29.816 INFO:tasks.workunit.client.0.smithi027.stdout:3/503: dread d1/d6f/f81 [0,4194304] 0 2021-11-03T17:14:29.816 INFO:tasks.workunit.client.0.smithi027.stdout:3/504: write d1/f4c [1765822,42448] 0 2021-11-03T17:14:29.816 INFO:tasks.workunit.client.0.smithi027.stdout:6/570: dread dc/d16/d1b/d3c/d51/f5b [0,4194304] 0 2021-11-03T17:14:29.818 INFO:tasks.workunit.client.0.smithi027.stdout:2/630: dread d0/d5/f53 [0,4194304] 0 2021-11-03T17:14:29.818 INFO:tasks.workunit.client.0.smithi027.stdout:2/631: fsync d0/d1b/d1c/f7e 0 2021-11-03T17:14:29.819 INFO:tasks.workunit.client.0.smithi027.stdout:8/926: creat d7/f133 x:0 0 0 2021-11-03T17:14:29.820 INFO:tasks.workunit.client.0.smithi027.stdout:0/782: dwrite d9/d12/d8f/fe2 [0,4194304] 0 2021-11-03T17:14:29.820 INFO:tasks.workunit.client.0.smithi027.stdout:0/783: mknod d9/da/d30/dac/dba/ddc/c10d 1 2021-11-03T17:14:29.827 INFO:tasks.workunit.client.0.smithi027.stdout:6/571: truncate dc/f14 3387783 0 2021-11-03T17:14:29.828 INFO:tasks.workunit.client.0.smithi027.stdout:0/784: dread d9/d12/f1d [0,4194304] 0 2021-11-03T17:14:29.828 INFO:tasks.workunit.client.0.smithi027.stdout:9/635: dwrite db/d1b/d23/d2c/f89 [0,4194304] 0 2021-11-03T17:14:29.829 INFO:tasks.workunit.client.0.smithi027.stdout:8/927: creat d7/dcc/dfe/f134 x:0 0 0 2021-11-03T17:14:29.829 INFO:tasks.workunit.client.0.smithi027.stdout:8/928: mknod d7/d53/d67/c135 1 2021-11-03T17:14:29.830 INFO:tasks.workunit.client.0.smithi027.stdout:8/929: stat d7/d2d/d38/l3a 0 2021-11-03T17:14:29.830 INFO:tasks.workunit.client.0.smithi027.stdout:7/495: mkdir d0/d2/d5e/d36/d3a/da7 0 2021-11-03T17:14:29.835 INFO:tasks.workunit.client.0.smithi027.stdout:6/572: unlink dc/l29 0 2021-11-03T17:14:29.835 INFO:tasks.workunit.client.0.smithi027.stdout:6/573: stat dc/d82/d57/f60 0 2021-11-03T17:14:29.836 INFO:tasks.workunit.client.0.smithi027.stdout:6/574: mknod dc/d16/d1b/d3c/d51/d5f/d79/ca8 1 2021-11-03T17:14:29.843 INFO:tasks.workunit.client.0.smithi027.stdout:2/632: dread d0/d1b/d29/f4b [0,4194304] 0 2021-11-03T17:14:29.844 INFO:tasks.workunit.client.0.smithi027.stdout:2/633: write d0/f91 [718223,13228] 0 2021-11-03T17:14:29.844 INFO:tasks.workunit.client.0.smithi027.stdout:2/634: stat d0/d1b/d29/d58/l6f 0 2021-11-03T17:14:29.845 INFO:tasks.workunit.client.0.smithi027.stdout:9/636: creat db/d1b/d2b/d41/dad/fda x:0 0 0 2021-11-03T17:14:29.846 INFO:tasks.workunit.client.0.smithi027.stdout:8/930: mkdir d7/d10/d3f/d55/db7/dc9/d136 0 2021-11-03T17:14:29.854 INFO:tasks.workunit.client.0.smithi027.stdout:6/575: dwrite dc/d82/f5c [4194304,4194304] 0 2021-11-03T17:14:29.854 INFO:tasks.workunit.client.0.smithi027.stdout:6/576: write dc/d16/d1b/d3c/f9e [523744,66131] 0 2021-11-03T17:14:29.855 INFO:tasks.workunit.client.0.smithi027.stdout:6/577: chown dc/d37 1610 1 2021-11-03T17:14:29.855 INFO:tasks.workunit.client.0.smithi027.stdout:0/785: dwrite d9/da/d30/f43 [0,4194304] 0 2021-11-03T17:14:29.858 INFO:tasks.workunit.client.0.smithi027.stdout:4/471: rename d6/d9/d16/d36/d70/d3e/d92 to d6/d9/d16/d36/d70/d3e/d58/d84/d9b 0 2021-11-03T17:14:29.860 INFO:tasks.workunit.client.0.smithi027.stdout:9/637: mkdir db/d1b/d1c/d1d/ddb 0 2021-11-03T17:14:29.865 INFO:tasks.workunit.client.0.smithi027.stdout:0/786: link d9/da/d30/d92/dca/le4 d9/da/d30/da7/l10e 0 2021-11-03T17:14:29.871 INFO:tasks.workunit.client.0.smithi027.stdout:3/505: rename d1/f14 to d1/db/d52/d57/d8b/f9c 0 2021-11-03T17:14:29.871 INFO:tasks.workunit.client.0.smithi027.stdout:3/506: chown d1/d40/l63 57 1 2021-11-03T17:14:29.872 INFO:tasks.workunit.client.0.smithi027.stdout:9/638: truncate db/d1b/d23/d2c/f4c 2033422 0 2021-11-03T17:14:29.873 INFO:tasks.workunit.client.0.smithi027.stdout:9/639: read db/d1b/d1c/f25 [80574,55937] 0 2021-11-03T17:14:29.873 INFO:tasks.workunit.client.0.smithi027.stdout:9/640: chown db/d1b/d2b/d41/dad 11900672 1 2021-11-03T17:14:29.874 INFO:tasks.workunit.client.0.smithi027.stdout:2/635: dwrite d0/d5/d2c/f3d [0,4194304] 0 2021-11-03T17:14:29.874 INFO:tasks.workunit.client.0.smithi027.stdout:4/472: dread d6/d9/d28/d40/f83 [0,4194304] 0 2021-11-03T17:14:29.875 INFO:tasks.workunit.client.0.smithi027.stdout:4/473: write d6/d9/f73 [258578,9431] 0 2021-11-03T17:14:29.875 INFO:tasks.workunit.client.0.smithi027.stdout:6/578: dwrite dc/f4d [0,4194304] 0 2021-11-03T17:14:29.878 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:29 smithi115 conmon[34670]: cephadm 2021-11-03T17:14:28.844070+0000 mgr.smithi115.tcyodz (mgr.24395 2021-11-03T17:14:29.878 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:29 smithi115 conmon[34670]: ) 11 : cephadm [INF] poll took 4546.712 ms: 1 events 2021-11-03T17:14:29.879 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:29 smithi115 conmon[34670]: audit 2021-11-03T17:14:28.873628+0000 mon.smithi027 (mon.0) 600 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' 2021-11-03T17:14:29.879 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:29 smithi115 conmon[34670]: audit 2021-11-03T17:14:29.063601+0000 mon.smithi027 (mon.0) 601 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' 2021-11-03T17:14:29.879 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:29 smithi115 conmon[34670]: cluster 2021-11-03T17:14:29.187371+0000 mgr.smithi115.tcyodz (mgr.24395) 12 : cluster [DBG] pgmap v8: 65 pgs: 65 active+clean; 2.7 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 21 MiB/s rd, 43 MiB/s wr, 96 op/s 2021-11-03T17:14:29.879 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:29 smithi115 conmon[34670]: audit 2021-11-03T17:14:29.509574+0000 mon.smithi027 (mon.0) 602 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' 2021-11-03T17:14:29.880 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:29 smithi115 conmon[34670]: audit 2021-11-03T17:14:29.510921+0000 mon.smithi115 (mon.1) 44 : audit [INF] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2021-11-03T17:14:29.880 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:29 smithi115 conmon[34670]: audit 2021-11-03T17:14:29.511226+0000 mon.smithi027 (mon.0) 603 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2021-11-03T17:14:29.882 INFO:tasks.workunit.client.0.smithi027.stdout:7/496: rename d0/d2/d10/d1e/d20/l67 to d0/d2/d1f/la8 0 2021-11-03T17:14:29.883 INFO:tasks.workunit.client.0.smithi027.stdout:3/507: dread d1/fa [0,4194304] 0 2021-11-03T17:14:29.883 INFO:tasks.workunit.client.0.smithi027.stdout:9/641: symlink db/d1b/ldc 0 2021-11-03T17:14:29.885 INFO:tasks.workunit.client.0.smithi027.stdout:2/636: mkdir d0/d1b/d29/db5 0 2021-11-03T17:14:29.888 INFO:tasks.workunit.client.0.smithi027.stdout:4/474: link d6/d9/d16/d36/d70/d3e/l43 d6/d9/d16/d36/d57/d5a/l9c 0 2021-11-03T17:14:29.889 INFO:tasks.workunit.client.0.smithi027.stdout:6/579: mkdir dc/d82/da9 0 2021-11-03T17:14:29.891 INFO:tasks.workunit.client.0.smithi027.stdout:0/787: dwrite d9/da/d64/f76 [0,4194304] 0 2021-11-03T17:14:29.900 INFO:tasks.workunit.client.0.smithi027.stdout:8/931: rename d7/dcc/dcf to d7/d111/d137 0 2021-11-03T17:14:29.900 INFO:tasks.workunit.client.0.smithi027.stdout:8/932: truncate d7/d10/d3f/d55/db7/dc9/d117/d41/fa3 9207803 0 2021-11-03T17:14:29.900 INFO:tasks.workunit.client.0.smithi027.stdout:7/497: rmdir d0/d2/d1c 39 2021-11-03T17:14:29.903 INFO:tasks.workunit.client.0.smithi027.stdout:3/508: symlink d1/db/l9d 0 2021-11-03T17:14:29.904 INFO:tasks.workunit.client.0.smithi027.stdout:3/509: dread - d1/d40/f6d zero size 2021-11-03T17:14:29.909 INFO:tasks.workunit.client.0.smithi027.stdout:8/933: read d7/d2d/d38/f6e [1564579,29554] 0 2021-11-03T17:14:29.909 INFO:tasks.workunit.client.0.smithi027.stdout:9/642: unlink l5 0 2021-11-03T17:14:29.910 INFO:tasks.workunit.client.0.smithi027.stdout:9/643: stat db/d1b/d2b/dc7 0 2021-11-03T17:14:29.916 INFO:tasks.workunit.client.0.smithi027.stdout:0/788: mkdir d9/da/d10f 0 2021-11-03T17:14:29.920 INFO:tasks.workunit.client.0.smithi027.stdout:3/510: creat d1/d6f/d97/f9e x:0 0 0 2021-11-03T17:14:29.920 INFO:tasks.workunit.client.0.smithi027.stdout:3/511: fsync d1/d6f/f85 0 2021-11-03T17:14:29.920 INFO:tasks.workunit.client.0.smithi027.stdout:3/512: write d1/db/f3f [142534,16858] 0 2021-11-03T17:14:29.921 INFO:tasks.workunit.client.0.smithi027.stdout:3/513: readlink d1/d40/l42 0 2021-11-03T17:14:29.922 INFO:tasks.workunit.client.0.smithi027.stdout:3/514: write d1/db/f86 [1599550,102338] 0 2021-11-03T17:14:29.922 INFO:tasks.workunit.client.0.smithi027.stdout:9/644: rename db/d1b/d1c/d1d/d40/lae to db/d1b/d2b/d41/ldd 0 2021-11-03T17:14:29.923 INFO:tasks.workunit.client.0.smithi027.stdout:9/645: mknod db/d1b/d2b/d41/d95/d9f/cde 1 2021-11-03T17:14:29.923 INFO:tasks.workunit.client.0.smithi027.stdout:6/580: dread dc/d16/d30/f32 [0,4194304] 0 2021-11-03T17:14:29.924 INFO:tasks.workunit.client.0.smithi027.stdout:2/637: dwrite d0/d5/f83 [0,4194304] 0 2021-11-03T17:14:29.924 INFO:tasks.workunit.client.0.smithi027.stdout:4/475: dwrite d6/d24/f46 [0,4194304] 0 2021-11-03T17:14:29.925 INFO:tasks.workunit.client.0.smithi027.stdout:2/638: stat d0/d1b/d29/d54/d64/daa/d5d/f8f 0 2021-11-03T17:14:29.926 INFO:tasks.workunit.client.0.smithi027.stdout:7/498: dwrite d0/d2/d10/d1e/d20/d74/f9d [0,4194304] 0 2021-11-03T17:14:29.929 INFO:tasks.workunit.client.0.smithi027.stdout:3/515: rename d1/db/f3e to d1/db/d52/d6a/d7c/d88/d98/f9f 0 2021-11-03T17:14:29.931 INFO:tasks.workunit.client.0.smithi027.stdout:9/646: creat db/d1b/d1c/d1d/d40/dcc/fdf x:0 0 0 2021-11-03T17:14:29.931 INFO:tasks.workunit.client.0.smithi027.stdout:9/647: mknod db/d1b/d1c/d1d/ddb/ce0 1 2021-11-03T17:14:29.933 INFO:tasks.workunit.client.0.smithi027.stdout:4/476: getdents d6 0 2021-11-03T17:14:29.940 INFO:tasks.workunit.client.0.smithi027.stdout:8/934: dwrite d7/d2d/d38/dc3/fdc [0,4194304] 0 2021-11-03T17:14:29.941 INFO:tasks.workunit.client.0.smithi027.stdout:0/789: dread d9/fc1 [0,4194304] 0 2021-11-03T17:14:29.941 INFO:tasks.workunit.client.0.smithi027.stdout:8/935: write d7/d10/d3f/d55/db7/dc9/d117/f8d [908114,13724] 0 2021-11-03T17:14:29.943 INFO:tasks.workunit.client.0.smithi027.stdout:7/499: symlink d0/d2/d10/d1e/d20/d74/la9 0 2021-11-03T17:14:29.951 INFO:tasks.workunit.client.0.smithi027.stdout:8/936: dread d7/d10/d3f/f47 [0,4194304] 0 2021-11-03T17:14:29.959 INFO:tasks.workunit.client.0.smithi027.stdout:0/790: dwrite d9/da/d30/dac/dba/feb [0,4194304] 0 2021-11-03T17:14:29.964 INFO:tasks.workunit.client.0.smithi027.stdout:3/516: symlink d1/db/d52/d6a/d7c/la0 0 2021-11-03T17:14:29.964 INFO:tasks.workunit.client.0.smithi027.stdout:3/517: mknod d1/d6f/ca1 1 2021-11-03T17:14:29.966 INFO:tasks.workunit.client.0.smithi027.stdout:9/648: link db/d1b/d1c/d1d/l72 db/d1b/d2b/dc7/le1 0 2021-11-03T17:14:29.968 INFO:tasks.workunit.client.0.smithi027.stdout:4/477: mknod d6/d9/d16/c9d 1 2021-11-03T17:14:29.969 INFO:tasks.workunit.client.0.smithi027.stdout:4/478: getdents d6/d8c 0 2021-11-03T17:14:29.973 INFO:tasks.workunit.client.0.smithi027.stdout:8/937: rename d7/d10/d3f/d55/db7/dc9/d103/df0/f49 to d7/d111/d137/f138 0 2021-11-03T17:14:29.982 INFO:tasks.workunit.client.0.smithi027.stdout:0/791: mkdir d9/da/d30/d92/d110 0 2021-11-03T17:14:29.982 INFO:tasks.workunit.client.0.smithi027.stdout:0/792: fsync d9/f1a 0 2021-11-03T17:14:29.982 INFO:tasks.workunit.client.0.smithi027.stdout:4/479: dread d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/f69 [0,4194304] 0 2021-11-03T17:14:29.983 INFO:tasks.workunit.client.0.smithi027.stdout:0/793: dread d9/da/d64/d70/d5f/fad [0,4194304] 0 2021-11-03T17:14:29.983 INFO:tasks.workunit.client.0.smithi027.stdout:0/794: mknod d9/da/d10f/c111 1 2021-11-03T17:14:29.984 INFO:tasks.workunit.client.0.smithi027.stdout:0/795: write d9/da/d64/f76 [1282877,25461] 0 2021-11-03T17:14:29.984 INFO:tasks.workunit.client.0.smithi027.stdout:0/796: stat d9/f23 0 2021-11-03T17:14:29.985 INFO:tasks.workunit.client.0.smithi027.stdout:0/797: chown d9/da/d30/dac/dba 199 1 2021-11-03T17:14:29.985 INFO:tasks.workunit.client.0.smithi027.stdout:3/518: creat d1/db/d52/d6a/fa2 x:0 0 0 2021-11-03T17:14:29.986 INFO:tasks.workunit.client.0.smithi027.stdout:3/519: readlink d1/l4f 0 2021-11-03T17:14:29.986 INFO:tasks.workunit.client.0.smithi027.stdout:3/520: chown d1/db/f7d 440201 1 2021-11-03T17:14:29.986 INFO:tasks.workunit.client.0.smithi027.stdout:3/521: mknod d1/db/d52/d57/d99/ca3 1 2021-11-03T17:14:29.986 INFO:tasks.workunit.client.0.smithi027.stdout:3/522: chown d1/db/f60 13 1 2021-11-03T17:14:29.987 INFO:tasks.workunit.client.0.smithi027.stdout:8/938: rename d7/d53/d67/la0 to d7/d111/d11d/l139 0 2021-11-03T17:14:29.989 INFO:tasks.workunit.client.0.smithi027.stdout:4/480: getdents d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98 0 2021-11-03T17:14:29.989 INFO:tasks.workunit.client.0.smithi027.stdout:4/481: mknod d6/d9/d16/d36/d57/d5a/d63/c9e 1 2021-11-03T17:14:29.990 INFO:tasks.workunit.client.0.smithi027.stdout:4/482: chown d6/d9/d28/d40 1414468 1 2021-11-03T17:14:29.996 INFO:tasks.workunit.client.0.smithi027.stdout:0/798: dread d9/da/d30/dac/dba/fbc [0,4194304] 0 2021-11-03T17:14:29.996 INFO:tasks.workunit.client.0.smithi027.stdout:0/799: read d9/da/d64/fbb [598487,126755] 0 2021-11-03T17:14:29.998 INFO:tasks.workunit.client.0.smithi027.stdout:9/649: truncate db/d1b/d1c/d1d/f49 3703184 0 2021-11-03T17:14:29.999 INFO:tasks.workunit.client.0.smithi027.stdout:8/939: symlink d7/l13a 0 2021-11-03T17:14:30.001 INFO:tasks.workunit.client.0.smithi027.stdout:4/483: mkdir d6/d8c/d9f 0 2021-11-03T17:14:30.002 INFO:tasks.workunit.client.0.smithi027.stdout:0/800: link d9/da/d30/f43 d9/da/d30/dfe/f112 0 2021-11-03T17:14:30.003 INFO:tasks.workunit.client.0.smithi027.stdout:0/801: stat d9/d12/f109 0 2021-11-03T17:14:30.003 INFO:tasks.workunit.client.0.smithi027.stdout:0/802: truncate d9/da/d30/dac/fbe 146599 0 2021-11-03T17:14:30.003 INFO:tasks.workunit.client.0.smithi027.stdout:0/803: write d9/dd3/ffc [986229,105160] 0 2021-11-03T17:14:30.003 INFO:tasks.workunit.client.0.smithi027.stdout:0/804: write d9/da/d30/dac/fec [766745,100106] 0 2021-11-03T17:14:30.004 INFO:tasks.workunit.client.0.smithi027.stdout:9/650: symlink db/d1b/d2b/dc7/le2 0 2021-11-03T17:14:30.005 INFO:tasks.workunit.client.0.smithi027.stdout:8/940: creat d7/d10/d3f/d9b/f13b x:0 0 0 2021-11-03T17:14:30.005 INFO:tasks.workunit.client.0.smithi027.stdout:0/805: creat d9/dd3/f113 x:0 0 0 2021-11-03T17:14:30.006 INFO:tasks.workunit.client.0.smithi027.stdout:3/523: dwrite d1/db/f3f [0,4194304] 0 2021-11-03T17:14:30.006 INFO:tasks.workunit.client.0.smithi027.stdout:3/524: write d1/db/d52/d6a/f95 [112963,100167] 0 2021-11-03T17:14:30.008 INFO:tasks.workunit.client.0.smithi027.stdout:4/484: dread d6/d9/d16/d36/d57/d5a/f67 [0,4194304] 0 2021-11-03T17:14:30.011 INFO:tasks.workunit.client.0.smithi027.stdout:8/941: creat d7/d2d/da8/f13c x:0 0 0 2021-11-03T17:14:30.011 INFO:tasks.workunit.client.0.smithi027.stdout:8/942: chown d7/dcc/dfe/d11f 7 1 2021-11-03T17:14:30.012 INFO:tasks.workunit.client.0.smithi027.stdout:0/806: truncate d9/da/d64/d70/d5f/fad 439095 0 2021-11-03T17:14:30.016 INFO:tasks.workunit.client.0.smithi027.stdout:4/485: mkdir d6/d9/d16/d36/d70/d3e/da0 0 2021-11-03T17:14:30.017 INFO:tasks.workunit.client.0.smithi027.stdout:4/486: fsync d6/d9/d28/f87 0 2021-11-03T17:14:30.018 INFO:tasks.workunit.client.0.smithi027.stdout:9/651: unlink db/d1b/d1c/d1d/d40/d60/fa4 0 2021-11-03T17:14:30.021 INFO:tasks.workunit.client.0.smithi027.stdout:4/487: rmdir d6/d9/d8f 39 2021-11-03T17:14:30.025 INFO:tasks.workunit.client.0.smithi027.stdout:0/807: dread d9/da/d30/dac/dba/feb [0,4194304] 0 2021-11-03T17:14:30.026 INFO:tasks.workunit.client.0.smithi027.stdout:0/808: write d9/dd3/f108 [634262,89568] 0 2021-11-03T17:14:30.026 INFO:tasks.workunit.client.0.smithi027.stdout:0/809: write d9/da/f9d [921107,127918] 0 2021-11-03T17:14:30.026 INFO:tasks.workunit.client.0.smithi027.stdout:0/810: chown d9/da/d64/d70/d5f/l9e 17982 1 2021-11-03T17:14:30.026 INFO:tasks.workunit.client.0.smithi027.stdout:0/811: write d9/da/d30/dac/fbe [377700,44894] 0 2021-11-03T17:14:30.028 INFO:tasks.workunit.client.0.smithi027.stdout:9/652: creat db/d1b/d2b/fe3 x:0 0 0 2021-11-03T17:14:30.029 INFO:tasks.workunit.client.0.smithi027.stdout:9/653: dread - db/f66 zero size 2021-11-03T17:14:30.029 INFO:tasks.workunit.client.0.smithi027.stdout:9/654: mknod db/d1b/d2b/d41/dad/dbb/ce4 1 2021-11-03T17:14:30.033 INFO:tasks.workunit.client.0.smithi027.stdout:3/525: dwrite d1/db/d52/d6a/f95 [0,4194304] 0 2021-11-03T17:14:30.033 INFO:tasks.workunit.client.0.smithi027.stdout:3/526: stat d1/f4c 0 2021-11-03T17:14:30.036 INFO:tasks.workunit.client.0.smithi027.stdout:4/488: rmdir d6/d9/d16/d36/d70/d3e/da0 0 2021-11-03T17:14:30.036 INFO:tasks.workunit.client.0.smithi027.stdout:4/489: chown d6/l59 515635 1 2021-11-03T17:14:30.039 INFO:tasks.workunit.client.0.smithi027.stdout:9/655: link db/d1b/d2b/f86 db/d1b/d2b/dc7/fe5 0 2021-11-03T17:14:30.040 INFO:tasks.workunit.client.0.smithi027.stdout:9/656: write db/d1b/d2b/fe3 [863491,124009] 0 2021-11-03T17:14:30.042 INFO:tasks.workunit.client.0.smithi027.stdout:3/527: creat d1/db/d52/d57/d8b/fa4 x:0 0 0 2021-11-03T17:14:30.044 INFO:tasks.workunit.client.0.smithi027.stdout:4/490: creat d6/d9/d16/d36/d70/d3e/d58/d84/d9b/fa1 x:0 0 0 2021-11-03T17:14:30.048 INFO:tasks.workunit.client.0.smithi027.stdout:9/657: creat db/d1b/d1c/fe6 x:0 0 0 2021-11-03T17:14:30.048 INFO:tasks.workunit.client.0.smithi027.stdout:9/658: mknod db/d1b/d23/ce7 1 2021-11-03T17:14:30.049 INFO:tasks.workunit.client.0.smithi027.stdout:4/491: dread d6/d9/d16/f19 [0,4194304] 0 2021-11-03T17:14:30.049 INFO:tasks.workunit.client.0.smithi027.stdout:4/492: chown d6/d9/d16/d36/d70/d3e/d58/d84/d9b 248 1 2021-11-03T17:14:30.050 INFO:tasks.workunit.client.0.smithi027.stdout:4/493: chown d6/d24/d6d 0 1 2021-11-03T17:14:30.053 INFO:tasks.workunit.client.0.smithi027.stdout:3/528: creat d1/db/fa5 x:0 0 0 2021-11-03T17:14:30.054 INFO:tasks.workunit.client.0.smithi027.stdout:9/659: creat db/fe8 x:0 0 0 2021-11-03T17:14:30.057 INFO:tasks.workunit.client.0.smithi027.stdout:4/494: creat d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/fa2 x:0 0 0 2021-11-03T17:14:30.057 INFO:tasks.workunit.client.0.smithi027.stdout:9/660: mknod db/d1b/d1c/d1d/ddb/ce9 1 2021-11-03T17:14:30.058 INFO:tasks.workunit.client.0.smithi027.stdout:9/661: mknod db/d1b/d8e/cea 1 2021-11-03T17:14:30.058 INFO:tasks.workunit.client.0.smithi027.stdout:4/495: read d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/f8a [108935,99633] 0 2021-11-03T17:14:30.059 INFO:tasks.workunit.client.0.smithi027.stdout:9/662: mkdir db/d1b/d2b/d41/d95/deb 0 2021-11-03T17:14:30.060 INFO:tasks.workunit.client.0.smithi027.stdout:4/496: getdents d6/d8c 0 2021-11-03T17:14:30.060 INFO:tasks.workunit.client.0.smithi027.stdout:4/497: chown d6/d9/d16/d36 4 1 2021-11-03T17:14:30.060 INFO:tasks.workunit.client.0.smithi027.stdout:4/498: chown d6/d9/d16/d36/f42 2074 1 2021-11-03T17:14:30.061 INFO:tasks.workunit.client.0.smithi027.stdout:9/663: creat db/d1b/d1c/dc5/fec x:0 0 0 2021-11-03T17:14:30.061 INFO:tasks.workunit.client.0.smithi027.stdout:9/664: write db/d1b/d2b/fba [993999,10396] 0 2021-11-03T17:14:30.062 INFO:tasks.workunit.client.0.smithi027.stdout:9/665: chown db/d1b/d2b/d41/d6c/l81 128 1 2021-11-03T17:14:30.065 INFO:tasks.workunit.client.0.smithi027.stdout:9/666: symlink db/d1b/d2b/led 0 2021-11-03T17:14:30.068 INFO:tasks.workunit.client.0.smithi027.stdout:9/667: link db/d1b/d2b/d5e/daa/fbf db/d1b/d2b/fee 0 2021-11-03T17:14:30.072 INFO:tasks.workunit.client.0.smithi027.stdout:9/668: rename db/d1b/d2b/d5e/daa/fb7 to db/d1b/d2b/d41/d6c/fef 0 2021-11-03T17:14:30.074 INFO:tasks.workunit.client.0.smithi027.stdout:9/669: symlink db/d1b/d1c/d1d/d40/lf0 0 2021-11-03T17:14:30.075 INFO:tasks.workunit.client.0.smithi027.stdout:3/529: dwrite d1/d40/f6d [0,4194304] 0 2021-11-03T17:14:30.080 INFO:tasks.workunit.client.0.smithi027.stdout:9/670: unlink db/f75 0 2021-11-03T17:14:30.082 INFO:tasks.workunit.client.0.smithi027.stdout:3/530: dread d1/db/f35 [0,4194304] 0 2021-11-03T17:14:30.082 INFO:tasks.workunit.client.0.smithi027.stdout:4/499: dwrite d6/d9/d28/d40/f83 [0,4194304] 0 2021-11-03T17:14:30.082 INFO:tasks.workunit.client.0.smithi027.stdout:9/671: creat db/d1b/d2b/d5e/daa/ff1 x:0 0 0 2021-11-03T17:14:30.083 INFO:tasks.workunit.client.0.smithi027.stdout:3/531: creat d1/d6f/d97/fa6 x:0 0 0 2021-11-03T17:14:30.094 INFO:tasks.workunit.client.0.smithi027.stdout:4/500: creat d6/d8c/d9f/fa3 x:0 0 0 2021-11-03T17:14:30.095 INFO:tasks.workunit.client.0.smithi027.stdout:4/501: write d6/d9/d16/d36/d70/f71 [1913946,125301] 0 2021-11-03T17:14:30.095 INFO:tasks.workunit.client.0.smithi027.stdout:9/672: creat db/d1b/d1c/ff2 x:0 0 0 2021-11-03T17:14:30.097 INFO:tasks.workunit.client.0.smithi027.stdout:3/532: mkdir d1/db/d52/d57/d99/da7 0 2021-11-03T17:14:30.099 INFO:tasks.workunit.client.0.smithi027.stdout:4/502: truncate f4 2927800 0 2021-11-03T17:14:30.101 INFO:tasks.workunit.client.0.smithi027.stdout:9/673: link db/f1f db/d1b/d2b/d41/dad/ff3 0 2021-11-03T17:14:30.104 INFO:tasks.workunit.client.0.smithi027.stdout:4/503: creat d6/d9/d16/d36/d70/d3e/d58/fa4 x:0 0 0 2021-11-03T17:14:30.104 INFO:tasks.workunit.client.0.smithi027.stdout:4/504: stat d6/d9/d28/d40/d81 0 2021-11-03T17:14:30.105 INFO:tasks.workunit.client.0.smithi027.stdout:4/505: readlink d6/d24/d6d/l7d 0 2021-11-03T17:14:30.109 INFO:tasks.workunit.client.0.smithi027.stdout:4/506: dread d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d41/f88 [0,4194304] 0 2021-11-03T17:14:30.113 INFO:tasks.workunit.client.0.smithi027.stdout:9/674: write db/d1b/f45 [3078184,77674] 0 2021-11-03T17:14:30.114 INFO:tasks.workunit.client.0.smithi027.stdout:4/507: symlink d6/d9/d16/d36/la5 0 2021-11-03T17:14:30.114 INFO:tasks.workunit.client.0.smithi027.stdout:4/508: readlink d6/d24/l80 0 2021-11-03T17:14:30.115 INFO:tasks.workunit.client.0.smithi027.stdout:9/675: creat db/d1b/d1c/d1d/ddb/ff4 x:0 0 0 2021-11-03T17:14:30.116 INFO:tasks.workunit.client.0.smithi027.stdout:9/676: dread db/d1b/d2b/f91 [0,4194304] 0 2021-11-03T17:14:30.116 INFO:tasks.workunit.client.0.smithi027.stdout:9/677: mknod db/d1b/d2b/d41/d95/d9f/cf5 1 2021-11-03T17:14:30.117 INFO:tasks.workunit.client.0.smithi027.stdout:4/509: rmdir d6/d9/d28/d40 39 2021-11-03T17:14:30.117 INFO:tasks.workunit.client.0.smithi027.stdout:4/510: mknod d6/d9/ca6 1 2021-11-03T17:14:30.118 INFO:tasks.workunit.client.0.smithi027.stdout:3/533: dwrite d1/f34 [0,4194304] 0 2021-11-03T17:14:30.119 INFO:tasks.workunit.client.0.smithi027.stdout:3/534: read - d1/db/f5b zero size 2021-11-03T17:14:30.119 INFO:tasks.workunit.client.0.smithi027.stdout:9/678: symlink db/d1b/d2b/dc7/lf6 0 2021-11-03T17:14:30.120 INFO:tasks.workunit.client.0.smithi027.stdout:4/511: symlink d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d41/d44/la7 0 2021-11-03T17:14:30.120 INFO:tasks.workunit.client.0.smithi027.stdout:4/512: truncate d6/d9/d28/d40/f53 937814 0 2021-11-03T17:14:30.121 INFO:tasks.workunit.client.0.smithi027.stdout:3/535: link d1/l96 d1/db/d52/d6a/d7c/la8 0 2021-11-03T17:14:30.123 INFO:tasks.workunit.client.0.smithi027.stdout:3/536: link d1/d40/f59 d1/db/d52/d6a/d7c/fa9 0 2021-11-03T17:14:30.124 INFO:tasks.workunit.client.0.smithi027.stdout:3/537: read d1/f50 [2106907,12636] 0 2021-11-03T17:14:30.124 INFO:tasks.workunit.client.0.smithi027.stdout:3/538: dread - d1/d6f/d97/fa6 zero size 2021-11-03T17:14:30.156 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:29 smithi027 conmon[32322]: cephadm 2021-11-03T17:14:28. 2021-11-03T17:14:30.156 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:29 smithi027 conmon[32322]: 844070+0000 mgr.smithi115.tcyodz (mgr.24395) 11 : cephadm [INF] poll took 4546.712 ms: 1 events 2021-11-03T17:14:30.156 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:29 smithi027 conmon[32322]: audit 2021-11-03T17:14:28.873628+0000 mon.smithi027 (mon.0) 600 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' 2021-11-03T17:14:30.157 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:29 smithi027 conmon[32322]: audit 2021-11-03T17:14:29.063601+0000 mon.smithi027 (mon.0) 601 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' 2021-11-03T17:14:30.157 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:29 smithi027 conmon[32322]: cluster 2021-11-03T17:14:29.187371+0000 mgr.smithi115.tcyodz ( 2021-11-03T17:14:30.157 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:29 smithi027 conmon[32322]: mgr.24395) 12 : cluster [DBG] pgmap v8: 65 pgs: 65 active+clean; 2.7 GiB data, 8.0 GiB used, 528 GiB / 536 GiB avail; 21 MiB/s rd, 43 MiB/s wr, 96 op/s 2021-11-03T17:14:30.157 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:29 smithi027 conmon[32322]: audit 2021-11- 2021-11-03T17:14:30.158 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:29 smithi027 conmon[32322]: 03T17:14:29.509574+0000 mon.smithi027 ( 2021-11-03T17:14:30.158 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:29 smithi027 conmon[32322]: mon.0) 602 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' 2021-11-03T17:14:30.158 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:29 smithi027 conmon[32322]: audit 2021- 2021-11-03T17:14:30.158 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:29 smithi027 conmon[32322]: 11-03T17:14:29.510921+0000 2021-11-03T17:14:30.159 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:29 smithi027 conmon[32322]: mon.smithi115 (mon.1) 44 : audit [INF] 2021-11-03T17:14:30.159 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:29 smithi027 conmon[32322]: from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2021-11-03T17:14:30.159 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:29 smithi027 conmon[32322]: audit 2021-11-03 2021-11-03T17:14:30.159 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:29 smithi027 conmon[32322]: T17:14:29.511226+0000 mon.smithi027 (mon 2021-11-03T17:14:30.159 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:29 smithi027 conmon[32322]: .0) 603 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2021-11-03T17:14:30.189 INFO:tasks.workunit.client.0.smithi027.stdout:3/539: creat d1/db/d52/d57/d99/faa x:0 0 0 2021-11-03T17:14:30.189 INFO:tasks.workunit.client.0.smithi027.stdout:3/540: truncate d1/db/d52/f9a 717642 0 2021-11-03T17:14:30.191 INFO:tasks.workunit.client.0.smithi027.stdout:3/541: unlink d1/f55 0 2021-11-03T17:14:30.199 INFO:tasks.workunit.client.0.smithi027.stdout:3/542: creat d1/db/d52/d6a/d7c/fab x:0 0 0 2021-11-03T17:14:30.214 INFO:tasks.workunit.client.0.smithi027.stdout:3/543: dwrite d1/db/d52/d57/d8b/f9c [0,4194304] 0 2021-11-03T17:14:30.216 INFO:tasks.workunit.client.0.smithi027.stdout:3/544: symlink d1/db/d52/d57/d99/da7/lac 0 2021-11-03T17:14:30.216 INFO:tasks.workunit.client.0.smithi027.stdout:3/545: write d1/d40/f6d [5234972,103445] 0 2021-11-03T17:14:30.222 INFO:tasks.workunit.client.0.smithi027.stdout:3/546: dread d1/db/f93 [0,4194304] 0 2021-11-03T17:14:30.223 INFO:tasks.workunit.client.0.smithi027.stdout:3/547: mkdir d1/d6f/dad 0 2021-11-03T17:14:30.225 INFO:tasks.workunit.client.0.smithi027.stdout:3/548: write d1/db/f93 [4677793,52147] 0 2021-11-03T17:14:30.225 INFO:tasks.workunit.client.0.smithi027.stdout:3/549: truncate d1/db/f93 7942887 0 2021-11-03T17:14:30.233 INFO:tasks.workunit.client.0.smithi027.stdout:3/550: dread d1/fa [4194304,4194304] 0 2021-11-03T17:14:30.234 INFO:tasks.workunit.client.0.smithi027.stdout:3/551: read d1/db/f3c [736703,82988] 0 2021-11-03T17:14:30.234 INFO:tasks.workunit.client.0.smithi027.stdout:3/552: mknod d1/db/d52/d57/d8b/cae 1 2021-11-03T17:14:30.234 INFO:tasks.workunit.client.0.smithi027.stdout:3/553: chown d1/l2c 110 1 2021-11-03T17:14:30.235 INFO:tasks.workunit.client.0.smithi027.stdout:3/554: rmdir d1 39 2021-11-03T17:14:30.243 INFO:tasks.workunit.client.0.smithi027.stdout:3/555: rmdir d1/db/d52/d6a 39 2021-11-03T17:14:30.243 INFO:tasks.workunit.client.0.smithi027.stdout:3/556: chown d1/db/l9d 87 1 2021-11-03T17:14:30.243 INFO:tasks.workunit.client.0.smithi027.stdout:3/557: chown d1/d40/l63 10 1 2021-11-03T17:14:30.260 INFO:tasks.workunit.client.0.smithi027.stdout:3/558: dwrite d1/db/d52/d57/d8b/fa4 [0,4194304] 0 2021-11-03T17:14:30.263 INFO:tasks.workunit.client.0.smithi027.stdout:3/559: link d1/l36 d1/db/d52/laf 0 2021-11-03T17:14:30.264 INFO:tasks.workunit.client.0.smithi027.stdout:3/560: symlink d1/d40/lb0 0 2021-11-03T17:14:30.264 INFO:tasks.workunit.client.0.smithi027.stdout:3/561: write d1/db/f7d [388512,35843] 0 2021-11-03T17:14:30.281 INFO:tasks.workunit.client.0.smithi027.stdout:3/562: dwrite d1/d40/f8a [0,4194304] 0 2021-11-03T17:14:30.288 INFO:tasks.workunit.client.0.smithi027.stdout:3/563: dread d1/d40/f87 [0,4194304] 0 2021-11-03T17:14:30.291 INFO:tasks.workunit.client.0.smithi027.stdout:3/564: link d1/db/f35 d1/db/d52/d57/d99/da7/fb1 0 2021-11-03T17:14:30.292 INFO:tasks.workunit.client.0.smithi027.stdout:3/565: write d1/d40/f6d [362155,84047] 0 2021-11-03T17:14:30.292 INFO:tasks.workunit.client.0.smithi027.stdout:3/566: chown d1/db/d52 51 1 2021-11-03T17:14:30.292 INFO:tasks.workunit.client.0.smithi027.stdout:3/567: readlink d1/l2c 0 2021-11-03T17:14:30.295 INFO:tasks.workunit.client.0.smithi027.stdout:3/568: unlink d1/db/d52/d6a/d7c/d88/d98/f9f 0 2021-11-03T17:14:30.299 INFO:tasks.workunit.client.0.smithi027.stdout:3/569: dread d1/d6f/f81 [0,4194304] 0 2021-11-03T17:14:30.301 INFO:tasks.workunit.client.0.smithi027.stdout:3/570: mkdir d1/db/db2 0 2021-11-03T17:14:30.302 INFO:tasks.workunit.client.0.smithi027.stdout:3/571: getdents d1/db/d52/d57/d8b 0 2021-11-03T17:14:30.318 INFO:tasks.workunit.client.0.smithi027.stdout:3/572: dwrite d1/db/f5b [0,4194304] 0 2021-11-03T17:14:30.322 INFO:tasks.workunit.client.0.smithi027.stdout:3/573: rename d1/db/l39 to d1/d6f/dad/lb3 0 2021-11-03T17:14:30.323 INFO:tasks.workunit.client.0.smithi027.stdout:3/574: unlink d1/db/d52/d6a/l77 0 2021-11-03T17:14:30.324 INFO:tasks.workunit.client.0.smithi027.stdout:3/575: mkdir d1/db/d52/d6a/db4 0 2021-11-03T17:14:30.325 INFO:tasks.workunit.client.0.smithi027.stdout:3/576: fdatasync d1/f41 0 2021-11-03T17:14:30.325 INFO:tasks.workunit.client.0.smithi027.stdout:3/577: fdatasync d1/d40/f5c 0 2021-11-03T17:14:30.330 INFO:tasks.workunit.client.0.smithi027.stdout:3/578: rename d1/d6f to d1/db/d52/d6a/db4/db5 0 2021-11-03T17:14:30.331 INFO:tasks.workunit.client.0.smithi027.stdout:3/579: write d1/db/d52/d6a/d7c/fa9 [900086,101279] 0 2021-11-03T17:14:30.332 INFO:tasks.workunit.client.0.smithi027.stdout:3/580: creat d1/d40/fb6 x:0 0 0 2021-11-03T17:14:30.340 INFO:tasks.workunit.client.0.smithi027.stdout:3/581: dread d1/db/d52/d57/d8b/f9c [0,4194304] 0 2021-11-03T17:14:30.341 INFO:tasks.workunit.client.0.smithi027.stdout:3/582: creat d1/db/d52/d57/d8b/fb7 x:0 0 0 2021-11-03T17:14:30.343 INFO:tasks.workunit.client.0.smithi027.stdout:3/583: link d1/d40/f48 d1/fb8 0 2021-11-03T17:14:30.345 INFO:tasks.workunit.client.0.smithi027.stdout:3/584: truncate d1/db/d52/d57/d99/da7/fb1 3441070 0 2021-11-03T17:14:30.348 INFO:tasks.workunit.client.0.smithi027.stdout:3/585: dread d1/db/d52/d57/d8b/f9c [0,4194304] 0 2021-11-03T17:14:30.348 INFO:tasks.workunit.client.0.smithi027.stdout:3/586: dread - d1/db/d52/d6a/db4/db5/d97/f9e zero size 2021-11-03T17:14:30.350 INFO:tasks.workunit.client.0.smithi027.stdout:3/587: mkdir d1/db/d52/db9 0 2021-11-03T17:14:30.351 INFO:tasks.workunit.client.0.smithi027.stdout:3/588: write d1/d40/f6d [3656145,94613] 0 2021-11-03T17:14:30.354 INFO:tasks.workunit.client.0.smithi027.stdout:3/589: symlink d1/lba 0 2021-11-03T17:14:30.354 INFO:tasks.workunit.client.0.smithi027.stdout:3/590: stat d1/d40/l63 0 2021-11-03T17:14:30.354 INFO:tasks.workunit.client.0.smithi027.stdout:3/591: write d1/db/d52/d6a/db4/db5/d97/f9e [506624,114551] 0 2021-11-03T17:14:30.355 INFO:tasks.workunit.client.0.smithi027.stdout:3/592: symlink d1/db/d52/d6a/db4/lbb 0 2021-11-03T17:14:30.374 INFO:tasks.workunit.client.0.smithi027.stdout:3/593: dwrite d1/db/f1c [4194304,4194304] 0 2021-11-03T17:14:30.388 INFO:tasks.workunit.client.0.smithi027.stdout:3/594: dwrite d1/db/d52/d6a/fa2 [0,4194304] 0 2021-11-03T17:14:30.402 INFO:tasks.workunit.client.0.smithi027.stdout:3/595: dwrite d1/d40/f5c [0,4194304] 0 2021-11-03T17:14:30.419 INFO:tasks.workunit.client.0.smithi027.stdout:3/596: dwrite d1/db/f60 [0,4194304] 0 2021-11-03T17:14:30.424 INFO:tasks.workunit.client.0.smithi027.stdout:3/597: rmdir d1/db 39 2021-11-03T17:14:30.426 INFO:tasks.workunit.client.0.smithi027.stdout:3/598: creat d1/db/d52/d6a/d7c/d88/d98/fbc x:0 0 0 2021-11-03T17:14:30.426 INFO:tasks.workunit.client.0.smithi027.stdout:3/599: getdents d1/db/db2 0 2021-11-03T17:14:30.428 INFO:tasks.workunit.client.0.smithi027.stdout:3/600: mkdir d1/db/db2/dbd 0 2021-11-03T17:14:30.429 INFO:tasks.workunit.client.0.smithi027.stdout:3/601: readlink d1/d40/lb0 0 2021-11-03T17:14:30.430 INFO:tasks.workunit.client.0.smithi027.stdout:3/602: read d1/f1d [3241997,66689] 0 2021-11-03T17:14:30.430 INFO:tasks.workunit.client.0.smithi027.stdout:3/603: chown d1/l36 64106942 1 2021-11-03T17:14:30.430 INFO:tasks.workunit.client.0.smithi027.stdout:3/604: fdatasync d1/db/d52/d57/d99/faa 0 2021-11-03T17:14:30.431 INFO:tasks.workunit.client.0.smithi027.stdout:3/605: readlink d1/l2b 0 2021-11-03T17:14:30.431 INFO:tasks.workunit.client.0.smithi027.stdout:3/606: dread d1/db/d52/d6a/f7f [0,4194304] 0 2021-11-03T17:14:30.450 INFO:tasks.workunit.client.0.smithi027.stdout:3/607: dwrite d1/f90 [0,4194304] 0 2021-11-03T17:14:30.451 INFO:tasks.workunit.client.0.smithi027.stdout:3/608: mknod d1/db/d52/d6a/db4/db5/d97/cbe 1 2021-11-03T17:14:30.451 INFO:tasks.workunit.client.0.smithi027.stdout:3/609: truncate d1/d40/f8f 763857 0 2021-11-03T17:14:30.472 INFO:tasks.workunit.client.0.smithi027.stdout:3/610: dwrite d1/db/f86 [0,4194304] 0 2021-11-03T17:14:30.473 INFO:tasks.workunit.client.0.smithi027.stdout:3/611: dread d1/db/d52/d57/f62 [0,4194304] 0 2021-11-03T17:14:30.476 INFO:tasks.workunit.client.0.smithi027.stdout:3/612: write d1/db/d52/f64 [3440209,98024] 0 2021-11-03T17:14:30.481 INFO:tasks.workunit.client.0.smithi027.stdout:3/613: dread d1/db/d52/d6a/db4/db5/f81 [0,4194304] 0 2021-11-03T17:14:30.482 INFO:tasks.workunit.client.0.smithi027.stdout:3/614: getdents d1/db 0 2021-11-03T17:14:30.485 INFO:tasks.workunit.client.0.smithi027.stdout:3/615: fdatasync d1/d40/f59 0 2021-11-03T17:14:30.485 INFO:tasks.workunit.client.0.smithi027.stdout:3/616: fsync d1/db/d52/d6a/d7c/d88/d98/fbc 0 2021-11-03T17:14:30.494 INFO:tasks.workunit.client.0.smithi027.stdout:3/617: dread d1/f1d [0,4194304] 0 2021-11-03T17:14:30.495 INFO:tasks.workunit.client.0.smithi027.stdout:3/618: mkdir d1/db/d52/d57/d99/da7/dbf 0 2021-11-03T17:14:30.495 INFO:tasks.workunit.client.0.smithi027.stdout:3/619: fdatasync d1/d40/f4a 0 2021-11-03T17:14:30.496 INFO:tasks.workunit.client.0.smithi027.stdout:3/620: symlink d1/db/d52/d57/d99/da7/dbf/lc0 0 2021-11-03T17:14:30.504 INFO:tasks.workunit.client.0.smithi027.stdout:3/621: truncate d1/db/d52/d57/d8b/f9c 3921312 0 2021-11-03T17:14:30.504 INFO:tasks.workunit.client.0.smithi027.stdout:3/622: chown d1/db/d52/d6a/db4/db5/l75 21489 1 2021-11-03T17:14:30.504 INFO:tasks.workunit.client.0.smithi027.stdout:3/623: fsync d1/db/d52/d57/f71 0 2021-11-03T17:14:30.504 INFO:tasks.workunit.client.0.smithi027.stdout:3/624: dread - d1/db/d52/d57/d8b/fb7 zero size 2021-11-03T17:14:30.520 INFO:tasks.workunit.client.0.smithi027.stdout:3/625: dwrite d1/d40/f8f [0,4194304] 0 2021-11-03T17:14:30.523 INFO:tasks.workunit.client.0.smithi027.stdout:3/626: rmdir d1/db/db2 39 2021-11-03T17:14:30.523 INFO:tasks.workunit.client.0.smithi027.stdout:3/627: mknod d1/db/d52/d6a/db4/db5/d97/cc1 1 2021-11-03T17:14:30.524 INFO:tasks.workunit.client.0.smithi027.stdout:3/628: truncate d1/db/d52/d57/d8b/f9c 294804 0 2021-11-03T17:14:30.542 INFO:tasks.workunit.client.0.smithi027.stdout:3/629: dwrite d1/db/d52/f5d [0,4194304] 0 2021-11-03T17:14:30.559 INFO:tasks.workunit.client.0.smithi027.stdout:3/630: dwrite d1/db/d52/d6a/db4/db5/f81 [0,4194304] 0 2021-11-03T17:14:30.561 INFO:tasks.workunit.client.0.smithi027.stdout:3/631: rename d1/db/fa5 to d1/db/d52/d6a/db4/db5/d97/fc2 0 2021-11-03T17:14:30.562 INFO:tasks.workunit.client.0.smithi027.stdout:3/632: symlink d1/db/d52/d57/lc3 0 2021-11-03T17:14:30.563 INFO:tasks.workunit.client.0.smithi027.stdout:3/633: chown d1/db/d52/d6a/d7c/la0 132505208 1 2021-11-03T17:14:30.585 INFO:tasks.workunit.client.0.smithi027.stdout:3/634: dwrite d1/d40/f59 [0,4194304] 0 2021-11-03T17:14:30.598 INFO:tasks.workunit.client.0.smithi027.stdout:3/635: dwrite d1/f30 [4194304,4194304] 0 2021-11-03T17:14:30.598 INFO:tasks.workunit.client.0.smithi027.stdout:3/636: readlink d1/db/d52/d6a/l72 0 2021-11-03T17:14:30.599 INFO:tasks.workunit.client.0.smithi027.stdout:3/637: dread - d1/db/d52/d6a/d7c/d88/d98/fbc zero size 2021-11-03T17:14:30.599 INFO:tasks.workunit.client.0.smithi027.stdout:3/638: mknod d1/db/d52/d57/cc4 1 2021-11-03T17:14:30.615 INFO:tasks.workunit.client.0.smithi027.stdout:3/639: dwrite d1/db/d52/d6a/db4/db5/f85 [0,4194304] 0 2021-11-03T17:14:30.615 INFO:tasks.workunit.client.0.smithi027.stdout:3/640: read d1/db/d52/d57/d8b/f9c [12735,41750] 0 2021-11-03T17:14:30.616 INFO:tasks.workunit.client.0.smithi027.stdout:3/641: creat d1/db/d52/db9/fc5 x:0 0 0 2021-11-03T17:14:30.617 INFO:tasks.workunit.client.0.smithi027.stdout:3/642: chown d1/f1d 0 1 2021-11-03T17:14:30.618 INFO:tasks.workunit.client.0.smithi027.stdout:3/643: mkdir d1/db/d52/d6a/db4/db5/dad/dc6 0 2021-11-03T17:14:30.618 INFO:tasks.workunit.client.0.smithi027.stdout:3/644: write d1/db/d52/d57/d99/faa [454425,113443] 0 2021-11-03T17:14:30.619 INFO:tasks.workunit.client.0.smithi027.stdout:3/645: chown d1/db/f32 233 1 2021-11-03T17:14:30.620 INFO:tasks.workunit.client.0.smithi027.stdout:3/646: rename d1/l3a to d1/db/d52/d6a/db4/db5/d97/lc7 0 2021-11-03T17:14:30.621 INFO:tasks.workunit.client.0.smithi027.stdout:3/647: write d1/d40/f5c [259731,7127] 0 2021-11-03T17:14:30.622 INFO:tasks.workunit.client.0.smithi027.stdout:3/648: rename d1/f90 to d1/db/d52/d6a/d7c/d88/fc8 0 2021-11-03T17:14:30.624 INFO:tasks.workunit.client.0.smithi027.stdout:3/649: creat d1/db/d52/d6a/db4/db5/d97/fc9 x:0 0 0 2021-11-03T17:14:30.625 INFO:tasks.workunit.client.0.smithi027.stdout:3/650: write d1/d40/f44 [898787,105493] 0 2021-11-03T17:14:30.626 INFO:tasks.workunit.client.0.smithi027.stdout:3/651: write d1/f5e [2274361,8696] 0 2021-11-03T17:14:30.644 INFO:tasks.workunit.client.0.smithi027.stdout:3/652: dwrite d1/f5e [0,4194304] 0 2021-11-03T17:14:30.646 INFO:tasks.workunit.client.0.smithi027.stdout:3/653: symlink d1/db/db2/lca 0 2021-11-03T17:14:30.646 INFO:tasks.workunit.client.0.smithi027.stdout:3/654: fsync d1/db/f35 0 2021-11-03T17:14:30.648 INFO:tasks.workunit.client.0.smithi027.stdout:3/655: unlink d1/db/l2d 0 2021-11-03T17:14:30.649 INFO:tasks.workunit.client.0.smithi027.stdout:3/656: truncate d1/d40/f4a 3157895 0 2021-11-03T17:14:30.652 INFO:tasks.workunit.client.0.smithi027.stdout:3/657: unlink d1/db/l7e 0 2021-11-03T17:14:30.652 INFO:tasks.workunit.client.0.smithi027.stdout:3/658: mknod d1/db/d52/d57/d99/da7/dbf/ccb 1 2021-11-03T17:14:30.653 INFO:tasks.workunit.client.0.smithi027.stdout:3/659: creat d1/db/fcc x:0 0 0 2021-11-03T17:14:30.653 INFO:tasks.workunit.client.0.smithi027.stdout:3/660: readlink d1/db/d52/d6a/db4/db5/dad/lb3 0 2021-11-03T17:14:30.654 INFO:tasks.workunit.client.0.smithi027.stdout:3/661: write d1/fb8 [2761570,8413] 0 2021-11-03T17:14:30.660 INFO:tasks.workunit.client.0.smithi027.stdout:3/662: link d1/l1b d1/db/lcd 0 2021-11-03T17:14:30.679 INFO:tasks.workunit.client.0.smithi027.stdout:3/663: dwrite d1/db/d52/d6a/db4/db5/d97/f9e [0,4194304] 0 2021-11-03T17:14:30.694 INFO:tasks.workunit.client.0.smithi027.stdout:3/664: dwrite d1/db/d52/d57/d8b/f9c [0,4194304] 0 2021-11-03T17:14:30.760 INFO:tasks.workunit.client.0.smithi027.stdout:3/665: dwrite d1/db/d52/d57/d8b/f9c [0,4194304] 0 2021-11-03T17:14:30.761 INFO:tasks.workunit.client.0.smithi027.stdout:3/666: chown d1/db/d52/d6a/db4/db5/f81 40 1 2021-11-03T17:14:30.762 INFO:tasks.workunit.client.0.smithi027.stdout:3/667: link d1/db/d52/d6a/d7c/f89 d1/db/d52/d57/d99/fce 0 2021-11-03T17:14:30.763 INFO:tasks.workunit.client.0.smithi027.stdout:3/668: chown d1/d40/f76 6895619 1 2021-11-03T17:14:30.763 INFO:tasks.workunit.client.0.smithi027.stdout:3/669: mknod d1/db/d52/d6a/d7c/ccf 1 2021-11-03T17:14:30.764 INFO:tasks.workunit.client.0.smithi027.stdout:3/670: link d1/f1d d1/db/d52/d6a/d7c/fd0 0 2021-11-03T17:14:30.765 INFO:tasks.workunit.client.0.smithi027.stdout:3/671: symlink d1/d40/ld1 0 2021-11-03T17:14:30.766 INFO:tasks.workunit.client.0.smithi027.stdout:3/672: fdatasync d1/db/d52/d57/d8b/fb7 0 2021-11-03T17:14:30.767 INFO:tasks.workunit.client.0.smithi027.stdout:3/673: rename d1/db/d52/db9/fc5 to d1/db/d52/d57/fd2 0 2021-11-03T17:14:30.768 INFO:tasks.workunit.client.0.smithi027.stdout:3/674: getdents d1/db/d52/d6a/db4/db5/d97 0 2021-11-03T17:14:30.771 INFO:tasks.workunit.client.0.smithi027.stdout:3/675: creat d1/db/d52/d6a/db4/fd3 x:0 0 0 2021-11-03T17:14:30.772 INFO:tasks.workunit.client.0.smithi027.stdout:3/676: fdatasync d1/d40/f59 0 2021-11-03T17:14:30.779 INFO:tasks.workunit.client.0.smithi027.stdout:3/677: write d1/f1d [2881434,76954] 0 2021-11-03T17:14:30.784 INFO:tasks.workunit.client.0.smithi027.stdout:3/678: truncate d1/d40/f44 3928451 0 2021-11-03T17:14:30.786 INFO:tasks.workunit.client.0.smithi027.stdout:3/679: creat d1/db/fd4 x:0 0 0 2021-11-03T17:14:30.787 INFO:tasks.workunit.client.0.smithi027.stdout:3/680: truncate d1/d40/f8a 2715594 0 2021-11-03T17:14:30.788 INFO:tasks.workunit.client.0.smithi027.stdout:3/681: write d1/db/f7d [1175025,64990] 0 2021-11-03T17:14:30.788 INFO:tasks.workunit.client.0.smithi027.stdout:3/682: write d1/f50 [9402764,24857] 0 2021-11-03T17:14:30.789 INFO:tasks.workunit.client.0.smithi027.stdout:3/683: mkdir d1/db/d52/d6a/dd5 0 2021-11-03T17:14:30.790 INFO:tasks.workunit.client.0.smithi027.stdout:3/684: creat d1/db/d52/d6a/d7c/fd6 x:0 0 0 2021-11-03T17:14:30.793 INFO:tasks.workunit.client.0.smithi027.stdout:3/685: rmdir d1/db/d52/d57/d99/da7/dbf 39 2021-11-03T17:14:30.806 INFO:tasks.workunit.client.0.smithi027.stdout:3/686: dread d1/db/d52/f65 [0,4194304] 0 2021-11-03T17:14:30.808 INFO:tasks.workunit.client.0.smithi027.stdout:3/687: rename d1/d40/l42 to d1/db/d52/ld7 0 2021-11-03T17:14:30.817 INFO:tasks.workunit.client.0.smithi027.stdout:3/688: dread d1/f13 [0,4194304] 0 2021-11-03T17:14:30.817 INFO:tasks.workunit.client.0.smithi027.stdout:3/689: write d1/d40/fb6 [1013661,87679] 0 2021-11-03T17:14:30.823 INFO:tasks.workunit.client.0.smithi027.stdout:3/690: dread d1/db/d52/d57/d8b/f9c [0,4194304] 0 2021-11-03T17:14:30.824 INFO:tasks.workunit.client.0.smithi027.stdout:3/691: chown d1/f18 534 1 2021-11-03T17:14:30.824 INFO:tasks.workunit.client.0.smithi027.stdout:3/692: chown d1/d40/f46 39 1 2021-11-03T17:14:30.824 INFO:tasks.workunit.client.0.smithi027.stdout:3/693: getdents d1/db/d52/d6a/dd5 0 2021-11-03T17:14:30.825 INFO:tasks.workunit.client.0.smithi027.stdout:3/694: rename d1/db/d52/d57/f62 to d1/db/d52/db9/fd8 0 2021-11-03T17:14:30.825 INFO:tasks.workunit.client.0.smithi027.stdout:3/695: chown d1/d40/l69 659960 1 2021-11-03T17:14:30.826 INFO:tasks.workunit.client.0.smithi027.stdout:3/696: mknod d1/db/d52/d6a/dd5/cd9 1 2021-11-03T17:14:30.827 INFO:tasks.workunit.client.0.smithi027.stdout:3/697: unlink d1/f2f 0 2021-11-03T17:14:30.827 INFO:tasks.workunit.client.0.smithi027.stdout:3/698: write d1/db/d52/d57/d99/fce [1589724,84009] 0 2021-11-03T17:14:30.828 INFO:tasks.workunit.client.0.smithi027.stdout:3/699: symlink d1/db/d52/d6a/dd5/lda 0 2021-11-03T17:14:30.829 INFO:tasks.workunit.client.0.smithi027.stdout:3/700: symlink d1/db/d52/d57/ldb 0 2021-11-03T17:14:30.830 INFO:tasks.workunit.client.0.smithi027.stdout:3/701: stat d1/db/f3f 0 2021-11-03T17:14:30.848 INFO:tasks.workunit.client.0.smithi027.stdout:3/702: dwrite d1/db/f7d [0,4194304] 0 2021-11-03T17:14:30.860 INFO:tasks.workunit.client.0.smithi027.stdout:3/703: dread d1/db/f7d [0,4194304] 0 2021-11-03T17:14:30.861 INFO:tasks.workunit.client.0.smithi027.stdout:3/704: fsync d1/f41 0 2021-11-03T17:14:30.861 INFO:tasks.workunit.client.0.smithi027.stdout:3/705: mknod d1/db/d52/db9/cdc 1 2021-11-03T17:14:30.861 INFO:tasks.workunit.client.0.smithi027.stdout:3/706: write d1/db/d52/d57/d99/fce [1144851,108454] 0 2021-11-03T17:14:31.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:31 smithi115 conmon[34670]: cluster 2021-11-03T17:14:30.159703+0000 mon.smithi027 (mon.0) 604 : 2021-11-03T17:14:31.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:31 smithi115 conmon[34670]: cluster [DBG] Standby manager daemon smithi027.ufhhvb restarted 2021-11-03T17:14:31.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:31 smithi115 conmon[34670]: cluster 2021-11-03T17:14:30.159811+0000 mon.smithi027 (mon.0) 605 : cluster [DBG] Standby manager daemon smithi027.ufhhvb started 2021-11-03T17:14:31.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:31 smithi115 conmon[34670]: audit 2021-11-03T17:14:30.162431+0000 mon.smithi027 ( 2021-11-03T17:14:31.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:31 smithi115 conmon[34670]: mon.0) 606 : audit [DBG] from='mgr.? 172.21.15.27:0/833939901' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi027.ufhhvb/crt"}]: dispatch 2021-11-03T17:14:31.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:31 smithi115 conmon[34670]: audit 2021-11-03T17:14:30.163834+0000 mon.smithi027 (mon 2021-11-03T17:14:31.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:31 smithi115 conmon[34670]: .0) 607 : audit [DBG] from='mgr.? 172.21.15.27:0/833939901' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-11-03T17:14:31.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:31 smithi115 conmon[34670]: audit 2021-11-03T17:14:31.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:31 smithi115 conmon[34670]: 2021-11-03T17:14:30.166377+0000 mon.smithi027 (mon.0) 608 : audit [DBG] from='mgr.? 172.21.15.27:0/833939901' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi027.ufhhvb/key"}]: dispatch 2021-11-03T17:14:31.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:31 smithi115 conmon[34670]: audit 2021-11- 2021-11-03T17:14:31.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:31 smithi115 conmon[34670]: 03T17:14:30.166770+0000 mon.smithi027 (mon.0) 609 : audit 2021-11-03T17:14:31.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:31 smithi115 conmon[34670]: [DBG] from='mgr.? 172.21.15.27:0/833939901' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-11-03T17:14:31.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:31 smithi115 conmon[34670]: audit 2021-11-03T17:14:30.229779+0000 mon.smithi027 (mon. 2021-11-03T17:14:31.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:31 smithi115 conmon[34670]: 0) 610 : audit [INF] from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2021-11-03T17:14:31.510 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:31 smithi027 conmon[32322]: cluster 2021-11-03T17:14:30.159703+0000 2021-11-03T17:14:31.510 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:31 smithi027 conmon[32322]: mon.smithi027 (mon.0) 604 : cluster [DBG] Standby manager daemon smithi027.ufhhvb restarted 2021-11-03T17:14:31.511 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:31 smithi027 conmon[32322]: cluster 2021-11-03T17:14:30.159811+0000 2021-11-03T17:14:31.511 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:31 smithi027 conmon[32322]: mon.smithi027 (mon.0) 605 : cluster [DBG] Standby manager daemon smithi027.ufhhvb started 2021-11-03T17:14:31.511 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:31 smithi027 conmon[32322]: audit 2021-11-03T17 2021-11-03T17:14:31.511 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:31 smithi027 conmon[32322]: :14:30.162431+0000 mon.smithi027 (mon.0 2021-11-03T17:14:31.512 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:31 smithi027 conmon[32322]: ) 606 : audit [DBG] from='mgr.? 172.21.15.27:0/833939901' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi027.ufhhvb/crt"}]: dispatch 2021-11-03T17:14:31.512 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:31 smithi027 conmon[32322]: audit 2021-11-03T17:14:31.512 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:31 smithi027 conmon[32322]: 2021-11-03T17:14:30.163834+0000 2021-11-03T17:14:31.512 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:31 smithi027 conmon[32322]: mon.smithi027 (mon.0) 607 : audit [DBG] from='mgr.? 172.21.15.27:0/833939901' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-11-03T17:14:31.512 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:31 smithi027 conmon[32322]: audit 2021-11 2021-11-03T17:14:31.513 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:31 smithi027 conmon[32322]: -03T17:14:30.166377+0000 mon.smithi027 (mon.0) 608 2021-11-03T17:14:31.513 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:31 smithi027 conmon[32322]: : audit [DBG] from='mgr.? 172.21.15.27:0/833939901' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi027.ufhhvb/key"}]: dispatch 2021-11-03T17:14:31.513 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:31 smithi027 conmon[32322]: audit 2021- 2021-11-03T17:14:31.513 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:31 smithi027 conmon[32322]: 11-03T17:14:30.166770+0000 mon.smithi027 2021-11-03T17:14:31.513 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:31 smithi027 conmon[32322]: (mon.0) 609 : audit [DBG] from='mgr.? 172.21.15.27:0/833939901' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-11-03T17:14:31.514 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:31 smithi027 conmon[32322]: audit 2021- 2021-11-03T17:14:31.514 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:31 smithi027 conmon[32322]: 11-03T17:14:30.229779+0000 mon.smithi027 (mon.0) 610 : 2021-11-03T17:14:31.514 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:31 smithi027 conmon[32322]: audit [INF] from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2021-11-03T17:14:31.560 INFO:tasks.workunit.client.0.smithi027.stdout:1/686: sync 2021-11-03T17:14:31.560 INFO:tasks.workunit.client.0.smithi027.stdout:1/687: read d6/d11/dac/d79/d37/de0/de4/fa0 [643093,113362] 0 2021-11-03T17:14:31.578 INFO:tasks.workunit.client.0.smithi027.stdout:1/688: dwrite d6/d11/dac/d79/d25/d28/dec/fd0 [0,4194304] 0 2021-11-03T17:14:31.584 INFO:tasks.workunit.client.0.smithi027.stdout:1/689: rename d6/d11/dac/d79/d25/d28/d5c/d81/dd5 to d6/d11/dac/d79/d25/d28/d5c/d7a/dee 0 2021-11-03T17:14:31.584 INFO:tasks.workunit.client.0.smithi027.stdout:1/690: mknod d6/d11/dac/d79/d37/de0/de4/cef 1 2021-11-03T17:14:31.584 INFO:tasks.workunit.client.0.smithi027.stdout:1/691: mknod d6/d11/dac/d79/d25/d28/dec/cf0 1 2021-11-03T17:14:32.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:32 smithi115 conmon[34670]: audit 2021-11-03T17:14:30.229977+0000 mon.smithi115 (mon.1) 2021-11-03T17:14:32.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:32 smithi115 conmon[34670]: 45 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2021-11-03T17:14:32.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:32 smithi115 conmon[34670]: audit 2021-11-03T17:14:30.230152+0000 mon.smithi115 (mon.1) 46 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2021-11-03T17:14:32.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:32 smithi115 conmon[34670]: audit 2021-11-03T17:14:32.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:32 smithi115 conmon[34670]: 2021-11-03T17:14:31.175542+0000 mon.smithi027 (mon.0) 611 : audit [INF] from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2021-11-03T17:14:32.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:32 smithi115 conmon[34670]: audit 2021-11 2021-11-03T17:14:32.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:32 smithi115 conmon[34670]: -03T17:14:31.179624+0000 mon.smithi115 ( 2021-11-03T17:14:32.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:32 smithi115 conmon[34670]: mon.1) 47 : audit [INF] from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2021-11-03T17:14:32.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:32 smithi115 conmon[34670]: cluster 2021- 2021-11-03T17:14:32.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:32 smithi115 conmon[34670]: 11-03T17:14:31.187717+0000 2021-11-03T17:14:32.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:32 smithi115 conmon[34670]: mgr.smithi115.tcyodz (mgr.24395) 13 : cluster [DBG] 2021-11-03T17:14:32.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:32 smithi115 conmon[34670]: pgmap v9: 65 pgs: 65 active+clean; 2.7 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 19 MiB/s rd, 41 MiB/s wr, 112 op/s 2021-11-03T17:14:32.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:32 smithi115 conmon[34670]: audit 2021-11-03T17:14: 2021-11-03T17:14:32.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:32 smithi115 conmon[34670]: 31.682926+0000 mon.smithi115 (mon.1) 48 : 2021-11-03T17:14:32.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:32 smithi115 conmon[34670]: audit [INF] from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2021-11-03T17:14:32.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:32 smithi115 conmon[34670]: audit 2021-11-03 2021-11-03T17:14:32.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:32 smithi115 conmon[34670]: T17:14:31.683806+0000 mon.smithi115 (mon 2021-11-03T17:14:32.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:32 smithi115 conmon[34670]: .1) 49 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2021-11-03T17:14:32.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:32 smithi115 conmon[34670]: audit 2021- 2021-11-03T17:14:32.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:32 smithi115 conmon[34670]: 11-03T17:14:31.683939+0000 2021-11-03T17:14:32.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:32 smithi115 conmon[34670]: mon.smithi115 (mon.1) 50 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2021-11-03T17:14:32.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:32 smithi115 conmon[34670]: 2021-11-03T17:14:32.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:32 smithi115 conmon[34670]: cluster 2021-11-03T17:14:32.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:32 smithi115 conmon[34670]: 2021-11-03T17:14:31.688995+0000 mon.smithi027 2021-11-03T17:14:32.490 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:32 smithi115 conmon[34670]: (mon.0) 612 : cluster [DBG] mgrmap e23: smithi115.tcyodz(active, since 8s), standbys: smithi027.ufhhvb 2021-11-03T17:14:32.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:32 smithi027 conmon[32322]: audit 2021-11-03T17:14:30.229977+0000 mon.smithi115 2021-11-03T17:14:32.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:32 smithi027 conmon[32322]: (mon.1) 45 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2021-11-03T17:14:32.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:32 smithi027 conmon[32322]: audit 2021-11-03T17:14:30.230152+0000 mon.smithi115 (mon.1) 46 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2021-11-03T17:14:32.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:32 smithi027 conmon[32322]: 2021-11-03T17:14:32.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:32 smithi027 conmon[32322]: audit 2021-11-03T17:14:31.175542+0000 mon.smithi027 (mon. 2021-11-03T17:14:32.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:32 smithi027 conmon[32322]: 0) 611 : audit [INF] from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2021-11-03T17:14:32.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:32 smithi027 conmon[32322]: audit 2021-11-03T17:14:31.179624+0000 mon.smithi115 2021-11-03T17:14:32.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:32 smithi027 conmon[32322]: (mon.1) 47 : audit [INF] from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2021-11-03T17:14:32.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:32 smithi027 conmon[32322]: cluster 2021-11-03 2021-11-03T17:14:32.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:32 smithi027 conmon[32322]: T17:14:31.187717+0000 mgr.smithi115.tcyodz (mgr.24395) 13 : 2021-11-03T17:14:32.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:32 smithi027 conmon[32322]: cluster [DBG] pgmap v9: 65 pgs: 65 active+clean; 2.7 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 19 MiB/s rd, 41 MiB/s wr, 112 op/s 2021-11-03T17:14:32.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:32 smithi027 conmon[32322]: audit 2021 2021-11-03T17:14:32.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:32 smithi027 conmon[32322]: -11-03T17:14:31.682926+0000 mon.smithi115 (mon.1) 48 : audit [INF] 2021-11-03T17:14:32.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:32 smithi027 conmon[32322]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2021-11-03T17:14:32.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:32 smithi027 conmon[32322]: audit 2021-11-03 2021-11-03T17:14:32.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:32 smithi027 conmon[32322]: T17:14:31.683806+0000 mon.smithi115 (mon.1) 49 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2021-11-03T17:14:32.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:32 smithi027 conmon[32322]: audit 2021-11-03T17: 2021-11-03T17:14:32.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:32 smithi027 conmon[32322]: 14:31.683939+0000 mon.smithi115 (mon.1) 50 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2021-11-03T17:14:32.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:32 smithi027 conmon[32322]: cluster 2021-11-03T17:14 2021-11-03T17:14:32.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:32 smithi027 conmon[32322]: :31.688995+0000 mon.smithi027 (mon.0) 612 : cluster [DBG] mgrmap e23: smithi115.tcyodz(active, since 8s), standbys: smithi027.ufhhvb 2021-11-03T17:14:32.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:32 smithi027 conmon[32322]: 2021-11-03T17:14:34.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:34 smithi027 conmon[32322]: cluster 2021-11-03T17:14:33.188083+0000 mgr.smithi115.tcyodz (mgr. 2021-11-03T17:14:34.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:34 smithi027 conmon[32322]: 24395) 14 : cluster [DBG] pgmap v10: 65 pgs: 65 active+clean; 2.7 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 15 MiB/s rd, 32 MiB/s wr, 87 op/s 2021-11-03T17:14:34.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:34 smithi027 conmon[32322]: cephadm 2021-11-03T17:14:33.212466+0000 mgr.smithi115.tcyodz (mgr.24395) 15 : cephadm [INF] poll took 3708.276 ms: 1 events 2021-11-03T17:14:34.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:34 smithi027 conmon[32322]: audit 2021-11-03T17:14:33.241136+0000 mon.smithi027 (mon.0) 613 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' 2021-11-03T17:14:34.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:34 smithi027 conmon[32322]: audit 2021-11-03T17:14:33.889687+0000 mon.smithi027 (mon.0) 614 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' 2021-11-03T17:14:34.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:34 smithi027 conmon[32322]: audit 2021-11-03T17:14:33.892310+0000 mon.smithi115 (mon.1) 51 : audit [INF] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"}]: dispatch 2021-11-03T17:14:34.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:34 smithi027 conmon[32322]: audit 2021-11-03T17:14:33.892838+0000 mon.smithi027 (mon.0) 615 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"}]: dispatch 2021-11-03T17:14:34.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:34 smithi027 conmon[32322]: audit 2021-11-03T17:14:33.897471+0000 mon.smithi027 (mon.0) 616 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' 2021-11-03T17:14:34.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:34 smithi027 conmon[32322]: audit 2021-11-03T 2021-11-03T17:14:34.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:34 smithi027 conmon[32322]: 17:14:33.902454+0000 mon.smithi027 (mon.0) 617 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' 2021-11-03T17:14:34.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:34 smithi027 conmon[32322]: audit 2021-11-03T17:14:33.911730+0000 mon.smithi115 (mon.1) 52 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:14:34.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:34 smithi115 conmon[34670]: cluster 2021-11-03T17:14:33.188083+0000 mgr.smithi115.tcyodz (mgr.24395) 14 : 2021-11-03T17:14:34.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:34 smithi115 conmon[34670]: cluster [DBG] pgmap v10: 65 pgs: 65 active+clean; 2.7 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 15 MiB/s rd, 32 MiB/s wr, 87 op/s 2021-11-03T17:14:34.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:34 smithi115 conmon[34670]: cephadm 2021-11-03T17:14:33.212466+0000 mgr.smithi115.tcyodz (mgr.24395) 15 : cephadm [INF] poll took 3708.276 ms: 1 events 2021-11-03T17:14:34.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:34 smithi115 conmon[34670]: audit 2021-11 2021-11-03T17:14:34.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:34 smithi115 conmon[34670]: -03T17:14:33.241136+0000 mon.smithi027 (mon.0) 613 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' 2021-11-03T17:14:34.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:34 smithi115 conmon[34670]: audit 2021-11-03T17:14:33.889687 2021-11-03T17:14:34.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:34 smithi115 conmon[34670]: +0000 mon.smithi027 (mon.0) 614 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' 2021-11-03T17:14:34.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:34 smithi115 conmon[34670]: audit 2021-11-03T17:14:34.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:34 smithi115 conmon[34670]: 2021-11-03T17:14:33.892310+0000 mon.smithi115 (mon.1) 51 : audit [INF] 2021-11-03T17:14:34.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:34 smithi115 conmon[34670]: from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"}]: dispatch 2021-11-03T17:14:34.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:34 smithi115 conmon[34670]: audit 2021-11-03 2021-11-03T17:14:34.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:34 smithi115 conmon[34670]: T17:14:33.892838+0000 mon.smithi027 (mon.0) 615 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"}]: dispatch 2021-11-03T17:14:34.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:34 smithi115 conmon[34670]: audit 2021-11-03T17:14:33.897471+0000 mon.smithi027 2021-11-03T17:14:34.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:34 smithi115 conmon[34670]: (mon.0) 616 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' 2021-11-03T17:14:34.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:34 smithi115 conmon[34670]: audit 2021-11-03T17:14:33.902454+0000 mon.smithi027 (mon 2021-11-03T17:14:34.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:34 smithi115 conmon[34670]: .0) 617 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' 2021-11-03T17:14:34.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:34 smithi115 conmon[34670]: audit 2021-11-03T17:14:33.911730+0000 mon.smithi115 2021-11-03T17:14:34.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:34 smithi115 conmon[34670]: (mon.1) 52 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:14:34.817 INFO:tasks.workunit.client.0.smithi027.stdout:1/692: sync 2021-11-03T17:14:34.817 INFO:tasks.workunit.client.0.smithi027.stdout:2/639: sync 2021-11-03T17:14:34.818 INFO:tasks.workunit.client.0.smithi027.stdout:9/679: sync 2021-11-03T17:14:34.818 INFO:tasks.workunit.client.0.smithi027.stdout:0/812: sync 2021-11-03T17:14:34.818 INFO:tasks.workunit.client.0.smithi027.stdout:5/818: sync 2021-11-03T17:14:34.818 INFO:tasks.workunit.client.0.smithi027.stdout:4/513: sync 2021-11-03T17:14:34.818 INFO:tasks.workunit.client.0.smithi027.stdout:7/500: sync 2021-11-03T17:14:34.819 INFO:tasks.workunit.client.0.smithi027.stdout:8/943: sync 2021-11-03T17:14:34.819 INFO:tasks.workunit.client.0.smithi027.stdout:2/640: chown d0/d1b/d29/d54/fb4 124 1 2021-11-03T17:14:34.819 INFO:tasks.workunit.client.0.smithi027.stdout:0/813: fsync d9/da/f9d 0 2021-11-03T17:14:34.819 INFO:tasks.workunit.client.0.smithi027.stdout:2/641: chown d0/d1b/d29/d54 0 1 2021-11-03T17:14:34.820 INFO:tasks.workunit.client.0.smithi027.stdout:7/501: truncate d0/d2/d5e/d36/f64 2054028 0 2021-11-03T17:14:34.820 INFO:tasks.workunit.client.0.smithi027.stdout:3/707: sync 2021-11-03T17:14:34.820 INFO:tasks.workunit.client.0.smithi027.stdout:6/581: sync 2021-11-03T17:14:34.821 INFO:tasks.workunit.client.0.smithi027.stdout:1/693: getdents d6 0 2021-11-03T17:14:34.821 INFO:tasks.workunit.client.0.smithi027.stdout:1/694: readlink d6/lea 0 2021-11-03T17:14:34.822 INFO:tasks.workunit.client.0.smithi027.stdout:8/944: write d7/d2d/d38/dc3/fdc [3426854,58959] 0 2021-11-03T17:14:34.822 INFO:tasks.workunit.client.0.smithi027.stdout:4/514: link d6/d9/d16/d36/d70/d3e/d58/f61 d6/d8c/d9f/fa8 0 2021-11-03T17:14:34.822 INFO:tasks.workunit.client.0.smithi027.stdout:4/515: write d6/d9/d16/d36/d70/d3e/f6b [319775,36831] 0 2021-11-03T17:14:34.822 INFO:tasks.workunit.client.0.smithi027.stdout:4/516: readlink d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/l5e 0 2021-11-03T17:14:34.823 INFO:tasks.workunit.client.0.smithi027.stdout:4/517: read d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d2b/f3f [2742909,100654] 0 2021-11-03T17:14:34.823 INFO:tasks.workunit.client.0.smithi027.stdout:9/680: creat db/d1b/d2b/d41/d6c/ff7 x:0 0 0 2021-11-03T17:14:34.824 INFO:tasks.workunit.client.0.smithi027.stdout:9/681: fsync db/f5b 0 2021-11-03T17:14:34.825 INFO:tasks.workunit.client.0.smithi027.stdout:5/819: symlink d2/d4/l10a 0 2021-11-03T17:14:34.825 INFO:tasks.workunit.client.0.smithi027.stdout:2/642: dread d0/d1b/f6e [0,4194304] 0 2021-11-03T17:14:34.826 INFO:tasks.workunit.client.0.smithi027.stdout:5/820: write d2/d12/f9a [559751,12947] 0 2021-11-03T17:14:34.826 INFO:tasks.workunit.client.0.smithi027.stdout:5/821: stat d2/d2c/d9b/db3 0 2021-11-03T17:14:34.827 INFO:tasks.workunit.client.0.smithi027.stdout:0/814: link d9/da/f38 d9/da/d30/da7/f114 0 2021-11-03T17:14:34.827 INFO:tasks.workunit.client.0.smithi027.stdout:0/815: fsync d9/fb4 0 2021-11-03T17:14:34.828 INFO:tasks.workunit.client.0.smithi027.stdout:3/708: dread d1/d40/f8f [0,4194304] 0 2021-11-03T17:14:34.828 INFO:tasks.workunit.client.0.smithi027.stdout:0/816: dread d9/fbf [0,4194304] 0 2021-11-03T17:14:34.829 INFO:tasks.workunit.client.0.smithi027.stdout:7/502: creat d0/d2/d10/d1e/d20/faa x:0 0 0 2021-11-03T17:14:34.830 INFO:tasks.workunit.client.0.smithi027.stdout:6/582: readlink dc/d16/d1b/d3c/d51/d5f/l63 0 2021-11-03T17:14:34.832 INFO:tasks.workunit.client.0.smithi027.stdout:1/695: mkdir d6/de/df1 0 2021-11-03T17:14:34.836 INFO:tasks.workunit.client.0.smithi027.stdout:8/945: mkdir d7/d10/d3f/d9b/d95/d107/d13d 0 2021-11-03T17:14:34.839 INFO:tasks.workunit.client.0.smithi027.stdout:4/518: creat d6/d9/d28/d40/d81/fa9 x:0 0 0 2021-11-03T17:14:34.839 INFO:tasks.workunit.client.0.smithi027.stdout:8/946: dread d7/d10/d3f/d9b/f3e [0,4194304] 0 2021-11-03T17:14:34.844 INFO:tasks.workunit.client.0.smithi027.stdout:9/682: rename db/d1b/d2b/d41/ldd to db/d1b/d2b/d9c/lf8 0 2021-11-03T17:14:34.869 INFO:tasks.workunit.client.0.smithi027.stdout:0/817: rmdir d9/d12/d8f 39 2021-11-03T17:14:34.869 INFO:tasks.workunit.client.0.smithi027.stdout:6/583: truncate dc/d16/d1b/d24/f46 4319932 0 2021-11-03T17:14:34.869 INFO:tasks.workunit.client.0.smithi027.stdout:6/584: chown dc/d16/d1b/d24/f46 106040120 1 2021-11-03T17:14:34.870 INFO:tasks.workunit.client.0.smithi027.stdout:2/643: mkdir d0/d1b/db6 0 2021-11-03T17:14:34.870 INFO:tasks.workunit.client.0.smithi027.stdout:1/696: link d6/d11/dac/d79/d25/d28/dec/fe9 d6/d11/de2/ff2 0 2021-11-03T17:14:34.870 INFO:tasks.workunit.client.0.smithi027.stdout:2/644: truncate d0/d1b/f27 1015517 0 2021-11-03T17:14:34.870 INFO:tasks.workunit.client.0.smithi027.stdout:2/645: chown d0/d1b/d29/d54/d67 205457 1 2021-11-03T17:14:34.870 INFO:tasks.workunit.client.0.smithi027.stdout:2/646: chown d0/f91 28744819 1 2021-11-03T17:14:34.871 INFO:tasks.workunit.client.0.smithi027.stdout:2/647: getdents d0/da7 0 2021-11-03T17:14:34.871 INFO:tasks.workunit.client.0.smithi027.stdout:5/822: rename d2/d2c/l8b to d2/d2c/d9b/de1/l10b 0 2021-11-03T17:14:34.871 INFO:tasks.workunit.client.0.smithi027.stdout:5/823: chown d2/d4/d4e/d67/d8e/lfa 8 1 2021-11-03T17:14:34.871 INFO:tasks.workunit.client.0.smithi027.stdout:5/824: fsync d2/d1f/fe9 0 2021-11-03T17:14:34.872 INFO:tasks.workunit.client.0.smithi027.stdout:3/709: dwrite d1/db/f60 [0,4194304] 0 2021-11-03T17:14:34.872 INFO:tasks.workunit.client.0.smithi027.stdout:6/585: creat dc/d82/da9/faa x:0 0 0 2021-11-03T17:14:34.873 INFO:tasks.workunit.client.0.smithi027.stdout:6/586: readlink dc/d16/l31 0 2021-11-03T17:14:34.873 INFO:tasks.workunit.client.0.smithi027.stdout:6/587: fsync dc/d16/d1b/d3c/d51/f58 0 2021-11-03T17:14:34.873 INFO:tasks.workunit.client.0.smithi027.stdout:6/588: chown dc/d16/d1b/d2d/l41 180019 1 2021-11-03T17:14:34.873 INFO:tasks.workunit.client.0.smithi027.stdout:6/589: fdatasync dc/d16/f2c 0 2021-11-03T17:14:34.874 INFO:tasks.workunit.client.0.smithi027.stdout:6/590: mknod dc/d16/d62/cab 1 2021-11-03T17:14:34.874 INFO:tasks.workunit.client.0.smithi027.stdout:1/697: creat d6/d11/dac/d79/d25/d5e/dd1/ff3 x:0 0 0 2021-11-03T17:14:34.874 INFO:tasks.workunit.client.0.smithi027.stdout:1/698: fdatasync d6/de/faf 0 2021-11-03T17:14:34.875 INFO:tasks.workunit.client.0.smithi027.stdout:8/947: dwrite d7/d10/d3f/d55/ff2 [0,4194304] 0 2021-11-03T17:14:34.875 INFO:tasks.workunit.client.0.smithi027.stdout:8/948: chown d7/d10/d3f/d55/db7/dc9/d117/lf6 206 1 2021-11-03T17:14:34.882 INFO:tasks.workunit.client.0.smithi027.stdout:2/648: unlink d0/d1b/d29/d54/d64/daa/l22 0 2021-11-03T17:14:34.882 INFO:tasks.workunit.client.0.smithi027.stdout:2/649: write d0/d45/f9a [255425,81261] 0 2021-11-03T17:14:34.883 INFO:tasks.workunit.client.0.smithi027.stdout:4/519: dwrite d6/d9/d16/f19 [0,4194304] 0 2021-11-03T17:14:34.887 INFO:tasks.workunit.client.0.smithi027.stdout:0/818: rename d9/f95 to d9/da/d64/dcc/f115 0 2021-11-03T17:14:34.898 INFO:tasks.workunit.client.0.smithi027.stdout:1/699: rmdir d6/d11/dac/d79/d25/d5e 39 2021-11-03T17:14:34.899 INFO:tasks.workunit.client.0.smithi027.stdout:1/700: chown d6/d11/dac/d79/d37/de0/de4/d58/fcd 13 1 2021-11-03T17:14:34.901 INFO:tasks.workunit.client.0.smithi027.stdout:5/825: dwrite d2/d2c/d3c/d58/f100 [0,4194304] 0 2021-11-03T17:14:34.902 INFO:tasks.workunit.client.0.smithi027.stdout:4/520: creat d6/d9/d16/d36/d97/faa x:0 0 0 2021-11-03T17:14:34.902 INFO:tasks.workunit.client.0.smithi027.stdout:4/521: mknod d6/d9/d16/d36/d70/d3e/d58/d84/d9b/cab 1 2021-11-03T17:14:34.903 INFO:tasks.workunit.client.0.smithi027.stdout:4/522: rename d6/d9/d16/d36/d70 to d6/d9/d16/d36/d70/dac 22 2021-11-03T17:14:34.904 INFO:tasks.workunit.client.0.smithi027.stdout:0/819: symlink d9/da/d30/d92/dca/l116 0 2021-11-03T17:14:34.904 INFO:tasks.workunit.client.0.smithi027.stdout:1/701: getdents d6/d11/dac/d79/d37 0 2021-11-03T17:14:34.908 INFO:tasks.workunit.client.0.smithi027.stdout:5/826: truncate d2/d2c/d3c/d48/d65/f56 1516496 0 2021-11-03T17:14:34.908 INFO:tasks.workunit.client.0.smithi027.stdout:5/827: mknod d2/d2c/d3c/d48/d65/d80/d9e/c10c 1 2021-11-03T17:14:34.908 INFO:tasks.workunit.client.0.smithi027.stdout:5/828: chown d2/d12/f9a 726 1 2021-11-03T17:14:34.908 INFO:tasks.workunit.client.0.smithi027.stdout:5/829: chown d2/d12 94 1 2021-11-03T17:14:34.909 INFO:tasks.workunit.client.0.smithi027.stdout:5/830: chown d2/d1f/f7c 8 1 2021-11-03T17:14:34.909 INFO:tasks.workunit.client.0.smithi027.stdout:4/523: mkdir d6/d9/d8f/dad 0 2021-11-03T17:14:34.910 INFO:tasks.workunit.client.0.smithi027.stdout:4/524: write d6/d9/d16/d36/d70/d3e/d58/f60 [373816,108428] 0 2021-11-03T17:14:34.910 INFO:tasks.workunit.client.0.smithi027.stdout:0/820: mkdir d9/da/d30/d92/d110/d117 0 2021-11-03T17:14:34.912 INFO:tasks.workunit.client.0.smithi027.stdout:1/702: rmdir d6/d11/dac/d79/d25/d28/d41 39 2021-11-03T17:14:34.915 INFO:tasks.workunit.client.0.smithi027.stdout:3/710: dwrite d1/db/d52/d57/d99/da7/fb1 [0,4194304] 0 2021-11-03T17:14:34.916 INFO:tasks.workunit.client.0.smithi027.stdout:4/525: symlink d6/d24/d6d/lae 0 2021-11-03T17:14:34.916 INFO:tasks.workunit.client.0.smithi027.stdout:4/526: fdatasync d6/d9/d16/f7f 0 2021-11-03T17:14:34.917 INFO:tasks.workunit.client.0.smithi027.stdout:4/527: stat d6/d9/d28/d40/d81 0 2021-11-03T17:14:34.917 INFO:tasks.workunit.client.0.smithi027.stdout:6/591: dwrite dc/d82/d57/f60 [0,4194304] 0 2021-11-03T17:14:34.917 INFO:tasks.workunit.client.0.smithi027.stdout:6/592: dread - dc/d16/d1b/d2d/d34/d89/f9d zero size 2021-11-03T17:14:34.917 INFO:tasks.workunit.client.0.smithi027.stdout:6/593: chown dc/d16/d1b/la6 69498427 1 2021-11-03T17:14:34.918 INFO:tasks.workunit.client.0.smithi027.stdout:8/949: dwrite d7/d10/d3f/d9b/fee [0,4194304] 0 2021-11-03T17:14:34.918 INFO:tasks.workunit.client.0.smithi027.stdout:0/821: creat d9/d12/d8f/dc7/f118 x:0 0 0 2021-11-03T17:14:34.920 INFO:tasks.workunit.client.0.smithi027.stdout:1/703: unlink d6/d11/dac/d79/d25/d28/d5c/d7a/lad 0 2021-11-03T17:14:34.921 INFO:tasks.workunit.client.0.smithi027.stdout:2/650: dwrite d0/d7c/f9f [0,4194304] 0 2021-11-03T17:14:34.922 INFO:tasks.workunit.client.0.smithi027.stdout:5/831: getdents d2/d2c 0 2021-11-03T17:14:34.922 INFO:tasks.workunit.client.0.smithi027.stdout:6/594: dread dc/f18 [0,4194304] 0 2021-11-03T17:14:34.928 INFO:tasks.workunit.client.0.smithi027.stdout:4/528: creat d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d9a/faf x:0 0 0 2021-11-03T17:14:34.929 INFO:tasks.workunit.client.0.smithi027.stdout:3/711: dread d1/db/d52/d57/d8b/fa4 [0,4194304] 0 2021-11-03T17:14:34.930 INFO:tasks.workunit.client.0.smithi027.stdout:8/950: symlink d7/l13e 0 2021-11-03T17:14:34.933 INFO:tasks.workunit.client.0.smithi027.stdout:0/822: rmdir d9/da/d30/dac/dba/ddc 39 2021-11-03T17:14:34.949 INFO:tasks.workunit.client.0.smithi027.stdout:2/651: rename d0/d1b/d29/d58/d5b/lb3 to d0/d45/lb7 0 2021-11-03T17:14:34.949 INFO:tasks.workunit.client.0.smithi027.stdout:6/595: rmdir dc 39 2021-11-03T17:14:34.950 INFO:tasks.workunit.client.0.smithi027.stdout:6/596: mknod dc/d16/d9a/cac 1 2021-11-03T17:14:34.950 INFO:tasks.workunit.client.0.smithi027.stdout:1/704: dwrite d6/d11/dac/d79/d25/d28/d32/dd7/f6c [4194304,4194304] 0 2021-11-03T17:14:34.951 INFO:tasks.workunit.client.0.smithi027.stdout:4/529: dwrite d6/f48 [0,4194304] 0 2021-11-03T17:14:34.952 INFO:tasks.workunit.client.0.smithi027.stdout:3/712: rmdir d1/db/d52/d57/d8b 39 2021-11-03T17:14:34.953 INFO:tasks.workunit.client.0.smithi027.stdout:0/823: creat d9/da/d30/d92/f119 x:0 0 0 2021-11-03T17:14:34.956 INFO:tasks.workunit.client.0.smithi027.stdout:5/832: link d2/d2c/d3c/d48/laf d2/d2c/d3c/d48/d65/d80/l10d 0 2021-11-03T17:14:34.957 INFO:tasks.workunit.client.0.smithi027.stdout:2/652: creat d0/d45/fb8 x:0 0 0 2021-11-03T17:14:34.958 INFO:tasks.workunit.client.0.smithi027.stdout:2/653: mknod d0/d1b/d29/d54/d67/d6a/cb9 1 2021-11-03T17:14:34.960 INFO:tasks.workunit.client.0.smithi027.stdout:8/951: dwrite d7/d10/dbd/f120 [0,4194304] 0 2021-11-03T17:14:34.961 INFO:tasks.workunit.client.0.smithi027.stdout:6/597: mkdir dc/d82/d57/dad 0 2021-11-03T17:14:34.961 INFO:tasks.workunit.client.0.smithi027.stdout:6/598: chown dc/d16/d1b/d3c/d51/d5f 4093 1 2021-11-03T17:14:34.962 INFO:tasks.workunit.client.0.smithi027.stdout:1/705: dread d6/d11/dac/d79/f99 [0,4194304] 0 2021-11-03T17:14:34.965 INFO:tasks.workunit.client.0.smithi027.stdout:4/530: rename d6/d9/d28/d40 to d6/d24/db0 0 2021-11-03T17:14:34.972 INFO:tasks.workunit.client.0.smithi027.stdout:3/713: truncate d1/f13 2178019 0 2021-11-03T17:14:34.973 INFO:tasks.workunit.client.0.smithi027.stdout:5/833: dwrite d2/f3 [0,4194304] 0 2021-11-03T17:14:34.973 INFO:tasks.workunit.client.0.smithi027.stdout:0/824: getdents d9/da/d64/d70/d5f 0 2021-11-03T17:14:34.974 INFO:tasks.workunit.client.0.smithi027.stdout:8/952: dread d7/d111/d137/f85 [0,4194304] 0 2021-11-03T17:14:34.974 INFO:tasks.workunit.client.0.smithi027.stdout:8/953: truncate d7/f133 713126 0 2021-11-03T17:14:34.974 INFO:tasks.workunit.client.0.smithi027.stdout:3/714: dread d1/f4e [0,4194304] 0 2021-11-03T17:14:34.975 INFO:tasks.workunit.client.0.smithi027.stdout:3/715: mknod d1/db/db2/cdd 1 2021-11-03T17:14:34.976 INFO:tasks.workunit.client.0.smithi027.stdout:2/654: truncate d0/d5/f53 1552826 0 2021-11-03T17:14:34.976 INFO:tasks.workunit.client.0.smithi027.stdout:3/716: read d1/d40/f44 [3574666,95283] 0 2021-11-03T17:14:34.976 INFO:tasks.workunit.client.0.smithi027.stdout:3/717: write d1/db/f93 [1045728,24832] 0 2021-11-03T17:14:34.977 INFO:tasks.workunit.client.0.smithi027.stdout:3/718: write d1/d40/fb6 [1838115,17773] 0 2021-11-03T17:14:34.978 INFO:tasks.workunit.client.0.smithi027.stdout:6/599: mkdir dc/d82/d57/dad/dae 0 2021-11-03T17:14:34.978 INFO:tasks.workunit.client.0.smithi027.stdout:1/706: rmdir d6/d11/dac/d79/d37/de0/de4 39 2021-11-03T17:14:34.983 INFO:tasks.workunit.client.0.smithi027.stdout:4/531: mkdir d6/d24/d6d/db1 0 2021-11-03T17:14:34.988 INFO:tasks.workunit.client.0.smithi027.stdout:5/834: rename d2/d4/d4e/d67/d20/l5a to d2/d2c/d3c/d48/d65/d80/dd2/l10e 0 2021-11-03T17:14:34.990 INFO:tasks.workunit.client.0.smithi027.stdout:0/825: mkdir d9/da/d64/d70/d6d/d9f/d11a 0 2021-11-03T17:14:34.990 INFO:tasks.workunit.client.0.smithi027.stdout:0/826: chown d9/fb4 57214 1 2021-11-03T17:14:34.991 INFO:tasks.workunit.client.0.smithi027.stdout:0/827: fdatasync d9/da/d64/d70/d5f/f74 0 2021-11-03T17:14:34.992 INFO:tasks.workunit.client.0.smithi027.stdout:8/954: creat d7/d10/d3f/f13f x:0 0 0 2021-11-03T17:14:34.994 INFO:tasks.workunit.client.0.smithi027.stdout:3/719: creat d1/db/d52/d57/d99/da7/fde x:0 0 0 2021-11-03T17:14:34.995 INFO:tasks.workunit.client.0.smithi027.stdout:5/835: dread d2/d12/f9a [0,4194304] 0 2021-11-03T17:14:34.995 INFO:tasks.workunit.client.0.smithi027.stdout:1/707: dwrite f2 [0,4194304] 0 2021-11-03T17:14:34.996 INFO:tasks.workunit.client.0.smithi027.stdout:1/708: chown d6/d11/d14/f76 243 1 2021-11-03T17:14:34.997 INFO:tasks.workunit.client.0.smithi027.stdout:6/600: stat dc/d16/d1b/d3c/l6b 0 2021-11-03T17:14:34.997 INFO:tasks.workunit.client.0.smithi027.stdout:6/601: readlink dc/d16/d1b/l20 0 2021-11-03T17:14:35.002 INFO:tasks.workunit.client.0.smithi027.stdout:3/720: dread d1/fa [0,4194304] 0 2021-11-03T17:14:35.003 INFO:tasks.workunit.client.0.smithi027.stdout:3/721: stat d1/f5e 0 2021-11-03T17:14:35.003 INFO:tasks.workunit.client.0.smithi027.stdout:3/722: write d1/db/d52/d6a/d7c/fab [46295,55557] 0 2021-11-03T17:14:35.007 INFO:tasks.workunit.client.0.smithi027.stdout:2/655: rename d0/d1b/d29/d54/d67/d6a/la4 to d0/d1b/d29/db5/lba 0 2021-11-03T17:14:35.007 INFO:tasks.workunit.client.0.smithi027.stdout:1/709: rmdir d6/d11/dac/d79/db0 39 2021-11-03T17:14:35.008 INFO:tasks.workunit.client.0.smithi027.stdout:0/828: dwrite d9/da/d30/dac/dba/ddc/fe1 [0,4194304] 0 2021-11-03T17:14:35.008 INFO:tasks.workunit.client.0.smithi027.stdout:0/829: readlink d9/l21 0 2021-11-03T17:14:35.013 INFO:tasks.workunit.client.0.smithi027.stdout:4/532: write d6/d24/db0/f53 [196140,62450] 0 2021-11-03T17:14:35.013 INFO:tasks.workunit.client.0.smithi027.stdout:4/533: mknod d6/d24/cb2 1 2021-11-03T17:14:35.026 INFO:tasks.workunit.client.0.smithi027.stdout:8/955: rename d7/d10/d3f/d55/db7/dc9/d103/da5 to d7/dcc/dfe/d140 0 2021-11-03T17:14:35.027 INFO:tasks.workunit.client.0.smithi027.stdout:1/710: truncate d6/fa 2016392 0 2021-11-03T17:14:35.032 INFO:tasks.workunit.client.0.smithi027.stdout:2/656: dwrite d0/d1b/d29/d58/f5c [0,4194304] 0 2021-11-03T17:14:35.033 INFO:tasks.workunit.client.0.smithi027.stdout:6/602: dwrite dc/d16/d1b/d3c/f9e [0,4194304] 0 2021-11-03T17:14:35.037 INFO:tasks.workunit.client.0.smithi027.stdout:0/830: rmdir d9/da/d64/d70/d5f 39 2021-11-03T17:14:35.038 INFO:tasks.workunit.client.0.smithi027.stdout:3/723: truncate d1/db/f93 7072997 0 2021-11-03T17:14:35.039 INFO:tasks.workunit.client.0.smithi027.stdout:4/534: creat d6/d9/d16/d36/d70/d3e/d58/d84/d9b/fb3 x:0 0 0 2021-11-03T17:14:35.042 INFO:tasks.workunit.client.0.smithi027.stdout:5/836: rename d2/d2c/d9b/d9d/lc6 to d2/d2c/d3c/d48/d65/dd6/l10f 0 2021-11-03T17:14:35.045 INFO:tasks.workunit.client.0.smithi027.stdout:1/711: creat d6/d11/dac/d79/d37/de0/de4/d58/ff4 x:0 0 0 2021-11-03T17:14:35.047 INFO:tasks.workunit.client.0.smithi027.stdout:2/657: getdents d0/d1b/d29/d54/d67 0 2021-11-03T17:14:35.052 INFO:tasks.workunit.client.0.smithi027.stdout:0/831: truncate d9/da/f16 799885 0 2021-11-03T17:14:35.054 INFO:tasks.workunit.client.0.smithi027.stdout:3/724: rmdir d1/db/d52/d6a 39 2021-11-03T17:14:35.055 INFO:tasks.workunit.client.0.smithi027.stdout:6/603: dwrite dc/f64 [0,4194304] 0 2021-11-03T17:14:35.056 INFO:tasks.workunit.client.0.smithi027.stdout:4/535: truncate d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/f38 1278124 0 2021-11-03T17:14:35.056 INFO:tasks.workunit.client.0.smithi027.stdout:4/536: truncate d6/d9/d16/d36/d70/d3e/d58/f61 2276877 0 2021-11-03T17:14:35.056 INFO:tasks.workunit.client.0.smithi027.stdout:4/537: fsync d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d41/f77 0 2021-11-03T17:14:35.057 INFO:tasks.workunit.client.0.smithi027.stdout:4/538: fsync d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/f7e 0 2021-11-03T17:14:35.057 INFO:tasks.workunit.client.0.smithi027.stdout:4/539: mknod d6/d9/d8f/cb4 1 2021-11-03T17:14:35.059 INFO:tasks.workunit.client.0.smithi027.stdout:8/956: rename d7/d10/d3f/d55/db7/dc9/d117/d7f/f129 to d7/d10/d3f/d9b/d95/d107/d13d/f141 0 2021-11-03T17:14:35.060 INFO:tasks.workunit.client.0.smithi027.stdout:1/712: creat d6/d11/dac/d79/d25/d28/d5c/ff5 x:0 0 0 2021-11-03T17:14:35.061 INFO:tasks.workunit.client.0.smithi027.stdout:1/713: chown d6/d11/dac/d79/d25/d28/d5c/d81/f86 1301941855 1 2021-11-03T17:14:35.062 INFO:tasks.workunit.client.0.smithi027.stdout:2/658: creat d0/d1b/d29/d54/fbb x:0 0 0 2021-11-03T17:14:35.065 INFO:tasks.workunit.client.0.smithi027.stdout:8/957: dread d7/d10/f12 [0,4194304] 0 2021-11-03T17:14:35.066 INFO:tasks.workunit.client.0.smithi027.stdout:8/958: stat d7/d10/d3f/d55/db7/f118 0 2021-11-03T17:14:35.066 INFO:tasks.workunit.client.0.smithi027.stdout:4/540: dread d6/d9/d16/d17/f6e [0,4194304] 0 2021-11-03T17:14:35.069 INFO:tasks.workunit.client.0.smithi027.stdout:0/832: rmdir d9/da/d30/d93 39 2021-11-03T17:14:35.069 INFO:tasks.workunit.client.0.smithi027.stdout:0/833: chown d9/da/d30/fd8 20814 1 2021-11-03T17:14:35.070 INFO:tasks.workunit.client.0.smithi027.stdout:8/959: dread d7/d10/d3f/d55/db7/dc9/d117/fda [0,4194304] 0 2021-11-03T17:14:35.071 INFO:tasks.workunit.client.0.smithi027.stdout:3/725: creat d1/db/d52/d6a/d7c/d88/d98/fdf x:0 0 0 2021-11-03T17:14:35.073 INFO:tasks.workunit.client.0.smithi027.stdout:5/837: dwrite d2/d12/dad/d6a/f106 [0,4194304] 0 2021-11-03T17:14:35.074 INFO:tasks.workunit.client.0.smithi027.stdout:6/604: dread dc/f10 [0,4194304] 0 2021-11-03T17:14:35.074 INFO:tasks.workunit.client.0.smithi027.stdout:6/605: stat dc/d16/d1b/d2d/l86 0 2021-11-03T17:14:35.074 INFO:tasks.workunit.client.0.smithi027.stdout:6/606: mknod dc/d16/caf 1 2021-11-03T17:14:35.077 INFO:tasks.workunit.client.0.smithi027.stdout:1/714: mkdir d6/d11/d14/df6 0 2021-11-03T17:14:35.079 INFO:tasks.workunit.client.0.smithi027.stdout:2/659: truncate d0/d7c/f9f 460140 0 2021-11-03T17:14:35.080 INFO:tasks.workunit.client.0.smithi027.stdout:6/607: write dc/d16/d1b/d3c/f9e [436781,94313] 0 2021-11-03T17:14:35.080 INFO:tasks.workunit.client.0.smithi027.stdout:6/608: fdatasync dc/d16/d30/f38 0 2021-11-03T17:14:35.086 INFO:tasks.workunit.client.0.smithi027.stdout:0/834: getdents d9/da 0 2021-11-03T17:14:35.224 INFO:tasks.workunit.client.0.smithi027.stdout:4/541: dwrite d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/f93 [0,4194304] 0 2021-11-03T17:14:35.224 INFO:tasks.workunit.client.0.smithi027.stdout:4/542: mknod d6/d9/d16/d36/d97/cb5 1 2021-11-03T17:14:35.224 INFO:tasks.workunit.client.0.smithi027.stdout:4/543: mknod d6/d9/d16/d36/d99/cb6 1 2021-11-03T17:14:35.224 INFO:tasks.workunit.client.0.smithi027.stdout:4/544: fdatasync d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/f8a 0 2021-11-03T17:14:35.225 INFO:tasks.workunit.client.0.smithi027.stdout:4/545: chown d6/d9/d16/f19 7856186 1 2021-11-03T17:14:35.225 INFO:tasks.workunit.client.0.smithi027.stdout:3/726: creat d1/db/d52/d57/d8b/fe0 x:0 0 0 2021-11-03T17:14:35.225 INFO:tasks.workunit.client.0.smithi027.stdout:3/727: mknod d1/db/d52/d6a/d7c/d88/ce1 1 2021-11-03T17:14:35.225 INFO:tasks.workunit.client.0.smithi027.stdout:3/728: fsync d1/db/d52/f5d 0 2021-11-03T17:14:35.226 INFO:tasks.workunit.client.0.smithi027.stdout:8/960: dwrite d7/d10/d3f/d9b/f3c [0,4194304] 0 2021-11-03T17:14:35.226 INFO:tasks.workunit.client.0.smithi027.stdout:8/961: chown d7/d10/d3f/d55/db7/dc9/d117/d7f/dd9 646042 1 2021-11-03T17:14:35.226 INFO:tasks.workunit.client.0.smithi027.stdout:5/838: link d2/d2c/d3c/l54 d2/d4/d4e/d67/l110 0 2021-11-03T17:14:35.226 INFO:tasks.workunit.client.0.smithi027.stdout:5/839: fsync d2/d12/d7d/f101 0 2021-11-03T17:14:35.226 INFO:tasks.workunit.client.0.smithi027.stdout:5/840: stat d2/d12/f9a 0 2021-11-03T17:14:35.227 INFO:tasks.workunit.client.0.smithi027.stdout:2/660: unlink d0/d1b/d87/f8c 0 2021-11-03T17:14:35.227 INFO:tasks.workunit.client.0.smithi027.stdout:6/609: mkdir dc/d82/d57/d9b/db0 0 2021-11-03T17:14:35.227 INFO:tasks.workunit.client.0.smithi027.stdout:0/835: symlink d9/d12/def/l11b 0 2021-11-03T17:14:35.227 INFO:tasks.workunit.client.0.smithi027.stdout:5/841: write d2/d2c/d3c/d58/f100 [1901879,6421] 0 2021-11-03T17:14:35.228 INFO:tasks.workunit.client.0.smithi027.stdout:6/610: dread dc/d16/d1b/d3c/d51/f70 [0,4194304] 0 2021-11-03T17:14:35.228 INFO:tasks.workunit.client.0.smithi027.stdout:3/729: creat d1/db/d52/d6a/db4/db5/dad/fe2 x:0 0 0 2021-11-03T17:14:35.228 INFO:tasks.workunit.client.0.smithi027.stdout:5/842: link d2/d2c/d3c/d48/d65/d80/d9e/fbd d2/d12/dad/da8/f111 0 2021-11-03T17:14:35.228 INFO:tasks.workunit.client.0.smithi027.stdout:1/715: dwrite d6/d11/dac/d79/fbb [0,4194304] 0 2021-11-03T17:14:35.228 INFO:tasks.workunit.client.0.smithi027.stdout:1/716: mknod d6/d11/d14/df6/cf7 1 2021-11-03T17:14:35.229 INFO:tasks.workunit.client.0.smithi027.stdout:6/611: rename dc/d16/d1b/d2d to dc/d82/d57/dad/dae/db1 0 2021-11-03T17:14:35.229 INFO:tasks.workunit.client.0.smithi027.stdout:6/612: write dc/d16/d1b/f75 [377507,4882] 0 2021-11-03T17:14:35.229 INFO:tasks.workunit.client.0.smithi027.stdout:5/843: creat d2/d12/dad/da8/f112 x:0 0 0 2021-11-03T17:14:35.229 INFO:tasks.workunit.client.0.smithi027.stdout:6/613: mkdir dc/d16/db2 0 2021-11-03T17:14:35.230 INFO:tasks.workunit.client.0.smithi027.stdout:6/614: mkdir dc/d16/d1b/d24/d7e/db3 0 2021-11-03T17:14:35.230 INFO:tasks.workunit.client.0.smithi027.stdout:6/615: rename dc to dc/d82/d57/d9b/db0/db4 22 2021-11-03T17:14:35.230 INFO:tasks.workunit.client.0.smithi027.stdout:4/546: dwrite d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/f37 [0,4194304] 0 2021-11-03T17:14:35.230 INFO:tasks.workunit.client.0.smithi027.stdout:5/844: truncate d2/d12/dad/d6a/f106 72705 0 2021-11-03T17:14:35.230 INFO:tasks.workunit.client.0.smithi027.stdout:2/661: dread d0/d1b/d29/d54/f7f [0,4194304] 0 2021-11-03T17:14:35.231 INFO:tasks.workunit.client.0.smithi027.stdout:2/662: write d0/f10 [890,4951] 0 2021-11-03T17:14:35.231 INFO:tasks.workunit.client.0.smithi027.stdout:2/663: rename d0/d1b/d29/d58/d5b/l85 to d0/d45/lbc 0 2021-11-03T17:14:35.231 INFO:tasks.workunit.client.0.smithi027.stdout:6/616: write dc/f64 [1474455,71582] 0 2021-11-03T17:14:35.231 INFO:tasks.workunit.client.0.smithi027.stdout:6/617: mknod dc/d37/cb5 1 2021-11-03T17:14:35.232 INFO:tasks.workunit.client.0.smithi027.stdout:6/618: mknod dc/d16/d1b/cb6 1 2021-11-03T17:14:35.232 INFO:tasks.workunit.client.0.smithi027.stdout:5/845: dread d2/d4/f13 [0,4194304] 0 2021-11-03T17:14:35.232 INFO:tasks.workunit.client.0.smithi027.stdout:5/846: mknod d2/d4/d4e/d67/d20/c113 1 2021-11-03T17:14:35.232 INFO:tasks.workunit.client.0.smithi027.stdout:0/836: dwrite d9/d12/f28 [0,4194304] 0 2021-11-03T17:14:35.232 INFO:tasks.workunit.client.0.smithi027.stdout:8/962: dwrite d7/d53/d67/fc7 [0,4194304] 0 2021-11-03T17:14:35.233 INFO:tasks.workunit.client.0.smithi027.stdout:3/730: dwrite d1/db/f86 [0,4194304] 0 2021-11-03T17:14:35.233 INFO:tasks.workunit.client.0.smithi027.stdout:2/664: dwrite d0/d45/f9a [0,4194304] 0 2021-11-03T17:14:35.233 INFO:tasks.workunit.client.0.smithi027.stdout:5/847: dwrite d2/d12/dad/d6a/fc5 [0,4194304] 0 2021-11-03T17:14:35.233 INFO:tasks.workunit.client.0.smithi027.stdout:6/619: dwrite dc/d82/d57/dad/dae/db1/d34/d89/f9d [0,4194304] 0 2021-11-03T17:14:35.234 INFO:tasks.workunit.client.0.smithi027.stdout:6/620: chown dc/d16/l31 257235814 1 2021-11-03T17:14:35.234 INFO:tasks.workunit.client.0.smithi027.stdout:5/848: dread - d2/d12/d7d/da7/ff8 zero size 2021-11-03T17:14:35.234 INFO:tasks.workunit.client.0.smithi027.stdout:5/849: dread - d2/d1f/fbc zero size 2021-11-03T17:14:35.234 INFO:tasks.workunit.client.0.smithi027.stdout:6/621: truncate dc/d82/d57/dad/dae/db1/d34/f8a 1638686 0 2021-11-03T17:14:35.234 INFO:tasks.workunit.client.0.smithi027.stdout:6/622: truncate dc/d16/d1b/d3c/d51/f5b 144597 0 2021-11-03T17:14:35.235 INFO:tasks.workunit.client.0.smithi027.stdout:8/963: unlink d7/fc8 0 2021-11-03T17:14:35.235 INFO:tasks.workunit.client.0.smithi027.stdout:0/837: dread d9/da/d30/dac/dba/ddc/fe1 [0,4194304] 0 2021-11-03T17:14:35.235 INFO:tasks.workunit.client.0.smithi027.stdout:4/547: dwrite d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/f69 [0,4194304] 0 2021-11-03T17:14:35.235 INFO:tasks.workunit.client.0.smithi027.stdout:2/665: rename d0/d1b/d29/d58/d5b to d0/d1b/d29/d92/dbd 0 2021-11-03T17:14:35.235 INFO:tasks.workunit.client.0.smithi027.stdout:5/850: creat d2/d4/d4e/f114 x:0 0 0 2021-11-03T17:14:35.236 INFO:tasks.workunit.client.0.smithi027.stdout:5/851: truncate d2/f52 89344 0 2021-11-03T17:14:35.236 INFO:tasks.workunit.client.0.smithi027.stdout:8/964: creat d7/dff/f142 x:0 0 0 2021-11-03T17:14:35.236 INFO:tasks.workunit.client.0.smithi027.stdout:0/838: link d9/ffa d9/d12/def/f11c 0 2021-11-03T17:14:35.236 INFO:tasks.workunit.client.0.smithi027.stdout:0/839: chown d9/da/d64/d70/d6d/l82 7 1 2021-11-03T17:14:35.237 INFO:tasks.workunit.client.0.smithi027.stdout:4/548: creat d6/d24/db0/d81/fb7 x:0 0 0 2021-11-03T17:14:35.237 INFO:tasks.workunit.client.0.smithi027.stdout:4/549: fdatasync d6/d8c/d9f/fa8 0 2021-11-03T17:14:35.237 INFO:tasks.workunit.client.0.smithi027.stdout:6/623: dwrite dc/d16/d1b/d24/f46 [0,4194304] 0 2021-11-03T17:14:35.237 INFO:tasks.workunit.client.0.smithi027.stdout:3/731: dwrite d1/db/d52/d57/f71 [0,4194304] 0 2021-11-03T17:14:35.237 INFO:tasks.workunit.client.0.smithi027.stdout:3/732: write d1/db/d52/d6a/f7f [2214213,124871] 0 2021-11-03T17:14:35.238 INFO:tasks.workunit.client.0.smithi027.stdout:3/733: fsync d1/d40/f4a 0 2021-11-03T17:14:35.238 INFO:tasks.workunit.client.0.smithi027.stdout:2/666: creat d0/d1b/db6/fbe x:0 0 0 2021-11-03T17:14:35.238 INFO:tasks.workunit.client.0.smithi027.stdout:5/852: creat d2/d4/d4e/d67/d8e/d99/f115 x:0 0 0 2021-11-03T17:14:35.238 INFO:tasks.workunit.client.0.smithi027.stdout:8/965: unlink d7/l6d 0 2021-11-03T17:14:35.239 INFO:tasks.workunit.client.0.smithi027.stdout:8/966: mknod d7/d10/dbd/c143 1 2021-11-03T17:14:35.239 INFO:tasks.workunit.client.0.smithi027.stdout:8/967: chown d7/fb8 13 1 2021-11-03T17:14:35.239 INFO:tasks.workunit.client.0.smithi027.stdout:0/840: truncate d9/d12/f3b 720247 0 2021-11-03T17:14:35.239 INFO:tasks.workunit.client.0.smithi027.stdout:3/734: mkdir d1/db/d52/d57/de3 0 2021-11-03T17:14:35.239 INFO:tasks.workunit.client.0.smithi027.stdout:3/735: dread - d1/db/d52/d57/d8b/fe0 zero size 2021-11-03T17:14:35.240 INFO:tasks.workunit.client.0.smithi027.stdout:0/841: write d9/da/d30/f43 [1430952,78260] 0 2021-11-03T17:14:35.241 INFO:tasks.workunit.client.0.smithi027.stdout:0/842: symlink d9/da/d64/l11d 0 2021-11-03T17:14:35.241 INFO:tasks.workunit.client.0.smithi027.stdout:0/843: mkdir d9/da/d30/dac/dba/ddc/d11e 0 2021-11-03T17:14:35.241 INFO:tasks.workunit.client.0.smithi027.stdout:0/844: mkdir d9/da/d30/dfe/d11f 0 2021-11-03T17:14:35.241 INFO:tasks.workunit.client.0.smithi027.stdout:2/667: dread d0/d1b/d29/d54/d64/daa/d5d/f75 [0,4194304] 0 2021-11-03T17:14:35.242 INFO:tasks.workunit.client.0.smithi027.stdout:6/624: dwrite dc/d16/f7a [0,4194304] 0 2021-11-03T17:14:35.242 INFO:tasks.workunit.client.0.smithi027.stdout:5/853: dread d2/d2c/d3c/d58/f74 [0,4194304] 0 2021-11-03T17:14:35.242 INFO:tasks.workunit.client.0.smithi027.stdout:6/625: mknod dc/d82/d57/d9b/db0/cb7 1 2021-11-03T17:14:35.242 INFO:tasks.workunit.client.0.smithi027.stdout:5/854: truncate d2/d2c/d9b/faa 113370 0 2021-11-03T17:14:35.242 INFO:tasks.workunit.client.0.smithi027.stdout:3/736: dread d1/d40/f76 [0,4194304] 0 2021-11-03T17:14:35.243 INFO:tasks.workunit.client.0.smithi027.stdout:8/968: dwrite d7/d111/d137/d9d/fb4 [0,4194304] 0 2021-11-03T17:14:35.244 INFO:tasks.workunit.client.0.smithi027.stdout:2/668: dread d0/d1b/d29/d92/dbd/f62 [0,4194304] 0 2021-11-03T17:14:35.244 INFO:tasks.workunit.client.0.smithi027.stdout:2/669: chown d0/d1b/d29/f2b 13735 1 2021-11-03T17:14:35.244 INFO:tasks.workunit.client.0.smithi027.stdout:3/737: symlink d1/db/d52/d6a/db4/db5/le4 0 2021-11-03T17:14:35.244 INFO:tasks.workunit.client.0.smithi027.stdout:3/738: mknod d1/db/d52/d6a/db4/ce5 1 2021-11-03T17:14:35.244 INFO:tasks.workunit.client.0.smithi027.stdout:5/855: write d2/f3e [6919768,45486] 0 2021-11-03T17:14:35.245 INFO:tasks.workunit.client.0.smithi027.stdout:0/845: dread d9/da/d64/d70/d6f/fa1 [0,4194304] 0 2021-11-03T17:14:35.245 INFO:tasks.workunit.client.0.smithi027.stdout:8/969: write d7/d10/d3f/d9b/fd4 [2800665,93310] 0 2021-11-03T17:14:35.249 INFO:tasks.workunit.client.0.smithi027.stdout:2/670: link d0/d5/f4 d0/da7/fbf 0 2021-11-03T17:14:35.251 INFO:tasks.workunit.client.0.smithi027.stdout:3/739: creat d1/db/d52/d6a/db4/db5/dad/fe6 x:0 0 0 2021-11-03T17:14:35.256 INFO:tasks.workunit.client.0.smithi027.stdout:2/671: unlink d0/d1b/d29/d54/d64/daa/d5d/f75 0 2021-11-03T17:14:35.259 INFO:tasks.workunit.client.0.smithi027.stdout:6/626: write dc/d82/d57/dad/dae/db1/d34/f8a [148004,124165] 0 2021-11-03T17:14:35.260 INFO:tasks.workunit.client.0.smithi027.stdout:3/740: truncate d1/d40/f5c 1509418 0 2021-11-03T17:14:35.263 INFO:tasks.workunit.client.0.smithi027.stdout:2/672: dread d0/d1b/d1c/f33 [0,4194304] 0 2021-11-03T17:14:35.267 INFO:tasks.workunit.client.0.smithi027.stdout:6/627: creat dc/d16/d1b/d3c/d51/fb8 x:0 0 0 2021-11-03T17:14:35.267 INFO:tasks.workunit.client.0.smithi027.stdout:3/741: creat d1/db/d52/d6a/d7c/d88/d98/fe7 x:0 0 0 2021-11-03T17:14:35.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:35 smithi027 conmon[32322]: cluster 2021-11-03T17:14:33. 2021-11-03T17:14:35.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:35 smithi027 conmon[32322]: 894432+0000 mgr.smithi115.tcyodz (mgr.24395) 16 : cluster [DBG] pgmap v11: 65 pgs: 65 active+clean; 2.7 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 16 MiB/s rd, 33 MiB/s wr, 90 op/s 2021-11-03T17:14:35.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:35 smithi027 conmon[32322]: cephadm 2021-11-03T17:14:33.913061+0000 mgr.smithi115.tcyodz (mgr.24395) 17 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.smithi115.tcyodz) 2021-11-03T17:14:35.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:35 smithi027 conmon[32322]: cluster 2021-11-03T17:14:34.887964+0000 mon.smithi027 (mon.0) 618 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2021-11-03T17:14:35.270 INFO:tasks.workunit.client.0.smithi027.stdout:5/856: dwrite d2/d2c/d9b/db3/fee [0,4194304] 0 2021-11-03T17:14:35.270 INFO:tasks.workunit.client.0.smithi027.stdout:8/970: dwrite f6 [4194304,4194304] 0 2021-11-03T17:14:35.270 INFO:tasks.workunit.client.0.smithi027.stdout:6/628: creat dc/d82/d87/fb9 x:0 0 0 2021-11-03T17:14:35.270 INFO:tasks.workunit.client.0.smithi027.stdout:5/857: write d2/d4/d4e/d67/f11 [1933382,95478] 0 2021-11-03T17:14:35.279 INFO:tasks.workunit.client.0.smithi027.stdout:3/742: getdents d1/db/d52/d6a/d7c/d88/d98 0 2021-11-03T17:14:35.284 INFO:tasks.workunit.client.0.smithi027.stdout:8/971: rmdir d7/d10/d3f/d55/db7/dc9/d117/d7f/dd9 0 2021-11-03T17:14:35.284 INFO:tasks.workunit.client.0.smithi027.stdout:8/972: write d7/d10/dbd/fe1 [1089519,107320] 0 2021-11-03T17:14:35.284 INFO:tasks.workunit.client.0.smithi027.stdout:8/973: write d7/d10/fed [281474,38300] 0 2021-11-03T17:14:35.285 INFO:tasks.workunit.client.0.smithi027.stdout:2/673: dwrite d0/d1b/d29/d54/d64/daa/f30 [0,4194304] 0 2021-11-03T17:14:35.287 INFO:tasks.workunit.client.0.smithi027.stdout:3/743: dread d1/db/d52/d6a/d7c/f89 [0,4194304] 0 2021-11-03T17:14:35.287 INFO:tasks.workunit.client.0.smithi027.stdout:6/629: unlink dc/d16/d1b/d3c/d51/f73 0 2021-11-03T17:14:35.289 INFO:tasks.workunit.client.0.smithi027.stdout:5/858: mkdir d2/d2c/d9b/db3/d116 0 2021-11-03T17:14:35.290 INFO:tasks.workunit.client.0.smithi027.stdout:2/674: mknod d0/cc0 1 2021-11-03T17:14:35.290 INFO:tasks.workunit.client.0.smithi027.stdout:2/675: fsync d0/fa9 0 2021-11-03T17:14:35.291 INFO:tasks.workunit.client.0.smithi027.stdout:3/744: symlink d1/db/d52/d6a/d7c/d88/d98/le8 0 2021-11-03T17:14:35.293 INFO:tasks.workunit.client.0.smithi027.stdout:2/676: creat d0/d1b/d29/d54/fc1 x:0 0 0 2021-11-03T17:14:35.293 INFO:tasks.workunit.client.0.smithi027.stdout:2/677: write d0/d1b/d29/d92/dbd/f8b [114464,126098] 0 2021-11-03T17:14:35.293 INFO:tasks.workunit.client.0.smithi027.stdout:2/678: truncate d0/d1b/d29/d92/dbd/f98 464312 0 2021-11-03T17:14:35.295 INFO:tasks.workunit.client.0.smithi027.stdout:2/679: write d0/f91 [436921,48736] 0 2021-11-03T17:14:35.295 INFO:tasks.workunit.client.0.smithi027.stdout:2/680: chown d0/d45/lbc 105360504 1 2021-11-03T17:14:35.296 INFO:tasks.workunit.client.0.smithi027.stdout:3/745: unlink d1/db/d52/d57/lc3 0 2021-11-03T17:14:35.297 INFO:tasks.workunit.client.0.smithi027.stdout:2/681: symlink d0/d45/lc2 0 2021-11-03T17:14:35.297 INFO:tasks.workunit.client.0.smithi027.stdout:2/682: mknod d0/d1b/d29/d92/dbd/cc3 1 2021-11-03T17:14:35.299 INFO:tasks.workunit.client.0.smithi027.stdout:3/746: link d1/d40/f8f d1/db/d52/d57/d99/da7/dbf/fe9 0 2021-11-03T17:14:35.300 INFO:tasks.workunit.client.0.smithi027.stdout:3/747: getdents d1/db/db2/dbd 0 2021-11-03T17:14:35.300 INFO:tasks.workunit.client.0.smithi027.stdout:3/748: chown d1/db/d52/f64 46052 1 2021-11-03T17:14:35.301 INFO:tasks.workunit.client.0.smithi027.stdout:3/749: getdents d1/db/d52/d57/d99 0 2021-11-03T17:14:35.301 INFO:tasks.workunit.client.0.smithi027.stdout:3/750: mknod d1/db/d52/d6a/db4/db5/dad/dc6/cea 1 2021-11-03T17:14:35.305 INFO:tasks.workunit.client.0.smithi027.stdout:8/974: dwrite d7/d10/dbd/fd2 [4194304,4194304] 0 2021-11-03T17:14:35.305 INFO:tasks.workunit.client.0.smithi027.stdout:8/975: readlink d7/d53/laf 0 2021-11-03T17:14:35.306 INFO:tasks.workunit.client.0.smithi027.stdout:2/683: fdatasync d0/f91 0 2021-11-03T17:14:35.307 INFO:tasks.workunit.client.0.smithi027.stdout:2/684: getdents d0/d1b/d29/d54/d64/daa/d88 0 2021-11-03T17:14:35.307 INFO:tasks.workunit.client.0.smithi027.stdout:6/630: dread dc/f64 [0,4194304] 0 2021-11-03T17:14:35.307 INFO:tasks.workunit.client.0.smithi027.stdout:6/631: readlink dc/d16/d1b/l2e 0 2021-11-03T17:14:35.308 INFO:tasks.workunit.client.0.smithi027.stdout:8/976: creat d7/d10/d3f/d9b/dfc/f144 x:0 0 0 2021-11-03T17:14:35.309 INFO:tasks.workunit.client.0.smithi027.stdout:5/859: dwrite d2/d12/d7d/da7/ff8 [0,4194304] 0 2021-11-03T17:14:35.309 INFO:tasks.workunit.client.0.smithi027.stdout:5/860: write d2/d12/dad/da8/fc4 [166590,92593] 0 2021-11-03T17:14:35.310 INFO:tasks.workunit.client.0.smithi027.stdout:2/685: creat d0/d1b/d29/d54/d64/daa/fc4 x:0 0 0 2021-11-03T17:14:35.310 INFO:tasks.workunit.client.0.smithi027.stdout:2/686: fdatasync d0/d1b/d29/d54/d67/f6c 0 2021-11-03T17:14:35.312 INFO:tasks.workunit.client.0.smithi027.stdout:6/632: creat dc/d16/d9a/fba x:0 0 0 2021-11-03T17:14:35.312 INFO:tasks.workunit.client.0.smithi027.stdout:6/633: write dc/d16/f2c [3898530,125507] 0 2021-11-03T17:14:35.312 INFO:tasks.workunit.client.0.smithi027.stdout:6/634: fdatasync dc/d16/d1b/d3c/d51/d5f/d79/f93 0 2021-11-03T17:14:35.313 INFO:tasks.workunit.client.0.smithi027.stdout:3/751: dread d1/db/d52/d57/d99/da7/fb1 [0,4194304] 0 2021-11-03T17:14:35.315 INFO:tasks.workunit.client.0.smithi027.stdout:3/752: write d1/d40/f8a [523538,104376] 0 2021-11-03T17:14:35.324 INFO:tasks.workunit.client.0.smithi027.stdout:8/977: unlink d7/d10/dbd/f120 0 2021-11-03T17:14:35.324 INFO:tasks.workunit.client.0.smithi027.stdout:8/978: truncate d7/d111/d137/f101 285234 0 2021-11-03T17:14:35.326 INFO:tasks.workunit.client.0.smithi027.stdout:5/861: link d2/d12/dad/l64 d2/d4/d4e/d67/d8e/l117 0 2021-11-03T17:14:35.326 INFO:tasks.workunit.client.0.smithi027.stdout:5/862: chown d2/d4/d61 3 1 2021-11-03T17:14:35.328 INFO:tasks.workunit.client.0.smithi027.stdout:5/863: dread d2/d4/d4e/d67/f2e [0,4194304] 0 2021-11-03T17:14:35.328 INFO:tasks.workunit.client.0.smithi027.stdout:5/864: chown d2/d4/d4e/d67/d20 1433 1 2021-11-03T17:14:35.328 INFO:tasks.workunit.client.0.smithi027.stdout:5/865: fdatasync d2/d1f/fbc 0 2021-11-03T17:14:35.328 INFO:tasks.workunit.client.0.smithi027.stdout:5/866: fdatasync d2/d2c/d3c/f84 0 2021-11-03T17:14:35.329 INFO:tasks.workunit.client.0.smithi027.stdout:2/687: getdents d0/d1b/d29/d92/dbd 0 2021-11-03T17:14:35.330 INFO:tasks.workunit.client.0.smithi027.stdout:6/635: link dc/f7b dc/d82/d57/dad/dae/fbb 0 2021-11-03T17:14:35.331 INFO:tasks.workunit.client.0.smithi027.stdout:6/636: read dc/d16/d1b/f75 [107995,75135] 0 2021-11-03T17:14:35.331 INFO:tasks.workunit.client.0.smithi027.stdout:3/753: symlink d1/db/d52/d57/d99/leb 0 2021-11-03T17:14:35.332 INFO:tasks.workunit.client.0.smithi027.stdout:8/979: rmdir d7/d10 39 2021-11-03T17:14:35.333 INFO:tasks.workunit.client.0.smithi027.stdout:5/867: creat d2/d4/d4e/d67/f118 x:0 0 0 2021-11-03T17:14:35.335 INFO:tasks.workunit.client.0.smithi027.stdout:2/688: creat d0/d1b/d29/fc5 x:0 0 0 2021-11-03T17:14:35.337 INFO:tasks.workunit.client.0.smithi027.stdout:3/754: mkdir d1/dec 0 2021-11-03T17:14:35.341 INFO:tasks.workunit.client.0.smithi027.stdout:8/980: rename d7/dcc/dfe/d11f to d7/d10/d3f/d55/db7/dc9/d103/df0/d145 0 2021-11-03T17:14:35.341 INFO:tasks.workunit.client.0.smithi027.stdout:8/981: mknod d7/d10/c146 1 2021-11-03T17:14:35.342 INFO:tasks.workunit.client.0.smithi027.stdout:8/982: creat d7/d111/f147 x:0 0 0 2021-11-03T17:14:35.342 INFO:tasks.workunit.client.0.smithi027.stdout:6/637: dread dc/d82/f81 [0,4194304] 0 2021-11-03T17:14:35.364 INFO:tasks.workunit.client.0.smithi027.stdout:8/983: dread d7/f87 [0,4194304] 0 2021-11-03T17:14:35.366 INFO:tasks.workunit.client.0.smithi027.stdout:5/868: dwrite d2/d12/dad/da8/f112 [0,4194304] 0 2021-11-03T17:14:35.366 INFO:tasks.workunit.client.0.smithi027.stdout:8/984: write d7/d10/d3f/d55/db7/dc9/d117/f8c [2542970,100214] 0 2021-11-03T17:14:35.367 INFO:tasks.workunit.client.0.smithi027.stdout:2/689: dwrite d0/d5/f83 [0,4194304] 0 2021-11-03T17:14:35.367 INFO:tasks.workunit.client.0.smithi027.stdout:3/755: dwrite d1/f1d [0,4194304] 0 2021-11-03T17:14:35.368 INFO:tasks.workunit.client.0.smithi027.stdout:3/756: readlink d1/db/d52/d57/d8b/l8c 0 2021-11-03T17:14:35.369 INFO:tasks.workunit.client.0.smithi027.stdout:5/869: creat d2/d2c/d9b/de1/f119 x:0 0 0 2021-11-03T17:14:35.369 INFO:tasks.workunit.client.0.smithi027.stdout:5/870: read - d2/d2c/d3c/dc8/fe7 zero size 2021-11-03T17:14:35.402 INFO:tasks.workunit.client.0.smithi027.stdout:2/690: dwrite d0/d1b/d1c/f26 [0,4194304] 0 2021-11-03T17:14:35.403 INFO:tasks.workunit.client.0.smithi027.stdout:2/691: fsync d0/d45/f7b 0 2021-11-03T17:14:35.403 INFO:tasks.workunit.client.0.smithi027.stdout:2/692: fsync d0/d1b/d1c/f33 0 2021-11-03T17:14:35.403 INFO:tasks.workunit.client.0.smithi027.stdout:2/693: stat d0/d45/f9a 0 2021-11-03T17:14:35.404 INFO:tasks.workunit.client.0.smithi027.stdout:3/757: dwrite d1/db/fd4 [0,4194304] 0 2021-11-03T17:14:35.404 INFO:tasks.workunit.client.0.smithi027.stdout:5/871: dwrite d2/d2c/d3c/d58/fa0 [4194304,4194304] 0 2021-11-03T17:14:35.415 INFO:tasks.workunit.client.0.smithi027.stdout:2/694: dread d0/da7/fbf [4194304,4194304] 0 2021-11-03T17:14:35.425 INFO:tasks.workunit.client.0.smithi027.stdout:8/985: creat d7/d10/d3f/d55/db7/dc9/f148 x:0 0 0 2021-11-03T17:14:35.428 INFO:tasks.workunit.client.0.smithi027.stdout:3/758: dwrite d1/db/d52/d6a/fa2 [4194304,4194304] 0 2021-11-03T17:14:35.428 INFO:tasks.workunit.client.0.smithi027.stdout:3/759: fdatasync d1/d40/f5c 0 2021-11-03T17:14:35.429 INFO:tasks.workunit.client.0.smithi027.stdout:8/986: dread d7/d10/d3f/d9b/d95/fd5 [0,4194304] 0 2021-11-03T17:14:35.430 INFO:tasks.workunit.client.0.smithi027.stdout:2/695: dwrite d0/d1b/d29/f2f [0,4194304] 0 2021-11-03T17:14:35.439 INFO:tasks.workunit.client.0.smithi027.stdout:5/872: rename d2/d2c/d9b/db3/fee to d2/d4/d61/f11a 0 2021-11-03T17:14:35.440 INFO:tasks.workunit.client.0.smithi027.stdout:3/760: dread d1/d40/f8f [0,4194304] 0 2021-11-03T17:14:35.444 INFO:tasks.workunit.client.0.smithi027.stdout:8/987: creat d7/d2d/da8/f149 x:0 0 0 2021-11-03T17:14:35.445 INFO:tasks.workunit.client.0.smithi027.stdout:8/988: readlink d7/d53/d67/lba 0 2021-11-03T17:14:35.445 INFO:tasks.workunit.client.0.smithi027.stdout:8/989: fsync d7/d10/d3f/d9b/fd4 0 2021-11-03T17:14:35.446 INFO:tasks.workunit.client.0.smithi027.stdout:2/696: unlink d0/d1b/d1c/dad/lb0 0 2021-11-03T17:14:35.449 INFO:tasks.workunit.client.0.smithi027.stdout:3/761: dread d1/db/d52/f64 [0,4194304] 0 2021-11-03T17:14:35.459 INFO:tasks.workunit.client.0.smithi027.stdout:5/873: link d2/d12/l3b d2/d12/dad/da8/dd9/dde/l11b 0 2021-11-03T17:14:35.461 INFO:tasks.workunit.client.0.smithi027.stdout:5/874: write d2/d12/dad/d6a/fc5 [224550,37782] 0 2021-11-03T17:14:35.462 INFO:tasks.workunit.client.0.smithi027.stdout:5/875: chown d2/d4/d4e/d67 18086869 1 2021-11-03T17:14:35.462 INFO:tasks.workunit.client.0.smithi027.stdout:5/876: mknod d2/d12/dad/da8/c11c 1 2021-11-03T17:14:35.462 INFO:tasks.workunit.client.0.smithi027.stdout:5/877: chown d2/d2c/d9b/db3/lf2 1396 1 2021-11-03T17:14:35.463 INFO:tasks.workunit.client.0.smithi027.stdout:8/990: dwrite d7/d10/d3f/d55/db7/dc9/d117/d7f/f10d [0,4194304] 0 2021-11-03T17:14:35.463 INFO:tasks.workunit.client.0.smithi027.stdout:2/697: creat d0/d1b/d29/d54/d64/daa/d88/fc6 x:0 0 0 2021-11-03T17:14:35.464 INFO:tasks.workunit.client.0.smithi027.stdout:2/698: truncate d0/f9b 322476 0 2021-11-03T17:14:35.464 INFO:tasks.workunit.client.0.smithi027.stdout:5/878: dread d2/d2c/d3c/fba [0,4194304] 0 2021-11-03T17:14:35.474 INFO:tasks.workunit.client.0.smithi027.stdout:8/991: rename d7/d10/d3f/f78 to d7/d10/d3f/d55/db7/dc9/f14a 0 2021-11-03T17:14:35.474 INFO:tasks.workunit.client.0.smithi027.stdout:8/992: truncate d7/d10/d3f/d55/db7/dc9/d117/f45 1787370 0 2021-11-03T17:14:35.477 INFO:tasks.workunit.client.0.smithi027.stdout:5/879: getdents d2/d2c/d3c/d48/d65/d80 0 2021-11-03T17:14:35.478 INFO:tasks.workunit.client.0.smithi027.stdout:2/699: dread d0/d5/f53 [0,4194304] 0 2021-11-03T17:14:35.479 INFO:tasks.workunit.client.0.smithi027.stdout:8/993: symlink d7/d10/d3f/d55/db7/dc9/d117/d41/l14b 0 2021-11-03T17:14:35.480 INFO:tasks.workunit.client.0.smithi027.stdout:5/880: mkdir d2/d4/d4e/d11d 0 2021-11-03T17:14:35.482 INFO:tasks.workunit.client.0.smithi027.stdout:5/881: symlink d2/d2c/d9b/l11e 0 2021-11-03T17:14:35.488 INFO:tasks.workunit.client.0.smithi027.stdout:3/762: dwrite d1/db/d52/db9/fd8 [0,4194304] 0 2021-11-03T17:14:35.493 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:35 smithi115 conmon[34670]: cluster 2021-11-03T17:14:33.894432+0000 mgr.smithi115.tcyodz 2021-11-03T17:14:35.494 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:35 smithi115 conmon[34670]: (mgr.24395) 16 : cluster [DBG] pgmap v11: 65 pgs: 65 active+clean; 2.7 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 16 MiB/s rd, 33 MiB/s wr, 90 op/s 2021-11-03T17:14:35.494 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:35 smithi115 conmon[34670]: cephadm 2021-11-03T17:14:33.913061+0000 mgr.smithi115.tcyodz (mgr.24395) 17 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.smithi115.tcyodz) 2021-11-03T17:14:35.495 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:35 smithi115 conmon[34670]: cluster 2021-11-03T17:14:34.887964+0000 2021-11-03T17:14:35.495 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:35 smithi115 conmon[34670]: mon.smithi027 (mon.0) 618 : cluster [WRN] 2021-11-03T17:14:35.495 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:35 smithi115 conmon[34670]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2021-11-03T17:14:35.497 INFO:tasks.workunit.client.0.smithi027.stdout:3/763: write d1/f4c [3050381,122988] 0 2021-11-03T17:14:35.508 INFO:tasks.workunit.client.0.smithi027.stdout:8/994: dwrite d7/d53/f102 [0,4194304] 0 2021-11-03T17:14:35.508 INFO:tasks.workunit.client.0.smithi027.stdout:8/995: mknod d7/d10/d3f/d9b/d11c/c14c 1 2021-11-03T17:14:35.510 INFO:tasks.workunit.client.0.smithi027.stdout:5/882: dwrite d2/d2c/d3c/d48/d65/f8f [0,4194304] 0 2021-11-03T17:14:35.511 INFO:tasks.workunit.client.0.smithi027.stdout:5/883: creat d2/d2c/d3c/dc8/f11f x:0 0 0 2021-11-03T17:14:35.512 INFO:tasks.workunit.client.0.smithi027.stdout:5/884: mkdir d2/d2c/d3c/d48/d65/d80/dd2/d120 0 2021-11-03T17:14:35.516 INFO:tasks.workunit.client.0.smithi027.stdout:8/996: dread d7/d10/d3f/ffa [0,4194304] 0 2021-11-03T17:14:35.516 INFO:tasks.workunit.client.0.smithi027.stdout:8/997: readlink d7/d53/l5d 0 2021-11-03T17:14:35.517 INFO:tasks.workunit.client.0.smithi027.stdout:8/998: chown d7/d111/d11d/l132 39 1 2021-11-03T17:14:35.518 INFO:tasks.workunit.client.0.smithi027.stdout:3/764: dread d1/d40/f44 [0,4194304] 0 2021-11-03T17:14:35.519 INFO:tasks.workunit.client.0.smithi027.stdout:8/999: write d7/d10/fa1 [1566615,63505] 0 2021-11-03T17:14:35.519 INFO:tasks.workunit.client.0.smithi027.stdout:3/765: symlink d1/db/db2/led 0 2021-11-03T17:14:35.519 INFO:tasks.workunit.client.0.smithi027.stdout:3/766: truncate d1/db/d52/d6a/d7c/d88/d98/fbc 275924 0 2021-11-03T17:14:35.519 INFO:tasks.workunit.client.0.smithi027.stdout:3/767: fdatasync d1/db/f1c 0 2021-11-03T17:14:35.520 INFO:tasks.workunit.client.0.smithi027.stdout:3/768: mknod d1/db/d52/d6a/db4/db5/dad/cee 1 2021-11-03T17:14:35.520 INFO:tasks.workunit.client.0.smithi027.stdout:3/769: link d1/fb8 d1/db/d52/d6a/d7c/fef 0 2021-11-03T17:14:35.524 INFO:tasks.workunit.client.0.smithi027.stdout:3/770: rename d1/db/d52/d57/d8b/fe0 to d1/db/d52/d6a/dd5/ff0 0 2021-11-03T17:14:35.524 INFO:tasks.workunit.client.0.smithi027.stdout:3/771: write d1/db/f7d [4892866,75443] 0 2021-11-03T17:14:35.525 INFO:tasks.workunit.client.0.smithi027.stdout:3/772: write d1/d40/f46 [4045007,13795] 0 2021-11-03T17:14:35.526 INFO:tasks.workunit.client.0.smithi027.stdout:3/773: symlink d1/db/d52/d57/de3/lf1 0 2021-11-03T17:14:35.528 INFO:tasks.workunit.client.0.smithi027.stdout:3/774: rename d1/db/l84 to d1/db/db2/lf2 0 2021-11-03T17:14:35.529 INFO:tasks.workunit.client.0.smithi027.stdout:3/775: truncate d1/db/d52/d6a/d7c/d88/fc8 2230610 0 2021-11-03T17:14:35.530 INFO:tasks.workunit.client.0.smithi027.stdout:3/776: write d1/d40/f87 [2627727,21592] 0 2021-11-03T17:14:35.532 INFO:tasks.workunit.client.0.smithi027.stdout:3/777: creat d1/db/db2/dbd/ff3 x:0 0 0 2021-11-03T17:14:35.533 INFO:tasks.workunit.client.0.smithi027.stdout:3/778: write d1/fb8 [5134427,49791] 0 2021-11-03T17:14:35.539 INFO:tasks.workunit.client.0.smithi027.stdout:3/779: dread d1/db/d52/d6a/fa2 [0,4194304] 0 2021-11-03T17:14:35.540 INFO:tasks.workunit.client.0.smithi027.stdout:3/780: write d1/fb8 [507020,124851] 0 2021-11-03T17:14:35.540 INFO:tasks.workunit.client.0.smithi027.stdout:3/781: chown d1/f50 28864 1 2021-11-03T17:14:35.542 INFO:tasks.workunit.client.0.smithi027.stdout:3/782: link d1/db/d52/d57/d99/da7/fb1 d1/db/d52/d57/de3/ff4 0 2021-11-03T17:14:35.547 INFO:tasks.workunit.client.0.smithi027.stdout:3/783: dread d1/d40/f8a [0,4194304] 0 2021-11-03T17:14:35.549 INFO:tasks.workunit.client.0.smithi027.stdout:3/784: symlink d1/db/d52/d6a/db4/db5/d97/lf5 0 2021-11-03T17:14:35.549 INFO:tasks.workunit.client.0.smithi027.stdout:3/785: write d1/db/fd4 [4906088,10108] 0 2021-11-03T17:14:35.550 INFO:tasks.workunit.client.0.smithi027.stdout:3/786: mkdir d1/db/d52/d57/d99/df6 0 2021-11-03T17:14:35.555 INFO:tasks.workunit.client.0.smithi027.stdout:3/787: dread d1/d40/f4a [0,4194304] 0 2021-11-03T17:14:35.557 INFO:tasks.workunit.client.0.smithi027.stdout:3/788: creat d1/db/d52/d57/d99/ff7 x:0 0 0 2021-11-03T17:14:35.574 INFO:tasks.workunit.client.0.smithi027.stdout:3/789: dwrite d1/d40/fb6 [0,4194304] 0 2021-11-03T17:14:35.579 INFO:tasks.workunit.client.0.smithi027.stdout:3/790: dread d1/db/f35 [0,4194304] 0 2021-11-03T17:14:35.580 INFO:tasks.workunit.client.0.smithi027.stdout:3/791: getdents d1/db/d52/d6a/d7c/d88 0 2021-11-03T17:14:35.583 INFO:tasks.workunit.client.0.smithi027.stdout:3/792: creat d1/db/ff8 x:0 0 0 2021-11-03T17:14:35.586 INFO:tasks.workunit.client.0.smithi027.stdout:3/793: creat d1/db/d52/d6a/db4/db5/dad/dc6/ff9 x:0 0 0 2021-11-03T17:14:35.595 INFO:tasks.workunit.client.0.smithi027.stdout:3/794: rename d1/d40/f6d to d1/db/db2/dbd/ffa 0 2021-11-03T17:14:35.596 INFO:tasks.workunit.client.0.smithi027.stdout:3/795: symlink d1/db/d52/d6a/db4/db5/dad/dc6/lfb 0 2021-11-03T17:14:35.603 INFO:tasks.workunit.client.0.smithi027.stdout:3/796: dread d1/db/d52/d6a/db4/db5/d97/f9e [0,4194304] 0 2021-11-03T17:14:35.603 INFO:tasks.workunit.client.0.smithi027.stdout:3/797: write d1/d40/f46 [4615639,109914] 0 2021-11-03T17:14:35.604 INFO:tasks.workunit.client.0.smithi027.stdout:3/798: write d1/f2a [4672038,96513] 0 2021-11-03T17:14:35.605 INFO:tasks.workunit.client.0.smithi027.stdout:3/799: rename d1/db/d52/d57/d8b/fa4 to d1/db/d52/d57/d99/df6/ffc 0 2021-11-03T17:14:35.607 INFO:tasks.workunit.client.0.smithi027.stdout:3/800: write d1/db/f5b [1157096,63211] 0 2021-11-03T17:14:35.607 INFO:tasks.workunit.client.0.smithi027.stdout:3/801: dread - d1/db/db2/dbd/ff3 zero size 2021-11-03T17:14:36.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:36 smithi115 conmon[34670]: cluster 2021-11-03T17:14:36.193458+0000 mon.smithi027 2021-11-03T17:14:36.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:36 smithi115 conmon[34670]: (mon.0) 619 : cluster [INF] daemon mds.cephfs.smithi115.zsfgnv restarted 2021-11-03T17:14:36.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:36 smithi115 conmon[34670]: cluster 2021-11-03T17:14:36.197748+0000 mon.smithi027 (mon.0) 620 : cluster [DBG] 2021-11-03T17:14:36.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:36 smithi115 conmon[34670]: osdmap e49: 6 total, 6 up, 6 in 2021-11-03T17:14:36.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:36 smithi027 conmon[32322]: cluster 2021-11-03T17:14:36.193458+0000 mon.smithi027 (mon.0) 619 : cluster [INF] daemon mds.cephfs.smithi115.zsfgnv restarted 2021-11-03T17:14:36.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:36 smithi027 conmon[32322]: cluster 2021-11-03T17:14:36.197748+0000 mon.smithi027 (mon.0) 620 : cluster [DBG] osdmap e49: 6 total, 6 up, 6 in 2021-11-03T17:14:37.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:37 smithi027 conmon[32322]: cluster 2021-11-03T17: 2021-11-03T17:14:37.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:37 smithi027 conmon[32322]: 14:35.894901+0000 mgr.smithi115.tcyodz (mgr.24395) 18 : cluster [DBG] pgmap v12: 65 pgs: 65 active+clean; 2.8 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 25 MiB/s rd, 45 MiB/s wr, 107 op/s 2021-11-03T17:14:37.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:37 smithi027 conmon[32322]: cluster 2021-11-03T17:14:36.246454+0000 mon.smithi027 (mon.0) 621 : cluster [WRN] Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2021-11-03T17:14:37.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:37 smithi027 conmon[32322]: cluster 2021-11-03T17:14:36.253713+0000 mon.smithi027 (mon.0) 622 : cluster [DBG] mds.? [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] up:boot 2021-11-03T17:14:37.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:37 smithi027 conmon[32322]: cluster 2021-11-03T 2021-11-03T17:14:37.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:37 smithi027 conmon[32322]: 17:14:36.253757+0000 mon.smithi027 (mon.0) 623 : cluster [INF] Standby daemon mds.cephfs.smithi027.fksixo assigned to filesystem cephfs as rank 0 2021-11-03T17:14:37.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:37 smithi027 conmon[32322]: cluster 2021-11-03 2021-11-03T17:14:37.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:37 smithi027 conmon[32322]: T17:14:36.253994+0000 mon.smithi027 (mon.0) 624 : cluster [DBG] 2021-11-03T17:14:37.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:37 smithi027 conmon[32322]: fsmap cephfs:1/2 {1=cephfs.smithi027.jdwohn=up:active} 3 up:standby, 1 failed 2021-11-03T17:14:37.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:37 smithi027 conmon[32322]: audit 2021-11-03 2021-11-03T17:14:37.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:37 smithi027 conmon[32322]: T17:14:36.256831+0000 mon.smithi115 (mon.1) 53 2021-11-03T17:14:37.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:37 smithi027 conmon[32322]: : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.zsfgnv"}]: dispatch 2021-11-03T17:14:37.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:37 smithi027 conmon[32322]: cluster 2021-11-03T17 2021-11-03T17:14:37.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:37 smithi027 conmon[32322]: :14:36.259062+0000 mon.smithi027 (mon 2021-11-03T17:14:37.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:37 smithi027 conmon[32322]: .0) 625 : cluster [DBG] 2021-11-03T17:14:37.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:37 smithi027 conmon[32322]: fsmap cephfs:2/2 {0=cephfs.smithi027.fksixo=up:replay,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:14:37.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:37 smithi027 conmon[32322]: audit 2021-11 2021-11-03T17:14:37.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:37 smithi027 conmon[32322]: -03T17:14:36.957495+0000 2021-11-03T17:14:37.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:37 smithi027 conmon[32322]: mon.smithi027 (mon.0) 626 : audit 2021-11-03T17:14:37.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:37 smithi027 conmon[32322]: [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' 2021-11-03T17:14:37.524 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:37 smithi027 conmon[32322]: audit 2021-11-03T17:14:37.524 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:37 smithi027 conmon[32322]: 2021-11-03T17:14 2021-11-03T17:14:37.524 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:37 smithi027 conmon[32322]: :36.959072+0000 mon.smithi115 ( 2021-11-03T17:14:37.524 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:37 smithi027 conmon[32322]: mon.1) 54 : audit 2021-11-03T17:14:37.524 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:37 smithi027 conmon[32322]: [INF] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi027.ufhhvb", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2021-11-03T17:14:37.525 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:37 smithi027 conmon[32322]: audit 2021 2021-11-03T17:14:37.525 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:37 smithi027 conmon[32322]: -11-03T17:14:36.959381+0000 mon.smithi027 2021-11-03T17:14:37.525 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:37 smithi027 conmon[32322]: (mon.0) 627 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi027.ufhhvb", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2021-11-03T17:14:37.525 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:37 smithi027 conmon[32322]: 2021-11-03T17:14:37.526 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:37 smithi027 conmon[32322]: audit 2021-11-03T17:14:37.526 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:37 smithi027 conmon[32322]: 2021-11-03T17:14:36.960257+0000 mon.smithi115 2021-11-03T17:14:37.526 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:37 smithi027 conmon[32322]: (mon.1) 55 : audit [DBG] 2021-11-03T17:14:37.526 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:37 smithi027 conmon[32322]: from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "mgr services"}]: dispatch 2021-11-03T17:14:37.526 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:37 smithi027 conmon[32322]: audit 2021-11- 2021-11-03T17:14:37.527 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:37 smithi027 conmon[32322]: 03T17:14:36.961057+0000 mon.smithi115 (mon. 2021-11-03T17:14:37.527 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:37 smithi027 conmon[32322]: 1) 56 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:14:37.543 INFO:tasks.workunit.client.0.smithi027.stdout:7/503: sync 2021-11-03T17:14:37.543 INFO:tasks.workunit.client.0.smithi027.stdout:7/504: chown d0/d2/d10/d35/l37 487794 1 2021-11-03T17:14:37.545 INFO:tasks.workunit.client.0.smithi027.stdout:7/505: mkdir d0/d2/d5e/d36/d3a/da7/dab 0 2021-11-03T17:14:37.546 INFO:tasks.workunit.client.0.smithi027.stdout:7/506: symlink d0/d2/d5e/d36/d3a/da7/lac 0 2021-11-03T17:14:37.548 INFO:tasks.workunit.client.0.smithi027.stdout:7/507: unlink d0/d2/d1c/d4d/l94 0 2021-11-03T17:14:37.549 INFO:tasks.workunit.client.0.smithi027.stdout:7/508: rmdir d0/d2/d10/d1e 39 2021-11-03T17:14:37.550 INFO:tasks.workunit.client.0.smithi027.stdout:7/509: write d0/d2/d10/d1e/d20/f26 [177621,127914] 0 2021-11-03T17:14:37.551 INFO:tasks.workunit.client.0.smithi027.stdout:7/510: readlink d0/d2/d1c/d4d/l5d 0 2021-11-03T17:14:37.551 INFO:tasks.workunit.client.0.smithi027.stdout:7/511: dread - d0/d2/d1f/f9e zero size 2021-11-03T17:14:37.552 INFO:tasks.workunit.client.0.smithi027.stdout:7/512: mkdir d0/d2/d1c/d4d/d6e/d84/dad 0 2021-11-03T17:14:37.572 INFO:tasks.workunit.client.0.smithi027.stdout:7/513: dwrite d0/d2/d10/d1e/d20/d74/f7b [0,4194304] 0 2021-11-03T17:14:37.575 INFO:tasks.workunit.client.0.smithi027.stdout:7/514: mkdir d0/d2d/d2f/dae 0 2021-11-03T17:14:37.585 INFO:tasks.workunit.client.0.smithi027.stdout:7/515: dread d0/f4c [0,4194304] 0 2021-11-03T17:14:37.586 INFO:tasks.workunit.client.0.smithi027.stdout:7/516: creat d0/d2/d1c/faf x:0 0 0 2021-11-03T17:14:37.601 INFO:tasks.workunit.client.0.smithi027.stdout:7/517: dwrite d0/f13 [4194304,4194304] 0 2021-11-03T17:14:37.610 INFO:tasks.workunit.client.0.smithi027.stdout:7/518: dread d0/d2d/d2f/f39 [0,4194304] 0 2021-11-03T17:14:37.611 INFO:tasks.workunit.client.0.smithi027.stdout:7/519: creat d0/d2/d5e/d36/fb0 x:0 0 0 2021-11-03T17:14:37.613 INFO:tasks.workunit.client.0.smithi027.stdout:7/520: read d0/d2/d10/d1e/d20/d74/f7b [3415345,103918] 0 2021-11-03T17:14:37.615 INFO:tasks.workunit.client.0.smithi027.stdout:7/521: unlink d0/d2/l9a 0 2021-11-03T17:14:37.616 INFO:tasks.workunit.client.0.smithi027.stdout:7/522: mknod d0/d2d/d2f/cb1 1 2021-11-03T17:14:37.618 INFO:tasks.workunit.client.0.smithi027.stdout:7/523: unlink d0/d2d/d58/f7f 0 2021-11-03T17:14:37.619 INFO:tasks.workunit.client.0.smithi027.stdout:7/524: fsync d0/d2d/d66/f70 0 2021-11-03T17:14:37.623 INFO:tasks.workunit.client.0.smithi027.stdout:7/525: dread d0/f4c [0,4194304] 0 2021-11-03T17:14:37.626 INFO:tasks.workunit.client.0.smithi027.stdout:7/526: link d0/d2/d1c/l9f d0/d2d/d58/lb2 0 2021-11-03T17:14:37.626 INFO:tasks.workunit.client.0.smithi027.stdout:7/527: chown d0/d2/d10/f1b 2085917 1 2021-11-03T17:14:37.630 INFO:tasks.workunit.client.0.smithi027.stdout:7/528: rename d0/d2/d1c/f97 to d0/d2/d10/d35/d96/fb3 0 2021-11-03T17:14:37.630 INFO:tasks.workunit.client.0.smithi027.stdout:7/529: mknod d0/d2/d10/cb4 1 2021-11-03T17:14:37.631 INFO:tasks.workunit.client.0.smithi027.stdout:7/530: creat d0/d2/d10/d35/fb5 x:0 0 0 2021-11-03T17:14:37.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:37 smithi115 conmon[34670]: cluster 2021-11-03T17:14:35.894901+0000 mgr.smithi115.tcyodz (mgr.24395) 18 : cluster [DBG] pgmap v12: 65 pgs: 65 active+clean; 2.8 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 25 MiB/s rd, 45 MiB/s wr, 107 op/s 2021-11-03T17:14:37.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:37 smithi115 conmon[34670]: cluster 2021-11-03T17:14:36.246454+0000 2021-11-03T17:14:37.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:37 smithi115 conmon[34670]: mon.smithi027 (mon.0) 621 : cluster [WRN] Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2021-11-03T17:14:37.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:37 smithi115 conmon[34670]: cluster 2021-11-03T17:14:36.253713+0000 mon.smithi027 (mon.0) 622 : cluster [DBG] mds.? [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] up:boot 2021-11-03T17:14:37.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:37 smithi115 conmon[34670]: cluster 2021-11-03T17:14:36.253757+0000 mon.smithi027 (mon.0) 623 : cluster 2021-11-03T17:14:37.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:37 smithi115 conmon[34670]: [INF] Standby daemon mds.cephfs.smithi027.fksixo assigned to filesystem cephfs as rank 0 2021-11-03T17:14:37.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:37 smithi115 conmon[34670]: cluster 2021-11-03T17:14: 2021-11-03T17:14:37.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:37 smithi115 conmon[34670]: 36.253994+0000 mon.smithi027 (mon.0) 624 : cluster [DBG] fsmap cephfs:1/2 {1=cephfs.smithi027.jdwohn=up:active} 3 up:standby, 1 failed 2021-11-03T17:14:37.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:37 smithi115 conmon[34670]: 2021-11-03T17:14:37.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:37 smithi115 conmon[34670]: audit 2021-11-03T17:14:37.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:37 smithi115 conmon[34670]: 2021-11-03T17:14:36.256831+0000 mon.smithi115 ( 2021-11-03T17:14:37.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:37 smithi115 conmon[34670]: mon.1) 53 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.zsfgnv"}]: dispatch 2021-11-03T17:14:37.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:37 smithi115 conmon[34670]: cluster 2021- 2021-11-03T17:14:37.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:37 smithi115 conmon[34670]: 11-03T17:14:36.259062+0000 2021-11-03T17:14:37.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:37 smithi115 conmon[34670]: mon.smithi027 (mon.0) 625 : cluster [DBG] 2021-11-03T17:14:37.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:37 smithi115 conmon[34670]: fsmap cephfs:2/2 {0=cephfs.smithi027.fksixo=up:replay,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:14:37.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:37 smithi115 conmon[34670]: audit 2021-11-03T17:14:36 2021-11-03T17:14:37.741 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:37 smithi115 conmon[34670]: .957495+0000 mon.smithi027 (mon.0) 626 : 2021-11-03T17:14:37.741 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:37 smithi115 conmon[34670]: audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' 2021-11-03T17:14:37.742 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:37 smithi115 conmon[34670]: audit 2021-11-03T 2021-11-03T17:14:37.742 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:37 smithi115 conmon[34670]: 17:14:36.959072+0000 mon.smithi115 (mon 2021-11-03T17:14:37.742 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:37 smithi115 conmon[34670]: .1) 54 : audit [INF] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi027.ufhhvb", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2021-11-03T17:14:37.743 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:37 smithi115 conmon[34670]: audit 2021- 2021-11-03T17:14:37.743 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:37 smithi115 conmon[34670]: 11-03T17:14:36.959381+0000 2021-11-03T17:14:37.744 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:37 smithi115 conmon[34670]: mon.smithi027 (mon.0) 627 : audit [INF] 2021-11-03T17:14:37.744 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:37 smithi115 conmon[34670]: from='mgr.24395 ' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi027.ufhhvb", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2021-11-03T17:14:37.744 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:37 smithi115 conmon[34670]: audit 2021-11-03T17:14:36 2021-11-03T17:14:37.745 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:37 smithi115 conmon[34670]: .960257+0000 mon.smithi115 (mon.1) 55 : audit 2021-11-03T17:14:37.745 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:37 smithi115 conmon[34670]: [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "mgr services"}]: dispatch 2021-11-03T17:14:37.746 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:37 smithi115 conmon[34670]: audit 2021-11-03T17:14: 2021-11-03T17:14:37.746 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:37 smithi115 conmon[34670]: 36.961057+0000 mon.smithi115 (mon.1) 56 : 2021-11-03T17:14:37.746 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:37 smithi115 conmon[34670]: audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:14:37.818 INFO:tasks.workunit.client.0.smithi027.stdout:9/683: sync 2021-11-03T17:14:37.821 INFO:tasks.workunit.client.0.smithi027.stdout:9/684: creat db/d1b/d1c/d1d/d40/dcc/ff9 x:0 0 0 2021-11-03T17:14:37.823 INFO:tasks.workunit.client.0.smithi027.stdout:9/685: dread db/d1b/d2b/d41/f4d [0,4194304] 0 2021-11-03T17:14:37.825 INFO:tasks.workunit.client.0.smithi027.stdout:9/686: mkdir db/d1b/dfa 0 2021-11-03T17:14:37.828 INFO:tasks.workunit.client.0.smithi027.stdout:9/687: rename db/d1b/d2b/f91 to db/d1b/d23/ffb 0 2021-11-03T17:14:37.830 INFO:tasks.workunit.client.0.smithi027.stdout:9/688: symlink db/d1b/d1c/d1d/d40/d60/lfc 0 2021-11-03T17:14:37.831 INFO:tasks.workunit.client.0.smithi027.stdout:9/689: chown db/d1b/d23 31258804 1 2021-11-03T17:14:38.367 INFO:tasks.workunit.client.0.smithi027.stdout:0/846: sync 2021-11-03T17:14:38.368 INFO:tasks.workunit.client.0.smithi027.stdout:0/847: chown d9/da/d30/dac/l100 2951274 1 2021-11-03T17:14:38.368 INFO:tasks.workunit.client.0.smithi027.stdout:0/848: chown d9/da/d31/l88 0 1 2021-11-03T17:14:38.368 INFO:tasks.workunit.client.0.smithi027.stdout:0/849: fdatasync d9/f6c 0 2021-11-03T17:14:38.369 INFO:tasks.workunit.client.0.smithi027.stdout:0/850: chown d9/da/d30/f43 31 1 2021-11-03T17:14:38.369 INFO:tasks.workunit.client.0.smithi027.stdout:0/851: write d9/d12/f28 [3306591,121139] 0 2021-11-03T17:14:38.370 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:38 smithi027 conmon[32322]: cephadm 2021-11-03T17:14: 2021-11-03T17:14:38.370 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:38 smithi027 conmon[32322]: 36.923398+0000 mgr.smithi115.tcyodz (mgr.24395) 19 : cephadm [INF] poll took 2973.218 ms: 1 events 2021-11-03T17:14:38.370 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:38 smithi027 conmon[32322]: cephadm 2021-11-03T17:14:36.952972+0000 mgr.smithi115.tcyodz (mgr.24395) 20 : cephadm [INF] Upgrade: Updating mgr.smithi027.ufhhvb 2021-11-03T17:14:38.371 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:38 smithi027 conmon[32322]: cephadm 2021-11-03T17:14:36.961681+0000 mgr.smithi115.tcyodz (mgr.24395) 21 : cephadm 2021-11-03T17:14:38.371 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:38 smithi027 conmon[32322]: [INF] Deploying daemon mgr.smithi027.ufhhvb on smithi027 2021-11-03T17:14:38.373 INFO:tasks.workunit.client.0.smithi027.stdout:0/852: creat d9/da/d30/f120 x:0 0 0 2021-11-03T17:14:38.373 INFO:tasks.workunit.client.0.smithi027.stdout:0/853: mknod d9/da/d64/d70/d6d/d9f/c121 1 2021-11-03T17:14:38.382 INFO:tasks.workunit.client.0.smithi027.stdout:0/854: dread d9/da/d30/f5e [0,4194304] 0 2021-11-03T17:14:38.643 INFO:tasks.workunit.client.0.smithi027.stdout:4/550: sync 2021-11-03T17:14:38.644 INFO:tasks.workunit.client.0.smithi027.stdout:1/717: sync 2021-11-03T17:14:38.644 INFO:tasks.workunit.client.0.smithi027.stdout:4/551: chown d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/f93 2 1 2021-11-03T17:14:38.644 INFO:tasks.workunit.client.0.smithi027.stdout:4/552: stat d6/d24/d6d 0 2021-11-03T17:14:38.644 INFO:tasks.workunit.client.0.smithi027.stdout:4/553: read - d6/d9/d16/d36/d70/d3e/d58/d84/d9b/fb3 zero size 2021-11-03T17:14:38.645 INFO:tasks.workunit.client.0.smithi027.stdout:4/554: mknod d6/d24/db0/d81/cb8 1 2021-11-03T17:14:38.646 INFO:tasks.workunit.client.0.smithi027.stdout:4/555: rename d6/d9/d28 to d6/d9/d16/d36/d99/db9 0 2021-11-03T17:14:38.648 INFO:tasks.workunit.client.0.smithi027.stdout:1/718: symlink d6/d11/d14/df6/lf8 0 2021-11-03T17:14:38.650 INFO:tasks.workunit.client.0.smithi027.stdout:4/556: creat d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d9a/fba x:0 0 0 2021-11-03T17:14:38.651 INFO:tasks.workunit.client.0.smithi027.stdout:1/719: rmdir d6/de/d13 39 2021-11-03T17:14:38.652 INFO:tasks.workunit.client.0.smithi027.stdout:4/557: rmdir d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d41 39 2021-11-03T17:14:38.654 INFO:tasks.workunit.client.0.smithi027.stdout:1/720: mkdir d6/d11/dac/d79/d37/de0/de4/df9 0 2021-11-03T17:14:38.654 INFO:tasks.workunit.client.0.smithi027.stdout:1/721: chown d6/d11/dac/d79/d25/d28/d41/l9e 33988 1 2021-11-03T17:14:38.655 INFO:tasks.workunit.client.0.smithi027.stdout:1/722: readlink d6/de/l45 0 2021-11-03T17:14:38.655 INFO:tasks.workunit.client.0.smithi027.stdout:4/558: mkdir d6/d9/d16/d36/d57/dbb 0 2021-11-03T17:14:38.657 INFO:tasks.workunit.client.0.smithi027.stdout:1/723: mkdir d6/d11/dac/d79/dfa 0 2021-11-03T17:14:38.657 INFO:tasks.workunit.client.0.smithi027.stdout:1/724: chown d6/d11 9 1 2021-11-03T17:14:38.657 INFO:tasks.workunit.client.0.smithi027.stdout:1/725: fdatasync d6/de/f19 0 2021-11-03T17:14:38.671 INFO:tasks.workunit.client.0.smithi027.stdout:1/726: write d6/d11/dac/d79/d25/d28/d5c/f73 [3098099,55183] 0 2021-11-03T17:14:38.672 INFO:tasks.workunit.client.0.smithi027.stdout:4/559: dwrite d6/d8c/d9f/fa8 [0,4194304] 0 2021-11-03T17:14:38.672 INFO:tasks.workunit.client.0.smithi027.stdout:4/560: readlink d6/d24/l80 0 2021-11-03T17:14:38.674 INFO:tasks.workunit.client.0.smithi027.stdout:4/561: link d6/d9/d16/d36/d70/l6f d6/d9/d16/d36/d70/d3e/d58/d84/d9b/lbc 0 2021-11-03T17:14:38.675 INFO:tasks.workunit.client.0.smithi027.stdout:4/562: write d6/d9/d16/d36/d99/db9/f96 [4959,98819] 0 2021-11-03T17:14:38.675 INFO:tasks.workunit.client.0.smithi027.stdout:1/727: write d6/d11/d14/f17 [2630668,20342] 0 2021-11-03T17:14:38.675 INFO:tasks.workunit.client.0.smithi027.stdout:1/728: stat d6/d11/dac/d79/d37/de0/l6e 0 2021-11-03T17:14:38.675 INFO:tasks.workunit.client.0.smithi027.stdout:1/729: chown d6/d11/dac/d79/d25/d28 7 1 2021-11-03T17:14:38.676 INFO:tasks.workunit.client.0.smithi027.stdout:1/730: dread - d6/d11/dac/d79/d37/de0/d43/f5d zero size 2021-11-03T17:14:38.676 INFO:tasks.workunit.client.0.smithi027.stdout:1/731: readlink d6/d11/dac/d79/d37/d49/la1 0 2021-11-03T17:14:38.676 INFO:tasks.workunit.client.0.smithi027.stdout:4/563: read d6/d9/d16/d36/f4c [1528897,64436] 0 2021-11-03T17:14:38.676 INFO:tasks.workunit.client.0.smithi027.stdout:4/564: fsync d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/f93 0 2021-11-03T17:14:38.676 INFO:tasks.workunit.client.0.smithi027.stdout:4/565: fdatasync d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d2b/f45 0 2021-11-03T17:14:38.677 INFO:tasks.workunit.client.0.smithi027.stdout:1/732: creat d6/d11/de2/ffb x:0 0 0 2021-11-03T17:14:38.678 INFO:tasks.workunit.client.0.smithi027.stdout:1/733: truncate d6/d11/dac/d79/d25/d5e/dd1/ff3 863998 0 2021-11-03T17:14:38.678 INFO:tasks.workunit.client.0.smithi027.stdout:4/566: symlink d6/d9/d8f/dad/lbd 0 2021-11-03T17:14:38.678 INFO:tasks.workunit.client.0.smithi027.stdout:4/567: getdents d6/d8c 0 2021-11-03T17:14:38.679 INFO:tasks.workunit.client.0.smithi027.stdout:4/568: mkdir d6/d24/dbe 0 2021-11-03T17:14:38.679 INFO:tasks.workunit.client.0.smithi027.stdout:4/569: mknod d6/d9/d16/d36/d70/cbf 1 2021-11-03T17:14:38.679 INFO:tasks.workunit.client.0.smithi027.stdout:4/570: chown d6/d9/d16/d36/d70/d3e/d58/f61 6 1 2021-11-03T17:14:38.680 INFO:tasks.workunit.client.0.smithi027.stdout:4/571: creat d6/d9/d16/d36/fc0 x:0 0 0 2021-11-03T17:14:38.702 INFO:tasks.workunit.client.0.smithi027.stdout:1/734: dwrite d6/d11/d14/f88 [0,4194304] 0 2021-11-03T17:14:38.704 INFO:tasks.workunit.client.0.smithi027.stdout:1/735: truncate d6/de/f1a 873799 0 2021-11-03T17:14:38.704 INFO:tasks.workunit.client.0.smithi027.stdout:4/572: dwrite d6/d24/db0/f76 [0,4194304] 0 2021-11-03T17:14:38.708 INFO:tasks.workunit.client.0.smithi027.stdout:4/573: mkdir d6/d9/d16/d36/dc1 0 2021-11-03T17:14:38.721 INFO:tasks.workunit.client.0.smithi027.stdout:4/574: truncate d6/d8c/d9f/fa8 1334939 0 2021-11-03T17:14:38.722 INFO:tasks.workunit.client.0.smithi027.stdout:4/575: fsync d6/d9/d16/d36/f42 0 2021-11-03T17:14:38.723 INFO:tasks.workunit.client.0.smithi027.stdout:4/576: truncate d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/f37 1760259 0 2021-11-03T17:14:38.725 INFO:tasks.workunit.client.0.smithi027.stdout:4/577: rmdir d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d9a 39 2021-11-03T17:14:38.725 INFO:tasks.workunit.client.0.smithi027.stdout:4/578: chown d6/d9/d16/d36/d70/l6f 5 1 2021-11-03T17:14:38.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:38 smithi115 conmon[34670]: cephadm 2021- 2021-11-03T17:14:38.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:38 smithi115 conmon[34670]: 11-03T17:14:36.923398+0000 mgr.smithi115.tcyodz (mgr.24395) 19 : cephadm [INF] poll took 2973.218 ms: 1 events 2021-11-03T17:14:38.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:38 smithi115 conmon[34670]: cephadm 2021-11-03T17:14:36.952972+0000 mgr.smithi115.tcyodz (mgr.24395) 20 : cephadm [INF] Upgrade: Updating mgr.smithi027.ufhhvb 2021-11-03T17:14:38.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:38 smithi115 conmon[34670]: cephadm 2021-11-03T17:14:36.961681+0000 mgr.smithi115.tcyodz (mgr.24395 2021-11-03T17:14:38.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:38 smithi115 conmon[34670]: ) 21 : cephadm [INF] Deploying daemon mgr.smithi027.ufhhvb on smithi027 2021-11-03T17:14:38.737 INFO:tasks.workunit.client.0.smithi027.stdout:4/579: dread d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/f37 [0,4194304] 0 2021-11-03T17:14:38.741 INFO:tasks.workunit.client.0.smithi027.stdout:4/580: dread d6/d9/d16/d36/d70/d3e/d58/d84/f86 [0,4194304] 0 2021-11-03T17:14:38.742 INFO:tasks.workunit.client.0.smithi027.stdout:4/581: creat d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/fc2 x:0 0 0 2021-11-03T17:14:38.744 INFO:tasks.workunit.client.0.smithi027.stdout:4/582: dread d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/f8a [0,4194304] 0 2021-11-03T17:14:38.760 INFO:tasks.workunit.client.0.smithi027.stdout:5/885: sync 2021-11-03T17:14:38.761 INFO:tasks.workunit.client.0.smithi027.stdout:5/886: read d2/d2c/d3c/d58/f74 [2675857,106973] 0 2021-11-03T17:14:38.761 INFO:tasks.workunit.client.0.smithi027.stdout:5/887: read - d2/d4/d4e/d67/d20/f103 zero size 2021-11-03T17:14:38.764 INFO:tasks.workunit.client.0.smithi027.stdout:5/888: rmdir d2/d2c/d3c/d48/d65/dd6 39 2021-11-03T17:14:38.769 INFO:tasks.workunit.client.0.smithi027.stdout:4/583: dwrite d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/f38 [0,4194304] 0 2021-11-03T17:14:38.776 INFO:tasks.workunit.client.0.smithi027.stdout:4/584: mkdir d6/d9/d16/dc3 0 2021-11-03T17:14:38.776 INFO:tasks.workunit.client.0.smithi027.stdout:4/585: fsync d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/f38 0 2021-11-03T17:14:38.777 INFO:tasks.workunit.client.0.smithi027.stdout:4/586: stat d6/d9/d16/d36 0 2021-11-03T17:14:38.777 INFO:tasks.workunit.client.0.smithi027.stdout:4/587: mknod d6/d9/d16/d17/cc4 1 2021-11-03T17:14:38.779 INFO:tasks.workunit.client.0.smithi027.stdout:4/588: truncate f4 3595143 0 2021-11-03T17:14:38.782 INFO:tasks.workunit.client.0.smithi027.stdout:4/589: rename d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/f94 to d6/d9/d16/d36/d70/d3e/d58/fc5 0 2021-11-03T17:14:38.783 INFO:tasks.workunit.client.0.smithi027.stdout:4/590: write d6/d9/d16/d36/d70/d3e/d58/f61 [932675,10509] 0 2021-11-03T17:14:38.783 INFO:tasks.workunit.client.0.smithi027.stdout:4/591: write d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d41/f88 [3671708,9146] 0 2021-11-03T17:14:38.784 INFO:tasks.workunit.client.0.smithi027.stdout:5/889: dwrite d2/d2c/d3c/d58/fa0 [4194304,4194304] 0 2021-11-03T17:14:38.784 INFO:tasks.workunit.client.0.smithi027.stdout:5/890: fsync d2/f3 0 2021-11-03T17:14:38.785 INFO:tasks.workunit.client.0.smithi027.stdout:4/592: rename d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d41/f77 to d6/d24/dbe/fc6 0 2021-11-03T17:14:38.787 INFO:tasks.workunit.client.0.smithi027.stdout:5/891: mkdir d2/d2c/d3c/dc8/d121 0 2021-11-03T17:14:38.788 INFO:tasks.workunit.client.0.smithi027.stdout:5/892: fsync d2/d2c/d3c/d58/f100 0 2021-11-03T17:14:38.788 INFO:tasks.workunit.client.0.smithi027.stdout:5/893: write d2/d12/dad/d6a/f106 [1007954,77004] 0 2021-11-03T17:14:38.788 INFO:tasks.workunit.client.0.smithi027.stdout:5/894: chown d2/d1f/l6d 679 1 2021-11-03T17:14:38.790 INFO:tasks.workunit.client.0.smithi027.stdout:5/895: mkdir d2/d2c/d9b/db3/d116/d122 0 2021-11-03T17:14:38.790 INFO:tasks.workunit.client.0.smithi027.stdout:5/896: mknod d2/d1f/c123 1 2021-11-03T17:14:38.791 INFO:tasks.workunit.client.0.smithi027.stdout:5/897: mknod d2/d4/d4e/d67/d20/c124 1 2021-11-03T17:14:38.791 INFO:tasks.workunit.client.0.smithi027.stdout:5/898: truncate d2/d4/f34 2037636 0 2021-11-03T17:14:38.792 INFO:tasks.workunit.client.0.smithi027.stdout:5/899: getdents d2/d2c/d3c/d48/d65/d80/dd2 0 2021-11-03T17:14:38.792 INFO:tasks.workunit.client.0.smithi027.stdout:5/900: chown d2/d12/dad/da8/dd9/dde/l11b 802523284 1 2021-11-03T17:14:38.796 INFO:tasks.workunit.client.0.smithi027.stdout:5/901: rename d2/f52 to d2/d12/dad/d6a/f125 0 2021-11-03T17:14:38.813 INFO:tasks.workunit.client.0.smithi027.stdout:4/593: dwrite d6/d8c/d9f/fa8 [0,4194304] 0 2021-11-03T17:14:38.815 INFO:tasks.workunit.client.0.smithi027.stdout:4/594: link d6/d24/l80 d6/d9/d16/d36/d70/d3e/d58/d84/lc7 0 2021-11-03T17:14:38.816 INFO:tasks.workunit.client.0.smithi027.stdout:5/902: dwrite d2/d2c/d3c/d48/d65/d80/d9e/ffb [0,4194304] 0 2021-11-03T17:14:38.821 INFO:tasks.workunit.client.0.smithi027.stdout:4/595: rename d6/d24/l80 to d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/lc8 0 2021-11-03T17:14:38.823 INFO:tasks.workunit.client.0.smithi027.stdout:5/903: creat d2/d4/d4e/d67/d8e/f126 x:0 0 0 2021-11-03T17:14:38.828 INFO:tasks.workunit.client.0.smithi027.stdout:4/596: mkdir d6/d24/dc9 0 2021-11-03T17:14:38.830 INFO:tasks.workunit.client.0.smithi027.stdout:4/597: write d6/d9/d16/d17/f56 [3422054,88812] 0 2021-11-03T17:14:38.832 INFO:tasks.workunit.client.0.smithi027.stdout:5/904: dread d2/d12/dad/da9/f108 [0,4194304] 0 2021-11-03T17:14:38.833 INFO:tasks.workunit.client.0.smithi027.stdout:5/905: readlink d2/d2c/d3c/d48/l59 0 2021-11-03T17:14:38.848 INFO:tasks.workunit.client.0.smithi027.stdout:4/598: dread d6/f48 [0,4194304] 0 2021-11-03T17:14:38.848 INFO:tasks.workunit.client.0.smithi027.stdout:4/599: chown d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d2b/f3f 3906860 1 2021-11-03T17:14:38.851 INFO:tasks.workunit.client.0.smithi027.stdout:4/600: rename d6/f48 to d6/d9/d8f/fca 0 2021-11-03T17:14:38.851 INFO:tasks.workunit.client.0.smithi027.stdout:4/601: readlink d6/d24/d6d/l7d 0 2021-11-03T17:14:38.851 INFO:tasks.workunit.client.0.smithi027.stdout:4/602: chown d6/d9/d8f/dad 213901 1 2021-11-03T17:14:38.852 INFO:tasks.workunit.client.0.smithi027.stdout:4/603: creat d6/d9/d16/d36/d70/d3e/d58/d84/d9b/fcb x:0 0 0 2021-11-03T17:14:38.852 INFO:tasks.workunit.client.0.smithi027.stdout:4/604: truncate d6/d9/d16/d17/f6e 4358388 0 2021-11-03T17:14:38.853 INFO:tasks.workunit.client.0.smithi027.stdout:4/605: stat d6/d9/d16/f7f 0 2021-11-03T17:14:38.853 INFO:tasks.workunit.client.0.smithi027.stdout:4/606: stat d6/d9/d16/dc3 0 2021-11-03T17:14:38.853 INFO:tasks.workunit.client.0.smithi027.stdout:5/906: dwrite d2/d4/d4e/d67/d20/fb9 [0,4194304] 0 2021-11-03T17:14:38.855 INFO:tasks.workunit.client.0.smithi027.stdout:5/907: symlink d2/d4/d4e/d67/d8e/l127 0 2021-11-03T17:14:38.856 INFO:tasks.workunit.client.0.smithi027.stdout:4/607: symlink d6/d9/d16/d17/lcc 0 2021-11-03T17:14:38.859 INFO:tasks.workunit.client.0.smithi027.stdout:4/608: rename d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/l5e to d6/d9/d16/lcd 0 2021-11-03T17:14:38.866 INFO:tasks.workunit.client.0.smithi027.stdout:5/908: dread d2/d2c/d3c/d58/f100 [0,4194304] 0 2021-11-03T17:14:38.867 INFO:tasks.workunit.client.0.smithi027.stdout:5/909: mknod d2/d2c/d3c/d48/d65/d80/dd2/c128 1 2021-11-03T17:14:38.870 INFO:tasks.workunit.client.0.smithi027.stdout:4/609: dread d6/d8c/d9f/fa8 [0,4194304] 0 2021-11-03T17:14:38.871 INFO:tasks.workunit.client.0.smithi027.stdout:4/610: creat d6/d9/d16/d36/d57/fce x:0 0 0 2021-11-03T17:14:38.871 INFO:tasks.workunit.client.0.smithi027.stdout:4/611: rename d6/d24/db0 to d6/d24/db0/d81/dcf 22 2021-11-03T17:14:38.872 INFO:tasks.workunit.client.0.smithi027.stdout:4/612: mkdir d6/d9/d16/d36/d99/dd0 0 2021-11-03T17:14:38.874 INFO:tasks.workunit.client.0.smithi027.stdout:4/613: write d6/d24/f46 [2526501,12470] 0 2021-11-03T17:14:38.886 INFO:tasks.workunit.client.0.smithi027.stdout:5/910: dwrite d2/d2c/d3c/fdd [0,4194304] 0 2021-11-03T17:14:38.891 INFO:tasks.workunit.client.0.smithi027.stdout:5/911: link d2/d4/d4e/d67/f7f d2/d12/dad/da8/f129 0 2021-11-03T17:14:38.891 INFO:tasks.workunit.client.0.smithi027.stdout:5/912: readlink d2/d2c/l8a 0 2021-11-03T17:14:38.892 INFO:tasks.workunit.client.0.smithi027.stdout:5/913: truncate d2/d12/dad/da9/f109 280645 0 2021-11-03T17:14:38.893 INFO:tasks.workunit.client.0.smithi027.stdout:5/914: creat d2/d2c/d9b/f12a x:0 0 0 2021-11-03T17:14:38.895 INFO:tasks.workunit.client.0.smithi027.stdout:5/915: rename d2/d4/d4e/f114 to d2/d4/d4e/d67/d8e/d99/f12b 0 2021-11-03T17:14:38.895 INFO:tasks.workunit.client.0.smithi027.stdout:4/614: dwrite d6/d9/d16/d36/d70/d3e/d58/f61 [4194304,4194304] 0 2021-11-03T17:14:38.912 INFO:tasks.workunit.client.0.smithi027.stdout:5/916: rename d2/d12/d7d/da7 to d2/d12/dad/da8/dd9/d12c 0 2021-11-03T17:14:38.915 INFO:tasks.workunit.client.0.smithi027.stdout:4/615: dwrite d6/d9/d16/d36/d70/d3e/d58/f61 [8388608,4194304] 0 2021-11-03T17:14:38.923 INFO:tasks.workunit.client.0.smithi027.stdout:4/616: dread d6/d9/d16/f32 [4194304,4194304] 0 2021-11-03T17:14:38.925 INFO:tasks.workunit.client.0.smithi027.stdout:4/617: write d6/d9/d16/d17/f56 [1533714,60754] 0 2021-11-03T17:14:38.926 INFO:tasks.workunit.client.0.smithi027.stdout:4/618: chown d6/d9/d16/d36/d70/d3e/l85 4 1 2021-11-03T17:14:38.926 INFO:tasks.workunit.client.0.smithi027.stdout:4/619: chown d6/d24/db0/f53 672419 1 2021-11-03T17:14:38.926 INFO:tasks.workunit.client.0.smithi027.stdout:4/620: stat d6/d9/d16/d36/d57/d5a 0 2021-11-03T17:14:38.926 INFO:tasks.workunit.client.0.smithi027.stdout:4/621: stat d6/d9/f1b 0 2021-11-03T17:14:38.927 INFO:tasks.workunit.client.0.smithi027.stdout:4/622: write d6/d9/d16/d36/d97/faa [449447,68915] 0 2021-11-03T17:14:38.927 INFO:tasks.workunit.client.0.smithi027.stdout:5/917: stat d2/l8 0 2021-11-03T17:14:38.928 INFO:tasks.workunit.client.0.smithi027.stdout:5/918: mknod d2/c12d 1 2021-11-03T17:14:38.929 INFO:tasks.workunit.client.0.smithi027.stdout:5/919: symlink d2/d12/dad/d6a/l12e 0 2021-11-03T17:14:38.929 INFO:tasks.workunit.client.0.smithi027.stdout:5/920: chown d2/f18 5 1 2021-11-03T17:14:38.929 INFO:tasks.workunit.client.0.smithi027.stdout:5/921: mknod d2/d2c/d3c/d58/c12f 1 2021-11-03T17:14:38.935 INFO:tasks.workunit.client.0.smithi027.stdout:5/922: creat d2/d12/f130 x:0 0 0 2021-11-03T17:14:38.943 INFO:tasks.workunit.client.0.smithi027.stdout:5/923: dread d2/d2c/d3c/f84 [0,4194304] 0 2021-11-03T17:14:38.944 INFO:tasks.workunit.client.0.smithi027.stdout:4/623: dwrite d6/d9/d16/d36/d57/f75 [0,4194304] 0 2021-11-03T17:14:38.944 INFO:tasks.workunit.client.0.smithi027.stdout:5/924: truncate d2/d1f/fbf 274081 0 2021-11-03T17:14:38.945 INFO:tasks.workunit.client.0.smithi027.stdout:5/925: mknod d2/d12/dad/da9/c131 1 2021-11-03T17:14:38.948 INFO:tasks.workunit.client.0.smithi027.stdout:4/624: creat d6/d9/d16/d36/d70/d3e/fd1 x:0 0 0 2021-11-03T17:14:38.952 INFO:tasks.workunit.client.0.smithi027.stdout:5/926: rmdir d2/d4/d4e/d11d 0 2021-11-03T17:14:38.952 INFO:tasks.workunit.client.0.smithi027.stdout:5/927: mknod d2/d4/d4e/c132 1 2021-11-03T17:14:38.958 INFO:tasks.workunit.client.0.smithi027.stdout:4/625: link d6/d9/d16/d36/f42 d6/d24/d6d/db1/fd2 0 2021-11-03T17:14:38.961 INFO:tasks.workunit.client.0.smithi027.stdout:5/928: rename d2/d2c/d3c/d58/f100 to d2/d2c/d3c/d48/d65/f133 0 2021-11-03T17:14:38.961 INFO:tasks.workunit.client.0.smithi027.stdout:5/929: write d2/d2c/d9b/de1/f119 [876062,75520] 0 2021-11-03T17:14:38.963 INFO:tasks.workunit.client.0.smithi027.stdout:4/626: getdents d6/d9/d8f/dad 0 2021-11-03T17:14:38.965 INFO:tasks.workunit.client.0.smithi027.stdout:5/930: truncate d2/d4/d61/f89 3170600 0 2021-11-03T17:14:38.965 INFO:tasks.workunit.client.0.smithi027.stdout:5/931: chown d2/d12/dad/da8/dd9/d12c 4275085 1 2021-11-03T17:14:38.967 INFO:tasks.workunit.client.0.smithi027.stdout:4/627: rename d6/d8c/d9f/fa3 to d6/d9/d16/dc3/fd3 0 2021-11-03T17:14:38.973 INFO:tasks.workunit.client.0.smithi027.stdout:5/932: dread d2/d12/dad/da9/fe4 [0,4194304] 0 2021-11-03T17:14:38.977 INFO:tasks.workunit.client.0.smithi027.stdout:5/933: rename d2/d12/dad/l4d to d2/d2c/l134 0 2021-11-03T17:14:38.977 INFO:tasks.workunit.client.0.smithi027.stdout:5/934: mknod d2/d12/dad/da9/c135 1 2021-11-03T17:14:38.977 INFO:tasks.workunit.client.0.smithi027.stdout:4/628: dread d6/d9/d16/d36/f4c [0,4194304] 0 2021-11-03T17:14:38.988 INFO:tasks.workunit.client.0.smithi027.stdout:5/935: dread d2/d4/d4e/d67/f11 [0,4194304] 0 2021-11-03T17:14:38.989 INFO:tasks.workunit.client.0.smithi027.stdout:5/936: creat d2/d2c/d9b/de1/f136 x:0 0 0 2021-11-03T17:14:38.990 INFO:tasks.workunit.client.0.smithi027.stdout:5/937: mknod d2/d4/d4e/c137 1 2021-11-03T17:14:38.992 INFO:tasks.workunit.client.0.smithi027.stdout:5/938: unlink d2/d12/l3b 0 2021-11-03T17:14:38.993 INFO:tasks.workunit.client.0.smithi027.stdout:5/939: creat d2/d2c/d3c/d58/f138 x:0 0 0 2021-11-03T17:14:38.999 INFO:tasks.workunit.client.0.smithi027.stdout:5/940: write d2/d12/dad/f71 [1580934,90385] 0 2021-11-03T17:14:39.000 INFO:tasks.workunit.client.0.smithi027.stdout:4/629: dwrite d6/d9/d16/d36/d57/fce [0,4194304] 0 2021-11-03T17:14:39.005 INFO:tasks.workunit.client.0.smithi027.stdout:4/630: rename d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/f38 to d6/d9/d16/d36/d57/d5a/d63/fd4 0 2021-11-03T17:14:39.018 INFO:tasks.workunit.client.0.smithi027.stdout:5/941: dwrite d2/d2c/d3c/d48/d65/d80/fd8 [0,4194304] 0 2021-11-03T17:14:39.027 INFO:tasks.workunit.client.0.smithi027.stdout:4/631: dwrite d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/fa2 [0,4194304] 0 2021-11-03T17:14:39.032 INFO:tasks.workunit.client.0.smithi027.stdout:4/632: symlink d6/d9/d16/d36/d99/db9/ld5 0 2021-11-03T17:14:39.035 INFO:tasks.workunit.client.0.smithi027.stdout:4/633: write d6/d24/db0/f83 [3280624,87542] 0 2021-11-03T17:14:39.036 INFO:tasks.workunit.client.0.smithi027.stdout:5/942: dwrite d2/d4/d4e/d67/d20/f103 [0,4194304] 0 2021-11-03T17:14:39.037 INFO:tasks.workunit.client.0.smithi027.stdout:4/634: rename d6/d9/d16/d36/d57/d5a/f5c to d6/d9/d16/d36/d99/db9/fd6 0 2021-11-03T17:14:39.038 INFO:tasks.workunit.client.0.smithi027.stdout:4/635: write d6/d9/d16/f26 [3704775,47366] 0 2021-11-03T17:14:39.039 INFO:tasks.workunit.client.0.smithi027.stdout:4/636: chown d6/d9/d16/d36 3769 1 2021-11-03T17:14:39.040 INFO:tasks.workunit.client.0.smithi027.stdout:5/943: creat d2/d1f/f139 x:0 0 0 2021-11-03T17:14:39.041 INFO:tasks.workunit.client.0.smithi027.stdout:5/944: write d2/d2c/d9b/fcd [4666992,27664] 0 2021-11-03T17:14:39.042 INFO:tasks.workunit.client.0.smithi027.stdout:4/637: stat d6/d9/d16/d36/d57/d5a/l9c 0 2021-11-03T17:14:39.045 INFO:tasks.workunit.client.0.smithi027.stdout:5/945: link d2/f42 d2/d2c/d3c/d48/d65/f13a 0 2021-11-03T17:14:39.050 INFO:tasks.workunit.client.0.smithi027.stdout:4/638: mkdir d6/d8c/d9f/dd7 0 2021-11-03T17:14:39.052 INFO:tasks.workunit.client.0.smithi027.stdout:5/946: symlink d2/d2c/l13b 0 2021-11-03T17:14:39.053 INFO:tasks.workunit.client.0.smithi027.stdout:5/947: truncate d2/d12/f5c 1118551 0 2021-11-03T17:14:39.053 INFO:tasks.workunit.client.0.smithi027.stdout:5/948: stat d2/d2c/d9b 0 2021-11-03T17:14:39.053 INFO:tasks.workunit.client.0.smithi027.stdout:5/949: fdatasync d2/d4/d4e/d67/d20/f3a 0 2021-11-03T17:14:39.053 INFO:tasks.workunit.client.0.smithi027.stdout:5/950: chown d2/d4/l10a 166015964 1 2021-11-03T17:14:39.054 INFO:tasks.workunit.client.0.smithi027.stdout:5/951: fdatasync d2/d12/dad/da9/f108 0 2021-11-03T17:14:39.054 INFO:tasks.workunit.client.0.smithi027.stdout:5/952: fdatasync d2/d1f/fbc 0 2021-11-03T17:14:39.054 INFO:tasks.workunit.client.0.smithi027.stdout:4/639: creat d6/d24/db0/d81/fd8 x:0 0 0 2021-11-03T17:14:39.055 INFO:tasks.workunit.client.0.smithi027.stdout:5/953: mkdir d2/d2c/d3c/d48/d65/dd6/d13c 0 2021-11-03T17:14:39.055 INFO:tasks.workunit.client.0.smithi027.stdout:4/640: creat d6/d9/d16/d36/d57/d5a/d63/fd9 x:0 0 0 2021-11-03T17:14:39.056 INFO:tasks.workunit.client.0.smithi027.stdout:4/641: mknod d6/cda 1 2021-11-03T17:14:39.056 INFO:tasks.workunit.client.0.smithi027.stdout:4/642: fsync d6/d24/db0/d81/fb7 0 2021-11-03T17:14:39.056 INFO:tasks.workunit.client.0.smithi027.stdout:4/643: chown d6/d9/d16/d36/d70/d3e/d58/d84/d9b/fa1 583896171 1 2021-11-03T17:14:39.057 INFO:tasks.workunit.client.0.smithi027.stdout:4/644: readlink d6/d9/d8f/dad/lbd 0 2021-11-03T17:14:39.059 INFO:tasks.workunit.client.0.smithi027.stdout:4/645: rename d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/f8e to d6/d9/d16/d36/d57/fdb 0 2021-11-03T17:14:39.063 INFO:tasks.workunit.client.0.smithi027.stdout:4/646: read d6/d8c/d9f/fa8 [11739981,77473] 0 2021-11-03T17:14:39.063 INFO:tasks.workunit.client.0.smithi027.stdout:4/647: readlink d6/d9/d16/d17/l1d 0 2021-11-03T17:14:39.078 INFO:tasks.workunit.client.0.smithi027.stdout:4/648: dwrite d6/d9/d8f/f95 [0,4194304] 0 2021-11-03T17:14:39.079 INFO:tasks.workunit.client.0.smithi027.stdout:4/649: dread - d6/d9/d16/d36/d57/fdb zero size 2021-11-03T17:14:39.079 INFO:tasks.workunit.client.0.smithi027.stdout:4/650: write d6/d9/d16/d36/d70/d3e/d58/d84/d9b/fa1 [688702,33033] 0 2021-11-03T17:14:39.084 INFO:tasks.workunit.client.0.smithi027.stdout:4/651: dread d6/d9/d16/f32 [4194304,4194304] 0 2021-11-03T17:14:39.084 INFO:tasks.workunit.client.0.smithi027.stdout:4/652: rename d6 to d6/d9/d16/d36/d97/ddc 22 2021-11-03T17:14:39.103 INFO:tasks.workunit.client.0.smithi027.stdout:4/653: dwrite d6/d9/d16/d36/d70/d3e/d58/f61 [4194304,4194304] 0 2021-11-03T17:14:39.103 INFO:tasks.workunit.client.0.smithi027.stdout:4/654: write d6/d24/db0/d81/fb7 [284844,71388] 0 2021-11-03T17:14:39.104 INFO:tasks.workunit.client.0.smithi027.stdout:4/655: chown d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21 2 1 2021-11-03T17:14:39.352 INFO:tasks.workunit.client.0.smithi027.stdout:6/638: sync 2021-11-03T17:14:39.353 INFO:tasks.workunit.client.0.smithi027.stdout:6/639: readlink dc/d16/d1b/la6 0 2021-11-03T17:14:39.354 INFO:tasks.workunit.client.0.smithi027.stdout:6/640: link dc/f7b dc/d82/d57/fbc 0 2021-11-03T17:14:39.356 INFO:tasks.workunit.client.0.smithi027.stdout:6/641: creat dc/d37/fbd x:0 0 0 2021-11-03T17:14:39.357 INFO:tasks.workunit.client.0.smithi027.stdout:6/642: read dc/f88 [3890345,15376] 0 2021-11-03T17:14:39.360 INFO:tasks.workunit.client.0.smithi027.stdout:6/643: rename dc/d16/d1b/d3c/d51/fa3 to dc/d37/fbe 0 2021-11-03T17:14:39.360 INFO:tasks.workunit.client.0.smithi027.stdout:6/644: chown dc/d16/l3d 66 1 2021-11-03T17:14:39.361 INFO:tasks.workunit.client.0.smithi027.stdout:6/645: symlink dc/d82/d57/dad/dae/db1/lbf 0 2021-11-03T17:14:39.362 INFO:tasks.workunit.client.0.smithi027.stdout:6/646: mknod dc/d82/cc0 1 2021-11-03T17:14:39.362 INFO:tasks.workunit.client.0.smithi027.stdout:6/647: chown dc/da1 648979 1 2021-11-03T17:14:39.363 INFO:tasks.workunit.client.0.smithi027.stdout:6/648: readlink dc/d82/d57/l59 0 2021-11-03T17:14:39.363 INFO:tasks.workunit.client.0.smithi027.stdout:6/649: stat dc/d16/d9a 0 2021-11-03T17:14:39.375 INFO:tasks.workunit.client.0.smithi027.stdout:6/650: dread dc/d82/d57/fbc [0,4194304] 0 2021-11-03T17:14:39.431 INFO:tasks.workunit.client.0.smithi027.stdout:6/651: write dc/d16/d30/f32 [2482763,117808] 0 2021-11-03T17:14:39.433 INFO:tasks.workunit.client.0.smithi027.stdout:6/652: rename dc/d16/d1b/d24/d7e/l95 to dc/d16/d1b/d24/lc1 0 2021-11-03T17:14:39.433 INFO:tasks.workunit.client.0.smithi027.stdout:6/653: fsync dc/f1f 0 2021-11-03T17:14:39.453 INFO:tasks.workunit.client.0.smithi027.stdout:6/654: dwrite dc/d16/d1b/d3c/d51/f78 [0,4194304] 0 2021-11-03T17:14:39.456 INFO:tasks.workunit.client.0.smithi027.stdout:6/655: creat dc/d16/fc2 x:0 0 0 2021-11-03T17:14:39.456 INFO:tasks.workunit.client.0.smithi027.stdout:6/656: truncate dc/d16/f2c 4128031 0 2021-11-03T17:14:39.456 INFO:tasks.workunit.client.0.smithi027.stdout:6/657: dread - dc/d82/d87/fb9 zero size 2021-11-03T17:14:39.457 INFO:tasks.workunit.client.0.smithi027.stdout:6/658: creat dc/d82/d57/fc3 x:0 0 0 2021-11-03T17:14:39.457 INFO:tasks.workunit.client.0.smithi027.stdout:6/659: write dc/d82/d57/f97 [324246,107307] 0 2021-11-03T17:14:39.458 INFO:tasks.workunit.client.0.smithi027.stdout:6/660: write dc/d82/d57/dad/dae/db1/d34/f4e [3700057,52551] 0 2021-11-03T17:14:39.458 INFO:tasks.workunit.client.0.smithi027.stdout:6/661: chown dc/d16/d9a 50590331 1 2021-11-03T17:14:39.463 INFO:tasks.workunit.client.0.smithi027.stdout:6/662: dread dc/d82/d57/fbc [0,4194304] 0 2021-11-03T17:14:39.463 INFO:tasks.workunit.client.0.smithi027.stdout:6/663: fsync dc/d16/d9a/fba 0 2021-11-03T17:14:39.464 INFO:tasks.workunit.client.0.smithi027.stdout:6/664: creat dc/d82/d57/d9b/fc4 x:0 0 0 2021-11-03T17:14:39.465 INFO:tasks.workunit.client.0.smithi027.stdout:6/665: creat dc/d82/d57/fc5 x:0 0 0 2021-11-03T17:14:39.466 INFO:tasks.workunit.client.0.smithi027.stdout:6/666: write dc/d82/d57/f91 [1994031,14356] 0 2021-11-03T17:14:39.467 INFO:tasks.workunit.client.0.smithi027.stdout:6/667: truncate dc/f4d 245045 0 2021-11-03T17:14:39.469 INFO:tasks.workunit.client.0.smithi027.stdout:6/668: creat dc/d16/db2/fc6 x:0 0 0 2021-11-03T17:14:39.487 INFO:tasks.workunit.client.0.smithi027.stdout:2/700: sync 2021-11-03T17:14:39.487 INFO:tasks.workunit.client.0.smithi027.stdout:3/802: sync 2021-11-03T17:14:39.488 INFO:tasks.workunit.client.0.smithi027.stdout:3/803: dread d1/db/d52/d6a/d7c/d88/d98/fbc [0,4194304] 0 2021-11-03T17:14:39.489 INFO:tasks.workunit.client.0.smithi027.stdout:2/701: creat d0/d1b/d29/d54/d64/daa/d61/fc7 x:0 0 0 2021-11-03T17:14:39.490 INFO:tasks.workunit.client.0.smithi027.stdout:6/669: dwrite dc/d16/f40 [0,4194304] 0 2021-11-03T17:14:39.491 INFO:tasks.workunit.client.0.smithi027.stdout:3/804: symlink d1/dec/lfd 0 2021-11-03T17:14:39.493 INFO:tasks.workunit.client.0.smithi027.stdout:6/670: creat dc/d16/d9a/fc7 x:0 0 0 2021-11-03T17:14:39.493 INFO:tasks.workunit.client.0.smithi027.stdout:6/671: chown dc/d16/d30/f32 10 1 2021-11-03T17:14:39.495 INFO:tasks.workunit.client.0.smithi027.stdout:3/805: rename d1/db/fd4 to d1/db/db2/dbd/ffe 0 2021-11-03T17:14:39.496 INFO:tasks.workunit.client.0.smithi027.stdout:2/702: dread d0/d1b/d29/d54/d64/daa/f66 [4194304,4194304] 0 2021-11-03T17:14:39.497 INFO:tasks.workunit.client.0.smithi027.stdout:3/806: symlink d1/db/d52/d57/d99/da7/lff 0 2021-11-03T17:14:39.498 INFO:tasks.workunit.client.0.smithi027.stdout:2/703: mkdir d0/d5/d2c/dc8 0 2021-11-03T17:14:39.501 INFO:tasks.workunit.client.0.smithi027.stdout:3/807: rename d1/l2c to d1/db/d52/d6a/dd5/l100 0 2021-11-03T17:14:39.502 INFO:tasks.workunit.client.0.smithi027.stdout:3/808: creat d1/d40/f101 x:0 0 0 2021-11-03T17:14:39.504 INFO:tasks.workunit.client.0.smithi027.stdout:3/809: mkdir d1/db/d52/d57/d99/da7/dbf/d102 0 2021-11-03T17:14:39.505 INFO:tasks.workunit.client.0.smithi027.stdout:3/810: creat d1/db/d52/d6a/db4/f103 x:0 0 0 2021-11-03T17:14:39.506 INFO:tasks.workunit.client.0.smithi027.stdout:2/704: dread d0/d1b/d29/d58/f5c [0,4194304] 0 2021-11-03T17:14:39.511 INFO:tasks.workunit.client.0.smithi027.stdout:2/705: rename d0/d1b/d29/d54/d67/d6a to d0/da7/dc9 0 2021-11-03T17:14:39.511 INFO:tasks.workunit.client.0.smithi027.stdout:2/706: getdents d0/d1b/d29/d92/dbd 0 2021-11-03T17:14:39.513 INFO:tasks.workunit.client.0.smithi027.stdout:2/707: rmdir d0/d1b/d1c 39 2021-11-03T17:14:39.513 INFO:tasks.workunit.client.0.smithi027.stdout:2/708: chown d0/d1b/d29/d54/d67/f6c 90 1 2021-11-03T17:14:39.513 INFO:tasks.workunit.client.0.smithi027.stdout:2/709: mknod d0/d1b/d29/d54/d64/daa/d88/cca 1 2021-11-03T17:14:39.514 INFO:tasks.workunit.client.0.smithi027.stdout:2/710: write d0/d1b/d29/d54/d64/daa/f66 [8625039,97703] 0 2021-11-03T17:14:39.514 INFO:tasks.workunit.client.0.smithi027.stdout:2/711: fsync d0/d1b/db6/fbe 0 2021-11-03T17:14:39.514 INFO:tasks.workunit.client.0.smithi027.stdout:6/672: dwrite dc/d37/fbe [0,4194304] 0 2021-11-03T17:14:39.514 INFO:tasks.workunit.client.0.smithi027.stdout:6/673: fdatasync dc/d16/f7a 0 2021-11-03T17:14:39.515 INFO:tasks.workunit.client.0.smithi027.stdout:2/712: creat d0/d1b/d29/fcb x:0 0 0 2021-11-03T17:14:39.517 INFO:tasks.workunit.client.0.smithi027.stdout:6/674: rename dc/d82/da9 to dc/d82/d57/d9b/dc8 0 2021-11-03T17:14:39.517 INFO:tasks.workunit.client.0.smithi027.stdout:6/675: mknod dc/d82/d57/dad/dae/db1/d34/d89/cc9 1 2021-11-03T17:14:39.518 INFO:tasks.workunit.client.0.smithi027.stdout:6/676: mknod dc/d82/d87/cca 1 2021-11-03T17:14:39.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:39 smithi027 conmon[32322]: cluster 2021-11-03T17:14:37.895372+0000 mgr.smithi115.tcyodz (mgr.24395) 22 2021-11-03T17:14:39.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:39 smithi027 conmon[32322]: : cluster [DBG] pgmap v14: 65 pgs: 65 active+clean; 2.8 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 18 MiB/s rd, 31 MiB/s wr, 75 op/s 2021-11-03T17:14:39.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:39 smithi027 conmon[32322]: audit 2021-11-03T17:14:38.260681+0000 mon.smithi027 (mon.0) 628 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' 2021-11-03T17:14:39.519 INFO:tasks.workunit.client.0.smithi027.stdout:2/713: mkdir d0/d1b/d29/db5/dcc 0 2021-11-03T17:14:39.520 INFO:tasks.workunit.client.0.smithi027.stdout:6/677: truncate dc/f7b 4218159 0 2021-11-03T17:14:39.520 INFO:tasks.workunit.client.0.smithi027.stdout:6/678: write dc/d16/d1b/d3c/d51/f70 [2413019,15577] 0 2021-11-03T17:14:39.521 INFO:tasks.workunit.client.0.smithi027.stdout:2/714: link d0/f10 d0/d1b/d29/fcd 0 2021-11-03T17:14:39.522 INFO:tasks.workunit.client.0.smithi027.stdout:2/715: stat d0/d45/lc2 0 2021-11-03T17:14:39.522 INFO:tasks.workunit.client.0.smithi027.stdout:6/679: symlink dc/d82/d57/d96/lcb 0 2021-11-03T17:14:39.530 INFO:tasks.workunit.client.0.smithi027.stdout:3/811: dwrite d1/db/ff8 [0,4194304] 0 2021-11-03T17:14:39.537 INFO:tasks.workunit.client.0.smithi027.stdout:6/680: dread dc/d82/d57/dad/dae/db1/d34/f4e [0,4194304] 0 2021-11-03T17:14:39.539 INFO:tasks.workunit.client.0.smithi027.stdout:2/716: dread d0/d45/f9a [0,4194304] 0 2021-11-03T17:14:39.539 INFO:tasks.workunit.client.0.smithi027.stdout:2/717: truncate d0/d1b/db6/fbe 55567 0 2021-11-03T17:14:39.540 INFO:tasks.workunit.client.0.smithi027.stdout:2/718: chown d0/d1b/d29/d54/d64/daa/f21 4173674 1 2021-11-03T17:14:39.556 INFO:tasks.workunit.client.0.smithi027.stdout:2/719: dwrite d0/d1b/d1c/f33 [0,4194304] 0 2021-11-03T17:14:39.566 INFO:tasks.workunit.client.0.smithi027.stdout:2/720: dread d0/d1b/d1c/f26 [0,4194304] 0 2021-11-03T17:14:39.567 INFO:tasks.workunit.client.0.smithi027.stdout:2/721: creat d0/d1b/d29/db5/fce x:0 0 0 2021-11-03T17:14:39.569 INFO:tasks.workunit.client.0.smithi027.stdout:3/812: dwrite d1/db/d52/d6a/d7c/d88/fc8 [0,4194304] 0 2021-11-03T17:14:39.570 INFO:tasks.workunit.client.0.smithi027.stdout:3/813: mkdir d1/db/d52/d57/d99/d104 0 2021-11-03T17:14:39.572 INFO:tasks.workunit.client.0.smithi027.stdout:3/814: truncate d1/db/f93 3613947 0 2021-11-03T17:14:39.572 INFO:tasks.workunit.client.0.smithi027.stdout:3/815: fdatasync d1/db/d52/d6a/d7c/d88/d98/fdf 0 2021-11-03T17:14:39.574 INFO:tasks.workunit.client.0.smithi027.stdout:3/816: dread d1/db/f3c [0,4194304] 0 2021-11-03T17:14:39.575 INFO:tasks.workunit.client.0.smithi027.stdout:3/817: getdents d1/db/d52/d6a/db4 0 2021-11-03T17:14:39.576 INFO:tasks.workunit.client.0.smithi027.stdout:3/818: readlink d1/d40/l7a 0 2021-11-03T17:14:39.576 INFO:tasks.workunit.client.0.smithi027.stdout:3/819: read d1/f5e [1667977,4954] 0 2021-11-03T17:14:39.578 INFO:tasks.workunit.client.0.smithi027.stdout:3/820: link d1/db/d52/d6a/d7c/d88/d98/fdf d1/db/d52/d6a/d7c/f105 0 2021-11-03T17:14:39.584 INFO:tasks.workunit.client.0.smithi027.stdout:3/821: getdents d1/db/d52/d6a/dd5 0 2021-11-03T17:14:39.586 INFO:tasks.workunit.client.0.smithi027.stdout:3/822: creat d1/db/db2/f106 x:0 0 0 2021-11-03T17:14:39.586 INFO:tasks.workunit.client.0.smithi027.stdout:2/722: dwrite d0/d1b/d29/d92/dbd/f80 [4194304,4194304] 0 2021-11-03T17:14:39.587 INFO:tasks.workunit.client.0.smithi027.stdout:2/723: dread - d0/d1b/d29/d92/dbd/fa6 zero size 2021-11-03T17:14:39.587 INFO:tasks.workunit.client.0.smithi027.stdout:2/724: readlink d0/l2d 0 2021-11-03T17:14:39.587 INFO:tasks.workunit.client.0.smithi027.stdout:2/725: chown d0/d45/f7b 259566 1 2021-11-03T17:14:39.588 INFO:tasks.workunit.client.0.smithi027.stdout:3/823: symlink d1/db/d52/d6a/d7c/d88/d98/l107 0 2021-11-03T17:14:39.589 INFO:tasks.workunit.client.0.smithi027.stdout:2/726: write d0/d5/f6d [4003065,32360] 0 2021-11-03T17:14:39.591 INFO:tasks.workunit.client.0.smithi027.stdout:2/727: link d0/d5/faf d0/d1b/d29/d54/fcf 0 2021-11-03T17:14:39.598 INFO:tasks.workunit.client.0.smithi027.stdout:2/728: dread d0/d1b/d1c/f7e [0,4194304] 0 2021-11-03T17:14:39.598 INFO:tasks.workunit.client.0.smithi027.stdout:2/729: truncate d0/d7c/f9f 881309 0 2021-11-03T17:14:39.599 INFO:tasks.workunit.client.0.smithi027.stdout:2/730: creat d0/d1b/d1c/fd0 x:0 0 0 2021-11-03T17:14:39.616 INFO:tasks.workunit.client.0.smithi027.stdout:2/731: dwrite d0/d5/faf [0,4194304] 0 2021-11-03T17:14:39.619 INFO:tasks.workunit.client.0.smithi027.stdout:2/732: symlink d0/d1b/ld1 0 2021-11-03T17:14:39.620 INFO:tasks.workunit.client.0.smithi027.stdout:2/733: symlink d0/ld2 0 2021-11-03T17:14:39.623 INFO:tasks.workunit.client.0.smithi027.stdout:2/734: symlink d0/d1b/db6/ld3 0 2021-11-03T17:14:39.624 INFO:tasks.workunit.client.0.smithi027.stdout:2/735: mkdir d0/da7/dc9/dd4 0 2021-11-03T17:14:39.632 INFO:tasks.workunit.client.0.smithi027.stdout:2/736: dread d0/d1b/d29/d54/fcf [0,4194304] 0 2021-11-03T17:14:39.634 INFO:tasks.workunit.client.0.smithi027.stdout:2/737: rename d0/d1b/d29/d54/fc1 to d0/d45/fd5 0 2021-11-03T17:14:39.635 INFO:tasks.workunit.client.0.smithi027.stdout:2/738: chown d0/d1b/d1c/f7e 345 1 2021-11-03T17:14:39.636 INFO:tasks.workunit.client.0.smithi027.stdout:2/739: creat d0/d1b/d29/d58/fd6 x:0 0 0 2021-11-03T17:14:39.636 INFO:tasks.workunit.client.0.smithi027.stdout:2/740: write d0/d1b/d29/d54/fbb [681703,37253] 0 2021-11-03T17:14:39.637 INFO:tasks.workunit.client.0.smithi027.stdout:2/741: mkdir d0/d1b/d29/dd7 0 2021-11-03T17:14:39.637 INFO:tasks.workunit.client.0.smithi027.stdout:2/742: mknod d0/d1b/d29/dd7/cd8 1 2021-11-03T17:14:39.655 INFO:tasks.workunit.client.0.smithi027.stdout:2/743: dwrite d0/d5/faf [0,4194304] 0 2021-11-03T17:14:39.656 INFO:tasks.workunit.client.0.smithi027.stdout:2/744: mknod d0/d1b/d1c/cd9 1 2021-11-03T17:14:39.680 INFO:tasks.workunit.client.0.smithi027.stdout:2/745: dwrite d0/f9b [0,4194304] 0 2021-11-03T17:14:39.681 INFO:tasks.workunit.client.0.smithi027.stdout:2/746: dread - d0/d1b/d29/d54/fae zero size 2021-11-03T17:14:39.681 INFO:tasks.workunit.client.0.smithi027.stdout:2/747: readlink d0/d45/lb7 0 2021-11-03T17:14:39.685 INFO:tasks.workunit.client.0.smithi027.stdout:2/748: rename d0/d1b/d1c/f26 to d0/d1b/d29/fda 0 2021-11-03T17:14:39.687 INFO:tasks.workunit.client.0.smithi027.stdout:2/749: unlink d0/d1b/d29/d58/l6f 0 2021-11-03T17:14:39.688 INFO:tasks.workunit.client.0.smithi027.stdout:2/750: creat d0/d1b/db6/fdb x:0 0 0 2021-11-03T17:14:39.689 INFO:tasks.workunit.client.0.smithi027.stdout:2/751: write d0/d1b/d29/fcd [686331,61866] 0 2021-11-03T17:14:39.690 INFO:tasks.workunit.client.0.smithi027.stdout:2/752: mknod d0/d1b/d29/d54/cdc 1 2021-11-03T17:14:39.690 INFO:tasks.workunit.client.0.smithi027.stdout:2/753: write d0/d1b/d29/d54/d64/daa/f94 [1030895,57820] 0 2021-11-03T17:14:39.690 INFO:tasks.workunit.client.0.smithi027.stdout:2/754: chown d0/da7/dc9/l73 72667 1 2021-11-03T17:14:39.691 INFO:tasks.workunit.client.0.smithi027.stdout:2/755: dread - d0/d1b/d29/fcb zero size 2021-11-03T17:14:39.711 INFO:tasks.workunit.client.0.smithi027.stdout:2/756: dwrite d0/d1b/f6e [0,4194304] 0 2021-11-03T17:14:39.712 INFO:tasks.workunit.client.0.smithi027.stdout:2/757: symlink d0/d1b/d29/dd7/ldd 0 2021-11-03T17:14:39.714 INFO:tasks.workunit.client.0.smithi027.stdout:2/758: dread d0/d7c/f9f [0,4194304] 0 2021-11-03T17:14:39.717 INFO:tasks.workunit.client.0.smithi027.stdout:2/759: creat d0/d1b/d29/d92/dbd/fde x:0 0 0 2021-11-03T17:14:39.719 INFO:tasks.workunit.client.0.smithi027.stdout:2/760: symlink d0/d5/d2c/ldf 0 2021-11-03T17:14:39.719 INFO:tasks.workunit.client.0.smithi027.stdout:2/761: chown d0/d1b/db6 2760626 1 2021-11-03T17:14:39.720 INFO:tasks.workunit.client.0.smithi027.stdout:2/762: symlink d0/d5/le0 0 2021-11-03T17:14:39.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:39 smithi115 conmon[34670]: cluster 2021-11-03T17:14:37.895372+0000 mgr.smithi115.tcyodz (mgr. 2021-11-03T17:14:39.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:39 smithi115 conmon[34670]: 24395) 22 : cluster [DBG] pgmap v14: 65 pgs: 65 active+clean; 2.8 GiB data, 8.4 GiB used, 528 GiB / 536 GiB avail; 18 MiB/s rd, 31 MiB/s wr, 75 op/s 2021-11-03T17:14:39.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:39 smithi115 conmon[34670]: audit 2021-11-03T17:14:38.260681+0000 mon.smithi027 (mon.0) 628 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' 2021-11-03T17:14:41.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:41 smithi027 conmon[32322]: cluster 2021-11-03T17:14:39.895898+0000 mgr.smithi115.tcyodz (mgr.24395) 23 : 2021-11-03T17:14:41.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:41 smithi027 conmon[32322]: cluster [DBG] pgmap v15: 65 pgs: 65 active+clean; 2.9 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail; 46 MiB/s rd, 56 MiB/s wr, 100 op/s 2021-11-03T17:14:41.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:41 smithi027 conmon[32322]: cluster 2021-11-03T17:14:40.267651+0000 mon.smithi027 (mon.0) 629 : cluster [DBG] mds.? [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] up:standby 2021-11-03T17:14:41.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:41 smithi027 conmon[32322]: cluster 2021-11-03T 2021-11-03T17:14:41.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:41 smithi027 conmon[32322]: 17:14:40.267727+0000 mon.smithi027 (mon.0) 630 : cluster [DBG] fsmap cephfs:2/2 {0=cephfs.smithi027.fksixo=up:replay,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:14:41.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:41 smithi115 conmon[34670]: cluster 2021-11-03T17:14:39.895898+0000 mgr.smithi115.tcyodz (mgr.24395) 23 : cluster [DBG] pgmap v15: 65 pgs: 65 active+clean; 2.9 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail; 46 MiB/s rd, 56 MiB/s wr, 100 op/s 2021-11-03T17:14:41.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:41 smithi115 conmon[34670]: cluster 2021-11-03T17:14:40.267651+0000 mon.smithi027 (mon.0) 629 : cluster [DBG] mds.? [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] up:standby 2021-11-03T17:14:41.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:41 smithi115 conmon[34670]: cluster 2021-11-03T17:14:40.267727+0000 mon.smithi027 (mon.0) 630 : cluster [DBG] fsmap cephfs:2/2 {0=cephfs.smithi027.fksixo=up:replay,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:14:42.611 INFO:tasks.workunit.client.0.smithi027.stdout:7/531: sync 2021-11-03T17:14:42.612 INFO:tasks.workunit.client.0.smithi027.stdout:7/532: fsync d0/d2/d10/d1e/d20/f5b 0 2021-11-03T17:14:42.629 INFO:tasks.workunit.client.0.smithi027.stdout:7/533: dwrite d0/d2/d5e/d36/fb0 [0,4194304] 0 2021-11-03T17:14:42.630 INFO:tasks.workunit.client.0.smithi027.stdout:7/534: chown d0/d2/d10/d35/f7a 0 1 2021-11-03T17:14:42.630 INFO:tasks.workunit.client.0.smithi027.stdout:7/535: truncate d0/d2d/d58/f69 3168170 0 2021-11-03T17:14:42.632 INFO:tasks.workunit.client.0.smithi027.stdout:7/536: mkdir d0/db6 0 2021-11-03T17:14:42.634 INFO:tasks.workunit.client.0.smithi027.stdout:7/537: symlink d0/d2/d1c/d4d/d6e/lb7 0 2021-11-03T17:14:42.641 INFO:tasks.workunit.client.0.smithi027.stdout:7/538: symlink d0/d2/d10/d1e/lb8 0 2021-11-03T17:14:42.641 INFO:tasks.workunit.client.0.smithi027.stdout:7/539: fsync d0/d2/d1c/f2b 0 2021-11-03T17:14:42.659 INFO:tasks.workunit.client.0.smithi027.stdout:7/540: dwrite d0/d2/d10/d35/f7a [0,4194304] 0 2021-11-03T17:14:42.662 INFO:tasks.workunit.client.0.smithi027.stdout:7/541: unlink d0/d2/d10/d35/l37 0 2021-11-03T17:14:42.663 INFO:tasks.workunit.client.0.smithi027.stdout:7/542: creat d0/d2d/d2f/fb9 x:0 0 0 2021-11-03T17:14:42.663 INFO:tasks.workunit.client.0.smithi027.stdout:7/543: dread - d0/d2/d10/d35/d96/fb3 zero size 2021-11-03T17:14:42.670 INFO:tasks.workunit.client.0.smithi027.stdout:7/544: dread d0/d2d/d53/f65 [0,4194304] 0 2021-11-03T17:14:42.679 INFO:tasks.workunit.client.0.smithi027.stdout:7/545: dread d0/d2/d10/d1e/d20/d74/f7b [0,4194304] 0 2021-11-03T17:14:42.679 INFO:tasks.workunit.client.0.smithi027.stdout:7/546: chown d0/d2/d10/d1e/l21 200381100 1 2021-11-03T17:14:42.696 INFO:tasks.workunit.client.0.smithi027.stdout:7/547: dwrite d0/d2/d5e/f32 [0,4194304] 0 2021-11-03T17:14:42.697 INFO:tasks.workunit.client.0.smithi027.stdout:7/548: mkdir d0/d2d/d58/dba 0 2021-11-03T17:14:42.699 INFO:tasks.workunit.client.0.smithi027.stdout:7/549: getdents d0/d2 0 2021-11-03T17:14:42.699 INFO:tasks.workunit.client.0.smithi027.stdout:7/550: chown d0/d2/d1c/d4d/d6e/d84/f92 17361129 1 2021-11-03T17:14:42.699 INFO:tasks.workunit.client.0.smithi027.stdout:7/551: write d0/d2/d1c/faf [91452,92932] 0 2021-11-03T17:14:42.699 INFO:tasks.workunit.client.0.smithi027.stdout:7/552: write d0/d2d/d2f/fb9 [566800,58807] 0 2021-11-03T17:14:42.702 INFO:tasks.workunit.client.0.smithi027.stdout:7/553: write d0/d2d/d53/fa0 [832698,61676] 0 2021-11-03T17:14:42.702 INFO:tasks.workunit.client.0.smithi027.stdout:7/554: readlink d0/d2/d10/l2e 0 2021-11-03T17:14:42.704 INFO:tasks.workunit.client.0.smithi027.stdout:7/555: symlink d0/d2d/lbb 0 2021-11-03T17:14:42.704 INFO:tasks.workunit.client.0.smithi027.stdout:7/556: chown d0/l55 0 1 2021-11-03T17:14:42.705 INFO:tasks.workunit.client.0.smithi027.stdout:7/557: write d0/f25 [3246550,111168] 0 2021-11-03T17:14:42.705 INFO:tasks.workunit.client.0.smithi027.stdout:7/558: read d0/d2/d1c/f27 [2115001,102502] 0 2021-11-03T17:14:42.705 INFO:tasks.workunit.client.0.smithi027.stdout:7/559: write d0/d2d/d2f/fb9 [1133376,104038] 0 2021-11-03T17:14:42.709 INFO:tasks.workunit.client.0.smithi027.stdout:9/690: sync 2021-11-03T17:14:42.709 INFO:tasks.workunit.client.0.smithi027.stdout:1/736: sync 2021-11-03T17:14:42.709 INFO:tasks.workunit.client.0.smithi027.stdout:1/737: truncate d6/d11/dac/d79/d25/d28/dec/fe9 643231 0 2021-11-03T17:14:42.709 INFO:tasks.workunit.client.0.smithi027.stdout:1/738: dread - d6/d11/de2/ffb zero size 2021-11-03T17:14:42.712 INFO:tasks.workunit.client.0.smithi027.stdout:9/691: stat db/d1b/d2b/d41/l58 0 2021-11-03T17:14:42.713 INFO:tasks.workunit.client.0.smithi027.stdout:7/560: dread d0/d2/d10/d1e/f5c [0,4194304] 0 2021-11-03T17:14:42.714 INFO:tasks.workunit.client.0.smithi027.stdout:1/739: creat d6/d11/ffc x:0 0 0 2021-11-03T17:14:42.719 INFO:tasks.workunit.client.0.smithi027.stdout:1/740: link d6/d11/d14/f35 d6/de/ffd 0 2021-11-03T17:14:42.721 INFO:tasks.workunit.client.0.smithi027.stdout:1/741: symlink d6/d11/d14/df6/lfe 0 2021-11-03T17:14:42.724 INFO:tasks.workunit.client.0.smithi027.stdout:7/561: fsync d0/d2d/d53/fa0 0 2021-11-03T17:14:42.725 INFO:tasks.workunit.client.0.smithi027.stdout:1/742: rename d6/d11/dac/d79/d25/d28/d32 to d6/d11/dac/d79/db0/de5/dff 0 2021-11-03T17:14:42.726 INFO:tasks.workunit.client.0.smithi027.stdout:1/743: stat d6/d11/dac/d79/d37/de0/f16 0 2021-11-03T17:14:42.735 INFO:tasks.workunit.client.0.smithi027.stdout:9/692: dwrite db/d1b/d1c/fe6 [0,4194304] 0 2021-11-03T17:14:42.736 INFO:tasks.workunit.client.0.smithi027.stdout:7/562: dread d0/d2/d5e/f32 [0,4194304] 0 2021-11-03T17:14:42.744 INFO:tasks.workunit.client.0.smithi027.stdout:1/744: dread d6/d11/dac/d79/f99 [0,4194304] 0 2021-11-03T17:14:42.768 INFO:tasks.workunit.client.0.smithi027.stdout:7/563: dwrite d0/d2/d1f/f54 [0,4194304] 0 2021-11-03T17:14:42.769 INFO:tasks.workunit.client.0.smithi027.stdout:9/693: dwrite db/f16 [4194304,4194304] 0 2021-11-03T17:14:42.770 INFO:tasks.workunit.client.0.smithi027.stdout:9/694: write db/d1b/d2b/d41/dad/fda [589946,27973] 0 2021-11-03T17:14:42.770 INFO:tasks.workunit.client.0.smithi027.stdout:1/745: dwrite d6/de/d13/f38 [0,4194304] 0 2021-11-03T17:14:42.770 INFO:tasks.workunit.client.0.smithi027.stdout:1/746: fdatasync d6/fa3 0 2021-11-03T17:14:42.771 INFO:tasks.workunit.client.0.smithi027.stdout:9/695: link db/lc1 db/d1b/d2b/d41/lfd 0 2021-11-03T17:14:42.772 INFO:tasks.workunit.client.0.smithi027.stdout:7/564: dread d0/d2d/d58/f69 [0,4194304] 0 2021-11-03T17:14:42.774 INFO:tasks.workunit.client.0.smithi027.stdout:1/747: symlink d6/d11/dac/d79/d25/d28/d5c/d7a/dee/l100 0 2021-11-03T17:14:42.778 INFO:tasks.workunit.client.0.smithi027.stdout:7/565: creat d0/d2d/fbc x:0 0 0 2021-11-03T17:14:42.779 INFO:tasks.workunit.client.0.smithi027.stdout:1/748: creat d6/d11/f101 x:0 0 0 2021-11-03T17:14:42.781 INFO:tasks.workunit.client.0.smithi027.stdout:7/566: rmdir d0/d2/d5e/d75 0 2021-11-03T17:14:42.782 INFO:tasks.workunit.client.0.smithi027.stdout:1/749: rmdir d6/d11/dac/d79/d25 39 2021-11-03T17:14:42.782 INFO:tasks.workunit.client.0.smithi027.stdout:1/750: truncate d6/d11/dac/d79/d37/de0/d43/f5d 733395 0 2021-11-03T17:14:42.784 INFO:tasks.workunit.client.0.smithi027.stdout:9/696: write db/d1b/d23/ffb [1182971,73101] 0 2021-11-03T17:14:42.787 INFO:tasks.workunit.client.0.smithi027.stdout:1/751: rename d6/de/d13 to d6/d11/dac/d79/db0/d102 0 2021-11-03T17:14:42.797 INFO:tasks.workunit.client.0.smithi027.stdout:9/697: dread db/d1b/d2b/f3f [4194304,4194304] 0 2021-11-03T17:14:42.797 INFO:tasks.workunit.client.0.smithi027.stdout:9/698: chown db/d1b/d8c 13 1 2021-11-03T17:14:42.799 INFO:tasks.workunit.client.0.smithi027.stdout:9/699: creat db/d1b/d8c/dcd/ffe x:0 0 0 2021-11-03T17:14:42.802 INFO:tasks.workunit.client.0.smithi027.stdout:9/700: unlink db/d1b/d8e/fa5 0 2021-11-03T17:14:42.802 INFO:tasks.workunit.client.0.smithi027.stdout:9/701: fsync db/d1b/d23/d2c/f89 0 2021-11-03T17:14:42.802 INFO:tasks.workunit.client.0.smithi027.stdout:9/702: mknod db/d1b/d2b/dc7/cff 1 2021-11-03T17:14:42.804 INFO:tasks.workunit.client.0.smithi027.stdout:9/703: creat db/d1b/d2b/d41/d95/d9f/f100 x:0 0 0 2021-11-03T17:14:42.805 INFO:tasks.workunit.client.0.smithi027.stdout:9/704: symlink db/d1b/d1c/d1d/d40/dcc/l101 0 2021-11-03T17:14:42.805 INFO:tasks.workunit.client.0.smithi027.stdout:9/705: truncate db/d1b/f38 4983928 0 2021-11-03T17:14:42.806 INFO:tasks.workunit.client.0.smithi027.stdout:9/706: creat db/d1b/d1c/f102 x:0 0 0 2021-11-03T17:14:42.808 INFO:tasks.workunit.client.0.smithi027.stdout:1/752: dwrite d6/d11/dac/d79/d37/de0/f42 [0,4194304] 0 2021-11-03T17:14:42.809 INFO:tasks.workunit.client.0.smithi027.stdout:1/753: rmdir d6/d11/dac/d79/dab/db4 39 2021-11-03T17:14:42.816 INFO:tasks.workunit.client.0.smithi027.stdout:9/707: dread db/d1b/f45 [0,4194304] 0 2021-11-03T17:14:42.818 INFO:tasks.workunit.client.0.smithi027.stdout:9/708: link db/d1b/d2b/d41/d97/lb1 db/d1b/d2b/d41/dad/l103 0 2021-11-03T17:14:42.819 INFO:tasks.workunit.client.0.smithi027.stdout:9/709: mkdir db/d1b/d1c/d1d/d40/d104 0 2021-11-03T17:14:42.821 INFO:tasks.workunit.client.0.smithi027.stdout:9/710: rmdir db/d1b/d1c/d1d/ddb 39 2021-11-03T17:14:42.821 INFO:tasks.workunit.client.0.smithi027.stdout:9/711: stat db/d1b/d8e 0 2021-11-03T17:14:42.823 INFO:tasks.workunit.client.0.smithi027.stdout:9/712: dread db/f1f [0,4194304] 0 2021-11-03T17:14:42.824 INFO:tasks.workunit.client.0.smithi027.stdout:9/713: symlink db/d1b/d2b/d41/l105 0 2021-11-03T17:14:42.825 INFO:tasks.workunit.client.0.smithi027.stdout:9/714: mkdir db/d1b/d106 0 2021-11-03T17:14:42.826 INFO:tasks.workunit.client.0.smithi027.stdout:9/715: mkdir db/d107 0 2021-11-03T17:14:42.826 INFO:tasks.workunit.client.0.smithi027.stdout:9/716: fsync db/d1b/d2b/d41/f4d 0 2021-11-03T17:14:42.827 INFO:tasks.workunit.client.0.smithi027.stdout:9/717: creat db/d1b/d2b/f108 x:0 0 0 2021-11-03T17:14:42.827 INFO:tasks.workunit.client.0.smithi027.stdout:9/718: readlink db/d1b/d23/l80 0 2021-11-03T17:14:42.830 INFO:tasks.workunit.client.0.smithi027.stdout:9/719: rename db/d1b/d1c/d1d/f78 to db/d1b/d2b/d5e/daa/f109 0 2021-11-03T17:14:42.830 INFO:tasks.workunit.client.0.smithi027.stdout:9/720: fdatasync db/fd 0 2021-11-03T17:14:42.830 INFO:tasks.workunit.client.0.smithi027.stdout:9/721: write db/d1b/fb2 [733800,16948] 0 2021-11-03T17:14:42.832 INFO:tasks.workunit.client.0.smithi027.stdout:1/754: dwrite d6/d11/dac/d79/d25/d28/dec/fe9 [0,4194304] 0 2021-11-03T17:14:42.832 INFO:tasks.workunit.client.0.smithi027.stdout:1/755: truncate d6/de/faf 939292 0 2021-11-03T17:14:42.874 INFO:tasks.workunit.client.0.smithi027.stdout:9/722: creat db/d1b/d1c/d1d/d40/f10a x:0 0 0 2021-11-03T17:14:42.895 INFO:tasks.workunit.client.0.smithi027.stdout:1/756: write d6/d11/f27 [1438497,24844] 0 2021-11-03T17:14:42.896 INFO:tasks.workunit.client.0.smithi027.stdout:1/757: read - d6/d11/dac/d79/d37/d49/fcb zero size 2021-11-03T17:14:42.896 INFO:tasks.workunit.client.0.smithi027.stdout:1/758: truncate d6/d11/dac/d79/d37/de0/de4/f34 652497 0 2021-11-03T17:14:42.896 INFO:tasks.workunit.client.0.smithi027.stdout:1/759: chown d6/d11/dac/f6d 128468479 1 2021-11-03T17:14:42.897 INFO:tasks.workunit.client.0.smithi027.stdout:1/760: write d6/d11/dac/d79/d25/d28/d5c/f78 [6664617,107718] 0 2021-11-03T17:14:42.913 INFO:tasks.workunit.client.0.smithi027.stdout:1/761: dwrite d6/d11/dac/d79/d25/d28/d5c/d81/f86 [4194304,4194304] 0 2021-11-03T17:14:42.915 INFO:tasks.workunit.client.0.smithi027.stdout:1/762: link d6/d11/dac/d79/d37/de0/l90 d6/d11/dac/d79/db0/d102/l103 0 2021-11-03T17:14:42.933 INFO:tasks.workunit.client.0.smithi027.stdout:1/763: dwrite d6/d11/dac/d79/d37/de0/de4/d58/fcd [0,4194304] 0 2021-11-03T17:14:42.934 INFO:tasks.workunit.client.0.smithi027.stdout:1/764: getdents d6/d11/dac/d79/dab/db4 0 2021-11-03T17:14:42.934 INFO:tasks.workunit.client.0.smithi027.stdout:1/765: chown d6/d11/dac/d79/db0/de5/dff/dd7/d8b 1 1 2021-11-03T17:14:42.935 INFO:tasks.workunit.client.0.smithi027.stdout:1/766: write d6/d11/dac/d79/f9d [1010556,44617] 0 2021-11-03T17:14:42.935 INFO:tasks.workunit.client.0.smithi027.stdout:1/767: truncate d6/d11/dac/d79/db0/de5/dff/f8e 3626319 0 2021-11-03T17:14:42.936 INFO:tasks.workunit.client.0.smithi027.stdout:1/768: stat d6/d11/dac/d79/d37/de0/d43/f5d 0 2021-11-03T17:14:42.936 INFO:tasks.workunit.client.0.smithi027.stdout:1/769: dread d6/d11/dac/d79/d37/d49/f50 [0,4194304] 0 2021-11-03T17:14:42.937 INFO:tasks.workunit.client.0.smithi027.stdout:1/770: dread d6/d11/dac/d79/d37/de0/de4/fa0 [0,4194304] 0 2021-11-03T17:14:42.954 INFO:tasks.workunit.client.0.smithi027.stdout:1/771: dwrite d6/d11/dac/d79/d25/f74 [0,4194304] 0 2021-11-03T17:14:42.956 INFO:tasks.workunit.client.0.smithi027.stdout:1/772: rmdir d6/d11/dac/d79/d37/de0/d4e 39 2021-11-03T17:14:42.957 INFO:tasks.workunit.client.0.smithi027.stdout:1/773: truncate d6/de/f19 6618199 0 2021-11-03T17:14:42.968 INFO:tasks.workunit.client.0.smithi027.stdout:1/774: truncate d6/d11/f27 127630 0 2021-11-03T17:14:42.969 INFO:tasks.workunit.client.0.smithi027.stdout:1/775: mkdir d6/d11/dac/d79/d104 0 2021-11-03T17:14:42.987 INFO:tasks.workunit.client.0.smithi027.stdout:1/776: dwrite d6/d11/dac/d79/d37/de0/de4/d58/fcd [0,4194304] 0 2021-11-03T17:14:42.989 INFO:tasks.workunit.client.0.smithi027.stdout:1/777: truncate d6/d11/dac/d79/d25/d28/fd4 1197313 0 2021-11-03T17:14:42.990 INFO:tasks.workunit.client.0.smithi027.stdout:1/778: chown d6/d11/dac/d79/d25/d28/d5c/d81/l52 8265716 1 2021-11-03T17:14:42.990 INFO:tasks.workunit.client.0.smithi027.stdout:1/779: fdatasync d6/d11/dac/d79/db0/de5/dff/f84 0 2021-11-03T17:14:42.994 INFO:tasks.workunit.client.0.smithi027.stdout:1/780: write d6/d11/f89 [1227191,100251] 0 2021-11-03T17:14:42.995 INFO:tasks.workunit.client.0.smithi027.stdout:1/781: chown d6/d11/dac/d79/f63 645255 1 2021-11-03T17:14:42.995 INFO:tasks.workunit.client.0.smithi027.stdout:1/782: rmdir d6/d11/dac/d79/d25/d28 39 2021-11-03T17:14:42.997 INFO:tasks.workunit.client.0.smithi027.stdout:1/783: link d6/d11/dac/d79/d37/l3d d6/d11/l105 0 2021-11-03T17:14:42.998 INFO:tasks.workunit.client.0.smithi027.stdout:1/784: write d6/d11/dac/f6d [353616,95912] 0 2021-11-03T17:14:42.998 INFO:tasks.workunit.client.0.smithi027.stdout:1/785: chown d6/d11/dac/d79/d25/d28/l39 121610269 1 2021-11-03T17:14:42.998 INFO:tasks.workunit.client.0.smithi027.stdout:1/786: fsync d6/d11/dac/d79/d25/d28/dec/fd0 0 2021-11-03T17:14:42.998 INFO:tasks.workunit.client.0.smithi027.stdout:1/787: readlink d6/d11/dac/d79/db0/d102/le7 0 2021-11-03T17:14:42.999 INFO:tasks.workunit.client.0.smithi027.stdout:1/788: chown d6/d11/dac/d79/d25/d28/d41/fd8 41519 1 2021-11-03T17:14:42.999 INFO:tasks.workunit.client.0.smithi027.stdout:1/789: chown d6/fa3 3599508 1 2021-11-03T17:14:42.999 INFO:tasks.workunit.client.0.smithi027.stdout:1/790: rename d6/l3e to d6/d11/dac/d79/d25/d5e/l106 0 2021-11-03T17:14:43.028 INFO:tasks.workunit.client.0.smithi027.stdout:1/791: dwrite d6/d11/dac/d79/db0/d102/f66 [0,4194304] 0 2021-11-03T17:14:43.029 INFO:tasks.workunit.client.0.smithi027.stdout:1/792: readlink d6/d11/dac/d79/d25/d28/d5c/d81/ldd 0 2021-11-03T17:14:43.029 INFO:tasks.workunit.client.0.smithi027.stdout:1/793: chown d6/d11/d14/f35 5 1 2021-11-03T17:14:43.029 INFO:tasks.workunit.client.0.smithi027.stdout:1/794: dread d6/d11/dac/d79/d37/d49/f50 [0,4194304] 0 2021-11-03T17:14:43.096 INFO:tasks.workunit.client.0.smithi027.stdout:1/795: dwrite d6/de/faf [0,4194304] 0 2021-11-03T17:14:43.097 INFO:tasks.workunit.client.0.smithi027.stdout:1/796: chown d6/de/ffd 175936 1 2021-11-03T17:14:43.098 INFO:tasks.workunit.client.0.smithi027.stdout:1/797: write d6/d11/dac/d79/d37/de0/de4/f8a [1983461,18245] 0 2021-11-03T17:14:43.098 INFO:tasks.workunit.client.0.smithi027.stdout:1/798: read - d6/d11/dac/d79/d25/d5e/d72/fc2 zero size 2021-11-03T17:14:43.099 INFO:tasks.workunit.client.0.smithi027.stdout:1/799: creat d6/d11/dac/d79/d25/d28/f107 x:0 0 0 2021-11-03T17:14:43.101 INFO:tasks.workunit.client.0.smithi027.stdout:1/800: symlink d6/d11/dac/d79/l108 0 2021-11-03T17:14:43.102 INFO:tasks.workunit.client.0.smithi027.stdout:1/801: rmdir d6/d11/dac/d79/d37/de0/d4e 39 2021-11-03T17:14:43.105 INFO:tasks.workunit.client.0.smithi027.stdout:1/802: link d6/d11/dac/d79/d37/de0/d4e/f91 d6/d11/dac/d79/d25/d5e/f109 0 2021-11-03T17:14:43.105 INFO:tasks.workunit.client.0.smithi027.stdout:1/803: getdents d6/d11/dac/d79/dfa 0 2021-11-03T17:14:43.105 INFO:tasks.workunit.client.0.smithi027.stdout:1/804: chown f2 3297 1 2021-11-03T17:14:43.105 INFO:tasks.workunit.client.0.smithi027.stdout:1/805: chown d6/d11/dac/d79/db0/d102/f6b 1779 1 2021-11-03T17:14:43.106 INFO:tasks.workunit.client.0.smithi027.stdout:1/806: write d6/d11/dac/d79/dab/db4/fc5 [1013934,42345] 0 2021-11-03T17:14:43.106 INFO:tasks.workunit.client.0.smithi027.stdout:1/807: stat d6/d11/dac/d79/d25/d28/dec/fd0 0 2021-11-03T17:14:43.106 INFO:tasks.workunit.client.0.smithi027.stdout:1/808: mknod d6/d11/dac/d79/d37/de0/d4e/dbd/c10a 1 2021-11-03T17:14:43.107 INFO:tasks.workunit.client.0.smithi027.stdout:1/809: rmdir d6/d11/dac/d79/d25/d5e/d72 39 2021-11-03T17:14:43.107 INFO:tasks.workunit.client.0.smithi027.stdout:1/810: chown d6/d11/dac/d79/d25/d28 311247676 1 2021-11-03T17:14:43.107 INFO:tasks.workunit.client.0.smithi027.stdout:1/811: dread d6/d11/dac/d79/d25/d28/d5c/d81/d7d/fca [0,4194304] 0 2021-11-03T17:14:43.109 INFO:tasks.workunit.client.0.smithi027.stdout:1/812: rename d6/d11/dac/d79/d37/de0/d4e/dbd to d6/d11/dac/d10b 0 2021-11-03T17:14:43.109 INFO:tasks.workunit.client.0.smithi027.stdout:1/813: getdents d6/d11/dac/d79/d37/d49 0 2021-11-03T17:14:43.119 INFO:tasks.workunit.client.0.smithi027.stdout:1/814: dread d6/f18 [0,4194304] 0 2021-11-03T17:14:43.121 INFO:tasks.workunit.client.0.smithi027.stdout:1/815: dread d6/d11/dac/d79/d37/de0/de4/d58/f70 [0,4194304] 0 2021-11-03T17:14:43.122 INFO:tasks.workunit.client.0.smithi027.stdout:1/816: stat d6/d11/dac/d79/db0/de5/dff/dd7/f6c 0 2021-11-03T17:14:43.122 INFO:tasks.workunit.client.0.smithi027.stdout:1/817: chown d6/d11/dac/d79/d25/d28/d5c/d81/l52 14 1 2021-11-03T17:14:43.124 INFO:tasks.workunit.client.0.smithi027.stdout:1/818: unlink d6/d11/de2/fed 0 2021-11-03T17:14:43.124 INFO:tasks.workunit.client.0.smithi027.stdout:1/819: mknod d6/d11/dac/d79/db0/c10c 1 2021-11-03T17:14:43.138 INFO:tasks.workunit.client.0.smithi027.stdout:3/824: sync 2021-11-03T17:14:43.138 INFO:tasks.workunit.client.0.smithi027.stdout:5/954: sync 2021-11-03T17:14:43.138 INFO:tasks.workunit.client.0.smithi027.stdout:0/855: sync 2021-11-03T17:14:43.138 INFO:tasks.workunit.client.0.smithi027.stdout:3/825: chown d1/db/d52/d57/f71 6 1 2021-11-03T17:14:43.139 INFO:tasks.workunit.client.0.smithi027.stdout:3/826: getdents d1/db/d52/d6a/d7c/d88 0 2021-11-03T17:14:43.139 INFO:tasks.workunit.client.0.smithi027.stdout:5/955: write d2/d2c/d9b/db3/fb4 [211560,60000] 0 2021-11-03T17:14:43.142 INFO:tasks.workunit.client.0.smithi027.stdout:1/820: dwrite d6/d11/dac/d79/d25/f47 [4194304,4194304] 0 2021-11-03T17:14:43.143 INFO:tasks.workunit.client.0.smithi027.stdout:5/956: symlink d2/d2c/d3c/d48/d65/d80/dd2/d120/l13d 0 2021-11-03T17:14:43.143 INFO:tasks.workunit.client.0.smithi027.stdout:5/957: mknod d2/d1f/c13e 1 2021-11-03T17:14:43.146 INFO:tasks.workunit.client.0.smithi027.stdout:5/958: creat d2/d4/f13f x:0 0 0 2021-11-03T17:14:43.146 INFO:tasks.workunit.client.0.smithi027.stdout:5/959: truncate d2/d4/fcc 1316488 0 2021-11-03T17:14:43.149 INFO:tasks.workunit.client.0.smithi027.stdout:3/827: dread d1/fb8 [0,4194304] 0 2021-11-03T17:14:43.150 INFO:tasks.workunit.client.0.smithi027.stdout:5/960: rename d2/d4/d4e/d67/d20 to d2/d2c/d3c/d48/d65/d80/dd2/d120/d140 0 2021-11-03T17:14:43.158 INFO:tasks.workunit.client.0.smithi027.stdout:3/828: creat d1/db/d52/d57/d99/da7/dbf/f108 x:0 0 0 2021-11-03T17:14:43.158 INFO:tasks.workunit.client.0.smithi027.stdout:3/829: write d1/db/d52/d6a/dd5/ff0 [98502,65010] 0 2021-11-03T17:14:43.159 INFO:tasks.workunit.client.0.smithi027.stdout:5/961: rmdir d2/d12/dad/da8/dd9/dde 39 2021-11-03T17:14:43.159 INFO:tasks.workunit.client.0.smithi027.stdout:5/962: chown d2/d2c/d3c/d58/f138 717682 1 2021-11-03T17:14:43.169 INFO:tasks.workunit.client.0.smithi027.stdout:1/821: dwrite d6/d11/dac/d79/d37/de0/f4b [0,4194304] 0 2021-11-03T17:14:43.178 INFO:tasks.workunit.client.0.smithi027.stdout:1/822: rmdir d6/d11 39 2021-11-03T17:14:43.180 INFO:tasks.workunit.client.0.smithi027.stdout:3/830: dwrite d1/db/d52/d57/de3/ff4 [0,4194304] 0 2021-11-03T17:14:43.193 INFO:tasks.workunit.client.0.smithi027.stdout:3/831: dread d1/db/d52/d6a/d7c/fa9 [0,4194304] 0 2021-11-03T17:14:43.194 INFO:tasks.workunit.client.0.smithi027.stdout:3/832: dread - d1/db/d52/d6a/d7c/d88/d98/fe7 zero size 2021-11-03T17:14:43.194 INFO:tasks.workunit.client.0.smithi027.stdout:3/833: chown d1/d40/f76 22 1 2021-11-03T17:14:43.199 INFO:tasks.workunit.client.0.smithi027.stdout:1/823: dwrite d6/de/faf [0,4194304] 0 2021-11-03T17:14:43.208 INFO:tasks.workunit.client.0.smithi027.stdout:1/824: dread d6/d11/dac/d79/db0/de5/dff/dd7/fb3 [0,4194304] 0 2021-11-03T17:14:43.208 INFO:tasks.workunit.client.0.smithi027.stdout:1/825: fsync d6/d11/dac/d79/db0/de5/dff/f8e 0 2021-11-03T17:14:43.209 INFO:tasks.workunit.client.0.smithi027.stdout:3/834: dwrite d1/db/d52/d57/d8b/f9c [0,4194304] 0 2021-11-03T17:14:43.209 INFO:tasks.workunit.client.0.smithi027.stdout:3/835: truncate d1/db/db2/dbd/ffe 5827839 0 2021-11-03T17:14:43.214 INFO:tasks.workunit.client.0.smithi027.stdout:3/836: rename d1/db/db2/lf2 to d1/db/d52/d6a/db4/db5/d97/l109 0 2021-11-03T17:14:43.222 INFO:tasks.workunit.client.0.smithi027.stdout:3/837: rename d1/db/f1c to d1/db/f10a 0 2021-11-03T17:14:43.224 INFO:tasks.workunit.client.0.smithi027.stdout:3/838: symlink d1/db/d52/d6a/d7c/d88/l10b 0 2021-11-03T17:14:43.224 INFO:tasks.workunit.client.0.smithi027.stdout:3/839: chown d1/db/d52/d6a/d7c/fd0 1 1 2021-11-03T17:14:43.225 INFO:tasks.workunit.client.0.smithi027.stdout:3/840: symlink d1/db/d52/d57/d99/da7/dbf/d102/l10c 0 2021-11-03T17:14:43.227 INFO:tasks.workunit.client.0.smithi027.stdout:1/826: dwrite d6/d11/dac/d79/d37/de0/de4/f7f [4194304,4194304] 0 2021-11-03T17:14:43.229 INFO:tasks.workunit.client.0.smithi027.stdout:3/841: rename d1/db/d52/d6a/d7c/d88/d98/fbc to d1/db/d52/d57/f10d 0 2021-11-03T17:14:43.229 INFO:tasks.workunit.client.0.smithi027.stdout:3/842: truncate d1/db/db2/f106 107333 0 2021-11-03T17:14:43.230 INFO:tasks.workunit.client.0.smithi027.stdout:1/827: read d6/d11/dac/d79/d37/de0/de4/d58/fcd [6987084,107899] 0 2021-11-03T17:14:43.231 INFO:tasks.workunit.client.0.smithi027.stdout:3/843: creat d1/db/d52/d6a/f10e x:0 0 0 2021-11-03T17:14:43.235 INFO:tasks.workunit.client.0.smithi027.stdout:1/828: rename d6/d11/dac/d79/d25/d28/d5c/fc6 to d6/d11/dac/d79/d25/d28/d5c/d7a/dee/f10d 0 2021-11-03T17:14:43.237 INFO:tasks.workunit.client.0.smithi027.stdout:3/844: creat d1/db/db2/f10f x:0 0 0 2021-11-03T17:14:43.240 INFO:tasks.workunit.client.0.smithi027.stdout:3/845: creat d1/db/db2/f110 x:0 0 0 2021-11-03T17:14:43.242 INFO:tasks.workunit.client.0.smithi027.stdout:3/846: write d1/d40/f4a [604230,35164] 0 2021-11-03T17:14:43.244 INFO:tasks.workunit.client.0.smithi027.stdout:3/847: mkdir d1/db/d52/d6a/dd5/d111 0 2021-11-03T17:14:43.244 INFO:tasks.workunit.client.0.smithi027.stdout:3/848: fsync d1/db/d52/d57/d99/df6/ffc 0 2021-11-03T17:14:43.246 INFO:tasks.workunit.client.0.smithi027.stdout:3/849: creat d1/db/d52/d57/d99/da7/f112 x:0 0 0 2021-11-03T17:14:43.254 INFO:tasks.workunit.client.0.smithi027.stdout:1/829: dwrite d6/d11/dac/d79/d25/d28/dec/fd0 [0,4194304] 0 2021-11-03T17:14:43.254 INFO:tasks.workunit.client.0.smithi027.stdout:1/830: fdatasync d6/d11/dac/d79/db0/d102/f66 0 2021-11-03T17:14:43.254 INFO:tasks.workunit.client.0.smithi027.stdout:1/831: truncate d6/d11/dac/d79/d37/de0/de4/fc7 238826 0 2021-11-03T17:14:43.258 INFO:tasks.workunit.client.0.smithi027.stdout:1/832: rename d6/l10 to d6/d11/dac/d79/d25/d5e/d72/l10e 0 2021-11-03T17:14:43.261 INFO:tasks.workunit.client.0.smithi027.stdout:1/833: mkdir d6/d11/dac/d79/db0/de5/d10f 0 2021-11-03T17:14:43.263 INFO:tasks.workunit.client.0.smithi027.stdout:1/834: symlink d6/d11/dac/d79/dab/l110 0 2021-11-03T17:14:43.263 INFO:tasks.workunit.client.0.smithi027.stdout:1/835: truncate d6/d11/dac/d79/db0/de5/dff/fae 1145705 0 2021-11-03T17:14:43.267 INFO:tasks.workunit.client.0.smithi027.stdout:1/836: link d6/d11/dac/d79/d25/d28/d41/l9e d6/d11/dac/d79/l111 0 2021-11-03T17:14:43.276 INFO:tasks.workunit.client.0.smithi027.stdout:3/850: dwrite d1/d40/f4a [0,4194304] 0 2021-11-03T17:14:43.279 INFO:tasks.workunit.client.0.smithi027.stdout:3/851: creat d1/db/d52/d57/d99/da7/dbf/d102/f113 x:0 0 0 2021-11-03T17:14:43.281 INFO:tasks.workunit.client.0.smithi027.stdout:3/852: creat d1/db/d52/d6a/db4/db5/dad/f114 x:0 0 0 2021-11-03T17:14:43.281 INFO:tasks.workunit.client.0.smithi027.stdout:3/853: fsync d1/db/d52/d57/d99/da7/dbf/fe9 0 2021-11-03T17:14:43.283 INFO:tasks.workunit.client.0.smithi027.stdout:3/854: creat d1/db/d52/d57/d99/f115 x:0 0 0 2021-11-03T17:14:43.284 INFO:tasks.workunit.client.0.smithi027.stdout:1/837: dwrite d6/d11/dac/f6d [0,4194304] 0 2021-11-03T17:14:43.289 INFO:tasks.workunit.client.0.smithi027.stdout:1/838: creat d6/d11/dac/d79/f112 x:0 0 0 2021-11-03T17:14:43.291 INFO:tasks.workunit.client.0.smithi027.stdout:1/839: rmdir d6/d11/d14/df6 39 2021-11-03T17:14:43.291 INFO:tasks.workunit.client.0.smithi027.stdout:1/840: chown d6/d11/dac/d79/d25/d28/d5c/d81/f85 5125252 1 2021-11-03T17:14:43.291 INFO:tasks.workunit.client.0.smithi027.stdout:1/841: mknod d6/d11/dac/d79/c113 1 2021-11-03T17:14:43.292 INFO:tasks.workunit.client.0.smithi027.stdout:1/842: mknod d6/d11/dac/d10b/c114 1 2021-11-03T17:14:43.292 INFO:tasks.workunit.client.0.smithi027.stdout:1/843: fsync d6/d11/dac/d79/f63 0 2021-11-03T17:14:43.294 INFO:tasks.workunit.client.0.smithi027.stdout:1/844: mkdir d6/de/df1/d115 0 2021-11-03T17:14:43.298 INFO:tasks.workunit.client.0.smithi027.stdout:3/855: dwrite d1/db/d52/d57/fd2 [0,4194304] 0 2021-11-03T17:14:43.313 INFO:tasks.workunit.client.0.smithi027.stdout:1/845: dwrite d6/d11/dac/d79/d37/de0/d4e/f91 [0,4194304] 0 2021-11-03T17:14:43.313 INFO:tasks.workunit.client.0.smithi027.stdout:1/846: mknod d6/d11/dac/d79/db0/d102/c116 1 2021-11-03T17:14:43.314 INFO:tasks.workunit.client.0.smithi027.stdout:3/856: creat d1/db/d52/d57/f116 x:0 0 0 2021-11-03T17:14:43.315 INFO:tasks.workunit.client.0.smithi027.stdout:3/857: fdatasync d1/db/d52/d6a/d7c/fa9 0 2021-11-03T17:14:43.315 INFO:tasks.workunit.client.0.smithi027.stdout:3/858: chown d1/db/d52/d6a/d7c/fd6 3328 1 2021-11-03T17:14:43.315 INFO:tasks.workunit.client.0.smithi027.stdout:3/859: mknod d1/dec/c117 1 2021-11-03T17:14:43.315 INFO:tasks.workunit.client.0.smithi027.stdout:3/860: read - d1/db/d52/d57/d99/ff7 zero size 2021-11-03T17:14:43.316 INFO:tasks.workunit.client.0.smithi027.stdout:3/861: chown d1/db/d52/d6a/d7c/fd6 22 1 2021-11-03T17:14:43.316 INFO:tasks.workunit.client.0.smithi027.stdout:1/847: rename d6/d11/dac/d79/d25/d28/f107 to d6/de/f117 0 2021-11-03T17:14:43.318 INFO:tasks.workunit.client.0.smithi027.stdout:3/862: getdents d1/db/d52/d57/d99/da7/dbf 0 2021-11-03T17:14:43.320 INFO:tasks.workunit.client.0.smithi027.stdout:1/848: rename d6/d11/dac/d79/d25/d28/d5c/d81/l8c to d6/d11/dac/d79/d37/de0/de4/d58/l118 0 2021-11-03T17:14:43.324 INFO:tasks.workunit.client.0.smithi027.stdout:3/863: symlink d1/db/d52/d6a/dd5/d111/l118 0 2021-11-03T17:14:43.329 INFO:tasks.workunit.client.0.smithi027.stdout:1/849: dread d6/d11/dac/d79/d25/d28/f55 [4194304,4194304] 0 2021-11-03T17:14:43.331 INFO:tasks.workunit.client.0.smithi027.stdout:3/864: write d1/db/d52/d57/f10d [962387,57306] 0 2021-11-03T17:14:43.333 INFO:tasks.workunit.client.0.smithi027.stdout:3/865: creat d1/db/d52/d6a/f119 x:0 0 0 2021-11-03T17:14:43.333 INFO:tasks.workunit.client.0.smithi027.stdout:3/866: chown d1/db/ff8 1 1 2021-11-03T17:14:43.336 INFO:tasks.workunit.client.0.smithi027.stdout:3/867: link d1/db/d52/d6a/d7c/fef d1/db/d52/d57/d99/da7/dbf/f11a 0 2021-11-03T17:14:43.337 INFO:tasks.workunit.client.0.smithi027.stdout:1/850: dread d6/d11/dac/d79/d25/d28/d5c/f78 [0,4194304] 0 2021-11-03T17:14:43.338 INFO:tasks.workunit.client.0.smithi027.stdout:4/656: sync 2021-11-03T17:14:43.339 INFO:tasks.workunit.client.0.smithi027.stdout:4/657: read - d6/d9/d16/d36/d70/d3e/d58/fc5 zero size 2021-11-03T17:14:43.339 INFO:tasks.workunit.client.0.smithi027.stdout:4/658: readlink d6/l5b 0 2021-11-03T17:14:43.339 INFO:tasks.workunit.client.0.smithi027.stdout:4/659: chown d6/d9/d16/d36/d57/d5a/d63 0 1 2021-11-03T17:14:43.339 INFO:tasks.workunit.client.0.smithi027.stdout:3/868: dread d1/db/f3c [0,4194304] 0 2021-11-03T17:14:43.340 INFO:tasks.workunit.client.0.smithi027.stdout:3/869: write d1/db/db2/dbd/ffa [1031290,3902] 0 2021-11-03T17:14:43.340 INFO:tasks.workunit.client.0.smithi027.stdout:4/660: write d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/f2c [5268014,124110] 0 2021-11-03T17:14:43.340 INFO:tasks.workunit.client.0.smithi027.stdout:1/851: creat d6/d11/dac/d79/d37/d49/f119 x:0 0 0 2021-11-03T17:14:43.341 INFO:tasks.workunit.client.0.smithi027.stdout:3/870: write d1/db/d52/d6a/f7f [354985,63646] 0 2021-11-03T17:14:43.349 INFO:tasks.workunit.client.0.smithi027.stdout:1/852: creat d6/d11/dac/d79/d25/f11a x:0 0 0 2021-11-03T17:14:43.351 INFO:tasks.workunit.client.0.smithi027.stdout:4/661: mkdir d6/d9/d16/d17/ddd 0 2021-11-03T17:14:43.352 INFO:tasks.workunit.client.0.smithi027.stdout:4/662: chown d6/d9/d16/d36/d57 813851 1 2021-11-03T17:14:43.352 INFO:tasks.workunit.client.0.smithi027.stdout:4/663: truncate d6/d24/db0/f53 1239618 0 2021-11-03T17:14:43.352 INFO:tasks.workunit.client.0.smithi027.stdout:1/853: rmdir d6/de 39 2021-11-03T17:14:43.353 INFO:tasks.workunit.client.0.smithi027.stdout:1/854: dread - d6/d11/dac/d79/d25/d28/d5c/d81/d7d/fdb zero size 2021-11-03T17:14:43.356 INFO:tasks.workunit.client.0.smithi027.stdout:4/664: symlink d6/d9/d16/d36/d70/d3e/d58/d84/d9b/lde 0 2021-11-03T17:14:43.358 INFO:tasks.workunit.client.0.smithi027.stdout:4/665: truncate d6/d9/d16/d36/d57/d5a/d63/fd4 2997007 0 2021-11-03T17:14:43.360 INFO:tasks.workunit.client.0.smithi027.stdout:3/871: dwrite d1/db/db2/dbd/ff3 [0,4194304] 0 2021-11-03T17:14:43.365 INFO:tasks.workunit.client.0.smithi027.stdout:4/666: unlink d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d41/l52 0 2021-11-03T17:14:43.378 INFO:tasks.workunit.client.0.smithi027.stdout:3/872: rename d1/db/d52/d6a/db4/db5/dad to d1/db/db2/dbd/d11b 0 2021-11-03T17:14:43.381 INFO:tasks.workunit.client.0.smithi027.stdout:4/667: rmdir d6/d9/d16/d36/d70 39 2021-11-03T17:14:43.383 INFO:tasks.workunit.client.0.smithi027.stdout:4/668: write d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d2b/f3f [750940,96123] 0 2021-11-03T17:14:43.383 INFO:tasks.workunit.client.0.smithi027.stdout:3/873: symlink d1/db/d52/d57/l11c 0 2021-11-03T17:14:43.384 INFO:tasks.workunit.client.0.smithi027.stdout:3/874: readlink d1/db/d52/l80 0 2021-11-03T17:14:43.386 INFO:tasks.workunit.client.0.smithi027.stdout:3/875: link d1/l43 d1/db/d52/d57/de3/l11d 0 2021-11-03T17:14:43.386 INFO:tasks.workunit.client.0.smithi027.stdout:3/876: stat d1/d40 0 2021-11-03T17:14:43.386 INFO:tasks.workunit.client.0.smithi027.stdout:3/877: chown d1/db/d52/d6a/dd5/d111/l118 258678 1 2021-11-03T17:14:43.388 INFO:tasks.workunit.client.0.smithi027.stdout:3/878: unlink d1/l1b 0 2021-11-03T17:14:43.388 INFO:tasks.workunit.client.0.smithi027.stdout:3/879: chown d1/db/f3f 6396836 1 2021-11-03T17:14:43.390 INFO:tasks.workunit.client.0.smithi027.stdout:3/880: rename d1/l96 to d1/db/d52/d57/d99/l11e 0 2021-11-03T17:14:43.396 INFO:tasks.workunit.client.0.smithi027.stdout:4/669: dread d6/d9/d16/d36/d57/d5a/f67 [0,4194304] 0 2021-11-03T17:14:43.404 INFO:tasks.workunit.client.0.smithi027.stdout:4/670: rename d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/f8a to d6/d9/d16/d17/fdf 0 2021-11-03T17:14:43.405 INFO:tasks.workunit.client.0.smithi027.stdout:3/881: rmdir d1/db/d52/d57/d99 39 2021-11-03T17:14:43.409 INFO:tasks.workunit.client.0.smithi027.stdout:4/671: symlink d6/d9/d16/d36/d99/dd0/le0 0 2021-11-03T17:14:43.413 INFO:tasks.workunit.client.0.smithi027.stdout:3/882: mkdir d1/d40/d11f 0 2021-11-03T17:14:43.418 INFO:tasks.workunit.client.0.smithi027.stdout:3/883: dread d1/d40/f8a [0,4194304] 0 2021-11-03T17:14:43.423 INFO:tasks.workunit.client.0.smithi027.stdout:3/884: getdents d1/db/d52/d57 0 2021-11-03T17:14:43.429 INFO:tasks.workunit.client.0.smithi027.stdout:4/672: dwrite d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/f93 [0,4194304] 0 2021-11-03T17:14:43.429 INFO:tasks.workunit.client.0.smithi027.stdout:3/885: dread d1/d40/f5c [0,4194304] 0 2021-11-03T17:14:43.430 INFO:tasks.workunit.client.0.smithi027.stdout:4/673: readlink d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/l54 0 2021-11-03T17:14:43.433 INFO:tasks.workunit.client.0.smithi027.stdout:3/886: mkdir d1/db/d52/d6a/dd5/d120 0 2021-11-03T17:14:43.450 INFO:tasks.workunit.client.0.smithi027.stdout:4/674: dwrite d6/d9/d16/d36/d99/db9/fd6 [0,4194304] 0 2021-11-03T17:14:43.465 INFO:tasks.workunit.client.0.smithi027.stdout:4/675: dread d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/fa2 [0,4194304] 0 2021-11-03T17:14:43.465 INFO:tasks.workunit.client.0.smithi027.stdout:4/676: write d6/d9/d16/d36/d70/d3e/d58/d84/d9b/fcb [423580,64509] 0 2021-11-03T17:14:43.466 INFO:tasks.workunit.client.0.smithi027.stdout:4/677: mknod d6/d9/d16/d17/ce1 1 2021-11-03T17:14:43.466 INFO:tasks.workunit.client.0.smithi027.stdout:4/678: dread - d6/d9/d16/d36/d57/d5a/d63/fd9 zero size 2021-11-03T17:14:43.481 INFO:tasks.workunit.client.0.smithi027.stdout:4/679: dwrite d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/fc2 [0,4194304] 0 2021-11-03T17:14:43.484 INFO:tasks.workunit.client.0.smithi027.stdout:4/680: symlink d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d41/le2 0 2021-11-03T17:14:43.488 INFO:tasks.workunit.client.0.smithi027.stdout:4/681: dread d6/d9/d16/d36/d70/d3e/d58/f61 [0,4194304] 0 2021-11-03T17:14:43.496 INFO:tasks.workunit.client.0.smithi027.stdout:4/682: dread d6/d9/f73 [0,4194304] 0 2021-11-03T17:14:43.498 INFO:tasks.workunit.client.0.smithi027.stdout:4/683: truncate d6/d24/db0/f76 1680722 0 2021-11-03T17:14:43.500 INFO:tasks.workunit.client.0.smithi027.stdout:4/684: rename d6/d8c to d6/d24/de3 0 2021-11-03T17:14:43.501 INFO:tasks.workunit.client.0.smithi027.stdout:4/685: mknod d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d41/d44/ce4 1 2021-11-03T17:14:43.501 INFO:tasks.workunit.client.0.smithi027.stdout:4/686: stat d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d9a/fba 0 2021-11-03T17:14:43.516 INFO:tasks.workunit.client.0.smithi027.stdout:4/687: dwrite d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/f93 [0,4194304] 0 2021-11-03T17:14:43.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:43 smithi027 conmon[32322]: cluster 2021-11-03T17:14 2021-11-03T17:14:43.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:43 smithi027 conmon[32322]: :41.896462+0000 mgr.smithi115.tcyodz (mgr.24395) 24 : cluster [DBG] pgmap v16: 65 pgs: 65 active+clean; 2.9 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail; 46 MiB/s rd, 56 MiB/s wr, 100 op/s 2021-11-03T17:14:43.520 INFO:tasks.workunit.client.0.smithi027.stdout:4/688: mkdir d6/d24/dbe/de5 0 2021-11-03T17:14:43.521 INFO:tasks.workunit.client.0.smithi027.stdout:4/689: write d6/d9/d16/d36/d70/d3e/f6b [204287,49595] 0 2021-11-03T17:14:43.522 INFO:tasks.workunit.client.0.smithi027.stdout:4/690: mkdir d6/d24/d6d/de6 0 2021-11-03T17:14:43.540 INFO:tasks.workunit.client.0.smithi027.stdout:4/691: dwrite d6/d24/f46 [0,4194304] 0 2021-11-03T17:14:43.541 INFO:tasks.workunit.client.0.smithi027.stdout:4/692: readlink d6/d9/d16/d36/d70/d3e/d58/d84/lc7 0 2021-11-03T17:14:43.541 INFO:tasks.workunit.client.0.smithi027.stdout:4/693: symlink d6/d24/dbe/de5/le7 0 2021-11-03T17:14:43.542 INFO:tasks.workunit.client.0.smithi027.stdout:4/694: fsync d6/d9/d16/d36/d70/d3e/f6c 0 2021-11-03T17:14:43.543 INFO:tasks.workunit.client.0.smithi027.stdout:4/695: dread d6/d9/d16/d17/fdf [0,4194304] 0 2021-11-03T17:14:43.545 INFO:tasks.workunit.client.0.smithi027.stdout:4/696: symlink d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/le8 0 2021-11-03T17:14:43.545 INFO:tasks.workunit.client.0.smithi027.stdout:4/697: write d6/d9/d16/d17/fdf [888896,10369] 0 2021-11-03T17:14:43.548 INFO:tasks.workunit.client.0.smithi027.stdout:4/698: unlink d6/d24/d6d/lae 0 2021-11-03T17:14:43.548 INFO:tasks.workunit.client.0.smithi027.stdout:4/699: dread - d6/d9/d16/d36/d70/d3e/fd1 zero size 2021-11-03T17:14:43.550 INFO:tasks.workunit.client.0.smithi027.stdout:4/700: mkdir d6/d9/d16/d36/d70/d3e/de9 0 2021-11-03T17:14:43.550 INFO:tasks.workunit.client.0.smithi027.stdout:4/701: dread - d6/d9/d16/d36/d70/d3e/d58/fa4 zero size 2021-11-03T17:14:43.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:43 smithi115 conmon[34670]: cluster 2021-11-03T17:14:41.896462+0000 mgr.smithi115.tcyodz ( 2021-11-03T17:14:43.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:43 smithi115 conmon[34670]: mgr.24395) 24 : cluster [DBG] pgmap v16: 65 pgs: 65 active+clean; 2.9 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail; 46 MiB/s rd, 56 MiB/s wr, 100 op/s 2021-11-03T17:14:43.792 INFO:tasks.workunit.client.0.smithi027.stdout:6/681: sync 2021-11-03T17:14:43.792 INFO:tasks.workunit.client.0.smithi027.stdout:2/763: sync 2021-11-03T17:14:43.793 INFO:tasks.workunit.client.0.smithi027.stdout:6/682: readlink dc/d82/d57/l8d 0 2021-11-03T17:14:43.793 INFO:tasks.workunit.client.0.smithi027.stdout:6/683: write dc/d37/fbd [525623,17508] 0 2021-11-03T17:14:43.798 INFO:tasks.workunit.client.0.smithi027.stdout:6/684: rename dc/d16/d1b/d3c/d51/d5f/d79 to dc/d37/dcc 0 2021-11-03T17:14:43.799 INFO:tasks.workunit.client.0.smithi027.stdout:6/685: write dc/f18 [85083,129888] 0 2021-11-03T17:14:43.800 INFO:tasks.workunit.client.0.smithi027.stdout:6/686: truncate dc/d37/fbe 1797560 0 2021-11-03T17:14:43.809 INFO:tasks.workunit.client.0.smithi027.stdout:6/687: dread dc/d16/d1b/d3c/d51/f70 [0,4194304] 0 2021-11-03T17:14:43.809 INFO:tasks.workunit.client.0.smithi027.stdout:6/688: read dc/d82/d57/f97 [104861,42501] 0 2021-11-03T17:14:43.813 INFO:tasks.workunit.client.0.smithi027.stdout:2/764: dwrite d0/d5/fac [0,4194304] 0 2021-11-03T17:14:43.814 INFO:tasks.workunit.client.0.smithi027.stdout:2/765: mknod d0/d1b/d29/d54/d64/daa/d88/ce1 1 2021-11-03T17:14:43.818 INFO:tasks.workunit.client.0.smithi027.stdout:2/766: unlink d0/d1b/d29/d54/d64/daa/f21 0 2021-11-03T17:14:43.820 INFO:tasks.workunit.client.0.smithi027.stdout:2/767: truncate d0/d5/f3 6858383 0 2021-11-03T17:14:43.828 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:43.831+0000 7f1a8b4cc700 1 -- 172.21.15.27:0/2817865836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a840f51a0 msgr2=0x7f1a840f7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:14:43.829 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:43.831+0000 7f1a8b4cc700 1 --2- 172.21.15.27:0/2817865836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a840f51a0 0x7f1a840f7920 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7f1a80004660 tx=0x7f1a80009b30).stop 2021-11-03T17:14:43.829 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:43.831+0000 7f1a8b4cc700 1 -- 172.21.15.27:0/2817865836 shutdown_connections 2021-11-03T17:14:43.829 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:43.831+0000 7f1a8b4cc700 1 --2- 172.21.15.27:0/2817865836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a840f51a0 0x7f1a840f7920 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:43.831 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:43.831+0000 7f1a8b4cc700 1 --2- 172.21.15.27:0/2817865836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a840f4800 0x7f1a840f4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:43.831 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:43.831+0000 7f1a8b4cc700 1 -- 172.21.15.27:0/2817865836 >> 172.21.15.27:0/2817865836 conn(0x7f1a840f0320 msgr2=0x7f1a840f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:14:43.831 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:43.832+0000 7f1a8b4cc700 1 -- 172.21.15.27:0/2817865836 shutdown_connections 2021-11-03T17:14:43.831 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:43.832+0000 7f1a8b4cc700 1 -- 172.21.15.27:0/2817865836 wait complete. 2021-11-03T17:14:43.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:43.832+0000 7f1a8b4cc700 1 Processor -- start 2021-11-03T17:14:43.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:43.832+0000 7f1a8b4cc700 1 -- start start 2021-11-03T17:14:43.833 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:43.833+0000 7f1a8b4cc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a840f4800 0x7f1a84108f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:14:43.833 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:43.833+0000 7f1a8b4cc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a840f51a0 0x7f1a84109400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:14:43.833 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:43.833+0000 7f1a8b4cc700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a841099b0 con 0x7f1a840f51a0 2021-11-03T17:14:43.833 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:43.833+0000 7f1a8b4cc700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a84109af0 con 0x7f1a840f4800 2021-11-03T17:14:43.834 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:43.833+0000 7f1a89cc9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a840f51a0 0x7f1a84109400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:14:43.834 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:43.833+0000 7f1a89cc9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a840f51a0 0x7f1a84109400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55420/0 (socket says 172.21.15.27:55420) 2021-11-03T17:14:43.834 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:43.833+0000 7f1a89cc9700 1 -- 172.21.15.27:0/815023159 learned_addr learned my addr 172.21.15.27:0/815023159 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:14:43.835 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:43.833+0000 7f1a8a4ca700 1 --2- 172.21.15.27:0/815023159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a840f4800 0x7f1a84108f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:14:43.835 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:43.833+0000 7f1a89cc9700 1 -- 172.21.15.27:0/815023159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a840f4800 msgr2=0x7f1a84108f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:14:43.835 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:43.833+0000 7f1a89cc9700 1 --2- 172.21.15.27:0/815023159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a840f4800 0x7f1a84108f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:43.835 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:43.833+0000 7f1a89cc9700 1 -- 172.21.15.27:0/815023159 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a80005040 con 0x7f1a840f51a0 2021-11-03T17:14:43.835 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:43.833+0000 7f1a8a4ca700 1 --2- 172.21.15.27:0/815023159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a840f4800 0x7f1a84108f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:14:43.836 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:43.833+0000 7f1a89cc9700 1 --2- 172.21.15.27:0/815023159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a840f51a0 0x7f1a84109400 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f1a80014040 tx=0x7f1a8000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:14:43.836 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:43.834+0000 7f1a76ffd700 1 -- 172.21.15.27:0/815023159 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a80019070 con 0x7f1a840f51a0 2021-11-03T17:14:43.836 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:43.834+0000 7f1a8b4cc700 1 -- 172.21.15.27:0/815023159 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a8410a430 con 0x7f1a840f51a0 2021-11-03T17:14:43.837 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:43.834+0000 7f1a8b4cc700 1 -- 172.21.15.27:0/815023159 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a8410a990 con 0x7f1a840f51a0 2021-11-03T17:14:43.837 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:43.834+0000 7f1a76ffd700 1 -- 172.21.15.27:0/815023159 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f1a80007e70 con 0x7f1a840f51a0 2021-11-03T17:14:43.837 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:43.834+0000 7f1a76ffd700 1 -- 172.21.15.27:0/815023159 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a80014d50 con 0x7f1a840f51a0 2021-11-03T17:14:43.838 INFO:tasks.workunit.client.0.smithi027.stdout:2/768: dread d0/d1b/d29/f4b [0,4194304] 0 2021-11-03T17:14:43.839 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:43.835+0000 7f1a8b4cc700 1 -- 172.21.15.27:0/815023159 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a6c004fa0 con 0x7f1a840f51a0 2021-11-03T17:14:43.839 INFO:tasks.workunit.client.0.smithi027.stdout:2/769: read d0/d5/f83 [3112587,57273] 0 2021-11-03T17:14:43.840 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:43.836+0000 7f1a76ffd700 1 -- 172.21.15.27:0/815023159 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 23) v1 ==== 77155+0+0 (secure 0 0 0) 0x7f1a80025070 con 0x7f1a840f51a0 2021-11-03T17:14:43.840 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:43.836+0000 7f1a76ffd700 1 --2- 172.21.15.27:0/815023159 >> [v2:172.21.15.115:6828/1733074450,v1:172.21.15.115:6829/1733074450] conn(0x7f1a700602d0 0x7f1a70062790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:14:43.841 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:43.836+0000 7f1a76ffd700 1 -- 172.21.15.27:0/815023159 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5601+0+0 (secure 0 0 0) 0x7f1a8007c2a0 con 0x7f1a840f51a0 2021-11-03T17:14:43.841 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:43.836+0000 7f1a8a4ca700 1 --2- 172.21.15.27:0/815023159 >> [v2:172.21.15.115:6828/1733074450,v1:172.21.15.115:6829/1733074450] conn(0x7f1a700602d0 0x7f1a70062790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:14:43.841 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:43.837+0000 7f1a8a4ca700 1 --2- 172.21.15.27:0/815023159 >> [v2:172.21.15.115:6828/1733074450,v1:172.21.15.115:6829/1733074450] conn(0x7f1a700602d0 0x7f1a70062790 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f1a78009da0 tx=0x7f1a7800b040).ready entity=mgr.24395 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:14:43.841 INFO:tasks.workunit.client.0.smithi027.stdout:2/770: write d0/f10 [1363740,83123] 0 2021-11-03T17:14:43.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:43.841+0000 7f1a76ffd700 1 -- 172.21.15.27:0/815023159 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156383 (secure 0 0 0) 0x7f1a8004ea90 con 0x7f1a840f51a0 2021-11-03T17:14:43.843 INFO:tasks.workunit.client.0.smithi027.stdout:2/771: dread - d0/d1b/d29/d54/d64/daa/fc4 zero size 2021-11-03T17:14:43.843 INFO:tasks.workunit.client.0.smithi027.stdout:2/772: creat d0/da7/fe2 x:0 0 0 2021-11-03T17:14:43.843 INFO:tasks.workunit.client.0.smithi027.stdout:2/773: fdatasync d0/fa9 0 2021-11-03T17:14:43.844 INFO:tasks.workunit.client.0.smithi027.stdout:2/774: write d0/da7/dc9/fa3 [835408,65643] 0 2021-11-03T17:14:43.844 INFO:tasks.workunit.client.0.smithi027.stdout:2/775: write d0/d1b/d29/d54/d64/daa/d5d/f65 [738799,103726] 0 2021-11-03T17:14:43.856 INFO:tasks.workunit.client.0.smithi027.stdout:2/776: dwrite d0/d1b/d29/d54/d67/f6c [0,4194304] 0 2021-11-03T17:14:43.857 INFO:tasks.workunit.client.0.smithi027.stdout:2/777: stat d0/d5/d2c 0 2021-11-03T17:14:43.858 INFO:tasks.workunit.client.0.smithi027.stdout:2/778: write d0/d5/fac [1874699,105891] 0 2021-11-03T17:14:43.859 INFO:tasks.workunit.client.0.smithi027.stdout:2/779: mknod d0/d1b/d29/d92/ce3 1 2021-11-03T17:14:43.860 INFO:tasks.workunit.client.0.smithi027.stdout:2/780: symlink d0/d1b/d87/le4 0 2021-11-03T17:14:43.861 INFO:tasks.workunit.client.0.smithi027.stdout:2/781: read d0/d5/f83 [168644,93065] 0 2021-11-03T17:14:43.862 INFO:tasks.workunit.client.0.smithi027.stdout:2/782: write d0/d1b/db6/fdb [827310,26046] 0 2021-11-03T17:14:44.029 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.034+0000 7f1a8b4cc700 1 -- 172.21.15.27:0/815023159 --> [v2:172.21.15.115:6828/1733074450,v1:172.21.15.115:6829/1733074450] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a6c000bc0 con 0x7f1a700602d0 2021-11-03T17:14:44.030 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.036+0000 7f1a76ffd700 1 -- 172.21.15.27:0/815023159 <== mgr.24395 v2:172.21.15.115:6828/1733074450 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+269 (secure 0 0 0) 0x7f1a6c000bc0 con 0x7f1a700602d0 2021-11-03T17:14:44.032 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.037+0000 7f1a8b4cc700 1 -- 172.21.15.27:0/815023159 >> [v2:172.21.15.115:6828/1733074450,v1:172.21.15.115:6829/1733074450] conn(0x7f1a700602d0 msgr2=0x7f1a70062790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:14:44.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.037+0000 7f1a8b4cc700 1 --2- 172.21.15.27:0/815023159 >> [v2:172.21.15.115:6828/1733074450,v1:172.21.15.115:6829/1733074450] conn(0x7f1a700602d0 0x7f1a70062790 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f1a78009da0 tx=0x7f1a7800b040).stop 2021-11-03T17:14:44.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.037+0000 7f1a8b4cc700 1 -- 172.21.15.27:0/815023159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a840f51a0 msgr2=0x7f1a84109400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:14:44.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.037+0000 7f1a8b4cc700 1 --2- 172.21.15.27:0/815023159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a840f51a0 0x7f1a84109400 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f1a80014040 tx=0x7f1a8000b040).stop 2021-11-03T17:14:44.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.037+0000 7f1a8b4cc700 1 -- 172.21.15.27:0/815023159 shutdown_connections 2021-11-03T17:14:44.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.037+0000 7f1a8b4cc700 1 --2- 172.21.15.27:0/815023159 >> [v2:172.21.15.115:6828/1733074450,v1:172.21.15.115:6829/1733074450] conn(0x7f1a700602d0 0x7f1a70062790 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:44.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.037+0000 7f1a8b4cc700 1 --2- 172.21.15.27:0/815023159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a840f4800 0x7f1a84108f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:44.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.037+0000 7f1a8b4cc700 1 --2- 172.21.15.27:0/815023159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a840f51a0 0x7f1a84109400 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:44.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.037+0000 7f1a8b4cc700 1 -- 172.21.15.27:0/815023159 >> 172.21.15.27:0/815023159 conn(0x7f1a840f0320 msgr2=0x7f1a840f5f30 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:14:44.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.038+0000 7f1a8b4cc700 1 -- 172.21.15.27:0/815023159 shutdown_connections 2021-11-03T17:14:44.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.038+0000 7f1a8b4cc700 1 -- 172.21.15.27:0/815023159 wait complete. 2021-11-03T17:14:44.042 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:14:44.138 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.138+0000 7f353cb21700 1 -- 172.21.15.27:0/2400192741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35380f6ff0 msgr2=0x7f35380f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:14:44.139 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.138+0000 7f353cb21700 1 --2- 172.21.15.27:0/2400192741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35380f6ff0 0x7f35380f7a20 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f352c004660 tx=0x7f352c009b30).stop 2021-11-03T17:14:44.139 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.139+0000 7f353cb21700 1 -- 172.21.15.27:0/2400192741 shutdown_connections 2021-11-03T17:14:44.139 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.139+0000 7f353cb21700 1 --2- 172.21.15.27:0/2400192741 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35380f6ff0 0x7f35380f7a20 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:44.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.139+0000 7f353cb21700 1 --2- 172.21.15.27:0/2400192741 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35380f66e0 0x7f35380f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:44.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.139+0000 7f353cb21700 1 -- 172.21.15.27:0/2400192741 >> 172.21.15.27:0/2400192741 conn(0x7f35380f1f00 msgr2=0x7f35380f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:14:44.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.139+0000 7f353cb21700 1 -- 172.21.15.27:0/2400192741 shutdown_connections 2021-11-03T17:14:44.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.139+0000 7f353cb21700 1 -- 172.21.15.27:0/2400192741 wait complete. 2021-11-03T17:14:44.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.139+0000 7f353cb21700 1 Processor -- start 2021-11-03T17:14:44.142 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.139+0000 7f353cb21700 1 -- start start 2021-11-03T17:14:44.142 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.139+0000 7f353cb21700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35380f66e0 0x7f3538106ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:14:44.142 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.139+0000 7f353cb21700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35380f6ff0 0x7f35381073e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:14:44.143 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.139+0000 7f353cb21700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3538107990 con 0x7f35380f6ff0 2021-11-03T17:14:44.143 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.139+0000 7f353cb21700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3538107ad0 con 0x7f35380f66e0 2021-11-03T17:14:44.144 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.140+0000 7f3535d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35380f6ff0 0x7f35381073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:14:44.144 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.140+0000 7f3535d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35380f6ff0 0x7f35381073e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55430/0 (socket says 172.21.15.27:55430) 2021-11-03T17:14:44.144 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.140+0000 7f3535d9b700 1 -- 172.21.15.27:0/40528563 learned_addr learned my addr 172.21.15.27:0/40528563 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:14:44.145 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.140+0000 7f353659c700 1 --2- 172.21.15.27:0/40528563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35380f66e0 0x7f3538106ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:14:44.145 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.140+0000 7f3535d9b700 1 -- 172.21.15.27:0/40528563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35380f66e0 msgr2=0x7f3538106ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:14:44.146 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.140+0000 7f3535d9b700 1 --2- 172.21.15.27:0/40528563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35380f66e0 0x7f3538106ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:44.146 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.140+0000 7f3535d9b700 1 -- 172.21.15.27:0/40528563 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f352c005040 con 0x7f35380f6ff0 2021-11-03T17:14:44.146 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.140+0000 7f3535d9b700 1 --2- 172.21.15.27:0/40528563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35380f6ff0 0x7f35381073e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f352c009990 tx=0x7f352c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:14:44.147 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.140+0000 7f3526ffd700 1 -- 172.21.15.27:0/40528563 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f352c018070 con 0x7f35380f6ff0 2021-11-03T17:14:44.147 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.140+0000 7f3526ffd700 1 -- 172.21.15.27:0/40528563 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f352c007ba0 con 0x7f35380f6ff0 2021-11-03T17:14:44.147 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.140+0000 7f353cb21700 1 -- 172.21.15.27:0/40528563 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3538108410 con 0x7f35380f6ff0 2021-11-03T17:14:44.147 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.140+0000 7f3526ffd700 1 -- 172.21.15.27:0/40528563 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f352c01c3c0 con 0x7f35380f6ff0 2021-11-03T17:14:44.148 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.140+0000 7f353cb21700 1 -- 172.21.15.27:0/40528563 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3538108970 con 0x7f35380f6ff0 2021-11-03T17:14:44.148 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.141+0000 7f353cb21700 1 -- 172.21.15.27:0/40528563 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3538045ad0 con 0x7f35380f6ff0 2021-11-03T17:14:44.148 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.141+0000 7f3526ffd700 1 -- 172.21.15.27:0/40528563 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 23) v1 ==== 77155+0+0 (secure 0 0 0) 0x7f352c01d420 con 0x7f35380f6ff0 2021-11-03T17:14:44.148 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.142+0000 7f3526ffd700 1 --2- 172.21.15.27:0/40528563 >> [v2:172.21.15.115:6828/1733074450,v1:172.21.15.115:6829/1733074450] conn(0x7f35200602e0 0x7f35200627a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:14:44.148 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.142+0000 7f3526ffd700 1 -- 172.21.15.27:0/40528563 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5601+0+0 (secure 0 0 0) 0x7f352c07cda0 con 0x7f35380f6ff0 2021-11-03T17:14:44.149 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.142+0000 7f353659c700 1 --2- 172.21.15.27:0/40528563 >> [v2:172.21.15.115:6828/1733074450,v1:172.21.15.115:6829/1733074450] conn(0x7f35200602e0 0x7f35200627a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:14:44.149 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.143+0000 7f353659c700 1 --2- 172.21.15.27:0/40528563 >> [v2:172.21.15.115:6828/1733074450,v1:172.21.15.115:6829/1733074450] conn(0x7f35200602e0 0x7f35200627a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f3528004ce0 tx=0x7f352800b040).ready entity=mgr.24395 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:14:44.149 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.146+0000 7f3526ffd700 1 -- 172.21.15.27:0/40528563 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156383 (secure 0 0 0) 0x7f352c04f630 con 0x7f35380f6ff0 2021-11-03T17:14:44.357 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.362+0000 7f353cb21700 1 -- 172.21.15.27:0/40528563 --> [v2:172.21.15.115:6828/1733074450,v1:172.21.15.115:6829/1733074450] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f35380fefe0 con 0x7f35200602e0 2021-11-03T17:14:44.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.378+0000 7f3526ffd700 1 -- 172.21.15.27:0/40528563 <== mgr.24395 v2:172.21.15.115:6828/1733074450 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3388 (secure 0 0 0) 0x7f35380fefe0 con 0x7f35200602e0 2021-11-03T17:14:44.374 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:14:44.375 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (6m) 11s ago 7m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:14:44.375 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (7m) 11s ago 7m 7013k - 16.2.4 8d91d370c2b8 d494472a47c0 2021-11-03T17:14:44.375 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (6m) 15s ago 6m 7134k - 16.2.4 8d91d370c2b8 f84fd4042530 2021-11-03T17:14:44.375 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (6m) 11s ago 7m 36.1M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:14:44.376 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (2m) 11s ago 2m 14.5M - 16.2.4 8d91d370c2b8 509a42d9cc82 2021-11-03T17:14:44.376 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (2m) 11s ago 2m 1720M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:14:44.376 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (2m) 15s ago 2m 17.1M - 16.2.4 8d91d370c2b8 bfd990d6415e 2021-11-03T17:14:44.376 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 unknown 15s ago 2m - - 2021-11-03T17:14:44.376 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:9283 running (8m) 11s ago 8m 105M - 16.2.4 8d91d370c2b8 999298a5769c 2021-11-03T17:14:44.377 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8443,9283 running (30s) 15s ago 6m 446M - 17.0.0-8776-g6e90212d 50416d063274 c53838a65398 2021-11-03T17:14:44.377 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (8m) 11s ago 8m 95.4M 2048M 16.2.4 8d91d370c2b8 060d5938214f 2021-11-03T17:14:44.377 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (6m) 15s ago 6m 77.4M 2048M 16.2.4 8d91d370c2b8 319323f09834 2021-11-03T17:14:44.377 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (7m) 11s ago 7m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:14:44.378 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (6m) 15s ago 6m 18.2M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:14:44.378 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (5m) 11s ago 5m 487M 4096M 16.2.4 8d91d370c2b8 17f79a815a0f 2021-11-03T17:14:44.378 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (5m) 11s ago 5m 571M 4096M 16.2.4 8d91d370c2b8 422be0cac0db 2021-11-03T17:14:44.378 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (5m) 11s ago 5m 436M 4096M 16.2.4 8d91d370c2b8 c211ef5551c9 2021-11-03T17:14:44.379 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (4m) 15s ago 4m 624M 4096M 16.2.4 8d91d370c2b8 b1486ec1b619 2021-11-03T17:14:44.379 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (4m) 15s ago 4m 451M 4096M 16.2.4 8d91d370c2b8 8b91230c69e5 2021-11-03T17:14:44.379 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (4m) 15s ago 4m 639M 4096M 16.2.4 8d91d370c2b8 4a54c4d6b6f5 2021-11-03T17:14:44.379 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (6m) 11s ago 6m 38.4M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:14:44.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.381+0000 7f353cb21700 1 -- 172.21.15.27:0/40528563 >> [v2:172.21.15.115:6828/1733074450,v1:172.21.15.115:6829/1733074450] conn(0x7f35200602e0 msgr2=0x7f35200627a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:14:44.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.381+0000 7f353cb21700 1 --2- 172.21.15.27:0/40528563 >> [v2:172.21.15.115:6828/1733074450,v1:172.21.15.115:6829/1733074450] conn(0x7f35200602e0 0x7f35200627a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f3528004ce0 tx=0x7f352800b040).stop 2021-11-03T17:14:44.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.381+0000 7f353cb21700 1 -- 172.21.15.27:0/40528563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35380f6ff0 msgr2=0x7f35381073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:14:44.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.381+0000 7f353cb21700 1 --2- 172.21.15.27:0/40528563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35380f6ff0 0x7f35381073e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f352c009990 tx=0x7f352c009b30).stop 2021-11-03T17:14:44.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.381+0000 7f353cb21700 1 -- 172.21.15.27:0/40528563 shutdown_connections 2021-11-03T17:14:44.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.382+0000 7f353cb21700 1 --2- 172.21.15.27:0/40528563 >> [v2:172.21.15.115:6828/1733074450,v1:172.21.15.115:6829/1733074450] conn(0x7f35200602e0 0x7f35200627a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:44.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.382+0000 7f353cb21700 1 --2- 172.21.15.27:0/40528563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35380f66e0 0x7f3538106ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:44.382 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.382+0000 7f353cb21700 1 --2- 172.21.15.27:0/40528563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35380f6ff0 0x7f35381073e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:44.382 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.382+0000 7f353cb21700 1 -- 172.21.15.27:0/40528563 >> 172.21.15.27:0/40528563 conn(0x7f35380f1f00 msgr2=0x7f35380fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:14:44.382 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.382+0000 7f353cb21700 1 -- 172.21.15.27:0/40528563 shutdown_connections 2021-11-03T17:14:44.382 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.382+0000 7f353cb21700 1 -- 172.21.15.27:0/40528563 wait complete. 2021-11-03T17:14:44.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.493+0000 7f80ab9eb700 1 -- 172.21.15.27:0/3680786125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80a40f53e0 msgr2=0x7f80a40f7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:14:44.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.493+0000 7f80ab9eb700 1 --2- 172.21.15.27:0/3680786125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80a40f53e0 0x7f80a40f7d20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f8094004660 tx=0x7f8094009b30).stop 2021-11-03T17:14:44.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.493+0000 7f80ab9eb700 1 -- 172.21.15.27:0/3680786125 shutdown_connections 2021-11-03T17:14:44.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.493+0000 7f80ab9eb700 1 --2- 172.21.15.27:0/3680786125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80a40f53e0 0x7f80a40f7d20 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:44.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.493+0000 7f80ab9eb700 1 --2- 172.21.15.27:0/3680786125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80a40f4a40 0x7f80a40f4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:44.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.493+0000 7f80ab9eb700 1 -- 172.21.15.27:0/3680786125 >> 172.21.15.27:0/3680786125 conn(0x7f80a40f04c0 msgr2=0x7f80a40f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:14:44.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.493+0000 7f80ab9eb700 1 -- 172.21.15.27:0/3680786125 shutdown_connections 2021-11-03T17:14:44.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.493+0000 7f80ab9eb700 1 -- 172.21.15.27:0/3680786125 wait complete. 2021-11-03T17:14:44.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.494+0000 7f80ab9eb700 1 Processor -- start 2021-11-03T17:14:44.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.494+0000 7f80ab9eb700 1 -- start start 2021-11-03T17:14:44.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.494+0000 7f80ab9eb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80a40f4a40 0x7f80a40f7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:14:44.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.494+0000 7f80ab9eb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80a40f53e0 0x7f80a40f5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:14:44.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.494+0000 7f80ab9eb700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a40f6010 con 0x7f80a40f4a40 2021-11-03T17:14:44.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.494+0000 7f80ab9eb700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80a40f6150 con 0x7f80a40f53e0 2021-11-03T17:14:44.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.494+0000 7f80a9787700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80a40f4a40 0x7f80a40f7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:14:44.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.494+0000 7f80a9787700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80a40f4a40 0x7f80a40f7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55440/0 (socket says 172.21.15.27:55440) 2021-11-03T17:14:44.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.494+0000 7f80a9787700 1 -- 172.21.15.27:0/1336584369 learned_addr learned my addr 172.21.15.27:0/1336584369 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:14:44.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.494+0000 7f80a8f86700 1 --2- 172.21.15.27:0/1336584369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80a40f53e0 0x7f80a40f5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:14:44.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.494+0000 7f80a9787700 1 -- 172.21.15.27:0/1336584369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80a40f53e0 msgr2=0x7f80a40f5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:14:44.494 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.494+0000 7f80a9787700 1 --2- 172.21.15.27:0/1336584369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80a40f53e0 0x7f80a40f5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:44.494 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.494+0000 7f80a9787700 1 -- 172.21.15.27:0/1336584369 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8094005040 con 0x7f80a40f4a40 2021-11-03T17:14:44.494 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.494+0000 7f80a9787700 1 --2- 172.21.15.27:0/1336584369 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80a40f4a40 0x7f80a40f7630 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f80a00026d0 tx=0x7f80a0009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:14:44.494 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.495+0000 7f8099ffb700 1 -- 172.21.15.27:0/1336584369 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80a00079c0 con 0x7f80a40f4a40 2021-11-03T17:14:44.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.495+0000 7f80ab9eb700 1 -- 172.21.15.27:0/1336584369 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80a40f6430 con 0x7f80a40f4a40 2021-11-03T17:14:44.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.495+0000 7f8099ffb700 1 -- 172.21.15.27:0/1336584369 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f80a000f040 con 0x7f80a40f4a40 2021-11-03T17:14:44.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.495+0000 7f8099ffb700 1 -- 172.21.15.27:0/1336584369 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80a00148b0 con 0x7f80a40f4a40 2021-11-03T17:14:44.496 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.495+0000 7f80ab9eb700 1 -- 172.21.15.27:0/1336584369 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80a410f270 con 0x7f80a40f4a40 2021-11-03T17:14:44.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.496+0000 7f80ab9eb700 1 -- 172.21.15.27:0/1336584369 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80a4045ad0 con 0x7f80a40f4a40 2021-11-03T17:14:44.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.496+0000 7f8099ffb700 1 -- 172.21.15.27:0/1336584369 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 23) v1 ==== 77155+0+0 (secure 0 0 0) 0x7f80a0007b20 con 0x7f80a40f4a40 2021-11-03T17:14:44.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.497+0000 7f8099ffb700 1 --2- 172.21.15.27:0/1336584369 >> [v2:172.21.15.115:6828/1733074450,v1:172.21.15.115:6829/1733074450] conn(0x7f8090060280 0x7f8090062740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:14:44.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.497+0000 7f8099ffb700 1 -- 172.21.15.27:0/1336584369 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5601+0+0 (secure 0 0 0) 0x7f80a007b2b0 con 0x7f80a40f4a40 2021-11-03T17:14:44.498 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.497+0000 7f80a8f86700 1 --2- 172.21.15.27:0/1336584369 >> [v2:172.21.15.115:6828/1733074450,v1:172.21.15.115:6829/1733074450] conn(0x7f8090060280 0x7f8090062740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:14:44.498 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.497+0000 7f80a8f86700 1 --2- 172.21.15.27:0/1336584369 >> [v2:172.21.15.115:6828/1733074450,v1:172.21.15.115:6829/1733074450] conn(0x7f8090060280 0x7f8090062740 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f8094004630 tx=0x7f8094009b30).ready entity=mgr.24395 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:14:44.498 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.501+0000 7f8099ffb700 1 -- 172.21.15.27:0/1336584369 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156383 (secure 0 0 0) 0x7f80a004daa0 con 0x7f80a40f4a40 2021-11-03T17:14:44.637 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:44 smithi027 conmon[32322]: cluster 2021-11-03T17:14:43.284024+0000 2021-11-03T17:14:44.638 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:44 smithi027 conmon[32322]: mon.smithi027 (mon.0) 631 : cluster [DBG] mds.? [v2:172.21.15.27:6828/1337044739,v1:172.21.15.27:6829/1337044739] up:resolve 2021-11-03T17:14:44.639 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:44 smithi027 conmon[32322]: cluster 2021-11-03T17:14:43.284074+0000 mon.smithi027 (mon.0) 632 : cluster [DBG] fsmap cephfs:2/2 {0=cephfs.smithi027.fksixo=up:resolve,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:14:44.639 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:44 smithi115 conmon[34670]: cluster 2021-11-03T17:14:43.284024+0000 mon.smithi027 (mon.0) 631 : cluster [DBG] mds.? [v2:172.21.15.27:6828/1337044739,v1:172.21.15.27:6829/1337044739] up:resolve 2021-11-03T17:14:44.640 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:44 smithi115 conmon[34670]: cluster 2021-11-03T17:14:43.284074+0000 mon.smithi027 (mon.0) 632 : cluster [DBG] fsmap cephfs:2/2 {0=cephfs.smithi027.fksixo=up:resolve,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:14:44.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.789+0000 7f80ab9eb700 1 -- 172.21.15.27:0/1336584369 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f80a40ff0f0 con 0x7f80a40f4a40 2021-11-03T17:14:44.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.789+0000 7f8099ffb700 1 -- 172.21.15.27:0/1336584369 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+783 (secure 0 0 0) 0x7f80a004d690 con 0x7f80a40f4a40 2021-11-03T17:14:44.785 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:14:44.786 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:14:44.786 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 2 2021-11-03T17:14:44.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:14:44.787 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:14:44.788 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 1, 2021-11-03T17:14:44.788 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 1 2021-11-03T17:14:44.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:14:44.789 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:14:44.789 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 6 2021-11-03T17:14:44.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:14:44.790 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:14:44.791 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:14:44.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:14:44.792 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:14:44.792 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 13, 2021-11-03T17:14:44.792 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 1 2021-11-03T17:14:44.793 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:14:44.793 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:14:44.795 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.791+0000 7f80ab9eb700 1 -- 172.21.15.27:0/1336584369 >> [v2:172.21.15.115:6828/1733074450,v1:172.21.15.115:6829/1733074450] conn(0x7f8090060280 msgr2=0x7f8090062740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:14:44.795 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.791+0000 7f80ab9eb700 1 --2- 172.21.15.27:0/1336584369 >> [v2:172.21.15.115:6828/1733074450,v1:172.21.15.115:6829/1733074450] conn(0x7f8090060280 0x7f8090062740 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f8094004630 tx=0x7f8094009b30).stop 2021-11-03T17:14:44.795 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.791+0000 7f80ab9eb700 1 -- 172.21.15.27:0/1336584369 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80a40f4a40 msgr2=0x7f80a40f7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:14:44.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.791+0000 7f80ab9eb700 1 --2- 172.21.15.27:0/1336584369 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80a40f4a40 0x7f80a40f7630 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f80a00026d0 tx=0x7f80a0009de0).stop 2021-11-03T17:14:44.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.791+0000 7f80ab9eb700 1 -- 172.21.15.27:0/1336584369 shutdown_connections 2021-11-03T17:14:44.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.791+0000 7f80ab9eb700 1 --2- 172.21.15.27:0/1336584369 >> [v2:172.21.15.115:6828/1733074450,v1:172.21.15.115:6829/1733074450] conn(0x7f8090060280 0x7f8090062740 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:44.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.791+0000 7f80ab9eb700 1 --2- 172.21.15.27:0/1336584369 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80a40f4a40 0x7f80a40f7630 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:44.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.791+0000 7f80ab9eb700 1 --2- 172.21.15.27:0/1336584369 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80a40f53e0 0x7f80a40f5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:44.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.791+0000 7f80ab9eb700 1 -- 172.21.15.27:0/1336584369 >> 172.21.15.27:0/1336584369 conn(0x7f80a40f04c0 msgr2=0x7f80a40fb770 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:14:44.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.791+0000 7f80ab9eb700 1 -- 172.21.15.27:0/1336584369 shutdown_connections 2021-11-03T17:14:44.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.791+0000 7f80ab9eb700 1 -- 172.21.15.27:0/1336584369 wait complete. 2021-11-03T17:14:44.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.892+0000 7f1a8fcd0700 1 -- 172.21.15.27:0/1267813773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a880f7960 msgr2=0x7f1a880f7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:14:44.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.892+0000 7f1a8fcd0700 1 --2- 172.21.15.27:0/1267813773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a880f7960 0x7f1a880f7dc0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f1a7c004660 tx=0x7f1a7c009b30).stop 2021-11-03T17:14:44.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.892+0000 7f1a8fcd0700 1 -- 172.21.15.27:0/1267813773 shutdown_connections 2021-11-03T17:14:44.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.892+0000 7f1a8fcd0700 1 --2- 172.21.15.27:0/1267813773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a880f7960 0x7f1a880f7dc0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:44.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.892+0000 7f1a8fcd0700 1 --2- 172.21.15.27:0/1267813773 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a880f66c0 0x7f1a880f6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:44.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.892+0000 7f1a8fcd0700 1 -- 172.21.15.27:0/1267813773 >> 172.21.15.27:0/1267813773 conn(0x7f1a880f1e20 msgr2=0x7f1a880f4240 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:14:44.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.892+0000 7f1a8fcd0700 1 -- 172.21.15.27:0/1267813773 shutdown_connections 2021-11-03T17:14:44.891 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.892+0000 7f1a8fcd0700 1 -- 172.21.15.27:0/1267813773 wait complete. 2021-11-03T17:14:44.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.893+0000 7f1a8fcd0700 1 Processor -- start 2021-11-03T17:14:44.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.893+0000 7f1a8fcd0700 1 -- start start 2021-11-03T17:14:44.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.893+0000 7f1a8fcd0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a880f66c0 0x7f1a88106f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:14:44.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.893+0000 7f1a8fcd0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a880f7960 0x7f1a88107430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:14:44.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.893+0000 7f1a8fcd0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a881079e0 con 0x7f1a880f7960 2021-11-03T17:14:44.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.893+0000 7f1a8fcd0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a88107b20 con 0x7f1a880f66c0 2021-11-03T17:14:44.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.893+0000 7f1a8d26b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a880f7960 0x7f1a88107430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:14:44.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.893+0000 7f1a8d26b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a880f7960 0x7f1a88107430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55454/0 (socket says 172.21.15.27:55454) 2021-11-03T17:14:44.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.893+0000 7f1a8d26b700 1 -- 172.21.15.27:0/1402176068 learned_addr learned my addr 172.21.15.27:0/1402176068 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:14:44.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.893+0000 7f1a8da6c700 1 --2- 172.21.15.27:0/1402176068 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a880f66c0 0x7f1a88106f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:14:44.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.893+0000 7f1a8d26b700 1 -- 172.21.15.27:0/1402176068 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a880f66c0 msgr2=0x7f1a88106f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:14:44.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.893+0000 7f1a8d26b700 1 --2- 172.21.15.27:0/1402176068 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a880f66c0 0x7f1a88106f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:44.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.893+0000 7f1a8d26b700 1 -- 172.21.15.27:0/1402176068 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a7c005040 con 0x7f1a880f7960 2021-11-03T17:14:44.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.893+0000 7f1a8d26b700 1 --2- 172.21.15.27:0/1402176068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a880f7960 0x7f1a88107430 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f1a7c009830 tx=0x7f1a7c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:14:44.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.894+0000 7f1a867fc700 1 -- 172.21.15.27:0/1402176068 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a7c018070 con 0x7f1a880f7960 2021-11-03T17:14:44.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.894+0000 7f1a867fc700 1 -- 172.21.15.27:0/1402176068 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f1a7c007ba0 con 0x7f1a880f7960 2021-11-03T17:14:44.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.894+0000 7f1a8fcd0700 1 -- 172.21.15.27:0/1402176068 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a88108460 con 0x7f1a880f7960 2021-11-03T17:14:44.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.894+0000 7f1a867fc700 1 -- 172.21.15.27:0/1402176068 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a7c01c3c0 con 0x7f1a880f7960 2021-11-03T17:14:44.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.894+0000 7f1a8fcd0700 1 -- 172.21.15.27:0/1402176068 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a881089c0 con 0x7f1a880f7960 2021-11-03T17:14:44.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.895+0000 7f1a8fcd0700 1 -- 172.21.15.27:0/1402176068 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a88045ad0 con 0x7f1a880f7960 2021-11-03T17:14:44.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.895+0000 7f1a867fc700 1 -- 172.21.15.27:0/1402176068 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 23) v1 ==== 77155+0+0 (secure 0 0 0) 0x7f1a7c007d10 con 0x7f1a880f7960 2021-11-03T17:14:44.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.896+0000 7f1a867fc700 1 --2- 172.21.15.27:0/1402176068 >> [v2:172.21.15.115:6828/1733074450,v1:172.21.15.115:6829/1733074450] conn(0x7f1a70060280 0x7f1a70062740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:14:44.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.896+0000 7f1a867fc700 1 -- 172.21.15.27:0/1402176068 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5601+0+0 (secure 0 0 0) 0x7f1a7c07d2d0 con 0x7f1a880f7960 2021-11-03T17:14:44.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.896+0000 7f1a8da6c700 1 --2- 172.21.15.27:0/1402176068 >> [v2:172.21.15.115:6828/1733074450,v1:172.21.15.115:6829/1733074450] conn(0x7f1a70060280 0x7f1a70062740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:14:44.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.897+0000 7f1a8da6c700 1 --2- 172.21.15.27:0/1402176068 >> [v2:172.21.15.115:6828/1733074450,v1:172.21.15.115:6829/1733074450] conn(0x7f1a70060280 0x7f1a70062740 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f1a880f77c0 tx=0x7f1a7800b040).ready entity=mgr.24395 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:14:44.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:44.899+0000 7f1a867fc700 1 -- 172.21.15.27:0/1402176068 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156383 (secure 0 0 0) 0x7f1a7c04fb40 con 0x7f1a880f7960 2021-11-03T17:14:45.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:45.116+0000 7f1a8fcd0700 1 -- 172.21.15.27:0/1402176068 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1a88027a00 con 0x7f1a880f7960 2021-11-03T17:14:45.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:45.117+0000 7f1a867fc700 1 -- 172.21.15.27:0/1402176068 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) v1 ==== 76+0+1650 (secure 0 0 0) 0x7f1a7c04f730 con 0x7f1a880f7960 2021-11-03T17:14:45.112 INFO:teuthology.orchestra.run.smithi027.stdout:e20 2021-11-03T17:14:45.112 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:14:45.113 INFO:teuthology.orchestra.run.smithi027.stdout:compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:14:45.113 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:14:45.113 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:14:45.113 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:14:45.113 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:14:45.114 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 20 2021-11-03T17:14:45.114 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 2021-11-03T17:14:45.114 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:14:45.115 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:14:44.281748+0000 2021-11-03T17:14:45.115 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:14:45.115 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:14:45.115 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:14:45.115 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:14:45.116 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:14:45.116 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:14:45.116 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:14:45.116 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 49 2021-11-03T17:14:45.117 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:14:45.117 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 2 2021-11-03T17:14:45.117 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:14:45.118 INFO:teuthology.orchestra.run.smithi027.stdout:up {0=14448,1=14436} 2021-11-03T17:14:45.118 INFO:teuthology.orchestra.run.smithi027.stdout:failed 2021-11-03T17:14:45.118 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:14:45.118 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:14:45.118 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:14:45.119 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:14:45.119 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:14:45.119 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:14:45.119 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:14:45.119 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{0:14448} state up:reconnect seq 42 join_fscid=1 addr [v2:172.21.15.27:6828/1337044739,v1:172.21.15.27:6829/1337044739]] 2021-11-03T17:14:45.120 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 3 export targets 0 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060]] 2021-11-03T17:14:45.120 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:14:45.120 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:14:45.120 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:14:45.121 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:14:45.121 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:24289} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.115:6826/3367380088,v1:172.21.15.115:6827/3367380088]] 2021-11-03T17:14:45.121 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060]] 2021-11-03T17:14:45.122 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:45.119+0000 7f1a8fcd0700 1 -- 172.21.15.27:0/1402176068 >> [v2:172.21.15.115:6828/1733074450,v1:172.21.15.115:6829/1733074450] conn(0x7f1a70060280 msgr2=0x7f1a70062740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:14:45.122 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:45.119+0000 7f1a8fcd0700 1 --2- 172.21.15.27:0/1402176068 >> [v2:172.21.15.115:6828/1733074450,v1:172.21.15.115:6829/1733074450] conn(0x7f1a70060280 0x7f1a70062740 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f1a880f77c0 tx=0x7f1a7800b040).stop 2021-11-03T17:14:45.123 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:45.119+0000 7f1a8fcd0700 1 -- 172.21.15.27:0/1402176068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a880f7960 msgr2=0x7f1a88107430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:14:45.123 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:45.119+0000 7f1a8fcd0700 1 --2- 172.21.15.27:0/1402176068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a880f7960 0x7f1a88107430 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f1a7c009830 tx=0x7f1a7c009b30).stop 2021-11-03T17:14:45.123 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:45.119+0000 7f1a8fcd0700 1 -- 172.21.15.27:0/1402176068 shutdown_connections 2021-11-03T17:14:45.123 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:45.119+0000 7f1a8fcd0700 1 --2- 172.21.15.27:0/1402176068 >> [v2:172.21.15.115:6828/1733074450,v1:172.21.15.115:6829/1733074450] conn(0x7f1a70060280 0x7f1a70062740 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:45.123 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:45.119+0000 7f1a8fcd0700 1 --2- 172.21.15.27:0/1402176068 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a880f66c0 0x7f1a88106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:45.124 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:45.119+0000 7f1a8fcd0700 1 --2- 172.21.15.27:0/1402176068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a880f7960 0x7f1a88107430 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:14:45.124 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:45.119+0000 7f1a8fcd0700 1 -- 172.21.15.27:0/1402176068 >> 172.21.15.27:0/1402176068 conn(0x7f1a880f1e20 msgr2=0x7f1a880fb3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:14:45.124 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:45.119+0000 7f1a8fcd0700 1 -- 172.21.15.27:0/1402176068 shutdown_connections 2021-11-03T17:14:45.124 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:14:45.119+0000 7f1a8fcd0700 1 -- 172.21.15.27:0/1402176068 wait complete. 2021-11-03T17:14:45.125 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 20 2021-11-03T17:14:45.426 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:45 smithi027 conmon[32322]: cluster 2021-11-03T17:14:43.896978 2021-11-03T17:14:45.426 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:45 smithi027 conmon[32322]: +0000 mgr.smithi115.tcyodz (mgr.24395) 25 : cluster [DBG] pgmap v17: 65 pgs: 65 active+clean; 2.9 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail; 40 MiB/s rd, 49 MiB/s wr, 87 op/s 2021-11-03T17:14:45.427 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:45 smithi027 conmon[32322]: audit 2021-11-03T17:14:44.036093+0000 mgr.smithi115.tcyodz (mgr.24395) 26 : audit 2021-11-03T17:14:45.427 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:45 smithi027 conmon[32322]: [DBG] from='client.14606 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:14:45.427 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:45 smithi027 conmon[32322]: cluster 2021-11-03T 2021-11-03T17:14:45.427 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:45 smithi027 conmon[32322]: 17:14:44.288049+0000 mon.smithi027 (mon.0) 633 : cluster [DBG] 2021-11-03T17:14:45.428 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:45 smithi027 conmon[32322]: mds.? [v2:172.21.15.27:6828/1337044739,v1:172.21.15.27:6829/1337044739] up:reconnect 2021-11-03T17:14:45.428 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:45 smithi027 conmon[32322]: cluster 2021-11-03T17 2021-11-03T17:14:45.428 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:45 smithi027 conmon[32322]: :14:44.288118+0000 mon.smithi027 (mon.0 2021-11-03T17:14:45.428 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:45 smithi027 conmon[32322]: ) 634 : cluster [DBG] fsmap cephfs:2/2 {0=cephfs.smithi027.fksixo=up:reconnect,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:14:45.428 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:45 smithi027 conmon[32322]: audit 2021- 2021-11-03T17:14:45.429 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:45 smithi027 conmon[32322]: 11-03T17:14:44.364041+0000 mgr.smithi115.tcyodz 2021-11-03T17:14:45.429 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:45 smithi027 conmon[32322]: (mgr.24395) 27 : audit [DBG] from='client.14610 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:14:45.429 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:45 smithi027 conmon[32322]: audit 2021-11-03T17:14:44. 2021-11-03T17:14:45.429 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:45 smithi027 conmon[32322]: 789984+0000 mon.smithi027 (mon.0) 635 : audit [DBG] from='client.? 172.21.15.27:0/1336584369' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:14:45.430 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:45 smithi027 conmon[32322]: audit 2021-11-03T17 2021-11-03T17:14:45.430 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:45 smithi027 conmon[32322]: :14:45.117639+0000 mon.smithi027 (mon.0) 636 : audit 2021-11-03T17:14:45.430 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:45 smithi027 conmon[32322]: [DBG] from='client.? 172.21.15.27:0/1402176068' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:14:45.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:45 smithi115 conmon[34670]: cluster 2021-11-03T17:14:43.896978+0000 mgr.smithi115.tcyodz (mgr.24395) 25 : cluster [DBG] pgmap v17: 65 pgs: 65 active+clean; 2.9 GiB data, 8.6 GiB used, 528 GiB / 536 GiB avail; 40 MiB/s rd, 49 MiB/s wr, 87 op/s 2021-11-03T17:14:45.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:45 smithi115 conmon[34670]: audit 2021-11-03T17: 2021-11-03T17:14:45.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:45 smithi115 conmon[34670]: 14:44.036093+0000 mgr.smithi115.tcyodz (mgr.24395) 26 : audit [DBG] from='client.14606 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:14:45.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:45 smithi115 conmon[34670]: cluster 2021-11-03T17:14:44.288049+0000 mon.smithi027 (mon.0) 633 : cluster [DBG] mds.? [v2:172.21.15.27:6828/1337044739,v1:172.21.15.27:6829/1337044739] up:reconnect 2021-11-03T17:14:45.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:45 smithi115 conmon[34670]: cluster 2021-11-03T17:14:44.288118 2021-11-03T17:14:45.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:45 smithi115 conmon[34670]: +0000 mon.smithi027 (mon.0) 634 : cluster 2021-11-03T17:14:45.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:45 smithi115 conmon[34670]: [DBG] fsmap cephfs:2/2 {0=cephfs.smithi027.fksixo=up:reconnect,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:14:45.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:45 smithi115 conmon[34670]: audit 2021-11-03 2021-11-03T17:14:45.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:45 smithi115 conmon[34670]: T17:14:44.364041+0000 mgr.smithi115.tcyodz ( 2021-11-03T17:14:45.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:45 smithi115 conmon[34670]: mgr.24395) 27 : audit [DBG] from='client.14610 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:14:45.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:45 smithi115 conmon[34670]: audit 2021-11-03T17:14:45.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:45 smithi115 conmon[34670]: 2021-11-03T17:14:44.789984+0000 mon.smithi027 ( 2021-11-03T17:14:45.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:45 smithi115 conmon[34670]: mon.0) 635 : audit [DBG] from='client.? 172.21.15.27:0/1336584369' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:14:45.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:45 smithi115 conmon[34670]: audit 2021- 2021-11-03T17:14:45.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:45 smithi115 conmon[34670]: 11-03T17:14:45.117639 2021-11-03T17:14:45.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:45 smithi115 conmon[34670]: +0000 mon.smithi027 (mon.0) 636 2021-11-03T17:14:45.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:45 smithi115 conmon[34670]: : audit [DBG] from='client.? 172.21.15.27:0/1402176068' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:14:47.510 INFO:tasks.workunit.client.0.smithi027.stdout:0/856: sync 2021-11-03T17:14:47.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:47 smithi115 conmon[34670]: cluster 2021-11-03T17:14:45.897566+0000 mgr.smithi115.tcyodz (mgr.24395) 28 : cluster [DBG] 2021-11-03T17:14:47.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:47 smithi115 conmon[34670]: pgmap v18: 65 pgs: 65 active+clean; 3.0 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 40 MiB/s rd, 47 MiB/s wr, 85 op/s 2021-11-03T17:14:47.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:47 smithi115 conmon[34670]: audit 2021-11-03T17:14:46.746546+0000 mon.smithi027 (mon.0) 637 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' 2021-11-03T17:14:47.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:47 smithi115 conmon[34670]: audit 2021-11-03T17:14:46.748463+0000 mon.smithi115 ( 2021-11-03T17:14:47.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:47 smithi115 conmon[34670]: mon.1) 57 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:14:47.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:47 smithi027 conmon[32322]: cluster 2021-11-03T17:14:45.897566+0000 mgr.smithi115.tcyodz (mgr.24395) 2021-11-03T17:14:47.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:47 smithi027 conmon[32322]: 28 : cluster [DBG] pgmap v18: 65 pgs: 65 active+clean; 3.0 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 40 MiB/s rd, 47 MiB/s wr, 85 op/s 2021-11-03T17:14:47.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:47 smithi027 conmon[32322]: audit 2021-11-03T17:14:46.746546+0000 mon.smithi027 (mon.0) 637 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' 2021-11-03T17:14:47.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:47 smithi027 conmon[32322]: audit 2021-11-03T17:14:46.748463+0000 mon.smithi115 (mon.1) 57 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:14:47.854 INFO:tasks.workunit.client.0.smithi027.stdout:9/723: sync 2021-11-03T17:14:47.854 INFO:tasks.workunit.client.0.smithi027.stdout:7/567: sync 2021-11-03T17:14:48.080 INFO:tasks.workunit.client.0.smithi027.stdout:5/963: sync 2021-11-03T17:14:48.200 INFO:tasks.workunit.client.0.smithi027.stdout:5/964: creat d2/d4/d61/f141 x:0 0 0 2021-11-03T17:14:48.201 INFO:tasks.workunit.client.0.smithi027.stdout:9/724: mknod db/d1b/d2b/d5e/c10b 1 2021-11-03T17:14:48.201 INFO:tasks.workunit.client.0.smithi027.stdout:0/857: dread d9/da/d31/f36 [0,4194304] 0 2021-11-03T17:14:48.253 INFO:tasks.workunit.client.0.smithi027.stdout:0/858: rmdir d9/da/d30/dac/dba 39 2021-11-03T17:14:48.307 INFO:tasks.workunit.client.0.smithi027.stdout:9/725: rename db/d1b/d23/l80 to db/d1b/d2b/d41/d95/l10c 0 2021-11-03T17:14:48.363 INFO:tasks.workunit.client.0.smithi027.stdout:3/887: sync 2021-11-03T17:14:48.441 INFO:tasks.workunit.client.0.smithi027.stdout:1/855: sync 2021-11-03T17:14:48.441 INFO:tasks.workunit.client.0.smithi027.stdout:4/702: sync 2021-11-03T17:14:48.483 INFO:tasks.workunit.client.0.smithi027.stdout:7/568: write d0/d2/d10/f95 [962330,39761] 0 2021-11-03T17:14:48.484 INFO:tasks.workunit.client.0.smithi027.stdout:5/965: write d2/d12/f5c [1751494,22071] 0 2021-11-03T17:14:48.485 INFO:tasks.workunit.client.0.smithi027.stdout:0/859: write d9/f72 [3769130,53191] 0 2021-11-03T17:14:48.492 INFO:tasks.workunit.client.0.smithi027.stdout:9/726: truncate db/d1b/d2b/d41/d95/d9f/f100 1016128 0 2021-11-03T17:14:48.498 INFO:tasks.workunit.client.0.smithi027.stdout:7/569: mknod d0/d2d/cbd 1 2021-11-03T17:14:48.509 INFO:tasks.workunit.client.0.smithi027.stdout:9/727: truncate db/d1b/fb0 999958 0 2021-11-03T17:14:48.518 INFO:tasks.workunit.client.0.smithi027.stdout:7/570: mknod d0/d2/d10/d1e/d20/cbe 1 2021-11-03T17:14:48.519 INFO:tasks.workunit.client.0.smithi027.stdout:3/888: dwrite d1/f34 [0,4194304] 0 2021-11-03T17:14:48.521 INFO:tasks.workunit.client.0.smithi027.stdout:5/966: mknod d2/d2c/d3c/d48/d65/d80/c142 1 2021-11-03T17:14:48.527 INFO:tasks.workunit.client.0.smithi027.stdout:9/728: dread db/f14 [0,4194304] 0 2021-11-03T17:14:48.528 INFO:tasks.workunit.client.0.smithi027.stdout:0/860: dwrite d9/d12/f1d [0,4194304] 0 2021-11-03T17:14:48.533 INFO:tasks.workunit.client.0.smithi027.stdout:3/889: truncate d1/db/f35 2849275 0 2021-11-03T17:14:48.535 INFO:tasks.workunit.client.0.smithi027.stdout:0/861: fdatasync d9/da/d30/f5e 0 2021-11-03T17:14:48.536 INFO:tasks.workunit.client.0.smithi027.stdout:1/856: getdents d6/d11/dac/d79/d25/d28/d5c/d7a 0 2021-11-03T17:14:48.551 INFO:tasks.workunit.client.0.smithi027.stdout:5/967: fdatasync d2/d4/d4e/d67/d8e/f126 0 2021-11-03T17:14:48.553 INFO:tasks.workunit.client.0.smithi027.stdout:7/571: mknod d0/d2/d1c/d4d/d6e/d8b/cbf 1 2021-11-03T17:14:48.554 INFO:tasks.workunit.client.0.smithi027.stdout:0/862: chown d9/da/d64/dcc 1227 1 2021-11-03T17:14:48.558 INFO:tasks.workunit.client.0.smithi027.stdout:9/729: mknod db/d1b/d1c/d1d/ddb/c10d 1 2021-11-03T17:14:48.566 INFO:tasks.workunit.client.0.smithi027.stdout:0/863: fdatasync d9/da/d30/dac/fbe 0 2021-11-03T17:14:48.566 INFO:tasks.workunit.client.0.smithi027.stdout:1/857: symlink d6/d11/de2/l11b 0 2021-11-03T17:14:48.572 INFO:tasks.workunit.client.0.smithi027.stdout:5/968: truncate d2/d2c/d3c/d48/ffc 4220984 0 2021-11-03T17:14:48.576 INFO:tasks.workunit.client.0.smithi027.stdout:7/572: unlink d0/d2/d10/d1e/d20/l76 0 2021-11-03T17:14:48.578 INFO:tasks.workunit.client.0.smithi027.stdout:3/890: readlink d1/db/db2/dbd/d11b/lb3 0 2021-11-03T17:14:48.580 INFO:tasks.workunit.client.0.smithi027.stdout:4/703: write d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/f93 [513306,90278] 0 2021-11-03T17:14:48.581 INFO:tasks.workunit.client.0.smithi027.stdout:0/864: fdatasync d9/d12/fa6 0 2021-11-03T17:14:48.585 INFO:tasks.workunit.client.0.smithi027.stdout:9/730: write db/fd [3649652,125190] 0 2021-11-03T17:14:48.587 INFO:tasks.workunit.client.0.smithi027.stdout:5/969: chown d2/d4/f13 56 1 2021-11-03T17:14:48.593 INFO:tasks.workunit.client.0.smithi027.stdout:0/865: chown d9/da/f8b 16492034 1 2021-11-03T17:14:48.596 INFO:tasks.workunit.client.0.smithi027.stdout:5/970: chown d2/d4/d4e/d67/l31 19145 1 2021-11-03T17:14:48.600 INFO:tasks.workunit.client.0.smithi027.stdout:9/731: chown db/d1b/d2b/dc7/le2 13035 1 2021-11-03T17:14:48.601 INFO:tasks.workunit.client.0.smithi027.stdout:5/971: chown d2/d2c/d3c/d48/d65/d80/l10d 29 1 2021-11-03T17:14:48.603 INFO:tasks.workunit.client.0.smithi027.stdout:0/866: mkdir d9/da/d64/d122 0 2021-11-03T17:14:48.605 INFO:tasks.workunit.client.0.smithi027.stdout:1/858: creat d6/d11/dac/d79/db0/de5/d10f/f11c x:0 0 0 2021-11-03T17:14:48.608 INFO:tasks.workunit.client.0.smithi027.stdout:7/573: dwrite d0/d2d/fbc [0,4194304] 0 2021-11-03T17:14:48.610 INFO:tasks.workunit.client.0.smithi027.stdout:9/732: dread - db/d1b/d1c/d1d/d40/f10a zero size 2021-11-03T17:14:48.613 INFO:tasks.workunit.client.0.smithi027.stdout:7/574: mknod d0/d2/d1c/cc0 1 2021-11-03T17:14:48.615 INFO:tasks.workunit.client.0.smithi027.stdout:4/704: link d6/d9/d16/d36/d70/f71 d6/d24/dbe/fea 0 2021-11-03T17:14:48.619 INFO:tasks.workunit.client.0.smithi027.stdout:0/867: stat d9/d12/d8f/fbd 0 2021-11-03T17:14:48.621 INFO:tasks.workunit.client.0.smithi027.stdout:9/733: mknod db/d1b/d23/c10e 1 2021-11-03T17:14:48.635 INFO:tasks.workunit.client.0.smithi027.stdout:3/891: rename d1/db/d52/d57/f116 to d1/db/d52/d6a/db4/f121 0 2021-11-03T17:14:48.636 INFO:tasks.workunit.client.0.smithi027.stdout:0/868: write d9/da/d64/ff5 [83355,126795] 0 2021-11-03T17:14:48.637 INFO:tasks.workunit.client.0.smithi027.stdout:7/575: truncate d0/d2/d1c/f61 2379187 0 2021-11-03T17:14:48.640 INFO:tasks.workunit.client.0.smithi027.stdout:9/734: chown db/d1b/d2b/d41/d97/fbd 9 1 2021-11-03T17:14:48.658 INFO:tasks.workunit.client.0.smithi027.stdout:1/859: dwrite d6/d11/dac/d79/d37/de0/f16 [4194304,4194304] 0 2021-11-03T17:14:48.660 INFO:tasks.workunit.client.0.smithi027.stdout:9/735: mkdir db/d1b/d1c/d10f 0 2021-11-03T17:14:48.660 INFO:tasks.workunit.client.0.smithi027.stdout:3/892: write d1/db/ff8 [1999688,97399] 0 2021-11-03T17:14:48.660 INFO:tasks.workunit.client.0.smithi027.stdout:7/576: write d0/d2/fd [2610605,63829] 0 2021-11-03T17:14:48.661 INFO:tasks.workunit.client.0.smithi027.stdout:5/972: rename d2/d4/d61/l70 to d2/d12/dad/da8/l143 0 2021-11-03T17:14:48.673 INFO:tasks.workunit.client.0.smithi027.stdout:4/705: link d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d2b/l49 d6/d24/de3/d9f/dd7/leb 0 2021-11-03T17:14:48.673 INFO:tasks.workunit.client.0.smithi027.stdout:3/893: fsync d1/db/f10a 0 2021-11-03T17:14:48.674 INFO:tasks.workunit.client.0.smithi027.stdout:9/736: unlink db/fc8 0 2021-11-03T17:14:48.676 INFO:tasks.workunit.client.0.smithi027.stdout:4/706: getdents d6/d9/d16 0 2021-11-03T17:14:48.678 INFO:tasks.workunit.client.0.smithi027.stdout:0/869: dwrite d9/da/d30/dac/dba/fbc [0,4194304] 0 2021-11-03T17:14:48.684 INFO:tasks.workunit.client.0.smithi027.stdout:7/577: chown d0/d2/d5e/d36/f63 0 1 2021-11-03T17:14:48.690 INFO:tasks.workunit.client.0.smithi027.stdout:5/973: unlink d2/d2c/d3c/dc8/fe7 0 2021-11-03T17:14:48.693 INFO:tasks.workunit.client.0.smithi027.stdout:0/870: dread - d9/d12/f109 zero size 2021-11-03T17:14:48.699 INFO:tasks.workunit.client.0.smithi027.stdout:1/860: rename d6/d11/dac/d79/d37/de0/de4/f34 to d6/d11/dac/d79/dfa/f11d 0 2021-11-03T17:14:48.700 INFO:tasks.workunit.client.0.smithi027.stdout:9/737: fdatasync db/d1b/d2b/dc7/fe5 0 2021-11-03T17:14:48.702 INFO:tasks.workunit.client.0.smithi027.stdout:7/578: creat d0/d2/d1c/d4d/d6e/fc1 x:0 0 0 2021-11-03T17:14:48.704 INFO:tasks.workunit.client.0.smithi027.stdout:4/707: chown d6/d9/d16/d36/d57/l7a 455806241 1 2021-11-03T17:14:48.705 INFO:tasks.workunit.client.0.smithi027.stdout:4/708: chown d6/d9/d16/d36/d57/l7a 12243 1 2021-11-03T17:14:48.710 INFO:tasks.workunit.client.0.smithi027.stdout:3/894: dwrite d1/f41 [0,4194304] 0 2021-11-03T17:14:48.714 INFO:tasks.workunit.client.0.smithi027.stdout:9/738: dread db/f16 [4194304,4194304] 0 2021-11-03T17:14:48.717 INFO:tasks.workunit.client.0.smithi027.stdout:5/974: chown d2/d2c/d3c/d48/d65/dd6/l10f 178295 1 2021-11-03T17:14:48.725 INFO:tasks.workunit.client.0.smithi027.stdout:9/739: creat db/d1b/d2b/d41/d95/f110 x:0 0 0 2021-11-03T17:14:48.731 INFO:tasks.workunit.client.0.smithi027.stdout:1/861: rename d6/d11/f101 to d6/d11/dac/d79/d25/d28/d5c/d7a/f11e 0 2021-11-03T17:14:48.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:48 smithi115 conmon[34670]: cephadm 2021-11-03T17:14:46.737432+0000 mgr.smithi115.tcyodz (mgr.24395) 29 : cephadm [INF] poll took 9499.574 ms: 1 events 2021-11-03T17:14:48.734 INFO:tasks.workunit.client.0.smithi027.stdout:4/709: symlink d6/d9/d8f/lec 0 2021-11-03T17:14:48.736 INFO:tasks.workunit.client.0.smithi027.stdout:3/895: symlink d1/db/d52/d57/de3/l122 0 2021-11-03T17:14:48.738 INFO:tasks.workunit.client.0.smithi027.stdout:5/975: mkdir d2/d1f/d144 0 2021-11-03T17:14:48.740 INFO:tasks.workunit.client.0.smithi027.stdout:4/710: readlink d6/l1c 0 2021-11-03T17:14:48.743 INFO:tasks.workunit.client.0.smithi027.stdout:9/740: link db/fa8 db/d1b/d106/f111 0 2021-11-03T17:14:48.746 INFO:tasks.workunit.client.0.smithi027.stdout:1/862: creat d6/d11/d14/df6/f11f x:0 0 0 2021-11-03T17:14:48.750 INFO:tasks.workunit.client.0.smithi027.stdout:3/896: dwrite d1/f41 [4194304,4194304] 0 2021-11-03T17:14:48.751 INFO:tasks.workunit.client.0.smithi027.stdout:5/976: unlink d2/d4/l5f 0 2021-11-03T17:14:48.753 INFO:tasks.workunit.client.0.smithi027.stdout:4/711: creat d6/d9/d16/d17/ddd/fed x:0 0 0 2021-11-03T17:14:48.760 INFO:tasks.workunit.client.0.smithi027.stdout:3/897: dread - d1/db/db2/dbd/d11b/fe2 zero size 2021-11-03T17:14:48.761 INFO:tasks.workunit.client.0.smithi027.stdout:9/741: write db/d1b/d23/f73 [1044558,18650] 0 2021-11-03T17:14:48.765 INFO:tasks.workunit.client.0.smithi027.stdout:9/742: creat db/d1b/d106/f112 x:0 0 0 2021-11-03T17:14:48.767 INFO:tasks.workunit.client.0.smithi027.stdout:4/712: rename d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d2b to d6/d9/d16/d36/d70/dee 0 2021-11-03T17:14:48.767 INFO:tasks.workunit.client.0.smithi027.stdout:4/713: readlink d6/d9/d16/d36/d57/l7a 0 2021-11-03T17:14:48.768 INFO:tasks.workunit.client.0.smithi027.stdout:1/863: stat d6/d11/dac/d79/d25/d5e/dd1 0 2021-11-03T17:14:48.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:48 smithi027 conmon[32322]: cephadm 2021-11-03T17:14:46.737432 2021-11-03T17:14:48.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:48 smithi027 conmon[32322]: +0000 mgr.smithi115.tcyodz (mgr.24395) 29 : cephadm [INF] poll took 9499.574 ms: 1 events 2021-11-03T17:14:48.772 INFO:tasks.workunit.client.0.smithi027.stdout:5/977: dread d2/d2c/d3c/fdd [0,4194304] 0 2021-11-03T17:14:48.778 INFO:tasks.workunit.client.0.smithi027.stdout:3/898: unlink d1/db/d52/d6a/dd5/d111/l118 0 2021-11-03T17:14:48.788 INFO:tasks.workunit.client.0.smithi027.stdout:1/864: chown d6/d11/dac/d79/d25/d28/d5c/d81/d7d/fca 193860379 1 2021-11-03T17:14:48.789 INFO:tasks.workunit.client.0.smithi027.stdout:4/714: write d6/d9/d16/f32 [3305710,70708] 0 2021-11-03T17:14:48.796 INFO:tasks.workunit.client.0.smithi027.stdout:5/978: rename d2/l8 to d2/d2c/d3c/d48/d65/dd6/l145 0 2021-11-03T17:14:48.797 INFO:tasks.workunit.client.0.smithi027.stdout:3/899: truncate d1/f4c 3949445 0 2021-11-03T17:14:48.806 INFO:tasks.workunit.client.0.smithi027.stdout:4/715: dread d6/d24/db0/f53 [0,4194304] 0 2021-11-03T17:14:48.809 INFO:tasks.workunit.client.0.smithi027.stdout:9/743: rename db/d1b/d2b/f3f to db/d1b/d8c/dcd/f113 0 2021-11-03T17:14:48.809 INFO:tasks.workunit.client.0.smithi027.stdout:1/865: dwrite d6/d11/dac/d79/d37/de0/f16 [4194304,4194304] 0 2021-11-03T17:14:48.814 INFO:tasks.workunit.client.0.smithi027.stdout:3/900: mkdir d1/db/d52/d6a/d7c/d123 0 2021-11-03T17:14:48.818 INFO:tasks.workunit.client.0.smithi027.stdout:4/716: symlink d6/d24/de3/d9f/lef 0 2021-11-03T17:14:48.828 INFO:tasks.workunit.client.0.smithi027.stdout:3/901: truncate d1/db/f5b 1060093 0 2021-11-03T17:14:48.828 INFO:tasks.workunit.client.0.smithi027.stdout:3/902: chown d1/db/db2/dbd/d11b 61874 1 2021-11-03T17:14:48.831 INFO:tasks.workunit.client.0.smithi027.stdout:1/866: rename d6/d11/dac/d79/d37/lce to d6/d11/dac/l120 0 2021-11-03T17:14:48.835 INFO:tasks.workunit.client.0.smithi027.stdout:3/903: fdatasync d1/db/f86 0 2021-11-03T17:14:48.843 INFO:tasks.workunit.client.0.smithi027.stdout:4/717: dread d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/f35 [0,4194304] 0 2021-11-03T17:14:48.844 INFO:tasks.workunit.client.0.smithi027.stdout:1/867: creat d6/d11/dac/d79/d25/d28/dec/f121 x:0 0 0 2021-11-03T17:14:48.853 INFO:tasks.workunit.client.0.smithi027.stdout:5/979: dwrite d2/d4/d4e/d67/d8e/d99/f96 [0,4194304] 0 2021-11-03T17:14:48.854 INFO:tasks.workunit.client.0.smithi027.stdout:9/744: dwrite db/d1b/d2b/d41/fb8 [0,4194304] 0 2021-11-03T17:14:48.855 INFO:tasks.workunit.client.0.smithi027.stdout:4/718: dread d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/f37 [0,4194304] 0 2021-11-03T17:14:48.855 INFO:tasks.workunit.client.0.smithi027.stdout:4/719: readlink d6/d9/d16/d36/d70/dee/l49 0 2021-11-03T17:14:48.858 INFO:tasks.workunit.client.0.smithi027.stdout:3/904: dread d1/db/f60 [0,4194304] 0 2021-11-03T17:14:48.859 INFO:tasks.workunit.client.0.smithi027.stdout:5/980: readlink d2/d4/d4e/d67/d8e/lfa 0 2021-11-03T17:14:48.862 INFO:tasks.workunit.client.0.smithi027.stdout:4/720: mknod d6/d9/d16/d17/ddd/cf0 1 2021-11-03T17:14:48.865 INFO:tasks.workunit.client.0.smithi027.stdout:9/745: stat db/d1b/d1c/d1d/f1e 0 2021-11-03T17:14:48.868 INFO:tasks.workunit.client.0.smithi027.stdout:3/905: creat d1/db/db2/dbd/d11b/dc6/f124 x:0 0 0 2021-11-03T17:14:48.872 INFO:tasks.workunit.client.0.smithi027.stdout:5/981: truncate d2/d2c/d3c/d48/ffc 2301791 0 2021-11-03T17:14:48.874 INFO:tasks.workunit.client.0.smithi027.stdout:1/868: dwrite d6/d11/dac/d79/d25/d28/d5c/d81/f86 [8388608,4194304] 0 2021-11-03T17:14:48.875 INFO:tasks.workunit.client.0.smithi027.stdout:4/721: link d6/d9/d16/d36/fc0 d6/d9/ff1 0 2021-11-03T17:14:48.875 INFO:tasks.workunit.client.0.smithi027.stdout:6/689: sync 2021-11-03T17:14:48.876 INFO:tasks.workunit.client.0.smithi027.stdout:2/783: sync 2021-11-03T17:14:48.878 INFO:tasks.workunit.client.0.smithi027.stdout:9/746: rename db/d1b/d1c/d1d/d40/lf0 to db/d1b/d8c/dcd/l114 0 2021-11-03T17:14:48.879 INFO:tasks.workunit.client.0.smithi027.stdout:9/747: fsync db/d1b/d106/f111 0 2021-11-03T17:14:48.884 INFO:tasks.workunit.client.0.smithi027.stdout:3/906: dread d1/db/d52/d6a/d7c/d88/fc8 [0,4194304] 0 2021-11-03T17:14:48.887 INFO:tasks.workunit.client.0.smithi027.stdout:4/722: dread - d6/d9/d16/d36/fc0 zero size 2021-11-03T17:14:48.899 INFO:tasks.workunit.client.0.smithi027.stdout:9/748: unlink db/fd6 0 2021-11-03T17:14:48.903 INFO:tasks.workunit.client.0.smithi027.stdout:9/749: dread - db/d1b/d2b/d41/d95/fcf zero size 2021-11-03T17:14:48.905 INFO:tasks.workunit.client.0.smithi027.stdout:3/907: creat d1/db/d52/d57/d99/f125 x:0 0 0 2021-11-03T17:14:48.906 INFO:tasks.workunit.client.0.smithi027.stdout:3/908: chown d1/db/d52/d6a/dd5 105 1 2021-11-03T17:14:48.909 INFO:tasks.workunit.client.0.smithi027.stdout:3/909: dread - d1/db/d52/d57/d99/ff7 zero size 2021-11-03T17:14:48.909 INFO:tasks.workunit.client.0.smithi027.stdout:3/910: mknod d1/db/d52/d6a/dd5/d111/c126 1 2021-11-03T17:14:48.911 INFO:tasks.workunit.client.0.smithi027.stdout:4/723: rename d6/d9/d16/d17/l1d to d6/d9/d16/d36/d70/d3e/d58/d84/lf2 0 2021-11-03T17:14:48.913 INFO:tasks.workunit.client.0.smithi027.stdout:2/784: symlink d0/d5/d2c/le5 0 2021-11-03T17:14:48.917 INFO:tasks.workunit.client.0.smithi027.stdout:1/869: dwrite d6/d11/dac/d79/db0/d102/f6b [0,4194304] 0 2021-11-03T17:14:48.918 INFO:tasks.workunit.client.0.smithi027.stdout:3/911: rmdir d1/db/db2 39 2021-11-03T17:14:48.925 INFO:tasks.workunit.client.0.smithi027.stdout:6/690: write dc/d16/d1b/d24/f46 [1067039,99933] 0 2021-11-03T17:14:48.928 INFO:tasks.workunit.client.0.smithi027.stdout:1/870: getdents d6/d11/dac/d79/d25/d28/dec 0 2021-11-03T17:14:48.929 INFO:tasks.workunit.client.0.smithi027.stdout:6/691: fsync dc/d16/d1b/d24/f46 0 2021-11-03T17:14:48.934 INFO:tasks.workunit.client.0.smithi027.stdout:4/724: dread d6/d9/d16/d36/d70/d3e/d58/d84/d9b/fcb [0,4194304] 0 2021-11-03T17:14:48.937 INFO:tasks.workunit.client.0.smithi027.stdout:9/750: link db/d1b/d1c/d1d/d40/dcc/ff9 db/f115 0 2021-11-03T17:14:48.938 INFO:tasks.workunit.client.0.smithi027.stdout:1/871: mknod d6/d11/dac/d79/d25/d28/d5c/d81/c122 1 2021-11-03T17:14:48.947 INFO:tasks.workunit.client.0.smithi027.stdout:5/982: rename d2/d12/dad/da8/dd9/d12c/fac to d2/d4/d4e/d67/d8e/d99/f146 0 2021-11-03T17:14:48.956 INFO:tasks.workunit.client.0.smithi027.stdout:4/725: write d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/f7e [484551,97679] 0 2021-11-03T17:14:48.956 INFO:tasks.workunit.client.0.smithi027.stdout:4/726: chown d6/d9/d16/d36/fc0 1401918842 1 2021-11-03T17:14:48.956 INFO:tasks.workunit.client.0.smithi027.stdout:6/692: dread - dc/d16/d9a/fc7 zero size 2021-11-03T17:14:48.957 INFO:tasks.workunit.client.0.smithi027.stdout:3/912: creat d1/db/d52/d57/d99/d104/f127 x:0 0 0 2021-11-03T17:14:48.959 INFO:tasks.workunit.client.0.smithi027.stdout:2/785: symlink d0/d1b/d87/le6 0 2021-11-03T17:14:48.964 INFO:tasks.workunit.client.0.smithi027.stdout:5/983: readlink d2/d1f/l6d 0 2021-11-03T17:14:48.966 INFO:tasks.workunit.client.0.smithi027.stdout:9/751: mkdir db/d1b/d2b/d41/d6c/d116 0 2021-11-03T17:14:48.966 INFO:tasks.workunit.client.0.smithi027.stdout:9/752: chown db/d1b/d23 6 1 2021-11-03T17:14:48.967 INFO:tasks.workunit.client.0.smithi027.stdout:1/872: symlink d6/d11/dac/d79/d37/de0/l123 0 2021-11-03T17:14:48.968 INFO:tasks.workunit.client.0.smithi027.stdout:4/727: truncate d6/d9/d16/d36/d70/d3e/f6b 912921 0 2021-11-03T17:14:48.969 INFO:tasks.workunit.client.0.smithi027.stdout:6/693: mknod dc/d16/d1b/d24/d7e/ccd 1 2021-11-03T17:14:49.026 INFO:tasks.workunit.client.0.smithi027.stdout:3/913: dwrite d1/db/d52/d57/d99/faa [0,4194304] 0 2021-11-03T17:14:49.028 INFO:tasks.workunit.client.0.smithi027.stdout:4/728: chown d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/f69 9 1 2021-11-03T17:14:49.050 INFO:tasks.workunit.client.0.smithi027.stdout:2/786: symlink d0/d45/le7 0 2021-11-03T17:14:49.053 INFO:tasks.workunit.client.0.smithi027.stdout:9/753: link db/d1b/d2b/d41/d95/d9f/f100 db/d1b/d2b/d41/d95/f117 0 2021-11-03T17:14:49.055 INFO:tasks.workunit.client.0.smithi027.stdout:1/873: truncate d6/d11/dac/d79/d25/d28/d5c/f78 5867799 0 2021-11-03T17:14:49.055 INFO:tasks.workunit.client.0.smithi027.stdout:4/729: rmdir d6/d9 39 2021-11-03T17:14:49.058 INFO:tasks.workunit.client.0.smithi027.stdout:3/914: creat d1/db/d52/d6a/dd5/d120/f128 x:0 0 0 2021-11-03T17:14:49.064 INFO:tasks.workunit.client.0.smithi027.stdout:5/984: rmdir d2/d2c/d9b/db3/d116 39 2021-11-03T17:14:49.066 INFO:tasks.workunit.client.0.smithi027.stdout:9/754: creat db/d1b/d2b/d41/d95/deb/f118 x:0 0 0 2021-11-03T17:14:49.068 INFO:tasks.workunit.client.0.smithi027.stdout:1/874: unlink d6/d11/dac/d79/d25/d28/d5c/d81/d7d/f7e 0 2021-11-03T17:14:49.068 INFO:tasks.workunit.client.0.smithi027.stdout:4/730: stat d6/d9/d16/d36/f42 0 2021-11-03T17:14:49.070 INFO:tasks.workunit.client.0.smithi027.stdout:3/915: mkdir d1/db/d52/d57/d99/d129 0 2021-11-03T17:14:49.072 INFO:tasks.workunit.client.0.smithi027.stdout:5/985: creat d2/d2c/d9b/f147 x:0 0 0 2021-11-03T17:14:49.073 INFO:tasks.workunit.client.0.smithi027.stdout:2/787: rename d0/l8e to d0/d5/le8 0 2021-11-03T17:14:49.074 INFO:tasks.workunit.client.0.smithi027.stdout:9/755: chown db/d1b/d1c/d1d/l56 3701336 1 2021-11-03T17:14:49.077 INFO:tasks.workunit.client.0.smithi027.stdout:6/694: mkdir dc/d16/d1b/d3c/d51/d5f/dce 0 2021-11-03T17:14:49.082 INFO:tasks.workunit.client.0.smithi027.stdout:9/756: creat db/d1b/d8c/dcd/f119 x:0 0 0 2021-11-03T17:14:49.083 INFO:tasks.workunit.client.0.smithi027.stdout:1/875: dread d6/d11/d14/f76 [0,4194304] 0 2021-11-03T17:14:49.096 INFO:tasks.workunit.client.0.smithi027.stdout:2/788: dread d0/d45/f7b [0,4194304] 0 2021-11-03T17:14:49.102 INFO:tasks.workunit.client.0.smithi027.stdout:3/916: write d1/d40/f4a [4775936,5873] 0 2021-11-03T17:14:49.103 INFO:tasks.workunit.client.0.smithi027.stdout:4/731: write d6/d24/db0/d81/fd8 [54067,14446] 0 2021-11-03T17:14:49.106 INFO:tasks.workunit.client.0.smithi027.stdout:5/986: link d2/d12/dad/da8/fcb d2/d2c/d3c/dc8/d121/f148 0 2021-11-03T17:14:49.108 INFO:tasks.workunit.client.0.smithi027.stdout:9/757: rename db/d1b/d2b/d41/d97/fbd to db/d1b/d2b/d41/d6c/f11a 0 2021-11-03T17:14:49.109 INFO:tasks.workunit.client.0.smithi027.stdout:1/876: mkdir d6/d11/dac/d79/d25/d28/d5c/d7a/dee/d124 0 2021-11-03T17:14:49.114 INFO:tasks.workunit.client.0.smithi027.stdout:6/695: chown dc/d82/d87/fb9 0 1 2021-11-03T17:14:49.115 INFO:tasks.workunit.client.0.smithi027.stdout:5/987: stat d2/d1f/f7c 0 2021-11-03T17:14:49.115 INFO:tasks.workunit.client.0.smithi027.stdout:5/988: mknod d2/d1f/d144/c149 1 2021-11-03T17:14:49.119 INFO:tasks.workunit.client.0.smithi027.stdout:1/877: dread d6/d11/dac/d79/d25/d28/d5c/d81/f86 [0,4194304] 0 2021-11-03T17:14:49.119 INFO:tasks.workunit.client.0.smithi027.stdout:2/789: fdatasync d0/d5/f6d 0 2021-11-03T17:14:49.124 INFO:tasks.workunit.client.0.smithi027.stdout:4/732: chown d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d41/d44 56601192 1 2021-11-03T17:14:49.125 INFO:tasks.workunit.client.0.smithi027.stdout:3/917: write d1/db/f32 [2905206,77543] 0 2021-11-03T17:14:49.128 INFO:tasks.workunit.client.0.smithi027.stdout:5/989: read d2/d4/d4e/d67/f11 [308351,37871] 0 2021-11-03T17:14:49.130 INFO:tasks.workunit.client.0.smithi027.stdout:1/878: read d6/d11/dac/d79/d25/d28/d5c/d7a/fbe [12464333,34900] 0 2021-11-03T17:14:49.134 INFO:tasks.workunit.client.0.smithi027.stdout:6/696: fdatasync dc/d82/d57/f97 0 2021-11-03T17:14:49.135 INFO:tasks.workunit.client.0.smithi027.stdout:5/990: mknod d2/c14a 1 2021-11-03T17:14:49.149 INFO:tasks.workunit.client.0.smithi027.stdout:6/697: symlink dc/d16/d1b/d24/lcf 0 2021-11-03T17:14:49.149 INFO:tasks.workunit.client.0.smithi027.stdout:9/758: mkdir db/d1b/d2b/d41/dad/dbb/d11b 0 2021-11-03T17:14:49.159 INFO:tasks.workunit.client.0.smithi027.stdout:4/733: rename d6/d9/d16/d36/d57/d5a/f67 to d6/d9/d16/d36/d97/ff3 0 2021-11-03T17:14:49.161 INFO:tasks.workunit.client.0.smithi027.stdout:3/918: dwrite d1/f50 [0,4194304] 0 2021-11-03T17:14:49.161 INFO:tasks.workunit.client.0.smithi027.stdout:9/759: symlink db/d1b/d2b/d41/l11c 0 2021-11-03T17:14:49.165 INFO:tasks.workunit.client.0.smithi027.stdout:4/734: fsync d6/d9/d16/d36/d70/d3e/d58/f60 0 2021-11-03T17:14:49.167 INFO:tasks.workunit.client.0.smithi027.stdout:9/760: creat db/d1b/d106/f11d x:0 0 0 2021-11-03T17:14:49.169 INFO:tasks.workunit.client.0.smithi027.stdout:1/879: dwrite d6/d11/dac/d79/d25/f47 [0,4194304] 0 2021-11-03T17:14:49.169 INFO:tasks.workunit.client.0.smithi027.stdout:2/790: dread d0/d1b/d29/d54/d67/f6c [0,4194304] 0 2021-11-03T17:14:49.170 INFO:tasks.workunit.client.0.smithi027.stdout:6/698: link dc/d16/d1b/d3c/f9e dc/d82/d57/d9b/fd0 0 2021-11-03T17:14:49.183 INFO:tasks.workunit.client.0.smithi027.stdout:9/761: rmdir db/d1b/d2b/dc7 39 2021-11-03T17:14:49.187 INFO:tasks.workunit.client.0.smithi027.stdout:2/791: mknod d0/d1b/d29/d58/ce9 1 2021-11-03T17:14:49.188 INFO:tasks.workunit.client.0.smithi027.stdout:4/735: write d6/d9/d16/d36/d97/faa [193578,48622] 0 2021-11-03T17:14:49.190 INFO:tasks.workunit.client.0.smithi027.stdout:1/880: dread d6/d11/dac/d79/d37/de0/de4/fb7 [0,4194304] 0 2021-11-03T17:14:49.191 INFO:tasks.workunit.client.0.smithi027.stdout:5/991: dwrite d2/d4/d61/f89 [0,4194304] 0 2021-11-03T17:14:49.194 INFO:tasks.workunit.client.0.smithi027.stdout:6/699: stat dc/d37/dcc/f93 0 2021-11-03T17:14:49.195 INFO:tasks.workunit.client.0.smithi027.stdout:9/762: truncate db/d1b/d1c/d1d/fa1 472137 0 2021-11-03T17:14:49.197 INFO:tasks.workunit.client.0.smithi027.stdout:4/736: dread d6/d24/db0/d81/fb7 [0,4194304] 0 2021-11-03T17:14:49.200 INFO:tasks.workunit.client.0.smithi027.stdout:5/992: truncate d2/f3 2579864 0 2021-11-03T17:14:49.203 INFO:tasks.workunit.client.0.smithi027.stdout:1/881: dread d6/d11/dac/d79/db0/d102/f66 [0,4194304] 0 2021-11-03T17:14:49.204 INFO:tasks.workunit.client.0.smithi027.stdout:3/919: dwrite d1/db/db2/dbd/d11b/fe6 [0,4194304] 0 2021-11-03T17:14:49.217 INFO:tasks.workunit.client.0.smithi027.stdout:3/920: creat d1/db/d52/d6a/dd5/d111/f12a x:0 0 0 2021-11-03T17:14:49.219 INFO:tasks.workunit.client.0.smithi027.stdout:5/993: mknod d2/d2c/d3c/d58/c14b 1 2021-11-03T17:14:49.221 INFO:tasks.workunit.client.0.smithi027.stdout:2/792: symlink d0/d1b/d29/d54/d64/daa/d5d/lea 0 2021-11-03T17:14:49.225 INFO:tasks.workunit.client.0.smithi027.stdout:3/921: creat d1/db/d52/d6a/d7c/d123/f12b x:0 0 0 2021-11-03T17:14:49.231 INFO:tasks.workunit.client.0.smithi027.stdout:6/700: mkdir dc/d82/d57/dad/dae/db1/dd1 0 2021-11-03T17:14:49.233 INFO:tasks.workunit.client.0.smithi027.stdout:1/882: link d6/f18 d6/d11/dac/f125 0 2021-11-03T17:14:49.233 INFO:tasks.workunit.client.0.smithi027.stdout:4/737: dwrite d6/d9/d8f/fca [0,4194304] 0 2021-11-03T17:14:49.234 INFO:tasks.workunit.client.0.smithi027.stdout:5/994: truncate d2/d1f/f6f 1618761 0 2021-11-03T17:14:49.237 INFO:tasks.workunit.client.0.smithi027.stdout:1/883: fsync d6/d11/dac/f6d 0 2021-11-03T17:14:49.238 INFO:tasks.workunit.client.0.smithi027.stdout:4/738: read d6/d9/d16/f19 [1756813,11889] 0 2021-11-03T17:14:49.242 INFO:tasks.workunit.client.0.smithi027.stdout:3/922: dwrite d1/db/db2/dbd/d11b/fe6 [0,4194304] 0 2021-11-03T17:14:49.242 INFO:tasks.workunit.client.0.smithi027.stdout:1/884: creat d6/d11/dac/d79/d25/d28/dec/f126 x:0 0 0 2021-11-03T17:14:49.244 INFO:tasks.workunit.client.0.smithi027.stdout:5/995: creat d2/d2c/d9b/d9d/f14c x:0 0 0 2021-11-03T17:14:49.245 INFO:tasks.workunit.client.0.smithi027.stdout:6/701: dread dc/d16/d30/f38 [0,4194304] 0 2021-11-03T17:14:49.246 INFO:tasks.workunit.client.0.smithi027.stdout:3/923: rmdir d1/db 39 2021-11-03T17:14:49.254 INFO:tasks.workunit.client.0.smithi027.stdout:4/739: symlink d6/d9/d16/d36/d70/d3e/d58/d84/lf4 0 2021-11-03T17:14:49.257 INFO:tasks.workunit.client.0.smithi027.stdout:6/702: creat dc/d82/d57/d9b/fd2 x:0 0 0 2021-11-03T17:14:49.303 INFO:tasks.workunit.client.0.smithi027.stdout:2/793: dwrite d0/d1b/d29/f2f [0,4194304] 0 2021-11-03T17:14:49.322 INFO:tasks.workunit.client.0.smithi027.stdout:4/740: rmdir d6/d9/d16/d36/d70/d3e/d58/d84 39 2021-11-03T17:14:49.366 INFO:tasks.workunit.client.0.smithi027.stdout:6/703: creat dc/d16/d9a/fd3 x:0 0 0 2021-11-03T17:14:49.367 INFO:tasks.workunit.client.0.smithi027.stdout:3/924: getdents d1/db/d52/d6a/d7c/d88 0 2021-11-03T17:14:49.376 INFO:tasks.workunit.client.0.smithi027.stdout:1/885: getdents d6/d11/dac/d79/db0/de5/dff/dd7/d8b 0 2021-11-03T17:14:49.381 INFO:tasks.workunit.client.0.smithi027.stdout:6/704: fsync dc/d16/d30/f5a 0 2021-11-03T17:14:49.386 INFO:tasks.workunit.client.0.smithi027.stdout:2/794: chown d0/d1b/db6/fdb 1433 1 2021-11-03T17:14:49.401 INFO:tasks.workunit.client.0.smithi027.stdout:1/886: dread d6/d11/dac/d79/d25/d28/dec/fe9 [0,4194304] 0 2021-11-03T17:14:49.401 INFO:tasks.workunit.client.0.smithi027.stdout:6/705: mkdir dc/d82/d57/d96/dd4 0 2021-11-03T17:14:49.402 INFO:tasks.workunit.client.0.smithi027.stdout:5/996: write d2/d4/f13 [3340452,70191] 0 2021-11-03T17:14:49.405 INFO:tasks.workunit.client.0.smithi027.stdout:5/997: rmdir d2/d12/dad/da8/dd9/dde 39 2021-11-03T17:14:49.407 INFO:tasks.workunit.client.0.smithi027.stdout:5/998: creat d2/d2c/d3c/dc8/f14d x:0 0 0 2021-11-03T17:14:49.411 INFO:tasks.workunit.client.0.smithi027.stdout:6/706: rename dc/d82/d57/d96/la0 to dc/d16/d1b/d24/d7e/db3/ld5 0 2021-11-03T17:14:49.414 INFO:tasks.workunit.client.0.smithi027.stdout:1/887: dread d6/d11/dac/d79/fbb [0,4194304] 0 2021-11-03T17:14:49.414 INFO:tasks.workunit.client.0.smithi027.stdout:3/925: dwrite d1/db/d52/d6a/d7c/f89 [0,4194304] 0 2021-11-03T17:14:49.415 INFO:tasks.workunit.client.0.smithi027.stdout:4/741: dwrite f4 [0,4194304] 0 2021-11-03T17:14:49.425 INFO:tasks.workunit.client.0.smithi027.stdout:6/707: write dc/f64 [4606240,115878] 0 2021-11-03T17:14:49.426 INFO:tasks.workunit.client.0.smithi027.stdout:1/888: creat d6/d11/dac/d79/d104/f127 x:0 0 0 2021-11-03T17:14:49.433 INFO:tasks.workunit.client.0.smithi027.stdout:1/889: readlink d6/d11/l105 0 2021-11-03T17:14:49.456 INFO:tasks.workunit.client.0.smithi027.stdout:3/926: dread d1/d40/fb6 [0,4194304] 0 2021-11-03T17:14:49.458 INFO:tasks.workunit.client.0.smithi027.stdout:2/795: dwrite d0/d5/faf [0,4194304] 0 2021-11-03T17:14:49.462 INFO:tasks.workunit.client.0.smithi027.stdout:2/796: mkdir d0/d5/d2c/deb 0 2021-11-03T17:14:49.467 INFO:tasks.workunit.client.0.smithi027.stdout:5/999: dwrite d2/d4/d4e/d67/f118 [0,4194304] 0 2021-11-03T17:14:49.467 INFO:tasks.workunit.client.0.smithi027.stdout:1/890: dwrite d6/d11/d14/df6/f11f [0,4194304] 0 2021-11-03T17:14:49.467 INFO:tasks.workunit.client.0.smithi027.stdout:4/742: dwrite d6/d24/db0/f53 [0,4194304] 0 2021-11-03T17:14:49.468 INFO:tasks.workunit.client.0.smithi027.stdout:4/743: stat d6/d9/d16 0 2021-11-03T17:14:49.481 INFO:tasks.workunit.client.0.smithi027.stdout:4/744: symlink d6/d9/d16/d36/d57/d5a/d63/lf5 0 2021-11-03T17:14:49.483 INFO:tasks.workunit.client.0.smithi027.stdout:1/891: truncate d6/d11/dac/d79/dab/db4/fc5 1699066 0 2021-11-03T17:14:49.484 INFO:tasks.workunit.client.0.smithi027.stdout:4/745: read d6/d24/dbe/fc6 [611170,59466] 0 2021-11-03T17:14:49.485 INFO:tasks.workunit.client.0.smithi027.stdout:4/746: creat d6/d9/d16/d36/d57/d5a/d63/ff6 x:0 0 0 2021-11-03T17:14:49.490 INFO:tasks.workunit.client.0.smithi027.stdout:1/892: symlink d6/de/df1/l128 0 2021-11-03T17:14:49.491 INFO:tasks.workunit.client.0.smithi027.stdout:4/747: truncate d6/d9/d16/d36/d70/d3e/d58/d84/f86 922421 0 2021-11-03T17:14:49.491 INFO:tasks.workunit.client.0.smithi027.stdout:4/748: fdatasync d6/d9/d16/f26 0 2021-11-03T17:14:49.493 INFO:tasks.workunit.client.0.smithi027.stdout:4/749: chown d6/d24/de3/d9f/fa8 1246 1 2021-11-03T17:14:49.493 INFO:tasks.workunit.client.0.smithi027.stdout:6/708: dwrite dc/d82/d57/d9b/fc4 [0,4194304] 0 2021-11-03T17:14:49.497 INFO:tasks.workunit.client.0.smithi027.stdout:2/797: dwrite d0/fa9 [0,4194304] 0 2021-11-03T17:14:49.507 INFO:tasks.workunit.client.0.smithi027.stdout:1/893: dwrite d6/d11/dac/d79/d25/d28/dec/f121 [0,4194304] 0 2021-11-03T17:14:49.510 INFO:tasks.workunit.client.0.smithi027.stdout:4/750: unlink d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d9a/fba 0 2021-11-03T17:14:49.517 INFO:tasks.workunit.client.0.smithi027.stdout:1/894: chown d6/de/f92 2970722 1 2021-11-03T17:14:49.518 INFO:tasks.workunit.client.0.smithi027.stdout:4/751: chown d6/d9/d16/d36/d70/f71 148 1 2021-11-03T17:14:49.519 INFO:tasks.workunit.client.0.smithi027.stdout:6/709: write dc/d82/d57/f91 [5110024,30783] 0 2021-11-03T17:14:49.521 INFO:tasks.workunit.client.0.smithi027.stdout:2/798: write d0/d1b/f82 [452939,96596] 0 2021-11-03T17:14:49.523 INFO:tasks.workunit.client.0.smithi027.stdout:1/895: creat d6/d11/d14/df6/f129 x:0 0 0 2021-11-03T17:14:49.524 INFO:tasks.workunit.client.0.smithi027.stdout:4/752: chown d6/d9/d16/d17/lcc 206 1 2021-11-03T17:14:49.526 INFO:tasks.workunit.client.0.smithi027.stdout:2/799: creat d0/d1b/d29/d54/d64/daa/fec x:0 0 0 2021-11-03T17:14:49.527 INFO:tasks.workunit.client.0.smithi027.stdout:1/896: rmdir d6/d11/dac/d79/d37 39 2021-11-03T17:14:49.530 INFO:tasks.workunit.client.0.smithi027.stdout:6/710: symlink dc/d16/ld6 0 2021-11-03T17:14:49.537 INFO:tasks.workunit.client.0.smithi027.stdout:4/753: link d6/d24/de3/d9f/dd7/leb d6/d24/de3/d9f/dd7/lf7 0 2021-11-03T17:14:49.543 INFO:tasks.workunit.client.0.smithi027.stdout:6/711: link dc/d16/f40 dc/d82/d57/dad/dae/fd7 0 2021-11-03T17:14:49.543 INFO:tasks.workunit.client.0.smithi027.stdout:2/800: symlink d0/da7/dc9/led 0 2021-11-03T17:14:49.547 INFO:tasks.workunit.client.0.smithi027.stdout:1/897: write d6/d11/dac/d79/d37/de0/d4e/f91 [1014545,47789] 0 2021-11-03T17:14:49.550 INFO:tasks.workunit.client.0.smithi027.stdout:6/712: fdatasync dc/f88 0 2021-11-03T17:14:49.553 INFO:tasks.workunit.client.0.smithi027.stdout:4/754: dread d6/d24/d6d/db1/fd2 [0,4194304] 0 2021-11-03T17:14:49.553 INFO:tasks.workunit.client.0.smithi027.stdout:1/898: creat d6/d11/d14/f12a x:0 0 0 2021-11-03T17:14:49.555 INFO:tasks.workunit.client.0.smithi027.stdout:4/755: dread d6/d24/dbe/fea [0,4194304] 0 2021-11-03T17:14:49.558 INFO:tasks.workunit.client.0.smithi027.stdout:2/801: symlink d0/d1b/d1c/lee 0 2021-11-03T17:14:49.560 INFO:tasks.workunit.client.0.smithi027.stdout:4/756: creat d6/d9/d16/ff8 x:0 0 0 2021-11-03T17:14:49.561 INFO:tasks.workunit.client.0.smithi027.stdout:4/757: chown d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d41/d44 1185712628 1 2021-11-03T17:14:49.563 INFO:tasks.workunit.client.0.smithi027.stdout:6/713: mkdir dc/d16/d1b/d3c/d51/dd8 0 2021-11-03T17:14:49.564 INFO:tasks.workunit.client.0.smithi027.stdout:1/899: mkdir d6/d11/dac/d79/d25/d28/d5c/d7a/dee/d12b 0 2021-11-03T17:14:49.567 INFO:tasks.workunit.client.0.smithi027.stdout:6/714: creat dc/d16/d1b/d3c/d51/dd8/fd9 x:0 0 0 2021-11-03T17:14:49.568 INFO:tasks.workunit.client.0.smithi027.stdout:2/802: symlink d0/d1b/d29/db5/dcc/lef 0 2021-11-03T17:14:49.570 INFO:tasks.workunit.client.0.smithi027.stdout:4/758: rename d6/d9/d16/d36/d70/dee/f3f to d6/d9/d16/d36/dc1/ff9 0 2021-11-03T17:14:49.571 INFO:tasks.workunit.client.0.smithi027.stdout:6/715: truncate dc/d82/d87/fb9 96119 0 2021-11-03T17:14:49.571 INFO:tasks.workunit.client.0.smithi027.stdout:6/716: fdatasync dc/d82/d57/d9b/fd0 0 2021-11-03T17:14:49.573 INFO:tasks.workunit.client.0.smithi027.stdout:1/900: mkdir d6/d11/dac/d79/d25/d28/d12c 0 2021-11-03T17:14:49.597 INFO:tasks.workunit.client.0.smithi027.stdout:4/759: creat d6/d9/d16/d36/d57/dbb/ffa x:0 0 0 2021-11-03T17:14:49.600 INFO:tasks.workunit.client.0.smithi027.stdout:2/803: dread d0/d1b/d29/d92/dbd/f98 [0,4194304] 0 2021-11-03T17:14:49.673 INFO:tasks.workunit.client.0.smithi027.stdout:4/760: mknod d6/d24/de3/d9f/cfb 1 2021-11-03T17:14:49.678 INFO:tasks.workunit.client.0.smithi027.stdout:2/804: mknod d0/d1b/d1c/cf0 1 2021-11-03T17:14:49.683 INFO:tasks.workunit.client.0.smithi027.stdout:4/761: rmdir d6/d9/d16/d36/d57/dbb 39 2021-11-03T17:14:49.686 INFO:tasks.workunit.client.0.smithi027.stdout:2/805: chown d0/d1b/d29/d92/dbd/fa6 39519325 1 2021-11-03T17:14:49.686 INFO:tasks.workunit.client.0.smithi027.stdout:4/762: readlink d6/d9/d16/d36/d70/d3e/d58/l74 0 2021-11-03T17:14:49.687 INFO:tasks.workunit.client.0.smithi027.stdout:4/763: chown d6/d24/dbe/fc6 67941 1 2021-11-03T17:14:49.690 INFO:tasks.workunit.client.0.smithi027.stdout:2/806: truncate d0/f10 3776807 0 2021-11-03T17:14:49.693 INFO:tasks.workunit.client.0.smithi027.stdout:2/807: mkdir d0/d7c/df1 0 2021-11-03T17:14:49.696 INFO:tasks.workunit.client.0.smithi027.stdout:2/808: creat d0/d1b/d29/d54/ff2 x:0 0 0 2021-11-03T17:14:49.698 INFO:tasks.workunit.client.0.smithi027.stdout:2/809: mkdir d0/d1b/d29/d54/df3 0 2021-11-03T17:14:49.703 INFO:tasks.workunit.client.0.smithi027.stdout:2/810: rename d0/d1b/d29/d54/d64/daa/fc4 to d0/d7c/ff4 0 2021-11-03T17:14:49.705 INFO:tasks.workunit.client.0.smithi027.stdout:4/764: dwrite d6/d9/f73 [0,4194304] 0 2021-11-03T17:14:49.706 INFO:tasks.workunit.client.0.smithi027.stdout:2/811: unlink d0/d5/f83 0 2021-11-03T17:14:49.707 INFO:tasks.workunit.client.0.smithi027.stdout:4/765: dread d6/d24/db0/d81/fd8 [0,4194304] 0 2021-11-03T17:14:49.707 INFO:tasks.workunit.client.0.smithi027.stdout:2/812: chown d0/d1b/d29/d54/d64/daa/d61/fc7 95 1 2021-11-03T17:14:49.710 INFO:tasks.workunit.client.0.smithi027.stdout:2/813: creat d0/d5/d2c/deb/ff5 x:0 0 0 2021-11-03T17:14:49.711 INFO:tasks.workunit.client.0.smithi027.stdout:4/766: symlink d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d41/d44/lfc 0 2021-11-03T17:14:49.712 INFO:tasks.workunit.client.0.smithi027.stdout:2/814: readlink d0/d45/lbc 0 2021-11-03T17:14:49.715 INFO:tasks.workunit.client.0.smithi027.stdout:2/815: creat d0/d1b/d29/d92/dbd/ff6 x:0 0 0 2021-11-03T17:14:49.715 INFO:tasks.workunit.client.0.smithi027.stdout:4/767: mknod d6/d9/d16/d36/d57/d5a/cfd 1 2021-11-03T17:14:49.717 INFO:tasks.workunit.client.0.smithi027.stdout:4/768: dread - d6/d9/d16/d36/d57/fdb zero size 2021-11-03T17:14:49.717 INFO:tasks.workunit.client.0.smithi027.stdout:2/816: rmdir d0/d1b/d29/d54/d67 39 2021-11-03T17:14:49.720 INFO:tasks.workunit.client.0.smithi027.stdout:2/817: unlink d0/d5/d2c/le5 0 2021-11-03T17:14:49.722 INFO:tasks.workunit.client.0.smithi027.stdout:4/769: creat d6/d24/dbe/de5/ffe x:0 0 0 2021-11-03T17:14:49.724 INFO:tasks.workunit.client.0.smithi027.stdout:4/770: creat d6/d9/fff x:0 0 0 2021-11-03T17:14:49.724 INFO:tasks.workunit.client.0.smithi027.stdout:2/818: getdents d0/d1b/d29/d54/d64/daa/d5d 0 2021-11-03T17:14:49.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:49 smithi115 conmon[34670]: cluster 2021-11-03T17:14:47.897997 2021-11-03T17:14:49.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:49 smithi115 conmon[34670]: +0000 mgr.smithi115.tcyodz (mgr.24395) 30 : cluster [DBG] pgmap v19: 65 pgs: 65 active+clean; 3.0 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 34 MiB/s rd, 41 MiB/s wr, 73 op/s 2021-11-03T17:14:49.735 INFO:tasks.workunit.client.0.smithi027.stdout:4/771: dread d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/f2c [4194304,4194304] 0 2021-11-03T17:14:49.740 INFO:tasks.workunit.client.0.smithi027.stdout:2/819: write d0/d1b/db6/fbe [1079953,96137] 0 2021-11-03T17:14:49.743 INFO:tasks.workunit.client.0.smithi027.stdout:4/772: rmdir d6/d9/d16/d36/d99/dd0 39 2021-11-03T17:14:49.744 INFO:tasks.workunit.client.0.smithi027.stdout:4/773: mknod d6/d9/d16/d36/d57/dbb/c100 1 2021-11-03T17:14:49.747 INFO:tasks.workunit.client.0.smithi027.stdout:4/774: truncate d6/d24/f46 3022216 0 2021-11-03T17:14:49.747 INFO:tasks.workunit.client.0.smithi027.stdout:4/775: write d6/d24/dbe/de5/ffe [958164,7620] 0 2021-11-03T17:14:49.748 INFO:tasks.workunit.client.0.smithi027.stdout:4/776: fdatasync d6/d24/de3/d9f/fa8 0 2021-11-03T17:14:49.748 INFO:tasks.workunit.client.0.smithi027.stdout:4/777: write d6/d24/db0/f53 [4238018,94332] 0 2021-11-03T17:14:49.756 INFO:tasks.workunit.client.0.smithi027.stdout:4/778: write d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d41/f88 [806299,73595] 0 2021-11-03T17:14:49.757 INFO:tasks.workunit.client.0.smithi027.stdout:4/779: creat d6/d9/d16/d36/dc1/f101 x:0 0 0 2021-11-03T17:14:49.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:49 smithi027 conmon[32322]: cluster 2021-11-03T17:14:47.897997+0000 mgr.smithi115.tcyodz (mgr.24395) 30 : cluster [DBG] pgmap v19: 65 pgs: 65 active+clean; 3.0 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 34 MiB/s rd, 41 MiB/s wr, 73 op/s 2021-11-03T17:14:50.696 INFO:tasks.workunit.client.0.smithi027.stdout:7/579: sync 2021-11-03T17:14:50.699 INFO:tasks.workunit.client.0.smithi027.stdout:7/580: creat d0/d2/d1f/fc2 x:0 0 0 2021-11-03T17:14:50.729 INFO:tasks.workunit.client.0.smithi027.stdout:7/581: dwrite d0/d2d/d58/f69 [0,4194304] 0 2021-11-03T17:14:50.732 INFO:tasks.workunit.client.0.smithi027.stdout:7/582: unlink d0/d2/d1f/la8 0 2021-11-03T17:14:50.733 INFO:tasks.workunit.client.0.smithi027.stdout:7/583: chown d0/lc 0 1 2021-11-03T17:14:50.736 INFO:tasks.workunit.client.0.smithi027.stdout:7/584: creat d0/d2/d1c/d4d/d6e/d84/dad/fc3 x:0 0 0 2021-11-03T17:14:50.745 INFO:tasks.workunit.client.0.smithi027.stdout:7/585: dread d0/d2d/d58/f69 [0,4194304] 0 2021-11-03T17:14:50.745 INFO:tasks.workunit.client.0.smithi027.stdout:7/586: write d0/d2/fd [5143976,51689] 0 2021-11-03T17:14:50.747 INFO:tasks.workunit.client.0.smithi027.stdout:7/587: truncate d0/d2/d1c/d4d/d6e/d84/f92 774230 0 2021-11-03T17:14:50.748 INFO:tasks.workunit.client.0.smithi027.stdout:7/588: dread - d0/d2/d10/d1e/d20/faa zero size 2021-11-03T17:14:50.748 INFO:tasks.workunit.client.0.smithi027.stdout:7/589: dread - d0/d2/d5e/d36/f63 zero size 2021-11-03T17:14:50.769 INFO:tasks.workunit.client.0.smithi027.stdout:7/590: dwrite d0/d2/fd [0,4194304] 0 2021-11-03T17:14:50.769 INFO:tasks.workunit.client.0.smithi027.stdout:7/591: fsync d0/d2/d59/f73 0 2021-11-03T17:14:50.772 INFO:tasks.workunit.client.0.smithi027.stdout:7/592: symlink d0/d2/d1c/d4d/d6e/lc4 0 2021-11-03T17:14:50.774 INFO:tasks.workunit.client.0.smithi027.stdout:7/593: creat d0/d2d/fc5 x:0 0 0 2021-11-03T17:14:50.798 INFO:tasks.workunit.client.0.smithi027.stdout:7/594: dwrite d0/d2/d1f/f9e [0,4194304] 0 2021-11-03T17:14:50.801 INFO:tasks.workunit.client.0.smithi027.stdout:7/595: mknod d0/d2/d1c/d4d/d6e/d8b/cc6 1 2021-11-03T17:14:50.803 INFO:tasks.workunit.client.0.smithi027.stdout:7/596: creat d0/d2/d1c/d4d/d6e/d8b/fc7 x:0 0 0 2021-11-03T17:14:50.805 INFO:tasks.workunit.client.0.smithi027.stdout:7/597: chown d0/d2d/d2f/fb9 1796 1 2021-11-03T17:14:50.806 INFO:tasks.workunit.client.0.smithi027.stdout:0/871: sync 2021-11-03T17:14:50.806 INFO:tasks.workunit.client.0.smithi027.stdout:7/598: dread - d0/d2/fa3 zero size 2021-11-03T17:14:50.811 INFO:tasks.workunit.client.0.smithi027.stdout:0/872: creat d9/da/d64/d122/f123 x:0 0 0 2021-11-03T17:14:50.811 INFO:tasks.workunit.client.0.smithi027.stdout:7/599: stat d0/d2/d10/d82 0 2021-11-03T17:14:50.819 INFO:tasks.workunit.client.0.smithi027.stdout:7/600: mkdir d0/d2d/d2f/dae/dc8 0 2021-11-03T17:14:50.822 INFO:tasks.workunit.client.0.smithi027.stdout:7/601: getdents d0/d2d/d58 0 2021-11-03T17:14:50.823 INFO:tasks.workunit.client.0.smithi027.stdout:7/602: mknod d0/d2/d1c/d4d/d6e/d84/dad/cc9 1 2021-11-03T17:14:50.826 INFO:tasks.workunit.client.0.smithi027.stdout:7/603: unlink d0/d2d/fbc 0 2021-11-03T17:14:50.828 INFO:tasks.workunit.client.0.smithi027.stdout:7/604: truncate d0/d2d/d66/f98 794136 0 2021-11-03T17:14:50.828 INFO:tasks.workunit.client.0.smithi027.stdout:7/605: getdents d0/d2/d59 0 2021-11-03T17:14:50.867 INFO:tasks.workunit.client.0.smithi027.stdout:0/873: dwrite d9/fb4 [0,4194304] 0 2021-11-03T17:14:50.882 INFO:tasks.workunit.client.0.smithi027.stdout:0/874: dread d9/da/d64/d70/d5f/f74 [0,4194304] 0 2021-11-03T17:14:50.884 INFO:tasks.workunit.client.0.smithi027.stdout:0/875: write d9/d12/fa6 [859478,94855] 0 2021-11-03T17:14:50.886 INFO:tasks.workunit.client.0.smithi027.stdout:0/876: mknod d9/da/d30/da7/c124 1 2021-11-03T17:14:50.886 INFO:tasks.workunit.client.0.smithi027.stdout:7/606: dwrite d0/d2/fa3 [0,4194304] 0 2021-11-03T17:14:50.887 INFO:tasks.workunit.client.0.smithi027.stdout:0/877: chown d9/dd3/f113 315219 1 2021-11-03T17:14:50.892 INFO:tasks.workunit.client.0.smithi027.stdout:0/878: dread d9/dd3/f108 [0,4194304] 0 2021-11-03T17:14:50.896 INFO:tasks.workunit.client.0.smithi027.stdout:0/879: symlink d9/da/d64/d70/d6f/l125 0 2021-11-03T17:14:50.900 INFO:tasks.workunit.client.0.smithi027.stdout:0/880: unlink d9/da/d64/fbb 0 2021-11-03T17:14:50.903 INFO:tasks.workunit.client.0.smithi027.stdout:7/607: dwrite d0/d2/d1f/fc2 [0,4194304] 0 2021-11-03T17:14:50.905 INFO:tasks.workunit.client.0.smithi027.stdout:0/881: rename d9/d12/l2c to d9/da/d30/d92/l126 0 2021-11-03T17:14:50.908 INFO:tasks.workunit.client.0.smithi027.stdout:7/608: mknod d0/d2d/d53/cca 1 2021-11-03T17:14:50.913 INFO:tasks.workunit.client.0.smithi027.stdout:7/609: chown d0/d2/d5e/d36/l71 991 1 2021-11-03T17:14:50.915 INFO:tasks.workunit.client.0.smithi027.stdout:7/610: read d0/d2/d10/d1e/f88 [97806,9747] 0 2021-11-03T17:14:50.916 INFO:tasks.workunit.client.0.smithi027.stdout:0/882: symlink d9/da/d64/d70/d6d/d7a/l127 0 2021-11-03T17:14:50.918 INFO:tasks.workunit.client.0.smithi027.stdout:0/883: mkdir d9/da/d64/d70/d6d/d9f/d11a/d128 0 2021-11-03T17:14:50.921 INFO:tasks.workunit.client.0.smithi027.stdout:0/884: mkdir d9/da/d64/d70/d6d/d9f/d11a/d129 0 2021-11-03T17:14:50.963 INFO:tasks.workunit.client.0.smithi027.stdout:0/885: symlink d9/da/d64/d70/d6d/d7a/dc9/l12a 0 2021-11-03T17:14:50.966 INFO:tasks.workunit.client.0.smithi027.stdout:0/886: creat d9/d12/d8f/dc7/f12b x:0 0 0 2021-11-03T17:14:50.967 INFO:tasks.workunit.client.0.smithi027.stdout:0/887: creat d9/da/d64/d70/d6d/d7a/f12c x:0 0 0 2021-11-03T17:14:50.970 INFO:tasks.workunit.client.0.smithi027.stdout:0/888: rename d9/da/d31/f34 to d9/d12/f12d 0 2021-11-03T17:14:50.974 INFO:tasks.workunit.client.0.smithi027.stdout:0/889: rename d9/da/d30/f43 to d9/da/d64/d70/d6d/d9f/f12e 0 2021-11-03T17:14:50.978 INFO:tasks.workunit.client.0.smithi027.stdout:0/890: creat d9/da/d30/dac/dba/ddc/f12f x:0 0 0 2021-11-03T17:14:50.985 INFO:tasks.workunit.client.0.smithi027.stdout:0/891: dread d9/fb4 [0,4194304] 0 2021-11-03T17:14:50.992 INFO:tasks.workunit.client.0.smithi027.stdout:0/892: dread d9/da/f39 [0,4194304] 0 2021-11-03T17:14:50.993 INFO:tasks.workunit.client.0.smithi027.stdout:0/893: chown d9/le5 252522359 1 2021-11-03T17:14:50.993 INFO:tasks.workunit.client.0.smithi027.stdout:0/894: readlink d9/da/d31/l58 0 2021-11-03T17:14:50.994 INFO:tasks.workunit.client.0.smithi027.stdout:0/895: mknod d9/da/d64/d70/c130 1 2021-11-03T17:14:50.995 INFO:tasks.workunit.client.0.smithi027.stdout:0/896: creat d9/f131 x:0 0 0 2021-11-03T17:14:50.999 INFO:tasks.workunit.client.0.smithi027.stdout:0/897: dread d9/da/d64/f68 [0,4194304] 0 2021-11-03T17:14:51.016 INFO:tasks.workunit.client.0.smithi027.stdout:0/898: dwrite d9/da/d30/dac/dba/ddc/f12f [0,4194304] 0 2021-11-03T17:14:51.027 INFO:tasks.workunit.client.0.smithi027.stdout:0/899: write d9/ffa [524172,34631] 0 2021-11-03T17:14:51.028 INFO:tasks.workunit.client.0.smithi027.stdout:0/900: write d9/f131 [932368,27327] 0 2021-11-03T17:14:51.029 INFO:tasks.workunit.client.0.smithi027.stdout:0/901: chown d9/da/d30/da7/l105 575 1 2021-11-03T17:14:51.030 INFO:tasks.workunit.client.0.smithi027.stdout:0/902: symlink d9/da/d64/d70/d6f/l132 0 2021-11-03T17:14:51.058 INFO:tasks.workunit.client.0.smithi027.stdout:0/903: dwrite d9/d12/f75 [0,4194304] 0 2021-11-03T17:14:51.060 INFO:tasks.workunit.client.0.smithi027.stdout:0/904: creat d9/da/d64/d122/f133 x:0 0 0 2021-11-03T17:14:51.066 INFO:tasks.workunit.client.0.smithi027.stdout:0/905: write d9/da/d64/d70/d6d/fa4 [4231276,977] 0 2021-11-03T17:14:51.070 INFO:tasks.workunit.client.0.smithi027.stdout:0/906: dread d9/da/d64/d70/fdb [4194304,4194304] 0 2021-11-03T17:14:51.073 INFO:tasks.workunit.client.0.smithi027.stdout:0/907: mkdir d9/d12/d8f/d134 0 2021-11-03T17:14:51.074 INFO:tasks.workunit.client.0.smithi027.stdout:0/908: truncate d9/da/d30/dac/fcf 3381893 0 2021-11-03T17:14:51.076 INFO:tasks.workunit.client.0.smithi027.stdout:0/909: fdatasync d9/f6c 0 2021-11-03T17:14:51.077 INFO:tasks.workunit.client.0.smithi027.stdout:0/910: chown d9/da/d64/d70/d6f/fa1 372001 1 2021-11-03T17:14:51.078 INFO:tasks.workunit.client.0.smithi027.stdout:0/911: readlink d9/da/d64/d70/d6d/l89 0 2021-11-03T17:14:51.080 INFO:tasks.workunit.client.0.smithi027.stdout:0/912: truncate d9/da/d64/d70/fcb 320312 0 2021-11-03T17:14:51.085 INFO:tasks.workunit.client.0.smithi027.stdout:0/913: dread d9/d12/def/ffb [0,4194304] 0 2021-11-03T17:14:51.085 INFO:tasks.workunit.client.0.smithi027.stdout:0/914: mknod d9/da/d64/dcc/c135 1 2021-11-03T17:14:51.087 INFO:tasks.workunit.client.0.smithi027.stdout:0/915: read d9/da/d30/da7/fab [31808,65734] 0 2021-11-03T17:14:51.088 INFO:tasks.workunit.client.0.smithi027.stdout:0/916: chown d9/da/d30/d92/fd2 241 1 2021-11-03T17:14:51.111 INFO:tasks.workunit.client.0.smithi027.stdout:0/917: dwrite d9/da/d64/d70/d6f/f80 [0,4194304] 0 2021-11-03T17:14:51.112 INFO:tasks.workunit.client.0.smithi027.stdout:0/918: rmdir d9/da/d64/d122 39 2021-11-03T17:14:51.114 INFO:tasks.workunit.client.0.smithi027.stdout:0/919: creat d9/d12/d8f/dc7/f136 x:0 0 0 2021-11-03T17:14:51.115 INFO:tasks.workunit.client.0.smithi027.stdout:0/920: chown d9/da/d30/dac/dba/ddc/lde 4914 1 2021-11-03T17:14:51.136 INFO:tasks.workunit.client.0.smithi027.stdout:0/921: dwrite d9/da/d64/d70/d6f/f10b [0,4194304] 0 2021-11-03T17:14:51.138 INFO:tasks.workunit.client.0.smithi027.stdout:0/922: dread - d9/dd3/f113 zero size 2021-11-03T17:14:51.145 INFO:tasks.workunit.client.0.smithi027.stdout:0/923: write d9/fbf [1649853,51129] 0 2021-11-03T17:14:51.146 INFO:tasks.workunit.client.0.smithi027.stdout:0/924: symlink d9/d12/d8f/l137 0 2021-11-03T17:14:51.148 INFO:tasks.workunit.client.0.smithi027.stdout:0/925: getdents d9/da 0 2021-11-03T17:14:51.155 INFO:tasks.workunit.client.0.smithi027.stdout:0/926: unlink d9/dfd/f10c 0 2021-11-03T17:14:51.167 INFO:tasks.workunit.client.0.smithi027.stdout:0/927: dread d9/f1f [0,4194304] 0 2021-11-03T17:14:51.168 INFO:tasks.workunit.client.0.smithi027.stdout:0/928: fdatasync d9/da/d30/f120 0 2021-11-03T17:14:51.175 INFO:tasks.workunit.client.0.smithi027.stdout:0/929: write d9/d12/f12d [1774922,115973] 0 2021-11-03T17:14:51.177 INFO:tasks.workunit.client.0.smithi027.stdout:0/930: symlink d9/d12/def/l138 0 2021-11-03T17:14:51.180 INFO:tasks.workunit.client.0.smithi027.stdout:0/931: mkdir d9/dd3/d139 0 2021-11-03T17:14:51.189 INFO:tasks.workunit.client.0.smithi027.stdout:0/932: dread d9/d12/d8f/fe2 [0,4194304] 0 2021-11-03T17:14:51.197 INFO:tasks.workunit.client.0.smithi027.stdout:0/933: write d9/d12/f17 [4284082,56454] 0 2021-11-03T17:14:51.217 INFO:tasks.workunit.client.0.smithi027.stdout:0/934: dwrite d9/da/f78 [4194304,4194304] 0 2021-11-03T17:14:51.249 INFO:tasks.workunit.client.0.smithi027.stdout:0/935: dwrite d9/da/d64/d70/d6f/fa1 [0,4194304] 0 2021-11-03T17:14:51.250 INFO:tasks.workunit.client.0.smithi027.stdout:0/936: read d9/da/f9d [613856,42502] 0 2021-11-03T17:14:51.251 INFO:tasks.workunit.client.0.smithi027.stdout:0/937: unlink d9/fbf 0 2021-11-03T17:14:51.252 INFO:tasks.workunit.client.0.smithi027.stdout:0/938: mknod d9/da/d64/d70/d6d/d9f/c13a 1 2021-11-03T17:14:51.253 INFO:tasks.workunit.client.0.smithi027.stdout:0/939: chown d9/da/d64/d70/d6d/lb0 842177 1 2021-11-03T17:14:51.254 INFO:tasks.workunit.client.0.smithi027.stdout:0/940: fdatasync d9/d12/f75 0 2021-11-03T17:14:51.254 INFO:tasks.workunit.client.0.smithi027.stdout:0/941: chown d9/da/d30/dac 512123 1 2021-11-03T17:14:51.258 INFO:tasks.workunit.client.0.smithi027.stdout:0/942: write d9/f42 [632082,7389] 0 2021-11-03T17:14:51.259 INFO:tasks.workunit.client.0.smithi027.stdout:0/943: write d9/d12/f17 [203789,96642] 0 2021-11-03T17:14:51.263 INFO:tasks.workunit.client.0.smithi027.stdout:0/944: unlink d9/da/d64/d70/d5f/f107 0 2021-11-03T17:14:51.266 INFO:tasks.workunit.client.0.smithi027.stdout:0/945: link d9/da/l90 d9/da/d64/d70/d6d/d7a/dc9/l13b 0 2021-11-03T17:14:51.268 INFO:tasks.workunit.client.0.smithi027.stdout:0/946: dread - d9/da/d30/d92/dca/fe0 zero size 2021-11-03T17:14:51.272 INFO:tasks.workunit.client.0.smithi027.stdout:0/947: creat d9/da/d30/dfe/d11f/f13c x:0 0 0 2021-11-03T17:14:51.274 INFO:tasks.workunit.client.0.smithi027.stdout:0/948: creat d9/da/d30/d92/dca/f13d x:0 0 0 2021-11-03T17:14:51.276 INFO:tasks.workunit.client.0.smithi027.stdout:0/949: creat d9/da/d64/dcc/f13e x:0 0 0 2021-11-03T17:14:51.276 INFO:tasks.workunit.client.0.smithi027.stdout:0/950: fsync d9/da/d64/d70/d6f/f80 0 2021-11-03T17:14:51.276 INFO:tasks.workunit.client.0.smithi027.stdout:0/951: fdatasync d9/d12/d8f/dc7/f136 0 2021-11-03T17:14:51.277 INFO:tasks.workunit.client.0.smithi027.stdout:0/952: creat d9/da/d10f/f13f x:0 0 0 2021-11-03T17:14:51.277 INFO:tasks.workunit.client.0.smithi027.stdout:0/953: chown d9/d12/d8f/l7f 4849 1 2021-11-03T17:14:51.278 INFO:tasks.workunit.client.0.smithi027.stdout:0/954: truncate d9/f6c 3182899 0 2021-11-03T17:14:51.279 INFO:tasks.workunit.client.0.smithi027.stdout:0/955: mkdir d9/da/d64/d70/d6d/d9f/d11a/d128/d140 0 2021-11-03T17:14:51.281 INFO:tasks.workunit.client.0.smithi027.stdout:0/956: getdents d9/da/d30/d93 0 2021-11-03T17:14:51.282 INFO:tasks.workunit.client.0.smithi027.stdout:0/957: creat d9/da/d30/dfe/d11f/f141 x:0 0 0 2021-11-03T17:14:51.284 INFO:tasks.workunit.client.0.smithi027.stdout:0/958: truncate d9/da/d30/fd8 1305381 0 2021-11-03T17:14:51.314 INFO:tasks.workunit.client.0.smithi027.stdout:0/959: dwrite d9/f13 [0,4194304] 0 2021-11-03T17:14:51.315 INFO:tasks.workunit.client.0.smithi027.stdout:0/960: fdatasync d9/f23 0 2021-11-03T17:14:51.341 INFO:tasks.workunit.client.0.smithi027.stdout:0/961: dwrite d9/da/d30/d93/fb2 [0,4194304] 0 2021-11-03T17:14:51.344 INFO:tasks.workunit.client.0.smithi027.stdout:0/962: getdents d9/da/d64/d70 0 2021-11-03T17:14:51.345 INFO:tasks.workunit.client.0.smithi027.stdout:0/963: chown d9/da/d30/f3c 554476 1 2021-11-03T17:14:51.439 INFO:tasks.workunit.client.0.smithi027.stdout:3/927: sync 2021-11-03T17:14:51.439 INFO:tasks.workunit.client.0.smithi027.stdout:9/763: sync 2021-11-03T17:14:51.440 INFO:tasks.workunit.client.0.smithi027.stdout:9/764: mknod db/d1b/dfa/c11e 1 2021-11-03T17:14:51.442 INFO:tasks.workunit.client.0.smithi027.stdout:3/928: stat d1/db/d52/d6a/db4/db5/f85 0 2021-11-03T17:14:51.448 INFO:tasks.workunit.client.0.smithi027.stdout:3/929: rename d1/l31 to d1/dec/l12c 0 2021-11-03T17:14:51.451 INFO:tasks.workunit.client.0.smithi027.stdout:3/930: link d1/db/d52/d6a/d7c/fd6 d1/db/d52/d6a/db4/f12d 0 2021-11-03T17:14:51.455 INFO:tasks.workunit.client.0.smithi027.stdout:3/931: read d1/db/db2/dbd/ffa [5267631,102243] 0 2021-11-03T17:14:51.458 INFO:tasks.workunit.client.0.smithi027.stdout:9/765: dwrite db/fa8 [0,4194304] 0 2021-11-03T17:14:51.458 INFO:tasks.workunit.client.0.smithi027.stdout:3/932: symlink d1/db/d52/d6a/d7c/l12e 0 2021-11-03T17:14:51.463 INFO:tasks.workunit.client.0.smithi027.stdout:9/766: symlink db/d1b/d1c/d1d/d40/l11f 0 2021-11-03T17:14:51.464 INFO:tasks.workunit.client.0.smithi027.stdout:9/767: chown db/f5b 0 1 2021-11-03T17:14:51.468 INFO:tasks.workunit.client.0.smithi027.stdout:3/933: write d1/d40/f48 [2164359,69182] 0 2021-11-03T17:14:51.471 INFO:tasks.workunit.client.0.smithi027.stdout:3/934: symlink d1/dec/l12f 0 2021-11-03T17:14:51.471 INFO:tasks.workunit.client.0.smithi027.stdout:3/935: stat d1/db/d52/d57/de3/l122 0 2021-11-03T17:14:51.472 INFO:tasks.workunit.client.0.smithi027.stdout:9/768: dread db/d1b/d2b/fe3 [0,4194304] 0 2021-11-03T17:14:51.472 INFO:tasks.workunit.client.0.smithi027.stdout:3/936: creat d1/db/f130 x:0 0 0 2021-11-03T17:14:51.474 INFO:tasks.workunit.client.0.smithi027.stdout:3/937: dread - d1/db/d52/d6a/db4/f121 zero size 2021-11-03T17:14:51.476 INFO:tasks.workunit.client.0.smithi027.stdout:9/769: truncate db/d1b/d2b/fd8 616832 0 2021-11-03T17:14:51.478 INFO:tasks.workunit.client.0.smithi027.stdout:3/938: write d1/f2a [2370634,13244] 0 2021-11-03T17:14:51.480 INFO:tasks.workunit.client.0.smithi027.stdout:9/770: dread db/d1b/f20 [0,4194304] 0 2021-11-03T17:14:51.481 INFO:tasks.workunit.client.0.smithi027.stdout:3/939: truncate d1/db/d52/d57/f71 3352389 0 2021-11-03T17:14:51.482 INFO:tasks.workunit.client.0.smithi027.stdout:9/771: mknod db/d1b/d2b/d41/dad/dbb/d11b/c120 1 2021-11-03T17:14:51.487 INFO:tasks.workunit.client.0.smithi027.stdout:3/940: fdatasync d1/fb8 0 2021-11-03T17:14:51.490 INFO:tasks.workunit.client.0.smithi027.stdout:3/941: creat d1/db/d52/d6a/d7c/d88/f131 x:0 0 0 2021-11-03T17:14:51.506 INFO:tasks.workunit.client.0.smithi027.stdout:3/942: dwrite d1/db/d52/d57/d99/d104/f127 [0,4194304] 0 2021-11-03T17:14:51.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:51 smithi027 conmon[32322]: cluster 2021-11-03T17:14:49.898770+0000 2021-11-03T17:14:51.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:51 smithi027 conmon[32322]: mgr.smithi115.tcyodz (mgr.24395) 31 : cluster [DBG] pgmap v20: 65 pgs: 65 active+clean; 3.0 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 37 MiB/s rd, 45 MiB/s wr, 97 op/s 2021-11-03T17:14:51.530 INFO:tasks.workunit.client.0.smithi027.stdout:3/943: dwrite d1/db/f60 [0,4194304] 0 2021-11-03T17:14:51.535 INFO:tasks.workunit.client.0.smithi027.stdout:3/944: write d1/db/db2/dbd/ffa [5150653,55936] 0 2021-11-03T17:14:51.535 INFO:tasks.workunit.client.0.smithi027.stdout:3/945: chown d1/d40 121 1 2021-11-03T17:14:51.537 INFO:tasks.workunit.client.0.smithi027.stdout:3/946: creat d1/db/f132 x:0 0 0 2021-11-03T17:14:51.537 INFO:tasks.workunit.client.0.smithi027.stdout:3/947: chown d1/db/d52/d6a/db4/db5 61408864 1 2021-11-03T17:14:51.554 INFO:tasks.workunit.client.0.smithi027.stdout:3/948: dread d1/db/f7d [0,4194304] 0 2021-11-03T17:14:51.558 INFO:tasks.workunit.client.0.smithi027.stdout:3/949: truncate d1/db/d52/d57/d8b/f9c 2106542 0 2021-11-03T17:14:51.560 INFO:tasks.workunit.client.0.smithi027.stdout:3/950: symlink d1/db/db2/l133 0 2021-11-03T17:14:51.585 INFO:tasks.workunit.client.0.smithi027.stdout:3/951: dwrite d1/f18 [4194304,4194304] 0 2021-11-03T17:14:51.589 INFO:tasks.workunit.client.0.smithi027.stdout:3/952: mkdir d1/db/d52/d57/d99/da7/d134 0 2021-11-03T17:14:51.615 INFO:tasks.workunit.client.0.smithi027.stdout:3/953: dwrite d1/db/d52/d6a/db4/f12d [0,4194304] 0 2021-11-03T17:14:51.620 INFO:tasks.workunit.client.0.smithi027.stdout:3/954: symlink d1/db/d52/d6a/d7c/d88/d98/l135 0 2021-11-03T17:14:51.621 INFO:tasks.workunit.client.0.smithi027.stdout:3/955: mknod d1/d40/d11f/c136 1 2021-11-03T17:14:51.622 INFO:tasks.workunit.client.0.smithi027.stdout:3/956: getdents d1/db/d52/d6a/db4 0 2021-11-03T17:14:51.622 INFO:tasks.workunit.client.0.smithi027.stdout:3/957: stat d1/db/d52/d57/d99/d104/f127 0 2021-11-03T17:14:51.635 INFO:tasks.workunit.client.0.smithi027.stdout:3/958: dwrite d1/db/d52/d6a/d7c/f89 [0,4194304] 0 2021-11-03T17:14:51.639 INFO:tasks.workunit.client.0.smithi027.stdout:3/959: unlink d1/db/f35 0 2021-11-03T17:14:51.642 INFO:tasks.workunit.client.0.smithi027.stdout:3/960: truncate d1/db/d52/d6a/d7c/f89 2851628 0 2021-11-03T17:14:51.643 INFO:tasks.workunit.client.0.smithi027.stdout:3/961: chown d1/l2b 1066 1 2021-11-03T17:14:51.643 INFO:tasks.workunit.client.0.smithi027.stdout:3/962: chown d1/dec 187222 1 2021-11-03T17:14:51.645 INFO:tasks.workunit.client.0.smithi027.stdout:3/963: creat d1/db/d52/d57/d8b/f137 x:0 0 0 2021-11-03T17:14:51.648 INFO:tasks.workunit.client.0.smithi027.stdout:3/964: creat d1/db/db2/dbd/d11b/dc6/f138 x:0 0 0 2021-11-03T17:14:51.655 INFO:tasks.workunit.client.0.smithi027.stdout:3/965: rename d1/d40/f46 to d1/db/d52/d6a/f139 0 2021-11-03T17:14:51.657 INFO:tasks.workunit.client.0.smithi027.stdout:3/966: creat d1/db/d52/d57/d99/d104/f13a x:0 0 0 2021-11-03T17:14:51.659 INFO:tasks.workunit.client.0.smithi027.stdout:4/780: sync 2021-11-03T17:14:51.662 INFO:tasks.workunit.client.0.smithi027.stdout:3/967: mknod d1/db/d52/d6a/db4/db5/d97/c13b 1 2021-11-03T17:14:51.663 INFO:tasks.workunit.client.0.smithi027.stdout:4/781: write d6/d9/d16/d36/d70/d3e/d58/f61 [837981,20515] 0 2021-11-03T17:14:51.665 INFO:tasks.workunit.client.0.smithi027.stdout:3/968: symlink d1/l13c 0 2021-11-03T17:14:51.668 INFO:tasks.workunit.client.0.smithi027.stdout:4/782: unlink d6/d9/d16/d17/ddd/fed 0 2021-11-03T17:14:51.668 INFO:tasks.workunit.client.0.smithi027.stdout:4/783: write d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/f7e [175125,102628] 0 2021-11-03T17:14:51.669 INFO:tasks.workunit.client.0.smithi027.stdout:3/969: truncate d1/db/f5b 1199432 0 2021-11-03T17:14:51.670 INFO:tasks.workunit.client.0.smithi027.stdout:4/784: chown d6/d24/db0/f83 31567527 1 2021-11-03T17:14:51.670 INFO:tasks.workunit.client.0.smithi027.stdout:3/970: chown d1/db/d52/l80 302773 1 2021-11-03T17:14:51.672 INFO:tasks.workunit.client.0.smithi027.stdout:3/971: creat d1/db/d52/d6a/db4/db5/d97/f13d x:0 0 0 2021-11-03T17:14:51.674 INFO:tasks.workunit.client.0.smithi027.stdout:4/785: mkdir d6/d24/de3/d102 0 2021-11-03T17:14:51.677 INFO:tasks.workunit.client.0.smithi027.stdout:3/972: creat d1/db/d52/d6a/d7c/d88/d98/f13e x:0 0 0 2021-11-03T17:14:51.678 INFO:tasks.workunit.client.0.smithi027.stdout:4/786: truncate d6/d9/d16/d36/d70/d3e/d58/fa4 527962 0 2021-11-03T17:14:51.685 INFO:tasks.workunit.client.0.smithi027.stdout:4/787: rename d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/f35 to d6/d24/de3/d9f/dd7/f103 0 2021-11-03T17:14:51.693 INFO:tasks.workunit.client.0.smithi027.stdout:3/973: write d1/db/d52/d57/d8b/f9c [1112184,15895] 0 2021-11-03T17:14:51.695 INFO:tasks.workunit.client.0.smithi027.stdout:3/974: creat d1/db/db2/dbd/d11b/f13f x:0 0 0 2021-11-03T17:14:51.697 INFO:tasks.workunit.client.0.smithi027.stdout:4/788: symlink d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d41/l104 0 2021-11-03T17:14:51.699 INFO:tasks.workunit.client.0.smithi027.stdout:4/789: creat d6/d9/f105 x:0 0 0 2021-11-03T17:14:51.701 INFO:tasks.workunit.client.0.smithi027.stdout:4/790: creat d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d9a/f106 x:0 0 0 2021-11-03T17:14:51.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:51 smithi115 conmon[34670]: cluster 2021-11-03T17:14:49.898770+0000 2021-11-03T17:14:51.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:51 smithi115 conmon[34670]: mgr.smithi115.tcyodz (mgr.24395) 31 : cluster [DBG] pgmap v20: 65 pgs: 65 active+clean; 3.0 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 37 MiB/s rd, 45 MiB/s wr, 97 op/s 2021-11-03T17:14:51.934 INFO:tasks.workunit.client.0.smithi027.stdout:2/820: sync 2021-11-03T17:14:51.934 INFO:tasks.workunit.client.0.smithi027.stdout:6/717: sync 2021-11-03T17:14:51.935 INFO:tasks.workunit.client.0.smithi027.stdout:1/901: sync 2021-11-03T17:14:51.936 INFO:tasks.workunit.client.0.smithi027.stdout:2/821: dread d0/d45/f7b [0,4194304] 0 2021-11-03T17:14:51.937 INFO:tasks.workunit.client.0.smithi027.stdout:6/718: dread dc/d16/d1b/d3c/d51/f5b [0,4194304] 0 2021-11-03T17:14:51.937 INFO:tasks.workunit.client.0.smithi027.stdout:1/902: read d6/d11/dac/d79/d25/d28/d41/fd8 [3289668,99369] 0 2021-11-03T17:14:51.938 INFO:tasks.workunit.client.0.smithi027.stdout:2/822: read d0/d1b/d29/d54/f7f [2033334,52988] 0 2021-11-03T17:14:51.941 INFO:tasks.workunit.client.0.smithi027.stdout:6/719: mkdir dc/d16/d1b/dda 0 2021-11-03T17:14:51.942 INFO:tasks.workunit.client.0.smithi027.stdout:1/903: rmdir d6/d11/dac/d79/d25/d28/d5c/d7a/dee 39 2021-11-03T17:14:51.944 INFO:tasks.workunit.client.0.smithi027.stdout:2/823: symlink d0/d1b/d87/lf7 0 2021-11-03T17:14:51.945 INFO:tasks.workunit.client.0.smithi027.stdout:2/824: chown d0/d1b/d29/d54/d64/daa 2296 1 2021-11-03T17:14:51.947 INFO:tasks.workunit.client.0.smithi027.stdout:2/825: dread d0/d1b/d29/d54/d67/f6c [0,4194304] 0 2021-11-03T17:14:51.949 INFO:tasks.workunit.client.0.smithi027.stdout:1/904: creat d6/d11/dac/d79/db0/f12d x:0 0 0 2021-11-03T17:14:51.950 INFO:tasks.workunit.client.0.smithi027.stdout:2/826: mknod d0/d1b/d29/d92/cf8 1 2021-11-03T17:14:51.950 INFO:tasks.workunit.client.0.smithi027.stdout:1/905: mknod d6/d11/dac/d79/d37/c12e 1 2021-11-03T17:14:51.963 INFO:tasks.workunit.client.0.smithi027.stdout:2/827: write d0/d5/fac [4756619,80746] 0 2021-11-03T17:14:51.963 INFO:tasks.workunit.client.0.smithi027.stdout:6/720: write dc/d82/d57/dad/dae/db1/d34/d89/f9d [1254560,8229] 0 2021-11-03T17:14:51.966 INFO:tasks.workunit.client.0.smithi027.stdout:1/906: unlink d6/lb 0 2021-11-03T17:14:51.968 INFO:tasks.workunit.client.0.smithi027.stdout:2/828: unlink d0/d1b/d29/d54/f7f 0 2021-11-03T17:14:51.968 INFO:tasks.workunit.client.0.smithi027.stdout:2/829: fdatasync d0/d1b/d29/d54/d64/daa/fec 0 2021-11-03T17:14:52.002 INFO:tasks.workunit.client.0.smithi027.stdout:2/830: dwrite d0/d1b/d29/d92/dbd/f80 [0,4194304] 0 2021-11-03T17:14:52.003 INFO:tasks.workunit.client.0.smithi027.stdout:1/907: dwrite d6/d11/dac/d79/d25/d28/d5c/d7a/f11e [0,4194304] 0 2021-11-03T17:14:52.003 INFO:tasks.workunit.client.0.smithi027.stdout:1/908: fdatasync d6/d11/dac/d79/d37/de0/de4/fb7 0 2021-11-03T17:14:52.005 INFO:tasks.workunit.client.0.smithi027.stdout:2/831: link d0/f10 d0/d1b/d29/d54/d64/daa/ff9 0 2021-11-03T17:14:52.009 INFO:tasks.workunit.client.0.smithi027.stdout:1/909: mknod d6/d11/dac/d79/d37/de0/de4/df9/c12f 1 2021-11-03T17:14:52.014 INFO:tasks.workunit.client.0.smithi027.stdout:2/832: dread d0/d5/f53 [0,4194304] 0 2021-11-03T17:14:52.014 INFO:tasks.workunit.client.0.smithi027.stdout:2/833: fsync d0/d1b/d29/d92/dbd/ff6 0 2021-11-03T17:14:52.020 INFO:tasks.workunit.client.0.smithi027.stdout:2/834: link d0/d1b/d29/d54/d64/daa/l4d d0/da7/lfa 0 2021-11-03T17:14:52.020 INFO:tasks.workunit.client.0.smithi027.stdout:1/910: write d6/d11/dac/d79/d37/de0/de4/d58/ff4 [896563,47638] 0 2021-11-03T17:14:52.028 INFO:tasks.workunit.client.0.smithi027.stdout:1/911: write d6/d11/dac/d79/d25/d28/d5c/ff5 [235155,420] 0 2021-11-03T17:14:52.029 INFO:tasks.workunit.client.0.smithi027.stdout:1/912: stat d6/d11/dac/d79 0 2021-11-03T17:14:52.032 INFO:tasks.workunit.client.0.smithi027.stdout:1/913: creat d6/d11/dac/d79/d25/d5e/dd1/f130 x:0 0 0 2021-11-03T17:14:52.047 INFO:tasks.workunit.client.0.smithi027.stdout:2/835: dwrite d0/d1b/d29/d54/d64/daa/d88/fc6 [0,4194304] 0 2021-11-03T17:14:52.048 INFO:tasks.workunit.client.0.smithi027.stdout:2/836: write d0/d1b/d29/d54/d64/daa/fec [891227,8356] 0 2021-11-03T17:14:52.056 INFO:tasks.workunit.client.0.smithi027.stdout:1/914: dwrite d6/de/f1a [0,4194304] 0 2021-11-03T17:14:52.060 INFO:tasks.workunit.client.0.smithi027.stdout:2/837: write d0/d1b/d29/d58/f5c [9303028,112242] 0 2021-11-03T17:14:52.066 INFO:tasks.workunit.client.0.smithi027.stdout:1/915: write d6/d11/dac/d79/d37/d49/fcb [702856,99266] 0 2021-11-03T17:14:52.068 INFO:tasks.workunit.client.0.smithi027.stdout:2/838: write d0/d1b/d1c/f57 [3846129,78397] 0 2021-11-03T17:14:52.068 INFO:tasks.workunit.client.0.smithi027.stdout:1/916: mkdir d6/d11/dac/d79/d37/de0/de4/df9/d131 0 2021-11-03T17:14:52.069 INFO:tasks.workunit.client.0.smithi027.stdout:2/839: creat d0/d1b/d29/d54/d64/daa/d88/ffb x:0 0 0 2021-11-03T17:14:52.071 INFO:tasks.workunit.client.0.smithi027.stdout:1/917: rmdir d6/d11/dac/d79/d25/d28/d5c/d7a/dee/d124 0 2021-11-03T17:14:52.072 INFO:tasks.workunit.client.0.smithi027.stdout:2/840: fdatasync d0/d1b/d29/d58/fd6 0 2021-11-03T17:14:52.075 INFO:tasks.workunit.client.0.smithi027.stdout:1/918: mkdir d6/d11/dac/d79/d25/d28/d5c/d132 0 2021-11-03T17:14:52.076 INFO:tasks.workunit.client.0.smithi027.stdout:1/919: chown d6/d11/dac/d79/l111 2066 1 2021-11-03T17:14:52.079 INFO:tasks.workunit.client.0.smithi027.stdout:1/920: rename d6/d11/dac/d79/d25/d28/la4 to d6/d11/dac/d79/d25/d28/d12c/l133 0 2021-11-03T17:14:52.097 INFO:tasks.workunit.client.0.smithi027.stdout:2/841: dwrite d0/d1b/d29/d58/f5c [0,4194304] 0 2021-11-03T17:14:52.098 INFO:tasks.workunit.client.0.smithi027.stdout:1/921: write d6/d11/dac/d79/d25/d28/d41/f6f [3212771,101025] 0 2021-11-03T17:14:52.102 INFO:tasks.workunit.client.0.smithi027.stdout:1/922: read d6/d11/d14/f17 [2107811,105638] 0 2021-11-03T17:14:52.102 INFO:tasks.workunit.client.0.smithi027.stdout:2/842: mkdir d0/da7/dfc 0 2021-11-03T17:14:52.104 INFO:tasks.workunit.client.0.smithi027.stdout:2/843: creat d0/d45/ffd x:0 0 0 2021-11-03T17:14:52.104 INFO:tasks.workunit.client.0.smithi027.stdout:2/844: stat d0/d1b/d87/le4 0 2021-11-03T17:14:52.107 INFO:tasks.workunit.client.0.smithi027.stdout:2/845: rename d0/da7/dfc to d0/dfe 0 2021-11-03T17:14:52.109 INFO:tasks.workunit.client.0.smithi027.stdout:2/846: symlink d0/d1b/d29/d54/lff 0 2021-11-03T17:14:52.109 INFO:tasks.workunit.client.0.smithi027.stdout:2/847: fdatasync d0/d1b/d29/d58/f5c 0 2021-11-03T17:14:52.110 INFO:tasks.workunit.client.0.smithi027.stdout:2/848: stat d0/d5/d2c/l52 0 2021-11-03T17:14:52.110 INFO:tasks.workunit.client.0.smithi027.stdout:2/849: chown d0/d1b/d29/d54/d64/daa/d5d 79240 1 2021-11-03T17:14:52.111 INFO:tasks.workunit.client.0.smithi027.stdout:1/923: dread d6/d11/dac/d79/d25/f47 [0,4194304] 0 2021-11-03T17:14:52.115 INFO:tasks.workunit.client.0.smithi027.stdout:2/850: mkdir d0/d1b/d1c/d100 0 2021-11-03T17:14:52.117 INFO:tasks.workunit.client.0.smithi027.stdout:2/851: dread - d0/d45/fb8 zero size 2021-11-03T17:14:52.118 INFO:tasks.workunit.client.0.smithi027.stdout:2/852: chown d0/d1b/d29/d54/d64/daa/l68 117692 1 2021-11-03T17:14:52.133 INFO:tasks.workunit.client.0.smithi027.stdout:1/924: dwrite d6/d11/dac/d79/d25/d28/d5c/d81/d7d/fca [0,4194304] 0 2021-11-03T17:14:52.133 INFO:tasks.workunit.client.0.smithi027.stdout:1/925: chown d6/d11/dac/d79/d37/d49/fcb 223427 1 2021-11-03T17:14:52.143 INFO:tasks.workunit.client.0.smithi027.stdout:1/926: mkdir d6/d11/de2/d134 0 2021-11-03T17:14:52.144 INFO:tasks.workunit.client.0.smithi027.stdout:1/927: creat d6/d11/dac/d79/d25/d28/d5c/d7a/dee/f135 x:0 0 0 2021-11-03T17:14:52.149 INFO:tasks.workunit.client.0.smithi027.stdout:1/928: mkdir d6/d11/dac/d79/db0/de5/d136 0 2021-11-03T17:14:52.151 INFO:tasks.workunit.client.0.smithi027.stdout:1/929: creat d6/d11/dac/d79/db0/de5/dff/dd7/d8b/f137 x:0 0 0 2021-11-03T17:14:52.154 INFO:tasks.workunit.client.0.smithi027.stdout:2/853: dwrite d0/d1b/d29/d54/d64/daa/d5d/f65 [0,4194304] 0 2021-11-03T17:14:52.157 INFO:tasks.workunit.client.0.smithi027.stdout:2/854: mkdir d0/d7c/d101 0 2021-11-03T17:14:52.158 INFO:tasks.workunit.client.0.smithi027.stdout:1/930: dread d6/d11/dac/d79/f59 [0,4194304] 0 2021-11-03T17:14:52.158 INFO:tasks.workunit.client.0.smithi027.stdout:2/855: dread - d0/da7/fe2 zero size 2021-11-03T17:14:52.158 INFO:tasks.workunit.client.0.smithi027.stdout:2/856: stat d0/d1b/d1c/lee 0 2021-11-03T17:14:52.262 INFO:tasks.workunit.client.0.smithi027.stdout:1/931: dread d6/d11/dac/d79/d37/de0/d43/f5d [0,4194304] 0 2021-11-03T17:14:52.275 INFO:tasks.workunit.client.0.smithi027.stdout:1/932: dread d6/d11/dac/d79/d25/d28/dec/f121 [0,4194304] 0 2021-11-03T17:14:52.276 INFO:tasks.workunit.client.0.smithi027.stdout:1/933: chown d6/d11/d14/l30 0 1 2021-11-03T17:14:52.281 INFO:tasks.workunit.client.0.smithi027.stdout:1/934: dread d6/d11/dac/d79/d25/d28/fd4 [0,4194304] 0 2021-11-03T17:14:52.282 INFO:tasks.workunit.client.0.smithi027.stdout:1/935: getdents d6/d11/dac/d10b 0 2021-11-03T17:14:52.284 INFO:tasks.workunit.client.0.smithi027.stdout:2/857: dwrite d0/da7/fbf [4194304,4194304] 0 2021-11-03T17:14:52.318 INFO:tasks.workunit.client.0.smithi027.stdout:1/936: truncate d6/d11/dac/d79/d25/d28/f55 1044668 0 2021-11-03T17:14:52.324 INFO:tasks.workunit.client.0.smithi027.stdout:2/858: write d0/d1b/d29/d54/d64/daa/d5d/f8f [5161430,124645] 0 2021-11-03T17:14:52.325 INFO:tasks.workunit.client.0.smithi027.stdout:2/859: mknod d0/d5/d2c/dc8/c102 1 2021-11-03T17:14:52.327 INFO:tasks.workunit.client.0.smithi027.stdout:2/860: creat d0/d1b/d29/d54/df3/f103 x:0 0 0 2021-11-03T17:14:52.356 INFO:tasks.workunit.client.0.smithi027.stdout:2/861: dwrite d0/d1b/d1c/f7e [0,4194304] 0 2021-11-03T17:14:52.362 INFO:tasks.workunit.client.0.smithi027.stdout:2/862: link d0/d5/f6d d0/d5/f104 0 2021-11-03T17:14:52.366 INFO:tasks.workunit.client.0.smithi027.stdout:2/863: unlink d0/d1b/d29/d92/dbd/f62 0 2021-11-03T17:14:52.368 INFO:tasks.workunit.client.0.smithi027.stdout:2/864: creat d0/d1b/d87/f105 x:0 0 0 2021-11-03T17:14:52.369 INFO:tasks.workunit.client.0.smithi027.stdout:2/865: stat d0/d1b/d29/d92/dbd/f8b 0 2021-11-03T17:14:52.370 INFO:tasks.workunit.client.0.smithi027.stdout:2/866: mknod d0/d1b/d29/d58/c106 1 2021-11-03T17:14:52.371 INFO:tasks.workunit.client.0.smithi027.stdout:2/867: chown d0/d1b/d29/d58/fb2 61 1 2021-11-03T17:14:52.374 INFO:tasks.workunit.client.0.smithi027.stdout:2/868: rmdir d0/da7/dc9/dd4 0 2021-11-03T17:14:52.377 INFO:tasks.workunit.client.0.smithi027.stdout:2/869: unlink d0/d1b/d29/d54/d64/daa/f23 0 2021-11-03T17:14:52.378 INFO:tasks.workunit.client.0.smithi027.stdout:2/870: creat d0/d1b/d29/d54/d64/daa/d5d/f107 x:0 0 0 2021-11-03T17:14:52.380 INFO:tasks.workunit.client.0.smithi027.stdout:2/871: symlink d0/d1b/d29/d54/d67/l108 0 2021-11-03T17:14:52.383 INFO:tasks.workunit.client.0.smithi027.stdout:2/872: unlink d0/f91 0 2021-11-03T17:14:52.383 INFO:tasks.workunit.client.0.smithi027.stdout:2/873: chown d0/d7c/df1 68233 1 2021-11-03T17:14:52.383 INFO:tasks.workunit.client.0.smithi027.stdout:2/874: fdatasync d0/d1b/d29/d54/d64/daa/fec 0 2021-11-03T17:14:52.384 INFO:tasks.workunit.client.0.smithi027.stdout:2/875: rmdir d0/da7/dc9 39 2021-11-03T17:14:52.386 INFO:tasks.workunit.client.0.smithi027.stdout:2/876: link d0/f10 d0/d45/f109 0 2021-11-03T17:14:52.387 INFO:tasks.workunit.client.0.smithi027.stdout:2/877: read d0/d1b/d29/fda [1936720,12975] 0 2021-11-03T17:14:52.390 INFO:tasks.workunit.client.0.smithi027.stdout:2/878: rename d0/d1b/d29/d58/f5c to d0/d1b/d29/d54/d64/f10a 0 2021-11-03T17:14:52.390 INFO:tasks.workunit.client.0.smithi027.stdout:2/879: mknod d0/d1b/d1c/dad/c10b 1 2021-11-03T17:14:52.398 INFO:tasks.workunit.client.0.smithi027.stdout:2/880: dread d0/d1b/d1c/f33 [0,4194304] 0 2021-11-03T17:14:52.400 INFO:tasks.workunit.client.0.smithi027.stdout:2/881: read d0/d1b/f27 [712390,19400] 0 2021-11-03T17:14:52.401 INFO:tasks.workunit.client.0.smithi027.stdout:2/882: getdents d0/d1b/d1c 0 2021-11-03T17:14:52.442 INFO:tasks.workunit.client.0.smithi027.stdout:2/883: dwrite d0/d5/d2c/f3d [0,4194304] 0 2021-11-03T17:14:52.464 INFO:tasks.workunit.client.0.smithi027.stdout:2/884: dwrite d0/d1b/d1c/f7e [0,4194304] 0 2021-11-03T17:14:52.466 INFO:tasks.workunit.client.0.smithi027.stdout:2/885: truncate d0/d45/f109 1381162 0 2021-11-03T17:14:52.466 INFO:tasks.workunit.client.0.smithi027.stdout:2/886: fsync d0/d45/fb8 0 2021-11-03T17:14:52.468 INFO:tasks.workunit.client.0.smithi027.stdout:2/887: mkdir d0/d10c 0 2021-11-03T17:14:52.472 INFO:tasks.workunit.client.0.smithi027.stdout:2/888: creat d0/d1b/d29/d54/d64/daa/f10d x:0 0 0 2021-11-03T17:14:52.475 INFO:tasks.workunit.client.0.smithi027.stdout:2/889: truncate d0/d1b/d29/d54/d64/daa/f94 251811 0 2021-11-03T17:14:52.475 INFO:tasks.workunit.client.0.smithi027.stdout:2/890: dread - d0/d1b/d29/d92/dbd/ff6 zero size 2021-11-03T17:14:52.483 INFO:tasks.workunit.client.0.smithi027.stdout:2/891: dread d0/d1b/d29/d54/d67/f6c [0,4194304] 0 2021-11-03T17:14:52.486 INFO:tasks.workunit.client.0.smithi027.stdout:2/892: symlink d0/d1b/d29/d54/d64/daa/d5d/l10e 0 2021-11-03T17:14:52.489 INFO:tasks.workunit.client.0.smithi027.stdout:2/893: symlink d0/d1b/d29/db5/dcc/l10f 0 2021-11-03T17:14:52.498 INFO:tasks.workunit.client.0.smithi027.stdout:2/894: dread d0/f90 [0,4194304] 0 2021-11-03T17:14:52.500 INFO:tasks.workunit.client.0.smithi027.stdout:2/895: creat d0/d5/f110 x:0 0 0 2021-11-03T17:14:52.501 INFO:tasks.workunit.client.0.smithi027.stdout:2/896: rmdir d0/d1b/d29/d54 39 2021-11-03T17:14:52.512 INFO:tasks.workunit.client.0.smithi027.stdout:2/897: write d0/f46 [1796683,62012] 0 2021-11-03T17:14:52.513 INFO:tasks.workunit.client.0.smithi027.stdout:2/898: mknod d0/da7/dc9/c111 1 2021-11-03T17:14:52.513 INFO:tasks.workunit.client.0.smithi027.stdout:2/899: chown d0/da7 40 1 2021-11-03T17:14:52.515 INFO:tasks.workunit.client.0.smithi027.stdout:2/900: creat d0/d1b/f112 x:0 0 0 2021-11-03T17:14:52.516 INFO:tasks.workunit.client.0.smithi027.stdout:2/901: read d0/d1b/f27 [84850,53833] 0 2021-11-03T17:14:52.525 INFO:tasks.workunit.client.0.smithi027.stdout:2/902: rmdir d0 39 2021-11-03T17:14:52.527 INFO:tasks.workunit.client.0.smithi027.stdout:2/903: link d0/d5/f53 d0/d10c/f113 0 2021-11-03T17:14:52.529 INFO:tasks.workunit.client.0.smithi027.stdout:2/904: mkdir d0/d1b/d114 0 2021-11-03T17:14:52.548 INFO:tasks.workunit.client.0.smithi027.stdout:2/905: dwrite d0/d5/fac [0,4194304] 0 2021-11-03T17:14:52.549 INFO:tasks.workunit.client.0.smithi027.stdout:2/906: symlink d0/da7/dc9/l115 0 2021-11-03T17:14:52.552 INFO:tasks.workunit.client.0.smithi027.stdout:2/907: getdents d0/d1b/d29/d54/d67 0 2021-11-03T17:14:52.575 INFO:tasks.workunit.client.0.smithi027.stdout:2/908: dwrite d0/f3b [0,4194304] 0 2021-11-03T17:14:52.577 INFO:tasks.workunit.client.0.smithi027.stdout:2/909: creat d0/d5/f116 x:0 0 0 2021-11-03T17:14:52.584 INFO:tasks.workunit.client.0.smithi027.stdout:2/910: write d0/d1b/d29/d54/fb4 [1762555,119198] 0 2021-11-03T17:14:52.585 INFO:tasks.workunit.client.0.smithi027.stdout:2/911: unlink d0/da7/lfa 0 2021-11-03T17:14:52.587 INFO:tasks.workunit.client.0.smithi027.stdout:2/912: creat d0/da7/dc9/f117 x:0 0 0 2021-11-03T17:14:52.588 INFO:tasks.workunit.client.0.smithi027.stdout:2/913: mknod d0/d5/d2c/c118 1 2021-11-03T17:14:52.589 INFO:tasks.workunit.client.0.smithi027.stdout:2/914: read d0/f9b [1143814,91131] 0 2021-11-03T17:14:52.598 INFO:tasks.workunit.client.0.smithi027.stdout:2/915: truncate d0/d1b/d29/fda 3159135 0 2021-11-03T17:14:53.194 INFO:tasks.workunit.client.0.smithi027.stdout:7/611: sync 2021-11-03T17:14:53.200 INFO:tasks.workunit.client.0.smithi027.stdout:7/612: link d0/d2d/d53/f65 d0/d2/d10/fcb 0 2021-11-03T17:14:53.202 INFO:tasks.workunit.client.0.smithi027.stdout:7/613: truncate d0/d2/d1c/d4d/d6e/d84/f92 1361672 0 2021-11-03T17:14:53.203 INFO:tasks.workunit.client.0.smithi027.stdout:7/614: readlink d0/d2d/d58/lb2 0 2021-11-03T17:14:53.225 INFO:tasks.workunit.client.0.smithi027.stdout:0/964: sync 2021-11-03T17:14:53.228 INFO:tasks.workunit.client.0.smithi027.stdout:0/965: link d9/da/d30/lc6 d9/dfd/l142 0 2021-11-03T17:14:53.236 INFO:tasks.workunit.client.0.smithi027.stdout:0/966: write d9/da/d30/dac/fea [840125,91946] 0 2021-11-03T17:14:53.237 INFO:tasks.workunit.client.0.smithi027.stdout:0/967: write d9/d12/f1d [5071637,59655] 0 2021-11-03T17:14:53.237 INFO:tasks.workunit.client.0.smithi027.stdout:0/968: dread - d9/da/d64/d122/f133 zero size 2021-11-03T17:14:53.238 INFO:tasks.workunit.client.0.smithi027.stdout:0/969: truncate d9/f6c 302554 0 2021-11-03T17:14:53.241 INFO:tasks.workunit.client.0.smithi027.stdout:0/970: link d9/da/f39 d9/da/d64/d70/d6f/f143 0 2021-11-03T17:14:53.241 INFO:tasks.workunit.client.0.smithi027.stdout:0/971: stat d9/da/d30/f120 0 2021-11-03T17:14:53.242 INFO:tasks.workunit.client.0.smithi027.stdout:0/972: creat d9/da/d64/dcc/f144 x:0 0 0 2021-11-03T17:14:53.250 INFO:tasks.workunit.client.0.smithi027.stdout:0/973: rename d9/d12/d8f/dc7 to d9/da/d64/d70/d6f/d145 0 2021-11-03T17:14:53.252 INFO:tasks.workunit.client.0.smithi027.stdout:0/974: dread d9/f42 [0,4194304] 0 2021-11-03T17:14:53.260 INFO:tasks.workunit.client.0.smithi027.stdout:0/975: dread d9/da/d64/d70/d6f/f80 [0,4194304] 0 2021-11-03T17:14:53.267 INFO:tasks.workunit.client.0.smithi027.stdout:0/976: dread d9/fb7 [0,4194304] 0 2021-11-03T17:14:53.270 INFO:tasks.workunit.client.0.smithi027.stdout:0/977: symlink d9/da/d30/d92/d110/l146 0 2021-11-03T17:14:53.272 INFO:tasks.workunit.client.0.smithi027.stdout:0/978: creat d9/da/d30/dac/f147 x:0 0 0 2021-11-03T17:14:53.272 INFO:tasks.workunit.client.0.smithi027.stdout:0/979: rmdir d9/dd3 39 2021-11-03T17:14:53.297 INFO:tasks.workunit.client.0.smithi027.stdout:0/980: dwrite d9/f1a [0,4194304] 0 2021-11-03T17:14:53.298 INFO:tasks.workunit.client.0.smithi027.stdout:0/981: dread - d9/da/d64/d70/d6d/d9f/fc3 zero size 2021-11-03T17:14:53.300 INFO:tasks.workunit.client.0.smithi027.stdout:0/982: unlink d9/ffa 0 2021-11-03T17:14:53.324 INFO:tasks.workunit.client.0.smithi027.stdout:0/983: dwrite d9/da/f10 [0,4194304] 0 2021-11-03T17:14:53.327 INFO:tasks.workunit.client.0.smithi027.stdout:3/975: sync 2021-11-03T17:14:53.328 INFO:tasks.workunit.client.0.smithi027.stdout:9/772: sync 2021-11-03T17:14:53.339 INFO:tasks.workunit.client.0.smithi027.stdout:3/976: write d1/db/f3c [397305,84545] 0 2021-11-03T17:14:53.339 INFO:tasks.workunit.client.0.smithi027.stdout:0/984: dread d9/f1f [0,4194304] 0 2021-11-03T17:14:53.340 INFO:tasks.workunit.client.0.smithi027.stdout:0/985: chown d9/da/d30/dac/dba/ddc/d11e 1780 1 2021-11-03T17:14:53.342 INFO:tasks.workunit.client.0.smithi027.stdout:3/977: mkdir d1/d140 0 2021-11-03T17:14:53.345 INFO:tasks.workunit.client.0.smithi027.stdout:0/986: creat d9/da/d64/d70/d6d/d9f/d11a/d129/f148 x:0 0 0 2021-11-03T17:14:53.345 INFO:tasks.workunit.client.0.smithi027.stdout:0/987: chown d9/da/f8b 83 1 2021-11-03T17:14:53.347 INFO:tasks.workunit.client.0.smithi027.stdout:3/978: getdents d1/d40 0 2021-11-03T17:14:53.352 INFO:tasks.workunit.client.0.smithi027.stdout:3/979: rename d1/db/d52/d57/d99/l11e to d1/db/d52/d6a/l141 0 2021-11-03T17:14:53.354 INFO:tasks.workunit.client.0.smithi027.stdout:3/980: chown d1/db/d52/d6a/db4/db5/f81 101 1 2021-11-03T17:14:53.355 INFO:tasks.workunit.client.0.smithi027.stdout:3/981: chown d1/f50 5 1 2021-11-03T17:14:53.356 INFO:tasks.workunit.client.0.smithi027.stdout:3/982: chown d1/d40/fb6 3316 1 2021-11-03T17:14:53.357 INFO:tasks.workunit.client.0.smithi027.stdout:3/983: chown d1/db/d52/d6a/l72 1 1 2021-11-03T17:14:53.357 INFO:tasks.workunit.client.0.smithi027.stdout:3/984: chown d1/db/d52/d6a/db4/lbb 70388 1 2021-11-03T17:14:53.358 INFO:tasks.workunit.client.0.smithi027.stdout:3/985: write d1/db/f3c [601632,29617] 0 2021-11-03T17:14:53.360 INFO:tasks.workunit.client.0.smithi027.stdout:3/986: truncate d1/d40/f48 4644914 0 2021-11-03T17:14:53.360 INFO:tasks.workunit.client.0.smithi027.stdout:3/987: stat d1/d40/fb6 0 2021-11-03T17:14:53.365 INFO:tasks.workunit.client.0.smithi027.stdout:3/988: dread d1/db/d52/d6a/d7c/f89 [0,4194304] 0 2021-11-03T17:14:53.372 INFO:tasks.workunit.client.0.smithi027.stdout:0/988: dwrite d9/da/f16 [0,4194304] 0 2021-11-03T17:14:53.375 INFO:tasks.workunit.client.0.smithi027.stdout:0/989: read d9/da/d64/d70/d6f/f10b [843428,80123] 0 2021-11-03T17:14:53.376 INFO:tasks.workunit.client.0.smithi027.stdout:0/990: fdatasync d9/da/d30/dac/f147 0 2021-11-03T17:14:53.377 INFO:tasks.workunit.client.0.smithi027.stdout:3/989: rename d1/db/d52/d6a to d1/db/d52/d57/d99/df6/d142 0 2021-11-03T17:14:53.391 INFO:tasks.workunit.client.0.smithi027.stdout:3/990: rename d1/db/fcc to d1/db/d52/d57/d99/df6/d142/dd5/d111/f143 0 2021-11-03T17:14:53.405 INFO:tasks.workunit.client.0.smithi027.stdout:0/991: dwrite d9/da/d64/d70/d6f/fe9 [0,4194304] 0 2021-11-03T17:14:53.405 INFO:tasks.workunit.client.0.smithi027.stdout:0/992: stat d9/da/d64/d70/d6d/d9f 0 2021-11-03T17:14:53.432 INFO:tasks.workunit.client.0.smithi027.stdout:3/991: readlink d1/db/d52/l56 0 2021-11-03T17:14:53.435 INFO:tasks.workunit.client.0.smithi027.stdout:0/993: creat d9/da/d64/d70/d6d/d7a/f149 x:0 0 0 2021-11-03T17:14:53.436 INFO:tasks.workunit.client.0.smithi027.stdout:3/992: creat d1/d40/d11f/f144 x:0 0 0 2021-11-03T17:14:53.440 INFO:tasks.workunit.client.0.smithi027.stdout:3/993: creat d1/d40/f145 x:0 0 0 2021-11-03T17:14:53.469 INFO:tasks.workunit.client.0.smithi027.stdout:0/994: dwrite d9/da/f9d [0,4194304] 0 2021-11-03T17:14:53.470 INFO:tasks.workunit.client.0.smithi027.stdout:3/994: dwrite d1/db/d52/d57/fd2 [0,4194304] 0 2021-11-03T17:14:53.475 INFO:tasks.workunit.client.0.smithi027.stdout:0/995: symlink d9/da/d30/d92/d110/d117/l14a 0 2021-11-03T17:14:53.476 INFO:tasks.workunit.client.0.smithi027.stdout:0/996: mkdir d9/da/d64/d70/d14b 0 2021-11-03T17:14:53.477 INFO:tasks.workunit.client.0.smithi027.stdout:0/997: mknod d9/da/d30/dac/dba/c14c 1 2021-11-03T17:14:53.478 INFO:tasks.workunit.client.0.smithi027.stdout:3/995: write d1/db/d52/d57/d99/df6/d142/db4/db5/d97/fc9 [6239,111802] 0 2021-11-03T17:14:53.479 INFO:tasks.workunit.client.0.smithi027.stdout:0/998: creat d9/da/d30/f14d x:0 0 0 2021-11-03T17:14:53.483 INFO:tasks.workunit.client.0.smithi027.stdout:0/999: chown d9/da/d64/d70/d6d/f7d 1 1 2021-11-03T17:14:53.487 INFO:tasks.workunit.client.0.smithi027.stdout:3/996: write d1/db/d52/f5d [1351206,102107] 0 2021-11-03T17:14:53.487 INFO:tasks.workunit.client.0.smithi027.stdout:3/997: mknod d1/db/d52/d57/d99/df6/d142/dd5/d120/c146 1 2021-11-03T17:14:53.489 INFO:tasks.workunit.client.0.smithi027.stdout:3/998: dread d1/db/db2/f106 [0,4194304] 0 2021-11-03T17:14:53.494 INFO:tasks.workunit.client.0.smithi027.stdout:3/999: write d1/d40/f8a [1455905,88209] 0 2021-11-03T17:14:53.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:53 smithi027 conmon[32322]: cluster 2021-11-03T17:14:51.899160+0000 mgr.smithi115.tcyodz 2021-11-03T17:14:53.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:53 smithi027 conmon[32322]: (mgr.24395) 32 : cluster [DBG] pgmap v21: 65 pgs: 65 active+clean; 3.0 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 14 MiB/s rd, 21 MiB/s wr, 53 op/s 2021-11-03T17:14:53.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:53 smithi115 conmon[34670]: cluster 2021-11-03T17:14:51.899160+0000 mgr.smithi115.tcyodz (mgr.24395) 32 : 2021-11-03T17:14:53.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:53 smithi115 conmon[34670]: cluster [DBG] pgmap v21: 65 pgs: 65 active+clean; 3.0 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 14 MiB/s rd, 21 MiB/s wr, 53 op/s 2021-11-03T17:14:53.814 INFO:tasks.workunit.client.0.smithi027.stdout:4/791: sync 2021-11-03T17:14:53.817 INFO:tasks.workunit.client.0.smithi027.stdout:4/792: dread d6/d9/d16/d36/d70/d3e/d58/d84/f86 [0,4194304] 0 2021-11-03T17:14:53.828 INFO:tasks.workunit.client.0.smithi027.stdout:4/793: write d6/d9/d16/d36/d99/db9/fd6 [2302551,24217] 0 2021-11-03T17:14:53.834 INFO:tasks.workunit.client.0.smithi027.stdout:4/794: write d6/d9/d16/d17/f6e [1479422,32303] 0 2021-11-03T17:14:53.837 INFO:tasks.workunit.client.0.smithi027.stdout:4/795: creat d6/d9/d16/d36/d70/d3e/de9/f107 x:0 0 0 2021-11-03T17:14:53.837 INFO:tasks.workunit.client.0.smithi027.stdout:4/796: mknod d6/d24/d6d/db1/c108 1 2021-11-03T17:14:53.861 INFO:tasks.workunit.client.0.smithi027.stdout:4/797: dwrite d6/d9/d16/d36/d70/d3e/f6c [4194304,4194304] 0 2021-11-03T17:14:53.872 INFO:tasks.workunit.client.0.smithi027.stdout:4/798: dread d6/d9/d16/d36/d57/f75 [0,4194304] 0 2021-11-03T17:14:53.877 INFO:tasks.workunit.client.0.smithi027.stdout:4/799: write d6/d9/d16/d36/d99/db9/f96 [331501,118165] 0 2021-11-03T17:14:53.896 INFO:tasks.workunit.client.0.smithi027.stdout:6/721: sync 2021-11-03T17:14:53.899 INFO:tasks.workunit.client.0.smithi027.stdout:6/722: symlink dc/d16/d1b/d3c/d51/ldb 0 2021-11-03T17:14:53.901 INFO:tasks.workunit.client.0.smithi027.stdout:6/723: dread - dc/d16/d9a/fc7 zero size 2021-11-03T17:14:53.903 INFO:tasks.workunit.client.0.smithi027.stdout:6/724: mkdir dc/d16/d1b/d3c/ddc 0 2021-11-03T17:14:53.906 INFO:tasks.workunit.client.0.smithi027.stdout:6/725: chown dc/d16/d1b/l42 117 1 2021-11-03T17:14:53.910 INFO:tasks.workunit.client.0.smithi027.stdout:6/726: rename dc/d16/d1b/d3c/l6b to dc/d16/d1b/dda/ldd 0 2021-11-03T17:14:53.912 INFO:tasks.workunit.client.0.smithi027.stdout:6/727: creat dc/d16/d1b/d3c/ddc/fde x:0 0 0 2021-11-03T17:14:53.927 INFO:tasks.workunit.client.0.smithi027.stdout:6/728: dwrite dc/d82/d57/d9b/fd0 [0,4194304] 0 2021-11-03T17:14:53.928 INFO:tasks.workunit.client.0.smithi027.stdout:6/729: mknod dc/cdf 1 2021-11-03T17:14:53.928 INFO:tasks.workunit.client.0.smithi027.stdout:6/730: mknod dc/d82/ce0 1 2021-11-03T17:14:53.929 INFO:tasks.workunit.client.0.smithi027.stdout:6/731: chown dc/d16/d1b/f75 7497017 1 2021-11-03T17:14:53.930 INFO:tasks.workunit.client.0.smithi027.stdout:6/732: fsync dc/d16/d1b/d3c/d51/f5b 0 2021-11-03T17:14:53.934 INFO:tasks.workunit.client.0.smithi027.stdout:6/733: creat dc/d37/fe1 x:0 0 0 2021-11-03T17:14:53.935 INFO:tasks.workunit.client.0.smithi027.stdout:6/734: chown dc/d82/d57/dad/dae/db1/d34/f8a 289 1 2021-11-03T17:14:53.938 INFO:tasks.workunit.client.0.smithi027.stdout:6/735: chown dc/d82/d57/dad/dae/db1/l41 7 1 2021-11-03T17:14:54.005 INFO:tasks.workunit.client.0.smithi027.stdout:1/937: sync 2021-11-03T17:14:54.007 INFO:tasks.workunit.client.0.smithi027.stdout:1/938: stat d6/d11/dac/d79/dab/db4/fc5 0 2021-11-03T17:14:54.010 INFO:tasks.workunit.client.0.smithi027.stdout:1/939: mkdir d6/d11/dac/d79/d25/d28/d12c/d138 0 2021-11-03T17:14:54.012 INFO:tasks.workunit.client.0.smithi027.stdout:1/940: fdatasync d6/d11/dac/d79/d37/de0/de4/f54 0 2021-11-03T17:14:54.022 INFO:tasks.workunit.client.0.smithi027.stdout:1/941: dread d6/d11/dac/d79/d37/de0/f4b [0,4194304] 0 2021-11-03T17:14:54.024 INFO:tasks.workunit.client.0.smithi027.stdout:1/942: creat d6/d11/dac/d79/d104/f139 x:0 0 0 2021-11-03T17:14:54.048 INFO:tasks.workunit.client.0.smithi027.stdout:1/943: dwrite d6/d11/dac/d79/d25/d28/d5c/d81/d7d/fdb [0,4194304] 0 2021-11-03T17:14:54.060 INFO:tasks.workunit.client.0.smithi027.stdout:1/944: dread d6/d11/dac/d79/d25/f2d [0,4194304] 0 2021-11-03T17:14:54.060 INFO:tasks.workunit.client.0.smithi027.stdout:1/945: write d6/d11/dac/d79/d25/d28/d5c/d7a/fbe [12541249,66075] 0 2021-11-03T17:14:54.061 INFO:tasks.workunit.client.0.smithi027.stdout:1/946: readlink d6/d11/dac/d79/d25/d28/l39 0 2021-11-03T17:14:54.064 INFO:tasks.workunit.client.0.smithi027.stdout:1/947: unlink d6/d11/dac/d79/d37/de0/la5 0 2021-11-03T17:14:54.073 INFO:tasks.workunit.client.0.smithi027.stdout:1/948: dread d6/d11/dac/d79/d37/de0/d43/f48 [4194304,4194304] 0 2021-11-03T17:14:54.075 INFO:tasks.workunit.client.0.smithi027.stdout:1/949: creat d6/d11/dac/d79/f13a x:0 0 0 2021-11-03T17:14:54.092 INFO:tasks.workunit.client.0.smithi027.stdout:1/950: dwrite d6/d11/dac/d79/d25/d28/d5c/d7a/dee/f135 [0,4194304] 0 2021-11-03T17:14:54.097 INFO:tasks.workunit.client.0.smithi027.stdout:1/951: unlink d6/d11/dac/d79/db0/de5/dff/dd7/d8b/f137 0 2021-11-03T17:14:54.106 INFO:tasks.workunit.client.0.smithi027.stdout:1/952: write d6/d11/dac/d79/d25/d28/f55 [2051639,12851] 0 2021-11-03T17:14:54.108 INFO:tasks.workunit.client.0.smithi027.stdout:1/953: chown d6/d11/dac/d79/d25/d28/d41/f6f 1450 1 2021-11-03T17:14:54.132 INFO:tasks.workunit.client.0.smithi027.stdout:1/954: dwrite d6/de/f117 [0,4194304] 0 2021-11-03T17:14:54.134 INFO:tasks.workunit.client.0.smithi027.stdout:1/955: chown d6/d11/dac/d79/d25/d28/d5c/d81/fdc 33 1 2021-11-03T17:14:54.134 INFO:tasks.workunit.client.0.smithi027.stdout:1/956: fsync d6/d11/dac/d79/d25/d28/d5c/d7a/dee/f135 0 2021-11-03T17:14:54.155 INFO:tasks.workunit.client.0.smithi027.stdout:1/957: dwrite d6/d11/dac/d79/db0/de5/dff/f8e [0,4194304] 0 2021-11-03T17:14:54.156 INFO:tasks.workunit.client.0.smithi027.stdout:1/958: chown d6/d11/dac/d79/d25/d5e/d72/l10e 41 1 2021-11-03T17:14:54.508 INFO:tasks.workunit.client.0.smithi027.stdout:2/916: sync 2021-11-03T17:14:54.540 INFO:tasks.workunit.client.0.smithi027.stdout:2/917: symlink d0/d1b/l119 0 2021-11-03T17:14:54.547 INFO:tasks.workunit.client.0.smithi027.stdout:2/918: mkdir d0/d45/d11a 0 2021-11-03T17:14:54.547 INFO:tasks.workunit.client.0.smithi027.stdout:2/919: write d0/d1b/d29/d54/d64/daa/d88/ffb [506398,62334] 0 2021-11-03T17:14:54.551 INFO:tasks.workunit.client.0.smithi027.stdout:2/920: creat d0/d1b/d29/db5/dcc/f11b x:0 0 0 2021-11-03T17:14:54.551 INFO:tasks.workunit.client.0.smithi027.stdout:2/921: chown d0/da7/dc9/led 45 1 2021-11-03T17:14:54.576 INFO:tasks.workunit.client.0.smithi027.stdout:2/922: dwrite d0/d1b/d29/f4b [0,4194304] 0 2021-11-03T17:14:54.580 INFO:tasks.workunit.client.0.smithi027.stdout:2/923: mkdir d0/d1b/d1c/d100/d11c 0 2021-11-03T17:14:54.601 INFO:tasks.workunit.client.0.smithi027.stdout:2/924: dwrite d0/d1b/d29/f4b [4194304,4194304] 0 2021-11-03T17:14:54.601 INFO:tasks.workunit.client.0.smithi027.stdout:2/925: rmdir d0/d1b/d29/d54/d64/daa/d5d 39 2021-11-03T17:14:54.607 INFO:tasks.workunit.client.0.smithi027.stdout:2/926: symlink d0/d1b/d29/db5/l11d 0 2021-11-03T17:14:54.607 INFO:tasks.workunit.client.0.smithi027.stdout:2/927: fdatasync d0/d5/f104 0 2021-11-03T17:14:54.608 INFO:tasks.workunit.client.0.smithi027.stdout:2/928: chown d0/d1b/d29/f2b 80 1 2021-11-03T17:14:54.614 INFO:tasks.workunit.client.0.smithi027.stdout:2/929: dread d0/d45/f9a [0,4194304] 0 2021-11-03T17:14:54.629 INFO:tasks.workunit.client.0.smithi027.stdout:2/930: dwrite d0/d5/fac [4194304,4194304] 0 2021-11-03T17:14:54.651 INFO:tasks.workunit.client.0.smithi027.stdout:2/931: dwrite d0/d1b/d1c/f7e [0,4194304] 0 2021-11-03T17:14:54.652 INFO:tasks.workunit.client.0.smithi027.stdout:2/932: creat d0/da7/f11e x:0 0 0 2021-11-03T17:14:54.654 INFO:tasks.workunit.client.0.smithi027.stdout:2/933: symlink d0/d1b/d29/d54/d67/l11f 0 2021-11-03T17:14:54.658 INFO:tasks.workunit.client.0.smithi027.stdout:2/934: link d0/fa9 d0/d5/f120 0 2021-11-03T17:14:54.663 INFO:tasks.workunit.client.0.smithi027.stdout:2/935: rename d0/d1b/l4c to d0/d1b/d29/d92/l121 0 2021-11-03T17:14:54.663 INFO:tasks.workunit.client.0.smithi027.stdout:2/936: chown d0/d1b/d29/d54/d64/daa/d88 252405 1 2021-11-03T17:14:54.664 INFO:tasks.workunit.client.0.smithi027.stdout:2/937: fdatasync d0/f10 0 2021-11-03T17:14:54.665 INFO:tasks.workunit.client.0.smithi027.stdout:2/938: write d0/fa9 [3220253,55550] 0 2021-11-03T17:14:54.675 INFO:tasks.workunit.client.0.smithi027.stdout:2/939: write d0/d1b/d29/d92/dbd/f8b [356267,52184] 0 2021-11-03T17:14:55.235 INFO:tasks.workunit.client.0.smithi027.stdout:7/615: sync 2021-11-03T17:14:55.235 INFO:tasks.workunit.client.0.smithi027.stdout:9/773: sync 2021-11-03T17:14:55.235 INFO:tasks.workunit.client.0.smithi027.stdout:9/774: chown db/d1b/d23 100 1 2021-11-03T17:14:55.256 INFO:tasks.workunit.client.0.smithi027.stdout:9/775: dwrite db/d1b/d2b/d41/d6c/f93 [0,4194304] 0 2021-11-03T17:14:55.264 INFO:tasks.workunit.client.0.smithi027.stdout:9/776: write db/d1b/d1c/fd0 [487363,13677] 0 2021-11-03T17:14:55.265 INFO:tasks.workunit.client.0.smithi027.stdout:9/777: creat db/f121 x:0 0 0 2021-11-03T17:14:55.267 INFO:tasks.workunit.client.0.smithi027.stdout:9/778: mknod db/d1b/d2b/d5e/daa/c122 1 2021-11-03T17:14:55.267 INFO:tasks.workunit.client.0.smithi027.stdout:9/779: chown db/d1b/fb2 0 1 2021-11-03T17:14:55.270 INFO:tasks.workunit.client.0.smithi027.stdout:9/780: dread db/d1b/d2b/d41/f74 [0,4194304] 0 2021-11-03T17:14:55.274 INFO:tasks.workunit.client.0.smithi027.stdout:9/781: rename db/f1f to db/d1b/d23/f123 0 2021-11-03T17:14:55.278 INFO:tasks.workunit.client.0.smithi027.stdout:9/782: link db/d1b/d2b/d41/f82 db/d1b/d2b/d9c/f124 0 2021-11-03T17:14:55.287 INFO:tasks.workunit.client.0.smithi027.stdout:9/783: dread db/d1b/f45 [0,4194304] 0 2021-11-03T17:14:55.288 INFO:tasks.workunit.client.0.smithi027.stdout:9/784: chown db/fa8 1 1 2021-11-03T17:14:55.308 INFO:tasks.workunit.client.0.smithi027.stdout:9/785: dwrite db/d1b/d2b/d5e/daa/f109 [0,4194304] 0 2021-11-03T17:14:55.313 INFO:tasks.workunit.client.0.smithi027.stdout:9/786: symlink db/d1b/d23/d2c/l125 0 2021-11-03T17:14:55.314 INFO:tasks.workunit.client.0.smithi027.stdout:9/787: creat db/d1b/d1c/d10f/f126 x:0 0 0 2021-11-03T17:14:55.322 INFO:tasks.workunit.client.0.smithi027.stdout:9/788: write db/d1b/d8c/dcd/ffe [860859,19753] 0 2021-11-03T17:14:55.327 INFO:tasks.workunit.client.0.smithi027.stdout:9/789: write db/d1b/d1c/d1d/d40/dcc/fdf [506122,79872] 0 2021-11-03T17:14:55.328 INFO:tasks.workunit.client.0.smithi027.stdout:9/790: creat db/d1b/dfa/f127 x:0 0 0 2021-11-03T17:14:55.331 INFO:tasks.workunit.client.0.smithi027.stdout:9/791: link db/d1b/d2b/d41/d95/fcf db/d1b/d8c/f128 0 2021-11-03T17:14:55.332 INFO:tasks.workunit.client.0.smithi027.stdout:9/792: symlink db/d1b/d2b/l129 0 2021-11-03T17:14:55.334 INFO:tasks.workunit.client.0.smithi027.stdout:9/793: mkdir db/d1b/d2b/d41/dad/dbb/d12a 0 2021-11-03T17:14:55.336 INFO:tasks.workunit.client.0.smithi027.stdout:9/794: unlink db/d1b/d2b/d41/l11c 0 2021-11-03T17:14:55.345 INFO:tasks.workunit.client.0.smithi027.stdout:9/795: dread db/fd [0,4194304] 0 2021-11-03T17:14:55.347 INFO:tasks.workunit.client.0.smithi027.stdout:9/796: mkdir db/d1b/d106/d12b 0 2021-11-03T17:14:55.347 INFO:tasks.workunit.client.0.smithi027.stdout:9/797: fsync db/d1b/d2b/fd8 0 2021-11-03T17:14:55.350 INFO:tasks.workunit.client.0.smithi027.stdout:9/798: read db/d1b/d1c/d1d/f49 [2814222,75073] 0 2021-11-03T17:14:55.359 INFO:tasks.workunit.client.0.smithi027.stdout:9/799: write db/d1b/d2b/d41/d95/fcf [1008284,98607] 0 2021-11-03T17:14:55.361 INFO:tasks.workunit.client.0.smithi027.stdout:9/800: getdents db/d1b/d1c/d1d 0 2021-11-03T17:14:55.362 INFO:tasks.workunit.client.0.smithi027.stdout:9/801: readlink db/d1b/d2b/d41/lfd 0 2021-11-03T17:14:55.363 INFO:tasks.workunit.client.0.smithi027.stdout:9/802: truncate db/d1b/d23/f123 1403025 0 2021-11-03T17:14:55.364 INFO:tasks.workunit.client.0.smithi027.stdout:9/803: readlink db/d1b/d2b/d5e/la0 0 2021-11-03T17:14:55.365 INFO:tasks.workunit.client.0.smithi027.stdout:9/804: fsync db/d1b/d2b/f108 0 2021-11-03T17:14:55.366 INFO:tasks.workunit.client.0.smithi027.stdout:9/805: readlink db/d1b/d2b/l7f 0 2021-11-03T17:14:55.374 INFO:tasks.workunit.client.0.smithi027.stdout:9/806: write db/d1b/d1c/d1d/d40/d60/fc6 [705560,98127] 0 2021-11-03T17:14:55.375 INFO:tasks.workunit.client.0.smithi027.stdout:9/807: readlink db/lc4 0 2021-11-03T17:14:55.376 INFO:tasks.workunit.client.0.smithi027.stdout:9/808: mknod db/c12c 1 2021-11-03T17:14:55.377 INFO:tasks.workunit.client.0.smithi027.stdout:9/809: mkdir db/d1b/d1c/d1d/d12d 0 2021-11-03T17:14:55.377 INFO:tasks.workunit.client.0.smithi027.stdout:9/810: creat db/d1b/d106/f12e x:0 0 0 2021-11-03T17:14:55.379 INFO:tasks.workunit.client.0.smithi027.stdout:9/811: mkdir db/d1b/d2b/d41/d95/d9f/d12f 0 2021-11-03T17:14:55.380 INFO:tasks.workunit.client.0.smithi027.stdout:9/812: dread - db/f115 zero size 2021-11-03T17:14:55.383 INFO:tasks.workunit.client.0.smithi027.stdout:9/813: rename db/d1b/d1c/d1d/f8f to db/d1b/d106/d12b/f130 0 2021-11-03T17:14:55.421 INFO:tasks.workunit.client.0.smithi027.stdout:9/814: dwrite db/d1b/d1c/d1d/d40/f53 [0,4194304] 0 2021-11-03T17:14:55.423 INFO:tasks.workunit.client.0.smithi027.stdout:9/815: stat db/d1b/d2b/d5e/daa/fbf 0 2021-11-03T17:14:55.424 INFO:tasks.workunit.client.0.smithi027.stdout:9/816: fsync db/d1b/d2b/d41/d95/f117 0 2021-11-03T17:14:55.424 INFO:tasks.workunit.client.0.smithi027.stdout:9/817: readlink db/d1b/l61 0 2021-11-03T17:14:55.425 INFO:tasks.workunit.client.0.smithi027.stdout:9/818: dread - db/f115 zero size 2021-11-03T17:14:55.425 INFO:tasks.workunit.client.0.smithi027.stdout:9/819: truncate db/d1b/d1c/d10f/f126 45944 0 2021-11-03T17:14:55.430 INFO:tasks.workunit.client.0.smithi027.stdout:9/820: write db/fe8 [954362,42451] 0 2021-11-03T17:14:55.431 INFO:tasks.workunit.client.0.smithi027.stdout:9/821: write db/d1b/d2b/dc7/fe5 [818247,123977] 0 2021-11-03T17:14:55.432 INFO:tasks.workunit.client.0.smithi027.stdout:9/822: dread db/d1b/d23/f123 [0,4194304] 0 2021-11-03T17:14:55.436 INFO:tasks.workunit.client.0.smithi027.stdout:9/823: link db/d1b/d2b/d41/d6c/ff7 db/d107/f131 0 2021-11-03T17:14:55.436 INFO:tasks.workunit.client.0.smithi027.stdout:9/824: getdents db/d1b/d2b/d41/dad/dbb/d11b 0 2021-11-03T17:14:55.437 INFO:tasks.workunit.client.0.smithi027.stdout:9/825: symlink db/d1b/d2b/d41/d6c/l132 0 2021-11-03T17:14:55.438 INFO:tasks.workunit.client.0.smithi027.stdout:9/826: creat db/f133 x:0 0 0 2021-11-03T17:14:55.441 INFO:tasks.workunit.client.0.smithi027.stdout:9/827: rename db/d1b/d106/d12b/f130 to db/d1b/d1c/f134 0 2021-11-03T17:14:55.441 INFO:tasks.workunit.client.0.smithi027.stdout:9/828: stat db/d1b/d2b/d5e/daa/fbf 0 2021-11-03T17:14:55.448 INFO:tasks.workunit.client.0.smithi027.stdout:9/829: write db/d1b/f51 [5202321,31298] 0 2021-11-03T17:14:55.450 INFO:tasks.workunit.client.0.smithi027.stdout:9/830: mkdir db/d1b/d135 0 2021-11-03T17:14:55.452 INFO:tasks.workunit.client.0.smithi027.stdout:9/831: unlink db/d1b/d2b/l7f 0 2021-11-03T17:14:55.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:55 smithi027 conmon[32322]: cluster 2021-11-03T17:14:53.899612+0000 mgr.smithi115.tcyodz 2021-11-03T17:14:55.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:55 smithi027 conmon[32322]: (mgr.24395) 33 : cluster [DBG] pgmap v22: 65 pgs: 65 active+clean; 3.0 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 14 MiB/s rd, 21 MiB/s wr, 53 op/s 2021-11-03T17:14:55.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:55 smithi027 conmon[32322]: cluster 2021-11-03T17:14:54.610585+0000 mon.smithi027 (mon.0) 638 : cluster [DBG] Standby manager daemon smithi027.ufhhvb restarted 2021-11-03T17:14:55.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:55 smithi027 conmon[32322]: cluster 2021-11-03T17:14:54.610704+0000 mon.smithi027 (mon.0) 639 : cluster [DBG] Standby manager daemon smithi027.ufhhvb started 2021-11-03T17:14:55.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:55 smithi027 conmon[32322]: audit 2021-11-03T17:14:54.613434+0000 mon.smithi027 (mon.0) 640 : audit [DBG] 2021-11-03T17:14:55.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:55 smithi027 conmon[32322]: from='mgr.? 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi027.ufhhvb/crt"}]: dispatch 2021-11-03T17:14:55.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:55 smithi027 conmon[32322]: audit 2021-11-03T17:14:54.614021+0000 mon.smithi027 2021-11-03T17:14:55.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:55 smithi027 conmon[32322]: (mon.0) 641 : audit [DBG] from='mgr.? 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-11-03T17:14:55.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:55 smithi027 conmon[32322]: audit 2021-11- 2021-11-03T17:14:55.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:55 smithi027 conmon[32322]: 03T17:14:54.616793+0000 mon.smithi027 (mon 2021-11-03T17:14:55.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:55 smithi027 conmon[32322]: .0) 642 : audit [DBG] from='mgr.? 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi027.ufhhvb/key"}]: dispatch 2021-11-03T17:14:55.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:55 smithi027 conmon[32322]: audit 2021 2021-11-03T17:14:55.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:55 smithi027 conmon[32322]: -11-03T17:14:54.617804+0000 2021-11-03T17:14:55.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:55 smithi027 conmon[32322]: mon.smithi027 (mon.0) 643 : audit [DBG] from='mgr.? 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-11-03T17:14:55.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:55 smithi027 conmon[32322]: 2021-11-03T17:14:55.564 INFO:tasks.workunit.client.0.smithi027.stdout:6/736: sync 2021-11-03T17:14:55.564 INFO:tasks.workunit.client.0.smithi027.stdout:4/800: sync 2021-11-03T17:14:55.566 INFO:tasks.workunit.client.0.smithi027.stdout:4/801: creat d6/d24/de3/d102/f109 x:0 0 0 2021-11-03T17:14:55.567 INFO:tasks.workunit.client.0.smithi027.stdout:4/802: write d6/d9/d8f/fca [4522254,86623] 0 2021-11-03T17:14:55.568 INFO:tasks.workunit.client.0.smithi027.stdout:6/737: symlink dc/d82/d57/le2 0 2021-11-03T17:14:55.574 INFO:tasks.workunit.client.0.smithi027.stdout:4/803: mkdir d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d41/d10a 0 2021-11-03T17:14:55.575 INFO:tasks.workunit.client.0.smithi027.stdout:6/738: symlink dc/d16/d9a/le3 0 2021-11-03T17:14:55.583 INFO:tasks.workunit.client.0.smithi027.stdout:4/804: dread d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/fc2 [0,4194304] 0 2021-11-03T17:14:55.586 INFO:tasks.workunit.client.0.smithi027.stdout:6/739: rename dc/d16/d1b/l2e to dc/le4 0 2021-11-03T17:14:55.601 INFO:tasks.workunit.client.0.smithi027.stdout:4/805: dwrite d6/d9/d16/d36/d57/dbb/ffa [0,4194304] 0 2021-11-03T17:14:55.602 INFO:tasks.workunit.client.0.smithi027.stdout:4/806: read - d6/d9/f105 zero size 2021-11-03T17:14:55.604 INFO:tasks.workunit.client.0.smithi027.stdout:6/740: write dc/d16/d1b/d24/f85 [3117276,129309] 0 2021-11-03T17:14:55.606 INFO:tasks.workunit.client.0.smithi027.stdout:4/807: symlink d6/d9/d16/dc3/l10b 0 2021-11-03T17:14:55.608 INFO:tasks.workunit.client.0.smithi027.stdout:6/741: getdents dc/d37 0 2021-11-03T17:14:55.611 INFO:tasks.workunit.client.0.smithi027.stdout:6/742: read dc/d82/d57/f91 [4296352,128242] 0 2021-11-03T17:14:55.613 INFO:tasks.workunit.client.0.smithi027.stdout:6/743: truncate dc/d16/d1b/d3c/f6c 6263060 0 2021-11-03T17:14:55.617 INFO:tasks.workunit.client.0.smithi027.stdout:6/744: rmdir dc/d82/d57/d9b/dc8 39 2021-11-03T17:14:55.619 INFO:tasks.workunit.client.0.smithi027.stdout:6/745: creat dc/d16/d1b/d3c/d51/fe5 x:0 0 0 2021-11-03T17:14:55.622 INFO:tasks.workunit.client.0.smithi027.stdout:6/746: creat dc/d82/d57/dad/dae/db1/d34/fe6 x:0 0 0 2021-11-03T17:14:55.625 INFO:tasks.workunit.client.0.smithi027.stdout:6/747: mkdir dc/d82/d57/d9b/db0/de7 0 2021-11-03T17:14:55.626 INFO:tasks.workunit.client.0.smithi027.stdout:6/748: truncate dc/d16/f7a 2624702 0 2021-11-03T17:14:55.631 INFO:tasks.workunit.client.0.smithi027.stdout:4/808: dwrite d6/d9/d16/d36/d70/d3e/d58/d84/d9b/fcb [0,4194304] 0 2021-11-03T17:14:55.636 INFO:tasks.workunit.client.0.smithi027.stdout:4/809: truncate d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/f7e 1352798 0 2021-11-03T17:14:55.639 INFO:tasks.workunit.client.0.smithi027.stdout:4/810: write f4 [1535694,1668] 0 2021-11-03T17:14:55.641 INFO:tasks.workunit.client.0.smithi027.stdout:4/811: creat d6/d9/d16/d36/d57/d5a/d63/f10c x:0 0 0 2021-11-03T17:14:55.643 INFO:tasks.workunit.client.0.smithi027.stdout:4/812: creat d6/d9/d16/d36/d99/f10d x:0 0 0 2021-11-03T17:14:55.649 INFO:tasks.workunit.client.0.smithi027.stdout:6/749: dwrite dc/d82/f81 [0,4194304] 0 2021-11-03T17:14:55.650 INFO:tasks.workunit.client.0.smithi027.stdout:6/750: dread - dc/d82/d57/fc5 zero size 2021-11-03T17:14:55.654 INFO:tasks.workunit.client.0.smithi027.stdout:4/813: write d6/d9/d16/d36/dc1/ff9 [4913168,6606] 0 2021-11-03T17:14:55.655 INFO:tasks.workunit.client.0.smithi027.stdout:4/814: mknod d6/d24/de3/d9f/c10e 1 2021-11-03T17:14:55.656 INFO:tasks.workunit.client.0.smithi027.stdout:4/815: chown d6/d24/db0/f76 1154390 1 2021-11-03T17:14:55.656 INFO:tasks.workunit.client.0.smithi027.stdout:4/816: mknod d6/d9/d16/d17/c10f 1 2021-11-03T17:14:55.658 INFO:tasks.workunit.client.0.smithi027.stdout:4/817: write d6/d9/d16/d36/d57/dbb/ffa [2111083,63196] 0 2021-11-03T17:14:55.660 INFO:tasks.workunit.client.0.smithi027.stdout:4/818: mkdir d6/d9/d16/d36/dc1/d110 0 2021-11-03T17:14:55.661 INFO:tasks.workunit.client.0.smithi027.stdout:4/819: rmdir d6/d9/d16/d36 39 2021-11-03T17:14:55.663 INFO:tasks.workunit.client.0.smithi027.stdout:4/820: symlink d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d9a/l111 0 2021-11-03T17:14:55.664 INFO:tasks.workunit.client.0.smithi027.stdout:4/821: creat d6/d9/d16/d36/d99/f112 x:0 0 0 2021-11-03T17:14:55.664 INFO:tasks.workunit.client.0.smithi027.stdout:4/822: write d6/d24/de3/d102/f109 [1009560,130917] 0 2021-11-03T17:14:55.665 INFO:tasks.workunit.client.0.smithi027.stdout:4/823: chown d6/d9/d16/d36/d57/d5a/d63/l90 1 1 2021-11-03T17:14:55.672 INFO:tasks.workunit.client.0.smithi027.stdout:6/751: dwrite dc/d16/d1b/d3c/f44 [8388608,4194304] 0 2021-11-03T17:14:55.681 INFO:tasks.workunit.client.0.smithi027.stdout:4/824: dread d6/d9/d8f/fca [0,4194304] 0 2021-11-03T17:14:55.683 INFO:tasks.workunit.client.0.smithi027.stdout:4/825: symlink d6/d24/db0/l113 0 2021-11-03T17:14:55.685 INFO:tasks.workunit.client.0.smithi027.stdout:4/826: rename d6/d9/d16/d36/d70/d3e/l65 to d6/d9/d16/d36/dc1/d110/l114 0 2021-11-03T17:14:55.697 INFO:tasks.workunit.client.0.smithi027.stdout:6/752: dwrite dc/d16/f2c [0,4194304] 0 2021-11-03T17:14:55.700 INFO:tasks.workunit.client.0.smithi027.stdout:6/753: mkdir dc/d16/d1b/d24/d7e/db3/de8 0 2021-11-03T17:14:55.701 INFO:tasks.workunit.client.0.smithi027.stdout:6/754: getdents dc/da1 0 2021-11-03T17:14:55.704 INFO:tasks.workunit.client.0.smithi027.stdout:6/755: rename dc/d16/d1b/d3c/d51/dd8/fd9 to dc/d16/d1b/d3c/d51/fe9 0 2021-11-03T17:14:55.705 INFO:tasks.workunit.client.0.smithi027.stdout:4/827: dwrite d6/d9/d16/d17/f6e [0,4194304] 0 2021-11-03T17:14:55.706 INFO:tasks.workunit.client.0.smithi027.stdout:6/756: getdents dc/d16/d62 0 2021-11-03T17:14:55.708 INFO:tasks.workunit.client.0.smithi027.stdout:6/757: creat dc/d37/dcc/fea x:0 0 0 2021-11-03T17:14:55.711 INFO:tasks.workunit.client.0.smithi027.stdout:4/828: symlink d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d41/l115 0 2021-11-03T17:14:55.711 INFO:tasks.workunit.client.0.smithi027.stdout:4/829: read - d6/d9/d16/d36/d57/d5a/d63/ff6 zero size 2021-11-03T17:14:55.721 INFO:tasks.workunit.client.0.smithi027.stdout:4/830: dread f4 [0,4194304] 0 2021-11-03T17:14:55.725 INFO:tasks.workunit.client.0.smithi027.stdout:4/831: rename d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/le8 to d6/d9/d16/d36/d70/d3e/d58/d84/l116 0 2021-11-03T17:14:55.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:55 smithi115 conmon[34670]: cluster 2021-11-03T17:14:53.899612+0000 mgr.smithi115.tcyodz (mgr.24395) 33 : 2021-11-03T17:14:55.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:55 smithi115 conmon[34670]: cluster [DBG] pgmap v22: 65 pgs: 65 active+clean; 3.0 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 14 MiB/s rd, 21 MiB/s wr, 53 op/s 2021-11-03T17:14:55.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:55 smithi115 conmon[34670]: cluster 2021-11-03T17:14:54.610585+0000 mon.smithi027 (mon.0) 638 : cluster [DBG] Standby manager daemon smithi027.ufhhvb restarted 2021-11-03T17:14:55.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:55 smithi115 conmon[34670]: cluster 2021- 2021-11-03T17:14:55.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:55 smithi115 conmon[34670]: 11-03T17:14:54.610704+0000 mon.smithi027 (mon.0) 639 : cluster [DBG] Standby manager daemon smithi027.ufhhvb started 2021-11-03T17:14:55.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:55 smithi115 conmon[34670]: audit 2021-11- 2021-11-03T17:14:55.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:55 smithi115 conmon[34670]: 03T17:14:54.613434+0000 mon.smithi027 (mon 2021-11-03T17:14:55.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:55 smithi115 conmon[34670]: .0) 640 : audit [DBG] from='mgr.? 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi027.ufhhvb/crt"}]: dispatch 2021-11-03T17:14:55.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:55 smithi115 conmon[34670]: audit 2021-11-03T17:14:55.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:55 smithi115 conmon[34670]: 2021-11-03T17:14:54.614021+0000 mon.smithi027 2021-11-03T17:14:55.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:55 smithi115 conmon[34670]: (mon.0) 641 : audit [DBG] from='mgr.? 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-11-03T17:14:55.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:55 smithi115 conmon[34670]: audit 2021-11-03T 2021-11-03T17:14:55.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:55 smithi115 conmon[34670]: 17:14:54.616793+0000 mon.smithi027 (mon.0) 642 : audit 2021-11-03T17:14:55.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:55 smithi115 conmon[34670]: [DBG] from='mgr.? 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi027.ufhhvb/key"}]: dispatch 2021-11-03T17:14:55.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:55 smithi115 conmon[34670]: audit 2021-11-03T17: 2021-11-03T17:14:55.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:55 smithi115 conmon[34670]: 14:54.617804+0000 mon.smithi027 (mon.0) 2021-11-03T17:14:55.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:55 smithi115 conmon[34670]: 643 : audit [DBG] from='mgr.? 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-11-03T17:14:55.738 INFO:tasks.workunit.client.0.smithi027.stdout:6/758: dwrite dc/d16/d30/f61 [4194304,4194304] 0 2021-11-03T17:14:55.744 INFO:tasks.workunit.client.0.smithi027.stdout:6/759: unlink dc/d82/d57/dad/dae/db1/d34/d89/la7 0 2021-11-03T17:14:55.753 INFO:tasks.workunit.client.0.smithi027.stdout:6/760: dread dc/d16/d1b/d3c/f44 [4194304,4194304] 0 2021-11-03T17:14:55.754 INFO:tasks.workunit.client.0.smithi027.stdout:6/761: fsync dc/d16/d1b/d3c/d51/fe5 0 2021-11-03T17:14:55.755 INFO:tasks.workunit.client.0.smithi027.stdout:4/832: dwrite d6/d9/d16/d36/d57/fce [0,4194304] 0 2021-11-03T17:14:55.760 INFO:tasks.workunit.client.0.smithi027.stdout:4/833: dread d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/f2c [4194304,4194304] 0 2021-11-03T17:14:55.761 INFO:tasks.workunit.client.0.smithi027.stdout:4/834: read - d6/d9/d16/d36/d57/d5a/d63/fd9 zero size 2021-11-03T17:14:55.762 INFO:tasks.workunit.client.0.smithi027.stdout:4/835: mknod d6/d24/de3/d9f/c117 1 2021-11-03T17:14:55.765 INFO:tasks.workunit.client.0.smithi027.stdout:6/762: dread dc/f14 [0,4194304] 0 2021-11-03T17:14:55.783 INFO:tasks.workunit.client.0.smithi027.stdout:6/763: dwrite dc/d82/d57/f97 [0,4194304] 0 2021-11-03T17:14:55.799 INFO:tasks.workunit.client.0.smithi027.stdout:6/764: dwrite dc/d16/d30/f5a [0,4194304] 0 2021-11-03T17:14:55.805 INFO:tasks.workunit.client.0.smithi027.stdout:6/765: truncate dc/d82/d57/f97 2732314 0 2021-11-03T17:14:55.810 INFO:tasks.workunit.client.0.smithi027.stdout:6/766: write dc/d16/d1b/d3c/d51/f58 [653763,69247] 0 2021-11-03T17:14:55.810 INFO:tasks.workunit.client.0.smithi027.stdout:6/767: chown dc/d37/fe1 32199563 1 2021-11-03T17:14:55.810 INFO:tasks.workunit.client.0.smithi027.stdout:6/768: write dc/d16/d30/f5a [2753559,20090] 0 2021-11-03T17:14:55.813 INFO:tasks.workunit.client.0.smithi027.stdout:6/769: creat dc/d16/db2/feb x:0 0 0 2021-11-03T17:14:55.813 INFO:tasks.workunit.client.0.smithi027.stdout:6/770: getdents dc/d16/d1b/d24 0 2021-11-03T17:14:55.890 INFO:tasks.workunit.client.0.smithi027.stdout:6/771: fsync dc/d16/fc2 0 2021-11-03T17:14:55.893 INFO:tasks.workunit.client.0.smithi027.stdout:6/772: mkdir dc/d82/d87/dec 0 2021-11-03T17:14:55.895 INFO:tasks.workunit.client.0.smithi027.stdout:6/773: truncate dc/d82/d57/fc3 782936 0 2021-11-03T17:14:55.896 INFO:tasks.workunit.client.0.smithi027.stdout:6/774: dread dc/d82/d87/fb9 [0,4194304] 0 2021-11-03T17:14:55.912 INFO:tasks.workunit.client.0.smithi027.stdout:6/775: dwrite dc/d16/fc2 [0,4194304] 0 2021-11-03T17:14:55.917 INFO:tasks.workunit.client.0.smithi027.stdout:6/776: getdents dc/d82/d87 0 2021-11-03T17:14:55.934 INFO:tasks.workunit.client.0.smithi027.stdout:6/777: dwrite dc/d16/d1b/d3c/d51/f5b [0,4194304] 0 2021-11-03T17:14:55.937 INFO:tasks.workunit.client.0.smithi027.stdout:6/778: mknod dc/d16/d1b/ced 1 2021-11-03T17:14:55.937 INFO:tasks.workunit.client.0.smithi027.stdout:6/779: getdents dc/d82/d57/d9b/db0/de7 0 2021-11-03T17:14:55.939 INFO:tasks.workunit.client.0.smithi027.stdout:6/780: fdatasync dc/f7b 0 2021-11-03T17:14:55.943 INFO:tasks.workunit.client.0.smithi027.stdout:6/781: symlink dc/d16/d1b/d3c/d51/lee 0 2021-11-03T17:14:55.963 INFO:tasks.workunit.client.0.smithi027.stdout:6/782: dwrite dc/d82/d57/f91 [0,4194304] 0 2021-11-03T17:14:55.970 INFO:tasks.workunit.client.0.smithi027.stdout:6/783: dread dc/d16/d1b/f28 [0,4194304] 0 2021-11-03T17:14:55.971 INFO:tasks.workunit.client.0.smithi027.stdout:6/784: creat dc/d82/d57/dad/fef x:0 0 0 2021-11-03T17:14:55.973 INFO:tasks.workunit.client.0.smithi027.stdout:6/785: getdents dc/d82/d57/d9b/dc8 0 2021-11-03T17:14:55.974 INFO:tasks.workunit.client.0.smithi027.stdout:6/786: truncate dc/d16/d1b/f28 847412 0 2021-11-03T17:14:55.978 INFO:tasks.workunit.client.0.smithi027.stdout:6/787: rmdir dc/d16/d1b/d3c/d51/dd8 0 2021-11-03T17:14:55.980 INFO:tasks.workunit.client.0.smithi027.stdout:6/788: creat dc/d16/d1b/d3c/d51/ff0 x:0 0 0 2021-11-03T17:14:55.990 INFO:tasks.workunit.client.0.smithi027.stdout:6/789: dread dc/d16/f7a [0,4194304] 0 2021-11-03T17:14:55.990 INFO:tasks.workunit.client.0.smithi027.stdout:6/790: stat dc/d16/d62 0 2021-11-03T17:14:55.998 INFO:tasks.workunit.client.0.smithi027.stdout:1/959: sync 2021-11-03T17:14:56.001 INFO:tasks.workunit.client.0.smithi027.stdout:1/960: mkdir d6/d11/dac/d79/dab/db4/d13b 0 2021-11-03T17:14:56.001 INFO:tasks.workunit.client.0.smithi027.stdout:1/961: read - d6/d11/d14/f12a zero size 2021-11-03T17:14:56.004 INFO:tasks.workunit.client.0.smithi027.stdout:6/791: dwrite dc/d16/d1b/d3c/d51/fe9 [0,4194304] 0 2021-11-03T17:14:56.009 INFO:tasks.workunit.client.0.smithi027.stdout:6/792: symlink dc/d16/d30/lf1 0 2021-11-03T17:14:56.014 INFO:tasks.workunit.client.0.smithi027.stdout:6/793: write dc/f14 [3679982,54116] 0 2021-11-03T17:14:56.016 INFO:tasks.workunit.client.0.smithi027.stdout:6/794: creat dc/d16/d1b/dda/ff2 x:0 0 0 2021-11-03T17:14:56.016 INFO:tasks.workunit.client.0.smithi027.stdout:6/795: chown dc/d16/d1b/d3c/d51/fe5 312240 1 2021-11-03T17:14:56.020 INFO:tasks.workunit.client.0.smithi027.stdout:6/796: rename dc/d16/d9a/fba to dc/d16/d1b/ff3 0 2021-11-03T17:14:56.021 INFO:tasks.workunit.client.0.smithi027.stdout:6/797: link dc/d16/d9a/le3 dc/d16/d62/lf4 0 2021-11-03T17:14:56.023 INFO:tasks.workunit.client.0.smithi027.stdout:6/798: mkdir dc/d82/df5 0 2021-11-03T17:14:56.024 INFO:tasks.workunit.client.0.smithi027.stdout:6/799: fsync dc/d82/d57/dad/dae/fd7 0 2021-11-03T17:14:56.029 INFO:tasks.workunit.client.0.smithi027.stdout:6/800: dread dc/d82/d57/f97 [0,4194304] 0 2021-11-03T17:14:56.030 INFO:tasks.workunit.client.0.smithi027.stdout:6/801: symlink dc/d16/d30/lf6 0 2021-11-03T17:14:56.031 INFO:tasks.workunit.client.0.smithi027.stdout:6/802: mknod dc/d82/d57/dad/dae/db1/d34/d89/cf7 1 2021-11-03T17:14:56.031 INFO:tasks.workunit.client.0.smithi027.stdout:6/803: readlink dc/d16/d1b/l42 0 2021-11-03T17:14:56.032 INFO:tasks.workunit.client.0.smithi027.stdout:6/804: chown dc/d16/d1b/dda/ff2 55039 1 2021-11-03T17:14:56.032 INFO:tasks.workunit.client.0.smithi027.stdout:6/805: mknod dc/d16/d1b/d24/cf8 1 2021-11-03T17:14:56.033 INFO:tasks.workunit.client.0.smithi027.stdout:6/806: read dc/d16/d1b/d24/f2a [2034128,79528] 0 2021-11-03T17:14:56.035 INFO:tasks.workunit.client.0.smithi027.stdout:6/807: truncate dc/d16/db2/fc6 198795 0 2021-11-03T17:14:56.038 INFO:tasks.workunit.client.0.smithi027.stdout:6/808: rename dc/d82/d57/d9b/db0 to dc/d16/d1b/d3c/d51/df9 0 2021-11-03T17:14:56.074 INFO:tasks.workunit.client.0.smithi027.stdout:1/962: dwrite d6/de/faf [0,4194304] 0 2021-11-03T17:14:56.074 INFO:tasks.workunit.client.0.smithi027.stdout:1/963: chown d6/d11/dac/d79/d37/de0/d4e 25 1 2021-11-03T17:14:56.079 INFO:tasks.workunit.client.0.smithi027.stdout:1/964: rename d6/d11/d14/df6/lf8 to d6/d11/dac/d79/db0/d102/l13c 0 2021-11-03T17:14:56.080 INFO:tasks.workunit.client.0.smithi027.stdout:6/809: dwrite dc/f64 [0,4194304] 0 2021-11-03T17:14:56.159 INFO:tasks.workunit.client.0.smithi027.stdout:1/965: link d6/d11/dac/d79/d25/d28/dec/fd2 d6/d11/dac/f13d 0 2021-11-03T17:14:56.161 INFO:tasks.workunit.client.0.smithi027.stdout:1/966: creat d6/d11/dac/d79/dab/db4/f13e x:0 0 0 2021-11-03T17:14:56.162 INFO:tasks.workunit.client.0.smithi027.stdout:6/810: symlink dc/d16/d1b/d24/lfa 0 2021-11-03T17:14:56.162 INFO:tasks.workunit.client.0.smithi027.stdout:1/967: readlink d6/d11/dac/d79/db0/d102/l103 0 2021-11-03T17:14:56.213 INFO:tasks.workunit.client.0.smithi027.stdout:6/811: rename dc/d37/fe1 to dc/ffb 0 2021-11-03T17:14:56.213 INFO:tasks.workunit.client.0.smithi027.stdout:6/812: chown dc/d82/d57/dad/dae/db1/l41 0 1 2021-11-03T17:14:56.222 INFO:tasks.workunit.client.0.smithi027.stdout:6/813: dread - dc/d16/d1b/d3c/d51/fb8 zero size 2021-11-03T17:14:56.222 INFO:tasks.workunit.client.0.smithi027.stdout:1/968: write d6/d11/dac/d79/d37/de0/f42 [3221231,50026] 0 2021-11-03T17:14:56.227 INFO:tasks.workunit.client.0.smithi027.stdout:6/814: rename dc/d16/d1b/d3c/d51/f5b to dc/d16/db2/ffc 0 2021-11-03T17:14:56.229 INFO:tasks.workunit.client.0.smithi027.stdout:6/815: symlink dc/d37/lfd 0 2021-11-03T17:14:56.231 INFO:tasks.workunit.client.0.smithi027.stdout:6/816: creat dc/d82/d57/dad/dae/db1/d34/ffe x:0 0 0 2021-11-03T17:14:56.240 INFO:tasks.workunit.client.0.smithi027.stdout:1/969: dwrite d6/d11/dac/d79/d25/d5e/dd1/f130 [0,4194304] 0 2021-11-03T17:14:56.252 INFO:tasks.workunit.client.0.smithi027.stdout:1/970: dread d6/d11/dac/f82 [0,4194304] 0 2021-11-03T17:14:56.515 INFO:tasks.workunit.client.0.smithi027.stdout:2/940: sync 2021-11-03T17:14:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:56 smithi027 conmon[32322]: cluster 2021-11-03T17:14:55 2021-11-03T17:14:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:56 smithi027 conmon[32322]: .323392+0000 mon.smithi027 (mon.0) 644 : cluster [DBG] mgrmap e24: smithi115.tcyodz(active, since 32s), standbys: smithi027.ufhhvb 2021-11-03T17:14:56.519 INFO:tasks.workunit.client.0.smithi027.stdout:2/941: unlink d0/d1b/d29/d54/d64/daa/l4d 0 2021-11-03T17:14:56.520 INFO:tasks.workunit.client.0.smithi027.stdout:2/942: creat d0/d7c/df1/f122 x:0 0 0 2021-11-03T17:14:56.521 INFO:tasks.workunit.client.0.smithi027.stdout:2/943: rmdir d0/d45 39 2021-11-03T17:14:56.521 INFO:tasks.workunit.client.0.smithi027.stdout:2/944: rmdir d0/d5/d2c 39 2021-11-03T17:14:56.523 INFO:tasks.workunit.client.0.smithi027.stdout:2/945: dread d0/d1b/d29/d92/dbd/f8b [0,4194304] 0 2021-11-03T17:14:56.532 INFO:tasks.workunit.client.0.smithi027.stdout:2/946: dread d0/d5/fac [0,4194304] 0 2021-11-03T17:14:56.533 INFO:tasks.workunit.client.0.smithi027.stdout:2/947: fdatasync d0/d1b/d29/d92/dbd/fa6 0 2021-11-03T17:14:56.533 INFO:tasks.workunit.client.0.smithi027.stdout:2/948: fdatasync d0/d10c/f113 0 2021-11-03T17:14:56.546 INFO:tasks.workunit.client.0.smithi027.stdout:2/949: dwrite d0/da7/f11e [0,4194304] 0 2021-11-03T17:14:56.547 INFO:tasks.workunit.client.0.smithi027.stdout:2/950: chown d0/d1b/d29/d54/d64/daa/d5d/l10e 329587131 1 2021-11-03T17:14:56.551 INFO:tasks.workunit.client.0.smithi027.stdout:2/951: truncate d0/d1b/d29/fcb 932426 0 2021-11-03T17:14:56.552 INFO:tasks.workunit.client.0.smithi027.stdout:2/952: mkdir d0/d7c/d101/d123 0 2021-11-03T17:14:56.554 INFO:tasks.workunit.client.0.smithi027.stdout:2/953: creat d0/d1b/f124 x:0 0 0 2021-11-03T17:14:56.555 INFO:tasks.workunit.client.0.smithi027.stdout:2/954: write d0/d5/d2c/f3d [816496,54430] 0 2021-11-03T17:14:56.556 INFO:tasks.workunit.client.0.smithi027.stdout:2/955: write d0/d5/faf [2491230,39909] 0 2021-11-03T17:14:56.557 INFO:tasks.workunit.client.0.smithi027.stdout:2/956: getdents d0/d1b/d29/d54/d64/daa/d5d 0 2021-11-03T17:14:56.559 INFO:tasks.workunit.client.0.smithi027.stdout:2/957: unlink d0/d1b/d29/d54/d64/daa/d5d/l10e 0 2021-11-03T17:14:56.559 INFO:tasks.workunit.client.0.smithi027.stdout:2/958: mknod d0/d5/d2c/dc8/c125 1 2021-11-03T17:14:56.560 INFO:tasks.workunit.client.0.smithi027.stdout:2/959: readlink d0/l31 0 2021-11-03T17:14:56.563 INFO:tasks.workunit.client.0.smithi027.stdout:2/960: unlink d0/d1b/d29/dd7/ldd 0 2021-11-03T17:14:56.564 INFO:tasks.workunit.client.0.smithi027.stdout:2/961: symlink d0/d5/d2c/l126 0 2021-11-03T17:14:56.586 INFO:tasks.workunit.client.0.smithi027.stdout:2/962: dwrite d0/d45/fd5 [0,4194304] 0 2021-11-03T17:14:56.589 INFO:tasks.workunit.client.0.smithi027.stdout:2/963: creat d0/d7c/df1/f127 x:0 0 0 2021-11-03T17:14:56.591 INFO:tasks.workunit.client.0.smithi027.stdout:2/964: dread d0/d7c/f9f [0,4194304] 0 2021-11-03T17:14:56.593 INFO:tasks.workunit.client.0.smithi027.stdout:2/965: dread d0/d1b/f27 [0,4194304] 0 2021-11-03T17:14:56.594 INFO:tasks.workunit.client.0.smithi027.stdout:2/966: getdents d0/d1b/d29/d54/df3 0 2021-11-03T17:14:56.595 INFO:tasks.workunit.client.0.smithi027.stdout:2/967: chown d0/d1b 43 1 2021-11-03T17:14:56.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:56 smithi115 conmon[34670]: cluster 2021-11-03T17:14:55.323392+0000 mon.smithi027 (mon.0) 2021-11-03T17:14:56.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:56 smithi115 conmon[34670]: 644 : cluster [DBG] mgrmap e24: smithi115.tcyodz(active, since 32s), standbys: smithi027.ufhhvb 2021-11-03T17:14:57.663 INFO:tasks.workunit.client.0.smithi027.stdout:7/616: sync 2021-11-03T17:14:57.665 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:57 smithi027 conmon[32322]: cluster 2021-11-03T17:14:55.900108+0000 2021-11-03T17:14:57.665 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:57 smithi027 conmon[32322]: mgr.smithi115.tcyodz (mgr.24395) 34 : cluster [DBG] pgmap v23: 65 pgs: 65 active+clean; 3.2 GiB data, 9.3 GiB used, 527 GiB / 536 GiB avail; 24 MiB/s rd, 42 MiB/s wr, 99 op/s 2021-11-03T17:14:57.667 INFO:tasks.workunit.client.0.smithi027.stdout:7/617: fsync d0/d2/d10/d35/f7a 0 2021-11-03T17:14:57.669 INFO:tasks.workunit.client.0.smithi027.stdout:7/618: unlink d0/d2/d10/l2e 0 2021-11-03T17:14:57.670 INFO:tasks.workunit.client.0.smithi027.stdout:7/619: read d0/d2/d1f/fc2 [2734369,98808] 0 2021-11-03T17:14:57.699 INFO:tasks.workunit.client.0.smithi027.stdout:7/620: dwrite d0/d2/d5e/f32 [0,4194304] 0 2021-11-03T17:14:57.711 INFO:tasks.workunit.client.0.smithi027.stdout:7/621: dread d0/d2/d10/d35/d96/fa2 [0,4194304] 0 2021-11-03T17:14:57.717 INFO:tasks.workunit.client.0.smithi027.stdout:7/622: truncate d0/d2/d5e/f32 3959497 0 2021-11-03T17:14:57.719 INFO:tasks.workunit.client.0.smithi027.stdout:7/623: mkdir d0/d2/d1c/d4d/dcc 0 2021-11-03T17:14:57.730 INFO:tasks.workunit.client.0.smithi027.stdout:7/624: dread d0/d2/d5e/d36/fb0 [0,4194304] 0 2021-11-03T17:14:57.731 INFO:tasks.workunit.client.0.smithi027.stdout:7/625: creat d0/d2/d10/d35/d96/fcd x:0 0 0 2021-11-03T17:14:57.732 INFO:tasks.workunit.client.0.smithi027.stdout:7/626: rmdir d0/d2/d1c/d4d/d6e/d84 39 2021-11-03T17:14:57.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:57 smithi115 conmon[34670]: cluster 2021-11-03T17:14:55.900108+0000 mgr.smithi115.tcyodz (mgr.24395) 34 : cluster [DBG] pgmap v23: 65 pgs: 65 active+clean; 3.2 GiB data, 9.3 GiB used, 527 GiB / 536 GiB avail; 24 MiB/s rd, 42 MiB/s wr, 99 op/s 2021-11-03T17:14:57.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:57 smithi115 conmon[34670]: 2021-11-03T17:14:57.740 INFO:tasks.workunit.client.0.smithi027.stdout:7/627: write d0/d2/d10/d1e/d20/f5b [243240,109040] 0 2021-11-03T17:14:57.741 INFO:tasks.workunit.client.0.smithi027.stdout:7/628: symlink d0/d2/d10/lce 0 2021-11-03T17:14:57.742 INFO:tasks.workunit.client.0.smithi027.stdout:7/629: dread - d0/d2/d10/d35/fb5 zero size 2021-11-03T17:14:57.747 INFO:tasks.workunit.client.0.smithi027.stdout:7/630: write d0/d2/d10/d1e/f88 [669806,40623] 0 2021-11-03T17:14:57.748 INFO:tasks.workunit.client.0.smithi027.stdout:7/631: chown d0/d2/d10/l72 415 1 2021-11-03T17:14:57.754 INFO:tasks.workunit.client.0.smithi027.stdout:7/632: write d0/d2/d1c/f61 [1090019,72961] 0 2021-11-03T17:14:57.756 INFO:tasks.workunit.client.0.smithi027.stdout:7/633: creat d0/d2d/d2f/dae/dc8/fcf x:0 0 0 2021-11-03T17:14:57.760 INFO:tasks.workunit.client.0.smithi027.stdout:7/634: dread d0/d2/f5 [0,4194304] 0 2021-11-03T17:14:57.761 INFO:tasks.workunit.client.0.smithi027.stdout:7/635: readlink d0/d2/l81 0 2021-11-03T17:14:57.761 INFO:tasks.workunit.client.0.smithi027.stdout:7/636: dread - d0/d2/d5e/d36/f63 zero size 2021-11-03T17:14:57.853 INFO:tasks.workunit.client.0.smithi027.stdout:9/832: sync 2021-11-03T17:14:57.855 INFO:tasks.workunit.client.0.smithi027.stdout:9/833: fsync db/d1b/d2b/d41/dad/fd3 0 2021-11-03T17:14:57.858 INFO:tasks.workunit.client.0.smithi027.stdout:9/834: creat db/d1b/d2b/d41/d97/f136 x:0 0 0 2021-11-03T17:14:57.879 INFO:tasks.workunit.client.0.smithi027.stdout:9/835: creat db/d1b/d8c/f137 x:0 0 0 2021-11-03T17:14:57.880 INFO:tasks.workunit.client.0.smithi027.stdout:9/836: chown db/d1b/d1c/d10f/f126 11 1 2021-11-03T17:14:57.882 INFO:tasks.workunit.client.0.smithi027.stdout:9/837: dread db/d1b/d23/f4b [0,4194304] 0 2021-11-03T17:14:57.884 INFO:tasks.workunit.client.0.smithi027.stdout:9/838: read db/d1b/d1c/f134 [5784489,81777] 0 2021-11-03T17:14:57.901 INFO:tasks.workunit.client.0.smithi027.stdout:9/839: dwrite db/fa8 [4194304,4194304] 0 2021-11-03T17:14:57.909 INFO:tasks.workunit.client.0.smithi027.stdout:9/840: write db/d1b/d2b/d41/dad/ff3 [824862,114592] 0 2021-11-03T17:14:57.913 INFO:tasks.workunit.client.0.smithi027.stdout:9/841: dread db/f5b [0,4194304] 0 2021-11-03T17:14:57.930 INFO:tasks.workunit.client.0.smithi027.stdout:9/842: dwrite db/d1b/d2b/d41/fb8 [0,4194304] 0 2021-11-03T17:14:57.932 INFO:tasks.workunit.client.0.smithi027.stdout:9/843: unlink db/d1b/d1c/d1d/l56 0 2021-11-03T17:14:57.950 INFO:tasks.workunit.client.0.smithi027.stdout:9/844: dwrite db/d1b/d1c/fd0 [0,4194304] 0 2021-11-03T17:14:57.952 INFO:tasks.workunit.client.0.smithi027.stdout:9/845: mknod db/d1b/d2b/d41/d6c/c138 1 2021-11-03T17:14:57.955 INFO:tasks.workunit.client.0.smithi027.stdout:9/846: link db/d1b/d1c/d1d/l31 db/d1b/d1c/d1d/d40/dcc/l139 0 2021-11-03T17:14:57.959 INFO:tasks.workunit.client.0.smithi027.stdout:9/847: unlink db/d1b/d2b/d41/f74 0 2021-11-03T17:14:57.984 INFO:tasks.workunit.client.0.smithi027.stdout:9/848: dwrite db/f5b [0,4194304] 0 2021-11-03T17:14:57.985 INFO:tasks.workunit.client.0.smithi027.stdout:9/849: creat db/f13a x:0 0 0 2021-11-03T17:14:57.990 INFO:tasks.workunit.client.0.smithi027.stdout:9/850: write db/d1b/d1c/d1d/d40/d60/fa6 [219350,126618] 0 2021-11-03T17:14:57.992 INFO:tasks.workunit.client.0.smithi027.stdout:9/851: mkdir db/d1b/d2b/d41/d95/d9f/d12f/d13b 0 2021-11-03T17:14:58.000 INFO:tasks.workunit.client.0.smithi027.stdout:9/852: dread db/d1b/d23/f7a [0,4194304] 0 2021-11-03T17:14:58.016 INFO:tasks.workunit.client.0.smithi027.stdout:4/836: sync 2021-11-03T17:14:58.017 INFO:tasks.workunit.client.0.smithi027.stdout:9/853: dwrite db/d1b/d2b/d41/d95/fcf [0,4194304] 0 2021-11-03T17:14:58.017 INFO:tasks.workunit.client.0.smithi027.stdout:4/837: mknod d6/d24/d6d/db1/c118 1 2021-11-03T17:14:58.023 INFO:tasks.workunit.client.0.smithi027.stdout:4/838: rename d6/d9/d16/d17 to d6/d24/de3/d102/d119 0 2021-11-03T17:14:58.034 INFO:tasks.workunit.client.0.smithi027.stdout:4/839: write d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d9a/faf [677216,77959] 0 2021-11-03T17:14:58.034 INFO:tasks.workunit.client.0.smithi027.stdout:4/840: chown d6/d24/de3 160575558 1 2021-11-03T17:14:58.036 INFO:tasks.workunit.client.0.smithi027.stdout:9/854: truncate db/d1b/d106/f111 1523413 0 2021-11-03T17:14:58.036 INFO:tasks.workunit.client.0.smithi027.stdout:9/855: readlink db/d1b/d1c/d1d/d40/l11f 0 2021-11-03T17:14:58.038 INFO:tasks.workunit.client.0.smithi027.stdout:9/856: mkdir db/d1b/d106/d13c 0 2021-11-03T17:14:58.046 INFO:tasks.workunit.client.0.smithi027.stdout:9/857: write db/d1b/d1c/f102 [710177,29125] 0 2021-11-03T17:14:58.049 INFO:tasks.workunit.client.0.smithi027.stdout:9/858: link db/d1b/d1c/d10f/f126 db/d1b/d1c/d1d/d12d/f13d 0 2021-11-03T17:14:58.049 INFO:tasks.workunit.client.0.smithi027.stdout:9/859: rmdir db/d1b/d1c 39 2021-11-03T17:14:58.050 INFO:tasks.workunit.client.0.smithi027.stdout:9/860: stat db/d1b/d1c/f25 0 2021-11-03T17:14:58.051 INFO:tasks.workunit.client.0.smithi027.stdout:9/861: readlink db/d1b/d2b/d41/d97/lb1 0 2021-11-03T17:14:58.053 INFO:tasks.workunit.client.0.smithi027.stdout:9/862: dread db/d1b/d2b/d41/dad/ff3 [0,4194304] 0 2021-11-03T17:14:58.054 INFO:tasks.workunit.client.0.smithi027.stdout:9/863: creat db/d1b/d2b/d5e/f13e x:0 0 0 2021-11-03T17:14:58.057 INFO:tasks.workunit.client.0.smithi027.stdout:9/864: rename db/f5b to db/d1b/d2b/d9c/f13f 0 2021-11-03T17:14:58.060 INFO:tasks.workunit.client.0.smithi027.stdout:9/865: rename db/d1b/d2b/d5e/la0 to db/d1b/d1c/l140 0 2021-11-03T17:14:58.061 INFO:tasks.workunit.client.0.smithi027.stdout:9/866: dread - db/d1b/d1c/d1d/fb6 zero size 2021-11-03T17:14:58.061 INFO:tasks.workunit.client.0.smithi027.stdout:9/867: readlink db/d1b/d1c/l79 0 2021-11-03T17:14:58.070 INFO:tasks.workunit.client.0.smithi027.stdout:9/868: write db/d1b/f45 [505386,5179] 0 2021-11-03T17:14:58.071 INFO:tasks.workunit.client.0.smithi027.stdout:9/869: write db/d1b/d1c/d1d/d40/dcc/fdf [545272,113919] 0 2021-11-03T17:14:58.072 INFO:tasks.workunit.client.0.smithi027.stdout:9/870: symlink db/d1b/d2b/l141 0 2021-11-03T17:14:58.074 INFO:tasks.workunit.client.0.smithi027.stdout:9/871: rmdir db/d1b/d135 0 2021-11-03T17:14:58.076 INFO:tasks.workunit.client.0.smithi027.stdout:9/872: rmdir db/d1b/d2b/d5e 39 2021-11-03T17:14:58.089 INFO:tasks.workunit.client.0.smithi027.stdout:9/873: dwrite db/d1b/d106/f112 [0,4194304] 0 2021-11-03T17:14:58.091 INFO:tasks.workunit.client.0.smithi027.stdout:9/874: creat db/d1b/d2b/d41/f142 x:0 0 0 2021-11-03T17:14:58.423 INFO:tasks.workunit.client.0.smithi027.stdout:6/817: sync 2021-11-03T17:14:58.424 INFO:tasks.workunit.client.0.smithi027.stdout:6/818: truncate dc/f7b 2855705 0 2021-11-03T17:14:58.517 INFO:tasks.workunit.client.0.smithi027.stdout:1/971: sync 2021-11-03T17:14:58.521 INFO:tasks.workunit.client.0.smithi027.stdout:1/972: rename d6/d11/dac/d79/db0/f12d to d6/d11/dac/d79/dab/db4/f13f 0 2021-11-03T17:14:58.547 INFO:tasks.workunit.client.0.smithi027.stdout:1/973: dwrite d6/d11/dac/d79/d25/d28/d5c/fc0 [0,4194304] 0 2021-11-03T17:14:58.550 INFO:tasks.workunit.client.0.smithi027.stdout:1/974: stat d6/d11/dac/d79/dab/lb8 0 2021-11-03T17:14:58.552 INFO:tasks.workunit.client.0.smithi027.stdout:1/975: symlink d6/d11/d14/df6/l140 0 2021-11-03T17:14:58.553 INFO:tasks.workunit.client.0.smithi027.stdout:1/976: stat d6/d11/dac/d79/d25/d5e/l106 0 2021-11-03T17:14:58.573 INFO:tasks.workunit.client.0.smithi027.stdout:1/977: dwrite d6/d11/dac/d79/d25/d28/f55 [0,4194304] 0 2021-11-03T17:14:58.575 INFO:tasks.workunit.client.0.smithi027.stdout:1/978: chown d6/d11/de2/ffb 99864050 1 2021-11-03T17:14:58.596 INFO:tasks.workunit.client.0.smithi027.stdout:1/979: dwrite d6/d11/dac/d79/d37/de0/de4/fc7 [0,4194304] 0 2021-11-03T17:14:58.599 INFO:tasks.workunit.client.0.smithi027.stdout:1/980: creat d6/d11/dac/d79/d25/f141 x:0 0 0 2021-11-03T17:14:58.601 INFO:tasks.workunit.client.0.smithi027.stdout:1/981: creat d6/d11/dac/d79/d37/de0/d4e/f142 x:0 0 0 2021-11-03T17:14:58.602 INFO:tasks.workunit.client.0.smithi027.stdout:1/982: readlink d6/d11/dac/d79/d25/d5e/d72/l10e 0 2021-11-03T17:14:58.625 INFO:tasks.workunit.client.0.smithi027.stdout:1/983: dwrite d6/d11/dac/d79/d25/d28/d5c/f78 [0,4194304] 0 2021-11-03T17:14:58.625 INFO:tasks.workunit.client.0.smithi027.stdout:1/984: getdents d6/d11/dac/d79/db0/de5/d136 0 2021-11-03T17:14:58.626 INFO:tasks.workunit.client.0.smithi027.stdout:1/985: mknod d6/d11/dac/d79/d25/d28/d5c/d7a/dee/c143 1 2021-11-03T17:14:58.634 INFO:tasks.workunit.client.0.smithi027.stdout:1/986: dread d6/d11/dac/d79/d37/de0/de4/fc7 [0,4194304] 0 2021-11-03T17:14:58.642 INFO:tasks.workunit.client.0.smithi027.stdout:1/987: link d6/fa3 d6/d11/dac/d79/dfa/f144 0 2021-11-03T17:14:58.642 INFO:tasks.workunit.client.0.smithi027.stdout:1/988: readlink d6/d11/dac/d79/db0/d102/le7 0 2021-11-03T17:14:58.642 INFO:tasks.workunit.client.0.smithi027.stdout:1/989: write d6/d11/dac/d79/d37/de0/d4e/f91 [1082522,42645] 0 2021-11-03T17:14:58.652 INFO:tasks.workunit.client.0.smithi027.stdout:1/990: dread d6/d11/dac/d79/d25/d5e/f109 [0,4194304] 0 2021-11-03T17:14:58.654 INFO:tasks.workunit.client.0.smithi027.stdout:1/991: creat d6/d11/dac/d79/db0/de5/f145 x:0 0 0 2021-11-03T17:14:58.655 INFO:tasks.workunit.client.0.smithi027.stdout:1/992: creat d6/d11/dac/d79/d37/de0/d4e/f146 x:0 0 0 2021-11-03T17:14:58.657 INFO:tasks.workunit.client.0.smithi027.stdout:1/993: creat d6/d11/dac/d79/d37/de0/de4/f147 x:0 0 0 2021-11-03T17:14:58.659 INFO:tasks.workunit.client.0.smithi027.stdout:1/994: fsync d6/de/f19 0 2021-11-03T17:14:58.667 INFO:tasks.workunit.client.0.smithi027.stdout:1/995: write d6/d11/dac/d79/d25/d28/d5c/d7a/dee/fda [3897391,32696] 0 2021-11-03T17:14:58.684 INFO:tasks.workunit.client.0.smithi027.stdout:1/996: dwrite d6/d11/dac/d79/d37/d49/f50 [0,4194304] 0 2021-11-03T17:14:58.687 INFO:tasks.workunit.client.0.smithi027.stdout:1/997: creat d6/d11/dac/d79/d25/d28/d5c/f148 x:0 0 0 2021-11-03T17:14:58.688 INFO:tasks.workunit.client.0.smithi027.stdout:1/998: getdents d6/d11 0 2021-11-03T17:14:58.691 INFO:tasks.workunit.client.0.smithi027.stdout:1/999: symlink d6/d11/dac/d79/d25/d28/d5c/d81/d7d/l149 0 2021-11-03T17:14:58.865 INFO:tasks.workunit.client.0.smithi027.stdout:2/968: sync 2021-11-03T17:14:58.867 INFO:tasks.workunit.client.0.smithi027.stdout:2/969: symlink d0/d1b/d1c/l128 0 2021-11-03T17:14:58.868 INFO:tasks.workunit.client.0.smithi027.stdout:2/970: mknod d0/d1b/d29/d58/c129 1 2021-11-03T17:14:58.870 INFO:tasks.workunit.client.0.smithi027.stdout:2/971: creat d0/d1b/d29/d54/d64/daa/d61/f12a x:0 0 0 2021-11-03T17:14:58.878 INFO:tasks.workunit.client.0.smithi027.stdout:2/972: write d0/f9b [1375210,27062] 0 2021-11-03T17:14:58.879 INFO:tasks.workunit.client.0.smithi027.stdout:2/973: readlink d0/d1b/d29/d54/d64/daa/d61/l63 0 2021-11-03T17:14:58.880 INFO:tasks.workunit.client.0.smithi027.stdout:2/974: read d0/da7/f11e [3413774,35386] 0 2021-11-03T17:14:58.884 INFO:tasks.workunit.client.0.smithi027.stdout:2/975: unlink d0/da7/dc9/l73 0 2021-11-03T17:14:58.885 INFO:tasks.workunit.client.0.smithi027.stdout:2/976: mknod d0/d1b/d1c/d100/c12b 1 2021-11-03T17:14:58.886 INFO:tasks.workunit.client.0.smithi027.stdout:2/977: creat d0/d1b/d29/d54/d64/f12c x:0 0 0 2021-11-03T17:14:58.887 INFO:tasks.workunit.client.0.smithi027.stdout:2/978: creat d0/d5/f12d x:0 0 0 2021-11-03T17:14:58.889 INFO:tasks.workunit.client.0.smithi027.stdout:2/979: creat d0/d7c/f12e x:0 0 0 2021-11-03T17:14:58.890 INFO:tasks.workunit.client.0.smithi027.stdout:2/980: write d0/d1b/d87/f105 [619673,44324] 0 2021-11-03T17:14:58.894 INFO:tasks.workunit.client.0.smithi027.stdout:2/981: link d0/d1b/d29/d92/dbd/l5f d0/d5/l12f 0 2021-11-03T17:14:58.894 INFO:tasks.workunit.client.0.smithi027.stdout:2/982: chown d0/d7c/ff4 6 1 2021-11-03T17:14:58.895 INFO:tasks.workunit.client.0.smithi027.stdout:2/983: dread - d0/d45/fb8 zero size 2021-11-03T17:14:58.895 INFO:tasks.workunit.client.0.smithi027.stdout:2/984: rmdir d0/d1b/d87 39 2021-11-03T17:14:58.897 INFO:tasks.workunit.client.0.smithi027.stdout:2/985: rename d0/l28 to d0/d1b/d29/dd7/l130 0 2021-11-03T17:14:58.915 INFO:tasks.workunit.client.0.smithi027.stdout:2/986: dwrite d0/d5/f110 [0,4194304] 0 2021-11-03T17:14:58.915 INFO:tasks.workunit.client.0.smithi027.stdout:2/987: dread - d0/d7c/df1/f127 zero size 2021-11-03T17:14:58.917 INFO:tasks.workunit.client.0.smithi027.stdout:2/988: creat d0/d5/d2c/dc8/f131 x:0 0 0 2021-11-03T17:14:58.919 INFO:tasks.workunit.client.0.smithi027.stdout:2/989: symlink d0/d45/d11a/l132 0 2021-11-03T17:14:58.930 INFO:tasks.workunit.client.0.smithi027.stdout:2/990: link d0/d5/f3 d0/d7c/d101/d123/f133 0 2021-11-03T17:14:58.930 INFO:tasks.workunit.client.0.smithi027.stdout:2/991: mknod d0/d7c/d101/d123/c134 1 2021-11-03T17:14:58.940 INFO:tasks.workunit.client.0.smithi027.stdout:2/992: dread d0/d1b/d29/d54/d64/daa/f30 [0,4194304] 0 2021-11-03T17:14:58.941 INFO:tasks.workunit.client.0.smithi027.stdout:2/993: dread - d0/d1b/d29/d54/ff2 zero size 2021-11-03T17:14:58.941 INFO:tasks.workunit.client.0.smithi027.stdout:2/994: stat d0/da7/dc9/l115 0 2021-11-03T17:14:58.943 INFO:tasks.workunit.client.0.smithi027.stdout:2/995: rmdir d0/d1b/d1c/d100/d11c 0 2021-11-03T17:14:58.944 INFO:tasks.workunit.client.0.smithi027.stdout:2/996: readlink d0/d5/l17 0 2021-11-03T17:14:58.945 INFO:tasks.workunit.client.0.smithi027.stdout:2/997: readlink d0/d5/la 0 2021-11-03T17:14:58.945 INFO:tasks.workunit.client.0.smithi027.stdout:2/998: fdatasync d0/d45/fb8 0 2021-11-03T17:14:58.947 INFO:tasks.workunit.client.0.smithi027.stdout:2/999: link d0/d5/f116 d0/d45/f135 0 2021-11-03T17:14:59.572 INFO:tasks.workunit.client.0.smithi027.stdout:7/637: sync 2021-11-03T17:14:59.594 INFO:tasks.workunit.client.0.smithi027.stdout:7/638: dwrite d0/d2/d10/d35/d96/fb3 [0,4194304] 0 2021-11-03T17:14:59.599 INFO:tasks.workunit.client.0.smithi027.stdout:7/639: creat d0/d2/d5e/d36/d3a/fd0 x:0 0 0 2021-11-03T17:14:59.605 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:59 smithi027 conmon[32322]: cluster 2021-11-03T17:14:57.900466 2021-11-03T17:14:59.606 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:14:59 smithi027 conmon[32322]: +0000 mgr.smithi115.tcyodz (mgr.24395) 35 : cluster [DBG] pgmap v24: 65 pgs: 65 active+clean; 3.2 GiB data, 9.3 GiB used, 527 GiB / 536 GiB avail; 14 MiB/s rd, 26 MiB/s wr, 71 op/s 2021-11-03T17:14:59.608 INFO:tasks.workunit.client.0.smithi027.stdout:7/640: dread d0/d2/d10/d35/f89 [0,4194304] 0 2021-11-03T17:14:59.614 INFO:tasks.workunit.client.0.smithi027.stdout:7/641: dread d0/d2d/d58/f6d [0,4194304] 0 2021-11-03T17:14:59.616 INFO:tasks.workunit.client.0.smithi027.stdout:7/642: creat d0/d2d/d2f/fd1 x:0 0 0 2021-11-03T17:14:59.617 INFO:tasks.workunit.client.0.smithi027.stdout:7/643: getdents d0/d2d/d58/dba 0 2021-11-03T17:14:59.617 INFO:tasks.workunit.client.0.smithi027.stdout:7/644: readlink d0/d2d/lbb 0 2021-11-03T17:14:59.619 INFO:tasks.workunit.client.0.smithi027.stdout:7/645: truncate d0/d2/d10/f1b 1046955 0 2021-11-03T17:14:59.627 INFO:tasks.workunit.client.0.smithi027.stdout:7/646: write d0/d2d/d2f/fb9 [114363,105098] 0 2021-11-03T17:14:59.628 INFO:tasks.workunit.client.0.smithi027.stdout:7/647: chown d0/d2/d5e/d36 121031 1 2021-11-03T17:14:59.632 INFO:tasks.workunit.client.0.smithi027.stdout:7/648: symlink d0/d2/d10/d1e/d20/d74/ld2 0 2021-11-03T17:14:59.647 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:59 smithi115 conmon[34670]: cluster 2021-11-03T17:14:57.900466+0000 2021-11-03T17:14:59.648 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:14:59 smithi115 conmon[34670]: mgr.smithi115.tcyodz (mgr.24395) 35 : cluster [DBG] pgmap v24: 65 pgs: 65 active+clean; 3.2 GiB data, 9.3 GiB used, 527 GiB / 536 GiB avail; 14 MiB/s rd, 26 MiB/s wr, 71 op/s 2021-11-03T17:14:59.649 INFO:tasks.workunit.client.0.smithi027.stdout:7/649: write d0/d2/d1c/f79 [1885767,125875] 0 2021-11-03T17:14:59.651 INFO:tasks.workunit.client.0.smithi027.stdout:7/650: dread d0/d2/d10/d1e/f88 [0,4194304] 0 2021-11-03T17:14:59.651 INFO:tasks.workunit.client.0.smithi027.stdout:4/841: sync 2021-11-03T17:14:59.654 INFO:tasks.workunit.client.0.smithi027.stdout:4/842: rename d6/d9/d16/lcd to d6/d9/d16/d36/d57/d5a/l11a 0 2021-11-03T17:14:59.656 INFO:tasks.workunit.client.0.smithi027.stdout:4/843: dread d6/d24/dbe/de5/ffe [0,4194304] 0 2021-11-03T17:14:59.658 INFO:tasks.workunit.client.0.smithi027.stdout:4/844: symlink d6/d24/db0/l11b 0 2021-11-03T17:14:59.659 INFO:tasks.workunit.client.0.smithi027.stdout:7/651: dread d0/f16 [0,4194304] 0 2021-11-03T17:14:59.662 INFO:tasks.workunit.client.0.smithi027.stdout:7/652: symlink d0/d2/d5e/d36/d3a/ld3 0 2021-11-03T17:14:59.663 INFO:tasks.workunit.client.0.smithi027.stdout:7/653: chown d0/d2/d5e/la1 1857461 1 2021-11-03T17:14:59.665 INFO:tasks.workunit.client.0.smithi027.stdout:7/654: creat d0/d2/d1c/d4d/dcc/fd4 x:0 0 0 2021-11-03T17:14:59.669 INFO:tasks.workunit.client.0.smithi027.stdout:7/655: creat d0/d2/d1c/d4d/fd5 x:0 0 0 2021-11-03T17:14:59.683 INFO:tasks.workunit.client.0.smithi027.stdout:4/845: dwrite d6/d9/d16/d36/f6a [0,4194304] 0 2021-11-03T17:14:59.693 INFO:tasks.workunit.client.0.smithi027.stdout:7/656: dwrite d0/d2d/d2f/fb9 [0,4194304] 0 2021-11-03T17:14:59.696 INFO:tasks.workunit.client.0.smithi027.stdout:7/657: getdents d0/d2d 0 2021-11-03T17:14:59.711 INFO:tasks.workunit.client.0.smithi027.stdout:4/846: dwrite d6/d24/db0/d81/fa9 [0,4194304] 0 2021-11-03T17:14:59.714 INFO:tasks.workunit.client.0.smithi027.stdout:4/847: link d6/d9/d16/d36/d70/d3e/d58/d84/l116 d6/d24/de3/d9f/l11c 0 2021-11-03T17:14:59.716 INFO:tasks.workunit.client.0.smithi027.stdout:4/848: truncate d6/d9/d8f/f95 4966567 0 2021-11-03T17:14:59.717 INFO:tasks.workunit.client.0.smithi027.stdout:4/849: dread - d6/d9/d16/dc3/fd3 zero size 2021-11-03T17:14:59.719 INFO:tasks.workunit.client.0.smithi027.stdout:7/658: dwrite d0/d2/d10/f11 [0,4194304] 0 2021-11-03T17:14:59.722 INFO:tasks.workunit.client.0.smithi027.stdout:4/850: dread d6/d24/de3/d9f/dd7/f103 [0,4194304] 0 2021-11-03T17:14:59.722 INFO:tasks.workunit.client.0.smithi027.stdout:4/851: chown d6/d24/de3/d102/d119/ddd 72401839 1 2021-11-03T17:14:59.724 INFO:tasks.workunit.client.0.smithi027.stdout:4/852: link d6/d9/d16/d36/d70/d3e/d58/d84/d9b/fcb d6/d9/d16/d36/d70/d3e/d58/d84/d9b/f11d 0 2021-11-03T17:14:59.729 INFO:tasks.workunit.client.0.smithi027.stdout:4/853: symlink d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/l11e 0 2021-11-03T17:14:59.730 INFO:tasks.workunit.client.0.smithi027.stdout:7/659: write d0/f18 [1720459,60740] 0 2021-11-03T17:14:59.731 INFO:tasks.workunit.client.0.smithi027.stdout:7/660: creat d0/d2/d1c/d4d/d6e/fd6 x:0 0 0 2021-11-03T17:14:59.734 INFO:tasks.workunit.client.0.smithi027.stdout:4/854: rename d6/d9/d16/d36/d70/d3e/d58/l74 to d6/d24/de3/d102/d119/l11f 0 2021-11-03T17:14:59.735 INFO:tasks.workunit.client.0.smithi027.stdout:7/661: symlink d0/d2/d10/d35/ld7 0 2021-11-03T17:14:59.743 INFO:tasks.workunit.client.0.smithi027.stdout:4/855: dread d6/d24/de3/d9f/fa8 [8388608,4194304] 0 2021-11-03T17:14:59.745 INFO:tasks.workunit.client.0.smithi027.stdout:4/856: rmdir d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21 39 2021-11-03T17:14:59.747 INFO:tasks.workunit.client.0.smithi027.stdout:4/857: mkdir d6/d24/d6d/de6/d120 0 2021-11-03T17:14:59.747 INFO:tasks.workunit.client.0.smithi027.stdout:9/875: sync 2021-11-03T17:14:59.748 INFO:tasks.workunit.client.0.smithi027.stdout:4/858: creat d6/d9/d16/d36/d99/f121 x:0 0 0 2021-11-03T17:14:59.750 INFO:tasks.workunit.client.0.smithi027.stdout:9/876: creat db/d1b/d2b/d41/d95/d9f/d12f/d13b/f143 x:0 0 0 2021-11-03T17:14:59.751 INFO:tasks.workunit.client.0.smithi027.stdout:4/859: truncate d6/d9/d16/d36/f42 1475824 0 2021-11-03T17:14:59.751 INFO:tasks.workunit.client.0.smithi027.stdout:4/860: chown d6/d24/db0/d81 565 1 2021-11-03T17:14:59.754 INFO:tasks.workunit.client.0.smithi027.stdout:4/861: unlink d6/d9/d16/d36/d57/d5a/l9c 0 2021-11-03T17:14:59.764 INFO:tasks.workunit.client.0.smithi027.stdout:9/877: write db/d1b/d1c/d1d/fa9 [856289,8315] 0 2021-11-03T17:14:59.778 INFO:tasks.workunit.client.0.smithi027.stdout:9/878: dread db/d1b/d23/ffb [0,4194304] 0 2021-11-03T17:14:59.779 INFO:tasks.workunit.client.0.smithi027.stdout:9/879: creat db/d1b/d1c/f144 x:0 0 0 2021-11-03T17:14:59.780 INFO:tasks.workunit.client.0.smithi027.stdout:4/862: dwrite d6/d24/dbe/fea [0,4194304] 0 2021-11-03T17:14:59.790 INFO:tasks.workunit.client.0.smithi027.stdout:4/863: dread d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/fa2 [0,4194304] 0 2021-11-03T17:14:59.790 INFO:tasks.workunit.client.0.smithi027.stdout:4/864: dread - d6/d9/d16/d36/dc1/f101 zero size 2021-11-03T17:14:59.801 INFO:tasks.workunit.client.0.smithi027.stdout:9/880: dwrite db/d1b/d2b/fee [0,4194304] 0 2021-11-03T17:14:59.809 INFO:tasks.workunit.client.0.smithi027.stdout:4/865: dwrite d6/d9/d16/d36/d57/d5a/d63/fd9 [0,4194304] 0 2021-11-03T17:14:59.812 INFO:tasks.workunit.client.0.smithi027.stdout:4/866: chown d6/d9/d16/d36/d99/db9/ld5 2827 1 2021-11-03T17:14:59.813 INFO:tasks.workunit.client.0.smithi027.stdout:4/867: creat d6/d9/d16/d36/d70/f122 x:0 0 0 2021-11-03T17:14:59.816 INFO:tasks.workunit.client.0.smithi027.stdout:4/868: symlink d6/d9/d16/d36/l123 0 2021-11-03T17:14:59.817 INFO:tasks.workunit.client.0.smithi027.stdout:4/869: write d6/d24/dbe/fea [888363,22181] 0 2021-11-03T17:14:59.819 INFO:tasks.workunit.client.0.smithi027.stdout:4/870: creat d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d9a/f124 x:0 0 0 2021-11-03T17:14:59.820 INFO:tasks.workunit.client.0.smithi027.stdout:4/871: mkdir d6/d9/d16/d125 0 2021-11-03T17:14:59.822 INFO:tasks.workunit.client.0.smithi027.stdout:4/872: getdents d6/d9/d8f/dad 0 2021-11-03T17:14:59.825 INFO:tasks.workunit.client.0.smithi027.stdout:4/873: rename d6/d9/d16/d36/fc0 to d6/d9/d16/d36/d70/d3e/d58/f126 0 2021-11-03T17:14:59.826 INFO:tasks.workunit.client.0.smithi027.stdout:4/874: getdents d6/d24/de3/d9f/dd7 0 2021-11-03T17:14:59.828 INFO:tasks.workunit.client.0.smithi027.stdout:4/875: truncate d6/d9/d16/f7f 712091 0 2021-11-03T17:14:59.829 INFO:tasks.workunit.client.0.smithi027.stdout:4/876: write f4 [5070812,10043] 0 2021-11-03T17:14:59.829 INFO:tasks.workunit.client.0.smithi027.stdout:4/877: truncate d6/d9/d16/f26 4937299 0 2021-11-03T17:14:59.830 INFO:tasks.workunit.client.0.smithi027.stdout:4/878: readlink d6/d24/l7c 0 2021-11-03T17:14:59.830 INFO:tasks.workunit.client.0.smithi027.stdout:9/881: dwrite db/d1b/d2b/d41/d95/fce [0,4194304] 0 2021-11-03T17:14:59.832 INFO:tasks.workunit.client.0.smithi027.stdout:4/879: mkdir d6/d9/d16/d36/d70/d3e/de9/d127 0 2021-11-03T17:14:59.836 INFO:tasks.workunit.client.0.smithi027.stdout:4/880: creat d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d41/d10a/f128 x:0 0 0 2021-11-03T17:14:59.836 INFO:tasks.workunit.client.0.smithi027.stdout:4/881: stat d6/d24/dbe/fc6 0 2021-11-03T17:14:59.837 INFO:tasks.workunit.client.0.smithi027.stdout:9/882: symlink db/d107/l145 0 2021-11-03T17:14:59.843 INFO:tasks.workunit.client.0.smithi027.stdout:4/882: readlink d6/d24/l33 0 2021-11-03T17:14:59.845 INFO:tasks.workunit.client.0.smithi027.stdout:9/883: unlink db/d1b/d2b/d41/d95/d9f/d12f/d13b/f143 0 2021-11-03T17:14:59.846 INFO:tasks.workunit.client.0.smithi027.stdout:9/884: read db/d1b/d2b/d41/fb8 [2095786,45717] 0 2021-11-03T17:14:59.930 INFO:tasks.workunit.client.0.smithi027.stdout:4/883: dwrite d6/d9/d16/d36/d57/fce [4194304,4194304] 0 2021-11-03T17:14:59.931 INFO:tasks.workunit.client.0.smithi027.stdout:9/885: dwrite db/f16 [0,4194304] 0 2021-11-03T17:14:59.933 INFO:tasks.workunit.client.0.smithi027.stdout:4/884: creat d6/d9/d16/d36/d99/db9/f129 x:0 0 0 2021-11-03T17:14:59.935 INFO:tasks.workunit.client.0.smithi027.stdout:9/886: truncate db/d1b/d2b/d9c/f13f 3693301 0 2021-11-03T17:14:59.936 INFO:tasks.workunit.client.0.smithi027.stdout:4/885: fdatasync d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d41/f88 0 2021-11-03T17:14:59.940 INFO:tasks.workunit.client.0.smithi027.stdout:9/887: symlink db/d1b/d2b/dc7/l146 0 2021-11-03T17:14:59.945 INFO:tasks.workunit.client.0.smithi027.stdout:9/888: rename db/d1b/l61 to db/d1b/d2b/l147 0 2021-11-03T17:14:59.946 INFO:tasks.workunit.client.0.smithi027.stdout:9/889: chown db/d1b/d8c/dcd/f113 802 1 2021-11-03T17:14:59.949 INFO:tasks.workunit.client.0.smithi027.stdout:9/890: creat db/d1b/dfa/f148 x:0 0 0 2021-11-03T17:14:59.951 INFO:tasks.workunit.client.0.smithi027.stdout:9/891: dread db/d1b/f20 [0,4194304] 0 2021-11-03T17:14:59.954 INFO:tasks.workunit.client.0.smithi027.stdout:9/892: creat db/d1b/d2b/d41/d95/f149 x:0 0 0 2021-11-03T17:14:59.957 INFO:tasks.workunit.client.0.smithi027.stdout:9/893: mkdir db/d1b/d2b/d41/dad/dbb/d14a 0 2021-11-03T17:14:59.957 INFO:tasks.workunit.client.0.smithi027.stdout:9/894: mknod db/d1b/d1c/d1d/c14b 1 2021-11-03T17:14:59.958 INFO:tasks.workunit.client.0.smithi027.stdout:9/895: chown db/d1b/d106/f112 16209175 1 2021-11-03T17:14:59.960 INFO:tasks.workunit.client.0.smithi027.stdout:9/896: mkdir db/d1b/d8e/d14c 0 2021-11-03T17:14:59.962 INFO:tasks.workunit.client.0.smithi027.stdout:4/886: dwrite d6/d9/d16/f19 [0,4194304] 0 2021-11-03T17:14:59.963 INFO:tasks.workunit.client.0.smithi027.stdout:4/887: stat d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d41/d10a 0 2021-11-03T17:14:59.963 INFO:tasks.workunit.client.0.smithi027.stdout:4/888: dread - d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d41/d10a/f128 zero size 2021-11-03T17:14:59.964 INFO:tasks.workunit.client.0.smithi027.stdout:4/889: creat d6/d24/dbe/de5/f12a x:0 0 0 2021-11-03T17:14:59.970 INFO:tasks.workunit.client.0.smithi027.stdout:4/890: dread d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/f2c [0,4194304] 0 2021-11-03T17:14:59.972 INFO:tasks.workunit.client.0.smithi027.stdout:4/891: symlink d6/d24/db0/d81/l12b 0 2021-11-03T17:14:59.973 INFO:tasks.workunit.client.0.smithi027.stdout:4/892: rmdir d6/d9/d8f 39 2021-11-03T17:14:59.974 INFO:tasks.workunit.client.0.smithi027.stdout:4/893: chown d6/d9/f73 916634 1 2021-11-03T17:14:59.975 INFO:tasks.workunit.client.0.smithi027.stdout:4/894: symlink d6/d24/de3/d102/l12c 0 2021-11-03T17:14:59.976 INFO:tasks.workunit.client.0.smithi027.stdout:4/895: readlink d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/l54 0 2021-11-03T17:14:59.978 INFO:tasks.workunit.client.0.smithi027.stdout:4/896: creat d6/d24/de3/d9f/dd7/f12d x:0 0 0 2021-11-03T17:14:59.980 INFO:tasks.workunit.client.0.smithi027.stdout:4/897: creat d6/d24/db0/f12e x:0 0 0 2021-11-03T17:14:59.980 INFO:tasks.workunit.client.0.smithi027.stdout:4/898: dread - d6/d9/fff zero size 2021-11-03T17:14:59.982 INFO:tasks.workunit.client.0.smithi027.stdout:4/899: mkdir d6/d9/d16/d36/d57/d12f 0 2021-11-03T17:14:59.983 INFO:tasks.workunit.client.0.smithi027.stdout:4/900: creat d6/d24/db0/f130 x:0 0 0 2021-11-03T17:14:59.984 INFO:tasks.workunit.client.0.smithi027.stdout:4/901: dread - d6/d9/f105 zero size 2021-11-03T17:14:59.984 INFO:tasks.workunit.client.0.smithi027.stdout:4/902: write d6/d9/d16/d36/d70/d3e/d58/f60 [1507440,50238] 0 2021-11-03T17:14:59.985 INFO:tasks.workunit.client.0.smithi027.stdout:4/903: dread - d6/d9/d16/d36/d99/f10d zero size 2021-11-03T17:14:59.985 INFO:tasks.workunit.client.0.smithi027.stdout:4/904: mknod d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d41/d10a/c131 1 2021-11-03T17:14:59.985 INFO:tasks.workunit.client.0.smithi027.stdout:4/905: chown d6/d9/d16/ff8 608838 1 2021-11-03T17:14:59.988 INFO:tasks.workunit.client.0.smithi027.stdout:9/897: dwrite db/d1b/d8c/dcd/f113 [0,4194304] 0 2021-11-03T17:14:59.988 INFO:tasks.workunit.client.0.smithi027.stdout:9/898: chown db/d1b/d8e/d14c 251065 1 2021-11-03T17:14:59.988 INFO:tasks.workunit.client.0.smithi027.stdout:9/899: chown db/d1b/d2b/d41/d95/d9f 43584 1 2021-11-03T17:14:59.990 INFO:tasks.workunit.client.0.smithi027.stdout:9/900: mkdir db/d1b/d1c/d1d/d40/d104/d14d 0 2021-11-03T17:14:59.993 INFO:tasks.workunit.client.0.smithi027.stdout:6/819: sync 2021-11-03T17:14:59.995 INFO:tasks.workunit.client.0.smithi027.stdout:9/901: rmdir db/d1b/d1c/dc5 39 2021-11-03T17:14:59.998 INFO:tasks.workunit.client.0.smithi027.stdout:6/820: mkdir dc/dff 0 2021-11-03T17:15:00.002 INFO:tasks.workunit.client.0.smithi027.stdout:4/906: write d6/d9/d8f/f95 [4833540,37578] 0 2021-11-03T17:15:00.006 INFO:tasks.workunit.client.0.smithi027.stdout:6/821: symlink dc/d82/d57/dad/dae/db1/d34/l100 0 2021-11-03T17:15:00.008 INFO:tasks.workunit.client.0.smithi027.stdout:6/822: unlink dc/d16/d1b/d3c/d51/lee 0 2021-11-03T17:15:00.010 INFO:tasks.workunit.client.0.smithi027.stdout:6/823: unlink dc/d16/d1b/d3c/d51/fe9 0 2021-11-03T17:15:00.011 INFO:tasks.workunit.client.0.smithi027.stdout:6/824: readlink dc/d82/d57/d96/la2 0 2021-11-03T17:15:00.014 INFO:tasks.workunit.client.0.smithi027.stdout:6/825: rename dc/l8c to dc/da1/l101 0 2021-11-03T17:15:00.014 INFO:tasks.workunit.client.0.smithi027.stdout:6/826: stat dc/d16/d1b/d24/f2a 0 2021-11-03T17:15:00.015 INFO:tasks.workunit.client.0.smithi027.stdout:9/902: dwrite db/d1b/f51 [0,4194304] 0 2021-11-03T17:15:00.016 INFO:tasks.workunit.client.0.smithi027.stdout:6/827: creat dc/d16/d1b/d3c/d51/f102 x:0 0 0 2021-11-03T17:15:00.019 INFO:tasks.workunit.client.0.smithi027.stdout:6/828: unlink dc/f10 0 2021-11-03T17:15:00.020 INFO:tasks.workunit.client.0.smithi027.stdout:6/829: getdents dc/d82/d57/d9b/dc8 0 2021-11-03T17:15:00.022 INFO:tasks.workunit.client.0.smithi027.stdout:4/907: dwrite d6/d9/d16/d36/d99/f121 [0,4194304] 0 2021-11-03T17:15:00.023 INFO:tasks.workunit.client.0.smithi027.stdout:6/830: link dc/d82/d57/d9b/dc8/faa dc/d82/d57/dad/dae/f103 0 2021-11-03T17:15:00.023 INFO:tasks.workunit.client.0.smithi027.stdout:9/903: dread db/d1b/d2b/d41/f82 [0,4194304] 0 2021-11-03T17:15:00.024 INFO:tasks.workunit.client.0.smithi027.stdout:4/908: chown d6/d9/d16/d36/d99/db9/ld5 58976550 1 2021-11-03T17:15:00.025 INFO:tasks.workunit.client.0.smithi027.stdout:6/831: chown dc/d82/d57/dad/dae/db1/d34/d89/f9d 1042258587 1 2021-11-03T17:15:00.027 INFO:tasks.workunit.client.0.smithi027.stdout:4/909: stat d6/d9/d16/d36/d70/d3e/d58/d84/lc7 0 2021-11-03T17:15:00.027 INFO:tasks.workunit.client.0.smithi027.stdout:4/910: dread - d6/d9/d16/d36/d70/d3e/de9/f107 zero size 2021-11-03T17:15:00.028 INFO:tasks.workunit.client.0.smithi027.stdout:6/832: dread dc/d82/d57/f97 [0,4194304] 0 2021-11-03T17:15:00.029 INFO:tasks.workunit.client.0.smithi027.stdout:9/904: symlink db/d1b/d8e/l14e 0 2021-11-03T17:15:00.036 INFO:tasks.workunit.client.0.smithi027.stdout:6/833: creat dc/d16/d1b/d3c/d51/df9/de7/f104 x:0 0 0 2021-11-03T17:15:00.045 INFO:tasks.workunit.client.0.smithi027.stdout:6/834: creat dc/d16/d1b/f105 x:0 0 0 2021-11-03T17:15:00.045 INFO:tasks.workunit.client.0.smithi027.stdout:6/835: dread - dc/d16/d1b/dda/ff2 zero size 2021-11-03T17:15:00.046 INFO:tasks.workunit.client.0.smithi027.stdout:9/905: dwrite db/f13a [0,4194304] 0 2021-11-03T17:15:00.048 INFO:tasks.workunit.client.0.smithi027.stdout:6/836: symlink dc/d16/d1b/d24/d7e/db3/de8/l106 0 2021-11-03T17:15:00.051 INFO:tasks.workunit.client.0.smithi027.stdout:9/906: rmdir db/d1b/d2b/d41/dad 39 2021-11-03T17:15:00.056 INFO:tasks.workunit.client.0.smithi027.stdout:6/837: mkdir dc/d82/d57/d96/d107 0 2021-11-03T17:15:00.057 INFO:tasks.workunit.client.0.smithi027.stdout:6/838: chown dc/d16/l6d 4504042 1 2021-11-03T17:15:00.058 INFO:tasks.workunit.client.0.smithi027.stdout:6/839: getdents dc/d16 0 2021-11-03T17:15:00.062 INFO:tasks.workunit.client.0.smithi027.stdout:4/911: dwrite d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/fc2 [0,4194304] 0 2021-11-03T17:15:00.064 INFO:tasks.workunit.client.0.smithi027.stdout:6/840: write dc/d16/f40 [1924467,20694] 0 2021-11-03T17:15:00.065 INFO:tasks.workunit.client.0.smithi027.stdout:6/841: mknod dc/d82/d57/dad/dae/db1/c108 1 2021-11-03T17:15:00.067 INFO:tasks.workunit.client.0.smithi027.stdout:9/907: write db/d1b/d23/d2c/f89 [3377706,129617] 0 2021-11-03T17:15:00.068 INFO:tasks.workunit.client.0.smithi027.stdout:9/908: getdents db 0 2021-11-03T17:15:00.073 INFO:tasks.workunit.client.0.smithi027.stdout:6/842: dread dc/d16/d1b/d3c/f9e [0,4194304] 0 2021-11-03T17:15:00.077 INFO:tasks.workunit.client.0.smithi027.stdout:6/843: creat dc/d16/d1b/d3c/d51/d5f/f109 x:0 0 0 2021-11-03T17:15:00.081 INFO:tasks.workunit.client.0.smithi027.stdout:4/912: dwrite d6/d9/d16/f26 [0,4194304] 0 2021-11-03T17:15:00.084 INFO:tasks.workunit.client.0.smithi027.stdout:4/913: mkdir d6/d9/d16/d36/d99/dd0/d132 0 2021-11-03T17:15:00.084 INFO:tasks.workunit.client.0.smithi027.stdout:4/914: write d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d9a/f106 [146523,95461] 0 2021-11-03T17:15:00.089 INFO:tasks.workunit.client.0.smithi027.stdout:9/909: dwrite db/d1b/d23/f4b [0,4194304] 0 2021-11-03T17:15:00.090 INFO:tasks.workunit.client.0.smithi027.stdout:6/844: dread dc/f88 [0,4194304] 0 2021-11-03T17:15:00.092 INFO:tasks.workunit.client.0.smithi027.stdout:6/845: truncate dc/d82/d57/dad/dae/fbb 3493872 0 2021-11-03T17:15:00.094 INFO:tasks.workunit.client.0.smithi027.stdout:6/846: creat dc/d16/d1b/d24/d7e/f10a x:0 0 0 2021-11-03T17:15:00.096 INFO:tasks.workunit.client.0.smithi027.stdout:9/910: rename db/d1b/d1c/dc5 to db/d1b/d8c/d14f 0 2021-11-03T17:15:00.096 INFO:tasks.workunit.client.0.smithi027.stdout:9/911: read - db/d1b/d1c/f144 zero size 2021-11-03T17:15:00.101 INFO:tasks.workunit.client.0.smithi027.stdout:6/847: unlink dc/d82/d87/l9f 0 2021-11-03T17:15:00.102 INFO:tasks.workunit.client.0.smithi027.stdout:6/848: chown dc/d82/d57/d96/d107 7603680 1 2021-11-03T17:15:00.112 INFO:tasks.workunit.client.0.smithi027.stdout:4/915: dwrite d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/f2c [0,4194304] 0 2021-11-03T17:15:00.112 INFO:tasks.workunit.client.0.smithi027.stdout:9/912: rename db/d1b/d23/f73 to db/d1b/d106/f150 0 2021-11-03T17:15:00.117 INFO:tasks.workunit.client.0.smithi027.stdout:6/849: rmdir dc 39 2021-11-03T17:15:00.118 INFO:tasks.workunit.client.0.smithi027.stdout:6/850: mknod dc/d82/d57/dad/dae/db1/c10b 1 2021-11-03T17:15:00.120 INFO:tasks.workunit.client.0.smithi027.stdout:6/851: symlink dc/d16/d1b/dda/l10c 0 2021-11-03T17:15:00.120 INFO:tasks.workunit.client.0.smithi027.stdout:6/852: chown dc/d82/d57/dad/dae/db1/d34/l100 3509222 1 2021-11-03T17:15:00.123 INFO:tasks.workunit.client.0.smithi027.stdout:6/853: unlink dc/d16/d1b/f105 0 2021-11-03T17:15:00.136 INFO:tasks.workunit.client.0.smithi027.stdout:4/916: dwrite d6/d24/db0/f130 [0,4194304] 0 2021-11-03T17:15:00.136 INFO:tasks.workunit.client.0.smithi027.stdout:4/917: chown d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d9a/f106 14911 1 2021-11-03T17:15:00.137 INFO:tasks.workunit.client.0.smithi027.stdout:9/913: dwrite db/d1b/d1c/d1d/d40/d60/fc6 [0,4194304] 0 2021-11-03T17:15:00.139 INFO:tasks.workunit.client.0.smithi027.stdout:4/918: chown d6/d24/db0/f8b 26 1 2021-11-03T17:15:00.141 INFO:tasks.workunit.client.0.smithi027.stdout:4/919: dread d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/f7e [0,4194304] 0 2021-11-03T17:15:00.141 INFO:tasks.workunit.client.0.smithi027.stdout:4/920: truncate d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d41/d10a/f128 543866 0 2021-11-03T17:15:00.150 INFO:tasks.workunit.client.0.smithi027.stdout:4/921: truncate d6/d9/d16/d36/f6a 258108 0 2021-11-03T17:15:00.154 INFO:tasks.workunit.client.0.smithi027.stdout:9/914: symlink db/d1b/d2b/l151 0 2021-11-03T17:15:00.156 INFO:tasks.workunit.client.0.smithi027.stdout:4/922: mknod d6/d9/d16/d36/d70/d3e/de9/c133 1 2021-11-03T17:15:00.157 INFO:tasks.workunit.client.0.smithi027.stdout:9/915: mknod db/d1b/d23/c152 1 2021-11-03T17:15:00.159 INFO:tasks.workunit.client.0.smithi027.stdout:9/916: link db/d1b/d2b/d9c/f124 db/d1b/d106/d12b/f153 0 2021-11-03T17:15:00.161 INFO:tasks.workunit.client.0.smithi027.stdout:4/923: symlink d6/d9/d16/d36/d99/dd0/l134 0 2021-11-03T17:15:00.162 INFO:tasks.workunit.client.0.smithi027.stdout:4/924: creat d6/d24/de3/d102/f135 x:0 0 0 2021-11-03T17:15:00.165 INFO:tasks.workunit.client.0.smithi027.stdout:9/917: rename db/d1b/d2b/d41/d97 to db/d1b/d1c/d1d/d40/d60/d154 0 2021-11-03T17:15:00.171 INFO:tasks.workunit.client.0.smithi027.stdout:9/918: creat db/d1b/d2b/d9c/f155 x:0 0 0 2021-11-03T17:15:00.172 INFO:tasks.workunit.client.0.smithi027.stdout:9/919: read db/d1b/d23/d2c/f89 [971783,42709] 0 2021-11-03T17:15:00.173 INFO:tasks.workunit.client.0.smithi027.stdout:9/920: mkdir db/d1b/d1c/d10f/d156 0 2021-11-03T17:15:00.174 INFO:tasks.workunit.client.0.smithi027.stdout:9/921: creat db/d1b/d8c/f157 x:0 0 0 2021-11-03T17:15:00.176 INFO:tasks.workunit.client.0.smithi027.stdout:9/922: creat db/d1b/d2b/d41/d95/d9f/d12f/f158 x:0 0 0 2021-11-03T17:15:00.184 INFO:tasks.workunit.client.0.smithi027.stdout:4/925: dwrite d6/d24/db0/f53 [0,4194304] 0 2021-11-03T17:15:00.187 INFO:tasks.workunit.client.0.smithi027.stdout:9/923: write db/d1b/d1c/f134 [3639920,79361] 0 2021-11-03T17:15:00.189 INFO:tasks.workunit.client.0.smithi027.stdout:9/924: creat db/d1b/d2b/d41/dad/f159 x:0 0 0 2021-11-03T17:15:00.196 INFO:tasks.workunit.client.0.smithi027.stdout:4/926: dread d6/d9/d16/d36/d57/dbb/ffa [0,4194304] 0 2021-11-03T17:15:00.197 INFO:tasks.workunit.client.0.smithi027.stdout:4/927: readlink d6/d24/de3/d9f/dd7/lf7 0 2021-11-03T17:15:00.222 INFO:tasks.workunit.client.0.smithi027.stdout:4/928: dwrite d6/d9/d16/d36/d70/dee/f45 [0,4194304] 0 2021-11-03T17:15:00.223 INFO:tasks.workunit.client.0.smithi027.stdout:9/925: dwrite db/f14 [8388608,4194304] 0 2021-11-03T17:15:00.223 INFO:tasks.workunit.client.0.smithi027.stdout:4/929: write d6/d9/d16/d36/d57/dbb/ffa [1504396,37676] 0 2021-11-03T17:15:00.236 INFO:tasks.workunit.client.0.smithi027.stdout:4/930: dread d6/d9/d8f/fca [0,4194304] 0 2021-11-03T17:15:00.238 INFO:tasks.workunit.client.0.smithi027.stdout:4/931: symlink d6/d9/d16/l136 0 2021-11-03T17:15:00.239 INFO:tasks.workunit.client.0.smithi027.stdout:4/932: stat d6/d9/d16/d36/d57/d5a/d63/f10c 0 2021-11-03T17:15:00.239 INFO:tasks.workunit.client.0.smithi027.stdout:9/926: dwrite db/d1b/d2b/f108 [0,4194304] 0 2021-11-03T17:15:00.242 INFO:tasks.workunit.client.0.smithi027.stdout:9/927: creat db/d1b/d1c/d1d/d40/d104/d14d/f15a x:0 0 0 2021-11-03T17:15:00.244 INFO:tasks.workunit.client.0.smithi027.stdout:4/933: link d6/d9/d16/d36/d70/d3e/d58/d84/f86 d6/d9/d16/d36/d70/f137 0 2021-11-03T17:15:00.249 INFO:tasks.workunit.client.0.smithi027.stdout:4/934: creat d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d41/f138 x:0 0 0 2021-11-03T17:15:00.252 INFO:tasks.workunit.client.0.smithi027.stdout:9/928: write db/d1b/d2b/fba [1490783,71710] 0 2021-11-03T17:15:00.253 INFO:tasks.workunit.client.0.smithi027.stdout:4/935: symlink d6/d24/de3/d102/d119/ddd/l139 0 2021-11-03T17:15:00.256 INFO:tasks.workunit.client.0.smithi027.stdout:4/936: mkdir d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d41/d44/d13a 0 2021-11-03T17:15:00.276 INFO:tasks.workunit.client.0.smithi027.stdout:9/929: dwrite db/d1b/d2b/d41/d95/fcf [0,4194304] 0 2021-11-03T17:15:00.276 INFO:tasks.workunit.client.0.smithi027.stdout:4/937: dwrite d6/d9/fff [0,4194304] 0 2021-11-03T17:15:00.279 INFO:tasks.workunit.client.0.smithi027.stdout:4/938: truncate d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d9a/faf 477495 0 2021-11-03T17:15:00.281 INFO:tasks.workunit.client.0.smithi027.stdout:9/930: symlink db/d1b/d1c/d1d/d40/d60/d154/l15b 0 2021-11-03T17:15:00.281 INFO:tasks.workunit.client.0.smithi027.stdout:9/931: mknod db/d1b/d106/d12b/c15c 1 2021-11-03T17:15:00.283 INFO:tasks.workunit.client.0.smithi027.stdout:4/939: rename d6/d9/d16/d36/d57/dbb/ffa to d6/d24/d6d/de6/f13b 0 2021-11-03T17:15:00.285 INFO:tasks.workunit.client.0.smithi027.stdout:4/940: read d6/d9/d16/d36/d70/d3e/d58/d84/d9b/fa1 [6625,100773] 0 2021-11-03T17:15:00.286 INFO:tasks.workunit.client.0.smithi027.stdout:4/941: truncate d6/d24/de3/d102/d119/f6e 2277297 0 2021-11-03T17:15:00.286 INFO:tasks.workunit.client.0.smithi027.stdout:4/942: chown d6/d9 20 1 2021-11-03T17:15:00.290 INFO:tasks.workunit.client.0.smithi027.stdout:4/943: dread d6/d24/db0/d81/fb7 [0,4194304] 0 2021-11-03T17:15:00.291 INFO:tasks.workunit.client.0.smithi027.stdout:4/944: dread - d6/d9/d16/d36/d99/f112 zero size 2021-11-03T17:15:00.291 INFO:tasks.workunit.client.0.smithi027.stdout:9/932: dread db/d1b/d2b/f108 [0,4194304] 0 2021-11-03T17:15:00.292 INFO:tasks.workunit.client.0.smithi027.stdout:9/933: stat db/d1b/d1c/d1d/d40/d60/lfc 0 2021-11-03T17:15:00.298 INFO:tasks.workunit.client.0.smithi027.stdout:9/934: write db/d1b/d2b/d5e/daa/f109 [545229,56423] 0 2021-11-03T17:15:00.300 INFO:tasks.workunit.client.0.smithi027.stdout:9/935: symlink db/d1b/d2b/d41/dad/dbb/d11b/l15d 0 2021-11-03T17:15:00.309 INFO:tasks.workunit.client.0.smithi027.stdout:9/936: dread db/d1b/f51 [0,4194304] 0 2021-11-03T17:15:00.309 INFO:tasks.workunit.client.0.smithi027.stdout:4/945: dwrite d6/d9/d16/d36/d70/d3e/d58/f60 [0,4194304] 0 2021-11-03T17:15:00.310 INFO:tasks.workunit.client.0.smithi027.stdout:4/946: chown d6/d24/dbe/de5/f12a 3709 1 2021-11-03T17:15:00.312 INFO:tasks.workunit.client.0.smithi027.stdout:9/937: creat db/d1b/d2b/d41/dad/dbb/d12a/f15e x:0 0 0 2021-11-03T17:15:00.313 INFO:tasks.workunit.client.0.smithi027.stdout:9/938: dread db/d1b/d2b/d41/d95/f117 [0,4194304] 0 2021-11-03T17:15:00.314 INFO:tasks.workunit.client.0.smithi027.stdout:4/947: mkdir d6/d24/d6d/de6/d120/d13c 0 2021-11-03T17:15:00.318 INFO:tasks.workunit.client.0.smithi027.stdout:4/948: symlink d6/d9/d16/d36/l13d 0 2021-11-03T17:15:00.320 INFO:tasks.workunit.client.0.smithi027.stdout:9/939: rename db/d1b/d2b/dc7/lf6 to db/d1b/d106/l15f 0 2021-11-03T17:15:00.321 INFO:tasks.workunit.client.0.smithi027.stdout:9/940: write db/d1b/d2b/d41/d95/fce [3859353,43381] 0 2021-11-03T17:15:00.322 INFO:tasks.workunit.client.0.smithi027.stdout:4/949: symlink d6/d9/d16/d36/d57/d5a/d63/l13e 0 2021-11-03T17:15:00.325 INFO:tasks.workunit.client.0.smithi027.stdout:9/941: rename db/d1b/d2b/d41/d6c/f93 to db/d1b/d2b/d41/dad/dbb/d14a/f160 0 2021-11-03T17:15:00.326 INFO:tasks.workunit.client.0.smithi027.stdout:9/942: dread - db/d1b/d2b/d5e/daa/ff1 zero size 2021-11-03T17:15:00.327 INFO:tasks.workunit.client.0.smithi027.stdout:9/943: write db/fe8 [1308119,4881] 0 2021-11-03T17:15:00.327 INFO:tasks.workunit.client.0.smithi027.stdout:9/944: stat db/d1b/d2b/d41/dad/ff3 0 2021-11-03T17:15:00.327 INFO:tasks.workunit.client.0.smithi027.stdout:9/945: stat db/d1b/d106/f11d 0 2021-11-03T17:15:00.328 INFO:tasks.workunit.client.0.smithi027.stdout:9/946: creat db/d1b/d106/d12b/f161 x:0 0 0 2021-11-03T17:15:00.332 INFO:tasks.workunit.client.0.smithi027.stdout:9/947: rename db/d1b/d1c/d1d/d40/f70 to db/d1b/d1c/f162 0 2021-11-03T17:15:00.334 INFO:tasks.workunit.client.0.smithi027.stdout:9/948: unlink db/d1b/d2b/l141 0 2021-11-03T17:15:00.336 INFO:tasks.workunit.client.0.smithi027.stdout:4/950: dwrite d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d41/f88 [0,4194304] 0 2021-11-03T17:15:00.346 INFO:tasks.workunit.client.0.smithi027.stdout:4/951: dread d6/d9/d16/d36/d70/d3e/f6c [4194304,4194304] 0 2021-11-03T17:15:00.346 INFO:tasks.workunit.client.0.smithi027.stdout:4/952: stat d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/f37 0 2021-11-03T17:15:00.347 INFO:tasks.workunit.client.0.smithi027.stdout:4/953: fdatasync d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/f69 0 2021-11-03T17:15:00.347 INFO:tasks.workunit.client.0.smithi027.stdout:4/954: dread - d6/d9/d16/d36/d70/d3e/de9/f107 zero size 2021-11-03T17:15:00.351 INFO:tasks.workunit.client.0.smithi027.stdout:9/949: dwrite db/d1b/d2b/d41/dad/dbb/d12a/f15e [0,4194304] 0 2021-11-03T17:15:00.354 INFO:tasks.workunit.client.0.smithi027.stdout:9/950: creat db/d1b/d1c/f163 x:0 0 0 2021-11-03T17:15:00.355 INFO:tasks.workunit.client.0.smithi027.stdout:4/955: dread d6/d9/d16/d36/d70/f71 [0,4194304] 0 2021-11-03T17:15:00.356 INFO:tasks.workunit.client.0.smithi027.stdout:9/951: symlink db/d1b/d2b/d41/d95/l164 0 2021-11-03T17:15:00.356 INFO:tasks.workunit.client.0.smithi027.stdout:9/952: fsync db/f16 0 2021-11-03T17:15:00.357 INFO:tasks.workunit.client.0.smithi027.stdout:4/956: rmdir d6/d9/d16/d36/d70/dee 39 2021-11-03T17:15:00.364 INFO:tasks.workunit.client.0.smithi027.stdout:9/953: mknod db/d1b/d8e/c165 1 2021-11-03T17:15:00.366 INFO:tasks.workunit.client.0.smithi027.stdout:4/957: symlink d6/d9/d16/d36/d57/d12f/l13f 0 2021-11-03T17:15:00.367 INFO:tasks.workunit.client.0.smithi027.stdout:4/958: symlink d6/d24/d6d/de6/d120/d13c/l140 0 2021-11-03T17:15:00.368 INFO:tasks.workunit.client.0.smithi027.stdout:9/954: dread db/d1b/d1c/f25 [0,4194304] 0 2021-11-03T17:15:00.370 INFO:tasks.workunit.client.0.smithi027.stdout:9/955: write db/d1b/d2b/d41/fb8 [2219468,100306] 0 2021-11-03T17:15:00.389 INFO:tasks.workunit.client.0.smithi027.stdout:4/959: dwrite d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/f69 [4194304,4194304] 0 2021-11-03T17:15:00.389 INFO:tasks.workunit.client.0.smithi027.stdout:4/960: stat d6/d24/l7c 0 2021-11-03T17:15:00.403 INFO:tasks.workunit.client.0.smithi027.stdout:4/961: truncate d6/d9/d16/d36/d70/f137 1011971 0 2021-11-03T17:15:00.637 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:00 smithi027 conmon[32322]: cephadm 2021-11-03T17:14:59.588069+0000 mgr.smithi115.tcyodz (mgr. 2021-11-03T17:15:00.638 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:00 smithi027 conmon[32322]: 24395) 36 : cephadm [INF] poll took 12800.515 ms: 1 events 2021-11-03T17:15:00.638 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:00 smithi027 conmon[32322]: audit 2021-11-03T17:14:59.622906+0000 mon.smithi027 (mon.0) 645 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' 2021-11-03T17:15:00.638 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:00 smithi027 conmon[32322]: cluster 2021-11-03T17:14:59.624417+0000 mgr.smithi115.tcyodz (mgr.24395) 37 : cluster [DBG] 2021-11-03T17:15:00.639 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:00 smithi027 conmon[32322]: pgmap v25: 65 pgs: 65 active+clean; 3.2 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 21 MiB/s rd, 40 MiB/s wr, 111 op/s 2021-11-03T17:15:00.639 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:00 smithi027 conmon[32322]: cluster 2021- 2021-11-03T17:15:00.639 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:00 smithi027 conmon[32322]: 11-03T17:14:59.624606+0000 mgr.smithi115.tcyodz (mgr.24395) 38 : cluster [DBG] 2021-11-03T17:15:00.639 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:00 smithi027 conmon[32322]: pgmap v26: 65 pgs: 65 active+clean; 3.2 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 21 MiB/s rd, 42 MiB/s wr, 101 op/s 2021-11-03T17:15:00.639 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:00 smithi027 conmon[32322]: audit 2021-11-03T17:15:00.640 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:00 smithi027 conmon[32322]: 2021-11-03T17:14:59.628425+0000 mon.smithi027 (mon.0) 646 : audit 2021-11-03T17:15:00.640 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:00 smithi027 conmon[32322]: [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' 2021-11-03T17:15:00.640 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:00 smithi027 conmon[32322]: audit 2021-11-03T17:14:59.632995+0000 mon.smithi115 2021-11-03T17:15:00.640 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:00 smithi027 conmon[32322]: (mon.1) 58 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:15:00.641 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:00 smithi027 conmon[32322]: cephadm 2021-11-03T17:14:59.634311+0000 mgr.smithi115.tcyodz 2021-11-03T17:15:00.641 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:00 smithi027 conmon[32322]: (mgr.24395) 39 : cephadm [INF] Upgrade: Need to upgrade myself (mgr.smithi115.tcyodz) 2021-11-03T17:15:00.641 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:00 smithi027 conmon[32322]: cephadm 2021-11-03T17 2021-11-03T17:15:00.641 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:00 smithi027 conmon[32322]: :14:59.636672+0000 mgr.smithi115.tcyodz (mgr.24395) 40 : cephadm 2021-11-03T17:15:00.641 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:00 smithi027 conmon[32322]: [INF] Failing over to other MGR 2021-11-03T17:15:00.642 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:00 smithi027 conmon[32322]: audit 2021-11-03T17:14:59.636950+0000 2021-11-03T17:15:00.642 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:00 smithi027 conmon[32322]: mon.smithi115 (mon.1) 59 : audit [INF] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "mgr fail", "who": "smithi115.tcyodz"}]: dispatch 2021-11-03T17:15:00.642 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:00 smithi027 conmon[32322]: audit 2021-11-03T17: 2021-11-03T17:15:00.642 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:00 smithi027 conmon[32322]: 14:59.637337+0000 mon.smithi027 (mon.0) 647 : audit [INF] 2021-11-03T17:15:00.643 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:00 smithi027 conmon[32322]: from='mgr.24395 ' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "mgr fail", "who": "smithi115.tcyodz"}]: dispatch 2021-11-03T17:15:00.643 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:00 smithi027 conmon[32322]: cluster 2021-11-03T17:15:00.643 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:00 smithi027 conmon[32322]: 2021-11-03T17:14:59.640676+0000 mon.smithi027 (mon.0) 648 : cluster 2021-11-03T17:15:00.643 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:00 smithi027 conmon[32322]: [DBG] osdmap e50: 6 total, 6 up, 6 in 2021-11-03T17:15:00.689 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:00 smithi115 conmon[34670]: cephadm 2021-11-03T17:14:59.588069+0000 mgr.smithi115.tcyodz (mgr. 2021-11-03T17:15:00.690 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:00 smithi115 conmon[34670]: 24395) 36 : cephadm [INF] poll took 12800.515 ms: 1 events 2021-11-03T17:15:00.690 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:00 smithi115 conmon[34670]: audit 2021-11-03T17:14:59.622906+0000 mon.smithi027 (mon.0) 2021-11-03T17:15:00.690 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:00 smithi115 conmon[34670]: 645 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' 2021-11-03T17:15:00.690 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:00 smithi115 conmon[34670]: cluster 2021-11-03T17:14:59. 2021-11-03T17:15:00.691 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:00 smithi115 conmon[34670]: 624417+0000 mgr.smithi115.tcyodz (mgr.24395) 37 : cluster [DBG] 2021-11-03T17:15:00.691 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:00 smithi115 conmon[34670]: pgmap v25: 65 pgs: 65 active+clean; 3.2 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 21 MiB/s rd, 40 MiB/s wr, 111 op/s 2021-11-03T17:15:00.691 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:00 smithi115 conmon[34670]: cluster 2021-11-03T17: 2021-11-03T17:15:00.692 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:00 smithi115 conmon[34670]: 14:59.624606+0000 mgr.smithi115.tcyodz (mgr.24395) 2021-11-03T17:15:00.692 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:00 smithi115 conmon[34670]: 38 : cluster [DBG] pgmap v26: 65 pgs: 65 active+clean; 3.2 GiB data, 9.6 GiB used, 527 GiB / 536 GiB avail; 21 MiB/s rd, 42 MiB/s wr, 101 op/s 2021-11-03T17:15:00.692 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:00 smithi115 conmon[34670]: audit 2021- 2021-11-03T17:15:00.692 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:00 smithi115 conmon[34670]: 11-03T17:14:59.628425+0000 mon.smithi027 2021-11-03T17:15:00.693 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:00 smithi115 conmon[34670]: (mon.0) 646 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' 2021-11-03T17:15:00.693 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:00 smithi115 conmon[34670]: audit 2021-11- 2021-11-03T17:15:00.693 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:00 smithi115 conmon[34670]: 03T17:14:59.632995+0000 mon.smithi115 (mon.1 2021-11-03T17:15:00.693 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:00 smithi115 conmon[34670]: ) 58 : audit [DBG] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:15:00.693 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:00 smithi115 conmon[34670]: cephadm 2021 2021-11-03T17:15:00.694 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:00 smithi115 conmon[34670]: -11-03T17:14:59.634311+0000 2021-11-03T17:15:00.694 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:00 smithi115 conmon[34670]: mgr.smithi115.tcyodz (mgr.24395) 39 : cephadm [INF] 2021-11-03T17:15:00.694 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:00 smithi115 conmon[34670]: Upgrade: Need to upgrade myself (mgr.smithi115.tcyodz) 2021-11-03T17:15:00.695 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:00 smithi115 conmon[34670]: cephadm 2021-11-03T17 2021-11-03T17:15:00.695 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:00 smithi115 conmon[34670]: :14:59.636672+0000 mgr.smithi115.tcyodz (mgr.24395) 2021-11-03T17:15:00.695 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:00 smithi115 conmon[34670]: 40 : cephadm [INF] Failing over to other MGR 2021-11-03T17:15:00.695 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:00 smithi115 conmon[34670]: audit 2021 2021-11-03T17:15:00.696 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:00 smithi115 conmon[34670]: -11-03T17:14:59.636950+0000 2021-11-03T17:15:00.696 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:00 smithi115 conmon[34670]: mon.smithi115 (mon.1) 59 : audit [INF] from='mgr.24395 172.21.15.115:0/1235451268' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "mgr fail", "who": "smithi115.tcyodz"}]: dispatch 2021-11-03T17:15:00.696 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:00 smithi115 conmon[34670]: 2021-11-03T17:15:00.697 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:00 smithi115 conmon[34670]: audit 2021-11-03T17:14: 2021-11-03T17:15:00.697 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:00 smithi115 conmon[34670]: 59.637337+0000 mon.smithi027 (mon.0) 647 2021-11-03T17:15:00.697 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:00 smithi115 conmon[34670]: : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "mgr fail", "who": "smithi115.tcyodz"}]: dispatch 2021-11-03T17:15:00.697 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:00 smithi115 conmon[34670]: cluster 2021-11- 2021-11-03T17:15:00.697 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:00 smithi115 conmon[34670]: 03T17:14:59.640676+0000 mon.smithi027 ( 2021-11-03T17:15:00.698 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:00 smithi115 conmon[34670]: mon.0) 648 : cluster [DBG] osdmap e50: 6 total, 6 up, 6 in 2021-11-03T17:15:01.448 INFO:tasks.workunit.client.0.smithi027.stdout:7/662: sync 2021-11-03T17:15:01.456 INFO:tasks.workunit.client.0.smithi027.stdout:7/663: dread d0/d2/d10/d1e/d20/d74/f9d [0,4194304] 0 2021-11-03T17:15:01.464 INFO:tasks.workunit.client.0.smithi027.stdout:7/664: write d0/d2/d1f/f46 [5026864,108770] 0 2021-11-03T17:15:01.467 INFO:tasks.workunit.client.0.smithi027.stdout:7/665: mkdir d0/d2d/d2f/dae/dd8 0 2021-11-03T17:15:01.467 INFO:tasks.workunit.client.0.smithi027.stdout:7/666: write d0/d2/d1c/d4d/d6e/fc1 [779577,25731] 0 2021-11-03T17:15:01.774 INFO:tasks.workunit.client.0.smithi027.stdout:6/854: sync 2021-11-03T17:15:01.775 INFO:tasks.workunit.client.0.smithi027.stdout:6/855: truncate dc/d82/d57/dad/dae/fd7 5037752 0 2021-11-03T17:15:01.776 INFO:tasks.workunit.client.0.smithi027.stdout:6/856: mknod dc/d16/d1b/d24/c10d 1 2021-11-03T17:15:01.776 INFO:tasks.workunit.client.0.smithi027.stdout:6/857: link dc/d16/d1b/d3c/d51/ldb dc/d16/d1b/d3c/d51/df9/de7/l10e 0 2021-11-03T17:15:01.778 INFO:tasks.workunit.client.0.smithi027.stdout:6/858: stat dc/d82/d57/d96/lcb 0 2021-11-03T17:15:01.778 INFO:tasks.workunit.client.0.smithi027.stdout:6/859: mknod dc/d82/d57/dad/dae/db1/dd1/c10f 1 2021-11-03T17:15:01.778 INFO:tasks.workunit.client.0.smithi027.stdout:6/860: stat dc/d16/l3d 0 2021-11-03T17:15:01.779 INFO:tasks.workunit.client.0.smithi027.stdout:6/861: mknod dc/d16/d30/c110 1 2021-11-03T17:15:01.812 INFO:tasks.workunit.client.0.smithi027.stdout:6/862: creat dc/d16/d1b/d3c/ddc/f111 x:0 0 0 2021-11-03T17:15:01.814 INFO:tasks.workunit.client.0.smithi027.stdout:6/863: symlink dc/d16/d1b/d3c/d51/df9/de7/l112 0 2021-11-03T17:15:01.815 INFO:tasks.workunit.client.0.smithi027.stdout:6/864: mkdir dc/d82/d57/dad/d113 0 2021-11-03T17:15:01.816 INFO:tasks.workunit.client.0.smithi027.stdout:6/865: getdents dc/d16/d1b/d3c/d51/d5f 0 2021-11-03T17:15:01.818 INFO:tasks.workunit.client.0.smithi027.stdout:6/866: creat dc/d82/d57/dad/dae/db1/d34/d89/f114 x:0 0 0 2021-11-03T17:15:01.821 INFO:tasks.workunit.client.0.smithi027.stdout:6/867: dread dc/d16/d1b/d3c/d51/f58 [0,4194304] 0 2021-11-03T17:15:01.821 INFO:tasks.workunit.client.0.smithi027.stdout:6/868: chown dc/d82/d87 210059027 1 2021-11-03T17:15:01.822 INFO:tasks.workunit.client.0.smithi027.stdout:6/869: creat dc/d16/d1b/d3c/d51/d5f/f115 x:0 0 0 2021-11-03T17:15:01.823 INFO:tasks.workunit.client.0.smithi027.stdout:6/870: stat dc/d16/d1b/d3c/d51 0 2021-11-03T17:15:01.833 INFO:tasks.workunit.client.0.smithi027.stdout:6/871: write dc/d16/d1b/d24/f2a [4719742,122586] 0 2021-11-03T17:15:01.835 INFO:tasks.workunit.client.0.smithi027.stdout:6/872: unlink dc/d82/d57/d9b/fd0 0 2021-11-03T17:15:01.844 INFO:tasks.workunit.client.0.smithi027.stdout:6/873: write dc/d37/dcc/f93 [3142037,57166] 0 2021-11-03T17:15:01.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:01 smithi115 conmon[34670]: audit 2021-11-03T17:15:00.634196+0000 mon.smithi027 (mon.0) 649 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' cmd='[{"prefix": "mgr fail", "who": "smithi115.tcyodz"}]': finished 2021-11-03T17:15:01.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:01 smithi115 conmon[34670]: cluster 2021-11-03T17:15:00.634259+0000 mon.smithi027 (mon.0) 650 : cluster [DBG] mgrmap e25: smithi027.ufhhvb(active, starting, since 0.996295s) 2021-11-03T17:15:01.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:01 smithi115 conmon[34670]: audit 2021-11-03T17:15:00.638010+0000 mon.smithi027 (mon.0) 651 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2021-11-03T17:15:01.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:01 smithi115 conmon[34670]: audit 2021-11-03T17:15:00.638222+0000 mon.smithi027 (mon.0) 652 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2021-11-03T17:15:01.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:01 smithi115 conmon[34670]: audit 2021-11-03T17:15:00.638585+0000 mon.smithi027 (mon.0) 653 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.jdwohn"}]: dispatch 2021-11-03T17:15:01.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:01 smithi115 conmon[34670]: audit 2021-11-03T17:15:00.638809+0000 mon.smithi027 (mon.0) 654 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.fksixo"}]: dispatch 2021-11-03T17:15:01.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:01 smithi115 conmon[34670]: audit 2021-11-03T17:15:00.639028+0000 mon.smithi027 (mon.0) 655 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.semywj"}]: dispatch 2021-11-03T17:15:01.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:01 smithi115 conmon[34670]: audit 2021-11-03T17:15:00.639271+0000 mon.smithi027 (mon.0) 656 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.zsfgnv"}]: dispatch 2021-11-03T17:15:01.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:01 smithi115 conmon[34670]: audit 2021-11-03T17:15:00.639514+0000 mon.smithi027 (mon.0) 657 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mgr metadata", "who": "smithi027.ufhhvb", "id": "smithi027.ufhhvb"}]: dispatch 2021-11-03T17:15:01.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:01 smithi115 conmon[34670]: audit 2021-11-03T17:15:00.639706+0000 mon.smithi027 (mon.0) 658 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-11-03T17:15:01.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:01 smithi115 conmon[34670]: audit 2021-11-03T17:15:00.639942+0000 mon.smithi027 (mon.0) 659 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-11-03T17:15:01.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:01 smithi115 conmon[34670]: audit 2021-11-03T17:15:00.640166+0000 mon.smithi027 (mon.0) 660 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-11-03T17:15:01.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:01 smithi115 conmon[34670]: audit 2021-11-03T17:15:00.640381+0000 mon.smithi027 (mon.0) 661 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-11-03T17:15:01.991 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:01 smithi115 conmon[34670]: audit 2021- 2021-11-03T17:15:01.991 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:01 smithi115 conmon[34670]: 11-03T17:15:00.640606+0000 mon.smithi027 (mon.0) 662 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-11-03T17:15:01.992 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:01 smithi115 conmon[34670]: audit 2021-11-03T17:15:00.640825+0000 mon.smithi027 (mon.0) 663 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-11-03T17:15:01.992 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:01 smithi115 conmon[34670]: audit 2021-11-03T17:15:00.641066+0000 mon.smithi027 (mon.0) 664 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata"}]: dispatch 2021-11-03T17:15:01.992 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:01 smithi115 conmon[34670]: audit 2021-11-03T17:15:00.641337+0000 mon.smithi027 (mon.0) 665 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata"}]: dispatch 2021-11-03T17:15:01.993 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:01 smithi115 conmon[34670]: audit 2021-11-03T17:15:00.641732+0000 mon.smithi027 (mon.0) 666 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata"}]: dispatch 2021-11-03T17:15:01.993 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:01 smithi115 conmon[34670]: cluster 2021-11-03T17:15:00.661667+0000 mon.smithi027 (mon.0) 667 : cluster [INF] Manager daemon smithi027.ufhhvb is now available 2021-11-03T17:15:01.994 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:01 smithi115 conmon[34670]: cephadm 2021-11-03T17:15:00.676076+0000 mgr.smithi027.ufhhvb (mgr.14622) 1 : cephadm [INF] inventory: adjusted host smithi115 addr 'smithi115' -> '172.21.15.115' 2021-11-03T17:15:01.994 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:01 smithi115 conmon[34670]: audit 2021-11-03T17:15:00.680471+0000 mon.smithi027 (mon.0) 668 : audit [INF] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:15:01.994 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:01 smithi115 conmon[34670]: audit 2021-11-03T17:15:00.691047+0000 mon.smithi027 (mon.0) 669 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:15:01.995 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:01 smithi115 conmon[34670]: audit 2021-11-03T17:15:00.697559+0000 mon.smithi027 (mon.0) 670 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:15:01.995 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:01 smithi115 conmon[34670]: audit 2021-11-03T17:15:00.720791+0000 mon.smithi027 (mon.0) 671 : audit [INF] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:15:01.996 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:01 smithi115 conmon[34670]: audit 2021-11-03T17:15:00.727368+0000 mon.smithi027 (mon.0) 672 : audit [INF] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:15:01.996 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:01 smithi115 conmon[34670]: cephadm 2021-11-03T17:15:00.976842+0000 mgr.smithi027.ufhhvb (mgr.14622) 2 : cephadm [WRN] Executing result=None created at /lib/python3.6/site-packages/asyncssh/connection.py:841> took 0.107 seconds 2021-11-03T17:15:02.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:01 smithi027 conmon[32322]: audit 2021-11-03T17:15:00.634196+0000 mon.smithi027 (mon.0) 649 : audit [INF] from='mgr.24395 ' entity='mgr.smithi115.tcyodz' cmd='[{"prefix": "mgr fail", "who": "smithi115.tcyodz"}]': finished 2021-11-03T17:15:02.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:01 smithi027 conmon[32322]: cluster 2021-11-03T17:15:00.634259+0000 mon.smithi027 (mon.0) 650 : cluster [DBG] mgrmap e25: smithi027.ufhhvb(active, starting, since 0.996295s) 2021-11-03T17:15:02.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:01 smithi027 conmon[32322]: audit 2021-11-03T17:15:00.638010+0000 mon.smithi027 (mon.0) 651 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2021-11-03T17:15:02.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:01 smithi027 conmon[32322]: audit 2021-11-03T17:15:00.638222+0000 mon.smithi027 (mon.0) 652 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2021-11-03T17:15:02.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:01 smithi027 conmon[32322]: audit 2021-11-03T17:15:00.638585+0000 mon.smithi027 (mon.0) 653 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.jdwohn"}]: dispatch 2021-11-03T17:15:02.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:01 smithi027 conmon[32322]: audit 2021-11-03T17:15:00. 2021-11-03T17:15:02.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:01 smithi027 conmon[32322]: 638809+0000 mon.smithi027 (mon.0) 654 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.fksixo"}]: dispatch 2021-11-03T17:15:02.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:01 smithi027 conmon[32322]: audit 2021-11-03T17:15:00.639028+0000 mon.smithi027 (mon.0) 655 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.semywj"}]: dispatch 2021-11-03T17:15:02.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:01 smithi027 conmon[32322]: audit 2021-11-03T17:15:00.639271+0000 mon.smithi027 (mon.0) 656 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.zsfgnv"}]: dispatch 2021-11-03T17:15:02.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:01 smithi027 conmon[32322]: audit 2021-11-03T17:15:00.639514+0000 mon.smithi027 (mon.0) 657 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mgr metadata", "who": "smithi027.ufhhvb", "id": "smithi027.ufhhvb"}]: dispatch 2021-11-03T17:15:02.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:01 smithi027 conmon[32322]: audit 2021-11-03T17:15:00.639706+0000 mon.smithi027 (mon.0) 658 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-11-03T17:15:02.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:01 smithi027 conmon[32322]: audit 2021-11-03T17:15:00.639942+0000 mon.smithi027 (mon.0) 659 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-11-03T17:15:02.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:01 smithi027 conmon[32322]: audit 2021-11-03T17:15:00.640166+0000 mon.smithi027 (mon.0) 660 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-11-03T17:15:02.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:01 smithi027 conmon[32322]: audit 2021-11-03T17:15:00.640381+0000 mon.smithi027 (mon.0) 661 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-11-03T17:15:02.026 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:01 smithi027 conmon[32322]: audit 2021-11-03T17:15:00.640606+0000 mon.smithi027 (mon.0) 662 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-11-03T17:15:02.026 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:01 smithi027 conmon[32322]: audit 2021-11-03T17:15:00.640825+0000 mon.smithi027 (mon.0) 663 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-11-03T17:15:02.026 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:01 smithi027 conmon[32322]: audit 2021-11-03T17:15:00.641066+0000 mon.smithi027 (mon.0) 664 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata"}]: dispatch 2021-11-03T17:15:02.026 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:01 smithi027 conmon[32322]: audit 2021-11-03T17:15:00.641337+0000 mon.smithi027 (mon.0) 665 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata"}]: dispatch 2021-11-03T17:15:02.027 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:01 smithi027 conmon[32322]: audit 2021-11-03T17:15:00.641732+0000 mon.smithi027 (mon.0) 666 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata"}]: dispatch 2021-11-03T17:15:02.027 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:01 smithi027 conmon[32322]: cluster 2021-11-03T17:15:00.661667+0000 mon.smithi027 (mon.0) 667 : cluster [INF] Manager daemon smithi027.ufhhvb is now available 2021-11-03T17:15:02.027 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:01 smithi027 conmon[32322]: cephadm 2021-11-03T17:15:00.676076+0000 mgr.smithi027.ufhhvb (mgr.14622) 1 : cephadm [INF] inventory: adjusted host smithi115 addr 'smithi115' -> '172.21.15.115' 2021-11-03T17:15:02.028 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:01 smithi027 conmon[32322]: audit 2021-11-03T17:15:00.680471+0000 mon.smithi027 (mon.0) 668 : audit [INF] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:15:02.028 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:01 smithi027 conmon[32322]: audit 2021-11-03T17:15:00.691047+0000 mon.smithi027 (mon.0) 669 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:15:02.028 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:01 smithi027 conmon[32322]: audit 2021-11-03T17:15:00.697559+0000 mon.smithi027 (mon.0) 670 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:15:02.028 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:01 smithi027 conmon[32322]: audit 2021-11-03T17:15:00.720791+0000 mon.smithi027 (mon.0) 671 : audit [INF] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:15:02.028 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:01 smithi027 conmon[32322]: audit 2021-11-03T17:15:00.727368+0000 mon.smithi027 (mon.0) 672 : audit [INF] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:15:02.029 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:01 smithi027 conmon[32322]: cephadm 2021-11-03T17:15:00.976842+0000 mgr.smithi027.ufhhvb ( 2021-11-03T17:15:02.029 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:01 smithi027 conmon[32322]: mgr.14622) 2 : cephadm [WRN] Executing result=None created at /lib/python3.6/site-packages/asyncssh/connection.py:841> took 0.107 seconds 2021-11-03T17:15:02.030 INFO:tasks.workunit.client.0.smithi027.stdout:9/956: sync 2021-11-03T17:15:02.031 INFO:tasks.workunit.client.0.smithi027.stdout:4/962: sync 2021-11-03T17:15:02.031 INFO:tasks.workunit.client.0.smithi027.stdout:4/963: creat d6/d9/d16/dc3/f141 x:0 0 0 2021-11-03T17:15:02.033 INFO:tasks.workunit.client.0.smithi027.stdout:9/957: rename db/d1b/d2b/d41/d95/l10c to db/d1b/d106/d13c/l166 0 2021-11-03T17:15:02.034 INFO:tasks.workunit.client.0.smithi027.stdout:9/958: mknod db/d1b/d106/d12b/c167 1 2021-11-03T17:15:02.036 INFO:tasks.workunit.client.0.smithi027.stdout:4/964: unlink d6/d24/l33 0 2021-11-03T17:15:02.038 INFO:tasks.workunit.client.0.smithi027.stdout:4/965: fdatasync d6/d9/d16/d36/d99/db9/f96 0 2021-11-03T17:15:02.039 INFO:tasks.workunit.client.0.smithi027.stdout:4/966: dread - d6/d24/de3/d9f/dd7/f12d zero size 2021-11-03T17:15:02.042 INFO:tasks.workunit.client.0.smithi027.stdout:4/967: symlink d6/d9/d16/d36/d70/l142 0 2021-11-03T17:15:02.045 INFO:tasks.workunit.client.0.smithi027.stdout:4/968: creat d6/d24/dbe/de5/f143 x:0 0 0 2021-11-03T17:15:02.047 INFO:tasks.workunit.client.0.smithi027.stdout:4/969: creat d6/d9/d16/d36/d70/d3e/f144 x:0 0 0 2021-11-03T17:15:02.056 INFO:tasks.workunit.client.0.smithi027.stdout:4/970: symlink d6/d24/de3/d102/d119/l145 0 2021-11-03T17:15:02.059 INFO:tasks.workunit.client.0.smithi027.stdout:4/971: symlink d6/d24/d6d/de6/d120/d13c/l146 0 2021-11-03T17:15:02.060 INFO:tasks.workunit.client.0.smithi027.stdout:4/972: read d6/d9/d16/d36/d57/d5a/d63/fd9 [620853,120298] 0 2021-11-03T17:15:02.066 INFO:tasks.workunit.client.0.smithi027.stdout:4/973: creat d6/d24/de3/d102/d119/f147 x:0 0 0 2021-11-03T17:15:02.069 INFO:tasks.workunit.client.0.smithi027.stdout:4/974: getdents d6/d9/d16/d36/d70/d3e/de9 0 2021-11-03T17:15:02.070 INFO:tasks.workunit.client.0.smithi027.stdout:4/975: fdatasync d6/d9/d16/d36/d99/db9/fd6 0 2021-11-03T17:15:02.072 INFO:tasks.workunit.client.0.smithi027.stdout:4/976: truncate d6/d9/d16/d36/d70/f71 559107 0 2021-11-03T17:15:02.075 INFO:tasks.workunit.client.0.smithi027.stdout:4/977: symlink d6/d9/d16/d36/dc1/l148 0 2021-11-03T17:15:02.078 INFO:tasks.workunit.client.0.smithi027.stdout:9/959: dwrite db/d1b/d2b/d41/fb9 [0,4194304] 0 2021-11-03T17:15:02.078 INFO:tasks.workunit.client.0.smithi027.stdout:9/960: chown db/d1b/d2b/d41/d95/deb/f118 0 1 2021-11-03T17:15:02.078 INFO:tasks.workunit.client.0.smithi027.stdout:9/961: mknod db/d1b/d1c/d10f/d156/c168 1 2021-11-03T17:15:02.079 INFO:tasks.workunit.client.0.smithi027.stdout:9/962: write db/d1b/d8c/dcd/ffe [1393153,33586] 0 2021-11-03T17:15:02.083 INFO:tasks.workunit.client.0.smithi027.stdout:9/963: truncate db/f16 6473031 0 2021-11-03T17:15:02.085 INFO:tasks.workunit.client.0.smithi027.stdout:9/964: mknod db/d1b/d2b/d5e/c169 1 2021-11-03T17:15:02.085 INFO:tasks.workunit.client.0.smithi027.stdout:9/965: truncate db/d1b/d1c/d1d/d12d/f13d 861387 0 2021-11-03T17:15:02.086 INFO:tasks.workunit.client.0.smithi027.stdout:4/978: mkdir d6/d24/db0/d149 0 2021-11-03T17:15:02.094 INFO:tasks.workunit.client.0.smithi027.stdout:4/979: rename d6/d24/d6d/de6 to d6/d24/d6d/d14a 0 2021-11-03T17:15:02.095 INFO:tasks.workunit.client.0.smithi027.stdout:4/980: truncate d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d9a/f124 97020 0 2021-11-03T17:15:02.096 INFO:tasks.workunit.client.0.smithi027.stdout:4/981: fsync d6/d9/d16/d36/d57/d5a/d63/ff6 0 2021-11-03T17:15:02.096 INFO:tasks.workunit.client.0.smithi027.stdout:4/982: chown d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/fc2 2604 1 2021-11-03T17:15:02.097 INFO:tasks.workunit.client.0.smithi027.stdout:4/983: write d6/d9/d16/d36/d97/faa [116455,6255] 0 2021-11-03T17:15:02.097 INFO:tasks.workunit.client.0.smithi027.stdout:4/984: read - d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d41/f138 zero size 2021-11-03T17:15:02.100 INFO:tasks.workunit.client.0.smithi027.stdout:9/966: dread db/d1b/d8c/dcd/f113 [4194304,4194304] 0 2021-11-03T17:15:02.100 INFO:tasks.workunit.client.0.smithi027.stdout:9/967: chown db/d1b/d1c/f144 22983 1 2021-11-03T17:15:02.103 INFO:tasks.workunit.client.0.smithi027.stdout:4/985: mkdir d6/d9/d16/d36/dc1/d110/d14b 0 2021-11-03T17:15:02.123 INFO:tasks.workunit.client.0.smithi027.stdout:4/986: dread d6/d9/d16/d36/d70/d3e/f6c [0,4194304] 0 2021-11-03T17:15:02.125 INFO:tasks.workunit.client.0.smithi027.stdout:4/987: creat d6/d24/d6d/d14a/d120/d13c/f14c x:0 0 0 2021-11-03T17:15:02.135 INFO:tasks.workunit.client.0.smithi027.stdout:4/988: dread d6/d24/db0/f8b [0,4194304] 0 2021-11-03T17:15:02.140 INFO:tasks.workunit.client.0.smithi027.stdout:4/989: link d6/d9/d16/d36/d97/ff3 d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/f14d 0 2021-11-03T17:15:02.140 INFO:tasks.workunit.client.0.smithi027.stdout:4/990: truncate d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/d9a/faf 898764 0 2021-11-03T17:15:02.142 INFO:tasks.workunit.client.0.smithi027.stdout:4/991: dread d6/d24/de3/d9f/dd7/f103 [0,4194304] 0 2021-11-03T17:15:02.144 INFO:tasks.workunit.client.0.smithi027.stdout:4/992: creat d6/d9/d16/d36/d57/d12f/f14e x:0 0 0 2021-11-03T17:15:02.147 INFO:tasks.workunit.client.0.smithi027.stdout:4/993: fsync d6/d9/d16/d36/d57/d5a/d63/fd4 0 2021-11-03T17:15:02.150 INFO:tasks.workunit.client.0.smithi027.stdout:4/994: write d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/d21/f2c [5560219,59303] 0 2021-11-03T17:15:02.151 INFO:tasks.workunit.client.0.smithi027.stdout:4/995: read d6/d9/d16/d36/d70/d3e/d58/d84/d9b/d98/f7e [621968,109439] 0 2021-11-03T17:15:02.151 INFO:tasks.workunit.client.0.smithi027.stdout:4/996: stat d6/d9/d16/d36/d70/d3e/l43 0 2021-11-03T17:15:02.155 INFO:tasks.workunit.client.0.smithi027.stdout:4/997: unlink d6/d9/d16/d36/d70/d3e/d58/d84/d9b/fcb 0 2021-11-03T17:15:02.159 INFO:tasks.workunit.client.0.smithi027.stdout:4/998: creat d6/d9/d16/d36/f14f x:0 0 0 2021-11-03T17:15:02.162 INFO:tasks.workunit.client.0.smithi027.stdout:4/999: rmdir d6/d9/d16/d36/d57/d5a/d63 39 2021-11-03T17:15:02.939 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: cluster 2021-11-03T17:15:01.645148+0000 2021-11-03T17:15:02.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: mon.smithi027 (mon.0) 673 : cluster [DBG] mgrmap e26: smithi027.ufhhvb(active, since 2s) 2021-11-03T17:15:02.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: cluster 2021-11-03T17:15 2021-11-03T17:15:02.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: :01.654998+0000 mgr.smithi027.ufhhvb (mgr.14622) 3 : cluster [DBG] pgmap v3: 65 pgs: 65 active+clean; 3.3 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail 2021-11-03T17:15:02.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: cluster 2021-11-03 2021-11-03T17:15:02.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: T17:15:01.907040+0000 mon.smithi027 (mon.0 2021-11-03T17:15:02.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: ) 674 : cluster [INF] daemon mds.cephfs.smithi027.fksixo restarted 2021-11-03T17:15:02.943 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: cluster 2021 2021-11-03T17:15:02.943 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: -11-03T17:15:01.910458+0000 mon.smithi027 2021-11-03T17:15:02.943 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: (mon.0) 675 : cluster [DBG] osdmap e51: 6 total, 6 up, 6 in 2021-11-03T17:15:02.944 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: cluster 2021 2021-11-03T17:15:02.944 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: -11-03T17:15:02.059224+0000 mon.smithi027 ( 2021-11-03T17:15:02.944 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: mon.0) 676 : cluster [INF] Standby daemon mds.cephfs.smithi115.semywj assigned to filesystem cephfs as rank 0 2021-11-03T17:15:02.945 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: cluster 2021-11-03T17:15:02.945 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: 2021-11-03T17: 2021-11-03T17:15:02.945 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: 15:02.065297+0000 mon.smithi027 (mon.0) 2021-11-03T17:15:02.946 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: 677 : cluster [DBG] mds.? [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] up:boot 2021-11-03T17:15:02.946 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: cluster 2021-11-03T17:15:02.947 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: 2021-11-03T17:15:02.065379+0000 2021-11-03T17:15:02.947 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: mon.smithi027 (mon.0) 678 : cluster [DBG] 2021-11-03T17:15:02.947 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: fsmap cephfs:2/2 {0=cephfs.smithi115.semywj=up:replay,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:15:02.947 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: audit 2021- 2021-11-03T17:15:02.947 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: 11-03T17:15:02.066614+0000 2021-11-03T17:15:02.948 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: mon.smithi027 (mon.0) 679 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.fksixo"}]: dispatch 2021-11-03T17:15:02.948 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: 2021-11-03T17:15:02.948 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: cephadm 2021-11-03T17: 2021-11-03T17:15:02.948 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: 15:02.295762+0000 mgr.smithi027.ufhhvb (mgr.14622 2021-11-03T17:15:02.949 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: ) 4 : cephadm [INF] Starting cherrypy engine... 2021-11-03T17:15:02.949 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: cephadm 2021-11-03T17:15:02.949 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: 2021-11-03T17:15: 2021-11-03T17:15:02.949 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: 02.295913+0000 mgr.smithi027.ufhhvb (mgr.14622) 5 2021-11-03T17:15:02.949 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: : cephadm [INF] [03/Nov/2021:17:15:02] ENGINE Bus STARTING 2021-11-03T17:15:02.950 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: cephadm 2021-11-03T17:15:02.950 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: 2021-11-03T17:15:02. 2021-11-03T17:15:02.950 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: 413078+0000 mgr.smithi027.ufhhvb (mgr.14622) 6 : cephadm 2021-11-03T17:15:02.950 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: [INF] [03/Nov/2021:17:15:02] ENGINE Serving on https://172.21.15.27:8499 2021-11-03T17:15:02.951 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: cephadm 2021-11-03T17:15:02.951 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: 2021-11-03T17:15:02.413291 2021-11-03T17:15:02.951 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: +0000 mgr.smithi027.ufhhvb (mgr.14622) 7 : cephadm [INF] 2021-11-03T17:15:02.951 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: [03/Nov/2021:17:15:02] ENGINE Bus STARTED 2021-11-03T17:15:02.952 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: cephadm 2021 2021-11-03T17:15:02.952 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: -11-03T17:15:02.413364+0000 2021-11-03T17:15:02.952 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14622) 8 : cephadm [INF] Cherrypy engine started. 2021-11-03T17:15:02.952 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:02 smithi027 conmon[32322]: 2021-11-03T17:15:02.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:02 smithi115 conmon[34670]: cluster 2021-11-03T17:15:01.645148+0000 mon.smithi027 (mon.0 2021-11-03T17:15:02.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:02 smithi115 conmon[34670]: ) 673 : cluster [DBG] mgrmap e26: smithi027.ufhhvb(active, since 2s) 2021-11-03T17:15:02.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:02 smithi115 conmon[34670]: cluster 2021-11-03T17:15:01.654998+0000 mgr.smithi027.ufhhvb (mgr.14622) 3 : cluster [DBG] pgmap v3: 65 pgs: 65 active+clean; 3.3 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail 2021-11-03T17:15:02.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:02 smithi115 conmon[34670]: cluster 2021-11-03T17:15:01.907040+0000 mon.smithi027 (mon.0) 674 : cluster [INF] daemon mds.cephfs.smithi027.fksixo restarted 2021-11-03T17:15:02.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:02 smithi115 conmon[34670]: cluster 2021-11-03T17:15:01.910458+0000 mon.smithi027 (mon.0) 675 : cluster [DBG] osdmap e51: 6 total, 6 up, 6 in 2021-11-03T17:15:02.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:02 smithi115 conmon[34670]: cluster 2021-11-03T17:15:02.059224+0000 mon.smithi027 (mon.0) 676 : cluster [INF] Standby daemon mds.cephfs.smithi115.semywj assigned to filesystem cephfs as rank 0 2021-11-03T17:15:02.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:02 smithi115 conmon[34670]: cluster 2021-11-03T17:15:02.065297+0000 mon.smithi027 (mon.0) 677 : cluster [DBG] mds.? [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] up:boot 2021-11-03T17:15:02.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:02 smithi115 conmon[34670]: cluster 2021-11-03T17:15:02.065379+0000 mon.smithi027 (mon.0) 678 : cluster [DBG] fsmap cephfs:2/2 {0=cephfs.smithi115.semywj=up:replay,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:15:02.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:02 smithi115 conmon[34670]: audit 2021-11-03T17:15:02.066614+0000 mon.smithi027 (mon.0) 679 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.fksixo"}]: dispatch 2021-11-03T17:15:02.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:02 smithi115 conmon[34670]: cephadm 2021-11-03T17:15:02.295762+0000 mgr.smithi027.ufhhvb (mgr.14622) 4 : cephadm [INF] Starting cherrypy engine... 2021-11-03T17:15:02.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:02 smithi115 conmon[34670]: cephadm 2021-11-03T17:15:02.295913+0000 mgr.smithi027.ufhhvb (mgr.14622) 5 : cephadm [INF] [03/Nov/2021:17:15:02] ENGINE Bus STARTING 2021-11-03T17:15:02.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:02 smithi115 conmon[34670]: cephadm 2021-11-03T17:15:02.413078+0000 mgr.smithi027.ufhhvb (mgr.14622) 6 : cephadm [INF] [03/Nov/2021:17:15:02] ENGINE Serving on https://172.21.15.27:8499 2021-11-03T17:15:02.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:02 smithi115 conmon[34670]: cephadm 2021-11-03T17:15:02.413291+0000 mgr.smithi027.ufhhvb (mgr.14622) 7 : cephadm [INF] [03/Nov/2021:17:15:02] ENGINE Bus STARTED 2021-11-03T17:15:02.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:02 smithi115 conmon[34670]: cephadm 2021-11-03T17:15:02.413364+0000 mgr.smithi027.ufhhvb (mgr.14622) 8 : cephadm [INF] Cherrypy engine started. 2021-11-03T17:15:03.230 INFO:tasks.workunit.client.0.smithi027.stdout:7/667: sync 2021-11-03T17:15:03.232 INFO:tasks.workunit.client.0.smithi027.stdout:7/668: symlink d0/d2d/d53/ld9 0 2021-11-03T17:15:03.267 INFO:tasks.workunit.client.0.smithi027.stdout:7/669: rename d0/d2/d5e/d36/d3a/da7/lac to d0/d2/d5e/d36/d3a/da7/lda 0 2021-11-03T17:15:03.268 INFO:tasks.workunit.client.0.smithi027.stdout:7/670: creat d0/d2/d1c/d4d/dcc/fdb x:0 0 0 2021-11-03T17:15:03.269 INFO:tasks.workunit.client.0.smithi027.stdout:7/671: write d0/d2/d10/d1e/d20/f5b [2531377,39469] 0 2021-11-03T17:15:03.272 INFO:tasks.workunit.client.0.smithi027.stdout:7/672: symlink d0/d2/d10/d35/d96/ldc 0 2021-11-03T17:15:03.273 INFO:tasks.workunit.client.0.smithi027.stdout:7/673: write d0/d2/d1c/d4d/d6e/fc1 [34925,57028] 0 2021-11-03T17:15:03.274 INFO:tasks.workunit.client.0.smithi027.stdout:7/674: creat d0/d2/d10/d1e/d20/fdd x:0 0 0 2021-11-03T17:15:03.275 INFO:tasks.workunit.client.0.smithi027.stdout:7/675: chown d0/d2/d10/l72 422 1 2021-11-03T17:15:03.276 INFO:tasks.workunit.client.0.smithi027.stdout:7/676: creat d0/d2d/d2f/dae/dc8/fde x:0 0 0 2021-11-03T17:15:03.278 INFO:tasks.workunit.client.0.smithi027.stdout:7/677: creat d0/db6/fdf x:0 0 0 2021-11-03T17:15:03.309 INFO:tasks.workunit.client.0.smithi027.stdout:7/678: dwrite d0/f4c [0,4194304] 0 2021-11-03T17:15:03.311 INFO:tasks.workunit.client.0.smithi027.stdout:7/679: truncate d0/d2/d10/f95 135713 0 2021-11-03T17:15:03.311 INFO:tasks.workunit.client.0.smithi027.stdout:7/680: stat d0/d2/d10/d35/ld7 0 2021-11-03T17:15:03.312 INFO:tasks.workunit.client.0.smithi027.stdout:7/681: creat d0/d2/d10/d35/d96/fe0 x:0 0 0 2021-11-03T17:15:03.313 INFO:tasks.workunit.client.0.smithi027.stdout:7/682: dread d0/d2d/d66/f98 [0,4194304] 0 2021-11-03T17:15:03.314 INFO:tasks.workunit.client.0.smithi027.stdout:7/683: mknod d0/d2/d1c/d4d/ce1 1 2021-11-03T17:15:03.315 INFO:tasks.workunit.client.0.smithi027.stdout:7/684: getdents d0/d2/d10 0 2021-11-03T17:15:03.316 INFO:tasks.workunit.client.0.smithi027.stdout:7/685: chown d0/f25 1 1 2021-11-03T17:15:03.318 INFO:tasks.workunit.client.0.smithi027.stdout:7/686: symlink d0/d2/d10/le2 0 2021-11-03T17:15:03.318 INFO:tasks.workunit.client.0.smithi027.stdout:7/687: mknod d0/d2d/d2f/dae/dd8/ce3 1 2021-11-03T17:15:03.342 INFO:tasks.workunit.client.0.smithi027.stdout:7/688: dwrite d0/d2/d5e/d36/d3a/fd0 [0,4194304] 0 2021-11-03T17:15:03.345 INFO:tasks.workunit.client.0.smithi027.stdout:7/689: mkdir d0/d2/d10/d82/de4 0 2021-11-03T17:15:03.362 INFO:tasks.workunit.client.0.smithi027.stdout:7/690: dwrite d0/d2/d10/d35/d96/fe0 [0,4194304] 0 2021-11-03T17:15:03.363 INFO:tasks.workunit.client.0.smithi027.stdout:7/691: dread d0/d2/f5 [0,4194304] 0 2021-11-03T17:15:03.364 INFO:tasks.workunit.client.0.smithi027.stdout:7/692: mknod d0/d2/d10/d82/ce5 1 2021-11-03T17:15:03.393 INFO:tasks.workunit.client.0.smithi027.stdout:7/693: dwrite d0/d2/d10/d35/f89 [0,4194304] 0 2021-11-03T17:15:03.394 INFO:tasks.workunit.client.0.smithi027.stdout:7/694: chown d0/d2/d1c/f61 18698475 1 2021-11-03T17:15:03.394 INFO:tasks.workunit.client.0.smithi027.stdout:7/695: fsync d0/d2d/fc5 0 2021-11-03T17:15:03.395 INFO:tasks.workunit.client.0.smithi027.stdout:7/696: rmdir d0/d2/d10/d82 39 2021-11-03T17:15:03.396 INFO:tasks.workunit.client.0.smithi027.stdout:7/697: mkdir d0/d2/d10/d1e/de6 0 2021-11-03T17:15:03.403 INFO:tasks.workunit.client.0.smithi027.stdout:7/698: dread d0/d2/d1c/f61 [0,4194304] 0 2021-11-03T17:15:03.418 INFO:tasks.workunit.client.0.smithi027.stdout:7/699: dwrite d0/db6/fdf [0,4194304] 0 2021-11-03T17:15:03.421 INFO:tasks.workunit.client.0.smithi027.stdout:7/700: truncate d0/d2d/d53/fa0 2786467 0 2021-11-03T17:15:03.423 INFO:tasks.workunit.client.0.smithi027.stdout:7/701: creat d0/d2/d1c/d4d/d6e/d8b/fe7 x:0 0 0 2021-11-03T17:15:03.424 INFO:tasks.workunit.client.0.smithi027.stdout:7/702: write d0/d2/d5e/f32 [2996622,41377] 0 2021-11-03T17:15:03.424 INFO:tasks.workunit.client.0.smithi027.stdout:7/703: fdatasync d0/d2/d5e/d36/d3a/fd0 0 2021-11-03T17:15:03.448 INFO:tasks.workunit.client.0.smithi027.stdout:7/704: dwrite d0/d2/d1c/d4d/d6e/d8b/fc7 [0,4194304] 0 2021-11-03T17:15:03.448 INFO:tasks.workunit.client.0.smithi027.stdout:7/705: chown d0/d2/d1c/d4d/d6e/d84/f92 11755 1 2021-11-03T17:15:03.466 INFO:tasks.workunit.client.0.smithi027.stdout:7/706: dread d0/d2d/d53/fa0 [0,4194304] 0 2021-11-03T17:15:03.468 INFO:tasks.workunit.client.0.smithi027.stdout:7/707: mkdir d0/d2/d10/d82/de8 0 2021-11-03T17:15:03.469 INFO:tasks.workunit.client.0.smithi027.stdout:7/708: readlink d0/d2/d5e/d36/d3a/la4 0 2021-11-03T17:15:03.489 INFO:tasks.workunit.client.0.smithi027.stdout:7/709: dwrite d0/d2/d1c/d4d/d6e/d84/dad/fc3 [0,4194304] 0 2021-11-03T17:15:03.490 INFO:tasks.workunit.client.0.smithi027.stdout:7/710: read d0/d2/d1f/fc2 [461703,79590] 0 2021-11-03T17:15:03.495 INFO:tasks.workunit.client.0.smithi027.stdout:7/711: getdents d0/d2/d10/d82 0 2021-11-03T17:15:03.498 INFO:tasks.workunit.client.0.smithi027.stdout:7/712: truncate d0/d2d/d2f/f39 1249915 0 2021-11-03T17:15:03.499 INFO:tasks.workunit.client.0.smithi027.stdout:7/713: getdents d0/d2/d10/d82 0 2021-11-03T17:15:03.499 INFO:tasks.workunit.client.0.smithi027.stdout:7/714: fsync d0/d2/d1c/d4d/d6e/d84/dad/fc3 0 2021-11-03T17:15:03.499 INFO:tasks.workunit.client.0.smithi027.stdout:7/715: stat d0/d2d/d2f/dae 0 2021-11-03T17:15:03.500 INFO:tasks.workunit.client.0.smithi027.stdout:7/716: truncate d0/d2/d5e/d36/f64 74870 0 2021-11-03T17:15:03.501 INFO:tasks.workunit.client.0.smithi027.stdout:7/717: creat d0/d2/d1c/d4d/fe9 x:0 0 0 2021-11-03T17:15:03.502 INFO:tasks.workunit.client.0.smithi027.stdout:7/718: mkdir d0/d2/d1c/d4d/d6e/d8b/dea 0 2021-11-03T17:15:03.503 INFO:tasks.workunit.client.0.smithi027.stdout:7/719: readlink d0/d2/d1c/l8c 0 2021-11-03T17:15:03.504 INFO:tasks.workunit.client.0.smithi027.stdout:7/720: read d0/d2/d10/d35/d96/fa2 [976567,51986] 0 2021-11-03T17:15:03.505 INFO:tasks.workunit.client.0.smithi027.stdout:7/721: mknod d0/d2/d10/d35/ceb 1 2021-11-03T17:15:03.505 INFO:tasks.workunit.client.0.smithi027.stdout:7/722: chown d0/d2/d5e/f32 11606 1 2021-11-03T17:15:03.506 INFO:tasks.workunit.client.0.smithi027.stdout:7/723: creat d0/d2/d1f/fec x:0 0 0 2021-11-03T17:15:03.529 INFO:tasks.workunit.client.0.smithi027.stdout:7/724: dwrite d0/d2/d10/d35/d96/fb3 [0,4194304] 0 2021-11-03T17:15:03.529 INFO:tasks.workunit.client.0.smithi027.stdout:7/725: write d0/d2d/d53/fa0 [748843,57240] 0 2021-11-03T17:15:03.531 INFO:tasks.workunit.client.0.smithi027.stdout:7/726: mknod d0/d2/d10/d1e/d20/d74/ced 1 2021-11-03T17:15:03.539 INFO:tasks.workunit.client.0.smithi027.stdout:6/874: sync 2021-11-03T17:15:03.541 INFO:tasks.workunit.client.0.smithi027.stdout:7/727: write d0/d2d/d53/f65 [2013297,49201] 0 2021-11-03T17:15:03.559 INFO:tasks.workunit.client.0.smithi027.stdout:6/875: dwrite dc/d82/d57/dad/dae/db1/f35 [4194304,4194304] 0 2021-11-03T17:15:03.561 INFO:tasks.workunit.client.0.smithi027.stdout:6/876: mkdir dc/d82/d116 0 2021-11-03T17:15:03.568 INFO:tasks.workunit.client.0.smithi027.stdout:7/728: dwrite d0/d2/d1c/faf [0,4194304] 0 2021-11-03T17:15:03.568 INFO:tasks.workunit.client.0.smithi027.stdout:7/729: read - d0/d2/d1c/d4d/d6e/d8b/fe7 zero size 2021-11-03T17:15:03.571 INFO:tasks.workunit.client.0.smithi027.stdout:7/730: mknod d0/d2d/d2f/cee 1 2021-11-03T17:15:03.573 INFO:tasks.workunit.client.0.smithi027.stdout:7/731: symlink d0/d2/d10/d1e/lef 0 2021-11-03T17:15:03.581 INFO:tasks.workunit.client.0.smithi027.stdout:6/877: dwrite dc/f88 [0,4194304] 0 2021-11-03T17:15:03.582 INFO:tasks.workunit.client.0.smithi027.stdout:6/878: chown dc/d16/d1b/dda/ff2 103793 1 2021-11-03T17:15:03.582 INFO:tasks.workunit.client.0.smithi027.stdout:6/879: write dc/d16/d9a/fd3 [422875,110285] 0 2021-11-03T17:15:03.582 INFO:tasks.workunit.client.0.smithi027.stdout:6/880: mknod dc/d82/d57/dad/dae/db1/c117 1 2021-11-03T17:15:03.583 INFO:tasks.workunit.client.0.smithi027.stdout:6/881: read dc/d82/d87/fb9 [13749,104788] 0 2021-11-03T17:15:03.583 INFO:tasks.workunit.client.0.smithi027.stdout:6/882: write dc/d16/d1b/d3c/d51/f58 [1306192,32655] 0 2021-11-03T17:15:03.584 INFO:tasks.workunit.client.0.smithi027.stdout:6/883: stat dc/d37/lfd 0 2021-11-03T17:15:03.584 INFO:tasks.workunit.client.0.smithi027.stdout:7/732: write d0/d2/d10/d1e/f99 [4577494,47298] 0 2021-11-03T17:15:03.585 INFO:tasks.workunit.client.0.smithi027.stdout:7/733: chown d0/d2/d1c/d4d/l5d 2454 1 2021-11-03T17:15:03.600 INFO:tasks.workunit.client.0.smithi027.stdout:6/884: dwrite dc/d82/d57/dad/dae/db1/f35 [0,4194304] 0 2021-11-03T17:15:03.605 INFO:tasks.workunit.client.0.smithi027.stdout:6/885: rename dc/d16/d1b/d3c/d51/f58 to dc/d16/d1b/d24/d7e/f118 0 2021-11-03T17:15:03.605 INFO:tasks.workunit.client.0.smithi027.stdout:6/886: getdents dc/d82/df5 0 2021-11-03T17:15:03.605 INFO:tasks.workunit.client.0.smithi027.stdout:6/887: mknod dc/d82/d57/dad/dae/db1/d34/c119 1 2021-11-03T17:15:03.608 INFO:tasks.workunit.client.0.smithi027.stdout:6/888: unlink dc/d16/d1b/la6 0 2021-11-03T17:15:03.608 INFO:tasks.workunit.client.0.smithi027.stdout:6/889: fsync dc/d16/d1b/f28 0 2021-11-03T17:15:03.609 INFO:tasks.workunit.client.0.smithi027.stdout:6/890: stat dc/d16/fc2 0 2021-11-03T17:15:03.609 INFO:tasks.workunit.client.0.smithi027.stdout:6/891: mknod dc/d82/d57/d9b/dc8/c11a 1 2021-11-03T17:15:03.611 INFO:tasks.workunit.client.0.smithi027.stdout:6/892: unlink dc/f14 0 2021-11-03T17:15:03.612 INFO:tasks.workunit.client.0.smithi027.stdout:7/734: dwrite d0/d2/d10/d1e/d20/f26 [0,4194304] 0 2021-11-03T17:15:03.613 INFO:tasks.workunit.client.0.smithi027.stdout:7/735: read d0/d2/d10/d35/f89 [516597,15055] 0 2021-11-03T17:15:03.614 INFO:tasks.workunit.client.0.smithi027.stdout:6/893: rename dc/d82/d57/dad/dae/db1/d34/d89/f114 to dc/d16/d1b/dda/f11b 0 2021-11-03T17:15:03.617 INFO:tasks.workunit.client.0.smithi027.stdout:6/894: mkdir dc/d82/d57/dad/dae/db1/dd1/d11c 0 2021-11-03T17:15:03.618 INFO:tasks.workunit.client.0.smithi027.stdout:6/895: mknod dc/d16/d62/c11d 1 2021-11-03T17:15:03.618 INFO:tasks.workunit.client.0.smithi027.stdout:6/896: fsync dc/d16/d1b/ff3 0 2021-11-03T17:15:03.649 INFO:tasks.workunit.client.0.smithi027.stdout:7/736: dwrite d0/d2/d10/d1e/d20/d74/f7b [0,4194304] 0 2021-11-03T17:15:03.649 INFO:tasks.workunit.client.0.smithi027.stdout:7/737: fdatasync d0/f18 0 2021-11-03T17:15:03.649 INFO:tasks.workunit.client.0.smithi027.stdout:7/738: dread - d0/d2/d59/f73 zero size 2021-11-03T17:15:03.650 INFO:tasks.workunit.client.0.smithi027.stdout:7/739: mkdir d0/d2d/d2f/df0 0 2021-11-03T17:15:03.652 INFO:tasks.workunit.client.0.smithi027.stdout:7/740: creat d0/d2/ff1 x:0 0 0 2021-11-03T17:15:03.659 INFO:tasks.workunit.client.0.smithi027.stdout:6/897: dwrite dc/d16/d30/f38 [0,4194304] 0 2021-11-03T17:15:03.663 INFO:tasks.workunit.client.0.smithi027.stdout:7/741: dread d0/d2/d1c/d4d/d6e/d8b/fc7 [0,4194304] 0 2021-11-03T17:15:03.664 INFO:tasks.workunit.client.0.smithi027.stdout:6/898: rename dc/d16/d1b/d3c/ddc/f111 to dc/d82/f11e 0 2021-11-03T17:15:03.665 INFO:tasks.workunit.client.0.smithi027.stdout:7/742: chown d0/d2/d10/l72 7 1 2021-11-03T17:15:03.668 INFO:tasks.workunit.client.0.smithi027.stdout:6/899: creat dc/d82/d57/d96/dd4/f11f x:0 0 0 2021-11-03T17:15:03.671 INFO:tasks.workunit.client.0.smithi027.stdout:7/743: rename d0/d2/d5e/d36/d3a/da7 to d0/d2/d10/d82/df2 0 2021-11-03T17:15:03.673 INFO:tasks.workunit.client.0.smithi027.stdout:6/900: link dc/d82/d57/dad/dae/db1/l69 dc/d82/d57/dad/d113/l120 0 2021-11-03T17:15:03.673 INFO:tasks.workunit.client.0.smithi027.stdout:7/744: truncate d0/d2/d5e/d36/f63 514629 0 2021-11-03T17:15:03.676 INFO:tasks.workunit.client.0.smithi027.stdout:6/901: creat dc/d82/d87/dec/f121 x:0 0 0 2021-11-03T17:15:03.681 INFO:tasks.workunit.client.0.smithi027.stdout:6/902: fdatasync dc/d16/d1b/d24/d7e/f118 0 2021-11-03T17:15:03.684 INFO:tasks.workunit.client.0.smithi027.stdout:6/903: unlink dc/d82/d57/dad/dae/db1/d34/d89/f9d 0 2021-11-03T17:15:03.722 INFO:tasks.workunit.client.0.smithi027.stdout:6/904: dwrite dc/d82/d57/dad/dae/f103 [0,4194304] 0 2021-11-03T17:15:03.725 INFO:tasks.workunit.client.0.smithi027.stdout:6/905: readlink dc/d82/d57/dad/dae/db1/lbf 0 2021-11-03T17:15:03.725 INFO:tasks.workunit.client.0.smithi027.stdout:6/906: chown dc/d16/l31 313 1 2021-11-03T17:15:03.725 INFO:tasks.workunit.client.0.smithi027.stdout:6/907: fsync dc/d16/fc2 0 2021-11-03T17:15:03.725 INFO:tasks.workunit.client.0.smithi027.stdout:6/908: mknod dc/d16/d1b/d3c/d51/d5f/dce/c122 1 2021-11-03T17:15:03.726 INFO:tasks.workunit.client.0.smithi027.stdout:6/909: symlink dc/d16/d30/l123 0 2021-11-03T17:15:03.728 INFO:tasks.workunit.client.0.smithi027.stdout:6/910: creat dc/d82/d57/d9b/f124 x:0 0 0 2021-11-03T17:15:03.728 INFO:tasks.workunit.client.0.smithi027.stdout:6/911: write dc/d16/d1b/ff3 [579602,28563] 0 2021-11-03T17:15:03.729 INFO:tasks.workunit.client.0.smithi027.stdout:6/912: readlink dc/d16/d1b/la5 0 2021-11-03T17:15:03.729 INFO:tasks.workunit.client.0.smithi027.stdout:6/913: mknod dc/d82/d57/dad/d113/c125 1 2021-11-03T17:15:03.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:03 smithi027 conmon[32322]: cluster 2021-11-03T17:15:02.640501+0000 2021-11-03T17:15:03.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:03 smithi027 conmon[32322]: mon.smithi027 (mon.0) 680 : cluster [INF] Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2021-11-03T17:15:03.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:03 smithi027 conmon[32322]: cluster 2021-11-03T17:15:02.641150+0000 mgr.smithi027.ufhhvb 2021-11-03T17:15:03.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:03 smithi027 conmon[32322]: (mgr.14622) 9 : cluster [DBG] pgmap v5: 65 pgs: 65 active+clean; 3.3 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail 2021-11-03T17:15:03.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:03 smithi027 conmon[32322]: cluster 2021-11-03T17:15:03.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:03 smithi027 conmon[32322]: 2021-11-03T17:15:03.076359 2021-11-03T17:15:03.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:03 smithi027 conmon[32322]: +0000 mon.smithi027 (mon.0) 681 : cluster [DBG] 2021-11-03T17:15:03.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:03 smithi027 conmon[32322]: mgrmap e27: smithi027.ufhhvb(active, since 3s) 2021-11-03T17:15:03.849 INFO:tasks.workunit.client.0.smithi027.stdout:9/968: sync 2021-11-03T17:15:03.869 INFO:tasks.workunit.client.0.smithi027.stdout:9/969: dwrite db/d1b/f38 [4194304,4194304] 0 2021-11-03T17:15:03.871 INFO:tasks.workunit.client.0.smithi027.stdout:9/970: getdents db/d1b/d8c 0 2021-11-03T17:15:03.888 INFO:tasks.workunit.client.0.smithi027.stdout:9/971: dwrite db/d1b/d1c/d1d/d40/d104/d14d/f15a [0,4194304] 0 2021-11-03T17:15:03.894 INFO:tasks.workunit.client.0.smithi027.stdout:9/972: rename db/d1b/f45 to db/d1b/d2b/d41/f16a 0 2021-11-03T17:15:03.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:03 smithi115 conmon[34670]: cluster 2021-11-03T17:15:02.640501+0000 mon.smithi027 ( 2021-11-03T17:15:03.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:03 smithi115 conmon[34670]: mon.0) 680 : cluster [INF] Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2021-11-03T17:15:03.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:03 smithi115 conmon[34670]: cluster 2021-11-03T17:15:02.641150+0000 mgr.smithi027.ufhhvb (mgr.14622) 9 : cluster [DBG] pgmap v5: 65 pgs: 65 active+clean; 3.3 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail 2021-11-03T17:15:03.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:03 smithi115 conmon[34670]: cluster 2021-11-03T17:15 2021-11-03T17:15:03.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:03 smithi115 conmon[34670]: :03.076359+0000 mon.smithi027 (mon.0) 681 : 2021-11-03T17:15:03.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:03 smithi115 conmon[34670]: cluster [DBG] mgrmap e27: smithi027.ufhhvb(active, since 3s) 2021-11-03T17:15:05.045 INFO:tasks.workunit.client.0.smithi027.stdout:6/914: sync 2021-11-03T17:15:05.047 INFO:tasks.workunit.client.0.smithi027.stdout:6/915: link dc/d16/f7a dc/d16/d30/f126 0 2021-11-03T17:15:05.048 INFO:tasks.workunit.client.0.smithi027.stdout:6/916: creat dc/d16/f127 x:0 0 0 2021-11-03T17:15:05.049 INFO:tasks.workunit.client.0.smithi027.stdout:6/917: write dc/d82/f11e [1012194,60595] 0 2021-11-03T17:15:05.049 INFO:tasks.workunit.client.0.smithi027.stdout:6/918: readlink dc/d16/l6d 0 2021-11-03T17:15:05.119 INFO:tasks.workunit.client.0.smithi027.stdout:7/745: sync 2021-11-03T17:15:05.119 INFO:tasks.workunit.client.0.smithi027.stdout:7/746: mknod d0/d2/d1c/d4d/d6e/d84/dad/cf3 1 2021-11-03T17:15:05.120 INFO:tasks.workunit.client.0.smithi027.stdout:7/747: read - d0/d2/d10/d35/d96/fcd zero size 2021-11-03T17:15:05.202 INFO:tasks.workunit.client.0.smithi027.stdout:9/973: sync 2021-11-03T17:15:05.207 INFO:tasks.workunit.client.0.smithi027.stdout:9/974: rmdir db/d1b/d2b 39 2021-11-03T17:15:05.223 INFO:tasks.workunit.client.0.smithi027.stdout:9/975: dwrite db/d1b/d1c/f134 [0,4194304] 0 2021-11-03T17:15:05.223 INFO:tasks.workunit.client.0.smithi027.stdout:9/976: getdents db/d1b/d2b/d41/d95/d9f 0 2021-11-03T17:15:05.227 INFO:tasks.workunit.client.0.smithi027.stdout:9/977: rename db/d1b/d2b/d41/dad/dbb/ld1 to db/d1b/d2b/d41/dad/dbb/d11b/l16b 0 2021-11-03T17:15:05.227 INFO:tasks.workunit.client.0.smithi027.stdout:9/978: dread - db/d1b/d2b/d41/f142 zero size 2021-11-03T17:15:05.227 INFO:tasks.workunit.client.0.smithi027.stdout:9/979: mknod db/d1b/d2b/d41/dad/dbb/d12a/c16c 1 2021-11-03T17:15:05.228 INFO:tasks.workunit.client.0.smithi027.stdout:9/980: rmdir db/d1b/d8e/d14c 0 2021-11-03T17:15:05.230 INFO:tasks.workunit.client.0.smithi027.stdout:9/981: dread db/d1b/d1c/f102 [0,4194304] 0 2021-11-03T17:15:05.236 INFO:tasks.workunit.client.0.smithi027.stdout:9/982: dread db/d1b/d2b/d41/dad/fd3 [0,4194304] 0 2021-11-03T17:15:05.236 INFO:tasks.workunit.client.0.smithi027.stdout:9/983: write db/d1b/d2b/f108 [5225405,97057] 0 2021-11-03T17:15:05.246 INFO:tasks.workunit.client.0.smithi027.stdout:9/984: write db/d1b/d1c/f65 [5237473,47670] 0 2021-11-03T17:15:05.249 INFO:tasks.workunit.client.0.smithi027.stdout:9/985: rename db/d1b/d1c/d1d/d40/d60/d154 to db/d1b/d8c/d14f/d16d 0 2021-11-03T17:15:05.251 INFO:tasks.workunit.client.0.smithi027.stdout:9/986: mkdir db/d1b/d2b/d9c/d16e 0 2021-11-03T17:15:05.270 INFO:tasks.workunit.client.0.smithi027.stdout:9/987: dwrite db/d1b/d8c/dcd/f119 [0,4194304] 0 2021-11-03T17:15:05.272 INFO:tasks.workunit.client.0.smithi027.stdout:9/988: rmdir db/d1b/d2b/d41/d95/d9f/d12f 39 2021-11-03T17:15:05.273 INFO:tasks.workunit.client.0.smithi027.stdout:9/989: mknod db/d1b/dfa/c16f 1 2021-11-03T17:15:05.279 INFO:tasks.workunit.client.0.smithi027.stdout:9/990: dread db/d1b/d2b/dc7/fe5 [0,4194304] 0 2021-11-03T17:15:05.280 INFO:tasks.workunit.client.0.smithi027.stdout:9/991: getdents db/d1b/d2b/d41/d95/deb 0 2021-11-03T17:15:05.281 INFO:tasks.workunit.client.0.smithi027.stdout:9/992: mknod db/c170 1 2021-11-03T17:15:05.282 INFO:tasks.workunit.client.0.smithi027.stdout:9/993: creat db/d1b/d1c/d1d/d40/d104/f171 x:0 0 0 2021-11-03T17:15:05.305 INFO:tasks.workunit.client.0.smithi027.stdout:9/994: dwrite db/f115 [0,4194304] 0 2021-11-03T17:15:05.308 INFO:tasks.workunit.client.0.smithi027.stdout:9/995: rename db/d1b/d8e/l14e to db/d1b/d2b/d41/d6c/d116/l172 0 2021-11-03T17:15:06.189 INFO:tasks.workunit.client.0.smithi027.stdout:6/919: sync 2021-11-03T17:15:06.288 INFO:tasks.workunit.client.0.smithi027.stdout:7/748: sync 2021-11-03T17:15:06.313 INFO:tasks.workunit.client.0.smithi027.stdout:7/749: dwrite d0/d2/d1f/fa6 [0,4194304] 0 2021-11-03T17:15:06.318 INFO:tasks.workunit.client.0.smithi027.stdout:7/750: link d0/d2d/d58/f69 d0/d2d/d2f/dae/ff4 0 2021-11-03T17:15:06.327 INFO:tasks.workunit.client.0.smithi027.stdout:7/751: write d0/d2/d10/d1e/d20/d74/f9d [4140506,2101] 0 2021-11-03T17:15:06.328 INFO:tasks.workunit.client.0.smithi027.stdout:7/752: readlink d0/d2/d1f/l3b 0 2021-11-03T17:15:06.329 INFO:tasks.workunit.client.0.smithi027.stdout:7/753: symlink d0/d2/d10/d82/de4/lf5 0 2021-11-03T17:15:06.331 INFO:tasks.workunit.client.0.smithi027.stdout:7/754: rmdir d0/d2/d1c/d4d/d6e 39 2021-11-03T17:15:06.333 INFO:tasks.workunit.client.0.smithi027.stdout:7/755: fdatasync d0/d2/d10/d1e/f5c 0 2021-11-03T17:15:06.336 INFO:tasks.workunit.client.0.smithi027.stdout:7/756: creat d0/d2/d5e/ff6 x:0 0 0 2021-11-03T17:15:06.339 INFO:tasks.workunit.client.0.smithi027.stdout:7/757: symlink d0/d2/d5e/d36/lf7 0 2021-11-03T17:15:06.341 INFO:tasks.workunit.client.0.smithi027.stdout:7/758: creat d0/d2d/ff8 x:0 0 0 2021-11-03T17:15:06.341 INFO:tasks.workunit.client.0.smithi027.stdout:7/759: dread - d0/d2d/d2f/dae/dc8/fde zero size 2021-11-03T17:15:06.342 INFO:tasks.workunit.client.0.smithi027.stdout:7/760: mkdir d0/d2d/d2f/dae/dc8/df9 0 2021-11-03T17:15:06.346 INFO:tasks.workunit.client.0.smithi027.stdout:7/761: getdents d0/d2/d5e/d36 0 2021-11-03T17:15:06.348 INFO:tasks.workunit.client.0.smithi027.stdout:7/762: write d0/d2/d1c/f27 [1207045,91664] 0 2021-11-03T17:15:06.364 INFO:tasks.workunit.client.0.smithi027.stdout:7/763: dwrite d0/d2/d1c/d4d/d6e/fd6 [0,4194304] 0 2021-11-03T17:15:06.383 INFO:tasks.workunit.client.0.smithi027.stdout:7/764: dwrite d0/d2d/d2f/fb9 [0,4194304] 0 2021-11-03T17:15:06.383 INFO:tasks.workunit.client.0.smithi027.stdout:7/765: dread - d0/d2/d1c/d4d/dcc/fdb zero size 2021-11-03T17:15:06.383 INFO:tasks.workunit.client.0.smithi027.stdout:7/766: read - d0/d2/d1f/fec zero size 2021-11-03T17:15:06.384 INFO:tasks.workunit.client.0.smithi027.stdout:7/767: creat d0/d2/d1c/ffa x:0 0 0 2021-11-03T17:15:06.386 INFO:tasks.workunit.client.0.smithi027.stdout:7/768: symlink d0/d2/d10/d35/lfb 0 2021-11-03T17:15:06.392 INFO:tasks.workunit.client.0.smithi027.stdout:7/769: write d0/d2/d10/d35/fb5 [209523,79273] 0 2021-11-03T17:15:06.393 INFO:tasks.workunit.client.0.smithi027.stdout:7/770: creat d0/d2/d10/d1e/d20/d74/ffc x:0 0 0 2021-11-03T17:15:06.394 INFO:tasks.workunit.client.0.smithi027.stdout:7/771: write d0/d2/d10/d35/f89 [1640827,23989] 0 2021-11-03T17:15:06.395 INFO:tasks.workunit.client.0.smithi027.stdout:7/772: fsync d0/d2/d1c/f61 0 2021-11-03T17:15:06.396 INFO:tasks.workunit.client.0.smithi027.stdout:7/773: mkdir d0/d2/d10/d82/de4/dfd 0 2021-11-03T17:15:06.398 INFO:tasks.workunit.client.0.smithi027.stdout:7/774: mkdir d0/d2/d10/d82/de4/dfd/dfe 0 2021-11-03T17:15:06.399 INFO:tasks.workunit.client.0.smithi027.stdout:7/775: symlink d0/d2/d10/d82/de4/lff 0 2021-11-03T17:15:06.400 INFO:tasks.workunit.client.0.smithi027.stdout:7/776: creat d0/d2/d10/d82/df2/f100 x:0 0 0 2021-11-03T17:15:06.408 INFO:tasks.workunit.client.0.smithi027.stdout:7/777: truncate d0/d2/d10/d1e/d20/d74/f9d 350882 0 2021-11-03T17:15:06.409 INFO:tasks.workunit.client.0.smithi027.stdout:7/778: mknod d0/d2d/d2f/dae/dc8/c101 1 2021-11-03T17:15:06.415 INFO:tasks.workunit.client.0.smithi027.stdout:7/779: write d0/d2/d5e/d36/fb0 [288848,40690] 0 2021-11-03T17:15:06.430 INFO:tasks.workunit.client.0.smithi027.stdout:7/780: dwrite d0/d2/d1f/fc2 [4194304,4194304] 0 2021-11-03T17:15:06.432 INFO:tasks.workunit.client.0.smithi027.stdout:7/781: unlink d0/d2/d5e/d36/l71 0 2021-11-03T17:15:06.443 INFO:tasks.workunit.client.0.smithi027.stdout:7/782: dread d0/d2/d10/d35/f7a [0,4194304] 0 2021-11-03T17:15:06.445 INFO:tasks.workunit.client.0.smithi027.stdout:7/783: rmdir d0/d2d/d2f/df0 0 2021-11-03T17:15:06.445 INFO:tasks.workunit.client.0.smithi027.stdout:7/784: chown d0/d2/d10/d1e/d20/d74/f7b 0 1 2021-11-03T17:15:06.460 INFO:tasks.workunit.client.0.smithi027.stdout:7/785: dwrite d0/d2/d1c/d4d/fe9 [0,4194304] 0 2021-11-03T17:15:06.464 INFO:tasks.workunit.client.0.smithi027.stdout:7/786: link d0/d2d/d2f/fd1 d0/d2/d10/d82/df2/f102 0 2021-11-03T17:15:06.469 INFO:tasks.workunit.client.0.smithi027.stdout:7/787: getdents d0/d2d/d2f/dae/dc8 0 2021-11-03T17:15:06.469 INFO:tasks.workunit.client.0.smithi027.stdout:7/788: write d0/d2/d10/d1e/f5c [4828596,78559] 0 2021-11-03T17:15:06.471 INFO:tasks.workunit.client.0.smithi027.stdout:7/789: creat d0/d2/d10/d82/f103 x:0 0 0 2021-11-03T17:15:06.471 INFO:tasks.workunit.client.0.smithi027.stdout:7/790: write d0/d2/d1c/d4d/d6e/d8b/fc7 [2012470,122201] 0 2021-11-03T17:15:06.474 INFO:tasks.workunit.client.0.smithi027.stdout:7/791: unlink d0/d2d/d2f/fb9 0 2021-11-03T17:15:06.476 INFO:tasks.workunit.client.0.smithi027.stdout:7/792: fsync d0/d2/d1c/f2b 0 2021-11-03T17:15:06.486 INFO:tasks.workunit.client.0.smithi027.stdout:7/793: dread d0/d2/d1f/fa6 [0,4194304] 0 2021-11-03T17:15:06.487 INFO:tasks.workunit.client.0.smithi027.stdout:7/794: truncate d0/f18 3913375 0 2021-11-03T17:15:06.489 INFO:tasks.workunit.client.0.smithi027.stdout:7/795: creat d0/d2/d10/d35/f104 x:0 0 0 2021-11-03T17:15:06.492 INFO:tasks.workunit.client.0.smithi027.stdout:7/796: rename d0/d2/d10/d35/f7a to d0/d2/d1c/d4d/dcc/f105 0 2021-11-03T17:15:06.495 INFO:tasks.workunit.client.0.smithi027.stdout:7/797: truncate d0/d2/d1c/faf 954436 0 2021-11-03T17:15:06.497 INFO:tasks.workunit.client.0.smithi027.stdout:7/798: link d0/f25 d0/d2/d10/d82/de4/f106 0 2021-11-03T17:15:06.498 INFO:tasks.workunit.client.0.smithi027.stdout:7/799: symlink d0/d2d/d2f/dae/dc8/l107 0 2021-11-03T17:15:06.501 INFO:tasks.workunit.client.0.smithi027.stdout:7/800: mkdir d0/d2/d10/d1e/d108 0 2021-11-03T17:15:06.503 INFO:tasks.workunit.client.0.smithi027.stdout:7/801: rename d0/d2/d1c/d4d/d6e/fc1 to d0/d2/d10/d82/de8/f109 0 2021-11-03T17:15:06.676 INFO:tasks.workunit.client.0.smithi027.stdout:9/996: sync 2021-11-03T17:15:06.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:06 smithi115 conmon[34670]: cluster 2021-11-03T17:15:04.641565+0000 mgr.smithi027.ufhhvb (mgr.14622) 10 : cluster [DBG] pgmap v6: 65 pgs: 65 active+clean; 3.3 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail 2021-11-03T17:15:06.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:06 smithi115 conmon[34670]: cephadm 2021-11-03T17:15:05.253715+0000 mgr.smithi027.ufhhvb (mgr.14622) 11 : cephadm [INF] poll took 4074.563 ms: 1 events 2021-11-03T17:15:06.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:06 smithi115 conmon[34670]: audit 2021-11-03T17:15:05.302279+0000 mon.smithi027 (mon.0) 682 : audit [INF] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:15:06.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:06 smithi115 conmon[34670]: audit 2021-11-03T17:15:05.977224+0000 mon.smithi027 (mon.0) 683 : audit [INF] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:15:06.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:06 smithi115 conmon[34670]: audit 2021-11-03T17:15:05.978347+0000 mon.smithi027 (mon.0) 684 : audit [INF] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2021-11-03T17:15:06.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:06 smithi027 conmon[32322]: cluster 2021-11-03T17:15: 2021-11-03T17:15:06.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:06 smithi027 conmon[32322]: 04.641565+0000 mgr.smithi027.ufhhvb (mgr.14622) 10 : cluster [DBG] pgmap v6: 65 pgs: 65 active+clean; 3.3 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail 2021-11-03T17:15:06.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:06 smithi027 conmon[32322]: cephadm 2021-11-03T17 2021-11-03T17:15:06.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:06 smithi027 conmon[32322]: :15:05.253715+0000 mgr.smithi027.ufhhvb 2021-11-03T17:15:06.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:06 smithi027 conmon[32322]: (mgr.14622) 11 : cephadm 2021-11-03T17:15:06.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:06 smithi027 conmon[32322]: [INF] poll took 4074.563 ms: 1 events 2021-11-03T17:15:06.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:06 smithi027 conmon[32322]: audit 2021 2021-11-03T17:15:06.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:06 smithi027 conmon[32322]: -11-03T17:15: 2021-11-03T17:15:06.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:06 smithi027 conmon[32322]: 05.302279+0000 mon.smithi027 (mon 2021-11-03T17:15:06.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:06 smithi027 conmon[32322]: .0) 682 : audit [INF] 2021-11-03T17:15:06.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:06 smithi027 conmon[32322]: from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:15:06.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:06 smithi027 conmon[32322]: audit 2021 2021-11-03T17:15:06.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:06 smithi027 conmon[32322]: -11-03T17:15 2021-11-03T17:15:06.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:06 smithi027 conmon[32322]: :05.977224+0000 mon.smithi027 2021-11-03T17:15:06.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:06 smithi027 conmon[32322]: (mon.0) 683 : audit 2021-11-03T17:15:06.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:06 smithi027 conmon[32322]: [INF] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:15:06.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:06 smithi027 conmon[32322]: audit 2021 2021-11-03T17:15:06.776 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:06 smithi027 conmon[32322]: -11-03T17:15 2021-11-03T17:15:06.776 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:06 smithi027 conmon[32322]: :05.978347+0000 mon.smithi027 ( 2021-11-03T17:15:06.777 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:06 smithi027 conmon[32322]: mon.0) 684 : audit 2021-11-03T17:15:06.777 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:06 smithi027 conmon[32322]: [INF] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2021-11-03T17:15:07.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:07 smithi115 conmon[34670]: cluster 2021-11-03T17:15:06.309051+0000 mon.smithi027 2021-11-03T17:15:07.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:07 smithi115 conmon[34670]: (mon.0) 685 : cluster [DBG] mds.? [v2:172.21.15.115:6826/3367380088,v1:172.21.15.115:6827/3367380088] up:resolve 2021-11-03T17:15:07.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:07 smithi115 conmon[34670]: cluster 2021-11-03T17:15:06.309101+0000 mon.smithi027 (mon.0 2021-11-03T17:15:07.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:07 smithi115 conmon[34670]: ) 686 : cluster [DBG] mds.? [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] up:standby 2021-11-03T17:15:07.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:07 smithi115 conmon[34670]: cluster 2021-11-03 2021-11-03T17:15:07.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:07 smithi115 conmon[34670]: T17:15:06.309151+0000 mon.smithi027 (mon.0) 687 : cluster 2021-11-03T17:15:07.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:07 smithi115 conmon[34670]: [DBG] fsmap cephfs:2/2 {0=cephfs.smithi115.semywj=up:resolve,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:15:07.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:07 smithi115 conmon[34670]: cluster 2021-11-03T17:15:07. 2021-11-03T17:15:07.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:07 smithi115 conmon[34670]: 298388+0000 mon.smithi027 (mon.0) 688 : cluster [DBG] Standby manager daemon smithi115.tcyodz started 2021-11-03T17:15:07.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:07 smithi115 conmon[34670]: audit 2021-11-03T17:15:07.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:07 smithi115 conmon[34670]: 2021-11-03T17:15:07.299800+0000 mon.smithi115 (mon 2021-11-03T17:15:07.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:07 smithi115 conmon[34670]: .1) 60 : audit [DBG] from='mgr.? 172.21.15.115:0/3320853155' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi115.tcyodz/crt"}]: dispatch 2021-11-03T17:15:07.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:07 smithi115 conmon[34670]: audit 2021-11-03T17:15:07.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:07 smithi115 conmon[34670]: 2021-11-03T 2021-11-03T17:15:07.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:07 smithi115 conmon[34670]: 17:15:07.300254+0000 mon.smithi115 (mon.1) 61 : audit [DBG] 2021-11-03T17:15:07.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:07 smithi115 conmon[34670]: from='mgr.? 172.21.15.115:0/3320853155' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-11-03T17:15:07.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:07 smithi115 conmon[34670]: audit 2021-11- 2021-11-03T17:15:07.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:07 smithi115 conmon[34670]: 03T17:15:07.302130+0000 mon.smithi115 (mon. 2021-11-03T17:15:07.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:07 smithi115 conmon[34670]: 1) 62 : audit [DBG] from='mgr.? 172.21.15.115:0/3320853155' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi115.tcyodz/key"}]: dispatch 2021-11-03T17:15:07.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:07 smithi115 conmon[34670]: audit 2021-11-03T17: 2021-11-03T17:15:07.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:07 smithi115 conmon[34670]: 15:07.303002+0000 mon.smithi115 (mon.1) 63 2021-11-03T17:15:07.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:07 smithi115 conmon[34670]: : audit [DBG] from='mgr.? 172.21.15.115:0/3320853155' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-11-03T17:15:07.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:07 smithi027 conmon[32322]: cluster 2021-11-03T17:15:06.309051+0000 mon.smithi027 (mon.0) 685 : cluster [DBG] mds.? [v2:172.21.15.115:6826/3367380088,v1:172.21.15.115:6827/3367380088] up:resolve 2021-11-03T17:15:07.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:07 smithi027 conmon[32322]: cluster 2021-11-03T17:15:06.309101+0000 mon.smithi027 (mon.0) 686 : cluster 2021-11-03T17:15:07.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:07 smithi027 conmon[32322]: [DBG] mds.? [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] up:standby 2021-11-03T17:15:07.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:07 smithi027 conmon[32322]: cluster 2021-11-03T17:15:06.309151+0000 mon.smithi027 (mon.0) 687 : cluster [DBG] fsmap cephfs:2/2 {0=cephfs.smithi115.semywj=up:resolve,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:15:07.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:07 smithi027 conmon[32322]: cluster 2021 2021-11-03T17:15:07.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:07 smithi027 conmon[32322]: -11-03T17:15:07.298388+0000 mon.smithi027 (mon.0) 688 : cluster [DBG] Standby manager daemon smithi115.tcyodz started 2021-11-03T17:15:07.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:07 smithi027 conmon[32322]: audit 2021 2021-11-03T17:15:07.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:07 smithi027 conmon[32322]: -11-03T17:15:07.299800+0000 mon.smithi115 (mon.1 2021-11-03T17:15:07.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:07 smithi027 conmon[32322]: ) 60 : audit [DBG] from='mgr.? 172.21.15.115:0/3320853155' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi115.tcyodz/crt"}]: dispatch 2021-11-03T17:15:07.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:07 smithi027 conmon[32322]: audit 2021-11-03T 2021-11-03T17:15:07.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:07 smithi027 conmon[32322]: 17:15:07.300254+0000 mon.smithi115 (mon.1) 61 : audit [DBG] from='mgr.? 172.21.15.115:0/3320853155' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-11-03T17:15:07.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:07 smithi027 conmon[32322]: 2021-11-03T17:15:07.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:07 smithi027 conmon[32322]: audit 2021-11-03T17:15:07. 2021-11-03T17:15:07.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:07 smithi027 conmon[32322]: 302130+0000 mon.smithi115 (mon.1) 62 : audit 2021-11-03T17:15:07.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:07 smithi027 conmon[32322]: [DBG] from='mgr.? 172.21.15.115:0/3320853155' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi115.tcyodz/key"}]: dispatch 2021-11-03T17:15:07.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:07 smithi027 conmon[32322]: audit 2021-11-03T17:15:07. 2021-11-03T17:15:07.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:07 smithi027 conmon[32322]: 303002+0000 mon.smithi115 (mon.1) 63 : audit [DBG] 2021-11-03T17:15:07.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:07 smithi027 conmon[32322]: from='mgr.? 172.21.15.115:0/3320853155' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-11-03T17:15:07.901 INFO:tasks.workunit.client.0.smithi027.stdout:6/920: sync 2021-11-03T17:15:07.905 INFO:tasks.workunit.client.0.smithi027.stdout:6/921: truncate dc/d16/d30/f32 1189644 0 2021-11-03T17:15:07.909 INFO:tasks.workunit.client.0.smithi027.stdout:6/922: mknod dc/d82/d57/d96/dd4/c128 1 2021-11-03T17:15:07.952 INFO:tasks.workunit.client.0.smithi027.stdout:6/923: write dc/d16/db2/fc6 [798873,62871] 0 2021-11-03T17:15:07.960 INFO:tasks.workunit.client.0.smithi027.stdout:6/924: rename dc/d82/d57/dad/dae/db1/d34/f8a to dc/d37/f129 0 2021-11-03T17:15:07.968 INFO:tasks.workunit.client.0.smithi027.stdout:6/925: write dc/d82/d57/dad/dae/f103 [325024,98517] 0 2021-11-03T17:15:07.969 INFO:tasks.workunit.client.0.smithi027.stdout:6/926: rmdir dc/d16/d9a 39 2021-11-03T17:15:07.971 INFO:tasks.workunit.client.0.smithi027.stdout:6/927: symlink dc/d82/d57/dad/dae/db1/l12a 0 2021-11-03T17:15:07.973 INFO:tasks.workunit.client.0.smithi027.stdout:6/928: creat dc/d16/d9a/f12b x:0 0 0 2021-11-03T17:15:07.974 INFO:tasks.workunit.client.0.smithi027.stdout:6/929: fsync dc/d82/d57/fc5 0 2021-11-03T17:15:07.975 INFO:tasks.workunit.client.0.smithi027.stdout:6/930: readlink dc/d16/l6d 0 2021-11-03T17:15:07.977 INFO:tasks.workunit.client.0.smithi027.stdout:6/931: rename dc/d82 to dc/d82/d57/d96/d107/d12c 22 2021-11-03T17:15:07.981 INFO:tasks.workunit.client.0.smithi027.stdout:6/932: creat dc/d82/d57/d9b/f12d x:0 0 0 2021-11-03T17:15:07.983 INFO:tasks.workunit.client.0.smithi027.stdout:6/933: creat dc/d82/d57/d96/d107/f12e x:0 0 0 2021-11-03T17:15:07.983 INFO:tasks.workunit.client.0.smithi027.stdout:6/934: readlink dc/d16/l6d 0 2021-11-03T17:15:07.984 INFO:tasks.workunit.client.0.smithi027.stdout:6/935: chown dc/d82/d57/d96/lcb 174461345 1 2021-11-03T17:15:07.989 INFO:tasks.workunit.client.0.smithi027.stdout:6/936: fdatasync dc/d16/d1b/d3c/d51/fe5 0 2021-11-03T17:15:08.008 INFO:tasks.workunit.client.0.smithi027.stdout:6/937: dwrite dc/d16/d1b/f28 [0,4194304] 0 2021-11-03T17:15:08.017 INFO:tasks.workunit.client.0.smithi027.stdout:6/938: write dc/d16/d1b/d24/d7e/f10a [991315,86511] 0 2021-11-03T17:15:08.019 INFO:tasks.workunit.client.0.smithi027.stdout:6/939: fdatasync dc/f4d 0 2021-11-03T17:15:08.024 INFO:tasks.workunit.client.0.smithi027.stdout:6/940: write dc/d82/d87/fb9 [395789,75804] 0 2021-11-03T17:15:08.028 INFO:tasks.workunit.client.0.smithi027.stdout:6/941: rename dc/d16/d1b/dda/ldd to dc/d16/d1b/d3c/l12f 0 2021-11-03T17:15:08.029 INFO:tasks.workunit.client.0.smithi027.stdout:6/942: chown dc/d16/d1b/d3c/d51/f78 88545 1 2021-11-03T17:15:08.030 INFO:tasks.workunit.client.0.smithi027.stdout:6/943: read - dc/d16/d9a/fc7 zero size 2021-11-03T17:15:08.052 INFO:tasks.workunit.client.0.smithi027.stdout:6/944: dwrite dc/d82/d57/f60 [4194304,4194304] 0 2021-11-03T17:15:08.054 INFO:tasks.workunit.client.0.smithi027.stdout:6/945: creat dc/d16/d1b/d24/f130 x:0 0 0 2021-11-03T17:15:08.056 INFO:tasks.workunit.client.0.smithi027.stdout:6/946: getdents dc/d16/d1b/d24/d7e 0 2021-11-03T17:15:08.063 INFO:tasks.workunit.client.0.smithi027.stdout:6/947: mkdir dc/d82/d57/dad/dae/d131 0 2021-11-03T17:15:08.064 INFO:tasks.workunit.client.0.smithi027.stdout:6/948: chown dc/d16/d1b/dda/ff2 7213 1 2021-11-03T17:15:08.098 INFO:tasks.workunit.client.0.smithi027.stdout:6/949: truncate dc/d16/d1b/d24/f46 1011679 0 2021-11-03T17:15:08.099 INFO:tasks.workunit.client.0.smithi027.stdout:6/950: read dc/d82/d57/dad/dae/fd7 [915349,68411] 0 2021-11-03T17:15:08.103 INFO:tasks.workunit.client.0.smithi027.stdout:6/951: stat dc/d16/d1b/d3c/d51/df9/de7/l10e 0 2021-11-03T17:15:08.105 INFO:tasks.workunit.client.0.smithi027.stdout:6/952: mkdir dc/d82/d57/d96/d107/d132 0 2021-11-03T17:15:08.125 INFO:tasks.workunit.client.0.smithi027.stdout:6/953: dwrite dc/d82/f5c [4194304,4194304] 0 2021-11-03T17:15:08.129 INFO:tasks.workunit.client.0.smithi027.stdout:6/954: link dc/d16/d62/lf4 dc/dff/l133 0 2021-11-03T17:15:08.129 INFO:tasks.workunit.client.0.smithi027.stdout:6/955: dread - dc/d16/d1b/d3c/d51/f102 zero size 2021-11-03T17:15:08.132 INFO:tasks.workunit.client.0.smithi027.stdout:6/956: mkdir dc/d16/d1b/d3c/d51/df9/d134 0 2021-11-03T17:15:08.132 INFO:tasks.workunit.client.0.smithi027.stdout:6/957: write dc/d82/d57/d9b/f12d [534319,85593] 0 2021-11-03T17:15:08.140 INFO:tasks.workunit.client.0.smithi027.stdout:6/958: write dc/d16/d1b/d3c/d51/df9/de7/f104 [478163,24719] 0 2021-11-03T17:15:08.141 INFO:tasks.workunit.client.0.smithi027.stdout:6/959: mknod dc/d82/d87/dec/c135 1 2021-11-03T17:15:08.143 INFO:tasks.workunit.client.0.smithi027.stdout:6/960: symlink dc/d82/d57/dad/l136 0 2021-11-03T17:15:08.152 INFO:tasks.workunit.client.0.smithi027.stdout:6/961: dread dc/d16/d1b/d3c/f44 [8388608,4194304] 0 2021-11-03T17:15:08.153 INFO:tasks.workunit.client.0.smithi027.stdout:6/962: truncate dc/d16/d1b/d24/d7e/f118 904562 0 2021-11-03T17:15:08.155 INFO:tasks.workunit.client.0.smithi027.stdout:6/963: mkdir dc/d16/d1b/dda/d137 0 2021-11-03T17:15:08.174 INFO:tasks.workunit.client.0.smithi027.stdout:6/964: dwrite dc/d82/f11e [0,4194304] 0 2021-11-03T17:15:08.177 INFO:tasks.workunit.client.0.smithi027.stdout:6/965: creat dc/d16/d1b/d3c/d51/f138 x:0 0 0 2021-11-03T17:15:08.179 INFO:tasks.workunit.client.0.smithi027.stdout:6/966: fdatasync dc/d37/dcc/f93 0 2021-11-03T17:15:08.197 INFO:tasks.workunit.client.0.smithi027.stdout:6/967: dwrite dc/d82/d87/f8b [0,4194304] 0 2021-11-03T17:15:08.201 INFO:tasks.workunit.client.0.smithi027.stdout:6/968: rmdir dc/d82/d57/d96/d107/d132 0 2021-11-03T17:15:08.206 INFO:tasks.workunit.client.0.smithi027.stdout:6/969: write dc/d37/f129 [450046,117195] 0 2021-11-03T17:15:08.226 INFO:tasks.workunit.client.0.smithi027.stdout:6/970: dwrite dc/d37/fbd [0,4194304] 0 2021-11-03T17:15:08.233 INFO:tasks.workunit.client.0.smithi027.stdout:6/971: write dc/d82/d57/fc3 [1283106,28092] 0 2021-11-03T17:15:08.235 INFO:tasks.workunit.client.0.smithi027.stdout:6/972: creat dc/d16/d1b/d3c/d51/d5f/f139 x:0 0 0 2021-11-03T17:15:08.235 INFO:tasks.workunit.client.0.smithi027.stdout:6/973: chown dc/d16/d62 145 1 2021-11-03T17:15:08.238 INFO:tasks.workunit.client.0.smithi027.stdout:6/974: link dc/d16/d30/l123 dc/d82/d116/l13a 0 2021-11-03T17:15:08.241 INFO:tasks.workunit.client.0.smithi027.stdout:6/975: mkdir dc/d82/d57/dad/d113/d13b 0 2021-11-03T17:15:08.242 INFO:tasks.workunit.client.0.smithi027.stdout:6/976: fsync dc/d16/d1b/dda/ff2 0 2021-11-03T17:15:08.244 INFO:tasks.workunit.client.0.smithi027.stdout:6/977: truncate dc/d16/f2c 1078797 0 2021-11-03T17:15:08.252 INFO:tasks.workunit.client.0.smithi027.stdout:6/978: write dc/d16/d1b/d3c/d51/d5f/f109 [1021707,74095] 0 2021-11-03T17:15:08.253 INFO:tasks.workunit.client.0.smithi027.stdout:6/979: truncate dc/d16/d30/f38 3837363 0 2021-11-03T17:15:08.261 INFO:tasks.workunit.client.0.smithi027.stdout:6/980: write dc/d16/d1b/d24/f2a [8340443,32980] 0 2021-11-03T17:15:08.288 INFO:tasks.workunit.client.0.smithi027.stdout:6/981: dwrite dc/d16/d1b/d24/d7e/f118 [0,4194304] 0 2021-11-03T17:15:08.290 INFO:tasks.workunit.client.0.smithi027.stdout:6/982: read dc/d37/f129 [895426,110054] 0 2021-11-03T17:15:08.292 INFO:tasks.workunit.client.0.smithi027.stdout:6/983: symlink dc/d16/d1b/dda/l13c 0 2021-11-03T17:15:08.297 INFO:tasks.workunit.client.0.smithi027.stdout:6/984: rename dc/d16/d1b/d3c/l3e to dc/d82/d57/dad/d113/d13b/l13d 0 2021-11-03T17:15:08.298 INFO:tasks.workunit.client.0.smithi027.stdout:6/985: creat dc/d82/d57/d96/d107/f13e x:0 0 0 2021-11-03T17:15:08.321 INFO:tasks.workunit.client.0.smithi027.stdout:7/802: sync 2021-11-03T17:15:08.329 INFO:tasks.workunit.client.0.smithi027.stdout:6/986: dwrite dc/d82/d57/dad/fef [0,4194304] 0 2021-11-03T17:15:08.366 INFO:tasks.workunit.client.0.smithi027.stdout:6/987: mkdir dc/d16/d30/d13f 0 2021-11-03T17:15:08.367 INFO:tasks.workunit.client.0.smithi027.stdout:7/803: rmdir d0/d2 39 2021-11-03T17:15:08.378 INFO:tasks.workunit.client.0.smithi027.stdout:6/988: unlink dc/d82/d57/dad/dae/db1/d34/l4a 0 2021-11-03T17:15:08.387 INFO:tasks.workunit.client.0.smithi027.stdout:7/804: write d0/f4c [168338,93307] 0 2021-11-03T17:15:08.388 INFO:tasks.workunit.client.0.smithi027.stdout:6/989: fdatasync dc/d16/d1b/d3c/d51/ff0 0 2021-11-03T17:15:08.396 INFO:tasks.workunit.client.0.smithi027.stdout:7/805: unlink d0/d2/d1c/f79 0 2021-11-03T17:15:08.398 INFO:tasks.workunit.client.0.smithi027.stdout:6/990: rename dc/d16/d1b/d24/d7e/db3/de8/l106 to dc/d16/d62/l140 0 2021-11-03T17:15:08.399 INFO:tasks.workunit.client.0.smithi027.stdout:6/991: creat dc/d16/d1b/d24/f141 x:0 0 0 2021-11-03T17:15:08.400 INFO:tasks.workunit.client.0.smithi027.stdout:7/806: fdatasync d0/d2/d5e/f32 0 2021-11-03T17:15:08.403 INFO:tasks.workunit.client.0.smithi027.stdout:6/992: creat dc/d16/d1b/d24/d7e/db3/f142 x:0 0 0 2021-11-03T17:15:08.405 INFO:tasks.workunit.client.0.smithi027.stdout:6/993: mknod dc/d82/d116/c143 1 2021-11-03T17:15:08.407 INFO:tasks.workunit.client.0.smithi027.stdout:6/994: readlink dc/d16/d1b/d3c/d51/ldb 0 2021-11-03T17:15:08.408 INFO:tasks.workunit.client.0.smithi027.stdout:6/995: dread - dc/d16/d1b/d3c/d51/ff0 zero size 2021-11-03T17:15:08.408 INFO:tasks.workunit.client.0.smithi027.stdout:6/996: truncate dc/d16/d1b/d24/d7e/f118 5225046 0 2021-11-03T17:15:08.412 INFO:tasks.workunit.client.0.smithi027.stdout:7/807: dread - d0/d2/d10/d1e/d20/faa zero size 2021-11-03T17:15:08.420 INFO:tasks.workunit.client.0.smithi027.stdout:7/808: readlink d0/d2/d1c/d4d/d6e/d84/l5a 0 2021-11-03T17:15:08.420 INFO:tasks.workunit.client.0.smithi027.stdout:7/809: chown d0/d2/d10/d1e/de6 1 1 2021-11-03T17:15:08.425 INFO:tasks.workunit.client.0.smithi027.stdout:7/810: mkdir d0/d2/d10/d82/de4/d10a 0 2021-11-03T17:15:08.426 INFO:tasks.workunit.client.0.smithi027.stdout:6/997: dwrite dc/d16/d9a/f12b [0,4194304] 0 2021-11-03T17:15:08.427 INFO:tasks.workunit.client.0.smithi027.stdout:6/998: fsync dc/d16/d1b/d3c/f44 0 2021-11-03T17:15:08.433 INFO:tasks.workunit.client.0.smithi027.stdout:7/811: dread - d0/d2d/d2f/dae/dc8/fcf zero size 2021-11-03T17:15:08.441 INFO:tasks.workunit.client.0.smithi027.stdout:7/812: write d0/d2/f5 [1809542,7439] 0 2021-11-03T17:15:08.444 INFO:tasks.workunit.client.0.smithi027.stdout:7/813: dread - d0/d2/d10/d1e/f24 zero size 2021-11-03T17:15:08.454 INFO:tasks.workunit.client.0.smithi027.stdout:7/814: write d0/d2/d10/d35/d96/fb3 [206561,11765] 0 2021-11-03T17:15:08.458 INFO:tasks.workunit.client.0.smithi027.stdout:7/815: rename d0/l14 to d0/d2/d1c/d4d/d6e/d84/l10b 0 2021-11-03T17:15:08.463 INFO:tasks.workunit.client.0.smithi027.stdout:7/816: mkdir d0/d2/d1c/d4d/d6e/d8b/dea/d10c 0 2021-11-03T17:15:08.463 INFO:tasks.workunit.client.0.smithi027.stdout:7/817: stat d0/d2/d1c/f61 0 2021-11-03T17:15:08.467 INFO:tasks.workunit.client.0.smithi027.stdout:7/818: unlink d0/d2/d1c/f27 0 2021-11-03T17:15:08.474 INFO:tasks.workunit.client.0.smithi027.stdout:7/819: rename d0/f16 to d0/d2/d10/d82/df2/dab/f10d 0 2021-11-03T17:15:08.475 INFO:tasks.workunit.client.0.smithi027.stdout:7/820: chown d0/f25 435561921 1 2021-11-03T17:15:08.476 INFO:tasks.workunit.client.0.smithi027.stdout:7/821: mknod d0/d2/d10/d35/d96/c10e 1 2021-11-03T17:15:08.482 INFO:tasks.workunit.client.0.smithi027.stdout:7/822: rename d0/d2/d10/d1e/f5c to d0/d2d/d58/f10f 0 2021-11-03T17:15:08.490 INFO:tasks.workunit.client.0.smithi027.stdout:7/823: dread d0/d2/d5e/f32 [0,4194304] 0 2021-11-03T17:15:08.494 INFO:tasks.workunit.client.0.smithi027.stdout:7/824: unlink d0/d2/d10/d82/f103 0 2021-11-03T17:15:08.498 INFO:tasks.workunit.client.0.smithi027.stdout:7/825: link d0/d2/d10/d1e/d20/faa d0/d2/d5e/f110 0 2021-11-03T17:15:08.501 INFO:tasks.workunit.client.0.smithi027.stdout:7/826: creat d0/d2/d1c/d4d/d6e/d8b/f111 x:0 0 0 2021-11-03T17:15:08.504 INFO:tasks.workunit.client.0.smithi027.stdout:7/827: mkdir d0/d2/d10/d82/d112 0 2021-11-03T17:15:08.504 INFO:tasks.workunit.client.0.smithi027.stdout:7/828: dread - d0/d2/d5e/f110 zero size 2021-11-03T17:15:08.506 INFO:tasks.workunit.client.0.smithi027.stdout:7/829: fsync d0/d2d/d66/f70 0 2021-11-03T17:15:08.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:08 smithi027 conmon[32322]: cluster 2021-11-03T17:15:06.641964+0000 2021-11-03T17:15:08.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:08 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14622) 12 : cluster [DBG] pgmap v7: 65 pgs: 65 active+clean; 3.3 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail 2021-11-03T17:15:08.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:08 smithi027 conmon[32322]: cluster 2021-11-03T17:15:07.313558+0000 mon.smithi027 (mon.0) 689 : cluster [DBG] mds.? [v2:172.21.15.115:6826/3367380088,v1:172.21.15.115:6827/3367380088] up:reconnect 2021-11-03T17:15:08.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:08 smithi027 conmon[32322]: cluster 2021-11-03T17:15:07.313606+0000 mon.smithi027 (mon.0) 690 : cluster [DBG] fsmap cephfs:2/2 {0=cephfs.smithi115.semywj=up:reconnect,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:15:08.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:08 smithi027 conmon[32322]: cluster 2021-11-03T17:15:07.333220+0000 mds.cephfs.smithi115.semywj (mds.0) 1 : cluster [DBG] reconnect by client.24331 v1:192.168.0.1:0/2120483524 after 0 2021-11-03T17:15:08.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:08 smithi027 conmon[32322]: cluster 2021-11-03T17:15:07.357450+0000 mds.cephfs.smithi115.semywj (mds.0) 2 : cluster [DBG] reconnect by client.14486 v1:192.168.0.1:0/2105322717 after 0.0239995 2021-11-03T17:15:08.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:08 smithi027 conmon[32322]: cluster 2021-11-03T17:15:08.070854+0000 mon.smithi027 (mon.0) 691 : cluster [DBG] mgrmap e28: smithi027.ufhhvb(active, since 8s), standbys: smithi115.tcyodz 2021-11-03T17:15:08.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:08 smithi027 conmon[32322]: audit 2021-11-03T17:15:08.071032+0000 mon.smithi027 (mon.0) 692 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mgr metadata", "who": "smithi115.tcyodz", "id": "smithi115.tcyodz"}]: dispatch 2021-11-03T17:15:08.521 INFO:tasks.workunit.client.0.smithi027.stdout:7/830: write d0/d2/d10/d82/df2/f102 [662729,7152] 0 2021-11-03T17:15:08.524 INFO:tasks.workunit.client.0.smithi027.stdout:7/831: symlink d0/d2d/d2f/dae/dc8/l113 0 2021-11-03T17:15:08.525 INFO:tasks.workunit.client.0.smithi027.stdout:7/832: chown d0/d2d/fc5 770 1 2021-11-03T17:15:08.527 INFO:tasks.workunit.client.0.smithi027.stdout:7/833: creat d0/d2d/d58/f114 x:0 0 0 2021-11-03T17:15:08.555 INFO:tasks.workunit.client.0.smithi027.stdout:7/834: dwrite d0/d2d/d66/f98 [0,4194304] 0 2021-11-03T17:15:08.566 INFO:tasks.workunit.client.0.smithi027.stdout:7/835: dread d0/d2/fa3 [0,4194304] 0 2021-11-03T17:15:08.567 INFO:tasks.workunit.client.0.smithi027.stdout:7/836: rmdir d0/d2d/d2f/dae/dc8 39 2021-11-03T17:15:08.567 INFO:tasks.workunit.client.0.smithi027.stdout:7/837: dread - d0/d2/d10/d1e/d20/faa zero size 2021-11-03T17:15:08.576 INFO:tasks.workunit.client.0.smithi027.stdout:7/838: dread d0/d2/d10/d1e/d20/f43 [0,4194304] 0 2021-11-03T17:15:08.601 INFO:tasks.workunit.client.0.smithi027.stdout:7/839: dwrite d0/d2/d10/d82/de4/f106 [0,4194304] 0 2021-11-03T17:15:08.627 INFO:tasks.workunit.client.0.smithi027.stdout:7/840: chown d0/d2/d5e/f32 45 1 2021-11-03T17:15:08.632 INFO:tasks.workunit.client.0.smithi027.stdout:7/841: symlink d0/d2/l115 0 2021-11-03T17:15:08.634 INFO:tasks.workunit.client.0.smithi027.stdout:7/842: chown d0/d2/d5e/ff6 1 1 2021-11-03T17:15:08.636 INFO:tasks.workunit.client.0.smithi027.stdout:7/843: dread d0/d2/d10/f95 [0,4194304] 0 2021-11-03T17:15:08.638 INFO:tasks.workunit.client.0.smithi027.stdout:7/844: read d0/d2d/d58/f69 [2407065,32218] 0 2021-11-03T17:15:08.639 INFO:tasks.workunit.client.0.smithi027.stdout:7/845: dread - d0/d2d/ff8 zero size 2021-11-03T17:15:08.646 INFO:tasks.workunit.client.0.smithi027.stdout:7/846: write d0/d2/d1c/faf [864249,91608] 0 2021-11-03T17:15:08.651 INFO:tasks.workunit.client.0.smithi027.stdout:7/847: write d0/d2/d1f/fc2 [3022389,76840] 0 2021-11-03T17:15:08.651 INFO:tasks.workunit.client.0.smithi027.stdout:7/848: dread - d0/d2/d10/d1e/d20/faa zero size 2021-11-03T17:15:08.657 INFO:tasks.workunit.client.0.smithi027.stdout:7/849: write d0/d2/d10/d1e/d20/d74/f7b [295089,7561] 0 2021-11-03T17:15:08.678 INFO:tasks.workunit.client.0.smithi027.stdout:7/850: dwrite d0/d2/fd [0,4194304] 0 2021-11-03T17:15:08.682 INFO:tasks.workunit.client.0.smithi027.stdout:7/851: creat d0/d2d/d2f/dae/dc8/df9/f116 x:0 0 0 2021-11-03T17:15:08.682 INFO:tasks.workunit.client.0.smithi027.stdout:9/997: sync 2021-11-03T17:15:08.704 INFO:tasks.workunit.client.0.smithi027.stdout:9/998: creat db/d1b/d2b/d41/dad/dbb/d12a/f173 x:0 0 0 2021-11-03T17:15:08.705 INFO:tasks.workunit.client.0.smithi027.stdout:9/999: stat db/d1b/d2b/d41/dad/f159 0 2021-11-03T17:15:08.706 INFO:tasks.workunit.client.0.smithi027.stdout:7/852: dwrite d0/d2/d1c/d4d/d6e/fd6 [4194304,4194304] 0 2021-11-03T17:15:08.714 INFO:tasks.workunit.client.0.smithi027.stdout:7/853: rename d0/l1 to d0/d2/d1c/d4d/d6e/d84/dad/l117 0 2021-11-03T17:15:08.720 INFO:tasks.workunit.client.0.smithi027.stdout:7/854: link d0/lc d0/d2/d10/d82/de4/dfd/dfe/l118 0 2021-11-03T17:15:08.721 INFO:tasks.workunit.client.0.smithi027.stdout:7/855: mknod d0/d2d/d2f/c119 1 2021-11-03T17:15:08.724 INFO:tasks.workunit.client.0.smithi027.stdout:7/856: link d0/lc d0/d2d/d58/dba/l11a 0 2021-11-03T17:15:08.729 INFO:tasks.workunit.client.0.smithi027.stdout:7/857: rename d0/d2/d5e/d36/f63 to d0/d2/d1c/d4d/d6e/d8b/dea/d10c/f11b 0 2021-11-03T17:15:08.730 INFO:tasks.workunit.client.0.smithi027.stdout:7/858: read d0/d2/d10/d35/d96/fb3 [1666257,130036] 0 2021-11-03T17:15:08.731 INFO:tasks.workunit.client.0.smithi027.stdout:7/859: stat d0/d2d/d53 0 2021-11-03T17:15:08.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:08 smithi115 conmon[34670]: cluster 2021-11-03T17:15:06.641964+0000 mgr.smithi027.ufhhvb (mgr.14622) 12 : cluster [DBG] pgmap v7: 65 pgs: 65 active+clean; 3.3 GiB data, 9.8 GiB used, 527 GiB / 536 GiB avail 2021-11-03T17:15:08.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:08 smithi115 conmon[34670]: cluster 2021-11-03T17:15:07 2021-11-03T17:15:08.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:08 smithi115 conmon[34670]: .313558+0000 mon.smithi027 (mon.0) 689 : cluster [DBG] mds.? [v2:172.21.15.115:6826/3367380088,v1:172.21.15.115:6827/3367380088] up:reconnect 2021-11-03T17:15:08.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:08 smithi115 conmon[34670]: cluster 2021-11-03T17:15:07.313606+0000 mon.smithi027 (mon.0) 690 : cluster [DBG] fsmap cephfs:2/2 {0=cephfs.smithi115.semywj=up:reconnect,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:15:08.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:08 smithi115 conmon[34670]: cluster 2021-11-03T17:15:07. 2021-11-03T17:15:08.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:08 smithi115 conmon[34670]: 333220+0000 mds.cephfs.smithi115.semywj (mds.0) 1 : cluster [DBG] reconnect by client.24331 v1:192.168.0.1:0/2120483524 after 0 2021-11-03T17:15:08.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:08 smithi115 conmon[34670]: cluster 2021-11-03T17:15: 2021-11-03T17:15:08.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:08 smithi115 conmon[34670]: 07.357450+0000 mds.cephfs.smithi115.semywj (mds.0) 2 2021-11-03T17:15:08.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:08 smithi115 conmon[34670]: : cluster [DBG] reconnect by client.14486 v1:192.168.0.1:0/2105322717 after 0.0239995 2021-11-03T17:15:08.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:08 smithi115 conmon[34670]: cluster 2021-11- 2021-11-03T17:15:08.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:08 smithi115 conmon[34670]: 03T17:15:08.070854+0000 mon.smithi027 ( 2021-11-03T17:15:08.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:08 smithi115 conmon[34670]: mon.0) 691 : cluster [DBG] mgrmap e28: smithi027.ufhhvb(active, since 8s), standbys: smithi115.tcyodz 2021-11-03T17:15:08.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:08 smithi115 conmon[34670]: audit 2021 2021-11-03T17:15:08.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:08 smithi115 conmon[34670]: -11-03T17:15:08.071032 2021-11-03T17:15:08.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:08 smithi115 conmon[34670]: +0000 mon.smithi027 (mon.0) 692 : audit [DBG] 2021-11-03T17:15:08.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:08 smithi115 conmon[34670]: from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mgr metadata", "who": "smithi115.tcyodz", "id": "smithi115.tcyodz"}]: dispatch 2021-11-03T17:15:08.738 INFO:tasks.workunit.client.0.smithi027.stdout:7/860: dread d0/d2/d5e/d36/fb0 [0,4194304] 0 2021-11-03T17:15:08.742 INFO:tasks.workunit.client.0.smithi027.stdout:7/861: dread d0/d2/d5e/f32 [0,4194304] 0 2021-11-03T17:15:08.743 INFO:tasks.workunit.client.0.smithi027.stdout:7/862: readlink d0/d2/d1c/d4d/d6e/d84/l5a 0 2021-11-03T17:15:08.743 INFO:tasks.workunit.client.0.smithi027.stdout:7/863: read - d0/d2/d5e/f110 zero size 2021-11-03T17:15:09.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:09 smithi115 conmon[34670]: cluster 2021-11-03T17:15:08.318995+0000 mon.smithi027 (mon 2021-11-03T17:15:09.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:09 smithi115 conmon[34670]: .0) 693 : cluster [DBG] mds.? [v2:172.21.15.115:6826/3367380088,v1:172.21.15.115:6827/3367380088] up:rejoin 2021-11-03T17:15:09.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:09 smithi115 conmon[34670]: cluster 2021-11-03T17:15:08.319065+0000 mon.smithi027 (mon.0) 694 2021-11-03T17:15:09.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:09 smithi115 conmon[34670]: : cluster [DBG] fsmap cephfs:2/2 {0=cephfs.smithi115.semywj=up:rejoin,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:15:09.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:09 smithi027 conmon[32322]: cluster 2021-11-03T17:15:08.318995+0000 mon.smithi027 (mon.0) 693 : cluster [DBG] mds.? [v2:172.21.15.115:6826/3367380088,v1:172.21.15.115:6827/3367380088] up:rejoin 2021-11-03T17:15:09.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:09 smithi027 conmon[32322]: cluster 2021-11-03T17:15:08.319065+0000 mon.smithi027 (mon.0) 694 : cluster [DBG] fsmap cephfs:2/2 {0=cephfs.smithi115.semywj=up:rejoin,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:15:10.095 INFO:tasks.workunit.client.0.smithi027.stdout:6/999: sync 2021-11-03T17:15:10.095 INFO:tasks.workunit.client.0.smithi027.stdout:7/864: sync 2021-11-03T17:15:10.095 INFO:tasks.workunit.client.0.smithi027.stdout:7/865: fdatasync d0/d2/d10/d1e/d20/f43 0 2021-11-03T17:15:10.098 INFO:tasks.workunit.client.0.smithi027.stdout:7/866: unlink d0/d2/f5 0 2021-11-03T17:15:10.100 INFO:tasks.workunit.client.0.smithi027.stdout:7/867: creat d0/d2/d10/d1e/d20/d74/f11c x:0 0 0 2021-11-03T17:15:10.103 INFO:tasks.workunit.client.0.smithi027.stdout:7/868: creat d0/f11d x:0 0 0 2021-11-03T17:15:10.105 INFO:tasks.workunit.client.0.smithi027.stdout:7/869: truncate d0/d2/d1c/d4d/d6e/fd6 7269249 0 2021-11-03T17:15:10.108 INFO:tasks.workunit.client.0.smithi027.stdout:7/870: mkdir d0/d2/d10/d82/df2/dab/d11e 0 2021-11-03T17:15:10.115 INFO:tasks.workunit.client.0.smithi027.stdout:7/871: write d0/d2d/ff8 [378709,31610] 0 2021-11-03T17:15:10.115 INFO:tasks.workunit.client.0.smithi027.stdout:7/872: chown d0/d2/d10/d1e/d20/fdd 114672 1 2021-11-03T17:15:10.116 INFO:tasks.workunit.client.0.smithi027.stdout:7/873: rmdir d0/d2/d5e/d36 39 2021-11-03T17:15:10.118 INFO:tasks.workunit.client.0.smithi027.stdout:7/874: readlink d0/d2/d5e/d36/d3a/la4 0 2021-11-03T17:15:10.141 INFO:tasks.workunit.client.0.smithi027.stdout:7/875: dwrite d0/d2/ff1 [0,4194304] 0 2021-11-03T17:15:10.142 INFO:tasks.workunit.client.0.smithi027.stdout:7/876: fsync d0/d2d/fc5 0 2021-11-03T17:15:10.147 INFO:tasks.workunit.client.0.smithi027.stdout:7/877: link d0/d2/fa3 d0/d2/d1f/f11f 0 2021-11-03T17:15:10.147 INFO:tasks.workunit.client.0.smithi027.stdout:7/878: mknod d0/d2/d10/d82/df2/dab/d11e/c120 1 2021-11-03T17:15:10.148 INFO:tasks.workunit.client.0.smithi027.stdout:7/879: truncate d0/d2/d5e/ff6 1038451 0 2021-11-03T17:15:10.150 INFO:tasks.workunit.client.0.smithi027.stdout:7/880: mknod d0/db6/c121 1 2021-11-03T17:15:10.151 INFO:tasks.workunit.client.0.smithi027.stdout:7/881: chown d0/d2/d10/d1e/d20/d74/ld2 6831 1 2021-11-03T17:15:10.153 INFO:tasks.workunit.client.0.smithi027.stdout:7/882: read d0/f13 [3528692,128449] 0 2021-11-03T17:15:10.153 INFO:tasks.workunit.client.0.smithi027.stdout:7/883: mknod d0/d2d/d2f/dae/dd8/c122 1 2021-11-03T17:15:10.155 INFO:tasks.workunit.client.0.smithi027.stdout:7/884: fdatasync d0/d2/d10/d1e/d20/f26 0 2021-11-03T17:15:10.158 INFO:tasks.workunit.client.0.smithi027.stdout:7/885: fsync d0/d2/d1c/d4d/fd5 0 2021-11-03T17:15:10.161 INFO:tasks.workunit.client.0.smithi027.stdout:7/886: rename d0/d2d/d2f/f39 to d0/d2d/d66/f123 0 2021-11-03T17:15:10.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:10 smithi115 conmon[34670]: cluster 2021-11-03T17:15:08.642494+0000 mgr.smithi027.ufhhvb (mgr.14622) 13 2021-11-03T17:15:10.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:10 smithi115 conmon[34670]: : cluster [DBG] pgmap v8: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 16 MiB/s rd, 16 MiB/s wr, 48 op/s 2021-11-03T17:15:10.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:10 smithi115 conmon[34670]: cluster 2021-11-03T17:15:09.326475+0000 mon.smithi027 (mon.0) 695 : cluster [DBG] mds.? [v2:172.21.15.115:6826/3367380088,v1:172.21.15.115:6827/3367380088] up:clientreplay 2021-11-03T17:15:10.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:10 smithi115 conmon[34670]: cluster 2021-11-03T17:15:09.326552+0000 mon.smithi027 (mon.0) 696 : cluster [DBG] fsmap cephfs:2/2 {0=cephfs.smithi115.semywj=up:clientreplay,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:15:10.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:10 smithi115 conmon[34670]: cluster 2021-11-03T17:15:09.601035+0000 mon.smithi027 (mon.0) 697 : cluster [INF] daemon mds.cephfs.smithi115.semywj is now active in filesystem cephfs as rank 0 2021-11-03T17:15:10.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:10 smithi027 conmon[32322]: cluster 2021-11-03T17:15:08.642494+0000 2021-11-03T17:15:10.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:10 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14622) 13 : cluster [DBG] pgmap v8: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 16 MiB/s rd, 16 MiB/s wr, 48 op/s 2021-11-03T17:15:10.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:10 smithi027 conmon[32322]: cluster 2021-11-03T17:15:09.326475 2021-11-03T17:15:10.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:10 smithi027 conmon[32322]: +0000 mon.smithi027 (mon.0) 695 : cluster [DBG] mds.? [v2:172.21.15.115:6826/3367380088,v1:172.21.15.115:6827/3367380088] up:clientreplay 2021-11-03T17:15:10.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:10 smithi027 conmon[32322]: cluster 2021-11 2021-11-03T17:15:10.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:10 smithi027 conmon[32322]: -03T17:15:09.326552+0000 mon.smithi027 (mon.0) 2021-11-03T17:15:10.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:10 smithi027 conmon[32322]: 696 : cluster [DBG] fsmap cephfs:2/2 {0=cephfs.smithi115.semywj=up:clientreplay,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:15:10.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:10 smithi027 conmon[32322]: cluster 2021- 2021-11-03T17:15:10.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:10 smithi027 conmon[32322]: 11-03T17:15:09.601035+0000 2021-11-03T17:15:10.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:10 smithi027 conmon[32322]: mon.smithi027 (mon.0) 697 : cluster [INF] daemon mds.cephfs.smithi115.semywj is now active in filesystem cephfs as rank 0 2021-11-03T17:15:10.771 INFO:tasks.workunit.client.1.smithi115.stdout:0/624: creat d9/d15/d1a/d1d/dcc/d46/d63/d70/fd2 x:0 0 0 2021-11-03T17:15:10.772 INFO:tasks.workunit.client.1.smithi115.stdout:7/436: symlink d3/da/d18/d73/l8b 0 2021-11-03T17:15:10.772 INFO:tasks.workunit.client.1.smithi115.stdout:6/441: mkdir d0/d85/d8f 0 2021-11-03T17:15:10.772 INFO:tasks.workunit.client.1.smithi115.stdout:5/636: symlink d1a/d28/d4c/d75/d68/lcd 0 2021-11-03T17:15:10.772 INFO:tasks.workunit.client.1.smithi115.stdout:4/646: creat d7/dc/d6a/d3c/dbf/fdb x:0 0 0 2021-11-03T17:15:10.773 INFO:tasks.workunit.client.1.smithi115.stdout:3/636: rename d13/d50/f4d to d13/d2a/d8a/d99/fd9 0 2021-11-03T17:15:10.788 INFO:tasks.workunit.client.1.smithi115.stdout:3/637: getdents d13/d2a/d8a/d99 0 2021-11-03T17:15:10.794 INFO:tasks.workunit.client.1.smithi115.stdout:4/647: mkdir d7/dc/d6a/d3c/dbf/ddc 0 2021-11-03T17:15:10.803 INFO:tasks.workunit.client.1.smithi115.stdout:0/625: getdents d9/db/d89 0 2021-11-03T17:15:10.809 INFO:tasks.workunit.client.1.smithi115.stdout:5/637: creat d1a/d28/d4c/d75/dae/fce x:0 0 0 2021-11-03T17:15:10.810 INFO:tasks.workunit.client.1.smithi115.stdout:4/648: chown d7/l9 3843016 1 2021-11-03T17:15:10.831 INFO:tasks.workunit.client.1.smithi115.stdout:0/626: truncate d9/d15/d1a/d1d/dcc/d46/d8c/f74 4409238 0 2021-11-03T17:15:10.847 INFO:tasks.workunit.client.1.smithi115.stdout:0/627: mkdir d9/d37/dd3 0 2021-11-03T17:15:10.907 INFO:tasks.workunit.client.1.smithi115.stdout:5/638: write d1a/d23/f61 [5054150,51054] 0 2021-11-03T17:15:10.908 INFO:tasks.workunit.client.1.smithi115.stdout:5/639: readlink d1a/d28/l4b 0 2021-11-03T17:15:10.908 INFO:tasks.workunit.client.1.smithi115.stdout:0/628: write d9/db/f4a [2329597,64962] 0 2021-11-03T17:15:10.949 INFO:tasks.workunit.client.1.smithi115.stdout:4/649: stat d7/dc/d6a/d3c/f3e 0 2021-11-03T17:15:10.950 INFO:tasks.workunit.client.1.smithi115.stdout:5/640: write d1a/d28/d4c/d63/f9c [2858241,19249] 0 2021-11-03T17:15:10.953 INFO:tasks.workunit.client.1.smithi115.stdout:6/442: dread d0/d1/d10/d13/d1c/d1e/f47 [0,4194304] 0 2021-11-03T17:15:10.955 INFO:tasks.workunit.client.1.smithi115.stdout:6/443: creat d0/d1/d10/d13/f90 x:0 0 0 2021-11-03T17:15:10.955 INFO:tasks.workunit.client.1.smithi115.stdout:6/444: write d0/d1/d10/d13/f90 [329447,46317] 0 2021-11-03T17:15:10.956 INFO:tasks.workunit.client.1.smithi115.stdout:4/650: fsync d7/dc/d1a/d21/fa1 0 2021-11-03T17:15:10.960 INFO:tasks.workunit.client.1.smithi115.stdout:4/651: chown d7/dc/d6a/d3c/dbf/dc4 976756 1 2021-11-03T17:15:10.960 INFO:tasks.workunit.client.1.smithi115.stdout:5/641: readlink d1a/d28/d4a/d65/db5/l70 0 2021-11-03T17:15:10.962 INFO:tasks.workunit.client.1.smithi115.stdout:0/629: dwrite d9/db/d89/fa1 [0,4194304] 0 2021-11-03T17:15:10.969 INFO:tasks.workunit.client.1.smithi115.stdout:5/642: fsync d1a/d28/d4c/d75/d83/f88 0 2021-11-03T17:15:10.971 INFO:tasks.workunit.client.1.smithi115.stdout:0/630: fsync d9/d15/d1a/d1d/dcc/d6e/f72 0 2021-11-03T17:15:10.973 INFO:tasks.workunit.client.1.smithi115.stdout:0/631: creat d9/d37/fd4 x:0 0 0 2021-11-03T17:15:10.974 INFO:tasks.workunit.client.1.smithi115.stdout:5/643: chown d1a/d28/d4c/d75/d68/da5/l9a 754749 1 2021-11-03T17:15:10.975 INFO:tasks.workunit.client.1.smithi115.stdout:5/644: truncate d1a/d28/d4c/d75/d83/f88 4541799 0 2021-11-03T17:15:10.977 INFO:tasks.workunit.client.1.smithi115.stdout:5/645: mknod d1a/d28/d4c/d75/d83/da1/ccf 1 2021-11-03T17:15:10.989 INFO:tasks.workunit.client.1.smithi115.stdout:5/646: write d1a/d28/d7b/d48/d7f/fb6 [330036,14077] 0 2021-11-03T17:15:11.004 INFO:tasks.workunit.client.1.smithi115.stdout:6/445: creat d0/d1/d10/d13/d1c/d1e/f91 x:0 0 0 2021-11-03T17:15:11.010 INFO:tasks.workunit.client.1.smithi115.stdout:4/652: write d7/d7a/f91 [975832,90454] 0 2021-11-03T17:15:11.012 INFO:tasks.workunit.client.1.smithi115.stdout:7/437: dwrite d3/f9 [8388608,4194304] 0 2021-11-03T17:15:11.013 INFO:tasks.workunit.client.1.smithi115.stdout:3/638: dread fc [0,4194304] 0 2021-11-03T17:15:11.014 INFO:tasks.workunit.client.1.smithi115.stdout:7/438: getdents d3/da/d18/d73 0 2021-11-03T17:15:11.017 INFO:tasks.workunit.client.1.smithi115.stdout:3/639: chown d13/d51/d40/f9d 334984 1 2021-11-03T17:15:11.025 INFO:tasks.workunit.client.1.smithi115.stdout:7/439: dread d3/da/f62 [0,4194304] 0 2021-11-03T17:15:11.025 INFO:tasks.workunit.client.1.smithi115.stdout:7/440: chown d3/da/f62 1876 1 2021-11-03T17:15:11.026 INFO:tasks.workunit.client.1.smithi115.stdout:7/441: readlink d3/da/d18/d73/l4e 0 2021-11-03T17:15:11.026 INFO:tasks.workunit.client.1.smithi115.stdout:3/640: dread d13/d3b/f58 [0,4194304] 0 2021-11-03T17:15:11.027 INFO:tasks.workunit.client.1.smithi115.stdout:7/442: symlink d3/da/l8c 0 2021-11-03T17:15:11.029 INFO:tasks.workunit.client.1.smithi115.stdout:7/443: rmdir d3/da/d18/d73/d33 39 2021-11-03T17:15:11.029 INFO:tasks.workunit.client.1.smithi115.stdout:7/444: chown d3/da/d18/d73/d35/d53 31495 1 2021-11-03T17:15:11.031 INFO:tasks.workunit.client.1.smithi115.stdout:7/445: read d3/da/d18/d73/d35/d53/f58 [3593231,29103] 0 2021-11-03T17:15:11.032 INFO:tasks.workunit.client.1.smithi115.stdout:3/641: creat d13/d2a/d60/dd3/fda x:0 0 0 2021-11-03T17:15:11.053 INFO:tasks.workunit.client.1.smithi115.stdout:6/446: creat d0/d1/d2/d5d/d61/f92 x:0 0 0 2021-11-03T17:15:11.055 INFO:tasks.workunit.client.1.smithi115.stdout:6/447: creat d0/d1/d6a/f93 x:0 0 0 2021-11-03T17:15:11.057 INFO:tasks.workunit.client.1.smithi115.stdout:6/448: symlink d0/d1/d2/d5d/d61/l94 0 2021-11-03T17:15:11.067 INFO:tasks.workunit.client.1.smithi115.stdout:4/653: write d7/dc/d6a/fd3 [479884,83480] 0 2021-11-03T17:15:11.068 INFO:tasks.workunit.client.1.smithi115.stdout:4/654: stat d7/dc/d6a/d3c/dc5/dcc 0 2021-11-03T17:15:11.074 INFO:tasks.workunit.client.1.smithi115.stdout:7/446: write d3/da/d18/d73/d35/f3d [691487,51736] 0 2021-11-03T17:15:11.075 INFO:tasks.workunit.client.1.smithi115.stdout:7/447: mknod d3/da/d18/d73/d52/c8d 1 2021-11-03T17:15:11.076 INFO:tasks.workunit.client.1.smithi115.stdout:7/448: fdatasync d3/da/d18/d73/d35/f75 0 2021-11-03T17:15:11.078 INFO:tasks.workunit.client.1.smithi115.stdout:7/449: getdents d3/da/d18/d73/d35 0 2021-11-03T17:15:11.079 INFO:tasks.workunit.client.1.smithi115.stdout:7/450: chown d3/da/d18/d73/d35/d53/l6f 27190786 1 2021-11-03T17:15:11.081 INFO:tasks.workunit.client.1.smithi115.stdout:7/451: mkdir d3/da/d8e 0 2021-11-03T17:15:11.097 INFO:tasks.workunit.client.1.smithi115.stdout:0/632: read d9/d15/d1a/d1d/dcc/d6e/fa3 [3669538,48108] 0 2021-11-03T17:15:11.098 INFO:tasks.workunit.client.1.smithi115.stdout:3/642: dwrite d13/d2a/d8a/fa7 [0,4194304] 0 2021-11-03T17:15:11.100 INFO:tasks.workunit.client.1.smithi115.stdout:3/643: fsync d13/f4c 0 2021-11-03T17:15:11.103 INFO:tasks.workunit.client.1.smithi115.stdout:3/644: dread d13/d2a/f71 [0,4194304] 0 2021-11-03T17:15:11.105 INFO:tasks.workunit.client.1.smithi115.stdout:3/645: creat d13/d35/fdb x:0 0 0 2021-11-03T17:15:11.108 INFO:tasks.workunit.client.1.smithi115.stdout:3/646: stat d13/d2a/d53/d72/fbd 0 2021-11-03T17:15:11.115 INFO:tasks.workunit.client.1.smithi115.stdout:3/647: dread d13/f48 [0,4194304] 0 2021-11-03T17:15:11.117 INFO:tasks.workunit.client.1.smithi115.stdout:3/648: fsync d13/f1f 0 2021-11-03T17:15:11.118 INFO:tasks.workunit.client.1.smithi115.stdout:6/449: mkdir d0/d1/d10/d13/d59/d95 0 2021-11-03T17:15:11.120 INFO:tasks.workunit.client.1.smithi115.stdout:3/649: chown d13/da4/fb9 45 1 2021-11-03T17:15:11.140 INFO:tasks.workunit.client.1.smithi115.stdout:4/655: dread - d7/dc/fd4 zero size 2021-11-03T17:15:11.145 INFO:tasks.workunit.client.1.smithi115.stdout:0/633: write d9/d15/d1a/d1d/dcc/d6e/fa3 [2298587,91871] 0 2021-11-03T17:15:11.147 INFO:tasks.workunit.client.1.smithi115.stdout:0/634: mknod d9/d15/d1a/d1d/dcc/d46/d63/dc5/dc8/cd5 1 2021-11-03T17:15:11.160 INFO:tasks.workunit.client.1.smithi115.stdout:6/450: rename d0/d1/d2/d5d to d0/d1/d2e/d4c/d96 0 2021-11-03T17:15:11.165 INFO:tasks.workunit.client.1.smithi115.stdout:6/451: truncate d0/d1/d10/d13/d1c/d1e/f47 3095288 0 2021-11-03T17:15:11.165 INFO:tasks.workunit.client.1.smithi115.stdout:7/452: dwrite d3/d4c/f72 [0,4194304] 0 2021-11-03T17:15:11.168 INFO:tasks.workunit.client.1.smithi115.stdout:6/452: read - d0/f7e zero size 2021-11-03T17:15:11.169 INFO:tasks.workunit.client.1.smithi115.stdout:7/453: chown d3/da/d18/d73/d52/f83 7 1 2021-11-03T17:15:11.174 INFO:tasks.workunit.client.1.smithi115.stdout:6/453: symlink d0/d1/d10/d13/d1c/d1e/l97 0 2021-11-03T17:15:11.175 INFO:tasks.workunit.client.1.smithi115.stdout:3/650: dwrite d13/d2a/d8a/d99/fd9 [0,4194304] 0 2021-11-03T17:15:11.199 INFO:tasks.workunit.client.1.smithi115.stdout:0/635: rename d9/f23 to d9/d15/d1a/d1d/dcc/d46/d8c/fd6 0 2021-11-03T17:15:11.205 INFO:tasks.workunit.client.1.smithi115.stdout:0/636: creat d9/d15/d1a/d1d/dcc/d46/d63/dae/fd7 x:0 0 0 2021-11-03T17:15:11.207 INFO:tasks.workunit.client.1.smithi115.stdout:6/454: rename d0/d1/d10/d13/d1c/d1e/l5e to d0/d1/d2e/l98 0 2021-11-03T17:15:11.208 INFO:tasks.workunit.client.1.smithi115.stdout:0/637: creat d9/d15/d1a/d1d/dcc/d46/d63/dae/fd8 x:0 0 0 2021-11-03T17:15:11.212 INFO:tasks.workunit.client.1.smithi115.stdout:3/651: write d13/f27 [4108388,64491] 0 2021-11-03T17:15:11.213 INFO:tasks.workunit.client.1.smithi115.stdout:3/652: mknod d13/d2a/d53/cdc 1 2021-11-03T17:15:11.214 INFO:tasks.workunit.client.1.smithi115.stdout:3/653: getdents d13/d3b/db1/dbc 0 2021-11-03T17:15:11.216 INFO:tasks.workunit.client.1.smithi115.stdout:3/654: fdatasync d13/d51/d40/f4f 0 2021-11-03T17:15:11.216 INFO:tasks.workunit.client.1.smithi115.stdout:3/655: dread - d13/d2a/d60/dd3/fda zero size 2021-11-03T17:15:11.217 INFO:tasks.workunit.client.1.smithi115.stdout:3/656: truncate d13/d2a/f4e 678054 0 2021-11-03T17:15:11.221 INFO:tasks.workunit.client.1.smithi115.stdout:0/638: dread d9/db/f4f [0,4194304] 0 2021-11-03T17:15:11.224 INFO:tasks.workunit.client.1.smithi115.stdout:0/639: symlink d9/d15/d1a/d1d/ld9 0 2021-11-03T17:15:11.254 INFO:tasks.workunit.client.1.smithi115.stdout:7/454: dwrite d3/f13 [0,4194304] 0 2021-11-03T17:15:11.255 INFO:tasks.workunit.client.1.smithi115.stdout:7/455: readlink d3/da/d18/l28 0 2021-11-03T17:15:11.260 INFO:tasks.workunit.client.1.smithi115.stdout:6/455: dwrite d0/d1/d2e/f83 [0,4194304] 0 2021-11-03T17:15:11.261 INFO:tasks.workunit.client.1.smithi115.stdout:3/657: creat d13/d2a/d53/d70/d92/dc2/fdd x:0 0 0 2021-11-03T17:15:11.265 INFO:tasks.workunit.client.1.smithi115.stdout:6/456: rmdir d0/d1/d10/d13/d87 39 2021-11-03T17:15:11.265 INFO:tasks.workunit.client.1.smithi115.stdout:3/658: mknod d13/d50/d9e/cde 1 2021-11-03T17:15:11.267 INFO:tasks.workunit.client.1.smithi115.stdout:3/659: mknod d13/d2a/d53/d70/cdf 1 2021-11-03T17:15:11.268 INFO:tasks.workunit.client.1.smithi115.stdout:3/660: dread - d13/d2a/d53/f8c zero size 2021-11-03T17:15:11.269 INFO:tasks.workunit.client.1.smithi115.stdout:3/661: chown d13/d50/d9e 714 1 2021-11-03T17:15:11.270 INFO:tasks.workunit.client.1.smithi115.stdout:6/457: rename d0/d1/d2/f12 to d0/d1/d10/f99 0 2021-11-03T17:15:11.274 INFO:tasks.workunit.client.1.smithi115.stdout:3/662: symlink d13/da4/dcb/le0 0 2021-11-03T17:15:11.276 INFO:tasks.workunit.client.1.smithi115.stdout:3/663: fsync d13/d35/f36 0 2021-11-03T17:15:11.285 INFO:tasks.workunit.client.1.smithi115.stdout:0/640: write d9/d15/d1a/d1d/f29 [2578131,70351] 0 2021-11-03T17:15:11.285 INFO:tasks.workunit.client.1.smithi115.stdout:0/641: write d9/d37/fd4 [310066,34306] 0 2021-11-03T17:15:11.287 INFO:tasks.workunit.client.1.smithi115.stdout:0/642: mkdir d9/d15/d1a/d1d/dcc/d46/d63/d70/dda 0 2021-11-03T17:15:11.287 INFO:tasks.workunit.client.1.smithi115.stdout:0/643: readlink d9/d15/l56 0 2021-11-03T17:15:11.288 INFO:tasks.workunit.client.1.smithi115.stdout:0/644: mknod d9/d15/cdb 1 2021-11-03T17:15:11.301 INFO:tasks.workunit.client.1.smithi115.stdout:7/456: write d3/da/d18/f22 [1867345,52133] 0 2021-11-03T17:15:11.308 INFO:tasks.workunit.client.1.smithi115.stdout:4/656: dread d7/dc/d1a/d21/db1/d56/d63/f67 [0,4194304] 0 2021-11-03T17:15:11.310 INFO:tasks.workunit.client.1.smithi115.stdout:4/657: mkdir d7/dc/d6a/dab/ddd 0 2021-11-03T17:15:11.311 INFO:tasks.workunit.client.1.smithi115.stdout:4/658: readlink d7/dc/lc0 0 2021-11-03T17:15:11.313 INFO:tasks.workunit.client.1.smithi115.stdout:4/659: truncate d7/dc/d6a/f46 1561045 0 2021-11-03T17:15:11.317 INFO:tasks.workunit.client.1.smithi115.stdout:4/660: link d7/dc/d1a/d21/l31 d7/dc/d6a/d3c/lde 0 2021-11-03T17:15:11.330 INFO:tasks.workunit.client.1.smithi115.stdout:3/664: creat d13/d51/d40/fe1 x:0 0 0 2021-11-03T17:15:11.332 INFO:tasks.workunit.client.1.smithi115.stdout:6/458: dwrite d0/d1/d2/f35 [0,4194304] 0 2021-11-03T17:15:11.337 INFO:tasks.workunit.client.1.smithi115.stdout:6/459: fsync d0/d1/d2e/d4c/d96/f6b 0 2021-11-03T17:15:11.339 INFO:tasks.workunit.client.1.smithi115.stdout:6/460: write d0/d1/d2e/f83 [1120702,33303] 0 2021-11-03T17:15:11.353 INFO:tasks.workunit.client.1.smithi115.stdout:0/645: write d9/f35 [2956582,128258] 0 2021-11-03T17:15:11.366 INFO:tasks.workunit.client.1.smithi115.stdout:7/457: dwrite d3/f27 [0,4194304] 0 2021-11-03T17:15:11.371 INFO:tasks.workunit.client.1.smithi115.stdout:3/665: creat d13/da4/fe2 x:0 0 0 2021-11-03T17:15:11.373 INFO:tasks.workunit.client.1.smithi115.stdout:7/458: symlink d3/da/d8e/l8f 0 2021-11-03T17:15:11.375 INFO:tasks.workunit.client.1.smithi115.stdout:4/661: truncate d7/fb5 238791 0 2021-11-03T17:15:11.380 INFO:tasks.workunit.client.1.smithi115.stdout:7/459: dread d3/f9 [4194304,4194304] 0 2021-11-03T17:15:11.381 INFO:tasks.workunit.client.1.smithi115.stdout:7/460: fsync d3/da/d18/d73/d35/f5f 0 2021-11-03T17:15:11.391 INFO:tasks.workunit.client.1.smithi115.stdout:4/662: dread d7/dc/d1a/d21/db1/d56/f8d [0,4194304] 0 2021-11-03T17:15:11.395 INFO:tasks.workunit.client.1.smithi115.stdout:0/646: write d9/d15/d1a/d1d/dcc/d46/d8c/fc2 [132603,31026] 0 2021-11-03T17:15:11.398 INFO:tasks.workunit.client.1.smithi115.stdout:0/647: dread f8 [4194304,4194304] 0 2021-11-03T17:15:11.418 INFO:tasks.workunit.client.1.smithi115.stdout:3/666: write d13/d51/d2e/f59 [100979,97738] 0 2021-11-03T17:15:11.426 INFO:tasks.workunit.client.1.smithi115.stdout:3/667: dread d13/d2a/d8a/d99/fd9 [0,4194304] 0 2021-11-03T17:15:11.428 INFO:tasks.workunit.client.1.smithi115.stdout:3/668: truncate d13/d2a/d53/d70/d92/db7/fc5 680817 0 2021-11-03T17:15:11.430 INFO:tasks.workunit.client.1.smithi115.stdout:3/669: getdents d13/d35/d54 0 2021-11-03T17:15:11.437 INFO:tasks.workunit.client.1.smithi115.stdout:0/648: write d9/d15/d78/d80/d82/fb5 [251965,58819] 0 2021-11-03T17:15:11.440 INFO:tasks.workunit.client.1.smithi115.stdout:0/649: link d9/db/d89/fb1 d9/d37/fdc 0 2021-11-03T17:15:11.442 INFO:tasks.workunit.client.1.smithi115.stdout:0/650: mkdir d9/d15/d78/ddd 0 2021-11-03T17:15:11.443 INFO:tasks.workunit.client.1.smithi115.stdout:0/651: symlink d9/d37/dd3/lde 0 2021-11-03T17:15:11.444 INFO:tasks.workunit.client.1.smithi115.stdout:0/652: chown d9/d15/d1a/d1d/dcc/d85 174 1 2021-11-03T17:15:11.444 INFO:tasks.workunit.client.1.smithi115.stdout:6/461: dread d0/d1/d2/f58 [0,4194304] 0 2021-11-03T17:15:11.445 INFO:tasks.workunit.client.1.smithi115.stdout:0/653: mknod d9/d15/d1a/d1d/dcc/d46/d8c/cdf 1 2021-11-03T17:15:11.448 INFO:tasks.workunit.client.1.smithi115.stdout:6/462: fsync d0/d1/d2/d3f/f49 0 2021-11-03T17:15:11.450 INFO:tasks.workunit.client.1.smithi115.stdout:6/463: creat d0/d1/d10/f9a x:0 0 0 2021-11-03T17:15:11.452 INFO:tasks.workunit.client.1.smithi115.stdout:0/654: symlink d9/d15/d78/d80/le0 0 2021-11-03T17:15:11.456 INFO:tasks.workunit.client.1.smithi115.stdout:6/464: symlink d0/d1/d10/d13/d1c/l9b 0 2021-11-03T17:15:11.457 INFO:tasks.workunit.client.1.smithi115.stdout:6/465: fsync d0/d1/d2/d3f/f51 0 2021-11-03T17:15:11.498 INFO:tasks.workunit.client.1.smithi115.stdout:7/461: truncate d3/da/f1b 5319574 0 2021-11-03T17:15:11.507 INFO:tasks.workunit.client.1.smithi115.stdout:3/670: dwrite fc [4194304,4194304] 0 2021-11-03T17:15:11.507 INFO:tasks.workunit.client.1.smithi115.stdout:7/462: dread d3/da/d18/d73/d35/d53/f58 [0,4194304] 0 2021-11-03T17:15:11.509 INFO:tasks.workunit.client.1.smithi115.stdout:7/463: creat d3/da/f90 x:0 0 0 2021-11-03T17:15:11.509 INFO:tasks.workunit.client.1.smithi115.stdout:0/655: write d9/d15/d1a/d1d/dcc/d46/d8c/f48 [1089676,117337] 0 2021-11-03T17:15:11.513 INFO:tasks.workunit.client.1.smithi115.stdout:3/671: dread d13/d67/f6c [0,4194304] 0 2021-11-03T17:15:11.514 INFO:tasks.workunit.client.1.smithi115.stdout:0/656: unlink d9/d15/d1a/d1d/dcc/d46/d8c/fc2 0 2021-11-03T17:15:11.515 INFO:tasks.workunit.client.1.smithi115.stdout:7/464: creat d3/d4c/d7b/f91 x:0 0 0 2021-11-03T17:15:11.515 INFO:tasks.workunit.client.1.smithi115.stdout:7/465: dread - d3/da/d18/d73/d35/f75 zero size 2021-11-03T17:15:11.523 INFO:tasks.workunit.client.1.smithi115.stdout:7/466: link d3/da/d18/d73/d52/l7a d3/da/d18/d73/d52/l92 0 2021-11-03T17:15:11.524 INFO:tasks.workunit.client.1.smithi115.stdout:7/467: creat d3/da/d8e/f93 x:0 0 0 2021-11-03T17:15:11.525 INFO:tasks.workunit.client.1.smithi115.stdout:7/468: mknod d3/c94 1 2021-11-03T17:15:11.560 INFO:tasks.workunit.client.1.smithi115.stdout:4/663: dread d7/dc/d1a/d21/db1/d56/d63/f9b [4194304,4194304] 0 2021-11-03T17:15:11.563 INFO:tasks.workunit.client.1.smithi115.stdout:4/664: chown d7/dc/d1a/d21/db1/d56/d63/d7b/fb2 256095195 1 2021-11-03T17:15:11.564 INFO:tasks.workunit.client.1.smithi115.stdout:3/672: write d13/d2a/d8a/fc8 [604448,98334] 0 2021-11-03T17:15:11.565 INFO:tasks.workunit.client.1.smithi115.stdout:4/665: symlink d7/dc/d6a/d3c/dbf/ldf 0 2021-11-03T17:15:11.566 INFO:tasks.workunit.client.1.smithi115.stdout:3/673: symlink d13/d2a/d53/le3 0 2021-11-03T17:15:11.569 INFO:tasks.workunit.client.1.smithi115.stdout:0/657: write d9/d15/d1a/d1d/dcc/d46/d63/f7b [1315843,68865] 0 2021-11-03T17:15:11.570 INFO:tasks.workunit.client.1.smithi115.stdout:4/666: truncate d7/dc/d1a/f1c 955527 0 2021-11-03T17:15:11.573 INFO:tasks.workunit.client.1.smithi115.stdout:0/658: dread d9/d37/fd4 [0,4194304] 0 2021-11-03T17:15:11.577 INFO:tasks.workunit.client.1.smithi115.stdout:0/659: link d9/d15/d1a/d1d/lca d9/le1 0 2021-11-03T17:15:11.578 INFO:tasks.workunit.client.1.smithi115.stdout:0/660: stat d9/d15/d1a/d1d/dcc/d46 0 2021-11-03T17:15:11.579 INFO:tasks.workunit.client.1.smithi115.stdout:0/661: rmdir d9 39 2021-11-03T17:15:11.580 INFO:tasks.workunit.client.1.smithi115.stdout:0/662: symlink d9/d15/d1a/le2 0 2021-11-03T17:15:11.582 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:11 smithi027 conmon[32322]: cluster 2021-11-03T17:15:10.323802+0000 mon.smithi027 2021-11-03T17:15:11.582 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:11 smithi027 conmon[32322]: (mon.0) 698 : cluster [INF] Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2021-11-03T17:15:11.582 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:11 smithi027 conmon[32322]: cluster 2021-11-03T17:15:10.323836+0000 mon.smithi027 (mon.0) 699 : cluster [INF] 2021-11-03T17:15:11.583 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:11 smithi027 conmon[32322]: Cluster is now healthy 2021-11-03T17:15:11.583 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:11 smithi027 conmon[32322]: cluster 2021-11-03T17: 2021-11-03T17:15:11.583 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:11 smithi027 conmon[32322]: 15:10.330548+0000 mon.smithi027 (mon.0 2021-11-03T17:15:11.583 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:11 smithi027 conmon[32322]: ) 700 : cluster [DBG] mds.? [v2:172.21.15.115:6826/3367380088,v1:172.21.15.115:6827/3367380088] up:active 2021-11-03T17:15:11.584 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:11 smithi027 conmon[32322]: cluster 2021- 2021-11-03T17:15:11.584 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:11 smithi027 conmon[32322]: 11-03T17:15:10.330613+0000 2021-11-03T17:15:11.584 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:11 smithi027 conmon[32322]: mon.smithi027 (mon.0) 701 : cluster 2021-11-03T17:15:11.584 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:11 smithi027 conmon[32322]: [DBG] fsmap cephfs:2 {0=cephfs.smithi115.semywj=up:active,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:15:11.585 INFO:tasks.workunit.client.1.smithi115.stdout:0/663: dread d9/db/f8e [0,4194304] 0 2021-11-03T17:15:11.596 INFO:tasks.workunit.client.1.smithi115.stdout:7/469: dwrite d3/da/d18/d73/d33/f5e [0,4194304] 0 2021-11-03T17:15:11.620 INFO:tasks.workunit.client.1.smithi115.stdout:4/667: write d7/fa0 [20266,80471] 0 2021-11-03T17:15:11.622 INFO:tasks.workunit.client.1.smithi115.stdout:4/668: unlink d7/dc/d6a/dab/lbb 0 2021-11-03T17:15:11.639 INFO:tasks.workunit.client.1.smithi115.stdout:7/470: rename d3/da/d18/d73/d35/d53/l6f to d3/da/l95 0 2021-11-03T17:15:11.640 INFO:tasks.workunit.client.1.smithi115.stdout:7/471: read d3/da/d18/d73/d35/d53/f58 [3497309,114410] 0 2021-11-03T17:15:11.640 INFO:tasks.workunit.client.1.smithi115.stdout:7/472: readlink l0 0 2021-11-03T17:15:11.641 INFO:tasks.workunit.client.1.smithi115.stdout:7/473: fsync d3/f47 0 2021-11-03T17:15:11.643 INFO:tasks.workunit.client.1.smithi115.stdout:7/474: unlink d3/f13 0 2021-11-03T17:15:11.643 INFO:tasks.workunit.client.1.smithi115.stdout:7/475: readlink d3/da/d18/d73/l81 0 2021-11-03T17:15:11.647 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:11 smithi115 conmon[34670]: cluster 2021-11-03T17:15:10.323802+0000 2021-11-03T17:15:11.648 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:11 smithi115 conmon[34670]: mon.smithi027 (mon.0) 698 : cluster [INF] Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2021-11-03T17:15:11.648 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:11 smithi115 conmon[34670]: cluster 2021-11-03T17:15:10.323836+0000 mon.smithi027 (mon.0 2021-11-03T17:15:11.648 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:11 smithi115 conmon[34670]: ) 699 : cluster [INF] Cluster is now healthy 2021-11-03T17:15:11.648 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:11 smithi115 conmon[34670]: cluster 2021 2021-11-03T17:15:11.649 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:11 smithi115 conmon[34670]: -11-03T17:15:10.330548+0000 mon.smithi027 ( 2021-11-03T17:15:11.649 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:11 smithi115 conmon[34670]: mon.0) 700 : cluster [DBG] 2021-11-03T17:15:11.649 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:11 smithi115 conmon[34670]: mds.? [v2:172.21.15.115:6826/3367380088,v1:172.21.15.115:6827/3367380088] up:active 2021-11-03T17:15:11.649 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:11 smithi115 conmon[34670]: cluster 2021-11-03T17:15:11.649 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:11 smithi115 conmon[34670]: 2021-11-03T17: 2021-11-03T17:15:11.650 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:11 smithi115 conmon[34670]: 15:10.330613+0000 mon.smithi027 (mon 2021-11-03T17:15:11.650 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:11 smithi115 conmon[34670]: .0) 701 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi115.semywj=up:active,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:15:11.650 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:11 smithi115 conmon[34670]: 2021-11-03T17:15:11.701 INFO:tasks.workunit.client.1.smithi115.stdout:7/476: dread d3/da/d18/f6d [0,4194304] 0 2021-11-03T17:15:11.704 INFO:tasks.workunit.client.1.smithi115.stdout:0/664: truncate d9/f25 3740308 0 2021-11-03T17:15:11.707 INFO:tasks.workunit.client.1.smithi115.stdout:0/665: getdents d9/db 0 2021-11-03T17:15:11.708 INFO:tasks.workunit.client.1.smithi115.stdout:7/477: rename d3/l4b to d3/da/d78/l96 0 2021-11-03T17:15:11.710 INFO:tasks.workunit.client.1.smithi115.stdout:0/666: creat d9/d15/d78/d80/fe3 x:0 0 0 2021-11-03T17:15:11.717 INFO:tasks.workunit.client.1.smithi115.stdout:7/478: dread d3/da/d18/d73/d33/f74 [0,4194304] 0 2021-11-03T17:15:11.718 INFO:tasks.workunit.client.1.smithi115.stdout:7/479: write d3/d4c/d7b/f91 [490889,36420] 0 2021-11-03T17:15:11.720 INFO:tasks.workunit.client.1.smithi115.stdout:7/480: dread d3/da/d18/d73/d52/f66 [0,4194304] 0 2021-11-03T17:15:11.721 INFO:tasks.workunit.client.1.smithi115.stdout:7/481: fdatasync d3/da/d18/d73/d35/f57 0 2021-11-03T17:15:11.725 INFO:tasks.workunit.client.0.smithi027.stdout:7/887: sync 2021-11-03T17:15:11.726 INFO:tasks.workunit.client.0.smithi027.stdout:7/888: mknod d0/d2/d10/d1e/d20/d74/c124 1 2021-11-03T17:15:11.727 INFO:tasks.workunit.client.1.smithi115.stdout:7/482: rename d3/f6a to d3/f97 0 2021-11-03T17:15:11.728 INFO:tasks.workunit.client.1.smithi115.stdout:7/483: creat d3/f98 x:0 0 0 2021-11-03T17:15:11.728 INFO:tasks.workunit.client.1.smithi115.stdout:0/667: dwrite d9/f35 [0,4194304] 0 2021-11-03T17:15:11.729 INFO:tasks.workunit.client.1.smithi115.stdout:7/484: creat d3/da/d8e/f99 x:0 0 0 2021-11-03T17:15:11.729 INFO:tasks.workunit.client.1.smithi115.stdout:7/485: dread - d3/da/d18/d73/d35/f57 zero size 2021-11-03T17:15:11.729 INFO:tasks.workunit.client.1.smithi115.stdout:0/668: readlink d9/l52 0 2021-11-03T17:15:11.731 INFO:tasks.workunit.client.1.smithi115.stdout:0/669: readlink d9/d15/d1a/d1d/dcc/d46/d63/l95 0 2021-11-03T17:15:11.735 INFO:tasks.workunit.client.1.smithi115.stdout:0/670: unlink d9/lc3 0 2021-11-03T17:15:11.736 INFO:tasks.workunit.client.1.smithi115.stdout:0/671: mknod d9/db/ce4 1 2021-11-03T17:15:11.736 INFO:tasks.workunit.client.1.smithi115.stdout:0/672: rmdir d9/d15/d1a/d1d/dcc/d46/d63 39 2021-11-03T17:15:11.739 INFO:tasks.workunit.client.1.smithi115.stdout:0/673: mkdir d9/d15/d1a/de5 0 2021-11-03T17:15:11.743 INFO:tasks.workunit.client.0.smithi027.stdout:7/889: write d0/f18 [778524,82090] 0 2021-11-03T17:15:11.744 INFO:tasks.workunit.client.0.smithi027.stdout:7/890: creat d0/f125 x:0 0 0 2021-11-03T17:15:11.748 INFO:tasks.workunit.client.0.smithi027.stdout:7/891: link d0/d2/d10/d82/df2/f100 d0/d2/d10/d82/f126 0 2021-11-03T17:15:11.751 INFO:tasks.workunit.client.0.smithi027.stdout:7/892: link d0/f4c d0/d2/d10/d1e/d108/f127 0 2021-11-03T17:15:11.753 INFO:tasks.workunit.client.0.smithi027.stdout:7/893: symlink d0/d2/d10/d1e/d20/d74/l128 0 2021-11-03T17:15:11.759 INFO:tasks.workunit.client.0.smithi027.stdout:7/894: fdatasync d0/d2d/d53/f65 0 2021-11-03T17:15:11.767 INFO:tasks.workunit.client.0.smithi027.stdout:7/895: unlink d0/d2d/d66/f70 0 2021-11-03T17:15:11.769 INFO:tasks.workunit.client.0.smithi027.stdout:7/896: chown d0/d2d/d2f/dae/dc8/l113 36970994 1 2021-11-03T17:15:11.773 INFO:tasks.workunit.client.0.smithi027.stdout:7/897: read d0/d2/d5e/d36/f64 [59944,21715] 0 2021-11-03T17:15:11.774 INFO:tasks.workunit.client.0.smithi027.stdout:7/898: mknod d0/d2/d10/d82/df2/c129 1 2021-11-03T17:15:11.776 INFO:tasks.workunit.client.0.smithi027.stdout:7/899: truncate d0/d2d/d58/f10f 4506445 0 2021-11-03T17:15:11.778 INFO:tasks.workunit.client.0.smithi027.stdout:7/900: chown d0/d2/d1c/d4d/d6e/d8b/l8e 7 1 2021-11-03T17:15:11.780 INFO:tasks.workunit.client.0.smithi027.stdout:7/901: symlink d0/d2d/l12a 0 2021-11-03T17:15:11.782 INFO:tasks.workunit.client.0.smithi027.stdout:7/902: mknod d0/d2/d59/c12b 1 2021-11-03T17:15:11.784 INFO:tasks.workunit.client.0.smithi027.stdout:7/903: getdents d0/d2/d1c/d4d/dcc 0 2021-11-03T17:15:11.897 INFO:tasks.workunit.client.1.smithi115.stdout:7/486: write d3/da/d18/d73/f2f [2397523,130316] 0 2021-11-03T17:15:11.898 INFO:tasks.workunit.client.1.smithi115.stdout:7/487: chown d3/da/d18/d73/d52/f83 33533475 1 2021-11-03T17:15:11.915 INFO:tasks.workunit.client.1.smithi115.stdout:0/674: write d9/fa8 [384662,40935] 0 2021-11-03T17:15:12.160 INFO:tasks.workunit.client.1.smithi115.stdout:0/675: rmdir d9/d15/d78/d80 39 2021-11-03T17:15:12.168 INFO:tasks.workunit.client.1.smithi115.stdout:0/676: dread d9/d15/d78/fd0 [0,4194304] 0 2021-11-03T17:15:12.200 INFO:tasks.workunit.client.1.smithi115.stdout:0/677: dwrite d9/d15/d1a/d1d/dcc/d46/d63/f96 [0,4194304] 0 2021-11-03T17:15:12.206 INFO:tasks.workunit.client.1.smithi115.stdout:0/678: fdatasync d9/d15/d1a/fa4 0 2021-11-03T17:15:12.216 INFO:tasks.workunit.client.1.smithi115.stdout:7/488: dread d3/f14 [0,4194304] 0 2021-11-03T17:15:12.216 INFO:tasks.workunit.client.1.smithi115.stdout:7/489: chown d3/da/l61 28065 1 2021-11-03T17:15:12.217 INFO:tasks.workunit.client.1.smithi115.stdout:7/490: mknod d3/da/d18/d73/d52/c9a 1 2021-11-03T17:15:12.217 INFO:tasks.workunit.client.1.smithi115.stdout:7/491: write d3/da/d8e/f93 [345124,68234] 0 2021-11-03T17:15:12.232 INFO:tasks.workunit.client.1.smithi115.stdout:7/492: dwrite d3/da/f90 [0,4194304] 0 2021-11-03T17:15:12.239 INFO:tasks.workunit.client.1.smithi115.stdout:7/493: getdents d3/da/d18 0 2021-11-03T17:15:12.243 INFO:tasks.workunit.client.1.smithi115.stdout:0/679: creat d9/d15/d78/d80/d82/fe6 x:0 0 0 2021-11-03T17:15:12.246 INFO:tasks.workunit.client.1.smithi115.stdout:0/680: link d9/db/f58 d9/d15/d1a/d1d/dcc/d46/d63/dc5/fe7 0 2021-11-03T17:15:12.248 INFO:tasks.workunit.client.1.smithi115.stdout:0/681: getdents d9 0 2021-11-03T17:15:12.249 INFO:tasks.workunit.client.1.smithi115.stdout:0/682: symlink d9/d15/d1a/de5/le8 0 2021-11-03T17:15:12.278 INFO:tasks.workunit.client.1.smithi115.stdout:7/494: creat d3/da/d18/d73/d33/f9b x:0 0 0 2021-11-03T17:15:12.283 INFO:tasks.workunit.client.1.smithi115.stdout:7/495: dread d3/da/d8e/f93 [0,4194304] 0 2021-11-03T17:15:12.294 INFO:tasks.workunit.client.1.smithi115.stdout:7/496: dread d3/f47 [0,4194304] 0 2021-11-03T17:15:12.300 INFO:tasks.workunit.client.1.smithi115.stdout:0/683: dwrite d9/d15/d1a/d1d/dcc/d6e/d71/fa9 [0,4194304] 0 2021-11-03T17:15:12.304 INFO:tasks.workunit.client.1.smithi115.stdout:0/684: link d9/d15/l56 d9/d15/d1a/da7/le9 0 2021-11-03T17:15:12.378 INFO:tasks.workunit.client.1.smithi115.stdout:7/497: truncate d3/da/d18/f40 4588089 0 2021-11-03T17:15:12.378 INFO:tasks.workunit.client.1.smithi115.stdout:7/498: chown d3/da/d18/d73/l8b 3235760 1 2021-11-03T17:15:12.381 INFO:tasks.workunit.client.1.smithi115.stdout:7/499: dread - d3/da/f7e zero size 2021-11-03T17:15:12.382 INFO:tasks.workunit.client.1.smithi115.stdout:7/500: mkdir d3/da/d18/d9c 0 2021-11-03T17:15:12.384 INFO:tasks.workunit.client.1.smithi115.stdout:7/501: rmdir d3/da/d18 39 2021-11-03T17:15:12.385 INFO:tasks.workunit.client.1.smithi115.stdout:7/502: creat d3/da/d18/d73/d35/f9d x:0 0 0 2021-11-03T17:15:12.393 INFO:tasks.workunit.client.1.smithi115.stdout:0/685: dread d9/d15/d1a/d1d/dcc/d46/d63/dc5/fe7 [0,4194304] 0 2021-11-03T17:15:12.393 INFO:tasks.workunit.client.1.smithi115.stdout:0/686: dread - d9/f98 zero size 2021-11-03T17:15:12.394 INFO:tasks.workunit.client.1.smithi115.stdout:0/687: fsync d9/d15/d1a/d1d/dcc/d6e/d71/fa9 0 2021-11-03T17:15:12.395 INFO:tasks.workunit.client.1.smithi115.stdout:7/503: dread d3/da/f62 [0,4194304] 0 2021-11-03T17:15:12.396 INFO:tasks.workunit.client.1.smithi115.stdout:0/688: symlink d9/d15/d1a/d1d/dcc/lea 0 2021-11-03T17:15:12.397 INFO:tasks.workunit.client.1.smithi115.stdout:0/689: readlink d9/d15/d1a/d1d/dcc/d46/d63/la2 0 2021-11-03T17:15:12.398 INFO:tasks.workunit.client.1.smithi115.stdout:0/690: mkdir d9/d15/d78/ddd/deb 0 2021-11-03T17:15:12.401 INFO:tasks.workunit.client.1.smithi115.stdout:0/691: mknod d9/d15/d1a/d1d/dcc/d46/cec 1 2021-11-03T17:15:12.439 INFO:tasks.workunit.client.1.smithi115.stdout:0/692: write d9/db/f4f [1246642,50379] 0 2021-11-03T17:15:12.440 INFO:tasks.workunit.client.1.smithi115.stdout:0/693: creat d9/d15/d1a/d1d/dcc/d46/d63/dae/fed x:0 0 0 2021-11-03T17:15:12.449 INFO:tasks.workunit.client.1.smithi115.stdout:0/694: dread d9/d15/d1a/d1d/dcc/d46/d8c/f48 [0,4194304] 0 2021-11-03T17:15:12.449 INFO:tasks.workunit.client.1.smithi115.stdout:0/695: mknod d9/d15/d1a/d1d/dcc/d46/cee 1 2021-11-03T17:15:12.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:12 smithi027 conmon[32322]: cluster 2021-11-03T17:15:10.642834+0000 mgr.smithi027.ufhhvb (mgr.14622) 14 : cluster 2021-11-03T17:15:12.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:12 smithi027 conmon[32322]: [DBG] pgmap v9: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 13 MiB/s rd, 12 MiB/s wr, 37 op/s 2021-11-03T17:15:12.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:12 smithi027 conmon[32322]: cephadm 2021-11-03T17:15:11.499729+0000 mgr.smithi027.ufhhvb (mgr.14622) 15 : cephadm [INF] poll took 5527.041 ms: 1 events 2021-11-03T17:15:12.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:12 smithi027 conmon[32322]: audit 2021-11-03T17:15:11.534246+0000 mon.smithi027 (mon.0) 702 : audit [INF] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:15:12.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:12 smithi027 conmon[32322]: audit 2021-11-03T17:15:12.268017+0000 mon.smithi027 (mon.0) 703 : audit [INF] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:15:12.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:12 smithi027 conmon[32322]: audit 2021-11-03T17:15:12.269236+0000 mon.smithi027 (mon.0) 704 : audit [INF] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"}]: dispatch 2021-11-03T17:15:12.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:12 smithi027 conmon[32322]: audit 2021-11-03T17:15:12.275315+0000 mon.smithi027 (mon.0) 705 : audit [INF] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:15:12.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:12 smithi027 conmon[32322]: audit 2021-11-03T17:15:12.281192+0000 mon.smithi027 (mon.0) 706 : audit [DBG] 2021-11-03T17:15:12.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:12 smithi027 conmon[32322]: from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:15:12.524 INFO:tasks.workunit.client.1.smithi115.stdout:0/696: truncate d9/d15/d1a/d1d/f24 5105011 0 2021-11-03T17:15:12.524 INFO:tasks.workunit.client.1.smithi115.stdout:0/697: symlink d9/d15/d1a/d1d/dcc/d46/d63/dc5/lef 0 2021-11-03T17:15:12.524 INFO:tasks.workunit.client.1.smithi115.stdout:0/698: symlink d9/d15/dce/lf0 0 2021-11-03T17:15:12.526 INFO:tasks.workunit.client.1.smithi115.stdout:0/699: truncate d9/d15/d1a/d1d/dcc/d46/d8c/fd6 3484737 0 2021-11-03T17:15:12.564 INFO:tasks.workunit.client.1.smithi115.stdout:0/700: write d9/db/f75 [1315865,82560] 0 2021-11-03T17:15:12.610 INFO:tasks.workunit.client.1.smithi115.stdout:0/701: truncate d9/d15/d1a/d1d/dcc/d46/d8c/fd6 2936848 0 2021-11-03T17:15:12.656 INFO:tasks.workunit.client.1.smithi115.stdout:0/702: dwrite d9/f66 [0,4194304] 0 2021-11-03T17:15:12.718 INFO:tasks.workunit.client.1.smithi115.stdout:0/703: read d9/d15/d1a/f9e [98755,68962] 0 2021-11-03T17:15:12.727 INFO:tasks.workunit.client.1.smithi115.stdout:0/704: dread d9/d15/d78/d80/f9a [0,4194304] 0 2021-11-03T17:15:12.727 INFO:tasks.workunit.client.1.smithi115.stdout:0/705: write d9/d15/d78/fd0 [886725,83308] 0 2021-11-03T17:15:12.730 INFO:tasks.workunit.client.1.smithi115.stdout:0/706: rename d9/d15/d1a/d1d/dcc/lea to d9/d15/d1a/d1d/dcc/d46/d63/lf1 0 2021-11-03T17:15:12.731 INFO:tasks.workunit.client.1.smithi115.stdout:0/707: write d9/d15/d1a/fa4 [801438,32016] 0 2021-11-03T17:15:12.731 INFO:tasks.workunit.client.1.smithi115.stdout:0/708: getdents d9/db/d83 0 2021-11-03T17:15:12.732 INFO:tasks.workunit.client.1.smithi115.stdout:0/709: fsync d9/d15/d78/d80/f9a 0 2021-11-03T17:15:12.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:12 smithi115 conmon[34670]: cluster 2021-11-03T17:15:10.642834 2021-11-03T17:15:12.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:12 smithi115 conmon[34670]: +0000 mgr.smithi027.ufhhvb (mgr.14622) 14 : cluster [DBG] pgmap v9: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 13 MiB/s rd, 12 MiB/s wr, 37 op/s 2021-11-03T17:15:12.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:12 smithi115 conmon[34670]: cephadm 2021-11-03T17:15:12.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:12 smithi115 conmon[34670]: 2021-11-03T17:15:11.499729+0000 mgr.smithi027.ufhhvb (mgr.14622) 2021-11-03T17:15:12.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:12 smithi115 conmon[34670]: 15 : cephadm [INF] poll took 5527.041 ms: 1 events 2021-11-03T17:15:12.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:12 smithi115 conmon[34670]: audit 2021-11-03T17:15:12.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:12 smithi115 conmon[34670]: 2021-11-03T17:15:11.534246 2021-11-03T17:15:12.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:12 smithi115 conmon[34670]: +0000 mon.smithi027 (mon.0) 702 : audit 2021-11-03T17:15:12.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:12 smithi115 conmon[34670]: [INF] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:15:12.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:12 smithi115 conmon[34670]: audit 2021-11 2021-11-03T17:15:12.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:12 smithi115 conmon[34670]: -03T17:15:12.268017+0000 mon.smithi027 2021-11-03T17:15:12.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:12 smithi115 conmon[34670]: (mon.0) 703 : audit [INF] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:15:12.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:12 smithi115 conmon[34670]: audit 2021-11-03T17:15:12. 2021-11-03T17:15:12.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:12 smithi115 conmon[34670]: 269236+0000 mon.smithi027 (mon.0) 704 : audit [INF] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"}]: dispatch 2021-11-03T17:15:12.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:12 smithi115 conmon[34670]: 2021-11-03T17:15:12.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:12 smithi115 conmon[34670]: audit 2021-11-03T17 2021-11-03T17:15:12.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:12 smithi115 conmon[34670]: :15:12.275315+0000 mon.smithi027 (mon.0 2021-11-03T17:15:12.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:12 smithi115 conmon[34670]: ) 705 : audit [INF] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:15:12.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:12 smithi115 conmon[34670]: audit 2021-11-03T17:15:12.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:12 smithi115 conmon[34670]: 2021-11-03T17:15:12.281192 2021-11-03T17:15:12.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:12 smithi115 conmon[34670]: +0000 mon.smithi027 (mon.0) 706 : audit 2021-11-03T17:15:12.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:12 smithi115 conmon[34670]: [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:15:12.740 INFO:tasks.workunit.client.1.smithi115.stdout:0/710: dread d9/db/f4f [0,4194304] 0 2021-11-03T17:15:12.741 INFO:tasks.workunit.client.1.smithi115.stdout:0/711: write d9/d15/d1a/d1d/dcc/d6e/f72 [482482,46540] 0 2021-11-03T17:15:12.742 INFO:tasks.workunit.client.1.smithi115.stdout:0/712: truncate d9/d15/d1a/d1d/f24 5152789 0 2021-11-03T17:15:12.742 INFO:tasks.workunit.client.1.smithi115.stdout:0/713: chown d9/db/f75 32171321 1 2021-11-03T17:15:12.743 INFO:tasks.workunit.client.1.smithi115.stdout:0/714: chown d9/d15/d1a/f68 69280 1 2021-11-03T17:15:12.744 INFO:tasks.workunit.client.1.smithi115.stdout:0/715: chown d9/db/d89/d81/lbd 13472 1 2021-11-03T17:15:12.745 INFO:tasks.workunit.client.1.smithi115.stdout:0/716: truncate d9/d15/f67 3068531 0 2021-11-03T17:15:12.746 INFO:tasks.workunit.client.1.smithi115.stdout:0/717: creat d9/d15/d1a/d1d/dcc/d46/d63/dae/ff2 x:0 0 0 2021-11-03T17:15:12.765 INFO:tasks.workunit.client.1.smithi115.stdout:0/718: dwrite d9/d15/d78/d80/d82/fb5 [0,4194304] 0 2021-11-03T17:15:12.772 INFO:tasks.workunit.client.1.smithi115.stdout:0/719: chown d9/d15/d1a/f2e 1083143 1 2021-11-03T17:15:12.777 INFO:tasks.workunit.client.1.smithi115.stdout:0/720: dread d9/d15/f16 [0,4194304] 0 2021-11-03T17:15:12.778 INFO:tasks.workunit.client.1.smithi115.stdout:0/721: mknod d9/d15/d1a/d1d/dcc/d46/cf3 1 2021-11-03T17:15:12.778 INFO:tasks.workunit.client.1.smithi115.stdout:0/722: chown d9/d15/d78/d80/l94 130470 1 2021-11-03T17:15:12.808 INFO:tasks.workunit.client.1.smithi115.stdout:0/723: unlink d9/db/d89/fa1 0 2021-11-03T17:15:12.816 INFO:tasks.workunit.client.1.smithi115.stdout:0/724: dread d9/d15/d1a/d1d/dcc/d46/d8c/fd6 [0,4194304] 0 2021-11-03T17:15:12.818 INFO:tasks.workunit.client.1.smithi115.stdout:0/725: symlink d9/d15/d1a/d1d/dcc/d46/d63/lf4 0 2021-11-03T17:15:12.821 INFO:tasks.workunit.client.1.smithi115.stdout:0/726: fdatasync d9/d15/d78/fd0 0 2021-11-03T17:15:12.839 INFO:tasks.workunit.client.1.smithi115.stdout:0/727: dwrite d9/d15/d78/d80/d82/fe6 [0,4194304] 0 2021-11-03T17:15:12.841 INFO:tasks.workunit.client.1.smithi115.stdout:0/728: fsync d9/d15/d1a/f92 0 2021-11-03T17:15:12.844 INFO:tasks.workunit.client.1.smithi115.stdout:0/729: symlink d9/d37/dd3/lf5 0 2021-11-03T17:15:12.880 INFO:tasks.workunit.client.1.smithi115.stdout:0/730: write d9/d15/d1a/d1d/fb4 [273874,11963] 0 2021-11-03T17:15:12.880 INFO:tasks.workunit.client.1.smithi115.stdout:0/731: truncate d9/fa8 1002199 0 2021-11-03T17:15:12.904 INFO:tasks.workunit.client.1.smithi115.stdout:0/732: dwrite d9/d37/fd4 [0,4194304] 0 2021-11-03T17:15:12.946 INFO:tasks.workunit.client.1.smithi115.stdout:0/733: dread d9/d15/d1a/f3d [0,4194304] 0 2021-11-03T17:15:12.964 INFO:tasks.workunit.client.1.smithi115.stdout:0/734: dwrite d9/f35 [0,4194304] 0 2021-11-03T17:15:12.967 INFO:tasks.workunit.client.1.smithi115.stdout:0/735: symlink d9/d15/d1a/d1d/dcc/d46/d63/d70/lf6 0 2021-11-03T17:15:12.968 INFO:tasks.workunit.client.1.smithi115.stdout:0/736: write d9/d15/d78/fd0 [4104440,128973] 0 2021-11-03T17:15:13.271 INFO:tasks.workunit.client.0.smithi027.stdout:7/904: sync 2021-11-03T17:15:13.272 INFO:tasks.workunit.client.0.smithi027.stdout:7/905: mknod d0/d2/d10/d82/df2/dab/d11e/c12c 1 2021-11-03T17:15:13.272 INFO:tasks.workunit.client.0.smithi027.stdout:7/906: chown d0/d2/d10/f1b 10 1 2021-11-03T17:15:13.278 INFO:tasks.workunit.client.0.smithi027.stdout:7/907: write d0/d2/d10/d1e/d20/f5b [1881089,95976] 0 2021-11-03T17:15:13.279 INFO:tasks.workunit.client.0.smithi027.stdout:7/908: getdents d0/d2/d10/d82/d112 0 2021-11-03T17:15:13.280 INFO:tasks.workunit.client.0.smithi027.stdout:7/909: mkdir d0/d2/d1c/d4d/d12d 0 2021-11-03T17:15:13.281 INFO:tasks.workunit.client.0.smithi027.stdout:7/910: chown d0/d2d/lbb 232432116 1 2021-11-03T17:15:13.285 INFO:tasks.workunit.client.0.smithi027.stdout:7/911: link d0/d2/d1c/f2b d0/d2/d1c/d4d/d6e/f12e 0 2021-11-03T17:15:13.285 INFO:tasks.workunit.client.0.smithi027.stdout:7/912: write d0/f11d [142955,93136] 0 2021-11-03T17:15:13.287 INFO:tasks.workunit.client.0.smithi027.stdout:7/913: read d0/d2/d1c/d4d/d6e/d84/dad/fc3 [1366983,59744] 0 2021-11-03T17:15:13.289 INFO:tasks.workunit.client.0.smithi027.stdout:7/914: write d0/d2/d10/d82/de4/f106 [2034699,51872] 0 2021-11-03T17:15:13.373 INFO:tasks.workunit.client.0.smithi027.stdout:7/915: dwrite d0/d2/d10/d35/d96/fa2 [0,4194304] 0 2021-11-03T17:15:13.395 INFO:tasks.workunit.client.0.smithi027.stdout:7/916: dwrite d0/d2/d10/d1e/d20/d74/f9d [0,4194304] 0 2021-11-03T17:15:13.400 INFO:tasks.workunit.client.0.smithi027.stdout:7/917: unlink d0/d2/d10/d82/de4/lf5 0 2021-11-03T17:15:13.756 INFO:tasks.workunit.client.0.smithi027.stdout:7/918: sync 2021-11-03T17:15:13.776 INFO:tasks.workunit.client.0.smithi027.stdout:7/919: dwrite d0/d2/d5e/f110 [0,4194304] 0 2021-11-03T17:15:13.778 INFO:tasks.workunit.client.0.smithi027.stdout:7/920: write d0/d2d/d2f/dae/dc8/df9/f116 [888109,49144] 0 2021-11-03T17:15:13.780 INFO:tasks.workunit.client.0.smithi027.stdout:7/921: fdatasync d0/d2/d1f/f54 0 2021-11-03T17:15:13.785 INFO:tasks.workunit.client.0.smithi027.stdout:7/922: write d0/d2/fa3 [3999619,45401] 0 2021-11-03T17:15:13.786 INFO:tasks.workunit.client.0.smithi027.stdout:7/923: dread - d0/d2/d59/f73 zero size 2021-11-03T17:15:13.788 INFO:tasks.workunit.client.0.smithi027.stdout:7/924: truncate d0/d2/d10/d1e/d20/d74/ffc 662836 0 2021-11-03T17:15:13.791 INFO:tasks.workunit.client.0.smithi027.stdout:7/925: rename d0/d2/d10/d35/d96 to d0/d2d/d2f/d12f 0 2021-11-03T17:15:13.792 INFO:tasks.workunit.client.0.smithi027.stdout:7/926: fsync d0/d2/d1c/d4d/dcc/fdb 0 2021-11-03T17:15:13.792 INFO:tasks.workunit.client.0.smithi027.stdout:7/927: fdatasync d0/d2/d1c/f61 0 2021-11-03T17:15:13.793 INFO:tasks.workunit.client.0.smithi027.stdout:7/928: stat d0/d2/d1c/d4d/fd5 0 2021-11-03T17:15:13.795 INFO:tasks.workunit.client.0.smithi027.stdout:7/929: write d0/d2d/ff8 [1161118,96645] 0 2021-11-03T17:15:13.797 INFO:tasks.workunit.client.0.smithi027.stdout:7/930: creat d0/d2/d10/f130 x:0 0 0 2021-11-03T17:15:13.803 INFO:tasks.workunit.client.0.smithi027.stdout:7/931: truncate d0/d2/fa3 1408319 0 2021-11-03T17:15:13.805 INFO:tasks.workunit.client.0.smithi027.stdout:7/932: unlink d0/d2/d1c/d4d/d6e/lc4 0 2021-11-03T17:15:14.047 INFO:tasks.workunit.client.0.smithi027.stdout:7/933: sync 2021-11-03T17:15:14.054 INFO:tasks.workunit.client.0.smithi027.stdout:7/934: dread d0/d2/d10/d1e/d20/f43 [0,4194304] 0 2021-11-03T17:15:14.058 INFO:tasks.workunit.client.0.smithi027.stdout:7/935: dread d0/d2/d10/d35/fb5 [0,4194304] 0 2021-11-03T17:15:14.060 INFO:tasks.workunit.client.0.smithi027.stdout:7/936: symlink d0/d2/d1c/d4d/d6e/d8b/dea/d10c/l131 0 2021-11-03T17:15:14.061 INFO:tasks.workunit.client.0.smithi027.stdout:7/937: stat d0/d2/d1f/fec 0 2021-11-03T17:15:14.062 INFO:tasks.workunit.client.0.smithi027.stdout:7/938: truncate d0/d2d/d58/f114 662338 0 2021-11-03T17:15:14.079 INFO:tasks.workunit.client.0.smithi027.stdout:7/939: dwrite d0/d2d/fc5 [0,4194304] 0 2021-11-03T17:15:14.085 INFO:tasks.workunit.client.0.smithi027.stdout:7/940: rename d0/d2/d1c/d4d/d6e/d8b/f111 to d0/d2/d10/d82/de8/f132 0 2021-11-03T17:15:14.107 INFO:tasks.workunit.client.0.smithi027.stdout:7/941: dwrite d0/d2/d10/d1e/d20/fdd [0,4194304] 0 2021-11-03T17:15:14.110 INFO:tasks.workunit.client.0.smithi027.stdout:7/942: creat d0/d2/d5e/d36/f133 x:0 0 0 2021-11-03T17:15:14.126 INFO:tasks.workunit.client.0.smithi027.stdout:7/943: dwrite d0/d2/ff1 [0,4194304] 0 2021-11-03T17:15:14.127 INFO:tasks.workunit.client.0.smithi027.stdout:7/944: stat d0/d2d/d2f/dae/ff4 0 2021-11-03T17:15:14.128 INFO:tasks.workunit.client.0.smithi027.stdout:7/945: rmdir d0/d2d/d58 39 2021-11-03T17:15:14.131 INFO:tasks.workunit.client.0.smithi027.stdout:7/946: symlink d0/d2/d1c/d4d/d6e/l134 0 2021-11-03T17:15:14.149 INFO:tasks.workunit.client.0.smithi027.stdout:7/947: dwrite d0/d2/d10/d1e/d108/f127 [4194304,4194304] 0 2021-11-03T17:15:14.153 INFO:tasks.workunit.client.0.smithi027.stdout:7/948: rename d0/d2/d10/d1e/d20/f5b to d0/d2/d10/d82/de4/f135 0 2021-11-03T17:15:14.165 INFO:tasks.workunit.client.0.smithi027.stdout:7/949: dwrite d0/d2/d10/d82/de8/f132 [0,4194304] 0 2021-11-03T17:15:14.166 INFO:tasks.workunit.client.0.smithi027.stdout:7/950: fdatasync d0/d2/d1c/d4d/dcc/fd4 0 2021-11-03T17:15:14.174 INFO:tasks.workunit.client.0.smithi027.stdout:7/951: dread d0/f13 [0,4194304] 0 2021-11-03T17:15:14.197 INFO:tasks.workunit.client.0.smithi027.stdout:7/952: dwrite d0/d2d/d66/f98 [0,4194304] 0 2021-11-03T17:15:14.198 INFO:tasks.workunit.client.0.smithi027.stdout:7/953: rmdir d0/d2d/d58 39 2021-11-03T17:15:14.203 INFO:tasks.workunit.client.0.smithi027.stdout:7/954: dread d0/d2/fa3 [0,4194304] 0 2021-11-03T17:15:14.228 INFO:tasks.workunit.client.0.smithi027.stdout:7/955: dwrite d0/d2/d5e/d36/d3a/fd0 [0,4194304] 0 2021-11-03T17:15:14.229 INFO:tasks.workunit.client.0.smithi027.stdout:7/956: mknod d0/d2/d10/d1e/de6/c136 1 2021-11-03T17:15:14.233 INFO:tasks.workunit.client.0.smithi027.stdout:7/957: unlink d0/d2/d10/d1e/d20/f26 0 2021-11-03T17:15:14.248 INFO:tasks.workunit.client.0.smithi027.stdout:7/958: dwrite d0/f11d [0,4194304] 0 2021-11-03T17:15:14.256 INFO:tasks.workunit.client.0.smithi027.stdout:7/959: rename d0/d2/d1f/fc2 to d0/d2/d10/d82/de4/dfd/dfe/f137 0 2021-11-03T17:15:14.256 INFO:tasks.workunit.client.0.smithi027.stdout:7/960: readlink d0/d2/d1c/d4d/d6e/d8b/l8e 0 2021-11-03T17:15:14.257 INFO:tasks.workunit.client.0.smithi027.stdout:7/961: mknod d0/d2/d5e/d36/d3a/c138 1 2021-11-03T17:15:14.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:14 smithi115 conmon[34670]: cluster 2021-11-03T17:15:12.643427+0000 mgr.smithi027.ufhhvb (mgr.14622) 16 : cluster 2021-11-03T17:15:14.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:14 smithi115 conmon[34670]: [DBG] pgmap v10: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 18 MiB/s rd, 21 MiB/s wr, 466 op/s 2021-11-03T17:15:14.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:14 smithi027 conmon[32322]: cluster 2021-11-03T17:15:12.643427+0000 mgr.smithi027.ufhhvb (mgr. 2021-11-03T17:15:14.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:14 smithi027 conmon[32322]: 14622) 16 : cluster [DBG] pgmap v10: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 18 MiB/s rd, 21 MiB/s wr, 466 op/s 2021-11-03T17:15:14.959 INFO:tasks.workunit.client.1.smithi115.stdout:8/526: sync 2021-11-03T17:15:14.967 INFO:tasks.workunit.client.1.smithi115.stdout:8/527: read d4/d6/d18/d44/d6e/fa8 [4231999,2441] 0 2021-11-03T17:15:14.969 INFO:tasks.workunit.client.1.smithi115.stdout:8/528: chown d4/d6/d3f/l57 13 1 2021-11-03T17:15:14.978 INFO:tasks.workunit.client.1.smithi115.stdout:8/529: dread d4/d6/f2a [0,4194304] 0 2021-11-03T17:15:14.978 INFO:tasks.workunit.client.1.smithi115.stdout:8/530: mknod d4/d6/d18/d9d/cb5 1 2021-11-03T17:15:14.979 INFO:tasks.workunit.client.1.smithi115.stdout:8/531: dread - d4/d6/d18/d44/f99 zero size 2021-11-03T17:15:14.987 INFO:tasks.workunit.client.1.smithi115.stdout:8/532: rename d4/d6/d3f/d43/d88/f8f to d4/d6/d18/d9d/fb6 0 2021-11-03T17:15:14.990 INFO:tasks.workunit.client.1.smithi115.stdout:8/533: unlink d4/d6/d18/d30/d33/d93/l9b 0 2021-11-03T17:15:14.997 INFO:tasks.workunit.client.1.smithi115.stdout:8/534: unlink d4/d6/d18/d44/d47/f7c 0 2021-11-03T17:15:15.001 INFO:tasks.workunit.client.1.smithi115.stdout:8/535: write d4/d6/d18/d44/d47/fab [534313,115276] 0 2021-11-03T17:15:15.003 INFO:tasks.workunit.client.1.smithi115.stdout:8/536: symlink d4/d6/d18/d30/d33/lb7 0 2021-11-03T17:15:15.004 INFO:tasks.workunit.client.1.smithi115.stdout:8/537: read d4/d6/d18/d44/f62 [476613,40570] 0 2021-11-03T17:15:15.009 INFO:tasks.workunit.client.1.smithi115.stdout:8/538: write d4/d6/d18/d30/f36 [1601192,37065] 0 2021-11-03T17:15:15.014 INFO:tasks.workunit.client.1.smithi115.stdout:8/539: unlink d4/d6/d18/d44/d6e/fa8 0 2021-11-03T17:15:15.016 INFO:tasks.workunit.client.1.smithi115.stdout:8/540: dread d4/d6/d3f/d9a/f65 [0,4194304] 0 2021-11-03T17:15:15.051 INFO:tasks.workunit.client.1.smithi115.stdout:9/552: sync 2021-11-03T17:15:15.060 INFO:tasks.workunit.client.1.smithi115.stdout:9/553: rename d1/d1a/d48/d27/d2f/f59 to d1/d18/d65/fca 0 2021-11-03T17:15:15.061 INFO:tasks.workunit.client.1.smithi115.stdout:9/554: symlink d1/d1a/lcb 0 2021-11-03T17:15:15.063 INFO:tasks.workunit.client.1.smithi115.stdout:9/555: mkdir d1/d1a/d48/d27/d80/dcc 0 2021-11-03T17:15:15.068 INFO:tasks.workunit.client.1.smithi115.stdout:9/556: rename d1/d1a/d48/d27/da9 to d1/d1a/d48/d27/d53/d83/d88/dcd 0 2021-11-03T17:15:15.072 INFO:tasks.workunit.client.1.smithi115.stdout:9/557: link d1/d1a/d48/f2a d1/d18/d4a/d74/dc7/fce 0 2021-11-03T17:15:15.073 INFO:tasks.workunit.client.1.smithi115.stdout:9/558: mknod d1/d18/d4a/d74/dc7/ccf 1 2021-11-03T17:15:15.074 INFO:tasks.workunit.client.1.smithi115.stdout:9/559: creat d1/d18/d2b/fd0 x:0 0 0 2021-11-03T17:15:15.084 INFO:tasks.workunit.client.1.smithi115.stdout:9/560: rename d1/d1a/f76 to d1/d66/fd1 0 2021-11-03T17:15:15.091 INFO:tasks.workunit.client.1.smithi115.stdout:9/561: dread d1/d1a/d48/d27/d53/f77 [0,4194304] 0 2021-11-03T17:15:15.092 INFO:tasks.workunit.client.0.smithi027.stdout:7/962: sync 2021-11-03T17:15:15.094 INFO:tasks.workunit.client.1.smithi115.stdout:9/562: symlink d1/d18/d2b/d42/d6e/ld2 0 2021-11-03T17:15:15.094 INFO:tasks.workunit.client.1.smithi115.stdout:9/563: mknod d1/d18/d2b/d42/d6e/cd3 1 2021-11-03T17:15:15.097 INFO:tasks.workunit.client.1.smithi115.stdout:9/564: rename d1/d1a/d48/f8d to d1/d1a/d48/d27/d2f/d6a/fd4 0 2021-11-03T17:15:15.097 INFO:tasks.workunit.client.1.smithi115.stdout:9/565: fdatasync d1/d18/d2b/fd0 0 2021-11-03T17:15:15.111 INFO:tasks.workunit.client.1.smithi115.stdout:9/566: rename d1/d1a/d48/d27/d53/d83/d88/d9e/d58/lb3 to d1/d18/d2b/d45/ld5 0 2021-11-03T17:15:15.118 INFO:tasks.workunit.client.0.smithi027.stdout:7/963: dwrite d0/d2/d1f/fa6 [0,4194304] 0 2021-11-03T17:15:15.121 INFO:tasks.workunit.client.0.smithi027.stdout:7/964: read d0/d2/d1f/f46 [3327606,119692] 0 2021-11-03T17:15:15.121 INFO:tasks.workunit.client.0.smithi027.stdout:7/965: write d0/d2/d1f/f11f [2429294,39696] 0 2021-11-03T17:15:15.122 INFO:tasks.workunit.client.0.smithi027.stdout:7/966: fsync d0/d2d/d2f/dae/ff4 0 2021-11-03T17:15:15.122 INFO:tasks.workunit.client.0.smithi027.stdout:7/967: read - d0/f125 zero size 2021-11-03T17:15:15.142 INFO:tasks.workunit.client.1.smithi115.stdout:9/567: dwrite d1/d66/fc0 [0,4194304] 0 2021-11-03T17:15:15.142 INFO:tasks.workunit.client.0.smithi027.stdout:7/968: dwrite d0/d2d/d66/f98 [0,4194304] 0 2021-11-03T17:15:15.144 INFO:tasks.workunit.client.1.smithi115.stdout:9/568: mknod d1/d18/d2b/d42/d63/cd6 1 2021-11-03T17:15:15.145 INFO:tasks.workunit.client.1.smithi115.stdout:9/569: dread - d1/d1a/d48/d27/d53/d83/d88/d9e/f90 zero size 2021-11-03T17:15:15.146 INFO:tasks.workunit.client.0.smithi027.stdout:7/969: unlink d0/d2/d10/d1e/l3e 0 2021-11-03T17:15:15.147 INFO:tasks.workunit.client.0.smithi027.stdout:7/970: chown d0/d2/d10 5 1 2021-11-03T17:15:15.148 INFO:tasks.workunit.client.1.smithi115.stdout:9/570: dread - d1/d18/d2b/d42/d6e/f5c zero size 2021-11-03T17:15:15.148 INFO:tasks.workunit.client.1.smithi115.stdout:9/571: mknod d1/d18/d65/cd7 1 2021-11-03T17:15:15.150 INFO:tasks.workunit.client.1.smithi115.stdout:9/572: creat d1/d1a/d48/d27/d80/dcc/fd8 x:0 0 0 2021-11-03T17:15:15.151 INFO:tasks.workunit.client.1.smithi115.stdout:9/573: rmdir d1/d18/d2b/d42/d63 39 2021-11-03T17:15:15.151 INFO:tasks.workunit.client.0.smithi027.stdout:7/971: rename d0/d2/d10/d1e/d20/d74/l128 to d0/d2/d10/d82/df2/dab/d11e/l139 0 2021-11-03T17:15:15.153 INFO:tasks.workunit.client.1.smithi115.stdout:9/574: stat d1/l4 0 2021-11-03T17:15:15.153 INFO:tasks.workunit.client.1.smithi115.stdout:9/575: read - d1/d1a/d48/d27/f4c zero size 2021-11-03T17:15:15.154 INFO:tasks.workunit.client.0.smithi027.stdout:7/972: symlink d0/d2/d10/d82/l13a 0 2021-11-03T17:15:15.154 INFO:tasks.workunit.client.1.smithi115.stdout:9/576: mkdir d1/d1a/dd9 0 2021-11-03T17:15:15.155 INFO:tasks.workunit.client.1.smithi115.stdout:9/577: readlink d1/d1a/d48/d27/d2f/l5d 0 2021-11-03T17:15:15.156 INFO:tasks.workunit.client.0.smithi027.stdout:7/973: creat d0/d2/d10/d1e/de6/f13b x:0 0 0 2021-11-03T17:15:15.157 INFO:tasks.workunit.client.1.smithi115.stdout:9/578: mkdir d1/d1a/d48/d27/d53/d83/d88/dcd/dda 0 2021-11-03T17:15:15.159 INFO:tasks.workunit.client.1.smithi115.stdout:2/571: sync 2021-11-03T17:15:15.160 INFO:tasks.workunit.client.1.smithi115.stdout:1/669: sync 2021-11-03T17:15:15.169 INFO:tasks.workunit.client.0.smithi027.stdout:7/974: rename d0/d2/d10 to d0/d2d/d2f/dae/dd8/d13c 0 2021-11-03T17:15:15.170 INFO:tasks.workunit.client.0.smithi027.stdout:7/975: read d0/d2d/d2f/dae/dd8/d13c/d82/df2/f102 [228213,62700] 0 2021-11-03T17:15:15.172 INFO:tasks.workunit.client.0.smithi027.stdout:7/976: fsync d0/d2d/d2f/dae/dd8/d13c/d1e/f88 0 2021-11-03T17:15:15.177 INFO:tasks.workunit.client.0.smithi027.stdout:7/977: rename d0/d2d/d2f/dae/dd8/d13c/d1e/f24 to d0/d2d/d2f/dae/dd8/d13c/f13d 0 2021-11-03T17:15:15.179 INFO:tasks.workunit.client.0.smithi027.stdout:7/978: rmdir d0/d2/d1c/d4d/d6e/d84 39 2021-11-03T17:15:15.183 INFO:tasks.workunit.client.1.smithi115.stdout:9/579: dwrite d1/f75 [0,4194304] 0 2021-11-03T17:15:15.191 INFO:tasks.workunit.client.1.smithi115.stdout:1/670: symlink d7/dc/ldc 0 2021-11-03T17:15:15.197 INFO:tasks.workunit.client.1.smithi115.stdout:9/580: truncate d1/d18/d65/faf 837613 0 2021-11-03T17:15:15.201 INFO:tasks.workunit.client.0.smithi027.stdout:7/979: dwrite d0/d2d/d2f/dae/dc8/fde [0,4194304] 0 2021-11-03T17:15:15.207 INFO:tasks.workunit.client.0.smithi027.stdout:7/980: rename d0/d2/d1c/d4d/d6e/d84/l60 to d0/d2/d1c/l13e 0 2021-11-03T17:15:15.212 INFO:tasks.workunit.client.1.smithi115.stdout:9/581: creat d1/d18/d2b/db9/fdb x:0 0 0 2021-11-03T17:15:15.216 INFO:tasks.workunit.client.1.smithi115.stdout:9/582: link d1/d1a/l38 d1/d66/ldc 0 2021-11-03T17:15:15.216 INFO:tasks.workunit.client.1.smithi115.stdout:2/572: dwrite d4/d6/f48 [0,4194304] 0 2021-11-03T17:15:15.216 INFO:tasks.workunit.client.1.smithi115.stdout:9/583: chown d1/d1a/d48/d27/d53/d83/d88/d9e/f6d 1 1 2021-11-03T17:15:15.217 INFO:tasks.workunit.client.1.smithi115.stdout:1/671: write d7/dc/dd/d3e/dd1/d4d/f69 [972403,21002] 0 2021-11-03T17:15:15.217 INFO:tasks.workunit.client.1.smithi115.stdout:9/584: getdents d1/d1a/d48/d27/d2f/d6a 0 2021-11-03T17:15:15.219 INFO:tasks.workunit.client.1.smithi115.stdout:9/585: chown d1/d1a/d48/d27/d53/d83/d88/dcd/l68 1103627 1 2021-11-03T17:15:15.220 INFO:tasks.workunit.client.1.smithi115.stdout:1/672: chown d7/dc/dd/d3e/dd1/d4d/d60/fb8 517 1 2021-11-03T17:15:15.222 INFO:tasks.workunit.client.1.smithi115.stdout:9/586: creat d1/d18/d4a/d74/fdd x:0 0 0 2021-11-03T17:15:15.224 INFO:tasks.workunit.client.1.smithi115.stdout:1/673: getdents d7/dc/dd/d3e/dbe 0 2021-11-03T17:15:15.226 INFO:tasks.workunit.client.1.smithi115.stdout:9/587: rename d1/d18/d4a/lb6 to d1/d1a/lde 0 2021-11-03T17:15:15.229 INFO:tasks.workunit.client.0.smithi027.stdout:7/981: dwrite d0/d2/d5e/d36/d3a/fd0 [0,4194304] 0 2021-11-03T17:15:15.229 INFO:tasks.workunit.client.0.smithi027.stdout:7/982: fsync d0/d2/d1f/fec 0 2021-11-03T17:15:15.230 INFO:tasks.workunit.client.0.smithi027.stdout:7/983: chown d0/d2/d1c/d4d/d6e/d8b/dea/d10c 188 1 2021-11-03T17:15:15.230 INFO:tasks.workunit.client.0.smithi027.stdout:7/984: fdatasync d0/d2d/d2f/dae/dd8/d13c/d1e/d20/d74/f9d 0 2021-11-03T17:15:15.230 INFO:tasks.workunit.client.1.smithi115.stdout:2/573: truncate d4/d6/d56/d6c/d34/d43/d63/d77/f81 2359568 0 2021-11-03T17:15:15.231 INFO:tasks.workunit.client.1.smithi115.stdout:9/588: dread - d1/d18/d2b/d42/d6e/db5/f99 zero size 2021-11-03T17:15:15.233 INFO:tasks.workunit.client.1.smithi115.stdout:2/574: mkdir d4/d6/d56/d6c/d34/d42/d8d/dc3 0 2021-11-03T17:15:15.236 INFO:tasks.workunit.client.0.smithi027.stdout:7/985: dread d0/d2d/d2f/dae/dd8/d13c/d82/de4/dfd/dfe/f137 [4194304,4194304] 0 2021-11-03T17:15:15.237 INFO:tasks.workunit.client.0.smithi027.stdout:7/986: dread - d0/d2d/d2f/dae/dc8/fcf zero size 2021-11-03T17:15:15.238 INFO:tasks.workunit.client.0.smithi027.stdout:7/987: write d0/d2d/d2f/dae/dd8/d13c/d1e/d20/d74/f9d [1045649,108120] 0 2021-11-03T17:15:15.241 INFO:tasks.workunit.client.1.smithi115.stdout:2/575: write d4/d6/d56/d6c/d34/f52 [5060515,29234] 0 2021-11-03T17:15:15.242 INFO:tasks.workunit.client.1.smithi115.stdout:2/576: creat d4/d6/d56/d6c/d34/d42/d8d/da9/fc4 x:0 0 0 2021-11-03T17:15:15.249 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.254+0000 7f6adef88700 1 -- 172.21.15.27:0/2773786206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ad80f66d0 msgr2=0x7f6ad80f6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:15:15.250 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.254+0000 7f6adef88700 1 --2- 172.21.15.27:0/2773786206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ad80f66d0 0x7f6ad80f6b30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f6ad4004660 tx=0x7f6ad4009b30).stop 2021-11-03T17:15:15.250 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.254+0000 7f6adef88700 1 -- 172.21.15.27:0/2773786206 shutdown_connections 2021-11-03T17:15:15.250 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.254+0000 7f6adef88700 1 --2- 172.21.15.27:0/2773786206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ad80f66d0 0x7f6ad80f6b30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:15.250 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.254+0000 7f6adef88700 1 --2- 172.21.15.27:0/2773786206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ad80f5430 0x7f6ad80f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:15.251 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.254+0000 7f6adef88700 1 -- 172.21.15.27:0/2773786206 >> 172.21.15.27:0/2773786206 conn(0x7f6ad80f0b90 msgr2=0x7f6ad80f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:15:15.251 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.254+0000 7f6adef88700 1 -- 172.21.15.27:0/2773786206 shutdown_connections 2021-11-03T17:15:15.251 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.254+0000 7f6adef88700 1 -- 172.21.15.27:0/2773786206 wait complete. 2021-11-03T17:15:15.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.254+0000 7f6adef88700 1 Processor -- start 2021-11-03T17:15:15.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.254+0000 7f6adef88700 1 -- start start 2021-11-03T17:15:15.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.255+0000 7f6adef88700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ad80f5430 0x7f6ad80f9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:15:15.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.255+0000 7f6adef88700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ad80f66d0 0x7f6ad80f8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:15:15.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.255+0000 7f6adef88700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad80f8790 con 0x7f6ad80f5430 2021-11-03T17:15:15.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.255+0000 7f6adef88700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ad80f88d0 con 0x7f6ad80f66d0 2021-11-03T17:15:15.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.255+0000 7f6addf86700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ad80f5430 0x7f6ad80f9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:15:15.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.255+0000 7f6add785700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ad80f66d0 0x7f6ad80f8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:15:15.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.255+0000 7f6addf86700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ad80f5430 0x7f6ad80f9db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55618/0 (socket says 172.21.15.27:55618) 2021-11-03T17:15:15.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.255+0000 7f6addf86700 1 -- 172.21.15.27:0/3854912796 learned_addr learned my addr 172.21.15.27:0/3854912796 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:15:15.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.255+0000 7f6addf86700 1 -- 172.21.15.27:0/3854912796 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ad80f66d0 msgr2=0x7f6ad80f8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:15:15.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.255+0000 7f6addf86700 1 --2- 172.21.15.27:0/3854912796 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ad80f66d0 0x7f6ad80f8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:15.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.255+0000 7f6addf86700 1 -- 172.21.15.27:0/3854912796 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad4005040 con 0x7f6ad80f5430 2021-11-03T17:15:15.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.255+0000 7f6add785700 1 --2- 172.21.15.27:0/3854912796 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ad80f66d0 0x7f6ad80f8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-03T17:15:15.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.255+0000 7f6addf86700 1 --2- 172.21.15.27:0/3854912796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ad80f5430 0x7f6ad80f9db0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f6ac800e7c0 tx=0x7f6ac800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:15:15.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.255+0000 7f6ace7fc700 1 -- 172.21.15.27:0/3854912796 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ac80092e0 con 0x7f6ad80f5430 2021-11-03T17:15:15.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.255+0000 7f6ace7fc700 1 -- 172.21.15.27:0/3854912796 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f6ac800f040 con 0x7f6ad80f5430 2021-11-03T17:15:15.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.255+0000 7f6adef88700 1 -- 172.21.15.27:0/3854912796 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ad80f8bb0 con 0x7f6ad80f5430 2021-11-03T17:15:15.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.255+0000 7f6ace7fc700 1 -- 172.21.15.27:0/3854912796 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ac80147e0 con 0x7f6ad80f5430 2021-11-03T17:15:15.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.255+0000 7f6adef88700 1 -- 172.21.15.27:0/3854912796 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ad810ed00 con 0x7f6ad80f5430 2021-11-03T17:15:15.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.256+0000 7f6adef88700 1 -- 172.21.15.27:0/3854912796 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ad8045ad0 con 0x7f6ad80f5430 2021-11-03T17:15:15.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.257+0000 7f6ace7fc700 1 -- 172.21.15.27:0/3854912796 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 28) v1 ==== 81971+0+0 (secure 0 0 0) 0x7f6ac801b030 con 0x7f6ad80f5430 2021-11-03T17:15:15.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.257+0000 7f6ace7fc700 1 --2- 172.21.15.27:0/3854912796 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7f6ac40640f0 0x7f6ac40665b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:15:15.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.258+0000 7f6ace7fc700 1 -- 172.21.15.27:0/3854912796 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6031+0+0 (secure 0 0 0) 0x7f6ac8080b40 con 0x7f6ad80f5430 2021-11-03T17:15:15.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.258+0000 7f6add785700 1 --2- 172.21.15.27:0/3854912796 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7f6ac40640f0 0x7f6ac40665b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:15:15.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.258+0000 7f6add785700 1 --2- 172.21.15.27:0/3854912796 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7f6ac40640f0 0x7f6ac40665b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f6ad4005010 tx=0x7f6ad4016040).ready entity=mgr.14622 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:15:15.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.261+0000 7f6ace7fc700 1 -- 172.21.15.27:0/3854912796 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156383 (secure 0 0 0) 0x7f6ac8052f20 con 0x7f6ad80f5430 2021-11-03T17:15:15.302 INFO:tasks.workunit.client.1.smithi115.stdout:2/577: unlink d4/d6/d13/d75/l7d 0 2021-11-03T17:15:15.302 INFO:tasks.workunit.client.1.smithi115.stdout:2/578: fdatasync d4/d6/d56/d6c/d34/d43/f2b 0 2021-11-03T17:15:15.303 INFO:tasks.workunit.client.1.smithi115.stdout:2/579: mknod d4/d6/d56/d6c/cc5 1 2021-11-03T17:15:15.305 INFO:tasks.workunit.client.1.smithi115.stdout:2/580: read d4/d6/d13/d75/f7a [1682921,12986] 0 2021-11-03T17:15:15.305 INFO:tasks.workunit.client.1.smithi115.stdout:2/581: chown d4/d6/d56/d6c/d34/d84 66403938 1 2021-11-03T17:15:15.308 INFO:tasks.workunit.client.1.smithi115.stdout:2/582: mknod d4/d6/dad/d7f/cc6 1 2021-11-03T17:15:15.317 INFO:tasks.workunit.client.1.smithi115.stdout:2/583: dread d4/d6/dad/f5f [0,4194304] 0 2021-11-03T17:15:15.320 INFO:tasks.workunit.client.1.smithi115.stdout:2/584: creat d4/d6/d56/d6c/d34/d43/d63/d77/d79/d9c/d9f/fc7 x:0 0 0 2021-11-03T17:15:15.325 INFO:tasks.workunit.client.1.smithi115.stdout:2/585: rename d4/d6/d13/lbc to d4/d6/d56/d6c/d34/d43/d28/lc8 0 2021-11-03T17:15:15.326 INFO:tasks.workunit.client.1.smithi115.stdout:2/586: creat d4/d6/d56/d6c/d34/d42/d8d/fc9 x:0 0 0 2021-11-03T17:15:15.331 INFO:tasks.workunit.client.1.smithi115.stdout:2/587: dread d4/d6/dad/f5f [0,4194304] 0 2021-11-03T17:15:15.335 INFO:tasks.workunit.client.1.smithi115.stdout:2/588: dread d4/f35 [0,4194304] 0 2021-11-03T17:15:15.393 INFO:tasks.workunit.client.1.smithi115.stdout:2/589: dwrite d4/f5a [0,4194304] 0 2021-11-03T17:15:15.394 INFO:tasks.workunit.client.1.smithi115.stdout:2/590: chown d4/f3a 46789342 1 2021-11-03T17:15:15.395 INFO:tasks.workunit.client.1.smithi115.stdout:2/591: creat d4/fca x:0 0 0 2021-11-03T17:15:15.398 INFO:tasks.workunit.client.1.smithi115.stdout:2/592: rename d4/d6/d56/d6c/f14 to d4/d6/d56/d6c/d34/fcb 0 2021-11-03T17:15:15.399 INFO:tasks.workunit.client.1.smithi115.stdout:2/593: mknod d4/d6/dad/dc2/ccc 1 2021-11-03T17:15:15.418 INFO:tasks.workunit.client.1.smithi115.stdout:2/594: dwrite d4/d6/d13/f38 [0,4194304] 0 2021-11-03T17:15:15.421 INFO:tasks.workunit.client.1.smithi115.stdout:2/595: mkdir d4/d6/d56/d6c/d34/d43/d63/d77/d79/dac/dcd 0 2021-11-03T17:15:15.422 INFO:tasks.workunit.client.1.smithi115.stdout:2/596: dread - d4/d6/d56/d6c/d34/d42/d8d/da9/fc4 zero size 2021-11-03T17:15:15.424 INFO:tasks.workunit.client.1.smithi115.stdout:2/597: dread d4/d6/d13/f21 [0,4194304] 0 2021-11-03T17:15:15.425 INFO:tasks.workunit.client.1.smithi115.stdout:2/598: creat d4/d6/d56/d6c/d34/d42/d8d/fce x:0 0 0 2021-11-03T17:15:15.426 INFO:tasks.workunit.client.1.smithi115.stdout:2/599: chown d4/d6/f2a 1 1 2021-11-03T17:15:15.426 INFO:tasks.workunit.client.1.smithi115.stdout:2/600: mknod d4/d6/d56/ccf 1 2021-11-03T17:15:15.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.447+0000 7f6adef88700 1 -- 172.21.15.27:0/3854912796 --> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ad805fd30 con 0x7f6ac40640f0 2021-11-03T17:15:15.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.449+0000 7f6ace7fc700 1 -- 172.21.15.27:0/3854912796 <== mgr.14622 v2:172.21.15.27:6828/3927816089 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+288 (secure 0 0 0) 0x7f6ad805fd30 con 0x7f6ac40640f0 2021-11-03T17:15:15.445 INFO:tasks.workunit.client.1.smithi115.stdout:2/601: dwrite d4/d6/d13/d75/f7b [0,4194304] 0 2021-11-03T17:15:15.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.451+0000 7f6adef88700 1 -- 172.21.15.27:0/3854912796 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7f6ac40640f0 msgr2=0x7f6ac40665b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:15:15.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.451+0000 7f6adef88700 1 --2- 172.21.15.27:0/3854912796 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7f6ac40640f0 0x7f6ac40665b0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f6ad4005010 tx=0x7f6ad4016040).stop 2021-11-03T17:15:15.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.451+0000 7f6adef88700 1 -- 172.21.15.27:0/3854912796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ad80f5430 msgr2=0x7f6ad80f9db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:15:15.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.451+0000 7f6adef88700 1 --2- 172.21.15.27:0/3854912796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ad80f5430 0x7f6ad80f9db0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f6ac800e7c0 tx=0x7f6ac800b040).stop 2021-11-03T17:15:15.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.451+0000 7f6adef88700 1 -- 172.21.15.27:0/3854912796 shutdown_connections 2021-11-03T17:15:15.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.451+0000 7f6adef88700 1 --2- 172.21.15.27:0/3854912796 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7f6ac40640f0 0x7f6ac40665b0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:15.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.451+0000 7f6adef88700 1 --2- 172.21.15.27:0/3854912796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ad80f5430 0x7f6ad80f9db0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:15.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.451+0000 7f6adef88700 1 --2- 172.21.15.27:0/3854912796 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ad80f66d0 0x7f6ad80f8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:15.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.451+0000 7f6adef88700 1 -- 172.21.15.27:0/3854912796 >> 172.21.15.27:0/3854912796 conn(0x7f6ad80f0b90 msgr2=0x7f6ad805f620 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:15:15.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.451+0000 7f6adef88700 1 -- 172.21.15.27:0/3854912796 shutdown_connections 2021-11-03T17:15:15.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.451+0000 7f6adef88700 1 -- 172.21.15.27:0/3854912796 wait complete. 2021-11-03T17:15:15.458 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:15:15.471 INFO:tasks.workunit.client.1.smithi115.stdout:2/602: dwrite d4/d6/d13/f6b [0,4194304] 0 2021-11-03T17:15:15.472 INFO:tasks.workunit.client.1.smithi115.stdout:2/603: chown d4/d6/d56/d6c/d34/d43/d63/d77/d79 6785486 1 2021-11-03T17:15:15.476 INFO:tasks.workunit.client.1.smithi115.stdout:2/604: truncate d4/d6/d13/f6b 160349 0 2021-11-03T17:15:15.477 INFO:tasks.workunit.client.1.smithi115.stdout:2/605: truncate d4/d6/d13/f4e 411979 0 2021-11-03T17:15:15.478 INFO:tasks.workunit.client.1.smithi115.stdout:2/606: truncate d4/d6/d13/f21 575437 0 2021-11-03T17:15:15.478 INFO:tasks.workunit.client.1.smithi115.stdout:2/607: chown d4/d6/d56/d6c/d34/d43/d63/d77/d79/d9c/d9f/fc7 167387887 1 2021-11-03T17:15:15.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.557+0000 7f6ae8f5f700 1 -- 172.21.15.27:0/809887502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae40f4800 msgr2=0x7f6ae40f4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:15:15.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.557+0000 7f6ae8f5f700 1 --2- 172.21.15.27:0/809887502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae40f4800 0x7f6ae40f4c20 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f6acc004660 tx=0x7f6acc009b30).stop 2021-11-03T17:15:15.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.557+0000 7f6ae8f5f700 1 -- 172.21.15.27:0/809887502 shutdown_connections 2021-11-03T17:15:15.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.557+0000 7f6ae8f5f700 1 --2- 172.21.15.27:0/809887502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ae40f51a0 0x7f6ae40f7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:15.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.557+0000 7f6ae8f5f700 1 --2- 172.21.15.27:0/809887502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae40f4800 0x7f6ae40f4c20 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:15.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.557+0000 7f6ae8f5f700 1 -- 172.21.15.27:0/809887502 >> 172.21.15.27:0/809887502 conn(0x7f6ae40f0320 msgr2=0x7f6ae40f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:15:15.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.557+0000 7f6ae8f5f700 1 -- 172.21.15.27:0/809887502 shutdown_connections 2021-11-03T17:15:15.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.557+0000 7f6ae8f5f700 1 -- 172.21.15.27:0/809887502 wait complete. 2021-11-03T17:15:15.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.558+0000 7f6ae8f5f700 1 Processor -- start 2021-11-03T17:15:15.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.558+0000 7f6ae8f5f700 1 -- start start 2021-11-03T17:15:15.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.558+0000 7f6ae8f5f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae40f4800 0x7f6ae4106dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:15:15.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.558+0000 7f6ae8f5f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ae40f51a0 0x7f6ae41072b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:15:15.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.558+0000 7f6ae8f5f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae41078b0 con 0x7f6ae40f4800 2021-11-03T17:15:15.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.558+0000 7f6ae8f5f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae41080f0 con 0x7f6ae40f51a0 2021-11-03T17:15:15.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.558+0000 7f6ae37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae40f4800 0x7f6ae4106dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:15:15.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.558+0000 7f6ae37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae40f4800 0x7f6ae4106dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55628/0 (socket says 172.21.15.27:55628) 2021-11-03T17:15:15.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.558+0000 7f6ae37fe700 1 -- 172.21.15.27:0/766240827 learned_addr learned my addr 172.21.15.27:0/766240827 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:15:15.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.558+0000 7f6ae2ffd700 1 --2- 172.21.15.27:0/766240827 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ae40f51a0 0x7f6ae41072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:15:15.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.559+0000 7f6ae37fe700 1 -- 172.21.15.27:0/766240827 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ae40f51a0 msgr2=0x7f6ae41072b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:15:15.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.559+0000 7f6ae37fe700 1 --2- 172.21.15.27:0/766240827 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ae40f51a0 0x7f6ae41072b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:15.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.559+0000 7f6ae37fe700 1 -- 172.21.15.27:0/766240827 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6acc005040 con 0x7f6ae40f4800 2021-11-03T17:15:15.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.559+0000 7f6ae37fe700 1 --2- 172.21.15.27:0/766240827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae40f4800 0x7f6ae4106dc0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f6acc013a70 tx=0x7f6acc009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:15:15.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.559+0000 7f6adbfff700 1 -- 172.21.15.27:0/766240827 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6acc018070 con 0x7f6ae40f4800 2021-11-03T17:15:15.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.559+0000 7f6adbfff700 1 -- 172.21.15.27:0/766240827 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f6acc0097d0 con 0x7f6ae40f4800 2021-11-03T17:15:15.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.559+0000 7f6ae8f5f700 1 -- 172.21.15.27:0/766240827 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ae4108370 con 0x7f6ae40f4800 2021-11-03T17:15:15.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.559+0000 7f6adbfff700 1 -- 172.21.15.27:0/766240827 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6acc01de80 con 0x7f6ae40f4800 2021-11-03T17:15:15.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.559+0000 7f6ae8f5f700 1 -- 172.21.15.27:0/766240827 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ae4108850 con 0x7f6ae40f4800 2021-11-03T17:15:15.562 INFO:tasks.workunit.client.1.smithi115.stdout:2/608: rename d4/d6/dad/f5f to d4/d6/d56/d6c/d34/d43/d63/d77/d79/d9c/d9f/fd0 0 2021-11-03T17:15:15.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.560+0000 7f6ae8f5f700 1 -- 172.21.15.27:0/766240827 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ae4059100 con 0x7f6ae40f4800 2021-11-03T17:15:15.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.560+0000 7f6adbfff700 1 -- 172.21.15.27:0/766240827 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 28) v1 ==== 81971+0+0 (secure 0 0 0) 0x7f6acc007ba0 con 0x7f6ae40f4800 2021-11-03T17:15:15.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.561+0000 7f6adbfff700 1 --2- 172.21.15.27:0/766240827 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7f6ad00640f0 0x7f6ad00665b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:15:15.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.561+0000 7f6adbfff700 1 -- 172.21.15.27:0/766240827 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6031+0+0 (secure 0 0 0) 0x7f6acc0821e0 con 0x7f6ae40f4800 2021-11-03T17:15:15.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.561+0000 7f6ae2ffd700 1 --2- 172.21.15.27:0/766240827 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7f6ad00640f0 0x7f6ad00665b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:15:15.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.561+0000 7f6ae2ffd700 1 --2- 172.21.15.27:0/766240827 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7f6ad00640f0 0x7f6ad00665b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f6ae4106b50 tx=0x7f6ad400b040).ready entity=mgr.14622 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:15:15.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.565+0000 7f6adbfff700 1 -- 172.21.15.27:0/766240827 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156383 (secure 0 0 0) 0x7f6acc054830 con 0x7f6ae40f4800 2021-11-03T17:15:15.570 INFO:tasks.workunit.client.1.smithi115.stdout:2/609: unlink d4/d6/d56/d6c/d34/d43/d63/d69/fa7 0 2021-11-03T17:15:15.571 INFO:tasks.workunit.client.1.smithi115.stdout:2/610: chown d4/d6/d56/d6c/f39 74978578 1 2021-11-03T17:15:15.572 INFO:tasks.workunit.client.1.smithi115.stdout:2/611: mknod d4/d6/dad/dc2/cd1 1 2021-11-03T17:15:15.738 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.742+0000 7f6ae8f5f700 1 -- 172.21.15.27:0/766240827 --> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ae40fee80 con 0x7f6ad00640f0 2021-11-03T17:15:15.759 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.763+0000 7f6adbfff700 1 -- 172.21.15.27:0/766240827 <== mgr.14622 v2:172.21.15.27:6828/3927816089 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f6ae40fee80 con 0x7f6ad00640f0 2021-11-03T17:15:15.759 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:15:15.760 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (6m) 4s ago 8m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:15:15.760 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (8m) 4s ago 8m 7013k - 16.2.4 8d91d370c2b8 d494472a47c0 2021-11-03T17:15:15.760 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (7m) 10s ago 7m 7130k - 16.2.4 8d91d370c2b8 f84fd4042530 2021-11-03T17:15:15.761 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (6m) 4s ago 7m 38.3M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:15:15.761 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (19s) 4s ago 3m 12.0M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:15:15.761 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (3m) 4s ago 3m 2177M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:15:15.761 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (3m) 10s ago 3m 17.6M - 16.2.4 8d91d370c2b8 bfd990d6415e 2021-11-03T17:15:15.762 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (40s) 10s ago 3m 13.7M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:15:15.762 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (32s) 4s ago 9m 436M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:15:15.762 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8443,9283 running (61s) 10s ago 7m 99.8M - 17.0.0-8776-g6e90212d 50416d063274 c53838a65398 2021-11-03T17:15:15.762 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (9m) 4s ago 9m 104M 2048M 16.2.4 8d91d370c2b8 060d5938214f 2021-11-03T17:15:15.763 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (7m) 10s ago 7m 89.2M 2048M 16.2.4 8d91d370c2b8 319323f09834 2021-11-03T17:15:15.763 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (7m) 4s ago 7m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:15:15.763 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (7m) 10s ago 7m 18.2M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:15:15.764 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (6m) 4s ago 6m 532M 4096M 16.2.4 8d91d370c2b8 17f79a815a0f 2021-11-03T17:15:15.764 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (6m) 4s ago 6m 661M 4096M 16.2.4 8d91d370c2b8 422be0cac0db 2021-11-03T17:15:15.764 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (5m) 4s ago 5m 449M 4096M 16.2.4 8d91d370c2b8 c211ef5551c9 2021-11-03T17:15:15.764 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (5m) 10s ago 5m 688M 4096M 16.2.4 8d91d370c2b8 b1486ec1b619 2021-11-03T17:15:15.765 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (5m) 10s ago 5m 519M 4096M 16.2.4 8d91d370c2b8 8b91230c69e5 2021-11-03T17:15:15.765 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (4m) 10s ago 4m 699M 4096M 16.2.4 8d91d370c2b8 4a54c4d6b6f5 2021-11-03T17:15:15.765 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (6m) 4s ago 7m 42.8M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:15:15.766 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.765+0000 7f6ae8f5f700 1 -- 172.21.15.27:0/766240827 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7f6ad00640f0 msgr2=0x7f6ad00665b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:15:15.767 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.765+0000 7f6ae8f5f700 1 --2- 172.21.15.27:0/766240827 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7f6ad00640f0 0x7f6ad00665b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f6ae4106b50 tx=0x7f6ad400b040).stop 2021-11-03T17:15:15.767 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.765+0000 7f6ae8f5f700 1 -- 172.21.15.27:0/766240827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae40f4800 msgr2=0x7f6ae4106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:15:15.767 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.765+0000 7f6ae8f5f700 1 --2- 172.21.15.27:0/766240827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae40f4800 0x7f6ae4106dc0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f6acc013a70 tx=0x7f6acc009b30).stop 2021-11-03T17:15:15.767 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.765+0000 7f6ae8f5f700 1 -- 172.21.15.27:0/766240827 shutdown_connections 2021-11-03T17:15:15.768 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.765+0000 7f6ae8f5f700 1 --2- 172.21.15.27:0/766240827 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7f6ad00640f0 0x7f6ad00665b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:15.768 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.765+0000 7f6ae8f5f700 1 --2- 172.21.15.27:0/766240827 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae40f4800 0x7f6ae4106dc0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:15.768 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.765+0000 7f6ae8f5f700 1 --2- 172.21.15.27:0/766240827 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ae40f51a0 0x7f6ae41072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:15.768 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.765+0000 7f6ae8f5f700 1 -- 172.21.15.27:0/766240827 >> 172.21.15.27:0/766240827 conn(0x7f6ae40f0320 msgr2=0x7f6ae40fb370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:15:15.768 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.765+0000 7f6ae8f5f700 1 -- 172.21.15.27:0/766240827 shutdown_connections 2021-11-03T17:15:15.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.765+0000 7f6ae8f5f700 1 -- 172.21.15.27:0/766240827 wait complete. 2021-11-03T17:15:15.862 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.865+0000 7ff820ec9700 1 -- 172.21.15.27:0/3635303738 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff81c0f5ed0 msgr2=0x7ff81c0f6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:15:15.862 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.865+0000 7ff820ec9700 1 --2- 172.21.15.27:0/3635303738 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff81c0f5ed0 0x7ff81c0f6900 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7ff804004660 tx=0x7ff804009b30).stop 2021-11-03T17:15:15.862 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.865+0000 7ff820ec9700 1 -- 172.21.15.27:0/3635303738 shutdown_connections 2021-11-03T17:15:15.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.865+0000 7ff820ec9700 1 --2- 172.21.15.27:0/3635303738 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff81c0f5ed0 0x7ff81c0f6900 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:15.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.865+0000 7ff820ec9700 1 --2- 172.21.15.27:0/3635303738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff81c0f55c0 0x7ff81c0f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:15.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.865+0000 7ff820ec9700 1 -- 172.21.15.27:0/3635303738 >> 172.21.15.27:0/3635303738 conn(0x7ff81c0f0de0 msgr2=0x7ff81c0f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:15:15.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.865+0000 7ff820ec9700 1 -- 172.21.15.27:0/3635303738 shutdown_connections 2021-11-03T17:15:15.864 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.865+0000 7ff820ec9700 1 -- 172.21.15.27:0/3635303738 wait complete. 2021-11-03T17:15:15.864 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.865+0000 7ff820ec9700 1 Processor -- start 2021-11-03T17:15:15.864 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.865+0000 7ff820ec9700 1 -- start start 2021-11-03T17:15:15.864 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.866+0000 7ff820ec9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff81c0f55c0 0x7ff81c109bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:15:15.865 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.866+0000 7ff820ec9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff81c0f5ed0 0x7ff81c10c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:15:15.865 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.866+0000 7ff820ec9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c060540 con 0x7ff81c0f55c0 2021-11-03T17:15:15.865 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.866+0000 7ff820ec9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff81c060680 con 0x7ff81c0f5ed0 2021-11-03T17:15:15.865 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.866+0000 7ff81a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff81c0f55c0 0x7ff81c109bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:15:15.866 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.866+0000 7ff81a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff81c0f55c0 0x7ff81c109bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55638/0 (socket says 172.21.15.27:55638) 2021-11-03T17:15:15.866 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.866+0000 7ff81a59c700 1 -- 172.21.15.27:0/3558130565 learned_addr learned my addr 172.21.15.27:0/3558130565 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:15:15.866 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.866+0000 7ff819d9b700 1 --2- 172.21.15.27:0/3558130565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff81c0f5ed0 0x7ff81c10c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:15:15.867 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.866+0000 7ff81a59c700 1 -- 172.21.15.27:0/3558130565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff81c0f5ed0 msgr2=0x7ff81c10c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:15:15.867 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.866+0000 7ff81a59c700 1 --2- 172.21.15.27:0/3558130565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff81c0f5ed0 0x7ff81c10c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:15.867 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.866+0000 7ff81a59c700 1 -- 172.21.15.27:0/3558130565 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff804005040 con 0x7ff81c0f55c0 2021-11-03T17:15:15.868 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.866+0000 7ff81a59c700 1 --2- 172.21.15.27:0/3558130565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff81c0f55c0 0x7ff81c109bd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7ff81400e7f0 tx=0x7ff81400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:15:15.868 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.866+0000 7ff812ffd700 1 -- 172.21.15.27:0/3558130565 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8140092e0 con 0x7ff81c0f55c0 2021-11-03T17:15:15.871 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.866+0000 7ff812ffd700 1 -- 172.21.15.27:0/3558130565 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7ff81400f040 con 0x7ff81c0f55c0 2021-11-03T17:15:15.872 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.866+0000 7ff820ec9700 1 -- 172.21.15.27:0/3558130565 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff81c10c680 con 0x7ff81c0f55c0 2021-11-03T17:15:15.872 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.866+0000 7ff812ffd700 1 -- 172.21.15.27:0/3558130565 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff814014750 con 0x7ff81c0f55c0 2021-11-03T17:15:15.872 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.866+0000 7ff820ec9700 1 -- 172.21.15.27:0/3558130565 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff81c10cc40 con 0x7ff81c0f55c0 2021-11-03T17:15:15.872 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.867+0000 7ff820ec9700 1 -- 172.21.15.27:0/3558130565 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff81c045ad0 con 0x7ff81c0f55c0 2021-11-03T17:15:15.872 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.868+0000 7ff812ffd700 1 -- 172.21.15.27:0/3558130565 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 28) v1 ==== 81971+0+0 (secure 0 0 0) 0x7ff81401b030 con 0x7ff81c0f55c0 2021-11-03T17:15:15.873 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.868+0000 7ff812ffd700 1 --2- 172.21.15.27:0/3558130565 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7ff8080640a0 0x7ff808066560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:15:15.873 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.868+0000 7ff812ffd700 1 -- 172.21.15.27:0/3558130565 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6031+0+0 (secure 0 0 0) 0x7ff8140803e0 con 0x7ff81c0f55c0 2021-11-03T17:15:15.873 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.868+0000 7ff819d9b700 1 --2- 172.21.15.27:0/3558130565 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7ff8080640a0 0x7ff808066560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:15:15.873 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.869+0000 7ff819d9b700 1 --2- 172.21.15.27:0/3558130565 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7ff8080640a0 0x7ff808066560 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7ff804005010 tx=0x7ff804016040).ready entity=mgr.14622 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:15:15.874 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:15.872+0000 7ff812ffd700 1 -- 172.21.15.27:0/3558130565 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156383 (secure 0 0 0) 0x7ff814052930 con 0x7ff81c0f55c0 2021-11-03T17:15:16.100 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.104+0000 7ff820ec9700 1 -- 172.21.15.27:0/3558130565 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff81c0fdef0 con 0x7ff81c0f55c0 2021-11-03T17:15:16.101 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.105+0000 7ff812ffd700 1 -- 172.21.15.27:0/3558130565 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+689 (secure 0 0 0) 0x7ff8140192d0 con 0x7ff81c0f55c0 2021-11-03T17:15:16.101 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:15:16.101 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:15:16.102 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 2 2021-11-03T17:15:16.102 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:15:16.102 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:15:16.102 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:15:16.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:15:16.103 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:15:16.103 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 6 2021-11-03T17:15:16.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:15:16.103 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:15:16.104 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:15:16.104 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:15:16.104 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:15:16.104 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 12, 2021-11-03T17:15:16.105 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:15:16.105 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:15:16.105 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:15:16.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.106+0000 7ff820ec9700 1 -- 172.21.15.27:0/3558130565 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7ff8080640a0 msgr2=0x7ff808066560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:15:16.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.106+0000 7ff820ec9700 1 --2- 172.21.15.27:0/3558130565 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7ff8080640a0 0x7ff808066560 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7ff804005010 tx=0x7ff804016040).stop 2021-11-03T17:15:16.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.106+0000 7ff820ec9700 1 -- 172.21.15.27:0/3558130565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff81c0f55c0 msgr2=0x7ff81c109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:15:16.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.106+0000 7ff820ec9700 1 --2- 172.21.15.27:0/3558130565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff81c0f55c0 0x7ff81c109bd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7ff81400e7f0 tx=0x7ff81400b040).stop 2021-11-03T17:15:16.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.106+0000 7ff820ec9700 1 -- 172.21.15.27:0/3558130565 shutdown_connections 2021-11-03T17:15:16.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.106+0000 7ff820ec9700 1 --2- 172.21.15.27:0/3558130565 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7ff8080640a0 0x7ff808066560 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:16.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.106+0000 7ff820ec9700 1 --2- 172.21.15.27:0/3558130565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff81c0f55c0 0x7ff81c109bd0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:16.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.106+0000 7ff820ec9700 1 --2- 172.21.15.27:0/3558130565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff81c0f5ed0 0x7ff81c10c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:16.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.106+0000 7ff820ec9700 1 -- 172.21.15.27:0/3558130565 >> 172.21.15.27:0/3558130565 conn(0x7ff81c0f0de0 msgr2=0x7ff81c0fa350 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:15:16.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.106+0000 7ff820ec9700 1 -- 172.21.15.27:0/3558130565 shutdown_connections 2021-11-03T17:15:16.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.107+0000 7ff820ec9700 1 -- 172.21.15.27:0/3558130565 wait complete. 2021-11-03T17:15:16.197 INFO:tasks.workunit.client.0.smithi027.stdout:7/988: sync 2021-11-03T17:15:16.200 INFO:tasks.workunit.client.0.smithi027.stdout:7/989: creat d0/d2d/d2f/dae/f13f x:0 0 0 2021-11-03T17:15:16.201 INFO:tasks.workunit.client.0.smithi027.stdout:7/990: getdents d0/d2/d5e/d36/d3a 0 2021-11-03T17:15:16.202 INFO:tasks.workunit.client.0.smithi027.stdout:7/991: mknod d0/d2d/d2f/dae/dd8/d13c/d82/de4/c140 1 2021-11-03T17:15:16.204 INFO:tasks.workunit.client.0.smithi027.stdout:7/992: mkdir d0/d2/d1c/d4d/d6e/d8b/d141 0 2021-11-03T17:15:16.204 INFO:tasks.workunit.client.0.smithi027.stdout:7/993: chown d0/d2d/d58/dba/l11a 357 1 2021-11-03T17:15:16.205 INFO:tasks.workunit.client.0.smithi027.stdout:7/994: stat d0/d2d/d2f/dae/dd8/d13c 0 2021-11-03T17:15:16.205 INFO:tasks.workunit.client.0.smithi027.stdout:7/995: chown d0/d2d/d2f/dae/dd8/d13c/d82/df2/dab/d11e 834976440 1 2021-11-03T17:15:16.206 INFO:tasks.workunit.client.0.smithi027.stdout:7/996: chown d0/d2/l33 8 1 2021-11-03T17:15:16.206 INFO:tasks.workunit.client.0.smithi027.stdout:7/997: chown d0/lc 7 1 2021-11-03T17:15:16.206 INFO:tasks.workunit.client.0.smithi027.stdout:7/998: write d0/d2/d1c/d4d/dcc/fdb [985414,124728] 0 2021-11-03T17:15:16.209 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.210+0000 7f2bae931700 1 -- 172.21.15.27:0/3647237455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ba80f73f0 msgr2=0x7f2ba80f5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:15:16.209 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.210+0000 7f2bae931700 1 --2- 172.21.15.27:0/3647237455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ba80f73f0 0x7f2ba80f5810 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f2b98004660 tx=0x7f2b98009b30).stop 2021-11-03T17:15:16.210 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.210+0000 7f2bae931700 1 -- 172.21.15.27:0/3647237455 shutdown_connections 2021-11-03T17:15:16.210 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.210+0000 7f2bae931700 1 --2- 172.21.15.27:0/3647237455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ba80f73f0 0x7f2ba80f5810 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:16.210 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.210+0000 7f2bae931700 1 --2- 172.21.15.27:0/3647237455 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ba80f6a50 0x7f2ba80f6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:16.210 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.210+0000 7f2bae931700 1 -- 172.21.15.27:0/3647237455 >> 172.21.15.27:0/3647237455 conn(0x7f2ba80f0320 msgr2=0x7f2ba80f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:15:16.211 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.210+0000 7f2bae931700 1 -- 172.21.15.27:0/3647237455 shutdown_connections 2021-11-03T17:15:16.211 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.210+0000 7f2bae931700 1 -- 172.21.15.27:0/3647237455 wait complete. 2021-11-03T17:15:16.211 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.211+0000 7f2bae931700 1 Processor -- start 2021-11-03T17:15:16.211 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.211+0000 7f2bae931700 1 -- start start 2021-11-03T17:15:16.211 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.211+0000 7f2bae931700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ba80f6a50 0x7f2ba8106d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:15:16.212 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.211+0000 7f2bae931700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ba80f73f0 0x7f2ba8107220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:15:16.212 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.211+0000 7f2bae931700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba81077d0 con 0x7f2ba80f73f0 2021-11-03T17:15:16.212 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.211+0000 7f2bae931700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba8107fd0 con 0x7f2ba80f6a50 2021-11-03T17:15:16.212 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.211+0000 7f2bad12e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ba80f73f0 0x7f2ba8107220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:15:16.213 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.211+0000 7f2bad92f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ba80f6a50 0x7f2ba8106d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:15:16.213 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.211+0000 7f2bad12e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ba80f73f0 0x7f2ba8107220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55650/0 (socket says 172.21.15.27:55650) 2021-11-03T17:15:16.213 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.211+0000 7f2bad12e700 1 -- 172.21.15.27:0/563124819 learned_addr learned my addr 172.21.15.27:0/563124819 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:15:16.213 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.211+0000 7f2bad12e700 1 -- 172.21.15.27:0/563124819 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ba80f6a50 msgr2=0x7f2ba8106d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:15:16.213 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.211+0000 7f2bad12e700 1 --2- 172.21.15.27:0/563124819 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ba80f6a50 0x7f2ba8106d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:16.214 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.211+0000 7f2bad12e700 1 -- 172.21.15.27:0/563124819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b98005040 con 0x7f2ba80f73f0 2021-11-03T17:15:16.214 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.211+0000 7f2bad12e700 1 --2- 172.21.15.27:0/563124819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ba80f73f0 0x7f2ba8107220 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f2b98000c00 tx=0x7f2b9800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:15:16.214 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.212+0000 7f2b9e7fc700 1 -- 172.21.15.27:0/563124819 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b98019070 con 0x7f2ba80f73f0 2021-11-03T17:15:16.214 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.212+0000 7f2b9e7fc700 1 -- 172.21.15.27:0/563124819 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f2b98007e70 con 0x7f2ba80f73f0 2021-11-03T17:15:16.215 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.212+0000 7f2bae931700 1 -- 172.21.15.27:0/563124819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ba8108250 con 0x7f2ba80f73f0 2021-11-03T17:15:16.215 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.212+0000 7f2b9e7fc700 1 -- 172.21.15.27:0/563124819 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b98014ce0 con 0x7f2ba80f73f0 2021-11-03T17:15:16.216 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.212+0000 7f2bae931700 1 -- 172.21.15.27:0/563124819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ba81087b0 con 0x7f2ba80f73f0 2021-11-03T17:15:16.216 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.213+0000 7f2bae931700 1 -- 172.21.15.27:0/563124819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ba8059100 con 0x7f2ba80f73f0 2021-11-03T17:15:16.216 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.214+0000 7f2b9e7fc700 1 -- 172.21.15.27:0/563124819 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 28) v1 ==== 81971+0+0 (secure 0 0 0) 0x7f2b98025070 con 0x7f2ba80f73f0 2021-11-03T17:15:16.217 INFO:tasks.workunit.client.1.smithi115.stdout:3/674: sync 2021-11-03T17:15:16.218 INFO:tasks.workunit.client.1.smithi115.stdout:4/669: sync 2021-11-03T17:15:16.218 INFO:tasks.workunit.client.1.smithi115.stdout:0/737: sync 2021-11-03T17:15:16.219 INFO:tasks.workunit.client.1.smithi115.stdout:7/504: sync 2021-11-03T17:15:16.219 INFO:tasks.workunit.client.1.smithi115.stdout:6/466: sync 2021-11-03T17:15:16.219 INFO:tasks.workunit.client.1.smithi115.stdout:5/647: sync 2021-11-03T17:15:16.219 INFO:tasks.workunit.client.1.smithi115.stdout:4/670: dread - d7/dc/d1a/d21/fd5 zero size 2021-11-03T17:15:16.220 INFO:tasks.workunit.client.1.smithi115.stdout:0/738: truncate d9/db/d89/f9c 3541304 0 2021-11-03T17:15:16.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.214+0000 7f2b9e7fc700 1 --2- 172.21.15.27:0/563124819 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7f2b940640f0 0x7f2b940665b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:15:16.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.214+0000 7f2b9e7fc700 1 -- 172.21.15.27:0/563124819 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6031+0+0 (secure 0 0 0) 0x7f2b98081460 con 0x7f2ba80f73f0 2021-11-03T17:15:16.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.214+0000 7f2bad92f700 1 --2- 172.21.15.27:0/563124819 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7f2b940640f0 0x7f2b940665b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:15:16.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.215+0000 7f2bad92f700 1 --2- 172.21.15.27:0/563124819 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7f2b940640f0 0x7f2b940665b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f2ba4009fd0 tx=0x7f2ba400b040).ready entity=mgr.14622 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:15:16.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.218+0000 7f2b9e7fc700 1 -- 172.21.15.27:0/563124819 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+156383 (secure 0 0 0) 0x7f2b980539f0 con 0x7f2ba80f73f0 2021-11-03T17:15:16.222 INFO:tasks.workunit.client.0.smithi027.stdout:7/999: write d0/d2/d1c/d4d/d6e/f12e [1785293,105792] 0 2021-11-03T17:15:16.222 INFO:tasks.workunit.client.0.smithi027.stderr:+ rm -rf -- ./tmp.4PQBNuAN7C 2021-11-03T17:15:16.227 INFO:tasks.workunit.client.1.smithi115.stdout:7/505: dwrite d3/da/d18/d73/d35/f75 [0,4194304] 0 2021-11-03T17:15:16.227 INFO:tasks.workunit.client.1.smithi115.stdout:7/506: write d3/da/d18/d73/d33/f9b [763276,113690] 0 2021-11-03T17:15:16.246 INFO:tasks.workunit.client.1.smithi115.stdout:3/675: creat d13/d2a/d53/d70/d92/fe4 x:0 0 0 2021-11-03T17:15:16.247 INFO:tasks.workunit.client.1.smithi115.stdout:4/671: dread - d7/dc/d1a/fa2 zero size 2021-11-03T17:15:16.252 INFO:tasks.workunit.client.1.smithi115.stdout:4/672: mknod d7/dc/d6a/d3c/dc5/dcc/dd9/ce0 1 2021-11-03T17:15:16.254 INFO:tasks.workunit.client.1.smithi115.stdout:5/648: creat d1a/d3d/dc4/fd0 x:0 0 0 2021-11-03T17:15:16.255 INFO:tasks.workunit.client.1.smithi115.stdout:0/739: truncate d9/db/f47 814894 0 2021-11-03T17:15:16.255 INFO:tasks.workunit.client.1.smithi115.stdout:0/740: stat d9/d15/d1a/d1d/dcc/d46/d63/lf4 0 2021-11-03T17:15:16.256 INFO:tasks.workunit.client.1.smithi115.stdout:7/507: link d3/da/l1d d3/da/d8e/l9e 0 2021-11-03T17:15:16.257 INFO:tasks.workunit.client.1.smithi115.stdout:3/676: chown d13/d2c/ld6 25950 1 2021-11-03T17:15:16.257 INFO:tasks.workunit.client.1.smithi115.stdout:0/741: write d9/d15/d78/d80/d82/fb5 [2057761,57080] 0 2021-11-03T17:15:16.258 INFO:tasks.workunit.client.1.smithi115.stdout:3/677: creat d13/d2a/fe5 x:0 0 0 2021-11-03T17:15:16.259 INFO:tasks.workunit.client.1.smithi115.stdout:4/673: truncate d7/dc/d1a/d21/db1/d56/d63/d7b/fb8 1265983 0 2021-11-03T17:15:16.260 INFO:tasks.workunit.client.1.smithi115.stdout:5/649: truncate d1a/d3d/f62 3828732 0 2021-11-03T17:15:16.261 INFO:tasks.workunit.client.1.smithi115.stdout:6/467: dwrite d0/d1/f50 [0,4194304] 0 2021-11-03T17:15:16.265 INFO:tasks.workunit.client.1.smithi115.stdout:3/678: chown d13/d50/l8b 0 1 2021-11-03T17:15:16.266 INFO:tasks.workunit.client.1.smithi115.stdout:0/742: mknod d9/d15/db6/cf7 1 2021-11-03T17:15:16.271 INFO:tasks.workunit.client.1.smithi115.stdout:6/468: chown d0/d1/d2e/d4c/d96/d61/l6f 14 1 2021-11-03T17:15:16.273 INFO:tasks.workunit.client.1.smithi115.stdout:7/508: rename d3/d4c/f72 to d3/da/d78/f9f 0 2021-11-03T17:15:16.276 INFO:tasks.workunit.client.1.smithi115.stdout:5/650: getdents d1a/d3d/d58 0 2021-11-03T17:15:16.278 INFO:tasks.workunit.client.1.smithi115.stdout:7/509: rename d3/da/d18/d73 to d3/da/d18/d73/d33/da0 22 2021-11-03T17:15:16.280 INFO:tasks.workunit.client.1.smithi115.stdout:6/469: dread d0/d1/d2/f4a [0,4194304] 0 2021-11-03T17:15:16.280 INFO:tasks.workunit.client.1.smithi115.stdout:4/674: mkdir d7/dc/db6/de1 0 2021-11-03T17:15:16.284 INFO:tasks.workunit.client.1.smithi115.stdout:7/510: mkdir d3/da/d18/d9c/da1 0 2021-11-03T17:15:16.286 INFO:tasks.workunit.client.1.smithi115.stdout:6/470: creat d0/d1/d10/d13/d59/f9c x:0 0 0 2021-11-03T17:15:16.287 INFO:tasks.workunit.client.1.smithi115.stdout:3/679: write ff [607563,20044] 0 2021-11-03T17:15:16.289 INFO:tasks.workunit.client.1.smithi115.stdout:6/471: chown d0/d1/d2e/d4c/d96/l6c 6 1 2021-11-03T17:15:16.290 INFO:tasks.workunit.client.1.smithi115.stdout:4/675: symlink d7/dc/d6a/dab/le2 0 2021-11-03T17:15:16.290 INFO:tasks.workunit.client.1.smithi115.stdout:4/676: dread - d7/dc/d6a/d3c/dbf/fdb zero size 2021-11-03T17:15:16.291 INFO:tasks.workunit.client.1.smithi115.stdout:7/511: mknod d3/d4c/d7b/ca2 1 2021-11-03T17:15:16.292 INFO:tasks.workunit.client.1.smithi115.stdout:4/677: creat d7/dc/d1a/fe3 x:0 0 0 2021-11-03T17:15:16.293 INFO:tasks.workunit.client.1.smithi115.stdout:3/680: rename d13/d2a/d8a/d99/fc6 to d13/d2a/fe6 0 2021-11-03T17:15:16.294 INFO:tasks.workunit.client.1.smithi115.stdout:7/512: mkdir d3/da/d18/d73/d35/da3 0 2021-11-03T17:15:16.295 INFO:tasks.workunit.client.1.smithi115.stdout:6/472: mkdir d0/d1/d10/d13/d1c/d9d 0 2021-11-03T17:15:16.297 INFO:tasks.workunit.client.1.smithi115.stdout:4/678: creat d7/dc/d1a/d21/db1/fe4 x:0 0 0 2021-11-03T17:15:16.298 INFO:tasks.workunit.client.1.smithi115.stdout:3/681: creat d13/d51/d2e/db4/fe7 x:0 0 0 2021-11-03T17:15:16.303 INFO:tasks.workunit.client.1.smithi115.stdout:7/513: write d3/f47 [3054555,51041] 0 2021-11-03T17:15:16.304 INFO:tasks.workunit.client.1.smithi115.stdout:4/679: dread d7/dc/d6a/d3c/f3e [0,4194304] 0 2021-11-03T17:15:16.306 INFO:tasks.workunit.client.1.smithi115.stdout:0/743: dwrite d9/db/d83/fd1 [0,4194304] 0 2021-11-03T17:15:16.317 INFO:tasks.workunit.client.1.smithi115.stdout:5/651: dwrite d1a/d28/d4c/d75/d68/da5/fa9 [0,4194304] 0 2021-11-03T17:15:16.317 INFO:tasks.workunit.client.1.smithi115.stdout:5/652: stat d1a/d3d/d58/dab 0 2021-11-03T17:15:16.320 INFO:tasks.workunit.client.1.smithi115.stdout:3/682: dwrite d13/f1f [0,4194304] 0 2021-11-03T17:15:16.342 INFO:tasks.workunit.client.1.smithi115.stdout:7/514: getdents d3/da/d18/d73/d33 0 2021-11-03T17:15:16.342 INFO:tasks.workunit.client.1.smithi115.stdout:7/515: readlink d3/da/d18/d73/l8b 0 2021-11-03T17:15:16.343 INFO:tasks.workunit.client.1.smithi115.stdout:4/680: creat d7/fe5 x:0 0 0 2021-11-03T17:15:16.347 INFO:tasks.workunit.client.1.smithi115.stdout:5/653: creat d1a/d28/fd1 x:0 0 0 2021-11-03T17:15:16.349 INFO:tasks.workunit.client.1.smithi115.stdout:3/683: creat d13/d2a/d53/d70/d92/fe8 x:0 0 0 2021-11-03T17:15:16.350 INFO:tasks.workunit.client.1.smithi115.stdout:3/684: chown d13 2380736 1 2021-11-03T17:15:16.352 INFO:tasks.workunit.client.1.smithi115.stdout:4/681: chown d7/dc/f12 42301994 1 2021-11-03T17:15:16.355 INFO:tasks.workunit.client.1.smithi115.stdout:5/654: rmdir d1a/d28/d4c 39 2021-11-03T17:15:16.356 INFO:tasks.workunit.client.1.smithi115.stdout:7/516: rename d3/da/d18/d73/d52/l7a to d3/d4c/la4 0 2021-11-03T17:15:16.358 INFO:tasks.workunit.client.1.smithi115.stdout:4/682: getdents d7/dc/d6a 0 2021-11-03T17:15:16.358 INFO:tasks.workunit.client.1.smithi115.stdout:0/744: write d9/f25 [2196598,119911] 0 2021-11-03T17:15:16.359 INFO:tasks.workunit.client.1.smithi115.stdout:7/517: chown d3/da/d18/d73/d52/f56 1584 1 2021-11-03T17:15:16.360 INFO:tasks.workunit.client.1.smithi115.stdout:3/685: dread ff [0,4194304] 0 2021-11-03T17:15:16.361 INFO:tasks.workunit.client.1.smithi115.stdout:5/655: truncate d1a/d28/d7b/f74 522489 0 2021-11-03T17:15:16.362 INFO:tasks.workunit.client.1.smithi115.stdout:7/518: creat d3/d4c/fa5 x:0 0 0 2021-11-03T17:15:16.362 INFO:tasks.workunit.client.1.smithi115.stdout:3/686: mknod d13/d2a/d8a/ce9 1 2021-11-03T17:15:16.364 INFO:tasks.workunit.client.1.smithi115.stdout:0/745: link d9/d37/fdc d9/d15/d1a/d1d/dcc/d85/ff8 0 2021-11-03T17:15:16.364 INFO:tasks.workunit.client.1.smithi115.stdout:5/656: creat d1a/d28/d4c/d75/d68/fd2 x:0 0 0 2021-11-03T17:15:16.369 INFO:tasks.workunit.client.1.smithi115.stdout:0/746: unlink d9/db/f4a 0 2021-11-03T17:15:16.370 INFO:tasks.workunit.client.1.smithi115.stdout:5/657: readlink d1a/d28/d4c/d75/d83/da1/lc5 0 2021-11-03T17:15:16.372 INFO:tasks.workunit.client.1.smithi115.stdout:3/687: truncate d13/f19 885772 0 2021-11-03T17:15:16.374 INFO:tasks.workunit.client.1.smithi115.stdout:4/683: mkdir d7/dc/d1a/d21/db1/d56/d42/de6 0 2021-11-03T17:15:16.375 INFO:tasks.workunit.client.1.smithi115.stdout:5/658: rmdir d1a/d23 39 2021-11-03T17:15:16.376 INFO:tasks.workunit.client.1.smithi115.stdout:3/688: truncate d13/f48 3201106 0 2021-11-03T17:15:16.378 INFO:tasks.workunit.client.1.smithi115.stdout:0/747: symlink d9/d15/d1a/d1d/lf9 0 2021-11-03T17:15:16.378 INFO:tasks.workunit.client.1.smithi115.stdout:3/689: readlink d13/d2a/d53/d70/la5 0 2021-11-03T17:15:16.382 INFO:tasks.workunit.client.1.smithi115.stdout:3/690: truncate d13/d51/f29 55561 0 2021-11-03T17:15:16.382 INFO:tasks.workunit.client.1.smithi115.stdout:3/691: getdents d13/da4/db2 0 2021-11-03T17:15:16.391 INFO:tasks.workunit.client.1.smithi115.stdout:5/659: mkdir d1a/d3d/d58/dab/dd3 0 2021-11-03T17:15:16.396 INFO:tasks.workunit.client.1.smithi115.stdout:3/692: dread d13/d2a/d8a/fa7 [0,4194304] 0 2021-11-03T17:15:16.397 INFO:tasks.workunit.client.1.smithi115.stdout:5/660: mkdir d1a/d28/d7b/d48/dd4 0 2021-11-03T17:15:16.398 INFO:tasks.workunit.client.1.smithi115.stdout:3/693: creat d13/d51/d2e/db4/fea x:0 0 0 2021-11-03T17:15:16.398 INFO:tasks.workunit.client.1.smithi115.stdout:3/694: stat fc 0 2021-11-03T17:15:16.401 INFO:tasks.workunit.client.1.smithi115.stdout:7/519: dwrite d3/da/f64 [4194304,4194304] 0 2021-11-03T17:15:16.401 INFO:tasks.workunit.client.1.smithi115.stdout:7/520: readlink d3/da/d18/d73/l8b 0 2021-11-03T17:15:16.402 INFO:tasks.workunit.client.1.smithi115.stdout:7/521: rmdir d3 39 2021-11-03T17:15:16.402 INFO:tasks.workunit.client.1.smithi115.stdout:7/522: chown d3/da/l61 15 1 2021-11-03T17:15:16.416 INFO:tasks.workunit.client.1.smithi115.stdout:5/661: dread d1a/d28/d4a/d65/db5/f67 [0,4194304] 0 2021-11-03T17:15:16.416 INFO:tasks.workunit.client.1.smithi115.stdout:4/684: dwrite d7/dc/f11 [0,4194304] 0 2021-11-03T17:15:16.417 INFO:tasks.workunit.client.1.smithi115.stdout:5/662: creat d1a/d28/d4c/d75/d68/da5/fd5 x:0 0 0 2021-11-03T17:15:16.418 INFO:tasks.workunit.client.1.smithi115.stdout:4/685: chown d7/dc/d1a/d21/db1/f69 1008493106 1 2021-11-03T17:15:16.420 INFO:tasks.workunit.client.1.smithi115.stdout:0/748: dwrite d9/d15/d1a/d1d/dcc/d85/ff8 [0,4194304] 0 2021-11-03T17:15:16.422 INFO:tasks.workunit.client.1.smithi115.stdout:3/695: mkdir d13/d35/d54/dad/deb 0 2021-11-03T17:15:16.423 INFO:tasks.workunit.client.1.smithi115.stdout:7/523: write d3/da/d18/f40 [1206122,52707] 0 2021-11-03T17:15:16.423 INFO:tasks.workunit.client.1.smithi115.stdout:7/524: getdents d3/da/d18/d9c/da1 0 2021-11-03T17:15:16.429 INFO:tasks.workunit.client.1.smithi115.stdout:5/663: link d1a/d28/d4c/d75/d83/la4 d1a/d28/d7b/d48/ld6 0 2021-11-03T17:15:16.430 INFO:tasks.workunit.client.1.smithi115.stdout:0/749: link d9/d15/d1a/de5/le8 d9/d15/dce/lfa 0 2021-11-03T17:15:16.433 INFO:tasks.workunit.client.1.smithi115.stdout:3/696: unlink d13/da4/dcb/le0 0 2021-11-03T17:15:16.434 INFO:tasks.workunit.client.1.smithi115.stdout:5/664: stat d1a/d28/l4d 0 2021-11-03T17:15:16.435 INFO:tasks.workunit.client.1.smithi115.stdout:3/697: write d13/d35/f36 [741053,64621] 0 2021-11-03T17:15:16.435 INFO:tasks.workunit.client.1.smithi115.stdout:7/525: mkdir d3/d4c/da6 0 2021-11-03T17:15:16.436 INFO:tasks.workunit.client.1.smithi115.stdout:7/526: fsync d3/da/d18/d73/d33/f74 0 2021-11-03T17:15:16.436 INFO:tasks.workunit.client.1.smithi115.stdout:4/686: link d7/dc/d1a/d21/db1/d56/d63/f6e d7/dc/d1a/d21/db1/d56/db3/fe7 0 2021-11-03T17:15:16.437 INFO:tasks.workunit.client.1.smithi115.stdout:5/665: readlink d1a/d28/d7b/lbd 0 2021-11-03T17:15:16.440 INFO:tasks.workunit.client.1.smithi115.stdout:7/527: mknod d3/da/d78/ca7 1 2021-11-03T17:15:16.440 INFO:tasks.workunit.client.1.smithi115.stdout:0/750: symlink d9/d15/d1a/lfb 0 2021-11-03T17:15:16.470 INFO:tasks.workunit.client.1.smithi115.stdout:5/666: rename d1a/d28/d34/f33 to d1a/d28/d7b/d48/dd4/fd7 0 2021-11-03T17:15:16.472 INFO:tasks.workunit.client.1.smithi115.stdout:4/687: dwrite d7/dc/d1a/d21/f22 [0,4194304] 0 2021-11-03T17:15:16.480 INFO:tasks.workunit.client.1.smithi115.stdout:3/698: dwrite d13/f56 [0,4194304] 0 2021-11-03T17:15:16.485 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.489+0000 7f2bae931700 1 -- 172.21.15.27:0/563124819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2ba810d210 con 0x7f2ba80f73f0 2021-11-03T17:15:16.486 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.490+0000 7f2b9e7fc700 1 -- 172.21.15.27:0/563124819 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 26 v26) v1 ==== 76+0+1645 (secure 0 0 0) 0x7f2b98023020 con 0x7f2ba80f73f0 2021-11-03T17:15:16.486 INFO:teuthology.orchestra.run.smithi027.stdout:e26 2021-11-03T17:15:16.486 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:15:16.487 INFO:teuthology.orchestra.run.smithi027.stdout:compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:15:16.487 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:15:16.487 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:15:16.488 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:15:16.488 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:15:16.488 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 26 2021-11-03T17:15:16.488 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 2021-11-03T17:15:16.489 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:15:16.489 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:15:10.323757+0000 2021-11-03T17:15:16.489 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:15:16.489 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:15:16.490 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:15:16.490 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:15:16.490 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:15:16.490 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:15:16.491 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:15:16.491 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 51 2021-11-03T17:15:16.491 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:15:16.491 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 2 2021-11-03T17:15:16.491 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:15:16.492 INFO:teuthology.orchestra.run.smithi027.stdout:up {0=24289,1=14436} 2021-11-03T17:15:16.492 INFO:teuthology.orchestra.run.smithi027.stdout:failed 2021-11-03T17:15:16.492 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:15:16.492 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:15:16.493 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:15:16.493 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:15:16.493 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:15:16.493 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:15:16.493 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:15:16.494 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{0:24289} state up:active seq 51 join_fscid=1 addr [v2:172.21.15.115:6826/3367380088,v1:172.21.15.115:6827/3367380088]] 2021-11-03T17:15:16.494 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 3 export targets 0 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060]] 2021-11-03T17:15:16.494 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:15:16.494 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:15:16.495 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:15:16.495 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:15:16.495 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635]] 2021-11-03T17:15:16.495 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060]] 2021-11-03T17:15:16.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.492+0000 7f2bae931700 1 -- 172.21.15.27:0/563124819 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7f2b940640f0 msgr2=0x7f2b940665b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:15:16.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.492+0000 7f2bae931700 1 --2- 172.21.15.27:0/563124819 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7f2b940640f0 0x7f2b940665b0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f2ba4009fd0 tx=0x7f2ba400b040).stop 2021-11-03T17:15:16.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.492+0000 7f2bae931700 1 -- 172.21.15.27:0/563124819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ba80f73f0 msgr2=0x7f2ba8107220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:15:16.498 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.492+0000 7f2bae931700 1 --2- 172.21.15.27:0/563124819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ba80f73f0 0x7f2ba8107220 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f2b98000c00 tx=0x7f2b9800b040).stop 2021-11-03T17:15:16.498 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.492+0000 7f2bae931700 1 -- 172.21.15.27:0/563124819 shutdown_connections 2021-11-03T17:15:16.498 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.492+0000 7f2bae931700 1 --2- 172.21.15.27:0/563124819 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7f2b940640f0 0x7f2b940665b0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:16.498 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.492+0000 7f2bae931700 1 --2- 172.21.15.27:0/563124819 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2ba80f6a50 0x7f2ba8106d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:16.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.492+0000 7f2bae931700 1 --2- 172.21.15.27:0/563124819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2ba80f73f0 0x7f2ba8107220 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:16.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.492+0000 7f2bae931700 1 -- 172.21.15.27:0/563124819 >> 172.21.15.27:0/563124819 conn(0x7f2ba80f0320 msgr2=0x7f2ba80f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:15:16.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.492+0000 7f2bae931700 1 -- 172.21.15.27:0/563124819 shutdown_connections 2021-11-03T17:15:16.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:16.492+0000 7f2bae931700 1 -- 172.21.15.27:0/563124819 wait complete. 2021-11-03T17:15:16.500 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 26 2021-11-03T17:15:16.500 INFO:tasks.workunit.client.1.smithi115.stdout:5/667: dread d1a/d28/d7b/d48/d7f/f86 [0,4194304] 0 2021-11-03T17:15:16.501 INFO:tasks.workunit.client.1.smithi115.stdout:5/668: rename d1a/d28/d4a/f56 to d1a/d28/d7b/fd8 0 2021-11-03T17:15:16.501 INFO:tasks.workunit.client.1.smithi115.stdout:5/669: creat d1a/d28/d4c/d75/d68/da5/fd9 x:0 0 0 2021-11-03T17:15:16.501 INFO:tasks.workunit.client.1.smithi115.stdout:5/670: creat d1a/d3d/d58/dab/dd3/fda x:0 0 0 2021-11-03T17:15:16.501 INFO:tasks.workunit.client.1.smithi115.stdout:7/528: dwrite d3/da/d18/d73/d35/f7c [0,4194304] 0 2021-11-03T17:15:16.502 INFO:tasks.workunit.client.1.smithi115.stdout:3/699: creat d13/d2a/d53/d72/fec x:0 0 0 2021-11-03T17:15:16.502 INFO:tasks.workunit.client.1.smithi115.stdout:0/751: dwrite f8 [0,4194304] 0 2021-11-03T17:15:16.502 INFO:tasks.workunit.client.1.smithi115.stdout:3/700: dread d13/d51/d40/f9d [0,4194304] 0 2021-11-03T17:15:16.502 INFO:tasks.workunit.client.1.smithi115.stdout:0/752: rename d9/d15/d78/ddd to d9/d15/d1a/d1d/dcc/d46/d8c/dfc 0 2021-11-03T17:15:16.506 INFO:tasks.workunit.client.1.smithi115.stdout:7/529: dread d3/da/d18/f40 [0,4194304] 0 2021-11-03T17:15:16.506 INFO:tasks.workunit.client.1.smithi115.stdout:7/530: chown d3/l8a 26091 1 2021-11-03T17:15:16.507 INFO:tasks.workunit.client.1.smithi115.stdout:4/688: dwrite d7/dc/d6a/d72/f76 [0,4194304] 0 2021-11-03T17:15:16.514 INFO:tasks.workunit.client.1.smithi115.stdout:5/671: unlink d1a/d28/d7b/d4f/d81/fa6 0 2021-11-03T17:15:16.516 INFO:tasks.workunit.client.1.smithi115.stdout:0/753: rename d9/d15/d1a/d1d/dcc/d6e/d71/fa9 to d9/d37/ffd 0 2021-11-03T17:15:16.519 INFO:tasks.workunit.client.1.smithi115.stdout:0/754: symlink d9/d15/d1a/de5/lfe 0 2021-11-03T17:15:16.521 INFO:tasks.workunit.client.1.smithi115.stdout:0/755: readlink d9/db/ld 0 2021-11-03T17:15:16.522 INFO:tasks.workunit.client.1.smithi115.stdout:0/756: write d9/d15/d78/fd0 [2635436,61031] 0 2021-11-03T17:15:16.523 INFO:tasks.workunit.client.1.smithi115.stdout:7/531: rename d3/da/f69 to d3/da/d18/d73/d35/da3/fa8 0 2021-11-03T17:15:16.525 INFO:tasks.workunit.client.1.smithi115.stdout:7/532: creat d3/da/d18/d73/d52/fa9 x:0 0 0 2021-11-03T17:15:16.525 INFO:tasks.workunit.client.1.smithi115.stdout:0/757: symlink d9/d15/d1a/d1d/dcc/d46/d63/dae/lff 0 2021-11-03T17:15:16.526 INFO:tasks.workunit.client.1.smithi115.stdout:7/533: rmdir d3/da/d8e 39 2021-11-03T17:15:16.527 INFO:tasks.workunit.client.1.smithi115.stdout:0/758: creat d9/d15/d1a/d1d/dcc/d46/d63/dc5/f100 x:0 0 0 2021-11-03T17:15:16.528 INFO:tasks.workunit.client.1.smithi115.stdout:7/534: getdents d3/da/d18/d73/d35/d53 0 2021-11-03T17:15:16.533 INFO:tasks.workunit.client.1.smithi115.stdout:7/535: read d3/da/f1b [3540804,47979] 0 2021-11-03T17:15:16.533 INFO:tasks.workunit.client.1.smithi115.stdout:3/701: dwrite d13/d67/f87 [0,4194304] 0 2021-11-03T17:15:16.538 INFO:tasks.workunit.client.1.smithi115.stdout:7/536: symlink d3/d4c/d7b/laa 0 2021-11-03T17:15:16.538 INFO:tasks.workunit.client.1.smithi115.stdout:5/672: write d1a/d28/d7b/d48/d7f/f94 [431105,101348] 0 2021-11-03T17:15:16.540 INFO:tasks.workunit.client.1.smithi115.stdout:4/689: dwrite d7/dc/f12 [0,4194304] 0 2021-11-03T17:15:16.541 INFO:tasks.workunit.client.1.smithi115.stdout:3/702: link d13/f7c d13/d35/d54/dad/fed 0 2021-11-03T17:15:16.541 INFO:tasks.workunit.client.1.smithi115.stdout:3/703: dread - d13/d2a/d8a/f94 zero size 2021-11-03T17:15:16.544 INFO:tasks.workunit.client.1.smithi115.stdout:0/759: rmdir d9/d15/d1a/d1d/dcc/d46/d63/dc5 39 2021-11-03T17:15:16.545 INFO:tasks.workunit.client.1.smithi115.stdout:0/760: fsync d9/db/f8e 0 2021-11-03T17:15:16.547 INFO:tasks.workunit.client.1.smithi115.stdout:3/704: symlink d13/lee 0 2021-11-03T17:15:16.549 INFO:tasks.workunit.client.1.smithi115.stdout:0/761: rmdir d9/d15/d1a/d1d/dcc 39 2021-11-03T17:15:16.550 INFO:tasks.workunit.client.1.smithi115.stdout:0/762: mknod d9/db/d89/c101 1 2021-11-03T17:15:16.551 INFO:tasks.workunit.client.1.smithi115.stdout:0/763: read d9/db/d83/fd1 [3217285,74210] 0 2021-11-03T17:15:16.555 INFO:tasks.workunit.client.1.smithi115.stdout:5/673: rename d1a/d28/d4c/d63/l8a to d1a/d3d/dc0/ldb 0 2021-11-03T17:15:16.555 INFO:tasks.workunit.client.1.smithi115.stdout:5/674: dread - d1a/d28/fd1 zero size 2021-11-03T17:15:16.556 INFO:tasks.workunit.client.1.smithi115.stdout:0/764: unlink d9/db/f58 0 2021-11-03T17:15:16.557 INFO:tasks.workunit.client.1.smithi115.stdout:5/675: creat d1a/d28/d7b/d4f/fdc x:0 0 0 2021-11-03T17:15:16.557 INFO:tasks.workunit.client.1.smithi115.stdout:8/541: sync 2021-11-03T17:15:16.558 INFO:tasks.workunit.client.1.smithi115.stdout:0/765: creat d9/d15/d1a/d1d/dcc/d46/d8c/dfc/f102 x:0 0 0 2021-11-03T17:15:16.559 INFO:tasks.workunit.client.1.smithi115.stdout:3/705: symlink d13/d67/lef 0 2021-11-03T17:15:16.566 INFO:tasks.workunit.client.1.smithi115.stdout:8/542: dread d4/f6c [0,4194304] 0 2021-11-03T17:15:16.566 INFO:tasks.workunit.client.1.smithi115.stdout:0/766: mkdir d9/d15/d1a/d1d/dcc/d46/d63/d70/dda/d103 0 2021-11-03T17:15:16.568 INFO:tasks.workunit.client.1.smithi115.stdout:0/767: write d9/db/d83/fd1 [884824,82023] 0 2021-11-03T17:15:16.568 INFO:tasks.workunit.client.1.smithi115.stdout:5/676: write d1a/d28/d7b/d48/fca [502263,16752] 0 2021-11-03T17:15:16.569 INFO:tasks.workunit.client.1.smithi115.stdout:3/706: readlink d13/d51/d40/l6e 0 2021-11-03T17:15:16.570 INFO:tasks.workunit.client.1.smithi115.stdout:8/543: chown d4/d6/d18/d30/d33/f75 2 1 2021-11-03T17:15:16.571 INFO:tasks.workunit.client.1.smithi115.stdout:0/768: rmdir d9/d15/d78/d80 39 2021-11-03T17:15:16.577 INFO:tasks.workunit.client.1.smithi115.stdout:3/707: link d13/d50/f42 d13/da4/ff0 0 2021-11-03T17:15:16.578 INFO:tasks.workunit.client.1.smithi115.stdout:3/708: chown d13/f19 5766650 1 2021-11-03T17:15:16.578 INFO:tasks.workunit.client.1.smithi115.stdout:3/709: readlink d13/l7f 0 2021-11-03T17:15:16.582 INFO:tasks.workunit.client.1.smithi115.stdout:0/769: dread d9/f35 [0,4194304] 0 2021-11-03T17:15:16.583 INFO:tasks.workunit.client.1.smithi115.stdout:5/677: creat d1a/d28/d4a/d65/fdd x:0 0 0 2021-11-03T17:15:16.583 INFO:tasks.workunit.client.1.smithi115.stdout:3/710: dread d13/d67/f6c [0,4194304] 0 2021-11-03T17:15:16.584 INFO:tasks.workunit.client.1.smithi115.stdout:0/770: creat d9/d15/d1a/d1d/dcc/d46/f104 x:0 0 0 2021-11-03T17:15:16.584 INFO:tasks.workunit.client.1.smithi115.stdout:8/544: dread d4/d6/f2b [4194304,4194304] 0 2021-11-03T17:15:16.589 INFO:tasks.workunit.client.1.smithi115.stdout:5/678: rename d1a/d3d/d58/l7e to d1a/d28/d4c/d75/d68/da5/lde 0 2021-11-03T17:15:16.589 INFO:tasks.workunit.client.1.smithi115.stdout:8/545: mknod d4/d6/d18/d30/d33/d93/cb8 1 2021-11-03T17:15:16.590 INFO:tasks.workunit.client.1.smithi115.stdout:0/771: dread d9/d15/d1a/f3d [0,4194304] 0 2021-11-03T17:15:16.595 INFO:tasks.workunit.client.1.smithi115.stdout:8/546: creat d4/fb9 x:0 0 0 2021-11-03T17:15:16.599 INFO:tasks.workunit.client.1.smithi115.stdout:8/547: fsync d4/d6/d18/d30/d33/f59 0 2021-11-03T17:15:16.600 INFO:tasks.workunit.client.1.smithi115.stdout:8/548: creat d4/d6/d18/d9d/fba x:0 0 0 2021-11-03T17:15:16.624 INFO:tasks.workunit.client.1.smithi115.stdout:0/772: dwrite d9/d15/d78/d80/f9a [0,4194304] 0 2021-11-03T17:15:16.624 INFO:tasks.workunit.client.1.smithi115.stdout:0/773: read - d9/d15/d1a/d1d/dcc/d46/d63/dae/ff2 zero size 2021-11-03T17:15:16.625 INFO:tasks.workunit.client.1.smithi115.stdout:0/774: chown d9/d15 1443 1 2021-11-03T17:15:16.625 INFO:tasks.workunit.client.1.smithi115.stdout:0/775: write d9/d15/d1a/f92 [4714909,116003] 0 2021-11-03T17:15:16.625 INFO:tasks.workunit.client.1.smithi115.stdout:5/679: dwrite d1a/d28/d4c/f6e [0,4194304] 0 2021-11-03T17:15:16.625 INFO:tasks.workunit.client.1.smithi115.stdout:5/680: truncate d1a/d28/d4c/d75/d68/fd2 838040 0 2021-11-03T17:15:16.627 INFO:tasks.workunit.client.1.smithi115.stdout:0/776: mknod d9/db/c105 1 2021-11-03T17:15:16.627 INFO:tasks.workunit.client.1.smithi115.stdout:3/711: dwrite d13/d2a/d8a/f94 [0,4194304] 0 2021-11-03T17:15:16.627 INFO:tasks.workunit.client.1.smithi115.stdout:0/777: chown d9/d15/d1a/d1d/dcc/d46/d8c/f74 77 1 2021-11-03T17:15:16.628 INFO:tasks.workunit.client.1.smithi115.stdout:8/549: link d4/d6/d18/d44/d6e/f9f d4/d6/d18/d30/d79/d7f/fbb 0 2021-11-03T17:15:16.629 INFO:tasks.workunit.client.1.smithi115.stdout:5/681: mkdir d1a/d3d/dc0/ddf 0 2021-11-03T17:15:16.632 INFO:tasks.workunit.client.1.smithi115.stdout:0/778: symlink d9/d15/d1a/d1d/dcc/d46/d63/d70/dda/d103/l106 0 2021-11-03T17:15:16.639 INFO:tasks.workunit.client.1.smithi115.stdout:3/712: dread d13/d3b/f76 [0,4194304] 0 2021-11-03T17:15:16.639 INFO:tasks.workunit.client.1.smithi115.stdout:3/713: fdatasync d13/d2a/f4e 0 2021-11-03T17:15:16.640 INFO:tasks.workunit.client.1.smithi115.stdout:3/714: chown d13/d2c 830 1 2021-11-03T17:15:16.643 INFO:tasks.workunit.client.1.smithi115.stdout:0/779: rename d9/d15/d1a/d1d/dcc/d46/d63/d70/l86 to d9/d15/d1a/d1d/dcc/d46/d63/d70/dda/l107 0 2021-11-03T17:15:16.645 INFO:tasks.workunit.client.1.smithi115.stdout:8/550: link d4/d6/d18/d44/f62 d4/d6/d3f/da4/fbc 0 2021-11-03T17:15:16.646 INFO:tasks.workunit.client.1.smithi115.stdout:3/715: mkdir d13/d2a/d53/d72/dd2/df1 0 2021-11-03T17:15:16.647 INFO:tasks.workunit.client.1.smithi115.stdout:0/780: mkdir d9/d15/d78/d108 0 2021-11-03T17:15:16.648 INFO:tasks.workunit.client.1.smithi115.stdout:8/551: truncate d4/d6/d3f/d43/d88/f8b 161935 0 2021-11-03T17:15:16.650 INFO:tasks.workunit.client.1.smithi115.stdout:5/682: dwrite d1a/d3d/f62 [0,4194304] 0 2021-11-03T17:15:16.650 INFO:tasks.workunit.client.1.smithi115.stdout:8/552: creat d4/fbd x:0 0 0 2021-11-03T17:15:16.655 INFO:tasks.workunit.client.1.smithi115.stdout:8/553: creat d4/d6/d18/d30/fbe x:0 0 0 2021-11-03T17:15:16.657 INFO:tasks.workunit.client.1.smithi115.stdout:8/554: creat d4/d6/d18/d44/d6e/fbf x:0 0 0 2021-11-03T17:15:16.657 INFO:tasks.workunit.client.1.smithi115.stdout:8/555: chown d4/d6/d18/d44/d47 3495 1 2021-11-03T17:15:16.658 INFO:tasks.workunit.client.1.smithi115.stdout:8/556: truncate d4/d6/fa9 1404749 0 2021-11-03T17:15:16.661 INFO:tasks.workunit.client.1.smithi115.stdout:8/557: truncate d4/d6/d18/f77 895450 0 2021-11-03T17:15:16.665 INFO:tasks.workunit.client.1.smithi115.stdout:3/716: symlink d13/d2a/d53/d70/d92/db7/lf2 0 2021-11-03T17:15:16.666 INFO:tasks.workunit.client.1.smithi115.stdout:0/781: dwrite d9/d15/d78/d80/fe3 [0,4194304] 0 2021-11-03T17:15:16.667 INFO:tasks.workunit.client.1.smithi115.stdout:3/717: rename d13/d35/d54/f6f to d13/d2a/d53/d70/d92/ff3 0 2021-11-03T17:15:16.668 INFO:tasks.workunit.client.1.smithi115.stdout:3/718: write d13/da4/fe2 [267529,109567] 0 2021-11-03T17:15:16.672 INFO:tasks.workunit.client.1.smithi115.stdout:0/782: mknod d9/c109 1 2021-11-03T17:15:16.672 INFO:tasks.workunit.client.1.smithi115.stdout:0/783: chown d9/d15/db6 119 1 2021-11-03T17:15:16.673 INFO:tasks.workunit.client.1.smithi115.stdout:0/784: mknod d9/d15/d1a/c10a 1 2021-11-03T17:15:16.674 INFO:tasks.workunit.client.1.smithi115.stdout:0/785: chown d9/d15/d1a 507478 1 2021-11-03T17:15:16.675 INFO:tasks.workunit.client.1.smithi115.stdout:3/719: rename d13/d35/d54/dad/deb to d13/da4/df4 0 2021-11-03T17:15:16.677 INFO:tasks.workunit.client.1.smithi115.stdout:0/786: rmdir d9/db/d89/d81 39 2021-11-03T17:15:16.678 INFO:tasks.workunit.client.1.smithi115.stdout:0/787: dread f8 [4194304,4194304] 0 2021-11-03T17:15:16.680 INFO:tasks.workunit.client.1.smithi115.stdout:0/788: chown d9/d15/d1a/d1d/dcc/d46/d63/dc5/fe7 378 1 2021-11-03T17:15:16.680 INFO:tasks.workunit.client.1.smithi115.stdout:0/789: fdatasync d9/f98 0 2021-11-03T17:15:16.682 INFO:tasks.workunit.client.1.smithi115.stdout:3/720: getdents d13/d2a/d53/d70/d96 0 2021-11-03T17:15:16.682 INFO:tasks.workunit.client.1.smithi115.stdout:5/683: dwrite d1a/d28/d7b/f9f [0,4194304] 0 2021-11-03T17:15:16.683 INFO:tasks.workunit.client.1.smithi115.stdout:0/790: write d9/d15/d1a/d1d/f4d [536970,36684] 0 2021-11-03T17:15:16.687 INFO:tasks.workunit.client.1.smithi115.stdout:3/721: truncate d13/d2a/d53/d72/fbd 2638992 0 2021-11-03T17:15:16.688 INFO:tasks.workunit.client.1.smithi115.stdout:3/722: mknod d13/d2a/d60/dd3/cf5 1 2021-11-03T17:15:16.688 INFO:tasks.workunit.client.1.smithi115.stdout:3/723: fdatasync d13/d35/fdb 0 2021-11-03T17:15:16.690 INFO:tasks.workunit.client.1.smithi115.stdout:8/558: dwrite d4/d6/d18/d44/f99 [0,4194304] 0 2021-11-03T17:15:16.692 INFO:tasks.workunit.client.1.smithi115.stdout:3/724: mkdir d13/d35/d54/df6 0 2021-11-03T17:15:16.693 INFO:tasks.workunit.client.1.smithi115.stdout:0/791: write d9/d15/d1a/d1d/fb7 [612921,51627] 0 2021-11-03T17:15:16.699 INFO:tasks.workunit.client.1.smithi115.stdout:0/792: rmdir d9/d15/d1a/da7 39 2021-11-03T17:15:16.700 INFO:tasks.workunit.client.1.smithi115.stdout:8/559: mknod d4/d6/d18/d30/d33/cc0 1 2021-11-03T17:15:16.700 INFO:tasks.workunit.client.1.smithi115.stdout:8/560: fsync d4/d6/d18/d30/fbe 0 2021-11-03T17:15:16.700 INFO:tasks.workunit.client.1.smithi115.stdout:8/561: mknod d4/d6/d18/d9d/cc1 1 2021-11-03T17:15:16.705 INFO:tasks.workunit.client.1.smithi115.stdout:8/562: mkdir d4/d7d/dc2 0 2021-11-03T17:15:16.705 INFO:tasks.workunit.client.1.smithi115.stdout:8/563: chown d4/d6/d18/d44/fa3 1764633125 1 2021-11-03T17:15:16.713 INFO:tasks.workunit.client.1.smithi115.stdout:0/793: mkdir d9/d15/db6/d10b 0 2021-11-03T17:15:16.714 INFO:tasks.workunit.client.1.smithi115.stdout:3/725: write d13/d35/f3a [2592026,13633] 0 2021-11-03T17:15:16.714 INFO:tasks.workunit.client.1.smithi115.stdout:8/564: symlink d4/d6/d3f/d9a/lc3 0 2021-11-03T17:15:16.714 INFO:tasks.workunit.client.1.smithi115.stdout:5/684: dwrite f11 [0,4194304] 0 2021-11-03T17:15:16.715 INFO:tasks.workunit.client.1.smithi115.stdout:3/726: creat d13/d51/d40/ff7 x:0 0 0 2021-11-03T17:15:16.716 INFO:tasks.workunit.client.1.smithi115.stdout:8/565: creat d4/d6/d3f/d9a/fc4 x:0 0 0 2021-11-03T17:15:16.717 INFO:tasks.workunit.client.1.smithi115.stdout:3/727: readlink d13/d50/l8f 0 2021-11-03T17:15:16.718 INFO:tasks.workunit.client.1.smithi115.stdout:8/566: mknod d4/d6/d18/cc5 1 2021-11-03T17:15:16.719 INFO:tasks.workunit.client.1.smithi115.stdout:5/685: symlink d1a/d28/d4c/d75/d68/le0 0 2021-11-03T17:15:16.720 INFO:tasks.workunit.client.1.smithi115.stdout:5/686: chown d1a/d28/d7b/d48/d7f 2 1 2021-11-03T17:15:16.720 INFO:tasks.workunit.client.1.smithi115.stdout:3/728: rename d13/d2a/d53/d70/d92 to d13/d2a/d53/d70/d92/df8 22 2021-11-03T17:15:16.720 INFO:tasks.workunit.client.1.smithi115.stdout:5/687: write d1a/d23/f61 [931125,33697] 0 2021-11-03T17:15:16.724 INFO:tasks.workunit.client.1.smithi115.stdout:3/729: dread d13/d51/d2e/fa2 [0,4194304] 0 2021-11-03T17:15:16.724 INFO:tasks.workunit.client.1.smithi115.stdout:0/794: dread d9/d15/d1a/d1d/f24 [0,4194304] 0 2021-11-03T17:15:16.726 INFO:tasks.workunit.client.1.smithi115.stdout:3/730: unlink d13/d35/d54/fa0 0 2021-11-03T17:15:16.733 INFO:tasks.workunit.client.1.smithi115.stdout:5/688: write d1a/d28/d7b/d48/dd4/fd7 [1544894,91560] 0 2021-11-03T17:15:16.734 INFO:tasks.workunit.client.1.smithi115.stdout:5/689: write d1a/d3d/f62 [4704116,120501] 0 2021-11-03T17:15:16.734 INFO:tasks.workunit.client.1.smithi115.stdout:5/690: chown d1a/d28/d7b/d48/fca 1328730341 1 2021-11-03T17:15:16.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:16 smithi115 conmon[34670]: cluster 2021-11-03T17:15:14.643867+0000 mgr.smithi027.ufhhvb (mgr. 2021-11-03T17:15:16.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:16 smithi115 conmon[34670]: 14622) 17 : cluster [DBG] pgmap v11: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 16 MiB/s rd, 19 MiB/s wr, 417 op/s 2021-11-03T17:15:16.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:16 smithi115 conmon[34670]: audit 2021-11-03T17:15:15.450112+0000 mgr.smithi027.ufhhvb (mgr.14622) 18 : audit [DBG] from='client.14658 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:15:16.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:16 smithi115 conmon[34670]: audit 2021-11-03T17:15:15.719222+0000 mon.smithi027 (mon.0) 707 : audit [INF] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:15:16.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:16 smithi115 conmon[34670]: audit 2021-11-03T17:15:16.106461+0000 mon.smithi027 (mon.0) 708 : audit [DBG] from='client.? 172.21.15.27:0/3558130565' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:15:16.736 INFO:tasks.workunit.client.1.smithi115.stdout:3/731: mkdir d13/d50/df9 0 2021-11-03T17:15:16.736 INFO:tasks.workunit.client.1.smithi115.stdout:5/691: chown d1a/d23/db4 114 1 2021-11-03T17:15:16.739 INFO:tasks.workunit.client.1.smithi115.stdout:5/692: creat d1a/fe1 x:0 0 0 2021-11-03T17:15:16.740 INFO:tasks.workunit.client.1.smithi115.stdout:3/732: symlink d13/d51/d2e/lfa 0 2021-11-03T17:15:16.741 INFO:tasks.workunit.client.1.smithi115.stdout:5/693: chown d1a/d28/d7b/d4f/l73 31309 1 2021-11-03T17:15:16.744 INFO:tasks.workunit.client.1.smithi115.stdout:5/694: getdents d1a/d28 0 2021-11-03T17:15:16.745 INFO:tasks.workunit.client.1.smithi115.stdout:3/733: truncate d13/d67/fd8 712277 0 2021-11-03T17:15:16.745 INFO:tasks.workunit.client.1.smithi115.stdout:3/734: readlink d13/d2a/d53/d70/la5 0 2021-11-03T17:15:16.748 INFO:tasks.workunit.client.1.smithi115.stdout:5/695: dread d1a/d28/d4a/f8b [0,4194304] 0 2021-11-03T17:15:16.750 INFO:tasks.workunit.client.1.smithi115.stdout:5/696: read d1a/d28/d4a/d65/db5/f67 [2016947,114911] 0 2021-11-03T17:15:16.756 INFO:tasks.workunit.client.1.smithi115.stdout:0/795: dwrite d9/db/d89/f84 [0,4194304] 0 2021-11-03T17:15:16.758 INFO:tasks.workunit.client.1.smithi115.stdout:8/567: dwrite d4/d6/d18/d30/d33/f5c [0,4194304] 0 2021-11-03T17:15:16.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:16 smithi027 conmon[32322]: cluster 2021-11-03T17:15:14.643867+0000 mgr.smithi027.ufhhvb (mgr.14622) 17 : cluster [DBG] pgmap v11: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 16 MiB/s rd, 19 MiB/s wr, 417 op/s 2021-11-03T17:15:16.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:16 smithi027 conmon[32322]: audit 2021-11-03T17:15:15.450112+0000 mgr.smithi027.ufhhvb (mgr.14622) 18 : audit [DBG] from='client.14658 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:15:16.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:16 smithi027 conmon[32322]: audit 2021-11-03T17:15:15.719222+0000 mon.smithi027 (mon.0) 707 : audit [INF] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:15:16.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:16 smithi027 conmon[32322]: audit 2021-11-03T17:15:16.106461+0000 mon.smithi027 (mon.0) 708 : audit [DBG] from='client.? 172.21.15.27:0/3558130565' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:15:17.087 INFO:tasks.workunit.client.1.smithi115.stdout:9/589: sync 2021-11-03T17:15:17.415 INFO:tasks.workunit.client.1.smithi115.stdout:2/612: sync 2021-11-03T17:15:17.415 INFO:tasks.workunit.client.1.smithi115.stdout:1/674: sync 2021-11-03T17:15:17.421 INFO:tasks.workunit.client.1.smithi115.stdout:3/735: creat d13/d2c/ffb x:0 0 0 2021-11-03T17:15:17.711 INFO:tasks.workunit.client.1.smithi115.stdout:0/796: truncate d9/db/f47 851682 0 2021-11-03T17:15:17.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:17 smithi115 conmon[34670]: audit 2021-11-03T17:15:15.744834+0000 mgr.smithi027.ufhhvb (mgr.14622) 19 : audit [DBG] from='client.14662 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:15:17.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:17 smithi115 conmon[34670]: audit 2021-11-03T17:15:16.491683+0000 mon.smithi027 (mon.0) 709 : audit [DBG] from='client.? 172.21.15.27:0/563124819' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:15:17.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:17 smithi115 conmon[34670]: audit 2021-11-03T17:15:17.005877+0000 mon.smithi027 (mon.0) 710 : audit [INF] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi115.tcyodz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2021-11-03T17:15:17.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:17 smithi115 conmon[34670]: audit 2021-11-03T17:15:17.006895+0000 mon.smithi027 (mon.0) 711 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mgr services"}]: dispatch 2021-11-03T17:15:17.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:17 smithi115 conmon[34670]: audit 2021-11-03T17:15:17.007853+0000 mon.smithi027 (mon.0) 712 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:15:17.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:17 smithi027 conmon[32322]: audit 2021-11-03T17:15:15.744834+0000 2021-11-03T17:15:17.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:17 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14622) 19 : audit [DBG] from='client.14662 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:15:17.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:17 smithi027 conmon[32322]: audit 2021-11-03T17:15:16.491683+0000 mon.smithi027 (mon.0) 709 : audit [DBG] from='client.? 172.21.15.27:0/563124819' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:15:17.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:17 smithi027 conmon[32322]: audit 2021-11-03T17 2021-11-03T17:15:17.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:17 smithi027 conmon[32322]: :15:17.005877+0000 mon.smithi027 2021-11-03T17:15:17.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:17 smithi027 conmon[32322]: (mon.0) 710 : audit 2021-11-03T17:15:17.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:17 smithi027 conmon[32322]: [INF] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi115.tcyodz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2021-11-03T17:15:17.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:17 smithi027 conmon[32322]: audit 2021 2021-11-03T17:15:17.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:17 smithi027 conmon[32322]: -11-03T17:15:17 2021-11-03T17:15:17.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:17 smithi027 conmon[32322]: .006895+0000 mon.smithi027 (mon.0 2021-11-03T17:15:17.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:17 smithi027 conmon[32322]: ) 711 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mgr services"}]: dispatch 2021-11-03T17:15:17.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:17 smithi027 conmon[32322]: 2021-11-03T17:15:17.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:17 smithi027 conmon[32322]: audit 2021 2021-11-03T17:15:17.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:17 smithi027 conmon[32322]: -11-03T17 2021-11-03T17:15:17.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:17 smithi027 conmon[32322]: :15: 2021-11-03T17:15:17.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:17 smithi027 conmon[32322]: 17.007853+0000 mon.smithi027 (mon 2021-11-03T17:15:17.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:17 smithi027 conmon[32322]: .0) 712 : audit [DBG] 2021-11-03T17:15:17.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:17 smithi027 conmon[32322]: from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:15:17.853 INFO:tasks.workunit.client.1.smithi115.stdout:9/590: mkdir d1/ddf 0 2021-11-03T17:15:17.861 INFO:tasks.workunit.client.1.smithi115.stdout:0/797: fsync d9/d15/f53 0 2021-11-03T17:15:17.865 INFO:tasks.workunit.client.1.smithi115.stdout:2/613: symlink d4/d6/d56/d6c/d34/d42/d8d/ld2 0 2021-11-03T17:15:17.876 INFO:tasks.workunit.client.1.smithi115.stdout:5/697: link d1a/d3d/f3b d1a/d28/d4c/d75/d68/da5/da0/fe2 0 2021-11-03T17:15:17.877 INFO:tasks.workunit.client.1.smithi115.stdout:2/614: chown d4/d6/fbd 29164957 1 2021-11-03T17:15:17.879 INFO:tasks.workunit.client.1.smithi115.stdout:1/675: truncate d7/dc/dab/f33 977364 0 2021-11-03T17:15:17.883 INFO:tasks.workunit.client.1.smithi115.stdout:8/568: link d4/d6/d18/d30/d79/d7f/fae d4/d6/d3f/da4/da6/fc6 0 2021-11-03T17:15:17.887 INFO:tasks.workunit.client.1.smithi115.stdout:9/591: readlink d1/d1a/d48/d27/d53/d83/d88/dcd/lb1 0 2021-11-03T17:15:17.905 INFO:tasks.workunit.client.1.smithi115.stdout:2/615: symlink d4/d6/d13/d75/ld3 0 2021-11-03T17:15:17.907 INFO:tasks.workunit.client.1.smithi115.stdout:1/676: truncate d7/dc/dd/fc7 31744 0 2021-11-03T17:15:17.918 INFO:tasks.workunit.client.1.smithi115.stdout:8/569: rename d4/d6/d18/d30/d33/f5f to d4/d6/d3f/da4/da6/fc7 0 2021-11-03T17:15:17.919 INFO:tasks.workunit.client.1.smithi115.stdout:8/570: chown d4/d6/d3f/l57 1783536 1 2021-11-03T17:15:17.921 INFO:tasks.workunit.client.1.smithi115.stdout:9/592: unlink d1/d18/f22 0 2021-11-03T17:15:17.923 INFO:tasks.workunit.client.1.smithi115.stdout:0/798: dwrite d9/db/f7e [0,4194304] 0 2021-11-03T17:15:17.940 INFO:tasks.workunit.client.1.smithi115.stdout:8/571: symlink d4/d6/d3f/d43/d88/lc8 0 2021-11-03T17:15:17.940 INFO:tasks.workunit.client.1.smithi115.stdout:8/572: chown d4/d6/d3f/d9a/lc3 7 1 2021-11-03T17:15:17.944 INFO:tasks.workunit.client.1.smithi115.stdout:2/616: dwrite d4/d6/d13/d75/f7a [0,4194304] 0 2021-11-03T17:15:17.945 INFO:tasks.workunit.client.1.smithi115.stdout:0/799: symlink d9/d15/d1a/d1d/dcc/d46/d63/dae/l10c 0 2021-11-03T17:15:17.947 INFO:tasks.workunit.client.1.smithi115.stdout:8/573: creat d4/d6/d3f/da4/da6/fc9 x:0 0 0 2021-11-03T17:15:17.948 INFO:tasks.workunit.client.1.smithi115.stdout:0/800: dread d9/db/f8e [0,4194304] 0 2021-11-03T17:15:17.956 INFO:tasks.workunit.client.1.smithi115.stdout:0/801: dread d9/d15/d1a/f3d [0,4194304] 0 2021-11-03T17:15:17.957 INFO:tasks.workunit.client.1.smithi115.stdout:8/574: fsync d4/d6/d18/d30/f35 0 2021-11-03T17:15:17.957 INFO:tasks.workunit.client.1.smithi115.stdout:0/802: dread d9/d15/d1a/d1d/dcc/d46/d8c/f74 [4194304,4194304] 0 2021-11-03T17:15:17.971 INFO:tasks.workunit.client.1.smithi115.stdout:8/575: rename d4/f26 to d4/d6/d18/d44/fca 0 2021-11-03T17:15:17.972 INFO:tasks.workunit.client.1.smithi115.stdout:9/593: dwrite d1/d18/d2b/d42/d6e/f5c [0,4194304] 0 2021-11-03T17:15:17.980 INFO:tasks.workunit.client.1.smithi115.stdout:0/803: dwrite d9/d15/d1a/d1d/dcc/d6e/f72 [0,4194304] 0 2021-11-03T17:15:17.993 INFO:tasks.workunit.client.1.smithi115.stdout:0/804: truncate d9/d15/d78/d80/d82/f91 788047 0 2021-11-03T17:15:18.058 INFO:tasks.workunit.client.1.smithi115.stdout:0/805: mkdir d9/d37/d10d 0 2021-11-03T17:15:18.058 INFO:tasks.workunit.client.1.smithi115.stdout:0/806: chown d9/db 101491 1 2021-11-03T17:15:18.065 INFO:tasks.workunit.client.1.smithi115.stdout:8/576: write d4/d6/d3f/d43/d88/f8b [653990,93081] 0 2021-11-03T17:15:18.069 INFO:tasks.workunit.client.1.smithi115.stdout:0/807: getdents d9/d15/d78/d80/dc4 0 2021-11-03T17:15:18.070 INFO:tasks.workunit.client.1.smithi115.stdout:8/577: stat d4/d6/d18/d30/d79/d7f/fae 0 2021-11-03T17:15:18.076 INFO:tasks.workunit.client.1.smithi115.stdout:0/808: link d9/d15/d1a/d1d/dcc/d46/d63/dc5/lc1 d9/d37/dd3/l10e 0 2021-11-03T17:15:18.077 INFO:tasks.workunit.client.1.smithi115.stdout:8/578: symlink d4/d6/d18/d30/d79/lcb 0 2021-11-03T17:15:18.085 INFO:tasks.workunit.client.1.smithi115.stdout:8/579: fdatasync d4/d6/d18/d44/f62 0 2021-11-03T17:15:18.096 INFO:tasks.workunit.client.1.smithi115.stdout:8/580: mkdir d4/d6/d3f/da4/da6/dcc 0 2021-11-03T17:15:18.113 INFO:tasks.workunit.client.1.smithi115.stdout:0/809: write d9/db/d89/f9c [2735942,98180] 0 2021-11-03T17:15:18.114 INFO:tasks.workunit.client.1.smithi115.stdout:0/810: mkdir d9/d37/d10d/d10f 0 2021-11-03T17:15:18.117 INFO:tasks.workunit.client.1.smithi115.stdout:0/811: creat d9/f110 x:0 0 0 2021-11-03T17:15:18.128 INFO:tasks.workunit.client.1.smithi115.stdout:0/812: read d9/d15/d1a/f68 [352604,8381] 0 2021-11-03T17:15:18.132 INFO:tasks.workunit.client.1.smithi115.stdout:0/813: chown d9/d15/l21 1610 1 2021-11-03T17:15:18.154 INFO:tasks.workunit.client.1.smithi115.stdout:8/581: dwrite d4/d6/d18/d30/f34 [0,4194304] 0 2021-11-03T17:15:18.155 INFO:tasks.workunit.client.1.smithi115.stdout:8/582: stat d4/d6/d18/d30/d33/d93 0 2021-11-03T17:15:18.160 INFO:tasks.workunit.client.1.smithi115.stdout:8/583: truncate d4/d6/d3f/d9a/f54 4737914 0 2021-11-03T17:15:18.176 INFO:tasks.workunit.client.1.smithi115.stdout:0/814: write d9/d15/f1e [1956333,81679] 0 2021-11-03T17:15:18.195 INFO:tasks.workunit.client.1.smithi115.stdout:8/584: write d4/d6/fa9 [1369228,78001] 0 2021-11-03T17:15:18.199 INFO:tasks.workunit.client.1.smithi115.stdout:0/815: symlink d9/d15/d90/l111 0 2021-11-03T17:15:18.202 INFO:tasks.workunit.client.1.smithi115.stdout:0/816: write d9/f25 [3412089,75805] 0 2021-11-03T17:15:18.202 INFO:tasks.workunit.client.1.smithi115.stdout:8/585: mkdir d4/d6/d18/d44/dcd 0 2021-11-03T17:15:18.204 INFO:tasks.workunit.client.1.smithi115.stdout:0/817: mkdir d9/db/d83/d112 0 2021-11-03T17:15:18.205 INFO:tasks.workunit.client.1.smithi115.stdout:0/818: read d9/d15/d1a/f68 [832395,48127] 0 2021-11-03T17:15:18.205 INFO:tasks.workunit.client.1.smithi115.stdout:0/819: write d9/d15/d1a/d1d/dcc/d46/f104 [525181,13109] 0 2021-11-03T17:15:18.206 INFO:tasks.workunit.client.1.smithi115.stdout:0/820: write d9/d15/d78/d80/d82/fb5 [3851323,43652] 0 2021-11-03T17:15:18.211 INFO:tasks.workunit.client.1.smithi115.stdout:0/821: rmdir d9/d15/d78/d80/d82 39 2021-11-03T17:15:18.312 INFO:tasks.workunit.client.1.smithi115.stdout:0/822: dwrite d9/d15/d1a/d1d/dcc/d46/d63/dae/fed [0,4194304] 0 2021-11-03T17:15:18.313 INFO:tasks.workunit.client.1.smithi115.stdout:8/586: dwrite d4/d6/d18/d30/d33/f87 [0,4194304] 0 2021-11-03T17:15:18.320 INFO:tasks.workunit.client.1.smithi115.stdout:0/823: dread d9/d15/d78/fd0 [0,4194304] 0 2021-11-03T17:15:18.345 INFO:tasks.workunit.client.1.smithi115.stdout:0/824: truncate d9/d37/fdc 2808061 0 2021-11-03T17:15:18.346 INFO:tasks.workunit.client.1.smithi115.stdout:8/587: symlink d4/d6/d18/lce 0 2021-11-03T17:15:18.349 INFO:tasks.workunit.client.1.smithi115.stdout:0/825: truncate d9/d15/d1a/fa4 392419 0 2021-11-03T17:15:18.357 INFO:tasks.workunit.client.1.smithi115.stdout:8/588: getdents d4/d6/d18/d44/d6e 0 2021-11-03T17:15:18.359 INFO:tasks.workunit.client.1.smithi115.stdout:8/589: chown d4/d7d/fa0 1424 1 2021-11-03T17:15:18.360 INFO:tasks.workunit.client.1.smithi115.stdout:8/590: getdents d4/d6/d3f/da4/da6/dcc 0 2021-11-03T17:15:18.374 INFO:tasks.workunit.client.1.smithi115.stdout:8/591: truncate d4/d6/d18/d30/f31 1173225 0 2021-11-03T17:15:18.375 INFO:tasks.workunit.client.1.smithi115.stdout:0/826: dwrite d9/db/d89/f84 [0,4194304] 0 2021-11-03T17:15:18.379 INFO:tasks.workunit.client.1.smithi115.stdout:0/827: rename d9/d15/d1a/d1d/dcc/d85/lbf to d9/d37/d10d/d10f/l113 0 2021-11-03T17:15:18.381 INFO:tasks.workunit.client.1.smithi115.stdout:0/828: mknod d9/d15/db6/c114 1 2021-11-03T17:15:18.383 INFO:tasks.workunit.client.1.smithi115.stdout:8/592: write d4/d6/f2b [5189485,127260] 0 2021-11-03T17:15:18.383 INFO:tasks.workunit.client.1.smithi115.stdout:0/829: truncate d9/d15/db6/fba 3312262 0 2021-11-03T17:15:18.384 INFO:tasks.workunit.client.1.smithi115.stdout:8/593: mknod d4/d6/d3f/ccf 1 2021-11-03T17:15:18.384 INFO:tasks.workunit.client.1.smithi115.stdout:0/830: write d9/d15/d1a/d1d/dcc/d85/ff8 [882664,53279] 0 2021-11-03T17:15:18.387 INFO:tasks.workunit.client.1.smithi115.stdout:0/831: rmdir d9/db/d83/dcd 0 2021-11-03T17:15:18.390 INFO:tasks.workunit.client.1.smithi115.stdout:0/832: rename d9/db/f75 to d9/d15/d1a/d1d/dcc/d85/f115 0 2021-11-03T17:15:18.393 INFO:tasks.workunit.client.1.smithi115.stdout:0/833: unlink d9/d15/d1a/d1d/f24 0 2021-11-03T17:15:18.395 INFO:tasks.workunit.client.1.smithi115.stdout:0/834: creat d9/db/d83/d112/f116 x:0 0 0 2021-11-03T17:15:18.395 INFO:tasks.workunit.client.1.smithi115.stdout:0/835: fsync d9/db/f42 0 2021-11-03T17:15:18.414 INFO:tasks.workunit.client.1.smithi115.stdout:8/594: dwrite d4/d6/f3c [4194304,4194304] 0 2021-11-03T17:15:18.473 INFO:tasks.workunit.client.1.smithi115.stdout:8/595: dread d4/d6/d18/d30/d33/f95 [0,4194304] 0 2021-11-03T17:15:18.474 INFO:tasks.workunit.client.1.smithi115.stdout:8/596: chown d4/d6/d3f/da4/da6/fc7 170 1 2021-11-03T17:15:18.474 INFO:tasks.workunit.client.1.smithi115.stdout:8/597: stat d4/d6/d3f/d43 0 2021-11-03T17:15:18.481 INFO:tasks.workunit.client.1.smithi115.stdout:0/836: dwrite d9/d15/d1a/fa4 [0,4194304] 0 2021-11-03T17:15:18.483 INFO:tasks.workunit.client.1.smithi115.stdout:0/837: dread d9/d15/d1a/d1d/dcc/d46/d8c/f48 [4194304,4194304] 0 2021-11-03T17:15:18.487 INFO:tasks.workunit.client.1.smithi115.stdout:0/838: dread d9/d15/d1a/d1d/dcc/d46/d8c/fd6 [0,4194304] 0 2021-11-03T17:15:18.512 INFO:tasks.workunit.client.1.smithi115.stdout:0/839: dwrite d9/d15/d78/d80/f9a [0,4194304] 0 2021-11-03T17:15:18.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:18 smithi115 conmon[34670]: cluster 2021-11-03T17:15:16.644302+0000 mgr.smithi027.ufhhvb 2021-11-03T17:15:18.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:18 smithi115 conmon[34670]: (mgr.14622) 20 : cluster [DBG] pgmap v12: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 16 MiB/s rd, 19 MiB/s wr, 417 op/s 2021-11-03T17:15:18.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:18 smithi115 conmon[34670]: cephadm 2021-11-03T17:15:16.971835+0000 mgr.smithi027.ufhhvb (mgr.14622) 21 : cephadm [INF] poll took 4646.113 ms: 1 events 2021-11-03T17:15:18.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:18 smithi115 conmon[34670]: cephadm 2021-11-03T17:15:17.005370+0000 mgr.smithi027.ufhhvb (mgr.14622) 22 : cephadm [INF] Upgrade: Updating mgr.smithi115.tcyodz 2021-11-03T17:15:18.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:18 smithi115 conmon[34670]: cephadm 2021-11-03T17:15:17.008654+0000 2021-11-03T17:15:18.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:18 smithi115 conmon[34670]: mgr.smithi027.ufhhvb (mgr.14622) 23 : cephadm [INF] Deploying daemon mgr.smithi115.tcyodz on smithi115 2021-11-03T17:15:18.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:18 smithi027 conmon[32322]: cluster 2021-11-03T17:15:16. 2021-11-03T17:15:18.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:18 smithi027 conmon[32322]: 644302+0000 mgr.smithi027.ufhhvb (mgr.14622) 20 : cluster [DBG] pgmap v12: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 16 MiB/s rd, 19 MiB/s wr, 417 op/s 2021-11-03T17:15:18.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:18 smithi027 conmon[32322]: cephadm 2021-11-03T17:15:16.971835+0000 2021-11-03T17:15:18.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:18 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14622) 21 : cephadm [INF] poll took 4646.113 ms: 1 events 2021-11-03T17:15:18.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:18 smithi027 conmon[32322]: cephadm 2021-11-03T17 2021-11-03T17:15:18.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:18 smithi027 conmon[32322]: :15:17.005370+0000 mgr.smithi027.ufhhvb (mgr. 2021-11-03T17:15:18.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:18 smithi027 conmon[32322]: 14622) 22 : cephadm [INF] Upgrade: Updating mgr.smithi115.tcyodz 2021-11-03T17:15:18.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:18 smithi027 conmon[32322]: 2021-11-03T17:15:18.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:18 smithi027 conmon[32322]: cephadm 2021-11-03 2021-11-03T17:15:18.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:18 smithi027 conmon[32322]: T17:15:17.008654+0000 mgr.smithi027.ufhhvb ( 2021-11-03T17:15:18.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:18 smithi027 conmon[32322]: mgr.14622) 23 : cephadm [INF] 2021-11-03T17:15:18.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:18 smithi027 conmon[32322]: Deploying daemon mgr.smithi115.tcyodz on smithi115 2021-11-03T17:15:19.239 INFO:tasks.workunit.client.1.smithi115.stdout:0/840: creat d9/f117 x:0 0 0 2021-11-03T17:15:19.262 INFO:tasks.workunit.client.1.smithi115.stdout:0/841: dwrite d9/f117 [0,4194304] 0 2021-11-03T17:15:19.268 INFO:tasks.workunit.client.1.smithi115.stdout:0/842: write d9/db/d89/fb1 [745940,68851] 0 2021-11-03T17:15:19.328 INFO:tasks.workunit.client.1.smithi115.stdout:0/843: creat d9/d15/d1a/d1d/dcc/d46/d8c/dfc/deb/f118 x:0 0 0 2021-11-03T17:15:19.331 INFO:tasks.workunit.client.1.smithi115.stdout:0/844: mknod d9/d15/dce/c119 1 2021-11-03T17:15:19.336 INFO:tasks.workunit.client.1.smithi115.stdout:0/845: chown d9/d15/d1a/d1d/dcc/d46/d8c/lb2 23515 1 2021-11-03T17:15:19.338 INFO:tasks.workunit.client.1.smithi115.stdout:8/598: mkdir d4/d6/d18/d44/dcd/dd0 0 2021-11-03T17:15:19.342 INFO:tasks.workunit.client.1.smithi115.stdout:8/599: symlink d4/d6/d18/d30/ld1 0 2021-11-03T17:15:19.343 INFO:tasks.workunit.client.1.smithi115.stdout:8/600: truncate d4/d6/d3f/d43/d88/f8b 1571349 0 2021-11-03T17:15:19.345 INFO:tasks.workunit.client.1.smithi115.stdout:8/601: getdents d4/d7d 0 2021-11-03T17:15:19.346 INFO:tasks.workunit.client.1.smithi115.stdout:0/846: dread d9/d15/d1a/d1d/dcc/d46/d8c/f48 [0,4194304] 0 2021-11-03T17:15:19.346 INFO:tasks.workunit.client.1.smithi115.stdout:0/847: write d9/d15/d1a/d1d/dcc/d46/d63/dae/fd7 [662381,1342] 0 2021-11-03T17:15:19.349 INFO:tasks.workunit.client.1.smithi115.stdout:0/848: symlink d9/d15/d78/d80/dc4/l11a 0 2021-11-03T17:15:19.356 INFO:tasks.workunit.client.1.smithi115.stdout:0/849: truncate d9/d15/d1a/d1d/dcc/fcf 862030 0 2021-11-03T17:15:19.362 INFO:tasks.workunit.client.1.smithi115.stdout:0/850: creat d9/d15/d1a/d1d/dcc/d46/d63/d70/f11b x:0 0 0 2021-11-03T17:15:19.395 INFO:tasks.workunit.client.1.smithi115.stdout:0/851: dwrite d9/d15/d78/d80/d82/fe6 [4194304,4194304] 0 2021-11-03T17:15:19.406 INFO:tasks.workunit.client.1.smithi115.stdout:0/852: mkdir d9/d15/d1a/d1d/dcc/d46/d63/d70/d11c 0 2021-11-03T17:15:19.418 INFO:tasks.workunit.client.1.smithi115.stdout:0/853: rename d9/d15/d1a/d1d/dcc/d46/d63/d70/dda to d9/d15/d1a/d1d/dcc/d46/d11d 0 2021-11-03T17:15:19.586 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:19 smithi115 conmon[34670]: cluster 2021-11-03T17:15:18.359286+0000 mon.smithi027 (mon.0) 713 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi115.semywj=up:active,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:15:19.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:19 smithi027 conmon[32322]: cluster 2021-11-03T17:15:18.359286+0000 mon.smithi027 (mon 2021-11-03T17:15:19.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:19 smithi027 conmon[32322]: .0) 713 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi115.semywj=up:active,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:15:19.820 INFO:tasks.workunit.client.1.smithi115.stdout:6/473: sync 2021-11-03T17:15:19.848 INFO:tasks.workunit.client.1.smithi115.stdout:6/474: dwrite d0/d1/d10/f3b [0,4194304] 0 2021-11-03T17:15:19.863 INFO:tasks.workunit.client.1.smithi115.stdout:4/690: sync 2021-11-03T17:15:19.863 INFO:tasks.workunit.client.1.smithi115.stdout:7/537: sync 2021-11-03T17:15:19.863 INFO:tasks.workunit.client.1.smithi115.stdout:6/475: write d0/d1/d2/ff [4348251,130049] 0 2021-11-03T17:15:19.864 INFO:tasks.workunit.client.1.smithi115.stdout:6/476: write d0/d1/d2/d3f/f51 [655697,65126] 0 2021-11-03T17:15:19.870 INFO:tasks.workunit.client.1.smithi115.stdout:4/691: symlink d7/dc/d6a/dab/le8 0 2021-11-03T17:15:19.877 INFO:tasks.workunit.client.1.smithi115.stdout:4/692: truncate d7/dc/d1a/d21/fa1 3702232 0 2021-11-03T17:15:19.879 INFO:tasks.workunit.client.1.smithi115.stdout:6/477: rename d0/d1/d6a/f93 to d0/d1/f9e 0 2021-11-03T17:15:19.884 INFO:tasks.workunit.client.1.smithi115.stdout:7/538: write d3/da/d18/d73/d33/f59 [905957,60436] 0 2021-11-03T17:15:19.889 INFO:tasks.workunit.client.1.smithi115.stdout:4/693: creat d7/dc/fe9 x:0 0 0 2021-11-03T17:15:19.892 INFO:tasks.workunit.client.1.smithi115.stdout:6/478: readlink d0/d1/d2e/d4c/d96/d61/l6f 0 2021-11-03T17:15:19.892 INFO:tasks.workunit.client.1.smithi115.stdout:6/479: stat d0/d1/d2e/d4c 0 2021-11-03T17:15:19.898 INFO:tasks.workunit.client.1.smithi115.stdout:6/480: readlink d0/d1/d10/d13/l62 0 2021-11-03T17:15:19.902 INFO:tasks.workunit.client.1.smithi115.stdout:6/481: mknod d0/d1/d2/c9f 1 2021-11-03T17:15:19.905 INFO:tasks.workunit.client.1.smithi115.stdout:6/482: fdatasync d0/f2a 0 2021-11-03T17:15:19.909 INFO:tasks.workunit.client.1.smithi115.stdout:6/483: rename d0/d1/d10/d13/d1c/d9d to d0/d1/d6a/d88/da0 0 2021-11-03T17:15:19.911 INFO:tasks.workunit.client.1.smithi115.stdout:6/484: chown d0/d1/d2/l7d 433 1 2021-11-03T17:15:19.922 INFO:tasks.workunit.client.1.smithi115.stdout:6/485: creat d0/d1/d6a/fa1 x:0 0 0 2021-11-03T17:15:19.923 INFO:tasks.workunit.client.1.smithi115.stdout:7/539: dwrite d3/da/d18/f40 [4194304,4194304] 0 2021-11-03T17:15:19.923 INFO:tasks.workunit.client.1.smithi115.stdout:4/694: dread d7/dc/d1a/d21/db1/d56/d63/f87 [0,4194304] 0 2021-11-03T17:15:19.929 INFO:tasks.workunit.client.1.smithi115.stdout:6/486: dread d0/d1/d2/f35 [0,4194304] 0 2021-11-03T17:15:19.932 INFO:tasks.workunit.client.1.smithi115.stdout:7/540: unlink d3/da/d18/d73/d52/l88 0 2021-11-03T17:15:19.933 INFO:tasks.workunit.client.1.smithi115.stdout:7/541: fsync d3/da/d18/d73/d35/d53/f87 0 2021-11-03T17:15:19.933 INFO:tasks.workunit.client.1.smithi115.stdout:7/542: readlink d3/d4c/la4 0 2021-11-03T17:15:19.934 INFO:tasks.workunit.client.1.smithi115.stdout:4/695: creat d7/d7a/fea x:0 0 0 2021-11-03T17:15:19.935 INFO:tasks.workunit.client.1.smithi115.stdout:4/696: fsync d7/dc/d1a/d21/db1/d56/d63/f87 0 2021-11-03T17:15:19.936 INFO:tasks.workunit.client.1.smithi115.stdout:6/487: chown d0/d1/d2e/d4c/l84 296470 1 2021-11-03T17:15:19.937 INFO:tasks.workunit.client.1.smithi115.stdout:6/488: stat d0/d1/d2e/d4c/d96/f6b 0 2021-11-03T17:15:19.943 INFO:tasks.workunit.client.1.smithi115.stdout:4/697: creat d7/dc/d1a/d21/db1/d56/d42/feb x:0 0 0 2021-11-03T17:15:19.950 INFO:tasks.workunit.client.1.smithi115.stdout:6/489: dread d0/f4d [0,4194304] 0 2021-11-03T17:15:19.964 INFO:tasks.workunit.client.1.smithi115.stdout:6/490: creat d0/fa2 x:0 0 0 2021-11-03T17:15:19.964 INFO:tasks.workunit.client.1.smithi115.stdout:6/491: chown d0/d1/d2e/f83 418 1 2021-11-03T17:15:19.965 INFO:tasks.workunit.client.1.smithi115.stdout:6/492: dread - d0/f7e zero size 2021-11-03T17:15:19.967 INFO:tasks.workunit.client.1.smithi115.stdout:4/698: unlink d7/dc/db6/l65 0 2021-11-03T17:15:19.968 INFO:tasks.workunit.client.1.smithi115.stdout:6/493: read d0/d1/d2/d3f/f4b [321269,98092] 0 2021-11-03T17:15:19.973 INFO:tasks.workunit.client.1.smithi115.stdout:4/699: truncate d7/dc/d1a/d21/db1/d56/d42/fa5 1641493 0 2021-11-03T17:15:19.983 INFO:tasks.workunit.client.1.smithi115.stdout:6/494: link d0/d1/d10/d13/d1c/f38 d0/d1/d10/d13/d59/fa3 0 2021-11-03T17:15:19.983 INFO:tasks.workunit.client.1.smithi115.stdout:6/495: readlink d0/d1/d10/d13/l62 0 2021-11-03T17:15:19.985 INFO:tasks.workunit.client.1.smithi115.stdout:6/496: readlink d0/d1/d2e/d4c/d96/d61/l78 0 2021-11-03T17:15:19.987 INFO:tasks.workunit.client.1.smithi115.stdout:4/700: rename d7/dc/d6a/l6d to d7/dc/d1a/d21/db1/d56/d42/daf/lec 0 2021-11-03T17:15:19.994 INFO:tasks.workunit.client.1.smithi115.stdout:4/701: creat d7/dc/d1a/d21/fed x:0 0 0 2021-11-03T17:15:19.999 INFO:tasks.workunit.client.1.smithi115.stdout:6/497: link d0/d1/d10/d13/f90 d0/d1/d10/d13/d87/fa4 0 2021-11-03T17:15:20.000 INFO:tasks.workunit.client.1.smithi115.stdout:4/702: readlink d7/l26 0 2021-11-03T17:15:20.006 INFO:tasks.workunit.client.1.smithi115.stdout:4/703: readlink d7/dc/l33 0 2021-11-03T17:15:20.024 INFO:tasks.workunit.client.1.smithi115.stdout:4/704: getdents d7/dc/d1a 0 2021-11-03T17:15:20.028 INFO:tasks.workunit.client.1.smithi115.stdout:6/498: rename d0/d1/f50 to d0/d1/d10/d13/d87/fa5 0 2021-11-03T17:15:20.032 INFO:tasks.workunit.client.1.smithi115.stdout:6/499: mknod d0/d1/ca6 1 2021-11-03T17:15:20.035 INFO:tasks.workunit.client.1.smithi115.stdout:4/705: mkdir d7/dd1/dee 0 2021-11-03T17:15:20.035 INFO:tasks.workunit.client.1.smithi115.stdout:4/706: stat d7/dc/d6a/d3c/f3e 0 2021-11-03T17:15:20.036 INFO:tasks.workunit.client.1.smithi115.stdout:6/500: creat d0/d1/d10/fa7 x:0 0 0 2021-11-03T17:15:20.040 INFO:tasks.workunit.client.1.smithi115.stdout:4/707: creat d7/dc/d6a/dab/ddd/fef x:0 0 0 2021-11-03T17:15:20.041 INFO:tasks.workunit.client.1.smithi115.stdout:3/736: sync 2021-11-03T17:15:20.042 INFO:tasks.workunit.client.1.smithi115.stdout:6/501: fdatasync d0/f22 0 2021-11-03T17:15:20.047 INFO:tasks.workunit.client.1.smithi115.stdout:3/737: fsync d13/d35/f3d 0 2021-11-03T17:15:20.074 INFO:tasks.workunit.client.1.smithi115.stdout:4/708: rename d7/dc/ld2 to d7/dc/d6a/d3c/dc5/dcc/lf0 0 2021-11-03T17:15:20.079 INFO:tasks.workunit.client.1.smithi115.stdout:3/738: dread d13/f1c [0,4194304] 0 2021-11-03T17:15:20.081 INFO:tasks.workunit.client.1.smithi115.stdout:4/709: read d7/dc/d1a/d21/db1/d56/d63/f9b [9682,46228] 0 2021-11-03T17:15:20.083 INFO:tasks.workunit.client.1.smithi115.stdout:4/710: mknod d7/dc/d1a/d21/db1/cf1 1 2021-11-03T17:15:20.086 INFO:tasks.workunit.client.1.smithi115.stdout:4/711: creat d7/d7a/ff2 x:0 0 0 2021-11-03T17:15:20.089 INFO:tasks.workunit.client.1.smithi115.stdout:4/712: creat d7/dc/d6a/d3c/dbf/ff3 x:0 0 0 2021-11-03T17:15:20.092 INFO:tasks.workunit.client.1.smithi115.stdout:4/713: mkdir d7/dc/d1a/d21/db1/d56/df4 0 2021-11-03T17:15:20.093 INFO:tasks.workunit.client.1.smithi115.stdout:4/714: read d7/dc/d6a/fac [239921,127458] 0 2021-11-03T17:15:20.100 INFO:tasks.workunit.client.1.smithi115.stdout:3/739: dwrite d13/d2a/d8a/fa6 [0,4194304] 0 2021-11-03T17:15:20.123 INFO:tasks.workunit.client.1.smithi115.stdout:3/740: mkdir d13/d50/dfc 0 2021-11-03T17:15:20.136 INFO:tasks.workunit.client.1.smithi115.stdout:4/715: dwrite d7/dc/f95 [0,4194304] 0 2021-11-03T17:15:20.139 INFO:tasks.workunit.client.1.smithi115.stdout:4/716: symlink d7/dc/d6a/d72/lf5 0 2021-11-03T17:15:20.139 INFO:tasks.workunit.client.1.smithi115.stdout:4/717: mknod d7/dc/d1a/cf6 1 2021-11-03T17:15:20.141 INFO:tasks.workunit.client.1.smithi115.stdout:4/718: mknod d7/dc/d1a/d21/db1/d56/d63/cf7 1 2021-11-03T17:15:20.143 INFO:tasks.workunit.client.1.smithi115.stdout:4/719: chown d7/dc/d1a/d21/db1/d56/d63/d7b/d96/fa7 17 1 2021-11-03T17:15:20.144 INFO:tasks.workunit.client.1.smithi115.stdout:4/720: read - d7/fe5 zero size 2021-11-03T17:15:20.145 INFO:tasks.workunit.client.1.smithi115.stdout:3/741: dwrite d13/d51/d40/f4f [0,4194304] 0 2021-11-03T17:15:20.156 INFO:tasks.workunit.client.1.smithi115.stdout:4/721: write d7/dc/d1a/d21/db1/d56/d63/fd7 [221316,85357] 0 2021-11-03T17:15:20.158 INFO:tasks.workunit.client.1.smithi115.stdout:3/742: read - d13/d2a/d53/d70/d92/dc2/fca zero size 2021-11-03T17:15:20.160 INFO:tasks.workunit.client.1.smithi115.stdout:4/722: fsync d7/dc/f60 0 2021-11-03T17:15:20.162 INFO:tasks.workunit.client.1.smithi115.stdout:3/743: chown d13/d51/d2e/db4 1568172 1 2021-11-03T17:15:20.168 INFO:tasks.workunit.client.1.smithi115.stdout:3/744: creat d13/d3b/db1/ffd x:0 0 0 2021-11-03T17:15:20.172 INFO:tasks.workunit.client.1.smithi115.stdout:4/723: symlink d7/dc/d1a/d21/db1/d56/d63/d9e/lf8 0 2021-11-03T17:15:20.173 INFO:tasks.workunit.client.1.smithi115.stdout:3/745: creat d13/d35/d54/ffe x:0 0 0 2021-11-03T17:15:20.175 INFO:tasks.workunit.client.1.smithi115.stdout:4/724: fsync d7/dc/d1a/d21/db1/d56/f93 0 2021-11-03T17:15:20.182 INFO:tasks.workunit.client.1.smithi115.stdout:4/725: dread d7/d7a/d83/f8c [0,4194304] 0 2021-11-03T17:15:20.182 INFO:tasks.workunit.client.1.smithi115.stdout:4/726: dread - d7/dc/fe9 zero size 2021-11-03T17:15:20.188 INFO:tasks.workunit.client.1.smithi115.stdout:4/727: fsync d7/dc/fb0 0 2021-11-03T17:15:20.191 INFO:tasks.workunit.client.1.smithi115.stdout:4/728: getdents d7/dc/d1a/d21/db1 0 2021-11-03T17:15:20.206 INFO:tasks.workunit.client.1.smithi115.stdout:3/746: dwrite d13/d2a/d8a/fd0 [0,4194304] 0 2021-11-03T17:15:20.215 INFO:tasks.workunit.client.1.smithi115.stdout:3/747: symlink d13/d3b/db1/dbc/lff 0 2021-11-03T17:15:20.216 INFO:tasks.workunit.client.1.smithi115.stdout:3/748: fdatasync d13/d2a/d8a/d99/fb8 0 2021-11-03T17:15:20.218 INFO:tasks.workunit.client.1.smithi115.stdout:3/749: readlink d13/d2a/lcc 0 2021-11-03T17:15:20.225 INFO:tasks.workunit.client.1.smithi115.stdout:3/750: creat d13/d35/d54/dad/f100 x:0 0 0 2021-11-03T17:15:20.230 INFO:tasks.workunit.client.1.smithi115.stdout:3/751: rename d13/d3b/f58 to d13/d35/f101 0 2021-11-03T17:15:20.236 INFO:tasks.workunit.client.1.smithi115.stdout:4/729: dwrite d7/dc/d1a/d21/db1/d56/d42/f58 [0,4194304] 0 2021-11-03T17:15:20.248 INFO:tasks.workunit.client.1.smithi115.stdout:4/730: unlink d7/dc/d1a/d21/db1/d56/d63/f87 0 2021-11-03T17:15:20.249 INFO:tasks.workunit.client.1.smithi115.stdout:9/594: sync 2021-11-03T17:15:20.253 INFO:tasks.workunit.client.1.smithi115.stdout:4/731: readlink d7/dc/l34 0 2021-11-03T17:15:20.258 INFO:tasks.workunit.client.1.smithi115.stdout:9/595: creat d1/d1a/fe0 x:0 0 0 2021-11-03T17:15:20.262 INFO:tasks.workunit.client.1.smithi115.stdout:4/732: mkdir d7/d7a/d83/df9 0 2021-11-03T17:15:20.267 INFO:tasks.workunit.client.1.smithi115.stdout:3/752: dwrite d13/d51/d2e/fa2 [0,4194304] 0 2021-11-03T17:15:20.282 INFO:tasks.workunit.client.1.smithi115.stdout:4/733: creat d7/dc/d6a/d3c/dc5/ffa x:0 0 0 2021-11-03T17:15:20.288 INFO:tasks.workunit.client.1.smithi115.stdout:4/734: fsync d7/dc/d6a/d3c/fa8 0 2021-11-03T17:15:20.292 INFO:tasks.workunit.client.1.smithi115.stdout:9/596: dwrite d1/d1a/d48/d27/d2f/d41/f9d [0,4194304] 0 2021-11-03T17:15:20.294 INFO:tasks.workunit.client.1.smithi115.stdout:3/753: write d13/d51/fcf [745793,117184] 0 2021-11-03T17:15:20.301 INFO:tasks.workunit.client.1.smithi115.stdout:4/735: symlink d7/dc/d1a/d21/lfb 0 2021-11-03T17:15:20.313 INFO:tasks.workunit.client.1.smithi115.stdout:4/736: dread d7/dc/d6a/d3c/fbd [0,4194304] 0 2021-11-03T17:15:20.313 INFO:tasks.workunit.client.1.smithi115.stdout:4/737: chown d7/d7a/ff2 63062 1 2021-11-03T17:15:20.320 INFO:tasks.workunit.client.1.smithi115.stdout:9/597: dwrite d1/d1a/d48/f26 [0,4194304] 0 2021-11-03T17:15:20.326 INFO:tasks.workunit.client.1.smithi115.stdout:4/738: write d7/dc/d1a/d21/db1/d56/d63/fa6 [1150380,83724] 0 2021-11-03T17:15:20.327 INFO:tasks.workunit.client.1.smithi115.stdout:4/739: chown d7/dc/d1a/d21/db1/d56/d42/f58 14 1 2021-11-03T17:15:20.334 INFO:tasks.workunit.client.1.smithi115.stdout:4/740: dread d7/dc/d1a/d21/db1/d56/d42/fa5 [0,4194304] 0 2021-11-03T17:15:20.336 INFO:tasks.workunit.client.1.smithi115.stdout:9/598: creat d1/d18/d4a/d74/d8b/dbc/fe1 x:0 0 0 2021-11-03T17:15:20.336 INFO:tasks.workunit.client.1.smithi115.stdout:9/599: dread - d1/d1a/d48/d27/d80/dcc/fd8 zero size 2021-11-03T17:15:20.337 INFO:tasks.workunit.client.1.smithi115.stdout:9/600: dread - d1/d18/d4a/fc8 zero size 2021-11-03T17:15:20.340 INFO:tasks.workunit.client.1.smithi115.stdout:3/754: dwrite d13/da4/fb9 [0,4194304] 0 2021-11-03T17:15:20.345 INFO:tasks.workunit.client.1.smithi115.stdout:9/601: dread d1/d1a/d48/f2a [0,4194304] 0 2021-11-03T17:15:20.353 INFO:tasks.workunit.client.1.smithi115.stdout:9/602: creat d1/d1a/d48/d27/d53/fe2 x:0 0 0 2021-11-03T17:15:20.355 INFO:tasks.workunit.client.1.smithi115.stdout:3/755: dread d13/d51/f41 [0,4194304] 0 2021-11-03T17:15:20.359 INFO:tasks.workunit.client.1.smithi115.stdout:3/756: rmdir d13/d2a/d53/d70/d92/db7 39 2021-11-03T17:15:20.367 INFO:tasks.workunit.client.1.smithi115.stdout:3/757: truncate d13/da4/fe2 165058 0 2021-11-03T17:15:20.373 INFO:tasks.workunit.client.1.smithi115.stdout:3/758: write d13/da4/fe2 [761471,9178] 0 2021-11-03T17:15:20.375 INFO:tasks.workunit.client.1.smithi115.stdout:3/759: creat d13/d2a/d53/d70/f102 x:0 0 0 2021-11-03T17:15:20.378 INFO:tasks.workunit.client.1.smithi115.stdout:3/760: unlink d13/d2a/d53/d70/d92/db7/lf2 0 2021-11-03T17:15:20.380 INFO:tasks.workunit.client.1.smithi115.stdout:3/761: mkdir d13/d50/d9e/d103 0 2021-11-03T17:15:20.380 INFO:tasks.workunit.client.1.smithi115.stdout:3/762: readlink d13/d51/d2e/db4/lc7 0 2021-11-03T17:15:20.383 INFO:tasks.workunit.client.1.smithi115.stdout:3/763: mkdir d13/d51/d2e/db4/d104 0 2021-11-03T17:15:20.384 INFO:tasks.workunit.client.1.smithi115.stdout:3/764: write d13/d51/d2e/fa2 [3516106,58392] 0 2021-11-03T17:15:20.385 INFO:tasks.workunit.client.1.smithi115.stdout:3/765: rmdir d13/d2c 39 2021-11-03T17:15:20.391 INFO:tasks.workunit.client.1.smithi115.stdout:9/603: dwrite d1/d1a/d48/d27/f47 [0,4194304] 0 2021-11-03T17:15:20.395 INFO:tasks.workunit.client.1.smithi115.stdout:9/604: chown d1/d18/d2b/d42/la0 79059 1 2021-11-03T17:15:20.395 INFO:tasks.workunit.client.1.smithi115.stdout:9/605: fdatasync d1/d1a/d48/d27/f4c 0 2021-11-03T17:15:20.413 INFO:tasks.workunit.client.1.smithi115.stdout:9/606: link d1/d18/d2b/d42/d6e/db5/f99 d1/d1a/d48/d27/d2f/d7b/fe3 0 2021-11-03T17:15:20.416 INFO:tasks.workunit.client.1.smithi115.stdout:9/607: truncate d1/d1a/d48/d27/f7a 1019578 0 2021-11-03T17:15:20.416 INFO:tasks.workunit.client.1.smithi115.stdout:9/608: dread - d1/d1a/d48/d27/f4c zero size 2021-11-03T17:15:20.424 INFO:tasks.workunit.client.1.smithi115.stdout:9/609: dread d1/d18/d2b/d42/fbd [4194304,4194304] 0 2021-11-03T17:15:20.435 INFO:tasks.workunit.client.1.smithi115.stdout:9/610: unlink d1/d18/d2b/d42/d6e/db5/f99 0 2021-11-03T17:15:20.436 INFO:tasks.workunit.client.1.smithi115.stdout:9/611: creat d1/d1a/d48/d27/d53/d83/d88/dcd/dda/fe4 x:0 0 0 2021-11-03T17:15:20.436 INFO:tasks.workunit.client.1.smithi115.stdout:9/612: truncate d1/d1a/d48/d27/d2f/d6a/fd4 2445567 0 2021-11-03T17:15:20.437 INFO:tasks.workunit.client.1.smithi115.stdout:9/613: chown d1/d18/d2b/d42/d6e/f52 501209 1 2021-11-03T17:15:20.448 INFO:tasks.workunit.client.1.smithi115.stdout:9/614: dread d1/f25 [0,4194304] 0 2021-11-03T17:15:20.450 INFO:tasks.workunit.client.1.smithi115.stdout:9/615: fsync d1/d1a/d48/d27/d53/f95 0 2021-11-03T17:15:20.458 INFO:tasks.workunit.client.1.smithi115.stdout:5/698: sync 2021-11-03T17:15:20.459 INFO:tasks.workunit.client.1.smithi115.stdout:2/617: sync 2021-11-03T17:15:20.459 INFO:tasks.workunit.client.1.smithi115.stdout:1/677: sync 2021-11-03T17:15:20.464 INFO:tasks.workunit.client.1.smithi115.stdout:9/616: write d1/d18/d2b/f35 [1594278,48652] 0 2021-11-03T17:15:20.469 INFO:tasks.workunit.client.1.smithi115.stdout:5/699: unlink d1a/d28/d4c/d75/d68/lcd 0 2021-11-03T17:15:20.469 INFO:tasks.workunit.client.1.smithi115.stdout:5/700: readlink d1a/d28/d4c/d75/d68/da5/lde 0 2021-11-03T17:15:20.472 INFO:tasks.workunit.client.1.smithi115.stdout:2/618: mknod d4/d6/d56/d6c/d34/d42/d8d/cd4 1 2021-11-03T17:15:20.475 INFO:tasks.workunit.client.1.smithi115.stdout:1/678: read d7/dc/dd/d3e/dd1/d4d/d60/f6d [233942,71590] 0 2021-11-03T17:15:20.494 INFO:tasks.workunit.client.1.smithi115.stdout:9/617: unlink d1/d1a/d48/d27/d2f/d7b/f85 0 2021-11-03T17:15:20.498 INFO:tasks.workunit.client.1.smithi115.stdout:2/619: rename d4/d6/d56/d6c/d34/d42/d8d/fc9 to d4/d6/d13/d75/fd5 0 2021-11-03T17:15:20.502 INFO:tasks.workunit.client.1.smithi115.stdout:1/679: truncate d7/dc/dd/f95 1227822 0 2021-11-03T17:15:20.508 INFO:tasks.workunit.client.1.smithi115.stdout:5/701: dwrite d1a/d28/d7b/f77 [0,4194304] 0 2021-11-03T17:15:20.509 INFO:tasks.workunit.client.1.smithi115.stdout:9/618: truncate d1/d18/d4a/d74/f7e 1755954 0 2021-11-03T17:15:20.510 INFO:tasks.workunit.client.1.smithi115.stdout:2/620: dread - d4/d6/dad/d7f/f8e zero size 2021-11-03T17:15:20.514 INFO:tasks.workunit.client.1.smithi115.stdout:9/619: rmdir d1/d1a/d48/d27/d53/d83/d88/dcd 39 2021-11-03T17:15:20.515 INFO:tasks.workunit.client.1.smithi115.stdout:2/621: mknod d4/d6/d56/d6c/d34/d43/d63/d77/d79/cd6 1 2021-11-03T17:15:20.516 INFO:tasks.workunit.client.1.smithi115.stdout:1/680: dread d7/dc/ff [0,4194304] 0 2021-11-03T17:15:20.518 INFO:tasks.workunit.client.1.smithi115.stdout:5/702: symlink d1a/d28/d4c/d75/dae/le3 0 2021-11-03T17:15:20.520 INFO:tasks.workunit.client.1.smithi115.stdout:2/622: chown d4/d6/d56/d6c/d34/d42/la1 6894 1 2021-11-03T17:15:20.527 INFO:tasks.workunit.client.1.smithi115.stdout:9/620: truncate d1/d1a/d48/f73 4093707 0 2021-11-03T17:15:20.528 INFO:tasks.workunit.client.1.smithi115.stdout:5/703: creat d1a/d28/d4a/fe4 x:0 0 0 2021-11-03T17:15:20.551 INFO:tasks.workunit.client.1.smithi115.stdout:5/704: getdents d1a/d28/d7b/d4f/d81 0 2021-11-03T17:15:20.555 INFO:tasks.workunit.client.1.smithi115.stdout:1/681: dwrite d7/fb3 [0,4194304] 0 2021-11-03T17:15:20.555 INFO:tasks.workunit.client.1.smithi115.stdout:2/623: link d4/d6/d56/d6c/lf d4/d6/d56/d6c/d34/d43/d28/ld7 0 2021-11-03T17:15:20.564 INFO:tasks.workunit.client.1.smithi115.stdout:5/705: symlink d1a/d28/daa/le5 0 2021-11-03T17:15:20.576 INFO:tasks.workunit.client.1.smithi115.stdout:2/624: dread d4/d6/d13/f4e [0,4194304] 0 2021-11-03T17:15:20.576 INFO:tasks.workunit.client.1.smithi115.stdout:1/682: dread - d7/dc/dd/d3e/dd1/d4d/d60/d8f/fa7 zero size 2021-11-03T17:15:20.582 INFO:tasks.workunit.client.1.smithi115.stdout:5/706: dread d1a/d28/d4c/d75/d83/f98 [0,4194304] 0 2021-11-03T17:15:20.587 INFO:tasks.workunit.client.1.smithi115.stdout:5/707: chown d1a/d3d/d58/dab/lc2 3591 1 2021-11-03T17:15:20.603 INFO:tasks.workunit.client.1.smithi115.stdout:5/708: mkdir d1a/d23/de6 0 2021-11-03T17:15:20.605 INFO:tasks.workunit.client.1.smithi115.stdout:5/709: getdents d1a/d28/d4c/d75 0 2021-11-03T17:15:20.605 INFO:tasks.workunit.client.1.smithi115.stdout:5/710: mknod d1a/d3d/d58/dab/dd3/ce7 1 2021-11-03T17:15:20.613 INFO:tasks.workunit.client.1.smithi115.stdout:5/711: rename d1a/d28/d4c/d75/d68/da5/l79 to d1a/d28/d4c/d75/d83/da1/le8 0 2021-11-03T17:15:20.614 INFO:tasks.workunit.client.1.smithi115.stdout:5/712: creat d1a/d3d/dc4/fe9 x:0 0 0 2021-11-03T17:15:20.616 INFO:tasks.workunit.client.1.smithi115.stdout:5/713: creat d1a/d28/daa/fea x:0 0 0 2021-11-03T17:15:20.617 INFO:tasks.workunit.client.1.smithi115.stdout:5/714: truncate d1a/d28/f31 4401264 0 2021-11-03T17:15:20.618 INFO:tasks.workunit.client.1.smithi115.stdout:2/625: dwrite d4/f35 [0,4194304] 0 2021-11-03T17:15:20.619 INFO:tasks.workunit.client.1.smithi115.stdout:1/683: dwrite d7/dc/dd/d3e/dd1/d43/f99 [0,4194304] 0 2021-11-03T17:15:20.619 INFO:tasks.workunit.client.1.smithi115.stdout:5/715: mkdir d1a/d28/d34/deb 0 2021-11-03T17:15:20.639 INFO:tasks.workunit.client.1.smithi115.stdout:1/684: dread d7/dc/dd/d3e/dd1/d43/f7e [0,4194304] 0 2021-11-03T17:15:20.644 INFO:tasks.workunit.client.1.smithi115.stdout:5/716: creat d1a/d28/fec x:0 0 0 2021-11-03T17:15:20.649 INFO:tasks.workunit.client.1.smithi115.stdout:2/626: mkdir d4/d6/d56/d6c/d34/d42/d8d/dd8 0 2021-11-03T17:15:20.649 INFO:tasks.workunit.client.1.smithi115.stdout:2/627: chown d4/d6/d56/d6c/d34/d43/d63/d77/d79/dac 2510563 1 2021-11-03T17:15:20.653 INFO:tasks.workunit.client.1.smithi115.stdout:1/685: write d7/f7c [601790,62799] 0 2021-11-03T17:15:20.656 INFO:tasks.workunit.client.1.smithi115.stdout:1/686: creat d7/fdd x:0 0 0 2021-11-03T17:15:20.659 INFO:tasks.workunit.client.1.smithi115.stdout:5/717: mkdir d1a/d28/d4c/d75/d83/da1/ded 0 2021-11-03T17:15:20.664 INFO:tasks.workunit.client.1.smithi115.stdout:5/718: unlink d1a/d3d/d58/f7c 0 2021-11-03T17:15:20.665 INFO:tasks.workunit.client.1.smithi115.stdout:5/719: write d1a/d28/d7b/f9f [1120753,23362] 0 2021-11-03T17:15:20.669 INFO:tasks.workunit.client.1.smithi115.stdout:5/720: fsync d1a/d28/d4c/d75/d68/da5/fa8 0 2021-11-03T17:15:20.670 INFO:tasks.workunit.client.1.smithi115.stdout:5/721: dread - d1a/d28/fd1 zero size 2021-11-03T17:15:20.681 INFO:tasks.workunit.client.1.smithi115.stdout:1/687: read d7/dc/dd/d3e/dd1/d4b/d55/f6c [1631063,14649] 0 2021-11-03T17:15:20.684 INFO:tasks.workunit.client.1.smithi115.stdout:5/722: write d1a/d28/d34/f2a [184565,90490] 0 2021-11-03T17:15:20.701 INFO:tasks.workunit.client.1.smithi115.stdout:1/688: write d7/dc/dd/d3e/dd1/d26/f4f [3505655,103961] 0 2021-11-03T17:15:20.701 INFO:tasks.workunit.client.1.smithi115.stdout:5/723: link d1a/d28/fec d1a/d28/d4c/fee 0 2021-11-03T17:15:20.702 INFO:tasks.workunit.client.1.smithi115.stdout:5/724: mknod d1a/d3d/d58/dab/dd3/cef 1 2021-11-03T17:15:20.702 INFO:tasks.workunit.client.1.smithi115.stdout:5/725: chown d1a/d3d/dc0/ldb 153516861 1 2021-11-03T17:15:20.702 INFO:tasks.workunit.client.1.smithi115.stdout:5/726: truncate d1a/fe1 316065 0 2021-11-03T17:15:20.706 INFO:tasks.workunit.client.1.smithi115.stdout:1/689: getdents d7/dc/dd/d3e/dbe 0 2021-11-03T17:15:20.712 INFO:tasks.workunit.client.1.smithi115.stdout:5/727: write d1a/d28/d4a/f8b [415644,5432] 0 2021-11-03T17:15:20.721 INFO:tasks.workunit.client.1.smithi115.stdout:5/728: link d1a/d3d/f3b d1a/d28/d4a/d65/ff0 0 2021-11-03T17:15:20.721 INFO:tasks.workunit.client.1.smithi115.stdout:1/690: dread d7/dc/dd/d3e/dd1/d4d/d60/f81 [0,4194304] 0 2021-11-03T17:15:20.723 INFO:tasks.workunit.client.1.smithi115.stdout:5/729: creat d1a/d28/d7b/d48/dd4/ff1 x:0 0 0 2021-11-03T17:15:20.724 INFO:tasks.workunit.client.1.smithi115.stdout:1/691: readlink d7/dc/dd/d3e/dd1/d4d/lb5 0 2021-11-03T17:15:20.730 INFO:tasks.workunit.client.1.smithi115.stdout:5/730: link d1a/d28/d4c/fee d1a/d28/d7b/d48/ff2 0 2021-11-03T17:15:20.733 INFO:tasks.workunit.client.1.smithi115.stdout:1/692: mkdir d7/dc/dd/d3e/dd1/d4d/d60/dde 0 2021-11-03T17:15:20.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:20 smithi115 conmon[34670]: cluster 2021-11-03T17:15:18.644872+0000 2021-11-03T17:15:20.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:20 smithi115 conmon[34670]: mgr.smithi027.ufhhvb (mgr.14622) 24 : cluster [DBG] pgmap v13: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 26 MiB/s rd, 36 MiB/s wr, 709 op/s 2021-11-03T17:15:20.734 INFO:tasks.workunit.client.1.smithi115.stdout:5/731: creat d1a/d3d/dc0/ddf/ff3 x:0 0 0 2021-11-03T17:15:20.738 INFO:tasks.workunit.client.1.smithi115.stdout:5/732: creat d1a/d28/d7b/ff4 x:0 0 0 2021-11-03T17:15:20.742 INFO:tasks.workunit.client.1.smithi115.stdout:1/693: mkdir d7/dc/dd/d3e/dd1/d4d/d60/d8f/ddf 0 2021-11-03T17:15:20.748 INFO:tasks.workunit.client.1.smithi115.stdout:1/694: truncate d7/dc/f15 7177155 0 2021-11-03T17:15:20.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:20 smithi027 conmon[32322]: cluster 2021-11-03T17:15:18.644872 2021-11-03T17:15:20.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:20 smithi027 conmon[32322]: +0000 mgr.smithi027.ufhhvb (mgr.14622) 24 : cluster [DBG] pgmap v13: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 26 MiB/s rd, 36 MiB/s wr, 709 op/s 2021-11-03T17:15:20.769 INFO:tasks.workunit.client.1.smithi115.stdout:1/695: write d7/dc/dd/d3e/dd1/d4b/f72 [1666274,36613] 0 2021-11-03T17:15:20.875 INFO:tasks.workunit.client.1.smithi115.stdout:0/854: sync 2021-11-03T17:15:20.950 INFO:tasks.workunit.client.1.smithi115.stdout:8/602: sync 2021-11-03T17:15:21.437 INFO:tasks.workunit.client.1.smithi115.stdout:7/543: sync 2021-11-03T17:15:21.443 INFO:tasks.workunit.client.1.smithi115.stdout:7/544: symlink d3/da/d18/d73/d35/lab 0 2021-11-03T17:15:21.474 INFO:tasks.workunit.client.1.smithi115.stdout:7/545: dwrite d3/da/d18/d73/d52/f66 [0,4194304] 0 2021-11-03T17:15:21.474 INFO:tasks.workunit.client.1.smithi115.stdout:7/546: dread - d3/da/d18/d73/d52/f83 zero size 2021-11-03T17:15:21.475 INFO:tasks.workunit.client.1.smithi115.stdout:7/547: read d3/da/d18/d73/d33/f5e [77917,35208] 0 2021-11-03T17:15:21.496 INFO:tasks.workunit.client.1.smithi115.stdout:7/548: dwrite d3/d4c/d7b/f91 [0,4194304] 0 2021-11-03T17:15:21.497 INFO:tasks.workunit.client.1.smithi115.stdout:7/549: write d3/da/d78/f9f [4198617,8143] 0 2021-11-03T17:15:21.497 INFO:tasks.workunit.client.1.smithi115.stdout:7/550: write d3/da/d18/d73/d35/f5f [15814,17401] 0 2021-11-03T17:15:21.506 INFO:tasks.workunit.client.1.smithi115.stdout:7/551: symlink d3/lac 0 2021-11-03T17:15:21.508 INFO:tasks.workunit.client.1.smithi115.stdout:7/552: creat d3/da/d18/d73/d35/fad x:0 0 0 2021-11-03T17:15:21.508 INFO:tasks.workunit.client.1.smithi115.stdout:7/553: write d3/da/d18/d73/d35/fad [73206,14865] 0 2021-11-03T17:15:21.509 INFO:tasks.workunit.client.1.smithi115.stdout:7/554: dread - d3/d4c/fa5 zero size 2021-11-03T17:15:21.539 INFO:tasks.workunit.client.1.smithi115.stdout:7/555: dwrite d3/d4c/f5b [0,4194304] 0 2021-11-03T17:15:21.540 INFO:tasks.workunit.client.1.smithi115.stdout:7/556: write d3/da/d18/d73/f2f [437814,38420] 0 2021-11-03T17:15:21.544 INFO:tasks.workunit.client.1.smithi115.stdout:6/502: sync 2021-11-03T17:15:21.547 INFO:tasks.workunit.client.1.smithi115.stdout:7/557: creat d3/da/d18/d73/d35/fae x:0 0 0 2021-11-03T17:15:21.547 INFO:tasks.workunit.client.1.smithi115.stdout:7/558: fdatasync d3/da/d18/d73/d52/f66 0 2021-11-03T17:15:21.547 INFO:tasks.workunit.client.1.smithi115.stdout:7/559: chown d3/da/d18/d73 2 1 2021-11-03T17:15:21.559 INFO:tasks.workunit.client.1.smithi115.stdout:7/560: rename d3/da/d18/f6d to d3/da/d18/faf 0 2021-11-03T17:15:21.559 INFO:tasks.workunit.client.1.smithi115.stdout:7/561: dread - d3/da/d18/d73/d52/f83 zero size 2021-11-03T17:15:21.563 INFO:tasks.workunit.client.1.smithi115.stdout:7/562: write d3/da/d8e/f93 [62957,129475] 0 2021-11-03T17:15:21.563 INFO:tasks.workunit.client.1.smithi115.stdout:7/563: write d3/da/d18/d73/d52/f66 [2393749,89303] 0 2021-11-03T17:15:21.563 INFO:tasks.workunit.client.1.smithi115.stdout:6/503: link d0/d1/d10/fa7 d0/d1/d10/d13/d87/d89/fa8 0 2021-11-03T17:15:21.563 INFO:tasks.workunit.client.1.smithi115.stdout:6/504: stat d0/d1/d10/d13/d59 0 2021-11-03T17:15:21.564 INFO:tasks.workunit.client.1.smithi115.stdout:6/505: dread d0/d1/d2/f4a [0,4194304] 0 2021-11-03T17:15:21.571 INFO:tasks.workunit.client.1.smithi115.stdout:6/506: creat d0/d7f/fa9 x:0 0 0 2021-11-03T17:15:21.592 INFO:tasks.workunit.client.1.smithi115.stdout:6/507: dwrite d0/d1/d10/d13/d87/fa4 [0,4194304] 0 2021-11-03T17:15:21.592 INFO:tasks.workunit.client.1.smithi115.stdout:7/564: dwrite d3/da/d18/faf [0,4194304] 0 2021-11-03T17:15:21.592 INFO:tasks.workunit.client.1.smithi115.stdout:7/565: dread - d3/da/d8e/f99 zero size 2021-11-03T17:15:21.595 INFO:tasks.workunit.client.1.smithi115.stdout:6/508: mknod d0/d1/d10/d13/caa 1 2021-11-03T17:15:21.600 INFO:tasks.workunit.client.1.smithi115.stdout:7/566: link d3/da/d18/d73/f2f d3/d4c/fb0 0 2021-11-03T17:15:21.601 INFO:tasks.workunit.client.1.smithi115.stdout:6/509: mkdir d0/d7f/dab 0 2021-11-03T17:15:21.606 INFO:tasks.workunit.client.1.smithi115.stdout:6/510: creat d0/d1/d10/d13/d87/d89/fac x:0 0 0 2021-11-03T17:15:21.611 INFO:tasks.workunit.client.1.smithi115.stdout:7/567: unlink d3/ld 0 2021-11-03T17:15:21.615 INFO:tasks.workunit.client.1.smithi115.stdout:7/568: truncate d3/da/d18/d73/d33/f59 419374 0 2021-11-03T17:15:21.621 INFO:tasks.workunit.client.1.smithi115.stdout:7/569: fsync d3/da/d18/d73/f2f 0 2021-11-03T17:15:21.630 INFO:tasks.workunit.client.1.smithi115.stdout:7/570: rmdir d3/da/d18/d9c 39 2021-11-03T17:15:21.643 INFO:tasks.workunit.client.1.smithi115.stdout:7/571: dwrite d3/da/d18/d73/d35/f75 [4194304,4194304] 0 2021-11-03T17:15:21.643 INFO:tasks.workunit.client.1.smithi115.stdout:7/572: stat d3/da/d18/d73/f89 0 2021-11-03T17:15:21.648 INFO:tasks.workunit.client.1.smithi115.stdout:7/573: rename d3/da/d18/d73/f2f to d3/da/d18/d9c/da1/fb1 0 2021-11-03T17:15:21.649 INFO:tasks.workunit.client.1.smithi115.stdout:7/574: chown d3/da/d18/d73/d33 23 1 2021-11-03T17:15:21.650 INFO:tasks.workunit.client.1.smithi115.stdout:7/575: write d3/f47 [38381,82400] 0 2021-11-03T17:15:21.650 INFO:tasks.workunit.client.1.smithi115.stdout:7/576: chown d3/d4c/f5b 1 1 2021-11-03T17:15:21.650 INFO:tasks.workunit.client.1.smithi115.stdout:7/577: write d3/da/d8e/f93 [21157,46314] 0 2021-11-03T17:15:21.655 INFO:tasks.workunit.client.1.smithi115.stdout:7/578: creat d3/da/d18/fb2 x:0 0 0 2021-11-03T17:15:21.661 INFO:tasks.workunit.client.1.smithi115.stdout:7/579: dread d3/f47 [0,4194304] 0 2021-11-03T17:15:21.668 INFO:tasks.workunit.client.1.smithi115.stdout:7/580: dread d3/da/d18/f40 [4194304,4194304] 0 2021-11-03T17:15:21.671 INFO:tasks.workunit.client.1.smithi115.stdout:7/581: rename d3/da/d8e/l9e to d3/d4c/da6/lb3 0 2021-11-03T17:15:21.675 INFO:tasks.workunit.client.1.smithi115.stdout:7/582: mkdir d3/da/d18/d73/d52/db4 0 2021-11-03T17:15:21.677 INFO:tasks.workunit.client.1.smithi115.stdout:7/583: fsync d3/f25 0 2021-11-03T17:15:21.680 INFO:tasks.workunit.client.1.smithi115.stdout:7/584: creat d3/da/d18/d73/d35/fb5 x:0 0 0 2021-11-03T17:15:21.681 INFO:tasks.workunit.client.1.smithi115.stdout:7/585: write d3/f47 [191297,30852] 0 2021-11-03T17:15:21.686 INFO:tasks.workunit.client.1.smithi115.stdout:7/586: symlink d3/da/d18/d9c/lb6 0 2021-11-03T17:15:21.689 INFO:tasks.workunit.client.1.smithi115.stdout:7/587: link d3/da/d18/d73/d35/fb5 d3/da/d18/d73/d35/d53/fb7 0 2021-11-03T17:15:21.694 INFO:tasks.workunit.client.1.smithi115.stdout:4/741: sync 2021-11-03T17:15:21.702 INFO:tasks.workunit.client.1.smithi115.stdout:4/742: creat d7/dc/d1a/d21/db1/ffc x:0 0 0 2021-11-03T17:15:21.712 INFO:tasks.workunit.client.1.smithi115.stdout:4/743: creat d7/dc/d6a/d3c/dc5/dcc/ffd x:0 0 0 2021-11-03T17:15:21.716 INFO:tasks.workunit.client.1.smithi115.stdout:4/744: chown d7/dc/d1a/d21/fbe 60924579 1 2021-11-03T17:15:21.721 INFO:tasks.workunit.client.1.smithi115.stdout:7/588: dwrite d3/da/d18/d73/d35/da3/fa8 [0,4194304] 0 2021-11-03T17:15:21.723 INFO:tasks.workunit.client.1.smithi115.stdout:4/745: link d7/dc/d1a/d21/db1/d56/d63/d9e/lf8 d7/dc/d6a/d3c/dc5/dcc/lfe 0 2021-11-03T17:15:21.736 INFO:tasks.workunit.client.1.smithi115.stdout:4/746: link d7/dc/d6a/d3c/dbf/fdb d7/dc/d1a/d21/db1/d56/d63/d7b/fff 0 2021-11-03T17:15:21.740 INFO:tasks.workunit.client.1.smithi115.stdout:7/589: dwrite d3/da/d8e/f99 [0,4194304] 0 2021-11-03T17:15:21.749 INFO:tasks.workunit.client.1.smithi115.stdout:7/590: rename d3/da/d18/d73/d35/lab to d3/da/d18/d9c/da1/lb8 0 2021-11-03T17:15:21.754 INFO:tasks.workunit.client.1.smithi115.stdout:7/591: dread d3/da/d18/f22 [0,4194304] 0 2021-11-03T17:15:21.755 INFO:tasks.workunit.client.1.smithi115.stdout:4/747: dwrite d7/dc/fb0 [0,4194304] 0 2021-11-03T17:15:21.771 INFO:tasks.workunit.client.1.smithi115.stdout:7/592: write d3/da/d18/d73/d52/f56 [2977982,93004] 0 2021-11-03T17:15:21.773 INFO:tasks.workunit.client.1.smithi115.stdout:4/748: write d7/dc/d6a/fac [421227,27598] 0 2021-11-03T17:15:21.776 INFO:tasks.workunit.client.1.smithi115.stdout:4/749: symlink d7/dc/d6a/d3c/dbf/dc4/dc6/l100 0 2021-11-03T17:15:21.782 INFO:tasks.workunit.client.1.smithi115.stdout:7/593: dread d3/f25 [0,4194304] 0 2021-11-03T17:15:21.788 INFO:tasks.workunit.client.1.smithi115.stdout:4/750: dread d7/dc/d1a/d21/db1/d56/f8d [0,4194304] 0 2021-11-03T17:15:21.809 INFO:tasks.workunit.client.1.smithi115.stdout:7/594: dwrite d3/da/d18/d73/d35/fb5 [0,4194304] 0 2021-11-03T17:15:21.834 INFO:tasks.workunit.client.1.smithi115.stdout:7/595: dwrite d3/da/d18/d9c/da1/fb1 [0,4194304] 0 2021-11-03T17:15:21.838 INFO:tasks.workunit.client.1.smithi115.stdout:7/596: creat d3/da/d18/d9c/da1/fb9 x:0 0 0 2021-11-03T17:15:21.839 INFO:tasks.workunit.client.1.smithi115.stdout:7/597: chown d3/da/d18/d73/d35/f3f 45468532 1 2021-11-03T17:15:21.847 INFO:tasks.workunit.client.1.smithi115.stdout:7/598: rename d3/da/d18/d73/d35/fae to d3/da/d78/fba 0 2021-11-03T17:15:21.848 INFO:tasks.workunit.client.1.smithi115.stdout:3/766: sync 2021-11-03T17:15:21.851 INFO:tasks.workunit.client.1.smithi115.stdout:4/751: dread d7/dc/f17 [0,4194304] 0 2021-11-03T17:15:21.851 INFO:tasks.workunit.client.1.smithi115.stdout:4/752: chown d7/dc/d1a/d21/db1/ffc 6419601 1 2021-11-03T17:15:21.854 INFO:tasks.workunit.client.1.smithi115.stdout:3/767: creat d13/d67/f105 x:0 0 0 2021-11-03T17:15:21.857 INFO:tasks.workunit.client.1.smithi115.stdout:4/753: stat d7/dc/d1a/d21/fa1 0 2021-11-03T17:15:21.858 INFO:tasks.workunit.client.1.smithi115.stdout:3/768: write d13/d35/f93 [1297803,30860] 0 2021-11-03T17:15:21.859 INFO:tasks.workunit.client.1.smithi115.stdout:4/754: mkdir d7/dc/d6a/d101 0 2021-11-03T17:15:21.860 INFO:tasks.workunit.client.1.smithi115.stdout:7/599: dread d3/da/d18/d9c/da1/fb1 [0,4194304] 0 2021-11-03T17:15:21.865 INFO:tasks.workunit.client.1.smithi115.stdout:4/755: mknod d7/dc/d1a/d21/db1/d56/c102 1 2021-11-03T17:15:21.865 INFO:tasks.workunit.client.1.smithi115.stdout:3/769: creat d13/d2a/d8a/f106 x:0 0 0 2021-11-03T17:15:21.868 INFO:tasks.workunit.client.1.smithi115.stdout:7/600: mkdir d3/da/d18/d73/d35/dbb 0 2021-11-03T17:15:21.873 INFO:tasks.workunit.client.1.smithi115.stdout:3/770: stat d13/d2a/l47 0 2021-11-03T17:15:21.877 INFO:tasks.workunit.client.1.smithi115.stdout:3/771: stat d13/l15 0 2021-11-03T17:15:21.880 INFO:tasks.workunit.client.1.smithi115.stdout:4/756: write d7/dc/f17 [9352538,25511] 0 2021-11-03T17:15:21.882 INFO:tasks.workunit.client.1.smithi115.stdout:4/757: getdents d7/dc/d1a/d21/db1/d56/d63 0 2021-11-03T17:15:21.883 INFO:tasks.workunit.client.1.smithi115.stdout:3/772: write d13/d2a/d8a/f94 [1762769,90281] 0 2021-11-03T17:15:21.885 INFO:tasks.workunit.client.1.smithi115.stdout:4/758: dread d7/dc/d1a/d21/db1/d56/d63/fa6 [0,4194304] 0 2021-11-03T17:15:21.885 INFO:tasks.workunit.client.1.smithi115.stdout:4/759: readlink d7/dc/d1a/d21/db1/d56/d63/d9e/lf8 0 2021-11-03T17:15:21.890 INFO:tasks.workunit.client.1.smithi115.stdout:4/760: link d7/l9 d7/dc/d6a/d3c/dc5/dcc/dd9/l103 0 2021-11-03T17:15:21.898 INFO:tasks.workunit.client.1.smithi115.stdout:7/601: dwrite d3/da/d18/d73/d33/f74 [0,4194304] 0 2021-11-03T17:15:21.899 INFO:tasks.workunit.client.1.smithi115.stdout:3/773: dread d13/f21 [0,4194304] 0 2021-11-03T17:15:21.910 INFO:tasks.workunit.client.1.smithi115.stdout:3/774: creat d13/d3b/db1/f107 x:0 0 0 2021-11-03T17:15:21.910 INFO:tasks.workunit.client.1.smithi115.stdout:7/602: rmdir d3/da/d18 39 2021-11-03T17:15:21.912 INFO:tasks.workunit.client.1.smithi115.stdout:3/775: mknod d13/d3b/db1/c108 1 2021-11-03T17:15:21.913 INFO:tasks.workunit.client.1.smithi115.stdout:7/603: mknod d3/da/d18/d73/d52/cbc 1 2021-11-03T17:15:21.913 INFO:tasks.workunit.client.1.smithi115.stdout:7/604: write d3/da/d18/d73/d35/f5f [4466374,50621] 0 2021-11-03T17:15:21.916 INFO:tasks.workunit.client.1.smithi115.stdout:7/605: symlink d3/da/d18/d73/d35/d53/lbd 0 2021-11-03T17:15:21.919 INFO:tasks.workunit.client.1.smithi115.stdout:3/776: rmdir d13/d51/d2e 39 2021-11-03T17:15:21.932 INFO:tasks.workunit.client.1.smithi115.stdout:3/777: dread d13/d51/d2e/f37 [0,4194304] 0 2021-11-03T17:15:21.935 INFO:tasks.workunit.client.1.smithi115.stdout:3/778: creat d13/d2a/d53/d70/d92/db7/f109 x:0 0 0 2021-11-03T17:15:21.936 INFO:tasks.workunit.client.1.smithi115.stdout:3/779: mknod d13/d50/d9e/c10a 1 2021-11-03T17:15:21.936 INFO:tasks.workunit.client.1.smithi115.stdout:3/780: write d13/d2a/d53/d70/d92/dc2/fdd [732562,32746] 0 2021-11-03T17:15:21.939 INFO:tasks.workunit.client.1.smithi115.stdout:3/781: fsync d13/d35/d54/dad/fb5 0 2021-11-03T17:15:21.942 INFO:tasks.workunit.client.1.smithi115.stdout:3/782: stat d13/d50/l6d 0 2021-11-03T17:15:21.964 INFO:tasks.workunit.client.1.smithi115.stdout:3/783: dread d13/f25 [0,4194304] 0 2021-11-03T17:15:21.965 INFO:tasks.workunit.client.1.smithi115.stdout:3/784: getdents d13/d50/df9 0 2021-11-03T17:15:21.965 INFO:tasks.workunit.client.1.smithi115.stdout:3/785: dread - d13/d35/fdb zero size 2021-11-03T17:15:21.965 INFO:tasks.workunit.client.1.smithi115.stdout:3/786: mkdir d13/d50/d10b 0 2021-11-03T17:15:21.973 INFO:tasks.workunit.client.1.smithi115.stdout:4/761: write d7/fb5 [233511,35307] 0 2021-11-03T17:15:21.973 INFO:tasks.workunit.client.1.smithi115.stdout:4/762: chown d7/dc/d6a/d3c/dc5/dcc/ffd 10757410 1 2021-11-03T17:15:21.976 INFO:tasks.workunit.client.1.smithi115.stdout:3/787: unlink d13/d51/d40/f4f 0 2021-11-03T17:15:21.978 INFO:tasks.workunit.client.1.smithi115.stdout:4/763: creat d7/dc/d6a/d3c/dc5/dcc/f104 x:0 0 0 2021-11-03T17:15:21.998 INFO:tasks.workunit.client.1.smithi115.stdout:3/788: mknod d13/d67/c10c 1 2021-11-03T17:15:21.999 INFO:tasks.workunit.client.1.smithi115.stdout:4/764: creat d7/dc/d1a/d21/f105 x:0 0 0 2021-11-03T17:15:22.001 INFO:tasks.workunit.client.1.smithi115.stdout:3/789: readlink d13/l1a 0 2021-11-03T17:15:22.006 INFO:tasks.workunit.client.1.smithi115.stdout:4/765: read d7/dc/d1a/d21/db1/d56/d63/d7b/fb8 [128003,8603] 0 2021-11-03T17:15:22.021 INFO:tasks.workunit.client.1.smithi115.stdout:4/766: dwrite d7/d7a/fea [0,4194304] 0 2021-11-03T17:15:22.021 INFO:tasks.workunit.client.1.smithi115.stdout:4/767: getdents d7/dc/d1a/d21/db1/d56/d63 0 2021-11-03T17:15:22.054 INFO:tasks.workunit.client.1.smithi115.stdout:4/768: dread d7/dc/d6a/d72/f76 [0,4194304] 0 2021-11-03T17:15:22.054 INFO:tasks.workunit.client.1.smithi115.stdout:4/769: fdatasync d7/dc/d1a/f49 0 2021-11-03T17:15:22.074 INFO:tasks.workunit.client.1.smithi115.stdout:4/770: dread d7/d7a/d83/f64 [0,4194304] 0 2021-11-03T17:15:22.074 INFO:tasks.workunit.client.1.smithi115.stdout:4/771: fdatasync d7/d7a/fea 0 2021-11-03T17:15:22.074 INFO:tasks.workunit.client.1.smithi115.stdout:4/772: chown d7/dc/d1a/d21/lfb 152378691 1 2021-11-03T17:15:22.074 INFO:tasks.workunit.client.1.smithi115.stdout:4/773: write d7/dc/d6a/fd3 [409761,90505] 0 2021-11-03T17:15:22.075 INFO:tasks.workunit.client.1.smithi115.stdout:4/774: fsync d7/dc/fe9 0 2021-11-03T17:15:22.080 INFO:tasks.workunit.client.1.smithi115.stdout:9/621: sync 2021-11-03T17:15:22.109 INFO:tasks.workunit.client.1.smithi115.stdout:4/775: write d7/dc/d1a/f49 [71277,109668] 0 2021-11-03T17:15:22.112 INFO:tasks.workunit.client.1.smithi115.stdout:4/776: symlink d7/dc/d1a/l106 0 2021-11-03T17:15:22.117 INFO:tasks.workunit.client.1.smithi115.stdout:4/777: creat d7/dc/d1a/d21/db1/d56/d63/f107 x:0 0 0 2021-11-03T17:15:22.118 INFO:tasks.workunit.client.1.smithi115.stdout:9/622: write d1/d18/d65/faf [1011900,117624] 0 2021-11-03T17:15:22.131 INFO:tasks.workunit.client.1.smithi115.stdout:9/623: dread d1/d1a/d48/d27/d2f/f8f [0,4194304] 0 2021-11-03T17:15:22.131 INFO:tasks.workunit.client.1.smithi115.stdout:4/778: symlink d7/dc/d6a/d3c/dbf/l108 0 2021-11-03T17:15:22.132 INFO:tasks.workunit.client.1.smithi115.stdout:4/779: chown d7/dc/d6a/f46 40447453 1 2021-11-03T17:15:22.136 INFO:tasks.workunit.client.1.smithi115.stdout:9/624: rmdir d1/d66 39 2021-11-03T17:15:22.140 INFO:tasks.workunit.client.1.smithi115.stdout:4/780: rename d7/d7a/fea to d7/dc/d1a/d21/db1/d56/d42/f109 0 2021-11-03T17:15:22.140 INFO:tasks.workunit.client.1.smithi115.stdout:4/781: stat d7/dc/d6a/d3c/dbf/dc4 0 2021-11-03T17:15:22.145 INFO:tasks.workunit.client.1.smithi115.stdout:4/782: creat d7/dc/d6a/d101/f10a x:0 0 0 2021-11-03T17:15:22.152 INFO:tasks.workunit.client.1.smithi115.stdout:4/783: read d7/dc/d1a/d21/db1/d56/d63/f6e [2961339,42060] 0 2021-11-03T17:15:22.154 INFO:tasks.workunit.client.1.smithi115.stdout:4/784: symlink d7/dc/d6a/dab/ddd/l10b 0 2021-11-03T17:15:22.165 INFO:tasks.workunit.client.1.smithi115.stdout:9/625: write d1/d18/d4a/d74/d8b/faa [3177877,1458] 0 2021-11-03T17:15:22.168 INFO:tasks.workunit.client.1.smithi115.stdout:4/785: symlink d7/l10c 0 2021-11-03T17:15:22.171 INFO:tasks.workunit.client.1.smithi115.stdout:9/626: readlink d1/d1a/d48/d27/d53/d83/d88/d9e/la8 0 2021-11-03T17:15:22.175 INFO:tasks.workunit.client.1.smithi115.stdout:4/786: dread d7/dc/d1a/d21/db1/d56/d63/d7b/d96/fa7 [0,4194304] 0 2021-11-03T17:15:22.175 INFO:tasks.workunit.client.1.smithi115.stdout:4/787: chown d7/dc/d6a/d3c/dc5/dcc/lfe 1 1 2021-11-03T17:15:22.176 INFO:tasks.workunit.client.1.smithi115.stdout:9/627: creat d1/d1a/d48/d27/d2f/d41/fe5 x:0 0 0 2021-11-03T17:15:22.176 INFO:tasks.workunit.client.1.smithi115.stdout:9/628: stat d1/d18/d4a/d74 0 2021-11-03T17:15:22.178 INFO:tasks.workunit.client.1.smithi115.stdout:9/629: read d1/d18/d2b/d42/ff [4284563,17185] 0 2021-11-03T17:15:22.190 INFO:tasks.workunit.client.1.smithi115.stdout:4/788: unlink d7/dc/d6a/dab/lce 0 2021-11-03T17:15:22.194 INFO:tasks.workunit.client.1.smithi115.stdout:9/630: mkdir d1/d1a/d48/d27/d2f/d7b/de6 0 2021-11-03T17:15:22.202 INFO:tasks.workunit.client.1.smithi115.stdout:9/631: creat d1/d18/d2b/d45/dc2/fe7 x:0 0 0 2021-11-03T17:15:22.212 INFO:tasks.workunit.client.1.smithi115.stdout:9/632: dread d1/d1a/d48/f73 [0,4194304] 0 2021-11-03T17:15:22.216 INFO:tasks.workunit.client.1.smithi115.stdout:9/633: rmdir d1/d1a/d48/d27 39 2021-11-03T17:15:22.217 INFO:tasks.workunit.client.1.smithi115.stdout:9/634: write d1/d18/d4a/d74/fdd [238359,23998] 0 2021-11-03T17:15:22.218 INFO:tasks.workunit.client.1.smithi115.stdout:4/789: dwrite d7/dc/d6a/d72/f76 [4194304,4194304] 0 2021-11-03T17:15:22.219 INFO:tasks.workunit.client.1.smithi115.stdout:4/790: stat d7/dd1/dee 0 2021-11-03T17:15:22.224 INFO:tasks.workunit.client.1.smithi115.stdout:4/791: creat d7/dc/f10d x:0 0 0 2021-11-03T17:15:22.237 INFO:tasks.workunit.client.1.smithi115.stdout:9/635: write d1/d1a/d48/d27/d53/d83/d88/dcd/fba [238861,34612] 0 2021-11-03T17:15:22.238 INFO:tasks.workunit.client.1.smithi115.stdout:4/792: mkdir d7/dc/d6a/d3c/d10e 0 2021-11-03T17:15:22.239 INFO:tasks.workunit.client.1.smithi115.stdout:4/793: fsync d7/d7a/ff2 0 2021-11-03T17:15:22.245 INFO:tasks.workunit.client.1.smithi115.stdout:9/636: mkdir d1/d18/d2b/de8 0 2021-11-03T17:15:22.246 INFO:tasks.workunit.client.1.smithi115.stdout:4/794: creat d7/dc/d1a/d21/db1/d56/db3/f10f x:0 0 0 2021-11-03T17:15:22.251 INFO:tasks.workunit.client.1.smithi115.stdout:4/795: creat d7/dc/db6/de1/f110 x:0 0 0 2021-11-03T17:15:22.261 INFO:tasks.workunit.client.1.smithi115.stdout:9/637: write d1/d18/d4a/d74/f7e [13904,127994] 0 2021-11-03T17:15:22.263 INFO:tasks.workunit.client.1.smithi115.stdout:2/628: sync 2021-11-03T17:15:22.264 INFO:tasks.workunit.client.1.smithi115.stdout:1/696: sync 2021-11-03T17:15:22.265 INFO:tasks.workunit.client.1.smithi115.stdout:1/697: write d7/dc/dd/d3e/dd1/d4b/f72 [2517856,125256] 0 2021-11-03T17:15:22.266 INFO:tasks.workunit.client.1.smithi115.stdout:4/796: write d7/dc/fa4 [1290781,91160] 0 2021-11-03T17:15:22.268 INFO:tasks.workunit.client.1.smithi115.stdout:2/629: getdents d4/d6/d56/d6c/d34/d43/d63/d69 0 2021-11-03T17:15:22.271 INFO:tasks.workunit.client.1.smithi115.stdout:4/797: creat d7/dc/d1a/d21/db1/d56/d63/f111 x:0 0 0 2021-11-03T17:15:22.271 INFO:tasks.workunit.client.1.smithi115.stdout:4/798: write d7/dc/d6a/dab/ddd/fef [159211,96685] 0 2021-11-03T17:15:22.273 INFO:tasks.workunit.client.1.smithi115.stdout:9/638: unlink d1/l67 0 2021-11-03T17:15:22.276 INFO:tasks.workunit.client.1.smithi115.stdout:1/698: dread d7/dc/dd/d3e/dd1/d43/d78/fb1 [0,4194304] 0 2021-11-03T17:15:22.281 INFO:tasks.workunit.client.1.smithi115.stdout:9/639: dread d1/d1a/d48/d27/d53/d83/d88/d9e/d58/fa4 [4194304,4194304] 0 2021-11-03T17:15:22.284 INFO:tasks.workunit.client.1.smithi115.stdout:4/799: rename d7/dc/d6a/d3c/dbf to d7/dc/d6a/d3c/d112 0 2021-11-03T17:15:22.284 INFO:tasks.workunit.client.1.smithi115.stdout:4/800: write d7/dc/f10d [88536,73336] 0 2021-11-03T17:15:22.285 INFO:tasks.workunit.client.1.smithi115.stdout:1/699: read d7/dc/f15 [5493017,68929] 0 2021-11-03T17:15:22.288 INFO:tasks.workunit.client.1.smithi115.stdout:9/640: symlink d1/d18/d2b/de8/le9 0 2021-11-03T17:15:22.291 INFO:tasks.workunit.client.1.smithi115.stdout:2/630: dwrite d4/d6/d56/d6c/d34/d42/d8d/fce [0,4194304] 0 2021-11-03T17:15:22.291 INFO:tasks.workunit.client.1.smithi115.stdout:2/631: dread - d4/d6/dad/d7f/f8e zero size 2021-11-03T17:15:22.298 INFO:tasks.workunit.client.1.smithi115.stdout:1/700: creat d7/dc/dd/db4/fe0 x:0 0 0 2021-11-03T17:15:22.298 INFO:tasks.workunit.client.1.smithi115.stdout:9/641: read - d1/d1a/d48/d27/d2f/d7b/fe3 zero size 2021-11-03T17:15:22.341 INFO:tasks.workunit.client.1.smithi115.stdout:9/642: rename d1/d18/d65/fca to d1/d1a/d48/d27/d2f/d41/fea 0 2021-11-03T17:15:22.354 INFO:tasks.workunit.client.1.smithi115.stdout:2/632: rename d4/d6/d13/l94 to d4/d6/d13/ld9 0 2021-11-03T17:15:22.358 INFO:tasks.workunit.client.1.smithi115.stdout:1/701: creat d7/d38/d84/dca/fe1 x:0 0 0 2021-11-03T17:15:22.369 INFO:tasks.workunit.client.1.smithi115.stdout:2/633: creat d4/d6/d13/d75/fda x:0 0 0 2021-11-03T17:15:22.369 INFO:tasks.workunit.client.1.smithi115.stdout:2/634: fdatasync d4/d6/d13/d75/f7b 0 2021-11-03T17:15:22.370 INFO:tasks.workunit.client.1.smithi115.stdout:9/643: dread d1/d18/d2b/d42/d63/f9f [0,4194304] 0 2021-11-03T17:15:22.370 INFO:tasks.workunit.client.1.smithi115.stdout:9/644: chown d1/d1a/d48/d27/d53/d83/d88/d9e/la8 1054280 1 2021-11-03T17:15:22.373 INFO:tasks.workunit.client.1.smithi115.stdout:5/733: sync 2021-11-03T17:15:22.380 INFO:tasks.workunit.client.1.smithi115.stdout:5/734: mkdir d1a/d28/d4c/d75/d83/da1/df5 0 2021-11-03T17:15:22.381 INFO:tasks.workunit.client.1.smithi115.stdout:9/645: dread d1/d18/d4a/d74/dc7/fce [0,4194304] 0 2021-11-03T17:15:22.384 INFO:tasks.workunit.client.1.smithi115.stdout:9/646: mknod d1/d18/d65/ceb 1 2021-11-03T17:15:22.387 INFO:tasks.workunit.client.1.smithi115.stdout:9/647: creat d1/d18/d4a/fec x:0 0 0 2021-11-03T17:15:22.391 INFO:tasks.workunit.client.1.smithi115.stdout:1/702: dwrite d7/dc/f16 [0,4194304] 0 2021-11-03T17:15:22.400 INFO:tasks.workunit.client.1.smithi115.stdout:5/735: write d1a/f1d [1445334,16152] 0 2021-11-03T17:15:22.406 INFO:tasks.workunit.client.1.smithi115.stdout:5/736: creat d1a/d28/d4a/ff6 x:0 0 0 2021-11-03T17:15:22.407 INFO:tasks.workunit.client.1.smithi115.stdout:5/737: chown d1a/d28/d4c/d75/dae/lcc 7177965 1 2021-11-03T17:15:22.410 INFO:tasks.workunit.client.1.smithi115.stdout:5/738: creat d1a/d28/d34/ff7 x:0 0 0 2021-11-03T17:15:22.410 INFO:tasks.workunit.client.1.smithi115.stdout:5/739: write d1a/d3d/dc0/ddf/ff3 [749415,106733] 0 2021-11-03T17:15:22.413 INFO:tasks.workunit.client.1.smithi115.stdout:1/703: dwrite d7/dc/dd/d3e/dd1/d43/f6f [4194304,4194304] 0 2021-11-03T17:15:22.414 INFO:tasks.workunit.client.1.smithi115.stdout:1/704: chown d7/d38/d84/dca/fe1 263253405 1 2021-11-03T17:15:22.415 INFO:tasks.workunit.client.1.smithi115.stdout:1/705: write d7/dc/f16 [1590599,4690] 0 2021-11-03T17:15:22.416 INFO:tasks.workunit.client.1.smithi115.stdout:5/740: creat d1a/d3d/d58/dab/daf/ff8 x:0 0 0 2021-11-03T17:15:22.421 INFO:tasks.workunit.client.1.smithi115.stdout:1/706: mknod d7/dc/dd/d3e/dd1/d43/d78/ce2 1 2021-11-03T17:15:22.430 INFO:tasks.workunit.client.1.smithi115.stdout:5/741: rename d1a/d28/d4c/d75/d68/da5/da0 to d1a/d28/df9 0 2021-11-03T17:15:22.439 INFO:tasks.workunit.client.1.smithi115.stdout:1/707: write d7/d9d/fda [382613,65789] 0 2021-11-03T17:15:22.446 INFO:tasks.workunit.client.1.smithi115.stdout:0/855: sync 2021-11-03T17:15:22.446 INFO:tasks.workunit.client.1.smithi115.stdout:8/603: sync 2021-11-03T17:15:22.446 INFO:tasks.workunit.client.1.smithi115.stdout:0/856: dread - d9/d15/f53 zero size 2021-11-03T17:15:22.450 INFO:tasks.workunit.client.1.smithi115.stdout:1/708: rename d7/dc/dd/d3e to d7/dc/dd/d3e/dd1/d4d/d60/de3 22 2021-11-03T17:15:22.454 INFO:tasks.workunit.client.1.smithi115.stdout:5/742: symlink d1a/lfa 0 2021-11-03T17:15:22.458 INFO:tasks.workunit.client.1.smithi115.stdout:8/604: creat d4/d6/fd2 x:0 0 0 2021-11-03T17:15:22.461 INFO:tasks.workunit.client.1.smithi115.stdout:1/709: mknod d7/d38/ce4 1 2021-11-03T17:15:22.462 INFO:tasks.workunit.client.1.smithi115.stdout:5/743: truncate d1a/d28/d7b/d48/dd4/fd7 4156260 0 2021-11-03T17:15:22.463 INFO:tasks.workunit.client.1.smithi115.stdout:5/744: fsync d1a/d3d/dc4/fe9 0 2021-11-03T17:15:22.463 INFO:tasks.workunit.client.1.smithi115.stdout:5/745: write d1a/d28/d7b/d48/fca [11991,32267] 0 2021-11-03T17:15:22.463 INFO:tasks.workunit.client.1.smithi115.stdout:5/746: write d1a/d28/d4c/d75/dae/fce [567251,107599] 0 2021-11-03T17:15:22.467 INFO:tasks.workunit.client.1.smithi115.stdout:8/605: mknod d4/d6/d18/d30/d79/cd3 1 2021-11-03T17:15:22.472 INFO:tasks.workunit.client.1.smithi115.stdout:8/606: mknod d4/d6/d18/d30/d33/d93/cd4 1 2021-11-03T17:15:22.473 INFO:tasks.workunit.client.1.smithi115.stdout:5/747: dread d1a/d28/f2c [0,4194304] 0 2021-11-03T17:15:22.474 INFO:tasks.workunit.client.1.smithi115.stdout:1/710: dread d7/dc/dd/d3e/d58/dbf/fd8 [0,4194304] 0 2021-11-03T17:15:22.484 INFO:tasks.workunit.client.1.smithi115.stdout:8/607: read d4/d6/d18/fa5 [1400628,89628] 0 2021-11-03T17:15:22.485 INFO:tasks.workunit.client.1.smithi115.stdout:5/748: rename d1a/d28/l59 to d1a/d28/d34/deb/lfb 0 2021-11-03T17:15:22.489 INFO:tasks.workunit.client.1.smithi115.stdout:8/608: mknod d4/cd5 1 2021-11-03T17:15:22.490 INFO:tasks.workunit.client.1.smithi115.stdout:5/749: mknod d1a/d28/d4c/d75/dae/cfc 1 2021-11-03T17:15:22.496 INFO:tasks.workunit.client.1.smithi115.stdout:1/711: write d7/dc/d1a/f6b [1753059,39266] 0 2021-11-03T17:15:22.497 INFO:tasks.workunit.client.1.smithi115.stdout:5/750: mknod d1a/d28/cfd 1 2021-11-03T17:15:22.506 INFO:tasks.workunit.client.1.smithi115.stdout:5/751: symlink d1a/d28/d4a/lfe 0 2021-11-03T17:15:22.509 INFO:tasks.workunit.client.1.smithi115.stdout:5/752: readlink d1a/d28/d7b/d48/d7f/lbf 0 2021-11-03T17:15:22.512 INFO:tasks.workunit.client.1.smithi115.stdout:5/753: unlink d1a/d28/daa/le5 0 2021-11-03T17:15:22.531 INFO:tasks.workunit.client.1.smithi115.stdout:8/609: dwrite d4/d6/d18/d9d/fb6 [0,4194304] 0 2021-11-03T17:15:22.552 INFO:tasks.workunit.client.1.smithi115.stdout:8/610: write d4/d6/d18/d30/d79/d7f/fbb [1209509,78948] 0 2021-11-03T17:15:22.573 INFO:tasks.workunit.client.1.smithi115.stdout:8/611: dwrite d4/d6/f8a [0,4194304] 0 2021-11-03T17:15:22.588 INFO:tasks.workunit.client.1.smithi115.stdout:8/612: unlink d4/d6/d3f/d9a/l53 0 2021-11-03T17:15:22.590 INFO:tasks.workunit.client.1.smithi115.stdout:8/613: creat d4/d6/d3f/d9a/fd6 x:0 0 0 2021-11-03T17:15:22.592 INFO:tasks.workunit.client.1.smithi115.stdout:8/614: getdents d4 0 2021-11-03T17:15:22.595 INFO:tasks.workunit.client.1.smithi115.stdout:8/615: dread d4/fa [0,4194304] 0 2021-11-03T17:15:22.602 INFO:tasks.workunit.client.1.smithi115.stdout:8/616: symlink d4/d6/d18/d30/d33/ld7 0 2021-11-03T17:15:22.604 INFO:tasks.workunit.client.1.smithi115.stdout:8/617: chown d4/d6/f1d 10742503 1 2021-11-03T17:15:22.606 INFO:tasks.workunit.client.1.smithi115.stdout:8/618: fsync d4/d6/d18/f63 0 2021-11-03T17:15:22.610 INFO:tasks.workunit.client.1.smithi115.stdout:8/619: chown d4/d6/d18/d30/d33/l40 312331 1 2021-11-03T17:15:22.614 INFO:tasks.workunit.client.1.smithi115.stdout:8/620: truncate d4/d6/f3c 7862331 0 2021-11-03T17:15:22.624 INFO:tasks.workunit.client.1.smithi115.stdout:8/621: creat d4/d6/d18/d44/d47/fd8 x:0 0 0 2021-11-03T17:15:22.634 INFO:tasks.workunit.client.1.smithi115.stdout:8/622: rename d4/l91 to d4/d6/d18/d44/d47/ld9 0 2021-11-03T17:15:22.636 INFO:tasks.workunit.client.1.smithi115.stdout:8/623: symlink d4/d6/d18/d44/dcd/lda 0 2021-11-03T17:15:22.637 INFO:tasks.workunit.client.1.smithi115.stdout:8/624: creat d4/d6/d3f/da4/fdb x:0 0 0 2021-11-03T17:15:22.642 INFO:tasks.workunit.client.1.smithi115.stdout:8/625: creat d4/d6/d3f/d43/d88/fdc x:0 0 0 2021-11-03T17:15:22.644 INFO:tasks.workunit.client.1.smithi115.stdout:8/626: creat d4/d7d/fdd x:0 0 0 2021-11-03T17:15:22.653 INFO:tasks.workunit.client.1.smithi115.stdout:8/627: mkdir d4/d6/d3f/dde 0 2021-11-03T17:15:22.658 INFO:tasks.workunit.client.1.smithi115.stdout:8/628: symlink d4/d6/d18/d30/d79/d7f/ldf 0 2021-11-03T17:15:22.661 INFO:tasks.workunit.client.1.smithi115.stdout:8/629: truncate d4/d7d/fa0 2045715 0 2021-11-03T17:15:22.679 INFO:tasks.workunit.client.1.smithi115.stdout:8/630: dwrite d4/d6/d18/f63 [0,4194304] 0 2021-11-03T17:15:22.681 INFO:tasks.workunit.client.1.smithi115.stdout:8/631: chown d4/d6/d18/d30/d79/l97 3798065 1 2021-11-03T17:15:22.684 INFO:tasks.workunit.client.1.smithi115.stdout:8/632: symlink d4/d6/d3f/da4/le0 0 2021-11-03T17:15:22.689 INFO:tasks.workunit.client.1.smithi115.stdout:8/633: dread - d4/d6/d18/d44/fa3 zero size 2021-11-03T17:15:22.697 INFO:tasks.workunit.client.1.smithi115.stdout:8/634: rename d4/d6/d18/l8d to d4/d6/d18/le1 0 2021-11-03T17:15:22.712 INFO:tasks.workunit.client.1.smithi115.stdout:8/635: dwrite d4/d6/f8a [0,4194304] 0 2021-11-03T17:15:22.712 INFO:tasks.workunit.client.1.smithi115.stdout:8/636: mknod d4/d6/d18/d44/ce2 1 2021-11-03T17:15:22.718 INFO:tasks.workunit.client.1.smithi115.stdout:8/637: mkdir d4/d6/d3f/d43/d88/de3 0 2021-11-03T17:15:22.721 INFO:tasks.workunit.client.1.smithi115.stdout:8/638: mknod d4/d6/d18/d44/d47/ce4 1 2021-11-03T17:15:22.724 INFO:tasks.workunit.client.1.smithi115.stdout:8/639: creat d4/d6/d3f/d43/d88/de3/fe5 x:0 0 0 2021-11-03T17:15:22.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:22 smithi115 conmon[34670]: cluster 2021-11-03T17:15: 2021-11-03T17:15:22.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:22 smithi115 conmon[34670]: 20.645228+0000 mgr.smithi027.ufhhvb (mgr.14622) 25 : cluster [DBG] pgmap v14: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 16 MiB/s rd, 27 MiB/s wr, 681 op/s 2021-11-03T17:15:22.736 INFO:tasks.workunit.client.1.smithi115.stdout:8/640: dread d4/d6/d18/d30/d79/fb2 [0,4194304] 0 2021-11-03T17:15:22.749 INFO:tasks.workunit.client.1.smithi115.stdout:8/641: dread d4/d6/d18/f23 [8388608,4194304] 0 2021-11-03T17:15:22.749 INFO:tasks.workunit.client.1.smithi115.stdout:8/642: chown d4/d6/d3f/d43/d88/de3/fe5 1 1 2021-11-03T17:15:22.750 INFO:tasks.workunit.client.1.smithi115.stdout:8/643: creat d4/d7d/fe6 x:0 0 0 2021-11-03T17:15:22.752 INFO:tasks.workunit.client.1.smithi115.stdout:8/644: dread d4/d6/d3f/d9a/f65 [0,4194304] 0 2021-11-03T17:15:22.755 INFO:tasks.workunit.client.1.smithi115.stdout:8/645: mkdir d4/d6/d18/d30/d79/de7 0 2021-11-03T17:15:22.757 INFO:tasks.workunit.client.1.smithi115.stdout:8/646: creat d4/d6/d3f/dde/fe8 x:0 0 0 2021-11-03T17:15:22.761 INFO:tasks.workunit.client.1.smithi115.stdout:8/647: stat d4/d6/f2a 0 2021-11-03T17:15:22.764 INFO:tasks.workunit.client.1.smithi115.stdout:8/648: unlink d4/d6/d18/d44/fca 0 2021-11-03T17:15:22.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:22 smithi027 conmon[32322]: cluster 2021-11-03T17:15:20.645228+0000 mgr.smithi027.ufhhvb 2021-11-03T17:15:22.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:22 smithi027 conmon[32322]: (mgr.14622) 25 : cluster [DBG] pgmap v14: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 16 MiB/s rd, 27 MiB/s wr, 681 op/s 2021-11-03T17:15:22.968 INFO:tasks.workunit.client.1.smithi115.stdout:6/511: sync 2021-11-03T17:15:22.968 INFO:tasks.workunit.client.1.smithi115.stdout:6/512: write d0/d1/d10/fa7 [1027906,19119] 0 2021-11-03T17:15:22.969 INFO:tasks.workunit.client.1.smithi115.stdout:6/513: write d0/d1/d2/ff [2337560,95684] 0 2021-11-03T17:15:22.983 INFO:tasks.workunit.client.1.smithi115.stdout:6/514: unlink d0/d1/d10/f53 0 2021-11-03T17:15:22.983 INFO:tasks.workunit.client.1.smithi115.stdout:6/515: symlink d0/d1/d10/d13/d1c/lad 0 2021-11-03T17:15:22.984 INFO:tasks.workunit.client.1.smithi115.stdout:6/516: chown d0/d1/d10/d13/d59/f64 115437223 1 2021-11-03T17:15:22.984 INFO:tasks.workunit.client.1.smithi115.stdout:6/517: mkdir d0/d74/dae 0 2021-11-03T17:15:22.989 INFO:tasks.workunit.client.1.smithi115.stdout:6/518: link d0/f4d d0/d1/d10/d13/d87/d89/faf 0 2021-11-03T17:15:22.991 INFO:tasks.workunit.client.1.smithi115.stdout:6/519: mkdir d0/d1/d6a/d88/da0/db0 0 2021-11-03T17:15:22.991 INFO:tasks.workunit.client.1.smithi115.stdout:6/520: stat d0/d1/d10/d13/d87 0 2021-11-03T17:15:22.995 INFO:tasks.workunit.client.1.smithi115.stdout:6/521: symlink d0/d1/d2/d3f/lb1 0 2021-11-03T17:15:22.997 INFO:tasks.workunit.client.1.smithi115.stdout:6/522: rmdir d0/d1/d2 39 2021-11-03T17:15:22.998 INFO:tasks.workunit.client.1.smithi115.stdout:6/523: write d0/f22 [2635629,61060] 0 2021-11-03T17:15:22.998 INFO:tasks.workunit.client.1.smithi115.stdout:6/524: truncate d0/d1/d10/fa7 2068562 0 2021-11-03T17:15:23.002 INFO:tasks.workunit.client.1.smithi115.stdout:6/525: mkdir d0/d74/db2 0 2021-11-03T17:15:23.025 INFO:tasks.workunit.client.1.smithi115.stdout:6/526: dwrite d0/fa2 [0,4194304] 0 2021-11-03T17:15:23.057 INFO:tasks.workunit.client.1.smithi115.stdout:6/527: truncate d0/f2a 3021937 0 2021-11-03T17:15:23.063 INFO:tasks.workunit.client.1.smithi115.stdout:6/528: readlink d0/d1/d10/d13/d1c/d1e/l3c 0 2021-11-03T17:15:23.068 INFO:tasks.workunit.client.1.smithi115.stdout:6/529: mkdir d0/d74/dae/db3 0 2021-11-03T17:15:23.079 INFO:tasks.workunit.client.1.smithi115.stdout:6/530: mkdir d0/d1/d10/d13/d59/d95/db4 0 2021-11-03T17:15:23.135 INFO:tasks.workunit.client.1.smithi115.stdout:6/531: dwrite d0/d1/d10/d13/d87/d89/faf [0,4194304] 0 2021-11-03T17:15:23.139 INFO:tasks.workunit.client.1.smithi115.stdout:6/532: rename d0/d1/d6a/d88/da0/db0 to d0/d1/d6a/d88/da0/db5 0 2021-11-03T17:15:23.184 INFO:tasks.workunit.client.1.smithi115.stdout:6/533: read d0/d1/d2/f35 [393249,126040] 0 2021-11-03T17:15:23.186 INFO:tasks.workunit.client.1.smithi115.stdout:6/534: unlink d0/d1/d2e/d4c/l84 0 2021-11-03T17:15:23.198 INFO:tasks.workunit.client.1.smithi115.stdout:6/535: dread d0/d1/d2e/f31 [0,4194304] 0 2021-11-03T17:15:23.201 INFO:tasks.workunit.client.1.smithi115.stdout:6/536: creat d0/d1/d2e/d4c/fb6 x:0 0 0 2021-11-03T17:15:23.201 INFO:tasks.workunit.client.1.smithi115.stdout:6/537: mknod d0/d1/d6a/d88/da0/db5/cb7 1 2021-11-03T17:15:23.203 INFO:tasks.workunit.client.1.smithi115.stdout:6/538: mknod d0/d1/d10/d13/d87/cb8 1 2021-11-03T17:15:23.203 INFO:tasks.workunit.client.1.smithi115.stdout:6/539: write d0/d1/d10/d13/d59/f9c [959535,54133] 0 2021-11-03T17:15:23.212 INFO:tasks.workunit.client.1.smithi115.stdout:6/540: link d0/f41 d0/d1/d2e/fb9 0 2021-11-03T17:15:23.220 INFO:tasks.workunit.client.1.smithi115.stdout:6/541: dread d0/d1/d10/d13/d87/fa4 [0,4194304] 0 2021-11-03T17:15:23.222 INFO:tasks.workunit.client.1.smithi115.stdout:6/542: creat d0/d1/fba x:0 0 0 2021-11-03T17:15:23.225 INFO:tasks.workunit.client.1.smithi115.stdout:6/543: creat d0/d1/d6a/d88/da0/db5/fbb x:0 0 0 2021-11-03T17:15:23.226 INFO:tasks.workunit.client.1.smithi115.stdout:6/544: chown d0/d1/d2/f35 9 1 2021-11-03T17:15:23.226 INFO:tasks.workunit.client.1.smithi115.stdout:6/545: write d0/d1/d10/d13/d1c/d1e/f91 [689865,29615] 0 2021-11-03T17:15:23.226 INFO:tasks.workunit.client.1.smithi115.stdout:6/546: dread - d0/d1/fba zero size 2021-11-03T17:15:23.235 INFO:tasks.workunit.client.1.smithi115.stdout:6/547: symlink d0/d1/d2/d3f/lbc 0 2021-11-03T17:15:23.236 INFO:tasks.workunit.client.1.smithi115.stdout:6/548: stat d0/d1/d10/d13 0 2021-11-03T17:15:23.245 INFO:tasks.workunit.client.1.smithi115.stdout:6/549: read d0/d1/d2/f1f [1896763,30765] 0 2021-11-03T17:15:23.276 INFO:tasks.workunit.client.1.smithi115.stdout:6/550: dwrite d0/d1/d10/d13/d87/d89/faf [0,4194304] 0 2021-11-03T17:15:23.284 INFO:tasks.workunit.client.1.smithi115.stdout:6/551: write d0/f40 [1549370,17605] 0 2021-11-03T17:15:23.284 INFO:tasks.workunit.client.1.smithi115.stdout:6/552: write d0/d1/d10/d13/d87/d89/fa8 [2586390,26539] 0 2021-11-03T17:15:23.285 INFO:tasks.workunit.client.1.smithi115.stdout:6/553: creat d0/d1/d6a/d88/da0/db5/fbd x:0 0 0 2021-11-03T17:15:23.287 INFO:tasks.workunit.client.1.smithi115.stdout:6/554: rmdir d0/d1/d2e 39 2021-11-03T17:15:23.287 INFO:tasks.workunit.client.1.smithi115.stdout:6/555: readlink d0/d1/d2/l7d 0 2021-11-03T17:15:23.289 INFO:tasks.workunit.client.1.smithi115.stdout:6/556: mknod d0/d1/d2e/d4c/cbe 1 2021-11-03T17:15:23.292 INFO:tasks.workunit.client.1.smithi115.stdout:6/557: fsync d0/d1/fb 0 2021-11-03T17:15:23.299 INFO:tasks.workunit.client.1.smithi115.stdout:6/558: unlink d0/d1/d10/d13/d59/l7a 0 2021-11-03T17:15:23.304 INFO:tasks.workunit.client.1.smithi115.stdout:6/559: readlink d0/d1/d10/l26 0 2021-11-03T17:15:23.307 INFO:tasks.workunit.client.1.smithi115.stdout:6/560: creat d0/d1/d10/d13/d1c/d1e/fbf x:0 0 0 2021-11-03T17:15:23.329 INFO:tasks.workunit.client.1.smithi115.stdout:6/561: dwrite d0/d1/d6a/d88/da0/db5/fbb [0,4194304] 0 2021-11-03T17:15:23.356 INFO:tasks.workunit.client.1.smithi115.stdout:6/562: creat d0/d1/d2/d3f/fc0 x:0 0 0 2021-11-03T17:15:23.360 INFO:tasks.workunit.client.1.smithi115.stdout:6/563: symlink d0/d1/d10/d13/d1c/lc1 0 2021-11-03T17:15:23.361 INFO:tasks.workunit.client.1.smithi115.stdout:6/564: stat d0/l5a 0 2021-11-03T17:15:23.363 INFO:tasks.workunit.client.1.smithi115.stdout:6/565: dread - d0/d1/d2e/d4c/d96/d61/f8a zero size 2021-11-03T17:15:23.364 INFO:tasks.workunit.client.1.smithi115.stdout:6/566: truncate d0/d1/d2/ff 4705766 0 2021-11-03T17:15:23.370 INFO:tasks.workunit.client.1.smithi115.stdout:6/567: rename d0/d1/d10/d13/d87/d89/fa8 to d0/fc2 0 2021-11-03T17:15:23.370 INFO:tasks.workunit.client.1.smithi115.stdout:6/568: write d0/d1/d10/d13/d87/fa4 [4326307,82134] 0 2021-11-03T17:15:23.371 INFO:tasks.workunit.client.1.smithi115.stdout:6/569: rmdir d0/d1/d6a/d88/da0 39 2021-11-03T17:15:23.386 INFO:tasks.workunit.client.1.smithi115.stdout:6/570: write d0/d1/d2/f58 [2859177,61113] 0 2021-11-03T17:15:23.408 INFO:tasks.workunit.client.1.smithi115.stdout:6/571: dwrite d0/d1/d6a/d88/da0/db5/fbb [0,4194304] 0 2021-11-03T17:15:23.412 INFO:tasks.workunit.client.1.smithi115.stdout:6/572: read d0/d1/d2e/fb9 [302888,72908] 0 2021-11-03T17:15:23.449 INFO:tasks.workunit.client.1.smithi115.stdout:6/573: dwrite d0/d1/d10/d13/f72 [0,4194304] 0 2021-11-03T17:15:23.452 INFO:tasks.workunit.client.1.smithi115.stdout:6/574: rename d0/d1/d2/f86 to d0/d1/d6a/d88/da0/db5/fc3 0 2021-11-03T17:15:23.453 INFO:tasks.workunit.client.1.smithi115.stdout:6/575: read d0/d1/fb [6425939,42794] 0 2021-11-03T17:15:23.454 INFO:tasks.workunit.client.1.smithi115.stdout:6/576: write d0/d1/d10/d13/d87/fa5 [3708328,110297] 0 2021-11-03T17:15:23.460 INFO:tasks.workunit.client.1.smithi115.stdout:6/577: rename d0/d1/d10/d13/l33 to d0/d1/d2/lc4 0 2021-11-03T17:15:23.467 INFO:tasks.workunit.client.1.smithi115.stdout:6/578: symlink d0/d1/d10/d13/d87/lc5 0 2021-11-03T17:15:23.476 INFO:tasks.workunit.client.1.smithi115.stdout:6/579: dread d0/d1/d10/fa7 [0,4194304] 0 2021-11-03T17:15:23.477 INFO:tasks.workunit.client.1.smithi115.stdout:6/580: chown d0/d1/d2e/f31 125539 1 2021-11-03T17:15:23.477 INFO:tasks.workunit.client.1.smithi115.stdout:6/581: chown d0/d1/d10/d13/d1c/lad 114 1 2021-11-03T17:15:23.493 INFO:tasks.workunit.client.1.smithi115.stdout:6/582: dwrite d0/f6 [4194304,4194304] 0 2021-11-03T17:15:23.500 INFO:tasks.workunit.client.1.smithi115.stdout:6/583: dread d0/d1/d10/d13/d1c/f21 [0,4194304] 0 2021-11-03T17:15:23.502 INFO:tasks.workunit.client.1.smithi115.stdout:6/584: mkdir d0/d7f/dab/dc6 0 2021-11-03T17:15:23.506 INFO:tasks.workunit.client.1.smithi115.stdout:6/585: creat d0/d1/d2e/d4c/fc7 x:0 0 0 2021-11-03T17:15:23.508 INFO:tasks.workunit.client.1.smithi115.stdout:6/586: dread d0/d1/d10/d13/d1c/f2f [4194304,4194304] 0 2021-11-03T17:15:23.528 INFO:tasks.workunit.client.1.smithi115.stdout:6/587: write d0/d1/d6a/d88/da0/db5/fc3 [4637821,66156] 0 2021-11-03T17:15:23.529 INFO:tasks.workunit.client.1.smithi115.stdout:6/588: readlink d0/d1/d10/d13/d1c/d1e/l75 0 2021-11-03T17:15:23.530 INFO:tasks.workunit.client.1.smithi115.stdout:6/589: read d0/d1/d10/d13/d87/fa5 [2581713,82941] 0 2021-11-03T17:15:23.541 INFO:tasks.workunit.client.1.smithi115.stdout:3/790: sync 2021-11-03T17:15:23.542 INFO:tasks.workunit.client.1.smithi115.stdout:7/606: sync 2021-11-03T17:15:23.542 INFO:tasks.workunit.client.1.smithi115.stdout:7/607: chown d3/da/d18/d73/d33/f74 41102 1 2021-11-03T17:15:23.543 INFO:tasks.workunit.client.1.smithi115.stdout:7/608: write d3/da/d18/d73/d33/f59 [91097,80600] 0 2021-11-03T17:15:23.545 INFO:tasks.workunit.client.1.smithi115.stdout:7/609: creat d3/da/d78/fbe x:0 0 0 2021-11-03T17:15:23.549 INFO:tasks.workunit.client.1.smithi115.stdout:3/791: truncate d13/d51/d40/f88 5406117 0 2021-11-03T17:15:23.559 INFO:tasks.workunit.client.1.smithi115.stdout:3/792: symlink d13/d35/d54/df6/l10d 0 2021-11-03T17:15:23.560 INFO:tasks.workunit.client.1.smithi115.stdout:3/793: write d13/d2a/d8a/fa7 [1618757,61051] 0 2021-11-03T17:15:23.560 INFO:tasks.workunit.client.1.smithi115.stdout:7/610: dwrite d3/da/d18/faf [0,4194304] 0 2021-11-03T17:15:23.570 INFO:tasks.workunit.client.1.smithi115.stdout:7/611: creat d3/d4c/da6/fbf x:0 0 0 2021-11-03T17:15:23.573 INFO:tasks.workunit.client.1.smithi115.stdout:3/794: rename d13/d3b/f76 to d13/d35/d54/df6/f10e 0 2021-11-03T17:15:23.575 INFO:tasks.workunit.client.1.smithi115.stdout:3/795: creat d13/d2a/f10f x:0 0 0 2021-11-03T17:15:23.608 INFO:tasks.workunit.client.1.smithi115.stdout:3/796: dwrite d13/d51/f55 [0,4194304] 0 2021-11-03T17:15:23.610 INFO:tasks.workunit.client.1.smithi115.stdout:7/612: dwrite d3/da/d18/d73/d35/f3f [0,4194304] 0 2021-11-03T17:15:23.618 INFO:tasks.workunit.client.1.smithi115.stdout:7/613: mkdir d3/da/d18/dc0 0 2021-11-03T17:15:23.619 INFO:tasks.workunit.client.1.smithi115.stdout:7/614: read d3/d4c/fb0 [466051,15800] 0 2021-11-03T17:15:23.619 INFO:tasks.workunit.client.1.smithi115.stdout:7/615: truncate d3/d4c/f5b 4650647 0 2021-11-03T17:15:23.627 INFO:tasks.workunit.client.1.smithi115.stdout:3/797: creat d13/d51/d2e/dbf/f110 x:0 0 0 2021-11-03T17:15:23.628 INFO:tasks.workunit.client.1.smithi115.stdout:3/798: chown d13/d35/d54/dad/f100 367451 1 2021-11-03T17:15:23.628 INFO:tasks.workunit.client.1.smithi115.stdout:3/799: fsync d13/d2a/d53/d70/d92/fe4 0 2021-11-03T17:15:23.630 INFO:tasks.workunit.client.1.smithi115.stdout:7/616: creat d3/da/d18/d73/d35/d53/fc1 x:0 0 0 2021-11-03T17:15:23.639 INFO:tasks.workunit.client.1.smithi115.stdout:3/800: creat d13/da4/db2/f111 x:0 0 0 2021-11-03T17:15:23.646 INFO:tasks.workunit.client.1.smithi115.stdout:7/617: fsync d3/d4c/f7f 0 2021-11-03T17:15:23.646 INFO:tasks.workunit.client.1.smithi115.stdout:7/618: chown d3/da/d18/d73/d35/f75 68 1 2021-11-03T17:15:23.647 INFO:tasks.workunit.client.1.smithi115.stdout:7/619: write d3/da/f64 [9356459,93260] 0 2021-11-03T17:15:23.647 INFO:tasks.workunit.client.1.smithi115.stdout:7/620: chown d3/da/l61 207 1 2021-11-03T17:15:23.647 INFO:tasks.workunit.client.1.smithi115.stdout:3/801: link d13/d51/d2e/lfa d13/d35/d54/df6/l112 0 2021-11-03T17:15:23.656 INFO:tasks.workunit.client.1.smithi115.stdout:3/802: dread d13/d2a/d60/f77 [0,4194304] 0 2021-11-03T17:15:23.664 INFO:tasks.workunit.client.1.smithi115.stdout:3/803: rename d13/l17 to d13/d2a/d8a/l113 0 2021-11-03T17:15:23.669 INFO:tasks.workunit.client.1.smithi115.stdout:3/804: chown d13/d51 22793528 1 2021-11-03T17:15:23.687 INFO:tasks.workunit.client.1.smithi115.stdout:3/805: dwrite d13/d2a/d60/f84 [0,4194304] 0 2021-11-03T17:15:23.693 INFO:tasks.workunit.client.1.smithi115.stdout:3/806: symlink d13/d51/d2e/l114 0 2021-11-03T17:15:23.703 INFO:tasks.workunit.client.1.smithi115.stdout:3/807: truncate d13/d51/d40/f88 66458 0 2021-11-03T17:15:23.704 INFO:tasks.workunit.client.1.smithi115.stdout:3/808: write d13/d35/f36 [1544599,118767] 0 2021-11-03T17:15:23.704 INFO:tasks.workunit.client.1.smithi115.stdout:3/809: truncate d13/d51/d40/ff7 417739 0 2021-11-03T17:15:23.712 INFO:tasks.workunit.client.1.smithi115.stdout:3/810: readlink d13/d3b/db1/ld1 0 2021-11-03T17:15:23.716 INFO:tasks.workunit.client.1.smithi115.stdout:3/811: creat d13/d2a/d53/d72/dd2/df1/f115 x:0 0 0 2021-11-03T17:15:23.719 INFO:tasks.workunit.client.1.smithi115.stdout:3/812: mkdir d13/d2a/d53/d72/dd2/df1/d116 0 2021-11-03T17:15:23.719 INFO:tasks.workunit.client.1.smithi115.stdout:3/813: write d13/d2a/d8a/d99/fb8 [110879,39566] 0 2021-11-03T17:15:23.721 INFO:tasks.workunit.client.1.smithi115.stdout:3/814: dread - d13/d2a/d53/f8c zero size 2021-11-03T17:15:23.722 INFO:tasks.workunit.client.1.smithi115.stdout:3/815: truncate d13/d35/d54/dad/fed 11636 0 2021-11-03T17:15:23.733 INFO:tasks.workunit.client.1.smithi115.stdout:3/816: write d13/fa9 [3413806,5291] 0 2021-11-03T17:15:23.734 INFO:tasks.workunit.client.1.smithi115.stdout:3/817: chown d13/d2a/d53/d70/f102 763105264 1 2021-11-03T17:15:23.736 INFO:tasks.workunit.client.1.smithi115.stdout:3/818: getdents d13 0 2021-11-03T17:15:23.738 INFO:tasks.workunit.client.1.smithi115.stdout:2/635: sync 2021-11-03T17:15:23.738 INFO:tasks.workunit.client.1.smithi115.stdout:4/801: sync 2021-11-03T17:15:23.742 INFO:tasks.workunit.client.1.smithi115.stdout:3/819: creat d13/d2a/d60/f117 x:0 0 0 2021-11-03T17:15:23.743 INFO:tasks.workunit.client.1.smithi115.stdout:3/820: chown d13/d50/f42 6524 1 2021-11-03T17:15:23.754 INFO:tasks.workunit.client.1.smithi115.stdout:4/802: mknod d7/dc/d1a/d21/db1/d56/d42/c113 1 2021-11-03T17:15:23.757 INFO:tasks.workunit.client.1.smithi115.stdout:2/636: readlink d4/d44/l74 0 2021-11-03T17:15:23.763 INFO:tasks.workunit.client.1.smithi115.stdout:2/637: dread d4/d6/d13/d75/f7b [0,4194304] 0 2021-11-03T17:15:23.767 INFO:tasks.workunit.client.1.smithi115.stdout:4/803: mkdir d7/dc/d6a/d3c/d112/ddc/d114 0 2021-11-03T17:15:23.767 INFO:tasks.workunit.client.1.smithi115.stdout:3/821: rmdir d13/d51/d40 39 2021-11-03T17:15:23.776 INFO:tasks.workunit.client.1.smithi115.stdout:4/804: symlink d7/dc/d6a/d3c/d10e/l115 0 2021-11-03T17:15:23.776 INFO:tasks.workunit.client.1.smithi115.stdout:4/805: readlink d7/dc/d1a/d21/db1/d56/d63/d9e/lf8 0 2021-11-03T17:15:23.781 INFO:tasks.workunit.client.1.smithi115.stdout:2/638: dread d4/d6/d56/d6c/d34/d43/d28/f3b [0,4194304] 0 2021-11-03T17:15:23.784 INFO:tasks.workunit.client.1.smithi115.stdout:4/806: mkdir d7/dc/d6a/d116 0 2021-11-03T17:15:23.788 INFO:tasks.workunit.client.1.smithi115.stdout:3/822: symlink d13/d2a/d60/l118 0 2021-11-03T17:15:23.790 INFO:tasks.workunit.client.1.smithi115.stdout:2/639: creat d4/d6/dad/dc2/fdb x:0 0 0 2021-11-03T17:15:23.794 INFO:tasks.workunit.client.1.smithi115.stdout:4/807: link d7/dc/d6a/d3c/fa8 d7/dc/d6a/d116/f117 0 2021-11-03T17:15:23.794 INFO:tasks.workunit.client.1.smithi115.stdout:4/808: readlink d7/dc/d1a/d21/db1/d56/d63/l61 0 2021-11-03T17:15:23.795 INFO:tasks.workunit.client.1.smithi115.stdout:4/809: fdatasync d7/dc/fe9 0 2021-11-03T17:15:23.796 INFO:tasks.workunit.client.1.smithi115.stdout:3/823: creat d13/d2a/d53/d70/f119 x:0 0 0 2021-11-03T17:15:23.797 INFO:tasks.workunit.client.1.smithi115.stdout:3/824: write d13/d51/d2e/dbf/f110 [365270,78840] 0 2021-11-03T17:15:23.798 INFO:tasks.workunit.client.1.smithi115.stdout:3/825: write d13/da4/fb9 [2492908,83581] 0 2021-11-03T17:15:23.798 INFO:tasks.workunit.client.1.smithi115.stdout:3/826: fdatasync d13/d2a/d53/d70/d92/fe4 0 2021-11-03T17:15:23.798 INFO:tasks.workunit.client.1.smithi115.stdout:3/827: write d13/d2a/d8a/d99/fb8 [503371,12820] 0 2021-11-03T17:15:23.799 INFO:tasks.workunit.client.1.smithi115.stdout:3/828: truncate d13/d67/f105 828115 0 2021-11-03T17:15:23.799 INFO:tasks.workunit.client.1.smithi115.stdout:2/640: stat d4/d6/d13/f25 0 2021-11-03T17:15:23.800 INFO:tasks.workunit.client.1.smithi115.stdout:4/810: chown d7/dc/l35 3944731 1 2021-11-03T17:15:23.803 INFO:tasks.workunit.client.1.smithi115.stdout:3/829: mkdir d13/d50/dfc/d11a 0 2021-11-03T17:15:23.804 INFO:tasks.workunit.client.1.smithi115.stdout:4/811: readlink d7/d7a/l8f 0 2021-11-03T17:15:23.817 INFO:tasks.workunit.client.1.smithi115.stdout:3/830: link d13/d35/d54/ffe d13/d2a/d53/d72/dd2/f11b 0 2021-11-03T17:15:23.817 INFO:tasks.workunit.client.1.smithi115.stdout:3/831: dread - d13/d2a/d53/d72/fec zero size 2021-11-03T17:15:23.818 INFO:tasks.workunit.client.1.smithi115.stdout:3/832: dread d13/f22 [0,4194304] 0 2021-11-03T17:15:23.837 INFO:tasks.workunit.client.1.smithi115.stdout:3/833: symlink d13/d2a/d53/d72/l11c 0 2021-11-03T17:15:23.840 INFO:tasks.workunit.client.1.smithi115.stdout:3/834: getdents d13/d2a/d53/d70/d92 0 2021-11-03T17:15:23.846 INFO:tasks.workunit.client.1.smithi115.stdout:3/835: creat d13/d51/f11d x:0 0 0 2021-11-03T17:15:23.850 INFO:tasks.workunit.client.1.smithi115.stdout:3/836: rename d13/d3b/lac to d13/d2a/d53/d70/d92/dc2/l11e 0 2021-11-03T17:15:23.852 INFO:tasks.workunit.client.1.smithi115.stdout:2/641: dwrite d4/f3a [0,4194304] 0 2021-11-03T17:15:23.854 INFO:tasks.workunit.client.1.smithi115.stdout:3/837: rename d13/da4/df4 to d13/d3b/d11f 0 2021-11-03T17:15:23.856 INFO:tasks.workunit.client.1.smithi115.stdout:4/812: dwrite d7/dc/d1a/d21/fbe [0,4194304] 0 2021-11-03T17:15:23.860 INFO:tasks.workunit.client.1.smithi115.stdout:2/642: fdatasync d4/d6/d56/d6c/d34/d42/f8f 0 2021-11-03T17:15:23.875 INFO:tasks.workunit.client.1.smithi115.stdout:4/813: dwrite d7/dc/d6a/fac [0,4194304] 0 2021-11-03T17:15:23.877 INFO:tasks.workunit.client.1.smithi115.stdout:2/643: symlink d4/d6/d56/d6c/d34/d43/d63/d77/d79/d9c/d9f/ldc 0 2021-11-03T17:15:23.878 INFO:tasks.workunit.client.1.smithi115.stdout:4/814: read - d7/d7a/d83/fd0 zero size 2021-11-03T17:15:23.885 INFO:tasks.workunit.client.1.smithi115.stdout:2/644: creat d4/d6/dad/fdd x:0 0 0 2021-11-03T17:15:23.888 INFO:tasks.workunit.client.1.smithi115.stdout:2/645: write d4/f35 [3228553,53211] 0 2021-11-03T17:15:23.891 INFO:tasks.workunit.client.1.smithi115.stdout:2/646: truncate d4/d6/f45 4944035 0 2021-11-03T17:15:23.896 INFO:tasks.workunit.client.1.smithi115.stdout:3/838: dwrite f12 [0,4194304] 0 2021-11-03T17:15:23.896 INFO:tasks.workunit.client.1.smithi115.stdout:4/815: dwrite d7/dc/d6a/fd3 [0,4194304] 0 2021-11-03T17:15:23.901 INFO:tasks.workunit.client.1.smithi115.stdout:4/816: truncate d7/dc/d6a/f3a 346023 0 2021-11-03T17:15:23.905 INFO:tasks.workunit.client.1.smithi115.stdout:3/839: symlink d13/d50/d9e/l120 0 2021-11-03T17:15:23.907 INFO:tasks.workunit.client.1.smithi115.stdout:4/817: creat d7/dc/d1a/d21/db1/d56/d42/f118 x:0 0 0 2021-11-03T17:15:23.907 INFO:tasks.workunit.client.1.smithi115.stdout:4/818: fdatasync d7/dc/d1a/f1c 0 2021-11-03T17:15:23.907 INFO:tasks.workunit.client.1.smithi115.stdout:4/819: fsync d7/dc/d1a/d21/db1/ffc 0 2021-11-03T17:15:23.908 INFO:tasks.workunit.client.1.smithi115.stdout:4/820: stat d7/dc/d6a/fac 0 2021-11-03T17:15:23.909 INFO:tasks.workunit.client.1.smithi115.stdout:2/647: dwrite d4/d6/d56/d6c/d34/d42/f8f [0,4194304] 0 2021-11-03T17:15:23.927 INFO:tasks.workunit.client.1.smithi115.stdout:4/821: dwrite d7/dc/d1a/d21/db1/d56/d63/f67 [0,4194304] 0 2021-11-03T17:15:23.928 INFO:tasks.workunit.client.1.smithi115.stdout:3/840: write d13/d2a/d53/f8c [748255,49760] 0 2021-11-03T17:15:23.935 INFO:tasks.workunit.client.1.smithi115.stdout:4/822: unlink d7/dc/l34 0 2021-11-03T17:15:23.935 INFO:tasks.workunit.client.1.smithi115.stdout:4/823: readlink d7/dc/d6a/d3c/d112/l108 0 2021-11-03T17:15:23.936 INFO:tasks.workunit.client.1.smithi115.stdout:3/841: truncate d13/d3b/db1/f107 1000696 0 2021-11-03T17:15:23.941 INFO:tasks.workunit.client.1.smithi115.stdout:4/824: readlink d7/dc/d1a/d21/l29 0 2021-11-03T17:15:23.944 INFO:tasks.workunit.client.1.smithi115.stdout:2/648: rename d4/d6/d56/d6c/d34/d43/d28/f41 to d4/d6/d56/d6c/d34/d43/d63/d77/fde 0 2021-11-03T17:15:23.956 INFO:tasks.workunit.client.1.smithi115.stdout:3/842: mkdir d13/d51/d2e/db4/d121 0 2021-11-03T17:15:23.957 INFO:tasks.workunit.client.1.smithi115.stdout:2/649: creat d4/d6/d56/d6c/d34/d43/d63/fdf x:0 0 0 2021-11-03T17:15:23.961 INFO:tasks.workunit.client.1.smithi115.stdout:3/843: getdents d13/d2a 0 2021-11-03T17:15:23.962 INFO:tasks.workunit.client.1.smithi115.stdout:3/844: truncate d13/d2c/ffb 756026 0 2021-11-03T17:15:23.963 INFO:tasks.workunit.client.1.smithi115.stdout:9/648: sync 2021-11-03T17:15:23.963 INFO:tasks.workunit.client.1.smithi115.stdout:1/712: sync 2021-11-03T17:15:23.964 INFO:tasks.workunit.client.1.smithi115.stdout:4/825: dread d7/dc/fc8 [0,4194304] 0 2021-11-03T17:15:23.975 INFO:tasks.workunit.client.1.smithi115.stdout:9/649: write d1/d18/d2b/d42/d63/f9f [403808,35126] 0 2021-11-03T17:15:23.977 INFO:tasks.workunit.client.1.smithi115.stdout:2/650: link d4/d6/f2a d4/d6/d13/d75/fe0 0 2021-11-03T17:15:23.979 INFO:tasks.workunit.client.1.smithi115.stdout:1/713: dread d7/dc/dd/d3e/dd1/d43/f3c [0,4194304] 0 2021-11-03T17:15:23.980 INFO:tasks.workunit.client.1.smithi115.stdout:4/826: chown d7/dc/d1a/d21/db1/d56/d63/d7b/lcf 92126 1 2021-11-03T17:15:24.000 INFO:tasks.workunit.client.1.smithi115.stdout:9/650: creat d1/d18/d2b/db9/fed x:0 0 0 2021-11-03T17:15:24.014 INFO:tasks.workunit.client.1.smithi115.stdout:4/827: unlink d7/dc/d1a/d21/lfb 0 2021-11-03T17:15:24.017 INFO:tasks.workunit.client.1.smithi115.stdout:9/651: mkdir d1/d1a/d48/d27/d53/d83/d88/dcd/dda/dee 0 2021-11-03T17:15:24.020 INFO:tasks.workunit.client.1.smithi115.stdout:1/714: getdents d7/d38/d59 0 2021-11-03T17:15:24.023 INFO:tasks.workunit.client.1.smithi115.stdout:4/828: mknod d7/dc/d6a/d72/c119 1 2021-11-03T17:15:24.023 INFO:tasks.workunit.client.1.smithi115.stdout:4/829: dread - d7/dc/d6a/d3c/d112/ff3 zero size 2021-11-03T17:15:24.024 INFO:tasks.workunit.client.1.smithi115.stdout:4/830: write d7/dc/d6a/d3c/dc5/ffa [2566,125625] 0 2021-11-03T17:15:24.024 INFO:tasks.workunit.client.1.smithi115.stdout:1/715: dread d7/dc/d1a/f6b [0,4194304] 0 2021-11-03T17:15:24.024 INFO:tasks.workunit.client.1.smithi115.stdout:1/716: fdatasync d7/dc/dd/d3e/dd1/d43/f3c 0 2021-11-03T17:15:24.025 INFO:tasks.workunit.client.1.smithi115.stdout:9/652: creat d1/d1a/d48/d27/d2f/d7b/fef x:0 0 0 2021-11-03T17:15:24.031 INFO:tasks.workunit.client.1.smithi115.stdout:1/717: rmdir d7/dc/dd/d3e/d58 39 2021-11-03T17:15:24.033 INFO:tasks.workunit.client.1.smithi115.stdout:9/653: symlink d1/d1a/d48/d27/d2f/d7b/de6/lf0 0 2021-11-03T17:15:24.033 INFO:tasks.workunit.client.1.smithi115.stdout:9/654: dread - d1/d1a/d48/d27/f4c zero size 2021-11-03T17:15:24.045 INFO:tasks.workunit.client.1.smithi115.stdout:2/651: write d4/d6/d56/d6c/d34/d43/d63/d77/d79/d82/fa3 [3780047,6675] 0 2021-11-03T17:15:24.050 INFO:tasks.workunit.client.1.smithi115.stdout:9/655: symlink d1/d1a/d48/d27/d2f/d6a/lf1 0 2021-11-03T17:15:24.064 INFO:tasks.workunit.client.1.smithi115.stdout:1/718: fdatasync d7/dc/dd/d3e/dd1/d26/dc3/fd3 0 2021-11-03T17:15:24.068 INFO:tasks.workunit.client.1.smithi115.stdout:9/656: mkdir d1/d18/d2b/db9/df2 0 2021-11-03T17:15:24.068 INFO:tasks.workunit.client.1.smithi115.stdout:9/657: truncate d1/d1a/d48/d27/d53/d83/d88/dcd/dda/fe4 368875 0 2021-11-03T17:15:24.072 INFO:tasks.workunit.client.1.smithi115.stdout:4/831: dwrite d7/d7a/d83/f8c [0,4194304] 0 2021-11-03T17:15:24.072 INFO:tasks.workunit.client.1.smithi115.stdout:9/658: creat d1/d18/d2b/d42/d6e/db5/ff3 x:0 0 0 2021-11-03T17:15:24.072 INFO:tasks.workunit.client.1.smithi115.stdout:2/652: dwrite d4/d6/d13/d75/f7b [0,4194304] 0 2021-11-03T17:15:24.091 INFO:tasks.workunit.client.1.smithi115.stdout:9/659: write d1/d1a/d48/d27/d53/d83/d88/d9e/f3b [663354,113145] 0 2021-11-03T17:15:24.095 INFO:tasks.workunit.client.1.smithi115.stdout:1/719: symlink d7/dc/dd/d3e/d58/dc2/dd2/le5 0 2021-11-03T17:15:24.095 INFO:tasks.workunit.client.1.smithi115.stdout:4/832: write d7/dc/d6a/d3c/fbd [729752,106577] 0 2021-11-03T17:15:24.096 INFO:tasks.workunit.client.1.smithi115.stdout:4/833: chown d7/dc/d6a/d3c/f3e 0 1 2021-11-03T17:15:24.104 INFO:tasks.workunit.client.1.smithi115.stdout:4/834: dread d7/dc/d1a/d21/fa1 [0,4194304] 0 2021-11-03T17:15:24.105 INFO:tasks.workunit.client.1.smithi115.stdout:2/653: dwrite d4/f5a [4194304,4194304] 0 2021-11-03T17:15:24.105 INFO:tasks.workunit.client.1.smithi115.stdout:4/835: read d7/dc/d1a/d21/db1/d56/d63/f6e [3856016,32142] 0 2021-11-03T17:15:24.106 INFO:tasks.workunit.client.1.smithi115.stdout:2/654: write d4/d6/d56/d6c/d34/d42/f8f [3926108,77259] 0 2021-11-03T17:15:24.108 INFO:tasks.workunit.client.1.smithi115.stdout:9/660: write d1/f1d [2965786,92694] 0 2021-11-03T17:15:24.130 INFO:tasks.workunit.client.1.smithi115.stdout:4/836: truncate d7/dc/d6a/d3c/f3e 560018 0 2021-11-03T17:15:24.131 INFO:tasks.workunit.client.1.smithi115.stdout:1/720: dwrite d7/dc/dd/d42/f47 [0,4194304] 0 2021-11-03T17:15:24.135 INFO:tasks.workunit.client.1.smithi115.stdout:9/661: mknod d1/d1a/d48/d27/d53/d83/d88/dcd/cf4 1 2021-11-03T17:15:24.135 INFO:tasks.workunit.client.1.smithi115.stdout:9/662: truncate d1/d18/d2b/db9/fed 465679 0 2021-11-03T17:15:24.140 INFO:tasks.workunit.client.1.smithi115.stdout:1/721: rmdir d7/dc/dab/d73 39 2021-11-03T17:15:24.143 INFO:tasks.workunit.client.1.smithi115.stdout:2/655: dread d4/d6/d56/f5d [12582912,4194304] 0 2021-11-03T17:15:24.143 INFO:tasks.workunit.client.1.smithi115.stdout:2/656: chown d4/d6/dad/fdd 10941713 1 2021-11-03T17:15:24.150 INFO:tasks.workunit.client.1.smithi115.stdout:2/657: mknod d4/d6/dad/d7f/ce1 1 2021-11-03T17:15:24.153 INFO:tasks.workunit.client.1.smithi115.stdout:1/722: creat d7/dc/dab/d73/d7f/fe6 x:0 0 0 2021-11-03T17:15:24.158 INFO:tasks.workunit.client.1.smithi115.stdout:2/658: link d4/d6/d13/d75/ld3 d4/d6/d56/d6c/d34/d43/d63/d77/d79/d9c/d9f/le2 0 2021-11-03T17:15:24.161 INFO:tasks.workunit.client.1.smithi115.stdout:4/837: dwrite d7/dc/d1a/d21/db1/f74 [0,4194304] 0 2021-11-03T17:15:24.162 INFO:tasks.workunit.client.1.smithi115.stdout:9/663: dwrite d1/d18/d4a/d74/fdd [0,4194304] 0 2021-11-03T17:15:24.162 INFO:tasks.workunit.client.1.smithi115.stdout:9/664: dread - d1/d18/d4a/d74/d8b/dbc/fe1 zero size 2021-11-03T17:15:24.163 INFO:tasks.workunit.client.1.smithi115.stdout:1/723: dread d7/dc/f16 [0,4194304] 0 2021-11-03T17:15:24.165 INFO:tasks.workunit.client.1.smithi115.stdout:4/838: mkdir d7/dd1/dee/d11a 0 2021-11-03T17:15:24.165 INFO:tasks.workunit.client.1.smithi115.stdout:4/839: chown d7/dc/d1a/d21/l29 74729 1 2021-11-03T17:15:24.172 INFO:tasks.workunit.client.1.smithi115.stdout:2/659: write d4/d6/d56/d6c/d34/d43/d63/d77/fb1 [671242,5170] 0 2021-11-03T17:15:24.172 INFO:tasks.workunit.client.1.smithi115.stdout:9/665: dread d1/d18/d2b/d42/d6e/f5c [0,4194304] 0 2021-11-03T17:15:24.173 INFO:tasks.workunit.client.1.smithi115.stdout:9/666: readlink d1/d1a/d48/d27/d2f/d6a/lf1 0 2021-11-03T17:15:24.178 INFO:tasks.workunit.client.1.smithi115.stdout:1/724: write d7/f8 [1592819,2691] 0 2021-11-03T17:15:24.179 INFO:tasks.workunit.client.1.smithi115.stdout:2/660: chown d4/d6/dad/d7f/fa5 12 1 2021-11-03T17:15:24.185 INFO:tasks.workunit.client.1.smithi115.stdout:4/840: link d7/dc/d1a/d21/db1/d56/f93 d7/dc/d1a/d21/db1/d56/d42/daf/f11b 0 2021-11-03T17:15:24.188 INFO:tasks.workunit.client.1.smithi115.stdout:9/667: write d1/d18/d2b/d42/d6e/fa6 [293150,28151] 0 2021-11-03T17:15:24.188 INFO:tasks.workunit.client.1.smithi115.stdout:2/661: mknod d4/d6/d56/d6c/d34/d43/d63/d77/d79/d9c/d9f/ce3 1 2021-11-03T17:15:24.191 INFO:tasks.workunit.client.1.smithi115.stdout:4/841: getdents d7/dc/d1a/d21/db1/d56/d42 0 2021-11-03T17:15:24.196 INFO:tasks.workunit.client.1.smithi115.stdout:9/668: creat d1/d1a/d48/d27/d53/d83/d88/ff5 x:0 0 0 2021-11-03T17:15:24.199 INFO:tasks.workunit.client.1.smithi115.stdout:2/662: dread - d4/d6/dad/dc2/fdb zero size 2021-11-03T17:15:24.202 INFO:tasks.workunit.client.1.smithi115.stdout:4/842: rmdir d7/dc/d6a/d3c/d112/ddc 39 2021-11-03T17:15:24.205 INFO:tasks.workunit.client.1.smithi115.stdout:4/843: chown d7/d7a/d83/l73 71 1 2021-11-03T17:15:24.206 INFO:tasks.workunit.client.1.smithi115.stdout:4/844: chown d7/d7a/ff2 126 1 2021-11-03T17:15:24.207 INFO:tasks.workunit.client.1.smithi115.stdout:4/845: symlink d7/dc/d6a/d101/l11c 0 2021-11-03T17:15:24.208 INFO:tasks.workunit.client.1.smithi115.stdout:4/846: write d7/dc/d6a/d3c/fa8 [940396,69997] 0 2021-11-03T17:15:24.209 INFO:tasks.workunit.client.1.smithi115.stdout:9/669: rename d1/d18/d2b/d42/d6e/lbe to d1/d18/d4a/d74/d8b/lf6 0 2021-11-03T17:15:24.212 INFO:tasks.workunit.client.1.smithi115.stdout:4/847: chown d7/dc/d1a/d21/db1/d56/l3b 8310292 1 2021-11-03T17:15:24.218 INFO:tasks.workunit.client.1.smithi115.stdout:1/725: dwrite d7/d38/d59/f9b [0,4194304] 0 2021-11-03T17:15:24.226 INFO:tasks.workunit.client.1.smithi115.stdout:2/663: rename d4/f3a to d4/d6/dad/dc2/fe4 0 2021-11-03T17:15:24.227 INFO:tasks.workunit.client.1.smithi115.stdout:4/848: write d7/fba [574587,100497] 0 2021-11-03T17:15:24.234 INFO:tasks.workunit.client.1.smithi115.stdout:9/670: write d1/d1a/d48/d27/d53/d83/d88/dcd/d79/fad [920332,77474] 0 2021-11-03T17:15:24.241 INFO:tasks.workunit.client.1.smithi115.stdout:9/671: dread - d1/d18/d2b/d42/fb7 zero size 2021-11-03T17:15:24.241 INFO:tasks.workunit.client.1.smithi115.stdout:2/664: dwrite d4/d6/d13/d75/f7b [0,4194304] 0 2021-11-03T17:15:24.241 INFO:tasks.workunit.client.1.smithi115.stdout:2/665: write d4/d6/d13/d75/fda [568879,103454] 0 2021-11-03T17:15:24.242 INFO:tasks.workunit.client.1.smithi115.stdout:2/666: chown d4/fca 37490 1 2021-11-03T17:15:24.246 INFO:tasks.workunit.client.1.smithi115.stdout:9/672: creat d1/d1a/d48/d27/ff7 x:0 0 0 2021-11-03T17:15:24.250 INFO:tasks.workunit.client.1.smithi115.stdout:1/726: truncate d7/dc/dd/d3e/dd1/d4d/d60/fb8 1930483 0 2021-11-03T17:15:24.250 INFO:tasks.workunit.client.1.smithi115.stdout:4/849: write d7/dc/db6/fc9 [1023021,60603] 0 2021-11-03T17:15:24.260 INFO:tasks.workunit.client.1.smithi115.stdout:2/667: unlink d4/f68 0 2021-11-03T17:15:24.260 INFO:tasks.workunit.client.1.smithi115.stdout:2/668: fsync d4/d6/d56/d6c/d34/d43/d63/fdf 0 2021-11-03T17:15:24.261 INFO:tasks.workunit.client.1.smithi115.stdout:2/669: mknod d4/d6/d56/d6c/d34/d42/d8d/dd8/ce5 1 2021-11-03T17:15:24.266 INFO:tasks.workunit.client.1.smithi115.stdout:4/850: unlink d7/dc/d1a/d21/db1/d56/d42/daf/f11b 0 2021-11-03T17:15:24.273 INFO:tasks.workunit.client.1.smithi115.stdout:9/673: truncate d1/d1a/d48/d27/d53/d83/d88/d9e/d58/fc1 957817 0 2021-11-03T17:15:24.279 INFO:tasks.workunit.client.1.smithi115.stdout:9/674: creat d1/d1a/d48/d27/d80/dcc/ff8 x:0 0 0 2021-11-03T17:15:24.280 INFO:tasks.workunit.client.1.smithi115.stdout:9/675: chown d1/d66 329585 1 2021-11-03T17:15:24.284 INFO:tasks.workunit.client.1.smithi115.stdout:2/670: dread d4/d6/d56/d6c/d34/d43/d63/f99 [0,4194304] 0 2021-11-03T17:15:24.287 INFO:tasks.workunit.client.1.smithi115.stdout:1/727: dwrite d7/dc/dd/d3e/d58/f64 [0,4194304] 0 2021-11-03T17:15:24.288 INFO:tasks.workunit.client.1.smithi115.stdout:4/851: dwrite d7/dc/d6a/d3c/fa8 [0,4194304] 0 2021-11-03T17:15:24.289 INFO:tasks.workunit.client.1.smithi115.stdout:4/852: fdatasync d7/dc/d6a/d116/f117 0 2021-11-03T17:15:24.289 INFO:tasks.workunit.client.1.smithi115.stdout:2/671: symlink d4/d6/dad/d7f/le6 0 2021-11-03T17:15:24.293 INFO:tasks.workunit.client.1.smithi115.stdout:1/728: creat d7/dc/dd/d3e/dd1/d4d/d60/d8f/ddf/fe7 x:0 0 0 2021-11-03T17:15:24.300 INFO:tasks.workunit.client.1.smithi115.stdout:9/676: write d1/d1a/d48/d27/d2f/d7b/fe3 [949917,82551] 0 2021-11-03T17:15:24.301 INFO:tasks.workunit.client.1.smithi115.stdout:9/677: write d1/d18/d2b/d45/dc2/fe7 [80180,3682] 0 2021-11-03T17:15:24.309 INFO:tasks.workunit.client.1.smithi115.stdout:9/678: dread d1/d18/d2b/d42/d6e/f52 [0,4194304] 0 2021-11-03T17:15:24.337 INFO:tasks.workunit.client.1.smithi115.stdout:9/679: dwrite d1/d1a/d48/d27/d53/d83/d88/dcd/d79/fad [0,4194304] 0 2021-11-03T17:15:24.337 INFO:tasks.workunit.client.1.smithi115.stdout:9/680: dread - d1/d1a/d48/d27/d80/dcc/fd8 zero size 2021-11-03T17:15:24.338 INFO:tasks.workunit.client.1.smithi115.stdout:4/853: link d7/l26 d7/dc/d6a/d101/l11d 0 2021-11-03T17:15:24.351 INFO:tasks.workunit.client.1.smithi115.stdout:0/857: sync 2021-11-03T17:15:24.352 INFO:tasks.workunit.client.1.smithi115.stdout:5/754: sync 2021-11-03T17:15:24.372 INFO:tasks.workunit.client.1.smithi115.stdout:1/729: creat d7/dc/dab/d73/d7f/fe8 x:0 0 0 2021-11-03T17:15:24.396 INFO:tasks.workunit.client.1.smithi115.stdout:2/672: creat d4/d6/d56/d6c/d34/d84/fe7 x:0 0 0 2021-11-03T17:15:24.400 INFO:tasks.workunit.client.1.smithi115.stdout:1/730: mknod d7/dc/dd/ce9 1 2021-11-03T17:15:24.400 INFO:tasks.workunit.client.1.smithi115.stdout:1/731: chown d7/d38/d59 20409309 1 2021-11-03T17:15:24.403 INFO:tasks.workunit.client.1.smithi115.stdout:2/673: write d4/d6/d13/f6b [449919,68125] 0 2021-11-03T17:15:24.403 INFO:tasks.workunit.client.1.smithi115.stdout:2/674: chown d4/d6/d56/d6c/d34/d42/d8d 215228191 1 2021-11-03T17:15:24.403 INFO:tasks.workunit.client.1.smithi115.stdout:2/675: fdatasync d4/d6/f45 0 2021-11-03T17:15:24.406 INFO:tasks.workunit.client.1.smithi115.stdout:0/858: dread d9/db/d83/fd1 [0,4194304] 0 2021-11-03T17:15:24.407 INFO:tasks.workunit.client.1.smithi115.stdout:0/859: dread - d9/db/d83/d112/f116 zero size 2021-11-03T17:15:24.407 INFO:tasks.workunit.client.1.smithi115.stdout:0/860: write d9/db/f8e [1768016,97728] 0 2021-11-03T17:15:24.407 INFO:tasks.workunit.client.1.smithi115.stdout:2/676: write d4/d6/d13/f25 [120584,40176] 0 2021-11-03T17:15:24.409 INFO:tasks.workunit.client.1.smithi115.stdout:1/732: dread d7/f7c [0,4194304] 0 2021-11-03T17:15:24.410 INFO:tasks.workunit.client.1.smithi115.stdout:9/681: dwrite d1/d18/d2b/d42/fbd [4194304,4194304] 0 2021-11-03T17:15:24.411 INFO:tasks.workunit.client.1.smithi115.stdout:0/861: creat d9/d15/dce/f11e x:0 0 0 2021-11-03T17:15:24.413 INFO:tasks.workunit.client.1.smithi115.stdout:2/677: chown d4/d6/d56/d6c/d34/d43/d63/d77/d79/dac/lb3 9 1 2021-11-03T17:15:24.413 INFO:tasks.workunit.client.1.smithi115.stdout:2/678: chown d4/d6/d56/d6c/d34/d84 53755383 1 2021-11-03T17:15:24.421 INFO:tasks.workunit.client.1.smithi115.stdout:1/733: mknod d7/dc/dd/d3e/dd1/d43/cea 1 2021-11-03T17:15:24.425 INFO:tasks.workunit.client.1.smithi115.stdout:0/862: dwrite d9/db/d89/f84 [0,4194304] 0 2021-11-03T17:15:24.426 INFO:tasks.workunit.client.1.smithi115.stdout:0/863: dread - d9/db/d83/d112/f116 zero size 2021-11-03T17:15:24.435 INFO:tasks.workunit.client.1.smithi115.stdout:9/682: write d1/d1a/d48/d27/d2f/fc6 [695799,7885] 0 2021-11-03T17:15:24.437 INFO:tasks.workunit.client.1.smithi115.stdout:0/864: read d9/d15/d1a/d1d/dcc/d46/d8c/f79 [3143383,70802] 0 2021-11-03T17:15:24.442 INFO:tasks.workunit.client.1.smithi115.stdout:1/734: creat d7/dc/dd/d3e/dd1/d4d/d60/db2/feb x:0 0 0 2021-11-03T17:15:24.442 INFO:tasks.workunit.client.1.smithi115.stdout:1/735: fsync d7/d38/d59/f9b 0 2021-11-03T17:15:24.446 INFO:tasks.workunit.client.1.smithi115.stdout:0/865: readlink d9/db/d89/d81/lc0 0 2021-11-03T17:15:24.450 INFO:tasks.workunit.client.1.smithi115.stdout:1/736: rmdir d7/dc/dd/d3e/dd1/d4d/d60/db2 39 2021-11-03T17:15:24.452 INFO:tasks.workunit.client.1.smithi115.stdout:9/683: dwrite d1/f1d [0,4194304] 0 2021-11-03T17:15:24.455 INFO:tasks.workunit.client.1.smithi115.stdout:0/866: symlink d9/d15/d78/d108/l11f 0 2021-11-03T17:15:24.455 INFO:tasks.workunit.client.1.smithi115.stdout:0/867: write d9/d15/d1a/d1d/dcc/d6e/fa3 [4394256,95083] 0 2021-11-03T17:15:24.464 INFO:tasks.workunit.client.1.smithi115.stdout:1/737: creat d7/dc/dd/d3e/d58/dc2/fec x:0 0 0 2021-11-03T17:15:24.483 INFO:tasks.workunit.client.1.smithi115.stdout:1/738: symlink d7/d38/d84/dca/led 0 2021-11-03T17:15:24.496 INFO:tasks.workunit.client.1.smithi115.stdout:0/868: rmdir d9/d15/d1a/d1d/dcc/d46/d63/d70/d11c 0 2021-11-03T17:15:24.500 INFO:tasks.workunit.client.1.smithi115.stdout:1/739: write f2 [2441941,130439] 0 2021-11-03T17:15:24.501 INFO:tasks.workunit.client.1.smithi115.stdout:9/684: dwrite d1/f72 [0,4194304] 0 2021-11-03T17:15:24.506 INFO:tasks.workunit.client.1.smithi115.stdout:4/854: creat d7/dc/d1a/f11e x:0 0 0 2021-11-03T17:15:24.511 INFO:tasks.workunit.client.1.smithi115.stdout:1/740: dread d7/dc/dd/d3e/fa1 [0,4194304] 0 2021-11-03T17:15:24.514 INFO:tasks.workunit.client.1.smithi115.stdout:0/869: dwrite d9/d15/d1a/d1d/dcc/d46/d63/dae/fd7 [0,4194304] 0 2021-11-03T17:15:24.514 INFO:tasks.workunit.client.1.smithi115.stdout:0/870: dread - d9/d15/d1a/f2e zero size 2021-11-03T17:15:24.520 INFO:tasks.workunit.client.1.smithi115.stdout:1/741: dread d7/dc/dd/d3e/dd1/d4b/f72 [0,4194304] 0 2021-11-03T17:15:24.523 INFO:tasks.workunit.client.1.smithi115.stdout:5/755: rename d1a/d28/d4c/d75/d68/da5/fc3 to d1a/d28/d7b/d48/dd4/fff 0 2021-11-03T17:15:24.523 INFO:tasks.workunit.client.1.smithi115.stdout:5/756: write d1a/d28/d34/ff7 [371670,3739] 0 2021-11-03T17:15:24.527 INFO:tasks.workunit.client.1.smithi115.stdout:9/685: creat d1/d1a/d48/d27/d2f/d41/ff9 x:0 0 0 2021-11-03T17:15:24.529 INFO:tasks.workunit.client.1.smithi115.stdout:5/757: dread d1a/d28/d4c/d75/d83/f98 [0,4194304] 0 2021-11-03T17:15:24.532 INFO:tasks.workunit.client.1.smithi115.stdout:4/855: creat d7/dc/d1a/d21/db1/d56/d63/d7b/d96/f11f x:0 0 0 2021-11-03T17:15:24.534 INFO:tasks.workunit.client.1.smithi115.stdout:4/856: dread d7/d7a/d83/f64 [0,4194304] 0 2021-11-03T17:15:24.543 INFO:tasks.workunit.client.1.smithi115.stdout:1/742: getdents d7/dc/dd/d3e/d58/dc2 0 2021-11-03T17:15:24.546 INFO:tasks.workunit.client.1.smithi115.stdout:0/871: rmdir d9/db 39 2021-11-03T17:15:24.546 INFO:tasks.workunit.client.1.smithi115.stdout:1/743: dread d7/dc/dd/d3e/fa1 [0,4194304] 0 2021-11-03T17:15:24.547 INFO:tasks.workunit.client.1.smithi115.stdout:9/686: getdents d1 0 2021-11-03T17:15:24.549 INFO:tasks.workunit.client.1.smithi115.stdout:5/758: chown d1a/d28/d4c/d63/f9c 32 1 2021-11-03T17:15:24.553 INFO:tasks.workunit.client.1.smithi115.stdout:4/857: dwrite d7/dc/d6a/d3c/dc5/ffa [0,4194304] 0 2021-11-03T17:15:24.553 INFO:tasks.workunit.client.1.smithi115.stdout:4/858: stat d7/dc/d1a/d21/db1/d56/d63/lc2 0 2021-11-03T17:15:24.553 INFO:tasks.workunit.client.1.smithi115.stdout:4/859: chown d7/dc/d1a/d21/db1/d56/d63/d7b 5364 1 2021-11-03T17:15:24.562 INFO:tasks.workunit.client.1.smithi115.stdout:5/759: creat d1a/d28/d4a/d65/f100 x:0 0 0 2021-11-03T17:15:24.571 INFO:tasks.workunit.client.1.smithi115.stdout:8/649: sync 2021-11-03T17:15:24.571 INFO:tasks.workunit.client.1.smithi115.stdout:8/650: chown d4/d7d/fe6 517197763 1 2021-11-03T17:15:24.572 INFO:tasks.workunit.client.1.smithi115.stdout:4/860: creat d7/dc/d1a/d21/db1/d56/d63/f120 x:0 0 0 2021-11-03T17:15:24.572 INFO:tasks.workunit.client.1.smithi115.stdout:4/861: truncate d7/dc/d1a/f1c 1540261 0 2021-11-03T17:15:24.572 INFO:tasks.workunit.client.1.smithi115.stdout:1/744: truncate d7/dc/dab/f33 1798855 0 2021-11-03T17:15:24.575 INFO:tasks.workunit.client.1.smithi115.stdout:5/760: stat d1a/d3d/fc7 0 2021-11-03T17:15:24.576 INFO:tasks.workunit.client.1.smithi115.stdout:5/761: truncate d1a/d28/d4c/d75/d68/da5/fd5 977598 0 2021-11-03T17:15:24.576 INFO:tasks.workunit.client.1.smithi115.stdout:5/762: mknod d1a/d3d/d58/dab/dd3/c101 1 2021-11-03T17:15:24.577 INFO:tasks.workunit.client.1.smithi115.stdout:4/862: mkdir d7/dc/d1a/d21/db1/d56/d63/d121 0 2021-11-03T17:15:24.580 INFO:tasks.workunit.client.1.smithi115.stdout:5/763: creat d1a/d28/d4c/d75/d83/da1/ded/f102 x:0 0 0 2021-11-03T17:15:24.587 INFO:tasks.workunit.client.1.smithi115.stdout:5/764: chown d1a/d23/fcb 1813329634 1 2021-11-03T17:15:24.588 INFO:tasks.workunit.client.1.smithi115.stdout:9/687: dwrite d1/d8a/f94 [0,4194304] 0 2021-11-03T17:15:24.589 INFO:tasks.workunit.client.1.smithi115.stdout:2/679: rename d4/d6/d56/d6c/d34/d43/d63/d77/d79 to d4/d6/dad/de8 0 2021-11-03T17:15:24.594 INFO:tasks.workunit.client.1.smithi115.stdout:4/863: symlink d7/dc/d6a/d3c/dc5/dcc/l122 0 2021-11-03T17:15:24.597 INFO:tasks.workunit.client.1.smithi115.stdout:4/864: write d7/fa0 [3653719,82175] 0 2021-11-03T17:15:24.623 INFO:tasks.workunit.client.1.smithi115.stdout:5/765: dwrite d1a/d28/d4a/d65/ff0 [4194304,4194304] 0 2021-11-03T17:15:24.628 INFO:tasks.workunit.client.1.smithi115.stdout:0/872: rename d9/d15/d1a/d1d/dcc/d46/d63/lf4 to d9/d15/d78/d80/l120 0 2021-11-03T17:15:24.637 INFO:tasks.workunit.client.1.smithi115.stdout:5/766: dwrite d1a/d28/d7b/f77 [0,4194304] 0 2021-11-03T17:15:24.643 INFO:tasks.workunit.client.1.smithi115.stdout:2/680: unlink d4/d6/d56/l73 0 2021-11-03T17:15:24.650 INFO:tasks.workunit.client.1.smithi115.stdout:0/873: readlink d9/d15/d1a/l55 0 2021-11-03T17:15:24.650 INFO:tasks.workunit.client.1.smithi115.stdout:0/874: chown d9/db/f7e 86699 1 2021-11-03T17:15:24.650 INFO:tasks.workunit.client.1.smithi115.stdout:0/875: write d9/d15/d1a/d1d/dcc/d46/d63/d70/fd2 [799411,67069] 0 2021-11-03T17:15:24.653 INFO:tasks.workunit.client.1.smithi115.stdout:4/865: mkdir d7/dc/d6a/dab/d123 0 2021-11-03T17:15:24.665 INFO:tasks.workunit.client.1.smithi115.stdout:2/681: dread d4/d6/f2a [0,4194304] 0 2021-11-03T17:15:24.666 INFO:tasks.workunit.client.1.smithi115.stdout:4/866: symlink d7/dc/d1a/l124 0 2021-11-03T17:15:24.668 INFO:tasks.workunit.client.1.smithi115.stdout:5/767: mkdir d1a/d28/d4c/d75/d83/da1/d103 0 2021-11-03T17:15:24.668 INFO:tasks.workunit.client.1.smithi115.stdout:5/768: write d1a/d28/d4a/d65/f100 [500600,37092] 0 2021-11-03T17:15:24.691 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:24 smithi115 conmon[34670]: cluster 2021-11-03T17:15:22.645735+0000 mgr.smithi027.ufhhvb (mgr.14622) 26 : cluster [DBG] pgmap v15: 65 pgs: 65 active+clean; 3.0 GiB data, 9.1 GiB used, 527 GiB / 536 GiB avail; 22 MiB/s rd, 42 MiB/s wr, 799 op/s 2021-11-03T17:15:24.692 INFO:tasks.workunit.client.1.smithi115.stdout:0/876: symlink d9/d15/l121 0 2021-11-03T17:15:24.692 INFO:tasks.workunit.client.1.smithi115.stdout:0/877: dread d9/d15/d1a/d1d/dcc/d46/d63/dc5/fe7 [4194304,4194304] 0 2021-11-03T17:15:24.692 INFO:tasks.workunit.client.1.smithi115.stdout:8/651: rename d4/d6/f42 to d4/d6/d18/d30/d33/fe9 0 2021-11-03T17:15:24.693 INFO:tasks.workunit.client.1.smithi115.stdout:2/682: fsync d4/d6/d56/d6c/d34/d42/f8f 0 2021-11-03T17:15:24.693 INFO:tasks.workunit.client.1.smithi115.stdout:2/683: fdatasync d4/d6/d56/d6c/d34/fcb 0 2021-11-03T17:15:24.700 INFO:tasks.workunit.client.1.smithi115.stdout:8/652: write d4/d6/d18/d44/f99 [646016,113587] 0 2021-11-03T17:15:24.701 INFO:tasks.workunit.client.1.smithi115.stdout:8/653: truncate d4/d6/d3f/da4/fdb 385741 0 2021-11-03T17:15:24.703 INFO:tasks.workunit.client.1.smithi115.stdout:4/867: dwrite d7/dc/d1a/f49 [0,4194304] 0 2021-11-03T17:15:24.712 INFO:tasks.workunit.client.1.smithi115.stdout:5/769: truncate d1a/d28/d7b/d48/dd4/fd7 4371863 0 2021-11-03T17:15:24.712 INFO:tasks.workunit.client.1.smithi115.stdout:5/770: chown d1a/d28/d7b/d48/ff2 1521 1 2021-11-03T17:15:24.716 INFO:tasks.workunit.client.1.smithi115.stdout:1/745: rename d7/dc/dd/d3e/dd1/d43/f99 to d7/dc/dd/d3e/dd1/d4d/fee 0 2021-11-03T17:15:24.720 INFO:tasks.workunit.client.1.smithi115.stdout:4/868: dwrite d7/dc/d6a/d3c/fa8 [0,4194304] 0 2021-11-03T17:15:24.720 INFO:tasks.workunit.client.1.smithi115.stdout:4/869: dread - d7/dc/db6/de1/f110 zero size 2021-11-03T17:15:24.730 INFO:tasks.workunit.client.1.smithi115.stdout:4/870: dread d7/dc/d6a/f46 [0,4194304] 0 2021-11-03T17:15:24.731 INFO:tasks.workunit.client.1.smithi115.stdout:0/878: dwrite d9/d15/d1a/f3d [0,4194304] 0 2021-11-03T17:15:24.732 INFO:tasks.workunit.client.1.smithi115.stdout:0/879: mknod d9/d15/d1a/d1d/dcc/d46/d11d/d103/c122 1 2021-11-03T17:15:24.747 INFO:tasks.workunit.client.1.smithi115.stdout:0/880: dwrite d9/d15/d1a/d1d/f4d [0,4194304] 0 2021-11-03T17:15:24.749 INFO:tasks.workunit.client.1.smithi115.stdout:8/654: creat d4/d6/d3f/da4/fea x:0 0 0 2021-11-03T17:15:24.752 INFO:tasks.workunit.client.1.smithi115.stdout:2/684: mkdir d4/d6/dad/d7f/de9 0 2021-11-03T17:15:24.758 INFO:tasks.workunit.client.1.smithi115.stdout:4/871: creat d7/dc/d1a/f125 x:0 0 0 2021-11-03T17:15:24.758 INFO:tasks.workunit.client.1.smithi115.stdout:4/872: chown d7/dc/d1a/d21/db1/d56/d42/f58 64943 1 2021-11-03T17:15:24.763 INFO:tasks.workunit.client.1.smithi115.stdout:8/655: getdents d4/d6/d3f/dde 0 2021-11-03T17:15:24.766 INFO:tasks.workunit.client.1.smithi115.stdout:2/685: fsync d4/d6/d56/d6c/f46 0 2021-11-03T17:15:24.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:24 smithi027 conmon[32322]: cluster 2021-11-03T17:15:22.645735+0000 mgr.smithi027.ufhhvb ( 2021-11-03T17:15:24.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:24 smithi027 conmon[32322]: mgr.14622) 26 : cluster [DBG] pgmap v15: 65 pgs: 65 active+clean; 3.0 GiB data, 9.1 GiB used, 527 GiB / 536 GiB avail; 22 MiB/s rd, 42 MiB/s wr, 799 op/s 2021-11-03T17:15:24.769 INFO:tasks.workunit.client.1.smithi115.stdout:1/746: symlink d7/dc/d1a/lef 0 2021-11-03T17:15:24.769 INFO:tasks.workunit.client.1.smithi115.stdout:1/747: dread - d7/dc/dab/d73/d7f/fe6 zero size 2021-11-03T17:15:24.771 INFO:tasks.workunit.client.1.smithi115.stdout:1/748: dread d7/dc/dd/d3e/fa1 [0,4194304] 0 2021-11-03T17:15:24.773 INFO:tasks.workunit.client.1.smithi115.stdout:0/881: write d9/d15/d78/d80/f97 [2718454,52038] 0 2021-11-03T17:15:24.773 INFO:tasks.workunit.client.1.smithi115.stdout:0/882: fsync d9/d15/f53 0 2021-11-03T17:15:24.774 INFO:tasks.workunit.client.1.smithi115.stdout:4/873: fsync d7/d7a/fa3 0 2021-11-03T17:15:24.777 INFO:tasks.workunit.client.1.smithi115.stdout:8/656: chown d4/d6/f2c 57 1 2021-11-03T17:15:24.778 INFO:tasks.workunit.client.1.smithi115.stdout:2/686: mknod d4/d6/dad/d7f/cea 1 2021-11-03T17:15:24.779 INFO:tasks.workunit.client.1.smithi115.stdout:0/883: dread d9/d15/d1a/d1d/dcc/d85/ff8 [0,4194304] 0 2021-11-03T17:15:24.788 INFO:tasks.workunit.client.1.smithi115.stdout:1/749: write d7/dc/dd/d3e/dd1/d4b/f3a [323061,5545] 0 2021-11-03T17:15:24.789 INFO:tasks.workunit.client.1.smithi115.stdout:2/687: chown d4/f62 552 1 2021-11-03T17:15:24.791 INFO:tasks.workunit.client.1.smithi115.stdout:0/884: creat d9/d15/d1a/d1d/dcc/d46/d11d/d103/f123 x:0 0 0 2021-11-03T17:15:24.792 INFO:tasks.workunit.client.1.smithi115.stdout:0/885: truncate d9/d15/d1a/d1d/f4d 4965094 0 2021-11-03T17:15:24.792 INFO:tasks.workunit.client.1.smithi115.stdout:0/886: truncate d9/d15/dce/f11e 510673 0 2021-11-03T17:15:24.793 INFO:tasks.workunit.client.1.smithi115.stdout:0/887: chown d9/db/d89/f9c 26969 1 2021-11-03T17:15:24.793 INFO:tasks.workunit.client.1.smithi115.stdout:4/874: symlink d7/dc/l126 0 2021-11-03T17:15:24.794 INFO:tasks.workunit.client.1.smithi115.stdout:8/657: mkdir d4/d6/d18/d44/d47/deb 0 2021-11-03T17:15:24.795 INFO:tasks.workunit.client.1.smithi115.stdout:4/875: dread d7/dc/f17 [8388608,4194304] 0 2021-11-03T17:15:24.795 INFO:tasks.workunit.client.1.smithi115.stdout:1/750: chown d7/lc0 761 1 2021-11-03T17:15:24.798 INFO:tasks.workunit.client.1.smithi115.stdout:2/688: creat d4/d6/dad/d7f/feb x:0 0 0 2021-11-03T17:15:24.798 INFO:tasks.workunit.client.1.smithi115.stdout:2/689: fdatasync d4/d6/fbd 0 2021-11-03T17:15:24.801 INFO:tasks.workunit.client.1.smithi115.stdout:0/888: rename d9/d15/d78/d108 to d9/d15/db6/d10b/d124 0 2021-11-03T17:15:24.803 INFO:tasks.workunit.client.1.smithi115.stdout:8/658: truncate d4/f6c 669671 0 2021-11-03T17:15:24.838 INFO:tasks.workunit.client.1.smithi115.stdout:0/889: rename d9/d37/d10d/d10f to d9/d15/d1a/de5/d125 0 2021-11-03T17:15:24.839 INFO:tasks.workunit.client.1.smithi115.stdout:8/659: creat d4/d6/d3f/d43/d88/fec x:0 0 0 2021-11-03T17:15:24.840 INFO:tasks.workunit.client.1.smithi115.stdout:4/876: symlink d7/dc/d6a/dab/l127 0 2021-11-03T17:15:24.848 INFO:tasks.workunit.client.1.smithi115.stdout:1/751: dwrite d7/dc/dab/d73/d7f/fa8 [0,4194304] 0 2021-11-03T17:15:24.849 INFO:tasks.workunit.client.1.smithi115.stdout:2/690: mkdir d4/d6/dad/de8/dac/dec 0 2021-11-03T17:15:24.851 INFO:tasks.workunit.client.1.smithi115.stdout:4/877: truncate d7/dc/d1a/d21/db1/d56/d63/fa6 688994 0 2021-11-03T17:15:24.859 INFO:tasks.workunit.client.1.smithi115.stdout:4/878: rename d7/dc/d1a/d21/db1/d56/d63/d7b/fb8 to d7/dc/d1a/d21/db1/d56/d63/d121/f128 0 2021-11-03T17:15:24.861 INFO:tasks.workunit.client.1.smithi115.stdout:2/691: mkdir d4/d6/d56/d6c/d34/ded 0 2021-11-03T17:15:24.862 INFO:tasks.workunit.client.1.smithi115.stdout:8/660: dwrite d4/d6/d18/d30/d33/f87 [4194304,4194304] 0 2021-11-03T17:15:24.862 INFO:tasks.workunit.client.1.smithi115.stdout:8/661: getdents d4/d6/d3f/dde 0 2021-11-03T17:15:24.863 INFO:tasks.workunit.client.1.smithi115.stdout:4/879: read d7/dc/d1a/d21/db1/d56/d63/f6e [3827927,14157] 0 2021-11-03T17:15:24.863 INFO:tasks.workunit.client.1.smithi115.stdout:4/880: truncate d7/dc/d6a/d3c/dc5/dcc/ffd 131885 0 2021-11-03T17:15:24.864 INFO:tasks.workunit.client.1.smithi115.stdout:0/890: dwrite d9/d15/d1a/d1d/dcc/d6e/f72 [0,4194304] 0 2021-11-03T17:15:24.871 INFO:tasks.workunit.client.1.smithi115.stdout:1/752: link d7/dc/dd/d3e/d58/dc2/fec d7/dc/dd/d3e/dd1/d4d/d60/db2/ff0 0 2021-11-03T17:15:24.871 INFO:tasks.workunit.client.1.smithi115.stdout:2/692: creat d4/d6/d56/fee x:0 0 0 2021-11-03T17:15:24.879 INFO:tasks.workunit.client.1.smithi115.stdout:4/881: dwrite d7/dc/fb0 [0,4194304] 0 2021-11-03T17:15:24.879 INFO:tasks.workunit.client.1.smithi115.stdout:1/753: dread d7/dc/dd/f95 [0,4194304] 0 2021-11-03T17:15:24.888 INFO:tasks.workunit.client.1.smithi115.stdout:0/891: symlink d9/d15/l126 0 2021-11-03T17:15:24.889 INFO:tasks.workunit.client.1.smithi115.stdout:4/882: creat d7/dc/d6a/d101/f129 x:0 0 0 2021-11-03T17:15:24.890 INFO:tasks.workunit.client.1.smithi115.stdout:4/883: chown d7 15 1 2021-11-03T17:15:24.895 INFO:tasks.workunit.client.1.smithi115.stdout:8/662: rename d4/d6/d3f/da4/le0 to d4/d6/d18/d30/d79/d7f/led 0 2021-11-03T17:15:24.896 INFO:tasks.workunit.client.1.smithi115.stdout:2/693: symlink d4/d6/d56/d6c/lef 0 2021-11-03T17:15:24.898 INFO:tasks.workunit.client.1.smithi115.stdout:0/892: truncate d9/d15/d1a/d1d/dcc/d6e/fa3 2760902 0 2021-11-03T17:15:24.911 INFO:tasks.workunit.client.1.smithi115.stdout:8/663: dwrite d4/d7d/fa0 [0,4194304] 0 2021-11-03T17:15:24.919 INFO:tasks.workunit.client.1.smithi115.stdout:4/884: rename d7/d7a/d83/l73 to d7/dc/d1a/d21/db1/d56/df4/l12a 0 2021-11-03T17:15:24.920 INFO:tasks.workunit.client.1.smithi115.stdout:4/885: chown d7/dc/d1a/d21/db1/ffc 56125 1 2021-11-03T17:15:24.926 INFO:tasks.workunit.client.1.smithi115.stdout:0/893: creat d9/db/d83/f127 x:0 0 0 2021-11-03T17:15:24.928 INFO:tasks.workunit.client.1.smithi115.stdout:8/664: mknod d4/d6/d3f/da4/cee 1 2021-11-03T17:15:24.929 INFO:tasks.workunit.client.1.smithi115.stdout:1/754: link d7/dc/dd/d3e/dd1/d4d/d60/lc1 d7/dc/dd/d3e/dd1/d26/lf1 0 2021-11-03T17:15:24.932 INFO:tasks.workunit.client.1.smithi115.stdout:4/886: mkdir d7/dc/d6a/d116/d12b 0 2021-11-03T17:15:24.936 INFO:tasks.workunit.client.1.smithi115.stdout:8/665: link d4/d7d/l96 d4/d6/d3f/d43/d88/de3/lef 0 2021-11-03T17:15:24.937 INFO:tasks.workunit.client.1.smithi115.stdout:8/666: fdatasync d4/d6/d3f/da4/fbc 0 2021-11-03T17:15:24.938 INFO:tasks.workunit.client.1.smithi115.stdout:1/755: creat d7/d38/d59/ff2 x:0 0 0 2021-11-03T17:15:24.938 INFO:tasks.workunit.client.1.smithi115.stdout:8/667: write d4/d6/d18/d30/d33/f5c [4117014,121723] 0 2021-11-03T17:15:24.939 INFO:tasks.workunit.client.1.smithi115.stdout:8/668: stat d4/d6/d18/d30/d79/d7f 0 2021-11-03T17:15:24.939 INFO:tasks.workunit.client.1.smithi115.stdout:8/669: mknod d4/d6/d3f/dde/cf0 1 2021-11-03T17:15:24.939 INFO:tasks.workunit.client.1.smithi115.stdout:8/670: getdents d4/d6/d18/d44/d47/deb 0 2021-11-03T17:15:24.940 INFO:tasks.workunit.client.1.smithi115.stdout:4/887: link d7/d7a/fa3 d7/dc/d6a/d101/f12c 0 2021-11-03T17:15:24.943 INFO:tasks.workunit.client.1.smithi115.stdout:2/694: dread d4/d6/d56/d6c/d34/f52 [0,4194304] 0 2021-11-03T17:15:24.943 INFO:tasks.workunit.client.1.smithi115.stdout:8/671: link d4/d6/d18/d44/d6e/f9f d4/d6/d18/d30/d79/d7f/ff1 0 2021-11-03T17:15:24.943 INFO:tasks.workunit.client.1.smithi115.stdout:4/888: write d7/dc/d6a/f6f [233504,13791] 0 2021-11-03T17:15:24.944 INFO:tasks.workunit.client.1.smithi115.stdout:4/889: dread - d7/dc/d1a/d21/db1/d56/d63/f111 zero size 2021-11-03T17:15:24.944 INFO:tasks.workunit.client.1.smithi115.stdout:4/890: chown d7/dc/d1a/d21/db1 1390 1 2021-11-03T17:15:24.950 INFO:tasks.workunit.client.1.smithi115.stdout:0/894: dwrite d9/d15/d1a/d1d/dcc/d46/d63/dae/fed [0,4194304] 0 2021-11-03T17:15:24.952 INFO:tasks.workunit.client.1.smithi115.stdout:8/672: readlink d4/d6/d18/l4d 0 2021-11-03T17:15:24.952 INFO:tasks.workunit.client.1.smithi115.stdout:2/695: rmdir d4/d6 39 2021-11-03T17:15:24.964 INFO:tasks.workunit.client.1.smithi115.stdout:0/895: creat d9/d15/d1a/d1d/dcc/d46/d8c/dfc/f128 x:0 0 0 2021-11-03T17:15:24.967 INFO:tasks.workunit.client.1.smithi115.stdout:2/696: dwrite d4/d6/d56/d6c/d34/d43/d63/fdf [0,4194304] 0 2021-11-03T17:15:24.970 INFO:tasks.workunit.client.1.smithi115.stdout:8/673: symlink d4/d6/lf2 0 2021-11-03T17:15:24.978 INFO:tasks.workunit.client.1.smithi115.stdout:0/896: dread d9/d15/d1a/d1d/dcc/d46/d8c/f49 [0,4194304] 0 2021-11-03T17:15:24.980 INFO:tasks.workunit.client.1.smithi115.stdout:8/674: creat d4/d6/d3f/dde/ff3 x:0 0 0 2021-11-03T17:15:24.982 INFO:tasks.workunit.client.1.smithi115.stdout:4/891: dwrite d7/dc/f13 [4194304,4194304] 0 2021-11-03T17:15:24.983 INFO:tasks.workunit.client.1.smithi115.stdout:4/892: chown d7/dc/d1a/d21/f105 22305 1 2021-11-03T17:15:24.983 INFO:tasks.workunit.client.1.smithi115.stdout:4/893: dread - d7/dc/d1a/d21/db1/d56/d63/d7b/fff zero size 2021-11-03T17:15:24.983 INFO:tasks.workunit.client.1.smithi115.stdout:0/897: mknod d9/d15/d90/c129 1 2021-11-03T17:15:24.989 INFO:tasks.workunit.client.1.smithi115.stdout:0/898: dread d9/d15/d1a/d1d/dcc/d85/f115 [0,4194304] 0 2021-11-03T17:15:24.997 INFO:tasks.workunit.client.1.smithi115.stdout:4/894: mkdir d7/dc/d1a/d21/db1/d56/d42/de6/d12d 0 2021-11-03T17:15:24.998 INFO:tasks.workunit.client.1.smithi115.stdout:0/899: dread d9/d15/d1a/f92 [0,4194304] 0 2021-11-03T17:15:25.001 INFO:tasks.workunit.client.1.smithi115.stdout:4/895: creat d7/dc/d6a/f12e x:0 0 0 2021-11-03T17:15:25.002 INFO:tasks.workunit.client.1.smithi115.stdout:4/896: dread - d7/dc/d1a/f11e zero size 2021-11-03T17:15:25.005 INFO:tasks.workunit.client.1.smithi115.stdout:0/900: mkdir d9/d15/d1a/da7/d12a 0 2021-11-03T17:15:25.008 INFO:tasks.workunit.client.1.smithi115.stdout:0/901: symlink d9/db/l12b 0 2021-11-03T17:15:25.010 INFO:tasks.workunit.client.1.smithi115.stdout:0/902: read d9/d15/d1a/d1d/dcc/d6e/fa3 [102013,107604] 0 2021-11-03T17:15:25.012 INFO:tasks.workunit.client.1.smithi115.stdout:4/897: dread d7/dc/d6a/d3c/dc5/ffa [0,4194304] 0 2021-11-03T17:15:25.022 INFO:tasks.workunit.client.1.smithi115.stdout:2/697: dwrite d4/d6/d13/f4e [0,4194304] 0 2021-11-03T17:15:25.027 INFO:tasks.workunit.client.1.smithi115.stdout:2/698: creat d4/d6/dad/de8/dac/dec/ff0 x:0 0 0 2021-11-03T17:15:25.027 INFO:tasks.workunit.client.1.smithi115.stdout:2/699: fsync d4/d6/d13/d75/fe0 0 2021-11-03T17:15:25.028 INFO:tasks.workunit.client.1.smithi115.stdout:0/903: dread d9/d15/d1a/d1d/dcc/d6e/f72 [0,4194304] 0 2021-11-03T17:15:25.038 INFO:tasks.workunit.client.1.smithi115.stdout:2/700: dread d4/d6/fbd [0,4194304] 0 2021-11-03T17:15:25.038 INFO:tasks.workunit.client.1.smithi115.stdout:2/701: chown d4/d6/f2a 1625378 1 2021-11-03T17:15:25.040 INFO:tasks.workunit.client.1.smithi115.stdout:4/898: dwrite d7/dc/d1a/f38 [0,4194304] 0 2021-11-03T17:15:25.041 INFO:tasks.workunit.client.1.smithi115.stdout:4/899: chown d7/dc/d6a/fd3 430945944 1 2021-11-03T17:15:25.041 INFO:tasks.workunit.client.1.smithi115.stdout:2/702: rmdir d4/d6/d56/d6c/d34/ded 0 2021-11-03T17:15:25.049 INFO:tasks.workunit.client.1.smithi115.stdout:4/900: symlink d7/dc/d6a/d101/l12f 0 2021-11-03T17:15:25.051 INFO:tasks.workunit.client.1.smithi115.stdout:0/904: dwrite d9/db/d83/f127 [0,4194304] 0 2021-11-03T17:15:25.052 INFO:tasks.workunit.client.1.smithi115.stdout:4/901: dread d7/dc/d1a/d21/db1/d56/d42/fa5 [0,4194304] 0 2021-11-03T17:15:25.054 INFO:tasks.workunit.client.1.smithi115.stdout:2/703: dread d4/d6/d56/d6c/d34/d43/f2b [0,4194304] 0 2021-11-03T17:15:25.071 INFO:tasks.workunit.client.1.smithi115.stdout:0/905: truncate d9/d15/d78/d80/d82/fe6 4951239 0 2021-11-03T17:15:25.075 INFO:tasks.workunit.client.1.smithi115.stdout:2/704: mknod d4/d6/dad/de8/d9c/d9f/cf1 1 2021-11-03T17:15:25.081 INFO:tasks.workunit.client.1.smithi115.stdout:2/705: creat d4/d6/d56/d6c/d34/ff2 x:0 0 0 2021-11-03T17:15:25.083 INFO:tasks.workunit.client.1.smithi115.stdout:2/706: readlink d4/d6/d13/l88 0 2021-11-03T17:15:25.095 INFO:tasks.workunit.client.1.smithi115.stdout:2/707: rename d4/d6/d56/fee to d4/d6/dad/de8/dac/ff3 0 2021-11-03T17:15:25.108 INFO:tasks.workunit.client.1.smithi115.stdout:2/708: dread d4/f78 [0,4194304] 0 2021-11-03T17:15:25.111 INFO:tasks.workunit.client.1.smithi115.stdout:2/709: stat d4/d6/dad/d7f/f8e 0 2021-11-03T17:15:25.124 INFO:tasks.workunit.client.1.smithi115.stdout:2/710: dread d4/d6/d56/d6c/d34/d42/d8d/fce [0,4194304] 0 2021-11-03T17:15:25.140 INFO:tasks.workunit.client.1.smithi115.stdout:2/711: dread d4/d6/dad/f8b [0,4194304] 0 2021-11-03T17:15:25.140 INFO:tasks.workunit.client.1.smithi115.stdout:2/712: dread - d4/d6/d56/d6c/d34/d84/fe7 zero size 2021-11-03T17:15:25.148 INFO:tasks.workunit.client.1.smithi115.stdout:2/713: link d4/d6/dad/de8/d82/l90 d4/d6/d56/d6c/d34/d43/lf4 0 2021-11-03T17:15:25.158 INFO:tasks.workunit.client.1.smithi115.stdout:6/590: sync 2021-11-03T17:15:25.191 INFO:tasks.workunit.client.1.smithi115.stdout:2/714: symlink d4/d6/d13/d75/lf5 0 2021-11-03T17:15:25.196 INFO:tasks.workunit.client.1.smithi115.stdout:6/591: rename d0/d1/d6a/d88/da0/db5/fbd to d0/d74/fc8 0 2021-11-03T17:15:25.200 INFO:tasks.workunit.client.1.smithi115.stdout:6/592: read d0/d1/d10/d13/f39 [2754421,120281] 0 2021-11-03T17:15:25.203 INFO:tasks.workunit.client.1.smithi115.stdout:2/715: creat d4/d6/d56/d6c/d34/d43/d63/d77/ff6 x:0 0 0 2021-11-03T17:15:25.210 INFO:tasks.workunit.client.1.smithi115.stdout:6/593: rename d0/d1/d2e/d4c/d96/d61/l78 to d0/d1/d10/d13/d59/d95/lc9 0 2021-11-03T17:15:25.210 INFO:tasks.workunit.client.1.smithi115.stdout:2/716: mknod d4/d6/d13/cf7 1 2021-11-03T17:15:25.218 INFO:tasks.workunit.client.1.smithi115.stdout:6/594: mkdir d0/d1/d2e/d4c/dca 0 2021-11-03T17:15:25.220 INFO:tasks.workunit.client.1.smithi115.stdout:6/595: symlink d0/d1/d6a/d88/da0/lcb 0 2021-11-03T17:15:25.235 INFO:tasks.workunit.client.1.smithi115.stdout:2/717: creat d4/d6/dad/de8/d82/ff8 x:0 0 0 2021-11-03T17:15:25.238 INFO:tasks.workunit.client.1.smithi115.stdout:2/718: symlink d4/d6/d13/d75/lf9 0 2021-11-03T17:15:25.241 INFO:tasks.workunit.client.1.smithi115.stdout:2/719: creat d4/d6/d13/ffa x:0 0 0 2021-11-03T17:15:25.253 INFO:tasks.workunit.client.1.smithi115.stdout:6/596: dwrite d0/d1/d2/d3f/f49 [0,4194304] 0 2021-11-03T17:15:25.255 INFO:tasks.workunit.client.1.smithi115.stdout:6/597: write d0/d1/d10/d13/d1c/f36 [13068543,79268] 0 2021-11-03T17:15:25.256 INFO:tasks.workunit.client.1.smithi115.stdout:7/621: sync 2021-11-03T17:15:25.257 INFO:tasks.workunit.client.1.smithi115.stdout:6/598: mkdir d0/d1/d10/d13/d1c/d1e/dcc 0 2021-11-03T17:15:25.259 INFO:tasks.workunit.client.1.smithi115.stdout:6/599: rmdir d0/d74 39 2021-11-03T17:15:25.270 INFO:tasks.workunit.client.1.smithi115.stdout:6/600: creat d0/d7f/fcd x:0 0 0 2021-11-03T17:15:25.271 INFO:tasks.workunit.client.1.smithi115.stdout:6/601: write d0/d1/d10/d13/d59/fa3 [3225534,51695] 0 2021-11-03T17:15:25.280 INFO:tasks.workunit.client.1.smithi115.stdout:6/602: write d0/d1/d10/f3b [829607,89290] 0 2021-11-03T17:15:25.284 INFO:tasks.workunit.client.1.smithi115.stdout:2/720: dwrite d4/d6/d56/d6c/d34/d43/d63/d77/f81 [0,4194304] 0 2021-11-03T17:15:25.289 INFO:tasks.workunit.client.1.smithi115.stdout:2/721: symlink d4/d44/lfb 0 2021-11-03T17:15:25.290 INFO:tasks.workunit.client.1.smithi115.stdout:2/722: truncate d4/d6/dad/de8/d82/ff8 742081 0 2021-11-03T17:15:25.293 INFO:tasks.workunit.client.1.smithi115.stdout:2/723: unlink d4/d6/dad/de8/dac/ff3 0 2021-11-03T17:15:25.301 INFO:tasks.workunit.client.1.smithi115.stdout:6/603: dwrite d0/d1/d2/f1f [8388608,4194304] 0 2021-11-03T17:15:25.306 INFO:tasks.workunit.client.1.smithi115.stdout:6/604: write d0/d1/d10/d13/d87/d89/faf [2649483,97792] 0 2021-11-03T17:15:25.308 INFO:tasks.workunit.client.1.smithi115.stdout:2/724: dwrite d4/d6/dad/d7f/feb [0,4194304] 0 2021-11-03T17:15:25.309 INFO:tasks.workunit.client.1.smithi115.stdout:6/605: fdatasync d0/d1/f8 0 2021-11-03T17:15:25.324 INFO:tasks.workunit.client.1.smithi115.stdout:2/725: getdents d4/d6/d56/d6c/d34/d43/d63/d77 0 2021-11-03T17:15:25.336 INFO:tasks.workunit.client.1.smithi115.stdout:6/606: fsync d0/d1/f19 0 2021-11-03T17:15:25.351 INFO:tasks.workunit.client.1.smithi115.stdout:6/607: write d0/f7e [194942,104789] 0 2021-11-03T17:15:25.351 INFO:tasks.workunit.client.1.smithi115.stdout:6/608: mknod d0/d1/d10/d13/d1c/d1e/dcc/cce 1 2021-11-03T17:15:25.352 INFO:tasks.workunit.client.1.smithi115.stdout:6/609: mknod d0/d1/d2/d3f/ccf 1 2021-11-03T17:15:25.355 INFO:tasks.workunit.client.1.smithi115.stdout:6/610: creat d0/d1/d2/d3f/fd0 x:0 0 0 2021-11-03T17:15:25.393 INFO:tasks.workunit.client.1.smithi115.stdout:6/611: dwrite d0/d1/d2e/f8e [0,4194304] 0 2021-11-03T17:15:25.396 INFO:tasks.workunit.client.1.smithi115.stdout:6/612: symlink d0/d1/d2e/d4c/d96/d61/ld1 0 2021-11-03T17:15:25.397 INFO:tasks.workunit.client.1.smithi115.stdout:6/613: chown d0/d1/d10/d13/d59/d95 864 1 2021-11-03T17:15:25.402 INFO:tasks.workunit.client.1.smithi115.stdout:6/614: mkdir d0/d1/d10/d13/d87/dd2 0 2021-11-03T17:15:25.411 INFO:tasks.workunit.client.1.smithi115.stdout:6/615: dread d0/d1/d2/fd [0,4194304] 0 2021-11-03T17:15:25.418 INFO:tasks.workunit.client.1.smithi115.stdout:6/616: dread d0/fa2 [0,4194304] 0 2021-11-03T17:15:25.420 INFO:tasks.workunit.client.1.smithi115.stdout:6/617: symlink d0/d7f/dab/ld3 0 2021-11-03T17:15:25.422 INFO:tasks.workunit.client.1.smithi115.stdout:6/618: chown d0/d1/l30 35 1 2021-11-03T17:15:25.436 INFO:tasks.workunit.client.1.smithi115.stdout:3/845: sync 2021-11-03T17:15:25.436 INFO:tasks.workunit.client.1.smithi115.stdout:3/846: fsync d13/d2a/d8a/fa6 0 2021-11-03T17:15:25.436 INFO:tasks.workunit.client.1.smithi115.stdout:3/847: write d13/d35/d54/dad/f100 [608277,71882] 0 2021-11-03T17:15:25.442 INFO:tasks.workunit.client.1.smithi115.stdout:3/848: chown d13/d2a/d60/dd3 0 1 2021-11-03T17:15:25.448 INFO:tasks.workunit.client.1.smithi115.stdout:3/849: mkdir d13/da4/d122 0 2021-11-03T17:15:25.452 INFO:tasks.workunit.client.1.smithi115.stdout:3/850: write d13/f4c [7705945,81161] 0 2021-11-03T17:15:25.457 INFO:tasks.workunit.client.1.smithi115.stdout:6/619: dwrite d0/d1/d10/d13/d1c/f2f [0,4194304] 0 2021-11-03T17:15:25.460 INFO:tasks.workunit.client.1.smithi115.stdout:6/620: rmdir d0/d1/d10/d13/d59/d95 39 2021-11-03T17:15:25.465 INFO:tasks.workunit.client.1.smithi115.stdout:6/621: dread d0/d1/d2/f4a [0,4194304] 0 2021-11-03T17:15:25.469 INFO:tasks.workunit.client.1.smithi115.stdout:3/851: write d13/d50/f42 [393385,6812] 0 2021-11-03T17:15:25.475 INFO:tasks.workunit.client.1.smithi115.stdout:3/852: dread d13/d51/d40/ff7 [0,4194304] 0 2021-11-03T17:15:25.476 INFO:tasks.workunit.client.1.smithi115.stdout:6/622: dread d0/d1/d2/d3f/f51 [0,4194304] 0 2021-11-03T17:15:25.478 INFO:tasks.workunit.client.1.smithi115.stdout:6/623: dread d0/d1/d10/d13/d59/f9c [0,4194304] 0 2021-11-03T17:15:25.482 INFO:tasks.workunit.client.1.smithi115.stdout:6/624: symlink d0/d1/d10/d13/ld4 0 2021-11-03T17:15:25.484 INFO:tasks.workunit.client.1.smithi115.stdout:3/853: write d13/d2c/ffb [1146403,6425] 0 2021-11-03T17:15:25.485 INFO:tasks.workunit.client.1.smithi115.stdout:6/625: truncate d0/d1/f19 1673149 0 2021-11-03T17:15:25.486 INFO:tasks.workunit.client.1.smithi115.stdout:6/626: fsync d0/d1/d2e/f8e 0 2021-11-03T17:15:25.487 INFO:tasks.workunit.client.1.smithi115.stdout:6/627: creat d0/d74/fd5 x:0 0 0 2021-11-03T17:15:25.490 INFO:tasks.workunit.client.1.smithi115.stdout:3/854: write d13/d35/f36 [8046004,19431] 0 2021-11-03T17:15:25.491 INFO:tasks.workunit.client.1.smithi115.stdout:6/628: truncate d0/d1/d2e/fb9 2582429 0 2021-11-03T17:15:25.496 INFO:tasks.workunit.client.1.smithi115.stdout:6/629: symlink d0/d7f/dab/ld6 0 2021-11-03T17:15:25.503 INFO:tasks.workunit.client.1.smithi115.stdout:6/630: mkdir d0/d1/d10/d13/d87/dd2/dd7 0 2021-11-03T17:15:25.534 INFO:tasks.workunit.client.1.smithi115.stdout:6/631: dwrite d0/d1/d2/d3f/f49 [0,4194304] 0 2021-11-03T17:15:25.535 INFO:tasks.workunit.client.1.smithi115.stdout:6/632: getdents d0/d1/d2e/d4c/dca 0 2021-11-03T17:15:25.536 INFO:tasks.workunit.client.1.smithi115.stdout:3/855: dwrite d13/d51/d40/f9d [0,4194304] 0 2021-11-03T17:15:25.555 INFO:tasks.workunit.client.1.smithi115.stdout:6/633: dwrite d0/d1/d2/d3f/fc0 [0,4194304] 0 2021-11-03T17:15:25.555 INFO:tasks.workunit.client.1.smithi115.stdout:3/856: rename f10 to d13/d3b/f123 0 2021-11-03T17:15:25.563 INFO:tasks.workunit.client.1.smithi115.stdout:6/634: unlink d0/d1/d2/d3f/l81 0 2021-11-03T17:15:25.574 INFO:tasks.workunit.client.1.smithi115.stdout:6/635: mkdir d0/d1/d6a/d88/dd8 0 2021-11-03T17:15:25.584 INFO:tasks.workunit.client.1.smithi115.stdout:3/857: dwrite d13/f25 [0,4194304] 0 2021-11-03T17:15:25.596 INFO:tasks.workunit.client.1.smithi115.stdout:3/858: rmdir d13/da4/db2 39 2021-11-03T17:15:25.597 INFO:tasks.workunit.client.1.smithi115.stdout:3/859: truncate d13/d3b/db1/f107 1488312 0 2021-11-03T17:15:25.597 INFO:tasks.workunit.client.1.smithi115.stdout:6/636: dwrite d0/d1/d2/f58 [0,4194304] 0 2021-11-03T17:15:25.598 INFO:tasks.workunit.client.1.smithi115.stdout:6/637: chown d0/d1/d10/d13/d87/d89/faf 3372 1 2021-11-03T17:15:25.604 INFO:tasks.workunit.client.1.smithi115.stdout:3/860: write d13/d51/d40/f88 [1057712,121395] 0 2021-11-03T17:15:25.605 INFO:tasks.workunit.client.1.smithi115.stdout:3/861: write d13/d51/d2e/db4/fe7 [312754,29568] 0 2021-11-03T17:15:25.606 INFO:tasks.workunit.client.1.smithi115.stdout:6/638: rmdir d0/d74 39 2021-11-03T17:15:25.607 INFO:tasks.workunit.client.1.smithi115.stdout:6/639: chown d0/d1/d10/f99 1 1 2021-11-03T17:15:25.611 INFO:tasks.workunit.client.1.smithi115.stdout:3/862: dread d13/da4/ff0 [0,4194304] 0 2021-11-03T17:15:25.612 INFO:tasks.workunit.client.1.smithi115.stdout:6/640: symlink d0/ld9 0 2021-11-03T17:15:25.612 INFO:tasks.workunit.client.1.smithi115.stdout:6/641: fsync d0/d1/d10/d13/d87/fa4 0 2021-11-03T17:15:25.613 INFO:tasks.workunit.client.1.smithi115.stdout:6/642: dread d0/d1/d2/d3f/f51 [0,4194304] 0 2021-11-03T17:15:25.613 INFO:tasks.workunit.client.1.smithi115.stdout:6/643: chown d0/d1/d10/f9a 20370700 1 2021-11-03T17:15:25.614 INFO:tasks.workunit.client.1.smithi115.stdout:6/644: mknod d0/d85/cda 1 2021-11-03T17:15:25.614 INFO:tasks.workunit.client.1.smithi115.stdout:3/863: creat d13/d2a/d53/f124 x:0 0 0 2021-11-03T17:15:25.615 INFO:tasks.workunit.client.1.smithi115.stdout:3/864: dread - d13/d2a/f10f zero size 2021-11-03T17:15:25.620 INFO:tasks.workunit.client.1.smithi115.stdout:3/865: creat d13/d50/d10b/f125 x:0 0 0 2021-11-03T17:15:25.628 INFO:tasks.workunit.client.1.smithi115.stdout:3/866: dread d13/f27 [0,4194304] 0 2021-11-03T17:15:25.640 INFO:tasks.workunit.client.1.smithi115.stdout:3/867: symlink d13/da4/dcb/l126 0 2021-11-03T17:15:25.641 INFO:tasks.workunit.client.1.smithi115.stdout:3/868: dread - d13/d50/d10b/f125 zero size 2021-11-03T17:15:25.655 INFO:tasks.workunit.client.1.smithi115.stdout:6/645: dwrite d0/d1/d10/f99 [0,4194304] 0 2021-11-03T17:15:25.657 INFO:tasks.workunit.client.1.smithi115.stdout:3/869: dwrite d13/d2a/d8a/fc8 [0,4194304] 0 2021-11-03T17:15:25.658 INFO:tasks.workunit.client.1.smithi115.stdout:6/646: link d0/d1/d10/l26 d0/d1/d10/d13/d87/d89/ldb 0 2021-11-03T17:15:25.659 INFO:tasks.workunit.client.1.smithi115.stdout:6/647: getdents d0/d1/d2 0 2021-11-03T17:15:25.660 INFO:tasks.workunit.client.1.smithi115.stdout:3/870: read d13/d2a/d8a/fd0 [3941796,126567] 0 2021-11-03T17:15:25.663 INFO:tasks.workunit.client.1.smithi115.stdout:6/648: unlink d0/d1/d10/d13/d1c/d1e/f47 0 2021-11-03T17:15:25.699 INFO:tasks.workunit.client.1.smithi115.stdout:3/871: dwrite d13/d51/f41 [4194304,4194304] 0 2021-11-03T17:15:25.717 INFO:tasks.workunit.client.1.smithi115.stdout:3/872: dwrite d13/d2a/d53/f8c [0,4194304] 0 2021-11-03T17:15:25.717 INFO:tasks.workunit.client.1.smithi115.stdout:3/873: chown d13/f22 60 1 2021-11-03T17:15:25.731 INFO:tasks.workunit.client.1.smithi115.stdout:3/874: dread d13/d67/f87 [0,4194304] 0 2021-11-03T17:15:25.731 INFO:tasks.workunit.client.1.smithi115.stdout:3/875: chown d13/f7c 13494952 1 2021-11-03T17:15:25.733 INFO:tasks.workunit.client.1.smithi115.stdout:3/876: mknod d13/da4/c127 1 2021-11-03T17:15:25.734 INFO:tasks.workunit.client.1.smithi115.stdout:3/877: creat d13/d51/d2e/dbf/f128 x:0 0 0 2021-11-03T17:15:25.743 INFO:tasks.workunit.client.1.smithi115.stdout:3/878: dread d13/d2a/d8a/f94 [0,4194304] 0 2021-11-03T17:15:25.744 INFO:tasks.workunit.client.1.smithi115.stdout:3/879: write d13/d51/d40/f9d [5211830,121826] 0 2021-11-03T17:15:25.745 INFO:tasks.workunit.client.1.smithi115.stdout:3/880: mkdir d13/d3b/d11f/d129 0 2021-11-03T17:15:26.101 INFO:tasks.workunit.client.1.smithi115.stdout:5/771: sync 2021-11-03T17:15:26.101 INFO:tasks.workunit.client.1.smithi115.stdout:9/688: sync 2021-11-03T17:15:26.102 INFO:tasks.workunit.client.1.smithi115.stdout:9/689: chown d1/d18/d2b 3037929 1 2021-11-03T17:15:26.102 INFO:tasks.workunit.client.1.smithi115.stdout:9/690: stat d1/d18/d4a/d74/dc7/fce 0 2021-11-03T17:15:26.107 INFO:tasks.workunit.client.1.smithi115.stdout:5/772: symlink d1a/d23/de6/l104 0 2021-11-03T17:15:26.109 INFO:tasks.workunit.client.1.smithi115.stdout:5/773: creat d1a/d28/d78/f105 x:0 0 0 2021-11-03T17:15:26.112 INFO:tasks.workunit.client.1.smithi115.stdout:5/774: mkdir d1a/d23/de6/d106 0 2021-11-03T17:15:26.115 INFO:tasks.workunit.client.1.smithi115.stdout:5/775: mkdir d1a/d28/d7b/d4f/d81/d107 0 2021-11-03T17:15:26.161 INFO:tasks.workunit.client.1.smithi115.stdout:5/776: write d1a/d28/d7b/d48/d7f/f86 [457018,104731] 0 2021-11-03T17:15:26.167 INFO:tasks.workunit.client.1.smithi115.stdout:5/777: creat d1a/d28/d4a/d65/f108 x:0 0 0 2021-11-03T17:15:26.167 INFO:tasks.workunit.client.1.smithi115.stdout:5/778: fdatasync d1a/d28/d7b/f77 0 2021-11-03T17:15:26.174 INFO:tasks.workunit.client.1.smithi115.stdout:5/779: rename d1a/d28/d4c/d75/d83/da1/ded to d1a/d28/daa/d109 0 2021-11-03T17:15:26.181 INFO:tasks.workunit.client.1.smithi115.stdout:5/780: dread - d1a/d23/f5c zero size 2021-11-03T17:15:26.213 INFO:tasks.workunit.client.1.smithi115.stdout:5/781: write d1a/d28/f91 [3578705,43407] 0 2021-11-03T17:15:26.214 INFO:tasks.workunit.client.1.smithi115.stdout:5/782: mknod d1a/d3d/d58/dab/daf/c10a 1 2021-11-03T17:15:26.225 INFO:tasks.workunit.client.1.smithi115.stdout:5/783: rename d1a/d3d/dc4/fd0 to d1a/d28/d4c/d75/d83/f10b 0 2021-11-03T17:15:26.229 INFO:tasks.workunit.client.1.smithi115.stdout:5/784: creat d1a/d23/de6/f10c x:0 0 0 2021-11-03T17:15:26.237 INFO:tasks.workunit.client.1.smithi115.stdout:5/785: unlink d1a/d28/d4c/d75/d68/da5/lbc 0 2021-11-03T17:15:26.241 INFO:tasks.workunit.client.1.smithi115.stdout:5/786: chown d1a/d3d/f47 1820129108 1 2021-11-03T17:15:26.262 INFO:tasks.workunit.client.1.smithi115.stdout:5/787: dwrite d1a/d28/d7b/d48/d7f/f94 [0,4194304] 0 2021-11-03T17:15:26.274 INFO:tasks.workunit.client.1.smithi115.stdout:5/788: mkdir d1a/d3d/d58/dab/d10d 0 2021-11-03T17:15:26.280 INFO:tasks.workunit.client.1.smithi115.stdout:5/789: mkdir d1a/d28/d4c/d10e 0 2021-11-03T17:15:26.280 INFO:tasks.workunit.client.1.smithi115.stdout:5/790: chown d1a/d28/l8c 21287 1 2021-11-03T17:15:26.283 INFO:tasks.workunit.client.1.smithi115.stdout:5/791: mkdir d1a/d10f 0 2021-11-03T17:15:26.283 INFO:tasks.workunit.client.1.smithi115.stdout:5/792: write d1a/d28/d4c/d75/d68/fd2 [191423,41387] 0 2021-11-03T17:15:26.285 INFO:tasks.workunit.client.1.smithi115.stdout:5/793: creat d1a/d28/daa/d109/f110 x:0 0 0 2021-11-03T17:15:26.291 INFO:tasks.workunit.client.1.smithi115.stdout:5/794: link d1a/d3d/d58/dab/daf/ff8 d1a/d28/daa/f111 0 2021-11-03T17:15:26.291 INFO:tasks.workunit.client.1.smithi115.stdout:5/795: chown d1a/d28/d4c/d75/dae/fce 372924 1 2021-11-03T17:15:26.294 INFO:tasks.workunit.client.1.smithi115.stdout:5/796: getdents d1a/d28/d4a/d65 0 2021-11-03T17:15:26.302 INFO:tasks.workunit.client.1.smithi115.stdout:5/797: dread d1a/d28/d7b/f77 [0,4194304] 0 2021-11-03T17:15:26.334 INFO:tasks.workunit.client.1.smithi115.stdout:5/798: dwrite d1a/d28/d34/f66 [0,4194304] 0 2021-11-03T17:15:26.339 INFO:tasks.workunit.client.1.smithi115.stdout:5/799: symlink d1a/d3d/d58/dab/dd3/l112 0 2021-11-03T17:15:26.340 INFO:tasks.workunit.client.1.smithi115.stdout:5/800: chown d1a/d3d/d58/fb9 397540072 1 2021-11-03T17:15:26.342 INFO:tasks.workunit.client.1.smithi115.stdout:5/801: read d1a/d28/d7b/fd8 [268360,58898] 0 2021-11-03T17:15:26.350 INFO:tasks.workunit.client.1.smithi115.stdout:5/802: chown d1a/d28/d4c/d63/l87 100939 1 2021-11-03T17:15:26.350 INFO:tasks.workunit.client.1.smithi115.stdout:5/803: chown d1a/d28/d7b/f9f 21925 1 2021-11-03T17:15:26.350 INFO:tasks.workunit.client.1.smithi115.stdout:5/804: write d1a/d3d/d58/dab/daf/ff8 [762553,58096] 0 2021-11-03T17:15:26.353 INFO:tasks.workunit.client.1.smithi115.stdout:5/805: link d1a/d28/d7b/d48/ld6 d1a/d28/d7b/l113 0 2021-11-03T17:15:26.353 INFO:tasks.workunit.client.1.smithi115.stdout:5/806: readlink d1a/d28/d4a/d65/db5/l70 0 2021-11-03T17:15:26.368 INFO:tasks.workunit.client.1.smithi115.stdout:5/807: dwrite d1a/d3d/f3b [0,4194304] 0 2021-11-03T17:15:26.376 INFO:tasks.workunit.client.1.smithi115.stdout:5/808: fdatasync d1a/d28/d4a/d65/db5/f8f 0 2021-11-03T17:15:26.378 INFO:tasks.workunit.client.1.smithi115.stdout:5/809: dread - d1a/d28/d7b/d48/fb1 zero size 2021-11-03T17:15:26.379 INFO:tasks.workunit.client.1.smithi115.stdout:5/810: truncate d1a/d28/d7b/d48/dd4/fff 668839 0 2021-11-03T17:15:26.577 INFO:tasks.workunit.client.1.smithi115.stdout:0/906: sync 2021-11-03T17:15:26.578 INFO:tasks.workunit.client.1.smithi115.stdout:8/675: sync 2021-11-03T17:15:26.578 INFO:tasks.workunit.client.1.smithi115.stdout:7/622: sync 2021-11-03T17:15:26.578 INFO:tasks.workunit.client.1.smithi115.stdout:4/902: sync 2021-11-03T17:15:26.579 INFO:tasks.workunit.client.1.smithi115.stdout:7/623: write d3/f9 [542742,103264] 0 2021-11-03T17:15:26.579 INFO:tasks.workunit.client.1.smithi115.stdout:7/624: fsync d3/f98 0 2021-11-03T17:15:26.582 INFO:tasks.workunit.client.1.smithi115.stdout:0/907: truncate d9/d15/d1a/f68 854427 0 2021-11-03T17:15:26.586 INFO:tasks.workunit.client.1.smithi115.stdout:8/676: creat d4/d7d/ff4 x:0 0 0 2021-11-03T17:15:26.591 INFO:tasks.workunit.client.1.smithi115.stdout:4/903: mkdir d7/dc/d6a/d3c/d112/d130 0 2021-11-03T17:15:26.591 INFO:tasks.workunit.client.1.smithi115.stdout:4/904: mknod d7/dc/d1a/d21/db1/c131 1 2021-11-03T17:15:26.593 INFO:tasks.workunit.client.1.smithi115.stdout:0/908: getdents d9/d15/d1a/d1d/dcc/d46/d63/dc5 0 2021-11-03T17:15:26.602 INFO:tasks.workunit.client.1.smithi115.stdout:7/625: link d3/da/d18/d73/d35/f9d d3/da/d18/d73/fc2 0 2021-11-03T17:15:26.602 INFO:tasks.workunit.client.1.smithi115.stdout:7/626: read - d3/da/d18/d73/d52/f83 zero size 2021-11-03T17:15:26.603 INFO:tasks.workunit.client.1.smithi115.stdout:8/677: symlink d4/d6/d3f/d43/lf5 0 2021-11-03T17:15:26.603 INFO:tasks.workunit.client.1.smithi115.stdout:8/678: read - d4/d6/d18/d44/d47/fd8 zero size 2021-11-03T17:15:26.603 INFO:tasks.workunit.client.1.smithi115.stdout:8/679: fsync d4/d6/d3f/d9a/f54 0 2021-11-03T17:15:26.607 INFO:tasks.workunit.client.1.smithi115.stdout:1/756: sync 2021-11-03T17:15:26.622 INFO:tasks.workunit.client.1.smithi115.stdout:1/757: creat d7/d38/ff3 x:0 0 0 2021-11-03T17:15:26.623 INFO:tasks.workunit.client.1.smithi115.stdout:1/758: write d7/dc/dd/d3e/dd1/d4d/d60/fb8 [1599898,69984] 0 2021-11-03T17:15:26.624 INFO:tasks.workunit.client.1.smithi115.stdout:7/627: symlink d3/d4c/lc3 0 2021-11-03T17:15:26.633 INFO:tasks.workunit.client.1.smithi115.stdout:1/759: rename d7/dc/dd/d3e/dd1/d4d to d7/dc/dd/d3e/dd1/d43/df4 0 2021-11-03T17:15:26.635 INFO:tasks.workunit.client.1.smithi115.stdout:2/726: sync 2021-11-03T17:15:26.635 INFO:tasks.workunit.client.1.smithi115.stdout:6/649: sync 2021-11-03T17:15:26.638 INFO:tasks.workunit.client.1.smithi115.stdout:7/628: symlink d3/d4c/lc4 0 2021-11-03T17:15:26.639 INFO:tasks.workunit.client.1.smithi115.stdout:8/680: dwrite d4/d6/f6b [0,4194304] 0 2021-11-03T17:15:26.640 INFO:tasks.workunit.client.1.smithi115.stdout:1/760: creat d7/dc/d1a/ff5 x:0 0 0 2021-11-03T17:15:26.648 INFO:tasks.workunit.client.1.smithi115.stdout:6/650: unlink d0/d7f/dab/ld3 0 2021-11-03T17:15:26.648 INFO:tasks.workunit.client.1.smithi115.stdout:6/651: fsync d0/d1/d10/f99 0 2021-11-03T17:15:26.649 INFO:tasks.workunit.client.1.smithi115.stdout:6/652: fdatasync d0/d1/d2/ff 0 2021-11-03T17:15:26.649 INFO:tasks.workunit.client.1.smithi115.stdout:6/653: write d0/d1/d2e/d4c/fb6 [971473,42031] 0 2021-11-03T17:15:26.651 INFO:tasks.workunit.client.1.smithi115.stdout:7/629: getdents d3/da/d18/d73/d35/da3 0 2021-11-03T17:15:26.653 INFO:tasks.workunit.client.1.smithi115.stdout:7/630: dread d3/da/d18/d73/d33/f9b [0,4194304] 0 2021-11-03T17:15:26.655 INFO:tasks.workunit.client.1.smithi115.stdout:8/681: unlink d4/d6/f8a 0 2021-11-03T17:15:26.666 INFO:tasks.workunit.client.1.smithi115.stdout:8/682: chown d4/d6/d18/d30/d33/l73 108618 1 2021-11-03T17:15:26.667 INFO:tasks.workunit.client.1.smithi115.stdout:6/654: dread d0/d1/d2e/d4c/f4e [0,4194304] 0 2021-11-03T17:15:26.667 INFO:tasks.workunit.client.1.smithi115.stdout:6/655: chown d0/d1/d2/fd 726 1 2021-11-03T17:15:26.667 INFO:tasks.workunit.client.1.smithi115.stdout:6/656: fsync d0/d7f/fcd 0 2021-11-03T17:15:26.670 INFO:tasks.workunit.client.1.smithi115.stdout:1/761: link d7/dc/dab/d73/d7f/fe6 d7/ff6 0 2021-11-03T17:15:26.673 INFO:tasks.workunit.client.1.smithi115.stdout:7/631: symlink d3/lc5 0 2021-11-03T17:15:26.674 INFO:tasks.workunit.client.1.smithi115.stdout:7/632: write d3/da/f64 [10355133,102857] 0 2021-11-03T17:15:26.680 INFO:tasks.workunit.client.1.smithi115.stdout:6/657: creat d0/d1/d6a/fdc x:0 0 0 2021-11-03T17:15:26.684 INFO:tasks.workunit.client.1.smithi115.stdout:2/727: rename d4/d6/d13/f2f to d4/d6/d56/d6c/d34/d43/d28/ffc 0 2021-11-03T17:15:26.687 INFO:tasks.workunit.client.1.smithi115.stdout:7/633: unlink d3/da/l95 0 2021-11-03T17:15:26.690 INFO:tasks.workunit.client.1.smithi115.stdout:2/728: fdatasync d4/d6/d13/f36 0 2021-11-03T17:15:26.693 INFO:tasks.workunit.client.1.smithi115.stdout:8/683: dread d4/d6/d18/d30/d33/fe9 [0,4194304] 0 2021-11-03T17:15:26.695 INFO:tasks.workunit.client.1.smithi115.stdout:1/762: rename d7/dc/dab/d73/d7f/fe6 to d7/dc/dd/ff7 0 2021-11-03T17:15:26.696 INFO:tasks.workunit.client.1.smithi115.stdout:1/763: dread - d7/fdd zero size 2021-11-03T17:15:26.701 INFO:tasks.workunit.client.1.smithi115.stdout:7/634: write d3/da/d18/d73/d33/f74 [1979338,77881] 0 2021-11-03T17:15:26.706 INFO:tasks.workunit.client.1.smithi115.stdout:2/729: mkdir d4/d6/d56/d6c/d34/d42/d8d/dfd 0 2021-11-03T17:15:26.711 INFO:tasks.workunit.client.1.smithi115.stdout:7/635: truncate d3/da/d18/faf 3247359 0 2021-11-03T17:15:26.711 INFO:tasks.workunit.client.1.smithi115.stdout:7/636: truncate d3/da/d78/fbe 966318 0 2021-11-03T17:15:26.712 INFO:tasks.workunit.client.1.smithi115.stdout:7/637: write d3/d4c/f7f [4015037,26012] 0 2021-11-03T17:15:26.713 INFO:tasks.workunit.client.1.smithi115.stdout:6/658: rename d0/d1/d10/d13/ld4 to d0/d7f/ldd 0 2021-11-03T17:15:26.714 INFO:tasks.workunit.client.1.smithi115.stdout:6/659: mknod d0/d1/d10/d13/d59/d95/db4/cde 1 2021-11-03T17:15:26.716 INFO:tasks.workunit.client.1.smithi115.stdout:2/730: symlink d4/d6/dad/d7f/de9/lfe 0 2021-11-03T17:15:26.717 INFO:tasks.workunit.client.1.smithi115.stdout:2/731: fsync d4/d6/d56/d6c/d34/d43/d63/d77/fa6 0 2021-11-03T17:15:26.720 INFO:tasks.workunit.client.1.smithi115.stdout:2/732: symlink d4/d6/d56/d6c/d34/d42/d8d/da9/lff 0 2021-11-03T17:15:26.729 INFO:tasks.workunit.client.1.smithi115.stdout:7/638: dwrite d3/da/d18/d73/d33/f9b [0,4194304] 0 2021-11-03T17:15:26.730 INFO:tasks.workunit.client.1.smithi115.stdout:7/639: chown d3 16682771 1 2021-11-03T17:15:26.730 INFO:tasks.workunit.client.1.smithi115.stdout:7/640: write d3/da/d18/d73/d52/fa9 [730393,83460] 0 2021-11-03T17:15:26.730 INFO:tasks.workunit.client.1.smithi115.stdout:7/641: mknod d3/da/d18/cc6 1 2021-11-03T17:15:26.730 INFO:tasks.workunit.client.1.smithi115.stdout:2/733: write d4/d6/fbd [118299,102834] 0 2021-11-03T17:15:26.731 INFO:tasks.workunit.client.1.smithi115.stdout:2/734: chown d4/d6/d56/d6c/d34/d42/d8d/dd8 1596488 1 2021-11-03T17:15:26.731 INFO:tasks.workunit.client.1.smithi115.stdout:2/735: read - d4/fca zero size 2021-11-03T17:15:26.731 INFO:tasks.workunit.client.1.smithi115.stdout:2/736: write d4/d6/d13/f4e [1178429,126711] 0 2021-11-03T17:15:26.732 INFO:tasks.workunit.client.1.smithi115.stdout:7/642: rmdir d3/da/d18/d73/d33 39 2021-11-03T17:15:26.732 INFO:tasks.workunit.client.1.smithi115.stdout:2/737: mknod d4/d6/dad/d7f/c100 1 2021-11-03T17:15:26.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:26 smithi115 conmon[34670]: cluster 2021-11-03T17:15:24.646157+0000 mgr.smithi027.ufhhvb 2021-11-03T17:15:26.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:26 smithi115 conmon[34670]: (mgr.14622) 27 : cluster [DBG] pgmap v16: 65 pgs: 65 active+clean; 3.0 GiB data, 9.1 GiB used, 527 GiB / 536 GiB avail; 16 MiB/s rd, 33 MiB/s wr, 410 op/s 2021-11-03T17:15:26.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:26 smithi115 conmon[34670]: cephadm 2021-11-03T17:15:25.314455+0000 mgr.smithi027.ufhhvb (mgr.14622) 28 : cephadm [INF] poll took 7905.618 ms: 1 events 2021-11-03T17:15:26.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:26 smithi115 conmon[34670]: audit 2021-11-03T17:15:25.324888+0000 mon.smithi027 (mon.0) 714 : audit [INF] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:15:26.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:26 smithi115 conmon[34670]: audit 2021-11-03T17:15:25.325773+0000 mon.smithi027 (mon.0) 715 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:15:26.735 INFO:tasks.workunit.client.1.smithi115.stdout:2/738: fdatasync d4/d6/d13/f58 0 2021-11-03T17:15:26.736 INFO:tasks.workunit.client.1.smithi115.stdout:6/660: dread d0/d1/fb [0,4194304] 0 2021-11-03T17:15:26.739 INFO:tasks.workunit.client.1.smithi115.stdout:6/661: symlink d0/d7f/ldf 0 2021-11-03T17:15:26.740 INFO:tasks.workunit.client.1.smithi115.stdout:6/662: fdatasync d0/d1/d10/d13/d87/d89/faf 0 2021-11-03T17:15:26.740 INFO:tasks.workunit.client.1.smithi115.stdout:7/643: dread d3/f97 [0,4194304] 0 2021-11-03T17:15:26.743 INFO:tasks.workunit.client.1.smithi115.stdout:7/644: fdatasync d3/da/d18/d73/d52/fa9 0 2021-11-03T17:15:26.762 INFO:tasks.workunit.client.1.smithi115.stdout:2/739: write d4/d6/dad/d7f/fc0 [1004922,44198] 0 2021-11-03T17:15:26.763 INFO:tasks.workunit.client.1.smithi115.stdout:1/764: write d7/d38/d84/f9a [746645,45202] 0 2021-11-03T17:15:26.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:26 smithi027 conmon[32322]: cluster 2021-11-03T17:15:24.646157+0000 2021-11-03T17:15:26.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:26 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14622) 27 : cluster [DBG] pgmap v16: 65 pgs: 65 active+clean; 3.0 GiB data, 9.1 GiB used, 527 GiB / 536 GiB avail; 16 MiB/s rd, 33 MiB/s wr, 410 op/s 2021-11-03T17:15:26.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:26 smithi027 conmon[32322]: cephadm 2021-11-03T17:15:25.314455+0000 mgr.smithi027.ufhhvb (mgr.14622) 2021-11-03T17:15:26.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:26 smithi027 conmon[32322]: 28 : cephadm [INF] poll took 7905.618 ms: 1 events 2021-11-03T17:15:26.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:26 smithi027 conmon[32322]: audit 2021-11 2021-11-03T17:15:26.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:26 smithi027 conmon[32322]: -03T17:15:25.324888+0000 mon.smithi027 (mon.0) 714 2021-11-03T17:15:26.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:26 smithi027 conmon[32322]: : audit [INF] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:15:26.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:26 smithi027 conmon[32322]: audit 2021-11-03T17 2021-11-03T17:15:26.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:26 smithi027 conmon[32322]: :15:25.325773+0000 mon.smithi027 (mon.0) 2021-11-03T17:15:26.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:26 smithi027 conmon[32322]: 715 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:15:26.771 INFO:tasks.workunit.client.1.smithi115.stdout:2/740: symlink d4/d6/dad/de8/d9c/dbe/l101 0 2021-11-03T17:15:26.771 INFO:tasks.workunit.client.1.smithi115.stdout:1/765: creat d7/dc/dd/d3e/d58/dc2/ff8 x:0 0 0 2021-11-03T17:15:26.778 INFO:tasks.workunit.client.1.smithi115.stdout:2/741: mknod d4/d6/d56/c102 1 2021-11-03T17:15:26.778 INFO:tasks.workunit.client.1.smithi115.stdout:1/766: rmdir d7/dc/dd/d3e/dd1/d43 39 2021-11-03T17:15:26.790 INFO:tasks.workunit.client.1.smithi115.stdout:1/767: dread d7/dc/f87 [0,4194304] 0 2021-11-03T17:15:26.798 INFO:tasks.workunit.client.1.smithi115.stdout:1/768: unlink d7/dc/d1a/l25 0 2021-11-03T17:15:26.800 INFO:tasks.workunit.client.1.smithi115.stdout:1/769: rmdir d7/dc/dd/d3e/d58 39 2021-11-03T17:15:26.804 INFO:tasks.workunit.client.1.smithi115.stdout:1/770: rename d7/dc/dab/f33 to d7/dc/dd/d3e/d58/dc2/dd2/ff9 0 2021-11-03T17:15:26.808 INFO:tasks.workunit.client.1.smithi115.stdout:1/771: symlink d7/dc/dd/d3e/d58/dbf/lfa 0 2021-11-03T17:15:26.821 INFO:tasks.workunit.client.1.smithi115.stdout:1/772: dread d7/dc/dd/d3e/dd1/f2b [0,4194304] 0 2021-11-03T17:15:26.830 INFO:tasks.workunit.client.1.smithi115.stdout:1/773: mknod d7/dc/dd/d42/cfb 1 2021-11-03T17:15:26.836 INFO:tasks.workunit.client.1.smithi115.stdout:1/774: dread d7/dc/dd/d3e/dd1/d26/f3b [4194304,4194304] 0 2021-11-03T17:15:26.848 INFO:tasks.workunit.client.1.smithi115.stdout:1/775: dread d7/dc/dd/d3e/dd1/d4b/d55/f66 [0,4194304] 0 2021-11-03T17:15:26.852 INFO:tasks.workunit.client.1.smithi115.stdout:1/776: creat d7/dc/dd/d3e/ffc x:0 0 0 2021-11-03T17:15:26.855 INFO:tasks.workunit.client.1.smithi115.stdout:1/777: unlink d7/dc/dd/f5a 0 2021-11-03T17:15:26.860 INFO:tasks.workunit.client.1.smithi115.stdout:1/778: link d7/dc/dab/d73/d7f/fa8 d7/dc/dd/d3e/d58/ffd 0 2021-11-03T17:15:26.861 INFO:tasks.workunit.client.1.smithi115.stdout:1/779: chown d7/dc/dd/d3e/dd1/d4b/d55/f97 1529788 1 2021-11-03T17:15:26.868 INFO:tasks.workunit.client.1.smithi115.stdout:1/780: rmdir d7/dc/d1a 39 2021-11-03T17:15:26.869 INFO:tasks.workunit.client.1.smithi115.stdout:1/781: chown d7/dc/dd/d3e/l56 1048358 1 2021-11-03T17:15:26.876 INFO:tasks.workunit.client.1.smithi115.stdout:1/782: creat d7/dc/dd/d3e/dd1/d43/d78/ffe x:0 0 0 2021-11-03T17:15:26.886 INFO:tasks.workunit.client.1.smithi115.stdout:1/783: rename d7/dc/dd/d3e/dd1/d26/dc3 to d7/dc/dd/d3e/dbe/dff 0 2021-11-03T17:15:26.887 INFO:tasks.workunit.client.1.smithi115.stdout:1/784: chown d7/d9d/fda 942186058 1 2021-11-03T17:15:26.888 INFO:tasks.workunit.client.1.smithi115.stdout:1/785: truncate d7/d38/d84/f9a 4052693 0 2021-11-03T17:15:27.022 INFO:tasks.workunit.client.1.smithi115.stdout:3/881: sync 2021-11-03T17:15:27.036 INFO:tasks.workunit.client.1.smithi115.stdout:3/882: dread d13/d35/d54/fab [0,4194304] 0 2021-11-03T17:15:27.038 INFO:tasks.workunit.client.1.smithi115.stdout:3/883: unlink d13/d35/f101 0 2021-11-03T17:15:27.041 INFO:tasks.workunit.client.1.smithi115.stdout:3/884: getdents d13/d2a/d53/d72/dd2/df1 0 2021-11-03T17:15:27.041 INFO:tasks.workunit.client.1.smithi115.stdout:3/885: chown d13/f1c 3219 1 2021-11-03T17:15:27.052 INFO:tasks.workunit.client.1.smithi115.stdout:3/886: rename d13/ld5 to d13/d50/l12a 0 2021-11-03T17:15:27.052 INFO:tasks.workunit.client.1.smithi115.stdout:3/887: mknod d13/d3b/db1/dbc/c12b 1 2021-11-03T17:15:27.054 INFO:tasks.workunit.client.1.smithi115.stdout:3/888: creat d13/d2c/f12c x:0 0 0 2021-11-03T17:15:27.055 INFO:tasks.workunit.client.1.smithi115.stdout:3/889: write d13/d51/d40/f9d [776730,127331] 0 2021-11-03T17:15:27.061 INFO:tasks.workunit.client.1.smithi115.stdout:3/890: rename d13/d51/f29 to d13/d35/d54/f12d 0 2021-11-03T17:15:27.065 INFO:tasks.workunit.client.1.smithi115.stdout:3/891: mkdir d13/d2a/d53/d72/d12e 0 2021-11-03T17:15:27.068 INFO:tasks.workunit.client.1.smithi115.stdout:3/892: link d13/d3b/db1/dbc/lff d13/da4/db2/l12f 0 2021-11-03T17:15:27.070 INFO:tasks.workunit.client.1.smithi115.stdout:3/893: creat d13/d2a/d60/f130 x:0 0 0 2021-11-03T17:15:27.094 INFO:tasks.workunit.client.1.smithi115.stdout:3/894: dwrite d13/d51/d2e/fa2 [4194304,4194304] 0 2021-11-03T17:15:27.095 INFO:tasks.workunit.client.1.smithi115.stdout:3/895: write d13/d2a/d53/d72/dd2/df1/f115 [575671,57261] 0 2021-11-03T17:15:27.103 INFO:tasks.workunit.client.1.smithi115.stdout:3/896: creat d13/d35/d54/f131 x:0 0 0 2021-11-03T17:15:27.108 INFO:tasks.workunit.client.1.smithi115.stdout:3/897: mkdir d13/d50/dfc/d11a/d132 0 2021-11-03T17:15:27.113 INFO:tasks.workunit.client.1.smithi115.stdout:3/898: creat d13/d2a/d53/d72/d12e/f133 x:0 0 0 2021-11-03T17:15:27.117 INFO:tasks.workunit.client.1.smithi115.stdout:3/899: rmdir d13/d2a/d8a 39 2021-11-03T17:15:27.121 INFO:tasks.workunit.client.1.smithi115.stdout:3/900: symlink d13/d2a/d53/d72/dd2/l134 0 2021-11-03T17:15:27.123 INFO:tasks.workunit.client.1.smithi115.stdout:3/901: creat d13/d35/d54/f135 x:0 0 0 2021-11-03T17:15:27.175 INFO:tasks.workunit.client.1.smithi115.stdout:3/902: dwrite d13/d67/f87 [0,4194304] 0 2021-11-03T17:15:27.213 INFO:tasks.workunit.client.1.smithi115.stdout:3/903: write d13/f1e [1790140,40569] 0 2021-11-03T17:15:27.223 INFO:tasks.workunit.client.1.smithi115.stdout:3/904: readlink d13/d2a/d53/d70/d96/lc4 0 2021-11-03T17:15:27.228 INFO:tasks.workunit.client.1.smithi115.stdout:3/905: unlink d13/d2a/d53/d72/d12e/f133 0 2021-11-03T17:15:27.243 INFO:tasks.workunit.client.1.smithi115.stdout:3/906: dwrite d13/d51/d40/fe1 [0,4194304] 0 2021-11-03T17:15:27.243 INFO:tasks.workunit.client.1.smithi115.stdout:3/907: chown d13/d67 14 1 2021-11-03T17:15:27.243 INFO:tasks.workunit.client.1.smithi115.stdout:3/908: chown d13/d51/d2e 3 1 2021-11-03T17:15:27.260 INFO:tasks.workunit.client.1.smithi115.stdout:3/909: mkdir d13/d35/d54/dad/d136 0 2021-11-03T17:15:27.266 INFO:tasks.workunit.client.1.smithi115.stdout:3/910: read - d13/d3b/f7a zero size 2021-11-03T17:15:27.277 INFO:tasks.workunit.client.1.smithi115.stdout:3/911: rename d13/d2a/d53/d70/d92/dc2/l11e to d13/d3b/db1/l137 0 2021-11-03T17:15:27.284 INFO:tasks.workunit.client.1.smithi115.stdout:3/912: unlink d13/d2c/f12c 0 2021-11-03T17:15:27.286 INFO:tasks.workunit.client.1.smithi115.stdout:3/913: mknod d13/da4/dcb/c138 1 2021-11-03T17:15:27.286 INFO:tasks.workunit.client.1.smithi115.stdout:3/914: stat d13/d50/l8b 0 2021-11-03T17:15:27.287 INFO:tasks.workunit.client.1.smithi115.stdout:3/915: fsync d13/d2a/fe5 0 2021-11-03T17:15:27.289 INFO:tasks.workunit.client.1.smithi115.stdout:3/916: mknod d13/d2a/d53/c139 1 2021-11-03T17:15:27.304 INFO:tasks.workunit.client.1.smithi115.stdout:3/917: dwrite d13/d2a/d8a/d99/fd9 [4194304,4194304] 0 2021-11-03T17:15:27.306 INFO:tasks.workunit.client.1.smithi115.stdout:3/918: dread d13/d2a/d53/d70/d92/dc2/fdd [0,4194304] 0 2021-11-03T17:15:27.312 INFO:tasks.workunit.client.1.smithi115.stdout:3/919: truncate d13/d3b/f43 378898 0 2021-11-03T17:15:27.316 INFO:tasks.workunit.client.1.smithi115.stdout:3/920: symlink d13/d35/l13a 0 2021-11-03T17:15:27.633 INFO:tasks.workunit.client.1.smithi115.stdout:9/691: sync 2021-11-03T17:15:27.636 INFO:tasks.workunit.client.1.smithi115.stdout:9/692: read d1/d1a/d48/d27/d53/d83/d88/d9e/d58/fc1 [643785,117880] 0 2021-11-03T17:15:27.665 INFO:tasks.workunit.client.1.smithi115.stdout:9/693: dwrite d1/d18/d2b/d42/d6e/f52 [0,4194304] 0 2021-11-03T17:15:27.667 INFO:tasks.workunit.client.1.smithi115.stdout:9/694: mkdir d1/d18/d2b/d42/d6e/db5/dfa 0 2021-11-03T17:15:27.670 INFO:tasks.workunit.client.1.smithi115.stdout:9/695: rmdir d1/d1a/d48/d27/d53 39 2021-11-03T17:15:27.670 INFO:tasks.workunit.client.1.smithi115.stdout:9/696: fdatasync d1/d1a/d48/d27/d2f/f8f 0 2021-11-03T17:15:27.839 INFO:tasks.workunit.client.1.smithi115.stdout:5/811: sync 2021-11-03T17:15:27.840 INFO:tasks.workunit.client.1.smithi115.stdout:5/812: dread - d1a/d28/d4c/fee zero size 2021-11-03T17:15:27.840 INFO:tasks.workunit.client.1.smithi115.stdout:5/813: fsync d1a/f1d 0 2021-11-03T17:15:27.846 INFO:tasks.workunit.client.1.smithi115.stdout:5/814: dread d1a/d28/d4a/d65/ff0 [4194304,4194304] 0 2021-11-03T17:15:27.853 INFO:tasks.workunit.client.1.smithi115.stdout:5/815: unlink d1a/d3d/d58/fb2 0 2021-11-03T17:15:27.870 INFO:tasks.workunit.client.1.smithi115.stdout:5/816: rename d1a/d28/d4c/d75/d68/da5 to d1a/d28/d4c/d75/dae/d114 0 2021-11-03T17:15:27.873 INFO:tasks.workunit.client.1.smithi115.stdout:5/817: dread d1a/d23/fcb [0,4194304] 0 2021-11-03T17:15:27.875 INFO:tasks.workunit.client.1.smithi115.stdout:5/818: creat d1a/d28/d4c/d75/f115 x:0 0 0 2021-11-03T17:15:27.877 INFO:tasks.workunit.client.1.smithi115.stdout:5/819: truncate d1a/d28/d7b/f77 1157852 0 2021-11-03T17:15:27.879 INFO:tasks.workunit.client.1.smithi115.stdout:5/820: getdents d1a/d3d/d58/dab/daf 0 2021-11-03T17:15:27.882 INFO:tasks.workunit.client.1.smithi115.stdout:5/821: symlink d1a/d28/d7b/d48/dd4/l116 0 2021-11-03T17:15:27.883 INFO:tasks.workunit.client.1.smithi115.stdout:5/822: fdatasync d1a/d28/daa/fea 0 2021-11-03T17:15:27.893 INFO:tasks.workunit.client.1.smithi115.stdout:5/823: rename d1a/d3d/f47 to d1a/d28/d4c/f117 0 2021-11-03T17:15:27.897 INFO:tasks.workunit.client.1.smithi115.stdout:5/824: unlink d1a/d28/d7b/f9f 0 2021-11-03T17:15:27.907 INFO:tasks.workunit.client.1.smithi115.stdout:5/825: rename d1a/d28/d7b/d48/dd4/fff to d1a/d3d/dc4/f118 0 2021-11-03T17:15:27.910 INFO:tasks.workunit.client.1.smithi115.stdout:5/826: dread d1a/d3d/dc0/ddf/ff3 [0,4194304] 0 2021-11-03T17:15:27.911 INFO:tasks.workunit.client.1.smithi115.stdout:5/827: dread d1a/d3d/dc0/ddf/ff3 [0,4194304] 0 2021-11-03T17:15:27.913 INFO:tasks.workunit.client.1.smithi115.stdout:5/828: truncate d1a/d28/d4c/d75/dae/fce 445454 0 2021-11-03T17:15:27.918 INFO:tasks.workunit.client.1.smithi115.stdout:5/829: creat d1a/d3d/dc0/f119 x:0 0 0 2021-11-03T17:15:27.918 INFO:tasks.workunit.client.1.smithi115.stdout:5/830: readlink d1a/d28/d7b/l113 0 2021-11-03T17:15:27.919 INFO:tasks.workunit.client.1.smithi115.stdout:5/831: fdatasync d1a/d3d/dc4/f118 0 2021-11-03T17:15:27.924 INFO:tasks.workunit.client.1.smithi115.stdout:5/832: symlink d1a/d3d/d58/dab/dd3/l11a 0 2021-11-03T17:15:27.925 INFO:tasks.workunit.client.1.smithi115.stdout:5/833: fsync d1a/d28/daa/f111 0 2021-11-03T17:15:27.931 INFO:tasks.workunit.client.1.smithi115.stdout:5/834: chown d1a/d28/d4c/d75/d83/da1/d103 76 1 2021-11-03T17:15:27.948 INFO:tasks.workunit.client.1.smithi115.stdout:5/835: dread d1a/d28/d4a/d65/db5/f67 [0,4194304] 0 2021-11-03T17:15:27.950 INFO:tasks.workunit.client.1.smithi115.stdout:5/836: readlink d1a/d28/d4c/d63/lc8 0 2021-11-03T17:15:27.953 INFO:tasks.workunit.client.1.smithi115.stdout:5/837: mkdir d1a/d28/d7b/d4f/d81/d11b 0 2021-11-03T17:15:27.954 INFO:tasks.workunit.client.1.smithi115.stdout:5/838: mknod d1a/d3d/c11c 1 2021-11-03T17:15:27.972 INFO:tasks.workunit.client.1.smithi115.stdout:5/839: getdents d1a/d28/d7b/d4f/d81 0 2021-11-03T17:15:27.976 INFO:tasks.workunit.client.1.smithi115.stdout:5/840: mkdir d1a/d28/d4a/d65/d11d 0 2021-11-03T17:15:27.995 INFO:tasks.workunit.client.1.smithi115.stdout:5/841: dwrite d1a/d28/d4c/d75/dae/d114/fd9 [0,4194304] 0 2021-11-03T17:15:27.995 INFO:tasks.workunit.client.1.smithi115.stdout:5/842: chown d1a/d28/d7b/fd8 196 1 2021-11-03T17:15:28.000 INFO:tasks.workunit.client.1.smithi115.stdout:5/843: truncate d1a/d23/f5c 926054 0 2021-11-03T17:15:28.001 INFO:tasks.workunit.client.1.smithi115.stdout:5/844: fsync d1a/d28/d34/f66 0 2021-11-03T17:15:28.018 INFO:tasks.workunit.client.1.smithi115.stdout:5/845: dwrite d1a/d28/d4c/d75/dae/d114/fd9 [0,4194304] 0 2021-11-03T17:15:28.034 INFO:tasks.workunit.client.1.smithi115.stdout:5/846: dwrite d1a/d28/d4c/d75/f115 [0,4194304] 0 2021-11-03T17:15:28.034 INFO:tasks.workunit.client.1.smithi115.stdout:5/847: write d1a/d28/d4c/d75/dae/d114/fa8 [1267323,116859] 0 2021-11-03T17:15:28.040 INFO:tasks.workunit.client.1.smithi115.stdout:5/848: creat d1a/d3d/f11e x:0 0 0 2021-11-03T17:15:28.054 INFO:tasks.workunit.client.1.smithi115.stdout:5/849: dwrite d1a/d28/d7b/d48/fca [0,4194304] 0 2021-11-03T17:15:28.060 INFO:tasks.workunit.client.1.smithi115.stdout:5/850: creat d1a/d28/daa/d109/f11f x:0 0 0 2021-11-03T17:15:28.089 INFO:tasks.workunit.client.1.smithi115.stdout:5/851: dwrite d1a/d28/d7b/d48/fb1 [0,4194304] 0 2021-11-03T17:15:28.113 INFO:tasks.workunit.client.1.smithi115.stdout:5/852: rename d1a/d23/de6/l104 to d1a/d28/d34/l120 0 2021-11-03T17:15:28.113 INFO:tasks.workunit.client.1.smithi115.stdout:5/853: chown d1a/d28/d4c/d63/f9c 287 1 2021-11-03T17:15:28.114 INFO:tasks.workunit.client.1.smithi115.stdout:5/854: mknod d1a/d23/db4/c121 1 2021-11-03T17:15:28.141 INFO:tasks.workunit.client.1.smithi115.stdout:0/909: sync 2021-11-03T17:15:28.142 INFO:tasks.workunit.client.1.smithi115.stdout:4/905: sync 2021-11-03T17:15:28.143 INFO:tasks.workunit.client.1.smithi115.stdout:4/906: read d7/dc/d6a/d116/f117 [2191456,66583] 0 2021-11-03T17:15:28.147 INFO:tasks.workunit.client.1.smithi115.stdout:0/910: mknod d9/d15/c12c 1 2021-11-03T17:15:28.151 INFO:tasks.workunit.client.1.smithi115.stdout:4/907: symlink d7/dc/d6a/l132 0 2021-11-03T17:15:28.153 INFO:tasks.workunit.client.1.smithi115.stdout:0/911: mkdir d9/d15/db6/d10b/d124/d12d 0 2021-11-03T17:15:28.158 INFO:tasks.workunit.client.1.smithi115.stdout:5/855: dwrite d1a/d28/f31 [4194304,4194304] 0 2021-11-03T17:15:28.158 INFO:tasks.workunit.client.1.smithi115.stdout:5/856: fsync d1a/d28/d7b/fd8 0 2021-11-03T17:15:28.159 INFO:tasks.workunit.client.1.smithi115.stdout:5/857: read d1a/d23/fcb [151793,48967] 0 2021-11-03T17:15:28.159 INFO:tasks.workunit.client.1.smithi115.stdout:5/858: chown d1a/d28/f91 26509 1 2021-11-03T17:15:28.160 INFO:tasks.workunit.client.1.smithi115.stdout:4/908: symlink d7/dc/d1a/d21/db1/d56/d63/d121/l133 0 2021-11-03T17:15:28.163 INFO:tasks.workunit.client.1.smithi115.stdout:0/912: creat d9/d15/db6/d10b/f12e x:0 0 0 2021-11-03T17:15:28.171 INFO:tasks.workunit.client.1.smithi115.stdout:4/909: mknod d7/dc/d6a/d3c/d112/c134 1 2021-11-03T17:15:28.171 INFO:tasks.workunit.client.1.smithi115.stdout:0/913: readlink d9/d15/d1a/l6c 0 2021-11-03T17:15:28.175 INFO:tasks.workunit.client.1.smithi115.stdout:0/914: truncate d9/d15/d1a/d1d/f29 2831581 0 2021-11-03T17:15:28.175 INFO:tasks.workunit.client.1.smithi115.stdout:5/859: mkdir d1a/d28/df9/d122 0 2021-11-03T17:15:28.175 INFO:tasks.workunit.client.1.smithi115.stdout:5/860: stat d1a/d28/d7b/d4f/d81/d107 0 2021-11-03T17:15:28.180 INFO:tasks.workunit.client.1.smithi115.stdout:5/861: rmdir d1a/d28/d4c/d75/dae 39 2021-11-03T17:15:28.191 INFO:tasks.workunit.client.1.smithi115.stdout:0/915: dread d9/d15/d1a/d1d/dcc/d46/d63/dae/fed [0,4194304] 0 2021-11-03T17:15:28.203 INFO:tasks.workunit.client.1.smithi115.stdout:4/910: dwrite d7/d7a/d83/f8c [0,4194304] 0 2021-11-03T17:15:28.203 INFO:tasks.workunit.client.1.smithi115.stdout:4/911: stat d7/dc/d1a/d21/db1/d56/d42/feb 0 2021-11-03T17:15:28.203 INFO:tasks.workunit.client.1.smithi115.stdout:4/912: stat d7/dc/l126 0 2021-11-03T17:15:28.203 INFO:tasks.workunit.client.1.smithi115.stdout:4/913: dread - d7/dc/d1a/d21/f105 zero size 2021-11-03T17:15:28.205 INFO:tasks.workunit.client.1.smithi115.stdout:5/862: dwrite d1a/d28/f31 [8388608,4194304] 0 2021-11-03T17:15:28.205 INFO:tasks.workunit.client.1.smithi115.stdout:4/914: mkdir d7/dc/d1a/d21/db1/d56/df4/d135 0 2021-11-03T17:15:28.209 INFO:tasks.workunit.client.1.smithi115.stdout:4/915: dread d7/dc/f95 [0,4194304] 0 2021-11-03T17:15:28.210 INFO:tasks.workunit.client.1.smithi115.stdout:5/863: dread - d1a/d3d/d58/dab/fbb zero size 2021-11-03T17:15:28.216 INFO:tasks.workunit.client.1.smithi115.stdout:4/916: rmdir d7/d7a/d83/df9 0 2021-11-03T17:15:28.216 INFO:tasks.workunit.client.1.smithi115.stdout:4/917: dread - d7/dc/d1a/f11e zero size 2021-11-03T17:15:28.238 INFO:tasks.workunit.client.1.smithi115.stdout:5/864: dwrite f11 [4194304,4194304] 0 2021-11-03T17:15:28.259 INFO:tasks.workunit.client.1.smithi115.stdout:4/918: write d7/d7a/fc3 [263608,102601] 0 2021-11-03T17:15:28.259 INFO:tasks.workunit.client.1.smithi115.stdout:5/865: dwrite d1a/d28/d7b/d48/ff2 [0,4194304] 0 2021-11-03T17:15:28.264 INFO:tasks.workunit.client.1.smithi115.stdout:0/916: dread d9/d15/d1a/d1d/f29 [0,4194304] 0 2021-11-03T17:15:28.275 INFO:tasks.workunit.client.1.smithi115.stdout:5/866: unlink d1a/d28/d7b/l90 0 2021-11-03T17:15:28.276 INFO:tasks.workunit.client.1.smithi115.stdout:5/867: write d1a/d28/df9/fe2 [1411036,82248] 0 2021-11-03T17:15:28.282 INFO:tasks.workunit.client.1.smithi115.stdout:5/868: mkdir d1a/d28/d4c/d75/d123 0 2021-11-03T17:15:28.283 INFO:tasks.workunit.client.1.smithi115.stdout:5/869: stat d1a/d3d/d58/dab/dd3/l11a 0 2021-11-03T17:15:28.283 INFO:tasks.workunit.client.1.smithi115.stdout:7/645: sync 2021-11-03T17:15:28.284 INFO:tasks.workunit.client.1.smithi115.stdout:2/742: sync 2021-11-03T17:15:28.284 INFO:tasks.workunit.client.1.smithi115.stdout:2/743: fsync d4/d6/dad/de8/d82/ff8 0 2021-11-03T17:15:28.284 INFO:tasks.workunit.client.1.smithi115.stdout:8/684: sync 2021-11-03T17:15:28.285 INFO:tasks.workunit.client.1.smithi115.stdout:8/685: dread - d4/d6/d3f/d9a/fd6 zero size 2021-11-03T17:15:28.285 INFO:tasks.workunit.client.1.smithi115.stdout:6/663: sync 2021-11-03T17:15:28.288 INFO:tasks.workunit.client.1.smithi115.stdout:4/919: write d7/dc/d1a/d21/db1/d56/d63/d121/f128 [573724,28784] 0 2021-11-03T17:15:28.290 INFO:tasks.workunit.client.1.smithi115.stdout:7/646: mkdir d3/d4c/da6/dc7 0 2021-11-03T17:15:28.296 INFO:tasks.workunit.client.1.smithi115.stdout:8/686: unlink d4/d6/d18/d44/d47/fab 0 2021-11-03T17:15:28.301 INFO:tasks.workunit.client.1.smithi115.stdout:5/870: truncate d1a/d28/d4c/fee 1615207 0 2021-11-03T17:15:28.304 INFO:tasks.workunit.client.1.smithi115.stdout:0/917: dwrite d9/d15/d1a/d1d/dcc/d46/d63/dae/fed [0,4194304] 0 2021-11-03T17:15:28.305 INFO:tasks.workunit.client.1.smithi115.stdout:5/871: dread d1a/d28/d4a/d65/f100 [0,4194304] 0 2021-11-03T17:15:28.309 INFO:tasks.workunit.client.1.smithi115.stdout:2/744: symlink d4/d6/d56/l103 0 2021-11-03T17:15:28.310 INFO:tasks.workunit.client.1.smithi115.stdout:8/687: truncate d4/d6/d3f/d43/d88/f8b 1916416 0 2021-11-03T17:15:28.316 INFO:tasks.workunit.client.1.smithi115.stdout:7/647: dwrite d3/f9 [4194304,4194304] 0 2021-11-03T17:15:28.316 INFO:tasks.workunit.client.1.smithi115.stdout:7/648: chown d3/da/d18/fb2 36490701 1 2021-11-03T17:15:28.317 INFO:tasks.workunit.client.1.smithi115.stdout:7/649: write d3/da/d18/fb2 [290103,5092] 0 2021-11-03T17:15:28.318 INFO:tasks.workunit.client.1.smithi115.stdout:6/664: dwrite d0/d1/d10/d13/d1c/f2f [0,4194304] 0 2021-11-03T17:15:28.318 INFO:tasks.workunit.client.1.smithi115.stdout:6/665: mknod d0/d74/dae/db3/ce0 1 2021-11-03T17:15:28.321 INFO:tasks.workunit.client.1.smithi115.stdout:7/650: read d3/f47 [3925771,77066] 0 2021-11-03T17:15:28.322 INFO:tasks.workunit.client.1.smithi115.stdout:7/651: truncate d3/da/d18/d9c/da1/fb9 350168 0 2021-11-03T17:15:28.323 INFO:tasks.workunit.client.1.smithi115.stdout:4/920: rename l1 to d7/dc/d6a/d3c/d112/d130/l136 0 2021-11-03T17:15:28.326 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:28 smithi115 conmon[34670]: cluster 2021-11-03T17:15:26.646533+0000 mgr.smithi027.ufhhvb ( 2021-11-03T17:15:28.327 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:28 smithi115 conmon[34670]: mgr.14622) 29 : cluster [DBG] pgmap v17: 65 pgs: 65 active+clean; 3.0 GiB data, 9.1 GiB used, 527 GiB / 536 GiB avail; 16 MiB/s rd, 33 MiB/s wr, 410 op/s 2021-11-03T17:15:28.329 INFO:tasks.workunit.client.1.smithi115.stdout:5/872: rmdir d1a/d28/d4c/d10e 0 2021-11-03T17:15:28.329 INFO:tasks.workunit.client.1.smithi115.stdout:2/745: getdents d4/d6/dad/de8/d82 0 2021-11-03T17:15:28.329 INFO:tasks.workunit.client.1.smithi115.stdout:2/746: chown d4/d6/d56/d6c/d34/d43/d28/lc8 0 1 2021-11-03T17:15:28.329 INFO:tasks.workunit.client.1.smithi115.stdout:2/747: truncate d4/d6/d13/f6b 1297214 0 2021-11-03T17:15:28.333 INFO:tasks.workunit.client.1.smithi115.stdout:6/666: creat d0/d1/d2e/d4c/d96/fe1 x:0 0 0 2021-11-03T17:15:28.339 INFO:tasks.workunit.client.1.smithi115.stdout:5/873: mkdir d1a/d23/db4/d124 0 2021-11-03T17:15:28.343 INFO:tasks.workunit.client.1.smithi115.stdout:2/748: mkdir d4/d6/d56/d6c/d34/d42/d8d/dd8/d104 0 2021-11-03T17:15:28.344 INFO:tasks.workunit.client.1.smithi115.stdout:6/667: chown d0/d1/d10/d13/d59/f80 88940 1 2021-11-03T17:15:28.347 INFO:tasks.workunit.client.1.smithi115.stdout:4/921: dwrite d7/dc/f60 [4194304,4194304] 0 2021-11-03T17:15:28.357 INFO:tasks.workunit.client.1.smithi115.stdout:7/652: mkdir d3/da/d18/d73/dc8 0 2021-11-03T17:15:28.362 INFO:tasks.workunit.client.1.smithi115.stdout:0/918: rename d9/le1 to d9/d15/d1a/l12f 0 2021-11-03T17:15:28.364 INFO:tasks.workunit.client.1.smithi115.stdout:5/874: creat d1a/d3d/dc0/ddf/f125 x:0 0 0 2021-11-03T17:15:28.364 INFO:tasks.workunit.client.1.smithi115.stdout:5/875: chown d1a/d28/d7b/l113 75216 1 2021-11-03T17:15:28.364 INFO:tasks.workunit.client.1.smithi115.stdout:5/876: chown d1a/d28/l2e 17157694 1 2021-11-03T17:15:28.367 INFO:tasks.workunit.client.1.smithi115.stdout:2/749: rmdir d4/d6/dad/d7f 39 2021-11-03T17:15:28.368 INFO:tasks.workunit.client.1.smithi115.stdout:2/750: write d4/d6/d13/d75/f7a [1864032,69355] 0 2021-11-03T17:15:28.368 INFO:tasks.workunit.client.1.smithi115.stdout:2/751: readlink d4/d6/dad/d7f/de9/lfe 0 2021-11-03T17:15:28.369 INFO:tasks.workunit.client.1.smithi115.stdout:4/922: creat d7/dc/d6a/d116/f137 x:0 0 0 2021-11-03T17:15:28.380 INFO:tasks.workunit.client.1.smithi115.stdout:0/919: mknod d9/d15/d1a/d1d/dcc/d6e/d71/c130 1 2021-11-03T17:15:28.387 INFO:tasks.workunit.client.1.smithi115.stdout:4/923: creat d7/dc/d1a/d21/f138 x:0 0 0 2021-11-03T17:15:28.388 INFO:tasks.workunit.client.1.smithi115.stdout:8/688: rename d4/d6/d18/f85 to d4/d6/d3f/da4/da6/dcc/ff6 0 2021-11-03T17:15:28.390 INFO:tasks.workunit.client.1.smithi115.stdout:8/689: creat d4/d6/d18/d44/dcd/ff7 x:0 0 0 2021-11-03T17:15:28.392 INFO:tasks.workunit.client.1.smithi115.stdout:4/924: dread d7/d7a/fc3 [0,4194304] 0 2021-11-03T17:15:28.395 INFO:tasks.workunit.client.1.smithi115.stdout:4/925: creat d7/f139 x:0 0 0 2021-11-03T17:15:28.396 INFO:tasks.workunit.client.1.smithi115.stdout:4/926: fsync d7/dc/d1a/d21/db1/f74 0 2021-11-03T17:15:28.396 INFO:tasks.workunit.client.1.smithi115.stdout:8/690: write d4/d6/d18/d44/f62 [597730,34982] 0 2021-11-03T17:15:28.396 INFO:tasks.workunit.client.1.smithi115.stdout:8/691: dread - d4/d6/d18/d30/d79/d7f/fae zero size 2021-11-03T17:15:28.397 INFO:tasks.workunit.client.1.smithi115.stdout:7/653: rename d3/d4c/lc3 to d3/lc9 0 2021-11-03T17:15:28.399 INFO:tasks.workunit.client.1.smithi115.stdout:5/877: write d1a/d28/df9/fc1 [88903,27518] 0 2021-11-03T17:15:28.401 INFO:tasks.workunit.client.1.smithi115.stdout:0/920: dwrite d9/d15/d1a/d1d/dcc/d46/d8c/dfc/f128 [0,4194304] 0 2021-11-03T17:15:28.404 INFO:tasks.workunit.client.1.smithi115.stdout:0/921: write d9/d15/d1a/d1d/dcc/d85/ff8 [2685351,47322] 0 2021-11-03T17:15:28.406 INFO:tasks.workunit.client.1.smithi115.stdout:4/927: mkdir d7/dc/d6a/d3c/d112/d13a 0 2021-11-03T17:15:28.406 INFO:tasks.workunit.client.1.smithi115.stdout:4/928: write d7/dc/d6a/f12e [120948,35021] 0 2021-11-03T17:15:28.406 INFO:tasks.workunit.client.1.smithi115.stdout:8/692: dread d4/d6/d18/fa5 [0,4194304] 0 2021-11-03T17:15:28.407 INFO:tasks.workunit.client.1.smithi115.stdout:8/693: chown d4/d6/fd2 8 1 2021-11-03T17:15:28.409 INFO:tasks.workunit.client.1.smithi115.stdout:7/654: symlink d3/da/d18/d73/d52/lca 0 2021-11-03T17:15:28.409 INFO:tasks.workunit.client.1.smithi115.stdout:7/655: chown d3/da/d8e 118187 1 2021-11-03T17:15:28.409 INFO:tasks.workunit.client.1.smithi115.stdout:7/656: chown d3/d4c/fa5 1657945 1 2021-11-03T17:15:28.410 INFO:tasks.workunit.client.1.smithi115.stdout:5/878: mkdir d1a/d28/d34/d126 0 2021-11-03T17:15:28.412 INFO:tasks.workunit.client.1.smithi115.stdout:4/929: mkdir d7/dc/d6a/d3c/d112/d130/d13b 0 2021-11-03T17:15:28.413 INFO:tasks.workunit.client.1.smithi115.stdout:0/922: mknod d9/d15/c131 1 2021-11-03T17:15:28.413 INFO:tasks.workunit.client.1.smithi115.stdout:0/923: fsync d9/d15/d1a/d1d/dcc/d46/d63/dae/fd7 0 2021-11-03T17:15:28.415 INFO:tasks.workunit.client.1.smithi115.stdout:4/930: unlink d7/dc/d6a/dab/le2 0 2021-11-03T17:15:28.416 INFO:tasks.workunit.client.1.smithi115.stdout:0/924: fdatasync d9/d37/fdc 0 2021-11-03T17:15:28.417 INFO:tasks.workunit.client.1.smithi115.stdout:7/657: symlink d3/da/d18/d73/d52/lcb 0 2021-11-03T17:15:28.418 INFO:tasks.workunit.client.1.smithi115.stdout:4/931: symlink d7/dc/d1a/d21/db1/d56/df4/l13c 0 2021-11-03T17:15:28.419 INFO:tasks.workunit.client.1.smithi115.stdout:7/658: write d3/da/d18/d73/d35/d53/fb7 [3367574,112936] 0 2021-11-03T17:15:28.419 INFO:tasks.workunit.client.1.smithi115.stdout:7/659: readlink d3/lc9 0 2021-11-03T17:15:28.419 INFO:tasks.workunit.client.1.smithi115.stdout:7/660: fdatasync d3/da/d18/d73/d35/f9d 0 2021-11-03T17:15:28.420 INFO:tasks.workunit.client.1.smithi115.stdout:0/925: truncate d9/d15/f1e 1789221 0 2021-11-03T17:15:28.421 INFO:tasks.workunit.client.1.smithi115.stdout:2/752: dwrite d4/d6/d56/d6c/d34/d43/d28/f3b [4194304,4194304] 0 2021-11-03T17:15:28.421 INFO:tasks.workunit.client.1.smithi115.stdout:2/753: write d4/d6/d13/f4e [4392962,41819] 0 2021-11-03T17:15:28.435 INFO:tasks.workunit.client.1.smithi115.stdout:7/661: getdents d3/da/d18/d73 0 2021-11-03T17:15:28.436 INFO:tasks.workunit.client.1.smithi115.stdout:5/879: dread d1a/d28/d4c/f6e [0,4194304] 0 2021-11-03T17:15:28.464 INFO:tasks.workunit.client.1.smithi115.stdout:2/754: chown d4/d6/d56/d6c/l5e 88957945 1 2021-11-03T17:15:28.464 INFO:tasks.workunit.client.1.smithi115.stdout:2/755: dread - d4/d6/dad/dc2/fdb zero size 2021-11-03T17:15:28.488 INFO:tasks.workunit.client.1.smithi115.stdout:0/926: mkdir d9/d15/db6/d132 0 2021-11-03T17:15:28.502 INFO:tasks.workunit.client.1.smithi115.stdout:7/662: link d3/da/d18/l28 d3/d4c/d7b/lcc 0 2021-11-03T17:15:28.502 INFO:tasks.workunit.client.1.smithi115.stdout:7/663: truncate d3/da/d8e/f99 4940769 0 2021-11-03T17:15:28.504 INFO:tasks.workunit.client.1.smithi115.stdout:4/932: write d7/dc/d1a/d21/db1/d56/d63/d7b/fb2 [2340776,6734] 0 2021-11-03T17:15:28.504 INFO:tasks.workunit.client.1.smithi115.stdout:4/933: read d7/dc/d1a/d21/db1/d56/db3/fe7 [2957012,4729] 0 2021-11-03T17:15:28.506 INFO:tasks.workunit.client.1.smithi115.stdout:5/880: symlink d1a/d28/d4c/d75/dae/d114/l127 0 2021-11-03T17:15:28.507 INFO:tasks.workunit.client.1.smithi115.stdout:0/927: creat d9/db/d83/f133 x:0 0 0 2021-11-03T17:15:28.513 INFO:tasks.workunit.client.1.smithi115.stdout:2/756: rename d4/d6/d56/d6c/d34/d43/lae to d4/d6/dad/de8/dac/dcd/l105 0 2021-11-03T17:15:28.514 INFO:tasks.workunit.client.1.smithi115.stdout:0/928: dread d9/f66 [0,4194304] 0 2021-11-03T17:15:28.515 INFO:tasks.workunit.client.1.smithi115.stdout:2/757: write d4/f5a [462772,60532] 0 2021-11-03T17:15:28.515 INFO:tasks.workunit.client.1.smithi115.stdout:2/758: truncate d4/d6/d13/ffa 317327 0 2021-11-03T17:15:28.521 INFO:tasks.workunit.client.1.smithi115.stdout:4/934: rmdir d7/dc/d6a/d3c/dc5 39 2021-11-03T17:15:28.525 INFO:tasks.workunit.client.1.smithi115.stdout:5/881: mkdir d1a/d28/d4c/d75/d83/da1/df5/d128 0 2021-11-03T17:15:28.535 INFO:tasks.workunit.client.1.smithi115.stdout:7/664: mkdir d3/da/d78/dcd 0 2021-11-03T17:15:28.538 INFO:tasks.workunit.client.1.smithi115.stdout:0/929: dwrite d9/db/f8e [0,4194304] 0 2021-11-03T17:15:28.541 INFO:tasks.workunit.client.1.smithi115.stdout:5/882: rmdir d1a/d28/d7b/d48/d7f 39 2021-11-03T17:15:28.547 INFO:tasks.workunit.client.1.smithi115.stdout:2/759: symlink d4/d6/d56/d6c/d34/d42/d8d/da9/l106 0 2021-11-03T17:15:28.555 INFO:tasks.workunit.client.1.smithi115.stdout:7/665: dwrite d3/da/d18/d73/d33/f5e [0,4194304] 0 2021-11-03T17:15:28.558 INFO:tasks.workunit.client.1.smithi115.stdout:0/930: creat d9/d15/d1a/d1d/dcc/d46/d63/d70/f134 x:0 0 0 2021-11-03T17:15:28.569 INFO:tasks.workunit.client.1.smithi115.stdout:5/883: dread d1a/d28/d4c/d75/dae/d114/fa9 [0,4194304] 0 2021-11-03T17:15:28.569 INFO:tasks.workunit.client.1.smithi115.stdout:4/935: dwrite d7/dc/d1a/d21/db1/d56/d63/f6e [0,4194304] 0 2021-11-03T17:15:28.570 INFO:tasks.workunit.client.1.smithi115.stdout:0/931: symlink d9/d15/d1a/d1d/dcc/d6e/d71/l135 0 2021-11-03T17:15:28.572 INFO:tasks.workunit.client.1.smithi115.stdout:5/884: creat d1a/d28/d7b/d48/d7f/f129 x:0 0 0 2021-11-03T17:15:28.572 INFO:tasks.workunit.client.1.smithi115.stdout:5/885: chown d1a/d28/d34/deb 4175795 1 2021-11-03T17:15:28.573 INFO:tasks.workunit.client.1.smithi115.stdout:1/786: sync 2021-11-03T17:15:28.573 INFO:tasks.workunit.client.1.smithi115.stdout:5/886: mknod d1a/d28/daa/c12a 1 2021-11-03T17:15:28.576 INFO:tasks.workunit.client.1.smithi115.stdout:1/787: stat d7/dc/dd/d3e/dd1/d26/lf1 0 2021-11-03T17:15:28.577 INFO:tasks.workunit.client.1.smithi115.stdout:5/887: mkdir d1a/d3d/d58/dab/dd3/d12b 0 2021-11-03T17:15:28.581 INFO:tasks.workunit.client.1.smithi115.stdout:0/932: write d9/d15/d1a/d1d/dcc/d46/d63/f96 [2286569,4970] 0 2021-11-03T17:15:28.584 INFO:tasks.workunit.client.1.smithi115.stdout:1/788: dread d7/dc/dd/d3e/dd1/d43/df4/f69 [0,4194304] 0 2021-11-03T17:15:28.585 INFO:tasks.workunit.client.1.smithi115.stdout:4/936: dwrite d7/dc/d6a/d116/f137 [0,4194304] 0 2021-11-03T17:15:28.586 INFO:tasks.workunit.client.1.smithi115.stdout:5/888: truncate d1a/d28/d34/ff7 91424 0 2021-11-03T17:15:28.588 INFO:tasks.workunit.client.1.smithi115.stdout:5/889: mknod d1a/d28/d4c/d75/d83/da1/c12c 1 2021-11-03T17:15:28.596 INFO:tasks.workunit.client.1.smithi115.stdout:1/789: unlink d7/d9d/l9e 0 2021-11-03T17:15:28.604 INFO:tasks.workunit.client.1.smithi115.stdout:1/790: dread d7/dc/f15 [4194304,4194304] 0 2021-11-03T17:15:28.605 INFO:tasks.workunit.client.1.smithi115.stdout:1/791: dread - d7/dc/dd/d3e/dd1/d43/d78/ffe zero size 2021-11-03T17:15:28.621 INFO:tasks.workunit.client.1.smithi115.stdout:0/933: link d9/d15/d1a/laa d9/d15/d1a/d1d/l136 0 2021-11-03T17:15:28.626 INFO:tasks.workunit.client.1.smithi115.stdout:1/792: creat d7/dc/dd/d3e/dd1/d4b/f100 x:0 0 0 2021-11-03T17:15:28.633 INFO:tasks.workunit.client.1.smithi115.stdout:5/890: link d1a/d28/d4a/f8b d1a/d28/d4a/f12d 0 2021-11-03T17:15:28.633 INFO:tasks.workunit.client.1.smithi115.stdout:5/891: dread - d1a/d3d/dc0/f119 zero size 2021-11-03T17:15:28.633 INFO:tasks.workunit.client.1.smithi115.stdout:5/892: stat d1a/d28/d4c/f6e 0 2021-11-03T17:15:28.636 INFO:tasks.workunit.client.1.smithi115.stdout:0/934: dwrite d9/d15/d78/d80/f97 [0,4194304] 0 2021-11-03T17:15:28.636 INFO:tasks.workunit.client.1.smithi115.stdout:0/935: write d9/f110 [409423,91528] 0 2021-11-03T17:15:28.636 INFO:tasks.workunit.client.1.smithi115.stdout:1/793: write d7/dc/dd/d3e/dd1/d4b/f3a [1248795,52474] 0 2021-11-03T17:15:28.637 INFO:tasks.workunit.client.1.smithi115.stdout:5/893: mkdir d1a/d23/de6/d12e 0 2021-11-03T17:15:28.640 INFO:tasks.workunit.client.1.smithi115.stdout:1/794: write d7/d38/d84/f9a [1609688,45495] 0 2021-11-03T17:15:28.641 INFO:tasks.workunit.client.1.smithi115.stdout:5/894: symlink d1a/d28/d7b/d4f/d81/l12f 0 2021-11-03T17:15:28.646 INFO:tasks.workunit.client.1.smithi115.stdout:1/795: chown d7/dc/dd/d3e/dd1/d43/d78/fb1 757 1 2021-11-03T17:15:28.646 INFO:tasks.workunit.client.1.smithi115.stdout:5/895: creat d1a/d3d/dc0/f130 x:0 0 0 2021-11-03T17:15:28.647 INFO:tasks.workunit.client.1.smithi115.stdout:5/896: truncate d1a/d28/d7b/d4f/fdc 40641 0 2021-11-03T17:15:28.651 INFO:tasks.workunit.client.1.smithi115.stdout:1/796: truncate d7/dc/d1a/fd7 1862327 0 2021-11-03T17:15:28.661 INFO:tasks.workunit.client.1.smithi115.stdout:5/897: mkdir d1a/d3d/d58/dab/d131 0 2021-11-03T17:15:28.664 INFO:tasks.workunit.client.1.smithi115.stdout:5/898: mkdir d1a/d28/d7b/d48/dd4/d132 0 2021-11-03T17:15:28.665 INFO:tasks.workunit.client.1.smithi115.stdout:5/899: read d1a/d28/d4a/d65/f100 [160488,41006] 0 2021-11-03T17:15:28.667 INFO:tasks.workunit.client.1.smithi115.stdout:1/797: dwrite d7/dc/dd/d42/f47 [0,4194304] 0 2021-11-03T17:15:28.672 INFO:tasks.workunit.client.1.smithi115.stdout:1/798: getdents d7/d9d 0 2021-11-03T17:15:28.684 INFO:tasks.workunit.client.1.smithi115.stdout:1/799: rename d7/dc/dd/d3e/dd1 to d7/dc/d101 0 2021-11-03T17:15:28.686 INFO:tasks.workunit.client.1.smithi115.stdout:5/900: dwrite d1a/d3d/dc4/f118 [0,4194304] 0 2021-11-03T17:15:28.687 INFO:tasks.workunit.client.1.smithi115.stdout:5/901: chown d1a/d28/daa/d109/f11f 0 1 2021-11-03T17:15:28.703 INFO:tasks.workunit.client.1.smithi115.stdout:1/800: write d7/d38/fd4 [796018,24820] 0 2021-11-03T17:15:28.707 INFO:tasks.workunit.client.1.smithi115.stdout:1/801: readlink d7/dc/dd/d3e/d58/dc2/dd2/le5 0 2021-11-03T17:15:28.708 INFO:tasks.workunit.client.1.smithi115.stdout:5/902: mkdir d1a/d133 0 2021-11-03T17:15:28.710 INFO:tasks.workunit.client.1.smithi115.stdout:1/802: write d7/d38/d59/f9b [2056348,2828] 0 2021-11-03T17:15:28.713 INFO:tasks.workunit.client.1.smithi115.stdout:5/903: symlink d1a/d3d/d58/dab/daf/l134 0 2021-11-03T17:15:28.718 INFO:tasks.workunit.client.1.smithi115.stdout:5/904: link d1a/d28/daa/d109/f110 d1a/d3d/d58/dab/d131/f135 0 2021-11-03T17:15:28.718 INFO:tasks.workunit.client.1.smithi115.stdout:5/905: dread - d1a/d3d/dc0/ddf/f125 zero size 2021-11-03T17:15:28.735 INFO:tasks.workunit.client.1.smithi115.stdout:5/906: dwrite d1a/d3d/dc4/fe9 [0,4194304] 0 2021-11-03T17:15:28.756 INFO:tasks.workunit.client.1.smithi115.stdout:5/907: write d1a/f1f [976462,82664] 0 2021-11-03T17:15:28.758 INFO:tasks.workunit.client.1.smithi115.stdout:5/908: write d1a/d3d/f3b [8800290,50467] 0 2021-11-03T17:15:28.765 INFO:tasks.workunit.client.1.smithi115.stdout:5/909: dread d1a/d28/f31 [8388608,4194304] 0 2021-11-03T17:15:28.766 INFO:tasks.workunit.client.1.smithi115.stdout:5/910: dread d1a/d28/d34/f2a [0,4194304] 0 2021-11-03T17:15:28.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:28 smithi027 conmon[32322]: cluster 2021-11-03T17:15:26.646533 2021-11-03T17:15:28.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:28 smithi027 conmon[32322]: +0000 mgr.smithi027.ufhhvb (mgr.14622) 29 : cluster [DBG] pgmap v17: 65 pgs: 65 active+clean; 3.0 GiB data, 9.1 GiB used, 527 GiB / 536 GiB avail; 16 MiB/s rd, 33 MiB/s wr, 410 op/s 2021-11-03T17:15:28.769 INFO:tasks.workunit.client.1.smithi115.stdout:5/911: rename d1a/d28/d34/ff7 to d1a/d28/d34/f136 0 2021-11-03T17:15:28.807 INFO:tasks.workunit.client.1.smithi115.stdout:5/912: dwrite d1a/d28/d78/f105 [0,4194304] 0 2021-11-03T17:15:28.807 INFO:tasks.workunit.client.1.smithi115.stdout:5/913: chown d1a/d28/fec 84 1 2021-11-03T17:15:28.809 INFO:tasks.workunit.client.1.smithi115.stdout:5/914: dread - d1a/d28/d7b/d4f/fc9 zero size 2021-11-03T17:15:28.812 INFO:tasks.workunit.client.1.smithi115.stdout:5/915: mkdir d1a/d28/df9/d137 0 2021-11-03T17:15:28.826 INFO:tasks.workunit.client.1.smithi115.stdout:5/916: mkdir d1a/d23/d138 0 2021-11-03T17:15:28.827 INFO:tasks.workunit.client.1.smithi115.stdout:3/921: sync 2021-11-03T17:15:28.830 INFO:tasks.workunit.client.1.smithi115.stdout:3/922: mknod d13/d35/d54/dad/c13b 1 2021-11-03T17:15:28.831 INFO:tasks.workunit.client.1.smithi115.stdout:3/923: write d13/f4c [6481004,130231] 0 2021-11-03T17:15:28.834 INFO:tasks.workunit.client.1.smithi115.stdout:5/917: unlink d1a/d28/f29 0 2021-11-03T17:15:28.838 INFO:tasks.workunit.client.1.smithi115.stdout:5/918: unlink d1a/d28/d34/f2a 0 2021-11-03T17:15:28.839 INFO:tasks.workunit.client.1.smithi115.stdout:3/924: dread d13/d2a/d60/f95 [0,4194304] 0 2021-11-03T17:15:28.839 INFO:tasks.workunit.client.1.smithi115.stdout:5/919: stat d1a/d28/d4c/d75/d68/f6c 0 2021-11-03T17:15:28.840 INFO:tasks.workunit.client.1.smithi115.stdout:5/920: write d1a/d28/df9/fc1 [259552,83869] 0 2021-11-03T17:15:28.841 INFO:tasks.workunit.client.1.smithi115.stdout:3/925: creat d13/d2a/d53/f13c x:0 0 0 2021-11-03T17:15:28.853 INFO:tasks.workunit.client.1.smithi115.stdout:3/926: unlink d13/d51/d2e/l114 0 2021-11-03T17:15:28.853 INFO:tasks.workunit.client.1.smithi115.stdout:3/927: fsync d13/d67/f105 0 2021-11-03T17:15:28.854 INFO:tasks.workunit.client.1.smithi115.stdout:3/928: chown d13/d2a/d8a/fc8 3 1 2021-11-03T17:15:28.858 INFO:tasks.workunit.client.1.smithi115.stdout:3/929: mkdir d13/d35/d54/d13d 0 2021-11-03T17:15:28.860 INFO:tasks.workunit.client.1.smithi115.stdout:3/930: mkdir d13/d2a/d60/dd3/d13e 0 2021-11-03T17:15:28.861 INFO:tasks.workunit.client.1.smithi115.stdout:3/931: mknod d13/d3b/c13f 1 2021-11-03T17:15:28.862 INFO:tasks.workunit.client.1.smithi115.stdout:5/921: dwrite d1a/d28/f91 [0,4194304] 0 2021-11-03T17:15:28.862 INFO:tasks.workunit.client.1.smithi115.stdout:3/932: creat d13/d2a/d8a/f140 x:0 0 0 2021-11-03T17:15:28.870 INFO:tasks.workunit.client.1.smithi115.stdout:5/922: mknod d1a/d28/d7b/d48/c139 1 2021-11-03T17:15:28.871 INFO:tasks.workunit.client.1.smithi115.stdout:5/923: write d1a/d28/d4a/f8b [574473,113276] 0 2021-11-03T17:15:28.871 INFO:tasks.workunit.client.1.smithi115.stdout:5/924: dread - d1a/d23/de6/f10c zero size 2021-11-03T17:15:28.881 INFO:tasks.workunit.client.1.smithi115.stdout:3/933: dwrite fc [4194304,4194304] 0 2021-11-03T17:15:28.882 INFO:tasks.workunit.client.1.smithi115.stdout:5/925: rmdir d1a/d28/d4c/d75/d83/da1/d103 0 2021-11-03T17:15:28.882 INFO:tasks.workunit.client.1.smithi115.stdout:3/934: read d13/d35/d54/fab [1567822,121246] 0 2021-11-03T17:15:28.882 INFO:tasks.workunit.client.1.smithi115.stdout:3/935: mknod d13/d2a/d53/d72/d12e/c141 1 2021-11-03T17:15:28.882 INFO:tasks.workunit.client.1.smithi115.stdout:3/936: mknod d13/d50/dfc/c142 1 2021-11-03T17:15:28.882 INFO:tasks.workunit.client.1.smithi115.stdout:3/937: truncate d13/d51/d2e/fa2 8898233 0 2021-11-03T17:15:28.894 INFO:tasks.workunit.client.1.smithi115.stdout:3/938: dread ff [0,4194304] 0 2021-11-03T17:15:28.897 INFO:tasks.workunit.client.1.smithi115.stdout:3/939: link d13/d2a/l8d d13/d50/df9/l143 0 2021-11-03T17:15:28.898 INFO:tasks.workunit.client.1.smithi115.stdout:3/940: chown d13/d3b/db1/dbc 2639070 1 2021-11-03T17:15:28.906 INFO:tasks.workunit.client.1.smithi115.stdout:5/926: dwrite d1a/d28/d4a/f8b [0,4194304] 0 2021-11-03T17:15:28.908 INFO:tasks.workunit.client.1.smithi115.stdout:5/927: chown d1a/d28/df9/fa7 1505 1 2021-11-03T17:15:28.912 INFO:tasks.workunit.client.1.smithi115.stdout:5/928: creat d1a/d28/d7b/d48/f13a x:0 0 0 2021-11-03T17:15:28.914 INFO:tasks.workunit.client.1.smithi115.stdout:3/941: dwrite d13/d3b/f98 [0,4194304] 0 2021-11-03T17:15:28.919 INFO:tasks.workunit.client.1.smithi115.stdout:5/929: stat d1a/d3d/f62 0 2021-11-03T17:15:28.942 INFO:tasks.workunit.client.1.smithi115.stdout:5/930: link d1a/d28/d4a/d65/f100 d1a/d28/daa/f13b 0 2021-11-03T17:15:28.946 INFO:tasks.workunit.client.1.smithi115.stdout:5/931: truncate d1a/d28/d7b/d48/fca 1454811 0 2021-11-03T17:15:28.948 INFO:tasks.workunit.client.1.smithi115.stdout:5/932: creat d1a/d3d/d58/dab/d131/f13c x:0 0 0 2021-11-03T17:15:28.950 INFO:tasks.workunit.client.1.smithi115.stdout:5/933: truncate d1a/d28/fec 373852 0 2021-11-03T17:15:28.951 INFO:tasks.workunit.client.1.smithi115.stdout:3/942: dwrite d13/d51/d40/ff7 [0,4194304] 0 2021-11-03T17:15:28.974 INFO:tasks.workunit.client.1.smithi115.stdout:5/934: dwrite d1a/d23/f5c [0,4194304] 0 2021-11-03T17:15:28.978 INFO:tasks.workunit.client.1.smithi115.stdout:5/935: creat d1a/d23/de6/d12e/f13d x:0 0 0 2021-11-03T17:15:28.980 INFO:tasks.workunit.client.1.smithi115.stdout:5/936: unlink d1a/d28/d78/f105 0 2021-11-03T17:15:28.983 INFO:tasks.workunit.client.1.smithi115.stdout:5/937: unlink d1a/d3d/f62 0 2021-11-03T17:15:28.984 INFO:tasks.workunit.client.1.smithi115.stdout:5/938: creat d1a/d28/d7b/d48/dd4/f13e x:0 0 0 2021-11-03T17:15:28.984 INFO:tasks.workunit.client.1.smithi115.stdout:5/939: getdents d1a/d28/d4c/d63 0 2021-11-03T17:15:29.012 INFO:tasks.workunit.client.1.smithi115.stdout:5/940: dwrite d1a/d3d/dc0/ddf/f125 [0,4194304] 0 2021-11-03T17:15:29.013 INFO:tasks.workunit.client.1.smithi115.stdout:3/943: dwrite d13/d3b/db1/fd7 [0,4194304] 0 2021-11-03T17:15:29.015 INFO:tasks.workunit.client.1.smithi115.stdout:9/697: sync 2021-11-03T17:15:29.021 INFO:tasks.workunit.client.1.smithi115.stdout:5/941: creat d1a/d28/df9/f13f x:0 0 0 2021-11-03T17:15:29.021 INFO:tasks.workunit.client.1.smithi115.stdout:5/942: chown d1a/d28/d4a/d65/db5/f67 1790 1 2021-11-03T17:15:29.021 INFO:tasks.workunit.client.1.smithi115.stdout:5/943: truncate d1a/d23/de6/f10c 148149 0 2021-11-03T17:15:29.029 INFO:tasks.workunit.client.1.smithi115.stdout:3/944: rmdir d13/d51/d2e/db4/d121 0 2021-11-03T17:15:29.035 INFO:tasks.workunit.client.1.smithi115.stdout:9/698: symlink d1/d1a/d48/d27/d53/d83/d88/dcd/dda/lfb 0 2021-11-03T17:15:29.044 INFO:tasks.workunit.client.1.smithi115.stdout:5/944: creat d1a/d10f/f140 x:0 0 0 2021-11-03T17:15:29.044 INFO:tasks.workunit.client.1.smithi115.stdout:5/945: chown d1a/d28/d7b/ff4 340 1 2021-11-03T17:15:29.045 INFO:tasks.workunit.client.1.smithi115.stdout:3/945: getdents d13/d35/d54/df6 0 2021-11-03T17:15:29.052 INFO:tasks.workunit.client.1.smithi115.stdout:9/699: mknod d1/d1a/d48/d27/d53/cfc 1 2021-11-03T17:15:29.056 INFO:tasks.workunit.client.1.smithi115.stdout:9/700: symlink d1/lfd 0 2021-11-03T17:15:29.059 INFO:tasks.workunit.client.1.smithi115.stdout:5/946: dwrite d1a/d28/daa/f13b [0,4194304] 0 2021-11-03T17:15:29.062 INFO:tasks.workunit.client.1.smithi115.stdout:9/701: unlink d1/d1a/d48/d27/d80/lc5 0 2021-11-03T17:15:29.063 INFO:tasks.workunit.client.1.smithi115.stdout:3/946: dread d13/d2a/d60/f77 [0,4194304] 0 2021-11-03T17:15:29.064 INFO:tasks.workunit.client.1.smithi115.stdout:3/947: mknod d13/d2a/c144 1 2021-11-03T17:15:29.074 INFO:tasks.workunit.client.1.smithi115.stdout:3/948: symlink d13/l145 0 2021-11-03T17:15:29.074 INFO:tasks.workunit.client.1.smithi115.stdout:3/949: chown d13/d3b/db1/ffd 1399741844 1 2021-11-03T17:15:29.077 INFO:tasks.workunit.client.1.smithi115.stdout:3/950: mkdir d13/d50/dfc/d11a/d146 0 2021-11-03T17:15:29.090 INFO:tasks.workunit.client.1.smithi115.stdout:3/951: write d13/d2a/d53/d70/d92/faa [590382,19308] 0 2021-11-03T17:15:29.091 INFO:tasks.workunit.client.1.smithi115.stdout:3/952: fdatasync d13/d2a/d53/d70/f102 0 2021-11-03T17:15:29.098 INFO:tasks.workunit.client.1.smithi115.stdout:3/953: rmdir d13/d2a/d8a/d99 39 2021-11-03T17:15:29.098 INFO:tasks.workunit.client.1.smithi115.stdout:3/954: truncate d13/d50/d10b/f125 574753 0 2021-11-03T17:15:29.140 INFO:tasks.workunit.client.1.smithi115.stdout:3/955: dwrite d13/d3b/f123 [0,4194304] 0 2021-11-03T17:15:29.202 INFO:tasks.workunit.client.1.smithi115.stdout:3/956: dwrite d13/d35/d54/dad/fed [0,4194304] 0 2021-11-03T17:15:29.209 INFO:tasks.workunit.client.1.smithi115.stdout:3/957: mkdir d13/d2a/d53/d72/dd2/df1/d116/d147 0 2021-11-03T17:15:29.210 INFO:tasks.workunit.client.1.smithi115.stdout:3/958: read d13/d2a/d53/f8c [1815028,98014] 0 2021-11-03T17:15:29.252 INFO:tasks.workunit.client.1.smithi115.stdout:3/959: write d13/d2c/f65 [661869,33998] 0 2021-11-03T17:15:29.252 INFO:tasks.workunit.client.1.smithi115.stdout:3/960: truncate d13/d2a/d60/dd3/fda 741465 0 2021-11-03T17:15:29.319 INFO:tasks.workunit.client.1.smithi115.stdout:3/961: dwrite d13/d2a/d60/f95 [0,4194304] 0 2021-11-03T17:15:29.417 INFO:tasks.workunit.client.1.smithi115.stdout:3/962: symlink d13/d35/l148 0 2021-11-03T17:15:29.479 INFO:tasks.workunit.client.1.smithi115.stdout:3/963: rmdir d13/d2a/d53/d72 39 2021-11-03T17:15:29.495 INFO:tasks.workunit.client.1.smithi115.stdout:3/964: rename d13/d2a/d53/d70/d92 to d13/d2a/d53/d72/d12e/d149 0 2021-11-03T17:15:29.500 INFO:tasks.workunit.client.1.smithi115.stdout:3/965: rename d13/d67/f7e to d13/d3b/d11f/f14a 0 2021-11-03T17:15:29.507 INFO:tasks.workunit.client.1.smithi115.stdout:3/966: mknod d13/d51/d2e/db4/c14b 1 2021-11-03T17:15:29.510 INFO:tasks.workunit.client.1.smithi115.stdout:3/967: truncate d13/d2a/d60/f84 3941186 0 2021-11-03T17:15:29.511 INFO:tasks.workunit.client.1.smithi115.stdout:3/968: rmdir d13/d51/d40 39 2021-11-03T17:15:29.512 INFO:tasks.workunit.client.1.smithi115.stdout:3/969: stat d13/d51/d2e/dbf/f128 0 2021-11-03T17:15:29.536 INFO:tasks.workunit.client.1.smithi115.stdout:3/970: dwrite d13/d51/d40/f88 [0,4194304] 0 2021-11-03T17:15:29.550 INFO:tasks.workunit.client.1.smithi115.stdout:3/971: dwrite d13/d35/d54/f131 [0,4194304] 0 2021-11-03T17:15:29.553 INFO:tasks.workunit.client.1.smithi115.stdout:3/972: stat d13/d67/fd8 0 2021-11-03T17:15:29.553 INFO:tasks.workunit.client.1.smithi115.stdout:3/973: mknod d13/d2a/d8a/d99/c14c 1 2021-11-03T17:15:29.553 INFO:tasks.workunit.client.1.smithi115.stdout:3/974: mknod d13/d50/dfc/d11a/d132/c14d 1 2021-11-03T17:15:29.555 INFO:tasks.workunit.client.1.smithi115.stdout:3/975: write d13/d35/d54/f131 [3287353,107918] 0 2021-11-03T17:15:29.555 INFO:tasks.workunit.client.1.smithi115.stdout:3/976: truncate d13/d35/d54/dad/f100 955201 0 2021-11-03T17:15:29.555 INFO:tasks.workunit.client.1.smithi115.stdout:3/977: mknod d13/d50/df9/c14e 1 2021-11-03T17:15:29.571 INFO:tasks.workunit.client.1.smithi115.stdout:3/978: dwrite d13/d51/d40/f5a [0,4194304] 0 2021-11-03T17:15:29.590 INFO:tasks.workunit.client.1.smithi115.stdout:3/979: dread d13/d51/f41 [0,4194304] 0 2021-11-03T17:15:29.598 INFO:tasks.workunit.client.1.smithi115.stdout:3/980: write d13/d2a/d53/d72/d12e/d149/dc2/fca [729422,48748] 0 2021-11-03T17:15:29.598 INFO:tasks.workunit.client.1.smithi115.stdout:3/981: dread - d13/d2a/fe6 zero size 2021-11-03T17:15:29.614 INFO:tasks.workunit.client.1.smithi115.stdout:3/982: dwrite d13/fa9 [0,4194304] 0 2021-11-03T17:15:29.623 INFO:tasks.workunit.client.1.smithi115.stdout:3/983: write d13/d51/d40/f9d [1941557,896] 0 2021-11-03T17:15:29.626 INFO:tasks.workunit.client.1.smithi115.stdout:3/984: dread d13/d2a/d60/f77 [0,4194304] 0 2021-11-03T17:15:29.626 INFO:tasks.workunit.client.1.smithi115.stdout:3/985: chown f12 24 1 2021-11-03T17:15:29.626 INFO:tasks.workunit.client.1.smithi115.stdout:3/986: chown fc 349227719 1 2021-11-03T17:15:29.630 INFO:tasks.workunit.client.1.smithi115.stdout:3/987: fdatasync d13/d2a/d53/d72/d12e/d149/dc2/fca 0 2021-11-03T17:15:29.648 INFO:tasks.workunit.client.1.smithi115.stdout:3/988: dwrite d13/d2a/f4e [0,4194304] 0 2021-11-03T17:15:29.652 INFO:tasks.workunit.client.1.smithi115.stdout:3/989: mknod d13/d3b/db1/c14f 1 2021-11-03T17:15:29.964 INFO:tasks.workunit.client.1.smithi115.stdout:6/668: sync 2021-11-03T17:15:29.964 INFO:tasks.workunit.client.1.smithi115.stdout:6/669: dread - d0/d7f/fa9 zero size 2021-11-03T17:15:29.965 INFO:tasks.workunit.client.1.smithi115.stdout:6/670: dread - d0/d1/d10/f9a zero size 2021-11-03T17:15:29.975 INFO:tasks.workunit.client.1.smithi115.stdout:6/671: rmdir d0/d1/d10/d13/d59 39 2021-11-03T17:15:29.991 INFO:tasks.workunit.client.1.smithi115.stdout:6/672: dwrite d0/d1/f19 [0,4194304] 0 2021-11-03T17:15:30.008 INFO:tasks.workunit.client.1.smithi115.stdout:6/673: dwrite d0/d1/f8 [0,4194304] 0 2021-11-03T17:15:30.033 INFO:tasks.workunit.client.1.smithi115.stdout:8/694: sync 2021-11-03T17:15:30.034 INFO:tasks.workunit.client.1.smithi115.stdout:8/695: chown d4/d6/d3f/l4b 84877039 1 2021-11-03T17:15:30.034 INFO:tasks.workunit.client.1.smithi115.stdout:8/696: chown d4/d6/f2c 23436 1 2021-11-03T17:15:30.046 INFO:tasks.workunit.client.1.smithi115.stdout:8/697: write d4/d6/d18/d30/f31 [314857,1266] 0 2021-11-03T17:15:30.048 INFO:tasks.workunit.client.1.smithi115.stdout:8/698: creat d4/d6/d18/d30/d79/d7f/ff8 x:0 0 0 2021-11-03T17:15:30.049 INFO:tasks.workunit.client.1.smithi115.stdout:8/699: write d4/d6/d18/d30/d33/f5c [1207475,58947] 0 2021-11-03T17:15:30.051 INFO:tasks.workunit.client.1.smithi115.stdout:8/700: mkdir d4/d7d/dc2/df9 0 2021-11-03T17:15:30.051 INFO:tasks.workunit.client.1.smithi115.stdout:8/701: dread - d4/d6/d3f/dde/fe8 zero size 2021-11-03T17:15:30.054 INFO:tasks.workunit.client.1.smithi115.stdout:8/702: symlink d4/d6/d18/d30/d79/de7/lfa 0 2021-11-03T17:15:30.054 INFO:tasks.workunit.client.1.smithi115.stdout:8/703: fsync d4/d6/d3f/da4/fea 0 2021-11-03T17:15:30.057 INFO:tasks.workunit.client.1.smithi115.stdout:8/704: creat d4/d7d/ffb x:0 0 0 2021-11-03T17:15:30.057 INFO:tasks.workunit.client.1.smithi115.stdout:8/705: write d4/d7d/fe6 [14535,11857] 0 2021-11-03T17:15:30.060 INFO:tasks.workunit.client.1.smithi115.stdout:8/706: mkdir d4/d6/d18/d30/dfc 0 2021-11-03T17:15:30.063 INFO:tasks.workunit.client.1.smithi115.stdout:8/707: read d4/d6/f8 [4725881,72918] 0 2021-11-03T17:15:30.063 INFO:tasks.workunit.client.1.smithi115.stdout:8/708: dread - d4/d6/d18/d44/d47/fd8 zero size 2021-11-03T17:15:30.064 INFO:tasks.workunit.client.1.smithi115.stdout:8/709: truncate d4/d6/d18/d9d/fba 536154 0 2021-11-03T17:15:30.065 INFO:tasks.workunit.client.1.smithi115.stdout:8/710: mkdir d4/d6/d3f/da4/dfd 0 2021-11-03T17:15:30.065 INFO:tasks.workunit.client.1.smithi115.stdout:8/711: readlink d4/d6/d3f/l4b 0 2021-11-03T17:15:30.066 INFO:tasks.workunit.client.1.smithi115.stdout:8/712: write d4/d6/d3f/d43/d88/fec [822024,128074] 0 2021-11-03T17:15:30.081 INFO:tasks.workunit.client.1.smithi115.stdout:4/937: sync 2021-11-03T17:15:30.081 INFO:tasks.workunit.client.1.smithi115.stdout:1/803: sync 2021-11-03T17:15:30.081 INFO:tasks.workunit.client.1.smithi115.stdout:2/760: sync 2021-11-03T17:15:30.081 INFO:tasks.workunit.client.1.smithi115.stdout:7/666: sync 2021-11-03T17:15:30.082 INFO:tasks.workunit.client.1.smithi115.stdout:4/938: read d7/dc/f95 [322242,66525] 0 2021-11-03T17:15:30.083 INFO:tasks.workunit.client.1.smithi115.stdout:4/939: fsync d7/dc/d6a/f6f 0 2021-11-03T17:15:30.083 INFO:tasks.workunit.client.1.smithi115.stdout:4/940: read d7/d7a/fa3 [306727,103442] 0 2021-11-03T17:15:30.083 INFO:tasks.workunit.client.1.smithi115.stdout:1/804: chown d7/dc/dd/f95 1432 1 2021-11-03T17:15:30.084 INFO:tasks.workunit.client.1.smithi115.stdout:2/761: getdents d4/d6/dad/dc2 0 2021-11-03T17:15:30.084 INFO:tasks.workunit.client.1.smithi115.stdout:2/762: write d4/d6/dad/de8/dac/dec/ff0 [873804,70457] 0 2021-11-03T17:15:30.086 INFO:tasks.workunit.client.1.smithi115.stdout:8/713: dwrite d4/d6/d18/d30/d33/f59 [4194304,4194304] 0 2021-11-03T17:15:30.092 INFO:tasks.workunit.client.1.smithi115.stdout:7/667: truncate d3/da/d18/d73/f89 3794840 0 2021-11-03T17:15:30.099 INFO:tasks.workunit.client.1.smithi115.stdout:4/941: dread d7/dc/db6/f9a [0,4194304] 0 2021-11-03T17:15:30.100 INFO:tasks.workunit.client.1.smithi115.stdout:4/942: dread - d7/dc/d1a/d21/db1/d56/d63/f107 zero size 2021-11-03T17:15:30.109 INFO:tasks.workunit.client.1.smithi115.stdout:8/714: creat d4/d6/d18/d30/d79/ffe x:0 0 0 2021-11-03T17:15:30.115 INFO:tasks.workunit.client.1.smithi115.stdout:7/668: creat d3/da/d18/fce x:0 0 0 2021-11-03T17:15:30.117 INFO:tasks.workunit.client.1.smithi115.stdout:1/805: truncate d7/dc/d101/d43/f6f 482459 0 2021-11-03T17:15:30.120 INFO:tasks.workunit.client.1.smithi115.stdout:4/943: symlink d7/dc/d6a/d116/l13d 0 2021-11-03T17:15:30.121 INFO:tasks.workunit.client.1.smithi115.stdout:4/944: mknod d7/dc/d1a/d21/db1/d56/df4/c13e 1 2021-11-03T17:15:30.121 INFO:tasks.workunit.client.1.smithi115.stdout:4/945: stat d7/dc/d1a/d21/f105 0 2021-11-03T17:15:30.121 INFO:tasks.workunit.client.1.smithi115.stdout:4/946: chown d7/d7a/fc3 307 1 2021-11-03T17:15:30.122 INFO:tasks.workunit.client.1.smithi115.stdout:4/947: write d7/dc/d1a/d21/fbe [8061374,12032] 0 2021-11-03T17:15:30.123 INFO:tasks.workunit.client.1.smithi115.stdout:2/763: rmdir d4/d6/d56/d6c/d34/d43/d63/d69 39 2021-11-03T17:15:30.124 INFO:tasks.workunit.client.1.smithi115.stdout:8/715: truncate d4/d6/fa9 1700394 0 2021-11-03T17:15:30.127 INFO:tasks.workunit.client.1.smithi115.stdout:1/806: mkdir d7/d38/d59/d102 0 2021-11-03T17:15:30.127 INFO:tasks.workunit.client.1.smithi115.stdout:8/716: creat d4/d6/d18/d30/d79/de7/fff x:0 0 0 2021-11-03T17:15:30.129 INFO:tasks.workunit.client.1.smithi115.stdout:7/669: mkdir d3/d4c/d7b/dcf 0 2021-11-03T17:15:30.129 INFO:tasks.workunit.client.1.smithi115.stdout:7/670: read - d3/d4c/fa5 zero size 2021-11-03T17:15:30.130 INFO:tasks.workunit.client.1.smithi115.stdout:8/717: creat d4/d6/d18/d30/f100 x:0 0 0 2021-11-03T17:15:30.134 INFO:tasks.workunit.client.1.smithi115.stdout:7/671: read d3/da/d18/d73/d35/d53/fb7 [116648,46273] 0 2021-11-03T17:15:30.134 INFO:tasks.workunit.client.1.smithi115.stdout:2/764: dread d4/d6/d56/d6c/d34/d43/d63/d77/fde [4194304,4194304] 0 2021-11-03T17:15:30.143 INFO:tasks.workunit.client.1.smithi115.stdout:2/765: creat d4/d6/d56/d6c/d34/d43/d63/d69/f107 x:0 0 0 2021-11-03T17:15:30.144 INFO:tasks.workunit.client.1.smithi115.stdout:4/948: dwrite d7/dc/db6/fc9 [0,4194304] 0 2021-11-03T17:15:30.153 INFO:tasks.workunit.client.1.smithi115.stdout:1/807: dwrite d7/dc/d1a/f6b [0,4194304] 0 2021-11-03T17:15:30.155 INFO:tasks.workunit.client.1.smithi115.stdout:7/672: mkdir d3/d4c/d7b/dd0 0 2021-11-03T17:15:30.157 INFO:tasks.workunit.client.1.smithi115.stdout:2/766: mkdir d4/d6/d13/d108 0 2021-11-03T17:15:30.165 INFO:tasks.workunit.client.1.smithi115.stdout:2/767: chown d4/d6/dad/de8/d9c/lb9 44050 1 2021-11-03T17:15:30.167 INFO:tasks.workunit.client.1.smithi115.stdout:1/808: mkdir d7/d38/d84/dca/d103 0 2021-11-03T17:15:30.173 INFO:tasks.workunit.client.1.smithi115.stdout:0/936: sync 2021-11-03T17:15:30.178 INFO:tasks.workunit.client.1.smithi115.stdout:8/718: dwrite d4/d6/f2a [0,4194304] 0 2021-11-03T17:15:30.180 INFO:tasks.workunit.client.1.smithi115.stdout:0/937: getdents d9/d15 0 2021-11-03T17:15:30.180 INFO:tasks.workunit.client.1.smithi115.stdout:0/938: chown d9/d15/d1a/d1d/dcc/d46/d63/dc5/lc1 168793 1 2021-11-03T17:15:30.180 INFO:tasks.workunit.client.1.smithi115.stdout:4/949: dwrite d7/dc/d1a/f38 [0,4194304] 0 2021-11-03T17:15:30.181 INFO:tasks.workunit.client.1.smithi115.stdout:4/950: write d7/dc/d1a/d21/db1/fe4 [176083,19179] 0 2021-11-03T17:15:30.181 INFO:tasks.workunit.client.1.smithi115.stdout:2/768: dread d4/f35 [0,4194304] 0 2021-11-03T17:15:30.187 INFO:tasks.workunit.client.1.smithi115.stdout:7/673: dwrite d3/da/d18/d73/d35/f5f [4194304,4194304] 0 2021-11-03T17:15:30.188 INFO:tasks.workunit.client.1.smithi115.stdout:1/809: dwrite d7/dc/d101/d26/f4f [0,4194304] 0 2021-11-03T17:15:30.195 INFO:tasks.workunit.client.1.smithi115.stdout:4/951: symlink d7/dc/d1a/d21/db1/d56/d42/l13f 0 2021-11-03T17:15:30.195 INFO:tasks.workunit.client.1.smithi115.stdout:4/952: chown d7/dc/d1a/d21/db1/d56/f8d 5 1 2021-11-03T17:15:30.197 INFO:tasks.workunit.client.1.smithi115.stdout:2/769: truncate d4/f78 2701181 0 2021-11-03T17:15:30.200 INFO:tasks.workunit.client.1.smithi115.stdout:8/719: rename d4/d6/d18/l78 to d4/d6/d18/d44/dcd/dd0/l101 0 2021-11-03T17:15:30.206 INFO:tasks.workunit.client.1.smithi115.stdout:7/674: dwrite d3/da/f90 [0,4194304] 0 2021-11-03T17:15:30.206 INFO:tasks.workunit.client.1.smithi115.stdout:7/675: chown d3/da/d18/f22 15844768 1 2021-11-03T17:15:30.215 INFO:tasks.workunit.client.1.smithi115.stdout:0/939: write d9/d15/d1a/d1d/dcc/d6e/fc9 [563297,31981] 0 2021-11-03T17:15:30.215 INFO:tasks.workunit.client.1.smithi115.stdout:0/940: mknod d9/d15/d1a/d1d/dcc/d46/d63/dc5/c137 1 2021-11-03T17:15:30.222 INFO:tasks.workunit.client.1.smithi115.stdout:4/953: dwrite d7/dc/d1a/f1c [0,4194304] 0 2021-11-03T17:15:30.222 INFO:tasks.workunit.client.1.smithi115.stdout:4/954: chown d7/dc/d6a/d116/l13d 1322 1 2021-11-03T17:15:30.223 INFO:tasks.workunit.client.1.smithi115.stdout:4/955: readlink d7/dc/d1a/d21/db1/d56/d42/daf/lec 0 2021-11-03T17:15:30.233 INFO:tasks.workunit.client.1.smithi115.stdout:0/941: dwrite d9/fa8 [0,4194304] 0 2021-11-03T17:15:30.233 INFO:tasks.workunit.client.1.smithi115.stdout:0/942: getdents d9/d15/d1a/da7/d12a 0 2021-11-03T17:15:30.233 INFO:tasks.workunit.client.1.smithi115.stdout:0/943: fsync d9/db/f47 0 2021-11-03T17:15:30.233 INFO:tasks.workunit.client.1.smithi115.stdout:0/944: write d9/d15/d1a/d1d/dcc/d46/d63/dae/ff2 [890478,20075] 0 2021-11-03T17:15:30.244 INFO:tasks.workunit.client.1.smithi115.stdout:7/676: readlink d3/l10 0 2021-11-03T17:15:30.245 INFO:tasks.workunit.client.1.smithi115.stdout:7/677: stat d3/da/d18/d73/l81 0 2021-11-03T17:15:30.245 INFO:tasks.workunit.client.1.smithi115.stdout:7/678: readlink d3/da/d18/d73/d33/l38 0 2021-11-03T17:15:30.246 INFO:tasks.workunit.client.1.smithi115.stdout:4/956: mkdir d7/dc/d6a/d3c/d112/d130/d13b/d140 0 2021-11-03T17:15:30.265 INFO:tasks.workunit.client.1.smithi115.stdout:0/945: creat d9/d15/d1a/d1d/dcc/d46/d63/dc5/f138 x:0 0 0 2021-11-03T17:15:30.265 INFO:tasks.workunit.client.1.smithi115.stdout:0/946: write d9/f110 [825048,15818] 0 2021-11-03T17:15:30.266 INFO:tasks.workunit.client.1.smithi115.stdout:2/770: link d4/d6/d56/d6c/d34/ff2 d4/d6/dad/de8/d9c/d9f/f109 0 2021-11-03T17:15:30.266 INFO:tasks.workunit.client.1.smithi115.stdout:2/771: mknod d4/d6/dad/d7f/de9/c10a 1 2021-11-03T17:15:30.266 INFO:tasks.workunit.client.1.smithi115.stdout:2/772: write d4/d6/d56/d6c/d34/d43/d28/f3b [9088404,39929] 0 2021-11-03T17:15:30.267 INFO:tasks.workunit.client.1.smithi115.stdout:7/679: mkdir d3/da/d18/d73/d52/db4/dd1 0 2021-11-03T17:15:30.269 INFO:tasks.workunit.client.1.smithi115.stdout:4/957: rmdir d7/dc/d6a/d3c/d112 39 2021-11-03T17:15:30.269 INFO:tasks.workunit.client.1.smithi115.stdout:4/958: stat d7/dc/d1a/d21/db1/d56/d63/f9b 0 2021-11-03T17:15:30.270 INFO:tasks.workunit.client.1.smithi115.stdout:0/947: write d9/db/f8e [3111249,127020] 0 2021-11-03T17:15:30.274 INFO:tasks.workunit.client.1.smithi115.stdout:2/773: getdents d4/d6/dad/de8/d9c 0 2021-11-03T17:15:30.274 INFO:tasks.workunit.client.1.smithi115.stdout:2/774: write d4/d6/d56/d6c/d34/d42/f8f [4893969,7865] 0 2021-11-03T17:15:30.276 INFO:tasks.workunit.client.1.smithi115.stdout:7/680: rmdir d3/da/d78/dcd 0 2021-11-03T17:15:30.276 INFO:tasks.workunit.client.1.smithi115.stdout:4/959: truncate d7/d7a/d83/fd0 49577 0 2021-11-03T17:15:30.278 INFO:tasks.workunit.client.1.smithi115.stdout:2/775: getdents d4/d6/d56/d6c/d34/d43 0 2021-11-03T17:15:30.279 INFO:tasks.workunit.client.1.smithi115.stdout:8/720: dwrite d4/d6/d18/f23 [0,4194304] 0 2021-11-03T17:15:30.280 INFO:tasks.workunit.client.1.smithi115.stdout:8/721: dread - d4/d6/d18/d44/fa3 zero size 2021-11-03T17:15:30.280 INFO:tasks.workunit.client.1.smithi115.stdout:8/722: dread - d4/d6/d3f/d9a/fd6 zero size 2021-11-03T17:15:30.281 INFO:tasks.workunit.client.1.smithi115.stdout:4/960: creat d7/dc/d1a/d21/f141 x:0 0 0 2021-11-03T17:15:30.282 INFO:tasks.workunit.client.1.smithi115.stdout:1/810: dwrite d7/dc/d101/d43/df4/d60/f6d [0,4194304] 0 2021-11-03T17:15:30.282 INFO:tasks.workunit.client.1.smithi115.stdout:8/723: write d4/d6/d3f/d9a/f54 [1465507,64403] 0 2021-11-03T17:15:30.282 INFO:tasks.workunit.client.1.smithi115.stdout:8/724: readlink d4/d6/d18/d44/dcd/lda 0 2021-11-03T17:15:30.282 INFO:tasks.workunit.client.1.smithi115.stdout:8/725: write d4/d6/d18/d30/f100 [771034,49359] 0 2021-11-03T17:15:30.284 INFO:tasks.workunit.client.1.smithi115.stdout:2/776: creat d4/d6/dad/d7f/f10b x:0 0 0 2021-11-03T17:15:30.284 INFO:tasks.workunit.client.1.smithi115.stdout:2/777: chown d4/d6/d13/d75/fd5 84536279 1 2021-11-03T17:15:30.284 INFO:tasks.workunit.client.1.smithi115.stdout:2/778: dread - d4/d6/dad/fdd zero size 2021-11-03T17:15:30.285 INFO:tasks.workunit.client.1.smithi115.stdout:4/961: creat d7/d7a/d83/f142 x:0 0 0 2021-11-03T17:15:30.297 INFO:tasks.workunit.client.1.smithi115.stdout:7/681: dwrite d3/da/d18/d73/d35/fad [0,4194304] 0 2021-11-03T17:15:30.298 INFO:tasks.workunit.client.1.smithi115.stdout:7/682: read d3/d4c/d7b/f91 [3123714,91402] 0 2021-11-03T17:15:30.313 INFO:tasks.workunit.client.1.smithi115.stdout:2/779: rmdir d4/d6/dad/d7f 39 2021-11-03T17:15:30.313 INFO:tasks.workunit.client.1.smithi115.stdout:2/780: stat d4/d6/dad/d7f/fc0 0 2021-11-03T17:15:30.317 INFO:tasks.workunit.client.1.smithi115.stdout:8/726: write d4/d6/d3f/d9a/f84 [3369082,29492] 0 2021-11-03T17:15:30.332 INFO:tasks.workunit.client.1.smithi115.stdout:1/811: dwrite d7/f8b [0,4194304] 0 2021-11-03T17:15:30.344 INFO:tasks.workunit.client.1.smithi115.stdout:2/781: creat d4/d6/d13/d75/f10c x:0 0 0 2021-11-03T17:15:30.348 INFO:tasks.workunit.client.1.smithi115.stdout:1/812: dwrite d7/dc/d1a/f6b [0,4194304] 0 2021-11-03T17:15:30.348 INFO:tasks.workunit.client.1.smithi115.stdout:1/813: chown d7/dc/d101/d43/df4/fee 312638 1 2021-11-03T17:15:30.348 INFO:tasks.workunit.client.1.smithi115.stdout:1/814: truncate d7/dc/dd/d3e/d58/dc2/ff8 91989 0 2021-11-03T17:15:30.352 INFO:tasks.workunit.client.1.smithi115.stdout:4/962: rename d7/d7a/ff2 to d7/dc/d6a/d3c/d112/dc4/dc6/f143 0 2021-11-03T17:15:30.358 INFO:tasks.workunit.client.1.smithi115.stdout:7/683: mkdir d3/d4c/d7b/dd2 0 2021-11-03T17:15:30.363 INFO:tasks.workunit.client.1.smithi115.stdout:8/727: rmdir d4/d6/d18/d30/d33 39 2021-11-03T17:15:30.377 INFO:tasks.workunit.client.1.smithi115.stdout:4/963: rename d7/dc/l33 to d7/dc/d6a/l144 0 2021-11-03T17:15:30.378 INFO:tasks.workunit.client.1.smithi115.stdout:7/684: chown d3/da/d18/d73/d33/l55 355 1 2021-11-03T17:15:30.381 INFO:tasks.workunit.client.1.smithi115.stdout:8/728: link d4/d6/d18/d30/f34 d4/d6/d18/d44/d47/deb/f102 0 2021-11-03T17:15:30.387 INFO:tasks.workunit.client.1.smithi115.stdout:2/782: link d4/d6/dad/f60 d4/d6/dad/de8/d9c/f10d 0 2021-11-03T17:15:30.389 INFO:tasks.workunit.client.1.smithi115.stdout:4/964: rmdir d7/dc/db6 39 2021-11-03T17:15:30.390 INFO:tasks.workunit.client.1.smithi115.stdout:4/965: fdatasync d7/dc/d1a/d21/db1/d56/d63/d7b/fb2 0 2021-11-03T17:15:30.393 INFO:tasks.workunit.client.1.smithi115.stdout:7/685: rename d3/da/d18/d73/d35/d53/lbd to d3/da/d18/d73/d35/ld3 0 2021-11-03T17:15:30.401 INFO:tasks.workunit.client.1.smithi115.stdout:8/729: truncate d4/d6/d18/fa5 3718657 0 2021-11-03T17:15:30.401 INFO:tasks.workunit.client.1.smithi115.stdout:8/730: write d4/d7d/ff4 [879160,78826] 0 2021-11-03T17:15:30.402 INFO:tasks.workunit.client.1.smithi115.stdout:8/731: readlink d4/d6/d18/le1 0 2021-11-03T17:15:30.403 INFO:tasks.workunit.client.1.smithi115.stdout:1/815: truncate d7/dc/dd/d3e/d58/dbf/fd8 49938 0 2021-11-03T17:15:30.403 INFO:tasks.workunit.client.1.smithi115.stdout:1/816: fdatasync d7/dc/dd/d3e/d58/dc2/dd2/ff9 0 2021-11-03T17:15:30.407 INFO:tasks.workunit.client.1.smithi115.stdout:2/783: fdatasync d4/d6/d56/d6c/d34/f70 0 2021-11-03T17:15:30.411 INFO:tasks.workunit.client.1.smithi115.stdout:1/817: dread d7/dc/d1a/f6b [0,4194304] 0 2021-11-03T17:15:30.414 INFO:tasks.workunit.client.1.smithi115.stdout:4/966: creat d7/dc/d6a/d3c/dc5/dcc/f145 x:0 0 0 2021-11-03T17:15:30.414 INFO:tasks.workunit.client.1.smithi115.stdout:4/967: readlink d7/dc/d6a/l45 0 2021-11-03T17:15:30.419 INFO:tasks.workunit.client.1.smithi115.stdout:7/686: truncate d3/f47 3475112 0 2021-11-03T17:15:30.423 INFO:tasks.workunit.client.1.smithi115.stdout:5/947: sync 2021-11-03T17:15:30.434 INFO:tasks.workunit.client.1.smithi115.stdout:1/818: dwrite f2 [0,4194304] 0 2021-11-03T17:15:30.437 INFO:tasks.workunit.client.1.smithi115.stdout:7/687: readlink d3/da/d18/d73/l29 0 2021-11-03T17:15:30.437 INFO:tasks.workunit.client.1.smithi115.stdout:7/688: write d3/da/d18/fce [428771,13403] 0 2021-11-03T17:15:30.437 INFO:tasks.workunit.client.1.smithi115.stdout:7/689: mknod d3/da/d18/cd4 1 2021-11-03T17:15:30.438 INFO:tasks.workunit.client.1.smithi115.stdout:7/690: write d3/da/d18/d73/d35/f3f [76450,15350] 0 2021-11-03T17:15:30.445 INFO:tasks.workunit.client.1.smithi115.stdout:5/948: dwrite d1a/d28/d4a/f12d [0,4194304] 0 2021-11-03T17:15:30.445 INFO:tasks.workunit.client.1.smithi115.stdout:5/949: mknod d1a/d3d/d58/dab/d10d/c141 1 2021-11-03T17:15:30.448 INFO:tasks.workunit.client.1.smithi115.stdout:8/732: write d4/d6/d3f/da4/da6/dcc/ff6 [1453147,110184] 0 2021-11-03T17:15:30.449 INFO:tasks.workunit.client.1.smithi115.stdout:2/784: truncate d4/d6/dad/f60 767414 0 2021-11-03T17:15:30.457 INFO:tasks.workunit.client.1.smithi115.stdout:4/968: symlink d7/d7a/l146 0 2021-11-03T17:15:30.458 INFO:tasks.workunit.client.1.smithi115.stdout:7/691: creat d3/fd5 x:0 0 0 2021-11-03T17:15:30.465 INFO:tasks.workunit.client.1.smithi115.stdout:8/733: rmdir d4/d7d/dc2 39 2021-11-03T17:15:30.465 INFO:tasks.workunit.client.1.smithi115.stdout:8/734: chown d4/d6/d18/d30/f35 283249892 1 2021-11-03T17:15:30.466 INFO:tasks.workunit.client.1.smithi115.stdout:8/735: getdents d4/d6/d3f/da4/dfd 0 2021-11-03T17:15:30.467 INFO:tasks.workunit.client.1.smithi115.stdout:1/819: write d7/dc/dd/d42/f75 [618551,43104] 0 2021-11-03T17:15:30.467 INFO:tasks.workunit.client.1.smithi115.stdout:1/820: chown d7/dc/dd/d42 18511 1 2021-11-03T17:15:30.468 INFO:tasks.workunit.client.1.smithi115.stdout:1/821: write d7/dc/d101/d43/df4/fee [2823885,65699] 0 2021-11-03T17:15:30.469 INFO:tasks.workunit.client.1.smithi115.stdout:5/950: dwrite d1a/d28/d7b/d48/dd4/ff1 [0,4194304] 0 2021-11-03T17:15:30.469 INFO:tasks.workunit.client.1.smithi115.stdout:5/951: dread - d1a/d3d/dc0/f119 zero size 2021-11-03T17:15:30.470 INFO:tasks.workunit.client.1.smithi115.stdout:5/952: write d1a/d28/df9/f13f [123577,47788] 0 2021-11-03T17:15:30.470 INFO:tasks.workunit.client.1.smithi115.stdout:1/822: write d7/dc/d101/f2b [1005583,45705] 0 2021-11-03T17:15:30.474 INFO:tasks.workunit.client.1.smithi115.stdout:8/736: link d4/d6/d3f/d43/d88/fdc d4/d6/d18/d44/d47/deb/f103 0 2021-11-03T17:15:30.479 INFO:tasks.workunit.client.1.smithi115.stdout:2/785: link d4/d6/d56/d6c/d34/d42/d8d/fce d4/d6/d56/d6c/d34/d43/d28/f10e 0 2021-11-03T17:15:30.480 INFO:tasks.workunit.client.1.smithi115.stdout:7/692: dwrite d3/fd5 [0,4194304] 0 2021-11-03T17:15:30.480 INFO:tasks.workunit.client.1.smithi115.stdout:7/693: truncate d3/da/d18/faf 4206364 0 2021-11-03T17:15:30.481 INFO:tasks.workunit.client.1.smithi115.stdout:7/694: chown d3/da/f7e 466422491 1 2021-11-03T17:15:30.482 INFO:tasks.workunit.client.1.smithi115.stdout:4/969: dwrite d7/d7a/fc3 [0,4194304] 0 2021-11-03T17:15:30.486 INFO:tasks.workunit.client.1.smithi115.stdout:7/695: write d3/d4c/f5b [4047166,48352] 0 2021-11-03T17:15:30.490 INFO:tasks.workunit.client.1.smithi115.stdout:1/823: dwrite d7/dc/dd/ff7 [0,4194304] 0 2021-11-03T17:15:30.491 INFO:tasks.workunit.client.1.smithi115.stdout:5/953: symlink d1a/d133/l142 0 2021-11-03T17:15:30.491 INFO:tasks.workunit.client.1.smithi115.stdout:5/954: readlink d1a/d3d/d58/dab/dd3/l11a 0 2021-11-03T17:15:30.506 INFO:tasks.workunit.client.1.smithi115.stdout:2/786: creat d4/d6/dad/de8/f10f x:0 0 0 2021-11-03T17:15:30.508 INFO:tasks.workunit.client.1.smithi115.stdout:2/787: dread d4/d6/d56/d6c/d34/d43/d63/d77/fb1 [0,4194304] 0 2021-11-03T17:15:30.508 INFO:tasks.workunit.client.1.smithi115.stdout:2/788: chown d4/d6/d13/f4e 3726524 1 2021-11-03T17:15:30.509 INFO:tasks.workunit.client.1.smithi115.stdout:8/737: rmdir d4/d6 39 2021-11-03T17:15:30.509 INFO:tasks.workunit.client.1.smithi115.stdout:8/738: fdatasync d4/d6/d18/d30/d79/d7f/ff1 0 2021-11-03T17:15:30.531 INFO:tasks.workunit.client.1.smithi115.stdout:5/955: truncate d1a/d28/d4a/d65/f100 82469 0 2021-11-03T17:15:30.533 INFO:tasks.workunit.client.1.smithi115.stdout:7/696: symlink d3/d4c/d7b/ld6 0 2021-11-03T17:15:30.534 INFO:tasks.workunit.client.1.smithi115.stdout:5/956: creat d1a/d28/df9/d137/f143 x:0 0 0 2021-11-03T17:15:30.536 INFO:tasks.workunit.client.1.smithi115.stdout:5/957: dread d1a/d3d/d58/dab/daf/ff8 [0,4194304] 0 2021-11-03T17:15:30.569 INFO:tasks.workunit.client.1.smithi115.stdout:8/739: dwrite d4/d6/fd2 [0,4194304] 0 2021-11-03T17:15:30.569 INFO:tasks.workunit.client.1.smithi115.stdout:8/740: fsync d4/d6/d18/d30/d33/f59 0 2021-11-03T17:15:30.570 INFO:tasks.workunit.client.1.smithi115.stdout:8/741: fdatasync d4/d6/f3c 0 2021-11-03T17:15:30.573 INFO:tasks.workunit.client.1.smithi115.stdout:8/742: creat d4/d7d/dc2/df9/f104 x:0 0 0 2021-11-03T17:15:30.582 INFO:tasks.workunit.client.1.smithi115.stdout:5/958: dwrite d1a/d28/f91 [4194304,4194304] 0 2021-11-03T17:15:30.582 INFO:tasks.workunit.client.1.smithi115.stdout:5/959: read d1a/d28/df9/fc1 [278542,108626] 0 2021-11-03T17:15:30.583 INFO:tasks.workunit.client.1.smithi115.stdout:7/697: dwrite d3/d4c/f4d [4194304,4194304] 0 2021-11-03T17:15:30.584 INFO:tasks.workunit.client.1.smithi115.stdout:8/743: symlink d4/d6/d18/d44/l105 0 2021-11-03T17:15:30.592 INFO:tasks.workunit.client.1.smithi115.stdout:5/960: mkdir d1a/d28/d7b/d48/dd4/d132/d144 0 2021-11-03T17:15:30.600 INFO:tasks.workunit.client.1.smithi115.stdout:7/698: creat d3/d4c/d7b/dd2/fd7 x:0 0 0 2021-11-03T17:15:30.601 INFO:tasks.workunit.client.1.smithi115.stdout:1/824: dwrite d7/dc/d101/d43/fc9 [0,4194304] 0 2021-11-03T17:15:30.601 INFO:tasks.workunit.client.1.smithi115.stdout:1/825: write d7/dc/dd/d3e/ffc [696584,16457] 0 2021-11-03T17:15:30.602 INFO:tasks.workunit.client.1.smithi115.stdout:9/702: sync 2021-11-03T17:15:30.607 INFO:tasks.workunit.client.1.smithi115.stdout:5/961: creat d1a/d23/d138/f145 x:0 0 0 2021-11-03T17:15:30.611 INFO:tasks.workunit.client.1.smithi115.stdout:7/699: creat d3/da/d18/d9c/fd8 x:0 0 0 2021-11-03T17:15:30.616 INFO:tasks.workunit.client.1.smithi115.stdout:1/826: dwrite d7/fdd [0,4194304] 0 2021-11-03T17:15:30.621 INFO:tasks.workunit.client.1.smithi115.stdout:5/962: dread d1a/d28/f2c [0,4194304] 0 2021-11-03T17:15:30.624 INFO:tasks.workunit.client.1.smithi115.stdout:9/703: creat d1/d18/d4a/d74/d8b/ffe x:0 0 0 2021-11-03T17:15:30.634 INFO:tasks.workunit.client.1.smithi115.stdout:5/963: dwrite d1a/d28/d7b/d48/fca [0,4194304] 0 2021-11-03T17:15:30.634 INFO:tasks.workunit.client.1.smithi115.stdout:5/964: write d1a/d28/d7b/d48/dd4/f13e [528862,128461] 0 2021-11-03T17:15:30.635 INFO:tasks.workunit.client.1.smithi115.stdout:5/965: stat d1a/d3d/d58/dab/daf 0 2021-11-03T17:15:30.639 INFO:tasks.workunit.client.1.smithi115.stdout:9/704: mknod d1/d1a/d48/d27/d53/d83/d88/cff 1 2021-11-03T17:15:30.646 INFO:tasks.workunit.client.1.smithi115.stdout:7/700: unlink d3/da/d18/d73/d52/lcb 0 2021-11-03T17:15:30.649 INFO:tasks.workunit.client.1.smithi115.stdout:5/966: mknod d1a/d28/df9/c146 1 2021-11-03T17:15:30.654 INFO:tasks.workunit.client.1.smithi115.stdout:7/701: getdents d3 0 2021-11-03T17:15:30.662 INFO:tasks.workunit.client.1.smithi115.stdout:8/744: dwrite d4/d6/d18/f77 [0,4194304] 0 2021-11-03T17:15:30.664 INFO:tasks.workunit.client.1.smithi115.stdout:9/705: rename d1/d18/d2b/d42/d63/l69 to d1/d1a/d48/d27/d53/d83/d88/d9e/l100 0 2021-11-03T17:15:30.664 INFO:tasks.workunit.client.1.smithi115.stdout:7/702: creat d3/da/d18/d73/d35/dbb/fd9 x:0 0 0 2021-11-03T17:15:30.665 INFO:tasks.workunit.client.1.smithi115.stdout:7/703: dread - d3/da/d18/d73/d35/d53/f87 zero size 2021-11-03T17:15:30.665 INFO:tasks.workunit.client.1.smithi115.stdout:7/704: chown d3/fd5 108 1 2021-11-03T17:15:30.665 INFO:tasks.workunit.client.1.smithi115.stdout:1/827: write d7/dc/d101/d4b/d55/f6c [4347941,58038] 0 2021-11-03T17:15:30.670 INFO:tasks.workunit.client.1.smithi115.stdout:9/706: mknod d1/d66/c101 1 2021-11-03T17:15:30.670 INFO:tasks.workunit.client.1.smithi115.stdout:8/745: dread d4/d7d/fa0 [0,4194304] 0 2021-11-03T17:15:30.670 INFO:tasks.workunit.client.1.smithi115.stdout:8/746: mknod d4/d6/d18/d44/d47/deb/c106 1 2021-11-03T17:15:30.682 INFO:tasks.workunit.client.1.smithi115.stdout:7/705: dread d3/da/d18/d73/f89 [0,4194304] 0 2021-11-03T17:15:30.693 INFO:tasks.workunit.client.1.smithi115.stdout:8/747: dread d4/d6/d18/d30/d79/d7f/f89 [0,4194304] 0 2021-11-03T17:15:30.693 INFO:tasks.workunit.client.1.smithi115.stdout:8/748: readlink d4/d6/d18/d30/d33/ld7 0 2021-11-03T17:15:30.693 INFO:tasks.workunit.client.1.smithi115.stdout:7/706: truncate d3/da/d18/d73/fc2 686051 0 2021-11-03T17:15:30.693 INFO:tasks.workunit.client.1.smithi115.stdout:7/707: truncate d3/d4c/d7b/f91 4607643 0 2021-11-03T17:15:30.695 INFO:tasks.workunit.client.1.smithi115.stdout:8/749: getdents d4/d6/d18/d30/d79/de7 0 2021-11-03T17:15:30.697 INFO:tasks.workunit.client.1.smithi115.stdout:9/707: dwrite d1/d1a/d48/d27/d53/d83/d88/d9e/f3b [0,4194304] 0 2021-11-03T17:15:30.706 INFO:tasks.workunit.client.1.smithi115.stdout:5/967: rename d1a/d28/d7b/d4f/d81 to d1a/d3d/d58/dab/d147 0 2021-11-03T17:15:30.707 INFO:tasks.workunit.client.1.smithi115.stdout:5/968: chown d1a/d3d/d58 0 1 2021-11-03T17:15:30.707 INFO:tasks.workunit.client.1.smithi115.stdout:5/969: chown d1a/d28/df9/d122 8 1 2021-11-03T17:15:30.715 INFO:tasks.workunit.client.1.smithi115.stdout:8/750: mkdir d4/d6/d3f/d43/d107 0 2021-11-03T17:15:30.721 INFO:tasks.workunit.client.1.smithi115.stdout:7/708: rename d3/da/d18/d73/d33 to d3/da/d18/d73/dda 0 2021-11-03T17:15:30.722 INFO:tasks.workunit.client.1.smithi115.stdout:7/709: write d3/d4c/f4d [6576417,84804] 0 2021-11-03T17:15:30.722 INFO:tasks.workunit.client.1.smithi115.stdout:7/710: stat d3/da/d8e 0 2021-11-03T17:15:30.725 INFO:tasks.workunit.client.1.smithi115.stdout:8/751: rmdir d4/d6/d3f/d43/d107 0 2021-11-03T17:15:30.726 INFO:tasks.workunit.client.1.smithi115.stdout:1/828: rename d7/dc/d101/d43/f3c to d7/dc/d101/d43/df4/d60/dde/f104 0 2021-11-03T17:15:30.728 INFO:tasks.workunit.client.1.smithi115.stdout:9/708: symlink d1/d1a/d48/d27/d53/d83/d88/d9e/d58/l102 0 2021-11-03T17:15:30.733 INFO:tasks.workunit.client.1.smithi115.stdout:8/752: rmdir d4/d6/d18/d30/d79 39 2021-11-03T17:15:30.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:30 smithi115 conmon[34670]: cluster 2021-11-03T17:15:28.647169+0000 mgr.smithi027.ufhhvb (mgr.14622) 30 : cluster 2021-11-03T17:15:30.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:30 smithi115 conmon[34670]: [DBG] pgmap v18: 65 pgs: 65 active+clean; 2.5 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 35 MiB/s rd, 63 MiB/s wr, 518 op/s 2021-11-03T17:15:30.735 INFO:tasks.workunit.client.1.smithi115.stdout:1/829: readlink d7/dc/d101/d4b/d55/lcc 0 2021-11-03T17:15:30.740 INFO:tasks.workunit.client.1.smithi115.stdout:7/711: unlink d3/da/d18/d73/l81 0 2021-11-03T17:15:30.742 INFO:tasks.workunit.client.1.smithi115.stdout:8/753: creat d4/d6/d18/d30/d33/d93/f108 x:0 0 0 2021-11-03T17:15:30.743 INFO:tasks.workunit.client.1.smithi115.stdout:7/712: dread d3/da/d18/d73/f89 [0,4194304] 0 2021-11-03T17:15:30.743 INFO:tasks.workunit.client.1.smithi115.stdout:7/713: fsync d3/da/d78/f9f 0 2021-11-03T17:15:30.745 INFO:tasks.workunit.client.1.smithi115.stdout:1/830: write d7/dc/dd/d3e/d58/f64 [2264904,50839] 0 2021-11-03T17:15:30.746 INFO:tasks.workunit.client.1.smithi115.stdout:5/970: rename d1a/d28/d7b/l41 to d1a/d23/db4/d124/l148 0 2021-11-03T17:15:30.747 INFO:tasks.workunit.client.1.smithi115.stdout:9/709: dwrite d1/d1a/d48/d27/d80/dcc/ff8 [0,4194304] 0 2021-11-03T17:15:30.747 INFO:tasks.workunit.client.1.smithi115.stdout:9/710: chown d1/d1a/d48/d27/d53/fe2 3419980 1 2021-11-03T17:15:30.749 INFO:tasks.workunit.client.1.smithi115.stdout:8/754: mkdir d4/d7d/dc2/d109 0 2021-11-03T17:15:30.750 INFO:tasks.workunit.client.1.smithi115.stdout:7/714: creat d3/d4c/da6/fdb x:0 0 0 2021-11-03T17:15:30.750 INFO:tasks.workunit.client.1.smithi115.stdout:7/715: dread - d3/da/d18/d9c/fd8 zero size 2021-11-03T17:15:30.751 INFO:tasks.workunit.client.1.smithi115.stdout:7/716: dread d3/da/d18/d73/fc2 [0,4194304] 0 2021-11-03T17:15:30.751 INFO:tasks.workunit.client.1.smithi115.stdout:1/831: creat d7/dc/dab/d73/d7f/f105 x:0 0 0 2021-11-03T17:15:30.758 INFO:tasks.workunit.client.1.smithi115.stdout:5/971: getdents d1a/d28/d7b/d48 0 2021-11-03T17:15:30.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:30 smithi027 conmon[32322]: cluster 2021-11-03T17:15:28.647169 2021-11-03T17:15:30.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:30 smithi027 conmon[32322]: +0000 mgr.smithi027.ufhhvb (mgr.14622) 30 : cluster [DBG] pgmap v18: 65 pgs: 65 active+clean; 2.5 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 35 MiB/s rd, 63 MiB/s wr, 518 op/s 2021-11-03T17:15:30.778 INFO:tasks.workunit.client.1.smithi115.stdout:9/711: truncate d1/d1a/d48/d27/f4c 664231 0 2021-11-03T17:15:30.779 INFO:tasks.workunit.client.1.smithi115.stdout:9/712: fdatasync d1/d66/fd1 0 2021-11-03T17:15:30.782 INFO:tasks.workunit.client.1.smithi115.stdout:7/717: rename d3/d4c/d7b/laa to d3/da/d18/d73/d52/db4/dd1/ldc 0 2021-11-03T17:15:30.786 INFO:tasks.workunit.client.1.smithi115.stdout:1/832: chown d7/dc/dd/db4/fe0 0 1 2021-11-03T17:15:30.786 INFO:tasks.workunit.client.1.smithi115.stdout:1/833: stat d7/d38 0 2021-11-03T17:15:30.788 INFO:tasks.workunit.client.1.smithi115.stdout:5/972: truncate d1a/d28/d4c/d75/dae/d114/fd5 343507 0 2021-11-03T17:15:30.792 INFO:tasks.workunit.client.1.smithi115.stdout:9/713: symlink d1/d18/d2b/d42/d6e/db5/dfa/l103 0 2021-11-03T17:15:30.794 INFO:tasks.workunit.client.1.smithi115.stdout:7/718: creat d3/d4c/d7b/dd2/fdd x:0 0 0 2021-11-03T17:15:30.805 INFO:tasks.workunit.client.1.smithi115.stdout:9/714: creat d1/d1a/d48/d27/d80/f104 x:0 0 0 2021-11-03T17:15:30.805 INFO:tasks.workunit.client.1.smithi115.stdout:9/715: readlink d1/d1a/d48/d27/d2f/l5d 0 2021-11-03T17:15:30.806 INFO:tasks.workunit.client.1.smithi115.stdout:5/973: dwrite d1a/d28/df9/fc1 [0,4194304] 0 2021-11-03T17:15:30.807 INFO:tasks.workunit.client.1.smithi115.stdout:7/719: dread d3/da/f62 [0,4194304] 0 2021-11-03T17:15:30.807 INFO:tasks.workunit.client.1.smithi115.stdout:7/720: readlink d3/da/l1d 0 2021-11-03T17:15:30.808 INFO:tasks.workunit.client.1.smithi115.stdout:8/755: link d4/d6/d18/d30/f3a d4/f10a 0 2021-11-03T17:15:30.808 INFO:tasks.workunit.client.1.smithi115.stdout:5/974: dread d1a/d28/fec [0,4194304] 0 2021-11-03T17:15:30.808 INFO:tasks.workunit.client.1.smithi115.stdout:8/756: write d4/d6/d18/d30/f31 [1086023,12302] 0 2021-11-03T17:15:30.809 INFO:tasks.workunit.client.1.smithi115.stdout:9/716: stat d1/d18/d2b/d42/d63/l86 0 2021-11-03T17:15:30.826 INFO:tasks.workunit.client.1.smithi115.stdout:7/721: mkdir d3/da/d18/d73/d35/d53/dde 0 2021-11-03T17:15:30.830 INFO:tasks.workunit.client.1.smithi115.stdout:5/975: mkdir d1a/d23/d138/d149 0 2021-11-03T17:15:30.838 INFO:tasks.workunit.client.1.smithi115.stdout:8/757: mkdir d4/d6/d3f/d9a/d10b 0 2021-11-03T17:15:30.844 INFO:tasks.workunit.client.1.smithi115.stdout:7/722: dwrite d3/da/d18/d73/dda/f5e [0,4194304] 0 2021-11-03T17:15:30.845 INFO:tasks.workunit.client.1.smithi115.stdout:9/717: link d1/d1a/d48/d27/d80/dcc/fd8 d1/d18/d2b/d42/f105 0 2021-11-03T17:15:30.848 INFO:tasks.workunit.client.1.smithi115.stdout:5/976: rmdir d1a/d28/d7b 39 2021-11-03T17:15:30.855 INFO:tasks.workunit.client.1.smithi115.stdout:5/977: symlink d1a/d28/l14a 0 2021-11-03T17:15:30.859 INFO:tasks.workunit.client.1.smithi115.stdout:7/723: dread d3/da/d18/d73/d35/d53/f58 [0,4194304] 0 2021-11-03T17:15:30.862 INFO:tasks.workunit.client.1.smithi115.stdout:9/718: dread d1/f75 [0,4194304] 0 2021-11-03T17:15:30.862 INFO:tasks.workunit.client.1.smithi115.stdout:9/719: dread - d1/d18/d2b/db9/fdb zero size 2021-11-03T17:15:30.871 INFO:tasks.workunit.client.1.smithi115.stdout:7/724: write d3/da/d18/d73/d35/d53/f58 [1939154,105438] 0 2021-11-03T17:15:30.872 INFO:tasks.workunit.client.1.smithi115.stdout:7/725: dread d3/da/d18/fce [0,4194304] 0 2021-11-03T17:15:30.873 INFO:tasks.workunit.client.1.smithi115.stdout:9/720: link d1/d1a/d48/d27/d53/l97 d1/d18/d4a/d74/d8b/dbc/l106 0 2021-11-03T17:15:30.879 INFO:tasks.workunit.client.1.smithi115.stdout:5/978: dwrite d1a/d3d/dc4/fe9 [0,4194304] 0 2021-11-03T17:15:30.885 INFO:tasks.workunit.client.1.smithi115.stdout:5/979: write d1a/d3d/dc4/f118 [2016492,35239] 0 2021-11-03T17:15:30.886 INFO:tasks.workunit.client.1.smithi115.stdout:7/726: dread d3/f47 [0,4194304] 0 2021-11-03T17:15:30.888 INFO:tasks.workunit.client.1.smithi115.stdout:9/721: dwrite d1/d18/d65/faf [0,4194304] 0 2021-11-03T17:15:30.896 INFO:tasks.workunit.client.1.smithi115.stdout:7/727: chown d3/da/d18/d73/l8b 257471347 1 2021-11-03T17:15:30.897 INFO:tasks.workunit.client.1.smithi115.stdout:9/722: creat d1/d1a/d48/d27/d53/d83/d88/d9e/d58/f107 x:0 0 0 2021-11-03T17:15:30.898 INFO:tasks.workunit.client.1.smithi115.stdout:7/728: symlink d3/da/d18/d73/d52/db4/dd1/ldf 0 2021-11-03T17:15:30.900 INFO:tasks.workunit.client.1.smithi115.stdout:9/723: rmdir d1/d1a/d48/d27/d53/d83/d88/dcd/dda/dee 0 2021-11-03T17:15:30.901 INFO:tasks.workunit.client.1.smithi115.stdout:7/729: truncate d3/da/d78/f9f 2242263 0 2021-11-03T17:15:30.902 INFO:tasks.workunit.client.1.smithi115.stdout:9/724: creat d1/d66/f108 x:0 0 0 2021-11-03T17:15:30.904 INFO:tasks.workunit.client.1.smithi115.stdout:7/730: creat d3/d4c/d7b/dcf/fe0 x:0 0 0 2021-11-03T17:15:30.905 INFO:tasks.workunit.client.1.smithi115.stdout:9/725: creat d1/d1a/dd9/f109 x:0 0 0 2021-11-03T17:15:30.905 INFO:tasks.workunit.client.1.smithi115.stdout:9/726: chown d1/d18/d65 13117 1 2021-11-03T17:15:30.906 INFO:tasks.workunit.client.1.smithi115.stdout:7/731: creat d3/da/d18/d73/d35/dbb/fe1 x:0 0 0 2021-11-03T17:15:30.907 INFO:tasks.workunit.client.1.smithi115.stdout:7/732: write d3/f98 [766622,19487] 0 2021-11-03T17:15:30.907 INFO:tasks.workunit.client.1.smithi115.stdout:7/733: fsync d3/da/d18/fce 0 2021-11-03T17:15:30.907 INFO:tasks.workunit.client.1.smithi115.stdout:9/727: creat d1/d1a/d48/d27/d53/d83/f10a x:0 0 0 2021-11-03T17:15:30.909 INFO:tasks.workunit.client.1.smithi115.stdout:7/734: creat d3/da/d8e/fe2 x:0 0 0 2021-11-03T17:15:30.909 INFO:tasks.workunit.client.1.smithi115.stdout:9/728: getdents d1/d18/d2b 0 2021-11-03T17:15:30.910 INFO:tasks.workunit.client.1.smithi115.stdout:9/729: chown d1/d18/d2b/d42/ff 2 1 2021-11-03T17:15:30.911 INFO:tasks.workunit.client.1.smithi115.stdout:7/735: dread d3/da/d18/d73/d52/fa9 [0,4194304] 0 2021-11-03T17:15:30.913 INFO:tasks.workunit.client.1.smithi115.stdout:9/730: rename d1/d1a/d48/d27/d53/d83/d88/d9e/l100 to d1/l10b 0 2021-11-03T17:15:30.914 INFO:tasks.workunit.client.1.smithi115.stdout:9/731: truncate d1/d18/d2b/d42/d6e/db5/f93 522384 0 2021-11-03T17:15:30.930 INFO:tasks.workunit.client.1.smithi115.stdout:7/736: read d3/da/d18/faf [255045,35104] 0 2021-11-03T17:15:30.930 INFO:tasks.workunit.client.1.smithi115.stdout:7/737: write d3/da/d18/d73/d52/f56 [4460981,37778] 0 2021-11-03T17:15:30.930 INFO:tasks.workunit.client.1.smithi115.stdout:7/738: dread - d3/d4c/d7b/dd2/fd7 zero size 2021-11-03T17:15:30.936 INFO:tasks.workunit.client.1.smithi115.stdout:9/732: dread d1/d1a/d48/d27/d2f/d41/f9d [0,4194304] 0 2021-11-03T17:15:30.940 INFO:tasks.workunit.client.1.smithi115.stdout:7/739: dread d3/da/d18/d73/dda/f74 [0,4194304] 0 2021-11-03T17:15:30.941 INFO:tasks.workunit.client.1.smithi115.stdout:7/740: stat d3/d4c/f5b 0 2021-11-03T17:15:30.941 INFO:tasks.workunit.client.1.smithi115.stdout:9/733: symlink d1/d1a/d48/d27/d53/d83/d88/d9e/l10c 0 2021-11-03T17:15:30.947 INFO:tasks.workunit.client.1.smithi115.stdout:7/741: getdents d3/da/d18/d73 0 2021-11-03T17:15:30.947 INFO:tasks.workunit.client.1.smithi115.stdout:7/742: mknod d3/da/d18/d73/d52/db4/dd1/ce3 1 2021-11-03T17:15:30.949 INFO:tasks.workunit.client.1.smithi115.stdout:9/734: truncate d1/d18/d2b/d42/fb7 759710 0 2021-11-03T17:15:30.949 INFO:tasks.workunit.client.1.smithi115.stdout:9/735: stat d1/d18/d2b/d42/d63/f9f 0 2021-11-03T17:15:30.953 INFO:tasks.workunit.client.1.smithi115.stdout:7/743: rename d3/da/d18/d73/d35/f57 to d3/da/d18/d73/fe4 0 2021-11-03T17:15:30.954 INFO:tasks.workunit.client.1.smithi115.stdout:7/744: mknod d3/d4c/da6/dc7/ce5 1 2021-11-03T17:15:30.986 INFO:tasks.workunit.client.1.smithi115.stdout:9/736: dwrite d1/d18/d2b/d42/d6e/db5/f93 [0,4194304] 0 2021-11-03T17:15:30.992 INFO:tasks.workunit.client.1.smithi115.stdout:9/737: dread d1/f75 [0,4194304] 0 2021-11-03T17:15:30.994 INFO:tasks.workunit.client.1.smithi115.stdout:9/738: write d1/d1a/d48/d27/d53/d83/d88/d9e/d58/fa4 [7698214,82193] 0 2021-11-03T17:15:30.995 INFO:tasks.workunit.client.1.smithi115.stdout:9/739: fsync d1/d1a/d48/d27/ff7 0 2021-11-03T17:15:30.999 INFO:tasks.workunit.client.1.smithi115.stdout:9/740: rename d1/d1a/d48/d27/d2f/d41/f8c to d1/d1a/d48/d27/f10d 0 2021-11-03T17:15:31.003 INFO:tasks.workunit.client.1.smithi115.stdout:9/741: truncate d1/d1a/d48/d27/f47 2053224 0 2021-11-03T17:15:31.028 INFO:tasks.workunit.client.1.smithi115.stdout:9/742: dwrite d1/d66/fc0 [0,4194304] 0 2021-11-03T17:15:31.029 INFO:tasks.workunit.client.1.smithi115.stdout:9/743: creat d1/d1a/dd9/f10e x:0 0 0 2021-11-03T17:15:31.030 INFO:tasks.workunit.client.1.smithi115.stdout:9/744: write d1/d1a/d48/d27/d53/f95 [1499268,87587] 0 2021-11-03T17:15:31.030 INFO:tasks.workunit.client.1.smithi115.stdout:9/745: truncate d1/d1a/d48/d27/d53/fe2 760182 0 2021-11-03T17:15:31.030 INFO:tasks.workunit.client.1.smithi115.stdout:9/746: dread - d1/d1a/dd9/f109 zero size 2021-11-03T17:15:31.034 INFO:tasks.workunit.client.1.smithi115.stdout:9/747: rename d1/d18/d2b/d42/d6e/fa6 to d1/d18/d2b/f10f 0 2021-11-03T17:15:31.036 INFO:tasks.workunit.client.1.smithi115.stdout:9/748: creat d1/d1a/d48/d27/d2f/d6a/f110 x:0 0 0 2021-11-03T17:15:31.037 INFO:tasks.workunit.client.1.smithi115.stdout:9/749: fsync d1/d1a/d48/d27/d2f/d41/fe5 0 2021-11-03T17:15:31.039 INFO:tasks.workunit.client.1.smithi115.stdout:9/750: creat d1/d18/d2b/de8/f111 x:0 0 0 2021-11-03T17:15:31.044 INFO:tasks.workunit.client.1.smithi115.stdout:9/751: rename d1/d18/d2b/de8/le9 to d1/d1a/d48/d27/d53/d83/d88/dcd/l112 0 2021-11-03T17:15:31.046 INFO:tasks.workunit.client.1.smithi115.stdout:9/752: creat d1/f113 x:0 0 0 2021-11-03T17:15:31.048 INFO:tasks.workunit.client.1.smithi115.stdout:9/753: creat d1/d18/d2b/d45/f114 x:0 0 0 2021-11-03T17:15:31.050 INFO:tasks.workunit.client.1.smithi115.stdout:9/754: rmdir d1/d1a/d48/d27/d2f/d7b 39 2021-11-03T17:15:31.051 INFO:tasks.workunit.client.1.smithi115.stdout:9/755: stat d1/d18/d2b/d45/dc2/fe7 0 2021-11-03T17:15:31.054 INFO:tasks.workunit.client.1.smithi115.stdout:9/756: rename d1/d18/d2b/d42/f105 to d1/d18/d2b/db9/df2/f115 0 2021-11-03T17:15:31.054 INFO:tasks.workunit.client.1.smithi115.stdout:9/757: chown d1/f72 1320 1 2021-11-03T17:15:31.056 INFO:tasks.workunit.client.1.smithi115.stdout:9/758: creat d1/d18/d2b/d42/f116 x:0 0 0 2021-11-03T17:15:31.060 INFO:tasks.workunit.client.1.smithi115.stdout:9/759: link d1/d18/d4a/d74/d8b/ffe d1/d1a/d48/d27/d80/dcc/f117 0 2021-11-03T17:15:31.062 INFO:tasks.workunit.client.1.smithi115.stdout:9/760: creat d1/d8a/f118 x:0 0 0 2021-11-03T17:15:31.066 INFO:tasks.workunit.client.1.smithi115.stdout:9/761: mkdir d1/d1a/d48/d27/d2f/d119 0 2021-11-03T17:15:31.089 INFO:tasks.workunit.client.1.smithi115.stdout:9/762: write d1/d18/d2b/d42/fb7 [458526,36327] 0 2021-11-03T17:15:31.093 INFO:tasks.workunit.client.1.smithi115.stdout:3/990: sync 2021-11-03T17:15:31.107 INFO:tasks.workunit.client.1.smithi115.stdout:3/991: write d13/d2a/d60/f77 [2034563,18736] 0 2021-11-03T17:15:31.111 INFO:tasks.workunit.client.1.smithi115.stdout:3/992: rename d13/d2a/d60/f117 to d13/d3b/db1/dbc/f150 0 2021-11-03T17:15:31.402 INFO:tasks.workunit.client.1.smithi115.stdout:6/674: sync 2021-11-03T17:15:31.403 INFO:tasks.workunit.client.1.smithi115.stdout:6/675: truncate d0/d1/d2e/d4c/d96/f6b 686943 0 2021-11-03T17:15:31.407 INFO:tasks.workunit.client.1.smithi115.stdout:6/676: creat d0/d1/d10/d13/d1c/fe2 x:0 0 0 2021-11-03T17:15:31.407 INFO:tasks.workunit.client.1.smithi115.stdout:6/677: chown d0/d7f 128703 1 2021-11-03T17:15:31.410 INFO:tasks.workunit.client.1.smithi115.stdout:6/678: truncate d0/d1/d2e/f8e 3280142 0 2021-11-03T17:15:31.411 INFO:tasks.workunit.client.1.smithi115.stdout:6/679: chown d0/d1/d10/d13/d87/d89 352 1 2021-11-03T17:15:31.434 INFO:tasks.workunit.client.1.smithi115.stdout:6/680: write d0/d1/d2e/d4c/f4e [1259143,90317] 0 2021-11-03T17:15:31.435 INFO:tasks.workunit.client.1.smithi115.stdout:6/681: chown d0/f22 0 1 2021-11-03T17:15:31.447 INFO:tasks.workunit.client.1.smithi115.stdout:6/682: write d0/d1/d10/d13/d59/f67 [1930068,118589] 0 2021-11-03T17:15:31.448 INFO:tasks.workunit.client.1.smithi115.stdout:6/683: getdents d0/d1/d10/d13 0 2021-11-03T17:15:31.450 INFO:tasks.workunit.client.1.smithi115.stdout:6/684: creat d0/d1/d10/d13/d59/fe3 x:0 0 0 2021-11-03T17:15:31.455 INFO:tasks.workunit.client.1.smithi115.stdout:6/685: mkdir d0/de4 0 2021-11-03T17:15:31.457 INFO:tasks.workunit.client.1.smithi115.stdout:6/686: creat d0/fe5 x:0 0 0 2021-11-03T17:15:31.459 INFO:tasks.workunit.client.1.smithi115.stdout:6/687: dread d0/d1/d10/d13/d1c/d1e/f91 [0,4194304] 0 2021-11-03T17:15:31.460 INFO:tasks.workunit.client.1.smithi115.stdout:6/688: dread d0/d1/d10/d13/d1c/f2f [4194304,4194304] 0 2021-11-03T17:15:31.464 INFO:tasks.workunit.client.1.smithi115.stdout:6/689: mkdir d0/d1/d10/d13/d1c/d1e/de6 0 2021-11-03T17:15:31.464 INFO:tasks.workunit.client.1.smithi115.stdout:6/690: stat d0/d1/d6a/fa1 0 2021-11-03T17:15:31.466 INFO:tasks.workunit.client.1.smithi115.stdout:6/691: creat d0/d1/fe7 x:0 0 0 2021-11-03T17:15:31.475 INFO:tasks.workunit.client.1.smithi115.stdout:6/692: dread d0/f6 [8388608,4194304] 0 2021-11-03T17:15:31.478 INFO:tasks.workunit.client.1.smithi115.stdout:6/693: creat d0/d1/d10/d13/d1c/d1e/de6/fe8 x:0 0 0 2021-11-03T17:15:31.479 INFO:tasks.workunit.client.1.smithi115.stdout:6/694: write d0/d1/d10/d13/d87/fa5 [3023643,74197] 0 2021-11-03T17:15:31.482 INFO:tasks.workunit.client.1.smithi115.stdout:6/695: creat d0/d1/d6a/d88/da0/db5/fe9 x:0 0 0 2021-11-03T17:15:31.489 INFO:tasks.workunit.client.1.smithi115.stdout:6/696: dread d0/d1/d10/f99 [0,4194304] 0 2021-11-03T17:15:31.491 INFO:tasks.workunit.client.1.smithi115.stdout:6/697: unlink d0/d1/d2e/d4c/d96/d61/ld1 0 2021-11-03T17:15:31.506 INFO:tasks.workunit.client.1.smithi115.stdout:6/698: write d0/d1/d2/fd [253021,122792] 0 2021-11-03T17:15:31.506 INFO:tasks.workunit.client.1.smithi115.stdout:6/699: dread - d0/d1/d2e/d4c/fc7 zero size 2021-11-03T17:15:31.508 INFO:tasks.workunit.client.1.smithi115.stdout:6/700: getdents d0 0 2021-11-03T17:15:31.513 INFO:tasks.workunit.client.1.smithi115.stdout:6/701: creat d0/d74/dae/fea x:0 0 0 2021-11-03T17:15:31.532 INFO:tasks.workunit.client.1.smithi115.stdout:6/702: dwrite d0/f22 [4194304,4194304] 0 2021-11-03T17:15:31.533 INFO:tasks.workunit.client.1.smithi115.stdout:6/703: write d0/d1/d10/d13/d59/fe3 [569427,104458] 0 2021-11-03T17:15:31.539 INFO:tasks.workunit.client.1.smithi115.stdout:6/704: rename d0/d1/d2e to d0/d1/d2/d3f/deb 0 2021-11-03T17:15:31.540 INFO:tasks.workunit.client.1.smithi115.stdout:6/705: write d0/d1/d10/d13/d1c/fe2 [574443,41909] 0 2021-11-03T17:15:31.540 INFO:tasks.workunit.client.1.smithi115.stdout:6/706: chown d0/d1/d10/d13/d59 17797 1 2021-11-03T17:15:31.547 INFO:tasks.workunit.client.1.smithi115.stdout:6/707: symlink d0/d1/d2/d3f/deb/d4c/d96/d61/lec 0 2021-11-03T17:15:31.548 INFO:tasks.workunit.client.1.smithi115.stdout:6/708: write d0/fc2 [2395511,19371] 0 2021-11-03T17:15:31.566 INFO:tasks.workunit.client.1.smithi115.stdout:6/709: dwrite d0/d1/d2/d3f/deb/f83 [4194304,4194304] 0 2021-11-03T17:15:31.568 INFO:tasks.workunit.client.1.smithi115.stdout:6/710: symlink d0/led 0 2021-11-03T17:15:31.571 INFO:tasks.workunit.client.1.smithi115.stdout:6/711: getdents d0/d1/d6a/d88/da0/db5 0 2021-11-03T17:15:31.575 INFO:tasks.workunit.client.1.smithi115.stdout:6/712: rename d0/d1/d10/d13/d59/d95/lc9 to d0/d1/d10/d13/d1c/d1e/de6/lee 0 2021-11-03T17:15:31.575 INFO:tasks.workunit.client.1.smithi115.stdout:6/713: chown d0/d74/dae/db3 6934166 1 2021-11-03T17:15:31.575 INFO:tasks.workunit.client.1.smithi115.stdout:6/714: truncate d0/d1/d6a/d88/da0/db5/fe9 1038762 0 2021-11-03T17:15:31.576 INFO:tasks.workunit.client.1.smithi115.stdout:6/715: stat d0/d1/d10/d13/d59 0 2021-11-03T17:15:31.582 INFO:tasks.workunit.client.1.smithi115.stdout:6/716: creat d0/d1/d10/d13/d87/dd2/dd7/fef x:0 0 0 2021-11-03T17:15:31.584 INFO:tasks.workunit.client.1.smithi115.stdout:6/717: symlink d0/d74/dae/lf0 0 2021-11-03T17:15:31.601 INFO:tasks.workunit.client.1.smithi115.stdout:6/718: dwrite d0/d1/fe7 [0,4194304] 0 2021-11-03T17:15:31.612 INFO:tasks.workunit.client.1.smithi115.stdout:6/719: chown d0/d1/d2/d3f/lbc 23243 1 2021-11-03T17:15:31.613 INFO:tasks.workunit.client.1.smithi115.stdout:6/720: write d0/d1/d2/f58 [1507013,63758] 0 2021-11-03T17:15:31.631 INFO:tasks.workunit.client.1.smithi115.stdout:6/721: creat d0/de4/ff1 x:0 0 0 2021-11-03T17:15:31.654 INFO:tasks.workunit.client.1.smithi115.stdout:6/722: dwrite d0/d1/fe7 [0,4194304] 0 2021-11-03T17:15:31.820 INFO:teuthology.orchestra.run:Running command with timeout 3600 2021-11-03T17:15:31.821 DEBUG:teuthology.orchestra.run.smithi027:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2021-11-03T17:15:31.922 INFO:tasks.workunit.client.1.smithi115.stdout:2/789: sync 2021-11-03T17:15:31.922 INFO:tasks.workunit.client.1.smithi115.stdout:0/948: sync 2021-11-03T17:15:31.922 INFO:tasks.workunit.client.1.smithi115.stdout:0/949: fsync d9/d15/d78/fd0 0 2021-11-03T17:15:31.928 INFO:tasks.workunit.client.1.smithi115.stdout:0/950: creat d9/d15/d78/d80/dc4/f139 x:0 0 0 2021-11-03T17:15:31.936 INFO:tasks.workunit.client.1.smithi115.stdout:0/951: creat d9/d37/f13a x:0 0 0 2021-11-03T17:15:31.958 INFO:tasks.workunit.client.1.smithi115.stdout:0/952: dwrite d9/db/f42 [0,4194304] 0 2021-11-03T17:15:31.963 INFO:tasks.workunit.client.1.smithi115.stdout:0/953: creat d9/d15/d1a/d1d/dcc/d46/d8c/dfc/f13b x:0 0 0 2021-11-03T17:15:31.963 INFO:tasks.workunit.client.1.smithi115.stdout:0/954: fdatasync d9/d15/d78/d80/d82/f91 0 2021-11-03T17:15:31.963 INFO:tasks.workunit.client.1.smithi115.stdout:0/955: fsync d9/d15/d78/d80/f9a 0 2021-11-03T17:15:31.970 INFO:tasks.workunit.client.1.smithi115.stdout:0/956: link d9/d15/d1a/d1d/dcc/d46/d11d/l107 d9/d15/d78/d80/l13c 0 2021-11-03T17:15:31.978 INFO:tasks.workunit.client.1.smithi115.stdout:2/790: dwrite d4/d6/d56/d6c/d34/d43/d63/d69/fbf [4194304,4194304] 0 2021-11-03T17:15:31.979 INFO:tasks.workunit.client.1.smithi115.stdout:2/791: read d4/f62 [2994341,29477] 0 2021-11-03T17:15:31.980 INFO:tasks.workunit.client.1.smithi115.stdout:2/792: fdatasync d4/d6/dad/d7f/fa5 0 2021-11-03T17:15:31.984 INFO:tasks.workunit.client.1.smithi115.stdout:0/957: write d9/d15/d78/d80/fe3 [2130325,125597] 0 2021-11-03T17:15:31.985 INFO:tasks.workunit.client.1.smithi115.stdout:0/958: write d9/d15/d1a/d1d/dcc/d46/d63/f7b [5068347,36588] 0 2021-11-03T17:15:31.988 INFO:tasks.workunit.client.1.smithi115.stdout:2/793: rename d4/d6/d56/d6c/d34/d43/d28/l30 to d4/d6/dad/d7f/l110 0 2021-11-03T17:15:31.989 INFO:tasks.workunit.client.1.smithi115.stdout:0/959: symlink d9/d37/l13d 0 2021-11-03T17:15:31.990 INFO:tasks.workunit.client.1.smithi115.stdout:0/960: write d9/db/f42 [1959768,91006] 0 2021-11-03T17:15:31.995 INFO:tasks.workunit.client.1.smithi115.stdout:4/970: sync 2021-11-03T17:15:31.996 INFO:tasks.workunit.client.1.smithi115.stdout:2/794: truncate d4/d6/d13/f21 878030 0 2021-11-03T17:15:32.007 INFO:tasks.workunit.client.1.smithi115.stdout:4/971: stat d7/dc/d1a/d21/db1/d56/d63/d9e/lf8 0 2021-11-03T17:15:32.007 INFO:tasks.workunit.client.1.smithi115.stdout:4/972: chown d7/d7a/l8f 7 1 2021-11-03T17:15:32.008 INFO:tasks.workunit.client.1.smithi115.stdout:0/961: truncate d9/d15/d1a/d1d/dcc/d6e/f9f 4078554 0 2021-11-03T17:15:32.008 INFO:tasks.workunit.client.1.smithi115.stdout:0/962: chown d9/d15/d1a/d1d/f4d 97402899 1 2021-11-03T17:15:32.014 INFO:tasks.workunit.client.1.smithi115.stdout:4/973: creat d7/dc/d6a/f147 x:0 0 0 2021-11-03T17:15:32.020 INFO:tasks.workunit.client.1.smithi115.stdout:2/795: symlink d4/d6/dad/d7f/l111 0 2021-11-03T17:15:32.020 INFO:tasks.workunit.client.1.smithi115.stdout:2/796: chown d4/d6/d13/ffa 3 1 2021-11-03T17:15:32.023 INFO:tasks.workunit.client.1.smithi115.stdout:4/974: mknod d7/dc/db6/c148 1 2021-11-03T17:15:32.027 INFO:tasks.workunit.client.1.smithi115.stdout:2/797: mkdir d4/d6/d13/d108/d112 0 2021-11-03T17:15:32.029 INFO:tasks.workunit.client.1.smithi115.stdout:0/963: rename d9/d15/d1a/d1d/dcc/d46/d8c/f79 to d9/d15/d1a/d1d/dcc/d46/d63/dae/f13e 0 2021-11-03T17:15:32.030 INFO:tasks.workunit.client.1.smithi115.stdout:0/964: dread - d9/d15/d1a/d1d/dcc/d46/d63/dc5/f100 zero size 2021-11-03T17:15:32.036 INFO:tasks.workunit.client.1.smithi115.stdout:2/798: rename d4/d6/d56/d6c/d34/d42/d8d/fce to d4/d6/dad/d7f/de9/f113 0 2021-11-03T17:15:32.038 INFO:tasks.workunit.client.1.smithi115.stdout:0/965: getdents d9/d37 0 2021-11-03T17:15:32.040 INFO:tasks.workunit.client.1.smithi115.stdout:4/975: mkdir d7/dc/d6a/d3c/d112/dc4/dc6/d149 0 2021-11-03T17:15:32.050 INFO:tasks.workunit.client.1.smithi115.stdout:2/799: rmdir d4/d6/d56/d6c/d34/d42/d8d/dd8 39 2021-11-03T17:15:32.051 INFO:tasks.workunit.client.1.smithi115.stdout:4/976: getdents d7/dd1/dee 0 2021-11-03T17:15:32.055 INFO:tasks.workunit.client.1.smithi115.stdout:4/977: rename d7/d7a/d83/f8c to d7/dc/d6a/d3c/f14a 0 2021-11-03T17:15:32.057 INFO:tasks.workunit.client.1.smithi115.stdout:4/978: mkdir d7/dd1/d14b 0 2021-11-03T17:15:32.058 INFO:tasks.workunit.client.1.smithi115.stdout:2/800: dread d4/d6/d13/d75/f7b [0,4194304] 0 2021-11-03T17:15:32.060 INFO:tasks.workunit.client.1.smithi115.stdout:4/979: creat d7/dc/d1a/d21/db1/d56/df4/d135/f14c x:0 0 0 2021-11-03T17:15:32.062 INFO:tasks.workunit.client.1.smithi115.stdout:2/801: read d4/d6/dad/d7f/feb [1813291,99514] 0 2021-11-03T17:15:32.065 INFO:tasks.workunit.client.1.smithi115.stdout:4/980: mkdir d7/dc/db6/de1/d14d 0 2021-11-03T17:15:32.065 INFO:tasks.workunit.client.1.smithi115.stdout:4/981: mknod d7/dc/d6a/d3c/d112/d130/c14e 1 2021-11-03T17:15:32.066 INFO:tasks.workunit.client.1.smithi115.stdout:0/966: dwrite d9/d15/d1a/d1d/dcc/d85/ff8 [0,4194304] 0 2021-11-03T17:15:32.066 INFO:tasks.workunit.client.1.smithi115.stdout:0/967: truncate d9/f110 1181339 0 2021-11-03T17:15:32.070 INFO:tasks.workunit.client.1.smithi115.stdout:0/968: getdents d9/d15/d78/d80/dc4 0 2021-11-03T17:15:32.081 INFO:tasks.workunit.client.1.smithi115.stdout:0/969: dread d9/d15/d1a/d1d/dcc/d46/d63/dae/fd7 [0,4194304] 0 2021-11-03T17:15:32.084 INFO:tasks.workunit.client.1.smithi115.stdout:0/970: dread d9/d15/d1a/f68 [0,4194304] 0 2021-11-03T17:15:32.085 INFO:tasks.workunit.client.1.smithi115.stdout:0/971: write d9/d15/d1a/d1d/dcc/d46/d63/d70/fd2 [1243791,3459] 0 2021-11-03T17:15:32.085 INFO:tasks.workunit.client.1.smithi115.stdout:0/972: chown d9/db/f47 441 1 2021-11-03T17:15:32.085 INFO:tasks.workunit.client.1.smithi115.stdout:0/973: chown d9/d15/d78/d80/f9a 1960 1 2021-11-03T17:15:32.085 INFO:tasks.workunit.client.1.smithi115.stdout:4/982: dwrite d7/dc/d1a/d21/f141 [0,4194304] 0 2021-11-03T17:15:32.086 INFO:tasks.workunit.client.1.smithi115.stdout:4/983: fsync d7/dc/d6a/d101/f129 0 2021-11-03T17:15:32.086 INFO:tasks.workunit.client.1.smithi115.stdout:4/984: mknod d7/dc/d6a/d3c/d10e/c14f 1 2021-11-03T17:15:32.090 INFO:tasks.workunit.client.1.smithi115.stdout:0/974: read d9/d15/d78/d80/d82/fe6 [3580398,99316] 0 2021-11-03T17:15:32.091 INFO:tasks.workunit.client.1.smithi115.stdout:0/975: write d9/d15/d1a/d1d/dcc/d46/d63/d70/fd2 [690065,1483] 0 2021-11-03T17:15:32.099 INFO:tasks.workunit.client.1.smithi115.stdout:4/985: mknod d7/dc/d1a/d21/db1/d56/d63/d7b/d96/c150 1 2021-11-03T17:15:32.099 INFO:tasks.workunit.client.1.smithi115.stdout:4/986: write d7/dc/d1a/d21/db1/d56/d63/f120 [877027,66064] 0 2021-11-03T17:15:32.105 INFO:tasks.workunit.client.1.smithi115.stdout:0/976: rename d9/d15/d78/d80 to d9/d15/db6/d10b/d124/d12d/d13f 0 2021-11-03T17:15:32.119 INFO:tasks.workunit.client.1.smithi115.stdout:4/987: dwrite d7/dc/d1a/d21/db1/f74 [4194304,4194304] 0 2021-11-03T17:15:32.120 INFO:tasks.workunit.client.1.smithi115.stdout:4/988: dread d7/dc/d6a/f12e [0,4194304] 0 2021-11-03T17:15:32.121 INFO:tasks.workunit.client.1.smithi115.stdout:4/989: write d7/dc/d6a/d3c/f3e [232820,54806] 0 2021-11-03T17:15:32.126 INFO:tasks.workunit.client.1.smithi115.stdout:0/977: write d9/d15/f67 [648152,124521] 0 2021-11-03T17:15:32.127 INFO:tasks.workunit.client.1.smithi115.stdout:0/978: write d9/d15/d1a/d1d/dcc/d46/d8c/dfc/f128 [912681,23331] 0 2021-11-03T17:15:32.128 INFO:tasks.workunit.client.1.smithi115.stdout:0/979: write d9/db/f8e [3225468,75987] 0 2021-11-03T17:15:32.162 INFO:tasks.workunit.client.1.smithi115.stdout:0/980: symlink d9/d15/d78/l140 0 2021-11-03T17:15:32.244 INFO:tasks.workunit.client.1.smithi115.stdout:0/981: dwrite d9/d15/d1a/d1d/dcc/fcf [0,4194304] 0 2021-11-03T17:15:32.269 INFO:tasks.workunit.client.1.smithi115.stdout:0/982: rename d9/d15/d1a/d1d/dcc/fcf to d9/d15/db6/d10b/d124/d12d/d13f/f141 0 2021-11-03T17:15:32.456 INFO:tasks.workunit.client.1.smithi115.stdout:8/758: sync 2021-11-03T17:15:32.456 INFO:tasks.workunit.client.1.smithi115.stdout:1/834: sync 2021-11-03T17:15:32.456 INFO:tasks.workunit.client.1.smithi115.stdout:9/763: sync 2021-11-03T17:15:32.457 INFO:tasks.workunit.client.1.smithi115.stdout:5/980: sync 2021-11-03T17:15:32.457 INFO:tasks.workunit.client.1.smithi115.stdout:7/745: sync 2021-11-03T17:15:32.457 INFO:tasks.workunit.client.1.smithi115.stdout:3/993: sync 2021-11-03T17:15:32.457 INFO:tasks.workunit.client.1.smithi115.stdout:5/981: stat d1a/d28/d7b/d48/d7f/f86 0 2021-11-03T17:15:32.458 INFO:tasks.workunit.client.1.smithi115.stdout:1/835: mknod d7/d9d/c106 1 2021-11-03T17:15:32.462 INFO:tasks.workunit.client.1.smithi115.stdout:9/764: chown d1/d18/d2b/d42/d6e/la2 472 1 2021-11-03T17:15:32.466 INFO:tasks.workunit.client.1.smithi115.stdout:8/759: creat d4/d7d/dc2/d109/f10c x:0 0 0 2021-11-03T17:15:32.468 INFO:tasks.workunit.client.1.smithi115.stdout:9/765: dread d1/d1a/d48/f26 [0,4194304] 0 2021-11-03T17:15:32.471 INFO:tasks.workunit.client.1.smithi115.stdout:5/982: rename d1a/d28/d4c/d63/l87 to d1a/d3d/d58/dab/dd3/d12b/l14b 0 2021-11-03T17:15:32.477 INFO:tasks.workunit.client.1.smithi115.stdout:7/746: symlink d3/da/d18/d73/d52/db4/le6 0 2021-11-03T17:15:32.486 INFO:tasks.workunit.client.1.smithi115.stdout:1/836: write d7/f8a [9341544,81559] 0 2021-11-03T17:15:32.489 INFO:tasks.workunit.client.1.smithi115.stdout:8/760: creat d4/d6/d3f/da4/da6/dcc/f10d x:0 0 0 2021-11-03T17:15:32.491 INFO:tasks.workunit.client.1.smithi115.stdout:9/766: truncate d1/f1d 2670796 0 2021-11-03T17:15:32.495 INFO:tasks.workunit.client.1.smithi115.stdout:5/983: rmdir d1a/d28/d78 39 2021-11-03T17:15:32.495 INFO:tasks.workunit.client.1.smithi115.stdout:5/984: dread - d1a/d28/d7b/ff4 zero size 2021-11-03T17:15:32.499 INFO:tasks.workunit.client.1.smithi115.stdout:7/747: rmdir d3/d4c/d7b/dd0 0 2021-11-03T17:15:32.508 INFO:tasks.workunit.client.1.smithi115.stdout:3/994: link d13/d2a/d53/d72/d12e/d149/db7/fc5 d13/d50/f151 0 2021-11-03T17:15:32.509 INFO:tasks.workunit.client.1.smithi115.stdout:1/837: mknod d7/dc/dd/c107 1 2021-11-03T17:15:32.511 INFO:tasks.workunit.client.1.smithi115.stdout:3/995: dread d13/d2a/d60/f77 [0,4194304] 0 2021-11-03T17:15:32.515 INFO:tasks.workunit.client.1.smithi115.stdout:8/761: dread d4/d6/fa9 [0,4194304] 0 2021-11-03T17:15:32.519 INFO:tasks.workunit.client.1.smithi115.stdout:7/748: symlink d3/da/d18/d73/d52/db4/le7 0 2021-11-03T17:15:32.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: cluster 2021-11-03T17:15:30.647578+0000 2021-11-03T17:15:32.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14622) 31 : cluster [DBG] pgmap v19: 65 pgs: 65 active+clean; 2.5 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 26 MiB/s rd, 45 MiB/s wr, 226 op/s 2021-11-03T17:15:32.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: audit 2021-11-03T17:15:32.068444+0000 mon.smithi027 (mon.0) 716 : audit [INF] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:15:32.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: audit 2021-11-03T17:15:32.082872+0000 mon.smithi027 (mon.0) 717 : audit [INF] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:15:32.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: cluster 2021-11-03T17:15:32.088870+0000 mon.smithi027 (mon.0) 718 : 2021-11-03T17:15:32.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: cluster [DBG] Standby manager daemon smithi115.tcyodz restarted 2021-11-03T17:15:32.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: cluster 2021-11-03T17:15:32.089020+0000 2021-11-03T17:15:32.524 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: mon.smithi027 (mon.0) 719 : cluster [DBG] Standby manager daemon smithi115.tcyodz started 2021-11-03T17:15:32.524 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: audit 2021-11-03 2021-11-03T17:15:32.524 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: T17:15:32.090682+0000 mon.smithi027 (mon 2021-11-03T17:15:32.524 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: .0) 720 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:15:32.525 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: audit 2021 2021-11-03T17:15:32.525 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: -11-03T17:15:32.091105 2021-11-03T17:15:32.525 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: +0000 mon.smithi027 (mon.0) 721 : audit 2021-11-03T17:15:32.525 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: [DBG] from='mgr.? 172.21.15.115:0/4204420773' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi115.tcyodz/crt"}]: dispatch 2021-11-03T17:15:32.526 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: audit 2021-11-03T17: 2021-11-03T17:15:32.526 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: 15:32.091556+0000 mon.smithi027 (mon.0) 2021-11-03T17:15:32.526 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: 722 : audit [DBG] from='mgr.? 172.21.15.115:0/4204420773' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-11-03T17:15:32.526 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: audit 2021-11- 2021-11-03T17:15:32.527 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: 03T17:15:32.092149+0000 mon.smithi027 2021-11-03T17:15:32.527 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: (mon.0) 723 : audit [DBG] 2021-11-03T17:15:32.527 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:15:32.527 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: audit 2021-11-03T17:15:32.528 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: 2021-11-03T17:15:32.095669+0000 2021-11-03T17:15:32.528 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: mon.smithi027 (mon.0) 724 : audit 2021-11-03T17:15:32.528 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: [DBG] from='mgr.? 172.21.15.115:0/4204420773' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi115.tcyodz/key"}]: dispatch 2021-11-03T17:15:32.528 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: audit 2021-11-03T17:15: 2021-11-03T17:15:32.528 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: 32.099799+0000 mon.smithi027 (mon.0) 725 2021-11-03T17:15:32.529 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: : audit [INF] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:15:32.529 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: audit 2021-11-03 2021-11-03T17:15:32.529 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: T17:15:32.100135+0000 mon.smithi027 ( 2021-11-03T17:15:32.529 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: mon.0) 726 : audit [DBG] from='mgr.? 172.21.15.115:0/4204420773' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-11-03T17:15:32.530 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: 2021-11-03T17:15:32.530 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: audit 2021-11-03T17:15:32.530 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: 2021-11-03T17:15:32.100888+0000 mon.smithi027 (mon.0 2021-11-03T17:15:32.530 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: ) 727 : audit [INF] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi027.ufhhvb"}]: dispatch 2021-11-03T17:15:32.530 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: audit 2021-11- 2021-11-03T17:15:32.531 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: 03T17:15:32.105746+0000 2021-11-03T17:15:32.531 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: mon.smithi027 (mon.0) 728 : audit [INF] 2021-11-03T17:15:32.531 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi027.ufhhvb"}]': finished 2021-11-03T17:15:32.531 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: audit 2021-11-03T17:15:32.532 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: 2021-11-03T17:15:32.106655+0000 2021-11-03T17:15:32.532 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: mon.smithi027 (mon.0) 729 : audit [INF] 2021-11-03T17:15:32.532 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi115.tcyodz"}]: dispatch 2021-11-03T17:15:32.532 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: audit 2021-11-03T17 2021-11-03T17:15:32.533 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: :15:32.110875+0000 mon.smithi027 (mon. 2021-11-03T17:15:32.533 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: 0) 730 : audit [INF] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi115.tcyodz"}]': finished 2021-11-03T17:15:32.533 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: audit 2021-11-03T 2021-11-03T17:15:32.533 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: 17:15:32.112613+0000 mon.smithi027 ( 2021-11-03T17:15:32.533 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: mon.0) 731 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "quorum_status"}]: dispatch 2021-11-03T17:15:32.534 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:32 smithi027 conmon[32322]: 2021-11-03T17:15:32.534 INFO:tasks.workunit.client.1.smithi115.stdout:1/838: unlink d7/dc/d1a/f6b 0 2021-11-03T17:15:32.537 INFO:tasks.workunit.client.1.smithi115.stdout:8/762: creat d4/d6/d3f/d9a/d10b/f10e x:0 0 0 2021-11-03T17:15:32.539 INFO:tasks.workunit.client.1.smithi115.stdout:3/996: dwrite d13/d35/f3a [0,4194304] 0 2021-11-03T17:15:32.540 INFO:tasks.workunit.client.1.smithi115.stdout:3/997: read d13/d2a/d53/d72/fbd [1185970,815] 0 2021-11-03T17:15:32.541 INFO:tasks.workunit.client.1.smithi115.stdout:3/998: write d13/d51/d2e/dbf/f128 [651295,58741] 0 2021-11-03T17:15:32.541 INFO:tasks.workunit.client.1.smithi115.stdout:7/749: dwrite d3/da/d18/d9c/fd8 [0,4194304] 0 2021-11-03T17:15:32.544 INFO:tasks.workunit.client.1.smithi115.stdout:1/839: dread - d7/dc/dab/d73/fb0 zero size 2021-11-03T17:15:32.556 INFO:tasks.workunit.client.1.smithi115.stdout:3/999: rename d13/d2a/d53/d72/l11c to d13/d51/d2e/db4/d104/l152 0 2021-11-03T17:15:32.556 INFO:tasks.workunit.client.1.smithi115.stdout:9/767: dwrite d1/d18/d4a/d74/f84 [0,4194304] 0 2021-11-03T17:15:32.557 INFO:tasks.workunit.client.1.smithi115.stdout:7/750: getdents d3/d4c 0 2021-11-03T17:15:32.558 INFO:tasks.workunit.client.1.smithi115.stdout:7/751: mknod d3/da/d18/ce8 1 2021-11-03T17:15:32.561 INFO:tasks.workunit.client.1.smithi115.stdout:1/840: rmdir d7/dc/d101/d43/df4/d60/dde 39 2021-11-03T17:15:32.574 INFO:tasks.workunit.client.1.smithi115.stdout:1/841: chown d7/dc/d1a/f3d 91 1 2021-11-03T17:15:32.576 INFO:tasks.workunit.client.1.smithi115.stdout:9/768: rename d1/d8a/f118 to d1/d18/f11a 0 2021-11-03T17:15:32.577 INFO:tasks.workunit.client.1.smithi115.stdout:1/842: creat d7/f108 x:0 0 0 2021-11-03T17:15:32.579 INFO:tasks.workunit.client.1.smithi115.stdout:9/769: creat d1/d18/d4a/d74/f11b x:0 0 0 2021-11-03T17:15:32.579 INFO:tasks.workunit.client.1.smithi115.stdout:1/843: readlink d7/d9d/lcd 0 2021-11-03T17:15:32.582 INFO:tasks.workunit.client.1.smithi115.stdout:9/770: dread d1/d18/d2b/d45/dc2/fe7 [0,4194304] 0 2021-11-03T17:15:32.582 INFO:tasks.workunit.client.1.smithi115.stdout:9/771: dread - d1/d1a/fe0 zero size 2021-11-03T17:15:32.584 INFO:tasks.workunit.client.1.smithi115.stdout:8/763: dwrite d4/d6/d18/d30/d79/d7f/f89 [0,4194304] 0 2021-11-03T17:15:32.585 INFO:tasks.workunit.client.1.smithi115.stdout:1/844: write d7/d38/d59/f9b [1242415,85034] 0 2021-11-03T17:15:32.589 INFO:tasks.workunit.client.1.smithi115.stdout:8/764: fdatasync d4/d6/d3f/d43/f92 0 2021-11-03T17:15:32.594 INFO:tasks.workunit.client.1.smithi115.stdout:9/772: symlink d1/d1a/d48/d27/d2f/d6a/l11c 0 2021-11-03T17:15:32.595 INFO:tasks.workunit.client.1.smithi115.stdout:9/773: stat d1/d18/d2b/db9/df2 0 2021-11-03T17:15:32.596 INFO:tasks.workunit.client.1.smithi115.stdout:7/752: dwrite d3/da/d18/d73/d35/da3/fa8 [0,4194304] 0 2021-11-03T17:15:32.599 INFO:tasks.workunit.client.1.smithi115.stdout:8/765: creat d4/d6/d18/d30/d79/d7f/f10f x:0 0 0 2021-11-03T17:15:32.604 INFO:tasks.workunit.client.1.smithi115.stdout:1/845: rename d7/dc/dd/d3e/d58/dbf/lfa to d7/dc/dd/d3e/d58/l109 0 2021-11-03T17:15:32.607 INFO:tasks.workunit.client.1.smithi115.stdout:8/766: readlink d4/d6/l24 0 2021-11-03T17:15:32.607 INFO:tasks.workunit.client.1.smithi115.stdout:8/767: mknod d4/d6/d3f/d9a/d10b/c110 1 2021-11-03T17:15:32.608 INFO:tasks.workunit.client.1.smithi115.stdout:1/846: dread d7/dc/dd/d3e/ffc [0,4194304] 0 2021-11-03T17:15:32.609 INFO:tasks.workunit.client.1.smithi115.stdout:8/768: dread d4/f10a [0,4194304] 0 2021-11-03T17:15:32.611 INFO:tasks.workunit.client.1.smithi115.stdout:8/769: write d4/d6/d18/f3d [1494200,112007] 0 2021-11-03T17:15:32.614 INFO:tasks.workunit.client.1.smithi115.stdout:7/753: dread d3/da/d18/d73/dda/f9b [0,4194304] 0 2021-11-03T17:15:32.614 INFO:tasks.workunit.client.1.smithi115.stdout:9/774: dwrite d1/d18/d2b/d42/fbd [0,4194304] 0 2021-11-03T17:15:32.615 INFO:tasks.workunit.client.1.smithi115.stdout:1/847: mkdir d7/dc/d101/d43/df4/d60/db2/d10a 0 2021-11-03T17:15:32.616 INFO:tasks.workunit.client.1.smithi115.stdout:7/754: creat d3/da/d18/d73/d35/d53/fe9 x:0 0 0 2021-11-03T17:15:32.619 INFO:tasks.workunit.client.1.smithi115.stdout:7/755: creat d3/d4c/d7b/dd2/fea x:0 0 0 2021-11-03T17:15:32.622 INFO:tasks.workunit.client.1.smithi115.stdout:7/756: dread d3/da/d8e/f93 [0,4194304] 0 2021-11-03T17:15:32.623 INFO:tasks.workunit.client.1.smithi115.stdout:9/775: dread d1/d1a/d48/f73 [0,4194304] 0 2021-11-03T17:15:32.628 INFO:tasks.workunit.client.1.smithi115.stdout:1/848: write d7/dc/d101/d43/f6f [1494110,14605] 0 2021-11-03T17:15:32.629 INFO:tasks.workunit.client.1.smithi115.stdout:7/757: dread d3/f97 [0,4194304] 0 2021-11-03T17:15:32.630 INFO:tasks.workunit.client.1.smithi115.stdout:7/758: stat d3/da/d78/fba 0 2021-11-03T17:15:32.630 INFO:tasks.workunit.client.1.smithi115.stdout:7/759: write d3/da/d18/d73/d35/f7c [2238111,87636] 0 2021-11-03T17:15:32.633 INFO:tasks.workunit.client.1.smithi115.stdout:9/776: dread - d1/d1a/d48/d27/d2f/d7b/fef zero size 2021-11-03T17:15:32.634 INFO:tasks.workunit.client.1.smithi115.stdout:9/777: read d1/d18/d2b/d42/d6e/f5c [3135128,87102] 0 2021-11-03T17:15:32.634 INFO:tasks.workunit.client.1.smithi115.stdout:7/760: truncate d3/da/d78/f9f 1897303 0 2021-11-03T17:15:32.642 INFO:tasks.workunit.client.1.smithi115.stdout:9/778: symlink d1/d66/l11d 0 2021-11-03T17:15:32.646 INFO:tasks.workunit.client.1.smithi115.stdout:1/849: dwrite d7/f8 [0,4194304] 0 2021-11-03T17:15:32.647 INFO:tasks.workunit.client.1.smithi115.stdout:7/761: dread d3/da/f62 [0,4194304] 0 2021-11-03T17:15:32.647 INFO:tasks.workunit.client.1.smithi115.stdout:7/762: chown d3/da/d18/d9c/da1/fb9 6015222 1 2021-11-03T17:15:32.656 INFO:tasks.workunit.client.1.smithi115.stdout:9/779: mknod d1/d1a/c11e 1 2021-11-03T17:15:32.661 INFO:tasks.workunit.client.1.smithi115.stdout:7/763: symlink d3/d4c/da6/leb 0 2021-11-03T17:15:32.665 INFO:tasks.workunit.client.1.smithi115.stdout:9/780: getdents d1/d18/d2b/d45/dc2 0 2021-11-03T17:15:32.672 INFO:tasks.workunit.client.1.smithi115.stdout:1/850: unlink d7/dc/d101/d43/df4/lc6 0 2021-11-03T17:15:32.677 INFO:tasks.workunit.client.1.smithi115.stdout:1/851: creat d7/dc/dab/d73/f10b x:0 0 0 2021-11-03T17:15:32.677 INFO:tasks.workunit.client.1.smithi115.stdout:1/852: write d7/dc/d101/d43/df4/f69 [1356840,25040] 0 2021-11-03T17:15:32.681 INFO:tasks.workunit.client.1.smithi115.stdout:9/781: link d1/d18/d2b/d42/d6e/db5/ff3 d1/d18/d4a/f11f 0 2021-11-03T17:15:32.681 INFO:tasks.workunit.client.1.smithi115.stdout:9/782: fsync d1/d1a/d48/d27/d80/f104 0 2021-11-03T17:15:32.681 INFO:tasks.workunit.client.1.smithi115.stdout:9/783: fsync d1/d1a/d48/f73 0 2021-11-03T17:15:32.681 INFO:tasks.workunit.client.1.smithi115.stdout:9/784: dread - d1/d1a/dd9/f10e zero size 2021-11-03T17:15:32.683 INFO:tasks.workunit.client.1.smithi115.stdout:7/764: dwrite d3/da/d18/d73/dda/f5e [4194304,4194304] 0 2021-11-03T17:15:32.685 INFO:tasks.workunit.client.1.smithi115.stdout:9/785: creat d1/d1a/d48/d27/d2f/d6a/f120 x:0 0 0 2021-11-03T17:15:32.685 INFO:tasks.workunit.client.1.smithi115.stdout:9/786: chown d1/d1a 182 1 2021-11-03T17:15:32.685 INFO:tasks.workunit.client.1.smithi115.stdout:9/787: chown d1/d18/d2b/d42/fbd 4 1 2021-11-03T17:15:32.685 INFO:tasks.workunit.client.1.smithi115.stdout:1/853: symlink d7/dc/d101/d26/l10c 0 2021-11-03T17:15:32.687 INFO:tasks.workunit.client.1.smithi115.stdout:7/765: creat d3/da/d18/d73/d52/db4/fec x:0 0 0 2021-11-03T17:15:32.688 INFO:tasks.workunit.client.1.smithi115.stdout:7/766: write d3/da/d18/d73/d52/f56 [4118921,22276] 0 2021-11-03T17:15:32.688 INFO:tasks.workunit.client.1.smithi115.stdout:7/767: write d3/da/d18/d73/d35/d53/fc1 [881413,17656] 0 2021-11-03T17:15:32.697 INFO:tasks.workunit.client.1.smithi115.stdout:1/854: readlink d7/dc/d101/d4b/l71 0 2021-11-03T17:15:32.701 INFO:tasks.workunit.client.1.smithi115.stdout:7/768: rmdir d3/da/d18/d9c 39 2021-11-03T17:15:32.703 INFO:tasks.workunit.client.1.smithi115.stdout:9/788: symlink d1/d18/d4a/d74/dc7/l121 0 2021-11-03T17:15:32.712 INFO:tasks.workunit.client.1.smithi115.stdout:1/855: symlink d7/l10d 0 2021-11-03T17:15:32.716 INFO:tasks.workunit.client.1.smithi115.stdout:9/789: mkdir d1/d1a/d48/d27/d122 0 2021-11-03T17:15:32.718 INFO:tasks.workunit.client.1.smithi115.stdout:1/856: rmdir d7/d38/d84 39 2021-11-03T17:15:32.720 INFO:tasks.workunit.client.1.smithi115.stdout:1/857: getdents d7/dc/d101/d43/df4/d60 0 2021-11-03T17:15:32.721 INFO:tasks.workunit.client.1.smithi115.stdout:1/858: chown d7/dc/d101/d43/df4/d60/d8f 2016592 1 2021-11-03T17:15:32.721 INFO:tasks.workunit.client.1.smithi115.stdout:1/859: chown d7/dc/ldc 292 1 2021-11-03T17:15:32.722 INFO:tasks.workunit.client.1.smithi115.stdout:1/860: mknod d7/dc/d101/d43/df4/d60/dde/c10e 1 2021-11-03T17:15:32.728 INFO:tasks.workunit.client.1.smithi115.stdout:1/861: dread d7/dc/f16 [0,4194304] 0 2021-11-03T17:15:32.731 INFO:tasks.workunit.client.1.smithi115.stdout:1/862: mknod d7/c10f 1 2021-11-03T17:15:32.733 INFO:tasks.workunit.client.1.smithi115.stdout:1/863: chown d7/fb3 26271871 1 2021-11-03T17:15:32.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: cluster 2021-11-03T17:15:30.647578+0000 2021-11-03T17:15:32.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: mgr.smithi027.ufhhvb (mgr.14622) 31 : cluster [DBG] pgmap v19: 65 pgs: 65 active+clean; 2.5 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 26 MiB/s rd, 45 MiB/s wr, 226 op/s 2021-11-03T17:15:32.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: audit 2021-11-03T17:15:32.068444+0000 mon.smithi027 (mon.0) 2021-11-03T17:15:32.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: 716 : audit [INF] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:15:32.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: audit 2021-11-03T17:15:32.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: 2021-11-03T17:15:32.082872+0000 mon.smithi027 (mon. 2021-11-03T17:15:32.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: 0) 717 : audit [INF] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:15:32.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: 2021-11-03T17:15:32.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: cluster 2021-11-03T17:15:32.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: 2021-11-03T17:15:32.088870+0000 mon.smithi027 2021-11-03T17:15:32.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: (mon.0) 718 : cluster [DBG] Standby manager daemon smithi115.tcyodz restarted 2021-11-03T17:15:32.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: cluster 2021-11-03T17:15:32.089020+0000 mon.smithi027 (mon. 2021-11-03T17:15:32.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: 0) 719 : cluster [DBG] Standby manager daemon smithi115.tcyodz started 2021-11-03T17:15:32.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: audit 2021-11-03T17 2021-11-03T17:15:32.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: :15:32.090682+0000 mon.smithi027 (mon.0) 720 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:15:32.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: 2021-11-03T17:15:32.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: audit 2021-11-03T17:15:32.091105+0000 mon.smithi027 (mon 2021-11-03T17:15:32.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: .0) 721 : audit [DBG] from='mgr.? 172.21.15.115:0/4204420773' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi115.tcyodz/crt"}]: dispatch 2021-11-03T17:15:32.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: audit 2021 2021-11-03T17:15:32.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: -11-03T17:15:32.091556+0000 mon.smithi027 (mon.0) 722 : audit [DBG] 2021-11-03T17:15:32.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: from='mgr.? 172.21.15.115:0/4204420773' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-11-03T17:15:32.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: audit 2021 2021-11-03T17:15:32.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: -11-03T17:15:32.092149+0000 mon.smithi027 (mon.0) 723 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:15:32.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: audit 2021-11-03T17:15:32.095669+0000 mon.smithi027 2021-11-03T17:15:32.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: (mon.0) 724 : audit [DBG] from='mgr.? 172.21.15.115:0/4204420773' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi115.tcyodz/key"}]: dispatch 2021-11-03T17:15:32.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: audit 2021-11-03T17: 2021-11-03T17:15:32.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: 15:32.099799+0000 mon.smithi027 (mon.0) 725 : audit [INF] 2021-11-03T17:15:32.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:15:32.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: audit 2021 2021-11-03T17:15:32.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: -11-03T17:15:32.100135+0000 mon.smithi027 (mon.0) 726 : 2021-11-03T17:15:32.741 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: audit [DBG] from='mgr.? 172.21.15.115:0/4204420773' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-11-03T17:15:32.741 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: audit 2021-11-03T17:15:32.741 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: 2021-11-03T17:15:32.100888+0000 mon.smithi027 (mon.0) 727 2021-11-03T17:15:32.741 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: : audit [INF] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi027.ufhhvb"}]: dispatch 2021-11-03T17:15:32.741 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: audit 2021-11-03T17:15: 2021-11-03T17:15:32.742 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: 32.105746+0000 mon.smithi027 (mon.0) 728 : audit [INF] 2021-11-03T17:15:32.742 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi027.ufhhvb"}]': finished 2021-11-03T17:15:32.742 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: audit 2021- 2021-11-03T17:15:32.742 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: 11-03T17:15:32.106655+0000 mon.smithi027 (mon.0) 729 : audit 2021-11-03T17:15:32.743 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: [INF] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi115.tcyodz"}]: dispatch 2021-11-03T17:15:32.743 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: audit 2021-11-03T17:15:32.743 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: 2021-11-03T17:15:32.110875+0000 mon.smithi027 (mon.0) 730 : audit 2021-11-03T17:15:32.743 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: [INF] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi115.tcyodz"}]': finished 2021-11-03T17:15:32.743 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: audit 2021-11-03T17:15:32.744 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: 2021-11-03T17:15:32.112613+0000 mon.smithi027 (mon.0) 731 : audit [DBG] 2021-11-03T17:15:32.744 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:32 smithi115 conmon[34670]: from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "quorum_status"}]: dispatch 2021-11-03T17:15:32.745 INFO:tasks.workunit.client.1.smithi115.stdout:1/864: rmdir d7/dc/dd/d42 39 2021-11-03T17:15:32.745 INFO:tasks.workunit.client.1.smithi115.stdout:9/790: dwrite d1/f113 [0,4194304] 0 2021-11-03T17:15:32.745 INFO:tasks.workunit.client.1.smithi115.stdout:1/865: symlink d7/dc/d101/d43/l110 0 2021-11-03T17:15:32.746 INFO:tasks.workunit.client.1.smithi115.stdout:7/769: dwrite d3/da/d18/d9c/da1/fb9 [0,4194304] 0 2021-11-03T17:15:32.746 INFO:tasks.workunit.client.1.smithi115.stdout:7/770: dread - d3/da/d18/d73/d35/d53/fe9 zero size 2021-11-03T17:15:32.766 INFO:tasks.workunit.client.1.smithi115.stdout:1/866: rename d7/fdd to d7/dc/d1a/f111 0 2021-11-03T17:15:32.770 INFO:tasks.workunit.client.1.smithi115.stdout:1/867: creat d7/d38/d84/f112 x:0 0 0 2021-11-03T17:15:32.773 INFO:tasks.workunit.client.1.smithi115.stdout:1/868: dread - d7/dc/d1a/f4a zero size 2021-11-03T17:15:32.779 INFO:tasks.workunit.client.1.smithi115.stdout:9/791: dwrite d1/d18/d2b/db9/fc3 [0,4194304] 0 2021-11-03T17:15:32.784 INFO:tasks.workunit.client.1.smithi115.stdout:9/792: creat d1/d1a/d48/d27/d53/d83/d88/f123 x:0 0 0 2021-11-03T17:15:32.784 INFO:tasks.workunit.client.1.smithi115.stdout:9/793: read - d1/d1a/d48/d27/ff7 zero size 2021-11-03T17:15:32.787 INFO:tasks.workunit.client.1.smithi115.stdout:9/794: creat d1/d18/d2b/d42/f124 x:0 0 0 2021-11-03T17:15:32.787 INFO:tasks.workunit.client.1.smithi115.stdout:9/795: readlink d1/l4 0 2021-11-03T17:15:32.791 INFO:tasks.workunit.client.1.smithi115.stdout:9/796: link d1/d1a/d48/d27/l40 d1/d1a/d48/d27/d2f/l125 0 2021-11-03T17:15:32.791 INFO:tasks.workunit.client.1.smithi115.stdout:9/797: dread - d1/d1a/d48/d27/ff7 zero size 2021-11-03T17:15:32.792 INFO:tasks.workunit.client.1.smithi115.stdout:9/798: truncate d1/d1a/d48/d27/d53/f95 1389733 0 2021-11-03T17:15:32.792 INFO:tasks.workunit.client.1.smithi115.stdout:9/799: truncate d1/d1a/d48/d27/d53/d83/d88/ff5 957464 0 2021-11-03T17:15:32.792 INFO:tasks.workunit.client.1.smithi115.stdout:9/800: readlink d1/d1a/d48/d27/d53/d83/d88/dcd/lb1 0 2021-11-03T17:15:32.793 INFO:tasks.workunit.client.1.smithi115.stdout:9/801: mkdir d1/d1a/dd9/d126 0 2021-11-03T17:15:32.797 INFO:tasks.workunit.client.1.smithi115.stdout:9/802: unlink d1/d1a/d48/d27/d2f/d41/fe5 0 2021-11-03T17:15:32.799 INFO:tasks.workunit.client.1.smithi115.stdout:1/869: dwrite d7/dc/d101/d43/f6f [0,4194304] 0 2021-11-03T17:15:32.799 INFO:tasks.workunit.client.1.smithi115.stdout:1/870: fsync d7/ff6 0 2021-11-03T17:15:32.806 INFO:tasks.workunit.client.1.smithi115.stdout:9/803: rename d1/d18/d2b/d42/d6e/la2 to d1/d18/d4a/d74/l127 0 2021-11-03T17:15:32.831 INFO:tasks.workunit.client.1.smithi115.stdout:1/871: creat d7/dc/d1a/f113 x:0 0 0 2021-11-03T17:15:32.840 INFO:tasks.workunit.client.1.smithi115.stdout:9/804: truncate d1/d1a/d48/d27/d53/f77 3867677 0 2021-11-03T17:15:32.840 INFO:tasks.workunit.client.1.smithi115.stdout:9/805: write d1/d1a/fe0 [316636,112089] 0 2021-11-03T17:15:32.841 INFO:tasks.workunit.client.1.smithi115.stdout:9/806: write d1/d1a/d48/d27/d53/d83/d88/d9e/f3b [3446127,67670] 0 2021-11-03T17:15:32.843 INFO:tasks.workunit.client.1.smithi115.stdout:1/872: write d7/dc/d101/d43/df4/f7b [83693,119744] 0 2021-11-03T17:15:32.847 INFO:tasks.workunit.client.1.smithi115.stdout:9/807: unlink d1/d18/d2b/db9/fc3 0 2021-11-03T17:15:32.848 INFO:tasks.workunit.client.1.smithi115.stdout:1/873: fsync d7/dc/dab/d73/d7f/fb6 0 2021-11-03T17:15:32.851 INFO:tasks.workunit.client.1.smithi115.stdout:9/808: chown d1/d18/d2b/d45/lb0 211 1 2021-11-03T17:15:32.851 INFO:tasks.workunit.client.1.smithi115.stdout:9/809: chown d1/d1a/d48/d27/d53/d83/d88 493843480 1 2021-11-03T17:15:32.852 INFO:tasks.workunit.client.1.smithi115.stdout:9/810: read - d1/d18/d2b/db9/df2/f115 zero size 2021-11-03T17:15:32.854 INFO:tasks.workunit.client.1.smithi115.stdout:1/874: truncate d7/dc/d101/d4b/d55/f66 237356 0 2021-11-03T17:15:32.858 INFO:tasks.workunit.client.1.smithi115.stdout:9/811: rmdir d1/d1a/d48/d27/d53/d83 39 2021-11-03T17:15:32.869 INFO:tasks.workunit.client.1.smithi115.stdout:1/875: creat d7/dc/dd/d3e/f114 x:0 0 0 2021-11-03T17:15:32.869 INFO:tasks.workunit.client.1.smithi115.stdout:1/876: fdatasync d7/dc/dab/d73/d7f/fe8 0 2021-11-03T17:15:32.869 INFO:tasks.workunit.client.1.smithi115.stdout:1/877: fsync d7/f8 0 2021-11-03T17:15:32.870 INFO:tasks.workunit.client.1.smithi115.stdout:9/812: dread d1/d1a/d48/d27/d53/d83/d88/ff5 [0,4194304] 0 2021-11-03T17:15:32.872 INFO:tasks.workunit.client.1.smithi115.stdout:1/878: unlink d7/d38/d84/dca/led 0 2021-11-03T17:15:32.878 INFO:tasks.workunit.client.1.smithi115.stdout:9/813: rename d1/l4 to d1/d1a/d48/d27/d53/d83/d88/dcd/l128 0 2021-11-03T17:15:32.878 INFO:tasks.workunit.client.1.smithi115.stdout:9/814: fsync d1/d18/d2b/d42/ff 0 2021-11-03T17:15:32.893 INFO:tasks.workunit.client.1.smithi115.stdout:9/815: dwrite d1/d18/d4a/d74/d8b/dbc/fe1 [0,4194304] 0 2021-11-03T17:15:32.902 INFO:tasks.workunit.client.1.smithi115.stdout:9/816: creat d1/d18/d2b/d42/d63/f129 x:0 0 0 2021-11-03T17:15:32.913 INFO:tasks.workunit.client.1.smithi115.stdout:9/817: unlink d1/d1a/d48/d27/d53/d83/d88/d9e/d58/fc1 0 2021-11-03T17:15:32.914 INFO:tasks.workunit.client.1.smithi115.stdout:9/818: creat d1/d1a/d48/d27/d2f/d41/f12a x:0 0 0 2021-11-03T17:15:32.919 INFO:tasks.workunit.client.1.smithi115.stdout:9/819: mkdir d1/d18/d4a/d74/d8b/dbc/d12b 0 2021-11-03T17:15:32.919 INFO:tasks.workunit.client.1.smithi115.stdout:9/820: mknod d1/d1a/d48/d27/d53/c12c 1 2021-11-03T17:15:32.919 INFO:tasks.workunit.client.1.smithi115.stdout:9/821: creat d1/d8a/f12d x:0 0 0 2021-11-03T17:15:32.919 INFO:tasks.workunit.client.1.smithi115.stdout:9/822: write d1/d18/d2b/d42/fbd [8698516,75202] 0 2021-11-03T17:15:32.920 INFO:tasks.workunit.client.1.smithi115.stdout:9/823: chown d1/d1a/d48/d27/d2f/l125 252157 1 2021-11-03T17:15:32.920 INFO:tasks.workunit.client.1.smithi115.stdout:9/824: write d1/d18/d2b/d42/d6e/f5c [2001978,6110] 0 2021-11-03T17:15:32.923 INFO:tasks.workunit.client.1.smithi115.stdout:9/825: mkdir d1/d1a/dd9/d126/d12e 0 2021-11-03T17:15:32.923 INFO:tasks.workunit.client.1.smithi115.stdout:9/826: chown d1/d18/d2b/db9/fed 110 1 2021-11-03T17:15:32.926 INFO:tasks.workunit.client.1.smithi115.stdout:9/827: mkdir d1/d1a/d48/d27/d53/d83/d88/dcd/dda/d12f 0 2021-11-03T17:15:32.928 INFO:tasks.workunit.client.1.smithi115.stdout:9/828: read d1/d1a/d48/d27/d2f/fc6 [29722,6525] 0 2021-11-03T17:15:32.930 INFO:tasks.workunit.client.1.smithi115.stdout:9/829: symlink d1/d1a/d48/d27/d53/d83/d88/dcd/dda/l130 0 2021-11-03T17:15:32.930 INFO:tasks.workunit.client.1.smithi115.stdout:9/830: fdatasync d1/d1a/d48/d27/ff7 0 2021-11-03T17:15:32.933 INFO:tasks.workunit.client.1.smithi115.stdout:6/723: sync 2021-11-03T17:15:32.933 INFO:tasks.workunit.client.1.smithi115.stdout:6/724: mknod d0/d1/d10/d13/d59/d95/cf2 1 2021-11-03T17:15:32.935 INFO:tasks.workunit.client.1.smithi115.stdout:6/725: rmdir d0/d74 39 2021-11-03T17:15:32.940 INFO:tasks.workunit.client.1.smithi115.stdout:6/726: rename d0/de4 to d0/d1/d2/df3 0 2021-11-03T17:15:32.941 INFO:tasks.workunit.client.1.smithi115.stdout:6/727: fdatasync d0/d1/d2/ff 0 2021-11-03T17:15:32.944 INFO:tasks.workunit.client.1.smithi115.stdout:6/728: creat d0/d74/dae/db3/ff4 x:0 0 0 2021-11-03T17:15:32.945 INFO:tasks.workunit.client.1.smithi115.stdout:6/729: mknod d0/d7f/cf5 1 2021-11-03T17:15:32.945 INFO:tasks.workunit.client.1.smithi115.stdout:6/730: chown d0/d1 63203087 1 2021-11-03T17:15:32.946 INFO:tasks.workunit.client.1.smithi115.stdout:9/831: dwrite d1/d18/d4a/d74/f84 [0,4194304] 0 2021-11-03T17:15:32.947 INFO:tasks.workunit.client.1.smithi115.stdout:9/832: truncate d1/d18/d2b/d42/d6e/db5/f93 4433993 0 2021-11-03T17:15:32.963 INFO:tasks.workunit.client.1.smithi115.stdout:6/731: dwrite d0/d1/d2/f58 [0,4194304] 0 2021-11-03T17:15:32.966 INFO:tasks.workunit.client.1.smithi115.stdout:6/732: rename d0/f7e to d0/d7f/dab/dc6/ff6 0 2021-11-03T17:15:32.973 INFO:tasks.workunit.client.1.smithi115.stdout:6/733: dread d0/d1/fe7 [0,4194304] 0 2021-11-03T17:15:32.983 INFO:tasks.workunit.client.1.smithi115.stdout:9/833: dwrite d1/d18/d4a/fc4 [0,4194304] 0 2021-11-03T17:15:32.988 INFO:tasks.workunit.client.1.smithi115.stdout:6/734: dread d0/d1/d2/d3f/fc0 [0,4194304] 0 2021-11-03T17:15:32.988 INFO:tasks.workunit.client.1.smithi115.stdout:6/735: readlink d0/l5a 0 2021-11-03T17:15:32.999 INFO:tasks.workunit.client.1.smithi115.stdout:9/834: write d1/d18/d2b/d42/f21 [178556,106066] 0 2021-11-03T17:15:33.005 INFO:tasks.workunit.client.1.smithi115.stdout:6/736: write d0/d1/d2/f4a [194951,104583] 0 2021-11-03T17:15:33.005 INFO:tasks.workunit.client.1.smithi115.stdout:6/737: chown d0/d1/d6a/d88 18 1 2021-11-03T17:15:33.006 INFO:tasks.workunit.client.1.smithi115.stdout:6/738: write d0/d1/d10/d13/d59/fa3 [3290142,25199] 0 2021-11-03T17:15:33.009 INFO:tasks.workunit.client.1.smithi115.stdout:6/739: truncate d0/d1/d10/d13/d59/f9c 931104 0 2021-11-03T17:15:33.009 INFO:tasks.workunit.client.1.smithi115.stdout:6/740: read d0/d1/d10/d13/d1c/f2f [4261787,79422] 0 2021-11-03T17:15:33.015 INFO:tasks.workunit.client.1.smithi115.stdout:6/741: creat d0/d1/d10/d13/d87/ff7 x:0 0 0 2021-11-03T17:15:33.017 INFO:tasks.workunit.client.1.smithi115.stdout:9/835: dwrite d1/d18/d2b/d45/f114 [0,4194304] 0 2021-11-03T17:15:33.017 INFO:tasks.workunit.client.1.smithi115.stdout:6/742: mknod d0/d1/d6a/cf8 1 2021-11-03T17:15:33.049 INFO:tasks.workunit.client.1.smithi115.stdout:6/743: creat d0/d74/dae/ff9 x:0 0 0 2021-11-03T17:15:33.050 INFO:tasks.workunit.client.1.smithi115.stdout:6/744: fsync d0/d1/d2/d3f/deb/fb9 0 2021-11-03T17:15:33.050 INFO:tasks.workunit.client.1.smithi115.stdout:6/745: truncate d0/d1/d6a/d88/da0/db5/fe9 1936898 0 2021-11-03T17:15:33.058 INFO:tasks.workunit.client.1.smithi115.stdout:6/746: mkdir d0/d85/d8f/dfa 0 2021-11-03T17:15:33.060 INFO:tasks.workunit.client.1.smithi115.stdout:9/836: link d1/d18/d2b/d42/f20 d1/d1a/f131 0 2021-11-03T17:15:33.061 INFO:tasks.workunit.client.1.smithi115.stdout:9/837: readlink d1/d18/d2b/d42/d6e/ld2 0 2021-11-03T17:15:33.061 INFO:tasks.workunit.client.1.smithi115.stdout:9/838: truncate d1/d18/d2b/d45/dc2/fe7 983242 0 2021-11-03T17:15:33.061 INFO:tasks.workunit.client.1.smithi115.stdout:9/839: write d1/d18/d2b/d42/f124 [412473,57207] 0 2021-11-03T17:15:33.064 INFO:tasks.workunit.client.1.smithi115.stdout:6/747: truncate d0/d1/d2/d3f/deb/d4c/d96/d61/f8a 85935 0 2021-11-03T17:15:33.085 INFO:tasks.workunit.client.1.smithi115.stdout:6/748: write d0/d1/d2/d3f/deb/d4c/d96/d61/f8a [852322,93738] 0 2021-11-03T17:15:33.086 INFO:tasks.workunit.client.1.smithi115.stdout:6/749: mknod d0/d85/d8f/cfb 1 2021-11-03T17:15:33.086 INFO:tasks.workunit.client.1.smithi115.stdout:6/750: write d0/d1/f9e [324043,61327] 0 2021-11-03T17:15:33.088 INFO:tasks.workunit.client.1.smithi115.stdout:6/751: symlink d0/d85/d8f/lfc 0 2021-11-03T17:15:33.105 INFO:tasks.workunit.client.1.smithi115.stdout:6/752: dwrite d0/d1/d2/d3f/deb/d4c/d96/d61/f92 [0,4194304] 0 2021-11-03T17:15:33.116 INFO:tasks.workunit.client.1.smithi115.stdout:6/753: symlink d0/d1/lfd 0 2021-11-03T17:15:33.118 INFO:tasks.workunit.client.1.smithi115.stdout:6/754: unlink d0/d7f/ldd 0 2021-11-03T17:15:33.118 INFO:tasks.workunit.client.1.smithi115.stdout:6/755: fdatasync d0/d1/d10/d13/d59/f67 0 2021-11-03T17:15:33.120 INFO:tasks.workunit.client.1.smithi115.stdout:6/756: truncate d0/d1/d2/ff 3430216 0 2021-11-03T17:15:33.120 INFO:tasks.workunit.client.1.smithi115.stdout:6/757: write d0/d1/d10/d13/d87/d89/fac [519467,43484] 0 2021-11-03T17:15:33.139 INFO:tasks.workunit.client.1.smithi115.stdout:6/758: dwrite d0/d1/d2/f35 [4194304,4194304] 0 2021-11-03T17:15:33.141 INFO:tasks.workunit.client.1.smithi115.stdout:6/759: symlink d0/d1/d2/df3/lfe 0 2021-11-03T17:15:33.156 INFO:tasks.workunit.client.1.smithi115.stdout:6/760: dwrite d0/d1/d10/d13/d59/f67 [0,4194304] 0 2021-11-03T17:15:33.167 INFO:tasks.workunit.client.1.smithi115.stdout:6/761: mkdir d0/d7f/dff 0 2021-11-03T17:15:33.171 INFO:tasks.workunit.client.1.smithi115.stdout:6/762: chown d0/d1/d10/d13/d1c/d1e/l43 181350 1 2021-11-03T17:15:33.172 INFO:tasks.workunit.client.1.smithi115.stdout:6/763: mknod d0/d1/d10/d13/d87/dd2/c100 1 2021-11-03T17:15:33.197 INFO:tasks.workunit.client.1.smithi115.stdout:6/764: dwrite d0/d1/d2/d3f/f49 [0,4194304] 0 2021-11-03T17:15:33.199 INFO:tasks.workunit.client.1.smithi115.stdout:6/765: truncate d0/d1/d10/d13/d59/fa3 3064444 0 2021-11-03T17:15:33.207 INFO:tasks.workunit.client.1.smithi115.stdout:6/766: dread d0/d1/d2/d3f/deb/f83 [0,4194304] 0 2021-11-03T17:15:33.214 INFO:tasks.workunit.client.1.smithi115.stdout:6/767: symlink d0/d74/dae/db3/l101 0 2021-11-03T17:15:33.341 INFO:tasks.workunit.client.1.smithi115.stdout:4/990: sync 2021-11-03T17:15:33.341 INFO:tasks.workunit.client.1.smithi115.stdout:2/802: sync 2021-11-03T17:15:33.342 INFO:tasks.workunit.client.1.smithi115.stdout:4/991: fsync d7/dc/d1a/d21/db1/d56/d63/fd7 0 2021-11-03T17:15:33.342 INFO:tasks.workunit.client.1.smithi115.stdout:4/992: mknod d7/dc/d1a/d21/db1/d56/d63/c151 1 2021-11-03T17:15:33.342 INFO:tasks.workunit.client.1.smithi115.stdout:2/803: read d4/d6/d56/d6c/d34/d42/f8f [3571361,67148] 0 2021-11-03T17:15:33.343 INFO:tasks.workunit.client.1.smithi115.stdout:4/993: mknod d7/dc/d6a/d3c/dc5/dcc/dd9/c152 1 2021-11-03T17:15:33.348 INFO:tasks.workunit.client.1.smithi115.stdout:4/994: rmdir d7/dc/d6a/d116 39 2021-11-03T17:15:33.357 INFO:tasks.workunit.client.1.smithi115.stdout:4/995: mknod d7/dc/d1a/d21/db1/d56/c153 1 2021-11-03T17:15:33.358 INFO:tasks.workunit.client.1.smithi115.stdout:4/996: dread d7/dc/f95 [0,4194304] 0 2021-11-03T17:15:33.369 INFO:tasks.workunit.client.1.smithi115.stdout:4/997: dread d7/dc/d1a/d21/db1/d56/d63/d7b/fb2 [0,4194304] 0 2021-11-03T17:15:33.372 INFO:tasks.workunit.client.1.smithi115.stdout:4/998: unlink d7/dc/d6a/d101/l11c 0 2021-11-03T17:15:33.373 INFO:tasks.workunit.client.1.smithi115.stdout:4/999: write d7/dc/d6a/d72/f76 [944266,44130] 0 2021-11-03T17:15:33.409 INFO:tasks.workunit.client.1.smithi115.stdout:2/804: dwrite d4/d6/d56/d6c/d34/d43/d63/f99 [0,4194304] 0 2021-11-03T17:15:33.410 INFO:tasks.workunit.client.1.smithi115.stdout:2/805: fdatasync d4/d6/d13/d75/f10c 0 2021-11-03T17:15:33.410 INFO:tasks.workunit.client.1.smithi115.stdout:2/806: chown d4/d6/d56/d6c/lef 1098091324 1 2021-11-03T17:15:33.417 INFO:tasks.workunit.client.1.smithi115.stdout:2/807: write d4/d6/dad/f60 [1017757,103326] 0 2021-11-03T17:15:33.419 INFO:tasks.workunit.client.1.smithi115.stdout:2/808: truncate d4/d6/dad/d7f/fa5 3788979 0 2021-11-03T17:15:33.420 INFO:tasks.workunit.client.1.smithi115.stdout:2/809: readlink d4/d6/dad/d7f/l96 0 2021-11-03T17:15:33.424 INFO:tasks.workunit.client.1.smithi115.stdout:2/810: truncate d4/d6/d56/d6c/d34/d43/d63/d77/fa6 630589 0 2021-11-03T17:15:33.442 INFO:tasks.workunit.client.1.smithi115.stdout:0/983: sync 2021-11-03T17:15:33.443 INFO:tasks.workunit.client.1.smithi115.stdout:2/811: dwrite d4/d6/d13/f36 [0,4194304] 0 2021-11-03T17:15:33.443 INFO:tasks.workunit.client.1.smithi115.stdout:2/812: readlink d4/d6/dad/d7f/l96 0 2021-11-03T17:15:33.447 INFO:tasks.workunit.client.1.smithi115.stdout:0/984: rename d9/d15/db6/d10b/f12e to d9/d15/db6/d132/f142 0 2021-11-03T17:15:33.447 INFO:tasks.workunit.client.1.smithi115.stdout:0/985: dread - d9/d15/db6/d132/f142 zero size 2021-11-03T17:15:33.464 INFO:tasks.workunit.client.1.smithi115.stdout:0/986: dwrite d9/f66 [0,4194304] 0 2021-11-03T17:15:33.471 INFO:tasks.workunit.client.1.smithi115.stdout:2/813: rename d4/d6/dad/de8/d9c/d9f/f109 to d4/d6/d56/d6c/d34/d43/d63/d77/f114 0 2021-11-03T17:15:33.475 INFO:tasks.workunit.client.1.smithi115.stdout:2/814: symlink d4/d6/d56/d6c/d34/d43/d28/l115 0 2021-11-03T17:15:33.479 INFO:tasks.workunit.client.1.smithi115.stdout:2/815: unlink d4/d6/d13/d75/fe0 0 2021-11-03T17:15:33.485 INFO:tasks.workunit.client.1.smithi115.stdout:2/816: mkdir d4/d6/d56/d6c/d34/d42/d8d/dfd/d116 0 2021-11-03T17:15:33.489 INFO:tasks.workunit.client.1.smithi115.stdout:2/817: unlink d4/f35 0 2021-11-03T17:15:33.489 INFO:tasks.workunit.client.1.smithi115.stdout:2/818: fdatasync d4/d6/f45 0 2021-11-03T17:15:33.490 INFO:tasks.workunit.client.1.smithi115.stdout:0/987: dwrite d9/d15/db6/d10b/d124/d12d/d13f/f97 [0,4194304] 0 2021-11-03T17:15:33.491 INFO:tasks.workunit.client.1.smithi115.stdout:0/988: write d9/f98 [750617,19105] 0 2021-11-03T17:15:33.491 INFO:tasks.workunit.client.1.smithi115.stdout:0/989: fsync d9/d15/d1a/d1d/dcc/d46/d63/d70/f134 0 2021-11-03T17:15:33.500 INFO:tasks.workunit.client.1.smithi115.stdout:2/819: getdents d4/d6/dad/de8/d9c/d9f 0 2021-11-03T17:15:33.521 INFO:tasks.workunit.client.1.smithi115.stdout:0/990: dwrite d9/db/d89/fb1 [0,4194304] 0 2021-11-03T17:15:33.522 INFO:tasks.workunit.client.1.smithi115.stdout:0/991: read d9/d15/d1a/d1d/dcc/d46/d8c/f48 [2022574,85394] 0 2021-11-03T17:15:33.522 INFO:tasks.workunit.client.1.smithi115.stdout:0/992: chown d9/d15/db6/d10b/d124/d12d/d13f/f9a 16 1 2021-11-03T17:15:33.523 INFO:tasks.workunit.client.1.smithi115.stdout:0/993: write d9/d15/d1a/d1d/dcc/d46/d8c/dfc/deb/f118 [160918,87319] 0 2021-11-03T17:15:33.523 INFO:tasks.workunit.client.1.smithi115.stdout:2/820: dwrite d4/d6/dad/d7f/f10b [0,4194304] 0 2021-11-03T17:15:33.526 INFO:tasks.workunit.client.1.smithi115.stdout:0/994: symlink d9/d15/db6/d10b/d124/d12d/l143 0 2021-11-03T17:15:33.536 INFO:tasks.workunit.client.1.smithi115.stdout:0/995: link d9/d15/db6/d10b/d124/d12d/d13f/l120 d9/d37/dd3/l144 0 2021-11-03T17:15:33.544 INFO:tasks.workunit.client.1.smithi115.stdout:0/996: mknod d9/d15/d1a/d1d/c145 1 2021-11-03T17:15:33.562 INFO:tasks.workunit.client.1.smithi115.stdout:0/997: dwrite d9/d15/d1a/d1d/dcc/d46/d63/dae/fd7 [0,4194304] 0 2021-11-03T17:15:33.563 INFO:tasks.workunit.client.1.smithi115.stdout:0/998: fdatasync d9/d15/d1a/d1d/dcc/d46/d63/dae/fd7 0 2021-11-03T17:15:33.580 INFO:tasks.workunit.client.1.smithi115.stdout:0/999: dwrite d9/d15/d1a/fa4 [4194304,4194304] 0 2021-11-03T17:15:33.592 INFO:tasks.workunit.client.1.smithi115.stdout:5/985: sync 2021-11-03T17:15:33.598 INFO:tasks.workunit.client.1.smithi115.stdout:5/986: mkdir d1a/d10f/d14c 0 2021-11-03T17:15:33.600 INFO:tasks.workunit.client.1.smithi115.stdout:5/987: unlink d1a/d3d/dc4/f118 0 2021-11-03T17:15:33.619 INFO:tasks.workunit.client.1.smithi115.stdout:5/988: dwrite d1a/d28/daa/d109/f110 [0,4194304] 0 2021-11-03T17:15:33.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:33 smithi115 conmon[34670]: cephadm 2021-11-03T17:15:32.030441+0000 mgr.smithi027.ufhhvb (mgr.14622) 32 2021-11-03T17:15:33.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:33 smithi115 conmon[34670]: : cephadm [INF] poll took 6662.866 ms: 1 events 2021-11-03T17:15:33.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:33 smithi115 conmon[34670]: cephadm 2021-11-03T17:15:32.093016+0000 mgr.smithi027.ufhhvb (mgr.14622) 33 : cephadm [INF] Upgrade: Setting container_image for all mgr 2021-11-03T17:15:33.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:33 smithi115 conmon[34670]: cluster 2021-11-03T 2021-11-03T17:15:33.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:33 smithi115 conmon[34670]: 17:15:33.090651+0000 mon.smithi027 (mon.0) 732 : cluster [DBG] mgrmap e29: smithi027.ufhhvb(active, since 33s), standbys: smithi115.tcyodz 2021-11-03T17:15:33.758 INFO:tasks.workunit.client.1.smithi115.stdout:8/770: sync 2021-11-03T17:15:33.761 INFO:tasks.workunit.client.1.smithi115.stdout:8/771: creat d4/d6/d3f/d43/f111 x:0 0 0 2021-11-03T17:15:33.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:33 smithi027 conmon[32322]: cephadm 2021-11-03T17:15:32 2021-11-03T17:15:33.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:33 smithi027 conmon[32322]: .030441+0000 mgr.smithi027.ufhhvb (mgr.14622) 32 : cephadm [INF] poll took 6662.866 ms: 1 events 2021-11-03T17:15:33.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:33 smithi027 conmon[32322]: cephadm 2021-11-03T17:15:33.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:33 smithi027 conmon[32322]: 2021-11- 2021-11-03T17:15:33.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:33 smithi027 conmon[32322]: 03T17:15:32. 2021-11-03T17:15:33.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:33 smithi027 conmon[32322]: 093016+0000 mgr.smithi027.ufhhvb (mgr. 2021-11-03T17:15:33.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:33 smithi027 conmon[32322]: 14622) 33 : cephadm [INF] 2021-11-03T17:15:33.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:33 smithi027 conmon[32322]: Upgrade: Setting container_image for all mgr 2021-11-03T17:15:33.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:33 smithi027 conmon[32322]: cluster 2021-11-03T17:15:33.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:33 smithi027 conmon[32322]: 2021-11 2021-11-03T17:15:33.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:33 smithi027 conmon[32322]: -03T17:15:33 2021-11-03T17:15:33.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:33 smithi027 conmon[32322]: .090651+0000 mon.smithi027 (mon. 2021-11-03T17:15:33.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:33 smithi027 conmon[32322]: 0) 732 : cluster [DBG] 2021-11-03T17:15:33.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:33 smithi027 conmon[32322]: mgrmap e29: smithi027.ufhhvb(active, since 33s), standbys: smithi115.tcyodz 2021-11-03T17:15:33.775 INFO:tasks.workunit.client.1.smithi115.stdout:8/772: dwrite d4/d6/d18/d30/d79/d7f/ff1 [0,4194304] 0 2021-11-03T17:15:33.788 INFO:tasks.workunit.client.1.smithi115.stdout:8/773: rename d4/d6/d18/d44/d47/deb/f103 to d4/d6/d18/d44/dcd/f112 0 2021-11-03T17:15:33.817 INFO:tasks.workunit.client.1.smithi115.stdout:8/774: dwrite d4/d6/d18/d30/f3a [0,4194304] 0 2021-11-03T17:15:33.836 INFO:tasks.workunit.client.1.smithi115.stdout:8/775: dwrite d4/d6/d18/d44/f99 [0,4194304] 0 2021-11-03T17:15:33.838 INFO:tasks.workunit.client.1.smithi115.stdout:8/776: symlink d4/d6/d18/d30/dfc/l113 0 2021-11-03T17:15:33.840 INFO:tasks.workunit.client.1.smithi115.stdout:8/777: write d4/d6/d18/d30/d33/d93/fad [4265574,17773] 0 2021-11-03T17:15:33.842 INFO:tasks.workunit.client.1.smithi115.stdout:8/778: mknod d4/d7d/c114 1 2021-11-03T17:15:33.843 INFO:tasks.workunit.client.1.smithi115.stdout:8/779: chown d4/d6/d3f/d43/d88/de3/lef 811431794 1 2021-11-03T17:15:33.843 INFO:tasks.workunit.client.1.smithi115.stdout:8/780: write d4/d6/d18/d44/dcd/ff7 [853265,70158] 0 2021-11-03T17:15:33.845 INFO:tasks.workunit.client.1.smithi115.stdout:8/781: mkdir d4/d6/d18/d44/d115 0 2021-11-03T17:15:33.846 INFO:tasks.workunit.client.1.smithi115.stdout:8/782: chown d4/d6/d3f/d43/d88/fdc 12103820 1 2021-11-03T17:15:33.854 INFO:tasks.workunit.client.1.smithi115.stdout:8/783: dread d4/d6/d18/d30/f35 [4194304,4194304] 0 2021-11-03T17:15:33.859 INFO:tasks.workunit.client.1.smithi115.stdout:7/771: sync 2021-11-03T17:15:33.860 INFO:tasks.workunit.client.1.smithi115.stdout:7/772: write d3/f9 [2448521,61698] 0 2021-11-03T17:15:33.861 INFO:tasks.workunit.client.1.smithi115.stdout:7/773: readlink d3/d4c/da6/leb 0 2021-11-03T17:15:33.861 INFO:tasks.workunit.client.1.smithi115.stdout:7/774: chown d3/da/d18/d73/d35/d53/fc1 16664 1 2021-11-03T17:15:33.867 INFO:tasks.workunit.client.1.smithi115.stdout:7/775: dread d3/d4c/f5b [4194304,4194304] 0 2021-11-03T17:15:33.869 INFO:tasks.workunit.client.1.smithi115.stdout:7/776: mkdir d3/da/d18/d9c/da1/ded 0 2021-11-03T17:15:33.869 INFO:tasks.workunit.client.1.smithi115.stdout:7/777: write d3/da/d18/d73/d35/f9d [1273003,122498] 0 2021-11-03T17:15:33.870 INFO:tasks.workunit.client.1.smithi115.stdout:7/778: write d3/da/d18/d73/fc2 [1642608,50120] 0 2021-11-03T17:15:33.871 INFO:tasks.workunit.client.1.smithi115.stdout:7/779: getdents d3/da/d18/d9c/da1/ded 0 2021-11-03T17:15:33.872 INFO:tasks.workunit.client.1.smithi115.stdout:7/780: creat d3/da/d18/d73/d35/dbb/fee x:0 0 0 2021-11-03T17:15:33.872 INFO:tasks.workunit.client.1.smithi115.stdout:7/781: stat d3/da/d18/d73/d35/f5f 0 2021-11-03T17:15:33.872 INFO:tasks.workunit.client.1.smithi115.stdout:7/782: chown d3/lc9 219767746 1 2021-11-03T17:15:33.872 INFO:tasks.workunit.client.1.smithi115.stdout:7/783: chown d3/da/d18/d73/dda/l55 169566388 1 2021-11-03T17:15:33.880 INFO:tasks.workunit.client.1.smithi115.stdout:7/784: dread d3/da/d18/d73/d35/f5f [4194304,4194304] 0 2021-11-03T17:15:33.883 INFO:tasks.workunit.client.1.smithi115.stdout:7/785: link l0 d3/da/d18/d73/d35/lef 0 2021-11-03T17:15:33.885 INFO:tasks.workunit.client.1.smithi115.stdout:7/786: truncate d3/da/d18/d73/d35/fad 865905 0 2021-11-03T17:15:33.886 INFO:tasks.workunit.client.1.smithi115.stdout:8/784: dwrite d4/d7d/dc2/df9/f104 [0,4194304] 0 2021-11-03T17:15:33.887 INFO:tasks.workunit.client.1.smithi115.stdout:7/787: mkdir d3/da/d18/d73/d35/df0 0 2021-11-03T17:15:33.896 INFO:tasks.workunit.client.1.smithi115.stdout:7/788: dread d3/da/f1b [0,4194304] 0 2021-11-03T17:15:33.901 INFO:tasks.workunit.client.1.smithi115.stdout:7/789: mkdir d3/da/d18/d73/d52/df1 0 2021-11-03T17:15:33.902 INFO:tasks.workunit.client.1.smithi115.stdout:7/790: stat d3/da/d18/fce 0 2021-11-03T17:15:33.902 INFO:tasks.workunit.client.1.smithi115.stdout:8/785: dwrite d4/d6/d18/d44/f62 [0,4194304] 0 2021-11-03T17:15:33.915 INFO:tasks.workunit.client.1.smithi115.stdout:7/791: dread d3/da/f64 [0,4194304] 0 2021-11-03T17:15:33.919 INFO:tasks.workunit.client.1.smithi115.stdout:8/786: write d4/d6/f1d [1085342,69579] 0 2021-11-03T17:15:33.927 INFO:tasks.workunit.client.1.smithi115.stdout:8/787: truncate d4/d6/f2b 244825 0 2021-11-03T17:15:33.928 INFO:tasks.workunit.client.1.smithi115.stdout:8/788: dread d4/d6/d3f/da4/fdb [0,4194304] 0 2021-11-03T17:15:33.929 INFO:tasks.workunit.client.1.smithi115.stdout:8/789: creat d4/d6/d18/d44/d6e/f116 x:0 0 0 2021-11-03T17:15:33.931 INFO:tasks.workunit.client.1.smithi115.stdout:8/790: creat d4/d6/d3f/d9a/f117 x:0 0 0 2021-11-03T17:15:33.931 INFO:tasks.workunit.client.1.smithi115.stdout:8/791: mknod d4/d6/d18/d30/d79/de7/c118 1 2021-11-03T17:15:33.932 INFO:tasks.workunit.client.1.smithi115.stdout:7/792: dwrite d3/da/d8e/fe2 [0,4194304] 0 2021-11-03T17:15:33.932 INFO:tasks.workunit.client.1.smithi115.stdout:7/793: truncate d3/d4c/d7b/dd2/fdd 585368 0 2021-11-03T17:15:33.932 INFO:tasks.workunit.client.1.smithi115.stdout:7/794: readlink d3/da/d8e/l8f 0 2021-11-03T17:15:33.932 INFO:tasks.workunit.client.1.smithi115.stdout:7/795: read - d3/d4c/d7b/dd2/fea zero size 2021-11-03T17:15:33.938 INFO:tasks.workunit.client.1.smithi115.stdout:8/792: unlink d4/d6/d18/d44/f62 0 2021-11-03T17:15:33.938 INFO:tasks.workunit.client.1.smithi115.stdout:8/793: chown d4/d6/f2c 42 1 2021-11-03T17:15:33.950 INFO:tasks.workunit.client.1.smithi115.stdout:8/794: unlink d4/d6/f3c 0 2021-11-03T17:15:33.954 INFO:tasks.workunit.client.1.smithi115.stdout:7/796: rename d3/da/d18/d73/d35/d53/f87 to d3/da/d18/d73/dc8/ff2 0 2021-11-03T17:15:33.954 INFO:tasks.workunit.client.1.smithi115.stdout:7/797: truncate d3/d4c/fa5 170187 0 2021-11-03T17:15:33.955 INFO:tasks.workunit.client.1.smithi115.stdout:8/795: creat d4/d6/d3f/d9a/d10b/f119 x:0 0 0 2021-11-03T17:15:33.956 INFO:tasks.workunit.client.1.smithi115.stdout:8/796: read - d4/d6/d3f/da4/da6/dcc/f10d zero size 2021-11-03T17:15:33.956 INFO:tasks.workunit.client.1.smithi115.stdout:8/797: fsync d4/d6/d18/d44/d6e/f116 0 2021-11-03T17:15:33.957 INFO:tasks.workunit.client.1.smithi115.stdout:8/798: read d4/d6/d3f/da4/fdb [79079,101726] 0 2021-11-03T17:15:33.962 INFO:tasks.workunit.client.1.smithi115.stdout:7/798: dread d3/da/d18/f22 [0,4194304] 0 2021-11-03T17:15:33.967 INFO:tasks.workunit.client.1.smithi115.stdout:8/799: rename d4/d7d/dc2 to d4/d6/d18/d30/d79/d7f/d11a 0 2021-11-03T17:15:33.972 INFO:tasks.workunit.client.1.smithi115.stdout:8/800: write d4/d7d/fa0 [4104689,41109] 0 2021-11-03T17:15:33.982 INFO:tasks.workunit.client.1.smithi115.stdout:7/799: dwrite d3/da/d8e/f93 [0,4194304] 0 2021-11-03T17:15:33.988 INFO:tasks.workunit.client.1.smithi115.stdout:7/800: link d3/da/f62 d3/da/d18/d73/d35/df0/ff3 0 2021-11-03T17:15:33.989 INFO:tasks.workunit.client.1.smithi115.stdout:7/801: chown d3/da/d18/d73/d35/fad 0 1 2021-11-03T17:15:33.990 INFO:tasks.workunit.client.1.smithi115.stdout:8/801: dwrite d4/d6/d18/d44/d6e/fbf [0,4194304] 0 2021-11-03T17:15:34.002 INFO:tasks.workunit.client.1.smithi115.stdout:8/802: readlink d4/d6/l2e 0 2021-11-03T17:15:34.004 INFO:tasks.workunit.client.1.smithi115.stdout:7/802: symlink d3/da/d78/lf4 0 2021-11-03T17:15:34.005 INFO:tasks.workunit.client.1.smithi115.stdout:7/803: fdatasync d3/da/d18/d73/d35/fad 0 2021-11-03T17:15:34.008 INFO:tasks.workunit.client.1.smithi115.stdout:8/803: truncate d4/d6/d18/d44/d47/deb/f102 3479028 0 2021-11-03T17:15:34.011 INFO:tasks.workunit.client.1.smithi115.stdout:1/879: sync 2021-11-03T17:15:34.026 INFO:tasks.workunit.client.1.smithi115.stdout:8/804: rename d4/d6/d18/d44 to d4/d6/d3f/d9a/d11b 0 2021-11-03T17:15:34.031 INFO:tasks.workunit.client.1.smithi115.stdout:8/805: dread d4/d6/d18/d30/d79/fb2 [0,4194304] 0 2021-11-03T17:15:34.032 INFO:tasks.workunit.client.1.smithi115.stdout:8/806: dread - d4/d6/d18/d30/d33/d93/f108 zero size 2021-11-03T17:15:34.032 INFO:tasks.workunit.client.1.smithi115.stdout:7/804: dwrite d3/d4c/f5b [0,4194304] 0 2021-11-03T17:15:34.039 INFO:tasks.workunit.client.1.smithi115.stdout:7/805: mkdir d3/d4c/da6/df5 0 2021-11-03T17:15:34.040 INFO:tasks.workunit.client.1.smithi115.stdout:1/880: dwrite d7/dc/dab/d73/f10b [0,4194304] 0 2021-11-03T17:15:34.043 INFO:tasks.workunit.client.1.smithi115.stdout:8/807: symlink d4/d7d/l11c 0 2021-11-03T17:15:34.045 INFO:tasks.workunit.client.1.smithi115.stdout:7/806: creat d3/da/d78/ff6 x:0 0 0 2021-11-03T17:15:34.055 INFO:tasks.workunit.client.1.smithi115.stdout:8/808: truncate d4/d6/d3f/d9a/f84 155427 0 2021-11-03T17:15:34.055 INFO:tasks.workunit.client.1.smithi115.stdout:8/809: mknod d4/d6/d18/d30/d79/d7f/d11a/c11d 1 2021-11-03T17:15:34.060 INFO:tasks.workunit.client.1.smithi115.stdout:8/810: rename d4/d6/d18/d30/d79/de7 to d4/d6/d18/d30/d79/d11e 0 2021-11-03T17:15:34.062 INFO:tasks.workunit.client.1.smithi115.stdout:8/811: rmdir d4/d6/d3f/d9a/d10b 39 2021-11-03T17:15:34.063 INFO:tasks.workunit.client.1.smithi115.stdout:8/812: dread - d4/d6/d3f/d9a/d11b/dcd/f112 zero size 2021-11-03T17:15:34.064 INFO:tasks.workunit.client.1.smithi115.stdout:8/813: dread d4/d6/d3f/f61 [0,4194304] 0 2021-11-03T17:15:34.064 INFO:tasks.workunit.client.1.smithi115.stdout:8/814: readlink d4/d6/lf2 0 2021-11-03T17:15:34.065 INFO:tasks.workunit.client.1.smithi115.stdout:8/815: dread - d4/d6/d3f/dde/fe8 zero size 2021-11-03T17:15:34.065 INFO:tasks.workunit.client.1.smithi115.stdout:8/816: rmdir d4/d6/d18/d30/d79/d7f/d11a 39 2021-11-03T17:15:34.067 INFO:tasks.workunit.client.1.smithi115.stdout:1/881: dwrite d7/d38/fd4 [0,4194304] 0 2021-11-03T17:15:34.068 INFO:tasks.workunit.client.1.smithi115.stdout:7/807: dwrite d3/da/d18/d73/d35/f5f [4194304,4194304] 0 2021-11-03T17:15:34.070 INFO:tasks.workunit.client.1.smithi115.stdout:1/882: mknod d7/dc/d101/d43/df4/d60/db2/c115 1 2021-11-03T17:15:34.084 INFO:tasks.workunit.client.1.smithi115.stdout:1/883: dread d7/d38/d84/f9a [0,4194304] 0 2021-11-03T17:15:34.085 INFO:tasks.workunit.client.1.smithi115.stdout:8/817: dwrite d4/d6/d18/f63 [0,4194304] 0 2021-11-03T17:15:34.089 INFO:tasks.workunit.client.1.smithi115.stdout:1/884: link d7/dc/ff d7/dc/d101/d43/df4/d60/d8f/ddf/f116 0 2021-11-03T17:15:34.089 INFO:tasks.workunit.client.1.smithi115.stdout:1/885: fsync d7/dc/d101/d43/f6f 0 2021-11-03T17:15:34.095 INFO:tasks.workunit.client.1.smithi115.stdout:8/818: rename d4/d6/d18/f23 to d4/d6/d3f/da4/f11f 0 2021-11-03T17:15:34.095 INFO:tasks.workunit.client.1.smithi115.stdout:8/819: write d4/d6/d3f/d9a/d11b/dcd/f112 [624111,94592] 0 2021-11-03T17:15:34.095 INFO:tasks.workunit.client.1.smithi115.stdout:7/808: dwrite d3/f9 [4194304,4194304] 0 2021-11-03T17:15:34.096 INFO:tasks.workunit.client.1.smithi115.stdout:7/809: fdatasync d3/fd5 0 2021-11-03T17:15:34.096 INFO:tasks.workunit.client.1.smithi115.stdout:1/886: symlink d7/dc/d101/d43/df4/d60/d8f/l117 0 2021-11-03T17:15:34.097 INFO:tasks.workunit.client.1.smithi115.stdout:1/887: write d7/d38/d84/f9a [832671,61385] 0 2021-11-03T17:15:34.105 INFO:tasks.workunit.client.1.smithi115.stdout:7/810: dread d3/da/d18/d9c/da1/fb1 [0,4194304] 0 2021-11-03T17:15:34.124 INFO:tasks.workunit.client.1.smithi115.stdout:7/811: dwrite d3/da/d18/d73/d52/f56 [0,4194304] 0 2021-11-03T17:15:34.128 INFO:tasks.workunit.client.1.smithi115.stdout:1/888: mknod d7/d9d/c118 1 2021-11-03T17:15:34.130 INFO:tasks.workunit.client.1.smithi115.stdout:8/820: link d4/d6/d18/d30/l74 d4/d6/d3f/d43/d88/de3/l120 0 2021-11-03T17:15:34.133 INFO:tasks.workunit.client.1.smithi115.stdout:7/812: mkdir d3/df7 0 2021-11-03T17:15:34.133 INFO:tasks.workunit.client.1.smithi115.stdout:7/813: truncate d3/da/d18/d73/d35/d53/fb7 4263817 0 2021-11-03T17:15:34.133 INFO:tasks.workunit.client.1.smithi115.stdout:7/814: write d3/d4c/fa5 [870029,65928] 0 2021-11-03T17:15:34.134 INFO:tasks.workunit.client.1.smithi115.stdout:7/815: stat d3/f98 0 2021-11-03T17:15:34.134 INFO:tasks.workunit.client.1.smithi115.stdout:7/816: chown d3/da/d18/d73/fc2 1296 1 2021-11-03T17:15:34.135 INFO:tasks.workunit.client.1.smithi115.stdout:7/817: dread d3/da/d18/d73/dda/f59 [0,4194304] 0 2021-11-03T17:15:34.150 INFO:tasks.workunit.client.1.smithi115.stdout:1/889: unlink d7/dc/fa2 0 2021-11-03T17:15:34.155 INFO:tasks.workunit.client.1.smithi115.stdout:8/821: unlink d4/l49 0 2021-11-03T17:15:34.155 INFO:tasks.workunit.client.1.smithi115.stdout:8/822: fsync d4/d6/d18/d30/f31 0 2021-11-03T17:15:34.158 INFO:tasks.workunit.client.1.smithi115.stdout:7/818: dwrite d3/da/d18/d73/d35/da3/fa8 [0,4194304] 0 2021-11-03T17:15:34.172 INFO:tasks.workunit.client.1.smithi115.stdout:1/890: dwrite d7/dc/d1a/f111 [0,4194304] 0 2021-11-03T17:15:34.173 INFO:tasks.workunit.client.1.smithi115.stdout:7/819: mkdir d3/da/d18/d9c/da1/ded/df8 0 2021-11-03T17:15:34.174 INFO:tasks.workunit.client.1.smithi115.stdout:7/820: fdatasync d3/da/d78/fbe 0 2021-11-03T17:15:34.186 INFO:tasks.workunit.client.1.smithi115.stdout:1/891: truncate d7/dc/f15 3413022 0 2021-11-03T17:15:34.187 INFO:tasks.workunit.client.1.smithi115.stdout:1/892: write d7/dc/dd/d3e/d58/dbf/fd8 [334936,39649] 0 2021-11-03T17:15:34.187 INFO:tasks.workunit.client.1.smithi115.stdout:7/821: truncate d3/da/d18/f22 1835878 0 2021-11-03T17:15:34.192 INFO:tasks.workunit.client.1.smithi115.stdout:8/823: rename d4/d6/l3e to d4/d6/d18/d9d/l121 0 2021-11-03T17:15:34.192 INFO:tasks.workunit.client.1.smithi115.stdout:8/824: dread - d4/d6/d18/d30/d79/ffe zero size 2021-11-03T17:15:34.225 INFO:tasks.workunit.client.1.smithi115.stdout:7/822: creat d3/da/d18/ff9 x:0 0 0 2021-11-03T17:15:34.254 INFO:tasks.workunit.client.1.smithi115.stdout:7/823: rename d3/da/d8e/f93 to d3/df7/ffa 0 2021-11-03T17:15:34.263 INFO:tasks.workunit.client.1.smithi115.stdout:8/825: mkdir d4/d6/d18/d30/d79/d7f/d122 0 2021-11-03T17:15:34.267 INFO:tasks.workunit.client.1.smithi115.stdout:1/893: write d7/dc/d101/d4b/d55/f97 [784866,79644] 0 2021-11-03T17:15:34.275 INFO:tasks.workunit.client.1.smithi115.stdout:7/824: readlink d3/da/d18/d73/l4e 0 2021-11-03T17:15:34.278 INFO:tasks.workunit.client.1.smithi115.stdout:7/825: getdents d3/da/d18/d73/d52/db4/dd1 0 2021-11-03T17:15:34.279 INFO:tasks.workunit.client.1.smithi115.stdout:8/826: write d4/d6/f2a [881027,22950] 0 2021-11-03T17:15:34.280 INFO:tasks.workunit.client.1.smithi115.stdout:8/827: fdatasync d4/d6/d3f/d9a/fc4 0 2021-11-03T17:15:34.282 INFO:tasks.workunit.client.1.smithi115.stdout:7/826: rmdir d3/da/d18/d73/d35/df0 39 2021-11-03T17:15:34.282 INFO:tasks.workunit.client.1.smithi115.stdout:7/827: dread - d3/da/f7e zero size 2021-11-03T17:15:34.283 INFO:tasks.workunit.client.1.smithi115.stdout:7/828: dread d3/da/d78/fbe [0,4194304] 0 2021-11-03T17:15:34.286 INFO:tasks.workunit.client.1.smithi115.stdout:8/828: rename d4/d6/d3f/d9a/d10b/f10e to d4/d6/d18/d30/d33/d93/f123 0 2021-11-03T17:15:34.287 INFO:tasks.workunit.client.1.smithi115.stdout:8/829: write d4/d6/d18/d30/d33/d93/f108 [667488,88119] 0 2021-11-03T17:15:34.288 INFO:tasks.workunit.client.1.smithi115.stdout:7/829: mkdir d3/da/d18/dfb 0 2021-11-03T17:15:34.288 INFO:tasks.workunit.client.1.smithi115.stdout:7/830: dread - d3/da/d78/ff6 zero size 2021-11-03T17:15:34.288 INFO:tasks.workunit.client.1.smithi115.stdout:7/831: fdatasync d3/da/d18/d73/d35/f3d 0 2021-11-03T17:15:34.291 INFO:tasks.workunit.client.1.smithi115.stdout:7/832: rename d3/da/d18/d9c/da1/fb9 to d3/d4c/ffc 0 2021-11-03T17:15:34.291 INFO:tasks.workunit.client.1.smithi115.stdout:7/833: dread - d3/da/d78/ff6 zero size 2021-11-03T17:15:34.291 INFO:tasks.workunit.client.1.smithi115.stdout:7/834: readlink d3/d4c/da6/leb 0 2021-11-03T17:15:34.291 INFO:tasks.workunit.client.1.smithi115.stdout:7/835: chown d3/da/d18/d73/d35/d53/f58 932 1 2021-11-03T17:15:34.292 INFO:tasks.workunit.client.1.smithi115.stdout:7/836: readlink d3/da/d18/d9c/lb6 0 2021-11-03T17:15:34.292 INFO:tasks.workunit.client.1.smithi115.stdout:7/837: chown d3/l10 231099277 1 2021-11-03T17:15:34.292 INFO:tasks.workunit.client.1.smithi115.stdout:7/838: chown d3/da/d18/d73/d35/d53/dde 1 1 2021-11-03T17:15:34.292 INFO:tasks.workunit.client.1.smithi115.stdout:8/830: dread d4/d6/d18/d30/d33/fe9 [0,4194304] 0 2021-11-03T17:15:34.295 INFO:tasks.workunit.client.1.smithi115.stdout:1/894: dwrite d7/dc/d1a/f113 [0,4194304] 0 2021-11-03T17:15:34.295 INFO:tasks.workunit.client.1.smithi115.stdout:8/831: rename d4/d6/d3f/d43/d88/f8b to d4/d6/d3f/d9a/d11b/dcd/dd0/f124 0 2021-11-03T17:15:34.295 INFO:tasks.workunit.client.1.smithi115.stdout:8/832: read d4/d6/d3f/f61 [284314,81386] 0 2021-11-03T17:15:34.306 INFO:tasks.workunit.client.1.smithi115.stdout:1/895: symlink d7/dc/d101/d4b/d55/l119 0 2021-11-03T17:15:34.318 INFO:tasks.workunit.client.1.smithi115.stdout:8/833: dwrite d4/d6/fd2 [4194304,4194304] 0 2021-11-03T17:15:34.325 INFO:tasks.workunit.client.1.smithi115.stdout:7/839: dwrite d3/d4c/f5b [0,4194304] 0 2021-11-03T17:15:34.325 INFO:tasks.workunit.client.1.smithi115.stdout:7/840: chown d3/da/d18/d73/l4e 104 1 2021-11-03T17:15:34.336 INFO:tasks.workunit.client.1.smithi115.stdout:1/896: creat d7/d38/d84/f11a x:0 0 0 2021-11-03T17:15:34.346 INFO:tasks.workunit.client.1.smithi115.stdout:1/897: dread d7/f8b [0,4194304] 0 2021-11-03T17:15:34.371 INFO:tasks.workunit.client.1.smithi115.stdout:8/834: write d4/d6/d18/d30/f34 [3904943,50396] 0 2021-11-03T17:15:34.371 INFO:tasks.workunit.client.1.smithi115.stdout:8/835: truncate d4/d6/d3f/dde/fe8 22908 0 2021-11-03T17:15:34.375 INFO:tasks.workunit.client.1.smithi115.stdout:7/841: symlink d3/da/d18/d73/d52/df1/lfd 0 2021-11-03T17:15:34.380 INFO:tasks.workunit.client.1.smithi115.stdout:1/898: creat d7/dc/dd/d3e/d58/dbf/f11b x:0 0 0 2021-11-03T17:15:34.384 INFO:tasks.workunit.client.1.smithi115.stdout:8/836: symlink d4/d6/d3f/dde/l125 0 2021-11-03T17:15:34.397 INFO:tasks.workunit.client.1.smithi115.stdout:7/842: write d3/da/f62 [365611,82619] 0 2021-11-03T17:15:34.397 INFO:tasks.workunit.client.1.smithi115.stdout:1/899: readlink d7/dc/d101/d43/df4/l89 0 2021-11-03T17:15:34.402 INFO:tasks.workunit.client.1.smithi115.stdout:8/837: creat d4/d6/d3f/da4/dfd/f126 x:0 0 0 2021-11-03T17:15:34.403 INFO:tasks.workunit.client.1.smithi115.stdout:8/838: mknod d4/d6/d3f/dde/c127 1 2021-11-03T17:15:34.406 INFO:tasks.workunit.client.1.smithi115.stdout:8/839: fdatasync d4/d6/d3f/d9a/d11b/dcd/ff7 0 2021-11-03T17:15:34.406 INFO:tasks.workunit.client.1.smithi115.stdout:8/840: dread - d4/d6/d3f/da4/da6/fc6 zero size 2021-11-03T17:15:34.407 INFO:tasks.workunit.client.1.smithi115.stdout:8/841: write d4/d6/d3f/d43/d88/fec [443737,118258] 0 2021-11-03T17:15:34.407 INFO:tasks.workunit.client.1.smithi115.stdout:7/843: creat d3/da/d18/d73/d35/df0/ffe x:0 0 0 2021-11-03T17:15:34.408 INFO:tasks.workunit.client.1.smithi115.stdout:7/844: write d3/da/d18/d9c/da1/fb1 [3429276,95417] 0 2021-11-03T17:15:34.457 INFO:tasks.workunit.client.1.smithi115.stdout:7/845: mkdir d3/da/d78/dff 0 2021-11-03T17:15:34.463 INFO:tasks.workunit.client.1.smithi115.stdout:9/840: sync 2021-11-03T17:15:34.465 INFO:tasks.workunit.client.1.smithi115.stdout:7/846: readlink d3/da/l7d 0 2021-11-03T17:15:34.475 INFO:tasks.workunit.client.1.smithi115.stdout:7/847: rename d3/da/d18/d73/d35/f9d to d3/da/f100 0 2021-11-03T17:15:34.476 INFO:tasks.workunit.client.1.smithi115.stdout:7/848: chown d3/da/d18/d73/dda/f9b 16 1 2021-11-03T17:15:34.480 INFO:tasks.workunit.client.1.smithi115.stdout:7/849: rmdir d3/da/d18/d9c 39 2021-11-03T17:15:34.480 INFO:tasks.workunit.client.1.smithi115.stdout:7/850: readlink d3/da/d18/d73/d52/df1/lfd 0 2021-11-03T17:15:34.485 INFO:tasks.workunit.client.1.smithi115.stdout:7/851: mkdir d3/d4c/da6/dc7/d101 0 2021-11-03T17:15:34.485 INFO:tasks.workunit.client.1.smithi115.stdout:7/852: write d3/da/d18/d9c/fd8 [4560069,115472] 0 2021-11-03T17:15:34.487 INFO:tasks.workunit.client.1.smithi115.stdout:7/853: write d3/fd5 [3119711,44511] 0 2021-11-03T17:15:34.487 INFO:tasks.workunit.client.1.smithi115.stdout:7/854: fsync d3/da/d18/d73/d35/dbb/fd9 0 2021-11-03T17:15:34.488 INFO:tasks.workunit.client.1.smithi115.stdout:7/855: chown d3/da/d18/ff9 3 1 2021-11-03T17:15:34.488 INFO:tasks.workunit.client.1.smithi115.stdout:7/856: write d3/da/d18/d73/d35/f3f [4436703,23503] 0 2021-11-03T17:15:34.497 INFO:tasks.workunit.client.1.smithi115.stdout:7/857: mkdir d3/da/d18/d73/d35/da3/d102 0 2021-11-03T17:15:34.500 INFO:tasks.workunit.client.1.smithi115.stdout:7/858: mknod d3/da/c103 1 2021-11-03T17:15:34.513 INFO:tasks.workunit.client.1.smithi115.stdout:7/859: dread d3/da/d18/d73/d35/f5f [4194304,4194304] 0 2021-11-03T17:15:34.516 INFO:tasks.workunit.client.1.smithi115.stdout:7/860: dread d3/da/f1b [0,4194304] 0 2021-11-03T17:15:34.516 INFO:tasks.workunit.client.1.smithi115.stdout:7/861: chown d3/d4c/d7b/dcf/fe0 9333697 1 2021-11-03T17:15:34.523 INFO:tasks.workunit.client.1.smithi115.stdout:7/862: getdents d3/da/d18/d9c/da1/ded 0 2021-11-03T17:15:34.530 INFO:tasks.workunit.client.1.smithi115.stdout:7/863: rmdir d3/da/d18/d73/d35/da3/d102 0 2021-11-03T17:15:34.537 INFO:tasks.workunit.client.1.smithi115.stdout:7/864: truncate d3/f98 138335 0 2021-11-03T17:15:34.541 INFO:tasks.workunit.client.1.smithi115.stdout:7/865: rename d3/da/d18/d73/d52/lca to d3/da/d18/d73/d35/l104 0 2021-11-03T17:15:34.547 INFO:tasks.workunit.client.1.smithi115.stdout:7/866: creat d3/da/f105 x:0 0 0 2021-11-03T17:15:34.547 INFO:tasks.workunit.client.1.smithi115.stdout:7/867: write d3/da/d18/d73/dc8/ff2 [498848,22564] 0 2021-11-03T17:15:34.548 INFO:tasks.workunit.client.1.smithi115.stdout:7/868: chown d3/da/d18/d73/d35/d53/fe9 498110273 1 2021-11-03T17:15:34.548 INFO:tasks.workunit.client.1.smithi115.stdout:7/869: chown d3/da/d18/d73/d35/da3/fa8 6 1 2021-11-03T17:15:34.558 INFO:tasks.workunit.client.1.smithi115.stdout:7/870: rename d3/da/d18/d73/dda/l6b to d3/da/d18/d73/d35/d53/dde/l106 0 2021-11-03T17:15:34.558 INFO:tasks.workunit.client.1.smithi115.stdout:7/871: fsync d3/d4c/d7b/dd2/fd7 0 2021-11-03T17:15:34.561 INFO:tasks.workunit.client.1.smithi115.stdout:7/872: symlink d3/da/d78/l107 0 2021-11-03T17:15:34.563 INFO:tasks.workunit.client.1.smithi115.stdout:7/873: mkdir d3/d4c/d108 0 2021-11-03T17:15:34.564 INFO:tasks.workunit.client.1.smithi115.stdout:7/874: write d3/da/d18/d73/d35/d53/fc1 [1207410,128918] 0 2021-11-03T17:15:34.573 INFO:tasks.workunit.client.1.smithi115.stdout:7/875: dread d3/da/d8e/f99 [0,4194304] 0 2021-11-03T17:15:34.576 INFO:tasks.workunit.client.1.smithi115.stdout:7/876: dread d3/da/f100 [0,4194304] 0 2021-11-03T17:15:34.580 INFO:tasks.workunit.client.1.smithi115.stdout:7/877: symlink d3/da/l109 0 2021-11-03T17:15:34.583 INFO:tasks.workunit.client.1.smithi115.stdout:7/878: mkdir d3/d10a 0 2021-11-03T17:15:34.583 INFO:tasks.workunit.client.1.smithi115.stdout:7/879: stat d3/da/d78/lf4 0 2021-11-03T17:15:34.585 INFO:tasks.workunit.client.1.smithi115.stdout:7/880: mkdir d3/d10a/d10b 0 2021-11-03T17:15:34.618 INFO:tasks.workunit.client.1.smithi115.stdout:7/881: dwrite d3/f97 [4194304,4194304] 0 2021-11-03T17:15:34.619 INFO:tasks.workunit.client.1.smithi115.stdout:7/882: chown d3/da/d18 1965527 1 2021-11-03T17:15:34.619 INFO:tasks.workunit.client.1.smithi115.stdout:7/883: truncate d3/da/d18/d73/dda/f9b 1033810 0 2021-11-03T17:15:34.623 INFO:tasks.workunit.client.1.smithi115.stdout:7/884: mknod d3/da/c10c 1 2021-11-03T17:15:34.629 INFO:tasks.workunit.client.1.smithi115.stdout:7/885: mkdir d3/d4c/da6/d10d 0 2021-11-03T17:15:34.629 INFO:tasks.workunit.client.1.smithi115.stdout:7/886: mknod d3/da/d18/d73/d52/db4/c10e 1 2021-11-03T17:15:34.630 INFO:tasks.workunit.client.1.smithi115.stdout:6/768: sync 2021-11-03T17:15:34.630 INFO:tasks.workunit.client.1.smithi115.stdout:7/887: write d3/f27 [1693139,50109] 0 2021-11-03T17:15:34.638 INFO:tasks.workunit.client.1.smithi115.stdout:7/888: mknod d3/da/d78/c10f 1 2021-11-03T17:15:34.639 INFO:tasks.workunit.client.1.smithi115.stdout:6/769: dread d0/f22 [0,4194304] 0 2021-11-03T17:15:34.641 INFO:tasks.workunit.client.1.smithi115.stdout:7/889: mkdir d3/d4c/d7b/dd2/d110 0 2021-11-03T17:15:34.649 INFO:tasks.workunit.client.1.smithi115.stdout:6/770: unlink d0/d1/d2/lc4 0 2021-11-03T17:15:34.650 INFO:tasks.workunit.client.1.smithi115.stdout:6/771: readlink d0/d1/d10/d13/d1c/d1e/l75 0 2021-11-03T17:15:34.650 INFO:tasks.workunit.client.1.smithi115.stdout:7/890: creat d3/da/f111 x:0 0 0 2021-11-03T17:15:34.654 INFO:tasks.workunit.client.1.smithi115.stdout:6/772: mkdir d0/d7f/dab/dc6/d102 0 2021-11-03T17:15:34.673 INFO:tasks.workunit.client.1.smithi115.stdout:6/773: write d0/d1/d2/d3f/deb/d4c/d96/f6b [1635598,45781] 0 2021-11-03T17:15:34.675 INFO:tasks.workunit.client.1.smithi115.stdout:6/774: symlink d0/d1/d2/df3/l103 0 2021-11-03T17:15:34.677 INFO:tasks.workunit.client.1.smithi115.stdout:6/775: dread d0/d1/d2/d3f/deb/d4c/fb6 [0,4194304] 0 2021-11-03T17:15:34.677 INFO:tasks.workunit.client.1.smithi115.stdout:6/776: stat d0/d1/f19 0 2021-11-03T17:15:34.677 INFO:tasks.workunit.client.1.smithi115.stdout:6/777: chown d0/d1/d2/d3f/deb 909 1 2021-11-03T17:15:34.678 INFO:tasks.workunit.client.1.smithi115.stdout:6/778: write d0/d1/d2/fd [460328,79091] 0 2021-11-03T17:15:34.682 INFO:tasks.workunit.client.1.smithi115.stdout:6/779: symlink d0/d1/d2/d3f/deb/d4c/l104 0 2021-11-03T17:15:34.682 INFO:tasks.workunit.client.1.smithi115.stdout:6/780: fdatasync d0/d1/f19 0 2021-11-03T17:15:34.701 INFO:tasks.workunit.client.1.smithi115.stdout:6/781: dwrite d0/d1/d6a/d88/da0/db5/fe9 [0,4194304] 0 2021-11-03T17:15:34.702 INFO:tasks.workunit.client.1.smithi115.stdout:6/782: write d0/d1/d6a/d88/da0/db5/fc3 [29289,28637] 0 2021-11-03T17:15:34.705 INFO:tasks.workunit.client.1.smithi115.stdout:6/783: dread - d0/d1/d10/d13/d1c/d1e/de6/fe8 zero size 2021-11-03T17:15:34.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:34 smithi115 conmon[34670]: cluster 2021-11-03T17:15:32.648127+0000 mgr.smithi027.ufhhvb (mgr.14622) 34 : 2021-11-03T17:15:34.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:34 smithi115 conmon[34670]: cluster [DBG] pgmap v20: 65 pgs: 65 active+clean; 2.1 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 40 MiB/s rd, 76 MiB/s wr, 313 op/s 2021-11-03T17:15:34.739 INFO:tasks.workunit.client.1.smithi115.stdout:2/821: sync 2021-11-03T17:15:34.743 INFO:tasks.workunit.client.1.smithi115.stdout:2/822: rename d4/d6/dad/de8/dac/dcd to d4/d6/d56/d6c/d34/d42/d8d/dc3/d117 0 2021-11-03T17:15:34.746 INFO:tasks.workunit.client.1.smithi115.stdout:2/823: rmdir d4/d6/d56/d6c/d34/d43/d63/d77 39 2021-11-03T17:15:34.752 INFO:tasks.workunit.client.1.smithi115.stdout:2/824: rename d4/d6/dad/de8/d9c/dbe/l101 to d4/d6/d13/d108/l118 0 2021-11-03T17:15:34.753 INFO:tasks.workunit.client.1.smithi115.stdout:2/825: chown d4/d6/dad/d7f/feb 1924 1 2021-11-03T17:15:34.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:34 smithi027 conmon[32322]: cluster 2021-11-03T17:15:32.648127+0000 2021-11-03T17:15:34.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:34 smithi027 conmon[32322]: mgr.smithi027.ufhhvb (mgr.14622) 34 : cluster [DBG] pgmap v20: 65 pgs: 65 active+clean; 2.1 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 40 MiB/s rd, 76 MiB/s wr, 313 op/s 2021-11-03T17:15:34.781 INFO:tasks.workunit.client.1.smithi115.stdout:2/826: dwrite d4/d6/d56/d6c/d34/d43/d63/d77/fde [8388608,4194304] 0 2021-11-03T17:15:34.790 INFO:tasks.workunit.client.1.smithi115.stdout:2/827: dread d4/d6/f7e [4194304,4194304] 0 2021-11-03T17:15:34.791 INFO:tasks.workunit.client.1.smithi115.stdout:2/828: dread d4/d6/dad/d7f/fc0 [0,4194304] 0 2021-11-03T17:15:34.798 INFO:tasks.workunit.client.1.smithi115.stdout:5/989: sync 2021-11-03T17:15:34.803 INFO:tasks.workunit.client.1.smithi115.stdout:2/829: write d4/f78 [2381105,72292] 0 2021-11-03T17:15:34.807 INFO:tasks.workunit.client.1.smithi115.stdout:5/990: mkdir d1a/d3d/d14d 0 2021-11-03T17:15:34.807 INFO:tasks.workunit.client.1.smithi115.stdout:5/991: chown d1a/d28/d4c/d75/d83/da1 39864 1 2021-11-03T17:15:34.809 INFO:tasks.workunit.client.1.smithi115.stdout:2/830: mkdir d4/d6/d56/d6c/d34/d119 0 2021-11-03T17:15:34.814 INFO:tasks.workunit.client.1.smithi115.stdout:5/992: symlink d1a/d28/d4a/d65/db5/l14e 0 2021-11-03T17:15:34.817 INFO:tasks.workunit.client.1.smithi115.stdout:2/831: link d4/d6/dad/d7f/l110 d4/d6/d56/d6c/d34/d84/l11a 0 2021-11-03T17:15:34.820 INFO:tasks.workunit.client.1.smithi115.stdout:5/993: link d1a/d3d/dc0/ddf/ff3 d1a/d28/df9/d122/f14f 0 2021-11-03T17:15:34.822 INFO:tasks.workunit.client.1.smithi115.stdout:5/994: dread - d1a/d23/d138/f145 zero size 2021-11-03T17:15:34.824 INFO:tasks.workunit.client.1.smithi115.stdout:2/832: mkdir d4/d6/d56/d6c/d11b 0 2021-11-03T17:15:34.825 INFO:tasks.workunit.client.1.smithi115.stdout:5/995: mknod d1a/d28/d34/c150 1 2021-11-03T17:15:34.826 INFO:tasks.workunit.client.1.smithi115.stdout:5/996: readlink d1a/d28/d4c/d75/dae/lcc 0 2021-11-03T17:15:34.826 INFO:tasks.workunit.client.1.smithi115.stdout:5/997: chown d1a/d3d/dc0/f130 708248442 1 2021-11-03T17:15:34.827 INFO:tasks.workunit.client.1.smithi115.stdout:2/833: read d4/d6/d56/d6c/d34/d43/d63/d77/fde [12586244,57854] 0 2021-11-03T17:15:34.828 INFO:tasks.workunit.client.1.smithi115.stdout:5/998: unlink d1a/d28/d4a/f8b 0 2021-11-03T17:15:34.828 INFO:tasks.workunit.client.1.smithi115.stdout:5/999: fsync d1a/d28/fec 0 2021-11-03T17:15:35.207 INFO:tasks.workunit.client.1.smithi115.stdout:9/841: sync 2021-11-03T17:15:35.207 INFO:tasks.workunit.client.1.smithi115.stdout:8/842: sync 2021-11-03T17:15:35.207 INFO:tasks.workunit.client.1.smithi115.stdout:1/900: sync 2021-11-03T17:15:35.208 INFO:tasks.workunit.client.1.smithi115.stdout:8/843: fdatasync d4/d6/d18/d30/d33/d93/f108 0 2021-11-03T17:15:35.208 INFO:tasks.workunit.client.1.smithi115.stdout:1/901: fsync d7/dc/d101/f2b 0 2021-11-03T17:15:35.208 INFO:tasks.workunit.client.1.smithi115.stdout:8/844: dread - d4/d6/d3f/da4/da6/fc9 zero size 2021-11-03T17:15:35.209 INFO:tasks.workunit.client.1.smithi115.stdout:9/842: write d1/d18/d4a/d74/f84 [655579,50160] 0 2021-11-03T17:15:35.210 INFO:tasks.workunit.client.1.smithi115.stdout:8/845: rename d4/d6/f2b to d4/d6/d3f/d9a/d11b/d6e/f128 0 2021-11-03T17:15:35.212 INFO:tasks.workunit.client.1.smithi115.stdout:9/843: creat d1/d18/d2b/d45/f132 x:0 0 0 2021-11-03T17:15:35.213 INFO:tasks.workunit.client.1.smithi115.stdout:9/844: mknod d1/d1a/dd9/d126/d12e/c133 1 2021-11-03T17:15:35.213 INFO:tasks.workunit.client.1.smithi115.stdout:1/902: dread d7/d38/d84/f9a [0,4194304] 0 2021-11-03T17:15:35.215 INFO:tasks.workunit.client.1.smithi115.stdout:8/846: symlink d4/d6/d3f/d9a/d11b/dcd/l129 0 2021-11-03T17:15:35.216 INFO:tasks.workunit.client.1.smithi115.stdout:9/845: creat d1/d18/d4a/d74/f134 x:0 0 0 2021-11-03T17:15:35.222 INFO:tasks.workunit.client.1.smithi115.stdout:1/903: truncate d7/dc/d101/d4b/f72 2384615 0 2021-11-03T17:15:35.224 INFO:tasks.workunit.client.1.smithi115.stdout:8/847: symlink d4/d6/d3f/da4/da6/l12a 0 2021-11-03T17:15:35.225 INFO:tasks.workunit.client.1.smithi115.stdout:9/846: symlink d1/d18/d2b/d42/l135 0 2021-11-03T17:15:35.225 INFO:tasks.workunit.client.1.smithi115.stdout:9/847: truncate d1/d18/d2b/d42/fb7 895967 0 2021-11-03T17:15:35.229 INFO:tasks.workunit.client.1.smithi115.stdout:1/904: fsync d7/dc/d101/d43/df4/d60/d8f/fae 0 2021-11-03T17:15:35.232 INFO:tasks.workunit.client.1.smithi115.stdout:8/848: dread d4/d6/d18/d9d/fba [0,4194304] 0 2021-11-03T17:15:35.235 INFO:tasks.workunit.client.1.smithi115.stdout:9/848: unlink d1/d1a/d48/d27/d2f/d7b/de6/lf0 0 2021-11-03T17:15:35.239 INFO:tasks.workunit.client.1.smithi115.stdout:8/849: dread d4/d6/d18/d30/d33/f5c [0,4194304] 0 2021-11-03T17:15:35.240 INFO:tasks.workunit.client.1.smithi115.stdout:8/850: write d4/d6/d18/d30/d79/d7f/d11a/df9/f104 [4032366,49902] 0 2021-11-03T17:15:35.240 INFO:tasks.workunit.client.1.smithi115.stdout:8/851: fdatasync d4/d6/d3f/d9a/d11b/dcd/ff7 0 2021-11-03T17:15:35.241 INFO:tasks.workunit.client.1.smithi115.stdout:8/852: write d4/d6/d18/d30/d79/ffe [322263,90664] 0 2021-11-03T17:15:35.253 INFO:tasks.workunit.client.1.smithi115.stdout:9/849: stat d1/d18/d4a/d74/l127 0 2021-11-03T17:15:35.255 INFO:tasks.workunit.client.1.smithi115.stdout:1/905: truncate d7/dc/d101/d43/df4/d60/d8f/fa7 224743 0 2021-11-03T17:15:35.258 INFO:tasks.workunit.client.1.smithi115.stdout:8/853: symlink d4/d6/d18/d30/d79/d7f/d11a/l12b 0 2021-11-03T17:15:35.265 INFO:tasks.workunit.client.1.smithi115.stdout:8/854: dread d4/d6/f2a [0,4194304] 0 2021-11-03T17:15:35.266 INFO:tasks.workunit.client.1.smithi115.stdout:9/850: rename d1/f25 to d1/d18/d4a/d74/d8b/dbc/d12b/f136 0 2021-11-03T17:15:35.267 INFO:tasks.workunit.client.1.smithi115.stdout:9/851: dread d1/d18/d2b/db9/fed [0,4194304] 0 2021-11-03T17:15:35.269 INFO:tasks.workunit.client.1.smithi115.stdout:1/906: creat d7/dc/dab/f11c x:0 0 0 2021-11-03T17:15:35.270 INFO:tasks.workunit.client.1.smithi115.stdout:1/907: write d7/dc/dd/d3e/fa1 [454984,90712] 0 2021-11-03T17:15:35.273 INFO:tasks.workunit.client.1.smithi115.stdout:8/855: getdents d4/d6/d3f/d9a/d11b/dcd/dd0 0 2021-11-03T17:15:35.280 INFO:tasks.workunit.client.1.smithi115.stdout:1/908: creat d7/d38/f11d x:0 0 0 2021-11-03T17:15:35.283 INFO:tasks.workunit.client.1.smithi115.stdout:8/856: chown d4/d7d/fe6 3 1 2021-11-03T17:15:35.283 INFO:tasks.workunit.client.1.smithi115.stdout:8/857: fsync d4/d6/d3f/da4/da6/fc7 0 2021-11-03T17:15:35.289 INFO:tasks.workunit.client.1.smithi115.stdout:9/852: rename d1/d1a/d48/d27/d80/dcc/f117 to d1/d1a/d48/d27/d80/f137 0 2021-11-03T17:15:35.294 INFO:tasks.workunit.client.1.smithi115.stdout:8/858: link d4/d6/d18/d9d/fba d4/d6/d3f/d9a/d11b/d6e/f12c 0 2021-11-03T17:15:35.297 INFO:tasks.workunit.client.1.smithi115.stdout:9/853: mknod d1/d1a/d48/d27/d80/c138 1 2021-11-03T17:15:35.298 INFO:tasks.workunit.client.1.smithi115.stdout:9/854: dread - d1/d1a/d48/d27/d2f/d6a/f120 zero size 2021-11-03T17:15:35.298 INFO:tasks.workunit.client.1.smithi115.stdout:9/855: fdatasync d1/d18/d2b/d45/f132 0 2021-11-03T17:15:35.309 INFO:tasks.workunit.client.1.smithi115.stdout:8/859: creat d4/d6/d18/d9d/f12d x:0 0 0 2021-11-03T17:15:35.309 INFO:tasks.workunit.client.1.smithi115.stdout:8/860: chown d4/d7d/fdd 13620 1 2021-11-03T17:15:35.310 INFO:tasks.workunit.client.1.smithi115.stdout:8/861: fsync d4/d6/d18/d30/d79/d7f/f10f 0 2021-11-03T17:15:35.310 INFO:tasks.workunit.client.1.smithi115.stdout:8/862: write d4/d6/d3f/d43/f111 [87356,74303] 0 2021-11-03T17:15:35.311 INFO:tasks.workunit.client.1.smithi115.stdout:8/863: readlink d4/d6/d3f/d9a/d11b/l105 0 2021-11-03T17:15:35.311 INFO:tasks.workunit.client.1.smithi115.stdout:8/864: write d4/d7d/fdd [227063,15400] 0 2021-11-03T17:15:35.314 INFO:tasks.workunit.client.1.smithi115.stdout:9/856: creat d1/d1a/d48/d27/d80/dcc/f139 x:0 0 0 2021-11-03T17:15:35.314 INFO:tasks.workunit.client.1.smithi115.stdout:9/857: write d1/d8a/f12d [68553,116930] 0 2021-11-03T17:15:35.318 INFO:tasks.workunit.client.1.smithi115.stdout:9/858: creat d1/d1a/d48/d27/d80/dcc/f13a x:0 0 0 2021-11-03T17:15:35.320 INFO:tasks.workunit.client.1.smithi115.stdout:9/859: creat d1/d1a/d48/d27/d2f/d41/f13b x:0 0 0 2021-11-03T17:15:35.321 INFO:tasks.workunit.client.1.smithi115.stdout:9/860: write d1/d18/d2b/d42/d63/f9f [1334996,91278] 0 2021-11-03T17:15:35.321 INFO:tasks.workunit.client.1.smithi115.stdout:9/861: dread - d1/d1a/d48/d27/d2f/d7b/fef zero size 2021-11-03T17:15:35.322 INFO:tasks.workunit.client.1.smithi115.stdout:9/862: truncate d1/d66/fd1 2165803 0 2021-11-03T17:15:35.323 INFO:tasks.workunit.client.1.smithi115.stdout:8/865: dread d4/d6/d18/d30/f36 [0,4194304] 0 2021-11-03T17:15:35.325 INFO:tasks.workunit.client.1.smithi115.stdout:8/866: symlink d4/d6/d18/d30/d79/l12e 0 2021-11-03T17:15:35.326 INFO:tasks.workunit.client.1.smithi115.stdout:9/863: dread d1/d1a/d48/d27/d2f/fc6 [0,4194304] 0 2021-11-03T17:15:35.326 INFO:tasks.workunit.client.1.smithi115.stdout:9/864: truncate d1/d18/d2b/db9/fdb 981735 0 2021-11-03T17:15:35.327 INFO:tasks.workunit.client.1.smithi115.stdout:9/865: write d1/d18/d2b/d42/f12 [4013462,75053] 0 2021-11-03T17:15:35.329 INFO:tasks.workunit.client.1.smithi115.stdout:8/867: write d4/d6/f6b [2634046,77739] 0 2021-11-03T17:15:35.329 INFO:tasks.workunit.client.1.smithi115.stdout:1/909: dwrite d7/dc/f87 [0,4194304] 0 2021-11-03T17:15:35.330 INFO:tasks.workunit.client.1.smithi115.stdout:1/910: chown d7/dc/d101 7109 1 2021-11-03T17:15:35.331 INFO:tasks.workunit.client.1.smithi115.stdout:8/868: read d4/d6/d18/d30/d79/d7f/ff1 [3577094,15979] 0 2021-11-03T17:15:35.331 INFO:tasks.workunit.client.1.smithi115.stdout:6/784: sync 2021-11-03T17:15:35.332 INFO:tasks.workunit.client.1.smithi115.stdout:7/891: sync 2021-11-03T17:15:35.332 INFO:tasks.workunit.client.1.smithi115.stdout:6/785: chown d0/d1/f9e 416 1 2021-11-03T17:15:35.333 INFO:tasks.workunit.client.1.smithi115.stdout:7/892: chown d3/da/d78/l107 13845501 1 2021-11-03T17:15:35.333 INFO:tasks.workunit.client.1.smithi115.stdout:6/786: chown d0/d1/d2/d3f/deb/fb9 1541332 1 2021-11-03T17:15:35.333 INFO:tasks.workunit.client.1.smithi115.stdout:6/787: stat d0/d1/d2/f35 0 2021-11-03T17:15:35.333 INFO:tasks.workunit.client.1.smithi115.stdout:7/893: fsync d3/da/d78/fbe 0 2021-11-03T17:15:35.334 INFO:tasks.workunit.client.1.smithi115.stdout:6/788: chown d0/d1/d2/df3 0 1 2021-11-03T17:15:35.340 INFO:tasks.workunit.client.1.smithi115.stdout:7/894: symlink d3/da/d18/d73/d35/d53/dde/l112 0 2021-11-03T17:15:35.344 INFO:tasks.workunit.client.1.smithi115.stdout:7/895: read d3/da/d18/d9c/fd8 [4115045,125066] 0 2021-11-03T17:15:35.346 INFO:tasks.workunit.client.1.smithi115.stdout:9/866: dwrite d1/d18/d4a/d74/f7e [0,4194304] 0 2021-11-03T17:15:35.346 INFO:tasks.workunit.client.1.smithi115.stdout:1/911: rename d7/d38/d84/dca/fe1 to d7/dc/dd/d3e/dbe/f11e 0 2021-11-03T17:15:35.357 INFO:tasks.workunit.client.1.smithi115.stdout:7/896: creat d3/da/d18/d73/d35/da3/f113 x:0 0 0 2021-11-03T17:15:35.358 INFO:tasks.workunit.client.1.smithi115.stdout:7/897: fsync d3/d4c/da6/fdb 0 2021-11-03T17:15:35.379 INFO:tasks.workunit.client.1.smithi115.stdout:8/869: rename d4/d6/l2e to d4/d6/d3f/dde/l12f 0 2021-11-03T17:15:35.381 INFO:tasks.workunit.client.1.smithi115.stdout:7/898: dwrite d3/da/d18/d73/d35/f3d [4194304,4194304] 0 2021-11-03T17:15:35.381 INFO:tasks.workunit.client.1.smithi115.stdout:7/899: readlink d3/lc5 0 2021-11-03T17:15:35.390 INFO:tasks.workunit.client.1.smithi115.stdout:9/867: symlink d1/d18/d2b/d42/d63/l13c 0 2021-11-03T17:15:35.397 INFO:tasks.workunit.client.1.smithi115.stdout:9/868: truncate d1/d1a/d48/f2a 4097273 0 2021-11-03T17:15:35.399 INFO:tasks.workunit.client.1.smithi115.stdout:7/900: dwrite d3/f9 [0,4194304] 0 2021-11-03T17:15:35.400 INFO:tasks.workunit.client.1.smithi115.stdout:7/901: stat d3/da/d18/d73/d52/fa9 0 2021-11-03T17:15:35.401 INFO:tasks.workunit.client.1.smithi115.stdout:8/870: unlink d4/d6/d18/d30/d33/l58 0 2021-11-03T17:15:35.402 INFO:tasks.workunit.client.1.smithi115.stdout:1/912: link d7/d38/d84/lb7 d7/dc/dd/d3e/d58/l11f 0 2021-11-03T17:15:35.404 INFO:tasks.workunit.client.1.smithi115.stdout:7/902: symlink d3/d4c/d7b/dd2/l114 0 2021-11-03T17:15:35.407 INFO:tasks.workunit.client.1.smithi115.stdout:7/903: dread d3/da/d18/fce [0,4194304] 0 2021-11-03T17:15:35.407 INFO:tasks.workunit.client.1.smithi115.stdout:7/904: truncate d3/da/d18/d73/dda/f59 1420476 0 2021-11-03T17:15:35.407 INFO:tasks.workunit.client.1.smithi115.stdout:7/905: readlink d3/l10 0 2021-11-03T17:15:35.413 INFO:tasks.workunit.client.1.smithi115.stdout:8/871: creat d4/d6/d3f/d9a/d11b/dcd/dd0/f130 x:0 0 0 2021-11-03T17:15:35.414 INFO:tasks.workunit.client.1.smithi115.stdout:8/872: stat d4/d6/d18/d30/d79/d7f/ff1 0 2021-11-03T17:15:35.414 INFO:tasks.workunit.client.1.smithi115.stdout:8/873: dread - d4/d6/d3f/da4/da6/fc6 zero size 2021-11-03T17:15:35.414 INFO:tasks.workunit.client.1.smithi115.stdout:9/869: symlink d1/d66/l13d 0 2021-11-03T17:15:35.424 INFO:tasks.workunit.client.1.smithi115.stdout:8/874: symlink d4/d6/d3f/d43/d88/l131 0 2021-11-03T17:15:35.428 INFO:tasks.workunit.client.1.smithi115.stdout:8/875: creat d4/d6/d18/d30/d33/d93/f132 x:0 0 0 2021-11-03T17:15:35.432 INFO:tasks.workunit.client.1.smithi115.stdout:8/876: mkdir d4/d6/d18/d133 0 2021-11-03T17:15:35.432 INFO:tasks.workunit.client.1.smithi115.stdout:8/877: chown d4/d6/d3f/d9a/d11b/d6e/f9f 18164513 1 2021-11-03T17:15:35.438 INFO:tasks.workunit.client.1.smithi115.stdout:1/913: dwrite d7/d38/d59/f9b [0,4194304] 0 2021-11-03T17:15:35.438 INFO:tasks.workunit.client.1.smithi115.stdout:1/914: write d7/dc/dab/d73/f10b [4247266,108335] 0 2021-11-03T17:15:35.439 INFO:tasks.workunit.client.1.smithi115.stdout:7/906: dwrite d3/da/d18/d73/d35/fad [0,4194304] 0 2021-11-03T17:15:35.440 INFO:tasks.workunit.client.1.smithi115.stdout:1/915: chown d7/d38/d84/fd0 6262 1 2021-11-03T17:15:35.446 INFO:tasks.workunit.client.1.smithi115.stdout:7/907: rename d3/da/d18/dc0 to d3/d4c/d7b/dcf/d115 0 2021-11-03T17:15:35.458 INFO:tasks.workunit.client.1.smithi115.stdout:1/916: dwrite d7/dc/d101/d43/df4/d60/d8f/ddf/fe7 [0,4194304] 0 2021-11-03T17:15:35.465 INFO:tasks.workunit.client.1.smithi115.stdout:1/917: rmdir d7/d38/d59/d83 0 2021-11-03T17:15:35.471 INFO:tasks.workunit.client.1.smithi115.stdout:7/908: dwrite d3/da/d18/fce [0,4194304] 0 2021-11-03T17:15:35.472 INFO:tasks.workunit.client.1.smithi115.stdout:9/870: dwrite d1/d18/d2b/d42/d63/f91 [0,4194304] 0 2021-11-03T17:15:35.473 INFO:tasks.workunit.client.1.smithi115.stdout:1/918: mkdir d7/d38/d84/d120 0 2021-11-03T17:15:35.477 INFO:tasks.workunit.client.1.smithi115.stdout:7/909: symlink d3/da/d18/l116 0 2021-11-03T17:15:35.479 INFO:tasks.workunit.client.1.smithi115.stdout:1/919: chown d7/dc/d101/d26/l41 45 1 2021-11-03T17:15:35.479 INFO:tasks.workunit.client.1.smithi115.stdout:1/920: fsync d7/dc/dab/d73/d7f/f105 0 2021-11-03T17:15:35.490 INFO:tasks.workunit.client.1.smithi115.stdout:7/910: unlink d3/da/f111 0 2021-11-03T17:15:35.500 INFO:tasks.workunit.client.1.smithi115.stdout:1/921: mkdir d7/dc/dd/d3e/dbe/dff/d121 0 2021-11-03T17:15:35.500 INFO:tasks.workunit.client.1.smithi115.stdout:7/911: dread d3/df7/ffa [0,4194304] 0 2021-11-03T17:15:35.501 INFO:tasks.workunit.client.1.smithi115.stdout:7/912: dread d3/d4c/d7b/dd2/fdd [0,4194304] 0 2021-11-03T17:15:35.501 INFO:tasks.workunit.client.1.smithi115.stdout:7/913: fsync d3/da/d78/f9f 0 2021-11-03T17:15:35.502 INFO:tasks.workunit.client.1.smithi115.stdout:1/922: dread d7/dc/dd/d3e/ffc [0,4194304] 0 2021-11-03T17:15:35.502 INFO:tasks.workunit.client.1.smithi115.stdout:1/923: mknod d7/dc/dd/d3e/dbe/dff/d121/c122 1 2021-11-03T17:15:35.502 INFO:tasks.workunit.client.1.smithi115.stdout:1/924: dread - d7/f108 zero size 2021-11-03T17:15:35.504 INFO:tasks.workunit.client.1.smithi115.stdout:7/914: creat d3/d4c/d7b/dd2/d110/f117 x:0 0 0 2021-11-03T17:15:35.504 INFO:tasks.workunit.client.1.smithi115.stdout:7/915: fdatasync d3/da/d18/d73/d52/fa9 0 2021-11-03T17:15:35.506 INFO:tasks.workunit.client.1.smithi115.stdout:7/916: read d3/da/d18/d73/fc2 [681180,122511] 0 2021-11-03T17:15:35.506 INFO:tasks.workunit.client.1.smithi115.stdout:1/925: mkdir d7/d38/d84/dca/d103/d123 0 2021-11-03T17:15:35.506 INFO:tasks.workunit.client.1.smithi115.stdout:1/926: write d7/dc/dd/d3e/d58/dc2/fec [522475,123900] 0 2021-11-03T17:15:35.506 INFO:tasks.workunit.client.1.smithi115.stdout:2/834: sync 2021-11-03T17:15:35.507 INFO:tasks.workunit.client.1.smithi115.stdout:2/835: fdatasync d4/d6/d13/f25 0 2021-11-03T17:15:35.507 INFO:tasks.workunit.client.1.smithi115.stdout:1/927: write d7/dc/dd/d3e/ffc [124394,66830] 0 2021-11-03T17:15:35.511 INFO:tasks.workunit.client.1.smithi115.stdout:7/917: creat d3/da/d18/d73/dda/f118 x:0 0 0 2021-11-03T17:15:35.515 INFO:tasks.workunit.client.1.smithi115.stdout:2/836: symlink d4/d6/dad/dc2/l11c 0 2021-11-03T17:15:35.516 INFO:tasks.workunit.client.1.smithi115.stdout:1/928: creat d7/dc/d101/f124 x:0 0 0 2021-11-03T17:15:35.520 INFO:tasks.workunit.client.1.smithi115.stdout:7/918: symlink d3/da/d18/dfb/l119 0 2021-11-03T17:15:35.524 INFO:tasks.workunit.client.1.smithi115.stdout:2/837: creat d4/d6/dad/de8/d9c/f11d x:0 0 0 2021-11-03T17:15:35.524 INFO:tasks.workunit.client.1.smithi115.stdout:2/838: stat d4/d6/d13/d75/f10c 0 2021-11-03T17:15:35.524 INFO:tasks.workunit.client.1.smithi115.stdout:2/839: mknod d4/d6/d56/d6c/d34/d119/c11e 1 2021-11-03T17:15:35.524 INFO:tasks.workunit.client.1.smithi115.stdout:2/840: truncate d4/d6/d56/d6c/d34/d42/d8d/da9/fc4 651646 0 2021-11-03T17:15:35.527 INFO:tasks.workunit.client.1.smithi115.stdout:7/919: dread d3/da/d18/d73/d35/f3f [0,4194304] 0 2021-11-03T17:15:35.527 INFO:tasks.workunit.client.1.smithi115.stdout:1/929: rename d7/dc/dab to d7/d38/d59/d125 0 2021-11-03T17:15:35.540 INFO:tasks.workunit.client.1.smithi115.stdout:2/841: mkdir d4/d6/dad/d7f/d11f 0 2021-11-03T17:15:35.544 INFO:tasks.workunit.client.1.smithi115.stdout:2/842: symlink d4/d6/d56/d6c/d34/d119/l120 0 2021-11-03T17:15:35.544 INFO:tasks.workunit.client.1.smithi115.stdout:2/843: getdents d4/d6/d56/d6c/d34/d42/d8d/dd8/d104 0 2021-11-03T17:15:35.553 INFO:tasks.workunit.client.1.smithi115.stdout:2/844: truncate d4/d6/d13/d75/f7a 2765394 0 2021-11-03T17:15:35.554 INFO:tasks.workunit.client.1.smithi115.stdout:2/845: fsync d4/d6/d56/d6c/d34/ff2 0 2021-11-03T17:15:35.554 INFO:tasks.workunit.client.1.smithi115.stdout:7/920: dwrite d3/da/d18/d73/d35/df0/ffe [0,4194304] 0 2021-11-03T17:15:35.555 INFO:tasks.workunit.client.1.smithi115.stdout:7/921: getdents d3/d4c/d108 0 2021-11-03T17:15:35.555 INFO:tasks.workunit.client.1.smithi115.stdout:7/922: mknod d3/d4c/d7b/dd2/d110/c11a 1 2021-11-03T17:15:35.556 INFO:tasks.workunit.client.1.smithi115.stdout:1/930: dwrite d7/dc/dd/d3e/d58/dbf/fd8 [0,4194304] 0 2021-11-03T17:15:35.556 INFO:tasks.workunit.client.1.smithi115.stdout:1/931: truncate d7/f8b 4358872 0 2021-11-03T17:15:35.559 INFO:tasks.workunit.client.1.smithi115.stdout:2/846: mknod d4/d6/d56/d6c/d34/d43/d28/c121 1 2021-11-03T17:15:35.564 INFO:tasks.workunit.client.1.smithi115.stdout:7/923: creat d3/da/d18/d73/d35/df0/f11b x:0 0 0 2021-11-03T17:15:35.564 INFO:tasks.workunit.client.1.smithi115.stdout:7/924: truncate d3/f9 13586900 0 2021-11-03T17:15:35.582 INFO:tasks.workunit.client.1.smithi115.stdout:7/925: dwrite d3/da/d18/d73/fc2 [0,4194304] 0 2021-11-03T17:15:35.582 INFO:tasks.workunit.client.1.smithi115.stdout:7/926: mknod d3/d4c/da6/c11c 1 2021-11-03T17:15:35.582 INFO:tasks.workunit.client.1.smithi115.stdout:2/847: mknod d4/d6/dad/de8/dac/db4/c122 1 2021-11-03T17:15:35.583 INFO:tasks.workunit.client.1.smithi115.stdout:2/848: write d4/d6/dad/d7f/fc0 [1171702,36189] 0 2021-11-03T17:15:35.583 INFO:tasks.workunit.client.1.smithi115.stdout:2/849: readlink d4/d6/dad/de8/dac/lb3 0 2021-11-03T17:15:35.591 INFO:tasks.workunit.client.1.smithi115.stdout:7/927: symlink d3/da/d18/d9c/da1/l11d 0 2021-11-03T17:15:35.597 INFO:tasks.workunit.client.1.smithi115.stdout:2/850: rmdir d4/d6/d56/d6c/d34/d42/d8d/da9 39 2021-11-03T17:15:35.599 INFO:tasks.workunit.client.1.smithi115.stdout:1/932: dwrite d7/dc/d101/d43/df4/f7b [4194304,4194304] 0 2021-11-03T17:15:35.612 INFO:tasks.workunit.client.1.smithi115.stdout:2/851: truncate d4/d6/d13/fb2 698622 0 2021-11-03T17:15:35.619 INFO:tasks.workunit.client.1.smithi115.stdout:1/933: truncate d7/dc/d101/d43/f7e 1793736 0 2021-11-03T17:15:35.621 INFO:tasks.workunit.client.1.smithi115.stdout:1/934: write d7/f8a [757052,61626] 0 2021-11-03T17:15:35.622 INFO:tasks.workunit.client.1.smithi115.stdout:2/852: symlink d4/d6/dad/de8/dac/dec/l123 0 2021-11-03T17:15:35.633 INFO:tasks.workunit.client.1.smithi115.stdout:2/853: dread d4/d6/dad/d7f/f10b [0,4194304] 0 2021-11-03T17:15:35.635 INFO:tasks.workunit.client.1.smithi115.stdout:2/854: mknod d4/d6/d56/d6c/d34/d43/c124 1 2021-11-03T17:15:35.639 INFO:tasks.workunit.client.1.smithi115.stdout:1/935: dwrite d7/f8b [0,4194304] 0 2021-11-03T17:15:35.640 INFO:tasks.workunit.client.1.smithi115.stdout:2/855: rename d4/d6/dad/de8/dac/lc1 to d4/d6/dad/d7f/d11f/l125 0 2021-11-03T17:15:35.643 INFO:tasks.workunit.client.1.smithi115.stdout:2/856: rename d4/d6/dad/d7f/l111 to d4/d6/d13/d108/l126 0 2021-11-03T17:15:35.649 INFO:tasks.workunit.client.1.smithi115.stdout:1/936: dread d7/dc/dd/ff7 [0,4194304] 0 2021-11-03T17:15:35.651 INFO:tasks.workunit.client.1.smithi115.stdout:1/937: write d7/dc/dd/d3e/d58/ffd [2721651,57989] 0 2021-11-03T17:15:35.653 INFO:tasks.workunit.client.1.smithi115.stdout:2/857: dread d4/d6/f48 [0,4194304] 0 2021-11-03T17:15:35.653 INFO:tasks.workunit.client.1.smithi115.stdout:2/858: write d4/d6/d13/d75/f10c [169664,114548] 0 2021-11-03T17:15:35.654 INFO:tasks.workunit.client.1.smithi115.stdout:1/938: symlink d7/dc/d101/d43/l126 0 2021-11-03T17:15:35.655 INFO:tasks.workunit.client.1.smithi115.stdout:1/939: write d7/f8a [8262871,68376] 0 2021-11-03T17:15:35.655 INFO:tasks.workunit.client.1.smithi115.stdout:1/940: chown d7/f7c 1 1 2021-11-03T17:15:35.657 INFO:tasks.workunit.client.1.smithi115.stdout:2/859: unlink d4/d6/d56/d6c/d34/ff2 0 2021-11-03T17:15:35.657 INFO:tasks.workunit.client.1.smithi115.stdout:2/860: write d4/fca [1009369,129259] 0 2021-11-03T17:15:35.658 INFO:tasks.workunit.client.1.smithi115.stdout:2/861: fdatasync d4/d6/dad/d7f/fa5 0 2021-11-03T17:15:35.658 INFO:tasks.workunit.client.1.smithi115.stdout:1/941: readlink d7/dc/d1a/l22 0 2021-11-03T17:15:35.679 INFO:tasks.workunit.client.1.smithi115.stdout:2/862: rename d4/d6/d13/d108/l126 to d4/d6/d56/d6c/d34/d84/l127 0 2021-11-03T17:15:35.679 INFO:tasks.workunit.client.1.smithi115.stdout:2/863: fsync d4/d6/dad/d7f/f10b 0 2021-11-03T17:15:35.694 INFO:tasks.workunit.client.1.smithi115.stdout:2/864: truncate d4/d6/f7e 1166326 0 2021-11-03T17:15:35.694 INFO:tasks.workunit.client.1.smithi115.stdout:2/865: dread - d4/d6/d56/d6c/d34/d84/fe7 zero size 2021-11-03T17:15:35.710 INFO:tasks.workunit.client.1.smithi115.stdout:1/942: rename d7/d38/d59/f9b to d7/d38/d59/d125/d73/f127 0 2021-11-03T17:15:35.719 INFO:tasks.workunit.client.1.smithi115.stdout:1/943: getdents d7/d38/d84/dca/d103 0 2021-11-03T17:15:35.719 INFO:tasks.workunit.client.1.smithi115.stdout:1/944: chown d7/dc/d101/d43/df4/d60/f6d 15 1 2021-11-03T17:15:35.720 INFO:tasks.workunit.client.1.smithi115.stdout:2/866: symlink d4/d6/d56/d6c/d34/d43/d28/l128 0 2021-11-03T17:15:35.722 INFO:tasks.workunit.client.1.smithi115.stdout:1/945: creat d7/dc/d101/d4b/f128 x:0 0 0 2021-11-03T17:15:35.724 INFO:tasks.workunit.client.1.smithi115.stdout:2/867: mknod d4/d6/dad/de8/c129 1 2021-11-03T17:15:35.724 INFO:tasks.workunit.client.1.smithi115.stdout:1/946: mknod d7/d38/d59/d125/c129 1 2021-11-03T17:15:35.729 INFO:tasks.workunit.client.1.smithi115.stdout:2/868: truncate d4/d6/d56/d6c/d34/d43/d63/d77/f81 717083 0 2021-11-03T17:15:35.729 INFO:tasks.workunit.client.1.smithi115.stdout:2/869: write d4/d6/dad/d7f/fc0 [1517114,102046] 0 2021-11-03T17:15:35.731 INFO:tasks.workunit.client.1.smithi115.stdout:1/947: symlink d7/dc/d101/l12a 0 2021-11-03T17:15:35.733 INFO:tasks.workunit.client.1.smithi115.stdout:2/870: creat d4/d6/dad/de8/d9c/f12a x:0 0 0 2021-11-03T17:15:35.736 INFO:tasks.workunit.client.1.smithi115.stdout:1/948: rename d7/dc/ff to d7/dc/dd/d3e/dbe/dff/d121/f12b 0 2021-11-03T17:15:35.736 INFO:tasks.workunit.client.1.smithi115.stdout:1/949: write d7/dc/d101/d43/df4/f69 [1549008,48174] 0 2021-11-03T17:15:35.737 INFO:tasks.workunit.client.1.smithi115.stdout:1/950: dread - d7/d38/d59/d125/f11c zero size 2021-11-03T17:15:35.737 INFO:tasks.workunit.client.1.smithi115.stdout:1/951: dread d7/f7c [0,4194304] 0 2021-11-03T17:15:35.739 INFO:tasks.workunit.client.1.smithi115.stdout:2/871: read d4/d6/d56/f5d [452969,91026] 0 2021-11-03T17:15:35.739 INFO:tasks.workunit.client.1.smithi115.stdout:2/872: chown d4/d6/d56/d6c/l5e 86 1 2021-11-03T17:15:35.740 INFO:tasks.workunit.client.1.smithi115.stdout:1/952: fsync d7/dc/d101/d43/f74 0 2021-11-03T17:15:35.747 INFO:tasks.workunit.client.1.smithi115.stdout:1/953: creat d7/dc/dd/d3e/d58/f12c x:0 0 0 2021-11-03T17:15:35.747 INFO:tasks.workunit.client.1.smithi115.stdout:1/954: readlink d7/dc/dd/d3e/d58/l109 0 2021-11-03T17:15:35.748 INFO:tasks.workunit.client.1.smithi115.stdout:1/955: chown d7/dc/d1a/fd7 4684 1 2021-11-03T17:15:35.751 INFO:tasks.workunit.client.1.smithi115.stdout:2/873: rename d4/d6/d56/d6c/d34/d42/d8d/dc3/d117/l105 to d4/d6/dad/de8/l12b 0 2021-11-03T17:15:35.752 INFO:tasks.workunit.client.1.smithi115.stdout:1/956: mknod d7/d38/d84/c12d 1 2021-11-03T17:15:35.762 INFO:tasks.workunit.client.1.smithi115.stdout:1/957: write d7/dc/d101/d4b/d55/f97 [244876,128359] 0 2021-11-03T17:15:35.763 INFO:tasks.workunit.client.1.smithi115.stdout:1/958: fdatasync d7/f7c 0 2021-11-03T17:15:35.775 INFO:tasks.workunit.client.1.smithi115.stdout:2/874: dwrite d4/d6/d13/d75/f10c [0,4194304] 0 2021-11-03T17:15:35.787 INFO:tasks.workunit.client.1.smithi115.stdout:1/959: dwrite d7/d38/d59/d125/d73/f10b [0,4194304] 0 2021-11-03T17:15:35.802 INFO:tasks.workunit.client.1.smithi115.stdout:1/960: readlink d7/dc/d1a/l21 0 2021-11-03T17:15:35.812 INFO:tasks.workunit.client.1.smithi115.stdout:1/961: unlink d7/d38/d59/d125/f11c 0 2021-11-03T17:15:35.812 INFO:tasks.workunit.client.1.smithi115.stdout:1/962: read - d7/d38/d84/f11a zero size 2021-11-03T17:15:35.812 INFO:tasks.workunit.client.1.smithi115.stdout:1/963: mknod d7/d38/d84/d120/c12e 1 2021-11-03T17:15:35.813 INFO:tasks.workunit.client.1.smithi115.stdout:1/964: fsync d7/dc/d101/d43/df4/d60/db2/ff0 0 2021-11-03T17:15:35.821 INFO:tasks.workunit.client.1.smithi115.stdout:1/965: readlink d7/dc/d101/d43/l34 0 2021-11-03T17:15:35.827 INFO:tasks.workunit.client.1.smithi115.stdout:1/966: dread d7/dc/d101/d43/df4/d60/d8f/ddf/fe7 [0,4194304] 0 2021-11-03T17:15:35.828 INFO:tasks.workunit.client.1.smithi115.stdout:1/967: write d7/dc/d101/f124 [211851,28381] 0 2021-11-03T17:15:35.829 INFO:tasks.workunit.client.1.smithi115.stdout:1/968: dread - d7/dc/d101/d43/df4/d60/d8f/fce zero size 2021-11-03T17:15:35.830 INFO:tasks.workunit.client.1.smithi115.stdout:1/969: write d7/dc/d101/d43/df4/fee [1633795,111974] 0 2021-11-03T17:15:35.834 INFO:tasks.workunit.client.1.smithi115.stdout:1/970: getdents d7/dc/dd/d3e/d58/dbf 0 2021-11-03T17:15:35.844 INFO:tasks.workunit.client.1.smithi115.stdout:1/971: symlink d7/dc/dd/d3e/d58/dc2/dd2/l12f 0 2021-11-03T17:15:35.984 INFO:tasks.workunit.client.1.smithi115.stdout:6/789: sync 2021-11-03T17:15:35.985 INFO:tasks.workunit.client.1.smithi115.stdout:6/790: dread - d0/d1/d2/df3/ff1 zero size 2021-11-03T17:15:35.988 INFO:tasks.workunit.client.1.smithi115.stdout:6/791: dread d0/d1/fb [8388608,4194304] 0 2021-11-03T17:15:35.992 INFO:tasks.workunit.client.1.smithi115.stdout:6/792: mkdir d0/d7f/dab/dc6/d102/d105 0 2021-11-03T17:15:35.993 INFO:tasks.workunit.client.1.smithi115.stdout:6/793: fdatasync d0/d1/d2/d3f/f49 0 2021-11-03T17:15:35.995 INFO:tasks.workunit.client.1.smithi115.stdout:6/794: creat d0/d74/db2/f106 x:0 0 0 2021-11-03T17:15:36.008 INFO:tasks.workunit.client.1.smithi115.stdout:6/795: write d0/d1/d10/d13/d59/f9c [1585363,116927] 0 2021-11-03T17:15:36.011 INFO:tasks.workunit.client.1.smithi115.stdout:6/796: rename d0/d7f/dab/dc6 to d0/d1/d2/df3/d107 0 2021-11-03T17:15:36.025 INFO:tasks.workunit.client.1.smithi115.stdout:8/878: sync 2021-11-03T17:15:36.033 INFO:tasks.workunit.client.1.smithi115.stdout:6/797: dwrite d0/d1/d2/d3f/fd0 [0,4194304] 0 2021-11-03T17:15:36.037 INFO:tasks.workunit.client.1.smithi115.stdout:6/798: truncate d0/d1/d2/d3f/f4b 3495293 0 2021-11-03T17:15:36.048 INFO:tasks.workunit.client.1.smithi115.stdout:6/799: dread d0/f4d [0,4194304] 0 2021-11-03T17:15:36.054 INFO:tasks.workunit.client.1.smithi115.stdout:7/928: sync 2021-11-03T17:15:36.060 INFO:tasks.workunit.client.1.smithi115.stdout:7/929: symlink d3/d10a/d10b/l11e 0 2021-11-03T17:15:36.067 INFO:tasks.workunit.client.1.smithi115.stdout:6/800: dwrite d0/d7f/fcd [0,4194304] 0 2021-11-03T17:15:36.071 INFO:tasks.workunit.client.1.smithi115.stdout:7/930: dread d3/da/f90 [0,4194304] 0 2021-11-03T17:15:36.071 INFO:tasks.workunit.client.1.smithi115.stdout:6/801: mkdir d0/d1/d10/d13/d59/d95/d108 0 2021-11-03T17:15:36.074 INFO:tasks.workunit.client.1.smithi115.stdout:7/931: link d3/da/d18/d73/d35/f5f d3/da/d18/d73/d35/d53/f11f 0 2021-11-03T17:15:36.075 INFO:tasks.workunit.client.1.smithi115.stdout:7/932: chown d3/d10a/d10b/l11e 0 1 2021-11-03T17:15:36.082 INFO:tasks.workunit.client.1.smithi115.stdout:6/802: unlink d0/d1/d10/d13/d1c/d1e/l75 0 2021-11-03T17:15:36.086 INFO:tasks.workunit.client.1.smithi115.stdout:6/803: creat d0/d74/dae/db3/f109 x:0 0 0 2021-11-03T17:15:36.086 INFO:tasks.workunit.client.1.smithi115.stdout:6/804: write d0/f4f [1053674,94422] 0 2021-11-03T17:15:36.086 INFO:tasks.workunit.client.1.smithi115.stdout:6/805: mknod d0/d1/d6a/d88/dd8/c10a 1 2021-11-03T17:15:36.086 INFO:tasks.workunit.client.1.smithi115.stdout:6/806: dread - d0/d1/d10/d13/d87/ff7 zero size 2021-11-03T17:15:36.087 INFO:tasks.workunit.client.1.smithi115.stdout:6/807: truncate d0/d1/d2/d3f/f51 1812099 0 2021-11-03T17:15:36.095 INFO:tasks.workunit.client.1.smithi115.stdout:6/808: symlink d0/d74/dae/db3/l10b 0 2021-11-03T17:15:36.097 INFO:tasks.workunit.client.1.smithi115.stdout:6/809: creat d0/d1/f10c x:0 0 0 2021-11-03T17:15:36.109 INFO:tasks.workunit.client.1.smithi115.stdout:9/871: sync 2021-11-03T17:15:36.110 INFO:tasks.workunit.client.1.smithi115.stdout:6/810: rmdir d0/d1 39 2021-11-03T17:15:36.115 INFO:tasks.workunit.client.1.smithi115.stdout:9/872: fsync d1/d8a/f94 0 2021-11-03T17:15:36.116 INFO:tasks.workunit.client.1.smithi115.stdout:6/811: dread - d0/d1/d10/d13/d1c/d1e/fbf zero size 2021-11-03T17:15:36.117 INFO:tasks.workunit.client.1.smithi115.stdout:6/812: read d0/d1/d2/f4a [834379,78506] 0 2021-11-03T17:15:36.117 INFO:tasks.workunit.client.1.smithi115.stdout:6/813: chown d0/f2a 13 1 2021-11-03T17:15:36.118 INFO:tasks.workunit.client.1.smithi115.stdout:6/814: write d0/f41 [2072214,68506] 0 2021-11-03T17:15:36.119 INFO:tasks.workunit.client.1.smithi115.stdout:9/873: creat d1/d1a/d48/d27/d80/dcc/f13e x:0 0 0 2021-11-03T17:15:36.121 INFO:tasks.workunit.client.1.smithi115.stdout:6/815: creat d0/d74/dae/db3/f10d x:0 0 0 2021-11-03T17:15:36.122 INFO:tasks.workunit.client.1.smithi115.stdout:6/816: chown d0/d1/d2/fd 786581571 1 2021-11-03T17:15:36.123 INFO:tasks.workunit.client.1.smithi115.stdout:9/874: rename d1/d18/d2b/d42/d63/f129 to d1/d18/d2b/d45/dc2/f13f 0 2021-11-03T17:15:36.125 INFO:tasks.workunit.client.1.smithi115.stdout:6/817: readlink d0/d1/d10/d13/d1c/l57 0 2021-11-03T17:15:36.128 INFO:tasks.workunit.client.1.smithi115.stdout:9/875: unlink d1/d1a/d48/d27/d2f/d41/ff9 0 2021-11-03T17:15:36.128 INFO:tasks.workunit.client.1.smithi115.stdout:9/876: dread - d1/d1a/d48/d27/ff7 zero size 2021-11-03T17:15:36.129 INFO:tasks.workunit.client.1.smithi115.stdout:9/877: readlink d1/d1a/d48/d27/d53/d83/d88/d9e/d58/l102 0 2021-11-03T17:15:36.129 INFO:tasks.workunit.client.1.smithi115.stdout:9/878: chown d1/d18/d2b/d42/fb7 171356 1 2021-11-03T17:15:36.130 INFO:tasks.workunit.client.1.smithi115.stdout:6/818: truncate d0/d1/d10/d13/d59/fe3 67267 0 2021-11-03T17:15:36.130 INFO:tasks.workunit.client.1.smithi115.stdout:9/879: write d1/d18/d2b/d42/fbd [5681489,94468] 0 2021-11-03T17:15:36.133 INFO:tasks.workunit.client.1.smithi115.stdout:9/880: rmdir d1/d1a/dd9/d126 39 2021-11-03T17:15:36.138 INFO:tasks.workunit.client.1.smithi115.stdout:9/881: dread d1/d1a/f131 [0,4194304] 0 2021-11-03T17:15:36.139 INFO:tasks.workunit.client.1.smithi115.stdout:9/882: truncate d1/d1a/d48/d27/d2f/d41/f12a 666353 0 2021-11-03T17:15:36.139 INFO:tasks.workunit.client.1.smithi115.stdout:9/883: fdatasync d1/d18/d4a/d74/d8b/dbc/fe1 0 2021-11-03T17:15:36.140 INFO:tasks.workunit.client.1.smithi115.stdout:6/819: dread d0/d1/d10/f3b [0,4194304] 0 2021-11-03T17:15:36.140 INFO:tasks.workunit.client.1.smithi115.stdout:6/820: chown d0/f22 228 1 2021-11-03T17:15:36.157 INFO:tasks.workunit.client.1.smithi115.stdout:6/821: dwrite d0/d1/d10/d13/d59/f67 [0,4194304] 0 2021-11-03T17:15:36.158 INFO:tasks.workunit.client.1.smithi115.stdout:9/884: write d1/d1a/f28 [2431135,94604] 0 2021-11-03T17:15:36.158 INFO:tasks.workunit.client.1.smithi115.stdout:9/885: chown d1/d1a/d48/d27/d80 111519158 1 2021-11-03T17:15:36.165 INFO:tasks.workunit.client.1.smithi115.stdout:9/886: link d1/d1a/d48/d27/d2f/d41/l78 d1/d18/d2b/de8/l140 0 2021-11-03T17:15:36.165 INFO:tasks.workunit.client.1.smithi115.stdout:6/822: rmdir d0/d1/d10/d13/d59 39 2021-11-03T17:15:36.167 INFO:tasks.workunit.client.1.smithi115.stdout:6/823: dread d0/d1/d2/f4a [0,4194304] 0 2021-11-03T17:15:36.171 INFO:tasks.workunit.client.1.smithi115.stdout:6/824: rename d0/d74/dae/db3/f10d to d0/f10e 0 2021-11-03T17:15:36.188 INFO:tasks.workunit.client.1.smithi115.stdout:9/887: link d1/d1a/d48/d27/d80/f104 d1/d1a/d48/d27/d53/d83/f141 0 2021-11-03T17:15:36.192 INFO:tasks.workunit.client.1.smithi115.stdout:6/825: symlink d0/d1/d10/d13/d59/d95/db4/l10f 0 2021-11-03T17:15:36.193 INFO:tasks.workunit.client.1.smithi115.stdout:6/826: read d0/d7f/fcd [1337210,47121] 0 2021-11-03T17:15:36.193 INFO:tasks.workunit.client.1.smithi115.stdout:6/827: truncate d0/d1/d2/d3f/deb/d4c/d96/d61/f8a 1991197 0 2021-11-03T17:15:36.193 INFO:tasks.workunit.client.1.smithi115.stdout:6/828: fdatasync d0/d1/d2/d3f/fd0 0 2021-11-03T17:15:36.200 INFO:tasks.workunit.client.1.smithi115.stdout:6/829: mkdir d0/d1/d6a/d88/da0/d110 0 2021-11-03T17:15:36.201 INFO:tasks.workunit.client.1.smithi115.stdout:9/888: mkdir d1/d66/d142 0 2021-11-03T17:15:36.206 INFO:tasks.workunit.client.1.smithi115.stdout:6/830: creat d0/d1/d2/d3f/deb/d4c/dca/f111 x:0 0 0 2021-11-03T17:15:36.212 INFO:tasks.workunit.client.1.smithi115.stdout:6/831: chown d0/d1/d10/d13/d1c/d1e/l97 1843 1 2021-11-03T17:15:36.217 INFO:tasks.workunit.client.1.smithi115.stdout:9/889: rename d1/d18/d2b/d42/d6e/db5/dfa/l103 to d1/d18/d2b/db9/l143 0 2021-11-03T17:15:36.218 INFO:tasks.workunit.client.1.smithi115.stdout:9/890: write d1/f72 [1881077,48544] 0 2021-11-03T17:15:36.218 INFO:tasks.workunit.client.1.smithi115.stdout:9/891: stat d1/d1a/d48/d27/d53/f95 0 2021-11-03T17:15:36.219 INFO:tasks.workunit.client.1.smithi115.stdout:6/832: creat d0/d1/d6a/d88/f112 x:0 0 0 2021-11-03T17:15:36.223 INFO:tasks.workunit.client.1.smithi115.stdout:9/892: rename d1/d66/fd1 to d1/d18/d2b/f144 0 2021-11-03T17:15:36.228 INFO:tasks.workunit.client.1.smithi115.stdout:6/833: creat d0/d1/d2/d3f/deb/d4c/d96/d61/f113 x:0 0 0 2021-11-03T17:15:36.228 INFO:tasks.workunit.client.1.smithi115.stdout:6/834: write d0/d1/d2/d3f/f51 [1909351,9242] 0 2021-11-03T17:15:36.231 INFO:tasks.workunit.client.1.smithi115.stdout:9/893: symlink d1/d18/d2b/d42/d6e/db5/dfa/l145 0 2021-11-03T17:15:36.244 INFO:tasks.workunit.client.1.smithi115.stdout:6/835: unlink d0/d1/d10/d13/d1c/d1e/l43 0 2021-11-03T17:15:36.245 INFO:tasks.workunit.client.1.smithi115.stdout:6/836: read - d0/d1/d6a/fdc zero size 2021-11-03T17:15:36.245 INFO:tasks.workunit.client.1.smithi115.stdout:6/837: readlink d0/d1/d2/d3f/deb/d4c/d96/d61/l6f 0 2021-11-03T17:15:36.250 INFO:tasks.workunit.client.1.smithi115.stdout:6/838: getdents d0/d1/d6a 0 2021-11-03T17:15:36.250 INFO:tasks.workunit.client.1.smithi115.stdout:6/839: mknod d0/d1/d2/d3f/deb/d4c/dca/c114 1 2021-11-03T17:15:36.251 INFO:tasks.workunit.client.1.smithi115.stdout:6/840: chown d0/d1/fe7 876495355 1 2021-11-03T17:15:36.251 INFO:tasks.workunit.client.1.smithi115.stdout:6/841: write d0/d74/dae/db3/ff4 [984941,95172] 0 2021-11-03T17:15:36.251 INFO:tasks.workunit.client.1.smithi115.stdout:6/842: truncate d0/d74/fc8 43706 0 2021-11-03T17:15:36.251 INFO:tasks.workunit.client.1.smithi115.stdout:6/843: getdents d0/d1/d6a/d88/dd8 0 2021-11-03T17:15:36.267 INFO:tasks.workunit.client.1.smithi115.stdout:6/844: dwrite d0/d1/d10/d13/d87/dd2/dd7/fef [0,4194304] 0 2021-11-03T17:15:36.267 INFO:tasks.workunit.client.1.smithi115.stdout:6/845: mknod d0/c115 1 2021-11-03T17:15:36.313 INFO:tasks.workunit.client.1.smithi115.stdout:9/894: write d1/d1a/d48/d27/d53/d83/d88/d9e/f90 [850580,10227] 0 2021-11-03T17:15:36.313 INFO:tasks.workunit.client.1.smithi115.stdout:9/895: chown d1/d1a/d48/f73 156 1 2021-11-03T17:15:36.313 INFO:tasks.workunit.client.1.smithi115.stdout:9/896: fdatasync d1/d18/d4a/fc4 0 2021-11-03T17:15:36.315 INFO:tasks.workunit.client.1.smithi115.stdout:6/846: symlink d0/d1/d10/d13/d59/d95/db4/l116 0 2021-11-03T17:15:36.318 INFO:tasks.workunit.client.1.smithi115.stdout:9/897: creat d1/d18/d2b/d45/dc2/f146 x:0 0 0 2021-11-03T17:15:36.319 INFO:tasks.workunit.client.1.smithi115.stdout:2/875: sync 2021-11-03T17:15:36.320 INFO:tasks.workunit.client.1.smithi115.stdout:1/972: sync 2021-11-03T17:15:36.320 INFO:tasks.workunit.client.1.smithi115.stdout:1/973: fdatasync d7/dc/d1a/f4a 0 2021-11-03T17:15:36.320 INFO:tasks.workunit.client.1.smithi115.stdout:1/974: readlink d7/dc/dd/d3e/d58/dc2/lcb 0 2021-11-03T17:15:36.320 INFO:tasks.workunit.client.1.smithi115.stdout:2/876: write d4/d6/dad/de8/d9c/f11d [517772,18045] 0 2021-11-03T17:15:36.323 INFO:tasks.workunit.client.1.smithi115.stdout:1/975: stat d7/dc/dd/d3e/d58/l11f 0 2021-11-03T17:15:36.327 INFO:tasks.workunit.client.1.smithi115.stdout:1/976: mknod d7/dc/daf/c130 1 2021-11-03T17:15:36.329 INFO:tasks.workunit.client.1.smithi115.stdout:2/877: truncate d4/d6/d56/d6c/d34/d43/d63/d77/f81 444700 0 2021-11-03T17:15:36.329 INFO:tasks.workunit.client.1.smithi115.stdout:2/878: stat d4/d6/d56/d6c/d34/d119/l120 0 2021-11-03T17:15:36.333 INFO:tasks.workunit.client.1.smithi115.stdout:9/898: chown d1/d1a/d48/d27/d53/f82 300519775 1 2021-11-03T17:15:36.340 INFO:tasks.workunit.client.1.smithi115.stdout:9/899: getdents d1/d1a/d48/d27/d80 0 2021-11-03T17:15:36.341 INFO:tasks.workunit.client.1.smithi115.stdout:6/847: dwrite d0/d1/d2/f1f [8388608,4194304] 0 2021-11-03T17:15:36.342 INFO:tasks.workunit.client.1.smithi115.stdout:6/848: mknod d0/d85/d8f/dfa/c117 1 2021-11-03T17:15:36.342 INFO:tasks.workunit.client.1.smithi115.stdout:6/849: mknod d0/d1/d6a/d88/da0/c118 1 2021-11-03T17:15:36.342 INFO:tasks.workunit.client.1.smithi115.stdout:6/850: write d0/d74/dae/db3/f109 [812329,34072] 0 2021-11-03T17:15:36.342 INFO:tasks.workunit.client.1.smithi115.stdout:6/851: dread - d0/d1/d2/d3f/deb/d4c/fc7 zero size 2021-11-03T17:15:36.342 INFO:tasks.workunit.client.1.smithi115.stdout:6/852: write d0/d1/d6a/fa1 [181021,37197] 0 2021-11-03T17:15:36.343 INFO:tasks.workunit.client.1.smithi115.stdout:2/879: symlink d4/d6/d56/d6c/d34/d43/l12c 0 2021-11-03T17:15:36.346 INFO:tasks.workunit.client.1.smithi115.stdout:9/900: chown d1/d1a/d48/d27/d53/fbb 6063382 1 2021-11-03T17:15:36.349 INFO:tasks.workunit.client.1.smithi115.stdout:6/853: creat d0/d85/d8f/f119 x:0 0 0 2021-11-03T17:15:36.352 INFO:tasks.workunit.client.1.smithi115.stdout:1/977: dwrite d7/dc/dd/d3e/ffc [0,4194304] 0 2021-11-03T17:15:36.363 INFO:tasks.workunit.client.1.smithi115.stdout:1/978: write d7/d38/d59/d125/d73/d7f/fa8 [4110329,72802] 0 2021-11-03T17:15:36.370 INFO:tasks.workunit.client.1.smithi115.stdout:9/901: symlink d1/d1a/d48/d27/l147 0 2021-11-03T17:15:36.371 INFO:tasks.workunit.client.1.smithi115.stdout:6/854: dwrite d0/d1/fe7 [4194304,4194304] 0 2021-11-03T17:15:36.372 INFO:tasks.workunit.client.1.smithi115.stdout:6/855: read d0/fa2 [1498801,48586] 0 2021-11-03T17:15:36.386 INFO:tasks.workunit.client.1.smithi115.stdout:2/880: write d4/f62 [3256605,4799] 0 2021-11-03T17:15:36.394 INFO:tasks.workunit.client.1.smithi115.stdout:9/902: symlink d1/d1a/d48/l148 0 2021-11-03T17:15:36.395 INFO:tasks.workunit.client.1.smithi115.stdout:1/979: dread d7/dc/f15 [0,4194304] 0 2021-11-03T17:15:36.395 INFO:tasks.workunit.client.1.smithi115.stdout:6/856: dwrite d0/d74/fc8 [0,4194304] 0 2021-11-03T17:15:36.395 INFO:tasks.workunit.client.1.smithi115.stdout:6/857: fdatasync d0/d7f/fcd 0 2021-11-03T17:15:36.396 INFO:tasks.workunit.client.1.smithi115.stdout:6/858: fdatasync d0/d1/d2/d3f/deb/d4c/fc7 0 2021-11-03T17:15:36.396 INFO:tasks.workunit.client.1.smithi115.stdout:6/859: fsync d0/d1/d2/d3f/deb/d4c/d96/fe1 0 2021-11-03T17:15:36.396 INFO:tasks.workunit.client.1.smithi115.stdout:6/860: stat d0/d1/d2/d3f/lbc 0 2021-11-03T17:15:36.397 INFO:tasks.workunit.client.1.smithi115.stdout:2/881: mkdir d4/d6/dad/de8/dac/d12d 0 2021-11-03T17:15:36.410 INFO:tasks.workunit.client.1.smithi115.stdout:6/861: link d0/fa2 d0/d1/d10/d13/d87/dd2/f11a 0 2021-11-03T17:15:36.411 INFO:tasks.workunit.client.1.smithi115.stdout:6/862: write d0/d1/d10/f9a [925114,100593] 0 2021-11-03T17:15:36.412 INFO:tasks.workunit.client.1.smithi115.stdout:9/903: dwrite d1/d1a/d48/f73 [0,4194304] 0 2021-11-03T17:15:36.413 INFO:tasks.workunit.client.1.smithi115.stdout:1/980: dread d7/dc/d101/d4b/f72 [0,4194304] 0 2021-11-03T17:15:36.424 INFO:tasks.workunit.client.1.smithi115.stdout:1/981: dread d7/dc/dd/d3e/d58/ffd [0,4194304] 0 2021-11-03T17:15:36.437 INFO:tasks.workunit.client.1.smithi115.stdout:1/982: dwrite d7/dc/d101/d43/f74 [0,4194304] 0 2021-11-03T17:15:36.438 INFO:tasks.workunit.client.1.smithi115.stdout:1/983: write d7/dc/dd/d3e/d58/dc2/dd2/ff9 [1237001,54275] 0 2021-11-03T17:15:36.444 INFO:tasks.workunit.client.1.smithi115.stdout:6/863: mknod d0/d1/d10/d13/d87/c11b 1 2021-11-03T17:15:36.444 INFO:tasks.workunit.client.1.smithi115.stdout:6/864: fdatasync d0/d1/d10/d13/d59/f80 0 2021-11-03T17:15:36.445 INFO:tasks.workunit.client.1.smithi115.stdout:6/865: fsync d0/d1/d10/d13/d87/fa5 0 2021-11-03T17:15:36.482 INFO:tasks.workunit.client.1.smithi115.stdout:1/984: dread - d7/dc/d101/d43/d78/fbc zero size 2021-11-03T17:15:36.488 INFO:tasks.workunit.client.1.smithi115.stdout:9/904: creat d1/d1a/d48/d27/d53/d83/d88/dcd/dda/d12f/f149 x:0 0 0 2021-11-03T17:15:36.488 INFO:tasks.workunit.client.1.smithi115.stdout:9/905: stat d1/d1a/d48/d27/d2f/d6a/f120 0 2021-11-03T17:15:36.493 INFO:tasks.workunit.client.1.smithi115.stdout:6/866: mkdir d0/d7f/d11c 0 2021-11-03T17:15:36.496 INFO:tasks.workunit.client.1.smithi115.stdout:6/867: creat d0/d85/d8f/dfa/f11d x:0 0 0 2021-11-03T17:15:36.496 INFO:tasks.workunit.client.1.smithi115.stdout:6/868: getdents d0/d1/d6a/d88/dd8 0 2021-11-03T17:15:36.497 INFO:tasks.workunit.client.1.smithi115.stdout:6/869: chown d0/f10e 105012 1 2021-11-03T17:15:36.497 INFO:tasks.workunit.client.1.smithi115.stdout:6/870: dread - d0/fe5 zero size 2021-11-03T17:15:36.498 INFO:tasks.workunit.client.1.smithi115.stdout:6/871: link d0/d1/d2/df3/lfe d0/d85/l11e 0 2021-11-03T17:15:36.499 INFO:tasks.workunit.client.1.smithi115.stdout:6/872: dread - d0/d1/d6a/fdc zero size 2021-11-03T17:15:36.500 INFO:tasks.workunit.client.1.smithi115.stdout:6/873: rename d0/d1/d10/f3b to d0/d1/d6a/d88/dd8/f11f 0 2021-11-03T17:15:36.521 INFO:tasks.workunit.client.1.smithi115.stdout:1/985: dwrite f2 [0,4194304] 0 2021-11-03T17:15:36.529 INFO:tasks.workunit.client.1.smithi115.stdout:1/986: dread d7/ff6 [0,4194304] 0 2021-11-03T17:15:36.529 INFO:tasks.workunit.client.1.smithi115.stdout:1/987: write d7/d38/d84/f112 [851590,10900] 0 2021-11-03T17:15:36.530 INFO:tasks.workunit.client.1.smithi115.stdout:9/906: dwrite d1/d18/d4a/fec [0,4194304] 0 2021-11-03T17:15:36.530 INFO:tasks.workunit.client.1.smithi115.stdout:9/907: truncate d1/d1a/d48/d27/d80/dcc/f13a 340737 0 2021-11-03T17:15:36.531 INFO:tasks.workunit.client.1.smithi115.stdout:6/874: dwrite d0/d1/d6a/d88/da0/db5/fe9 [0,4194304] 0 2021-11-03T17:15:36.533 INFO:tasks.workunit.client.1.smithi115.stdout:6/875: dread d0/f4d [0,4194304] 0 2021-11-03T17:15:36.539 INFO:tasks.workunit.client.1.smithi115.stdout:9/908: unlink d1/d18/d2b/db9/fed 0 2021-11-03T17:15:36.540 INFO:tasks.workunit.client.1.smithi115.stdout:9/909: write d1/d18/d2b/d42/d63/f9f [3246391,67149] 0 2021-11-03T17:15:36.541 INFO:tasks.workunit.client.1.smithi115.stdout:9/910: read - d1/d1a/d48/d27/d53/d83/d88/f123 zero size 2021-11-03T17:15:36.547 INFO:tasks.workunit.client.1.smithi115.stdout:9/911: dread d1/d18/d2b/f35 [0,4194304] 0 2021-11-03T17:15:36.548 INFO:tasks.workunit.client.1.smithi115.stdout:9/912: write d1/d1a/d48/d27/d80/dcc/f13e [224826,35323] 0 2021-11-03T17:15:36.548 INFO:tasks.workunit.client.1.smithi115.stdout:1/988: symlink d7/dc/d101/d4b/d55/l131 0 2021-11-03T17:15:36.550 INFO:tasks.workunit.client.1.smithi115.stdout:6/876: dwrite d0/d1/d10/d13/d1c/f36 [12582912,4194304] 0 2021-11-03T17:15:36.551 INFO:tasks.workunit.client.1.smithi115.stdout:6/877: stat d0/d1/d10/d13/d87/fa4 0 2021-11-03T17:15:36.551 INFO:tasks.workunit.client.1.smithi115.stdout:6/878: dread - d0/d1/fba zero size 2021-11-03T17:15:36.552 INFO:tasks.workunit.client.1.smithi115.stdout:6/879: chown d0/d1/d10/d13/f72 1542 1 2021-11-03T17:15:36.552 INFO:tasks.workunit.client.1.smithi115.stdout:9/913: read - d1/d18/d4a/fc8 zero size 2021-11-03T17:15:36.555 INFO:tasks.workunit.client.1.smithi115.stdout:9/914: write d1/d18/d2b/d45/f114 [1787668,102101] 0 2021-11-03T17:15:36.556 INFO:tasks.workunit.client.1.smithi115.stdout:9/915: creat d1/d8a/f14a x:0 0 0 2021-11-03T17:15:36.556 INFO:tasks.workunit.client.1.smithi115.stdout:9/916: stat d1/d18/d4a/fc8 0 2021-11-03T17:15:36.557 INFO:tasks.workunit.client.1.smithi115.stdout:6/880: mkdir d0/d1/d10/d13/d1c/d120 0 2021-11-03T17:15:36.562 INFO:tasks.workunit.client.1.smithi115.stdout:9/917: mknod d1/d18/d65/c14b 1 2021-11-03T17:15:36.563 INFO:tasks.workunit.client.1.smithi115.stdout:9/918: chown d1/l4d 4 1 2021-11-03T17:15:36.564 INFO:tasks.workunit.client.1.smithi115.stdout:6/881: symlink d0/d1/d2/l121 0 2021-11-03T17:15:36.566 INFO:tasks.workunit.client.1.smithi115.stdout:6/882: getdents d0/d1/d10/d13/d87/dd2 0 2021-11-03T17:15:36.566 INFO:tasks.workunit.client.1.smithi115.stdout:6/883: readlink d0/d1/d10/d13/d1c/d1e/l97 0 2021-11-03T17:15:36.572 INFO:tasks.workunit.client.1.smithi115.stdout:6/884: unlink d0/d1/d10/d13/d87/fa5 0 2021-11-03T17:15:36.647 INFO:tasks.workunit.client.1.smithi115.stdout:6/885: getdents d0/d1/d10/d13/d1c/d1e/dcc 0 2021-11-03T17:15:36.647 INFO:tasks.workunit.client.1.smithi115.stdout:6/886: write d0/fa2 [1782198,2742] 0 2021-11-03T17:15:36.647 INFO:tasks.workunit.client.1.smithi115.stdout:6/887: creat d0/d1/d2/df3/d107/d102/d105/f122 x:0 0 0 2021-11-03T17:15:36.647 INFO:tasks.workunit.client.1.smithi115.stdout:6/888: rename d0/led to d0/d1/d10/d13/d87/l123 0 2021-11-03T17:15:36.648 INFO:tasks.workunit.client.1.smithi115.stdout:6/889: creat d0/d85/d8f/f124 x:0 0 0 2021-11-03T17:15:36.648 INFO:tasks.workunit.client.1.smithi115.stdout:6/890: creat d0/d1/d6a/d88/da0/db5/f125 x:0 0 0 2021-11-03T17:15:36.648 INFO:tasks.workunit.client.1.smithi115.stdout:6/891: rmdir d0/d74 39 2021-11-03T17:15:36.648 INFO:tasks.workunit.client.1.smithi115.stdout:6/892: truncate d0/d85/d8f/dfa/f11d 486252 0 2021-11-03T17:15:36.648 INFO:tasks.workunit.client.1.smithi115.stdout:6/893: chown d0/d74/dae/db3/ff4 434 1 2021-11-03T17:15:36.649 INFO:tasks.workunit.client.1.smithi115.stdout:1/989: dwrite d7/d38/d59/d125/d73/fb0 [0,4194304] 0 2021-11-03T17:15:36.649 INFO:tasks.workunit.client.1.smithi115.stdout:9/919: dwrite d1/d1a/dd9/f10e [0,4194304] 0 2021-11-03T17:15:36.649 INFO:tasks.workunit.client.1.smithi115.stdout:1/990: unlink d7/dc/d101/d43/df4/fee 0 2021-11-03T17:15:36.649 INFO:tasks.workunit.client.1.smithi115.stdout:9/920: write d1/d1a/d48/d27/d53/d83/d88/ff5 [1928682,341] 0 2021-11-03T17:15:36.650 INFO:tasks.workunit.client.1.smithi115.stdout:1/991: mkdir d7/d38/d84/d120/d132 0 2021-11-03T17:15:36.650 INFO:tasks.workunit.client.1.smithi115.stdout:1/992: truncate d7/dc/dd/d3e/d58/ffd 4738972 0 2021-11-03T17:15:36.650 INFO:tasks.workunit.client.1.smithi115.stdout:9/921: mknod d1/d1a/d48/d27/d2f/d41/c14c 1 2021-11-03T17:15:36.663 INFO:tasks.workunit.client.1.smithi115.stdout:9/922: creat d1/d18/d4a/d74/f14d x:0 0 0 2021-11-03T17:15:36.664 INFO:tasks.workunit.client.1.smithi115.stdout:9/923: write d1/d8a/f14a [186687,60858] 0 2021-11-03T17:15:36.664 INFO:tasks.workunit.client.1.smithi115.stdout:9/924: write d1/d1a/d48/f73 [5154203,68648] 0 2021-11-03T17:15:36.665 INFO:tasks.workunit.client.1.smithi115.stdout:1/993: symlink d7/dc/l133 0 2021-11-03T17:15:36.668 INFO:tasks.workunit.client.1.smithi115.stdout:1/994: link d7/dc/dd/d3e/d58/dc2/dd2/l12f d7/dc/dd/d3e/dbe/dff/d121/l134 0 2021-11-03T17:15:36.674 INFO:tasks.workunit.client.1.smithi115.stdout:9/925: rename d1/d18/d2b/d42 to d1/d1a/d48/d27/d2f/d41/d14e 0 2021-11-03T17:15:36.686 INFO:tasks.workunit.client.1.smithi115.stdout:9/926: rename d1/d1a/d48/d27/d80/f137 to d1/d18/d2b/db9/f14f 0 2021-11-03T17:15:36.696 INFO:tasks.workunit.client.1.smithi115.stdout:1/995: dwrite d7/d38/d59/d125/d73/fb0 [0,4194304] 0 2021-11-03T17:15:36.699 INFO:tasks.workunit.client.1.smithi115.stdout:1/996: mknod d7/dc/d101/d4b/c135 1 2021-11-03T17:15:36.700 INFO:tasks.workunit.client.1.smithi115.stdout:9/927: dread d1/d1a/d48/d27/d53/fbb [0,4194304] 0 2021-11-03T17:15:36.701 INFO:tasks.workunit.client.1.smithi115.stdout:1/997: symlink d7/dc/d101/d43/df4/d60/l136 0 2021-11-03T17:15:36.705 INFO:tasks.workunit.client.1.smithi115.stdout:1/998: creat d7/dc/d101/d43/df4/d60/f137 x:0 0 0 2021-11-03T17:15:36.706 INFO:tasks.workunit.client.1.smithi115.stdout:9/928: fdatasync d1/d1a/d48/d27/d53/f7f 0 2021-11-03T17:15:36.709 INFO:tasks.workunit.client.1.smithi115.stdout:9/929: getdents d1/d1a/d48/d27/d2f/d7b 0 2021-11-03T17:15:36.710 INFO:tasks.workunit.client.1.smithi115.stdout:1/999: mkdir d7/d38/d59/d125/d73/d7f/d138 0 2021-11-03T17:15:36.723 INFO:tasks.workunit.client.1.smithi115.stdout:9/930: dwrite d1/d1a/d48/d27/d53/d83/d88/d9e/f90 [0,4194304] 0 2021-11-03T17:15:36.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:36 smithi115 conmon[34670]: cluster 2021-11-03T17:15:34.648508+0000 mgr.smithi027.ufhhvb (mgr.14622) 35 : 2021-11-03T17:15:36.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:36 smithi115 conmon[34670]: cluster [DBG] pgmap v21: 65 pgs: 65 active+clean; 2.1 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 34 MiB/s rd, 61 MiB/s wr, 195 op/s 2021-11-03T17:15:36.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:36 smithi027 conmon[32322]: cluster 2021-11-03T17:15:34.648508+0000 mgr.smithi027.ufhhvb ( 2021-11-03T17:15:36.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:36 smithi027 conmon[32322]: mgr.14622) 35 : cluster [DBG] pgmap v21: 65 pgs: 65 active+clean; 2.1 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 34 MiB/s rd, 61 MiB/s wr, 195 op/s 2021-11-03T17:15:36.928 INFO:tasks.workunit.client.1.smithi115.stdout:8/879: sync 2021-11-03T17:15:36.928 INFO:tasks.workunit.client.1.smithi115.stdout:8/880: fsync d4/d6/d18/d30/f3a 0 2021-11-03T17:15:36.954 INFO:tasks.workunit.client.1.smithi115.stdout:8/881: dwrite d4/d6/d3f/da4/f11f [4194304,4194304] 0 2021-11-03T17:15:36.954 INFO:tasks.workunit.client.1.smithi115.stdout:8/882: readlink d4/d7d/l11c 0 2021-11-03T17:15:36.957 INFO:tasks.workunit.client.1.smithi115.stdout:8/883: creat d4/d6/d3f/da4/da6/f134 x:0 0 0 2021-11-03T17:15:36.959 INFO:tasks.workunit.client.1.smithi115.stdout:8/884: symlink d4/d6/d3f/dde/l135 0 2021-11-03T17:15:36.960 INFO:tasks.workunit.client.1.smithi115.stdout:8/885: write d4/d6/d18/d30/d33/d93/f123 [12612,129699] 0 2021-11-03T17:15:36.961 INFO:tasks.workunit.client.1.smithi115.stdout:8/886: write d4/d6/d3f/d9a/d11b/d6e/fbf [3491883,47095] 0 2021-11-03T17:15:36.972 INFO:tasks.workunit.client.1.smithi115.stdout:8/887: symlink d4/d6/d18/d30/d33/l136 0 2021-11-03T17:15:36.973 INFO:tasks.workunit.client.1.smithi115.stdout:8/888: mknod d4/d6/d18/d30/d33/d93/c137 1 2021-11-03T17:15:36.978 INFO:tasks.workunit.client.1.smithi115.stdout:8/889: link d4/d6/d18/d30/d79/d7f/ff8 d4/d6/d3f/d9a/d11b/d47/deb/f138 0 2021-11-03T17:15:36.978 INFO:tasks.workunit.client.1.smithi115.stdout:7/933: sync 2021-11-03T17:15:36.980 INFO:tasks.workunit.client.1.smithi115.stdout:7/934: dread d3/da/d18/d73/dc8/ff2 [0,4194304] 0 2021-11-03T17:15:36.985 INFO:tasks.workunit.client.1.smithi115.stdout:7/935: rmdir d3/d4c/da6/dc7/d101 0 2021-11-03T17:15:36.986 INFO:tasks.workunit.client.1.smithi115.stdout:8/890: symlink d4/d6/d3f/d43/l139 0 2021-11-03T17:15:36.986 INFO:tasks.workunit.client.1.smithi115.stdout:8/891: write d4/d6/d3f/d9a/f84 [979553,13204] 0 2021-11-03T17:15:36.988 INFO:tasks.workunit.client.1.smithi115.stdout:7/936: symlink d3/d10a/l120 0 2021-11-03T17:15:36.991 INFO:tasks.workunit.client.1.smithi115.stdout:7/937: truncate d3/da/d18/d73/d35/d53/f11f 8256851 0 2021-11-03T17:15:36.991 INFO:tasks.workunit.client.1.smithi115.stdout:7/938: stat d3/da/d18/d73/d52/db4 0 2021-11-03T17:15:36.992 INFO:tasks.workunit.client.1.smithi115.stdout:7/939: mknod d3/c121 1 2021-11-03T17:15:36.994 INFO:tasks.workunit.client.1.smithi115.stdout:7/940: creat d3/da/d18/d73/d35/d53/dde/f122 x:0 0 0 2021-11-03T17:15:36.994 INFO:tasks.workunit.client.1.smithi115.stdout:7/941: write d3/da/d18/d73/d35/dbb/fee [363199,35574] 0 2021-11-03T17:15:37.003 INFO:tasks.workunit.client.1.smithi115.stdout:8/892: dwrite d4/d6/d3f/d9a/d11b/dcd/ff7 [0,4194304] 0 2021-11-03T17:15:37.007 INFO:tasks.workunit.client.1.smithi115.stdout:7/942: creat d3/d4c/da6/f123 x:0 0 0 2021-11-03T17:15:37.014 INFO:tasks.workunit.client.1.smithi115.stdout:8/893: symlink d4/d6/d3f/d9a/d11b/d47/deb/l13a 0 2021-11-03T17:15:37.014 INFO:tasks.workunit.client.1.smithi115.stdout:8/894: truncate d4/d6/d18/d9d/f12d 42273 0 2021-11-03T17:15:37.024 INFO:tasks.workunit.client.1.smithi115.stdout:7/943: dwrite d3/da/d18/d73/fc2 [4194304,4194304] 0 2021-11-03T17:15:37.025 INFO:tasks.workunit.client.1.smithi115.stdout:7/944: fsync d3/d4c/da6/fbf 0 2021-11-03T17:15:37.025 INFO:tasks.workunit.client.1.smithi115.stdout:8/895: dread d4/d6/d18/f77 [0,4194304] 0 2021-11-03T17:15:37.025 INFO:tasks.workunit.client.1.smithi115.stdout:7/945: write d3/da/f90 [85673,13684] 0 2021-11-03T17:15:37.027 INFO:tasks.workunit.client.1.smithi115.stdout:8/896: symlink d4/d6/d3f/d9a/d11b/d115/l13b 0 2021-11-03T17:15:37.028 INFO:tasks.workunit.client.1.smithi115.stdout:7/946: symlink d3/da/d18/dfb/l124 0 2021-11-03T17:15:37.036 INFO:tasks.workunit.client.1.smithi115.stdout:7/947: symlink d3/da/d18/d9c/da1/ded/df8/l125 0 2021-11-03T17:15:37.040 INFO:tasks.workunit.client.1.smithi115.stdout:8/897: dread d4/d6/d18/d30/d33/f7e [0,4194304] 0 2021-11-03T17:15:37.045 INFO:tasks.workunit.client.1.smithi115.stdout:7/948: dread d3/da/d18/d73/dda/f5e [0,4194304] 0 2021-11-03T17:15:37.045 INFO:tasks.workunit.client.1.smithi115.stdout:8/898: read d4/d6/d3f/d9a/d11b/f99 [2662219,121841] 0 2021-11-03T17:15:37.046 INFO:tasks.workunit.client.1.smithi115.stdout:7/949: mkdir d3/d4c/d7b/dd2/d126 0 2021-11-03T17:15:37.048 INFO:tasks.workunit.client.1.smithi115.stdout:8/899: rename d4/d6/d3f/dde/ff3 to d4/d6/d3f/d9a/d11b/d47/deb/f13c 0 2021-11-03T17:15:37.062 INFO:tasks.workunit.client.1.smithi115.stdout:7/950: mkdir d3/d10a/d10b/d127 0 2021-11-03T17:15:37.069 INFO:tasks.workunit.client.1.smithi115.stdout:8/900: link d4/d6/d18/d30/d33/l136 d4/d7d/l13d 0 2021-11-03T17:15:37.077 INFO:tasks.workunit.client.1.smithi115.stdout:7/951: dwrite d3/d4c/da6/fdb [0,4194304] 0 2021-11-03T17:15:37.094 INFO:tasks.workunit.client.1.smithi115.stdout:8/901: dread d4/f28 [0,4194304] 0 2021-11-03T17:15:37.094 INFO:tasks.workunit.client.1.smithi115.stdout:7/952: dwrite d3/d4c/f5b [4194304,4194304] 0 2021-11-03T17:15:37.118 INFO:tasks.workunit.client.1.smithi115.stdout:7/953: dwrite d3/da/f62 [0,4194304] 0 2021-11-03T17:15:37.141 INFO:tasks.workunit.client.1.smithi115.stdout:8/902: dwrite d4/f6c [0,4194304] 0 2021-11-03T17:15:37.141 INFO:tasks.workunit.client.1.smithi115.stdout:8/903: chown d4/d6/d3f/d9a/fd6 6918406 1 2021-11-03T17:15:37.143 INFO:tasks.workunit.client.1.smithi115.stdout:7/954: dwrite d3/d4c/d7b/dd2/fd7 [0,4194304] 0 2021-11-03T17:15:37.146 INFO:tasks.workunit.client.1.smithi115.stdout:8/904: creat d4/d6/d3f/da4/dfd/f13e x:0 0 0 2021-11-03T17:15:37.160 INFO:tasks.workunit.client.1.smithi115.stdout:7/955: mkdir d3/d10a/d10b/d127/d128 0 2021-11-03T17:15:37.164 INFO:tasks.workunit.client.1.smithi115.stdout:8/905: mkdir d4/d6/d3f/da4/da6/dcc/d13f 0 2021-11-03T17:15:37.166 INFO:tasks.workunit.client.1.smithi115.stdout:7/956: rmdir d3/da/d18/d9c 39 2021-11-03T17:15:37.169 INFO:tasks.workunit.client.1.smithi115.stdout:7/957: creat d3/d4c/d7b/dcf/d115/f129 x:0 0 0 2021-11-03T17:15:37.172 INFO:tasks.workunit.client.1.smithi115.stdout:8/906: rename d4/d6/d3f/d43/f5b to d4/d6/d3f/d9a/d11b/d115/f140 0 2021-11-03T17:15:37.172 INFO:tasks.workunit.client.1.smithi115.stdout:8/907: chown d4/d6/d3f/d9a/d11b/dcd/l129 44693166 1 2021-11-03T17:15:37.174 INFO:tasks.workunit.client.1.smithi115.stdout:7/958: symlink d3/d10a/d10b/l12a 0 2021-11-03T17:15:37.177 INFO:tasks.workunit.client.1.smithi115.stdout:8/908: symlink d4/d6/l141 0 2021-11-03T17:15:37.180 INFO:tasks.workunit.client.1.smithi115.stdout:8/909: dread d4/d6/d18/d30/d79/fb2 [0,4194304] 0 2021-11-03T17:15:37.181 INFO:tasks.workunit.client.1.smithi115.stdout:7/959: dread d3/da/d18/d73/d35/d53/f11f [0,4194304] 0 2021-11-03T17:15:37.182 INFO:tasks.workunit.client.1.smithi115.stdout:7/960: mknod d3/d4c/da6/dc7/c12b 1 2021-11-03T17:15:37.197 INFO:tasks.workunit.client.1.smithi115.stdout:7/961: creat d3/da/d78/dff/f12c x:0 0 0 2021-11-03T17:15:37.197 INFO:tasks.workunit.client.1.smithi115.stdout:7/962: dread - d3/d4c/d7b/dd2/fea zero size 2021-11-03T17:15:37.216 INFO:tasks.workunit.client.1.smithi115.stdout:7/963: creat d3/d4c/d7b/dd2/d110/f12d x:0 0 0 2021-11-03T17:15:37.216 INFO:tasks.workunit.client.1.smithi115.stdout:7/964: write d3/da/d8e/fe2 [4561130,29126] 0 2021-11-03T17:15:37.216 INFO:tasks.workunit.client.1.smithi115.stdout:7/965: mknod d3/da/d18/d73/d35/c12e 1 2021-11-03T17:15:37.217 INFO:tasks.workunit.client.1.smithi115.stdout:7/966: mknod d3/d4c/da6/c12f 1 2021-11-03T17:15:37.219 INFO:tasks.workunit.client.1.smithi115.stdout:7/967: rename d3/da/d18/d73/d52/df1/lfd to d3/d4c/d7b/dd2/d110/l130 0 2021-11-03T17:15:37.225 INFO:tasks.workunit.client.1.smithi115.stdout:7/968: dread d3/f25 [0,4194304] 0 2021-11-03T17:15:37.225 INFO:tasks.workunit.client.1.smithi115.stdout:8/910: dwrite d4/d6/d3f/d9a/d11b/fa3 [0,4194304] 0 2021-11-03T17:15:37.225 INFO:tasks.workunit.client.1.smithi115.stdout:8/911: readlink d4/d6/d18/d30/d33/l40 0 2021-11-03T17:15:37.238 INFO:tasks.workunit.client.1.smithi115.stdout:8/912: symlink d4/d6/d18/d30/d79/d7f/d11a/d109/l142 0 2021-11-03T17:15:37.241 INFO:tasks.workunit.client.1.smithi115.stdout:8/913: truncate d4/d6/d3f/da4/da6/fc6 14144 0 2021-11-03T17:15:37.246 INFO:tasks.workunit.client.1.smithi115.stdout:7/969: dwrite d3/d4c/da6/fdb [0,4194304] 0 2021-11-03T17:15:37.258 INFO:tasks.workunit.client.1.smithi115.stdout:2/882: sync 2021-11-03T17:15:37.263 INFO:tasks.workunit.client.1.smithi115.stdout:8/914: dwrite d4/d6/d3f/da4/da6/f134 [0,4194304] 0 2021-11-03T17:15:37.269 INFO:tasks.workunit.client.1.smithi115.stdout:7/970: dwrite d3/da/d18/d9c/da1/fb1 [0,4194304] 0 2021-11-03T17:15:37.276 INFO:tasks.workunit.client.1.smithi115.stdout:2/883: dwrite d4/d6/dad/dc2/fdb [0,4194304] 0 2021-11-03T17:15:37.296 INFO:tasks.workunit.client.1.smithi115.stdout:8/915: fsync d4/fb9 0 2021-11-03T17:15:37.303 INFO:tasks.workunit.client.1.smithi115.stdout:7/971: symlink d3/da/d18/d73/d35/d53/dde/l131 0 2021-11-03T17:15:37.303 INFO:tasks.workunit.client.1.smithi115.stdout:7/972: chown d3/da/f100 0 1 2021-11-03T17:15:37.310 INFO:tasks.workunit.client.1.smithi115.stdout:7/973: link d3/da/d18/d73/d52/db4/dd1/ldf d3/da/d18/d73/d35/da3/l132 0 2021-11-03T17:15:37.311 INFO:tasks.workunit.client.1.smithi115.stdout:7/974: readlink d3/da/d18/d73/d52/db4/le6 0 2021-11-03T17:15:37.314 INFO:tasks.workunit.client.1.smithi115.stdout:7/975: readlink d3/da/d18/d73/d52/l54 0 2021-11-03T17:15:37.315 INFO:tasks.workunit.client.1.smithi115.stdout:8/916: dwrite d4/d6/d3f/da4/da6/f134 [0,4194304] 0 2021-11-03T17:15:37.326 INFO:tasks.workunit.client.1.smithi115.stdout:2/884: write d4/d6/dad/f8b [4517081,86825] 0 2021-11-03T17:15:37.331 INFO:tasks.workunit.client.1.smithi115.stdout:8/917: creat d4/d6/d3f/d9a/d11b/dcd/f143 x:0 0 0 2021-11-03T17:15:37.337 INFO:tasks.workunit.client.1.smithi115.stdout:7/976: rename d3/da/d18/d73/d35/d53 to d3/da/d18/d73/dda/d133 0 2021-11-03T17:15:37.339 INFO:tasks.workunit.client.1.smithi115.stdout:7/977: mknod d3/da/d18/d73/dda/c134 1 2021-11-03T17:15:37.339 INFO:tasks.workunit.client.1.smithi115.stdout:7/978: dread - d3/da/f7e zero size 2021-11-03T17:15:37.339 INFO:tasks.workunit.client.1.smithi115.stdout:7/979: fdatasync d3/da/d18/d73/d35/f3f 0 2021-11-03T17:15:37.340 INFO:tasks.workunit.client.1.smithi115.stdout:7/980: readlink d3/da/d18/d9c/lb6 0 2021-11-03T17:15:37.349 INFO:tasks.workunit.client.1.smithi115.stdout:7/981: link d3/da/d78/fba d3/da/d18/d73/d35/df0/f135 0 2021-11-03T17:15:37.354 INFO:tasks.workunit.client.1.smithi115.stdout:7/982: dread d3/d4c/f7f [0,4194304] 0 2021-11-03T17:15:37.356 INFO:tasks.workunit.client.1.smithi115.stdout:8/918: dwrite d4/d6/f2c [0,4194304] 0 2021-11-03T17:15:37.357 INFO:tasks.workunit.client.1.smithi115.stdout:2/885: dwrite d4/d6/d56/d6c/d34/d43/d63/d69/fbf [4194304,4194304] 0 2021-11-03T17:15:37.357 INFO:tasks.workunit.client.1.smithi115.stdout:2/886: chown d4/f62 27 1 2021-11-03T17:15:37.358 INFO:tasks.workunit.client.1.smithi115.stdout:7/983: write d3/f9 [9850620,80919] 0 2021-11-03T17:15:37.358 INFO:tasks.workunit.client.1.smithi115.stdout:7/984: stat d3/da/d18/d73/dda/f118 0 2021-11-03T17:15:37.358 INFO:tasks.workunit.client.1.smithi115.stdout:7/985: read - d3/d4c/d7b/dd2/fea zero size 2021-11-03T17:15:37.360 INFO:tasks.workunit.client.1.smithi115.stdout:8/919: creat d4/d6/d18/d133/f144 x:0 0 0 2021-11-03T17:15:37.362 INFO:tasks.workunit.client.1.smithi115.stdout:2/887: dread d4/d6/f48 [0,4194304] 0 2021-11-03T17:15:37.365 INFO:tasks.workunit.client.1.smithi115.stdout:7/986: symlink d3/da/d78/l136 0 2021-11-03T17:15:37.368 INFO:tasks.workunit.client.1.smithi115.stdout:7/987: creat d3/da/d18/d73/d52/db4/dd1/f137 x:0 0 0 2021-11-03T17:15:37.371 INFO:tasks.workunit.client.1.smithi115.stdout:8/920: dread d4/d6/d3f/d9a/f54 [0,4194304] 0 2021-11-03T17:15:37.381 INFO:tasks.workunit.client.1.smithi115.stdout:8/921: dread d4/d6/d3f/d9a/d11b/fa3 [0,4194304] 0 2021-11-03T17:15:37.382 INFO:tasks.workunit.client.1.smithi115.stdout:8/922: dread - d4/d6/d18/d30/d79/d11e/fff zero size 2021-11-03T17:15:37.383 INFO:tasks.workunit.client.1.smithi115.stdout:8/923: symlink d4/d6/d3f/l145 0 2021-11-03T17:15:37.383 INFO:tasks.workunit.client.1.smithi115.stdout:2/888: dwrite d4/f5a [0,4194304] 0 2021-11-03T17:15:37.385 INFO:tasks.workunit.client.1.smithi115.stdout:8/924: creat d4/d6/d3f/d9a/d11b/d47/f146 x:0 0 0 2021-11-03T17:15:37.385 INFO:tasks.workunit.client.1.smithi115.stdout:8/925: fdatasync d4/d6/d3f/d9a/f117 0 2021-11-03T17:15:37.391 INFO:tasks.workunit.client.1.smithi115.stdout:7/988: dread d3/f9 [0,4194304] 0 2021-11-03T17:15:37.395 INFO:tasks.workunit.client.1.smithi115.stdout:7/989: mkdir d3/da/d18/d73/dda/d133/dde/d138 0 2021-11-03T17:15:37.396 INFO:tasks.workunit.client.1.smithi115.stdout:7/990: truncate d3/da/d18/d73/d35/da3/f113 39449 0 2021-11-03T17:15:37.402 INFO:tasks.workunit.client.1.smithi115.stdout:2/889: dread d4/d6/d13/d75/f7a [0,4194304] 0 2021-11-03T17:15:37.404 INFO:tasks.workunit.client.1.smithi115.stdout:8/926: dwrite d4/d6/d3f/d9a/f117 [0,4194304] 0 2021-11-03T17:15:37.408 INFO:tasks.workunit.client.1.smithi115.stdout:7/991: rename d3/da/d18/d73/dda/f9b to d3/da/d18/d73/f139 0 2021-11-03T17:15:37.413 INFO:tasks.workunit.client.1.smithi115.stdout:8/927: getdents d4/d6/d3f/d43/d88 0 2021-11-03T17:15:37.417 INFO:tasks.workunit.client.1.smithi115.stdout:7/992: symlink d3/da/d18/d73/dda/d133/l13a 0 2021-11-03T17:15:37.418 INFO:tasks.workunit.client.1.smithi115.stdout:7/993: fsync d3/d4c/fa5 0 2021-11-03T17:15:37.419 INFO:tasks.workunit.client.1.smithi115.stdout:2/890: dwrite d4/d6/d13/f58 [0,4194304] 0 2021-11-03T17:15:37.424 INFO:tasks.workunit.client.1.smithi115.stdout:8/928: rename d4/d7d/l13d to d4/d6/d18/d30/d79/d7f/d122/l147 0 2021-11-03T17:15:37.429 INFO:tasks.workunit.client.1.smithi115.stdout:2/891: link d4/d6/d13/f58 d4/d6/d56/d6c/d34/d42/d8d/dfd/d116/f12e 0 2021-11-03T17:15:37.431 INFO:tasks.workunit.client.1.smithi115.stdout:8/929: symlink d4/d6/d18/d30/d79/d7f/d11a/df9/l148 0 2021-11-03T17:15:37.431 INFO:tasks.workunit.client.1.smithi115.stdout:8/930: dread - d4/d6/d18/d30/fbe zero size 2021-11-03T17:15:37.432 INFO:tasks.workunit.client.1.smithi115.stdout:8/931: mknod d4/d6/d3f/da4/da6/dcc/d13f/c149 1 2021-11-03T17:15:37.432 INFO:tasks.workunit.client.1.smithi115.stdout:2/892: creat d4/d6/d56/f12f x:0 0 0 2021-11-03T17:15:37.435 INFO:tasks.workunit.client.1.smithi115.stdout:8/932: mkdir d4/d6/d3f/da4/da6/dcc/d14a 0 2021-11-03T17:15:37.439 INFO:tasks.workunit.client.1.smithi115.stdout:9/931: sync 2021-11-03T17:15:37.440 INFO:tasks.workunit.client.1.smithi115.stdout:6/894: sync 2021-11-03T17:15:37.444 INFO:tasks.workunit.client.1.smithi115.stdout:2/893: dread d4/d6/f2a [0,4194304] 0 2021-11-03T17:15:37.445 INFO:tasks.workunit.client.1.smithi115.stdout:9/932: mkdir d1/d1a/d48/d27/d2f/d41/d14e/d6e/db5/d150 0 2021-11-03T17:15:37.445 INFO:tasks.workunit.client.1.smithi115.stdout:9/933: truncate d1/d1a/d48/d27/d2f/d41/f12a 769857 0 2021-11-03T17:15:37.445 INFO:tasks.workunit.client.1.smithi115.stdout:7/994: dwrite d3/da/d18/d73/d35/fad [0,4194304] 0 2021-11-03T17:15:37.447 INFO:tasks.workunit.client.1.smithi115.stdout:6/895: rename d0/d1/d2/l7d to d0/d1/d10/l126 0 2021-11-03T17:15:37.448 INFO:tasks.workunit.client.1.smithi115.stdout:2/894: chown d4/d44/l74 56754 1 2021-11-03T17:15:37.458 INFO:tasks.workunit.client.1.smithi115.stdout:9/934: rmdir d1/d1a/d48/d27/d2f/d41/d14e/d6e/db5 39 2021-11-03T17:15:37.458 INFO:tasks.workunit.client.1.smithi115.stdout:9/935: mknod d1/d18/d2b/c151 1 2021-11-03T17:15:37.459 INFO:tasks.workunit.client.1.smithi115.stdout:9/936: write d1/d1a/d48/d27/d53/d83/d88/d9e/f3b [4542411,20000] 0 2021-11-03T17:15:37.459 INFO:tasks.workunit.client.1.smithi115.stdout:9/937: write d1/d1a/dd9/f109 [853382,18769] 0 2021-11-03T17:15:37.460 INFO:tasks.workunit.client.1.smithi115.stdout:7/995: creat d3/da/d18/d73/dda/f13b x:0 0 0 2021-11-03T17:15:37.461 INFO:tasks.workunit.client.1.smithi115.stdout:8/933: dwrite d4/d6/d3f/da4/da6/dcc/ff6 [0,4194304] 0 2021-11-03T17:15:37.461 INFO:tasks.workunit.client.1.smithi115.stdout:8/934: write d4/d6/d3f/d9a/d11b/d47/deb/f13c [553550,89828] 0 2021-11-03T17:15:37.462 INFO:tasks.workunit.client.1.smithi115.stdout:8/935: dread - d4/fb9 zero size 2021-11-03T17:15:37.462 INFO:tasks.workunit.client.1.smithi115.stdout:6/896: truncate d0/f79 1449996 0 2021-11-03T17:15:37.463 INFO:tasks.workunit.client.1.smithi115.stdout:2/895: mknod d4/d6/d56/d6c/d34/d42/d8d/c130 1 2021-11-03T17:15:37.464 INFO:tasks.workunit.client.1.smithi115.stdout:2/896: chown d4/d6/dad/dc2/l11c 22 1 2021-11-03T17:15:37.470 INFO:tasks.workunit.client.1.smithi115.stdout:8/936: rmdir d4/d6/d3f/d9a/d10b 39 2021-11-03T17:15:37.471 INFO:tasks.workunit.client.1.smithi115.stdout:2/897: symlink d4/d6/dad/de8/dac/d12d/l131 0 2021-11-03T17:15:37.477 INFO:tasks.workunit.client.1.smithi115.stdout:9/938: write d1/d18/d4a/fc8 [646538,94983] 0 2021-11-03T17:15:37.478 INFO:tasks.workunit.client.1.smithi115.stdout:8/937: mknod d4/d6/d3f/d9a/c14b 1 2021-11-03T17:15:37.479 INFO:tasks.workunit.client.1.smithi115.stdout:6/897: dwrite d0/d1/d2/df3/d107/ff6 [0,4194304] 0 2021-11-03T17:15:37.480 INFO:tasks.workunit.client.1.smithi115.stdout:2/898: write d4/d6/dad/de8/d82/ff8 [297873,37669] 0 2021-11-03T17:15:37.484 INFO:tasks.workunit.client.1.smithi115.stdout:7/996: rmdir d3/d4c/d108 0 2021-11-03T17:15:37.485 INFO:tasks.workunit.client.1.smithi115.stdout:7/997: write d3/d4c/f4d [1781949,12197] 0 2021-11-03T17:15:37.493 INFO:tasks.workunit.client.1.smithi115.stdout:9/939: mknod d1/d18/d4a/d74/c152 1 2021-11-03T17:15:37.493 INFO:tasks.workunit.client.1.smithi115.stdout:9/940: stat d1/d1a/d48/d27/d80/dcc 0 2021-11-03T17:15:37.494 INFO:tasks.workunit.client.1.smithi115.stdout:8/938: chown d4/d6/d18/d30/d79/d7f/fa1 116 1 2021-11-03T17:15:37.494 INFO:tasks.workunit.client.1.smithi115.stdout:8/939: mknod d4/d6/d3f/d43/d88/de3/c14c 1 2021-11-03T17:15:37.508 INFO:tasks.workunit.client.1.smithi115.stdout:2/899: creat d4/d6/dad/d7f/d11f/f132 x:0 0 0 2021-11-03T17:15:37.510 INFO:tasks.workunit.client.1.smithi115.stdout:7/998: link d3/da/d18/d73/dda/d133/fc1 d3/df7/f13c 0 2021-11-03T17:15:37.516 INFO:tasks.workunit.client.1.smithi115.stdout:9/941: dwrite d1/d1a/d48/d27/d53/d83/d88/dcd/d79/fad [0,4194304] 0 2021-11-03T17:15:37.516 INFO:tasks.workunit.client.1.smithi115.stdout:9/942: fsync d1/d1a/d48/d27/d53/d83/d88/d9e/f90 0 2021-11-03T17:15:37.518 INFO:tasks.workunit.client.1.smithi115.stdout:7/999: rename d3/f47 to d3/da/d18/d9c/da1/ded/f13d 0 2021-11-03T17:15:37.522 INFO:tasks.workunit.client.1.smithi115.stdout:8/940: link d4/d6/d18/d9d/fba d4/d6/d18/f14d 0 2021-11-03T17:15:37.522 INFO:tasks.workunit.client.1.smithi115.stdout:8/941: fsync d4/d6/d3f/d9a/d11b/dcd/dd0/f130 0 2021-11-03T17:15:37.522 INFO:tasks.workunit.client.1.smithi115.stdout:8/942: chown d4/d6/d3f/d43/d88 200 1 2021-11-03T17:15:37.525 INFO:tasks.workunit.client.1.smithi115.stdout:8/943: symlink d4/d6/d18/d30/d79/d11e/l14e 0 2021-11-03T17:15:37.526 INFO:tasks.workunit.client.1.smithi115.stdout:2/900: dread d4/d6/d56/d6c/d34/d43/d63/d69/fbf [4194304,4194304] 0 2021-11-03T17:15:37.531 INFO:tasks.workunit.client.1.smithi115.stdout:6/898: dwrite d0/d1/d10/d13/f39 [0,4194304] 0 2021-11-03T17:15:37.535 INFO:tasks.workunit.client.1.smithi115.stdout:8/944: rmdir d4/d6/d3f/da4/da6 39 2021-11-03T17:15:37.536 INFO:tasks.workunit.client.1.smithi115.stdout:6/899: read d0/f41 [397480,110319] 0 2021-11-03T17:15:37.539 INFO:tasks.workunit.client.1.smithi115.stdout:6/900: creat d0/d1/d2/d8c/f127 x:0 0 0 2021-11-03T17:15:37.539 INFO:tasks.workunit.client.1.smithi115.stdout:6/901: write d0/d1/fba [857100,24160] 0 2021-11-03T17:15:37.540 INFO:tasks.workunit.client.1.smithi115.stdout:6/902: dread - d0/d1/d10/d13/d1c/d1e/fbf zero size 2021-11-03T17:15:37.540 INFO:tasks.workunit.client.1.smithi115.stdout:8/945: write d4/d6/d3f/da4/da6/fc7 [7242869,54397] 0 2021-11-03T17:15:37.541 INFO:tasks.workunit.client.1.smithi115.stdout:8/946: write d4/d6/d18/d30/fbe [138427,88779] 0 2021-11-03T17:15:37.542 INFO:tasks.workunit.client.1.smithi115.stdout:6/903: mkdir d0/d1/d2/d3f/deb/d4c/d96/d128 0 2021-11-03T17:15:37.542 INFO:tasks.workunit.client.1.smithi115.stdout:8/947: chown d4/d6/d18/d30/d79/lcb 854 1 2021-11-03T17:15:37.545 INFO:tasks.workunit.client.1.smithi115.stdout:6/904: mkdir d0/d1/d6a/d88/da0/d129 0 2021-11-03T17:15:37.547 INFO:tasks.workunit.client.1.smithi115.stdout:6/905: creat d0/d1/d2/d3f/deb/d4c/d96/d128/f12a x:0 0 0 2021-11-03T17:15:37.549 INFO:tasks.workunit.client.1.smithi115.stdout:9/943: dwrite d1/d66/fc0 [0,4194304] 0 2021-11-03T17:15:37.549 INFO:tasks.workunit.client.1.smithi115.stdout:6/906: symlink d0/d1/d10/d13/d1c/d1e/de6/l12b 0 2021-11-03T17:15:37.550 INFO:tasks.workunit.client.1.smithi115.stdout:6/907: fdatasync d0/d1/d10/d13/d1c/fe2 0 2021-11-03T17:15:37.560 INFO:tasks.workunit.client.1.smithi115.stdout:8/948: dwrite d4/d6/d3f/d9a/d11b/d47/deb/f138 [0,4194304] 0 2021-11-03T17:15:37.561 INFO:tasks.workunit.client.1.smithi115.stdout:8/949: truncate d4/d6/d3f/d9a/d11b/d47/fd8 333951 0 2021-11-03T17:15:37.563 INFO:tasks.workunit.client.1.smithi115.stdout:9/944: mkdir d1/d1a/d48/d27/d2f/d41/d153 0 2021-11-03T17:15:37.566 INFO:tasks.workunit.client.1.smithi115.stdout:9/945: read d1/f72 [2101979,63748] 0 2021-11-03T17:15:37.569 INFO:tasks.workunit.client.1.smithi115.stdout:8/950: getdents d4/d6/d3f/da4/da6 0 2021-11-03T17:15:37.569 INFO:tasks.workunit.client.1.smithi115.stdout:9/946: read d1/d1a/d48/d27/d53/d83/d88/d9e/d58/fa4 [4827219,38419] 0 2021-11-03T17:15:37.572 INFO:tasks.workunit.client.1.smithi115.stdout:6/908: dwrite d0/d1/d2/d3f/deb/d4c/dca/f111 [0,4194304] 0 2021-11-03T17:15:37.573 INFO:tasks.workunit.client.1.smithi115.stdout:6/909: mknod d0/d1/d10/d13/d59/d95/c12c 1 2021-11-03T17:15:37.573 INFO:tasks.workunit.client.1.smithi115.stdout:6/910: dread - d0/d1/d6a/d88/da0/db5/f125 zero size 2021-11-03T17:15:37.574 INFO:tasks.workunit.client.1.smithi115.stdout:9/947: creat d1/d1a/d48/d27/d2f/d41/d14e/d6e/db5/d150/f154 x:0 0 0 2021-11-03T17:15:37.574 INFO:tasks.workunit.client.1.smithi115.stdout:9/948: write d1/d1a/d48/d27/d2f/d41/d14e/d6e/db5/d150/f154 [415641,73733] 0 2021-11-03T17:15:37.575 INFO:tasks.workunit.client.1.smithi115.stdout:6/911: write d0/d1/d2/d3f/deb/d4c/dca/f111 [4054708,11903] 0 2021-11-03T17:15:37.576 INFO:tasks.workunit.client.1.smithi115.stdout:8/951: rename d4/d6/d3f/d43/d88/de3/l120 to d4/d6/d18/d30/d79/d7f/l14f 0 2021-11-03T17:15:37.577 INFO:tasks.workunit.client.1.smithi115.stdout:8/952: rename d4/d6 to d4/d6/d3f/d9a/d11b/d6e/d150 22 2021-11-03T17:15:37.577 INFO:tasks.workunit.client.1.smithi115.stdout:9/949: write d1/d8a/f94 [3033855,76339] 0 2021-11-03T17:15:37.583 INFO:tasks.workunit.client.1.smithi115.stdout:6/912: creat d0/d1/d6a/d88/da0/db5/f12d x:0 0 0 2021-11-03T17:15:37.584 INFO:tasks.workunit.client.1.smithi115.stdout:6/913: write d0/d1/d2/d3f/deb/fb9 [3251686,24735] 0 2021-11-03T17:15:37.585 INFO:tasks.workunit.client.1.smithi115.stdout:6/914: write d0/d1/f19 [4187649,64843] 0 2021-11-03T17:15:37.586 INFO:tasks.workunit.client.1.smithi115.stdout:2/901: dwrite d4/d6/d56/d6c/f6f [4194304,4194304] 0 2021-11-03T17:15:37.588 INFO:tasks.workunit.client.1.smithi115.stdout:6/915: rename d0/d1/d10/d13/d87/lc5 to d0/d1/d10/d13/l12e 0 2021-11-03T17:15:37.588 INFO:tasks.workunit.client.1.smithi115.stdout:6/916: chown d0/d1/d6a/d88/dd8 734 1 2021-11-03T17:15:37.588 INFO:tasks.workunit.client.1.smithi115.stdout:6/917: write d0/fe5 [945760,61923] 0 2021-11-03T17:15:37.597 INFO:tasks.workunit.client.1.smithi115.stdout:8/953: dwrite d4/d6/fa9 [4194304,4194304] 0 2021-11-03T17:15:37.598 INFO:tasks.workunit.client.1.smithi115.stdout:6/918: dread d0/d1/fb [4194304,4194304] 0 2021-11-03T17:15:37.598 INFO:tasks.workunit.client.1.smithi115.stdout:6/919: read d0/d1/d6a/d88/da0/db5/fbb [1437631,72199] 0 2021-11-03T17:15:37.599 INFO:tasks.workunit.client.1.smithi115.stdout:6/920: read d0/f40 [1363982,84375] 0 2021-11-03T17:15:37.623 INFO:tasks.workunit.client.1.smithi115.stdout:8/954: symlink d4/d6/d18/d30/dfc/l151 0 2021-11-03T17:15:37.627 INFO:tasks.workunit.client.1.smithi115.stdout:2/902: dwrite d4/d6/dad/d7f/f8e [0,4194304] 0 2021-11-03T17:15:37.652 INFO:tasks.workunit.client.1.smithi115.stdout:2/903: dwrite d4/d6/d13/d75/fda [0,4194304] 0 2021-11-03T17:15:37.656 INFO:tasks.workunit.client.1.smithi115.stdout:2/904: write d4/d6/d56/d6c/d34/fcb [7388537,94087] 0 2021-11-03T17:15:37.659 INFO:tasks.workunit.client.1.smithi115.stdout:2/905: dread d4/d6/dad/de8/dac/dec/ff0 [0,4194304] 0 2021-11-03T17:15:37.663 INFO:tasks.workunit.client.1.smithi115.stdout:6/921: creat d0/d1/d2/d3f/deb/d4c/d96/d61/f12f x:0 0 0 2021-11-03T17:15:37.664 INFO:tasks.workunit.client.1.smithi115.stdout:6/922: truncate d0/d85/d8f/f119 305456 0 2021-11-03T17:15:37.693 INFO:tasks.workunit.client.1.smithi115.stdout:8/955: dwrite d4/d6/d3f/d9a/d11b/f48 [0,4194304] 0 2021-11-03T17:15:37.694 INFO:tasks.workunit.client.1.smithi115.stdout:6/923: dwrite d0/d1/d10/d13/d87/d89/fac [0,4194304] 0 2021-11-03T17:15:37.699 INFO:tasks.workunit.client.1.smithi115.stdout:6/924: dread d0/d1/d6a/d88/dd8/f11f [0,4194304] 0 2021-11-03T17:15:37.699 INFO:tasks.workunit.client.1.smithi115.stdout:8/956: mknod d4/d6/d3f/d9a/c152 1 2021-11-03T17:15:37.700 INFO:tasks.workunit.client.1.smithi115.stdout:8/957: stat d4/d6/d3f/d9a/f54 0 2021-11-03T17:15:37.701 INFO:tasks.workunit.client.1.smithi115.stdout:6/925: write d0/d1/d10/d13/d1c/f38 [1575874,124938] 0 2021-11-03T17:15:37.701 INFO:tasks.workunit.client.1.smithi115.stdout:6/926: write d0/d1/d10/d13/f90 [5289215,38817] 0 2021-11-03T17:15:37.701 INFO:tasks.workunit.client.1.smithi115.stdout:6/927: chown d0/f4d 281 1 2021-11-03T17:15:37.711 INFO:tasks.workunit.client.1.smithi115.stdout:6/928: mkdir d0/d1/d6a/d88/da0/db5/d130 0 2021-11-03T17:15:37.712 INFO:tasks.workunit.client.1.smithi115.stdout:2/906: rename d4/d6/dad to d4/d6/d56/d6c/d34/d42/d8d/dfd/d116/d133 0 2021-11-03T17:15:37.713 INFO:tasks.workunit.client.1.smithi115.stdout:6/929: rmdir d0/d1/d2/d3f 39 2021-11-03T17:15:37.716 INFO:tasks.workunit.client.1.smithi115.stdout:2/907: write d4/d6/f48 [4345117,68768] 0 2021-11-03T17:15:37.717 INFO:tasks.workunit.client.1.smithi115.stdout:6/930: truncate d0/d1/d2/d3f/deb/fb9 1509301 0 2021-11-03T17:15:37.717 INFO:tasks.workunit.client.1.smithi115.stdout:6/931: fdatasync d0/d7f/fcd 0 2021-11-03T17:15:37.724 INFO:tasks.workunit.client.1.smithi115.stdout:6/932: getdents d0/d1/d2/d8c 0 2021-11-03T17:15:37.725 INFO:tasks.workunit.client.1.smithi115.stdout:8/958: dwrite d4/d6/d18/f3d [0,4194304] 0 2021-11-03T17:15:37.726 INFO:tasks.workunit.client.1.smithi115.stdout:8/959: chown d4/d6/d3f/d9a/d11b/dcd/dd0 14026740 1 2021-11-03T17:15:37.726 INFO:tasks.workunit.client.1.smithi115.stdout:6/933: write d0/d1/d10/d13/f90 [1293282,76805] 0 2021-11-03T17:15:37.727 INFO:tasks.workunit.client.1.smithi115.stdout:2/908: mkdir d4/d6/d56/d6c/d34/d42/d8d/dfd/d116/d133/de8/dac/db4/d134 0 2021-11-03T17:15:37.728 INFO:tasks.workunit.client.1.smithi115.stdout:2/909: write d4/d6/d56/d6c/d34/d42/d8d/dfd/d116/d133/d7f/d11f/f132 [677014,1438] 0 2021-11-03T17:15:37.759 INFO:tasks.workunit.client.1.smithi115.stdout:6/934: dwrite d0/d1/d10/d13/d87/d89/faf [0,4194304] 0 2021-11-03T17:15:37.759 INFO:tasks.workunit.client.1.smithi115.stdout:6/935: stat d0/l5a 0 2021-11-03T17:15:37.760 INFO:tasks.workunit.client.1.smithi115.stdout:6/936: mknod d0/d1/d6a/d88/da0/c131 1 2021-11-03T17:15:37.760 INFO:tasks.workunit.client.1.smithi115.stdout:8/960: dwrite d4/d6/d18/d30/d79/d7f/d11a/d109/f10c [0,4194304] 0 2021-11-03T17:15:37.763 INFO:tasks.workunit.client.1.smithi115.stdout:6/937: creat d0/d1/d10/d13/d59/d95/d108/f132 x:0 0 0 2021-11-03T17:15:37.765 INFO:tasks.workunit.client.1.smithi115.stdout:2/910: write d4/d6/d13/f21 [1037247,35152] 0 2021-11-03T17:15:37.767 INFO:tasks.workunit.client.1.smithi115.stdout:8/961: creat d4/d6/d3f/d9a/d11b/d47/deb/f153 x:0 0 0 2021-11-03T17:15:37.779 INFO:tasks.workunit.client.1.smithi115.stdout:6/938: rename d0/d1/d10/d13/d1c/l46 to d0/d1/d10/d13/d59/d95/d108/l133 0 2021-11-03T17:15:37.779 INFO:tasks.workunit.client.1.smithi115.stdout:2/911: rename d4 to d4/d6/d56/d6c/d34/d42/d8d/dfd/d116/d133/d7f/de9/d135 22 2021-11-03T17:15:37.784 INFO:tasks.workunit.client.1.smithi115.stdout:6/939: unlink d0/d1/d2/d3f/lb1 0 2021-11-03T17:15:37.785 INFO:tasks.workunit.client.1.smithi115.stdout:8/962: dwrite d4/d6/d18/d133/f144 [0,4194304] 0 2021-11-03T17:15:37.787 INFO:tasks.workunit.client.1.smithi115.stdout:6/940: link d0/d1/d6a/d88/da0/db5/fbb d0/d1/d6a/f134 0 2021-11-03T17:15:37.803 INFO:tasks.workunit.client.1.smithi115.stdout:8/963: dwrite d4/d6/d3f/d9a/d11b/dcd/ff7 [0,4194304] 0 2021-11-03T17:15:37.818 INFO:tasks.workunit.client.1.smithi115.stdout:8/964: creat d4/d6/d18/d133/f154 x:0 0 0 2021-11-03T17:15:37.818 INFO:tasks.workunit.client.1.smithi115.stdout:8/965: readlink d4/d6/d18/d30/d79/d7f/d11a/d109/l142 0 2021-11-03T17:15:37.826 INFO:tasks.workunit.client.1.smithi115.stdout:6/941: mkdir d0/d1/d2/d3f/deb/d4c/d96/d61/d135 0 2021-11-03T17:15:37.826 INFO:tasks.workunit.client.1.smithi115.stdout:6/942: stat d0/d1/d2/d3f/deb/d4c/d96/f6b 0 2021-11-03T17:15:37.829 INFO:tasks.workunit.client.1.smithi115.stdout:6/943: unlink d0/d1/d2/d3f/deb/d4c/d96/d61/f8a 0 2021-11-03T17:15:37.833 INFO:tasks.workunit.client.1.smithi115.stdout:6/944: creat d0/d1/d2/df3/f136 x:0 0 0 2021-11-03T17:15:37.833 INFO:tasks.workunit.client.1.smithi115.stdout:8/966: dwrite d4/d6/d18/d30/d79/d7f/d11a/df9/f104 [4194304,4194304] 0 2021-11-03T17:15:37.837 INFO:tasks.workunit.client.1.smithi115.stdout:8/967: rmdir d4/d6/d3f/da4/da6/dcc 39 2021-11-03T17:15:37.840 INFO:tasks.workunit.client.1.smithi115.stdout:6/945: mkdir d0/d1/d10/d13/d59/d137 0 2021-11-03T17:15:37.840 INFO:tasks.workunit.client.1.smithi115.stdout:6/946: truncate d0/d1/d6a/f134 4772768 0 2021-11-03T17:15:37.840 INFO:tasks.workunit.client.1.smithi115.stdout:6/947: dread - d0/d1/d10/d13/d59/f80 zero size 2021-11-03T17:15:37.841 INFO:tasks.workunit.client.1.smithi115.stdout:6/948: chown d0/d1/d2/d3f/deb 0 1 2021-11-03T17:15:37.841 INFO:tasks.workunit.client.1.smithi115.stdout:6/949: write d0/d1/d2/d3f/deb/d4c/d96/d61/f113 [463665,16619] 0 2021-11-03T17:15:37.856 INFO:tasks.workunit.client.1.smithi115.stdout:6/950: dwrite d0/d1/d10/d13/d1c/f36 [4194304,4194304] 0 2021-11-03T17:15:37.861 INFO:tasks.workunit.client.1.smithi115.stdout:6/951: rename d0/d1/d10/d13/d87/fa4 to d0/d1/d2/d3f/deb/d4c/d96/d61/f138 0 2021-11-03T17:15:37.864 INFO:tasks.workunit.client.1.smithi115.stdout:6/952: creat d0/d1/d10/d13/d59/f139 x:0 0 0 2021-11-03T17:15:37.866 INFO:tasks.workunit.client.1.smithi115.stdout:6/953: creat d0/d1/d2/d3f/deb/d4c/d96/d128/f13a x:0 0 0 2021-11-03T17:15:37.871 INFO:tasks.workunit.client.1.smithi115.stdout:6/954: link d0/d1/d2/f1f d0/d1/d10/d13/d1c/f13b 0 2021-11-03T17:15:37.872 INFO:tasks.workunit.client.1.smithi115.stdout:6/955: read d0/d74/fc8 [3078686,43533] 0 2021-11-03T17:15:37.874 INFO:tasks.workunit.client.1.smithi115.stdout:6/956: mknod d0/d1/d10/c13c 1 2021-11-03T17:15:37.874 INFO:tasks.workunit.client.1.smithi115.stdout:6/957: fsync d0/d74/fc8 0 2021-11-03T17:15:37.874 INFO:tasks.workunit.client.1.smithi115.stdout:6/958: readlink d0/d1/l30 0 2021-11-03T17:15:37.876 INFO:tasks.workunit.client.1.smithi115.stdout:6/959: creat d0/d1/d2/d3f/deb/d4c/f13d x:0 0 0 2021-11-03T17:15:37.894 INFO:tasks.workunit.client.1.smithi115.stdout:6/960: write d0/d1/fe [2702800,88717] 0 2021-11-03T17:15:37.894 INFO:tasks.workunit.client.1.smithi115.stdout:6/961: fsync d0/d1/f10c 0 2021-11-03T17:15:37.894 INFO:tasks.workunit.client.1.smithi115.stdout:6/962: write d0/d1/d10/d13/d59/f80 [139067,60298] 0 2021-11-03T17:15:37.920 INFO:tasks.workunit.client.1.smithi115.stdout:6/963: dwrite d0/f41 [0,4194304] 0 2021-11-03T17:15:37.920 INFO:tasks.workunit.client.1.smithi115.stdout:6/964: write d0/d74/fd5 [492164,9582] 0 2021-11-03T17:15:37.936 INFO:tasks.workunit.client.1.smithi115.stdout:6/965: dwrite d0/d1/d10/d13/d59/d95/d108/f132 [0,4194304] 0 2021-11-03T17:15:37.937 INFO:tasks.workunit.client.1.smithi115.stdout:6/966: chown d0/d1/d10/d13/f72 7222558 1 2021-11-03T17:15:37.937 INFO:tasks.workunit.client.1.smithi115.stdout:6/967: mknod d0/d1/d2/d8c/c13e 1 2021-11-03T17:15:37.937 INFO:tasks.workunit.client.1.smithi115.stdout:6/968: mknod d0/d1/d2/df3/d107/d102/d105/c13f 1 2021-11-03T17:15:37.941 INFO:tasks.workunit.client.1.smithi115.stdout:6/969: creat d0/f140 x:0 0 0 2021-11-03T17:15:37.954 INFO:tasks.workunit.client.1.smithi115.stdout:6/970: mkdir d0/d1/d2/df3/d141 0 2021-11-03T17:15:37.956 INFO:tasks.workunit.client.1.smithi115.stdout:6/971: mkdir d0/d1/d10/d13/d87/dd2/d142 0 2021-11-03T17:15:37.959 INFO:tasks.workunit.client.1.smithi115.stdout:6/972: symlink d0/d74/dae/db3/l143 0 2021-11-03T17:15:37.962 INFO:tasks.workunit.client.1.smithi115.stdout:6/973: truncate d0/d1/d6a/d88/da0/db5/fbb 875698 0 2021-11-03T17:15:37.964 INFO:tasks.workunit.client.1.smithi115.stdout:6/974: truncate d0/d1/d6a/d88/dd8/f11f 1879021 0 2021-11-03T17:15:37.968 INFO:tasks.workunit.client.1.smithi115.stdout:6/975: creat d0/d1/d10/d13/f144 x:0 0 0 2021-11-03T17:15:37.969 INFO:tasks.workunit.client.1.smithi115.stdout:6/976: stat d0/d1/d10/d13/f144 0 2021-11-03T17:15:37.969 INFO:tasks.workunit.client.1.smithi115.stdout:6/977: write d0/d1/d2/df3/ff1 [465931,77736] 0 2021-11-03T17:15:37.978 INFO:tasks.workunit.client.1.smithi115.stdout:6/978: dread d0/d1/d2/f58 [0,4194304] 0 2021-11-03T17:15:37.979 INFO:tasks.workunit.client.1.smithi115.stdout:6/979: write d0/d1/d2/d3f/deb/d4c/dca/f111 [714043,5865] 0 2021-11-03T17:15:37.982 INFO:tasks.workunit.client.1.smithi115.stdout:6/980: creat d0/d85/d8f/dfa/f145 x:0 0 0 2021-11-03T17:15:37.983 INFO:tasks.workunit.client.1.smithi115.stdout:6/981: chown d0/d1/d2/d3f/deb/d4c/d96/l6c 45190 1 2021-11-03T17:15:38.010 INFO:tasks.workunit.client.1.smithi115.stdout:6/982: dwrite d0/d1/fe [4194304,4194304] 0 2021-11-03T17:15:38.010 INFO:tasks.workunit.client.1.smithi115.stdout:6/983: chown d0/d1/d2/d8c 374 1 2021-11-03T17:15:38.034 INFO:tasks.workunit.client.1.smithi115.stdout:6/984: dwrite d0/d1/d10/d13/d87/d89/faf [0,4194304] 0 2021-11-03T17:15:38.034 INFO:tasks.workunit.client.1.smithi115.stdout:6/985: mknod d0/d1/d2/df3/d141/c146 1 2021-11-03T17:15:38.036 INFO:tasks.workunit.client.1.smithi115.stdout:6/986: unlink d0/f140 0 2021-11-03T17:15:38.037 INFO:tasks.workunit.client.1.smithi115.stdout:6/987: stat d0/d74/dae/db3/l143 0 2021-11-03T17:15:38.038 INFO:tasks.workunit.client.1.smithi115.stdout:6/988: dread d0/d1/d6a/d88/da0/db5/fc3 [4194304,4194304] 0 2021-11-03T17:15:38.041 INFO:tasks.workunit.client.1.smithi115.stdout:6/989: rename d0/d1/d2/l121 to d0/d1/d6a/d88/dd8/l147 0 2021-11-03T17:15:38.041 INFO:tasks.workunit.client.1.smithi115.stdout:6/990: write d0/d1/d2/d3f/deb/d4c/d96/f6b [1783579,6618] 0 2021-11-03T17:15:38.041 INFO:tasks.workunit.client.1.smithi115.stdout:6/991: write d0/d1/d2/d3f/deb/d4c/d96/d61/f138 [1313514,11363] 0 2021-11-03T17:15:38.440 INFO:tasks.workunit.client.1.smithi115.stdout:9/950: sync 2021-11-03T17:15:38.441 INFO:tasks.workunit.client.1.smithi115.stdout:9/951: write d1/d1a/d48/d27/d2f/fc6 [837663,12329] 0 2021-11-03T17:15:38.441 INFO:tasks.workunit.client.1.smithi115.stdout:9/952: readlink d1/d1a/d48/d27/l40 0 2021-11-03T17:15:38.442 INFO:tasks.workunit.client.1.smithi115.stdout:9/953: creat d1/d18/d4a/d74/f155 x:0 0 0 2021-11-03T17:15:38.442 INFO:tasks.workunit.client.1.smithi115.stdout:9/954: chown d1/d1a/d48/d27/f4c 935429 1 2021-11-03T17:15:38.447 INFO:tasks.workunit.client.1.smithi115.stdout:9/955: unlink d1/d18/d4a/fc8 0 2021-11-03T17:15:38.447 INFO:tasks.workunit.client.1.smithi115.stdout:9/956: write d1/d18/d2b/fd0 [953621,62715] 0 2021-11-03T17:15:38.448 INFO:tasks.workunit.client.1.smithi115.stdout:9/957: mknod d1/d18/d2b/db9/df2/c156 1 2021-11-03T17:15:38.450 INFO:tasks.workunit.client.1.smithi115.stdout:9/958: creat d1/d18/d2b/d45/dc2/f157 x:0 0 0 2021-11-03T17:15:38.453 INFO:tasks.workunit.client.1.smithi115.stdout:9/959: truncate d1/d1a/f28 2985111 0 2021-11-03T17:15:38.454 INFO:tasks.workunit.client.1.smithi115.stdout:9/960: write d1/d1a/d48/d27/d2f/d7b/fef [1037808,72936] 0 2021-11-03T17:15:38.454 INFO:tasks.workunit.client.1.smithi115.stdout:9/961: write d1/d18/d2b/db9/fdb [2014659,89757] 0 2021-11-03T17:15:38.462 INFO:tasks.workunit.client.1.smithi115.stdout:9/962: dread d1/d1a/d48/f2a [0,4194304] 0 2021-11-03T17:15:38.465 INFO:tasks.workunit.client.1.smithi115.stdout:9/963: dread d1/d1a/f131 [0,4194304] 0 2021-11-03T17:15:38.465 INFO:tasks.workunit.client.1.smithi115.stdout:9/964: write d1/d18/d4a/d74/d8b/dbc/fe1 [5067122,37392] 0 2021-11-03T17:15:38.465 INFO:tasks.workunit.client.1.smithi115.stdout:9/965: write d1/d1a/d48/d27/ff7 [62572,59246] 0 2021-11-03T17:15:38.465 INFO:tasks.workunit.client.1.smithi115.stdout:9/966: dread - d1/d18/d4a/d74/f14d zero size 2021-11-03T17:15:38.466 INFO:tasks.workunit.client.1.smithi115.stdout:9/967: truncate d1/d1a/d48/d27/d53/d83/d88/dcd/d79/fad 436125 0 2021-11-03T17:15:38.475 INFO:tasks.workunit.client.1.smithi115.stdout:9/968: symlink d1/d1a/d48/d27/d2f/d41/l158 0 2021-11-03T17:15:38.475 INFO:tasks.workunit.client.1.smithi115.stdout:9/969: chown d1/d1a/dd9 341994574 1 2021-11-03T17:15:38.475 INFO:tasks.workunit.client.1.smithi115.stdout:9/970: stat d1/d1a/d48/d27/d2f/d41/d14e/d6e/db5/d150/f154 0 2021-11-03T17:15:38.475 INFO:tasks.workunit.client.1.smithi115.stdout:9/971: fdatasync d1/d1a/d48/d27/f4c 0 2021-11-03T17:15:38.476 INFO:tasks.workunit.client.1.smithi115.stdout:9/972: readlink d1/l10b 0 2021-11-03T17:15:38.479 INFO:tasks.workunit.client.1.smithi115.stdout:9/973: truncate d1/d18/d4a/d74/fdd 2386414 0 2021-11-03T17:15:38.484 INFO:tasks.workunit.client.1.smithi115.stdout:9/974: mkdir d1/d18/d2b/d45/dc2/d159 0 2021-11-03T17:15:38.511 INFO:tasks.workunit.client.1.smithi115.stdout:9/975: dwrite d1/d18/d4a/d74/d8b/faa [0,4194304] 0 2021-11-03T17:15:38.511 INFO:tasks.workunit.client.1.smithi115.stdout:9/976: fdatasync d1/d1a/d48/d27/f4c 0 2021-11-03T17:15:38.516 INFO:tasks.workunit.client.1.smithi115.stdout:9/977: creat d1/d1a/d48/d27/d53/d83/d88/dcd/dda/f15a x:0 0 0 2021-11-03T17:15:38.516 INFO:tasks.workunit.client.1.smithi115.stdout:9/978: dread - d1/d1a/d48/d27/d2f/d6a/f110 zero size 2021-11-03T17:15:38.535 INFO:tasks.workunit.client.1.smithi115.stdout:9/979: dwrite d1/d1a/d48/d27/d2f/d41/d14e/f21 [0,4194304] 0 2021-11-03T17:15:38.556 INFO:tasks.workunit.client.1.smithi115.stdout:9/980: symlink d1/d18/d2b/l15b 0 2021-11-03T17:15:38.568 INFO:tasks.workunit.client.1.smithi115.stdout:9/981: dread d1/d1a/d48/d27/f3d [0,4194304] 0 2021-11-03T17:15:38.599 INFO:tasks.workunit.client.1.smithi115.stdout:9/982: dwrite d1/d18/d4a/d74/fdd [0,4194304] 0 2021-11-03T17:15:38.600 INFO:tasks.workunit.client.1.smithi115.stdout:9/983: stat d1/d1a/d48/d27/d2f/d6a/f110 0 2021-11-03T17:15:38.600 INFO:tasks.workunit.client.1.smithi115.stdout:9/984: write d1/d1a/d48/d27/d2f/d41/f12a [951860,68754] 0 2021-11-03T17:15:38.608 INFO:tasks.workunit.client.1.smithi115.stdout:2/912: sync 2021-11-03T17:15:38.635 INFO:tasks.workunit.client.1.smithi115.stdout:9/985: dwrite d1/d1a/d48/d27/d53/f77 [0,4194304] 0 2021-11-03T17:15:38.636 INFO:tasks.workunit.client.1.smithi115.stdout:9/986: mknod d1/ddf/c15c 1 2021-11-03T17:15:38.643 INFO:tasks.workunit.client.1.smithi115.stdout:9/987: truncate d1/d18/d2b/f35 88703 0 2021-11-03T17:15:38.643 INFO:tasks.workunit.client.1.smithi115.stdout:9/988: dread - d1/fae zero size 2021-11-03T17:15:38.651 INFO:tasks.workunit.client.1.smithi115.stdout:2/913: dwrite d4/d6/f2a [0,4194304] 0 2021-11-03T17:15:38.656 INFO:tasks.workunit.client.1.smithi115.stdout:2/914: symlink d4/d6/d13/d75/l136 0 2021-11-03T17:15:38.659 INFO:tasks.workunit.client.1.smithi115.stdout:2/915: truncate d4/d6/d56/d6c/d34/d42/d8d/dfd/d116/d133/de8/d82/ff8 706070 0 2021-11-03T17:15:38.659 INFO:tasks.workunit.client.1.smithi115.stdout:9/989: dwrite d1/d18/d2b/d45/dc2/f13f [0,4194304] 0 2021-11-03T17:15:38.670 INFO:tasks.workunit.client.1.smithi115.stdout:9/990: link d1/d18/d4a/d74/d8b/dbc/fe1 d1/d18/d2b/d45/dc2/d159/f15d 0 2021-11-03T17:15:38.679 INFO:tasks.workunit.client.1.smithi115.stdout:9/991: rmdir d1/d1a/dd9 39 2021-11-03T17:15:38.688 INFO:tasks.workunit.client.1.smithi115.stdout:2/916: dwrite d4/d6/d56/d6c/d34/d42/d8d/dfd/d116/d133/de8/d82/ff8 [0,4194304] 0 2021-11-03T17:15:38.693 INFO:tasks.workunit.client.1.smithi115.stdout:8/968: sync 2021-11-03T17:15:38.693 INFO:tasks.workunit.client.1.smithi115.stdout:8/969: write d4/d6/d18/d30/d33/d93/f132 [213410,39942] 0 2021-11-03T17:15:38.709 INFO:tasks.workunit.client.1.smithi115.stdout:2/917: rename d4/d6/d56/d6c/d34/d42/d8d/dfd/d116/d133/l59 to d4/d6/d56/d6c/d34/d43/d63/d77/l137 0 2021-11-03T17:15:38.710 INFO:tasks.workunit.client.1.smithi115.stdout:9/992: dwrite d1/fae [0,4194304] 0 2021-11-03T17:15:38.710 INFO:tasks.workunit.client.1.smithi115.stdout:9/993: truncate d1/d1a/d48/d27/d2f/fc6 1630634 0 2021-11-03T17:15:38.733 INFO:tasks.workunit.client.1.smithi115.stdout:9/994: mkdir d1/d18/d2b/db9/df2/d15e 0 2021-11-03T17:15:38.734 INFO:tasks.workunit.client.1.smithi115.stdout:9/995: chown d1/d1a/d48/d27/d53/d83/d88/dcd/d79/fad 987716714 1 2021-11-03T17:15:38.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:38 smithi115 conmon[34670]: cluster 2021-11-03T17:15:36.648852+0000 mgr.smithi027.ufhhvb ( 2021-11-03T17:15:38.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:38 smithi115 conmon[34670]: mgr.14622) 36 : cluster [DBG] pgmap v22: 65 pgs: 65 active+clean; 2.1 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 37 MiB/s rd, 67 MiB/s wr, 207 op/s 2021-11-03T17:15:38.735 INFO:tasks.workunit.client.1.smithi115.stdout:2/918: dwrite d4/d6/d56/d6c/d34/d42/d8d/dfd/d116/d133/dc2/fe4 [0,4194304] 0 2021-11-03T17:15:38.736 INFO:tasks.workunit.client.1.smithi115.stdout:8/970: dwrite d4/d6/d3f/f61 [0,4194304] 0 2021-11-03T17:15:38.738 INFO:tasks.workunit.client.1.smithi115.stdout:9/996: mkdir d1/d1a/d48/d27/d80/da7/d15f 0 2021-11-03T17:15:38.738 INFO:tasks.workunit.client.1.smithi115.stdout:9/997: write d1/d18/d2b/db9/df2/f115 [845740,33600] 0 2021-11-03T17:15:38.738 INFO:tasks.workunit.client.1.smithi115.stdout:9/998: read - d1/d1a/d48/d27/d2f/d6a/f110 zero size 2021-11-03T17:15:38.739 INFO:tasks.workunit.client.1.smithi115.stdout:2/919: creat d4/d6/d56/d6c/d34/d42/d8d/dfd/d116/d133/de8/dac/d12d/f138 x:0 0 0 2021-11-03T17:15:38.746 INFO:tasks.workunit.client.1.smithi115.stdout:2/920: dread d4/d6/f2a [0,4194304] 0 2021-11-03T17:15:38.746 INFO:tasks.workunit.client.1.smithi115.stdout:2/921: read - d4/d6/d56/d6c/d34/d84/fe7 zero size 2021-11-03T17:15:38.748 INFO:tasks.workunit.client.1.smithi115.stdout:8/971: getdents d4/d6/d18/d30/d79/d7f/d122 0 2021-11-03T17:15:38.764 INFO:tasks.workunit.client.1.smithi115.stdout:9/999: unlink d1/d1a/d48/d27/d53/d83/d88/d9e/f6d 0 2021-11-03T17:15:38.768 INFO:tasks.workunit.client.1.smithi115.stdout:8/972: write d4/d6/d18/d30/d79/d7f/fae [311265,91288] 0 2021-11-03T17:15:38.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:38 smithi027 conmon[32322]: cluster 2021-11-03T17:15:36. 2021-11-03T17:15:38.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:38 smithi027 conmon[32322]: 648852+0000 mgr.smithi027.ufhhvb (mgr.14622) 36 : cluster [DBG] pgmap v22: 65 pgs: 65 active+clean; 2.1 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 37 MiB/s rd, 67 MiB/s wr, 207 op/s 2021-11-03T17:15:38.772 INFO:tasks.workunit.client.1.smithi115.stdout:8/973: mknod d4/d6/d18/d30/d33/d93/c155 1 2021-11-03T17:15:38.774 INFO:tasks.workunit.client.1.smithi115.stdout:2/922: rename d4/l23 to d4/d6/d56/d6c/d34/d42/l139 0 2021-11-03T17:15:38.778 INFO:tasks.workunit.client.1.smithi115.stdout:8/974: unlink d4/d6/d18/f3d 0 2021-11-03T17:15:38.781 INFO:tasks.workunit.client.1.smithi115.stdout:8/975: symlink d4/d6/l156 0 2021-11-03T17:15:38.794 INFO:tasks.workunit.client.1.smithi115.stdout:2/923: write d4/d6/d56/d6c/d34/d43/d63/d77/fa6 [535811,12612] 0 2021-11-03T17:15:38.796 INFO:tasks.workunit.client.1.smithi115.stdout:2/924: fsync d4/d6/d56/d6c/d34/d42/d8d/dfd/d116/d133/f60 0 2021-11-03T17:15:38.798 INFO:tasks.workunit.client.1.smithi115.stdout:2/925: getdents d4/d6/d56/d6c/d34/d119 0 2021-11-03T17:15:38.802 INFO:tasks.workunit.client.1.smithi115.stdout:2/926: truncate d4/d6/d56/d6c/d34/d42/d8d/dfd/d116/d133/d7f/feb 4056763 0 2021-11-03T17:15:38.802 INFO:tasks.workunit.client.1.smithi115.stdout:8/976: dwrite d4/d6/d3f/da4/da6/fc6 [0,4194304] 0 2021-11-03T17:15:38.806 INFO:tasks.workunit.client.1.smithi115.stdout:2/927: creat d4/d6/d56/d6c/d34/d43/d28/f13a x:0 0 0 2021-11-03T17:15:38.810 INFO:tasks.workunit.client.1.smithi115.stdout:8/977: creat d4/d6/d3f/d9a/d11b/d47/f157 x:0 0 0 2021-11-03T17:15:38.812 INFO:tasks.workunit.client.1.smithi115.stdout:2/928: unlink d4/d6/d56/d6c/d34/d42/d8d/dfd/d116/d133/d7f/de9/f113 0 2021-11-03T17:15:38.820 INFO:tasks.workunit.client.1.smithi115.stdout:8/978: rename d4/d6/d18/d30/d33/ld7 to d4/d6/d3f/dde/l158 0 2021-11-03T17:15:38.826 INFO:tasks.workunit.client.1.smithi115.stdout:8/979: creat d4/d6/d18/d133/f159 x:0 0 0 2021-11-03T17:15:38.827 INFO:tasks.workunit.client.1.smithi115.stdout:8/980: mknod d4/d6/d3f/da4/da6/dcc/d13f/c15a 1 2021-11-03T17:15:38.827 INFO:tasks.workunit.client.1.smithi115.stdout:8/981: write d4/d6/d3f/da4/dfd/f126 [290212,33077] 0 2021-11-03T17:15:38.831 INFO:tasks.workunit.client.1.smithi115.stdout:2/929: dwrite d4/d6/d13/d75/fd5 [0,4194304] 0 2021-11-03T17:15:38.849 INFO:tasks.workunit.client.1.smithi115.stdout:2/930: rename d4/d44/l74 to d4/l13b 0 2021-11-03T17:15:38.851 INFO:tasks.workunit.client.1.smithi115.stdout:6/992: sync 2021-11-03T17:15:38.854 INFO:tasks.workunit.client.1.smithi115.stdout:2/931: creat d4/d6/d56/d6c/d34/d42/d8d/dfd/d116/d133/d7f/f13c x:0 0 0 2021-11-03T17:15:38.855 INFO:tasks.workunit.client.1.smithi115.stdout:6/993: creat d0/d1/d2/d8c/f148 x:0 0 0 2021-11-03T17:15:38.863 INFO:tasks.workunit.client.1.smithi115.stdout:8/982: dwrite d4/f28 [0,4194304] 0 2021-11-03T17:15:38.863 INFO:tasks.workunit.client.1.smithi115.stdout:6/994: getdents d0/d1/d10/d13/d59/d95/db4 0 2021-11-03T17:15:38.864 INFO:tasks.workunit.client.1.smithi115.stdout:6/995: read - d0/d85/d8f/dfa/f145 zero size 2021-11-03T17:15:38.864 INFO:tasks.workunit.client.1.smithi115.stdout:6/996: truncate d0/d74/dae/db3/f109 1419810 0 2021-11-03T17:15:38.877 INFO:tasks.workunit.client.1.smithi115.stdout:6/997: symlink d0/d1/d10/d13/d87/dd2/dd7/l149 0 2021-11-03T17:15:38.885 INFO:tasks.workunit.client.1.smithi115.stdout:8/983: dwrite d4/fbd [0,4194304] 0 2021-11-03T17:15:38.885 INFO:tasks.workunit.client.1.smithi115.stdout:8/984: fdatasync d4/d6/d3f/d9a/d11b/dcd/dd0/f124 0 2021-11-03T17:15:38.894 INFO:tasks.workunit.client.1.smithi115.stdout:8/985: rename d4/d6/d18/d30/d79/d7f/ff1 to d4/d6/d3f/d9a/d11b/dcd/dd0/f15b 0 2021-11-03T17:15:38.896 INFO:tasks.workunit.client.1.smithi115.stdout:6/998: dwrite d0/d1/d10/d13/d59/f67 [0,4194304] 0 2021-11-03T17:15:38.898 INFO:tasks.workunit.client.1.smithi115.stdout:2/932: dwrite d4/d6/d56/d6c/d34/d43/d28/f10e [0,4194304] 0 2021-11-03T17:15:38.908 INFO:tasks.workunit.client.1.smithi115.stdout:6/999: mkdir d0/d1/d10/d13/d1c/d1e/de6/d14a 0 2021-11-03T17:15:38.913 INFO:tasks.workunit.client.1.smithi115.stdout:8/986: rmdir d4/d6/d18/d9d 39 2021-11-03T17:15:38.914 INFO:tasks.workunit.client.1.smithi115.stdout:8/987: chown d4/d6/d18/d30/ld1 35 1 2021-11-03T17:15:38.915 INFO:tasks.workunit.client.1.smithi115.stdout:2/933: dwrite d4/d6/d56/d6c/d34/d42/d8d/dfd/d116/d133/d7f/d11f/f132 [0,4194304] 0 2021-11-03T17:15:38.923 INFO:tasks.workunit.client.1.smithi115.stdout:2/934: creat d4/d6/d56/d6c/d34/d43/d63/f13d x:0 0 0 2021-11-03T17:15:38.924 INFO:tasks.workunit.client.1.smithi115.stdout:2/935: mknod d4/d6/d13/d108/d112/c13e 1 2021-11-03T17:15:38.924 INFO:tasks.workunit.client.1.smithi115.stdout:2/936: chown d4/d6/d56/d6c/d34/d42/d8d/dfd/d116/f12e 8 1 2021-11-03T17:15:38.924 INFO:tasks.workunit.client.1.smithi115.stdout:8/988: write d4/d6/d18/d30/d79/fb2 [811301,27931] 0 2021-11-03T17:15:38.930 INFO:tasks.workunit.client.1.smithi115.stdout:2/937: truncate d4/d6/d56/d6c/f46 1803651 0 2021-11-03T17:15:38.931 INFO:tasks.workunit.client.1.smithi115.stdout:2/938: write d4/d6/d56/d6c/d34/d42/d8d/dfd/d116/d133/de8/d9c/f12a [736464,103056] 0 2021-11-03T17:15:38.942 INFO:tasks.workunit.client.1.smithi115.stdout:8/989: dwrite d4/d6/d18/d30/d33/f5c [0,4194304] 0 2021-11-03T17:15:38.942 INFO:tasks.workunit.client.1.smithi115.stdout:2/939: dread d4/d6/d56/d6c/d34/d43/d63/fdf [0,4194304] 0 2021-11-03T17:15:38.944 INFO:tasks.workunit.client.1.smithi115.stdout:8/990: unlink d4/d6/d18/d30/d33/l40 0 2021-11-03T17:15:38.944 INFO:tasks.workunit.client.1.smithi115.stdout:2/940: mknod d4/d6/d13/d75/c13f 1 2021-11-03T17:15:38.948 INFO:tasks.workunit.client.1.smithi115.stdout:2/941: unlink d4/d6/d56/d6c/d34/d43/d28/f13a 0 2021-11-03T17:15:38.953 INFO:tasks.workunit.client.1.smithi115.stdout:2/942: creat d4/d6/d56/f140 x:0 0 0 2021-11-03T17:15:38.959 INFO:tasks.workunit.client.1.smithi115.stdout:8/991: write d4/d6/d18/d9d/fba [285334,74836] 0 2021-11-03T17:15:38.962 INFO:tasks.workunit.client.1.smithi115.stdout:8/992: symlink d4/d6/d18/d9d/l15c 0 2021-11-03T17:15:38.963 INFO:tasks.workunit.client.1.smithi115.stdout:2/943: dread d4/d6/fbd [0,4194304] 0 2021-11-03T17:15:38.965 INFO:tasks.workunit.client.1.smithi115.stdout:8/993: unlink d4/d6/d3f/d9a/d11b/d6e/f9f 0 2021-11-03T17:15:38.965 INFO:tasks.workunit.client.1.smithi115.stdout:8/994: mknod d4/d6/d3f/d9a/d10b/c15d 1 2021-11-03T17:15:38.970 INFO:tasks.workunit.client.1.smithi115.stdout:2/944: symlink d4/d6/d56/d6c/d34/d42/l141 0 2021-11-03T17:15:38.993 INFO:tasks.workunit.client.1.smithi115.stdout:8/995: dwrite d4/d6/d3f/d9a/d11b/d47/f157 [0,4194304] 0 2021-11-03T17:15:38.995 INFO:tasks.workunit.client.1.smithi115.stdout:2/945: dwrite d4/d6/d56/d6c/d34/d42/d8d/dfd/d116/d133/dc2/fdb [4194304,4194304] 0 2021-11-03T17:15:38.998 INFO:tasks.workunit.client.1.smithi115.stdout:8/996: mknod d4/d6/d3f/d9a/d11b/c15e 1 2021-11-03T17:15:39.018 INFO:tasks.workunit.client.1.smithi115.stdout:2/946: dwrite d4/d6/f48 [4194304,4194304] 0 2021-11-03T17:15:39.020 INFO:tasks.workunit.client.1.smithi115.stdout:8/997: rename d4/d7d/l11c to d4/d6/d3f/d9a/d10b/l15f 0 2021-11-03T17:15:39.024 INFO:tasks.workunit.client.1.smithi115.stdout:2/947: creat d4/d6/d56/d6c/d34/d42/d8d/dfd/d116/d133/de8/d9c/dbe/f142 x:0 0 0 2021-11-03T17:15:39.024 INFO:tasks.workunit.client.1.smithi115.stdout:2/948: fdatasync d4/d6/d13/d75/fda 0 2021-11-03T17:15:39.030 INFO:tasks.workunit.client.1.smithi115.stdout:8/998: creat d4/d6/d3f/da4/da6/dcc/d13f/f160 x:0 0 0 2021-11-03T17:15:39.039 INFO:tasks.workunit.client.1.smithi115.stdout:8/999: mkdir d4/d6/d18/d30/d79/d7f/d122/d161 0 2021-11-03T17:15:39.046 INFO:tasks.workunit.client.1.smithi115.stdout:2/949: dwrite d4/d6/d56/d6c/d34/d43/d63/d77/fa6 [0,4194304] 0 2021-11-03T17:15:39.049 INFO:tasks.workunit.client.1.smithi115.stdout:2/950: readlink d4/l2c 0 2021-11-03T17:15:39.051 INFO:tasks.workunit.client.1.smithi115.stdout:2/951: mknod d4/d6/d56/d6c/d34/d42/d8d/dfd/d116/d133/de8/dac/c143 1 2021-11-03T17:15:39.051 INFO:tasks.workunit.client.1.smithi115.stdout:2/952: readlink d4/d6/d56/d6c/d34/d42/d8d/dfd/d116/d133/dc2/l11c 0 2021-11-03T17:15:39.052 INFO:tasks.workunit.client.1.smithi115.stdout:2/953: mknod d4/d6/d56/d6c/d34/d42/d8d/dfd/c144 1 2021-11-03T17:15:39.058 INFO:tasks.workunit.client.1.smithi115.stdout:2/954: dread d4/d6/d56/d6c/f6f [4194304,4194304] 0 2021-11-03T17:15:39.062 INFO:tasks.workunit.client.1.smithi115.stdout:2/955: rename d4/d6/d56/d6c/d34/d42/d8d/dfd/d116/d133/de8/d82/ff8 to d4/d6/d56/d6c/d34/d43/d28/f145 0 2021-11-03T17:15:39.063 INFO:tasks.workunit.client.1.smithi115.stdout:2/956: mknod d4/d6/d56/d6c/d34/d42/d8d/dfd/c146 1 2021-11-03T17:15:39.066 INFO:tasks.workunit.client.1.smithi115.stdout:2/957: creat d4/d6/d56/d6c/d34/d42/d8d/dfd/d116/f147 x:0 0 0 2021-11-03T17:15:39.073 INFO:tasks.workunit.client.1.smithi115.stdout:2/958: dread d4/d6/d56/d6c/d34/d42/d8d/dfd/d116/d133/dc2/fe4 [0,4194304] 0 2021-11-03T17:15:39.711 INFO:tasks.workunit.client.1.smithi115.stdout:2/959: sync 2021-11-03T17:15:39.713 INFO:tasks.workunit.client.1.smithi115.stdout:2/960: creat d4/d6/d56/d6c/d34/f148 x:0 0 0 2021-11-03T17:15:39.714 INFO:tasks.workunit.client.1.smithi115.stdout:2/961: mknod d4/d6/d56/d6c/d34/d43/d28/c149 1 2021-11-03T17:15:39.717 INFO:tasks.workunit.client.1.smithi115.stdout:2/962: creat d4/d44/f14a x:0 0 0 2021-11-03T17:15:39.719 INFO:tasks.workunit.client.1.smithi115.stdout:2/963: mkdir d4/d6/d56/d6c/d34/d42/d8d/dfd/d116/d133/de8/d9c/d9f/d14b 0 2021-11-03T17:15:39.730 INFO:tasks.workunit.client.1.smithi115.stdout:2/964: dread d4/d6/d56/d6c/d34/d42/d8d/dfd/d116/d133/d7f/fa5 [0,4194304] 0 2021-11-03T17:15:39.736 INFO:tasks.workunit.client.1.smithi115.stdout:2/965: fsync d4/d6/d56/d6c/f39 0 2021-11-03T17:15:39.739 INFO:tasks.workunit.client.1.smithi115.stdout:2/966: write d4/d6/f45 [4644852,82234] 0 2021-11-03T17:15:39.742 INFO:tasks.workunit.client.1.smithi115.stdout:2/967: dread d4/d6/d56/d6c/d34/d42/d8d/dfd/d116/d133/d7f/fc0 [0,4194304] 0 2021-11-03T17:15:39.742 INFO:tasks.workunit.client.1.smithi115.stdout:2/968: getdents d4/d6/d56/d6c/d34/d42/d8d/dfd/d116/d133/de8/dac/db4/d134 0 2021-11-03T17:15:40.172 INFO:tasks.workunit.client.1.smithi115.stdout:2/969: sync 2021-11-03T17:15:40.172 INFO:tasks.workunit.client.1.smithi115.stdout:2/970: stat d4/d6/d56/d6c/d34/d42/d8d/da9 0 2021-11-03T17:15:40.199 INFO:tasks.workunit.client.1.smithi115.stdout:2/971: truncate d4/d6/d56/d6c/d34/fcb 8791341 0 2021-11-03T17:15:40.199 INFO:tasks.workunit.client.1.smithi115.stdout:2/972: fsync d4/d6/d56/d6c/d34/d43/d63/d77/f81 0 2021-11-03T17:15:40.199 INFO:tasks.workunit.client.1.smithi115.stdout:2/973: readlink d4/d6/d56/d6c/d34/d42/l141 0 2021-11-03T17:15:40.200 INFO:tasks.workunit.client.1.smithi115.stdout:2/974: write d4/d6/d13/f58 [383783,78311] 0 2021-11-03T17:15:40.204 INFO:tasks.workunit.client.1.smithi115.stdout:2/975: creat d4/d6/d56/d6c/d34/d43/d63/d69/f14c x:0 0 0 2021-11-03T17:15:40.362 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:40 smithi027 conmon[32322]: cluster 2021-11-03T17:15:38 2021-11-03T17:15:40.363 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:40 smithi027 conmon[32322]: .649437+0000 mgr.smithi027.ufhhvb (mgr.14622) 37 : cluster [DBG] pgmap v23: 65 pgs: 65 active+clean; 2.1 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 50 MiB/s rd, 92 MiB/s wr, 298 op/s 2021-11-03T17:15:40.363 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:40 smithi027 conmon[32322]: audit 2021-11-03T17:15:39.598711+0000 mon.smithi027 (mon.0) 733 : audit [INF] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:15:40.363 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:40 smithi027 conmon[32322]: audit 2021-11-03T17:15:39.599475+0000 mon.smithi027 (mon.0) 734 : audit [INF] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-11-03T17:15:40.364 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:40 smithi027 conmon[32322]: audit 2021-11-03T17:15:39.600206+0000 mon.smithi027 (mon.0) 735 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-11-03T17:15:40.364 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:40 smithi027 conmon[32322]: audit 2021-11-03T17:15:39.600902+0000 mon.smithi027 (mon.0) 736 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:15:40.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:40 smithi115 conmon[34670]: cluster 2021-11-03T17:15:38.649437+0000 mgr.smithi027.ufhhvb (mgr.14622) 37 : cluster [DBG] pgmap v23: 65 pgs: 65 active+clean; 2.1 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 50 MiB/s rd, 92 MiB/s wr, 298 op/s 2021-11-03T17:15:40.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:40 smithi115 conmon[34670]: audit 2021-11-03T17:15:39.598711+0000 mon.smithi027 (mon.0) 733 : audit [INF] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:15:40.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:40 smithi115 conmon[34670]: audit 2021-11-03T17:15:39.599475+0000 mon.smithi027 (mon.0) 734 : audit [INF] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-11-03T17:15:40.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:40 smithi115 conmon[34670]: audit 2021-11-03T17:15:39.600206+0000 mon.smithi027 (mon.0) 735 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-11-03T17:15:40.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:40 smithi115 conmon[34670]: audit 2021-11-03T17:15:39.600902+0000 mon.smithi027 (mon.0) 736 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:15:40.790 INFO:tasks.workunit.client.1.smithi115.stdout:2/976: sync 2021-11-03T17:15:40.791 INFO:tasks.workunit.client.1.smithi115.stdout:2/977: read d4/d6/d56/d6c/d34/d42/d8d/dfd/d116/d133/de8/d9c/f11d [170459,26038] 0 2021-11-03T17:15:40.795 INFO:tasks.workunit.client.1.smithi115.stdout:2/978: symlink d4/d6/d56/d6c/d34/l14d 0 2021-11-03T17:15:40.796 INFO:tasks.workunit.client.1.smithi115.stdout:2/979: dread - d4/d6/d56/d6c/d34/d84/fe7 zero size 2021-11-03T17:15:40.799 INFO:tasks.workunit.client.1.smithi115.stdout:2/980: creat d4/d6/d56/d6c/d34/d42/d8d/dfd/d116/d133/d7f/de9/f14e x:0 0 0 2021-11-03T17:15:40.821 INFO:tasks.workunit.client.1.smithi115.stdout:2/981: dwrite d4/d6/d56/d6c/d34/d43/d63/d77/f81 [0,4194304] 0 2021-11-03T17:15:40.826 INFO:tasks.workunit.client.1.smithi115.stdout:2/982: unlink d4/d6/d56/d6c/d34/d43/d63/d77/l137 0 2021-11-03T17:15:41.151 INFO:tasks.workunit.client.1.smithi115.stdout:2/983: sync 2021-11-03T17:15:41.159 INFO:tasks.workunit.client.1.smithi115.stdout:2/984: rename d4/d6/d56/d6c/d34/d43/d28 to d4/d6/d56/d6c/d34/d42/d8d/dfd/d116/d133/de8/dac/db4/d134/d14f 0 2021-11-03T17:15:41.179 INFO:tasks.workunit.client.1.smithi115.stdout:2/985: dwrite d4/d6/d56/d6c/d34/d42/f8f [0,4194304] 0 2021-11-03T17:15:41.179 INFO:tasks.workunit.client.1.smithi115.stdout:2/986: chown d4/d6/d56/d6c/d34/d84/l11a 2271992 1 2021-11-03T17:15:41.180 INFO:tasks.workunit.client.1.smithi115.stdout:2/987: write d4/d6/d56/d6c/d34/fcb [6789034,128138] 0 2021-11-03T17:15:41.195 INFO:tasks.workunit.client.1.smithi115.stdout:2/988: dread d4/d6/d56/d6c/d34/d43/d63/f99 [0,4194304] 0 2021-11-03T17:15:41.211 INFO:tasks.workunit.client.1.smithi115.stdout:2/989: rename d4/d6/d56/d6c/d34/d42/d8d/dfd/d116 to d4/d6/d56/d6c/d34/d119/d150 0 2021-11-03T17:15:41.212 INFO:tasks.workunit.client.1.smithi115.stdout:2/990: chown d4/d6/d56/d6c/d34/d119/d150/d133/d7f/l96 26 1 2021-11-03T17:15:41.218 INFO:tasks.workunit.client.1.smithi115.stdout:2/991: rename d4/d6/d56/d6c/d34/d119/d150/d133/de8/dac/db4/d134/d14f/f3b to d4/d6/d56/f151 0 2021-11-03T17:15:41.219 INFO:tasks.workunit.client.1.smithi115.stdout:2/992: creat d4/d6/d56/d6c/d34/d119/d150/d133/d7f/d11f/f152 x:0 0 0 2021-11-03T17:15:41.228 INFO:tasks.workunit.client.1.smithi115.stdout:2/993: dread d4/d6/f45 [0,4194304] 0 2021-11-03T17:15:41.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:41 smithi115 conmon[34670]: cephadm 2021-11-03T17:15:39 2021-11-03T17:15:41.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:41 smithi115 conmon[34670]: .567069+0000 mgr.smithi027.ufhhvb (mgr.14622) 38 : cephadm [INF] poll took 7414.928 ms: 1 events 2021-11-03T17:15:41.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:41 smithi115 conmon[34670]: cephadm 2021-11-03T17:15:39.594436+0000 mgr.smithi027.ufhhvb (mgr.14622) 39 : cephadm [INF] Upgrade: Updating mon.smithi027 2021-11-03T17:15:41.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:41 smithi115 conmon[34670]: cephadm 2021-11-03T17:15:39.601563+0000 mgr.smithi027.ufhhvb (mgr.14622 2021-11-03T17:15:41.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:41 smithi115 conmon[34670]: ) 40 : cephadm [INF] Deploying daemon mon.smithi027 on smithi027 2021-11-03T17:15:41.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:41 smithi027 conmon[32322]: cephadm 2021-11-03T17:15:39.567069+0000 mgr.smithi027.ufhhvb (mgr.14622 2021-11-03T17:15:41.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:41 smithi027 conmon[32322]: ) 38 : cephadm [INF] poll took 7414.928 ms: 1 events 2021-11-03T17:15:41.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:41 smithi027 conmon[32322]: cephadm 2021-11-03T17:15:39.594436+0000 mgr.smithi027.ufhhvb (mgr.14622) 39 : cephadm [INF] Upgrade: Updating mon.smithi027 2021-11-03T17:15:41.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:41 smithi027 conmon[32322]: cephadm 2021- 2021-11-03T17:15:41.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:41 smithi027 conmon[32322]: 11-03T17:15:39 2021-11-03T17:15:41.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:41 smithi027 conmon[32322]: .601563+0000 mgr.smithi027.ufhhvb (mgr.14622 2021-11-03T17:15:41.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:41 smithi027 conmon[32322]: ) 40 : cephadm [INF] Deploying daemon mon.smithi027 on smithi027 2021-11-03T17:15:41.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:41 smithi027 conmon[32322]: 2021-11-03T17:15:41.950 INFO:tasks.workunit.client.1.smithi115.stdout:2/994: sync 2021-11-03T17:15:42.048 INFO:tasks.workunit.client.1.smithi115.stdout:2/995: mkdir d4/d6/d13/d153 0 2021-11-03T17:15:42.090 INFO:tasks.workunit.client.1.smithi115.stdout:2/996: creat d4/d6/d56/d6c/d11b/f154 x:0 0 0 2021-11-03T17:15:42.092 INFO:tasks.workunit.client.1.smithi115.stdout:2/997: link d4/d6/d13/fb2 d4/d6/d13/d153/f155 0 2021-11-03T17:15:42.120 INFO:tasks.workunit.client.1.smithi115.stdout:2/998: dwrite d4/d6/d56/d6c/d34/fcb [4194304,4194304] 0 2021-11-03T17:15:42.120 INFO:tasks.workunit.client.1.smithi115.stdout:2/999: truncate d4/d6/d56/d6c/d34/d43/d63/d69/f14c 442901 0 2021-11-03T17:15:42.337 INFO:tasks.workunit.client.1.smithi115.stderr:+ rm -rf -- ./tmp.ibnEjVpTuX 2021-11-03T17:15:42.408 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:42 smithi027 systemd[1]: Stopping Ceph mon.smithi027 for 32ab977e-3cc8-11ec-8c28-001a4aab830c... 2021-11-03T17:15:42.409 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:42 smithi027 conmon[32322]: cluster 2021-11-03T17:15:40.649875 2021-11-03T17:15:42.409 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:42 smithi027 conmon[32322]: +0000 mgr.smithi027.ufhhvb (mgr.14622) 41 : cluster [DBG] pgmap v24: 65 pgs: 65 active+clean; 2.1 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 31 MiB/s rd, 62 MiB/s wr, 190 op/s 2021-11-03T17:15:42.409 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:42 smithi027 conmon[32322]: debug 2021-11-03T17:15:42.412+0000 7ff1bf8ca700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.smithi027 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2021-11-03T17:15:42.410 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:42 smithi027 conmon[32322]: debug 2021-11-03T17:15:42.412+0000 7ff1bf8ca700 -1 mon.smithi027@0(leader) e2 *** Got Signal Terminated *** 2021-11-03T17:15:42.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:42 smithi115 conmon[34670]: cluster 2021-11-03T17:15:40.649875+0000 2021-11-03T17:15:42.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:42 smithi115 conmon[34670]: mgr.smithi027.ufhhvb (mgr.14622) 41 : cluster [DBG] pgmap v24: 65 pgs: 65 active+clean; 2.1 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 31 MiB/s rd, 62 MiB/s wr, 190 op/s 2021-11-03T17:15:43.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:42 smithi027 bash[96813]: 060d5938214f76bc057d33ac29fdfd199f0136f33cb61527d59b00179ee25f36 2021-11-03T17:15:43.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:42 smithi027 bash[96813]: Error: no container with name or ID ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c-mon-smithi027 found: no such container 2021-11-03T17:15:44.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:43 smithi027 bash[96813]: Error: container 060d5938214f76bc057d33ac29fdfd199f0136f33cb61527d59b00179ee25f36 does not exist in database: no such container 2021-11-03T17:15:44.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:43 smithi027 systemd[1]: ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c@mon.smithi027.service: Succeeded. 2021-11-03T17:15:44.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:43 smithi027 systemd[1]: Stopped Ceph mon.smithi027 for 32ab977e-3cc8-11ec-8c28-001a4aab830c. 2021-11-03T17:15:44.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:43 smithi027 systemd[1]: Starting Ceph mon.smithi027 for 32ab977e-3cc8-11ec-8c28-001a4aab830c... 2021-11-03T17:15:44.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 bash[97021]: ce77ef590f9efdbf9c06ab9f87c3cfff8cf5bc9991bd335986ddd1a30f1cbc9f 2021-11-03T17:15:44.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 systemd[1]: Started Ceph mon.smithi027 for 32ab977e-3cc8-11ec-8c28-001a4aab830c. 2021-11-03T17:15:44.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: set uid:gid to 167:167 (ceph:ceph) 2021-11-03T17:15:44.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev), process ceph-mon, pid 8 2021-11-03T17:15:44.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: pidfile_write: ignore empty --pid-file 2021-11-03T17:15:44.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: load: jerasure load: lrc 2021-11-03T17:15:44.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: RocksDB version: 6.15.5 2021-11-03T17:15:44.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2021-11-03T17:15:44.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Compile date Nov 3 2021 2021-11-03T17:15:44.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: DB SUMMARY 2021-11-03T17:15:44.776 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: DB Session ID: 6L1WNZYAKHTZ3N4J2ADH 2021-11-03T17:15:44.777 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: CURRENT file: CURRENT 2021-11-03T17:15:44.777 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: IDENTITY file: IDENTITY 2021-11-03T17:15:44.778 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: MANIFEST file: MANIFEST-000008 size: 779 Bytes 2021-11-03T17:15:44.778 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi027/store.db dir, Total Num: 1, files: 000020.sst 2021-11-03T17:15:44.778 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi027/store.db: 000018.log size: 69157 ; 2021-11-03T17:15:44.779 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.error_if_exists: 0 2021-11-03T17:15:44.779 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.create_if_missing: 0 2021-11-03T17:15:44.780 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.paranoid_checks: 1 2021-11-03T17:15:44.780 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2021-11-03T17:15:44.781 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.env: 0x55fb1e27fc00 2021-11-03T17:15:44.781 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.fs: Posix File System 2021-11-03T17:15:44.782 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.info_log: 0x55fb20a06460 2021-11-03T17:15:44.782 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.max_file_opening_threads: 16 2021-11-03T17:15:44.783 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.statistics: (nil) 2021-11-03T17:15:44.783 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.use_fsync: 0 2021-11-03T17:15:44.783 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.max_log_file_size: 0 2021-11-03T17:15:44.784 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.max_manifest_file_size: 1073741824 2021-11-03T17:15:44.784 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.log_file_time_to_roll: 0 2021-11-03T17:15:44.785 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.keep_log_file_num: 1000 2021-11-03T17:15:44.785 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.recycle_log_file_num: 0 2021-11-03T17:15:44.786 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.allow_fallocate: 1 2021-11-03T17:15:44.786 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.allow_mmap_reads: 0 2021-11-03T17:15:44.786 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.allow_mmap_writes: 0 2021-11-03T17:15:44.787 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.use_direct_reads: 0 2021-11-03T17:15:44.788 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2021-11-03T17:15:44.788 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.create_missing_column_families: 0 2021-11-03T17:15:44.789 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.db_log_dir: 2021-11-03T17:15:44.789 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi027/store.db 2021-11-03T17:15:44.789 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.table_cache_numshardbits: 6 2021-11-03T17:15:44.790 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.WAL_ttl_seconds: 0 2021-11-03T17:15:44.790 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.WAL_size_limit_MB: 0 2021-11-03T17:15:44.790 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2021-11-03T17:15:44.791 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.manifest_preallocation_size: 4194304 2021-11-03T17:15:44.791 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.is_fd_close_on_exec: 1 2021-11-03T17:15:44.791 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.advise_random_on_open: 1 2021-11-03T17:15:44.792 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.db_write_buffer_size: 0 2021-11-03T17:15:44.792 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.write_buffer_manager: 0x55fb20a7bf20 2021-11-03T17:15:44.792 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.access_hint_on_compaction_start: 1 2021-11-03T17:15:44.792 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2021-11-03T17:15:44.793 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.random_access_max_buffer_size: 1048576 2021-11-03T17:15:44.793 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.use_adaptive_mutex: 0 2021-11-03T17:15:44.793 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.rate_limiter: (nil) 2021-11-03T17:15:44.793 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2021-11-03T17:15:44.794 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.wal_recovery_mode: 2 2021-11-03T17:15:44.794 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.enable_thread_tracking: 0 2021-11-03T17:15:44.794 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.enable_pipelined_write: 0 2021-11-03T17:15:44.794 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.unordered_write: 0 2021-11-03T17:15:44.795 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.allow_concurrent_memtable_write: 1 2021-11-03T17:15:44.795 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2021-11-03T17:15:44.795 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.write_thread_max_yield_usec: 100 2021-11-03T17:15:44.795 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.write_thread_slow_yield_usec: 3 2021-11-03T17:15:44.796 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.row_cache: None 2021-11-03T17:15:44.796 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.wal_filter: None 2021-11-03T17:15:44.796 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.avoid_flush_during_recovery: 0 2021-11-03T17:15:44.796 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.allow_ingest_behind: 0 2021-11-03T17:15:44.797 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.preserve_deletes: 0 2021-11-03T17:15:44.797 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.two_write_queues: 0 2021-11-03T17:15:44.797 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.manual_wal_flush: 0 2021-11-03T17:15:44.798 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.atomic_flush: 0 2021-11-03T17:15:44.798 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2021-11-03T17:15:44.798 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.persist_stats_to_disk: 0 2021-11-03T17:15:44.798 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.write_dbid_to_manifest: 0 2021-11-03T17:15:44.799 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.log_readahead_size: 0 2021-11-03T17:15:44.799 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.file_checksum_gen_factory: Unknown 2021-11-03T17:15:44.799 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.best_efforts_recovery: 0 2021-11-03T17:15:44.799 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2021-11-03T17:15:44.800 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2021-11-03T17:15:44.800 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.allow_data_in_errors: 0 2021-11-03T17:15:44.800 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.db_host_id: __hostname__ 2021-11-03T17:15:44.800 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.max_background_jobs: 2 2021-11-03T17:15:44.800 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.max_background_compactions: -1 2021-11-03T17:15:44.801 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.max_subcompactions: 1 2021-11-03T17:15:44.801 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.avoid_flush_during_shutdown: 0 2021-11-03T17:15:44.801 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2021-11-03T17:15:44.802 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.delayed_write_rate : 16777216 2021-11-03T17:15:44.802 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.max_total_wal_size: 0 2021-11-03T17:15:44.802 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2021-11-03T17:15:44.803 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.stats_dump_period_sec: 600 2021-11-03T17:15:44.803 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.stats_persist_period_sec: 600 2021-11-03T17:15:44.803 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.stats_history_buffer_size: 1048576 2021-11-03T17:15:44.803 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.max_open_files: -1 2021-11-03T17:15:44.804 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.bytes_per_sync: 0 2021-11-03T17:15:44.804 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.wal_bytes_per_sync: 0 2021-11-03T17:15:44.804 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.strict_bytes_per_sync: 0 2021-11-03T17:15:44.804 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.compaction_readahead_size: 0 2021-11-03T17:15:44.805 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.max_background_flushes: -1 2021-11-03T17:15:44.805 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Compression algorithms supported: 2021-11-03T17:15:44.805 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: kZSTDNotFinalCompression supported: 0 2021-11-03T17:15:44.805 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: kZSTD supported: 0 2021-11-03T17:15:44.806 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: kXpressCompression supported: 0 2021-11-03T17:15:44.806 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: kLZ4HCCompression supported: 1 2021-11-03T17:15:44.806 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: kLZ4Compression supported: 1 2021-11-03T17:15:44.806 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: kBZip2Compression supported: 0 2021-11-03T17:15:44.807 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: kZlibCompression supported: 1 2021-11-03T17:15:44.807 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: kSnappyCompression supported: 1 2021-11-03T17:15:44.807 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Fast CRC32 supported: Supported on x86 2021-11-03T17:15:44.807 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: [db/version_set.cc:4725] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi027/store.db/MANIFEST-000008 2021-11-03T17:15:44.807 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: [db/column_family.cc:597] --------------- Options for column family [default]: 2021-11-03T17:15:44.808 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2021-11-03T17:15:44.808 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.merge_operator: 2021-11-03T17:15:44.809 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.compaction_filter: None 2021-11-03T17:15:44.809 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.compaction_filter_factory: None 2021-11-03T17:15:44.809 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.sst_partitioner_factory: None 2021-11-03T17:15:44.810 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.memtable_factory: SkipListFactory 2021-11-03T17:15:44.810 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.table_factory: BlockBasedTable 2021-11-03T17:15:44.810 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55fb209cfbc0) 2021-11-03T17:15:44.811 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: cache_index_and_filter_blocks: 1 2021-11-03T17:15:44.811 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2021-11-03T17:15:44.811 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2021-11-03T17:15:44.811 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: pin_top_level_index_and_filter: 1 2021-11-03T17:15:44.812 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: index_type: 0 2021-11-03T17:15:44.812 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: data_block_index_type: 0 2021-11-03T17:15:44.812 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: index_shortening: 1 2021-11-03T17:15:44.812 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: data_block_hash_table_util_ratio: 0.750000 2021-11-03T17:15:44.813 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: hash_index_allow_collision: 1 2021-11-03T17:15:44.813 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: checksum: 1 2021-11-03T17:15:44.813 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: no_block_cache: 0 2021-11-03T17:15:44.814 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: block_cache: 0x55fb20a04d10 2021-11-03T17:15:44.814 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: block_cache_name: BinnedLRUCache 2021-11-03T17:15:44.814 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: block_cache_options: 2021-11-03T17:15:44.814 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: capacity : 536870912 2021-11-03T17:15:44.815 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: num_shard_bits : 4 2021-11-03T17:15:44.815 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: strict_capacity_limit : 0 2021-11-03T17:15:44.815 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: high_pri_pool_ratio: 0.000 2021-11-03T17:15:44.815 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: block_cache_compressed: (nil) 2021-11-03T17:15:44.816 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: persistent_cache: (nil) 2021-11-03T17:15:44.816 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: block_size: 4096 2021-11-03T17:15:44.816 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: block_size_deviation: 10 2021-11-03T17:15:44.816 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: block_restart_interval: 16 2021-11-03T17:15:44.816 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: index_block_restart_interval: 1 2021-11-03T17:15:44.817 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: metadata_block_size: 4096 2021-11-03T17:15:44.817 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: partition_filters: 0 2021-11-03T17:15:44.817 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: use_delta_encoding: 1 2021-11-03T17:15:44.817 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2021-11-03T17:15:44.817 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: whole_key_filtering: 1 2021-11-03T17:15:44.818 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: verify_compression: 0 2021-11-03T17:15:44.818 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: read_amp_bytes_per_bit: 0 2021-11-03T17:15:44.818 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: format_version: 4 2021-11-03T17:15:44.818 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: enable_index_compression: 1 2021-11-03T17:15:44.819 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: block_align: 0 2021-11-03T17:15:44.819 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.write_buffer_size: 33554432 2021-11-03T17:15:44.819 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.max_write_buffer_number: 2 2021-11-03T17:15:44.819 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.compression: NoCompression 2021-11-03T17:15:44.819 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.bottommost_compression: Disabled 2021-11-03T17:15:44.820 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.prefix_extractor: nullptr 2021-11-03T17:15:44.820 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2021-11-03T17:15:44.820 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.num_levels: 7 2021-11-03T17:15:44.820 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2021-11-03T17:15:44.821 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2021-11-03T17:15:44.821 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2021-11-03T17:15:44.821 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2021-11-03T17:15:44.821 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.bottommost_compression_opts.level: 32767 2021-11-03T17:15:44.821 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2021-11-03T17:15:44.822 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2021-11-03T17:15:44.822 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2021-11-03T17:15:44.822 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2021-11-03T17:15:44.822 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.bottommost_compression_opts.enabled: false 2021-11-03T17:15:44.823 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.compression_opts.window_bits: -14 2021-11-03T17:15:44.823 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.compression_opts.level: 32767 2021-11-03T17:15:44.823 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.compression_opts.strategy: 0 2021-11-03T17:15:44.823 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2021-11-03T17:15:44.824 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2021-11-03T17:15:44.824 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.compression_opts.parallel_threads: 1 2021-11-03T17:15:44.824 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.compression_opts.enabled: false 2021-11-03T17:15:44.824 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2021-11-03T17:15:44.824 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2021-11-03T17:15:44.825 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.level0_stop_writes_trigger: 36 2021-11-03T17:15:44.825 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.target_file_size_base: 67108864 2021-11-03T17:15:44.825 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.target_file_size_multiplier: 1 2021-11-03T17:15:44.826 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.max_bytes_for_level_base: 268435456 2021-11-03T17:15:44.826 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2021-11-03T17:15:44.826 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2021-11-03T17:15:44.826 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2021-11-03T17:15:44.827 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2021-11-03T17:15:44.827 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2021-11-03T17:15:44.827 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2021-11-03T17:15:44.827 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2021-11-03T17:15:44.828 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2021-11-03T17:15:44.828 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2021-11-03T17:15:44.828 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2021-11-03T17:15:44.828 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.max_compaction_bytes: 1677721600 2021-11-03T17:15:44.828 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.arena_block_size: 4194304 2021-11-03T17:15:44.829 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2021-11-03T17:15:44.829 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2021-11-03T17:15:44.829 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2021-11-03T17:15:44.829 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.disable_auto_compactions: 0 2021-11-03T17:15:44.830 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2021-11-03T17:15:44.830 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2021-11-03T17:15:44.830 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2021-11-03T17:15:44.830 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2021-11-03T17:15:44.831 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2021-11-03T17:15:44.831 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2021-11-03T17:15:44.831 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2021-11-03T17:15:44.831 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2021-11-03T17:15:44.832 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2021-11-03T17:15:44.832 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2021-11-03T17:15:44.832 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.table_properties_collectors: 2021-11-03T17:15:44.832 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.inplace_update_support: 0 2021-11-03T17:15:44.832 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.inplace_update_num_locks: 10000 2021-11-03T17:15:44.833 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2021-11-03T17:15:44.833 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.memtable_whole_key_filtering: 0 2021-11-03T17:15:44.833 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.memtable_huge_page_size: 0 2021-11-03T17:15:44.833 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.bloom_locality: 0 2021-11-03T17:15:44.834 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.max_successive_merges: 0 2021-11-03T17:15:44.834 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.optimize_filters_for_hits: 0 2021-11-03T17:15:44.834 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.paranoid_file_checks: 0 2021-11-03T17:15:44.835 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.force_consistency_checks: 1 2021-11-03T17:15:44.835 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.report_bg_io_stats: 0 2021-11-03T17:15:44.835 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.ttl: 2592000 2021-11-03T17:15:44.835 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.periodic_compaction_seconds: 0 2021-11-03T17:15:44.836 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.enable_blob_files: false 2021-11-03T17:15:44.836 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.min_blob_size: 0 2021-11-03T17:15:44.836 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.blob_file_size: 268435456 2021-11-03T17:15:44.836 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.blob_compression_type: NoCompression 2021-11-03T17:15:44.837 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.enable_blob_garbage_collection: false 2021-11-03T17:15:44.837 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2021-11-03T17:15:44.837 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: [db/version_set.cc:3459] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2021-11-03T17:15:44.837 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: [db/version_set.cc:4773] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi027/store.db/MANIFEST-000008 succeeded,manifest_file_number is 8, next_file_number is 22, last_sequence is 9394, log_number is 18,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2021-11-03T17:15:44.838 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: [db/version_set.cc:4782] Column family [default] (ID 0), log number is 18 2021-11-03T17:15:44.838 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: [db/version_set.cc:4083] Creating manifest 22 2021-11-03T17:15:44.838 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: [db/version_set.cc:3459] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2021-11-03T17:15:44.838 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: EVENT_LOG_v1 {"time_micros": 1635959744388437, "job": 1, "event": "recovery_started", "wal_files": [18]} 2021-11-03T17:15:44.839 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: [db/db_impl/db_impl_open.cc:847] Recovering log #18 mode 2 2021-11-03T17:15:44.839 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: [table/block_based/filter_policy.cc:996] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2021-11-03T17:15:44.839 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: EVENT_LOG_v1 {"time_micros": 1635959744389055, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 23, "file_size": 70136, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 69066, "index_size": 74, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 311, "raw_average_key_size": 22, "raw_value_size": 68736, "raw_average_value_size": 4909, "num_data_blocks": 3, "num_entries": 14, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1635959744, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "078e4ea8-1042-4a5c-baf0-50de99772fd1", "db_session_id": "6L1WNZYAKHTZ3N4J2ADH"}} 2021-11-03T17:15:44.839 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: [db/version_set.cc:4083] Creating manifest 24 2021-11-03T17:15:44.840 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: [db/version_set.cc:3459] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2021-11-03T17:15:44.840 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: EVENT_LOG_v1 {"time_micros": 1635959744389541, "job": 1, "event": "recovery_finished"} 2021-11-03T17:15:44.840 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: [file/delete_scheduler.cc:73] Deleted file /var/lib/ceph/mon/ceph-smithi027/store.db/000018.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2021-11-03T17:15:44.841 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: [db/db_impl/db_impl_open.cc:1701] SstFileManager instance 0x55fb20a20a80 2021-11-03T17:15:44.841 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: DB pointer 0x55fb20a30000 2021-11-03T17:15:44.841 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: [db/db_impl/db_impl.cc:902] ------- DUMPING STATS ------- 2021-11-03T17:15:44.842 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: rocksdb: [db/db_impl/db_impl.cc:903] 2021-11-03T17:15:44.842 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: ** DB Stats ** 2021-11-03T17:15:44.842 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Uptime(secs): 0.0 total, 0.0 interval 2021-11-03T17:15:44.842 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2021-11-03T17:15:44.843 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2021-11-03T17:15:44.843 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2021-11-03T17:15:44.843 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2021-11-03T17:15:44.843 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2021-11-03T17:15:44.844 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2021-11-03T17:15:44.844 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: 2021-11-03T17:15:44.844 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: ** Compaction Stats [default] ** 2021-11-03T17:15:44.844 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-11-03T17:15:44.844 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-11-03T17:15:44.845 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: L0 1/0 68.49 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 169.8 0.00 0.00 1 0.000 0 0 2021-11-03T17:15:44.845 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: L6 1/0 22.67 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2021-11-03T17:15:44.845 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Sum 2/0 22.74 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 169.8 0.00 0.00 1 0.000 0 0 2021-11-03T17:15:44.845 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 169.8 0.00 0.00 1 0.000 0 0 2021-11-03T17:15:44.846 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: 2021-11-03T17:15:44.846 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: ** Compaction Stats [default] ** 2021-11-03T17:15:44.846 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-11-03T17:15:44.846 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-11-03T17:15:44.847 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 169.8 0.00 0.00 1 0.000 0 0 2021-11-03T17:15:44.847 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Uptime(secs): 0.0 total, 0.0 interval 2021-11-03T17:15:44.847 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Flush(GB): cumulative 0.000, interval 0.000 2021-11-03T17:15:44.847 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2021-11-03T17:15:44.848 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: AddFile(Total Files): cumulative 0, interval 0 2021-11-03T17:15:44.848 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: AddFile(L0 Files): cumulative 0, interval 0 2021-11-03T17:15:44.848 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: AddFile(Keys): cumulative 0, interval 0 2021-11-03T17:15:44.848 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Cumulative compaction: 0.00 GB write, 16.94 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-11-03T17:15:44.848 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Interval compaction: 0.00 GB write, 16.94 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-11-03T17:15:44.849 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2021-11-03T17:15:44.849 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: 2021-11-03T17:15:44.849 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: ** File Read Latency Histogram By Level [default] ** 2021-11-03T17:15:44.849 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: 2021-11-03T17:15:44.850 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: ** Compaction Stats [default] ** 2021-11-03T17:15:44.850 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-11-03T17:15:44.850 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-11-03T17:15:44.850 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: L0 1/0 68.49 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 169.8 0.00 0.00 1 0.000 0 0 2021-11-03T17:15:44.851 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: L6 1/0 22.67 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2021-11-03T17:15:44.851 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Sum 2/0 22.74 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 169.8 0.00 0.00 1 0.000 0 0 2021-11-03T17:15:44.851 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2021-11-03T17:15:44.851 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: 2021-11-03T17:15:44.852 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: ** Compaction Stats [default] ** 2021-11-03T17:15:44.852 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-11-03T17:15:44.852 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-11-03T17:15:44.852 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 169.8 0.00 0.00 1 0.000 0 0 2021-11-03T17:15:44.853 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Uptime(secs): 0.0 total, 0.0 interval 2021-11-03T17:15:44.853 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Flush(GB): cumulative 0.000, interval 0.000 2021-11-03T17:15:44.853 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2021-11-03T17:15:44.853 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: AddFile(Total Files): cumulative 0, interval 0 2021-11-03T17:15:44.854 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: AddFile(L0 Files): cumulative 0, interval 0 2021-11-03T17:15:44.854 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: AddFile(Keys): cumulative 0, interval 0 2021-11-03T17:15:44.854 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Cumulative compaction: 0.00 GB write, 16.79 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-11-03T17:15:44.854 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-11-03T17:15:44.854 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2021-11-03T17:15:44.855 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: 2021-11-03T17:15:44.855 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: ** File Read Latency Histogram By Level [default] ** 2021-11-03T17:15:44.855 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: starting mon.smithi027 rank 0 at public addrs [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] at bind addrs [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi027 fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c 2021-11-03T17:15:44.855 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: mon.smithi027@-1(???) e2 preinit fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c 2021-11-03T17:15:44.856 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: mon.smithi027@-1(???).mds e27 new map 2021-11-03T17:15:44.856 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: mon.smithi027@-1(???).mds e27 print_map 2021-11-03T17:15:44.856 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: e27 2021-11-03T17:15:44.856 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:15:44.856 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:15:44.857 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: legacy client fscid: 1 2021-11-03T17:15:44.857 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: 2021-11-03T17:15:44.857 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Filesystem 'cephfs' (1) 2021-11-03T17:15:44.857 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: fs_name cephfs 2021-11-03T17:15:44.858 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: epoch 27 2021-11-03T17:15:44.858 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:15:44.858 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:15:44.858 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: modified 2021-11-03T17:15:18.348938+0000 2021-11-03T17:15:44.859 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: tableserver 0 2021-11-03T17:15:44.859 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: root 0 2021-11-03T17:15:44.859 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: session_timeout 60 2021-11-03T17:15:44.859 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: session_autoclose 300 2021-11-03T17:15:44.859 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: max_file_size 1099511627776 2021-11-03T17:15:44.860 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: required_client_features {} 2021-11-03T17:15:44.860 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: last_failure 0 2021-11-03T17:15:44.860 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: last_failure_osd_epoch 51 2021-11-03T17:15:44.860 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:15:44.860 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: max_mds 2 2021-11-03T17:15:44.861 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: in 0,1 2021-11-03T17:15:44.861 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: up {0=24289,1=14436} 2021-11-03T17:15:44.861 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: failed 2021-11-03T17:15:44.861 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: damaged 2021-11-03T17:15:44.862 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: stopped 2021-11-03T17:15:44.862 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: data_pools [3] 2021-11-03T17:15:44.862 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: metadata_pool 2 2021-11-03T17:15:44.862 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: inline_data disabled 2021-11-03T17:15:44.862 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: balancer 2021-11-03T17:15:44.863 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: standby_count_wanted 1 2021-11-03T17:15:44.863 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: [mds.cephfs.smithi115.semywj{0:24289} state up:active seq 51 export targets 1 join_fscid=1 addr [v2:172.21.15.115:6826/3367380088,v1:172.21.15.115:6827/3367380088] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:15:44.863 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: [mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 3 export targets 0 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:15:44.863 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: 2021-11-03T17:15:44.864 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: 2021-11-03T17:15:44.864 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: Standby daemons: 2021-11-03T17:15:44.864 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: 2021-11-03T17:15:44.864 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: [mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:15:44.864 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout: [mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:15:44.865 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: mon.smithi027@-1(???).osd e51 crush map has features 3314933000852226048, adjusting msgr requires 2021-11-03T17:15:44.865 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: mon.smithi027@-1(???).osd e51 crush map has features 288514051259236352, adjusting msgr requires 2021-11-03T17:15:44.865 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: mon.smithi027@-1(???).osd e51 crush map has features 288514051259236352, adjusting msgr requires 2021-11-03T17:15:44.865 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: mon.smithi027@-1(???).osd e51 crush map has features 288514051259236352, adjusting msgr requires 2021-11-03T17:15:44.865 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:15:44 smithi027 ceph-mon[97244]: mon.smithi027@-1(???).paxosservice(auth 1..21) refresh upgraded, format 0 -> 3 2021-11-03T17:15:46.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:45 smithi115 conmon[34670]: audit 2021-11-03T17:15:42.423820+0000 mon.smithi115 (mon.1) 64 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2021-11-03T17:15:46.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:45 smithi115 conmon[34670]: audit 2021-11-03T17:15:42.424425+0000 mon.smithi115 (mon.1) 65 : audit [DBG] from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2021-11-03T17:15:46.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:45 smithi115 conmon[34670]: cluster 2021-11-03T17:15:42.650383+0000 mgr.smithi027.ufhhvb (mgr.14622) 42 : cluster [DBG] pgmap v25: 65 pgs: 65 active+clean; 2.2 GiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 41 MiB/s rd, 86 MiB/s wr, 309 op/s 2021-11-03T17:15:46.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:45 smithi115 conmon[34670]: cephadm 2021-11-03T17:15:44.420241+0000 mgr.smithi027.ufhhvb (mgr.14622) 43 : cephadm [INF] poll took 4489.861 ms: 1 events 2021-11-03T17:15:46.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:45 smithi115 conmon[34670]: cluster 2021-11-03T17:15:44.650886+0000 mgr.smithi027.ufhhvb (mgr.14622) 44 : cluster [DBG] pgmap v26: 65 pgs: 65 active+clean; 2.2 GiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 27 MiB/s rd, 56 MiB/s wr, 222 op/s 2021-11-03T17:15:46.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:45 smithi115 conmon[34670]: cluster 2021-11-03T17:15:45.805916+0000 mon.smithi027 (mon.0) 1 : cluster [INF] mon.smithi027 calling monitor election 2021-11-03T17:15:46.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:45 smithi115 conmon[34670]: cluster 2021-11-03T17:15:45.821167+0000 mon.smithi027 (mon.0) 2 : cluster [INF] mon.smithi027 is new leader, mons smithi027,smithi115 in quorum (ranks 0,1) 2021-11-03T17:15:46.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:45 smithi115 conmon[34670]: cluster 2021-11-03T 2021-11-03T17:15:46.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:45 smithi115 conmon[34670]: 17:15:45.829657+0000 mon.smithi027 (mon.0) 3 : cluster [DBG] monmap e2: 2 mons at {smithi027=[v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0],smithi115=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0]} 2021-11-03T17:15:46.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:45 smithi115 conmon[34670]: cluster 2021-11-03T17:15:45.829761+0000 mon.smithi027 (mon.0) 4 : cluster [DBG] fsmap cephfs:2 {0=cephfs.smithi115.semywj=up:active,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:15:46.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:45 smithi115 conmon[34670]: cluster 2021-11-03T17:15:45.829801+0000 mon.smithi027 (mon.0) 5 : cluster [DBG] osdmap e51: 6 total, 6 up, 6 in 2021-11-03T17:15:46.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:45 smithi115 conmon[34670]: cluster 2021-11-03T17:15:45.831353+0000 mon.smithi027 (mon.0) 6 2021-11-03T17:15:46.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:45 smithi115 conmon[34670]: : cluster [DBG] mgrmap e29: smithi027.ufhhvb(active, since 46s), standbys: smithi115.tcyodz 2021-11-03T17:15:46.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:45 smithi115 conmon[34670]: cluster 2021-11-03T17:15:46.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:45 smithi115 conmon[34670]: 2021-11-03T17:15:45.845432+0000 mon.smithi027 (mon.0) 7 : cluster [DBG] 2021-11-03T17:15:46.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:45 smithi115 conmon[34670]: mgrmap e30: smithi027.ufhhvb(active, since 46s), standbys: smithi115.tcyodz 2021-11-03T17:15:46.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:45 smithi115 conmon[34670]: cluster 2021-11-03T17: 2021-11-03T17:15:46.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:45 smithi115 conmon[34670]: 15:45.845512+0000 mon.smithi027 (mon.0) 8 : 2021-11-03T17:15:46.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:45 smithi115 conmon[34670]: cluster [INF] overall HEALTH_OK 2021-11-03T17:15:46.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:45 smithi115 conmon[34670]: audit 2021-11-03T17: 2021-11-03T17:15:46.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:45 smithi115 conmon[34670]: 15:45.845743+0000 mon.smithi027 (mon.0) 9 : audit [INF] 2021-11-03T17:15:46.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:45 smithi115 conmon[34670]: from='mgr.14622 ' entity='' 2021-11-03T17:15:46.656 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:46.658+0000 7fb323fff700 1 -- 172.21.15.27:0/994781885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb324068780 msgr2=0x7fb324102af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:15:46.656 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:46.658+0000 7fb323fff700 1 --2- 172.21.15.27:0/994781885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb324068780 0x7fb324102af0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fb30c0099e0 tx=0x7fb30c004d10).stop 2021-11-03T17:15:46.656 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:46.659+0000 7fb323fff700 1 -- 172.21.15.27:0/994781885 shutdown_connections 2021-11-03T17:15:46.657 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:46.659+0000 7fb323fff700 1 --2- 172.21.15.27:0/994781885 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb324068780 0x7fb324102af0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:46.657 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:46.659+0000 7fb323fff700 1 --2- 172.21.15.27:0/994781885 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb324067e70 0x7fb324068290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:46.657 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:46.659+0000 7fb323fff700 1 -- 172.21.15.27:0/994781885 >> 172.21.15.27:0/994781885 conn(0x7fb324063980 msgr2=0x7fb324065da0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:15:46.657 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:46.659+0000 7fb323fff700 1 -- 172.21.15.27:0/994781885 shutdown_connections 2021-11-03T17:15:46.657 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:46.659+0000 7fb323fff700 1 -- 172.21.15.27:0/994781885 wait complete. 2021-11-03T17:15:46.658 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:46.660+0000 7fb323fff700 1 Processor -- start 2021-11-03T17:15:46.659 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:46.661+0000 7fb323fff700 1 -- start start 2021-11-03T17:15:46.659 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:46.661+0000 7fb323fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb324067e70 0x7fb324073950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:15:46.659 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:46.661+0000 7fb323fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb324068780 0x7fb32407a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:15:46.659 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:46.661+0000 7fb323fff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb324073e40 con 0x7fb324067e70 2021-11-03T17:15:46.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:46.661+0000 7fb323fff700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb324073f80 con 0x7fb324068780 2021-11-03T17:15:46.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:46.662+0000 7fb31a7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb324068780 0x7fb32407a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:15:46.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:46.662+0000 7fb31a7fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb324068780 0x7fb32407a740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.27:46930/0 (socket says 172.21.15.27:46930) 2021-11-03T17:15:46.661 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:46.662+0000 7fb31a7fc700 1 -- 172.21.15.27:0/2519329624 learned_addr learned my addr 172.21.15.27:0/2519329624 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:15:46.661 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:46.662+0000 7fb31a7fc700 1 -- 172.21.15.27:0/2519329624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb324067e70 msgr2=0x7fb324073950 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2021-11-03T17:15:46.661 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:46.662+0000 7fb31a7fc700 1 --2- 172.21.15.27:0/2519329624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb324067e70 0x7fb324073950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:15:46.661 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:46.662+0000 7fb31a7fc700 1 -- 172.21.15.27:0/2519329624 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb30c009640 con 0x7fb324068780 2021-11-03T17:15:46.662 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:46.663+0000 7fb31a7fc700 1 --2- 172.21.15.27:0/2519329624 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb324068780 0x7fb32407a740 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fb30c00b3b0 tx=0x7fb30c004d10).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:15:46.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:46.665+0000 7fb31bfff700 1 -- 172.21.15.27:0/2519329624 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb30c00e070 con 0x7fb324068780 2021-11-03T17:15:46.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:46.665+0000 7fb31bfff700 1 -- 172.21.15.27:0/2519329624 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fb30c0049b0 con 0x7fb324068780 2021-11-03T17:15:46.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:46.665+0000 7fb31bfff700 1 -- 172.21.15.27:0/2519329624 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb30c018070 con 0x7fb324068780 2021-11-03T17:15:46.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:46.665+0000 7fb323fff700 1 -- 172.21.15.27:0/2519329624 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb324074200 con 0x7fb324068780 2021-11-03T17:15:46.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:46.665+0000 7fb323fff700 1 -- 172.21.15.27:0/2519329624 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3241263d0 con 0x7fb324068780 2021-11-03T17:15:46.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:46.666+0000 7fb323fff700 1 -- 172.21.15.27:0/2519329624 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb324059100 con 0x7fb324068780 2021-11-03T17:15:46.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:46.674+0000 7fb31bfff700 1 -- 172.21.15.27:0/2519329624 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 30) v1 ==== 81997+0+0 (secure 0 0 0) 0x7fb30c01d420 con 0x7fb324068780 2021-11-03T17:15:46.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:46.675+0000 7fb31bfff700 1 --2- 172.21.15.27:0/2519329624 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7fb304064250 0x7fb304066710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:15:46.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:46.675+0000 7fb31bfff700 1 -- 172.21.15.27:0/2519329624 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6031+0+0 (secure 0 0 0) 0x7fb30c082580 con 0x7fb324068780 2021-11-03T17:15:46.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:46.675+0000 7fb31bfff700 1 -- 172.21.15.27:0/2519329624 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fb30c082ac0 con 0x7fb324068780 2021-11-03T17:15:46.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:46.675+0000 7fb322ffd700 1 -- 172.21.15.27:0/2519329624 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7fb304064250 msgr2=0x7fb304066710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6828/3927816089 2021-11-03T17:15:46.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:46.675+0000 7fb322ffd700 1 --2- 172.21.15.27:0/2519329624 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7fb304064250 0x7fb304066710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2021-11-03T17:15:46.871 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:46.876+0000 7fb322ffd700 1 -- 172.21.15.27:0/2519329624 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7fb304064250 msgr2=0x7fb304066710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6828/3927816089 2021-11-03T17:15:46.872 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:46.876+0000 7fb322ffd700 1 --2- 172.21.15.27:0/2519329624 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7fb304064250 0x7fb304066710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2021-11-03T17:15:46.913 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:46.918+0000 7fb323fff700 1 -- 172.21.15.27:0/2519329624 --> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb32406b3e0 con 0x7fb304064250 2021-11-03T17:15:47.272 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:47.277+0000 7fb322ffd700 1 -- 172.21.15.27:0/2519329624 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7fb304064250 msgr2=0x7fb304066710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6828/3927816089 2021-11-03T17:15:47.273 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:47.277+0000 7fb322ffd700 1 --2- 172.21.15.27:0/2519329624 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7fb304064250 0x7fb304066710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2021-11-03T17:15:48.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:48.078+0000 7fb322ffd700 1 -- 172.21.15.27:0/2519329624 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7fb304064250 msgr2=0x7fb304066710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6828/3927816089 2021-11-03T17:15:48.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:48.078+0000 7fb322ffd700 1 --2- 172.21.15.27:0/2519329624 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7fb304064250 0x7fb304066710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2021-11-03T17:15:49.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:49.679+0000 7fb322ffd700 1 -- 172.21.15.27:0/2519329624 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7fb304064250 msgr2=0x7fb304066710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6828/3927816089 2021-11-03T17:15:49.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:49.679+0000 7fb322ffd700 1 --2- 172.21.15.27:0/2519329624 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7fb304064250 0x7fb304066710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2021-11-03T17:15:52.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:52.882+0000 7fb322ffd700 1 -- 172.21.15.27:0/2519329624 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7fb304064250 msgr2=0x7fb304066710 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.27:6828/3927816089 2021-11-03T17:15:52.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:52.882+0000 7fb322ffd700 1 --2- 172.21.15.27:0/2519329624 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7fb304064250 0x7fb304066710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 6.400000 2021-11-03T17:15:53.274 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:53.279+0000 7fb31bfff700 1 -- 172.21.15.27:0/2519329624 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 31) v1 ==== 81997+0+0 (secure 0 0 0) 0x7fb30c080810 con 0x7fb324068780 2021-11-03T17:15:53.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:53 smithi115 conmon[34670]: cluster 2021-11-03T17:15:53.220923+0000 mon.smithi027 (mon.0) 10 : cluster 2021-11-03T17:15:53.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:53 smithi115 conmon[34670]: [DBG] Standby manager daemon smithi115.tcyodz restarted 2021-11-03T17:15:53.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:53 smithi115 conmon[34670]: cluster 2021-11-03T17:15:53.221014+0000 mon.smithi027 (mon.0) 11 : cluster [DBG] Standby manager daemon smithi115.tcyodz started 2021-11-03T17:15:53.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:53 smithi115 conmon[34670]: audit 2021-11-03T17:15:53.223104+0000 mon.smithi115 (mon.1) 66 2021-11-03T17:15:53.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:53 smithi115 conmon[34670]: : audit [DBG] from='mgr.? 172.21.15.115:0/2569420103' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi115.tcyodz/crt"}]: dispatch 2021-11-03T17:15:53.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:53 smithi115 conmon[34670]: audit 2021-11-03T17:15:53 2021-11-03T17:15:53.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:53 smithi115 conmon[34670]: .223785+0000 mon.smithi115 (mon.1) 67 : audit [DBG] from='mgr.? 172.21.15.115:0/2569420103' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-11-03T17:15:53.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:53 smithi115 conmon[34670]: audit 2021-11-03 2021-11-03T17:15:53.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:53 smithi115 conmon[34670]: T17:15:53.225864+0000 mon.smithi115 (mon.1 2021-11-03T17:15:53.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:53 smithi115 conmon[34670]: ) 68 : audit [DBG] from='mgr.? 172.21.15.115:0/2569420103' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi115.tcyodz/key"}]: dispatch 2021-11-03T17:15:53.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:53 smithi115 conmon[34670]: audit 2021-11 2021-11-03T17:15:53.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:53 smithi115 conmon[34670]: -03T17:15:53.226933+0000 mon.smithi115 ( 2021-11-03T17:15:53.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:53 smithi115 conmon[34670]: mon.1) 69 : audit [DBG] 2021-11-03T17:15:53.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:53 smithi115 conmon[34670]: from='mgr.? 172.21.15.115:0/2569420103' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-11-03T17:15:54.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:54 smithi115 conmon[34670]: cluster 2021-11-03T17:15:53.277329+0000 mon.smithi027 (mon.0 2021-11-03T17:15:54.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:54 smithi115 conmon[34670]: ) 12 : cluster [DBG] mgrmap e31: smithi027.ufhhvb(active, since 53s), standbys: smithi115.tcyodz 2021-11-03T17:15:55.405 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:55.410+0000 7fb31bfff700 1 -- 172.21.15.27:0/2519329624 <== mon.1 v2:172.21.15.115:3300/0 8 ==== mgrmap(e 32) v1 ==== 81560+0+0 (secure 0 0 0) 0x7fb30c051540 con 0x7fb324068780 2021-11-03T17:15:55.406 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:55.410+0000 7fb31bfff700 1 -- 172.21.15.27:0/2519329624 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7fb304064250 msgr2=0x7fb304066710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2021-11-03T17:15:55.406 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:55.410+0000 7fb31bfff700 1 --2- 172.21.15.27:0/2519329624 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7fb304064250 0x7fb304066710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:15:55.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:55 smithi115 conmon[34670]: cluster 2021-11-03T17:15:54.438255+0000 mon.smithi027 (mon.0) 2021-11-03T17:15:55.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:55 smithi115 conmon[34670]: 13 : cluster [INF] Active manager daemon smithi027.ufhhvb restarted 2021-11-03T17:15:55.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:55 smithi115 conmon[34670]: cluster 2021-11-03T17:15:54.438798+0000 mon.smithi027 (mon.0) 14 : cluster [INF] Activating manager daemon smithi027.ufhhvb 2021-11-03T17:15:55.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:55 smithi115 conmon[34670]: cluster 2021-11-03T17:15:55.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:55 smithi115 conmon[34670]: 2021-11-03T17:15:54.442018+0000 mon.smithi027 (mon.0) 15 : cluster [DBG] osdmap e52: 6 total, 6 up, 6 in 2021-11-03T17:15:56.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.418+0000 7fb31bfff700 1 -- 172.21.15.27:0/2519329624 <== mon.1 v2:172.21.15.115:3300/0 9 ==== mgrmap(e 33) v1 ==== 81670+0+0 (secure 0 0 0) 0x7fb30c051e10 con 0x7fb324068780 2021-11-03T17:15:56.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.418+0000 7fb31bfff700 1 --2- 172.21.15.27:0/2519329624 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb3040702a0 0x7fb304072690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:15:56.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.418+0000 7fb31bfff700 1 -- 172.21.15.27:0/2519329624 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb32406b3e0 con 0x7fb3040702a0 2021-11-03T17:15:56.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.418+0000 7fb322ffd700 1 --2- 172.21.15.27:0/2519329624 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb3040702a0 0x7fb304072690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:15:56.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.419+0000 7fb322ffd700 1 --2- 172.21.15.27:0/2519329624 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb3040702a0 0x7fb304072690 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fb32406c230 tx=0x7fb314007a40).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:15:56.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.423+0000 7fb31bfff700 1 -- 172.21.15.27:0/2519329624 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+257 (secure 0 0 0) 0x7fb32406b3e0 con 0x7fb3040702a0 2021-11-03T17:15:56.426 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.430+0000 7fb323fff700 1 -- 172.21.15.27:0/2519329624 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb3040702a0 msgr2=0x7fb304072690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:15:56.426 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.430+0000 7fb323fff700 1 --2- 172.21.15.27:0/2519329624 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb3040702a0 0x7fb304072690 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fb32406c230 tx=0x7fb314007a40).stop 2021-11-03T17:15:56.427 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.430+0000 7fb323fff700 1 -- 172.21.15.27:0/2519329624 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb324068780 msgr2=0x7fb32407a740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:15:56.427 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.430+0000 7fb323fff700 1 --2- 172.21.15.27:0/2519329624 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb324068780 0x7fb32407a740 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fb30c00b3b0 tx=0x7fb30c004d10).stop 2021-11-03T17:15:56.429 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.433+0000 7fb323fff700 1 -- 172.21.15.27:0/2519329624 shutdown_connections 2021-11-03T17:15:56.430 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.433+0000 7fb323fff700 1 --2- 172.21.15.27:0/2519329624 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb3040702a0 0x7fb304072690 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:56.430 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.433+0000 7fb323fff700 1 --2- 172.21.15.27:0/2519329624 >> [v2:172.21.15.27:6828/3927816089,v1:172.21.15.27:6829/3927816089] conn(0x7fb304064250 0x7fb304066710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:15:56.430 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.433+0000 7fb323fff700 1 --2- 172.21.15.27:0/2519329624 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb324067e70 0x7fb324073950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:15:56.431 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.433+0000 7fb323fff700 1 --2- 172.21.15.27:0/2519329624 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb324068780 0x7fb32407a740 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:56.431 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.433+0000 7fb323fff700 1 -- 172.21.15.27:0/2519329624 >> 172.21.15.27:0/2519329624 conn(0x7fb324063980 msgr2=0x7fb324101300 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:15:56.431 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.433+0000 7fb323fff700 1 -- 172.21.15.27:0/2519329624 shutdown_connections 2021-11-03T17:15:56.431 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.433+0000 7fb323fff700 1 -- 172.21.15.27:0/2519329624 wait complete. 2021-11-03T17:15:56.445 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:15:56.605 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.609+0000 7feb04b6f700 1 -- 172.21.15.27:0/1370830227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb000f7620 msgr2=0x7feb000f9ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:15:56.605 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.609+0000 7feb04b6f700 1 --2- 172.21.15.27:0/1370830227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb000f7620 0x7feb000f9ab0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7feaf40045a0 tx=0x7feaf4009b30).stop 2021-11-03T17:15:56.605 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.609+0000 7feb04b6f700 1 -- 172.21.15.27:0/1370830227 shutdown_connections 2021-11-03T17:15:56.606 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.609+0000 7feb04b6f700 1 --2- 172.21.15.27:0/1370830227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb000f7620 0x7feb000f9ab0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:56.606 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.609+0000 7feb04b6f700 1 --2- 172.21.15.27:0/1370830227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb00060410 0x7feb000f70a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:56.606 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.609+0000 7feb04b6f700 1 -- 172.21.15.27:0/1370830227 >> 172.21.15.27:0/1370830227 conn(0x7feb000f0d30 msgr2=0x7feb000f3150 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:15:56.606 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.609+0000 7feb04b6f700 1 -- 172.21.15.27:0/1370830227 shutdown_connections 2021-11-03T17:15:56.608 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.609+0000 7feb04b6f700 1 -- 172.21.15.27:0/1370830227 wait complete. 2021-11-03T17:15:56.608 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.610+0000 7feb04b6f700 1 Processor -- start 2021-11-03T17:15:56.608 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.610+0000 7feb04b6f700 1 -- start start 2021-11-03T17:15:56.608 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.610+0000 7feb04b6f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb00060410 0x7feb00068c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:15:56.609 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.610+0000 7feb04b6f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb000f7620 0x7feb00069170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:15:56.609 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.610+0000 7feb04b6f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb00069720 con 0x7feb000f7620 2021-11-03T17:15:56.609 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.610+0000 7feb04b6f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb00116fe0 con 0x7feb00060410 2021-11-03T17:15:56.609 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.611+0000 7feafe59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb00060410 0x7feb00068c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:15:56.610 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.611+0000 7feafe59c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb00060410 0x7feb00068c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.27:47096/0 (socket says 172.21.15.27:47096) 2021-11-03T17:15:56.610 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.611+0000 7feafe59c700 1 -- 172.21.15.27:0/721663252 learned_addr learned my addr 172.21.15.27:0/721663252 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:15:56.610 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.611+0000 7feafdd9b700 1 --2- 172.21.15.27:0/721663252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb000f7620 0x7feb00069170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:15:56.610 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.612+0000 7feafe59c700 1 -- 172.21.15.27:0/721663252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb000f7620 msgr2=0x7feb00069170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:15:56.611 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.612+0000 7feafe59c700 1 --2- 172.21.15.27:0/721663252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb000f7620 0x7feb00069170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:56.611 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.612+0000 7feafe59c700 1 -- 172.21.15.27:0/721663252 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaf4005040 con 0x7feb00060410 2021-11-03T17:15:56.611 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.612+0000 7feafe59c700 1 --2- 172.21.15.27:0/721663252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb00060410 0x7feb00068c80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7feaf8007fd0 tx=0x7feaf8004ee0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:15:56.611 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.612+0000 7feaeeffd700 1 -- 172.21.15.27:0/721663252 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feaf800b4f0 con 0x7feb00060410 2021-11-03T17:15:56.612 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.612+0000 7feaeeffd700 1 -- 172.21.15.27:0/721663252 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7feaf800f040 con 0x7feb00060410 2021-11-03T17:15:56.612 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.612+0000 7feb04b6f700 1 -- 172.21.15.27:0/721663252 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb001172c0 con 0x7feb00060410 2021-11-03T17:15:56.613 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.612+0000 7feaeeffd700 1 -- 172.21.15.27:0/721663252 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feaf8013a90 con 0x7feb00060410 2021-11-03T17:15:56.613 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.612+0000 7feb04b6f700 1 -- 172.21.15.27:0/721663252 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb00117880 con 0x7feb00060410 2021-11-03T17:15:56.613 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.614+0000 7feb04b6f700 1 -- 172.21.15.27:0/721663252 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb000f49c0 con 0x7feb00060410 2021-11-03T17:15:56.613 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.614+0000 7feaeeffd700 1 -- 172.21.15.27:0/721663252 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 33) v1 ==== 81670+0+0 (secure 0 0 0) 0x7feaf801a030 con 0x7feb00060410 2021-11-03T17:15:56.613 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.615+0000 7feaeeffd700 1 --2- 172.21.15.27:0/721663252 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7feae8063dd0 0x7feae8066290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:15:56.614 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.615+0000 7feaeeffd700 1 -- 172.21.15.27:0/721663252 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6375+0+0 (secure 0 0 0) 0x7feaf80801f0 con 0x7feb00060410 2021-11-03T17:15:56.614 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.615+0000 7feafdd9b700 1 --2- 172.21.15.27:0/721663252 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7feae8063dd0 0x7feae8066290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:15:56.614 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.615+0000 7feafdd9b700 1 --2- 172.21.15.27:0/721663252 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7feae8063dd0 0x7feae8066290 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7feaf4004d70 tx=0x7feaf4016040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:15:56.618 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.623+0000 7feaeeffd700 1 -- 172.21.15.27:0/721663252 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7feaf804ec10 con 0x7feb00060410 2021-11-03T17:15:56.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:56 smithi115 conmon[34670]: cluster 2021-11-03T17:15:55.408418+0000 mon.smithi027 (mon.0) 16 : cluster [DBG] mgrmap e32: smithi027.ufhhvb(active, starting, since 0.9697s), standbys: smithi115.tcyodz 2021-11-03T17:15:56.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:56 smithi115 conmon[34670]: audit 2021-11-03T17:15:55.411224+0000 mon.smithi027 (mon.0) 17 : audit [DBG] from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2021-11-03T17:15:56.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:56 smithi115 conmon[34670]: audit 2021-11-03T17:15:55.411383+0000 mon.smithi027 (mon.0) 18 : audit [DBG] from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2021-11-03T17:15:56.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:56 smithi115 conmon[34670]: audit 2021-11-03T17:15:55.412080+0000 mon.smithi027 (mon.0) 19 : audit [DBG] from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.jdwohn"}]: dispatch 2021-11-03T17:15:56.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:56 smithi115 conmon[34670]: audit 2021-11-03T17:15:55.412434+0000 mon.smithi027 (mon.0) 20 : audit [DBG] from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.fksixo"}]: dispatch 2021-11-03T17:15:56.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:56 smithi115 conmon[34670]: audit 2021-11-03T17:15:55.412696+0000 mon.smithi027 (mon.0) 21 : audit [DBG] from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.semywj"}]: dispatch 2021-11-03T17:15:56.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:56 smithi115 conmon[34670]: audit 2021-11-03T17:15:55.412939+0000 mon.smithi027 (mon.0) 22 : audit [DBG] from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.zsfgnv"}]: dispatch 2021-11-03T17:15:56.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:56 smithi115 conmon[34670]: audit 2021-11-03T17:15:55.413366+0000 mon.smithi027 (mon.0) 23 : audit [DBG] from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mgr metadata", "who": "smithi027.ufhhvb", "id": "smithi027.ufhhvb"}]: dispatch 2021-11-03T17:15:56.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:56 smithi115 conmon[34670]: audit 2021-11-03T17:15:55.413642+0000 mon.smithi027 (mon.0) 24 : audit [DBG] from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mgr metadata", "who": "smithi115.tcyodz", "id": "smithi115.tcyodz"}]: dispatch 2021-11-03T17:15:56.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:56 smithi115 conmon[34670]: audit 2021-11-03T17:15:55.413782+0000 mon.smithi027 (mon.0) 25 : audit [DBG] from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-11-03T17:15:56.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:56 smithi115 conmon[34670]: audit 2021-11-03T17:15:55.414031+0000 mon.smithi027 (mon.0) 26 : audit [DBG] from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-11-03T17:15:56.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:56 smithi115 conmon[34670]: audit 2021-11-03T17:15:55.414262+0000 mon.smithi027 (mon.0) 27 : audit [DBG] from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-11-03T17:15:56.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:56 smithi115 conmon[34670]: audit 2021-11-03T17:15:55.414577+0000 mon.smithi027 (mon.0) 28 : audit [DBG] from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-11-03T17:15:56.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:56 smithi115 conmon[34670]: audit 2021-11-03T17:15:55.414906+0000 mon.smithi027 (mon.0) 29 : audit [DBG] from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-11-03T17:15:56.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:56 smithi115 conmon[34670]: audit 2021-11-03T17:15:55.415273+0000 mon.smithi027 (mon.0) 30 : audit [DBG] from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-11-03T17:15:56.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:56 smithi115 conmon[34670]: audit 2021-11-03T17:15:55.416186+0000 mon.smithi027 (mon.0) 31 : audit [DBG] from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata"}]: dispatch 2021-11-03T17:15:56.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:56 smithi115 conmon[34670]: audit 2021-11-03T17:15:55.416611+0000 mon.smithi027 (mon.0) 32 : audit [DBG] from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata"}]: dispatch 2021-11-03T17:15:56.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:56 smithi115 conmon[34670]: audit 2021-11-03T17:15:55.417049+0000 mon.smithi027 (mon.0) 33 : audit [DBG] from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata"}]: dispatch 2021-11-03T17:15:56.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:56 smithi115 conmon[34670]: cluster 2021-11-03T17:15:55.424888+0000 mon.smithi027 (mon.0) 34 : cluster [INF] Manager daemon smithi027.ufhhvb is now available 2021-11-03T17:15:56.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:56 smithi115 conmon[34670]: audit 2021-11-03T17:15:55.464443+0000 mon.smithi027 (mon.0) 35 : audit [DBG] from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:15:56.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:56 smithi115 conmon[34670]: audit 2021-11-03T17:15:55.493801+0000 mon.smithi027 (mon.0) 36 : audit [DBG] from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:15:56.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:56 smithi115 conmon[34670]: audit 2021-11-03T17:15:55.553340+0000 mon.smithi027 (mon.0) 37 : audit [INF] from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:15:56.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:56 smithi115 conmon[34670]: audit 2021-11-03T17:15:55.579762+0000 mon.smithi027 (mon.0) 38 : audit [INF] from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:15:56.909 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.913+0000 7feb04b6f700 1 -- 172.21.15.27:0/721663252 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7feb00105630 con 0x7feae8063dd0 2021-11-03T17:15:56.930 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.934+0000 7feaeeffd700 1 -- 172.21.15.27:0/721663252 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7feb00105630 con 0x7feae8063dd0 2021-11-03T17:15:56.931 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:15:56.931 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (7m) 45s ago 8m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:15:56.931 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (8m) 45s ago 8m 7013k - 16.2.4 8d91d370c2b8 d494472a47c0 2021-11-03T17:15:56.932 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (8m) 24s ago 7m 7130k - 16.2.4 8d91d370c2b8 f84fd4042530 2021-11-03T17:15:56.932 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (7m) 45s ago 8m 38.3M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:15:56.932 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (60s) 45s ago 3m 12.0M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:15:56.932 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (3m) 45s ago 3m 2177M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:15:56.933 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (3m) 24s ago 3m 812M - 16.2.4 8d91d370c2b8 bfd990d6415e 2021-11-03T17:15:56.933 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (81s) 24s ago 4m 14.1M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:15:56.933 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (73s) 45s ago 10m 436M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:15:56.933 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (33s) 24s ago 7m 53.4M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:15:56.934 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 starting - - - 2048M 2021-11-03T17:15:56.934 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (7m) 24s ago 7m 73.2M 2048M 16.2.4 8d91d370c2b8 319323f09834 2021-11-03T17:15:56.934 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (8m) 45s ago 8m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:15:56.934 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (7m) 24s ago 7m 18.2M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:15:56.935 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (7m) 45s ago 7m 532M 4096M 16.2.4 8d91d370c2b8 17f79a815a0f 2021-11-03T17:15:56.935 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (6m) 45s ago 6m 661M 4096M 16.2.4 8d91d370c2b8 422be0cac0db 2021-11-03T17:15:56.935 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (6m) 45s ago 6m 449M 4096M 16.2.4 8d91d370c2b8 c211ef5551c9 2021-11-03T17:15:56.935 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (6m) 24s ago 6m 847M 4096M 16.2.4 8d91d370c2b8 b1486ec1b619 2021-11-03T17:15:56.935 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (5m) 24s ago 5m 652M 4096M 16.2.4 8d91d370c2b8 8b91230c69e5 2021-11-03T17:15:56.936 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (5m) 24s ago 5m 817M 4096M 16.2.4 8d91d370c2b8 4a54c4d6b6f5 2021-11-03T17:15:56.936 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (7m) 45s ago 8m 42.8M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:15:56.937 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.936+0000 7feb04b6f700 1 -- 172.21.15.27:0/721663252 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7feae8063dd0 msgr2=0x7feae8066290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:15:56.937 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.936+0000 7feb04b6f700 1 --2- 172.21.15.27:0/721663252 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7feae8063dd0 0x7feae8066290 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7feaf4004d70 tx=0x7feaf4016040).stop 2021-11-03T17:15:56.937 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.936+0000 7feb04b6f700 1 -- 172.21.15.27:0/721663252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb00060410 msgr2=0x7feb00068c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:15:56.938 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.936+0000 7feb04b6f700 1 --2- 172.21.15.27:0/721663252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb00060410 0x7feb00068c80 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7feaf8007fd0 tx=0x7feaf8004ee0).stop 2021-11-03T17:15:56.938 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.936+0000 7feb04b6f700 1 -- 172.21.15.27:0/721663252 shutdown_connections 2021-11-03T17:15:56.938 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.936+0000 7feb04b6f700 1 --2- 172.21.15.27:0/721663252 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7feae8063dd0 0x7feae8066290 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:56.939 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.936+0000 7feb04b6f700 1 --2- 172.21.15.27:0/721663252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb00060410 0x7feb00068c80 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:56.939 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.936+0000 7feb04b6f700 1 --2- 172.21.15.27:0/721663252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb000f7620 0x7feb00069170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:56.939 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.936+0000 7feb04b6f700 1 -- 172.21.15.27:0/721663252 >> 172.21.15.27:0/721663252 conn(0x7feb000f0d30 msgr2=0x7feb000f3120 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:15:56.939 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.936+0000 7feb04b6f700 1 -- 172.21.15.27:0/721663252 shutdown_connections 2021-11-03T17:15:56.940 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:56.936+0000 7feb04b6f700 1 -- 172.21.15.27:0/721663252 wait complete. 2021-11-03T17:15:57.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.043+0000 7fd92c149700 1 -- 172.21.15.27:0/2960415240 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9240f6860 msgr2=0x7fd9240f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:15:57.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.043+0000 7fd92c149700 1 --2- 172.21.15.27:0/2960415240 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9240f6860 0x7fd9240f6cc0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fd914004660 tx=0x7fd914009b30).stop 2021-11-03T17:15:57.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.043+0000 7fd92c149700 1 -- 172.21.15.27:0/2960415240 shutdown_connections 2021-11-03T17:15:57.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.043+0000 7fd92c149700 1 --2- 172.21.15.27:0/2960415240 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9240f6860 0x7fd9240f6cc0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:57.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.043+0000 7fd92c149700 1 --2- 172.21.15.27:0/2960415240 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9240f55c0 0x7fd9240f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:57.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.043+0000 7fd92c149700 1 -- 172.21.15.27:0/2960415240 >> 172.21.15.27:0/2960415240 conn(0x7fd9240f0d40 msgr2=0x7fd9240f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:15:57.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.044+0000 7fd92c149700 1 -- 172.21.15.27:0/2960415240 shutdown_connections 2021-11-03T17:15:57.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.044+0000 7fd92c149700 1 -- 172.21.15.27:0/2960415240 wait complete. 2021-11-03T17:15:57.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.044+0000 7fd92c149700 1 Processor -- start 2021-11-03T17:15:57.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.044+0000 7fd92c149700 1 -- start start 2021-11-03T17:15:57.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.044+0000 7fd92c149700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9240f55c0 0x7fd924106ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:15:57.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.044+0000 7fd92c149700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9240f6860 0x7fd9241073d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:15:57.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.044+0000 7fd92c149700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd924107980 con 0x7fd9240f6860 2021-11-03T17:15:57.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.044+0000 7fd92c149700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd924107ac0 con 0x7fd9240f55c0 2021-11-03T17:15:57.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.044+0000 7fd9296e4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9240f6860 0x7fd9241073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:15:57.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.045+0000 7fd929ee5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9240f55c0 0x7fd924106ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:15:57.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.045+0000 7fd9296e4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9240f6860 0x7fd9241073d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:55950/0 (socket says 172.21.15.27:55950) 2021-11-03T17:15:57.046 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.045+0000 7fd9296e4700 1 -- 172.21.15.27:0/4261618275 learned_addr learned my addr 172.21.15.27:0/4261618275 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:15:57.046 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.045+0000 7fd929ee5700 1 -- 172.21.15.27:0/4261618275 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9240f6860 msgr2=0x7fd9241073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:15:57.046 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.045+0000 7fd929ee5700 1 --2- 172.21.15.27:0/4261618275 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9240f6860 0x7fd9241073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:57.046 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.045+0000 7fd929ee5700 1 -- 172.21.15.27:0/4261618275 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd914005040 con 0x7fd9240f55c0 2021-11-03T17:15:57.047 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.045+0000 7fd9296e4700 1 --2- 172.21.15.27:0/4261618275 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9240f6860 0x7fd9241073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-03T17:15:57.047 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.045+0000 7fd929ee5700 1 --2- 172.21.15.27:0/4261618275 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9240f55c0 0x7fd924106ee0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fd9240f66c0 tx=0x7fd92000b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:15:57.048 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.045+0000 7fd91a7fc700 1 -- 172.21.15.27:0/4261618275 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9200092e0 con 0x7fd9240f55c0 2021-11-03T17:15:57.048 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.045+0000 7fd92c149700 1 -- 172.21.15.27:0/4261618275 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd924108460 con 0x7fd9240f55c0 2021-11-03T17:15:57.048 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.045+0000 7fd91a7fc700 1 -- 172.21.15.27:0/4261618275 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fd92000f040 con 0x7fd9240f55c0 2021-11-03T17:15:57.048 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.045+0000 7fd92c149700 1 -- 172.21.15.27:0/4261618275 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd924108a20 con 0x7fd9240f55c0 2021-11-03T17:15:57.049 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.045+0000 7fd91a7fc700 1 -- 172.21.15.27:0/4261618275 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9200147a0 con 0x7fd9240f55c0 2021-11-03T17:15:57.049 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.046+0000 7fd92c149700 1 -- 172.21.15.27:0/4261618275 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd90c004fa0 con 0x7fd9240f55c0 2021-11-03T17:15:57.049 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.047+0000 7fd91a7fc700 1 -- 172.21.15.27:0/4261618275 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 33) v1 ==== 81670+0+0 (secure 0 0 0) 0x7fd92001b030 con 0x7fd9240f55c0 2021-11-03T17:15:57.049 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.047+0000 7fd91a7fc700 1 --2- 172.21.15.27:0/4261618275 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd910063dd0 0x7fd910066290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:15:57.050 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.047+0000 7fd91a7fc700 1 -- 172.21.15.27:0/4261618275 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6375+0+0 (secure 0 0 0) 0x7fd920080030 con 0x7fd9240f55c0 2021-11-03T17:15:57.050 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.047+0000 7fd9296e4700 1 --2- 172.21.15.27:0/4261618275 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd910063dd0 0x7fd910066290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:15:57.050 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.048+0000 7fd9296e4700 1 --2- 172.21.15.27:0/4261618275 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd910063dd0 0x7fd910066290 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fd914009d60 tx=0x7fd914016040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:15:57.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.051+0000 7fd91a7fc700 1 -- 172.21.15.27:0/4261618275 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fd92004ea50 con 0x7fd9240f55c0 2021-11-03T17:15:57.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.299+0000 7fd92c149700 1 -- 172.21.15.27:0/4261618275 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd90c005e80 con 0x7fd9240f55c0 2021-11-03T17:15:57.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.300+0000 7fd91a7fc700 1 -- 172.21.15.27:0/4261618275 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+794 (secure 0 0 0) 0x7fd920019070 con 0x7fd9240f55c0 2021-11-03T17:15:57.296 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:15:57.297 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:15:57.297 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 1, 2021-11-03T17:15:57.297 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 1 2021-11-03T17:15:57.297 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:15:57.298 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:15:57.298 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:15:57.298 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:15:57.298 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:15:57.298 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 6 2021-11-03T17:15:57.299 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:15:57.299 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:15:57.300 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:15:57.300 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:15:57.300 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:15:57.300 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 11, 2021-11-03T17:15:57.300 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 3 2021-11-03T17:15:57.301 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:15:57.301 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:15:57.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.302+0000 7fd92c149700 1 -- 172.21.15.27:0/4261618275 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd910063dd0 msgr2=0x7fd910066290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:15:57.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.302+0000 7fd92c149700 1 --2- 172.21.15.27:0/4261618275 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd910063dd0 0x7fd910066290 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fd914009d60 tx=0x7fd914016040).stop 2021-11-03T17:15:57.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.302+0000 7fd92c149700 1 -- 172.21.15.27:0/4261618275 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9240f55c0 msgr2=0x7fd924106ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:15:57.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.302+0000 7fd92c149700 1 --2- 172.21.15.27:0/4261618275 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9240f55c0 0x7fd924106ee0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fd9240f66c0 tx=0x7fd92000b040).stop 2021-11-03T17:15:57.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.302+0000 7fd92c149700 1 -- 172.21.15.27:0/4261618275 shutdown_connections 2021-11-03T17:15:57.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.302+0000 7fd92c149700 1 --2- 172.21.15.27:0/4261618275 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd910063dd0 0x7fd910066290 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:57.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.302+0000 7fd92c149700 1 --2- 172.21.15.27:0/4261618275 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9240f55c0 0x7fd924106ee0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:57.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.302+0000 7fd92c149700 1 --2- 172.21.15.27:0/4261618275 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9240f6860 0x7fd9241073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:57.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.302+0000 7fd92c149700 1 -- 172.21.15.27:0/4261618275 >> 172.21.15.27:0/4261618275 conn(0x7fd9240f0d40 msgr2=0x7fd92405fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:15:57.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.302+0000 7fd92c149700 1 -- 172.21.15.27:0/4261618275 shutdown_connections 2021-11-03T17:15:57.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.302+0000 7fd92c149700 1 -- 172.21.15.27:0/4261618275 wait complete. 2021-11-03T17:15:57.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.420+0000 7f63f79e6700 1 -- 172.21.15.27:0/710008647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63f00f7590 msgr2=0x7f63f00f59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:15:57.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.420+0000 7f63f79e6700 1 --2- 172.21.15.27:0/710008647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63f00f7590 0x7f63f00f59b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f63e0004660 tx=0x7f63e0009b30).stop 2021-11-03T17:15:57.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.421+0000 7f63f79e6700 1 -- 172.21.15.27:0/710008647 shutdown_connections 2021-11-03T17:15:57.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.421+0000 7f63f79e6700 1 --2- 172.21.15.27:0/710008647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63f00f7590 0x7f63f00f59b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:57.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.421+0000 7f63f79e6700 1 --2- 172.21.15.27:0/710008647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63f00f6bf0 0x7f63f00f7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:57.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.421+0000 7f63f79e6700 1 -- 172.21.15.27:0/710008647 >> 172.21.15.27:0/710008647 conn(0x7f63f00f04c0 msgr2=0x7f63f00f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:15:57.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.421+0000 7f63f79e6700 1 -- 172.21.15.27:0/710008647 shutdown_connections 2021-11-03T17:15:57.419 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.421+0000 7f63f79e6700 1 -- 172.21.15.27:0/710008647 wait complete. 2021-11-03T17:15:57.420 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.422+0000 7f63f79e6700 1 Processor -- start 2021-11-03T17:15:57.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.422+0000 7f63f79e6700 1 -- start start 2021-11-03T17:15:57.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.422+0000 7f63f79e6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63f00f6bf0 0x7f63f0109c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:15:57.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.422+0000 7f63f79e6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63f00f7590 0x7f63f010c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:15:57.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.422+0000 7f63f79e6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f00f5680 con 0x7f63f00f7590 2021-11-03T17:15:57.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.422+0000 7f63f79e6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f00f57c0 con 0x7f63f00f6bf0 2021-11-03T17:15:57.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.422+0000 7f63f5782700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63f00f6bf0 0x7f63f0109c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2021-11-03T17:15:57.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.422+0000 7f63f5782700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63f00f6bf0 0x7f63f0109c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.27:47126/0 (socket says 172.21.15.27:47126) 2021-11-03T17:15:57.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.422+0000 7f63f5782700 1 -- 172.21.15.27:0/2027359171 learned_addr learned my addr 172.21.15.27:0/2027359171 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:15:57.423 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.422+0000 7f63f4f81700 1 --2- 172.21.15.27:0/2027359171 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63f00f7590 0x7f63f010c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:15:57.423 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.422+0000 7f63f5782700 1 -- 172.21.15.27:0/2027359171 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63f00f7590 msgr2=0x7f63f010c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:15:57.423 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.422+0000 7f63f5782700 1 --2- 172.21.15.27:0/2027359171 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63f00f7590 0x7f63f010c130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:57.423 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.422+0000 7f63f5782700 1 -- 172.21.15.27:0/2027359171 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e0005040 con 0x7f63f00f6bf0 2021-11-03T17:15:57.423 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.422+0000 7f63f4f81700 1 --2- 172.21.15.27:0/2027359171 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63f00f7590 0x7f63f010c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:15:57.424 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.423+0000 7f63f5782700 1 --2- 172.21.15.27:0/2027359171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63f00f6bf0 0x7f63f0109c30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f63ec00e7c0 tx=0x7f63ec00b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:15:57.424 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.424+0000 7f63e5ffb700 1 -- 172.21.15.27:0/2027359171 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63ec0092e0 con 0x7f63f00f6bf0 2021-11-03T17:15:57.424 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.424+0000 7f63e5ffb700 1 -- 172.21.15.27:0/2027359171 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f63ec00f040 con 0x7f63f00f6bf0 2021-11-03T17:15:57.424 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.424+0000 7f63e5ffb700 1 -- 172.21.15.27:0/2027359171 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63ec014750 con 0x7f63f00f6bf0 2021-11-03T17:15:57.425 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.424+0000 7f63f79e6700 1 -- 172.21.15.27:0/2027359171 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63f010c740 con 0x7f63f00f6bf0 2021-11-03T17:15:57.425 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.425+0000 7f63f79e6700 1 -- 172.21.15.27:0/2027359171 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f010ccd0 con 0x7f63f00f6bf0 2021-11-03T17:15:57.426 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.426+0000 7f63f79e6700 1 -- 172.21.15.27:0/2027359171 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63f00f4150 con 0x7f63f00f6bf0 2021-11-03T17:15:57.426 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.426+0000 7f63e5ffb700 1 -- 172.21.15.27:0/2027359171 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 33) v1 ==== 81670+0+0 (secure 0 0 0) 0x7f63ec01b030 con 0x7f63f00f6bf0 2021-11-03T17:15:57.426 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.427+0000 7f63e5ffb700 1 --2- 172.21.15.27:0/2027359171 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f63dc063dd0 0x7f63dc066290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:15:57.427 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.427+0000 7f63e5ffb700 1 -- 172.21.15.27:0/2027359171 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6375+0+0 (secure 0 0 0) 0x7f63ec080580 con 0x7f63f00f6bf0 2021-11-03T17:15:57.427 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.427+0000 7f63f4f81700 1 --2- 172.21.15.27:0/2027359171 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f63dc063dd0 0x7f63dc066290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:15:57.427 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.427+0000 7f63f4f81700 1 --2- 172.21.15.27:0/2027359171 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f63dc063dd0 0x7f63dc066290 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f63e0009720 tx=0x7f63e000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:15:57.428 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.431+0000 7f63e5ffb700 1 -- 172.21.15.27:0/2027359171 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f63ec04f0b0 con 0x7f63f00f6bf0 2021-11-03T17:15:57.432 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.436+0000 7f63e5ffb700 1 -- 172.21.15.27:0/2027359171 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mgrmap(e 34) v1 ==== 81956+0+0 (secure 0 0 0) 0x7f63ec07e700 con 0x7f63f00f6bf0 2021-11-03T17:15:57.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.650+0000 7f63f79e6700 1 -- 172.21.15.27:0/2027359171 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f63f00fdef0 con 0x7f63f00f6bf0 2021-11-03T17:15:57.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.651+0000 7f63e5ffb700 1 -- 172.21.15.27:0/2027359171 <== mon.1 v2:172.21.15.115:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 27 v27) v1 ==== 76+0+1662 (secure 0 0 0) 0x7f63ec039df0 con 0x7f63f00f6bf0 2021-11-03T17:15:57.647 INFO:teuthology.orchestra.run.smithi027.stdout:e27 2021-11-03T17:15:57.647 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:15:57.647 INFO:teuthology.orchestra.run.smithi027.stdout:compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:15:57.647 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:15:57.648 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:15:57.648 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:15:57.648 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:15:57.648 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 27 2021-11-03T17:15:57.649 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 2021-11-03T17:15:57.649 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:15:57.649 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:15:18.348938+0000 2021-11-03T17:15:57.649 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:15:57.650 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:15:57.650 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:15:57.650 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:15:57.650 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:15:57.651 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:15:57.651 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:15:57.651 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 51 2021-11-03T17:15:57.651 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:15:57.652 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 2 2021-11-03T17:15:57.652 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:15:57.652 INFO:teuthology.orchestra.run.smithi027.stdout:up {0=24289,1=14436} 2021-11-03T17:15:57.652 INFO:teuthology.orchestra.run.smithi027.stdout:failed 2021-11-03T17:15:57.652 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:15:57.653 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:15:57.653 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:15:57.653 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:15:57.653 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:15:57.654 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:15:57.654 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:15:57.654 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{0:24289} state up:active seq 51 export targets 1 join_fscid=1 addr [v2:172.21.15.115:6826/3367380088,v1:172.21.15.115:6827/3367380088]] 2021-11-03T17:15:57.654 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 3 export targets 0 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060]] 2021-11-03T17:15:57.654 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:15:57.655 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:15:57.655 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:15:57.655 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:15:57.655 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635]] 2021-11-03T17:15:57.656 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060]] 2021-11-03T17:15:57.657 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.653+0000 7f63f79e6700 1 -- 172.21.15.27:0/2027359171 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f63dc063dd0 msgr2=0x7f63dc066290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:15:57.657 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.653+0000 7f63f79e6700 1 --2- 172.21.15.27:0/2027359171 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f63dc063dd0 0x7f63dc066290 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f63e0009720 tx=0x7f63e000b040).stop 2021-11-03T17:15:57.657 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.653+0000 7f63f79e6700 1 -- 172.21.15.27:0/2027359171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63f00f6bf0 msgr2=0x7f63f0109c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:15:57.657 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.653+0000 7f63f79e6700 1 --2- 172.21.15.27:0/2027359171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63f00f6bf0 0x7f63f0109c30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f63ec00e7c0 tx=0x7f63ec00b040).stop 2021-11-03T17:15:57.658 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.653+0000 7f63f79e6700 1 -- 172.21.15.27:0/2027359171 shutdown_connections 2021-11-03T17:15:57.658 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.653+0000 7f63f79e6700 1 --2- 172.21.15.27:0/2027359171 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f63dc063dd0 0x7f63dc066290 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:57.658 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.653+0000 7f63f79e6700 1 --2- 172.21.15.27:0/2027359171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f63f00f6bf0 0x7f63f0109c30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:57.658 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.653+0000 7f63f79e6700 1 --2- 172.21.15.27:0/2027359171 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63f00f7590 0x7f63f010c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:15:57.659 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.653+0000 7f63f79e6700 1 -- 172.21.15.27:0/2027359171 >> 172.21.15.27:0/2027359171 conn(0x7f63f00f04c0 msgr2=0x7f63f00fa3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:15:57.659 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.653+0000 7f63f79e6700 1 -- 172.21.15.27:0/2027359171 shutdown_connections 2021-11-03T17:15:57.659 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:15:57.653+0000 7f63f79e6700 1 -- 172.21.15.27:0/2027359171 wait complete. 2021-11-03T17:15:57.659 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 27 2021-11-03T17:15:57.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:57 smithi115 conmon[34670]: cluster 2021-11-03T17:15:56.414274+0000 mon.smithi027 (mon.0) 39 : 2021-11-03T17:15:57.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:57 smithi115 conmon[34670]: cluster [DBG] mgrmap e33: smithi027.ufhhvb(active, since 1.97556s), standbys: smithi115.tcyodz 2021-11-03T17:15:57.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:57 smithi115 conmon[34670]: audit 2021-11-03T17:15:56.423097+0000 mgr.smithi027.ufhhvb (mgr.34104) 1 : audit [DBG] from='client.24499 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:15:57.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:57 smithi115 conmon[34670]: cluster 2021-11-03T17:15:56.429945+0000 mgr.smithi027.ufhhvb (mgr.34104 2021-11-03T17:15:57.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:57 smithi115 conmon[34670]: ) 2 : cluster [DBG] pgmap v3: 65 pgs: 65 active+clean; 823 MiB data, 2.9 GiB used, 533 GiB / 536 GiB avail 2021-11-03T17:15:57.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:57 smithi115 conmon[34670]: audit 2021-11- 2021-11-03T17:15:57.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:57 smithi115 conmon[34670]: 03T17:15:56.916083+0000 mgr.smithi027.ufhhvb (mgr.34104) 2021-11-03T17:15:57.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:57 smithi115 conmon[34670]: 3 : audit [DBG] from='client.24525 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:15:57.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:57 smithi115 conmon[34670]: audit 2021-11-03T17: 2021-11-03T17:15:57.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:57 smithi115 conmon[34670]: 15:57.301543+0000 mon.smithi115 (mon.1) 70 : audit [DBG] from='client.? 172.21.15.27:0/4261618275' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:15:57.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:57 smithi115 conmon[34670]: 2021-11-03T17:15:58.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:58 smithi115 conmon[34670]: cluster 2021-11-03T17:15:57.416279+0000 mgr.smithi027.ufhhvb (mgr.34104) 4 : cluster [DBG] pgmap v4: 65 pgs: 65 active+clean; 823 MiB data, 2.9 GiB used, 533 GiB / 536 GiB avail 2021-11-03T17:15:58.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:58 smithi115 conmon[34670]: 2021-11-03T17:15:58.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:58 smithi115 conmon[34670]: cluster 2021-11-03T17:15:57.433186+0000 mon.smithi027 (mon.0) 40 : cluster [DBG] mgrmap e34: smithi027.ufhhvb(active, since 2s), standbys: smithi115.tcyodz 2021-11-03T17:15:58.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:15:58 smithi115 conmon[34670]: audit 2021-11-03T17:15:57.652249+0000 mon.smithi115 (mon.1) 71 : audit [DBG] from='client.? 172.21.15.27:0/2027359171' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:16:00.603 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:00 smithi115 conmon[34670]: cephadm 2021-11-03T17:15:59.165268+0000 mgr.smithi027.ufhhvb (mgr.34104) 5 : cephadm [INF] Starting cherrypy engine... 2021-11-03T17:16:00.603 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:00 smithi115 conmon[34670]: 2021-11-03T17:16:00.603 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:00 smithi115 conmon[34670]: cephadm 2021-11-03T17:15:59.165830+0000 mgr.smithi027.ufhhvb (mgr.34104) 6 : cephadm [INF] [03/Nov/2021:17:15:59] ENGINE Bus STARTING 2021-11-03T17:16:00.604 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:00 smithi115 conmon[34670]: cephadm 2021-11-03T17:15:59.283080+0000 mgr.smithi027.ufhhvb (mgr.34104) 7 : cephadm [INF] [03/Nov/2021:17:15:59] ENGINE Serving on https://172.21.15.27:8499 2021-11-03T17:16:00.604 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:00 smithi115 conmon[34670]: cephadm 2021-11-03T17:15:59.283291+0000 mgr.smithi027.ufhhvb (mgr.34104) 8 : cephadm [INF] [03/Nov/2021:17:15:59] ENGINE Bus STARTED 2021-11-03T17:16:00.604 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:00 smithi115 conmon[34670]: cephadm 2021-11-03T17:15:59.283364+0000 mgr.smithi027.ufhhvb (mgr.34104) 9 : cephadm [INF] Cherrypy engine started. 2021-11-03T17:16:00.604 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:00 smithi115 conmon[34670]: cluster 2021-11-03T17:15:59.416630+0000 mgr.smithi027.ufhhvb (mgr.34104) 10 : cluster [DBG] pgmap v5: 65 pgs: 65 active+clean; 823 MiB data, 2.9 GiB used, 533 GiB / 536 GiB avail 2021-11-03T17:16:02.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:02 smithi115 conmon[34670]: cluster 2021-11-03T17:16:01.412213+0000 mon.smithi027 (mon.0) 41 : cluster [DBG] mgrmap e35: smithi027.ufhhvb(active, since 6s), standbys: smithi115.tcyodz 2021-11-03T17:16:02.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:02 smithi115 conmon[34670]: cluster 2021-11-03T17:16:01.417057+0000 mgr.smithi027.ufhhvb (mgr.34104) 11 : cluster [DBG] pgmap v6: 65 pgs: 65 active+clean; 774 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 86 KiB/s wr, 13 op/s 2021-11-03T17:16:02.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:02 smithi115 conmon[34670]: cephadm 2021-11-03T17:16:01.602405+0000 mgr.smithi027.ufhhvb (mgr.34104) 12 : cephadm [INF] poll took 5537.853 ms: 1 events 2021-11-03T17:16:02.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:02 smithi115 conmon[34670]: audit 2021-11-03T17:16:01.630687+0000 mon.smithi027 (mon.0) 42 : audit [INF] from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:02.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:02 smithi115 conmon[34670]: audit 2021-11-03T17:16:02.276712+0000 mon.smithi027 (mon.0) 43 : audit [INF] from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:02.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:02 smithi115 conmon[34670]: audit 2021-11-03T17:16:02.283320+0000 mon.smithi027 (mon.0) 44 : audit [INF] from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"}]: dispatch 2021-11-03T17:16:04.499 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:04 smithi115 conmon[34670]: cluster 2021-11-03T17:16:03.417771+0000 mgr.smithi027.ufhhvb (mgr.34104) 13 : cluster [DBG] 2021-11-03T17:16:04.500 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:04 smithi115 conmon[34670]: pgmap v7: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 4.4 KiB/s rd, 92 KiB/s wr, 34 op/s 2021-11-03T17:16:05.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:05 smithi115 conmon[34670]: cephadm 2021-11-03T17:16:04.488577+0000 mgr.smithi027.ufhhvb (mgr.34104) 14 : cephadm 2021-11-03T17:16:05.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:05 smithi115 conmon[34670]: [INF] poll took 2223.456 ms: 1 events 2021-11-03T17:16:05.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:05 smithi115 conmon[34670]: audit 2021-11-03T17:16:04.518382+0000 mon.smithi027 (mon.0) 45 : audit [INF] from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:05.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:05 smithi115 conmon[34670]: audit 2021-11-03T17:16:05.135940+0000 mon.smithi027 (mon.0) 46 : audit [INF] from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:05.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:05 smithi115 conmon[34670]: audit 2021-11-03T17:16:05.137340+0000 mon.smithi027 (mon.0) 47 : audit [INF] from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2021-11-03T17:16:05.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:05 smithi115 conmon[34670]: audit 2021-11-03T17:16:05.144349+0000 mon.smithi027 (mon.0) 48 : audit [INF] from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:05.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:05 smithi115 conmon[34670]: audit 2021-11-03T17:16:05.151315+0000 mon.smithi027 (mon.0) 49 : audit [DBG] from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:16:05.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:05 smithi115 conmon[34670]: audit 2021-11-03T17: 2021-11-03T17:16:05.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:05 smithi115 conmon[34670]: 16:05.152828+0000 mon.smithi027 (mon.0) 50 : audit [DBG] from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:16:05.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:05 smithi115 conmon[34670]: audit 2021-11-03T17:16:05.154015+0000 mon.smithi027 (mon.0) 51 : audit [DBG] from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "quorum_status"}]: dispatch 2021-11-03T17:16:06.765 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:06 smithi115 conmon[34670]: cluster 2021-11-03T17:16:05.138858+0000 mgr.smithi027.ufhhvb (mgr.34104 2021-11-03T17:16:06.766 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:06 smithi115 conmon[34670]: ) 15 : cluster [DBG] pgmap v8: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 3.6 KiB/s rd, 73 KiB/s wr, 27 op/s 2021-11-03T17:16:06.766 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:06 smithi115 conmon[34670]: cluster 2021-11-03T17:16:06.133853+0000 mon.smithi027 (mon.0) 52 : cluster [WRN] Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2021-11-03T17:16:07.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:07 smithi115 conmon[34670]: cephadm 2021-11-03T17:16:06.712800+0000 mgr.smithi027.ufhhvb (mgr.34104) 16 : cephadm [INF] poll took 1524.765 ms: 1 events 2021-11-03T17:16:07.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:07 smithi115 conmon[34670]: cephadm 2021-11-03T17:16:06.737194+0000 mgr.smithi027.ufhhvb (mgr.34104) 17 : cephadm [INF] Upgrade: Updating mon.smithi115 2021-11-03T17:16:07.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:07 smithi115 conmon[34670]: audit 2021-11-03T17:16:06.741847+0000 mon.smithi027 (mon.0) 53 : audit [INF] from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:07.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:07 smithi115 conmon[34670]: audit 2021-11-03T17:16:06.742846+0000 mon.smithi027 (mon.0) 54 : audit [INF] from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-11-03T17:16:07.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:07 smithi115 conmon[34670]: audit 2021-11-03T17:16:06.743796+0000 mon.smithi027 (mon.0) 55 : audit [DBG] from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-11-03T17:16:07.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:07 smithi115 conmon[34670]: audit 2021-11-03T17:16:06.744703+0000 mon.smithi027 (mon.0) 56 : audit [DBG] from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:16:07.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:07 smithi115 conmon[34670]: cephadm 2021-11-03T17:16:06.745861+0000 mgr.smithi027.ufhhvb (mgr.34104) 18 : cephadm [INF] Deploying daemon mon.smithi115 on smithi115 2021-11-03T17:16:08.452 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:08 smithi115 systemd[1]: Stopping Ceph mon.smithi115 for 32ab977e-3cc8-11ec-8c28-001a4aab830c... 2021-11-03T17:16:08.453 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:08 smithi115 conmon[34670]: debug 2021-11-03T17:16:08.415+0000 7f7b49b18700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.smithi115 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2021-11-03T17:16:08.453 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:08 smithi115 conmon[34670]: debug 2021-11-03T17:16:08.415+0000 7f7b49b18700 -1 mon.smithi115@1(peon) e2 *** Got Signal Terminated *** 2021-11-03T17:16:09.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 bash[65035]: 319323f0983442a469e49342deb68dcf097e9049c2caef2669ebf60c3a070d28 2021-11-03T17:16:09.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 bash[65035]: Error: no container with name or ID ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c-mon-smithi115 found: no such container 2021-11-03T17:16:09.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 bash[65035]: Error: no container with name or ID ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c-mon.smithi115 found: no such container 2021-11-03T17:16:09.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 systemd[1]: ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c@mon.smithi115.service: Succeeded. 2021-11-03T17:16:09.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 systemd[1]: Stopped Ceph mon.smithi115 for 32ab977e-3cc8-11ec-8c28-001a4aab830c. 2021-11-03T17:16:09.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 systemd[1]: Starting Ceph mon.smithi115 for 32ab977e-3cc8-11ec-8c28-001a4aab830c... 2021-11-03T17:16:09.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: set uid:gid to 167:167 (ceph:ceph) 2021-11-03T17:16:09.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev), process ceph-mon, pid 7 2021-11-03T17:16:09.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: pidfile_write: ignore empty --pid-file 2021-11-03T17:16:09.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: load: jerasure load: lrc 2021-11-03T17:16:09.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: RocksDB version: 6.15.5 2021-11-03T17:16:09.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2021-11-03T17:16:09.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Compile date Nov 3 2021 2021-11-03T17:16:09.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: DB SUMMARY 2021-11-03T17:16:09.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: DB Session ID: PTJ84C27UI8LW0RF4KW2 2021-11-03T17:16:09.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: CURRENT file: CURRENT 2021-11-03T17:16:09.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: IDENTITY file: IDENTITY 2021-11-03T17:16:09.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: MANIFEST file: MANIFEST-000005 size: 703 Bytes 2021-11-03T17:16:09.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi115/store.db dir, Total Num: 1, files: 000017.sst 2021-11-03T17:16:09.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi115/store.db: 000015.log size: 3718655 ; 2021-11-03T17:16:09.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.error_if_exists: 0 2021-11-03T17:16:09.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.create_if_missing: 0 2021-11-03T17:16:09.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.paranoid_checks: 1 2021-11-03T17:16:09.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2021-11-03T17:16:09.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.env: 0x564127ac6c00 2021-11-03T17:16:09.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.fs: Posix File System 2021-11-03T17:16:09.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.info_log: 0x56412924e460 2021-11-03T17:16:09.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.max_file_opening_threads: 16 2021-11-03T17:16:09.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.statistics: (nil) 2021-11-03T17:16:09.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.use_fsync: 0 2021-11-03T17:16:09.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.max_log_file_size: 0 2021-11-03T17:16:09.991 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.max_manifest_file_size: 1073741824 2021-11-03T17:16:09.991 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.log_file_time_to_roll: 0 2021-11-03T17:16:09.991 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.keep_log_file_num: 1000 2021-11-03T17:16:09.991 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.recycle_log_file_num: 0 2021-11-03T17:16:09.992 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.allow_fallocate: 1 2021-11-03T17:16:09.992 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.allow_mmap_reads: 0 2021-11-03T17:16:09.992 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.allow_mmap_writes: 0 2021-11-03T17:16:09.992 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.use_direct_reads: 0 2021-11-03T17:16:09.993 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2021-11-03T17:16:09.993 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.create_missing_column_families: 0 2021-11-03T17:16:09.993 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.db_log_dir: 2021-11-03T17:16:09.993 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-smithi115/store.db 2021-11-03T17:16:09.994 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.table_cache_numshardbits: 6 2021-11-03T17:16:09.994 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.WAL_ttl_seconds: 0 2021-11-03T17:16:09.994 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.WAL_size_limit_MB: 0 2021-11-03T17:16:09.994 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2021-11-03T17:16:09.994 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.manifest_preallocation_size: 4194304 2021-11-03T17:16:09.995 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.is_fd_close_on_exec: 1 2021-11-03T17:16:09.995 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.advise_random_on_open: 1 2021-11-03T17:16:09.995 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.db_write_buffer_size: 0 2021-11-03T17:16:09.995 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.write_buffer_manager: 0x5641292c3f20 2021-11-03T17:16:09.996 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.access_hint_on_compaction_start: 1 2021-11-03T17:16:09.996 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2021-11-03T17:16:09.996 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.random_access_max_buffer_size: 1048576 2021-11-03T17:16:09.996 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.use_adaptive_mutex: 0 2021-11-03T17:16:09.997 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.rate_limiter: (nil) 2021-11-03T17:16:09.997 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2021-11-03T17:16:09.997 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.wal_recovery_mode: 2 2021-11-03T17:16:09.997 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.enable_thread_tracking: 0 2021-11-03T17:16:09.997 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.enable_pipelined_write: 0 2021-11-03T17:16:09.998 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.unordered_write: 0 2021-11-03T17:16:09.998 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.allow_concurrent_memtable_write: 1 2021-11-03T17:16:09.998 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2021-11-03T17:16:09.998 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.write_thread_max_yield_usec: 100 2021-11-03T17:16:09.999 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.write_thread_slow_yield_usec: 3 2021-11-03T17:16:09.999 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.row_cache: None 2021-11-03T17:16:09.999 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.wal_filter: None 2021-11-03T17:16:09.999 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.avoid_flush_during_recovery: 0 2021-11-03T17:16:10.000 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.allow_ingest_behind: 0 2021-11-03T17:16:10.000 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.preserve_deletes: 0 2021-11-03T17:16:10.000 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.two_write_queues: 0 2021-11-03T17:16:10.000 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.manual_wal_flush: 0 2021-11-03T17:16:10.000 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.atomic_flush: 0 2021-11-03T17:16:10.001 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2021-11-03T17:16:10.001 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.persist_stats_to_disk: 0 2021-11-03T17:16:10.001 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.write_dbid_to_manifest: 0 2021-11-03T17:16:10.001 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.log_readahead_size: 0 2021-11-03T17:16:10.002 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.file_checksum_gen_factory: Unknown 2021-11-03T17:16:10.002 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.best_efforts_recovery: 0 2021-11-03T17:16:10.002 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2021-11-03T17:16:10.002 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2021-11-03T17:16:10.003 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.allow_data_in_errors: 0 2021-11-03T17:16:10.003 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.db_host_id: __hostname__ 2021-11-03T17:16:10.003 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.max_background_jobs: 2 2021-11-03T17:16:10.003 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.max_background_compactions: -1 2021-11-03T17:16:10.004 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.max_subcompactions: 1 2021-11-03T17:16:10.004 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.avoid_flush_during_shutdown: 0 2021-11-03T17:16:10.004 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2021-11-03T17:16:10.004 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.delayed_write_rate : 16777216 2021-11-03T17:16:10.005 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.max_total_wal_size: 0 2021-11-03T17:16:10.005 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2021-11-03T17:16:10.005 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.stats_dump_period_sec: 600 2021-11-03T17:16:10.005 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.stats_persist_period_sec: 600 2021-11-03T17:16:10.006 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.stats_history_buffer_size: 1048576 2021-11-03T17:16:10.006 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.max_open_files: -1 2021-11-03T17:16:10.006 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.bytes_per_sync: 0 2021-11-03T17:16:10.006 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.wal_bytes_per_sync: 0 2021-11-03T17:16:10.007 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.strict_bytes_per_sync: 0 2021-11-03T17:16:10.007 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.compaction_readahead_size: 0 2021-11-03T17:16:10.007 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.max_background_flushes: -1 2021-11-03T17:16:10.007 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Compression algorithms supported: 2021-11-03T17:16:10.007 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: kZSTDNotFinalCompression supported: 0 2021-11-03T17:16:10.008 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: kZSTD supported: 0 2021-11-03T17:16:10.008 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: kXpressCompression supported: 0 2021-11-03T17:16:10.008 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: kLZ4HCCompression supported: 1 2021-11-03T17:16:10.008 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: kLZ4Compression supported: 1 2021-11-03T17:16:10.009 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: kBZip2Compression supported: 0 2021-11-03T17:16:10.009 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: kZlibCompression supported: 1 2021-11-03T17:16:10.009 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: kSnappyCompression supported: 1 2021-11-03T17:16:10.009 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Fast CRC32 supported: Supported on x86 2021-11-03T17:16:10.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: [db/version_set.cc:4725] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi115/store.db/MANIFEST-000005 2021-11-03T17:16:10.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: [db/column_family.cc:597] --------------- Options for column family [default]: 2021-11-03T17:16:10.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2021-11-03T17:16:10.010 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.merge_operator: 2021-11-03T17:16:10.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.compaction_filter: None 2021-11-03T17:16:10.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.compaction_filter_factory: None 2021-11-03T17:16:10.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.sst_partitioner_factory: None 2021-11-03T17:16:10.011 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.memtable_factory: SkipListFactory 2021-11-03T17:16:10.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.table_factory: BlockBasedTable 2021-11-03T17:16:10.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x564129217bc0) 2021-11-03T17:16:10.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: cache_index_and_filter_blocks: 1 2021-11-03T17:16:10.012 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2021-11-03T17:16:10.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2021-11-03T17:16:10.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: pin_top_level_index_and_filter: 1 2021-11-03T17:16:10.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: index_type: 0 2021-11-03T17:16:10.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: data_block_index_type: 0 2021-11-03T17:16:10.013 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: index_shortening: 1 2021-11-03T17:16:10.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: data_block_hash_table_util_ratio: 0.750000 2021-11-03T17:16:10.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: hash_index_allow_collision: 1 2021-11-03T17:16:10.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: checksum: 1 2021-11-03T17:16:10.014 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: no_block_cache: 0 2021-11-03T17:16:10.015 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: block_cache: 0x56412924cd10 2021-11-03T17:16:10.015 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: block_cache_name: BinnedLRUCache 2021-11-03T17:16:10.015 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: block_cache_options: 2021-11-03T17:16:10.015 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: capacity : 536870912 2021-11-03T17:16:10.016 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: num_shard_bits : 4 2021-11-03T17:16:10.016 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: strict_capacity_limit : 0 2021-11-03T17:16:10.016 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: high_pri_pool_ratio: 0.000 2021-11-03T17:16:10.016 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: block_cache_compressed: (nil) 2021-11-03T17:16:10.017 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: persistent_cache: (nil) 2021-11-03T17:16:10.017 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: block_size: 4096 2021-11-03T17:16:10.017 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: block_size_deviation: 10 2021-11-03T17:16:10.017 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: block_restart_interval: 16 2021-11-03T17:16:10.018 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: index_block_restart_interval: 1 2021-11-03T17:16:10.018 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: metadata_block_size: 4096 2021-11-03T17:16:10.018 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: partition_filters: 0 2021-11-03T17:16:10.018 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: use_delta_encoding: 1 2021-11-03T17:16:10.019 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2021-11-03T17:16:10.019 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: whole_key_filtering: 1 2021-11-03T17:16:10.019 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: verify_compression: 0 2021-11-03T17:16:10.019 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: read_amp_bytes_per_bit: 0 2021-11-03T17:16:10.020 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: format_version: 4 2021-11-03T17:16:10.020 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: enable_index_compression: 1 2021-11-03T17:16:10.020 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: block_align: 0 2021-11-03T17:16:10.020 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.write_buffer_size: 33554432 2021-11-03T17:16:10.020 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.max_write_buffer_number: 2 2021-11-03T17:16:10.021 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.compression: NoCompression 2021-11-03T17:16:10.021 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.bottommost_compression: Disabled 2021-11-03T17:16:10.021 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.prefix_extractor: nullptr 2021-11-03T17:16:10.021 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2021-11-03T17:16:10.022 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.num_levels: 7 2021-11-03T17:16:10.022 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2021-11-03T17:16:10.022 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2021-11-03T17:16:10.022 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2021-11-03T17:16:10.023 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2021-11-03T17:16:10.023 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.bottommost_compression_opts.level: 32767 2021-11-03T17:16:10.023 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2021-11-03T17:16:10.023 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2021-11-03T17:16:10.024 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2021-11-03T17:16:10.024 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2021-11-03T17:16:10.024 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.bottommost_compression_opts.enabled: false 2021-11-03T17:16:10.024 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.compression_opts.window_bits: -14 2021-11-03T17:16:10.025 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.compression_opts.level: 32767 2021-11-03T17:16:10.026 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.compression_opts.strategy: 0 2021-11-03T17:16:10.026 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2021-11-03T17:16:10.027 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2021-11-03T17:16:10.027 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.compression_opts.parallel_threads: 1 2021-11-03T17:16:10.027 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.compression_opts.enabled: false 2021-11-03T17:16:10.027 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2021-11-03T17:16:10.028 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2021-11-03T17:16:10.028 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.level0_stop_writes_trigger: 36 2021-11-03T17:16:10.028 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.target_file_size_base: 67108864 2021-11-03T17:16:10.028 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.target_file_size_multiplier: 1 2021-11-03T17:16:10.028 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.max_bytes_for_level_base: 268435456 2021-11-03T17:16:10.029 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2021-11-03T17:16:10.029 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2021-11-03T17:16:10.029 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2021-11-03T17:16:10.029 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2021-11-03T17:16:10.030 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2021-11-03T17:16:10.030 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2021-11-03T17:16:10.030 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2021-11-03T17:16:10.030 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2021-11-03T17:16:10.031 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2021-11-03T17:16:10.031 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2021-11-03T17:16:10.031 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.max_compaction_bytes: 1677721600 2021-11-03T17:16:10.031 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.arena_block_size: 4194304 2021-11-03T17:16:10.032 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2021-11-03T17:16:10.032 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2021-11-03T17:16:10.032 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2021-11-03T17:16:10.032 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.disable_auto_compactions: 0 2021-11-03T17:16:10.033 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2021-11-03T17:16:10.033 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2021-11-03T17:16:10.033 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2021-11-03T17:16:10.033 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2021-11-03T17:16:10.034 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2021-11-03T17:16:10.034 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2021-11-03T17:16:10.034 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2021-11-03T17:16:10.034 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2021-11-03T17:16:10.035 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2021-11-03T17:16:10.035 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2021-11-03T17:16:10.035 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.table_properties_collectors: 2021-11-03T17:16:10.035 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.inplace_update_support: 0 2021-11-03T17:16:10.036 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.inplace_update_num_locks: 10000 2021-11-03T17:16:10.036 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2021-11-03T17:16:10.036 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.memtable_whole_key_filtering: 0 2021-11-03T17:16:10.036 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.memtable_huge_page_size: 0 2021-11-03T17:16:10.037 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.bloom_locality: 0 2021-11-03T17:16:10.037 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.max_successive_merges: 0 2021-11-03T17:16:10.037 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.optimize_filters_for_hits: 0 2021-11-03T17:16:10.037 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.paranoid_file_checks: 0 2021-11-03T17:16:10.037 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.force_consistency_checks: 1 2021-11-03T17:16:10.038 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.report_bg_io_stats: 0 2021-11-03T17:16:10.038 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.ttl: 2592000 2021-11-03T17:16:10.038 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.periodic_compaction_seconds: 0 2021-11-03T17:16:10.038 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.enable_blob_files: false 2021-11-03T17:16:10.039 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.min_blob_size: 0 2021-11-03T17:16:10.039 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.blob_file_size: 268435456 2021-11-03T17:16:10.039 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.blob_compression_type: NoCompression 2021-11-03T17:16:10.039 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.enable_blob_garbage_collection: false 2021-11-03T17:16:10.040 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2021-11-03T17:16:10.040 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: [db/version_set.cc:3459] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2021-11-03T17:16:10.040 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: [db/version_set.cc:4773] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi115/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 19, last_sequence is 9556, log_number is 15,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2021-11-03T17:16:10.040 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: [db/version_set.cc:4782] Column family [default] (ID 0), log number is 15 2021-11-03T17:16:10.040 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: [db/version_set.cc:4083] Creating manifest 19 2021-11-03T17:16:10.041 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: [db/version_set.cc:3459] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2021-11-03T17:16:10.041 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: EVENT_LOG_v1 {"time_micros": 1635959769810004, "job": 1, "event": "recovery_started", "wal_files": [15]} 2021-11-03T17:16:10.041 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: [db/db_impl/db_impl_open.cc:847] Recovering log #15 mode 2 2021-11-03T17:16:10.041 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: [table/block_based/filter_policy.cc:996] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2021-11-03T17:16:10.042 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: EVENT_LOG_v1 {"time_micros": 1635959769815991, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 20, "file_size": 2438532, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 2435501, "index_size": 1643, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 453, "raw_key_size": 2778, "raw_average_key_size": 21, "raw_value_size": 2431618, "raw_average_value_size": 18849, "num_data_blocks": 74, "num_entries": 129, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1635959769, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "a3be1bc0-54a5-4a49-90b7-9544ecc19941", "db_session_id": "PTJ84C27UI8LW0RF4KW2"}} 2021-11-03T17:16:10.042 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: [db/version_set.cc:4083] Creating manifest 21 2021-11-03T17:16:10.042 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: [db/version_set.cc:3459] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2021-11-03T17:16:10.042 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: EVENT_LOG_v1 {"time_micros": 1635959769816383, "job": 1, "event": "recovery_finished"} 2021-11-03T17:16:10.043 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: [file/delete_scheduler.cc:73] Deleted file /var/lib/ceph/mon/ceph-smithi115/store.db/000015.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2021-11-03T17:16:10.043 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 bash[65236]: e3f7967a24d5a8527ffaaf99907f9dc3ddee0b2d11c1047ef145f3a81a246264 2021-11-03T17:16:10.043 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: [db/db_impl/db_impl_open.cc:1701] SstFileManager instance 0x564129268a80 2021-11-03T17:16:10.043 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: DB pointer 0x564129278000 2021-11-03T17:16:10.044 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: [db/db_impl/db_impl.cc:902] ------- DUMPING STATS ------- 2021-11-03T17:16:10.044 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: rocksdb: [db/db_impl/db_impl.cc:903] 2021-11-03T17:16:10.044 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: ** DB Stats ** 2021-11-03T17:16:10.044 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Uptime(secs): 0.0 total, 0.0 interval 2021-11-03T17:16:10.044 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2021-11-03T17:16:10.045 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2021-11-03T17:16:10.045 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2021-11-03T17:16:10.045 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2021-11-03T17:16:10.045 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2021-11-03T17:16:10.046 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2021-11-03T17:16:10.046 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: 2021-11-03T17:16:10.046 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: ** Compaction Stats [default] ** 2021-11-03T17:16:10.046 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-11-03T17:16:10.047 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-11-03T17:16:10.047 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: L0 1/0 2.33 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 837.4 0.00 0.00 1 0.003 0 0 2021-11-03T17:16:10.047 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: L6 1/0 22.67 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2021-11-03T17:16:10.047 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Sum 2/0 25.00 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 837.4 0.00 0.00 1 0.003 0 0 2021-11-03T17:16:10.047 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 837.4 0.00 0.00 1 0.003 0 0 2021-11-03T17:16:10.048 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: 2021-11-03T17:16:10.048 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: ** Compaction Stats [default] ** 2021-11-03T17:16:10.048 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-11-03T17:16:10.048 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-11-03T17:16:10.049 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 837.4 0.00 0.00 1 0.003 0 0 2021-11-03T17:16:10.049 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Uptime(secs): 0.0 total, 0.0 interval 2021-11-03T17:16:10.049 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Flush(GB): cumulative 0.002, interval 0.002 2021-11-03T17:16:10.049 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2021-11-03T17:16:10.050 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: AddFile(Total Files): cumulative 0, interval 0 2021-11-03T17:16:10.050 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: AddFile(L0 Files): cumulative 0, interval 0 2021-11-03T17:16:10.050 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: AddFile(Keys): cumulative 0, interval 0 2021-11-03T17:16:10.050 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Cumulative compaction: 0.00 GB write, 264.39 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-11-03T17:16:10.050 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Interval compaction: 0.00 GB write, 264.39 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-11-03T17:16:10.051 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2021-11-03T17:16:10.051 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: 2021-11-03T17:16:10.051 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: ** File Read Latency Histogram By Level [default] ** 2021-11-03T17:16:10.051 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: 2021-11-03T17:16:10.052 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: ** Compaction Stats [default] ** 2021-11-03T17:16:10.052 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-11-03T17:16:10.052 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-11-03T17:16:10.052 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: L0 1/0 2.33 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 837.4 0.00 0.00 1 0.003 0 0 2021-11-03T17:16:10.053 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: L6 1/0 22.67 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2021-11-03T17:16:10.053 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Sum 2/0 25.00 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 837.4 0.00 0.00 1 0.003 0 0 2021-11-03T17:16:10.053 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2021-11-03T17:16:10.053 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: 2021-11-03T17:16:10.053 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: ** Compaction Stats [default] ** 2021-11-03T17:16:10.054 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2021-11-03T17:16:10.054 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2021-11-03T17:16:10.054 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 837.4 0.00 0.00 1 0.003 0 0 2021-11-03T17:16:10.054 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Uptime(secs): 0.0 total, 0.0 interval 2021-11-03T17:16:10.055 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Flush(GB): cumulative 0.002, interval 0.000 2021-11-03T17:16:10.055 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2021-11-03T17:16:10.055 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: AddFile(Total Files): cumulative 0, interval 0 2021-11-03T17:16:10.055 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: AddFile(L0 Files): cumulative 0, interval 0 2021-11-03T17:16:10.056 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: AddFile(Keys): cumulative 0, interval 0 2021-11-03T17:16:10.056 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Cumulative compaction: 0.00 GB write, 263.37 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-11-03T17:16:10.056 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2021-11-03T17:16:10.056 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2021-11-03T17:16:10.057 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: 2021-11-03T17:16:10.057 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: ** File Read Latency Histogram By Level [default] ** 2021-11-03T17:16:10.057 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: starting mon.smithi115 rank 1 at public addrs [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] at bind addrs [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi115 fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c 2021-11-03T17:16:10.057 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: mon.smithi115@-1(???) e2 preinit fsid 32ab977e-3cc8-11ec-8c28-001a4aab830c 2021-11-03T17:16:10.058 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: mon.smithi115@-1(???).mds e27 new map 2021-11-03T17:16:10.058 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: mon.smithi115@-1(???).mds e27 print_map 2021-11-03T17:16:10.058 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: e27 2021-11-03T17:16:10.058 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:16:10.059 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:16:10.059 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: legacy client fscid: 1 2021-11-03T17:16:10.059 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: 2021-11-03T17:16:10.059 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Filesystem 'cephfs' (1) 2021-11-03T17:16:10.059 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: fs_name cephfs 2021-11-03T17:16:10.060 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: epoch 27 2021-11-03T17:16:10.060 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:16:10.060 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:16:10.060 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: modified 2021-11-03T17:15:18.348938+0000 2021-11-03T17:16:10.061 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: tableserver 0 2021-11-03T17:16:10.061 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: root 0 2021-11-03T17:16:10.061 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: session_timeout 60 2021-11-03T17:16:10.061 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: session_autoclose 300 2021-11-03T17:16:10.062 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: max_file_size 1099511627776 2021-11-03T17:16:10.062 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: required_client_features {} 2021-11-03T17:16:10.062 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: last_failure 0 2021-11-03T17:16:10.062 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: last_failure_osd_epoch 51 2021-11-03T17:16:10.063 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:16:10.063 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: max_mds 2 2021-11-03T17:16:10.063 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: in 0,1 2021-11-03T17:16:10.063 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: up {0=24289,1=14436} 2021-11-03T17:16:10.064 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: failed 2021-11-03T17:16:10.064 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: damaged 2021-11-03T17:16:10.064 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: stopped 2021-11-03T17:16:10.064 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: data_pools [3] 2021-11-03T17:16:10.064 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: metadata_pool 2 2021-11-03T17:16:10.065 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: inline_data disabled 2021-11-03T17:16:10.065 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: balancer 2021-11-03T17:16:10.065 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: standby_count_wanted 1 2021-11-03T17:16:10.065 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: [mds.cephfs.smithi115.semywj{0:24289} state up:active seq 51 export targets 1 join_fscid=1 addr [v2:172.21.15.115:6826/3367380088,v1:172.21.15.115:6827/3367380088] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:16:10.066 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: [mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 3 export targets 0 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:16:10.066 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: 2021-11-03T17:16:10.066 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: 2021-11-03T17:16:10.066 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: Standby daemons: 2021-11-03T17:16:10.067 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: 2021-11-03T17:16:10.067 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: [mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:16:10.067 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout: [mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:16:10.067 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: mon.smithi115@-1(???).osd e52 crush map has features 3314933000852226048, adjusting msgr requires 2021-11-03T17:16:10.068 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: mon.smithi115@-1(???).osd e52 crush map has features 288514051259236352, adjusting msgr requires 2021-11-03T17:16:10.068 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: mon.smithi115@-1(???).osd e52 crush map has features 288514051259236352, adjusting msgr requires 2021-11-03T17:16:10.068 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: mon.smithi115@-1(???).osd e52 crush map has features 288514051259236352, adjusting msgr requires 2021-11-03T17:16:10.068 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 ceph-mon[65453]: mon.smithi115@-1(???).paxosservice(auth 1..22) refresh upgraded, format 0 -> 3 2021-11-03T17:16:10.069 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:09 smithi115 systemd[1]: Started Ceph mon.smithi115 for 32ab977e-3cc8-11ec-8c28-001a4aab830c. 2021-11-03T17:16:11.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: mkfs 32ab977e-3cc8-11ec-8c28-001a4aab830c 2021-11-03T17:16:11.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: Upgrade: Updating mgr.smithi115.tcyodz 2021-11-03T17:16:11.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: Deploying daemon mgr.smithi115.tcyodz on smithi115 2021-11-03T17:16:11.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: Upgrade: Need to upgrade myself (mgr.smithi027.ufhhvb) 2021-11-03T17:16:11.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: Failing over to other MGR 2021-11-03T17:16:11.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: Deploying cephadm binary to smithi115 2021-11-03T17:16:11.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: Deploying cephadm binary to smithi027 2021-11-03T17:16:11.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: Starting cherrypy engine... 2021-11-03T17:16:11.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: [03/Nov/2021:17:14:24] ENGINE Bus STARTING 2021-11-03T17:16:11.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: [03/Nov/2021:17:14:24] ENGINE Serving on https://172.21.15.115:8499 2021-11-03T17:16:11.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: [03/Nov/2021:17:14:24] ENGINE Bus STARTED 2021-11-03T17:16:11.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: Cherrypy engine started. 2021-11-03T17:16:11.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: poll took 4546.712 ms: 1 events 2021-11-03T17:16:11.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: poll took 3708.276 ms: 1 events 2021-11-03T17:16:11.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: Upgrade: Need to upgrade myself (mgr.smithi115.tcyodz) 2021-11-03T17:16:11.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: poll took 2973.218 ms: 1 events 2021-11-03T17:16:11.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: Upgrade: Updating mgr.smithi027.ufhhvb 2021-11-03T17:16:11.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: Deploying daemon mgr.smithi027.ufhhvb on smithi027 2021-11-03T17:16:11.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: poll took 9499.574 ms: 1 events 2021-11-03T17:16:11.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: poll took 12800.515 ms: 1 events 2021-11-03T17:16:11.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: Upgrade: Need to upgrade myself (mgr.smithi115.tcyodz) 2021-11-03T17:16:11.490 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: Failing over to other MGR 2021-11-03T17:16:11.490 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: inventory: adjusted host smithi115 addr 'smithi115' -> '172.21.15.115' 2021-11-03T17:16:11.490 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: Executing result=None created at /lib/python3.6/site-packages/asyncssh/connection.py:841> took 0.107 seconds 2021-11-03T17:16:11.491 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: Starting cherrypy engine... 2021-11-03T17:16:11.491 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: [03/Nov/2021:17:15:02] ENGINE Bus STARTING 2021-11-03T17:16:11.491 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: [03/Nov/2021:17:15:02] ENGINE Serving on https://172.21.15.27:8499 2021-11-03T17:16:11.491 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: [03/Nov/2021:17:15:02] ENGINE Bus STARTED 2021-11-03T17:16:11.492 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: Cherrypy engine started. 2021-11-03T17:16:11.492 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: poll took 4074.563 ms: 1 events 2021-11-03T17:16:11.492 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: fsmap cephfs:2 {0=cephfs.smithi115.semywj=up:active,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:16:11.492 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: pgmap v9: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 13 MiB/s rd, 12 MiB/s wr, 37 op/s 2021-11-03T17:16:11.492 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: poll took 5527.041 ms: 1 events 2021-11-03T17:16:11.493 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: pgmap v10: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 18 MiB/s rd, 21 MiB/s wr, 466 op/s 2021-11-03T17:16:11.493 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: pgmap v11: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 16 MiB/s rd, 19 MiB/s wr, 417 op/s 2021-11-03T17:16:11.493 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: pgmap v12: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 16 MiB/s rd, 19 MiB/s wr, 417 op/s 2021-11-03T17:16:11.493 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: poll took 4646.113 ms: 1 events 2021-11-03T17:16:11.494 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: Upgrade: Updating mgr.smithi115.tcyodz 2021-11-03T17:16:11.494 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: Deploying daemon mgr.smithi115.tcyodz on smithi115 2021-11-03T17:16:11.494 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: fsmap cephfs:2 {0=cephfs.smithi115.semywj=up:active,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:16:11.494 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: pgmap v13: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 26 MiB/s rd, 36 MiB/s wr, 709 op/s 2021-11-03T17:16:11.495 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: pgmap v14: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 16 MiB/s rd, 27 MiB/s wr, 681 op/s 2021-11-03T17:16:11.495 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: pgmap v15: 65 pgs: 65 active+clean; 3.0 GiB data, 9.1 GiB used, 527 GiB / 536 GiB avail; 22 MiB/s rd, 42 MiB/s wr, 799 op/s 2021-11-03T17:16:11.495 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: pgmap v16: 65 pgs: 65 active+clean; 3.0 GiB data, 9.1 GiB used, 527 GiB / 536 GiB avail; 16 MiB/s rd, 33 MiB/s wr, 410 op/s 2021-11-03T17:16:11.495 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: poll took 7905.618 ms: 1 events 2021-11-03T17:16:11.496 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: pgmap v17: 65 pgs: 65 active+clean; 3.0 GiB data, 9.1 GiB used, 527 GiB / 536 GiB avail; 16 MiB/s rd, 33 MiB/s wr, 410 op/s 2021-11-03T17:16:11.496 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: pgmap v18: 65 pgs: 65 active+clean; 2.5 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 35 MiB/s rd, 63 MiB/s wr, 518 op/s 2021-11-03T17:16:11.496 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: pgmap v19: 65 pgs: 65 active+clean; 2.5 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 26 MiB/s rd, 45 MiB/s wr, 226 op/s 2021-11-03T17:16:11.496 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: poll took 6662.866 ms: 1 events 2021-11-03T17:16:11.496 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: Standby manager daemon smithi115.tcyodz restarted 2021-11-03T17:16:11.497 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: Standby manager daemon smithi115.tcyodz started 2021-11-03T17:16:11.497 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: Upgrade: Setting container_image for all mgr 2021-11-03T17:16:11.497 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: pgmap v20: 65 pgs: 65 active+clean; 2.1 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 40 MiB/s rd, 76 MiB/s wr, 313 op/s 2021-11-03T17:16:11.497 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: mgrmap e29: smithi027.ufhhvb(active, since 33s), standbys: smithi115.tcyodz 2021-11-03T17:16:11.498 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: pgmap v21: 65 pgs: 65 active+clean; 2.1 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 34 MiB/s rd, 61 MiB/s wr, 195 op/s 2021-11-03T17:16:11.498 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: pgmap v22: 65 pgs: 65 active+clean; 2.1 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 37 MiB/s rd, 67 MiB/s wr, 207 op/s 2021-11-03T17:16:11.498 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: pgmap v23: 65 pgs: 65 active+clean; 2.1 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 50 MiB/s rd, 92 MiB/s wr, 298 op/s 2021-11-03T17:16:11.499 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: poll took 7414.928 ms: 1 events 2021-11-03T17:16:11.499 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: Upgrade: Updating mon.smithi027 2021-11-03T17:16:11.499 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:11.499 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-11-03T17:16:11.500 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-11-03T17:16:11.500 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:16:11.500 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: Deploying daemon mon.smithi027 on smithi027 2021-11-03T17:16:11.500 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: pgmap v24: 65 pgs: 65 active+clean; 2.1 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 31 MiB/s rd, 62 MiB/s wr, 190 op/s 2021-11-03T17:16:11.500 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2021-11-03T17:16:11.501 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2021-11-03T17:16:11.501 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: pgmap v25: 65 pgs: 65 active+clean; 2.2 GiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 41 MiB/s rd, 86 MiB/s wr, 309 op/s 2021-11-03T17:16:11.501 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: poll took 4489.861 ms: 1 events 2021-11-03T17:16:11.501 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: pgmap v26: 65 pgs: 65 active+clean; 2.2 GiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 27 MiB/s rd, 56 MiB/s wr, 222 op/s 2021-11-03T17:16:11.502 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: mon.smithi027 calling monitor election 2021-11-03T17:16:11.502 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: mon.smithi027 is new leader, mons smithi027,smithi115 in quorum (ranks 0,1) 2021-11-03T17:16:11.502 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: monmap e2: 2 mons at {smithi027=[v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0],smithi115=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0]} 2021-11-03T17:16:11.502 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: fsmap cephfs:2 {0=cephfs.smithi115.semywj=up:active,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:16:11.503 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: osdmap e51: 6 total, 6 up, 6 in 2021-11-03T17:16:11.503 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: mgrmap e29: smithi027.ufhhvb(active, since 46s), standbys: smithi115.tcyodz 2021-11-03T17:16:11.503 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: mgrmap e30: smithi027.ufhhvb(active, since 46s), standbys: smithi115.tcyodz 2021-11-03T17:16:11.503 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: overall HEALTH_OK 2021-11-03T17:16:11.503 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.14622 ' entity='' 2021-11-03T17:16:11.504 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: Standby manager daemon smithi115.tcyodz restarted 2021-11-03T17:16:11.504 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: Standby manager daemon smithi115.tcyodz started 2021-11-03T17:16:11.504 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.? 172.21.15.115:0/2569420103' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi115.tcyodz/crt"}]: dispatch 2021-11-03T17:16:11.504 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.? 172.21.15.115:0/2569420103' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-11-03T17:16:11.505 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.? 172.21.15.115:0/2569420103' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi115.tcyodz/key"}]: dispatch 2021-11-03T17:16:11.505 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.? 172.21.15.115:0/2569420103' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-11-03T17:16:11.505 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: mgrmap e31: smithi027.ufhhvb(active, since 53s), standbys: smithi115.tcyodz 2021-11-03T17:16:11.505 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: Active manager daemon smithi027.ufhhvb restarted 2021-11-03T17:16:11.505 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: Activating manager daemon smithi027.ufhhvb 2021-11-03T17:16:11.506 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: osdmap e52: 6 total, 6 up, 6 in 2021-11-03T17:16:11.506 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: mgrmap e32: smithi027.ufhhvb(active, starting, since 0.9697s), standbys: smithi115.tcyodz 2021-11-03T17:16:11.506 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2021-11-03T17:16:11.506 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2021-11-03T17:16:11.507 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.jdwohn"}]: dispatch 2021-11-03T17:16:11.507 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.fksixo"}]: dispatch 2021-11-03T17:16:11.507 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.semywj"}]: dispatch 2021-11-03T17:16:11.507 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.zsfgnv"}]: dispatch 2021-11-03T17:16:11.508 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mgr metadata", "who": "smithi027.ufhhvb", "id": "smithi027.ufhhvb"}]: dispatch 2021-11-03T17:16:11.508 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mgr metadata", "who": "smithi115.tcyodz", "id": "smithi115.tcyodz"}]: dispatch 2021-11-03T17:16:11.508 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-11-03T17:16:11.508 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-11-03T17:16:11.508 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-11-03T17:16:11.509 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-11-03T17:16:11.509 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-11-03T17:16:11.509 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-11-03T17:16:11.509 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata"}]: dispatch 2021-11-03T17:16:11.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata"}]: dispatch 2021-11-03T17:16:11.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata"}]: dispatch 2021-11-03T17:16:11.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: Manager daemon smithi027.ufhhvb is now available 2021-11-03T17:16:11.510 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:16:11.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:16:11.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:16:11.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:16:11.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: mgrmap e33: smithi027.ufhhvb(active, since 1.97556s), standbys: smithi115.tcyodz 2021-11-03T17:16:11.511 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='client.24499 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:16:11.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: pgmap v3: 65 pgs: 65 active+clean; 823 MiB data, 2.9 GiB used, 533 GiB / 536 GiB avail 2021-11-03T17:16:11.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='client.24525 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:16:11.512 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/4261618275' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:16:11.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: pgmap v4: 65 pgs: 65 active+clean; 823 MiB data, 2.9 GiB used, 533 GiB / 536 GiB avail 2021-11-03T17:16:11.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: mgrmap e34: smithi027.ufhhvb(active, since 2s), standbys: smithi115.tcyodz 2021-11-03T17:16:11.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2027359171' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:16:11.513 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: Starting cherrypy engine... 2021-11-03T17:16:11.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: [03/Nov/2021:17:15:59] ENGINE Bus STARTING 2021-11-03T17:16:11.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: [03/Nov/2021:17:15:59] ENGINE Serving on https://172.21.15.27:8499 2021-11-03T17:16:11.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: [03/Nov/2021:17:15:59] ENGINE Bus STARTED 2021-11-03T17:16:11.514 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: Cherrypy engine started. 2021-11-03T17:16:11.515 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: pgmap v5: 65 pgs: 65 active+clean; 823 MiB data, 2.9 GiB used, 533 GiB / 536 GiB avail 2021-11-03T17:16:11.515 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: mgrmap e35: smithi027.ufhhvb(active, since 6s), standbys: smithi115.tcyodz 2021-11-03T17:16:11.515 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: pgmap v6: 65 pgs: 65 active+clean; 774 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 86 KiB/s wr, 13 op/s 2021-11-03T17:16:11.515 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: poll took 5537.853 ms: 1 events 2021-11-03T17:16:11.516 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:11.516 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:11.516 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"}]: dispatch 2021-11-03T17:16:11.516 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: pgmap v7: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 4.4 KiB/s rd, 92 KiB/s wr, 34 op/s 2021-11-03T17:16:11.517 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: poll took 2223.456 ms: 1 events 2021-11-03T17:16:11.517 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:11.517 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:11.517 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2021-11-03T17:16:11.517 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: pgmap v8: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 3.6 KiB/s rd, 73 KiB/s wr, 27 op/s 2021-11-03T17:16:11.518 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:11.518 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:16:11.518 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:16:11.518 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "quorum_status"}]: dispatch 2021-11-03T17:16:11.519 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2021-11-03T17:16:11.519 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: poll took 1524.765 ms: 1 events 2021-11-03T17:16:11.519 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: Upgrade: Updating mon.smithi115 2021-11-03T17:16:11.519 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:11.519 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-11-03T17:16:11.520 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-11-03T17:16:11.520 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:16:11.520 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: Deploying daemon mon.smithi115 on smithi115 2021-11-03T17:16:11.520 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: pgmap v9: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.9 KiB/s rd, 65 KiB/s wr, 23 op/s 2021-11-03T17:16:11.521 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: Health detail: HEALTH_WARN 1 failed cephadm daemon(s) 2021-11-03T17:16:11.521 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2021-11-03T17:16:11.521 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: daemon mds.cephfs.smithi115.semywj on smithi115 is in unknown state 2021-11-03T17:16:11.521 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2021-11-03T17:16:11.521 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: mon.smithi027 calling monitor election 2021-11-03T17:16:11.522 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: mon.smithi115 calling monitor election 2021-11-03T17:16:11.522 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: mon.smithi027 is new leader, mons smithi027,smithi115 in quorum (ranks 0,1) 2021-11-03T17:16:11.522 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2021-11-03T17:16:11.522 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: monmap e3: 2 mons at {smithi027=[v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0],smithi115=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0]} 2021-11-03T17:16:11.523 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: fsmap cephfs:2 {0=cephfs.smithi115.semywj=up:active,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:16:11.523 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: osdmap e52: 6 total, 6 up, 6 in 2021-11-03T17:16:11.523 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: mgrmap e35: smithi027.ufhhvb(active, since 15s), standbys: smithi115.tcyodz 2021-11-03T17:16:11.523 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: Health detail: HEALTH_WARN 1 failed cephadm daemon(s) 2021-11-03T17:16:11.523 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2021-11-03T17:16:11.524 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: daemon mds.cephfs.smithi115.semywj on smithi115 is in unknown state 2021-11-03T17:16:11.524 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:11.524 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:16:11.524 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:11.526 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: mkfs 32ab977e-3cc8-11ec-8c28-001a4aab830c 2021-11-03T17:16:11.526 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: Upgrade: Updating mgr.smithi115.tcyodz 2021-11-03T17:16:11.526 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: Deploying daemon mgr.smithi115.tcyodz on smithi115 2021-11-03T17:16:11.526 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: Upgrade: Need to upgrade myself (mgr.smithi027.ufhhvb) 2021-11-03T17:16:11.527 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: Failing over to other MGR 2021-11-03T17:16:11.527 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: Deploying cephadm binary to smithi115 2021-11-03T17:16:11.527 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: Deploying cephadm binary to smithi027 2021-11-03T17:16:11.527 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: Starting cherrypy engine... 2021-11-03T17:16:11.528 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: [03/Nov/2021:17:14:24] ENGINE Bus STARTING 2021-11-03T17:16:11.528 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: [03/Nov/2021:17:14:24] ENGINE Serving on https://172.21.15.115:8499 2021-11-03T17:16:11.528 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: [03/Nov/2021:17:14:24] ENGINE Bus STARTED 2021-11-03T17:16:11.528 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: Cherrypy engine started. 2021-11-03T17:16:11.528 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: poll took 4546.712 ms: 1 events 2021-11-03T17:16:11.529 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: poll took 3708.276 ms: 1 events 2021-11-03T17:16:11.529 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: Upgrade: Need to upgrade myself (mgr.smithi115.tcyodz) 2021-11-03T17:16:11.529 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: poll took 2973.218 ms: 1 events 2021-11-03T17:16:11.529 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: Upgrade: Updating mgr.smithi027.ufhhvb 2021-11-03T17:16:11.530 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: Deploying daemon mgr.smithi027.ufhhvb on smithi027 2021-11-03T17:16:11.530 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: poll took 9499.574 ms: 1 events 2021-11-03T17:16:11.530 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: poll took 12800.515 ms: 1 events 2021-11-03T17:16:11.530 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: Upgrade: Need to upgrade myself (mgr.smithi115.tcyodz) 2021-11-03T17:16:11.531 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: Failing over to other MGR 2021-11-03T17:16:11.531 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: inventory: adjusted host smithi115 addr 'smithi115' -> '172.21.15.115' 2021-11-03T17:16:11.531 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: Executing result=None created at /lib/python3.6/site-packages/asyncssh/connection.py:841> took 0.107 seconds 2021-11-03T17:16:11.531 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: Starting cherrypy engine... 2021-11-03T17:16:11.531 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: [03/Nov/2021:17:15:02] ENGINE Bus STARTING 2021-11-03T17:16:11.532 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: [03/Nov/2021:17:15:02] ENGINE Serving on https://172.21.15.27:8499 2021-11-03T17:16:11.532 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: [03/Nov/2021:17:15:02] ENGINE Bus STARTED 2021-11-03T17:16:11.532 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: Cherrypy engine started. 2021-11-03T17:16:11.532 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: poll took 4074.563 ms: 1 events 2021-11-03T17:16:11.532 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: fsmap cephfs:2 {0=cephfs.smithi115.semywj=up:active,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:16:11.533 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: pgmap v9: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 13 MiB/s rd, 12 MiB/s wr, 37 op/s 2021-11-03T17:16:11.533 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: poll took 5527.041 ms: 1 events 2021-11-03T17:16:11.533 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: pgmap v10: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 18 MiB/s rd, 21 MiB/s wr, 466 op/s 2021-11-03T17:16:11.533 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: pgmap v11: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 16 MiB/s rd, 19 MiB/s wr, 417 op/s 2021-11-03T17:16:11.534 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: pgmap v12: 65 pgs: 65 active+clean; 3.4 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 16 MiB/s rd, 19 MiB/s wr, 417 op/s 2021-11-03T17:16:11.534 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: poll took 4646.113 ms: 1 events 2021-11-03T17:16:11.534 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: Upgrade: Updating mgr.smithi115.tcyodz 2021-11-03T17:16:11.534 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: Deploying daemon mgr.smithi115.tcyodz on smithi115 2021-11-03T17:16:11.535 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: fsmap cephfs:2 {0=cephfs.smithi115.semywj=up:active,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:16:11.535 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: pgmap v13: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 26 MiB/s rd, 36 MiB/s wr, 709 op/s 2021-11-03T17:16:11.535 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: pgmap v14: 65 pgs: 65 active+clean; 3.5 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 16 MiB/s rd, 27 MiB/s wr, 681 op/s 2021-11-03T17:16:11.535 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: pgmap v15: 65 pgs: 65 active+clean; 3.0 GiB data, 9.1 GiB used, 527 GiB / 536 GiB avail; 22 MiB/s rd, 42 MiB/s wr, 799 op/s 2021-11-03T17:16:11.535 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: pgmap v16: 65 pgs: 65 active+clean; 3.0 GiB data, 9.1 GiB used, 527 GiB / 536 GiB avail; 16 MiB/s rd, 33 MiB/s wr, 410 op/s 2021-11-03T17:16:11.536 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: poll took 7905.618 ms: 1 events 2021-11-03T17:16:11.536 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: pgmap v17: 65 pgs: 65 active+clean; 3.0 GiB data, 9.1 GiB used, 527 GiB / 536 GiB avail; 16 MiB/s rd, 33 MiB/s wr, 410 op/s 2021-11-03T17:16:11.536 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: pgmap v18: 65 pgs: 65 active+clean; 2.5 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 35 MiB/s rd, 63 MiB/s wr, 518 op/s 2021-11-03T17:16:11.536 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: pgmap v19: 65 pgs: 65 active+clean; 2.5 GiB data, 7.6 GiB used, 529 GiB / 536 GiB avail; 26 MiB/s rd, 45 MiB/s wr, 226 op/s 2021-11-03T17:16:11.537 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: poll took 6662.866 ms: 1 events 2021-11-03T17:16:11.537 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: Standby manager daemon smithi115.tcyodz restarted 2021-11-03T17:16:11.537 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: Standby manager daemon smithi115.tcyodz started 2021-11-03T17:16:11.538 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: Upgrade: Setting container_image for all mgr 2021-11-03T17:16:11.538 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: pgmap v20: 65 pgs: 65 active+clean; 2.1 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 40 MiB/s rd, 76 MiB/s wr, 313 op/s 2021-11-03T17:16:11.538 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: mgrmap e29: smithi027.ufhhvb(active, since 33s), standbys: smithi115.tcyodz 2021-11-03T17:16:11.538 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: pgmap v21: 65 pgs: 65 active+clean; 2.1 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 34 MiB/s rd, 61 MiB/s wr, 195 op/s 2021-11-03T17:16:11.538 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: pgmap v22: 65 pgs: 65 active+clean; 2.1 GiB data, 6.7 GiB used, 530 GiB / 536 GiB avail; 37 MiB/s rd, 67 MiB/s wr, 207 op/s 2021-11-03T17:16:11.539 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: pgmap v23: 65 pgs: 65 active+clean; 2.1 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 50 MiB/s rd, 92 MiB/s wr, 298 op/s 2021-11-03T17:16:11.539 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: poll took 7414.928 ms: 1 events 2021-11-03T17:16:11.539 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: Upgrade: Updating mon.smithi027 2021-11-03T17:16:11.540 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:11.540 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-11-03T17:16:11.540 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-11-03T17:16:11.540 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:16:11.541 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: Deploying daemon mon.smithi027 on smithi027 2021-11-03T17:16:11.541 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: pgmap v24: 65 pgs: 65 active+clean; 2.1 GiB data, 6.4 GiB used, 530 GiB / 536 GiB avail; 31 MiB/s rd, 62 MiB/s wr, 190 op/s 2021-11-03T17:16:11.541 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2021-11-03T17:16:11.541 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.14622 172.21.15.27:0/1600409905' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2021-11-03T17:16:11.542 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: pgmap v25: 65 pgs: 65 active+clean; 2.2 GiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 41 MiB/s rd, 86 MiB/s wr, 309 op/s 2021-11-03T17:16:11.542 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: poll took 4489.861 ms: 1 events 2021-11-03T17:16:11.542 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: pgmap v26: 65 pgs: 65 active+clean; 2.2 GiB data, 6.9 GiB used, 530 GiB / 536 GiB avail; 27 MiB/s rd, 56 MiB/s wr, 222 op/s 2021-11-03T17:16:11.543 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: mon.smithi027 calling monitor election 2021-11-03T17:16:11.543 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: mon.smithi027 is new leader, mons smithi027,smithi115 in quorum (ranks 0,1) 2021-11-03T17:16:11.543 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: monmap e2: 2 mons at {smithi027=[v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0],smithi115=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0]} 2021-11-03T17:16:11.543 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: fsmap cephfs:2 {0=cephfs.smithi115.semywj=up:active,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:16:11.544 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: osdmap e51: 6 total, 6 up, 6 in 2021-11-03T17:16:11.544 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: mgrmap e29: smithi027.ufhhvb(active, since 46s), standbys: smithi115.tcyodz 2021-11-03T17:16:11.544 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: mgrmap e30: smithi027.ufhhvb(active, since 46s), standbys: smithi115.tcyodz 2021-11-03T17:16:11.544 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: overall HEALTH_OK 2021-11-03T17:16:11.545 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.14622 ' entity='' 2021-11-03T17:16:11.545 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: Standby manager daemon smithi115.tcyodz restarted 2021-11-03T17:16:11.545 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: Standby manager daemon smithi115.tcyodz started 2021-11-03T17:16:11.545 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.? 172.21.15.115:0/2569420103' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi115.tcyodz/crt"}]: dispatch 2021-11-03T17:16:11.545 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.? 172.21.15.115:0/2569420103' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2021-11-03T17:16:11.546 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.? 172.21.15.115:0/2569420103' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi115.tcyodz/key"}]: dispatch 2021-11-03T17:16:11.546 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.? 172.21.15.115:0/2569420103' entity='mgr.smithi115.tcyodz' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2021-11-03T17:16:11.546 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: mgrmap e31: smithi027.ufhhvb(active, since 53s), standbys: smithi115.tcyodz 2021-11-03T17:16:11.546 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: Active manager daemon smithi027.ufhhvb restarted 2021-11-03T17:16:11.547 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: Activating manager daemon smithi027.ufhhvb 2021-11-03T17:16:11.547 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: osdmap e52: 6 total, 6 up, 6 in 2021-11-03T17:16:11.547 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: mgrmap e32: smithi027.ufhhvb(active, starting, since 0.9697s), standbys: smithi115.tcyodz 2021-11-03T17:16:11.547 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2021-11-03T17:16:11.547 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2021-11-03T17:16:11.548 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.jdwohn"}]: dispatch 2021-11-03T17:16:11.548 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi027.fksixo"}]: dispatch 2021-11-03T17:16:11.548 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.semywj"}]: dispatch 2021-11-03T17:16:11.548 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.zsfgnv"}]: dispatch 2021-11-03T17:16:11.549 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mgr metadata", "who": "smithi027.ufhhvb", "id": "smithi027.ufhhvb"}]: dispatch 2021-11-03T17:16:11.549 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mgr metadata", "who": "smithi115.tcyodz", "id": "smithi115.tcyodz"}]: dispatch 2021-11-03T17:16:11.549 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-11-03T17:16:11.549 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-11-03T17:16:11.549 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-11-03T17:16:11.550 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-11-03T17:16:11.550 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-11-03T17:16:11.550 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-11-03T17:16:11.550 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata"}]: dispatch 2021-11-03T17:16:11.551 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata"}]: dispatch 2021-11-03T17:16:11.551 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata"}]: dispatch 2021-11-03T17:16:11.551 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: Manager daemon smithi027.ufhhvb is now available 2021-11-03T17:16:11.551 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:16:11.552 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:16:11.552 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:16:11.552 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:16:11.552 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: mgrmap e33: smithi027.ufhhvb(active, since 1.97556s), standbys: smithi115.tcyodz 2021-11-03T17:16:11.552 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='client.24499 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:16:11.553 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: pgmap v3: 65 pgs: 65 active+clean; 823 MiB data, 2.9 GiB used, 533 GiB / 536 GiB avail 2021-11-03T17:16:11.553 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='client.24525 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:16:11.553 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/4261618275' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:16:11.553 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: pgmap v4: 65 pgs: 65 active+clean; 823 MiB data, 2.9 GiB used, 533 GiB / 536 GiB avail 2021-11-03T17:16:11.554 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: mgrmap e34: smithi027.ufhhvb(active, since 2s), standbys: smithi115.tcyodz 2021-11-03T17:16:11.554 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2027359171' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:16:11.554 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: Starting cherrypy engine... 2021-11-03T17:16:11.554 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: [03/Nov/2021:17:15:59] ENGINE Bus STARTING 2021-11-03T17:16:11.554 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: [03/Nov/2021:17:15:59] ENGINE Serving on https://172.21.15.27:8499 2021-11-03T17:16:11.555 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: [03/Nov/2021:17:15:59] ENGINE Bus STARTED 2021-11-03T17:16:11.555 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: Cherrypy engine started. 2021-11-03T17:16:11.555 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: pgmap v5: 65 pgs: 65 active+clean; 823 MiB data, 2.9 GiB used, 533 GiB / 536 GiB avail 2021-11-03T17:16:11.555 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: mgrmap e35: smithi027.ufhhvb(active, since 6s), standbys: smithi115.tcyodz 2021-11-03T17:16:11.556 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: pgmap v6: 65 pgs: 65 active+clean; 774 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 86 KiB/s wr, 13 op/s 2021-11-03T17:16:11.556 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: poll took 5537.853 ms: 1 events 2021-11-03T17:16:11.556 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:11.556 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:11.556 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"}]: dispatch 2021-11-03T17:16:11.557 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: pgmap v7: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 4.4 KiB/s rd, 92 KiB/s wr, 34 op/s 2021-11-03T17:16:11.557 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: poll took 2223.456 ms: 1 events 2021-11-03T17:16:11.557 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:11.557 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:11.558 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2021-11-03T17:16:11.558 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: pgmap v8: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 3.6 KiB/s rd, 73 KiB/s wr, 27 op/s 2021-11-03T17:16:11.558 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:11.558 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:16:11.559 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:16:11.559 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "quorum_status"}]: dispatch 2021-11-03T17:16:11.559 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: Health check failed: 1 failed cephadm daemon(s) (CEPHADM_FAILED_DAEMON) 2021-11-03T17:16:11.559 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: poll took 1524.765 ms: 1 events 2021-11-03T17:16:11.560 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: Upgrade: Updating mon.smithi115 2021-11-03T17:16:11.560 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:11.560 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-11-03T17:16:11.560 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-11-03T17:16:11.560 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:16:11.561 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: Deploying daemon mon.smithi115 on smithi115 2021-11-03T17:16:11.561 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: pgmap v9: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.9 KiB/s rd, 65 KiB/s wr, 23 op/s 2021-11-03T17:16:11.561 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: Health detail: HEALTH_WARN 1 failed cephadm daemon(s) 2021-11-03T17:16:11.561 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2021-11-03T17:16:11.562 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: daemon mds.cephfs.smithi115.semywj on smithi115 is in unknown state 2021-11-03T17:16:11.562 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi027"}]: dispatch 2021-11-03T17:16:11.562 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: mon.smithi027 calling monitor election 2021-11-03T17:16:11.562 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: mon.smithi115 calling monitor election 2021-11-03T17:16:11.562 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: mon.smithi027 is new leader, mons smithi027,smithi115 in quorum (ranks 0,1) 2021-11-03T17:16:11.563 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mon metadata", "id": "smithi115"}]: dispatch 2021-11-03T17:16:11.563 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: monmap e3: 2 mons at {smithi027=[v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0],smithi115=[v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0]} 2021-11-03T17:16:11.563 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: fsmap cephfs:2 {0=cephfs.smithi115.semywj=up:active,1=cephfs.smithi027.jdwohn=up:active} 2 up:standby 2021-11-03T17:16:11.563 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: osdmap e52: 6 total, 6 up, 6 in 2021-11-03T17:16:11.564 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: mgrmap e35: smithi027.ufhhvb(active, since 15s), standbys: smithi115.tcyodz 2021-11-03T17:16:11.564 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: Health detail: HEALTH_WARN 1 failed cephadm daemon(s) 2021-11-03T17:16:11.564 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: [WRN] CEPHADM_FAILED_DAEMON: 1 failed cephadm daemon(s) 2021-11-03T17:16:11.564 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: daemon mds.cephfs.smithi115.semywj on smithi115 is in unknown state 2021-11-03T17:16:11.564 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:11.565 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:16:11.565 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:12.894 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:12 smithi115 ceph-mon[65453]: pgmap v11: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.6 KiB/s rd, 60 KiB/s wr, 21 op/s 2021-11-03T17:16:12.937 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:12 smithi027 ceph-mon[97244]: pgmap v11: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.6 KiB/s rd, 60 KiB/s wr, 21 op/s 2021-11-03T17:16:13.955 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:13 smithi115 ceph-mon[65453]: daemon mds.cephfs.smithi115.semywj restarted 2021-11-03T17:16:13.955 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:13 smithi115 ceph-mon[65453]: osdmap e53: 6 total, 6 up, 6 in 2021-11-03T17:16:14.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:13 smithi027 ceph-mon[97244]: daemon mds.cephfs.smithi115.semywj restarted 2021-11-03T17:16:14.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:13 smithi027 ceph-mon[97244]: osdmap e53: 6 total, 6 up, 6 in 2021-11-03T17:16:14.845 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:14 smithi027 ceph-mon[97244]: pgmap v13: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 6.0 KiB/s wr, 1 op/s 2021-11-03T17:16:14.846 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:14 smithi027 ceph-mon[97244]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2021-11-03T17:16:14.846 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:14 smithi027 ceph-mon[97244]: Health check failed: 1 filesystem has a failed mds daemon (FS_WITH_FAILED_MDS) 2021-11-03T17:16:14.846 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:14 smithi027 ceph-mon[97244]: mds.? [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] up:boot 2021-11-03T17:16:14.847 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:14 smithi027 ceph-mon[97244]: fsmap cephfs:1/2 {1=cephfs.smithi027.jdwohn=up:active} 3 up:standby, 1 failed 2021-11-03T17:16:14.847 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.semywj"}]: dispatch 2021-11-03T17:16:14.847 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:14 smithi027 ceph-mon[97244]: poll took 3329.425 ms: 1 events 2021-11-03T17:16:14.847 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:14.848 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:14 smithi027 ceph-mon[97244]: pgmap v14: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 6.9 KiB/s wr, 1 op/s 2021-11-03T17:16:14.848 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:14.848 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:14 smithi027 ceph-mon[97244]: Reconfiguring mon.smithi027 (monmap changed)... 2021-11-03T17:16:14.848 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-11-03T17:16:14.849 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-11-03T17:16:14.849 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:16:14.849 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:14 smithi027 ceph-mon[97244]: Reconfiguring daemon mon.smithi027 on smithi027 2021-11-03T17:16:14.849 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:14 smithi027 ceph-mon[97244]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2021-11-03T17:16:14.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:14 smithi115 ceph-mon[65453]: pgmap v13: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 6.0 KiB/s wr, 1 op/s 2021-11-03T17:16:14.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:14 smithi115 ceph-mon[65453]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2021-11-03T17:16:14.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:14 smithi115 ceph-mon[65453]: Health check failed: 1 filesystem has a failed mds daemon (FS_WITH_FAILED_MDS) 2021-11-03T17:16:14.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:14 smithi115 ceph-mon[65453]: mds.? [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] up:boot 2021-11-03T17:16:14.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:14 smithi115 ceph-mon[65453]: fsmap cephfs:1/2 {1=cephfs.smithi027.jdwohn=up:active} 3 up:standby, 1 failed 2021-11-03T17:16:14.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi115.semywj"}]: dispatch 2021-11-03T17:16:14.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:14 smithi115 ceph-mon[65453]: poll took 3329.425 ms: 1 events 2021-11-03T17:16:14.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:14.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:14 smithi115 ceph-mon[65453]: pgmap v14: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 6.9 KiB/s wr, 1 op/s 2021-11-03T17:16:14.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:14.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:14 smithi115 ceph-mon[65453]: Reconfiguring mon.smithi027 (monmap changed)... 2021-11-03T17:16:14.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-11-03T17:16:14.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-11-03T17:16:14.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:16:14.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:14 smithi115 ceph-mon[65453]: Reconfiguring daemon mon.smithi027 on smithi027 2021-11-03T17:16:14.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:14 smithi115 ceph-mon[65453]: Health check cleared: CEPHADM_FAILED_DAEMON (was: 1 failed cephadm daemon(s)) 2021-11-03T17:16:17.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:16 smithi115 ceph-mon[65453]: pgmap v15: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2021-11-03T17:16:17.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:16 smithi027 ceph-mon[97244]: pgmap v15: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2021-11-03T17:16:19.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:18 smithi027 ceph-mon[97244]: pgmap v16: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2021-11-03T17:16:19.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:19.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi027.ufhhvb", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2021-11-03T17:16:19.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mgr services"}]: dispatch 2021-11-03T17:16:19.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:16:19.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:18 smithi115 ceph-mon[65453]: pgmap v16: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2021-11-03T17:16:19.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:19.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi027.ufhhvb", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2021-11-03T17:16:19.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mgr services"}]: dispatch 2021-11-03T17:16:19.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:16:20.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:19 smithi027 ceph-mon[97244]: poll took 4412.304 ms: 1 events 2021-11-03T17:16:20.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:19 smithi027 ceph-mon[97244]: Reconfiguring mgr.smithi027.ufhhvb (monmap changed)... 2021-11-03T17:16:20.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:19 smithi027 ceph-mon[97244]: Reconfiguring daemon mgr.smithi027.ufhhvb on smithi027 2021-11-03T17:16:20.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:20.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi027", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2021-11-03T17:16:20.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:16:20.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:19 smithi115 ceph-mon[65453]: poll took 4412.304 ms: 1 events 2021-11-03T17:16:20.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:19 smithi115 ceph-mon[65453]: Reconfiguring mgr.smithi027.ufhhvb (monmap changed)... 2021-11-03T17:16:20.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:19 smithi115 ceph-mon[65453]: Reconfiguring daemon mgr.smithi027.ufhhvb on smithi027 2021-11-03T17:16:20.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:20.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi027", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2021-11-03T17:16:20.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:16:21.101 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:20 smithi027 ceph-mon[97244]: pgmap v17: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2021-11-03T17:16:21.101 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:20 smithi027 ceph-mon[97244]: poll took 1157.228 ms: 1 events 2021-11-03T17:16:21.102 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:20 smithi027 ceph-mon[97244]: Reconfiguring crash.smithi027 (monmap changed)... 2021-11-03T17:16:21.102 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:20 smithi027 ceph-mon[97244]: Reconfiguring daemon crash.smithi027 on smithi027 2021-11-03T17:16:21.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:20 smithi115 ceph-mon[65453]: pgmap v17: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2021-11-03T17:16:21.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:20 smithi115 ceph-mon[65453]: poll took 1157.228 ms: 1 events 2021-11-03T17:16:21.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:20 smithi115 ceph-mon[65453]: Reconfiguring crash.smithi027 (monmap changed)... 2021-11-03T17:16:21.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:20 smithi115 ceph-mon[65453]: Reconfiguring daemon crash.smithi027 on smithi027 2021-11-03T17:16:22.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:22 smithi027 ceph-mon[97244]: poll took 1115.657 ms: 1 events 2021-11-03T17:16:22.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:22.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:22 smithi027 ceph-mon[97244]: Reconfiguring osd.0 (monmap changed)... 2021-11-03T17:16:22.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:22 smithi027 ceph-mon[97244]: Reconfiguring daemon osd.0 on smithi027 2021-11-03T17:16:22.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:22 smithi027 ceph-mon[97244]: pgmap v18: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:16:22.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:22 smithi115 ceph-mon[65453]: poll took 1115.657 ms: 1 events 2021-11-03T17:16:22.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:22.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:22 smithi115 ceph-mon[65453]: Reconfiguring osd.0 (monmap changed)... 2021-11-03T17:16:22.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:22 smithi115 ceph-mon[65453]: Reconfiguring daemon osd.0 on smithi027 2021-11-03T17:16:22.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:22 smithi115 ceph-mon[65453]: pgmap v18: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:16:23.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:23 smithi027 ceph-mon[97244]: poll took 1209.280 ms: 1 events 2021-11-03T17:16:23.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:23.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:23 smithi027 ceph-mon[97244]: Reconfiguring osd.1 (monmap changed)... 2021-11-03T17:16:23.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:23 smithi027 ceph-mon[97244]: Reconfiguring daemon osd.1 on smithi027 2021-11-03T17:16:23.955 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:23 smithi115 ceph-mon[65453]: poll took 1209.280 ms: 1 events 2021-11-03T17:16:23.955 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:23.956 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:23 smithi115 ceph-mon[65453]: Reconfiguring osd.1 (monmap changed)... 2021-11-03T17:16:23.956 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:23 smithi115 ceph-mon[65453]: Reconfiguring daemon osd.1 on smithi027 2021-11-03T17:16:24.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:24 smithi027 ceph-mon[97244]: pgmap v19: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:16:24.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:24 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:24.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:24 smithi115 ceph-mon[65453]: pgmap v19: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:16:24.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:24 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:25.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:25 smithi027 ceph-mon[97244]: poll took 1169.140 ms: 1 events 2021-11-03T17:16:25.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:25 smithi027 ceph-mon[97244]: Reconfiguring osd.2 (monmap changed)... 2021-11-03T17:16:25.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:25 smithi027 ceph-mon[97244]: Reconfiguring daemon osd.2 on smithi027 2021-11-03T17:16:25.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:25.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi027.jdwohn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2021-11-03T17:16:25.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:16:25.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:25.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:25 smithi115 ceph-mon[65453]: poll took 1169.140 ms: 1 events 2021-11-03T17:16:25.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:25 smithi115 ceph-mon[65453]: Reconfiguring osd.2 (monmap changed)... 2021-11-03T17:16:25.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:25 smithi115 ceph-mon[65453]: Reconfiguring daemon osd.2 on smithi027 2021-11-03T17:16:25.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:25.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi027.jdwohn", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2021-11-03T17:16:25.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:16:25.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:26.858 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:26 smithi027 ceph-mon[97244]: poll took 1140.120 ms: 1 events 2021-11-03T17:16:26.859 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:26 smithi027 ceph-mon[97244]: Reconfiguring mds.cephfs.smithi027.jdwohn (monmap changed)... 2021-11-03T17:16:26.859 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:26 smithi027 ceph-mon[97244]: Reconfiguring daemon mds.cephfs.smithi027.jdwohn on smithi027 2021-11-03T17:16:26.859 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:26 smithi027 ceph-mon[97244]: pgmap v20: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:16:26.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:26 smithi115 ceph-mon[65453]: poll took 1140.120 ms: 1 events 2021-11-03T17:16:26.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:26 smithi115 ceph-mon[65453]: Reconfiguring mds.cephfs.smithi027.jdwohn (monmap changed)... 2021-11-03T17:16:26.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:26 smithi115 ceph-mon[65453]: Reconfiguring daemon mds.cephfs.smithi027.jdwohn on smithi027 2021-11-03T17:16:26.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:26 smithi115 ceph-mon[65453]: pgmap v20: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:16:27.762 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.760+0000 7fbbda990700 1 -- 172.21.15.27:0/3003024303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbd4060330 msgr2=0x7fbbd40f93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:16:27.763 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.760+0000 7fbbda990700 1 --2- 172.21.15.27:0/3003024303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbd4060330 0x7fbbd40f93c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fbbc8004660 tx=0x7fbbc8009b30).stop 2021-11-03T17:16:27.763 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.760+0000 7fbbda990700 1 -- 172.21.15.27:0/3003024303 shutdown_connections 2021-11-03T17:16:27.764 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.760+0000 7fbbda990700 1 --2- 172.21.15.27:0/3003024303 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbd40f98b0 0x7fbbd40fbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:27.764 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.760+0000 7fbbda990700 1 --2- 172.21.15.27:0/3003024303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbd4060330 0x7fbbd40f93c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:27.765 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.760+0000 7fbbda990700 1 -- 172.21.15.27:0/3003024303 >> 172.21.15.27:0/3003024303 conn(0x7fbbd40f0de0 msgr2=0x7fbbd40f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:16:27.765 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.760+0000 7fbbda990700 1 -- 172.21.15.27:0/3003024303 shutdown_connections 2021-11-03T17:16:27.766 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.760+0000 7fbbda990700 1 -- 172.21.15.27:0/3003024303 wait complete. 2021-11-03T17:16:27.766 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.760+0000 7fbbda990700 1 Processor -- start 2021-11-03T17:16:27.767 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.761+0000 7fbbda990700 1 -- start start 2021-11-03T17:16:27.767 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.761+0000 7fbbda990700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbd4060330 0x7fbbd40f6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:16:27.767 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.761+0000 7fbbda990700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbd40f98b0 0x7fbbd40f52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:16:27.768 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.761+0000 7fbbda990700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd40f5820 con 0x7fbbd40f98b0 2021-11-03T17:16:27.768 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.761+0000 7fbbda990700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd40f5960 con 0x7fbbd4060330 2021-11-03T17:16:27.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.761+0000 7fbbd37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbd40f98b0 0x7fbbd40f52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:16:27.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.761+0000 7fbbd37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbd40f98b0 0x7fbbd40f52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56064/0 (socket says 172.21.15.27:56064) 2021-11-03T17:16:27.770 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.761+0000 7fbbd37fe700 1 -- 172.21.15.27:0/3685450327 learned_addr learned my addr 172.21.15.27:0/3685450327 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:16:27.770 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.761+0000 7fbbd3fff700 1 --2- 172.21.15.27:0/3685450327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbd4060330 0x7fbbd40f6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:16:27.771 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.761+0000 7fbbd37fe700 1 -- 172.21.15.27:0/3685450327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbd4060330 msgr2=0x7fbbd40f6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:16:27.772 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.761+0000 7fbbd37fe700 1 --2- 172.21.15.27:0/3685450327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbd4060330 0x7fbbd40f6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:27.773 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.761+0000 7fbbd37fe700 1 -- 172.21.15.27:0/3685450327 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc8005040 con 0x7fbbd40f98b0 2021-11-03T17:16:27.773 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.761+0000 7fbbd37fe700 1 --2- 172.21.15.27:0/3685450327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbd40f98b0 0x7fbbd40f52c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fbbc400c790 tx=0x7fbbc4007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:16:27.774 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.761+0000 7fbbd0ff9700 1 -- 172.21.15.27:0/3685450327 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbc400e680 con 0x7fbbd40f98b0 2021-11-03T17:16:27.774 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.761+0000 7fbbd0ff9700 1 -- 172.21.15.27:0/3685450327 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fbbc400ec90 con 0x7fbbd40f98b0 2021-11-03T17:16:27.774 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.761+0000 7fbbda990700 1 -- 172.21.15.27:0/3685450327 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbd40f5c40 con 0x7fbbd40f98b0 2021-11-03T17:16:27.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.762+0000 7fbbd0ff9700 1 -- 172.21.15.27:0/3685450327 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbc4013d60 con 0x7fbbd40f98b0 2021-11-03T17:16:27.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.762+0000 7fbbda990700 1 -- 172.21.15.27:0/3685450327 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbd410f000 con 0x7fbbd40f98b0 2021-11-03T17:16:27.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.762+0000 7fbbda990700 1 -- 172.21.15.27:0/3685450327 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbd4045ad0 con 0x7fbbd40f98b0 2021-11-03T17:16:27.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.763+0000 7fbbd0ff9700 1 -- 172.21.15.27:0/3685450327 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 35) v1 ==== 81997+0+0 (secure 0 0 0) 0x7fbbc401a060 con 0x7fbbd40f98b0 2021-11-03T17:16:27.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.763+0000 7fbbd0ff9700 1 --2- 172.21.15.27:0/3685450327 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbbbc064120 0x7fbbbc0665e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:16:27.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.763+0000 7fbbd0ff9700 1 -- 172.21.15.27:0/3685450327 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fbbc4080b40 con 0x7fbbd40f98b0 2021-11-03T17:16:27.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.763+0000 7fbbd3fff700 1 --2- 172.21.15.27:0/3685450327 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbbbc064120 0x7fbbbc0665e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:16:27.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.764+0000 7fbbd3fff700 1 --2- 172.21.15.27:0/3685450327 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbbbc064120 0x7fbbbc0665e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fbbc8009da0 tx=0x7fbbc8009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:16:27.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.768+0000 7fbbd0ff9700 1 -- 172.21.15.27:0/3685450327 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fbbc4052dc0 con 0x7fbbd40f98b0 2021-11-03T17:16:27.955 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.959+0000 7fbbda990700 1 -- 172.21.15.27:0/3685450327 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbbd40fefc0 con 0x7fbbbc064120 2021-11-03T17:16:27.956 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.961+0000 7fbbd0ff9700 1 -- 172.21.15.27:0/3685450327 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+303 (secure 0 0 0) 0x7fbbd40fefc0 con 0x7fbbbc064120 2021-11-03T17:16:27.958 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.962+0000 7fbbda990700 1 -- 172.21.15.27:0/3685450327 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbbbc064120 msgr2=0x7fbbbc0665e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:16:27.958 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.962+0000 7fbbda990700 1 --2- 172.21.15.27:0/3685450327 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbbbc064120 0x7fbbbc0665e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fbbc8009da0 tx=0x7fbbc8009b30).stop 2021-11-03T17:16:27.959 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.962+0000 7fbbda990700 1 -- 172.21.15.27:0/3685450327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbd40f98b0 msgr2=0x7fbbd40f52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:16:27.959 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.962+0000 7fbbda990700 1 --2- 172.21.15.27:0/3685450327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbd40f98b0 0x7fbbd40f52c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fbbc400c790 tx=0x7fbbc4007d70).stop 2021-11-03T17:16:27.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.962+0000 7fbbda990700 1 -- 172.21.15.27:0/3685450327 shutdown_connections 2021-11-03T17:16:27.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.962+0000 7fbbda990700 1 --2- 172.21.15.27:0/3685450327 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbbbc064120 0x7fbbbc0665e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:27.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.962+0000 7fbbda990700 1 --2- 172.21.15.27:0/3685450327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbd4060330 0x7fbbd40f6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:27.961 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.962+0000 7fbbda990700 1 --2- 172.21.15.27:0/3685450327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbd40f98b0 0x7fbbd40f52c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:27.961 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.962+0000 7fbbda990700 1 -- 172.21.15.27:0/3685450327 >> 172.21.15.27:0/3685450327 conn(0x7fbbd40f0de0 msgr2=0x7fbbd40f31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:16:27.961 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.963+0000 7fbbda990700 1 -- 172.21.15.27:0/3685450327 shutdown_connections 2021-11-03T17:16:27.961 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:27.963+0000 7fbbda990700 1 -- 172.21.15.27:0/3685450327 wait complete. 2021-11-03T17:16:27.968 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:16:28.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:27 smithi027 ceph-mon[97244]: poll took 1182.734 ms: 1 events 2021-11-03T17:16:28.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:28.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:27 smithi027 ceph-mon[97244]: Reconfiguring mds.cephfs.smithi027.fksixo (monmap changed)... 2021-11-03T17:16:28.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi027.fksixo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2021-11-03T17:16:28.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:16:28.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:27 smithi027 ceph-mon[97244]: Reconfiguring daemon mds.cephfs.smithi027.fksixo on smithi027 2021-11-03T17:16:28.060 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.063+0000 7f72f0683700 1 -- 172.21.15.27:0/2641226195 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72e80f6ff0 msgr2=0x7f72e80f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:16:28.061 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.063+0000 7f72f0683700 1 --2- 172.21.15.27:0/2641226195 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72e80f6ff0 0x7f72e80f7a20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f72dc004660 tx=0x7f72dc009b30).stop 2021-11-03T17:16:28.061 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.063+0000 7f72f0683700 1 -- 172.21.15.27:0/2641226195 shutdown_connections 2021-11-03T17:16:28.061 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.063+0000 7f72f0683700 1 --2- 172.21.15.27:0/2641226195 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72e80f6ff0 0x7f72e80f7a20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:28.062 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.063+0000 7f72f0683700 1 --2- 172.21.15.27:0/2641226195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72e80f66e0 0x7f72e80f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:28.062 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.063+0000 7f72f0683700 1 -- 172.21.15.27:0/2641226195 >> 172.21.15.27:0/2641226195 conn(0x7f72e80f1f00 msgr2=0x7f72e80f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:16:28.062 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.063+0000 7f72f0683700 1 -- 172.21.15.27:0/2641226195 shutdown_connections 2021-11-03T17:16:28.062 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.063+0000 7f72f0683700 1 -- 172.21.15.27:0/2641226195 wait complete. 2021-11-03T17:16:28.063 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.064+0000 7f72f0683700 1 Processor -- start 2021-11-03T17:16:28.063 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.064+0000 7f72f0683700 1 -- start start 2021-11-03T17:16:28.063 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.064+0000 7f72f0683700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72e80f66e0 0x7f72e806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:16:28.063 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.064+0000 7f72f0683700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72e80f6ff0 0x7f72e8069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:16:28.063 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.064+0000 7f72f0683700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e806a380 con 0x7f72e80f66e0 2021-11-03T17:16:28.064 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.064+0000 7f72f0683700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72e806a4c0 con 0x7f72e80f6ff0 2021-11-03T17:16:28.064 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.064+0000 7f72ee41f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72e80f66e0 0x7f72e806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:16:28.065 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.064+0000 7f72ee41f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72e80f66e0 0x7f72e806b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56072/0 (socket says 172.21.15.27:56072) 2021-11-03T17:16:28.065 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.064+0000 7f72ee41f700 1 -- 172.21.15.27:0/3187858201 learned_addr learned my addr 172.21.15.27:0/3187858201 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:16:28.065 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.064+0000 7f72edc1e700 1 --2- 172.21.15.27:0/3187858201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72e80f6ff0 0x7f72e8069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:16:28.065 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.064+0000 7f72ee41f700 1 -- 172.21.15.27:0/3187858201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72e80f6ff0 msgr2=0x7f72e8069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:16:28.066 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.064+0000 7f72ee41f700 1 --2- 172.21.15.27:0/3187858201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72e80f6ff0 0x7f72e8069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:28.066 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.064+0000 7f72ee41f700 1 -- 172.21.15.27:0/3187858201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72dc005040 con 0x7f72e80f66e0 2021-11-03T17:16:28.066 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.064+0000 7f72edc1e700 1 --2- 172.21.15.27:0/3187858201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72e80f6ff0 0x7f72e8069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:16:28.067 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.064+0000 7f72ee41f700 1 --2- 172.21.15.27:0/3187858201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72e80f66e0 0x7f72e806b9a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f72e0007fd0 tx=0x7f72e0004ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:16:28.067 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.065+0000 7f72e6ffd700 1 -- 172.21.15.27:0/3187858201 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72e000b5a0 con 0x7f72e80f66e0 2021-11-03T17:16:28.067 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.065+0000 7f72e6ffd700 1 -- 172.21.15.27:0/3187858201 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f72e000f040 con 0x7f72e80f66e0 2021-11-03T17:16:28.067 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.065+0000 7f72f0683700 1 -- 172.21.15.27:0/3187858201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72e806a7a0 con 0x7f72e80f66e0 2021-11-03T17:16:28.067 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.065+0000 7f72e6ffd700 1 -- 172.21.15.27:0/3187858201 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72e0013b60 con 0x7f72e80f66e0 2021-11-03T17:16:28.068 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.065+0000 7f72f0683700 1 -- 172.21.15.27:0/3187858201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72e810efa0 con 0x7f72e80f66e0 2021-11-03T17:16:28.069 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.065+0000 7f72f0683700 1 -- 172.21.15.27:0/3187858201 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72e8045ad0 con 0x7f72e80f66e0 2021-11-03T17:16:28.069 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.066+0000 7f72e6ffd700 1 -- 172.21.15.27:0/3187858201 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 35) v1 ==== 81997+0+0 (secure 0 0 0) 0x7f72e001a030 con 0x7f72e80f66e0 2021-11-03T17:16:28.069 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.066+0000 7f72e6ffd700 1 --2- 172.21.15.27:0/3187858201 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f72d0064120 0x7f72d00665e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:16:28.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.066+0000 7f72e6ffd700 1 -- 172.21.15.27:0/3187858201 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f72e0080750 con 0x7f72e80f66e0 2021-11-03T17:16:28.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.066+0000 7f72edc1e700 1 --2- 172.21.15.27:0/3187858201 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f72d0064120 0x7f72d00665e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:16:28.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.067+0000 7f72edc1e700 1 --2- 172.21.15.27:0/3187858201 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f72d0064120 0x7f72d00665e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f72dc004630 tx=0x7f72dc009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:16:28.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.070+0000 7f72e6ffd700 1 -- 172.21.15.27:0/3187858201 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f72e0052a80 con 0x7f72e80f66e0 2021-11-03T17:16:28.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:27 smithi115 ceph-mon[65453]: poll took 1182.734 ms: 1 events 2021-11-03T17:16:28.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:28.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:27 smithi115 ceph-mon[65453]: Reconfiguring mds.cephfs.smithi027.fksixo (monmap changed)... 2021-11-03T17:16:28.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi027.fksixo", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2021-11-03T17:16:28.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:16:28.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:27 smithi115 ceph-mon[65453]: Reconfiguring daemon mds.cephfs.smithi027.fksixo on smithi027 2021-11-03T17:16:28.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.249+0000 7f72f0683700 1 -- 172.21.15.27:0/3187858201 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f72e80fefe0 con 0x7f72d0064120 2021-11-03T17:16:28.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.263+0000 7f72e6ffd700 1 -- 172.21.15.27:0/3187858201 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f72e80fefe0 con 0x7f72d0064120 2021-11-03T17:16:28.259 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:16:28.260 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (8m) 26s ago 9m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:16:28.260 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (9m) 26s ago 9m 7013k - 16.2.4 8d91d370c2b8 d494472a47c0 2021-11-03T17:16:28.260 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (8m) 14s ago 8m 7134k - 16.2.4 8d91d370c2b8 f84fd4042530 2021-11-03T17:16:28.260 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (7m) 26s ago 8m 38.5M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:16:28.261 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (91s) 26s ago 4m 14.5M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:16:28.262 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (4m) 26s ago 4m 3820M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:16:28.262 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (15s) 14s ago 4m - - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:16:28.263 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (112s) 14s ago 4m 15.0M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:16:28.263 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (104s) 26s ago 10m 442M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:16:28.263 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (64s) 14s ago 8m 410M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:16:28.263 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (44s) 26s ago 10m 35.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:16:28.264 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (18s) 14s ago 8m 24.2M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:16:28.264 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (8m) 26s ago 8m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:16:28.264 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (8m) 14s ago 8m 18.2M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:16:28.264 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (7m) 26s ago 7m 748M 4096M 16.2.4 8d91d370c2b8 17f79a815a0f 2021-11-03T17:16:28.265 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (7m) 26s ago 7m 851M 4096M 16.2.4 8d91d370c2b8 422be0cac0db 2021-11-03T17:16:28.265 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (6m) 26s ago 6m 612M 4096M 16.2.4 8d91d370c2b8 c211ef5551c9 2021-11-03T17:16:28.265 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (6m) 14s ago 6m 920M 4096M 16.2.4 8d91d370c2b8 b1486ec1b619 2021-11-03T17:16:28.265 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (6m) 14s ago 6m 740M 4096M 16.2.4 8d91d370c2b8 8b91230c69e5 2021-11-03T17:16:28.265 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (6m) 14s ago 6m 843M 4096M 16.2.4 8d91d370c2b8 4a54c4d6b6f5 2021-11-03T17:16:28.266 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (7m) 26s ago 8m 43.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:16:28.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.265+0000 7f72f0683700 1 -- 172.21.15.27:0/3187858201 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f72d0064120 msgr2=0x7f72d00665e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:16:28.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.265+0000 7f72f0683700 1 --2- 172.21.15.27:0/3187858201 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f72d0064120 0x7f72d00665e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f72dc004630 tx=0x7f72dc009b30).stop 2021-11-03T17:16:28.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.265+0000 7f72f0683700 1 -- 172.21.15.27:0/3187858201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72e80f66e0 msgr2=0x7f72e806b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:16:28.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.265+0000 7f72f0683700 1 --2- 172.21.15.27:0/3187858201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72e80f66e0 0x7f72e806b9a0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f72e0007fd0 tx=0x7f72e0004ee0).stop 2021-11-03T17:16:28.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.265+0000 7f72f0683700 1 -- 172.21.15.27:0/3187858201 shutdown_connections 2021-11-03T17:16:28.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.265+0000 7f72f0683700 1 --2- 172.21.15.27:0/3187858201 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f72d0064120 0x7f72d00665e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:28.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.265+0000 7f72f0683700 1 --2- 172.21.15.27:0/3187858201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72e80f66e0 0x7f72e806b9a0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:28.270 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.265+0000 7f72f0683700 1 --2- 172.21.15.27:0/3187858201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72e80f6ff0 0x7f72e8069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:28.270 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.265+0000 7f72f0683700 1 -- 172.21.15.27:0/3187858201 >> 172.21.15.27:0/3187858201 conn(0x7f72e80f1f00 msgr2=0x7f72e80fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:16:28.271 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.265+0000 7f72f0683700 1 -- 172.21.15.27:0/3187858201 shutdown_connections 2021-11-03T17:16:28.271 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.265+0000 7f72f0683700 1 -- 172.21.15.27:0/3187858201 wait complete. 2021-11-03T17:16:28.365 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.365+0000 7fa7ba6fd700 1 -- 172.21.15.27:0/1054792595 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7b406a3b0 msgr2=0x7fa7b406a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:16:28.366 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.365+0000 7fa7ba6fd700 1 --2- 172.21.15.27:0/1054792595 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7b406a3b0 0x7fa7b406a830 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fa7a8004660 tx=0x7fa7a8009b30).stop 2021-11-03T17:16:28.366 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.365+0000 7fa7ba6fd700 1 -- 172.21.15.27:0/1054792595 shutdown_connections 2021-11-03T17:16:28.366 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.365+0000 7fa7ba6fd700 1 --2- 172.21.15.27:0/1054792595 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7b406a3b0 0x7fa7b406a830 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:28.367 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.365+0000 7fa7ba6fd700 1 --2- 172.21.15.27:0/1054792595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7b406b9c0 0x7fa7b4069ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:28.367 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.365+0000 7fa7ba6fd700 1 -- 172.21.15.27:0/1054792595 >> 172.21.15.27:0/1054792595 conn(0x7fa7b40f1e20 msgr2=0x7fa7b40f4240 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:16:28.367 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.365+0000 7fa7ba6fd700 1 -- 172.21.15.27:0/1054792595 shutdown_connections 2021-11-03T17:16:28.367 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.365+0000 7fa7ba6fd700 1 -- 172.21.15.27:0/1054792595 wait complete. 2021-11-03T17:16:28.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.366+0000 7fa7ba6fd700 1 Processor -- start 2021-11-03T17:16:28.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.366+0000 7fa7ba6fd700 1 -- start start 2021-11-03T17:16:28.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.366+0000 7fa7ba6fd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7b406a3b0 0x7fa7b4109150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:16:28.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.366+0000 7fa7ba6fd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7b406b9c0 0x7fa7b4109640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:16:28.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.366+0000 7fa7ba6fd700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b4109bf0 con 0x7fa7b406b9c0 2021-11-03T17:16:28.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.366+0000 7fa7ba6fd700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b4109d30 con 0x7fa7b406a3b0 2021-11-03T17:16:28.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.366+0000 7fa7b37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7b406b9c0 0x7fa7b4109640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:16:28.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.366+0000 7fa7b37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7b406b9c0 0x7fa7b4109640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56084/0 (socket says 172.21.15.27:56084) 2021-11-03T17:16:28.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.366+0000 7fa7b37fe700 1 -- 172.21.15.27:0/466276407 learned_addr learned my addr 172.21.15.27:0/466276407 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:16:28.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.366+0000 7fa7b37fe700 1 -- 172.21.15.27:0/466276407 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7b406a3b0 msgr2=0x7fa7b4109150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:16:28.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.366+0000 7fa7b37fe700 1 --2- 172.21.15.27:0/466276407 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7b406a3b0 0x7fa7b4109150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:16:28.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.366+0000 7fa7b37fe700 1 -- 172.21.15.27:0/466276407 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a8005040 con 0x7fa7b406b9c0 2021-11-03T17:16:28.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.366+0000 7fa7b37fe700 1 --2- 172.21.15.27:0/466276407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7b406b9c0 0x7fa7b4109640 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fa7a8009990 tx=0x7fa7a8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:16:28.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.367+0000 7fa7b0ff9700 1 -- 172.21.15.27:0/466276407 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7a8018070 con 0x7fa7b406b9c0 2021-11-03T17:16:28.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.367+0000 7fa7b0ff9700 1 -- 172.21.15.27:0/466276407 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fa7a8007ba0 con 0x7fa7b406b9c0 2021-11-03T17:16:28.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.367+0000 7fa7ba6fd700 1 -- 172.21.15.27:0/466276407 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7b410a670 con 0x7fa7b406b9c0 2021-11-03T17:16:28.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.367+0000 7fa7b0ff9700 1 -- 172.21.15.27:0/466276407 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7a801c3c0 con 0x7fa7b406b9c0 2021-11-03T17:16:28.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.367+0000 7fa7ba6fd700 1 -- 172.21.15.27:0/466276407 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7b410abd0 con 0x7fa7b406b9c0 2021-11-03T17:16:28.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.368+0000 7fa7ba6fd700 1 -- 172.21.15.27:0/466276407 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7b4045ad0 con 0x7fa7b406b9c0 2021-11-03T17:16:28.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.368+0000 7fa7b0ff9700 1 -- 172.21.15.27:0/466276407 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 35) v1 ==== 81997+0+0 (secure 0 0 0) 0x7fa7a801d420 con 0x7fa7b406b9c0 2021-11-03T17:16:28.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.368+0000 7fa7b0ff9700 1 --2- 172.21.15.27:0/466276407 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa79c064120 0x7fa79c0665e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:16:28.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.369+0000 7fa7b0ff9700 1 -- 172.21.15.27:0/466276407 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fa7a8082420 con 0x7fa7b406b9c0 2021-11-03T17:16:28.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.369+0000 7fa7b3fff700 1 --2- 172.21.15.27:0/466276407 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa79c064120 0x7fa79c0665e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:16:28.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.369+0000 7fa7b3fff700 1 --2- 172.21.15.27:0/466276407 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa79c064120 0x7fa79c0665e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fa7a4004a10 tx=0x7fa7a40099e0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:16:28.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.373+0000 7fa7b0ff9700 1 -- 172.21.15.27:0/466276407 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fa7a80547f0 con 0x7fa7b406b9c0 2021-11-03T17:16:28.601 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.605+0000 7fa7ba6fd700 1 -- 172.21.15.27:0/466276407 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa7b410ae40 con 0x7fa7b406b9c0 2021-11-03T17:16:28.602 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.605+0000 7fa7b0ff9700 1 -- 172.21.15.27:0/466276407 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+700 (secure 0 0 0) 0x7fa7a80543e0 con 0x7fa7b406b9c0 2021-11-03T17:16:28.603 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:16:28.603 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:16:28.604 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:16:28.604 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:16:28.605 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:16:28.605 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:16:28.605 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:16:28.606 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:16:28.607 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 6 2021-11-03T17:16:28.607 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:16:28.608 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:16:28.608 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:16:28.608 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:16:28.609 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:16:28.609 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 10, 2021-11-03T17:16:28.610 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 4 2021-11-03T17:16:28.611 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:16:28.611 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:16:28.613 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.607+0000 7fa7ba6fd700 1 -- 172.21.15.27:0/466276407 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa79c064120 msgr2=0x7fa79c0665e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:16:28.613 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.607+0000 7fa7ba6fd700 1 --2- 172.21.15.27:0/466276407 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa79c064120 0x7fa79c0665e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fa7a4004a10 tx=0x7fa7a40099e0).stop 2021-11-03T17:16:28.614 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.607+0000 7fa7ba6fd700 1 -- 172.21.15.27:0/466276407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7b406b9c0 msgr2=0x7fa7b4109640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:16:28.614 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.607+0000 7fa7ba6fd700 1 --2- 172.21.15.27:0/466276407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7b406b9c0 0x7fa7b4109640 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fa7a8009990 tx=0x7fa7a8009b30).stop 2021-11-03T17:16:28.615 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.607+0000 7fa7ba6fd700 1 -- 172.21.15.27:0/466276407 shutdown_connections 2021-11-03T17:16:28.615 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.607+0000 7fa7ba6fd700 1 --2- 172.21.15.27:0/466276407 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa79c064120 0x7fa79c0665e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:28.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.607+0000 7fa7ba6fd700 1 --2- 172.21.15.27:0/466276407 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7b406a3b0 0x7fa7b4109150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:16:28.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.607+0000 7fa7ba6fd700 1 --2- 172.21.15.27:0/466276407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7b406b9c0 0x7fa7b4109640 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:28.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.607+0000 7fa7ba6fd700 1 -- 172.21.15.27:0/466276407 >> 172.21.15.27:0/466276407 conn(0x7fa7b40f1e20 msgr2=0x7fa7b40fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:16:28.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.607+0000 7fa7ba6fd700 1 -- 172.21.15.27:0/466276407 shutdown_connections 2021-11-03T17:16:28.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.607+0000 7fa7ba6fd700 1 -- 172.21.15.27:0/466276407 wait complete. 2021-11-03T17:16:28.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.708+0000 7f200c29d700 1 -- 172.21.15.27:0/2357755586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20040f66e0 msgr2=0x7f20040f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:16:28.707 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.708+0000 7f200c29d700 1 --2- 172.21.15.27:0/2357755586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20040f66e0 0x7f20040f6b00 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f1ffc0099e0 tx=0x7f1ffc004d10).stop 2021-11-03T17:16:28.707 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.708+0000 7f200c29d700 1 -- 172.21.15.27:0/2357755586 shutdown_connections 2021-11-03T17:16:28.707 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.708+0000 7f200c29d700 1 --2- 172.21.15.27:0/2357755586 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20040f7980 0x7f20040f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:28.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.708+0000 7f200c29d700 1 --2- 172.21.15.27:0/2357755586 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20040f66e0 0x7f20040f6b00 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:28.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.708+0000 7f200c29d700 1 -- 172.21.15.27:0/2357755586 >> 172.21.15.27:0/2357755586 conn(0x7f20040f1e40 msgr2=0x7f20040f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:16:28.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.708+0000 7f200c29d700 1 -- 172.21.15.27:0/2357755586 shutdown_connections 2021-11-03T17:16:28.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.708+0000 7f200c29d700 1 -- 172.21.15.27:0/2357755586 wait complete. 2021-11-03T17:16:28.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.708+0000 7f200c29d700 1 Processor -- start 2021-11-03T17:16:28.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.708+0000 7f200c29d700 1 -- start start 2021-11-03T17:16:28.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.709+0000 7f200c29d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20040f66e0 0x7f2004107080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:16:28.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.709+0000 7f200c29d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20040f7980 0x7f2004107570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:16:28.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.709+0000 7f200c29d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2004107b20 con 0x7f20040f66e0 2021-11-03T17:16:28.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.709+0000 7f200c29d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2004107c60 con 0x7f20040f7980 2021-11-03T17:16:28.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.709+0000 7f200a039700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20040f66e0 0x7f2004107080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:16:28.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.709+0000 7f200a039700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20040f66e0 0x7f2004107080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56092/0 (socket says 172.21.15.27:56092) 2021-11-03T17:16:28.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.709+0000 7f200a039700 1 -- 172.21.15.27:0/4090979606 learned_addr learned my addr 172.21.15.27:0/4090979606 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:16:28.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.709+0000 7f2009838700 1 --2- 172.21.15.27:0/4090979606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20040f7980 0x7f2004107570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:16:28.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.709+0000 7f200a039700 1 -- 172.21.15.27:0/4090979606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20040f7980 msgr2=0x7f2004107570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:16:28.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.709+0000 7f200a039700 1 --2- 172.21.15.27:0/4090979606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20040f7980 0x7f2004107570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:28.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.709+0000 7f200a039700 1 -- 172.21.15.27:0/4090979606 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ffc009640 con 0x7f20040f66e0 2021-11-03T17:16:28.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.709+0000 7f200a039700 1 --2- 172.21.15.27:0/4090979606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20040f66e0 0x7f2004107080 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f1ffc00baa0 tx=0x7f1ffc004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:16:28.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.709+0000 7f2002ffd700 1 -- 172.21.15.27:0/4090979606 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ffc00e070 con 0x7f20040f66e0 2021-11-03T17:16:28.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.709+0000 7f2002ffd700 1 -- 172.21.15.27:0/4090979606 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f1ffc007d40 con 0x7f20040f66e0 2021-11-03T17:16:28.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.709+0000 7f200c29d700 1 -- 172.21.15.27:0/4090979606 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20041085a0 con 0x7f20040f66e0 2021-11-03T17:16:28.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.709+0000 7f2002ffd700 1 -- 172.21.15.27:0/4090979606 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1ffc018070 con 0x7f20040f66e0 2021-11-03T17:16:28.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.709+0000 7f200c29d700 1 -- 172.21.15.27:0/4090979606 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2004108b00 con 0x7f20040f66e0 2021-11-03T17:16:28.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.710+0000 7f2000ff9700 1 -- 172.21.15.27:0/4090979606 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fe8004fa0 con 0x7f20040f66e0 2021-11-03T17:16:28.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.711+0000 7f2002ffd700 1 -- 172.21.15.27:0/4090979606 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 35) v1 ==== 81997+0+0 (secure 0 0 0) 0x7f1ffc024060 con 0x7f20040f66e0 2021-11-03T17:16:28.715 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.711+0000 7f2002ffd700 1 --2- 172.21.15.27:0/4090979606 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1fec064120 0x7f1fec0665e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:16:28.716 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.711+0000 7f2002ffd700 1 -- 172.21.15.27:0/4090979606 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f1ffc081e20 con 0x7f20040f66e0 2021-11-03T17:16:28.716 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.711+0000 7f2009838700 1 --2- 172.21.15.27:0/4090979606 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1fec064120 0x7f1fec0665e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:16:28.716 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.712+0000 7f2009838700 1 --2- 172.21.15.27:0/4090979606 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1fec064120 0x7f1fec0665e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f1ff8009fd0 tx=0x7f1ff800b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:16:28.717 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.715+0000 7f2002ffd700 1 -- 172.21.15.27:0/4090979606 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f1ffc054190 con 0x7f20040f66e0 2021-11-03T17:16:28.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.921+0000 7f2000ff9700 1 -- 172.21.15.27:0/4090979606 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1fe8005e80 con 0x7f20040f66e0 2021-11-03T17:16:28.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.922+0000 7f2002ffd700 1 -- 172.21.15.27:0/4090979606 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1795 (secure 0 0 0) 0x7f1ffc022720 con 0x7f20040f66e0 2021-11-03T17:16:28.918 INFO:teuthology.orchestra.run.smithi027.stdout:e28 2021-11-03T17:16:28.918 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:16:28.919 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:16:28.919 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:16:28.919 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:16:28.919 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:16:28.920 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:16:28.920 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 28 2021-11-03T17:16:28.920 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:16:28.921 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:16:28.921 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:16:13.536828+0000 2021-11-03T17:16:28.922 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:16:28.923 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:16:28.923 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:16:28.923 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:16:28.924 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:16:28.924 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:16:28.924 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:16:28.925 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:16:28.925 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:16:28.925 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 2 2021-11-03T17:16:28.926 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:16:28.926 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:16:28.927 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:16:28.927 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:16:28.928 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:16:28.928 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:16:28.929 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:16:28.929 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:16:28.930 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:16:28.930 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:16:28.930 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 3 export targets 0 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:16:28.931 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:16:28.931 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:16:28.931 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:16:28.931 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:16:28.932 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:16:28.932 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:16:28.932 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:16:28.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.924+0000 7f2000ff9700 1 -- 172.21.15.27:0/4090979606 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1fec064120 msgr2=0x7f1fec0665e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:16:28.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.924+0000 7f2000ff9700 1 --2- 172.21.15.27:0/4090979606 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1fec064120 0x7f1fec0665e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f1ff8009fd0 tx=0x7f1ff800b040).stop 2021-11-03T17:16:28.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.924+0000 7f2000ff9700 1 -- 172.21.15.27:0/4090979606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20040f66e0 msgr2=0x7f2004107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:16:28.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.924+0000 7f2000ff9700 1 --2- 172.21.15.27:0/4090979606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20040f66e0 0x7f2004107080 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f1ffc00baa0 tx=0x7f1ffc004d10).stop 2021-11-03T17:16:28.935 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.924+0000 7f2000ff9700 1 -- 172.21.15.27:0/4090979606 shutdown_connections 2021-11-03T17:16:28.935 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.924+0000 7f2000ff9700 1 --2- 172.21.15.27:0/4090979606 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1fec064120 0x7f1fec0665e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:28.935 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.924+0000 7f2000ff9700 1 --2- 172.21.15.27:0/4090979606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20040f66e0 0x7f2004107080 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:28.935 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.924+0000 7f2000ff9700 1 --2- 172.21.15.27:0/4090979606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20040f7980 0x7f2004107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:28.936 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.924+0000 7f2000ff9700 1 -- 172.21.15.27:0/4090979606 >> 172.21.15.27:0/4090979606 conn(0x7f20040f1e40 msgr2=0x7f20040fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:16:28.936 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.924+0000 7f2000ff9700 1 -- 172.21.15.27:0/4090979606 shutdown_connections 2021-11-03T17:16:28.936 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:28.924+0000 7f2000ff9700 1 -- 172.21.15.27:0/4090979606 wait complete. 2021-11-03T17:16:28.936 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 28 2021-11-03T17:16:29.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:28 smithi115 ceph-mon[65453]: pgmap v21: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s wr, 0 op/s 2021-11-03T17:16:29.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:29.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi115", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2021-11-03T17:16:29.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:16:29.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:28 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/466276407' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:16:29.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:28 smithi027 ceph-mon[97244]: pgmap v21: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s wr, 0 op/s 2021-11-03T17:16:29.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:29.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi115", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2021-11-03T17:16:29.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:16:29.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:28 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/466276407' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:16:30.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:29 smithi115 ceph-mon[65453]: from='client.34138 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:16:30.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:29 smithi115 ceph-mon[65453]: from='client.34142 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:16:30.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:29 smithi115 ceph-mon[65453]: poll took 1248.255 ms: 1 events 2021-11-03T17:16:30.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:29 smithi115 ceph-mon[65453]: Reconfiguring crash.smithi115 (monmap changed)... 2021-11-03T17:16:30.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:29 smithi115 ceph-mon[65453]: Reconfiguring daemon crash.smithi115 on smithi115 2021-11-03T17:16:30.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:29 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/4090979606' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:16:30.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:29 smithi027 ceph-mon[97244]: from='client.34138 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:16:30.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:29 smithi027 ceph-mon[97244]: from='client.34142 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:16:30.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:29 smithi027 ceph-mon[97244]: poll took 1248.255 ms: 1 events 2021-11-03T17:16:30.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:29 smithi027 ceph-mon[97244]: Reconfiguring crash.smithi115 (monmap changed)... 2021-11-03T17:16:30.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:29 smithi027 ceph-mon[97244]: Reconfiguring daemon crash.smithi115 on smithi115 2021-11-03T17:16:30.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:29 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/4090979606' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:16:31.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:30 smithi115 ceph-mon[65453]: pgmap v22: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s wr, 0 op/s 2021-11-03T17:16:31.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:31.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi115.tcyodz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2021-11-03T17:16:31.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mgr services"}]: dispatch 2021-11-03T17:16:31.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:16:31.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:30 smithi027 ceph-mon[97244]: pgmap v22: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 1.6 KiB/s wr, 0 op/s 2021-11-03T17:16:31.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:31.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi115.tcyodz", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2021-11-03T17:16:31.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "mgr services"}]: dispatch 2021-11-03T17:16:31.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:16:32.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:31 smithi115 ceph-mon[65453]: poll took 1283.250 ms: 1 events 2021-11-03T17:16:32.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:31 smithi115 ceph-mon[65453]: Reconfiguring mgr.smithi115.tcyodz (monmap changed)... 2021-11-03T17:16:32.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:31 smithi115 ceph-mon[65453]: Reconfiguring daemon mgr.smithi115.tcyodz on smithi115 2021-11-03T17:16:32.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:32.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-11-03T17:16:32.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-11-03T17:16:32.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:16:32.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:31 smithi027 ceph-mon[97244]: poll took 1283.250 ms: 1 events 2021-11-03T17:16:32.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:31 smithi027 ceph-mon[97244]: Reconfiguring mgr.smithi115.tcyodz (monmap changed)... 2021-11-03T17:16:32.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:31 smithi027 ceph-mon[97244]: Reconfiguring daemon mgr.smithi115.tcyodz on smithi115 2021-11-03T17:16:32.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:32.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2021-11-03T17:16:32.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2021-11-03T17:16:32.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:16:33.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:32 smithi115 ceph-mon[65453]: poll took 1223.516 ms: 1 events 2021-11-03T17:16:33.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:32 smithi115 ceph-mon[65453]: Reconfiguring mon.smithi115 (monmap changed)... 2021-11-03T17:16:33.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:32 smithi115 ceph-mon[65453]: Reconfiguring daemon mon.smithi115 on smithi115 2021-11-03T17:16:33.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:32 smithi115 ceph-mon[65453]: pgmap v23: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.9 KiB/s wr, 0 op/s 2021-11-03T17:16:33.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:32 smithi027 ceph-mon[97244]: poll took 1223.516 ms: 1 events 2021-11-03T17:16:33.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:32 smithi027 ceph-mon[97244]: Reconfiguring mon.smithi115 (monmap changed)... 2021-11-03T17:16:33.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:32 smithi027 ceph-mon[97244]: Reconfiguring daemon mon.smithi115 on smithi115 2021-11-03T17:16:33.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:32 smithi027 ceph-mon[97244]: pgmap v23: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.9 KiB/s wr, 0 op/s 2021-11-03T17:16:34.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:34 smithi115 ceph-mon[65453]: poll took 1264.083 ms: 1 events 2021-11-03T17:16:34.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:34.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:34 smithi115 ceph-mon[65453]: Reconfiguring osd.3 (monmap changed)... 2021-11-03T17:16:34.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:34 smithi115 ceph-mon[65453]: Reconfiguring daemon osd.3 on smithi115 2021-11-03T17:16:34.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:34 smithi115 ceph-mon[65453]: pgmap v24: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.9 KiB/s wr, 0 op/s 2021-11-03T17:16:34.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:34 smithi027 ceph-mon[97244]: poll took 1264.083 ms: 1 events 2021-11-03T17:16:34.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:34.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:34 smithi027 ceph-mon[97244]: Reconfiguring osd.3 (monmap changed)... 2021-11-03T17:16:34.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:34 smithi027 ceph-mon[97244]: Reconfiguring daemon osd.3 on smithi115 2021-11-03T17:16:34.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:34 smithi027 ceph-mon[97244]: pgmap v24: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.9 KiB/s wr, 0 op/s 2021-11-03T17:16:35.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:35 smithi115 ceph-mon[65453]: poll took 1159.684 ms: 1 events 2021-11-03T17:16:35.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:35.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:35 smithi115 ceph-mon[65453]: Reconfiguring osd.4 (monmap changed)... 2021-11-03T17:16:35.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:35 smithi115 ceph-mon[65453]: Reconfiguring daemon osd.4 on smithi115 2021-11-03T17:16:35.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:35 smithi027 ceph-mon[97244]: poll took 1159.684 ms: 1 events 2021-11-03T17:16:35.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:35.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:35 smithi027 ceph-mon[97244]: Reconfiguring osd.4 (monmap changed)... 2021-11-03T17:16:35.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:35 smithi027 ceph-mon[97244]: Reconfiguring daemon osd.4 on smithi115 2021-11-03T17:16:36.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:36 smithi115 ceph-mon[65453]: pgmap v25: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.9 KiB/s wr, 0 op/s 2021-11-03T17:16:36.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:36 smithi115 ceph-mon[65453]: poll took 1176.275 ms: 1 events 2021-11-03T17:16:36.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:36.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:36 smithi115 ceph-mon[65453]: Reconfiguring osd.5 (monmap changed)... 2021-11-03T17:16:36.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:36 smithi115 ceph-mon[65453]: Reconfiguring daemon osd.5 on smithi115 2021-11-03T17:16:37.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:36 smithi027 ceph-mon[97244]: pgmap v25: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.9 KiB/s wr, 0 op/s 2021-11-03T17:16:37.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:36 smithi027 ceph-mon[97244]: poll took 1176.275 ms: 1 events 2021-11-03T17:16:37.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:37.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:36 smithi027 ceph-mon[97244]: Reconfiguring osd.5 (monmap changed)... 2021-11-03T17:16:37.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:36 smithi027 ceph-mon[97244]: Reconfiguring daemon osd.5 on smithi115 2021-11-03T17:16:38.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:38 smithi115 ceph-mon[65453]: poll took 1164.106 ms: 1 events 2021-11-03T17:16:38.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:38.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:38 smithi115 ceph-mon[65453]: Reconfiguring mds.cephfs.smithi115.zsfgnv (monmap changed)... 2021-11-03T17:16:38.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi115.zsfgnv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2021-11-03T17:16:38.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:16:38.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:38 smithi115 ceph-mon[65453]: Reconfiguring daemon mds.cephfs.smithi115.zsfgnv on smithi115 2021-11-03T17:16:38.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:38 smithi115 ceph-mon[65453]: pgmap v26: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 18 KiB/s wr, 1 op/s 2021-11-03T17:16:38.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:38 smithi027 ceph-mon[97244]: poll took 1164.106 ms: 1 events 2021-11-03T17:16:38.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:38.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:38 smithi027 ceph-mon[97244]: Reconfiguring mds.cephfs.smithi115.zsfgnv (monmap changed)... 2021-11-03T17:16:38.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi115.zsfgnv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2021-11-03T17:16:38.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:16:38.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:38 smithi027 ceph-mon[97244]: Reconfiguring daemon mds.cephfs.smithi115.zsfgnv on smithi115 2021-11-03T17:16:38.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:38 smithi027 ceph-mon[97244]: pgmap v26: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 18 KiB/s wr, 1 op/s 2021-11-03T17:16:39.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:39 smithi115 ceph-mon[65453]: poll took 1210.932 ms: 1 events 2021-11-03T17:16:39.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:39.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:39 smithi115 ceph-mon[65453]: Reconfiguring mds.cephfs.smithi115.semywj (monmap changed)... 2021-11-03T17:16:39.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi115.semywj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2021-11-03T17:16:39.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:16:39.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:39 smithi115 ceph-mon[65453]: Reconfiguring daemon mds.cephfs.smithi115.semywj on smithi115 2021-11-03T17:16:40.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:39 smithi027 ceph-mon[97244]: poll took 1210.932 ms: 1 events 2021-11-03T17:16:40.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:40.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:39 smithi027 ceph-mon[97244]: Reconfiguring mds.cephfs.smithi115.semywj (monmap changed)... 2021-11-03T17:16:40.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi115.semywj", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2021-11-03T17:16:40.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:16:40.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:39 smithi027 ceph-mon[97244]: Reconfiguring daemon mds.cephfs.smithi115.semywj on smithi115 2021-11-03T17:16:41.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:40 smithi115 ceph-mon[65453]: pgmap v27: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s wr, 0 op/s 2021-11-03T17:16:41.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:41.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:16:41.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:16:41.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:16:41.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:41.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi027"}]: dispatch 2021-11-03T17:16:41.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi027"}]': finished 2021-11-03T17:16:41.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi115"}]: dispatch 2021-11-03T17:16:41.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi115"}]': finished 2021-11-03T17:16:41.240 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:40 smithi027 ceph-mon[97244]: pgmap v27: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 17 KiB/s wr, 0 op/s 2021-11-03T17:16:41.240 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:41.241 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:16:41.241 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:16:41.241 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:16:41.241 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:41.242 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi027"}]: dispatch 2021-11-03T17:16:41.242 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi027"}]': finished 2021-11-03T17:16:41.242 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi115"}]: dispatch 2021-11-03T17:16:41.242 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi115"}]': finished 2021-11-03T17:16:42.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:41 smithi115 ceph-mon[65453]: poll took 1193.535 ms: 1 events 2021-11-03T17:16:42.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:41 smithi115 ceph-mon[65453]: Upgrade: Setting container_image for all mon 2021-11-03T17:16:42.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:41 smithi027 ceph-mon[97244]: poll took 1193.535 ms: 1 events 2021-11-03T17:16:42.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:41 smithi027 ceph-mon[97244]: Upgrade: Setting container_image for all mon 2021-11-03T17:16:43.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:42 smithi027 ceph-mon[97244]: pgmap v28: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s wr, 2 op/s 2021-11-03T17:16:43.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:42 smithi027 ceph-mon[97244]: poll took 1538.590 ms: 1 events 2021-11-03T17:16:43.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:42 smithi027 ceph-mon[97244]: Upgrade: Updating crash.smithi027 2021-11-03T17:16:43.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:43.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi027", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2021-11-03T17:16:43.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:16:43.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:42 smithi027 ceph-mon[97244]: Deploying daemon crash.smithi027 on smithi027 2021-11-03T17:16:43.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:42 smithi115 ceph-mon[65453]: pgmap v28: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 20 KiB/s wr, 2 op/s 2021-11-03T17:16:43.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:42 smithi115 ceph-mon[65453]: poll took 1538.590 ms: 1 events 2021-11-03T17:16:43.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:42 smithi115 ceph-mon[65453]: Upgrade: Updating crash.smithi027 2021-11-03T17:16:43.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:43.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi027", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2021-11-03T17:16:43.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:16:43.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:42 smithi115 ceph-mon[65453]: Deploying daemon crash.smithi027 on smithi027 2021-11-03T17:16:45.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:44 smithi115 ceph-mon[65453]: pgmap v29: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 18 KiB/s wr, 1 op/s 2021-11-03T17:16:45.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:44 smithi027 ceph-mon[97244]: pgmap v29: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 18 KiB/s wr, 1 op/s 2021-11-03T17:16:46.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:46 smithi027 ceph-mon[97244]: poll took 2842.499 ms: 1 events 2021-11-03T17:16:46.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:46.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:16:46.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:46.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:46 smithi027 ceph-mon[97244]: from='client.34154 -' entity='client.crash.smithi027' cmd=[{"prefix": "crash post", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:16:46.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:46.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:46 smithi027 ceph-mon[97244]: pgmap v30: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s wr, 1 op/s 2021-11-03T17:16:46.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:46 smithi115 ceph-mon[65453]: poll took 2842.499 ms: 1 events 2021-11-03T17:16:46.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:46.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:16:46.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:46.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:46 smithi115 ceph-mon[65453]: from='client.34154 -' entity='client.crash.smithi027' cmd=[{"prefix": "crash post", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:16:46.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:46.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:46 smithi115 ceph-mon[65453]: pgmap v30: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s wr, 1 op/s 2021-11-03T17:16:47.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:47 smithi027 ceph-mon[97244]: Health check failed: 1 daemons have recently crashed (RECENT_CRASH) 2021-11-03T17:16:47.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:47 smithi115 ceph-mon[65453]: Health check failed: 1 daemons have recently crashed (RECENT_CRASH) 2021-11-03T17:16:48.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:48 smithi027 ceph-mon[97244]: pgmap v31: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s wr, 1 op/s 2021-11-03T17:16:48.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:48 smithi115 ceph-mon[65453]: pgmap v31: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 19 KiB/s wr, 1 op/s 2021-11-03T17:16:50.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:50 smithi115 ceph-mon[65453]: pgmap v32: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s wr, 1 op/s 2021-11-03T17:16:50.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:50 smithi027 ceph-mon[97244]: pgmap v32: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s wr, 1 op/s 2021-11-03T17:16:51.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:51 smithi115 ceph-mon[65453]: poll took 5068.909 ms: 1 events 2021-11-03T17:16:51.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:51.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:51.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:16:51.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:16:51.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:16:51.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:51 smithi027 ceph-mon[97244]: poll took 5068.909 ms: 1 events 2021-11-03T17:16:51.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:51.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:51.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:16:51.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:16:51.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:16:52.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:52 smithi115 ceph-mon[65453]: pgmap v33: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s wr, 1 op/s 2021-11-03T17:16:52.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:52.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi115", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2021-11-03T17:16:52.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:16:52.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:52 smithi027 ceph-mon[97244]: pgmap v33: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 2.7 KiB/s wr, 1 op/s 2021-11-03T17:16:52.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:52.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi115", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2021-11-03T17:16:52.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2021-11-03T17:16:53.654 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:53 smithi115 ceph-mon[65453]: poll took 1465.165 ms: 1 events 2021-11-03T17:16:53.654 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:53 smithi115 ceph-mon[65453]: Upgrade: Updating crash.smithi115 2021-11-03T17:16:53.655 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:53 smithi115 ceph-mon[65453]: Deploying daemon crash.smithi115 on smithi115 2021-11-03T17:16:53.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:53 smithi027 ceph-mon[97244]: poll took 1465.165 ms: 1 events 2021-11-03T17:16:53.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:53 smithi027 ceph-mon[97244]: Upgrade: Updating crash.smithi115 2021-11-03T17:16:53.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:53 smithi027 ceph-mon[97244]: Deploying daemon crash.smithi115 on smithi115 2021-11-03T17:16:54.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:54 smithi115 ceph-mon[65453]: pgmap v34: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:16:54.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:54 smithi027 ceph-mon[97244]: pgmap v34: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:16:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:56 smithi115 ceph-mon[65453]: poll took 2921.286 ms: 1 events 2021-11-03T17:16:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:56.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:16:56.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:56 smithi115 ceph-mon[65453]: from='client.44115 -' entity='client.crash.smithi115' cmd=[{"prefix": "crash post", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:16:56.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:56.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:56 smithi115 ceph-mon[65453]: pgmap v35: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:16:56.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:56.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:16:56.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:16:56.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:56 smithi115 ceph-mon[65453]: from='client.44119 -' entity='client.crash.smithi115' cmd=[{"prefix": "crash post", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:16:56.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:56.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:56 smithi115 ceph-mon[65453]: pgmap v36: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:16:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:56 smithi027 ceph-mon[97244]: poll took 2921.286 ms: 1 events 2021-11-03T17:16:56.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:56.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:16:56.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:56 smithi027 ceph-mon[97244]: from='client.44115 -' entity='client.crash.smithi115' cmd=[{"prefix": "crash post", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:16:56.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:56.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:56 smithi027 ceph-mon[97244]: pgmap v35: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:16:56.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:56.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:16:56.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:16:56.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:56 smithi027 ceph-mon[97244]: from='client.44119 -' entity='client.crash.smithi115' cmd=[{"prefix": "crash post", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:16:56.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:56.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:56 smithi027 ceph-mon[97244]: pgmap v36: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:16:57.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:57 smithi115 ceph-mon[65453]: Health check update: 3 daemons have recently crashed (RECENT_CRASH) 2021-11-03T17:16:57.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:57 smithi027 ceph-mon[97244]: Health check update: 3 daemons have recently crashed (RECENT_CRASH) 2021-11-03T17:16:58.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:58 smithi115 ceph-mon[65453]: pgmap v37: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:16:58.623 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:58 smithi027 ceph-mon[97244]: pgmap v37: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:16:59.024 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.028+0000 7f0f7b1eb700 1 -- 172.21.15.27:0/368679606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f740f7980 msgr2=0x7f0f740f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:16:59.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.028+0000 7f0f7b1eb700 1 --2- 172.21.15.27:0/368679606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f740f7980 0x7f0f740f7de0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f0f60004660 tx=0x7f0f60009b30).stop 2021-11-03T17:16:59.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.028+0000 7f0f7b1eb700 1 -- 172.21.15.27:0/368679606 shutdown_connections 2021-11-03T17:16:59.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.028+0000 7f0f7b1eb700 1 --2- 172.21.15.27:0/368679606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f740f7980 0x7f0f740f7de0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:59.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.028+0000 7f0f7b1eb700 1 --2- 172.21.15.27:0/368679606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f740f66e0 0x7f0f740f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:59.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.028+0000 7f0f7b1eb700 1 -- 172.21.15.27:0/368679606 >> 172.21.15.27:0/368679606 conn(0x7f0f740f1e40 msgr2=0x7f0f740f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:16:59.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.028+0000 7f0f7b1eb700 1 -- 172.21.15.27:0/368679606 shutdown_connections 2021-11-03T17:16:59.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.028+0000 7f0f7b1eb700 1 -- 172.21.15.27:0/368679606 wait complete. 2021-11-03T17:16:59.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.028+0000 7f0f7b1eb700 1 Processor -- start 2021-11-03T17:16:59.027 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.028+0000 7f0f7b1eb700 1 -- start start 2021-11-03T17:16:59.029 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.029+0000 7f0f7b1eb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f740f66e0 0x7f0f74106f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:16:59.029 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.029+0000 7f0f7b1eb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f740f7980 0x7f0f74107400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:16:59.029 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.029+0000 7f0f7b1eb700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f741079b0 con 0x7f0f740f7980 2021-11-03T17:16:59.029 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.029+0000 7f0f7b1eb700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f74107af0 con 0x7f0f740f66e0 2021-11-03T17:16:59.030 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.029+0000 7f0f73fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f740f7980 0x7f0f74107400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:16:59.030 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.029+0000 7f0f73fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f740f7980 0x7f0f74107400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56114/0 (socket says 172.21.15.27:56114) 2021-11-03T17:16:59.030 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.029+0000 7f0f73fff700 1 -- 172.21.15.27:0/1980655711 learned_addr learned my addr 172.21.15.27:0/1980655711 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:16:59.030 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.029+0000 7f0f78f87700 1 --2- 172.21.15.27:0/1980655711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f740f66e0 0x7f0f74106f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:16:59.031 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.029+0000 7f0f73fff700 1 -- 172.21.15.27:0/1980655711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f740f66e0 msgr2=0x7f0f74106f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:16:59.031 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.029+0000 7f0f73fff700 1 --2- 172.21.15.27:0/1980655711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f740f66e0 0x7f0f74106f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:59.031 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.029+0000 7f0f73fff700 1 -- 172.21.15.27:0/1980655711 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f60005040 con 0x7f0f740f7980 2021-11-03T17:16:59.031 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.029+0000 7f0f73fff700 1 --2- 172.21.15.27:0/1980655711 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f740f7980 0x7f0f74107400 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f0f60002040 tx=0x7f0f6000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:16:59.032 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.030+0000 7f0f717fa700 1 -- 172.21.15.27:0/1980655711 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f60019070 con 0x7f0f740f7980 2021-11-03T17:16:59.032 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.030+0000 7f0f717fa700 1 -- 172.21.15.27:0/1980655711 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f0f60007e70 con 0x7f0f740f7980 2021-11-03T17:16:59.032 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.030+0000 7f0f7b1eb700 1 -- 172.21.15.27:0/1980655711 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f74108430 con 0x7f0f740f7980 2021-11-03T17:16:59.032 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.030+0000 7f0f717fa700 1 -- 172.21.15.27:0/1980655711 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f60014db0 con 0x7f0f740f7980 2021-11-03T17:16:59.032 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.030+0000 7f0f7b1eb700 1 -- 172.21.15.27:0/1980655711 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f74108990 con 0x7f0f740f7980 2021-11-03T17:16:59.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.031+0000 7f0f7b1eb700 1 -- 172.21.15.27:0/1980655711 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f74045ad0 con 0x7f0f740f7980 2021-11-03T17:16:59.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.031+0000 7f0f717fa700 1 -- 172.21.15.27:0/1980655711 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 35) v1 ==== 81997+0+0 (secure 0 0 0) 0x7f0f60025070 con 0x7f0f740f7980 2021-11-03T17:16:59.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.031+0000 7f0f717fa700 1 --2- 172.21.15.27:0/1980655711 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0f640640d0 0x7f0f64066590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:16:59.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.032+0000 7f0f717fa700 1 -- 172.21.15.27:0/1980655711 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f0f60081710 con 0x7f0f740f7980 2021-11-03T17:16:59.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.032+0000 7f0f78f87700 1 --2- 172.21.15.27:0/1980655711 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0f640640d0 0x7f0f64066590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:16:59.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.032+0000 7f0f78f87700 1 --2- 172.21.15.27:0/1980655711 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0f640640d0 0x7f0f64066590 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f0f68009fd0 tx=0x7f0f6800b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:16:59.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.036+0000 7f0f717fa700 1 -- 172.21.15.27:0/1980655711 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f0f60053a80 con 0x7f0f740f7980 2021-11-03T17:16:59.210 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.215+0000 7f0f7b1eb700 1 -- 172.21.15.27:0/1980655711 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f74108c00 con 0x7f0f640640d0 2021-11-03T17:16:59.211 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.216+0000 7f0f717fa700 1 -- 172.21.15.27:0/1980655711 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+320 (secure 0 0 0) 0x7f0f74108c00 con 0x7f0f640640d0 2021-11-03T17:16:59.213 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.218+0000 7f0f7b1eb700 1 -- 172.21.15.27:0/1980655711 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0f640640d0 msgr2=0x7f0f64066590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:16:59.214 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.218+0000 7f0f7b1eb700 1 --2- 172.21.15.27:0/1980655711 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0f640640d0 0x7f0f64066590 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f0f68009fd0 tx=0x7f0f6800b040).stop 2021-11-03T17:16:59.214 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.218+0000 7f0f7b1eb700 1 -- 172.21.15.27:0/1980655711 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f740f7980 msgr2=0x7f0f74107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:16:59.214 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.218+0000 7f0f7b1eb700 1 --2- 172.21.15.27:0/1980655711 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f740f7980 0x7f0f74107400 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f0f60002040 tx=0x7f0f6000b040).stop 2021-11-03T17:16:59.214 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.218+0000 7f0f7b1eb700 1 -- 172.21.15.27:0/1980655711 shutdown_connections 2021-11-03T17:16:59.215 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.218+0000 7f0f7b1eb700 1 --2- 172.21.15.27:0/1980655711 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0f640640d0 0x7f0f64066590 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:59.215 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.218+0000 7f0f7b1eb700 1 --2- 172.21.15.27:0/1980655711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f740f66e0 0x7f0f74106f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:59.215 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.218+0000 7f0f7b1eb700 1 --2- 172.21.15.27:0/1980655711 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f740f7980 0x7f0f74107400 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:59.215 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.218+0000 7f0f7b1eb700 1 -- 172.21.15.27:0/1980655711 >> 172.21.15.27:0/1980655711 conn(0x7f0f740f1e40 msgr2=0x7f0f740fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:16:59.216 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.218+0000 7f0f7b1eb700 1 -- 172.21.15.27:0/1980655711 shutdown_connections 2021-11-03T17:16:59.216 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.218+0000 7f0f7b1eb700 1 -- 172.21.15.27:0/1980655711 wait complete. 2021-11-03T17:16:59.225 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:16:59.320 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.324+0000 7f99c9506700 1 -- 172.21.15.27:0/3609905281 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99c40f5c30 msgr2=0x7f99c40f6050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:16:59.320 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.324+0000 7f99c9506700 1 --2- 172.21.15.27:0/3609905281 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99c40f5c30 0x7f99c40f6050 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f99b4004660 tx=0x7f99b4009b30).stop 2021-11-03T17:16:59.320 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.324+0000 7f99c9506700 1 -- 172.21.15.27:0/3609905281 shutdown_connections 2021-11-03T17:16:59.321 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.324+0000 7f99c9506700 1 --2- 172.21.15.27:0/3609905281 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99c40f6540 0x7f99c40f69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:59.321 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.324+0000 7f99c9506700 1 --2- 172.21.15.27:0/3609905281 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99c40f5c30 0x7f99c40f6050 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:59.321 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.324+0000 7f99c9506700 1 -- 172.21.15.27:0/3609905281 >> 172.21.15.27:0/3609905281 conn(0x7f99c40f0540 msgr2=0x7f99c40f2960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:16:59.321 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.324+0000 7f99c9506700 1 -- 172.21.15.27:0/3609905281 shutdown_connections 2021-11-03T17:16:59.322 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.324+0000 7f99c9506700 1 -- 172.21.15.27:0/3609905281 wait complete. 2021-11-03T17:16:59.324 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.325+0000 7f99c9506700 1 Processor -- start 2021-11-03T17:16:59.324 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.325+0000 7f99c9506700 1 -- start start 2021-11-03T17:16:59.324 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.325+0000 7f99c9506700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99c40f5c30 0x7f99c4109be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:16:59.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.325+0000 7f99c9506700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99c40f6540 0x7f99c410c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:16:59.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.325+0000 7f99c9506700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c40f5760 con 0x7f99c40f6540 2021-11-03T17:16:59.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.325+0000 7f99c9506700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c40f58a0 con 0x7f99c40f5c30 2021-11-03T17:16:59.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.325+0000 7f99c27fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99c40f6540 0x7f99c410c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:16:59.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.325+0000 7f99c27fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99c40f6540 0x7f99c410c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56124/0 (socket says 172.21.15.27:56124) 2021-11-03T17:16:59.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.325+0000 7f99c27fc700 1 -- 172.21.15.27:0/1308842163 learned_addr learned my addr 172.21.15.27:0/1308842163 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:16:59.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.325+0000 7f99c2ffd700 1 --2- 172.21.15.27:0/1308842163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99c40f5c30 0x7f99c4109be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:16:59.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.325+0000 7f99c27fc700 1 -- 172.21.15.27:0/1308842163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99c40f5c30 msgr2=0x7f99c4109be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:16:59.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.325+0000 7f99c27fc700 1 --2- 172.21.15.27:0/1308842163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99c40f5c30 0x7f99c4109be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:59.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.325+0000 7f99c27fc700 1 -- 172.21.15.27:0/1308842163 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b4005040 con 0x7f99c40f6540 2021-11-03T17:16:59.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.325+0000 7f99c2ffd700 1 --2- 172.21.15.27:0/1308842163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99c40f5c30 0x7f99c4109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-03T17:16:59.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.326+0000 7f99c27fc700 1 --2- 172.21.15.27:0/1308842163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99c40f6540 0x7f99c410c0e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f99ac00e7f0 tx=0x7f99ac00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:16:59.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.326+0000 7f99bb7fe700 1 -- 172.21.15.27:0/1308842163 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99ac0092e0 con 0x7f99c40f6540 2021-11-03T17:16:59.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.326+0000 7f99c9506700 1 -- 172.21.15.27:0/1308842163 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c410c6f0 con 0x7f99c40f6540 2021-11-03T17:16:59.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.326+0000 7f99bb7fe700 1 -- 172.21.15.27:0/1308842163 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f99ac00f040 con 0x7f99c40f6540 2021-11-03T17:16:59.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.326+0000 7f99bb7fe700 1 -- 172.21.15.27:0/1308842163 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99ac014750 con 0x7f99c40f6540 2021-11-03T17:16:59.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.326+0000 7f99c9506700 1 -- 172.21.15.27:0/1308842163 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c410cc80 con 0x7f99c40f6540 2021-11-03T17:16:59.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.327+0000 7f99c9506700 1 -- 172.21.15.27:0/1308842163 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99c40f41d0 con 0x7f99c40f6540 2021-11-03T17:16:59.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.327+0000 7f99bb7fe700 1 -- 172.21.15.27:0/1308842163 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 35) v1 ==== 81997+0+0 (secure 0 0 0) 0x7f99ac01b030 con 0x7f99c40f6540 2021-11-03T17:16:59.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.328+0000 7f99bb7fe700 1 --2- 172.21.15.27:0/1308842163 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f99b0064120 0x7f99b00665e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:16:59.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.328+0000 7f99bb7fe700 1 -- 172.21.15.27:0/1308842163 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f99ac0805e0 con 0x7f99c40f6540 2021-11-03T17:16:59.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.328+0000 7f99c2ffd700 1 --2- 172.21.15.27:0/1308842163 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f99b0064120 0x7f99b00665e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:16:59.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.328+0000 7f99c2ffd700 1 --2- 172.21.15.27:0/1308842163 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f99b0064120 0x7f99b00665e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f99b4004d80 tx=0x7f99b4016040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:16:59.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.332+0000 7f99bb7fe700 1 -- 172.21.15.27:0/1308842163 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f99ac052910 con 0x7f99c40f6540 2021-11-03T17:16:59.506 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.510+0000 7f99c9506700 1 -- 172.21.15.27:0/1308842163 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f99c40fdf30 con 0x7f99b0064120 2021-11-03T17:16:59.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.525+0000 7f99bb7fe700 1 -- 172.21.15.27:0/1308842163 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f99c40fdf30 con 0x7f99b0064120 2021-11-03T17:16:59.521 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:16:59.521 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (8m) 9s ago 9m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:16:59.521 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (15s) 9s ago 9m 7205k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:16:59.522 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (4s) 1s ago 9m 18.1M - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:16:59.522 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (8m) 9s ago 9m 38.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:16:59.522 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (2m) 9s ago 4m 14.9M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:16:59.522 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (5m) 9s ago 5m 3835M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:16:59.523 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (47s) 1s ago 4m 13.5M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:16:59.523 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (2m) 1s ago 5m 15.2M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:16:59.523 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (2m) 9s ago 11m 463M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:16:59.523 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (95s) 1s ago 8m 410M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:16:59.523 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (75s) 9s ago 11m 53.5M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:16:59.524 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (49s) 1s ago 8m 40.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:16:59.524 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (9m) 9s ago 9m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:16:59.524 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (8m) 1s ago 8m 18.2M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:16:59.524 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (8m) 9s ago 8m 747M 4096M 16.2.4 8d91d370c2b8 17f79a815a0f 2021-11-03T17:16:59.525 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (7m) 9s ago 7m 846M 4096M 16.2.4 8d91d370c2b8 422be0cac0db 2021-11-03T17:16:59.525 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (7m) 9s ago 7m 610M 4096M 16.2.4 8d91d370c2b8 c211ef5551c9 2021-11-03T17:16:59.525 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (7m) 1s ago 7m 922M 4096M 16.2.4 8d91d370c2b8 b1486ec1b619 2021-11-03T17:16:59.525 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (6m) 1s ago 6m 742M 4096M 16.2.4 8d91d370c2b8 8b91230c69e5 2021-11-03T17:16:59.525 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (6m) 1s ago 6m 844M 4096M 16.2.4 8d91d370c2b8 4a54c4d6b6f5 2021-11-03T17:16:59.526 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (8m) 9s ago 9m 45.4M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:16:59.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.527+0000 7f99c9506700 1 -- 172.21.15.27:0/1308842163 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f99b0064120 msgr2=0x7f99b00665e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:16:59.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.527+0000 7f99c9506700 1 --2- 172.21.15.27:0/1308842163 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f99b0064120 0x7f99b00665e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f99b4004d80 tx=0x7f99b4016040).stop 2021-11-03T17:16:59.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.527+0000 7f99c9506700 1 -- 172.21.15.27:0/1308842163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99c40f6540 msgr2=0x7f99c410c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:16:59.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.527+0000 7f99c9506700 1 --2- 172.21.15.27:0/1308842163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99c40f6540 0x7f99c410c0e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f99ac00e7f0 tx=0x7f99ac00b040).stop 2021-11-03T17:16:59.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.527+0000 7f99c9506700 1 -- 172.21.15.27:0/1308842163 shutdown_connections 2021-11-03T17:16:59.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.527+0000 7f99c9506700 1 --2- 172.21.15.27:0/1308842163 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f99b0064120 0x7f99b00665e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:59.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.527+0000 7f99c9506700 1 --2- 172.21.15.27:0/1308842163 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99c40f5c30 0x7f99c4109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:59.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.527+0000 7f99c9506700 1 --2- 172.21.15.27:0/1308842163 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99c40f6540 0x7f99c410c0e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:59.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.527+0000 7f99c9506700 1 -- 172.21.15.27:0/1308842163 >> 172.21.15.27:0/1308842163 conn(0x7f99c40f0540 msgr2=0x7f99c40fa4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:16:59.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.527+0000 7f99c9506700 1 -- 172.21.15.27:0/1308842163 shutdown_connections 2021-11-03T17:16:59.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.527+0000 7f99c9506700 1 -- 172.21.15.27:0/1308842163 wait complete. 2021-11-03T17:16:59.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.630+0000 7fe99a59e700 1 -- 172.21.15.27:0/1705112056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9940f5440 msgr2=0x7fe9940f5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:16:59.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.630+0000 7fe99a59e700 1 --2- 172.21.15.27:0/1705112056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9940f5440 0x7fe9940f5860 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fe9840099e0 tx=0x7fe984004d10).stop 2021-11-03T17:16:59.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.630+0000 7fe99a59e700 1 -- 172.21.15.27:0/1705112056 shutdown_connections 2021-11-03T17:16:59.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.630+0000 7fe99a59e700 1 --2- 172.21.15.27:0/1705112056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9940f66e0 0x7fe9940f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:59.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.630+0000 7fe99a59e700 1 --2- 172.21.15.27:0/1705112056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9940f5440 0x7fe9940f5860 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:59.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.630+0000 7fe99a59e700 1 -- 172.21.15.27:0/1705112056 >> 172.21.15.27:0/1705112056 conn(0x7fe9940f0ba0 msgr2=0x7fe9940f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:16:59.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.630+0000 7fe99a59e700 1 -- 172.21.15.27:0/1705112056 shutdown_connections 2021-11-03T17:16:59.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.630+0000 7fe99a59e700 1 -- 172.21.15.27:0/1705112056 wait complete. 2021-11-03T17:16:59.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.631+0000 7fe99a59e700 1 Processor -- start 2021-11-03T17:16:59.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.631+0000 7fe99a59e700 1 -- start start 2021-11-03T17:16:59.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.631+0000 7fe99a59e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9940f5440 0x7fe994109b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:16:59.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.631+0000 7fe99a59e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9940f66e0 0x7fe99410c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:16:59.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.631+0000 7fe99a59e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9940603e0 con 0x7fe9940f5440 2021-11-03T17:16:59.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.631+0000 7fe99a59e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe994060520 con 0x7fe9940f66e0 2021-11-03T17:16:59.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.631+0000 7fe99959c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9940f5440 0x7fe994109b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:16:59.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.631+0000 7fe99959c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9940f5440 0x7fe994109b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56132/0 (socket says 172.21.15.27:56132) 2021-11-03T17:16:59.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.631+0000 7fe99959c700 1 -- 172.21.15.27:0/398786796 learned_addr learned my addr 172.21.15.27:0/398786796 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:16:59.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.631+0000 7fe998d9b700 1 --2- 172.21.15.27:0/398786796 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9940f66e0 0x7fe99410c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:16:59.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.631+0000 7fe99959c700 1 -- 172.21.15.27:0/398786796 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9940f66e0 msgr2=0x7fe99410c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:16:59.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.631+0000 7fe99959c700 1 --2- 172.21.15.27:0/398786796 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9940f66e0 0x7fe99410c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:59.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.631+0000 7fe99959c700 1 -- 172.21.15.27:0/398786796 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe984009640 con 0x7fe9940f5440 2021-11-03T17:16:59.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.631+0000 7fe998d9b700 1 --2- 172.21.15.27:0/398786796 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9940f66e0 0x7fe99410c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:16:59.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.631+0000 7fe99959c700 1 --2- 172.21.15.27:0/398786796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9940f5440 0x7fe994109b90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fe98400bb10 tx=0x7fe984004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:16:59.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.632+0000 7fe991ffb700 1 -- 172.21.15.27:0/398786796 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe98400e070 con 0x7fe9940f5440 2021-11-03T17:16:59.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.632+0000 7fe991ffb700 1 -- 172.21.15.27:0/398786796 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fe98401d8d0 con 0x7fe9940f5440 2021-11-03T17:16:59.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.632+0000 7fe991ffb700 1 -- 172.21.15.27:0/398786796 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe984018070 con 0x7fe9940f5440 2021-11-03T17:16:59.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.632+0000 7fe99a59e700 1 -- 172.21.15.27:0/398786796 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe99410c5e0 con 0x7fe9940f5440 2021-11-03T17:16:59.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.632+0000 7fe99a59e700 1 -- 172.21.15.27:0/398786796 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe99410cac0 con 0x7fe9940f5440 2021-11-03T17:16:59.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.633+0000 7fe99a59e700 1 -- 172.21.15.27:0/398786796 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe994045ad0 con 0x7fe9940f5440 2021-11-03T17:16:59.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.633+0000 7fe991ffb700 1 -- 172.21.15.27:0/398786796 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 35) v1 ==== 81997+0+0 (secure 0 0 0) 0x7fe984024060 con 0x7fe9940f5440 2021-11-03T17:16:59.637 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.634+0000 7fe991ffb700 1 --2- 172.21.15.27:0/398786796 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe97c064180 0x7fe97c066640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:16:59.637 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.634+0000 7fe991ffb700 1 -- 172.21.15.27:0/398786796 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fe984082160 con 0x7fe9940f5440 2021-11-03T17:16:59.637 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.634+0000 7fe998d9b700 1 --2- 172.21.15.27:0/398786796 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe97c064180 0x7fe97c066640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:16:59.637 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.634+0000 7fe998d9b700 1 --2- 172.21.15.27:0/398786796 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe97c064180 0x7fe97c066640 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fe9880041d0 tx=0x7fe988009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:16:59.638 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.638+0000 7fe991ffb700 1 -- 172.21.15.27:0/398786796 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fe9840544d0 con 0x7fe9940f5440 2021-11-03T17:16:59.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:59 smithi115 ceph-mon[65453]: poll took 3044.502 ms: 1 events 2021-11-03T17:16:59.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:59.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:59.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:16:59.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:16:59.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:16:59.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:16:59.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:59 smithi115 ceph-mon[65453]: Upgrade: Setting container_image for all crash 2021-11-03T17:16:59.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:59.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi027"}]: dispatch 2021-11-03T17:16:59.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi027"}]': finished 2021-11-03T17:16:59.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi115"}]: dispatch 2021-11-03T17:16:59.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi115"}]': finished 2021-11-03T17:16:59.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2021-11-03T17:16:59.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:59 smithi115 ceph-mon[65453]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2021-11-03T17:16:59.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:16:59 smithi115 ceph-mon[65453]: Upgrade: osd.0 is safe to restart 2021-11-03T17:16:59.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:59 smithi027 ceph-mon[97244]: poll took 3044.502 ms: 1 events 2021-11-03T17:16:59.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:59.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:59.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:16:59.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:16:59.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:16:59.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:16:59.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:59 smithi027 ceph-mon[97244]: Upgrade: Setting container_image for all crash 2021-11-03T17:16:59.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:16:59.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi027"}]: dispatch 2021-11-03T17:16:59.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi027"}]': finished 2021-11-03T17:16:59.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi115"}]: dispatch 2021-11-03T17:16:59.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi115"}]': finished 2021-11-03T17:16:59.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2021-11-03T17:16:59.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:59 smithi027 ceph-mon[97244]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2021-11-03T17:16:59.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:16:59 smithi027 ceph-mon[97244]: Upgrade: osd.0 is safe to restart 2021-11-03T17:16:59.864 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.868+0000 7fe99a59e700 1 -- 172.21.15.27:0/398786796 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe9940fdcd0 con 0x7fe9940f5440 2021-11-03T17:16:59.864 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.868+0000 7fe991ffb700 1 -- 172.21.15.27:0/398786796 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+700 (secure 0 0 0) 0x7fe9840222d0 con 0x7fe9940f5440 2021-11-03T17:16:59.865 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:16:59.865 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:16:59.865 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:16:59.865 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:16:59.866 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:16:59.866 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:16:59.866 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:16:59.866 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:16:59.867 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 6 2021-11-03T17:16:59.867 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:16:59.867 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:16:59.868 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:16:59.868 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:16:59.868 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:16:59.868 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 10, 2021-11-03T17:16:59.869 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 4 2021-11-03T17:16:59.869 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:16:59.869 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:16:59.870 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.870+0000 7fe99a59e700 1 -- 172.21.15.27:0/398786796 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe97c064180 msgr2=0x7fe97c066640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:16:59.870 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.870+0000 7fe99a59e700 1 --2- 172.21.15.27:0/398786796 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe97c064180 0x7fe97c066640 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fe9880041d0 tx=0x7fe988009040).stop 2021-11-03T17:16:59.870 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.870+0000 7fe99a59e700 1 -- 172.21.15.27:0/398786796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9940f5440 msgr2=0x7fe994109b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:16:59.871 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.870+0000 7fe99a59e700 1 --2- 172.21.15.27:0/398786796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9940f5440 0x7fe994109b90 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7fe98400bb10 tx=0x7fe984004d10).stop 2021-11-03T17:16:59.871 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.870+0000 7fe99a59e700 1 -- 172.21.15.27:0/398786796 shutdown_connections 2021-11-03T17:16:59.872 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.870+0000 7fe99a59e700 1 --2- 172.21.15.27:0/398786796 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe97c064180 0x7fe97c066640 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:59.872 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.870+0000 7fe99a59e700 1 --2- 172.21.15.27:0/398786796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9940f5440 0x7fe994109b90 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:59.872 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.870+0000 7fe99a59e700 1 --2- 172.21.15.27:0/398786796 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9940f66e0 0x7fe99410c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:59.872 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.870+0000 7fe99a59e700 1 -- 172.21.15.27:0/398786796 >> 172.21.15.27:0/398786796 conn(0x7fe9940f0ba0 msgr2=0x7fe9940fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:16:59.873 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.870+0000 7fe99a59e700 1 -- 172.21.15.27:0/398786796 shutdown_connections 2021-11-03T17:16:59.873 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.870+0000 7fe99a59e700 1 -- 172.21.15.27:0/398786796 wait complete. 2021-11-03T17:16:59.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.970+0000 7f9b92592700 1 -- 172.21.15.27:0/766988030 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b8c0f55e0 msgr2=0x7f9b8c0f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:16:59.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.970+0000 7f9b92592700 1 --2- 172.21.15.27:0/766988030 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b8c0f55e0 0x7f9b8c0f5a00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f9b840099e0 tx=0x7f9b84004d10).stop 2021-11-03T17:16:59.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.970+0000 7f9b92592700 1 -- 172.21.15.27:0/766988030 shutdown_connections 2021-11-03T17:16:59.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.970+0000 7f9b92592700 1 --2- 172.21.15.27:0/766988030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b8c0f5ef0 0x7f9b8c0f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:59.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.970+0000 7f9b92592700 1 --2- 172.21.15.27:0/766988030 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b8c0f55e0 0x7f9b8c0f5a00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:59.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.970+0000 7f9b92592700 1 -- 172.21.15.27:0/766988030 >> 172.21.15.27:0/766988030 conn(0x7f9b8c0f0de0 msgr2=0x7f9b8c0f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:16:59.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.970+0000 7f9b92592700 1 -- 172.21.15.27:0/766988030 shutdown_connections 2021-11-03T17:16:59.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.970+0000 7f9b92592700 1 -- 172.21.15.27:0/766988030 wait complete. 2021-11-03T17:16:59.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.970+0000 7f9b92592700 1 Processor -- start 2021-11-03T17:16:59.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.971+0000 7f9b92592700 1 -- start start 2021-11-03T17:16:59.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.971+0000 7f9b92592700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b8c0f55e0 0x7f9b8c107b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:16:59.973 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.971+0000 7f9b92592700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b8c0f5ef0 0x7f9b8c10a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:16:59.973 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.971+0000 7f9b92592700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c060520 con 0x7f9b8c0f5ef0 2021-11-03T17:16:59.973 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.971+0000 7f9b92592700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b8c060660 con 0x7f9b8c0f55e0 2021-11-03T17:16:59.973 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.971+0000 7f9b8b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b8c0f5ef0 0x7f9b8c10a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:16:59.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.971+0000 7f9b8bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b8c0f55e0 0x7f9b8c107b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:16:59.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.971+0000 7f9b8b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b8c0f5ef0 0x7f9b8c10a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56144/0 (socket says 172.21.15.27:56144) 2021-11-03T17:16:59.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.971+0000 7f9b8b7fe700 1 -- 172.21.15.27:0/2086990303 learned_addr learned my addr 172.21.15.27:0/2086990303 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:16:59.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.971+0000 7f9b8bfff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b8c0f55e0 0x7f9b8c107b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.27:47308/0 (socket says 172.21.15.27:47308) 2021-11-03T17:16:59.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.971+0000 7f9b8b7fe700 1 -- 172.21.15.27:0/2086990303 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b8c0f55e0 msgr2=0x7f9b8c107b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:16:59.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.971+0000 7f9b8b7fe700 1 --2- 172.21.15.27:0/2086990303 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b8c0f55e0 0x7f9b8c107b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:16:59.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.971+0000 7f9b8b7fe700 1 -- 172.21.15.27:0/2086990303 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b84009640 con 0x7f9b8c0f5ef0 2021-11-03T17:16:59.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.971+0000 7f9b8b7fe700 1 --2- 172.21.15.27:0/2086990303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b8c0f5ef0 0x7f9b8c10a010 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f9b7c0026d0 tx=0x7f9b7c009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:16:59.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.972+0000 7f9b88ff9700 1 -- 172.21.15.27:0/2086990303 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b7c0079c0 con 0x7f9b8c0f5ef0 2021-11-03T17:16:59.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.972+0000 7f9b88ff9700 1 -- 172.21.15.27:0/2086990303 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f9b7c00f040 con 0x7f9b8c0f5ef0 2021-11-03T17:16:59.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.972+0000 7f9b92592700 1 -- 172.21.15.27:0/2086990303 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b8c10a620 con 0x7f9b8c0f5ef0 2021-11-03T17:16:59.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.972+0000 7f9b88ff9700 1 -- 172.21.15.27:0/2086990303 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b7c0148b0 con 0x7f9b8c0f5ef0 2021-11-03T17:16:59.977 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.972+0000 7f9b92592700 1 -- 172.21.15.27:0/2086990303 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b8c10ab60 con 0x7f9b8c0f5ef0 2021-11-03T17:16:59.977 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.973+0000 7f9b92592700 1 -- 172.21.15.27:0/2086990303 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b8c0f4a70 con 0x7f9b8c0f5ef0 2021-11-03T17:16:59.977 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.973+0000 7f9b88ff9700 1 -- 172.21.15.27:0/2086990303 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 35) v1 ==== 81997+0+0 (secure 0 0 0) 0x7f9b7c007b20 con 0x7f9b8c0f5ef0 2021-11-03T17:16:59.978 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.974+0000 7f9b88ff9700 1 --2- 172.21.15.27:0/2086990303 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9b78064120 0x7f9b780665e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:16:59.978 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.974+0000 7f9b88ff9700 1 -- 172.21.15.27:0/2086990303 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f9b7c080770 con 0x7f9b8c0f5ef0 2021-11-03T17:16:59.978 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.974+0000 7f9b8bfff700 1 --2- 172.21.15.27:0/2086990303 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9b78064120 0x7f9b780665e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:16:59.979 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.974+0000 7f9b8bfff700 1 --2- 172.21.15.27:0/2086990303 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9b78064120 0x7f9b780665e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f9b8400b730 tx=0x7f9b84004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:16:59.979 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:16:59.978+0000 7f9b88ff9700 1 -- 172.21.15.27:0/2086990303 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f9b7c052ae0 con 0x7f9b8c0f5ef0 2021-11-03T17:17:00.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:00.189+0000 7f9b92592700 1 -- 172.21.15.27:0/2086990303 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9b8c045ad0 con 0x7f9b8c0f5ef0 2021-11-03T17:17:00.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:00.190+0000 7f9b88ff9700 1 -- 172.21.15.27:0/2086990303 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1795 (secure 0 0 0) 0x7f9b7c0526d0 con 0x7f9b8c0f5ef0 2021-11-03T17:17:00.186 INFO:teuthology.orchestra.run.smithi027.stdout:e28 2021-11-03T17:17:00.186 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:17:00.186 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:17:00.187 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:17:00.187 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:17:00.187 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:17:00.187 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:17:00.188 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 28 2021-11-03T17:17:00.188 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:17:00.188 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:17:00.188 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:16:13.536828+0000 2021-11-03T17:17:00.189 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:17:00.189 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:17:00.189 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:17:00.189 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:17:00.189 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:17:00.190 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:17:00.190 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:17:00.190 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:17:00.190 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:17:00.191 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 2 2021-11-03T17:17:00.191 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:17:00.191 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:17:00.191 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:17:00.192 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:17:00.192 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:17:00.192 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:17:00.192 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:17:00.193 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:17:00.193 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:17:00.193 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:17:00.193 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 3 export targets 0 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:17:00.193 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:17:00.194 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:17:00.194 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:17:00.194 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:17:00.194 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:17:00.195 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:17:00.195 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:17:00.196 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:00.192+0000 7f9b92592700 1 -- 172.21.15.27:0/2086990303 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9b78064120 msgr2=0x7f9b780665e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:17:00.196 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:00.192+0000 7f9b92592700 1 --2- 172.21.15.27:0/2086990303 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9b78064120 0x7f9b780665e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f9b8400b730 tx=0x7f9b84004d10).stop 2021-11-03T17:17:00.196 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:00.192+0000 7f9b92592700 1 -- 172.21.15.27:0/2086990303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b8c0f5ef0 msgr2=0x7f9b8c10a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:17:00.197 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:00.192+0000 7f9b92592700 1 --2- 172.21.15.27:0/2086990303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b8c0f5ef0 0x7f9b8c10a010 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f9b7c0026d0 tx=0x7f9b7c009de0).stop 2021-11-03T17:17:00.197 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:00.192+0000 7f9b92592700 1 -- 172.21.15.27:0/2086990303 shutdown_connections 2021-11-03T17:17:00.197 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:00.192+0000 7f9b92592700 1 --2- 172.21.15.27:0/2086990303 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9b78064120 0x7f9b780665e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:17:00.197 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:00.192+0000 7f9b92592700 1 --2- 172.21.15.27:0/2086990303 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b8c0f55e0 0x7f9b8c107b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:17:00.198 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:00.192+0000 7f9b92592700 1 --2- 172.21.15.27:0/2086990303 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b8c0f5ef0 0x7f9b8c10a010 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:17:00.198 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:00.192+0000 7f9b92592700 1 -- 172.21.15.27:0/2086990303 >> 172.21.15.27:0/2086990303 conn(0x7f9b8c0f0de0 msgr2=0x7f9b8c0fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:17:00.198 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:00.192+0000 7f9b92592700 1 -- 172.21.15.27:0/2086990303 shutdown_connections 2021-11-03T17:17:00.198 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:00.192+0000 7f9b92592700 1 -- 172.21.15.27:0/2086990303 wait complete. 2021-11-03T17:17:00.199 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 28 2021-11-03T17:17:00.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:00 smithi115 ceph-mon[65453]: from='client.34166 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:17:00.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:00 smithi115 ceph-mon[65453]: from='client.34170 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:17:00.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:00 smithi115 ceph-mon[65453]: pgmap v38: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:17:00.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:00 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/398786796' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:17:00.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:17:00.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:00 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2086990303' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:17:00.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:00 smithi027 ceph-mon[97244]: from='client.34166 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:17:00.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:00 smithi027 ceph-mon[97244]: from='client.34170 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:17:00.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:00 smithi027 ceph-mon[97244]: pgmap v38: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:17:00.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:00 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/398786796' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:17:00.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:17:00.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:00 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2086990303' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:17:01.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:01 smithi115 ceph-mon[65453]: poll took 1723.676 ms: 1 events 2021-11-03T17:17:01.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:01 smithi115 ceph-mon[65453]: Upgrade: Updating osd.0 2021-11-03T17:17:01.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:01 smithi115 ceph-mon[65453]: Deploying daemon osd.0 on smithi027 2021-11-03T17:17:01.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:01 smithi027 ceph-mon[97244]: poll took 1723.676 ms: 1 events 2021-11-03T17:17:01.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:01 smithi027 ceph-mon[97244]: Upgrade: Updating osd.0 2021-11-03T17:17:01.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:01 smithi027 ceph-mon[97244]: Deploying daemon osd.0 on smithi027 2021-11-03T17:17:02.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:02 smithi027 ceph-mon[97244]: pgmap v39: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:17:02.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:02 smithi115 ceph-mon[65453]: pgmap v39: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:17:03.047 INFO:journalctl@ceph.osd.0.smithi027.stdout:Nov 03 17:17:02 smithi027 systemd[1]: Stopping Ceph osd.0 for 32ab977e-3cc8-11ec-8c28-001a4aab830c... 2021-11-03T17:17:03.383 INFO:journalctl@ceph.osd.0.smithi027.stdout:Nov 03 17:17:03 smithi027 conmon[50393]: debug 2021-11-03T17:17:03.051+0000 7faf5d8d4700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2021-11-03T17:17:03.384 INFO:journalctl@ceph.osd.0.smithi027.stdout:Nov 03 17:17:03 smithi027 conmon[50393]: debug 2021-11-03T17:17:03.051+0000 7faf5d8d4700 -1 osd.0 53 *** Got signal Terminated *** 2021-11-03T17:17:03.384 INFO:journalctl@ceph.osd.0.smithi027.stdout:Nov 03 17:17:03 smithi027 conmon[50393]: debug 2021-11-03T17:17:03.051+0000 7faf5d8d4700 -1 osd.0 53 *** Immediate shutdown (osd_fast_shutdown=true) *** 2021-11-03T17:17:03.385 INFO:journalctl@ceph.osd.0.smithi027.stdout:Nov 03 17:17:03 smithi027 bash[104398]: 17f79a815a0fc57e9a5d81f8e9fb2f982109031a5dd415aa51ef8ac2c0e49600 2021-11-03T17:17:03.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:03 smithi115 ceph-mon[65453]: osd.0 reported immediately failed by osd.1 2021-11-03T17:17:03.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:03 smithi115 ceph-mon[65453]: osd.0 failed (root=default,host=smithi027) (connection refused reported by osd.1) 2021-11-03T17:17:03.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:03 smithi115 ceph-mon[65453]: osd.0 reported immediately failed by osd.3 2021-11-03T17:17:03.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:03 smithi115 ceph-mon[65453]: osd.0 reported immediately failed by osd.4 2021-11-03T17:17:03.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:03 smithi115 ceph-mon[65453]: osd.0 reported immediately failed by osd.5 2021-11-03T17:17:03.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:03 smithi115 ceph-mon[65453]: osd.0 reported immediately failed by osd.1 2021-11-03T17:17:03.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:03 smithi115 ceph-mon[65453]: osd.0 reported immediately failed by osd.2 2021-11-03T17:17:03.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:03 smithi115 ceph-mon[65453]: osd.0 reported immediately failed by osd.3 2021-11-03T17:17:03.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:03 smithi115 ceph-mon[65453]: osd.0 reported immediately failed by osd.4 2021-11-03T17:17:03.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:03 smithi115 ceph-mon[65453]: osd.0 reported immediately failed by osd.5 2021-11-03T17:17:03.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:03 smithi115 ceph-mon[65453]: osd.0 reported immediately failed by osd.2 2021-11-03T17:17:03.768 INFO:journalctl@ceph.osd.0.smithi027.stdout:Nov 03 17:17:03 smithi027 bash[104398]: Error: no container with name or ID ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c-osd-0 found: no such container 2021-11-03T17:17:03.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:03 smithi027 ceph-mon[97244]: osd.0 reported immediately failed by osd.1 2021-11-03T17:17:03.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:03 smithi027 ceph-mon[97244]: osd.0 failed (root=default,host=smithi027) (connection refused reported by osd.1) 2021-11-03T17:17:03.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:03 smithi027 ceph-mon[97244]: osd.0 reported immediately failed by osd.3 2021-11-03T17:17:03.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:03 smithi027 ceph-mon[97244]: osd.0 reported immediately failed by osd.4 2021-11-03T17:17:03.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:03 smithi027 ceph-mon[97244]: osd.0 reported immediately failed by osd.5 2021-11-03T17:17:03.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:03 smithi027 ceph-mon[97244]: osd.0 reported immediately failed by osd.1 2021-11-03T17:17:03.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:03 smithi027 ceph-mon[97244]: osd.0 reported immediately failed by osd.2 2021-11-03T17:17:03.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:03 smithi027 ceph-mon[97244]: osd.0 reported immediately failed by osd.3 2021-11-03T17:17:03.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:03 smithi027 ceph-mon[97244]: osd.0 reported immediately failed by osd.4 2021-11-03T17:17:03.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:03 smithi027 ceph-mon[97244]: osd.0 reported immediately failed by osd.5 2021-11-03T17:17:03.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:03 smithi027 ceph-mon[97244]: osd.0 reported immediately failed by osd.2 2021-11-03T17:17:04.237 INFO:journalctl@ceph.osd.0.smithi027.stdout:Nov 03 17:17:03 smithi027 bash[104398]: Error: container 17f79a815a0fc57e9a5d81f8e9fb2f982109031a5dd415aa51ef8ac2c0e49600 does not exist in database: no such container 2021-11-03T17:17:04.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:04 smithi027 ceph-mon[97244]: Health check failed: 1 osds down (OSD_DOWN) 2021-11-03T17:17:04.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:04 smithi027 ceph-mon[97244]: osdmap e54: 6 total, 5 up, 6 in 2021-11-03T17:17:04.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:04 smithi027 ceph-mon[97244]: pgmap v41: 65 pgs: 9 stale+active+clean, 56 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:17:04.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:04 smithi115 ceph-mon[65453]: Health check failed: 1 osds down (OSD_DOWN) 2021-11-03T17:17:04.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:04 smithi115 ceph-mon[65453]: osdmap e54: 6 total, 5 up, 6 in 2021-11-03T17:17:04.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:04 smithi115 ceph-mon[65453]: pgmap v41: 65 pgs: 9 stale+active+clean, 56 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:17:05.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:05 smithi115 ceph-mon[65453]: osdmap e55: 6 total, 5 up, 6 in 2021-11-03T17:17:05.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:05 smithi027 ceph-mon[97244]: osdmap e55: 6 total, 5 up, 6 in 2021-11-03T17:17:05.769 INFO:journalctl@ceph.osd.0.smithi027.stdout:Nov 03 17:17:05 smithi027 systemd[1]: ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c@osd.0.service: Succeeded. 2021-11-03T17:17:05.769 INFO:journalctl@ceph.osd.0.smithi027.stdout:Nov 03 17:17:05 smithi027 systemd[1]: Stopped Ceph osd.0 for 32ab977e-3cc8-11ec-8c28-001a4aab830c. 2021-11-03T17:17:05.770 INFO:journalctl@ceph.osd.0.smithi027.stdout:Nov 03 17:17:05 smithi027 systemd[1]: Starting Ceph osd.0 for 32ab977e-3cc8-11ec-8c28-001a4aab830c... 2021-11-03T17:17:06.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:06 smithi115 ceph-mon[65453]: pgmap v43: 65 pgs: 9 stale+active+clean, 56 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:17:06.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:06 smithi027 ceph-mon[97244]: pgmap v43: 65 pgs: 9 stale+active+clean, 56 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:17:06.770 INFO:journalctl@ceph.osd.0.smithi027.stdout:Nov 03 17:17:06 smithi027 bash[104886]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2021-11-03T17:17:06.770 INFO:journalctl@ceph.osd.0.smithi027.stdout:Nov 03 17:17:06 smithi027 bash[104886]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --dev /dev/mapper/vg_nvme-lv_4 --path /var/lib/ceph/osd/ceph-0 --no-mon-config 2021-11-03T17:17:06.771 INFO:journalctl@ceph.osd.0.smithi027.stdout:Nov 03 17:17:06 smithi027 bash[104886]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2021-11-03T17:17:06.771 INFO:journalctl@ceph.osd.0.smithi027.stdout:Nov 03 17:17:06 smithi027 bash[104886]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2021-11-03T17:17:06.772 INFO:journalctl@ceph.osd.0.smithi027.stdout:Nov 03 17:17:06 smithi027 bash[104886]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2021-11-03T17:17:06.772 INFO:journalctl@ceph.osd.0.smithi027.stdout:Nov 03 17:17:06 smithi027 bash[104886]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2021-11-03T17:17:06.772 INFO:journalctl@ceph.osd.0.smithi027.stdout:Nov 03 17:17:06 smithi027 bash[104886]: --> ceph-volume raw activate successful for osd ID: 0 2021-11-03T17:17:08.019 INFO:journalctl@ceph.osd.0.smithi027.stdout:Nov 03 17:17:07 smithi027 bash[104886]: da109dc25a0588dacd346dc66a76937b98b6963e7552ad56993d6fb9ec038951 2021-11-03T17:17:08.020 INFO:journalctl@ceph.osd.0.smithi027.stdout:Nov 03 17:17:07 smithi027 ceph-osd[105415]: -- 172.21.15.27:0/1706120172 <== mon.0 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563c8ec48340 con 0x563c8ec0ec00 2021-11-03T17:17:08.020 INFO:journalctl@ceph.osd.0.smithi027.stdout:Nov 03 17:17:07 smithi027 systemd[1]: Started Ceph osd.0 for 32ab977e-3cc8-11ec-8c28-001a4aab830c. 2021-11-03T17:17:08.519 INFO:journalctl@ceph.osd.0.smithi027.stdout:Nov 03 17:17:08 smithi027 conmon[105391]: 2021-11-03T17:17:08.190+0000 7f4181bb20c0 -1 Falling back to public interface 2021-11-03T17:17:08.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:08 smithi115 ceph-mon[65453]: poll took 7318.684 ms: 1 events 2021-11-03T17:17:08.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:17:08.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:17:08.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:08 smithi115 ceph-mon[65453]: pgmap v44: 65 pgs: 33 active+undersized+degraded, 1 active+undersized, 31 active+clean; 564 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 7723/52479 objects degraded (14.716%) 2021-11-03T17:17:09.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:08 smithi027 ceph-mon[97244]: poll took 7318.684 ms: 1 events 2021-11-03T17:17:09.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:17:09.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:17:09.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:08 smithi027 ceph-mon[97244]: pgmap v44: 65 pgs: 33 active+undersized+degraded, 1 active+undersized, 31 active+clean; 564 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 7723/52479 objects degraded (14.716%) 2021-11-03T17:17:09.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:09 smithi115 ceph-mon[65453]: Health check failed: Degraded data redundancy: 7723/52479 objects degraded (14.716%), 33 pgs degraded (PG_DEGRADED) 2021-11-03T17:17:10.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:09 smithi027 ceph-mon[97244]: Health check failed: Degraded data redundancy: 7723/52479 objects degraded (14.716%), 33 pgs degraded (PG_DEGRADED) 2021-11-03T17:17:10.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:10 smithi115 ceph-mon[65453]: pgmap v45: 65 pgs: 33 active+undersized+degraded, 1 active+undersized, 31 active+clean; 564 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 7723/52479 objects degraded (14.716%) 2021-11-03T17:17:10.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:17:11.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:10 smithi027 ceph-mon[97244]: pgmap v45: 65 pgs: 33 active+undersized+degraded, 1 active+undersized, 31 active+clean; 564 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 7723/52479 objects degraded (14.716%) 2021-11-03T17:17:11.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:17:12.938 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:12 smithi027 ceph-mon[97244]: pgmap v46: 65 pgs: 33 active+undersized+degraded, 1 active+undersized, 31 active+clean; 564 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 7723/52479 objects degraded (14.716%) 2021-11-03T17:17:12.938 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:17:13.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:12 smithi115 ceph-mon[65453]: pgmap v46: 65 pgs: 33 active+undersized+degraded, 1 active+undersized, 31 active+clean; 564 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 7723/52479 objects degraded (14.716%) 2021-11-03T17:17:13.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:17:14.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:13 smithi115 ceph-mon[65453]: poll took 4586.480 ms: 1 events 2021-11-03T17:17:14.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:17:14.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:17:14.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:17:14.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:17:14.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:17:14.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2021-11-03T17:17:14.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:13 smithi027 ceph-mon[97244]: poll took 4586.480 ms: 1 events 2021-11-03T17:17:14.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:17:14.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:17:14.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:17:14.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:17:14.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:17:14.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2021-11-03T17:17:15.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:14 smithi115 ceph-mon[65453]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2021-11-03T17:17:15.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:14 smithi115 ceph-mon[65453]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2021-11-03T17:17:15.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:14 smithi115 ceph-mon[65453]: pgmap v47: 65 pgs: 33 active+undersized+degraded, 1 active+undersized, 31 active+clean; 564 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 7723/52479 objects degraded (14.716%) 2021-11-03T17:17:15.268 INFO:journalctl@ceph.osd.0.smithi027.stdout:Nov 03 17:17:15 smithi027 conmon[105391]: 2021-11-03T17:17:15.011+0000 7f4181bb20c0 -1 osd.0 53 log_to_monitors {default=true} 2021-11-03T17:17:15.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:14 smithi027 ceph-mon[97244]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2021-11-03T17:17:15.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:14 smithi027 ceph-mon[97244]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2021-11-03T17:17:15.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:14 smithi027 ceph-mon[97244]: pgmap v47: 65 pgs: 33 active+undersized+degraded, 1 active+undersized, 31 active+clean; 564 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 7723/52479 objects degraded (14.716%) 2021-11-03T17:17:16.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:15 smithi115 ceph-mon[65453]: from='osd.0 [v2:172.21.15.27:6802/2052665725,v1:172.21.15.27:6803/2052665725]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2021-11-03T17:17:16.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:15 smithi027 ceph-mon[97244]: from='osd.0 [v2:172.21.15.27:6802/2052665725,v1:172.21.15.27:6803/2052665725]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2021-11-03T17:17:16.269 INFO:journalctl@ceph.osd.0.smithi027.stdout:Nov 03 17:17:15 smithi027 conmon[105391]: 2021-11-03T17:17:15.828+0000 7f41744f4700 -1 osd.0 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2021-11-03T17:17:17.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:16 smithi115 ceph-mon[65453]: pgmap v48: 65 pgs: 33 active+undersized+degraded, 1 active+undersized, 31 active+clean; 564 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 7723/52479 objects degraded (14.716%) 2021-11-03T17:17:17.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:16 smithi115 ceph-mon[65453]: from='osd.0 [v2:172.21.15.27:6802/2052665725,v1:172.21.15.27:6803/2052665725]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2021-11-03T17:17:17.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:16 smithi115 ceph-mon[65453]: osdmap e56: 6 total, 5 up, 6 in 2021-11-03T17:17:17.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:16 smithi115 ceph-mon[65453]: from='osd.0 [v2:172.21.15.27:6802/2052665725,v1:172.21.15.27:6803/2052665725]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2021-11-03T17:17:17.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:16 smithi027 ceph-mon[97244]: pgmap v48: 65 pgs: 33 active+undersized+degraded, 1 active+undersized, 31 active+clean; 564 MiB data, 2.2 GiB used, 534 GiB / 536 GiB avail; 7723/52479 objects degraded (14.716%) 2021-11-03T17:17:17.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:16 smithi027 ceph-mon[97244]: from='osd.0 [v2:172.21.15.27:6802/2052665725,v1:172.21.15.27:6803/2052665725]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2021-11-03T17:17:17.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:16 smithi027 ceph-mon[97244]: osdmap e56: 6 total, 5 up, 6 in 2021-11-03T17:17:17.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:16 smithi027 ceph-mon[97244]: from='osd.0 [v2:172.21.15.27:6802/2052665725,v1:172.21.15.27:6803/2052665725]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2021-11-03T17:17:18.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:17 smithi115 ceph-mon[65453]: Health check cleared: OSD_DOWN (was: 1 osds down) 2021-11-03T17:17:18.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:17 smithi115 ceph-mon[65453]: osd.0 [v2:172.21.15.27:6802/2052665725,v1:172.21.15.27:6803/2052665725] boot 2021-11-03T17:17:18.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:17 smithi115 ceph-mon[65453]: osdmap e57: 6 total, 6 up, 6 in 2021-11-03T17:17:18.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-11-03T17:17:18.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:17 smithi115 ceph-mon[65453]: osdmap e58: 6 total, 6 up, 6 in 2021-11-03T17:17:18.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:17 smithi027 ceph-mon[97244]: Health check cleared: OSD_DOWN (was: 1 osds down) 2021-11-03T17:17:18.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:17 smithi027 ceph-mon[97244]: osd.0 [v2:172.21.15.27:6802/2052665725,v1:172.21.15.27:6803/2052665725] boot 2021-11-03T17:17:18.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:17 smithi027 ceph-mon[97244]: osdmap e57: 6 total, 6 up, 6 in 2021-11-03T17:17:18.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2021-11-03T17:17:18.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:17 smithi027 ceph-mon[97244]: osdmap e58: 6 total, 6 up, 6 in 2021-11-03T17:17:19.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:18 smithi115 ceph-mon[65453]: pgmap v51: 65 pgs: 33 active+undersized+degraded, 1 active+undersized, 31 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 7723/52479 objects degraded (14.716%) 2021-11-03T17:17:19.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:18 smithi027 ceph-mon[97244]: pgmap v51: 65 pgs: 33 active+undersized+degraded, 1 active+undersized, 31 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 7723/52479 objects degraded (14.716%) 2021-11-03T17:17:20.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:20 smithi115 ceph-mon[65453]: pgmap v53: 65 pgs: 33 active+undersized+degraded, 1 active+undersized, 31 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 7723/52479 objects degraded (14.716%) 2021-11-03T17:17:20.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:20 smithi027 ceph-mon[97244]: pgmap v53: 65 pgs: 33 active+undersized+degraded, 1 active+undersized, 31 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 7723/52479 objects degraded (14.716%) 2021-11-03T17:17:22.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:21 smithi115 ceph-mon[65453]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 7723/52479 objects degraded (14.716%), 33 pgs degraded) 2021-11-03T17:17:22.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:21 smithi027 ceph-mon[97244]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 7723/52479 objects degraded (14.716%), 33 pgs degraded) 2021-11-03T17:17:23.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:22 smithi115 ceph-mon[65453]: pgmap v54: 65 pgs: 9 active+clean+wait, 56 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:17:23.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:22 smithi027 ceph-mon[97244]: pgmap v54: 65 pgs: 9 active+clean+wait, 56 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:17:25.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:24 smithi115 ceph-mon[65453]: pgmap v55: 65 pgs: 9 active+clean+wait, 56 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:17:25.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:24 smithi027 ceph-mon[97244]: pgmap v55: 65 pgs: 9 active+clean+wait, 56 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:17:26.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:17:26.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:26 smithi115 ceph-mon[65453]: pgmap v56: 65 pgs: 9 active+clean+wait, 56 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:17:27.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:17:27.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:26 smithi027 ceph-mon[97244]: pgmap v56: 65 pgs: 9 active+clean+wait, 56 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:17:29.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:28 smithi027 ceph-mon[97244]: pgmap v57: 65 pgs: 9 active+clean+wait, 56 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 307 B/s wr, 0 op/s 2021-11-03T17:17:29.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2021-11-03T17:17:29.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:28 smithi115 ceph-mon[65453]: pgmap v57: 65 pgs: 9 active+clean+wait, 56 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 307 B/s wr, 0 op/s 2021-11-03T17:17:29.455 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2021-11-03T17:17:30.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:29 smithi027 ceph-mon[97244]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2021-11-03T17:17:30.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:29 smithi027 ceph-mon[97244]: Upgrade: osd.1 is safe to restart 2021-11-03T17:17:30.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:29 smithi027 ceph-mon[97244]: poll took 15021.624 ms: 1 events 2021-11-03T17:17:30.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:17:30.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:29 smithi115 ceph-mon[65453]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2021-11-03T17:17:30.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:29 smithi115 ceph-mon[65453]: Upgrade: osd.1 is safe to restart 2021-11-03T17:17:30.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:29 smithi115 ceph-mon[65453]: poll took 15021.624 ms: 1 events 2021-11-03T17:17:30.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:17:30.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.303+0000 7f582f7b4700 1 -- 172.21.15.27:0/1806129092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58280f6860 msgr2=0x7f58280f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:17:30.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.303+0000 7f582f7b4700 1 --2- 172.21.15.27:0/1806129092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58280f6860 0x7f58280f6cc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f5818004660 tx=0x7f5818009b30).stop 2021-11-03T17:17:30.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.303+0000 7f582f7b4700 1 -- 172.21.15.27:0/1806129092 shutdown_connections 2021-11-03T17:17:30.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.303+0000 7f582f7b4700 1 --2- 172.21.15.27:0/1806129092 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58280f6860 0x7f58280f6cc0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:17:30.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.303+0000 7f582f7b4700 1 --2- 172.21.15.27:0/1806129092 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58280f55c0 0x7f58280f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:17:30.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.303+0000 7f582f7b4700 1 -- 172.21.15.27:0/1806129092 >> 172.21.15.27:0/1806129092 conn(0x7f58280f0d40 msgr2=0x7f58280f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:17:30.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.303+0000 7f582f7b4700 1 -- 172.21.15.27:0/1806129092 shutdown_connections 2021-11-03T17:17:30.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.303+0000 7f582f7b4700 1 -- 172.21.15.27:0/1806129092 wait complete. 2021-11-03T17:17:30.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.303+0000 7f582f7b4700 1 Processor -- start 2021-11-03T17:17:30.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.303+0000 7f582f7b4700 1 -- start start 2021-11-03T17:17:30.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.304+0000 7f582f7b4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58280f55c0 0x7f5828106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:17:30.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.304+0000 7f582f7b4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58280f6860 0x7f5828107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:17:30.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.304+0000 7f582f7b4700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5828107a10 con 0x7f58280f55c0 2021-11-03T17:17:30.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.304+0000 7f582f7b4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5828107b50 con 0x7f58280f6860 2021-11-03T17:17:30.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.304+0000 7f582d550700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58280f55c0 0x7f5828106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:17:30.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.304+0000 7f582d550700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58280f55c0 0x7f5828106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56222/0 (socket says 172.21.15.27:56222) 2021-11-03T17:17:30.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.304+0000 7f582d550700 1 -- 172.21.15.27:0/916797826 learned_addr learned my addr 172.21.15.27:0/916797826 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:17:30.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.304+0000 7f582cd4f700 1 --2- 172.21.15.27:0/916797826 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58280f6860 0x7f5828107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:17:30.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.304+0000 7f582d550700 1 -- 172.21.15.27:0/916797826 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58280f6860 msgr2=0x7f5828107460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:17:30.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.304+0000 7f582d550700 1 --2- 172.21.15.27:0/916797826 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58280f6860 0x7f5828107460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:17:30.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.304+0000 7f582d550700 1 -- 172.21.15.27:0/916797826 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5818005040 con 0x7f58280f55c0 2021-11-03T17:17:30.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.304+0000 7f582d550700 1 --2- 172.21.15.27:0/916797826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58280f55c0 0x7f5828106f70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f582400c530 tx=0x7f5824007e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:17:30.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.304+0000 7f581dffb700 1 -- 172.21.15.27:0/916797826 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f582400f650 con 0x7f58280f55c0 2021-11-03T17:17:30.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.304+0000 7f581dffb700 1 -- 172.21.15.27:0/916797826 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f582400fc60 con 0x7f58280f55c0 2021-11-03T17:17:30.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.304+0000 7f581dffb700 1 -- 172.21.15.27:0/916797826 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5824012cb0 con 0x7f58280f55c0 2021-11-03T17:17:30.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.304+0000 7f582f7b4700 1 -- 172.21.15.27:0/916797826 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58281084f0 con 0x7f58280f55c0 2021-11-03T17:17:30.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.305+0000 7f582f7b4700 1 -- 172.21.15.27:0/916797826 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5828108a60 con 0x7f58280f55c0 2021-11-03T17:17:30.307 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.305+0000 7f582f7b4700 1 -- 172.21.15.27:0/916797826 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5828045ad0 con 0x7f58280f55c0 2021-11-03T17:17:30.307 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.306+0000 7f581dffb700 1 -- 172.21.15.27:0/916797826 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 35) v1 ==== 81997+0+0 (secure 0 0 0) 0x7f5824019030 con 0x7f58280f55c0 2021-11-03T17:17:30.307 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.306+0000 7f581dffb700 1 --2- 172.21.15.27:0/916797826 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5814064130 0x7f58140665f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:17:30.307 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.306+0000 7f581dffb700 1 -- 172.21.15.27:0/916797826 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f58240805f0 con 0x7f58280f55c0 2021-11-03T17:17:30.308 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.306+0000 7f582cd4f700 1 --2- 172.21.15.27:0/916797826 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5814064130 0x7f58140665f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:17:30.308 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.307+0000 7f582cd4f700 1 --2- 172.21.15.27:0/916797826 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5814064130 0x7f58140665f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f5818004e30 tx=0x7f5818009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:17:30.308 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.310+0000 7f581dffb700 1 -- 172.21.15.27:0/916797826 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f5824052920 con 0x7f58280f55c0 2021-11-03T17:17:30.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.504+0000 7f582f7b4700 1 -- 172.21.15.27:0/916797826 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f582805fd30 con 0x7f5814064130 2021-11-03T17:17:30.500 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.505+0000 7f581dffb700 1 -- 172.21.15.27:0/916797826 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+320 (secure 0 0 0) 0x7f582805fd30 con 0x7f5814064130 2021-11-03T17:17:30.502 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.507+0000 7f582f7b4700 1 -- 172.21.15.27:0/916797826 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5814064130 msgr2=0x7f58140665f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:17:30.502 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.507+0000 7f582f7b4700 1 --2- 172.21.15.27:0/916797826 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5814064130 0x7f58140665f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f5818004e30 tx=0x7f5818009b30).stop 2021-11-03T17:17:30.503 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.507+0000 7f582f7b4700 1 -- 172.21.15.27:0/916797826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58280f55c0 msgr2=0x7f5828106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:17:30.503 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.507+0000 7f582f7b4700 1 --2- 172.21.15.27:0/916797826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58280f55c0 0x7f5828106f70 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f582400c530 tx=0x7f5824007e10).stop 2021-11-03T17:17:30.503 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.507+0000 7f582f7b4700 1 -- 172.21.15.27:0/916797826 shutdown_connections 2021-11-03T17:17:30.503 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.507+0000 7f582f7b4700 1 --2- 172.21.15.27:0/916797826 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5814064130 0x7f58140665f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:17:30.504 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.507+0000 7f582f7b4700 1 --2- 172.21.15.27:0/916797826 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58280f55c0 0x7f5828106f70 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:17:30.504 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.507+0000 7f582f7b4700 1 --2- 172.21.15.27:0/916797826 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58280f6860 0x7f5828107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:17:30.504 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.507+0000 7f582f7b4700 1 -- 172.21.15.27:0/916797826 >> 172.21.15.27:0/916797826 conn(0x7f58280f0d40 msgr2=0x7f582805f620 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:17:30.504 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.507+0000 7f582f7b4700 1 -- 172.21.15.27:0/916797826 shutdown_connections 2021-11-03T17:17:30.504 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.508+0000 7f582f7b4700 1 -- 172.21.15.27:0/916797826 wait complete. 2021-11-03T17:17:30.512 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:17:30.608 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.611+0000 7f090bd1b700 1 -- 172.21.15.27:0/800627542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09040f7980 msgr2=0x7f09040f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:17:30.609 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.611+0000 7f090bd1b700 1 --2- 172.21.15.27:0/800627542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09040f7980 0x7f09040f7de0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f08f4004660 tx=0x7f08f4009b30).stop 2021-11-03T17:17:30.609 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.611+0000 7f090bd1b700 1 -- 172.21.15.27:0/800627542 shutdown_connections 2021-11-03T17:17:30.609 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.611+0000 7f090bd1b700 1 --2- 172.21.15.27:0/800627542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09040f7980 0x7f09040f7de0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:17:30.609 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.611+0000 7f090bd1b700 1 --2- 172.21.15.27:0/800627542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09040f66e0 0x7f09040f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:17:30.610 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.611+0000 7f090bd1b700 1 -- 172.21.15.27:0/800627542 >> 172.21.15.27:0/800627542 conn(0x7f09040f1e40 msgr2=0x7f09040f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:17:30.610 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.611+0000 7f090bd1b700 1 -- 172.21.15.27:0/800627542 shutdown_connections 2021-11-03T17:17:30.610 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.612+0000 7f090bd1b700 1 -- 172.21.15.27:0/800627542 wait complete. 2021-11-03T17:17:30.610 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.612+0000 7f090bd1b700 1 Processor -- start 2021-11-03T17:17:30.611 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.612+0000 7f090bd1b700 1 -- start start 2021-11-03T17:17:30.611 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.612+0000 7f090bd1b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09040f66e0 0x7f0904106f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:17:30.611 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.612+0000 7f090bd1b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09040f7980 0x7f0904107430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:17:30.611 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.612+0000 7f090bd1b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09041079e0 con 0x7f09040f7980 2021-11-03T17:17:30.611 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.612+0000 7f090bd1b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0904107b20 con 0x7f09040f66e0 2021-11-03T17:17:30.612 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.612+0000 7f09092b6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09040f7980 0x7f0904107430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:17:30.612 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.612+0000 7f09092b6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09040f7980 0x7f0904107430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56234/0 (socket says 172.21.15.27:56234) 2021-11-03T17:17:30.612 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.612+0000 7f09092b6700 1 -- 172.21.15.27:0/687016216 learned_addr learned my addr 172.21.15.27:0/687016216 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:17:30.612 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.612+0000 7f0909ab7700 1 --2- 172.21.15.27:0/687016216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09040f66e0 0x7f0904106f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:17:30.613 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.613+0000 7f09092b6700 1 -- 172.21.15.27:0/687016216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09040f66e0 msgr2=0x7f0904106f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:17:30.613 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.613+0000 7f09092b6700 1 --2- 172.21.15.27:0/687016216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09040f66e0 0x7f0904106f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:17:30.613 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.613+0000 7f09092b6700 1 -- 172.21.15.27:0/687016216 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f4005040 con 0x7f09040f7980 2021-11-03T17:17:30.613 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.613+0000 7f09092b6700 1 --2- 172.21.15.27:0/687016216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09040f7980 0x7f0904107430 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f08f4009990 tx=0x7f08f4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:17:30.614 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.613+0000 7f08fa7fc700 1 -- 172.21.15.27:0/687016216 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08f4018070 con 0x7f09040f7980 2021-11-03T17:17:30.614 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.613+0000 7f08fa7fc700 1 -- 172.21.15.27:0/687016216 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f08f4007ba0 con 0x7f09040f7980 2021-11-03T17:17:30.614 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.613+0000 7f090bd1b700 1 -- 172.21.15.27:0/687016216 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0904108460 con 0x7f09040f7980 2021-11-03T17:17:30.614 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.613+0000 7f08fa7fc700 1 -- 172.21.15.27:0/687016216 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08f401c3c0 con 0x7f09040f7980 2021-11-03T17:17:30.615 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.613+0000 7f090bd1b700 1 -- 172.21.15.27:0/687016216 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09041089c0 con 0x7f09040f7980 2021-11-03T17:17:30.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.614+0000 7f090bd1b700 1 -- 172.21.15.27:0/687016216 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0904045ad0 con 0x7f09040f7980 2021-11-03T17:17:30.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.614+0000 7f08fa7fc700 1 -- 172.21.15.27:0/687016216 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 35) v1 ==== 81997+0+0 (secure 0 0 0) 0x7f08f401d420 con 0x7f09040f7980 2021-11-03T17:17:30.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.615+0000 7f08fa7fc700 1 --2- 172.21.15.27:0/687016216 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f08f0064120 0x7f08f00665e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:17:30.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.615+0000 7f08fa7fc700 1 -- 172.21.15.27:0/687016216 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f08f4082450 con 0x7f09040f7980 2021-11-03T17:17:30.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.615+0000 7f0909ab7700 1 --2- 172.21.15.27:0/687016216 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f08f0064120 0x7f08f00665e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:17:30.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.615+0000 7f0909ab7700 1 --2- 172.21.15.27:0/687016216 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f08f0064120 0x7f08f00665e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f09040f77e0 tx=0x7f090000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:17:30.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.619+0000 7f08fa7fc700 1 -- 172.21.15.27:0/687016216 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f08f4054760 con 0x7f09040f7980 2021-11-03T17:17:30.791 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.797+0000 7f090bd1b700 1 -- 172.21.15.27:0/687016216 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f09040fefb0 con 0x7f08f0064120 2021-11-03T17:17:30.805 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.811+0000 7f08fa7fc700 1 -- 172.21.15.27:0/687016216 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f09040fefb0 con 0x7f08f0064120 2021-11-03T17:17:30.806 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:17:30.806 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (9m) 18s ago 10m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:17:30.806 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (46s) 18s ago 10m 7205k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:17:30.807 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (36s) 32s ago 9m 18.1M - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:17:30.807 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (8m) 18s ago 9m 38.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:17:30.807 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (2m) 18s ago 5m 15.1M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:17:30.807 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (5m) 18s ago 5m 3835M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:17:30.808 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (78s) 32s ago 5m 13.5M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:17:30.808 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (2m) 32s ago 5m 15.2M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:17:30.808 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (2m) 18s ago 11m 466M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:17:30.808 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (2m) 32s ago 9m 410M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:17:30.809 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (106s) 18s ago 11m 56.2M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:17:30.809 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (81s) 32s ago 9m 40.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:17:30.809 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (9m) 18s ago 9m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:17:30.809 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (9m) 32s ago 9m 18.2M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:17:30.809 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (23s) 18s ago 8m 11.8M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:17:30.810 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (8m) 18s ago 8m 847M 4096M 16.2.4 8d91d370c2b8 422be0cac0db 2021-11-03T17:17:30.810 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (8m) 18s ago 8m 610M 4096M 16.2.4 8d91d370c2b8 c211ef5551c9 2021-11-03T17:17:30.810 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (7m) 32s ago 7m 922M 4096M 16.2.4 8d91d370c2b8 b1486ec1b619 2021-11-03T17:17:30.810 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (7m) 32s ago 7m 742M 4096M 16.2.4 8d91d370c2b8 8b91230c69e5 2021-11-03T17:17:30.811 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (7m) 32s ago 7m 844M 4096M 16.2.4 8d91d370c2b8 4a54c4d6b6f5 2021-11-03T17:17:30.811 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (8m) 18s ago 9m 45.5M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:17:30.812 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.813+0000 7f090bd1b700 1 -- 172.21.15.27:0/687016216 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f08f0064120 msgr2=0x7f08f00665e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:17:30.812 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.813+0000 7f090bd1b700 1 --2- 172.21.15.27:0/687016216 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f08f0064120 0x7f08f00665e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f09040f77e0 tx=0x7f090000b040).stop 2021-11-03T17:17:30.813 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.813+0000 7f090bd1b700 1 -- 172.21.15.27:0/687016216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09040f7980 msgr2=0x7f0904107430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:17:30.813 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.813+0000 7f090bd1b700 1 --2- 172.21.15.27:0/687016216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09040f7980 0x7f0904107430 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f08f4009990 tx=0x7f08f4009b30).stop 2021-11-03T17:17:30.813 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.813+0000 7f090bd1b700 1 -- 172.21.15.27:0/687016216 shutdown_connections 2021-11-03T17:17:30.813 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.813+0000 7f090bd1b700 1 --2- 172.21.15.27:0/687016216 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f08f0064120 0x7f08f00665e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:17:30.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.813+0000 7f090bd1b700 1 --2- 172.21.15.27:0/687016216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09040f66e0 0x7f0904106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:17:30.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.813+0000 7f090bd1b700 1 --2- 172.21.15.27:0/687016216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09040f7980 0x7f0904107430 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:17:30.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.813+0000 7f090bd1b700 1 -- 172.21.15.27:0/687016216 >> 172.21.15.27:0/687016216 conn(0x7f09040f1e40 msgr2=0x7f09040fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:17:30.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.813+0000 7f090bd1b700 1 -- 172.21.15.27:0/687016216 shutdown_connections 2021-11-03T17:17:30.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.813+0000 7f090bd1b700 1 -- 172.21.15.27:0/687016216 wait complete. 2021-11-03T17:17:30.915 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.918+0000 7fe6cf250700 1 -- 172.21.15.27:0/4130649371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6c80f5430 msgr2=0x7fe6c80f5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:17:30.915 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.918+0000 7fe6cf250700 1 --2- 172.21.15.27:0/4130649371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6c80f5430 0x7fe6c80f5850 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fe6c00099e0 tx=0x7fe6c0004d10).stop 2021-11-03T17:17:30.916 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.918+0000 7fe6cf250700 1 -- 172.21.15.27:0/4130649371 shutdown_connections 2021-11-03T17:17:30.916 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.918+0000 7fe6cf250700 1 --2- 172.21.15.27:0/4130649371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6c80f66d0 0x7fe6c80f6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:17:30.916 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.918+0000 7fe6cf250700 1 --2- 172.21.15.27:0/4130649371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6c80f5430 0x7fe6c80f5850 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:17:30.916 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.918+0000 7fe6cf250700 1 -- 172.21.15.27:0/4130649371 >> 172.21.15.27:0/4130649371 conn(0x7fe6c80f0b90 msgr2=0x7fe6c80f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:17:30.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.918+0000 7fe6cf250700 1 -- 172.21.15.27:0/4130649371 shutdown_connections 2021-11-03T17:17:30.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.918+0000 7fe6cf250700 1 -- 172.21.15.27:0/4130649371 wait complete. 2021-11-03T17:17:30.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.919+0000 7fe6cf250700 1 Processor -- start 2021-11-03T17:17:30.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.919+0000 7fe6cf250700 1 -- start start 2021-11-03T17:17:30.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.919+0000 7fe6cf250700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6c80f5430 0x7fe6c8109bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:17:30.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.919+0000 7fe6cf250700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6c80f66d0 0x7fe6c810c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:17:30.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.919+0000 7fe6cf250700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c80603c0 con 0x7fe6c80f66d0 2021-11-03T17:17:30.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.919+0000 7fe6cf250700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6c8060500 con 0x7fe6c80f5430 2021-11-03T17:17:30.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.919+0000 7fe6cda4d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6c80f66d0 0x7fe6c810c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:17:30.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.919+0000 7fe6cda4d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6c80f66d0 0x7fe6c810c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56244/0 (socket says 172.21.15.27:56244) 2021-11-03T17:17:30.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.919+0000 7fe6cda4d700 1 -- 172.21.15.27:0/1250587912 learned_addr learned my addr 172.21.15.27:0/1250587912 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:17:30.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.919+0000 7fe6ce24e700 1 --2- 172.21.15.27:0/1250587912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6c80f5430 0x7fe6c8109bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:17:30.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.919+0000 7fe6cda4d700 1 -- 172.21.15.27:0/1250587912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6c80f5430 msgr2=0x7fe6c8109bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:17:30.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.919+0000 7fe6cda4d700 1 --2- 172.21.15.27:0/1250587912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6c80f5430 0x7fe6c8109bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:17:30.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.919+0000 7fe6cda4d700 1 -- 172.21.15.27:0/1250587912 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6c0009640 con 0x7fe6c80f66d0 2021-11-03T17:17:30.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.919+0000 7fe6ce24e700 1 --2- 172.21.15.27:0/1250587912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6c80f5430 0x7fe6c8109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:17:30.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.920+0000 7fe6cda4d700 1 --2- 172.21.15.27:0/1250587912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6c80f66d0 0x7fe6c810c0b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fe6bc00c530 tx=0x7fe6bc007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:17:30.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.920+0000 7fe6c6ffd700 1 -- 172.21.15.27:0/1250587912 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6bc00f650 con 0x7fe6c80f66d0 2021-11-03T17:17:30.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.920+0000 7fe6c6ffd700 1 -- 172.21.15.27:0/1250587912 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fe6bc00fc60 con 0x7fe6c80f66d0 2021-11-03T17:17:30.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.920+0000 7fe6cf250700 1 -- 172.21.15.27:0/1250587912 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6c810c660 con 0x7fe6c80f66d0 2021-11-03T17:17:30.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.920+0000 7fe6c6ffd700 1 -- 172.21.15.27:0/1250587912 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6bc012cd0 con 0x7fe6c80f66d0 2021-11-03T17:17:30.922 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.920+0000 7fe6cf250700 1 -- 172.21.15.27:0/1250587912 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6c810cb70 con 0x7fe6c80f66d0 2021-11-03T17:17:30.923 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.921+0000 7fe6cf250700 1 -- 172.21.15.27:0/1250587912 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6c8045ad0 con 0x7fe6c80f66d0 2021-11-03T17:17:30.923 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.921+0000 7fe6c6ffd700 1 -- 172.21.15.27:0/1250587912 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 35) v1 ==== 81997+0+0 (secure 0 0 0) 0x7fe6bc010070 con 0x7fe6c80f66d0 2021-11-03T17:17:30.923 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.922+0000 7fe6c6ffd700 1 --2- 172.21.15.27:0/1250587912 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe6b0064180 0x7fe6b0066640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:17:30.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.922+0000 7fe6c6ffd700 1 -- 172.21.15.27:0/1250587912 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fe6bc080750 con 0x7fe6c80f66d0 2021-11-03T17:17:30.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.922+0000 7fe6ce24e700 1 --2- 172.21.15.27:0/1250587912 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe6b0064180 0x7fe6b0066640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:17:30.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.922+0000 7fe6ce24e700 1 --2- 172.21.15.27:0/1250587912 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe6b0064180 0x7fe6b0066640 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fe6c0000f80 tx=0x7fe6c0004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:17:30.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:30.926+0000 7fe6c6ffd700 1 -- 172.21.15.27:0/1250587912 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fe6bc0529d0 con 0x7fe6c80f66d0 2021-11-03T17:17:31.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:30 smithi027 ceph-mon[97244]: poll took 1641.683 ms: 1 events 2021-11-03T17:17:31.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:30 smithi027 ceph-mon[97244]: Upgrade: Updating osd.1 2021-11-03T17:17:31.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:30 smithi027 ceph-mon[97244]: pgmap v58: 65 pgs: 9 active+clean+wait, 56 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 257 B/s wr, 0 op/s 2021-11-03T17:17:31.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:30 smithi027 ceph-mon[97244]: Deploying daemon osd.1 on smithi027 2021-11-03T17:17:31.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:17:31.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.156+0000 7fe6cf250700 1 -- 172.21.15.27:0/1250587912 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe6c80586e0 con 0x7fe6c80f66d0 2021-11-03T17:17:31.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.156+0000 7fe6c6ffd700 1 -- 172.21.15.27:0/1250587912 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+804 (secure 0 0 0) 0x7fe6bc0525c0 con 0x7fe6c80f66d0 2021-11-03T17:17:31.152 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:17:31.152 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:17:31.153 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:17:31.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:17:31.153 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:17:31.153 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:17:31.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:17:31.154 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:17:31.154 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 5, 2021-11-03T17:17:31.154 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 1 2021-11-03T17:17:31.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:17:31.155 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:17:31.155 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:17:31.155 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:17:31.155 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:17:31.156 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 9, 2021-11-03T17:17:31.156 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 5 2021-11-03T17:17:31.156 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:17:31.157 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:17:31.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.158+0000 7fe6cf250700 1 -- 172.21.15.27:0/1250587912 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe6b0064180 msgr2=0x7fe6b0066640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:17:31.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.158+0000 7fe6cf250700 1 --2- 172.21.15.27:0/1250587912 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe6b0064180 0x7fe6b0066640 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fe6c0000f80 tx=0x7fe6c0004d10).stop 2021-11-03T17:17:31.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.158+0000 7fe6cf250700 1 -- 172.21.15.27:0/1250587912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6c80f66d0 msgr2=0x7fe6c810c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:17:31.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.158+0000 7fe6cf250700 1 --2- 172.21.15.27:0/1250587912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6c80f66d0 0x7fe6c810c0b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7fe6bc00c530 tx=0x7fe6bc007e20).stop 2021-11-03T17:17:31.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.158+0000 7fe6cf250700 1 -- 172.21.15.27:0/1250587912 shutdown_connections 2021-11-03T17:17:31.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.158+0000 7fe6cf250700 1 --2- 172.21.15.27:0/1250587912 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe6b0064180 0x7fe6b0066640 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:17:31.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.158+0000 7fe6cf250700 1 --2- 172.21.15.27:0/1250587912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6c80f5430 0x7fe6c8109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:17:31.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.158+0000 7fe6cf250700 1 --2- 172.21.15.27:0/1250587912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6c80f66d0 0x7fe6c810c0b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:17:31.160 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.158+0000 7fe6cf250700 1 -- 172.21.15.27:0/1250587912 >> 172.21.15.27:0/1250587912 conn(0x7fe6c80f0b90 msgr2=0x7fe6c80fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:17:31.160 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.158+0000 7fe6cf250700 1 -- 172.21.15.27:0/1250587912 shutdown_connections 2021-11-03T17:17:31.160 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.158+0000 7fe6cf250700 1 -- 172.21.15.27:0/1250587912 wait complete. 2021-11-03T17:17:31.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:30 smithi115 ceph-mon[65453]: poll took 1641.683 ms: 1 events 2021-11-03T17:17:31.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:30 smithi115 ceph-mon[65453]: Upgrade: Updating osd.1 2021-11-03T17:17:31.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:30 smithi115 ceph-mon[65453]: pgmap v58: 65 pgs: 9 active+clean+wait, 56 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 257 B/s wr, 0 op/s 2021-11-03T17:17:31.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:30 smithi115 ceph-mon[65453]: Deploying daemon osd.1 on smithi027 2021-11-03T17:17:31.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:17:31.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.261+0000 7f1cc3919700 1 -- 172.21.15.27:0/3144304717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cbc0f66e0 msgr2=0x7f1cbc0f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:17:31.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.261+0000 7f1cc3919700 1 --2- 172.21.15.27:0/3144304717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cbc0f66e0 0x7f1cbc0f6b00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f1cac004660 tx=0x7f1cac009b30).stop 2021-11-03T17:17:31.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.261+0000 7f1cc3919700 1 -- 172.21.15.27:0/3144304717 shutdown_connections 2021-11-03T17:17:31.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.261+0000 7f1cc3919700 1 --2- 172.21.15.27:0/3144304717 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1cbc0f7980 0x7f1cbc0f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:17:31.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.261+0000 7f1cc3919700 1 --2- 172.21.15.27:0/3144304717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cbc0f66e0 0x7f1cbc0f6b00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:17:31.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.261+0000 7f1cc3919700 1 -- 172.21.15.27:0/3144304717 >> 172.21.15.27:0/3144304717 conn(0x7f1cbc0f1e40 msgr2=0x7f1cbc0f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:17:31.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.261+0000 7f1cc3919700 1 -- 172.21.15.27:0/3144304717 shutdown_connections 2021-11-03T17:17:31.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.261+0000 7f1cc3919700 1 -- 172.21.15.27:0/3144304717 wait complete. 2021-11-03T17:17:31.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.261+0000 7f1cc3919700 1 Processor -- start 2021-11-03T17:17:31.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.262+0000 7f1cc3919700 1 -- start start 2021-11-03T17:17:31.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.262+0000 7f1cc3919700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cbc0f66e0 0x7f1cbc106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:17:31.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.262+0000 7f1cc3919700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1cbc0f7980 0x7f1cbc107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:17:31.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.262+0000 7f1cc3919700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc107a10 con 0x7f1cbc0f66e0 2021-11-03T17:17:31.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.262+0000 7f1cc3919700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc107b50 con 0x7f1cbc0f7980 2021-11-03T17:17:31.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.262+0000 7f1cc16b5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cbc0f66e0 0x7f1cbc106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:17:31.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.262+0000 7f1cc16b5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cbc0f66e0 0x7f1cbc106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56252/0 (socket says 172.21.15.27:56252) 2021-11-03T17:17:31.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.262+0000 7f1cc16b5700 1 -- 172.21.15.27:0/4142006104 learned_addr learned my addr 172.21.15.27:0/4142006104 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:17:31.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.262+0000 7f1cc0eb4700 1 --2- 172.21.15.27:0/4142006104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1cbc0f7980 0x7f1cbc107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:17:31.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.262+0000 7f1cc16b5700 1 -- 172.21.15.27:0/4142006104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1cbc0f7980 msgr2=0x7f1cbc107460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:17:31.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.262+0000 7f1cc16b5700 1 --2- 172.21.15.27:0/4142006104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1cbc0f7980 0x7f1cbc107460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:17:31.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.262+0000 7f1cc16b5700 1 -- 172.21.15.27:0/4142006104 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cac005040 con 0x7f1cbc0f66e0 2021-11-03T17:17:31.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.262+0000 7f1cc16b5700 1 --2- 172.21.15.27:0/4142006104 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cbc0f66e0 0x7f1cbc106f70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f1cac004f60 tx=0x7f1cac009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:17:31.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.262+0000 7f1cb1ffb700 1 -- 172.21.15.27:0/4142006104 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cac018070 con 0x7f1cbc0f66e0 2021-11-03T17:17:31.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.262+0000 7f1cb1ffb700 1 -- 172.21.15.27:0/4142006104 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f1cac009770 con 0x7f1cbc0f66e0 2021-11-03T17:17:31.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.262+0000 7f1cc3919700 1 -- 172.21.15.27:0/4142006104 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cbc108490 con 0x7f1cbc0f66e0 2021-11-03T17:17:31.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.263+0000 7f1cb1ffb700 1 -- 172.21.15.27:0/4142006104 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cac01c3c0 con 0x7f1cbc0f66e0 2021-11-03T17:17:31.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.263+0000 7f1cc3919700 1 -- 172.21.15.27:0/4142006104 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cbc1089f0 con 0x7f1cbc0f66e0 2021-11-03T17:17:31.265 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.264+0000 7f1cc3919700 1 -- 172.21.15.27:0/4142006104 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cbc045ad0 con 0x7f1cbc0f66e0 2021-11-03T17:17:31.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.264+0000 7f1cb1ffb700 1 -- 172.21.15.27:0/4142006104 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 35) v1 ==== 81997+0+0 (secure 0 0 0) 0x7f1cac01d420 con 0x7f1cbc0f66e0 2021-11-03T17:17:31.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.264+0000 7f1cb1ffb700 1 --2- 172.21.15.27:0/4142006104 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1ca8064120 0x7f1ca80665e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:17:31.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.264+0000 7f1cb1ffb700 1 -- 172.21.15.27:0/4142006104 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f1cac082600 con 0x7f1cbc0f66e0 2021-11-03T17:17:31.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.265+0000 7f1cc0eb4700 1 --2- 172.21.15.27:0/4142006104 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1ca8064120 0x7f1ca80665e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:17:31.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.265+0000 7f1cc0eb4700 1 --2- 172.21.15.27:0/4142006104 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1ca8064120 0x7f1ca80665e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f1cb8007fd0 tx=0x7f1cb8009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:17:31.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.268+0000 7f1cb1ffb700 1 -- 172.21.15.27:0/4142006104 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f1cac054a30 con 0x7f1cbc0f66e0 2021-11-03T17:17:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.482+0000 7f1cc3919700 1 -- 172.21.15.27:0/4142006104 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1cbc0feea0 con 0x7f1cbc0f66e0 2021-11-03T17:17:31.478 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.483+0000 7f1cb1ffb700 1 -- 172.21.15.27:0/4142006104 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1795 (secure 0 0 0) 0x7f1cac054620 con 0x7f1cbc0f66e0 2021-11-03T17:17:31.479 INFO:teuthology.orchestra.run.smithi027.stdout:e28 2021-11-03T17:17:31.479 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:17:31.479 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:17:31.479 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:17:31.480 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:17:31.480 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:17:31.480 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:17:31.480 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 28 2021-11-03T17:17:31.481 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:17:31.481 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:17:31.481 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:16:13.536828+0000 2021-11-03T17:17:31.481 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:17:31.482 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:17:31.482 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:17:31.482 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:17:31.482 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:17:31.482 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:17:31.483 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:17:31.483 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:17:31.483 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:17:31.483 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 2 2021-11-03T17:17:31.484 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:17:31.484 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:17:31.484 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:17:31.484 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:17:31.484 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:17:31.485 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:17:31.485 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:17:31.485 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:17:31.485 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:17:31.486 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:17:31.486 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 3 export targets 0 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:17:31.486 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:17:31.486 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:17:31.487 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:17:31.487 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:17:31.487 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:17:31.487 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:17:31.488 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:17:31.488 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.485+0000 7f1cc3919700 1 -- 172.21.15.27:0/4142006104 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1ca8064120 msgr2=0x7f1ca80665e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:17:31.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.485+0000 7f1cc3919700 1 --2- 172.21.15.27:0/4142006104 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1ca8064120 0x7f1ca80665e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f1cb8007fd0 tx=0x7f1cb8009040).stop 2021-11-03T17:17:31.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.485+0000 7f1cc3919700 1 -- 172.21.15.27:0/4142006104 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cbc0f66e0 msgr2=0x7f1cbc106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:17:31.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.485+0000 7f1cc3919700 1 --2- 172.21.15.27:0/4142006104 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cbc0f66e0 0x7f1cbc106f70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f1cac004f60 tx=0x7f1cac009b30).stop 2021-11-03T17:17:31.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.485+0000 7f1cc3919700 1 -- 172.21.15.27:0/4142006104 shutdown_connections 2021-11-03T17:17:31.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.485+0000 7f1cc3919700 1 --2- 172.21.15.27:0/4142006104 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1ca8064120 0x7f1ca80665e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:17:31.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.485+0000 7f1cc3919700 1 --2- 172.21.15.27:0/4142006104 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cbc0f66e0 0x7f1cbc106f70 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:17:31.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.485+0000 7f1cc3919700 1 --2- 172.21.15.27:0/4142006104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1cbc0f7980 0x7f1cbc107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:17:31.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.485+0000 7f1cc3919700 1 -- 172.21.15.27:0/4142006104 >> 172.21.15.27:0/4142006104 conn(0x7f1cbc0f1e40 msgr2=0x7f1cbc0fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:17:31.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.485+0000 7f1cc3919700 1 -- 172.21.15.27:0/4142006104 shutdown_connections 2021-11-03T17:17:31.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:17:31.485+0000 7f1cc3919700 1 -- 172.21.15.27:0/4142006104 wait complete. 2021-11-03T17:17:31.491 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 28 2021-11-03T17:17:32.111 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:31 smithi027 ceph-mon[97244]: from='client.34186 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:17:32.112 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:31 smithi027 ceph-mon[97244]: from='client.34190 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:17:32.112 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:31 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1250587912' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:17:32.112 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:31 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/4142006104' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:17:32.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:31 smithi115 ceph-mon[65453]: from='client.34186 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:17:32.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:31 smithi115 ceph-mon[65453]: from='client.34190 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:17:32.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:31 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1250587912' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:17:32.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:31 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/4142006104' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:17:32.818 INFO:journalctl@ceph.osd.1.smithi027.stdout:Nov 03 17:17:32 smithi027 systemd[1]: Stopping Ceph osd.1 for 32ab977e-3cc8-11ec-8c28-001a4aab830c... 2021-11-03T17:17:33.071 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:32 smithi027 ceph-mon[97244]: pgmap v59: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2021-11-03T17:17:33.072 INFO:journalctl@ceph.osd.1.smithi027.stdout:Nov 03 17:17:32 smithi027 conmon[56703]: debug 2021-11-03T17:17:32.822+0000 7fb3f7296700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2021-11-03T17:17:33.072 INFO:journalctl@ceph.osd.1.smithi027.stdout:Nov 03 17:17:32 smithi027 conmon[56703]: debug 2021-11-03T17:17:32.822+0000 7fb3f7296700 -1 osd.1 58 *** Got signal Terminated *** 2021-11-03T17:17:33.072 INFO:journalctl@ceph.osd.1.smithi027.stdout:Nov 03 17:17:32 smithi027 conmon[56703]: debug 2021-11-03T17:17:32.822+0000 7fb3f7296700 -1 osd.1 58 *** Immediate shutdown (osd_fast_shutdown=true) *** 2021-11-03T17:17:33.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:32 smithi115 ceph-mon[65453]: pgmap v59: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2021-11-03T17:17:33.519 INFO:journalctl@ceph.osd.1.smithi027.stdout:Nov 03 17:17:33 smithi027 bash[108618]: 422be0cac0db5a7301297c48885ace3e5eeff72c9c389dfc5dff22de617f6fd2 2021-11-03T17:17:33.519 INFO:journalctl@ceph.osd.1.smithi027.stdout:Nov 03 17:17:33 smithi027 bash[108618]: Error: no container with name or ID ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c-osd-1 found: no such container 2021-11-03T17:17:34.018 INFO:journalctl@ceph.osd.1.smithi027.stdout:Nov 03 17:17:33 smithi027 bash[108618]: Error: container 422be0cac0db5a7301297c48885ace3e5eeff72c9c389dfc5dff22de617f6fd2 does not exist in database: no such container 2021-11-03T17:17:34.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:33 smithi027 ceph-mon[97244]: osd.1 reported immediately failed by osd.3 2021-11-03T17:17:34.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:33 smithi027 ceph-mon[97244]: osd.1 failed (root=default,host=smithi027) (connection refused reported by osd.3) 2021-11-03T17:17:34.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:33 smithi027 ceph-mon[97244]: osd.1 reported immediately failed by osd.3 2021-11-03T17:17:34.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:33 smithi027 ceph-mon[97244]: osd.1 reported immediately failed by osd.0 2021-11-03T17:17:34.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:33 smithi027 ceph-mon[97244]: osd.1 reported immediately failed by osd.4 2021-11-03T17:17:34.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:33 smithi027 ceph-mon[97244]: osd.1 reported immediately failed by osd.5 2021-11-03T17:17:34.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:33 smithi027 ceph-mon[97244]: osd.1 reported immediately failed by osd.2 2021-11-03T17:17:34.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:33 smithi027 ceph-mon[97244]: osd.1 reported immediately failed by osd.0 2021-11-03T17:17:34.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:33 smithi027 ceph-mon[97244]: osd.1 reported immediately failed by osd.4 2021-11-03T17:17:34.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:33 smithi027 ceph-mon[97244]: osd.1 reported immediately failed by osd.5 2021-11-03T17:17:34.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:33 smithi027 ceph-mon[97244]: osd.1 reported immediately failed by osd.2 2021-11-03T17:17:34.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:33 smithi027 ceph-mon[97244]: osd.1 reported immediately failed by osd.4 2021-11-03T17:17:34.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:33 smithi027 ceph-mon[97244]: osd.1 reported immediately failed by osd.4 2021-11-03T17:17:34.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:33 smithi027 ceph-mon[97244]: osd.1 reported immediately failed by osd.5 2021-11-03T17:17:34.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:33 smithi027 ceph-mon[97244]: osd.1 reported immediately failed by osd.2 2021-11-03T17:17:34.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:33 smithi027 ceph-mon[97244]: osd.1 reported immediately failed by osd.3 2021-11-03T17:17:34.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:33 smithi027 ceph-mon[97244]: osd.1 reported immediately failed by osd.0 2021-11-03T17:17:34.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:33 smithi027 ceph-mon[97244]: osd.1 reported immediately failed by osd.5 2021-11-03T17:17:34.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:33 smithi027 ceph-mon[97244]: osd.1 reported immediately failed by osd.2 2021-11-03T17:17:34.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:33 smithi027 ceph-mon[97244]: osd.1 reported immediately failed by osd.3 2021-11-03T17:17:34.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:33 smithi027 ceph-mon[97244]: osd.1 reported immediately failed by osd.0 2021-11-03T17:17:34.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:33 smithi115 ceph-mon[65453]: osd.1 reported immediately failed by osd.3 2021-11-03T17:17:34.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:33 smithi115 ceph-mon[65453]: osd.1 failed (root=default,host=smithi027) (connection refused reported by osd.3) 2021-11-03T17:17:34.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:33 smithi115 ceph-mon[65453]: osd.1 reported immediately failed by osd.3 2021-11-03T17:17:34.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:33 smithi115 ceph-mon[65453]: osd.1 reported immediately failed by osd.0 2021-11-03T17:17:34.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:33 smithi115 ceph-mon[65453]: osd.1 reported immediately failed by osd.4 2021-11-03T17:17:34.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:33 smithi115 ceph-mon[65453]: osd.1 reported immediately failed by osd.5 2021-11-03T17:17:34.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:33 smithi115 ceph-mon[65453]: osd.1 reported immediately failed by osd.2 2021-11-03T17:17:34.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:33 smithi115 ceph-mon[65453]: osd.1 reported immediately failed by osd.0 2021-11-03T17:17:34.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:33 smithi115 ceph-mon[65453]: osd.1 reported immediately failed by osd.4 2021-11-03T17:17:34.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:33 smithi115 ceph-mon[65453]: osd.1 reported immediately failed by osd.5 2021-11-03T17:17:34.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:33 smithi115 ceph-mon[65453]: osd.1 reported immediately failed by osd.2 2021-11-03T17:17:34.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:33 smithi115 ceph-mon[65453]: osd.1 reported immediately failed by osd.4 2021-11-03T17:17:34.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:33 smithi115 ceph-mon[65453]: osd.1 reported immediately failed by osd.4 2021-11-03T17:17:34.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:33 smithi115 ceph-mon[65453]: osd.1 reported immediately failed by osd.5 2021-11-03T17:17:34.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:33 smithi115 ceph-mon[65453]: osd.1 reported immediately failed by osd.2 2021-11-03T17:17:34.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:33 smithi115 ceph-mon[65453]: osd.1 reported immediately failed by osd.3 2021-11-03T17:17:34.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:33 smithi115 ceph-mon[65453]: osd.1 reported immediately failed by osd.0 2021-11-03T17:17:34.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:33 smithi115 ceph-mon[65453]: osd.1 reported immediately failed by osd.5 2021-11-03T17:17:34.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:33 smithi115 ceph-mon[65453]: osd.1 reported immediately failed by osd.2 2021-11-03T17:17:34.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:33 smithi115 ceph-mon[65453]: osd.1 reported immediately failed by osd.3 2021-11-03T17:17:34.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:33 smithi115 ceph-mon[65453]: osd.1 reported immediately failed by osd.0 2021-11-03T17:17:35.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:34 smithi115 ceph-mon[65453]: pgmap v60: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2021-11-03T17:17:35.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:34 smithi115 ceph-mon[65453]: Health check failed: 1 osds down (OSD_DOWN) 2021-11-03T17:17:35.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:34 smithi115 ceph-mon[65453]: osdmap e59: 6 total, 5 up, 6 in 2021-11-03T17:17:35.245 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:34 smithi027 ceph-mon[97244]: pgmap v60: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2021-11-03T17:17:35.245 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:34 smithi027 ceph-mon[97244]: Health check failed: 1 osds down (OSD_DOWN) 2021-11-03T17:17:35.245 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:34 smithi027 ceph-mon[97244]: osdmap e59: 6 total, 5 up, 6 in 2021-11-03T17:17:35.519 INFO:journalctl@ceph.osd.1.smithi027.stdout:Nov 03 17:17:35 smithi027 systemd[1]: ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c@osd.1.service: Succeeded. 2021-11-03T17:17:35.519 INFO:journalctl@ceph.osd.1.smithi027.stdout:Nov 03 17:17:35 smithi027 systemd[1]: Stopped Ceph osd.1 for 32ab977e-3cc8-11ec-8c28-001a4aab830c. 2021-11-03T17:17:35.519 INFO:journalctl@ceph.osd.1.smithi027.stdout:Nov 03 17:17:35 smithi027 systemd[1]: Starting Ceph osd.1 for 32ab977e-3cc8-11ec-8c28-001a4aab830c... 2021-11-03T17:17:36.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:35 smithi115 ceph-mon[65453]: osdmap e60: 6 total, 5 up, 6 in 2021-11-03T17:17:36.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:35 smithi027 ceph-mon[97244]: osdmap e60: 6 total, 5 up, 6 in 2021-11-03T17:17:36.769 INFO:journalctl@ceph.osd.1.smithi027.stdout:Nov 03 17:17:36 smithi027 bash[109111]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2021-11-03T17:17:36.769 INFO:journalctl@ceph.osd.1.smithi027.stdout:Nov 03 17:17:36 smithi027 bash[109111]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --dev /dev/mapper/vg_nvme-lv_3 --path /var/lib/ceph/osd/ceph-1 --no-mon-config 2021-11-03T17:17:36.769 INFO:journalctl@ceph.osd.1.smithi027.stdout:Nov 03 17:17:36 smithi027 bash[109111]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2021-11-03T17:17:36.770 INFO:journalctl@ceph.osd.1.smithi027.stdout:Nov 03 17:17:36 smithi027 bash[109111]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2021-11-03T17:17:36.770 INFO:journalctl@ceph.osd.1.smithi027.stdout:Nov 03 17:17:36 smithi027 bash[109111]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2021-11-03T17:17:36.770 INFO:journalctl@ceph.osd.1.smithi027.stdout:Nov 03 17:17:36 smithi027 bash[109111]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2021-11-03T17:17:36.770 INFO:journalctl@ceph.osd.1.smithi027.stdout:Nov 03 17:17:36 smithi027 bash[109111]: --> ceph-volume raw activate successful for osd ID: 1 2021-11-03T17:17:37.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:36 smithi115 ceph-mon[65453]: pgmap v63: 65 pgs: 12 stale+active+clean, 53 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:17:37.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:36 smithi027 ceph-mon[97244]: pgmap v63: 65 pgs: 12 stale+active+clean, 53 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:17:38.018 INFO:journalctl@ceph.osd.1.smithi027.stdout:Nov 03 17:17:37 smithi027 ceph-osd[109634]: -- 172.21.15.27:0/3443688432 <== mon.0 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ae14a80340 con 0x55ae14a46c00 2021-11-03T17:17:38.019 INFO:journalctl@ceph.osd.1.smithi027.stdout:Nov 03 17:17:37 smithi027 bash[109111]: cf75166ddccfee1be967e32510a991defa12c1ecb186eb37f7784068e153ac10 2021-11-03T17:17:38.019 INFO:journalctl@ceph.osd.1.smithi027.stdout:Nov 03 17:17:37 smithi027 systemd[1]: Started Ceph osd.1 for 32ab977e-3cc8-11ec-8c28-001a4aab830c. 2021-11-03T17:17:38.518 INFO:journalctl@ceph.osd.1.smithi027.stdout:Nov 03 17:17:38 smithi027 conmon[109611]: 2021-11-03T17:17:38.148+0000 7f0ae77310c0 -1 Falling back to public interface 2021-11-03T17:17:38.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:38 smithi115 ceph-mon[65453]: poll took 7667.297 ms: 1 events 2021-11-03T17:17:38.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:17:38.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:17:38.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:38 smithi115 ceph-mon[65453]: pgmap v64: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 9892/52479 objects degraded (18.849%) 2021-11-03T17:17:39.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:38 smithi027 ceph-mon[97244]: poll took 7667.297 ms: 1 events 2021-11-03T17:17:39.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:17:39.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:17:39.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:38 smithi027 ceph-mon[97244]: pgmap v64: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 9892/52479 objects degraded (18.849%) 2021-11-03T17:17:39.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:39 smithi115 ceph-mon[65453]: Health check failed: Degraded data redundancy: 9892/52479 objects degraded (18.849%), 34 pgs degraded (PG_DEGRADED) 2021-11-03T17:17:40.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:39 smithi027 ceph-mon[97244]: Health check failed: Degraded data redundancy: 9892/52479 objects degraded (18.849%), 34 pgs degraded (PG_DEGRADED) 2021-11-03T17:17:40.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:40 smithi115 ceph-mon[65453]: pgmap v65: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 9892/52479 objects degraded (18.849%) 2021-11-03T17:17:40.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:17:41.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:40 smithi027 ceph-mon[97244]: pgmap v65: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 9892/52479 objects degraded (18.849%) 2021-11-03T17:17:41.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:17:43.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:42 smithi027 ceph-mon[97244]: pgmap v66: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 9892/52479 objects degraded (18.849%) 2021-11-03T17:17:43.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:17:43.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:17:43.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:17:43.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:17:43.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:17:43.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:17:43.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2021-11-03T17:17:43.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:42 smithi115 ceph-mon[65453]: pgmap v66: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 9892/52479 objects degraded (18.849%) 2021-11-03T17:17:43.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:17:43.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:17:43.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:17:43.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:17:43.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:17:43.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:17:43.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2021-11-03T17:17:44.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:43 smithi115 ceph-mon[65453]: poll took 5005.845 ms: 1 events 2021-11-03T17:17:44.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:43 smithi115 ceph-mon[65453]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2021-11-03T17:17:44.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:43 smithi115 ceph-mon[65453]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2021-11-03T17:17:44.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:43 smithi027 ceph-mon[97244]: poll took 5005.845 ms: 1 events 2021-11-03T17:17:44.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:43 smithi027 ceph-mon[97244]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2021-11-03T17:17:44.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:43 smithi027 ceph-mon[97244]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2021-11-03T17:17:45.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:44 smithi115 ceph-mon[65453]: pgmap v67: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 9892/52479 objects degraded (18.849%) 2021-11-03T17:17:45.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:44 smithi027 ceph-mon[97244]: pgmap v67: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 9892/52479 objects degraded (18.849%) 2021-11-03T17:17:45.769 INFO:journalctl@ceph.osd.1.smithi027.stdout:Nov 03 17:17:45 smithi027 conmon[109611]: 2021-11-03T17:17:45.420+0000 7f0ae77310c0 -1 osd.1 58 log_to_monitors {default=true} 2021-11-03T17:17:46.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:45 smithi115 ceph-mon[65453]: from='osd.1 [v2:172.21.15.27:6810/1432450709,v1:172.21.15.27:6811/1432450709]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2021-11-03T17:17:46.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:45 smithi027 ceph-mon[97244]: from='osd.1 [v2:172.21.15.27:6810/1432450709,v1:172.21.15.27:6811/1432450709]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2021-11-03T17:17:46.269 INFO:journalctl@ceph.osd.1.smithi027.stdout:Nov 03 17:17:45 smithi027 conmon[109611]: 2021-11-03T17:17:45.845+0000 7f0ad786e700 -1 osd.1 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2021-11-03T17:17:47.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:46 smithi115 ceph-mon[65453]: pgmap v68: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 9892/52479 objects degraded (18.849%) 2021-11-03T17:17:47.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:46 smithi115 ceph-mon[65453]: from='osd.1 [v2:172.21.15.27:6810/1432450709,v1:172.21.15.27:6811/1432450709]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2021-11-03T17:17:47.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:46 smithi115 ceph-mon[65453]: osdmap e61: 6 total, 5 up, 6 in 2021-11-03T17:17:47.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:46 smithi115 ceph-mon[65453]: from='osd.1 [v2:172.21.15.27:6810/1432450709,v1:172.21.15.27:6811/1432450709]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2021-11-03T17:17:47.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:46 smithi027 ceph-mon[97244]: pgmap v68: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 9892/52479 objects degraded (18.849%) 2021-11-03T17:17:47.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:46 smithi027 ceph-mon[97244]: from='osd.1 [v2:172.21.15.27:6810/1432450709,v1:172.21.15.27:6811/1432450709]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2021-11-03T17:17:47.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:46 smithi027 ceph-mon[97244]: osdmap e61: 6 total, 5 up, 6 in 2021-11-03T17:17:47.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:46 smithi027 ceph-mon[97244]: from='osd.1 [v2:172.21.15.27:6810/1432450709,v1:172.21.15.27:6811/1432450709]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2021-11-03T17:17:48.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:47 smithi115 ceph-mon[65453]: Health check cleared: OSD_DOWN (was: 1 osds down) 2021-11-03T17:17:48.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:47 smithi115 ceph-mon[65453]: osd.1 [v2:172.21.15.27:6810/1432450709,v1:172.21.15.27:6811/1432450709] boot 2021-11-03T17:17:48.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:47 smithi115 ceph-mon[65453]: osdmap e62: 6 total, 6 up, 6 in 2021-11-03T17:17:48.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-11-03T17:17:48.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:47 smithi027 ceph-mon[97244]: Health check cleared: OSD_DOWN (was: 1 osds down) 2021-11-03T17:17:48.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:47 smithi027 ceph-mon[97244]: osd.1 [v2:172.21.15.27:6810/1432450709,v1:172.21.15.27:6811/1432450709] boot 2021-11-03T17:17:48.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:47 smithi027 ceph-mon[97244]: osdmap e62: 6 total, 6 up, 6 in 2021-11-03T17:17:48.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2021-11-03T17:17:49.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:48 smithi115 ceph-mon[65453]: pgmap v71: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s wr, 0 op/s; 9892/52479 objects degraded (18.849%) 2021-11-03T17:17:49.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:48 smithi115 ceph-mon[65453]: osdmap e63: 6 total, 6 up, 6 in 2021-11-03T17:17:49.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:48 smithi027 ceph-mon[97244]: pgmap v71: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s wr, 0 op/s; 9892/52479 objects degraded (18.849%) 2021-11-03T17:17:49.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:48 smithi027 ceph-mon[97244]: osdmap e63: 6 total, 6 up, 6 in 2021-11-03T17:17:50.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:50 smithi115 ceph-mon[65453]: pgmap v73: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s wr, 0 op/s; 9892/52479 objects degraded (18.849%) 2021-11-03T17:17:50.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:50 smithi027 ceph-mon[97244]: pgmap v73: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s wr, 0 op/s; 9892/52479 objects degraded (18.849%) 2021-11-03T17:17:52.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:51 smithi115 ceph-mon[65453]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 9892/52479 objects degraded (18.849%), 34 pgs degraded) 2021-11-03T17:17:52.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:51 smithi027 ceph-mon[97244]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 9892/52479 objects degraded (18.849%), 34 pgs degraded) 2021-11-03T17:17:53.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:52 smithi115 ceph-mon[65453]: pgmap v74: 65 pgs: 12 active+clean+wait, 53 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s wr, 0 op/s; 0 B/s, 15 keys/s, 0 objects/s recovering 2021-11-03T17:17:53.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:52 smithi027 ceph-mon[97244]: pgmap v74: 65 pgs: 12 active+clean+wait, 53 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s wr, 0 op/s; 0 B/s, 15 keys/s, 0 objects/s recovering 2021-11-03T17:17:55.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:54 smithi115 ceph-mon[65453]: pgmap v75: 65 pgs: 12 active+clean+wait, 53 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s wr, 0 op/s; 0 B/s, 12 keys/s, 0 objects/s recovering 2021-11-03T17:17:55.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:54 smithi027 ceph-mon[97244]: pgmap v75: 65 pgs: 12 active+clean+wait, 53 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s wr, 0 op/s; 0 B/s, 12 keys/s, 0 objects/s recovering 2021-11-03T17:17:56.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:17:56.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:17:56.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:17:56.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:56 smithi115 ceph-mon[65453]: pgmap v76: 65 pgs: 12 active+clean+wait, 53 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s, 10 keys/s, 0 objects/s recovering 2021-11-03T17:17:57.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:17:57.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:17:57.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:17:57.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:56 smithi027 ceph-mon[97244]: pgmap v76: 65 pgs: 12 active+clean+wait, 53 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s, 10 keys/s, 0 objects/s recovering 2021-11-03T17:17:58.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:58 smithi027 ceph-mon[97244]: mgrmap e36: smithi027.ufhhvb(active, since 2m), standbys: smithi115.tcyodz 2021-11-03T17:17:58.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:58 smithi027 ceph-mon[97244]: pgmap v77: 65 pgs: 12 active+clean+wait, 53 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s, 9 keys/s, 0 objects/s recovering 2021-11-03T17:17:58.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2021-11-03T17:17:58.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:58 smithi027 ceph-mon[97244]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2021-11-03T17:17:58.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:58 smithi027 ceph-mon[97244]: Upgrade: osd.2 is safe to restart 2021-11-03T17:17:58.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:17:58 smithi027 ceph-mon[97244]: poll took 15035.872 ms: 1 events 2021-11-03T17:17:58.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:58 smithi115 ceph-mon[65453]: mgrmap e36: smithi027.ufhhvb(active, since 2m), standbys: smithi115.tcyodz 2021-11-03T17:17:58.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:58 smithi115 ceph-mon[65453]: pgmap v77: 65 pgs: 12 active+clean+wait, 53 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s, 9 keys/s, 0 objects/s recovering 2021-11-03T17:17:58.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2021-11-03T17:17:58.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:58 smithi115 ceph-mon[65453]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2021-11-03T17:17:58.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:58 smithi115 ceph-mon[65453]: Upgrade: osd.2 is safe to restart 2021-11-03T17:17:58.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:17:58 smithi115 ceph-mon[65453]: poll took 15035.872 ms: 1 events 2021-11-03T17:18:00.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:00 smithi027 ceph-mon[97244]: poll took 1732.142 ms: 1 events 2021-11-03T17:18:00.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:00 smithi027 ceph-mon[97244]: Upgrade: Updating osd.2 2021-11-03T17:18:00.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:18:00.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:00 smithi027 ceph-mon[97244]: Deploying daemon osd.2 on smithi027 2021-11-03T17:18:00.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:00 smithi027 ceph-mon[97244]: pgmap v78: 65 pgs: 12 active+clean+wait, 53 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s, 8 keys/s, 0 objects/s recovering 2021-11-03T17:18:00.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:00 smithi115 ceph-mon[65453]: poll took 1732.142 ms: 1 events 2021-11-03T17:18:00.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:00 smithi115 ceph-mon[65453]: Upgrade: Updating osd.2 2021-11-03T17:18:00.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:18:00.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:00 smithi115 ceph-mon[65453]: Deploying daemon osd.2 on smithi027 2021-11-03T17:18:00.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:00 smithi115 ceph-mon[65453]: pgmap v78: 65 pgs: 12 active+clean+wait, 53 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s, 8 keys/s, 0 objects/s recovering 2021-11-03T17:18:01.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.592+0000 7f471b9fe700 1 -- 172.21.15.27:0/1700327647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47140f66e0 msgr2=0x7f47140f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:18:01.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.592+0000 7f471b9fe700 1 --2- 172.21.15.27:0/1700327647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47140f66e0 0x7f47140f6b00 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f4710004660 tx=0x7f4710009b30).stop 2021-11-03T17:18:01.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.592+0000 7f471b9fe700 1 -- 172.21.15.27:0/1700327647 shutdown_connections 2021-11-03T17:18:01.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.592+0000 7f471b9fe700 1 --2- 172.21.15.27:0/1700327647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47140f7980 0x7f47140f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:01.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.592+0000 7f471b9fe700 1 --2- 172.21.15.27:0/1700327647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47140f66e0 0x7f47140f6b00 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:01.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.592+0000 7f471b9fe700 1 -- 172.21.15.27:0/1700327647 >> 172.21.15.27:0/1700327647 conn(0x7f47140f1e40 msgr2=0x7f47140f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:18:01.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.592+0000 7f471b9fe700 1 -- 172.21.15.27:0/1700327647 shutdown_connections 2021-11-03T17:18:01.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.592+0000 7f471b9fe700 1 -- 172.21.15.27:0/1700327647 wait complete. 2021-11-03T17:18:01.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.593+0000 7f471b9fe700 1 Processor -- start 2021-11-03T17:18:01.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.593+0000 7f471b9fe700 1 -- start start 2021-11-03T17:18:01.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.593+0000 7f471b9fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47140f66e0 0x7f4714106fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:18:01.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.593+0000 7f471b9fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47140f7980 0x7f47141074a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:18:01.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.593+0000 7f471b9fe700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4714107a50 con 0x7f47140f7980 2021-11-03T17:18:01.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.593+0000 7f471b9fe700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4714107b90 con 0x7f47140f66e0 2021-11-03T17:18:01.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.593+0000 7f4718f99700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47140f7980 0x7f47141074a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:18:01.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.593+0000 7f4718f99700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47140f7980 0x7f47141074a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56342/0 (socket says 172.21.15.27:56342) 2021-11-03T17:18:01.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.593+0000 7f4718f99700 1 -- 172.21.15.27:0/2568853233 learned_addr learned my addr 172.21.15.27:0/2568853233 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:18:01.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.593+0000 7f471979a700 1 --2- 172.21.15.27:0/2568853233 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47140f66e0 0x7f4714106fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:18:01.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.593+0000 7f4718f99700 1 -- 172.21.15.27:0/2568853233 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47140f66e0 msgr2=0x7f4714106fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:18:01.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.593+0000 7f4718f99700 1 --2- 172.21.15.27:0/2568853233 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47140f66e0 0x7f4714106fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:01.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.593+0000 7f4718f99700 1 -- 172.21.15.27:0/2568853233 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4710005040 con 0x7f47140f7980 2021-11-03T17:18:01.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.593+0000 7f4718f99700 1 --2- 172.21.15.27:0/2568853233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47140f7980 0x7f47141074a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f470400c530 tx=0x7f4704007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:18:01.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.594+0000 7f4709ffb700 1 -- 172.21.15.27:0/2568853233 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f470400f650 con 0x7f47140f7980 2021-11-03T17:18:01.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.594+0000 7f4709ffb700 1 -- 172.21.15.27:0/2568853233 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f470400fc60 con 0x7f47140f7980 2021-11-03T17:18:01.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.594+0000 7f471b9fe700 1 -- 172.21.15.27:0/2568853233 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4714108530 con 0x7f47140f7980 2021-11-03T17:18:01.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.594+0000 7f4709ffb700 1 -- 172.21.15.27:0/2568853233 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4704012cd0 con 0x7f47140f7980 2021-11-03T17:18:01.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.594+0000 7f471b9fe700 1 -- 172.21.15.27:0/2568853233 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4714108af0 con 0x7f47140f7980 2021-11-03T17:18:01.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.595+0000 7f471b9fe700 1 -- 172.21.15.27:0/2568853233 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4714045ad0 con 0x7f47140f7980 2021-11-03T17:18:01.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.595+0000 7f4709ffb700 1 -- 172.21.15.27:0/2568853233 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f4704010070 con 0x7f47140f7980 2021-11-03T17:18:01.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.595+0000 7f4709ffb700 1 --2- 172.21.15.27:0/2568853233 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f47000640b0 0x7f4700066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:18:01.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.596+0000 7f4709ffb700 1 -- 172.21.15.27:0/2568853233 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f47040806e0 con 0x7f47140f7980 2021-11-03T17:18:01.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.596+0000 7f471979a700 1 --2- 172.21.15.27:0/2568853233 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f47000640b0 0x7f4700066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:18:01.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.596+0000 7f471979a700 1 --2- 172.21.15.27:0/2568853233 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f47000640b0 0x7f4700066570 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f4710009da0 tx=0x7f4710009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:18:01.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.600+0000 7f4709ffb700 1 -- 172.21.15.27:0/2568853233 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f4704052950 con 0x7f47140f7980 2021-11-03T17:18:01.764 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:18:01.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.781+0000 7f471b9fe700 1 -- 172.21.15.27:0/2568853233 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4714108d10 con 0x7f47000640b0 2021-11-03T17:18:01.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.782+0000 7f4709ffb700 1 -- 172.21.15.27:0/2568853233 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+320 (secure 0 0 0) 0x7f4714108d10 con 0x7f47000640b0 2021-11-03T17:18:01.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.784+0000 7f471b9fe700 1 -- 172.21.15.27:0/2568853233 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f47000640b0 msgr2=0x7f4700066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:18:01.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.784+0000 7f471b9fe700 1 --2- 172.21.15.27:0/2568853233 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f47000640b0 0x7f4700066570 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f4710009da0 tx=0x7f4710009b30).stop 2021-11-03T17:18:01.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.784+0000 7f471b9fe700 1 -- 172.21.15.27:0/2568853233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47140f7980 msgr2=0x7f47141074a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:18:01.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.784+0000 7f471b9fe700 1 --2- 172.21.15.27:0/2568853233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47140f7980 0x7f47141074a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f470400c530 tx=0x7f4704007e20).stop 2021-11-03T17:18:01.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.785+0000 7f471b9fe700 1 -- 172.21.15.27:0/2568853233 shutdown_connections 2021-11-03T17:18:01.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.785+0000 7f471b9fe700 1 --2- 172.21.15.27:0/2568853233 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f47000640b0 0x7f4700066570 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:01.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.785+0000 7f471b9fe700 1 --2- 172.21.15.27:0/2568853233 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f47140f66e0 0x7f4714106fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:01.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.785+0000 7f471b9fe700 1 --2- 172.21.15.27:0/2568853233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f47140f7980 0x7f47141074a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:01.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.785+0000 7f471b9fe700 1 -- 172.21.15.27:0/2568853233 >> 172.21.15.27:0/2568853233 conn(0x7f47140f1e40 msgr2=0x7f47140fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:18:01.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.785+0000 7f471b9fe700 1 -- 172.21.15.27:0/2568853233 shutdown_connections 2021-11-03T17:18:01.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.785+0000 7f471b9fe700 1 -- 172.21.15.27:0/2568853233 wait complete. 2021-11-03T17:18:01.789 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:18:01.885 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.890+0000 7fa3e2319700 1 -- 172.21.15.27:0/1125657193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3dc0f6c70 msgr2=0x7fa3dc0f7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:18:01.886 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.890+0000 7fa3e2319700 1 --2- 172.21.15.27:0/1125657193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3dc0f6c70 0x7fa3dc0f7090 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fa3c4004660 tx=0x7fa3c4009b30).stop 2021-11-03T17:18:01.886 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.890+0000 7fa3e2319700 1 -- 172.21.15.27:0/1125657193 shutdown_connections 2021-11-03T17:18:01.886 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.890+0000 7fa3e2319700 1 --2- 172.21.15.27:0/1125657193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3dc0f7610 0x7fa3dc0fdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:01.886 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.890+0000 7fa3e2319700 1 --2- 172.21.15.27:0/1125657193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3dc0f6c70 0x7fa3dc0f7090 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:01.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.890+0000 7fa3e2319700 1 -- 172.21.15.27:0/1125657193 >> 172.21.15.27:0/1125657193 conn(0x7fa3dc0f04c0 msgr2=0x7fa3dc0f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:18:01.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.890+0000 7fa3e2319700 1 -- 172.21.15.27:0/1125657193 shutdown_connections 2021-11-03T17:18:01.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.890+0000 7fa3e2319700 1 -- 172.21.15.27:0/1125657193 wait complete. 2021-11-03T17:18:01.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.890+0000 7fa3e2319700 1 Processor -- start 2021-11-03T17:18:01.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.890+0000 7fa3e2319700 1 -- start start 2021-11-03T17:18:01.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.891+0000 7fa3e2319700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3dc0f6c70 0x7fa3dc1091f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:18:01.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.891+0000 7fa3e2319700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3dc0f7610 0x7fa3dc1096e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:18:01.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.891+0000 7fa3e2319700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc109c90 con 0x7fa3dc0f6c70 2021-11-03T17:18:01.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.891+0000 7fa3e2319700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc109dd0 con 0x7fa3dc0f7610 2021-11-03T17:18:01.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.891+0000 7fa3dbfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3dc0f6c70 0x7fa3dc1091f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:18:01.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.891+0000 7fa3dbfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3dc0f6c70 0x7fa3dc1091f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56350/0 (socket says 172.21.15.27:56350) 2021-11-03T17:18:01.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.891+0000 7fa3dbfff700 1 -- 172.21.15.27:0/2571555421 learned_addr learned my addr 172.21.15.27:0/2571555421 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:18:01.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.891+0000 7fa3db7fe700 1 --2- 172.21.15.27:0/2571555421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3dc0f7610 0x7fa3dc1096e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:18:01.891 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.891+0000 7fa3dbfff700 1 -- 172.21.15.27:0/2571555421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3dc0f7610 msgr2=0x7fa3dc1096e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:18:01.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.891+0000 7fa3dbfff700 1 --2- 172.21.15.27:0/2571555421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3dc0f7610 0x7fa3dc1096e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:01.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.891+0000 7fa3dbfff700 1 -- 172.21.15.27:0/2571555421 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c4005040 con 0x7fa3dc0f6c70 2021-11-03T17:18:01.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.891+0000 7fa3dbfff700 1 --2- 172.21.15.27:0/2571555421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3dc0f6c70 0x7fa3dc1091f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fa3c4009990 tx=0x7fa3c4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:18:01.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.892+0000 7fa3d8ff9700 1 -- 172.21.15.27:0/2571555421 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3c4018070 con 0x7fa3dc0f6c70 2021-11-03T17:18:01.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.892+0000 7fa3d8ff9700 1 -- 172.21.15.27:0/2571555421 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fa3c4007ba0 con 0x7fa3dc0f6c70 2021-11-03T17:18:01.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.892+0000 7fa3e2319700 1 -- 172.21.15.27:0/2571555421 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3dc10a710 con 0x7fa3dc0f6c70 2021-11-03T17:18:01.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.892+0000 7fa3d8ff9700 1 -- 172.21.15.27:0/2571555421 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3c401de50 con 0x7fa3dc0f6c70 2021-11-03T17:18:01.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.892+0000 7fa3e2319700 1 -- 172.21.15.27:0/2571555421 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3dc10ac70 con 0x7fa3dc0f6c70 2021-11-03T17:18:01.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.893+0000 7fa3d8ff9700 1 -- 172.21.15.27:0/2571555421 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fa3c401c4a0 con 0x7fa3dc0f6c70 2021-11-03T17:18:01.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.893+0000 7fa3e2319700 1 -- 172.21.15.27:0/2571555421 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3dc045ad0 con 0x7fa3dc0f6c70 2021-11-03T17:18:01.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.893+0000 7fa3d8ff9700 1 --2- 172.21.15.27:0/2571555421 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa3c80640b0 0x7fa3c8066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:18:01.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.894+0000 7fa3d8ff9700 1 -- 172.21.15.27:0/2571555421 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fa3c4082480 con 0x7fa3dc0f6c70 2021-11-03T17:18:01.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.894+0000 7fa3db7fe700 1 --2- 172.21.15.27:0/2571555421 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa3c80640b0 0x7fa3c8066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:18:01.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.894+0000 7fa3db7fe700 1 --2- 172.21.15.27:0/2571555421 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa3c80640b0 0x7fa3c8066570 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fa3dc0f7a90 tx=0x7fa3cc00b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:18:01.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:01.898+0000 7fa3d8ff9700 1 -- 172.21.15.27:0/2571555421 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fa3c40547e0 con 0x7fa3dc0f6c70 2021-11-03T17:18:01.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:18:02.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.075+0000 7fa3e2319700 1 -- 172.21.15.27:0/2571555421 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa3dc1011e0 con 0x7fa3c80640b0 2021-11-03T17:18:02.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.090+0000 7fa3d8ff9700 1 -- 172.21.15.27:0/2571555421 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fa3dc1011e0 con 0x7fa3c80640b0 2021-11-03T17:18:02.085 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:18:02.086 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (9m) 19s ago 10m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:18:02.086 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (77s) 19s ago 10m 7205k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:18:02.086 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (67s) 63s ago 10m 18.1M - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:18:02.086 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (9m) 19s ago 10m 38.9M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:18:02.087 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (3m) 19s ago 5m 15.2M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:18:02.087 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (6m) 19s ago 6m 3836M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:18:02.087 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (109s) 63s ago 6m 13.5M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:18:02.087 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (3m) 63s ago 6m 15.2M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:18:02.087 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (3m) 19s ago 12m 468M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:18:02.088 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (2m) 63s ago 10m 410M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:18:02.088 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (2m) 19s ago 12m 75.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:18:02.088 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (112s) 63s ago 9m 40.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:18:02.088 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (10m) 19s ago 10m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:18:02.089 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (9m) 63s ago 9m 18.2M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:18:02.089 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (54s) 19s ago 9m 238M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:18:02.089 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (24s) 19s ago 8m 11.5M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:18:02.089 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (8m) 19s ago 8m 614M 4096M 16.2.4 8d91d370c2b8 c211ef5551c9 2021-11-03T17:18:02.089 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (8m) 63s ago 8m 922M 4096M 16.2.4 8d91d370c2b8 b1486ec1b619 2021-11-03T17:18:02.090 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (7m) 63s ago 7m 742M 4096M 16.2.4 8d91d370c2b8 8b91230c69e5 2021-11-03T17:18:02.090 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (7m) 63s ago 7m 844M 4096M 16.2.4 8d91d370c2b8 4a54c4d6b6f5 2021-11-03T17:18:02.090 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (9m) 19s ago 10m 45.6M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:18:02.091 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.092+0000 7fa3e2319700 1 -- 172.21.15.27:0/2571555421 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa3c80640b0 msgr2=0x7fa3c8066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:18:02.091 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.092+0000 7fa3e2319700 1 --2- 172.21.15.27:0/2571555421 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa3c80640b0 0x7fa3c8066570 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fa3dc0f7a90 tx=0x7fa3cc00b040).stop 2021-11-03T17:18:02.091 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.092+0000 7fa3e2319700 1 -- 172.21.15.27:0/2571555421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3dc0f6c70 msgr2=0x7fa3dc1091f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:18:02.092 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.092+0000 7fa3e2319700 1 --2- 172.21.15.27:0/2571555421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3dc0f6c70 0x7fa3dc1091f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fa3c4009990 tx=0x7fa3c4009b30).stop 2021-11-03T17:18:02.092 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.092+0000 7fa3e2319700 1 -- 172.21.15.27:0/2571555421 shutdown_connections 2021-11-03T17:18:02.092 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.092+0000 7fa3e2319700 1 --2- 172.21.15.27:0/2571555421 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa3c80640b0 0x7fa3c8066570 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:02.092 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.092+0000 7fa3e2319700 1 --2- 172.21.15.27:0/2571555421 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3dc0f6c70 0x7fa3dc1091f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:02.093 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.092+0000 7fa3e2319700 1 --2- 172.21.15.27:0/2571555421 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3dc0f7610 0x7fa3dc1096e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:02.093 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.092+0000 7fa3e2319700 1 -- 172.21.15.27:0/2571555421 >> 172.21.15.27:0/2571555421 conn(0x7fa3dc0f04c0 msgr2=0x7fa3dc0f28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:18:02.093 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.092+0000 7fa3e2319700 1 -- 172.21.15.27:0/2571555421 shutdown_connections 2021-11-03T17:18:02.093 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.092+0000 7fa3e2319700 1 -- 172.21.15.27:0/2571555421 wait complete. 2021-11-03T17:18:02.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.192+0000 7f589759e700 1 -- 172.21.15.27:0/575859197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58980f6a10 msgr2=0x7f58980f6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:18:02.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.192+0000 7f589759e700 1 --2- 172.21.15.27:0/575859197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58980f6a10 0x7f58980f6e30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f5888004660 tx=0x7f5888009b30).stop 2021-11-03T17:18:02.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.193+0000 7f589759e700 1 -- 172.21.15.27:0/575859197 shutdown_connections 2021-11-03T17:18:02.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.193+0000 7f589759e700 1 --2- 172.21.15.27:0/575859197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58980f5710 0x7f58980f5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:02.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.193+0000 7f589759e700 1 --2- 172.21.15.27:0/575859197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58980f6a10 0x7f58980f6e30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:02.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.193+0000 7f589759e700 1 -- 172.21.15.27:0/575859197 >> 172.21.15.27:0/575859197 conn(0x7f58980f0320 msgr2=0x7f58980f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:18:02.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.193+0000 7f589759e700 1 -- 172.21.15.27:0/575859197 shutdown_connections 2021-11-03T17:18:02.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.193+0000 7f589759e700 1 -- 172.21.15.27:0/575859197 wait complete. 2021-11-03T17:18:02.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.193+0000 7f589759e700 1 Processor -- start 2021-11-03T17:18:02.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.193+0000 7f589759e700 1 -- start start 2021-11-03T17:18:02.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.193+0000 7f589759e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58980f5710 0x7f5898109a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:18:02.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.193+0000 7f589759e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58980f6a10 0x7f589810bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:18:02.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.193+0000 7f589759e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58980f5400 con 0x7f58980f5710 2021-11-03T17:18:02.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.193+0000 7f589759e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58980f5540 con 0x7f58980f6a10 2021-11-03T17:18:02.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.193+0000 7f589659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58980f5710 0x7f5898109a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:18:02.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.193+0000 7f589659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58980f5710 0x7f5898109a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56360/0 (socket says 172.21.15.27:56360) 2021-11-03T17:18:02.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.193+0000 7f589659c700 1 -- 172.21.15.27:0/714151691 learned_addr learned my addr 172.21.15.27:0/714151691 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:18:02.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.194+0000 7f5895d9b700 1 --2- 172.21.15.27:0/714151691 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58980f6a10 0x7f589810bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:18:02.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.194+0000 7f589659c700 1 -- 172.21.15.27:0/714151691 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58980f6a10 msgr2=0x7f589810bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:18:02.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.194+0000 7f589659c700 1 --2- 172.21.15.27:0/714151691 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58980f6a10 0x7f589810bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:02.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.194+0000 7f589659c700 1 -- 172.21.15.27:0/714151691 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5888005040 con 0x7f58980f5710 2021-11-03T17:18:02.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.194+0000 7f589659c700 1 --2- 172.21.15.27:0/714151691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58980f5710 0x7f5898109a30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f5888004f60 tx=0x7f5888009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:18:02.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.194+0000 7f5886ffd700 1 -- 172.21.15.27:0/714151691 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5888018070 con 0x7f58980f5710 2021-11-03T17:18:02.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.194+0000 7f5886ffd700 1 -- 172.21.15.27:0/714151691 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f5888007ba0 con 0x7f58980f5710 2021-11-03T17:18:02.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.194+0000 7f589759e700 1 -- 172.21.15.27:0/714151691 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f589810c4e0 con 0x7f58980f5710 2021-11-03T17:18:02.196 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.194+0000 7f5886ffd700 1 -- 172.21.15.27:0/714151691 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f588801c620 con 0x7f58980f5710 2021-11-03T17:18:02.196 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.194+0000 7f589759e700 1 -- 172.21.15.27:0/714151691 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f589810ca10 con 0x7f58980f5710 2021-11-03T17:18:02.197 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.195+0000 7f589759e700 1 -- 172.21.15.27:0/714151691 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5898059100 con 0x7f58980f5710 2021-11-03T17:18:02.197 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.196+0000 7f5886ffd700 1 -- 172.21.15.27:0/714151691 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f588801c8e0 con 0x7f58980f5710 2021-11-03T17:18:02.198 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.196+0000 7f5886ffd700 1 --2- 172.21.15.27:0/714151691 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5880064100 0x7f58800665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:18:02.198 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.196+0000 7f5886ffd700 1 -- 172.21.15.27:0/714151691 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f58880826e0 con 0x7f58980f5710 2021-11-03T17:18:02.198 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.196+0000 7f5895d9b700 1 --2- 172.21.15.27:0/714151691 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5880064100 0x7f58800665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:18:02.198 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.196+0000 7f5895d9b700 1 --2- 172.21.15.27:0/714151691 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5880064100 0x7f58800665c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f58980f67e0 tx=0x7f588c009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:18:02.198 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.200+0000 7f5886ffd700 1 -- 172.21.15.27:0/714151691 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f5888054a40 con 0x7f58980f5710 2021-11-03T17:18:02.430 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.435+0000 7f589759e700 1 -- 172.21.15.27:0/714151691 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f58980fdc30 con 0x7f58980f5710 2021-11-03T17:18:02.430 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.435+0000 7f5886ffd700 1 -- 172.21.15.27:0/714151691 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+804 (secure 0 0 0) 0x7f5888054630 con 0x7f58980f5710 2021-11-03T17:18:02.431 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:18:02.431 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:18:02.431 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:18:02.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:18:02.432 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:18:02.432 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:18:02.432 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:18:02.433 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:18:02.433 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:18:02.433 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:18:02.433 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:18:02.434 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:18:02.434 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:18:02.434 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:18:02.434 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:18:02.435 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 8, 2021-11-03T17:18:02.435 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:18:02.435 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:18:02.435 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:18:02.436 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.437+0000 7f589759e700 1 -- 172.21.15.27:0/714151691 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5880064100 msgr2=0x7f58800665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:18:02.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.437+0000 7f589759e700 1 --2- 172.21.15.27:0/714151691 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5880064100 0x7f58800665c0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f58980f67e0 tx=0x7f588c009040).stop 2021-11-03T17:18:02.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.437+0000 7f589759e700 1 -- 172.21.15.27:0/714151691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58980f5710 msgr2=0x7f5898109a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:18:02.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.437+0000 7f589759e700 1 --2- 172.21.15.27:0/714151691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58980f5710 0x7f5898109a30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f5888004f60 tx=0x7f5888009b30).stop 2021-11-03T17:18:02.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.437+0000 7f589759e700 1 -- 172.21.15.27:0/714151691 shutdown_connections 2021-11-03T17:18:02.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.437+0000 7f589759e700 1 --2- 172.21.15.27:0/714151691 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5880064100 0x7f58800665c0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:02.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.437+0000 7f589759e700 1 --2- 172.21.15.27:0/714151691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f58980f5710 0x7f5898109a30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:02.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.437+0000 7f589759e700 1 --2- 172.21.15.27:0/714151691 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f58980f6a10 0x7f589810bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:02.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.437+0000 7f589759e700 1 -- 172.21.15.27:0/714151691 >> 172.21.15.27:0/714151691 conn(0x7f58980f0320 msgr2=0x7f58980fa170 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:18:02.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.437+0000 7f589759e700 1 -- 172.21.15.27:0/714151691 shutdown_connections 2021-11-03T17:18:02.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.437+0000 7f589759e700 1 -- 172.21.15.27:0/714151691 wait complete. 2021-11-03T17:18:02.548 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.552+0000 7f53229c0700 1 -- 172.21.15.27:0/2125326638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f531c0f9690 msgr2=0x7f531c0fbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:18:02.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.552+0000 7f53229c0700 1 --2- 172.21.15.27:0/2125326638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f531c0f9690 0x7f531c0fbae0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f5310004660 tx=0x7f5310009b30).stop 2021-11-03T17:18:02.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.552+0000 7f53229c0700 1 -- 172.21.15.27:0/2125326638 shutdown_connections 2021-11-03T17:18:02.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.552+0000 7f53229c0700 1 --2- 172.21.15.27:0/2125326638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f531c0f9690 0x7f531c0fbae0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:02.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.552+0000 7f53229c0700 1 --2- 172.21.15.27:0/2125326638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f531c060310 0x7f531c0f9110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:02.550 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.552+0000 7f53229c0700 1 -- 172.21.15.27:0/2125326638 >> 172.21.15.27:0/2125326638 conn(0x7f531c0f0b90 msgr2=0x7f531c0f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:18:02.550 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.552+0000 7f53229c0700 1 -- 172.21.15.27:0/2125326638 shutdown_connections 2021-11-03T17:18:02.550 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.552+0000 7f53229c0700 1 -- 172.21.15.27:0/2125326638 wait complete. 2021-11-03T17:18:02.550 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.553+0000 7f53229c0700 1 Processor -- start 2021-11-03T17:18:02.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.553+0000 7f53229c0700 1 -- start start 2021-11-03T17:18:02.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.553+0000 7f53229c0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f531c060310 0x7f531c0f6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:18:02.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.553+0000 7f53229c0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f531c0f9690 0x7f531c0f5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:18:02.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.553+0000 7f53229c0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531c0f5570 con 0x7f531c060310 2021-11-03T17:18:02.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.553+0000 7f53229c0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f531c0f56b0 con 0x7f531c0f9690 2021-11-03T17:18:02.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.553+0000 7f53219be700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f531c060310 0x7f531c0f6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:18:02.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.553+0000 7f53219be700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f531c060310 0x7f531c0f6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56370/0 (socket says 172.21.15.27:56370) 2021-11-03T17:18:02.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.553+0000 7f53219be700 1 -- 172.21.15.27:0/1115883040 learned_addr learned my addr 172.21.15.27:0/1115883040 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:18:02.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.553+0000 7f53211bd700 1 --2- 172.21.15.27:0/1115883040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f531c0f9690 0x7f531c0f5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:18:02.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.553+0000 7f53219be700 1 -- 172.21.15.27:0/1115883040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f531c0f9690 msgr2=0x7f531c0f5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:18:02.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.553+0000 7f53219be700 1 --2- 172.21.15.27:0/1115883040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f531c0f9690 0x7f531c0f5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:02.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.553+0000 7f53219be700 1 -- 172.21.15.27:0/1115883040 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5310005040 con 0x7f531c060310 2021-11-03T17:18:02.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.553+0000 7f53219be700 1 --2- 172.21.15.27:0/1115883040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f531c060310 0x7f531c0f6b90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f531800e7f0 tx=0x7f531800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:18:02.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.553+0000 7f530e7fc700 1 -- 172.21.15.27:0/1115883040 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53180092e0 con 0x7f531c060310 2021-11-03T17:18:02.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.554+0000 7f530e7fc700 1 -- 172.21.15.27:0/1115883040 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f531800f040 con 0x7f531c060310 2021-11-03T17:18:02.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.554+0000 7f53229c0700 1 -- 172.21.15.27:0/1115883040 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f531c0f5990 con 0x7f531c060310 2021-11-03T17:18:02.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.554+0000 7f530e7fc700 1 -- 172.21.15.27:0/1115883040 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5318014750 con 0x7f531c060310 2021-11-03T17:18:02.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.554+0000 7f53229c0700 1 -- 172.21.15.27:0/1115883040 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f531c10ecf0 con 0x7f531c060310 2021-11-03T17:18:02.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.555+0000 7f53229c0700 1 -- 172.21.15.27:0/1115883040 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f531c045ad0 con 0x7f531c060310 2021-11-03T17:18:02.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.555+0000 7f530e7fc700 1 -- 172.21.15.27:0/1115883040 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f531801b030 con 0x7f531c060310 2021-11-03T17:18:02.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.556+0000 7f530e7fc700 1 --2- 172.21.15.27:0/1115883040 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5308064100 0x7f53080665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:18:02.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.556+0000 7f530e7fc700 1 -- 172.21.15.27:0/1115883040 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f5318080650 con 0x7f531c060310 2021-11-03T17:18:02.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.556+0000 7f53211bd700 1 --2- 172.21.15.27:0/1115883040 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5308064100 0x7f53080665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:18:02.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.556+0000 7f53211bd700 1 --2- 172.21.15.27:0/1115883040 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5308064100 0x7f53080665c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f5310005010 tx=0x7f531000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:18:02.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.560+0000 7f530e7fc700 1 -- 172.21.15.27:0/1115883040 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f5318052970 con 0x7f531c060310 2021-11-03T17:18:02.690 INFO:journalctl@ceph.osd.2.smithi027.stdout:Nov 03 17:18:02 smithi027 systemd[1]: Stopping Ceph osd.2 for 32ab977e-3cc8-11ec-8c28-001a4aab830c... 2021-11-03T17:18:02.691 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:02 smithi027 ceph-mon[97244]: pgmap v79: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s, 7 keys/s, 0 objects/s recovering 2021-11-03T17:18:02.691 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:02 smithi027 ceph-mon[97244]: from='client.34206 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:18:02.691 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:02 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/714151691' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:18:02.774 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.779+0000 7f53229c0700 1 -- 172.21.15.27:0/1115883040 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f531c027a00 con 0x7f531c060310 2021-11-03T17:18:02.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.780+0000 7f530e7fc700 1 -- 172.21.15.27:0/1115883040 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1795 (secure 0 0 0) 0x7f5318019070 con 0x7f531c060310 2021-11-03T17:18:02.775 INFO:teuthology.orchestra.run.smithi027.stdout:e28 2021-11-03T17:18:02.776 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:18:02.776 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:18:02.776 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:18:02.777 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:18:02.777 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:18:02.777 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:18:02.777 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 28 2021-11-03T17:18:02.777 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:18:02.778 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:18:02.778 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:16:13.536828+0000 2021-11-03T17:18:02.778 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:18:02.779 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:18:02.779 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:18:02.779 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:18:02.780 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:18:02.780 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:18:02.780 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:18:02.780 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:18:02.781 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:18:02.781 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 2 2021-11-03T17:18:02.781 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:18:02.781 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:18:02.781 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:18:02.782 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:18:02.782 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:18:02.782 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:18:02.782 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:18:02.783 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:18:02.783 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:18:02.783 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:18:02.783 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 3 export targets 0 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:18:02.784 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:18:02.784 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:18:02.784 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:18:02.784 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:18:02.785 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:18:02.785 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:18:02.785 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:18:02.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.781+0000 7f53229c0700 1 -- 172.21.15.27:0/1115883040 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5308064100 msgr2=0x7f53080665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:18:02.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.781+0000 7f53229c0700 1 --2- 172.21.15.27:0/1115883040 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5308064100 0x7f53080665c0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f5310005010 tx=0x7f531000b040).stop 2021-11-03T17:18:02.787 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.782+0000 7f53229c0700 1 -- 172.21.15.27:0/1115883040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f531c060310 msgr2=0x7f531c0f6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:18:02.787 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.782+0000 7f53229c0700 1 --2- 172.21.15.27:0/1115883040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f531c060310 0x7f531c0f6b90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f531800e7f0 tx=0x7f531800b040).stop 2021-11-03T17:18:02.787 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.782+0000 7f53229c0700 1 -- 172.21.15.27:0/1115883040 shutdown_connections 2021-11-03T17:18:02.787 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.782+0000 7f53229c0700 1 --2- 172.21.15.27:0/1115883040 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5308064100 0x7f53080665c0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:02.788 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.782+0000 7f53229c0700 1 --2- 172.21.15.27:0/1115883040 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f531c060310 0x7f531c0f6b90 secure :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0x7f531800e7f0 tx=0x7f531800b040).stop 2021-11-03T17:18:02.788 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.782+0000 7f53229c0700 1 --2- 172.21.15.27:0/1115883040 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f531c0f9690 0x7f531c0f5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:02.788 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.782+0000 7f53229c0700 1 -- 172.21.15.27:0/1115883040 >> 172.21.15.27:0/1115883040 conn(0x7f531c0f0b90 msgr2=0x7f531c0f2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:18:02.788 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.782+0000 7f53229c0700 1 -- 172.21.15.27:0/1115883040 shutdown_connections 2021-11-03T17:18:02.788 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:02.782+0000 7f53229c0700 1 -- 172.21.15.27:0/1115883040 wait complete. 2021-11-03T17:18:02.789 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 28 2021-11-03T17:18:02.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:02 smithi115 ceph-mon[65453]: pgmap v79: 65 pgs: 65 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 0 B/s, 7 keys/s, 0 objects/s recovering 2021-11-03T17:18:02.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:02 smithi115 ceph-mon[65453]: from='client.34206 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:18:02.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:02 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/714151691' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:18:03.018 INFO:journalctl@ceph.osd.2.smithi027.stdout:Nov 03 17:18:02 smithi027 conmon[63140]: debug 2021-11-03T17:18:02.693+0000 7fc29cb8b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2021-11-03T17:18:03.019 INFO:journalctl@ceph.osd.2.smithi027.stdout:Nov 03 17:18:02 smithi027 conmon[63140]: debug 2021-11-03T17:18:02.694+0000 7fc29cb8b700 -1 osd.2 63 *** Got signal Terminated *** 2021-11-03T17:18:03.019 INFO:journalctl@ceph.osd.2.smithi027.stdout:Nov 03 17:18:02 smithi027 conmon[63140]: debug 2021-11-03T17:18:02.694+0000 7fc29cb8b700 -1 osd.2 63 *** Immediate shutdown (osd_fast_shutdown=true) *** 2021-11-03T17:18:03.019 INFO:journalctl@ceph.osd.2.smithi027.stdout:Nov 03 17:18:02 smithi027 bash[112727]: c211ef5551c941ed5775b2c3420a7c841088e8dc56c3bb73c8acba7ce5563595 2021-11-03T17:18:03.464 INFO:journalctl@ceph.osd.2.smithi027.stdout:Nov 03 17:18:03 smithi027 bash[112727]: Error: no container with name or ID ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c-osd-2 found: no such container 2021-11-03T17:18:03.768 INFO:journalctl@ceph.osd.2.smithi027.stdout:Nov 03 17:18:03 smithi027 bash[112727]: Error: container c211ef5551c941ed5775b2c3420a7c841088e8dc56c3bb73c8acba7ce5563595 does not exist in database: no such container 2021-11-03T17:18:03.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:03 smithi027 ceph-mon[97244]: from='client.34210 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:18:03.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:03 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1115883040' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:18:03.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:03 smithi027 ceph-mon[97244]: osd.2 reported immediately failed by osd.0 2021-11-03T17:18:03.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:03 smithi027 ceph-mon[97244]: osd.2 failed (root=default,host=smithi027) (connection refused reported by osd.0) 2021-11-03T17:18:03.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:03 smithi027 ceph-mon[97244]: osd.2 reported immediately failed by osd.1 2021-11-03T17:18:03.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:03 smithi027 ceph-mon[97244]: osd.2 reported immediately failed by osd.4 2021-11-03T17:18:03.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:03 smithi027 ceph-mon[97244]: osd.2 reported immediately failed by osd.5 2021-11-03T17:18:03.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:03 smithi027 ceph-mon[97244]: osd.2 reported immediately failed by osd.3 2021-11-03T17:18:03.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:03 smithi027 ceph-mon[97244]: osd.2 reported immediately failed by osd.0 2021-11-03T17:18:03.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:03 smithi027 ceph-mon[97244]: osd.2 reported immediately failed by osd.1 2021-11-03T17:18:03.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:03 smithi027 ceph-mon[97244]: osd.2 reported immediately failed by osd.4 2021-11-03T17:18:03.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:03 smithi027 ceph-mon[97244]: osd.2 reported immediately failed by osd.5 2021-11-03T17:18:03.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:03 smithi027 ceph-mon[97244]: osd.2 reported immediately failed by osd.3 2021-11-03T17:18:03.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:03 smithi027 ceph-mon[97244]: osd.2 reported immediately failed by osd.3 2021-11-03T17:18:03.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:03 smithi027 ceph-mon[97244]: osd.2 reported immediately failed by osd.0 2021-11-03T17:18:03.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:03 smithi027 ceph-mon[97244]: osd.2 reported immediately failed by osd.1 2021-11-03T17:18:03.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:03 smithi027 ceph-mon[97244]: osd.2 reported immediately failed by osd.0 2021-11-03T17:18:03.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:03 smithi027 ceph-mon[97244]: osd.2 reported immediately failed by osd.1 2021-11-03T17:18:03.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:03 smithi027 ceph-mon[97244]: osd.2 reported immediately failed by osd.5 2021-11-03T17:18:03.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:03 smithi027 ceph-mon[97244]: osd.2 reported immediately failed by osd.3 2021-11-03T17:18:03.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:03 smithi027 ceph-mon[97244]: osd.2 reported immediately failed by osd.5 2021-11-03T17:18:03.776 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:03 smithi027 ceph-mon[97244]: osd.2 reported immediately failed by osd.4 2021-11-03T17:18:03.776 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:03 smithi027 ceph-mon[97244]: osd.2 reported immediately failed by osd.4 2021-11-03T17:18:03.955 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:03 smithi115 ceph-mon[65453]: from='client.34210 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:18:03.956 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:03 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1115883040' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:18:03.956 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:03 smithi115 ceph-mon[65453]: osd.2 reported immediately failed by osd.0 2021-11-03T17:18:03.957 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:03 smithi115 ceph-mon[65453]: osd.2 failed (root=default,host=smithi027) (connection refused reported by osd.0) 2021-11-03T17:18:03.957 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:03 smithi115 ceph-mon[65453]: osd.2 reported immediately failed by osd.1 2021-11-03T17:18:03.957 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:03 smithi115 ceph-mon[65453]: osd.2 reported immediately failed by osd.4 2021-11-03T17:18:03.958 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:03 smithi115 ceph-mon[65453]: osd.2 reported immediately failed by osd.5 2021-11-03T17:18:03.958 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:03 smithi115 ceph-mon[65453]: osd.2 reported immediately failed by osd.3 2021-11-03T17:18:03.958 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:03 smithi115 ceph-mon[65453]: osd.2 reported immediately failed by osd.0 2021-11-03T17:18:03.959 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:03 smithi115 ceph-mon[65453]: osd.2 reported immediately failed by osd.1 2021-11-03T17:18:03.959 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:03 smithi115 ceph-mon[65453]: osd.2 reported immediately failed by osd.4 2021-11-03T17:18:03.959 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:03 smithi115 ceph-mon[65453]: osd.2 reported immediately failed by osd.5 2021-11-03T17:18:03.960 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:03 smithi115 ceph-mon[65453]: osd.2 reported immediately failed by osd.3 2021-11-03T17:18:03.960 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:03 smithi115 ceph-mon[65453]: osd.2 reported immediately failed by osd.3 2021-11-03T17:18:03.960 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:03 smithi115 ceph-mon[65453]: osd.2 reported immediately failed by osd.0 2021-11-03T17:18:03.960 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:03 smithi115 ceph-mon[65453]: osd.2 reported immediately failed by osd.1 2021-11-03T17:18:03.960 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:03 smithi115 ceph-mon[65453]: osd.2 reported immediately failed by osd.0 2021-11-03T17:18:03.961 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:03 smithi115 ceph-mon[65453]: osd.2 reported immediately failed by osd.1 2021-11-03T17:18:03.961 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:03 smithi115 ceph-mon[65453]: osd.2 reported immediately failed by osd.5 2021-11-03T17:18:03.961 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:03 smithi115 ceph-mon[65453]: osd.2 reported immediately failed by osd.3 2021-11-03T17:18:03.961 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:03 smithi115 ceph-mon[65453]: osd.2 reported immediately failed by osd.5 2021-11-03T17:18:03.962 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:03 smithi115 ceph-mon[65453]: osd.2 reported immediately failed by osd.4 2021-11-03T17:18:03.962 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:03 smithi115 ceph-mon[65453]: osd.2 reported immediately failed by osd.4 2021-11-03T17:18:04.967 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:04 smithi027 ceph-mon[97244]: Health check failed: 1 osds down (OSD_DOWN) 2021-11-03T17:18:04.967 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:04 smithi027 ceph-mon[97244]: osdmap e64: 6 total, 5 up, 6 in 2021-11-03T17:18:04.968 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:04 smithi027 ceph-mon[97244]: pgmap v81: 65 pgs: 9 stale+active+clean, 56 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:18:04.968 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:04 smithi027 ceph-mon[97244]: osdmap e65: 6 total, 5 up, 6 in 2021-11-03T17:18:04.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:04 smithi115 ceph-mon[65453]: Health check failed: 1 osds down (OSD_DOWN) 2021-11-03T17:18:04.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:04 smithi115 ceph-mon[65453]: osdmap e64: 6 total, 5 up, 6 in 2021-11-03T17:18:04.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:04 smithi115 ceph-mon[65453]: pgmap v81: 65 pgs: 9 stale+active+clean, 56 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:18:04.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:04 smithi115 ceph-mon[65453]: osdmap e65: 6 total, 5 up, 6 in 2021-11-03T17:18:05.268 INFO:journalctl@ceph.osd.2.smithi027.stdout:Nov 03 17:18:04 smithi027 systemd[1]: ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c@osd.2.service: Succeeded. 2021-11-03T17:18:05.269 INFO:journalctl@ceph.osd.2.smithi027.stdout:Nov 03 17:18:04 smithi027 systemd[1]: Stopped Ceph osd.2 for 32ab977e-3cc8-11ec-8c28-001a4aab830c. 2021-11-03T17:18:05.269 INFO:journalctl@ceph.osd.2.smithi027.stdout:Nov 03 17:18:05 smithi027 systemd[1]: Starting Ceph osd.2 for 32ab977e-3cc8-11ec-8c28-001a4aab830c... 2021-11-03T17:18:06.519 INFO:journalctl@ceph.osd.2.smithi027.stdout:Nov 03 17:18:06 smithi027 bash[113212]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2021-11-03T17:18:06.519 INFO:journalctl@ceph.osd.2.smithi027.stdout:Nov 03 17:18:06 smithi027 bash[113212]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --dev /dev/mapper/vg_nvme-lv_2 --path /var/lib/ceph/osd/ceph-2 --no-mon-config 2021-11-03T17:18:06.519 INFO:journalctl@ceph.osd.2.smithi027.stdout:Nov 03 17:18:06 smithi027 bash[113212]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2021-11-03T17:18:06.520 INFO:journalctl@ceph.osd.2.smithi027.stdout:Nov 03 17:18:06 smithi027 bash[113212]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2021-11-03T17:18:06.520 INFO:journalctl@ceph.osd.2.smithi027.stdout:Nov 03 17:18:06 smithi027 bash[113212]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2021-11-03T17:18:06.520 INFO:journalctl@ceph.osd.2.smithi027.stdout:Nov 03 17:18:06 smithi027 bash[113212]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2021-11-03T17:18:06.520 INFO:journalctl@ceph.osd.2.smithi027.stdout:Nov 03 17:18:06 smithi027 bash[113212]: --> ceph-volume raw activate successful for osd ID: 2 2021-11-03T17:18:07.233 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:06 smithi027 ceph-mon[97244]: pgmap v83: 65 pgs: 9 stale+active+clean, 56 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:18:07.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:06 smithi115 ceph-mon[65453]: pgmap v83: 65 pgs: 9 stale+active+clean, 56 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:18:07.769 INFO:journalctl@ceph.osd.2.smithi027.stdout:Nov 03 17:18:07 smithi027 ceph-osd[113742]: -- 172.21.15.27:0/2078109479 <== mon.0 v2:172.21.15.27:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555dab780340 con 0x555dab747000 2021-11-03T17:18:07.769 INFO:journalctl@ceph.osd.2.smithi027.stdout:Nov 03 17:18:07 smithi027 bash[113212]: 8e134c0e6fe16a63b4661b7f9de6c6a7f49d5b74142b5bcb3656df37303814c5 2021-11-03T17:18:07.769 INFO:journalctl@ceph.osd.2.smithi027.stdout:Nov 03 17:18:07 smithi027 systemd[1]: Started Ceph osd.2 for 32ab977e-3cc8-11ec-8c28-001a4aab830c. 2021-11-03T17:18:08.269 INFO:journalctl@ceph.osd.2.smithi027.stdout:Nov 03 17:18:07 smithi027 conmon[113719]: 2021-11-03T17:18:07.865+0000 7f62165e20c0 -1 Falling back to public interface 2021-11-03T17:18:08.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:08 smithi115 ceph-mon[65453]: poll took 7495.001 ms: 1 events 2021-11-03T17:18:08.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:18:08.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:18:08.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:18:08.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:08 smithi115 ceph-mon[65453]: pgmap v84: 65 pgs: 26 active+undersized+degraded, 1 active+undersized, 38 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 7620/52479 objects degraded (14.520%) 2021-11-03T17:18:08.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:08 smithi027 ceph-mon[97244]: poll took 7495.001 ms: 1 events 2021-11-03T17:18:08.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:18:08.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:18:08.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:18:08.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:08 smithi027 ceph-mon[97244]: pgmap v84: 65 pgs: 26 active+undersized+degraded, 1 active+undersized, 38 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 7620/52479 objects degraded (14.520%) 2021-11-03T17:18:09.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:09 smithi115 ceph-mon[65453]: Health check failed: Degraded data redundancy: 7620/52479 objects degraded (14.520%), 26 pgs degraded (PG_DEGRADED) 2021-11-03T17:18:09.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:09 smithi027 ceph-mon[97244]: Health check failed: Degraded data redundancy: 7620/52479 objects degraded (14.520%), 26 pgs degraded (PG_DEGRADED) 2021-11-03T17:18:10.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:10 smithi115 ceph-mon[65453]: pgmap v85: 65 pgs: 26 active+undersized+degraded, 1 active+undersized, 38 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 7620/52479 objects degraded (14.520%) 2021-11-03T17:18:10.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:10 smithi027 ceph-mon[97244]: pgmap v85: 65 pgs: 26 active+undersized+degraded, 1 active+undersized, 38 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 7620/52479 objects degraded (14.520%) 2021-11-03T17:18:11.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:18:12.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:18:12.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:12 smithi027 ceph-mon[97244]: pgmap v86: 65 pgs: 26 active+undersized+degraded, 1 active+undersized, 38 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 7620/52479 objects degraded (14.520%) 2021-11-03T17:18:12.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:18:12.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:18:12.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:18:12.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:18:12.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:18:12.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:18:12.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2021-11-03T17:18:12.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:12 smithi115 ceph-mon[65453]: pgmap v86: 65 pgs: 26 active+undersized+degraded, 1 active+undersized, 38 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 7620/52479 objects degraded (14.520%) 2021-11-03T17:18:12.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:18:12.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:18:12.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:18:12.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:18:12.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:18:12.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:18:12.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2021-11-03T17:18:13.956 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:13 smithi115 ceph-mon[65453]: poll took 4776.566 ms: 1 events 2021-11-03T17:18:13.956 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:13 smithi115 ceph-mon[65453]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2021-11-03T17:18:13.957 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:13 smithi115 ceph-mon[65453]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2021-11-03T17:18:14.019 INFO:journalctl@ceph.osd.2.smithi027.stdout:Nov 03 17:18:13 smithi027 conmon[113719]: 2021-11-03T17:18:13.720+0000 7f62165e20c0 -1 osd.2 63 log_to_monitors {default=true} 2021-11-03T17:18:14.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:13 smithi027 ceph-mon[97244]: poll took 4776.566 ms: 1 events 2021-11-03T17:18:14.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:13 smithi027 ceph-mon[97244]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2021-11-03T17:18:14.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:13 smithi027 ceph-mon[97244]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2021-11-03T17:18:14.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:14 smithi115 ceph-mon[65453]: from='osd.2 [v2:172.21.15.27:6818/4134200224,v1:172.21.15.27:6819/4134200224]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2021-11-03T17:18:14.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:14 smithi115 ceph-mon[65453]: pgmap v87: 65 pgs: 26 active+undersized+degraded, 1 active+undersized, 38 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 7620/52479 objects degraded (14.520%) 2021-11-03T17:18:15.019 INFO:journalctl@ceph.osd.2.smithi027.stdout:Nov 03 17:18:14 smithi027 conmon[113719]: 2021-11-03T17:18:14.566+0000 7f6207f22700 -1 osd.2 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2021-11-03T17:18:15.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:14 smithi027 ceph-mon[97244]: from='osd.2 [v2:172.21.15.27:6818/4134200224,v1:172.21.15.27:6819/4134200224]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2021-11-03T17:18:15.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:14 smithi027 ceph-mon[97244]: pgmap v87: 65 pgs: 26 active+undersized+degraded, 1 active+undersized, 38 active+clean; 564 MiB data, 2.1 GiB used, 534 GiB / 536 GiB avail; 7620/52479 objects degraded (14.520%) 2021-11-03T17:18:15.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:15 smithi115 ceph-mon[65453]: from='osd.2 [v2:172.21.15.27:6818/4134200224,v1:172.21.15.27:6819/4134200224]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2021-11-03T17:18:15.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:15 smithi115 ceph-mon[65453]: osdmap e66: 6 total, 5 up, 6 in 2021-11-03T17:18:15.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:15 smithi115 ceph-mon[65453]: from='osd.2 [v2:172.21.15.27:6818/4134200224,v1:172.21.15.27:6819/4134200224]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2021-11-03T17:18:16.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:15 smithi027 ceph-mon[97244]: from='osd.2 [v2:172.21.15.27:6818/4134200224,v1:172.21.15.27:6819/4134200224]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2021-11-03T17:18:16.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:15 smithi027 ceph-mon[97244]: osdmap e66: 6 total, 5 up, 6 in 2021-11-03T17:18:16.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:15 smithi027 ceph-mon[97244]: from='osd.2 [v2:172.21.15.27:6818/4134200224,v1:172.21.15.27:6819/4134200224]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi027", "root=default"]}]: dispatch 2021-11-03T17:18:16.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:16 smithi115 ceph-mon[65453]: Health check cleared: OSD_DOWN (was: 1 osds down) 2021-11-03T17:18:16.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:16 smithi115 ceph-mon[65453]: osd.2 [v2:172.21.15.27:6818/4134200224,v1:172.21.15.27:6819/4134200224] boot 2021-11-03T17:18:16.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:16 smithi115 ceph-mon[65453]: osdmap e67: 6 total, 6 up, 6 in 2021-11-03T17:18:16.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-11-03T17:18:16.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:16 smithi115 ceph-mon[65453]: pgmap v90: 65 pgs: 26 active+undersized+degraded, 1 active+undersized, 38 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 7620/52479 objects degraded (14.520%) 2021-11-03T17:18:16.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:16 smithi115 ceph-mon[65453]: osdmap e68: 6 total, 6 up, 6 in 2021-11-03T17:18:17.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:16 smithi027 ceph-mon[97244]: Health check cleared: OSD_DOWN (was: 1 osds down) 2021-11-03T17:18:17.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:16 smithi027 ceph-mon[97244]: osd.2 [v2:172.21.15.27:6818/4134200224,v1:172.21.15.27:6819/4134200224] boot 2021-11-03T17:18:17.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:16 smithi027 ceph-mon[97244]: osdmap e67: 6 total, 6 up, 6 in 2021-11-03T17:18:17.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2021-11-03T17:18:17.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:16 smithi027 ceph-mon[97244]: pgmap v90: 65 pgs: 26 active+undersized+degraded, 1 active+undersized, 38 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 7620/52479 objects degraded (14.520%) 2021-11-03T17:18:17.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:16 smithi027 ceph-mon[97244]: osdmap e68: 6 total, 6 up, 6 in 2021-11-03T17:18:18.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:17 smithi115 ceph-mon[65453]: Health check update: Degraded data redundancy: 2211/52479 objects degraded (4.213%), 8 pgs degraded (PG_DEGRADED) 2021-11-03T17:18:18.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:17 smithi027 ceph-mon[97244]: Health check update: Degraded data redundancy: 2211/52479 objects degraded (4.213%), 8 pgs degraded (PG_DEGRADED) 2021-11-03T17:18:19.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:18 smithi115 ceph-mon[65453]: pgmap v92: 65 pgs: 18 peering, 8 active+undersized+degraded, 1 active+undersized, 38 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2211/52479 objects degraded (4.213%) 2021-11-03T17:18:19.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:18 smithi027 ceph-mon[97244]: pgmap v92: 65 pgs: 18 peering, 8 active+undersized+degraded, 1 active+undersized, 38 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 2211/52479 objects degraded (4.213%) 2021-11-03T17:18:20.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:20 smithi115 ceph-mon[65453]: pgmap v93: 65 pgs: 9 active+clean+wait, 18 peering, 38 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:18:20.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:20 smithi027 ceph-mon[97244]: pgmap v93: 65 pgs: 9 active+clean+wait, 18 peering, 38 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:18:21.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:21 smithi115 ceph-mon[65453]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2211/52479 objects degraded (4.213%), 8 pgs degraded) 2021-11-03T17:18:21.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:21 smithi027 ceph-mon[97244]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2211/52479 objects degraded (4.213%), 8 pgs degraded) 2021-11-03T17:18:22.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:22 smithi115 ceph-mon[65453]: pgmap v94: 65 pgs: 9 active+clean+wait, 56 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:18:22.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:22 smithi027 ceph-mon[97244]: pgmap v94: 65 pgs: 9 active+clean+wait, 56 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:18:25.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:24 smithi115 ceph-mon[65453]: pgmap v95: 65 pgs: 9 active+clean+wait, 56 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:18:25.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:24 smithi027 ceph-mon[97244]: pgmap v95: 65 pgs: 9 active+clean+wait, 56 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:18:26.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:18:26.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:26 smithi115 ceph-mon[65453]: pgmap v96: 65 pgs: 9 active+clean+wait, 56 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:18:27.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:18:27.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:26 smithi027 ceph-mon[97244]: pgmap v96: 65 pgs: 9 active+clean+wait, 56 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:18:27.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2021-11-03T17:18:28.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2021-11-03T17:18:28.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:28 smithi115 ceph-mon[65453]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2021-11-03T17:18:28.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:28 smithi115 ceph-mon[65453]: Upgrade: osd.3 is safe to restart 2021-11-03T17:18:28.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:28 smithi115 ceph-mon[65453]: poll took 15039.258 ms: 1 events 2021-11-03T17:18:28.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:28 smithi115 ceph-mon[65453]: pgmap v97: 65 pgs: 9 active+clean+wait, 56 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:18:29.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:28 smithi027 ceph-mon[97244]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2021-11-03T17:18:29.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:28 smithi027 ceph-mon[97244]: Upgrade: osd.3 is safe to restart 2021-11-03T17:18:29.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:28 smithi027 ceph-mon[97244]: poll took 15039.258 ms: 1 events 2021-11-03T17:18:29.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:28 smithi027 ceph-mon[97244]: pgmap v97: 65 pgs: 9 active+clean+wait, 56 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:18:30.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:30 smithi115 ceph-mon[65453]: poll took 1523.071 ms: 1 events 2021-11-03T17:18:30.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:30 smithi115 ceph-mon[65453]: Upgrade: Updating osd.3 2021-11-03T17:18:30.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:18:30.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:30 smithi115 ceph-mon[65453]: Deploying daemon osd.3 on smithi115 2021-11-03T17:18:30.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:30 smithi115 ceph-mon[65453]: pgmap v98: 65 pgs: 65 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:18:30.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:30 smithi027 ceph-mon[97244]: poll took 1523.071 ms: 1 events 2021-11-03T17:18:30.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:30 smithi027 ceph-mon[97244]: Upgrade: Updating osd.3 2021-11-03T17:18:30.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:18:30.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:30 smithi027 ceph-mon[97244]: Deploying daemon osd.3 on smithi115 2021-11-03T17:18:30.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:30 smithi027 ceph-mon[97244]: pgmap v98: 65 pgs: 65 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:18:31.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:18:31.985 INFO:journalctl@ceph.osd.3.smithi115.stdout:Nov 03 17:18:31 smithi115 systemd[1]: Stopping Ceph osd.3 for 32ab977e-3cc8-11ec-8c28-001a4aab830c... 2021-11-03T17:18:31.985 INFO:journalctl@ceph.osd.3.smithi115.stdout:Nov 03 17:18:31 smithi115 conmon[41051]: debug 2021-11-03T17:18:31.899+0000 7f552739d700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2021-11-03T17:18:31.985 INFO:journalctl@ceph.osd.3.smithi115.stdout:Nov 03 17:18:31 smithi115 conmon[41051]: debug 2021-11-03T17:18:31.899+0000 7f552739d700 -1 osd.3 68 *** Got signal Terminated *** 2021-11-03T17:18:31.986 INFO:journalctl@ceph.osd.3.smithi115.stdout:Nov 03 17:18:31 smithi115 conmon[41051]: debug 2021-11-03T17:18:31.899+0000 7f552739d700 -1 osd.3 68 *** Immediate shutdown (osd_fast_shutdown=true) *** 2021-11-03T17:18:32.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:18:32.484 INFO:journalctl@ceph.osd.3.smithi115.stdout:Nov 03 17:18:32 smithi115 bash[71047]: b1486ec1b6198cabe6909cf7ccb69c5e37e8b22aa59a25f3aa6b3e3a51107379 2021-11-03T17:18:32.484 INFO:journalctl@ceph.osd.3.smithi115.stdout:Nov 03 17:18:32 smithi115 bash[71047]: Error: no container with name or ID ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c-osd-3 found: no such container 2021-11-03T17:18:32.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:32.899+0000 7f0e3e9e5700 1 -- 172.21.15.27:0/753645123 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e380f5ef0 msgr2=0x7f0e380f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:18:32.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:32.899+0000 7f0e3e9e5700 1 --2- 172.21.15.27:0/753645123 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e380f5ef0 0x7f0e380f6920 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f0e2c004660 tx=0x7f0e2c009b30).stop 2021-11-03T17:18:32.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:32.899+0000 7f0e3e9e5700 1 -- 172.21.15.27:0/753645123 shutdown_connections 2021-11-03T17:18:32.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:32.899+0000 7f0e3e9e5700 1 --2- 172.21.15.27:0/753645123 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e380f5ef0 0x7f0e380f6920 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:32.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:32.899+0000 7f0e3e9e5700 1 --2- 172.21.15.27:0/753645123 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e380f55e0 0x7f0e380f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:32.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:32.899+0000 7f0e3e9e5700 1 -- 172.21.15.27:0/753645123 >> 172.21.15.27:0/753645123 conn(0x7f0e380f0de0 msgr2=0x7f0e380f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:18:32.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:32.899+0000 7f0e3e9e5700 1 -- 172.21.15.27:0/753645123 shutdown_connections 2021-11-03T17:18:32.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:32.899+0000 7f0e3e9e5700 1 -- 172.21.15.27:0/753645123 wait complete. 2021-11-03T17:18:32.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:32.900+0000 7f0e3e9e5700 1 Processor -- start 2021-11-03T17:18:32.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:32.900+0000 7f0e3e9e5700 1 -- start start 2021-11-03T17:18:32.903 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:32.900+0000 7f0e3e9e5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e380f55e0 0x7f0e38106ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:18:32.903 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:32.900+0000 7f0e3e9e5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e380f5ef0 0x7f0e381073e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:18:32.903 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:32.900+0000 7f0e3e9e5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e38107990 con 0x7f0e380f55e0 2021-11-03T17:18:32.903 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:32.900+0000 7f0e3e9e5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e38107ad0 con 0x7f0e380f5ef0 2021-11-03T17:18:32.904 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:32.900+0000 7f0e37fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e380f55e0 0x7f0e38106ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:18:32.904 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:32.900+0000 7f0e377fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e380f5ef0 0x7f0e381073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:18:32.904 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:32.900+0000 7f0e37fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e380f55e0 0x7f0e38106ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56528/0 (socket says 172.21.15.27:56528) 2021-11-03T17:18:32.904 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:32.900+0000 7f0e37fff700 1 -- 172.21.15.27:0/2640385106 learned_addr learned my addr 172.21.15.27:0/2640385106 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:18:32.905 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:32.901+0000 7f0e37fff700 1 -- 172.21.15.27:0/2640385106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e380f5ef0 msgr2=0x7f0e381073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:18:32.905 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:32.901+0000 7f0e37fff700 1 --2- 172.21.15.27:0/2640385106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e380f5ef0 0x7f0e381073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:32.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:32.901+0000 7f0e37fff700 1 -- 172.21.15.27:0/2640385106 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e2c005040 con 0x7f0e380f55e0 2021-11-03T17:18:32.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:32.901+0000 7f0e377fe700 1 --2- 172.21.15.27:0/2640385106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e380f5ef0 0x7f0e381073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-03T17:18:32.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:32.901+0000 7f0e37fff700 1 --2- 172.21.15.27:0/2640385106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e380f55e0 0x7f0e38106ef0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f0e2800e7f0 tx=0x7f0e2800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:18:32.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:32.901+0000 7f0e34ff9700 1 -- 172.21.15.27:0/2640385106 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e280092e0 con 0x7f0e380f55e0 2021-11-03T17:18:32.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:32.901+0000 7f0e3e9e5700 1 -- 172.21.15.27:0/2640385106 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e38108470 con 0x7f0e380f55e0 2021-11-03T17:18:32.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:32.901+0000 7f0e34ff9700 1 -- 172.21.15.27:0/2640385106 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f0e2800f040 con 0x7f0e380f55e0 2021-11-03T17:18:32.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:32.901+0000 7f0e3e9e5700 1 -- 172.21.15.27:0/2640385106 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e38108a30 con 0x7f0e380f55e0 2021-11-03T17:18:32.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:32.901+0000 7f0e34ff9700 1 -- 172.21.15.27:0/2640385106 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e28014750 con 0x7f0e380f55e0 2021-11-03T17:18:32.909 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:32.902+0000 7f0e3e9e5700 1 -- 172.21.15.27:0/2640385106 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e38045ad0 con 0x7f0e380f55e0 2021-11-03T17:18:32.909 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:32.902+0000 7f0e34ff9700 1 -- 172.21.15.27:0/2640385106 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f0e2801b030 con 0x7f0e380f55e0 2021-11-03T17:18:32.910 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:32.903+0000 7f0e34ff9700 1 --2- 172.21.15.27:0/2640385106 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0e200640b0 0x7f0e20066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:18:32.910 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:32.903+0000 7f0e34ff9700 1 -- 172.21.15.27:0/2640385106 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f0e28080680 con 0x7f0e380f55e0 2021-11-03T17:18:32.911 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:32.903+0000 7f0e377fe700 1 --2- 172.21.15.27:0/2640385106 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0e200640b0 0x7f0e20066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:18:32.911 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:32.903+0000 7f0e377fe700 1 --2- 172.21.15.27:0/2640385106 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0e200640b0 0x7f0e20066570 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f0e2c005010 tx=0x7f0e2c016040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:18:32.912 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:32.907+0000 7f0e34ff9700 1 -- 172.21.15.27:0/2640385106 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f0e280529a0 con 0x7f0e380f55e0 2021-11-03T17:18:32.937 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:32 smithi027 ceph-mon[97244]: pgmap v99: 65 pgs: 65 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:18:32.937 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:32 smithi027 ceph-mon[97244]: osd.3 reported immediately failed by osd.1 2021-11-03T17:18:32.938 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:32 smithi027 ceph-mon[97244]: osd.3 failed (root=default,host=smithi115) (connection refused reported by osd.1) 2021-11-03T17:18:32.938 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:32 smithi027 ceph-mon[97244]: osd.3 reported immediately failed by osd.1 2021-11-03T17:18:32.938 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:32 smithi027 ceph-mon[97244]: osd.3 reported immediately failed by osd.2 2021-11-03T17:18:32.939 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:32 smithi027 ceph-mon[97244]: osd.3 reported immediately failed by osd.4 2021-11-03T17:18:32.939 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:32 smithi027 ceph-mon[97244]: osd.3 reported immediately failed by osd.5 2021-11-03T17:18:32.939 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:32 smithi027 ceph-mon[97244]: osd.3 reported immediately failed by osd.0 2021-11-03T17:18:32.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:32 smithi027 ceph-mon[97244]: osd.3 reported immediately failed by osd.2 2021-11-03T17:18:32.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:32 smithi027 ceph-mon[97244]: osd.3 reported immediately failed by osd.4 2021-11-03T17:18:32.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:32 smithi027 ceph-mon[97244]: osd.3 reported immediately failed by osd.5 2021-11-03T17:18:32.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:32 smithi027 ceph-mon[97244]: osd.3 reported immediately failed by osd.0 2021-11-03T17:18:32.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:32 smithi027 ceph-mon[97244]: osd.3 reported immediately failed by osd.5 2021-11-03T17:18:32.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:32 smithi027 ceph-mon[97244]: osd.3 reported immediately failed by osd.4 2021-11-03T17:18:32.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:32 smithi027 ceph-mon[97244]: osd.3 reported immediately failed by osd.5 2021-11-03T17:18:32.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:32 smithi027 ceph-mon[97244]: osd.3 reported immediately failed by osd.0 2021-11-03T17:18:32.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:32 smithi027 ceph-mon[97244]: osd.3 reported immediately failed by osd.1 2021-11-03T17:18:32.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:32 smithi027 ceph-mon[97244]: osd.3 reported immediately failed by osd.2 2021-11-03T17:18:32.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:32 smithi027 ceph-mon[97244]: osd.3 reported immediately failed by osd.4 2021-11-03T17:18:32.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:32 smithi027 ceph-mon[97244]: osd.3 reported immediately failed by osd.0 2021-11-03T17:18:32.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:32 smithi027 ceph-mon[97244]: osd.3 reported immediately failed by osd.1 2021-11-03T17:18:32.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:32 smithi027 ceph-mon[97244]: osd.3 reported immediately failed by osd.2 2021-11-03T17:18:32.943 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:32 smithi027 ceph-mon[97244]: osd.3 reported immediately failed by osd.2 2021-11-03T17:18:32.943 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:32 smithi027 ceph-mon[97244]: osd.3 reported immediately failed by osd.0 2021-11-03T17:18:32.943 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:32 smithi027 ceph-mon[97244]: osd.3 reported immediately failed by osd.5 2021-11-03T17:18:32.944 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:32 smithi027 ceph-mon[97244]: osd.3 reported immediately failed by osd.1 2021-11-03T17:18:32.984 INFO:journalctl@ceph.osd.3.smithi115.stdout:Nov 03 17:18:32 smithi115 bash[71047]: Error: container b1486ec1b6198cabe6909cf7ccb69c5e37e8b22aa59a25f3aa6b3e3a51107379 does not exist in database: no such container 2021-11-03T17:18:32.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:32 smithi115 ceph-mon[65453]: pgmap v99: 65 pgs: 65 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:18:32.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:32 smithi115 ceph-mon[65453]: osd.3 reported immediately failed by osd.1 2021-11-03T17:18:32.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:32 smithi115 ceph-mon[65453]: osd.3 failed (root=default,host=smithi115) (connection refused reported by osd.1) 2021-11-03T17:18:32.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:32 smithi115 ceph-mon[65453]: osd.3 reported immediately failed by osd.1 2021-11-03T17:18:32.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:32 smithi115 ceph-mon[65453]: osd.3 reported immediately failed by osd.2 2021-11-03T17:18:32.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:32 smithi115 ceph-mon[65453]: osd.3 reported immediately failed by osd.4 2021-11-03T17:18:32.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:32 smithi115 ceph-mon[65453]: osd.3 reported immediately failed by osd.5 2021-11-03T17:18:32.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:32 smithi115 ceph-mon[65453]: osd.3 reported immediately failed by osd.0 2021-11-03T17:18:32.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:32 smithi115 ceph-mon[65453]: osd.3 reported immediately failed by osd.2 2021-11-03T17:18:32.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:32 smithi115 ceph-mon[65453]: osd.3 reported immediately failed by osd.4 2021-11-03T17:18:32.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:32 smithi115 ceph-mon[65453]: osd.3 reported immediately failed by osd.5 2021-11-03T17:18:32.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:32 smithi115 ceph-mon[65453]: osd.3 reported immediately failed by osd.0 2021-11-03T17:18:32.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:32 smithi115 ceph-mon[65453]: osd.3 reported immediately failed by osd.5 2021-11-03T17:18:32.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:32 smithi115 ceph-mon[65453]: osd.3 reported immediately failed by osd.4 2021-11-03T17:18:32.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:32 smithi115 ceph-mon[65453]: osd.3 reported immediately failed by osd.5 2021-11-03T17:18:32.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:32 smithi115 ceph-mon[65453]: osd.3 reported immediately failed by osd.0 2021-11-03T17:18:32.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:32 smithi115 ceph-mon[65453]: osd.3 reported immediately failed by osd.1 2021-11-03T17:18:32.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:32 smithi115 ceph-mon[65453]: osd.3 reported immediately failed by osd.2 2021-11-03T17:18:32.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:32 smithi115 ceph-mon[65453]: osd.3 reported immediately failed by osd.4 2021-11-03T17:18:32.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:32 smithi115 ceph-mon[65453]: osd.3 reported immediately failed by osd.0 2021-11-03T17:18:32.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:32 smithi115 ceph-mon[65453]: osd.3 reported immediately failed by osd.1 2021-11-03T17:18:32.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:32 smithi115 ceph-mon[65453]: osd.3 reported immediately failed by osd.2 2021-11-03T17:18:32.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:32 smithi115 ceph-mon[65453]: osd.3 reported immediately failed by osd.2 2021-11-03T17:18:32.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:32 smithi115 ceph-mon[65453]: osd.3 reported immediately failed by osd.0 2021-11-03T17:18:32.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:32 smithi115 ceph-mon[65453]: osd.3 reported immediately failed by osd.5 2021-11-03T17:18:32.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:32 smithi115 ceph-mon[65453]: osd.3 reported immediately failed by osd.1 2021-11-03T17:18:33.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.089+0000 7f0e3e9e5700 1 -- 172.21.15.27:0/2640385106 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e380fad60 con 0x7f0e200640b0 2021-11-03T17:18:33.086 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.091+0000 7f0e34ff9700 1 -- 172.21.15.27:0/2640385106 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+320 (secure 0 0 0) 0x7f0e380fad60 con 0x7f0e200640b0 2021-11-03T17:18:33.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.092+0000 7f0e3e9e5700 1 -- 172.21.15.27:0/2640385106 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0e200640b0 msgr2=0x7f0e20066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:18:33.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.092+0000 7f0e3e9e5700 1 --2- 172.21.15.27:0/2640385106 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0e200640b0 0x7f0e20066570 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f0e2c005010 tx=0x7f0e2c016040).stop 2021-11-03T17:18:33.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.092+0000 7f0e3e9e5700 1 -- 172.21.15.27:0/2640385106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e380f55e0 msgr2=0x7f0e38106ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:18:33.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.092+0000 7f0e3e9e5700 1 --2- 172.21.15.27:0/2640385106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e380f55e0 0x7f0e38106ef0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f0e2800e7f0 tx=0x7f0e2800b040).stop 2021-11-03T17:18:33.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.093+0000 7f0e3e9e5700 1 -- 172.21.15.27:0/2640385106 shutdown_connections 2021-11-03T17:18:33.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.093+0000 7f0e3e9e5700 1 --2- 172.21.15.27:0/2640385106 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0e200640b0 0x7f0e20066570 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:33.090 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.093+0000 7f0e3e9e5700 1 --2- 172.21.15.27:0/2640385106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e380f55e0 0x7f0e38106ef0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:33.090 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.093+0000 7f0e3e9e5700 1 --2- 172.21.15.27:0/2640385106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e380f5ef0 0x7f0e381073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:33.090 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.093+0000 7f0e3e9e5700 1 -- 172.21.15.27:0/2640385106 >> 172.21.15.27:0/2640385106 conn(0x7f0e380f0de0 msgr2=0x7f0e3805ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:18:33.090 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.093+0000 7f0e3e9e5700 1 -- 172.21.15.27:0/2640385106 shutdown_connections 2021-11-03T17:18:33.091 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.093+0000 7f0e3e9e5700 1 -- 172.21.15.27:0/2640385106 wait complete. 2021-11-03T17:18:33.098 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:18:33.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.193+0000 7f9f5fa5c700 1 -- 172.21.15.27:0/174857282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f580f66c0 msgr2=0x7f9f580f6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:18:33.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.193+0000 7f9f5fa5c700 1 --2- 172.21.15.27:0/174857282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f580f66c0 0x7f9f580f6ae0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f9f54004660 tx=0x7f9f54009b30).stop 2021-11-03T17:18:33.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.193+0000 7f9f5fa5c700 1 -- 172.21.15.27:0/174857282 shutdown_connections 2021-11-03T17:18:33.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.193+0000 7f9f5fa5c700 1 --2- 172.21.15.27:0/174857282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f580f7960 0x7f9f580f7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:33.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.193+0000 7f9f5fa5c700 1 --2- 172.21.15.27:0/174857282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f580f66c0 0x7f9f580f6ae0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:33.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.193+0000 7f9f5fa5c700 1 -- 172.21.15.27:0/174857282 >> 172.21.15.27:0/174857282 conn(0x7f9f580f1e20 msgr2=0x7f9f580f4240 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:18:33.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.193+0000 7f9f5fa5c700 1 -- 172.21.15.27:0/174857282 shutdown_connections 2021-11-03T17:18:33.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.193+0000 7f9f5fa5c700 1 -- 172.21.15.27:0/174857282 wait complete. 2021-11-03T17:18:33.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.194+0000 7f9f5fa5c700 1 Processor -- start 2021-11-03T17:18:33.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.194+0000 7f9f5fa5c700 1 -- start start 2021-11-03T17:18:33.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.194+0000 7f9f5fa5c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f580f66c0 0x7f9f58106f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:18:33.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.194+0000 7f9f5fa5c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f580f7960 0x7f9f58107400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:18:33.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.194+0000 7f9f5fa5c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f581079b0 con 0x7f9f580f66c0 2021-11-03T17:18:33.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.194+0000 7f9f5fa5c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f58107af0 con 0x7f9f580f7960 2021-11-03T17:18:33.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.194+0000 7f9f5d7f8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f580f66c0 0x7f9f58106f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:18:33.196 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.194+0000 7f9f5d7f8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f580f66c0 0x7f9f58106f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56538/0 (socket says 172.21.15.27:56538) 2021-11-03T17:18:33.196 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.194+0000 7f9f5d7f8700 1 -- 172.21.15.27:0/2894173522 learned_addr learned my addr 172.21.15.27:0/2894173522 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:18:33.196 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.194+0000 7f9f5cff7700 1 --2- 172.21.15.27:0/2894173522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f580f7960 0x7f9f58107400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:18:33.196 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.195+0000 7f9f5d7f8700 1 -- 172.21.15.27:0/2894173522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f580f7960 msgr2=0x7f9f58107400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:18:33.196 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.195+0000 7f9f5d7f8700 1 --2- 172.21.15.27:0/2894173522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f580f7960 0x7f9f58107400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:33.197 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.195+0000 7f9f5d7f8700 1 -- 172.21.15.27:0/2894173522 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f54005040 con 0x7f9f580f66c0 2021-11-03T17:18:33.197 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.195+0000 7f9f5d7f8700 1 --2- 172.21.15.27:0/2894173522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f580f66c0 0x7f9f58106f10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f9f54014040 tx=0x7f9f5400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:18:33.197 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.195+0000 7f9f4dffb700 1 -- 172.21.15.27:0/2894173522 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f54019070 con 0x7f9f580f66c0 2021-11-03T17:18:33.197 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.195+0000 7f9f4dffb700 1 -- 172.21.15.27:0/2894173522 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f9f54007e70 con 0x7f9f580f66c0 2021-11-03T17:18:33.198 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.195+0000 7f9f4dffb700 1 -- 172.21.15.27:0/2894173522 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f54014d10 con 0x7f9f580f66c0 2021-11-03T17:18:33.198 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.195+0000 7f9f5fa5c700 1 -- 172.21.15.27:0/2894173522 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f58108430 con 0x7f9f580f66c0 2021-11-03T17:18:33.198 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.195+0000 7f9f5fa5c700 1 -- 172.21.15.27:0/2894173522 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f58108990 con 0x7f9f580f66c0 2021-11-03T17:18:33.198 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.196+0000 7f9f5fa5c700 1 -- 172.21.15.27:0/2894173522 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f58045ad0 con 0x7f9f580f66c0 2021-11-03T17:18:33.199 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.196+0000 7f9f4dffb700 1 -- 172.21.15.27:0/2894173522 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f9f54025070 con 0x7f9f580f66c0 2021-11-03T17:18:33.202 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.197+0000 7f9f4dffb700 1 --2- 172.21.15.27:0/2894173522 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9f440640b0 0x7f9f44066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:18:33.202 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.197+0000 7f9f4dffb700 1 -- 172.21.15.27:0/2894173522 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f9f540816e0 con 0x7f9f580f66c0 2021-11-03T17:18:33.202 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.197+0000 7f9f5cff7700 1 --2- 172.21.15.27:0/2894173522 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9f440640b0 0x7f9f44066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:18:33.203 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.197+0000 7f9f5cff7700 1 --2- 172.21.15.27:0/2894173522 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9f440640b0 0x7f9f44066570 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f9f48009fd0 tx=0x7f9f4800b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:18:33.203 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.201+0000 7f9f4dffb700 1 -- 172.21.15.27:0/2894173522 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f9f54053a40 con 0x7f9f580f66c0 2021-11-03T17:18:33.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.375+0000 7f9f5fa5c700 1 -- 172.21.15.27:0/2894173522 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f580ff000 con 0x7f9f440640b0 2021-11-03T17:18:33.385 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.389+0000 7f9f4dffb700 1 -- 172.21.15.27:0/2894173522 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f9f580ff000 con 0x7f9f440640b0 2021-11-03T17:18:33.386 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:18:33.386 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (10m) 20s ago 11m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:18:33.386 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (108s) 20s ago 11m 7205k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:18:33.386 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (98s) 95s ago 10m 18.1M - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:18:33.387 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (9m) 20s ago 11m 38.9M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:18:33.387 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (3m) 20s ago 6m 15.4M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:18:33.387 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (6m) 20s ago 6m 3836M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:18:33.387 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (2m) 95s ago 6m 13.5M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:18:33.388 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (3m) 95s ago 6m 15.2M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:18:33.388 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (3m) 20s ago 12m 470M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:18:33.388 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (3m) 95s ago 10m 410M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:18:33.388 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (2m) 20s ago 12m 81.2M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:18:33.389 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (2m) 95s ago 10m 40.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:18:33.389 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (10m) 20s ago 10m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:18:33.389 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (10m) 95s ago 10m 18.2M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:18:33.389 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (86s) 20s ago 9m 243M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:18:33.390 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (56s) 20s ago 9m 270M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:18:33.390 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (26s) 20s ago 9m 11.8M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:18:33.390 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (8m) 95s ago 8m 922M 4096M 16.2.4 8d91d370c2b8 b1486ec1b619 2021-11-03T17:18:33.390 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (8m) 95s ago 8m 742M 4096M 16.2.4 8d91d370c2b8 8b91230c69e5 2021-11-03T17:18:33.391 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (8m) 95s ago 8m 844M 4096M 16.2.4 8d91d370c2b8 4a54c4d6b6f5 2021-11-03T17:18:33.391 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (9m) 20s ago 10m 45.7M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:18:33.392 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.391+0000 7f9f5fa5c700 1 -- 172.21.15.27:0/2894173522 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9f440640b0 msgr2=0x7f9f44066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:18:33.392 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.391+0000 7f9f5fa5c700 1 --2- 172.21.15.27:0/2894173522 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9f440640b0 0x7f9f44066570 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f9f48009fd0 tx=0x7f9f4800b040).stop 2021-11-03T17:18:33.392 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.391+0000 7f9f5fa5c700 1 -- 172.21.15.27:0/2894173522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f580f66c0 msgr2=0x7f9f58106f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:18:33.393 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.391+0000 7f9f5fa5c700 1 --2- 172.21.15.27:0/2894173522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f580f66c0 0x7f9f58106f10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f9f54014040 tx=0x7f9f5400b040).stop 2021-11-03T17:18:33.393 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.391+0000 7f9f5fa5c700 1 -- 172.21.15.27:0/2894173522 shutdown_connections 2021-11-03T17:18:33.393 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.391+0000 7f9f5fa5c700 1 --2- 172.21.15.27:0/2894173522 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9f440640b0 0x7f9f44066570 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:33.393 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.391+0000 7f9f5fa5c700 1 --2- 172.21.15.27:0/2894173522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f580f66c0 0x7f9f58106f10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:33.393 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.391+0000 7f9f5fa5c700 1 --2- 172.21.15.27:0/2894173522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f580f7960 0x7f9f58107400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:33.394 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.391+0000 7f9f5fa5c700 1 -- 172.21.15.27:0/2894173522 >> 172.21.15.27:0/2894173522 conn(0x7f9f580f1e20 msgr2=0x7f9f580fb3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:18:33.394 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.392+0000 7f9f5fa5c700 1 -- 172.21.15.27:0/2894173522 shutdown_connections 2021-11-03T17:18:33.394 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.392+0000 7f9f5fa5c700 1 -- 172.21.15.27:0/2894173522 wait complete. 2021-11-03T17:18:33.488 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.491+0000 7f7f57f13700 1 -- 172.21.15.27:0/2866289172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f500f55e0 msgr2=0x7f7f500f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:18:33.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.491+0000 7f7f57f13700 1 --2- 172.21.15.27:0/2866289172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f500f55e0 0x7f7f500f5a00 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f7f4c004660 tx=0x7f7f4c009b30).stop 2021-11-03T17:18:33.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.491+0000 7f7f57f13700 1 -- 172.21.15.27:0/2866289172 shutdown_connections 2021-11-03T17:18:33.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.491+0000 7f7f57f13700 1 --2- 172.21.15.27:0/2866289172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f500f5ef0 0x7f7f500f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:33.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.491+0000 7f7f57f13700 1 --2- 172.21.15.27:0/2866289172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f500f55e0 0x7f7f500f5a00 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:33.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.491+0000 7f7f57f13700 1 -- 172.21.15.27:0/2866289172 >> 172.21.15.27:0/2866289172 conn(0x7f7f500f0de0 msgr2=0x7f7f500f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:18:33.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.491+0000 7f7f57f13700 1 -- 172.21.15.27:0/2866289172 shutdown_connections 2021-11-03T17:18:33.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.491+0000 7f7f57f13700 1 -- 172.21.15.27:0/2866289172 wait complete. 2021-11-03T17:18:33.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.492+0000 7f7f57f13700 1 Processor -- start 2021-11-03T17:18:33.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.492+0000 7f7f57f13700 1 -- start start 2021-11-03T17:18:33.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.492+0000 7f7f57f13700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f500f55e0 0x7f7f50109bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:18:33.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.492+0000 7f7f57f13700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f500f5ef0 0x7f7f5010c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:18:33.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.492+0000 7f7f57f13700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f50060540 con 0x7f7f500f55e0 2021-11-03T17:18:33.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.492+0000 7f7f57f13700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f50060680 con 0x7f7f500f5ef0 2021-11-03T17:18:33.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.492+0000 7f7f55caf700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f500f55e0 0x7f7f50109bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:18:33.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.492+0000 7f7f55caf700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f500f55e0 0x7f7f50109bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56552/0 (socket says 172.21.15.27:56552) 2021-11-03T17:18:33.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.492+0000 7f7f55caf700 1 -- 172.21.15.27:0/1625949622 learned_addr learned my addr 172.21.15.27:0/1625949622 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:18:33.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.492+0000 7f7f554ae700 1 --2- 172.21.15.27:0/1625949622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f500f5ef0 0x7f7f5010c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:18:33.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.492+0000 7f7f55caf700 1 -- 172.21.15.27:0/1625949622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f500f5ef0 msgr2=0x7f7f5010c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:18:33.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.492+0000 7f7f55caf700 1 --2- 172.21.15.27:0/1625949622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f500f5ef0 0x7f7f5010c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:33.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.492+0000 7f7f55caf700 1 -- 172.21.15.27:0/1625949622 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f4c005040 con 0x7f7f500f55e0 2021-11-03T17:18:33.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.492+0000 7f7f55caf700 1 --2- 172.21.15.27:0/1625949622 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f500f55e0 0x7f7f50109bb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f7f4c000c00 tx=0x7f7f4c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:18:33.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.492+0000 7f7f467fc700 1 -- 172.21.15.27:0/1625949622 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f4c019070 con 0x7f7f500f55e0 2021-11-03T17:18:33.494 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.492+0000 7f7f467fc700 1 -- 172.21.15.27:0/1625949622 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f7f4c007e70 con 0x7f7f500f55e0 2021-11-03T17:18:33.494 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.492+0000 7f7f57f13700 1 -- 172.21.15.27:0/1625949622 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f5010c660 con 0x7f7f500f55e0 2021-11-03T17:18:33.494 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.492+0000 7f7f467fc700 1 -- 172.21.15.27:0/1625949622 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f4c014ce0 con 0x7f7f500f55e0 2021-11-03T17:18:33.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.493+0000 7f7f57f13700 1 -- 172.21.15.27:0/1625949622 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f5010cb90 con 0x7f7f500f55e0 2021-11-03T17:18:33.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.493+0000 7f7f57f13700 1 -- 172.21.15.27:0/1625949622 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f500f4a70 con 0x7f7f500f55e0 2021-11-03T17:18:33.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.494+0000 7f7f467fc700 1 -- 172.21.15.27:0/1625949622 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f7f4c025070 con 0x7f7f500f55e0 2021-11-03T17:18:33.498 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.494+0000 7f7f467fc700 1 --2- 172.21.15.27:0/1625949622 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7f3c0640b0 0x7f7f3c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:18:33.498 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.494+0000 7f7f467fc700 1 -- 172.21.15.27:0/1625949622 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f7f4c081750 con 0x7f7f500f55e0 2021-11-03T17:18:33.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.494+0000 7f7f554ae700 1 --2- 172.21.15.27:0/1625949622 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7f3c0640b0 0x7f7f3c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:18:33.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.495+0000 7f7f554ae700 1 --2- 172.21.15.27:0/1625949622 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7f3c0640b0 0x7f7f3c066570 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f7f40009fd0 tx=0x7f7f4000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:18:33.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.498+0000 7f7f467fc700 1 -- 172.21.15.27:0/1625949622 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f7f4c053ab0 con 0x7f7f500f55e0 2021-11-03T17:18:33.736 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.740+0000 7f7f57f13700 1 -- 172.21.15.27:0/1625949622 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7f50045ad0 con 0x7f7f500f55e0 2021-11-03T17:18:33.736 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.740+0000 7f7f467fc700 1 -- 172.21.15.27:0/1625949622 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+804 (secure 0 0 0) 0x7f7f4c023070 con 0x7f7f500f55e0 2021-11-03T17:18:33.737 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:18:33.737 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:18:33.738 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:18:33.738 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:18:33.738 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:18:33.738 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:18:33.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:18:33.739 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:18:33.739 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 2, 2021-11-03T17:18:33.739 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 3 2021-11-03T17:18:33.739 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:18:33.740 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:18:33.740 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:18:33.740 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:18:33.741 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:18:33.741 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 6, 2021-11-03T17:18:33.741 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 7 2021-11-03T17:18:33.741 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:18:33.742 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:18:33.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.742+0000 7f7f57f13700 1 -- 172.21.15.27:0/1625949622 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7f3c0640b0 msgr2=0x7f7f3c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:18:33.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.742+0000 7f7f57f13700 1 --2- 172.21.15.27:0/1625949622 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7f3c0640b0 0x7f7f3c066570 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f7f40009fd0 tx=0x7f7f4000b040).stop 2021-11-03T17:18:33.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.742+0000 7f7f57f13700 1 -- 172.21.15.27:0/1625949622 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f500f55e0 msgr2=0x7f7f50109bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:18:33.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.742+0000 7f7f57f13700 1 --2- 172.21.15.27:0/1625949622 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f500f55e0 0x7f7f50109bb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f7f4c000c00 tx=0x7f7f4c00b040).stop 2021-11-03T17:18:33.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.742+0000 7f7f57f13700 1 -- 172.21.15.27:0/1625949622 shutdown_connections 2021-11-03T17:18:33.744 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.742+0000 7f7f57f13700 1 --2- 172.21.15.27:0/1625949622 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7f3c0640b0 0x7f7f3c066570 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:33.744 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.742+0000 7f7f57f13700 1 --2- 172.21.15.27:0/1625949622 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f500f55e0 0x7f7f50109bb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:33.744 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.742+0000 7f7f57f13700 1 --2- 172.21.15.27:0/1625949622 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f500f5ef0 0x7f7f5010c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:33.745 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.742+0000 7f7f57f13700 1 -- 172.21.15.27:0/1625949622 >> 172.21.15.27:0/1625949622 conn(0x7f7f500f0de0 msgr2=0x7f7f500fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:18:33.745 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.742+0000 7f7f57f13700 1 -- 172.21.15.27:0/1625949622 shutdown_connections 2021-11-03T17:18:33.745 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.742+0000 7f7f57f13700 1 -- 172.21.15.27:0/1625949622 wait complete. 2021-11-03T17:18:33.839 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.843+0000 7f03577c2700 1 -- 172.21.15.27:0/1012103075 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03500f5ef0 msgr2=0x7f03500f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:18:33.840 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.843+0000 7f03577c2700 1 --2- 172.21.15.27:0/1012103075 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03500f5ef0 0x7f03500f6920 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f034c004660 tx=0x7f034c009b30).stop 2021-11-03T17:18:33.840 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.843+0000 7f03577c2700 1 -- 172.21.15.27:0/1012103075 shutdown_connections 2021-11-03T17:18:33.840 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.843+0000 7f03577c2700 1 --2- 172.21.15.27:0/1012103075 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03500f5ef0 0x7f03500f6920 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:33.841 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.843+0000 7f03577c2700 1 --2- 172.21.15.27:0/1012103075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03500f55e0 0x7f03500f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:33.841 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.843+0000 7f03577c2700 1 -- 172.21.15.27:0/1012103075 >> 172.21.15.27:0/1012103075 conn(0x7f03500f0de0 msgr2=0x7f03500f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:18:33.841 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.843+0000 7f03577c2700 1 -- 172.21.15.27:0/1012103075 shutdown_connections 2021-11-03T17:18:33.841 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.843+0000 7f03577c2700 1 -- 172.21.15.27:0/1012103075 wait complete. 2021-11-03T17:18:33.841 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.844+0000 7f03577c2700 1 Processor -- start 2021-11-03T17:18:33.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.844+0000 7f03577c2700 1 -- start start 2021-11-03T17:18:33.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.844+0000 7f03577c2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03500f55e0 0x7f035010c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:18:33.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.844+0000 7f03577c2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03500f5ef0 0x7f035010e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:18:33.843 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.844+0000 7f03577c2700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03500603b0 con 0x7f03500f5ef0 2021-11-03T17:18:33.843 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.844+0000 7f03577c2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03500604f0 con 0x7f03500f55e0 2021-11-03T17:18:33.843 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.844+0000 7f0354d5d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03500f5ef0 0x7f035010e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:18:33.843 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.844+0000 7f0354d5d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03500f5ef0 0x7f035010e510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56564/0 (socket says 172.21.15.27:56564) 2021-11-03T17:18:33.844 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.844+0000 7f0354d5d700 1 -- 172.21.15.27:0/2747728915 learned_addr learned my addr 172.21.15.27:0/2747728915 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:18:33.844 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.844+0000 7f035555e700 1 --2- 172.21.15.27:0/2747728915 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03500f55e0 0x7f035010c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:18:33.846 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.844+0000 7f0354d5d700 1 -- 172.21.15.27:0/2747728915 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03500f55e0 msgr2=0x7f035010c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:18:33.846 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.844+0000 7f0354d5d700 1 --2- 172.21.15.27:0/2747728915 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03500f55e0 0x7f035010c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:33.846 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.844+0000 7f0354d5d700 1 -- 172.21.15.27:0/2747728915 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f034c005040 con 0x7f03500f5ef0 2021-11-03T17:18:33.847 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.844+0000 7f035555e700 1 --2- 172.21.15.27:0/2747728915 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03500f55e0 0x7f035010c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-03T17:18:33.847 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.845+0000 7f0354d5d700 1 --2- 172.21.15.27:0/2747728915 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03500f5ef0 0x7f035010e510 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f034c000c00 tx=0x7f034c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:18:33.847 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.845+0000 7f0345ffb700 1 -- 172.21.15.27:0/2747728915 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f034c019070 con 0x7f03500f5ef0 2021-11-03T17:18:33.847 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.845+0000 7f0345ffb700 1 -- 172.21.15.27:0/2747728915 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f034c007e70 con 0x7f03500f5ef0 2021-11-03T17:18:33.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.845+0000 7f03577c2700 1 -- 172.21.15.27:0/2747728915 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f035010ea00 con 0x7f03500f5ef0 2021-11-03T17:18:33.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.845+0000 7f0345ffb700 1 -- 172.21.15.27:0/2747728915 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f034c014ce0 con 0x7f03500f5ef0 2021-11-03T17:18:33.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.845+0000 7f03577c2700 1 -- 172.21.15.27:0/2747728915 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f035010edd0 con 0x7f03500f5ef0 2021-11-03T17:18:33.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.846+0000 7f03577c2700 1 -- 172.21.15.27:0/2747728915 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03500f4a70 con 0x7f03500f5ef0 2021-11-03T17:18:33.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.847+0000 7f0345ffb700 1 -- 172.21.15.27:0/2747728915 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f034c025070 con 0x7f03500f5ef0 2021-11-03T17:18:33.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.847+0000 7f0345ffb700 1 --2- 172.21.15.27:0/2747728915 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f033c0640b0 0x7f033c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:18:33.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.848+0000 7f0345ffb700 1 -- 172.21.15.27:0/2747728915 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f034c081ba0 con 0x7f03500f5ef0 2021-11-03T17:18:33.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.848+0000 7f035555e700 1 --2- 172.21.15.27:0/2747728915 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f033c0640b0 0x7f033c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:18:33.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.848+0000 7f035555e700 1 --2- 172.21.15.27:0/2747728915 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f033c0640b0 0x7f033c066570 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f0350060140 tx=0x7f034000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:18:33.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:33.851+0000 7f0345ffb700 1 -- 172.21.15.27:0/2747728915 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f034c053e40 con 0x7f03500f5ef0 2021-11-03T17:18:33.955 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:33 smithi115 ceph-mon[65453]: Health check failed: 1 osds down (OSD_DOWN) 2021-11-03T17:18:33.955 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:33 smithi115 ceph-mon[65453]: osdmap e69: 6 total, 5 up, 6 in 2021-11-03T17:18:34.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:33 smithi027 ceph-mon[97244]: Health check failed: 1 osds down (OSD_DOWN) 2021-11-03T17:18:34.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:33 smithi027 ceph-mon[97244]: osdmap e69: 6 total, 5 up, 6 in 2021-11-03T17:18:34.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:34.055+0000 7f03577c2700 1 -- 172.21.15.27:0/2747728915 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0350045ad0 con 0x7f03500f5ef0 2021-11-03T17:18:34.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:34.055+0000 7f0345ffb700 1 -- 172.21.15.27:0/2747728915 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) v1 ==== 76+0+1795 (secure 0 0 0) 0x7f034c023070 con 0x7f03500f5ef0 2021-11-03T17:18:34.052 INFO:teuthology.orchestra.run.smithi027.stdout:e28 2021-11-03T17:18:34.052 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:18:34.052 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:18:34.053 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:18:34.053 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:18:34.053 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:18:34.053 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:18:34.054 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 28 2021-11-03T17:18:34.054 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:18:34.054 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:18:34.055 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:16:13.536828+0000 2021-11-03T17:18:34.055 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:18:34.055 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:18:34.055 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:18:34.056 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:18:34.056 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:18:34.056 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:18:34.057 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:18:34.058 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:18:34.058 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:18:34.059 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 2 2021-11-03T17:18:34.059 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:18:34.060 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:18:34.061 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:18:34.061 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:18:34.061 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:18:34.061 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:18:34.062 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:18:34.062 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:18:34.062 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:18:34.062 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:18:34.063 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 3 export targets 0 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:18:34.063 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:18:34.063 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:18:34.063 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:18:34.064 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:18:34.064 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:18:34.064 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:18:34.064 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:18:34.065 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:34.057+0000 7f03577c2700 1 -- 172.21.15.27:0/2747728915 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f033c0640b0 msgr2=0x7f033c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:18:34.066 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:34.057+0000 7f03577c2700 1 --2- 172.21.15.27:0/2747728915 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f033c0640b0 0x7f033c066570 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f0350060140 tx=0x7f034000b040).stop 2021-11-03T17:18:34.066 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:34.057+0000 7f03577c2700 1 -- 172.21.15.27:0/2747728915 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03500f5ef0 msgr2=0x7f035010e510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:18:34.066 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:34.057+0000 7f03577c2700 1 --2- 172.21.15.27:0/2747728915 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03500f5ef0 0x7f035010e510 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f034c000c00 tx=0x7f034c00b040).stop 2021-11-03T17:18:34.066 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:34.057+0000 7f03577c2700 1 -- 172.21.15.27:0/2747728915 shutdown_connections 2021-11-03T17:18:34.067 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:34.057+0000 7f03577c2700 1 --2- 172.21.15.27:0/2747728915 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f033c0640b0 0x7f033c066570 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:34.067 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:34.057+0000 7f03577c2700 1 --2- 172.21.15.27:0/2747728915 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03500f55e0 0x7f035010c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:34.067 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:34.057+0000 7f03577c2700 1 --2- 172.21.15.27:0/2747728915 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03500f5ef0 0x7f035010e510 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:18:34.067 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:34.057+0000 7f03577c2700 1 -- 172.21.15.27:0/2747728915 >> 172.21.15.27:0/2747728915 conn(0x7f03500f0de0 msgr2=0x7f03500fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:18:34.068 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:34.057+0000 7f03577c2700 1 -- 172.21.15.27:0/2747728915 shutdown_connections 2021-11-03T17:18:34.068 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:18:34.057+0000 7f03577c2700 1 -- 172.21.15.27:0/2747728915 wait complete. 2021-11-03T17:18:34.068 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 28 2021-11-03T17:18:34.234 INFO:journalctl@ceph.osd.3.smithi115.stdout:Nov 03 17:18:33 smithi115 systemd[1]: ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c@osd.3.service: Succeeded. 2021-11-03T17:18:34.234 INFO:journalctl@ceph.osd.3.smithi115.stdout:Nov 03 17:18:33 smithi115 systemd[1]: Stopped Ceph osd.3 for 32ab977e-3cc8-11ec-8c28-001a4aab830c. 2021-11-03T17:18:34.235 INFO:journalctl@ceph.osd.3.smithi115.stdout:Nov 03 17:18:34 smithi115 systemd[1]: Starting Ceph osd.3 for 32ab977e-3cc8-11ec-8c28-001a4aab830c... 2021-11-03T17:18:34.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:34 smithi115 ceph-mon[65453]: from='client.34226 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:18:34.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:34 smithi115 ceph-mon[65453]: from='client.34230 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:18:34.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:34 smithi115 ceph-mon[65453]: osdmap e70: 6 total, 5 up, 6 in 2021-11-03T17:18:34.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:34 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1625949622' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:18:34.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:34 smithi115 ceph-mon[65453]: pgmap v102: 65 pgs: 16 stale+active+clean, 49 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:18:34.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:34 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2747728915' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:18:35.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:34 smithi027 ceph-mon[97244]: from='client.34226 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:18:35.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:34 smithi027 ceph-mon[97244]: from='client.34230 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:18:35.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:34 smithi027 ceph-mon[97244]: osdmap e70: 6 total, 5 up, 6 in 2021-11-03T17:18:35.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:34 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1625949622' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:18:35.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:34 smithi027 ceph-mon[97244]: pgmap v102: 65 pgs: 16 stale+active+clean, 49 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:18:35.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:34 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2747728915' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:18:35.484 INFO:journalctl@ceph.osd.3.smithi115.stdout:Nov 03 17:18:35 smithi115 bash[71527]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2021-11-03T17:18:35.484 INFO:journalctl@ceph.osd.3.smithi115.stdout:Nov 03 17:18:35 smithi115 bash[71527]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --dev /dev/mapper/vg_nvme-lv_4 --path /var/lib/ceph/osd/ceph-3 --no-mon-config 2021-11-03T17:18:35.484 INFO:journalctl@ceph.osd.3.smithi115.stdout:Nov 03 17:18:35 smithi115 bash[71527]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2021-11-03T17:18:35.485 INFO:journalctl@ceph.osd.3.smithi115.stdout:Nov 03 17:18:35 smithi115 bash[71527]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2021-11-03T17:18:35.485 INFO:journalctl@ceph.osd.3.smithi115.stdout:Nov 03 17:18:35 smithi115 bash[71527]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2021-11-03T17:18:35.485 INFO:journalctl@ceph.osd.3.smithi115.stdout:Nov 03 17:18:35 smithi115 bash[71527]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2021-11-03T17:18:35.485 INFO:journalctl@ceph.osd.3.smithi115.stdout:Nov 03 17:18:35 smithi115 bash[71527]: --> ceph-volume raw activate successful for osd ID: 3 2021-11-03T17:18:36.202 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:35 smithi115 ceph-mon[65453]: Health check failed: Degraded data redundancy: 1081/52479 objects degraded (2.060%), 3 pgs degraded (PG_DEGRADED) 2021-11-03T17:18:36.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:35 smithi027 ceph-mon[97244]: Health check failed: Degraded data redundancy: 1081/52479 objects degraded (2.060%), 3 pgs degraded (PG_DEGRADED) 2021-11-03T17:18:36.484 INFO:journalctl@ceph.osd.3.smithi115.stdout:Nov 03 17:18:36 smithi115 ceph-osd[72048]: -- 172.21.15.115:0/1740002848 <== mon.1 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b08bd8e340 con 0x55b08bd54c00 2021-11-03T17:18:36.484 INFO:journalctl@ceph.osd.3.smithi115.stdout:Nov 03 17:18:36 smithi115 bash[71527]: 731e672c66d0490ed2790e65a3257952775a8ae95e6b986be205ff2bc8e8884a 2021-11-03T17:18:36.485 INFO:journalctl@ceph.osd.3.smithi115.stdout:Nov 03 17:18:36 smithi115 systemd[1]: Started Ceph osd.3 for 32ab977e-3cc8-11ec-8c28-001a4aab830c. 2021-11-03T17:18:36.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:36 smithi115 ceph-mon[65453]: pgmap v103: 65 pgs: 3 active+undersized+degraded, 15 stale+active+clean, 1 active+undersized, 46 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1081/52479 objects degraded (2.060%) 2021-11-03T17:18:36.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:18:36.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:18:36.985 INFO:journalctl@ceph.osd.3.smithi115.stdout:Nov 03 17:18:36 smithi115 conmon[72025]: 2021-11-03T17:18:36.730+0000 7fa5470210c0 -1 Falling back to public interface 2021-11-03T17:18:37.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:36 smithi027 ceph-mon[97244]: pgmap v103: 65 pgs: 3 active+undersized+degraded, 15 stale+active+clean, 1 active+undersized, 46 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1081/52479 objects degraded (2.060%) 2021-11-03T17:18:37.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:18:37.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:18:38.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:37 smithi115 ceph-mon[65453]: poll took 6855.159 ms: 1 events 2021-11-03T17:18:38.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:37 smithi027 ceph-mon[97244]: poll took 6855.159 ms: 1 events 2021-11-03T17:18:39.173 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:38 smithi115 ceph-mon[65453]: pgmap v104: 65 pgs: 37 active+undersized+degraded, 1 active+undersized, 27 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 9826/52479 objects degraded (18.724%) 2021-11-03T17:18:39.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:38 smithi027 ceph-mon[97244]: pgmap v104: 65 pgs: 37 active+undersized+degraded, 1 active+undersized, 27 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 9826/52479 objects degraded (18.724%) 2021-11-03T17:18:40.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:40 smithi115 ceph-mon[65453]: poll took 2514.975 ms: 1 events 2021-11-03T17:18:40.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:18:40.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:18:40.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:18:40.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:18:40.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:18:40.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:18:40.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2021-11-03T17:18:40.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:40 smithi115 ceph-mon[65453]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2021-11-03T17:18:40.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:40 smithi115 ceph-mon[65453]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2021-11-03T17:18:40.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:40 smithi115 ceph-mon[65453]: pgmap v105: 65 pgs: 37 active+undersized+degraded, 1 active+undersized, 27 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 9826/52479 objects degraded (18.724%) 2021-11-03T17:18:40.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:40 smithi027 ceph-mon[97244]: poll took 2514.975 ms: 1 events 2021-11-03T17:18:40.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:18:40.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:18:40.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:18:40.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:18:40.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:18:40.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:18:40.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2021-11-03T17:18:40.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:40 smithi027 ceph-mon[97244]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2021-11-03T17:18:40.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:40 smithi027 ceph-mon[97244]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2021-11-03T17:18:40.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:40 smithi027 ceph-mon[97244]: pgmap v105: 65 pgs: 37 active+undersized+degraded, 1 active+undersized, 27 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 9826/52479 objects degraded (18.724%) 2021-11-03T17:18:41.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:18:42.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:18:42.937 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:42 smithi027 ceph-mon[97244]: pgmap v106: 65 pgs: 37 active+undersized+degraded, 1 active+undersized, 27 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 9826/52479 objects degraded (18.724%) 2021-11-03T17:18:42.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:42 smithi115 ceph-mon[65453]: pgmap v106: 65 pgs: 37 active+undersized+degraded, 1 active+undersized, 27 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 9826/52479 objects degraded (18.724%) 2021-11-03T17:18:43.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:43 smithi115 ceph-mon[65453]: from='osd.3 [v2:172.21.15.115:6800/679677417,v1:172.21.15.115:6801/679677417]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2021-11-03T17:18:43.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:43 smithi115 ceph-mon[65453]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2021-11-03T17:18:43.985 INFO:journalctl@ceph.osd.3.smithi115.stdout:Nov 03 17:18:43 smithi115 conmon[72025]: 2021-11-03T17:18:43.737+0000 7fa5470210c0 -1 osd.3 68 log_to_monitors {default=true} 2021-11-03T17:18:43.985 INFO:journalctl@ceph.osd.3.smithi115.stdout:Nov 03 17:18:43 smithi115 conmon[72025]: 2021-11-03T17:18:43.822+0000 7fa538961700 -1 osd.3 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2021-11-03T17:18:44.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:43 smithi027 ceph-mon[97244]: from='osd.3 [v2:172.21.15.115:6800/679677417,v1:172.21.15.115:6801/679677417]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2021-11-03T17:18:44.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:43 smithi027 ceph-mon[97244]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2021-11-03T17:18:45.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:44 smithi115 ceph-mon[65453]: pgmap v107: 65 pgs: 37 active+undersized+degraded, 1 active+undersized, 27 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 9826/52479 objects degraded (18.724%) 2021-11-03T17:18:45.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:44 smithi115 ceph-mon[65453]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2021-11-03T17:18:45.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:44 smithi115 ceph-mon[65453]: osdmap e71: 6 total, 5 up, 6 in 2021-11-03T17:18:45.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:44 smithi115 ceph-mon[65453]: from='osd.3 [v2:172.21.15.115:6800/679677417,v1:172.21.15.115:6801/679677417]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2021-11-03T17:18:45.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:44 smithi115 ceph-mon[65453]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2021-11-03T17:18:45.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:44 smithi115 ceph-mon[65453]: Health check update: Degraded data redundancy: 9826/52479 objects degraded (18.724%), 37 pgs degraded (PG_DEGRADED) 2021-11-03T17:18:45.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:44 smithi027 ceph-mon[97244]: pgmap v107: 65 pgs: 37 active+undersized+degraded, 1 active+undersized, 27 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 9826/52479 objects degraded (18.724%) 2021-11-03T17:18:45.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:44 smithi027 ceph-mon[97244]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2021-11-03T17:18:45.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:44 smithi027 ceph-mon[97244]: osdmap e71: 6 total, 5 up, 6 in 2021-11-03T17:18:45.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:44 smithi027 ceph-mon[97244]: from='osd.3 [v2:172.21.15.115:6800/679677417,v1:172.21.15.115:6801/679677417]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2021-11-03T17:18:45.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:44 smithi027 ceph-mon[97244]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2021-11-03T17:18:45.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:44 smithi027 ceph-mon[97244]: Health check update: Degraded data redundancy: 9826/52479 objects degraded (18.724%), 37 pgs degraded (PG_DEGRADED) 2021-11-03T17:18:46.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:45 smithi115 ceph-mon[65453]: Health check cleared: OSD_DOWN (was: 1 osds down) 2021-11-03T17:18:46.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:45 smithi115 ceph-mon[65453]: osd.3 [v2:172.21.15.115:6800/679677417,v1:172.21.15.115:6801/679677417] boot 2021-11-03T17:18:46.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:45 smithi115 ceph-mon[65453]: osdmap e72: 6 total, 6 up, 6 in 2021-11-03T17:18:46.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-11-03T17:18:46.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:45 smithi115 ceph-mon[65453]: osdmap e73: 6 total, 6 up, 6 in 2021-11-03T17:18:46.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:45 smithi027 ceph-mon[97244]: Health check cleared: OSD_DOWN (was: 1 osds down) 2021-11-03T17:18:46.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:45 smithi027 ceph-mon[97244]: osd.3 [v2:172.21.15.115:6800/679677417,v1:172.21.15.115:6801/679677417] boot 2021-11-03T17:18:46.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:45 smithi027 ceph-mon[97244]: osdmap e72: 6 total, 6 up, 6 in 2021-11-03T17:18:46.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2021-11-03T17:18:46.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:45 smithi027 ceph-mon[97244]: osdmap e73: 6 total, 6 up, 6 in 2021-11-03T17:18:47.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:46 smithi115 ceph-mon[65453]: pgmap v110: 65 pgs: 37 active+undersized+degraded, 1 active+undersized, 27 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 9826/52479 objects degraded (18.724%) 2021-11-03T17:18:47.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:46 smithi027 ceph-mon[97244]: pgmap v110: 65 pgs: 37 active+undersized+degraded, 1 active+undersized, 27 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 9826/52479 objects degraded (18.724%) 2021-11-03T17:18:49.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:48 smithi115 ceph-mon[65453]: pgmap v112: 65 pgs: 18 active+undersized+degraded, 1 active+undersized, 46 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 3831/52479 objects degraded (7.300%) 2021-11-03T17:18:49.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:48 smithi027 ceph-mon[97244]: pgmap v112: 65 pgs: 18 active+undersized+degraded, 1 active+undersized, 46 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 3831/52479 objects degraded (7.300%) 2021-11-03T17:18:50.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:49 smithi115 ceph-mon[65453]: Health check update: Degraded data redundancy: 3831/52479 objects degraded (7.300%), 18 pgs degraded (PG_DEGRADED) 2021-11-03T17:18:50.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:49 smithi027 ceph-mon[97244]: Health check update: Degraded data redundancy: 3831/52479 objects degraded (7.300%), 18 pgs degraded (PG_DEGRADED) 2021-11-03T17:18:51.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:50 smithi115 ceph-mon[65453]: pgmap v113: 65 pgs: 16 active+clean+wait, 49 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:18:51.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:50 smithi115 ceph-mon[65453]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3831/52479 objects degraded (7.300%), 18 pgs degraded) 2021-11-03T17:18:51.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:50 smithi027 ceph-mon[97244]: pgmap v113: 65 pgs: 16 active+clean+wait, 49 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:18:51.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:50 smithi027 ceph-mon[97244]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 3831/52479 objects degraded (7.300%), 18 pgs degraded) 2021-11-03T17:18:53.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:52 smithi115 ceph-mon[65453]: pgmap v114: 65 pgs: 16 active+clean+wait, 49 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:18:53.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:52 smithi027 ceph-mon[97244]: pgmap v114: 65 pgs: 16 active+clean+wait, 49 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:18:54.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:54 smithi115 ceph-mon[65453]: pgmap v115: 65 pgs: 16 active+clean+wait, 49 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:18:54.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2021-11-03T17:18:55.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:54 smithi027 ceph-mon[97244]: pgmap v115: 65 pgs: 16 active+clean+wait, 49 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:18:55.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2021-11-03T17:18:56.149 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:55 smithi115 ceph-mon[65453]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2021-11-03T17:18:56.150 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:55 smithi115 ceph-mon[65453]: Upgrade: osd.4 is safe to restart 2021-11-03T17:18:56.150 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:55 smithi115 ceph-mon[65453]: poll took 15038.089 ms: 1 events 2021-11-03T17:18:56.150 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:18:56.151 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:18:56.151 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:18:56.151 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:18:56.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:55 smithi027 ceph-mon[97244]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2021-11-03T17:18:56.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:55 smithi027 ceph-mon[97244]: Upgrade: osd.4 is safe to restart 2021-11-03T17:18:56.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:55 smithi027 ceph-mon[97244]: poll took 15038.089 ms: 1 events 2021-11-03T17:18:56.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:18:56.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:18:56.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:18:56.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:18:56.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:56 smithi115 ceph-mon[65453]: pgmap v116: 65 pgs: 16 active+clean+wait, 49 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:18:56.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:18:57.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:56 smithi027 ceph-mon[97244]: pgmap v116: 65 pgs: 16 active+clean+wait, 49 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:18:57.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:18:58.101 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:57 smithi115 ceph-mon[65453]: poll took 1533.922 ms: 1 events 2021-11-03T17:18:58.102 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:57 smithi115 ceph-mon[65453]: Upgrade: Updating osd.4 2021-11-03T17:18:58.102 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:57 smithi115 ceph-mon[65453]: Deploying daemon osd.4 on smithi115 2021-11-03T17:18:58.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:57 smithi027 ceph-mon[97244]: poll took 1533.922 ms: 1 events 2021-11-03T17:18:58.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:57 smithi027 ceph-mon[97244]: Upgrade: Updating osd.4 2021-11-03T17:18:58.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:57 smithi027 ceph-mon[97244]: Deploying daemon osd.4 on smithi115 2021-11-03T17:18:58.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:58 smithi115 ceph-mon[65453]: pgmap v117: 65 pgs: 16 active+clean+wait, 49 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:18:58.984 INFO:journalctl@ceph.osd.4.smithi115.stdout:Nov 03 17:18:58 smithi115 systemd[1]: Stopping Ceph osd.4 for 32ab977e-3cc8-11ec-8c28-001a4aab830c... 2021-11-03T17:18:58.984 INFO:journalctl@ceph.osd.4.smithi115.stdout:Nov 03 17:18:58 smithi115 conmon[45920]: debug 2021-11-03T17:18:58.886+0000 7fef4f99c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2021-11-03T17:18:58.985 INFO:journalctl@ceph.osd.4.smithi115.stdout:Nov 03 17:18:58 smithi115 conmon[45920]: debug 2021-11-03T17:18:58.886+0000 7fef4f99c700 -1 osd.4 73 *** Got signal Terminated *** 2021-11-03T17:18:58.985 INFO:journalctl@ceph.osd.4.smithi115.stdout:Nov 03 17:18:58 smithi115 conmon[45920]: debug 2021-11-03T17:18:58.886+0000 7fef4f99c700 -1 osd.4 73 *** Immediate shutdown (osd_fast_shutdown=true) *** 2021-11-03T17:18:59.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:58 smithi027 ceph-mon[97244]: pgmap v117: 65 pgs: 16 active+clean+wait, 49 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:18:59.484 INFO:journalctl@ceph.osd.4.smithi115.stdout:Nov 03 17:18:59 smithi115 bash[74584]: 8b91230c69e54ce6f2968350e2c902a672bfd46b024ce87d75e800f389624f67 2021-11-03T17:18:59.485 INFO:journalctl@ceph.osd.4.smithi115.stdout:Nov 03 17:18:59 smithi115 bash[74584]: Error: no container with name or ID ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c-osd-4 found: no such container 2021-11-03T17:18:59.984 INFO:journalctl@ceph.osd.4.smithi115.stdout:Nov 03 17:18:59 smithi115 bash[74584]: Error: container 8b91230c69e54ce6f2968350e2c902a672bfd46b024ce87d75e800f389624f67 does not exist in database: no such container 2021-11-03T17:18:59.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:59 smithi115 ceph-mon[65453]: osd.4 reported immediately failed by osd.0 2021-11-03T17:18:59.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:59 smithi115 ceph-mon[65453]: osd.4 failed (root=default,host=smithi115) (connection refused reported by osd.0) 2021-11-03T17:18:59.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:59 smithi115 ceph-mon[65453]: osd.4 reported immediately failed by osd.1 2021-11-03T17:18:59.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:59 smithi115 ceph-mon[65453]: osd.4 reported immediately failed by osd.2 2021-11-03T17:18:59.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:59 smithi115 ceph-mon[65453]: osd.4 reported immediately failed by osd.5 2021-11-03T17:18:59.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:59 smithi115 ceph-mon[65453]: osd.4 reported immediately failed by osd.0 2021-11-03T17:18:59.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:59 smithi115 ceph-mon[65453]: osd.4 reported immediately failed by osd.1 2021-11-03T17:18:59.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:59 smithi115 ceph-mon[65453]: osd.4 reported immediately failed by osd.2 2021-11-03T17:18:59.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:59 smithi115 ceph-mon[65453]: osd.4 reported immediately failed by osd.3 2021-11-03T17:18:59.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:59 smithi115 ceph-mon[65453]: osd.4 reported immediately failed by osd.5 2021-11-03T17:18:59.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:59 smithi115 ceph-mon[65453]: osd.4 reported immediately failed by osd.3 2021-11-03T17:18:59.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:59 smithi115 ceph-mon[65453]: 1 slow requests, 1 included below; oldest blocked for > 182.363209 secs 2021-11-03T17:18:59.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:59 smithi115 ceph-mon[65453]: slow request 182.363208 seconds old, received at 2021-11-03T17:15:56.956064+0000: client_request(client.24331:21043 rmdir #0x10000003957/db1 2021-11-03T17:15:56.954824+0000 caller_uid=1000, caller_gid=1254{6,36,1000,1254,}) currently failed to xlock, waiting 2021-11-03T17:18:59.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:59 smithi115 ceph-mon[65453]: osd.4 reported immediately failed by osd.5 2021-11-03T17:18:59.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:59 smithi115 ceph-mon[65453]: osd.4 reported immediately failed by osd.5 2021-11-03T17:18:59.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:59 smithi115 ceph-mon[65453]: osd.4 reported immediately failed by osd.0 2021-11-03T17:18:59.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:59 smithi115 ceph-mon[65453]: osd.4 reported immediately failed by osd.1 2021-11-03T17:18:59.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:59 smithi115 ceph-mon[65453]: osd.4 reported immediately failed by osd.2 2021-11-03T17:18:59.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:59 smithi115 ceph-mon[65453]: osd.4 reported immediately failed by osd.3 2021-11-03T17:18:59.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:59 smithi115 ceph-mon[65453]: osd.4 reported immediately failed by osd.0 2021-11-03T17:18:59.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:59 smithi115 ceph-mon[65453]: osd.4 reported immediately failed by osd.1 2021-11-03T17:18:59.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:59 smithi115 ceph-mon[65453]: osd.4 reported immediately failed by osd.2 2021-11-03T17:18:59.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:18:59 smithi115 ceph-mon[65453]: osd.4 reported immediately failed by osd.3 2021-11-03T17:19:00.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:59 smithi027 ceph-mon[97244]: osd.4 reported immediately failed by osd.0 2021-11-03T17:19:00.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:59 smithi027 ceph-mon[97244]: osd.4 failed (root=default,host=smithi115) (connection refused reported by osd.0) 2021-11-03T17:19:00.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:59 smithi027 ceph-mon[97244]: osd.4 reported immediately failed by osd.1 2021-11-03T17:19:00.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:59 smithi027 ceph-mon[97244]: osd.4 reported immediately failed by osd.2 2021-11-03T17:19:00.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:59 smithi027 ceph-mon[97244]: osd.4 reported immediately failed by osd.5 2021-11-03T17:19:00.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:59 smithi027 ceph-mon[97244]: osd.4 reported immediately failed by osd.0 2021-11-03T17:19:00.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:59 smithi027 ceph-mon[97244]: osd.4 reported immediately failed by osd.1 2021-11-03T17:19:00.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:59 smithi027 ceph-mon[97244]: osd.4 reported immediately failed by osd.2 2021-11-03T17:19:00.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:59 smithi027 ceph-mon[97244]: osd.4 reported immediately failed by osd.3 2021-11-03T17:19:00.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:59 smithi027 ceph-mon[97244]: osd.4 reported immediately failed by osd.5 2021-11-03T17:19:00.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:59 smithi027 ceph-mon[97244]: osd.4 reported immediately failed by osd.3 2021-11-03T17:19:00.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:59 smithi027 ceph-mon[97244]: 1 slow requests, 1 included below; oldest blocked for > 182.363209 secs 2021-11-03T17:19:00.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:59 smithi027 ceph-mon[97244]: slow request 182.363208 seconds old, received at 2021-11-03T17:15:56.956064+0000: client_request(client.24331:21043 rmdir #0x10000003957/db1 2021-11-03T17:15:56.954824+0000 caller_uid=1000, caller_gid=1254{6,36,1000,1254,}) currently failed to xlock, waiting 2021-11-03T17:19:00.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:59 smithi027 ceph-mon[97244]: osd.4 reported immediately failed by osd.5 2021-11-03T17:19:00.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:59 smithi027 ceph-mon[97244]: osd.4 reported immediately failed by osd.5 2021-11-03T17:19:00.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:59 smithi027 ceph-mon[97244]: osd.4 reported immediately failed by osd.0 2021-11-03T17:19:00.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:59 smithi027 ceph-mon[97244]: osd.4 reported immediately failed by osd.1 2021-11-03T17:19:00.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:59 smithi027 ceph-mon[97244]: osd.4 reported immediately failed by osd.2 2021-11-03T17:19:00.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:59 smithi027 ceph-mon[97244]: osd.4 reported immediately failed by osd.3 2021-11-03T17:19:00.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:59 smithi027 ceph-mon[97244]: osd.4 reported immediately failed by osd.0 2021-11-03T17:19:00.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:59 smithi027 ceph-mon[97244]: osd.4 reported immediately failed by osd.1 2021-11-03T17:19:00.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:59 smithi027 ceph-mon[97244]: osd.4 reported immediately failed by osd.2 2021-11-03T17:19:00.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:18:59 smithi027 ceph-mon[97244]: osd.4 reported immediately failed by osd.3 2021-11-03T17:19:01.102 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:00 smithi115 ceph-mon[65453]: pgmap v118: 65 pgs: 65 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:19:01.103 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:00 smithi115 ceph-mon[65453]: Health check failed: 1 osds down (OSD_DOWN) 2021-11-03T17:19:01.103 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:00 smithi115 ceph-mon[65453]: osdmap e74: 6 total, 5 up, 6 in 2021-11-03T17:19:01.104 INFO:journalctl@ceph.osd.4.smithi115.stdout:Nov 03 17:19:00 smithi115 systemd[1]: ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c@osd.4.service: Succeeded. 2021-11-03T17:19:01.104 INFO:journalctl@ceph.osd.4.smithi115.stdout:Nov 03 17:19:00 smithi115 systemd[1]: Stopped Ceph osd.4 for 32ab977e-3cc8-11ec-8c28-001a4aab830c. 2021-11-03T17:19:01.104 INFO:journalctl@ceph.osd.4.smithi115.stdout:Nov 03 17:19:01 smithi115 systemd[1]: Starting Ceph osd.4 for 32ab977e-3cc8-11ec-8c28-001a4aab830c... 2021-11-03T17:19:01.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:00 smithi027 ceph-mon[97244]: pgmap v118: 65 pgs: 65 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:19:01.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:00 smithi027 ceph-mon[97244]: Health check failed: 1 osds down (OSD_DOWN) 2021-11-03T17:19:01.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:00 smithi027 ceph-mon[97244]: osdmap e74: 6 total, 5 up, 6 in 2021-11-03T17:19:02.174 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:01 smithi115 ceph-mon[65453]: Health check failed: 1 MDSs report slow requests (MDS_SLOW_REQUEST) 2021-11-03T17:19:02.175 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:01 smithi115 ceph-mon[65453]: mds.1 [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] up:active 2021-11-03T17:19:02.175 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:01 smithi115 ceph-mon[65453]: fsmap cephfs:1/2 {1=cephfs.smithi027.jdwohn=up:active} 3 up:standby, 1 failed 2021-11-03T17:19:02.175 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:01 smithi115 ceph-mon[65453]: osdmap e75: 6 total, 5 up, 6 in 2021-11-03T17:19:02.175 INFO:journalctl@ceph.osd.4.smithi115.stdout:Nov 03 17:19:02 smithi115 bash[75066]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2021-11-03T17:19:02.176 INFO:journalctl@ceph.osd.4.smithi115.stdout:Nov 03 17:19:02 smithi115 bash[75066]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --dev /dev/mapper/vg_nvme-lv_3 --path /var/lib/ceph/osd/ceph-4 --no-mon-config 2021-11-03T17:19:02.176 INFO:journalctl@ceph.osd.4.smithi115.stdout:Nov 03 17:19:02 smithi115 bash[75066]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2021-11-03T17:19:02.176 INFO:journalctl@ceph.osd.4.smithi115.stdout:Nov 03 17:19:02 smithi115 bash[75066]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2021-11-03T17:19:02.176 INFO:journalctl@ceph.osd.4.smithi115.stdout:Nov 03 17:19:02 smithi115 bash[75066]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2021-11-03T17:19:02.177 INFO:journalctl@ceph.osd.4.smithi115.stdout:Nov 03 17:19:02 smithi115 bash[75066]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2021-11-03T17:19:02.177 INFO:journalctl@ceph.osd.4.smithi115.stdout:Nov 03 17:19:02 smithi115 bash[75066]: --> ceph-volume raw activate successful for osd ID: 4 2021-11-03T17:19:02.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:01 smithi027 ceph-mon[97244]: Health check failed: 1 MDSs report slow requests (MDS_SLOW_REQUEST) 2021-11-03T17:19:02.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:01 smithi027 ceph-mon[97244]: mds.1 [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] up:active 2021-11-03T17:19:02.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:01 smithi027 ceph-mon[97244]: fsmap cephfs:1/2 {1=cephfs.smithi027.jdwohn=up:active} 3 up:standby, 1 failed 2021-11-03T17:19:02.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:01 smithi027 ceph-mon[97244]: osdmap e75: 6 total, 5 up, 6 in 2021-11-03T17:19:03.155 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:02 smithi115 ceph-mon[65453]: pgmap v121: 65 pgs: 5 active+undersized+degraded, 16 peering, 2 stale+active+clean, 42 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1564/52479 objects degraded (2.980%) 2021-11-03T17:19:03.156 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:02 smithi115 ceph-mon[65453]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2021-11-03T17:19:03.156 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:02 smithi115 ceph-mon[65453]: Health check failed: Degraded data redundancy: 1564/52479 objects degraded (2.980%), 5 pgs degraded (PG_DEGRADED) 2021-11-03T17:19:03.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:02 smithi027 ceph-mon[97244]: pgmap v121: 65 pgs: 5 active+undersized+degraded, 16 peering, 2 stale+active+clean, 42 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1564/52479 objects degraded (2.980%) 2021-11-03T17:19:03.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:02 smithi027 ceph-mon[97244]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2021-11-03T17:19:03.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:02 smithi027 ceph-mon[97244]: Health check failed: Degraded data redundancy: 1564/52479 objects degraded (2.980%), 5 pgs degraded (PG_DEGRADED) 2021-11-03T17:19:03.484 INFO:journalctl@ceph.osd.4.smithi115.stdout:Nov 03 17:19:03 smithi115 ceph-osd[75585]: -- 172.21.15.115:0/944074181 <== mon.1 v2:172.21.15.115:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5571d3394340 con 0x5571d335ac00 2021-11-03T17:19:03.484 INFO:journalctl@ceph.osd.4.smithi115.stdout:Nov 03 17:19:03 smithi115 bash[75066]: 3a8535b47627201abef05a05a417b953acdef538492214db15a44e5837a44c22 2021-11-03T17:19:03.484 INFO:journalctl@ceph.osd.4.smithi115.stdout:Nov 03 17:19:03 smithi115 systemd[1]: Started Ceph osd.4 for 32ab977e-3cc8-11ec-8c28-001a4aab830c. 2021-11-03T17:19:03.955 INFO:journalctl@ceph.osd.4.smithi115.stdout:Nov 03 17:19:03 smithi115 conmon[75562]: 2021-11-03T17:19:03.681+0000 7f0c3eb9b0c0 -1 Falling back to public interface 2021-11-03T17:19:04.172 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.174+0000 7fcdcb50f700 1 -- 172.21.15.27:0/1550942106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdc40f66e0 msgr2=0x7fcdc40f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:19:04.172 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.174+0000 7fcdcb50f700 1 --2- 172.21.15.27:0/1550942106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdc40f66e0 0x7fcdc40f6b00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fcdb4004660 tx=0x7fcdb4009b30).stop 2021-11-03T17:19:04.173 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.174+0000 7fcdcb50f700 1 -- 172.21.15.27:0/1550942106 shutdown_connections 2021-11-03T17:19:04.173 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.174+0000 7fcdcb50f700 1 --2- 172.21.15.27:0/1550942106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcdc40f6ff0 0x7fcdc40f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:04.173 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.174+0000 7fcdcb50f700 1 --2- 172.21.15.27:0/1550942106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdc40f66e0 0x7fcdc40f6b00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:04.174 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.174+0000 7fcdcb50f700 1 -- 172.21.15.27:0/1550942106 >> 172.21.15.27:0/1550942106 conn(0x7fcdc40f1f00 msgr2=0x7fcdc40f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:19:04.174 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.174+0000 7fcdcb50f700 1 -- 172.21.15.27:0/1550942106 shutdown_connections 2021-11-03T17:19:04.174 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.175+0000 7fcdcb50f700 1 -- 172.21.15.27:0/1550942106 wait complete. 2021-11-03T17:19:04.174 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.175+0000 7fcdcb50f700 1 Processor -- start 2021-11-03T17:19:04.174 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.175+0000 7fcdcb50f700 1 -- start start 2021-11-03T17:19:04.175 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.175+0000 7fcdcb50f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdc40f66e0 0x7fcdc4106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:19:04.175 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.175+0000 7fcdcb50f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcdc40f6ff0 0x7fcdc4107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:19:04.175 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.175+0000 7fcdcb50f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc4107a00 con 0x7fcdc40f66e0 2021-11-03T17:19:04.175 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.175+0000 7fcdcb50f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdc4107b40 con 0x7fcdc40f6ff0 2021-11-03T17:19:04.176 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.175+0000 7fcdc92ab700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdc40f66e0 0x7fcdc4106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:19:04.176 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.175+0000 7fcdc92ab700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdc40f66e0 0x7fcdc4106f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56652/0 (socket says 172.21.15.27:56652) 2021-11-03T17:19:04.176 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.175+0000 7fcdc92ab700 1 -- 172.21.15.27:0/426839719 learned_addr learned my addr 172.21.15.27:0/426839719 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:19:04.176 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.176+0000 7fcdc8aaa700 1 --2- 172.21.15.27:0/426839719 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcdc40f6ff0 0x7fcdc4107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:19:04.177 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.176+0000 7fcdc92ab700 1 -- 172.21.15.27:0/426839719 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcdc40f6ff0 msgr2=0x7fcdc4107450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:19:04.177 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.176+0000 7fcdc92ab700 1 --2- 172.21.15.27:0/426839719 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcdc40f6ff0 0x7fcdc4107450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:04.177 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.176+0000 7fcdc92ab700 1 -- 172.21.15.27:0/426839719 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb4005040 con 0x7fcdc40f66e0 2021-11-03T17:19:04.177 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.176+0000 7fcdc92ab700 1 --2- 172.21.15.27:0/426839719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdc40f66e0 0x7fcdc4106f60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fcdb4009990 tx=0x7fcdb4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:19:04.178 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.176+0000 7fcdb9ffb700 1 -- 172.21.15.27:0/426839719 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdb4018070 con 0x7fcdc40f66e0 2021-11-03T17:19:04.178 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.176+0000 7fcdcb50f700 1 -- 172.21.15.27:0/426839719 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdc4108480 con 0x7fcdc40f66e0 2021-11-03T17:19:04.178 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.176+0000 7fcdb9ffb700 1 -- 172.21.15.27:0/426839719 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fcdb4007ca0 con 0x7fcdc40f66e0 2021-11-03T17:19:04.178 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.176+0000 7fcdb9ffb700 1 -- 172.21.15.27:0/426839719 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcdb401c620 con 0x7fcdc40f66e0 2021-11-03T17:19:04.178 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.176+0000 7fcdcb50f700 1 -- 172.21.15.27:0/426839719 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdc41089e0 con 0x7fcdc40f66e0 2021-11-03T17:19:04.180 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.177+0000 7fcdcb50f700 1 -- 172.21.15.27:0/426839719 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdc4045ad0 con 0x7fcdc40f66e0 2021-11-03T17:19:04.180 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.178+0000 7fcdb9ffb700 1 -- 172.21.15.27:0/426839719 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fcdb401d960 con 0x7fcdc40f66e0 2021-11-03T17:19:04.180 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.178+0000 7fcdb9ffb700 1 --2- 172.21.15.27:0/426839719 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcdb00640b0 0x7fcdb0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:19:04.180 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.178+0000 7fcdb9ffb700 1 -- 172.21.15.27:0/426839719 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fcdb4082520 con 0x7fcdc40f66e0 2021-11-03T17:19:04.181 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.178+0000 7fcdc8aaa700 1 --2- 172.21.15.27:0/426839719 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcdb00640b0 0x7fcdb0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:19:04.181 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.179+0000 7fcdc8aaa700 1 --2- 172.21.15.27:0/426839719 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcdb00640b0 0x7fcdb0066570 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fcdc00049e0 tx=0x7fcdc0009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:19:04.181 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.183+0000 7fcdb9ffb700 1 -- 172.21.15.27:0/426839719 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fcdb4054940 con 0x7fcdc40f66e0 2021-11-03T17:19:04.359 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.363+0000 7fcdcb50f700 1 -- 172.21.15.27:0/426839719 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcdc40fef40 con 0x7fcdb00640b0 2021-11-03T17:19:04.360 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.364+0000 7fcdb9ffb700 1 -- 172.21.15.27:0/426839719 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+321 (secure 0 0 0) 0x7fcdc40fef40 con 0x7fcdb00640b0 2021-11-03T17:19:04.362 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.366+0000 7fcdcb50f700 1 -- 172.21.15.27:0/426839719 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcdb00640b0 msgr2=0x7fcdb0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:19:04.362 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.366+0000 7fcdcb50f700 1 --2- 172.21.15.27:0/426839719 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcdb00640b0 0x7fcdb0066570 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fcdc00049e0 tx=0x7fcdc0009040).stop 2021-11-03T17:19:04.362 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.366+0000 7fcdcb50f700 1 -- 172.21.15.27:0/426839719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdc40f66e0 msgr2=0x7fcdc4106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:19:04.363 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.366+0000 7fcdcb50f700 1 --2- 172.21.15.27:0/426839719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdc40f66e0 0x7fcdc4106f60 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fcdb4009990 tx=0x7fcdb4009b30).stop 2021-11-03T17:19:04.363 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.366+0000 7fcdcb50f700 1 -- 172.21.15.27:0/426839719 shutdown_connections 2021-11-03T17:19:04.363 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.366+0000 7fcdcb50f700 1 --2- 172.21.15.27:0/426839719 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcdb00640b0 0x7fcdb0066570 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:04.363 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.366+0000 7fcdcb50f700 1 --2- 172.21.15.27:0/426839719 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcdc40f66e0 0x7fcdc4106f60 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:04.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.366+0000 7fcdcb50f700 1 --2- 172.21.15.27:0/426839719 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcdc40f6ff0 0x7fcdc4107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:04.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.366+0000 7fcdcb50f700 1 -- 172.21.15.27:0/426839719 >> 172.21.15.27:0/426839719 conn(0x7fcdc40f1f00 msgr2=0x7fcdc40fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:19:04.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.366+0000 7fcdcb50f700 1 -- 172.21.15.27:0/426839719 shutdown_connections 2021-11-03T17:19:04.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.366+0000 7fcdcb50f700 1 -- 172.21.15.27:0/426839719 wait complete. 2021-11-03T17:19:04.372 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:19:04.463 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.467+0000 7ffbb106d700 1 -- 172.21.15.27:0/3963491297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbac0f5ef0 msgr2=0x7ffbac0f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:19:04.463 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.467+0000 7ffbb106d700 1 --2- 172.21.15.27:0/3963491297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbac0f5ef0 0x7ffbac0f6920 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7ffba0004660 tx=0x7ffba0009b30).stop 2021-11-03T17:19:04.464 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.467+0000 7ffbb106d700 1 -- 172.21.15.27:0/3963491297 shutdown_connections 2021-11-03T17:19:04.464 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.467+0000 7ffbb106d700 1 --2- 172.21.15.27:0/3963491297 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbac0f5ef0 0x7ffbac0f6920 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:04.464 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.467+0000 7ffbb106d700 1 --2- 172.21.15.27:0/3963491297 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffbac0f55e0 0x7ffbac0f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:04.464 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.467+0000 7ffbb106d700 1 -- 172.21.15.27:0/3963491297 >> 172.21.15.27:0/3963491297 conn(0x7ffbac0f0de0 msgr2=0x7ffbac0f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:19:04.465 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.467+0000 7ffbb106d700 1 -- 172.21.15.27:0/3963491297 shutdown_connections 2021-11-03T17:19:04.465 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.467+0000 7ffbb106d700 1 -- 172.21.15.27:0/3963491297 wait complete. 2021-11-03T17:19:04.467 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.468+0000 7ffbb106d700 1 Processor -- start 2021-11-03T17:19:04.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.468+0000 7ffbb106d700 1 -- start start 2021-11-03T17:19:04.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.468+0000 7ffbb106d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffbac0f55e0 0x7ffbac0f97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:19:04.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.468+0000 7ffbb106d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbac0f5ef0 0x7ffbac0f7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:19:04.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.468+0000 7ffbb106d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac0f81c0 con 0x7ffbac0f5ef0 2021-11-03T17:19:04.469 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.468+0000 7ffbb106d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac0f8300 con 0x7ffbac0f55e0 2021-11-03T17:19:04.469 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.468+0000 7ffbaa59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbac0f5ef0 0x7ffbac0f7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:19:04.469 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.468+0000 7ffbaa59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbac0f5ef0 0x7ffbac0f7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56664/0 (socket says 172.21.15.27:56664) 2021-11-03T17:19:04.469 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.468+0000 7ffbaa59c700 1 -- 172.21.15.27:0/424467069 learned_addr learned my addr 172.21.15.27:0/424467069 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:19:04.470 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.468+0000 7ffbaad9d700 1 --2- 172.21.15.27:0/424467069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffbac0f55e0 0x7ffbac0f97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:19:04.470 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.469+0000 7ffbaa59c700 1 -- 172.21.15.27:0/424467069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffbac0f55e0 msgr2=0x7ffbac0f97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:19:04.470 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.469+0000 7ffbaa59c700 1 --2- 172.21.15.27:0/424467069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffbac0f55e0 0x7ffbac0f97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:04.470 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.469+0000 7ffbaa59c700 1 -- 172.21.15.27:0/424467069 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba0005040 con 0x7ffbac0f5ef0 2021-11-03T17:19:04.470 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.469+0000 7ffbaa59c700 1 --2- 172.21.15.27:0/424467069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbac0f5ef0 0x7ffbac0f7c60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7ffba0009990 tx=0x7ffba0009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:19:04.471 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.469+0000 7ffb937fe700 1 -- 172.21.15.27:0/424467069 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffba0018070 con 0x7ffbac0f5ef0 2021-11-03T17:19:04.471 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.469+0000 7ffb937fe700 1 -- 172.21.15.27:0/424467069 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7ffba0007ba0 con 0x7ffbac0f5ef0 2021-11-03T17:19:04.471 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.469+0000 7ffbb106d700 1 -- 172.21.15.27:0/424467069 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbac0f8580 con 0x7ffbac0f5ef0 2021-11-03T17:19:04.471 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.469+0000 7ffb937fe700 1 -- 172.21.15.27:0/424467069 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffba001c3c0 con 0x7ffbac0f5ef0 2021-11-03T17:19:04.472 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.469+0000 7ffbb106d700 1 -- 172.21.15.27:0/424467069 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbac0f8ae0 con 0x7ffbac0f5ef0 2021-11-03T17:19:04.472 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.470+0000 7ffbb106d700 1 -- 172.21.15.27:0/424467069 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbac045ad0 con 0x7ffbac0f5ef0 2021-11-03T17:19:04.472 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.470+0000 7ffb937fe700 1 -- 172.21.15.27:0/424467069 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7ffba001d420 con 0x7ffbac0f5ef0 2021-11-03T17:19:04.472 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.471+0000 7ffb937fe700 1 --2- 172.21.15.27:0/424467069 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffb940640b0 0x7ffb94066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:19:04.473 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.471+0000 7ffb937fe700 1 -- 172.21.15.27:0/424467069 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6461+0+0 (secure 0 0 0) 0x7ffba0082410 con 0x7ffbac0f5ef0 2021-11-03T17:19:04.473 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.471+0000 7ffbaad9d700 1 --2- 172.21.15.27:0/424467069 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffb940640b0 0x7ffb94066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:19:04.473 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.471+0000 7ffbaad9d700 1 --2- 172.21.15.27:0/424467069 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffb940640b0 0x7ffb94066570 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7ffbac0f9570 tx=0x7ffb9c00b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:19:04.474 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.475+0000 7ffb937fe700 1 -- 172.21.15.27:0/424467069 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7ffba0054830 con 0x7ffbac0f5ef0 2021-11-03T17:19:04.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:04 smithi115 ceph-mon[65453]: poll took 6767.725 ms: 1 events 2021-11-03T17:19:04.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:19:04.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:19:04.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:04 smithi115 ceph-mon[65453]: pgmap v122: 65 pgs: 5 active+undersized+degraded, 16 peering, 2 stale+active+clean, 42 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1564/52479 objects degraded (2.980%) 2021-11-03T17:19:04.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:04 smithi027 ceph-mon[97244]: poll took 6767.725 ms: 1 events 2021-11-03T17:19:04.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:19:04.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:19:04.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:04 smithi027 ceph-mon[97244]: pgmap v122: 65 pgs: 5 active+undersized+degraded, 16 peering, 2 stale+active+clean, 42 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 1564/52479 objects degraded (2.980%) 2021-11-03T17:19:04.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.649+0000 7ffbb106d700 1 -- 172.21.15.27:0/424467069 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ffbac0fad60 con 0x7ffb940640b0 2021-11-03T17:19:04.659 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.664+0000 7ffb937fe700 1 -- 172.21.15.27:0/424467069 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7ffbac0fad60 con 0x7ffb940640b0 2021-11-03T17:19:04.660 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:19:04.660 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (10m) 52s ago 11m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:19:04.660 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (2m) 52s ago 11m 7205k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:19:04.660 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (2m) 25s ago 11m 7247k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:19:04.661 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (10m) 52s ago 11m 38.9M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:19:04.661 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (4m) 52s ago 6m 15.4M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:19:04.661 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (7m) 52s ago 7m 3836M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:19:04.661 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (2m) 25s ago 7m 14.3M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:19:04.662 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (4m) 25s ago 7m 16.0M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:19:04.662 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (4m) 52s ago 13m 470M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:19:04.662 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (3m) 25s ago 11m 412M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:19:04.662 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (3m) 52s ago 13m 81.2M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:19:04.662 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (2m) 25s ago 11m 66.1M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:19:04.663 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (11m) 52s ago 11m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:19:04.663 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (10m) 25s ago 10m 18.2M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:19:04.663 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (117s) 52s ago 10m 243M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:19:04.663 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (87s) 52s ago 9m 270M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:19:04.664 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (57s) 52s ago 9m 11.8M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:19:04.664 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (28s) 25s ago 9m 11.9M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:19:04.664 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 starting - - - 4096M 2021-11-03T17:19:04.664 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (8m) 25s ago 8m 849M 4096M 16.2.4 8d91d370c2b8 4a54c4d6b6f5 2021-11-03T17:19:04.665 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (10m) 52s ago 11m 45.7M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:19:04.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.665+0000 7ffbb106d700 1 -- 172.21.15.27:0/424467069 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffb940640b0 msgr2=0x7ffb94066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:19:04.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.665+0000 7ffbb106d700 1 --2- 172.21.15.27:0/424467069 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffb940640b0 0x7ffb94066570 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7ffbac0f9570 tx=0x7ffb9c00b040).stop 2021-11-03T17:19:04.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.665+0000 7ffbb106d700 1 -- 172.21.15.27:0/424467069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbac0f5ef0 msgr2=0x7ffbac0f7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:19:04.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.665+0000 7ffbb106d700 1 --2- 172.21.15.27:0/424467069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbac0f5ef0 0x7ffbac0f7c60 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7ffba0009990 tx=0x7ffba0009b30).stop 2021-11-03T17:19:04.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.666+0000 7ffbb106d700 1 -- 172.21.15.27:0/424467069 shutdown_connections 2021-11-03T17:19:04.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.666+0000 7ffbb106d700 1 --2- 172.21.15.27:0/424467069 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffb940640b0 0x7ffb94066570 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:04.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.666+0000 7ffbb106d700 1 --2- 172.21.15.27:0/424467069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffbac0f55e0 0x7ffbac0f97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:04.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.666+0000 7ffbb106d700 1 --2- 172.21.15.27:0/424467069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbac0f5ef0 0x7ffbac0f7c60 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:04.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.666+0000 7ffbb106d700 1 -- 172.21.15.27:0/424467069 >> 172.21.15.27:0/424467069 conn(0x7ffbac0f0de0 msgr2=0x7ffbac05ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:19:04.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.666+0000 7ffbb106d700 1 -- 172.21.15.27:0/424467069 shutdown_connections 2021-11-03T17:19:04.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.666+0000 7ffbb106d700 1 -- 172.21.15.27:0/424467069 wait complete. 2021-11-03T17:19:04.762 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.767+0000 7f1cc1910700 1 -- 172.21.15.27:0/2835570413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cbc0f66e0 msgr2=0x7f1cbc0f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:19:04.763 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.767+0000 7f1cc1910700 1 --2- 172.21.15.27:0/2835570413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cbc0f66e0 0x7f1cbc0f6b00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f1cb40099e0 tx=0x7f1cb4004d10).stop 2021-11-03T17:19:04.763 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.767+0000 7f1cc1910700 1 -- 172.21.15.27:0/2835570413 shutdown_connections 2021-11-03T17:19:04.763 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.767+0000 7f1cc1910700 1 --2- 172.21.15.27:0/2835570413 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1cbc0f7980 0x7f1cbc0f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:04.764 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.767+0000 7f1cc1910700 1 --2- 172.21.15.27:0/2835570413 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cbc0f66e0 0x7f1cbc0f6b00 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:04.764 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.767+0000 7f1cc1910700 1 -- 172.21.15.27:0/2835570413 >> 172.21.15.27:0/2835570413 conn(0x7f1cbc0f1e40 msgr2=0x7f1cbc0f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:19:04.764 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.767+0000 7f1cc1910700 1 -- 172.21.15.27:0/2835570413 shutdown_connections 2021-11-03T17:19:04.764 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.767+0000 7f1cc1910700 1 -- 172.21.15.27:0/2835570413 wait complete. 2021-11-03T17:19:04.767 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.767+0000 7f1cc1910700 1 Processor -- start 2021-11-03T17:19:04.768 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.767+0000 7f1cc1910700 1 -- start start 2021-11-03T17:19:04.768 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.768+0000 7f1cc1910700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cbc0f66e0 0x7f1cbc06b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:19:04.768 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.768+0000 7f1cc1910700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1cbc0f7980 0x7f1cbc069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:19:04.768 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.768+0000 7f1cc1910700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc06a380 con 0x7f1cbc0f66e0 2021-11-03T17:19:04.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.768+0000 7f1cc1910700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc06a4c0 con 0x7f1cbc0f7980 2021-11-03T17:19:04.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.768+0000 7f1cbaffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cbc0f66e0 0x7f1cbc06b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:19:04.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.768+0000 7f1cbaffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cbc0f66e0 0x7f1cbc06b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56672/0 (socket says 172.21.15.27:56672) 2021-11-03T17:19:04.770 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.768+0000 7f1cbaffd700 1 -- 172.21.15.27:0/2550112361 learned_addr learned my addr 172.21.15.27:0/2550112361 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:19:04.770 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.768+0000 7f1cba7fc700 1 --2- 172.21.15.27:0/2550112361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1cbc0f7980 0x7f1cbc069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:19:04.770 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.768+0000 7f1cbaffd700 1 -- 172.21.15.27:0/2550112361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1cbc0f7980 msgr2=0x7f1cbc069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:19:04.770 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.768+0000 7f1cbaffd700 1 --2- 172.21.15.27:0/2550112361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1cbc0f7980 0x7f1cbc069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:04.771 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.768+0000 7f1cbaffd700 1 -- 172.21.15.27:0/2550112361 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cb4009640 con 0x7f1cbc0f66e0 2021-11-03T17:19:04.771 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.768+0000 7f1cbaffd700 1 --2- 172.21.15.27:0/2550112361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cbc0f66e0 0x7f1cbc06b9a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f1cb400c010 tx=0x7f1cb4004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:19:04.771 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.768+0000 7f1ca37fe700 1 -- 172.21.15.27:0/2550112361 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cb400e070 con 0x7f1cbc0f66e0 2021-11-03T17:19:04.771 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.768+0000 7f1cc1910700 1 -- 172.21.15.27:0/2550112361 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cbc06a740 con 0x7f1cbc0f66e0 2021-11-03T17:19:04.772 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.768+0000 7f1ca37fe700 1 -- 172.21.15.27:0/2550112361 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f1cb4007d10 con 0x7f1cbc0f66e0 2021-11-03T17:19:04.772 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.768+0000 7f1ca37fe700 1 -- 172.21.15.27:0/2550112361 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1cb4018070 con 0x7f1cbc0f66e0 2021-11-03T17:19:04.772 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.768+0000 7f1cc1910700 1 -- 172.21.15.27:0/2550112361 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cbc06ac20 con 0x7f1cbc0f66e0 2021-11-03T17:19:04.772 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.769+0000 7f1cc1910700 1 -- 172.21.15.27:0/2550112361 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cbc045ad0 con 0x7f1cbc0f66e0 2021-11-03T17:19:04.773 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.770+0000 7f1ca37fe700 1 -- 172.21.15.27:0/2550112361 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f1cb4024060 con 0x7f1cbc0f66e0 2021-11-03T17:19:04.773 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.770+0000 7f1ca37fe700 1 --2- 172.21.15.27:0/2550112361 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1ca40640b0 0x7f1ca4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:19:04.773 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.770+0000 7f1ca37fe700 1 -- 172.21.15.27:0/2550112361 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f1cb4081f90 con 0x7f1cbc0f66e0 2021-11-03T17:19:04.773 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.770+0000 7f1cba7fc700 1 --2- 172.21.15.27:0/2550112361 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1ca40640b0 0x7f1ca4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:19:04.773 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.771+0000 7f1cba7fc700 1 --2- 172.21.15.27:0/2550112361 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1ca40640b0 0x7f1ca4066570 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f1cb0004d80 tx=0x7f1cb000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:19:04.774 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:04.774+0000 7f1ca37fe700 1 -- 172.21.15.27:0/2550112361 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f1cb40542f0 con 0x7f1cbc0f66e0 2021-11-03T17:19:05.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.004+0000 7f1cc1910700 1 -- 172.21.15.27:0/2550112361 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1cbc0008d0 con 0x7f1cbc0f66e0 2021-11-03T17:19:05.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.005+0000 7f1ca37fe700 1 -- 172.21.15.27:0/2550112361 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+804 (secure 0 0 0) 0x7f1cb4022340 con 0x7f1cbc0f66e0 2021-11-03T17:19:05.001 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:19:05.001 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:19:05.001 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:19:05.002 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:19:05.002 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:19:05.002 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:19:05.002 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:19:05.003 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:19:05.003 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 1, 2021-11-03T17:19:05.003 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 4 2021-11-03T17:19:05.003 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:19:05.004 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:19:05.004 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:19:05.004 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:19:05.004 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:19:05.004 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 5, 2021-11-03T17:19:05.005 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 8 2021-11-03T17:19:05.005 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:19:05.005 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:19:05.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.007+0000 7f1cc1910700 1 -- 172.21.15.27:0/2550112361 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1ca40640b0 msgr2=0x7f1ca4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:19:05.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.007+0000 7f1cc1910700 1 --2- 172.21.15.27:0/2550112361 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1ca40640b0 0x7f1ca4066570 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f1cb0004d80 tx=0x7f1cb000b040).stop 2021-11-03T17:19:05.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.007+0000 7f1cc1910700 1 -- 172.21.15.27:0/2550112361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cbc0f66e0 msgr2=0x7f1cbc06b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:19:05.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.007+0000 7f1cc1910700 1 --2- 172.21.15.27:0/2550112361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cbc0f66e0 0x7f1cbc06b9a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f1cb400c010 tx=0x7f1cb4004d10).stop 2021-11-03T17:19:05.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.007+0000 7f1cc1910700 1 -- 172.21.15.27:0/2550112361 shutdown_connections 2021-11-03T17:19:05.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.007+0000 7f1cc1910700 1 --2- 172.21.15.27:0/2550112361 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1ca40640b0 0x7f1ca4066570 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:05.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.007+0000 7f1cc1910700 1 --2- 172.21.15.27:0/2550112361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1cbc0f66e0 0x7f1cbc06b9a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:05.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.007+0000 7f1cc1910700 1 --2- 172.21.15.27:0/2550112361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1cbc0f7980 0x7f1cbc069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:05.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.007+0000 7f1cc1910700 1 -- 172.21.15.27:0/2550112361 >> 172.21.15.27:0/2550112361 conn(0x7f1cbc0f1e40 msgr2=0x7f1cbc0fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:19:05.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.007+0000 7f1cc1910700 1 -- 172.21.15.27:0/2550112361 shutdown_connections 2021-11-03T17:19:05.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.007+0000 7f1cc1910700 1 -- 172.21.15.27:0/2550112361 wait complete. 2021-11-03T17:19:05.103 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.107+0000 7f53ca931700 1 -- 172.21.15.27:0/3771993975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c4060330 msgr2=0x7f53c40f93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:19:05.104 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.107+0000 7f53ca931700 1 --2- 172.21.15.27:0/3771993975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c4060330 0x7f53c40f93c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f53b8004660 tx=0x7f53b8009b30).stop 2021-11-03T17:19:05.104 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.107+0000 7f53ca931700 1 -- 172.21.15.27:0/3771993975 shutdown_connections 2021-11-03T17:19:05.104 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.108+0000 7f53ca931700 1 --2- 172.21.15.27:0/3771993975 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53c40f98b0 0x7f53c40fbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:05.104 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.108+0000 7f53ca931700 1 --2- 172.21.15.27:0/3771993975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c4060330 0x7f53c40f93c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:05.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.108+0000 7f53ca931700 1 -- 172.21.15.27:0/3771993975 >> 172.21.15.27:0/3771993975 conn(0x7f53c40f0de0 msgr2=0x7f53c40f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:19:05.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.108+0000 7f53ca931700 1 -- 172.21.15.27:0/3771993975 shutdown_connections 2021-11-03T17:19:05.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.108+0000 7f53ca931700 1 -- 172.21.15.27:0/3771993975 wait complete. 2021-11-03T17:19:05.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.108+0000 7f53ca931700 1 Processor -- start 2021-11-03T17:19:05.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.108+0000 7f53ca931700 1 -- start start 2021-11-03T17:19:05.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.108+0000 7f53ca931700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53c4060330 0x7f53c40f6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:19:05.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.108+0000 7f53ca931700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c40f98b0 0x7f53c40f52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:19:05.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.108+0000 7f53ca931700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c40f5820 con 0x7f53c40f98b0 2021-11-03T17:19:05.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.108+0000 7f53ca931700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c40f5960 con 0x7f53c4060330 2021-11-03T17:19:05.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.109+0000 7f53c37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c40f98b0 0x7f53c40f52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:19:05.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.109+0000 7f53c37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c40f98b0 0x7f53c40f52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56684/0 (socket says 172.21.15.27:56684) 2021-11-03T17:19:05.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.109+0000 7f53c37fe700 1 -- 172.21.15.27:0/3142883725 learned_addr learned my addr 172.21.15.27:0/3142883725 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:19:05.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.109+0000 7f53c3fff700 1 --2- 172.21.15.27:0/3142883725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53c4060330 0x7f53c40f6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:19:05.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.109+0000 7f53c37fe700 1 -- 172.21.15.27:0/3142883725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53c4060330 msgr2=0x7f53c40f6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:19:05.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.109+0000 7f53c37fe700 1 --2- 172.21.15.27:0/3142883725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53c4060330 0x7f53c40f6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:05.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.109+0000 7f53c37fe700 1 -- 172.21.15.27:0/3142883725 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b8005040 con 0x7f53c40f98b0 2021-11-03T17:19:05.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.109+0000 7f53c37fe700 1 --2- 172.21.15.27:0/3142883725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c40f98b0 0x7f53c40f52c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f53b400e7f0 tx=0x7f53b400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:19:05.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.109+0000 7f53c0ff9700 1 -- 172.21.15.27:0/3142883725 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53b40092e0 con 0x7f53c40f98b0 2021-11-03T17:19:05.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.109+0000 7f53c0ff9700 1 -- 172.21.15.27:0/3142883725 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f53b400f040 con 0x7f53c40f98b0 2021-11-03T17:19:05.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.109+0000 7f53ca931700 1 -- 172.21.15.27:0/3142883725 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c40f5c40 con 0x7f53c40f98b0 2021-11-03T17:19:05.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.109+0000 7f53c0ff9700 1 -- 172.21.15.27:0/3142883725 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53b4014750 con 0x7f53c40f98b0 2021-11-03T17:19:05.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.109+0000 7f53ca931700 1 -- 172.21.15.27:0/3142883725 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c410ef80 con 0x7f53c40f98b0 2021-11-03T17:19:05.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.110+0000 7f53ca931700 1 -- 172.21.15.27:0/3142883725 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53c4045ad0 con 0x7f53c40f98b0 2021-11-03T17:19:05.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.111+0000 7f53c0ff9700 1 -- 172.21.15.27:0/3142883725 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f53b401b030 con 0x7f53c40f98b0 2021-11-03T17:19:05.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.111+0000 7f53c0ff9700 1 --2- 172.21.15.27:0/3142883725 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f53ac0640b0 0x7f53ac066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:19:05.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.111+0000 7f53c0ff9700 1 -- 172.21.15.27:0/3142883725 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f53b4080680 con 0x7f53c40f98b0 2021-11-03T17:19:05.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.111+0000 7f53c3fff700 1 --2- 172.21.15.27:0/3142883725 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f53ac0640b0 0x7f53ac066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:19:05.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.112+0000 7f53c3fff700 1 --2- 172.21.15.27:0/3142883725 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f53ac0640b0 0x7f53ac066570 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f53b8000c00 tx=0x7f53b8016040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:19:05.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.115+0000 7f53c0ff9700 1 -- 172.21.15.27:0/3142883725 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f53b40529a0 con 0x7f53c40f98b0 2021-11-03T17:19:05.319 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.323+0000 7f53ca931700 1 -- 172.21.15.27:0/3142883725 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f53c4027a00 con 0x7f53c40f98b0 2021-11-03T17:19:05.319 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.323+0000 7f53c0ff9700 1 -- 172.21.15.27:0/3142883725 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 30 v30) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f53b4052590 con 0x7f53c40f98b0 2021-11-03T17:19:05.320 INFO:teuthology.orchestra.run.smithi027.stdout:e30 2021-11-03T17:19:05.320 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:19:05.320 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:19:05.321 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:19:05.321 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:19:05.321 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:19:05.321 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:19:05.322 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 30 2021-11-03T17:19:05.322 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:19:05.322 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:19:05.322 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:05.222141+0000 2021-11-03T17:19:05.323 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:19:05.323 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:19:05.323 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:19:05.323 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:19:05.324 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:19:05.324 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:19:05.324 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:19:05.324 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:19:05.324 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:19:05.325 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 2 2021-11-03T17:19:05.325 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:19:05.325 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:19:05.325 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:19:05.325 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:19:05.326 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:19:05.326 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:19:05.326 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:19:05.327 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:19:05.327 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:19:05.327 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:19:05.327 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:19:05.328 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:19:05.328 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:19:05.328 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:19:05.328 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:19:05.329 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:19:05.329 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:19:05.329 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:19:05.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.325+0000 7f53ca931700 1 -- 172.21.15.27:0/3142883725 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f53ac0640b0 msgr2=0x7f53ac066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:19:05.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.325+0000 7f53ca931700 1 --2- 172.21.15.27:0/3142883725 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f53ac0640b0 0x7f53ac066570 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f53b8000c00 tx=0x7f53b8016040).stop 2021-11-03T17:19:05.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.325+0000 7f53ca931700 1 -- 172.21.15.27:0/3142883725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c40f98b0 msgr2=0x7f53c40f52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:19:05.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.325+0000 7f53ca931700 1 --2- 172.21.15.27:0/3142883725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c40f98b0 0x7f53c40f52c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f53b400e7f0 tx=0x7f53b400b040).stop 2021-11-03T17:19:05.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.325+0000 7f53ca931700 1 -- 172.21.15.27:0/3142883725 shutdown_connections 2021-11-03T17:19:05.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.325+0000 7f53ca931700 1 --2- 172.21.15.27:0/3142883725 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f53ac0640b0 0x7f53ac066570 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:05.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.325+0000 7f53ca931700 1 --2- 172.21.15.27:0/3142883725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53c4060330 0x7f53c40f6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:05.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.325+0000 7f53ca931700 1 --2- 172.21.15.27:0/3142883725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c40f98b0 0x7f53c40f52c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:05.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.325+0000 7f53ca931700 1 -- 172.21.15.27:0/3142883725 >> 172.21.15.27:0/3142883725 conn(0x7f53c40f0de0 msgr2=0x7f53c40f31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:19:05.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.325+0000 7f53ca931700 1 -- 172.21.15.27:0/3142883725 shutdown_connections 2021-11-03T17:19:05.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:05.325+0000 7f53ca931700 1 -- 172.21.15.27:0/3142883725 wait complete. 2021-11-03T17:19:05.333 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 30 2021-11-03T17:19:05.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 187.363288 secs 2021-11-03T17:19:05.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:05 smithi115 ceph-mon[65453]: from='client.34244 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:19:05.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:05 smithi115 ceph-mon[65453]: from='client.34248 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:19:05.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:05 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2550112361' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:05.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 187.363288 secs 2021-11-03T17:19:05.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:05 smithi027 ceph-mon[97244]: from='client.34244 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:19:05.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:05 smithi027 ceph-mon[97244]: from='client.34248 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:19:05.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:05 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2550112361' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:06.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:06 smithi115 ceph-mon[65453]: fsmap cephfs:1/2 {1=cephfs.smithi027.jdwohn=up:active} 3 up:standby, 1 failed 2021-11-03T17:19:06.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:06 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3142883725' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:19:06.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:06 smithi115 ceph-mon[65453]: pgmap v123: 65 pgs: 18 active+undersized+degraded, 16 peering, 31 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 5354/52479 objects degraded (10.202%) 2021-11-03T17:19:06.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:19:06.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:19:06.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:19:06.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:06.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:06.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:06.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2021-11-03T17:19:06.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:06 smithi027 ceph-mon[97244]: fsmap cephfs:1/2 {1=cephfs.smithi027.jdwohn=up:active} 3 up:standby, 1 failed 2021-11-03T17:19:06.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:06 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3142883725' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:19:06.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:06 smithi027 ceph-mon[97244]: pgmap v123: 65 pgs: 18 active+undersized+degraded, 16 peering, 31 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 5354/52479 objects degraded (10.202%) 2021-11-03T17:19:06.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:19:06.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:19:06.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:19:06.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:06.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:06.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:06.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2021-11-03T17:19:07.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:07 smithi115 ceph-mon[65453]: poll took 2824.440 ms: 1 events 2021-11-03T17:19:07.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:07 smithi115 ceph-mon[65453]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2021-11-03T17:19:07.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:07 smithi115 ceph-mon[65453]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2021-11-03T17:19:07.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:07 smithi027 ceph-mon[97244]: poll took 2824.440 ms: 1 events 2021-11-03T17:19:07.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:07 smithi027 ceph-mon[97244]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2021-11-03T17:19:07.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:07 smithi027 ceph-mon[97244]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2021-11-03T17:19:08.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:08 smithi115 ceph-mon[65453]: pgmap v124: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 9778/52479 objects degraded (18.632%) 2021-11-03T17:19:08.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:08 smithi027 ceph-mon[97244]: pgmap v124: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 9778/52479 objects degraded (18.632%) 2021-11-03T17:19:09.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:09 smithi115 ceph-mon[65453]: Health check update: Degraded data redundancy: 9778/52479 objects degraded (18.632%), 34 pgs degraded (PG_DEGRADED) 2021-11-03T17:19:09.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:09 smithi115 ceph-mon[65453]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2021-11-03T17:19:09.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:09 smithi027 ceph-mon[97244]: Health check update: Degraded data redundancy: 9778/52479 objects degraded (18.632%), 34 pgs degraded (PG_DEGRADED) 2021-11-03T17:19:09.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:09 smithi027 ceph-mon[97244]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2021-11-03T17:19:10.233 INFO:journalctl@ceph.osd.4.smithi115.stdout:Nov 03 17:19:09 smithi115 conmon[75562]: 2021-11-03T17:19:09.982+0000 7f0c3eb9b0c0 -1 osd.4 73 log_to_monitors {default=true} 2021-11-03T17:19:10.484 INFO:journalctl@ceph.osd.4.smithi115.stdout:Nov 03 17:19:10 smithi115 conmon[75562]: 2021-11-03T17:19:10.455+0000 7f0c2e4d7700 -1 osd.4 73 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2021-11-03T17:19:10.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 192.363403 secs 2021-11-03T17:19:10.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:10 smithi115 ceph-mon[65453]: pgmap v125: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 9778/52479 objects degraded (18.632%) 2021-11-03T17:19:10.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:10 smithi115 ceph-mon[65453]: from='osd.4 [v2:172.21.15.115:6808/2050657468,v1:172.21.15.115:6809/2050657468]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2021-11-03T17:19:10.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:10 smithi115 ceph-mon[65453]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2021-11-03T17:19:10.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 192.363403 secs 2021-11-03T17:19:10.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:10 smithi027 ceph-mon[97244]: pgmap v125: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 564 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 9778/52479 objects degraded (18.632%) 2021-11-03T17:19:10.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:10 smithi027 ceph-mon[97244]: from='osd.4 [v2:172.21.15.115:6808/2050657468,v1:172.21.15.115:6809/2050657468]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2021-11-03T17:19:10.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:10 smithi027 ceph-mon[97244]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2021-11-03T17:19:11.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:11 smithi115 ceph-mon[65453]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2021-11-03T17:19:11.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:11 smithi115 ceph-mon[65453]: osdmap e76: 6 total, 5 up, 6 in 2021-11-03T17:19:11.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:11 smithi115 ceph-mon[65453]: from='osd.4 [v2:172.21.15.115:6808/2050657468,v1:172.21.15.115:6809/2050657468]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2021-11-03T17:19:11.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:11 smithi115 ceph-mon[65453]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2021-11-03T17:19:11.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:19:11.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:11 smithi027 ceph-mon[97244]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2021-11-03T17:19:11.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:11 smithi027 ceph-mon[97244]: osdmap e76: 6 total, 5 up, 6 in 2021-11-03T17:19:11.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:11 smithi027 ceph-mon[97244]: from='osd.4 [v2:172.21.15.115:6808/2050657468,v1:172.21.15.115:6809/2050657468]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2021-11-03T17:19:11.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:11 smithi027 ceph-mon[97244]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2021-11-03T17:19:11.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:19:12.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:12 smithi115 ceph-mon[65453]: Health check cleared: OSD_DOWN (was: 1 osds down) 2021-11-03T17:19:12.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:12 smithi115 ceph-mon[65453]: osd.4 [v2:172.21.15.115:6808/2050657468,v1:172.21.15.115:6809/2050657468] boot 2021-11-03T17:19:12.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:12 smithi115 ceph-mon[65453]: osdmap e77: 6 total, 6 up, 6 in 2021-11-03T17:19:12.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-11-03T17:19:12.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:12 smithi115 ceph-mon[65453]: pgmap v128: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 564 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 9778/52479 objects degraded (18.632%) 2021-11-03T17:19:12.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:12 smithi115 ceph-mon[65453]: osdmap e78: 6 total, 6 up, 6 in 2021-11-03T17:19:12.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:12 smithi027 ceph-mon[97244]: Health check cleared: OSD_DOWN (was: 1 osds down) 2021-11-03T17:19:12.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:12 smithi027 ceph-mon[97244]: osd.4 [v2:172.21.15.115:6808/2050657468,v1:172.21.15.115:6809/2050657468] boot 2021-11-03T17:19:12.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:12 smithi027 ceph-mon[97244]: osdmap e77: 6 total, 6 up, 6 in 2021-11-03T17:19:12.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2021-11-03T17:19:12.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:12 smithi027 ceph-mon[97244]: pgmap v128: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 564 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 9778/52479 objects degraded (18.632%) 2021-11-03T17:19:12.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:12 smithi027 ceph-mon[97244]: osdmap e78: 6 total, 6 up, 6 in 2021-11-03T17:19:15.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:14 smithi115 ceph-mon[65453]: pgmap v130: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 564 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 9778/52479 objects degraded (18.632%) 2021-11-03T17:19:15.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:14 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 197.363530 secs 2021-11-03T17:19:15.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:14 smithi027 ceph-mon[97244]: pgmap v130: 65 pgs: 34 active+undersized+degraded, 31 active+clean; 564 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 9778/52479 objects degraded (18.632%) 2021-11-03T17:19:15.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:14 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 197.363530 secs 2021-11-03T17:19:16.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:15 smithi115 ceph-mon[65453]: Health check update: Degraded data redundancy: 4930/52479 objects degraded (9.394%), 17 pgs degraded (PG_DEGRADED) 2021-11-03T17:19:16.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:15 smithi027 ceph-mon[97244]: Health check update: Degraded data redundancy: 4930/52479 objects degraded (9.394%), 17 pgs degraded (PG_DEGRADED) 2021-11-03T17:19:17.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:16 smithi115 ceph-mon[65453]: pgmap v131: 65 pgs: 6 active+clean+wait, 17 active+undersized+degraded, 42 active+clean; 564 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 4930/52479 objects degraded (9.394%) 2021-11-03T17:19:17.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:16 smithi027 ceph-mon[97244]: pgmap v131: 65 pgs: 6 active+clean+wait, 17 active+undersized+degraded, 42 active+clean; 564 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 4930/52479 objects degraded (9.394%) 2021-11-03T17:19:18.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:17 smithi115 ceph-mon[65453]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 4930/52479 objects degraded (9.394%), 17 pgs degraded) 2021-11-03T17:19:18.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:17 smithi027 ceph-mon[97244]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 4930/52479 objects degraded (9.394%), 17 pgs degraded) 2021-11-03T17:19:19.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:18 smithi115 ceph-mon[65453]: pgmap v132: 65 pgs: 6 active+clean+wait, 59 active+clean; 564 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:19:19.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:18 smithi027 ceph-mon[97244]: pgmap v132: 65 pgs: 6 active+clean+wait, 59 active+clean; 564 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:19:20.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:19 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 202.363612 secs 2021-11-03T17:19:20.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:19 smithi115 ceph-mon[65453]: pgmap v133: 65 pgs: 6 active+clean+wait, 59 active+clean; 564 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:19:20.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:19 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 202.363612 secs 2021-11-03T17:19:20.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:19 smithi027 ceph-mon[97244]: pgmap v133: 65 pgs: 6 active+clean+wait, 59 active+clean; 564 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:19:21.440 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2021-11-03T17:19:21.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2021-11-03T17:19:22.710 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:22 smithi115 ceph-mon[65453]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2021-11-03T17:19:22.710 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:22 smithi115 ceph-mon[65453]: Upgrade: osd.5 is safe to restart 2021-11-03T17:19:22.710 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:22 smithi115 ceph-mon[65453]: poll took 15038.934 ms: 1 events 2021-11-03T17:19:22.711 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:22 smithi115 ceph-mon[65453]: pgmap v134: 65 pgs: 6 active+clean+wait, 59 active+clean; 564 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:19:22.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:22 smithi027 ceph-mon[97244]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2021-11-03T17:19:22.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:22 smithi027 ceph-mon[97244]: Upgrade: osd.5 is safe to restart 2021-11-03T17:19:22.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:22 smithi027 ceph-mon[97244]: poll took 15038.934 ms: 1 events 2021-11-03T17:19:22.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:22 smithi027 ceph-mon[97244]: pgmap v134: 65 pgs: 6 active+clean+wait, 59 active+clean; 564 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:19:23.956 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:23 smithi115 ceph-mon[65453]: poll took 1522.348 ms: 1 events 2021-11-03T17:19:23.956 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:23 smithi115 ceph-mon[65453]: Upgrade: Updating osd.5 2021-11-03T17:19:23.957 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:19:23.957 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:23 smithi115 ceph-mon[65453]: Deploying daemon osd.5 on smithi115 2021-11-03T17:19:24.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:23 smithi027 ceph-mon[97244]: poll took 1522.348 ms: 1 events 2021-11-03T17:19:24.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:23 smithi027 ceph-mon[97244]: Upgrade: Updating osd.5 2021-11-03T17:19:24.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:19:24.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:23 smithi027 ceph-mon[97244]: Deploying daemon osd.5 on smithi115 2021-11-03T17:19:24.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:24 smithi115 ceph-mon[65453]: pgmap v135: 65 pgs: 6 active+clean+wait, 59 active+clean; 564 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:19:24.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:24 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 207.363736 secs 2021-11-03T17:19:25.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:24 smithi027 ceph-mon[97244]: pgmap v135: 65 pgs: 6 active+clean+wait, 59 active+clean; 564 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:19:25.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:24 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 207.363736 secs 2021-11-03T17:19:25.484 INFO:journalctl@ceph.osd.5.smithi115.stdout:Nov 03 17:19:25 smithi115 systemd[1]: Stopping Ceph osd.5 for 32ab977e-3cc8-11ec-8c28-001a4aab830c... 2021-11-03T17:19:25.487 INFO:journalctl@ceph.osd.5.smithi115.stdout:Nov 03 17:19:25 smithi115 conmon[50881]: debug 2021-11-03T17:19:25.454+0000 7f797368c700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2021-11-03T17:19:25.487 INFO:journalctl@ceph.osd.5.smithi115.stdout:Nov 03 17:19:25 smithi115 conmon[50881]: debug 2021-11-03T17:19:25.455+0000 7f797368c700 -1 osd.5 78 *** Got signal Terminated *** 2021-11-03T17:19:25.487 INFO:journalctl@ceph.osd.5.smithi115.stdout:Nov 03 17:19:25 smithi115 conmon[50881]: debug 2021-11-03T17:19:25.455+0000 7f797368c700 -1 osd.5 78 *** Immediate shutdown (osd_fast_shutdown=true) *** 2021-11-03T17:19:25.984 INFO:journalctl@ceph.osd.5.smithi115.stdout:Nov 03 17:19:25 smithi115 bash[78008]: 4a54c4d6b6f5e0011f186d4e347766cc193554c1345cff306bff6bf341cdf890 2021-11-03T17:19:25.985 INFO:journalctl@ceph.osd.5.smithi115.stdout:Nov 03 17:19:25 smithi115 bash[78008]: Error: no container with name or ID ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c-osd-5 found: no such container 2021-11-03T17:19:26.484 INFO:journalctl@ceph.osd.5.smithi115.stdout:Nov 03 17:19:26 smithi115 bash[78008]: Error: container 4a54c4d6b6f5e0011f186d4e347766cc193554c1345cff306bff6bf341cdf890 does not exist in database: no such container 2021-11-03T17:19:26.870 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:19:26.870 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:26 smithi115 ceph-mon[65453]: osd.5 reported immediately failed by osd.2 2021-11-03T17:19:26.871 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:26 smithi115 ceph-mon[65453]: osd.5 failed (root=default,host=smithi115) (connection refused reported by osd.2) 2021-11-03T17:19:26.871 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:26 smithi115 ceph-mon[65453]: osd.5 reported immediately failed by osd.1 2021-11-03T17:19:26.871 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:26 smithi115 ceph-mon[65453]: osd.5 reported immediately failed by osd.2 2021-11-03T17:19:26.871 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:26 smithi115 ceph-mon[65453]: osd.5 reported immediately failed by osd.4 2021-11-03T17:19:26.872 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:26 smithi115 ceph-mon[65453]: osd.5 reported immediately failed by osd.0 2021-11-03T17:19:26.872 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:26 smithi115 ceph-mon[65453]: osd.5 reported immediately failed by osd.1 2021-11-03T17:19:26.872 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:26 smithi115 ceph-mon[65453]: osd.5 reported immediately failed by osd.3 2021-11-03T17:19:26.872 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:26 smithi115 ceph-mon[65453]: osd.5 reported immediately failed by osd.0 2021-11-03T17:19:26.873 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:26 smithi115 ceph-mon[65453]: osd.5 reported immediately failed by osd.4 2021-11-03T17:19:26.873 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:26 smithi115 ceph-mon[65453]: osd.5 reported immediately failed by osd.3 2021-11-03T17:19:26.873 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:19:26.873 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:26 smithi115 ceph-mon[65453]: pgmap v136: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:19:26.874 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:26 smithi115 ceph-mon[65453]: osd.5 reported immediately failed by osd.3 2021-11-03T17:19:26.874 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:26 smithi115 ceph-mon[65453]: osd.5 reported immediately failed by osd.1 2021-11-03T17:19:26.874 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:26 smithi115 ceph-mon[65453]: osd.5 reported immediately failed by osd.1 2021-11-03T17:19:26.874 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:26 smithi115 ceph-mon[65453]: osd.5 reported immediately failed by osd.1 2021-11-03T17:19:26.875 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:26 smithi115 ceph-mon[65453]: osd.5 reported immediately failed by osd.1 2021-11-03T17:19:26.875 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:26 smithi115 ceph-mon[65453]: osd.5 reported immediately failed by osd.2 2021-11-03T17:19:26.875 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:26 smithi115 ceph-mon[65453]: osd.5 reported immediately failed by osd.4 2021-11-03T17:19:26.875 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:26 smithi115 ceph-mon[65453]: osd.5 reported immediately failed by osd.0 2021-11-03T17:19:26.875 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:26 smithi115 ceph-mon[65453]: osd.5 reported immediately failed by osd.2 2021-11-03T17:19:26.876 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:26 smithi115 ceph-mon[65453]: osd.5 reported immediately failed by osd.3 2021-11-03T17:19:26.876 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:26 smithi115 ceph-mon[65453]: osd.5 reported immediately failed by osd.0 2021-11-03T17:19:26.876 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:26 smithi115 ceph-mon[65453]: osd.5 reported immediately failed by osd.4 2021-11-03T17:19:26.876 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:26 smithi115 ceph-mon[65453]: osd.5 reported immediately failed by osd.3 2021-11-03T17:19:26.877 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:26 smithi115 ceph-mon[65453]: osd.5 reported immediately failed by osd.1 2021-11-03T17:19:27.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:19:27.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:26 smithi027 ceph-mon[97244]: osd.5 reported immediately failed by osd.2 2021-11-03T17:19:27.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:26 smithi027 ceph-mon[97244]: osd.5 failed (root=default,host=smithi115) (connection refused reported by osd.2) 2021-11-03T17:19:27.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:26 smithi027 ceph-mon[97244]: osd.5 reported immediately failed by osd.1 2021-11-03T17:19:27.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:26 smithi027 ceph-mon[97244]: osd.5 reported immediately failed by osd.2 2021-11-03T17:19:27.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:26 smithi027 ceph-mon[97244]: osd.5 reported immediately failed by osd.4 2021-11-03T17:19:27.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:26 smithi027 ceph-mon[97244]: osd.5 reported immediately failed by osd.0 2021-11-03T17:19:27.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:26 smithi027 ceph-mon[97244]: osd.5 reported immediately failed by osd.1 2021-11-03T17:19:27.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:26 smithi027 ceph-mon[97244]: osd.5 reported immediately failed by osd.3 2021-11-03T17:19:27.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:26 smithi027 ceph-mon[97244]: osd.5 reported immediately failed by osd.0 2021-11-03T17:19:27.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:26 smithi027 ceph-mon[97244]: osd.5 reported immediately failed by osd.4 2021-11-03T17:19:27.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:26 smithi027 ceph-mon[97244]: osd.5 reported immediately failed by osd.3 2021-11-03T17:19:27.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:19:27.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:26 smithi027 ceph-mon[97244]: pgmap v136: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:19:27.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:26 smithi027 ceph-mon[97244]: osd.5 reported immediately failed by osd.3 2021-11-03T17:19:27.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:26 smithi027 ceph-mon[97244]: osd.5 reported immediately failed by osd.1 2021-11-03T17:19:27.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:26 smithi027 ceph-mon[97244]: osd.5 reported immediately failed by osd.1 2021-11-03T17:19:27.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:26 smithi027 ceph-mon[97244]: osd.5 reported immediately failed by osd.1 2021-11-03T17:19:27.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:26 smithi027 ceph-mon[97244]: osd.5 reported immediately failed by osd.1 2021-11-03T17:19:27.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:26 smithi027 ceph-mon[97244]: osd.5 reported immediately failed by osd.2 2021-11-03T17:19:27.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:26 smithi027 ceph-mon[97244]: osd.5 reported immediately failed by osd.4 2021-11-03T17:19:27.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:26 smithi027 ceph-mon[97244]: osd.5 reported immediately failed by osd.0 2021-11-03T17:19:27.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:26 smithi027 ceph-mon[97244]: osd.5 reported immediately failed by osd.2 2021-11-03T17:19:27.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:26 smithi027 ceph-mon[97244]: osd.5 reported immediately failed by osd.3 2021-11-03T17:19:27.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:26 smithi027 ceph-mon[97244]: osd.5 reported immediately failed by osd.0 2021-11-03T17:19:27.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:26 smithi027 ceph-mon[97244]: osd.5 reported immediately failed by osd.4 2021-11-03T17:19:27.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:26 smithi027 ceph-mon[97244]: osd.5 reported immediately failed by osd.3 2021-11-03T17:19:27.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:26 smithi027 ceph-mon[97244]: osd.5 reported immediately failed by osd.1 2021-11-03T17:19:27.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:27 smithi115 ceph-mon[65453]: Health check failed: 1 osds down (OSD_DOWN) 2021-11-03T17:19:27.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:27 smithi115 ceph-mon[65453]: osdmap e79: 6 total, 5 up, 6 in 2021-11-03T17:19:27.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:27 smithi115 ceph-mon[65453]: osdmap e80: 6 total, 5 up, 6 in 2021-11-03T17:19:27.735 INFO:journalctl@ceph.osd.5.smithi115.stdout:Nov 03 17:19:27 smithi115 systemd[1]: ceph-32ab977e-3cc8-11ec-8c28-001a4aab830c@osd.5.service: Succeeded. 2021-11-03T17:19:27.736 INFO:journalctl@ceph.osd.5.smithi115.stdout:Nov 03 17:19:27 smithi115 systemd[1]: Stopped Ceph osd.5 for 32ab977e-3cc8-11ec-8c28-001a4aab830c. 2021-11-03T17:19:27.736 INFO:journalctl@ceph.osd.5.smithi115.stdout:Nov 03 17:19:27 smithi115 systemd[1]: Starting Ceph osd.5 for 32ab977e-3cc8-11ec-8c28-001a4aab830c... 2021-11-03T17:19:28.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:27 smithi027 ceph-mon[97244]: Health check failed: 1 osds down (OSD_DOWN) 2021-11-03T17:19:28.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:27 smithi027 ceph-mon[97244]: osdmap e79: 6 total, 5 up, 6 in 2021-11-03T17:19:28.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:27 smithi027 ceph-mon[97244]: osdmap e80: 6 total, 5 up, 6 in 2021-11-03T17:19:28.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:28 smithi115 ceph-mon[65453]: pgmap v139: 65 pgs: 13 stale+active+clean, 52 active+clean; 564 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:19:28.734 INFO:journalctl@ceph.osd.5.smithi115.stdout:Nov 03 17:19:28 smithi115 bash[78487]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2021-11-03T17:19:28.735 INFO:journalctl@ceph.osd.5.smithi115.stdout:Nov 03 17:19:28 smithi115 bash[78487]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --dev /dev/mapper/vg_nvme-lv_2 --path /var/lib/ceph/osd/ceph-5 --no-mon-config 2021-11-03T17:19:28.735 INFO:journalctl@ceph.osd.5.smithi115.stdout:Nov 03 17:19:28 smithi115 bash[78487]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2021-11-03T17:19:28.735 INFO:journalctl@ceph.osd.5.smithi115.stdout:Nov 03 17:19:28 smithi115 bash[78487]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2021-11-03T17:19:28.735 INFO:journalctl@ceph.osd.5.smithi115.stdout:Nov 03 17:19:28 smithi115 bash[78487]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2021-11-03T17:19:28.736 INFO:journalctl@ceph.osd.5.smithi115.stdout:Nov 03 17:19:28 smithi115 bash[78487]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2021-11-03T17:19:28.736 INFO:journalctl@ceph.osd.5.smithi115.stdout:Nov 03 17:19:28 smithi115 bash[78487]: --> ceph-volume raw activate successful for osd ID: 5 2021-11-03T17:19:29.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:28 smithi027 ceph-mon[97244]: pgmap v139: 65 pgs: 13 stale+active+clean, 52 active+clean; 564 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail 2021-11-03T17:19:29.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:29 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 212.363857 secs 2021-11-03T17:19:29.734 INFO:journalctl@ceph.osd.5.smithi115.stdout:Nov 03 17:19:29 smithi115 bash[78487]: 059cccf0e5c80910c963c212a52b974af97430fff4ae2e7c855c6ea1d6b2af33 2021-11-03T17:19:29.735 INFO:journalctl@ceph.osd.5.smithi115.stdout:Nov 03 17:19:29 smithi115 systemd[1]: Started Ceph osd.5 for 32ab977e-3cc8-11ec-8c28-001a4aab830c. 2021-11-03T17:19:30.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:29 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 212.363857 secs 2021-11-03T17:19:30.484 INFO:journalctl@ceph.osd.5.smithi115.stdout:Nov 03 17:19:30 smithi115 conmon[78984]: 2021-11-03T17:19:30.093+0000 7feafe05b0c0 -1 Falling back to public interface 2021-11-03T17:19:30.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:30 smithi115 ceph-mon[65453]: poll took 6615.039 ms: 1 events 2021-11-03T17:19:30.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:19:30.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:19:30.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:30 smithi115 ceph-mon[65453]: pgmap v140: 65 pgs: 15 active+undersized+degraded, 7 stale+active+clean, 43 active+clean; 564 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 3303/52479 objects degraded (6.294%) 2021-11-03T17:19:31.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:30 smithi027 ceph-mon[97244]: poll took 6615.039 ms: 1 events 2021-11-03T17:19:31.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:19:31.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:19:31.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:30 smithi027 ceph-mon[97244]: pgmap v140: 65 pgs: 15 active+undersized+degraded, 7 stale+active+clean, 43 active+clean; 564 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 3303/52479 objects degraded (6.294%) 2021-11-03T17:19:31.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:31 smithi115 ceph-mon[65453]: Health check failed: Degraded data redundancy: 3303/52479 objects degraded (6.294%), 15 pgs degraded (PG_DEGRADED) 2021-11-03T17:19:32.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:31 smithi027 ceph-mon[97244]: Health check failed: Degraded data redundancy: 3303/52479 objects degraded (6.294%), 15 pgs degraded (PG_DEGRADED) 2021-11-03T17:19:32.937 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:32 smithi027 ceph-mon[97244]: pgmap v141: 65 pgs: 28 active+undersized+degraded, 37 active+clean; 564 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 7640/52479 objects degraded (14.558%) 2021-11-03T17:19:32.938 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:19:32.938 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:19:32.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:32 smithi115 ceph-mon[65453]: pgmap v141: 65 pgs: 28 active+undersized+degraded, 37 active+clean; 564 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 7640/52479 objects degraded (14.558%) 2021-11-03T17:19:32.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:19:32.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:19:33.956 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:33 smithi115 ceph-mon[65453]: poll took 2928.426 ms: 1 events 2021-11-03T17:19:33.956 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:19:33.956 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:33.956 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:33.957 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:33.957 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:33.957 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:33 smithi115 ceph-mon[65453]: Upgrade: Setting container_image for all osd 2021-11-03T17:19:33.957 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:19:33.958 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2021-11-03T17:19:33.958 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2021-11-03T17:19:33.958 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2021-11-03T17:19:33.959 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2021-11-03T17:19:33.959 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2021-11-03T17:19:33.959 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2021-11-03T17:19:33.959 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2021-11-03T17:19:33.960 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2021-11-03T17:19:33.960 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2021-11-03T17:19:33.960 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2021-11-03T17:19:33.960 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2021-11-03T17:19:33.961 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2021-11-03T17:19:33.961 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:33 smithi115 ceph-mon[65453]: Upgrade: Setting require_osd_release to 17 quincy 2021-11-03T17:19:33.961 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd require-osd-release", "release": "quincy"}]: dispatch 2021-11-03T17:19:34.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:33 smithi027 ceph-mon[97244]: poll took 2928.426 ms: 1 events 2021-11-03T17:19:34.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:19:34.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:34.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:34.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:34.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:34.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:33 smithi027 ceph-mon[97244]: Upgrade: Setting container_image for all osd 2021-11-03T17:19:34.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:19:34.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2021-11-03T17:19:34.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2021-11-03T17:19:34.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2021-11-03T17:19:34.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2021-11-03T17:19:34.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2021-11-03T17:19:34.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2021-11-03T17:19:34.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2021-11-03T17:19:34.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2021-11-03T17:19:34.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2021-11-03T17:19:34.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2021-11-03T17:19:34.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2021-11-03T17:19:34.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2021-11-03T17:19:34.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:33 smithi027 ceph-mon[97244]: Upgrade: Setting require_osd_release to 17 quincy 2021-11-03T17:19:34.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd require-osd-release", "release": "quincy"}]: dispatch 2021-11-03T17:19:34.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "osd require-osd-release", "release": "quincy"}]': finished 2021-11-03T17:19:34.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:34 smithi115 ceph-mon[65453]: osdmap e81: 6 total, 5 up, 6 in 2021-11-03T17:19:34.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:34 smithi115 ceph-mon[65453]: Upgrade: Scaling down filesystem cephfs 2021-11-03T17:19:34.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:19:34.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2021-11-03T17:19:34.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:34 smithi115 ceph-mon[65453]: pgmap v143: 65 pgs: 28 active+undersized+degraded, 37 active+clean; 564 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 7640/52479 objects degraded (14.558%) 2021-11-03T17:19:34.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:34 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 217.363979 secs 2021-11-03T17:19:35.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "osd require-osd-release", "release": "quincy"}]': finished 2021-11-03T17:19:35.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:34 smithi027 ceph-mon[97244]: osdmap e81: 6 total, 5 up, 6 in 2021-11-03T17:19:35.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:34 smithi027 ceph-mon[97244]: Upgrade: Scaling down filesystem cephfs 2021-11-03T17:19:35.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:19:35.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]: dispatch 2021-11-03T17:19:35.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:34 smithi027 ceph-mon[97244]: pgmap v143: 65 pgs: 28 active+undersized+degraded, 37 active+clean; 564 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 7640/52479 objects degraded (14.558%) 2021-11-03T17:19:35.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:34 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 217.363979 secs 2021-11-03T17:19:35.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.440+0000 7fe05e0ec700 1 -- 172.21.15.27:0/434749999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0580f91d0 msgr2=0x7fe0580f9c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:19:35.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.440+0000 7fe05e0ec700 1 --2- 172.21.15.27:0/434749999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0580f91d0 0x7fe0580f9c00 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7fe040004660 tx=0x7fe040009b30).stop 2021-11-03T17:19:35.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.440+0000 7fe05e0ec700 1 -- 172.21.15.27:0/434749999 shutdown_connections 2021-11-03T17:19:35.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.440+0000 7fe05e0ec700 1 --2- 172.21.15.27:0/434749999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0580f91d0 0x7fe0580f9c00 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:35.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.440+0000 7fe05e0ec700 1 --2- 172.21.15.27:0/434749999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0580f88c0 0x7fe0580f8ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:35.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.440+0000 7fe05e0ec700 1 -- 172.21.15.27:0/434749999 >> 172.21.15.27:0/434749999 conn(0x7fe0580f40e0 msgr2=0x7fe0580f6500 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:19:35.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.440+0000 7fe05e0ec700 1 -- 172.21.15.27:0/434749999 shutdown_connections 2021-11-03T17:19:35.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.440+0000 7fe05e0ec700 1 -- 172.21.15.27:0/434749999 wait complete. 2021-11-03T17:19:35.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.441+0000 7fe05e0ec700 1 Processor -- start 2021-11-03T17:19:35.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.441+0000 7fe05e0ec700 1 -- start start 2021-11-03T17:19:35.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.441+0000 7fe05e0ec700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0580f88c0 0x7fe058109100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:19:35.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.441+0000 7fe05e0ec700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0580f91d0 0x7fe0581095f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:19:35.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.441+0000 7fe05e0ec700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe058109ba0 con 0x7fe0580f88c0 2021-11-03T17:19:35.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.441+0000 7fe05e0ec700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe058109ce0 con 0x7fe0580f91d0 2021-11-03T17:19:35.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.441+0000 7fe0577fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0580f88c0 0x7fe058109100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:19:35.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.441+0000 7fe0577fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0580f88c0 0x7fe058109100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56768/0 (socket says 172.21.15.27:56768) 2021-11-03T17:19:35.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.441+0000 7fe0577fe700 1 -- 172.21.15.27:0/1149018516 learned_addr learned my addr 172.21.15.27:0/1149018516 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:19:35.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.441+0000 7fe056ffd700 1 --2- 172.21.15.27:0/1149018516 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0580f91d0 0x7fe0581095f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:19:35.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.441+0000 7fe0577fe700 1 -- 172.21.15.27:0/1149018516 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0580f91d0 msgr2=0x7fe0581095f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:19:35.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.441+0000 7fe0577fe700 1 --2- 172.21.15.27:0/1149018516 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0580f91d0 0x7fe0581095f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:35.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.441+0000 7fe0577fe700 1 -- 172.21.15.27:0/1149018516 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe040005040 con 0x7fe0580f88c0 2021-11-03T17:19:35.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.442+0000 7fe0577fe700 1 --2- 172.21.15.27:0/1149018516 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0580f88c0 0x7fe058109100 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fe05000e7f0 tx=0x7fe05000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:19:35.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.442+0000 7fe04ffff700 1 -- 172.21.15.27:0/1149018516 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0500092e0 con 0x7fe0580f88c0 2021-11-03T17:19:35.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.442+0000 7fe04ffff700 1 -- 172.21.15.27:0/1149018516 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fe05000f040 con 0x7fe0580f88c0 2021-11-03T17:19:35.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.442+0000 7fe05e0ec700 1 -- 172.21.15.27:0/1149018516 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe05810a680 con 0x7fe0580f88c0 2021-11-03T17:19:35.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.442+0000 7fe04ffff700 1 -- 172.21.15.27:0/1149018516 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe050014750 con 0x7fe0580f88c0 2021-11-03T17:19:35.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.442+0000 7fe05e0ec700 1 -- 172.21.15.27:0/1149018516 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe05810ac40 con 0x7fe0580f88c0 2021-11-03T17:19:35.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.443+0000 7fe05e0ec700 1 -- 172.21.15.27:0/1149018516 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe058045ad0 con 0x7fe0580f88c0 2021-11-03T17:19:35.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.443+0000 7fe04ffff700 1 -- 172.21.15.27:0/1149018516 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fe05001b030 con 0x7fe0580f88c0 2021-11-03T17:19:35.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.444+0000 7fe04ffff700 1 --2- 172.21.15.27:0/1149018516 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe0440640b0 0x7fe044066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:19:35.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.444+0000 7fe04ffff700 1 -- 172.21.15.27:0/1149018516 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fe0500805d0 con 0x7fe0580f88c0 2021-11-03T17:19:35.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.444+0000 7fe056ffd700 1 --2- 172.21.15.27:0/1149018516 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe0440640b0 0x7fe044066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:19:35.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.444+0000 7fe056ffd700 1 --2- 172.21.15.27:0/1149018516 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe0440640b0 0x7fe044066570 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fe040005010 tx=0x7fe040016040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:19:35.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.448+0000 7fe04ffff700 1 -- 172.21.15.27:0/1149018516 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fe0500528f0 con 0x7fe0580f88c0 2021-11-03T17:19:35.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.631+0000 7fe05e0ec700 1 -- 172.21.15.27:0/1149018516 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe05810aeb0 con 0x7fe0440640b0 2021-11-03T17:19:35.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.632+0000 7fe04ffff700 1 -- 172.21.15.27:0/1149018516 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fe05810aeb0 con 0x7fe0440640b0 2021-11-03T17:19:35.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.634+0000 7fe05e0ec700 1 -- 172.21.15.27:0/1149018516 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe0440640b0 msgr2=0x7fe044066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:19:35.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.634+0000 7fe05e0ec700 1 --2- 172.21.15.27:0/1149018516 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe0440640b0 0x7fe044066570 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fe040005010 tx=0x7fe040016040).stop 2021-11-03T17:19:35.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.634+0000 7fe05e0ec700 1 -- 172.21.15.27:0/1149018516 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0580f88c0 msgr2=0x7fe058109100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:19:35.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.634+0000 7fe05e0ec700 1 --2- 172.21.15.27:0/1149018516 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0580f88c0 0x7fe058109100 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fe05000e7f0 tx=0x7fe05000b040).stop 2021-11-03T17:19:35.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.634+0000 7fe05e0ec700 1 -- 172.21.15.27:0/1149018516 shutdown_connections 2021-11-03T17:19:35.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.634+0000 7fe05e0ec700 1 --2- 172.21.15.27:0/1149018516 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe0440640b0 0x7fe044066570 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:35.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.634+0000 7fe05e0ec700 1 --2- 172.21.15.27:0/1149018516 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0580f88c0 0x7fe058109100 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:35.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.634+0000 7fe05e0ec700 1 --2- 172.21.15.27:0/1149018516 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0580f91d0 0x7fe0581095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:35.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.634+0000 7fe05e0ec700 1 -- 172.21.15.27:0/1149018516 >> 172.21.15.27:0/1149018516 conn(0x7fe0580f40e0 msgr2=0x7fe0580fd650 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:19:35.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.634+0000 7fe05e0ec700 1 -- 172.21.15.27:0/1149018516 shutdown_connections 2021-11-03T17:19:35.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.634+0000 7fe05e0ec700 1 -- 172.21.15.27:0/1149018516 wait complete. 2021-11-03T17:19:35.640 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:19:35.733 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.736+0000 7f2a5e658700 1 -- 172.21.15.27:0/1789620600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a580f5440 msgr2=0x7f2a580f5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:19:35.734 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.736+0000 7f2a5e658700 1 --2- 172.21.15.27:0/1789620600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a580f5440 0x7f2a580f5860 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f2a54004660 tx=0x7f2a54009b30).stop 2021-11-03T17:19:35.734 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.736+0000 7f2a5e658700 1 -- 172.21.15.27:0/1789620600 shutdown_connections 2021-11-03T17:19:35.734 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.736+0000 7f2a5e658700 1 --2- 172.21.15.27:0/1789620600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a580f66e0 0x7f2a580f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:35.735 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.736+0000 7f2a5e658700 1 --2- 172.21.15.27:0/1789620600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a580f5440 0x7f2a580f5860 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:35.735 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.736+0000 7f2a5e658700 1 -- 172.21.15.27:0/1789620600 >> 172.21.15.27:0/1789620600 conn(0x7f2a580f0ba0 msgr2=0x7f2a580f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:19:35.735 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.736+0000 7f2a5e658700 1 -- 172.21.15.27:0/1789620600 shutdown_connections 2021-11-03T17:19:35.735 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.736+0000 7f2a5e658700 1 -- 172.21.15.27:0/1789620600 wait complete. 2021-11-03T17:19:35.736 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.737+0000 7f2a5e658700 1 Processor -- start 2021-11-03T17:19:35.736 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.737+0000 7f2a5e658700 1 -- start start 2021-11-03T17:19:35.736 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.737+0000 7f2a5e658700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a580f5440 0x7f2a58109aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:19:35.736 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.737+0000 7f2a5e658700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a580f66e0 0x7f2a5810bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:19:35.737 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.737+0000 7f2a5e658700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a580603e0 con 0x7f2a580f66e0 2021-11-03T17:19:35.737 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.737+0000 7f2a5e658700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a58060520 con 0x7f2a580f5440 2021-11-03T17:19:35.737 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.737+0000 7f2a5ce55700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a580f66e0 0x7f2a5810bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:19:35.738 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.737+0000 7f2a5ce55700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a580f66e0 0x7f2a5810bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56780/0 (socket says 172.21.15.27:56780) 2021-11-03T17:19:35.738 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.737+0000 7f2a5ce55700 1 -- 172.21.15.27:0/2726060376 learned_addr learned my addr 172.21.15.27:0/2726060376 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:19:35.739 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.737+0000 7f2a5d656700 1 --2- 172.21.15.27:0/2726060376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a580f5440 0x7f2a58109aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:19:35.739 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.738+0000 7f2a5ce55700 1 -- 172.21.15.27:0/2726060376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a580f5440 msgr2=0x7f2a58109aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:19:35.739 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.738+0000 7f2a5ce55700 1 --2- 172.21.15.27:0/2726060376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a580f5440 0x7f2a58109aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:35.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.738+0000 7f2a5ce55700 1 -- 172.21.15.27:0/2726060376 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a54005040 con 0x7f2a580f66e0 2021-11-03T17:19:35.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.738+0000 7f2a5ce55700 1 --2- 172.21.15.27:0/2726060376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a580f66e0 0x7f2a5810bfa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f2a4c0026d0 tx=0x7f2a4c009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:19:35.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.738+0000 7f2a49ffb700 1 -- 172.21.15.27:0/2726060376 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a4c0079c0 con 0x7f2a580f66e0 2021-11-03T17:19:35.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.738+0000 7f2a5e658700 1 -- 172.21.15.27:0/2726060376 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a5810c550 con 0x7f2a580f66e0 2021-11-03T17:19:35.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.738+0000 7f2a49ffb700 1 -- 172.21.15.27:0/2726060376 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f2a4c00f040 con 0x7f2a580f66e0 2021-11-03T17:19:35.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.738+0000 7f2a49ffb700 1 -- 172.21.15.27:0/2726060376 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a4c0148b0 con 0x7f2a580f66e0 2021-11-03T17:19:35.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.738+0000 7f2a5e658700 1 -- 172.21.15.27:0/2726060376 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a58105c50 con 0x7f2a580f66e0 2021-11-03T17:19:35.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.739+0000 7f2a5e658700 1 -- 172.21.15.27:0/2726060376 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a58045ad0 con 0x7f2a580f66e0 2021-11-03T17:19:35.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.739+0000 7f2a49ffb700 1 -- 172.21.15.27:0/2726060376 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f2a4c007b20 con 0x7f2a580f66e0 2021-11-03T17:19:35.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.740+0000 7f2a49ffb700 1 --2- 172.21.15.27:0/2726060376 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2a44064100 0x7f2a440665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:19:35.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.740+0000 7f2a49ffb700 1 -- 172.21.15.27:0/2726060376 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f2a4c080730 con 0x7f2a580f66e0 2021-11-03T17:19:35.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.740+0000 7f2a5d656700 1 --2- 172.21.15.27:0/2726060376 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2a44064100 0x7f2a440665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:19:35.744 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.740+0000 7f2a5d656700 1 --2- 172.21.15.27:0/2726060376 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2a44064100 0x7f2a440665c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f2a54013690 tx=0x7f2a54009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:19:35.744 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.744+0000 7f2a49ffb700 1 -- 172.21.15.27:0/2726060376 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f2a4c052a90 con 0x7f2a580f66e0 2021-11-03T17:19:35.912 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.916+0000 7f2a5e658700 1 -- 172.21.15.27:0/2726060376 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a58057f10 con 0x7f2a44064100 2021-11-03T17:19:35.926 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.931+0000 7f2a49ffb700 1 -- 172.21.15.27:0/2726060376 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f2a58057f10 con 0x7f2a44064100 2021-11-03T17:19:35.927 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:19:35.927 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (11m) 83s ago 12m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:19:35.927 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (2m) 83s ago 12m 7205k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:19:35.928 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (2m) 3s ago 11m 7239k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:19:35.928 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (10m) 83s ago 12m 38.9M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:19:35.928 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (4m) 83s ago 7m 15.4M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:19:35.929 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (7m) 83s ago 7m 3836M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:19:35.929 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (3m) 3s ago 7m 14.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:19:35.929 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (5m) 3s ago 7m 16.3M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:19:35.930 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (4m) 83s ago 13m 470M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:19:35.930 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (4m) 3s ago 11m 412M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:19:35.930 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (3m) 83s ago 13m 81.2M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:19:35.930 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (3m) 3s ago 11m 69.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:19:35.931 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (12m) 83s ago 12m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:19:35.931 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (11m) 3s ago 11m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:19:35.931 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (2m) 83s ago 10m 243M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:19:35.932 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (118s) 83s ago 10m 270M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:19:35.932 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (88s) 83s ago 10m 11.8M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:19:35.932 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (60s) 3s ago 9m 289M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:19:35.933 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (33s) 3s ago 9m 235M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:19:35.933 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (6s) 3s ago 9m 11.5M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:19:35.933 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (10m) 83s ago 11m 45.7M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:19:35.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.932+0000 7f2a5e658700 1 -- 172.21.15.27:0/2726060376 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2a44064100 msgr2=0x7f2a440665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:19:35.935 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.932+0000 7f2a5e658700 1 --2- 172.21.15.27:0/2726060376 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2a44064100 0x7f2a440665c0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f2a54013690 tx=0x7f2a54009b30).stop 2021-11-03T17:19:35.935 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.932+0000 7f2a5e658700 1 -- 172.21.15.27:0/2726060376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a580f66e0 msgr2=0x7f2a5810bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:19:35.935 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.933+0000 7f2a5e658700 1 --2- 172.21.15.27:0/2726060376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a580f66e0 0x7f2a5810bfa0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f2a4c0026d0 tx=0x7f2a4c009de0).stop 2021-11-03T17:19:35.936 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.933+0000 7f2a5e658700 1 -- 172.21.15.27:0/2726060376 shutdown_connections 2021-11-03T17:19:35.936 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.933+0000 7f2a5e658700 1 --2- 172.21.15.27:0/2726060376 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2a44064100 0x7f2a440665c0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:35.936 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.933+0000 7f2a5e658700 1 --2- 172.21.15.27:0/2726060376 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a580f5440 0x7f2a58109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:35.937 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.933+0000 7f2a5e658700 1 --2- 172.21.15.27:0/2726060376 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a580f66e0 0x7f2a5810bfa0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:35.937 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.933+0000 7f2a5e658700 1 -- 172.21.15.27:0/2726060376 >> 172.21.15.27:0/2726060376 conn(0x7f2a580f0ba0 msgr2=0x7f2a580fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:19:35.937 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.933+0000 7f2a5e658700 1 -- 172.21.15.27:0/2726060376 shutdown_connections 2021-11-03T17:19:35.938 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:35.933+0000 7f2a5e658700 1 -- 172.21.15.27:0/2726060376 wait complete. 2021-11-03T17:19:35.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2021-11-03T17:19:35.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:35 smithi115 ceph-mon[65453]: fsmap cephfs:1/2 {1=cephfs.smithi027.jdwohn=up:active} 3 up:standby, 1 failed 2021-11-03T17:19:35.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:19:35.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:19:35.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:19:35.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:35.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:35.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:35.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:35.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:35 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:19:36.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "1"}]': finished 2021-11-03T17:19:36.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:35 smithi027 ceph-mon[97244]: fsmap cephfs:1/2 {1=cephfs.smithi027.jdwohn=up:active} 3 up:standby, 1 failed 2021-11-03T17:19:36.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:19:36.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:19:36.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:19:36.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:36.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:36.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:36.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:36.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:35 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:19:36.030 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.033+0000 7f85aac6a700 1 -- 172.21.15.27:0/2449797842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85a40f6860 msgr2=0x7f85a40f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:19:36.030 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.033+0000 7f85aac6a700 1 --2- 172.21.15.27:0/2449797842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85a40f6860 0x7f85a40f6cc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f8594004660 tx=0x7f8594009b30).stop 2021-11-03T17:19:36.031 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.033+0000 7f85aac6a700 1 -- 172.21.15.27:0/2449797842 shutdown_connections 2021-11-03T17:19:36.031 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.033+0000 7f85aac6a700 1 --2- 172.21.15.27:0/2449797842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85a40f6860 0x7f85a40f6cc0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:36.031 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.033+0000 7f85aac6a700 1 --2- 172.21.15.27:0/2449797842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85a40f55c0 0x7f85a40f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:36.031 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.033+0000 7f85aac6a700 1 -- 172.21.15.27:0/2449797842 >> 172.21.15.27:0/2449797842 conn(0x7f85a40f0d40 msgr2=0x7f85a40f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:19:36.032 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.033+0000 7f85aac6a700 1 -- 172.21.15.27:0/2449797842 shutdown_connections 2021-11-03T17:19:36.032 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.033+0000 7f85aac6a700 1 -- 172.21.15.27:0/2449797842 wait complete. 2021-11-03T17:19:36.032 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.033+0000 7f85aac6a700 1 Processor -- start 2021-11-03T17:19:36.032 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.033+0000 7f85aac6a700 1 -- start start 2021-11-03T17:19:36.032 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.034+0000 7f85aac6a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85a40f55c0 0x7f85a4105e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:19:36.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.034+0000 7f85aac6a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85a40f6860 0x7f85a4106320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:19:36.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.034+0000 7f85aac6a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a4106920 con 0x7f85a40f55c0 2021-11-03T17:19:36.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.034+0000 7f85aac6a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a4107130 con 0x7f85a40f6860 2021-11-03T17:19:36.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.034+0000 7f85a8a06700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85a40f55c0 0x7f85a4105e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:19:36.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.034+0000 7f85a8a06700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85a40f55c0 0x7f85a4105e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56786/0 (socket says 172.21.15.27:56786) 2021-11-03T17:19:36.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.034+0000 7f85a8a06700 1 -- 172.21.15.27:0/2493344238 learned_addr learned my addr 172.21.15.27:0/2493344238 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:19:36.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.034+0000 7f85a3fff700 1 --2- 172.21.15.27:0/2493344238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85a40f6860 0x7f85a4106320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:19:36.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.034+0000 7f85a8a06700 1 -- 172.21.15.27:0/2493344238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85a40f6860 msgr2=0x7f85a4106320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:19:36.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.034+0000 7f85a8a06700 1 --2- 172.21.15.27:0/2493344238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85a40f6860 0x7f85a4106320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:36.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.034+0000 7f85a8a06700 1 -- 172.21.15.27:0/2493344238 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8594005040 con 0x7f85a40f55c0 2021-11-03T17:19:36.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.034+0000 7f85a8a06700 1 --2- 172.21.15.27:0/2493344238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85a40f55c0 0x7f85a4105e30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f858c00e7c0 tx=0x7f858c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:19:36.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.034+0000 7f85a17fa700 1 -- 172.21.15.27:0/2493344238 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f858c0092e0 con 0x7f85a40f55c0 2021-11-03T17:19:36.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.034+0000 7f85aac6a700 1 -- 172.21.15.27:0/2493344238 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85a41073b0 con 0x7f85a40f55c0 2021-11-03T17:19:36.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.034+0000 7f85a17fa700 1 -- 172.21.15.27:0/2493344238 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f858c00f040 con 0x7f85a40f55c0 2021-11-03T17:19:36.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.034+0000 7f85a17fa700 1 -- 172.21.15.27:0/2493344238 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f858c0147e0 con 0x7f85a40f55c0 2021-11-03T17:19:36.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.034+0000 7f85aac6a700 1 -- 172.21.15.27:0/2493344238 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a4107970 con 0x7f85a40f55c0 2021-11-03T17:19:36.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.035+0000 7f85aac6a700 1 -- 172.21.15.27:0/2493344238 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85a40f49b0 con 0x7f85a40f55c0 2021-11-03T17:19:36.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.036+0000 7f85a17fa700 1 -- 172.21.15.27:0/2493344238 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f858c01b030 con 0x7f85a40f55c0 2021-11-03T17:19:36.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.036+0000 7f85a17fa700 1 --2- 172.21.15.27:0/2493344238 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f85900640b0 0x7f8590066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:19:36.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.036+0000 7f85a17fa700 1 -- 172.21.15.27:0/2493344238 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f858c080830 con 0x7f85a40f55c0 2021-11-03T17:19:36.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.037+0000 7f85a3fff700 1 --2- 172.21.15.27:0/2493344238 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f85900640b0 0x7f8590066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:19:36.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.037+0000 7f85a3fff700 1 --2- 172.21.15.27:0/2493344238 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f85900640b0 0x7f8590066570 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f8594005010 tx=0x7f8594016040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:19:36.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.041+0000 7f85a17fa700 1 -- 172.21.15.27:0/2493344238 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f858c052aa0 con 0x7f85a40f55c0 2021-11-03T17:19:36.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.272+0000 7f85aac6a700 1 -- 172.21.15.27:0/2493344238 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f85a40008d0 con 0x7f85a40f55c0 2021-11-03T17:19:36.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.272+0000 7f85a17fa700 1 -- 172.21.15.27:0/2493344238 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+710 (secure 0 0 0) 0x7f858c019070 con 0x7f85a40f55c0 2021-11-03T17:19:36.268 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:19:36.269 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:19:36.269 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:19:36.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:19:36.269 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:19:36.270 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:19:36.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:19:36.270 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:19:36.270 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 5 2021-11-03T17:19:36.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:19:36.271 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:19:36.271 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:19:36.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:19:36.271 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:19:36.272 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:19:36.272 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 9 2021-11-03T17:19:36.272 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:19:36.272 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:19:36.273 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.274+0000 7f85aac6a700 1 -- 172.21.15.27:0/2493344238 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f85900640b0 msgr2=0x7f8590066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:19:36.274 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.274+0000 7f85aac6a700 1 --2- 172.21.15.27:0/2493344238 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f85900640b0 0x7f8590066570 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f8594005010 tx=0x7f8594016040).stop 2021-11-03T17:19:36.274 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.274+0000 7f85aac6a700 1 -- 172.21.15.27:0/2493344238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85a40f55c0 msgr2=0x7f85a4105e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:19:36.274 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.274+0000 7f85aac6a700 1 --2- 172.21.15.27:0/2493344238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85a40f55c0 0x7f85a4105e30 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f858c00e7c0 tx=0x7f858c00b040).stop 2021-11-03T17:19:36.275 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.274+0000 7f85aac6a700 1 -- 172.21.15.27:0/2493344238 shutdown_connections 2021-11-03T17:19:36.275 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.274+0000 7f85aac6a700 1 --2- 172.21.15.27:0/2493344238 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f85900640b0 0x7f8590066570 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:36.276 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.274+0000 7f85aac6a700 1 --2- 172.21.15.27:0/2493344238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85a40f55c0 0x7f85a4105e30 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:36.276 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.274+0000 7f85aac6a700 1 --2- 172.21.15.27:0/2493344238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85a40f6860 0x7f85a4106320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:36.276 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.274+0000 7f85aac6a700 1 -- 172.21.15.27:0/2493344238 >> 172.21.15.27:0/2493344238 conn(0x7f85a40f0d40 msgr2=0x7f85a40fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:19:36.276 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.274+0000 7f85aac6a700 1 -- 172.21.15.27:0/2493344238 shutdown_connections 2021-11-03T17:19:36.277 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.274+0000 7f85aac6a700 1 -- 172.21.15.27:0/2493344238 wait complete. 2021-11-03T17:19:36.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.375+0000 7f6ea004d700 1 -- 172.21.15.27:0/1841271472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e980f7980 msgr2=0x7f6e980f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:19:36.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.375+0000 7f6ea004d700 1 --2- 172.21.15.27:0/1841271472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e980f7980 0x7f6e980f7de0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f6e94004660 tx=0x7f6e94009b30).stop 2021-11-03T17:19:36.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.375+0000 7f6ea004d700 1 -- 172.21.15.27:0/1841271472 shutdown_connections 2021-11-03T17:19:36.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.375+0000 7f6ea004d700 1 --2- 172.21.15.27:0/1841271472 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e980f7980 0x7f6e980f7de0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:36.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.375+0000 7f6ea004d700 1 --2- 172.21.15.27:0/1841271472 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e980f66e0 0x7f6e980f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:36.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.375+0000 7f6ea004d700 1 -- 172.21.15.27:0/1841271472 >> 172.21.15.27:0/1841271472 conn(0x7f6e980f1e40 msgr2=0x7f6e980f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:19:36.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.375+0000 7f6ea004d700 1 -- 172.21.15.27:0/1841271472 shutdown_connections 2021-11-03T17:19:36.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.375+0000 7f6ea004d700 1 -- 172.21.15.27:0/1841271472 wait complete. 2021-11-03T17:19:36.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.376+0000 7f6ea004d700 1 Processor -- start 2021-11-03T17:19:36.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.376+0000 7f6ea004d700 1 -- start start 2021-11-03T17:19:36.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.376+0000 7f6ea004d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e980f66e0 0x7f6e98106f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:19:36.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.376+0000 7f6ea004d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e980f7980 0x7f6e98107400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:19:36.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.376+0000 7f6ea004d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e981079b0 con 0x7f6e980f7980 2021-11-03T17:19:36.382 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.376+0000 7f6ea004d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e98107af0 con 0x7f6e980f66e0 2021-11-03T17:19:36.382 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.376+0000 7f6e9d5e8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e980f7980 0x7f6e98107400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:19:36.383 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.376+0000 7f6e9d5e8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e980f7980 0x7f6e98107400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56798/0 (socket says 172.21.15.27:56798) 2021-11-03T17:19:36.383 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.376+0000 7f6e9d5e8700 1 -- 172.21.15.27:0/2776760524 learned_addr learned my addr 172.21.15.27:0/2776760524 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:19:36.384 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.376+0000 7f6e9dde9700 1 --2- 172.21.15.27:0/2776760524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e980f66e0 0x7f6e98106f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:19:36.384 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.376+0000 7f6e9d5e8700 1 -- 172.21.15.27:0/2776760524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e980f66e0 msgr2=0x7f6e98106f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:19:36.384 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.376+0000 7f6e9d5e8700 1 --2- 172.21.15.27:0/2776760524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e980f66e0 0x7f6e98106f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:36.385 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.376+0000 7f6e9d5e8700 1 -- 172.21.15.27:0/2776760524 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e94005040 con 0x7f6e980f7980 2021-11-03T17:19:36.385 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.376+0000 7f6e9d5e8700 1 --2- 172.21.15.27:0/2776760524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e980f7980 0x7f6e98107400 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f6e94002040 tx=0x7f6e9400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:19:36.386 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.376+0000 7f6e8e7fc700 1 -- 172.21.15.27:0/2776760524 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e94019070 con 0x7f6e980f7980 2021-11-03T17:19:36.386 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.376+0000 7f6e8e7fc700 1 -- 172.21.15.27:0/2776760524 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f6e94007e70 con 0x7f6e980f7980 2021-11-03T17:19:36.387 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.377+0000 7f6ea004d700 1 -- 172.21.15.27:0/2776760524 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e98108430 con 0x7f6e980f7980 2021-11-03T17:19:36.387 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.377+0000 7f6e8e7fc700 1 -- 172.21.15.27:0/2776760524 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e94014db0 con 0x7f6e980f7980 2021-11-03T17:19:36.388 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.377+0000 7f6ea004d700 1 -- 172.21.15.27:0/2776760524 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e98108990 con 0x7f6e980f7980 2021-11-03T17:19:36.388 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.378+0000 7f6ea004d700 1 -- 172.21.15.27:0/2776760524 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e98045ad0 con 0x7f6e980f7980 2021-11-03T17:19:36.388 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.378+0000 7f6e8e7fc700 1 -- 172.21.15.27:0/2776760524 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f6e94025070 con 0x7f6e980f7980 2021-11-03T17:19:36.389 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.378+0000 7f6e8e7fc700 1 --2- 172.21.15.27:0/2776760524 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6e84064090 0x7f6e84066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:19:36.389 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.378+0000 7f6e8e7fc700 1 -- 172.21.15.27:0/2776760524 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f6e94081680 con 0x7f6e980f7980 2021-11-03T17:19:36.390 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.378+0000 7f6e9dde9700 1 --2- 172.21.15.27:0/2776760524 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6e84064090 0x7f6e84066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:19:36.390 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.379+0000 7f6e9dde9700 1 --2- 172.21.15.27:0/2776760524 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6e84064090 0x7f6e84066550 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f6e88009fd0 tx=0x7f6e8800b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:19:36.391 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.382+0000 7f6e8e7fc700 1 -- 172.21.15.27:0/2776760524 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f6e940539e0 con 0x7f6e980f7980 2021-11-03T17:19:36.585 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.589+0000 7f6ea004d700 1 -- 172.21.15.27:0/2776760524 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6e98027a00 con 0x7f6e980f7980 2021-11-03T17:19:36.586 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.590+0000 7f6e8e7fc700 1 -- 172.21.15.27:0/2776760524 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f6e94023020 con 0x7f6e980f7980 2021-11-03T17:19:36.586 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:19:36.586 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:19:36.587 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:19:36.587 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:19:36.587 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:19:36.587 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:19:36.588 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:19:36.588 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:19:36.588 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:19:36.588 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:19:36.589 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:19:36.589 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:19:36.589 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:19:36.589 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:19:36.589 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:19:36.590 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:19:36.590 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:19:36.590 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:19:36.590 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:19:36.591 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:19:36.591 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:19:36.591 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:19:36.591 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:19:36.591 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:19:36.592 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:19:36.592 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:19:36.592 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:19:36.592 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:19:36.593 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:19:36.593 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:19:36.593 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:19:36.593 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:19:36.593 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:19:36.594 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:19:36.594 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:19:36.594 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:19:36.594 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:19:36.595 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:19:36.595 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:19:36.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.592+0000 7f6ea004d700 1 -- 172.21.15.27:0/2776760524 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6e84064090 msgr2=0x7f6e84066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:19:36.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.592+0000 7f6ea004d700 1 --2- 172.21.15.27:0/2776760524 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6e84064090 0x7f6e84066550 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f6e88009fd0 tx=0x7f6e8800b040).stop 2021-11-03T17:19:36.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.592+0000 7f6ea004d700 1 -- 172.21.15.27:0/2776760524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e980f7980 msgr2=0x7f6e98107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:19:36.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.592+0000 7f6ea004d700 1 --2- 172.21.15.27:0/2776760524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e980f7980 0x7f6e98107400 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f6e94002040 tx=0x7f6e9400b040).stop 2021-11-03T17:19:36.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.592+0000 7f6ea004d700 1 -- 172.21.15.27:0/2776760524 shutdown_connections 2021-11-03T17:19:36.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.592+0000 7f6ea004d700 1 --2- 172.21.15.27:0/2776760524 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6e84064090 0x7f6e84066550 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:36.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.592+0000 7f6ea004d700 1 --2- 172.21.15.27:0/2776760524 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e980f66e0 0x7f6e98106f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:36.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.592+0000 7f6ea004d700 1 --2- 172.21.15.27:0/2776760524 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e980f7980 0x7f6e98107400 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:19:36.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.592+0000 7f6ea004d700 1 -- 172.21.15.27:0/2776760524 >> 172.21.15.27:0/2776760524 conn(0x7f6e980f1e40 msgr2=0x7f6e980fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:19:36.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.592+0000 7f6ea004d700 1 -- 172.21.15.27:0/2776760524 shutdown_connections 2021-11-03T17:19:36.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:19:36.592+0000 7f6ea004d700 1 -- 172.21.15.27:0/2776760524 wait complete. 2021-11-03T17:19:36.598 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:19:36.984 INFO:journalctl@ceph.osd.5.smithi115.stdout:Nov 03 17:19:36 smithi115 conmon[78984]: 2021-11-03T17:19:36.841+0000 7feafe05b0c0 -1 osd.5 78 log_to_monitors {default=true} 2021-11-03T17:19:36.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:36 smithi115 ceph-mon[65453]: from='client.34262 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:19:36.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:36 smithi115 ceph-mon[65453]: pgmap v144: 65 pgs: 28 active+undersized+degraded, 37 active+clean; 564 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 7640/52479 objects degraded (14.558%) 2021-11-03T17:19:36.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:36 smithi115 ceph-mon[65453]: from='client.34266 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:19:36.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:36 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2493344238' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:36.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:36 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2776760524' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:19:37.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:36 smithi027 ceph-mon[97244]: from='client.34262 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:19:37.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:36 smithi027 ceph-mon[97244]: pgmap v144: 65 pgs: 28 active+undersized+degraded, 37 active+clean; 564 MiB data, 1.9 GiB used, 534 GiB / 536 GiB avail; 7640/52479 objects degraded (14.558%) 2021-11-03T17:19:37.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:36 smithi027 ceph-mon[97244]: from='client.34266 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:19:37.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:36 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2493344238' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:37.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:36 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2776760524' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:19:37.984 INFO:journalctl@ceph.osd.5.smithi115.stdout:Nov 03 17:19:37 smithi115 conmon[78984]: 2021-11-03T17:19:37.711+0000 7feaef99b700 -1 osd.5 78 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2021-11-03T17:19:37.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:37 smithi115 ceph-mon[65453]: from='osd.5 [v2:172.21.15.115:6816/3641571191,v1:172.21.15.115:6817/3641571191]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2021-11-03T17:19:37.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:37 smithi115 ceph-mon[65453]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2021-11-03T17:19:38.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:37 smithi027 ceph-mon[97244]: from='osd.5 [v2:172.21.15.115:6816/3641571191,v1:172.21.15.115:6817/3641571191]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2021-11-03T17:19:38.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:37 smithi027 ceph-mon[97244]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2021-11-03T17:19:38.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:38 smithi115 ceph-mon[65453]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2021-11-03T17:19:38.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:38 smithi115 ceph-mon[65453]: osdmap e82: 6 total, 5 up, 6 in 2021-11-03T17:19:38.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:38 smithi115 ceph-mon[65453]: from='osd.5 [v2:172.21.15.115:6816/3641571191,v1:172.21.15.115:6817/3641571191]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2021-11-03T17:19:38.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:38 smithi115 ceph-mon[65453]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2021-11-03T17:19:38.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:38 smithi115 ceph-mon[65453]: pgmap v146: 65 pgs: 28 active+undersized+degraded, 37 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 7640/52479 objects degraded (14.558%) 2021-11-03T17:19:39.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:38 smithi027 ceph-mon[97244]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2021-11-03T17:19:39.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:38 smithi027 ceph-mon[97244]: osdmap e82: 6 total, 5 up, 6 in 2021-11-03T17:19:39.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:38 smithi027 ceph-mon[97244]: from='osd.5 [v2:172.21.15.115:6816/3641571191,v1:172.21.15.115:6817/3641571191]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2021-11-03T17:19:39.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:38 smithi027 ceph-mon[97244]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi115", "root=default"]}]: dispatch 2021-11-03T17:19:39.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:38 smithi027 ceph-mon[97244]: pgmap v146: 65 pgs: 28 active+undersized+degraded, 37 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 7640/52479 objects degraded (14.558%) 2021-11-03T17:19:39.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:39 smithi115 ceph-mon[65453]: Health check cleared: OSD_DOWN (was: 1 osds down) 2021-11-03T17:19:39.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:39 smithi115 ceph-mon[65453]: osd.5 [v2:172.21.15.115:6816/3641571191,v1:172.21.15.115:6817/3641571191] boot 2021-11-03T17:19:39.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:39 smithi115 ceph-mon[65453]: osdmap e83: 6 total, 6 up, 6 in 2021-11-03T17:19:39.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-11-03T17:19:39.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:39 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 222.364101 secs 2021-11-03T17:19:39.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:39 smithi115 ceph-mon[65453]: Health check update: Degraded data redundancy: 7640/52479 objects degraded (14.558%), 28 pgs degraded (PG_DEGRADED) 2021-11-03T17:19:40.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:39 smithi027 ceph-mon[97244]: Health check cleared: OSD_DOWN (was: 1 osds down) 2021-11-03T17:19:40.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:39 smithi027 ceph-mon[97244]: osd.5 [v2:172.21.15.115:6816/3641571191,v1:172.21.15.115:6817/3641571191] boot 2021-11-03T17:19:40.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:39 smithi027 ceph-mon[97244]: osdmap e83: 6 total, 6 up, 6 in 2021-11-03T17:19:40.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2021-11-03T17:19:40.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:39 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 222.364101 secs 2021-11-03T17:19:40.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:39 smithi027 ceph-mon[97244]: Health check update: Degraded data redundancy: 7640/52479 objects degraded (14.558%), 28 pgs degraded (PG_DEGRADED) 2021-11-03T17:19:40.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:40 smithi115 ceph-mon[65453]: osdmap e84: 6 total, 6 up, 6 in 2021-11-03T17:19:40.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:40 smithi115 ceph-mon[65453]: pgmap v149: 65 pgs: 9 peering, 19 active+undersized+degraded, 37 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 5998/52479 objects degraded (11.429%) 2021-11-03T17:19:40.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:19:41.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:40 smithi027 ceph-mon[97244]: osdmap e84: 6 total, 6 up, 6 in 2021-11-03T17:19:41.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:40 smithi027 ceph-mon[97244]: pgmap v149: 65 pgs: 9 peering, 19 active+undersized+degraded, 37 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 5998/52479 objects degraded (11.429%) 2021-11-03T17:19:41.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:19:42.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:19:42.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:19:43.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:42 smithi027 ceph-mon[97244]: pgmap v150: 65 pgs: 9 peering, 13 active+undersized+degraded, 43 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 4377/52479 objects degraded (8.340%) 2021-11-03T17:19:43.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:42 smithi115 ceph-mon[65453]: pgmap v150: 65 pgs: 9 peering, 13 active+undersized+degraded, 43 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 4377/52479 objects degraded (8.340%) 2021-11-03T17:19:44.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:43 smithi115 ceph-mon[65453]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 4377/52479 objects degraded (8.340%), 13 pgs degraded) 2021-11-03T17:19:44.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:43 smithi027 ceph-mon[97244]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 4377/52479 objects degraded (8.340%), 13 pgs degraded) 2021-11-03T17:19:45.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:44 smithi027 ceph-mon[97244]: pgmap v151: 65 pgs: 13 active+clean+wait, 9 peering, 43 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:19:45.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:44 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 227.364229 secs 2021-11-03T17:19:45.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:19:45.071 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:44 smithi115 ceph-mon[65453]: pgmap v151: 65 pgs: 13 active+clean+wait, 9 peering, 43 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:19:45.071 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:44 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 227.364229 secs 2021-11-03T17:19:45.072 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:19:46.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:45 smithi115 ceph-mon[65453]: poll took 12117.543 ms: 1 events 2021-11-03T17:19:46.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:19:46.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:19:46.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:46.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:46.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:46.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:46.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:45 smithi027 ceph-mon[97244]: poll took 12117.543 ms: 1 events 2021-11-03T17:19:46.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:19:46.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:19:46.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:46.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:46.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:46.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:47.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:46 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:19:47.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:46 smithi115 ceph-mon[65453]: pgmap v152: 65 pgs: 13 active+clean+wait, 52 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:19:47.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:46 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:19:47.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:46 smithi027 ceph-mon[97244]: pgmap v152: 65 pgs: 13 active+clean+wait, 52 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:19:49.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:48 smithi115 ceph-mon[65453]: pgmap v153: 65 pgs: 13 active+clean+wait, 52 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:19:49.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:48 smithi027 ceph-mon[97244]: pgmap v153: 65 pgs: 13 active+clean+wait, 52 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:19:50.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:49 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 232.364363 secs 2021-11-03T17:19:50.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:49 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 232.364363 secs 2021-11-03T17:19:51.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:50 smithi115 ceph-mon[65453]: pgmap v154: 65 pgs: 13 active+clean+wait, 52 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:19:51.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:19:51.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:50 smithi027 ceph-mon[97244]: pgmap v154: 65 pgs: 13 active+clean+wait, 52 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:19:51.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:19:53.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:52 smithi115 ceph-mon[65453]: pgmap v155: 65 pgs: 13 active+clean+wait, 52 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:19:53.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:52 smithi027 ceph-mon[97244]: pgmap v155: 65 pgs: 13 active+clean+wait, 52 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:19:55.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:54 smithi115 ceph-mon[65453]: pgmap v156: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:19:55.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:54 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 237.364498 secs 2021-11-03T17:19:55.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:54 smithi027 ceph-mon[97244]: pgmap v156: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:19:55.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:54 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 237.364498 secs 2021-11-03T17:19:56.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:19:56.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:19:56.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:19:56.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:56.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:56.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:56.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:56.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:19:56.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:19:56.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:19:56.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:19:56.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:19:56.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:19:56.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:56.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:56.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:56.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:19:56.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:19:56.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:19:56.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:19:57.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:56 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:19:57.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:56 smithi115 ceph-mon[65453]: pgmap v157: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:19:57.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:56 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:19:57.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:56 smithi027 ceph-mon[97244]: pgmap v157: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:19:59.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:58 smithi115 ceph-mon[65453]: pgmap v158: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:19:59.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:58 smithi027 ceph-mon[97244]: pgmap v158: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:00.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:19:59 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 242.364624 secs 2021-11-03T17:20:00.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:19:59 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 242.364624 secs 2021-11-03T17:20:01.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:00 smithi115 ceph-mon[65453]: pgmap v159: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:01.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:00 smithi115 ceph-mon[65453]: Health detail: HEALTH_WARN 1 filesystem is degraded; 1 filesystem has a failed mds daemon; 1 MDSs report slow requests; 3 daemons have recently crashed 2021-11-03T17:20:01.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:00 smithi115 ceph-mon[65453]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2021-11-03T17:20:01.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:00 smithi115 ceph-mon[65453]: fs cephfs is degraded 2021-11-03T17:20:01.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:00 smithi115 ceph-mon[65453]: [WRN] FS_WITH_FAILED_MDS: 1 filesystem has a failed mds daemon 2021-11-03T17:20:01.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:00 smithi115 ceph-mon[65453]: fs cephfs has 1 failed mds 2021-11-03T17:20:01.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:00 smithi115 ceph-mon[65453]: [WRN] MDS_SLOW_REQUEST: 1 MDSs report slow requests 2021-11-03T17:20:01.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:00 smithi115 ceph-mon[65453]: mds.cephfs.smithi027.jdwohn(mds.1): 1 slow requests are blocked > 180 secs 2021-11-03T17:20:01.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:00 smithi115 ceph-mon[65453]: [WRN] RECENT_CRASH: 3 daemons have recently crashed 2021-11-03T17:20:01.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:00 smithi115 ceph-mon[65453]: mds.cephfs.smithi027.fksixo crashed on host smithi027 at 2021-11-03T17:14:44.306015Z 2021-11-03T17:20:01.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:00 smithi115 ceph-mon[65453]: mds.cephfs.smithi115.zsfgnv crashed on host smithi115 at 2021-11-03T17:14:24.305452Z 2021-11-03T17:20:01.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:00 smithi115 ceph-mon[65453]: mds.cephfs.smithi115.semywj crashed on host smithi115 at 2021-11-03T17:15:56.818332Z 2021-11-03T17:20:01.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:00 smithi027 ceph-mon[97244]: pgmap v159: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:01.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:00 smithi027 ceph-mon[97244]: Health detail: HEALTH_WARN 1 filesystem is degraded; 1 filesystem has a failed mds daemon; 1 MDSs report slow requests; 3 daemons have recently crashed 2021-11-03T17:20:01.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:00 smithi027 ceph-mon[97244]: [WRN] FS_DEGRADED: 1 filesystem is degraded 2021-11-03T17:20:01.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:00 smithi027 ceph-mon[97244]: fs cephfs is degraded 2021-11-03T17:20:01.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:00 smithi027 ceph-mon[97244]: [WRN] FS_WITH_FAILED_MDS: 1 filesystem has a failed mds daemon 2021-11-03T17:20:01.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:00 smithi027 ceph-mon[97244]: fs cephfs has 1 failed mds 2021-11-03T17:20:01.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:00 smithi027 ceph-mon[97244]: [WRN] MDS_SLOW_REQUEST: 1 MDSs report slow requests 2021-11-03T17:20:01.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:00 smithi027 ceph-mon[97244]: mds.cephfs.smithi027.jdwohn(mds.1): 1 slow requests are blocked > 180 secs 2021-11-03T17:20:01.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:00 smithi027 ceph-mon[97244]: [WRN] RECENT_CRASH: 3 daemons have recently crashed 2021-11-03T17:20:01.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:00 smithi027 ceph-mon[97244]: mds.cephfs.smithi027.fksixo crashed on host smithi027 at 2021-11-03T17:14:44.306015Z 2021-11-03T17:20:01.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:00 smithi027 ceph-mon[97244]: mds.cephfs.smithi115.zsfgnv crashed on host smithi115 at 2021-11-03T17:14:24.305452Z 2021-11-03T17:20:01.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:00 smithi027 ceph-mon[97244]: mds.cephfs.smithi115.semywj crashed on host smithi115 at 2021-11-03T17:15:56.818332Z 2021-11-03T17:20:03.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:02 smithi115 ceph-mon[65453]: pgmap v160: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:03.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:02 smithi027 ceph-mon[97244]: pgmap v160: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:05.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:04 smithi115 ceph-mon[65453]: pgmap v161: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:05.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:04 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 247.364763 secs 2021-11-03T17:20:05.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:04 smithi027 ceph-mon[97244]: pgmap v161: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:05.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:04 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 247.364763 secs 2021-11-03T17:20:06.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:20:06.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:20:06.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:20:06.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:06.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:06.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:06.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:06.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:20:06.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:20:06.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:20:06.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:06.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:06.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:06.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:06.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.709+0000 7f77eaeed700 1 -- 172.21.15.27:0/2116959285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77e40f55c0 msgr2=0x7f77e40f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:20:06.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.709+0000 7f77eaeed700 1 --2- 172.21.15.27:0/2116959285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77e40f55c0 0x7f77e40f59e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f77d8004660 tx=0x7f77d8009b30).stop 2021-11-03T17:20:06.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.709+0000 7f77eaeed700 1 -- 172.21.15.27:0/2116959285 shutdown_connections 2021-11-03T17:20:06.707 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.709+0000 7f77eaeed700 1 --2- 172.21.15.27:0/2116959285 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77e40f6860 0x7f77e40f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:06.707 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.709+0000 7f77eaeed700 1 --2- 172.21.15.27:0/2116959285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77e40f55c0 0x7f77e40f59e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:06.707 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.709+0000 7f77eaeed700 1 -- 172.21.15.27:0/2116959285 >> 172.21.15.27:0/2116959285 conn(0x7f77e40f0d40 msgr2=0x7f77e40f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:20:06.707 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.709+0000 7f77eaeed700 1 -- 172.21.15.27:0/2116959285 shutdown_connections 2021-11-03T17:20:06.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.709+0000 7f77eaeed700 1 -- 172.21.15.27:0/2116959285 wait complete. 2021-11-03T17:20:06.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.710+0000 7f77eaeed700 1 Processor -- start 2021-11-03T17:20:06.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.710+0000 7f77eaeed700 1 -- start start 2021-11-03T17:20:06.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.710+0000 7f77eaeed700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77e40f55c0 0x7f77e4109c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:20:06.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.710+0000 7f77eaeed700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77e40f6860 0x7f77e410c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:20:06.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.710+0000 7f77eaeed700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e40603e0 con 0x7f77e40f6860 2021-11-03T17:20:06.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.710+0000 7f77eaeed700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77e4060520 con 0x7f77e40f55c0 2021-11-03T17:20:06.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.710+0000 7f77e3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77e40f6860 0x7f77e410c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:20:06.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.710+0000 7f77e3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77e40f6860 0x7f77e410c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56834/0 (socket says 172.21.15.27:56834) 2021-11-03T17:20:06.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.710+0000 7f77e3fff700 1 -- 172.21.15.27:0/3837311859 learned_addr learned my addr 172.21.15.27:0/3837311859 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:20:06.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.711+0000 7f77e8c89700 1 --2- 172.21.15.27:0/3837311859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77e40f55c0 0x7f77e4109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:20:06.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.711+0000 7f77e3fff700 1 -- 172.21.15.27:0/3837311859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77e40f55c0 msgr2=0x7f77e4109c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:20:06.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.711+0000 7f77e3fff700 1 --2- 172.21.15.27:0/3837311859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77e40f55c0 0x7f77e4109c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:06.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.711+0000 7f77e3fff700 1 -- 172.21.15.27:0/3837311859 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77d8005040 con 0x7f77e40f6860 2021-11-03T17:20:06.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.711+0000 7f77e3fff700 1 --2- 172.21.15.27:0/3837311859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77e40f6860 0x7f77e410c170 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f77d000c530 tx=0x7f77d0007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:20:06.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.711+0000 7f77e17fa700 1 -- 172.21.15.27:0/3837311859 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77d000f650 con 0x7f77e40f6860 2021-11-03T17:20:06.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.711+0000 7f77e17fa700 1 -- 172.21.15.27:0/3837311859 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f77d000fc60 con 0x7f77e40f6860 2021-11-03T17:20:06.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.711+0000 7f77e17fa700 1 -- 172.21.15.27:0/3837311859 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77d0012cd0 con 0x7f77e40f6860 2021-11-03T17:20:06.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.711+0000 7f77eaeed700 1 -- 172.21.15.27:0/3837311859 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77e410c720 con 0x7f77e40f6860 2021-11-03T17:20:06.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.711+0000 7f77eaeed700 1 -- 172.21.15.27:0/3837311859 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77e410cce0 con 0x7f77e40f6860 2021-11-03T17:20:06.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.713+0000 7f77eaeed700 1 -- 172.21.15.27:0/3837311859 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77e4045ad0 con 0x7f77e40f6860 2021-11-03T17:20:06.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.713+0000 7f77e17fa700 1 -- 172.21.15.27:0/3837311859 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f77d0010070 con 0x7f77e40f6860 2021-11-03T17:20:06.715 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.713+0000 7f77e17fa700 1 --2- 172.21.15.27:0/3837311859 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f77d40640b0 0x7f77d4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:20:06.715 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.713+0000 7f77e17fa700 1 -- 172.21.15.27:0/3837311859 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f77d00806e0 con 0x7f77e40f6860 2021-11-03T17:20:06.715 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.713+0000 7f77e8c89700 1 --2- 172.21.15.27:0/3837311859 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f77d40640b0 0x7f77d4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:20:06.715 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.714+0000 7f77e8c89700 1 --2- 172.21.15.27:0/3837311859 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f77d40640b0 0x7f77d4066570 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f77d8004e30 tx=0x7f77d8009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:20:06.716 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.717+0000 7f77e17fa700 1 -- 172.21.15.27:0/3837311859 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f77d0052950 con 0x7f77e40f6860 2021-11-03T17:20:06.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.898+0000 7f77eaeed700 1 -- 172.21.15.27:0/3837311859 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f77e40008d0 con 0x7f77d40640b0 2021-11-03T17:20:06.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.899+0000 7f77e17fa700 1 -- 172.21.15.27:0/3837311859 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f77e40008d0 con 0x7f77d40640b0 2021-11-03T17:20:06.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.901+0000 7f77eaeed700 1 -- 172.21.15.27:0/3837311859 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f77d40640b0 msgr2=0x7f77d4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:20:06.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.901+0000 7f77eaeed700 1 --2- 172.21.15.27:0/3837311859 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f77d40640b0 0x7f77d4066570 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f77d8004e30 tx=0x7f77d8009b30).stop 2021-11-03T17:20:06.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.901+0000 7f77eaeed700 1 -- 172.21.15.27:0/3837311859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77e40f6860 msgr2=0x7f77e410c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:20:06.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.901+0000 7f77eaeed700 1 --2- 172.21.15.27:0/3837311859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77e40f6860 0x7f77e410c170 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f77d000c530 tx=0x7f77d0007e20).stop 2021-11-03T17:20:06.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.901+0000 7f77eaeed700 1 -- 172.21.15.27:0/3837311859 shutdown_connections 2021-11-03T17:20:06.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.901+0000 7f77eaeed700 1 --2- 172.21.15.27:0/3837311859 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f77d40640b0 0x7f77d4066570 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:06.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.901+0000 7f77eaeed700 1 --2- 172.21.15.27:0/3837311859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77e40f55c0 0x7f77e4109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:06.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.901+0000 7f77eaeed700 1 --2- 172.21.15.27:0/3837311859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77e40f6860 0x7f77e410c170 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:06.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.901+0000 7f77eaeed700 1 -- 172.21.15.27:0/3837311859 >> 172.21.15.27:0/3837311859 conn(0x7f77e40f0d40 msgr2=0x7f77e40fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:20:06.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.901+0000 7f77eaeed700 1 -- 172.21.15.27:0/3837311859 shutdown_connections 2021-11-03T17:20:06.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:06.901+0000 7f77eaeed700 1 -- 172.21.15.27:0/3837311859 wait complete. 2021-11-03T17:20:06.905 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:20:06.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.002+0000 7fb77a1a5700 1 -- 172.21.15.27:0/4218991372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb774060310 msgr2=0x7fb7740fb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:20:06.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.002+0000 7fb77a1a5700 1 --2- 172.21.15.27:0/4218991372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb774060310 0x7fb7740fb520 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7fb770004660 tx=0x7fb770009b30).stop 2021-11-03T17:20:07.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.002+0000 7fb77a1a5700 1 -- 172.21.15.27:0/4218991372 shutdown_connections 2021-11-03T17:20:07.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.002+0000 7fb77a1a5700 1 --2- 172.21.15.27:0/4218991372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7740fba10 0x7fb7740fde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:07.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.002+0000 7fb77a1a5700 1 --2- 172.21.15.27:0/4218991372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb774060310 0x7fb7740fb520 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:07.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.002+0000 7fb77a1a5700 1 -- 172.21.15.27:0/4218991372 >> 172.21.15.27:0/4218991372 conn(0x7fb7740f0ba0 msgr2=0x7fb7740f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:20:07.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.002+0000 7fb77a1a5700 1 -- 172.21.15.27:0/4218991372 shutdown_connections 2021-11-03T17:20:07.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.002+0000 7fb77a1a5700 1 -- 172.21.15.27:0/4218991372 wait complete. 2021-11-03T17:20:07.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.003+0000 7fb77a1a5700 1 Processor -- start 2021-11-03T17:20:07.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.003+0000 7fb77a1a5700 1 -- start start 2021-11-03T17:20:07.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.003+0000 7fb77a1a5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb774060310 0x7fb774108f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:20:07.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.003+0000 7fb77a1a5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7740fba10 0x7fb774109460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:20:07.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.003+0000 7fb77a1a5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb774109a10 con 0x7fb774060310 2021-11-03T17:20:07.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.003+0000 7fb77a1a5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb774109b50 con 0x7fb7740fba10 2021-11-03T17:20:07.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.003+0000 7fb7791a3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb774060310 0x7fb774108f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:20:07.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.003+0000 7fb7791a3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb774060310 0x7fb774108f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56842/0 (socket says 172.21.15.27:56842) 2021-11-03T17:20:07.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.003+0000 7fb7791a3700 1 -- 172.21.15.27:0/4199842361 learned_addr learned my addr 172.21.15.27:0/4199842361 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:20:07.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.003+0000 7fb7789a2700 1 --2- 172.21.15.27:0/4199842361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7740fba10 0x7fb774109460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:20:07.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.004+0000 7fb7791a3700 1 -- 172.21.15.27:0/4199842361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7740fba10 msgr2=0x7fb774109460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:20:07.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.004+0000 7fb7791a3700 1 --2- 172.21.15.27:0/4199842361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7740fba10 0x7fb774109460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:07.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.004+0000 7fb7791a3700 1 -- 172.21.15.27:0/4199842361 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb770005040 con 0x7fb774060310 2021-11-03T17:20:07.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.004+0000 7fb7789a2700 1 --2- 172.21.15.27:0/4199842361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7740fba10 0x7fb774109460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:20:07.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.004+0000 7fb7791a3700 1 --2- 172.21.15.27:0/4199842361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb774060310 0x7fb774108f70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fb770009990 tx=0x7fb770009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:20:07.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.004+0000 7fb769ffb700 1 -- 172.21.15.27:0/4199842361 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb770018070 con 0x7fb774060310 2021-11-03T17:20:07.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.004+0000 7fb77a1a5700 1 -- 172.21.15.27:0/4199842361 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb77410a490 con 0x7fb774060310 2021-11-03T17:20:07.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.004+0000 7fb769ffb700 1 -- 172.21.15.27:0/4199842361 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fb770007ba0 con 0x7fb774060310 2021-11-03T17:20:07.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.004+0000 7fb769ffb700 1 -- 172.21.15.27:0/4199842361 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb77001c3c0 con 0x7fb774060310 2021-11-03T17:20:07.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.004+0000 7fb77a1a5700 1 -- 172.21.15.27:0/4199842361 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb77410a9f0 con 0x7fb774060310 2021-11-03T17:20:07.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.005+0000 7fb77a1a5700 1 -- 172.21.15.27:0/4199842361 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb774045ad0 con 0x7fb774060310 2021-11-03T17:20:07.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.006+0000 7fb769ffb700 1 -- 172.21.15.27:0/4199842361 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fb77001d420 con 0x7fb774060310 2021-11-03T17:20:07.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.006+0000 7fb769ffb700 1 --2- 172.21.15.27:0/4199842361 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb760064100 0x7fb7600665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:20:07.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.006+0000 7fb7789a2700 1 --2- 172.21.15.27:0/4199842361 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb760064100 0x7fb7600665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:20:07.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.006+0000 7fb769ffb700 1 -- 172.21.15.27:0/4199842361 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fb7700824f0 con 0x7fb774060310 2021-11-03T17:20:07.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.007+0000 7fb7789a2700 1 --2- 172.21.15.27:0/4199842361 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb760064100 0x7fb7600665c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fb7740600e0 tx=0x7fb764009be0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:20:07.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.010+0000 7fb769ffb700 1 -- 172.21.15.27:0/4199842361 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fb770054910 con 0x7fb774060310 2021-11-03T17:20:07.177 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.183+0000 7fb77a1a5700 1 -- 172.21.15.27:0/4199842361 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb774101070 con 0x7fb760064100 2021-11-03T17:20:07.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.197+0000 7fb769ffb700 1 -- 172.21.15.27:0/4199842361 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fb774101070 con 0x7fb760064100 2021-11-03T17:20:07.193 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:20:07.194 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (11m) 114s ago 12m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:20:07.194 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (3m) 114s ago 12m 7205k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:20:07.194 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (3m) 34s ago 12m 7239k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:20:07.194 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (11m) 114s ago 12m 38.9M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:20:07.195 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (5m) 114s ago 8m 15.4M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:20:07.195 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (8m) 114s ago 8m 3836M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:20:07.195 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (3m) 34s ago 8m 14.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:20:07.196 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (5m) 34s ago 8m 16.3M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:20:07.196 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (5m) 114s ago 14m 470M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:20:07.197 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (4m) 34s ago 12m 412M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:20:07.197 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (4m) 114s ago 14m 81.2M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:20:07.197 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (3m) 34s ago 12m 69.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:20:07.198 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (12m) 114s ago 12m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:20:07.198 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (11m) 34s ago 11m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:20:07.199 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (2m) 114s ago 11m 243M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:20:07.199 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (2m) 114s ago 10m 270M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:20:07.200 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (2m) 114s ago 10m 11.8M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:20:07.200 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (91s) 34s ago 10m 289M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:20:07.200 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (64s) 34s ago 9m 235M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:20:07.201 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (37s) 34s ago 9m 11.5M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:20:07.201 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (11m) 114s ago 12m 45.7M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:20:07.203 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.199+0000 7fb77a1a5700 1 -- 172.21.15.27:0/4199842361 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb760064100 msgr2=0x7fb7600665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:20:07.203 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.199+0000 7fb77a1a5700 1 --2- 172.21.15.27:0/4199842361 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb760064100 0x7fb7600665c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7fb7740600e0 tx=0x7fb764009be0).stop 2021-11-03T17:20:07.204 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.199+0000 7fb77a1a5700 1 -- 172.21.15.27:0/4199842361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb774060310 msgr2=0x7fb774108f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:20:07.204 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.199+0000 7fb77a1a5700 1 --2- 172.21.15.27:0/4199842361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb774060310 0x7fb774108f70 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fb770009990 tx=0x7fb770009b30).stop 2021-11-03T17:20:07.205 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.199+0000 7fb77a1a5700 1 -- 172.21.15.27:0/4199842361 shutdown_connections 2021-11-03T17:20:07.205 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.199+0000 7fb77a1a5700 1 --2- 172.21.15.27:0/4199842361 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb760064100 0x7fb7600665c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:07.206 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.199+0000 7fb77a1a5700 1 --2- 172.21.15.27:0/4199842361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb774060310 0x7fb774108f70 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:07.206 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.199+0000 7fb77a1a5700 1 --2- 172.21.15.27:0/4199842361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7740fba10 0x7fb774109460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:07.207 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.199+0000 7fb77a1a5700 1 -- 172.21.15.27:0/4199842361 >> 172.21.15.27:0/4199842361 conn(0x7fb7740f0ba0 msgr2=0x7fb7740f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:20:07.207 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.200+0000 7fb77a1a5700 1 -- 172.21.15.27:0/4199842361 shutdown_connections 2021-11-03T17:20:07.208 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.200+0000 7fb77a1a5700 1 -- 172.21.15.27:0/4199842361 wait complete. 2021-11-03T17:20:07.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:06 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:20:07.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:06 smithi115 ceph-mon[65453]: pgmap v162: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:07.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:06 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:20:07.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:06 smithi027 ceph-mon[97244]: pgmap v162: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:07.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.299+0000 7fa384982700 1 -- 172.21.15.27:0/825745576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3800f73b0 msgr2=0x7fa3800f4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:20:07.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.299+0000 7fa384982700 1 --2- 172.21.15.27:0/825745576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3800f73b0 0x7fa3800f4640 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fa374004660 tx=0x7fa374009b30).stop 2021-11-03T17:20:07.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.299+0000 7fa384982700 1 -- 172.21.15.27:0/825745576 shutdown_connections 2021-11-03T17:20:07.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.299+0000 7fa384982700 1 --2- 172.21.15.27:0/825745576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3800f73b0 0x7fa3800f4640 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:07.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.299+0000 7fa384982700 1 --2- 172.21.15.27:0/825745576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3800f6a10 0x7fa3800f6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:07.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.299+0000 7fa384982700 1 -- 172.21.15.27:0/825745576 >> 172.21.15.27:0/825745576 conn(0x7fa3800f0320 msgr2=0x7fa3800f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:20:07.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.299+0000 7fa384982700 1 -- 172.21.15.27:0/825745576 shutdown_connections 2021-11-03T17:20:07.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.299+0000 7fa384982700 1 -- 172.21.15.27:0/825745576 wait complete. 2021-11-03T17:20:07.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.300+0000 7fa384982700 1 Processor -- start 2021-11-03T17:20:07.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.300+0000 7fa384982700 1 -- start start 2021-11-03T17:20:07.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.300+0000 7fa384982700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3800f6a10 0x7fa380108f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:20:07.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.300+0000 7fa384982700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3800f73b0 0x7fa380109430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:20:07.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.300+0000 7fa384982700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3801099e0 con 0x7fa3800f6a10 2021-11-03T17:20:07.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.300+0000 7fa384982700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa380109b20 con 0x7fa3800f73b0 2021-11-03T17:20:07.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.300+0000 7fa37f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3800f6a10 0x7fa380108f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:20:07.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.300+0000 7fa37f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3800f6a10 0x7fa380108f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56852/0 (socket says 172.21.15.27:56852) 2021-11-03T17:20:07.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.300+0000 7fa37f7fe700 1 -- 172.21.15.27:0/3110737600 learned_addr learned my addr 172.21.15.27:0/3110737600 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:20:07.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.301+0000 7fa37f7fe700 1 -- 172.21.15.27:0/3110737600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3800f73b0 msgr2=0x7fa380109430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:20:07.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.301+0000 7fa37f7fe700 1 --2- 172.21.15.27:0/3110737600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3800f73b0 0x7fa380109430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:20:07.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.301+0000 7fa37f7fe700 1 -- 172.21.15.27:0/3110737600 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa374005040 con 0x7fa3800f6a10 2021-11-03T17:20:07.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.301+0000 7fa37f7fe700 1 --2- 172.21.15.27:0/3110737600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3800f6a10 0x7fa380108f40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fa3700026d0 tx=0x7fa370009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:20:07.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.301+0000 7fa367fff700 1 -- 172.21.15.27:0/3110737600 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa370009320 con 0x7fa3800f6a10 2021-11-03T17:20:07.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.301+0000 7fa367fff700 1 -- 172.21.15.27:0/3110737600 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fa37000f040 con 0x7fa3800f6a10 2021-11-03T17:20:07.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.301+0000 7fa384982700 1 -- 172.21.15.27:0/3110737600 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa38010a4c0 con 0x7fa3800f6a10 2021-11-03T17:20:07.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.301+0000 7fa367fff700 1 -- 172.21.15.27:0/3110737600 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3700147e0 con 0x7fa3800f6a10 2021-11-03T17:20:07.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.301+0000 7fa384982700 1 -- 172.21.15.27:0/3110737600 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa38010aa80 con 0x7fa3800f6a10 2021-11-03T17:20:07.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.302+0000 7fa384982700 1 -- 172.21.15.27:0/3110737600 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa380045ad0 con 0x7fa3800f6a10 2021-11-03T17:20:07.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.302+0000 7fa367fff700 1 -- 172.21.15.27:0/3110737600 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fa3700077b0 con 0x7fa3800f6a10 2021-11-03T17:20:07.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.303+0000 7fa367fff700 1 --2- 172.21.15.27:0/3110737600 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa368064100 0x7fa3680665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:20:07.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.303+0000 7fa367fff700 1 -- 172.21.15.27:0/3110737600 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fa370080eb0 con 0x7fa3800f6a10 2021-11-03T17:20:07.307 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.303+0000 7fa37effd700 1 --2- 172.21.15.27:0/3110737600 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa368064100 0x7fa3680665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:20:07.307 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.303+0000 7fa37effd700 1 --2- 172.21.15.27:0/3110737600 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa368064100 0x7fa3680665c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fa374004e30 tx=0x7fa374009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:20:07.308 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.307+0000 7fa367fff700 1 -- 172.21.15.27:0/3110737600 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fa370053210 con 0x7fa3800f6a10 2021-11-03T17:20:07.532 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.538+0000 7fa384982700 1 -- 172.21.15.27:0/3110737600 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa3801010b0 con 0x7fa3800f6a10 2021-11-03T17:20:07.533 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.538+0000 7fa367fff700 1 -- 172.21.15.27:0/3110737600 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fa370052e00 con 0x7fa3800f6a10 2021-11-03T17:20:07.534 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:20:07.534 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:20:07.534 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:20:07.534 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:20:07.535 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:20:07.535 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:20:07.535 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:20:07.535 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:20:07.536 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:20:07.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:20:07.536 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:20:07.536 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:20:07.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:20:07.537 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:20:07.537 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:20:07.537 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:20:07.537 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:20:07.538 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:20:07.539 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.540+0000 7fa384982700 1 -- 172.21.15.27:0/3110737600 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa368064100 msgr2=0x7fa3680665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:20:07.539 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.540+0000 7fa384982700 1 --2- 172.21.15.27:0/3110737600 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa368064100 0x7fa3680665c0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7fa374004e30 tx=0x7fa374009b30).stop 2021-11-03T17:20:07.539 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.540+0000 7fa384982700 1 -- 172.21.15.27:0/3110737600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3800f6a10 msgr2=0x7fa380108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:20:07.540 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.540+0000 7fa384982700 1 --2- 172.21.15.27:0/3110737600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3800f6a10 0x7fa380108f40 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fa3700026d0 tx=0x7fa370009de0).stop 2021-11-03T17:20:07.540 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.540+0000 7fa384982700 1 -- 172.21.15.27:0/3110737600 shutdown_connections 2021-11-03T17:20:07.540 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.540+0000 7fa384982700 1 --2- 172.21.15.27:0/3110737600 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa368064100 0x7fa3680665c0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:07.540 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.540+0000 7fa384982700 1 --2- 172.21.15.27:0/3110737600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3800f6a10 0x7fa380108f40 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:07.541 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.540+0000 7fa384982700 1 --2- 172.21.15.27:0/3110737600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3800f73b0 0x7fa380109430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:20:07.541 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.540+0000 7fa384982700 1 -- 172.21.15.27:0/3110737600 >> 172.21.15.27:0/3110737600 conn(0x7fa3800f0320 msgr2=0x7fa3800fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:20:07.541 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.541+0000 7fa384982700 1 -- 172.21.15.27:0/3110737600 shutdown_connections 2021-11-03T17:20:07.541 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.541+0000 7fa384982700 1 -- 172.21.15.27:0/3110737600 wait complete. 2021-11-03T17:20:07.638 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.641+0000 7f843351e700 1 -- 172.21.15.27:0/3441039470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f842c0f5ef0 msgr2=0x7f842c0f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:20:07.638 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.641+0000 7f843351e700 1 --2- 172.21.15.27:0/3441039470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f842c0f5ef0 0x7f842c0f6920 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f8428004660 tx=0x7f8428009b30).stop 2021-11-03T17:20:07.638 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.641+0000 7f843351e700 1 -- 172.21.15.27:0/3441039470 shutdown_connections 2021-11-03T17:20:07.638 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.641+0000 7f843351e700 1 --2- 172.21.15.27:0/3441039470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f842c0f5ef0 0x7f842c0f6920 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:07.639 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.641+0000 7f843351e700 1 --2- 172.21.15.27:0/3441039470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f842c0f55e0 0x7f842c0f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:07.639 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.641+0000 7f843351e700 1 -- 172.21.15.27:0/3441039470 >> 172.21.15.27:0/3441039470 conn(0x7f842c0f0de0 msgr2=0x7f842c0f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:20:07.639 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.641+0000 7f843351e700 1 -- 172.21.15.27:0/3441039470 shutdown_connections 2021-11-03T17:20:07.639 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.641+0000 7f843351e700 1 -- 172.21.15.27:0/3441039470 wait complete. 2021-11-03T17:20:07.640 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.642+0000 7f843351e700 1 Processor -- start 2021-11-03T17:20:07.640 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.642+0000 7f843351e700 1 -- start start 2021-11-03T17:20:07.640 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.642+0000 7f843351e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f842c0f55e0 0x7f842c109bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:20:07.640 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.642+0000 7f843351e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f842c0f5ef0 0x7f842c10c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:20:07.641 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.642+0000 7f843351e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842c0604e0 con 0x7f842c0f5ef0 2021-11-03T17:20:07.641 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.642+0000 7f843351e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842c060620 con 0x7f842c0f55e0 2021-11-03T17:20:07.641 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.642+0000 7f8430ab9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f842c0f5ef0 0x7f842c10c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:20:07.641 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.642+0000 7f8430ab9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f842c0f5ef0 0x7f842c10c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56866/0 (socket says 172.21.15.27:56866) 2021-11-03T17:20:07.642 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.642+0000 7f8430ab9700 1 -- 172.21.15.27:0/1135512667 learned_addr learned my addr 172.21.15.27:0/1135512667 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:20:07.642 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.642+0000 7f84312ba700 1 --2- 172.21.15.27:0/1135512667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f842c0f55e0 0x7f842c109bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:20:07.642 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.642+0000 7f8430ab9700 1 -- 172.21.15.27:0/1135512667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f842c0f55e0 msgr2=0x7f842c109bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:20:07.642 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.642+0000 7f8430ab9700 1 --2- 172.21.15.27:0/1135512667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f842c0f55e0 0x7f842c109bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:07.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.642+0000 7f8430ab9700 1 -- 172.21.15.27:0/1135512667 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8428005040 con 0x7f842c0f5ef0 2021-11-03T17:20:07.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.642+0000 7f84312ba700 1 --2- 172.21.15.27:0/1135512667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f842c0f55e0 0x7f842c109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:20:07.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.642+0000 7f8430ab9700 1 --2- 172.21.15.27:0/1135512667 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f842c0f5ef0 0x7f842c10c0f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f8428009990 tx=0x7f8428009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:20:07.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.643+0000 7f8421ffb700 1 -- 172.21.15.27:0/1135512667 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8428018070 con 0x7f842c0f5ef0 2021-11-03T17:20:07.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.643+0000 7f843351e700 1 -- 172.21.15.27:0/1135512667 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f842c10c640 con 0x7f842c0f5ef0 2021-11-03T17:20:07.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.643+0000 7f8421ffb700 1 -- 172.21.15.27:0/1135512667 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f8428007ba0 con 0x7f842c0f5ef0 2021-11-03T17:20:07.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.643+0000 7f8421ffb700 1 -- 172.21.15.27:0/1135512667 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f842801c3c0 con 0x7f842c0f5ef0 2021-11-03T17:20:07.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.643+0000 7f843351e700 1 -- 172.21.15.27:0/1135512667 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f842c10cba0 con 0x7f842c0f5ef0 2021-11-03T17:20:07.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.644+0000 7f843351e700 1 -- 172.21.15.27:0/1135512667 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f842c045ad0 con 0x7f842c0f5ef0 2021-11-03T17:20:07.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.644+0000 7f8421ffb700 1 -- 172.21.15.27:0/1135512667 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f842801d420 con 0x7f842c0f5ef0 2021-11-03T17:20:07.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.645+0000 7f8421ffb700 1 --2- 172.21.15.27:0/1135512667 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f84180640b0 0x7f8418066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:20:07.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.645+0000 7f8421ffb700 1 -- 172.21.15.27:0/1135512667 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f84280825d0 con 0x7f842c0f5ef0 2021-11-03T17:20:07.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.645+0000 7f84312ba700 1 --2- 172.21.15.27:0/1135512667 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f84180640b0 0x7f8418066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:20:07.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.645+0000 7f84312ba700 1 --2- 172.21.15.27:0/1135512667 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f84180640b0 0x7f8418066570 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f841c009880 tx=0x7f841c00b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:20:07.648 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.649+0000 7f8421ffb700 1 -- 172.21.15.27:0/1135512667 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f8428054990 con 0x7f842c0f5ef0 2021-11-03T17:20:07.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.855+0000 7f843351e700 1 -- 172.21.15.27:0/1135512667 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f842c027a00 con 0x7f842c0f5ef0 2021-11-03T17:20:07.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.856+0000 7f8421ffb700 1 -- 172.21.15.27:0/1135512667 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f8428054580 con 0x7f842c0f5ef0 2021-11-03T17:20:07.851 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:20:07.851 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:20:07.851 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:20:07.851 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:20:07.852 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:20:07.852 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:20:07.852 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:20:07.852 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:20:07.853 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:20:07.853 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:20:07.853 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:20:07.854 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:20:07.854 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:20:07.854 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:20:07.854 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:20:07.855 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:20:07.855 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:20:07.855 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:20:07.855 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:20:07.855 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:20:07.856 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:20:07.856 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:20:07.856 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:20:07.856 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:20:07.857 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:20:07.857 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:20:07.857 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:20:07.857 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:20:07.857 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:20:07.858 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:20:07.858 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:20:07.858 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:20:07.858 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:20:07.859 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:20:07.859 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:20:07.859 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:20:07.859 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:20:07.859 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:20:07.860 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:20:07.861 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.858+0000 7f843351e700 1 -- 172.21.15.27:0/1135512667 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f84180640b0 msgr2=0x7f8418066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:20:07.861 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.858+0000 7f843351e700 1 --2- 172.21.15.27:0/1135512667 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f84180640b0 0x7f8418066570 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f841c009880 tx=0x7f841c00b040).stop 2021-11-03T17:20:07.861 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.858+0000 7f843351e700 1 -- 172.21.15.27:0/1135512667 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f842c0f5ef0 msgr2=0x7f842c10c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:20:07.861 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.858+0000 7f843351e700 1 --2- 172.21.15.27:0/1135512667 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f842c0f5ef0 0x7f842c10c0f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f8428009990 tx=0x7f8428009b30).stop 2021-11-03T17:20:07.862 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.858+0000 7f843351e700 1 -- 172.21.15.27:0/1135512667 shutdown_connections 2021-11-03T17:20:07.862 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.858+0000 7f843351e700 1 --2- 172.21.15.27:0/1135512667 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f84180640b0 0x7f8418066570 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:07.862 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.858+0000 7f843351e700 1 --2- 172.21.15.27:0/1135512667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f842c0f55e0 0x7f842c109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:07.862 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.858+0000 7f843351e700 1 --2- 172.21.15.27:0/1135512667 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f842c0f5ef0 0x7f842c10c0f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:07.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.858+0000 7f843351e700 1 -- 172.21.15.27:0/1135512667 >> 172.21.15.27:0/1135512667 conn(0x7f842c0f0de0 msgr2=0x7f842c0fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:20:07.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.858+0000 7f843351e700 1 -- 172.21.15.27:0/1135512667 shutdown_connections 2021-11-03T17:20:07.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:07.858+0000 7f843351e700 1 -- 172.21.15.27:0/1135512667 wait complete. 2021-11-03T17:20:07.863 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:20:08.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:07 smithi115 ceph-mon[65453]: from='client.34280 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:20:08.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:07 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3110737600' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:08.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:07 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1135512667' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:20:08.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:07 smithi027 ceph-mon[97244]: from='client.34280 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:20:08.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:07 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3110737600' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:08.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:07 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1135512667' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:20:09.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:08 smithi115 ceph-mon[65453]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:20:09.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:08 smithi115 ceph-mon[65453]: pgmap v163: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:09.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:08 smithi027 ceph-mon[97244]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:20:09.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:08 smithi027 ceph-mon[97244]: pgmap v163: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:10.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:09 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 252.364880 secs 2021-11-03T17:20:10.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:09 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 252.364880 secs 2021-11-03T17:20:11.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:10 smithi115 ceph-mon[65453]: pgmap v164: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:11.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:10 smithi027 ceph-mon[97244]: pgmap v164: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:13.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:12 smithi115 ceph-mon[65453]: pgmap v165: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:13.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:12 smithi027 ceph-mon[97244]: pgmap v165: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:15.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:14 smithi115 ceph-mon[65453]: pgmap v166: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:15.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:14 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 257.364973 secs 2021-11-03T17:20:15.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:14 smithi027 ceph-mon[97244]: pgmap v166: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:15.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:14 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 257.364973 secs 2021-11-03T17:20:16.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:20:16.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:20:16.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:20:16.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:16.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:16.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:16.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:16.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:20:16.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:20:16.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:20:16.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:16.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:16.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:16.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:17.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:16 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:20:17.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:16 smithi115 ceph-mon[65453]: pgmap v167: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:17.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:16 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:20:17.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:16 smithi027 ceph-mon[97244]: pgmap v167: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:19.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:18 smithi115 ceph-mon[65453]: pgmap v168: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:19.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:18 smithi027 ceph-mon[97244]: pgmap v168: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:20.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:19 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 262.365119 secs 2021-11-03T17:20:20.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:19 smithi115 ceph-mon[65453]: pgmap v169: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:20.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:19 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 262.365119 secs 2021-11-03T17:20:20.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:19 smithi027 ceph-mon[97244]: pgmap v169: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:23.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:22 smithi115 ceph-mon[65453]: pgmap v170: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:23.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:22 smithi027 ceph-mon[97244]: pgmap v170: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:25.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:24 smithi115 ceph-mon[65453]: pgmap v171: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:25.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:24 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 267.365244 secs 2021-11-03T17:20:25.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:24 smithi027 ceph-mon[97244]: pgmap v171: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:25.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:24 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 267.365244 secs 2021-11-03T17:20:26.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:20:26.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:20:26.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:20:26.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:26.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:26.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:26.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:26.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:20:26.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:20:26.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:20:26.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:26.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:26.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:26.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:27.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:26 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:20:27.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:26 smithi115 ceph-mon[65453]: pgmap v172: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:27.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:26 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:20:27.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:26 smithi027 ceph-mon[97244]: pgmap v172: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:29.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:28 smithi115 ceph-mon[65453]: pgmap v173: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:29.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:28 smithi027 ceph-mon[97244]: pgmap v173: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:30.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:29 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 272.365368 secs 2021-11-03T17:20:30.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:29 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 272.365368 secs 2021-11-03T17:20:31.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:30 smithi115 ceph-mon[65453]: pgmap v174: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:31.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:30 smithi027 ceph-mon[97244]: pgmap v174: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:33.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:32 smithi115 ceph-mon[65453]: pgmap v175: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:33.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:32 smithi027 ceph-mon[97244]: pgmap v175: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:35.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:34 smithi115 ceph-mon[65453]: pgmap v176: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:35.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:34 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 277.365493 secs 2021-11-03T17:20:35.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:34 smithi027 ceph-mon[97244]: pgmap v176: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:35.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:34 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 277.365493 secs 2021-11-03T17:20:36.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:20:36.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:20:36.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:20:36.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:36.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:36.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:36.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:36.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:20:36.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:20:36.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:20:36.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:36.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:36.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:36.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:37.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:36 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:20:37.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:36 smithi115 ceph-mon[65453]: pgmap v177: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:37.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:36 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:20:37.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:36 smithi027 ceph-mon[97244]: pgmap v177: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:37.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:37.975+0000 7f5b7d25b700 1 -- 172.21.15.27:0/733715338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b780f66c0 msgr2=0x7f5b780f6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:20:37.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:37.975+0000 7f5b7d25b700 1 --2- 172.21.15.27:0/733715338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b780f66c0 0x7f5b780f6ae0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f5b70004660 tx=0x7f5b70009b30).stop 2021-11-03T17:20:37.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:37.975+0000 7f5b7d25b700 1 -- 172.21.15.27:0/733715338 shutdown_connections 2021-11-03T17:20:37.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:37.975+0000 7f5b7d25b700 1 --2- 172.21.15.27:0/733715338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b780f7960 0x7f5b780f7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:37.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:37.975+0000 7f5b7d25b700 1 --2- 172.21.15.27:0/733715338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b780f66c0 0x7f5b780f6ae0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:37.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:37.975+0000 7f5b7d25b700 1 -- 172.21.15.27:0/733715338 >> 172.21.15.27:0/733715338 conn(0x7f5b780f1e20 msgr2=0x7f5b780f4240 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:20:37.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:37.975+0000 7f5b7d25b700 1 -- 172.21.15.27:0/733715338 shutdown_connections 2021-11-03T17:20:37.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:37.975+0000 7f5b7d25b700 1 -- 172.21.15.27:0/733715338 wait complete. 2021-11-03T17:20:37.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:37.975+0000 7f5b7d25b700 1 Processor -- start 2021-11-03T17:20:37.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:37.976+0000 7f5b7d25b700 1 -- start start 2021-11-03T17:20:37.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:37.976+0000 7f5b7d25b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b780f66c0 0x7f5b7806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:20:37.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:37.976+0000 7f5b7d25b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b780f7960 0x7f5b78069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:20:37.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:37.976+0000 7f5b7d25b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7806a380 con 0x7f5b780f66c0 2021-11-03T17:20:37.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:37.976+0000 7f5b7d25b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b7806a4c0 con 0x7f5b780f7960 2021-11-03T17:20:37.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:37.976+0000 7f5b76d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b780f66c0 0x7f5b7806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:20:37.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:37.976+0000 7f5b76d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b780f66c0 0x7f5b7806b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56882/0 (socket says 172.21.15.27:56882) 2021-11-03T17:20:37.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:37.976+0000 7f5b76d9d700 1 -- 172.21.15.27:0/3941267672 learned_addr learned my addr 172.21.15.27:0/3941267672 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:20:37.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:37.976+0000 7f5b76d9d700 1 -- 172.21.15.27:0/3941267672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b780f7960 msgr2=0x7f5b78069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:20:37.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:37.976+0000 7f5b76d9d700 1 --2- 172.21.15.27:0/3941267672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b780f7960 0x7f5b78069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:20:37.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:37.976+0000 7f5b76d9d700 1 -- 172.21.15.27:0/3941267672 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b70005040 con 0x7f5b780f66c0 2021-11-03T17:20:37.977 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:37.976+0000 7f5b76d9d700 1 --2- 172.21.15.27:0/3941267672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b780f66c0 0x7f5b7806b9a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f5b70004f60 tx=0x7f5b70009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:20:37.977 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:37.976+0000 7f5b6f7fe700 1 -- 172.21.15.27:0/3941267672 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b70018070 con 0x7f5b780f66c0 2021-11-03T17:20:37.977 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:37.976+0000 7f5b6f7fe700 1 -- 172.21.15.27:0/3941267672 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f5b70007ba0 con 0x7f5b780f66c0 2021-11-03T17:20:37.977 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:37.977+0000 7f5b7d25b700 1 -- 172.21.15.27:0/3941267672 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b7806a740 con 0x7f5b780f66c0 2021-11-03T17:20:37.978 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:37.977+0000 7f5b6f7fe700 1 -- 172.21.15.27:0/3941267672 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b7001c620 con 0x7f5b780f66c0 2021-11-03T17:20:37.978 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:37.977+0000 7f5b7d25b700 1 -- 172.21.15.27:0/3941267672 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b7806aca0 con 0x7f5b780f66c0 2021-11-03T17:20:37.978 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:37.978+0000 7f5b7d25b700 1 -- 172.21.15.27:0/3941267672 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b78045ad0 con 0x7f5b780f66c0 2021-11-03T17:20:37.978 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:37.978+0000 7f5b6f7fe700 1 -- 172.21.15.27:0/3941267672 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f5b7001d420 con 0x7f5b780f66c0 2021-11-03T17:20:37.978 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:37.978+0000 7f5b6f7fe700 1 --2- 172.21.15.27:0/3941267672 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5b640640b0 0x7f5b64066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:20:37.979 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:37.978+0000 7f5b6f7fe700 1 -- 172.21.15.27:0/3941267672 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f5b70082620 con 0x7f5b780f66c0 2021-11-03T17:20:37.979 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:37.978+0000 7f5b7659c700 1 --2- 172.21.15.27:0/3941267672 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5b640640b0 0x7f5b64066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:20:37.979 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:37.979+0000 7f5b7659c700 1 --2- 172.21.15.27:0/3941267672 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5b640640b0 0x7f5b64066570 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f5b60004880 tx=0x7f5b60007ae0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:20:37.980 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:37.983+0000 7f5b6f7fe700 1 -- 172.21.15.27:0/3941267672 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f5b70054a40 con 0x7f5b780f66c0 2021-11-03T17:20:38.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.163+0000 7f5b7d25b700 1 -- 172.21.15.27:0/3941267672 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b78057f10 con 0x7f5b640640b0 2021-11-03T17:20:38.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.164+0000 7f5b6f7fe700 1 -- 172.21.15.27:0/3941267672 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f5b78057f10 con 0x7f5b640640b0 2021-11-03T17:20:38.160 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.166+0000 7f5b7d25b700 1 -- 172.21.15.27:0/3941267672 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5b640640b0 msgr2=0x7f5b64066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:20:38.161 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.166+0000 7f5b7d25b700 1 --2- 172.21.15.27:0/3941267672 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5b640640b0 0x7f5b64066570 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f5b60004880 tx=0x7f5b60007ae0).stop 2021-11-03T17:20:38.161 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.166+0000 7f5b7d25b700 1 -- 172.21.15.27:0/3941267672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b780f66c0 msgr2=0x7f5b7806b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:20:38.161 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.166+0000 7f5b7d25b700 1 --2- 172.21.15.27:0/3941267672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b780f66c0 0x7f5b7806b9a0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f5b70004f60 tx=0x7f5b70009b30).stop 2021-11-03T17:20:38.161 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.166+0000 7f5b7d25b700 1 -- 172.21.15.27:0/3941267672 shutdown_connections 2021-11-03T17:20:38.162 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.166+0000 7f5b7d25b700 1 --2- 172.21.15.27:0/3941267672 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5b640640b0 0x7f5b64066570 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:38.162 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.166+0000 7f5b7d25b700 1 --2- 172.21.15.27:0/3941267672 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b780f66c0 0x7f5b7806b9a0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:38.162 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.166+0000 7f5b7d25b700 1 --2- 172.21.15.27:0/3941267672 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b780f7960 0x7f5b78069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:20:38.162 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.166+0000 7f5b7d25b700 1 -- 172.21.15.27:0/3941267672 >> 172.21.15.27:0/3941267672 conn(0x7f5b780f1e20 msgr2=0x7f5b780fb3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:20:38.163 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.166+0000 7f5b7d25b700 1 -- 172.21.15.27:0/3941267672 shutdown_connections 2021-11-03T17:20:38.163 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.166+0000 7f5b7d25b700 1 -- 172.21.15.27:0/3941267672 wait complete. 2021-11-03T17:20:38.171 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:20:38.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.267+0000 7fbbdd322700 1 -- 172.21.15.27:0/1532171804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbd80602b0 msgr2=0x7fbbd80f9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:20:38.265 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.267+0000 7fbbdd322700 1 --2- 172.21.15.27:0/1532171804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbd80602b0 0x7fbbd80f9180 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fbbc8004660 tx=0x7fbbc8009b30).stop 2021-11-03T17:20:38.265 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.267+0000 7fbbdd322700 1 -- 172.21.15.27:0/1532171804 shutdown_connections 2021-11-03T17:20:38.265 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.267+0000 7fbbdd322700 1 --2- 172.21.15.27:0/1532171804 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbd80f9700 0x7fbbd80fbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:38.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.267+0000 7fbbdd322700 1 --2- 172.21.15.27:0/1532171804 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbd80602b0 0x7fbbd80f9180 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:38.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.267+0000 7fbbdd322700 1 -- 172.21.15.27:0/1532171804 >> 172.21.15.27:0/1532171804 conn(0x7fbbd80f0ba0 msgr2=0x7fbbd80f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:20:38.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.268+0000 7fbbdd322700 1 -- 172.21.15.27:0/1532171804 shutdown_connections 2021-11-03T17:20:38.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.268+0000 7fbbdd322700 1 -- 172.21.15.27:0/1532171804 wait complete. 2021-11-03T17:20:38.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.268+0000 7fbbdd322700 1 Processor -- start 2021-11-03T17:20:38.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.268+0000 7fbbdd322700 1 -- start start 2021-11-03T17:20:38.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.268+0000 7fbbdd322700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbd80602b0 0x7fbbd8106db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:20:38.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.268+0000 7fbbdd322700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbd80f9700 0x7fbbd81072a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:20:38.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.268+0000 7fbbdd322700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd81078a0 con 0x7fbbd80f9700 2021-11-03T17:20:38.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.268+0000 7fbbdd322700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbd81080e0 con 0x7fbbd80602b0 2021-11-03T17:20:38.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.269+0000 7fbbd77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbd80f9700 0x7fbbd81072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:20:38.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.269+0000 7fbbd77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbd80f9700 0x7fbbd81072a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56894/0 (socket says 172.21.15.27:56894) 2021-11-03T17:20:38.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.269+0000 7fbbd77fe700 1 -- 172.21.15.27:0/88241806 learned_addr learned my addr 172.21.15.27:0/88241806 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:20:38.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.269+0000 7fbbd7fff700 1 --2- 172.21.15.27:0/88241806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbd80602b0 0x7fbbd8106db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:20:38.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.269+0000 7fbbd77fe700 1 -- 172.21.15.27:0/88241806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbd80602b0 msgr2=0x7fbbd8106db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:20:38.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.269+0000 7fbbd77fe700 1 --2- 172.21.15.27:0/88241806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbd80602b0 0x7fbbd8106db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:38.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.269+0000 7fbbd77fe700 1 -- 172.21.15.27:0/88241806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc8005040 con 0x7fbbd80f9700 2021-11-03T17:20:38.270 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.269+0000 7fbbd7fff700 1 --2- 172.21.15.27:0/88241806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbd80602b0 0x7fbbd8106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:20:38.270 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.269+0000 7fbbd77fe700 1 --2- 172.21.15.27:0/88241806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbd80f9700 0x7fbbd81072a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fbbcc002700 tx=0x7fbbcc009d60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:20:38.270 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.269+0000 7fbbd4ff9700 1 -- 172.21.15.27:0/88241806 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbcc009320 con 0x7fbbd80f9700 2021-11-03T17:20:38.270 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.269+0000 7fbbd4ff9700 1 -- 172.21.15.27:0/88241806 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fbbcc00f040 con 0x7fbbd80f9700 2021-11-03T17:20:38.271 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.269+0000 7fbbdd322700 1 -- 172.21.15.27:0/88241806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbd81083c0 con 0x7fbbd80f9700 2021-11-03T17:20:38.271 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.269+0000 7fbbd4ff9700 1 -- 172.21.15.27:0/88241806 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbbcc014810 con 0x7fbbd80f9700 2021-11-03T17:20:38.271 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.269+0000 7fbbdd322700 1 -- 172.21.15.27:0/88241806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbd8108900 con 0x7fbbd80f9700 2021-11-03T17:20:38.271 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.270+0000 7fbbdd322700 1 -- 172.21.15.27:0/88241806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbd8059100 con 0x7fbbd80f9700 2021-11-03T17:20:38.272 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.271+0000 7fbbd4ff9700 1 -- 172.21.15.27:0/88241806 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fbbcc0077b0 con 0x7fbbd80f9700 2021-11-03T17:20:38.273 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.271+0000 7fbbd4ff9700 1 --2- 172.21.15.27:0/88241806 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbbc0064100 0x7fbbc00665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:20:38.273 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.271+0000 7fbbd4ff9700 1 -- 172.21.15.27:0/88241806 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fbbcc080d90 con 0x7fbbd80f9700 2021-11-03T17:20:38.273 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.271+0000 7fbbd7fff700 1 --2- 172.21.15.27:0/88241806 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbbc0064100 0x7fbbc00665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:20:38.273 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.272+0000 7fbbd7fff700 1 --2- 172.21.15.27:0/88241806 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbbc0064100 0x7fbbc00665c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fbbc8004e30 tx=0x7fbbc8009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:20:38.274 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.275+0000 7fbbd4ff9700 1 -- 172.21.15.27:0/88241806 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fbbcc0530f0 con 0x7fbbd80f9700 2021-11-03T17:20:38.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.448+0000 7fbbdd322700 1 -- 172.21.15.27:0/88241806 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbbd80fee90 con 0x7fbbc0064100 2021-11-03T17:20:38.457 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.463+0000 7fbbd4ff9700 1 -- 172.21.15.27:0/88241806 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fbbd80fee90 con 0x7fbbc0064100 2021-11-03T17:20:38.457 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:20:38.458 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (12m) 2m ago 13m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:20:38.458 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (3m) 2m ago 13m 7205k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:20:38.458 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (3m) 65s ago 12m 7239k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:20:38.458 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (12m) 2m ago 13m 38.9M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:20:38.459 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (5m) 2m ago 8m 15.4M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:20:38.459 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (8m) 2m ago 8m 3836M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:20:38.459 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (4m) 65s ago 8m 14.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:20:38.459 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (6m) 65s ago 8m 16.3M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:20:38.460 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (5m) 2m ago 14m 470M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:20:38.460 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (5m) 65s ago 12m 412M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:20:38.460 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (4m) 2m ago 14m 81.2M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:20:38.460 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (4m) 65s ago 12m 69.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:20:38.461 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (13m) 2m ago 13m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:20:38.461 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (12m) 65s ago 12m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:20:38.461 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (3m) 2m ago 11m 243M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:20:38.461 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (3m) 2m ago 11m 270M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:20:38.462 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (2m) 2m ago 11m 11.8M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:20:38.462 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (2m) 65s ago 10m 289M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:20:38.462 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (95s) 65s ago 10m 235M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:20:38.462 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (69s) 65s ago 10m 11.5M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:20:38.463 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (11m) 2m ago 12m 45.7M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:20:38.463 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.464+0000 7fbbdd322700 1 -- 172.21.15.27:0/88241806 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbbc0064100 msgr2=0x7fbbc00665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:20:38.464 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.464+0000 7fbbdd322700 1 --2- 172.21.15.27:0/88241806 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbbc0064100 0x7fbbc00665c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7fbbc8004e30 tx=0x7fbbc8009b30).stop 2021-11-03T17:20:38.464 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.464+0000 7fbbdd322700 1 -- 172.21.15.27:0/88241806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbd80f9700 msgr2=0x7fbbd81072a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:20:38.464 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.464+0000 7fbbdd322700 1 --2- 172.21.15.27:0/88241806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbd80f9700 0x7fbbd81072a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fbbcc002700 tx=0x7fbbcc009d60).stop 2021-11-03T17:20:38.464 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.465+0000 7fbbdd322700 1 -- 172.21.15.27:0/88241806 shutdown_connections 2021-11-03T17:20:38.465 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.465+0000 7fbbdd322700 1 --2- 172.21.15.27:0/88241806 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbbc0064100 0x7fbbc00665c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:38.465 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.465+0000 7fbbdd322700 1 --2- 172.21.15.27:0/88241806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbbd80602b0 0x7fbbd8106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:38.465 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.465+0000 7fbbdd322700 1 --2- 172.21.15.27:0/88241806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbbd80f9700 0x7fbbd81072a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:38.465 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.465+0000 7fbbdd322700 1 -- 172.21.15.27:0/88241806 >> 172.21.15.27:0/88241806 conn(0x7fbbd80f0ba0 msgr2=0x7fbbd80f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:20:38.466 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.465+0000 7fbbdd322700 1 -- 172.21.15.27:0/88241806 shutdown_connections 2021-11-03T17:20:38.466 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.465+0000 7fbbdd322700 1 -- 172.21.15.27:0/88241806 wait complete. 2021-11-03T17:20:38.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.565+0000 7fa8223e2700 1 -- 172.21.15.27:0/2662545476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa81c0f55e0 msgr2=0x7fa81c0f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:20:38.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.565+0000 7fa8223e2700 1 --2- 172.21.15.27:0/2662545476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa81c0f55e0 0x7fa81c0f5a00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fa80c004660 tx=0x7fa80c009b30).stop 2021-11-03T17:20:38.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.565+0000 7fa8223e2700 1 -- 172.21.15.27:0/2662545476 shutdown_connections 2021-11-03T17:20:38.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.565+0000 7fa8223e2700 1 --2- 172.21.15.27:0/2662545476 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa81c0f5ef0 0x7fa81c0f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:38.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.565+0000 7fa8223e2700 1 --2- 172.21.15.27:0/2662545476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa81c0f55e0 0x7fa81c0f5a00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:38.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.565+0000 7fa8223e2700 1 -- 172.21.15.27:0/2662545476 >> 172.21.15.27:0/2662545476 conn(0x7fa81c0f0de0 msgr2=0x7fa81c0f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:20:38.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.565+0000 7fa8223e2700 1 -- 172.21.15.27:0/2662545476 shutdown_connections 2021-11-03T17:20:38.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.565+0000 7fa8223e2700 1 -- 172.21.15.27:0/2662545476 wait complete. 2021-11-03T17:20:38.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.566+0000 7fa8223e2700 1 Processor -- start 2021-11-03T17:20:38.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.566+0000 7fa8223e2700 1 -- start start 2021-11-03T17:20:38.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.566+0000 7fa8223e2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa81c0f55e0 0x7fa81c109cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:20:38.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.566+0000 7fa8223e2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa81c0f5ef0 0x7fa81c10c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:20:38.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.566+0000 7fa8223e2700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81c0604e0 con 0x7fa81c0f55e0 2021-11-03T17:20:38.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.566+0000 7fa8223e2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa81c060620 con 0x7fa81c0f5ef0 2021-11-03T17:20:38.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.566+0000 7fa81bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa81c0f55e0 0x7fa81c109cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:20:38.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.566+0000 7fa81bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa81c0f55e0 0x7fa81c109cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56902/0 (socket says 172.21.15.27:56902) 2021-11-03T17:20:38.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.566+0000 7fa81bfff700 1 -- 172.21.15.27:0/3580402069 learned_addr learned my addr 172.21.15.27:0/3580402069 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:20:38.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.566+0000 7fa81bfff700 1 -- 172.21.15.27:0/3580402069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa81c0f5ef0 msgr2=0x7fa81c10c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:20:38.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.566+0000 7fa81bfff700 1 --2- 172.21.15.27:0/3580402069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa81c0f5ef0 0x7fa81c10c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:20:38.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.566+0000 7fa81bfff700 1 -- 172.21.15.27:0/3580402069 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa80c005040 con 0x7fa81c0f55e0 2021-11-03T17:20:38.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.567+0000 7fa81bfff700 1 --2- 172.21.15.27:0/3580402069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa81c0f55e0 0x7fa81c109cb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fa80c009990 tx=0x7fa80c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:20:38.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.567+0000 7fa818ff9700 1 -- 172.21.15.27:0/3580402069 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa80c018070 con 0x7fa81c0f55e0 2021-11-03T17:20:38.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.567+0000 7fa818ff9700 1 -- 172.21.15.27:0/3580402069 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fa80c007ca0 con 0x7fa81c0f55e0 2021-11-03T17:20:38.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.567+0000 7fa8223e2700 1 -- 172.21.15.27:0/3580402069 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa81c10c700 con 0x7fa81c0f55e0 2021-11-03T17:20:38.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.567+0000 7fa818ff9700 1 -- 172.21.15.27:0/3580402069 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa80c01c620 con 0x7fa81c0f55e0 2021-11-03T17:20:38.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.567+0000 7fa8223e2700 1 -- 172.21.15.27:0/3580402069 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa81c10cc30 con 0x7fa81c0f55e0 2021-11-03T17:20:38.569 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.568+0000 7fa8223e2700 1 -- 172.21.15.27:0/3580402069 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa81c0f4a70 con 0x7fa81c0f55e0 2021-11-03T17:20:38.569 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.568+0000 7fa818ff9700 1 -- 172.21.15.27:0/3580402069 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fa80c01d960 con 0x7fa81c0f55e0 2021-11-03T17:20:38.569 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.568+0000 7fa818ff9700 1 --2- 172.21.15.27:0/3580402069 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa8040640b0 0x7fa804066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:20:38.570 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.569+0000 7fa818ff9700 1 -- 172.21.15.27:0/3580402069 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fa80c082480 con 0x7fa81c0f55e0 2021-11-03T17:20:38.570 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.569+0000 7fa81b7fe700 1 --2- 172.21.15.27:0/3580402069 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa8040640b0 0x7fa804066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:20:38.570 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.569+0000 7fa81b7fe700 1 --2- 172.21.15.27:0/3580402069 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa8040640b0 0x7fa804066570 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fa8100047d0 tx=0x7fa810007a40).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:20:38.570 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.572+0000 7fa818ff9700 1 -- 172.21.15.27:0/3580402069 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fa80c0548a0 con 0x7fa81c0f55e0 2021-11-03T17:20:38.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.803+0000 7fa8223e2700 1 -- 172.21.15.27:0/3580402069 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa81c045ad0 con 0x7fa81c0f55e0 2021-11-03T17:20:38.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.803+0000 7fa818ff9700 1 -- 172.21.15.27:0/3580402069 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fa80c054490 con 0x7fa81c0f55e0 2021-11-03T17:20:38.798 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:20:38.799 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:20:38.799 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:20:38.799 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:20:38.799 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:20:38.800 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:20:38.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:20:38.800 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:20:38.800 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:20:38.800 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:20:38.801 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:20:38.801 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:20:38.801 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:20:38.802 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:20:38.802 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:20:38.802 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:20:38.802 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:20:38.803 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:20:38.804 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.805+0000 7fa8223e2700 1 -- 172.21.15.27:0/3580402069 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa8040640b0 msgr2=0x7fa804066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:20:38.804 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.805+0000 7fa8223e2700 1 --2- 172.21.15.27:0/3580402069 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa8040640b0 0x7fa804066570 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7fa8100047d0 tx=0x7fa810007a40).stop 2021-11-03T17:20:38.804 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.805+0000 7fa8223e2700 1 -- 172.21.15.27:0/3580402069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa81c0f55e0 msgr2=0x7fa81c109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:20:38.805 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.805+0000 7fa8223e2700 1 --2- 172.21.15.27:0/3580402069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa81c0f55e0 0x7fa81c109cb0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fa80c009990 tx=0x7fa80c009b30).stop 2021-11-03T17:20:38.805 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.805+0000 7fa8223e2700 1 -- 172.21.15.27:0/3580402069 shutdown_connections 2021-11-03T17:20:38.805 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.805+0000 7fa8223e2700 1 --2- 172.21.15.27:0/3580402069 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa8040640b0 0x7fa804066570 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:38.806 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.805+0000 7fa8223e2700 1 --2- 172.21.15.27:0/3580402069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa81c0f55e0 0x7fa81c109cb0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:38.806 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.805+0000 7fa8223e2700 1 --2- 172.21.15.27:0/3580402069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa81c0f5ef0 0x7fa81c10c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:20:38.806 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.805+0000 7fa8223e2700 1 -- 172.21.15.27:0/3580402069 >> 172.21.15.27:0/3580402069 conn(0x7fa81c0f0de0 msgr2=0x7fa81c0fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:20:38.807 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.805+0000 7fa8223e2700 1 -- 172.21.15.27:0/3580402069 shutdown_connections 2021-11-03T17:20:38.807 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.805+0000 7fa8223e2700 1 -- 172.21.15.27:0/3580402069 wait complete. 2021-11-03T17:20:38.905 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.908+0000 7f05a2629700 1 -- 172.21.15.27:0/3040448420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f059c0f77d0 msgr2=0x7f059c0f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:20:38.905 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.908+0000 7f05a2629700 1 --2- 172.21.15.27:0/3040448420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f059c0f77d0 0x7f059c0f7c30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f058c004660 tx=0x7f058c009b30).stop 2021-11-03T17:20:38.905 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.908+0000 7f05a2629700 1 -- 172.21.15.27:0/3040448420 shutdown_connections 2021-11-03T17:20:38.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.908+0000 7f05a2629700 1 --2- 172.21.15.27:0/3040448420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f059c0f77d0 0x7f059c0f7c30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:38.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.908+0000 7f05a2629700 1 --2- 172.21.15.27:0/3040448420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f059c0f6530 0x7f059c0f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:20:38.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.908+0000 7f05a2629700 1 -- 172.21.15.27:0/3040448420 >> 172.21.15.27:0/3040448420 conn(0x7f059c0f1cd0 msgr2=0x7f059c0f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:20:38.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.908+0000 7f05a2629700 1 -- 172.21.15.27:0/3040448420 shutdown_connections 2021-11-03T17:20:38.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.908+0000 7f05a2629700 1 -- 172.21.15.27:0/3040448420 wait complete. 2021-11-03T17:20:38.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.909+0000 7f05a2629700 1 Processor -- start 2021-11-03T17:20:38.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.909+0000 7f05a2629700 1 -- start start 2021-11-03T17:20:38.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.909+0000 7f05a2629700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f059c0f6530 0x7f059c106d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:20:38.908 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.909+0000 7f05a2629700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f059c0f77d0 0x7f059c107270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:20:38.908 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.909+0000 7f05a2629700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059c107900 con 0x7f059c0f77d0 2021-11-03T17:20:38.908 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.909+0000 7f05a2629700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059c1080b0 con 0x7f059c0f6530 2021-11-03T17:20:38.908 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.909+0000 7f05a0e26700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f059c0f77d0 0x7f059c107270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:20:38.908 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.909+0000 7f05a0e26700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f059c0f77d0 0x7f059c107270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56914/0 (socket says 172.21.15.27:56914) 2021-11-03T17:20:38.909 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.909+0000 7f05a0e26700 1 -- 172.21.15.27:0/1502854510 learned_addr learned my addr 172.21.15.27:0/1502854510 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:20:38.909 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.909+0000 7f05a1627700 1 --2- 172.21.15.27:0/1502854510 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f059c0f6530 0x7f059c106d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:20:38.909 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.910+0000 7f05a0e26700 1 -- 172.21.15.27:0/1502854510 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f059c0f6530 msgr2=0x7f059c106d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:20:38.909 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.910+0000 7f05a0e26700 1 --2- 172.21.15.27:0/1502854510 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f059c0f6530 0x7f059c106d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:38.910 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.910+0000 7f05a0e26700 1 -- 172.21.15.27:0/1502854510 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f058c005040 con 0x7f059c0f77d0 2021-11-03T17:20:38.910 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.910+0000 7f05a0e26700 1 --2- 172.21.15.27:0/1502854510 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f059c0f77d0 0x7f059c107270 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f058c009990 tx=0x7f058c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:20:38.910 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.910+0000 7f0591ffb700 1 -- 172.21.15.27:0/1502854510 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f058c018070 con 0x7f059c0f77d0 2021-11-03T17:20:38.910 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.910+0000 7f0591ffb700 1 -- 172.21.15.27:0/1502854510 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f058c007ca0 con 0x7f059c0f77d0 2021-11-03T17:20:38.911 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.910+0000 7f05a2629700 1 -- 172.21.15.27:0/1502854510 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f059c108330 con 0x7f059c0f77d0 2021-11-03T17:20:38.911 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.910+0000 7f0591ffb700 1 -- 172.21.15.27:0/1502854510 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f058c01c620 con 0x7f059c0f77d0 2021-11-03T17:20:38.911 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.910+0000 7f05a2629700 1 -- 172.21.15.27:0/1502854510 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f059c108890 con 0x7f059c0f77d0 2021-11-03T17:20:38.912 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.911+0000 7f05a2629700 1 -- 172.21.15.27:0/1502854510 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f059c059100 con 0x7f059c0f77d0 2021-11-03T17:20:38.912 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.911+0000 7f0591ffb700 1 -- 172.21.15.27:0/1502854510 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f058c01d960 con 0x7f059c0f77d0 2021-11-03T17:20:38.913 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.912+0000 7f0591ffb700 1 --2- 172.21.15.27:0/1502854510 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f05880641d0 0x7f0588066690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:20:38.913 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.912+0000 7f0591ffb700 1 -- 172.21.15.27:0/1502854510 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f058c0823a0 con 0x7f059c0f77d0 2021-11-03T17:20:38.913 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.912+0000 7f05a1627700 1 --2- 172.21.15.27:0/1502854510 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f05880641d0 0x7f0588066690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:20:38.913 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.912+0000 7f05a1627700 1 --2- 172.21.15.27:0/1502854510 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f05880641d0 0x7f0588066690 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f059c107b40 tx=0x7f0598008040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:20:38.914 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:38.916+0000 7f0591ffb700 1 -- 172.21.15.27:0/1502854510 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f058c054840 con 0x7f059c0f77d0 2021-11-03T17:20:39.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:39.122+0000 7f05a2629700 1 -- 172.21.15.27:0/1502854510 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f059c0fed90 con 0x7f059c0f77d0 2021-11-03T17:20:39.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:39.123+0000 7f0591ffb700 1 -- 172.21.15.27:0/1502854510 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f058c054430 con 0x7f059c0f77d0 2021-11-03T17:20:39.119 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:20:39.119 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:20:39.119 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:20:39.120 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:20:39.120 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:20:39.120 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:20:39.120 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:20:39.121 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:20:39.121 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:20:39.121 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:20:39.121 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:20:39.121 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:20:39.122 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:20:39.122 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:20:39.122 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:20:39.122 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:20:39.122 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:20:39.123 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:20:39.123 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:20:39.123 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:20:39.123 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:20:39.123 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:20:39.124 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:20:39.124 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:20:39.124 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:20:39.124 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:20:39.124 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:20:39.125 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:20:39.125 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:20:39.125 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:20:39.125 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:20:39.126 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:20:39.126 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:20:39.126 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:20:39.126 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:20:39.126 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:20:39.127 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:20:39.127 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:20:39.127 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:20:39.128 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:39.124+0000 7f05a2629700 1 -- 172.21.15.27:0/1502854510 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f05880641d0 msgr2=0x7f0588066690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:20:39.128 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:39.124+0000 7f05a2629700 1 --2- 172.21.15.27:0/1502854510 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f05880641d0 0x7f0588066690 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f059c107b40 tx=0x7f0598008040).stop 2021-11-03T17:20:39.129 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:39.124+0000 7f05a2629700 1 -- 172.21.15.27:0/1502854510 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f059c0f77d0 msgr2=0x7f059c107270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:20:39.129 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:39.124+0000 7f05a2629700 1 --2- 172.21.15.27:0/1502854510 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f059c0f77d0 0x7f059c107270 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f058c009990 tx=0x7f058c009b30).stop 2021-11-03T17:20:39.129 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:39.124+0000 7f05a2629700 1 -- 172.21.15.27:0/1502854510 shutdown_connections 2021-11-03T17:20:39.129 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:39.124+0000 7f05a2629700 1 --2- 172.21.15.27:0/1502854510 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f05880641d0 0x7f0588066690 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:39.129 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:39.124+0000 7f05a2629700 1 --2- 172.21.15.27:0/1502854510 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f059c0f6530 0x7f059c106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:39.130 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:39.124+0000 7f05a2629700 1 --2- 172.21.15.27:0/1502854510 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f059c0f77d0 0x7f059c107270 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:20:39.130 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:39.124+0000 7f05a2629700 1 -- 172.21.15.27:0/1502854510 >> 172.21.15.27:0/1502854510 conn(0x7f059c0f1cd0 msgr2=0x7f059c0fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:20:39.130 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:39.125+0000 7f05a2629700 1 -- 172.21.15.27:0/1502854510 shutdown_connections 2021-11-03T17:20:39.130 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:20:39.125+0000 7f05a2629700 1 -- 172.21.15.27:0/1502854510 wait complete. 2021-11-03T17:20:39.131 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:20:39.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:38 smithi115 ceph-mon[65453]: pgmap v178: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:39.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:38 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3580402069' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:39.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:38 smithi027 ceph-mon[97244]: pgmap v178: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:39.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:38 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3580402069' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:40.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:39 smithi115 ceph-mon[65453]: from='client.34296 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:20:40.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:39 smithi115 ceph-mon[65453]: from='client.34300 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:20:40.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:39 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1502854510' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:20:40.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:39 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 282.365597 secs 2021-11-03T17:20:40.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:39 smithi027 ceph-mon[97244]: from='client.34296 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:20:40.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:39 smithi027 ceph-mon[97244]: from='client.34300 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:20:40.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:39 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1502854510' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:20:40.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:39 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 282.365597 secs 2021-11-03T17:20:41.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:40 smithi115 ceph-mon[65453]: pgmap v179: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:41.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:40 smithi027 ceph-mon[97244]: pgmap v179: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:43.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:42 smithi115 ceph-mon[65453]: pgmap v180: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:43.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:42 smithi027 ceph-mon[97244]: pgmap v180: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:45.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:44 smithi115 ceph-mon[65453]: pgmap v181: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:45.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:44 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 287.365664 secs 2021-11-03T17:20:45.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:44 smithi027 ceph-mon[97244]: pgmap v181: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:45.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:44 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 287.365664 secs 2021-11-03T17:20:46.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:20:46.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:20:46.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:20:46.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:46.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:46.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:46.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:46.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:20:46.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:20:46.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:20:46.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:46.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:46.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:46.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:47.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:46 smithi115 ceph-mon[65453]: poll took 60184.140 ms: 1 events 2021-11-03T17:20:47.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:46 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:20:47.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:46 smithi115 ceph-mon[65453]: pgmap v182: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:47.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:46 smithi027 ceph-mon[97244]: poll took 60184.140 ms: 1 events 2021-11-03T17:20:47.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:46 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:20:47.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:46 smithi027 ceph-mon[97244]: pgmap v182: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:49.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:48 smithi115 ceph-mon[65453]: pgmap v183: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:49.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:48 smithi027 ceph-mon[97244]: pgmap v183: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:50.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:49 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 292.365799 secs 2021-11-03T17:20:50.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:49 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 292.365799 secs 2021-11-03T17:20:51.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:50 smithi115 ceph-mon[65453]: pgmap v184: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:51.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:50 smithi027 ceph-mon[97244]: pgmap v184: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:53.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:52 smithi115 ceph-mon[65453]: pgmap v185: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:53.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:52 smithi027 ceph-mon[97244]: pgmap v185: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:55.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:54 smithi115 ceph-mon[65453]: pgmap v186: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:55.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:54 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 297.365939 secs 2021-11-03T17:20:55.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:54 smithi027 ceph-mon[97244]: pgmap v186: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:55.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:54 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 297.365939 secs 2021-11-03T17:20:56.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:20:56.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:20:56.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:20:56.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:20:56.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:20:56.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:56.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:56.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:56.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:56.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:20:56.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:20:56.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:20:56.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:20:56.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:20:56.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:56.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:56.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:56.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:20:57.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:56 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:20:57.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:56 smithi115 ceph-mon[65453]: pgmap v187: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:57.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:56 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:20:57.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:56 smithi027 ceph-mon[97244]: pgmap v187: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:59.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:58 smithi115 ceph-mon[65453]: pgmap v188: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:20:59.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:58 smithi027 ceph-mon[97244]: pgmap v188: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:00.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:20:59 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 302.366097 secs 2021-11-03T17:21:00.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:20:59 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 302.366097 secs 2021-11-03T17:21:01.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:00 smithi115 ceph-mon[65453]: pgmap v189: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:01.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:00 smithi027 ceph-mon[97244]: pgmap v189: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:03.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:02 smithi115 ceph-mon[65453]: pgmap v190: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:03.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:02 smithi027 ceph-mon[97244]: pgmap v190: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:05.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:04 smithi115 ceph-mon[65453]: pgmap v191: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:05.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:04 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 307.366221 secs 2021-11-03T17:21:05.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:04 smithi027 ceph-mon[97244]: pgmap v191: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:05.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:04 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 307.366221 secs 2021-11-03T17:21:06.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:21:06.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:21:06.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:21:06.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:06.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:06.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:06.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:06.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:21:06.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:21:06.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:21:06.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:06.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:06.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:06.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:07.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:06 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:21:07.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:06 smithi115 ceph-mon[65453]: pgmap v192: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:07.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:06 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:21:07.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:06 smithi027 ceph-mon[97244]: pgmap v192: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:09.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:08 smithi115 ceph-mon[65453]: pgmap v193: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:09.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.240+0000 7f31bce0b700 1 -- 172.21.15.27:0/2260100521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31b80f9830 msgr2=0x7f31b80fbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:21:09.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.240+0000 7f31bce0b700 1 --2- 172.21.15.27:0/2260100521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31b80f9830 0x7f31b80fbc20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f31ac004660 tx=0x7f31ac009b30).stop 2021-11-03T17:21:09.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.240+0000 7f31bce0b700 1 -- 172.21.15.27:0/2260100521 shutdown_connections 2021-11-03T17:21:09.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.240+0000 7f31bce0b700 1 --2- 172.21.15.27:0/2260100521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31b80f9830 0x7f31b80fbc20 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:09.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.240+0000 7f31bce0b700 1 --2- 172.21.15.27:0/2260100521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31b8060350 0x7f31b80f9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:09.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.240+0000 7f31bce0b700 1 -- 172.21.15.27:0/2260100521 >> 172.21.15.27:0/2260100521 conn(0x7f31b80f0de0 msgr2=0x7f31b80f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:21:09.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.240+0000 7f31bce0b700 1 -- 172.21.15.27:0/2260100521 shutdown_connections 2021-11-03T17:21:09.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.240+0000 7f31bce0b700 1 -- 172.21.15.27:0/2260100521 wait complete. 2021-11-03T17:21:09.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.241+0000 7f31bce0b700 1 Processor -- start 2021-11-03T17:21:09.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.241+0000 7f31bce0b700 1 -- start start 2021-11-03T17:21:09.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.241+0000 7f31bce0b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31b8060350 0x7f31b8106f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:21:09.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.241+0000 7f31bce0b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31b80f9830 0x7f31b8107430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:21:09.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.241+0000 7f31bce0b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b81079e0 con 0x7f31b80f9830 2021-11-03T17:21:09.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.241+0000 7f31bce0b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b8107b20 con 0x7f31b8060350 2021-11-03T17:21:09.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.241+0000 7f31b5d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31b80f9830 0x7f31b8107430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:21:09.242 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.241+0000 7f31b5d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31b80f9830 0x7f31b8107430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56932/0 (socket says 172.21.15.27:56932) 2021-11-03T17:21:09.242 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.241+0000 7f31b5d9b700 1 -- 172.21.15.27:0/111044065 learned_addr learned my addr 172.21.15.27:0/111044065 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:21:09.242 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.241+0000 7f31b659c700 1 --2- 172.21.15.27:0/111044065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31b8060350 0x7f31b8106f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:21:09.242 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.242+0000 7f31b5d9b700 1 -- 172.21.15.27:0/111044065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31b8060350 msgr2=0x7f31b8106f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:21:09.243 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.242+0000 7f31b5d9b700 1 --2- 172.21.15.27:0/111044065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31b8060350 0x7f31b8106f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:09.243 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.242+0000 7f31b5d9b700 1 -- 172.21.15.27:0/111044065 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31ac005040 con 0x7f31b80f9830 2021-11-03T17:21:09.243 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.242+0000 7f31b659c700 1 --2- 172.21.15.27:0/111044065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31b8060350 0x7f31b8106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:21:09.243 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.242+0000 7f31b5d9b700 1 --2- 172.21.15.27:0/111044065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31b80f9830 0x7f31b8107430 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f31ac009990 tx=0x7f31ac009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:21:09.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.242+0000 7f31a6ffd700 1 -- 172.21.15.27:0/111044065 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31ac018070 con 0x7f31b80f9830 2021-11-03T17:21:09.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.242+0000 7f31a6ffd700 1 -- 172.21.15.27:0/111044065 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f31ac007ca0 con 0x7f31b80f9830 2021-11-03T17:21:09.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.242+0000 7f31bce0b700 1 -- 172.21.15.27:0/111044065 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31b8108460 con 0x7f31b80f9830 2021-11-03T17:21:09.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.242+0000 7f31a6ffd700 1 -- 172.21.15.27:0/111044065 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31ac01c620 con 0x7f31b80f9830 2021-11-03T17:21:09.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.242+0000 7f31bce0b700 1 -- 172.21.15.27:0/111044065 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31b81089c0 con 0x7f31b80f9830 2021-11-03T17:21:09.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.243+0000 7f31bce0b700 1 -- 172.21.15.27:0/111044065 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31b8045ad0 con 0x7f31b80f9830 2021-11-03T17:21:09.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.243+0000 7f31a6ffd700 1 -- 172.21.15.27:0/111044065 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f31ac01d960 con 0x7f31b80f9830 2021-11-03T17:21:09.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.244+0000 7f31a6ffd700 1 --2- 172.21.15.27:0/111044065 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f31a00640b0 0x7f31a0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:21:09.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.244+0000 7f31a6ffd700 1 -- 172.21.15.27:0/111044065 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f31ac082520 con 0x7f31b80f9830 2021-11-03T17:21:09.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.244+0000 7f31b659c700 1 --2- 172.21.15.27:0/111044065 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f31a00640b0 0x7f31a0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:21:09.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.244+0000 7f31b659c700 1 --2- 172.21.15.27:0/111044065 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f31a00640b0 0x7f31a0066570 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f31b8060120 tx=0x7f31a8009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:21:09.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.248+0000 7f31a6ffd700 1 -- 172.21.15.27:0/111044065 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f31ac054940 con 0x7f31b80f9830 2021-11-03T17:21:09.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:08 smithi027 ceph-mon[97244]: pgmap v193: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:09.426 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.431+0000 7f31bce0b700 1 -- 172.21.15.27:0/111044065 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f31b8108be0 con 0x7f31a00640b0 2021-11-03T17:21:09.427 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.432+0000 7f31a6ffd700 1 -- 172.21.15.27:0/111044065 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f31b8108be0 con 0x7f31a00640b0 2021-11-03T17:21:09.430 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.434+0000 7f31bce0b700 1 -- 172.21.15.27:0/111044065 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f31a00640b0 msgr2=0x7f31a0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:21:09.430 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.434+0000 7f31bce0b700 1 --2- 172.21.15.27:0/111044065 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f31a00640b0 0x7f31a0066570 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f31b8060120 tx=0x7f31a8009040).stop 2021-11-03T17:21:09.430 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.434+0000 7f31bce0b700 1 -- 172.21.15.27:0/111044065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31b80f9830 msgr2=0x7f31b8107430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:21:09.431 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.434+0000 7f31bce0b700 1 --2- 172.21.15.27:0/111044065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31b80f9830 0x7f31b8107430 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f31ac009990 tx=0x7f31ac009b30).stop 2021-11-03T17:21:09.431 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.434+0000 7f31bce0b700 1 -- 172.21.15.27:0/111044065 shutdown_connections 2021-11-03T17:21:09.431 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.434+0000 7f31bce0b700 1 --2- 172.21.15.27:0/111044065 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f31a00640b0 0x7f31a0066570 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:09.431 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.434+0000 7f31bce0b700 1 --2- 172.21.15.27:0/111044065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31b8060350 0x7f31b8106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:09.432 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.434+0000 7f31bce0b700 1 --2- 172.21.15.27:0/111044065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31b80f9830 0x7f31b8107430 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:09.432 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.434+0000 7f31bce0b700 1 -- 172.21.15.27:0/111044065 >> 172.21.15.27:0/111044065 conn(0x7f31b80f0de0 msgr2=0x7f31b80f31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:21:09.432 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.434+0000 7f31bce0b700 1 -- 172.21.15.27:0/111044065 shutdown_connections 2021-11-03T17:21:09.432 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.434+0000 7f31bce0b700 1 -- 172.21.15.27:0/111044065 wait complete. 2021-11-03T17:21:09.440 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:21:09.533 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.536+0000 7fb61f60b700 1 -- 172.21.15.27:0/3583258251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6180f5ef0 msgr2=0x7fb6180f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:21:09.534 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.536+0000 7fb61f60b700 1 --2- 172.21.15.27:0/3583258251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6180f5ef0 0x7fb6180f6920 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fb614004660 tx=0x7fb614009b30).stop 2021-11-03T17:21:09.534 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.536+0000 7fb61f60b700 1 -- 172.21.15.27:0/3583258251 shutdown_connections 2021-11-03T17:21:09.534 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.536+0000 7fb61f60b700 1 --2- 172.21.15.27:0/3583258251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6180f5ef0 0x7fb6180f6920 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:09.535 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.536+0000 7fb61f60b700 1 --2- 172.21.15.27:0/3583258251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6180f55e0 0x7fb6180f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:09.535 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.536+0000 7fb61f60b700 1 -- 172.21.15.27:0/3583258251 >> 172.21.15.27:0/3583258251 conn(0x7fb6180f0de0 msgr2=0x7fb6180f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:21:09.535 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.536+0000 7fb61f60b700 1 -- 172.21.15.27:0/3583258251 shutdown_connections 2021-11-03T17:21:09.535 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.536+0000 7fb61f60b700 1 -- 172.21.15.27:0/3583258251 wait complete. 2021-11-03T17:21:09.536 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.536+0000 7fb61f60b700 1 Processor -- start 2021-11-03T17:21:09.536 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.536+0000 7fb61f60b700 1 -- start start 2021-11-03T17:21:09.536 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.536+0000 7fb61f60b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6180f55e0 0x7fb618105e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:21:09.536 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.536+0000 7fb61f60b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6180f5ef0 0x7fb618106380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:21:09.537 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.536+0000 7fb61f60b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb618106930 con 0x7fb6180f5ef0 2021-11-03T17:21:09.537 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.536+0000 7fb61f60b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb618106a70 con 0x7fb6180f55e0 2021-11-03T17:21:09.537 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.537+0000 7fb61cba6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6180f5ef0 0x7fb618106380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:21:09.537 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.537+0000 7fb61cba6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6180f5ef0 0x7fb618106380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56942/0 (socket says 172.21.15.27:56942) 2021-11-03T17:21:09.537 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.537+0000 7fb61cba6700 1 -- 172.21.15.27:0/451022391 learned_addr learned my addr 172.21.15.27:0/451022391 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:21:09.538 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.537+0000 7fb61d3a7700 1 --2- 172.21.15.27:0/451022391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6180f55e0 0x7fb618105e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:21:09.538 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.537+0000 7fb61cba6700 1 -- 172.21.15.27:0/451022391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6180f55e0 msgr2=0x7fb618105e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:21:09.538 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.537+0000 7fb61cba6700 1 --2- 172.21.15.27:0/451022391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6180f55e0 0x7fb618105e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:09.538 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.537+0000 7fb61cba6700 1 -- 172.21.15.27:0/451022391 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb614005040 con 0x7fb6180f5ef0 2021-11-03T17:21:09.539 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.537+0000 7fb61d3a7700 1 --2- 172.21.15.27:0/451022391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6180f55e0 0x7fb618105e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:21:09.539 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.537+0000 7fb61cba6700 1 --2- 172.21.15.27:0/451022391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6180f5ef0 0x7fb618106380 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7fb614009990 tx=0x7fb614009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:21:09.539 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.537+0000 7fb60dffb700 1 -- 172.21.15.27:0/451022391 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb61401a070 con 0x7fb6180f5ef0 2021-11-03T17:21:09.539 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.537+0000 7fb61f60b700 1 -- 172.21.15.27:0/451022391 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6181073b0 con 0x7fb6180f5ef0 2021-11-03T17:21:09.540 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.537+0000 7fb60dffb700 1 -- 172.21.15.27:0/451022391 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fb614012a80 con 0x7fb6180f5ef0 2021-11-03T17:21:09.540 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.537+0000 7fb60dffb700 1 -- 172.21.15.27:0/451022391 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb61401f9a0 con 0x7fb6180f5ef0 2021-11-03T17:21:09.540 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.537+0000 7fb61f60b700 1 -- 172.21.15.27:0/451022391 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb618107910 con 0x7fb6180f5ef0 2021-11-03T17:21:09.540 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.538+0000 7fb61f60b700 1 -- 172.21.15.27:0/451022391 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb618045ad0 con 0x7fb6180f5ef0 2021-11-03T17:21:09.542 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.539+0000 7fb60dffb700 1 -- 172.21.15.27:0/451022391 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fb614012bf0 con 0x7fb6180f5ef0 2021-11-03T17:21:09.542 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.539+0000 7fb60dffb700 1 --2- 172.21.15.27:0/451022391 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb6040640b0 0x7fb604066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:21:09.542 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.539+0000 7fb60dffb700 1 -- 172.21.15.27:0/451022391 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fb614084ac0 con 0x7fb6180f5ef0 2021-11-03T17:21:09.542 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.539+0000 7fb61d3a7700 1 --2- 172.21.15.27:0/451022391 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb6040640b0 0x7fb604066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:21:09.543 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.539+0000 7fb61d3a7700 1 --2- 172.21.15.27:0/451022391 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb6040640b0 0x7fb604066570 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fb608004a90 tx=0x7fb608009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:21:09.543 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.543+0000 7fb60dffb700 1 -- 172.21.15.27:0/451022391 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fb614056e20 con 0x7fb6180f5ef0 2021-11-03T17:21:09.716 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.720+0000 7fb61f60b700 1 -- 172.21.15.27:0/451022391 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb6180fdf80 con 0x7fb6040640b0 2021-11-03T17:21:09.730 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.735+0000 7fb60dffb700 1 -- 172.21.15.27:0/451022391 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fb608004670 con 0x7fb6040640b0 2021-11-03T17:21:09.730 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:21:09.731 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (12m) 2m ago 13m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:21:09.731 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (4m) 2m ago 13m 7205k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:21:09.731 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (4m) 97s ago 13m 7239k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:21:09.731 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (12m) 2m ago 13m 38.9M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:21:09.732 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (6m) 2m ago 9m 15.4M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:21:09.732 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (9m) 2m ago 9m 3836M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:21:09.732 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (4m) 97s ago 9m 14.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:21:09.732 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (6m) 97s ago 9m 16.3M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:21:09.733 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (6m) 2m ago 15m 470M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:21:09.733 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (5m) 97s ago 13m 412M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:21:09.733 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (5m) 2m ago 15m 81.2M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:21:09.733 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (5m) 97s ago 13m 69.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:21:09.734 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (13m) 2m ago 13m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:21:09.734 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (12m) 97s ago 12m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:21:09.734 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (4m) 2m ago 12m 243M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:21:09.734 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (3m) 2m ago 12m 270M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:21:09.735 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (3m) 2m ago 11m 11.8M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:21:09.735 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (2m) 97s ago 11m 289M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:21:09.735 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (2m) 97s ago 11m 235M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:21:09.735 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (100s) 97s ago 10m 11.5M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:21:09.736 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (12m) 2m ago 13m 45.7M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:21:09.737 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.737+0000 7fb61f60b700 1 -- 172.21.15.27:0/451022391 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb6040640b0 msgr2=0x7fb604066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:21:09.737 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.737+0000 7fb61f60b700 1 --2- 172.21.15.27:0/451022391 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb6040640b0 0x7fb604066570 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fb608004a90 tx=0x7fb608009040).stop 2021-11-03T17:21:09.737 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.737+0000 7fb61f60b700 1 -- 172.21.15.27:0/451022391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6180f5ef0 msgr2=0x7fb618106380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:21:09.737 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.737+0000 7fb61f60b700 1 --2- 172.21.15.27:0/451022391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6180f5ef0 0x7fb618106380 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7fb614009990 tx=0x7fb614009b30).stop 2021-11-03T17:21:09.738 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.737+0000 7fb61f60b700 1 -- 172.21.15.27:0/451022391 shutdown_connections 2021-11-03T17:21:09.738 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.737+0000 7fb61f60b700 1 --2- 172.21.15.27:0/451022391 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb6040640b0 0x7fb604066570 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:09.738 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.737+0000 7fb61f60b700 1 --2- 172.21.15.27:0/451022391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6180f55e0 0x7fb618105e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:09.739 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.737+0000 7fb61f60b700 1 --2- 172.21.15.27:0/451022391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6180f5ef0 0x7fb618106380 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:09.739 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.737+0000 7fb61f60b700 1 -- 172.21.15.27:0/451022391 >> 172.21.15.27:0/451022391 conn(0x7fb6180f0de0 msgr2=0x7fb6180fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:21:09.739 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.737+0000 7fb61f60b700 1 -- 172.21.15.27:0/451022391 shutdown_connections 2021-11-03T17:21:09.739 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.737+0000 7fb61f60b700 1 -- 172.21.15.27:0/451022391 wait complete. 2021-11-03T17:21:09.835 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.838+0000 7f620ce6f700 1 -- 172.21.15.27:0/803909947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62080f6860 msgr2=0x7f62080f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:21:09.836 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.838+0000 7f620ce6f700 1 --2- 172.21.15.27:0/803909947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62080f6860 0x7f62080f6cc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7f61fc004660 tx=0x7f61fc009b30).stop 2021-11-03T17:21:09.836 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.838+0000 7f620ce6f700 1 -- 172.21.15.27:0/803909947 shutdown_connections 2021-11-03T17:21:09.836 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.838+0000 7f620ce6f700 1 --2- 172.21.15.27:0/803909947 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62080f6860 0x7f62080f6cc0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:09.837 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.838+0000 7f620ce6f700 1 --2- 172.21.15.27:0/803909947 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62080f55c0 0x7f62080f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:09.837 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.838+0000 7f620ce6f700 1 -- 172.21.15.27:0/803909947 >> 172.21.15.27:0/803909947 conn(0x7f62080f0d40 msgr2=0x7f62080f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:21:09.837 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.838+0000 7f620ce6f700 1 -- 172.21.15.27:0/803909947 shutdown_connections 2021-11-03T17:21:09.837 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.838+0000 7f620ce6f700 1 -- 172.21.15.27:0/803909947 wait complete. 2021-11-03T17:21:09.837 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.838+0000 7f620ce6f700 1 Processor -- start 2021-11-03T17:21:09.838 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.838+0000 7f620ce6f700 1 -- start start 2021-11-03T17:21:09.838 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.838+0000 7f620ce6f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62080f55c0 0x7f6208106f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:21:09.838 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.838+0000 7f620ce6f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62080f6860 0x7f6208107440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:21:09.838 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.838+0000 7f620ce6f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62081079f0 con 0x7f62080f6860 2021-11-03T17:21:09.839 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.838+0000 7f620ce6f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6208107b30 con 0x7f62080f55c0 2021-11-03T17:21:09.839 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.839+0000 7f6205d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62080f6860 0x7f6208107440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:21:09.839 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.839+0000 7f6205d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62080f6860 0x7f6208107440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56952/0 (socket says 172.21.15.27:56952) 2021-11-03T17:21:09.839 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.839+0000 7f6205d9b700 1 -- 172.21.15.27:0/3641827112 learned_addr learned my addr 172.21.15.27:0/3641827112 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:21:09.840 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.839+0000 7f620659c700 1 --2- 172.21.15.27:0/3641827112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62080f55c0 0x7f6208106f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:21:09.840 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.839+0000 7f6205d9b700 1 -- 172.21.15.27:0/3641827112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62080f55c0 msgr2=0x7f6208106f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:21:09.840 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.839+0000 7f6205d9b700 1 --2- 172.21.15.27:0/3641827112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62080f55c0 0x7f6208106f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:09.840 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.839+0000 7f6205d9b700 1 -- 172.21.15.27:0/3641827112 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61fc005040 con 0x7f62080f6860 2021-11-03T17:21:09.840 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.839+0000 7f6205d9b700 1 --2- 172.21.15.27:0/3641827112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62080f6860 0x7f6208107440 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f61fc009990 tx=0x7f61fc009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:21:09.841 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.839+0000 7f61f6ffd700 1 -- 172.21.15.27:0/3641827112 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61fc018070 con 0x7f62080f6860 2021-11-03T17:21:09.841 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.839+0000 7f620ce6f700 1 -- 172.21.15.27:0/3641827112 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6208108470 con 0x7f62080f6860 2021-11-03T17:21:09.841 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.839+0000 7f61f6ffd700 1 -- 172.21.15.27:0/3641827112 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f61fc007ba0 con 0x7f62080f6860 2021-11-03T17:21:09.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.839+0000 7f61f6ffd700 1 -- 172.21.15.27:0/3641827112 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61fc01c620 con 0x7f62080f6860 2021-11-03T17:21:09.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.839+0000 7f620ce6f700 1 -- 172.21.15.27:0/3641827112 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62081089d0 con 0x7f62080f6860 2021-11-03T17:21:09.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.840+0000 7f620ce6f700 1 -- 172.21.15.27:0/3641827112 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6208045ad0 con 0x7f62080f6860 2021-11-03T17:21:09.844 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.841+0000 7f61f6ffd700 1 -- 172.21.15.27:0/3641827112 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f61fc01c8e0 con 0x7f62080f6860 2021-11-03T17:21:09.844 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.841+0000 7f61f6ffd700 1 --2- 172.21.15.27:0/3641827112 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f61f00640b0 0x7f61f0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:21:09.845 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.842+0000 7f620659c700 1 --2- 172.21.15.27:0/3641827112 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f61f00640b0 0x7f61f0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:21:09.845 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.842+0000 7f61f6ffd700 1 -- 172.21.15.27:0/3641827112 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f61fc082870 con 0x7f62080f6860 2021-11-03T17:21:09.846 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.842+0000 7f620659c700 1 --2- 172.21.15.27:0/3641827112 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f61f00640b0 0x7f61f0066570 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f62080f66c0 tx=0x7f6200009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:21:09.846 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:09.845+0000 7f61f6ffd700 1 -- 172.21.15.27:0/3641827112 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f61fc054bd0 con 0x7f62080f6860 2021-11-03T17:21:10.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.076+0000 7f620ce6f700 1 -- 172.21.15.27:0/3641827112 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6208027a00 con 0x7f62080f6860 2021-11-03T17:21:10.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.076+0000 7f61f6ffd700 1 -- 172.21.15.27:0/3641827112 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f61fc0547c0 con 0x7f62080f6860 2021-11-03T17:21:10.072 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:21:10.073 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:21:10.073 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:21:10.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:21:10.074 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:21:10.074 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:21:10.074 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:21:10.074 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:21:10.075 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:21:10.075 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:21:10.075 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:21:10.075 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:21:10.076 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:21:10.076 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:21:10.076 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:21:10.076 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:21:10.077 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:21:10.077 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:21:10.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.078+0000 7f620ce6f700 1 -- 172.21.15.27:0/3641827112 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f61f00640b0 msgr2=0x7f61f0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:21:10.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.078+0000 7f620ce6f700 1 --2- 172.21.15.27:0/3641827112 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f61f00640b0 0x7f61f0066570 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f62080f66c0 tx=0x7f6200009040).stop 2021-11-03T17:21:10.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.078+0000 7f620ce6f700 1 -- 172.21.15.27:0/3641827112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62080f6860 msgr2=0x7f6208107440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:21:10.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.078+0000 7f620ce6f700 1 --2- 172.21.15.27:0/3641827112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62080f6860 0x7f6208107440 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f61fc009990 tx=0x7f61fc009b30).stop 2021-11-03T17:21:10.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.078+0000 7f620ce6f700 1 -- 172.21.15.27:0/3641827112 shutdown_connections 2021-11-03T17:21:10.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.078+0000 7f620ce6f700 1 --2- 172.21.15.27:0/3641827112 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f61f00640b0 0x7f61f0066570 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:10.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.078+0000 7f620ce6f700 1 --2- 172.21.15.27:0/3641827112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62080f55c0 0x7f6208106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:10.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.078+0000 7f620ce6f700 1 --2- 172.21.15.27:0/3641827112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62080f6860 0x7f6208107440 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:10.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.078+0000 7f620ce6f700 1 -- 172.21.15.27:0/3641827112 >> 172.21.15.27:0/3641827112 conn(0x7f62080f0d40 msgr2=0x7f62080fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:21:10.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.078+0000 7f620ce6f700 1 -- 172.21.15.27:0/3641827112 shutdown_connections 2021-11-03T17:21:10.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.078+0000 7f620ce6f700 1 -- 172.21.15.27:0/3641827112 wait complete. 2021-11-03T17:21:10.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.184+0000 7f3512fee700 1 -- 172.21.15.27:0/3254434534 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f350c0f98f0 msgr2=0x7f350c0fbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:21:10.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.184+0000 7f3512fee700 1 --2- 172.21.15.27:0/3254434534 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f350c0f98f0 0x7f350c0fbce0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f34f8004660 tx=0x7f34f8009b30).stop 2021-11-03T17:21:10.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.184+0000 7f3512fee700 1 -- 172.21.15.27:0/3254434534 shutdown_connections 2021-11-03T17:21:10.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.184+0000 7f3512fee700 1 --2- 172.21.15.27:0/3254434534 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f350c0f98f0 0x7f350c0fbce0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:10.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.185+0000 7f3512fee700 1 --2- 172.21.15.27:0/3254434534 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f350c0602d0 0x7f350c0f9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:10.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.185+0000 7f3512fee700 1 -- 172.21.15.27:0/3254434534 >> 172.21.15.27:0/3254434534 conn(0x7f350c0f0de0 msgr2=0x7f350c0f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:21:10.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.185+0000 7f3512fee700 1 -- 172.21.15.27:0/3254434534 shutdown_connections 2021-11-03T17:21:10.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.185+0000 7f3512fee700 1 -- 172.21.15.27:0/3254434534 wait complete. 2021-11-03T17:21:10.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.185+0000 7f3512fee700 1 Processor -- start 2021-11-03T17:21:10.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.185+0000 7f3512fee700 1 -- start start 2021-11-03T17:21:10.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.185+0000 7f3512fee700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f350c0602d0 0x7f350c0f6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:21:10.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.185+0000 7f3512fee700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f350c0f98f0 0x7f350c0f5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:21:10.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.185+0000 7f3512fee700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350c0f5860 con 0x7f350c0f98f0 2021-11-03T17:21:10.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.185+0000 7f3512fee700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350c0f59a0 con 0x7f350c0602d0 2021-11-03T17:21:10.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.185+0000 7f350bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f350c0f98f0 0x7f350c0f5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:21:10.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.186+0000 7f350bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f350c0f98f0 0x7f350c0f5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56962/0 (socket says 172.21.15.27:56962) 2021-11-03T17:21:10.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.186+0000 7f350bfff700 1 -- 172.21.15.27:0/101443498 learned_addr learned my addr 172.21.15.27:0/101443498 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:21:10.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.186+0000 7f3510d8a700 1 --2- 172.21.15.27:0/101443498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f350c0602d0 0x7f350c0f6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:21:10.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.186+0000 7f350bfff700 1 -- 172.21.15.27:0/101443498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f350c0602d0 msgr2=0x7f350c0f6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:21:10.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.186+0000 7f350bfff700 1 --2- 172.21.15.27:0/101443498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f350c0602d0 0x7f350c0f6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:10.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.186+0000 7f350bfff700 1 -- 172.21.15.27:0/101443498 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34f8005040 con 0x7f350c0f98f0 2021-11-03T17:21:10.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.186+0000 7f3510d8a700 1 --2- 172.21.15.27:0/101443498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f350c0602d0 0x7f350c0f6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:21:10.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.186+0000 7f350bfff700 1 --2- 172.21.15.27:0/101443498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f350c0f98f0 0x7f350c0f5300 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f34f8009990 tx=0x7f34f8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:21:10.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.186+0000 7f35097fa700 1 -- 172.21.15.27:0/101443498 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34f8018070 con 0x7f350c0f98f0 2021-11-03T17:21:10.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.186+0000 7f3512fee700 1 -- 172.21.15.27:0/101443498 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f350c0f5c20 con 0x7f350c0f98f0 2021-11-03T17:21:10.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.186+0000 7f35097fa700 1 -- 172.21.15.27:0/101443498 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f34f8007ba0 con 0x7f350c0f98f0 2021-11-03T17:21:10.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.186+0000 7f35097fa700 1 -- 172.21.15.27:0/101443498 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34f801c3c0 con 0x7f350c0f98f0 2021-11-03T17:21:10.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.186+0000 7f3512fee700 1 -- 172.21.15.27:0/101443498 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f350c0f6180 con 0x7f350c0f98f0 2021-11-03T17:21:10.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.187+0000 7f3512fee700 1 -- 172.21.15.27:0/101443498 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f350c045ad0 con 0x7f350c0f98f0 2021-11-03T17:21:10.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.188+0000 7f35097fa700 1 -- 172.21.15.27:0/101443498 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f34f801d420 con 0x7f350c0f98f0 2021-11-03T17:21:10.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.188+0000 7f35097fa700 1 --2- 172.21.15.27:0/101443498 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f34fc0640b0 0x7f34fc066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:21:10.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.188+0000 7f35097fa700 1 -- 172.21.15.27:0/101443498 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f34f8082500 con 0x7f350c0f98f0 2021-11-03T17:21:10.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.188+0000 7f3510d8a700 1 --2- 172.21.15.27:0/101443498 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f34fc0640b0 0x7f34fc066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:21:10.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.188+0000 7f3510d8a700 1 --2- 172.21.15.27:0/101443498 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f34fc0640b0 0x7f34fc066570 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f350c0f6c10 tx=0x7f3500009c80).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:21:10.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.192+0000 7f35097fa700 1 -- 172.21.15.27:0/101443498 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f34f80548c0 con 0x7f350c0f98f0 2021-11-03T17:21:10.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:09 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 312.366301 secs 2021-11-03T17:21:10.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:09 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 312.366301 secs 2021-11-03T17:21:10.393 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.396+0000 7f3512fee700 1 -- 172.21.15.27:0/101443498 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f350c027a00 con 0x7f350c0f98f0 2021-11-03T17:21:10.393 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.397+0000 7f35097fa700 1 -- 172.21.15.27:0/101443498 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f34f80544b0 con 0x7f350c0f98f0 2021-11-03T17:21:10.394 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:21:10.395 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:21:10.395 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:21:10.396 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:21:10.396 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:21:10.396 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:21:10.397 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:21:10.397 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:21:10.398 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:21:10.398 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:21:10.399 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:21:10.399 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:21:10.399 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:21:10.399 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:21:10.400 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:21:10.400 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:21:10.400 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:21:10.400 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:21:10.401 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:21:10.401 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:21:10.401 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:21:10.402 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:21:10.402 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:21:10.402 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:21:10.402 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:21:10.403 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:21:10.403 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:21:10.403 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:21:10.403 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:21:10.403 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:21:10.404 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:21:10.404 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:21:10.404 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:21:10.405 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:21:10.405 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:21:10.405 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:21:10.405 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:21:10.406 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:21:10.406 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:21:10.407 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.399+0000 7f3512fee700 1 -- 172.21.15.27:0/101443498 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f34fc0640b0 msgr2=0x7f34fc066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:21:10.407 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.399+0000 7f3512fee700 1 --2- 172.21.15.27:0/101443498 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f34fc0640b0 0x7f34fc066570 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f350c0f6c10 tx=0x7f3500009c80).stop 2021-11-03T17:21:10.407 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.399+0000 7f3512fee700 1 -- 172.21.15.27:0/101443498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f350c0f98f0 msgr2=0x7f350c0f5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:21:10.408 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.399+0000 7f3512fee700 1 --2- 172.21.15.27:0/101443498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f350c0f98f0 0x7f350c0f5300 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f34f8009990 tx=0x7f34f8009b30).stop 2021-11-03T17:21:10.408 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.399+0000 7f3512fee700 1 -- 172.21.15.27:0/101443498 shutdown_connections 2021-11-03T17:21:10.408 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.399+0000 7f3512fee700 1 --2- 172.21.15.27:0/101443498 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f34fc0640b0 0x7f34fc066570 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:10.408 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.399+0000 7f3512fee700 1 --2- 172.21.15.27:0/101443498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f350c0602d0 0x7f350c0f6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:10.408 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.399+0000 7f3512fee700 1 --2- 172.21.15.27:0/101443498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f350c0f98f0 0x7f350c0f5300 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:10.409 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.399+0000 7f3512fee700 1 -- 172.21.15.27:0/101443498 >> 172.21.15.27:0/101443498 conn(0x7f350c0f0de0 msgr2=0x7f350c0f31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:21:10.409 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.399+0000 7f3512fee700 1 -- 172.21.15.27:0/101443498 shutdown_connections 2021-11-03T17:21:10.409 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:10.399+0000 7f3512fee700 1 -- 172.21.15.27:0/101443498 wait complete. 2021-11-03T17:21:10.410 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:21:11.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:10 smithi115 ceph-mon[65453]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:21:11.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:10 smithi115 ceph-mon[65453]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:21:11.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:10 smithi115 ceph-mon[65453]: pgmap v194: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:11.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:10 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3641827112' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:11.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:10 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/101443498' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:21:11.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:10 smithi027 ceph-mon[97244]: from='client.34312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:21:11.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:10 smithi027 ceph-mon[97244]: from='client.34316 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:21:11.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:10 smithi027 ceph-mon[97244]: pgmap v194: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:11.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:10 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3641827112' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:11.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:10 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/101443498' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:21:13.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:12 smithi115 ceph-mon[65453]: pgmap v195: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:13.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:12 smithi027 ceph-mon[97244]: pgmap v195: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:15.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:14 smithi115 ceph-mon[65453]: pgmap v196: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:15.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:14 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 317.366442 secs 2021-11-03T17:21:15.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:14 smithi027 ceph-mon[97244]: pgmap v196: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:15.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:14 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 317.366442 secs 2021-11-03T17:21:16.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:21:16.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:21:16.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:21:16.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:16.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:16.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:16.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:16.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:21:16.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:21:16.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:21:16.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:16.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:16.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:16.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:17.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:16 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:21:17.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:16 smithi115 ceph-mon[65453]: pgmap v197: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:17.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:16 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:21:17.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:16 smithi027 ceph-mon[97244]: pgmap v197: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:19.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:18 smithi115 ceph-mon[65453]: pgmap v198: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:19.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:18 smithi027 ceph-mon[97244]: pgmap v198: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:20.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:19 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 322.366574 secs 2021-11-03T17:21:20.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:19 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 322.366574 secs 2021-11-03T17:21:21.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:20 smithi115 ceph-mon[65453]: pgmap v199: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:21.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:20 smithi027 ceph-mon[97244]: pgmap v199: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:23.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:22 smithi115 ceph-mon[65453]: pgmap v200: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:23.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:22 smithi027 ceph-mon[97244]: pgmap v200: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:25.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:24 smithi115 ceph-mon[65453]: pgmap v201: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:25.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:24 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 327.366639 secs 2021-11-03T17:21:25.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:24 smithi027 ceph-mon[97244]: pgmap v201: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:25.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:24 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 327.366639 secs 2021-11-03T17:21:26.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:21:26.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:21:26.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:21:26.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:26.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:26.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:26.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:26.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:25 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:21:26.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:25 smithi115 ceph-mon[65453]: pgmap v202: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:26.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:21:26.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:21:26.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:21:26.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:26.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:26.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:26.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:26.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:25 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:21:26.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:25 smithi027 ceph-mon[97244]: pgmap v202: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:29.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:28 smithi115 ceph-mon[65453]: pgmap v203: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:29.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:28 smithi027 ceph-mon[97244]: pgmap v203: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:30.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:29 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 332.366775 secs 2021-11-03T17:21:30.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:29 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 332.366775 secs 2021-11-03T17:21:31.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:30 smithi115 ceph-mon[65453]: pgmap v204: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:31.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:30 smithi027 ceph-mon[97244]: pgmap v204: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:33.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:32 smithi115 ceph-mon[65453]: pgmap v205: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:33.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:32 smithi027 ceph-mon[97244]: pgmap v205: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:35.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:34 smithi115 ceph-mon[65453]: pgmap v206: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:35.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:34 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 337.366903 secs 2021-11-03T17:21:35.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:34 smithi027 ceph-mon[97244]: pgmap v206: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:35.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:34 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 337.366903 secs 2021-11-03T17:21:36.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:21:36.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:21:36.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:21:36.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:36.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:36.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:36.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:36.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:21:36.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:21:36.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:21:36.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:36.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:36.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:36.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:37.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:36 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:21:37.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:36 smithi115 ceph-mon[65453]: pgmap v207: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:37.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:36 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:21:37.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:36 smithi027 ceph-mon[97244]: pgmap v207: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:39.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:38 smithi115 ceph-mon[65453]: pgmap v208: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:39.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:38 smithi027 ceph-mon[97244]: pgmap v208: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:40.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:39 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 342.367029 secs 2021-11-03T17:21:40.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:39 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 342.367029 secs 2021-11-03T17:21:40.512 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.515+0000 7f0330f0a700 1 -- 172.21.15.27:0/3165271881 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f032c0f66e0 msgr2=0x7f032c0f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:21:40.513 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.515+0000 7f0330f0a700 1 --2- 172.21.15.27:0/3165271881 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f032c0f66e0 0x7f032c0f6b00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f03240099e0 tx=0x7f0324004d10).stop 2021-11-03T17:21:40.513 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.515+0000 7f0330f0a700 1 -- 172.21.15.27:0/3165271881 shutdown_connections 2021-11-03T17:21:40.513 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.515+0000 7f0330f0a700 1 --2- 172.21.15.27:0/3165271881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f032c0f7980 0x7f032c0f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:40.514 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.515+0000 7f0330f0a700 1 --2- 172.21.15.27:0/3165271881 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f032c0f66e0 0x7f032c0f6b00 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:40.514 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.515+0000 7f0330f0a700 1 -- 172.21.15.27:0/3165271881 >> 172.21.15.27:0/3165271881 conn(0x7f032c0f1e40 msgr2=0x7f032c0f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:21:40.514 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.515+0000 7f0330f0a700 1 -- 172.21.15.27:0/3165271881 shutdown_connections 2021-11-03T17:21:40.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.515+0000 7f0330f0a700 1 -- 172.21.15.27:0/3165271881 wait complete. 2021-11-03T17:21:40.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.516+0000 7f0330f0a700 1 Processor -- start 2021-11-03T17:21:40.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.516+0000 7f0330f0a700 1 -- start start 2021-11-03T17:21:40.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.516+0000 7f0330f0a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f032c0f66e0 0x7f032c06b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:21:40.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.516+0000 7f0330f0a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f032c0f7980 0x7f032c069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:21:40.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.516+0000 7f0330f0a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032c06a380 con 0x7f032c0f66e0 2021-11-03T17:21:40.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.516+0000 7f0330f0a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f032c06a4c0 con 0x7f032c0f7980 2021-11-03T17:21:40.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.516+0000 7f032a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f032c0f66e0 0x7f032c06b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:21:40.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.516+0000 7f032a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f032c0f66e0 0x7f032c06b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56978/0 (socket says 172.21.15.27:56978) 2021-11-03T17:21:40.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.516+0000 7f032a59c700 1 -- 172.21.15.27:0/3029283400 learned_addr learned my addr 172.21.15.27:0/3029283400 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:21:40.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.516+0000 7f0329d9b700 1 --2- 172.21.15.27:0/3029283400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f032c0f7980 0x7f032c069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:21:40.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.516+0000 7f032a59c700 1 -- 172.21.15.27:0/3029283400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f032c0f7980 msgr2=0x7f032c069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:21:40.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.516+0000 7f032a59c700 1 --2- 172.21.15.27:0/3029283400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f032c0f7980 0x7f032c069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:40.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.516+0000 7f032a59c700 1 -- 172.21.15.27:0/3029283400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0324009640 con 0x7f032c0f66e0 2021-11-03T17:21:40.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.517+0000 7f032a59c700 1 --2- 172.21.15.27:0/3029283400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f032c0f66e0 0x7f032c06b9a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f032400c010 tx=0x7f0324004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:21:40.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.517+0000 7f0322ffd700 1 -- 172.21.15.27:0/3029283400 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f032400e070 con 0x7f032c0f66e0 2021-11-03T17:21:40.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.517+0000 7f0322ffd700 1 -- 172.21.15.27:0/3029283400 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f0324007dc0 con 0x7f032c0f66e0 2021-11-03T17:21:40.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.517+0000 7f0330f0a700 1 -- 172.21.15.27:0/3029283400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f032c06a740 con 0x7f032c0f66e0 2021-11-03T17:21:40.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.517+0000 7f0322ffd700 1 -- 172.21.15.27:0/3029283400 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0324018070 con 0x7f032c0f66e0 2021-11-03T17:21:40.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.517+0000 7f0330f0a700 1 -- 172.21.15.27:0/3029283400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f032c06ac20 con 0x7f032c0f66e0 2021-11-03T17:21:40.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.518+0000 7f0330f0a700 1 -- 172.21.15.27:0/3029283400 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f032c045ad0 con 0x7f032c0f66e0 2021-11-03T17:21:40.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.518+0000 7f0322ffd700 1 -- 172.21.15.27:0/3029283400 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f0324024060 con 0x7f032c0f66e0 2021-11-03T17:21:40.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.519+0000 7f0322ffd700 1 --2- 172.21.15.27:0/3029283400 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f03180640b0 0x7f0318066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:21:40.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.519+0000 7f0322ffd700 1 -- 172.21.15.27:0/3029283400 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f0324082220 con 0x7f032c0f66e0 2021-11-03T17:21:40.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.519+0000 7f0329d9b700 1 --2- 172.21.15.27:0/3029283400 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f03180640b0 0x7f0318066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:21:40.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.519+0000 7f0329d9b700 1 --2- 172.21.15.27:0/3029283400 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f03180640b0 0x7f0318066570 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f0314004a70 tx=0x7f0314009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:21:40.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.523+0000 7f0322ffd700 1 -- 172.21.15.27:0/3029283400 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f0324054580 con 0x7f032c0f66e0 2021-11-03T17:21:40.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.703+0000 7f0330f0a700 1 -- 172.21.15.27:0/3029283400 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f032c0fefe0 con 0x7f03180640b0 2021-11-03T17:21:40.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.704+0000 7f0322ffd700 1 -- 172.21.15.27:0/3029283400 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f032c0fefe0 con 0x7f03180640b0 2021-11-03T17:21:40.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.706+0000 7f0330f0a700 1 -- 172.21.15.27:0/3029283400 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f03180640b0 msgr2=0x7f0318066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:21:40.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.706+0000 7f0330f0a700 1 --2- 172.21.15.27:0/3029283400 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f03180640b0 0x7f0318066570 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f0314004a70 tx=0x7f0314009040).stop 2021-11-03T17:21:40.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.706+0000 7f0330f0a700 1 -- 172.21.15.27:0/3029283400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f032c0f66e0 msgr2=0x7f032c06b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:21:40.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.706+0000 7f0330f0a700 1 --2- 172.21.15.27:0/3029283400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f032c0f66e0 0x7f032c06b9a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f032400c010 tx=0x7f0324004d10).stop 2021-11-03T17:21:40.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.706+0000 7f0330f0a700 1 -- 172.21.15.27:0/3029283400 shutdown_connections 2021-11-03T17:21:40.704 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.706+0000 7f0330f0a700 1 --2- 172.21.15.27:0/3029283400 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f03180640b0 0x7f0318066570 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:40.704 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.706+0000 7f0330f0a700 1 --2- 172.21.15.27:0/3029283400 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f032c0f66e0 0x7f032c06b9a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:40.704 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.706+0000 7f0330f0a700 1 --2- 172.21.15.27:0/3029283400 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f032c0f7980 0x7f032c069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:40.704 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.706+0000 7f0330f0a700 1 -- 172.21.15.27:0/3029283400 >> 172.21.15.27:0/3029283400 conn(0x7f032c0f1e40 msgr2=0x7f032c0fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:21:40.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.706+0000 7f0330f0a700 1 -- 172.21.15.27:0/3029283400 shutdown_connections 2021-11-03T17:21:40.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.706+0000 7f0330f0a700 1 -- 172.21.15.27:0/3029283400 wait complete. 2021-11-03T17:21:40.711 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:21:40.804 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.807+0000 7f7f1b60c700 1 -- 172.21.15.27:0/3797496395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f140f5bb0 msgr2=0x7f7f140f5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:21:40.805 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.807+0000 7f7f1b60c700 1 --2- 172.21.15.27:0/3797496395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f140f5bb0 0x7f7f140f5fd0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f7f04004660 tx=0x7f7f04009b30).stop 2021-11-03T17:21:40.805 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.807+0000 7f7f1b60c700 1 -- 172.21.15.27:0/3797496395 shutdown_connections 2021-11-03T17:21:40.805 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.807+0000 7f7f1b60c700 1 --2- 172.21.15.27:0/3797496395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f140f6550 0x7f7f140f69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:40.806 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.807+0000 7f7f1b60c700 1 --2- 172.21.15.27:0/3797496395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f140f5bb0 0x7f7f140f5fd0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:40.806 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.807+0000 7f7f1b60c700 1 -- 172.21.15.27:0/3797496395 >> 172.21.15.27:0/3797496395 conn(0x7f7f140f04c0 msgr2=0x7f7f140f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:21:40.806 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.807+0000 7f7f1b60c700 1 -- 172.21.15.27:0/3797496395 shutdown_connections 2021-11-03T17:21:40.806 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.807+0000 7f7f1b60c700 1 -- 172.21.15.27:0/3797496395 wait complete. 2021-11-03T17:21:40.807 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.808+0000 7f7f1b60c700 1 Processor -- start 2021-11-03T17:21:40.807 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.808+0000 7f7f1b60c700 1 -- start start 2021-11-03T17:21:40.807 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.808+0000 7f7f1b60c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f140f5bb0 0x7f7f14105e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:21:40.807 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.808+0000 7f7f1b60c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f140f6550 0x7f7f14106380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:21:40.807 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.808+0000 7f7f1b60c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f14106930 con 0x7f7f140f5bb0 2021-11-03T17:21:40.808 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.808+0000 7f7f1b60c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f14106a70 con 0x7f7f140f6550 2021-11-03T17:21:40.808 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.808+0000 7f7f193a8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f140f5bb0 0x7f7f14105e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:21:40.808 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.808+0000 7f7f193a8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f140f5bb0 0x7f7f14105e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:56988/0 (socket says 172.21.15.27:56988) 2021-11-03T17:21:40.808 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.808+0000 7f7f193a8700 1 -- 172.21.15.27:0/2825557621 learned_addr learned my addr 172.21.15.27:0/2825557621 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:21:40.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.808+0000 7f7f18ba7700 1 --2- 172.21.15.27:0/2825557621 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f140f6550 0x7f7f14106380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:21:40.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.808+0000 7f7f193a8700 1 -- 172.21.15.27:0/2825557621 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f140f6550 msgr2=0x7f7f14106380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:21:40.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.808+0000 7f7f193a8700 1 --2- 172.21.15.27:0/2825557621 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f140f6550 0x7f7f14106380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:40.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.808+0000 7f7f193a8700 1 -- 172.21.15.27:0/2825557621 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f04005040 con 0x7f7f140f5bb0 2021-11-03T17:21:40.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.808+0000 7f7f18ba7700 1 --2- 172.21.15.27:0/2825557621 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f140f6550 0x7f7f14106380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:21:40.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.808+0000 7f7f193a8700 1 --2- 172.21.15.27:0/2825557621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f140f5bb0 0x7f7f14105e90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f7f04004f60 tx=0x7f7f04009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:21:40.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.808+0000 7f7f09ffb700 1 -- 172.21.15.27:0/2825557621 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f04018070 con 0x7f7f140f5bb0 2021-11-03T17:21:40.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.809+0000 7f7f09ffb700 1 -- 172.21.15.27:0/2825557621 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f7f04007ba0 con 0x7f7f140f5bb0 2021-11-03T17:21:40.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.809+0000 7f7f1b60c700 1 -- 172.21.15.27:0/2825557621 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f141073b0 con 0x7f7f140f5bb0 2021-11-03T17:21:40.811 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.809+0000 7f7f09ffb700 1 -- 172.21.15.27:0/2825557621 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f0401c620 con 0x7f7f140f5bb0 2021-11-03T17:21:40.811 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.809+0000 7f7f1b60c700 1 -- 172.21.15.27:0/2825557621 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f14107910 con 0x7f7f140f5bb0 2021-11-03T17:21:40.812 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.810+0000 7f7f1b60c700 1 -- 172.21.15.27:0/2825557621 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f14045ad0 con 0x7f7f140f5bb0 2021-11-03T17:21:40.812 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.810+0000 7f7f09ffb700 1 -- 172.21.15.27:0/2825557621 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f7f0401c8e0 con 0x7f7f140f5bb0 2021-11-03T17:21:40.813 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.810+0000 7f7f09ffb700 1 --2- 172.21.15.27:0/2825557621 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7f000640b0 0x7f7f00066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:21:40.813 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.811+0000 7f7f09ffb700 1 -- 172.21.15.27:0/2825557621 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f7f040826e0 con 0x7f7f140f5bb0 2021-11-03T17:21:40.813 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.811+0000 7f7f18ba7700 1 --2- 172.21.15.27:0/2825557621 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7f000640b0 0x7f7f00066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:21:40.813 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.811+0000 7f7f18ba7700 1 --2- 172.21.15.27:0/2825557621 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7f000640b0 0x7f7f00066570 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f7f10004a70 tx=0x7f7f10009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:21:40.813 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.815+0000 7f7f09ffb700 1 -- 172.21.15.27:0/2825557621 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f7f04054a40 con 0x7f7f140f5bb0 2021-11-03T17:21:40.985 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:40.990+0000 7f7f1b60c700 1 -- 172.21.15.27:0/2825557621 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f140fde10 con 0x7f7f000640b0 2021-11-03T17:21:41.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.004+0000 7f7f09ffb700 1 -- 172.21.15.27:0/2825557621 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f7f140fde10 con 0x7f7f000640b0 2021-11-03T17:21:41.001 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:21:41.001 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (13m) 3m ago 14m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:21:41.001 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (4m) 3m ago 14m 7205k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:21:41.002 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (4m) 2m ago 13m 7239k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:21:41.002 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (13m) 3m ago 14m 38.9M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:21:41.002 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (6m) 3m ago 9m 15.4M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:21:41.002 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (9m) 3m ago 9m 3836M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:21:41.003 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (5m) 2m ago 9m 14.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:21:41.003 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (7m) 2m ago 9m 16.3M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:21:41.003 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (6m) 3m ago 15m 470M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:21:41.003 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (6m) 2m ago 13m 412M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:21:41.004 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (5m) 3m ago 15m 81.2M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:21:41.004 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (5m) 2m ago 13m 69.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:21:41.004 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (14m) 3m ago 14m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:21:41.004 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (13m) 2m ago 13m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:21:41.005 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (4m) 3m ago 12m 243M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:21:41.005 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (4m) 3m ago 12m 270M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:21:41.005 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (3m) 3m ago 12m 11.8M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:21:41.006 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (3m) 2m ago 11m 289M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:21:41.006 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (2m) 2m ago 11m 235M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:21:41.006 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (2m) 2m ago 11m 11.5M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:21:41.006 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (12m) 3m ago 13m 45.7M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:21:41.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.006+0000 7f7f1b60c700 1 -- 172.21.15.27:0/2825557621 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7f000640b0 msgr2=0x7f7f00066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:21:41.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.006+0000 7f7f1b60c700 1 --2- 172.21.15.27:0/2825557621 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7f000640b0 0x7f7f00066570 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f7f10004a70 tx=0x7f7f10009040).stop 2021-11-03T17:21:41.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.006+0000 7f7f1b60c700 1 -- 172.21.15.27:0/2825557621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f140f5bb0 msgr2=0x7f7f14105e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:21:41.009 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.006+0000 7f7f1b60c700 1 --2- 172.21.15.27:0/2825557621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f140f5bb0 0x7f7f14105e90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f7f04004f60 tx=0x7f7f04009b30).stop 2021-11-03T17:21:41.009 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.006+0000 7f7f1b60c700 1 -- 172.21.15.27:0/2825557621 shutdown_connections 2021-11-03T17:21:41.010 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.006+0000 7f7f1b60c700 1 --2- 172.21.15.27:0/2825557621 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7f000640b0 0x7f7f00066570 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:41.010 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.006+0000 7f7f1b60c700 1 --2- 172.21.15.27:0/2825557621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f140f5bb0 0x7f7f14105e90 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:41.010 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.006+0000 7f7f1b60c700 1 --2- 172.21.15.27:0/2825557621 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f140f6550 0x7f7f14106380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:41.010 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.006+0000 7f7f1b60c700 1 -- 172.21.15.27:0/2825557621 >> 172.21.15.27:0/2825557621 conn(0x7f7f140f04c0 msgr2=0x7f7f140fa450 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:21:41.011 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.007+0000 7f7f1b60c700 1 -- 172.21.15.27:0/2825557621 shutdown_connections 2021-11-03T17:21:41.011 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.007+0000 7f7f1b60c700 1 -- 172.21.15.27:0/2825557621 wait complete. 2021-11-03T17:21:41.104 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.107+0000 7feeb80be700 1 -- 172.21.15.27:0/1236682325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feeb00f5ef0 msgr2=0x7feeb00f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:21:41.104 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.107+0000 7feeb80be700 1 --2- 172.21.15.27:0/1236682325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feeb00f5ef0 0x7feeb00f6920 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7feea0004660 tx=0x7feea0009b30).stop 2021-11-03T17:21:41.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.107+0000 7feeb80be700 1 -- 172.21.15.27:0/1236682325 shutdown_connections 2021-11-03T17:21:41.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.107+0000 7feeb80be700 1 --2- 172.21.15.27:0/1236682325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feeb00f5ef0 0x7feeb00f6920 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:41.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.107+0000 7feeb80be700 1 --2- 172.21.15.27:0/1236682325 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feeb00f55e0 0x7feeb00f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:41.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.107+0000 7feeb80be700 1 -- 172.21.15.27:0/1236682325 >> 172.21.15.27:0/1236682325 conn(0x7feeb00f0de0 msgr2=0x7feeb00f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:21:41.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.107+0000 7feeb80be700 1 -- 172.21.15.27:0/1236682325 shutdown_connections 2021-11-03T17:21:41.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.107+0000 7feeb80be700 1 -- 172.21.15.27:0/1236682325 wait complete. 2021-11-03T17:21:41.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.107+0000 7feeb80be700 1 Processor -- start 2021-11-03T17:21:41.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.107+0000 7feeb80be700 1 -- start start 2021-11-03T17:21:41.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.107+0000 7feeb80be700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feeb00f55e0 0x7feeb0106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:21:41.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.107+0000 7feeb80be700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feeb00f5ef0 0x7feeb0107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:21:41.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.107+0000 7feeb80be700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb0107a10 con 0x7feeb00f5ef0 2021-11-03T17:21:41.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.107+0000 7feeb80be700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeb0107b50 con 0x7feeb00f55e0 2021-11-03T17:21:41.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.108+0000 7feeb5659700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feeb00f5ef0 0x7feeb0107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:21:41.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.108+0000 7feeb5659700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feeb00f5ef0 0x7feeb0107460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57000/0 (socket says 172.21.15.27:57000) 2021-11-03T17:21:41.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.108+0000 7feeb5659700 1 -- 172.21.15.27:0/2267857172 learned_addr learned my addr 172.21.15.27:0/2267857172 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:21:41.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.108+0000 7feeb5e5a700 1 --2- 172.21.15.27:0/2267857172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feeb00f55e0 0x7feeb0106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:21:41.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.108+0000 7feeb5659700 1 -- 172.21.15.27:0/2267857172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feeb00f55e0 msgr2=0x7feeb0106f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:21:41.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.108+0000 7feeb5659700 1 --2- 172.21.15.27:0/2267857172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feeb00f55e0 0x7feeb0106f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:41.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.108+0000 7feeb5659700 1 -- 172.21.15.27:0/2267857172 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea0005040 con 0x7feeb00f5ef0 2021-11-03T17:21:41.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.108+0000 7feeb5659700 1 --2- 172.21.15.27:0/2267857172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feeb00f5ef0 0x7feeb0107460 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7feea0009990 tx=0x7feea0009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:21:41.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.108+0000 7feea67fc700 1 -- 172.21.15.27:0/2267857172 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feea0018070 con 0x7feeb00f5ef0 2021-11-03T17:21:41.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.108+0000 7feeb80be700 1 -- 172.21.15.27:0/2267857172 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeb0108490 con 0x7feeb00f5ef0 2021-11-03T17:21:41.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.108+0000 7feea67fc700 1 -- 172.21.15.27:0/2267857172 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7feea0007ba0 con 0x7feeb00f5ef0 2021-11-03T17:21:41.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.108+0000 7feea67fc700 1 -- 172.21.15.27:0/2267857172 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feea001c3c0 con 0x7feeb00f5ef0 2021-11-03T17:21:41.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.108+0000 7feeb80be700 1 -- 172.21.15.27:0/2267857172 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feeb01089f0 con 0x7feeb00f5ef0 2021-11-03T17:21:41.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.109+0000 7feeb80be700 1 -- 172.21.15.27:0/2267857172 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feeb0045ad0 con 0x7feeb00f5ef0 2021-11-03T17:21:41.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.110+0000 7feea67fc700 1 -- 172.21.15.27:0/2267857172 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7feea001d420 con 0x7feeb00f5ef0 2021-11-03T17:21:41.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.110+0000 7feea67fc700 1 --2- 172.21.15.27:0/2267857172 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fee9c0640b0 0x7fee9c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:21:41.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.110+0000 7feea67fc700 1 -- 172.21.15.27:0/2267857172 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6461+0+0 (secure 0 0 0) 0x7feea0082410 con 0x7feeb00f5ef0 2021-11-03T17:21:41.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.110+0000 7feeb5e5a700 1 --2- 172.21.15.27:0/2267857172 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fee9c0640b0 0x7fee9c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:21:41.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.111+0000 7feeb5e5a700 1 --2- 172.21.15.27:0/2267857172 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fee9c0640b0 0x7fee9c066570 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7feea8007db0 tx=0x7feea8009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:21:41.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.114+0000 7feea67fc700 1 -- 172.21.15.27:0/2267857172 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7feea0054710 con 0x7feeb00f5ef0 2021-11-03T17:21:41.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:40 smithi115 ceph-mon[65453]: pgmap v209: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:41.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:40 smithi027 ceph-mon[97244]: pgmap v209: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:41.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.343+0000 7feeb80be700 1 -- 172.21.15.27:0/2267857172 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7feeb01090f0 con 0x7feeb00f5ef0 2021-11-03T17:21:41.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.343+0000 7feea67fc700 1 -- 172.21.15.27:0/2267857172 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7feea0054300 con 0x7feeb00f5ef0 2021-11-03T17:21:41.339 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:21:41.340 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:21:41.340 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:21:41.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:21:41.341 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:21:41.341 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:21:41.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:21:41.341 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:21:41.342 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:21:41.342 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:21:41.342 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:21:41.342 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:21:41.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:21:41.343 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:21:41.343 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:21:41.343 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:21:41.343 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:21:41.344 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:21:41.344 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.345+0000 7feeb80be700 1 -- 172.21.15.27:0/2267857172 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fee9c0640b0 msgr2=0x7fee9c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:21:41.345 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.345+0000 7feeb80be700 1 --2- 172.21.15.27:0/2267857172 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fee9c0640b0 0x7fee9c066570 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7feea8007db0 tx=0x7feea8009040).stop 2021-11-03T17:21:41.345 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.345+0000 7feeb80be700 1 -- 172.21.15.27:0/2267857172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feeb00f5ef0 msgr2=0x7feeb0107460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:21:41.345 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.345+0000 7feeb80be700 1 --2- 172.21.15.27:0/2267857172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feeb00f5ef0 0x7feeb0107460 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7feea0009990 tx=0x7feea0009b30).stop 2021-11-03T17:21:41.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.345+0000 7feeb80be700 1 -- 172.21.15.27:0/2267857172 shutdown_connections 2021-11-03T17:21:41.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.345+0000 7feeb80be700 1 --2- 172.21.15.27:0/2267857172 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fee9c0640b0 0x7fee9c066570 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:41.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.345+0000 7feeb80be700 1 --2- 172.21.15.27:0/2267857172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feeb00f55e0 0x7feeb0106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:41.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.345+0000 7feeb80be700 1 --2- 172.21.15.27:0/2267857172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feeb00f5ef0 0x7feeb0107460 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:41.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.345+0000 7feeb80be700 1 -- 172.21.15.27:0/2267857172 >> 172.21.15.27:0/2267857172 conn(0x7feeb00f0de0 msgr2=0x7feeb005f720 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:21:41.347 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.345+0000 7feeb80be700 1 -- 172.21.15.27:0/2267857172 shutdown_connections 2021-11-03T17:21:41.347 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.345+0000 7feeb80be700 1 -- 172.21.15.27:0/2267857172 wait complete. 2021-11-03T17:21:41.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.446+0000 7fc30644b700 1 -- 172.21.15.27:0/3214205074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3000f66e0 msgr2=0x7fc3000f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:21:41.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.446+0000 7fc30644b700 1 --2- 172.21.15.27:0/3214205074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3000f66e0 0x7fc3000f6b00 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7fc2f0004660 tx=0x7fc2f0009b30).stop 2021-11-03T17:21:41.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.446+0000 7fc30644b700 1 -- 172.21.15.27:0/3214205074 shutdown_connections 2021-11-03T17:21:41.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.446+0000 7fc30644b700 1 --2- 172.21.15.27:0/3214205074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3000f6ff0 0x7fc3000f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:41.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.446+0000 7fc30644b700 1 --2- 172.21.15.27:0/3214205074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3000f66e0 0x7fc3000f6b00 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:41.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.446+0000 7fc30644b700 1 -- 172.21.15.27:0/3214205074 >> 172.21.15.27:0/3214205074 conn(0x7fc3000f1f00 msgr2=0x7fc3000f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:21:41.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.446+0000 7fc30644b700 1 -- 172.21.15.27:0/3214205074 shutdown_connections 2021-11-03T17:21:41.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.446+0000 7fc30644b700 1 -- 172.21.15.27:0/3214205074 wait complete. 2021-11-03T17:21:41.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.447+0000 7fc30644b700 1 Processor -- start 2021-11-03T17:21:41.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.447+0000 7fc30644b700 1 -- start start 2021-11-03T17:21:41.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.447+0000 7fc30644b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3000f66e0 0x7fc300106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:21:41.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.447+0000 7fc30644b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3000f6ff0 0x7fc300107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:21:41.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.447+0000 7fc30644b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc300107a00 con 0x7fc3000f66e0 2021-11-03T17:21:41.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.447+0000 7fc30644b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc300107b40 con 0x7fc3000f6ff0 2021-11-03T17:21:41.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.447+0000 7fc2fffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3000f66e0 0x7fc300106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:21:41.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.447+0000 7fc2fffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3000f66e0 0x7fc300106f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57008/0 (socket says 172.21.15.27:57008) 2021-11-03T17:21:41.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.447+0000 7fc2fffff700 1 -- 172.21.15.27:0/2687454155 learned_addr learned my addr 172.21.15.27:0/2687454155 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:21:41.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.447+0000 7fc2ff7fe700 1 --2- 172.21.15.27:0/2687454155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3000f6ff0 0x7fc300107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:21:41.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.447+0000 7fc2fffff700 1 -- 172.21.15.27:0/2687454155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3000f6ff0 msgr2=0x7fc300107450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:21:41.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.447+0000 7fc2fffff700 1 --2- 172.21.15.27:0/2687454155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3000f6ff0 0x7fc300107450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:41.457 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.447+0000 7fc2fffff700 1 -- 172.21.15.27:0/2687454155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f0005040 con 0x7fc3000f66e0 2021-11-03T17:21:41.457 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.447+0000 7fc2ff7fe700 1 --2- 172.21.15.27:0/2687454155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3000f6ff0 0x7fc300107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:21:41.458 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.448+0000 7fc2fffff700 1 --2- 172.21.15.27:0/2687454155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3000f66e0 0x7fc300106f60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fc2f0009990 tx=0x7fc2f0009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:21:41.458 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.448+0000 7fc2fcff9700 1 -- 172.21.15.27:0/2687454155 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2f0018070 con 0x7fc3000f66e0 2021-11-03T17:21:41.459 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.448+0000 7fc2fcff9700 1 -- 172.21.15.27:0/2687454155 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fc2f0007ba0 con 0x7fc3000f66e0 2021-11-03T17:21:41.459 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.448+0000 7fc30644b700 1 -- 172.21.15.27:0/2687454155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc300108480 con 0x7fc3000f66e0 2021-11-03T17:21:41.460 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.448+0000 7fc2fcff9700 1 -- 172.21.15.27:0/2687454155 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc2f001c3c0 con 0x7fc3000f66e0 2021-11-03T17:21:41.460 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.448+0000 7fc30644b700 1 -- 172.21.15.27:0/2687454155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3001089e0 con 0x7fc3000f66e0 2021-11-03T17:21:41.460 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.449+0000 7fc30644b700 1 -- 172.21.15.27:0/2687454155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc300045ad0 con 0x7fc3000f66e0 2021-11-03T17:21:41.461 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.450+0000 7fc2fcff9700 1 -- 172.21.15.27:0/2687454155 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fc2f001d420 con 0x7fc3000f66e0 2021-11-03T17:21:41.461 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.450+0000 7fc2fcff9700 1 --2- 172.21.15.27:0/2687454155 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc2ec0640b0 0x7fc2ec066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:21:41.461 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.450+0000 7fc2fcff9700 1 -- 172.21.15.27:0/2687454155 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fc2f00826e0 con 0x7fc3000f66e0 2021-11-03T17:21:41.462 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.450+0000 7fc2ff7fe700 1 --2- 172.21.15.27:0/2687454155 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc2ec0640b0 0x7fc2ec066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:21:41.462 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.451+0000 7fc2ff7fe700 1 --2- 172.21.15.27:0/2687454155 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc2ec0640b0 0x7fc2ec066570 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7fc2e8009740 tx=0x7fc2e8009be0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:21:41.463 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.454+0000 7fc2fcff9700 1 -- 172.21.15.27:0/2687454155 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fc2f00549a0 con 0x7fc3000f66e0 2021-11-03T17:21:41.655 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.660+0000 7fc30644b700 1 -- 172.21.15.27:0/2687454155 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc3000ff050 con 0x7fc3000f66e0 2021-11-03T17:21:41.656 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.660+0000 7fc2fcff9700 1 -- 172.21.15.27:0/2687454155 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7fc2f0054590 con 0x7fc3000f66e0 2021-11-03T17:21:41.656 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:21:41.656 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:21:41.657 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:21:41.657 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:21:41.657 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:21:41.657 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:21:41.657 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:21:41.658 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:21:41.658 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:21:41.658 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:21:41.658 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:21:41.659 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:21:41.659 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:21:41.659 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:21:41.660 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:21:41.660 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:21:41.660 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:21:41.660 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:21:41.660 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:21:41.661 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:21:41.661 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:21:41.661 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:21:41.661 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:21:41.662 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:21:41.662 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:21:41.662 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:21:41.662 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:21:41.663 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:21:41.663 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:21:41.663 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:21:41.663 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:21:41.664 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:21:41.664 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:21:41.664 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:21:41.664 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:21:41.665 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:21:41.665 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:21:41.665 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:21:41.665 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:21:41.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.662+0000 7fc30644b700 1 -- 172.21.15.27:0/2687454155 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc2ec0640b0 msgr2=0x7fc2ec066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:21:41.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.662+0000 7fc30644b700 1 --2- 172.21.15.27:0/2687454155 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc2ec0640b0 0x7fc2ec066570 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7fc2e8009740 tx=0x7fc2e8009be0).stop 2021-11-03T17:21:41.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.662+0000 7fc30644b700 1 -- 172.21.15.27:0/2687454155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3000f66e0 msgr2=0x7fc300106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:21:41.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.662+0000 7fc30644b700 1 --2- 172.21.15.27:0/2687454155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3000f66e0 0x7fc300106f60 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7fc2f0009990 tx=0x7fc2f0009b30).stop 2021-11-03T17:21:41.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.662+0000 7fc30644b700 1 -- 172.21.15.27:0/2687454155 shutdown_connections 2021-11-03T17:21:41.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.662+0000 7fc30644b700 1 --2- 172.21.15.27:0/2687454155 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc2ec0640b0 0x7fc2ec066570 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:41.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.662+0000 7fc30644b700 1 --2- 172.21.15.27:0/2687454155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3000f66e0 0x7fc300106f60 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:41.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.662+0000 7fc30644b700 1 --2- 172.21.15.27:0/2687454155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3000f6ff0 0x7fc300107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:21:41.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.662+0000 7fc30644b700 1 -- 172.21.15.27:0/2687454155 >> 172.21.15.27:0/2687454155 conn(0x7fc3000f1f00 msgr2=0x7fc3000fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:21:41.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.662+0000 7fc30644b700 1 -- 172.21.15.27:0/2687454155 shutdown_connections 2021-11-03T17:21:41.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:21:41.662+0000 7fc30644b700 1 -- 172.21.15.27:0/2687454155 wait complete. 2021-11-03T17:21:41.669 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:21:42.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:41 smithi115 ceph-mon[65453]: from='client.34328 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:21:42.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:41 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2267857172' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:42.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:41 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2687454155' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:21:42.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:41 smithi027 ceph-mon[97244]: from='client.34328 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:21:42.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:41 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2267857172' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:42.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:41 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2687454155' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:21:43.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:42 smithi115 ceph-mon[65453]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:21:43.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:42 smithi115 ceph-mon[65453]: pgmap v210: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:43.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:42 smithi027 ceph-mon[97244]: from='client.34332 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:21:43.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:42 smithi027 ceph-mon[97244]: pgmap v210: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:45.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:44 smithi115 ceph-mon[65453]: pgmap v211: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:45.289 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:44 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 347.367172 secs 2021-11-03T17:21:45.290 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:44 smithi027 ceph-mon[97244]: pgmap v211: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:45.291 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:44 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 347.367172 secs 2021-11-03T17:21:46.184 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:21:46.194 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:21:47.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:46 smithi115 ceph-mon[65453]: poll took 60170.363 ms: 1 events 2021-11-03T17:21:47.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:46 smithi115 ceph-mon[65453]: pgmap v212: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:47.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:21:47.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:21:47.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:47.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:47.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:47.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:47.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:46 smithi027 ceph-mon[97244]: poll took 60170.363 ms: 1 events 2021-11-03T17:21:47.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:46 smithi027 ceph-mon[97244]: pgmap v212: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:47.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:21:47.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:21:47.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:47.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:47.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:47.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:48.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:47 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:21:48.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:47 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:21:49.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:48 smithi115 ceph-mon[65453]: pgmap v213: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:49.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:48 smithi027 ceph-mon[97244]: pgmap v213: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:50.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:49 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 352.367318 secs 2021-11-03T17:21:50.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:49 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 352.367318 secs 2021-11-03T17:21:51.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:50 smithi115 ceph-mon[65453]: pgmap v214: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:51.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:50 smithi027 ceph-mon[97244]: pgmap v214: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:53.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:52 smithi115 ceph-mon[65453]: pgmap v215: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:53.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:52 smithi027 ceph-mon[97244]: pgmap v215: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:55.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:54 smithi115 ceph-mon[65453]: pgmap v216: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:55.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:54 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 357.367466 secs 2021-11-03T17:21:55.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:54 smithi027 ceph-mon[97244]: pgmap v216: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:55.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:54 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 357.367466 secs 2021-11-03T17:21:56.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:21:56.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:21:56.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:21:56.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:21:57.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:56 smithi115 ceph-mon[65453]: pgmap v217: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:57.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:21:57.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:21:57.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:21:57.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:57.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:57.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:57.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:57.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:56 smithi027 ceph-mon[97244]: pgmap v217: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:57.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:21:57.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:21:57.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:21:57.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:57.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:57.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:57.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:21:58.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:57 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:21:58.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:57 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:21:59.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:58 smithi115 ceph-mon[65453]: pgmap v218: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:21:59.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:58 smithi027 ceph-mon[97244]: pgmap v218: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:00.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:59 smithi115 ceph-mon[65453]: 1 slow requests, 1 included below; oldest blocked for > 362.367598 secs 2021-11-03T17:22:00.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:21:59 smithi115 ceph-mon[65453]: slow request 362.367597 seconds old, received at 2021-11-03T17:15:56.956064+0000: client_request(client.24331:21043 rmdir #0x10000003957/db1 2021-11-03T17:15:56.954824+0000 caller_uid=1000, caller_gid=1254{6,36,1000,1254,}) currently failed to xlock, waiting 2021-11-03T17:22:00.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:59 smithi027 ceph-mon[97244]: 1 slow requests, 1 included below; oldest blocked for > 362.367598 secs 2021-11-03T17:22:00.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:21:59 smithi027 ceph-mon[97244]: slow request 362.367597 seconds old, received at 2021-11-03T17:15:56.956064+0000: client_request(client.24331:21043 rmdir #0x10000003957/db1 2021-11-03T17:15:56.954824+0000 caller_uid=1000, caller_gid=1254{6,36,1000,1254,}) currently failed to xlock, waiting 2021-11-03T17:22:01.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:00 smithi115 ceph-mon[65453]: pgmap v219: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:01.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:00 smithi027 ceph-mon[97244]: pgmap v219: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:03.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:02 smithi115 ceph-mon[65453]: pgmap v220: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:03.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:02 smithi027 ceph-mon[97244]: pgmap v220: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:05.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:04 smithi115 ceph-mon[65453]: pgmap v221: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:05.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:04 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 367.367739 secs 2021-11-03T17:22:05.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:04 smithi027 ceph-mon[97244]: pgmap v221: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:05.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:04 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 367.367739 secs 2021-11-03T17:22:07.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:06 smithi115 ceph-mon[65453]: pgmap v222: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:07.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:22:07.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:22:07.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:22:07.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:07.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:07.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:07.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:07.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:06 smithi027 ceph-mon[97244]: pgmap v222: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:07.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:22:07.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:22:07.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:22:07.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:07.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:07.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:07.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:08.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:07 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:22:08.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:07 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:22:09.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:08 smithi115 ceph-mon[65453]: pgmap v223: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:09.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:08 smithi027 ceph-mon[97244]: pgmap v223: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:10.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:09 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 372.367878 secs 2021-11-03T17:22:10.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:09 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 372.367878 secs 2021-11-03T17:22:11.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:10 smithi115 ceph-mon[65453]: pgmap v224: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:11.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:10 smithi027 ceph-mon[97244]: pgmap v224: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:11.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.780+0000 7fdb026ea700 1 -- 172.21.15.27:0/901064679 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdafc0f6530 msgr2=0x7fdafc0f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:22:11.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.780+0000 7fdb026ea700 1 --2- 172.21.15.27:0/901064679 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdafc0f6530 0x7fdafc0f6950 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7fdaf8004660 tx=0x7fdaf8009b30).stop 2021-11-03T17:22:11.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.780+0000 7fdb026ea700 1 -- 172.21.15.27:0/901064679 shutdown_connections 2021-11-03T17:22:11.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.780+0000 7fdb026ea700 1 --2- 172.21.15.27:0/901064679 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdafc0f77d0 0x7fdafc0f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:11.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.780+0000 7fdb026ea700 1 --2- 172.21.15.27:0/901064679 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdafc0f6530 0x7fdafc0f6950 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:11.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.780+0000 7fdb026ea700 1 -- 172.21.15.27:0/901064679 >> 172.21.15.27:0/901064679 conn(0x7fdafc0f1cd0 msgr2=0x7fdafc0f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:22:11.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.780+0000 7fdb026ea700 1 -- 172.21.15.27:0/901064679 shutdown_connections 2021-11-03T17:22:11.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.780+0000 7fdb026ea700 1 -- 172.21.15.27:0/901064679 wait complete. 2021-11-03T17:22:11.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.780+0000 7fdb026ea700 1 Processor -- start 2021-11-03T17:22:11.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.780+0000 7fdb026ea700 1 -- start start 2021-11-03T17:22:11.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.781+0000 7fdb026ea700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdafc0f6530 0x7fdafc106db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:22:11.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.781+0000 7fdb026ea700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdafc0f77d0 0x7fdafc1072a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:22:11.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.781+0000 7fdb026ea700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdafc1078a0 con 0x7fdafc0f77d0 2021-11-03T17:22:11.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.781+0000 7fdb026ea700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdafc1080e0 con 0x7fdafc0f6530 2021-11-03T17:22:11.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.781+0000 7fdb00ee7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdafc0f77d0 0x7fdafc1072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:22:11.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.781+0000 7fdb00ee7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdafc0f77d0 0x7fdafc1072a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57028/0 (socket says 172.21.15.27:57028) 2021-11-03T17:22:11.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.781+0000 7fdb00ee7700 1 -- 172.21.15.27:0/3924619195 learned_addr learned my addr 172.21.15.27:0/3924619195 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:22:11.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.781+0000 7fdb016e8700 1 --2- 172.21.15.27:0/3924619195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdafc0f6530 0x7fdafc106db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:22:11.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.781+0000 7fdb00ee7700 1 -- 172.21.15.27:0/3924619195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdafc0f6530 msgr2=0x7fdafc106db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:22:11.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.781+0000 7fdb00ee7700 1 --2- 172.21.15.27:0/3924619195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdafc0f6530 0x7fdafc106db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:11.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.781+0000 7fdb00ee7700 1 -- 172.21.15.27:0/3924619195 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf8005040 con 0x7fdafc0f77d0 2021-11-03T17:22:11.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.781+0000 7fdb00ee7700 1 --2- 172.21.15.27:0/3924619195 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdafc0f77d0 0x7fdafc1072a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7fdaec00c790 tx=0x7fdaec007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:22:11.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.781+0000 7fdaf1ffb700 1 -- 172.21.15.27:0/3924619195 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdaec002590 con 0x7fdafc0f77d0 2021-11-03T17:22:11.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.781+0000 7fdb026ea700 1 -- 172.21.15.27:0/3924619195 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdafc1083c0 con 0x7fdafc0f77d0 2021-11-03T17:22:11.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.781+0000 7fdaf1ffb700 1 -- 172.21.15.27:0/3924619195 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fdaec002ba0 con 0x7fdafc0f77d0 2021-11-03T17:22:11.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.781+0000 7fdaf1ffb700 1 -- 172.21.15.27:0/3924619195 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdaec013d90 con 0x7fdafc0f77d0 2021-11-03T17:22:11.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.782+0000 7fdb026ea700 1 -- 172.21.15.27:0/3924619195 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdafc108900 con 0x7fdafc0f77d0 2021-11-03T17:22:11.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.783+0000 7fdb026ea700 1 -- 172.21.15.27:0/3924619195 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdafc059100 con 0x7fdafc0f77d0 2021-11-03T17:22:11.787 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.783+0000 7fdaf1ffb700 1 -- 172.21.15.27:0/3924619195 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fdaec01a060 con 0x7fdafc0f77d0 2021-11-03T17:22:11.787 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.783+0000 7fdaf1ffb700 1 --2- 172.21.15.27:0/3924619195 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdae8064100 0x7fdae80665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:22:11.787 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.783+0000 7fdaf1ffb700 1 -- 172.21.15.27:0/3924619195 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fdaec080b30 con 0x7fdafc0f77d0 2021-11-03T17:22:11.788 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.784+0000 7fdb016e8700 1 --2- 172.21.15.27:0/3924619195 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdae8064100 0x7fdae80665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:22:11.788 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.784+0000 7fdb016e8700 1 --2- 172.21.15.27:0/3924619195 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdae8064100 0x7fdae80665c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fdaf8013690 tx=0x7fdaf8009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:22:11.789 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.788+0000 7fdaf1ffb700 1 -- 172.21.15.27:0/3924619195 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fdaec052d40 con 0x7fdafc0f77d0 2021-11-03T17:22:11.964 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.969+0000 7fdb026ea700 1 -- 172.21.15.27:0/3924619195 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdafc0fede0 con 0x7fdae8064100 2021-11-03T17:22:11.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.970+0000 7fdaf1ffb700 1 -- 172.21.15.27:0/3924619195 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fdafc0fede0 con 0x7fdae8064100 2021-11-03T17:22:11.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.972+0000 7fdb026ea700 1 -- 172.21.15.27:0/3924619195 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdae8064100 msgr2=0x7fdae80665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:22:11.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.972+0000 7fdb026ea700 1 --2- 172.21.15.27:0/3924619195 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdae8064100 0x7fdae80665c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7fdaf8013690 tx=0x7fdaf8009b30).stop 2021-11-03T17:22:11.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.972+0000 7fdb026ea700 1 -- 172.21.15.27:0/3924619195 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdafc0f77d0 msgr2=0x7fdafc1072a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:22:11.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.972+0000 7fdb026ea700 1 --2- 172.21.15.27:0/3924619195 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdafc0f77d0 0x7fdafc1072a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7fdaec00c790 tx=0x7fdaec007d70).stop 2021-11-03T17:22:11.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.972+0000 7fdb026ea700 1 -- 172.21.15.27:0/3924619195 shutdown_connections 2021-11-03T17:22:11.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.972+0000 7fdb026ea700 1 --2- 172.21.15.27:0/3924619195 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdae8064100 0x7fdae80665c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:11.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.972+0000 7fdb026ea700 1 --2- 172.21.15.27:0/3924619195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdafc0f6530 0x7fdafc106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:11.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.972+0000 7fdb026ea700 1 --2- 172.21.15.27:0/3924619195 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdafc0f77d0 0x7fdafc1072a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:11.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.972+0000 7fdb026ea700 1 -- 172.21.15.27:0/3924619195 >> 172.21.15.27:0/3924619195 conn(0x7fdafc0f1cd0 msgr2=0x7fdafc0fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:22:11.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.972+0000 7fdb026ea700 1 -- 172.21.15.27:0/3924619195 shutdown_connections 2021-11-03T17:22:11.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:11.972+0000 7fdb026ea700 1 -- 172.21.15.27:0/3924619195 wait complete. 2021-11-03T17:22:11.978 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:22:12.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.074+0000 7f298e76c700 1 -- 172.21.15.27:0/3442362831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29880f55c0 msgr2=0x7f29880f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:22:12.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.074+0000 7f298e76c700 1 --2- 172.21.15.27:0/3442362831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29880f55c0 0x7f29880f59e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f2978004660 tx=0x7f2978009b30).stop 2021-11-03T17:22:12.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.074+0000 7f298e76c700 1 -- 172.21.15.27:0/3442362831 shutdown_connections 2021-11-03T17:22:12.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.074+0000 7f298e76c700 1 --2- 172.21.15.27:0/3442362831 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29880f6860 0x7f29880f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:22:12.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.074+0000 7f298e76c700 1 --2- 172.21.15.27:0/3442362831 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29880f55c0 0x7f29880f59e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:12.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.074+0000 7f298e76c700 1 -- 172.21.15.27:0/3442362831 >> 172.21.15.27:0/3442362831 conn(0x7f29880f0d40 msgr2=0x7f29880f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:22:12.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.074+0000 7f298e76c700 1 -- 172.21.15.27:0/3442362831 shutdown_connections 2021-11-03T17:22:12.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.074+0000 7f298e76c700 1 -- 172.21.15.27:0/3442362831 wait complete. 2021-11-03T17:22:12.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.074+0000 7f298e76c700 1 Processor -- start 2021-11-03T17:22:12.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.074+0000 7f298e76c700 1 -- start start 2021-11-03T17:22:12.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.075+0000 7f298e76c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29880f55c0 0x7f2988109c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:22:12.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.075+0000 7f298e76c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29880f6860 0x7f298810c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:22:12.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.075+0000 7f298e76c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2988060470 con 0x7f29880f55c0 2021-11-03T17:22:12.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.075+0000 7f298e76c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29880605b0 con 0x7f29880f6860 2021-11-03T17:22:12.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.075+0000 7f2987fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29880f55c0 0x7f2988109c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:22:12.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.075+0000 7f2987fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29880f55c0 0x7f2988109c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57036/0 (socket says 172.21.15.27:57036) 2021-11-03T17:22:12.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.075+0000 7f2987fff700 1 -- 172.21.15.27:0/2163676597 learned_addr learned my addr 172.21.15.27:0/2163676597 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:22:12.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.075+0000 7f2987fff700 1 -- 172.21.15.27:0/2163676597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29880f6860 msgr2=0x7f298810c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:22:12.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.075+0000 7f29877fe700 1 --2- 172.21.15.27:0/2163676597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29880f6860 0x7f298810c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:22:12.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.075+0000 7f2987fff700 1 --2- 172.21.15.27:0/2163676597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29880f6860 0x7f298810c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:12.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.075+0000 7f2987fff700 1 -- 172.21.15.27:0/2163676597 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2978005040 con 0x7f29880f55c0 2021-11-03T17:22:12.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.075+0000 7f2987fff700 1 --2- 172.21.15.27:0/2163676597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29880f55c0 0x7f2988109c50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f2978004f60 tx=0x7f2978009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:22:12.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.075+0000 7f2984ff9700 1 -- 172.21.15.27:0/2163676597 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2978018070 con 0x7f29880f55c0 2021-11-03T17:22:12.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.075+0000 7f2984ff9700 1 -- 172.21.15.27:0/2163676597 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f2978007ba0 con 0x7f29880f55c0 2021-11-03T17:22:12.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.075+0000 7f298e76c700 1 -- 172.21.15.27:0/2163676597 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f298810c750 con 0x7f29880f55c0 2021-11-03T17:22:12.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.075+0000 7f2984ff9700 1 -- 172.21.15.27:0/2163676597 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f297801c620 con 0x7f29880f55c0 2021-11-03T17:22:12.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.075+0000 7f298e76c700 1 -- 172.21.15.27:0/2163676597 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f298810cc80 con 0x7f29880f55c0 2021-11-03T17:22:12.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.076+0000 7f298e76c700 1 -- 172.21.15.27:0/2163676597 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29880f49b0 con 0x7f29880f55c0 2021-11-03T17:22:12.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.077+0000 7f2984ff9700 1 -- 172.21.15.27:0/2163676597 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f297801d420 con 0x7f29880f55c0 2021-11-03T17:22:12.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.077+0000 7f2984ff9700 1 --2- 172.21.15.27:0/2163676597 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f29700640b0 0x7f2970066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:22:12.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.077+0000 7f2984ff9700 1 -- 172.21.15.27:0/2163676597 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f2978082620 con 0x7f29880f55c0 2021-11-03T17:22:12.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.077+0000 7f29877fe700 1 --2- 172.21.15.27:0/2163676597 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f29700640b0 0x7f2970066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:22:12.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.078+0000 7f29877fe700 1 --2- 172.21.15.27:0/2163676597 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f29700640b0 0x7f2970066570 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f297c00a6f0 tx=0x7f297c008040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:22:12.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.081+0000 7f2984ff9700 1 -- 172.21.15.27:0/2163676597 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f2978054a40 con 0x7f29880f55c0 2021-11-03T17:22:12.252 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.256+0000 7f298e76c700 1 -- 172.21.15.27:0/2163676597 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f29880fdee0 con 0x7f29700640b0 2021-11-03T17:22:12.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.271+0000 7f2984ff9700 1 -- 172.21.15.27:0/2163676597 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f29880fdee0 con 0x7f29700640b0 2021-11-03T17:22:12.267 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:22:12.267 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (13m) 3m ago 15m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:22:12.267 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (5m) 3m ago 14m 7205k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:22:12.268 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (5m) 2m ago 14m 7239k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:22:12.268 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (13m) 3m ago 14m 38.9M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:22:12.268 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (7m) 3m ago 10m 15.4M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:22:12.268 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (10m) 3m ago 10m 3836M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:22:12.268 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (5m) 2m ago 10m 14.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:22:12.269 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (7m) 2m ago 10m 16.3M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:22:12.269 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (7m) 3m ago 16m 470M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:22:12.269 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (6m) 2m ago 14m 412M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:22:12.269 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (6m) 3m ago 16m 81.2M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:22:12.270 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (6m) 2m ago 14m 69.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:22:12.270 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (14m) 3m ago 14m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:22:12.270 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (13m) 2m ago 13m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:22:12.270 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (5m) 3m ago 13m 243M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:22:12.270 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (4m) 3m ago 13m 270M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:22:12.271 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (4m) 3m ago 12m 11.8M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:22:12.271 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (3m) 2m ago 12m 289M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:22:12.271 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (3m) 2m ago 12m 235M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:22:12.271 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (2m) 2m ago 11m 11.5M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:22:12.272 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (13m) 3m ago 14m 45.7M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:22:12.272 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.273+0000 7f298e76c700 1 -- 172.21.15.27:0/2163676597 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f29700640b0 msgr2=0x7f2970066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:22:12.273 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.273+0000 7f298e76c700 1 --2- 172.21.15.27:0/2163676597 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f29700640b0 0x7f2970066570 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f297c00a6f0 tx=0x7f297c008040).stop 2021-11-03T17:22:12.273 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.273+0000 7f298e76c700 1 -- 172.21.15.27:0/2163676597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29880f55c0 msgr2=0x7f2988109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:22:12.273 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.273+0000 7f298e76c700 1 --2- 172.21.15.27:0/2163676597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29880f55c0 0x7f2988109c50 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f2978004f60 tx=0x7f2978009b30).stop 2021-11-03T17:22:12.274 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.273+0000 7f298e76c700 1 -- 172.21.15.27:0/2163676597 shutdown_connections 2021-11-03T17:22:12.274 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.273+0000 7f298e76c700 1 --2- 172.21.15.27:0/2163676597 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f29700640b0 0x7f2970066570 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:12.274 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.273+0000 7f298e76c700 1 --2- 172.21.15.27:0/2163676597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29880f55c0 0x7f2988109c50 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:12.274 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.273+0000 7f298e76c700 1 --2- 172.21.15.27:0/2163676597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29880f6860 0x7f298810c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:12.275 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.273+0000 7f298e76c700 1 -- 172.21.15.27:0/2163676597 >> 172.21.15.27:0/2163676597 conn(0x7f29880f0d40 msgr2=0x7f29880fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:22:12.275 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.273+0000 7f298e76c700 1 -- 172.21.15.27:0/2163676597 shutdown_connections 2021-11-03T17:22:12.275 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.273+0000 7f298e76c700 1 -- 172.21.15.27:0/2163676597 wait complete. 2021-11-03T17:22:12.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.376+0000 7f3bb9a85700 1 -- 172.21.15.27:0/957521725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bb40f5340 msgr2=0x7f3bb40f7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:22:12.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.376+0000 7f3bb9a85700 1 --2- 172.21.15.27:0/957521725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bb40f5340 0x7f3bb40f7ac0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f3ba8004660 tx=0x7f3ba8009b30).stop 2021-11-03T17:22:12.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.376+0000 7f3bb9a85700 1 -- 172.21.15.27:0/957521725 shutdown_connections 2021-11-03T17:22:12.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.376+0000 7f3bb9a85700 1 --2- 172.21.15.27:0/957521725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bb40f5340 0x7f3bb40f7ac0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:12.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.376+0000 7f3bb9a85700 1 --2- 172.21.15.27:0/957521725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bb40f49a0 0x7f3bb40f4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:12.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.376+0000 7f3bb9a85700 1 -- 172.21.15.27:0/957521725 >> 172.21.15.27:0/957521725 conn(0x7f3bb40f04c0 msgr2=0x7f3bb40f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:22:12.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.376+0000 7f3bb9a85700 1 -- 172.21.15.27:0/957521725 shutdown_connections 2021-11-03T17:22:12.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.376+0000 7f3bb9a85700 1 -- 172.21.15.27:0/957521725 wait complete. 2021-11-03T17:22:12.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.377+0000 7f3bb9a85700 1 Processor -- start 2021-11-03T17:22:12.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.377+0000 7f3bb9a85700 1 -- start start 2021-11-03T17:22:12.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.377+0000 7f3bb9a85700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bb40f49a0 0x7f3bb4106fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:22:12.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.377+0000 7f3bb9a85700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bb40f5340 0x7f3bb4107490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:22:12.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.377+0000 7f3bb9a85700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb4107a40 con 0x7f3bb40f5340 2021-11-03T17:22:12.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.377+0000 7f3bb9a85700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb4107b80 con 0x7f3bb40f49a0 2021-11-03T17:22:12.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.377+0000 7f3bb2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bb40f5340 0x7f3bb4107490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:22:12.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.377+0000 7f3bb2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bb40f5340 0x7f3bb4107490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57048/0 (socket says 172.21.15.27:57048) 2021-11-03T17:22:12.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.377+0000 7f3bb2ffd700 1 -- 172.21.15.27:0/2658867686 learned_addr learned my addr 172.21.15.27:0/2658867686 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:22:12.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.377+0000 7f3bb37fe700 1 --2- 172.21.15.27:0/2658867686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bb40f49a0 0x7f3bb4106fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:22:12.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.377+0000 7f3bb2ffd700 1 -- 172.21.15.27:0/2658867686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bb40f49a0 msgr2=0x7f3bb4106fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:22:12.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.377+0000 7f3bb2ffd700 1 --2- 172.21.15.27:0/2658867686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bb40f49a0 0x7f3bb4106fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:12.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.377+0000 7f3bb2ffd700 1 -- 172.21.15.27:0/2658867686 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba8005040 con 0x7f3bb40f5340 2021-11-03T17:22:12.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.378+0000 7f3bb37fe700 1 --2- 172.21.15.27:0/2658867686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bb40f49a0 0x7f3bb4106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:22:12.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.378+0000 7f3bb2ffd700 1 --2- 172.21.15.27:0/2658867686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bb40f5340 0x7f3bb4107490 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f3ba8004f60 tx=0x7f3ba8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:22:12.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.378+0000 7f3b9bfff700 1 -- 172.21.15.27:0/2658867686 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ba8018070 con 0x7f3bb40f5340 2021-11-03T17:22:12.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.378+0000 7f3b9bfff700 1 -- 172.21.15.27:0/2658867686 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f3ba8007ba0 con 0x7f3bb40f5340 2021-11-03T17:22:12.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.378+0000 7f3bb9a85700 1 -- 172.21.15.27:0/2658867686 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bb41084c0 con 0x7f3bb40f5340 2021-11-03T17:22:12.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.378+0000 7f3b9bfff700 1 -- 172.21.15.27:0/2658867686 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ba801c620 con 0x7f3bb40f5340 2021-11-03T17:22:12.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.378+0000 7f3bb9a85700 1 -- 172.21.15.27:0/2658867686 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bb4108a20 con 0x7f3bb40f5340 2021-11-03T17:22:12.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.379+0000 7f3bb9a85700 1 -- 172.21.15.27:0/2658867686 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bb4045ad0 con 0x7f3bb40f5340 2021-11-03T17:22:12.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.379+0000 7f3b9bfff700 1 -- 172.21.15.27:0/2658867686 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f3ba801d420 con 0x7f3bb40f5340 2021-11-03T17:22:12.382 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.380+0000 7f3b9bfff700 1 --2- 172.21.15.27:0/2658867686 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3b9c0640b0 0x7f3b9c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:22:12.382 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.380+0000 7f3b9bfff700 1 -- 172.21.15.27:0/2658867686 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f3ba8082610 con 0x7f3bb40f5340 2021-11-03T17:22:12.382 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.380+0000 7f3bb37fe700 1 --2- 172.21.15.27:0/2658867686 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3b9c0640b0 0x7f3b9c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:22:12.382 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.380+0000 7f3bb37fe700 1 --2- 172.21.15.27:0/2658867686 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3b9c0640b0 0x7f3b9c066570 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f3bb405f4c0 tx=0x7f3ba4009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:22:12.383 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.384+0000 7f3b9bfff700 1 -- 172.21.15.27:0/2658867686 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f3ba8054a30 con 0x7f3bb40f5340 2021-11-03T17:22:12.609 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.614+0000 7f3bb9a85700 1 -- 172.21.15.27:0/2658867686 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3bb4027a00 con 0x7f3bb40f5340 2021-11-03T17:22:12.610 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.614+0000 7f3b9bfff700 1 -- 172.21.15.27:0/2658867686 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f3ba8054620 con 0x7f3bb40f5340 2021-11-03T17:22:12.611 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:22:12.611 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:22:12.611 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:22:12.611 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:22:12.612 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:22:12.612 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:22:12.612 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:22:12.612 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:22:12.613 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:22:12.613 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:22:12.613 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:22:12.613 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:22:12.614 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:22:12.614 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:22:12.614 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:22:12.614 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:22:12.615 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:22:12.615 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:22:12.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.616+0000 7f3bb9a85700 1 -- 172.21.15.27:0/2658867686 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3b9c0640b0 msgr2=0x7f3b9c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:22:12.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.616+0000 7f3bb9a85700 1 --2- 172.21.15.27:0/2658867686 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3b9c0640b0 0x7f3b9c066570 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f3bb405f4c0 tx=0x7f3ba4009040).stop 2021-11-03T17:22:12.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.616+0000 7f3bb9a85700 1 -- 172.21.15.27:0/2658867686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bb40f5340 msgr2=0x7f3bb4107490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:22:12.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.616+0000 7f3bb9a85700 1 --2- 172.21.15.27:0/2658867686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bb40f5340 0x7f3bb4107490 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f3ba8004f60 tx=0x7f3ba8009b30).stop 2021-11-03T17:22:12.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.616+0000 7f3bb9a85700 1 -- 172.21.15.27:0/2658867686 shutdown_connections 2021-11-03T17:22:12.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.616+0000 7f3bb9a85700 1 --2- 172.21.15.27:0/2658867686 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3b9c0640b0 0x7f3b9c066570 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:12.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.616+0000 7f3bb9a85700 1 --2- 172.21.15.27:0/2658867686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3bb40f49a0 0x7f3bb4106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:12.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.616+0000 7f3bb9a85700 1 --2- 172.21.15.27:0/2658867686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3bb40f5340 0x7f3bb4107490 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:12.618 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.616+0000 7f3bb9a85700 1 -- 172.21.15.27:0/2658867686 >> 172.21.15.27:0/2658867686 conn(0x7f3bb40f04c0 msgr2=0x7f3bb40fb510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:22:12.618 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.617+0000 7f3bb9a85700 1 -- 172.21.15.27:0/2658867686 shutdown_connections 2021-11-03T17:22:12.618 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.617+0000 7f3bb9a85700 1 -- 172.21.15.27:0/2658867686 wait complete. 2021-11-03T17:22:12.715 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.717+0000 7f3595d72700 1 -- 172.21.15.27:0/1731518208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35900f5440 msgr2=0x7f35900f5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:22:12.716 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.717+0000 7f3595d72700 1 --2- 172.21.15.27:0/1731518208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35900f5440 0x7f35900f5860 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f3584004660 tx=0x7f3584009b30).stop 2021-11-03T17:22:12.716 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.717+0000 7f3595d72700 1 -- 172.21.15.27:0/1731518208 shutdown_connections 2021-11-03T17:22:12.717 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.717+0000 7f3595d72700 1 --2- 172.21.15.27:0/1731518208 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35900f66e0 0x7f35900f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:22:12.717 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.717+0000 7f3595d72700 1 --2- 172.21.15.27:0/1731518208 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35900f5440 0x7f35900f5860 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:12.717 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.717+0000 7f3595d72700 1 -- 172.21.15.27:0/1731518208 >> 172.21.15.27:0/1731518208 conn(0x7f35900f0ba0 msgr2=0x7f35900f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:22:12.717 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.717+0000 7f3595d72700 1 -- 172.21.15.27:0/1731518208 shutdown_connections 2021-11-03T17:22:12.717 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.718+0000 7f3595d72700 1 -- 172.21.15.27:0/1731518208 wait complete. 2021-11-03T17:22:12.718 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.718+0000 7f3595d72700 1 Processor -- start 2021-11-03T17:22:12.718 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.718+0000 7f3595d72700 1 -- start start 2021-11-03T17:22:12.718 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.718+0000 7f3595d72700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35900f5440 0x7f3590109a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:22:12.718 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.718+0000 7f3595d72700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35900f66e0 0x7f359010bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:22:12.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.718+0000 7f3595d72700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3590060470 con 0x7f35900f5440 2021-11-03T17:22:12.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.718+0000 7f3595d72700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35900605b0 con 0x7f35900f66e0 2021-11-03T17:22:12.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.718+0000 7f3594d70700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35900f5440 0x7f3590109a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:22:12.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.718+0000 7f3594d70700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35900f5440 0x7f3590109a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57056/0 (socket says 172.21.15.27:57056) 2021-11-03T17:22:12.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.719+0000 7f3594d70700 1 -- 172.21.15.27:0/1903429308 learned_addr learned my addr 172.21.15.27:0/1903429308 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:22:12.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.719+0000 7f358ffff700 1 --2- 172.21.15.27:0/1903429308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35900f66e0 0x7f359010bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:22:12.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.719+0000 7f3594d70700 1 -- 172.21.15.27:0/1903429308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35900f66e0 msgr2=0x7f359010bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:22:12.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.719+0000 7f3594d70700 1 --2- 172.21.15.27:0/1903429308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35900f66e0 0x7f359010bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:12.721 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.719+0000 7f3594d70700 1 -- 172.21.15.27:0/1903429308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3584005040 con 0x7f35900f5440 2021-11-03T17:22:12.721 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.719+0000 7f358ffff700 1 --2- 172.21.15.27:0/1903429308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35900f66e0 0x7f359010bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:22:12.721 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.719+0000 7f3594d70700 1 --2- 172.21.15.27:0/1903429308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35900f5440 0x7f3590109a80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f3584004f60 tx=0x7f3584009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:22:12.721 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.719+0000 7f358d7fa700 1 -- 172.21.15.27:0/1903429308 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3584018070 con 0x7f35900f5440 2021-11-03T17:22:12.722 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.719+0000 7f358d7fa700 1 -- 172.21.15.27:0/1903429308 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f3584007ba0 con 0x7f35900f5440 2021-11-03T17:22:12.722 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.719+0000 7f3595d72700 1 -- 172.21.15.27:0/1903429308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f359010c580 con 0x7f35900f5440 2021-11-03T17:22:12.722 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.719+0000 7f358d7fa700 1 -- 172.21.15.27:0/1903429308 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f358401c620 con 0x7f35900f5440 2021-11-03T17:22:12.722 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.719+0000 7f3595d72700 1 -- 172.21.15.27:0/1903429308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f359010cab0 con 0x7f35900f5440 2021-11-03T17:22:12.723 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.720+0000 7f3595d72700 1 -- 172.21.15.27:0/1903429308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3590059100 con 0x7f35900f5440 2021-11-03T17:22:12.723 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.720+0000 7f358d7fa700 1 -- 172.21.15.27:0/1903429308 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f358401c8e0 con 0x7f35900f5440 2021-11-03T17:22:12.724 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.721+0000 7f358d7fa700 1 --2- 172.21.15.27:0/1903429308 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3580064100 0x7f35800665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:22:12.724 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.721+0000 7f358d7fa700 1 -- 172.21.15.27:0/1903429308 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f35840826c0 con 0x7f35900f5440 2021-11-03T17:22:12.724 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.721+0000 7f358ffff700 1 --2- 172.21.15.27:0/1903429308 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3580064100 0x7f35800665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:22:12.725 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.721+0000 7f358ffff700 1 --2- 172.21.15.27:0/1903429308 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3580064100 0x7f35800665c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f357c00a7e0 tx=0x7f357c008040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:22:12.725 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.725+0000 7f358d7fa700 1 -- 172.21.15.27:0/1903429308 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f3584054a20 con 0x7f35900f5440 2021-11-03T17:22:12.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.932+0000 7f3595d72700 1 -- 172.21.15.27:0/1903429308 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f35900fdd90 con 0x7f35900f5440 2021-11-03T17:22:12.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.933+0000 7f358d7fa700 1 -- 172.21.15.27:0/1903429308 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f3584054610 con 0x7f35900f5440 2021-11-03T17:22:12.929 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:22:12.929 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:22:12.929 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:22:12.930 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:22:12.930 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:22:12.930 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:22:12.931 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:22:12.931 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:22:12.931 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:22:12.931 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:22:12.931 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:22:12.932 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:22:12.932 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:22:12.932 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:22:12.932 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:22:12.933 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:22:12.933 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:22:12.933 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:22:12.933 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:22:12.933 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:22:12.934 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:22:12.934 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:22:12.934 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:22:12.934 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:22:12.935 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:22:12.935 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:22:12.935 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:22:12.935 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:22:12.936 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:22:12.936 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:22:12.936 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:22:12.936 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:22:12.937 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:22:12.937 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:22:12.937 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:22:12.937 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:22:12.937 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:22:12.938 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:22:12.938 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:22:12.939 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.934+0000 7f3595d72700 1 -- 172.21.15.27:0/1903429308 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3580064100 msgr2=0x7f35800665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:22:12.939 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.934+0000 7f3595d72700 1 --2- 172.21.15.27:0/1903429308 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3580064100 0x7f35800665c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f357c00a7e0 tx=0x7f357c008040).stop 2021-11-03T17:22:12.939 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.934+0000 7f3595d72700 1 -- 172.21.15.27:0/1903429308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35900f5440 msgr2=0x7f3590109a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:22:12.940 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.934+0000 7f3595d72700 1 --2- 172.21.15.27:0/1903429308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35900f5440 0x7f3590109a80 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f3584004f60 tx=0x7f3584009b30).stop 2021-11-03T17:22:12.940 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.934+0000 7f3595d72700 1 -- 172.21.15.27:0/1903429308 shutdown_connections 2021-11-03T17:22:12.940 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.934+0000 7f3595d72700 1 --2- 172.21.15.27:0/1903429308 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3580064100 0x7f35800665c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:12.940 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.934+0000 7f3595d72700 1 --2- 172.21.15.27:0/1903429308 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35900f5440 0x7f3590109a80 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:12.941 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.934+0000 7f3595d72700 1 --2- 172.21.15.27:0/1903429308 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35900f66e0 0x7f359010bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:12.941 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.934+0000 7f3595d72700 1 -- 172.21.15.27:0/1903429308 >> 172.21.15.27:0/1903429308 conn(0x7f35900f0ba0 msgr2=0x7f35900fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:22:12.941 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.935+0000 7f3595d72700 1 -- 172.21.15.27:0/1903429308 shutdown_connections 2021-11-03T17:22:12.941 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:12.935+0000 7f3595d72700 1 -- 172.21.15.27:0/1903429308 wait complete. 2021-11-03T17:22:12.942 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:22:13.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:12 smithi115 ceph-mon[65453]: pgmap v225: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:13.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:12 smithi115 ceph-mon[65453]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:22:13.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:12 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2658867686' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:13.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:12 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1903429308' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:22:13.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:12 smithi027 ceph-mon[97244]: pgmap v225: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:13.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:12 smithi027 ceph-mon[97244]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:22:13.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:12 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2658867686' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:13.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:12 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1903429308' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:22:14.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:13 smithi115 ceph-mon[65453]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:22:14.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:13 smithi027 ceph-mon[97244]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:22:15.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:14 smithi027 ceph-mon[97244]: pgmap v226: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:15.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:14 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 377.368020 secs 2021-11-03T17:22:15.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:14 smithi115 ceph-mon[65453]: pgmap v226: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:15.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:14 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 377.368020 secs 2021-11-03T17:22:16.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:15 smithi027 ceph-mon[97244]: pgmap v227: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:16.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:15 smithi115 ceph-mon[65453]: pgmap v227: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:17.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:22:17.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:22:17.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:22:17.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:17.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:17.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:17.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:17.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:16 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:22:17.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:22:17.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:22:17.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:22:17.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:17.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:17.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:17.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:17.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:16 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:22:18.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:18 smithi027 ceph-mon[97244]: pgmap v228: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:18.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:18 smithi115 ceph-mon[65453]: pgmap v228: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:19.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:19 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 382.368121 secs 2021-11-03T17:22:19.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:19 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 382.368121 secs 2021-11-03T17:22:20.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:20 smithi027 ceph-mon[97244]: pgmap v229: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:20.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:20 smithi115 ceph-mon[65453]: pgmap v229: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:23.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:22 smithi115 ceph-mon[65453]: pgmap v230: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:23.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:22 smithi027 ceph-mon[97244]: pgmap v230: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:25.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:24 smithi115 ceph-mon[65453]: pgmap v231: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:25.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:24 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 387.368243 secs 2021-11-03T17:22:25.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:24 smithi027 ceph-mon[97244]: pgmap v231: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:25.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:24 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 387.368243 secs 2021-11-03T17:22:27.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:26 smithi115 ceph-mon[65453]: pgmap v232: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:27.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:22:27.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:22:27.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:22:27.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:27.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:27.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:27.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:27.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:26 smithi027 ceph-mon[97244]: pgmap v232: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:27.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:22:27.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:22:27.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:22:27.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:27.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:27.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:27.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:28.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:27 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:22:28.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:27 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:22:29.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:28 smithi115 ceph-mon[65453]: pgmap v233: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:29.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:28 smithi027 ceph-mon[97244]: pgmap v233: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:30.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:29 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 392.368349 secs 2021-11-03T17:22:30.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:29 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 392.368349 secs 2021-11-03T17:22:31.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:30 smithi115 ceph-mon[65453]: pgmap v234: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:31.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:30 smithi027 ceph-mon[97244]: pgmap v234: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:33.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:32 smithi115 ceph-mon[65453]: pgmap v235: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:33.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:32 smithi027 ceph-mon[97244]: pgmap v235: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:35.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:34 smithi115 ceph-mon[65453]: pgmap v236: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:35.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:34 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 397.368483 secs 2021-11-03T17:22:35.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:34 smithi027 ceph-mon[97244]: pgmap v236: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:35.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:34 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 397.368483 secs 2021-11-03T17:22:37.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:36 smithi115 ceph-mon[65453]: pgmap v237: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:37.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:22:37.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:22:37.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:22:37.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:37.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:37.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:37.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:37.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:36 smithi027 ceph-mon[97244]: pgmap v237: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:37.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:22:37.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:22:37.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:22:37.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:37.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:37.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:37.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:38.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:37 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:22:38.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:37 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:22:39.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:38 smithi115 ceph-mon[65453]: pgmap v238: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:39.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:38 smithi027 ceph-mon[97244]: pgmap v238: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:40.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:39 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 402.368576 secs 2021-11-03T17:22:40.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:39 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 402.368576 secs 2021-11-03T17:22:41.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:40 smithi115 ceph-mon[65453]: pgmap v239: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:41.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:40 smithi027 ceph-mon[97244]: pgmap v239: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:43.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.050+0000 7f3249f2f700 1 -- 172.21.15.27:0/754878379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32440602d0 msgr2=0x7f32440f9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:22:43.046 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.050+0000 7f3249f2f700 1 --2- 172.21.15.27:0/754878379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32440602d0 0x7f32440f9160 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f3238004660 tx=0x7f3238009b30).stop 2021-11-03T17:22:43.046 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.050+0000 7f3249f2f700 1 -- 172.21.15.27:0/754878379 shutdown_connections 2021-11-03T17:22:43.046 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.050+0000 7f3249f2f700 1 --2- 172.21.15.27:0/754878379 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32440f96e0 0x7f32440fbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:22:43.047 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.050+0000 7f3249f2f700 1 --2- 172.21.15.27:0/754878379 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32440602d0 0x7f32440f9160 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:43.047 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.050+0000 7f3249f2f700 1 -- 172.21.15.27:0/754878379 >> 172.21.15.27:0/754878379 conn(0x7f32440f0ba0 msgr2=0x7f32440f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:22:43.047 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.051+0000 7f3249f2f700 1 -- 172.21.15.27:0/754878379 shutdown_connections 2021-11-03T17:22:43.048 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.051+0000 7f3249f2f700 1 -- 172.21.15.27:0/754878379 wait complete. 2021-11-03T17:22:43.048 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.051+0000 7f3249f2f700 1 Processor -- start 2021-11-03T17:22:43.048 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.051+0000 7f3249f2f700 1 -- start start 2021-11-03T17:22:43.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.051+0000 7f3249f2f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32440602d0 0x7f3244106e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:22:43.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.052+0000 7f3249f2f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32440f96e0 0x7f3244107300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:22:43.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.052+0000 7f3249f2f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3244107990 con 0x7f32440602d0 2021-11-03T17:22:43.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.052+0000 7f3249f2f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3244108110 con 0x7f32440f96e0 2021-11-03T17:22:43.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.052+0000 7f3248f2d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32440602d0 0x7f3244106e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:22:43.055 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.052+0000 7f3248f2d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32440602d0 0x7f3244106e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57074/0 (socket says 172.21.15.27:57074) 2021-11-03T17:22:43.055 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.052+0000 7f3248f2d700 1 -- 172.21.15.27:0/2681621835 learned_addr learned my addr 172.21.15.27:0/2681621835 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:22:43.055 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.052+0000 7f3243fff700 1 --2- 172.21.15.27:0/2681621835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32440f96e0 0x7f3244107300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:22:43.055 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.052+0000 7f3248f2d700 1 -- 172.21.15.27:0/2681621835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32440f96e0 msgr2=0x7f3244107300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:22:43.056 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.052+0000 7f3248f2d700 1 --2- 172.21.15.27:0/2681621835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32440f96e0 0x7f3244107300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:43.056 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.052+0000 7f3248f2d700 1 -- 172.21.15.27:0/2681621835 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3238005040 con 0x7f32440602d0 2021-11-03T17:22:43.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.052+0000 7f3243fff700 1 --2- 172.21.15.27:0/2681621835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32440f96e0 0x7f3244107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:22:43.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.052+0000 7f3248f2d700 1 --2- 172.21.15.27:0/2681621835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32440602d0 0x7f3244106e10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f3238009990 tx=0x7f3238009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:22:43.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.052+0000 7f32417fa700 1 -- 172.21.15.27:0/2681621835 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3238018070 con 0x7f32440602d0 2021-11-03T17:22:43.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.052+0000 7f32417fa700 1 -- 172.21.15.27:0/2681621835 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f3238007ca0 con 0x7f32440602d0 2021-11-03T17:22:43.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.052+0000 7f3249f2f700 1 -- 172.21.15.27:0/2681621835 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3244108330 con 0x7f32440602d0 2021-11-03T17:22:43.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.052+0000 7f32417fa700 1 -- 172.21.15.27:0/2681621835 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f323801c620 con 0x7f32440602d0 2021-11-03T17:22:43.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.053+0000 7f3249f2f700 1 -- 172.21.15.27:0/2681621835 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3244108890 con 0x7f32440602d0 2021-11-03T17:22:43.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.054+0000 7f3249f2f700 1 -- 172.21.15.27:0/2681621835 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3244059100 con 0x7f32440602d0 2021-11-03T17:22:43.059 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.054+0000 7f32417fa700 1 -- 172.21.15.27:0/2681621835 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f323801d960 con 0x7f32440602d0 2021-11-03T17:22:43.059 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.054+0000 7f32417fa700 1 --2- 172.21.15.27:0/2681621835 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3234064100 0x7f32340665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:22:43.059 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.055+0000 7f32417fa700 1 -- 172.21.15.27:0/2681621835 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f3238082520 con 0x7f32440602d0 2021-11-03T17:22:43.059 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.055+0000 7f3243fff700 1 --2- 172.21.15.27:0/2681621835 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3234064100 0x7f32340665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:22:43.060 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.055+0000 7f3243fff700 1 --2- 172.21.15.27:0/2681621835 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3234064100 0x7f32340665c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f32440600a0 tx=0x7f3230008040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:22:43.060 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.059+0000 7f32417fa700 1 -- 172.21.15.27:0/2681621835 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f3238054940 con 0x7f32440602d0 2021-11-03T17:22:43.232 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.238+0000 7f3249f2f700 1 -- 172.21.15.27:0/2681621835 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f32440fef30 con 0x7f3234064100 2021-11-03T17:22:43.234 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.239+0000 7f32417fa700 1 -- 172.21.15.27:0/2681621835 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f32440fef30 con 0x7f3234064100 2021-11-03T17:22:43.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:42 smithi115 ceph-mon[65453]: pgmap v240: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:43.235 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.241+0000 7f3249f2f700 1 -- 172.21.15.27:0/2681621835 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3234064100 msgr2=0x7f32340665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:22:43.236 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.241+0000 7f3249f2f700 1 --2- 172.21.15.27:0/2681621835 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3234064100 0x7f32340665c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f32440600a0 tx=0x7f3230008040).stop 2021-11-03T17:22:43.236 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.241+0000 7f3249f2f700 1 -- 172.21.15.27:0/2681621835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32440602d0 msgr2=0x7f3244106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:22:43.236 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.241+0000 7f3249f2f700 1 --2- 172.21.15.27:0/2681621835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32440602d0 0x7f3244106e10 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f3238009990 tx=0x7f3238009b30).stop 2021-11-03T17:22:43.236 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.241+0000 7f3249f2f700 1 -- 172.21.15.27:0/2681621835 shutdown_connections 2021-11-03T17:22:43.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.241+0000 7f3249f2f700 1 --2- 172.21.15.27:0/2681621835 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3234064100 0x7f32340665c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:43.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.241+0000 7f3249f2f700 1 --2- 172.21.15.27:0/2681621835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32440602d0 0x7f3244106e10 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:43.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.241+0000 7f3249f2f700 1 --2- 172.21.15.27:0/2681621835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32440f96e0 0x7f3244107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:43.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.241+0000 7f3249f2f700 1 -- 172.21.15.27:0/2681621835 >> 172.21.15.27:0/2681621835 conn(0x7f32440f0ba0 msgr2=0x7f32440f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:22:43.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.241+0000 7f3249f2f700 1 -- 172.21.15.27:0/2681621835 shutdown_connections 2021-11-03T17:22:43.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.241+0000 7f3249f2f700 1 -- 172.21.15.27:0/2681621835 wait complete. 2021-11-03T17:22:43.246 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:22:43.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:42 smithi027 ceph-mon[97244]: pgmap v240: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:43.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.342+0000 7f97ef745700 1 -- 172.21.15.27:0/2086651408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97e80f6820 msgr2=0x7f97e80f6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:22:43.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.342+0000 7f97ef745700 1 --2- 172.21.15.27:0/2086651408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97e80f6820 0x7f97e80f6c80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f97e4004660 tx=0x7f97e4009b30).stop 2021-11-03T17:22:43.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.342+0000 7f97ef745700 1 -- 172.21.15.27:0/2086651408 shutdown_connections 2021-11-03T17:22:43.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.342+0000 7f97ef745700 1 --2- 172.21.15.27:0/2086651408 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97e80f6820 0x7f97e80f6c80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:43.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.342+0000 7f97ef745700 1 --2- 172.21.15.27:0/2086651408 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f97e80f5580 0x7f97e80f59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:43.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.342+0000 7f97ef745700 1 -- 172.21.15.27:0/2086651408 >> 172.21.15.27:0/2086651408 conn(0x7f97e80f0ce0 msgr2=0x7f97e80f3100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:22:43.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.342+0000 7f97ef745700 1 -- 172.21.15.27:0/2086651408 shutdown_connections 2021-11-03T17:22:43.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.342+0000 7f97ef745700 1 -- 172.21.15.27:0/2086651408 wait complete. 2021-11-03T17:22:43.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.343+0000 7f97ef745700 1 Processor -- start 2021-11-03T17:22:43.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.343+0000 7f97ef745700 1 -- start start 2021-11-03T17:22:43.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.343+0000 7f97ef745700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97e80f5580 0x7f97e810bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:22:43.342 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.343+0000 7f97ef745700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f97e80f6820 0x7f97e810e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:22:43.342 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.343+0000 7f97ef745700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e8060430 con 0x7f97e80f5580 2021-11-03T17:22:43.342 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.343+0000 7f97ef745700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97e8060570 con 0x7f97e80f6820 2021-11-03T17:22:43.342 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.343+0000 7f97ed4e1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97e80f5580 0x7f97e810bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:22:43.343 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.343+0000 7f97ed4e1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97e80f5580 0x7f97e810bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57084/0 (socket says 172.21.15.27:57084) 2021-11-03T17:22:43.343 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.343+0000 7f97ed4e1700 1 -- 172.21.15.27:0/1920938114 learned_addr learned my addr 172.21.15.27:0/1920938114 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:22:43.343 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.343+0000 7f97ed4e1700 1 -- 172.21.15.27:0/1920938114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f97e80f6820 msgr2=0x7f97e810e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:22:43.343 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.343+0000 7f97ed4e1700 1 --2- 172.21.15.27:0/1920938114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f97e80f6820 0x7f97e810e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:22:43.344 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.343+0000 7f97ed4e1700 1 -- 172.21.15.27:0/1920938114 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97e4005040 con 0x7f97e80f5580 2021-11-03T17:22:43.344 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.343+0000 7f97ed4e1700 1 --2- 172.21.15.27:0/1920938114 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97e80f5580 0x7f97e810bfb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f97d8002700 tx=0x7f97d8009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:22:43.345 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.343+0000 7f97ddffb700 1 -- 172.21.15.27:0/1920938114 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97d8009320 con 0x7f97e80f5580 2021-11-03T17:22:43.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.343+0000 7f97ef745700 1 -- 172.21.15.27:0/1920938114 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97e810e9a0 con 0x7f97e80f5580 2021-11-03T17:22:43.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.343+0000 7f97ddffb700 1 -- 172.21.15.27:0/1920938114 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f97d800f040 con 0x7f97e80f5580 2021-11-03T17:22:43.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.343+0000 7f97ddffb700 1 -- 172.21.15.27:0/1920938114 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97d8014900 con 0x7f97e80f5580 2021-11-03T17:22:43.347 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.344+0000 7f97ef745700 1 -- 172.21.15.27:0/1920938114 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97e810ee20 con 0x7f97e80f5580 2021-11-03T17:22:43.348 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.345+0000 7f97ef745700 1 -- 172.21.15.27:0/1920938114 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97e8045ad0 con 0x7f97e80f5580 2021-11-03T17:22:43.348 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.345+0000 7f97ddffb700 1 -- 172.21.15.27:0/1920938114 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f97d8009480 con 0x7f97e80f5580 2021-11-03T17:22:43.348 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.345+0000 7f97ddffb700 1 --2- 172.21.15.27:0/1920938114 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f97d4064100 0x7f97d40665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:22:43.349 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.345+0000 7f97ddffb700 1 -- 172.21.15.27:0/1920938114 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f97d8080a50 con 0x7f97e80f5580 2021-11-03T17:22:43.349 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.345+0000 7f97ecce0700 1 --2- 172.21.15.27:0/1920938114 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f97d4064100 0x7f97d40665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:22:43.349 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.346+0000 7f97ecce0700 1 --2- 172.21.15.27:0/1920938114 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f97d4064100 0x7f97d40665c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f97e4004e30 tx=0x7f97e4009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:22:43.349 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.349+0000 7f97ddffb700 1 -- 172.21.15.27:0/1920938114 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f97d8052db0 con 0x7f97e80f5580 2021-11-03T17:22:43.526 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.531+0000 7f97ef745700 1 -- 172.21.15.27:0/1920938114 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f97e8057f10 con 0x7f97d4064100 2021-11-03T17:22:43.540 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.546+0000 7f97ddffb700 1 -- 172.21.15.27:0/1920938114 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f97e8057f10 con 0x7f97d4064100 2021-11-03T17:22:43.542 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:22:43.542 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (14m) 4m ago 15m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:22:43.542 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (5m) 4m ago 15m 7205k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:22:43.542 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (5m) 3m ago 14m 7239k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:22:43.543 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (14m) 4m ago 15m 38.9M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:22:43.543 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (7m) 4m ago 10m 15.4M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:22:43.543 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (10m) 4m ago 10m 3836M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:22:43.543 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (6m) 3m ago 10m 14.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:22:43.544 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (8m) 3m ago 10m 16.3M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:22:43.544 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (7m) 4m ago 16m 470M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:22:43.544 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (7m) 3m ago 14m 412M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:22:43.544 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (6m) 4m ago 16m 81.2M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:22:43.544 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (6m) 3m ago 14m 69.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:22:43.545 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (15m) 4m ago 15m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:22:43.545 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (14m) 3m ago 14m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:22:43.545 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (5m) 4m ago 13m 243M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:22:43.545 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (5m) 4m ago 13m 270M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:22:43.546 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (4m) 4m ago 13m 11.8M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:22:43.546 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (4m) 3m ago 12m 289M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:22:43.546 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (3m) 3m ago 12m 235M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:22:43.546 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (3m) 3m ago 12m 11.5M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:22:43.546 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (14m) 4m ago 14m 45.7M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:22:43.547 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.547+0000 7f97ef745700 1 -- 172.21.15.27:0/1920938114 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f97d4064100 msgr2=0x7f97d40665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:22:43.548 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.547+0000 7f97ef745700 1 --2- 172.21.15.27:0/1920938114 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f97d4064100 0x7f97d40665c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f97e4004e30 tx=0x7f97e4009b30).stop 2021-11-03T17:22:43.548 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.547+0000 7f97ef745700 1 -- 172.21.15.27:0/1920938114 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97e80f5580 msgr2=0x7f97e810bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:22:43.548 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.547+0000 7f97ef745700 1 --2- 172.21.15.27:0/1920938114 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97e80f5580 0x7f97e810bfb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f97d8002700 tx=0x7f97d8009d80).stop 2021-11-03T17:22:43.548 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.548+0000 7f97ef745700 1 -- 172.21.15.27:0/1920938114 shutdown_connections 2021-11-03T17:22:43.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.548+0000 7f97ef745700 1 --2- 172.21.15.27:0/1920938114 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f97d4064100 0x7f97d40665c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:43.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.548+0000 7f97ef745700 1 --2- 172.21.15.27:0/1920938114 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97e80f5580 0x7f97e810bfb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:43.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.548+0000 7f97ef745700 1 --2- 172.21.15.27:0/1920938114 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f97e80f6820 0x7f97e810e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:22:43.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.548+0000 7f97ef745700 1 -- 172.21.15.27:0/1920938114 >> 172.21.15.27:0/1920938114 conn(0x7f97e80f0ce0 msgr2=0x7f97e80fa270 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:22:43.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.548+0000 7f97ef745700 1 -- 172.21.15.27:0/1920938114 shutdown_connections 2021-11-03T17:22:43.550 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.548+0000 7f97ef745700 1 -- 172.21.15.27:0/1920938114 wait complete. 2021-11-03T17:22:43.642 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.647+0000 7f0d517e6700 1 -- 172.21.15.27:0/253120657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d4c060390 msgr2=0x7f0d4c0f9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:22:43.642 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.647+0000 7f0d517e6700 1 --2- 172.21.15.27:0/253120657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d4c060390 0x7f0d4c0f9360 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f0d440099e0 tx=0x7f0d44004d10).stop 2021-11-03T17:22:43.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.648+0000 7f0d517e6700 1 -- 172.21.15.27:0/253120657 shutdown_connections 2021-11-03T17:22:43.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.648+0000 7f0d517e6700 1 --2- 172.21.15.27:0/253120657 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d4c0f9850 0x7f0d4c0fbc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:43.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.648+0000 7f0d517e6700 1 --2- 172.21.15.27:0/253120657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d4c060390 0x7f0d4c0f9360 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:43.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.648+0000 7f0d517e6700 1 -- 172.21.15.27:0/253120657 >> 172.21.15.27:0/253120657 conn(0x7f0d4c0f0de0 msgr2=0x7f0d4c0f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:22:43.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.648+0000 7f0d517e6700 1 -- 172.21.15.27:0/253120657 shutdown_connections 2021-11-03T17:22:43.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.648+0000 7f0d517e6700 1 -- 172.21.15.27:0/253120657 wait complete. 2021-11-03T17:22:43.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.648+0000 7f0d517e6700 1 Processor -- start 2021-11-03T17:22:43.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.648+0000 7f0d517e6700 1 -- start start 2021-11-03T17:22:43.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.648+0000 7f0d517e6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d4c060390 0x7f0d4c0f6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:22:43.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.648+0000 7f0d517e6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d4c0f9850 0x7f0d4c0f5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:22:43.648 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.648+0000 7f0d517e6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c0f57c0 con 0x7f0d4c060390 2021-11-03T17:22:43.648 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.648+0000 7f0d517e6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d4c0f5900 con 0x7f0d4c0f9850 2021-11-03T17:22:43.648 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.649+0000 7f0d4affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d4c060390 0x7f0d4c0f6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:22:43.648 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.649+0000 7f0d4affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d4c060390 0x7f0d4c0f6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57094/0 (socket says 172.21.15.27:57094) 2021-11-03T17:22:43.649 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.649+0000 7f0d4affd700 1 -- 172.21.15.27:0/113970316 learned_addr learned my addr 172.21.15.27:0/113970316 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:22:43.649 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.649+0000 7f0d4a7fc700 1 --2- 172.21.15.27:0/113970316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d4c0f9850 0x7f0d4c0f5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:22:43.649 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.649+0000 7f0d4affd700 1 -- 172.21.15.27:0/113970316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d4c0f9850 msgr2=0x7f0d4c0f5260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:22:43.649 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.649+0000 7f0d4affd700 1 --2- 172.21.15.27:0/113970316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d4c0f9850 0x7f0d4c0f5260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:43.649 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.649+0000 7f0d4affd700 1 -- 172.21.15.27:0/113970316 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d44009640 con 0x7f0d4c060390 2021-11-03T17:22:43.650 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.649+0000 7f0d4affd700 1 --2- 172.21.15.27:0/113970316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d4c060390 0x7f0d4c0f6de0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f0d4400c010 tx=0x7f0d44004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:22:43.650 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.649+0000 7f0d337fe700 1 -- 172.21.15.27:0/113970316 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d4400e070 con 0x7f0d4c060390 2021-11-03T17:22:43.650 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.649+0000 7f0d337fe700 1 -- 172.21.15.27:0/113970316 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f0d44007d10 con 0x7f0d4c060390 2021-11-03T17:22:43.650 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.649+0000 7f0d517e6700 1 -- 172.21.15.27:0/113970316 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d4c0f5b80 con 0x7f0d4c060390 2021-11-03T17:22:43.651 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.649+0000 7f0d337fe700 1 -- 172.21.15.27:0/113970316 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d44018070 con 0x7f0d4c060390 2021-11-03T17:22:43.651 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.649+0000 7f0d517e6700 1 -- 172.21.15.27:0/113970316 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d4c0f6060 con 0x7f0d4c060390 2021-11-03T17:22:43.651 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.650+0000 7f0d517e6700 1 -- 172.21.15.27:0/113970316 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d4c045ad0 con 0x7f0d4c060390 2021-11-03T17:22:43.651 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.651+0000 7f0d337fe700 1 -- 172.21.15.27:0/113970316 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f0d44024060 con 0x7f0d4c060390 2021-11-03T17:22:43.651 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.651+0000 7f0d337fe700 1 --2- 172.21.15.27:0/113970316 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0d340640b0 0x7f0d34066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:22:43.652 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.651+0000 7f0d337fe700 1 -- 172.21.15.27:0/113970316 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f0d44081f10 con 0x7f0d4c060390 2021-11-03T17:22:43.652 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.651+0000 7f0d4a7fc700 1 --2- 172.21.15.27:0/113970316 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0d340640b0 0x7f0d34066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:22:43.652 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.652+0000 7f0d4a7fc700 1 --2- 172.21.15.27:0/113970316 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0d340640b0 0x7f0d34066570 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f0d40004e20 tx=0x7f0d4000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:22:43.653 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.655+0000 7f0d337fe700 1 -- 172.21.15.27:0/113970316 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f0d44054270 con 0x7f0d4c060390 2021-11-03T17:22:43.882 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.887+0000 7f0d517e6700 1 -- 172.21.15.27:0/113970316 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0d4c0586e0 con 0x7f0d4c060390 2021-11-03T17:22:43.883 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.888+0000 7f0d337fe700 1 -- 172.21.15.27:0/113970316 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f0d440222e0 con 0x7f0d4c060390 2021-11-03T17:22:43.883 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:22:43.883 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:22:43.883 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:22:43.884 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:22:43.884 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:22:43.884 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:22:43.884 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:22:43.885 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:22:43.885 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:22:43.885 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:22:43.885 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:22:43.885 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:22:43.886 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:22:43.886 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:22:43.887 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:22:43.887 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:22:43.887 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:22:43.887 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:22:43.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.890+0000 7f0d517e6700 1 -- 172.21.15.27:0/113970316 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0d340640b0 msgr2=0x7f0d34066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:22:43.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.890+0000 7f0d517e6700 1 --2- 172.21.15.27:0/113970316 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0d340640b0 0x7f0d34066570 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7f0d40004e20 tx=0x7f0d4000b040).stop 2021-11-03T17:22:43.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.890+0000 7f0d517e6700 1 -- 172.21.15.27:0/113970316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d4c060390 msgr2=0x7f0d4c0f6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:22:43.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.890+0000 7f0d517e6700 1 --2- 172.21.15.27:0/113970316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d4c060390 0x7f0d4c0f6de0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f0d4400c010 tx=0x7f0d44004d10).stop 2021-11-03T17:22:43.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.890+0000 7f0d517e6700 1 -- 172.21.15.27:0/113970316 shutdown_connections 2021-11-03T17:22:43.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.890+0000 7f0d517e6700 1 --2- 172.21.15.27:0/113970316 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0d340640b0 0x7f0d34066570 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:43.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.890+0000 7f0d517e6700 1 --2- 172.21.15.27:0/113970316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d4c060390 0x7f0d4c0f6de0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:43.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.890+0000 7f0d517e6700 1 --2- 172.21.15.27:0/113970316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d4c0f9850 0x7f0d4c0f5260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:43.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.890+0000 7f0d517e6700 1 -- 172.21.15.27:0/113970316 >> 172.21.15.27:0/113970316 conn(0x7f0d4c0f0de0 msgr2=0x7f0d4c0f31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:22:43.891 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.890+0000 7f0d517e6700 1 -- 172.21.15.27:0/113970316 shutdown_connections 2021-11-03T17:22:43.891 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.890+0000 7f0d517e6700 1 -- 172.21.15.27:0/113970316 wait complete. 2021-11-03T17:22:43.990 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.993+0000 7f5be5667700 1 -- 172.21.15.27:0/4241960337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5be00f55e0 msgr2=0x7f5be00f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:22:43.990 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.993+0000 7f5be5667700 1 --2- 172.21.15.27:0/4241960337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5be00f55e0 0x7f5be00f5a00 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f5bd80099e0 tx=0x7f5bd8004d10).stop 2021-11-03T17:22:43.990 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.993+0000 7f5be5667700 1 -- 172.21.15.27:0/4241960337 shutdown_connections 2021-11-03T17:22:43.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.993+0000 7f5be5667700 1 --2- 172.21.15.27:0/4241960337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5be00f6880 0x7f5be00f6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:22:43.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.993+0000 7f5be5667700 1 --2- 172.21.15.27:0/4241960337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5be00f55e0 0x7f5be00f5a00 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:43.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.993+0000 7f5be5667700 1 -- 172.21.15.27:0/4241960337 >> 172.21.15.27:0/4241960337 conn(0x7f5be00f0d40 msgr2=0x7f5be00f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:22:43.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.994+0000 7f5be5667700 1 -- 172.21.15.27:0/4241960337 shutdown_connections 2021-11-03T17:22:43.992 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.994+0000 7f5be5667700 1 -- 172.21.15.27:0/4241960337 wait complete. 2021-11-03T17:22:43.992 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.994+0000 7f5be5667700 1 Processor -- start 2021-11-03T17:22:43.992 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.994+0000 7f5be5667700 1 -- start start 2021-11-03T17:22:43.992 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.994+0000 7f5be5667700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5be00f55e0 0x7f5be0109e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:22:43.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.994+0000 7f5be5667700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5be00f6880 0x7f5be010c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:22:43.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.994+0000 7f5be5667700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be0060510 con 0x7f5be00f6880 2021-11-03T17:22:43.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.994+0000 7f5be5667700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5be0060650 con 0x7f5be00f55e0 2021-11-03T17:22:43.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.995+0000 7f5bde7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5be00f6880 0x7f5be010c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:22:43.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.995+0000 7f5bde7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5be00f6880 0x7f5be010c320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57106/0 (socket says 172.21.15.27:57106) 2021-11-03T17:22:43.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.995+0000 7f5bde7fc700 1 -- 172.21.15.27:0/4054877493 learned_addr learned my addr 172.21.15.27:0/4054877493 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:22:43.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.995+0000 7f5bdeffd700 1 --2- 172.21.15.27:0/4054877493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5be00f55e0 0x7f5be0109e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:22:43.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.995+0000 7f5bde7fc700 1 -- 172.21.15.27:0/4054877493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5be00f55e0 msgr2=0x7f5be0109e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:22:43.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.995+0000 7f5bde7fc700 1 --2- 172.21.15.27:0/4054877493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5be00f55e0 0x7f5be0109e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:43.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.995+0000 7f5bde7fc700 1 -- 172.21.15.27:0/4054877493 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd8009640 con 0x7f5be00f6880 2021-11-03T17:22:43.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.995+0000 7f5bde7fc700 1 --2- 172.21.15.27:0/4054877493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5be00f6880 0x7f5be010c320 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f5bc800b410 tx=0x7f5bc8006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:22:43.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.995+0000 7f5bd77fe700 1 -- 172.21.15.27:0/4054877493 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bc80125f0 con 0x7f5be00f6880 2021-11-03T17:22:43.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.995+0000 7f5bd77fe700 1 -- 172.21.15.27:0/4054877493 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f5bc8012c00 con 0x7f5be00f6880 2021-11-03T17:22:43.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.995+0000 7f5be5667700 1 -- 172.21.15.27:0/4054877493 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5be010c930 con 0x7f5be00f6880 2021-11-03T17:22:43.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.995+0000 7f5bd77fe700 1 -- 172.21.15.27:0/4054877493 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5bc8011d20 con 0x7f5be00f6880 2021-11-03T17:22:43.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.995+0000 7f5be5667700 1 -- 172.21.15.27:0/4054877493 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5be010ce70 con 0x7f5be00f6880 2021-11-03T17:22:43.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.996+0000 7f5be5667700 1 -- 172.21.15.27:0/4054877493 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5be0045ad0 con 0x7f5be00f6880 2021-11-03T17:22:43.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.997+0000 7f5bd77fe700 1 -- 172.21.15.27:0/4054877493 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f5bc800f030 con 0x7f5be00f6880 2021-11-03T17:22:43.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.997+0000 7f5bd77fe700 1 --2- 172.21.15.27:0/4054877493 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5bcc0640b0 0x7f5bcc066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:22:43.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.997+0000 7f5bd77fe700 1 -- 172.21.15.27:0/4054877493 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f5bc8080b10 con 0x7f5be00f6880 2021-11-03T17:22:43.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.997+0000 7f5bdeffd700 1 --2- 172.21.15.27:0/4054877493 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5bcc0640b0 0x7f5bcc066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:22:43.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:43.998+0000 7f5bdeffd700 1 --2- 172.21.15.27:0/4054877493 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5bcc0640b0 0x7f5bcc066570 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f5bd80099b0 tx=0x7f5bd8004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:22:43.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:44.001+0000 7f5bd77fe700 1 -- 172.21.15.27:0/4054877493 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f5bc8052e10 con 0x7f5be00f6880 2021-11-03T17:22:44.200 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:44.205+0000 7f5be5667700 1 -- 172.21.15.27:0/4054877493 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5be0027a00 con 0x7f5be00f6880 2021-11-03T17:22:44.200 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:44.206+0000 7f5bd77fe700 1 -- 172.21.15.27:0/4054877493 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f5bc8012d70 con 0x7f5be00f6880 2021-11-03T17:22:44.201 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:22:44.201 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:22:44.201 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:22:44.202 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:22:44.202 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:22:44.202 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:22:44.202 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:22:44.203 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:22:44.203 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:22:44.203 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:22:44.203 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:22:44.204 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:22:44.204 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:22:44.204 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:22:44.204 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:22:44.204 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:22:44.205 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:22:44.205 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:22:44.205 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:22:44.205 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:22:44.206 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:22:44.206 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:22:44.206 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:22:44.206 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:22:44.207 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:22:44.207 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:22:44.207 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:22:44.207 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:22:44.207 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:22:44.208 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:22:44.208 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:22:44.208 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:22:44.208 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:22:44.208 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:22:44.209 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:22:44.209 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:22:44.209 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:22:44.209 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:22:44.210 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:22:44.211 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:44.207+0000 7f5be5667700 1 -- 172.21.15.27:0/4054877493 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5bcc0640b0 msgr2=0x7f5bcc066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:22:44.211 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:44.207+0000 7f5be5667700 1 --2- 172.21.15.27:0/4054877493 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5bcc0640b0 0x7f5bcc066570 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f5bd80099b0 tx=0x7f5bd8004d10).stop 2021-11-03T17:22:44.211 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:44.208+0000 7f5be5667700 1 -- 172.21.15.27:0/4054877493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5be00f6880 msgr2=0x7f5be010c320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:22:44.211 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:44.208+0000 7f5be5667700 1 --2- 172.21.15.27:0/4054877493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5be00f6880 0x7f5be010c320 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f5bc800b410 tx=0x7f5bc8006ef0).stop 2021-11-03T17:22:44.212 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:44.208+0000 7f5be5667700 1 -- 172.21.15.27:0/4054877493 shutdown_connections 2021-11-03T17:22:44.212 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:44.208+0000 7f5be5667700 1 --2- 172.21.15.27:0/4054877493 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5bcc0640b0 0x7f5bcc066570 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:44.212 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:44.208+0000 7f5be5667700 1 --2- 172.21.15.27:0/4054877493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5be00f55e0 0x7f5be0109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:44.212 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:44.208+0000 7f5be5667700 1 --2- 172.21.15.27:0/4054877493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5be00f6880 0x7f5be010c320 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:22:44.212 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:44.208+0000 7f5be5667700 1 -- 172.21.15.27:0/4054877493 >> 172.21.15.27:0/4054877493 conn(0x7f5be00f0d40 msgr2=0x7f5be00fa2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:22:44.213 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:44.208+0000 7f5be5667700 1 -- 172.21.15.27:0/4054877493 shutdown_connections 2021-11-03T17:22:44.213 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:22:44.208+0000 7f5be5667700 1 -- 172.21.15.27:0/4054877493 wait complete. 2021-11-03T17:22:44.213 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:22:44.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:43 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/113970316' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:44.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:43 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/113970316' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:45.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:44 smithi115 ceph-mon[65453]: from='client.34360 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:22:45.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:44 smithi115 ceph-mon[65453]: from='client.34364 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:22:45.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:44 smithi115 ceph-mon[65453]: pgmap v241: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:45.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:44 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/4054877493' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:22:45.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:44 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 407.368654 secs 2021-11-03T17:22:45.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:44 smithi027 ceph-mon[97244]: from='client.34360 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:22:45.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:44 smithi027 ceph-mon[97244]: from='client.34364 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:22:45.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:44 smithi027 ceph-mon[97244]: pgmap v241: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:45.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:44 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/4054877493' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:22:45.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:44 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 407.368654 secs 2021-11-03T17:22:46.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:46 smithi115 ceph-mon[65453]: pgmap v242: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:46.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:22:46.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:22:46.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:22:46.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:46.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:46.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:46.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:47.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:46 smithi027 ceph-mon[97244]: pgmap v242: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:47.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:22:47.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:22:47.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:22:47.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:47.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:47.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:47.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:48.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:47 smithi115 ceph-mon[65453]: poll took 60181.052 ms: 1 events 2021-11-03T17:22:48.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:47 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:22:48.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:47 smithi027 ceph-mon[97244]: poll took 60181.052 ms: 1 events 2021-11-03T17:22:48.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:47 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:22:49.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:48 smithi115 ceph-mon[65453]: pgmap v243: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:49.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:48 smithi027 ceph-mon[97244]: pgmap v243: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:50.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:49 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 412.368737 secs 2021-11-03T17:22:50.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:49 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 412.368737 secs 2021-11-03T17:22:51.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:50 smithi115 ceph-mon[65453]: pgmap v244: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:51.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:50 smithi027 ceph-mon[97244]: pgmap v244: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:53.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:52 smithi115 ceph-mon[65453]: pgmap v245: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:53.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:52 smithi027 ceph-mon[97244]: pgmap v245: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:55.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:54 smithi115 ceph-mon[65453]: pgmap v246: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:55.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:54 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 417.368851 secs 2021-11-03T17:22:55.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:54 smithi027 ceph-mon[97244]: pgmap v246: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:55.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:54 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 417.368851 secs 2021-11-03T17:22:56.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:22:56.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:22:56.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:22:56.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:22:57.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:56 smithi115 ceph-mon[65453]: pgmap v247: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:57.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:22:57.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:22:57.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:22:57.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:57.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:57.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:57.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:57.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:56 smithi027 ceph-mon[97244]: pgmap v247: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:57.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:22:57.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:22:57.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:22:57.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:57.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:57.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:57.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:22:58.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:57 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:22:58.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:57 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:22:59.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:58 smithi115 ceph-mon[65453]: pgmap v248: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:22:59.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:58 smithi027 ceph-mon[97244]: pgmap v248: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:00.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:22:59 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 422.368991 secs 2021-11-03T17:23:00.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:22:59 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 422.368991 secs 2021-11-03T17:23:01.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:00 smithi027 ceph-mon[97244]: pgmap v249: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:01.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:00 smithi115 ceph-mon[65453]: pgmap v249: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:03.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:02 smithi027 ceph-mon[97244]: pgmap v250: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:03.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:02 smithi115 ceph-mon[65453]: pgmap v250: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:05.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:04 smithi027 ceph-mon[97244]: pgmap v251: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:05.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:04 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 427.369120 secs 2021-11-03T17:23:05.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:04 smithi115 ceph-mon[65453]: pgmap v251: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:05.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:04 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 427.369120 secs 2021-11-03T17:23:06.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:06 smithi115 ceph-mon[65453]: pgmap v252: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:06.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:06 smithi027 ceph-mon[97244]: pgmap v252: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:07.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:23:07.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:23:07.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:23:07.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:07.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:07.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:07.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:07.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:07 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:23:07.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:23:07.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:23:07.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:23:07.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:07.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:07.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:07.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:07.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:07 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:23:08.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:08 smithi115 ceph-mon[65453]: pgmap v253: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:08.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:08 smithi027 ceph-mon[97244]: pgmap v253: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:09.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:09 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 432.369255 secs 2021-11-03T17:23:09.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:09 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 432.369255 secs 2021-11-03T17:23:10.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:10 smithi027 ceph-mon[97244]: pgmap v254: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:10.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:10 smithi115 ceph-mon[65453]: pgmap v254: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:13.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:12 smithi115 ceph-mon[65453]: pgmap v255: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:13.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:12 smithi027 ceph-mon[97244]: pgmap v255: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:14.318 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.323+0000 7f66911c1700 1 -- 172.21.15.27:0/4187122411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f668c06b940 msgr2=0x7f668c069e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:23:14.318 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.323+0000 7f66911c1700 1 --2- 172.21.15.27:0/4187122411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f668c06b940 0x7f668c069e40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f667c004660 tx=0x7f667c009b30).stop 2021-11-03T17:23:14.319 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.323+0000 7f66911c1700 1 -- 172.21.15.27:0/4187122411 shutdown_connections 2021-11-03T17:23:14.319 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.323+0000 7f66911c1700 1 --2- 172.21.15.27:0/4187122411 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f668c06a330 0x7f668c06a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:14.319 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.323+0000 7f66911c1700 1 --2- 172.21.15.27:0/4187122411 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f668c06b940 0x7f668c069e40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:14.319 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.323+0000 7f66911c1700 1 -- 172.21.15.27:0/4187122411 >> 172.21.15.27:0/4187122411 conn(0x7f668c0f1e40 msgr2=0x7f668c0f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:23:14.320 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.323+0000 7f66911c1700 1 -- 172.21.15.27:0/4187122411 shutdown_connections 2021-11-03T17:23:14.320 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.323+0000 7f66911c1700 1 -- 172.21.15.27:0/4187122411 wait complete. 2021-11-03T17:23:14.320 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.324+0000 7f66911c1700 1 Processor -- start 2021-11-03T17:23:14.320 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.324+0000 7f66911c1700 1 -- start start 2021-11-03T17:23:14.323 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.324+0000 7f66911c1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f668c06a330 0x7f668c109160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:23:14.323 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.324+0000 7f66911c1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f668c06b940 0x7f668c109650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:23:14.323 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.324+0000 7f66911c1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668c109c00 con 0x7f668c06b940 2021-11-03T17:23:14.323 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.324+0000 7f66911c1700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668c109d40 con 0x7f668c06a330 2021-11-03T17:23:14.324 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.324+0000 7f6683fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f668c06b940 0x7f668c109650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:23:14.324 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.324+0000 7f6683fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f668c06b940 0x7f668c109650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57124/0 (socket says 172.21.15.27:57124) 2021-11-03T17:23:14.324 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.324+0000 7f6683fff700 1 -- 172.21.15.27:0/4049468525 learned_addr learned my addr 172.21.15.27:0/4049468525 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:23:14.324 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.324+0000 7f668ad9d700 1 --2- 172.21.15.27:0/4049468525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f668c06a330 0x7f668c109160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:23:14.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.325+0000 7f6683fff700 1 -- 172.21.15.27:0/4049468525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f668c06a330 msgr2=0x7f668c109160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:23:14.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.325+0000 7f6683fff700 1 --2- 172.21.15.27:0/4049468525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f668c06a330 0x7f668c109160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:14.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.325+0000 7f6683fff700 1 -- 172.21.15.27:0/4049468525 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f667c005040 con 0x7f668c06b940 2021-11-03T17:23:14.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.325+0000 7f6683fff700 1 --2- 172.21.15.27:0/4049468525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f668c06b940 0x7f668c109650 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f667400c530 tx=0x7f6674007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:23:14.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.325+0000 7f66837fe700 1 -- 172.21.15.27:0/4049468525 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f667400f650 con 0x7f668c06b940 2021-11-03T17:23:14.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.325+0000 7f66837fe700 1 -- 172.21.15.27:0/4049468525 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f667400fc60 con 0x7f668c06b940 2021-11-03T17:23:14.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.325+0000 7f66911c1700 1 -- 172.21.15.27:0/4049468525 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f668c10a6e0 con 0x7f668c06b940 2021-11-03T17:23:14.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.325+0000 7f66837fe700 1 -- 172.21.15.27:0/4049468525 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6674012c20 con 0x7f668c06b940 2021-11-03T17:23:14.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.325+0000 7f66911c1700 1 -- 172.21.15.27:0/4049468525 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f668c10aca0 con 0x7f668c06b940 2021-11-03T17:23:14.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.326+0000 7f66911c1700 1 -- 172.21.15.27:0/4049468525 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f668c045ad0 con 0x7f668c06b940 2021-11-03T17:23:14.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.326+0000 7f66837fe700 1 -- 172.21.15.27:0/4049468525 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f6674010070 con 0x7f668c06b940 2021-11-03T17:23:14.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.327+0000 7f66837fe700 1 --2- 172.21.15.27:0/4049468525 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f66780640b0 0x7f6678066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:23:14.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.327+0000 7f66837fe700 1 -- 172.21.15.27:0/4049468525 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f6674080550 con 0x7f668c06b940 2021-11-03T17:23:14.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.327+0000 7f668ad9d700 1 --2- 172.21.15.27:0/4049468525 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f66780640b0 0x7f6678066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:23:14.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.327+0000 7f668ad9d700 1 --2- 172.21.15.27:0/4049468525 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f66780640b0 0x7f6678066570 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f667c004630 tx=0x7f667c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:23:14.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.331+0000 7f66837fe700 1 -- 172.21.15.27:0/4049468525 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f66740527c0 con 0x7f668c06b940 2021-11-03T17:23:14.503 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.509+0000 7f66911c1700 1 -- 172.21.15.27:0/4049468525 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f668c10af10 con 0x7f66780640b0 2021-11-03T17:23:14.504 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.510+0000 7f66837fe700 1 -- 172.21.15.27:0/4049468525 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f668c10af10 con 0x7f66780640b0 2021-11-03T17:23:14.506 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.512+0000 7f66911c1700 1 -- 172.21.15.27:0/4049468525 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f66780640b0 msgr2=0x7f6678066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:23:14.507 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.512+0000 7f66911c1700 1 --2- 172.21.15.27:0/4049468525 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f66780640b0 0x7f6678066570 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f667c004630 tx=0x7f667c009b30).stop 2021-11-03T17:23:14.507 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.512+0000 7f66911c1700 1 -- 172.21.15.27:0/4049468525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f668c06b940 msgr2=0x7f668c109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:23:14.507 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.512+0000 7f66911c1700 1 --2- 172.21.15.27:0/4049468525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f668c06b940 0x7f668c109650 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f667400c530 tx=0x7f6674007e20).stop 2021-11-03T17:23:14.507 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.512+0000 7f66911c1700 1 -- 172.21.15.27:0/4049468525 shutdown_connections 2021-11-03T17:23:14.508 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.512+0000 7f66911c1700 1 --2- 172.21.15.27:0/4049468525 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f66780640b0 0x7f6678066570 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:14.508 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.512+0000 7f66911c1700 1 --2- 172.21.15.27:0/4049468525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f668c06a330 0x7f668c109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:14.508 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.512+0000 7f66911c1700 1 --2- 172.21.15.27:0/4049468525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f668c06b940 0x7f668c109650 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:14.508 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.512+0000 7f66911c1700 1 -- 172.21.15.27:0/4049468525 >> 172.21.15.27:0/4049468525 conn(0x7f668c0f1e40 msgr2=0x7f668c0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:23:14.509 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.512+0000 7f66911c1700 1 -- 172.21.15.27:0/4049468525 shutdown_connections 2021-11-03T17:23:14.509 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.512+0000 7f66911c1700 1 -- 172.21.15.27:0/4049468525 wait complete. 2021-11-03T17:23:14.516 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:23:14.610 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.613+0000 7f6ae9b27700 1 -- 172.21.15.27:0/3193412090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae40f6ff0 msgr2=0x7f6ae40f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:23:14.610 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.613+0000 7f6ae9b27700 1 --2- 172.21.15.27:0/3193412090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae40f6ff0 0x7f6ae40f7a20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f6acc004660 tx=0x7f6acc009b30).stop 2021-11-03T17:23:14.611 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.613+0000 7f6ae9b27700 1 -- 172.21.15.27:0/3193412090 shutdown_connections 2021-11-03T17:23:14.611 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.613+0000 7f6ae9b27700 1 --2- 172.21.15.27:0/3193412090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae40f6ff0 0x7f6ae40f7a20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:14.611 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.613+0000 7f6ae9b27700 1 --2- 172.21.15.27:0/3193412090 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ae40f66e0 0x7f6ae40f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:14.611 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.613+0000 7f6ae9b27700 1 -- 172.21.15.27:0/3193412090 >> 172.21.15.27:0/3193412090 conn(0x7f6ae40f1f00 msgr2=0x7f6ae40f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:23:14.612 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.613+0000 7f6ae9b27700 1 -- 172.21.15.27:0/3193412090 shutdown_connections 2021-11-03T17:23:14.612 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.613+0000 7f6ae9b27700 1 -- 172.21.15.27:0/3193412090 wait complete. 2021-11-03T17:23:14.612 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.613+0000 7f6ae9b27700 1 Processor -- start 2021-11-03T17:23:14.612 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.613+0000 7f6ae9b27700 1 -- start start 2021-11-03T17:23:14.613 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.614+0000 7f6ae9b27700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae40f66e0 0x7f6ae4106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:23:14.613 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.614+0000 7f6ae9b27700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ae40f6ff0 0x7f6ae4107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:23:14.613 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.614+0000 7f6ae9b27700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae4107a10 con 0x7f6ae40f66e0 2021-11-03T17:23:14.613 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.614+0000 7f6ae9b27700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae4107b50 con 0x7f6ae40f6ff0 2021-11-03T17:23:14.614 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.614+0000 7f6ae37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae40f66e0 0x7f6ae4106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:23:14.614 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.614+0000 7f6ae37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae40f66e0 0x7f6ae4106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57132/0 (socket says 172.21.15.27:57132) 2021-11-03T17:23:14.614 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.614+0000 7f6ae37fe700 1 -- 172.21.15.27:0/2311292183 learned_addr learned my addr 172.21.15.27:0/2311292183 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:23:14.614 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.614+0000 7f6ae2ffd700 1 --2- 172.21.15.27:0/2311292183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ae40f6ff0 0x7f6ae4107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:23:14.615 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.614+0000 7f6ae37fe700 1 -- 172.21.15.27:0/2311292183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ae40f6ff0 msgr2=0x7f6ae4107460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:23:14.615 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.614+0000 7f6ae37fe700 1 --2- 172.21.15.27:0/2311292183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ae40f6ff0 0x7f6ae4107460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:14.615 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.614+0000 7f6ae37fe700 1 -- 172.21.15.27:0/2311292183 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6acc005040 con 0x7f6ae40f66e0 2021-11-03T17:23:14.615 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.614+0000 7f6ae37fe700 1 --2- 172.21.15.27:0/2311292183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae40f66e0 0x7f6ae4106f70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f6ad4007fd0 tx=0x7f6ad4004ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:23:14.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.614+0000 7f6adbfff700 1 -- 172.21.15.27:0/2311292183 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ad400b4f0 con 0x7f6ae40f66e0 2021-11-03T17:23:14.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.614+0000 7f6adbfff700 1 -- 172.21.15.27:0/2311292183 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f6ad400f040 con 0x7f6ae40f66e0 2021-11-03T17:23:14.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.614+0000 7f6adbfff700 1 -- 172.21.15.27:0/2311292183 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ad4013a50 con 0x7f6ae40f66e0 2021-11-03T17:23:14.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.614+0000 7f6ae9b27700 1 -- 172.21.15.27:0/2311292183 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ae41084f0 con 0x7f6ae40f66e0 2021-11-03T17:23:14.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.614+0000 7f6ae9b27700 1 -- 172.21.15.27:0/2311292183 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ae4108ab0 con 0x7f6ae40f66e0 2021-11-03T17:23:14.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.615+0000 7f6ae9b27700 1 -- 172.21.15.27:0/2311292183 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ae4045ad0 con 0x7f6ae40f66e0 2021-11-03T17:23:14.618 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.616+0000 7f6adbfff700 1 -- 172.21.15.27:0/2311292183 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f6ad401a030 con 0x7f6ae40f66e0 2021-11-03T17:23:14.619 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.616+0000 7f6adbfff700 1 --2- 172.21.15.27:0/2311292183 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6ad00640b0 0x7f6ad0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:23:14.619 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.616+0000 7f6adbfff700 1 -- 172.21.15.27:0/2311292183 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f6ad4080710 con 0x7f6ae40f66e0 2021-11-03T17:23:14.619 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.616+0000 7f6ae2ffd700 1 --2- 172.21.15.27:0/2311292183 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6ad00640b0 0x7f6ad0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:23:14.620 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.617+0000 7f6ae2ffd700 1 --2- 172.21.15.27:0/2311292183 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6ad00640b0 0x7f6ad0066570 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f6acc004630 tx=0x7f6acc009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:23:14.620 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.620+0000 7f6adbfff700 1 -- 172.21.15.27:0/2311292183 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f6ad4052af0 con 0x7f6ae40f66e0 2021-11-03T17:23:14.788 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.793+0000 7f6ae9b27700 1 -- 172.21.15.27:0/2311292183 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ae40ff050 con 0x7f6ad00640b0 2021-11-03T17:23:14.802 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.808+0000 7f6adbfff700 1 -- 172.21.15.27:0/2311292183 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f6ae40ff050 con 0x7f6ad00640b0 2021-11-03T17:23:14.802 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:23:14.803 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (14m) 5m ago 16m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:23:14.803 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (6m) 5m ago 16m 7205k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:23:14.803 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (6m) 3m ago 15m 7239k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:23:14.803 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (14m) 5m ago 15m 38.9M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:23:14.804 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (8m) 5m ago 11m 15.4M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:23:14.804 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (11m) 5m ago 11m 3836M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:23:14.804 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (7m) 3m ago 11m 14.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:23:14.804 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (8m) 3m ago 11m 16.3M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:23:14.805 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (8m) 5m ago 17m 470M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:23:14.805 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (7m) 3m ago 15m 412M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:23:14.805 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (7m) 5m ago 17m 81.2M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:23:14.805 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (7m) 3m ago 15m 69.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:23:14.806 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (15m) 5m ago 15m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:23:14.806 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (15m) 3m ago 15m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:23:14.806 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (6m) 5m ago 14m 243M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:23:14.806 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (5m) 5m ago 14m 270M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:23:14.806 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (5m) 5m ago 13m 11.8M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:23:14.807 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (4m) 3m ago 13m 289M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:23:14.807 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (4m) 3m ago 13m 235M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:23:14.807 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (3m) 3m ago 12m 11.5M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:23:14.807 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (14m) 5m ago 15m 45.7M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:23:14.808 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.810+0000 7f6ae9b27700 1 -- 172.21.15.27:0/2311292183 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6ad00640b0 msgr2=0x7f6ad0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:23:14.808 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.810+0000 7f6ae9b27700 1 --2- 172.21.15.27:0/2311292183 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6ad00640b0 0x7f6ad0066570 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f6acc004630 tx=0x7f6acc009b30).stop 2021-11-03T17:23:14.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.810+0000 7f6ae9b27700 1 -- 172.21.15.27:0/2311292183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae40f66e0 msgr2=0x7f6ae4106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:23:14.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.810+0000 7f6ae9b27700 1 --2- 172.21.15.27:0/2311292183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae40f66e0 0x7f6ae4106f70 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f6ad4007fd0 tx=0x7f6ad4004ee0).stop 2021-11-03T17:23:14.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.810+0000 7f6ae9b27700 1 -- 172.21.15.27:0/2311292183 shutdown_connections 2021-11-03T17:23:14.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.810+0000 7f6ae9b27700 1 --2- 172.21.15.27:0/2311292183 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6ad00640b0 0x7f6ad0066570 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:14.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.810+0000 7f6ae9b27700 1 --2- 172.21.15.27:0/2311292183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ae40f66e0 0x7f6ae4106f70 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:14.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.810+0000 7f6ae9b27700 1 --2- 172.21.15.27:0/2311292183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ae40f6ff0 0x7f6ae4107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:14.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.810+0000 7f6ae9b27700 1 -- 172.21.15.27:0/2311292183 >> 172.21.15.27:0/2311292183 conn(0x7f6ae40f1f00 msgr2=0x7f6ae40fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:23:14.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.810+0000 7f6ae9b27700 1 -- 172.21.15.27:0/2311292183 shutdown_connections 2021-11-03T17:23:14.811 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.810+0000 7f6ae9b27700 1 -- 172.21.15.27:0/2311292183 wait complete. 2021-11-03T17:23:14.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.910+0000 7f3aeff4b700 1 -- 172.21.15.27:0/3387809934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ae80f55d0 msgr2=0x7f3ae80f59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:23:14.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.910+0000 7f3aeff4b700 1 --2- 172.21.15.27:0/3387809934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ae80f55d0 0x7f3ae80f59f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f3ae00099e0 tx=0x7f3ae0004d10).stop 2021-11-03T17:23:14.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.910+0000 7f3aeff4b700 1 -- 172.21.15.27:0/3387809934 shutdown_connections 2021-11-03T17:23:14.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.910+0000 7f3aeff4b700 1 --2- 172.21.15.27:0/3387809934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ae80f6870 0x7f3ae80f6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:14.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.910+0000 7f3aeff4b700 1 --2- 172.21.15.27:0/3387809934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ae80f55d0 0x7f3ae80f59f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:14.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.910+0000 7f3aeff4b700 1 -- 172.21.15.27:0/3387809934 >> 172.21.15.27:0/3387809934 conn(0x7f3ae80f0d70 msgr2=0x7f3ae80f3190 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:23:14.908 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.910+0000 7f3aeff4b700 1 -- 172.21.15.27:0/3387809934 shutdown_connections 2021-11-03T17:23:14.908 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.910+0000 7f3aeff4b700 1 -- 172.21.15.27:0/3387809934 wait complete. 2021-11-03T17:23:14.908 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.910+0000 7f3aeff4b700 1 Processor -- start 2021-11-03T17:23:14.908 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.911+0000 7f3aeff4b700 1 -- start start 2021-11-03T17:23:14.909 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.911+0000 7f3aeff4b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ae80f55d0 0x7f3ae8109d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:23:14.909 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.911+0000 7f3aeff4b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ae80f6870 0x7f3ae810c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:23:14.910 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.911+0000 7f3aeff4b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae8060500 con 0x7f3ae80f6870 2021-11-03T17:23:14.910 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.911+0000 7f3aeff4b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae8060640 con 0x7f3ae80f55d0 2021-11-03T17:23:14.910 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.911+0000 7f3aed4e6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ae80f6870 0x7f3ae810c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:23:14.910 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.911+0000 7f3aed4e6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ae80f6870 0x7f3ae810c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57144/0 (socket says 172.21.15.27:57144) 2021-11-03T17:23:14.911 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.911+0000 7f3aed4e6700 1 -- 172.21.15.27:0/3056244575 learned_addr learned my addr 172.21.15.27:0/3056244575 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:23:14.911 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.911+0000 7f3aedce7700 1 --2- 172.21.15.27:0/3056244575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ae80f55d0 0x7f3ae8109d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:23:14.911 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.911+0000 7f3aed4e6700 1 -- 172.21.15.27:0/3056244575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ae80f55d0 msgr2=0x7f3ae8109d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:23:14.911 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.911+0000 7f3aed4e6700 1 --2- 172.21.15.27:0/3056244575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ae80f55d0 0x7f3ae8109d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:14.912 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.911+0000 7f3aed4e6700 1 -- 172.21.15.27:0/3056244575 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae0009640 con 0x7f3ae80f6870 2021-11-03T17:23:14.912 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.911+0000 7f3aedce7700 1 --2- 172.21.15.27:0/3056244575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ae80f55d0 0x7f3ae8109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:23:14.912 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.911+0000 7f3aed4e6700 1 --2- 172.21.15.27:0/3056244575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ae80f6870 0x7f3ae810c220 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f3adc0026d0 tx=0x7f3adc009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:23:14.913 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.911+0000 7f3ae67fc700 1 -- 172.21.15.27:0/3056244575 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3adc009320 con 0x7f3ae80f6870 2021-11-03T17:23:14.913 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.911+0000 7f3ae67fc700 1 -- 172.21.15.27:0/3056244575 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f3adc00f040 con 0x7f3ae80f6870 2021-11-03T17:23:14.914 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.911+0000 7f3aeff4b700 1 -- 172.21.15.27:0/3056244575 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ae810c7d0 con 0x7f3ae80f6870 2021-11-03T17:23:14.914 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.911+0000 7f3ae67fc700 1 -- 172.21.15.27:0/3056244575 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3adc0147e0 con 0x7f3ae80f6870 2021-11-03T17:23:14.914 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.911+0000 7f3aeff4b700 1 -- 172.21.15.27:0/3056244575 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ae810cce0 con 0x7f3ae80f6870 2021-11-03T17:23:14.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.912+0000 7f3aeff4b700 1 -- 172.21.15.27:0/3056244575 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ae80f49c0 con 0x7f3ae80f6870 2021-11-03T17:23:14.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.913+0000 7f3ae67fc700 1 -- 172.21.15.27:0/3056244575 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f3adc009480 con 0x7f3ae80f6870 2021-11-03T17:23:14.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.913+0000 7f3ae67fc700 1 --2- 172.21.15.27:0/3056244575 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3ad00640b0 0x7f3ad0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:23:14.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.913+0000 7f3ae67fc700 1 -- 172.21.15.27:0/3056244575 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f3adc080eb0 con 0x7f3ae80f6870 2021-11-03T17:23:14.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.913+0000 7f3aedce7700 1 --2- 172.21.15.27:0/3056244575 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3ad00640b0 0x7f3ad0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:23:14.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.914+0000 7f3aedce7700 1 --2- 172.21.15.27:0/3056244575 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3ad00640b0 0x7f3ad0066570 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f3ae0000f80 tx=0x7f3ae0004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:23:14.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:14.917+0000 7f3ae67fc700 1 -- 172.21.15.27:0/3056244575 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f3adc053210 con 0x7f3ae80f6870 2021-11-03T17:23:15.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.157+0000 7f3aeff4b700 1 -- 172.21.15.27:0/3056244575 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3ae8045ad0 con 0x7f3ae80f6870 2021-11-03T17:23:15.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.158+0000 7f3ae67fc700 1 -- 172.21.15.27:0/3056244575 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f3adc052e00 con 0x7f3ae80f6870 2021-11-03T17:23:15.153 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:23:15.153 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:23:15.153 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:23:15.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:23:15.154 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:23:15.154 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:23:15.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:23:15.154 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:23:15.155 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:23:15.155 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:23:15.155 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:23:15.155 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:23:15.156 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:23:15.156 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:23:15.156 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:23:15.156 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:23:15.156 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:23:15.157 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:23:15.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.159+0000 7f3aeff4b700 1 -- 172.21.15.27:0/3056244575 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3ad00640b0 msgr2=0x7f3ad0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:23:15.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.159+0000 7f3aeff4b700 1 --2- 172.21.15.27:0/3056244575 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3ad00640b0 0x7f3ad0066570 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f3ae0000f80 tx=0x7f3ae0004d10).stop 2021-11-03T17:23:15.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.159+0000 7f3aeff4b700 1 -- 172.21.15.27:0/3056244575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ae80f6870 msgr2=0x7f3ae810c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:23:15.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.159+0000 7f3aeff4b700 1 --2- 172.21.15.27:0/3056244575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ae80f6870 0x7f3ae810c220 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f3adc0026d0 tx=0x7f3adc009de0).stop 2021-11-03T17:23:15.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.159+0000 7f3aeff4b700 1 -- 172.21.15.27:0/3056244575 shutdown_connections 2021-11-03T17:23:15.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.159+0000 7f3aeff4b700 1 --2- 172.21.15.27:0/3056244575 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3ad00640b0 0x7f3ad0066570 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:15.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.159+0000 7f3aeff4b700 1 --2- 172.21.15.27:0/3056244575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ae80f55d0 0x7f3ae8109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:15.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.159+0000 7f3aeff4b700 1 --2- 172.21.15.27:0/3056244575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ae80f6870 0x7f3ae810c220 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:15.160 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.159+0000 7f3aeff4b700 1 -- 172.21.15.27:0/3056244575 >> 172.21.15.27:0/3056244575 conn(0x7f3ae80f0d70 msgr2=0x7f3ae80fa2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:23:15.160 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.159+0000 7f3aeff4b700 1 -- 172.21.15.27:0/3056244575 shutdown_connections 2021-11-03T17:23:15.160 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.160+0000 7f3aeff4b700 1 -- 172.21.15.27:0/3056244575 wait complete. 2021-11-03T17:23:15.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:14 smithi115 ceph-mon[65453]: pgmap v256: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:15.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:14 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 437.369321 secs 2021-11-03T17:23:15.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.260+0000 7ff0dc6b5700 1 -- 172.21.15.27:0/3328622075 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0d406a3b0 msgr2=0x7ff0d406a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:23:15.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.260+0000 7ff0dc6b5700 1 --2- 172.21.15.27:0/3328622075 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0d406a3b0 0x7ff0d406a830 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7ff0c4004660 tx=0x7ff0c4009b30).stop 2021-11-03T17:23:15.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.261+0000 7ff0dc6b5700 1 -- 172.21.15.27:0/3328622075 shutdown_connections 2021-11-03T17:23:15.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.261+0000 7ff0dc6b5700 1 --2- 172.21.15.27:0/3328622075 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0d406a3b0 0x7ff0d406a830 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:15.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.261+0000 7ff0dc6b5700 1 --2- 172.21.15.27:0/3328622075 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0d406b920 0x7ff0d4069ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:15.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.261+0000 7ff0dc6b5700 1 -- 172.21.15.27:0/3328622075 >> 172.21.15.27:0/3328622075 conn(0x7ff0d40f1f00 msgr2=0x7ff0d40f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:23:15.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.261+0000 7ff0dc6b5700 1 -- 172.21.15.27:0/3328622075 shutdown_connections 2021-11-03T17:23:15.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.261+0000 7ff0dc6b5700 1 -- 172.21.15.27:0/3328622075 wait complete. 2021-11-03T17:23:15.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.261+0000 7ff0dc6b5700 1 Processor -- start 2021-11-03T17:23:15.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.261+0000 7ff0dc6b5700 1 -- start start 2021-11-03T17:23:15.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.261+0000 7ff0dc6b5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0d406a3b0 0x7ff0d410d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:23:15.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.261+0000 7ff0dc6b5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0d406b920 0x7ff0d410da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:23:15.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.261+0000 7ff0dc6b5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d410e030 con 0x7ff0d406b920 2021-11-03T17:23:15.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.261+0000 7ff0dc6b5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0d410e170 con 0x7ff0d406a3b0 2021-11-03T17:23:15.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.261+0000 7ff0d9c50700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0d406b920 0x7ff0d410da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:23:15.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.261+0000 7ff0d9c50700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0d406b920 0x7ff0d410da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57154/0 (socket says 172.21.15.27:57154) 2021-11-03T17:23:15.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.261+0000 7ff0d9c50700 1 -- 172.21.15.27:0/1910877846 learned_addr learned my addr 172.21.15.27:0/1910877846 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:23:15.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.262+0000 7ff0d9c50700 1 -- 172.21.15.27:0/1910877846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0d406a3b0 msgr2=0x7ff0d410d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:23:15.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.262+0000 7ff0d9c50700 1 --2- 172.21.15.27:0/1910877846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0d406a3b0 0x7ff0d410d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:23:15.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.262+0000 7ff0d9c50700 1 -- 172.21.15.27:0/1910877846 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c4005040 con 0x7ff0d406b920 2021-11-03T17:23:15.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.262+0000 7ff0d9c50700 1 --2- 172.21.15.27:0/1910877846 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0d406b920 0x7ff0d410da80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7ff0c4009990 tx=0x7ff0c4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:23:15.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.262+0000 7ff0caffd700 1 -- 172.21.15.27:0/1910877846 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0c4018070 con 0x7ff0d406b920 2021-11-03T17:23:15.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.262+0000 7ff0caffd700 1 -- 172.21.15.27:0/1910877846 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7ff0c4007ca0 con 0x7ff0d406b920 2021-11-03T17:23:15.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.262+0000 7ff0dc6b5700 1 -- 172.21.15.27:0/1910877846 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0d410eab0 con 0x7ff0d406b920 2021-11-03T17:23:15.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.262+0000 7ff0caffd700 1 -- 172.21.15.27:0/1910877846 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0c401c600 con 0x7ff0d406b920 2021-11-03T17:23:15.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.262+0000 7ff0dc6b5700 1 -- 172.21.15.27:0/1910877846 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d410f010 con 0x7ff0d406b920 2021-11-03T17:23:15.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.263+0000 7ff0dc6b5700 1 -- 172.21.15.27:0/1910877846 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0d4045ad0 con 0x7ff0d406b920 2021-11-03T17:23:15.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.264+0000 7ff0caffd700 1 -- 172.21.15.27:0/1910877846 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7ff0c401d9d0 con 0x7ff0d406b920 2021-11-03T17:23:15.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.264+0000 7ff0caffd700 1 --2- 172.21.15.27:0/1910877846 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff0c00640b0 0x7ff0c0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:23:15.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.264+0000 7ff0caffd700 1 -- 172.21.15.27:0/1910877846 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6461+0+0 (secure 0 0 0) 0x7ff0c4082550 con 0x7ff0d406b920 2021-11-03T17:23:15.265 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.264+0000 7ff0da451700 1 --2- 172.21.15.27:0/1910877846 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff0c00640b0 0x7ff0c0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:23:15.265 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.264+0000 7ff0da451700 1 --2- 172.21.15.27:0/1910877846 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff0c00640b0 0x7ff0c0066570 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7ff0d406b6f0 tx=0x7ff0d0007a60).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:23:15.265 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.268+0000 7ff0caffd700 1 -- 172.21.15.27:0/1910877846 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7ff0c4054970 con 0x7ff0d406b920 2021-11-03T17:23:15.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:14 smithi027 ceph-mon[97244]: pgmap v256: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:15.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:14 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 437.369321 secs 2021-11-03T17:23:15.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.473+0000 7ff0dc6b5700 1 -- 172.21.15.27:0/1910877846 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff0d410f280 con 0x7ff0d406b920 2021-11-03T17:23:15.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.474+0000 7ff0caffd700 1 -- 172.21.15.27:0/1910877846 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7ff0c4054560 con 0x7ff0d406b920 2021-11-03T17:23:15.469 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:23:15.469 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:23:15.469 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:23:15.469 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:23:15.470 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:23:15.470 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:23:15.470 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:23:15.470 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:23:15.471 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:23:15.471 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:23:15.471 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:23:15.471 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:23:15.472 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:23:15.472 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:23:15.472 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:23:15.472 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:23:15.472 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:23:15.473 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:23:15.473 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:23:15.473 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:23:15.473 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:23:15.474 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:23:15.474 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:23:15.474 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:23:15.474 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:23:15.474 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:23:15.475 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:23:15.475 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:23:15.475 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:23:15.476 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:23:15.476 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:23:15.476 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:23:15.476 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:23:15.476 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:23:15.477 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:23:15.477 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:23:15.477 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:23:15.477 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:23:15.477 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:23:15.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.475+0000 7ff0dc6b5700 1 -- 172.21.15.27:0/1910877846 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff0c00640b0 msgr2=0x7ff0c0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:23:15.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.475+0000 7ff0dc6b5700 1 --2- 172.21.15.27:0/1910877846 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff0c00640b0 0x7ff0c0066570 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7ff0d406b6f0 tx=0x7ff0d0007a60).stop 2021-11-03T17:23:15.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.475+0000 7ff0dc6b5700 1 -- 172.21.15.27:0/1910877846 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0d406b920 msgr2=0x7ff0d410da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:23:15.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.475+0000 7ff0dc6b5700 1 --2- 172.21.15.27:0/1910877846 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0d406b920 0x7ff0d410da80 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7ff0c4009990 tx=0x7ff0c4009b30).stop 2021-11-03T17:23:15.480 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.475+0000 7ff0dc6b5700 1 -- 172.21.15.27:0/1910877846 shutdown_connections 2021-11-03T17:23:15.480 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.476+0000 7ff0dc6b5700 1 --2- 172.21.15.27:0/1910877846 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff0c00640b0 0x7ff0c0066570 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:15.480 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.476+0000 7ff0dc6b5700 1 --2- 172.21.15.27:0/1910877846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff0d406a3b0 0x7ff0d410d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:23:15.480 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.476+0000 7ff0dc6b5700 1 --2- 172.21.15.27:0/1910877846 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff0d406b920 0x7ff0d410da80 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:15.480 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.476+0000 7ff0dc6b5700 1 -- 172.21.15.27:0/1910877846 >> 172.21.15.27:0/1910877846 conn(0x7ff0d40f1f00 msgr2=0x7ff0d40f8960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:23:15.481 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.476+0000 7ff0dc6b5700 1 -- 172.21.15.27:0/1910877846 shutdown_connections 2021-11-03T17:23:15.481 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:15.476+0000 7ff0dc6b5700 1 -- 172.21.15.27:0/1910877846 wait complete. 2021-11-03T17:23:15.481 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:23:16.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:15 smithi115 ceph-mon[65453]: from='client.34376 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:23:16.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:15 smithi115 ceph-mon[65453]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:23:16.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:15 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3056244575' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:16.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:15 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1910877846' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:23:16.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:15 smithi027 ceph-mon[97244]: from='client.34376 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:23:16.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:15 smithi027 ceph-mon[97244]: from='client.34380 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:23:16.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:15 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3056244575' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:16.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:15 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1910877846' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:23:17.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:16 smithi115 ceph-mon[65453]: pgmap v257: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:17.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:23:17.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:23:17.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:23:17.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:17.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:17.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:17.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:17.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:16 smithi027 ceph-mon[97244]: pgmap v257: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:17.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:23:17.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:23:17.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:23:17.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:17.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:17.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:17.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:18.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:17 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:23:18.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:17 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:23:19.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:18 smithi115 ceph-mon[65453]: pgmap v258: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:19.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:18 smithi027 ceph-mon[97244]: pgmap v258: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:20.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:19 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 442.369467 secs 2021-11-03T17:23:20.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:19 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 442.369467 secs 2021-11-03T17:23:21.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:20 smithi115 ceph-mon[65453]: pgmap v259: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:21.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:20 smithi027 ceph-mon[97244]: pgmap v259: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:23.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:22 smithi115 ceph-mon[65453]: pgmap v260: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:23.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:22 smithi027 ceph-mon[97244]: pgmap v260: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:25.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:24 smithi115 ceph-mon[65453]: pgmap v261: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:25.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:24 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 447.369592 secs 2021-11-03T17:23:25.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:24 smithi027 ceph-mon[97244]: pgmap v261: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:25.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:24 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 447.369592 secs 2021-11-03T17:23:27.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:26 smithi115 ceph-mon[65453]: pgmap v262: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:27.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:23:27.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:23:27.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:23:27.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:27.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:27.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:27.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:27.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:26 smithi027 ceph-mon[97244]: pgmap v262: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:27.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:23:27.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:23:27.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:23:27.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:27.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:27.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:27.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:28.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:27 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:23:28.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:27 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:23:29.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:28 smithi115 ceph-mon[65453]: pgmap v263: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:29.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:28 smithi027 ceph-mon[97244]: pgmap v263: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:30.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:29 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 452.369727 secs 2021-11-03T17:23:30.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:29 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 452.369727 secs 2021-11-03T17:23:31.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:30 smithi115 ceph-mon[65453]: pgmap v264: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:31.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:30 smithi027 ceph-mon[97244]: pgmap v264: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:33.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:32 smithi115 ceph-mon[65453]: pgmap v265: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:33.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:32 smithi027 ceph-mon[97244]: pgmap v265: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:35.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:34 smithi115 ceph-mon[65453]: pgmap v266: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:35.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:34 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 457.369872 secs 2021-11-03T17:23:35.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:34 smithi027 ceph-mon[97244]: pgmap v266: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:35.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:34 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 457.369872 secs 2021-11-03T17:23:37.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:36 smithi115 ceph-mon[65453]: pgmap v267: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:37.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:23:37.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:23:37.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:23:37.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:37.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:37.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:37.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:37.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:36 smithi027 ceph-mon[97244]: pgmap v267: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:37.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:23:37.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:23:37.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:23:37.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:37.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:37.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:37.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:38.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:37 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:23:38.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:37 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:23:39.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:38 smithi115 ceph-mon[65453]: pgmap v268: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:39.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:38 smithi027 ceph-mon[97244]: pgmap v268: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:40.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:39 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 462.369984 secs 2021-11-03T17:23:40.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:39 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 462.369984 secs 2021-11-03T17:23:41.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:40 smithi115 ceph-mon[65453]: pgmap v269: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:41.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:40 smithi027 ceph-mon[97244]: pgmap v269: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:43.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:42 smithi115 ceph-mon[65453]: pgmap v270: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:43.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:42 smithi027 ceph-mon[97244]: pgmap v270: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:45.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:44 smithi027 ceph-mon[97244]: pgmap v271: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:45.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:44 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 467.370108 secs 2021-11-03T17:23:45.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:44 smithi115 ceph-mon[65453]: pgmap v271: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:45.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:44 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 467.370108 secs 2021-11-03T17:23:45.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.590+0000 7f6bc1992700 1 -- 172.21.15.27:0/3804562203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bbc0f66e0 msgr2=0x7f6bbc0f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:23:45.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.590+0000 7f6bc1992700 1 --2- 172.21.15.27:0/3804562203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bbc0f66e0 0x7f6bbc0f6b00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f6bac004660 tx=0x7f6bac009b30).stop 2021-11-03T17:23:45.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.590+0000 7f6bc1992700 1 -- 172.21.15.27:0/3804562203 shutdown_connections 2021-11-03T17:23:45.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.590+0000 7f6bc1992700 1 --2- 172.21.15.27:0/3804562203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bbc0f6ff0 0x7f6bbc0f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:45.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.590+0000 7f6bc1992700 1 --2- 172.21.15.27:0/3804562203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bbc0f66e0 0x7f6bbc0f6b00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:45.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.590+0000 7f6bc1992700 1 -- 172.21.15.27:0/3804562203 >> 172.21.15.27:0/3804562203 conn(0x7f6bbc0f1f00 msgr2=0x7f6bbc0f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:23:45.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.590+0000 7f6bc1992700 1 -- 172.21.15.27:0/3804562203 shutdown_connections 2021-11-03T17:23:45.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.590+0000 7f6bc1992700 1 -- 172.21.15.27:0/3804562203 wait complete. 2021-11-03T17:23:45.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.590+0000 7f6bc1992700 1 Processor -- start 2021-11-03T17:23:45.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.590+0000 7f6bc1992700 1 -- start start 2021-11-03T17:23:45.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.591+0000 7f6bc1992700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bbc0f66e0 0x7f6bbc06b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:23:45.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.591+0000 7f6bc1992700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bbc0f6ff0 0x7f6bbc069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:23:45.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.591+0000 7f6bc1992700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc06a380 con 0x7f6bbc0f6ff0 2021-11-03T17:23:45.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.591+0000 7f6bc1992700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc06a4c0 con 0x7f6bbc0f66e0 2021-11-03T17:23:45.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.591+0000 7f6bb3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bbc0f6ff0 0x7f6bbc069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:23:45.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.591+0000 7f6bb3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bbc0f6ff0 0x7f6bbc069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57172/0 (socket says 172.21.15.27:57172) 2021-11-03T17:23:45.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.591+0000 7f6bb3fff700 1 -- 172.21.15.27:0/2851682042 learned_addr learned my addr 172.21.15.27:0/2851682042 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:23:45.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.591+0000 7f6bbaffd700 1 --2- 172.21.15.27:0/2851682042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bbc0f66e0 0x7f6bbc06b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:23:45.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.591+0000 7f6bb3fff700 1 -- 172.21.15.27:0/2851682042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bbc0f66e0 msgr2=0x7f6bbc06b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:23:45.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.591+0000 7f6bb3fff700 1 --2- 172.21.15.27:0/2851682042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bbc0f66e0 0x7f6bbc06b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:45.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.591+0000 7f6bb3fff700 1 -- 172.21.15.27:0/2851682042 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bac005040 con 0x7f6bbc0f6ff0 2021-11-03T17:23:45.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.591+0000 7f6bb3fff700 1 --2- 172.21.15.27:0/2851682042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bbc0f6ff0 0x7f6bbc069e20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f6ba40026d0 tx=0x7f6ba4009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:23:45.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.591+0000 7f6bb37fe700 1 -- 172.21.15.27:0/2851682042 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ba40079c0 con 0x7f6bbc0f6ff0 2021-11-03T17:23:45.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.591+0000 7f6bb37fe700 1 -- 172.21.15.27:0/2851682042 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f6ba400f040 con 0x7f6bbc0f6ff0 2021-11-03T17:23:45.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.591+0000 7f6bc1992700 1 -- 172.21.15.27:0/2851682042 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bbc06a7a0 con 0x7f6bbc0f6ff0 2021-11-03T17:23:45.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.591+0000 7f6bb37fe700 1 -- 172.21.15.27:0/2851682042 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ba40148b0 con 0x7f6bbc0f6ff0 2021-11-03T17:23:45.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.592+0000 7f6bc1992700 1 -- 172.21.15.27:0/2851682042 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bbc10ef90 con 0x7f6bbc0f6ff0 2021-11-03T17:23:45.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.593+0000 7f6bc1992700 1 -- 172.21.15.27:0/2851682042 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bbc045ad0 con 0x7f6bbc0f6ff0 2021-11-03T17:23:45.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.593+0000 7f6bb37fe700 1 -- 172.21.15.27:0/2851682042 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f6ba4007b20 con 0x7f6bbc0f6ff0 2021-11-03T17:23:45.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.594+0000 7f6bb37fe700 1 --2- 172.21.15.27:0/2851682042 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6ba8064090 0x7f6ba8066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:23:45.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.594+0000 7f6bb37fe700 1 -- 172.21.15.27:0/2851682042 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f6ba4080730 con 0x7f6bbc0f6ff0 2021-11-03T17:23:45.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.594+0000 7f6bbaffd700 1 --2- 172.21.15.27:0/2851682042 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6ba8064090 0x7f6ba8066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:23:45.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.594+0000 7f6bbaffd700 1 --2- 172.21.15.27:0/2851682042 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6ba8064090 0x7f6ba8066550 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f6bac013690 tx=0x7f6bac009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:23:45.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.598+0000 7f6bb37fe700 1 -- 172.21.15.27:0/2851682042 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f6ba4052a90 con 0x7f6bbc0f6ff0 2021-11-03T17:23:45.771 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.776+0000 7f6bc1992700 1 -- 172.21.15.27:0/2851682042 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6bbc0ff040 con 0x7f6ba8064090 2021-11-03T17:23:45.772 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.777+0000 7f6bb37fe700 1 -- 172.21.15.27:0/2851682042 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f6bbc0ff040 con 0x7f6ba8064090 2021-11-03T17:23:45.774 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.779+0000 7f6bc1992700 1 -- 172.21.15.27:0/2851682042 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6ba8064090 msgr2=0x7f6ba8066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:23:45.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.779+0000 7f6bc1992700 1 --2- 172.21.15.27:0/2851682042 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6ba8064090 0x7f6ba8066550 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7f6bac013690 tx=0x7f6bac009b30).stop 2021-11-03T17:23:45.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.779+0000 7f6bc1992700 1 -- 172.21.15.27:0/2851682042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bbc0f6ff0 msgr2=0x7f6bbc069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:23:45.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.779+0000 7f6bc1992700 1 --2- 172.21.15.27:0/2851682042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bbc0f6ff0 0x7f6bbc069e20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f6ba40026d0 tx=0x7f6ba4009de0).stop 2021-11-03T17:23:45.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.779+0000 7f6bc1992700 1 -- 172.21.15.27:0/2851682042 shutdown_connections 2021-11-03T17:23:45.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.779+0000 7f6bc1992700 1 --2- 172.21.15.27:0/2851682042 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6ba8064090 0x7f6ba8066550 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:45.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.779+0000 7f6bc1992700 1 --2- 172.21.15.27:0/2851682042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bbc0f66e0 0x7f6bbc06b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:45.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.779+0000 7f6bc1992700 1 --2- 172.21.15.27:0/2851682042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bbc0f6ff0 0x7f6bbc069e20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:45.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.779+0000 7f6bc1992700 1 -- 172.21.15.27:0/2851682042 >> 172.21.15.27:0/2851682042 conn(0x7f6bbc0f1f00 msgr2=0x7f6bbc0fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:23:45.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.779+0000 7f6bc1992700 1 -- 172.21.15.27:0/2851682042 shutdown_connections 2021-11-03T17:23:45.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.779+0000 7f6bc1992700 1 -- 172.21.15.27:0/2851682042 wait complete. 2021-11-03T17:23:45.784 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:23:45.877 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.880+0000 7f2f345cb700 1 -- 172.21.15.27:0/710640799 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f2c0f8610 msgr2=0x7f2f2c0f4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:23:45.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.880+0000 7f2f345cb700 1 --2- 172.21.15.27:0/710640799 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f2c0f8610 0x7f2f2c0f4820 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f2f1c004660 tx=0x7f2f1c009b30).stop 2021-11-03T17:23:45.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.880+0000 7f2f345cb700 1 -- 172.21.15.27:0/710640799 shutdown_connections 2021-11-03T17:23:45.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.880+0000 7f2f345cb700 1 --2- 172.21.15.27:0/710640799 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f2c0f8610 0x7f2f2c0f4820 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:45.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.880+0000 7f2f345cb700 1 --2- 172.21.15.27:0/710640799 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f2c0f7d00 0x7f2f2c0f8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:23:45.879 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.880+0000 7f2f345cb700 1 -- 172.21.15.27:0/710640799 >> 172.21.15.27:0/710640799 conn(0x7f2f2c0f0540 msgr2=0x7f2f2c0f2960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:23:45.880 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.880+0000 7f2f345cb700 1 -- 172.21.15.27:0/710640799 shutdown_connections 2021-11-03T17:23:45.880 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.880+0000 7f2f345cb700 1 -- 172.21.15.27:0/710640799 wait complete. 2021-11-03T17:23:45.880 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.881+0000 7f2f345cb700 1 Processor -- start 2021-11-03T17:23:45.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.881+0000 7f2f345cb700 1 -- start start 2021-11-03T17:23:45.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.881+0000 7f2f345cb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f2c0f7d00 0x7f2f2c106f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:23:45.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.881+0000 7f2f345cb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f2c0f8610 0x7f2f2c107440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:23:45.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.881+0000 7f2f345cb700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c107a80 con 0x7f2f2c0f7d00 2021-11-03T17:23:45.882 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.881+0000 7f2f345cb700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f2c108330 con 0x7f2f2c0f8610 2021-11-03T17:23:45.882 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.881+0000 7f2f32367700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f2c0f7d00 0x7f2f2c106f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:23:45.882 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.881+0000 7f2f32367700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f2c0f7d00 0x7f2f2c106f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57180/0 (socket says 172.21.15.27:57180) 2021-11-03T17:23:45.882 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.881+0000 7f2f32367700 1 -- 172.21.15.27:0/2298798915 learned_addr learned my addr 172.21.15.27:0/2298798915 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:23:45.883 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.881+0000 7f2f32367700 1 -- 172.21.15.27:0/2298798915 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f2c0f8610 msgr2=0x7f2f2c107440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:23:45.883 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.881+0000 7f2f32367700 1 --2- 172.21.15.27:0/2298798915 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f2c0f8610 0x7f2f2c107440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:23:45.883 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.881+0000 7f2f32367700 1 -- 172.21.15.27:0/2298798915 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f1c005040 con 0x7f2f2c0f7d00 2021-11-03T17:23:45.883 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.881+0000 7f2f32367700 1 --2- 172.21.15.27:0/2298798915 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f2c0f7d00 0x7f2f2c106f50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f2f2800b410 tx=0x7f2f28006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:23:45.884 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.882+0000 7f2f22ffd700 1 -- 172.21.15.27:0/2298798915 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f280125f0 con 0x7f2f2c0f7d00 2021-11-03T17:23:45.884 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.882+0000 7f2f22ffd700 1 -- 172.21.15.27:0/2298798915 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f2f28012c00 con 0x7f2f2c0f7d00 2021-11-03T17:23:45.884 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.882+0000 7f2f22ffd700 1 -- 172.21.15.27:0/2298798915 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f28002ae0 con 0x7f2f2c0f7d00 2021-11-03T17:23:45.884 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.882+0000 7f2f345cb700 1 -- 172.21.15.27:0/2298798915 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f2c108610 con 0x7f2f2c0f7d00 2021-11-03T17:23:45.884 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.882+0000 7f2f345cb700 1 -- 172.21.15.27:0/2298798915 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f2c108bd0 con 0x7f2f2c0f7d00 2021-11-03T17:23:45.886 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.883+0000 7f2f345cb700 1 -- 172.21.15.27:0/2298798915 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f2c0f41d0 con 0x7f2f2c0f7d00 2021-11-03T17:23:45.886 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.883+0000 7f2f22ffd700 1 -- 172.21.15.27:0/2298798915 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f2f2800f030 con 0x7f2f2c0f7d00 2021-11-03T17:23:45.886 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.883+0000 7f2f22ffd700 1 --2- 172.21.15.27:0/2298798915 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2f180640b0 0x7f2f18066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:23:45.886 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.883+0000 7f2f22ffd700 1 -- 172.21.15.27:0/2298798915 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f2f28080be0 con 0x7f2f2c0f7d00 2021-11-03T17:23:45.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.883+0000 7f2f31b66700 1 --2- 172.21.15.27:0/2298798915 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2f180640b0 0x7f2f18066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:23:45.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.884+0000 7f2f31b66700 1 --2- 172.21.15.27:0/2298798915 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2f180640b0 0x7f2f18066570 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f2f1c004e30 tx=0x7f2f1c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:23:45.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:45.887+0000 7f2f22ffd700 1 -- 172.21.15.27:0/2298798915 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f2f28052f40 con 0x7f2f2c0f7d00 2021-11-03T17:23:46.059 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.064+0000 7f2f345cb700 1 -- 172.21.15.27:0/2298798915 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f2f2c0ff040 con 0x7f2f180640b0 2021-11-03T17:23:46.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.078+0000 7f2f22ffd700 1 -- 172.21.15.27:0/2298798915 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f2f2c0ff040 con 0x7f2f180640b0 2021-11-03T17:23:46.074 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:23:46.074 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (15m) 5m ago 16m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:23:46.075 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (7m) 5m ago 16m 7205k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:23:46.075 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (6m) 4m ago 15m 7239k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:23:46.076 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (15m) 5m ago 16m 38.9M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:23:46.076 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (8m) 5m ago 11m 15.4M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:23:46.076 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (11m) 5m ago 11m 3836M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:23:46.077 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (7m) 4m ago 11m 14.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:23:46.077 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (9m) 4m ago 11m 16.3M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:23:46.077 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (9m) 5m ago 17m 470M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:23:46.078 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (8m) 4m ago 15m 412M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:23:46.078 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (8m) 5m ago 17m 81.2M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:23:46.079 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (7m) 4m ago 15m 69.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:23:46.079 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (16m) 5m ago 16m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:23:46.079 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (15m) 4m ago 15m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:23:46.080 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (6m) 5m ago 15m 243M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:23:46.080 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (6m) 5m ago 14m 270M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:23:46.080 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (5m) 5m ago 14m 11.8M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:23:46.081 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (5m) 4m ago 13m 289M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:23:46.081 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (4m) 4m ago 13m 235M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:23:46.081 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (4m) 4m ago 13m 11.5M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:23:46.082 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (15m) 5m ago 15m 45.7M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:23:46.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.080+0000 7f2f345cb700 1 -- 172.21.15.27:0/2298798915 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2f180640b0 msgr2=0x7f2f18066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:23:46.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.080+0000 7f2f345cb700 1 --2- 172.21.15.27:0/2298798915 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2f180640b0 0x7f2f18066570 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7f2f1c004e30 tx=0x7f2f1c009b30).stop 2021-11-03T17:23:46.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.080+0000 7f2f345cb700 1 -- 172.21.15.27:0/2298798915 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f2c0f7d00 msgr2=0x7f2f2c106f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:23:46.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.080+0000 7f2f345cb700 1 --2- 172.21.15.27:0/2298798915 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f2c0f7d00 0x7f2f2c106f50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f2f2800b410 tx=0x7f2f28006ef0).stop 2021-11-03T17:23:46.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.080+0000 7f2f345cb700 1 -- 172.21.15.27:0/2298798915 shutdown_connections 2021-11-03T17:23:46.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.080+0000 7f2f345cb700 1 --2- 172.21.15.27:0/2298798915 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2f180640b0 0x7f2f18066570 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:46.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.080+0000 7f2f345cb700 1 --2- 172.21.15.27:0/2298798915 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f2c0f7d00 0x7f2f2c106f50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:46.086 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.080+0000 7f2f345cb700 1 --2- 172.21.15.27:0/2298798915 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f2c0f8610 0x7f2f2c107440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:23:46.086 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.080+0000 7f2f345cb700 1 -- 172.21.15.27:0/2298798915 >> 172.21.15.27:0/2298798915 conn(0x7f2f2c0f0540 msgr2=0x7f2f2c0fb590 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:23:46.086 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.080+0000 7f2f345cb700 1 -- 172.21.15.27:0/2298798915 shutdown_connections 2021-11-03T17:23:46.087 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.080+0000 7f2f345cb700 1 -- 172.21.15.27:0/2298798915 wait complete. 2021-11-03T17:23:46.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.181+0000 7fb5de168700 1 -- 172.21.15.27:0/2410271241 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5d80f66e0 msgr2=0x7fb5d80f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:23:46.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.181+0000 7fb5de168700 1 --2- 172.21.15.27:0/2410271241 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5d80f66e0 0x7fb5d80f6b00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fb5c8004660 tx=0x7fb5c8009b30).stop 2021-11-03T17:23:46.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.181+0000 7fb5de168700 1 -- 172.21.15.27:0/2410271241 shutdown_connections 2021-11-03T17:23:46.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.181+0000 7fb5de168700 1 --2- 172.21.15.27:0/2410271241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5d80f6ff0 0x7fb5d80f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:46.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.181+0000 7fb5de168700 1 --2- 172.21.15.27:0/2410271241 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5d80f66e0 0x7fb5d80f6b00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:46.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.181+0000 7fb5de168700 1 -- 172.21.15.27:0/2410271241 >> 172.21.15.27:0/2410271241 conn(0x7fb5d80f1f00 msgr2=0x7fb5d80f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:23:46.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.181+0000 7fb5de168700 1 -- 172.21.15.27:0/2410271241 shutdown_connections 2021-11-03T17:23:46.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.181+0000 7fb5de168700 1 -- 172.21.15.27:0/2410271241 wait complete. 2021-11-03T17:23:46.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.182+0000 7fb5de168700 1 Processor -- start 2021-11-03T17:23:46.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.182+0000 7fb5de168700 1 -- start start 2021-11-03T17:23:46.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.182+0000 7fb5de168700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5d80f66e0 0x7fb5d806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:23:46.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.182+0000 7fb5de168700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5d80f6ff0 0x7fb5d8069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:23:46.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.182+0000 7fb5de168700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d806a380 con 0x7fb5d80f66e0 2021-11-03T17:23:46.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.182+0000 7fb5de168700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d806a4c0 con 0x7fb5d80f6ff0 2021-11-03T17:23:46.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.182+0000 7fb5d77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5d80f66e0 0x7fb5d806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:23:46.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.182+0000 7fb5d77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5d80f66e0 0x7fb5d806b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57190/0 (socket says 172.21.15.27:57190) 2021-11-03T17:23:46.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.182+0000 7fb5d77fe700 1 -- 172.21.15.27:0/786862959 learned_addr learned my addr 172.21.15.27:0/786862959 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:23:46.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.182+0000 7fb5d6ffd700 1 --2- 172.21.15.27:0/786862959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5d80f6ff0 0x7fb5d8069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:23:46.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.182+0000 7fb5d77fe700 1 -- 172.21.15.27:0/786862959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5d80f6ff0 msgr2=0x7fb5d8069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:23:46.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.182+0000 7fb5d77fe700 1 --2- 172.21.15.27:0/786862959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5d80f6ff0 0x7fb5d8069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:46.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.182+0000 7fb5d77fe700 1 -- 172.21.15.27:0/786862959 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c8005040 con 0x7fb5d80f66e0 2021-11-03T17:23:46.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.182+0000 7fb5d6ffd700 1 --2- 172.21.15.27:0/786862959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5d80f6ff0 0x7fb5d8069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:23:46.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.183+0000 7fb5d77fe700 1 --2- 172.21.15.27:0/786862959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5d80f66e0 0x7fb5d806b9a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fb5c8002040 tx=0x7fb5c800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:23:46.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.183+0000 7fb5bffff700 1 -- 172.21.15.27:0/786862959 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5c8019070 con 0x7fb5d80f66e0 2021-11-03T17:23:46.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.183+0000 7fb5bffff700 1 -- 172.21.15.27:0/786862959 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fb5c8007e70 con 0x7fb5d80f66e0 2021-11-03T17:23:46.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.183+0000 7fb5de168700 1 -- 172.21.15.27:0/786862959 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d806a740 con 0x7fb5d80f66e0 2021-11-03T17:23:46.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.183+0000 7fb5bffff700 1 -- 172.21.15.27:0/786862959 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5c8014e30 con 0x7fb5d80f66e0 2021-11-03T17:23:46.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.183+0000 7fb5de168700 1 -- 172.21.15.27:0/786862959 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d806aca0 con 0x7fb5d80f66e0 2021-11-03T17:23:46.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.184+0000 7fb5de168700 1 -- 172.21.15.27:0/786862959 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5d8045ad0 con 0x7fb5d80f66e0 2021-11-03T17:23:46.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.184+0000 7fb5bffff700 1 -- 172.21.15.27:0/786862959 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fb5c8025070 con 0x7fb5d80f66e0 2021-11-03T17:23:46.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.184+0000 7fb5bffff700 1 --2- 172.21.15.27:0/786862959 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb5c0064090 0x7fb5c0066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:23:46.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.185+0000 7fb5bffff700 1 -- 172.21.15.27:0/786862959 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fb5c80816f0 con 0x7fb5d80f66e0 2021-11-03T17:23:46.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.185+0000 7fb5d6ffd700 1 --2- 172.21.15.27:0/786862959 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb5c0064090 0x7fb5c0066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:23:46.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.185+0000 7fb5d6ffd700 1 --2- 172.21.15.27:0/786862959 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb5c0064090 0x7fb5c0066550 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fb5cc009170 tx=0x7fb5cc00b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:23:46.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.188+0000 7fb5bffff700 1 -- 172.21.15.27:0/786862959 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fb5c8053a50 con 0x7fb5d80f66e0 2021-11-03T17:23:46.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.418+0000 7fb5de168700 1 -- 172.21.15.27:0/786862959 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb5d80586e0 con 0x7fb5d80f66e0 2021-11-03T17:23:46.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.419+0000 7fb5bffff700 1 -- 172.21.15.27:0/786862959 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fb5c8023070 con 0x7fb5d80f66e0 2021-11-03T17:23:46.415 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:23:46.415 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:23:46.415 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:23:46.416 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:23:46.416 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:23:46.416 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:23:46.416 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:23:46.417 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:23:46.417 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:23:46.417 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:23:46.417 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:23:46.417 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:23:46.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:23:46.418 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:23:46.418 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:23:46.419 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:23:46.419 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:23:46.419 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:23:46.420 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.421+0000 7fb5de168700 1 -- 172.21.15.27:0/786862959 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb5c0064090 msgr2=0x7fb5c0066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:23:46.420 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.421+0000 7fb5de168700 1 --2- 172.21.15.27:0/786862959 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb5c0064090 0x7fb5c0066550 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fb5cc009170 tx=0x7fb5cc00b040).stop 2021-11-03T17:23:46.420 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.421+0000 7fb5de168700 1 -- 172.21.15.27:0/786862959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5d80f66e0 msgr2=0x7fb5d806b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:23:46.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.421+0000 7fb5de168700 1 --2- 172.21.15.27:0/786862959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5d80f66e0 0x7fb5d806b9a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fb5c8002040 tx=0x7fb5c800b040).stop 2021-11-03T17:23:46.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.421+0000 7fb5de168700 1 -- 172.21.15.27:0/786862959 shutdown_connections 2021-11-03T17:23:46.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.421+0000 7fb5de168700 1 --2- 172.21.15.27:0/786862959 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb5c0064090 0x7fb5c0066550 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:46.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.421+0000 7fb5de168700 1 --2- 172.21.15.27:0/786862959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5d80f66e0 0x7fb5d806b9a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:46.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.421+0000 7fb5de168700 1 --2- 172.21.15.27:0/786862959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5d80f6ff0 0x7fb5d8069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:46.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.421+0000 7fb5de168700 1 -- 172.21.15.27:0/786862959 >> 172.21.15.27:0/786862959 conn(0x7fb5d80f1f00 msgr2=0x7fb5d80fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:23:46.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.421+0000 7fb5de168700 1 -- 172.21.15.27:0/786862959 shutdown_connections 2021-11-03T17:23:46.423 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.421+0000 7fb5de168700 1 -- 172.21.15.27:0/786862959 wait complete. 2021-11-03T17:23:46.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.522+0000 7f5238d82700 1 -- 172.21.15.27:0/725440497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52340f96b0 msgr2=0x7f52340fbb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:23:46.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.522+0000 7f5238d82700 1 --2- 172.21.15.27:0/725440497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52340f96b0 0x7f52340fbb00 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f5228004660 tx=0x7f5228009b30).stop 2021-11-03T17:23:46.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.522+0000 7f5238d82700 1 -- 172.21.15.27:0/725440497 shutdown_connections 2021-11-03T17:23:46.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.522+0000 7f5238d82700 1 --2- 172.21.15.27:0/725440497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52340f96b0 0x7f52340fbb00 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:46.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.522+0000 7f5238d82700 1 --2- 172.21.15.27:0/725440497 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5234060270 0x7f52340f91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:46.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.522+0000 7f5238d82700 1 -- 172.21.15.27:0/725440497 >> 172.21.15.27:0/725440497 conn(0x7f52340f0ba0 msgr2=0x7f52340f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:23:46.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.522+0000 7f5238d82700 1 -- 172.21.15.27:0/725440497 shutdown_connections 2021-11-03T17:23:46.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.523+0000 7f5238d82700 1 -- 172.21.15.27:0/725440497 wait complete. 2021-11-03T17:23:46.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.523+0000 7f5238d82700 1 Processor -- start 2021-11-03T17:23:46.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.523+0000 7f5238d82700 1 -- start start 2021-11-03T17:23:46.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.523+0000 7f5238d82700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5234060270 0x7f52340f6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:23:46.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.523+0000 7f5238d82700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52340f96b0 0x7f52340f50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:23:46.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.523+0000 7f5238d82700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52340f5620 con 0x7f5234060270 2021-11-03T17:23:46.524 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.523+0000 7f5238d82700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52340f5760 con 0x7f52340f96b0 2021-11-03T17:23:46.524 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.523+0000 7f52337fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5234060270 0x7f52340f6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:23:46.524 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.523+0000 7f52337fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5234060270 0x7f52340f6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57200/0 (socket says 172.21.15.27:57200) 2021-11-03T17:23:46.524 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.523+0000 7f52337fe700 1 -- 172.21.15.27:0/1483875614 learned_addr learned my addr 172.21.15.27:0/1483875614 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:23:46.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.524+0000 7f5232ffd700 1 --2- 172.21.15.27:0/1483875614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52340f96b0 0x7f52340f50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:23:46.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.524+0000 7f52337fe700 1 -- 172.21.15.27:0/1483875614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52340f96b0 msgr2=0x7f52340f50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:23:46.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.524+0000 7f52337fe700 1 --2- 172.21.15.27:0/1483875614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52340f96b0 0x7f52340f50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:46.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.524+0000 7f52337fe700 1 -- 172.21.15.27:0/1483875614 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5228005040 con 0x7f5234060270 2021-11-03T17:23:46.526 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.524+0000 7f52337fe700 1 --2- 172.21.15.27:0/1483875614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5234060270 0x7f52340f6c40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f522400e7f0 tx=0x7f522400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:23:46.526 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.524+0000 7f521bfff700 1 -- 172.21.15.27:0/1483875614 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52240092e0 con 0x7f5234060270 2021-11-03T17:23:46.526 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.524+0000 7f521bfff700 1 -- 172.21.15.27:0/1483875614 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f522400f040 con 0x7f5234060270 2021-11-03T17:23:46.526 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.524+0000 7f5238d82700 1 -- 172.21.15.27:0/1483875614 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52340f5a40 con 0x7f5234060270 2021-11-03T17:23:46.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.524+0000 7f521bfff700 1 -- 172.21.15.27:0/1483875614 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5224014750 con 0x7f5234060270 2021-11-03T17:23:46.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.524+0000 7f5238d82700 1 -- 172.21.15.27:0/1483875614 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f523410ecf0 con 0x7f5234060270 2021-11-03T17:23:46.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.525+0000 7f5238d82700 1 -- 172.21.15.27:0/1483875614 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5234045ad0 con 0x7f5234060270 2021-11-03T17:23:46.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.525+0000 7f521bfff700 1 -- 172.21.15.27:0/1483875614 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f522401b030 con 0x7f5234060270 2021-11-03T17:23:46.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.526+0000 7f521bfff700 1 --2- 172.21.15.27:0/1483875614 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f521c064100 0x7f521c0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:23:46.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.526+0000 7f521bfff700 1 -- 172.21.15.27:0/1483875614 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f5224080680 con 0x7f5234060270 2021-11-03T17:23:46.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.526+0000 7f5232ffd700 1 --2- 172.21.15.27:0/1483875614 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f521c064100 0x7f521c0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:23:46.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.526+0000 7f5232ffd700 1 --2- 172.21.15.27:0/1483875614 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f521c064100 0x7f521c0665c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f5228004d80 tx=0x7f5228016040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:23:46.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.530+0000 7f521bfff700 1 -- 172.21.15.27:0/1483875614 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f52240529a0 con 0x7f5234060270 2021-11-03T17:23:46.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.733+0000 7f5238d82700 1 -- 172.21.15.27:0/1483875614 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5234027a00 con 0x7f5234060270 2021-11-03T17:23:46.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.734+0000 7f521bfff700 1 -- 172.21.15.27:0/1483875614 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f5224019710 con 0x7f5234060270 2021-11-03T17:23:46.730 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:23:46.730 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:23:46.730 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:23:46.730 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:23:46.731 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:23:46.731 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:23:46.731 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:23:46.731 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:23:46.732 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:23:46.732 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:23:46.732 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:23:46.732 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:23:46.733 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:23:46.733 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:23:46.733 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:23:46.733 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:23:46.733 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:23:46.734 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:23:46.734 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:23:46.734 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:23:46.734 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:23:46.735 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:23:46.735 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:23:46.735 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:23:46.735 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:23:46.735 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:23:46.736 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:23:46.736 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:23:46.736 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:23:46.736 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:23:46.737 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:23:46.737 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:23:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:23:46.737 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:23:46.737 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:23:46.738 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:23:46.738 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:23:46.738 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:23:46.738 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:23:46.739 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.736+0000 7f5238d82700 1 -- 172.21.15.27:0/1483875614 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f521c064100 msgr2=0x7f521c0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:23:46.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.736+0000 7f5238d82700 1 --2- 172.21.15.27:0/1483875614 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f521c064100 0x7f521c0665c0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f5228004d80 tx=0x7f5228016040).stop 2021-11-03T17:23:46.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.736+0000 7f5238d82700 1 -- 172.21.15.27:0/1483875614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5234060270 msgr2=0x7f52340f6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:23:46.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.736+0000 7f5238d82700 1 --2- 172.21.15.27:0/1483875614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5234060270 0x7f52340f6c40 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f522400e7f0 tx=0x7f522400b040).stop 2021-11-03T17:23:46.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.736+0000 7f5238d82700 1 -- 172.21.15.27:0/1483875614 shutdown_connections 2021-11-03T17:23:46.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.736+0000 7f5238d82700 1 --2- 172.21.15.27:0/1483875614 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f521c064100 0x7f521c0665c0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:46.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.736+0000 7f5238d82700 1 --2- 172.21.15.27:0/1483875614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5234060270 0x7f52340f6c40 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:46.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.736+0000 7f5238d82700 1 --2- 172.21.15.27:0/1483875614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52340f96b0 0x7f52340f50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:23:46.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.736+0000 7f5238d82700 1 -- 172.21.15.27:0/1483875614 >> 172.21.15.27:0/1483875614 conn(0x7f52340f0ba0 msgr2=0x7f52340f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:23:46.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.736+0000 7f5238d82700 1 -- 172.21.15.27:0/1483875614 shutdown_connections 2021-11-03T17:23:46.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:23:46.736+0000 7f5238d82700 1 -- 172.21.15.27:0/1483875614 wait complete. 2021-11-03T17:23:46.742 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:23:47.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:46 smithi115 ceph-mon[65453]: from='client.34392 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:23:47.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:46 smithi115 ceph-mon[65453]: pgmap v272: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:47.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:46 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/786862959' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:47.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:46 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1483875614' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:23:47.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:23:47.257 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:46 smithi027 ceph-mon[97244]: from='client.34392 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:23:47.257 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:46 smithi027 ceph-mon[97244]: pgmap v272: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:47.258 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:46 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/786862959' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:47.258 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:46 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1483875614' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:23:47.258 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:23:48.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:47 smithi027 ceph-mon[97244]: from='client.34396 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:23:48.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:47 smithi027 ceph-mon[97244]: poll took 60175.876 ms: 1 events 2021-11-03T17:23:48.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:23:48.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:23:48.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:48.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:48.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:48.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:48.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:47 smithi115 ceph-mon[65453]: from='client.34396 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:23:48.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:47 smithi115 ceph-mon[65453]: poll took 60175.876 ms: 1 events 2021-11-03T17:23:48.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:23:48.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:23:48.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:48.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:48.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:48.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:49.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:48 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:23:49.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:48 smithi027 ceph-mon[97244]: pgmap v273: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:49.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:48 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:23:49.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:48 smithi115 ceph-mon[65453]: pgmap v273: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:50.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:49 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 472.370235 secs 2021-11-03T17:23:50.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 472.370235 secs 2021-11-03T17:23:51.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:51 smithi027 ceph-mon[97244]: pgmap v274: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:51.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:51 smithi115 ceph-mon[65453]: pgmap v274: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:53.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:53 smithi027 ceph-mon[97244]: pgmap v275: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:53.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:53 smithi115 ceph-mon[65453]: pgmap v275: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:54.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:54 smithi115 ceph-mon[65453]: pgmap v276: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:54.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:54 smithi027 ceph-mon[97244]: pgmap v276: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:55.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 477.370372 secs 2021-11-03T17:23:55.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 477.370372 secs 2021-11-03T17:23:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:23:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:23:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:56 smithi115 ceph-mon[65453]: pgmap v277: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:23:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:23:56.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:56 smithi027 ceph-mon[97244]: pgmap v277: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:57.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:23:57.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:23:57.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:23:57.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:57.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:57.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:57.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:57.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:23:57.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:23:57.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:23:57.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:57.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:57.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:57.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:23:58.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:58 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:23:58.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:58 smithi115 ceph-mon[65453]: pgmap v278: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:58.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:58 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:23:58.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:58 smithi027 ceph-mon[97244]: pgmap v278: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:23:59.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:23:59 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 482.370505 secs 2021-11-03T17:23:59.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:23:59 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 482.370505 secs 2021-11-03T17:24:00.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:00 smithi027 ceph-mon[97244]: pgmap v279: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:24:00.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:00 smithi115 ceph-mon[65453]: pgmap v279: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:24:03.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:02 smithi115 ceph-mon[65453]: pgmap v280: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:24:03.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:02 smithi027 ceph-mon[97244]: pgmap v280: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:24:05.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:04 smithi115 ceph-mon[65453]: pgmap v281: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:24:05.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:04 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 487.370648 secs 2021-11-03T17:24:05.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:04 smithi027 ceph-mon[97244]: pgmap v281: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:24:05.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:04 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 487.370648 secs 2021-11-03T17:24:07.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:06 smithi115 ceph-mon[65453]: pgmap v282: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:24:07.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:06 smithi027 ceph-mon[97244]: pgmap v282: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:24:08.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:24:08.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:24:08.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:24:08.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:08.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:08.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:08.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:08.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:24:08.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:24:08.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:24:08.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:08.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:08.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:08.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:09.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:08 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:24:09.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:08 smithi115 ceph-mon[65453]: pgmap v283: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:24:09.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:08 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:24:09.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:08 smithi027 ceph-mon[97244]: pgmap v283: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:24:10.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:09 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 492.370764 secs 2021-11-03T17:24:10.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:09 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 492.370764 secs 2021-11-03T17:24:11.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:10 smithi115 ceph-mon[65453]: pgmap v284: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:24:11.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:10 smithi027 ceph-mon[97244]: pgmap v284: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:24:13.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:12 smithi115 ceph-mon[65453]: pgmap v285: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:24:13.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:12 smithi027 ceph-mon[97244]: pgmap v285: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:24:15.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:14 smithi115 ceph-mon[65453]: pgmap v286: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:24:15.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:14 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 497.370882 secs 2021-11-03T17:24:15.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:14 smithi027 ceph-mon[97244]: pgmap v286: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:24:15.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:14 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 497.370882 secs 2021-11-03T17:24:16.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:16.851+0000 7f451123a700 1 -- 172.21.15.27:0/3031189348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f450c0f77d0 msgr2=0x7f450c0f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:24:16.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:16.851+0000 7f451123a700 1 --2- 172.21.15.27:0/3031189348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f450c0f77d0 0x7f450c0f7c30 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f4500004660 tx=0x7f4500009b30).stop 2021-11-03T17:24:16.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:16.851+0000 7f451123a700 1 -- 172.21.15.27:0/3031189348 shutdown_connections 2021-11-03T17:24:16.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:16.851+0000 7f451123a700 1 --2- 172.21.15.27:0/3031189348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f450c0f77d0 0x7f450c0f7c30 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:16.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:16.851+0000 7f451123a700 1 --2- 172.21.15.27:0/3031189348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f450c0f6530 0x7f450c0f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:16.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:16.851+0000 7f451123a700 1 -- 172.21.15.27:0/3031189348 >> 172.21.15.27:0/3031189348 conn(0x7f450c0f1c90 msgr2=0x7f450c0f40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:24:16.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:16.851+0000 7f451123a700 1 -- 172.21.15.27:0/3031189348 shutdown_connections 2021-11-03T17:24:16.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:16.851+0000 7f451123a700 1 -- 172.21.15.27:0/3031189348 wait complete. 2021-11-03T17:24:16.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:16.851+0000 7f451123a700 1 Processor -- start 2021-11-03T17:24:16.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:16.851+0000 7f451123a700 1 -- start start 2021-11-03T17:24:16.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:16.852+0000 7f451123a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f450c0f6530 0x7f450c106d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:24:16.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:16.852+0000 7f451123a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f450c0f77d0 0x7f450c107280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:24:16.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:16.852+0000 7f451123a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450c107880 con 0x7f450c0f77d0 2021-11-03T17:24:16.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:16.852+0000 7f451123a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450c108090 con 0x7f450c0f6530 2021-11-03T17:24:16.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:16.852+0000 7f450b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f450c0f77d0 0x7f450c107280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:24:16.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:16.852+0000 7f450b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f450c0f77d0 0x7f450c107280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57220/0 (socket says 172.21.15.27:57220) 2021-11-03T17:24:16.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:16.852+0000 7f450b7fe700 1 -- 172.21.15.27:0/3606660256 learned_addr learned my addr 172.21.15.27:0/3606660256 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:24:16.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:16.852+0000 7f450bfff700 1 --2- 172.21.15.27:0/3606660256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f450c0f6530 0x7f450c106d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:24:16.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:16.852+0000 7f450b7fe700 1 -- 172.21.15.27:0/3606660256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f450c0f6530 msgr2=0x7f450c106d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:24:16.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:16.852+0000 7f450b7fe700 1 --2- 172.21.15.27:0/3606660256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f450c0f6530 0x7f450c106d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:16.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:16.852+0000 7f450b7fe700 1 -- 172.21.15.27:0/3606660256 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4500005040 con 0x7f450c0f77d0 2021-11-03T17:24:16.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:16.852+0000 7f450b7fe700 1 --2- 172.21.15.27:0/3606660256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f450c0f77d0 0x7f450c107280 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f4500009990 tx=0x7f4500009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:24:16.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:16.852+0000 7f4508ff9700 1 -- 172.21.15.27:0/3606660256 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4500018070 con 0x7f450c0f77d0 2021-11-03T17:24:16.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:16.852+0000 7f4508ff9700 1 -- 172.21.15.27:0/3606660256 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f4500007ca0 con 0x7f450c0f77d0 2021-11-03T17:24:16.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:16.852+0000 7f451123a700 1 -- 172.21.15.27:0/3606660256 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f450c1082b0 con 0x7f450c0f77d0 2021-11-03T17:24:16.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:16.852+0000 7f4508ff9700 1 -- 172.21.15.27:0/3606660256 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f450001c620 con 0x7f450c0f77d0 2021-11-03T17:24:16.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:16.853+0000 7f451123a700 1 -- 172.21.15.27:0/3606660256 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f450c108810 con 0x7f450c0f77d0 2021-11-03T17:24:16.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:16.854+0000 7f451123a700 1 -- 172.21.15.27:0/3606660256 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f450c059100 con 0x7f450c0f77d0 2021-11-03T17:24:16.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:16.854+0000 7f4508ff9700 1 -- 172.21.15.27:0/3606660256 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f450001d960 con 0x7f450c0f77d0 2021-11-03T17:24:16.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:16.854+0000 7f4508ff9700 1 --2- 172.21.15.27:0/3606660256 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f44f4064100 0x7f44f40665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:24:16.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:16.855+0000 7f4508ff9700 1 -- 172.21.15.27:0/3606660256 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f4500082540 con 0x7f450c0f77d0 2021-11-03T17:24:16.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:16.855+0000 7f450bfff700 1 --2- 172.21.15.27:0/3606660256 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f44f4064100 0x7f44f40665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:24:16.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:16.855+0000 7f450bfff700 1 --2- 172.21.15.27:0/3606660256 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f44f4064100 0x7f44f40665c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f450c106b00 tx=0x7f44fc009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:24:16.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:16.859+0000 7f4508ff9700 1 -- 172.21.15.27:0/3606660256 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f4500054960 con 0x7f450c0f77d0 2021-11-03T17:24:17.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.039+0000 7f451123a700 1 -- 172.21.15.27:0/3606660256 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f450c0fee80 con 0x7f44f4064100 2021-11-03T17:24:17.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.040+0000 7f4508ff9700 1 -- 172.21.15.27:0/3606660256 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f450c0fee80 con 0x7f44f4064100 2021-11-03T17:24:17.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.042+0000 7f451123a700 1 -- 172.21.15.27:0/3606660256 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f44f4064100 msgr2=0x7f44f40665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:24:17.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.042+0000 7f451123a700 1 --2- 172.21.15.27:0/3606660256 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f44f4064100 0x7f44f40665c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7f450c106b00 tx=0x7f44fc009040).stop 2021-11-03T17:24:17.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.042+0000 7f451123a700 1 -- 172.21.15.27:0/3606660256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f450c0f77d0 msgr2=0x7f450c107280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:24:17.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.042+0000 7f451123a700 1 --2- 172.21.15.27:0/3606660256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f450c0f77d0 0x7f450c107280 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f4500009990 tx=0x7f4500009b30).stop 2021-11-03T17:24:17.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.042+0000 7f451123a700 1 -- 172.21.15.27:0/3606660256 shutdown_connections 2021-11-03T17:24:17.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.042+0000 7f451123a700 1 --2- 172.21.15.27:0/3606660256 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f44f4064100 0x7f44f40665c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:17.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.042+0000 7f451123a700 1 --2- 172.21.15.27:0/3606660256 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f450c0f6530 0x7f450c106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:17.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.042+0000 7f451123a700 1 --2- 172.21.15.27:0/3606660256 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f450c0f77d0 0x7f450c107280 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:17.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.042+0000 7f451123a700 1 -- 172.21.15.27:0/3606660256 >> 172.21.15.27:0/3606660256 conn(0x7f450c0f1c90 msgr2=0x7f450c0fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:24:17.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.042+0000 7f451123a700 1 -- 172.21.15.27:0/3606660256 shutdown_connections 2021-11-03T17:24:17.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.042+0000 7f451123a700 1 -- 172.21.15.27:0/3606660256 wait complete. 2021-11-03T17:24:17.048 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:24:17.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.143+0000 7f55ffa47700 1 -- 172.21.15.27:0/3489470743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55f80f98d0 msgr2=0x7f55f80fbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:24:17.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.143+0000 7f55ffa47700 1 --2- 172.21.15.27:0/3489470743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55f80f98d0 0x7f55f80fbcc0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f55f4004660 tx=0x7f55f4009b30).stop 2021-11-03T17:24:17.142 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.143+0000 7f55ffa47700 1 -- 172.21.15.27:0/3489470743 shutdown_connections 2021-11-03T17:24:17.142 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.143+0000 7f55ffa47700 1 --2- 172.21.15.27:0/3489470743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55f80f98d0 0x7f55f80fbcc0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:17.143 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.143+0000 7f55ffa47700 1 --2- 172.21.15.27:0/3489470743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55f80602b0 0x7f55f80f93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:24:17.144 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.143+0000 7f55ffa47700 1 -- 172.21.15.27:0/3489470743 >> 172.21.15.27:0/3489470743 conn(0x7f55f80f0de0 msgr2=0x7f55f80f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:24:17.144 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.143+0000 7f55ffa47700 1 -- 172.21.15.27:0/3489470743 shutdown_connections 2021-11-03T17:24:17.144 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.143+0000 7f55ffa47700 1 -- 172.21.15.27:0/3489470743 wait complete. 2021-11-03T17:24:17.145 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.144+0000 7f55ffa47700 1 Processor -- start 2021-11-03T17:24:17.145 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.144+0000 7f55ffa47700 1 -- start start 2021-11-03T17:24:17.146 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.144+0000 7f55ffa47700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55f80602b0 0x7f55f8106f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:24:17.146 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.144+0000 7f55ffa47700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55f80f98d0 0x7f55f8107440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:24:17.147 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.144+0000 7f55ffa47700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f8107a80 con 0x7f55f80f98d0 2021-11-03T17:24:17.147 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.144+0000 7f55ffa47700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f8108330 con 0x7f55f80602b0 2021-11-03T17:24:17.148 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.144+0000 7f55fcfe2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55f80f98d0 0x7f55f8107440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:24:17.149 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.144+0000 7f55fcfe2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55f80f98d0 0x7f55f8107440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57230/0 (socket says 172.21.15.27:57230) 2021-11-03T17:24:17.149 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.144+0000 7f55fcfe2700 1 -- 172.21.15.27:0/4267894229 learned_addr learned my addr 172.21.15.27:0/4267894229 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:24:17.149 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.144+0000 7f55fd7e3700 1 --2- 172.21.15.27:0/4267894229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55f80602b0 0x7f55f8106f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:24:17.150 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.144+0000 7f55fcfe2700 1 -- 172.21.15.27:0/4267894229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55f80602b0 msgr2=0x7f55f8106f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:24:17.150 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.144+0000 7f55fcfe2700 1 --2- 172.21.15.27:0/4267894229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55f80602b0 0x7f55f8106f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:17.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.144+0000 7f55fcfe2700 1 -- 172.21.15.27:0/4267894229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f4005040 con 0x7f55f80f98d0 2021-11-03T17:24:17.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.144+0000 7f55fcfe2700 1 --2- 172.21.15.27:0/4267894229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55f80f98d0 0x7f55f8107440 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f55f4009990 tx=0x7f55f4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:24:17.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.145+0000 7f55edffb700 1 -- 172.21.15.27:0/4267894229 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55f4018070 con 0x7f55f80f98d0 2021-11-03T17:24:17.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.145+0000 7f55edffb700 1 -- 172.21.15.27:0/4267894229 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f55f4007ca0 con 0x7f55f80f98d0 2021-11-03T17:24:17.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.145+0000 7f55ffa47700 1 -- 172.21.15.27:0/4267894229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f81085b0 con 0x7f55f80f98d0 2021-11-03T17:24:17.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.145+0000 7f55edffb700 1 -- 172.21.15.27:0/4267894229 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55f401c620 con 0x7f55f80f98d0 2021-11-03T17:24:17.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.145+0000 7f55ffa47700 1 -- 172.21.15.27:0/4267894229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f8108b10 con 0x7f55f80f98d0 2021-11-03T17:24:17.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.146+0000 7f55ffa47700 1 -- 172.21.15.27:0/4267894229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f80f4a50 con 0x7f55f80f98d0 2021-11-03T17:24:17.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.146+0000 7f55edffb700 1 -- 172.21.15.27:0/4267894229 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f55f401d960 con 0x7f55f80f98d0 2021-11-03T17:24:17.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.147+0000 7f55edffb700 1 --2- 172.21.15.27:0/4267894229 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f55e40640b0 0x7f55e4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:24:17.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.147+0000 7f55edffb700 1 -- 172.21.15.27:0/4267894229 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f55f4082480 con 0x7f55f80f98d0 2021-11-03T17:24:17.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.147+0000 7f55fd7e3700 1 --2- 172.21.15.27:0/4267894229 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f55e40640b0 0x7f55e4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:24:17.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.147+0000 7f55fd7e3700 1 --2- 172.21.15.27:0/4267894229 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f55e40640b0 0x7f55e4066570 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f55f8060080 tx=0x7f55e8008040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:24:17.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.151+0000 7f55edffb700 1 -- 172.21.15.27:0/4267894229 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f55f40548a0 con 0x7f55f80f98d0 2021-11-03T17:24:17.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:16 smithi115 ceph-mon[65453]: pgmap v287: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:24:17.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:16 smithi027 ceph-mon[97244]: pgmap v287: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:24:17.320 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.324+0000 7f55ffa47700 1 -- 172.21.15.27:0/4267894229 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f55f80fef30 con 0x7f55e40640b0 2021-11-03T17:24:17.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.338+0000 7f55edffb700 1 -- 172.21.15.27:0/4267894229 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f55f80fef30 con 0x7f55e40640b0 2021-11-03T17:24:17.335 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:24:17.335 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (15m) 6m ago 17m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:24:17.335 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (7m) 6m ago 17m 7205k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:24:17.335 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (7m) 4m ago 16m 7239k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:24:17.336 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (15m) 6m ago 16m 38.9M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:24:17.336 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (9m) 6m ago 12m 15.4M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:24:17.336 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (12m) 6m ago 12m 3836M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:24:17.336 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (8m) 4m ago 12m 14.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:24:17.336 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (9m) 4m ago 12m 16.3M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:24:17.337 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (9m) 6m ago 18m 470M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:24:17.337 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (8m) 4m ago 16m 412M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:24:17.337 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (8m) 6m ago 18m 81.2M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:24:17.337 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (8m) 4m ago 16m 69.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:24:17.338 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (16m) 6m ago 16m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:24:17.338 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (16m) 4m ago 16m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:24:17.339 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (7m) 6m ago 15m 243M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:24:17.339 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (6m) 6m ago 15m 270M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:24:17.339 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (6m) 6m ago 14m 11.8M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:24:17.339 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (5m) 4m ago 14m 289M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:24:17.340 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (5m) 4m ago 14m 235M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:24:17.340 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (4m) 4m ago 13m 11.5M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:24:17.340 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (15m) 6m ago 16m 45.7M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:24:17.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.340+0000 7f55ffa47700 1 -- 172.21.15.27:0/4267894229 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f55e40640b0 msgr2=0x7f55e4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:24:17.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.340+0000 7f55ffa47700 1 --2- 172.21.15.27:0/4267894229 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f55e40640b0 0x7f55e4066570 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7f55f8060080 tx=0x7f55e8008040).stop 2021-11-03T17:24:17.342 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.340+0000 7f55ffa47700 1 -- 172.21.15.27:0/4267894229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55f80f98d0 msgr2=0x7f55f8107440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:24:17.342 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.340+0000 7f55ffa47700 1 --2- 172.21.15.27:0/4267894229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55f80f98d0 0x7f55f8107440 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f55f4009990 tx=0x7f55f4009b30).stop 2021-11-03T17:24:17.342 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.341+0000 7f55ffa47700 1 -- 172.21.15.27:0/4267894229 shutdown_connections 2021-11-03T17:24:17.342 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.341+0000 7f55ffa47700 1 --2- 172.21.15.27:0/4267894229 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f55e40640b0 0x7f55e4066570 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:17.343 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.341+0000 7f55ffa47700 1 --2- 172.21.15.27:0/4267894229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55f80602b0 0x7f55f8106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:17.343 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.341+0000 7f55ffa47700 1 --2- 172.21.15.27:0/4267894229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55f80f98d0 0x7f55f8107440 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:17.343 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.341+0000 7f55ffa47700 1 -- 172.21.15.27:0/4267894229 >> 172.21.15.27:0/4267894229 conn(0x7f55f80f0de0 msgr2=0x7f55f80f31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:24:17.343 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.341+0000 7f55ffa47700 1 -- 172.21.15.27:0/4267894229 shutdown_connections 2021-11-03T17:24:17.343 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.341+0000 7f55ffa47700 1 -- 172.21.15.27:0/4267894229 wait complete. 2021-11-03T17:24:17.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.442+0000 7f498be7b700 1 -- 172.21.15.27:0/3770498535 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49840f55c0 msgr2=0x7f49840f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:24:17.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.442+0000 7f498be7b700 1 --2- 172.21.15.27:0/3770498535 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49840f55c0 0x7f49840f59e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f497c004660 tx=0x7f497c009b30).stop 2021-11-03T17:24:17.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.442+0000 7f498be7b700 1 -- 172.21.15.27:0/3770498535 shutdown_connections 2021-11-03T17:24:17.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.442+0000 7f498be7b700 1 --2- 172.21.15.27:0/3770498535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49840f5ed0 0x7f49840f6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:17.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.442+0000 7f498be7b700 1 --2- 172.21.15.27:0/3770498535 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49840f55c0 0x7f49840f59e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:17.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.442+0000 7f498be7b700 1 -- 172.21.15.27:0/3770498535 >> 172.21.15.27:0/3770498535 conn(0x7f49840f0de0 msgr2=0x7f49840f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:24:17.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.442+0000 7f498be7b700 1 -- 172.21.15.27:0/3770498535 shutdown_connections 2021-11-03T17:24:17.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.442+0000 7f498be7b700 1 -- 172.21.15.27:0/3770498535 wait complete. 2021-11-03T17:24:17.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.443+0000 7f498be7b700 1 Processor -- start 2021-11-03T17:24:17.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.443+0000 7f498be7b700 1 -- start start 2021-11-03T17:24:17.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.443+0000 7f498be7b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49840f55c0 0x7f4984105e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:24:17.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.443+0000 7f498be7b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49840f5ed0 0x7f4984106360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:24:17.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.443+0000 7f498be7b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4984106910 con 0x7f49840f5ed0 2021-11-03T17:24:17.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.443+0000 7f498be7b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4984106a50 con 0x7f49840f55c0 2021-11-03T17:24:17.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.443+0000 7f4989416700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49840f5ed0 0x7f4984106360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:24:17.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.443+0000 7f4989416700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49840f5ed0 0x7f4984106360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57240/0 (socket says 172.21.15.27:57240) 2021-11-03T17:24:17.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.443+0000 7f4989416700 1 -- 172.21.15.27:0/1740939101 learned_addr learned my addr 172.21.15.27:0/1740939101 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:24:17.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.443+0000 7f4989c17700 1 --2- 172.21.15.27:0/1740939101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49840f55c0 0x7f4984105e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:24:17.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.443+0000 7f4989416700 1 -- 172.21.15.27:0/1740939101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49840f55c0 msgr2=0x7f4984105e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:24:17.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.443+0000 7f4989416700 1 --2- 172.21.15.27:0/1740939101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49840f55c0 0x7f4984105e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:17.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.443+0000 7f4989416700 1 -- 172.21.15.27:0/1740939101 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f497c005040 con 0x7f49840f5ed0 2021-11-03T17:24:17.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.444+0000 7f4989416700 1 --2- 172.21.15.27:0/1740939101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49840f5ed0 0x7f4984106360 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f497800c790 tx=0x7f4978007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:24:17.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.444+0000 7f49827fc700 1 -- 172.21.15.27:0/1740939101 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f497800e680 con 0x7f49840f5ed0 2021-11-03T17:24:17.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.444+0000 7f49827fc700 1 -- 172.21.15.27:0/1740939101 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f497800ec90 con 0x7f49840f5ed0 2021-11-03T17:24:17.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.444+0000 7f498be7b700 1 -- 172.21.15.27:0/1740939101 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49841073f0 con 0x7f49840f5ed0 2021-11-03T17:24:17.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.444+0000 7f49827fc700 1 -- 172.21.15.27:0/1740939101 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4978013d60 con 0x7f49840f5ed0 2021-11-03T17:24:17.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.444+0000 7f498be7b700 1 -- 172.21.15.27:0/1740939101 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49841079b0 con 0x7f49840f5ed0 2021-11-03T17:24:17.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.445+0000 7f498be7b700 1 -- 172.21.15.27:0/1740939101 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4984045ad0 con 0x7f49840f5ed0 2021-11-03T17:24:17.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.445+0000 7f49827fc700 1 -- 172.21.15.27:0/1740939101 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f497801a060 con 0x7f49840f5ed0 2021-11-03T17:24:17.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.446+0000 7f49827fc700 1 --2- 172.21.15.27:0/1740939101 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f496c0640b0 0x7f496c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:24:17.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.446+0000 7f49827fc700 1 -- 172.21.15.27:0/1740939101 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f4978080b00 con 0x7f49840f5ed0 2021-11-03T17:24:17.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.446+0000 7f4989c17700 1 --2- 172.21.15.27:0/1740939101 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f496c0640b0 0x7f496c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:24:17.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.446+0000 7f4989c17700 1 --2- 172.21.15.27:0/1740939101 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f496c0640b0 0x7f496c066570 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f497c009da0 tx=0x7f497c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:24:17.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.450+0000 7f49827fc700 1 -- 172.21.15.27:0/1740939101 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f4978052d70 con 0x7f49840f5ed0 2021-11-03T17:24:17.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.679+0000 7f498be7b700 1 -- 172.21.15.27:0/1740939101 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f49840fde50 con 0x7f49840f5ed0 2021-11-03T17:24:17.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.679+0000 7f49827fc700 1 -- 172.21.15.27:0/1740939101 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f4978018070 con 0x7f49840f5ed0 2021-11-03T17:24:17.675 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:24:17.676 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:24:17.676 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:24:17.676 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:24:17.676 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:24:17.676 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:24:17.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:24:17.677 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:24:17.677 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:24:17.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:24:17.678 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:24:17.678 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:24:17.678 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:24:17.678 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:24:17.679 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:24:17.679 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:24:17.679 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:24:17.679 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:24:17.680 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.681+0000 7f498be7b700 1 -- 172.21.15.27:0/1740939101 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f496c0640b0 msgr2=0x7f496c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:24:17.681 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.681+0000 7f498be7b700 1 --2- 172.21.15.27:0/1740939101 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f496c0640b0 0x7f496c066570 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7f497c009da0 tx=0x7f497c009b30).stop 2021-11-03T17:24:17.681 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.681+0000 7f498be7b700 1 -- 172.21.15.27:0/1740939101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49840f5ed0 msgr2=0x7f4984106360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:24:17.681 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.681+0000 7f498be7b700 1 --2- 172.21.15.27:0/1740939101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49840f5ed0 0x7f4984106360 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f497800c790 tx=0x7f4978007d70).stop 2021-11-03T17:24:17.681 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.681+0000 7f498be7b700 1 -- 172.21.15.27:0/1740939101 shutdown_connections 2021-11-03T17:24:17.682 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.681+0000 7f498be7b700 1 --2- 172.21.15.27:0/1740939101 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f496c0640b0 0x7f496c066570 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:17.682 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.681+0000 7f498be7b700 1 --2- 172.21.15.27:0/1740939101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49840f55c0 0x7f4984105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:17.682 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.681+0000 7f498be7b700 1 --2- 172.21.15.27:0/1740939101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49840f5ed0 0x7f4984106360 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:17.682 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.681+0000 7f498be7b700 1 -- 172.21.15.27:0/1740939101 >> 172.21.15.27:0/1740939101 conn(0x7f49840f0de0 msgr2=0x7f49840fa350 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:24:17.683 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.681+0000 7f498be7b700 1 -- 172.21.15.27:0/1740939101 shutdown_connections 2021-11-03T17:24:17.683 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.681+0000 7f498be7b700 1 -- 172.21.15.27:0/1740939101 wait complete. 2021-11-03T17:24:17.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.782+0000 7f54df39f700 1 -- 172.21.15.27:0/3193824117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54d80f66e0 msgr2=0x7f54d80f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:24:17.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.782+0000 7f54df39f700 1 --2- 172.21.15.27:0/3193824117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54d80f66e0 0x7f54d80f6b40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f54cc004660 tx=0x7f54cc009b30).stop 2021-11-03T17:24:17.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.782+0000 7f54df39f700 1 -- 172.21.15.27:0/3193824117 shutdown_connections 2021-11-03T17:24:17.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.782+0000 7f54df39f700 1 --2- 172.21.15.27:0/3193824117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54d80f66e0 0x7f54d80f6b40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:17.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.782+0000 7f54df39f700 1 --2- 172.21.15.27:0/3193824117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54d80f5440 0x7f54d80f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:17.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.782+0000 7f54df39f700 1 -- 172.21.15.27:0/3193824117 >> 172.21.15.27:0/3193824117 conn(0x7f54d80f0ba0 msgr2=0x7f54d80f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:24:17.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.782+0000 7f54df39f700 1 -- 172.21.15.27:0/3193824117 shutdown_connections 2021-11-03T17:24:17.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.782+0000 7f54df39f700 1 -- 172.21.15.27:0/3193824117 wait complete. 2021-11-03T17:24:17.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.783+0000 7f54df39f700 1 Processor -- start 2021-11-03T17:24:17.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.783+0000 7f54df39f700 1 -- start start 2021-11-03T17:24:17.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.783+0000 7f54df39f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54d80f5440 0x7f54d8106e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:24:17.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.783+0000 7f54df39f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54d80f66e0 0x7f54d81072f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:24:17.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.783+0000 7f54df39f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d81078f0 con 0x7f54d80f5440 2021-11-03T17:24:17.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.783+0000 7f54df39f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d8108100 con 0x7f54d80f66e0 2021-11-03T17:24:17.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.783+0000 7f54de39d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54d80f5440 0x7f54d8106e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:24:17.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.783+0000 7f54de39d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54d80f5440 0x7f54d8106e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57248/0 (socket says 172.21.15.27:57248) 2021-11-03T17:24:17.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.783+0000 7f54de39d700 1 -- 172.21.15.27:0/3491749712 learned_addr learned my addr 172.21.15.27:0/3491749712 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:24:17.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.783+0000 7f54ddb9c700 1 --2- 172.21.15.27:0/3491749712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54d80f66e0 0x7f54d81072f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:24:17.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.783+0000 7f54de39d700 1 -- 172.21.15.27:0/3491749712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54d80f66e0 msgr2=0x7f54d81072f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:24:17.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.783+0000 7f54de39d700 1 --2- 172.21.15.27:0/3491749712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54d80f66e0 0x7f54d81072f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:17.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.783+0000 7f54de39d700 1 -- 172.21.15.27:0/3491749712 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54cc005040 con 0x7f54d80f5440 2021-11-03T17:24:17.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.783+0000 7f54ddb9c700 1 --2- 172.21.15.27:0/3491749712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54d80f66e0 0x7f54d81072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:24:17.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.784+0000 7f54de39d700 1 --2- 172.21.15.27:0/3491749712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54d80f5440 0x7f54d8106e00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f54d4002700 tx=0x7f54d4009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:24:17.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.784+0000 7f54caffd700 1 -- 172.21.15.27:0/3491749712 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54d4009320 con 0x7f54d80f5440 2021-11-03T17:24:17.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.784+0000 7f54df39f700 1 -- 172.21.15.27:0/3491749712 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54d8108380 con 0x7f54d80f5440 2021-11-03T17:24:17.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.784+0000 7f54caffd700 1 -- 172.21.15.27:0/3491749712 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f54d400f040 con 0x7f54d80f5440 2021-11-03T17:24:17.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.784+0000 7f54caffd700 1 -- 172.21.15.27:0/3491749712 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54d4014900 con 0x7f54d80f5440 2021-11-03T17:24:17.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.784+0000 7f54df39f700 1 -- 172.21.15.27:0/3491749712 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54d8108940 con 0x7f54d80f5440 2021-11-03T17:24:17.787 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.785+0000 7f54df39f700 1 -- 172.21.15.27:0/3491749712 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54d8059100 con 0x7f54d80f5440 2021-11-03T17:24:17.788 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.785+0000 7f54caffd700 1 -- 172.21.15.27:0/3491749712 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f54d4009480 con 0x7f54d80f5440 2021-11-03T17:24:17.788 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.785+0000 7f54caffd700 1 --2- 172.21.15.27:0/3491749712 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f54c4064100 0x7f54c40665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:24:17.788 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.786+0000 7f54caffd700 1 -- 172.21.15.27:0/3491749712 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f54d4080a50 con 0x7f54d80f5440 2021-11-03T17:24:17.788 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.786+0000 7f54ddb9c700 1 --2- 172.21.15.27:0/3491749712 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f54c4064100 0x7f54c40665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:24:17.789 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.786+0000 7f54ddb9c700 1 --2- 172.21.15.27:0/3491749712 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f54c4064100 0x7f54c40665c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f54cc004630 tx=0x7f54cc009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:24:17.789 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.790+0000 7f54caffd700 1 -- 172.21.15.27:0/3491749712 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f54d4052db0 con 0x7f54d80f5440 2021-11-03T17:24:17.990 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.995+0000 7f54df39f700 1 -- 172.21.15.27:0/3491749712 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f54d80fed00 con 0x7f54d80f5440 2021-11-03T17:24:17.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.995+0000 7f54caffd700 1 -- 172.21.15.27:0/3491749712 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f54d40529a0 con 0x7f54d80f5440 2021-11-03T17:24:17.992 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:24:17.992 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:24:17.993 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:24:17.993 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:24:17.993 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:24:17.994 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:24:17.994 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:24:17.994 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:24:17.994 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:24:17.995 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:24:17.995 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:24:17.995 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:24:17.995 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:24:17.996 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:24:17.996 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:24:17.996 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:24:17.996 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:24:17.996 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:24:17.997 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:24:17.997 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:24:17.997 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:24:17.998 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:24:17.998 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:24:17.998 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:24:17.999 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:24:17.999 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:24:17.999 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:24:18.000 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:24:18.000 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:24:18.001 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:24:18.001 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:24:18.001 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:24:18.001 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:24:18.001 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:24:18.002 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:24:18.002 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:24:18.002 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:24:18.002 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:24:18.003 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:24:18.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.997+0000 7f54df39f700 1 -- 172.21.15.27:0/3491749712 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f54c4064100 msgr2=0x7f54c40665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:24:18.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.997+0000 7f54df39f700 1 --2- 172.21.15.27:0/3491749712 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f54c4064100 0x7f54c40665c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f54cc004630 tx=0x7f54cc009b30).stop 2021-11-03T17:24:18.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.997+0000 7f54df39f700 1 -- 172.21.15.27:0/3491749712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54d80f5440 msgr2=0x7f54d8106e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:24:18.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.997+0000 7f54df39f700 1 --2- 172.21.15.27:0/3491749712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54d80f5440 0x7f54d8106e00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7f54d4002700 tx=0x7f54d4009d80).stop 2021-11-03T17:24:18.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.997+0000 7f54df39f700 1 -- 172.21.15.27:0/3491749712 shutdown_connections 2021-11-03T17:24:18.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.997+0000 7f54df39f700 1 --2- 172.21.15.27:0/3491749712 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f54c4064100 0x7f54c40665c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:18.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.997+0000 7f54df39f700 1 --2- 172.21.15.27:0/3491749712 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54d80f5440 0x7f54d8106e00 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:18.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.997+0000 7f54df39f700 1 --2- 172.21.15.27:0/3491749712 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54d80f66e0 0x7f54d81072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:18.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.997+0000 7f54df39f700 1 -- 172.21.15.27:0/3491749712 >> 172.21.15.27:0/3491749712 conn(0x7f54d80f0ba0 msgr2=0x7f54d80fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:24:18.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.997+0000 7f54df39f700 1 -- 172.21.15.27:0/3491749712 shutdown_connections 2021-11-03T17:24:18.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:17.997+0000 7f54df39f700 1 -- 172.21.15.27:0/3491749712 wait complete. 2021-11-03T17:24:18.007 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:24:18.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:24:18.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:24:18.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:24:18.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:18.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:18.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:18.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:18.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:17 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1740939101' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:18.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:24:18.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:24:18.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:24:18.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:18.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:18.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:18.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:18.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:17 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1740939101' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:19.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:18 smithi115 ceph-mon[65453]: from='client.34408 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:24:19.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:18 smithi115 ceph-mon[65453]: from='client.34412 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:24:19.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:18 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:24:19.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:18 smithi115 ceph-mon[65453]: pgmap v288: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:24:19.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:18 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3491749712' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:24:19.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:18 smithi027 ceph-mon[97244]: from='client.34408 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:24:19.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:18 smithi027 ceph-mon[97244]: from='client.34412 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:24:19.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:18 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:24:19.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:18 smithi027 ceph-mon[97244]: pgmap v288: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:24:19.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:18 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3491749712' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:24:20.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:19 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 502.370985 secs 2021-11-03T17:24:20.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:19 smithi115 ceph-mon[65453]: evicting unresponsive client (24331), after 300.844 seconds 2021-11-03T17:24:20.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:19 smithi115 ceph-mon[65453]: Evicting (and blocklisting) client session 24331 (v1:192.168.0.1:0/2120483524) 2021-11-03T17:24:20.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:19 smithi115 ceph-mon[65453]: from='mds.1 [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060]' entity='mds.cephfs.smithi027.jdwohn' cmd=[{"prefix":"osd blocklist", "blocklistop":"add","addr":"v1:192.168.0.1:0/2120483524"}]: dispatch 2021-11-03T17:24:20.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:19 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 502.370985 secs 2021-11-03T17:24:20.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:19 smithi027 ceph-mon[97244]: evicting unresponsive client (24331), after 300.844 seconds 2021-11-03T17:24:20.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:19 smithi027 ceph-mon[97244]: Evicting (and blocklisting) client session 24331 (v1:192.168.0.1:0/2120483524) 2021-11-03T17:24:20.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:19 smithi027 ceph-mon[97244]: from='mds.1 [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060]' entity='mds.cephfs.smithi027.jdwohn' cmd=[{"prefix":"osd blocklist", "blocklistop":"add","addr":"v1:192.168.0.1:0/2120483524"}]: dispatch 2021-11-03T17:24:21.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:20 smithi115 ceph-mon[65453]: pgmap v289: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:24:21.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:20 smithi115 ceph-mon[65453]: from='mds.1 [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060]' entity='mds.cephfs.smithi027.jdwohn' cmd='[{"prefix":"osd blocklist", "blocklistop":"add","addr":"v1:192.168.0.1:0/2120483524"}]': finished 2021-11-03T17:24:21.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:20 smithi115 ceph-mon[65453]: osdmap e85: 6 total, 6 up, 6 in 2021-11-03T17:24:21.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:20 smithi027 ceph-mon[97244]: pgmap v289: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:24:21.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:20 smithi027 ceph-mon[97244]: from='mds.1 [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060]' entity='mds.cephfs.smithi027.jdwohn' cmd='[{"prefix":"osd blocklist", "blocklistop":"add","addr":"v1:192.168.0.1:0/2120483524"}]': finished 2021-11-03T17:24:21.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:20 smithi027 ceph-mon[97244]: osdmap e85: 6 total, 6 up, 6 in 2021-11-03T17:24:23.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:22 smithi115 ceph-mon[65453]: pgmap v291: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:24:23.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:22 smithi027 ceph-mon[97244]: pgmap v291: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:24:25.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:24 smithi115 ceph-mon[65453]: pgmap v292: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 307 B/s wr, 0 op/s 2021-11-03T17:24:25.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:24 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 507.371099 secs 2021-11-03T17:24:25.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:24 smithi027 ceph-mon[97244]: pgmap v292: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 307 B/s wr, 0 op/s 2021-11-03T17:24:25.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:24 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 507.371099 secs 2021-11-03T17:24:27.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:26 smithi115 ceph-mon[65453]: pgmap v293: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 307 B/s wr, 0 op/s 2021-11-03T17:24:27.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:26 smithi027 ceph-mon[97244]: pgmap v293: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 307 B/s wr, 0 op/s 2021-11-03T17:24:28.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:24:28.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:24:28.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:24:28.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:28.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:28.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:28.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:28.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:24:28.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:24:28.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:24:28.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:28.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:28.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:28.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:29.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:28 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:24:29.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:28 smithi115 ceph-mon[65453]: pgmap v294: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 307 B/s wr, 0 op/s 2021-11-03T17:24:29.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:28 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:24:29.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:28 smithi027 ceph-mon[97244]: pgmap v294: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 307 B/s wr, 0 op/s 2021-11-03T17:24:30.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:29 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 512.371230 secs 2021-11-03T17:24:30.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:29 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 512.371230 secs 2021-11-03T17:24:31.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:30 smithi115 ceph-mon[65453]: pgmap v295: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 307 B/s wr, 0 op/s 2021-11-03T17:24:31.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:30 smithi027 ceph-mon[97244]: pgmap v295: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 307 B/s wr, 0 op/s 2021-11-03T17:24:33.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:32 smithi115 ceph-mon[65453]: pgmap v296: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 257 B/s wr, 0 op/s 2021-11-03T17:24:33.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:32 smithi027 ceph-mon[97244]: pgmap v296: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 257 B/s wr, 0 op/s 2021-11-03T17:24:35.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:34 smithi027 ceph-mon[97244]: pgmap v297: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2021-11-03T17:24:35.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:34 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 517.371356 secs 2021-11-03T17:24:35.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:34 smithi027 ceph-mon[97244]: evicting unresponsive client smithi027:0 (14486), after 304.645 seconds 2021-11-03T17:24:35.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:34 smithi027 ceph-mon[97244]: Evicting (and blocklisting) client session 14486 (v1:192.168.0.1:0/2105322717) 2021-11-03T17:24:35.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:34 smithi027 ceph-mon[97244]: from='mds.1 [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060]' entity='mds.cephfs.smithi027.jdwohn' cmd=[{"prefix":"osd blocklist", "blocklistop":"add","addr":"v1:192.168.0.1:0/2105322717"}]: dispatch 2021-11-03T17:24:35.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:34 smithi115 ceph-mon[65453]: pgmap v297: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2021-11-03T17:24:35.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:34 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 517.371356 secs 2021-11-03T17:24:35.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:34 smithi115 ceph-mon[65453]: evicting unresponsive client smithi027:0 (14486), after 304.645 seconds 2021-11-03T17:24:35.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:34 smithi115 ceph-mon[65453]: Evicting (and blocklisting) client session 14486 (v1:192.168.0.1:0/2105322717) 2021-11-03T17:24:35.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:34 smithi115 ceph-mon[65453]: from='mds.1 [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060]' entity='mds.cephfs.smithi027.jdwohn' cmd=[{"prefix":"osd blocklist", "blocklistop":"add","addr":"v1:192.168.0.1:0/2105322717"}]: dispatch 2021-11-03T17:24:36.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:36 smithi027 ceph-mon[97244]: from='mds.1 [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060]' entity='mds.cephfs.smithi027.jdwohn' cmd='[{"prefix":"osd blocklist", "blocklistop":"add","addr":"v1:192.168.0.1:0/2105322717"}]': finished 2021-11-03T17:24:36.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:36 smithi027 ceph-mon[97244]: osdmap e86: 6 total, 6 up, 6 in 2021-11-03T17:24:36.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:36 smithi115 ceph-mon[65453]: from='mds.1 [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060]' entity='mds.cephfs.smithi027.jdwohn' cmd='[{"prefix":"osd blocklist", "blocklistop":"add","addr":"v1:192.168.0.1:0/2105322717"}]': finished 2021-11-03T17:24:36.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:36 smithi115 ceph-mon[65453]: osdmap e86: 6 total, 6 up, 6 in 2021-11-03T17:24:37.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:37 smithi027 ceph-mon[97244]: pgmap v299: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:24:37.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:37 smithi115 ceph-mon[65453]: pgmap v299: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:24:38.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:24:38.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:24:38.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:24:38.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:38.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:38.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:38.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:38.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:24:38.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:24:38.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:24:38.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:38.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:38.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:38.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:39.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:39 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:24:39.382 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:39 smithi027 ceph-mon[97244]: pgmap v300: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 102 B/s wr, 0 op/s 2021-11-03T17:24:39.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:39 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:24:39.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:39 smithi115 ceph-mon[65453]: pgmap v300: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 102 B/s wr, 0 op/s 2021-11-03T17:24:40.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 522.371491 secs 2021-11-03T17:24:40.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 522.371491 secs 2021-11-03T17:24:41.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:41 smithi027 ceph-mon[97244]: pgmap v301: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 307 B/s wr, 0 op/s 2021-11-03T17:24:41.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:41 smithi115 ceph-mon[65453]: pgmap v301: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 307 B/s wr, 0 op/s 2021-11-03T17:24:42.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:42 smithi115 ceph-mon[65453]: pgmap v302: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 307 B/s wr, 0 op/s 2021-11-03T17:24:42.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:42 smithi027 ceph-mon[97244]: pgmap v302: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 307 B/s wr, 0 op/s 2021-11-03T17:24:45.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:44 smithi115 ceph-mon[65453]: pgmap v303: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 307 B/s wr, 0 op/s 2021-11-03T17:24:45.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:44 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 527.371628 secs 2021-11-03T17:24:45.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:44 smithi027 ceph-mon[97244]: pgmap v303: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 307 B/s wr, 0 op/s 2021-11-03T17:24:45.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:44 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 527.371628 secs 2021-11-03T17:24:47.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:46 smithi115 ceph-mon[65453]: pgmap v304: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 281 B/s wr, 0 op/s 2021-11-03T17:24:47.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:46 smithi027 ceph-mon[97244]: pgmap v304: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 281 B/s wr, 0 op/s 2021-11-03T17:24:47.946 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:24:47.946 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:24:47.947 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:24:47.947 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:47.947 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:47.947 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:47.948 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:48.099 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.101+0000 7f329e325700 1 -- 172.21.15.27:0/1173584810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32980f5440 msgr2=0x7f32980f5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:24:48.099 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.101+0000 7f329e325700 1 --2- 172.21.15.27:0/1173584810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32980f5440 0x7f32980f5860 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 rx=0x7f3294004660 tx=0x7f3294009b30).stop 2021-11-03T17:24:48.100 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.101+0000 7f329e325700 1 -- 172.21.15.27:0/1173584810 shutdown_connections 2021-11-03T17:24:48.100 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.101+0000 7f329e325700 1 --2- 172.21.15.27:0/1173584810 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32980f66e0 0x7f32980f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:48.100 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.101+0000 7f329e325700 1 --2- 172.21.15.27:0/1173584810 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32980f5440 0x7f32980f5860 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:48.100 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.101+0000 7f329e325700 1 -- 172.21.15.27:0/1173584810 >> 172.21.15.27:0/1173584810 conn(0x7f32980f0ba0 msgr2=0x7f32980f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:24:48.101 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.102+0000 7f329e325700 1 -- 172.21.15.27:0/1173584810 shutdown_connections 2021-11-03T17:24:48.101 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.102+0000 7f329e325700 1 -- 172.21.15.27:0/1173584810 wait complete. 2021-11-03T17:24:48.101 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.102+0000 7f329e325700 1 Processor -- start 2021-11-03T17:24:48.101 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.102+0000 7f329e325700 1 -- start start 2021-11-03T17:24:48.102 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.102+0000 7f329e325700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32980f5440 0x7f3298109ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:24:48.102 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.103+0000 7f329e325700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32980f66e0 0x7f329810bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:24:48.102 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.103+0000 7f329e325700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32980603c0 con 0x7f32980f5440 2021-11-03T17:24:48.102 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.103+0000 7f329e325700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3298060500 con 0x7f32980f66e0 2021-11-03T17:24:48.103 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.103+0000 7f329d323700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32980f5440 0x7f3298109ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:24:48.103 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.103+0000 7f329d323700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32980f5440 0x7f3298109ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57280/0 (socket says 172.21.15.27:57280) 2021-11-03T17:24:48.103 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.103+0000 7f329d323700 1 -- 172.21.15.27:0/2039282517 learned_addr learned my addr 172.21.15.27:0/2039282517 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:24:48.103 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.103+0000 7f329cb22700 1 --2- 172.21.15.27:0/2039282517 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32980f66e0 0x7f329810bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:24:48.103 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.103+0000 7f329d323700 1 -- 172.21.15.27:0/2039282517 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32980f66e0 msgr2=0x7f329810bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:24:48.104 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.103+0000 7f329d323700 1 --2- 172.21.15.27:0/2039282517 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32980f66e0 0x7f329810bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:48.104 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.103+0000 7f329d323700 1 -- 172.21.15.27:0/2039282517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3294005040 con 0x7f32980f5440 2021-11-03T17:24:48.104 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.103+0000 7f329cb22700 1 --2- 172.21.15.27:0/2039282517 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32980f66e0 0x7f329810bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:24:48.104 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.103+0000 7f329d323700 1 --2- 172.21.15.27:0/2039282517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32980f5440 0x7f3298109ac0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f3294004f60 tx=0x7f3294009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:24:48.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.103+0000 7f328dffb700 1 -- 172.21.15.27:0/2039282517 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3294018070 con 0x7f32980f5440 2021-11-03T17:24:48.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.103+0000 7f328dffb700 1 -- 172.21.15.27:0/2039282517 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f3294007ba0 con 0x7f32980f5440 2021-11-03T17:24:48.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.103+0000 7f329e325700 1 -- 172.21.15.27:0/2039282517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f329810c510 con 0x7f32980f5440 2021-11-03T17:24:48.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.103+0000 7f328dffb700 1 -- 172.21.15.27:0/2039282517 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f329401c620 con 0x7f32980f5440 2021-11-03T17:24:48.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.103+0000 7f329e325700 1 -- 172.21.15.27:0/2039282517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f329810ca40 con 0x7f32980f5440 2021-11-03T17:24:48.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.104+0000 7f329e325700 1 -- 172.21.15.27:0/2039282517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3298059100 con 0x7f32980f5440 2021-11-03T17:24:48.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.105+0000 7f328dffb700 1 -- 172.21.15.27:0/2039282517 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f329401d420 con 0x7f32980f5440 2021-11-03T17:24:48.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.105+0000 7f328dffb700 1 --2- 172.21.15.27:0/2039282517 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3284064100 0x7f32840665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:24:48.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.105+0000 7f328dffb700 1 -- 172.21.15.27:0/2039282517 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f3294082670 con 0x7f32980f5440 2021-11-03T17:24:48.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.105+0000 7f329cb22700 1 --2- 172.21.15.27:0/2039282517 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3284064100 0x7f32840665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:24:48.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.106+0000 7f329cb22700 1 --2- 172.21.15.27:0/2039282517 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3284064100 0x7f32840665c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f32880041d0 tx=0x7f3288009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:24:48.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.109+0000 7f328dffb700 1 -- 172.21.15.27:0/2039282517 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f3294054a30 con 0x7f32980f5440 2021-11-03T17:24:48.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:24:48.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:24:48.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:24:48.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:48.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:48.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:48.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:48.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.288+0000 7f329e325700 1 -- 172.21.15.27:0/2039282517 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f32980fddb0 con 0x7f3284064100 2021-11-03T17:24:48.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.290+0000 7f328dffb700 1 -- 172.21.15.27:0/2039282517 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f3288004600 con 0x7f3284064100 2021-11-03T17:24:48.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.292+0000 7f329e325700 1 -- 172.21.15.27:0/2039282517 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3284064100 msgr2=0x7f32840665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:24:48.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.292+0000 7f329e325700 1 --2- 172.21.15.27:0/2039282517 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3284064100 0x7f32840665c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7f32880041d0 tx=0x7f3288009040).stop 2021-11-03T17:24:48.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.292+0000 7f329e325700 1 -- 172.21.15.27:0/2039282517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32980f5440 msgr2=0x7f3298109ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:24:48.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.292+0000 7f329e325700 1 --2- 172.21.15.27:0/2039282517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32980f5440 0x7f3298109ac0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f3294004f60 tx=0x7f3294009b30).stop 2021-11-03T17:24:48.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.292+0000 7f329e325700 1 -- 172.21.15.27:0/2039282517 shutdown_connections 2021-11-03T17:24:48.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.292+0000 7f329e325700 1 --2- 172.21.15.27:0/2039282517 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3284064100 0x7f32840665c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:48.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.292+0000 7f329e325700 1 --2- 172.21.15.27:0/2039282517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32980f5440 0x7f3298109ac0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:48.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.292+0000 7f329e325700 1 --2- 172.21.15.27:0/2039282517 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32980f66e0 0x7f329810bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:48.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.292+0000 7f329e325700 1 -- 172.21.15.27:0/2039282517 >> 172.21.15.27:0/2039282517 conn(0x7f32980f0ba0 msgr2=0x7f32980fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:24:48.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.292+0000 7f329e325700 1 -- 172.21.15.27:0/2039282517 shutdown_connections 2021-11-03T17:24:48.290 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.292+0000 7f329e325700 1 -- 172.21.15.27:0/2039282517 wait complete. 2021-11-03T17:24:48.297 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:24:48.389 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.392+0000 7fe1260e9700 1 -- 172.21.15.27:0/2469037972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1200f6810 msgr2=0x7fe1200f6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:24:48.389 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.392+0000 7fe1260e9700 1 --2- 172.21.15.27:0/2469037972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1200f6810 0x7fe1200f6c70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7fe114004660 tx=0x7fe114009b30).stop 2021-11-03T17:24:48.390 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.392+0000 7fe1260e9700 1 -- 172.21.15.27:0/2469037972 shutdown_connections 2021-11-03T17:24:48.390 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.392+0000 7fe1260e9700 1 --2- 172.21.15.27:0/2469037972 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1200f6810 0x7fe1200f6c70 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:48.390 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.392+0000 7fe1260e9700 1 --2- 172.21.15.27:0/2469037972 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1200f5570 0x7fe1200f5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:48.391 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.392+0000 7fe1260e9700 1 -- 172.21.15.27:0/2469037972 >> 172.21.15.27:0/2469037972 conn(0x7fe1200f0cd0 msgr2=0x7fe1200f30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:24:48.391 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.392+0000 7fe1260e9700 1 -- 172.21.15.27:0/2469037972 shutdown_connections 2021-11-03T17:24:48.391 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.392+0000 7fe1260e9700 1 -- 172.21.15.27:0/2469037972 wait complete. 2021-11-03T17:24:48.391 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.392+0000 7fe1260e9700 1 Processor -- start 2021-11-03T17:24:48.392 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.393+0000 7fe1260e9700 1 -- start start 2021-11-03T17:24:48.392 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.393+0000 7fe1260e9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1200f5570 0x7fe120105e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:24:48.392 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.393+0000 7fe1260e9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1200f6810 0x7fe120106320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:24:48.392 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.393+0000 7fe1260e9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe120106920 con 0x7fe1200f6810 2021-11-03T17:24:48.393 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.393+0000 7fe1260e9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe120107130 con 0x7fe1200f5570 2021-11-03T17:24:48.393 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.393+0000 7fe11effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1200f6810 0x7fe120106320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:24:48.393 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.393+0000 7fe11effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1200f6810 0x7fe120106320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57292/0 (socket says 172.21.15.27:57292) 2021-11-03T17:24:48.393 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.393+0000 7fe11effd700 1 -- 172.21.15.27:0/3793222606 learned_addr learned my addr 172.21.15.27:0/3793222606 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:24:48.394 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.393+0000 7fe11f7fe700 1 --2- 172.21.15.27:0/3793222606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1200f5570 0x7fe120105e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:24:48.394 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.393+0000 7fe11effd700 1 -- 172.21.15.27:0/3793222606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1200f5570 msgr2=0x7fe120105e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:24:48.394 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.393+0000 7fe11effd700 1 --2- 172.21.15.27:0/3793222606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1200f5570 0x7fe120105e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:48.394 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.393+0000 7fe11effd700 1 -- 172.21.15.27:0/3793222606 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe114005040 con 0x7fe1200f6810 2021-11-03T17:24:48.395 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.393+0000 7fe11effd700 1 --2- 172.21.15.27:0/3793222606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1200f6810 0x7fe120106320 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7fe114009990 tx=0x7fe114009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:24:48.395 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.393+0000 7fe107fff700 1 -- 172.21.15.27:0/3793222606 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe114018070 con 0x7fe1200f6810 2021-11-03T17:24:48.395 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.393+0000 7fe107fff700 1 -- 172.21.15.27:0/3793222606 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fe114007ba0 con 0x7fe1200f6810 2021-11-03T17:24:48.395 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.393+0000 7fe1260e9700 1 -- 172.21.15.27:0/3793222606 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe120107350 con 0x7fe1200f6810 2021-11-03T17:24:48.396 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.393+0000 7fe107fff700 1 -- 172.21.15.27:0/3793222606 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe11401c3c0 con 0x7fe1200f6810 2021-11-03T17:24:48.396 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.394+0000 7fe1260e9700 1 -- 172.21.15.27:0/3793222606 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1201078b0 con 0x7fe1200f6810 2021-11-03T17:24:48.397 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.395+0000 7fe1260e9700 1 -- 172.21.15.27:0/3793222606 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1200f4960 con 0x7fe1200f6810 2021-11-03T17:24:48.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.395+0000 7fe107fff700 1 -- 172.21.15.27:0/3793222606 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fe11401d420 con 0x7fe1200f6810 2021-11-03T17:24:48.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.395+0000 7fe107fff700 1 --2- 172.21.15.27:0/3793222606 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe108064100 0x7fe1080665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:24:48.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.395+0000 7fe107fff700 1 -- 172.21.15.27:0/3793222606 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fe114082470 con 0x7fe1200f6810 2021-11-03T17:24:48.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.395+0000 7fe11f7fe700 1 --2- 172.21.15.27:0/3793222606 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe108064100 0x7fe1080665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:24:48.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.396+0000 7fe11f7fe700 1 --2- 172.21.15.27:0/3793222606 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe108064100 0x7fe1080665c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fe120105ba0 tx=0x7fe11000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:24:48.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.399+0000 7fe107fff700 1 -- 172.21.15.27:0/3793222606 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fe114054830 con 0x7fe1200f6810 2021-11-03T17:24:48.569 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.573+0000 7fe1260e9700 1 -- 172.21.15.27:0/3793222606 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe1200fde10 con 0x7fe108064100 2021-11-03T17:24:48.583 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.588+0000 7fe107fff700 1 -- 172.21.15.27:0/3793222606 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fe1200fde10 con 0x7fe108064100 2021-11-03T17:24:48.584 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:24:48.584 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (16m) 6m ago 17m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:24:48.584 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (8m) 6m ago 17m 7205k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:24:48.584 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (7m) 5m ago 16m 7239k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:24:48.585 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (16m) 6m ago 17m 38.9M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:24:48.585 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (9m) 6m ago 12m 15.4M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:24:48.585 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (12m) 6m ago 12m 3836M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:24:48.586 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (8m) 5m ago 12m 14.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:24:48.586 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (10m) 5m ago 12m 16.3M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:24:48.586 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (10m) 6m ago 18m 470M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:24:48.586 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (9m) 5m ago 16m 412M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:24:48.587 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (9m) 6m ago 19m 81.2M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:24:48.587 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (8m) 5m ago 16m 69.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:24:48.587 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (17m) 6m ago 17m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:24:48.587 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (16m) 5m ago 16m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:24:48.587 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (7m) 6m ago 16m 243M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:24:48.588 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (7m) 6m ago 15m 270M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:24:48.588 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (6m) 6m ago 15m 11.8M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:24:48.588 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (6m) 5m ago 15m 289M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:24:48.588 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (5m) 5m ago 14m 235M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:24:48.589 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (5m) 5m ago 14m 11.5M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:24:48.589 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (16m) 6m ago 16m 45.7M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:24:48.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.590+0000 7fe1260e9700 1 -- 172.21.15.27:0/3793222606 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe108064100 msgr2=0x7fe1080665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:24:48.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.590+0000 7fe1260e9700 1 --2- 172.21.15.27:0/3793222606 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe108064100 0x7fe1080665c0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fe120105ba0 tx=0x7fe11000b040).stop 2021-11-03T17:24:48.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.590+0000 7fe1260e9700 1 -- 172.21.15.27:0/3793222606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1200f6810 msgr2=0x7fe120106320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:24:48.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.590+0000 7fe1260e9700 1 --2- 172.21.15.27:0/3793222606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1200f6810 0x7fe120106320 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7fe114009990 tx=0x7fe114009b30).stop 2021-11-03T17:24:48.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.590+0000 7fe1260e9700 1 -- 172.21.15.27:0/3793222606 shutdown_connections 2021-11-03T17:24:48.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.590+0000 7fe1260e9700 1 --2- 172.21.15.27:0/3793222606 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe108064100 0x7fe1080665c0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:48.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.590+0000 7fe1260e9700 1 --2- 172.21.15.27:0/3793222606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1200f5570 0x7fe120105e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:48.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.590+0000 7fe1260e9700 1 --2- 172.21.15.27:0/3793222606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1200f6810 0x7fe120106320 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:48.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.590+0000 7fe1260e9700 1 -- 172.21.15.27:0/3793222606 >> 172.21.15.27:0/3793222606 conn(0x7fe1200f0cd0 msgr2=0x7fe1200fa260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:24:48.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.590+0000 7fe1260e9700 1 -- 172.21.15.27:0/3793222606 shutdown_connections 2021-11-03T17:24:48.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.590+0000 7fe1260e9700 1 -- 172.21.15.27:0/3793222606 wait complete. 2021-11-03T17:24:48.687 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.690+0000 7f50e945f700 1 -- 172.21.15.27:0/3402840652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50e40f7b80 msgr2=0x7f50e40f7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:24:48.687 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.690+0000 7f50e945f700 1 --2- 172.21.15.27:0/3402840652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50e40f7b80 0x7f50e40f7fe0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f50cc004660 tx=0x7f50cc009b30).stop 2021-11-03T17:24:48.687 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.690+0000 7f50e945f700 1 -- 172.21.15.27:0/3402840652 shutdown_connections 2021-11-03T17:24:48.688 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.690+0000 7f50e945f700 1 --2- 172.21.15.27:0/3402840652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50e40f7b80 0x7f50e40f7fe0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:48.688 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.690+0000 7f50e945f700 1 --2- 172.21.15.27:0/3402840652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50e40f68e0 0x7f50e40f6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:48.688 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.690+0000 7f50e945f700 1 -- 172.21.15.27:0/3402840652 >> 172.21.15.27:0/3402840652 conn(0x7f50e40f2020 msgr2=0x7f50e40f4460 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:24:48.689 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.690+0000 7f50e945f700 1 -- 172.21.15.27:0/3402840652 shutdown_connections 2021-11-03T17:24:48.689 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.690+0000 7f50e945f700 1 -- 172.21.15.27:0/3402840652 wait complete. 2021-11-03T17:24:48.689 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.691+0000 7f50e945f700 1 Processor -- start 2021-11-03T17:24:48.689 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.691+0000 7f50e945f700 1 -- start start 2021-11-03T17:24:48.690 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.691+0000 7f50e945f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50e40f68e0 0x7f50e4107290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:24:48.690 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.691+0000 7f50e945f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50e4107780 0x7f50e4108670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:24:48.690 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.691+0000 7f50e945f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e4107c50 con 0x7f50e40f68e0 2021-11-03T17:24:48.690 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.691+0000 7f50e945f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e4107d90 con 0x7f50e4107780 2021-11-03T17:24:48.690 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.691+0000 7f50e2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50e40f68e0 0x7f50e4107290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:24:48.691 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.691+0000 7f50e2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50e40f68e0 0x7f50e4107290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57300/0 (socket says 172.21.15.27:57300) 2021-11-03T17:24:48.691 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.691+0000 7f50e2ffd700 1 -- 172.21.15.27:0/271995680 learned_addr learned my addr 172.21.15.27:0/271995680 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:24:48.691 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.691+0000 7f50e27fc700 1 --2- 172.21.15.27:0/271995680 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50e4107780 0x7f50e4108670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:24:48.691 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.691+0000 7f50e2ffd700 1 -- 172.21.15.27:0/271995680 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50e4107780 msgr2=0x7f50e4108670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:24:48.692 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.691+0000 7f50e2ffd700 1 --2- 172.21.15.27:0/271995680 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50e4107780 0x7f50e4108670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:48.692 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.691+0000 7f50e2ffd700 1 -- 172.21.15.27:0/271995680 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50cc005040 con 0x7f50e40f68e0 2021-11-03T17:24:48.692 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.691+0000 7f50e2ffd700 1 --2- 172.21.15.27:0/271995680 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50e40f68e0 0x7f50e4107290 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f50dc00c530 tx=0x7f50dc007e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:24:48.693 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.692+0000 7f50db7fe700 1 -- 172.21.15.27:0/271995680 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50dc00f650 con 0x7f50e40f68e0 2021-11-03T17:24:48.693 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.692+0000 7f50db7fe700 1 -- 172.21.15.27:0/271995680 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f50dc00fc60 con 0x7f50e40f68e0 2021-11-03T17:24:48.693 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.692+0000 7f50e945f700 1 -- 172.21.15.27:0/271995680 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50e4108c20 con 0x7f50e40f68e0 2021-11-03T17:24:48.693 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.692+0000 7f50db7fe700 1 -- 172.21.15.27:0/271995680 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50dc012cb0 con 0x7f50e40f68e0 2021-11-03T17:24:48.694 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.692+0000 7f50e945f700 1 -- 172.21.15.27:0/271995680 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50e41091e0 con 0x7f50e40f68e0 2021-11-03T17:24:48.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.692+0000 7f50e945f700 1 -- 172.21.15.27:0/271995680 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50e40f5cd0 con 0x7f50e40f68e0 2021-11-03T17:24:48.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.693+0000 7f50db7fe700 1 -- 172.21.15.27:0/271995680 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f50dc019030 con 0x7f50e40f68e0 2021-11-03T17:24:48.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.693+0000 7f50db7fe700 1 --2- 172.21.15.27:0/271995680 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f50d00640b0 0x7f50d0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:24:48.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.693+0000 7f50db7fe700 1 -- 172.21.15.27:0/271995680 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f50dc080620 con 0x7f50e40f68e0 2021-11-03T17:24:48.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.693+0000 7f50e27fc700 1 --2- 172.21.15.27:0/271995680 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f50d00640b0 0x7f50d0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:24:48.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.694+0000 7f50e27fc700 1 --2- 172.21.15.27:0/271995680 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f50d00640b0 0x7f50d0066570 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f50cc000c00 tx=0x7f50cc009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:24:48.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.697+0000 7f50db7fe700 1 -- 172.21.15.27:0/271995680 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f50dc0529a0 con 0x7f50e40f68e0 2021-11-03T17:24:48.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.933+0000 7f50e945f700 1 -- 172.21.15.27:0/271995680 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f50e4027a00 con 0x7f50e40f68e0 2021-11-03T17:24:48.929 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.933+0000 7f50db7fe700 1 -- 172.21.15.27:0/271995680 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f50dc0172e0 con 0x7f50e40f68e0 2021-11-03T17:24:48.929 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:24:48.929 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:24:48.930 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:24:48.930 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:24:48.930 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:24:48.930 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:24:48.931 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:24:48.931 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:24:48.931 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:24:48.931 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:24:48.932 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:24:48.932 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:24:48.932 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:24:48.932 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:24:48.933 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:24:48.933 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:24:48.933 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:24:48.933 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:24:48.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.935+0000 7f50e945f700 1 -- 172.21.15.27:0/271995680 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f50d00640b0 msgr2=0x7f50d0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:24:48.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.935+0000 7f50e945f700 1 --2- 172.21.15.27:0/271995680 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f50d00640b0 0x7f50d0066570 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f50cc000c00 tx=0x7f50cc009b30).stop 2021-11-03T17:24:48.935 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.935+0000 7f50e945f700 1 -- 172.21.15.27:0/271995680 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50e40f68e0 msgr2=0x7f50e4107290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:24:48.935 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.935+0000 7f50e945f700 1 --2- 172.21.15.27:0/271995680 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50e40f68e0 0x7f50e4107290 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f50dc00c530 tx=0x7f50dc007e10).stop 2021-11-03T17:24:48.935 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.935+0000 7f50e945f700 1 -- 172.21.15.27:0/271995680 shutdown_connections 2021-11-03T17:24:48.935 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.935+0000 7f50e945f700 1 --2- 172.21.15.27:0/271995680 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f50d00640b0 0x7f50d0066570 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:48.936 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.935+0000 7f50e945f700 1 --2- 172.21.15.27:0/271995680 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50e40f68e0 0x7f50e4107290 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:48.936 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.935+0000 7f50e945f700 1 --2- 172.21.15.27:0/271995680 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50e4107780 0x7f50e4108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:48.936 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.935+0000 7f50e945f700 1 -- 172.21.15.27:0/271995680 >> 172.21.15.27:0/271995680 conn(0x7f50e40f2020 msgr2=0x7f50e40fb5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:24:48.937 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.935+0000 7f50e945f700 1 -- 172.21.15.27:0/271995680 shutdown_connections 2021-11-03T17:24:48.937 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:48.935+0000 7f50e945f700 1 -- 172.21.15.27:0/271995680 wait complete. 2021-11-03T17:24:49.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.037+0000 7fa09e3a4700 1 -- 172.21.15.27:0/2238899756 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0980f77d0 msgr2=0x7fa0980f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:24:49.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.037+0000 7fa09e3a4700 1 --2- 172.21.15.27:0/2238899756 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0980f77d0 0x7fa0980f7c30 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 rx=0x7fa088004660 tx=0x7fa088009b30).stop 2021-11-03T17:24:49.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.037+0000 7fa09e3a4700 1 -- 172.21.15.27:0/2238899756 shutdown_connections 2021-11-03T17:24:49.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.037+0000 7fa09e3a4700 1 --2- 172.21.15.27:0/2238899756 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0980f77d0 0x7fa0980f7c30 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:49.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.037+0000 7fa09e3a4700 1 --2- 172.21.15.27:0/2238899756 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0980f6530 0x7fa0980f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:49.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.037+0000 7fa09e3a4700 1 -- 172.21.15.27:0/2238899756 >> 172.21.15.27:0/2238899756 conn(0x7fa0980f1cd0 msgr2=0x7fa0980f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:24:49.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.037+0000 7fa09e3a4700 1 -- 172.21.15.27:0/2238899756 shutdown_connections 2021-11-03T17:24:49.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.037+0000 7fa09e3a4700 1 -- 172.21.15.27:0/2238899756 wait complete. 2021-11-03T17:24:49.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.038+0000 7fa09e3a4700 1 Processor -- start 2021-11-03T17:24:49.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.038+0000 7fa09e3a4700 1 -- start start 2021-11-03T17:24:49.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.038+0000 7fa09e3a4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0980f6530 0x7fa098106d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:24:49.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.038+0000 7fa09e3a4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0980f77d0 0x7fa098107270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:24:49.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.038+0000 7fa09e3a4700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa098107870 con 0x7fa0980f77d0 2021-11-03T17:24:49.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.038+0000 7fa09e3a4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa098108080 con 0x7fa0980f6530 2021-11-03T17:24:49.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.038+0000 7fa09cba1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0980f77d0 0x7fa098107270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:24:49.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.038+0000 7fa09cba1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0980f77d0 0x7fa098107270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57312/0 (socket says 172.21.15.27:57312) 2021-11-03T17:24:49.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.038+0000 7fa09cba1700 1 -- 172.21.15.27:0/345060774 learned_addr learned my addr 172.21.15.27:0/345060774 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:24:49.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.038+0000 7fa09d3a2700 1 --2- 172.21.15.27:0/345060774 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0980f6530 0x7fa098106d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:24:49.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.039+0000 7fa09cba1700 1 -- 172.21.15.27:0/345060774 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0980f6530 msgr2=0x7fa098106d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:24:49.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.039+0000 7fa09cba1700 1 --2- 172.21.15.27:0/345060774 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0980f6530 0x7fa098106d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:49.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.039+0000 7fa09cba1700 1 -- 172.21.15.27:0/345060774 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa088005040 con 0x7fa0980f77d0 2021-11-03T17:24:49.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.039+0000 7fa09cba1700 1 --2- 172.21.15.27:0/345060774 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0980f77d0 0x7fa098107270 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7fa088009990 tx=0x7fa088009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:24:49.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.039+0000 7fa08dffb700 1 -- 172.21.15.27:0/345060774 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa088018070 con 0x7fa0980f77d0 2021-11-03T17:24:49.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.039+0000 7fa08dffb700 1 -- 172.21.15.27:0/345060774 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fa088007ba0 con 0x7fa0980f77d0 2021-11-03T17:24:49.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.039+0000 7fa09e3a4700 1 -- 172.21.15.27:0/345060774 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0981082a0 con 0x7fa0980f77d0 2021-11-03T17:24:49.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.039+0000 7fa08dffb700 1 -- 172.21.15.27:0/345060774 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa08801c620 con 0x7fa0980f77d0 2021-11-03T17:24:49.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.039+0000 7fa09e3a4700 1 -- 172.21.15.27:0/345060774 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa098108800 con 0x7fa0980f77d0 2021-11-03T17:24:49.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.040+0000 7fa09e3a4700 1 -- 172.21.15.27:0/345060774 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa098059100 con 0x7fa0980f77d0 2021-11-03T17:24:49.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.041+0000 7fa08dffb700 1 -- 172.21.15.27:0/345060774 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fa08801c8e0 con 0x7fa0980f77d0 2021-11-03T17:24:49.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.041+0000 7fa08dffb700 1 --2- 172.21.15.27:0/345060774 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa084064100 0x7fa0840665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:24:49.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.041+0000 7fa08dffb700 1 -- 172.21.15.27:0/345060774 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fa088082660 con 0x7fa0980f77d0 2021-11-03T17:24:49.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.041+0000 7fa09d3a2700 1 --2- 172.21.15.27:0/345060774 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa084064100 0x7fa0840665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:24:49.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.042+0000 7fa09d3a2700 1 --2- 172.21.15.27:0/345060774 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa084064100 0x7fa0840665c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7fa098106af0 tx=0x7fa094009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:24:49.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.045+0000 7fa08dffb700 1 -- 172.21.15.27:0/345060774 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fa088054a20 con 0x7fa0980f77d0 2021-11-03T17:24:49.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:48 smithi115 ceph-mon[65453]: poll took 60175.840 ms: 1 events 2021-11-03T17:24:49.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:48 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:24:49.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:48 smithi115 ceph-mon[65453]: pgmap v305: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2021-11-03T17:24:49.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:48 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/271995680' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:49.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.251+0000 7fa09e3a4700 1 -- 172.21.15.27:0/345060774 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa0980fedf0 con 0x7fa0980f77d0 2021-11-03T17:24:49.248 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.252+0000 7fa08dffb700 1 -- 172.21.15.27:0/345060774 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7fa088054610 con 0x7fa0980f77d0 2021-11-03T17:24:49.248 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:24:49.248 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:24:49.249 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:24:49.249 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:24:49.249 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:24:49.249 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:24:49.250 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:24:49.250 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:24:49.250 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:24:49.250 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:24:49.251 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:24:49.251 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:24:49.251 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:24:49.251 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:24:49.252 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:24:49.252 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:24:49.252 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:24:49.253 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:24:49.253 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:24:49.253 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:24:49.253 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:24:49.254 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:24:49.254 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:24:49.254 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:24:49.254 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:24:49.255 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:24:49.255 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:24:49.255 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:24:49.255 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:24:49.256 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:24:49.256 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:24:49.256 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:24:49.256 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:24:49.256 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:24:49.257 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:24:49.257 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:24:49.257 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:24:49.257 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:24:49.258 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:24:49.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.254+0000 7fa09e3a4700 1 -- 172.21.15.27:0/345060774 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa084064100 msgr2=0x7fa0840665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:24:49.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.254+0000 7fa09e3a4700 1 --2- 172.21.15.27:0/345060774 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa084064100 0x7fa0840665c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7fa098106af0 tx=0x7fa094009040).stop 2021-11-03T17:24:49.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.254+0000 7fa09e3a4700 1 -- 172.21.15.27:0/345060774 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0980f77d0 msgr2=0x7fa098107270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:24:49.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.254+0000 7fa09e3a4700 1 --2- 172.21.15.27:0/345060774 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0980f77d0 0x7fa098107270 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7fa088009990 tx=0x7fa088009b30).stop 2021-11-03T17:24:49.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.254+0000 7fa09e3a4700 1 -- 172.21.15.27:0/345060774 shutdown_connections 2021-11-03T17:24:49.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.254+0000 7fa09e3a4700 1 --2- 172.21.15.27:0/345060774 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa084064100 0x7fa0840665c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:49.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.254+0000 7fa09e3a4700 1 --2- 172.21.15.27:0/345060774 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0980f6530 0x7fa098106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:49.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.254+0000 7fa09e3a4700 1 --2- 172.21.15.27:0/345060774 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0980f77d0 0x7fa098107270 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:24:49.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.254+0000 7fa09e3a4700 1 -- 172.21.15.27:0/345060774 >> 172.21.15.27:0/345060774 conn(0x7fa0980f1cd0 msgr2=0x7fa0980fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:24:49.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.254+0000 7fa09e3a4700 1 -- 172.21.15.27:0/345060774 shutdown_connections 2021-11-03T17:24:49.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:24:49.254+0000 7fa09e3a4700 1 -- 172.21.15.27:0/345060774 wait complete. 2021-11-03T17:24:49.261 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:24:49.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:48 smithi027 ceph-mon[97244]: poll took 60175.840 ms: 1 events 2021-11-03T17:24:49.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:48 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:24:49.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:48 smithi027 ceph-mon[97244]: pgmap v305: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2021-11-03T17:24:49.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:48 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/271995680' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:50.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:49 smithi115 ceph-mon[65453]: from='client.34424 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:24:50.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:49 smithi115 ceph-mon[65453]: from='client.34428 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:24:50.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:49 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/345060774' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:24:50.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:49 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 532.371686 secs 2021-11-03T17:24:50.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:49 smithi027 ceph-mon[97244]: from='client.34424 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:24:50.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:49 smithi027 ceph-mon[97244]: from='client.34428 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:24:50.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:49 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/345060774' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:24:50.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:49 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 532.371686 secs 2021-11-03T17:24:51.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:50 smithi115 ceph-mon[65453]: pgmap v306: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2021-11-03T17:24:51.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:50 smithi027 ceph-mon[97244]: pgmap v306: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2021-11-03T17:24:53.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:52 smithi115 ceph-mon[65453]: pgmap v307: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2021-11-03T17:24:53.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:52 smithi027 ceph-mon[97244]: pgmap v307: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 0 B/s wr, 0 op/s 2021-11-03T17:24:55.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:54 smithi115 ceph-mon[65453]: pgmap v308: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:24:55.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:54 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 537.371801 secs 2021-11-03T17:24:55.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:54 smithi027 ceph-mon[97244]: pgmap v308: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:24:55.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:54 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 537.371801 secs 2021-11-03T17:24:56.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:24:56.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:24:56.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:24:56.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:24:57.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:56 smithi115 ceph-mon[65453]: pgmap v309: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:24:57.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:56 smithi027 ceph-mon[97244]: pgmap v309: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:24:58.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:24:58.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:24:58.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:24:58.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:58.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:58.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:58.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:58.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:24:58.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:24:58.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:24:58.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:58.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:58.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:58.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:24:59.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:58 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:24:59.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:58 smithi115 ceph-mon[65453]: pgmap v310: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:24:59.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:58 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:24:59.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:58 smithi027 ceph-mon[97244]: pgmap v310: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:00.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:24:59 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 542.371922 secs 2021-11-03T17:25:00.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:24:59 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 542.371922 secs 2021-11-03T17:25:01.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:00 smithi115 ceph-mon[65453]: pgmap v311: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:01.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:00 smithi027 ceph-mon[97244]: pgmap v311: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:03.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:02 smithi115 ceph-mon[65453]: pgmap v312: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:03.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:02 smithi027 ceph-mon[97244]: pgmap v312: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:05.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:04 smithi115 ceph-mon[65453]: pgmap v313: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:05.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:04 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 547.372046 secs 2021-11-03T17:25:05.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:04 smithi027 ceph-mon[97244]: pgmap v313: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:05.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:04 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 547.372046 secs 2021-11-03T17:25:07.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:06 smithi115 ceph-mon[65453]: pgmap v314: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:07.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:06 smithi027 ceph-mon[97244]: pgmap v314: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:08.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:25:08.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:25:08.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:25:08.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:08.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:08.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:08.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:08.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:25:08.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:25:08.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:25:08.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:08.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:08.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:08.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:09.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:08 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:25:09.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:08 smithi115 ceph-mon[65453]: pgmap v315: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:09.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:08 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:25:09.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:08 smithi027 ceph-mon[97244]: pgmap v315: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:10.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:09 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 552.372131 secs 2021-11-03T17:25:10.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:09 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 552.372131 secs 2021-11-03T17:25:11.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:10 smithi027 ceph-mon[97244]: pgmap v316: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:11.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:10 smithi115 ceph-mon[65453]: pgmap v316: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:13.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:12 smithi027 ceph-mon[97244]: pgmap v317: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:13.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:12 smithi115 ceph-mon[65453]: pgmap v317: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:15.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:14 smithi027 ceph-mon[97244]: pgmap v318: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:15.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:14 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 557.372269 secs 2021-11-03T17:25:15.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:14 smithi115 ceph-mon[65453]: pgmap v318: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:15.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:14 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 557.372269 secs 2021-11-03T17:25:17.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:16 smithi027 ceph-mon[97244]: pgmap v319: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:17.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:16 smithi115 ceph-mon[65453]: pgmap v319: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:18.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:25:18.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:25:18.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:25:18.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:18.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:18.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:18.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:18.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:25:18.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:25:18.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:25:18.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:18.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:18.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:18.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:19.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:19 smithi027 ceph-mon[97244]: pgmap v320: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:19.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:19 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:25:19.363 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.366+0000 7f0bf359e700 1 -- 172.21.15.27:0/1207279012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bf40f5440 msgr2=0x7f0bf40f5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:25:19.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.366+0000 7f0bf359e700 1 --2- 172.21.15.27:0/1207279012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bf40f5440 0x7f0bf40f5860 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f0be4004660 tx=0x7f0be4009b30).stop 2021-11-03T17:25:19.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.367+0000 7f0bf359e700 1 -- 172.21.15.27:0/1207279012 shutdown_connections 2021-11-03T17:25:19.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.367+0000 7f0bf359e700 1 --2- 172.21.15.27:0/1207279012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0bf40f66e0 0x7f0bf40f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:19.365 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.367+0000 7f0bf359e700 1 --2- 172.21.15.27:0/1207279012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bf40f5440 0x7f0bf40f5860 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:19.365 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.367+0000 7f0bf359e700 1 -- 172.21.15.27:0/1207279012 >> 172.21.15.27:0/1207279012 conn(0x7f0bf40f0ba0 msgr2=0x7f0bf40f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:25:19.365 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.367+0000 7f0bf359e700 1 -- 172.21.15.27:0/1207279012 shutdown_connections 2021-11-03T17:25:19.365 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.367+0000 7f0bf359e700 1 -- 172.21.15.27:0/1207279012 wait complete. 2021-11-03T17:25:19.366 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.367+0000 7f0bf359e700 1 Processor -- start 2021-11-03T17:25:19.366 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.367+0000 7f0bf359e700 1 -- start start 2021-11-03T17:25:19.366 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.367+0000 7f0bf359e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bf40f5440 0x7f0bf4106e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:25:19.366 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.367+0000 7f0bf359e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0bf40f66e0 0x7f0bf4107300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:25:19.367 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.368+0000 7f0bf359e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf4107900 con 0x7f0bf40f5440 2021-11-03T17:25:19.367 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.368+0000 7f0bf359e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf4108110 con 0x7f0bf40f66e0 2021-11-03T17:25:19.367 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.368+0000 7f0bf259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bf40f5440 0x7f0bf4106e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:25:19.367 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.368+0000 7f0bf259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bf40f5440 0x7f0bf4106e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57328/0 (socket says 172.21.15.27:57328) 2021-11-03T17:25:19.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.368+0000 7f0bf259c700 1 -- 172.21.15.27:0/3237291985 learned_addr learned my addr 172.21.15.27:0/3237291985 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:25:19.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.368+0000 7f0bf1d9b700 1 --2- 172.21.15.27:0/3237291985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0bf40f66e0 0x7f0bf4107300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:25:19.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.368+0000 7f0bf259c700 1 -- 172.21.15.27:0/3237291985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0bf40f66e0 msgr2=0x7f0bf4107300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:25:19.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.368+0000 7f0bf259c700 1 --2- 172.21.15.27:0/3237291985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0bf40f66e0 0x7f0bf4107300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:19.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.368+0000 7f0bf259c700 1 -- 172.21.15.27:0/3237291985 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0be4005040 con 0x7f0bf40f5440 2021-11-03T17:25:19.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.368+0000 7f0bf259c700 1 --2- 172.21.15.27:0/3237291985 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bf40f5440 0x7f0bf4106e10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f0be4004f60 tx=0x7f0be4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:25:19.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.368+0000 7f0be2ffd700 1 -- 172.21.15.27:0/3237291985 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0be4018070 con 0x7f0bf40f5440 2021-11-03T17:25:19.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.368+0000 7f0be2ffd700 1 -- 172.21.15.27:0/3237291985 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f0be4009770 con 0x7f0bf40f5440 2021-11-03T17:25:19.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.368+0000 7f0bf359e700 1 -- 172.21.15.27:0/3237291985 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bf4108330 con 0x7f0bf40f5440 2021-11-03T17:25:19.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.368+0000 7f0be2ffd700 1 -- 172.21.15.27:0/3237291985 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0be401c3c0 con 0x7f0bf40f5440 2021-11-03T17:25:19.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.368+0000 7f0bf359e700 1 -- 172.21.15.27:0/3237291985 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf4108890 con 0x7f0bf40f5440 2021-11-03T17:25:19.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.369+0000 7f0bf359e700 1 -- 172.21.15.27:0/3237291985 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bf4059100 con 0x7f0bf40f5440 2021-11-03T17:25:19.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.370+0000 7f0be2ffd700 1 -- 172.21.15.27:0/3237291985 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f0be401d420 con 0x7f0bf40f5440 2021-11-03T17:25:19.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.370+0000 7f0be2ffd700 1 --2- 172.21.15.27:0/3237291985 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0bdc064100 0x7f0bdc0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:25:19.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.370+0000 7f0be2ffd700 1 -- 172.21.15.27:0/3237291985 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f0be40825e0 con 0x7f0bf40f5440 2021-11-03T17:25:19.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.370+0000 7f0bf1d9b700 1 --2- 172.21.15.27:0/3237291985 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0bdc064100 0x7f0bdc0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:25:19.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.371+0000 7f0bf1d9b700 1 --2- 172.21.15.27:0/3237291985 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0bdc064100 0x7f0bdc0665c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f0be80041d0 tx=0x7f0be8009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:25:19.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.374+0000 7f0be2ffd700 1 -- 172.21.15.27:0/3237291985 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f0be40549a0 con 0x7f0bf40f5440 2021-11-03T17:25:19.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:19 smithi115 ceph-mon[65453]: pgmap v320: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:19.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:19 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:25:19.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.555+0000 7f0bf359e700 1 -- 172.21.15.27:0/3237291985 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0bf40fab20 con 0x7f0bdc064100 2021-11-03T17:25:19.550 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.556+0000 7f0be2ffd700 1 -- 172.21.15.27:0/3237291985 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f0bf40fab20 con 0x7f0bdc064100 2021-11-03T17:25:19.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.558+0000 7f0bf359e700 1 -- 172.21.15.27:0/3237291985 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0bdc064100 msgr2=0x7f0bdc0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:25:19.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.558+0000 7f0bf359e700 1 --2- 172.21.15.27:0/3237291985 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0bdc064100 0x7f0bdc0665c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f0be80041d0 tx=0x7f0be8009040).stop 2021-11-03T17:25:19.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.558+0000 7f0bf359e700 1 -- 172.21.15.27:0/3237291985 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bf40f5440 msgr2=0x7f0bf4106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:25:19.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.558+0000 7f0bf359e700 1 --2- 172.21.15.27:0/3237291985 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bf40f5440 0x7f0bf4106e10 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7f0be4004f60 tx=0x7f0be4009b30).stop 2021-11-03T17:25:19.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.558+0000 7f0bf359e700 1 -- 172.21.15.27:0/3237291985 shutdown_connections 2021-11-03T17:25:19.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.558+0000 7f0bf359e700 1 --2- 172.21.15.27:0/3237291985 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0bdc064100 0x7f0bdc0665c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:19.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.558+0000 7f0bf359e700 1 --2- 172.21.15.27:0/3237291985 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bf40f5440 0x7f0bf4106e10 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:19.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.558+0000 7f0bf359e700 1 --2- 172.21.15.27:0/3237291985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0bf40f66e0 0x7f0bf4107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:19.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.558+0000 7f0bf359e700 1 -- 172.21.15.27:0/3237291985 >> 172.21.15.27:0/3237291985 conn(0x7f0bf40f0ba0 msgr2=0x7f0bf405fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:25:19.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.558+0000 7f0bf359e700 1 -- 172.21.15.27:0/3237291985 shutdown_connections 2021-11-03T17:25:19.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.558+0000 7f0bf359e700 1 -- 172.21.15.27:0/3237291985 wait complete. 2021-11-03T17:25:19.562 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:25:19.655 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.659+0000 7f1a42a82700 1 -- 172.21.15.27:0/196336332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a3c0f7980 msgr2=0x7f1a3c0f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:25:19.655 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.659+0000 7f1a42a82700 1 --2- 172.21.15.27:0/196336332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a3c0f7980 0x7f1a3c0f7de0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f1a28004660 tx=0x7f1a28009b30).stop 2021-11-03T17:25:19.655 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.659+0000 7f1a42a82700 1 -- 172.21.15.27:0/196336332 shutdown_connections 2021-11-03T17:25:19.656 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.659+0000 7f1a42a82700 1 --2- 172.21.15.27:0/196336332 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a3c0f7980 0x7f1a3c0f7de0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:19.656 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.659+0000 7f1a42a82700 1 --2- 172.21.15.27:0/196336332 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a3c0f66e0 0x7f1a3c0f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:19.656 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.659+0000 7f1a42a82700 1 -- 172.21.15.27:0/196336332 >> 172.21.15.27:0/196336332 conn(0x7f1a3c0f1e40 msgr2=0x7f1a3c0f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:25:19.656 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.659+0000 7f1a42a82700 1 -- 172.21.15.27:0/196336332 shutdown_connections 2021-11-03T17:25:19.657 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.659+0000 7f1a42a82700 1 -- 172.21.15.27:0/196336332 wait complete. 2021-11-03T17:25:19.657 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.659+0000 7f1a42a82700 1 Processor -- start 2021-11-03T17:25:19.657 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.659+0000 7f1a42a82700 1 -- start start 2021-11-03T17:25:19.657 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.660+0000 7f1a42a82700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a3c0f66e0 0x7f1a3c106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:25:19.657 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.660+0000 7f1a42a82700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a3c0f7980 0x7f1a3c107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:25:19.658 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.660+0000 7f1a42a82700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c107a00 con 0x7f1a3c0f7980 2021-11-03T17:25:19.658 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.660+0000 7f1a42a82700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a3c107b40 con 0x7f1a3c0f66e0 2021-11-03T17:25:19.658 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.660+0000 7f1a3bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a3c0f7980 0x7f1a3c107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:25:19.658 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.660+0000 7f1a3bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a3c0f7980 0x7f1a3c107450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57340/0 (socket says 172.21.15.27:57340) 2021-11-03T17:25:19.659 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.660+0000 7f1a3bfff700 1 -- 172.21.15.27:0/3530131740 learned_addr learned my addr 172.21.15.27:0/3530131740 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:25:19.659 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.660+0000 7f1a4081e700 1 --2- 172.21.15.27:0/3530131740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a3c0f66e0 0x7f1a3c106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:25:19.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.660+0000 7f1a3bfff700 1 -- 172.21.15.27:0/3530131740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a3c0f66e0 msgr2=0x7f1a3c106f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:25:19.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.660+0000 7f1a3bfff700 1 --2- 172.21.15.27:0/3530131740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a3c0f66e0 0x7f1a3c106f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:19.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.660+0000 7f1a3bfff700 1 -- 172.21.15.27:0/3530131740 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a28005040 con 0x7f1a3c0f7980 2021-11-03T17:25:19.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.660+0000 7f1a3bfff700 1 --2- 172.21.15.27:0/3530131740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a3c0f7980 0x7f1a3c107450 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f1a28009990 tx=0x7f1a28009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:25:19.661 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.660+0000 7f1a397fa700 1 -- 172.21.15.27:0/3530131740 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a28018070 con 0x7f1a3c0f7980 2021-11-03T17:25:19.661 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.660+0000 7f1a42a82700 1 -- 172.21.15.27:0/3530131740 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a3c108480 con 0x7f1a3c0f7980 2021-11-03T17:25:19.661 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.660+0000 7f1a397fa700 1 -- 172.21.15.27:0/3530131740 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f1a28007ba0 con 0x7f1a3c0f7980 2021-11-03T17:25:19.661 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.660+0000 7f1a397fa700 1 -- 172.21.15.27:0/3530131740 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a2801c620 con 0x7f1a3c0f7980 2021-11-03T17:25:19.662 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.660+0000 7f1a42a82700 1 -- 172.21.15.27:0/3530131740 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a3c1089e0 con 0x7f1a3c0f7980 2021-11-03T17:25:19.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.661+0000 7f1a42a82700 1 -- 172.21.15.27:0/3530131740 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a3c045ad0 con 0x7f1a3c0f7980 2021-11-03T17:25:19.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.662+0000 7f1a397fa700 1 -- 172.21.15.27:0/3530131740 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f1a2801c890 con 0x7f1a3c0f7980 2021-11-03T17:25:19.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.662+0000 7f1a397fa700 1 --2- 172.21.15.27:0/3530131740 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1a2c0640b0 0x7f1a2c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:25:19.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.662+0000 7f1a397fa700 1 -- 172.21.15.27:0/3530131740 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f1a280825a0 con 0x7f1a3c0f7980 2021-11-03T17:25:19.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.662+0000 7f1a4081e700 1 --2- 172.21.15.27:0/3530131740 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1a2c0640b0 0x7f1a2c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:25:19.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.663+0000 7f1a4081e700 1 --2- 172.21.15.27:0/3530131740 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1a2c0640b0 0x7f1a2c066570 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f1a3c0f77e0 tx=0x7f1a30009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:25:19.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.667+0000 7f1a397fa700 1 -- 172.21.15.27:0/3530131740 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f1a28054960 con 0x7f1a3c0f7980 2021-11-03T17:25:19.834 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.840+0000 7f1a42a82700 1 -- 172.21.15.27:0/3530131740 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a3c0ff010 con 0x7f1a2c0640b0 2021-11-03T17:25:19.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.854+0000 7f1a397fa700 1 -- 172.21.15.27:0/3530131740 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f1a3c0ff010 con 0x7f1a2c0640b0 2021-11-03T17:25:19.849 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:25:19.849 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (16m) 7m ago 18m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:25:19.850 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (8m) 7m ago 18m 7205k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:25:19.850 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (8m) 5m ago 17m 7239k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:25:19.850 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (16m) 7m ago 17m 38.9M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:25:19.851 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (10m) 7m ago 13m 15.4M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:25:19.851 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (13m) 7m ago 13m 3836M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:25:19.851 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (9m) 5m ago 13m 14.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:25:19.851 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (10m) 5m ago 13m 16.3M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:25:19.851 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (10m) 7m ago 19m 470M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:25:19.852 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (9m) 5m ago 17m 412M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:25:19.852 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (9m) 7m ago 19m 81.2M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:25:19.852 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (9m) 5m ago 17m 69.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:25:19.852 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (17m) 7m ago 17m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:25:19.853 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (17m) 5m ago 17m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:25:19.853 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (8m) 7m ago 16m 243M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:25:19.853 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (7m) 7m ago 16m 270M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:25:19.853 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (7m) 7m ago 15m 11.8M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:25:19.853 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (6m) 5m ago 15m 289M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:25:19.854 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (6m) 5m ago 15m 235M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:25:19.854 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (5m) 5m ago 14m 11.5M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:25:19.854 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (16m) 7m ago 17m 45.7M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:25:19.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.856+0000 7f1a42a82700 1 -- 172.21.15.27:0/3530131740 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1a2c0640b0 msgr2=0x7f1a2c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:25:19.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.856+0000 7f1a42a82700 1 --2- 172.21.15.27:0/3530131740 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1a2c0640b0 0x7f1a2c066570 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f1a3c0f77e0 tx=0x7f1a30009040).stop 2021-11-03T17:25:19.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.856+0000 7f1a42a82700 1 -- 172.21.15.27:0/3530131740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a3c0f7980 msgr2=0x7f1a3c107450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:25:19.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.856+0000 7f1a42a82700 1 --2- 172.21.15.27:0/3530131740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a3c0f7980 0x7f1a3c107450 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f1a28009990 tx=0x7f1a28009b30).stop 2021-11-03T17:25:19.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.856+0000 7f1a42a82700 1 -- 172.21.15.27:0/3530131740 shutdown_connections 2021-11-03T17:25:19.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.856+0000 7f1a42a82700 1 --2- 172.21.15.27:0/3530131740 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1a2c0640b0 0x7f1a2c066570 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:19.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.856+0000 7f1a42a82700 1 --2- 172.21.15.27:0/3530131740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a3c0f66e0 0x7f1a3c106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:19.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.856+0000 7f1a42a82700 1 --2- 172.21.15.27:0/3530131740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a3c0f7980 0x7f1a3c107450 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:19.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.856+0000 7f1a42a82700 1 -- 172.21.15.27:0/3530131740 >> 172.21.15.27:0/3530131740 conn(0x7f1a3c0f1e40 msgr2=0x7f1a3c0fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:25:19.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.856+0000 7f1a42a82700 1 -- 172.21.15.27:0/3530131740 shutdown_connections 2021-11-03T17:25:19.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.856+0000 7f1a42a82700 1 -- 172.21.15.27:0/3530131740 wait complete. 2021-11-03T17:25:19.952 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.956+0000 7f94c375f700 1 -- 172.21.15.27:0/1617623629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94bc0f6870 msgr2=0x7f94bc0f6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:25:19.953 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.956+0000 7f94c375f700 1 --2- 172.21.15.27:0/1617623629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94bc0f6870 0x7f94bc0f6cd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f94b0004660 tx=0x7f94b0009b30).stop 2021-11-03T17:25:19.953 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.956+0000 7f94c375f700 1 -- 172.21.15.27:0/1617623629 shutdown_connections 2021-11-03T17:25:19.953 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.956+0000 7f94c375f700 1 --2- 172.21.15.27:0/1617623629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94bc0f6870 0x7f94bc0f6cd0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:19.954 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.956+0000 7f94c375f700 1 --2- 172.21.15.27:0/1617623629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94bc0f55d0 0x7f94bc0f59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:25:19.954 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.956+0000 7f94c375f700 1 -- 172.21.15.27:0/1617623629 >> 172.21.15.27:0/1617623629 conn(0x7f94bc0f0d30 msgr2=0x7f94bc0f3150 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:25:19.954 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.956+0000 7f94c375f700 1 -- 172.21.15.27:0/1617623629 shutdown_connections 2021-11-03T17:25:19.954 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.956+0000 7f94c375f700 1 -- 172.21.15.27:0/1617623629 wait complete. 2021-11-03T17:25:19.955 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.957+0000 7f94c375f700 1 Processor -- start 2021-11-03T17:25:19.955 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.957+0000 7f94c375f700 1 -- start start 2021-11-03T17:25:19.955 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.957+0000 7f94c375f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94bc0f55d0 0x7f94bc106f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:25:19.955 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.957+0000 7f94c375f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94bc0f6870 0x7f94bc107480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:25:19.956 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.957+0000 7f94c375f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc107ac0 con 0x7f94bc0f55d0 2021-11-03T17:25:19.956 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.957+0000 7f94c375f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94bc108370 con 0x7f94bc0f6870 2021-11-03T17:25:19.956 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.957+0000 7f94c14fb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94bc0f55d0 0x7f94bc106f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:25:19.956 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.957+0000 7f94c14fb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94bc0f55d0 0x7f94bc106f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57348/0 (socket says 172.21.15.27:57348) 2021-11-03T17:25:19.957 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.957+0000 7f94c14fb700 1 -- 172.21.15.27:0/4015006834 learned_addr learned my addr 172.21.15.27:0/4015006834 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:25:19.957 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.957+0000 7f94c0cfa700 1 --2- 172.21.15.27:0/4015006834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94bc0f6870 0x7f94bc107480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:25:19.957 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.957+0000 7f94c14fb700 1 -- 172.21.15.27:0/4015006834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94bc0f6870 msgr2=0x7f94bc107480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:25:19.957 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.957+0000 7f94c14fb700 1 --2- 172.21.15.27:0/4015006834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94bc0f6870 0x7f94bc107480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:19.958 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.957+0000 7f94c14fb700 1 -- 172.21.15.27:0/4015006834 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b0005040 con 0x7f94bc0f55d0 2021-11-03T17:25:19.958 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.957+0000 7f94c0cfa700 1 --2- 172.21.15.27:0/4015006834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94bc0f6870 0x7f94bc107480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:25:19.958 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.958+0000 7f94c14fb700 1 --2- 172.21.15.27:0/4015006834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94bc0f55d0 0x7f94bc106f90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f94b400b790 tx=0x7f94b40092b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:25:19.958 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.958+0000 7f94b9ffb700 1 -- 172.21.15.27:0/4015006834 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94b400d700 con 0x7f94bc0f55d0 2021-11-03T17:25:19.959 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.958+0000 7f94b9ffb700 1 -- 172.21.15.27:0/4015006834 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f94b400dd10 con 0x7f94bc0f55d0 2021-11-03T17:25:19.959 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.958+0000 7f94c375f700 1 -- 172.21.15.27:0/4015006834 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94bc108650 con 0x7f94bc0f55d0 2021-11-03T17:25:19.959 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.958+0000 7f94b9ffb700 1 -- 172.21.15.27:0/4015006834 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f94b4010e60 con 0x7f94bc0f55d0 2021-11-03T17:25:19.959 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.958+0000 7f94c375f700 1 -- 172.21.15.27:0/4015006834 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94bc108c10 con 0x7f94bc0f55d0 2021-11-03T17:25:19.961 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.959+0000 7f94c375f700 1 -- 172.21.15.27:0/4015006834 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94bc0f49c0 con 0x7f94bc0f55d0 2021-11-03T17:25:19.961 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.959+0000 7f94b9ffb700 1 -- 172.21.15.27:0/4015006834 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f94b4010670 con 0x7f94bc0f55d0 2021-11-03T17:25:19.961 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.960+0000 7f94b9ffb700 1 --2- 172.21.15.27:0/4015006834 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f94a40640b0 0x7f94a4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:25:19.961 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.960+0000 7f94b9ffb700 1 -- 172.21.15.27:0/4015006834 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f94b4080cc0 con 0x7f94bc0f55d0 2021-11-03T17:25:19.962 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.960+0000 7f94c0cfa700 1 --2- 172.21.15.27:0/4015006834 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f94a40640b0 0x7f94a4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:25:19.962 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.960+0000 7f94c0cfa700 1 --2- 172.21.15.27:0/4015006834 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f94a40640b0 0x7f94a4066570 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f94b0009da0 tx=0x7f94b0009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:25:19.962 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:19.964+0000 7f94b9ffb700 1 -- 172.21.15.27:0/4015006834 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f94b4052ed0 con 0x7f94bc0f55d0 2021-11-03T17:25:20.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.194+0000 7f94c375f700 1 -- 172.21.15.27:0/4015006834 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f94bc10d5b0 con 0x7f94bc0f55d0 2021-11-03T17:25:20.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.194+0000 7f94b9ffb700 1 -- 172.21.15.27:0/4015006834 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f94b4016070 con 0x7f94bc0f55d0 2021-11-03T17:25:20.189 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:25:20.189 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:25:20.190 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:25:20.190 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:25:20.190 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:25:20.190 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:25:20.191 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:25:20.191 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:25:20.191 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:25:20.191 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:25:20.192 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:25:20.192 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:25:20.192 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:25:20.192 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:25:20.193 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:25:20.193 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:25:20.193 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:25:20.193 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:25:20.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.196+0000 7f94c375f700 1 -- 172.21.15.27:0/4015006834 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f94a40640b0 msgr2=0x7f94a4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:25:20.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.196+0000 7f94c375f700 1 --2- 172.21.15.27:0/4015006834 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f94a40640b0 0x7f94a4066570 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f94b0009da0 tx=0x7f94b0009b30).stop 2021-11-03T17:25:20.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.196+0000 7f94c375f700 1 -- 172.21.15.27:0/4015006834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94bc0f55d0 msgr2=0x7f94bc106f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:25:20.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.196+0000 7f94c375f700 1 --2- 172.21.15.27:0/4015006834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94bc0f55d0 0x7f94bc106f90 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f94b400b790 tx=0x7f94b40092b0).stop 2021-11-03T17:25:20.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.196+0000 7f94c375f700 1 -- 172.21.15.27:0/4015006834 shutdown_connections 2021-11-03T17:25:20.196 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.196+0000 7f94c375f700 1 --2- 172.21.15.27:0/4015006834 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f94a40640b0 0x7f94a4066570 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:20.196 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.196+0000 7f94c375f700 1 --2- 172.21.15.27:0/4015006834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94bc0f55d0 0x7f94bc106f90 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:20.196 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.196+0000 7f94c375f700 1 --2- 172.21.15.27:0/4015006834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94bc0f6870 0x7f94bc107480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:20.196 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.196+0000 7f94c375f700 1 -- 172.21.15.27:0/4015006834 >> 172.21.15.27:0/4015006834 conn(0x7f94bc0f0d30 msgr2=0x7f94bc05f720 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:25:20.197 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.196+0000 7f94c375f700 1 -- 172.21.15.27:0/4015006834 shutdown_connections 2021-11-03T17:25:20.197 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.196+0000 7f94c375f700 1 -- 172.21.15.27:0/4015006834 wait complete. 2021-11-03T17:25:20.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 562.372355 secs 2021-11-03T17:25:20.291 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.296+0000 7fabe9d39700 1 -- 172.21.15.27:0/2312234590 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabe40f5ef0 msgr2=0x7fabe40f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:25:20.291 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.296+0000 7fabe9d39700 1 --2- 172.21.15.27:0/2312234590 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabe40f5ef0 0x7fabe40f6920 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7fabd8004660 tx=0x7fabd8009b30).stop 2021-11-03T17:25:20.291 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.297+0000 7fabe9d39700 1 -- 172.21.15.27:0/2312234590 shutdown_connections 2021-11-03T17:25:20.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.297+0000 7fabe9d39700 1 --2- 172.21.15.27:0/2312234590 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabe40f5ef0 0x7fabe40f6920 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:20.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.297+0000 7fabe9d39700 1 --2- 172.21.15.27:0/2312234590 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabe40f55e0 0x7fabe40f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:20.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.297+0000 7fabe9d39700 1 -- 172.21.15.27:0/2312234590 >> 172.21.15.27:0/2312234590 conn(0x7fabe40f0de0 msgr2=0x7fabe40f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:25:20.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.297+0000 7fabe9d39700 1 -- 172.21.15.27:0/2312234590 shutdown_connections 2021-11-03T17:25:20.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.297+0000 7fabe9d39700 1 -- 172.21.15.27:0/2312234590 wait complete. 2021-11-03T17:25:20.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.297+0000 7fabe9d39700 1 Processor -- start 2021-11-03T17:25:20.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.297+0000 7fabe9d39700 1 -- start start 2021-11-03T17:25:20.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.297+0000 7fabe9d39700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabe40f55e0 0x7fabe40fa000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:25:20.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.297+0000 7fabe9d39700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabe40f5ef0 0x7fabe40f8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:25:20.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.297+0000 7fabe9d39700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe40f89e0 con 0x7fabe40f55e0 2021-11-03T17:25:20.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.297+0000 7fabe9d39700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabe40f8b20 con 0x7fabe40f5ef0 2021-11-03T17:25:20.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.298+0000 7fabe37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabe40f55e0 0x7fabe40fa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:25:20.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.298+0000 7fabe37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabe40f55e0 0x7fabe40fa000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57358/0 (socket says 172.21.15.27:57358) 2021-11-03T17:25:20.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.298+0000 7fabe37fe700 1 -- 172.21.15.27:0/119812430 learned_addr learned my addr 172.21.15.27:0/119812430 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:25:20.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.298+0000 7fabe2ffd700 1 --2- 172.21.15.27:0/119812430 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabe40f5ef0 0x7fabe40f8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:25:20.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.298+0000 7fabe37fe700 1 -- 172.21.15.27:0/119812430 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabe40f5ef0 msgr2=0x7fabe40f8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:25:20.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.298+0000 7fabe37fe700 1 --2- 172.21.15.27:0/119812430 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabe40f5ef0 0x7fabe40f8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:20.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.298+0000 7fabe37fe700 1 -- 172.21.15.27:0/119812430 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd8005040 con 0x7fabe40f55e0 2021-11-03T17:25:20.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.298+0000 7fabe37fe700 1 --2- 172.21.15.27:0/119812430 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabe40f55e0 0x7fabe40fa000 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7fabd400c790 tx=0x7fabd4007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:25:20.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.298+0000 7fabcbfff700 1 -- 172.21.15.27:0/119812430 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabd4002590 con 0x7fabe40f55e0 2021-11-03T17:25:20.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.298+0000 7fabcbfff700 1 -- 172.21.15.27:0/119812430 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fabd4002ba0 con 0x7fabe40f55e0 2021-11-03T17:25:20.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.298+0000 7fabe9d39700 1 -- 172.21.15.27:0/119812430 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabe40f8e00 con 0x7fabe40f55e0 2021-11-03T17:25:20.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.298+0000 7fabcbfff700 1 -- 172.21.15.27:0/119812430 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fabd4013d90 con 0x7fabe40f55e0 2021-11-03T17:25:20.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.298+0000 7fabe9d39700 1 -- 172.21.15.27:0/119812430 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabe410efa0 con 0x7fabe40f55e0 2021-11-03T17:25:20.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.299+0000 7fabe9d39700 1 -- 172.21.15.27:0/119812430 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabe4045ad0 con 0x7fabe40f55e0 2021-11-03T17:25:20.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.300+0000 7fabcbfff700 1 -- 172.21.15.27:0/119812430 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fabd401a060 con 0x7fabe40f55e0 2021-11-03T17:25:20.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.300+0000 7fabcbfff700 1 --2- 172.21.15.27:0/119812430 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fabcc0640b0 0x7fabcc066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:25:20.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.300+0000 7fabe2ffd700 1 --2- 172.21.15.27:0/119812430 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fabcc0640b0 0x7fabcc066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:25:20.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.300+0000 7fabcbfff700 1 -- 172.21.15.27:0/119812430 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fabd4080af0 con 0x7fabe40f55e0 2021-11-03T17:25:20.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.301+0000 7fabe2ffd700 1 --2- 172.21.15.27:0/119812430 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fabcc0640b0 0x7fabcc066570 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fabd8004630 tx=0x7fabd8009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:25:20.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.304+0000 7fabcbfff700 1 -- 172.21.15.27:0/119812430 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fabd4052ca0 con 0x7fabe40f55e0 2021-11-03T17:25:20.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 562.372355 secs 2021-11-03T17:25:20.505 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.511+0000 7fabe9d39700 1 -- 172.21.15.27:0/119812430 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fabe40f8f90 con 0x7fabe40f55e0 2021-11-03T17:25:20.506 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.511+0000 7fabcbfff700 1 -- 172.21.15.27:0/119812430 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7fabe40f8f90 con 0x7fabe40f55e0 2021-11-03T17:25:20.506 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:25:20.506 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:25:20.507 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:25:20.507 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:25:20.507 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:25:20.507 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:25:20.508 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:25:20.508 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:25:20.508 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:25:20.508 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:25:20.508 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:25:20.509 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:25:20.509 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:25:20.509 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:25:20.509 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:25:20.510 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:25:20.510 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:25:20.510 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:25:20.510 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:25:20.511 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:25:20.511 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:25:20.511 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:25:20.511 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:25:20.512 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:25:20.512 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:25:20.512 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:25:20.512 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:25:20.513 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:25:20.513 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:25:20.513 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:25:20.513 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:25:20.514 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:25:20.514 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:25:20.514 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:25:20.514 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:25:20.514 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:25:20.515 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:25:20.515 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:25:20.515 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:25:20.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.513+0000 7fabe9d39700 1 -- 172.21.15.27:0/119812430 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fabcc0640b0 msgr2=0x7fabcc066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:25:20.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.513+0000 7fabe9d39700 1 --2- 172.21.15.27:0/119812430 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fabcc0640b0 0x7fabcc066570 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fabd8004630 tx=0x7fabd8009b30).stop 2021-11-03T17:25:20.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.513+0000 7fabe9d39700 1 -- 172.21.15.27:0/119812430 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabe40f55e0 msgr2=0x7fabe40fa000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:25:20.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.513+0000 7fabe9d39700 1 --2- 172.21.15.27:0/119812430 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabe40f55e0 0x7fabe40fa000 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7fabd400c790 tx=0x7fabd4007d70).stop 2021-11-03T17:25:20.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.513+0000 7fabe9d39700 1 -- 172.21.15.27:0/119812430 shutdown_connections 2021-11-03T17:25:20.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.513+0000 7fabe9d39700 1 --2- 172.21.15.27:0/119812430 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fabcc0640b0 0x7fabcc066570 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:20.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.513+0000 7fabe9d39700 1 --2- 172.21.15.27:0/119812430 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fabe40f55e0 0x7fabe40fa000 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:20.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.513+0000 7fabe9d39700 1 --2- 172.21.15.27:0/119812430 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fabe40f5ef0 0x7fabe40f8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:20.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.513+0000 7fabe9d39700 1 -- 172.21.15.27:0/119812430 >> 172.21.15.27:0/119812430 conn(0x7fabe40f0de0 msgr2=0x7fabe405f720 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:25:20.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.513+0000 7fabe9d39700 1 -- 172.21.15.27:0/119812430 shutdown_connections 2021-11-03T17:25:20.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:20.513+0000 7fabe9d39700 1 -- 172.21.15.27:0/119812430 wait complete. 2021-11-03T17:25:20.519 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:25:21.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:21 smithi027 ceph-mon[97244]: from='client.34440 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:25:21.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:21 smithi027 ceph-mon[97244]: from='client.34444 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:25:21.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:21 smithi027 ceph-mon[97244]: pgmap v321: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:21.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:21 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/4015006834' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:21.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:21 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/119812430' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:25:21.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:21 smithi115 ceph-mon[65453]: from='client.34440 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:25:21.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:21 smithi115 ceph-mon[65453]: from='client.34444 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:25:21.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:21 smithi115 ceph-mon[65453]: pgmap v321: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:21.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:21 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/4015006834' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:21.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:21 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/119812430' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:25:23.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:23 smithi027 ceph-mon[97244]: pgmap v322: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:23.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:23 smithi115 ceph-mon[65453]: pgmap v322: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:25.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:25 smithi027 ceph-mon[97244]: pgmap v323: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:25.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 567.372483 secs 2021-11-03T17:25:25.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:25 smithi115 ceph-mon[65453]: pgmap v323: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:25.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 567.372483 secs 2021-11-03T17:25:27.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:27 smithi027 ceph-mon[97244]: pgmap v324: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:27.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:27 smithi115 ceph-mon[65453]: pgmap v324: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:28.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:25:28.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:25:28.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:25:28.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:28.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:28.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:28.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:28.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:25:28.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:25:28.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:25:28.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:28.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:28.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:28.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:29.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:29 smithi115 ceph-mon[65453]: pgmap v325: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:29.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:29 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:25:29.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:29 smithi027 ceph-mon[97244]: pgmap v325: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:29.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:29 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:25:30.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 572.372596 secs 2021-11-03T17:25:30.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:30 smithi115 ceph-mon[65453]: pgmap v326: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:30.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 572.372596 secs 2021-11-03T17:25:30.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:30 smithi027 ceph-mon[97244]: pgmap v326: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:33.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:32 smithi115 ceph-mon[65453]: pgmap v327: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:33.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:32 smithi027 ceph-mon[97244]: pgmap v327: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:35.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:34 smithi115 ceph-mon[65453]: pgmap v328: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:35.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:34 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 577.372680 secs 2021-11-03T17:25:35.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:34 smithi027 ceph-mon[97244]: pgmap v328: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:35.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:34 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 577.372680 secs 2021-11-03T17:25:37.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:36 smithi115 ceph-mon[65453]: pgmap v329: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:37.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:36 smithi027 ceph-mon[97244]: pgmap v329: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:38.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:25:38.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:25:38.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:25:38.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:25:39.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:38 smithi115 ceph-mon[65453]: pgmap v330: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:39.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:25:39.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:39.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:39.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:39.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:39.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:38 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:25:39.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:38 smithi027 ceph-mon[97244]: pgmap v330: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:39.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:25:39.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:39.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:39.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:39.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:39.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:38 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:25:40.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:39 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 582.372790 secs 2021-11-03T17:25:40.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:39 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 582.372790 secs 2021-11-03T17:25:41.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:40 smithi115 ceph-mon[65453]: pgmap v331: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:41.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:40 smithi027 ceph-mon[97244]: pgmap v331: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:43.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:42 smithi115 ceph-mon[65453]: pgmap v332: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:43.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:42 smithi027 ceph-mon[97244]: pgmap v332: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:45.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:44 smithi115 ceph-mon[65453]: pgmap v333: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:45.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:44 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 587.372853 secs 2021-11-03T17:25:45.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:44 smithi027 ceph-mon[97244]: pgmap v333: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:45.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:44 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 587.372853 secs 2021-11-03T17:25:47.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:46 smithi115 ceph-mon[65453]: pgmap v334: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:47.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:46 smithi027 ceph-mon[97244]: pgmap v334: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:49.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:48 smithi115 ceph-mon[65453]: pgmap v335: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:49.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:25:49.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:48 smithi115 ceph-mon[65453]: poll took 60179.275 ms: 1 events 2021-11-03T17:25:49.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:25:49.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:25:49.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:49.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:49.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:49.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:49.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:48 smithi027 ceph-mon[97244]: pgmap v335: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:49.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:25:49.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:48 smithi027 ceph-mon[97244]: poll took 60179.275 ms: 1 events 2021-11-03T17:25:49.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:25:49.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:25:49.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:49.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:49.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:49.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:50.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:49 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:25:50.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:49 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 592.372990 secs 2021-11-03T17:25:50.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:49 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:25:50.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:49 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 592.372990 secs 2021-11-03T17:25:50.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.628+0000 7f9a93640700 1 -- 172.21.15.27:0/2410432424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a8c0f6ff0 msgr2=0x7f9a8c0f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:25:50.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.628+0000 7f9a93640700 1 --2- 172.21.15.27:0/2410432424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a8c0f6ff0 0x7f9a8c0f7a20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f9a80004660 tx=0x7f9a80009b30).stop 2021-11-03T17:25:50.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.628+0000 7f9a93640700 1 -- 172.21.15.27:0/2410432424 shutdown_connections 2021-11-03T17:25:50.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.628+0000 7f9a93640700 1 --2- 172.21.15.27:0/2410432424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a8c0f6ff0 0x7f9a8c0f7a20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:50.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.628+0000 7f9a93640700 1 --2- 172.21.15.27:0/2410432424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a8c0f66e0 0x7f9a8c0f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:50.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.628+0000 7f9a93640700 1 -- 172.21.15.27:0/2410432424 >> 172.21.15.27:0/2410432424 conn(0x7f9a8c0f1f00 msgr2=0x7f9a8c0f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:25:50.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.629+0000 7f9a93640700 1 -- 172.21.15.27:0/2410432424 shutdown_connections 2021-11-03T17:25:50.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.629+0000 7f9a93640700 1 -- 172.21.15.27:0/2410432424 wait complete. 2021-11-03T17:25:50.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.629+0000 7f9a93640700 1 Processor -- start 2021-11-03T17:25:50.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.629+0000 7f9a93640700 1 -- start start 2021-11-03T17:25:50.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.629+0000 7f9a93640700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a8c0f66e0 0x7f9a8c06b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:25:50.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.629+0000 7f9a93640700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a8c0f6ff0 0x7f9a8c069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:25:50.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.629+0000 7f9a93640700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c06a380 con 0x7f9a8c0f66e0 2021-11-03T17:25:50.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.629+0000 7f9a93640700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8c06a4c0 con 0x7f9a8c0f6ff0 2021-11-03T17:25:50.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.629+0000 7f9a913dc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a8c0f66e0 0x7f9a8c06b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:25:50.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.629+0000 7f9a913dc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a8c0f66e0 0x7f9a8c06b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57376/0 (socket says 172.21.15.27:57376) 2021-11-03T17:25:50.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.629+0000 7f9a913dc700 1 -- 172.21.15.27:0/1787228264 learned_addr learned my addr 172.21.15.27:0/1787228264 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:25:50.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.630+0000 7f9a90bdb700 1 --2- 172.21.15.27:0/1787228264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a8c0f6ff0 0x7f9a8c069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:25:50.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.630+0000 7f9a913dc700 1 -- 172.21.15.27:0/1787228264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a8c0f6ff0 msgr2=0x7f9a8c069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:25:50.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.630+0000 7f9a913dc700 1 --2- 172.21.15.27:0/1787228264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a8c0f6ff0 0x7f9a8c069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:50.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.630+0000 7f9a913dc700 1 -- 172.21.15.27:0/1787228264 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a80005040 con 0x7f9a8c0f66e0 2021-11-03T17:25:50.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.630+0000 7f9a913dc700 1 --2- 172.21.15.27:0/1787228264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a8c0f66e0 0x7f9a8c06b9a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f9a880026d0 tx=0x7f9a88009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:25:50.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.630+0000 7f9a7dffb700 1 -- 172.21.15.27:0/1787228264 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a88009320 con 0x7f9a8c0f66e0 2021-11-03T17:25:50.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.630+0000 7f9a7dffb700 1 -- 172.21.15.27:0/1787228264 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f9a8800f040 con 0x7f9a8c0f66e0 2021-11-03T17:25:50.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.630+0000 7f9a93640700 1 -- 172.21.15.27:0/1787228264 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a8c06a7a0 con 0x7f9a8c0f66e0 2021-11-03T17:25:50.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.630+0000 7f9a7dffb700 1 -- 172.21.15.27:0/1787228264 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a880147e0 con 0x7f9a8c0f66e0 2021-11-03T17:25:50.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.630+0000 7f9a93640700 1 -- 172.21.15.27:0/1787228264 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a8c10efa0 con 0x7f9a8c0f66e0 2021-11-03T17:25:50.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.631+0000 7f9a93640700 1 -- 172.21.15.27:0/1787228264 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a8c045ad0 con 0x7f9a8c0f66e0 2021-11-03T17:25:50.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.632+0000 7f9a7dffb700 1 -- 172.21.15.27:0/1787228264 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f9a880077b0 con 0x7f9a8c0f66e0 2021-11-03T17:25:50.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.632+0000 7f9a7dffb700 1 --2- 172.21.15.27:0/1787228264 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9a78064090 0x7f9a78066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:25:50.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.632+0000 7f9a7dffb700 1 -- 172.21.15.27:0/1787228264 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f9a88080e50 con 0x7f9a8c0f66e0 2021-11-03T17:25:50.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.632+0000 7f9a90bdb700 1 --2- 172.21.15.27:0/1787228264 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9a78064090 0x7f9a78066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:25:50.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.633+0000 7f9a90bdb700 1 --2- 172.21.15.27:0/1787228264 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9a78064090 0x7f9a78066550 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f9a80004e30 tx=0x7f9a80009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:25:50.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.636+0000 7f9a7dffb700 1 -- 172.21.15.27:0/1787228264 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f9a88053150 con 0x7f9a8c0f66e0 2021-11-03T17:25:50.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.816+0000 7f9a93640700 1 -- 172.21.15.27:0/1787228264 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9a8c0fefe0 con 0x7f9a78064090 2021-11-03T17:25:50.811 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.817+0000 7f9a7dffb700 1 -- 172.21.15.27:0/1787228264 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f9a8c0fefe0 con 0x7f9a78064090 2021-11-03T17:25:50.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.819+0000 7f9a93640700 1 -- 172.21.15.27:0/1787228264 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9a78064090 msgr2=0x7f9a78066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:25:50.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.819+0000 7f9a93640700 1 --2- 172.21.15.27:0/1787228264 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9a78064090 0x7f9a78066550 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f9a80004e30 tx=0x7f9a80009b30).stop 2021-11-03T17:25:50.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.819+0000 7f9a93640700 1 -- 172.21.15.27:0/1787228264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a8c0f66e0 msgr2=0x7f9a8c06b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:25:50.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.819+0000 7f9a93640700 1 --2- 172.21.15.27:0/1787228264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a8c0f66e0 0x7f9a8c06b9a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f9a880026d0 tx=0x7f9a88009de0).stop 2021-11-03T17:25:50.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.819+0000 7f9a93640700 1 -- 172.21.15.27:0/1787228264 shutdown_connections 2021-11-03T17:25:50.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.819+0000 7f9a93640700 1 --2- 172.21.15.27:0/1787228264 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9a78064090 0x7f9a78066550 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:50.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.819+0000 7f9a93640700 1 --2- 172.21.15.27:0/1787228264 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a8c0f66e0 0x7f9a8c06b9a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:50.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.819+0000 7f9a93640700 1 --2- 172.21.15.27:0/1787228264 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a8c0f6ff0 0x7f9a8c069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:50.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.819+0000 7f9a93640700 1 -- 172.21.15.27:0/1787228264 >> 172.21.15.27:0/1787228264 conn(0x7f9a8c0f1f00 msgr2=0x7f9a8c0fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:25:50.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.820+0000 7f9a93640700 1 -- 172.21.15.27:0/1787228264 shutdown_connections 2021-11-03T17:25:50.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.820+0000 7f9a93640700 1 -- 172.21.15.27:0/1787228264 wait complete. 2021-11-03T17:25:50.825 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:25:50.915 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.920+0000 7f8599ca4700 1 -- 172.21.15.27:0/3607835038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85940f6ff0 msgr2=0x7f85940f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:25:50.916 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.920+0000 7f8599ca4700 1 --2- 172.21.15.27:0/3607835038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85940f6ff0 0x7f85940f7a20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f8588004660 tx=0x7f8588009b30).stop 2021-11-03T17:25:50.916 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.920+0000 7f8599ca4700 1 -- 172.21.15.27:0/3607835038 shutdown_connections 2021-11-03T17:25:50.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.920+0000 7f8599ca4700 1 --2- 172.21.15.27:0/3607835038 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85940f6ff0 0x7f85940f7a20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:50.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.920+0000 7f8599ca4700 1 --2- 172.21.15.27:0/3607835038 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85940f66e0 0x7f85940f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:50.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.920+0000 7f8599ca4700 1 -- 172.21.15.27:0/3607835038 >> 172.21.15.27:0/3607835038 conn(0x7f85940f1f00 msgr2=0x7f85940f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:25:50.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.920+0000 7f8599ca4700 1 -- 172.21.15.27:0/3607835038 shutdown_connections 2021-11-03T17:25:50.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.920+0000 7f8599ca4700 1 -- 172.21.15.27:0/3607835038 wait complete. 2021-11-03T17:25:50.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.920+0000 7f8599ca4700 1 Processor -- start 2021-11-03T17:25:50.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.920+0000 7f8599ca4700 1 -- start start 2021-11-03T17:25:50.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.921+0000 7f8599ca4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85940f66e0 0x7f8594106ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:25:50.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.921+0000 7f8599ca4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85940f6ff0 0x7f85941073e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:25:50.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.921+0000 7f8599ca4700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8594107990 con 0x7f85940f66e0 2021-11-03T17:25:50.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.921+0000 7f8599ca4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8594107ad0 con 0x7f85940f6ff0 2021-11-03T17:25:50.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.921+0000 7f85937fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85940f66e0 0x7f8594106ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:25:50.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.921+0000 7f85937fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85940f66e0 0x7f8594106ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57386/0 (socket says 172.21.15.27:57386) 2021-11-03T17:25:50.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.921+0000 7f85937fe700 1 -- 172.21.15.27:0/482829049 learned_addr learned my addr 172.21.15.27:0/482829049 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:25:50.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.921+0000 7f8592ffd700 1 --2- 172.21.15.27:0/482829049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85940f6ff0 0x7f85941073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:25:50.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.921+0000 7f85937fe700 1 -- 172.21.15.27:0/482829049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85940f6ff0 msgr2=0x7f85941073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:25:50.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.921+0000 7f85937fe700 1 --2- 172.21.15.27:0/482829049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85940f6ff0 0x7f85941073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:50.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.921+0000 7f85937fe700 1 -- 172.21.15.27:0/482829049 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8588005040 con 0x7f85940f66e0 2021-11-03T17:25:50.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.921+0000 7f85937fe700 1 --2- 172.21.15.27:0/482829049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85940f66e0 0x7f8594106ef0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f8584002700 tx=0x7f8584009e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:25:50.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.921+0000 7f857bfff700 1 -- 172.21.15.27:0/482829049 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8584009320 con 0x7f85940f66e0 2021-11-03T17:25:50.922 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.921+0000 7f8599ca4700 1 -- 172.21.15.27:0/482829049 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8594108470 con 0x7f85940f66e0 2021-11-03T17:25:50.922 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.921+0000 7f857bfff700 1 -- 172.21.15.27:0/482829049 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f858400f040 con 0x7f85940f66e0 2021-11-03T17:25:50.922 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.921+0000 7f857bfff700 1 -- 172.21.15.27:0/482829049 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8584014820 con 0x7f85940f66e0 2021-11-03T17:25:50.922 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.921+0000 7f8599ca4700 1 -- 172.21.15.27:0/482829049 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8594108a30 con 0x7f85940f66e0 2021-11-03T17:25:50.923 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.922+0000 7f8599ca4700 1 -- 172.21.15.27:0/482829049 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8594045ad0 con 0x7f85940f66e0 2021-11-03T17:25:50.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.923+0000 7f857bfff700 1 -- 172.21.15.27:0/482829049 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f8584009480 con 0x7f85940f66e0 2021-11-03T17:25:50.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.923+0000 7f857bfff700 1 --2- 172.21.15.27:0/482829049 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f857c0640b0 0x7f857c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:25:50.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.923+0000 7f857bfff700 1 -- 172.21.15.27:0/482829049 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f8584080b70 con 0x7f85940f66e0 2021-11-03T17:25:50.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.923+0000 7f8592ffd700 1 --2- 172.21.15.27:0/482829049 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f857c0640b0 0x7f857c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:25:50.925 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.924+0000 7f8592ffd700 1 --2- 172.21.15.27:0/482829049 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f857c0640b0 0x7f857c066570 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f8588004630 tx=0x7f8588009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:25:50.925 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:50.927+0000 7f857bfff700 1 -- 172.21.15.27:0/482829049 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f8584052e70 con 0x7f85940f66e0 2021-11-03T17:25:51.095 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.101+0000 7f8599ca4700 1 -- 172.21.15.27:0/482829049 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f85940ff050 con 0x7f857c0640b0 2021-11-03T17:25:51.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.116+0000 7f857bfff700 1 -- 172.21.15.27:0/482829049 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f85940ff050 con 0x7f857c0640b0 2021-11-03T17:25:51.111 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:25:51.111 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (17m) 7m ago 18m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:25:51.112 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (9m) 7m ago 18m 7205k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:25:51.112 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (8m) 6m ago 17m 7239k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:25:51.112 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (17m) 7m ago 18m 38.9M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:25:51.113 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (10m) 7m ago 13m 15.4M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:25:51.113 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (13m) 7m ago 13m 3836M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:25:51.114 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (9m) 6m ago 13m 14.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:25:51.114 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (11m) 6m ago 13m 16.3M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:25:51.115 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (11m) 7m ago 19m 470M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:25:51.115 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (10m) 6m ago 17m 412M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:25:51.115 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (10m) 7m ago 20m 81.2M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:25:51.116 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (9m) 6m ago 17m 69.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:25:51.116 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (18m) 7m ago 18m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:25:51.117 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (17m) 6m ago 17m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:25:51.117 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (8m) 7m ago 17m 243M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:25:51.117 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (8m) 7m ago 16m 270M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:25:51.118 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (7m) 7m ago 16m 11.8M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:25:51.118 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (7m) 6m ago 16m 289M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:25:51.119 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (6m) 6m ago 15m 235M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:25:51.119 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (6m) 6m ago 15m 11.5M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:25:51.119 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (17m) 7m ago 18m 45.7M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:25:51.121 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.118+0000 7f8599ca4700 1 -- 172.21.15.27:0/482829049 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f857c0640b0 msgr2=0x7f857c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:25:51.121 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.118+0000 7f8599ca4700 1 --2- 172.21.15.27:0/482829049 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f857c0640b0 0x7f857c066570 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f8588004630 tx=0x7f8588009b30).stop 2021-11-03T17:25:51.121 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.118+0000 7f8599ca4700 1 -- 172.21.15.27:0/482829049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85940f66e0 msgr2=0x7f8594106ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:25:51.121 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.118+0000 7f8599ca4700 1 --2- 172.21.15.27:0/482829049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85940f66e0 0x7f8594106ef0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f8584002700 tx=0x7f8584009e00).stop 2021-11-03T17:25:51.122 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.118+0000 7f8599ca4700 1 -- 172.21.15.27:0/482829049 shutdown_connections 2021-11-03T17:25:51.122 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.118+0000 7f8599ca4700 1 --2- 172.21.15.27:0/482829049 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f857c0640b0 0x7f857c066570 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:51.122 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.118+0000 7f8599ca4700 1 --2- 172.21.15.27:0/482829049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85940f66e0 0x7f8594106ef0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:51.122 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.118+0000 7f8599ca4700 1 --2- 172.21.15.27:0/482829049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85940f6ff0 0x7f85941073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:51.123 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.118+0000 7f8599ca4700 1 -- 172.21.15.27:0/482829049 >> 172.21.15.27:0/482829049 conn(0x7f85940f1f00 msgr2=0x7f85940fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:25:51.123 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.118+0000 7f8599ca4700 1 -- 172.21.15.27:0/482829049 shutdown_connections 2021-11-03T17:25:51.123 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.118+0000 7f8599ca4700 1 -- 172.21.15.27:0/482829049 wait complete. 2021-11-03T17:25:51.214 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.217+0000 7fe63475d700 1 -- 172.21.15.27:0/146134718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe62c0f66e0 msgr2=0x7fe62c0f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:25:51.214 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.217+0000 7fe63475d700 1 --2- 172.21.15.27:0/146134718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe62c0f66e0 0x7fe62c0f6b00 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7fe61c004660 tx=0x7fe61c009b30).stop 2021-11-03T17:25:51.214 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.217+0000 7fe63475d700 1 -- 172.21.15.27:0/146134718 shutdown_connections 2021-11-03T17:25:51.214 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.217+0000 7fe63475d700 1 --2- 172.21.15.27:0/146134718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe62c0f6ff0 0x7fe62c0f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:51.215 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.217+0000 7fe63475d700 1 --2- 172.21.15.27:0/146134718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe62c0f66e0 0x7fe62c0f6b00 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:51.215 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.217+0000 7fe63475d700 1 -- 172.21.15.27:0/146134718 >> 172.21.15.27:0/146134718 conn(0x7fe62c0f1f00 msgr2=0x7fe62c0f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:25:51.215 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.217+0000 7fe63475d700 1 -- 172.21.15.27:0/146134718 shutdown_connections 2021-11-03T17:25:51.215 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.217+0000 7fe63475d700 1 -- 172.21.15.27:0/146134718 wait complete. 2021-11-03T17:25:51.216 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.218+0000 7fe63475d700 1 Processor -- start 2021-11-03T17:25:51.216 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.218+0000 7fe63475d700 1 -- start start 2021-11-03T17:25:51.216 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.218+0000 7fe63475d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe62c0f66e0 0x7fe62c06b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:25:51.216 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.218+0000 7fe63475d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe62c0f6ff0 0x7fe62c069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:25:51.216 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.218+0000 7fe63475d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe62c06a380 con 0x7fe62c0f66e0 2021-11-03T17:25:51.217 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.218+0000 7fe63475d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe62c06a4c0 con 0x7fe62c0f6ff0 2021-11-03T17:25:51.217 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.218+0000 7fe6324f9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe62c0f66e0 0x7fe62c06b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:25:51.217 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.218+0000 7fe6324f9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe62c0f66e0 0x7fe62c06b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57396/0 (socket says 172.21.15.27:57396) 2021-11-03T17:25:51.217 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.218+0000 7fe6324f9700 1 -- 172.21.15.27:0/1981408013 learned_addr learned my addr 172.21.15.27:0/1981408013 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:25:51.218 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.218+0000 7fe631cf8700 1 --2- 172.21.15.27:0/1981408013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe62c0f6ff0 0x7fe62c069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:25:51.218 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.218+0000 7fe6324f9700 1 -- 172.21.15.27:0/1981408013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe62c0f6ff0 msgr2=0x7fe62c069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:25:51.218 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.218+0000 7fe6324f9700 1 --2- 172.21.15.27:0/1981408013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe62c0f6ff0 0x7fe62c069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:51.218 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.218+0000 7fe6324f9700 1 -- 172.21.15.27:0/1981408013 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe61c005040 con 0x7fe62c0f66e0 2021-11-03T17:25:51.219 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.218+0000 7fe631cf8700 1 --2- 172.21.15.27:0/1981408013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe62c0f6ff0 0x7fe62c069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:25:51.219 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.218+0000 7fe6324f9700 1 --2- 172.21.15.27:0/1981408013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe62c0f66e0 0x7fe62c06b9a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7fe61c009990 tx=0x7fe61c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:25:51.219 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.219+0000 7fe622ffd700 1 -- 172.21.15.27:0/1981408013 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe61c018070 con 0x7fe62c0f66e0 2021-11-03T17:25:51.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.219+0000 7fe622ffd700 1 -- 172.21.15.27:0/1981408013 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fe61c007ca0 con 0x7fe62c0f66e0 2021-11-03T17:25:51.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.219+0000 7fe63475d700 1 -- 172.21.15.27:0/1981408013 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe62c06a740 con 0x7fe62c0f66e0 2021-11-03T17:25:51.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.219+0000 7fe622ffd700 1 -- 172.21.15.27:0/1981408013 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe61c01c620 con 0x7fe62c0f66e0 2021-11-03T17:25:51.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.219+0000 7fe63475d700 1 -- 172.21.15.27:0/1981408013 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe62c06aca0 con 0x7fe62c0f66e0 2021-11-03T17:25:51.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.220+0000 7fe63475d700 1 -- 172.21.15.27:0/1981408013 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe62c045ad0 con 0x7fe62c0f66e0 2021-11-03T17:25:51.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.220+0000 7fe622ffd700 1 -- 172.21.15.27:0/1981408013 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fe61c01d960 con 0x7fe62c0f66e0 2021-11-03T17:25:51.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.220+0000 7fe622ffd700 1 --2- 172.21.15.27:0/1981408013 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe6180640b0 0x7fe618066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:25:51.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.221+0000 7fe622ffd700 1 -- 172.21.15.27:0/1981408013 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fe61c082580 con 0x7fe62c0f66e0 2021-11-03T17:25:51.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.221+0000 7fe631cf8700 1 --2- 172.21.15.27:0/1981408013 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe6180640b0 0x7fe618066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:25:51.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.221+0000 7fe631cf8700 1 --2- 172.21.15.27:0/1981408013 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe6180640b0 0x7fe618066570 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fe628004a40 tx=0x7fe628009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:25:51.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.225+0000 7fe622ffd700 1 -- 172.21.15.27:0/1981408013 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fe61c054940 con 0x7fe62c0f66e0 2021-11-03T17:25:51.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:50 smithi027 ceph-mon[97244]: pgmap v336: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:51.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.456+0000 7fe63475d700 1 -- 172.21.15.27:0/1981408013 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe62c0ff030 con 0x7fe62c0f66e0 2021-11-03T17:25:51.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.456+0000 7fe622ffd700 1 -- 172.21.15.27:0/1981408013 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fe61c054530 con 0x7fe62c0f66e0 2021-11-03T17:25:51.451 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:25:51.452 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:25:51.452 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:25:51.452 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:25:51.452 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:25:51.452 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:25:51.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:25:51.453 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:25:51.453 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:25:51.453 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:25:51.454 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:25:51.454 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:25:51.454 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:25:51.454 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:25:51.455 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:25:51.455 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:25:51.455 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:25:51.455 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:25:51.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.458+0000 7fe63475d700 1 -- 172.21.15.27:0/1981408013 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe6180640b0 msgr2=0x7fe618066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:25:51.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.458+0000 7fe63475d700 1 --2- 172.21.15.27:0/1981408013 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe6180640b0 0x7fe618066570 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fe628004a40 tx=0x7fe628009040).stop 2021-11-03T17:25:51.457 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.458+0000 7fe63475d700 1 -- 172.21.15.27:0/1981408013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe62c0f66e0 msgr2=0x7fe62c06b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:25:51.457 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.458+0000 7fe63475d700 1 --2- 172.21.15.27:0/1981408013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe62c0f66e0 0x7fe62c06b9a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7fe61c009990 tx=0x7fe61c009b30).stop 2021-11-03T17:25:51.457 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.458+0000 7fe63475d700 1 -- 172.21.15.27:0/1981408013 shutdown_connections 2021-11-03T17:25:51.457 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.458+0000 7fe63475d700 1 --2- 172.21.15.27:0/1981408013 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe6180640b0 0x7fe618066570 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:51.457 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.458+0000 7fe63475d700 1 --2- 172.21.15.27:0/1981408013 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe62c0f66e0 0x7fe62c06b9a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:51.458 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.458+0000 7fe63475d700 1 --2- 172.21.15.27:0/1981408013 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe62c0f6ff0 0x7fe62c069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:51.458 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.458+0000 7fe63475d700 1 -- 172.21.15.27:0/1981408013 >> 172.21.15.27:0/1981408013 conn(0x7fe62c0f1f00 msgr2=0x7fe62c0fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:25:51.458 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.458+0000 7fe63475d700 1 -- 172.21.15.27:0/1981408013 shutdown_connections 2021-11-03T17:25:51.458 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.458+0000 7fe63475d700 1 -- 172.21.15.27:0/1981408013 wait complete. 2021-11-03T17:25:51.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:50 smithi115 ceph-mon[65453]: pgmap v336: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:51.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.559+0000 7f2d850e0700 1 -- 172.21.15.27:0/3226146429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d800f9830 msgr2=0x7f2d800fbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:25:51.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.559+0000 7f2d850e0700 1 --2- 172.21.15.27:0/3226146429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d800f9830 0x7f2d800fbc20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f2d74004660 tx=0x7f2d74009b30).stop 2021-11-03T17:25:51.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.559+0000 7f2d850e0700 1 -- 172.21.15.27:0/3226146429 shutdown_connections 2021-11-03T17:25:51.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.559+0000 7f2d850e0700 1 --2- 172.21.15.27:0/3226146429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d800f9830 0x7f2d800fbc20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:51.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.559+0000 7f2d850e0700 1 --2- 172.21.15.27:0/3226146429 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d80060350 0x7f2d800f9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:51.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.559+0000 7f2d850e0700 1 -- 172.21.15.27:0/3226146429 >> 172.21.15.27:0/3226146429 conn(0x7f2d800f0de0 msgr2=0x7f2d800f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:25:51.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.559+0000 7f2d850e0700 1 -- 172.21.15.27:0/3226146429 shutdown_connections 2021-11-03T17:25:51.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.559+0000 7f2d850e0700 1 -- 172.21.15.27:0/3226146429 wait complete. 2021-11-03T17:25:51.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.559+0000 7f2d850e0700 1 Processor -- start 2021-11-03T17:25:51.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.559+0000 7f2d850e0700 1 -- start start 2021-11-03T17:25:51.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.560+0000 7f2d850e0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d80060350 0x7f2d800f6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:25:51.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.560+0000 7f2d850e0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d800f9830 0x7f2d800f5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:25:51.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.560+0000 7f2d850e0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d800f57a0 con 0x7f2d80060350 2021-11-03T17:25:51.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.560+0000 7f2d850e0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d800f58e0 con 0x7f2d800f9830 2021-11-03T17:25:51.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.560+0000 7f2d7ed9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d80060350 0x7f2d800f6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:25:51.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.560+0000 7f2d7ed9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d80060350 0x7f2d800f6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57406/0 (socket says 172.21.15.27:57406) 2021-11-03T17:25:51.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.560+0000 7f2d7ed9d700 1 -- 172.21.15.27:0/1334107782 learned_addr learned my addr 172.21.15.27:0/1334107782 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:25:51.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.560+0000 7f2d7e59c700 1 --2- 172.21.15.27:0/1334107782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d800f9830 0x7f2d800f5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:25:51.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.560+0000 7f2d7ed9d700 1 -- 172.21.15.27:0/1334107782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d800f9830 msgr2=0x7f2d800f5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:25:51.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.560+0000 7f2d7ed9d700 1 --2- 172.21.15.27:0/1334107782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d800f9830 0x7f2d800f5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:51.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.560+0000 7f2d7ed9d700 1 -- 172.21.15.27:0/1334107782 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d74005040 con 0x7f2d80060350 2021-11-03T17:25:51.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.560+0000 7f2d7ed9d700 1 --2- 172.21.15.27:0/1334107782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d80060350 0x7f2d800f6dc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f2d70002700 tx=0x7f2d70009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:25:51.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.560+0000 7f2d677fe700 1 -- 172.21.15.27:0/1334107782 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d70009320 con 0x7f2d80060350 2021-11-03T17:25:51.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.560+0000 7f2d677fe700 1 -- 172.21.15.27:0/1334107782 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f2d7000f040 con 0x7f2d80060350 2021-11-03T17:25:51.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.560+0000 7f2d850e0700 1 -- 172.21.15.27:0/1334107782 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d800f5bc0 con 0x7f2d80060350 2021-11-03T17:25:51.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.560+0000 7f2d677fe700 1 -- 172.21.15.27:0/1334107782 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2d70014900 con 0x7f2d80060350 2021-11-03T17:25:51.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.560+0000 7f2d850e0700 1 -- 172.21.15.27:0/1334107782 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d8010ef70 con 0x7f2d80060350 2021-11-03T17:25:51.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.561+0000 7f2d850e0700 1 -- 172.21.15.27:0/1334107782 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d80045ad0 con 0x7f2d80060350 2021-11-03T17:25:51.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.562+0000 7f2d677fe700 1 -- 172.21.15.27:0/1334107782 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f2d70009480 con 0x7f2d80060350 2021-11-03T17:25:51.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.562+0000 7f2d677fe700 1 --2- 172.21.15.27:0/1334107782 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2d680640b0 0x7f2d68066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:25:51.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.562+0000 7f2d677fe700 1 -- 172.21.15.27:0/1334107782 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f2d70080ae0 con 0x7f2d80060350 2021-11-03T17:25:51.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.562+0000 7f2d7e59c700 1 --2- 172.21.15.27:0/1334107782 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2d680640b0 0x7f2d68066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:25:51.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.563+0000 7f2d7e59c700 1 --2- 172.21.15.27:0/1334107782 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2d680640b0 0x7f2d68066570 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f2d74004630 tx=0x7f2d74009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:25:51.568 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.566+0000 7f2d677fe700 1 -- 172.21.15.27:0/1334107782 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f2d70052de0 con 0x7f2d80060350 2021-11-03T17:25:51.763 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.768+0000 7f2d850e0700 1 -- 172.21.15.27:0/1334107782 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2d800fef60 con 0x7f2d80060350 2021-11-03T17:25:51.763 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.768+0000 7f2d677fe700 1 -- 172.21.15.27:0/1334107782 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f2d700529d0 con 0x7f2d80060350 2021-11-03T17:25:51.764 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:25:51.764 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:25:51.764 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:25:51.764 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:25:51.765 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:25:51.765 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:25:51.765 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:25:51.765 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:25:51.766 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:25:51.766 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:25:51.766 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:25:51.766 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:25:51.767 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:25:51.767 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:25:51.767 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:25:51.767 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:25:51.767 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:25:51.768 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:25:51.768 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:25:51.768 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:25:51.768 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:25:51.769 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:25:51.769 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:25:51.769 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:25:51.769 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:25:51.769 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:25:51.770 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:25:51.770 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:25:51.770 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:25:51.770 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:25:51.771 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:25:51.771 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:25:51.771 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:25:51.771 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:25:51.772 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:25:51.772 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:25:51.772 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:25:51.772 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:25:51.773 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:25:51.774 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.770+0000 7f2d850e0700 1 -- 172.21.15.27:0/1334107782 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2d680640b0 msgr2=0x7f2d68066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:25:51.774 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.770+0000 7f2d850e0700 1 --2- 172.21.15.27:0/1334107782 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2d680640b0 0x7f2d68066570 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f2d74004630 tx=0x7f2d74009b30).stop 2021-11-03T17:25:51.774 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.770+0000 7f2d850e0700 1 -- 172.21.15.27:0/1334107782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d80060350 msgr2=0x7f2d800f6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:25:51.774 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.770+0000 7f2d850e0700 1 --2- 172.21.15.27:0/1334107782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d80060350 0x7f2d800f6dc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f2d70002700 tx=0x7f2d70009d80).stop 2021-11-03T17:25:51.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.770+0000 7f2d850e0700 1 -- 172.21.15.27:0/1334107782 shutdown_connections 2021-11-03T17:25:51.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.770+0000 7f2d850e0700 1 --2- 172.21.15.27:0/1334107782 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2d680640b0 0x7f2d68066570 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:51.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.770+0000 7f2d850e0700 1 --2- 172.21.15.27:0/1334107782 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2d80060350 0x7f2d800f6dc0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:51.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.770+0000 7f2d850e0700 1 --2- 172.21.15.27:0/1334107782 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2d800f9830 0x7f2d800f5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:25:51.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.770+0000 7f2d850e0700 1 -- 172.21.15.27:0/1334107782 >> 172.21.15.27:0/1334107782 conn(0x7f2d800f0de0 msgr2=0x7f2d800f31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:25:51.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.771+0000 7f2d850e0700 1 -- 172.21.15.27:0/1334107782 shutdown_connections 2021-11-03T17:25:51.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:25:51.771+0000 7f2d850e0700 1 -- 172.21.15.27:0/1334107782 wait complete. 2021-11-03T17:25:51.776 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:25:52.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:52 smithi027 ceph-mon[97244]: from='client.34456 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:25:52.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:52 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1981408013' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:52.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:52 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1334107782' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:25:52.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:52 smithi115 ceph-mon[65453]: from='client.34456 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:25:52.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:52 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1981408013' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:52.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:52 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1334107782' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:25:53.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:53 smithi027 ceph-mon[97244]: from='client.34460 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:25:53.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:53 smithi027 ceph-mon[97244]: pgmap v337: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:53.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:53 smithi115 ceph-mon[65453]: from='client.34460 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:25:53.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:53 smithi115 ceph-mon[65453]: pgmap v337: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:55.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:55 smithi027 ceph-mon[97244]: pgmap v338: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:55.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 597.373135 secs 2021-11-03T17:25:55.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:55 smithi115 ceph-mon[65453]: pgmap v338: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:55.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 597.373135 secs 2021-11-03T17:25:56.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:25:56.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:25:56.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:25:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:25:57.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:57 smithi027 ceph-mon[97244]: pgmap v339: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:57.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:57 smithi115 ceph-mon[65453]: pgmap v339: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:59.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:59 smithi027 ceph-mon[97244]: pgmap v340: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:59.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:25:59.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:25:59.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:25:59.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:59.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:59.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:59.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:25:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:59.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:59 smithi115 ceph-mon[65453]: pgmap v340: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:25:59.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:25:59.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:25:59.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:25:59.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:59.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:59.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:25:59.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:25:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:00.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:00 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:26:00.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 602.373264 secs 2021-11-03T17:26:00.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:00 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:26:00.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 602.373264 secs 2021-11-03T17:26:01.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:01 smithi027 ceph-mon[97244]: pgmap v341: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:01.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:01 smithi115 ceph-mon[65453]: pgmap v341: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:03.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:03 smithi027 ceph-mon[97244]: pgmap v342: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:03.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:03 smithi115 ceph-mon[65453]: pgmap v342: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:05.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:05 smithi115 ceph-mon[65453]: pgmap v343: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:05.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 607.373407 secs 2021-11-03T17:26:05.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:05 smithi027 ceph-mon[97244]: pgmap v343: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:05.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 607.373407 secs 2021-11-03T17:26:07.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:07 smithi115 ceph-mon[65453]: pgmap v344: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:07.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:07 smithi027 ceph-mon[97244]: pgmap v344: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:08.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:08 smithi115 ceph-mon[65453]: pgmap v345: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:08.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:08 smithi027 ceph-mon[97244]: pgmap v345: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:09.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:26:09.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"}]: dispatch 2021-11-03T17:26:09.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2021-11-03T17:26:09.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:26:09.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:26:09.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:09.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:09.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:09.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:09.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:09 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:26:09.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:26:09.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"}]: dispatch 2021-11-03T17:26:09.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2021-11-03T17:26:09.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:26:09.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:26:09.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:09.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:09.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:09.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:09.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:09 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:26:10.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 612.373513 secs 2021-11-03T17:26:10.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:10 smithi115 ceph-mon[65453]: pgmap v346: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:10.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 612.373513 secs 2021-11-03T17:26:10.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:10 smithi027 ceph-mon[97244]: pgmap v346: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:13.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:12 smithi115 ceph-mon[65453]: pgmap v347: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:13.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:12 smithi027 ceph-mon[97244]: pgmap v347: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:15.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:14 smithi115 ceph-mon[65453]: pgmap v348: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:15.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:14 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 617.373655 secs 2021-11-03T17:26:15.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:14 smithi027 ceph-mon[97244]: pgmap v348: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:15.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:14 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 617.373655 secs 2021-11-03T17:26:17.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:16 smithi115 ceph-mon[65453]: pgmap v349: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:17.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:16 smithi027 ceph-mon[97244]: pgmap v349: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:19.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:18 smithi115 ceph-mon[65453]: pgmap v350: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:19.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:26:19.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:26:19.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:26:19.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:19.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:19.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:19.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:19.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:18 smithi027 ceph-mon[97244]: pgmap v350: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:19.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:26:19.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:26:19.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:26:19.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:19.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:19.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:19.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:20.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:19 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:26:20.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:19 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 622.373793 secs 2021-11-03T17:26:20.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:19 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:26:20.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:19 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 622.373793 secs 2021-11-03T17:26:21.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:20 smithi115 ceph-mon[65453]: pgmap v351: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:21.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:20 smithi027 ceph-mon[97244]: pgmap v351: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:21.880 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:21.886+0000 7fa8b9329700 1 -- 172.21.15.27:0/1846270634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8b40f66e0 msgr2=0x7fa8b40f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:26:21.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:21.886+0000 7fa8b9329700 1 --2- 172.21.15.27:0/1846270634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8b40f66e0 0x7fa8b40f6b40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7fa89c004660 tx=0x7fa89c009b30).stop 2021-11-03T17:26:21.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:21.886+0000 7fa8b9329700 1 -- 172.21.15.27:0/1846270634 shutdown_connections 2021-11-03T17:26:21.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:21.886+0000 7fa8b9329700 1 --2- 172.21.15.27:0/1846270634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8b40f66e0 0x7fa8b40f6b40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:21.882 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:21.886+0000 7fa8b9329700 1 --2- 172.21.15.27:0/1846270634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8b40f5440 0x7fa8b40f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:21.882 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:21.886+0000 7fa8b9329700 1 -- 172.21.15.27:0/1846270634 >> 172.21.15.27:0/1846270634 conn(0x7fa8b40f0ba0 msgr2=0x7fa8b40f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:26:21.882 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:21.886+0000 7fa8b9329700 1 -- 172.21.15.27:0/1846270634 shutdown_connections 2021-11-03T17:26:21.883 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:21.886+0000 7fa8b9329700 1 -- 172.21.15.27:0/1846270634 wait complete. 2021-11-03T17:26:21.883 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:21.886+0000 7fa8b9329700 1 Processor -- start 2021-11-03T17:26:21.885 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:21.887+0000 7fa8b9329700 1 -- start start 2021-11-03T17:26:21.885 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:21.887+0000 7fa8b9329700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8b40f5440 0x7fa8b4105d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:26:21.886 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:21.887+0000 7fa8b9329700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8b40f66e0 0x7fa8b4106220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:26:21.886 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:21.887+0000 7fa8b9329700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b41067d0 con 0x7fa8b40f5440 2021-11-03T17:26:21.886 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:21.887+0000 7fa8b9329700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b4106fd0 con 0x7fa8b40f66e0 2021-11-03T17:26:21.886 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:21.887+0000 7fa8b3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8b40f5440 0x7fa8b4105d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:26:21.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:21.887+0000 7fa8b3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8b40f5440 0x7fa8b4105d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57424/0 (socket says 172.21.15.27:57424) 2021-11-03T17:26:21.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:21.887+0000 7fa8b3fff700 1 -- 172.21.15.27:0/1565872794 learned_addr learned my addr 172.21.15.27:0/1565872794 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:26:21.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:21.887+0000 7fa8b37fe700 1 --2- 172.21.15.27:0/1565872794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8b40f66e0 0x7fa8b4106220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:26:21.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:21.887+0000 7fa8b3fff700 1 -- 172.21.15.27:0/1565872794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8b40f66e0 msgr2=0x7fa8b4106220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:26:21.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:21.887+0000 7fa8b3fff700 1 --2- 172.21.15.27:0/1565872794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8b40f66e0 0x7fa8b4106220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:21.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:21.887+0000 7fa8b3fff700 1 -- 172.21.15.27:0/1565872794 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa89c005040 con 0x7fa8b40f5440 2021-11-03T17:26:21.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:21.887+0000 7fa8b3fff700 1 --2- 172.21.15.27:0/1565872794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8b40f5440 0x7fa8b4105d30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7fa8a400e9a0 tx=0x7fa8a4007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:26:21.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:21.888+0000 7fa8b0ff9700 1 -- 172.21.15.27:0/1565872794 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8a400c980 con 0x7fa8b40f5440 2021-11-03T17:26:21.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:21.888+0000 7fa8b9329700 1 -- 172.21.15.27:0/1565872794 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8b4107250 con 0x7fa8b40f5440 2021-11-03T17:26:21.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:21.888+0000 7fa8b0ff9700 1 -- 172.21.15.27:0/1565872794 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fa8a4011470 con 0x7fa8b40f5440 2021-11-03T17:26:21.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:21.888+0000 7fa8b0ff9700 1 -- 172.21.15.27:0/1565872794 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8a4010630 con 0x7fa8b40f5440 2021-11-03T17:26:21.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:21.888+0000 7fa8b9329700 1 -- 172.21.15.27:0/1565872794 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8b41077c0 con 0x7fa8b40f5440 2021-11-03T17:26:21.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:21.889+0000 7fa8b9329700 1 -- 172.21.15.27:0/1565872794 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8b4045ad0 con 0x7fa8b40f5440 2021-11-03T17:26:21.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:21.889+0000 7fa8b0ff9700 1 -- 172.21.15.27:0/1565872794 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fa8a40115e0 con 0x7fa8b40f5440 2021-11-03T17:26:21.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:21.890+0000 7fa8b0ff9700 1 --2- 172.21.15.27:0/1565872794 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa8a00640a0 0x7fa8a0066560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:26:21.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:21.890+0000 7fa8b0ff9700 1 -- 172.21.15.27:0/1565872794 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fa8a4083940 con 0x7fa8b40f5440 2021-11-03T17:26:21.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:21.890+0000 7fa8b37fe700 1 --2- 172.21.15.27:0/1565872794 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa8a00640a0 0x7fa8a0066560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:26:21.891 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:21.890+0000 7fa8b37fe700 1 --2- 172.21.15.27:0/1565872794 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa8a00640a0 0x7fa8a0066560 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fa89c004630 tx=0x7fa89c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:26:21.891 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:21.894+0000 7fa8b0ff9700 1 -- 172.21.15.27:0/1565872794 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fa8a4055af0 con 0x7fa8b40f5440 2021-11-03T17:26:22.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.077+0000 7fa8b9329700 1 -- 172.21.15.27:0/1565872794 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa8b405f620 con 0x7fa8a00640a0 2021-11-03T17:26:22.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.078+0000 7fa8b0ff9700 1 -- 172.21.15.27:0/1565872794 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fa8b405f620 con 0x7fa8a00640a0 2021-11-03T17:26:22.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.080+0000 7fa8b9329700 1 -- 172.21.15.27:0/1565872794 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa8a00640a0 msgr2=0x7fa8a0066560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:26:22.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.080+0000 7fa8b9329700 1 --2- 172.21.15.27:0/1565872794 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa8a00640a0 0x7fa8a0066560 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7fa89c004630 tx=0x7fa89c009b30).stop 2021-11-03T17:26:22.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.080+0000 7fa8b9329700 1 -- 172.21.15.27:0/1565872794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8b40f5440 msgr2=0x7fa8b4105d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:26:22.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.080+0000 7fa8b9329700 1 --2- 172.21.15.27:0/1565872794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8b40f5440 0x7fa8b4105d30 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7fa8a400e9a0 tx=0x7fa8a4007e20).stop 2021-11-03T17:26:22.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.080+0000 7fa8b9329700 1 -- 172.21.15.27:0/1565872794 shutdown_connections 2021-11-03T17:26:22.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.080+0000 7fa8b9329700 1 --2- 172.21.15.27:0/1565872794 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa8a00640a0 0x7fa8a0066560 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:22.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.080+0000 7fa8b9329700 1 --2- 172.21.15.27:0/1565872794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8b40f5440 0x7fa8b4105d30 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:22.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.080+0000 7fa8b9329700 1 --2- 172.21.15.27:0/1565872794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8b40f66e0 0x7fa8b4106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:22.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.080+0000 7fa8b9329700 1 -- 172.21.15.27:0/1565872794 >> 172.21.15.27:0/1565872794 conn(0x7fa8b40f0ba0 msgr2=0x7fa8b40fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:26:22.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.080+0000 7fa8b9329700 1 -- 172.21.15.27:0/1565872794 shutdown_connections 2021-11-03T17:26:22.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.080+0000 7fa8b9329700 1 -- 172.21.15.27:0/1565872794 wait complete. 2021-11-03T17:26:22.084 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:26:22.180 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.182+0000 7f7795aea700 1 -- 172.21.15.27:0/670320603 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77900f6a10 msgr2=0x7f77900f6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:26:22.181 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.182+0000 7f7795aea700 1 --2- 172.21.15.27:0/670320603 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77900f6a10 0x7f77900f6e30 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7f7784004660 tx=0x7f7784009b30).stop 2021-11-03T17:26:22.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.182+0000 7f7795aea700 1 -- 172.21.15.27:0/670320603 shutdown_connections 2021-11-03T17:26:22.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.182+0000 7f7795aea700 1 --2- 172.21.15.27:0/670320603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77900f73b0 0x7f77900f4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:26:22.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.182+0000 7f7795aea700 1 --2- 172.21.15.27:0/670320603 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77900f6a10 0x7f77900f6e30 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:22.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.182+0000 7f7795aea700 1 -- 172.21.15.27:0/670320603 >> 172.21.15.27:0/670320603 conn(0x7f77900f0320 msgr2=0x7f77900f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:26:22.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.182+0000 7f7795aea700 1 -- 172.21.15.27:0/670320603 shutdown_connections 2021-11-03T17:26:22.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.182+0000 7f7795aea700 1 -- 172.21.15.27:0/670320603 wait complete. 2021-11-03T17:26:22.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.183+0000 7f7795aea700 1 Processor -- start 2021-11-03T17:26:22.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.183+0000 7f7795aea700 1 -- start start 2021-11-03T17:26:22.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.183+0000 7f7795aea700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77900f6a10 0x7f7790108f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:26:22.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.183+0000 7f7795aea700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77900f73b0 0x7f7790109430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:26:22.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.183+0000 7f7795aea700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7790109a70 con 0x7f77900f73b0 2021-11-03T17:26:22.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.183+0000 7f7795aea700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f779010a320 con 0x7f77900f6a10 2021-11-03T17:26:22.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.183+0000 7f778ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77900f73b0 0x7f7790109430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:26:22.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.183+0000 7f778ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77900f73b0 0x7f7790109430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57436/0 (socket says 172.21.15.27:57436) 2021-11-03T17:26:22.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.183+0000 7f778ffff700 1 -- 172.21.15.27:0/3061790591 learned_addr learned my addr 172.21.15.27:0/3061790591 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:26:22.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.184+0000 7f7794ae8700 1 --2- 172.21.15.27:0/3061790591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77900f6a10 0x7f7790108f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:26:22.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.184+0000 7f778ffff700 1 -- 172.21.15.27:0/3061790591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77900f6a10 msgr2=0x7f7790108f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:26:22.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.184+0000 7f778ffff700 1 --2- 172.21.15.27:0/3061790591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77900f6a10 0x7f7790108f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:22.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.184+0000 7f778ffff700 1 -- 172.21.15.27:0/3061790591 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7784005040 con 0x7f77900f73b0 2021-11-03T17:26:22.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.184+0000 7f7794ae8700 1 --2- 172.21.15.27:0/3061790591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77900f6a10 0x7f7790108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:26:22.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.184+0000 7f778ffff700 1 --2- 172.21.15.27:0/3061790591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77900f73b0 0x7f7790109430 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f777c00b410 tx=0x7f777c006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:26:22.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.184+0000 7f778d7fa700 1 -- 172.21.15.27:0/3061790591 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f777c0125f0 con 0x7f77900f73b0 2021-11-03T17:26:22.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.184+0000 7f778d7fa700 1 -- 172.21.15.27:0/3061790591 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f777c012c00 con 0x7f77900f73b0 2021-11-03T17:26:22.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.184+0000 7f7795aea700 1 -- 172.21.15.27:0/3061790591 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f779010a600 con 0x7f77900f73b0 2021-11-03T17:26:22.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.184+0000 7f778d7fa700 1 -- 172.21.15.27:0/3061790591 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f777c002ae0 con 0x7f77900f73b0 2021-11-03T17:26:22.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.184+0000 7f7795aea700 1 -- 172.21.15.27:0/3061790591 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f779010abc0 con 0x7f77900f73b0 2021-11-03T17:26:22.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.185+0000 7f7795aea700 1 -- 172.21.15.27:0/3061790591 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7790059100 con 0x7f77900f73b0 2021-11-03T17:26:22.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.186+0000 7f778d7fa700 1 -- 172.21.15.27:0/3061790591 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f777c00f030 con 0x7f77900f73b0 2021-11-03T17:26:22.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.186+0000 7f778d7fa700 1 --2- 172.21.15.27:0/3061790591 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7780064100 0x7f77800665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:26:22.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.186+0000 7f778d7fa700 1 -- 172.21.15.27:0/3061790591 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f777c080ce0 con 0x7f77900f73b0 2021-11-03T17:26:22.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.186+0000 7f7794ae8700 1 --2- 172.21.15.27:0/3061790591 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7780064100 0x7f77800665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:26:22.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.187+0000 7f7794ae8700 1 --2- 172.21.15.27:0/3061790591 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7780064100 0x7f77800665c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f7784009da0 tx=0x7f7784009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:26:22.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.190+0000 7f778d7fa700 1 -- 172.21.15.27:0/3061790591 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f777c052fe0 con 0x7f77900f73b0 2021-11-03T17:26:22.359 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.365+0000 7f7795aea700 1 -- 172.21.15.27:0/3061790591 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7790100fe0 con 0x7f7780064100 2021-11-03T17:26:22.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.381+0000 7f778d7fa700 1 -- 172.21.15.27:0/3061790591 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f7790100fe0 con 0x7f7780064100 2021-11-03T17:26:22.376 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:26:22.376 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (17m) 8m ago 19m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:26:22.376 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (9m) 8m ago 19m 7205k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:26:22.376 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (9m) 6m ago 18m 7239k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:26:22.376 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (17m) 8m ago 18m 38.9M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:26:22.377 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (11m) 8m ago 14m 15.4M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:26:22.377 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (14m) 8m ago 14m 3836M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:26:22.377 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (10m) 6m ago 14m 14.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:26:22.377 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (11m) 6m ago 14m 16.3M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:26:22.378 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (11m) 8m ago 20m 470M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:26:22.378 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (10m) 6m ago 18m 412M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:26:22.378 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (10m) 8m ago 20m 81.2M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:26:22.379 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (10m) 6m ago 18m 69.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:26:22.379 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (18m) 8m ago 18m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:26:22.379 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (18m) 6m ago 18m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:26:22.379 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (9m) 8m ago 17m 243M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:26:22.379 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (8m) 8m ago 17m 270M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:26:22.380 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (8m) 8m ago 16m 11.8M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:26:22.380 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (7m) 6m ago 16m 289M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:26:22.380 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (7m) 6m ago 16m 235M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:26:22.380 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (6m) 6m ago 15m 11.5M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:26:22.381 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (17m) 8m ago 18m 45.7M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:26:22.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.383+0000 7f7795aea700 1 -- 172.21.15.27:0/3061790591 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7780064100 msgr2=0x7f77800665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:26:22.382 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.383+0000 7f7795aea700 1 --2- 172.21.15.27:0/3061790591 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7780064100 0x7f77800665c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f7784009da0 tx=0x7f7784009b30).stop 2021-11-03T17:26:22.382 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.383+0000 7f7795aea700 1 -- 172.21.15.27:0/3061790591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77900f73b0 msgr2=0x7f7790109430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:26:22.382 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.383+0000 7f7795aea700 1 --2- 172.21.15.27:0/3061790591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77900f73b0 0x7f7790109430 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f777c00b410 tx=0x7f777c006ef0).stop 2021-11-03T17:26:22.382 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.383+0000 7f7795aea700 1 -- 172.21.15.27:0/3061790591 shutdown_connections 2021-11-03T17:26:22.383 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.383+0000 7f7795aea700 1 --2- 172.21.15.27:0/3061790591 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7780064100 0x7f77800665c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:22.383 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.383+0000 7f7795aea700 1 --2- 172.21.15.27:0/3061790591 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f77900f6a10 0x7f7790108f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:22.383 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.383+0000 7f7795aea700 1 --2- 172.21.15.27:0/3061790591 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f77900f73b0 0x7f7790109430 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:22.383 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.383+0000 7f7795aea700 1 -- 172.21.15.27:0/3061790591 >> 172.21.15.27:0/3061790591 conn(0x7f77900f0320 msgr2=0x7f77900fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:26:22.384 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.383+0000 7f7795aea700 1 -- 172.21.15.27:0/3061790591 shutdown_connections 2021-11-03T17:26:22.384 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.383+0000 7f7795aea700 1 -- 172.21.15.27:0/3061790591 wait complete. 2021-11-03T17:26:22.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.482+0000 7f2f8268a700 1 -- 172.21.15.27:0/3997914887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f7c0f66e0 msgr2=0x7f2f7c0f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:26:22.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.482+0000 7f2f8268a700 1 --2- 172.21.15.27:0/3997914887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f7c0f66e0 0x7f2f7c0f6b40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f2f70004660 tx=0x7f2f70009b30).stop 2021-11-03T17:26:22.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.482+0000 7f2f8268a700 1 -- 172.21.15.27:0/3997914887 shutdown_connections 2021-11-03T17:26:22.480 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.482+0000 7f2f8268a700 1 --2- 172.21.15.27:0/3997914887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f7c0f66e0 0x7f2f7c0f6b40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:22.480 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.482+0000 7f2f8268a700 1 --2- 172.21.15.27:0/3997914887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f7c0f5440 0x7f2f7c0f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:22.480 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.482+0000 7f2f8268a700 1 -- 172.21.15.27:0/3997914887 >> 172.21.15.27:0/3997914887 conn(0x7f2f7c0f0ba0 msgr2=0x7f2f7c0f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:26:22.481 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.482+0000 7f2f8268a700 1 -- 172.21.15.27:0/3997914887 shutdown_connections 2021-11-03T17:26:22.481 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.482+0000 7f2f8268a700 1 -- 172.21.15.27:0/3997914887 wait complete. 2021-11-03T17:26:22.481 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.483+0000 7f2f8268a700 1 Processor -- start 2021-11-03T17:26:22.482 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.483+0000 7f2f8268a700 1 -- start start 2021-11-03T17:26:22.482 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.483+0000 7f2f8268a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f7c0f5440 0x7f2f7c106dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:26:22.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.483+0000 7f2f8268a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f7c0f66e0 0x7f2f7c1072c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:26:22.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.483+0000 7f2f8268a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7c1078c0 con 0x7f2f7c0f5440 2021-11-03T17:26:22.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.483+0000 7f2f8268a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f7c108100 con 0x7f2f7c0f66e0 2021-11-03T17:26:22.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.483+0000 7f2f81688700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f7c0f5440 0x7f2f7c106dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:26:22.484 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.483+0000 7f2f81688700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f7c0f5440 0x7f2f7c106dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57446/0 (socket says 172.21.15.27:57446) 2021-11-03T17:26:22.484 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.483+0000 7f2f81688700 1 -- 172.21.15.27:0/1641678499 learned_addr learned my addr 172.21.15.27:0/1641678499 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:26:22.484 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.483+0000 7f2f80e87700 1 --2- 172.21.15.27:0/1641678499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f7c0f66e0 0x7f2f7c1072c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:26:22.484 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.484+0000 7f2f81688700 1 -- 172.21.15.27:0/1641678499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f7c0f66e0 msgr2=0x7f2f7c1072c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:26:22.485 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.484+0000 7f2f81688700 1 --2- 172.21.15.27:0/1641678499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f7c0f66e0 0x7f2f7c1072c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:22.486 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.484+0000 7f2f81688700 1 -- 172.21.15.27:0/1641678499 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f70005040 con 0x7f2f7c0f5440 2021-11-03T17:26:22.486 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.484+0000 7f2f81688700 1 --2- 172.21.15.27:0/1641678499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f7c0f5440 0x7f2f7c106dd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f2f6c00d7f0 tx=0x7f2f6c004e80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:26:22.486 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.484+0000 7f2f79ffb700 1 -- 172.21.15.27:0/1641678499 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f6c00f490 con 0x7f2f7c0f5440 2021-11-03T17:26:22.486 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.484+0000 7f2f79ffb700 1 -- 172.21.15.27:0/1641678499 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f2f6c00f040 con 0x7f2f7c0f5440 2021-11-03T17:26:22.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.484+0000 7f2f8268a700 1 -- 172.21.15.27:0/1641678499 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f7c1083e0 con 0x7f2f7c0f5440 2021-11-03T17:26:22.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.484+0000 7f2f79ffb700 1 -- 172.21.15.27:0/1641678499 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f6c0139b0 con 0x7f2f7c0f5440 2021-11-03T17:26:22.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.484+0000 7f2f8268a700 1 -- 172.21.15.27:0/1641678499 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f7c108920 con 0x7f2f7c0f5440 2021-11-03T17:26:22.488 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.485+0000 7f2f8268a700 1 -- 172.21.15.27:0/1641678499 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f7c059100 con 0x7f2f7c0f5440 2021-11-03T17:26:22.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.485+0000 7f2f79ffb700 1 -- 172.21.15.27:0/1641678499 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f2f6c01a030 con 0x7f2f7c0f5440 2021-11-03T17:26:22.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.486+0000 7f2f79ffb700 1 --2- 172.21.15.27:0/1641678499 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2f64064100 0x7f2f640665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:26:22.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.486+0000 7f2f79ffb700 1 -- 172.21.15.27:0/1641678499 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f2f6c080810 con 0x7f2f7c0f5440 2021-11-03T17:26:22.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.486+0000 7f2f80e87700 1 --2- 172.21.15.27:0/1641678499 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2f64064100 0x7f2f640665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:26:22.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.486+0000 7f2f80e87700 1 --2- 172.21.15.27:0/1641678499 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2f64064100 0x7f2f640665c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f2f70004630 tx=0x7f2f70009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:26:22.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.490+0000 7f2f79ffb700 1 -- 172.21.15.27:0/1641678499 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f2f6c052a20 con 0x7f2f7c0f5440 2021-11-03T17:26:22.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.718+0000 7f2f8268a700 1 -- 172.21.15.27:0/1641678499 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2f7c10d340 con 0x7f2f7c0f5440 2021-11-03T17:26:22.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.719+0000 7f2f79ffb700 1 -- 172.21.15.27:0/1641678499 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f2f6c0182e0 con 0x7f2f7c0f5440 2021-11-03T17:26:22.714 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:26:22.714 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:26:22.714 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:26:22.714 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:26:22.715 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:26:22.715 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:26:22.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:26:22.715 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:26:22.716 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:26:22.716 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:26:22.716 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:26:22.716 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:26:22.717 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:26:22.717 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:26:22.717 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:26:22.717 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:26:22.717 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:26:22.718 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:26:22.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.721+0000 7f2f8268a700 1 -- 172.21.15.27:0/1641678499 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2f64064100 msgr2=0x7f2f640665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:26:22.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.721+0000 7f2f8268a700 1 --2- 172.21.15.27:0/1641678499 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2f64064100 0x7f2f640665c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f2f70004630 tx=0x7f2f70009b30).stop 2021-11-03T17:26:22.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.721+0000 7f2f8268a700 1 -- 172.21.15.27:0/1641678499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f7c0f5440 msgr2=0x7f2f7c106dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:26:22.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.721+0000 7f2f8268a700 1 --2- 172.21.15.27:0/1641678499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f7c0f5440 0x7f2f7c106dd0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f2f6c00d7f0 tx=0x7f2f6c004e80).stop 2021-11-03T17:26:22.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.721+0000 7f2f8268a700 1 -- 172.21.15.27:0/1641678499 shutdown_connections 2021-11-03T17:26:22.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.721+0000 7f2f8268a700 1 --2- 172.21.15.27:0/1641678499 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2f64064100 0x7f2f640665c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:22.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.721+0000 7f2f8268a700 1 --2- 172.21.15.27:0/1641678499 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f7c0f5440 0x7f2f7c106dd0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:22.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.721+0000 7f2f8268a700 1 --2- 172.21.15.27:0/1641678499 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f7c0f66e0 0x7f2f7c1072c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:22.721 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.721+0000 7f2f8268a700 1 -- 172.21.15.27:0/1641678499 >> 172.21.15.27:0/1641678499 conn(0x7f2f7c0f0ba0 msgr2=0x7f2f7c05f620 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:26:22.721 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.721+0000 7f2f8268a700 1 -- 172.21.15.27:0/1641678499 shutdown_connections 2021-11-03T17:26:22.721 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.721+0000 7f2f8268a700 1 -- 172.21.15.27:0/1641678499 wait complete. 2021-11-03T17:26:22.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.821+0000 7f23830c9700 1 -- 172.21.15.27:0/838990786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f237c0f66e0 msgr2=0x7f237c0f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:26:22.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.821+0000 7f23830c9700 1 --2- 172.21.15.27:0/838990786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f237c0f66e0 0x7f237c0f6b00 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f2370004660 tx=0x7f2370009b30).stop 2021-11-03T17:26:22.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.821+0000 7f23830c9700 1 -- 172.21.15.27:0/838990786 shutdown_connections 2021-11-03T17:26:22.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.821+0000 7f23830c9700 1 --2- 172.21.15.27:0/838990786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f237c0f6ff0 0x7f237c0f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:22.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.821+0000 7f23830c9700 1 --2- 172.21.15.27:0/838990786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f237c0f66e0 0x7f237c0f6b00 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:22.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.821+0000 7f23830c9700 1 -- 172.21.15.27:0/838990786 >> 172.21.15.27:0/838990786 conn(0x7f237c0f1f00 msgr2=0x7f237c0f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:26:22.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.821+0000 7f23830c9700 1 -- 172.21.15.27:0/838990786 shutdown_connections 2021-11-03T17:26:22.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.821+0000 7f23830c9700 1 -- 172.21.15.27:0/838990786 wait complete. 2021-11-03T17:26:22.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.822+0000 7f23830c9700 1 Processor -- start 2021-11-03T17:26:22.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.822+0000 7f23830c9700 1 -- start start 2021-11-03T17:26:22.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.822+0000 7f23830c9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f237c0f66e0 0x7f237c06b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:26:22.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.822+0000 7f23830c9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f237c0f6ff0 0x7f237c069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:26:22.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.822+0000 7f23830c9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237c10abf0 con 0x7f237c0f66e0 2021-11-03T17:26:22.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.822+0000 7f23830c9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237c10ad30 con 0x7f237c0f6ff0 2021-11-03T17:26:22.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.822+0000 7f2380e65700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f237c0f66e0 0x7f237c06b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:26:22.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.822+0000 7f2380e65700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f237c0f66e0 0x7f237c06b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57456/0 (socket says 172.21.15.27:57456) 2021-11-03T17:26:22.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.822+0000 7f2380e65700 1 -- 172.21.15.27:0/1279298247 learned_addr learned my addr 172.21.15.27:0/1279298247 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:26:22.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.822+0000 7f237bfff700 1 --2- 172.21.15.27:0/1279298247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f237c0f6ff0 0x7f237c069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:26:22.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.822+0000 7f2380e65700 1 -- 172.21.15.27:0/1279298247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f237c0f6ff0 msgr2=0x7f237c069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:26:22.823 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.822+0000 7f2380e65700 1 --2- 172.21.15.27:0/1279298247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f237c0f6ff0 0x7f237c069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:22.823 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.822+0000 7f2380e65700 1 -- 172.21.15.27:0/1279298247 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2370005040 con 0x7f237c0f66e0 2021-11-03T17:26:22.823 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.823+0000 7f2380e65700 1 --2- 172.21.15.27:0/1279298247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f237c0f66e0 0x7f237c06b9a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f2370009990 tx=0x7f2370009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:26:22.823 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.823+0000 7f23797fa700 1 -- 172.21.15.27:0/1279298247 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2370018070 con 0x7f237c0f66e0 2021-11-03T17:26:22.823 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.823+0000 7f23797fa700 1 -- 172.21.15.27:0/1279298247 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f2370007ba0 con 0x7f237c0f66e0 2021-11-03T17:26:22.824 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.823+0000 7f23830c9700 1 -- 172.21.15.27:0/1279298247 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f237c06a370 con 0x7f237c0f66e0 2021-11-03T17:26:22.824 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.823+0000 7f23797fa700 1 -- 172.21.15.27:0/1279298247 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f237001c3c0 con 0x7f237c0f66e0 2021-11-03T17:26:22.824 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.823+0000 7f23830c9700 1 -- 172.21.15.27:0/1279298247 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f237c06a8d0 con 0x7f237c0f66e0 2021-11-03T17:26:22.824 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.824+0000 7f23830c9700 1 -- 172.21.15.27:0/1279298247 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f237c045ad0 con 0x7f237c0f66e0 2021-11-03T17:26:22.825 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.824+0000 7f23797fa700 1 -- 172.21.15.27:0/1279298247 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f237001d420 con 0x7f237c0f66e0 2021-11-03T17:26:22.825 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.825+0000 7f23797fa700 1 --2- 172.21.15.27:0/1279298247 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f236c0640b0 0x7f236c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:26:22.825 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.825+0000 7f23797fa700 1 -- 172.21.15.27:0/1279298247 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f2370082470 con 0x7f237c0f66e0 2021-11-03T17:26:22.825 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.825+0000 7f237bfff700 1 --2- 172.21.15.27:0/1279298247 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f236c0640b0 0x7f236c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:26:22.826 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.825+0000 7f237bfff700 1 --2- 172.21.15.27:0/1279298247 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f236c0640b0 0x7f236c066570 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f2368004e20 tx=0x7f236800b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:26:22.826 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:22.829+0000 7f23797fa700 1 -- 172.21.15.27:0/1279298247 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f2370054710 con 0x7f237c0f66e0 2021-11-03T17:26:23.029 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:23.035+0000 7f23830c9700 1 -- 172.21.15.27:0/1279298247 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f237c027aa0 con 0x7f237c0f66e0 2021-11-03T17:26:23.030 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:23.035+0000 7f23797fa700 1 -- 172.21.15.27:0/1279298247 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f2370054300 con 0x7f237c0f66e0 2021-11-03T17:26:23.030 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:26:23.030 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:26:23.031 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:26:23.031 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:26:23.031 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:26:23.031 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:26:23.032 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:26:23.032 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:26:23.032 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:26:23.032 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:26:23.033 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:26:23.033 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:26:23.033 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:26:23.033 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:26:23.034 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:26:23.034 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:26:23.034 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:26:23.034 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:26:23.034 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:26:23.035 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:26:23.035 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:26:23.035 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:26:23.035 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:26:23.036 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:26:23.036 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:26:23.036 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:26:23.036 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:26:23.036 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:26:23.037 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:26:23.037 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:26:23.037 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:26:23.037 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:26:23.038 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:26:23.038 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:26:23.038 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:26:23.038 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:26:23.039 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:26:23.039 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:26:23.039 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:26:23.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:23.037+0000 7f23830c9700 1 -- 172.21.15.27:0/1279298247 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f236c0640b0 msgr2=0x7f236c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:26:23.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:23.037+0000 7f23830c9700 1 --2- 172.21.15.27:0/1279298247 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f236c0640b0 0x7f236c066570 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f2368004e20 tx=0x7f236800b040).stop 2021-11-03T17:26:23.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:23.037+0000 7f23830c9700 1 -- 172.21.15.27:0/1279298247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f237c0f66e0 msgr2=0x7f237c06b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:26:23.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:23.037+0000 7f23830c9700 1 --2- 172.21.15.27:0/1279298247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f237c0f66e0 0x7f237c06b9a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f2370009990 tx=0x7f2370009b30).stop 2021-11-03T17:26:23.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:23.037+0000 7f23830c9700 1 -- 172.21.15.27:0/1279298247 shutdown_connections 2021-11-03T17:26:23.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:23.037+0000 7f23830c9700 1 --2- 172.21.15.27:0/1279298247 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f236c0640b0 0x7f236c066570 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:23.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:23.037+0000 7f23830c9700 1 --2- 172.21.15.27:0/1279298247 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f237c0f66e0 0x7f237c06b9a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:23.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:23.037+0000 7f23830c9700 1 --2- 172.21.15.27:0/1279298247 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f237c0f6ff0 0x7f237c069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:23.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:23.037+0000 7f23830c9700 1 -- 172.21.15.27:0/1279298247 >> 172.21.15.27:0/1279298247 conn(0x7f237c0f1f00 msgr2=0x7f237c0fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:26:23.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:23.037+0000 7f23830c9700 1 -- 172.21.15.27:0/1279298247 shutdown_connections 2021-11-03T17:26:23.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:23.037+0000 7f23830c9700 1 -- 172.21.15.27:0/1279298247 wait complete. 2021-11-03T17:26:23.043 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:26:23.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:22 smithi027 ceph-mon[97244]: pgmap v352: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:23.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:22 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1641678499' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:23.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:22 smithi115 ceph-mon[65453]: pgmap v352: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:23.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:22 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1641678499' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:24.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:23 smithi115 ceph-mon[65453]: from='client.34472 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:26:24.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:23 smithi115 ceph-mon[65453]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:26:24.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:23 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1279298247' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:26:24.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:23 smithi027 ceph-mon[97244]: from='client.34472 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:26:24.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:23 smithi027 ceph-mon[97244]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:26:24.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:23 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1279298247' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:26:25.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:25 smithi027 ceph-mon[97244]: pgmap v353: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:25.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 627.373934 secs 2021-11-03T17:26:25.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:25 smithi115 ceph-mon[65453]: pgmap v353: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:25.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 627.373934 secs 2021-11-03T17:26:27.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:27 smithi027 ceph-mon[97244]: pgmap v354: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:27.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:27 smithi115 ceph-mon[65453]: pgmap v354: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:29.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:29 smithi027 ceph-mon[97244]: pgmap v355: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:29.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:26:29.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:26:29.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:26:29.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:29.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:29.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:29.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:29.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:29 smithi115 ceph-mon[65453]: pgmap v355: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:29.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:26:29.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:26:29.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:26:29.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:29.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:29.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:29.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:30.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:30 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:26:30.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 632.374028 secs 2021-11-03T17:26:30.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:30 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:26:30.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 632.374028 secs 2021-11-03T17:26:31.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:31 smithi027 ceph-mon[97244]: pgmap v356: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:31.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:31 smithi115 ceph-mon[65453]: pgmap v356: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:33.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:33 smithi027 ceph-mon[97244]: pgmap v357: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:33.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:33 smithi115 ceph-mon[65453]: pgmap v357: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:35.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:35 smithi027 ceph-mon[97244]: pgmap v358: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:35.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 637.374170 secs 2021-11-03T17:26:35.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:35 smithi115 ceph-mon[65453]: pgmap v358: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:35.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 637.374170 secs 2021-11-03T17:26:37.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:37 smithi027 ceph-mon[97244]: pgmap v359: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:37.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:37 smithi115 ceph-mon[65453]: pgmap v359: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:39.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:39 smithi027 ceph-mon[97244]: pgmap v360: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:39.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:26:39.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:26:39.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:26:39.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:39.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:39.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:39.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:39.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:39 smithi115 ceph-mon[65453]: pgmap v360: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:39.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:26:39.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:26:39.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:26:39.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:39.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:39.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:39.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:40.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:40 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:26:40.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 642.374269 secs 2021-11-03T17:26:40.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:40 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:26:40.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 642.374269 secs 2021-11-03T17:26:41.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:41 smithi115 ceph-mon[65453]: pgmap v361: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:41.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:41 smithi027 ceph-mon[97244]: pgmap v361: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:43.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:43 smithi027 ceph-mon[97244]: pgmap v362: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:43.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:43 smithi115 ceph-mon[65453]: pgmap v362: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:45.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:45 smithi115 ceph-mon[65453]: pgmap v363: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:45.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 647.374394 secs 2021-11-03T17:26:45.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:45 smithi027 ceph-mon[97244]: pgmap v363: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:45.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 647.374394 secs 2021-11-03T17:26:47.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:47 smithi115 ceph-mon[65453]: pgmap v364: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:47.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:47 smithi027 ceph-mon[97244]: pgmap v364: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:49.212 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:49 smithi027 ceph-mon[97244]: pgmap v365: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:49.212 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:26:49.212 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:26:49.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:49 smithi115 ceph-mon[65453]: pgmap v365: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:49.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:26:49.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:26:50.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:50 smithi115 ceph-mon[65453]: poll took 60185.017 ms: 1 events 2021-11-03T17:26:50.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:26:50.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:26:50.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:50.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:50.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:50.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:50.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 652.374509 secs 2021-11-03T17:26:50.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:50 smithi027 ceph-mon[97244]: poll took 60185.017 ms: 1 events 2021-11-03T17:26:50.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:26:50.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:26:50.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:50.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:50.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:50.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:50.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 652.374509 secs 2021-11-03T17:26:51.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:51 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:26:51.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:51 smithi115 ceph-mon[65453]: pgmap v366: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:51.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:51 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:26:51.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:51 smithi027 ceph-mon[97244]: pgmap v366: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:52.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:52 smithi115 ceph-mon[65453]: pgmap v367: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:52.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:52 smithi027 ceph-mon[97244]: pgmap v367: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:53.150 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.152+0000 7fccc5d12700 1 -- 172.21.15.27:0/869366699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccc00f6880 msgr2=0x7fccc00f6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:26:53.150 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.152+0000 7fccc5d12700 1 --2- 172.21.15.27:0/869366699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccc00f6880 0x7fccc00f6ce0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7fccb4004660 tx=0x7fccb4009b30).stop 2021-11-03T17:26:53.150 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.152+0000 7fccc5d12700 1 -- 172.21.15.27:0/869366699 shutdown_connections 2021-11-03T17:26:53.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.152+0000 7fccc5d12700 1 --2- 172.21.15.27:0/869366699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccc00f6880 0x7fccc00f6ce0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:53.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.152+0000 7fccc5d12700 1 --2- 172.21.15.27:0/869366699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fccc00f55e0 0x7fccc00f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:53.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.152+0000 7fccc5d12700 1 -- 172.21.15.27:0/869366699 >> 172.21.15.27:0/869366699 conn(0x7fccc00f0d40 msgr2=0x7fccc00f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:26:53.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.152+0000 7fccc5d12700 1 -- 172.21.15.27:0/869366699 shutdown_connections 2021-11-03T17:26:53.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.152+0000 7fccc5d12700 1 -- 172.21.15.27:0/869366699 wait complete. 2021-11-03T17:26:53.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.153+0000 7fccc5d12700 1 Processor -- start 2021-11-03T17:26:53.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.153+0000 7fccc5d12700 1 -- start start 2021-11-03T17:26:53.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.153+0000 7fccc5d12700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccc00f55e0 0x7fccc0109cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:26:53.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.153+0000 7fccc5d12700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fccc00f6880 0x7fccc010c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:26:53.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.153+0000 7fccc5d12700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc0060480 con 0x7fccc00f55e0 2021-11-03T17:26:53.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.153+0000 7fccc5d12700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc00605c0 con 0x7fccc00f6880 2021-11-03T17:26:53.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.153+0000 7fccbf7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccc00f55e0 0x7fccc0109cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:26:53.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.153+0000 7fccbf7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccc00f55e0 0x7fccc0109cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57474/0 (socket says 172.21.15.27:57474) 2021-11-03T17:26:53.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.153+0000 7fccbf7fe700 1 -- 172.21.15.27:0/3132818701 learned_addr learned my addr 172.21.15.27:0/3132818701 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:26:53.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.153+0000 7fccbeffd700 1 --2- 172.21.15.27:0/3132818701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fccc00f6880 0x7fccc010c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:26:53.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.154+0000 7fccbf7fe700 1 -- 172.21.15.27:0/3132818701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fccc00f6880 msgr2=0x7fccc010c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:26:53.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.154+0000 7fccbf7fe700 1 --2- 172.21.15.27:0/3132818701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fccc00f6880 0x7fccc010c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:53.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.154+0000 7fccbf7fe700 1 -- 172.21.15.27:0/3132818701 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb4005040 con 0x7fccc00f55e0 2021-11-03T17:26:53.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.154+0000 7fccbf7fe700 1 --2- 172.21.15.27:0/3132818701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccc00f55e0 0x7fccc0109cb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7fccb000c530 tx=0x7fccb0007e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:26:53.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.154+0000 7fcca7fff700 1 -- 172.21.15.27:0/3132818701 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccb000f650 con 0x7fccc00f55e0 2021-11-03T17:26:53.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.154+0000 7fccc5d12700 1 -- 172.21.15.27:0/3132818701 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc010c760 con 0x7fccc00f55e0 2021-11-03T17:26:53.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.154+0000 7fcca7fff700 1 -- 172.21.15.27:0/3132818701 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fccb000fc60 con 0x7fccc00f55e0 2021-11-03T17:26:53.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.154+0000 7fcca7fff700 1 -- 172.21.15.27:0/3132818701 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fccb0012d20 con 0x7fccc00f55e0 2021-11-03T17:26:53.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.154+0000 7fccc5d12700 1 -- 172.21.15.27:0/3132818701 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc010ccf0 con 0x7fccc00f55e0 2021-11-03T17:26:53.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.155+0000 7fccc5d12700 1 -- 172.21.15.27:0/3132818701 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccc00f49d0 con 0x7fccc00f55e0 2021-11-03T17:26:53.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.155+0000 7fcca7fff700 1 -- 172.21.15.27:0/3132818701 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fccb0019030 con 0x7fccc00f55e0 2021-11-03T17:26:53.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.156+0000 7fcca7fff700 1 --2- 172.21.15.27:0/3132818701 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcca80640b0 0x7fcca8066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:26:53.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.156+0000 7fcca7fff700 1 -- 172.21.15.27:0/3132818701 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fccb00806b0 con 0x7fccc00f55e0 2021-11-03T17:26:53.160 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.156+0000 7fccbeffd700 1 --2- 172.21.15.27:0/3132818701 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcca80640b0 0x7fcca8066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:26:53.160 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.156+0000 7fccbeffd700 1 --2- 172.21.15.27:0/3132818701 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcca80640b0 0x7fcca8066570 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fccb4004e30 tx=0x7fccb4009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:26:53.160 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.160+0000 7fcca7fff700 1 -- 172.21.15.27:0/3132818701 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fccb00529d0 con 0x7fccc00f55e0 2021-11-03T17:26:53.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.337+0000 7fccc5d12700 1 -- 172.21.15.27:0/3132818701 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fccc0057f10 con 0x7fcca80640b0 2021-11-03T17:26:53.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.339+0000 7fcca7fff700 1 -- 172.21.15.27:0/3132818701 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fccc0057f10 con 0x7fcca80640b0 2021-11-03T17:26:53.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.341+0000 7fccc5d12700 1 -- 172.21.15.27:0/3132818701 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcca80640b0 msgr2=0x7fcca8066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:26:53.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.341+0000 7fccc5d12700 1 --2- 172.21.15.27:0/3132818701 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcca80640b0 0x7fcca8066570 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7fccb4004e30 tx=0x7fccb4009b30).stop 2021-11-03T17:26:53.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.341+0000 7fccc5d12700 1 -- 172.21.15.27:0/3132818701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccc00f55e0 msgr2=0x7fccc0109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:26:53.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.341+0000 7fccc5d12700 1 --2- 172.21.15.27:0/3132818701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccc00f55e0 0x7fccc0109cb0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7fccb000c530 tx=0x7fccb0007e10).stop 2021-11-03T17:26:53.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.341+0000 7fccc5d12700 1 -- 172.21.15.27:0/3132818701 shutdown_connections 2021-11-03T17:26:53.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.341+0000 7fccc5d12700 1 --2- 172.21.15.27:0/3132818701 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcca80640b0 0x7fcca8066570 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:53.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.341+0000 7fccc5d12700 1 --2- 172.21.15.27:0/3132818701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fccc00f55e0 0x7fccc0109cb0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:53.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.341+0000 7fccc5d12700 1 --2- 172.21.15.27:0/3132818701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fccc00f6880 0x7fccc010c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:53.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.341+0000 7fccc5d12700 1 -- 172.21.15.27:0/3132818701 >> 172.21.15.27:0/3132818701 conn(0x7fccc00f0d40 msgr2=0x7fccc00fa2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:26:53.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.341+0000 7fccc5d12700 1 -- 172.21.15.27:0/3132818701 shutdown_connections 2021-11-03T17:26:53.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.341+0000 7fccc5d12700 1 -- 172.21.15.27:0/3132818701 wait complete. 2021-11-03T17:26:53.346 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:26:53.436 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.441+0000 7fa7eeb8d700 1 -- 172.21.15.27:0/832514801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7e80f7980 msgr2=0x7fa7e80f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:26:53.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.441+0000 7fa7eeb8d700 1 --2- 172.21.15.27:0/832514801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7e80f7980 0x7fa7e80f7de0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7fa7dc004660 tx=0x7fa7dc009b30).stop 2021-11-03T17:26:53.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.441+0000 7fa7eeb8d700 1 -- 172.21.15.27:0/832514801 shutdown_connections 2021-11-03T17:26:53.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.441+0000 7fa7eeb8d700 1 --2- 172.21.15.27:0/832514801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7e80f7980 0x7fa7e80f7de0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:53.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.441+0000 7fa7eeb8d700 1 --2- 172.21.15.27:0/832514801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7e80f66e0 0x7fa7e80f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:53.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.441+0000 7fa7eeb8d700 1 -- 172.21.15.27:0/832514801 >> 172.21.15.27:0/832514801 conn(0x7fa7e80f1e40 msgr2=0x7fa7e80f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:26:53.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.441+0000 7fa7eeb8d700 1 -- 172.21.15.27:0/832514801 shutdown_connections 2021-11-03T17:26:53.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.441+0000 7fa7eeb8d700 1 -- 172.21.15.27:0/832514801 wait complete. 2021-11-03T17:26:53.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.441+0000 7fa7eeb8d700 1 Processor -- start 2021-11-03T17:26:53.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.441+0000 7fa7eeb8d700 1 -- start start 2021-11-03T17:26:53.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.442+0000 7fa7eeb8d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7e80f66e0 0x7fa7e806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:26:53.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.442+0000 7fa7eeb8d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7e80f7980 0x7fa7e8069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:26:53.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.442+0000 7fa7eeb8d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e806a380 con 0x7fa7e80f7980 2021-11-03T17:26:53.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.442+0000 7fa7eeb8d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e806a4c0 con 0x7fa7e80f66e0 2021-11-03T17:26:53.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.442+0000 7fa7e7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7e80f7980 0x7fa7e8069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:26:53.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.442+0000 7fa7e7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7e80f7980 0x7fa7e8069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57486/0 (socket says 172.21.15.27:57486) 2021-11-03T17:26:53.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.442+0000 7fa7e7fff700 1 -- 172.21.15.27:0/4145778018 learned_addr learned my addr 172.21.15.27:0/4145778018 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:26:53.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.442+0000 7fa7ec929700 1 --2- 172.21.15.27:0/4145778018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7e80f66e0 0x7fa7e806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:26:53.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.442+0000 7fa7e7fff700 1 -- 172.21.15.27:0/4145778018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7e80f66e0 msgr2=0x7fa7e806b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:26:53.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.442+0000 7fa7e7fff700 1 --2- 172.21.15.27:0/4145778018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7e80f66e0 0x7fa7e806b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:53.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.442+0000 7fa7e7fff700 1 -- 172.21.15.27:0/4145778018 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7dc005040 con 0x7fa7e80f7980 2021-11-03T17:26:53.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.442+0000 7fa7e7fff700 1 --2- 172.21.15.27:0/4145778018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7e80f7980 0x7fa7e8069e20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7fa7dc009990 tx=0x7fa7dc009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:26:53.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.442+0000 7fa7e57fa700 1 -- 172.21.15.27:0/4145778018 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7dc018070 con 0x7fa7e80f7980 2021-11-03T17:26:53.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.442+0000 7fa7e57fa700 1 -- 172.21.15.27:0/4145778018 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fa7dc007ca0 con 0x7fa7e80f7980 2021-11-03T17:26:53.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.442+0000 7fa7eeb8d700 1 -- 172.21.15.27:0/4145778018 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e806a740 con 0x7fa7e80f7980 2021-11-03T17:26:53.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.442+0000 7fa7e57fa700 1 -- 172.21.15.27:0/4145778018 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7dc01c620 con 0x7fa7e80f7980 2021-11-03T17:26:53.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.443+0000 7fa7eeb8d700 1 -- 172.21.15.27:0/4145778018 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e806aca0 con 0x7fa7e80f7980 2021-11-03T17:26:53.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.443+0000 7fa7eeb8d700 1 -- 172.21.15.27:0/4145778018 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7e8045ad0 con 0x7fa7e80f7980 2021-11-03T17:26:53.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.444+0000 7fa7e57fa700 1 -- 172.21.15.27:0/4145778018 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fa7dc01d960 con 0x7fa7e80f7980 2021-11-03T17:26:53.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.444+0000 7fa7e57fa700 1 --2- 172.21.15.27:0/4145778018 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa7d00640b0 0x7fa7d0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:26:53.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.444+0000 7fa7e57fa700 1 -- 172.21.15.27:0/4145778018 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fa7dc082580 con 0x7fa7e80f7980 2021-11-03T17:26:53.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.444+0000 7fa7ec929700 1 --2- 172.21.15.27:0/4145778018 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa7d00640b0 0x7fa7d0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:26:53.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.445+0000 7fa7ec929700 1 --2- 172.21.15.27:0/4145778018 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa7d00640b0 0x7fa7d0066570 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fa7e806b730 tx=0x7fa7e0009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:26:53.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.448+0000 7fa7e57fa700 1 -- 172.21.15.27:0/4145778018 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fa7dc054940 con 0x7fa7e80f7980 2021-11-03T17:26:53.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.622+0000 7fa7eeb8d700 1 -- 172.21.15.27:0/4145778018 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7e80fefb0 con 0x7fa7d00640b0 2021-11-03T17:26:53.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.637+0000 7fa7e57fa700 1 -- 172.21.15.27:0/4145778018 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fa7e80fefb0 con 0x7fa7d00640b0 2021-11-03T17:26:53.632 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:26:53.633 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (18m) 8m ago 19m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:26:53.633 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (10m) 8m ago 19m 7205k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:26:53.633 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (9m) 7m ago 18m 7239k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:26:53.633 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (18m) 8m ago 19m 38.9M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:26:53.633 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (11m) 8m ago 14m 15.4M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:26:53.634 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (14m) 8m ago 14m 3836M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:26:53.634 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (10m) 7m ago 14m 14.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:26:53.634 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (12m) 7m ago 14m 16.3M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:26:53.635 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (12m) 8m ago 20m 470M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:26:53.635 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (11m) 7m ago 18m 412M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:26:53.635 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (11m) 8m ago 21m 81.2M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:26:53.635 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (10m) 7m ago 18m 69.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:26:53.635 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (19m) 8m ago 19m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:26:53.636 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (18m) 7m ago 18m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:26:53.636 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (9m) 8m ago 18m 243M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:26:53.636 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (9m) 8m ago 17m 270M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:26:53.636 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (8m) 8m ago 17m 11.8M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:26:53.637 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (8m) 7m ago 17m 289M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:26:53.639 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (7m) 7m ago 16m 235M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:26:53.639 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (7m) 7m ago 16m 11.5M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:26:53.640 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (18m) 8m ago 19m 45.7M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:26:53.641 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.638+0000 7fa7eeb8d700 1 -- 172.21.15.27:0/4145778018 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa7d00640b0 msgr2=0x7fa7d0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:26:53.641 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.638+0000 7fa7eeb8d700 1 --2- 172.21.15.27:0/4145778018 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa7d00640b0 0x7fa7d0066570 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fa7e806b730 tx=0x7fa7e0009040).stop 2021-11-03T17:26:53.642 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.638+0000 7fa7eeb8d700 1 -- 172.21.15.27:0/4145778018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7e80f7980 msgr2=0x7fa7e8069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:26:53.642 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.638+0000 7fa7eeb8d700 1 --2- 172.21.15.27:0/4145778018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7e80f7980 0x7fa7e8069e20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7fa7dc009990 tx=0x7fa7dc009b30).stop 2021-11-03T17:26:53.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.638+0000 7fa7eeb8d700 1 -- 172.21.15.27:0/4145778018 shutdown_connections 2021-11-03T17:26:53.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.638+0000 7fa7eeb8d700 1 --2- 172.21.15.27:0/4145778018 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa7d00640b0 0x7fa7d0066570 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:53.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.638+0000 7fa7eeb8d700 1 --2- 172.21.15.27:0/4145778018 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7e80f66e0 0x7fa7e806b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:53.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.638+0000 7fa7eeb8d700 1 --2- 172.21.15.27:0/4145778018 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7e80f7980 0x7fa7e8069e20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:53.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.638+0000 7fa7eeb8d700 1 -- 172.21.15.27:0/4145778018 >> 172.21.15.27:0/4145778018 conn(0x7fa7e80f1e40 msgr2=0x7fa7e80fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:26:53.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.639+0000 7fa7eeb8d700 1 -- 172.21.15.27:0/4145778018 shutdown_connections 2021-11-03T17:26:53.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.639+0000 7fa7eeb8d700 1 -- 172.21.15.27:0/4145778018 wait complete. 2021-11-03T17:26:53.736 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.737+0000 7f7a27479700 1 -- 172.21.15.27:0/1466421965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a200f5440 msgr2=0x7f7a200f5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:26:53.737 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.737+0000 7f7a27479700 1 --2- 172.21.15.27:0/1466421965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a200f5440 0x7f7a200f5860 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f7a1c004660 tx=0x7f7a1c009b30).stop 2021-11-03T17:26:53.737 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.737+0000 7f7a27479700 1 -- 172.21.15.27:0/1466421965 shutdown_connections 2021-11-03T17:26:53.737 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.737+0000 7f7a27479700 1 --2- 172.21.15.27:0/1466421965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a200f66e0 0x7f7a200f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:53.738 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.737+0000 7f7a27479700 1 --2- 172.21.15.27:0/1466421965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a200f5440 0x7f7a200f5860 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:53.738 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.737+0000 7f7a27479700 1 -- 172.21.15.27:0/1466421965 >> 172.21.15.27:0/1466421965 conn(0x7f7a200f0ba0 msgr2=0x7f7a200f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:26:53.738 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.737+0000 7f7a27479700 1 -- 172.21.15.27:0/1466421965 shutdown_connections 2021-11-03T17:26:53.739 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.737+0000 7f7a27479700 1 -- 172.21.15.27:0/1466421965 wait complete. 2021-11-03T17:26:53.739 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.738+0000 7f7a27479700 1 Processor -- start 2021-11-03T17:26:53.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.738+0000 7f7a27479700 1 -- start start 2021-11-03T17:26:53.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.738+0000 7f7a27479700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a200f5440 0x7f7a20109ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:26:53.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.738+0000 7f7a27479700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a200f66e0 0x7f7a2010bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:26:53.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.738+0000 7f7a27479700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a200603c0 con 0x7f7a200f5440 2021-11-03T17:26:53.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.738+0000 7f7a27479700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a20060500 con 0x7f7a200f66e0 2021-11-03T17:26:53.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.738+0000 7f7a26477700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a200f5440 0x7f7a20109ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:26:53.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.738+0000 7f7a26477700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a200f5440 0x7f7a20109ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57494/0 (socket says 172.21.15.27:57494) 2021-11-03T17:26:53.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.738+0000 7f7a26477700 1 -- 172.21.15.27:0/2299703243 learned_addr learned my addr 172.21.15.27:0/2299703243 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:26:53.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.739+0000 7f7a25c76700 1 --2- 172.21.15.27:0/2299703243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a200f66e0 0x7f7a2010bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:26:53.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.739+0000 7f7a26477700 1 -- 172.21.15.27:0/2299703243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a200f66e0 msgr2=0x7f7a2010bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:26:53.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.739+0000 7f7a26477700 1 --2- 172.21.15.27:0/2299703243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a200f66e0 0x7f7a2010bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:53.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.739+0000 7f7a26477700 1 -- 172.21.15.27:0/2299703243 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a1c005040 con 0x7f7a200f5440 2021-11-03T17:26:53.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.739+0000 7f7a25c76700 1 --2- 172.21.15.27:0/2299703243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a200f66e0 0x7f7a2010bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:26:53.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.739+0000 7f7a26477700 1 --2- 172.21.15.27:0/2299703243 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a200f5440 0x7f7a20109ac0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f7a1c004f60 tx=0x7f7a1c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:26:53.744 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.739+0000 7f7a16ffd700 1 -- 172.21.15.27:0/2299703243 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a1c018070 con 0x7f7a200f5440 2021-11-03T17:26:53.744 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.739+0000 7f7a16ffd700 1 -- 172.21.15.27:0/2299703243 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f7a1c007ba0 con 0x7f7a200f5440 2021-11-03T17:26:53.744 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.739+0000 7f7a16ffd700 1 -- 172.21.15.27:0/2299703243 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a1c01c620 con 0x7f7a200f5440 2021-11-03T17:26:53.744 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.739+0000 7f7a27479700 1 -- 172.21.15.27:0/2299703243 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a2010c510 con 0x7f7a200f5440 2021-11-03T17:26:53.745 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.739+0000 7f7a27479700 1 -- 172.21.15.27:0/2299703243 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a20105ba0 con 0x7f7a200f5440 2021-11-03T17:26:53.745 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.740+0000 7f7a27479700 1 -- 172.21.15.27:0/2299703243 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a20045ad0 con 0x7f7a200f5440 2021-11-03T17:26:53.745 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.740+0000 7f7a16ffd700 1 -- 172.21.15.27:0/2299703243 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f7a1c01d420 con 0x7f7a200f5440 2021-11-03T17:26:53.745 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.741+0000 7f7a16ffd700 1 --2- 172.21.15.27:0/2299703243 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7a0c064100 0x7f7a0c0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:26:53.746 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.741+0000 7f7a16ffd700 1 -- 172.21.15.27:0/2299703243 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f7a1c082670 con 0x7f7a200f5440 2021-11-03T17:26:53.746 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.741+0000 7f7a25c76700 1 --2- 172.21.15.27:0/2299703243 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7a0c064100 0x7f7a0c0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:26:53.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.741+0000 7f7a25c76700 1 --2- 172.21.15.27:0/2299703243 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7a0c064100 0x7f7a0c0665c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f7a100041d0 tx=0x7f7a10009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:26:53.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.745+0000 7f7a16ffd700 1 -- 172.21.15.27:0/2299703243 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f7a1c054a30 con 0x7f7a200f5440 2021-11-03T17:26:53.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.974+0000 7f7a27479700 1 -- 172.21.15.27:0/2299703243 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7a20027a00 con 0x7f7a200f5440 2021-11-03T17:26:53.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.974+0000 7f7a16ffd700 1 -- 172.21.15.27:0/2299703243 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f7a1c054620 con 0x7f7a200f5440 2021-11-03T17:26:53.970 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:26:53.971 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:26:53.971 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:26:53.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:26:53.971 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:26:53.972 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:26:53.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:26:53.972 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:26:53.972 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:26:53.973 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:26:53.973 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:26:53.973 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:26:53.973 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:26:53.973 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:26:53.974 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:26:53.974 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:26:53.974 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:26:53.975 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:26:53.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.976+0000 7f7a27479700 1 -- 172.21.15.27:0/2299703243 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7a0c064100 msgr2=0x7f7a0c0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:26:53.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.976+0000 7f7a27479700 1 --2- 172.21.15.27:0/2299703243 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7a0c064100 0x7f7a0c0665c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f7a100041d0 tx=0x7f7a10009040).stop 2021-11-03T17:26:53.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.976+0000 7f7a27479700 1 -- 172.21.15.27:0/2299703243 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a200f5440 msgr2=0x7f7a20109ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:26:53.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.976+0000 7f7a27479700 1 --2- 172.21.15.27:0/2299703243 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a200f5440 0x7f7a20109ac0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f7a1c004f60 tx=0x7f7a1c009b30).stop 2021-11-03T17:26:53.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.976+0000 7f7a27479700 1 -- 172.21.15.27:0/2299703243 shutdown_connections 2021-11-03T17:26:53.977 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.976+0000 7f7a27479700 1 --2- 172.21.15.27:0/2299703243 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7a0c064100 0x7f7a0c0665c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:53.977 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.976+0000 7f7a27479700 1 --2- 172.21.15.27:0/2299703243 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7a200f5440 0x7f7a20109ac0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:53.977 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.976+0000 7f7a27479700 1 --2- 172.21.15.27:0/2299703243 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7a200f66e0 0x7f7a2010bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:53.977 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.976+0000 7f7a27479700 1 -- 172.21.15.27:0/2299703243 >> 172.21.15.27:0/2299703243 conn(0x7f7a200f0ba0 msgr2=0x7f7a200fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:26:53.978 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.976+0000 7f7a27479700 1 -- 172.21.15.27:0/2299703243 shutdown_connections 2021-11-03T17:26:53.978 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:53.976+0000 7f7a27479700 1 -- 172.21.15.27:0/2299703243 wait complete. 2021-11-03T17:26:54.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.078+0000 7f2c956e1700 1 -- 172.21.15.27:0/1170335087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c9006b9c0 msgr2=0x7f2c90069ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:26:54.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.078+0000 7f2c956e1700 1 --2- 172.21.15.27:0/1170335087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c9006b9c0 0x7f2c90069ec0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7f2c80004660 tx=0x7f2c80009b30).stop 2021-11-03T17:26:54.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.078+0000 7f2c956e1700 1 -- 172.21.15.27:0/1170335087 shutdown_connections 2021-11-03T17:26:54.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.078+0000 7f2c956e1700 1 --2- 172.21.15.27:0/1170335087 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c9006a3b0 0x7f2c9006a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:54.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.078+0000 7f2c956e1700 1 --2- 172.21.15.27:0/1170335087 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c9006b9c0 0x7f2c90069ec0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:54.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.078+0000 7f2c956e1700 1 -- 172.21.15.27:0/1170335087 >> 172.21.15.27:0/1170335087 conn(0x7f2c900f1e40 msgr2=0x7f2c900f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:26:54.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.078+0000 7f2c956e1700 1 -- 172.21.15.27:0/1170335087 shutdown_connections 2021-11-03T17:26:54.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.078+0000 7f2c956e1700 1 -- 172.21.15.27:0/1170335087 wait complete. 2021-11-03T17:26:54.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.078+0000 7f2c956e1700 1 Processor -- start 2021-11-03T17:26:54.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.078+0000 7f2c956e1700 1 -- start start 2021-11-03T17:26:54.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.078+0000 7f2c956e1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c9006a3b0 0x7f2c90109180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:26:54.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.079+0000 7f2c956e1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c9006b9c0 0x7f2c90109670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:26:54.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.079+0000 7f2c956e1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c90109c20 con 0x7f2c9006b9c0 2021-11-03T17:26:54.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.079+0000 7f2c956e1700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c90109d60 con 0x7f2c9006a3b0 2021-11-03T17:26:54.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.079+0000 7f2c8e7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c9006b9c0 0x7f2c90109670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:26:54.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.079+0000 7f2c8e7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c9006b9c0 0x7f2c90109670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57506/0 (socket says 172.21.15.27:57506) 2021-11-03T17:26:54.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.079+0000 7f2c8e7fc700 1 -- 172.21.15.27:0/1430048146 learned_addr learned my addr 172.21.15.27:0/1430048146 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:26:54.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.079+0000 7f2c8effd700 1 --2- 172.21.15.27:0/1430048146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c9006a3b0 0x7f2c90109180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:26:54.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.079+0000 7f2c8e7fc700 1 -- 172.21.15.27:0/1430048146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c9006a3b0 msgr2=0x7f2c90109180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:26:54.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.079+0000 7f2c8e7fc700 1 --2- 172.21.15.27:0/1430048146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c9006a3b0 0x7f2c90109180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:54.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.079+0000 7f2c8e7fc700 1 -- 172.21.15.27:0/1430048146 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c80005040 con 0x7f2c9006b9c0 2021-11-03T17:26:54.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.079+0000 7f2c8e7fc700 1 --2- 172.21.15.27:0/1430048146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c9006b9c0 0x7f2c90109670 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f2c840026d0 tx=0x7f2c84009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:26:54.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.079+0000 7f2c777fe700 1 -- 172.21.15.27:0/1430048146 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c840079c0 con 0x7f2c9006b9c0 2021-11-03T17:26:54.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.079+0000 7f2c777fe700 1 -- 172.21.15.27:0/1430048146 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f2c8400f040 con 0x7f2c9006b9c0 2021-11-03T17:26:54.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.079+0000 7f2c956e1700 1 -- 172.21.15.27:0/1430048146 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c9010a700 con 0x7f2c9006b9c0 2021-11-03T17:26:54.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.079+0000 7f2c777fe700 1 -- 172.21.15.27:0/1430048146 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2c840148b0 con 0x7f2c9006b9c0 2021-11-03T17:26:54.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.079+0000 7f2c956e1700 1 -- 172.21.15.27:0/1430048146 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c9010acc0 con 0x7f2c9006b9c0 2021-11-03T17:26:54.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.080+0000 7f2c956e1700 1 -- 172.21.15.27:0/1430048146 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c90045ad0 con 0x7f2c9006b9c0 2021-11-03T17:26:54.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.081+0000 7f2c777fe700 1 -- 172.21.15.27:0/1430048146 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f2c84007b20 con 0x7f2c9006b9c0 2021-11-03T17:26:54.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.081+0000 7f2c777fe700 1 --2- 172.21.15.27:0/1430048146 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2c780640b0 0x7f2c78066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:26:54.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.081+0000 7f2c777fe700 1 -- 172.21.15.27:0/1430048146 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f2c840807a0 con 0x7f2c9006b9c0 2021-11-03T17:26:54.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.081+0000 7f2c8effd700 1 --2- 172.21.15.27:0/1430048146 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2c780640b0 0x7f2c78066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:26:54.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.082+0000 7f2c8effd700 1 --2- 172.21.15.27:0/1430048146 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2c780640b0 0x7f2c78066570 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f2c80004630 tx=0x7f2c80009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:26:54.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.085+0000 7f2c777fe700 1 -- 172.21.15.27:0/1430048146 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f2c84052aa0 con 0x7f2c9006b9c0 2021-11-03T17:26:54.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:54 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2299703243' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:54.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.288+0000 7f2c956e1700 1 -- 172.21.15.27:0/1430048146 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2c90027a00 con 0x7f2c9006b9c0 2021-11-03T17:26:54.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.289+0000 7f2c777fe700 1 -- 172.21.15.27:0/1430048146 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f2c84052690 con 0x7f2c9006b9c0 2021-11-03T17:26:54.285 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:26:54.285 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:26:54.285 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:26:54.285 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:26:54.286 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:26:54.286 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:26:54.286 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:26:54.286 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:26:54.287 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:26:54.287 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:26:54.287 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:26:54.287 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:26:54.288 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:26:54.288 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:26:54.288 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:26:54.288 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:26:54.288 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:26:54.289 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:26:54.289 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:26:54.289 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:26:54.289 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:26:54.290 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:26:54.290 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:26:54.290 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:26:54.290 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:26:54.290 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:26:54.291 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:26:54.291 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:26:54.291 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:26:54.291 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:26:54.292 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:26:54.292 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:26:54.292 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:26:54.292 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:26:54.292 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:26:54.293 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:26:54.293 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:26:54.293 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:26:54.293 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:26:54.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.290+0000 7f2c956e1700 1 -- 172.21.15.27:0/1430048146 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2c780640b0 msgr2=0x7f2c78066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:26:54.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.290+0000 7f2c956e1700 1 --2- 172.21.15.27:0/1430048146 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2c780640b0 0x7f2c78066570 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f2c80004630 tx=0x7f2c80009b30).stop 2021-11-03T17:26:54.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.290+0000 7f2c956e1700 1 -- 172.21.15.27:0/1430048146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c9006b9c0 msgr2=0x7f2c90109670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:26:54.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.290+0000 7f2c956e1700 1 --2- 172.21.15.27:0/1430048146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c9006b9c0 0x7f2c90109670 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7f2c840026d0 tx=0x7f2c84009de0).stop 2021-11-03T17:26:54.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.291+0000 7f2c956e1700 1 -- 172.21.15.27:0/1430048146 shutdown_connections 2021-11-03T17:26:54.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.291+0000 7f2c956e1700 1 --2- 172.21.15.27:0/1430048146 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2c780640b0 0x7f2c78066570 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:54.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.291+0000 7f2c956e1700 1 --2- 172.21.15.27:0/1430048146 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c9006a3b0 0x7f2c90109180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:54.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.291+0000 7f2c956e1700 1 --2- 172.21.15.27:0/1430048146 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c9006b9c0 0x7f2c90109670 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:26:54.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.291+0000 7f2c956e1700 1 -- 172.21.15.27:0/1430048146 >> 172.21.15.27:0/1430048146 conn(0x7f2c900f1e40 msgr2=0x7f2c900fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:26:54.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.291+0000 7f2c956e1700 1 -- 172.21.15.27:0/1430048146 shutdown_connections 2021-11-03T17:26:54.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:26:54.291+0000 7f2c956e1700 1 -- 172.21.15.27:0/1430048146 wait complete. 2021-11-03T17:26:54.298 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:26:54.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:54 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2299703243' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:26:55.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:55 smithi115 ceph-mon[65453]: from='client.34488 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:26:55.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:55 smithi115 ceph-mon[65453]: from='client.34492 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:26:55.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:55 smithi115 ceph-mon[65453]: pgmap v368: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:55.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:55 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1430048146' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:26:55.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 657.374589 secs 2021-11-03T17:26:55.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:55 smithi027 ceph-mon[97244]: from='client.34488 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:26:55.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:55 smithi027 ceph-mon[97244]: from='client.34492 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:26:55.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:55 smithi027 ceph-mon[97244]: pgmap v368: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:55.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:55 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1430048146' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:26:55.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 657.374589 secs 2021-11-03T17:26:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:26:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:26:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:26:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:26:57.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:57 smithi115 ceph-mon[65453]: pgmap v369: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:57.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:57 smithi027 ceph-mon[97244]: pgmap v369: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:59.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:26:59 smithi115 ceph-mon[65453]: pgmap v370: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:26:59.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:26:59 smithi027 ceph-mon[97244]: pgmap v370: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:00.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:27:00.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:27:00.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:27:00.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:00.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:00.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:00.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:00.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:00 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:27:00.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 662.374669 secs 2021-11-03T17:27:00.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:00 smithi115 ceph-mon[65453]: pgmap v371: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:00.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:27:00.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:27:00.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:27:00.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:00.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:00.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:00.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:00.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:00 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:27:00.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 662.374669 secs 2021-11-03T17:27:00.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:00 smithi027 ceph-mon[97244]: pgmap v371: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:03.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:02 smithi115 ceph-mon[65453]: pgmap v372: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:03.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:02 smithi027 ceph-mon[97244]: pgmap v372: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:05.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:04 smithi027 ceph-mon[97244]: pgmap v373: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:05.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:04 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 667.374807 secs 2021-11-03T17:27:05.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:04 smithi115 ceph-mon[65453]: pgmap v373: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:05.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:04 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 667.374807 secs 2021-11-03T17:27:07.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:06 smithi027 ceph-mon[97244]: pgmap v374: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:07.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:06 smithi115 ceph-mon[65453]: pgmap v374: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:09.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:08 smithi027 ceph-mon[97244]: pgmap v375: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:09.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:08 smithi115 ceph-mon[65453]: pgmap v375: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:10.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:27:10.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:27:10.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:27:10.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:10.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:10.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:10.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:10.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:09 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 672.374911 secs 2021-11-03T17:27:10.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:27:10.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:27:10.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:27:10.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:10.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:10.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:10.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:10.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:09 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 672.374911 secs 2021-11-03T17:27:11.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:11 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:27:11.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:11 smithi027 ceph-mon[97244]: pgmap v376: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:11.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:11 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:27:11.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:11 smithi115 ceph-mon[65453]: pgmap v376: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:13.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:13 smithi027 ceph-mon[97244]: pgmap v377: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:13.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:13 smithi115 ceph-mon[65453]: pgmap v377: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:15.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:15 smithi027 ceph-mon[97244]: pgmap v378: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:15.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 677.375052 secs 2021-11-03T17:27:15.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:15 smithi115 ceph-mon[65453]: pgmap v378: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:15.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 677.375052 secs 2021-11-03T17:27:17.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:17 smithi027 ceph-mon[97244]: pgmap v379: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:17.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:17 smithi115 ceph-mon[65453]: pgmap v379: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:19.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:19 smithi027 ceph-mon[97244]: pgmap v380: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:19.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:19 smithi115 ceph-mon[65453]: pgmap v380: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:20.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:27:20.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:27:20.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:27:20.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:20.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:20.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:20.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:20.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 682.375117 secs 2021-11-03T17:27:20.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:27:20.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:27:20.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:27:20.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:20.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:20.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:20.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:20.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 682.375117 secs 2021-11-03T17:27:21.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:21 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:27:21.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:21 smithi027 ceph-mon[97244]: pgmap v381: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:21.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:21 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:27:21.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:21 smithi115 ceph-mon[65453]: pgmap v381: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:23.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:23 smithi027 ceph-mon[97244]: pgmap v382: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:23.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:23 smithi115 ceph-mon[65453]: pgmap v382: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:24.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.406+0000 7f81eb638700 1 -- 172.21.15.27:0/814208578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81e40f7960 msgr2=0x7f81e40f7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:27:24.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.406+0000 7f81eb638700 1 --2- 172.21.15.27:0/814208578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81e40f7960 0x7f81e40f7dc0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7f81d4004660 tx=0x7f81d4009b30).stop 2021-11-03T17:27:24.402 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.406+0000 7f81eb638700 1 -- 172.21.15.27:0/814208578 shutdown_connections 2021-11-03T17:27:24.402 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.406+0000 7f81eb638700 1 --2- 172.21.15.27:0/814208578 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81e40f7960 0x7f81e40f7dc0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:24.402 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.406+0000 7f81eb638700 1 --2- 172.21.15.27:0/814208578 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81e40f66c0 0x7f81e40f6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:24.403 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.406+0000 7f81eb638700 1 -- 172.21.15.27:0/814208578 >> 172.21.15.27:0/814208578 conn(0x7f81e40f1e20 msgr2=0x7f81e40f4240 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:27:24.403 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.406+0000 7f81eb638700 1 -- 172.21.15.27:0/814208578 shutdown_connections 2021-11-03T17:27:24.403 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.406+0000 7f81eb638700 1 -- 172.21.15.27:0/814208578 wait complete. 2021-11-03T17:27:24.406 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.406+0000 7f81eb638700 1 Processor -- start 2021-11-03T17:27:24.406 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.407+0000 7f81eb638700 1 -- start start 2021-11-03T17:27:24.407 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.407+0000 7f81eb638700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81e40f66c0 0x7f81e41090e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:27:24.407 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.407+0000 7f81eb638700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81e40f7960 0x7f81e41095d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:27:24.407 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.407+0000 7f81eb638700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e4109b80 con 0x7f81e40f7960 2021-11-03T17:27:24.408 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.407+0000 7f81eb638700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81e4109cc0 con 0x7f81e40f66c0 2021-11-03T17:27:24.408 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.407+0000 7f81e8bd3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81e40f7960 0x7f81e41095d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:27:24.408 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.407+0000 7f81e8bd3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81e40f7960 0x7f81e41095d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57524/0 (socket says 172.21.15.27:57524) 2021-11-03T17:27:24.408 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.407+0000 7f81e8bd3700 1 -- 172.21.15.27:0/1340915830 learned_addr learned my addr 172.21.15.27:0/1340915830 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:27:24.409 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.407+0000 7f81e93d4700 1 --2- 172.21.15.27:0/1340915830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81e40f66c0 0x7f81e41090e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:27:24.409 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.407+0000 7f81e8bd3700 1 -- 172.21.15.27:0/1340915830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81e40f66c0 msgr2=0x7f81e41090e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:27:24.409 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.407+0000 7f81e8bd3700 1 --2- 172.21.15.27:0/1340915830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81e40f66c0 0x7f81e41090e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:24.409 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.407+0000 7f81e8bd3700 1 -- 172.21.15.27:0/1340915830 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d4005040 con 0x7f81e40f7960 2021-11-03T17:27:24.409 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.407+0000 7f81e8bd3700 1 --2- 172.21.15.27:0/1340915830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81e40f7960 0x7f81e41095d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f81d4002040 tx=0x7f81d400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:27:24.410 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.407+0000 7f81d9ffb700 1 -- 172.21.15.27:0/1340915830 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81d4019070 con 0x7f81e40f7960 2021-11-03T17:27:24.410 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.408+0000 7f81d9ffb700 1 -- 172.21.15.27:0/1340915830 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f81d4007e70 con 0x7f81e40f7960 2021-11-03T17:27:24.410 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.408+0000 7f81d9ffb700 1 -- 172.21.15.27:0/1340915830 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81d4014db0 con 0x7f81e40f7960 2021-11-03T17:27:24.410 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.408+0000 7f81eb638700 1 -- 172.21.15.27:0/1340915830 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81e410a600 con 0x7f81e40f7960 2021-11-03T17:27:24.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.408+0000 7f81eb638700 1 -- 172.21.15.27:0/1340915830 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81e410ab60 con 0x7f81e40f7960 2021-11-03T17:27:24.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.409+0000 7f81eb638700 1 -- 172.21.15.27:0/1340915830 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81e4045ad0 con 0x7f81e40f7960 2021-11-03T17:27:24.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.409+0000 7f81d9ffb700 1 -- 172.21.15.27:0/1340915830 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f81d4025070 con 0x7f81e40f7960 2021-11-03T17:27:24.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.410+0000 7f81d9ffb700 1 --2- 172.21.15.27:0/1340915830 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f81d0064030 0x7f81d00664f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:27:24.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.410+0000 7f81d9ffb700 1 -- 172.21.15.27:0/1340915830 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f81d4081870 con 0x7f81e40f7960 2021-11-03T17:27:24.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.410+0000 7f81e93d4700 1 --2- 172.21.15.27:0/1340915830 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f81d0064030 0x7f81d00664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:27:24.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.410+0000 7f81e93d4700 1 --2- 172.21.15.27:0/1340915830 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f81d0064030 0x7f81d00664f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f81e0009fd0 tx=0x7f81e000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:27:24.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.414+0000 7f81d9ffb700 1 -- 172.21.15.27:0/1340915830 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f81d4053a50 con 0x7f81e40f7960 2021-11-03T17:27:24.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.591+0000 7f81eb638700 1 -- 172.21.15.27:0/1340915830 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f81e40fef90 con 0x7f81d0064030 2021-11-03T17:27:24.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.593+0000 7f81d9ffb700 1 -- 172.21.15.27:0/1340915830 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f81e40fef90 con 0x7f81d0064030 2021-11-03T17:27:24.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.594+0000 7f81eb638700 1 -- 172.21.15.27:0/1340915830 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f81d0064030 msgr2=0x7f81d00664f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:27:24.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.595+0000 7f81eb638700 1 --2- 172.21.15.27:0/1340915830 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f81d0064030 0x7f81d00664f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f81e0009fd0 tx=0x7f81e000b040).stop 2021-11-03T17:27:24.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.595+0000 7f81eb638700 1 -- 172.21.15.27:0/1340915830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81e40f7960 msgr2=0x7f81e41095d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:27:24.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.595+0000 7f81eb638700 1 --2- 172.21.15.27:0/1340915830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81e40f7960 0x7f81e41095d0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7f81d4002040 tx=0x7f81d400b040).stop 2021-11-03T17:27:24.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.595+0000 7f81eb638700 1 -- 172.21.15.27:0/1340915830 shutdown_connections 2021-11-03T17:27:24.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.595+0000 7f81eb638700 1 --2- 172.21.15.27:0/1340915830 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f81d0064030 0x7f81d00664f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:24.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.595+0000 7f81eb638700 1 --2- 172.21.15.27:0/1340915830 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81e40f66c0 0x7f81e41090e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:24.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.595+0000 7f81eb638700 1 --2- 172.21.15.27:0/1340915830 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81e40f7960 0x7f81e41095d0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:24.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.595+0000 7f81eb638700 1 -- 172.21.15.27:0/1340915830 >> 172.21.15.27:0/1340915830 conn(0x7f81e40f1e20 msgr2=0x7f81e40f93a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:27:24.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.595+0000 7f81eb638700 1 -- 172.21.15.27:0/1340915830 shutdown_connections 2021-11-03T17:27:24.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.595+0000 7f81eb638700 1 -- 172.21.15.27:0/1340915830 wait complete. 2021-11-03T17:27:24.600 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:27:24.690 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.695+0000 7f50a3fff700 1 -- 172.21.15.27:0/168936916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50a40f6530 msgr2=0x7f50a40f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:27:24.691 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.695+0000 7f50a3fff700 1 --2- 172.21.15.27:0/168936916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50a40f6530 0x7f50a40f6950 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f5094004660 tx=0x7f5094009b30).stop 2021-11-03T17:27:24.691 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.695+0000 7f50a3fff700 1 -- 172.21.15.27:0/168936916 shutdown_connections 2021-11-03T17:27:24.691 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.695+0000 7f50a3fff700 1 --2- 172.21.15.27:0/168936916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50a40f77d0 0x7f50a40f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:24.691 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.695+0000 7f50a3fff700 1 --2- 172.21.15.27:0/168936916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50a40f6530 0x7f50a40f6950 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:24.692 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.695+0000 7f50a3fff700 1 -- 172.21.15.27:0/168936916 >> 172.21.15.27:0/168936916 conn(0x7f50a40f1cd0 msgr2=0x7f50a40f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:27:24.692 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.695+0000 7f50a3fff700 1 -- 172.21.15.27:0/168936916 shutdown_connections 2021-11-03T17:27:24.692 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.695+0000 7f50a3fff700 1 -- 172.21.15.27:0/168936916 wait complete. 2021-11-03T17:27:24.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.696+0000 7f50a3fff700 1 Processor -- start 2021-11-03T17:27:24.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.696+0000 7f50a3fff700 1 -- start start 2021-11-03T17:27:24.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.696+0000 7f50a3fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50a40f6530 0x7f50a4106d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:27:24.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.696+0000 7f50a3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50a40f77d0 0x7f50a4107260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:27:24.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.696+0000 7f50a3fff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a4107860 con 0x7f50a40f77d0 2021-11-03T17:27:24.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.696+0000 7f50a3fff700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50a4108070 con 0x7f50a40f6530 2021-11-03T17:27:24.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.696+0000 7f50a27fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50a40f77d0 0x7f50a4107260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:27:24.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.696+0000 7f50a27fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50a40f77d0 0x7f50a4107260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57534/0 (socket says 172.21.15.27:57534) 2021-11-03T17:27:24.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.696+0000 7f50a27fc700 1 -- 172.21.15.27:0/668261128 learned_addr learned my addr 172.21.15.27:0/668261128 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:27:24.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.696+0000 7f50a2ffd700 1 --2- 172.21.15.27:0/668261128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50a40f6530 0x7f50a4106d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:27:24.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.696+0000 7f50a27fc700 1 -- 172.21.15.27:0/668261128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50a40f6530 msgr2=0x7f50a4106d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:27:24.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.696+0000 7f50a27fc700 1 --2- 172.21.15.27:0/668261128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50a40f6530 0x7f50a4106d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:24.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.696+0000 7f50a27fc700 1 -- 172.21.15.27:0/668261128 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5094005040 con 0x7f50a40f77d0 2021-11-03T17:27:24.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.696+0000 7f50a27fc700 1 --2- 172.21.15.27:0/668261128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50a40f77d0 0x7f50a4107260 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f508c002700 tx=0x7f508c009e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:27:24.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.697+0000 7f509b7fe700 1 -- 172.21.15.27:0/668261128 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f508c009320 con 0x7f50a40f77d0 2021-11-03T17:27:24.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.697+0000 7f509b7fe700 1 -- 172.21.15.27:0/668261128 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f508c00f040 con 0x7f50a40f77d0 2021-11-03T17:27:24.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.697+0000 7f50a3fff700 1 -- 172.21.15.27:0/668261128 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50a41082f0 con 0x7f50a40f77d0 2021-11-03T17:27:24.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.697+0000 7f509b7fe700 1 -- 172.21.15.27:0/668261128 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f508c014820 con 0x7f50a40f77d0 2021-11-03T17:27:24.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.697+0000 7f50a3fff700 1 -- 172.21.15.27:0/668261128 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50a41088b0 con 0x7f50a40f77d0 2021-11-03T17:27:24.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.698+0000 7f50a3fff700 1 -- 172.21.15.27:0/668261128 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50a4059100 con 0x7f50a40f77d0 2021-11-03T17:27:24.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.698+0000 7f509b7fe700 1 -- 172.21.15.27:0/668261128 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f508c009480 con 0x7f50a40f77d0 2021-11-03T17:27:24.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.699+0000 7f509b7fe700 1 --2- 172.21.15.27:0/668261128 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5090064100 0x7f50900665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:27:24.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.699+0000 7f509b7fe700 1 -- 172.21.15.27:0/668261128 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f508c080b70 con 0x7f50a40f77d0 2021-11-03T17:27:24.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.699+0000 7f50a2ffd700 1 --2- 172.21.15.27:0/668261128 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5090064100 0x7f50900665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:27:24.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.699+0000 7f50a2ffd700 1 --2- 172.21.15.27:0/668261128 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5090064100 0x7f50900665c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f50a4106ae0 tx=0x7f5094009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:27:24.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.703+0000 7f509b7fe700 1 -- 172.21.15.27:0/668261128 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f508c052e70 con 0x7f50a40f77d0 2021-11-03T17:27:24.871 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.876+0000 7f50a3fff700 1 -- 172.21.15.27:0/668261128 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f50a40fee80 con 0x7f5090064100 2021-11-03T17:27:24.885 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.890+0000 7f509b7fe700 1 -- 172.21.15.27:0/668261128 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f50a40fee80 con 0x7f5090064100 2021-11-03T17:27:24.886 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:27:24.886 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (18m) 9m ago 20m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:27:24.886 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (10m) 9m ago 20m 7205k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:27:24.887 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (10m) 7m ago 19m 7239k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:27:24.887 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (18m) 9m ago 19m 38.9M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:27:24.887 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (12m) 9m ago 15m 15.4M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:27:24.887 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (15m) 9m ago 15m 3836M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:27:24.888 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (11m) 7m ago 15m 14.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:27:24.888 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (12m) 7m ago 15m 16.3M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:27:24.888 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (12m) 9m ago 21m 470M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:27:24.888 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (12m) 7m ago 19m 412M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:27:24.889 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (11m) 9m ago 21m 81.2M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:27:24.889 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (11m) 7m ago 19m 69.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:27:24.889 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (19m) 9m ago 19m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:27:24.889 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (19m) 7m ago 19m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:27:24.889 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (10m) 9m ago 18m 243M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:27:24.890 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (9m) 9m ago 18m 270M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:27:24.890 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (9m) 9m ago 17m 11.8M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:27:24.890 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (8m) 7m ago 17m 289M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:27:24.890 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (8m) 7m ago 17m 235M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:27:24.891 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (7m) 7m ago 16m 11.5M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:27:24.891 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (18m) 9m ago 19m 45.7M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:27:24.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.892+0000 7f50a3fff700 1 -- 172.21.15.27:0/668261128 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5090064100 msgr2=0x7f50900665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:27:24.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.892+0000 7f50a3fff700 1 --2- 172.21.15.27:0/668261128 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5090064100 0x7f50900665c0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7f50a4106ae0 tx=0x7f5094009b30).stop 2021-11-03T17:27:24.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.892+0000 7f50a3fff700 1 -- 172.21.15.27:0/668261128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50a40f77d0 msgr2=0x7f50a4107260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:27:24.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.892+0000 7f50a3fff700 1 --2- 172.21.15.27:0/668261128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50a40f77d0 0x7f50a4107260 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f508c002700 tx=0x7f508c009e00).stop 2021-11-03T17:27:24.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.892+0000 7f50a3fff700 1 -- 172.21.15.27:0/668261128 shutdown_connections 2021-11-03T17:27:24.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.892+0000 7f50a3fff700 1 --2- 172.21.15.27:0/668261128 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5090064100 0x7f50900665c0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:24.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.892+0000 7f50a3fff700 1 --2- 172.21.15.27:0/668261128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f50a40f6530 0x7f50a4106d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:24.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.892+0000 7f50a3fff700 1 --2- 172.21.15.27:0/668261128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f50a40f77d0 0x7f50a4107260 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:24.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.892+0000 7f50a3fff700 1 -- 172.21.15.27:0/668261128 >> 172.21.15.27:0/668261128 conn(0x7f50a40f1cd0 msgr2=0x7f50a40fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:27:24.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.892+0000 7f50a3fff700 1 -- 172.21.15.27:0/668261128 shutdown_connections 2021-11-03T17:27:24.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.892+0000 7f50a3fff700 1 -- 172.21.15.27:0/668261128 wait complete. 2021-11-03T17:27:24.987 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.991+0000 7f407df84700 1 -- 172.21.15.27:0/3952640440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40780f5710 msgr2=0x7f40780f5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:27:24.987 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.991+0000 7f407df84700 1 --2- 172.21.15.27:0/3952640440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40780f5710 0x7f40780f5b90 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f40680099e0 tx=0x7f4068004d10).stop 2021-11-03T17:27:24.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.991+0000 7f407df84700 1 -- 172.21.15.27:0/3952640440 shutdown_connections 2021-11-03T17:27:24.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.991+0000 7f407df84700 1 --2- 172.21.15.27:0/3952640440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40780f5710 0x7f40780f5b90 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:24.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.991+0000 7f407df84700 1 --2- 172.21.15.27:0/3952640440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40780f6a10 0x7f40780f6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:24.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.991+0000 7f407df84700 1 -- 172.21.15.27:0/3952640440 >> 172.21.15.27:0/3952640440 conn(0x7f40780f0320 msgr2=0x7f40780f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:27:24.989 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.991+0000 7f407df84700 1 -- 172.21.15.27:0/3952640440 shutdown_connections 2021-11-03T17:27:24.989 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.991+0000 7f407df84700 1 -- 172.21.15.27:0/3952640440 wait complete. 2021-11-03T17:27:24.989 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.992+0000 7f407df84700 1 Processor -- start 2021-11-03T17:27:24.989 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.992+0000 7f407df84700 1 -- start start 2021-11-03T17:27:24.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.992+0000 7f407df84700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40780f5710 0x7f4078109b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:27:24.992 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.992+0000 7f407df84700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40780f6a10 0x7f407810c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:27:24.992 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.992+0000 7f407df84700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40780f5400 con 0x7f40780f5710 2021-11-03T17:27:24.992 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.992+0000 7f407df84700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40780f5540 con 0x7f40780f6a10 2021-11-03T17:27:24.992 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.992+0000 7f407cf82700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40780f5710 0x7f4078109b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:27:24.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.992+0000 7f407cf82700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40780f5710 0x7f4078109b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57542/0 (socket says 172.21.15.27:57542) 2021-11-03T17:27:24.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.992+0000 7f407cf82700 1 -- 172.21.15.27:0/1976762854 learned_addr learned my addr 172.21.15.27:0/1976762854 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:27:24.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.993+0000 7f4077fff700 1 --2- 172.21.15.27:0/1976762854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40780f6a10 0x7f407810c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:27:24.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.993+0000 7f407cf82700 1 -- 172.21.15.27:0/1976762854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40780f6a10 msgr2=0x7f407810c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:27:24.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.993+0000 7f407cf82700 1 --2- 172.21.15.27:0/1976762854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40780f6a10 0x7f407810c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:24.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.993+0000 7f407cf82700 1 -- 172.21.15.27:0/1976762854 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4068009640 con 0x7f40780f5710 2021-11-03T17:27:24.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.993+0000 7f4077fff700 1 --2- 172.21.15.27:0/1976762854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40780f6a10 0x7f407810c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:27:24.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.993+0000 7f407cf82700 1 --2- 172.21.15.27:0/1976762854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40780f5710 0x7f4078109b30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f407000c530 tx=0x7f4070007e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:27:24.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.993+0000 7f40757fa700 1 -- 172.21.15.27:0/1976762854 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f407000f650 con 0x7f40780f5710 2021-11-03T17:27:24.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.993+0000 7f407df84700 1 -- 172.21.15.27:0/1976762854 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f407810c640 con 0x7f40780f5710 2021-11-03T17:27:24.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.993+0000 7f40757fa700 1 -- 172.21.15.27:0/1976762854 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f407000fc60 con 0x7f40780f5710 2021-11-03T17:27:24.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.993+0000 7f40757fa700 1 -- 172.21.15.27:0/1976762854 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4070012d20 con 0x7f40780f5710 2021-11-03T17:27:24.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.993+0000 7f407df84700 1 -- 172.21.15.27:0/1976762854 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f407810cb50 con 0x7f40780f5710 2021-11-03T17:27:24.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.994+0000 7f407df84700 1 -- 172.21.15.27:0/1976762854 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4078045ad0 con 0x7f40780f5710 2021-11-03T17:27:24.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.995+0000 7f40757fa700 1 -- 172.21.15.27:0/1976762854 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f4070019030 con 0x7f40780f5710 2021-11-03T17:27:24.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.995+0000 7f40757fa700 1 --2- 172.21.15.27:0/1976762854 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4064064100 0x7f40640665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:27:24.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.995+0000 7f40757fa700 1 -- 172.21.15.27:0/1976762854 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f4070080690 con 0x7f40780f5710 2021-11-03T17:27:24.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.995+0000 7f4077fff700 1 --2- 172.21.15.27:0/1976762854 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4064064100 0x7f40640665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:27:24.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.996+0000 7f4077fff700 1 --2- 172.21.15.27:0/1976762854 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4064064100 0x7f40640665c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f406800b3b0 tx=0x7f4068004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:27:24.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:24.999+0000 7f40757fa700 1 -- 172.21.15.27:0/1976762854 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f40700529b0 con 0x7f40780f5710 2021-11-03T17:27:25.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.229+0000 7f407df84700 1 -- 172.21.15.27:0/1976762854 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f40780fdd30 con 0x7f40780f5710 2021-11-03T17:27:25.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.230+0000 7f40757fa700 1 -- 172.21.15.27:0/1976762854 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f40700525a0 con 0x7f40780f5710 2021-11-03T17:27:25.226 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:27:25.226 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:27:25.226 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:27:25.227 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:27:25.227 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:27:25.227 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:27:25.227 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:27:25.228 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:27:25.228 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:27:25.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:27:25.228 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:27:25.228 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:27:25.229 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:27:25.229 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:27:25.229 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:27:25.229 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:27:25.230 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:27:25.230 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:27:25.231 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.232+0000 7f407df84700 1 -- 172.21.15.27:0/1976762854 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4064064100 msgr2=0x7f40640665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:27:25.231 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.232+0000 7f407df84700 1 --2- 172.21.15.27:0/1976762854 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4064064100 0x7f40640665c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f406800b3b0 tx=0x7f4068004d10).stop 2021-11-03T17:27:25.231 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.232+0000 7f407df84700 1 -- 172.21.15.27:0/1976762854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40780f5710 msgr2=0x7f4078109b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:27:25.232 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.232+0000 7f407df84700 1 --2- 172.21.15.27:0/1976762854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40780f5710 0x7f4078109b30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7f407000c530 tx=0x7f4070007e10).stop 2021-11-03T17:27:25.232 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.232+0000 7f407df84700 1 -- 172.21.15.27:0/1976762854 shutdown_connections 2021-11-03T17:27:25.232 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.232+0000 7f407df84700 1 --2- 172.21.15.27:0/1976762854 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4064064100 0x7f40640665c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:25.232 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.232+0000 7f407df84700 1 --2- 172.21.15.27:0/1976762854 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40780f5710 0x7f4078109b30 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:25.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.232+0000 7f407df84700 1 --2- 172.21.15.27:0/1976762854 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40780f6a10 0x7f407810c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:25.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.232+0000 7f407df84700 1 -- 172.21.15.27:0/1976762854 >> 172.21.15.27:0/1976762854 conn(0x7f40780f0320 msgr2=0x7f40780fa170 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:27:25.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.232+0000 7f407df84700 1 -- 172.21.15.27:0/1976762854 shutdown_connections 2021-11-03T17:27:25.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.232+0000 7f407df84700 1 -- 172.21.15.27:0/1976762854 wait complete. 2021-11-03T17:27:25.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:25 smithi027 ceph-mon[97244]: pgmap v383: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:25.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 687.375193 secs 2021-11-03T17:27:25.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.332+0000 7fc9ef59e700 1 -- 172.21.15.27:0/1121518143 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9f00f77d0 msgr2=0x7fc9f00f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:27:25.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.332+0000 7fc9ef59e700 1 --2- 172.21.15.27:0/1121518143 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9f00f77d0 0x7fc9f00f7c30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7fc9e4004660 tx=0x7fc9e4009b30).stop 2021-11-03T17:27:25.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.332+0000 7fc9ef59e700 1 -- 172.21.15.27:0/1121518143 shutdown_connections 2021-11-03T17:27:25.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.332+0000 7fc9ef59e700 1 --2- 172.21.15.27:0/1121518143 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9f00f77d0 0x7fc9f00f7c30 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:25.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.332+0000 7fc9ef59e700 1 --2- 172.21.15.27:0/1121518143 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9f00f6530 0x7fc9f00f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:25.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.332+0000 7fc9ef59e700 1 -- 172.21.15.27:0/1121518143 >> 172.21.15.27:0/1121518143 conn(0x7fc9f00f1cd0 msgr2=0x7fc9f00f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:27:25.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.333+0000 7fc9ef59e700 1 -- 172.21.15.27:0/1121518143 shutdown_connections 2021-11-03T17:27:25.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.333+0000 7fc9ef59e700 1 -- 172.21.15.27:0/1121518143 wait complete. 2021-11-03T17:27:25.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.333+0000 7fc9ef59e700 1 Processor -- start 2021-11-03T17:27:25.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.333+0000 7fc9ef59e700 1 -- start start 2021-11-03T17:27:25.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.333+0000 7fc9ef59e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9f00f6530 0x7fc9f0106d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:27:25.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.333+0000 7fc9ef59e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9f00f77d0 0x7fc9f0107270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:27:25.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.333+0000 7fc9ef59e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f0107870 con 0x7fc9f00f77d0 2021-11-03T17:27:25.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.333+0000 7fc9ef59e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9f0108080 con 0x7fc9f00f6530 2021-11-03T17:27:25.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.333+0000 7fc9edd9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9f00f77d0 0x7fc9f0107270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:27:25.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.333+0000 7fc9edd9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9f00f77d0 0x7fc9f0107270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57554/0 (socket says 172.21.15.27:57554) 2021-11-03T17:27:25.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.333+0000 7fc9edd9b700 1 -- 172.21.15.27:0/2780980218 learned_addr learned my addr 172.21.15.27:0/2780980218 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:27:25.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.334+0000 7fc9ee59c700 1 --2- 172.21.15.27:0/2780980218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9f00f6530 0x7fc9f0106d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:27:25.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.334+0000 7fc9edd9b700 1 -- 172.21.15.27:0/2780980218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9f00f6530 msgr2=0x7fc9f0106d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:27:25.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.334+0000 7fc9edd9b700 1 --2- 172.21.15.27:0/2780980218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9f00f6530 0x7fc9f0106d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:25.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.334+0000 7fc9edd9b700 1 -- 172.21.15.27:0/2780980218 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e4005040 con 0x7fc9f00f77d0 2021-11-03T17:27:25.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.334+0000 7fc9ee59c700 1 --2- 172.21.15.27:0/2780980218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9f00f6530 0x7fc9f0106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:27:25.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.334+0000 7fc9edd9b700 1 --2- 172.21.15.27:0/2780980218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9f00f77d0 0x7fc9f0107270 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7fc9e4009990 tx=0x7fc9e4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:27:25.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.334+0000 7fc9deffd700 1 -- 172.21.15.27:0/2780980218 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9e4018070 con 0x7fc9f00f77d0 2021-11-03T17:27:25.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.334+0000 7fc9deffd700 1 -- 172.21.15.27:0/2780980218 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fc9e4007ca0 con 0x7fc9f00f77d0 2021-11-03T17:27:25.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.334+0000 7fc9deffd700 1 -- 172.21.15.27:0/2780980218 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9e401c620 con 0x7fc9f00f77d0 2021-11-03T17:27:25.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.334+0000 7fc9ef59e700 1 -- 172.21.15.27:0/2780980218 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9f01082a0 con 0x7fc9f00f77d0 2021-11-03T17:27:25.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.334+0000 7fc9ef59e700 1 -- 172.21.15.27:0/2780980218 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9f0108800 con 0x7fc9f00f77d0 2021-11-03T17:27:25.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.335+0000 7fc9ef59e700 1 -- 172.21.15.27:0/2780980218 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9f0059100 con 0x7fc9f00f77d0 2021-11-03T17:27:25.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.335+0000 7fc9deffd700 1 -- 172.21.15.27:0/2780980218 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fc9e401d960 con 0x7fc9f00f77d0 2021-11-03T17:27:25.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.336+0000 7fc9deffd700 1 --2- 172.21.15.27:0/2780980218 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc9d8064100 0x7fc9d80665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:27:25.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.336+0000 7fc9deffd700 1 -- 172.21.15.27:0/2780980218 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fc9e40824d0 con 0x7fc9f00f77d0 2021-11-03T17:27:25.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.336+0000 7fc9ee59c700 1 --2- 172.21.15.27:0/2780980218 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc9d8064100 0x7fc9d80665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:27:25.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.336+0000 7fc9ee59c700 1 --2- 172.21.15.27:0/2780980218 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc9d8064100 0x7fc9d80665c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fc9e0007fd0 tx=0x7fc9e0009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:27:25.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.340+0000 7fc9deffd700 1 -- 172.21.15.27:0/2780980218 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fc9e4054890 con 0x7fc9f00f77d0 2021-11-03T17:27:25.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:25 smithi115 ceph-mon[65453]: pgmap v383: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:25.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 687.375193 secs 2021-11-03T17:27:25.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.559+0000 7fc9ef59e700 1 -- 172.21.15.27:0/2780980218 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc9f0027a00 con 0x7fc9f00f77d0 2021-11-03T17:27:25.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.559+0000 7fc9deffd700 1 -- 172.21.15.27:0/2780980218 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7fc9e4054480 con 0x7fc9f00f77d0 2021-11-03T17:27:25.555 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:27:25.555 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:27:25.556 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:27:25.556 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:27:25.556 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:27:25.556 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:27:25.557 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:27:25.557 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:27:25.557 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:27:25.557 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:27:25.557 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:27:25.558 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:27:25.558 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:27:25.558 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:27:25.558 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:27:25.559 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:27:25.559 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:27:25.559 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:27:25.559 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:27:25.560 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:27:25.560 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:27:25.560 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:27:25.560 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:27:25.560 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:27:25.561 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:27:25.561 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:27:25.561 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:27:25.561 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:27:25.562 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:27:25.562 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:27:25.562 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:27:25.562 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:27:25.562 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:27:25.563 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:27:25.563 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:27:25.563 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:27:25.563 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:27:25.564 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:27:25.564 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:27:25.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.563+0000 7fc9ef59e700 1 -- 172.21.15.27:0/2780980218 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc9d8064100 msgr2=0x7fc9d80665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:27:25.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.563+0000 7fc9ef59e700 1 --2- 172.21.15.27:0/2780980218 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc9d8064100 0x7fc9d80665c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7fc9e0007fd0 tx=0x7fc9e0009040).stop 2021-11-03T17:27:25.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.563+0000 7fc9ef59e700 1 -- 172.21.15.27:0/2780980218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9f00f77d0 msgr2=0x7fc9f0107270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:27:25.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.563+0000 7fc9ef59e700 1 --2- 172.21.15.27:0/2780980218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9f00f77d0 0x7fc9f0107270 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7fc9e4009990 tx=0x7fc9e4009b30).stop 2021-11-03T17:27:25.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.563+0000 7fc9ef59e700 1 -- 172.21.15.27:0/2780980218 shutdown_connections 2021-11-03T17:27:25.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.563+0000 7fc9ef59e700 1 --2- 172.21.15.27:0/2780980218 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc9d8064100 0x7fc9d80665c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:25.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.563+0000 7fc9ef59e700 1 --2- 172.21.15.27:0/2780980218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9f00f6530 0x7fc9f0106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:25.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.563+0000 7fc9ef59e700 1 --2- 172.21.15.27:0/2780980218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9f00f77d0 0x7fc9f0107270 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:25.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.563+0000 7fc9ef59e700 1 -- 172.21.15.27:0/2780980218 >> 172.21.15.27:0/2780980218 conn(0x7fc9f00f1cd0 msgr2=0x7fc9f00fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:27:25.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.563+0000 7fc9ef59e700 1 -- 172.21.15.27:0/2780980218 shutdown_connections 2021-11-03T17:27:25.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:25.563+0000 7fc9ef59e700 1 -- 172.21.15.27:0/2780980218 wait complete. 2021-11-03T17:27:25.567 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:27:26.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:26 smithi115 ceph-mon[65453]: from='client.34504 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:27:26.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:26 smithi115 ceph-mon[65453]: from='client.34508 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:27:26.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:26 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1976762854' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:26.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:26 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2780980218' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:27:26.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:26 smithi027 ceph-mon[97244]: from='client.34504 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:27:26.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:26 smithi027 ceph-mon[97244]: from='client.34508 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:27:26.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:26 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1976762854' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:26.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:26 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2780980218' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:27:27.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:27 smithi115 ceph-mon[65453]: pgmap v384: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:27.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:27 smithi027 ceph-mon[97244]: pgmap v384: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:29.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:29 smithi115 ceph-mon[65453]: pgmap v385: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:29.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:29 smithi027 ceph-mon[97244]: pgmap v385: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:30.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 692.375331 secs 2021-11-03T17:27:30.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:27:30.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:27:30.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:27:30.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:30.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:30.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:30.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:30.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 692.375331 secs 2021-11-03T17:27:30.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:27:30.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:27:30.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:27:30.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:30.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:30.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:30.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:31.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:31 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:27:31.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:31 smithi115 ceph-mon[65453]: pgmap v386: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:31.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:31 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:27:31.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:31 smithi027 ceph-mon[97244]: pgmap v386: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:33.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:33 smithi027 ceph-mon[97244]: pgmap v387: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:33.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:33 smithi115 ceph-mon[65453]: pgmap v387: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:35.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:35 smithi115 ceph-mon[65453]: pgmap v388: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:35.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 697.375454 secs 2021-11-03T17:27:35.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:35 smithi027 ceph-mon[97244]: pgmap v388: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:35.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 697.375454 secs 2021-11-03T17:27:37.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:37 smithi115 ceph-mon[65453]: pgmap v389: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:37.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:37 smithi027 ceph-mon[97244]: pgmap v389: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:39.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:39 smithi115 ceph-mon[65453]: pgmap v390: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:39.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:39 smithi027 ceph-mon[97244]: pgmap v390: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:40.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 702.375595 secs 2021-11-03T17:27:40.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:27:40.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:27:40.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:27:40.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:40.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:40.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:40.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:40.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:40 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:27:40.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:40 smithi115 ceph-mon[65453]: pgmap v391: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:40.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 702.375595 secs 2021-11-03T17:27:40.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:27:40.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:27:40.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:27:40.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:40.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:40.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:40.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:40.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:40 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:27:40.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:40 smithi027 ceph-mon[97244]: pgmap v391: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:43.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:42 smithi027 ceph-mon[97244]: pgmap v392: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:43.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:42 smithi115 ceph-mon[65453]: pgmap v392: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:45.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:45 smithi027 ceph-mon[97244]: pgmap v393: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:45.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 707.375739 secs 2021-11-03T17:27:45.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:45 smithi115 ceph-mon[65453]: pgmap v393: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:45.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 707.375739 secs 2021-11-03T17:27:47.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:47 smithi027 ceph-mon[97244]: pgmap v394: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:47.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:47 smithi115 ceph-mon[65453]: pgmap v394: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:49.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:49 smithi027 ceph-mon[97244]: pgmap v395: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:49.426 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:49 smithi115 ceph-mon[65453]: pgmap v395: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:50.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 712.375833 secs 2021-11-03T17:27:50.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:27:50.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:27:50.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:27:50.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:50.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:50.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:50.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:50.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 712.375833 secs 2021-11-03T17:27:50.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:27:50.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:27:50.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:27:50.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:50.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:50.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:50.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:51.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:51 smithi027 ceph-mon[97244]: poll took 60180.112 ms: 1 events 2021-11-03T17:27:51.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:51 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:27:51.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:51 smithi027 ceph-mon[97244]: pgmap v396: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:51.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:51 smithi115 ceph-mon[65453]: poll took 60180.112 ms: 1 events 2021-11-03T17:27:51.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:51 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:27:51.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:51 smithi115 ceph-mon[65453]: pgmap v396: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:53.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:53 smithi027 ceph-mon[97244]: pgmap v397: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:53.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:53 smithi115 ceph-mon[65453]: pgmap v397: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:55.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:55 smithi027 ceph-mon[97244]: pgmap v398: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:55.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 717.375954 secs 2021-11-03T17:27:55.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:55 smithi115 ceph-mon[65453]: pgmap v398: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:55.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 717.375954 secs 2021-11-03T17:27:55.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.675+0000 7f1a12ac7700 1 -- 172.21.15.27:0/3635969726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a0c0f5440 msgr2=0x7f1a0c0f5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:27:55.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.675+0000 7f1a12ac7700 1 --2- 172.21.15.27:0/3635969726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a0c0f5440 0x7f1a0c0f5860 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f1a08004660 tx=0x7f1a08009b30).stop 2021-11-03T17:27:55.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.675+0000 7f1a12ac7700 1 -- 172.21.15.27:0/3635969726 shutdown_connections 2021-11-03T17:27:55.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.675+0000 7f1a12ac7700 1 --2- 172.21.15.27:0/3635969726 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a0c0f66e0 0x7f1a0c0f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:55.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.675+0000 7f1a12ac7700 1 --2- 172.21.15.27:0/3635969726 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a0c0f5440 0x7f1a0c0f5860 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:55.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.675+0000 7f1a12ac7700 1 -- 172.21.15.27:0/3635969726 >> 172.21.15.27:0/3635969726 conn(0x7f1a0c0f0ba0 msgr2=0x7f1a0c0f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:27:55.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.676+0000 7f1a12ac7700 1 -- 172.21.15.27:0/3635969726 shutdown_connections 2021-11-03T17:27:55.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.676+0000 7f1a12ac7700 1 -- 172.21.15.27:0/3635969726 wait complete. 2021-11-03T17:27:55.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.676+0000 7f1a12ac7700 1 Processor -- start 2021-11-03T17:27:55.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.676+0000 7f1a12ac7700 1 -- start start 2021-11-03T17:27:55.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.676+0000 7f1a12ac7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a0c0f5440 0x7f1a0c106e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:27:55.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.676+0000 7f1a12ac7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a0c0f66e0 0x7f1a0c107320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:27:55.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.676+0000 7f1a12ac7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c107920 con 0x7f1a0c0f66e0 2021-11-03T17:27:55.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.676+0000 7f1a12ac7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c108130 con 0x7f1a0c0f5440 2021-11-03T17:27:55.676 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.676+0000 7f1a112c4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a0c0f66e0 0x7f1a0c107320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:27:55.676 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.677+0000 7f1a112c4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a0c0f66e0 0x7f1a0c107320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57572/0 (socket says 172.21.15.27:57572) 2021-11-03T17:27:55.676 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.677+0000 7f1a112c4700 1 -- 172.21.15.27:0/608157476 learned_addr learned my addr 172.21.15.27:0/608157476 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:27:55.676 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.677+0000 7f1a11ac5700 1 --2- 172.21.15.27:0/608157476 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a0c0f5440 0x7f1a0c106e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:27:55.677 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.677+0000 7f1a112c4700 1 -- 172.21.15.27:0/608157476 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a0c0f5440 msgr2=0x7f1a0c106e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:27:55.677 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.677+0000 7f1a112c4700 1 --2- 172.21.15.27:0/608157476 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a0c0f5440 0x7f1a0c106e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:55.677 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.677+0000 7f1a112c4700 1 -- 172.21.15.27:0/608157476 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a08005040 con 0x7f1a0c0f66e0 2021-11-03T17:27:55.677 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.677+0000 7f1a112c4700 1 --2- 172.21.15.27:0/608157476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a0c0f66e0 0x7f1a0c107320 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f19fc00c530 tx=0x7f19fc007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:27:55.678 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.677+0000 7f1a027fc700 1 -- 172.21.15.27:0/608157476 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19fc00f650 con 0x7f1a0c0f66e0 2021-11-03T17:27:55.678 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.677+0000 7f1a027fc700 1 -- 172.21.15.27:0/608157476 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f19fc00fc60 con 0x7f1a0c0f66e0 2021-11-03T17:27:55.678 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.677+0000 7f1a12ac7700 1 -- 172.21.15.27:0/608157476 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a0c1083b0 con 0x7f1a0c0f66e0 2021-11-03T17:27:55.678 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.677+0000 7f1a027fc700 1 -- 172.21.15.27:0/608157476 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f19fc012c20 con 0x7f1a0c0f66e0 2021-11-03T17:27:55.680 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.677+0000 7f1a12ac7700 1 -- 172.21.15.27:0/608157476 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a0c108970 con 0x7f1a0c0f66e0 2021-11-03T17:27:55.680 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.678+0000 7f1a12ac7700 1 -- 172.21.15.27:0/608157476 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a0c059100 con 0x7f1a0c0f66e0 2021-11-03T17:27:55.680 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.679+0000 7f1a027fc700 1 -- 172.21.15.27:0/608157476 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f19fc010070 con 0x7f1a0c0f66e0 2021-11-03T17:27:55.680 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.679+0000 7f1a027fc700 1 --2- 172.21.15.27:0/608157476 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f19f8064100 0x7f19f80665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:27:55.681 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.679+0000 7f1a027fc700 1 -- 172.21.15.27:0/608157476 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f19fc0806c0 con 0x7f1a0c0f66e0 2021-11-03T17:27:55.681 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.679+0000 7f1a11ac5700 1 --2- 172.21.15.27:0/608157476 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f19f8064100 0x7f19f80665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:27:55.681 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.680+0000 7f1a11ac5700 1 --2- 172.21.15.27:0/608157476 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f19f8064100 0x7f19f80665c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f1a08013690 tx=0x7f1a08009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:27:55.681 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.683+0000 7f1a027fc700 1 -- 172.21.15.27:0/608157476 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f19fc0528d0 con 0x7f1a0c0f66e0 2021-11-03T17:27:55.860 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.865+0000 7f1a12ac7700 1 -- 172.21.15.27:0/608157476 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1a0c05fd30 con 0x7f19f8064100 2021-11-03T17:27:55.861 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.866+0000 7f1a027fc700 1 -- 172.21.15.27:0/608157476 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f1a0c05fd30 con 0x7f19f8064100 2021-11-03T17:27:55.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.868+0000 7f1a12ac7700 1 -- 172.21.15.27:0/608157476 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f19f8064100 msgr2=0x7f19f80665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:27:55.864 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.868+0000 7f1a12ac7700 1 --2- 172.21.15.27:0/608157476 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f19f8064100 0x7f19f80665c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f1a08013690 tx=0x7f1a08009b30).stop 2021-11-03T17:27:55.864 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.868+0000 7f1a12ac7700 1 -- 172.21.15.27:0/608157476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a0c0f66e0 msgr2=0x7f1a0c107320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:27:55.865 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.868+0000 7f1a12ac7700 1 --2- 172.21.15.27:0/608157476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a0c0f66e0 0x7f1a0c107320 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f19fc00c530 tx=0x7f19fc007e20).stop 2021-11-03T17:27:55.865 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.868+0000 7f1a12ac7700 1 -- 172.21.15.27:0/608157476 shutdown_connections 2021-11-03T17:27:55.865 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.868+0000 7f1a12ac7700 1 --2- 172.21.15.27:0/608157476 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f19f8064100 0x7f19f80665c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:55.866 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.868+0000 7f1a12ac7700 1 --2- 172.21.15.27:0/608157476 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a0c0f5440 0x7f1a0c106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:55.866 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.868+0000 7f1a12ac7700 1 --2- 172.21.15.27:0/608157476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a0c0f66e0 0x7f1a0c107320 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:55.867 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.868+0000 7f1a12ac7700 1 -- 172.21.15.27:0/608157476 >> 172.21.15.27:0/608157476 conn(0x7f1a0c0f0ba0 msgr2=0x7f1a0c05f620 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:27:55.867 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.868+0000 7f1a12ac7700 1 -- 172.21.15.27:0/608157476 shutdown_connections 2021-11-03T17:27:55.868 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.869+0000 7f1a12ac7700 1 -- 172.21.15.27:0/608157476 wait complete. 2021-11-03T17:27:55.873 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:27:55.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.970+0000 7f822caea700 1 -- 172.21.15.27:0/1198103292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82280f55c0 msgr2=0x7f82280f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:27:55.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.970+0000 7f822caea700 1 --2- 172.21.15.27:0/1198103292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82280f55c0 0x7f82280f59e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f8210004660 tx=0x7f8210009b30).stop 2021-11-03T17:27:55.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.970+0000 7f822caea700 1 -- 172.21.15.27:0/1198103292 shutdown_connections 2021-11-03T17:27:55.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.970+0000 7f822caea700 1 --2- 172.21.15.27:0/1198103292 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82280f5ed0 0x7f82280f6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:55.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.970+0000 7f822caea700 1 --2- 172.21.15.27:0/1198103292 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82280f55c0 0x7f82280f59e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:55.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.970+0000 7f822caea700 1 -- 172.21.15.27:0/1198103292 >> 172.21.15.27:0/1198103292 conn(0x7f82280f0de0 msgr2=0x7f82280f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:27:55.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.970+0000 7f822caea700 1 -- 172.21.15.27:0/1198103292 shutdown_connections 2021-11-03T17:27:55.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.970+0000 7f822caea700 1 -- 172.21.15.27:0/1198103292 wait complete. 2021-11-03T17:27:55.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.970+0000 7f822caea700 1 Processor -- start 2021-11-03T17:27:55.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.970+0000 7f822caea700 1 -- start start 2021-11-03T17:27:55.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.971+0000 7f822caea700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82280f55c0 0x7f8228106f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:27:55.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.971+0000 7f822caea700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82280f5ed0 0x7f82281073f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:27:55.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.971+0000 7f822caea700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82281079a0 con 0x7f82280f5ed0 2021-11-03T17:27:55.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.971+0000 7f822caea700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8228107ae0 con 0x7f82280f55c0 2021-11-03T17:27:55.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.971+0000 7f8225d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82280f5ed0 0x7f82281073f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:27:55.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.971+0000 7f8225d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82280f5ed0 0x7f82281073f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57580/0 (socket says 172.21.15.27:57580) 2021-11-03T17:27:55.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.971+0000 7f8225d9b700 1 -- 172.21.15.27:0/3142583187 learned_addr learned my addr 172.21.15.27:0/3142583187 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:27:55.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.971+0000 7f822659c700 1 --2- 172.21.15.27:0/3142583187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82280f55c0 0x7f8228106f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:27:55.973 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.971+0000 7f8225d9b700 1 -- 172.21.15.27:0/3142583187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82280f55c0 msgr2=0x7f8228106f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:27:55.973 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.971+0000 7f8225d9b700 1 --2- 172.21.15.27:0/3142583187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82280f55c0 0x7f8228106f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:55.973 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.971+0000 7f8225d9b700 1 -- 172.21.15.27:0/3142583187 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8210005040 con 0x7f82280f5ed0 2021-11-03T17:27:55.973 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.971+0000 7f8225d9b700 1 --2- 172.21.15.27:0/3142583187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82280f5ed0 0x7f82281073f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f821800e7f0 tx=0x7f821800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:27:55.973 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.971+0000 7f821effd700 1 -- 172.21.15.27:0/3142583187 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82180092e0 con 0x7f82280f5ed0 2021-11-03T17:27:55.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.971+0000 7f821effd700 1 -- 172.21.15.27:0/3142583187 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f821800f040 con 0x7f82280f5ed0 2021-11-03T17:27:55.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.971+0000 7f822caea700 1 -- 172.21.15.27:0/3142583187 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8228108480 con 0x7f82280f5ed0 2021-11-03T17:27:55.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.971+0000 7f821effd700 1 -- 172.21.15.27:0/3142583187 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8218014750 con 0x7f82280f5ed0 2021-11-03T17:27:55.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.971+0000 7f822caea700 1 -- 172.21.15.27:0/3142583187 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8228108a40 con 0x7f82280f5ed0 2021-11-03T17:27:55.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.972+0000 7f822caea700 1 -- 172.21.15.27:0/3142583187 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8228045ad0 con 0x7f82280f5ed0 2021-11-03T17:27:55.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.973+0000 7f821effd700 1 -- 172.21.15.27:0/3142583187 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f821801b030 con 0x7f82280f5ed0 2021-11-03T17:27:55.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.973+0000 7f821effd700 1 --2- 172.21.15.27:0/3142583187 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f82140640b0 0x7f8214066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:27:55.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.973+0000 7f821effd700 1 -- 172.21.15.27:0/3142583187 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f82180806c0 con 0x7f82280f5ed0 2021-11-03T17:27:55.977 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.973+0000 7f822659c700 1 --2- 172.21.15.27:0/3142583187 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f82140640b0 0x7f8214066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:27:55.977 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.974+0000 7f822659c700 1 --2- 172.21.15.27:0/3142583187 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f82140640b0 0x7f8214066570 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f82100043d0 tx=0x7f821000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:27:55.977 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:55.977+0000 7f821effd700 1 -- 172.21.15.27:0/3142583187 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f82180528c0 con 0x7f82280f5ed0 2021-11-03T17:27:56.150 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.155+0000 7f822caea700 1 -- 172.21.15.27:0/3142583187 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f822805fe90 con 0x7f82140640b0 2021-11-03T17:27:56.165 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.170+0000 7f821effd700 1 -- 172.21.15.27:0/3142583187 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f822805fe90 con 0x7f82140640b0 2021-11-03T17:27:56.165 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:27:56.165 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (19m) 9m ago 20m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:27:56.166 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (11m) 9m ago 20m 7205k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:27:56.166 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (11m) 8m ago 19m 7239k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:27:56.166 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (19m) 9m ago 20m 38.9M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:27:56.166 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (12m) 9m ago 15m 15.4M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:27:56.167 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (15m) 9m ago 15m 3836M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:27:56.167 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (11m) 8m ago 15m 14.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:27:56.167 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (13m) 8m ago 16m 16.3M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:27:56.168 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (13m) 9m ago 22m 470M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:27:56.168 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (12m) 8m ago 19m 412M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:27:56.168 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (12m) 9m ago 22m 81.2M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:27:56.168 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (11m) 8m ago 19m 69.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:27:56.169 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (20m) 9m ago 20m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:27:56.169 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (19m) 8m ago 19m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:27:56.169 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (10m) 9m ago 19m 243M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:27:56.169 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (10m) 9m ago 18m 270M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:27:56.170 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (9m) 9m ago 18m 11.8M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:27:56.170 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (9m) 8m ago 18m 289M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:27:56.170 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (8m) 8m ago 17m 235M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:27:56.170 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (8m) 8m ago 17m 11.5M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:27:56.171 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (19m) 9m ago 20m 45.7M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:27:56.171 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.171+0000 7f822caea700 1 -- 172.21.15.27:0/3142583187 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f82140640b0 msgr2=0x7f8214066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:27:56.172 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.171+0000 7f822caea700 1 --2- 172.21.15.27:0/3142583187 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f82140640b0 0x7f8214066570 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7f82100043d0 tx=0x7f821000b040).stop 2021-11-03T17:27:56.172 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.171+0000 7f822caea700 1 -- 172.21.15.27:0/3142583187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82280f5ed0 msgr2=0x7f82281073f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:27:56.172 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.171+0000 7f822caea700 1 --2- 172.21.15.27:0/3142583187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82280f5ed0 0x7f82281073f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f821800e7f0 tx=0x7f821800b040).stop 2021-11-03T17:27:56.173 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.171+0000 7f822caea700 1 -- 172.21.15.27:0/3142583187 shutdown_connections 2021-11-03T17:27:56.173 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.171+0000 7f822caea700 1 --2- 172.21.15.27:0/3142583187 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f82140640b0 0x7f8214066570 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:56.173 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.171+0000 7f822caea700 1 --2- 172.21.15.27:0/3142583187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82280f55c0 0x7f8228106f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:56.173 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.171+0000 7f822caea700 1 --2- 172.21.15.27:0/3142583187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82280f5ed0 0x7f82281073f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:56.173 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.172+0000 7f822caea700 1 -- 172.21.15.27:0/3142583187 >> 172.21.15.27:0/3142583187 conn(0x7f82280f0de0 msgr2=0x7f822805f780 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:27:56.174 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.172+0000 7f822caea700 1 -- 172.21.15.27:0/3142583187 shutdown_connections 2021-11-03T17:27:56.174 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.172+0000 7f822caea700 1 -- 172.21.15.27:0/3142583187 wait complete. 2021-11-03T17:27:56.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.271+0000 7fcb35339700 1 -- 172.21.15.27:0/3571276216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb300f77d0 msgr2=0x7fcb300f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:27:56.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.271+0000 7fcb35339700 1 --2- 172.21.15.27:0/3571276216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb300f77d0 0x7fcb300f7c30 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7fcb20004660 tx=0x7fcb20009b30).stop 2021-11-03T17:27:56.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.271+0000 7fcb35339700 1 -- 172.21.15.27:0/3571276216 shutdown_connections 2021-11-03T17:27:56.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.271+0000 7fcb35339700 1 --2- 172.21.15.27:0/3571276216 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb300f77d0 0x7fcb300f7c30 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:56.270 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.271+0000 7fcb35339700 1 --2- 172.21.15.27:0/3571276216 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb300f6530 0x7fcb300f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:56.270 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.271+0000 7fcb35339700 1 -- 172.21.15.27:0/3571276216 >> 172.21.15.27:0/3571276216 conn(0x7fcb300f1cd0 msgr2=0x7fcb300f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:27:56.270 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.272+0000 7fcb35339700 1 -- 172.21.15.27:0/3571276216 shutdown_connections 2021-11-03T17:27:56.270 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.272+0000 7fcb35339700 1 -- 172.21.15.27:0/3571276216 wait complete. 2021-11-03T17:27:56.271 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.272+0000 7fcb35339700 1 Processor -- start 2021-11-03T17:27:56.271 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.272+0000 7fcb35339700 1 -- start start 2021-11-03T17:27:56.271 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.272+0000 7fcb35339700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb300f6530 0x7fcb30106d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:27:56.271 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.272+0000 7fcb35339700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb300f77d0 0x7fcb30107270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:27:56.272 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.272+0000 7fcb35339700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb30107870 con 0x7fcb300f6530 2021-11-03T17:27:56.272 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.272+0000 7fcb35339700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb30108080 con 0x7fcb300f77d0 2021-11-03T17:27:56.272 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.272+0000 7fcb2ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb300f6530 0x7fcb30106d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:27:56.273 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.272+0000 7fcb2ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb300f6530 0x7fcb30106d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57588/0 (socket says 172.21.15.27:57588) 2021-11-03T17:27:56.273 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.273+0000 7fcb2ffff700 1 -- 172.21.15.27:0/3374390699 learned_addr learned my addr 172.21.15.27:0/3374390699 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:27:56.273 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.273+0000 7fcb2ffff700 1 -- 172.21.15.27:0/3374390699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb300f77d0 msgr2=0x7fcb30107270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:27:56.273 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.273+0000 7fcb2f7fe700 1 --2- 172.21.15.27:0/3374390699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb300f77d0 0x7fcb30107270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:27:56.274 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.273+0000 7fcb2ffff700 1 --2- 172.21.15.27:0/3374390699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb300f77d0 0x7fcb30107270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:56.275 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.273+0000 7fcb2ffff700 1 -- 172.21.15.27:0/3374390699 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb20005040 con 0x7fcb300f6530 2021-11-03T17:27:56.275 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.273+0000 7fcb2f7fe700 1 --2- 172.21.15.27:0/3374390699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb300f77d0 0x7fcb30107270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:27:56.276 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.273+0000 7fcb2ffff700 1 --2- 172.21.15.27:0/3374390699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb300f6530 0x7fcb30106d80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fcb24002700 tx=0x7fcb24009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:27:56.276 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.273+0000 7fcb2cff9700 1 -- 172.21.15.27:0/3374390699 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb24009320 con 0x7fcb300f6530 2021-11-03T17:27:56.276 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.273+0000 7fcb2cff9700 1 -- 172.21.15.27:0/3374390699 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fcb2400f040 con 0x7fcb300f6530 2021-11-03T17:27:56.277 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.273+0000 7fcb35339700 1 -- 172.21.15.27:0/3374390699 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb30108300 con 0x7fcb300f6530 2021-11-03T17:27:56.277 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.273+0000 7fcb2cff9700 1 -- 172.21.15.27:0/3374390699 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcb24014900 con 0x7fcb300f6530 2021-11-03T17:27:56.277 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.273+0000 7fcb35339700 1 -- 172.21.15.27:0/3374390699 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb301088c0 con 0x7fcb300f6530 2021-11-03T17:27:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.274+0000 7fcb35339700 1 -- 172.21.15.27:0/3374390699 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb30059100 con 0x7fcb300f6530 2021-11-03T17:27:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.275+0000 7fcb2cff9700 1 -- 172.21.15.27:0/3374390699 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fcb24009480 con 0x7fcb300f6530 2021-11-03T17:27:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.275+0000 7fcb2cff9700 1 --2- 172.21.15.27:0/3374390699 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcb18064100 0x7fcb180665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:27:56.279 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.275+0000 7fcb2cff9700 1 -- 172.21.15.27:0/3374390699 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fcb24080ab0 con 0x7fcb300f6530 2021-11-03T17:27:56.280 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.275+0000 7fcb2f7fe700 1 --2- 172.21.15.27:0/3374390699 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcb18064100 0x7fcb180665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:27:56.280 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.276+0000 7fcb2f7fe700 1 --2- 172.21.15.27:0/3374390699 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcb18064100 0x7fcb180665c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7fcb20009da0 tx=0x7fcb20009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:27:56.280 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.279+0000 7fcb2cff9700 1 -- 172.21.15.27:0/3374390699 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fcb24052db0 con 0x7fcb300f6530 2021-11-03T17:27:56.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:27:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:27:56.507 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.512+0000 7fcb35339700 1 -- 172.21.15.27:0/3374390699 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcb30108b40 con 0x7fcb300f6530 2021-11-03T17:27:56.508 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.512+0000 7fcb2cff9700 1 -- 172.21.15.27:0/3374390699 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fcb240529a0 con 0x7fcb300f6530 2021-11-03T17:27:56.508 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:27:56.508 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:27:56.509 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:27:56.509 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:27:56.509 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:27:56.509 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:27:56.510 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:27:56.510 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:27:56.511 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:27:56.511 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:27:56.511 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:27:56.511 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:27:56.512 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:27:56.512 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:27:56.512 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:27:56.512 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:27:56.512 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:27:56.513 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:27:56.513 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.514+0000 7fcb35339700 1 -- 172.21.15.27:0/3374390699 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcb18064100 msgr2=0x7fcb180665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:27:56.514 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.514+0000 7fcb35339700 1 --2- 172.21.15.27:0/3374390699 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcb18064100 0x7fcb180665c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7fcb20009da0 tx=0x7fcb20009b30).stop 2021-11-03T17:27:56.514 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.514+0000 7fcb35339700 1 -- 172.21.15.27:0/3374390699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb300f6530 msgr2=0x7fcb30106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:27:56.514 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.514+0000 7fcb35339700 1 --2- 172.21.15.27:0/3374390699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb300f6530 0x7fcb30106d80 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7fcb24002700 tx=0x7fcb24009d80).stop 2021-11-03T17:27:56.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.514+0000 7fcb35339700 1 -- 172.21.15.27:0/3374390699 shutdown_connections 2021-11-03T17:27:56.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.514+0000 7fcb35339700 1 --2- 172.21.15.27:0/3374390699 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcb18064100 0x7fcb180665c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:56.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.514+0000 7fcb35339700 1 --2- 172.21.15.27:0/3374390699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb300f6530 0x7fcb30106d80 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:56.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.514+0000 7fcb35339700 1 --2- 172.21.15.27:0/3374390699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb300f77d0 0x7fcb30107270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:56.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.514+0000 7fcb35339700 1 -- 172.21.15.27:0/3374390699 >> 172.21.15.27:0/3374390699 conn(0x7fcb300f1cd0 msgr2=0x7fcb300fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:27:56.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.514+0000 7fcb35339700 1 -- 172.21.15.27:0/3374390699 shutdown_connections 2021-11-03T17:27:56.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.514+0000 7fcb35339700 1 -- 172.21.15.27:0/3374390699 wait complete. 2021-11-03T17:27:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:27:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:27:56.615 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.614+0000 7fa811d9c700 1 -- 172.21.15.27:0/1758815735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa80c0f5ef0 msgr2=0x7fa80c0f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:27:56.615 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.614+0000 7fa811d9c700 1 --2- 172.21.15.27:0/1758815735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa80c0f5ef0 0x7fa80c0f6920 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7fa800004660 tx=0x7fa800009b30).stop 2021-11-03T17:27:56.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.614+0000 7fa811d9c700 1 -- 172.21.15.27:0/1758815735 shutdown_connections 2021-11-03T17:27:56.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.614+0000 7fa811d9c700 1 --2- 172.21.15.27:0/1758815735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa80c0f5ef0 0x7fa80c0f6920 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:56.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.614+0000 7fa811d9c700 1 --2- 172.21.15.27:0/1758815735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa80c0f55e0 0x7fa80c0f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:56.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.614+0000 7fa811d9c700 1 -- 172.21.15.27:0/1758815735 >> 172.21.15.27:0/1758815735 conn(0x7fa80c0f0de0 msgr2=0x7fa80c0f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:27:56.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.615+0000 7fa811d9c700 1 -- 172.21.15.27:0/1758815735 shutdown_connections 2021-11-03T17:27:56.618 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.615+0000 7fa811d9c700 1 -- 172.21.15.27:0/1758815735 wait complete. 2021-11-03T17:27:56.618 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.615+0000 7fa811d9c700 1 Processor -- start 2021-11-03T17:27:56.618 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.615+0000 7fa811d9c700 1 -- start start 2021-11-03T17:27:56.619 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.615+0000 7fa811d9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa80c0f55e0 0x7fa80c106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:27:56.619 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.615+0000 7fa811d9c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa80c0f5ef0 0x7fa80c107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:27:56.619 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.615+0000 7fa811d9c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80c107a10 con 0x7fa80c0f55e0 2021-11-03T17:27:56.620 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.615+0000 7fa811d9c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa80c107b50 con 0x7fa80c0f5ef0 2021-11-03T17:27:56.620 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.616+0000 7fa80b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa80c0f55e0 0x7fa80c106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:27:56.621 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.616+0000 7fa80b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa80c0f55e0 0x7fa80c106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57598/0 (socket says 172.21.15.27:57598) 2021-11-03T17:27:56.621 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.616+0000 7fa80b7fe700 1 -- 172.21.15.27:0/1797727127 learned_addr learned my addr 172.21.15.27:0/1797727127 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:27:56.621 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.616+0000 7fa80affd700 1 --2- 172.21.15.27:0/1797727127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa80c0f5ef0 0x7fa80c107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:27:56.621 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.616+0000 7fa80b7fe700 1 -- 172.21.15.27:0/1797727127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa80c0f5ef0 msgr2=0x7fa80c107460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:27:56.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.616+0000 7fa80b7fe700 1 --2- 172.21.15.27:0/1797727127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa80c0f5ef0 0x7fa80c107460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:56.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.616+0000 7fa80b7fe700 1 -- 172.21.15.27:0/1797727127 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa800005040 con 0x7fa80c0f55e0 2021-11-03T17:27:56.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.616+0000 7fa80b7fe700 1 --2- 172.21.15.27:0/1797727127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa80c0f55e0 0x7fa80c106f70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7fa7fc00c530 tx=0x7fa7fc007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:27:56.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.616+0000 7fa7f3fff700 1 -- 172.21.15.27:0/1797727127 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7fc00f650 con 0x7fa80c0f55e0 2021-11-03T17:27:56.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.616+0000 7fa7f3fff700 1 -- 172.21.15.27:0/1797727127 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fa7fc00fc60 con 0x7fa80c0f55e0 2021-11-03T17:27:56.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.616+0000 7fa811d9c700 1 -- 172.21.15.27:0/1797727127 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa80c1084f0 con 0x7fa80c0f55e0 2021-11-03T17:27:56.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.616+0000 7fa7f3fff700 1 -- 172.21.15.27:0/1797727127 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7fc012c20 con 0x7fa80c0f55e0 2021-11-03T17:27:56.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.616+0000 7fa811d9c700 1 -- 172.21.15.27:0/1797727127 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa80c108ab0 con 0x7fa80c0f55e0 2021-11-03T17:27:56.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.617+0000 7fa811d9c700 1 -- 172.21.15.27:0/1797727127 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa80c045ad0 con 0x7fa80c0f55e0 2021-11-03T17:27:56.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.618+0000 7fa7f3fff700 1 -- 172.21.15.27:0/1797727127 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fa7fc010070 con 0x7fa80c0f55e0 2021-11-03T17:27:56.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.618+0000 7fa7f3fff700 1 --2- 172.21.15.27:0/1797727127 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa7f40640b0 0x7fa7f4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:27:56.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.618+0000 7fa7f3fff700 1 -- 172.21.15.27:0/1797727127 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fa7fc0805d0 con 0x7fa80c0f55e0 2021-11-03T17:27:56.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.618+0000 7fa80affd700 1 --2- 172.21.15.27:0/1797727127 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa7f40640b0 0x7fa7f4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:27:56.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.619+0000 7fa80affd700 1 --2- 172.21.15.27:0/1797727127 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa7f40640b0 0x7fa7f4066570 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7fa800004630 tx=0x7fa800009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:27:56.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.622+0000 7fa7f3fff700 1 -- 172.21.15.27:0/1797727127 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fa7fc0528a0 con 0x7fa80c0f55e0 2021-11-03T17:27:56.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.825+0000 7fa811d9c700 1 -- 172.21.15.27:0/1797727127 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa80c10d590 con 0x7fa80c0f55e0 2021-11-03T17:27:56.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.825+0000 7fa7f3fff700 1 -- 172.21.15.27:0/1797727127 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7fa7fc052490 con 0x7fa80c0f55e0 2021-11-03T17:27:56.821 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:27:56.821 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:27:56.821 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:27:56.822 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:27:56.822 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:27:56.822 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:27:56.822 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:27:56.822 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:27:56.823 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:27:56.823 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:27:56.823 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:27:56.823 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:27:56.823 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:27:56.824 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:27:56.824 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:27:56.824 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:27:56.824 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:27:56.824 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:27:56.825 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:27:56.825 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:27:56.825 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:27:56.825 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:27:56.826 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:27:56.826 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:27:56.826 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:27:56.826 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:27:56.826 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:27:56.827 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:27:56.827 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:27:56.827 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:27:56.827 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:27:56.827 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:27:56.827 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:27:56.828 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:27:56.828 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:27:56.828 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:27:56.828 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:27:56.829 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:27:56.829 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:27:56.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.827+0000 7fa811d9c700 1 -- 172.21.15.27:0/1797727127 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa7f40640b0 msgr2=0x7fa7f4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:27:56.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.827+0000 7fa811d9c700 1 --2- 172.21.15.27:0/1797727127 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa7f40640b0 0x7fa7f4066570 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7fa800004630 tx=0x7fa800009b30).stop 2021-11-03T17:27:56.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.827+0000 7fa811d9c700 1 -- 172.21.15.27:0/1797727127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa80c0f55e0 msgr2=0x7fa80c106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:27:56.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.827+0000 7fa811d9c700 1 --2- 172.21.15.27:0/1797727127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa80c0f55e0 0x7fa80c106f70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7fa7fc00c530 tx=0x7fa7fc007e20).stop 2021-11-03T17:27:56.831 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.827+0000 7fa811d9c700 1 -- 172.21.15.27:0/1797727127 shutdown_connections 2021-11-03T17:27:56.831 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.827+0000 7fa811d9c700 1 --2- 172.21.15.27:0/1797727127 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa7f40640b0 0x7fa7f4066570 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:56.831 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.827+0000 7fa811d9c700 1 --2- 172.21.15.27:0/1797727127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa80c0f55e0 0x7fa80c106f70 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:56.831 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.827+0000 7fa811d9c700 1 --2- 172.21.15.27:0/1797727127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa80c0f5ef0 0x7fa80c107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:27:56.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.827+0000 7fa811d9c700 1 -- 172.21.15.27:0/1797727127 >> 172.21.15.27:0/1797727127 conn(0x7fa80c0f0de0 msgr2=0x7fa80c05f720 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:27:56.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.827+0000 7fa811d9c700 1 -- 172.21.15.27:0/1797727127 shutdown_connections 2021-11-03T17:27:56.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:27:56.827+0000 7fa811d9c700 1 -- 172.21.15.27:0/1797727127 wait complete. 2021-11-03T17:27:56.832 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:27:57.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:57 smithi115 ceph-mon[65453]: from='client.34520 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:27:57.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:57 smithi115 ceph-mon[65453]: pgmap v399: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:57.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:57 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3374390699' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:57.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:57 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1797727127' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:27:57.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:57 smithi027 ceph-mon[97244]: from='client.34520 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:27:57.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:57 smithi027 ceph-mon[97244]: pgmap v399: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:57.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:57 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3374390699' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:27:57.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:57 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1797727127' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:27:58.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:58 smithi115 ceph-mon[65453]: from='client.34524 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:27:58.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:58 smithi027 ceph-mon[97244]: from='client.34524 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:27:59.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:27:59 smithi115 ceph-mon[65453]: pgmap v400: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:27:59.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:27:59 smithi027 ceph-mon[97244]: pgmap v400: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:00.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:00 smithi115 ceph-mon[65453]: 1 slow requests, 1 included below; oldest blocked for > 722.376056 secs 2021-11-03T17:28:00.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:00 smithi115 ceph-mon[65453]: slow request 722.376055 seconds old, received at 2021-11-03T17:15:56.956064+0000: client_request(client.24331:21043 rmdir #0x10000003957/db1 2021-11-03T17:15:56.954824+0000 caller_uid=1000, caller_gid=1254{6,36,1000,1254,}) currently cleaned up request 2021-11-03T17:28:00.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:28:00.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:28:00.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:28:00.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:00.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:00.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:00.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:00.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:00 smithi027 ceph-mon[97244]: 1 slow requests, 1 included below; oldest blocked for > 722.376056 secs 2021-11-03T17:28:00.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:00 smithi027 ceph-mon[97244]: slow request 722.376055 seconds old, received at 2021-11-03T17:15:56.956064+0000: client_request(client.24331:21043 rmdir #0x10000003957/db1 2021-11-03T17:15:56.954824+0000 caller_uid=1000, caller_gid=1254{6,36,1000,1254,}) currently cleaned up request 2021-11-03T17:28:00.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:28:00.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:28:00.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:28:00.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:00.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:00.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:00.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:01.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:01 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:28:01.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:01 smithi115 ceph-mon[65453]: pgmap v401: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:01.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:01 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:28:01.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:01 smithi027 ceph-mon[97244]: pgmap v401: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:03.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:03 smithi027 ceph-mon[97244]: pgmap v402: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:03.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:03 smithi115 ceph-mon[65453]: pgmap v402: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:05.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:05 smithi115 ceph-mon[65453]: pgmap v403: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:05.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 727.376185 secs 2021-11-03T17:28:05.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:05 smithi027 ceph-mon[97244]: pgmap v403: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:05.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 727.376185 secs 2021-11-03T17:28:07.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:07 smithi115 ceph-mon[65453]: pgmap v404: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:07.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:07 smithi027 ceph-mon[97244]: pgmap v404: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:08.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:08 smithi115 ceph-mon[65453]: pgmap v405: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:08.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:08 smithi027 ceph-mon[97244]: pgmap v405: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:09.852 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:09 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 732.376310 secs 2021-11-03T17:28:10.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:09 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 732.376310 secs 2021-11-03T17:28:10.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:28:10.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:10 smithi115 ceph-mon[65453]: poll took 20057.251 ms: 1 events 2021-11-03T17:28:10.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:10 smithi115 ceph-mon[65453]: pgmap v406: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:10.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:28:10.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:28:10.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:28:10.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:10.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:10.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:10.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:11.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:28:11.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:10 smithi027 ceph-mon[97244]: poll took 20057.251 ms: 1 events 2021-11-03T17:28:11.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:10 smithi027 ceph-mon[97244]: pgmap v406: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:11.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:28:11.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:28:11.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:28:11.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:11.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:11.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:11.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:11.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:11 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:28:12.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:11 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:28:12.937 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:12 smithi027 ceph-mon[97244]: pgmap v407: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:12.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:12 smithi115 ceph-mon[65453]: pgmap v407: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:15.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:15 smithi027 ceph-mon[97244]: pgmap v408: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:15.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 737.376376 secs 2021-11-03T17:28:15.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:15 smithi115 ceph-mon[65453]: pgmap v408: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:15.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 737.376376 secs 2021-11-03T17:28:17.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:17 smithi027 ceph-mon[97244]: pgmap v409: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:17.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:17 smithi115 ceph-mon[65453]: pgmap v409: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:19.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:19 smithi027 ceph-mon[97244]: pgmap v410: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:19.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:19 smithi115 ceph-mon[65453]: pgmap v410: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:20.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 742.376517 secs 2021-11-03T17:28:20.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 742.376517 secs 2021-11-03T17:28:21.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:21 smithi027 ceph-mon[97244]: pgmap v411: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:21.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:28:21.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:21 smithi115 ceph-mon[65453]: pgmap v411: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:21.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:28:22.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:22 smithi027 ceph-mon[97244]: poll took 10038.464 ms: 1 events 2021-11-03T17:28:22.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:22 smithi115 ceph-mon[65453]: poll took 10038.464 ms: 1 events 2021-11-03T17:28:23.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:23 smithi027 ceph-mon[97244]: pgmap v412: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:23.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:23 smithi115 ceph-mon[65453]: pgmap v412: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:25.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:25 smithi115 ceph-mon[65453]: pgmap v413: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:25.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 747.376639 secs 2021-11-03T17:28:25.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:25 smithi027 ceph-mon[97244]: pgmap v413: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:25.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 747.376639 secs 2021-11-03T17:28:26.939 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:26.944+0000 7ff825ee8700 1 -- 172.21.15.27:0/3292712818 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8200f5340 msgr2=0x7ff8200f7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:28:26.939 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:26.944+0000 7ff825ee8700 1 --2- 172.21.15.27:0/3292712818 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8200f5340 0x7ff8200f7ac0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7ff814004660 tx=0x7ff814009b30).stop 2021-11-03T17:28:26.939 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:26.944+0000 7ff825ee8700 1 -- 172.21.15.27:0/3292712818 shutdown_connections 2021-11-03T17:28:26.940 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:26.944+0000 7ff825ee8700 1 --2- 172.21.15.27:0/3292712818 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8200f5340 0x7ff8200f7ac0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:26.940 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:26.944+0000 7ff825ee8700 1 --2- 172.21.15.27:0/3292712818 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8200f49a0 0x7ff8200f4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:28:26.940 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:26.944+0000 7ff825ee8700 1 -- 172.21.15.27:0/3292712818 >> 172.21.15.27:0/3292712818 conn(0x7ff8200f04c0 msgr2=0x7ff8200f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:28:26.940 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:26.944+0000 7ff825ee8700 1 -- 172.21.15.27:0/3292712818 shutdown_connections 2021-11-03T17:28:26.941 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:26.944+0000 7ff825ee8700 1 -- 172.21.15.27:0/3292712818 wait complete. 2021-11-03T17:28:26.941 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:26.945+0000 7ff825ee8700 1 Processor -- start 2021-11-03T17:28:26.941 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:26.945+0000 7ff825ee8700 1 -- start start 2021-11-03T17:28:26.945 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:26.945+0000 7ff825ee8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8200f49a0 0x7ff820106fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:28:26.946 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:26.945+0000 7ff825ee8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8200f5340 0x7ff8201074c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:28:26.947 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:26.945+0000 7ff825ee8700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff820107b00 con 0x7ff8200f49a0 2021-11-03T17:28:26.947 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:26.945+0000 7ff825ee8700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8201083b0 con 0x7ff8200f5340 2021-11-03T17:28:26.947 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:26.945+0000 7ff81f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8200f49a0 0x7ff820106fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:28:26.948 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:26.945+0000 7ff81f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8200f49a0 0x7ff820106fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57616/0 (socket says 172.21.15.27:57616) 2021-11-03T17:28:26.948 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:26.945+0000 7ff81f7fe700 1 -- 172.21.15.27:0/969306589 learned_addr learned my addr 172.21.15.27:0/969306589 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:28:26.949 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:26.946+0000 7ff81effd700 1 --2- 172.21.15.27:0/969306589 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8200f5340 0x7ff8201074c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:28:26.949 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:26.946+0000 7ff81f7fe700 1 -- 172.21.15.27:0/969306589 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8200f5340 msgr2=0x7ff8201074c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:28:26.950 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:26.946+0000 7ff81f7fe700 1 --2- 172.21.15.27:0/969306589 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8200f5340 0x7ff8201074c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:26.950 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:26.946+0000 7ff81f7fe700 1 -- 172.21.15.27:0/969306589 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff814005040 con 0x7ff8200f49a0 2021-11-03T17:28:26.951 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:26.946+0000 7ff81f7fe700 1 --2- 172.21.15.27:0/969306589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8200f49a0 0x7ff820106fd0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7ff81800b790 tx=0x7ff8180092b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:28:26.951 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:26.946+0000 7ff807fff700 1 -- 172.21.15.27:0/969306589 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff81800d700 con 0x7ff8200f49a0 2021-11-03T17:28:26.952 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:26.946+0000 7ff807fff700 1 -- 172.21.15.27:0/969306589 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7ff81800dd10 con 0x7ff8200f49a0 2021-11-03T17:28:26.952 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:26.946+0000 7ff825ee8700 1 -- 172.21.15.27:0/969306589 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff820108690 con 0x7ff8200f49a0 2021-11-03T17:28:26.953 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:26.946+0000 7ff807fff700 1 -- 172.21.15.27:0/969306589 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff818010e60 con 0x7ff8200f49a0 2021-11-03T17:28:26.953 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:26.946+0000 7ff825ee8700 1 -- 172.21.15.27:0/969306589 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff820108c50 con 0x7ff8200f49a0 2021-11-03T17:28:26.953 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:26.947+0000 7ff825ee8700 1 -- 172.21.15.27:0/969306589 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8200f4150 con 0x7ff8200f49a0 2021-11-03T17:28:26.954 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:26.947+0000 7ff807fff700 1 -- 172.21.15.27:0/969306589 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7ff818010670 con 0x7ff8200f49a0 2021-11-03T17:28:26.954 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:26.948+0000 7ff807fff700 1 --2- 172.21.15.27:0/969306589 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff8080640b0 0x7ff808066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:28:26.955 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:26.948+0000 7ff807fff700 1 -- 172.21.15.27:0/969306589 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7ff818080c40 con 0x7ff8200f49a0 2021-11-03T17:28:26.955 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:26.948+0000 7ff81effd700 1 --2- 172.21.15.27:0/969306589 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff8080640b0 0x7ff808066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:28:26.956 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:26.948+0000 7ff81effd700 1 --2- 172.21.15.27:0/969306589 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff8080640b0 0x7ff808066570 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7ff814009da0 tx=0x7ff814009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:28:26.957 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:26.952+0000 7ff807fff700 1 -- 172.21.15.27:0/969306589 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7ff818052e50 con 0x7ff8200f49a0 2021-11-03T17:28:26.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:26 smithi115 ceph-mon[65453]: poll took 5372.133 ms: 1 events 2021-11-03T17:28:26.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:28:26.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:28:26.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:28:26.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:26.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:26.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:26.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:26.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:26 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:28:26.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:26 smithi115 ceph-mon[65453]: pgmap v414: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:27.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:26 smithi027 ceph-mon[97244]: poll took 5372.133 ms: 1 events 2021-11-03T17:28:27.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:28:27.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:28:27.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:28:27.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:27.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:27.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:27.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:27.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:26 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:28:27.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:26 smithi027 ceph-mon[97244]: pgmap v414: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:27.124 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.130+0000 7ff825ee8700 1 -- 172.21.15.27:0/969306589 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff8200fefd0 con 0x7ff8080640b0 2021-11-03T17:28:27.125 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.131+0000 7ff807fff700 1 -- 172.21.15.27:0/969306589 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7ff8200fefd0 con 0x7ff8080640b0 2021-11-03T17:28:27.127 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.133+0000 7ff825ee8700 1 -- 172.21.15.27:0/969306589 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff8080640b0 msgr2=0x7ff808066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:28:27.128 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.133+0000 7ff825ee8700 1 --2- 172.21.15.27:0/969306589 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff8080640b0 0x7ff808066570 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7ff814009da0 tx=0x7ff814009b30).stop 2021-11-03T17:28:27.128 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.133+0000 7ff825ee8700 1 -- 172.21.15.27:0/969306589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8200f49a0 msgr2=0x7ff820106fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:28:27.128 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.133+0000 7ff825ee8700 1 --2- 172.21.15.27:0/969306589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8200f49a0 0x7ff820106fd0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7ff81800b790 tx=0x7ff8180092b0).stop 2021-11-03T17:28:27.129 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.133+0000 7ff825ee8700 1 -- 172.21.15.27:0/969306589 shutdown_connections 2021-11-03T17:28:27.129 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.133+0000 7ff825ee8700 1 --2- 172.21.15.27:0/969306589 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff8080640b0 0x7ff808066570 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:27.129 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.133+0000 7ff825ee8700 1 --2- 172.21.15.27:0/969306589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8200f49a0 0x7ff820106fd0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:27.129 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.133+0000 7ff825ee8700 1 --2- 172.21.15.27:0/969306589 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8200f5340 0x7ff8201074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:27.129 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.133+0000 7ff825ee8700 1 -- 172.21.15.27:0/969306589 >> 172.21.15.27:0/969306589 conn(0x7ff8200f04c0 msgr2=0x7ff8200fb510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:28:27.130 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.133+0000 7ff825ee8700 1 -- 172.21.15.27:0/969306589 shutdown_connections 2021-11-03T17:28:27.130 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.133+0000 7ff825ee8700 1 -- 172.21.15.27:0/969306589 wait complete. 2021-11-03T17:28:27.137 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:28:27.230 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.233+0000 7f5cd6a06700 1 -- 172.21.15.27:0/3458949026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cd00f6550 msgr2=0x7f5cd00f69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:28:27.231 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.233+0000 7f5cd6a06700 1 --2- 172.21.15.27:0/3458949026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cd00f6550 0x7f5cd00f69b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7f5cc4004660 tx=0x7f5cc4009b30).stop 2021-11-03T17:28:27.231 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.234+0000 7f5cd6a06700 1 -- 172.21.15.27:0/3458949026 shutdown_connections 2021-11-03T17:28:27.231 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.234+0000 7f5cd6a06700 1 --2- 172.21.15.27:0/3458949026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cd00f6550 0x7f5cd00f69b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:27.232 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.234+0000 7f5cd6a06700 1 --2- 172.21.15.27:0/3458949026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cd00f5bb0 0x7f5cd00f5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:27.232 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.234+0000 7f5cd6a06700 1 -- 172.21.15.27:0/3458949026 >> 172.21.15.27:0/3458949026 conn(0x7f5cd00f04c0 msgr2=0x7f5cd00f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:28:27.232 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.234+0000 7f5cd6a06700 1 -- 172.21.15.27:0/3458949026 shutdown_connections 2021-11-03T17:28:27.232 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.234+0000 7f5cd6a06700 1 -- 172.21.15.27:0/3458949026 wait complete. 2021-11-03T17:28:27.232 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.234+0000 7f5cd6a06700 1 Processor -- start 2021-11-03T17:28:27.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.234+0000 7f5cd6a06700 1 -- start start 2021-11-03T17:28:27.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.234+0000 7f5cd6a06700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cd00f5bb0 0x7f5cd0109c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:28:27.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.234+0000 7f5cd6a06700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cd00f6550 0x7f5cd010c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:28:27.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.234+0000 7f5cd6a06700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd00f56a0 con 0x7f5cd00f5bb0 2021-11-03T17:28:27.234 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.234+0000 7f5cd6a06700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cd00f57e0 con 0x7f5cd00f6550 2021-11-03T17:28:27.234 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.235+0000 7f5ccffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cd00f5bb0 0x7f5cd0109c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:28:27.234 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.235+0000 7f5ccffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cd00f5bb0 0x7f5cd0109c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57626/0 (socket says 172.21.15.27:57626) 2021-11-03T17:28:27.234 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.235+0000 7f5ccffff700 1 -- 172.21.15.27:0/2056621713 learned_addr learned my addr 172.21.15.27:0/2056621713 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:28:27.235 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.235+0000 7f5ccf7fe700 1 --2- 172.21.15.27:0/2056621713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cd00f6550 0x7f5cd010c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:28:27.235 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.235+0000 7f5ccffff700 1 -- 172.21.15.27:0/2056621713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cd00f6550 msgr2=0x7f5cd010c140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:28:27.235 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.235+0000 7f5ccffff700 1 --2- 172.21.15.27:0/2056621713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cd00f6550 0x7f5cd010c140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:27.235 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.235+0000 7f5ccffff700 1 -- 172.21.15.27:0/2056621713 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cc4005040 con 0x7f5cd00f5bb0 2021-11-03T17:28:27.236 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.235+0000 7f5ccf7fe700 1 --2- 172.21.15.27:0/2056621713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cd00f6550 0x7f5cd010c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:28:27.236 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.235+0000 7f5ccffff700 1 --2- 172.21.15.27:0/2056621713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cd00f5bb0 0x7f5cd0109c40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f5cc000c790 tx=0x7f5cc0007d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:28:27.236 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.235+0000 7f5cccff9700 1 -- 172.21.15.27:0/2056621713 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cc000e680 con 0x7f5cd00f5bb0 2021-11-03T17:28:27.236 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.235+0000 7f5cd6a06700 1 -- 172.21.15.27:0/2056621713 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cd010c6f0 con 0x7f5cd00f5bb0 2021-11-03T17:28:27.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.235+0000 7f5cccff9700 1 -- 172.21.15.27:0/2056621713 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f5cc000ec90 con 0x7f5cd00f5bb0 2021-11-03T17:28:27.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.235+0000 7f5cccff9700 1 -- 172.21.15.27:0/2056621713 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cc0012d00 con 0x7f5cd00f5bb0 2021-11-03T17:28:27.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.236+0000 7f5cd6a06700 1 -- 172.21.15.27:0/2056621713 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cd010ccb0 con 0x7f5cd00f5bb0 2021-11-03T17:28:27.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.236+0000 7f5cd6a06700 1 -- 172.21.15.27:0/2056621713 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cd0045ad0 con 0x7f5cd00f5bb0 2021-11-03T17:28:27.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.237+0000 7f5cccff9700 1 -- 172.21.15.27:0/2056621713 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f5cc0010070 con 0x7f5cd00f5bb0 2021-11-03T17:28:27.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.237+0000 7f5cccff9700 1 --2- 172.21.15.27:0/2056621713 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5cb80640b0 0x7f5cb8066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:28:27.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.237+0000 7f5cccff9700 1 -- 172.21.15.27:0/2056621713 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f5cc00806a0 con 0x7f5cd00f5bb0 2021-11-03T17:28:27.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.237+0000 7f5ccf7fe700 1 --2- 172.21.15.27:0/2056621713 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5cb80640b0 0x7f5cb8066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:28:27.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.237+0000 7f5ccf7fe700 1 --2- 172.21.15.27:0/2056621713 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5cb80640b0 0x7f5cb8066570 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f5cc4004e30 tx=0x7f5cc4009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:28:27.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.241+0000 7f5cccff9700 1 -- 172.21.15.27:0/2056621713 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f5cc00528b0 con 0x7f5cd00f5bb0 2021-11-03T17:28:27.407 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.413+0000 7f5cd6a06700 1 -- 172.21.15.27:0/2056621713 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5cd00fdf10 con 0x7f5cb80640b0 2021-11-03T17:28:27.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.428+0000 7f5cccff9700 1 -- 172.21.15.27:0/2056621713 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f5cd00fdf10 con 0x7f5cb80640b0 2021-11-03T17:28:27.422 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:28:27.422 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (20m) 1s ago 21m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:28:27.423 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (11m) 1s ago 21m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:28:27.423 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (11m) 8m ago 20m 7239k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:28:27.423 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (19m) 1s ago 20m 40.2M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:28:27.423 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (13m) 1s ago 16m 18.2M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:28:27.424 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (16m) 1s ago 16m 3841M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:28:27.424 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (12m) 8m ago 16m 14.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:28:27.424 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (13m) 8m ago 16m 16.3M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:28:27.425 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (13m) 1s ago 22m 497M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:28:27.425 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (13m) 8m ago 20m 412M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:28:27.425 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (12m) 1s ago 22m 119M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:28:27.425 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (12m) 8m ago 20m 69.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:28:27.425 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (20m) 1s ago 20m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:28:27.426 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (20m) 8m ago 20m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:28:27.426 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (11m) 1s ago 19m 290M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:28:27.426 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (10m) 1s ago 19m 317M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:28:27.426 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (10m) 1s ago 18m 229M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:28:27.427 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (9m) 8m ago 18m 289M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:28:27.427 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (9m) 8m ago 18m 235M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:28:27.427 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (8m) 8m ago 17m 11.5M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:28:27.428 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (19m) 1s ago 20m 48.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:28:27.428 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.429+0000 7f5cd6a06700 1 -- 172.21.15.27:0/2056621713 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5cb80640b0 msgr2=0x7f5cb8066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:28:27.429 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.429+0000 7f5cd6a06700 1 --2- 172.21.15.27:0/2056621713 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5cb80640b0 0x7f5cb8066570 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f5cc4004e30 tx=0x7f5cc4009b30).stop 2021-11-03T17:28:27.429 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.429+0000 7f5cd6a06700 1 -- 172.21.15.27:0/2056621713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cd00f5bb0 msgr2=0x7f5cd0109c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:28:27.429 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.429+0000 7f5cd6a06700 1 --2- 172.21.15.27:0/2056621713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cd00f5bb0 0x7f5cd0109c40 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f5cc000c790 tx=0x7f5cc0007d90).stop 2021-11-03T17:28:27.431 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.429+0000 7f5cd6a06700 1 -- 172.21.15.27:0/2056621713 shutdown_connections 2021-11-03T17:28:27.431 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.429+0000 7f5cd6a06700 1 --2- 172.21.15.27:0/2056621713 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5cb80640b0 0x7f5cb8066570 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:27.431 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.429+0000 7f5cd6a06700 1 --2- 172.21.15.27:0/2056621713 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5cd00f5bb0 0x7f5cd0109c40 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:27.431 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.429+0000 7f5cd6a06700 1 --2- 172.21.15.27:0/2056621713 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5cd00f6550 0x7f5cd010c140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:27.432 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.430+0000 7f5cd6a06700 1 -- 172.21.15.27:0/2056621713 >> 172.21.15.27:0/2056621713 conn(0x7f5cd00f04c0 msgr2=0x7f5cd00fa450 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:28:27.432 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.430+0000 7f5cd6a06700 1 -- 172.21.15.27:0/2056621713 shutdown_connections 2021-11-03T17:28:27.432 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.430+0000 7f5cd6a06700 1 -- 172.21.15.27:0/2056621713 wait complete. 2021-11-03T17:28:27.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.528+0000 7f45c537b700 1 -- 172.21.15.27:0/2321193314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45c0060310 msgr2=0x7f45c00f6f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:28:27.526 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.528+0000 7f45c537b700 1 --2- 172.21.15.27:0/2321193314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45c0060310 0x7f45c00f6f30 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f45b0004660 tx=0x7f45b0009b30).stop 2021-11-03T17:28:27.526 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.528+0000 7f45c537b700 1 -- 172.21.15.27:0/2321193314 shutdown_connections 2021-11-03T17:28:27.526 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.528+0000 7f45c537b700 1 --2- 172.21.15.27:0/2321193314 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45c00f74b0 0x7f45c00f9940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:27.526 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.528+0000 7f45c537b700 1 --2- 172.21.15.27:0/2321193314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45c0060310 0x7f45c00f6f30 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:27.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.528+0000 7f45c537b700 1 -- 172.21.15.27:0/2321193314 >> 172.21.15.27:0/2321193314 conn(0x7f45c00f0bc0 msgr2=0x7f45c00f2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:28:27.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.529+0000 7f45c537b700 1 -- 172.21.15.27:0/2321193314 shutdown_connections 2021-11-03T17:28:27.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.529+0000 7f45c537b700 1 -- 172.21.15.27:0/2321193314 wait complete. 2021-11-03T17:28:27.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.529+0000 7f45c537b700 1 Processor -- start 2021-11-03T17:28:27.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.529+0000 7f45c537b700 1 -- start start 2021-11-03T17:28:27.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.529+0000 7f45c537b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45c0060310 0x7f45c010d380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:28:27.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.529+0000 7f45c537b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45c00f74b0 0x7f45c010d870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:28:27.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.529+0000 7f45c537b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c010de20 con 0x7f45c00f74b0 2021-11-03T17:28:27.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.529+0000 7f45c537b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c010df60 con 0x7f45c0060310 2021-11-03T17:28:27.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.530+0000 7f45be7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45c00f74b0 0x7f45c010d870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:28:27.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.530+0000 7f45be7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45c00f74b0 0x7f45c010d870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57638/0 (socket says 172.21.15.27:57638) 2021-11-03T17:28:27.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.530+0000 7f45be7fc700 1 -- 172.21.15.27:0/2582938432 learned_addr learned my addr 172.21.15.27:0/2582938432 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:28:27.530 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.530+0000 7f45beffd700 1 --2- 172.21.15.27:0/2582938432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45c0060310 0x7f45c010d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:28:27.530 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.530+0000 7f45be7fc700 1 -- 172.21.15.27:0/2582938432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45c0060310 msgr2=0x7f45c010d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:28:27.530 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.530+0000 7f45be7fc700 1 --2- 172.21.15.27:0/2582938432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45c0060310 0x7f45c010d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:27.530 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.530+0000 7f45be7fc700 1 -- 172.21.15.27:0/2582938432 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45b0005040 con 0x7f45c00f74b0 2021-11-03T17:28:27.530 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.530+0000 7f45be7fc700 1 --2- 172.21.15.27:0/2582938432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45c00f74b0 0x7f45c010d870 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f45b400c790 tx=0x7f45b4007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:28:27.531 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.530+0000 7f45a77fe700 1 -- 172.21.15.27:0/2582938432 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45b400e680 con 0x7f45c00f74b0 2021-11-03T17:28:27.531 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.530+0000 7f45a77fe700 1 -- 172.21.15.27:0/2582938432 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f45b400ec90 con 0x7f45c00f74b0 2021-11-03T17:28:27.531 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.530+0000 7f45c537b700 1 -- 172.21.15.27:0/2582938432 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45c010e900 con 0x7f45c00f74b0 2021-11-03T17:28:27.531 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.530+0000 7f45a77fe700 1 -- 172.21.15.27:0/2582938432 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45b4013d60 con 0x7f45c00f74b0 2021-11-03T17:28:27.532 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.530+0000 7f45c537b700 1 -- 172.21.15.27:0/2582938432 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45c010eec0 con 0x7f45c00f74b0 2021-11-03T17:28:27.532 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.531+0000 7f45c537b700 1 -- 172.21.15.27:0/2582938432 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45c0045ad0 con 0x7f45c00f74b0 2021-11-03T17:28:27.533 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.532+0000 7f45a77fe700 1 -- 172.21.15.27:0/2582938432 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f45b401a060 con 0x7f45c00f74b0 2021-11-03T17:28:27.533 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.532+0000 7f45a77fe700 1 --2- 172.21.15.27:0/2582938432 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f45a8064100 0x7f45a80665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:28:27.534 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.532+0000 7f45a77fe700 1 -- 172.21.15.27:0/2582938432 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f45b4080b80 con 0x7f45c00f74b0 2021-11-03T17:28:27.534 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.532+0000 7f45beffd700 1 --2- 172.21.15.27:0/2582938432 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f45a8064100 0x7f45a80665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:28:27.534 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.533+0000 7f45beffd700 1 --2- 172.21.15.27:0/2582938432 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f45a8064100 0x7f45a80665c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f45b0004e30 tx=0x7f45b0009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:28:27.534 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.536+0000 7f45a77fe700 1 -- 172.21.15.27:0/2582938432 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f45b4052d90 con 0x7f45c00f74b0 2021-11-03T17:28:27.759 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.765+0000 7f45c537b700 1 -- 172.21.15.27:0/2582938432 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f45c0027a00 con 0x7f45c00f74b0 2021-11-03T17:28:27.760 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.765+0000 7f45a77fe700 1 -- 172.21.15.27:0/2582938432 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f45b4018070 con 0x7f45c00f74b0 2021-11-03T17:28:27.760 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:28:27.760 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:28:27.761 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:28:27.761 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:28:27.761 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:28:27.762 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:28:27.762 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:28:27.762 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:28:27.762 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:28:27.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:28:27.763 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:28:27.763 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:28:27.763 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:28:27.764 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:28:27.764 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:28:27.764 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:28:27.764 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:28:27.765 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:28:27.765 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.767+0000 7f45c537b700 1 -- 172.21.15.27:0/2582938432 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f45a8064100 msgr2=0x7f45a80665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:28:27.766 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.767+0000 7f45c537b700 1 --2- 172.21.15.27:0/2582938432 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f45a8064100 0x7f45a80665c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f45b0004e30 tx=0x7f45b0009b30).stop 2021-11-03T17:28:27.766 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.767+0000 7f45c537b700 1 -- 172.21.15.27:0/2582938432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45c00f74b0 msgr2=0x7f45c010d870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:28:27.766 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.767+0000 7f45c537b700 1 --2- 172.21.15.27:0/2582938432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45c00f74b0 0x7f45c010d870 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7f45b400c790 tx=0x7f45b4007d70).stop 2021-11-03T17:28:27.766 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.767+0000 7f45c537b700 1 -- 172.21.15.27:0/2582938432 shutdown_connections 2021-11-03T17:28:27.767 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.767+0000 7f45c537b700 1 --2- 172.21.15.27:0/2582938432 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f45a8064100 0x7f45a80665c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:27.767 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.767+0000 7f45c537b700 1 --2- 172.21.15.27:0/2582938432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45c0060310 0x7f45c010d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:27.767 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.767+0000 7f45c537b700 1 --2- 172.21.15.27:0/2582938432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45c00f74b0 0x7f45c010d870 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:27.767 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.767+0000 7f45c537b700 1 -- 172.21.15.27:0/2582938432 >> 172.21.15.27:0/2582938432 conn(0x7f45c00f0bc0 msgr2=0x7f45c00f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:28:27.768 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.767+0000 7f45c537b700 1 -- 172.21.15.27:0/2582938432 shutdown_connections 2021-11-03T17:28:27.768 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.767+0000 7f45c537b700 1 -- 172.21.15.27:0/2582938432 wait complete. 2021-11-03T17:28:27.865 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.868+0000 7f75f195e700 1 -- 172.21.15.27:0/3276780444 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75ec06b9c0 msgr2=0x7f75ec069ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:28:27.865 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.868+0000 7f75f195e700 1 --2- 172.21.15.27:0/3276780444 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75ec06b9c0 0x7f75ec069ec0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f75e0004660 tx=0x7f75e0009b30).stop 2021-11-03T17:28:27.865 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.868+0000 7f75f195e700 1 -- 172.21.15.27:0/3276780444 shutdown_connections 2021-11-03T17:28:27.866 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.868+0000 7f75f195e700 1 --2- 172.21.15.27:0/3276780444 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75ec06a3b0 0x7f75ec06a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:27.866 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.868+0000 7f75f195e700 1 --2- 172.21.15.27:0/3276780444 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75ec06b9c0 0x7f75ec069ec0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:27.866 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.868+0000 7f75f195e700 1 -- 172.21.15.27:0/3276780444 >> 172.21.15.27:0/3276780444 conn(0x7f75ec0f1cd0 msgr2=0x7f75ec0f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:28:27.866 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.868+0000 7f75f195e700 1 -- 172.21.15.27:0/3276780444 shutdown_connections 2021-11-03T17:28:27.867 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.868+0000 7f75f195e700 1 -- 172.21.15.27:0/3276780444 wait complete. 2021-11-03T17:28:27.867 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.869+0000 7f75f195e700 1 Processor -- start 2021-11-03T17:28:27.867 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.869+0000 7f75f195e700 1 -- start start 2021-11-03T17:28:27.867 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.869+0000 7f75f195e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75ec06a3b0 0x7f75ec10d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:28:27.867 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.869+0000 7f75f195e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75ec06b9c0 0x7f75ec10d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:28:27.868 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.869+0000 7f75f195e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec10ddc0 con 0x7f75ec06a3b0 2021-11-03T17:28:27.868 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.869+0000 7f75f195e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec10df00 con 0x7f75ec06b9c0 2021-11-03T17:28:27.868 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.869+0000 7f75f095c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75ec06a3b0 0x7f75ec10d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:28:27.868 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.869+0000 7f75f095c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75ec06a3b0 0x7f75ec10d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57646/0 (socket says 172.21.15.27:57646) 2021-11-03T17:28:27.869 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.869+0000 7f75f095c700 1 -- 172.21.15.27:0/338871983 learned_addr learned my addr 172.21.15.27:0/338871983 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:28:27.869 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.870+0000 7f75ebfff700 1 --2- 172.21.15.27:0/338871983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75ec06b9c0 0x7f75ec10d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:28:27.869 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.870+0000 7f75f095c700 1 -- 172.21.15.27:0/338871983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75ec06b9c0 msgr2=0x7f75ec10d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:28:27.869 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.870+0000 7f75f095c700 1 --2- 172.21.15.27:0/338871983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75ec06b9c0 0x7f75ec10d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:27.869 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.870+0000 7f75f095c700 1 -- 172.21.15.27:0/338871983 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e0005040 con 0x7f75ec06a3b0 2021-11-03T17:28:27.870 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.870+0000 7f75ebfff700 1 --2- 172.21.15.27:0/338871983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75ec06b9c0 0x7f75ec10d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:28:27.870 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.870+0000 7f75f095c700 1 --2- 172.21.15.27:0/338871983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75ec06a3b0 0x7f75ec10d320 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f75e0009990 tx=0x7f75e0009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:28:27.870 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.870+0000 7f75e97fa700 1 -- 172.21.15.27:0/338871983 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75e0018070 con 0x7f75ec06a3b0 2021-11-03T17:28:27.870 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.870+0000 7f75e97fa700 1 -- 172.21.15.27:0/338871983 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f75e0007ba0 con 0x7f75ec06a3b0 2021-11-03T17:28:27.871 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.870+0000 7f75f195e700 1 -- 172.21.15.27:0/338871983 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75ec10e840 con 0x7f75ec06a3b0 2021-11-03T17:28:27.871 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.870+0000 7f75e97fa700 1 -- 172.21.15.27:0/338871983 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75e001c3c0 con 0x7f75ec06a3b0 2021-11-03T17:28:27.871 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.870+0000 7f75f195e700 1 -- 172.21.15.27:0/338871983 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75ec10eda0 con 0x7f75ec06a3b0 2021-11-03T17:28:27.872 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.871+0000 7f75f195e700 1 -- 172.21.15.27:0/338871983 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75ec045ad0 con 0x7f75ec06a3b0 2021-11-03T17:28:27.872 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.871+0000 7f75e97fa700 1 -- 172.21.15.27:0/338871983 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f75e001d420 con 0x7f75ec06a3b0 2021-11-03T17:28:27.873 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.872+0000 7f75e97fa700 1 --2- 172.21.15.27:0/338871983 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f75dc064100 0x7f75dc0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:28:27.873 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.872+0000 7f75e97fa700 1 -- 172.21.15.27:0/338871983 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f75e0082630 con 0x7f75ec06a3b0 2021-11-03T17:28:27.873 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.872+0000 7f75ebfff700 1 --2- 172.21.15.27:0/338871983 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f75dc064100 0x7f75dc0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:28:27.873 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.872+0000 7f75ebfff700 1 --2- 172.21.15.27:0/338871983 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f75dc064100 0x7f75dc0665c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f75d8009740 tx=0x7f75d8009be0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:28:27.874 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:27.876+0000 7f75e97fa700 1 -- 172.21.15.27:0/338871983 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f75e0054990 con 0x7f75ec06a3b0 2021-11-03T17:28:28.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:28.081+0000 7f75f195e700 1 -- 172.21.15.27:0/338871983 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f75ec1053f0 con 0x7f75ec06a3b0 2021-11-03T17:28:28.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:28.081+0000 7f75e97fa700 1 -- 172.21.15.27:0/338871983 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f75e0054580 con 0x7f75ec06a3b0 2021-11-03T17:28:28.076 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:28:28.076 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:28:28.076 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:28:28.076 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:28:28.077 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:28:28.077 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:28:28.077 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:28:28.077 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:28:28.078 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:28:28.078 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:28:28.078 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:28:28.078 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:28:28.079 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:28:28.079 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:28:28.079 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:28:28.079 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:28:28.080 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:28:28.080 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:28:28.080 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:28:28.080 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:28:28.081 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:28:28.081 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:28:28.081 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:28:28.082 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:28:28.082 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:28:28.083 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:28:28.083 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:28:28.083 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:28:28.083 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:28:28.084 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:28:28.084 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:28:28.084 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:28:28.084 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:28:28.085 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:28:28.085 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:28:28.085 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:28:28.086 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:28:28.086 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:28:28.086 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:28:28.087 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:28.083+0000 7f75f195e700 1 -- 172.21.15.27:0/338871983 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f75dc064100 msgr2=0x7f75dc0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:28:28.087 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:28.083+0000 7f75f195e700 1 --2- 172.21.15.27:0/338871983 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f75dc064100 0x7f75dc0665c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f75d8009740 tx=0x7f75d8009be0).stop 2021-11-03T17:28:28.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:28.083+0000 7f75f195e700 1 -- 172.21.15.27:0/338871983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75ec06a3b0 msgr2=0x7f75ec10d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:28:28.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:28.083+0000 7f75f195e700 1 --2- 172.21.15.27:0/338871983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75ec06a3b0 0x7f75ec10d320 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f75e0009990 tx=0x7f75e0009b30).stop 2021-11-03T17:28:28.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:28.083+0000 7f75f195e700 1 -- 172.21.15.27:0/338871983 shutdown_connections 2021-11-03T17:28:28.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:28.083+0000 7f75f195e700 1 --2- 172.21.15.27:0/338871983 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f75dc064100 0x7f75dc0665c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:28.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:28.083+0000 7f75f195e700 1 --2- 172.21.15.27:0/338871983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75ec06a3b0 0x7f75ec10d320 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:28.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:28.083+0000 7f75f195e700 1 --2- 172.21.15.27:0/338871983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75ec06b9c0 0x7f75ec10d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:28.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:28.083+0000 7f75f195e700 1 -- 172.21.15.27:0/338871983 >> 172.21.15.27:0/338871983 conn(0x7f75ec0f1cd0 msgr2=0x7f75ec0f8820 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:28:28.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:28.083+0000 7f75f195e700 1 -- 172.21.15.27:0/338871983 shutdown_connections 2021-11-03T17:28:28.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:28.083+0000 7f75f195e700 1 -- 172.21.15.27:0/338871983 wait complete. 2021-11-03T17:28:28.090 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:28:28.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:27 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2582938432' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:28.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:27 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2582938432' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:29.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:28 smithi115 ceph-mon[65453]: from='client.34536 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:28:29.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:28 smithi115 ceph-mon[65453]: from='client.34540 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:28:29.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:28 smithi115 ceph-mon[65453]: pgmap v415: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:29.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:28 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/338871983' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:28:29.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:28 smithi027 ceph-mon[97244]: from='client.34536 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:28:29.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:28 smithi027 ceph-mon[97244]: from='client.34540 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:28:29.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:28 smithi027 ceph-mon[97244]: pgmap v415: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:29.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:28 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/338871983' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:28:30.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:29 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 752.376747 secs 2021-11-03T17:28:30.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:29 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 752.376747 secs 2021-11-03T17:28:31.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:30 smithi115 ceph-mon[65453]: pgmap v416: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:31.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:30 smithi027 ceph-mon[97244]: pgmap v416: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:33.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:33 smithi027 ceph-mon[97244]: pgmap v417: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:33.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:33 smithi115 ceph-mon[65453]: pgmap v417: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:35.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:35 smithi027 ceph-mon[97244]: pgmap v418: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:35.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 757.376859 secs 2021-11-03T17:28:35.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:35 smithi115 ceph-mon[65453]: pgmap v418: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:35.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 757.376859 secs 2021-11-03T17:28:36.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:28:36.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:28:36.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:28:36.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:36.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:36.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:36.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:36.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:28:36.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:28:36.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:28:36.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:36.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:36.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:36.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:37.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:37 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:28:37.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:37 smithi115 ceph-mon[65453]: pgmap v419: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:37.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:37 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:28:37.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:37 smithi027 ceph-mon[97244]: pgmap v419: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:39.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:39 smithi115 ceph-mon[65453]: pgmap v420: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:39.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:39 smithi027 ceph-mon[97244]: pgmap v420: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:40.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 762.376984 secs 2021-11-03T17:28:40.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 762.376984 secs 2021-11-03T17:28:41.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:41 smithi115 ceph-mon[65453]: pgmap v421: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:41.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:41 smithi027 ceph-mon[97244]: pgmap v421: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:43.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:43 smithi027 ceph-mon[97244]: pgmap v422: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:43.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:43 smithi115 ceph-mon[65453]: pgmap v422: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:45.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:45 smithi115 ceph-mon[65453]: pgmap v423: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:45.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 767.377099 secs 2021-11-03T17:28:45.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:45 smithi027 ceph-mon[97244]: pgmap v423: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:45.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 767.377099 secs 2021-11-03T17:28:46.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:28:46.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:28:46.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:28:46.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:46.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:46.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:46.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:46.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:28:46.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:28:46.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:28:46.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:46.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:46.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:46.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:47.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:47 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:28:47.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:47 smithi115 ceph-mon[65453]: pgmap v424: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:47.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:47 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:28:47.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:47 smithi027 ceph-mon[97244]: pgmap v424: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:49.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:49 smithi115 ceph-mon[65453]: pgmap v425: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:49.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:49 smithi027 ceph-mon[97244]: pgmap v425: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:50.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 772.377219 secs 2021-11-03T17:28:50.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 772.377219 secs 2021-11-03T17:28:51.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:51 smithi115 ceph-mon[65453]: pgmap v426: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:51.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:51 smithi027 ceph-mon[97244]: pgmap v426: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:53.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:53 smithi027 ceph-mon[97244]: pgmap v427: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:53.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:53 smithi115 ceph-mon[65453]: pgmap v427: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:55.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:55 smithi115 ceph-mon[65453]: pgmap v428: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:55.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 777.377350 secs 2021-11-03T17:28:55.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:55 smithi027 ceph-mon[97244]: pgmap v428: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:55.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 777.377350 secs 2021-11-03T17:28:56.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:28:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:28:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:28:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:28:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:28:56.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:28:57.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:57 smithi115 ceph-mon[65453]: poll took 30095.152 ms: 1 events 2021-11-03T17:28:57.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:57 smithi115 ceph-mon[65453]: pgmap v429: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:57.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:28:57.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:28:57.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:57.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:57.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:57.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:57.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:57 smithi027 ceph-mon[97244]: poll took 30095.152 ms: 1 events 2021-11-03T17:28:57.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:57 smithi027 ceph-mon[97244]: pgmap v429: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:57.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:28:57.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:28:57.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:57.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:57.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:57.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:58.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.199+0000 7f85154d1700 1 -- 172.21.15.27:0/143061160 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85100f55e0 msgr2=0x7f85100f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:28:58.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.199+0000 7f85154d1700 1 --2- 172.21.15.27:0/143061160 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85100f55e0 0x7f85100f5a00 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f8500004660 tx=0x7f8500009b30).stop 2021-11-03T17:28:58.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.200+0000 7f85154d1700 1 -- 172.21.15.27:0/143061160 shutdown_connections 2021-11-03T17:28:58.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.200+0000 7f85154d1700 1 --2- 172.21.15.27:0/143061160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85100f5ef0 0x7f85100f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:58.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.200+0000 7f85154d1700 1 --2- 172.21.15.27:0/143061160 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85100f55e0 0x7f85100f5a00 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:58.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.200+0000 7f85154d1700 1 -- 172.21.15.27:0/143061160 >> 172.21.15.27:0/143061160 conn(0x7f85100f0de0 msgr2=0x7f85100f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:28:58.196 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.200+0000 7f85154d1700 1 -- 172.21.15.27:0/143061160 shutdown_connections 2021-11-03T17:28:58.196 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.200+0000 7f85154d1700 1 -- 172.21.15.27:0/143061160 wait complete. 2021-11-03T17:28:58.198 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.200+0000 7f85154d1700 1 Processor -- start 2021-11-03T17:28:58.198 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.201+0000 7f85154d1700 1 -- start start 2021-11-03T17:28:58.198 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.201+0000 7f85154d1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85100f55e0 0x7f851010bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:28:58.199 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.201+0000 7f85154d1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85100f5ef0 0x7f851010e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:28:58.199 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.201+0000 7f85154d1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8510107710 con 0x7f85100f55e0 2021-11-03T17:28:58.199 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.201+0000 7f85154d1700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f851010e650 con 0x7f85100f5ef0 2021-11-03T17:28:58.199 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.201+0000 7f850effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85100f55e0 0x7f851010bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:28:58.200 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.201+0000 7f850effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85100f55e0 0x7f851010bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57668/0 (socket says 172.21.15.27:57668) 2021-11-03T17:28:58.200 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.201+0000 7f850effd700 1 -- 172.21.15.27:0/2303232072 learned_addr learned my addr 172.21.15.27:0/2303232072 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:28:58.200 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.201+0000 7f850e7fc700 1 --2- 172.21.15.27:0/2303232072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85100f5ef0 0x7f851010e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:28:58.200 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.202+0000 7f850effd700 1 -- 172.21.15.27:0/2303232072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85100f5ef0 msgr2=0x7f851010e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:28:58.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.202+0000 7f850effd700 1 --2- 172.21.15.27:0/2303232072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85100f5ef0 0x7f851010e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:58.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.202+0000 7f850effd700 1 -- 172.21.15.27:0/2303232072 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8500005040 con 0x7f85100f55e0 2021-11-03T17:28:58.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.202+0000 7f850e7fc700 1 --2- 172.21.15.27:0/2303232072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85100f5ef0 0x7f851010e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:28:58.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.202+0000 7f850effd700 1 --2- 172.21.15.27:0/2303232072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85100f55e0 0x7f851010bc60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f8500009990 tx=0x7f8500009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:28:58.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.202+0000 7f84f77fe700 1 -- 172.21.15.27:0/2303232072 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8500018070 con 0x7f85100f55e0 2021-11-03T17:28:58.202 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.202+0000 7f85154d1700 1 -- 172.21.15.27:0/2303232072 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f851010e8d0 con 0x7f85100f55e0 2021-11-03T17:28:58.202 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.202+0000 7f84f77fe700 1 -- 172.21.15.27:0/2303232072 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f8500007ca0 con 0x7f85100f55e0 2021-11-03T17:28:58.202 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.202+0000 7f84f77fe700 1 -- 172.21.15.27:0/2303232072 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f850001c620 con 0x7f85100f55e0 2021-11-03T17:28:58.202 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.202+0000 7f85154d1700 1 -- 172.21.15.27:0/2303232072 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f851010ee30 con 0x7f85100f55e0 2021-11-03T17:28:58.203 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.203+0000 7f85154d1700 1 -- 172.21.15.27:0/2303232072 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8510045ad0 con 0x7f85100f55e0 2021-11-03T17:28:58.203 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.204+0000 7f84f77fe700 1 -- 172.21.15.27:0/2303232072 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f850001d960 con 0x7f85100f55e0 2021-11-03T17:28:58.203 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.204+0000 7f84f77fe700 1 --2- 172.21.15.27:0/2303232072 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f84f80640b0 0x7f84f8066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:28:58.203 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.204+0000 7f84f77fe700 1 -- 172.21.15.27:0/2303232072 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f8500081f60 con 0x7f85100f55e0 2021-11-03T17:28:58.203 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.204+0000 7f850e7fc700 1 --2- 172.21.15.27:0/2303232072 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f84f80640b0 0x7f84f8066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:28:58.204 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.204+0000 7f850e7fc700 1 --2- 172.21.15.27:0/2303232072 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f84f80640b0 0x7f84f8066570 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f8504004a70 tx=0x7f8504009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:28:58.204 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.208+0000 7f84f77fe700 1 -- 172.21.15.27:0/2303232072 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f8500054260 con 0x7f85100f55e0 2021-11-03T17:28:58.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.386+0000 7f85154d1700 1 -- 172.21.15.27:0/2303232072 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f85100fdee0 con 0x7f84f80640b0 2021-11-03T17:28:58.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.387+0000 7f84f77fe700 1 -- 172.21.15.27:0/2303232072 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f85100fdee0 con 0x7f84f80640b0 2021-11-03T17:28:58.383 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.389+0000 7f85154d1700 1 -- 172.21.15.27:0/2303232072 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f84f80640b0 msgr2=0x7f84f8066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:28:58.384 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.389+0000 7f85154d1700 1 --2- 172.21.15.27:0/2303232072 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f84f80640b0 0x7f84f8066570 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f8504004a70 tx=0x7f8504009040).stop 2021-11-03T17:28:58.384 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.389+0000 7f85154d1700 1 -- 172.21.15.27:0/2303232072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85100f55e0 msgr2=0x7f851010bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:28:58.385 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.389+0000 7f85154d1700 1 --2- 172.21.15.27:0/2303232072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85100f55e0 0x7f851010bc60 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f8500009990 tx=0x7f8500009b30).stop 2021-11-03T17:28:58.385 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.389+0000 7f85154d1700 1 -- 172.21.15.27:0/2303232072 shutdown_connections 2021-11-03T17:28:58.385 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.389+0000 7f85154d1700 1 --2- 172.21.15.27:0/2303232072 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f84f80640b0 0x7f84f8066570 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:58.386 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.389+0000 7f85154d1700 1 --2- 172.21.15.27:0/2303232072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85100f55e0 0x7f851010bc60 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:58.386 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.389+0000 7f85154d1700 1 --2- 172.21.15.27:0/2303232072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85100f5ef0 0x7f851010e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:58.386 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.389+0000 7f85154d1700 1 -- 172.21.15.27:0/2303232072 >> 172.21.15.27:0/2303232072 conn(0x7f85100f0de0 msgr2=0x7f85100fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:28:58.386 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.389+0000 7f85154d1700 1 -- 172.21.15.27:0/2303232072 shutdown_connections 2021-11-03T17:28:58.387 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.389+0000 7f85154d1700 1 -- 172.21.15.27:0/2303232072 wait complete. 2021-11-03T17:28:58.393 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:28:58.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:58 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:28:58.486 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.490+0000 7f4d7a748700 1 -- 172.21.15.27:0/3666276054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d740f7980 msgr2=0x7f4d740f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:28:58.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.490+0000 7f4d7a748700 1 --2- 172.21.15.27:0/3666276054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d740f7980 0x7f4d740f7de0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f4d5c004660 tx=0x7f4d5c009b30).stop 2021-11-03T17:28:58.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.490+0000 7f4d7a748700 1 -- 172.21.15.27:0/3666276054 shutdown_connections 2021-11-03T17:28:58.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.490+0000 7f4d7a748700 1 --2- 172.21.15.27:0/3666276054 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d740f7980 0x7f4d740f7de0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:58.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.490+0000 7f4d7a748700 1 --2- 172.21.15.27:0/3666276054 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d740f66e0 0x7f4d740f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:58.488 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.490+0000 7f4d7a748700 1 -- 172.21.15.27:0/3666276054 >> 172.21.15.27:0/3666276054 conn(0x7f4d740f1e40 msgr2=0x7f4d740f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:28:58.488 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.490+0000 7f4d7a748700 1 -- 172.21.15.27:0/3666276054 shutdown_connections 2021-11-03T17:28:58.488 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.490+0000 7f4d7a748700 1 -- 172.21.15.27:0/3666276054 wait complete. 2021-11-03T17:28:58.488 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.491+0000 7f4d7a748700 1 Processor -- start 2021-11-03T17:28:58.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.491+0000 7f4d7a748700 1 -- start start 2021-11-03T17:28:58.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.491+0000 7f4d7a748700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d740f66e0 0x7f4d7406b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:28:58.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.491+0000 7f4d7a748700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d740f7980 0x7f4d74069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:28:58.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.491+0000 7f4d7a748700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7406a380 con 0x7f4d740f7980 2021-11-03T17:28:58.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.491+0000 7f4d7a748700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d7406a4c0 con 0x7f4d740f66e0 2021-11-03T17:28:58.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.491+0000 7f4d6bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d740f7980 0x7f4d74069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:28:58.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.491+0000 7f4d6bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d740f7980 0x7f4d74069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57680/0 (socket says 172.21.15.27:57680) 2021-11-03T17:28:58.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.491+0000 7f4d6bfff700 1 -- 172.21.15.27:0/1512956364 learned_addr learned my addr 172.21.15.27:0/1512956364 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:28:58.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.491+0000 7f4d73fff700 1 --2- 172.21.15.27:0/1512956364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d740f66e0 0x7f4d7406b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:28:58.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.491+0000 7f4d6bfff700 1 -- 172.21.15.27:0/1512956364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d740f66e0 msgr2=0x7f4d7406b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:28:58.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.491+0000 7f4d6bfff700 1 --2- 172.21.15.27:0/1512956364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d740f66e0 0x7f4d7406b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:58.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.491+0000 7f4d6bfff700 1 -- 172.21.15.27:0/1512956364 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d5c005040 con 0x7f4d740f7980 2021-11-03T17:28:58.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.491+0000 7f4d73fff700 1 --2- 172.21.15.27:0/1512956364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d740f66e0 0x7f4d7406b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:28:58.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.491+0000 7f4d6bfff700 1 --2- 172.21.15.27:0/1512956364 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d740f7980 0x7f4d74069e20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f4d5c000c00 tx=0x7f4d5c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:28:58.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.491+0000 7f4d717fa700 1 -- 172.21.15.27:0/1512956364 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d5c019070 con 0x7f4d740f7980 2021-11-03T17:28:58.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.492+0000 7f4d717fa700 1 -- 172.21.15.27:0/1512956364 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f4d5c007e70 con 0x7f4d740f7980 2021-11-03T17:28:58.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.492+0000 7f4d717fa700 1 -- 172.21.15.27:0/1512956364 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d5c014d90 con 0x7f4d740f7980 2021-11-03T17:28:58.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.492+0000 7f4d7a748700 1 -- 172.21.15.27:0/1512956364 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d7406a740 con 0x7f4d740f7980 2021-11-03T17:28:58.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.492+0000 7f4d7a748700 1 -- 172.21.15.27:0/1512956364 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d7406aca0 con 0x7f4d740f7980 2021-11-03T17:28:58.494 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.493+0000 7f4d7a748700 1 -- 172.21.15.27:0/1512956364 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d74045ad0 con 0x7f4d740f7980 2021-11-03T17:28:58.494 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.493+0000 7f4d717fa700 1 -- 172.21.15.27:0/1512956364 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f4d5c025070 con 0x7f4d740f7980 2021-11-03T17:28:58.494 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.493+0000 7f4d717fa700 1 --2- 172.21.15.27:0/1512956364 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4d600640b0 0x7f4d60066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:28:58.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.494+0000 7f4d717fa700 1 -- 172.21.15.27:0/1512956364 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f4d5c081880 con 0x7f4d740f7980 2021-11-03T17:28:58.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.494+0000 7f4d73fff700 1 --2- 172.21.15.27:0/1512956364 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4d600640b0 0x7f4d60066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:28:58.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.494+0000 7f4d73fff700 1 --2- 172.21.15.27:0/1512956364 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4d600640b0 0x7f4d60066570 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f4d7406b730 tx=0x7f4d6400b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:28:58.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.498+0000 7f4d717fa700 1 -- 172.21.15.27:0/1512956364 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f4d5c053b20 con 0x7f4d740f7980 2021-11-03T17:28:58.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:58 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:28:58.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.669+0000 7f4d7a748700 1 -- 172.21.15.27:0/1512956364 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d740fefb0 con 0x7f4d600640b0 2021-11-03T17:28:58.678 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.684+0000 7f4d717fa700 1 -- 172.21.15.27:0/1512956364 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f4d740fefb0 con 0x7f4d600640b0 2021-11-03T17:28:58.678 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:28:58.678 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (20m) 33s ago 21m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:28:58.679 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (12m) 33s ago 21m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:28:58.679 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (12m) 9m ago 21m 7239k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:28:58.679 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (20m) 33s ago 21m 40.2M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:28:58.679 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (14m) 33s ago 16m 18.2M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:28:58.680 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (17m) 33s ago 16m 3841M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:28:58.680 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (12m) 9m ago 16m 14.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:28:58.680 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (14m) 9m ago 17m 16.3M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:28:58.681 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (14m) 33s ago 23m 497M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:28:58.681 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (13m) 9m ago 20m 412M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:28:58.681 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (13m) 33s ago 23m 119M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:28:58.682 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (12m) 9m ago 20m 69.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:28:58.682 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (21m) 33s ago 21m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:28:58.682 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (20m) 9m ago 20m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:28:58.682 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (11m) 33s ago 20m 290M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:28:58.683 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (11m) 33s ago 19m 317M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:28:58.683 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (10m) 33s ago 19m 229M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:28:58.683 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (10m) 9m ago 19m 289M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:28:58.684 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (9m) 9m ago 18m 235M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:28:58.684 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (9m) 9m ago 18m 11.5M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:28:58.685 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (20m) 33s ago 21m 48.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:28:58.686 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.685+0000 7f4d7a748700 1 -- 172.21.15.27:0/1512956364 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4d600640b0 msgr2=0x7f4d60066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:28:58.686 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.685+0000 7f4d7a748700 1 --2- 172.21.15.27:0/1512956364 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4d600640b0 0x7f4d60066570 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f4d7406b730 tx=0x7f4d6400b040).stop 2021-11-03T17:28:58.686 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.685+0000 7f4d7a748700 1 -- 172.21.15.27:0/1512956364 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d740f7980 msgr2=0x7f4d74069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:28:58.686 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.685+0000 7f4d7a748700 1 --2- 172.21.15.27:0/1512956364 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d740f7980 0x7f4d74069e20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f4d5c000c00 tx=0x7f4d5c00b040).stop 2021-11-03T17:28:58.687 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.686+0000 7f4d7a748700 1 -- 172.21.15.27:0/1512956364 shutdown_connections 2021-11-03T17:28:58.687 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.686+0000 7f4d7a748700 1 --2- 172.21.15.27:0/1512956364 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4d600640b0 0x7f4d60066570 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:58.687 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.686+0000 7f4d7a748700 1 --2- 172.21.15.27:0/1512956364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d740f66e0 0x7f4d7406b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:58.687 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.686+0000 7f4d7a748700 1 --2- 172.21.15.27:0/1512956364 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d740f7980 0x7f4d74069e20 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:58.688 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.686+0000 7f4d7a748700 1 -- 172.21.15.27:0/1512956364 >> 172.21.15.27:0/1512956364 conn(0x7f4d740f1e40 msgr2=0x7f4d740fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:28:58.688 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.686+0000 7f4d7a748700 1 -- 172.21.15.27:0/1512956364 shutdown_connections 2021-11-03T17:28:58.688 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.686+0000 7f4d7a748700 1 -- 172.21.15.27:0/1512956364 wait complete. 2021-11-03T17:28:58.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.785+0000 7f7006434700 1 -- 172.21.15.27:0/2810386368 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70000f5ef0 msgr2=0x7f70000f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:28:58.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.785+0000 7f7006434700 1 --2- 172.21.15.27:0/2810386368 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70000f5ef0 0x7f70000f6920 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f6fe8004660 tx=0x7f6fe8009b30).stop 2021-11-03T17:28:58.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.785+0000 7f7006434700 1 -- 172.21.15.27:0/2810386368 shutdown_connections 2021-11-03T17:28:58.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.785+0000 7f7006434700 1 --2- 172.21.15.27:0/2810386368 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70000f5ef0 0x7f70000f6920 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:58.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.785+0000 7f7006434700 1 --2- 172.21.15.27:0/2810386368 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70000f55e0 0x7f70000f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:58.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.785+0000 7f7006434700 1 -- 172.21.15.27:0/2810386368 >> 172.21.15.27:0/2810386368 conn(0x7f70000f0de0 msgr2=0x7f70000f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:28:58.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.785+0000 7f7006434700 1 -- 172.21.15.27:0/2810386368 shutdown_connections 2021-11-03T17:28:58.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.785+0000 7f7006434700 1 -- 172.21.15.27:0/2810386368 wait complete. 2021-11-03T17:28:58.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.785+0000 7f7006434700 1 Processor -- start 2021-11-03T17:28:58.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.786+0000 7f7006434700 1 -- start start 2021-11-03T17:28:58.787 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.786+0000 7f7006434700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70000f55e0 0x7f70000f97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:28:58.787 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.786+0000 7f7006434700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70000f5ef0 0x7f70000f7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:28:58.787 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.786+0000 7f7006434700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70000f81c0 con 0x7f70000f55e0 2021-11-03T17:28:58.788 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.786+0000 7f7006434700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70000f8300 con 0x7f70000f5ef0 2021-11-03T17:28:58.788 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.786+0000 7f6ffffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70000f55e0 0x7f70000f97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:28:58.789 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.786+0000 7f6ffffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70000f55e0 0x7f70000f97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57688/0 (socket says 172.21.15.27:57688) 2021-11-03T17:28:58.789 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.786+0000 7f6ffffff700 1 -- 172.21.15.27:0/3904883890 learned_addr learned my addr 172.21.15.27:0/3904883890 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:28:58.789 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.786+0000 7f6fff7fe700 1 --2- 172.21.15.27:0/3904883890 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70000f5ef0 0x7f70000f7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:28:58.790 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.786+0000 7f6ffffff700 1 -- 172.21.15.27:0/3904883890 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70000f5ef0 msgr2=0x7f70000f7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:28:58.790 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.786+0000 7f6ffffff700 1 --2- 172.21.15.27:0/3904883890 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70000f5ef0 0x7f70000f7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:58.790 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.786+0000 7f6ffffff700 1 -- 172.21.15.27:0/3904883890 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fe8005040 con 0x7f70000f55e0 2021-11-03T17:28:58.791 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.786+0000 7f6fff7fe700 1 --2- 172.21.15.27:0/3904883890 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70000f5ef0 0x7f70000f7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:28:58.791 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.786+0000 7f6ffffff700 1 --2- 172.21.15.27:0/3904883890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70000f55e0 0x7f70000f97e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f6ff00026d0 tx=0x7f6ff0009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:28:58.791 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.786+0000 7f6ffcff9700 1 -- 172.21.15.27:0/3904883890 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ff0009320 con 0x7f70000f55e0 2021-11-03T17:28:58.791 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.787+0000 7f7006434700 1 -- 172.21.15.27:0/3904883890 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70000f85e0 con 0x7f70000f55e0 2021-11-03T17:28:58.792 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.787+0000 7f6ffcff9700 1 -- 172.21.15.27:0/3904883890 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f6ff000f040 con 0x7f70000f55e0 2021-11-03T17:28:58.792 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.787+0000 7f6ffcff9700 1 -- 172.21.15.27:0/3904883890 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ff00147e0 con 0x7f70000f55e0 2021-11-03T17:28:58.792 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.787+0000 7f7006434700 1 -- 172.21.15.27:0/3904883890 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f700010ef90 con 0x7f70000f55e0 2021-11-03T17:28:58.793 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.788+0000 7f7006434700 1 -- 172.21.15.27:0/3904883890 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7000045ad0 con 0x7f70000f55e0 2021-11-03T17:28:58.793 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.788+0000 7f6ffcff9700 1 -- 172.21.15.27:0/3904883890 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f6ff0009480 con 0x7f70000f55e0 2021-11-03T17:28:58.794 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.788+0000 7f6ffcff9700 1 --2- 172.21.15.27:0/3904883890 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6fec064090 0x7f6fec066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:28:58.794 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.788+0000 7f6ffcff9700 1 -- 172.21.15.27:0/3904883890 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f6ff0080fd0 con 0x7f70000f55e0 2021-11-03T17:28:58.794 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.788+0000 7f6fff7fe700 1 --2- 172.21.15.27:0/3904883890 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6fec064090 0x7f6fec066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:28:58.795 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.789+0000 7f6fff7fe700 1 --2- 172.21.15.27:0/3904883890 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6fec064090 0x7f6fec066550 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f6fe8004e30 tx=0x7f6fe8009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:28:58.795 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:58.792+0000 7f6ffcff9700 1 -- 172.21.15.27:0/3904883890 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f6ff0053210 con 0x7f70000f55e0 2021-11-03T17:28:59.017 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.022+0000 7f7006434700 1 -- 172.21.15.27:0/3904883890 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f700010f770 con 0x7f70000f55e0 2021-11-03T17:28:59.017 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.023+0000 7f6ffcff9700 1 -- 172.21.15.27:0/3904883890 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f6ff0052e00 con 0x7f70000f55e0 2021-11-03T17:28:59.018 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:28:59.018 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:28:59.018 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:28:59.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:28:59.019 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:28:59.019 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:28:59.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:28:59.020 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:28:59.020 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:28:59.020 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:28:59.021 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:28:59.021 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:28:59.021 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:28:59.021 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:28:59.021 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:28:59.022 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:28:59.022 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:28:59.022 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:28:59.023 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.024+0000 7f7006434700 1 -- 172.21.15.27:0/3904883890 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6fec064090 msgr2=0x7f6fec066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:28:59.024 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.024+0000 7f7006434700 1 --2- 172.21.15.27:0/3904883890 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6fec064090 0x7f6fec066550 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f6fe8004e30 tx=0x7f6fe8009b30).stop 2021-11-03T17:28:59.024 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.024+0000 7f7006434700 1 -- 172.21.15.27:0/3904883890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70000f55e0 msgr2=0x7f70000f97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:28:59.024 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.024+0000 7f7006434700 1 --2- 172.21.15.27:0/3904883890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70000f55e0 0x7f70000f97e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f6ff00026d0 tx=0x7f6ff0009de0).stop 2021-11-03T17:28:59.024 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.024+0000 7f7006434700 1 -- 172.21.15.27:0/3904883890 shutdown_connections 2021-11-03T17:28:59.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.024+0000 7f7006434700 1 --2- 172.21.15.27:0/3904883890 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6fec064090 0x7f6fec066550 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:59.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.024+0000 7f7006434700 1 --2- 172.21.15.27:0/3904883890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70000f55e0 0x7f70000f97e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:59.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.024+0000 7f7006434700 1 --2- 172.21.15.27:0/3904883890 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70000f5ef0 0x7f70000f7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:59.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.024+0000 7f7006434700 1 -- 172.21.15.27:0/3904883890 >> 172.21.15.27:0/3904883890 conn(0x7f70000f0de0 msgr2=0x7f700005ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:28:59.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.025+0000 7f7006434700 1 -- 172.21.15.27:0/3904883890 shutdown_connections 2021-11-03T17:28:59.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.025+0000 7f7006434700 1 -- 172.21.15.27:0/3904883890 wait complete. 2021-11-03T17:28:59.122 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.125+0000 7fc7b54b8700 1 -- 172.21.15.27:0/4120565327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7b00f5450 msgr2=0x7fc7b00f5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:28:59.123 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.125+0000 7fc7b54b8700 1 --2- 172.21.15.27:0/4120565327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7b00f5450 0x7fc7b00f5870 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7fc7a0004660 tx=0x7fc7a0009b30).stop 2021-11-03T17:28:59.123 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.125+0000 7fc7b54b8700 1 -- 172.21.15.27:0/4120565327 shutdown_connections 2021-11-03T17:28:59.123 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.125+0000 7fc7b54b8700 1 --2- 172.21.15.27:0/4120565327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7b00f66f0 0x7fc7b00f6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:59.124 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.125+0000 7fc7b54b8700 1 --2- 172.21.15.27:0/4120565327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7b00f5450 0x7fc7b00f5870 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:59.124 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.125+0000 7fc7b54b8700 1 -- 172.21.15.27:0/4120565327 >> 172.21.15.27:0/4120565327 conn(0x7fc7b00f0bd0 msgr2=0x7fc7b00f2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:28:59.124 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.125+0000 7fc7b54b8700 1 -- 172.21.15.27:0/4120565327 shutdown_connections 2021-11-03T17:28:59.124 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.125+0000 7fc7b54b8700 1 -- 172.21.15.27:0/4120565327 wait complete. 2021-11-03T17:28:59.125 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.126+0000 7fc7b54b8700 1 Processor -- start 2021-11-03T17:28:59.125 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.126+0000 7fc7b54b8700 1 -- start start 2021-11-03T17:28:59.125 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.126+0000 7fc7b54b8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7b00f5450 0x7fc7b0106e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:28:59.125 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.126+0000 7fc7b54b8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7b00f66f0 0x7fc7b0107300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:28:59.126 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.126+0000 7fc7b54b8700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b0107900 con 0x7fc7b00f66f0 2021-11-03T17:28:59.126 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.126+0000 7fc7b54b8700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7b0108110 con 0x7fc7b00f5450 2021-11-03T17:28:59.126 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.126+0000 7fc7ae7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7b00f66f0 0x7fc7b0107300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:28:59.127 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.126+0000 7fc7ae7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7b00f66f0 0x7fc7b0107300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57700/0 (socket says 172.21.15.27:57700) 2021-11-03T17:28:59.127 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.126+0000 7fc7ae7fc700 1 -- 172.21.15.27:0/3245054306 learned_addr learned my addr 172.21.15.27:0/3245054306 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:28:59.127 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.127+0000 7fc7aeffd700 1 --2- 172.21.15.27:0/3245054306 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7b00f5450 0x7fc7b0106e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:28:59.127 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.127+0000 7fc7ae7fc700 1 -- 172.21.15.27:0/3245054306 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7b00f5450 msgr2=0x7fc7b0106e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:28:59.128 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.127+0000 7fc7ae7fc700 1 --2- 172.21.15.27:0/3245054306 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7b00f5450 0x7fc7b0106e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:59.128 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.127+0000 7fc7ae7fc700 1 -- 172.21.15.27:0/3245054306 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a0005040 con 0x7fc7b00f66f0 2021-11-03T17:28:59.128 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.127+0000 7fc7aeffd700 1 --2- 172.21.15.27:0/3245054306 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7b00f5450 0x7fc7b0106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:28:59.128 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.127+0000 7fc7ae7fc700 1 --2- 172.21.15.27:0/3245054306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7b00f66f0 0x7fc7b0107300 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7fc7a400c530 tx=0x7fc7a4007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:28:59.129 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.127+0000 7fc7977fe700 1 -- 172.21.15.27:0/3245054306 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7a400f650 con 0x7fc7b00f66f0 2021-11-03T17:28:59.129 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.127+0000 7fc7b54b8700 1 -- 172.21.15.27:0/3245054306 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7b0108390 con 0x7fc7b00f66f0 2021-11-03T17:28:59.129 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.127+0000 7fc7977fe700 1 -- 172.21.15.27:0/3245054306 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fc7a400fc60 con 0x7fc7b00f66f0 2021-11-03T17:28:59.129 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.127+0000 7fc7977fe700 1 -- 172.21.15.27:0/3245054306 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7a4012cd0 con 0x7fc7b00f66f0 2021-11-03T17:28:59.130 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.127+0000 7fc7b54b8700 1 -- 172.21.15.27:0/3245054306 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7b0108950 con 0x7fc7b00f66f0 2021-11-03T17:28:59.130 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.128+0000 7fc7b54b8700 1 -- 172.21.15.27:0/3245054306 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7b0059100 con 0x7fc7b00f66f0 2021-11-03T17:28:59.131 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.128+0000 7fc7977fe700 1 -- 172.21.15.27:0/3245054306 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fc7a4010070 con 0x7fc7b00f66f0 2021-11-03T17:28:59.131 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.129+0000 7fc7977fe700 1 --2- 172.21.15.27:0/3245054306 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc798064100 0x7fc7980665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:28:59.132 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.129+0000 7fc7977fe700 1 -- 172.21.15.27:0/3245054306 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fc7a4080740 con 0x7fc7b00f66f0 2021-11-03T17:28:59.132 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.129+0000 7fc7aeffd700 1 --2- 172.21.15.27:0/3245054306 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc798064100 0x7fc7980665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:28:59.132 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.129+0000 7fc7aeffd700 1 --2- 172.21.15.27:0/3245054306 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc798064100 0x7fc7980665c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fc7a0009830 tx=0x7fc7a0009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:28:59.132 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.133+0000 7fc7977fe700 1 -- 172.21.15.27:0/3245054306 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fc7a4052950 con 0x7fc7b00f66f0 2021-11-03T17:28:59.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.336+0000 7fc7b54b8700 1 -- 172.21.15.27:0/3245054306 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc7b010d350 con 0x7fc7b00f66f0 2021-11-03T17:28:59.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.336+0000 7fc7977fe700 1 -- 172.21.15.27:0/3245054306 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7fc7a4052540 con 0x7fc7b00f66f0 2021-11-03T17:28:59.331 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:28:59.332 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:28:59.332 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:28:59.332 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:28:59.332 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:28:59.333 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:28:59.333 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:28:59.333 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:28:59.333 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:28:59.333 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:28:59.334 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:28:59.334 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:28:59.334 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:28:59.334 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:28:59.335 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:28:59.335 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:28:59.335 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:28:59.335 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:28:59.336 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:28:59.336 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:28:59.336 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:28:59.336 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:28:59.336 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:28:59.337 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:28:59.337 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:28:59.337 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:28:59.337 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:28:59.337 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:28:59.338 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:28:59.338 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:28:59.338 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:28:59.338 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:28:59.339 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:28:59.339 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:28:59.339 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:28:59.339 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:28:59.339 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:28:59.343 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:28:59.344 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:28:59.345 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.338+0000 7fc7b54b8700 1 -- 172.21.15.27:0/3245054306 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc798064100 msgr2=0x7fc7980665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:28:59.345 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.338+0000 7fc7b54b8700 1 --2- 172.21.15.27:0/3245054306 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc798064100 0x7fc7980665c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7fc7a0009830 tx=0x7fc7a0009b30).stop 2021-11-03T17:28:59.345 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.338+0000 7fc7b54b8700 1 -- 172.21.15.27:0/3245054306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7b00f66f0 msgr2=0x7fc7b0107300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:28:59.345 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.338+0000 7fc7b54b8700 1 --2- 172.21.15.27:0/3245054306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7b00f66f0 0x7fc7b0107300 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7fc7a400c530 tx=0x7fc7a4007e20).stop 2021-11-03T17:28:59.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.338+0000 7fc7b54b8700 1 -- 172.21.15.27:0/3245054306 shutdown_connections 2021-11-03T17:28:59.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.338+0000 7fc7b54b8700 1 --2- 172.21.15.27:0/3245054306 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc798064100 0x7fc7980665c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:59.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.338+0000 7fc7b54b8700 1 --2- 172.21.15.27:0/3245054306 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7b00f5450 0x7fc7b0106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:59.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.338+0000 7fc7b54b8700 1 --2- 172.21.15.27:0/3245054306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7b00f66f0 0x7fc7b0107300 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:28:59.347 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.338+0000 7fc7b54b8700 1 -- 172.21.15.27:0/3245054306 >> 172.21.15.27:0/3245054306 conn(0x7fc7b00f0bd0 msgr2=0x7fc7b005f660 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:28:59.347 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.338+0000 7fc7b54b8700 1 -- 172.21.15.27:0/3245054306 shutdown_connections 2021-11-03T17:28:59.347 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:28:59.338+0000 7fc7b54b8700 1 -- 172.21.15.27:0/3245054306 wait complete. 2021-11-03T17:28:59.347 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:28:59.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:59 smithi115 ceph-mon[65453]: pgmap v430: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:59.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:59 smithi115 ceph-mon[65453]: from='client.34552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:28:59.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:59 smithi115 ceph-mon[65453]: from='client.34556 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:28:59.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:28:59 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3904883890' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:28:59.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:59 smithi027 ceph-mon[97244]: pgmap v430: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:28:59.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:59 smithi027 ceph-mon[97244]: from='client.34552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:28:59.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:59 smithi027 ceph-mon[97244]: from='client.34556 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:28:59.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:28:59 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3904883890' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:00.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 782.377426 secs 2021-11-03T17:29:00.498 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:00 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3245054306' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:29:00.499 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:00 smithi115 ceph-mon[65453]: pgmap v431: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:00.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 782.377426 secs 2021-11-03T17:29:00.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:00 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3245054306' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:29:00.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:00 smithi027 ceph-mon[97244]: pgmap v431: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:03.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:03 smithi027 ceph-mon[97244]: pgmap v432: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:03.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:03 smithi115 ceph-mon[65453]: pgmap v432: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:05.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:05 smithi115 ceph-mon[65453]: pgmap v433: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:05.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 787.377568 secs 2021-11-03T17:29:05.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:05 smithi027 ceph-mon[97244]: pgmap v433: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:05.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 787.377568 secs 2021-11-03T17:29:07.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:07 smithi115 ceph-mon[65453]: pgmap v434: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:07.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:29:07.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:29:07.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:29:07.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:07.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:07.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:07.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:07.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:07 smithi027 ceph-mon[97244]: pgmap v434: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:07.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:29:07.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:29:07.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:29:07.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:07.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:07.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:07.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:08.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:08 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:29:08.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:08 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:29:09.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:09 smithi115 ceph-mon[65453]: pgmap v435: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:09.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:09 smithi027 ceph-mon[97244]: pgmap v435: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:10.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 792.377640 secs 2021-11-03T17:29:10.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 792.377640 secs 2021-11-03T17:29:11.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:11 smithi115 ceph-mon[65453]: pgmap v436: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:11.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:11 smithi027 ceph-mon[97244]: pgmap v436: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:13.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:13 smithi027 ceph-mon[97244]: pgmap v437: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:13.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:13 smithi115 ceph-mon[65453]: pgmap v437: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:15.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:15 smithi115 ceph-mon[65453]: pgmap v438: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:15.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 797.377783 secs 2021-11-03T17:29:15.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:15 smithi027 ceph-mon[97244]: pgmap v438: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:15.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 797.377783 secs 2021-11-03T17:29:17.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:17 smithi115 ceph-mon[65453]: pgmap v439: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:17.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:29:17.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:29:17.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:29:17.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:17.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:17.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:17.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:17.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:17 smithi027 ceph-mon[97244]: pgmap v439: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:17.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:29:17.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:29:17.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:29:17.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:17.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:17.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:17.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:18.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:18 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:29:18.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:18 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:29:19.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:19 smithi115 ceph-mon[65453]: pgmap v440: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:19.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:19 smithi027 ceph-mon[97244]: pgmap v440: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:20.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 802.377899 secs 2021-11-03T17:29:20.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 802.377899 secs 2021-11-03T17:29:21.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:21 smithi115 ceph-mon[65453]: pgmap v441: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:21.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:21 smithi027 ceph-mon[97244]: pgmap v441: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:23.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:23 smithi027 ceph-mon[97244]: pgmap v442: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:23.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:23 smithi115 ceph-mon[65453]: pgmap v442: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:25.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:25 smithi115 ceph-mon[65453]: pgmap v443: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:25.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 807.378049 secs 2021-11-03T17:29:25.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:25 smithi027 ceph-mon[97244]: pgmap v443: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:25.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 807.378049 secs 2021-11-03T17:29:27.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:27 smithi115 ceph-mon[65453]: pgmap v444: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:27.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:29:27.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:29:27.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:29:27.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:27.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:27.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:27.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:27.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:27 smithi027 ceph-mon[97244]: pgmap v444: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:27.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:29:27.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:29:27.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:29:27.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:27.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:27.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:27.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:28.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:28 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:29:28.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:28 smithi115 ceph-mon[65453]: pgmap v445: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:28.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:28 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:29:28.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:28 smithi027 ceph-mon[97244]: pgmap v445: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:29.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.453+0000 7fb215556700 1 -- 172.21.15.27:0/629296551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2100f66d0 msgr2=0x7fb2100f6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:29:29.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.453+0000 7fb215556700 1 --2- 172.21.15.27:0/629296551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2100f66d0 0x7fb2100f6b30 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7fb1f8004660 tx=0x7fb1f8009b30).stop 2021-11-03T17:29:29.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.453+0000 7fb215556700 1 -- 172.21.15.27:0/629296551 shutdown_connections 2021-11-03T17:29:29.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.453+0000 7fb215556700 1 --2- 172.21.15.27:0/629296551 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2100f66d0 0x7fb2100f6b30 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:29:29.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.453+0000 7fb215556700 1 --2- 172.21.15.27:0/629296551 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2100f5430 0x7fb2100f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:29:29.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.453+0000 7fb215556700 1 -- 172.21.15.27:0/629296551 >> 172.21.15.27:0/629296551 conn(0x7fb2100f0b90 msgr2=0x7fb2100f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:29:29.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.453+0000 7fb215556700 1 -- 172.21.15.27:0/629296551 shutdown_connections 2021-11-03T17:29:29.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.453+0000 7fb215556700 1 -- 172.21.15.27:0/629296551 wait complete. 2021-11-03T17:29:29.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.454+0000 7fb215556700 1 Processor -- start 2021-11-03T17:29:29.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.454+0000 7fb215556700 1 -- start start 2021-11-03T17:29:29.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.454+0000 7fb215556700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2100f5430 0x7fb210109a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:29:29.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.454+0000 7fb215556700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2100f66d0 0x7fb21010bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:29:29.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.454+0000 7fb215556700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2100603e0 con 0x7fb2100f66d0 2021-11-03T17:29:29.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.454+0000 7fb215556700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb210060520 con 0x7fb2100f5430 2021-11-03T17:29:29.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.454+0000 7fb207fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2100f66d0 0x7fb21010bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:29:29.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.454+0000 7fb207fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2100f66d0 0x7fb21010bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57718/0 (socket says 172.21.15.27:57718) 2021-11-03T17:29:29.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.454+0000 7fb207fff700 1 -- 172.21.15.27:0/4131204032 learned_addr learned my addr 172.21.15.27:0/4131204032 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:29:29.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.454+0000 7fb20ffff700 1 --2- 172.21.15.27:0/4131204032 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2100f5430 0x7fb210109a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:29:29.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.454+0000 7fb207fff700 1 -- 172.21.15.27:0/4131204032 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2100f5430 msgr2=0x7fb210109a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:29:29.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.454+0000 7fb207fff700 1 --2- 172.21.15.27:0/4131204032 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2100f5430 0x7fb210109a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:29:29.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.454+0000 7fb207fff700 1 -- 172.21.15.27:0/4131204032 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f8005040 con 0x7fb2100f66d0 2021-11-03T17:29:29.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.454+0000 7fb207fff700 1 --2- 172.21.15.27:0/4131204032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2100f66d0 0x7fb21010bf20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7fb1f8009990 tx=0x7fb1f8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:29:29.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.455+0000 7fb20d7fa700 1 -- 172.21.15.27:0/4131204032 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1f8018070 con 0x7fb2100f66d0 2021-11-03T17:29:29.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.455+0000 7fb20d7fa700 1 -- 172.21.15.27:0/4131204032 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fb1f8007ba0 con 0x7fb2100f66d0 2021-11-03T17:29:29.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.455+0000 7fb215556700 1 -- 172.21.15.27:0/4131204032 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb21010c470 con 0x7fb2100f66d0 2021-11-03T17:29:29.457 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.455+0000 7fb20d7fa700 1 -- 172.21.15.27:0/4131204032 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1f801c3c0 con 0x7fb2100f66d0 2021-11-03T17:29:29.457 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.455+0000 7fb215556700 1 -- 172.21.15.27:0/4131204032 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb21010c9d0 con 0x7fb2100f66d0 2021-11-03T17:29:29.458 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.456+0000 7fb215556700 1 -- 172.21.15.27:0/4131204032 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb210045ad0 con 0x7fb2100f66d0 2021-11-03T17:29:29.459 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.456+0000 7fb20d7fa700 1 -- 172.21.15.27:0/4131204032 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fb1f801d420 con 0x7fb2100f66d0 2021-11-03T17:29:29.459 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.457+0000 7fb20d7fa700 1 --2- 172.21.15.27:0/4131204032 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb1fc064100 0x7fb1fc0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:29:29.459 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.457+0000 7fb20d7fa700 1 -- 172.21.15.27:0/4131204032 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fb1f80823c0 con 0x7fb2100f66d0 2021-11-03T17:29:29.459 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.457+0000 7fb20ffff700 1 --2- 172.21.15.27:0/4131204032 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb1fc064100 0x7fb1fc0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:29:29.459 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.457+0000 7fb20ffff700 1 --2- 172.21.15.27:0/4131204032 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb1fc064100 0x7fb1fc0665c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fb210060060 tx=0x7fb20000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:29:29.460 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.461+0000 7fb20d7fa700 1 -- 172.21.15.27:0/4131204032 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fb1f8054720 con 0x7fb2100f66d0 2021-11-03T17:29:29.637 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.642+0000 7fb215556700 1 -- 172.21.15.27:0/4131204032 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2100fdda0 con 0x7fb1fc064100 2021-11-03T17:29:29.638 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.643+0000 7fb20d7fa700 1 -- 172.21.15.27:0/4131204032 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fb20000f040 con 0x7fb1fc064100 2021-11-03T17:29:29.641 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.645+0000 7fb215556700 1 -- 172.21.15.27:0/4131204032 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb1fc064100 msgr2=0x7fb1fc0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:29:29.641 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.645+0000 7fb215556700 1 --2- 172.21.15.27:0/4131204032 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb1fc064100 0x7fb1fc0665c0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7fb210060060 tx=0x7fb20000b040).stop 2021-11-03T17:29:29.641 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.645+0000 7fb215556700 1 -- 172.21.15.27:0/4131204032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2100f66d0 msgr2=0x7fb21010bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:29:29.642 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.645+0000 7fb215556700 1 --2- 172.21.15.27:0/4131204032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2100f66d0 0x7fb21010bf20 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7fb1f8009990 tx=0x7fb1f8009b30).stop 2021-11-03T17:29:29.642 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.645+0000 7fb215556700 1 -- 172.21.15.27:0/4131204032 shutdown_connections 2021-11-03T17:29:29.642 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.645+0000 7fb215556700 1 --2- 172.21.15.27:0/4131204032 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb1fc064100 0x7fb1fc0665c0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:29:29.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.645+0000 7fb215556700 1 --2- 172.21.15.27:0/4131204032 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2100f5430 0x7fb210109a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:29:29.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.645+0000 7fb215556700 1 --2- 172.21.15.27:0/4131204032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2100f66d0 0x7fb21010bf20 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:29:29.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.645+0000 7fb215556700 1 -- 172.21.15.27:0/4131204032 >> 172.21.15.27:0/4131204032 conn(0x7fb2100f0b90 msgr2=0x7fb2100fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:29:29.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.645+0000 7fb215556700 1 -- 172.21.15.27:0/4131204032 shutdown_connections 2021-11-03T17:29:29.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.645+0000 7fb215556700 1 -- 172.21.15.27:0/4131204032 wait complete. 2021-11-03T17:29:29.650 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:29:29.746 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.746+0000 7f7026dcd700 1 -- 172.21.15.27:0/1789824378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70200f55c0 msgr2=0x7f70200f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:29:29.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.746+0000 7f7026dcd700 1 --2- 172.21.15.27:0/1789824378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70200f55c0 0x7f70200f59e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f7018004660 tx=0x7f7018009b30).stop 2021-11-03T17:29:29.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.746+0000 7f7026dcd700 1 -- 172.21.15.27:0/1789824378 shutdown_connections 2021-11-03T17:29:29.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.746+0000 7f7026dcd700 1 --2- 172.21.15.27:0/1789824378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70200f6860 0x7f70200f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:29:29.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.746+0000 7f7026dcd700 1 --2- 172.21.15.27:0/1789824378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70200f55c0 0x7f70200f59e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:29:29.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.746+0000 7f7026dcd700 1 -- 172.21.15.27:0/1789824378 >> 172.21.15.27:0/1789824378 conn(0x7f70200f0d40 msgr2=0x7f70200f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:29:29.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.746+0000 7f7026dcd700 1 -- 172.21.15.27:0/1789824378 shutdown_connections 2021-11-03T17:29:29.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.746+0000 7f7026dcd700 1 -- 172.21.15.27:0/1789824378 wait complete. 2021-11-03T17:29:29.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.746+0000 7f7026dcd700 1 Processor -- start 2021-11-03T17:29:29.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.746+0000 7f7026dcd700 1 -- start start 2021-11-03T17:29:29.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.747+0000 7f7026dcd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70200f55c0 0x7f7020109c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:29:29.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.747+0000 7f7026dcd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70200f6860 0x7f702010c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:29:29.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.747+0000 7f7026dcd700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70200603c0 con 0x7f70200f55c0 2021-11-03T17:29:29.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.747+0000 7f7026dcd700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7020060500 con 0x7f70200f6860 2021-11-03T17:29:29.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.747+0000 7f7024b69700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70200f55c0 0x7f7020109c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:29:29.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.747+0000 7f7024b69700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70200f55c0 0x7f7020109c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57726/0 (socket says 172.21.15.27:57726) 2021-11-03T17:29:29.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.747+0000 7f7024b69700 1 -- 172.21.15.27:0/2852943941 learned_addr learned my addr 172.21.15.27:0/2852943941 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:29:29.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.747+0000 7f701ffff700 1 --2- 172.21.15.27:0/2852943941 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70200f6860 0x7f702010c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:29:29.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.747+0000 7f7024b69700 1 -- 172.21.15.27:0/2852943941 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70200f6860 msgr2=0x7f702010c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:29:29.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.747+0000 7f7024b69700 1 --2- 172.21.15.27:0/2852943941 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70200f6860 0x7f702010c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:29:29.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.747+0000 7f7024b69700 1 -- 172.21.15.27:0/2852943941 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7018005040 con 0x7f70200f55c0 2021-11-03T17:29:29.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.747+0000 7f701ffff700 1 --2- 172.21.15.27:0/2852943941 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70200f6860 0x7f702010c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:29:29.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.747+0000 7f7024b69700 1 --2- 172.21.15.27:0/2852943941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70200f55c0 0x7f7020109c90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f7018013a70 tx=0x7f7018009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:29:29.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.747+0000 7f701d7fa700 1 -- 172.21.15.27:0/2852943941 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7018018070 con 0x7f70200f55c0 2021-11-03T17:29:29.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.747+0000 7f701d7fa700 1 -- 172.21.15.27:0/2852943941 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f70180097d0 con 0x7f70200f55c0 2021-11-03T17:29:29.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.747+0000 7f7026dcd700 1 -- 172.21.15.27:0/2852943941 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f702010c6e0 con 0x7f70200f55c0 2021-11-03T17:29:29.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.747+0000 7f701d7fa700 1 -- 172.21.15.27:0/2852943941 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f701801c3c0 con 0x7f70200f55c0 2021-11-03T17:29:29.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.748+0000 7f7026dcd700 1 -- 172.21.15.27:0/2852943941 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f702010cc10 con 0x7f70200f55c0 2021-11-03T17:29:29.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.749+0000 7f7026dcd700 1 -- 172.21.15.27:0/2852943941 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70200f49b0 con 0x7f70200f55c0 2021-11-03T17:29:29.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.749+0000 7f701d7fa700 1 -- 172.21.15.27:0/2852943941 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f7018007ba0 con 0x7f70200f55c0 2021-11-03T17:29:29.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.749+0000 7f701d7fa700 1 --2- 172.21.15.27:0/2852943941 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f700c0640b0 0x7f700c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:29:29.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.749+0000 7f701d7fa700 1 -- 172.21.15.27:0/2852943941 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f70180823f0 con 0x7f70200f55c0 2021-11-03T17:29:29.754 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.750+0000 7f701ffff700 1 --2- 172.21.15.27:0/2852943941 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f700c0640b0 0x7f700c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:29:29.754 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.750+0000 7f701ffff700 1 --2- 172.21.15.27:0/2852943941 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f700c0640b0 0x7f700c066570 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f7010007d50 tx=0x7f7010004d00).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:29:29.754 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.754+0000 7f701d7fa700 1 -- 172.21.15.27:0/2852943941 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f70180547b0 con 0x7f70200f55c0 2021-11-03T17:29:29.923 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.928+0000 7f7026dcd700 1 -- 172.21.15.27:0/2852943941 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f70200fdf00 con 0x7f700c0640b0 2021-11-03T17:29:29.938 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.943+0000 7f701d7fa700 1 -- 172.21.15.27:0/2852943941 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f70200fdf00 con 0x7f700c0640b0 2021-11-03T17:29:29.938 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:29:29.939 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (21m) 64s ago 22m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:29:29.939 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (12m) 64s ago 22m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:29:29.939 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (12m) 9m ago 21m 7239k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:29:29.939 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (20m) 64s ago 21m 40.2M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:29:29.940 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (14m) 64s ago 17m 18.2M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:29:29.940 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (17m) 64s ago 17m 3841M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:29:29.940 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (13m) 9m ago 17m 14.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:29:29.940 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (14m) 9m ago 17m 16.3M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:29:29.940 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (14m) 64s ago 23m 497M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:29:29.941 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (14m) 9m ago 21m 412M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:29:29.941 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (13m) 64s ago 23m 119M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:29:29.941 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (13m) 9m ago 21m 69.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:29:29.942 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (21m) 64s ago 21m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:29:29.942 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (21m) 9m ago 21m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:29:29.942 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (12m) 64s ago 20m 290M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:29:29.942 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (11m) 64s ago 20m 317M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:29:29.943 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (11m) 64s ago 20m 229M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:29:29.943 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (10m) 9m ago 19m 289M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:29:29.943 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (10m) 9m ago 19m 235M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:29:29.943 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (10m) 9m ago 19m 11.5M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:29:29.944 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (20m) 64s ago 21m 48.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:29:29.944 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.945+0000 7f7026dcd700 1 -- 172.21.15.27:0/2852943941 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f700c0640b0 msgr2=0x7f700c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:29:29.945 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.945+0000 7f7026dcd700 1 --2- 172.21.15.27:0/2852943941 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f700c0640b0 0x7f700c066570 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f7010007d50 tx=0x7f7010004d00).stop 2021-11-03T17:29:29.945 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.945+0000 7f7026dcd700 1 -- 172.21.15.27:0/2852943941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70200f55c0 msgr2=0x7f7020109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:29:29.945 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.945+0000 7f7026dcd700 1 --2- 172.21.15.27:0/2852943941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70200f55c0 0x7f7020109c90 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7f7018013a70 tx=0x7f7018009b30).stop 2021-11-03T17:29:29.945 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.945+0000 7f7026dcd700 1 -- 172.21.15.27:0/2852943941 shutdown_connections 2021-11-03T17:29:29.946 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.945+0000 7f7026dcd700 1 --2- 172.21.15.27:0/2852943941 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f700c0640b0 0x7f700c066570 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:29:29.946 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.945+0000 7f7026dcd700 1 --2- 172.21.15.27:0/2852943941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70200f55c0 0x7f7020109c90 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:29:29.946 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.945+0000 7f7026dcd700 1 --2- 172.21.15.27:0/2852943941 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70200f6860 0x7f702010c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:29:29.946 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.945+0000 7f7026dcd700 1 -- 172.21.15.27:0/2852943941 >> 172.21.15.27:0/2852943941 conn(0x7f70200f0d40 msgr2=0x7f70200fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:29:29.947 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.945+0000 7f7026dcd700 1 -- 172.21.15.27:0/2852943941 shutdown_connections 2021-11-03T17:29:29.947 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:29.945+0000 7f7026dcd700 1 -- 172.21.15.27:0/2852943941 wait complete. 2021-11-03T17:29:29.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:29 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 812.378168 secs 2021-11-03T17:29:30.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:29 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 812.378168 secs 2021-11-03T17:29:30.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.045+0000 7f4f09b70700 1 -- 172.21.15.27:0/429946404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f040f66e0 msgr2=0x7f4f040f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:29:30.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.045+0000 7f4f09b70700 1 --2- 172.21.15.27:0/429946404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f040f66e0 0x7f4f040f6b00 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f4efc0099e0 tx=0x7f4efc004d10).stop 2021-11-03T17:29:30.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.045+0000 7f4f09b70700 1 -- 172.21.15.27:0/429946404 shutdown_connections 2021-11-03T17:29:30.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.045+0000 7f4f09b70700 1 --2- 172.21.15.27:0/429946404 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f040f6ff0 0x7f4f040f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:29:30.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.045+0000 7f4f09b70700 1 --2- 172.21.15.27:0/429946404 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f040f66e0 0x7f4f040f6b00 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:29:30.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.045+0000 7f4f09b70700 1 -- 172.21.15.27:0/429946404 >> 172.21.15.27:0/429946404 conn(0x7f4f040f1f00 msgr2=0x7f4f040f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:29:30.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.045+0000 7f4f09b70700 1 -- 172.21.15.27:0/429946404 shutdown_connections 2021-11-03T17:29:30.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.045+0000 7f4f09b70700 1 -- 172.21.15.27:0/429946404 wait complete. 2021-11-03T17:29:30.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.045+0000 7f4f09b70700 1 Processor -- start 2021-11-03T17:29:30.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.045+0000 7f4f09b70700 1 -- start start 2021-11-03T17:29:30.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.046+0000 7f4f09b70700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f040f66e0 0x7f4f04107070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:29:30.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.046+0000 7f4f09b70700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f040f6ff0 0x7f4f04107560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:29:30.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.046+0000 7f4f09b70700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f04107b10 con 0x7f4f040f6ff0 2021-11-03T17:29:30.046 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.046+0000 7f4f09b70700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f04107c50 con 0x7f4f040f66e0 2021-11-03T17:29:30.046 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.046+0000 7f4f02ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f040f6ff0 0x7f4f04107560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:29:30.046 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.046+0000 7f4f02ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f040f6ff0 0x7f4f04107560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57738/0 (socket says 172.21.15.27:57738) 2021-11-03T17:29:30.046 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.046+0000 7f4f02ffd700 1 -- 172.21.15.27:0/2835898241 learned_addr learned my addr 172.21.15.27:0/2835898241 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:29:30.047 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.046+0000 7f4f037fe700 1 --2- 172.21.15.27:0/2835898241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f040f66e0 0x7f4f04107070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:29:30.047 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.046+0000 7f4f02ffd700 1 -- 172.21.15.27:0/2835898241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f040f66e0 msgr2=0x7f4f04107070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:29:30.047 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.046+0000 7f4f02ffd700 1 --2- 172.21.15.27:0/2835898241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f040f66e0 0x7f4f04107070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:29:30.048 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.046+0000 7f4f02ffd700 1 -- 172.21.15.27:0/2835898241 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4efc009640 con 0x7f4f040f6ff0 2021-11-03T17:29:30.048 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.046+0000 7f4f02ffd700 1 --2- 172.21.15.27:0/2835898241 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f040f6ff0 0x7f4f04107560 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f4eec0026d0 tx=0x7f4eec009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:29:30.048 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.046+0000 7f4efbfff700 1 -- 172.21.15.27:0/2835898241 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4eec0079c0 con 0x7f4f040f6ff0 2021-11-03T17:29:30.048 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.046+0000 7f4efbfff700 1 -- 172.21.15.27:0/2835898241 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f4eec00f040 con 0x7f4f040f6ff0 2021-11-03T17:29:30.049 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.046+0000 7f4f09b70700 1 -- 172.21.15.27:0/2835898241 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f041085f0 con 0x7f4f040f6ff0 2021-11-03T17:29:30.049 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.046+0000 7f4efbfff700 1 -- 172.21.15.27:0/2835898241 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4eec0148b0 con 0x7f4f040f6ff0 2021-11-03T17:29:30.049 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.046+0000 7f4f09b70700 1 -- 172.21.15.27:0/2835898241 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f04108b30 con 0x7f4f040f6ff0 2021-11-03T17:29:30.049 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.047+0000 7f4f09b70700 1 -- 172.21.15.27:0/2835898241 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f04045ad0 con 0x7f4f040f6ff0 2021-11-03T17:29:30.050 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.048+0000 7f4efbfff700 1 -- 172.21.15.27:0/2835898241 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f4eec007b20 con 0x7f4f040f6ff0 2021-11-03T17:29:30.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.048+0000 7f4efbfff700 1 --2- 172.21.15.27:0/2835898241 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4ef00640b0 0x7f4ef0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:29:30.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.048+0000 7f4efbfff700 1 -- 172.21.15.27:0/2835898241 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f4eec080790 con 0x7f4f040f6ff0 2021-11-03T17:29:30.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.048+0000 7f4f037fe700 1 --2- 172.21.15.27:0/2835898241 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4ef00640b0 0x7f4ef0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:29:30.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.049+0000 7f4f037fe700 1 --2- 172.21.15.27:0/2835898241 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4ef00640b0 0x7f4ef0066570 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f4efc003380 tx=0x7f4efc004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:29:30.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.052+0000 7f4efbfff700 1 -- 172.21.15.27:0/2835898241 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f4eec052a90 con 0x7f4f040f6ff0 2021-11-03T17:29:30.276 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.281+0000 7f4f09b70700 1 -- 172.21.15.27:0/2835898241 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4f04027a00 con 0x7f4f040f6ff0 2021-11-03T17:29:30.277 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.282+0000 7f4efbfff700 1 -- 172.21.15.27:0/2835898241 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f4eec052680 con 0x7f4f040f6ff0 2021-11-03T17:29:30.278 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:29:30.278 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:29:30.278 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:29:30.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:29:30.279 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:29:30.279 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:29:30.279 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:29:30.280 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:29:30.280 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:29:30.280 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:29:30.280 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:29:30.281 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:29:30.281 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:29:30.281 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:29:30.281 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:29:30.281 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:29:30.282 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:29:30.282 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:29:30.283 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.283+0000 7f4f09b70700 1 -- 172.21.15.27:0/2835898241 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4ef00640b0 msgr2=0x7f4ef0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:29:30.283 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.283+0000 7f4f09b70700 1 --2- 172.21.15.27:0/2835898241 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4ef00640b0 0x7f4ef0066570 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7f4efc003380 tx=0x7f4efc004d10).stop 2021-11-03T17:29:30.283 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.283+0000 7f4f09b70700 1 -- 172.21.15.27:0/2835898241 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f040f6ff0 msgr2=0x7f4f04107560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:29:30.283 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.283+0000 7f4f09b70700 1 --2- 172.21.15.27:0/2835898241 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f040f6ff0 0x7f4f04107560 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f4eec0026d0 tx=0x7f4eec009de0).stop 2021-11-03T17:29:30.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.283+0000 7f4f09b70700 1 -- 172.21.15.27:0/2835898241 shutdown_connections 2021-11-03T17:29:30.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.283+0000 7f4f09b70700 1 --2- 172.21.15.27:0/2835898241 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4ef00640b0 0x7f4ef0066570 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:29:30.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.283+0000 7f4f09b70700 1 --2- 172.21.15.27:0/2835898241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f040f66e0 0x7f4f04107070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:29:30.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.283+0000 7f4f09b70700 1 --2- 172.21.15.27:0/2835898241 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f040f6ff0 0x7f4f04107560 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:29:30.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.283+0000 7f4f09b70700 1 -- 172.21.15.27:0/2835898241 >> 172.21.15.27:0/2835898241 conn(0x7f4f040f1f00 msgr2=0x7f4f040fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:29:30.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.284+0000 7f4f09b70700 1 -- 172.21.15.27:0/2835898241 shutdown_connections 2021-11-03T17:29:30.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.284+0000 7f4f09b70700 1 -- 172.21.15.27:0/2835898241 wait complete. 2021-11-03T17:29:30.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.384+0000 7f9b4f7b2700 1 -- 172.21.15.27:0/1974644627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b480f6ff0 msgr2=0x7f9b480f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:29:30.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.384+0000 7f9b4f7b2700 1 --2- 172.21.15.27:0/1974644627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b480f6ff0 0x7f9b480f7a20 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f9b44004660 tx=0x7f9b44009b30).stop 2021-11-03T17:29:30.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.384+0000 7f9b4f7b2700 1 -- 172.21.15.27:0/1974644627 shutdown_connections 2021-11-03T17:29:30.382 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.384+0000 7f9b4f7b2700 1 --2- 172.21.15.27:0/1974644627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b480f6ff0 0x7f9b480f7a20 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:29:30.382 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.384+0000 7f9b4f7b2700 1 --2- 172.21.15.27:0/1974644627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b480f66e0 0x7f9b480f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:29:30.382 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.384+0000 7f9b4f7b2700 1 -- 172.21.15.27:0/1974644627 >> 172.21.15.27:0/1974644627 conn(0x7f9b480f1f00 msgr2=0x7f9b480f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:29:30.382 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.384+0000 7f9b4f7b2700 1 -- 172.21.15.27:0/1974644627 shutdown_connections 2021-11-03T17:29:30.383 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.384+0000 7f9b4f7b2700 1 -- 172.21.15.27:0/1974644627 wait complete. 2021-11-03T17:29:30.383 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.384+0000 7f9b4f7b2700 1 Processor -- start 2021-11-03T17:29:30.383 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.384+0000 7f9b4f7b2700 1 -- start start 2021-11-03T17:29:30.384 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.385+0000 7f9b4f7b2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b480f66e0 0x7f9b48106f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:29:30.384 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.385+0000 7f9b4f7b2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b480f6ff0 0x7f9b48107480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:29:30.384 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.385+0000 7f9b4f7b2700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b48107a30 con 0x7f9b480f66e0 2021-11-03T17:29:30.384 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.385+0000 7f9b4f7b2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b48107b70 con 0x7f9b480f6ff0 2021-11-03T17:29:30.385 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.385+0000 7f9b4d54e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b480f66e0 0x7f9b48106f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:29:30.385 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.385+0000 7f9b4d54e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b480f66e0 0x7f9b48106f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57746/0 (socket says 172.21.15.27:57746) 2021-11-03T17:29:30.385 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.385+0000 7f9b4d54e700 1 -- 172.21.15.27:0/57019346 learned_addr learned my addr 172.21.15.27:0/57019346 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:29:30.385 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.385+0000 7f9b4cd4d700 1 --2- 172.21.15.27:0/57019346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b480f6ff0 0x7f9b48107480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:29:30.385 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.385+0000 7f9b4d54e700 1 -- 172.21.15.27:0/57019346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b480f6ff0 msgr2=0x7f9b48107480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:29:30.386 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.385+0000 7f9b4d54e700 1 --2- 172.21.15.27:0/57019346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b480f6ff0 0x7f9b48107480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:29:30.386 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.385+0000 7f9b4d54e700 1 -- 172.21.15.27:0/57019346 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b44005040 con 0x7f9b480f66e0 2021-11-03T17:29:30.386 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.385+0000 7f9b4d54e700 1 --2- 172.21.15.27:0/57019346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b480f66e0 0x7f9b48106f90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f9b3800c510 tx=0x7f9b38007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:29:30.386 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.385+0000 7f9b3dffb700 1 -- 172.21.15.27:0/57019346 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b3800f650 con 0x7f9b480f66e0 2021-11-03T17:29:30.387 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.385+0000 7f9b3dffb700 1 -- 172.21.15.27:0/57019346 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f9b3800fc60 con 0x7f9b480f66e0 2021-11-03T17:29:30.387 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.385+0000 7f9b4f7b2700 1 -- 172.21.15.27:0/57019346 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b48108510 con 0x7f9b480f66e0 2021-11-03T17:29:30.387 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.385+0000 7f9b3dffb700 1 -- 172.21.15.27:0/57019346 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b38011bb0 con 0x7f9b480f66e0 2021-11-03T17:29:30.387 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.386+0000 7f9b4f7b2700 1 -- 172.21.15.27:0/57019346 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b48108ad0 con 0x7f9b480f66e0 2021-11-03T17:29:30.389 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.386+0000 7f9b4f7b2700 1 -- 172.21.15.27:0/57019346 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b48045ad0 con 0x7f9b480f66e0 2021-11-03T17:29:30.389 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.387+0000 7f9b3dffb700 1 -- 172.21.15.27:0/57019346 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f9b38010070 con 0x7f9b480f66e0 2021-11-03T17:29:30.389 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.387+0000 7f9b3dffb700 1 --2- 172.21.15.27:0/57019346 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9b34064090 0x7f9b34066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:29:30.390 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.387+0000 7f9b3dffb700 1 -- 172.21.15.27:0/57019346 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f9b38080430 con 0x7f9b480f66e0 2021-11-03T17:29:30.390 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.387+0000 7f9b4cd4d700 1 --2- 172.21.15.27:0/57019346 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9b34064090 0x7f9b34066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:29:30.390 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.388+0000 7f9b4cd4d700 1 --2- 172.21.15.27:0/57019346 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9b34064090 0x7f9b34066550 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f9b44004e30 tx=0x7f9b44009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:29:30.390 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.391+0000 7f9b3dffb700 1 -- 172.21.15.27:0/57019346 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f9b38052640 con 0x7f9b480f66e0 2021-11-03T17:29:30.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.596+0000 7f9b4f7b2700 1 -- 172.21.15.27:0/57019346 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9b48027a00 con 0x7f9b480f66e0 2021-11-03T17:29:30.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.596+0000 7f9b3dffb700 1 -- 172.21.15.27:0/57019346 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f9b38017070 con 0x7f9b480f66e0 2021-11-03T17:29:30.592 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:29:30.592 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:29:30.592 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:29:30.593 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:29:30.593 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:29:30.593 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:29:30.594 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:29:30.594 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:29:30.594 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:29:30.594 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:29:30.594 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:29:30.595 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:29:30.595 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:29:30.595 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:29:30.595 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:29:30.595 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:29:30.596 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:29:30.596 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:29:30.596 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:29:30.596 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:29:30.597 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:29:30.597 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:29:30.597 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:29:30.597 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:29:30.598 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:29:30.598 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:29:30.598 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:29:30.598 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:29:30.598 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:29:30.599 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:29:30.599 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:29:30.599 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:29:30.599 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:29:30.599 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:29:30.600 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:29:30.600 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:29:30.600 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:29:30.600 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:29:30.601 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:29:30.601 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.598+0000 7f9b4f7b2700 1 -- 172.21.15.27:0/57019346 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9b34064090 msgr2=0x7f9b34066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:29:30.602 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.598+0000 7f9b4f7b2700 1 --2- 172.21.15.27:0/57019346 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9b34064090 0x7f9b34066550 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f9b44004e30 tx=0x7f9b44009b30).stop 2021-11-03T17:29:30.602 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.598+0000 7f9b4f7b2700 1 -- 172.21.15.27:0/57019346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b480f66e0 msgr2=0x7f9b48106f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:29:30.602 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.598+0000 7f9b4f7b2700 1 --2- 172.21.15.27:0/57019346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b480f66e0 0x7f9b48106f90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f9b3800c510 tx=0x7f9b38007e00).stop 2021-11-03T17:29:30.602 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.598+0000 7f9b4f7b2700 1 -- 172.21.15.27:0/57019346 shutdown_connections 2021-11-03T17:29:30.603 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.598+0000 7f9b4f7b2700 1 --2- 172.21.15.27:0/57019346 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9b34064090 0x7f9b34066550 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:29:30.603 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.598+0000 7f9b4f7b2700 1 --2- 172.21.15.27:0/57019346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9b480f66e0 0x7f9b48106f90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:29:30.603 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.598+0000 7f9b4f7b2700 1 --2- 172.21.15.27:0/57019346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9b480f6ff0 0x7f9b48107480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:29:30.603 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.598+0000 7f9b4f7b2700 1 -- 172.21.15.27:0/57019346 >> 172.21.15.27:0/57019346 conn(0x7f9b480f1f00 msgr2=0x7f9b480fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:29:30.604 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.598+0000 7f9b4f7b2700 1 -- 172.21.15.27:0/57019346 shutdown_connections 2021-11-03T17:29:30.604 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:29:30.598+0000 7f9b4f7b2700 1 -- 172.21.15.27:0/57019346 wait complete. 2021-11-03T17:29:30.604 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:29:30.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:30 smithi115 ceph-mon[65453]: from='client.34568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:29:30.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:30 smithi115 ceph-mon[65453]: from='client.34572 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:29:30.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:30 smithi115 ceph-mon[65453]: pgmap v446: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:30.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:30 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2835898241' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:31.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:30 smithi027 ceph-mon[97244]: from='client.34568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:29:31.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:30 smithi027 ceph-mon[97244]: from='client.34572 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:29:31.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:30 smithi027 ceph-mon[97244]: pgmap v446: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:31.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:30 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2835898241' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:31.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:31 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/57019346' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:29:32.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:31 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/57019346' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:29:32.937 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:32 smithi027 ceph-mon[97244]: pgmap v447: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:32.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:32 smithi115 ceph-mon[65453]: pgmap v447: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:35.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:35 smithi115 ceph-mon[65453]: pgmap v448: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:35.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 817.378315 secs 2021-11-03T17:29:35.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:35 smithi027 ceph-mon[97244]: pgmap v448: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:35.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 817.378315 secs 2021-11-03T17:29:37.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:37 smithi115 ceph-mon[65453]: pgmap v449: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:37.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:29:37.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:37 smithi027 ceph-mon[97244]: pgmap v449: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:37.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:29:38.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:38 smithi115 ceph-mon[65453]: poll took 40120.482 ms: 1 events 2021-11-03T17:29:38.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:38 smithi027 ceph-mon[97244]: poll took 40120.482 ms: 1 events 2021-11-03T17:29:39.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:39 smithi115 ceph-mon[65453]: pgmap v450: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:39.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:39 smithi027 ceph-mon[97244]: pgmap v450: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:40.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:40 smithi115 ceph-mon[65453]: poll took 2944.783 ms: 1 events 2021-11-03T17:29:40.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:29:40.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:29:40.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:29:40.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:40.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:40.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:40.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:40.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:40 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:29:40.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 822.378468 secs 2021-11-03T17:29:40.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:40 smithi115 ceph-mon[65453]: pgmap v451: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:40.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:40 smithi027 ceph-mon[97244]: poll took 2944.783 ms: 1 events 2021-11-03T17:29:40.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:29:40.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:29:40.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:29:40.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:40.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:40.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:40.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:40.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:40 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:29:40.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 822.378468 secs 2021-11-03T17:29:40.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:40 smithi027 ceph-mon[97244]: pgmap v451: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:43.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:43 smithi027 ceph-mon[97244]: pgmap v452: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:43.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:43 smithi115 ceph-mon[65453]: pgmap v452: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:45.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:45 smithi115 ceph-mon[65453]: pgmap v453: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:45.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 827.378601 secs 2021-11-03T17:29:45.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:45 smithi027 ceph-mon[97244]: pgmap v453: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:45.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 827.378601 secs 2021-11-03T17:29:47.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:47 smithi115 ceph-mon[65453]: pgmap v454: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:47.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:47 smithi027 ceph-mon[97244]: pgmap v454: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:49.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:49 smithi115 ceph-mon[65453]: pgmap v455: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:49.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:49 smithi027 ceph-mon[97244]: pgmap v455: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:50.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:29:50.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:29:50.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:29:50.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:50.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:50.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:50.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:50.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 832.378663 secs 2021-11-03T17:29:50.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:29:50.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:29:50.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:29:50.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:50.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:50.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:50.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:29:50.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 832.378663 secs 2021-11-03T17:29:51.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:51 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:29:51.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:51 smithi115 ceph-mon[65453]: pgmap v456: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:51.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:51 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:29:51.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:51 smithi027 ceph-mon[97244]: pgmap v456: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:53.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:53 smithi027 ceph-mon[97244]: pgmap v457: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:53.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:53 smithi115 ceph-mon[65453]: pgmap v457: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:55.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:55 smithi115 ceph-mon[65453]: pgmap v458: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:55.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 837.378805 secs 2021-11-03T17:29:55.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:55 smithi027 ceph-mon[97244]: pgmap v458: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:55.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 837.378805 secs 2021-11-03T17:29:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:29:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:29:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:29:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:29:57.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:57 smithi115 ceph-mon[65453]: pgmap v459: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:57.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:57 smithi027 ceph-mon[97244]: pgmap v459: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:59.312 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:29:59 smithi115 ceph-mon[65453]: pgmap v460: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:29:59.317 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:29:59 smithi027 ceph-mon[97244]: pgmap v460: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:00.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:30:00.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 842.378873 secs 2021-11-03T17:30:00.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:30:00.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:30:00.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:00.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:00.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:00.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:00.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:00 smithi115 ceph-mon[65453]: overall HEALTH_WARN 1 filesystem is degraded; 1 filesystem has a failed mds daemon; 1 MDSs report slow requests; 3 daemons have recently crashed 2021-11-03T17:30:00.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:30:00.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 842.378873 secs 2021-11-03T17:30:00.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:30:00.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:30:00.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:00.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:00.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:00.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:00.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:00 smithi027 ceph-mon[97244]: overall HEALTH_WARN 1 filesystem is degraded; 1 filesystem has a failed mds daemon; 1 MDSs report slow requests; 3 daemons have recently crashed 2021-11-03T17:30:00.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.713+0000 7fc34c2a5700 1 -- 172.21.15.27:0/2552589348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3440f6ff0 msgr2=0x7fc3440f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:30:00.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.713+0000 7fc34c2a5700 1 --2- 172.21.15.27:0/2552589348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3440f6ff0 0x7fc3440f7a20 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7fc338004660 tx=0x7fc338009b30).stop 2021-11-03T17:30:00.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.714+0000 7fc34c2a5700 1 -- 172.21.15.27:0/2552589348 shutdown_connections 2021-11-03T17:30:00.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.714+0000 7fc34c2a5700 1 --2- 172.21.15.27:0/2552589348 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3440f6ff0 0x7fc3440f7a20 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:00.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.714+0000 7fc34c2a5700 1 --2- 172.21.15.27:0/2552589348 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3440f66e0 0x7fc3440f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:00.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.714+0000 7fc34c2a5700 1 -- 172.21.15.27:0/2552589348 >> 172.21.15.27:0/2552589348 conn(0x7fc3440f1f00 msgr2=0x7fc3440f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:30:00.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.714+0000 7fc34c2a5700 1 -- 172.21.15.27:0/2552589348 shutdown_connections 2021-11-03T17:30:00.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.714+0000 7fc34c2a5700 1 -- 172.21.15.27:0/2552589348 wait complete. 2021-11-03T17:30:00.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.714+0000 7fc34c2a5700 1 Processor -- start 2021-11-03T17:30:00.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.714+0000 7fc34c2a5700 1 -- start start 2021-11-03T17:30:00.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.714+0000 7fc34c2a5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3440f66e0 0x7fc34406b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:30:00.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.714+0000 7fc34c2a5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3440f6ff0 0x7fc344069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:30:00.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.714+0000 7fc34c2a5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc34406a380 con 0x7fc3440f66e0 2021-11-03T17:30:00.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.714+0000 7fc34c2a5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc34406a4c0 con 0x7fc3440f6ff0 2021-11-03T17:30:00.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.714+0000 7fc34a041700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3440f66e0 0x7fc34406b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:30:00.715 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.714+0000 7fc34a041700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3440f66e0 0x7fc34406b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57762/0 (socket says 172.21.15.27:57762) 2021-11-03T17:30:00.715 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.714+0000 7fc34a041700 1 -- 172.21.15.27:0/3559270420 learned_addr learned my addr 172.21.15.27:0/3559270420 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:30:00.715 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.715+0000 7fc349840700 1 --2- 172.21.15.27:0/3559270420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3440f6ff0 0x7fc344069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:30:00.715 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.715+0000 7fc34a041700 1 -- 172.21.15.27:0/3559270420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3440f6ff0 msgr2=0x7fc344069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:30:00.716 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.715+0000 7fc34a041700 1 --2- 172.21.15.27:0/3559270420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3440f6ff0 0x7fc344069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:00.716 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.715+0000 7fc34a041700 1 -- 172.21.15.27:0/3559270420 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc338005040 con 0x7fc3440f66e0 2021-11-03T17:30:00.716 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.715+0000 7fc34a041700 1 --2- 172.21.15.27:0/3559270420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3440f66e0 0x7fc34406b9a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7fc33c007fd0 tx=0x7fc33c004ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:30:00.716 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.715+0000 7fc342ffd700 1 -- 172.21.15.27:0/3559270420 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc33c00b4f0 con 0x7fc3440f66e0 2021-11-03T17:30:00.717 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.715+0000 7fc342ffd700 1 -- 172.21.15.27:0/3559270420 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fc33c00f040 con 0x7fc3440f66e0 2021-11-03T17:30:00.717 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.715+0000 7fc34c2a5700 1 -- 172.21.15.27:0/3559270420 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc34406a7a0 con 0x7fc3440f66e0 2021-11-03T17:30:00.717 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.715+0000 7fc342ffd700 1 -- 172.21.15.27:0/3559270420 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc33c013a50 con 0x7fc3440f66e0 2021-11-03T17:30:00.717 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.715+0000 7fc34c2a5700 1 -- 172.21.15.27:0/3559270420 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc34410efa0 con 0x7fc3440f66e0 2021-11-03T17:30:00.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.716+0000 7fc34c2a5700 1 -- 172.21.15.27:0/3559270420 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc344045ad0 con 0x7fc3440f66e0 2021-11-03T17:30:00.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.717+0000 7fc342ffd700 1 -- 172.21.15.27:0/3559270420 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fc33c01a030 con 0x7fc3440f66e0 2021-11-03T17:30:00.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.717+0000 7fc342ffd700 1 --2- 172.21.15.27:0/3559270420 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc32c0640b0 0x7fc32c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:30:00.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.717+0000 7fc342ffd700 1 -- 172.21.15.27:0/3559270420 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fc33c0806a0 con 0x7fc3440f66e0 2021-11-03T17:30:00.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.717+0000 7fc349840700 1 --2- 172.21.15.27:0/3559270420 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc32c0640b0 0x7fc32c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:30:00.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.718+0000 7fc349840700 1 --2- 172.21.15.27:0/3559270420 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc32c0640b0 0x7fc32c066570 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7fc338004630 tx=0x7fc338009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:30:00.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.721+0000 7fc342ffd700 1 -- 172.21.15.27:0/3559270420 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fc33c052960 con 0x7fc3440f66e0 2021-11-03T17:30:00.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.899+0000 7fc34c2a5700 1 -- 172.21.15.27:0/3559270420 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc3440fefe0 con 0x7fc32c0640b0 2021-11-03T17:30:00.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.900+0000 7fc342ffd700 1 -- 172.21.15.27:0/3559270420 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fc3440fefe0 con 0x7fc32c0640b0 2021-11-03T17:30:00.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.902+0000 7fc34c2a5700 1 -- 172.21.15.27:0/3559270420 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc32c0640b0 msgr2=0x7fc32c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:30:00.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.902+0000 7fc34c2a5700 1 --2- 172.21.15.27:0/3559270420 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc32c0640b0 0x7fc32c066570 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7fc338004630 tx=0x7fc338009b30).stop 2021-11-03T17:30:00.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.902+0000 7fc34c2a5700 1 -- 172.21.15.27:0/3559270420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3440f66e0 msgr2=0x7fc34406b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:30:00.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.902+0000 7fc34c2a5700 1 --2- 172.21.15.27:0/3559270420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3440f66e0 0x7fc34406b9a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7fc33c007fd0 tx=0x7fc33c004ee0).stop 2021-11-03T17:30:00.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.902+0000 7fc34c2a5700 1 -- 172.21.15.27:0/3559270420 shutdown_connections 2021-11-03T17:30:00.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.902+0000 7fc34c2a5700 1 --2- 172.21.15.27:0/3559270420 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc32c0640b0 0x7fc32c066570 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:00.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.902+0000 7fc34c2a5700 1 --2- 172.21.15.27:0/3559270420 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3440f66e0 0x7fc34406b9a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:00.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.902+0000 7fc34c2a5700 1 --2- 172.21.15.27:0/3559270420 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3440f6ff0 0x7fc344069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:00.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.902+0000 7fc34c2a5700 1 -- 172.21.15.27:0/3559270420 >> 172.21.15.27:0/3559270420 conn(0x7fc3440f1f00 msgr2=0x7fc3440fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:30:00.900 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.902+0000 7fc34c2a5700 1 -- 172.21.15.27:0/3559270420 shutdown_connections 2021-11-03T17:30:00.900 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:00.902+0000 7fc34c2a5700 1 -- 172.21.15.27:0/3559270420 wait complete. 2021-11-03T17:30:00.907 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:30:00.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.002+0000 7fc9cc84f700 1 -- 172.21.15.27:0/261267908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9c80602b0 msgr2=0x7fc9c80f9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:30:00.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.002+0000 7fc9cc84f700 1 --2- 172.21.15.27:0/261267908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9c80602b0 0x7fc9c80f9180 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7fc9b8004660 tx=0x7fc9b8009b30).stop 2021-11-03T17:30:01.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.002+0000 7fc9cc84f700 1 -- 172.21.15.27:0/261267908 shutdown_connections 2021-11-03T17:30:01.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.002+0000 7fc9cc84f700 1 --2- 172.21.15.27:0/261267908 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9c80f9700 0x7fc9c80fbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:01.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.002+0000 7fc9cc84f700 1 --2- 172.21.15.27:0/261267908 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9c80602b0 0x7fc9c80f9180 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:01.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.002+0000 7fc9cc84f700 1 -- 172.21.15.27:0/261267908 >> 172.21.15.27:0/261267908 conn(0x7fc9c80f0ba0 msgr2=0x7fc9c80f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:30:01.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.002+0000 7fc9cc84f700 1 -- 172.21.15.27:0/261267908 shutdown_connections 2021-11-03T17:30:01.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.002+0000 7fc9cc84f700 1 -- 172.21.15.27:0/261267908 wait complete. 2021-11-03T17:30:01.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.003+0000 7fc9cc84f700 1 Processor -- start 2021-11-03T17:30:01.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.003+0000 7fc9cc84f700 1 -- start start 2021-11-03T17:30:01.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.003+0000 7fc9cc84f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9c80602b0 0x7fc9c8106db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:30:01.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.003+0000 7fc9cc84f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9c80f9700 0x7fc9c81072a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:30:01.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.003+0000 7fc9cc84f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c81078a0 con 0x7fc9c80602b0 2021-11-03T17:30:01.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.003+0000 7fc9cc84f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9c81080e0 con 0x7fc9c80f9700 2021-11-03T17:30:01.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.003+0000 7fc9c6d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9c80602b0 0x7fc9c8106db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:30:01.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.003+0000 7fc9c6d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9c80602b0 0x7fc9c8106db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57772/0 (socket says 172.21.15.27:57772) 2021-11-03T17:30:01.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.003+0000 7fc9c6d9d700 1 -- 172.21.15.27:0/2679005349 learned_addr learned my addr 172.21.15.27:0/2679005349 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:30:01.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.003+0000 7fc9c6d9d700 1 -- 172.21.15.27:0/2679005349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9c80f9700 msgr2=0x7fc9c81072a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:30:01.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.004+0000 7fc9c659c700 1 --2- 172.21.15.27:0/2679005349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9c80f9700 0x7fc9c81072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:30:01.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.004+0000 7fc9c6d9d700 1 --2- 172.21.15.27:0/2679005349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9c80f9700 0x7fc9c81072a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:01.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.004+0000 7fc9c6d9d700 1 -- 172.21.15.27:0/2679005349 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9b8005040 con 0x7fc9c80602b0 2021-11-03T17:30:01.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.004+0000 7fc9c659c700 1 --2- 172.21.15.27:0/2679005349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9c80f9700 0x7fc9c81072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:30:01.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.004+0000 7fc9c6d9d700 1 --2- 172.21.15.27:0/2679005349 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9c80602b0 0x7fc9c8106db0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7fc9b8009990 tx=0x7fc9b8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:30:01.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.004+0000 7fc9af7fe700 1 -- 172.21.15.27:0/2679005349 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9b8018070 con 0x7fc9c80602b0 2021-11-03T17:30:01.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.004+0000 7fc9cc84f700 1 -- 172.21.15.27:0/2679005349 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9c8108360 con 0x7fc9c80602b0 2021-11-03T17:30:01.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.004+0000 7fc9cc84f700 1 -- 172.21.15.27:0/2679005349 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9c8108840 con 0x7fc9c80602b0 2021-11-03T17:30:01.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.004+0000 7fc9af7fe700 1 -- 172.21.15.27:0/2679005349 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fc9b8007ba0 con 0x7fc9c80602b0 2021-11-03T17:30:01.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.004+0000 7fc9af7fe700 1 -- 172.21.15.27:0/2679005349 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9b801c3c0 con 0x7fc9c80602b0 2021-11-03T17:30:01.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.005+0000 7fc9cc84f700 1 -- 172.21.15.27:0/2679005349 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9c8059100 con 0x7fc9c80602b0 2021-11-03T17:30:01.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.005+0000 7fc9af7fe700 1 -- 172.21.15.27:0/2679005349 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fc9b801d420 con 0x7fc9c80602b0 2021-11-03T17:30:01.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.006+0000 7fc9af7fe700 1 --2- 172.21.15.27:0/2679005349 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc9b0064100 0x7fc9b00665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:30:01.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.006+0000 7fc9af7fe700 1 -- 172.21.15.27:0/2679005349 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fc9b8082500 con 0x7fc9c80602b0 2021-11-03T17:30:01.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.006+0000 7fc9c659c700 1 --2- 172.21.15.27:0/2679005349 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc9b0064100 0x7fc9b00665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:30:01.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.006+0000 7fc9c659c700 1 --2- 172.21.15.27:0/2679005349 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc9b0064100 0x7fc9b00665c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fc9bc009fd0 tx=0x7fc9bc00b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:30:01.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.010+0000 7fc9af7fe700 1 -- 172.21.15.27:0/2679005349 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fc9b8054860 con 0x7fc9c80602b0 2021-11-03T17:30:01.179 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.183+0000 7fc9cc84f700 1 -- 172.21.15.27:0/2679005349 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9c80fee30 con 0x7fc9b0064100 2021-11-03T17:30:01.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.198+0000 7fc9af7fe700 1 -- 172.21.15.27:0/2679005349 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fc9c80fee30 con 0x7fc9b0064100 2021-11-03T17:30:01.194 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:30:01.194 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (21m) 95s ago 22m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:30:01.194 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (13m) 95s ago 22m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:30:01.194 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (13m) 21s ago 22m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:30:01.195 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (21m) 95s ago 22m 40.2M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:30:01.195 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (15m) 95s ago 17m 18.2M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:30:01.195 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (18m) 95s ago 18m 3841M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:30:01.195 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (13m) 21s ago 17m 17.6M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:30:01.196 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (15m) 21s ago 18m 19.3M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:30:01.196 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (15m) 95s ago 24m 497M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:30:01.196 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (14m) 21s ago 22m 415M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:30:01.196 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (14m) 95s ago 24m 119M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:30:01.196 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (13m) 21s ago 21m 71.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:30:01.197 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (22m) 95s ago 22m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:30:01.197 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (21m) 21s ago 21m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:30:01.197 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (12m) 95s ago 21m 290M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:30:01.197 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (12m) 95s ago 20m 317M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:30:01.198 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (11m) 95s ago 20m 229M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:30:01.198 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (11m) 21s ago 20m 329M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:30:01.198 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (10m) 21s ago 19m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:30:01.198 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (10m) 21s ago 19m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:30:01.199 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (21m) 95s ago 22m 48.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:30:01.200 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.200+0000 7fc9cc84f700 1 -- 172.21.15.27:0/2679005349 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc9b0064100 msgr2=0x7fc9b00665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:30:01.200 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.200+0000 7fc9cc84f700 1 --2- 172.21.15.27:0/2679005349 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc9b0064100 0x7fc9b00665c0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7fc9bc009fd0 tx=0x7fc9bc00b040).stop 2021-11-03T17:30:01.200 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.200+0000 7fc9cc84f700 1 -- 172.21.15.27:0/2679005349 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9c80602b0 msgr2=0x7fc9c8106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:30:01.200 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.200+0000 7fc9cc84f700 1 --2- 172.21.15.27:0/2679005349 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9c80602b0 0x7fc9c8106db0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7fc9b8009990 tx=0x7fc9b8009b30).stop 2021-11-03T17:30:01.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.200+0000 7fc9cc84f700 1 -- 172.21.15.27:0/2679005349 shutdown_connections 2021-11-03T17:30:01.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.200+0000 7fc9cc84f700 1 --2- 172.21.15.27:0/2679005349 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc9b0064100 0x7fc9b00665c0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:01.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.200+0000 7fc9cc84f700 1 --2- 172.21.15.27:0/2679005349 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9c80602b0 0x7fc9c8106db0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:01.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.200+0000 7fc9cc84f700 1 --2- 172.21.15.27:0/2679005349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9c80f9700 0x7fc9c81072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:01.202 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.200+0000 7fc9cc84f700 1 -- 172.21.15.27:0/2679005349 >> 172.21.15.27:0/2679005349 conn(0x7fc9c80f0ba0 msgr2=0x7fc9c80f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:30:01.202 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.200+0000 7fc9cc84f700 1 -- 172.21.15.27:0/2679005349 shutdown_connections 2021-11-03T17:30:01.202 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.200+0000 7fc9cc84f700 1 -- 172.21.15.27:0/2679005349 wait complete. 2021-11-03T17:30:01.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.300+0000 7fb34d7fb700 1 -- 172.21.15.27:0/540305739 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3480f55c0 msgr2=0x7fb3480f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:30:01.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.300+0000 7fb34d7fb700 1 --2- 172.21.15.27:0/540305739 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3480f55c0 0x7fb3480f59e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7fb338004660 tx=0x7fb338009b30).stop 2021-11-03T17:30:01.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.300+0000 7fb34d7fb700 1 -- 172.21.15.27:0/540305739 shutdown_connections 2021-11-03T17:30:01.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.300+0000 7fb34d7fb700 1 --2- 172.21.15.27:0/540305739 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3480f6860 0x7fb3480f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:01.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.300+0000 7fb34d7fb700 1 --2- 172.21.15.27:0/540305739 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3480f55c0 0x7fb3480f59e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:01.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.300+0000 7fb34d7fb700 1 -- 172.21.15.27:0/540305739 >> 172.21.15.27:0/540305739 conn(0x7fb3480f0d40 msgr2=0x7fb3480f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:30:01.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.300+0000 7fb34d7fb700 1 -- 172.21.15.27:0/540305739 shutdown_connections 2021-11-03T17:30:01.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.300+0000 7fb34d7fb700 1 -- 172.21.15.27:0/540305739 wait complete. 2021-11-03T17:30:01.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.300+0000 7fb34d7fb700 1 Processor -- start 2021-11-03T17:30:01.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.301+0000 7fb34d7fb700 1 -- start start 2021-11-03T17:30:01.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.301+0000 7fb34d7fb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3480f55c0 0x7fb348106fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:30:01.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.301+0000 7fb34d7fb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3480f6860 0x7fb3481074c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:30:01.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.301+0000 7fb34d7fb700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb348107a70 con 0x7fb3480f6860 2021-11-03T17:30:01.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.301+0000 7fb34d7fb700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb348107bb0 con 0x7fb3480f55c0 2021-11-03T17:30:01.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.301+0000 7fb3467fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3480f6860 0x7fb3481074c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:30:01.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.301+0000 7fb3467fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3480f6860 0x7fb3481074c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57784/0 (socket says 172.21.15.27:57784) 2021-11-03T17:30:01.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.301+0000 7fb3467fc700 1 -- 172.21.15.27:0/478661060 learned_addr learned my addr 172.21.15.27:0/478661060 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:30:01.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.301+0000 7fb346ffd700 1 --2- 172.21.15.27:0/478661060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3480f55c0 0x7fb348106fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:30:01.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.301+0000 7fb3467fc700 1 -- 172.21.15.27:0/478661060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3480f55c0 msgr2=0x7fb348106fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:30:01.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.301+0000 7fb3467fc700 1 --2- 172.21.15.27:0/478661060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3480f55c0 0x7fb348106fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:01.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.301+0000 7fb3467fc700 1 -- 172.21.15.27:0/478661060 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb338005040 con 0x7fb3480f6860 2021-11-03T17:30:01.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.301+0000 7fb3467fc700 1 --2- 172.21.15.27:0/478661060 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3480f6860 0x7fb3481074c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7fb33c00c530 tx=0x7fb33c007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:30:01.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.301+0000 7fb32f7fe700 1 -- 172.21.15.27:0/478661060 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb33c00f650 con 0x7fb3480f6860 2021-11-03T17:30:01.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.302+0000 7fb32f7fe700 1 -- 172.21.15.27:0/478661060 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fb33c00fc60 con 0x7fb3480f6860 2021-11-03T17:30:01.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.302+0000 7fb32f7fe700 1 -- 172.21.15.27:0/478661060 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb33c012c20 con 0x7fb3480f6860 2021-11-03T17:30:01.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.302+0000 7fb34d7fb700 1 -- 172.21.15.27:0/478661060 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb348108550 con 0x7fb3480f6860 2021-11-03T17:30:01.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.302+0000 7fb34d7fb700 1 -- 172.21.15.27:0/478661060 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb348108b10 con 0x7fb3480f6860 2021-11-03T17:30:01.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.302+0000 7fb34d7fb700 1 -- 172.21.15.27:0/478661060 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb348045ad0 con 0x7fb3480f6860 2021-11-03T17:30:01.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.303+0000 7fb32f7fe700 1 -- 172.21.15.27:0/478661060 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fb33c010070 con 0x7fb3480f6860 2021-11-03T17:30:01.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.303+0000 7fb32f7fe700 1 --2- 172.21.15.27:0/478661060 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb330064090 0x7fb330066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:30:01.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.303+0000 7fb32f7fe700 1 -- 172.21.15.27:0/478661060 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fb33c0805b0 con 0x7fb3480f6860 2021-11-03T17:30:01.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.303+0000 7fb346ffd700 1 --2- 172.21.15.27:0/478661060 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb330064090 0x7fb330066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:30:01.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.304+0000 7fb346ffd700 1 --2- 172.21.15.27:0/478661060 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb330064090 0x7fb330066550 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fb338013690 tx=0x7fb338009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:30:01.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.307+0000 7fb32f7fe700 1 -- 172.21.15.27:0/478661060 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fb33c0527c0 con 0x7fb3480f6860 2021-11-03T17:30:01.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:01 smithi115 ceph-mon[65453]: poll took 20074.266 ms: 1 events 2021-11-03T17:30:01.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:01 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:30:01.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:01 smithi115 ceph-mon[65453]: pgmap v461: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:01.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:01 smithi027 ceph-mon[97244]: poll took 20074.266 ms: 1 events 2021-11-03T17:30:01.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:01 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:30:01.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:01 smithi027 ceph-mon[97244]: pgmap v461: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:01.536 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.541+0000 7fb34d7fb700 1 -- 172.21.15.27:0/478661060 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb348027a00 con 0x7fb3480f6860 2021-11-03T17:30:01.537 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.541+0000 7fb32f7fe700 1 -- 172.21.15.27:0/478661060 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fb33c0523b0 con 0x7fb3480f6860 2021-11-03T17:30:01.537 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:30:01.537 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:30:01.537 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:30:01.538 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:30:01.538 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:30:01.538 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:30:01.538 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:30:01.539 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:30:01.539 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:30:01.539 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:30:01.539 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:30:01.540 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:30:01.540 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:30:01.540 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:30:01.540 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:30:01.541 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:30:01.541 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:30:01.541 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:30:01.542 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.543+0000 7fb34d7fb700 1 -- 172.21.15.27:0/478661060 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb330064090 msgr2=0x7fb330066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:30:01.543 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.543+0000 7fb34d7fb700 1 --2- 172.21.15.27:0/478661060 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb330064090 0x7fb330066550 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7fb338013690 tx=0x7fb338009b30).stop 2021-11-03T17:30:01.543 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.543+0000 7fb34d7fb700 1 -- 172.21.15.27:0/478661060 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3480f6860 msgr2=0x7fb3481074c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:30:01.543 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.543+0000 7fb34d7fb700 1 --2- 172.21.15.27:0/478661060 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3480f6860 0x7fb3481074c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7fb33c00c530 tx=0x7fb33c007e20).stop 2021-11-03T17:30:01.543 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.543+0000 7fb34d7fb700 1 -- 172.21.15.27:0/478661060 shutdown_connections 2021-11-03T17:30:01.544 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.543+0000 7fb34d7fb700 1 --2- 172.21.15.27:0/478661060 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb330064090 0x7fb330066550 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:01.544 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.543+0000 7fb34d7fb700 1 --2- 172.21.15.27:0/478661060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3480f55c0 0x7fb348106fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:01.544 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.543+0000 7fb34d7fb700 1 --2- 172.21.15.27:0/478661060 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3480f6860 0x7fb3481074c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:01.544 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.543+0000 7fb34d7fb700 1 -- 172.21.15.27:0/478661060 >> 172.21.15.27:0/478661060 conn(0x7fb3480f0d40 msgr2=0x7fb3480fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:30:01.544 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.544+0000 7fb34d7fb700 1 -- 172.21.15.27:0/478661060 shutdown_connections 2021-11-03T17:30:01.545 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.544+0000 7fb34d7fb700 1 -- 172.21.15.27:0/478661060 wait complete. 2021-11-03T17:30:01.639 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.644+0000 7f479158d700 1 -- 172.21.15.27:0/2453211078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f478c0f6860 msgr2=0x7f478c0f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:30:01.639 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.644+0000 7f479158d700 1 --2- 172.21.15.27:0/2453211078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f478c0f6860 0x7f478c0f6cc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7f477c004660 tx=0x7f477c009b30).stop 2021-11-03T17:30:01.640 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.644+0000 7f479158d700 1 -- 172.21.15.27:0/2453211078 shutdown_connections 2021-11-03T17:30:01.640 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.644+0000 7f479158d700 1 --2- 172.21.15.27:0/2453211078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f478c0f6860 0x7f478c0f6cc0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:01.640 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.644+0000 7f479158d700 1 --2- 172.21.15.27:0/2453211078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f478c0f55c0 0x7f478c0f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:01.640 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.644+0000 7f479158d700 1 -- 172.21.15.27:0/2453211078 >> 172.21.15.27:0/2453211078 conn(0x7f478c0f0d40 msgr2=0x7f478c0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:30:01.641 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.644+0000 7f479158d700 1 -- 172.21.15.27:0/2453211078 shutdown_connections 2021-11-03T17:30:01.641 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.644+0000 7f479158d700 1 -- 172.21.15.27:0/2453211078 wait complete. 2021-11-03T17:30:01.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.644+0000 7f479158d700 1 Processor -- start 2021-11-03T17:30:01.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.645+0000 7f479158d700 1 -- start start 2021-11-03T17:30:01.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.645+0000 7f479158d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f478c0f55c0 0x7f478c109c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:30:01.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.645+0000 7f479158d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f478c0f6860 0x7f478c10c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:30:01.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.645+0000 7f479158d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478c0603e0 con 0x7f478c0f55c0 2021-11-03T17:30:01.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.645+0000 7f479158d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478c060520 con 0x7f478c0f6860 2021-11-03T17:30:01.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.645+0000 7f478affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f478c0f55c0 0x7f478c109c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:30:01.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.645+0000 7f478affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f478c0f55c0 0x7f478c109c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57792/0 (socket says 172.21.15.27:57792) 2021-11-03T17:30:01.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.645+0000 7f478affd700 1 -- 172.21.15.27:0/3587446509 learned_addr learned my addr 172.21.15.27:0/3587446509 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:30:01.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.645+0000 7f478a7fc700 1 --2- 172.21.15.27:0/3587446509 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f478c0f6860 0x7f478c10c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:30:01.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.645+0000 7f478affd700 1 -- 172.21.15.27:0/3587446509 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f478c0f6860 msgr2=0x7f478c10c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:30:01.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.645+0000 7f478affd700 1 --2- 172.21.15.27:0/3587446509 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f478c0f6860 0x7f478c10c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:01.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.645+0000 7f478affd700 1 -- 172.21.15.27:0/3587446509 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f477c005040 con 0x7f478c0f55c0 2021-11-03T17:30:01.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.645+0000 7f478affd700 1 --2- 172.21.15.27:0/3587446509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f478c0f55c0 0x7f478c109c50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f478000c530 tx=0x7f4780007e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:30:01.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.645+0000 7f47737fe700 1 -- 172.21.15.27:0/3587446509 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f478000f650 con 0x7f478c0f55c0 2021-11-03T17:30:01.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.645+0000 7f47737fe700 1 -- 172.21.15.27:0/3587446509 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f478000fc60 con 0x7f478c0f55c0 2021-11-03T17:30:01.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.645+0000 7f479158d700 1 -- 172.21.15.27:0/3587446509 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f478c10c700 con 0x7f478c0f55c0 2021-11-03T17:30:01.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.645+0000 7f479158d700 1 -- 172.21.15.27:0/3587446509 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f478c10ccc0 con 0x7f478c0f55c0 2021-11-03T17:30:01.648 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.646+0000 7f47737fe700 1 -- 172.21.15.27:0/3587446509 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4780012cb0 con 0x7f478c0f55c0 2021-11-03T17:30:01.648 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.646+0000 7f479158d700 1 -- 172.21.15.27:0/3587446509 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f476c004fa0 con 0x7f478c0f55c0 2021-11-03T17:30:01.648 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.647+0000 7f47737fe700 1 -- 172.21.15.27:0/3587446509 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f4780019030 con 0x7f478c0f55c0 2021-11-03T17:30:01.648 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.647+0000 7f47737fe700 1 --2- 172.21.15.27:0/3587446509 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f47740640b0 0x7f4774066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:30:01.648 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.647+0000 7f478a7fc700 1 --2- 172.21.15.27:0/3587446509 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f47740640b0 0x7f4774066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:30:01.649 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.647+0000 7f47737fe700 1 -- 172.21.15.27:0/3587446509 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f47800805c0 con 0x7f478c0f55c0 2021-11-03T17:30:01.649 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.648+0000 7f478a7fc700 1 --2- 172.21.15.27:0/3587446509 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f47740640b0 0x7f4774066570 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f477c004630 tx=0x7f477c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:30:01.649 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.651+0000 7f47737fe700 1 -- 172.21.15.27:0/3587446509 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f4780052820 con 0x7f478c0f55c0 2021-11-03T17:30:01.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.854+0000 7f479158d700 1 -- 172.21.15.27:0/3587446509 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f476c005e80 con 0x7f478c0f55c0 2021-11-03T17:30:01.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.855+0000 7f47737fe700 1 -- 172.21.15.27:0/3587446509 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f4780017710 con 0x7f478c0f55c0 2021-11-03T17:30:01.851 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:30:01.851 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:30:01.851 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:30:01.852 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:30:01.852 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:30:01.852 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:30:01.852 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:30:01.853 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:30:01.853 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:30:01.853 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:30:01.853 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:30:01.853 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:30:01.854 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:30:01.854 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:30:01.854 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:30:01.854 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:30:01.855 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:30:01.855 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:30:01.855 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:30:01.855 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:30:01.855 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:30:01.856 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:30:01.856 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:30:01.856 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:30:01.856 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:30:01.857 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:30:01.857 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:30:01.857 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:30:01.857 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:30:01.858 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:30:01.858 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:30:01.858 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:30:01.858 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:30:01.859 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:30:01.859 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:30:01.860 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:30:01.860 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:30:01.861 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:30:01.861 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:30:01.862 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.857+0000 7f479158d700 1 -- 172.21.15.27:0/3587446509 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f47740640b0 msgr2=0x7f4774066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:30:01.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.857+0000 7f479158d700 1 --2- 172.21.15.27:0/3587446509 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f47740640b0 0x7f4774066570 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 rx=0x7f477c004630 tx=0x7f477c009b30).stop 2021-11-03T17:30:01.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.857+0000 7f479158d700 1 -- 172.21.15.27:0/3587446509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f478c0f55c0 msgr2=0x7f478c109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:30:01.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.857+0000 7f479158d700 1 --2- 172.21.15.27:0/3587446509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f478c0f55c0 0x7f478c109c50 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f478000c530 tx=0x7f4780007e10).stop 2021-11-03T17:30:01.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.857+0000 7f479158d700 1 -- 172.21.15.27:0/3587446509 shutdown_connections 2021-11-03T17:30:01.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.857+0000 7f479158d700 1 --2- 172.21.15.27:0/3587446509 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f47740640b0 0x7f4774066570 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:01.864 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.857+0000 7f479158d700 1 --2- 172.21.15.27:0/3587446509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f478c0f55c0 0x7f478c109c50 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:01.864 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.857+0000 7f479158d700 1 --2- 172.21.15.27:0/3587446509 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f478c0f6860 0x7f478c10c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:01.864 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.857+0000 7f479158d700 1 -- 172.21.15.27:0/3587446509 >> 172.21.15.27:0/3587446509 conn(0x7f478c0f0d40 msgr2=0x7f478c0fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:30:01.864 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.857+0000 7f479158d700 1 -- 172.21.15.27:0/3587446509 shutdown_connections 2021-11-03T17:30:01.865 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:01.857+0000 7f479158d700 1 -- 172.21.15.27:0/3587446509 wait complete. 2021-11-03T17:30:01.866 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:30:02.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:02 smithi115 ceph-mon[65453]: from='client.34584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:30:02.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:02 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/478661060' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:02.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:02 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3587446509' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:30:02.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:02 smithi027 ceph-mon[97244]: from='client.34584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:30:02.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:02 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/478661060' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:02.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:02 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3587446509' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:30:03.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:03 smithi027 ceph-mon[97244]: from='client.34588 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:30:03.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:03 smithi027 ceph-mon[97244]: pgmap v462: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:03.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:03 smithi115 ceph-mon[65453]: from='client.34588 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:30:03.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:03 smithi115 ceph-mon[65453]: pgmap v462: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:04.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:04 smithi115 ceph-mon[65453]: pgmap v463: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:04.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:04 smithi027 ceph-mon[97244]: pgmap v463: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:05.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 847.379016 secs 2021-11-03T17:30:05.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 847.379016 secs 2021-11-03T17:30:06.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:06 smithi115 ceph-mon[65453]: pgmap v464: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:06.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:06 smithi027 ceph-mon[97244]: pgmap v464: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:09.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:09 smithi115 ceph-mon[65453]: pgmap v465: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:09.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:09 smithi027 ceph-mon[97244]: pgmap v465: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:10.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 852.379127 secs 2021-11-03T17:30:10.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:30:10.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:30:10.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:30:10.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:10.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:10.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:10.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:10.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 852.379127 secs 2021-11-03T17:30:10.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:30:10.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:30:10.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:30:10.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:10.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:10.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:10.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:11.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:11 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:30:11.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:11 smithi115 ceph-mon[65453]: pgmap v466: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:11.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:11 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:30:11.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:11 smithi027 ceph-mon[97244]: pgmap v466: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:13.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:13 smithi027 ceph-mon[97244]: pgmap v467: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:13.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:13 smithi115 ceph-mon[65453]: pgmap v467: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:15.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:15 smithi115 ceph-mon[65453]: pgmap v468: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:15.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 857.379267 secs 2021-11-03T17:30:15.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:15 smithi027 ceph-mon[97244]: pgmap v468: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:15.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 857.379267 secs 2021-11-03T17:30:17.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:17 smithi115 ceph-mon[65453]: pgmap v469: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:17.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:17 smithi027 ceph-mon[97244]: pgmap v469: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:19.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:19 smithi115 ceph-mon[65453]: pgmap v470: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:19.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:19 smithi027 ceph-mon[97244]: pgmap v470: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:20.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 862.379406 secs 2021-11-03T17:30:20.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:30:20.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:30:20.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:30:20.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:20.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:20.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:20.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:20.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 862.379406 secs 2021-11-03T17:30:20.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:30:20.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:30:20.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:30:20.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:20.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:20.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:20.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:21.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:21 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:30:21.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:21 smithi115 ceph-mon[65453]: pgmap v471: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:21.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:21 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:30:21.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:21 smithi027 ceph-mon[97244]: pgmap v471: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:23.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:23 smithi027 ceph-mon[97244]: pgmap v472: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:23.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:23 smithi115 ceph-mon[65453]: pgmap v472: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:24.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:24 smithi115 ceph-mon[65453]: pgmap v473: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:24.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:24 smithi027 ceph-mon[97244]: pgmap v473: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:25.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 867.379545 secs 2021-11-03T17:30:25.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 867.379545 secs 2021-11-03T17:30:26.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:26 smithi115 ceph-mon[65453]: pgmap v474: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:26.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:26 smithi027 ceph-mon[97244]: pgmap v474: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:29.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:29 smithi115 ceph-mon[65453]: pgmap v475: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:29.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:29 smithi027 ceph-mon[97244]: pgmap v475: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:30.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 872.379619 secs 2021-11-03T17:30:30.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:30:30.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:30:30.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:30:30.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:30.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:30.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:30.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:30.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 872.379619 secs 2021-11-03T17:30:30.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:30:30.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:30:30.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:30:30.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:30.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:30.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:30.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:31.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:31 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:30:31.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:31 smithi115 ceph-mon[65453]: pgmap v476: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:31.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:31 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:30:31.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:31 smithi027 ceph-mon[97244]: pgmap v476: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:31.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:31.972+0000 7f3fb1e0c700 1 -- 172.21.15.27:0/1620057900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fac0fbbd0 msgr2=0x7f3fac0fdfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:30:31.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:31.972+0000 7f3fb1e0c700 1 --2- 172.21.15.27:0/1620057900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fac0fbbd0 0x7f3fac0fdfc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f3f94004660 tx=0x7f3f94009b30).stop 2021-11-03T17:30:31.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:31.972+0000 7f3fb1e0c700 1 -- 172.21.15.27:0/1620057900 shutdown_connections 2021-11-03T17:30:31.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:31.972+0000 7f3fb1e0c700 1 --2- 172.21.15.27:0/1620057900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fac0fbbd0 0x7f3fac0fdfc0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:31.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:31.972+0000 7f3fb1e0c700 1 --2- 172.21.15.27:0/1620057900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3fac0602d0 0x7f3fac0fb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:31.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:31.972+0000 7f3fb1e0c700 1 -- 172.21.15.27:0/1620057900 >> 172.21.15.27:0/1620057900 conn(0x7f3fac0f0d40 msgr2=0x7f3fac0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:30:31.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:31.972+0000 7f3fb1e0c700 1 -- 172.21.15.27:0/1620057900 shutdown_connections 2021-11-03T17:30:31.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:31.972+0000 7f3fb1e0c700 1 -- 172.21.15.27:0/1620057900 wait complete. 2021-11-03T17:30:31.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:31.973+0000 7f3fb1e0c700 1 Processor -- start 2021-11-03T17:30:31.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:31.973+0000 7f3fb1e0c700 1 -- start start 2021-11-03T17:30:31.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:31.973+0000 7f3fb1e0c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3fac0602d0 0x7f3fac109120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:30:31.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:31.973+0000 7f3fb1e0c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fac0fbbd0 0x7f3fac109610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:30:31.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:31.973+0000 7f3fb1e0c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac109bc0 con 0x7f3fac0fbbd0 2021-11-03T17:30:31.973 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:31.973+0000 7f3fb1e0c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fac109d00 con 0x7f3fac0602d0 2021-11-03T17:30:31.973 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:31.973+0000 7f3faaffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fac0fbbd0 0x7f3fac109610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:30:31.973 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:31.973+0000 7f3faaffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fac0fbbd0 0x7f3fac109610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57812/0 (socket says 172.21.15.27:57812) 2021-11-03T17:30:31.973 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:31.973+0000 7f3faaffd700 1 -- 172.21.15.27:0/461433398 learned_addr learned my addr 172.21.15.27:0/461433398 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:30:31.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:31.973+0000 7f3fab7fe700 1 --2- 172.21.15.27:0/461433398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3fac0602d0 0x7f3fac109120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:30:31.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:31.973+0000 7f3faaffd700 1 -- 172.21.15.27:0/461433398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3fac0602d0 msgr2=0x7f3fac109120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:30:31.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:31.973+0000 7f3faaffd700 1 --2- 172.21.15.27:0/461433398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3fac0602d0 0x7f3fac109120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:31.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:31.973+0000 7f3faaffd700 1 -- 172.21.15.27:0/461433398 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f94005040 con 0x7f3fac0fbbd0 2021-11-03T17:30:31.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:31.974+0000 7f3faaffd700 1 --2- 172.21.15.27:0/461433398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fac0fbbd0 0x7f3fac109610 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f3f94013a70 tx=0x7f3f94009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:30:31.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:31.974+0000 7f3fa3fff700 1 -- 172.21.15.27:0/461433398 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f94018070 con 0x7f3fac0fbbd0 2021-11-03T17:30:31.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:31.974+0000 7f3fa3fff700 1 -- 172.21.15.27:0/461433398 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f3f940097d0 con 0x7f3fac0fbbd0 2021-11-03T17:30:31.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:31.974+0000 7f3fb1e0c700 1 -- 172.21.15.27:0/461433398 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fac10a640 con 0x7f3fac0fbbd0 2021-11-03T17:30:31.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:31.974+0000 7f3fa3fff700 1 -- 172.21.15.27:0/461433398 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f9401c3c0 con 0x7f3fac0fbbd0 2021-11-03T17:30:31.977 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:31.974+0000 7f3fb1e0c700 1 -- 172.21.15.27:0/461433398 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fac10aba0 con 0x7f3fac0fbbd0 2021-11-03T17:30:31.978 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:31.975+0000 7f3fb1e0c700 1 -- 172.21.15.27:0/461433398 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fac045ad0 con 0x7f3fac0fbbd0 2021-11-03T17:30:31.978 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:31.975+0000 7f3fa3fff700 1 -- 172.21.15.27:0/461433398 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f3f94007ba0 con 0x7f3fac0fbbd0 2021-11-03T17:30:31.978 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:31.976+0000 7f3fa3fff700 1 --2- 172.21.15.27:0/461433398 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3f980640b0 0x7f3f98066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:30:31.979 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:31.976+0000 7f3fa3fff700 1 -- 172.21.15.27:0/461433398 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f3f940824d0 con 0x7f3fac0fbbd0 2021-11-03T17:30:31.979 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:31.976+0000 7f3fab7fe700 1 --2- 172.21.15.27:0/461433398 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3f980640b0 0x7f3f98066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:30:31.979 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:31.976+0000 7f3fab7fe700 1 --2- 172.21.15.27:0/461433398 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3f980640b0 0x7f3f98066570 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f3fa4007d70 tx=0x7f3fa4004d00).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:30:31.979 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:31.980+0000 7f3fa3fff700 1 -- 172.21.15.27:0/461433398 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f3f940547d0 con 0x7f3fac0fbbd0 2021-11-03T17:30:32.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.162+0000 7f3fb1e0c700 1 -- 172.21.15.27:0/461433398 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3fac1010f0 con 0x7f3f980640b0 2021-11-03T17:30:32.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.163+0000 7f3fa3fff700 1 -- 172.21.15.27:0/461433398 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f3fac1010f0 con 0x7f3f980640b0 2021-11-03T17:30:32.160 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.165+0000 7f3fb1e0c700 1 -- 172.21.15.27:0/461433398 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3f980640b0 msgr2=0x7f3f98066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:30:32.160 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.165+0000 7f3fb1e0c700 1 --2- 172.21.15.27:0/461433398 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3f980640b0 0x7f3f98066570 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7f3fa4007d70 tx=0x7f3fa4004d00).stop 2021-11-03T17:30:32.161 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.165+0000 7f3fb1e0c700 1 -- 172.21.15.27:0/461433398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fac0fbbd0 msgr2=0x7f3fac109610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:30:32.161 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.165+0000 7f3fb1e0c700 1 --2- 172.21.15.27:0/461433398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fac0fbbd0 0x7f3fac109610 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f3f94013a70 tx=0x7f3f94009b30).stop 2021-11-03T17:30:32.161 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.165+0000 7f3fb1e0c700 1 -- 172.21.15.27:0/461433398 shutdown_connections 2021-11-03T17:30:32.161 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.165+0000 7f3fb1e0c700 1 --2- 172.21.15.27:0/461433398 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3f980640b0 0x7f3f98066570 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:32.161 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.165+0000 7f3fb1e0c700 1 --2- 172.21.15.27:0/461433398 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3fac0602d0 0x7f3fac109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:32.162 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.165+0000 7f3fb1e0c700 1 --2- 172.21.15.27:0/461433398 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3fac0fbbd0 0x7f3fac109610 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:32.162 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.165+0000 7f3fb1e0c700 1 -- 172.21.15.27:0/461433398 >> 172.21.15.27:0/461433398 conn(0x7f3fac0f0d40 msgr2=0x7f3fac0f3130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:30:32.162 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.165+0000 7f3fb1e0c700 1 -- 172.21.15.27:0/461433398 shutdown_connections 2021-11-03T17:30:32.162 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.165+0000 7f3fb1e0c700 1 -- 172.21.15.27:0/461433398 wait complete. 2021-11-03T17:30:32.170 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:30:32.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.265+0000 7ff294dd2700 1 -- 172.21.15.27:0/2294941177 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2900f7960 msgr2=0x7ff2900f7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:30:32.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.265+0000 7ff294dd2700 1 --2- 172.21.15.27:0/2294941177 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2900f7960 0x7ff2900f7dc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7ff284004660 tx=0x7ff284009b30).stop 2021-11-03T17:30:32.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.265+0000 7ff294dd2700 1 -- 172.21.15.27:0/2294941177 shutdown_connections 2021-11-03T17:30:32.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.265+0000 7ff294dd2700 1 --2- 172.21.15.27:0/2294941177 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2900f7960 0x7ff2900f7dc0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:32.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.265+0000 7ff294dd2700 1 --2- 172.21.15.27:0/2294941177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2900f66c0 0x7ff2900f6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:30:32.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.265+0000 7ff294dd2700 1 -- 172.21.15.27:0/2294941177 >> 172.21.15.27:0/2294941177 conn(0x7ff2900f1e20 msgr2=0x7ff2900f4240 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:30:32.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.266+0000 7ff294dd2700 1 -- 172.21.15.27:0/2294941177 shutdown_connections 2021-11-03T17:30:32.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.266+0000 7ff294dd2700 1 -- 172.21.15.27:0/2294941177 wait complete. 2021-11-03T17:30:32.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.266+0000 7ff294dd2700 1 Processor -- start 2021-11-03T17:30:32.265 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.266+0000 7ff294dd2700 1 -- start start 2021-11-03T17:30:32.265 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.266+0000 7ff294dd2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2900f66c0 0x7ff290106f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:30:32.265 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.266+0000 7ff294dd2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2900f7960 0x7ff290107480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:30:32.265 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.266+0000 7ff294dd2700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff290107ac0 con 0x7ff2900f66c0 2021-11-03T17:30:32.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.266+0000 7ff294dd2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff290108370 con 0x7ff2900f7960 2021-11-03T17:30:32.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.266+0000 7ff28e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2900f66c0 0x7ff290106f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:30:32.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.266+0000 7ff28e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2900f66c0 0x7ff290106f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57820/0 (socket says 172.21.15.27:57820) 2021-11-03T17:30:32.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.266+0000 7ff28e59c700 1 -- 172.21.15.27:0/4149157127 learned_addr learned my addr 172.21.15.27:0/4149157127 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:30:32.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.266+0000 7ff28dd9b700 1 --2- 172.21.15.27:0/4149157127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2900f7960 0x7ff290107480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:30:32.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.267+0000 7ff28e59c700 1 -- 172.21.15.27:0/4149157127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2900f7960 msgr2=0x7ff290107480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:30:32.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.267+0000 7ff28e59c700 1 --2- 172.21.15.27:0/4149157127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2900f7960 0x7ff290107480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:32.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.267+0000 7ff28e59c700 1 -- 172.21.15.27:0/4149157127 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff284005040 con 0x7ff2900f66c0 2021-11-03T17:30:32.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.267+0000 7ff28e59c700 1 --2- 172.21.15.27:0/4149157127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2900f66c0 0x7ff290106f90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7ff2800093f0 tx=0x7ff280006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:30:32.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.267+0000 7ff27effd700 1 -- 172.21.15.27:0/4149157127 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff280013640 con 0x7ff2900f66c0 2021-11-03T17:30:32.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.267+0000 7ff27effd700 1 -- 172.21.15.27:0/4149157127 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7ff280013c50 con 0x7ff2900f66c0 2021-11-03T17:30:32.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.267+0000 7ff294dd2700 1 -- 172.21.15.27:0/4149157127 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff290108650 con 0x7ff2900f66c0 2021-11-03T17:30:32.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.267+0000 7ff27effd700 1 -- 172.21.15.27:0/4149157127 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff280012c70 con 0x7ff2900f66c0 2021-11-03T17:30:32.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.267+0000 7ff294dd2700 1 -- 172.21.15.27:0/4149157127 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff290108c10 con 0x7ff2900f66c0 2021-11-03T17:30:32.270 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.268+0000 7ff294dd2700 1 -- 172.21.15.27:0/4149157127 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2900f5ab0 con 0x7ff2900f66c0 2021-11-03T17:30:32.270 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.269+0000 7ff27effd700 1 -- 172.21.15.27:0/4149157127 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7ff28001a060 con 0x7ff2900f66c0 2021-11-03T17:30:32.271 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.269+0000 7ff27effd700 1 --2- 172.21.15.27:0/4149157127 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff2780640b0 0x7ff278066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:30:32.271 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.269+0000 7ff27effd700 1 -- 172.21.15.27:0/4149157127 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7ff280080970 con 0x7ff2900f66c0 2021-11-03T17:30:32.271 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.269+0000 7ff28dd9b700 1 --2- 172.21.15.27:0/4149157127 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff2780640b0 0x7ff278066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:30:32.271 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.270+0000 7ff28dd9b700 1 --2- 172.21.15.27:0/4149157127 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff2780640b0 0x7ff278066570 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7ff284004630 tx=0x7ff284009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:30:32.272 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.273+0000 7ff27effd700 1 -- 172.21.15.27:0/4149157127 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7ff280052b20 con 0x7ff2900f66c0 2021-11-03T17:30:32.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.447+0000 7ff294dd2700 1 -- 172.21.15.27:0/4149157127 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2900fefe0 con 0x7ff2780640b0 2021-11-03T17:30:32.457 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.462+0000 7ff27effd700 1 -- 172.21.15.27:0/4149157127 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7ff2900fefe0 con 0x7ff2780640b0 2021-11-03T17:30:32.457 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:30:32.458 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (22m) 2m ago 23m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:30:32.458 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (13m) 2m ago 23m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:30:32.458 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (13m) 53s ago 22m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:30:32.458 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (21m) 2m ago 23m 40.2M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:30:32.459 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (15m) 2m ago 18m 18.2M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:30:32.460 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (18m) 2m ago 18m 3841M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:30:32.460 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (14m) 53s ago 18m 17.6M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:30:32.461 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (15m) 53s ago 18m 19.3M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:30:32.461 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (15m) 2m ago 24m 497M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:30:32.461 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (15m) 53s ago 22m 415M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:30:32.462 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (14m) 2m ago 24m 119M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:30:32.462 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (14m) 53s ago 22m 71.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:30:32.463 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (22m) 2m ago 22m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:30:32.463 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (22m) 53s ago 22m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:30:32.463 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (13m) 2m ago 21m 290M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:30:32.464 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (12m) 2m ago 21m 317M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:30:32.464 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (12m) 2m ago 21m 229M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:30:32.465 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (11m) 53s ago 20m 329M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:30:32.465 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (11m) 53s ago 20m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:30:32.465 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (11m) 53s ago 20m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:30:32.466 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (21m) 2m ago 22m 48.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:30:32.467 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.464+0000 7ff294dd2700 1 -- 172.21.15.27:0/4149157127 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff2780640b0 msgr2=0x7ff278066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:30:32.467 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.464+0000 7ff294dd2700 1 --2- 172.21.15.27:0/4149157127 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff2780640b0 0x7ff278066570 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7ff284004630 tx=0x7ff284009b30).stop 2021-11-03T17:30:32.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.464+0000 7ff294dd2700 1 -- 172.21.15.27:0/4149157127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2900f66c0 msgr2=0x7ff290106f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:30:32.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.464+0000 7ff294dd2700 1 --2- 172.21.15.27:0/4149157127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2900f66c0 0x7ff290106f90 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7ff2800093f0 tx=0x7ff280006ec0).stop 2021-11-03T17:30:32.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.464+0000 7ff294dd2700 1 -- 172.21.15.27:0/4149157127 shutdown_connections 2021-11-03T17:30:32.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.464+0000 7ff294dd2700 1 --2- 172.21.15.27:0/4149157127 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff2780640b0 0x7ff278066570 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:32.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.464+0000 7ff294dd2700 1 --2- 172.21.15.27:0/4149157127 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2900f66c0 0x7ff290106f90 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:32.469 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.464+0000 7ff294dd2700 1 --2- 172.21.15.27:0/4149157127 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2900f7960 0x7ff290107480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:32.469 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.464+0000 7ff294dd2700 1 -- 172.21.15.27:0/4149157127 >> 172.21.15.27:0/4149157127 conn(0x7ff2900f1e20 msgr2=0x7ff2900fb3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:30:32.469 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.464+0000 7ff294dd2700 1 -- 172.21.15.27:0/4149157127 shutdown_connections 2021-11-03T17:30:32.469 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.464+0000 7ff294dd2700 1 -- 172.21.15.27:0/4149157127 wait complete. 2021-11-03T17:30:32.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.564+0000 7f7af950f700 1 -- 172.21.15.27:0/1221146579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7af40f6a10 msgr2=0x7f7af40f6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:30:32.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.564+0000 7f7af950f700 1 --2- 172.21.15.27:0/1221146579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7af40f6a10 0x7f7af40f6e30 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f7ae4004660 tx=0x7f7ae4009b30).stop 2021-11-03T17:30:32.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.564+0000 7f7af950f700 1 -- 172.21.15.27:0/1221146579 shutdown_connections 2021-11-03T17:30:32.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.564+0000 7f7af950f700 1 --2- 172.21.15.27:0/1221146579 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7af40f73b0 0x7f7af40f4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:32.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.564+0000 7f7af950f700 1 --2- 172.21.15.27:0/1221146579 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7af40f6a10 0x7f7af40f6e30 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:32.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.564+0000 7f7af950f700 1 -- 172.21.15.27:0/1221146579 >> 172.21.15.27:0/1221146579 conn(0x7f7af40f0320 msgr2=0x7f7af40f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:30:32.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.564+0000 7f7af950f700 1 -- 172.21.15.27:0/1221146579 shutdown_connections 2021-11-03T17:30:32.568 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.565+0000 7f7af950f700 1 -- 172.21.15.27:0/1221146579 wait complete. 2021-11-03T17:30:32.568 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.565+0000 7f7af950f700 1 Processor -- start 2021-11-03T17:30:32.569 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.565+0000 7f7af950f700 1 -- start start 2021-11-03T17:30:32.569 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.565+0000 7f7af950f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7af40f6a10 0x7f7af4108f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:30:32.570 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.565+0000 7f7af950f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7af40f73b0 0x7f7af4109450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:30:32.570 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.565+0000 7f7af950f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af4109a00 con 0x7f7af40f73b0 2021-11-03T17:30:32.571 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.565+0000 7f7af950f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af4109b40 con 0x7f7af40f6a10 2021-11-03T17:30:32.571 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.565+0000 7f7af37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7af40f73b0 0x7f7af4109450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:30:32.571 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.565+0000 7f7af37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7af40f73b0 0x7f7af4109450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57832/0 (socket says 172.21.15.27:57832) 2021-11-03T17:30:32.572 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.565+0000 7f7af37fe700 1 -- 172.21.15.27:0/3161546259 learned_addr learned my addr 172.21.15.27:0/3161546259 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:30:32.572 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.566+0000 7f7af3fff700 1 --2- 172.21.15.27:0/3161546259 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7af40f6a10 0x7f7af4108f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:30:32.573 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.566+0000 7f7af37fe700 1 -- 172.21.15.27:0/3161546259 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7af40f6a10 msgr2=0x7f7af4108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:30:32.573 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.566+0000 7f7af37fe700 1 --2- 172.21.15.27:0/3161546259 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7af40f6a10 0x7f7af4108f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:32.574 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.566+0000 7f7af37fe700 1 -- 172.21.15.27:0/3161546259 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae4005040 con 0x7f7af40f73b0 2021-11-03T17:30:32.574 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.566+0000 7f7af37fe700 1 --2- 172.21.15.27:0/3161546259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7af40f73b0 0x7f7af4109450 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f7ae80026d0 tx=0x7f7ae8009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:30:32.575 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.566+0000 7f7af0ff9700 1 -- 172.21.15.27:0/3161546259 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ae8009320 con 0x7f7af40f73b0 2021-11-03T17:30:32.575 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.566+0000 7f7af0ff9700 1 -- 172.21.15.27:0/3161546259 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f7ae800f040 con 0x7f7af40f73b0 2021-11-03T17:30:32.575 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.566+0000 7f7af950f700 1 -- 172.21.15.27:0/3161546259 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af410a4e0 con 0x7f7af40f73b0 2021-11-03T17:30:32.576 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.566+0000 7f7af0ff9700 1 -- 172.21.15.27:0/3161546259 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ae80147e0 con 0x7f7af40f73b0 2021-11-03T17:30:32.576 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.566+0000 7f7af950f700 1 -- 172.21.15.27:0/3161546259 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af410aaa0 con 0x7f7af40f73b0 2021-11-03T17:30:32.577 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.567+0000 7f7af950f700 1 -- 172.21.15.27:0/3161546259 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7af4045ad0 con 0x7f7af40f73b0 2021-11-03T17:30:32.577 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.567+0000 7f7af0ff9700 1 -- 172.21.15.27:0/3161546259 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f7ae8009480 con 0x7f7af40f73b0 2021-11-03T17:30:32.578 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.568+0000 7f7af0ff9700 1 --2- 172.21.15.27:0/3161546259 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7adc064100 0x7f7adc0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:30:32.578 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.568+0000 7f7af0ff9700 1 -- 172.21.15.27:0/3161546259 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f7ae8080f10 con 0x7f7af40f73b0 2021-11-03T17:30:32.578 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.568+0000 7f7af3fff700 1 --2- 172.21.15.27:0/3161546259 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7adc064100 0x7f7adc0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:30:32.579 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.568+0000 7f7af3fff700 1 --2- 172.21.15.27:0/3161546259 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7adc064100 0x7f7adc0665c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f7ae4004630 tx=0x7f7ae4009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:30:32.580 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.572+0000 7f7af0ff9700 1 -- 172.21.15.27:0/3161546259 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f7ae8053210 con 0x7f7af40f73b0 2021-11-03T17:30:32.800 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.805+0000 7f7af950f700 1 -- 172.21.15.27:0/3161546259 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7af410ad20 con 0x7f7af40f73b0 2021-11-03T17:30:32.801 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.806+0000 7f7af0ff9700 1 -- 172.21.15.27:0/3161546259 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f7ae8052e00 con 0x7f7af40f73b0 2021-11-03T17:30:32.801 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:30:32.802 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:30:32.802 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:30:32.802 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:30:32.802 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:30:32.803 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:30:32.803 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:30:32.803 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:30:32.803 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:30:32.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:30:32.804 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:30:32.804 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:30:32.804 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:30:32.804 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:30:32.805 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:30:32.805 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:30:32.805 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:30:32.805 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:30:32.807 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.807+0000 7f7af950f700 1 -- 172.21.15.27:0/3161546259 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7adc064100 msgr2=0x7f7adc0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:30:32.807 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.807+0000 7f7af950f700 1 --2- 172.21.15.27:0/3161546259 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7adc064100 0x7f7adc0665c0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f7ae4004630 tx=0x7f7ae4009b30).stop 2021-11-03T17:30:32.807 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.807+0000 7f7af950f700 1 -- 172.21.15.27:0/3161546259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7af40f73b0 msgr2=0x7f7af4109450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:30:32.807 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.807+0000 7f7af950f700 1 --2- 172.21.15.27:0/3161546259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7af40f73b0 0x7f7af4109450 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f7ae80026d0 tx=0x7f7ae8009de0).stop 2021-11-03T17:30:32.808 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.808+0000 7f7af950f700 1 -- 172.21.15.27:0/3161546259 shutdown_connections 2021-11-03T17:30:32.808 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.808+0000 7f7af950f700 1 --2- 172.21.15.27:0/3161546259 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7adc064100 0x7f7adc0665c0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:32.808 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.808+0000 7f7af950f700 1 --2- 172.21.15.27:0/3161546259 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7af40f6a10 0x7f7af4108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:32.808 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.808+0000 7f7af950f700 1 --2- 172.21.15.27:0/3161546259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7af40f73b0 0x7f7af4109450 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:32.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.808+0000 7f7af950f700 1 -- 172.21.15.27:0/3161546259 >> 172.21.15.27:0/3161546259 conn(0x7f7af40f0320 msgr2=0x7f7af40fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:30:32.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.808+0000 7f7af950f700 1 -- 172.21.15.27:0/3161546259 shutdown_connections 2021-11-03T17:30:32.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.808+0000 7f7af950f700 1 -- 172.21.15.27:0/3161546259 wait complete. 2021-11-03T17:30:32.905 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.908+0000 7f76f9806700 1 -- 172.21.15.27:0/2391431801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76f40f6870 msgr2=0x7f76f40f6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:30:32.905 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.908+0000 7f76f9806700 1 --2- 172.21.15.27:0/2391431801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76f40f6870 0x7f76f40f6cd0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f76e4004660 tx=0x7f76e4009b30).stop 2021-11-03T17:30:32.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.908+0000 7f76f9806700 1 -- 172.21.15.27:0/2391431801 shutdown_connections 2021-11-03T17:30:32.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.908+0000 7f76f9806700 1 --2- 172.21.15.27:0/2391431801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76f40f6870 0x7f76f40f6cd0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:32.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.908+0000 7f76f9806700 1 --2- 172.21.15.27:0/2391431801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76f40f55d0 0x7f76f40f59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:32.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.908+0000 7f76f9806700 1 -- 172.21.15.27:0/2391431801 >> 172.21.15.27:0/2391431801 conn(0x7f76f40f0d30 msgr2=0x7f76f40f3150 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:30:32.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.908+0000 7f76f9806700 1 -- 172.21.15.27:0/2391431801 shutdown_connections 2021-11-03T17:30:32.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.908+0000 7f76f9806700 1 -- 172.21.15.27:0/2391431801 wait complete. 2021-11-03T17:30:32.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.909+0000 7f76f9806700 1 Processor -- start 2021-11-03T17:30:32.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.909+0000 7f76f9806700 1 -- start start 2021-11-03T17:30:32.908 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.909+0000 7f76f9806700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76f40f55d0 0x7f76f41090f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:30:32.908 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.909+0000 7f76f9806700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76f40f6870 0x7f76f41095e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:30:32.908 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.909+0000 7f76f9806700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f4109b90 con 0x7f76f40f55d0 2021-11-03T17:30:32.908 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.909+0000 7f76f9806700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f4109cd0 con 0x7f76f40f6870 2021-11-03T17:30:32.908 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.909+0000 7f76f2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76f40f55d0 0x7f76f41090f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:30:32.909 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.909+0000 7f76f2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76f40f55d0 0x7f76f41090f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57840/0 (socket says 172.21.15.27:57840) 2021-11-03T17:30:32.909 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.909+0000 7f76f2ffd700 1 -- 172.21.15.27:0/3478144179 learned_addr learned my addr 172.21.15.27:0/3478144179 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:30:32.909 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.909+0000 7f76f27fc700 1 --2- 172.21.15.27:0/3478144179 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76f40f6870 0x7f76f41095e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:30:32.909 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.909+0000 7f76f2ffd700 1 -- 172.21.15.27:0/3478144179 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76f40f6870 msgr2=0x7f76f41095e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:30:32.910 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.909+0000 7f76f2ffd700 1 --2- 172.21.15.27:0/3478144179 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76f40f6870 0x7f76f41095e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:32.910 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.909+0000 7f76f2ffd700 1 -- 172.21.15.27:0/3478144179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e4005040 con 0x7f76f40f55d0 2021-11-03T17:30:32.910 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.909+0000 7f76f27fc700 1 --2- 172.21.15.27:0/3478144179 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76f40f6870 0x7f76f41095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:30:32.910 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.909+0000 7f76f2ffd700 1 --2- 172.21.15.27:0/3478144179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76f40f55d0 0x7f76f41090f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f76e8002700 tx=0x7f76e8009e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:30:32.911 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.910+0000 7f76db7fe700 1 -- 172.21.15.27:0/3478144179 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76e8009320 con 0x7f76f40f55d0 2021-11-03T17:30:32.911 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.910+0000 7f76db7fe700 1 -- 172.21.15.27:0/3478144179 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f76e800f040 con 0x7f76f40f55d0 2021-11-03T17:30:32.911 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.910+0000 7f76f9806700 1 -- 172.21.15.27:0/3478144179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76f410a670 con 0x7f76f40f55d0 2021-11-03T17:30:32.911 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.910+0000 7f76db7fe700 1 -- 172.21.15.27:0/3478144179 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f76e80148e0 con 0x7f76f40f55d0 2021-11-03T17:30:32.912 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.910+0000 7f76f9806700 1 -- 172.21.15.27:0/3478144179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f410ac30 con 0x7f76f40f55d0 2021-11-03T17:30:32.913 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.911+0000 7f76f9806700 1 -- 172.21.15.27:0/3478144179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76f4045ad0 con 0x7f76f40f55d0 2021-11-03T17:30:32.913 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.911+0000 7f76db7fe700 1 -- 172.21.15.27:0/3478144179 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f76e8009480 con 0x7f76f40f55d0 2021-11-03T17:30:32.914 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.911+0000 7f76db7fe700 1 --2- 172.21.15.27:0/3478144179 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f76dc0640b0 0x7f76dc066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:30:32.914 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.912+0000 7f76db7fe700 1 -- 172.21.15.27:0/3478144179 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f76e8080a90 con 0x7f76f40f55d0 2021-11-03T17:30:32.915 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.912+0000 7f76f27fc700 1 --2- 172.21.15.27:0/3478144179 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f76dc0640b0 0x7f76dc066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:30:32.915 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.912+0000 7f76f27fc700 1 --2- 172.21.15.27:0/3478144179 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f76dc0640b0 0x7f76dc066570 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f76e4009da0 tx=0x7f76e4009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:30:32.915 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:32.916+0000 7f76db7fe700 1 -- 172.21.15.27:0/3478144179 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f76e8052d90 con 0x7f76f40f55d0 2021-11-03T17:30:33.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:33.122+0000 7f76f9806700 1 -- 172.21.15.27:0/3478144179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f76f410f840 con 0x7f76f40f55d0 2021-11-03T17:30:33.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:33.123+0000 7f76db7fe700 1 -- 172.21.15.27:0/3478144179 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f76e8052980 con 0x7f76f40f55d0 2021-11-03T17:30:33.118 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:30:33.119 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:30:33.119 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:30:33.119 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:30:33.119 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:30:33.120 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:30:33.120 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:30:33.120 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:30:33.120 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:30:33.120 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:30:33.121 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:30:33.121 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:30:33.121 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:30:33.121 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:30:33.122 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:30:33.122 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:30:33.122 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:30:33.122 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:30:33.122 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:30:33.123 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:30:33.123 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:30:33.123 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:30:33.123 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:30:33.124 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:30:33.124 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:30:33.124 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:30:33.124 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:30:33.125 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:30:33.125 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:30:33.125 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:30:33.125 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:30:33.126 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:30:33.126 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:30:33.126 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:30:33.126 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:30:33.127 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:30:33.127 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:30:33.127 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:30:33.127 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:30:33.128 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:33.125+0000 7f76f9806700 1 -- 172.21.15.27:0/3478144179 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f76dc0640b0 msgr2=0x7f76dc066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:30:33.129 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:33.125+0000 7f76f9806700 1 --2- 172.21.15.27:0/3478144179 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f76dc0640b0 0x7f76dc066570 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f76e4009da0 tx=0x7f76e4009b30).stop 2021-11-03T17:30:33.129 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:33.125+0000 7f76f9806700 1 -- 172.21.15.27:0/3478144179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76f40f55d0 msgr2=0x7f76f41090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:30:33.129 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:33.125+0000 7f76f9806700 1 --2- 172.21.15.27:0/3478144179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76f40f55d0 0x7f76f41090f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f76e8002700 tx=0x7f76e8009e00).stop 2021-11-03T17:30:33.129 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:33.125+0000 7f76f9806700 1 -- 172.21.15.27:0/3478144179 shutdown_connections 2021-11-03T17:30:33.130 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:33.125+0000 7f76f9806700 1 --2- 172.21.15.27:0/3478144179 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f76dc0640b0 0x7f76dc066570 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:33.130 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:33.125+0000 7f76f9806700 1 --2- 172.21.15.27:0/3478144179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f76f40f55d0 0x7f76f41090f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:33.130 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:33.125+0000 7f76f9806700 1 --2- 172.21.15.27:0/3478144179 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f76f40f6870 0x7f76f41095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:30:33.130 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:33.125+0000 7f76f9806700 1 -- 172.21.15.27:0/3478144179 >> 172.21.15.27:0/3478144179 conn(0x7f76f40f0d30 msgr2=0x7f76f405ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:30:33.131 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:33.125+0000 7f76f9806700 1 -- 172.21.15.27:0/3478144179 shutdown_connections 2021-11-03T17:30:33.131 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:30:33.125+0000 7f76f9806700 1 -- 172.21.15.27:0/3478144179 wait complete. 2021-11-03T17:30:33.131 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:30:33.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:33 smithi027 ceph-mon[97244]: pgmap v477: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:33.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:33 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3161546259' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:33.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:33 smithi115 ceph-mon[65453]: pgmap v477: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:33.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:33 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3161546259' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:34.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:34 smithi115 ceph-mon[65453]: from='client.34600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:30:34.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:34 smithi115 ceph-mon[65453]: from='client.34604 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:30:34.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:34 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3478144179' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:30:34.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:34 smithi027 ceph-mon[97244]: from='client.34600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:30:34.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:34 smithi027 ceph-mon[97244]: from='client.34604 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:30:34.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:34 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3478144179' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:30:35.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:35 smithi115 ceph-mon[65453]: pgmap v478: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:35.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 877.379758 secs 2021-11-03T17:30:35.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:35 smithi027 ceph-mon[97244]: pgmap v478: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:35.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 877.379758 secs 2021-11-03T17:30:37.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:37 smithi115 ceph-mon[65453]: pgmap v479: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:37.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:37 smithi027 ceph-mon[97244]: pgmap v479: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:39.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:39 smithi115 ceph-mon[65453]: pgmap v480: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:39.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:39 smithi027 ceph-mon[97244]: pgmap v480: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:40.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 882.379820 secs 2021-11-03T17:30:40.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:30:40.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:30:40.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:30:40.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:40.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:40.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:40.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:40.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 882.379820 secs 2021-11-03T17:30:40.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:30:40.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:30:40.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:30:40.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:40.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:40.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:40.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:41.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:41 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:30:41.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:41 smithi115 ceph-mon[65453]: pgmap v481: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:41.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:41 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:30:41.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:41 smithi027 ceph-mon[97244]: pgmap v481: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:43.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:43 smithi027 ceph-mon[97244]: pgmap v482: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:43.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:43 smithi115 ceph-mon[65453]: pgmap v482: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:45.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:45 smithi115 ceph-mon[65453]: pgmap v483: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:45.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 887.379960 secs 2021-11-03T17:30:45.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:45 smithi027 ceph-mon[97244]: pgmap v483: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:45.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 887.379960 secs 2021-11-03T17:30:47.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:47 smithi115 ceph-mon[65453]: pgmap v484: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:47.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:47 smithi027 ceph-mon[97244]: pgmap v484: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:48.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:48 smithi115 ceph-mon[65453]: pgmap v485: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:48.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:48 smithi027 ceph-mon[97244]: pgmap v485: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:49.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:49 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 892.380092 secs 2021-11-03T17:30:50.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:49 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 892.380092 secs 2021-11-03T17:30:50.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:30:50.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:30:50.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:30:50.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:50.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:50.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:50.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:50.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:50 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:30:50.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:50 smithi115 ceph-mon[65453]: pgmap v486: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:51.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:30:51.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:30:51.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:30:51.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:51.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:51.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:51.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:30:51.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:50 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:30:51.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:50 smithi027 ceph-mon[97244]: pgmap v486: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:53.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:53 smithi027 ceph-mon[97244]: pgmap v487: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:53.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:53 smithi115 ceph-mon[65453]: pgmap v487: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:55.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:55 smithi115 ceph-mon[65453]: pgmap v488: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:55.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 897.380220 secs 2021-11-03T17:30:55.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:55 smithi027 ceph-mon[97244]: pgmap v488: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:55.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 897.380220 secs 2021-11-03T17:30:56.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:30:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:30:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:30:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:30:57.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:57 smithi115 ceph-mon[65453]: pgmap v489: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:57.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:57 smithi027 ceph-mon[97244]: pgmap v489: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:59.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:30:59 smithi115 ceph-mon[65453]: pgmap v490: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:30:59.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:30:59 smithi027 ceph-mon[97244]: pgmap v490: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:00.188 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 902.380360 secs 2021-11-03T17:31:00.189 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:31:00.204 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 902.380360 secs 2021-11-03T17:31:00.205 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:31:01.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:01 smithi115 ceph-mon[65453]: poll took 60190.942 ms: 1 events 2021-11-03T17:31:01.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:01 smithi115 ceph-mon[65453]: pgmap v491: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:01.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:31:01.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:31:01.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:01.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:01.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:01.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:01.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:01 smithi027 ceph-mon[97244]: poll took 60190.942 ms: 1 events 2021-11-03T17:31:01.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:01 smithi027 ceph-mon[97244]: pgmap v491: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:01.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:31:01.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:31:01.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:01.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:01.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:01.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:02.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:02 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:31:02.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:02 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:31:03.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.242+0000 7f3e2b375700 1 -- 172.21.15.27:0/2722778072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e240f66e0 msgr2=0x7f3e240f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:31:03.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.242+0000 7f3e2b375700 1 --2- 172.21.15.27:0/2722778072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e240f66e0 0x7f3e240f6b40 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f3e14004660 tx=0x7f3e14009b30).stop 2021-11-03T17:31:03.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.242+0000 7f3e2b375700 1 -- 172.21.15.27:0/2722778072 shutdown_connections 2021-11-03T17:31:03.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.242+0000 7f3e2b375700 1 --2- 172.21.15.27:0/2722778072 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e240f66e0 0x7f3e240f6b40 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:03.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.242+0000 7f3e2b375700 1 --2- 172.21.15.27:0/2722778072 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e240f5440 0x7f3e240f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:03.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.242+0000 7f3e2b375700 1 -- 172.21.15.27:0/2722778072 >> 172.21.15.27:0/2722778072 conn(0x7f3e240f0ba0 msgr2=0x7f3e240f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:31:03.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.242+0000 7f3e2b375700 1 -- 172.21.15.27:0/2722778072 shutdown_connections 2021-11-03T17:31:03.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.242+0000 7f3e2b375700 1 -- 172.21.15.27:0/2722778072 wait complete. 2021-11-03T17:31:03.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.243+0000 7f3e2b375700 1 Processor -- start 2021-11-03T17:31:03.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.243+0000 7f3e2b375700 1 -- start start 2021-11-03T17:31:03.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.243+0000 7f3e2b375700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e240f5440 0x7f3e24106dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:31:03.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.243+0000 7f3e2b375700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e240f66e0 0x7f3e241072c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:31:03.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.243+0000 7f3e2b375700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e241078c0 con 0x7f3e240f66e0 2021-11-03T17:31:03.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.243+0000 7f3e2b375700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e24108100 con 0x7f3e240f5440 2021-11-03T17:31:03.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.243+0000 7f3e29b72700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e240f66e0 0x7f3e241072c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:31:03.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.243+0000 7f3e29b72700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e240f66e0 0x7f3e241072c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57874/0 (socket says 172.21.15.27:57874) 2021-11-03T17:31:03.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.243+0000 7f3e29b72700 1 -- 172.21.15.27:0/4096476818 learned_addr learned my addr 172.21.15.27:0/4096476818 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:31:03.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.243+0000 7f3e2a373700 1 --2- 172.21.15.27:0/4096476818 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e240f5440 0x7f3e24106dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:31:03.242 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.243+0000 7f3e29b72700 1 -- 172.21.15.27:0/4096476818 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e240f5440 msgr2=0x7f3e24106dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:31:03.242 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.243+0000 7f3e29b72700 1 --2- 172.21.15.27:0/4096476818 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e240f5440 0x7f3e24106dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:03.242 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.243+0000 7f3e29b72700 1 -- 172.21.15.27:0/4096476818 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e14005040 con 0x7f3e240f66e0 2021-11-03T17:31:03.242 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.243+0000 7f3e29b72700 1 --2- 172.21.15.27:0/4096476818 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e240f66e0 0x7f3e241072c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f3e14009990 tx=0x7f3e14009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:31:03.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.244+0000 7f3e1affd700 1 -- 172.21.15.27:0/4096476818 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e14018070 con 0x7f3e240f66e0 2021-11-03T17:31:03.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.244+0000 7f3e2b375700 1 -- 172.21.15.27:0/4096476818 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e24108380 con 0x7f3e240f66e0 2021-11-03T17:31:03.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.244+0000 7f3e1affd700 1 -- 172.21.15.27:0/4096476818 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f3e14007ba0 con 0x7f3e240f66e0 2021-11-03T17:31:03.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.244+0000 7f3e1affd700 1 -- 172.21.15.27:0/4096476818 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e1401c3c0 con 0x7f3e240f66e0 2021-11-03T17:31:03.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.244+0000 7f3e2b375700 1 -- 172.21.15.27:0/4096476818 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e24108800 con 0x7f3e240f66e0 2021-11-03T17:31:03.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.245+0000 7f3e2b375700 1 -- 172.21.15.27:0/4096476818 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e24059100 con 0x7f3e240f66e0 2021-11-03T17:31:03.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.245+0000 7f3e1affd700 1 -- 172.21.15.27:0/4096476818 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f3e1401d420 con 0x7f3e240f66e0 2021-11-03T17:31:03.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.245+0000 7f3e1affd700 1 --2- 172.21.15.27:0/4096476818 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3e10064100 0x7f3e100665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:31:03.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.246+0000 7f3e1affd700 1 -- 172.21.15.27:0/4096476818 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f3e140824f0 con 0x7f3e240f66e0 2021-11-03T17:31:03.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.246+0000 7f3e2a373700 1 --2- 172.21.15.27:0/4096476818 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3e10064100 0x7f3e100665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:31:03.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.246+0000 7f3e2a373700 1 --2- 172.21.15.27:0/4096476818 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3e10064100 0x7f3e100665c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f3e24107b90 tx=0x7f3e2000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:31:03.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.250+0000 7f3e1affd700 1 -- 172.21.15.27:0/4096476818 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f3e14054830 con 0x7f3e240f66e0 2021-11-03T17:31:03.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:03 smithi027 ceph-mon[97244]: pgmap v492: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:03.424 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.429+0000 7f3e2b375700 1 -- 172.21.15.27:0/4096476818 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e2405fd30 con 0x7f3e10064100 2021-11-03T17:31:03.425 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.431+0000 7f3e1affd700 1 -- 172.21.15.27:0/4096476818 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f3e2405fd30 con 0x7f3e10064100 2021-11-03T17:31:03.427 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.432+0000 7f3e2b375700 1 -- 172.21.15.27:0/4096476818 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3e10064100 msgr2=0x7f3e100665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:31:03.428 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.432+0000 7f3e2b375700 1 --2- 172.21.15.27:0/4096476818 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3e10064100 0x7f3e100665c0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f3e24107b90 tx=0x7f3e2000b040).stop 2021-11-03T17:31:03.428 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.432+0000 7f3e2b375700 1 -- 172.21.15.27:0/4096476818 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e240f66e0 msgr2=0x7f3e241072c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:31:03.428 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.432+0000 7f3e2b375700 1 --2- 172.21.15.27:0/4096476818 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e240f66e0 0x7f3e241072c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f3e14009990 tx=0x7f3e14009b30).stop 2021-11-03T17:31:03.429 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.432+0000 7f3e2b375700 1 -- 172.21.15.27:0/4096476818 shutdown_connections 2021-11-03T17:31:03.429 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.432+0000 7f3e2b375700 1 --2- 172.21.15.27:0/4096476818 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3e10064100 0x7f3e100665c0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:03.429 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.432+0000 7f3e2b375700 1 --2- 172.21.15.27:0/4096476818 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3e240f5440 0x7f3e24106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:03.429 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.432+0000 7f3e2b375700 1 --2- 172.21.15.27:0/4096476818 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3e240f66e0 0x7f3e241072c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:03.430 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.433+0000 7f3e2b375700 1 -- 172.21.15.27:0/4096476818 >> 172.21.15.27:0/4096476818 conn(0x7f3e240f0ba0 msgr2=0x7f3e2405f620 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:31:03.430 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.433+0000 7f3e2b375700 1 -- 172.21.15.27:0/4096476818 shutdown_connections 2021-11-03T17:31:03.430 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.433+0000 7f3e2b375700 1 -- 172.21.15.27:0/4096476818 wait complete. 2021-11-03T17:31:03.436 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:31:03.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:03 smithi115 ceph-mon[65453]: pgmap v492: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:03.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.533+0000 7f9732f1a700 1 -- 172.21.15.27:0/2693710137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f972c0f7980 msgr2=0x7f972c0f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:31:03.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.533+0000 7f9732f1a700 1 --2- 172.21.15.27:0/2693710137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f972c0f7980 0x7f972c0f7de0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f9718004660 tx=0x7f9718009b30).stop 2021-11-03T17:31:03.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.533+0000 7f9732f1a700 1 -- 172.21.15.27:0/2693710137 shutdown_connections 2021-11-03T17:31:03.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.533+0000 7f9732f1a700 1 --2- 172.21.15.27:0/2693710137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f972c0f7980 0x7f972c0f7de0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:03.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.533+0000 7f9732f1a700 1 --2- 172.21.15.27:0/2693710137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f972c0f66e0 0x7f972c0f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:03.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.533+0000 7f9732f1a700 1 -- 172.21.15.27:0/2693710137 >> 172.21.15.27:0/2693710137 conn(0x7f972c0f1e40 msgr2=0x7f972c0f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:31:03.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.533+0000 7f9732f1a700 1 -- 172.21.15.27:0/2693710137 shutdown_connections 2021-11-03T17:31:03.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.533+0000 7f9732f1a700 1 -- 172.21.15.27:0/2693710137 wait complete. 2021-11-03T17:31:03.531 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.533+0000 7f9732f1a700 1 Processor -- start 2021-11-03T17:31:03.531 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.533+0000 7f9732f1a700 1 -- start start 2021-11-03T17:31:03.532 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.534+0000 7f9732f1a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f972c0f66e0 0x7f972c106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:31:03.532 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.534+0000 7f9732f1a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f972c0f7980 0x7f972c107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:31:03.532 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.534+0000 7f9732f1a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972c107a00 con 0x7f972c0f7980 2021-11-03T17:31:03.532 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.534+0000 7f9732f1a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972c107b40 con 0x7f972c0f66e0 2021-11-03T17:31:03.533 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.534+0000 7f972bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f972c0f7980 0x7f972c107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:31:03.533 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.534+0000 7f972bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f972c0f7980 0x7f972c107450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57884/0 (socket says 172.21.15.27:57884) 2021-11-03T17:31:03.533 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.534+0000 7f972bfff700 1 -- 172.21.15.27:0/836310853 learned_addr learned my addr 172.21.15.27:0/836310853 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:31:03.533 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.534+0000 7f9730cb6700 1 --2- 172.21.15.27:0/836310853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f972c0f66e0 0x7f972c106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:31:03.534 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.534+0000 7f972bfff700 1 -- 172.21.15.27:0/836310853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f972c0f66e0 msgr2=0x7f972c106f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:31:03.534 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.534+0000 7f972bfff700 1 --2- 172.21.15.27:0/836310853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f972c0f66e0 0x7f972c106f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:03.534 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.534+0000 7f972bfff700 1 -- 172.21.15.27:0/836310853 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9718005040 con 0x7f972c0f7980 2021-11-03T17:31:03.534 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.534+0000 7f972bfff700 1 --2- 172.21.15.27:0/836310853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f972c0f7980 0x7f972c107450 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f9718009990 tx=0x7f9718009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:31:03.535 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.534+0000 7f97297fa700 1 -- 172.21.15.27:0/836310853 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9718018070 con 0x7f972c0f7980 2021-11-03T17:31:03.535 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.534+0000 7f9732f1a700 1 -- 172.21.15.27:0/836310853 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f972c108480 con 0x7f972c0f7980 2021-11-03T17:31:03.535 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.534+0000 7f97297fa700 1 -- 172.21.15.27:0/836310853 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f9718007ba0 con 0x7f972c0f7980 2021-11-03T17:31:03.535 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.534+0000 7f97297fa700 1 -- 172.21.15.27:0/836310853 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f971801c3c0 con 0x7f972c0f7980 2021-11-03T17:31:03.535 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.535+0000 7f9732f1a700 1 -- 172.21.15.27:0/836310853 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f972c1089e0 con 0x7f972c0f7980 2021-11-03T17:31:03.536 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.536+0000 7f9732f1a700 1 -- 172.21.15.27:0/836310853 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f972c045ad0 con 0x7f972c0f7980 2021-11-03T17:31:03.536 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.536+0000 7f97297fa700 1 -- 172.21.15.27:0/836310853 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f971801d420 con 0x7f972c0f7980 2021-11-03T17:31:03.536 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.536+0000 7f97297fa700 1 --2- 172.21.15.27:0/836310853 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f971c0640b0 0x7f971c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:31:03.536 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.536+0000 7f97297fa700 1 -- 172.21.15.27:0/836310853 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f9718082630 con 0x7f972c0f7980 2021-11-03T17:31:03.537 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.536+0000 7f9730cb6700 1 --2- 172.21.15.27:0/836310853 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f971c0640b0 0x7f971c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:31:03.537 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.537+0000 7f9730cb6700 1 --2- 172.21.15.27:0/836310853 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f971c0640b0 0x7f971c066570 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f972c0f77e0 tx=0x7f9720009c00).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:31:03.537 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.540+0000 7f97297fa700 1 -- 172.21.15.27:0/836310853 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f9718054990 con 0x7f972c0f7980 2021-11-03T17:31:03.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.713+0000 7f9732f1a700 1 -- 172.21.15.27:0/836310853 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f972c108c00 con 0x7f971c0640b0 2021-11-03T17:31:03.722 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.728+0000 7f97297fa700 1 -- 172.21.15.27:0/836310853 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f972c108c00 con 0x7f971c0640b0 2021-11-03T17:31:03.722 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:31:03.723 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (22m) 2m ago 23m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:31:03.723 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (14m) 2m ago 23m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:31:03.723 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (14m) 84s ago 23m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:31:03.723 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (22m) 2m ago 23m 40.2M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:31:03.724 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (16m) 2m ago 18m 18.2M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:31:03.724 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (19m) 2m ago 19m 3841M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:31:03.724 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (14m) 84s ago 19m 17.6M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:31:03.724 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (16m) 84s ago 19m 19.3M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:31:03.724 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (16m) 2m ago 25m 497M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:31:03.725 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (15m) 84s ago 23m 415M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:31:03.725 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (15m) 2m ago 25m 119M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:31:03.725 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (14m) 84s ago 22m 71.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:31:03.725 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (23m) 2m ago 23m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:31:03.726 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (22m) 84s ago 22m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:31:03.726 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (13m) 2m ago 22m 290M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:31:03.726 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (13m) 2m ago 21m 317M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:31:03.726 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (12m) 2m ago 21m 229M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:31:03.726 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (12m) 84s ago 21m 329M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:31:03.727 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (12m) 84s ago 20m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:31:03.727 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (11m) 84s ago 20m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:31:03.727 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (22m) 2m ago 23m 48.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:31:03.728 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.730+0000 7f9732f1a700 1 -- 172.21.15.27:0/836310853 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f971c0640b0 msgr2=0x7f971c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:31:03.728 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.730+0000 7f9732f1a700 1 --2- 172.21.15.27:0/836310853 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f971c0640b0 0x7f971c066570 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f972c0f77e0 tx=0x7f9720009c00).stop 2021-11-03T17:31:03.728 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.730+0000 7f9732f1a700 1 -- 172.21.15.27:0/836310853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f972c0f7980 msgr2=0x7f972c107450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:31:03.728 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.730+0000 7f9732f1a700 1 --2- 172.21.15.27:0/836310853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f972c0f7980 0x7f972c107450 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7f9718009990 tx=0x7f9718009b30).stop 2021-11-03T17:31:03.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.730+0000 7f9732f1a700 1 -- 172.21.15.27:0/836310853 shutdown_connections 2021-11-03T17:31:03.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.730+0000 7f9732f1a700 1 --2- 172.21.15.27:0/836310853 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f971c0640b0 0x7f971c066570 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:03.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.730+0000 7f9732f1a700 1 --2- 172.21.15.27:0/836310853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f972c0f66e0 0x7f972c106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:03.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.730+0000 7f9732f1a700 1 --2- 172.21.15.27:0/836310853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f972c0f7980 0x7f972c107450 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:03.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.730+0000 7f9732f1a700 1 -- 172.21.15.27:0/836310853 >> 172.21.15.27:0/836310853 conn(0x7f972c0f1e40 msgr2=0x7f972c0fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:31:03.730 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.730+0000 7f9732f1a700 1 -- 172.21.15.27:0/836310853 shutdown_connections 2021-11-03T17:31:03.730 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.730+0000 7f9732f1a700 1 -- 172.21.15.27:0/836310853 wait complete. 2021-11-03T17:31:03.824 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.830+0000 7f6f030ee700 1 -- 172.21.15.27:0/3490712441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6efc0f7980 msgr2=0x7f6efc0f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:31:03.824 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.830+0000 7f6f030ee700 1 --2- 172.21.15.27:0/3490712441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6efc0f7980 0x7f6efc0f7de0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f6eec004660 tx=0x7f6eec009b30).stop 2021-11-03T17:31:03.825 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.830+0000 7f6f030ee700 1 -- 172.21.15.27:0/3490712441 shutdown_connections 2021-11-03T17:31:03.825 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.830+0000 7f6f030ee700 1 --2- 172.21.15.27:0/3490712441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6efc0f7980 0x7f6efc0f7de0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:03.825 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.830+0000 7f6f030ee700 1 --2- 172.21.15.27:0/3490712441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6efc0f66e0 0x7f6efc0f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:03.825 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.830+0000 7f6f030ee700 1 -- 172.21.15.27:0/3490712441 >> 172.21.15.27:0/3490712441 conn(0x7f6efc0f1e40 msgr2=0x7f6efc0f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:31:03.826 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.830+0000 7f6f030ee700 1 -- 172.21.15.27:0/3490712441 shutdown_connections 2021-11-03T17:31:03.826 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.830+0000 7f6f030ee700 1 -- 172.21.15.27:0/3490712441 wait complete. 2021-11-03T17:31:03.826 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.830+0000 7f6f030ee700 1 Processor -- start 2021-11-03T17:31:03.826 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.830+0000 7f6f030ee700 1 -- start start 2021-11-03T17:31:03.828 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.831+0000 7f6f030ee700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6efc0f66e0 0x7f6efc106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:31:03.829 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.831+0000 7f6f030ee700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6efc0f7980 0x7f6efc107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:31:03.829 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.831+0000 7f6f030ee700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6efc107a00 con 0x7f6efc0f66e0 2021-11-03T17:31:03.829 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.831+0000 7f6f030ee700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6efc107b40 con 0x7f6efc0f7980 2021-11-03T17:31:03.829 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.831+0000 7f6f00e8a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6efc0f66e0 0x7f6efc106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:31:03.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.831+0000 7f6f00e8a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6efc0f66e0 0x7f6efc106f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57892/0 (socket says 172.21.15.27:57892) 2021-11-03T17:31:03.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.831+0000 7f6f00e8a700 1 -- 172.21.15.27:0/3446178788 learned_addr learned my addr 172.21.15.27:0/3446178788 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:31:03.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.831+0000 7f6efbfff700 1 --2- 172.21.15.27:0/3446178788 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6efc0f7980 0x7f6efc107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:31:03.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.831+0000 7f6f00e8a700 1 -- 172.21.15.27:0/3446178788 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6efc0f7980 msgr2=0x7f6efc107450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:31:03.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.831+0000 7f6f00e8a700 1 --2- 172.21.15.27:0/3446178788 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6efc0f7980 0x7f6efc107450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:03.831 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.831+0000 7f6f00e8a700 1 -- 172.21.15.27:0/3446178788 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eec005040 con 0x7f6efc0f66e0 2021-11-03T17:31:03.831 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.831+0000 7f6f00e8a700 1 --2- 172.21.15.27:0/3446178788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6efc0f66e0 0x7f6efc106f60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f6ef400d7f0 tx=0x7f6ef4004e80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:31:03.831 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.831+0000 7f6ef97fa700 1 -- 172.21.15.27:0/3446178788 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ef400f490 con 0x7f6efc0f66e0 2021-11-03T17:31:03.831 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.831+0000 7f6f030ee700 1 -- 172.21.15.27:0/3446178788 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6efc1084e0 con 0x7f6efc0f66e0 2021-11-03T17:31:03.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.831+0000 7f6ef97fa700 1 -- 172.21.15.27:0/3446178788 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f6ef400f040 con 0x7f6efc0f66e0 2021-11-03T17:31:03.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.831+0000 7f6ef97fa700 1 -- 172.21.15.27:0/3446178788 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ef40139b0 con 0x7f6efc0f66e0 2021-11-03T17:31:03.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.832+0000 7f6f030ee700 1 -- 172.21.15.27:0/3446178788 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6efc108aa0 con 0x7f6efc0f66e0 2021-11-03T17:31:03.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.832+0000 7f6f030ee700 1 -- 172.21.15.27:0/3446178788 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6efc045ad0 con 0x7f6efc0f66e0 2021-11-03T17:31:03.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.833+0000 7f6ef97fa700 1 -- 172.21.15.27:0/3446178788 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f6ef401a030 con 0x7f6efc0f66e0 2021-11-03T17:31:03.833 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.833+0000 7f6ef97fa700 1 --2- 172.21.15.27:0/3446178788 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6ee80640b0 0x7f6ee8066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:31:03.833 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.833+0000 7f6ef97fa700 1 -- 172.21.15.27:0/3446178788 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f6ef4080830 con 0x7f6efc0f66e0 2021-11-03T17:31:03.833 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.833+0000 7f6efbfff700 1 --2- 172.21.15.27:0/3446178788 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6ee80640b0 0x7f6ee8066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:31:03.833 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.834+0000 7f6efbfff700 1 --2- 172.21.15.27:0/3446178788 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6ee80640b0 0x7f6ee8066570 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f6eec004630 tx=0x7f6eec009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:31:03.834 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:03.837+0000 7f6ef97fa700 1 -- 172.21.15.27:0/3446178788 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f6ef4052a40 con 0x7f6efc0f66e0 2021-11-03T17:31:04.064 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.069+0000 7f6f030ee700 1 -- 172.21.15.27:0/3446178788 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6efc0008d0 con 0x7f6efc0f66e0 2021-11-03T17:31:04.064 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.070+0000 7f6ef97fa700 1 -- 172.21.15.27:0/3446178788 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f6ef4052630 con 0x7f6efc0f66e0 2021-11-03T17:31:04.065 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:31:04.065 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:31:04.065 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:31:04.065 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:31:04.066 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:31:04.066 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:31:04.066 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:31:04.066 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:31:04.067 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:31:04.067 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:31:04.067 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:31:04.067 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:31:04.067 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:31:04.068 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:31:04.068 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:31:04.068 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:31:04.068 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:31:04.069 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:31:04.069 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.072+0000 7f6f030ee700 1 -- 172.21.15.27:0/3446178788 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6ee80640b0 msgr2=0x7f6ee8066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:31:04.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.072+0000 7f6f030ee700 1 --2- 172.21.15.27:0/3446178788 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6ee80640b0 0x7f6ee8066570 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f6eec004630 tx=0x7f6eec009b30).stop 2021-11-03T17:31:04.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.072+0000 7f6f030ee700 1 -- 172.21.15.27:0/3446178788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6efc0f66e0 msgr2=0x7f6efc106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:31:04.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.072+0000 7f6f030ee700 1 --2- 172.21.15.27:0/3446178788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6efc0f66e0 0x7f6efc106f60 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f6ef400d7f0 tx=0x7f6ef4004e80).stop 2021-11-03T17:31:04.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.072+0000 7f6f030ee700 1 -- 172.21.15.27:0/3446178788 shutdown_connections 2021-11-03T17:31:04.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.072+0000 7f6f030ee700 1 --2- 172.21.15.27:0/3446178788 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6ee80640b0 0x7f6ee8066570 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:04.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.072+0000 7f6f030ee700 1 --2- 172.21.15.27:0/3446178788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6efc0f66e0 0x7f6efc106f60 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:04.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.072+0000 7f6f030ee700 1 --2- 172.21.15.27:0/3446178788 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6efc0f7980 0x7f6efc107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:04.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.072+0000 7f6f030ee700 1 -- 172.21.15.27:0/3446178788 >> 172.21.15.27:0/3446178788 conn(0x7f6efc0f1e40 msgr2=0x7f6efc0fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:31:04.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.072+0000 7f6f030ee700 1 -- 172.21.15.27:0/3446178788 shutdown_connections 2021-11-03T17:31:04.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.072+0000 7f6f030ee700 1 -- 172.21.15.27:0/3446178788 wait complete. 2021-11-03T17:31:04.168 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.174+0000 7f1269729700 1 -- 172.21.15.27:0/2239018971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12640f6860 msgr2=0x7f12640f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:31:04.169 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.174+0000 7f1269729700 1 --2- 172.21.15.27:0/2239018971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12640f6860 0x7f12640f6cc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 rx=0x7f1258004660 tx=0x7f1258009b30).stop 2021-11-03T17:31:04.169 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.174+0000 7f1269729700 1 -- 172.21.15.27:0/2239018971 shutdown_connections 2021-11-03T17:31:04.169 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.174+0000 7f1269729700 1 --2- 172.21.15.27:0/2239018971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12640f6860 0x7f12640f6cc0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:04.169 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.174+0000 7f1269729700 1 --2- 172.21.15.27:0/2239018971 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12640f55c0 0x7f12640f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.174+0000 7f1269729700 1 -- 172.21.15.27:0/2239018971 >> 172.21.15.27:0/2239018971 conn(0x7f12640f0d40 msgr2=0x7f12640f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:31:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.174+0000 7f1269729700 1 -- 172.21.15.27:0/2239018971 shutdown_connections 2021-11-03T17:31:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.174+0000 7f1269729700 1 -- 172.21.15.27:0/2239018971 wait complete. 2021-11-03T17:31:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.174+0000 7f1269729700 1 Processor -- start 2021-11-03T17:31:04.170 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.174+0000 7f1269729700 1 -- start start 2021-11-03T17:31:04.173 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.174+0000 7f1269729700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12640f55c0 0x7f1264109c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:31:04.173 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.175+0000 7f1269729700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12640f6860 0x7f126410c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:31:04.173 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.175+0000 7f1269729700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12640603e0 con 0x7f12640f55c0 2021-11-03T17:31:04.174 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.175+0000 7f1269729700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1264060520 con 0x7f12640f6860 2021-11-03T17:31:04.174 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.175+0000 7f1262ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12640f55c0 0x7f1264109c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:31:04.174 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.175+0000 7f1262ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12640f55c0 0x7f1264109c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57902/0 (socket says 172.21.15.27:57902) 2021-11-03T17:31:04.174 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.175+0000 7f1262ffd700 1 -- 172.21.15.27:0/965540493 learned_addr learned my addr 172.21.15.27:0/965540493 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:31:04.174 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.175+0000 7f12627fc700 1 --2- 172.21.15.27:0/965540493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12640f6860 0x7f126410c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:31:04.175 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.175+0000 7f1262ffd700 1 -- 172.21.15.27:0/965540493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12640f6860 msgr2=0x7f126410c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:31:04.175 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.175+0000 7f1262ffd700 1 --2- 172.21.15.27:0/965540493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12640f6860 0x7f126410c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:04.175 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.175+0000 7f1262ffd700 1 -- 172.21.15.27:0/965540493 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1258005040 con 0x7f12640f55c0 2021-11-03T17:31:04.175 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.175+0000 7f1262ffd700 1 --2- 172.21.15.27:0/965540493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12640f55c0 0x7f1264109c50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f125c00d7f0 tx=0x7f125c004e80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:31:04.176 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.175+0000 7f124b7fe700 1 -- 172.21.15.27:0/965540493 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f125c00f490 con 0x7f12640f55c0 2021-11-03T17:31:04.176 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.175+0000 7f124b7fe700 1 -- 172.21.15.27:0/965540493 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f125c00f040 con 0x7f12640f55c0 2021-11-03T17:31:04.176 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.175+0000 7f1269729700 1 -- 172.21.15.27:0/965540493 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f126410c700 con 0x7f12640f55c0 2021-11-03T17:31:04.176 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.175+0000 7f124b7fe700 1 -- 172.21.15.27:0/965540493 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f125c0139b0 con 0x7f12640f55c0 2021-11-03T17:31:04.177 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.175+0000 7f1269729700 1 -- 172.21.15.27:0/965540493 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f126410ccc0 con 0x7f12640f55c0 2021-11-03T17:31:04.177 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.176+0000 7f1269729700 1 -- 172.21.15.27:0/965540493 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1264045ad0 con 0x7f12640f55c0 2021-11-03T17:31:04.177 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.177+0000 7f124b7fe700 1 -- 172.21.15.27:0/965540493 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f125c01a030 con 0x7f12640f55c0 2021-11-03T17:31:04.177 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.177+0000 7f124b7fe700 1 --2- 172.21.15.27:0/965540493 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f124c0640b0 0x7f124c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:31:04.177 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.177+0000 7f124b7fe700 1 -- 172.21.15.27:0/965540493 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f125c080810 con 0x7f12640f55c0 2021-11-03T17:31:04.178 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.177+0000 7f12627fc700 1 --2- 172.21.15.27:0/965540493 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f124c0640b0 0x7f124c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:31:04.178 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.178+0000 7f12627fc700 1 --2- 172.21.15.27:0/965540493 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f124c0640b0 0x7f124c066570 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f1258004630 tx=0x7f1258009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:31:04.178 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.181+0000 7f124b7fe700 1 -- 172.21.15.27:0/965540493 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f125c052a20 con 0x7f12640f55c0 2021-11-03T17:31:04.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:04 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3446178788' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:04.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.384+0000 7f1269729700 1 -- 172.21.15.27:0/965540493 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f12640586e0 con 0x7f12640f55c0 2021-11-03T17:31:04.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.384+0000 7f124b7fe700 1 -- 172.21.15.27:0/965540493 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f125c018070 con 0x7f12640f55c0 2021-11-03T17:31:04.379 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:31:04.379 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:31:04.380 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:31:04.380 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:31:04.380 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:31:04.380 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:31:04.380 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:31:04.381 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:31:04.381 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:31:04.381 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:31:04.381 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:31:04.382 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:31:04.382 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:31:04.382 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:31:04.382 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:31:04.383 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:31:04.383 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:31:04.383 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:31:04.383 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:31:04.383 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:31:04.384 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:31:04.384 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:31:04.384 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:31:04.384 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:31:04.384 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:31:04.385 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:31:04.385 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:31:04.385 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:31:04.385 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:31:04.385 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:31:04.386 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:31:04.386 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:31:04.386 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:31:04.386 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:31:04.387 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:31:04.387 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:31:04.387 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:31:04.387 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:31:04.387 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:31:04.388 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.386+0000 7f1269729700 1 -- 172.21.15.27:0/965540493 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f124c0640b0 msgr2=0x7f124c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:31:04.389 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.386+0000 7f1269729700 1 --2- 172.21.15.27:0/965540493 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f124c0640b0 0x7f124c066570 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f1258004630 tx=0x7f1258009b30).stop 2021-11-03T17:31:04.389 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.386+0000 7f1269729700 1 -- 172.21.15.27:0/965540493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12640f55c0 msgr2=0x7f1264109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:31:04.389 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.386+0000 7f1269729700 1 --2- 172.21.15.27:0/965540493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12640f55c0 0x7f1264109c50 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f125c00d7f0 tx=0x7f125c004e80).stop 2021-11-03T17:31:04.389 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.386+0000 7f1269729700 1 -- 172.21.15.27:0/965540493 shutdown_connections 2021-11-03T17:31:04.390 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.386+0000 7f1269729700 1 --2- 172.21.15.27:0/965540493 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f124c0640b0 0x7f124c066570 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:04.390 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.386+0000 7f1269729700 1 --2- 172.21.15.27:0/965540493 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12640f55c0 0x7f1264109c50 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:04.390 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.386+0000 7f1269729700 1 --2- 172.21.15.27:0/965540493 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12640f6860 0x7f126410c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:04.390 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.386+0000 7f1269729700 1 -- 172.21.15.27:0/965540493 >> 172.21.15.27:0/965540493 conn(0x7f12640f0d40 msgr2=0x7f12640fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:31:04.391 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.386+0000 7f1269729700 1 -- 172.21.15.27:0/965540493 shutdown_connections 2021-11-03T17:31:04.391 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:04.386+0000 7f1269729700 1 -- 172.21.15.27:0/965540493 wait complete. 2021-11-03T17:31:04.391 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:31:04.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:04 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3446178788' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:05.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:05 smithi115 ceph-mon[65453]: from='client.34616 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:31:05.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:05 smithi115 ceph-mon[65453]: from='client.34620 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:31:05.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:05 smithi115 ceph-mon[65453]: pgmap v493: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:05.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 907.380443 secs 2021-11-03T17:31:05.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:05 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/965540493' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:31:05.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:05 smithi027 ceph-mon[97244]: from='client.34616 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:31:05.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:05 smithi027 ceph-mon[97244]: from='client.34620 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:31:05.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:05 smithi027 ceph-mon[97244]: pgmap v493: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:05.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 907.380443 secs 2021-11-03T17:31:05.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:05 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/965540493' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:31:06.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:06 smithi115 ceph-mon[65453]: pgmap v494: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:06.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:06 smithi027 ceph-mon[97244]: pgmap v494: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:09.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:09 smithi115 ceph-mon[65453]: pgmap v495: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:09.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:09 smithi027 ceph-mon[97244]: pgmap v495: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:10.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 912.380569 secs 2021-11-03T17:31:10.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 912.380569 secs 2021-11-03T17:31:11.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:11 smithi115 ceph-mon[65453]: pgmap v496: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:11.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:31:11.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:31:11.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:31:11.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:11.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:11.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:11.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:11.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:11 smithi027 ceph-mon[97244]: pgmap v496: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:11.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:31:11.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:31:11.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:31:11.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:11.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:11.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:11.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:12.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:12 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:31:12.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:12 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:31:13.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:13 smithi027 ceph-mon[97244]: pgmap v497: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:13.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:13 smithi115 ceph-mon[65453]: pgmap v497: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:15.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:15 smithi115 ceph-mon[65453]: pgmap v498: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:15.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 917.380659 secs 2021-11-03T17:31:15.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:15 smithi027 ceph-mon[97244]: pgmap v498: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:15.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 917.380659 secs 2021-11-03T17:31:17.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:17 smithi115 ceph-mon[65453]: pgmap v499: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:17.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:17 smithi027 ceph-mon[97244]: pgmap v499: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:19.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:19 smithi115 ceph-mon[65453]: pgmap v500: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:19.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:19 smithi027 ceph-mon[97244]: pgmap v500: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:20.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 922.380739 secs 2021-11-03T17:31:20.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 922.380739 secs 2021-11-03T17:31:21.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:21 smithi115 ceph-mon[65453]: pgmap v501: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:21.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:31:21.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:31:21.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:31:21.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:21.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:21.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:21.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:21.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:21 smithi027 ceph-mon[97244]: pgmap v501: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:21.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:31:21.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:31:21.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:31:21.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:21.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:21.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:21.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:22.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:22 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:31:22.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:22 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:31:23.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:23 smithi027 ceph-mon[97244]: pgmap v502: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:23.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:23 smithi115 ceph-mon[65453]: pgmap v502: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:24.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:24 smithi115 ceph-mon[65453]: pgmap v503: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:24.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:24 smithi027 ceph-mon[97244]: pgmap v503: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:25.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 927.380857 secs 2021-11-03T17:31:25.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 927.380857 secs 2021-11-03T17:31:26.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:26 smithi115 ceph-mon[65453]: pgmap v504: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:26.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:26 smithi027 ceph-mon[97244]: pgmap v504: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:29.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:29 smithi115 ceph-mon[65453]: pgmap v505: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:29.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:29 smithi027 ceph-mon[97244]: pgmap v505: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:30.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 932.380924 secs 2021-11-03T17:31:30.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 932.380924 secs 2021-11-03T17:31:31.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:31 smithi115 ceph-mon[65453]: pgmap v506: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:31.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:31:31.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:31:31.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:31:31.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:31.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:31.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:31.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:31.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:31 smithi027 ceph-mon[97244]: pgmap v506: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:31.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:31:31.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:31:31.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:31:31.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:31.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:31.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:31.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:32.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:32 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:31:32.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:32 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:31:33.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:33 smithi027 ceph-mon[97244]: pgmap v507: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:33.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:33 smithi115 ceph-mon[65453]: pgmap v507: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:34.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.500+0000 7f1e522ca700 1 -- 172.21.15.27:0/1084808371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e4c0f55e0 msgr2=0x7f1e4c0f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:31:34.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.500+0000 7f1e522ca700 1 --2- 172.21.15.27:0/1084808371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e4c0f55e0 0x7f1e4c0f5a00 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f1e3c004660 tx=0x7f1e3c009b30).stop 2021-11-03T17:31:34.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.500+0000 7f1e522ca700 1 -- 172.21.15.27:0/1084808371 shutdown_connections 2021-11-03T17:31:34.496 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.500+0000 7f1e522ca700 1 --2- 172.21.15.27:0/1084808371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e4c0f5ef0 0x7f1e4c0f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:34.496 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.500+0000 7f1e522ca700 1 --2- 172.21.15.27:0/1084808371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e4c0f55e0 0x7f1e4c0f5a00 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:34.496 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.500+0000 7f1e522ca700 1 -- 172.21.15.27:0/1084808371 >> 172.21.15.27:0/1084808371 conn(0x7f1e4c0f0de0 msgr2=0x7f1e4c0f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:31:34.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.500+0000 7f1e522ca700 1 -- 172.21.15.27:0/1084808371 shutdown_connections 2021-11-03T17:31:34.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.500+0000 7f1e522ca700 1 -- 172.21.15.27:0/1084808371 wait complete. 2021-11-03T17:31:34.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.501+0000 7f1e522ca700 1 Processor -- start 2021-11-03T17:31:34.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.501+0000 7f1e522ca700 1 -- start start 2021-11-03T17:31:34.500 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.501+0000 7f1e522ca700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e4c0f55e0 0x7f1e4c109150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:31:34.500 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.501+0000 7f1e522ca700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e4c0f5ef0 0x7f1e4c109640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:31:34.500 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.501+0000 7f1e522ca700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4c109bf0 con 0x7f1e4c0f55e0 2021-11-03T17:31:34.501 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.501+0000 7f1e522ca700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4c109d30 con 0x7f1e4c0f5ef0 2021-11-03T17:31:34.501 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.502+0000 7f1e4bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e4c0f55e0 0x7f1e4c109150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:31:34.501 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.502+0000 7f1e4bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e4c0f55e0 0x7f1e4c109150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57920/0 (socket says 172.21.15.27:57920) 2021-11-03T17:31:34.501 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.502+0000 7f1e4bfff700 1 -- 172.21.15.27:0/872948480 learned_addr learned my addr 172.21.15.27:0/872948480 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:31:34.502 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.502+0000 7f1e4b7fe700 1 --2- 172.21.15.27:0/872948480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e4c0f5ef0 0x7f1e4c109640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:31:34.502 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.502+0000 7f1e4bfff700 1 -- 172.21.15.27:0/872948480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e4c0f5ef0 msgr2=0x7f1e4c109640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:31:34.502 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.502+0000 7f1e4bfff700 1 --2- 172.21.15.27:0/872948480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e4c0f5ef0 0x7f1e4c109640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:34.503 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.502+0000 7f1e4bfff700 1 -- 172.21.15.27:0/872948480 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e3c005040 con 0x7f1e4c0f55e0 2021-11-03T17:31:34.503 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.502+0000 7f1e4bfff700 1 --2- 172.21.15.27:0/872948480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e4c0f55e0 0x7f1e4c109150 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f1e3c009830 tx=0x7f1e3c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:31:34.503 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.502+0000 7f1e48ff9700 1 -- 172.21.15.27:0/872948480 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e3c018070 con 0x7f1e4c0f55e0 2021-11-03T17:31:34.503 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.502+0000 7f1e48ff9700 1 -- 172.21.15.27:0/872948480 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f1e3c007ba0 con 0x7f1e4c0f55e0 2021-11-03T17:31:34.504 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.502+0000 7f1e48ff9700 1 -- 172.21.15.27:0/872948480 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e3c01c3c0 con 0x7f1e4c0f55e0 2021-11-03T17:31:34.504 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.502+0000 7f1e522ca700 1 -- 172.21.15.27:0/872948480 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e4c10a670 con 0x7f1e4c0f55e0 2021-11-03T17:31:34.504 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.502+0000 7f1e522ca700 1 -- 172.21.15.27:0/872948480 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e4c10abd0 con 0x7f1e4c0f55e0 2021-11-03T17:31:34.504 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.503+0000 7f1e522ca700 1 -- 172.21.15.27:0/872948480 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e4c045ad0 con 0x7f1e4c0f55e0 2021-11-03T17:31:34.504 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.504+0000 7f1e48ff9700 1 -- 172.21.15.27:0/872948480 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f1e3c01d420 con 0x7f1e4c0f55e0 2021-11-03T17:31:34.505 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.504+0000 7f1e48ff9700 1 --2- 172.21.15.27:0/872948480 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1e34064090 0x7f1e34066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:31:34.505 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.504+0000 7f1e48ff9700 1 -- 172.21.15.27:0/872948480 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f1e3c0822f0 con 0x7f1e4c0f55e0 2021-11-03T17:31:34.505 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.504+0000 7f1e4b7fe700 1 --2- 172.21.15.27:0/872948480 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1e34064090 0x7f1e34066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:31:34.505 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.505+0000 7f1e4b7fe700 1 --2- 172.21.15.27:0/872948480 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1e34064090 0x7f1e34066550 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f1e40004c40 tx=0x7f1e4000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:31:34.506 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.509+0000 7f1e48ff9700 1 -- 172.21.15.27:0/872948480 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f1e3c0545f0 con 0x7f1e4c0f55e0 2021-11-03T17:31:34.682 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.688+0000 7f1e522ca700 1 -- 172.21.15.27:0/872948480 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e4c0fab60 con 0x7f1e34064090 2021-11-03T17:31:34.683 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.689+0000 7f1e48ff9700 1 -- 172.21.15.27:0/872948480 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f1e4c0fab60 con 0x7f1e34064090 2021-11-03T17:31:34.685 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.691+0000 7f1e522ca700 1 -- 172.21.15.27:0/872948480 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1e34064090 msgr2=0x7f1e34066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:31:34.685 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.691+0000 7f1e522ca700 1 --2- 172.21.15.27:0/872948480 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1e34064090 0x7f1e34066550 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f1e40004c40 tx=0x7f1e4000b040).stop 2021-11-03T17:31:34.685 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.691+0000 7f1e522ca700 1 -- 172.21.15.27:0/872948480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e4c0f55e0 msgr2=0x7f1e4c109150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:31:34.686 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.691+0000 7f1e522ca700 1 --2- 172.21.15.27:0/872948480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e4c0f55e0 0x7f1e4c109150 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7f1e3c009830 tx=0x7f1e3c009b30).stop 2021-11-03T17:31:34.686 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.691+0000 7f1e522ca700 1 -- 172.21.15.27:0/872948480 shutdown_connections 2021-11-03T17:31:34.686 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.691+0000 7f1e522ca700 1 --2- 172.21.15.27:0/872948480 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1e34064090 0x7f1e34066550 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:34.686 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.691+0000 7f1e522ca700 1 --2- 172.21.15.27:0/872948480 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e4c0f55e0 0x7f1e4c109150 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:34.687 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.691+0000 7f1e522ca700 1 --2- 172.21.15.27:0/872948480 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e4c0f5ef0 0x7f1e4c109640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:34.687 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.691+0000 7f1e522ca700 1 -- 172.21.15.27:0/872948480 >> 172.21.15.27:0/872948480 conn(0x7f1e4c0f0de0 msgr2=0x7f1e4c05ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:31:34.687 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.691+0000 7f1e522ca700 1 -- 172.21.15.27:0/872948480 shutdown_connections 2021-11-03T17:31:34.687 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.691+0000 7f1e522ca700 1 -- 172.21.15.27:0/872948480 wait complete. 2021-11-03T17:31:34.695 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:31:34.788 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.791+0000 7fae6ba37700 1 -- 172.21.15.27:0/3136907761 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae640602f0 msgr2=0x7fae640f9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:31:34.788 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.791+0000 7fae6ba37700 1 --2- 172.21.15.27:0/3136907761 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae640602f0 0x7fae640f9400 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7fae60004660 tx=0x7fae60009b30).stop 2021-11-03T17:31:34.788 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.791+0000 7fae6ba37700 1 -- 172.21.15.27:0/3136907761 shutdown_connections 2021-11-03T17:31:34.789 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.791+0000 7fae6ba37700 1 --2- 172.21.15.27:0/3136907761 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae640f98f0 0x7fae640fbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:34.789 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.791+0000 7fae6ba37700 1 --2- 172.21.15.27:0/3136907761 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae640602f0 0x7fae640f9400 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:34.789 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.791+0000 7fae6ba37700 1 -- 172.21.15.27:0/3136907761 >> 172.21.15.27:0/3136907761 conn(0x7fae640f0de0 msgr2=0x7fae640f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:31:34.789 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.792+0000 7fae6ba37700 1 -- 172.21.15.27:0/3136907761 shutdown_connections 2021-11-03T17:31:34.790 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.792+0000 7fae6ba37700 1 -- 172.21.15.27:0/3136907761 wait complete. 2021-11-03T17:31:34.790 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.792+0000 7fae6ba37700 1 Processor -- start 2021-11-03T17:31:34.790 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.792+0000 7fae6ba37700 1 -- start start 2021-11-03T17:31:34.790 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.792+0000 7fae6ba37700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae640602f0 0x7fae64106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:31:34.791 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.792+0000 7fae6ba37700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae640f98f0 0x7fae64107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:31:34.791 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.792+0000 7fae6ba37700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae64107a10 con 0x7fae640602f0 2021-11-03T17:31:34.791 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.792+0000 7fae6ba37700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae64107b50 con 0x7fae640f98f0 2021-11-03T17:31:34.791 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.793+0000 7fae697d3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae640602f0 0x7fae64106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:31:34.792 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.793+0000 7fae697d3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae640602f0 0x7fae64106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57930/0 (socket says 172.21.15.27:57930) 2021-11-03T17:31:34.792 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.793+0000 7fae697d3700 1 -- 172.21.15.27:0/247803568 learned_addr learned my addr 172.21.15.27:0/247803568 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:31:34.792 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.793+0000 7fae68fd2700 1 --2- 172.21.15.27:0/247803568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae640f98f0 0x7fae64107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:31:34.792 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.793+0000 7fae697d3700 1 -- 172.21.15.27:0/247803568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae640f98f0 msgr2=0x7fae64107460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:31:34.793 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.793+0000 7fae697d3700 1 --2- 172.21.15.27:0/247803568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae640f98f0 0x7fae64107460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:34.793 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.793+0000 7fae697d3700 1 -- 172.21.15.27:0/247803568 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae60005040 con 0x7fae640602f0 2021-11-03T17:31:34.793 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.793+0000 7fae697d3700 1 --2- 172.21.15.27:0/247803568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae640602f0 0x7fae64106f70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7fae60013a70 tx=0x7fae60009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:31:34.793 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.793+0000 7fae59ffb700 1 -- 172.21.15.27:0/247803568 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae60018070 con 0x7fae640602f0 2021-11-03T17:31:34.793 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.793+0000 7fae59ffb700 1 -- 172.21.15.27:0/247803568 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fae600097d0 con 0x7fae640602f0 2021-11-03T17:31:34.794 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.793+0000 7fae6ba37700 1 -- 172.21.15.27:0/247803568 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae64108490 con 0x7fae640602f0 2021-11-03T17:31:34.794 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.793+0000 7fae59ffb700 1 -- 172.21.15.27:0/247803568 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae6001de80 con 0x7fae640602f0 2021-11-03T17:31:34.794 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.793+0000 7fae6ba37700 1 -- 172.21.15.27:0/247803568 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae641089f0 con 0x7fae640602f0 2021-11-03T17:31:34.795 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.794+0000 7fae6ba37700 1 -- 172.21.15.27:0/247803568 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae64045ad0 con 0x7fae640602f0 2021-11-03T17:31:34.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.795+0000 7fae59ffb700 1 -- 172.21.15.27:0/247803568 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fae6001c490 con 0x7fae640602f0 2021-11-03T17:31:34.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.795+0000 7fae59ffb700 1 --2- 172.21.15.27:0/247803568 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fae500640b0 0x7fae50066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:31:34.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.795+0000 7fae59ffb700 1 -- 172.21.15.27:0/247803568 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fae60082370 con 0x7fae640602f0 2021-11-03T17:31:34.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.795+0000 7fae68fd2700 1 --2- 172.21.15.27:0/247803568 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fae500640b0 0x7fae50066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:31:34.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.796+0000 7fae68fd2700 1 --2- 172.21.15.27:0/247803568 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fae500640b0 0x7fae50066570 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7fae54004c40 tx=0x7fae5400b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:31:34.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.799+0000 7fae59ffb700 1 -- 172.21.15.27:0/247803568 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fae600547c0 con 0x7fae640602f0 2021-11-03T17:31:34.964 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.970+0000 7fae6ba37700 1 -- 172.21.15.27:0/247803568 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fae640fefc0 con 0x7fae500640b0 2021-11-03T17:31:34.979 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.985+0000 7fae59ffb700 1 -- 172.21.15.27:0/247803568 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fae640fefc0 con 0x7fae500640b0 2021-11-03T17:31:34.980 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:31:34.980 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (23m) 3m ago 24m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:31:34.980 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (14m) 3m ago 24m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:31:34.980 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (14m) 115s ago 23m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:31:34.981 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (22m) 3m ago 24m 40.2M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:31:34.981 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (16m) 3m ago 19m 18.2M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:31:34.981 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (19m) 3m ago 19m 3841M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:31:34.981 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (15m) 115s ago 19m 17.6M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:31:34.982 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (16m) 115s ago 19m 19.3M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:31:34.982 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (16m) 3m ago 25m 497M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:31:34.982 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (16m) 115s ago 23m 415M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:31:34.982 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (15m) 3m ago 25m 119M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:31:34.983 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (15m) 115s ago 23m 71.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:31:34.983 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (24m) 3m ago 23m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:31:34.983 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (23m) 115s ago 23m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:31:34.983 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (14m) 3m ago 22m 290M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:31:34.984 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (13m) 3m ago 22m 317M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:31:34.984 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (13m) 3m ago 22m 229M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:31:34.984 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (12m) 115s ago 21m 329M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:31:34.984 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (12m) 115s ago 21m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:31:34.985 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (12m) 115s ago 21m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:31:34.985 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (22m) 3m ago 23m 48.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:31:34.986 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.987+0000 7fae6ba37700 1 -- 172.21.15.27:0/247803568 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fae500640b0 msgr2=0x7fae50066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:31:34.986 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.987+0000 7fae6ba37700 1 --2- 172.21.15.27:0/247803568 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fae500640b0 0x7fae50066570 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7fae54004c40 tx=0x7fae5400b040).stop 2021-11-03T17:31:34.987 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.987+0000 7fae6ba37700 1 -- 172.21.15.27:0/247803568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae640602f0 msgr2=0x7fae64106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:31:34.987 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.987+0000 7fae6ba37700 1 --2- 172.21.15.27:0/247803568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae640602f0 0x7fae64106f70 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7fae60013a70 tx=0x7fae60009b30).stop 2021-11-03T17:31:34.987 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.987+0000 7fae6ba37700 1 -- 172.21.15.27:0/247803568 shutdown_connections 2021-11-03T17:31:34.987 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.987+0000 7fae6ba37700 1 --2- 172.21.15.27:0/247803568 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fae500640b0 0x7fae50066570 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:34.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.987+0000 7fae6ba37700 1 --2- 172.21.15.27:0/247803568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae640602f0 0x7fae64106f70 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:34.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.987+0000 7fae6ba37700 1 --2- 172.21.15.27:0/247803568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae640f98f0 0x7fae64107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:34.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.987+0000 7fae6ba37700 1 -- 172.21.15.27:0/247803568 >> 172.21.15.27:0/247803568 conn(0x7fae640f0de0 msgr2=0x7fae640f31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:31:34.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.987+0000 7fae6ba37700 1 -- 172.21.15.27:0/247803568 shutdown_connections 2021-11-03T17:31:34.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:34.987+0000 7fae6ba37700 1 -- 172.21.15.27:0/247803568 wait complete. 2021-11-03T17:31:35.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.086+0000 7f268144d700 1 -- 172.21.15.27:0/1469721911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f267c0fcce0 msgr2=0x7f267c0fd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:31:35.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.087+0000 7f268144d700 1 --2- 172.21.15.27:0/1469721911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f267c0fcce0 0x7f267c0fd100 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f266c004660 tx=0x7f266c009b30).stop 2021-11-03T17:31:35.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.087+0000 7f268144d700 1 -- 172.21.15.27:0/1469721911 shutdown_connections 2021-11-03T17:31:35.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.087+0000 7f268144d700 1 --2- 172.21.15.27:0/1469721911 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f267c0fdf80 0x7f267c0fe3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:35.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.087+0000 7f268144d700 1 --2- 172.21.15.27:0/1469721911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f267c0fcce0 0x7f267c0fd100 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:35.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.087+0000 7f268144d700 1 -- 172.21.15.27:0/1469721911 >> 172.21.15.27:0/1469721911 conn(0x7f267c06c000 msgr2=0x7f267c06e420 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:31:35.086 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.087+0000 7f268144d700 1 -- 172.21.15.27:0/1469721911 shutdown_connections 2021-11-03T17:31:35.086 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.087+0000 7f268144d700 1 -- 172.21.15.27:0/1469721911 wait complete. 2021-11-03T17:31:35.086 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.087+0000 7f268144d700 1 Processor -- start 2021-11-03T17:31:35.086 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.087+0000 7f268144d700 1 -- start start 2021-11-03T17:31:35.087 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.087+0000 7f268144d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f267c0fcce0 0x7f267c10d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:31:35.087 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.087+0000 7f268144d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f267c0fdf80 0x7f267c10da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:31:35.087 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.087+0000 7f268144d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267c10dfd0 con 0x7f267c0fdf80 2021-11-03T17:31:35.087 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.087+0000 7f268144d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f267c10e110 con 0x7f267c0fcce0 2021-11-03T17:31:35.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.088+0000 7f267a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f267c0fdf80 0x7f267c10da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:31:35.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.088+0000 7f267a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f267c0fdf80 0x7f267c10da20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57942/0 (socket says 172.21.15.27:57942) 2021-11-03T17:31:35.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.088+0000 7f267a7fc700 1 -- 172.21.15.27:0/3788537229 learned_addr learned my addr 172.21.15.27:0/3788537229 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:31:35.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.088+0000 7f267affd700 1 --2- 172.21.15.27:0/3788537229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f267c0fcce0 0x7f267c10d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:31:35.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.088+0000 7f267a7fc700 1 -- 172.21.15.27:0/3788537229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f267c0fcce0 msgr2=0x7f267c10d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:31:35.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.088+0000 7f267a7fc700 1 --2- 172.21.15.27:0/3788537229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f267c0fcce0 0x7f267c10d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:35.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.088+0000 7f267a7fc700 1 -- 172.21.15.27:0/3788537229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f266c005040 con 0x7f267c0fdf80 2021-11-03T17:31:35.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.088+0000 7f267a7fc700 1 --2- 172.21.15.27:0/3788537229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f267c0fdf80 0x7f267c10da20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f267000e7f0 tx=0x7f267000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:31:35.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.088+0000 7f26637fe700 1 -- 172.21.15.27:0/3788537229 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26700092e0 con 0x7f267c0fdf80 2021-11-03T17:31:35.090 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.088+0000 7f26637fe700 1 -- 172.21.15.27:0/3788537229 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f267000f040 con 0x7f267c0fdf80 2021-11-03T17:31:35.090 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.088+0000 7f268144d700 1 -- 172.21.15.27:0/3788537229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f267c10eab0 con 0x7f267c0fdf80 2021-11-03T17:31:35.090 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.088+0000 7f26637fe700 1 -- 172.21.15.27:0/3788537229 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2670014750 con 0x7f267c0fdf80 2021-11-03T17:31:35.090 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.088+0000 7f268144d700 1 -- 172.21.15.27:0/3788537229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f267c10f070 con 0x7f267c0fdf80 2021-11-03T17:31:35.092 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.089+0000 7f268144d700 1 -- 172.21.15.27:0/3788537229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f267c045ad0 con 0x7f267c0fdf80 2021-11-03T17:31:35.092 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.090+0000 7f26637fe700 1 -- 172.21.15.27:0/3788537229 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f267001b030 con 0x7f267c0fdf80 2021-11-03T17:31:35.092 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.090+0000 7f26637fe700 1 --2- 172.21.15.27:0/3788537229 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f26640640b0 0x7f2664066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:31:35.093 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.090+0000 7f26637fe700 1 -- 172.21.15.27:0/3788537229 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f2670080630 con 0x7f267c0fdf80 2021-11-03T17:31:35.093 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.090+0000 7f267affd700 1 --2- 172.21.15.27:0/3788537229 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f26640640b0 0x7f2664066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:31:35.094 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.091+0000 7f267affd700 1 --2- 172.21.15.27:0/3788537229 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f26640640b0 0x7f2664066570 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f266c004d80 tx=0x7f266c016040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:31:35.094 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.094+0000 7f26637fe700 1 -- 172.21.15.27:0/3788537229 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f26700528f0 con 0x7f267c0fdf80 2021-11-03T17:31:35.318 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.324+0000 7f268144d700 1 -- 172.21.15.27:0/3788537229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f267c1055a0 con 0x7f267c0fdf80 2021-11-03T17:31:35.319 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.325+0000 7f26637fe700 1 -- 172.21.15.27:0/3788537229 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f26700192e0 con 0x7f267c0fdf80 2021-11-03T17:31:35.320 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:31:35.320 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:31:35.320 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:31:35.320 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:31:35.321 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:31:35.321 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:31:35.321 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:31:35.321 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:31:35.321 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:31:35.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:31:35.322 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:31:35.322 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:31:35.322 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:31:35.323 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:31:35.323 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:31:35.323 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:31:35.323 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:31:35.324 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:31:35.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.327+0000 7f268144d700 1 -- 172.21.15.27:0/3788537229 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f26640640b0 msgr2=0x7f2664066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:31:35.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.327+0000 7f268144d700 1 --2- 172.21.15.27:0/3788537229 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f26640640b0 0x7f2664066570 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f266c004d80 tx=0x7f266c016040).stop 2021-11-03T17:31:35.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.327+0000 7f268144d700 1 -- 172.21.15.27:0/3788537229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f267c0fdf80 msgr2=0x7f267c10da20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:31:35.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.327+0000 7f268144d700 1 --2- 172.21.15.27:0/3788537229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f267c0fdf80 0x7f267c10da20 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f267000e7f0 tx=0x7f267000b040).stop 2021-11-03T17:31:35.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.327+0000 7f268144d700 1 -- 172.21.15.27:0/3788537229 shutdown_connections 2021-11-03T17:31:35.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.327+0000 7f268144d700 1 --2- 172.21.15.27:0/3788537229 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f26640640b0 0x7f2664066570 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:35.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.327+0000 7f268144d700 1 --2- 172.21.15.27:0/3788537229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f267c0fcce0 0x7f267c10d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:35.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.327+0000 7f268144d700 1 --2- 172.21.15.27:0/3788537229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f267c0fdf80 0x7f267c10da20 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:35.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.327+0000 7f268144d700 1 -- 172.21.15.27:0/3788537229 >> 172.21.15.27:0/3788537229 conn(0x7f267c06c000 msgr2=0x7f267c1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:31:35.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.327+0000 7f268144d700 1 -- 172.21.15.27:0/3788537229 shutdown_connections 2021-11-03T17:31:35.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.327+0000 7f268144d700 1 -- 172.21.15.27:0/3788537229 wait complete. 2021-11-03T17:31:35.423 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.427+0000 7f0e2a348700 1 -- 172.21.15.27:0/2948535619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e240f53e0 msgr2=0x7f0e240f7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:31:35.424 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.427+0000 7f0e2a348700 1 --2- 172.21.15.27:0/2948535619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e240f53e0 0x7f0e240f7d20 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 rx=0x7f0e0c004660 tx=0x7f0e0c009b30).stop 2021-11-03T17:31:35.424 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.427+0000 7f0e2a348700 1 -- 172.21.15.27:0/2948535619 shutdown_connections 2021-11-03T17:31:35.424 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.427+0000 7f0e2a348700 1 --2- 172.21.15.27:0/2948535619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e240f53e0 0x7f0e240f7d20 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:35.424 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.427+0000 7f0e2a348700 1 --2- 172.21.15.27:0/2948535619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e240f4a40 0x7f0e240f4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:35.425 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.427+0000 7f0e2a348700 1 -- 172.21.15.27:0/2948535619 >> 172.21.15.27:0/2948535619 conn(0x7f0e240f04c0 msgr2=0x7f0e240f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:31:35.425 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.427+0000 7f0e2a348700 1 -- 172.21.15.27:0/2948535619 shutdown_connections 2021-11-03T17:31:35.425 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.427+0000 7f0e2a348700 1 -- 172.21.15.27:0/2948535619 wait complete. 2021-11-03T17:31:35.425 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.428+0000 7f0e2a348700 1 Processor -- start 2021-11-03T17:31:35.425 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.428+0000 7f0e2a348700 1 -- start start 2021-11-03T17:31:35.426 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.428+0000 7f0e2a348700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e240f4a40 0x7f0e241070c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:31:35.426 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.428+0000 7f0e2a348700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e240f53e0 0x7f0e241075b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:31:35.426 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.428+0000 7f0e2a348700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e24107b60 con 0x7f0e240f53e0 2021-11-03T17:31:35.426 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.428+0000 7f0e2a348700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e24107ca0 con 0x7f0e240f4a40 2021-11-03T17:31:35.427 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.428+0000 7f0e237fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e240f53e0 0x7f0e241075b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:31:35.427 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.428+0000 7f0e237fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e240f53e0 0x7f0e241075b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57952/0 (socket says 172.21.15.27:57952) 2021-11-03T17:31:35.427 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.428+0000 7f0e237fe700 1 -- 172.21.15.27:0/1898101600 learned_addr learned my addr 172.21.15.27:0/1898101600 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:31:35.427 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.428+0000 7f0e23fff700 1 --2- 172.21.15.27:0/1898101600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e240f4a40 0x7f0e241070c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:31:35.428 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.428+0000 7f0e237fe700 1 -- 172.21.15.27:0/1898101600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e240f4a40 msgr2=0x7f0e241070c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:31:35.428 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.428+0000 7f0e237fe700 1 --2- 172.21.15.27:0/1898101600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e240f4a40 0x7f0e241070c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:35.428 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.428+0000 7f0e237fe700 1 -- 172.21.15.27:0/1898101600 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e0c005040 con 0x7f0e240f53e0 2021-11-03T17:31:35.428 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.428+0000 7f0e23fff700 1 --2- 172.21.15.27:0/1898101600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e240f4a40 0x7f0e241070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:31:35.429 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.429+0000 7f0e237fe700 1 --2- 172.21.15.27:0/1898101600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e240f53e0 0x7f0e241075b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f0e0c013a70 tx=0x7f0e0c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:31:35.429 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.429+0000 7f0e20ff9700 1 -- 172.21.15.27:0/1898101600 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e0c018070 con 0x7f0e240f53e0 2021-11-03T17:31:35.429 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.429+0000 7f0e20ff9700 1 -- 172.21.15.27:0/1898101600 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f0e0c0097d0 con 0x7f0e240f53e0 2021-11-03T17:31:35.429 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.429+0000 7f0e2a348700 1 -- 172.21.15.27:0/1898101600 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e241085e0 con 0x7f0e240f53e0 2021-11-03T17:31:35.429 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.429+0000 7f0e20ff9700 1 -- 172.21.15.27:0/1898101600 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e0c01c3c0 con 0x7f0e240f53e0 2021-11-03T17:31:35.430 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.429+0000 7f0e2a348700 1 -- 172.21.15.27:0/1898101600 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e24108b40 con 0x7f0e240f53e0 2021-11-03T17:31:35.431 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.430+0000 7f0e2a348700 1 -- 172.21.15.27:0/1898101600 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e24045ad0 con 0x7f0e240f53e0 2021-11-03T17:31:35.431 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.430+0000 7f0e20ff9700 1 -- 172.21.15.27:0/1898101600 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f0e0c007ba0 con 0x7f0e240f53e0 2021-11-03T17:31:35.431 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.430+0000 7f0e20ff9700 1 --2- 172.21.15.27:0/1898101600 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0e100640b0 0x7f0e10066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:31:35.432 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.431+0000 7f0e20ff9700 1 -- 172.21.15.27:0/1898101600 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f0e0c082510 con 0x7f0e240f53e0 2021-11-03T17:31:35.432 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.431+0000 7f0e23fff700 1 --2- 172.21.15.27:0/1898101600 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0e100640b0 0x7f0e10066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:31:35.432 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.431+0000 7f0e23fff700 1 --2- 172.21.15.27:0/1898101600 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0e100640b0 0x7f0e10066570 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f0e14009880 tx=0x7f0e1400b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:31:35.432 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.435+0000 7f0e20ff9700 1 -- 172.21.15.27:0/1898101600 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f0e0c054810 con 0x7f0e240f53e0 2021-11-03T17:31:35.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:35 smithi115 ceph-mon[65453]: pgmap v508: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:35.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 937.381056 secs 2021-11-03T17:31:35.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:35 smithi027 ceph-mon[97244]: pgmap v508: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:35.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 937.381056 secs 2021-11-03T17:31:35.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.636+0000 7f0e2a348700 1 -- 172.21.15.27:0/1898101600 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0e240ff0f0 con 0x7f0e240f53e0 2021-11-03T17:31:35.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.636+0000 7f0e20ff9700 1 -- 172.21.15.27:0/1898101600 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f0e0c054400 con 0x7f0e240f53e0 2021-11-03T17:31:35.631 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:31:35.631 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:31:35.632 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:31:35.632 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:31:35.632 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:31:35.632 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:31:35.633 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:31:35.633 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:31:35.633 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:31:35.633 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:31:35.634 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:31:35.634 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:31:35.634 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:31:35.634 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:31:35.635 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:31:35.635 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:31:35.636 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:31:35.636 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:31:35.637 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:31:35.637 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:31:35.638 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:31:35.638 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:31:35.638 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:31:35.639 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:31:35.639 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:31:35.639 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:31:35.639 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:31:35.640 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:31:35.640 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:31:35.640 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:31:35.640 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:31:35.640 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:31:35.641 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:31:35.641 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:31:35.641 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:31:35.642 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:31:35.642 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:31:35.642 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:31:35.643 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:31:35.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.638+0000 7f0e2a348700 1 -- 172.21.15.27:0/1898101600 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0e100640b0 msgr2=0x7f0e10066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:31:35.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.638+0000 7f0e2a348700 1 --2- 172.21.15.27:0/1898101600 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0e100640b0 0x7f0e10066570 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f0e14009880 tx=0x7f0e1400b040).stop 2021-11-03T17:31:35.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.638+0000 7f0e2a348700 1 -- 172.21.15.27:0/1898101600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e240f53e0 msgr2=0x7f0e241075b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:31:35.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.638+0000 7f0e2a348700 1 --2- 172.21.15.27:0/1898101600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e240f53e0 0x7f0e241075b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f0e0c013a70 tx=0x7f0e0c009b30).stop 2021-11-03T17:31:35.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.638+0000 7f0e2a348700 1 -- 172.21.15.27:0/1898101600 shutdown_connections 2021-11-03T17:31:35.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.638+0000 7f0e2a348700 1 --2- 172.21.15.27:0/1898101600 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0e100640b0 0x7f0e10066570 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:35.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.638+0000 7f0e2a348700 1 --2- 172.21.15.27:0/1898101600 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e240f4a40 0x7f0e241070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:35.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.638+0000 7f0e2a348700 1 --2- 172.21.15.27:0/1898101600 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e240f53e0 0x7f0e241075b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:31:35.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.638+0000 7f0e2a348700 1 -- 172.21.15.27:0/1898101600 >> 172.21.15.27:0/1898101600 conn(0x7f0e240f04c0 msgr2=0x7f0e240fb770 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:31:35.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.638+0000 7f0e2a348700 1 -- 172.21.15.27:0/1898101600 shutdown_connections 2021-11-03T17:31:35.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:31:35.639+0000 7f0e2a348700 1 -- 172.21.15.27:0/1898101600 wait complete. 2021-11-03T17:31:35.646 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:31:36.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:36 smithi115 ceph-mon[65453]: from='client.34632 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:31:36.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:36 smithi115 ceph-mon[65453]: from='client.34636 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:31:36.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:36 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3788537229' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:36.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:36 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1898101600' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:31:36.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:36 smithi027 ceph-mon[97244]: from='client.34632 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:31:36.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:36 smithi027 ceph-mon[97244]: from='client.34636 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:31:36.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:36 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3788537229' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:36.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:36 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1898101600' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:31:37.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:37 smithi115 ceph-mon[65453]: pgmap v509: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:37.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:37 smithi027 ceph-mon[97244]: pgmap v509: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:38.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:38 smithi115 ceph-mon[65453]: pgmap v510: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:38.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:38 smithi027 ceph-mon[97244]: pgmap v510: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:39.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:39 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 942.381177 secs 2021-11-03T17:31:40.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:39 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 942.381177 secs 2021-11-03T17:31:40.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:40 smithi115 ceph-mon[65453]: pgmap v511: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:40.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:31:40.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:31:40.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:31:40.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:40.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:40.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:40.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:41.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:40 smithi027 ceph-mon[97244]: pgmap v511: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:41.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:31:41.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:31:41.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:31:41.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:41.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:41.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:41.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:41.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:41 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:31:42.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:41 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:31:42.936 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:42 smithi027 ceph-mon[97244]: pgmap v512: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:42.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:42 smithi115 ceph-mon[65453]: pgmap v512: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:45.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:45 smithi115 ceph-mon[65453]: pgmap v513: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:45.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 947.381316 secs 2021-11-03T17:31:45.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:45 smithi027 ceph-mon[97244]: pgmap v513: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:45.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 947.381316 secs 2021-11-03T17:31:47.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:47 smithi115 ceph-mon[65453]: pgmap v514: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:47.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:47 smithi027 ceph-mon[97244]: pgmap v514: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:49.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:49 smithi115 ceph-mon[65453]: pgmap v515: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:49.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:49 smithi027 ceph-mon[97244]: pgmap v515: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:50.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 952.381440 secs 2021-11-03T17:31:50.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 952.381440 secs 2021-11-03T17:31:51.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:51 smithi115 ceph-mon[65453]: pgmap v516: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:51.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:31:51.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:31:51.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:31:51.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:51.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:51.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:51.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:51.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:51 smithi027 ceph-mon[97244]: pgmap v516: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:51.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:31:51.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:31:51.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:31:51.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:51.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:51.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:51.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:31:52.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:52 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:31:52.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:52 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:31:53.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:53 smithi027 ceph-mon[97244]: pgmap v517: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:53.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:53 smithi115 ceph-mon[65453]: pgmap v517: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:55.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:55 smithi115 ceph-mon[65453]: pgmap v518: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:55.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 957.381577 secs 2021-11-03T17:31:55.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:55 smithi027 ceph-mon[97244]: pgmap v518: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:55.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 957.381577 secs 2021-11-03T17:31:56.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:31:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:31:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:56 smithi115 ceph-mon[65453]: pgmap v519: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:31:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:31:56.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:56 smithi027 ceph-mon[97244]: pgmap v519: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:59.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:31:59 smithi115 ceph-mon[65453]: pgmap v520: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:31:59.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:31:59 smithi027 ceph-mon[97244]: pgmap v520: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:00.418 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 962.381711 secs 2021-11-03T17:32:00.419 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 962.381711 secs 2021-11-03T17:32:01.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:01 smithi115 ceph-mon[65453]: pgmap v521: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:01.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:32:01.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:32:01.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:32:01.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:01.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:01.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:01.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:01.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:01 smithi027 ceph-mon[97244]: pgmap v521: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:01.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:32:01.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:32:01.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:32:01.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:01.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:01.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:01.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:02.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:02 smithi115 ceph-mon[65453]: poll took 60179.391 ms: 1 events 2021-11-03T17:32:02.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:02 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:32:02.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:02 smithi027 ceph-mon[97244]: poll took 60179.391 ms: 1 events 2021-11-03T17:32:02.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:02 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:32:03.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:03 smithi027 ceph-mon[97244]: pgmap v522: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:03.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:03 smithi115 ceph-mon[65453]: pgmap v522: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:05.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:05 smithi115 ceph-mon[65453]: pgmap v523: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:05.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 967.381856 secs 2021-11-03T17:32:05.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:05 smithi027 ceph-mon[97244]: pgmap v523: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:05.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 967.381856 secs 2021-11-03T17:32:05.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.754+0000 7f8666ab5700 1 -- 172.21.15.27:0/2866661517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86600f77d0 msgr2=0x7f86600f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:32:05.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.754+0000 7f8666ab5700 1 --2- 172.21.15.27:0/2866661517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86600f77d0 0x7f86600f7c30 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f8650004660 tx=0x7f8650009b30).stop 2021-11-03T17:32:05.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.754+0000 7f8666ab5700 1 -- 172.21.15.27:0/2866661517 shutdown_connections 2021-11-03T17:32:05.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.754+0000 7f8666ab5700 1 --2- 172.21.15.27:0/2866661517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86600f77d0 0x7f86600f7c30 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:05.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.754+0000 7f8666ab5700 1 --2- 172.21.15.27:0/2866661517 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86600f6530 0x7f86600f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:05.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.754+0000 7f8666ab5700 1 -- 172.21.15.27:0/2866661517 >> 172.21.15.27:0/2866661517 conn(0x7f86600f1cd0 msgr2=0x7f86600f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:32:05.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.754+0000 7f8666ab5700 1 -- 172.21.15.27:0/2866661517 shutdown_connections 2021-11-03T17:32:05.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.754+0000 7f8666ab5700 1 -- 172.21.15.27:0/2866661517 wait complete. 2021-11-03T17:32:05.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.754+0000 7f8666ab5700 1 Processor -- start 2021-11-03T17:32:05.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.755+0000 7f8666ab5700 1 -- start start 2021-11-03T17:32:05.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.755+0000 7f8666ab5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86600f6530 0x7f866006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:32:05.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.755+0000 7f8666ab5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86600f77d0 0x7f8660069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:32:05.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.755+0000 7f8666ab5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866006a380 con 0x7f86600f77d0 2021-11-03T17:32:05.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.755+0000 7f8666ab5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866006a4c0 con 0x7f86600f6530 2021-11-03T17:32:05.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.755+0000 7f86652b2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86600f77d0 0x7f8660069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:32:05.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.755+0000 7f86652b2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86600f77d0 0x7f8660069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57970/0 (socket says 172.21.15.27:57970) 2021-11-03T17:32:05.754 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.755+0000 7f86652b2700 1 -- 172.21.15.27:0/1333231935 learned_addr learned my addr 172.21.15.27:0/1333231935 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:32:05.754 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.755+0000 7f8665ab3700 1 --2- 172.21.15.27:0/1333231935 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86600f6530 0x7f866006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:32:05.754 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.755+0000 7f86652b2700 1 -- 172.21.15.27:0/1333231935 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86600f6530 msgr2=0x7f866006b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:32:05.754 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.755+0000 7f86652b2700 1 --2- 172.21.15.27:0/1333231935 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86600f6530 0x7f866006b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:05.755 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.755+0000 7f86652b2700 1 -- 172.21.15.27:0/1333231935 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8650005040 con 0x7f86600f77d0 2021-11-03T17:32:05.755 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.755+0000 7f86652b2700 1 --2- 172.21.15.27:0/1333231935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86600f77d0 0x7f8660069e20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f8650009990 tx=0x7f8650009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:32:05.755 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.756+0000 7f86567fc700 1 -- 172.21.15.27:0/1333231935 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8650018070 con 0x7f86600f77d0 2021-11-03T17:32:05.755 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.756+0000 7f8666ab5700 1 -- 172.21.15.27:0/1333231935 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f866006a740 con 0x7f86600f77d0 2021-11-03T17:32:05.756 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.756+0000 7f86567fc700 1 -- 172.21.15.27:0/1333231935 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f8650007ba0 con 0x7f86600f77d0 2021-11-03T17:32:05.756 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.756+0000 7f86567fc700 1 -- 172.21.15.27:0/1333231935 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f865001c620 con 0x7f86600f77d0 2021-11-03T17:32:05.756 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.756+0000 7f8666ab5700 1 -- 172.21.15.27:0/1333231935 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f866006aca0 con 0x7f86600f77d0 2021-11-03T17:32:05.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.757+0000 7f8666ab5700 1 -- 172.21.15.27:0/1333231935 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8660045ad0 con 0x7f86600f77d0 2021-11-03T17:32:05.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.757+0000 7f86567fc700 1 -- 172.21.15.27:0/1333231935 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f865001c8e0 con 0x7f86600f77d0 2021-11-03T17:32:05.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.757+0000 7f86567fc700 1 --2- 172.21.15.27:0/1333231935 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f864c064100 0x7f864c0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:32:05.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.758+0000 7f86567fc700 1 -- 172.21.15.27:0/1333231935 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f8650082640 con 0x7f86600f77d0 2021-11-03T17:32:05.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.758+0000 7f8665ab3700 1 --2- 172.21.15.27:0/1333231935 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f864c064100 0x7f864c0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:32:05.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.758+0000 7f8665ab3700 1 --2- 172.21.15.27:0/1333231935 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f864c064100 0x7f864c0665c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f866006b730 tx=0x7f865c009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:32:05.759 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.762+0000 7f86567fc700 1 -- 172.21.15.27:0/1333231935 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f8650054a00 con 0x7f86600f77d0 2021-11-03T17:32:05.936 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.941+0000 7f8666ab5700 1 -- 172.21.15.27:0/1333231935 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f86600fed60 con 0x7f864c064100 2021-11-03T17:32:05.937 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.943+0000 7f86567fc700 1 -- 172.21.15.27:0/1333231935 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f86600fed60 con 0x7f864c064100 2021-11-03T17:32:05.939 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.944+0000 7f8666ab5700 1 -- 172.21.15.27:0/1333231935 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f864c064100 msgr2=0x7f864c0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:32:05.939 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.944+0000 7f8666ab5700 1 --2- 172.21.15.27:0/1333231935 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f864c064100 0x7f864c0665c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f866006b730 tx=0x7f865c009040).stop 2021-11-03T17:32:05.939 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.944+0000 7f8666ab5700 1 -- 172.21.15.27:0/1333231935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86600f77d0 msgr2=0x7f8660069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:32:05.940 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.944+0000 7f8666ab5700 1 --2- 172.21.15.27:0/1333231935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86600f77d0 0x7f8660069e20 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f8650009990 tx=0x7f8650009b30).stop 2021-11-03T17:32:05.940 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.944+0000 7f8666ab5700 1 -- 172.21.15.27:0/1333231935 shutdown_connections 2021-11-03T17:32:05.940 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.945+0000 7f8666ab5700 1 --2- 172.21.15.27:0/1333231935 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f864c064100 0x7f864c0665c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:05.941 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.945+0000 7f8666ab5700 1 --2- 172.21.15.27:0/1333231935 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f86600f6530 0x7f866006b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:05.941 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.945+0000 7f8666ab5700 1 --2- 172.21.15.27:0/1333231935 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f86600f77d0 0x7f8660069e20 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:05.941 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.945+0000 7f8666ab5700 1 -- 172.21.15.27:0/1333231935 >> 172.21.15.27:0/1333231935 conn(0x7f86600f1cd0 msgr2=0x7f86600fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:32:05.941 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.945+0000 7f8666ab5700 1 -- 172.21.15.27:0/1333231935 shutdown_connections 2021-11-03T17:32:05.942 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:05.945+0000 7f8666ab5700 1 -- 172.21.15.27:0/1333231935 wait complete. 2021-11-03T17:32:05.948 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:32:06.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.045+0000 7fbcdb33d700 1 -- 172.21.15.27:0/2343070238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcd40f7980 msgr2=0x7fbcd40f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:32:06.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.045+0000 7fbcdb33d700 1 --2- 172.21.15.27:0/2343070238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcd40f7980 0x7fbcd40f7de0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7fbcc8004660 tx=0x7fbcc8009b30).stop 2021-11-03T17:32:06.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.045+0000 7fbcdb33d700 1 -- 172.21.15.27:0/2343070238 shutdown_connections 2021-11-03T17:32:06.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.045+0000 7fbcdb33d700 1 --2- 172.21.15.27:0/2343070238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcd40f7980 0x7fbcd40f7de0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:06.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.045+0000 7fbcdb33d700 1 --2- 172.21.15.27:0/2343070238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcd40f66e0 0x7fbcd40f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:06.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.045+0000 7fbcdb33d700 1 -- 172.21.15.27:0/2343070238 >> 172.21.15.27:0/2343070238 conn(0x7fbcd40f1e40 msgr2=0x7fbcd40f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:32:06.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.045+0000 7fbcdb33d700 1 -- 172.21.15.27:0/2343070238 shutdown_connections 2021-11-03T17:32:06.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.045+0000 7fbcdb33d700 1 -- 172.21.15.27:0/2343070238 wait complete. 2021-11-03T17:32:06.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.045+0000 7fbcdb33d700 1 Processor -- start 2021-11-03T17:32:06.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.045+0000 7fbcdb33d700 1 -- start start 2021-11-03T17:32:06.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.045+0000 7fbcdb33d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcd40f66e0 0x7fbcd4106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:32:06.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.045+0000 7fbcdb33d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcd40f7980 0x7fbcd4107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:32:06.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.046+0000 7fbcdb33d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd4107a00 con 0x7fbcd40f7980 2021-11-03T17:32:06.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.046+0000 7fbcdb33d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd4107b40 con 0x7fbcd40f66e0 2021-11-03T17:32:06.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.046+0000 7fbcd88d8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcd40f7980 0x7fbcd4107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:32:06.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.046+0000 7fbcd88d8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcd40f7980 0x7fbcd4107450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57980/0 (socket says 172.21.15.27:57980) 2021-11-03T17:32:06.046 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.046+0000 7fbcd88d8700 1 -- 172.21.15.27:0/3288030849 learned_addr learned my addr 172.21.15.27:0/3288030849 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:32:06.046 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.046+0000 7fbcd90d9700 1 --2- 172.21.15.27:0/3288030849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcd40f66e0 0x7fbcd4106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:32:06.046 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.046+0000 7fbcd88d8700 1 -- 172.21.15.27:0/3288030849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcd40f66e0 msgr2=0x7fbcd4106f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:32:06.046 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.046+0000 7fbcd88d8700 1 --2- 172.21.15.27:0/3288030849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcd40f66e0 0x7fbcd4106f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:06.047 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.046+0000 7fbcd88d8700 1 -- 172.21.15.27:0/3288030849 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc8005040 con 0x7fbcd40f7980 2021-11-03T17:32:06.047 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.046+0000 7fbcd88d8700 1 --2- 172.21.15.27:0/3288030849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcd40f7980 0x7fbcd4107450 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7fbcc8009990 tx=0x7fbcc8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:32:06.047 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.046+0000 7fbcd1ffb700 1 -- 172.21.15.27:0/3288030849 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcc8018070 con 0x7fbcd40f7980 2021-11-03T17:32:06.047 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.046+0000 7fbcdb33d700 1 -- 172.21.15.27:0/3288030849 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcd4108480 con 0x7fbcd40f7980 2021-11-03T17:32:06.048 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.046+0000 7fbcd1ffb700 1 -- 172.21.15.27:0/3288030849 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fbcc8007ba0 con 0x7fbcd40f7980 2021-11-03T17:32:06.048 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.046+0000 7fbcd1ffb700 1 -- 172.21.15.27:0/3288030849 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcc801c3c0 con 0x7fbcd40f7980 2021-11-03T17:32:06.048 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.047+0000 7fbcdb33d700 1 -- 172.21.15.27:0/3288030849 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcd41089e0 con 0x7fbcd40f7980 2021-11-03T17:32:06.048 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.048+0000 7fbcdb33d700 1 -- 172.21.15.27:0/3288030849 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcd4045ad0 con 0x7fbcd40f7980 2021-11-03T17:32:06.049 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.048+0000 7fbcd1ffb700 1 -- 172.21.15.27:0/3288030849 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fbcc801d420 con 0x7fbcd40f7980 2021-11-03T17:32:06.050 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.048+0000 7fbcd1ffb700 1 --2- 172.21.15.27:0/3288030849 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbcbc0640b0 0x7fbcbc066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:32:06.050 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.048+0000 7fbcd1ffb700 1 -- 172.21.15.27:0/3288030849 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fbcc8082470 con 0x7fbcd40f7980 2021-11-03T17:32:06.050 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.048+0000 7fbcd90d9700 1 --2- 172.21.15.27:0/3288030849 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbcbc0640b0 0x7fbcbc066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:32:06.050 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.049+0000 7fbcd90d9700 1 --2- 172.21.15.27:0/3288030849 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbcbc0640b0 0x7fbcbc066570 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fbcd40f77e0 tx=0x7fbccc009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:32:06.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.052+0000 7fbcd1ffb700 1 -- 172.21.15.27:0/3288030849 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fbcc8054710 con 0x7fbcd40f7980 2021-11-03T17:32:06.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.228+0000 7fbcdb33d700 1 -- 172.21.15.27:0/3288030849 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbcd40fefb0 con 0x7fbcbc0640b0 2021-11-03T17:32:06.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.243+0000 7fbcd1ffb700 1 -- 172.21.15.27:0/3288030849 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fbcd40fefb0 con 0x7fbcbc0640b0 2021-11-03T17:32:06.237 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:32:06.238 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (23m) 3m ago 24m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:32:06.238 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (15m) 3m ago 24m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:32:06.238 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (15m) 2m ago 24m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:32:06.238 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (23m) 3m ago 24m 40.2M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:32:06.239 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (17m) 3m ago 20m 18.2M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:32:06.239 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (20m) 3m ago 20m 3841M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:32:06.239 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (15m) 2m ago 20m 17.6M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:32:06.239 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (17m) 2m ago 20m 19.3M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:32:06.240 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (17m) 3m ago 26m 497M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:32:06.240 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (16m) 2m ago 24m 415M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:32:06.240 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (16m) 3m ago 26m 119M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:32:06.240 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (15m) 2m ago 24m 71.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:32:06.240 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (24m) 3m ago 24m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:32:06.241 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (23m) 2m ago 23m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:32:06.241 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (14m) 3m ago 23m 290M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:32:06.241 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (14m) 3m ago 22m 317M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:32:06.241 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (13m) 3m ago 22m 229M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:32:06.242 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (13m) 2m ago 22m 329M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:32:06.242 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (13m) 2m ago 21m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:32:06.242 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (12m) 2m ago 21m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:32:06.242 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (23m) 3m ago 24m 48.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:32:06.243 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.245+0000 7fbcdb33d700 1 -- 172.21.15.27:0/3288030849 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbcbc0640b0 msgr2=0x7fbcbc066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:32:06.243 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.245+0000 7fbcdb33d700 1 --2- 172.21.15.27:0/3288030849 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbcbc0640b0 0x7fbcbc066570 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7fbcd40f77e0 tx=0x7fbccc009040).stop 2021-11-03T17:32:06.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.245+0000 7fbcdb33d700 1 -- 172.21.15.27:0/3288030849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcd40f7980 msgr2=0x7fbcd4107450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:32:06.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.245+0000 7fbcdb33d700 1 --2- 172.21.15.27:0/3288030849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcd40f7980 0x7fbcd4107450 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7fbcc8009990 tx=0x7fbcc8009b30).stop 2021-11-03T17:32:06.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.245+0000 7fbcdb33d700 1 -- 172.21.15.27:0/3288030849 shutdown_connections 2021-11-03T17:32:06.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.245+0000 7fbcdb33d700 1 --2- 172.21.15.27:0/3288030849 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbcbc0640b0 0x7fbcbc066570 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:06.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.245+0000 7fbcdb33d700 1 --2- 172.21.15.27:0/3288030849 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcd40f66e0 0x7fbcd4106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:06.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.245+0000 7fbcdb33d700 1 --2- 172.21.15.27:0/3288030849 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcd40f7980 0x7fbcd4107450 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:06.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.245+0000 7fbcdb33d700 1 -- 172.21.15.27:0/3288030849 >> 172.21.15.27:0/3288030849 conn(0x7fbcd40f1e40 msgr2=0x7fbcd40fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:32:06.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.245+0000 7fbcdb33d700 1 -- 172.21.15.27:0/3288030849 shutdown_connections 2021-11-03T17:32:06.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.245+0000 7fbcdb33d700 1 -- 172.21.15.27:0/3288030849 wait complete. 2021-11-03T17:32:06.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.344+0000 7f695071e700 1 -- 172.21.15.27:0/2870029754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69480f55c0 msgr2=0x7f69480f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:32:06.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.344+0000 7f695071e700 1 --2- 172.21.15.27:0/2870029754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69480f55c0 0x7f69480f59e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7f6938004660 tx=0x7f6938009b30).stop 2021-11-03T17:32:06.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.344+0000 7f695071e700 1 -- 172.21.15.27:0/2870029754 shutdown_connections 2021-11-03T17:32:06.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.344+0000 7f695071e700 1 --2- 172.21.15.27:0/2870029754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69480f6860 0x7f69480f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:32:06.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.344+0000 7f695071e700 1 --2- 172.21.15.27:0/2870029754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69480f55c0 0x7f69480f59e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:06.342 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.344+0000 7f695071e700 1 -- 172.21.15.27:0/2870029754 >> 172.21.15.27:0/2870029754 conn(0x7f69480f0d40 msgr2=0x7f69480f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:32:06.343 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.344+0000 7f695071e700 1 -- 172.21.15.27:0/2870029754 shutdown_connections 2021-11-03T17:32:06.343 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.344+0000 7f695071e700 1 -- 172.21.15.27:0/2870029754 wait complete. 2021-11-03T17:32:06.343 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.344+0000 7f695071e700 1 Processor -- start 2021-11-03T17:32:06.343 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.344+0000 7f695071e700 1 -- start start 2021-11-03T17:32:06.344 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.345+0000 7f695071e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69480f55c0 0x7f6948105ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:32:06.344 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.345+0000 7f695071e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69480f6860 0x7f69481063b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:32:06.344 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.345+0000 7f695071e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69481069f0 con 0x7f69480f55c0 2021-11-03T17:32:06.344 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.345+0000 7f695071e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69481072a0 con 0x7f69480f6860 2021-11-03T17:32:06.345 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.345+0000 7f694e4ba700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69480f55c0 0x7f6948105ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:32:06.345 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.345+0000 7f694e4ba700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69480f55c0 0x7f6948105ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:57988/0 (socket says 172.21.15.27:57988) 2021-11-03T17:32:06.345 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.345+0000 7f694e4ba700 1 -- 172.21.15.27:0/959646061 learned_addr learned my addr 172.21.15.27:0/959646061 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:32:06.345 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.345+0000 7f694dcb9700 1 --2- 172.21.15.27:0/959646061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69480f6860 0x7f69481063b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:32:06.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.345+0000 7f694e4ba700 1 -- 172.21.15.27:0/959646061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69480f6860 msgr2=0x7f69481063b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:32:06.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.345+0000 7f694e4ba700 1 --2- 172.21.15.27:0/959646061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69480f6860 0x7f69481063b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:06.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.345+0000 7f694e4ba700 1 -- 172.21.15.27:0/959646061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6938005040 con 0x7f69480f55c0 2021-11-03T17:32:06.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.345+0000 7f694e4ba700 1 --2- 172.21.15.27:0/959646061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69480f55c0 0x7f6948105ec0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f6938004f60 tx=0x7f6938009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:32:06.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.345+0000 7f693effd700 1 -- 172.21.15.27:0/959646061 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6938018070 con 0x7f69480f55c0 2021-11-03T17:32:06.347 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.345+0000 7f693effd700 1 -- 172.21.15.27:0/959646061 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f6938009770 con 0x7f69480f55c0 2021-11-03T17:32:06.347 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.345+0000 7f695071e700 1 -- 172.21.15.27:0/959646061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6948107520 con 0x7f69480f55c0 2021-11-03T17:32:06.347 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.345+0000 7f693effd700 1 -- 172.21.15.27:0/959646061 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f693801c3c0 con 0x7f69480f55c0 2021-11-03T17:32:06.347 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.346+0000 7f695071e700 1 -- 172.21.15.27:0/959646061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6948107a80 con 0x7f69480f55c0 2021-11-03T17:32:06.348 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.346+0000 7f695071e700 1 -- 172.21.15.27:0/959646061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69480f49b0 con 0x7f69480f55c0 2021-11-03T17:32:06.349 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.347+0000 7f693effd700 1 -- 172.21.15.27:0/959646061 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f693801d420 con 0x7f69480f55c0 2021-11-03T17:32:06.349 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.347+0000 7f693effd700 1 --2- 172.21.15.27:0/959646061 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f69340640b0 0x7f6934066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:32:06.349 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.347+0000 7f693effd700 1 -- 172.21.15.27:0/959646061 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f69380825e0 con 0x7f69480f55c0 2021-11-03T17:32:06.350 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.347+0000 7f694dcb9700 1 --2- 172.21.15.27:0/959646061 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f69340640b0 0x7f6934066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:32:06.350 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.348+0000 7f694dcb9700 1 --2- 172.21.15.27:0/959646061 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f69340640b0 0x7f6934066570 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f6944006fd0 tx=0x7f6944008040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:32:06.350 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.351+0000 7f693effd700 1 -- 172.21.15.27:0/959646061 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f69380549a0 con 0x7f69480f55c0 2021-11-03T17:32:06.580 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.585+0000 7f695071e700 1 -- 172.21.15.27:0/959646061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6948027a00 con 0x7f69480f55c0 2021-11-03T17:32:06.580 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.586+0000 7f693effd700 1 -- 172.21.15.27:0/959646061 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f6938054590 con 0x7f69480f55c0 2021-11-03T17:32:06.581 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:32:06.581 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:32:06.581 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:32:06.582 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:32:06.582 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:32:06.582 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:32:06.582 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:32:06.583 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:32:06.583 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:32:06.583 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:32:06.583 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:32:06.583 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:32:06.584 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:32:06.584 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:32:06.584 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:32:06.584 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:32:06.585 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:32:06.585 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:32:06.586 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.588+0000 7f695071e700 1 -- 172.21.15.27:0/959646061 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f69340640b0 msgr2=0x7f6934066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:32:06.586 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.588+0000 7f695071e700 1 --2- 172.21.15.27:0/959646061 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f69340640b0 0x7f6934066570 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f6944006fd0 tx=0x7f6944008040).stop 2021-11-03T17:32:06.586 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.588+0000 7f695071e700 1 -- 172.21.15.27:0/959646061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69480f55c0 msgr2=0x7f6948105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:32:06.586 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.588+0000 7f695071e700 1 --2- 172.21.15.27:0/959646061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69480f55c0 0x7f6948105ec0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f6938004f60 tx=0x7f6938009b30).stop 2021-11-03T17:32:06.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.588+0000 7f695071e700 1 -- 172.21.15.27:0/959646061 shutdown_connections 2021-11-03T17:32:06.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.588+0000 7f695071e700 1 --2- 172.21.15.27:0/959646061 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f69340640b0 0x7f6934066570 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:06.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.588+0000 7f695071e700 1 --2- 172.21.15.27:0/959646061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69480f55c0 0x7f6948105ec0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:06.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.588+0000 7f695071e700 1 --2- 172.21.15.27:0/959646061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69480f6860 0x7f69481063b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:06.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.588+0000 7f695071e700 1 -- 172.21.15.27:0/959646061 >> 172.21.15.27:0/959646061 conn(0x7f69480f0d40 msgr2=0x7f69480fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:32:06.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.588+0000 7f695071e700 1 -- 172.21.15.27:0/959646061 shutdown_connections 2021-11-03T17:32:06.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.588+0000 7f695071e700 1 -- 172.21.15.27:0/959646061 wait complete. 2021-11-03T17:32:06.685 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.689+0000 7f8ae75e8700 1 -- 172.21.15.27:0/3013988789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ae00f4a60 msgr2=0x7f8ae00f4e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:32:06.685 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.689+0000 7f8ae75e8700 1 --2- 172.21.15.27:0/3013988789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ae00f4a60 0x7f8ae00f4e80 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f8ad4004660 tx=0x7f8ad4009b30).stop 2021-11-03T17:32:06.685 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.689+0000 7f8ae75e8700 1 -- 172.21.15.27:0/3013988789 shutdown_connections 2021-11-03T17:32:06.686 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.689+0000 7f8ae75e8700 1 --2- 172.21.15.27:0/3013988789 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ae00f5400 0x7f8ae00fbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:06.686 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.689+0000 7f8ae75e8700 1 --2- 172.21.15.27:0/3013988789 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ae00f4a60 0x7f8ae00f4e80 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:06.686 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.689+0000 7f8ae75e8700 1 -- 172.21.15.27:0/3013988789 >> 172.21.15.27:0/3013988789 conn(0x7f8ae00f04c0 msgr2=0x7f8ae00f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:32:06.686 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.689+0000 7f8ae75e8700 1 -- 172.21.15.27:0/3013988789 shutdown_connections 2021-11-03T17:32:06.687 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.689+0000 7f8ae75e8700 1 -- 172.21.15.27:0/3013988789 wait complete. 2021-11-03T17:32:06.687 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.689+0000 7f8ae75e8700 1 Processor -- start 2021-11-03T17:32:06.687 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.689+0000 7f8ae75e8700 1 -- start start 2021-11-03T17:32:06.687 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.690+0000 7f8ae75e8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ae00f4a60 0x7f8ae0106f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:32:06.688 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.690+0000 7f8ae75e8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ae00f5400 0x7f8ae0107440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:32:06.688 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.690+0000 7f8ae75e8700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae01079f0 con 0x7f8ae00f5400 2021-11-03T17:32:06.688 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.690+0000 7f8ae75e8700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae0107b30 con 0x7f8ae00f4a60 2021-11-03T17:32:06.688 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.690+0000 7f8ae4b83700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ae00f5400 0x7f8ae0107440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:32:06.688 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.690+0000 7f8ae4b83700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ae00f5400 0x7f8ae0107440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58000/0 (socket says 172.21.15.27:58000) 2021-11-03T17:32:06.689 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.690+0000 7f8ae4b83700 1 -- 172.21.15.27:0/4208467606 learned_addr learned my addr 172.21.15.27:0/4208467606 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:32:06.689 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.690+0000 7f8ae5384700 1 --2- 172.21.15.27:0/4208467606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ae00f4a60 0x7f8ae0106f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:32:06.689 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.690+0000 7f8ae4b83700 1 -- 172.21.15.27:0/4208467606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ae00f4a60 msgr2=0x7f8ae0106f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:32:06.689 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.690+0000 7f8ae4b83700 1 --2- 172.21.15.27:0/4208467606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ae00f4a60 0x7f8ae0106f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:06.690 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.690+0000 7f8ae4b83700 1 -- 172.21.15.27:0/4208467606 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad4005040 con 0x7f8ae00f5400 2021-11-03T17:32:06.690 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.690+0000 7f8ae4b83700 1 --2- 172.21.15.27:0/4208467606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ae00f5400 0x7f8ae0107440 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f8adc00c530 tx=0x7f8adc007e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:32:06.690 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.690+0000 7f8ad1ffb700 1 -- 172.21.15.27:0/4208467606 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8adc00f650 con 0x7f8ae00f5400 2021-11-03T17:32:06.690 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.690+0000 7f8ad1ffb700 1 -- 172.21.15.27:0/4208467606 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f8adc00fc60 con 0x7f8ae00f5400 2021-11-03T17:32:06.691 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.690+0000 7f8ae75e8700 1 -- 172.21.15.27:0/4208467606 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ae01084d0 con 0x7f8ae00f5400 2021-11-03T17:32:06.691 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.690+0000 7f8ad1ffb700 1 -- 172.21.15.27:0/4208467606 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8adc012cb0 con 0x7f8ae00f5400 2021-11-03T17:32:06.691 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.691+0000 7f8ae75e8700 1 -- 172.21.15.27:0/4208467606 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae0108a90 con 0x7f8ae00f5400 2021-11-03T17:32:06.692 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.692+0000 7f8ae75e8700 1 -- 172.21.15.27:0/4208467606 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ae0045ad0 con 0x7f8ae00f5400 2021-11-03T17:32:06.693 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.692+0000 7f8ad1ffb700 1 -- 172.21.15.27:0/4208467606 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f8adc019030 con 0x7f8ae00f5400 2021-11-03T17:32:06.693 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.692+0000 7f8ad1ffb700 1 --2- 172.21.15.27:0/4208467606 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8acc0640b0 0x7f8acc066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:32:06.693 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.692+0000 7f8ad1ffb700 1 -- 172.21.15.27:0/4208467606 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f8adc080730 con 0x7f8ae00f5400 2021-11-03T17:32:06.693 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.693+0000 7f8ae5384700 1 --2- 172.21.15.27:0/4208467606 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8acc0640b0 0x7f8acc066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:32:06.694 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.693+0000 7f8ae5384700 1 --2- 172.21.15.27:0/4208467606 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8acc0640b0 0x7f8acc066570 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f8ad4004630 tx=0x7f8ad4009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:32:06.694 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.696+0000 7f8ad1ffb700 1 -- 172.21.15.27:0/4208467606 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f8adc0529f0 con 0x7f8ae00f5400 2021-11-03T17:32:06.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.901+0000 7f8ae75e8700 1 -- 172.21.15.27:0/4208467606 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8ae0108d10 con 0x7f8ae00f5400 2021-11-03T17:32:06.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.901+0000 7f8ad1ffb700 1 -- 172.21.15.27:0/4208467606 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f8adc017070 con 0x7f8ae00f5400 2021-11-03T17:32:06.896 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:32:06.896 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:32:06.897 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:32:06.897 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:32:06.897 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:32:06.898 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:32:06.898 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:32:06.898 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:32:06.898 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:32:06.899 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:32:06.899 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:32:06.899 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:32:06.899 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:32:06.900 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:32:06.900 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:32:06.900 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:32:06.900 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:32:06.900 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:32:06.901 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:32:06.901 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:32:06.901 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:32:06.901 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:32:06.902 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:32:06.902 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:32:06.902 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:32:06.902 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:32:06.902 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:32:06.903 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:32:06.903 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:32:06.903 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:32:06.903 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:32:06.904 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:32:06.904 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:32:06.904 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:32:06.904 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:32:06.904 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:32:06.905 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:32:06.905 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:32:06.905 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:32:06.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.903+0000 7f8ae75e8700 1 -- 172.21.15.27:0/4208467606 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8acc0640b0 msgr2=0x7f8acc066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:32:06.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.903+0000 7f8ae75e8700 1 --2- 172.21.15.27:0/4208467606 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8acc0640b0 0x7f8acc066570 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f8ad4004630 tx=0x7f8ad4009b30).stop 2021-11-03T17:32:06.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.903+0000 7f8ae75e8700 1 -- 172.21.15.27:0/4208467606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ae00f5400 msgr2=0x7f8ae0107440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:32:06.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.903+0000 7f8ae75e8700 1 --2- 172.21.15.27:0/4208467606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ae00f5400 0x7f8ae0107440 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f8adc00c530 tx=0x7f8adc007e10).stop 2021-11-03T17:32:06.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.903+0000 7f8ae75e8700 1 -- 172.21.15.27:0/4208467606 shutdown_connections 2021-11-03T17:32:06.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.903+0000 7f8ae75e8700 1 --2- 172.21.15.27:0/4208467606 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8acc0640b0 0x7f8acc066570 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:06.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.903+0000 7f8ae75e8700 1 --2- 172.21.15.27:0/4208467606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ae00f4a60 0x7f8ae0106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:06.908 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.903+0000 7f8ae75e8700 1 --2- 172.21.15.27:0/4208467606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ae00f5400 0x7f8ae0107440 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:06.908 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.903+0000 7f8ae75e8700 1 -- 172.21.15.27:0/4208467606 >> 172.21.15.27:0/4208467606 conn(0x7f8ae00f04c0 msgr2=0x7f8ae00f8510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:32:06.908 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.903+0000 7f8ae75e8700 1 -- 172.21.15.27:0/4208467606 shutdown_connections 2021-11-03T17:32:06.908 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:06.903+0000 7f8ae75e8700 1 -- 172.21.15.27:0/4208467606 wait complete. 2021-11-03T17:32:06.909 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:32:07.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:07 smithi115 ceph-mon[65453]: from='client.34648 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:32:07.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:07 smithi115 ceph-mon[65453]: pgmap v524: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:07.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:07 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/959646061' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:07.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:07 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/4208467606' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:32:07.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:07 smithi027 ceph-mon[97244]: from='client.34648 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:32:07.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:07 smithi027 ceph-mon[97244]: pgmap v524: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:07.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:07 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/959646061' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:07.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:07 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/4208467606' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:32:08.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:08 smithi115 ceph-mon[65453]: from='client.34652 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:32:08.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:08 smithi027 ceph-mon[97244]: from='client.34652 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:32:09.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:09 smithi115 ceph-mon[65453]: pgmap v525: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:09.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:09 smithi027 ceph-mon[97244]: pgmap v525: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:10.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 972.381980 secs 2021-11-03T17:32:10.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:10 smithi115 ceph-mon[65453]: pgmap v526: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:10.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 972.381980 secs 2021-11-03T17:32:10.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:10 smithi027 ceph-mon[97244]: pgmap v526: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:11.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:32:11.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:32:11.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:32:11.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:11.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:11.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:11.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:11.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:11 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:32:12.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:32:12.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:32:12.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:32:12.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:12.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:12.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:12.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:12.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:11 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:32:12.936 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:12 smithi027 ceph-mon[97244]: pgmap v527: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:12.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:12 smithi115 ceph-mon[65453]: pgmap v527: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:15.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:15 smithi115 ceph-mon[65453]: pgmap v528: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:15.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 977.382125 secs 2021-11-03T17:32:15.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:15 smithi027 ceph-mon[97244]: pgmap v528: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:15.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 977.382125 secs 2021-11-03T17:32:17.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:17 smithi115 ceph-mon[65453]: pgmap v529: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:17.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:17 smithi027 ceph-mon[97244]: pgmap v529: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:19.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:19 smithi115 ceph-mon[65453]: pgmap v530: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:19.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:19 smithi027 ceph-mon[97244]: pgmap v530: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:20.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 982.382248 secs 2021-11-03T17:32:20.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 982.382248 secs 2021-11-03T17:32:21.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:21 smithi115 ceph-mon[65453]: pgmap v531: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:21.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:32:21.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:32:21.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:32:21.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:21.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:21.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:21.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:21.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:21 smithi027 ceph-mon[97244]: pgmap v531: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:21.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:32:21.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:32:21.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:32:21.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:21.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:21.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:21.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:22.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:22 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:32:22.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:22 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:32:23.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:23 smithi027 ceph-mon[97244]: pgmap v532: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:23.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:23 smithi115 ceph-mon[65453]: pgmap v532: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:25.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:25 smithi115 ceph-mon[65453]: pgmap v533: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:25.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 987.382370 secs 2021-11-03T17:32:25.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:25 smithi027 ceph-mon[97244]: pgmap v533: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:25.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 987.382370 secs 2021-11-03T17:32:27.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:27 smithi115 ceph-mon[65453]: pgmap v534: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:27.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:27 smithi027 ceph-mon[97244]: pgmap v534: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:28.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:28 smithi115 ceph-mon[65453]: pgmap v535: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:28.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:28 smithi027 ceph-mon[97244]: pgmap v535: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:29.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:29 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 992.382437 secs 2021-11-03T17:32:30.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:29 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 992.382437 secs 2021-11-03T17:32:30.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:30 smithi115 ceph-mon[65453]: pgmap v536: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:31.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:30 smithi027 ceph-mon[97244]: pgmap v536: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:31.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:32:31.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:32:31.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:32:31.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:31.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:31.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:31.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:31.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:31 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:32:32.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:32:32.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:32:32.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:32:32.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:32.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:32.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:32.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:32.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:31 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:32:32.936 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:32 smithi027 ceph-mon[97244]: pgmap v537: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:32.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:32 smithi115 ceph-mon[65453]: pgmap v537: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:35.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:35 smithi115 ceph-mon[65453]: pgmap v538: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:35.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 997.382577 secs 2021-11-03T17:32:35.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:35 smithi027 ceph-mon[97244]: pgmap v538: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:35.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 997.382577 secs 2021-11-03T17:32:37.013 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.017+0000 7fdbade62700 1 -- 172.21.15.27:0/3532432730 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdba80f7980 msgr2=0x7fdba80f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:32:37.013 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.017+0000 7fdbade62700 1 --2- 172.21.15.27:0/3532432730 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdba80f7980 0x7fdba80f7de0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7fdb9c004660 tx=0x7fdb9c009b30).stop 2021-11-03T17:32:37.013 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.017+0000 7fdbade62700 1 -- 172.21.15.27:0/3532432730 shutdown_connections 2021-11-03T17:32:37.014 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.017+0000 7fdbade62700 1 --2- 172.21.15.27:0/3532432730 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdba80f7980 0x7fdba80f7de0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:37.014 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.017+0000 7fdbade62700 1 --2- 172.21.15.27:0/3532432730 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdba80f66e0 0x7fdba80f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:37.014 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.017+0000 7fdbade62700 1 -- 172.21.15.27:0/3532432730 >> 172.21.15.27:0/3532432730 conn(0x7fdba80f1e40 msgr2=0x7fdba80f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:32:37.014 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.017+0000 7fdbade62700 1 -- 172.21.15.27:0/3532432730 shutdown_connections 2021-11-03T17:32:37.015 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.017+0000 7fdbade62700 1 -- 172.21.15.27:0/3532432730 wait complete. 2021-11-03T17:32:37.015 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.018+0000 7fdbade62700 1 Processor -- start 2021-11-03T17:32:37.015 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.018+0000 7fdbade62700 1 -- start start 2021-11-03T17:32:37.017 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.018+0000 7fdbade62700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdba80f66e0 0x7fdba806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:32:37.018 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.018+0000 7fdbade62700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdba80f7980 0x7fdba8069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:32:37.018 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.018+0000 7fdbade62700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba806a380 con 0x7fdba80f66e0 2021-11-03T17:32:37.018 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.018+0000 7fdbade62700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba806a4c0 con 0x7fdba80f7980 2021-11-03T17:32:37.019 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.018+0000 7fdba77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdba80f66e0 0x7fdba806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:32:37.019 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.018+0000 7fdba77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdba80f66e0 0x7fdba806b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58016/0 (socket says 172.21.15.27:58016) 2021-11-03T17:32:37.019 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.018+0000 7fdba77fe700 1 -- 172.21.15.27:0/1322620859 learned_addr learned my addr 172.21.15.27:0/1322620859 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:32:37.019 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.019+0000 7fdba6ffd700 1 --2- 172.21.15.27:0/1322620859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdba80f7980 0x7fdba8069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:32:37.019 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.019+0000 7fdba77fe700 1 -- 172.21.15.27:0/1322620859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdba80f7980 msgr2=0x7fdba8069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:32:37.020 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.019+0000 7fdba77fe700 1 --2- 172.21.15.27:0/1322620859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdba80f7980 0x7fdba8069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:37.020 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.019+0000 7fdba77fe700 1 -- 172.21.15.27:0/1322620859 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb9c005040 con 0x7fdba80f66e0 2021-11-03T17:32:37.020 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.019+0000 7fdba6ffd700 1 --2- 172.21.15.27:0/1322620859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdba80f7980 0x7fdba8069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:32:37.020 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.019+0000 7fdba77fe700 1 --2- 172.21.15.27:0/1322620859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdba80f66e0 0x7fdba806b9a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7fdb9800c530 tx=0x7fdb98007e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:32:37.021 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.019+0000 7fdb8ffff700 1 -- 172.21.15.27:0/1322620859 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb9800f650 con 0x7fdba80f66e0 2021-11-03T17:32:37.021 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.019+0000 7fdbade62700 1 -- 172.21.15.27:0/1322620859 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdba806a7a0 con 0x7fdba80f66e0 2021-11-03T17:32:37.021 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.019+0000 7fdb8ffff700 1 -- 172.21.15.27:0/1322620859 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fdb9800fc60 con 0x7fdba80f66e0 2021-11-03T17:32:37.021 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.019+0000 7fdb8ffff700 1 -- 172.21.15.27:0/1322620859 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb98012d20 con 0x7fdba80f66e0 2021-11-03T17:32:37.022 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.019+0000 7fdbade62700 1 -- 172.21.15.27:0/1322620859 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba810ef90 con 0x7fdba80f66e0 2021-11-03T17:32:37.022 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.020+0000 7fdbade62700 1 -- 172.21.15.27:0/1322620859 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdba8045ad0 con 0x7fdba80f66e0 2021-11-03T17:32:37.022 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.021+0000 7fdb8ffff700 1 -- 172.21.15.27:0/1322620859 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fdb98019030 con 0x7fdba80f66e0 2021-11-03T17:32:37.022 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.021+0000 7fdb8ffff700 1 --2- 172.21.15.27:0/1322620859 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdb900640b0 0x7fdb90066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:32:37.022 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.021+0000 7fdb8ffff700 1 -- 172.21.15.27:0/1322620859 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fdb98080680 con 0x7fdba80f66e0 2021-11-03T17:32:37.023 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.021+0000 7fdba6ffd700 1 --2- 172.21.15.27:0/1322620859 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdb900640b0 0x7fdb90066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:32:37.023 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.022+0000 7fdba6ffd700 1 --2- 172.21.15.27:0/1322620859 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdb900640b0 0x7fdb90066570 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fdb9c009da0 tx=0x7fdb9c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:32:37.023 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.025+0000 7fdb8ffff700 1 -- 172.21.15.27:0/1322620859 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fdb980528e0 con 0x7fdba80f66e0 2021-11-03T17:32:37.202 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.207+0000 7fdbade62700 1 -- 172.21.15.27:0/1322620859 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdba80ff020 con 0x7fdb900640b0 2021-11-03T17:32:37.203 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.208+0000 7fdb8ffff700 1 -- 172.21.15.27:0/1322620859 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fdba80ff020 con 0x7fdb900640b0 2021-11-03T17:32:37.205 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.210+0000 7fdbade62700 1 -- 172.21.15.27:0/1322620859 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdb900640b0 msgr2=0x7fdb90066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:32:37.206 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.210+0000 7fdbade62700 1 --2- 172.21.15.27:0/1322620859 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdb900640b0 0x7fdb90066570 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7fdb9c009da0 tx=0x7fdb9c009b30).stop 2021-11-03T17:32:37.206 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.210+0000 7fdbade62700 1 -- 172.21.15.27:0/1322620859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdba80f66e0 msgr2=0x7fdba806b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:32:37.206 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.210+0000 7fdbade62700 1 --2- 172.21.15.27:0/1322620859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdba80f66e0 0x7fdba806b9a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7fdb9800c530 tx=0x7fdb98007e10).stop 2021-11-03T17:32:37.206 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.210+0000 7fdbade62700 1 -- 172.21.15.27:0/1322620859 shutdown_connections 2021-11-03T17:32:37.207 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.210+0000 7fdbade62700 1 --2- 172.21.15.27:0/1322620859 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdb900640b0 0x7fdb90066570 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:37.207 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.210+0000 7fdbade62700 1 --2- 172.21.15.27:0/1322620859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdba80f66e0 0x7fdba806b9a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:37.207 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.210+0000 7fdbade62700 1 --2- 172.21.15.27:0/1322620859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdba80f7980 0x7fdba8069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:37.207 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.210+0000 7fdbade62700 1 -- 172.21.15.27:0/1322620859 >> 172.21.15.27:0/1322620859 conn(0x7fdba80f1e40 msgr2=0x7fdba80fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:32:37.207 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.210+0000 7fdbade62700 1 -- 172.21.15.27:0/1322620859 shutdown_connections 2021-11-03T17:32:37.208 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.210+0000 7fdbade62700 1 -- 172.21.15.27:0/1322620859 wait complete. 2021-11-03T17:32:37.215 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:32:37.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.310+0000 7fadad23d700 1 -- 172.21.15.27:0/1032399069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fada80f5ef0 msgr2=0x7fada80f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:32:37.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.310+0000 7fadad23d700 1 --2- 172.21.15.27:0/1032399069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fada80f5ef0 0x7fada80f6920 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7fad90004660 tx=0x7fad90009b30).stop 2021-11-03T17:32:37.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.310+0000 7fadad23d700 1 -- 172.21.15.27:0/1032399069 shutdown_connections 2021-11-03T17:32:37.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.310+0000 7fadad23d700 1 --2- 172.21.15.27:0/1032399069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fada80f5ef0 0x7fada80f6920 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:37.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.310+0000 7fadad23d700 1 --2- 172.21.15.27:0/1032399069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fada80f55e0 0x7fada80f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:37.307 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.310+0000 7fadad23d700 1 -- 172.21.15.27:0/1032399069 >> 172.21.15.27:0/1032399069 conn(0x7fada80f0de0 msgr2=0x7fada80f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:32:37.307 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.310+0000 7fadad23d700 1 -- 172.21.15.27:0/1032399069 shutdown_connections 2021-11-03T17:32:37.307 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.310+0000 7fadad23d700 1 -- 172.21.15.27:0/1032399069 wait complete. 2021-11-03T17:32:37.309 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.311+0000 7fadad23d700 1 Processor -- start 2021-11-03T17:32:37.310 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.311+0000 7fadad23d700 1 -- start start 2021-11-03T17:32:37.310 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.311+0000 7fadad23d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fada80f55e0 0x7fada8109c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:32:37.310 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.311+0000 7fadad23d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fada80f5ef0 0x7fada810c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:32:37.311 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.311+0000 7fadad23d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada80604e0 con 0x7fada80f55e0 2021-11-03T17:32:37.311 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.311+0000 7fadad23d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada8060620 con 0x7fada80f5ef0 2021-11-03T17:32:37.311 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.311+0000 7fada6d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fada80f55e0 0x7fada8109c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:32:37.311 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.311+0000 7fada6d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fada80f55e0 0x7fada8109c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58026/0 (socket says 172.21.15.27:58026) 2021-11-03T17:32:37.311 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.311+0000 7fada6d9d700 1 -- 172.21.15.27:0/1833819615 learned_addr learned my addr 172.21.15.27:0/1833819615 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:32:37.312 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.312+0000 7fada659c700 1 --2- 172.21.15.27:0/1833819615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fada80f5ef0 0x7fada810c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:32:37.312 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.312+0000 7fada6d9d700 1 -- 172.21.15.27:0/1833819615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fada80f5ef0 msgr2=0x7fada810c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:32:37.312 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.312+0000 7fada6d9d700 1 --2- 172.21.15.27:0/1833819615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fada80f5ef0 0x7fada810c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:37.312 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.312+0000 7fada6d9d700 1 -- 172.21.15.27:0/1833819615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad90005040 con 0x7fada80f55e0 2021-11-03T17:32:37.313 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.312+0000 7fada659c700 1 --2- 172.21.15.27:0/1833819615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fada80f5ef0 0x7fada810c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:32:37.313 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.312+0000 7fada6d9d700 1 --2- 172.21.15.27:0/1833819615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fada80f55e0 0x7fada8109c10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7fad9800e7f0 tx=0x7fad9800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:32:37.313 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.312+0000 7fad9f7fe700 1 -- 172.21.15.27:0/1833819615 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad980092e0 con 0x7fada80f55e0 2021-11-03T17:32:37.313 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.312+0000 7fadad23d700 1 -- 172.21.15.27:0/1833819615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fada810c6c0 con 0x7fada80f55e0 2021-11-03T17:32:37.313 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.312+0000 7fad9f7fe700 1 -- 172.21.15.27:0/1833819615 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fad9800f040 con 0x7fada80f55e0 2021-11-03T17:32:37.314 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.312+0000 7fad9f7fe700 1 -- 172.21.15.27:0/1833819615 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fad98014750 con 0x7fada80f55e0 2021-11-03T17:32:37.314 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.312+0000 7fadad23d700 1 -- 172.21.15.27:0/1833819615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fada810cc50 con 0x7fada80f55e0 2021-11-03T17:32:37.314 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.313+0000 7fadad23d700 1 -- 172.21.15.27:0/1833819615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fada8045ad0 con 0x7fada80f55e0 2021-11-03T17:32:37.314 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.314+0000 7fad9f7fe700 1 -- 172.21.15.27:0/1833819615 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fad9801b030 con 0x7fada80f55e0 2021-11-03T17:32:37.315 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.314+0000 7fad9f7fe700 1 --2- 172.21.15.27:0/1833819615 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fad940640b0 0x7fad94066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:32:37.315 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.314+0000 7fad9f7fe700 1 -- 172.21.15.27:0/1833819615 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fad980807f0 con 0x7fada80f55e0 2021-11-03T17:32:37.315 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.314+0000 7fada659c700 1 --2- 172.21.15.27:0/1833819615 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fad940640b0 0x7fad94066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:32:37.315 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.314+0000 7fada659c700 1 --2- 172.21.15.27:0/1833819615 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fad940640b0 0x7fad94066570 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7fad900043d0 tx=0x7fad9000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:32:37.316 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.317+0000 7fad9f7fe700 1 -- 172.21.15.27:0/1833819615 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fad98052ab0 con 0x7fada80f55e0 2021-11-03T17:32:37.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:37 smithi115 ceph-mon[65453]: pgmap v539: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:37.484 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.489+0000 7fadad23d700 1 -- 172.21.15.27:0/1833819615 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fada80fdf40 con 0x7fad940640b0 2021-11-03T17:32:37.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.504+0000 7fad9f7fe700 1 -- 172.21.15.27:0/1833819615 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fada80fdf40 con 0x7fad940640b0 2021-11-03T17:32:37.499 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:32:37.499 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (24m) 4m ago 25m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:32:37.500 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (15m) 4m ago 25m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:32:37.500 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (15m) 2m ago 24m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:32:37.500 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (24m) 4m ago 25m 40.2M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:32:37.500 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (17m) 4m ago 20m 18.2M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:32:37.501 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (20m) 4m ago 20m 3841M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:32:37.501 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (16m) 2m ago 20m 17.6M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:32:37.501 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (18m) 2m ago 20m 19.3M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:32:37.501 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (17m) 4m ago 26m 497M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:32:37.501 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (17m) 2m ago 24m 415M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:32:37.502 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (16m) 4m ago 26m 119M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:32:37.502 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (16m) 2m ago 24m 71.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:32:37.502 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (25m) 4m ago 25m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:32:37.502 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (24m) 2m ago 24m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:32:37.503 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (15m) 4m ago 23m 290M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:32:37.503 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (15m) 4m ago 23m 317M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:32:37.503 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (14m) 4m ago 23m 229M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:32:37.503 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (14m) 2m ago 22m 329M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:32:37.504 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (13m) 2m ago 22m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:32:37.504 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (13m) 2m ago 22m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:32:37.504 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (23m) 4m ago 24m 48.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:32:37.505 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.506+0000 7fadad23d700 1 -- 172.21.15.27:0/1833819615 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fad940640b0 msgr2=0x7fad94066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:32:37.505 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.506+0000 7fadad23d700 1 --2- 172.21.15.27:0/1833819615 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fad940640b0 0x7fad94066570 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7fad900043d0 tx=0x7fad9000b040).stop 2021-11-03T17:32:37.505 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.506+0000 7fadad23d700 1 -- 172.21.15.27:0/1833819615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fada80f55e0 msgr2=0x7fada8109c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:32:37.506 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.506+0000 7fadad23d700 1 --2- 172.21.15.27:0/1833819615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fada80f55e0 0x7fada8109c10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 rx=0x7fad9800e7f0 tx=0x7fad9800b040).stop 2021-11-03T17:32:37.506 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.506+0000 7fadad23d700 1 -- 172.21.15.27:0/1833819615 shutdown_connections 2021-11-03T17:32:37.506 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.506+0000 7fadad23d700 1 --2- 172.21.15.27:0/1833819615 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fad940640b0 0x7fad94066570 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:37.506 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.506+0000 7fadad23d700 1 --2- 172.21.15.27:0/1833819615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fada80f55e0 0x7fada8109c10 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:37.507 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.506+0000 7fadad23d700 1 --2- 172.21.15.27:0/1833819615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fada80f5ef0 0x7fada810c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:37.507 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.506+0000 7fadad23d700 1 -- 172.21.15.27:0/1833819615 >> 172.21.15.27:0/1833819615 conn(0x7fada80f0de0 msgr2=0x7fada80fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:32:37.507 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.506+0000 7fadad23d700 1 -- 172.21.15.27:0/1833819615 shutdown_connections 2021-11-03T17:32:37.507 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.506+0000 7fadad23d700 1 -- 172.21.15.27:0/1833819615 wait complete. 2021-11-03T17:32:37.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:37 smithi027 ceph-mon[97244]: pgmap v539: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:37.602 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.605+0000 7f6a913ca700 1 -- 172.21.15.27:0/1849088779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a8c0f5ed0 msgr2=0x7f6a8c0f6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:32:37.603 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.605+0000 7f6a913ca700 1 --2- 172.21.15.27:0/1849088779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a8c0f5ed0 0x7f6a8c0f6900 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f6a74004660 tx=0x7f6a74009b30).stop 2021-11-03T17:32:37.603 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.605+0000 7f6a913ca700 1 -- 172.21.15.27:0/1849088779 shutdown_connections 2021-11-03T17:32:37.603 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.605+0000 7f6a913ca700 1 --2- 172.21.15.27:0/1849088779 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a8c0f5ed0 0x7f6a8c0f6900 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:37.603 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.605+0000 7f6a913ca700 1 --2- 172.21.15.27:0/1849088779 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a8c0f55c0 0x7f6a8c0f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:37.604 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.605+0000 7f6a913ca700 1 -- 172.21.15.27:0/1849088779 >> 172.21.15.27:0/1849088779 conn(0x7f6a8c0f0dc0 msgr2=0x7f6a8c0f31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:32:37.604 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.605+0000 7f6a913ca700 1 -- 172.21.15.27:0/1849088779 shutdown_connections 2021-11-03T17:32:37.604 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.605+0000 7f6a913ca700 1 -- 172.21.15.27:0/1849088779 wait complete. 2021-11-03T17:32:37.605 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.606+0000 7f6a913ca700 1 Processor -- start 2021-11-03T17:32:37.605 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.606+0000 7f6a913ca700 1 -- start start 2021-11-03T17:32:37.605 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.606+0000 7f6a913ca700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a8c0f55c0 0x7f6a8c109c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:32:37.605 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.606+0000 7f6a913ca700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a8c0f5ed0 0x7f6a8c10c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:32:37.605 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.606+0000 7f6a913ca700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c060560 con 0x7f6a8c0f55c0 2021-11-03T17:32:37.606 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.606+0000 7f6a913ca700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8c0606a0 con 0x7f6a8c0f5ed0 2021-11-03T17:32:37.606 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.606+0000 7f6a8affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a8c0f55c0 0x7f6a8c109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:32:37.606 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.606+0000 7f6a8affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a8c0f55c0 0x7f6a8c109c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58038/0 (socket says 172.21.15.27:58038) 2021-11-03T17:32:37.606 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.606+0000 7f6a8affd700 1 -- 172.21.15.27:0/2865340907 learned_addr learned my addr 172.21.15.27:0/2865340907 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:32:37.607 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.607+0000 7f6a8affd700 1 -- 172.21.15.27:0/2865340907 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a8c0f5ed0 msgr2=0x7f6a8c10c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:32:37.607 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.607+0000 7f6a8affd700 1 --2- 172.21.15.27:0/2865340907 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a8c0f5ed0 0x7f6a8c10c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:32:37.607 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.607+0000 7f6a8affd700 1 -- 172.21.15.27:0/2865340907 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a74005040 con 0x7f6a8c0f55c0 2021-11-03T17:32:37.607 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.607+0000 7f6a8affd700 1 --2- 172.21.15.27:0/2865340907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a8c0f55c0 0x7f6a8c109c70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f6a8400c530 tx=0x7f6a84007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:32:37.608 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.607+0000 7f6a837fe700 1 -- 172.21.15.27:0/2865340907 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a8400f650 con 0x7f6a8c0f55c0 2021-11-03T17:32:37.608 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.607+0000 7f6a913ca700 1 -- 172.21.15.27:0/2865340907 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a8c10c720 con 0x7f6a8c0f55c0 2021-11-03T17:32:37.608 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.607+0000 7f6a837fe700 1 -- 172.21.15.27:0/2865340907 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f6a8400fc60 con 0x7f6a8c0f55c0 2021-11-03T17:32:37.608 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.607+0000 7f6a837fe700 1 -- 172.21.15.27:0/2865340907 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a84012cd0 con 0x7f6a8c0f55c0 2021-11-03T17:32:37.609 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.607+0000 7f6a913ca700 1 -- 172.21.15.27:0/2865340907 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a8c10cce0 con 0x7f6a8c0f55c0 2021-11-03T17:32:37.610 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.608+0000 7f6a913ca700 1 -- 172.21.15.27:0/2865340907 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a8c045ad0 con 0x7f6a8c0f55c0 2021-11-03T17:32:37.610 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.608+0000 7f6a837fe700 1 -- 172.21.15.27:0/2865340907 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f6a84010070 con 0x7f6a8c0f55c0 2021-11-03T17:32:37.610 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.609+0000 7f6a837fe700 1 --2- 172.21.15.27:0/2865340907 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6a780640b0 0x7f6a78066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:32:37.610 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.609+0000 7f6a837fe700 1 -- 172.21.15.27:0/2865340907 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f6a84080740 con 0x7f6a8c0f55c0 2021-11-03T17:32:37.611 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.609+0000 7f6a8a7fc700 1 --2- 172.21.15.27:0/2865340907 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6a780640b0 0x7f6a78066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:32:37.611 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.609+0000 7f6a8a7fc700 1 --2- 172.21.15.27:0/2865340907 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6a780640b0 0x7f6a78066570 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f6a74004e30 tx=0x7f6a74009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:32:37.611 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.613+0000 7f6a837fe700 1 -- 172.21.15.27:0/2865340907 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f6a84052950 con 0x7f6a8c0f55c0 2021-11-03T17:32:37.836 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.840+0000 7f6a913ca700 1 -- 172.21.15.27:0/2865340907 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6a8c0586e0 con 0x7f6a8c0f55c0 2021-11-03T17:32:37.836 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.841+0000 7f6a837fe700 1 -- 172.21.15.27:0/2865340907 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f6a84052540 con 0x7f6a8c0f55c0 2021-11-03T17:32:37.837 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:32:37.837 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:32:37.837 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:32:37.837 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:32:37.838 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:32:37.838 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:32:37.838 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:32:37.838 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:32:37.839 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:32:37.839 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:32:37.839 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:32:37.839 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:32:37.840 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:32:37.840 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:32:37.840 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:32:37.840 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:32:37.840 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:32:37.841 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:32:37.841 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.843+0000 7f6a913ca700 1 -- 172.21.15.27:0/2865340907 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6a780640b0 msgr2=0x7f6a78066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:32:37.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.843+0000 7f6a913ca700 1 --2- 172.21.15.27:0/2865340907 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6a780640b0 0x7f6a78066570 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f6a74004e30 tx=0x7f6a74009b30).stop 2021-11-03T17:32:37.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.843+0000 7f6a913ca700 1 -- 172.21.15.27:0/2865340907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a8c0f55c0 msgr2=0x7f6a8c109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:32:37.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.843+0000 7f6a913ca700 1 --2- 172.21.15.27:0/2865340907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a8c0f55c0 0x7f6a8c109c70 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f6a8400c530 tx=0x7f6a84007e20).stop 2021-11-03T17:32:37.843 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.843+0000 7f6a913ca700 1 -- 172.21.15.27:0/2865340907 shutdown_connections 2021-11-03T17:32:37.843 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.843+0000 7f6a913ca700 1 --2- 172.21.15.27:0/2865340907 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6a780640b0 0x7f6a78066570 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:37.843 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.843+0000 7f6a913ca700 1 --2- 172.21.15.27:0/2865340907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6a8c0f55c0 0x7f6a8c109c70 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:37.843 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.843+0000 7f6a913ca700 1 --2- 172.21.15.27:0/2865340907 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6a8c0f5ed0 0x7f6a8c10c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:32:37.844 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.843+0000 7f6a913ca700 1 -- 172.21.15.27:0/2865340907 >> 172.21.15.27:0/2865340907 conn(0x7f6a8c0f0dc0 msgr2=0x7f6a8c0fa350 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:32:37.844 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.843+0000 7f6a913ca700 1 -- 172.21.15.27:0/2865340907 shutdown_connections 2021-11-03T17:32:37.844 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.843+0000 7f6a913ca700 1 -- 172.21.15.27:0/2865340907 wait complete. 2021-11-03T17:32:37.941 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.944+0000 7f1821029700 1 -- 172.21.15.27:0/1636782898 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f181c0f74a0 msgr2=0x7f181c0f5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:32:37.942 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.944+0000 7f1821029700 1 --2- 172.21.15.27:0/1636782898 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f181c0f74a0 0x7f181c0f5930 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f1804004660 tx=0x7f1804009b30).stop 2021-11-03T17:32:37.942 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.944+0000 7f1821029700 1 -- 172.21.15.27:0/1636782898 shutdown_connections 2021-11-03T17:32:37.942 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.944+0000 7f1821029700 1 --2- 172.21.15.27:0/1636782898 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f181c0f74a0 0x7f181c0f5930 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:37.942 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.944+0000 7f1821029700 1 --2- 172.21.15.27:0/1636782898 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f181c0f6b90 0x7f181c0f6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:37.943 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.944+0000 7f1821029700 1 -- 172.21.15.27:0/1636782898 >> 172.21.15.27:0/1636782898 conn(0x7f181c0f0540 msgr2=0x7f181c0f2960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:32:37.943 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.944+0000 7f1821029700 1 -- 172.21.15.27:0/1636782898 shutdown_connections 2021-11-03T17:32:37.943 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.944+0000 7f1821029700 1 -- 172.21.15.27:0/1636782898 wait complete. 2021-11-03T17:32:37.943 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.945+0000 7f1821029700 1 Processor -- start 2021-11-03T17:32:37.944 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.945+0000 7f1821029700 1 -- start start 2021-11-03T17:32:37.944 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.945+0000 7f1821029700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f181c0f6b90 0x7f181c106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:32:37.944 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.945+0000 7f1821029700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f181c0f74a0 0x7f181c107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:32:37.944 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.945+0000 7f1821029700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181c107a10 con 0x7f181c0f6b90 2021-11-03T17:32:37.944 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.945+0000 7f1821029700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181c107b50 con 0x7f181c0f74a0 2021-11-03T17:32:37.945 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.945+0000 7f181ad9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f181c0f6b90 0x7f181c106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:32:37.945 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.945+0000 7f181ad9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f181c0f6b90 0x7f181c106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58048/0 (socket says 172.21.15.27:58048) 2021-11-03T17:32:37.945 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.945+0000 7f181ad9d700 1 -- 172.21.15.27:0/1540546691 learned_addr learned my addr 172.21.15.27:0/1540546691 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:32:37.945 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.945+0000 7f181ad9d700 1 -- 172.21.15.27:0/1540546691 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f181c0f74a0 msgr2=0x7f181c107460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:32:37.946 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.945+0000 7f181ad9d700 1 --2- 172.21.15.27:0/1540546691 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f181c0f74a0 0x7f181c107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:32:37.946 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.945+0000 7f181ad9d700 1 -- 172.21.15.27:0/1540546691 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1804005040 con 0x7f181c0f6b90 2021-11-03T17:32:37.946 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.945+0000 7f181ad9d700 1 --2- 172.21.15.27:0/1540546691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f181c0f6b90 0x7f181c106f70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f180c00c530 tx=0x7f180c007e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:32:37.946 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.946+0000 7f18137fe700 1 -- 172.21.15.27:0/1540546691 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f180c00f650 con 0x7f181c0f6b90 2021-11-03T17:32:37.947 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.946+0000 7f1821029700 1 -- 172.21.15.27:0/1540546691 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f181c1084f0 con 0x7f181c0f6b90 2021-11-03T17:32:37.947 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.946+0000 7f18137fe700 1 -- 172.21.15.27:0/1540546691 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f180c00fc60 con 0x7f181c0f6b90 2021-11-03T17:32:37.947 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.946+0000 7f18137fe700 1 -- 172.21.15.27:0/1540546691 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f180c012d20 con 0x7f181c0f6b90 2021-11-03T17:32:37.947 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.946+0000 7f1821029700 1 -- 172.21.15.27:0/1540546691 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f181c108ab0 con 0x7f181c0f6b90 2021-11-03T17:32:37.949 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.947+0000 7f1821029700 1 -- 172.21.15.27:0/1540546691 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f181c045ad0 con 0x7f181c0f6b90 2021-11-03T17:32:37.949 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.947+0000 7f18137fe700 1 -- 172.21.15.27:0/1540546691 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f180c019030 con 0x7f181c0f6b90 2021-11-03T17:32:37.949 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.947+0000 7f18137fe700 1 --2- 172.21.15.27:0/1540546691 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f18080640b0 0x7f1808066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:32:37.949 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.948+0000 7f18137fe700 1 -- 172.21.15.27:0/1540546691 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f180c080680 con 0x7f181c0f6b90 2021-11-03T17:32:37.950 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.948+0000 7f181a59c700 1 --2- 172.21.15.27:0/1540546691 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f18080640b0 0x7f1808066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:32:37.950 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.948+0000 7f181a59c700 1 --2- 172.21.15.27:0/1540546691 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f18080640b0 0x7f1808066570 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f1804004e30 tx=0x7f1804009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:32:37.950 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:37.952+0000 7f18137fe700 1 -- 172.21.15.27:0/1540546691 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f180c0528e0 con 0x7f181c0f6b90 2021-11-03T17:32:38.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:38.160+0000 7f1821029700 1 -- 172.21.15.27:0/1540546691 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f181c10d590 con 0x7f181c0f6b90 2021-11-03T17:32:38.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:38.160+0000 7f18137fe700 1 -- 172.21.15.27:0/1540546691 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f180c017710 con 0x7f181c0f6b90 2021-11-03T17:32:38.156 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:32:38.156 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:32:38.156 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:32:38.157 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:32:38.157 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:32:38.157 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:32:38.157 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:32:38.158 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:32:38.158 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:32:38.158 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:32:38.158 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:32:38.158 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:32:38.159 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:32:38.159 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:32:38.159 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:32:38.159 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:32:38.160 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:32:38.160 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:32:38.160 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:32:38.160 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:32:38.160 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:32:38.161 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:32:38.161 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:32:38.161 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:32:38.162 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:32:38.162 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:32:38.162 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:32:38.162 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:32:38.163 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:32:38.163 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:32:38.163 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:32:38.163 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:32:38.164 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:32:38.164 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:32:38.165 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:32:38.165 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:32:38.165 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:32:38.165 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:32:38.165 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:32:38.167 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:38.162+0000 7f1821029700 1 -- 172.21.15.27:0/1540546691 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f18080640b0 msgr2=0x7f1808066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:32:38.167 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:38.162+0000 7f1821029700 1 --2- 172.21.15.27:0/1540546691 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f18080640b0 0x7f1808066570 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7f1804004e30 tx=0x7f1804009b30).stop 2021-11-03T17:32:38.167 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:38.162+0000 7f1821029700 1 -- 172.21.15.27:0/1540546691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f181c0f6b90 msgr2=0x7f181c106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:32:38.167 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:38.162+0000 7f1821029700 1 --2- 172.21.15.27:0/1540546691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f181c0f6b90 0x7f181c106f70 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f180c00c530 tx=0x7f180c007e10).stop 2021-11-03T17:32:38.168 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:38.162+0000 7f1821029700 1 -- 172.21.15.27:0/1540546691 shutdown_connections 2021-11-03T17:32:38.168 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:38.162+0000 7f1821029700 1 --2- 172.21.15.27:0/1540546691 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f18080640b0 0x7f1808066570 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:38.169 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:38.162+0000 7f1821029700 1 --2- 172.21.15.27:0/1540546691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f181c0f6b90 0x7f181c106f70 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:32:38.169 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:38.162+0000 7f1821029700 1 --2- 172.21.15.27:0/1540546691 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f181c0f74a0 0x7f181c107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:32:38.169 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:38.162+0000 7f1821029700 1 -- 172.21.15.27:0/1540546691 >> 172.21.15.27:0/1540546691 conn(0x7f181c0f0540 msgr2=0x7f181c0f4820 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:32:38.169 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:38.162+0000 7f1821029700 1 -- 172.21.15.27:0/1540546691 shutdown_connections 2021-11-03T17:32:38.170 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:32:38.162+0000 7f1821029700 1 -- 172.21.15.27:0/1540546691 wait complete. 2021-11-03T17:32:38.170 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:32:38.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:38 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2865340907' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:38.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:38 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2865340907' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:39.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:39 smithi115 ceph-mon[65453]: from='client.34664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:32:39.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:39 smithi115 ceph-mon[65453]: from='client.34668 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:32:39.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:39 smithi115 ceph-mon[65453]: pgmap v540: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:39.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:39 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1540546691' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:32:39.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:39 smithi027 ceph-mon[97244]: from='client.34664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:32:39.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:39 smithi027 ceph-mon[97244]: from='client.34668 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:32:39.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:39 smithi027 ceph-mon[97244]: pgmap v540: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:39.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:39 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1540546691' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:32:40.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1002.382639 secs 2021-11-03T17:32:40.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1002.382639 secs 2021-11-03T17:32:41.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:41 smithi115 ceph-mon[65453]: pgmap v541: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:41.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:32:41.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:32:41.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:32:41.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:41.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:41.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:41.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:41.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:41 smithi027 ceph-mon[97244]: pgmap v541: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:41.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:32:41.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:32:41.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:32:41.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:41.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:41.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:41.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:42.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:42 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:32:42.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:42 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:32:43.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:43 smithi027 ceph-mon[97244]: pgmap v542: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:43.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:43 smithi115 ceph-mon[65453]: pgmap v542: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:44.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:44 smithi115 ceph-mon[65453]: pgmap v543: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:44.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:44 smithi027 ceph-mon[97244]: pgmap v543: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:45.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1007.382786 secs 2021-11-03T17:32:45.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1007.382786 secs 2021-11-03T17:32:46.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:46 smithi115 ceph-mon[65453]: pgmap v544: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:46.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:46 smithi027 ceph-mon[97244]: pgmap v544: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:49.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:49 smithi115 ceph-mon[65453]: pgmap v545: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:49.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:49 smithi027 ceph-mon[97244]: pgmap v545: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:50.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1012.382907 secs 2021-11-03T17:32:50.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1012.382907 secs 2021-11-03T17:32:51.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:51 smithi115 ceph-mon[65453]: pgmap v546: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:51.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:32:51.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:32:51.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:32:51.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:51.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:51.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:51.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:51.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:51 smithi027 ceph-mon[97244]: pgmap v546: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:51.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:32:51.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:32:51.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:32:51.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:51.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:51.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:51.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:32:52.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:52 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:32:52.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:52 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:32:53.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:53 smithi027 ceph-mon[97244]: pgmap v547: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:53.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:53 smithi115 ceph-mon[65453]: pgmap v547: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:55.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:55 smithi115 ceph-mon[65453]: pgmap v548: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:55.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1017.383039 secs 2021-11-03T17:32:55.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:55 smithi027 ceph-mon[97244]: pgmap v548: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:55.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1017.383039 secs 2021-11-03T17:32:56.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:32:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:32:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:32:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:32:57.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:57 smithi115 ceph-mon[65453]: pgmap v549: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:57.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:57 smithi027 ceph-mon[97244]: pgmap v549: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:58.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:58 smithi115 ceph-mon[65453]: pgmap v550: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:58.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:58 smithi027 ceph-mon[97244]: pgmap v550: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:32:59.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:32:59 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1022.383175 secs 2021-11-03T17:33:00.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:32:59 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1022.383175 secs 2021-11-03T17:33:00.978 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:00 smithi115 ceph-mon[65453]: pgmap v551: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:00.978 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:00 smithi027 ceph-mon[97244]: pgmap v551: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:01.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:33:01.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:01 smithi115 ceph-mon[65453]: poll took 60179.275 ms: 1 events 2021-11-03T17:33:01.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:33:01.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:33:01.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:01.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:01.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:01.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:02.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:33:02.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:01 smithi027 ceph-mon[97244]: poll took 60179.275 ms: 1 events 2021-11-03T17:33:02.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:33:02.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:33:02.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:02.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:02.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:02.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:02.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:02 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:33:02.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:02 smithi027 ceph-mon[97244]: pgmap v552: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:02.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:02 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:33:02.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:02 smithi115 ceph-mon[65453]: pgmap v552: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:05.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:05 smithi115 ceph-mon[65453]: pgmap v553: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:05.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1027.383242 secs 2021-11-03T17:33:05.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:05 smithi027 ceph-mon[97244]: pgmap v553: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:05.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1027.383242 secs 2021-11-03T17:33:07.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:07 smithi115 ceph-mon[65453]: pgmap v554: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:07.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:07 smithi027 ceph-mon[97244]: pgmap v554: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:08.274 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.277+0000 7f6cf18c8700 1 -- 172.21.15.27:0/2534278233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cec0f49a0 msgr2=0x7f6cec0f4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:33:08.274 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.277+0000 7f6cf18c8700 1 --2- 172.21.15.27:0/2534278233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cec0f49a0 0x7f6cec0f4dc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f6ce40099e0 tx=0x7f6ce4004d10).stop 2021-11-03T17:33:08.275 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.277+0000 7f6cf18c8700 1 -- 172.21.15.27:0/2534278233 shutdown_connections 2021-11-03T17:33:08.275 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.277+0000 7f6cf18c8700 1 --2- 172.21.15.27:0/2534278233 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cec0f5340 0x7f6cec0f7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:08.275 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.277+0000 7f6cf18c8700 1 --2- 172.21.15.27:0/2534278233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cec0f49a0 0x7f6cec0f4dc0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:08.276 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.277+0000 7f6cf18c8700 1 -- 172.21.15.27:0/2534278233 >> 172.21.15.27:0/2534278233 conn(0x7f6cec0f04c0 msgr2=0x7f6cec0f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:33:08.276 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.277+0000 7f6cf18c8700 1 -- 172.21.15.27:0/2534278233 shutdown_connections 2021-11-03T17:33:08.276 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.277+0000 7f6cf18c8700 1 -- 172.21.15.27:0/2534278233 wait complete. 2021-11-03T17:33:08.276 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.278+0000 7f6cf18c8700 1 Processor -- start 2021-11-03T17:33:08.277 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.278+0000 7f6cf18c8700 1 -- start start 2021-11-03T17:33:08.277 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.278+0000 7f6cf18c8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cec0f49a0 0x7f6cec1070e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:33:08.277 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.278+0000 7f6cf18c8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cec0f5340 0x7f6cec1075d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:33:08.277 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.278+0000 7f6cf18c8700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec107b80 con 0x7f6cec0f5340 2021-11-03T17:33:08.277 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.278+0000 7f6cf18c8700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec107cc0 con 0x7f6cec0f49a0 2021-11-03T17:33:08.278 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.278+0000 7f6cea7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cec0f5340 0x7f6cec1075d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:33:08.278 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.278+0000 7f6cea7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cec0f5340 0x7f6cec1075d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58070/0 (socket says 172.21.15.27:58070) 2021-11-03T17:33:08.278 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.278+0000 7f6cea7fc700 1 -- 172.21.15.27:0/3673400970 learned_addr learned my addr 172.21.15.27:0/3673400970 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:33:08.278 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.278+0000 7f6ceaffd700 1 --2- 172.21.15.27:0/3673400970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cec0f49a0 0x7f6cec1070e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:33:08.279 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.278+0000 7f6cea7fc700 1 -- 172.21.15.27:0/3673400970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cec0f49a0 msgr2=0x7f6cec1070e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:33:08.279 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.278+0000 7f6cea7fc700 1 --2- 172.21.15.27:0/3673400970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cec0f49a0 0x7f6cec1070e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:08.279 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.278+0000 7f6cea7fc700 1 -- 172.21.15.27:0/3673400970 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce4009640 con 0x7f6cec0f5340 2021-11-03T17:33:08.279 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.278+0000 7f6cea7fc700 1 --2- 172.21.15.27:0/3673400970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cec0f5340 0x7f6cec1075d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f6cd4002700 tx=0x7f6cd4009d60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:33:08.280 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.279+0000 7f6ce37fe700 1 -- 172.21.15.27:0/3673400970 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cd40079c0 con 0x7f6cec0f5340 2021-11-03T17:33:08.280 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.279+0000 7f6ce37fe700 1 -- 172.21.15.27:0/3673400970 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f6cd400f040 con 0x7f6cec0f5340 2021-11-03T17:33:08.280 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.279+0000 7f6ce37fe700 1 -- 172.21.15.27:0/3673400970 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cd4014740 con 0x7f6cec0f5340 2021-11-03T17:33:08.280 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.279+0000 7f6cf18c8700 1 -- 172.21.15.27:0/3673400970 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cec108660 con 0x7f6cec0f5340 2021-11-03T17:33:08.280 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.279+0000 7f6cf18c8700 1 -- 172.21.15.27:0/3673400970 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cec108ba0 con 0x7f6cec0f5340 2021-11-03T17:33:08.282 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.280+0000 7f6cf18c8700 1 -- 172.21.15.27:0/3673400970 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cec045ad0 con 0x7f6cec0f5340 2021-11-03T17:33:08.282 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.280+0000 7f6ce37fe700 1 -- 172.21.15.27:0/3673400970 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f6cd401b060 con 0x7f6cec0f5340 2021-11-03T17:33:08.282 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.281+0000 7f6ce37fe700 1 --2- 172.21.15.27:0/3673400970 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6cd80640b0 0x7f6cd8066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:33:08.282 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.281+0000 7f6ce37fe700 1 -- 172.21.15.27:0/3673400970 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f6cd40805e0 con 0x7f6cec0f5340 2021-11-03T17:33:08.283 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.281+0000 7f6ceaffd700 1 --2- 172.21.15.27:0/3673400970 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6cd80640b0 0x7f6cd8066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:33:08.283 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.281+0000 7f6ceaffd700 1 --2- 172.21.15.27:0/3673400970 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6cd80640b0 0x7f6cd8066570 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f6ce400b730 tx=0x7f6ce4004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:33:08.283 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.285+0000 7f6ce37fe700 1 -- 172.21.15.27:0/3673400970 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f6cd4052880 con 0x7f6cec0f5340 2021-11-03T17:33:08.460 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.465+0000 7f6cf18c8700 1 -- 172.21.15.27:0/3673400970 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6cec108e70 con 0x7f6cd80640b0 2021-11-03T17:33:08.461 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.466+0000 7f6ce37fe700 1 -- 172.21.15.27:0/3673400970 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f6cec108e70 con 0x7f6cd80640b0 2021-11-03T17:33:08.463 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.468+0000 7f6cf18c8700 1 -- 172.21.15.27:0/3673400970 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6cd80640b0 msgr2=0x7f6cd8066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:33:08.463 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.468+0000 7f6cf18c8700 1 --2- 172.21.15.27:0/3673400970 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6cd80640b0 0x7f6cd8066570 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7f6ce400b730 tx=0x7f6ce4004d10).stop 2021-11-03T17:33:08.463 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.468+0000 7f6cf18c8700 1 -- 172.21.15.27:0/3673400970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cec0f5340 msgr2=0x7f6cec1075d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:33:08.464 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.468+0000 7f6cf18c8700 1 --2- 172.21.15.27:0/3673400970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cec0f5340 0x7f6cec1075d0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f6cd4002700 tx=0x7f6cd4009d60).stop 2021-11-03T17:33:08.464 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.468+0000 7f6cf18c8700 1 -- 172.21.15.27:0/3673400970 shutdown_connections 2021-11-03T17:33:08.464 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.468+0000 7f6cf18c8700 1 --2- 172.21.15.27:0/3673400970 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6cd80640b0 0x7f6cd8066570 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:08.464 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.468+0000 7f6cf18c8700 1 --2- 172.21.15.27:0/3673400970 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cec0f49a0 0x7f6cec1070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:08.465 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.468+0000 7f6cf18c8700 1 --2- 172.21.15.27:0/3673400970 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cec0f5340 0x7f6cec1075d0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:08.465 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.468+0000 7f6cf18c8700 1 -- 172.21.15.27:0/3673400970 >> 172.21.15.27:0/3673400970 conn(0x7f6cec0f04c0 msgr2=0x7f6cec0fb510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:33:08.465 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.468+0000 7f6cf18c8700 1 -- 172.21.15.27:0/3673400970 shutdown_connections 2021-11-03T17:33:08.465 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.468+0000 7f6cf18c8700 1 -- 172.21.15.27:0/3673400970 wait complete. 2021-11-03T17:33:08.473 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:33:08.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.569+0000 7f069d578700 1 -- 172.21.15.27:0/3277472090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06980f7980 msgr2=0x7f06980f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:33:08.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.569+0000 7f069d578700 1 --2- 172.21.15.27:0/3277472090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06980f7980 0x7f06980f7de0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f068c004660 tx=0x7f068c009b30).stop 2021-11-03T17:33:08.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.569+0000 7f069d578700 1 -- 172.21.15.27:0/3277472090 shutdown_connections 2021-11-03T17:33:08.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.569+0000 7f069d578700 1 --2- 172.21.15.27:0/3277472090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06980f7980 0x7f06980f7de0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:08.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.569+0000 7f069d578700 1 --2- 172.21.15.27:0/3277472090 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06980f66e0 0x7f06980f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:08.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.569+0000 7f069d578700 1 -- 172.21.15.27:0/3277472090 >> 172.21.15.27:0/3277472090 conn(0x7f06980f1e40 msgr2=0x7f06980f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:33:08.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.569+0000 7f069d578700 1 -- 172.21.15.27:0/3277472090 shutdown_connections 2021-11-03T17:33:08.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.569+0000 7f069d578700 1 -- 172.21.15.27:0/3277472090 wait complete. 2021-11-03T17:33:08.569 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.569+0000 7f069d578700 1 Processor -- start 2021-11-03T17:33:08.569 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.570+0000 7f069d578700 1 -- start start 2021-11-03T17:33:08.569 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.570+0000 7f069d578700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06980f66e0 0x7f0698106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:33:08.570 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.570+0000 7f069d578700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06980f7980 0x7f0698107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:33:08.570 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.570+0000 7f069d578700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0698107a00 con 0x7f06980f66e0 2021-11-03T17:33:08.570 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.570+0000 7f069d578700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0698107b40 con 0x7f06980f7980 2021-11-03T17:33:08.570 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.570+0000 7f0696ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06980f66e0 0x7f0698106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:33:08.570 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.570+0000 7f0696ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06980f66e0 0x7f0698106f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58078/0 (socket says 172.21.15.27:58078) 2021-11-03T17:33:08.571 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.570+0000 7f0696ffd700 1 -- 172.21.15.27:0/2181944344 learned_addr learned my addr 172.21.15.27:0/2181944344 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:33:08.571 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.570+0000 7f06967fc700 1 --2- 172.21.15.27:0/2181944344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06980f7980 0x7f0698107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:33:08.571 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.570+0000 7f0696ffd700 1 -- 172.21.15.27:0/2181944344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06980f7980 msgr2=0x7f0698107450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:33:08.571 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.570+0000 7f0696ffd700 1 --2- 172.21.15.27:0/2181944344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06980f7980 0x7f0698107450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:08.572 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.570+0000 7f0696ffd700 1 -- 172.21.15.27:0/2181944344 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f068c005040 con 0x7f06980f66e0 2021-11-03T17:33:08.572 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.570+0000 7f06967fc700 1 --2- 172.21.15.27:0/2181944344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06980f7980 0x7f0698107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:33:08.572 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.570+0000 7f0696ffd700 1 --2- 172.21.15.27:0/2181944344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06980f66e0 0x7f0698106f60 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f0688002700 tx=0x7f0688009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:33:08.572 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.570+0000 7f067f7fe700 1 -- 172.21.15.27:0/2181944344 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0688009320 con 0x7f06980f66e0 2021-11-03T17:33:08.572 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.570+0000 7f067f7fe700 1 -- 172.21.15.27:0/2181944344 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f068800f040 con 0x7f06980f66e0 2021-11-03T17:33:08.573 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.570+0000 7f067f7fe700 1 -- 172.21.15.27:0/2181944344 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0688014900 con 0x7f06980f66e0 2021-11-03T17:33:08.573 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.570+0000 7f069d578700 1 -- 172.21.15.27:0/2181944344 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06981084e0 con 0x7f06980f66e0 2021-11-03T17:33:08.573 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.571+0000 7f069d578700 1 -- 172.21.15.27:0/2181944344 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0698108aa0 con 0x7f06980f66e0 2021-11-03T17:33:08.573 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.571+0000 7f069d578700 1 -- 172.21.15.27:0/2181944344 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0698045ad0 con 0x7f06980f66e0 2021-11-03T17:33:08.574 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.573+0000 7f067f7fe700 1 -- 172.21.15.27:0/2181944344 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f0688009480 con 0x7f06980f66e0 2021-11-03T17:33:08.574 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.573+0000 7f067f7fe700 1 --2- 172.21.15.27:0/2181944344 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f06800640b0 0x7f0680066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:33:08.574 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.573+0000 7f067f7fe700 1 -- 172.21.15.27:0/2181944344 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f0688080e30 con 0x7f06980f66e0 2021-11-03T17:33:08.574 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.573+0000 7f06967fc700 1 --2- 172.21.15.27:0/2181944344 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f06800640b0 0x7f0680066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:33:08.575 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.573+0000 7f06967fc700 1 --2- 172.21.15.27:0/2181944344 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f06800640b0 0x7f0680066570 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f068c004630 tx=0x7f068c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:33:08.575 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.577+0000 7f067f7fe700 1 -- 172.21.15.27:0/2181944344 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f06880531e0 con 0x7f06980f66e0 2021-11-03T17:33:08.744 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.749+0000 7f069d578700 1 -- 172.21.15.27:0/2181944344 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0698108d20 con 0x7f06800640b0 2021-11-03T17:33:08.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.763+0000 7f067f7fe700 1 -- 172.21.15.27:0/2181944344 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f0698108d20 con 0x7f06800640b0 2021-11-03T17:33:08.759 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:33:08.759 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (24m) 4m ago 25m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:33:08.759 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (16m) 4m ago 25m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:33:08.759 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (16m) 3m ago 25m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:33:08.760 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (24m) 4m ago 25m 40.2M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:33:08.760 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (18m) 4m ago 21m 18.2M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:33:08.760 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (21m) 4m ago 21m 3841M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:33:08.760 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (16m) 3m ago 21m 17.6M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:33:08.760 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (18m) 3m ago 21m 19.3M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:33:08.761 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (18m) 4m ago 27m 497M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:33:08.761 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (17m) 3m ago 25m 415M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:33:08.761 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (17m) 4m ago 27m 119M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:33:08.761 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (16m) 3m ago 25m 71.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:33:08.762 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (25m) 4m ago 25m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:33:08.762 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (24m) 3m ago 24m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:33:08.762 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (16m) 4m ago 24m 290M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:33:08.762 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (15m) 4m ago 24m 317M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:33:08.763 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (15m) 4m ago 23m 229M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:33:08.763 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (14m) 3m ago 23m 329M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:33:08.763 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (14m) 3m ago 23m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:33:08.763 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (13m) 3m ago 22m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:33:08.763 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (24m) 4m ago 25m 48.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:33:08.764 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.765+0000 7f069d578700 1 -- 172.21.15.27:0/2181944344 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f06800640b0 msgr2=0x7f0680066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:33:08.765 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.765+0000 7f069d578700 1 --2- 172.21.15.27:0/2181944344 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f06800640b0 0x7f0680066570 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f068c004630 tx=0x7f068c009b30).stop 2021-11-03T17:33:08.765 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.765+0000 7f069d578700 1 -- 172.21.15.27:0/2181944344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06980f66e0 msgr2=0x7f0698106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:33:08.765 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.765+0000 7f069d578700 1 --2- 172.21.15.27:0/2181944344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06980f66e0 0x7f0698106f60 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f0688002700 tx=0x7f0688009d80).stop 2021-11-03T17:33:08.765 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.765+0000 7f069d578700 1 -- 172.21.15.27:0/2181944344 shutdown_connections 2021-11-03T17:33:08.766 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.765+0000 7f069d578700 1 --2- 172.21.15.27:0/2181944344 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f06800640b0 0x7f0680066570 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:08.766 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.765+0000 7f069d578700 1 --2- 172.21.15.27:0/2181944344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06980f66e0 0x7f0698106f60 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:08.766 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.765+0000 7f069d578700 1 --2- 172.21.15.27:0/2181944344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06980f7980 0x7f0698107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:08.766 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.765+0000 7f069d578700 1 -- 172.21.15.27:0/2181944344 >> 172.21.15.27:0/2181944344 conn(0x7f06980f1e40 msgr2=0x7f06980fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:33:08.767 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.765+0000 7f069d578700 1 -- 172.21.15.27:0/2181944344 shutdown_connections 2021-11-03T17:33:08.767 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.765+0000 7f069d578700 1 -- 172.21.15.27:0/2181944344 wait complete. 2021-11-03T17:33:08.860 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.864+0000 7fe092c47700 1 -- 172.21.15.27:0/1309800757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe08c0f8530 msgr2=0x7fe08c0f46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:33:08.860 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.864+0000 7fe092c47700 1 --2- 172.21.15.27:0/1309800757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe08c0f8530 0x7fe08c0f46b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7fe07c004660 tx=0x7fe07c009b30).stop 2021-11-03T17:33:08.860 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.864+0000 7fe092c47700 1 -- 172.21.15.27:0/1309800757 shutdown_connections 2021-11-03T17:33:08.861 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.864+0000 7fe092c47700 1 --2- 172.21.15.27:0/1309800757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe08c0f8530 0x7fe08c0f46b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:08.861 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.864+0000 7fe092c47700 1 --2- 172.21.15.27:0/1309800757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe08c0f7b90 0x7fe08c0f7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:08.861 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.864+0000 7fe092c47700 1 -- 172.21.15.27:0/1309800757 >> 172.21.15.27:0/1309800757 conn(0x7fe08c0f0330 msgr2=0x7fe08c0f2750 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:33:08.861 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.865+0000 7fe092c47700 1 -- 172.21.15.27:0/1309800757 shutdown_connections 2021-11-03T17:33:08.862 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.865+0000 7fe092c47700 1 -- 172.21.15.27:0/1309800757 wait complete. 2021-11-03T17:33:08.862 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.865+0000 7fe092c47700 1 Processor -- start 2021-11-03T17:33:08.862 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.865+0000 7fe092c47700 1 -- start start 2021-11-03T17:33:08.865 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.865+0000 7fe092c47700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe08c0f7b90 0x7fe08c106d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:33:08.865 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.865+0000 7fe092c47700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe08c0f8530 0x7fe08c107250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:33:08.865 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.865+0000 7fe092c47700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c107850 con 0x7fe08c0f8530 2021-11-03T17:33:08.866 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.865+0000 7fe092c47700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c108060 con 0x7fe08c0f7b90 2021-11-03T17:33:08.866 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.865+0000 7fe091444700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe08c0f8530 0x7fe08c107250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:33:08.866 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.865+0000 7fe091444700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe08c0f8530 0x7fe08c107250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58090/0 (socket says 172.21.15.27:58090) 2021-11-03T17:33:08.866 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.865+0000 7fe091444700 1 -- 172.21.15.27:0/1158751245 learned_addr learned my addr 172.21.15.27:0/1158751245 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:33:08.867 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.866+0000 7fe091c45700 1 --2- 172.21.15.27:0/1158751245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe08c0f7b90 0x7fe08c106d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:33:08.867 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.866+0000 7fe091444700 1 -- 172.21.15.27:0/1158751245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe08c0f7b90 msgr2=0x7fe08c106d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:33:08.867 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.866+0000 7fe091444700 1 --2- 172.21.15.27:0/1158751245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe08c0f7b90 0x7fe08c106d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:08.867 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.866+0000 7fe091444700 1 -- 172.21.15.27:0/1158751245 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe07c005040 con 0x7fe08c0f8530 2021-11-03T17:33:08.868 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.866+0000 7fe091c45700 1 --2- 172.21.15.27:0/1158751245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe08c0f7b90 0x7fe08c106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:33:08.868 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.866+0000 7fe091444700 1 --2- 172.21.15.27:0/1158751245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe08c0f8530 0x7fe08c107250 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7fe07c000c00 tx=0x7fe07c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:33:08.868 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.866+0000 7fe0827fc700 1 -- 172.21.15.27:0/1158751245 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe07c019070 con 0x7fe08c0f8530 2021-11-03T17:33:08.868 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.866+0000 7fe0827fc700 1 -- 172.21.15.27:0/1158751245 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fe07c007e70 con 0x7fe08c0f8530 2021-11-03T17:33:08.869 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.866+0000 7fe092c47700 1 -- 172.21.15.27:0/1158751245 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe08c108280 con 0x7fe08c0f8530 2021-11-03T17:33:08.869 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.866+0000 7fe0827fc700 1 -- 172.21.15.27:0/1158751245 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe07c014d90 con 0x7fe08c0f8530 2021-11-03T17:33:08.869 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.866+0000 7fe092c47700 1 -- 172.21.15.27:0/1158751245 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe08c1087e0 con 0x7fe08c0f8530 2021-11-03T17:33:08.869 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.867+0000 7fe092c47700 1 -- 172.21.15.27:0/1158751245 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe08c059100 con 0x7fe08c0f8530 2021-11-03T17:33:08.870 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.867+0000 7fe0827fc700 1 -- 172.21.15.27:0/1158751245 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fe07c025070 con 0x7fe08c0f8530 2021-11-03T17:33:08.870 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.868+0000 7fe0827fc700 1 --2- 172.21.15.27:0/1158751245 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe078064100 0x7fe0780665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:33:08.870 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.868+0000 7fe0827fc700 1 -- 172.21.15.27:0/1158751245 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fe07c081800 con 0x7fe08c0f8530 2021-11-03T17:33:08.870 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.868+0000 7fe091c45700 1 --2- 172.21.15.27:0/1158751245 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe078064100 0x7fe0780665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:33:08.871 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.868+0000 7fe091c45700 1 --2- 172.21.15.27:0/1158751245 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe078064100 0x7fe0780665c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7fe088009da0 tx=0x7fe08800b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:33:08.871 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:08.872+0000 7fe0827fc700 1 -- 172.21.15.27:0/1158751245 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fe07c053b00 con 0x7fe08c0f8530 2021-11-03T17:33:09.097 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.102+0000 7fe092c47700 1 -- 172.21.15.27:0/1158751245 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe08c0fee00 con 0x7fe08c0f8530 2021-11-03T17:33:09.100 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.105+0000 7fe0827fc700 1 -- 172.21.15.27:0/1158751245 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fe07c023070 con 0x7fe08c0f8530 2021-11-03T17:33:09.100 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:33:09.100 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:33:09.101 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:33:09.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:33:09.101 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:33:09.101 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:33:09.101 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:33:09.102 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:33:09.102 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:33:09.102 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:33:09.102 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:33:09.103 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:33:09.103 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:33:09.103 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:33:09.103 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:33:09.104 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:33:09.104 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:33:09.104 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:33:09.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.107+0000 7fe092c47700 1 -- 172.21.15.27:0/1158751245 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe078064100 msgr2=0x7fe0780665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:33:09.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.107+0000 7fe092c47700 1 --2- 172.21.15.27:0/1158751245 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe078064100 0x7fe0780665c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7fe088009da0 tx=0x7fe08800b040).stop 2021-11-03T17:33:09.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.107+0000 7fe092c47700 1 -- 172.21.15.27:0/1158751245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe08c0f8530 msgr2=0x7fe08c107250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:33:09.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.107+0000 7fe092c47700 1 --2- 172.21.15.27:0/1158751245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe08c0f8530 0x7fe08c107250 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 rx=0x7fe07c000c00 tx=0x7fe07c00b040).stop 2021-11-03T17:33:09.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.107+0000 7fe092c47700 1 -- 172.21.15.27:0/1158751245 shutdown_connections 2021-11-03T17:33:09.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.107+0000 7fe092c47700 1 --2- 172.21.15.27:0/1158751245 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe078064100 0x7fe0780665c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:09.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.107+0000 7fe092c47700 1 --2- 172.21.15.27:0/1158751245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe08c0f7b90 0x7fe08c106d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:09.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.107+0000 7fe092c47700 1 --2- 172.21.15.27:0/1158751245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe08c0f8530 0x7fe08c107250 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:09.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.107+0000 7fe092c47700 1 -- 172.21.15.27:0/1158751245 >> 172.21.15.27:0/1158751245 conn(0x7fe08c0f0330 msgr2=0x7fe08c0fb420 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:33:09.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.107+0000 7fe092c47700 1 -- 172.21.15.27:0/1158751245 shutdown_connections 2021-11-03T17:33:09.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.107+0000 7fe092c47700 1 -- 172.21.15.27:0/1158751245 wait complete. 2021-11-03T17:33:09.202 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.207+0000 7febded25700 1 -- 172.21.15.27:0/3339894630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febd80f6530 msgr2=0x7febd80f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:33:09.203 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.207+0000 7febded25700 1 --2- 172.21.15.27:0/3339894630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febd80f6530 0x7febd80f6950 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7febd4004660 tx=0x7febd4009b30).stop 2021-11-03T17:33:09.203 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.207+0000 7febded25700 1 -- 172.21.15.27:0/3339894630 shutdown_connections 2021-11-03T17:33:09.203 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.207+0000 7febded25700 1 --2- 172.21.15.27:0/3339894630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7febd80f77d0 0x7febd80f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:33:09.203 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.207+0000 7febded25700 1 --2- 172.21.15.27:0/3339894630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febd80f6530 0x7febd80f6950 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:09.204 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.207+0000 7febded25700 1 -- 172.21.15.27:0/3339894630 >> 172.21.15.27:0/3339894630 conn(0x7febd80f1cd0 msgr2=0x7febd80f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:33:09.204 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.207+0000 7febded25700 1 -- 172.21.15.27:0/3339894630 shutdown_connections 2021-11-03T17:33:09.204 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.207+0000 7febded25700 1 -- 172.21.15.27:0/3339894630 wait complete. 2021-11-03T17:33:09.206 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.208+0000 7febded25700 1 Processor -- start 2021-11-03T17:33:09.207 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.208+0000 7febded25700 1 -- start start 2021-11-03T17:33:09.207 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.208+0000 7febded25700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febd80f6530 0x7febd806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:33:09.207 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.208+0000 7febded25700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7febd80f77d0 0x7febd8069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:33:09.207 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.208+0000 7febded25700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd806a410 con 0x7febd80f6530 2021-11-03T17:33:09.208 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.208+0000 7febded25700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febd806a550 con 0x7febd80f77d0 2021-11-03T17:33:09.208 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.208+0000 7febddd23700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febd80f6530 0x7febd806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:33:09.208 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.208+0000 7febddd23700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febd80f6530 0x7febd806b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58098/0 (socket says 172.21.15.27:58098) 2021-11-03T17:33:09.208 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.208+0000 7febddd23700 1 -- 172.21.15.27:0/4195686366 learned_addr learned my addr 172.21.15.27:0/4195686366 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:33:09.208 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.208+0000 7febddd23700 1 -- 172.21.15.27:0/4195686366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7febd80f77d0 msgr2=0x7febd8069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:33:09.209 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.208+0000 7febdd522700 1 --2- 172.21.15.27:0/4195686366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7febd80f77d0 0x7febd8069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:33:09.209 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.208+0000 7febddd23700 1 --2- 172.21.15.27:0/4195686366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7febd80f77d0 0x7febd8069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:09.209 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.208+0000 7febddd23700 1 -- 172.21.15.27:0/4195686366 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd4005040 con 0x7febd80f6530 2021-11-03T17:33:09.209 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.209+0000 7febddd23700 1 --2- 172.21.15.27:0/4195686366 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febd80f6530 0x7febd806b9a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7febd4004f60 tx=0x7febd4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:33:09.210 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.209+0000 7febca7fc700 1 -- 172.21.15.27:0/4195686366 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febd4018070 con 0x7febd80f6530 2021-11-03T17:33:09.210 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.209+0000 7febca7fc700 1 -- 172.21.15.27:0/4195686366 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7febd4007ba0 con 0x7febd80f6530 2021-11-03T17:33:09.210 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.209+0000 7febded25700 1 -- 172.21.15.27:0/4195686366 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febd806a7d0 con 0x7febd80f6530 2021-11-03T17:33:09.210 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.209+0000 7febca7fc700 1 -- 172.21.15.27:0/4195686366 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7febd401c620 con 0x7febd80f6530 2021-11-03T17:33:09.210 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.209+0000 7febded25700 1 -- 172.21.15.27:0/4195686366 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febd810ed80 con 0x7febd80f6530 2021-11-03T17:33:09.211 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.210+0000 7febded25700 1 -- 172.21.15.27:0/4195686366 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febd8045ad0 con 0x7febd80f6530 2021-11-03T17:33:09.211 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.210+0000 7febca7fc700 1 -- 172.21.15.27:0/4195686366 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7febd401c8e0 con 0x7febd80f6530 2021-11-03T17:33:09.211 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.211+0000 7febca7fc700 1 --2- 172.21.15.27:0/4195686366 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7febc4064100 0x7febc40665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:33:09.211 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.211+0000 7febdd522700 1 --2- 172.21.15.27:0/4195686366 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7febc4064100 0x7febc40665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:33:09.211 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.211+0000 7febca7fc700 1 -- 172.21.15.27:0/4195686366 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7febd4082740 con 0x7febd80f6530 2021-11-03T17:33:09.212 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.211+0000 7febdd522700 1 --2- 172.21.15.27:0/4195686366 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7febc4064100 0x7febc40665c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7febcc006fd0 tx=0x7febcc008040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:33:09.212 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.215+0000 7febca7fc700 1 -- 172.21.15.27:0/4195686366 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7febd4054a40 con 0x7febd80f6530 2021-11-03T17:33:09.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.421+0000 7febded25700 1 -- 172.21.15.27:0/4195686366 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7febd80fee80 con 0x7febd80f6530 2021-11-03T17:33:09.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.421+0000 7febca7fc700 1 -- 172.21.15.27:0/4195686366 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7febd4054630 con 0x7febd80f6530 2021-11-03T17:33:09.417 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:33:09.417 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:33:09.417 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:33:09.417 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:33:09.418 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:33:09.418 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:33:09.418 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:33:09.418 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:33:09.418 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:33:09.419 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:33:09.419 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:33:09.419 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:33:09.419 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:33:09.420 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:33:09.420 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:33:09.420 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:33:09.420 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:33:09.420 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:33:09.421 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:33:09.421 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:33:09.421 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:33:09.421 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:33:09.422 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:33:09.422 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:33:09.422 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:33:09.422 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:33:09.422 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:33:09.423 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:33:09.423 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:33:09.423 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:33:09.423 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:33:09.423 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:33:09.424 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:33:09.424 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:33:09.424 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:33:09.424 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:33:09.424 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:33:09.425 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:33:09.425 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:33:09.426 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.423+0000 7febded25700 1 -- 172.21.15.27:0/4195686366 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7febc4064100 msgr2=0x7febc40665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:33:09.426 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.423+0000 7febded25700 1 --2- 172.21.15.27:0/4195686366 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7febc4064100 0x7febc40665c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7febcc006fd0 tx=0x7febcc008040).stop 2021-11-03T17:33:09.426 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.423+0000 7febded25700 1 -- 172.21.15.27:0/4195686366 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febd80f6530 msgr2=0x7febd806b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:33:09.426 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.423+0000 7febded25700 1 --2- 172.21.15.27:0/4195686366 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febd80f6530 0x7febd806b9a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7febd4004f60 tx=0x7febd4009b30).stop 2021-11-03T17:33:09.427 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.423+0000 7febded25700 1 -- 172.21.15.27:0/4195686366 shutdown_connections 2021-11-03T17:33:09.427 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.423+0000 7febded25700 1 --2- 172.21.15.27:0/4195686366 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7febc4064100 0x7febc40665c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:09.427 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.423+0000 7febded25700 1 --2- 172.21.15.27:0/4195686366 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7febd80f6530 0x7febd806b9a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:09.427 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.423+0000 7febded25700 1 --2- 172.21.15.27:0/4195686366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7febd80f77d0 0x7febd8069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:09.427 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.423+0000 7febded25700 1 -- 172.21.15.27:0/4195686366 >> 172.21.15.27:0/4195686366 conn(0x7febd80f1cd0 msgr2=0x7febd80fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:33:09.428 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.423+0000 7febded25700 1 -- 172.21.15.27:0/4195686366 shutdown_connections 2021-11-03T17:33:09.428 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:09.423+0000 7febded25700 1 -- 172.21.15.27:0/4195686366 wait complete. 2021-11-03T17:33:09.428 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:33:09.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:09 smithi115 ceph-mon[65453]: pgmap v555: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:09.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:09 smithi027 ceph-mon[97244]: pgmap v555: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:10.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:10 smithi115 ceph-mon[65453]: from='client.34680 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:33:10.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:10 smithi115 ceph-mon[65453]: from='client.34684 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:33:10.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:10 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1158751245' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:10.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1032.383318 secs 2021-11-03T17:33:10.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:10 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/4195686366' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:33:10.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:10 smithi027 ceph-mon[97244]: from='client.34680 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:33:10.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:10 smithi027 ceph-mon[97244]: from='client.34684 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:33:10.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:10 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1158751245' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:10.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1032.383318 secs 2021-11-03T17:33:10.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:10 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/4195686366' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:33:11.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:11 smithi115 ceph-mon[65453]: pgmap v556: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:11.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:11 smithi027 ceph-mon[97244]: pgmap v556: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:12.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:33:12.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:33:12.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:33:12.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:12.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:12.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:12.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:12.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:33:12.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:33:12.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:33:12.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:12.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:12.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:12.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:13.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:13 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:33:13.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:13 smithi027 ceph-mon[97244]: pgmap v557: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:13.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:13 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:33:13.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:13 smithi115 ceph-mon[65453]: pgmap v557: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:14.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:14 smithi115 ceph-mon[65453]: pgmap v558: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:14.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:14 smithi027 ceph-mon[97244]: pgmap v558: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:15.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1037.383448 secs 2021-11-03T17:33:15.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1037.383448 secs 2021-11-03T17:33:16.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:16 smithi115 ceph-mon[65453]: pgmap v559: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:16.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:16 smithi027 ceph-mon[97244]: pgmap v559: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:19.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:19 smithi115 ceph-mon[65453]: pgmap v560: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:19.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:19 smithi027 ceph-mon[97244]: pgmap v560: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:20.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1042.383585 secs 2021-11-03T17:33:20.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1042.383585 secs 2021-11-03T17:33:21.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:21 smithi115 ceph-mon[65453]: pgmap v561: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:21.542 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:21 smithi027 ceph-mon[97244]: pgmap v561: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:22.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:33:22.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:33:22.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:33:22.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:22.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:22.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:22.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:22.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:33:22.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:33:22.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:33:22.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:22.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:22.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:22.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:23.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:23 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:33:23.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:23 smithi027 ceph-mon[97244]: pgmap v562: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:23.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:23 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:33:23.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:23 smithi115 ceph-mon[65453]: pgmap v562: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:25.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:25 smithi115 ceph-mon[65453]: pgmap v563: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:25.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1047.383710 secs 2021-11-03T17:33:25.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:25 smithi027 ceph-mon[97244]: pgmap v563: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:25.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1047.383710 secs 2021-11-03T17:33:27.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:27 smithi115 ceph-mon[65453]: pgmap v564: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:27.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:27 smithi027 ceph-mon[97244]: pgmap v564: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:28.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:28 smithi115 ceph-mon[65453]: pgmap v565: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:28.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:28 smithi027 ceph-mon[97244]: pgmap v565: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:29.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:29 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1052.383827 secs 2021-11-03T17:33:30.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:29 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1052.383827 secs 2021-11-03T17:33:30.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:30 smithi115 ceph-mon[65453]: pgmap v566: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:31.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:30 smithi027 ceph-mon[97244]: pgmap v566: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:31.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:33:31.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:33:31.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:33:31.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:31.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:31.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:31.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:32.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:33:32.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:33:32.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:33:32.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:32.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:32.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:32.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:32.937 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:32 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:33:32.937 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:32 smithi027 ceph-mon[97244]: pgmap v567: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:32.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:32 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:33:32.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:32 smithi115 ceph-mon[65453]: pgmap v567: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:35.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:35 smithi115 ceph-mon[65453]: pgmap v568: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:35.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1057.383966 secs 2021-11-03T17:33:35.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:35 smithi027 ceph-mon[97244]: pgmap v568: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:35.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1057.383966 secs 2021-11-03T17:33:37.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:37 smithi115 ceph-mon[65453]: pgmap v569: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:37.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:37 smithi027 ceph-mon[97244]: pgmap v569: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:39.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:39 smithi115 ceph-mon[65453]: pgmap v570: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:39.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:39 smithi027 ceph-mon[97244]: pgmap v570: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:39.532 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.537+0000 7f4a131fb700 1 -- 172.21.15.27:0/3110614074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a0c0f5580 msgr2=0x7f4a0c0f59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:33:39.532 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.537+0000 7f4a131fb700 1 --2- 172.21.15.27:0/3110614074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a0c0f5580 0x7f4a0c0f59a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f4a040099e0 tx=0x7f4a04004d10).stop 2021-11-03T17:33:39.533 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.537+0000 7f4a131fb700 1 -- 172.21.15.27:0/3110614074 shutdown_connections 2021-11-03T17:33:39.533 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.537+0000 7f4a131fb700 1 --2- 172.21.15.27:0/3110614074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a0c0f6820 0x7f4a0c0f6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:33:39.533 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.537+0000 7f4a131fb700 1 --2- 172.21.15.27:0/3110614074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a0c0f5580 0x7f4a0c0f59a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:39.534 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.537+0000 7f4a131fb700 1 -- 172.21.15.27:0/3110614074 >> 172.21.15.27:0/3110614074 conn(0x7f4a0c0f0ce0 msgr2=0x7f4a0c0f3100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:33:39.534 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.537+0000 7f4a131fb700 1 -- 172.21.15.27:0/3110614074 shutdown_connections 2021-11-03T17:33:39.534 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.537+0000 7f4a131fb700 1 -- 172.21.15.27:0/3110614074 wait complete. 2021-11-03T17:33:39.537 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.538+0000 7f4a131fb700 1 Processor -- start 2021-11-03T17:33:39.537 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.538+0000 7f4a131fb700 1 -- start start 2021-11-03T17:33:39.537 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.538+0000 7f4a131fb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a0c0f5580 0x7f4a0c109d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:33:39.537 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.538+0000 7f4a131fb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a0c0f6820 0x7f4a0c10c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:33:39.538 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.538+0000 7f4a131fb700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c060570 con 0x7f4a0c0f6820 2021-11-03T17:33:39.538 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.538+0000 7f4a131fb700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0c0606b0 con 0x7f4a0c0f5580 2021-11-03T17:33:39.538 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.538+0000 7f4a0bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a0c0f6820 0x7f4a0c10c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:33:39.538 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.539+0000 7f4a0bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a0c0f6820 0x7f4a0c10c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58120/0 (socket says 172.21.15.27:58120) 2021-11-03T17:33:39.539 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.539+0000 7f4a0bfff700 1 -- 172.21.15.27:0/1163745160 learned_addr learned my addr 172.21.15.27:0/1163745160 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:33:39.539 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.539+0000 7f4a10f97700 1 --2- 172.21.15.27:0/1163745160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a0c0f5580 0x7f4a0c109d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:33:39.539 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.539+0000 7f4a0bfff700 1 -- 172.21.15.27:0/1163745160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a0c0f5580 msgr2=0x7f4a0c109d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:33:39.539 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.539+0000 7f4a0bfff700 1 --2- 172.21.15.27:0/1163745160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a0c0f5580 0x7f4a0c109d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:39.540 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.539+0000 7f4a0bfff700 1 -- 172.21.15.27:0/1163745160 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a04009640 con 0x7f4a0c0f6820 2021-11-03T17:33:39.540 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.539+0000 7f4a0bfff700 1 --2- 172.21.15.27:0/1163745160 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a0c0f6820 0x7f4a0c10c270 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f49fc00b410 tx=0x7f49fc006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:33:39.540 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.539+0000 7f4a097fa700 1 -- 172.21.15.27:0/1163745160 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49fc0125f0 con 0x7f4a0c0f6820 2021-11-03T17:33:39.540 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.539+0000 7f4a097fa700 1 -- 172.21.15.27:0/1163745160 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f49fc00be80 con 0x7f4a0c0f6820 2021-11-03T17:33:39.540 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.539+0000 7f4a131fb700 1 -- 172.21.15.27:0/1163745160 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a0c10c880 con 0x7f4a0c0f6820 2021-11-03T17:33:39.541 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.539+0000 7f4a097fa700 1 -- 172.21.15.27:0/1163745160 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49fc002ac0 con 0x7f4a0c0f6820 2021-11-03T17:33:39.541 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.539+0000 7f4a131fb700 1 -- 172.21.15.27:0/1163745160 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a0c10cdc0 con 0x7f4a0c0f6820 2021-11-03T17:33:39.541 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.540+0000 7f4a131fb700 1 -- 172.21.15.27:0/1163745160 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a0c045ad0 con 0x7f4a0c0f6820 2021-11-03T17:33:39.541 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.541+0000 7f4a097fa700 1 -- 172.21.15.27:0/1163745160 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f49fc00f030 con 0x7f4a0c0f6820 2021-11-03T17:33:39.542 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.541+0000 7f4a097fa700 1 --2- 172.21.15.27:0/1163745160 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f49f8064100 0x7f49f80665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:33:39.542 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.541+0000 7f4a097fa700 1 -- 172.21.15.27:0/1163745160 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f49fc080c40 con 0x7f4a0c0f6820 2021-11-03T17:33:39.542 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.541+0000 7f4a10f97700 1 --2- 172.21.15.27:0/1163745160 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f49f8064100 0x7f49f80665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:33:39.542 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.542+0000 7f4a10f97700 1 --2- 172.21.15.27:0/1163745160 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f49f8064100 0x7f49f80665c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f4a0400b730 tx=0x7f4a04004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:33:39.543 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.545+0000 7f4a097fa700 1 -- 172.21.15.27:0/1163745160 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f49fc052ee0 con 0x7f4a0c0f6820 2021-11-03T17:33:39.718 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.722+0000 7f4a131fb700 1 -- 172.21.15.27:0/1163745160 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a0c0fdf30 con 0x7f49f8064100 2021-11-03T17:33:39.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.724+0000 7f4a097fa700 1 -- 172.21.15.27:0/1163745160 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f4a0c0fdf30 con 0x7f49f8064100 2021-11-03T17:33:39.721 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.725+0000 7f4a131fb700 1 -- 172.21.15.27:0/1163745160 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f49f8064100 msgr2=0x7f49f80665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:33:39.721 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.725+0000 7f4a131fb700 1 --2- 172.21.15.27:0/1163745160 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f49f8064100 0x7f49f80665c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f4a0400b730 tx=0x7f4a04004d10).stop 2021-11-03T17:33:39.721 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.725+0000 7f4a131fb700 1 -- 172.21.15.27:0/1163745160 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a0c0f6820 msgr2=0x7f4a0c10c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:33:39.722 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.725+0000 7f4a131fb700 1 --2- 172.21.15.27:0/1163745160 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a0c0f6820 0x7f4a0c10c270 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f49fc00b410 tx=0x7f49fc006ef0).stop 2021-11-03T17:33:39.722 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.726+0000 7f4a131fb700 1 -- 172.21.15.27:0/1163745160 shutdown_connections 2021-11-03T17:33:39.722 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.726+0000 7f4a131fb700 1 --2- 172.21.15.27:0/1163745160 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f49f8064100 0x7f49f80665c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:39.722 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.726+0000 7f4a131fb700 1 --2- 172.21.15.27:0/1163745160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a0c0f5580 0x7f4a0c109d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:39.723 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.726+0000 7f4a131fb700 1 --2- 172.21.15.27:0/1163745160 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a0c0f6820 0x7f4a0c10c270 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:39.723 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.726+0000 7f4a131fb700 1 -- 172.21.15.27:0/1163745160 >> 172.21.15.27:0/1163745160 conn(0x7f4a0c0f0ce0 msgr2=0x7f4a0c0fa270 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:33:39.723 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.726+0000 7f4a131fb700 1 -- 172.21.15.27:0/1163745160 shutdown_connections 2021-11-03T17:33:39.723 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.726+0000 7f4a131fb700 1 -- 172.21.15.27:0/1163745160 wait complete. 2021-11-03T17:33:39.730 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:33:39.824 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.827+0000 7f68a32d4700 1 -- 172.21.15.27:0/565439477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f689c0f5ef0 msgr2=0x7f689c0f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:33:39.824 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.827+0000 7f68a32d4700 1 --2- 172.21.15.27:0/565439477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f689c0f5ef0 0x7f689c0f6920 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f6898004660 tx=0x7f6898009b30).stop 2021-11-03T17:33:39.824 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.827+0000 7f68a32d4700 1 -- 172.21.15.27:0/565439477 shutdown_connections 2021-11-03T17:33:39.825 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.827+0000 7f68a32d4700 1 --2- 172.21.15.27:0/565439477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f689c0f5ef0 0x7f689c0f6920 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:39.825 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.827+0000 7f68a32d4700 1 --2- 172.21.15.27:0/565439477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f689c0f55e0 0x7f689c0f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:39.825 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.827+0000 7f68a32d4700 1 -- 172.21.15.27:0/565439477 >> 172.21.15.27:0/565439477 conn(0x7f689c0f0de0 msgr2=0x7f689c0f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:33:39.825 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.827+0000 7f68a32d4700 1 -- 172.21.15.27:0/565439477 shutdown_connections 2021-11-03T17:33:39.826 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.827+0000 7f68a32d4700 1 -- 172.21.15.27:0/565439477 wait complete. 2021-11-03T17:33:39.826 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.828+0000 7f68a32d4700 1 Processor -- start 2021-11-03T17:33:39.826 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.828+0000 7f68a32d4700 1 -- start start 2021-11-03T17:33:39.826 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.828+0000 7f68a32d4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f689c0f55e0 0x7f689c109bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:33:39.827 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.828+0000 7f68a32d4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f689c0f5ef0 0x7f689c10c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:33:39.827 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.828+0000 7f68a32d4700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f689c060520 con 0x7f689c0f5ef0 2021-11-03T17:33:39.827 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.828+0000 7f68a32d4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f689c060660 con 0x7f689c0f55e0 2021-11-03T17:33:39.827 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.828+0000 7f68a086f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f689c0f5ef0 0x7f689c10c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:33:39.828 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.828+0000 7f68a086f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f689c0f5ef0 0x7f689c10c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58130/0 (socket says 172.21.15.27:58130) 2021-11-03T17:33:39.828 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.828+0000 7f68a086f700 1 -- 172.21.15.27:0/4055454964 learned_addr learned my addr 172.21.15.27:0/4055454964 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:33:39.828 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.828+0000 7f68a1070700 1 --2- 172.21.15.27:0/4055454964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f689c0f55e0 0x7f689c109bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:33:39.828 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.828+0000 7f68a086f700 1 -- 172.21.15.27:0/4055454964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f689c0f55e0 msgr2=0x7f689c109bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:33:39.828 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.828+0000 7f68a086f700 1 --2- 172.21.15.27:0/4055454964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f689c0f55e0 0x7f689c109bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:39.829 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.828+0000 7f68a086f700 1 -- 172.21.15.27:0/4055454964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6898005040 con 0x7f689c0f5ef0 2021-11-03T17:33:39.829 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.828+0000 7f68a086f700 1 --2- 172.21.15.27:0/4055454964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f689c0f5ef0 0x7f689c10c0b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f6898000c00 tx=0x7f689800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:33:39.829 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.829+0000 7f6891ffb700 1 -- 172.21.15.27:0/4055454964 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6898019070 con 0x7f689c0f5ef0 2021-11-03T17:33:39.829 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.829+0000 7f6891ffb700 1 -- 172.21.15.27:0/4055454964 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f6898007e70 con 0x7f689c0f5ef0 2021-11-03T17:33:39.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.829+0000 7f68a32d4700 1 -- 172.21.15.27:0/4055454964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f689c10c660 con 0x7f689c0f5ef0 2021-11-03T17:33:39.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.829+0000 7f6891ffb700 1 -- 172.21.15.27:0/4055454964 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6898014ce0 con 0x7f689c0f5ef0 2021-11-03T17:33:39.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.829+0000 7f68a32d4700 1 -- 172.21.15.27:0/4055454964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f689c10cb90 con 0x7f689c0f5ef0 2021-11-03T17:33:39.831 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.830+0000 7f68a32d4700 1 -- 172.21.15.27:0/4055454964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f689c045ad0 con 0x7f689c0f5ef0 2021-11-03T17:33:39.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.830+0000 7f6891ffb700 1 -- 172.21.15.27:0/4055454964 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f6898025070 con 0x7f689c0f5ef0 2021-11-03T17:33:39.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.831+0000 7f6891ffb700 1 --2- 172.21.15.27:0/4055454964 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f68880640b0 0x7f6888066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:33:39.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.831+0000 7f6891ffb700 1 -- 172.21.15.27:0/4055454964 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f68980817b0 con 0x7f689c0f5ef0 2021-11-03T17:33:39.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.831+0000 7f68a1070700 1 --2- 172.21.15.27:0/4055454964 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f68880640b0 0x7f6888066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:33:39.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.831+0000 7f68a1070700 1 --2- 172.21.15.27:0/4055454964 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f68880640b0 0x7f6888066570 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f689c0601a0 tx=0x7f688c00b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:33:39.833 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:39.835+0000 7f6891ffb700 1 -- 172.21.15.27:0/4055454964 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f6898053ab0 con 0x7f689c0f5ef0 2021-11-03T17:33:40.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.010+0000 7f68a32d4700 1 -- 172.21.15.27:0/4055454964 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f689c0fdf80 con 0x7f68880640b0 2021-11-03T17:33:40.019 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.024+0000 7f6891ffb700 1 -- 172.21.15.27:0/4055454964 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f688c00f040 con 0x7f68880640b0 2021-11-03T17:33:40.020 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:33:40.020 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (25m) 5m ago 26m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:33:40.020 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (16m) 5m ago 26m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:33:40.020 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (16m) 4m ago 25m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:33:40.021 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (25m) 5m ago 26m 40.2M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:33:40.021 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (18m) 5m ago 21m 18.2M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:33:40.021 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (21m) 5m ago 21m 3841M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:33:40.021 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (17m) 4m ago 21m 17.6M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:33:40.022 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (19m) 4m ago 21m 19.3M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:33:40.022 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (18m) 5m ago 27m 497M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:33:40.022 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (18m) 4m ago 25m 415M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:33:40.022 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (17m) 5m ago 27m 119M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:33:40.023 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (17m) 4m ago 25m 71.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:33:40.023 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (26m) 5m ago 26m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:33:40.023 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (25m) 4m ago 25m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:33:40.023 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (16m) 5m ago 24m 290M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:33:40.023 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (16m) 5m ago 24m 317M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:33:40.024 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (15m) 5m ago 24m 229M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:33:40.024 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (15m) 4m ago 23m 329M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:33:40.024 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (14m) 4m ago 23m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:33:40.024 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (14m) 4m ago 23m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:33:40.025 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (24m) 5m ago 25m 48.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:33:40.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.026+0000 7f68a32d4700 1 -- 172.21.15.27:0/4055454964 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f68880640b0 msgr2=0x7f6888066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:33:40.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.026+0000 7f68a32d4700 1 --2- 172.21.15.27:0/4055454964 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f68880640b0 0x7f6888066570 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f689c0601a0 tx=0x7f688c00b040).stop 2021-11-03T17:33:40.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.026+0000 7f68a32d4700 1 -- 172.21.15.27:0/4055454964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f689c0f5ef0 msgr2=0x7f689c10c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:33:40.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.026+0000 7f68a32d4700 1 --2- 172.21.15.27:0/4055454964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f689c0f5ef0 0x7f689c10c0b0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f6898000c00 tx=0x7f689800b040).stop 2021-11-03T17:33:40.027 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.026+0000 7f68a32d4700 1 -- 172.21.15.27:0/4055454964 shutdown_connections 2021-11-03T17:33:40.027 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.026+0000 7f68a32d4700 1 --2- 172.21.15.27:0/4055454964 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f68880640b0 0x7f6888066570 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:40.027 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.026+0000 7f68a32d4700 1 --2- 172.21.15.27:0/4055454964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f689c0f55e0 0x7f689c109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:40.027 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.026+0000 7f68a32d4700 1 --2- 172.21.15.27:0/4055454964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f689c0f5ef0 0x7f689c10c0b0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:40.028 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.026+0000 7f68a32d4700 1 -- 172.21.15.27:0/4055454964 >> 172.21.15.27:0/4055454964 conn(0x7f689c0f0de0 msgr2=0x7f689c0fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:33:40.028 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.026+0000 7f68a32d4700 1 -- 172.21.15.27:0/4055454964 shutdown_connections 2021-11-03T17:33:40.028 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.026+0000 7f68a32d4700 1 -- 172.21.15.27:0/4055454964 wait complete. 2021-11-03T17:33:40.123 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.128+0000 7f0c1f9e4700 1 -- 172.21.15.27:0/2555320561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c180f55c0 msgr2=0x7f0c180f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:33:40.124 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.128+0000 7f0c1f9e4700 1 --2- 172.21.15.27:0/2555320561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c180f55c0 0x7f0c180f59e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7f0c14004660 tx=0x7f0c14009b30).stop 2021-11-03T17:33:40.124 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.128+0000 7f0c1f9e4700 1 -- 172.21.15.27:0/2555320561 shutdown_connections 2021-11-03T17:33:40.124 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.128+0000 7f0c1f9e4700 1 --2- 172.21.15.27:0/2555320561 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c180f6860 0x7f0c180f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:40.124 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.128+0000 7f0c1f9e4700 1 --2- 172.21.15.27:0/2555320561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c180f55c0 0x7f0c180f59e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:40.125 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.128+0000 7f0c1f9e4700 1 -- 172.21.15.27:0/2555320561 >> 172.21.15.27:0/2555320561 conn(0x7f0c180f0d40 msgr2=0x7f0c180f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:33:40.125 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.128+0000 7f0c1f9e4700 1 -- 172.21.15.27:0/2555320561 shutdown_connections 2021-11-03T17:33:40.125 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.128+0000 7f0c1f9e4700 1 -- 172.21.15.27:0/2555320561 wait complete. 2021-11-03T17:33:40.128 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.129+0000 7f0c1f9e4700 1 Processor -- start 2021-11-03T17:33:40.128 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.129+0000 7f0c1f9e4700 1 -- start start 2021-11-03T17:33:40.128 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.129+0000 7f0c1f9e4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c180f55c0 0x7f0c1810c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:33:40.128 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.129+0000 7f0c1f9e4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c180f6860 0x7f0c1810e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:33:40.129 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.129+0000 7f0c1f9e4700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c180602b0 con 0x7f0c180f6860 2021-11-03T17:33:40.129 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.129+0000 7f0c1f9e4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c180603f0 con 0x7f0c180f55c0 2021-11-03T17:33:40.129 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.129+0000 7f0c1cf7f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c180f6860 0x7f0c1810e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:33:40.129 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.129+0000 7f0c1cf7f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c180f6860 0x7f0c1810e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58140/0 (socket says 172.21.15.27:58140) 2021-11-03T17:33:40.130 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.129+0000 7f0c1cf7f700 1 -- 172.21.15.27:0/3160920321 learned_addr learned my addr 172.21.15.27:0/3160920321 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:33:40.130 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.129+0000 7f0c1d780700 1 --2- 172.21.15.27:0/3160920321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c180f55c0 0x7f0c1810c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:33:40.130 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.129+0000 7f0c1cf7f700 1 -- 172.21.15.27:0/3160920321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c180f55c0 msgr2=0x7f0c1810c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:33:40.130 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.129+0000 7f0c1cf7f700 1 --2- 172.21.15.27:0/3160920321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c180f55c0 0x7f0c1810c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:40.131 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.129+0000 7f0c1cf7f700 1 -- 172.21.15.27:0/3160920321 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c14005040 con 0x7f0c180f6860 2021-11-03T17:33:40.131 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.129+0000 7f0c1d780700 1 --2- 172.21.15.27:0/3160920321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c180f55c0 0x7f0c1810c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:33:40.131 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.129+0000 7f0c1cf7f700 1 --2- 172.21.15.27:0/3160920321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c180f6860 0x7f0c1810e580 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f0c080026d0 tx=0x7f0c08009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:33:40.131 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.129+0000 7f0c0dffb700 1 -- 172.21.15.27:0/3160920321 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c08009320 con 0x7f0c180f6860 2021-11-03T17:33:40.131 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.130+0000 7f0c1f9e4700 1 -- 172.21.15.27:0/3160920321 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c1810ea70 con 0x7f0c180f6860 2021-11-03T17:33:40.132 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.130+0000 7f0c0dffb700 1 -- 172.21.15.27:0/3160920321 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f0c0800f040 con 0x7f0c180f6860 2021-11-03T17:33:40.132 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.130+0000 7f0c0dffb700 1 -- 172.21.15.27:0/3160920321 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c080147e0 con 0x7f0c180f6860 2021-11-03T17:33:40.132 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.130+0000 7f0c1f9e4700 1 -- 172.21.15.27:0/3160920321 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c1810eef0 con 0x7f0c180f6860 2021-11-03T17:33:40.132 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.131+0000 7f0c0dffb700 1 -- 172.21.15.27:0/3160920321 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f0c080077b0 con 0x7f0c180f6860 2021-11-03T17:33:40.133 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.131+0000 7f0c1f9e4700 1 -- 172.21.15.27:0/3160920321 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c18045ad0 con 0x7f0c180f6860 2021-11-03T17:33:40.133 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.131+0000 7f0c0dffb700 1 --2- 172.21.15.27:0/3160920321 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0c04064090 0x7f0c04066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:33:40.133 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.131+0000 7f0c0dffb700 1 -- 172.21.15.27:0/3160920321 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f0c08080e40 con 0x7f0c180f6860 2021-11-03T17:33:40.133 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.131+0000 7f0c1d780700 1 --2- 172.21.15.27:0/3160920321 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0c04064090 0x7f0c04066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:33:40.134 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.132+0000 7f0c1d780700 1 --2- 172.21.15.27:0/3160920321 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0c04064090 0x7f0c04066550 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f0c18060040 tx=0x7f0c14009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:33:40.134 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.136+0000 7f0c0dffb700 1 -- 172.21.15.27:0/3160920321 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f0c08053140 con 0x7f0c180f6860 2021-11-03T17:33:40.359 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.364+0000 7f0c1f9e4700 1 -- 172.21.15.27:0/3160920321 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0c18060580 con 0x7f0c180f6860 2021-11-03T17:33:40.359 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.364+0000 7f0c0dffb700 1 -- 172.21.15.27:0/3160920321 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f0c08052d30 con 0x7f0c180f6860 2021-11-03T17:33:40.360 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:33:40.360 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:33:40.360 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:33:40.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:33:40.361 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:33:40.361 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:33:40.361 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:33:40.362 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:33:40.362 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:33:40.362 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:33:40.362 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:33:40.363 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:33:40.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:33:40.363 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:33:40.363 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:33:40.364 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:33:40.364 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:33:40.364 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:33:40.365 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.366+0000 7f0c1f9e4700 1 -- 172.21.15.27:0/3160920321 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0c04064090 msgr2=0x7f0c04066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:33:40.365 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.366+0000 7f0c1f9e4700 1 --2- 172.21.15.27:0/3160920321 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0c04064090 0x7f0c04066550 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 rx=0x7f0c18060040 tx=0x7f0c14009b30).stop 2021-11-03T17:33:40.365 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.366+0000 7f0c1f9e4700 1 -- 172.21.15.27:0/3160920321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c180f6860 msgr2=0x7f0c1810e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:33:40.366 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.366+0000 7f0c1f9e4700 1 --2- 172.21.15.27:0/3160920321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c180f6860 0x7f0c1810e580 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7f0c080026d0 tx=0x7f0c08009de0).stop 2021-11-03T17:33:40.366 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.366+0000 7f0c1f9e4700 1 -- 172.21.15.27:0/3160920321 shutdown_connections 2021-11-03T17:33:40.366 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.366+0000 7f0c1f9e4700 1 --2- 172.21.15.27:0/3160920321 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0c04064090 0x7f0c04066550 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:40.366 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.366+0000 7f0c1f9e4700 1 --2- 172.21.15.27:0/3160920321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c180f55c0 0x7f0c1810c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:40.366 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.366+0000 7f0c1f9e4700 1 --2- 172.21.15.27:0/3160920321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c180f6860 0x7f0c1810e580 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:40.367 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.366+0000 7f0c1f9e4700 1 -- 172.21.15.27:0/3160920321 >> 172.21.15.27:0/3160920321 conn(0x7f0c180f0d40 msgr2=0x7f0c180fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:33:40.367 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.366+0000 7f0c1f9e4700 1 -- 172.21.15.27:0/3160920321 shutdown_connections 2021-11-03T17:33:40.367 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.366+0000 7f0c1f9e4700 1 -- 172.21.15.27:0/3160920321 wait complete. 2021-11-03T17:33:40.463 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.466+0000 7f7d15305700 1 -- 172.21.15.27:0/3958099486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d100f5450 msgr2=0x7f7d100f5870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:33:40.464 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.466+0000 7f7d15305700 1 --2- 172.21.15.27:0/3958099486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d100f5450 0x7f7d100f5870 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f7d00004660 tx=0x7f7d00009b30).stop 2021-11-03T17:33:40.464 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.466+0000 7f7d15305700 1 -- 172.21.15.27:0/3958099486 shutdown_connections 2021-11-03T17:33:40.465 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.466+0000 7f7d15305700 1 --2- 172.21.15.27:0/3958099486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d100f66f0 0x7f7d100f6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:40.465 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.466+0000 7f7d15305700 1 --2- 172.21.15.27:0/3958099486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d100f5450 0x7f7d100f5870 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:40.465 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.466+0000 7f7d15305700 1 -- 172.21.15.27:0/3958099486 >> 172.21.15.27:0/3958099486 conn(0x7f7d100f0bd0 msgr2=0x7f7d100f2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:33:40.465 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.466+0000 7f7d15305700 1 -- 172.21.15.27:0/3958099486 shutdown_connections 2021-11-03T17:33:40.466 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.466+0000 7f7d15305700 1 -- 172.21.15.27:0/3958099486 wait complete. 2021-11-03T17:33:40.466 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.467+0000 7f7d15305700 1 Processor -- start 2021-11-03T17:33:40.466 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.467+0000 7f7d15305700 1 -- start start 2021-11-03T17:33:40.466 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.467+0000 7f7d15305700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d100f5450 0x7f7d10109a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:33:40.467 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.467+0000 7f7d15305700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d100f66f0 0x7f7d1010bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:33:40.467 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.467+0000 7f7d15305700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d100603e0 con 0x7f7d100f66f0 2021-11-03T17:33:40.467 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.467+0000 7f7d15305700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d10060520 con 0x7f7d100f5450 2021-11-03T17:33:40.467 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.467+0000 7f7d0e7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d100f66f0 0x7f7d1010bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:33:40.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.467+0000 7f7d0e7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d100f66f0 0x7f7d1010bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58150/0 (socket says 172.21.15.27:58150) 2021-11-03T17:33:40.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.467+0000 7f7d0e7fc700 1 -- 172.21.15.27:0/1499603621 learned_addr learned my addr 172.21.15.27:0/1499603621 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:33:40.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.467+0000 7f7d0effd700 1 --2- 172.21.15.27:0/1499603621 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d100f5450 0x7f7d10109a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:33:40.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.468+0000 7f7d0e7fc700 1 -- 172.21.15.27:0/1499603621 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d100f5450 msgr2=0x7f7d10109a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:33:40.469 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.468+0000 7f7d0e7fc700 1 --2- 172.21.15.27:0/1499603621 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d100f5450 0x7f7d10109a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:40.469 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.468+0000 7f7d0e7fc700 1 -- 172.21.15.27:0/1499603621 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d00005040 con 0x7f7d100f66f0 2021-11-03T17:33:40.469 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.468+0000 7f7d0effd700 1 --2- 172.21.15.27:0/1499603621 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d100f5450 0x7f7d10109a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:33:40.469 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.468+0000 7f7d0e7fc700 1 --2- 172.21.15.27:0/1499603621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d100f66f0 0x7f7d1010bf60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f7d04002700 tx=0x7f7d04009e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:33:40.469 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.468+0000 7f7cf77fe700 1 -- 172.21.15.27:0/1499603621 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d04009320 con 0x7f7d100f66f0 2021-11-03T17:33:40.470 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.468+0000 7f7d15305700 1 -- 172.21.15.27:0/1499603621 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d1010c510 con 0x7f7d100f66f0 2021-11-03T17:33:40.470 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.468+0000 7f7cf77fe700 1 -- 172.21.15.27:0/1499603621 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f7d0400f040 con 0x7f7d100f66f0 2021-11-03T17:33:40.470 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.468+0000 7f7cf77fe700 1 -- 172.21.15.27:0/1499603621 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d04014820 con 0x7f7d100f66f0 2021-11-03T17:33:40.472 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.468+0000 7f7d15305700 1 -- 172.21.15.27:0/1499603621 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d1010caa0 con 0x7f7d100f66f0 2021-11-03T17:33:40.472 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.469+0000 7f7d15305700 1 -- 172.21.15.27:0/1499603621 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d10059100 con 0x7f7d100f66f0 2021-11-03T17:33:40.472 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.469+0000 7f7cf77fe700 1 -- 172.21.15.27:0/1499603621 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f7d04009480 con 0x7f7d100f66f0 2021-11-03T17:33:40.472 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.470+0000 7f7cf77fe700 1 --2- 172.21.15.27:0/1499603621 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7cf8064100 0x7f7cf80665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:33:40.473 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.470+0000 7f7cf77fe700 1 -- 172.21.15.27:0/1499603621 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f7d04080b80 con 0x7f7d100f66f0 2021-11-03T17:33:40.473 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.470+0000 7f7d0effd700 1 --2- 172.21.15.27:0/1499603621 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7cf8064100 0x7f7cf80665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:33:40.473 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.470+0000 7f7d0effd700 1 --2- 172.21.15.27:0/1499603621 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7cf8064100 0x7f7cf80665c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f7d00004e30 tx=0x7f7d00009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:33:40.473 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.474+0000 7f7cf77fe700 1 -- 172.21.15.27:0/1499603621 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f7d04052e80 con 0x7f7d100f66f0 2021-11-03T17:33:40.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1062.384050 secs 2021-11-03T17:33:40.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1062.384050 secs 2021-11-03T17:33:40.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.678+0000 7f7d15305700 1 -- 172.21.15.27:0/1499603621 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7d10045ad0 con 0x7f7d100f66f0 2021-11-03T17:33:40.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.678+0000 7f7cf77fe700 1 -- 172.21.15.27:0/1499603621 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f7d04080070 con 0x7f7d100f66f0 2021-11-03T17:33:40.674 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:33:40.674 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:33:40.675 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:33:40.675 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:33:40.675 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:33:40.675 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:33:40.676 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:33:40.676 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:33:40.676 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:33:40.676 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:33:40.677 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:33:40.677 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:33:40.677 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:33:40.677 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:33:40.677 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:33:40.678 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:33:40.678 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:33:40.678 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:33:40.678 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:33:40.679 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:33:40.679 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:33:40.679 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:33:40.679 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:33:40.679 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:33:40.680 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:33:40.680 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:33:40.680 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:33:40.680 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:33:40.681 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:33:40.681 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:33:40.681 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:33:40.681 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:33:40.681 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:33:40.682 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:33:40.682 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:33:40.682 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:33:40.682 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:33:40.683 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:33:40.683 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:33:40.684 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.680+0000 7f7d15305700 1 -- 172.21.15.27:0/1499603621 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7cf8064100 msgr2=0x7f7cf80665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:33:40.684 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.680+0000 7f7d15305700 1 --2- 172.21.15.27:0/1499603621 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7cf8064100 0x7f7cf80665c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f7d00004e30 tx=0x7f7d00009b30).stop 2021-11-03T17:33:40.684 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.680+0000 7f7d15305700 1 -- 172.21.15.27:0/1499603621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d100f66f0 msgr2=0x7f7d1010bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:33:40.684 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.680+0000 7f7d15305700 1 --2- 172.21.15.27:0/1499603621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d100f66f0 0x7f7d1010bf60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f7d04002700 tx=0x7f7d04009e00).stop 2021-11-03T17:33:40.685 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.680+0000 7f7d15305700 1 -- 172.21.15.27:0/1499603621 shutdown_connections 2021-11-03T17:33:40.685 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.680+0000 7f7d15305700 1 --2- 172.21.15.27:0/1499603621 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7cf8064100 0x7f7cf80665c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:40.685 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.680+0000 7f7d15305700 1 --2- 172.21.15.27:0/1499603621 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d100f5450 0x7f7d10109a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:40.686 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.680+0000 7f7d15305700 1 --2- 172.21.15.27:0/1499603621 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d100f66f0 0x7f7d1010bf60 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:33:40.686 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.680+0000 7f7d15305700 1 -- 172.21.15.27:0/1499603621 >> 172.21.15.27:0/1499603621 conn(0x7f7d100f0bd0 msgr2=0x7f7d100fa140 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:33:40.686 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.680+0000 7f7d15305700 1 -- 172.21.15.27:0/1499603621 shutdown_connections 2021-11-03T17:33:40.686 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:33:40.680+0000 7f7d15305700 1 -- 172.21.15.27:0/1499603621 wait complete. 2021-11-03T17:33:40.687 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:33:41.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:41 smithi115 ceph-mon[65453]: from='client.34696 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:33:41.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:41 smithi115 ceph-mon[65453]: from='client.34700 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:33:41.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:41 smithi115 ceph-mon[65453]: pgmap v571: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:41.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:41 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3160920321' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:41.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:41 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1499603621' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:33:41.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:41 smithi027 ceph-mon[97244]: from='client.34696 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:33:41.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:41 smithi027 ceph-mon[97244]: from='client.34700 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:33:41.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:41 smithi027 ceph-mon[97244]: pgmap v571: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:41.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:41 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3160920321' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:41.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:41 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1499603621' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:33:42.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:33:42.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:33:42.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:33:42.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:42.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:42.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:42.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:42.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:33:42.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:33:42.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:33:42.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:42.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:42.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:42.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:43.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:43 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:33:43.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:43 smithi027 ceph-mon[97244]: pgmap v572: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:43.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:43 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:33:43.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:43 smithi115 ceph-mon[65453]: pgmap v572: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:44.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:44 smithi115 ceph-mon[65453]: pgmap v573: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:44.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:44 smithi027 ceph-mon[97244]: pgmap v573: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:45.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1067.384200 secs 2021-11-03T17:33:45.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1067.384200 secs 2021-11-03T17:33:46.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:46 smithi115 ceph-mon[65453]: pgmap v574: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:46.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:46 smithi027 ceph-mon[97244]: pgmap v574: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:49.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:49 smithi115 ceph-mon[65453]: pgmap v575: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:49.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:49 smithi027 ceph-mon[97244]: pgmap v575: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:50.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1072.384326 secs 2021-11-03T17:33:50.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1072.384326 secs 2021-11-03T17:33:51.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:51 smithi115 ceph-mon[65453]: pgmap v576: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:51.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:51 smithi027 ceph-mon[97244]: pgmap v576: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:52.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:33:52.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:33:52.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:33:52.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:52.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:52.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:52.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:52.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:33:52.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:33:52.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:33:52.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:52.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:52.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:52.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:33:53.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:53 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:33:53.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:53 smithi027 ceph-mon[97244]: pgmap v577: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:53.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:53 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:33:53.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:53 smithi115 ceph-mon[65453]: pgmap v577: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:54.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:54 smithi115 ceph-mon[65453]: pgmap v578: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:54.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:54 smithi027 ceph-mon[97244]: pgmap v578: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:55.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1077.384442 secs 2021-11-03T17:33:55.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1077.384442 secs 2021-11-03T17:33:56.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:33:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:33:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:56 smithi115 ceph-mon[65453]: pgmap v579: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:33:56.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:33:56.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:56 smithi027 ceph-mon[97244]: pgmap v579: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:59.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:33:59 smithi115 ceph-mon[65453]: pgmap v580: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:33:59.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:33:59 smithi027 ceph-mon[97244]: pgmap v580: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:00.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1082.384559 secs 2021-11-03T17:34:00.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1082.384559 secs 2021-11-03T17:34:01.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:01 smithi115 ceph-mon[65453]: pgmap v581: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:01.515 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:01 smithi027 ceph-mon[97244]: pgmap v581: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:02.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:34:02.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:34:02.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:34:02.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:02.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:02.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:02.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:02.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:34:02.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:34:02.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:34:02.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:02.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:02.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:02.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:03.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:03 smithi027 ceph-mon[97244]: poll took 60162.427 ms: 1 events 2021-11-03T17:34:03.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:03 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:34:03.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:03 smithi027 ceph-mon[97244]: pgmap v582: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:03.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:03 smithi115 ceph-mon[65453]: poll took 60162.427 ms: 1 events 2021-11-03T17:34:03.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:03 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:34:03.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:03 smithi115 ceph-mon[65453]: pgmap v582: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:04.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:04 smithi115 ceph-mon[65453]: pgmap v583: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:04.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:04 smithi027 ceph-mon[97244]: pgmap v583: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:05.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1087.384688 secs 2021-11-03T17:34:05.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1087.384688 secs 2021-11-03T17:34:06.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:06 smithi115 ceph-mon[65453]: pgmap v584: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:06.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:06 smithi027 ceph-mon[97244]: pgmap v584: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:09.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:09 smithi115 ceph-mon[65453]: pgmap v585: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:09.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:09 smithi027 ceph-mon[97244]: pgmap v585: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:10.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1092.384821 secs 2021-11-03T17:34:10.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1092.384821 secs 2021-11-03T17:34:10.793 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.797+0000 7fa34326b700 1 -- 172.21.15.27:0/3847717197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa33c0f66e0 msgr2=0x7fa33c0f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:34:10.793 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.797+0000 7fa34326b700 1 --2- 172.21.15.27:0/3847717197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa33c0f66e0 0x7fa33c0f6b00 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7fa334004660 tx=0x7fa334009b30).stop 2021-11-03T17:34:10.794 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.797+0000 7fa34326b700 1 -- 172.21.15.27:0/3847717197 shutdown_connections 2021-11-03T17:34:10.794 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.797+0000 7fa34326b700 1 --2- 172.21.15.27:0/3847717197 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa33c0f6ff0 0x7fa33c0f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:10.794 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.797+0000 7fa34326b700 1 --2- 172.21.15.27:0/3847717197 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa33c0f66e0 0x7fa33c0f6b00 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:10.794 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.797+0000 7fa34326b700 1 -- 172.21.15.27:0/3847717197 >> 172.21.15.27:0/3847717197 conn(0x7fa33c0f1f00 msgr2=0x7fa33c0f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:34:10.795 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.797+0000 7fa34326b700 1 -- 172.21.15.27:0/3847717197 shutdown_connections 2021-11-03T17:34:10.795 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.797+0000 7fa34326b700 1 -- 172.21.15.27:0/3847717197 wait complete. 2021-11-03T17:34:10.795 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.798+0000 7fa34326b700 1 Processor -- start 2021-11-03T17:34:10.795 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.798+0000 7fa34326b700 1 -- start start 2021-11-03T17:34:10.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.798+0000 7fa34326b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa33c0f66e0 0x7fa33c06b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:34:10.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.798+0000 7fa34326b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa33c0f6ff0 0x7fa33c069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:34:10.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.798+0000 7fa34326b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c06a380 con 0x7fa33c0f66e0 2021-11-03T17:34:10.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.798+0000 7fa34326b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa33c06a4c0 con 0x7fa33c0f6ff0 2021-11-03T17:34:10.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.798+0000 7fa341007700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa33c0f66e0 0x7fa33c06b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:34:10.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.798+0000 7fa341007700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa33c0f66e0 0x7fa33c06b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58166/0 (socket says 172.21.15.27:58166) 2021-11-03T17:34:10.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.798+0000 7fa341007700 1 -- 172.21.15.27:0/2860735668 learned_addr learned my addr 172.21.15.27:0/2860735668 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:34:10.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.798+0000 7fa341007700 1 -- 172.21.15.27:0/2860735668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa33c0f6ff0 msgr2=0x7fa33c069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:34:10.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.798+0000 7fa341007700 1 --2- 172.21.15.27:0/2860735668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa33c0f6ff0 0x7fa33c069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:34:10.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.798+0000 7fa341007700 1 -- 172.21.15.27:0/2860735668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa334005040 con 0x7fa33c0f66e0 2021-11-03T17:34:10.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.798+0000 7fa341007700 1 --2- 172.21.15.27:0/2860735668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa33c0f66e0 0x7fa33c06b9a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7fa334009830 tx=0x7fa334009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:34:10.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.799+0000 7fa339ffb700 1 -- 172.21.15.27:0/2860735668 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa334018070 con 0x7fa33c0f66e0 2021-11-03T17:34:10.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.799+0000 7fa34326b700 1 -- 172.21.15.27:0/2860735668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa33c06a740 con 0x7fa33c0f66e0 2021-11-03T17:34:10.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.799+0000 7fa339ffb700 1 -- 172.21.15.27:0/2860735668 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fa334007ba0 con 0x7fa33c0f66e0 2021-11-03T17:34:10.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.799+0000 7fa339ffb700 1 -- 172.21.15.27:0/2860735668 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa33401c620 con 0x7fa33c0f66e0 2021-11-03T17:34:10.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.799+0000 7fa34326b700 1 -- 172.21.15.27:0/2860735668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa33c06aca0 con 0x7fa33c0f66e0 2021-11-03T17:34:10.800 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.800+0000 7fa34326b700 1 -- 172.21.15.27:0/2860735668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa33c045ad0 con 0x7fa33c0f66e0 2021-11-03T17:34:10.801 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.800+0000 7fa339ffb700 1 -- 172.21.15.27:0/2860735668 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fa334007d10 con 0x7fa33c0f66e0 2021-11-03T17:34:10.801 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.801+0000 7fa339ffb700 1 --2- 172.21.15.27:0/2860735668 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa324064090 0x7fa324066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:34:10.801 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.801+0000 7fa339ffb700 1 -- 172.21.15.27:0/2860735668 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fa334082570 con 0x7fa33c0f66e0 2021-11-03T17:34:10.802 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.801+0000 7fa340806700 1 --2- 172.21.15.27:0/2860735668 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa324064090 0x7fa324066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:34:10.802 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.801+0000 7fa340806700 1 --2- 172.21.15.27:0/2860735668 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa324064090 0x7fa324066550 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7fa330004bd0 tx=0x7fa330009b10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:34:10.802 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.805+0000 7fa339ffb700 1 -- 172.21.15.27:0/2860735668 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fa334054930 con 0x7fa33c0f66e0 2021-11-03T17:34:10.977 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.983+0000 7fa34326b700 1 -- 172.21.15.27:0/2860735668 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa33c0008d0 con 0x7fa324064090 2021-11-03T17:34:10.978 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.985+0000 7fa339ffb700 1 -- 172.21.15.27:0/2860735668 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fa33c0008d0 con 0x7fa324064090 2021-11-03T17:34:10.981 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.986+0000 7fa34326b700 1 -- 172.21.15.27:0/2860735668 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa324064090 msgr2=0x7fa324066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:34:10.981 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.986+0000 7fa34326b700 1 --2- 172.21.15.27:0/2860735668 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa324064090 0x7fa324066550 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7fa330004bd0 tx=0x7fa330009b10).stop 2021-11-03T17:34:10.981 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.986+0000 7fa34326b700 1 -- 172.21.15.27:0/2860735668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa33c0f66e0 msgr2=0x7fa33c06b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:34:10.982 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.986+0000 7fa34326b700 1 --2- 172.21.15.27:0/2860735668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa33c0f66e0 0x7fa33c06b9a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7fa334009830 tx=0x7fa334009b30).stop 2021-11-03T17:34:10.982 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.986+0000 7fa34326b700 1 -- 172.21.15.27:0/2860735668 shutdown_connections 2021-11-03T17:34:10.982 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.986+0000 7fa34326b700 1 --2- 172.21.15.27:0/2860735668 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa324064090 0x7fa324066550 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:10.982 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.986+0000 7fa34326b700 1 --2- 172.21.15.27:0/2860735668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa33c0f66e0 0x7fa33c06b9a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:10.983 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.987+0000 7fa34326b700 1 --2- 172.21.15.27:0/2860735668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa33c0f6ff0 0x7fa33c069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:34:10.983 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.987+0000 7fa34326b700 1 -- 172.21.15.27:0/2860735668 >> 172.21.15.27:0/2860735668 conn(0x7fa33c0f1f00 msgr2=0x7fa33c0fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:34:10.983 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.987+0000 7fa34326b700 1 -- 172.21.15.27:0/2860735668 shutdown_connections 2021-11-03T17:34:10.983 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:10.987+0000 7fa34326b700 1 -- 172.21.15.27:0/2860735668 wait complete. 2021-11-03T17:34:10.990 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:34:11.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.087+0000 7f51a9709700 1 -- 172.21.15.27:0/1583510079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a40f55e0 msgr2=0x7f51a40f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:34:11.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.087+0000 7f51a9709700 1 --2- 172.21.15.27:0/1583510079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a40f55e0 0x7f51a40f5a00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7f518c004660 tx=0x7f518c009b30).stop 2021-11-03T17:34:11.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.087+0000 7f51a9709700 1 -- 172.21.15.27:0/1583510079 shutdown_connections 2021-11-03T17:34:11.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.087+0000 7f51a9709700 1 --2- 172.21.15.27:0/1583510079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51a40f5ef0 0x7f51a40f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:11.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.087+0000 7f51a9709700 1 --2- 172.21.15.27:0/1583510079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a40f55e0 0x7f51a40f5a00 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:11.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.087+0000 7f51a9709700 1 -- 172.21.15.27:0/1583510079 >> 172.21.15.27:0/1583510079 conn(0x7f51a40f0de0 msgr2=0x7f51a40f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:34:11.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.087+0000 7f51a9709700 1 -- 172.21.15.27:0/1583510079 shutdown_connections 2021-11-03T17:34:11.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.087+0000 7f51a9709700 1 -- 172.21.15.27:0/1583510079 wait complete. 2021-11-03T17:34:11.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.088+0000 7f51a9709700 1 Processor -- start 2021-11-03T17:34:11.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.088+0000 7f51a9709700 1 -- start start 2021-11-03T17:34:11.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.088+0000 7f51a9709700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51a40f55e0 0x7f51a4109c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:34:11.086 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.088+0000 7f51a9709700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a40f5ef0 0x7f51a410c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:34:11.086 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.088+0000 7f51a9709700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a4060500 con 0x7f51a40f5ef0 2021-11-03T17:34:11.086 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.088+0000 7f51a9709700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a4060640 con 0x7f51a40f55e0 2021-11-03T17:34:11.086 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.088+0000 7f51a27fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a40f5ef0 0x7f51a410c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:34:11.087 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.088+0000 7f51a27fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a40f5ef0 0x7f51a410c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58178/0 (socket says 172.21.15.27:58178) 2021-11-03T17:34:11.087 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.088+0000 7f51a27fc700 1 -- 172.21.15.27:0/2427126772 learned_addr learned my addr 172.21.15.27:0/2427126772 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:34:11.087 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.088+0000 7f51a27fc700 1 -- 172.21.15.27:0/2427126772 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51a40f55e0 msgr2=0x7f51a4109c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:34:11.087 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.088+0000 7f51a27fc700 1 --2- 172.21.15.27:0/2427126772 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51a40f55e0 0x7f51a4109c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:34:11.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.088+0000 7f51a27fc700 1 -- 172.21.15.27:0/2427126772 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f518c005040 con 0x7f51a40f5ef0 2021-11-03T17:34:11.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.089+0000 7f51a27fc700 1 --2- 172.21.15.27:0/2427126772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a40f5ef0 0x7f51a410c190 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f51940026d0 tx=0x7f5194009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:34:11.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.089+0000 7f519b7fe700 1 -- 172.21.15.27:0/2427126772 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5194009320 con 0x7f51a40f5ef0 2021-11-03T17:34:11.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.089+0000 7f51a9709700 1 -- 172.21.15.27:0/2427126772 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a410c740 con 0x7f51a40f5ef0 2021-11-03T17:34:11.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.089+0000 7f519b7fe700 1 -- 172.21.15.27:0/2427126772 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f519400f040 con 0x7f51a40f5ef0 2021-11-03T17:34:11.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.089+0000 7f519b7fe700 1 -- 172.21.15.27:0/2427126772 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51940147e0 con 0x7f51a40f5ef0 2021-11-03T17:34:11.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.089+0000 7f51a9709700 1 -- 172.21.15.27:0/2427126772 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a410cd00 con 0x7f51a40f5ef0 2021-11-03T17:34:11.090 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.090+0000 7f51a9709700 1 -- 172.21.15.27:0/2427126772 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51a4045ad0 con 0x7f51a40f5ef0 2021-11-03T17:34:11.090 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.090+0000 7f519b7fe700 1 -- 172.21.15.27:0/2427126772 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f5194009480 con 0x7f51a40f5ef0 2021-11-03T17:34:11.090 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.090+0000 7f519b7fe700 1 --2- 172.21.15.27:0/2427126772 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f51900640b0 0x7f5190066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:34:11.091 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.091+0000 7f519b7fe700 1 -- 172.21.15.27:0/2427126772 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f5194080f40 con 0x7f51a40f5ef0 2021-11-03T17:34:11.091 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.091+0000 7f51a2ffd700 1 --2- 172.21.15.27:0/2427126772 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f51900640b0 0x7f5190066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:34:11.091 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.091+0000 7f51a2ffd700 1 --2- 172.21.15.27:0/2427126772 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f51900640b0 0x7f5190066570 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f518c004e30 tx=0x7f518c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:34:11.091 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.095+0000 7f519b7fe700 1 -- 172.21.15.27:0/2427126772 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f51940addf0 con 0x7f51a40f5ef0 2021-11-03T17:34:11.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.268+0000 7f51a9709700 1 -- 172.21.15.27:0/2427126772 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f51a40fdf90 con 0x7f51900640b0 2021-11-03T17:34:11.276 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.282+0000 7f519b7fe700 1 -- 172.21.15.27:0/2427126772 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f51a40fdf90 con 0x7f51900640b0 2021-11-03T17:34:11.277 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:34:11.277 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (25m) 5m ago 26m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:34:11.277 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (17m) 5m ago 26m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:34:11.277 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (17m) 4m ago 26m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:34:11.278 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (25m) 5m ago 26m 40.2M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:34:11.278 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (19m) 5m ago 22m 18.2M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:34:11.278 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (22m) 5m ago 22m 3841M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:34:11.278 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (17m) 4m ago 22m 17.6M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:34:11.279 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (19m) 4m ago 22m 19.3M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:34:11.279 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (19m) 5m ago 28m 497M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:34:11.279 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (18m) 4m ago 26m 415M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:34:11.279 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (18m) 5m ago 28m 119M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:34:11.280 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (18m) 4m ago 26m 71.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:34:11.280 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (26m) 5m ago 26m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:34:11.280 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (25m) 4m ago 25m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:34:11.280 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (17m) 5m ago 25m 290M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:34:11.280 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (16m) 5m ago 25m 317M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:34:11.281 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (16m) 5m ago 24m 229M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:34:11.281 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (15m) 4m ago 24m 329M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:34:11.281 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (15m) 4m ago 24m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:34:11.281 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (14m) 4m ago 23m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:34:11.282 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (25m) 5m ago 26m 48.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:34:11.283 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.284+0000 7f51a9709700 1 -- 172.21.15.27:0/2427126772 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f51900640b0 msgr2=0x7f5190066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:34:11.283 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.284+0000 7f51a9709700 1 --2- 172.21.15.27:0/2427126772 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f51900640b0 0x7f5190066570 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f518c004e30 tx=0x7f518c009b30).stop 2021-11-03T17:34:11.283 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.284+0000 7f51a9709700 1 -- 172.21.15.27:0/2427126772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a40f5ef0 msgr2=0x7f51a410c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:34:11.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.284+0000 7f51a9709700 1 --2- 172.21.15.27:0/2427126772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a40f5ef0 0x7f51a410c190 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f51940026d0 tx=0x7f5194009de0).stop 2021-11-03T17:34:11.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.284+0000 7f51a9709700 1 -- 172.21.15.27:0/2427126772 shutdown_connections 2021-11-03T17:34:11.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.284+0000 7f51a9709700 1 --2- 172.21.15.27:0/2427126772 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f51900640b0 0x7f5190066570 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:11.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.284+0000 7f51a9709700 1 --2- 172.21.15.27:0/2427126772 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51a40f55e0 0x7f51a4109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:34:11.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.284+0000 7f51a9709700 1 --2- 172.21.15.27:0/2427126772 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51a40f5ef0 0x7f51a410c190 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:11.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.284+0000 7f51a9709700 1 -- 172.21.15.27:0/2427126772 >> 172.21.15.27:0/2427126772 conn(0x7f51a40f0de0 msgr2=0x7f51a40fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:34:11.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.284+0000 7f51a9709700 1 -- 172.21.15.27:0/2427126772 shutdown_connections 2021-11-03T17:34:11.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.284+0000 7f51a9709700 1 -- 172.21.15.27:0/2427126772 wait complete. 2021-11-03T17:34:11.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.383+0000 7ff975cf5700 1 -- 172.21.15.27:0/1434764476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9700f5ef0 msgr2=0x7ff9700f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:34:11.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.383+0000 7ff975cf5700 1 --2- 172.21.15.27:0/1434764476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9700f5ef0 0x7ff9700f6920 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7ff964004660 tx=0x7ff964009b30).stop 2021-11-03T17:34:11.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.384+0000 7ff975cf5700 1 -- 172.21.15.27:0/1434764476 shutdown_connections 2021-11-03T17:34:11.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.384+0000 7ff975cf5700 1 --2- 172.21.15.27:0/1434764476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9700f5ef0 0x7ff9700f6920 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:11.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.384+0000 7ff975cf5700 1 --2- 172.21.15.27:0/1434764476 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9700f55e0 0x7ff9700f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:11.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.384+0000 7ff975cf5700 1 -- 172.21.15.27:0/1434764476 >> 172.21.15.27:0/1434764476 conn(0x7ff9700f0de0 msgr2=0x7ff9700f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:34:11.382 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.384+0000 7ff975cf5700 1 -- 172.21.15.27:0/1434764476 shutdown_connections 2021-11-03T17:34:11.382 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.384+0000 7ff975cf5700 1 -- 172.21.15.27:0/1434764476 wait complete. 2021-11-03T17:34:11.382 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.384+0000 7ff975cf5700 1 Processor -- start 2021-11-03T17:34:11.382 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.384+0000 7ff975cf5700 1 -- start start 2021-11-03T17:34:11.383 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.384+0000 7ff975cf5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9700f55e0 0x7ff9700fa000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:34:11.383 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.384+0000 7ff975cf5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9700f5ef0 0x7ff9700f8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:34:11.383 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.384+0000 7ff975cf5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9700f89e0 con 0x7ff9700f55e0 2021-11-03T17:34:11.383 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.384+0000 7ff975cf5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9700f8b20 con 0x7ff9700f5ef0 2021-11-03T17:34:11.384 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.385+0000 7ff96f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9700f55e0 0x7ff9700fa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:34:11.384 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.385+0000 7ff96f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9700f55e0 0x7ff9700fa000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58186/0 (socket says 172.21.15.27:58186) 2021-11-03T17:34:11.384 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.385+0000 7ff96f7fe700 1 -- 172.21.15.27:0/1349341468 learned_addr learned my addr 172.21.15.27:0/1349341468 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:34:11.384 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.385+0000 7ff96effd700 1 --2- 172.21.15.27:0/1349341468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9700f5ef0 0x7ff9700f8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:34:11.385 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.385+0000 7ff96f7fe700 1 -- 172.21.15.27:0/1349341468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9700f5ef0 msgr2=0x7ff9700f8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:34:11.385 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.385+0000 7ff96f7fe700 1 --2- 172.21.15.27:0/1349341468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9700f5ef0 0x7ff9700f8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:11.385 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.385+0000 7ff96f7fe700 1 -- 172.21.15.27:0/1349341468 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff964005040 con 0x7ff9700f55e0 2021-11-03T17:34:11.386 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.385+0000 7ff96effd700 1 --2- 172.21.15.27:0/1349341468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9700f5ef0 0x7ff9700f8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:34:11.386 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.385+0000 7ff96f7fe700 1 --2- 172.21.15.27:0/1349341468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9700f55e0 0x7ff9700fa000 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7ff96000c790 tx=0x7ff960007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:34:11.386 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.385+0000 7ff957fff700 1 -- 172.21.15.27:0/1349341468 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff96000e680 con 0x7ff9700f55e0 2021-11-03T17:34:11.386 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.385+0000 7ff975cf5700 1 -- 172.21.15.27:0/1349341468 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9700f8e00 con 0x7ff9700f55e0 2021-11-03T17:34:11.387 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.385+0000 7ff957fff700 1 -- 172.21.15.27:0/1349341468 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7ff96000ec90 con 0x7ff9700f55e0 2021-11-03T17:34:11.387 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.385+0000 7ff957fff700 1 -- 172.21.15.27:0/1349341468 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff960013d60 con 0x7ff9700f55e0 2021-11-03T17:34:11.387 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.385+0000 7ff975cf5700 1 -- 172.21.15.27:0/1349341468 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff97010efa0 con 0x7ff9700f55e0 2021-11-03T17:34:11.389 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.386+0000 7ff975cf5700 1 -- 172.21.15.27:0/1349341468 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff970045ad0 con 0x7ff9700f55e0 2021-11-03T17:34:11.389 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.387+0000 7ff957fff700 1 -- 172.21.15.27:0/1349341468 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7ff96001a060 con 0x7ff9700f55e0 2021-11-03T17:34:11.389 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.387+0000 7ff957fff700 1 --2- 172.21.15.27:0/1349341468 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff958064090 0x7ff958066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:34:11.389 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.387+0000 7ff957fff700 1 -- 172.21.15.27:0/1349341468 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7ff960080b60 con 0x7ff9700f55e0 2021-11-03T17:34:11.390 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.387+0000 7ff96effd700 1 --2- 172.21.15.27:0/1349341468 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff958064090 0x7ff958066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:34:11.390 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.387+0000 7ff96effd700 1 --2- 172.21.15.27:0/1349341468 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff958064090 0x7ff958066550 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7ff964004e30 tx=0x7ff964009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:34:11.390 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.391+0000 7ff957fff700 1 -- 172.21.15.27:0/1349341468 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7ff960052d70 con 0x7ff9700f55e0 2021-11-03T17:34:11.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:11 smithi115 ceph-mon[65453]: pgmap v586: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:11.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:11 smithi027 ceph-mon[97244]: pgmap v586: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:11.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.621+0000 7ff975cf5700 1 -- 172.21.15.27:0/1349341468 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff97010f650 con 0x7ff9700f55e0 2021-11-03T17:34:11.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.622+0000 7ff957fff700 1 -- 172.21.15.27:0/1349341468 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7ff960018070 con 0x7ff9700f55e0 2021-11-03T17:34:11.617 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:34:11.617 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:34:11.617 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:34:11.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:34:11.618 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:34:11.618 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:34:11.618 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:34:11.618 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:34:11.619 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:34:11.619 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:34:11.619 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:34:11.619 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:34:11.620 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:34:11.620 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:34:11.620 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:34:11.620 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:34:11.621 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:34:11.621 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:34:11.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.623+0000 7ff975cf5700 1 -- 172.21.15.27:0/1349341468 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff958064090 msgr2=0x7ff958066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:34:11.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.624+0000 7ff975cf5700 1 --2- 172.21.15.27:0/1349341468 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff958064090 0x7ff958066550 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 rx=0x7ff964004e30 tx=0x7ff964009b30).stop 2021-11-03T17:34:11.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.624+0000 7ff975cf5700 1 -- 172.21.15.27:0/1349341468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9700f55e0 msgr2=0x7ff9700fa000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:34:11.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.624+0000 7ff975cf5700 1 --2- 172.21.15.27:0/1349341468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9700f55e0 0x7ff9700fa000 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7ff96000c790 tx=0x7ff960007d70).stop 2021-11-03T17:34:11.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.624+0000 7ff975cf5700 1 -- 172.21.15.27:0/1349341468 shutdown_connections 2021-11-03T17:34:11.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.624+0000 7ff975cf5700 1 --2- 172.21.15.27:0/1349341468 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff958064090 0x7ff958066550 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:11.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.624+0000 7ff975cf5700 1 --2- 172.21.15.27:0/1349341468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9700f55e0 0x7ff9700fa000 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:11.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.624+0000 7ff975cf5700 1 --2- 172.21.15.27:0/1349341468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9700f5ef0 0x7ff9700f8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:11.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.624+0000 7ff975cf5700 1 -- 172.21.15.27:0/1349341468 >> 172.21.15.27:0/1349341468 conn(0x7ff9700f0de0 msgr2=0x7ff97005f740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:34:11.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.624+0000 7ff975cf5700 1 -- 172.21.15.27:0/1349341468 shutdown_connections 2021-11-03T17:34:11.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.624+0000 7ff975cf5700 1 -- 172.21.15.27:0/1349341468 wait complete. 2021-11-03T17:34:11.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.724+0000 7f35de37c700 1 -- 172.21.15.27:0/46521008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35d80f7980 msgr2=0x7f35d80f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:34:11.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.724+0000 7f35de37c700 1 --2- 172.21.15.27:0/46521008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35d80f7980 0x7f35d80f7de0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f35cc004660 tx=0x7f35cc009b30).stop 2021-11-03T17:34:11.721 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.724+0000 7f35de37c700 1 -- 172.21.15.27:0/46521008 shutdown_connections 2021-11-03T17:34:11.721 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.724+0000 7f35de37c700 1 --2- 172.21.15.27:0/46521008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35d80f7980 0x7f35d80f7de0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:11.721 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.724+0000 7f35de37c700 1 --2- 172.21.15.27:0/46521008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35d80f66e0 0x7f35d80f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:11.721 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.724+0000 7f35de37c700 1 -- 172.21.15.27:0/46521008 >> 172.21.15.27:0/46521008 conn(0x7f35d80f1e40 msgr2=0x7f35d80f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:34:11.721 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.724+0000 7f35de37c700 1 -- 172.21.15.27:0/46521008 shutdown_connections 2021-11-03T17:34:11.722 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.724+0000 7f35de37c700 1 -- 172.21.15.27:0/46521008 wait complete. 2021-11-03T17:34:11.722 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.724+0000 7f35de37c700 1 Processor -- start 2021-11-03T17:34:11.722 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.725+0000 7f35de37c700 1 -- start start 2021-11-03T17:34:11.722 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.725+0000 7f35de37c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35d80f66e0 0x7f35d806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:34:11.723 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.725+0000 7f35de37c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35d80f7980 0x7f35d8069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:34:11.723 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.725+0000 7f35de37c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d806a380 con 0x7f35d80f66e0 2021-11-03T17:34:11.723 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.725+0000 7f35de37c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35d806a4c0 con 0x7f35d80f7980 2021-11-03T17:34:11.723 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.725+0000 7f35d7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35d80f66e0 0x7f35d806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:34:11.724 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.725+0000 7f35d7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35d80f66e0 0x7f35d806b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58196/0 (socket says 172.21.15.27:58196) 2021-11-03T17:34:11.724 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.725+0000 7f35d7fff700 1 -- 172.21.15.27:0/892942031 learned_addr learned my addr 172.21.15.27:0/892942031 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:34:11.724 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.725+0000 7f35d77fe700 1 --2- 172.21.15.27:0/892942031 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35d80f7980 0x7f35d8069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:34:11.724 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.725+0000 7f35d7fff700 1 -- 172.21.15.27:0/892942031 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35d80f7980 msgr2=0x7f35d8069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:34:11.725 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.725+0000 7f35d7fff700 1 --2- 172.21.15.27:0/892942031 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35d80f7980 0x7f35d8069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:11.725 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.725+0000 7f35d7fff700 1 -- 172.21.15.27:0/892942031 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35cc005040 con 0x7f35d80f66e0 2021-11-03T17:34:11.725 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.725+0000 7f35d7fff700 1 --2- 172.21.15.27:0/892942031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35d80f66e0 0x7f35d806b9a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f35c800c790 tx=0x7f35c8007d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:34:11.725 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.726+0000 7f35d4ff9700 1 -- 172.21.15.27:0/892942031 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35c8013630 con 0x7f35d80f66e0 2021-11-03T17:34:11.725 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.726+0000 7f35de37c700 1 -- 172.21.15.27:0/892942031 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35d806a7a0 con 0x7f35d80f66e0 2021-11-03T17:34:11.726 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.726+0000 7f35d4ff9700 1 -- 172.21.15.27:0/892942031 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f35c8012860 con 0x7f35d80f66e0 2021-11-03T17:34:11.726 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.726+0000 7f35d4ff9700 1 -- 172.21.15.27:0/892942031 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35c8002780 con 0x7f35d80f66e0 2021-11-03T17:34:11.726 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.726+0000 7f35de37c700 1 -- 172.21.15.27:0/892942031 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35d810ef90 con 0x7f35d80f66e0 2021-11-03T17:34:11.727 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.727+0000 7f35de37c700 1 -- 172.21.15.27:0/892942031 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35d8045ad0 con 0x7f35d80f66e0 2021-11-03T17:34:11.728 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.727+0000 7f35d4ff9700 1 -- 172.21.15.27:0/892942031 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f35c8010070 con 0x7f35d80f66e0 2021-11-03T17:34:11.728 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.727+0000 7f35d4ff9700 1 --2- 172.21.15.27:0/892942031 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f35c00640b0 0x7f35c0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:34:11.728 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.728+0000 7f35d4ff9700 1 -- 172.21.15.27:0/892942031 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f35c80809c0 con 0x7f35d80f66e0 2021-11-03T17:34:11.728 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.728+0000 7f35d77fe700 1 --2- 172.21.15.27:0/892942031 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f35c00640b0 0x7f35c0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:34:11.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.728+0000 7f35d77fe700 1 --2- 172.21.15.27:0/892942031 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f35c00640b0 0x7f35c0066570 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f35cc004630 tx=0x7f35cc009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:34:11.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.731+0000 7f35d4ff9700 1 -- 172.21.15.27:0/892942031 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f35c8052b70 con 0x7f35d80f66e0 2021-11-03T17:34:11.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.940+0000 7f35de37c700 1 -- 172.21.15.27:0/892942031 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f35d80fef40 con 0x7f35d80f66e0 2021-11-03T17:34:11.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.940+0000 7f35d4ff9700 1 -- 172.21.15.27:0/892942031 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f35c8052760 con 0x7f35d80f66e0 2021-11-03T17:34:11.935 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:34:11.935 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:34:11.935 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:34:11.936 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:34:11.936 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:34:11.936 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:34:11.936 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:34:11.937 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:34:11.937 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:34:11.937 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:34:11.937 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:34:11.937 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:34:11.938 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:34:11.938 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:34:11.938 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:34:11.938 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:34:11.939 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:34:11.939 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:34:11.939 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:34:11.939 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:34:11.939 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:34:11.940 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:34:11.940 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:34:11.940 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:34:11.940 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:34:11.941 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:34:11.941 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:34:11.941 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:34:11.941 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:34:11.942 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:34:11.942 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:34:11.942 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:34:11.942 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:34:11.943 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:34:11.943 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:34:11.943 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:34:11.943 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:34:11.944 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:34:11.944 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:34:11.945 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.942+0000 7f35de37c700 1 -- 172.21.15.27:0/892942031 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f35c00640b0 msgr2=0x7f35c0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:34:11.945 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.942+0000 7f35de37c700 1 --2- 172.21.15.27:0/892942031 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f35c00640b0 0x7f35c0066570 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7f35cc004630 tx=0x7f35cc009b30).stop 2021-11-03T17:34:11.946 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.942+0000 7f35de37c700 1 -- 172.21.15.27:0/892942031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35d80f66e0 msgr2=0x7f35d806b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:34:11.946 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.942+0000 7f35de37c700 1 --2- 172.21.15.27:0/892942031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35d80f66e0 0x7f35d806b9a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7f35c800c790 tx=0x7f35c8007d90).stop 2021-11-03T17:34:11.946 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.942+0000 7f35de37c700 1 -- 172.21.15.27:0/892942031 shutdown_connections 2021-11-03T17:34:11.946 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.942+0000 7f35de37c700 1 --2- 172.21.15.27:0/892942031 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f35c00640b0 0x7f35c0066570 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:11.947 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.942+0000 7f35de37c700 1 --2- 172.21.15.27:0/892942031 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35d80f66e0 0x7f35d806b9a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:11.947 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.942+0000 7f35de37c700 1 --2- 172.21.15.27:0/892942031 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35d80f7980 0x7f35d8069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:11.947 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.942+0000 7f35de37c700 1 -- 172.21.15.27:0/892942031 >> 172.21.15.27:0/892942031 conn(0x7f35d80f1e40 msgr2=0x7f35d80fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:34:11.947 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.942+0000 7f35de37c700 1 -- 172.21.15.27:0/892942031 shutdown_connections 2021-11-03T17:34:11.947 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:11.942+0000 7f35de37c700 1 -- 172.21.15.27:0/892942031 wait complete. 2021-11-03T17:34:11.948 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:34:12.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:12 smithi115 ceph-mon[65453]: from='client.34712 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:34:12.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:12 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1349341468' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:12.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:34:12.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:34:12.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:34:12.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:12.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:12.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:12.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:12.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:12 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/892942031' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:34:12.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:12 smithi027 ceph-mon[97244]: from='client.34712 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:34:12.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:12 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1349341468' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:12.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:34:12.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:34:12.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:34:12.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:12.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:12.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:12.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:12.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:12 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/892942031' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:34:13.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:13 smithi027 ceph-mon[97244]: from='client.34716 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:34:13.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:13 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:34:13.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:13 smithi027 ceph-mon[97244]: pgmap v587: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:13.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:13 smithi115 ceph-mon[65453]: from='client.34716 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:34:13.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:13 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:34:13.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:13 smithi115 ceph-mon[65453]: pgmap v587: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:14.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:14 smithi115 ceph-mon[65453]: pgmap v588: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:14.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:14 smithi027 ceph-mon[97244]: pgmap v588: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:15.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1097.384971 secs 2021-11-03T17:34:15.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1097.384971 secs 2021-11-03T17:34:16.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:16 smithi115 ceph-mon[65453]: pgmap v589: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:16.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:16 smithi027 ceph-mon[97244]: pgmap v589: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:19.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:19 smithi115 ceph-mon[65453]: pgmap v590: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:19.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:19 smithi027 ceph-mon[97244]: pgmap v590: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:20.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1102.385087 secs 2021-11-03T17:34:20.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1102.385087 secs 2021-11-03T17:34:21.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:21 smithi115 ceph-mon[65453]: pgmap v591: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:21.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:21 smithi027 ceph-mon[97244]: pgmap v591: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:22.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:34:22.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:34:22.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:34:22.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:22.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:22.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:22.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:22.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:34:22.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:34:22.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:34:22.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:22.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:22.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:22.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:23.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:23 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:34:23.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:23 smithi027 ceph-mon[97244]: pgmap v592: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:23.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:23 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:34:23.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:23 smithi115 ceph-mon[65453]: pgmap v592: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:24.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:24 smithi115 ceph-mon[65453]: pgmap v593: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:24.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:24 smithi027 ceph-mon[97244]: pgmap v593: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:25.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1107.385210 secs 2021-11-03T17:34:25.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1107.385210 secs 2021-11-03T17:34:26.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:26 smithi115 ceph-mon[65453]: pgmap v594: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:26.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:26 smithi027 ceph-mon[97244]: pgmap v594: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:29.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:29 smithi115 ceph-mon[65453]: pgmap v595: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:29.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:29 smithi027 ceph-mon[97244]: pgmap v595: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:30.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1112.385347 secs 2021-11-03T17:34:30.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1112.385347 secs 2021-11-03T17:34:31.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:31 smithi115 ceph-mon[65453]: pgmap v596: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:31.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:31 smithi027 ceph-mon[97244]: pgmap v596: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:32.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:34:32.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:34:32.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:34:32.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:32.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:32.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:32.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:32.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:34:32.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:34:32.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:34:32.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:32.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:32.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:32.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:33.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:33 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:34:33.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:33 smithi027 ceph-mon[97244]: pgmap v597: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:33.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:33 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:34:33.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:33 smithi115 ceph-mon[65453]: pgmap v597: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:34.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:34 smithi115 ceph-mon[65453]: pgmap v598: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:34.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:34 smithi027 ceph-mon[97244]: pgmap v598: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:35.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1117.385473 secs 2021-11-03T17:34:35.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1117.385473 secs 2021-11-03T17:34:36.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:36 smithi115 ceph-mon[65453]: pgmap v599: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:36.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:36 smithi027 ceph-mon[97244]: pgmap v599: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:39.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:39 smithi115 ceph-mon[65453]: pgmap v600: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:39.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:39 smithi027 ceph-mon[97244]: pgmap v600: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:40.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1122.385506 secs 2021-11-03T17:34:40.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1122.385506 secs 2021-11-03T17:34:41.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:41 smithi115 ceph-mon[65453]: pgmap v601: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:41.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:41 smithi027 ceph-mon[97244]: pgmap v601: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:42.055 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.059+0000 7fcd774c3700 1 -- 172.21.15.27:0/747419668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd70060310 msgr2=0x7fcd700f7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:34:42.055 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.059+0000 7fcd774c3700 1 --2- 172.21.15.27:0/747419668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd70060310 0x7fcd700f7090 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7fcd680099e0 tx=0x7fcd68004d10).stop 2021-11-03T17:34:42.056 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.059+0000 7fcd774c3700 1 -- 172.21.15.27:0/747419668 shutdown_connections 2021-11-03T17:34:42.056 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.059+0000 7fcd774c3700 1 --2- 172.21.15.27:0/747419668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd700f7610 0x7fcd700f9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:42.056 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.059+0000 7fcd774c3700 1 --2- 172.21.15.27:0/747419668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd70060310 0x7fcd700f7090 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:42.056 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.059+0000 7fcd774c3700 1 -- 172.21.15.27:0/747419668 >> 172.21.15.27:0/747419668 conn(0x7fcd700f0d40 msgr2=0x7fcd700f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:34:42.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.059+0000 7fcd774c3700 1 -- 172.21.15.27:0/747419668 shutdown_connections 2021-11-03T17:34:42.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.059+0000 7fcd774c3700 1 -- 172.21.15.27:0/747419668 wait complete. 2021-11-03T17:34:42.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.060+0000 7fcd774c3700 1 Processor -- start 2021-11-03T17:34:42.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.060+0000 7fcd774c3700 1 -- start start 2021-11-03T17:34:42.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.060+0000 7fcd774c3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd70060310 0x7fcd7010d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:34:42.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.060+0000 7fcd774c3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd700f7610 0x7fcd7010db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:34:42.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.060+0000 7fcd774c3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7010e0c0 con 0x7fcd70060310 2021-11-03T17:34:42.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.060+0000 7fcd774c3700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd7010e200 con 0x7fcd700f7610 2021-11-03T17:34:42.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.060+0000 7fcd7525f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd70060310 0x7fcd7010d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:34:42.062 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.060+0000 7fcd7525f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd70060310 0x7fcd7010d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58212/0 (socket says 172.21.15.27:58212) 2021-11-03T17:34:42.062 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.060+0000 7fcd7525f700 1 -- 172.21.15.27:0/2788377476 learned_addr learned my addr 172.21.15.27:0/2788377476 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:34:42.062 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.060+0000 7fcd7525f700 1 -- 172.21.15.27:0/2788377476 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd700f7610 msgr2=0x7fcd7010db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:34:42.063 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.060+0000 7fcd7525f700 1 --2- 172.21.15.27:0/2788377476 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd700f7610 0x7fcd7010db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:34:42.063 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.060+0000 7fcd7525f700 1 -- 172.21.15.27:0/2788377476 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd68009640 con 0x7fcd70060310 2021-11-03T17:34:42.063 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.061+0000 7fcd7525f700 1 --2- 172.21.15.27:0/2788377476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd70060310 0x7fcd7010d620 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7fcd6800c010 tx=0x7fcd68004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:34:42.064 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.061+0000 7fcd6dffb700 1 -- 172.21.15.27:0/2788377476 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd6800e070 con 0x7fcd70060310 2021-11-03T17:34:42.064 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.061+0000 7fcd6dffb700 1 -- 172.21.15.27:0/2788377476 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fcd6801d8d0 con 0x7fcd70060310 2021-11-03T17:34:42.064 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.061+0000 7fcd774c3700 1 -- 172.21.15.27:0/2788377476 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd7010eb40 con 0x7fcd70060310 2021-11-03T17:34:42.064 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.061+0000 7fcd6dffb700 1 -- 172.21.15.27:0/2788377476 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd68018070 con 0x7fcd70060310 2021-11-03T17:34:42.065 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.061+0000 7fcd774c3700 1 -- 172.21.15.27:0/2788377476 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd7010f020 con 0x7fcd70060310 2021-11-03T17:34:42.066 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.062+0000 7fcd774c3700 1 -- 172.21.15.27:0/2788377476 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd70045ad0 con 0x7fcd70060310 2021-11-03T17:34:42.066 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.062+0000 7fcd6dffb700 1 -- 172.21.15.27:0/2788377476 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fcd68024060 con 0x7fcd70060310 2021-11-03T17:34:42.067 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.063+0000 7fcd6dffb700 1 --2- 172.21.15.27:0/2788377476 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcd58064090 0x7fcd58066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:34:42.067 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.063+0000 7fcd6dffb700 1 -- 172.21.15.27:0/2788377476 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fcd680821d0 con 0x7fcd70060310 2021-11-03T17:34:42.067 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.063+0000 7fcd74a5e700 1 --2- 172.21.15.27:0/2788377476 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcd58064090 0x7fcd58066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:34:42.067 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.063+0000 7fcd74a5e700 1 --2- 172.21.15.27:0/2788377476 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcd58064090 0x7fcd58066550 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7fcd64004870 tx=0x7fcd640079c0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:34:42.068 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.067+0000 7fcd6dffb700 1 -- 172.21.15.27:0/2788377476 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fcd680544d0 con 0x7fcd70060310 2021-11-03T17:34:42.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.251+0000 7fcd774c3700 1 -- 172.21.15.27:0/2788377476 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd701055b0 con 0x7fcd58064090 2021-11-03T17:34:42.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.252+0000 7fcd6dffb700 1 -- 172.21.15.27:0/2788377476 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fcd701055b0 con 0x7fcd58064090 2021-11-03T17:34:42.249 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.254+0000 7fcd774c3700 1 -- 172.21.15.27:0/2788377476 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcd58064090 msgr2=0x7fcd58066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:34:42.249 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.254+0000 7fcd774c3700 1 --2- 172.21.15.27:0/2788377476 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcd58064090 0x7fcd58066550 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7fcd64004870 tx=0x7fcd640079c0).stop 2021-11-03T17:34:42.250 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.254+0000 7fcd774c3700 1 -- 172.21.15.27:0/2788377476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd70060310 msgr2=0x7fcd7010d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:34:42.250 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.254+0000 7fcd774c3700 1 --2- 172.21.15.27:0/2788377476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd70060310 0x7fcd7010d620 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7fcd6800c010 tx=0x7fcd68004d10).stop 2021-11-03T17:34:42.251 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.254+0000 7fcd774c3700 1 -- 172.21.15.27:0/2788377476 shutdown_connections 2021-11-03T17:34:42.251 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.254+0000 7fcd774c3700 1 --2- 172.21.15.27:0/2788377476 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcd58064090 0x7fcd58066550 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:42.252 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.254+0000 7fcd774c3700 1 --2- 172.21.15.27:0/2788377476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd70060310 0x7fcd7010d620 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:42.253 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.254+0000 7fcd774c3700 1 --2- 172.21.15.27:0/2788377476 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd700f7610 0x7fcd7010db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:34:42.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.254+0000 7fcd774c3700 1 -- 172.21.15.27:0/2788377476 >> 172.21.15.27:0/2788377476 conn(0x7fcd700f0d40 msgr2=0x7fcd700f3130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:34:42.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.254+0000 7fcd774c3700 1 -- 172.21.15.27:0/2788377476 shutdown_connections 2021-11-03T17:34:42.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.254+0000 7fcd774c3700 1 -- 172.21.15.27:0/2788377476 wait complete. 2021-11-03T17:34:42.258 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:34:42.350 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.356+0000 7f320b59e700 1 -- 172.21.15.27:0/2682473022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f320c0f66e0 msgr2=0x7f320c0f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:34:42.350 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.356+0000 7f320b59e700 1 --2- 172.21.15.27:0/2682473022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f320c0f66e0 0x7f320c0f6b40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7f31f4004660 tx=0x7f31f4009b30).stop 2021-11-03T17:34:42.351 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.356+0000 7f320b59e700 1 -- 172.21.15.27:0/2682473022 shutdown_connections 2021-11-03T17:34:42.351 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.356+0000 7f320b59e700 1 --2- 172.21.15.27:0/2682473022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f320c0f66e0 0x7f320c0f6b40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:42.351 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.356+0000 7f320b59e700 1 --2- 172.21.15.27:0/2682473022 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f320c0f5440 0x7f320c0f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:42.352 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.356+0000 7f320b59e700 1 -- 172.21.15.27:0/2682473022 >> 172.21.15.27:0/2682473022 conn(0x7f320c0f0ba0 msgr2=0x7f320c0f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:34:42.352 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.356+0000 7f320b59e700 1 -- 172.21.15.27:0/2682473022 shutdown_connections 2021-11-03T17:34:42.352 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.356+0000 7f320b59e700 1 -- 172.21.15.27:0/2682473022 wait complete. 2021-11-03T17:34:42.355 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.356+0000 7f320b59e700 1 Processor -- start 2021-11-03T17:34:42.355 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.357+0000 7f320b59e700 1 -- start start 2021-11-03T17:34:42.355 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.357+0000 7f320b59e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f320c0f5440 0x7f320c0f95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:34:42.356 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.357+0000 7f320b59e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f320c0f66e0 0x7f320c0f7a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:34:42.356 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.357+0000 7f320b59e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320c0f7f80 con 0x7f320c0f66e0 2021-11-03T17:34:42.356 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.357+0000 7f320b59e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f320c0f80c0 con 0x7f320c0f5440 2021-11-03T17:34:42.356 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.357+0000 7f3203fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f320c0f66e0 0x7f320c0f7a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:34:42.357 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.357+0000 7f3203fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f320c0f66e0 0x7f320c0f7a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58224/0 (socket says 172.21.15.27:58224) 2021-11-03T17:34:42.357 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.357+0000 7f3203fff700 1 -- 172.21.15.27:0/3867571858 learned_addr learned my addr 172.21.15.27:0/3867571858 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:34:42.357 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.357+0000 7f320a59c700 1 --2- 172.21.15.27:0/3867571858 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f320c0f5440 0x7f320c0f95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:34:42.357 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.357+0000 7f3203fff700 1 -- 172.21.15.27:0/3867571858 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f320c0f5440 msgr2=0x7f320c0f95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:34:42.358 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.357+0000 7f3203fff700 1 --2- 172.21.15.27:0/3867571858 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f320c0f5440 0x7f320c0f95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:42.358 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.357+0000 7f3203fff700 1 -- 172.21.15.27:0/3867571858 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f4005040 con 0x7f320c0f66e0 2021-11-03T17:34:42.358 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.357+0000 7f320a59c700 1 --2- 172.21.15.27:0/3867571858 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f320c0f5440 0x7f320c0f95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:34:42.358 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.357+0000 7f3203fff700 1 --2- 172.21.15.27:0/3867571858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f320c0f66e0 0x7f320c0f7a20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f31f4009990 tx=0x7f31f4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:34:42.358 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.358+0000 7f3202ffd700 1 -- 172.21.15.27:0/3867571858 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31f4018070 con 0x7f320c0f66e0 2021-11-03T17:34:42.359 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.358+0000 7f320b59e700 1 -- 172.21.15.27:0/3867571858 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f320c0f8340 con 0x7f320c0f66e0 2021-11-03T17:34:42.359 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.358+0000 7f3202ffd700 1 -- 172.21.15.27:0/3867571858 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f31f4007ba0 con 0x7f320c0f66e0 2021-11-03T17:34:42.359 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.358+0000 7f3202ffd700 1 -- 172.21.15.27:0/3867571858 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31f401c3c0 con 0x7f320c0f66e0 2021-11-03T17:34:42.359 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.358+0000 7f320b59e700 1 -- 172.21.15.27:0/3867571858 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f320c0f88a0 con 0x7f320c0f66e0 2021-11-03T17:34:42.360 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.359+0000 7f320b59e700 1 -- 172.21.15.27:0/3867571858 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f320c045ad0 con 0x7f320c0f66e0 2021-11-03T17:34:42.360 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.359+0000 7f3202ffd700 1 -- 172.21.15.27:0/3867571858 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f31f401d420 con 0x7f320c0f66e0 2021-11-03T17:34:42.360 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.359+0000 7f3202ffd700 1 --2- 172.21.15.27:0/3867571858 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f31f8064100 0x7f31f80665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:34:42.360 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.360+0000 7f3202ffd700 1 -- 172.21.15.27:0/3867571858 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f31f4082630 con 0x7f320c0f66e0 2021-11-03T17:34:42.361 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.360+0000 7f320a59c700 1 --2- 172.21.15.27:0/3867571858 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f31f8064100 0x7f31f80665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:34:42.361 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.360+0000 7f320a59c700 1 --2- 172.21.15.27:0/3867571858 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f31f8064100 0x7f31f80665c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f31fc009880 tx=0x7f31fc00b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:34:42.361 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.364+0000 7f3202ffd700 1 -- 172.21.15.27:0/3867571858 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f31f4054990 con 0x7f320c0f66e0 2021-11-03T17:34:42.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:34:42.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:34:42.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:34:42.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:42.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:42.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:42.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:42.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:42 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:34:42.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:42 smithi115 ceph-mon[65453]: pgmap v602: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:42.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:34:42.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:34:42.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:34:42.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:42.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:42.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:42.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:42.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:42 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:34:42.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:42 smithi027 ceph-mon[97244]: pgmap v602: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:42.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.535+0000 7f320b59e700 1 -- 172.21.15.27:0/3867571858 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f320c0fab20 con 0x7f31f8064100 2021-11-03T17:34:42.544 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.550+0000 7f3202ffd700 1 -- 172.21.15.27:0/3867571858 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f320c0fab20 con 0x7f31f8064100 2021-11-03T17:34:42.544 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:34:42.544 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (26m) 6m ago 27m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:34:42.545 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (17m) 6m ago 27m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:34:42.545 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (17m) 5m ago 26m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:34:42.545 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (26m) 6m ago 27m 40.2M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:34:42.545 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (19m) 6m ago 22m 18.2M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:34:42.546 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (22m) 6m ago 22m 3841M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:34:42.546 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (18m) 5m ago 22m 17.6M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:34:42.546 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (20m) 5m ago 22m 19.3M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:34:42.546 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (19m) 6m ago 28m 497M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:34:42.547 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (19m) 5m ago 26m 415M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:34:42.547 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (18m) 6m ago 28m 119M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:34:42.547 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (18m) 5m ago 26m 71.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:34:42.547 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (27m) 6m ago 27m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:34:42.547 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (26m) 5m ago 26m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:34:42.548 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (17m) 6m ago 25m 290M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:34:42.548 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (17m) 6m ago 25m 317M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:34:42.548 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (16m) 6m ago 25m 229M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:34:42.548 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (16m) 5m ago 24m 329M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:34:42.549 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (15m) 5m ago 24m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:34:42.549 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (15m) 5m ago 24m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:34:42.549 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (26m) 6m ago 26m 48.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:34:42.550 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.551+0000 7f320b59e700 1 -- 172.21.15.27:0/3867571858 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f31f8064100 msgr2=0x7f31f80665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:34:42.550 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.552+0000 7f320b59e700 1 --2- 172.21.15.27:0/3867571858 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f31f8064100 0x7f31f80665c0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f31fc009880 tx=0x7f31fc00b040).stop 2021-11-03T17:34:42.550 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.552+0000 7f320b59e700 1 -- 172.21.15.27:0/3867571858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f320c0f66e0 msgr2=0x7f320c0f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:34:42.550 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.552+0000 7f320b59e700 1 --2- 172.21.15.27:0/3867571858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f320c0f66e0 0x7f320c0f7a20 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f31f4009990 tx=0x7f31f4009b30).stop 2021-11-03T17:34:42.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.552+0000 7f320b59e700 1 -- 172.21.15.27:0/3867571858 shutdown_connections 2021-11-03T17:34:42.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.552+0000 7f320b59e700 1 --2- 172.21.15.27:0/3867571858 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f31f8064100 0x7f31f80665c0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:42.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.552+0000 7f320b59e700 1 --2- 172.21.15.27:0/3867571858 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f320c0f5440 0x7f320c0f95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:42.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.552+0000 7f320b59e700 1 --2- 172.21.15.27:0/3867571858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f320c0f66e0 0x7f320c0f7a20 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:42.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.552+0000 7f320b59e700 1 -- 172.21.15.27:0/3867571858 >> 172.21.15.27:0/3867571858 conn(0x7f320c0f0ba0 msgr2=0x7f320c05fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:34:42.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.552+0000 7f320b59e700 1 -- 172.21.15.27:0/3867571858 shutdown_connections 2021-11-03T17:34:42.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.552+0000 7f320b59e700 1 -- 172.21.15.27:0/3867571858 wait complete. 2021-11-03T17:34:42.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.651+0000 7f4d42f53700 1 -- 172.21.15.27:0/1563994688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d3c0f64e0 msgr2=0x7f4d3c0f6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:34:42.648 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.651+0000 7f4d42f53700 1 --2- 172.21.15.27:0/1563994688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d3c0f64e0 0x7f4d3c0f6960 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f4d2c0099e0 tx=0x7f4d2c004d10).stop 2021-11-03T17:34:42.648 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.651+0000 7f4d42f53700 1 -- 172.21.15.27:0/1563994688 shutdown_connections 2021-11-03T17:34:42.648 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.651+0000 7f4d42f53700 1 --2- 172.21.15.27:0/1563994688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d3c0f64e0 0x7f4d3c0f6960 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:42.648 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.651+0000 7f4d42f53700 1 --2- 172.21.15.27:0/1563994688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d3c0f5bd0 0x7f4d3c0f5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:42.649 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.651+0000 7f4d42f53700 1 -- 172.21.15.27:0/1563994688 >> 172.21.15.27:0/1563994688 conn(0x7f4d3c0f0540 msgr2=0x7f4d3c0f2960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:34:42.649 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.651+0000 7f4d42f53700 1 -- 172.21.15.27:0/1563994688 shutdown_connections 2021-11-03T17:34:42.649 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.651+0000 7f4d42f53700 1 -- 172.21.15.27:0/1563994688 wait complete. 2021-11-03T17:34:42.649 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.651+0000 7f4d42f53700 1 Processor -- start 2021-11-03T17:34:42.650 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.651+0000 7f4d42f53700 1 -- start start 2021-11-03T17:34:42.650 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.652+0000 7f4d42f53700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d3c0f5bd0 0x7f4d3c109d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:34:42.650 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.652+0000 7f4d42f53700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d3c0f64e0 0x7f4d3c10c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:34:42.651 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.652+0000 7f4d42f53700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c0f56c0 con 0x7f4d3c0f64e0 2021-11-03T17:34:42.651 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.652+0000 7f4d42f53700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3c0f5800 con 0x7f4d3c0f5bd0 2021-11-03T17:34:42.651 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.652+0000 7f4d3bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d3c0f64e0 0x7f4d3c10c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:34:42.652 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.652+0000 7f4d3bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d3c0f64e0 0x7f4d3c10c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58232/0 (socket says 172.21.15.27:58232) 2021-11-03T17:34:42.652 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.652+0000 7f4d3bfff700 1 -- 172.21.15.27:0/3348111611 learned_addr learned my addr 172.21.15.27:0/3348111611 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:34:42.652 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.652+0000 7f4d40cef700 1 --2- 172.21.15.27:0/3348111611 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d3c0f5bd0 0x7f4d3c109d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:34:42.652 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.652+0000 7f4d3bfff700 1 -- 172.21.15.27:0/3348111611 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d3c0f5bd0 msgr2=0x7f4d3c109d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:34:42.652 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.652+0000 7f4d3bfff700 1 --2- 172.21.15.27:0/3348111611 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d3c0f5bd0 0x7f4d3c109d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:42.653 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.652+0000 7f4d3bfff700 1 -- 172.21.15.27:0/3348111611 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d2c009640 con 0x7f4d3c0f64e0 2021-11-03T17:34:42.653 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.652+0000 7f4d3bfff700 1 --2- 172.21.15.27:0/3348111611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d3c0f64e0 0x7f4d3c10c230 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f4d2c00b3b0 tx=0x7f4d2c004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:34:42.653 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.652+0000 7f4d397fa700 1 -- 172.21.15.27:0/3348111611 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d2c00e030 con 0x7f4d3c0f64e0 2021-11-03T17:34:42.653 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.653+0000 7f4d397fa700 1 -- 172.21.15.27:0/3348111611 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f4d2c0049b0 con 0x7f4d3c0f64e0 2021-11-03T17:34:42.654 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.653+0000 7f4d42f53700 1 -- 172.21.15.27:0/3348111611 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d3c10c780 con 0x7f4d3c0f64e0 2021-11-03T17:34:42.654 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.653+0000 7f4d397fa700 1 -- 172.21.15.27:0/3348111611 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d2c018070 con 0x7f4d3c0f64e0 2021-11-03T17:34:42.654 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.653+0000 7f4d42f53700 1 -- 172.21.15.27:0/3348111611 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d3c10cc60 con 0x7f4d3c0f64e0 2021-11-03T17:34:42.655 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.653+0000 7f4d42f53700 1 -- 172.21.15.27:0/3348111611 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d3c045ad0 con 0x7f4d3c0f64e0 2021-11-03T17:34:42.656 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.654+0000 7f4d397fa700 1 -- 172.21.15.27:0/3348111611 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f4d2c007c90 con 0x7f4d3c0f64e0 2021-11-03T17:34:42.656 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.654+0000 7f4d397fa700 1 --2- 172.21.15.27:0/3348111611 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4d280640b0 0x7f4d28066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:34:42.656 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.654+0000 7f4d397fa700 1 -- 172.21.15.27:0/3348111611 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f4d2c0825a0 con 0x7f4d3c0f64e0 2021-11-03T17:34:42.657 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.654+0000 7f4d40cef700 1 --2- 172.21.15.27:0/3348111611 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4d280640b0 0x7f4d28066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:34:42.657 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.655+0000 7f4d40cef700 1 --2- 172.21.15.27:0/3348111611 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4d280640b0 0x7f4d28066570 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f4d3c0f5340 tx=0x7f4d3400b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:34:42.657 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.658+0000 7f4d397fa700 1 -- 172.21.15.27:0/3348111611 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f4d2c0547e0 con 0x7f4d3c0f64e0 2021-11-03T17:34:42.880 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.886+0000 7f4d42f53700 1 -- 172.21.15.27:0/3348111611 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4d3c027a00 con 0x7f4d3c0f64e0 2021-11-03T17:34:42.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.887+0000 7f4d397fa700 1 -- 172.21.15.27:0/3348111611 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f4d2c0543d0 con 0x7f4d3c0f64e0 2021-11-03T17:34:42.881 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:34:42.882 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:34:42.882 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:34:42.882 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:34:42.882 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:34:42.883 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:34:42.883 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:34:42.883 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:34:42.883 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:34:42.884 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:34:42.884 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:34:42.884 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:34:42.884 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:34:42.884 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:34:42.885 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:34:42.885 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:34:42.885 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:34:42.885 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:34:42.886 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.888+0000 7f4d42f53700 1 -- 172.21.15.27:0/3348111611 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4d280640b0 msgr2=0x7f4d28066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:34:42.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.888+0000 7f4d42f53700 1 --2- 172.21.15.27:0/3348111611 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4d280640b0 0x7f4d28066570 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f4d3c0f5340 tx=0x7f4d3400b040).stop 2021-11-03T17:34:42.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.888+0000 7f4d42f53700 1 -- 172.21.15.27:0/3348111611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d3c0f64e0 msgr2=0x7f4d3c10c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:34:42.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.888+0000 7f4d42f53700 1 --2- 172.21.15.27:0/3348111611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d3c0f64e0 0x7f4d3c10c230 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f4d2c00b3b0 tx=0x7f4d2c004d10).stop 2021-11-03T17:34:42.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.889+0000 7f4d42f53700 1 -- 172.21.15.27:0/3348111611 shutdown_connections 2021-11-03T17:34:42.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.889+0000 7f4d42f53700 1 --2- 172.21.15.27:0/3348111611 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4d280640b0 0x7f4d28066570 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:42.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.889+0000 7f4d42f53700 1 --2- 172.21.15.27:0/3348111611 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d3c0f5bd0 0x7f4d3c109d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:42.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.889+0000 7f4d42f53700 1 --2- 172.21.15.27:0/3348111611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d3c0f64e0 0x7f4d3c10c230 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:42.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.889+0000 7f4d42f53700 1 -- 172.21.15.27:0/3348111611 >> 172.21.15.27:0/3348111611 conn(0x7f4d3c0f0540 msgr2=0x7f4d3c0fa470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:34:42.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.889+0000 7f4d42f53700 1 -- 172.21.15.27:0/3348111611 shutdown_connections 2021-11-03T17:34:42.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.889+0000 7f4d42f53700 1 -- 172.21.15.27:0/3348111611 wait complete. 2021-11-03T17:34:42.986 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.989+0000 7f05dcc67700 1 -- 172.21.15.27:0/1205057268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05d806b960 msgr2=0x7f05d8069e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:34:42.986 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.989+0000 7f05dcc67700 1 --2- 172.21.15.27:0/1205057268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05d806b960 0x7f05d8069e60 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f05c8004660 tx=0x7f05c8009b30).stop 2021-11-03T17:34:42.986 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.990+0000 7f05dcc67700 1 -- 172.21.15.27:0/1205057268 shutdown_connections 2021-11-03T17:34:42.986 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.990+0000 7f05dcc67700 1 --2- 172.21.15.27:0/1205057268 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05d806a350 0x7f05d806a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:42.987 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.990+0000 7f05dcc67700 1 --2- 172.21.15.27:0/1205057268 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05d806b960 0x7f05d8069e60 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:42.987 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.990+0000 7f05dcc67700 1 -- 172.21.15.27:0/1205057268 >> 172.21.15.27:0/1205057268 conn(0x7f05d80f1e20 msgr2=0x7f05d80f4240 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:34:42.987 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.990+0000 7f05dcc67700 1 -- 172.21.15.27:0/1205057268 shutdown_connections 2021-11-03T17:34:42.987 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.990+0000 7f05dcc67700 1 -- 172.21.15.27:0/1205057268 wait complete. 2021-11-03T17:34:42.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.990+0000 7f05dcc67700 1 Processor -- start 2021-11-03T17:34:42.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.990+0000 7f05dcc67700 1 -- start start 2021-11-03T17:34:42.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.990+0000 7f05dcc67700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05d806a350 0x7f05d8109130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:34:42.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.990+0000 7f05dcc67700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05d806b960 0x7f05d8109620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:34:42.989 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.990+0000 7f05dcc67700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d8109bd0 con 0x7f05d806b960 2021-11-03T17:34:42.989 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.990+0000 7f05dcc67700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d8109d10 con 0x7f05d806a350 2021-11-03T17:34:42.989 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.991+0000 7f05d5d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05d806b960 0x7f05d8109620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:34:42.989 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.991+0000 7f05d5d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05d806b960 0x7f05d8109620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58244/0 (socket says 172.21.15.27:58244) 2021-11-03T17:34:42.990 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.991+0000 7f05d5d9b700 1 -- 172.21.15.27:0/3639025470 learned_addr learned my addr 172.21.15.27:0/3639025470 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:34:42.990 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.991+0000 7f05d659c700 1 --2- 172.21.15.27:0/3639025470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05d806a350 0x7f05d8109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:34:42.990 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.991+0000 7f05d5d9b700 1 -- 172.21.15.27:0/3639025470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05d806a350 msgr2=0x7f05d8109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:34:42.990 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.991+0000 7f05d5d9b700 1 --2- 172.21.15.27:0/3639025470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05d806a350 0x7f05d8109130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:42.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.991+0000 7f05d5d9b700 1 -- 172.21.15.27:0/3639025470 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c8005040 con 0x7f05d806b960 2021-11-03T17:34:42.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.991+0000 7f05d5d9b700 1 --2- 172.21.15.27:0/3639025470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05d806b960 0x7f05d8109620 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f05cc0026d0 tx=0x7f05cc009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:34:42.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.991+0000 7f05c6ffd700 1 -- 172.21.15.27:0/3639025470 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05cc0079c0 con 0x7f05d806b960 2021-11-03T17:34:42.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.991+0000 7f05c6ffd700 1 -- 172.21.15.27:0/3639025470 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f05cc00f040 con 0x7f05d806b960 2021-11-03T17:34:42.992 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.991+0000 7f05dcc67700 1 -- 172.21.15.27:0/3639025470 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05d810a6b0 con 0x7f05d806b960 2021-11-03T17:34:42.992 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.991+0000 7f05c6ffd700 1 -- 172.21.15.27:0/3639025470 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05cc0148b0 con 0x7f05d806b960 2021-11-03T17:34:42.992 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.991+0000 7f05dcc67700 1 -- 172.21.15.27:0/3639025470 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05d810ac70 con 0x7f05d806b960 2021-11-03T17:34:42.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.992+0000 7f05dcc67700 1 -- 172.21.15.27:0/3639025470 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05d8045ad0 con 0x7f05d806b960 2021-11-03T17:34:42.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.993+0000 7f05c6ffd700 1 -- 172.21.15.27:0/3639025470 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f05cc007b20 con 0x7f05d806b960 2021-11-03T17:34:42.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.993+0000 7f05c6ffd700 1 --2- 172.21.15.27:0/3639025470 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f05c00640b0 0x7f05c0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:34:42.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.993+0000 7f05c6ffd700 1 -- 172.21.15.27:0/3639025470 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f05cc0807a0 con 0x7f05d806b960 2021-11-03T17:34:42.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.993+0000 7f05d659c700 1 --2- 172.21.15.27:0/3639025470 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f05c00640b0 0x7f05c0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:34:42.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.993+0000 7f05d659c700 1 --2- 172.21.15.27:0/3639025470 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f05c00640b0 0x7f05c0066570 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f05c8004630 tx=0x7f05c8009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:34:42.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:42.997+0000 7f05c6ffd700 1 -- 172.21.15.27:0/3639025470 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f05cc052aa0 con 0x7f05d806b960 2021-11-03T17:34:43.196 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:43.201+0000 7f05dcc67700 1 -- 172.21.15.27:0/3639025470 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f05d810a840 con 0x7f05d806b960 2021-11-03T17:34:43.196 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:43.202+0000 7f05c6ffd700 1 -- 172.21.15.27:0/3639025470 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f05d810a840 con 0x7f05d806b960 2021-11-03T17:34:43.196 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:34:43.197 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:34:43.197 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:34:43.197 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:34:43.197 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:34:43.198 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:34:43.198 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:34:43.198 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:34:43.198 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:34:43.199 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:34:43.199 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:34:43.199 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:34:43.199 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:34:43.200 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:34:43.200 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:34:43.200 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:34:43.200 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:34:43.201 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:34:43.201 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:34:43.201 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:34:43.201 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:34:43.202 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:34:43.202 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:34:43.202 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:34:43.202 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:34:43.203 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:34:43.203 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:34:43.203 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:34:43.203 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:34:43.204 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:34:43.204 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:34:43.204 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:34:43.204 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:34:43.204 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:34:43.205 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:34:43.205 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:34:43.205 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:34:43.205 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:34:43.206 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:34:43.206 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:43.204+0000 7f05dcc67700 1 -- 172.21.15.27:0/3639025470 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f05c00640b0 msgr2=0x7f05c0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:34:43.207 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:43.204+0000 7f05dcc67700 1 --2- 172.21.15.27:0/3639025470 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f05c00640b0 0x7f05c0066570 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7f05c8004630 tx=0x7f05c8009b30).stop 2021-11-03T17:34:43.207 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:43.204+0000 7f05dcc67700 1 -- 172.21.15.27:0/3639025470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05d806b960 msgr2=0x7f05d8109620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:34:43.207 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:43.204+0000 7f05dcc67700 1 --2- 172.21.15.27:0/3639025470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05d806b960 0x7f05d8109620 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7f05cc0026d0 tx=0x7f05cc009de0).stop 2021-11-03T17:34:43.207 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:43.204+0000 7f05dcc67700 1 -- 172.21.15.27:0/3639025470 shutdown_connections 2021-11-03T17:34:43.208 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:43.204+0000 7f05dcc67700 1 --2- 172.21.15.27:0/3639025470 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f05c00640b0 0x7f05c0066570 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:43.208 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:43.204+0000 7f05dcc67700 1 --2- 172.21.15.27:0/3639025470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f05d806a350 0x7f05d8109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:43.208 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:43.204+0000 7f05dcc67700 1 --2- 172.21.15.27:0/3639025470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f05d806b960 0x7f05d8109620 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:34:43.208 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:43.204+0000 7f05dcc67700 1 -- 172.21.15.27:0/3639025470 >> 172.21.15.27:0/3639025470 conn(0x7f05d80f1e20 msgr2=0x7f05d80fce60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:34:43.209 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:43.204+0000 7f05dcc67700 1 -- 172.21.15.27:0/3639025470 shutdown_connections 2021-11-03T17:34:43.209 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:34:43.204+0000 7f05dcc67700 1 -- 172.21.15.27:0/3639025470 wait complete. 2021-11-03T17:34:43.209 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:34:43.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:43 smithi027 ceph-mon[97244]: from='client.34728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:34:43.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:43 smithi027 ceph-mon[97244]: from='client.34732 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:34:43.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:43 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3348111611' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:43.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:43 smithi115 ceph-mon[65453]: from='client.34728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:34:43.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:43 smithi115 ceph-mon[65453]: from='client.34732 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:34:43.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:43 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3348111611' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:44.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:44 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3639025470' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:34:44.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:44 smithi115 ceph-mon[65453]: pgmap v603: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:44.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:44 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3639025470' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:34:44.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:44 smithi027 ceph-mon[97244]: pgmap v603: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:45.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1127.385593 secs 2021-11-03T17:34:45.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1127.385593 secs 2021-11-03T17:34:46.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:46 smithi115 ceph-mon[65453]: pgmap v604: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:46.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:46 smithi027 ceph-mon[97244]: pgmap v604: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:49.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:49 smithi115 ceph-mon[65453]: pgmap v605: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:49.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:49 smithi027 ceph-mon[97244]: pgmap v605: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:50.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1132.385665 secs 2021-11-03T17:34:50.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1132.385665 secs 2021-11-03T17:34:51.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:51 smithi115 ceph-mon[65453]: pgmap v606: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:51.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:51 smithi027 ceph-mon[97244]: pgmap v606: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:52.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:34:52.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:34:52.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:34:52.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:52.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:52.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:52.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:52.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:52 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:34:52.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:52 smithi115 ceph-mon[65453]: pgmap v607: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:52.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:34:52.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:34:52.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:34:52.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:52.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:52.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:52.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:34:52.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:52 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:34:52.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:52 smithi027 ceph-mon[97244]: pgmap v607: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:55.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:55 smithi115 ceph-mon[65453]: pgmap v608: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:55.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1137.385790 secs 2021-11-03T17:34:55.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:55 smithi027 ceph-mon[97244]: pgmap v608: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:55.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1137.385790 secs 2021-11-03T17:34:56.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:34:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:34:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:56 smithi115 ceph-mon[65453]: pgmap v609: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:34:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:34:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:56 smithi027 ceph-mon[97244]: pgmap v609: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:59.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:34:59 smithi115 ceph-mon[65453]: pgmap v610: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:34:59.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:34:59 smithi027 ceph-mon[97244]: pgmap v610: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:00.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1142.385852 secs 2021-11-03T17:35:00.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:00 smithi115 ceph-mon[65453]: pgmap v611: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:00.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1142.385852 secs 2021-11-03T17:35:00.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:00 smithi027 ceph-mon[97244]: pgmap v611: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:02.379 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:35:02.401 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:35:03.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:03 smithi027 ceph-mon[97244]: poll took 60171.738 ms: 1 events 2021-11-03T17:35:03.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:03 smithi027 ceph-mon[97244]: pgmap v612: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:03.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:35:03.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:35:03.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:03.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:03.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:03.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:03.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:03 smithi115 ceph-mon[65453]: poll took 60171.738 ms: 1 events 2021-11-03T17:35:03.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:03 smithi115 ceph-mon[65453]: pgmap v612: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:03.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:35:03.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:35:03.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:03.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:03.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:03.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:04.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:04 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:35:04.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:04 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:35:05.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:05 smithi115 ceph-mon[65453]: pgmap v613: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:05.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1147.385996 secs 2021-11-03T17:35:05.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:05 smithi027 ceph-mon[97244]: pgmap v613: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:05.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1147.385996 secs 2021-11-03T17:35:07.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:07 smithi115 ceph-mon[65453]: pgmap v614: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:07.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:07 smithi027 ceph-mon[97244]: pgmap v614: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:09.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:09 smithi115 ceph-mon[65453]: pgmap v615: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:09.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:09 smithi027 ceph-mon[97244]: pgmap v615: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:10.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1152.386123 secs 2021-11-03T17:35:10.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1152.386123 secs 2021-11-03T17:35:11.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:11 smithi115 ceph-mon[65453]: pgmap v616: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:11.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:11 smithi027 ceph-mon[97244]: pgmap v616: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:13.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:13 smithi027 ceph-mon[97244]: pgmap v617: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:13.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:35:13.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:35:13.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:35:13.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:13.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:13.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:13.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:13.315 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.319+0000 7f3a3a398700 1 -- 172.21.15.27:0/302089964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a340f6540 msgr2=0x7f3a340f69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:35:13.315 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.319+0000 7f3a3a398700 1 --2- 172.21.15.27:0/302089964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a340f6540 0x7f3a340f69c0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f3a28004660 tx=0x7f3a28009b30).stop 2021-11-03T17:35:13.315 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.319+0000 7f3a3a398700 1 -- 172.21.15.27:0/302089964 shutdown_connections 2021-11-03T17:35:13.316 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.319+0000 7f3a3a398700 1 --2- 172.21.15.27:0/302089964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a340f6540 0x7f3a340f69c0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:13.316 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.319+0000 7f3a3a398700 1 --2- 172.21.15.27:0/302089964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a340f5c30 0x7f3a340f6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:13.316 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.319+0000 7f3a3a398700 1 -- 172.21.15.27:0/302089964 >> 172.21.15.27:0/302089964 conn(0x7f3a340f0540 msgr2=0x7f3a340f2960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:35:13.316 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.319+0000 7f3a3a398700 1 -- 172.21.15.27:0/302089964 shutdown_connections 2021-11-03T17:35:13.317 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.319+0000 7f3a3a398700 1 -- 172.21.15.27:0/302089964 wait complete. 2021-11-03T17:35:13.317 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.320+0000 7f3a3a398700 1 Processor -- start 2021-11-03T17:35:13.317 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.320+0000 7f3a3a398700 1 -- start start 2021-11-03T17:35:13.317 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.320+0000 7f3a3a398700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a340f5c30 0x7f3a34109c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:35:13.318 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.320+0000 7f3a3a398700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a340f6540 0x7f3a3410c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:35:13.318 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.320+0000 7f3a3a398700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a340f5700 con 0x7f3a340f6540 2021-11-03T17:35:13.318 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.320+0000 7f3a3a398700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a340f5840 con 0x7f3a340f5c30 2021-11-03T17:35:13.318 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.320+0000 7f3a337fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a340f6540 0x7f3a3410c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:35:13.318 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.320+0000 7f3a337fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a340f6540 0x7f3a3410c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58262/0 (socket says 172.21.15.27:58262) 2021-11-03T17:35:13.319 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.320+0000 7f3a337fe700 1 -- 172.21.15.27:0/2668354267 learned_addr learned my addr 172.21.15.27:0/2668354267 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:35:13.319 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.320+0000 7f3a33fff700 1 --2- 172.21.15.27:0/2668354267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a340f5c30 0x7f3a34109c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:35:13.319 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.321+0000 7f3a337fe700 1 -- 172.21.15.27:0/2668354267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a340f5c30 msgr2=0x7f3a34109c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:35:13.319 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.321+0000 7f3a337fe700 1 --2- 172.21.15.27:0/2668354267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a340f5c30 0x7f3a34109c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:13.320 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.321+0000 7f3a337fe700 1 -- 172.21.15.27:0/2668354267 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a28005040 con 0x7f3a340f6540 2021-11-03T17:35:13.320 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.321+0000 7f3a33fff700 1 --2- 172.21.15.27:0/2668354267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a340f5c30 0x7f3a34109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:35:13.320 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.321+0000 7f3a337fe700 1 --2- 172.21.15.27:0/2668354267 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a340f6540 0x7f3a3410c140 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f3a28009da0 tx=0x7f3a28009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:35:13.320 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.321+0000 7f3a30ff9700 1 -- 172.21.15.27:0/2668354267 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a28018070 con 0x7f3a340f6540 2021-11-03T17:35:13.320 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.321+0000 7f3a3a398700 1 -- 172.21.15.27:0/2668354267 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a3410c690 con 0x7f3a340f6540 2021-11-03T17:35:13.321 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.321+0000 7f3a30ff9700 1 -- 172.21.15.27:0/2668354267 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f3a28007d70 con 0x7f3a340f6540 2021-11-03T17:35:13.321 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.321+0000 7f3a3a398700 1 -- 172.21.15.27:0/2668354267 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a3410cbc0 con 0x7f3a340f6540 2021-11-03T17:35:13.321 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.321+0000 7f3a30ff9700 1 -- 172.21.15.27:0/2668354267 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a2801c620 con 0x7f3a340f6540 2021-11-03T17:35:13.322 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.322+0000 7f3a3a398700 1 -- 172.21.15.27:0/2668354267 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a34045ad0 con 0x7f3a340f6540 2021-11-03T17:35:13.323 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.322+0000 7f3a30ff9700 1 -- 172.21.15.27:0/2668354267 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f3a2801d960 con 0x7f3a340f6540 2021-11-03T17:35:13.323 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.323+0000 7f3a30ff9700 1 --2- 172.21.15.27:0/2668354267 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3a1c064090 0x7f3a1c066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:35:13.323 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.323+0000 7f3a30ff9700 1 -- 172.21.15.27:0/2668354267 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f3a28082810 con 0x7f3a340f6540 2021-11-03T17:35:13.323 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.323+0000 7f3a33fff700 1 --2- 172.21.15.27:0/2668354267 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3a1c064090 0x7f3a1c066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:35:13.323 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.323+0000 7f3a33fff700 1 --2- 172.21.15.27:0/2668354267 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3a1c064090 0x7f3a1c066550 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f3a240041d0 tx=0x7f3a24009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:35:13.324 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.327+0000 7f3a30ff9700 1 -- 172.21.15.27:0/2668354267 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f3a28054b10 con 0x7f3a340f6540 2021-11-03T17:35:13.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:13 smithi115 ceph-mon[65453]: pgmap v617: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:13.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:35:13.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:35:13.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:35:13.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:13.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:13.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:13.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:13.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.505+0000 7f3a3a398700 1 -- 172.21.15.27:0/2668354267 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a34057f10 con 0x7f3a1c064090 2021-11-03T17:35:13.500 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.506+0000 7f3a30ff9700 1 -- 172.21.15.27:0/2668354267 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f3a34057f10 con 0x7f3a1c064090 2021-11-03T17:35:13.502 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.508+0000 7f3a3a398700 1 -- 172.21.15.27:0/2668354267 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3a1c064090 msgr2=0x7f3a1c066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:35:13.503 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.508+0000 7f3a3a398700 1 --2- 172.21.15.27:0/2668354267 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3a1c064090 0x7f3a1c066550 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f3a240041d0 tx=0x7f3a24009040).stop 2021-11-03T17:35:13.503 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.508+0000 7f3a3a398700 1 -- 172.21.15.27:0/2668354267 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a340f6540 msgr2=0x7f3a3410c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:35:13.503 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.508+0000 7f3a3a398700 1 --2- 172.21.15.27:0/2668354267 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a340f6540 0x7f3a3410c140 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f3a28009da0 tx=0x7f3a28009b30).stop 2021-11-03T17:35:13.504 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.508+0000 7f3a3a398700 1 -- 172.21.15.27:0/2668354267 shutdown_connections 2021-11-03T17:35:13.504 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.508+0000 7f3a3a398700 1 --2- 172.21.15.27:0/2668354267 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3a1c064090 0x7f3a1c066550 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:13.504 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.508+0000 7f3a3a398700 1 --2- 172.21.15.27:0/2668354267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a340f5c30 0x7f3a34109c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:13.504 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.508+0000 7f3a3a398700 1 --2- 172.21.15.27:0/2668354267 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a340f6540 0x7f3a3410c140 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:13.504 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.508+0000 7f3a3a398700 1 -- 172.21.15.27:0/2668354267 >> 172.21.15.27:0/2668354267 conn(0x7f3a340f0540 msgr2=0x7f3a340fa4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:35:13.505 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.508+0000 7f3a3a398700 1 -- 172.21.15.27:0/2668354267 shutdown_connections 2021-11-03T17:35:13.505 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.509+0000 7f3a3a398700 1 -- 172.21.15.27:0/2668354267 wait complete. 2021-11-03T17:35:13.512 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:35:13.605 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.609+0000 7f8c22f4c700 1 -- 172.21.15.27:0/2089299649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c1c0f55e0 msgr2=0x7f8c1c0f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:35:13.605 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.609+0000 7f8c22f4c700 1 --2- 172.21.15.27:0/2089299649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c1c0f55e0 0x7f8c1c0f5a00 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f8c140099e0 tx=0x7f8c14004d10).stop 2021-11-03T17:35:13.605 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.609+0000 7f8c22f4c700 1 -- 172.21.15.27:0/2089299649 shutdown_connections 2021-11-03T17:35:13.606 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.609+0000 7f8c22f4c700 1 --2- 172.21.15.27:0/2089299649 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c1c0f5ef0 0x7f8c1c0f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:13.606 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.609+0000 7f8c22f4c700 1 --2- 172.21.15.27:0/2089299649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c1c0f55e0 0x7f8c1c0f5a00 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:13.606 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.609+0000 7f8c22f4c700 1 -- 172.21.15.27:0/2089299649 >> 172.21.15.27:0/2089299649 conn(0x7f8c1c0f0de0 msgr2=0x7f8c1c0f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:35:13.606 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.609+0000 7f8c22f4c700 1 -- 172.21.15.27:0/2089299649 shutdown_connections 2021-11-03T17:35:13.607 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.609+0000 7f8c22f4c700 1 -- 172.21.15.27:0/2089299649 wait complete. 2021-11-03T17:35:13.607 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.609+0000 7f8c22f4c700 1 Processor -- start 2021-11-03T17:35:13.607 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.609+0000 7f8c22f4c700 1 -- start start 2021-11-03T17:35:13.607 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.610+0000 7f8c22f4c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c1c0f55e0 0x7f8c1c109d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:35:13.608 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.610+0000 7f8c22f4c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c1c0f5ef0 0x7f8c1c10c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:35:13.608 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.610+0000 7f8c22f4c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c060520 con 0x7f8c1c0f55e0 2021-11-03T17:35:13.608 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.610+0000 7f8c22f4c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1c060660 con 0x7f8c1c0f5ef0 2021-11-03T17:35:13.608 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.610+0000 7f8c20ce8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c1c0f55e0 0x7f8c1c109d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:35:13.608 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.610+0000 7f8c20ce8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c1c0f55e0 0x7f8c1c109d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58270/0 (socket says 172.21.15.27:58270) 2021-11-03T17:35:13.609 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.610+0000 7f8c20ce8700 1 -- 172.21.15.27:0/3578970964 learned_addr learned my addr 172.21.15.27:0/3578970964 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:35:13.609 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.610+0000 7f8c1bfff700 1 --2- 172.21.15.27:0/3578970964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c1c0f5ef0 0x7f8c1c10c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:35:13.609 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.610+0000 7f8c20ce8700 1 -- 172.21.15.27:0/3578970964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c1c0f5ef0 msgr2=0x7f8c1c10c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:35:13.609 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.610+0000 7f8c20ce8700 1 --2- 172.21.15.27:0/3578970964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c1c0f5ef0 0x7f8c1c10c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:13.610 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.610+0000 7f8c20ce8700 1 -- 172.21.15.27:0/3578970964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c14009640 con 0x7f8c1c0f55e0 2021-11-03T17:35:13.610 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.610+0000 7f8c20ce8700 1 --2- 172.21.15.27:0/3578970964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c1c0f55e0 0x7f8c1c109d20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f8c1400b3b0 tx=0x7f8c14004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:35:13.610 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.610+0000 7f8c197fa700 1 -- 172.21.15.27:0/3578970964 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c1400e030 con 0x7f8c1c0f55e0 2021-11-03T17:35:13.610 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.610+0000 7f8c22f4c700 1 -- 172.21.15.27:0/3578970964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c1c10c7d0 con 0x7f8c1c0f55e0 2021-11-03T17:35:13.611 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.610+0000 7f8c197fa700 1 -- 172.21.15.27:0/3578970964 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f8c140049b0 con 0x7f8c1c0f55e0 2021-11-03T17:35:13.611 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.611+0000 7f8c22f4c700 1 -- 172.21.15.27:0/3578970964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c1c10cc80 con 0x7f8c1c0f55e0 2021-11-03T17:35:13.611 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.611+0000 7f8c197fa700 1 -- 172.21.15.27:0/3578970964 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c14018070 con 0x7f8c1c0f55e0 2021-11-03T17:35:13.612 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.611+0000 7f8c22f4c700 1 -- 172.21.15.27:0/3578970964 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c1c045ad0 con 0x7f8c1c0f55e0 2021-11-03T17:35:13.613 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.612+0000 7f8c197fa700 1 -- 172.21.15.27:0/3578970964 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f8c14007c90 con 0x7f8c1c0f55e0 2021-11-03T17:35:13.613 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.612+0000 7f8c197fa700 1 --2- 172.21.15.27:0/3578970964 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8c080640b0 0x7f8c08066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:35:13.613 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.612+0000 7f8c197fa700 1 -- 172.21.15.27:0/3578970964 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f8c140822f0 con 0x7f8c1c0f55e0 2021-11-03T17:35:13.614 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.613+0000 7f8c1bfff700 1 --2- 172.21.15.27:0/3578970964 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8c080640b0 0x7f8c08066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:35:13.614 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.613+0000 7f8c1bfff700 1 --2- 172.21.15.27:0/3578970964 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8c080640b0 0x7f8c08066570 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f8c0c004c40 tx=0x7f8c0c00b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:35:13.614 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.616+0000 7f8c197fa700 1 -- 172.21.15.27:0/3578970964 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f8c14054590 con 0x7f8c1c0f55e0 2021-11-03T17:35:13.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.792+0000 7f8c22f4c700 1 -- 172.21.15.27:0/3578970964 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c1c0fdf10 con 0x7f8c080640b0 2021-11-03T17:35:13.801 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.807+0000 7f8c197fa700 1 -- 172.21.15.27:0/3578970964 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f8c1c0fdf10 con 0x7f8c080640b0 2021-11-03T17:35:13.801 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:35:13.801 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (26m) 6m ago 28m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:35:13.802 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (18m) 6m ago 27m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:35:13.802 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (18m) 5m ago 27m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:35:13.802 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (26m) 6m ago 27m 40.2M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:35:13.802 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (20m) 6m ago 23m 18.2M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:35:13.803 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (23m) 6m ago 23m 3841M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:35:13.803 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (19m) 5m ago 23m 17.6M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:35:13.803 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (20m) 5m ago 23m 19.3M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:35:13.803 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (20m) 6m ago 29m 497M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:35:13.804 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (19m) 5m ago 27m 415M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:35:13.804 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (19m) 6m ago 29m 119M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:35:13.804 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (19m) 5m ago 27m 71.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:35:13.804 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (27m) 6m ago 27m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:35:13.805 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (27m) 5m ago 27m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:35:13.805 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (18m) 6m ago 26m 290M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:35:13.805 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (17m) 6m ago 26m 317M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:35:13.805 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (17m) 6m ago 25m 229M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:35:13.805 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (16m) 5m ago 25m 329M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:35:13.806 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (16m) 5m ago 25m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:35:13.806 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (15m) 5m ago 24m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:35:13.806 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (26m) 6m ago 27m 48.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:35:13.807 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.809+0000 7f8c22f4c700 1 -- 172.21.15.27:0/3578970964 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8c080640b0 msgr2=0x7f8c08066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:35:13.807 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.809+0000 7f8c22f4c700 1 --2- 172.21.15.27:0/3578970964 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8c080640b0 0x7f8c08066570 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f8c0c004c40 tx=0x7f8c0c00b040).stop 2021-11-03T17:35:13.808 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.809+0000 7f8c22f4c700 1 -- 172.21.15.27:0/3578970964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c1c0f55e0 msgr2=0x7f8c1c109d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:35:13.808 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.809+0000 7f8c22f4c700 1 --2- 172.21.15.27:0/3578970964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c1c0f55e0 0x7f8c1c109d20 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f8c1400b3b0 tx=0x7f8c14004d10).stop 2021-11-03T17:35:13.808 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.809+0000 7f8c22f4c700 1 -- 172.21.15.27:0/3578970964 shutdown_connections 2021-11-03T17:35:13.808 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.809+0000 7f8c22f4c700 1 --2- 172.21.15.27:0/3578970964 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8c080640b0 0x7f8c08066570 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:13.808 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.809+0000 7f8c22f4c700 1 --2- 172.21.15.27:0/3578970964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c1c0f55e0 0x7f8c1c109d20 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:13.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.809+0000 7f8c22f4c700 1 --2- 172.21.15.27:0/3578970964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c1c0f5ef0 0x7f8c1c10c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:13.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.809+0000 7f8c22f4c700 1 -- 172.21.15.27:0/3578970964 >> 172.21.15.27:0/3578970964 conn(0x7f8c1c0f0de0 msgr2=0x7f8c1c0fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:35:13.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.809+0000 7f8c22f4c700 1 -- 172.21.15.27:0/3578970964 shutdown_connections 2021-11-03T17:35:13.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.809+0000 7f8c22f4c700 1 -- 172.21.15.27:0/3578970964 wait complete. 2021-11-03T17:35:13.902 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.908+0000 7fc6d520f700 1 -- 172.21.15.27:0/802632350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6d00f6860 msgr2=0x7fc6d00f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:35:13.903 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.908+0000 7fc6d520f700 1 --2- 172.21.15.27:0/802632350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6d00f6860 0x7fc6d00f6cc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7fc6b8004660 tx=0x7fc6b8009b30).stop 2021-11-03T17:35:13.903 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.908+0000 7fc6d520f700 1 -- 172.21.15.27:0/802632350 shutdown_connections 2021-11-03T17:35:13.903 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.908+0000 7fc6d520f700 1 --2- 172.21.15.27:0/802632350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6d00f6860 0x7fc6d00f6cc0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:13.903 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.908+0000 7fc6d520f700 1 --2- 172.21.15.27:0/802632350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6d00f55c0 0x7fc6d00f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:13.904 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.908+0000 7fc6d520f700 1 -- 172.21.15.27:0/802632350 >> 172.21.15.27:0/802632350 conn(0x7fc6d00f0d40 msgr2=0x7fc6d00f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:35:13.904 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.908+0000 7fc6d520f700 1 -- 172.21.15.27:0/802632350 shutdown_connections 2021-11-03T17:35:13.904 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.908+0000 7fc6d520f700 1 -- 172.21.15.27:0/802632350 wait complete. 2021-11-03T17:35:13.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.909+0000 7fc6d520f700 1 Processor -- start 2021-11-03T17:35:13.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.909+0000 7fc6d520f700 1 -- start start 2021-11-03T17:35:13.908 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.909+0000 7fc6d520f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6d00f55c0 0x7fc6d0106f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:35:13.908 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.909+0000 7fc6d520f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6d00f6860 0x7fc6d0107440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:35:13.908 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.909+0000 7fc6d520f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d01079f0 con 0x7fc6d00f55c0 2021-11-03T17:35:13.908 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.909+0000 7fc6d520f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6d0107b30 con 0x7fc6d00f6860 2021-11-03T17:35:13.909 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.909+0000 7fc6ced9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6d00f55c0 0x7fc6d0106f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:35:13.909 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.909+0000 7fc6ced9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6d00f55c0 0x7fc6d0106f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58280/0 (socket says 172.21.15.27:58280) 2021-11-03T17:35:13.909 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.909+0000 7fc6ced9d700 1 -- 172.21.15.27:0/3967330378 learned_addr learned my addr 172.21.15.27:0/3967330378 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:35:13.909 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.909+0000 7fc6ce59c700 1 --2- 172.21.15.27:0/3967330378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6d00f6860 0x7fc6d0107440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:35:13.909 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.909+0000 7fc6ced9d700 1 -- 172.21.15.27:0/3967330378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6d00f6860 msgr2=0x7fc6d0107440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:35:13.910 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.909+0000 7fc6ced9d700 1 --2- 172.21.15.27:0/3967330378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6d00f6860 0x7fc6d0107440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:13.910 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.909+0000 7fc6ced9d700 1 -- 172.21.15.27:0/3967330378 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b8005040 con 0x7fc6d00f55c0 2021-11-03T17:35:13.910 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.910+0000 7fc6ced9d700 1 --2- 172.21.15.27:0/3967330378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6d00f55c0 0x7fc6d0106f50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7fc6c800d7f0 tx=0x7fc6c8004e80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:35:13.910 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.910+0000 7fc6c77fe700 1 -- 172.21.15.27:0/3967330378 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6c800f490 con 0x7fc6d00f55c0 2021-11-03T17:35:13.911 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.910+0000 7fc6c77fe700 1 -- 172.21.15.27:0/3967330378 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fc6c800f040 con 0x7fc6d00f55c0 2021-11-03T17:35:13.911 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.910+0000 7fc6d520f700 1 -- 172.21.15.27:0/3967330378 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6d01084d0 con 0x7fc6d00f55c0 2021-11-03T17:35:13.911 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.910+0000 7fc6c77fe700 1 -- 172.21.15.27:0/3967330378 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc6c80139b0 con 0x7fc6d00f55c0 2021-11-03T17:35:13.911 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.910+0000 7fc6d520f700 1 -- 172.21.15.27:0/3967330378 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6d0108a90 con 0x7fc6d00f55c0 2021-11-03T17:35:13.911 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.911+0000 7fc6d520f700 1 -- 172.21.15.27:0/3967330378 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6d0045ad0 con 0x7fc6d00f55c0 2021-11-03T17:35:13.912 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.911+0000 7fc6c77fe700 1 -- 172.21.15.27:0/3967330378 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fc6c801a030 con 0x7fc6d00f55c0 2021-11-03T17:35:13.912 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.912+0000 7fc6c77fe700 1 --2- 172.21.15.27:0/3967330378 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc6bc0640b0 0x7fc6bc066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:35:13.912 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.912+0000 7fc6c77fe700 1 -- 172.21.15.27:0/3967330378 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fc6c8080850 con 0x7fc6d00f55c0 2021-11-03T17:35:13.912 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.912+0000 7fc6ce59c700 1 --2- 172.21.15.27:0/3967330378 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc6bc0640b0 0x7fc6bc066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:35:13.913 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.912+0000 7fc6ce59c700 1 --2- 172.21.15.27:0/3967330378 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc6bc0640b0 0x7fc6bc066570 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7fc6b8004630 tx=0x7fc6b8009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:35:13.913 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:13.916+0000 7fc6c77fe700 1 -- 172.21.15.27:0/3967330378 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fc6c8052a60 con 0x7fc6d00f55c0 2021-11-03T17:35:14.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.147+0000 7fc6d520f700 1 -- 172.21.15.27:0/3967330378 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc6d00fee80 con 0x7fc6d00f55c0 2021-11-03T17:35:14.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.147+0000 7fc6c77fe700 1 -- 172.21.15.27:0/3967330378 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fc6c8018070 con 0x7fc6d00f55c0 2021-11-03T17:35:14.229 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:35:14.229 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:35:14.229 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:35:14.229 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:35:14.230 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:35:14.230 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:35:14.230 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:35:14.230 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:35:14.231 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:35:14.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:35:14.231 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:35:14.231 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:35:14.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:35:14.232 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:35:14.232 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:35:14.232 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:35:14.232 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:35:14.233 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:35:14.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.149+0000 7fc6d520f700 1 -- 172.21.15.27:0/3967330378 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc6bc0640b0 msgr2=0x7fc6bc066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:35:14.234 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.149+0000 7fc6d520f700 1 --2- 172.21.15.27:0/3967330378 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc6bc0640b0 0x7fc6bc066570 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7fc6b8004630 tx=0x7fc6b8009b30).stop 2021-11-03T17:35:14.234 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.149+0000 7fc6d520f700 1 -- 172.21.15.27:0/3967330378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6d00f55c0 msgr2=0x7fc6d0106f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:35:14.234 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.149+0000 7fc6d520f700 1 --2- 172.21.15.27:0/3967330378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6d00f55c0 0x7fc6d0106f50 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7fc6c800d7f0 tx=0x7fc6c8004e80).stop 2021-11-03T17:35:14.234 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.149+0000 7fc6d520f700 1 -- 172.21.15.27:0/3967330378 shutdown_connections 2021-11-03T17:35:14.235 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.149+0000 7fc6d520f700 1 --2- 172.21.15.27:0/3967330378 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc6bc0640b0 0x7fc6bc066570 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:14.235 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.149+0000 7fc6d520f700 1 --2- 172.21.15.27:0/3967330378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc6d00f55c0 0x7fc6d0106f50 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:14.235 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.149+0000 7fc6d520f700 1 --2- 172.21.15.27:0/3967330378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc6d00f6860 0x7fc6d0107440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:14.235 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.149+0000 7fc6d520f700 1 -- 172.21.15.27:0/3967330378 >> 172.21.15.27:0/3967330378 conn(0x7fc6d00f0d40 msgr2=0x7fc6d00fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:35:14.235 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.149+0000 7fc6d520f700 1 -- 172.21.15.27:0/3967330378 shutdown_connections 2021-11-03T17:35:14.236 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.149+0000 7fc6d520f700 1 -- 172.21.15.27:0/3967330378 wait complete. 2021-11-03T17:35:14.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:14 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:35:14.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.250+0000 7f698ec27700 1 -- 172.21.15.27:0/1171387037 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69880f5f60 msgr2=0x7f69880f63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:35:14.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.250+0000 7f698ec27700 1 --2- 172.21.15.27:0/1171387037 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69880f5f60 0x7f69880f63e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f697c004660 tx=0x7f697c009b30).stop 2021-11-03T17:35:14.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.250+0000 7f698ec27700 1 -- 172.21.15.27:0/1171387037 shutdown_connections 2021-11-03T17:35:14.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.250+0000 7f698ec27700 1 --2- 172.21.15.27:0/1171387037 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69880f5f60 0x7f69880f63e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:14.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.250+0000 7f698ec27700 1 --2- 172.21.15.27:0/1171387037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69880f4d60 0x7f69880f5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:14.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.250+0000 7f698ec27700 1 -- 172.21.15.27:0/1171387037 >> 172.21.15.27:0/1171387037 conn(0x7f69880f04c0 msgr2=0x7f69880f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:35:14.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.250+0000 7f698ec27700 1 -- 172.21.15.27:0/1171387037 shutdown_connections 2021-11-03T17:35:14.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.250+0000 7f698ec27700 1 -- 172.21.15.27:0/1171387037 wait complete. 2021-11-03T17:35:14.248 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.250+0000 7f698ec27700 1 Processor -- start 2021-11-03T17:35:14.248 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.250+0000 7f698ec27700 1 -- start start 2021-11-03T17:35:14.248 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.251+0000 7f698ec27700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69880f4d60 0x7f6988106f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:35:14.248 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.251+0000 7f698ec27700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69880f5f60 0x7f6988107420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:35:14.249 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.251+0000 7f698ec27700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69881079d0 con 0x7f69880f4d60 2021-11-03T17:35:14.249 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.251+0000 7f698ec27700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6988107b10 con 0x7f69880f5f60 2021-11-03T17:35:14.249 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.251+0000 7f698c9c3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69880f4d60 0x7f6988106f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:35:14.249 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.251+0000 7f698c9c3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69880f4d60 0x7f6988106f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58290/0 (socket says 172.21.15.27:58290) 2021-11-03T17:35:14.250 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.251+0000 7f698c9c3700 1 -- 172.21.15.27:0/3617653364 learned_addr learned my addr 172.21.15.27:0/3617653364 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:35:14.250 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.251+0000 7f6987fff700 1 --2- 172.21.15.27:0/3617653364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69880f5f60 0x7f6988107420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:35:14.250 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.251+0000 7f698c9c3700 1 -- 172.21.15.27:0/3617653364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69880f5f60 msgr2=0x7f6988107420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:35:14.250 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.251+0000 7f698c9c3700 1 --2- 172.21.15.27:0/3617653364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69880f5f60 0x7f6988107420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:14.251 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.251+0000 7f698c9c3700 1 -- 172.21.15.27:0/3617653364 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697c005040 con 0x7f69880f4d60 2021-11-03T17:35:14.251 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.251+0000 7f698c9c3700 1 --2- 172.21.15.27:0/3617653364 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69880f4d60 0x7f6988106f30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f698000c760 tx=0x7f6980004d20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:35:14.251 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.251+0000 7f69857fa700 1 -- 172.21.15.27:0/3617653364 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6980004200 con 0x7f69880f4d60 2021-11-03T17:35:14.251 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.251+0000 7f698ec27700 1 -- 172.21.15.27:0/3617653364 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69881084b0 con 0x7f69880f4d60 2021-11-03T17:35:14.251 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.251+0000 7f69857fa700 1 -- 172.21.15.27:0/3617653364 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f69800109c0 con 0x7f69880f4d60 2021-11-03T17:35:14.252 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.251+0000 7f69857fa700 1 -- 172.21.15.27:0/3617653364 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6980003980 con 0x7f69880f4d60 2021-11-03T17:35:14.252 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.251+0000 7f698ec27700 1 -- 172.21.15.27:0/3617653364 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6988108a70 con 0x7f69880f4d60 2021-11-03T17:35:14.252 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.252+0000 7f698ec27700 1 -- 172.21.15.27:0/3617653364 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6988045ad0 con 0x7f69880f4d60 2021-11-03T17:35:14.252 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.253+0000 7f69857fa700 1 -- 172.21.15.27:0/3617653364 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f6980018020 con 0x7f69880f4d60 2021-11-03T17:35:14.253 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.253+0000 7f69857fa700 1 --2- 172.21.15.27:0/3617653364 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6970064090 0x7f6970066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:35:14.253 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.253+0000 7f69857fa700 1 -- 172.21.15.27:0/3617653364 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f6980080710 con 0x7f69880f4d60 2021-11-03T17:35:14.253 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.253+0000 7f6987fff700 1 --2- 172.21.15.27:0/3617653364 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6970064090 0x7f6970066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:35:14.253 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.254+0000 7f6987fff700 1 --2- 172.21.15.27:0/3617653364 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6970064090 0x7f6970066550 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f697c004e30 tx=0x7f697c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:35:14.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.257+0000 7f69857fa700 1 -- 172.21.15.27:0/3617653364 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f6980052920 con 0x7f69880f4d60 2021-11-03T17:35:14.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.460+0000 7f698ec27700 1 -- 172.21.15.27:0/3617653364 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f69880fd650 con 0x7f69880f4d60 2021-11-03T17:35:14.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.460+0000 7f69857fa700 1 -- 172.21.15.27:0/3617653364 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f6980052510 con 0x7f69880f4d60 2021-11-03T17:35:14.455 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:35:14.455 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:35:14.455 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:35:14.456 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:35:14.456 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:35:14.456 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:35:14.456 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:35:14.457 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:35:14.457 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:35:14.457 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:35:14.457 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:35:14.458 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:35:14.458 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:35:14.458 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:35:14.458 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:35:14.458 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:35:14.459 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:35:14.459 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:35:14.459 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:35:14.459 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:35:14.460 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:35:14.460 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:35:14.460 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:35:14.460 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:35:14.461 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:35:14.461 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:35:14.461 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:35:14.461 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:35:14.461 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:35:14.462 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:35:14.462 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:35:14.462 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:35:14.462 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:35:14.463 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:35:14.463 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:35:14.463 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:35:14.463 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:35:14.464 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:35:14.464 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:35:14.465 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.462+0000 7f698ec27700 1 -- 172.21.15.27:0/3617653364 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6970064090 msgr2=0x7f6970066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:35:14.465 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.462+0000 7f698ec27700 1 --2- 172.21.15.27:0/3617653364 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6970064090 0x7f6970066550 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 rx=0x7f697c004e30 tx=0x7f697c009b30).stop 2021-11-03T17:35:14.465 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.462+0000 7f698ec27700 1 -- 172.21.15.27:0/3617653364 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69880f4d60 msgr2=0x7f6988106f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:35:14.466 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.462+0000 7f698ec27700 1 --2- 172.21.15.27:0/3617653364 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69880f4d60 0x7f6988106f30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f698000c760 tx=0x7f6980004d20).stop 2021-11-03T17:35:14.466 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.462+0000 7f698ec27700 1 -- 172.21.15.27:0/3617653364 shutdown_connections 2021-11-03T17:35:14.466 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.462+0000 7f698ec27700 1 --2- 172.21.15.27:0/3617653364 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6970064090 0x7f6970066550 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:14.466 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.462+0000 7f698ec27700 1 --2- 172.21.15.27:0/3617653364 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69880f4d60 0x7f6988106f30 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:14.467 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.462+0000 7f698ec27700 1 --2- 172.21.15.27:0/3617653364 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69880f5f60 0x7f6988107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:14.467 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.462+0000 7f698ec27700 1 -- 172.21.15.27:0/3617653364 >> 172.21.15.27:0/3617653364 conn(0x7f69880f04c0 msgr2=0x7f69880fa530 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:35:14.467 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.462+0000 7f698ec27700 1 -- 172.21.15.27:0/3617653364 shutdown_connections 2021-11-03T17:35:14.467 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:14.462+0000 7f698ec27700 1 -- 172.21.15.27:0/3617653364 wait complete. 2021-11-03T17:35:14.468 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:35:14.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:14 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:35:15.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:15 smithi115 ceph-mon[65453]: from='client.34744 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:35:15.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:15 smithi115 ceph-mon[65453]: from='client.34748 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:35:15.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:15 smithi115 ceph-mon[65453]: pgmap v618: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:15.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:15 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3967330378' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:15.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1157.386208 secs 2021-11-03T17:35:15.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:15 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3617653364' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:35:15.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:15 smithi027 ceph-mon[97244]: from='client.34744 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:35:15.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:15 smithi027 ceph-mon[97244]: from='client.34748 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:35:15.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:15 smithi027 ceph-mon[97244]: pgmap v618: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:15.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:15 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3967330378' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:15.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1157.386208 secs 2021-11-03T17:35:15.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:15 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3617653364' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:35:17.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:17 smithi115 ceph-mon[65453]: pgmap v619: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:17.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:17 smithi027 ceph-mon[97244]: pgmap v619: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:19.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:19 smithi115 ceph-mon[65453]: pgmap v620: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:19.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:19 smithi027 ceph-mon[97244]: pgmap v620: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:20.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1162.386354 secs 2021-11-03T17:35:20.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1162.386354 secs 2021-11-03T17:35:21.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:21 smithi115 ceph-mon[65453]: pgmap v621: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:21.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:21 smithi027 ceph-mon[97244]: pgmap v621: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:23.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:23 smithi027 ceph-mon[97244]: pgmap v622: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:23.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:35:23.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:35:23.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:35:23.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:23.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:23.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:23.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:23.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:23 smithi115 ceph-mon[65453]: pgmap v622: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:23.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:35:23.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:35:23.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:35:23.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:23.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:23.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:23.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:24.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:24 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:35:24.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:24 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:35:25.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:25 smithi115 ceph-mon[65453]: pgmap v623: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:25.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1167.386485 secs 2021-11-03T17:35:25.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:25 smithi027 ceph-mon[97244]: pgmap v623: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:25.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1167.386485 secs 2021-11-03T17:35:26.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:26 smithi115 ceph-mon[65453]: pgmap v624: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:26.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:26 smithi027 ceph-mon[97244]: pgmap v624: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:28.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:28 smithi115 ceph-mon[65453]: pgmap v625: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:28.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:28 smithi027 ceph-mon[97244]: pgmap v625: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:29.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:29 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1172.386632 secs 2021-11-03T17:35:30.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:29 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1172.386632 secs 2021-11-03T17:35:30.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:30 smithi115 ceph-mon[65453]: pgmap v626: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:31.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:30 smithi027 ceph-mon[97244]: pgmap v626: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:32.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:32 smithi115 ceph-mon[65453]: pgmap v627: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:32.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:32 smithi027 ceph-mon[97244]: pgmap v627: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:33.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:35:33.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:35:33.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:35:33.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:33.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:33.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:33.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:33.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:33 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:35:33.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:35:33.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:35:33.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:35:33.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:33.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:33.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:33.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:33.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:33 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:35:34.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:34 smithi115 ceph-mon[65453]: pgmap v628: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:34.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:34 smithi027 ceph-mon[97244]: pgmap v628: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:35.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1177.386773 secs 2021-11-03T17:35:35.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1177.386773 secs 2021-11-03T17:35:36.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:36 smithi115 ceph-mon[65453]: pgmap v629: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:36.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:36 smithi027 ceph-mon[97244]: pgmap v629: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:38.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:38 smithi115 ceph-mon[65453]: pgmap v630: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:38.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:38 smithi027 ceph-mon[97244]: pgmap v630: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:39.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:39 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1182.386899 secs 2021-11-03T17:35:40.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:39 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1182.386899 secs 2021-11-03T17:35:40.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:40 smithi115 ceph-mon[65453]: pgmap v631: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:41.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:40 smithi027 ceph-mon[97244]: pgmap v631: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:42.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:42 smithi115 ceph-mon[65453]: pgmap v632: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:42.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:42 smithi027 ceph-mon[97244]: pgmap v632: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:43.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:35:43.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:35:43.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:35:43.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:43.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:43.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:43.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:43.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:43 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:35:43.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:35:43.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:35:43.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:35:43.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:43.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:43.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:43.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:43.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:43 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:35:44.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:44 smithi115 ceph-mon[65453]: pgmap v633: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:44.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:44 smithi027 ceph-mon[97244]: pgmap v633: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:44.574 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.577+0000 7f45dff20700 1 -- 172.21.15.27:0/2564775916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45d8060310 msgr2=0x7f45d80fb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:35:44.575 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.577+0000 7f45dff20700 1 --2- 172.21.15.27:0/2564775916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45d8060310 0x7f45d80fb6a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f45d4004660 tx=0x7f45d4009b30).stop 2021-11-03T17:35:44.575 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.577+0000 7f45dff20700 1 -- 172.21.15.27:0/2564775916 shutdown_connections 2021-11-03T17:35:44.575 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.577+0000 7f45dff20700 1 --2- 172.21.15.27:0/2564775916 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45d80fbb90 0x7f45d80fdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:44.575 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.577+0000 7f45dff20700 1 --2- 172.21.15.27:0/2564775916 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45d8060310 0x7f45d80fb6a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:44.576 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.577+0000 7f45dff20700 1 -- 172.21.15.27:0/2564775916 >> 172.21.15.27:0/2564775916 conn(0x7f45d80f0d40 msgr2=0x7f45d80f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:35:44.576 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.577+0000 7f45dff20700 1 -- 172.21.15.27:0/2564775916 shutdown_connections 2021-11-03T17:35:44.576 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.577+0000 7f45dff20700 1 -- 172.21.15.27:0/2564775916 wait complete. 2021-11-03T17:35:44.576 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.578+0000 7f45dff20700 1 Processor -- start 2021-11-03T17:35:44.577 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.578+0000 7f45dff20700 1 -- start start 2021-11-03T17:35:44.577 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.578+0000 7f45dff20700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45d8060310 0x7f45d8109150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:35:44.577 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.578+0000 7f45dff20700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45d80fbb90 0x7f45d8109640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:35:44.577 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.578+0000 7f45dff20700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d8109bf0 con 0x7f45d8060310 2021-11-03T17:35:44.578 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.578+0000 7f45dff20700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45d8109d30 con 0x7f45d80fbb90 2021-11-03T17:35:44.578 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.578+0000 7f45ddcbc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45d8060310 0x7f45d8109150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:35:44.578 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.578+0000 7f45ddcbc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45d8060310 0x7f45d8109150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58308/0 (socket says 172.21.15.27:58308) 2021-11-03T17:35:44.578 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.578+0000 7f45ddcbc700 1 -- 172.21.15.27:0/4062315896 learned_addr learned my addr 172.21.15.27:0/4062315896 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:35:44.579 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.579+0000 7f45dd4bb700 1 --2- 172.21.15.27:0/4062315896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45d80fbb90 0x7f45d8109640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:35:44.579 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.579+0000 7f45ddcbc700 1 -- 172.21.15.27:0/4062315896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45d80fbb90 msgr2=0x7f45d8109640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:35:44.579 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.579+0000 7f45ddcbc700 1 --2- 172.21.15.27:0/4062315896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45d80fbb90 0x7f45d8109640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:44.579 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.579+0000 7f45ddcbc700 1 -- 172.21.15.27:0/4062315896 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d4005040 con 0x7f45d8060310 2021-11-03T17:35:44.580 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.579+0000 7f45dd4bb700 1 --2- 172.21.15.27:0/4062315896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45d80fbb90 0x7f45d8109640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:35:44.580 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.579+0000 7f45ddcbc700 1 --2- 172.21.15.27:0/4062315896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45d8060310 0x7f45d8109150 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f45d4009990 tx=0x7f45d4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:35:44.580 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.579+0000 7f45ce7fc700 1 -- 172.21.15.27:0/4062315896 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45d4018070 con 0x7f45d8060310 2021-11-03T17:35:44.580 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.579+0000 7f45dff20700 1 -- 172.21.15.27:0/4062315896 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45d810a670 con 0x7f45d8060310 2021-11-03T17:35:44.581 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.579+0000 7f45ce7fc700 1 -- 172.21.15.27:0/4062315896 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f45d4007ca0 con 0x7f45d8060310 2021-11-03T17:35:44.581 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.579+0000 7f45dff20700 1 -- 172.21.15.27:0/4062315896 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45d810abd0 con 0x7f45d8060310 2021-11-03T17:35:44.581 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.579+0000 7f45ce7fc700 1 -- 172.21.15.27:0/4062315896 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45d401c600 con 0x7f45d8060310 2021-11-03T17:35:44.582 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.580+0000 7f45dff20700 1 -- 172.21.15.27:0/4062315896 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45d8045ad0 con 0x7f45d8060310 2021-11-03T17:35:44.582 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.581+0000 7f45ce7fc700 1 -- 172.21.15.27:0/4062315896 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f45d401d9d0 con 0x7f45d8060310 2021-11-03T17:35:44.583 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.581+0000 7f45ce7fc700 1 --2- 172.21.15.27:0/4062315896 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f45c40640b0 0x7f45c4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:35:44.583 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.581+0000 7f45ce7fc700 1 -- 172.21.15.27:0/4062315896 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f45d40826d0 con 0x7f45d8060310 2021-11-03T17:35:44.583 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.581+0000 7f45dd4bb700 1 --2- 172.21.15.27:0/4062315896 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f45c40640b0 0x7f45c4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:35:44.583 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.582+0000 7f45dd4bb700 1 --2- 172.21.15.27:0/4062315896 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f45c40640b0 0x7f45c4066570 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f45c80041d0 tx=0x7f45c8009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:35:44.584 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.585+0000 7f45ce7fc700 1 -- 172.21.15.27:0/4062315896 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f45d40549d0 con 0x7f45d8060310 2021-11-03T17:35:44.763 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.768+0000 7f45dff20700 1 -- 172.21.15.27:0/4062315896 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f45d80008d0 con 0x7f45c40640b0 2021-11-03T17:35:44.764 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.770+0000 7f45ce7fc700 1 -- 172.21.15.27:0/4062315896 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f45d80008d0 con 0x7f45c40640b0 2021-11-03T17:35:44.767 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.772+0000 7f45dff20700 1 -- 172.21.15.27:0/4062315896 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f45c40640b0 msgr2=0x7f45c4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:35:44.767 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.772+0000 7f45dff20700 1 --2- 172.21.15.27:0/4062315896 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f45c40640b0 0x7f45c4066570 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f45c80041d0 tx=0x7f45c8009040).stop 2021-11-03T17:35:44.767 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.772+0000 7f45dff20700 1 -- 172.21.15.27:0/4062315896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45d8060310 msgr2=0x7f45d8109150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:35:44.768 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.772+0000 7f45dff20700 1 --2- 172.21.15.27:0/4062315896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45d8060310 0x7f45d8109150 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f45d4009990 tx=0x7f45d4009b30).stop 2021-11-03T17:35:44.768 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.772+0000 7f45dff20700 1 -- 172.21.15.27:0/4062315896 shutdown_connections 2021-11-03T17:35:44.768 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.772+0000 7f45dff20700 1 --2- 172.21.15.27:0/4062315896 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f45c40640b0 0x7f45c4066570 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:44.768 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.772+0000 7f45dff20700 1 --2- 172.21.15.27:0/4062315896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45d8060310 0x7f45d8109150 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:44.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.772+0000 7f45dff20700 1 --2- 172.21.15.27:0/4062315896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45d80fbb90 0x7f45d8109640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:44.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.772+0000 7f45dff20700 1 -- 172.21.15.27:0/4062315896 >> 172.21.15.27:0/4062315896 conn(0x7f45d80f0d40 msgr2=0x7f45d80f3130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:35:44.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.772+0000 7f45dff20700 1 -- 172.21.15.27:0/4062315896 shutdown_connections 2021-11-03T17:35:44.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.772+0000 7f45dff20700 1 -- 172.21.15.27:0/4062315896 wait complete. 2021-11-03T17:35:44.777 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:35:44.870 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.873+0000 7f9374975700 1 -- 172.21.15.27:0/2465289649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93700f6ff0 msgr2=0x7f93700f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:35:44.870 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.873+0000 7f9374975700 1 --2- 172.21.15.27:0/2465289649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93700f6ff0 0x7f93700f7a20 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f9358004660 tx=0x7f9358009b30).stop 2021-11-03T17:35:44.870 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.873+0000 7f9374975700 1 -- 172.21.15.27:0/2465289649 shutdown_connections 2021-11-03T17:35:44.871 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.873+0000 7f9374975700 1 --2- 172.21.15.27:0/2465289649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93700f6ff0 0x7f93700f7a20 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:44.871 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.873+0000 7f9374975700 1 --2- 172.21.15.27:0/2465289649 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93700f66e0 0x7f93700f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:44.871 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.873+0000 7f9374975700 1 -- 172.21.15.27:0/2465289649 >> 172.21.15.27:0/2465289649 conn(0x7f93700f1f00 msgr2=0x7f93700f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:35:44.871 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.873+0000 7f9374975700 1 -- 172.21.15.27:0/2465289649 shutdown_connections 2021-11-03T17:35:44.872 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.873+0000 7f9374975700 1 -- 172.21.15.27:0/2465289649 wait complete. 2021-11-03T17:35:44.872 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.873+0000 7f9374975700 1 Processor -- start 2021-11-03T17:35:44.872 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.873+0000 7f9374975700 1 -- start start 2021-11-03T17:35:44.872 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.873+0000 7f9374975700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93700f66e0 0x7f937006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:35:44.872 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.873+0000 7f9374975700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93700f6ff0 0x7f9370069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:35:44.873 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.873+0000 7f9374975700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f937006a380 con 0x7f93700f6ff0 2021-11-03T17:35:44.873 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.873+0000 7f9374975700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f937006a4c0 con 0x7f93700f66e0 2021-11-03T17:35:44.873 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.874+0000 7f936dd9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93700f6ff0 0x7f9370069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:35:44.873 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.874+0000 7f936dd9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93700f6ff0 0x7f9370069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58320/0 (socket says 172.21.15.27:58320) 2021-11-03T17:35:44.874 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.874+0000 7f936dd9b700 1 -- 172.21.15.27:0/2154126764 learned_addr learned my addr 172.21.15.27:0/2154126764 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:35:44.874 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.874+0000 7f936e59c700 1 --2- 172.21.15.27:0/2154126764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93700f66e0 0x7f937006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:35:44.874 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.874+0000 7f936dd9b700 1 -- 172.21.15.27:0/2154126764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93700f66e0 msgr2=0x7f937006b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:35:44.874 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.874+0000 7f936dd9b700 1 --2- 172.21.15.27:0/2154126764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93700f66e0 0x7f937006b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:44.875 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.874+0000 7f936dd9b700 1 -- 172.21.15.27:0/2154126764 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9358005040 con 0x7f93700f6ff0 2021-11-03T17:35:44.875 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.874+0000 7f936dd9b700 1 --2- 172.21.15.27:0/2154126764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93700f6ff0 0x7f9370069e20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f9358009990 tx=0x7f9358009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:35:44.875 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.874+0000 7f9366ffd700 1 -- 172.21.15.27:0/2154126764 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9358018070 con 0x7f93700f6ff0 2021-11-03T17:35:44.875 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.874+0000 7f9366ffd700 1 -- 172.21.15.27:0/2154126764 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f9358007ba0 con 0x7f93700f6ff0 2021-11-03T17:35:44.875 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.874+0000 7f9374975700 1 -- 172.21.15.27:0/2154126764 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f937006a740 con 0x7f93700f6ff0 2021-11-03T17:35:44.876 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.874+0000 7f9366ffd700 1 -- 172.21.15.27:0/2154126764 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f935801c3c0 con 0x7f93700f6ff0 2021-11-03T17:35:44.876 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.874+0000 7f9374975700 1 -- 172.21.15.27:0/2154126764 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f937006aca0 con 0x7f93700f6ff0 2021-11-03T17:35:44.877 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.875+0000 7f9374975700 1 -- 172.21.15.27:0/2154126764 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9370045ad0 con 0x7f93700f6ff0 2021-11-03T17:35:44.877 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.876+0000 7f9366ffd700 1 -- 172.21.15.27:0/2154126764 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f935801d420 con 0x7f93700f6ff0 2021-11-03T17:35:44.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.876+0000 7f9366ffd700 1 --2- 172.21.15.27:0/2154126764 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f935c0640b0 0x7f935c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:35:44.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.876+0000 7f9366ffd700 1 -- 172.21.15.27:0/2154126764 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f93580823a0 con 0x7f93700f6ff0 2021-11-03T17:35:44.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.876+0000 7f936e59c700 1 --2- 172.21.15.27:0/2154126764 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f935c0640b0 0x7f935c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:35:44.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.877+0000 7f936e59c700 1 --2- 172.21.15.27:0/2154126764 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f935c0640b0 0x7f935c066570 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f937006b730 tx=0x7f936800b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:35:44.879 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:44.880+0000 7f9366ffd700 1 -- 172.21.15.27:0/2154126764 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f9358054760 con 0x7f93700f6ff0 2021-11-03T17:35:45.046 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.051+0000 7f9374975700 1 -- 172.21.15.27:0/2154126764 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f93700fefe0 con 0x7f935c0640b0 2021-11-03T17:35:45.061 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.065+0000 7f9366ffd700 1 -- 172.21.15.27:0/2154126764 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f93700fefe0 con 0x7f935c0640b0 2021-11-03T17:35:45.061 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:35:45.062 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (27m) 7m ago 28m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:35:45.062 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (19m) 7m ago 28m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:35:45.062 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (18m) 6m ago 27m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:35:45.062 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (27m) 7m ago 28m 40.2M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:35:45.063 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (20m) 7m ago 23m 18.2M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:35:45.063 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (23m) 7m ago 23m 3841M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:35:45.063 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (19m) 6m ago 23m 17.6M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:35:45.063 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (21m) 6m ago 23m 19.3M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:35:45.063 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (21m) 7m ago 29m 497M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:35:45.064 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (20m) 6m ago 27m 415M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:35:45.064 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (20m) 7m ago 29m 119M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:35:45.064 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (19m) 6m ago 27m 71.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:35:45.064 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (28m) 7m ago 28m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:35:45.065 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (27m) 6m ago 27m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:35:45.065 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (18m) 7m ago 26m 290M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:35:45.065 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (18m) 7m ago 26m 317M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:35:45.065 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (17m) 7m ago 26m 229M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:35:45.065 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (17m) 6m ago 25m 329M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:35:45.066 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (16m) 6m ago 25m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:35:45.066 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (16m) 6m ago 25m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:35:45.066 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (27m) 7m ago 27m 48.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:35:45.067 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.067+0000 7f9374975700 1 -- 172.21.15.27:0/2154126764 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f935c0640b0 msgr2=0x7f935c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:35:45.067 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.067+0000 7f9374975700 1 --2- 172.21.15.27:0/2154126764 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f935c0640b0 0x7f935c066570 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f937006b730 tx=0x7f936800b040).stop 2021-11-03T17:35:45.067 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.067+0000 7f9374975700 1 -- 172.21.15.27:0/2154126764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93700f6ff0 msgr2=0x7f9370069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:35:45.068 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.067+0000 7f9374975700 1 --2- 172.21.15.27:0/2154126764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93700f6ff0 0x7f9370069e20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f9358009990 tx=0x7f9358009b30).stop 2021-11-03T17:35:45.068 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.067+0000 7f9374975700 1 -- 172.21.15.27:0/2154126764 shutdown_connections 2021-11-03T17:35:45.068 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.067+0000 7f9374975700 1 --2- 172.21.15.27:0/2154126764 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f935c0640b0 0x7f935c066570 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:45.068 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.067+0000 7f9374975700 1 --2- 172.21.15.27:0/2154126764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93700f66e0 0x7f937006b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:45.069 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.067+0000 7f9374975700 1 --2- 172.21.15.27:0/2154126764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93700f6ff0 0x7f9370069e20 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:45.069 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.067+0000 7f9374975700 1 -- 172.21.15.27:0/2154126764 >> 172.21.15.27:0/2154126764 conn(0x7f93700f1f00 msgr2=0x7f93700fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:35:45.069 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.067+0000 7f9374975700 1 -- 172.21.15.27:0/2154126764 shutdown_connections 2021-11-03T17:35:45.069 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.067+0000 7f9374975700 1 -- 172.21.15.27:0/2154126764 wait complete. 2021-11-03T17:35:45.161 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.166+0000 7f43c3f5d700 1 -- 172.21.15.27:0/2976165417 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43bc0f7980 msgr2=0x7f43bc0f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:35:45.162 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.166+0000 7f43c3f5d700 1 --2- 172.21.15.27:0/2976165417 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43bc0f7980 0x7f43bc0f7de0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f43b0004660 tx=0x7f43b0009b30).stop 2021-11-03T17:35:45.162 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.166+0000 7f43c3f5d700 1 -- 172.21.15.27:0/2976165417 shutdown_connections 2021-11-03T17:35:45.162 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.166+0000 7f43c3f5d700 1 --2- 172.21.15.27:0/2976165417 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43bc0f7980 0x7f43bc0f7de0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:45.162 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.166+0000 7f43c3f5d700 1 --2- 172.21.15.27:0/2976165417 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43bc0f66e0 0x7f43bc0f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:45.163 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.166+0000 7f43c3f5d700 1 -- 172.21.15.27:0/2976165417 >> 172.21.15.27:0/2976165417 conn(0x7f43bc0f1e40 msgr2=0x7f43bc0f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:35:45.163 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.166+0000 7f43c3f5d700 1 -- 172.21.15.27:0/2976165417 shutdown_connections 2021-11-03T17:35:45.163 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.166+0000 7f43c3f5d700 1 -- 172.21.15.27:0/2976165417 wait complete. 2021-11-03T17:35:45.163 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.167+0000 7f43c3f5d700 1 Processor -- start 2021-11-03T17:35:45.163 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.167+0000 7f43c3f5d700 1 -- start start 2021-11-03T17:35:45.167 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.167+0000 7f43c3f5d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43bc0f66e0 0x7f43bc06b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:35:45.167 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.167+0000 7f43c3f5d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43bc0f7980 0x7f43bc069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:35:45.168 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.167+0000 7f43c3f5d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc06a380 con 0x7f43bc0f66e0 2021-11-03T17:35:45.168 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.167+0000 7f43c3f5d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43bc06a4c0 con 0x7f43bc0f7980 2021-11-03T17:35:45.168 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.167+0000 7f43c1cf9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43bc0f66e0 0x7f43bc06b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:35:45.168 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.167+0000 7f43c1cf9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43bc0f66e0 0x7f43bc06b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58328/0 (socket says 172.21.15.27:58328) 2021-11-03T17:35:45.169 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.167+0000 7f43c1cf9700 1 -- 172.21.15.27:0/2758160337 learned_addr learned my addr 172.21.15.27:0/2758160337 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:35:45.169 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.167+0000 7f43c14f8700 1 --2- 172.21.15.27:0/2758160337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43bc0f7980 0x7f43bc069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:35:45.169 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.167+0000 7f43c1cf9700 1 -- 172.21.15.27:0/2758160337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43bc0f7980 msgr2=0x7f43bc069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:35:45.170 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.167+0000 7f43c1cf9700 1 --2- 172.21.15.27:0/2758160337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43bc0f7980 0x7f43bc069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:45.170 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.167+0000 7f43c1cf9700 1 -- 172.21.15.27:0/2758160337 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b0005040 con 0x7f43bc0f66e0 2021-11-03T17:35:45.170 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.167+0000 7f43c14f8700 1 --2- 172.21.15.27:0/2758160337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43bc0f7980 0x7f43bc069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:35:45.170 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.167+0000 7f43c1cf9700 1 --2- 172.21.15.27:0/2758160337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43bc0f66e0 0x7f43bc06b9a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7f43b400e7c0 tx=0x7f43b400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:35:45.171 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.168+0000 7f43ba7fc700 1 -- 172.21.15.27:0/2758160337 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43b40092e0 con 0x7f43bc0f66e0 2021-11-03T17:35:45.171 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.168+0000 7f43ba7fc700 1 -- 172.21.15.27:0/2758160337 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f43b400f040 con 0x7f43bc0f66e0 2021-11-03T17:35:45.171 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.168+0000 7f43c3f5d700 1 -- 172.21.15.27:0/2758160337 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43bc06a7a0 con 0x7f43bc0f66e0 2021-11-03T17:35:45.171 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.168+0000 7f43ba7fc700 1 -- 172.21.15.27:0/2758160337 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43b4014750 con 0x7f43bc0f66e0 2021-11-03T17:35:45.172 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.168+0000 7f43c3f5d700 1 -- 172.21.15.27:0/2758160337 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43bc10ef40 con 0x7f43bc0f66e0 2021-11-03T17:35:45.172 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.168+0000 7f43c3f5d700 1 -- 172.21.15.27:0/2758160337 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43bc045ad0 con 0x7f43bc0f66e0 2021-11-03T17:35:45.172 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.169+0000 7f43ba7fc700 1 -- 172.21.15.27:0/2758160337 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f43b401b030 con 0x7f43bc0f66e0 2021-11-03T17:35:45.172 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.169+0000 7f43ba7fc700 1 --2- 172.21.15.27:0/2758160337 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f43a40640b0 0x7f43a4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:35:45.172 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.170+0000 7f43ba7fc700 1 -- 172.21.15.27:0/2758160337 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f43b40809c0 con 0x7f43bc0f66e0 2021-11-03T17:35:45.173 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.170+0000 7f43c14f8700 1 --2- 172.21.15.27:0/2758160337 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f43a40640b0 0x7f43a4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:35:45.173 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.170+0000 7f43c14f8700 1 --2- 172.21.15.27:0/2758160337 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f43a40640b0 0x7f43a4066570 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f43b0004d80 tx=0x7f43b000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:35:45.173 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.174+0000 7f43ba7fc700 1 -- 172.21.15.27:0/2758160337 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f43b4052bd0 con 0x7f43bc0f66e0 2021-11-03T17:35:45.397 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.402+0000 7f43c3f5d700 1 -- 172.21.15.27:0/2758160337 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f43bc0fefe0 con 0x7f43bc0f66e0 2021-11-03T17:35:45.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.402+0000 7f43ba7fc700 1 -- 172.21.15.27:0/2758160337 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f43b40527c0 con 0x7f43bc0f66e0 2021-11-03T17:35:45.398 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:35:45.398 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:35:45.399 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:35:45.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:35:45.399 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:35:45.399 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:35:45.399 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:35:45.400 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:35:45.400 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:35:45.400 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:35:45.400 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:35:45.401 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:35:45.401 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:35:45.401 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:35:45.401 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:35:45.401 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:35:45.402 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:35:45.402 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:35:45.403 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.404+0000 7f43c3f5d700 1 -- 172.21.15.27:0/2758160337 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f43a40640b0 msgr2=0x7f43a4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:35:45.403 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.404+0000 7f43c3f5d700 1 --2- 172.21.15.27:0/2758160337 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f43a40640b0 0x7f43a4066570 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 rx=0x7f43b0004d80 tx=0x7f43b000b040).stop 2021-11-03T17:35:45.403 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.404+0000 7f43c3f5d700 1 -- 172.21.15.27:0/2758160337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43bc0f66e0 msgr2=0x7f43bc06b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:35:45.404 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.404+0000 7f43c3f5d700 1 --2- 172.21.15.27:0/2758160337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43bc0f66e0 0x7f43bc06b9a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7f43b400e7c0 tx=0x7f43b400b040).stop 2021-11-03T17:35:45.404 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.404+0000 7f43c3f5d700 1 -- 172.21.15.27:0/2758160337 shutdown_connections 2021-11-03T17:35:45.404 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.404+0000 7f43c3f5d700 1 --2- 172.21.15.27:0/2758160337 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f43a40640b0 0x7f43a4066570 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:45.404 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.404+0000 7f43c3f5d700 1 --2- 172.21.15.27:0/2758160337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43bc0f66e0 0x7f43bc06b9a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:45.405 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.404+0000 7f43c3f5d700 1 --2- 172.21.15.27:0/2758160337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43bc0f7980 0x7f43bc069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:45.405 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.404+0000 7f43c3f5d700 1 -- 172.21.15.27:0/2758160337 >> 172.21.15.27:0/2758160337 conn(0x7f43bc0f1e40 msgr2=0x7f43bc0fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:35:45.405 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.404+0000 7f43c3f5d700 1 -- 172.21.15.27:0/2758160337 shutdown_connections 2021-11-03T17:35:45.405 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.404+0000 7f43c3f5d700 1 -- 172.21.15.27:0/2758160337 wait complete. 2021-11-03T17:35:45.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1187.387017 secs 2021-11-03T17:35:45.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:45 smithi115 ceph-mon[65453]: from='client.34760 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:35:45.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:45 smithi115 ceph-mon[65453]: from='client.34764 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:35:45.501 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.504+0000 7f800b9e5700 1 -- 172.21.15.27:0/951929780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8004060290 msgr2=0x7f80040f9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:35:45.502 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.505+0000 7f800b9e5700 1 --2- 172.21.15.27:0/951929780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8004060290 0x7f80040f9400 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f8000004690 tx=0x7f8000009b30).stop 2021-11-03T17:35:45.502 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.505+0000 7f800b9e5700 1 -- 172.21.15.27:0/951929780 shutdown_connections 2021-11-03T17:35:45.502 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.505+0000 7f800b9e5700 1 --2- 172.21.15.27:0/951929780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80040f98f0 0x7f80040fbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:45.503 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.505+0000 7f800b9e5700 1 --2- 172.21.15.27:0/951929780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8004060290 0x7f80040f9400 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:45.503 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.505+0000 7f800b9e5700 1 -- 172.21.15.27:0/951929780 >> 172.21.15.27:0/951929780 conn(0x7f80040f0de0 msgr2=0x7f80040f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:35:45.503 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.505+0000 7f800b9e5700 1 -- 172.21.15.27:0/951929780 shutdown_connections 2021-11-03T17:35:45.503 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.505+0000 7f800b9e5700 1 -- 172.21.15.27:0/951929780 wait complete. 2021-11-03T17:35:45.504 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.505+0000 7f800b9e5700 1 Processor -- start 2021-11-03T17:35:45.504 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.505+0000 7f800b9e5700 1 -- start start 2021-11-03T17:35:45.504 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.505+0000 7f800b9e5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8004060290 0x7f80040f6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:35:45.504 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.505+0000 7f800b9e5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80040f98f0 0x7f80040f5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:35:45.505 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.505+0000 7f800b9e5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80040f5860 con 0x7f80040f98f0 2021-11-03T17:35:45.505 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.505+0000 7f800b9e5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80040f59a0 con 0x7f8004060290 2021-11-03T17:35:45.505 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.506+0000 7f8008f80700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80040f98f0 0x7f80040f5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:35:45.505 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.506+0000 7f8008f80700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80040f98f0 0x7f80040f5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58340/0 (socket says 172.21.15.27:58340) 2021-11-03T17:35:45.506 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.506+0000 7f8008f80700 1 -- 172.21.15.27:0/1267692172 learned_addr learned my addr 172.21.15.27:0/1267692172 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:35:45.506 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.506+0000 7f8009781700 1 --2- 172.21.15.27:0/1267692172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8004060290 0x7f80040f6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:35:45.506 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.506+0000 7f8008f80700 1 -- 172.21.15.27:0/1267692172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8004060290 msgr2=0x7f80040f6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:35:45.506 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.506+0000 7f8008f80700 1 --2- 172.21.15.27:0/1267692172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8004060290 0x7f80040f6e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:45.507 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.506+0000 7f8008f80700 1 -- 172.21.15.27:0/1267692172 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8000005040 con 0x7f80040f98f0 2021-11-03T17:35:45.507 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.506+0000 7f8009781700 1 --2- 172.21.15.27:0/1267692172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8004060290 0x7f80040f6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:35:45.507 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.506+0000 7f8008f80700 1 --2- 172.21.15.27:0/1267692172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80040f98f0 0x7f80040f5300 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f7ff40026d0 tx=0x7f7ff4009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:35:45.507 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.506+0000 7f7ff9ffb700 1 -- 172.21.15.27:0/1267692172 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ff4009320 con 0x7f80040f98f0 2021-11-03T17:35:45.508 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.506+0000 7f7ff9ffb700 1 -- 172.21.15.27:0/1267692172 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f7ff400f040 con 0x7f80040f98f0 2021-11-03T17:35:45.508 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.506+0000 7f800b9e5700 1 -- 172.21.15.27:0/1267692172 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80040f5c80 con 0x7f80040f98f0 2021-11-03T17:35:45.508 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.506+0000 7f7ff9ffb700 1 -- 172.21.15.27:0/1267692172 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ff40147e0 con 0x7f80040f98f0 2021-11-03T17:35:45.508 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.506+0000 7f800b9e5700 1 -- 172.21.15.27:0/1267692172 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f800410ef40 con 0x7f80040f98f0 2021-11-03T17:35:45.510 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.507+0000 7f800b9e5700 1 -- 172.21.15.27:0/1267692172 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8004045ad0 con 0x7f80040f98f0 2021-11-03T17:35:45.510 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.508+0000 7f7ff9ffb700 1 -- 172.21.15.27:0/1267692172 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f7ff4009480 con 0x7f80040f98f0 2021-11-03T17:35:45.510 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.508+0000 7f7ff9ffb700 1 --2- 172.21.15.27:0/1267692172 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7ff00640b0 0x7f7ff0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:35:45.511 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.508+0000 7f7ff9ffb700 1 -- 172.21.15.27:0/1267692172 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f7ff4080e50 con 0x7f80040f98f0 2021-11-03T17:35:45.511 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.508+0000 7f8009781700 1 --2- 172.21.15.27:0/1267692172 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7ff00640b0 0x7f7ff0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:35:45.511 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.509+0000 7f8009781700 1 --2- 172.21.15.27:0/1267692172 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7ff00640b0 0x7f7ff0066570 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f8000000c00 tx=0x7f8000009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:35:45.511 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.512+0000 7f7ff9ffb700 1 -- 172.21.15.27:0/1267692172 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f7ff4053210 con 0x7f80040f98f0 2021-11-03T17:35:45.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1187.387017 secs 2021-11-03T17:35:45.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:45 smithi027 ceph-mon[97244]: from='client.34760 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:35:45.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:45 smithi027 ceph-mon[97244]: from='client.34764 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:35:45.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.718+0000 7f800b9e5700 1 -- 172.21.15.27:0/1267692172 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f80040f5e10 con 0x7f80040f98f0 2021-11-03T17:35:45.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.718+0000 7f7ff9ffb700 1 -- 172.21.15.27:0/1267692172 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f80040f5e10 con 0x7f80040f98f0 2021-11-03T17:35:45.715 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:35:45.716 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:35:45.716 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:35:45.716 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:35:45.716 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:35:45.717 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:35:45.717 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:35:45.717 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:35:45.717 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:35:45.718 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:35:45.718 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:35:45.718 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:35:45.719 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:35:45.719 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:35:45.719 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:35:45.720 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:35:45.720 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:35:45.720 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:35:45.721 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:35:45.721 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:35:45.721 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:35:45.721 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:35:45.721 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:35:45.722 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:35:45.722 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:35:45.722 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:35:45.722 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:35:45.723 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:35:45.723 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:35:45.723 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:35:45.724 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:35:45.724 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:35:45.724 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:35:45.724 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:35:45.725 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:35:45.725 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:35:45.725 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:35:45.726 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:35:45.726 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:35:45.727 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.720+0000 7f800b9e5700 1 -- 172.21.15.27:0/1267692172 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7ff00640b0 msgr2=0x7f7ff0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:35:45.727 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.720+0000 7f800b9e5700 1 --2- 172.21.15.27:0/1267692172 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7ff00640b0 0x7f7ff0066570 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 rx=0x7f8000000c00 tx=0x7f8000009b30).stop 2021-11-03T17:35:45.728 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.720+0000 7f800b9e5700 1 -- 172.21.15.27:0/1267692172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80040f98f0 msgr2=0x7f80040f5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:35:45.728 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.720+0000 7f800b9e5700 1 --2- 172.21.15.27:0/1267692172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80040f98f0 0x7f80040f5300 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f7ff40026d0 tx=0x7f7ff4009de0).stop 2021-11-03T17:35:45.728 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.720+0000 7f800b9e5700 1 -- 172.21.15.27:0/1267692172 shutdown_connections 2021-11-03T17:35:45.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.720+0000 7f800b9e5700 1 --2- 172.21.15.27:0/1267692172 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7ff00640b0 0x7f7ff0066570 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:45.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.720+0000 7f800b9e5700 1 --2- 172.21.15.27:0/1267692172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8004060290 0x7f80040f6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:45.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.720+0000 7f800b9e5700 1 --2- 172.21.15.27:0/1267692172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80040f98f0 0x7f80040f5300 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:35:45.730 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.720+0000 7f800b9e5700 1 -- 172.21.15.27:0/1267692172 >> 172.21.15.27:0/1267692172 conn(0x7f80040f0de0 msgr2=0x7f80040f31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:35:45.730 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.721+0000 7f800b9e5700 1 -- 172.21.15.27:0/1267692172 shutdown_connections 2021-11-03T17:35:45.730 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:35:45.721+0000 7f800b9e5700 1 -- 172.21.15.27:0/1267692172 wait complete. 2021-11-03T17:35:45.731 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:35:46.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:46 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2758160337' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:46.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:46 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1267692172' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:35:46.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:46 smithi115 ceph-mon[65453]: pgmap v634: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:46.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:46 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2758160337' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:46.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:46 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1267692172' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:35:46.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:46 smithi027 ceph-mon[97244]: pgmap v634: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:48.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:48 smithi115 ceph-mon[65453]: pgmap v635: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:48.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:48 smithi027 ceph-mon[97244]: pgmap v635: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:49.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:49 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1192.387160 secs 2021-11-03T17:35:50.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:49 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1192.387160 secs 2021-11-03T17:35:50.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:50 smithi115 ceph-mon[65453]: pgmap v636: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:51.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:50 smithi027 ceph-mon[97244]: pgmap v636: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:52.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:52 smithi115 ceph-mon[65453]: pgmap v637: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:52.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:52 smithi027 ceph-mon[97244]: pgmap v637: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:53.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:53 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:35:53.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:53 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:35:53.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:53 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:35:53.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:53 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:53.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:53 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:53.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:53 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:53.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:53 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:53.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:53 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:35:53.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:53 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:35:53.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:53 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:35:53.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:53 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:35:53.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:53 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:53.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:53 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:53.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:53 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:53.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:53 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:35:53.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:53 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:35:54.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:54 smithi115 ceph-mon[65453]: pgmap v638: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:54.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:54 smithi027 ceph-mon[97244]: pgmap v638: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:55.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1197.387290 secs 2021-11-03T17:35:55.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1197.387290 secs 2021-11-03T17:35:56.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:35:56.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:35:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:56 smithi115 ceph-mon[65453]: pgmap v639: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:35:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:35:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:56 smithi027 ceph-mon[97244]: pgmap v639: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:58.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:58 smithi115 ceph-mon[65453]: pgmap v640: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:58.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:58 smithi027 ceph-mon[97244]: pgmap v640: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:35:59.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:35:59 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1202.387425 secs 2021-11-03T17:36:00.017 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:35:59 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1202.387425 secs 2021-11-03T17:36:00.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:00 smithi115 ceph-mon[65453]: pgmap v641: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:01.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:00 smithi027 ceph-mon[97244]: pgmap v641: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:02.448 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:02 smithi115 ceph-mon[65453]: pgmap v642: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:02.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:02 smithi027 ceph-mon[97244]: pgmap v642: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:03.450 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:36:03.450 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:03 smithi115 ceph-mon[65453]: poll took 60172.874 ms: 1 events 2021-11-03T17:36:03.451 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:36:03.451 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:36:03.451 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:03.451 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:03.452 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:03.452 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:03.452 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:03 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:36:03.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:36:03.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:03 smithi027 ceph-mon[97244]: poll took 60172.874 ms: 1 events 2021-11-03T17:36:03.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:36:03.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:36:03.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:03.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:03.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:03.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:03.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:03 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:36:04.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:04 smithi115 ceph-mon[65453]: pgmap v643: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:04.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:04 smithi027 ceph-mon[97244]: pgmap v643: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:05.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1207.387538 secs 2021-11-03T17:36:05.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1207.387538 secs 2021-11-03T17:36:06.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:06 smithi115 ceph-mon[65453]: pgmap v644: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:06.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:06 smithi027 ceph-mon[97244]: pgmap v644: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:08.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:08 smithi115 ceph-mon[65453]: pgmap v645: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:08.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:08 smithi027 ceph-mon[97244]: pgmap v645: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:09.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:09 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1212.387662 secs 2021-11-03T17:36:10.017 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:09 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1212.387662 secs 2021-11-03T17:36:10.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:10 smithi115 ceph-mon[65453]: pgmap v646: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:11.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:10 smithi027 ceph-mon[97244]: pgmap v646: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:12.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:12 smithi115 ceph-mon[65453]: pgmap v647: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:12.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:12 smithi027 ceph-mon[97244]: pgmap v647: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:13.186 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:36:13.187 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"}]: dispatch 2021-11-03T17:36:13.187 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2021-11-03T17:36:13.187 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:36:13.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:36:13.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"}]: dispatch 2021-11-03T17:36:13.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2021-11-03T17:36:13.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:36:13.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:36:13.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:13.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:13.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:13.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:13.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:13 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:36:13.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:36:13.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:13.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:13.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:13.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:13.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:13 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:36:14.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:14 smithi115 ceph-mon[65453]: pgmap v648: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:14.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:14 smithi027 ceph-mon[97244]: pgmap v648: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:15.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1217.387795 secs 2021-11-03T17:36:15.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1217.387795 secs 2021-11-03T17:36:15.833 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:15.836+0000 7f1e78698700 1 -- 172.21.15.27:0/834431666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e700f5ef0 msgr2=0x7f1e700f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:36:15.834 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:15.836+0000 7f1e78698700 1 --2- 172.21.15.27:0/834431666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e700f5ef0 0x7f1e700f6920 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f1e6c004660 tx=0x7f1e6c009b30).stop 2021-11-03T17:36:15.834 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:15.836+0000 7f1e78698700 1 -- 172.21.15.27:0/834431666 shutdown_connections 2021-11-03T17:36:15.834 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:15.836+0000 7f1e78698700 1 --2- 172.21.15.27:0/834431666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e700f5ef0 0x7f1e700f6920 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:15.835 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:15.836+0000 7f1e78698700 1 --2- 172.21.15.27:0/834431666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e700f55e0 0x7f1e700f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:15.835 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:15.836+0000 7f1e78698700 1 -- 172.21.15.27:0/834431666 >> 172.21.15.27:0/834431666 conn(0x7f1e700f0de0 msgr2=0x7f1e700f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:36:15.835 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:15.836+0000 7f1e78698700 1 -- 172.21.15.27:0/834431666 shutdown_connections 2021-11-03T17:36:15.835 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:15.836+0000 7f1e78698700 1 -- 172.21.15.27:0/834431666 wait complete. 2021-11-03T17:36:15.836 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:15.837+0000 7f1e78698700 1 Processor -- start 2021-11-03T17:36:15.836 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:15.837+0000 7f1e78698700 1 -- start start 2021-11-03T17:36:15.836 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:15.837+0000 7f1e78698700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e700f55e0 0x7f1e70105e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:36:15.836 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:15.837+0000 7f1e78698700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e700f5ef0 0x7f1e70106380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:36:15.837 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:15.837+0000 7f1e78698700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e70106930 con 0x7f1e700f55e0 2021-11-03T17:36:15.837 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:15.837+0000 7f1e78698700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e70106a70 con 0x7f1e700f5ef0 2021-11-03T17:36:15.837 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:15.838+0000 7f1e76434700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e700f55e0 0x7f1e70105e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:36:15.837 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:15.838+0000 7f1e76434700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e700f55e0 0x7f1e70105e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58356/0 (socket says 172.21.15.27:58356) 2021-11-03T17:36:15.838 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:15.838+0000 7f1e76434700 1 -- 172.21.15.27:0/3277081112 learned_addr learned my addr 172.21.15.27:0/3277081112 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:36:15.838 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:15.838+0000 7f1e75c33700 1 --2- 172.21.15.27:0/3277081112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e700f5ef0 0x7f1e70106380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:36:15.838 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:15.838+0000 7f1e76434700 1 -- 172.21.15.27:0/3277081112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e700f5ef0 msgr2=0x7f1e70106380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:36:15.838 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:15.838+0000 7f1e76434700 1 --2- 172.21.15.27:0/3277081112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e700f5ef0 0x7f1e70106380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:15.839 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:15.838+0000 7f1e76434700 1 -- 172.21.15.27:0/3277081112 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e6c005040 con 0x7f1e700f55e0 2021-11-03T17:36:15.839 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:15.838+0000 7f1e76434700 1 --2- 172.21.15.27:0/3277081112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e700f55e0 0x7f1e70105e90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f1e600026d0 tx=0x7f1e60009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:36:15.839 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:15.838+0000 7f1e66ffd700 1 -- 172.21.15.27:0/3277081112 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e600079c0 con 0x7f1e700f55e0 2021-11-03T17:36:15.839 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:15.838+0000 7f1e66ffd700 1 -- 172.21.15.27:0/3277081112 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f1e6000f040 con 0x7f1e700f55e0 2021-11-03T17:36:15.840 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:15.838+0000 7f1e78698700 1 -- 172.21.15.27:0/3277081112 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e70107410 con 0x7f1e700f55e0 2021-11-03T17:36:15.840 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:15.838+0000 7f1e66ffd700 1 -- 172.21.15.27:0/3277081112 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e600148b0 con 0x7f1e700f55e0 2021-11-03T17:36:15.840 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:15.838+0000 7f1e78698700 1 -- 172.21.15.27:0/3277081112 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e701079d0 con 0x7f1e700f55e0 2021-11-03T17:36:15.841 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:15.839+0000 7f1e78698700 1 -- 172.21.15.27:0/3277081112 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e70045ad0 con 0x7f1e700f55e0 2021-11-03T17:36:15.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:15.840+0000 7f1e66ffd700 1 -- 172.21.15.27:0/3277081112 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f1e60007b20 con 0x7f1e700f55e0 2021-11-03T17:36:15.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:15.840+0000 7f1e66ffd700 1 --2- 172.21.15.27:0/3277081112 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1e5c0640b0 0x7f1e5c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:36:15.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:15.840+0000 7f1e66ffd700 1 -- 172.21.15.27:0/3277081112 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f1e60080790 con 0x7f1e700f55e0 2021-11-03T17:36:15.843 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:15.840+0000 7f1e75c33700 1 --2- 172.21.15.27:0/3277081112 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1e5c0640b0 0x7f1e5c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:36:15.843 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:15.841+0000 7f1e75c33700 1 --2- 172.21.15.27:0/3277081112 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1e5c0640b0 0x7f1e5c066570 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f1e6c004630 tx=0x7f1e6c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:36:15.843 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:15.844+0000 7f1e66ffd700 1 -- 172.21.15.27:0/3277081112 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f1e60052a90 con 0x7f1e700f55e0 2021-11-03T17:36:16.018 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.023+0000 7f1e78698700 1 -- 172.21.15.27:0/3277081112 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e700fdf50 con 0x7f1e5c0640b0 2021-11-03T17:36:16.019 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.024+0000 7f1e66ffd700 1 -- 172.21.15.27:0/3277081112 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f1e700fdf50 con 0x7f1e5c0640b0 2021-11-03T17:36:16.021 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.026+0000 7f1e78698700 1 -- 172.21.15.27:0/3277081112 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1e5c0640b0 msgr2=0x7f1e5c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:36:16.022 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.026+0000 7f1e78698700 1 --2- 172.21.15.27:0/3277081112 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1e5c0640b0 0x7f1e5c066570 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 rx=0x7f1e6c004630 tx=0x7f1e6c009b30).stop 2021-11-03T17:36:16.022 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.026+0000 7f1e78698700 1 -- 172.21.15.27:0/3277081112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e700f55e0 msgr2=0x7f1e70105e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:36:16.023 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.026+0000 7f1e78698700 1 --2- 172.21.15.27:0/3277081112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e700f55e0 0x7f1e70105e90 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f1e600026d0 tx=0x7f1e60009de0).stop 2021-11-03T17:36:16.023 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.026+0000 7f1e78698700 1 -- 172.21.15.27:0/3277081112 shutdown_connections 2021-11-03T17:36:16.024 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.026+0000 7f1e78698700 1 --2- 172.21.15.27:0/3277081112 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1e5c0640b0 0x7f1e5c066570 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:16.024 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.026+0000 7f1e78698700 1 --2- 172.21.15.27:0/3277081112 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e700f55e0 0x7f1e70105e90 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:16.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.026+0000 7f1e78698700 1 --2- 172.21.15.27:0/3277081112 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e700f5ef0 0x7f1e70106380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:16.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.026+0000 7f1e78698700 1 -- 172.21.15.27:0/3277081112 >> 172.21.15.27:0/3277081112 conn(0x7f1e700f0de0 msgr2=0x7f1e700fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:36:16.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.026+0000 7f1e78698700 1 -- 172.21.15.27:0/3277081112 shutdown_connections 2021-11-03T17:36:16.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.026+0000 7f1e78698700 1 -- 172.21.15.27:0/3277081112 wait complete. 2021-11-03T17:36:16.031 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:36:16.122 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.127+0000 7f3eaa568700 1 -- 172.21.15.27:0/2948745565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ea40f66e0 msgr2=0x7f3ea40f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:36:16.123 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.127+0000 7f3eaa568700 1 --2- 172.21.15.27:0/2948745565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ea40f66e0 0x7f3ea40f6b00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7f3e94004660 tx=0x7f3e94009b30).stop 2021-11-03T17:36:16.123 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.127+0000 7f3eaa568700 1 -- 172.21.15.27:0/2948745565 shutdown_connections 2021-11-03T17:36:16.123 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.127+0000 7f3eaa568700 1 --2- 172.21.15.27:0/2948745565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ea40f7980 0x7f3ea40f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:16.124 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.127+0000 7f3eaa568700 1 --2- 172.21.15.27:0/2948745565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ea40f66e0 0x7f3ea40f6b00 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:16.124 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.127+0000 7f3eaa568700 1 -- 172.21.15.27:0/2948745565 >> 172.21.15.27:0/2948745565 conn(0x7f3ea40f1e40 msgr2=0x7f3ea40f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:36:16.124 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.127+0000 7f3eaa568700 1 -- 172.21.15.27:0/2948745565 shutdown_connections 2021-11-03T17:36:16.124 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.127+0000 7f3eaa568700 1 -- 172.21.15.27:0/2948745565 wait complete. 2021-11-03T17:36:16.127 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.128+0000 7f3eaa568700 1 Processor -- start 2021-11-03T17:36:16.127 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.128+0000 7f3eaa568700 1 -- start start 2021-11-03T17:36:16.127 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.129+0000 7f3eaa568700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ea40f66e0 0x7f3ea4106fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:36:16.127 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.129+0000 7f3eaa568700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ea40f7980 0x7f3ea4107490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:36:16.128 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.129+0000 7f3eaa568700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea4107a40 con 0x7f3ea40f66e0 2021-11-03T17:36:16.128 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.129+0000 7f3eaa568700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea4107b80 con 0x7f3ea40f7980 2021-11-03T17:36:16.128 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.129+0000 7f3ea3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ea40f66e0 0x7f3ea4106fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:36:16.128 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.129+0000 7f3ea3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ea40f66e0 0x7f3ea4106fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58366/0 (socket says 172.21.15.27:58366) 2021-11-03T17:36:16.129 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.129+0000 7f3ea3fff700 1 -- 172.21.15.27:0/3009358503 learned_addr learned my addr 172.21.15.27:0/3009358503 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:36:16.129 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.129+0000 7f3ea37fe700 1 --2- 172.21.15.27:0/3009358503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ea40f7980 0x7f3ea4107490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:36:16.129 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.129+0000 7f3ea3fff700 1 -- 172.21.15.27:0/3009358503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ea40f7980 msgr2=0x7f3ea4107490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:36:16.129 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.129+0000 7f3ea3fff700 1 --2- 172.21.15.27:0/3009358503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ea40f7980 0x7f3ea4107490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:16.130 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.129+0000 7f3ea3fff700 1 -- 172.21.15.27:0/3009358503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e94005040 con 0x7f3ea40f66e0 2021-11-03T17:36:16.130 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.129+0000 7f3ea3fff700 1 --2- 172.21.15.27:0/3009358503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ea40f66e0 0x7f3ea4106fa0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f3e94004f60 tx=0x7f3e94009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:36:16.130 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.129+0000 7f3ea0ff9700 1 -- 172.21.15.27:0/3009358503 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e94018070 con 0x7f3ea40f66e0 2021-11-03T17:36:16.130 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.129+0000 7f3ea0ff9700 1 -- 172.21.15.27:0/3009358503 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f3e94007ba0 con 0x7f3ea40f66e0 2021-11-03T17:36:16.130 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.129+0000 7f3ea0ff9700 1 -- 172.21.15.27:0/3009358503 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3e9401c620 con 0x7f3ea40f66e0 2021-11-03T17:36:16.131 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.129+0000 7f3eaa568700 1 -- 172.21.15.27:0/3009358503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ea41084c0 con 0x7f3ea40f66e0 2021-11-03T17:36:16.131 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.129+0000 7f3eaa568700 1 -- 172.21.15.27:0/3009358503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ea4108a20 con 0x7f3ea40f66e0 2021-11-03T17:36:16.131 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.131+0000 7f3ea0ff9700 1 -- 172.21.15.27:0/3009358503 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f3e9401c8e0 con 0x7f3ea40f66e0 2021-11-03T17:36:16.131 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.131+0000 7f3eaa568700 1 -- 172.21.15.27:0/3009358503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ea4045ad0 con 0x7f3ea40f66e0 2021-11-03T17:36:16.132 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.131+0000 7f3ea0ff9700 1 --2- 172.21.15.27:0/3009358503 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3e8c0640b0 0x7f3e8c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:36:16.132 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.131+0000 7f3ea0ff9700 1 -- 172.21.15.27:0/3009358503 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f3e94082740 con 0x7f3ea40f66e0 2021-11-03T17:36:16.132 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.131+0000 7f3ea37fe700 1 --2- 172.21.15.27:0/3009358503 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3e8c0640b0 0x7f3e8c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:36:16.132 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.132+0000 7f3ea37fe700 1 --2- 172.21.15.27:0/3009358503 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3e8c0640b0 0x7f3e8c066570 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f3e980041d0 tx=0x7f3e98009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:36:16.133 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.135+0000 7f3ea0ff9700 1 -- 172.21.15.27:0/3009358503 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f3e94054a40 con 0x7f3ea40f66e0 2021-11-03T17:36:16.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.310+0000 7f3eaa568700 1 -- 172.21.15.27:0/3009358503 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ea40fefb0 con 0x7f3e8c0640b0 2021-11-03T17:36:16.319 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.324+0000 7f3ea0ff9700 1 -- 172.21.15.27:0/3009358503 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f3ea40fefb0 con 0x7f3e8c0640b0 2021-11-03T17:36:16.320 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:36:16.320 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (27m) 7m ago 29m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:36:16.320 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (19m) 7m ago 29m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:36:16.320 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (19m) 6m ago 28m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:36:16.321 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (27m) 7m ago 28m 40.2M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:36:16.321 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (21m) 7m ago 24m 18.2M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:36:16.321 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (24m) 7m ago 24m 3841M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:36:16.321 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (20m) 6m ago 24m 17.6M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:36:16.321 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (21m) 6m ago 24m 19.3M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:36:16.322 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (21m) 7m ago 30m 497M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:36:16.322 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (20m) 6m ago 28m 415M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:36:16.322 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (20m) 7m ago 30m 119M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:36:16.322 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (20m) 6m ago 28m 71.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:36:16.323 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (28m) 7m ago 28m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:36:16.323 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (28m) 6m ago 28m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:36:16.323 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (19m) 7m ago 27m 290M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:36:16.323 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (18m) 7m ago 27m 317M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:36:16.323 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (18m) 7m ago 26m 229M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:36:16.324 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (17m) 6m ago 26m 329M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:36:16.324 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (17m) 6m ago 26m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:36:16.324 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (16m) 6m ago 25m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:36:16.324 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (27m) 7m ago 28m 48.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:36:16.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.326+0000 7f3eaa568700 1 -- 172.21.15.27:0/3009358503 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3e8c0640b0 msgr2=0x7f3e8c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:36:16.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.326+0000 7f3eaa568700 1 --2- 172.21.15.27:0/3009358503 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3e8c0640b0 0x7f3e8c066570 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f3e980041d0 tx=0x7f3e98009040).stop 2021-11-03T17:36:16.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.326+0000 7f3eaa568700 1 -- 172.21.15.27:0/3009358503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ea40f66e0 msgr2=0x7f3ea4106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:36:16.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.326+0000 7f3eaa568700 1 --2- 172.21.15.27:0/3009358503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ea40f66e0 0x7f3ea4106fa0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7f3e94004f60 tx=0x7f3e94009b30).stop 2021-11-03T17:36:16.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.326+0000 7f3eaa568700 1 -- 172.21.15.27:0/3009358503 shutdown_connections 2021-11-03T17:36:16.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.326+0000 7f3eaa568700 1 --2- 172.21.15.27:0/3009358503 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3e8c0640b0 0x7f3e8c066570 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:16.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.326+0000 7f3eaa568700 1 --2- 172.21.15.27:0/3009358503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ea40f66e0 0x7f3ea4106fa0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:16.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.326+0000 7f3eaa568700 1 --2- 172.21.15.27:0/3009358503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ea40f7980 0x7f3ea4107490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:16.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.326+0000 7f3eaa568700 1 -- 172.21.15.27:0/3009358503 >> 172.21.15.27:0/3009358503 conn(0x7f3ea40f1e40 msgr2=0x7f3ea40fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:36:16.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.326+0000 7f3eaa568700 1 -- 172.21.15.27:0/3009358503 shutdown_connections 2021-11-03T17:36:16.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.326+0000 7f3eaa568700 1 -- 172.21.15.27:0/3009358503 wait complete. 2021-11-03T17:36:16.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.426+0000 7ff9fda5d700 1 -- 172.21.15.27:0/3379970946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9f80f7b80 msgr2=0x7ff9f80f7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:36:16.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.426+0000 7ff9fda5d700 1 --2- 172.21.15.27:0/3379970946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9f80f7b80 0x7ff9f80f7fe0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7ff9e0004660 tx=0x7ff9e0009b30).stop 2021-11-03T17:36:16.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.426+0000 7ff9fda5d700 1 -- 172.21.15.27:0/3379970946 shutdown_connections 2021-11-03T17:36:16.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.426+0000 7ff9fda5d700 1 --2- 172.21.15.27:0/3379970946 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9f80f7b80 0x7ff9f80f7fe0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:16.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.426+0000 7ff9fda5d700 1 --2- 172.21.15.27:0/3379970946 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9f80f68e0 0x7ff9f80f6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:16.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.426+0000 7ff9fda5d700 1 -- 172.21.15.27:0/3379970946 >> 172.21.15.27:0/3379970946 conn(0x7ff9f80f2020 msgr2=0x7ff9f80f4460 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:36:16.423 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.426+0000 7ff9fda5d700 1 -- 172.21.15.27:0/3379970946 shutdown_connections 2021-11-03T17:36:16.423 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.426+0000 7ff9fda5d700 1 -- 172.21.15.27:0/3379970946 wait complete. 2021-11-03T17:36:16.425 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.427+0000 7ff9fda5d700 1 Processor -- start 2021-11-03T17:36:16.426 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.427+0000 7ff9fda5d700 1 -- start start 2021-11-03T17:36:16.426 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.427+0000 7ff9fda5d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9f80f68e0 0x7ff9f8107170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:36:16.426 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.427+0000 7ff9fda5d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9f80f7b80 0x7ff9f8107660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:36:16.426 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.427+0000 7ff9fda5d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f8107c10 con 0x7ff9f80f68e0 2021-11-03T17:36:16.427 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.427+0000 7ff9fda5d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9f8107d50 con 0x7ff9f80f7b80 2021-11-03T17:36:16.427 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.427+0000 7ff9f6ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9f80f68e0 0x7ff9f8107170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:36:16.427 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.427+0000 7ff9f6ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9f80f68e0 0x7ff9f8107170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58376/0 (socket says 172.21.15.27:58376) 2021-11-03T17:36:16.427 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.427+0000 7ff9f6ffd700 1 -- 172.21.15.27:0/752741792 learned_addr learned my addr 172.21.15.27:0/752741792 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:36:16.428 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.427+0000 7ff9f67fc700 1 --2- 172.21.15.27:0/752741792 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9f80f7b80 0x7ff9f8107660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:36:16.428 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.427+0000 7ff9f6ffd700 1 -- 172.21.15.27:0/752741792 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9f80f7b80 msgr2=0x7ff9f8107660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:36:16.428 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.427+0000 7ff9f6ffd700 1 --2- 172.21.15.27:0/752741792 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9f80f7b80 0x7ff9f8107660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:16.428 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.427+0000 7ff9f6ffd700 1 -- 172.21.15.27:0/752741792 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e0005040 con 0x7ff9f80f68e0 2021-11-03T17:36:16.428 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.427+0000 7ff9f6ffd700 1 --2- 172.21.15.27:0/752741792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9f80f68e0 0x7ff9f8107170 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7ff9f000d7f0 tx=0x7ff9f0004e80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:36:16.429 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.428+0000 7ff9ef7fe700 1 -- 172.21.15.27:0/752741792 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9f000b4f0 con 0x7ff9f80f68e0 2021-11-03T17:36:16.429 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.428+0000 7ff9ef7fe700 1 -- 172.21.15.27:0/752741792 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7ff9f000f040 con 0x7ff9f80f68e0 2021-11-03T17:36:16.429 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.428+0000 7ff9fda5d700 1 -- 172.21.15.27:0/752741792 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9f81086f0 con 0x7ff9f80f68e0 2021-11-03T17:36:16.429 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.428+0000 7ff9ef7fe700 1 -- 172.21.15.27:0/752741792 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9f0013980 con 0x7ff9f80f68e0 2021-11-03T17:36:16.430 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.428+0000 7ff9fda5d700 1 -- 172.21.15.27:0/752741792 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9f8108cb0 con 0x7ff9f80f68e0 2021-11-03T17:36:16.430 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.429+0000 7ff9fda5d700 1 -- 172.21.15.27:0/752741792 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9f8045ad0 con 0x7ff9f80f68e0 2021-11-03T17:36:16.430 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.429+0000 7ff9ef7fe700 1 -- 172.21.15.27:0/752741792 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7ff9f001a030 con 0x7ff9f80f68e0 2021-11-03T17:36:16.430 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.429+0000 7ff9ef7fe700 1 --2- 172.21.15.27:0/752741792 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff9e40640b0 0x7ff9e4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:36:16.431 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.429+0000 7ff9ef7fe700 1 -- 172.21.15.27:0/752741792 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7ff9f00808b0 con 0x7ff9f80f68e0 2021-11-03T17:36:16.431 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.429+0000 7ff9f67fc700 1 --2- 172.21.15.27:0/752741792 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff9e40640b0 0x7ff9e4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:36:16.431 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.430+0000 7ff9f67fc700 1 --2- 172.21.15.27:0/752741792 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff9e40640b0 0x7ff9e4066570 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7ff9e0004630 tx=0x7ff9e0009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:36:16.432 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.433+0000 7ff9ef7fe700 1 -- 172.21.15.27:0/752741792 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7ff9f0052ac0 con 0x7ff9f80f68e0 2021-11-03T17:36:16.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:16 smithi115 ceph-mon[65453]: from='client.34776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:36:16.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:16 smithi115 ceph-mon[65453]: pgmap v649: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:16.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:16 smithi027 ceph-mon[97244]: from='client.34776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:36:16.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:16 smithi027 ceph-mon[97244]: pgmap v649: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:16.662 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.667+0000 7ff9fda5d700 1 -- 172.21.15.27:0/752741792 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff9f80ff0a0 con 0x7ff9f80f68e0 2021-11-03T17:36:16.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.667+0000 7ff9ef7fe700 1 -- 172.21.15.27:0/752741792 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7ff9f00182e0 con 0x7ff9f80f68e0 2021-11-03T17:36:16.663 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:36:16.663 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:36:16.664 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:36:16.664 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:36:16.664 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:36:16.664 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:36:16.664 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:36:16.665 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:36:16.665 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:36:16.665 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:36:16.665 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:36:16.666 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:36:16.666 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:36:16.666 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:36:16.667 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:36:16.667 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:36:16.667 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:36:16.667 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:36:16.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.669+0000 7ff9fda5d700 1 -- 172.21.15.27:0/752741792 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff9e40640b0 msgr2=0x7ff9e4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:36:16.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.669+0000 7ff9fda5d700 1 --2- 172.21.15.27:0/752741792 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff9e40640b0 0x7ff9e4066570 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 rx=0x7ff9e0004630 tx=0x7ff9e0009b30).stop 2021-11-03T17:36:16.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.669+0000 7ff9fda5d700 1 -- 172.21.15.27:0/752741792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9f80f68e0 msgr2=0x7ff9f8107170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:36:16.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.669+0000 7ff9fda5d700 1 --2- 172.21.15.27:0/752741792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9f80f68e0 0x7ff9f8107170 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7ff9f000d7f0 tx=0x7ff9f0004e80).stop 2021-11-03T17:36:16.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.669+0000 7ff9fda5d700 1 -- 172.21.15.27:0/752741792 shutdown_connections 2021-11-03T17:36:16.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.669+0000 7ff9fda5d700 1 --2- 172.21.15.27:0/752741792 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff9e40640b0 0x7ff9e4066570 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:16.670 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.669+0000 7ff9fda5d700 1 --2- 172.21.15.27:0/752741792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9f80f68e0 0x7ff9f8107170 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:16.670 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.669+0000 7ff9fda5d700 1 --2- 172.21.15.27:0/752741792 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9f80f7b80 0x7ff9f8107660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:16.670 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.669+0000 7ff9fda5d700 1 -- 172.21.15.27:0/752741792 >> 172.21.15.27:0/752741792 conn(0x7ff9f80f2020 msgr2=0x7ff9f80fb5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:36:16.670 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.669+0000 7ff9fda5d700 1 -- 172.21.15.27:0/752741792 shutdown_connections 2021-11-03T17:36:16.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.669+0000 7ff9fda5d700 1 -- 172.21.15.27:0/752741792 wait complete. 2021-11-03T17:36:16.767 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.771+0000 7fbf1c859700 1 -- 172.21.15.27:0/3633516856 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf18060310 msgr2=0x7fbf180f9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:36:16.768 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.771+0000 7fbf1c859700 1 --2- 172.21.15.27:0/3633516856 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf18060310 0x7fbf180f9110 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7fbf100099e0 tx=0x7fbf10004d10).stop 2021-11-03T17:36:16.768 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.771+0000 7fbf1c859700 1 -- 172.21.15.27:0/3633516856 shutdown_connections 2021-11-03T17:36:16.768 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.771+0000 7fbf1c859700 1 --2- 172.21.15.27:0/3633516856 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf180f9690 0x7fbf180fbae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:36:16.768 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.771+0000 7fbf1c859700 1 --2- 172.21.15.27:0/3633516856 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf18060310 0x7fbf180f9110 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:16.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.771+0000 7fbf1c859700 1 -- 172.21.15.27:0/3633516856 >> 172.21.15.27:0/3633516856 conn(0x7fbf180f0b90 msgr2=0x7fbf180f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:36:16.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.771+0000 7fbf1c859700 1 -- 172.21.15.27:0/3633516856 shutdown_connections 2021-11-03T17:36:16.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.771+0000 7fbf1c859700 1 -- 172.21.15.27:0/3633516856 wait complete. 2021-11-03T17:36:16.770 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.771+0000 7fbf1c859700 1 Processor -- start 2021-11-03T17:36:16.770 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.771+0000 7fbf1c859700 1 -- start start 2021-11-03T17:36:16.770 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.771+0000 7fbf1c859700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf18060310 0x7fbf180f6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:36:16.771 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.772+0000 7fbf1c859700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf180f9690 0x7fbf180f5010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:36:16.771 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.772+0000 7fbf1c859700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf180f5600 con 0x7fbf180f9690 2021-11-03T17:36:16.772 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.772+0000 7fbf1c859700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf180f5740 con 0x7fbf18060310 2021-11-03T17:36:16.772 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.772+0000 7fbf1659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf180f9690 0x7fbf180f5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:36:16.773 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.772+0000 7fbf1659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf180f9690 0x7fbf180f5010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58388/0 (socket says 172.21.15.27:58388) 2021-11-03T17:36:16.773 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.772+0000 7fbf1659c700 1 -- 172.21.15.27:0/2575977052 learned_addr learned my addr 172.21.15.27:0/2575977052 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:36:16.773 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.772+0000 7fbf16d9d700 1 --2- 172.21.15.27:0/2575977052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf18060310 0x7fbf180f6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:36:16.774 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.772+0000 7fbf1659c700 1 -- 172.21.15.27:0/2575977052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf18060310 msgr2=0x7fbf180f6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:36:16.774 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.772+0000 7fbf1659c700 1 --2- 172.21.15.27:0/2575977052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf18060310 0x7fbf180f6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:16.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.772+0000 7fbf1659c700 1 -- 172.21.15.27:0/2575977052 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf10009640 con 0x7fbf180f9690 2021-11-03T17:36:16.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.772+0000 7fbf1659c700 1 --2- 172.21.15.27:0/2575977052 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf180f9690 0x7fbf180f5010 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7fbf000093f0 tx=0x7fbf00006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:36:16.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.772+0000 7fbf0f7fe700 1 -- 172.21.15.27:0/2575977052 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf00009ea0 con 0x7fbf180f9690 2021-11-03T17:36:16.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.772+0000 7fbf1c859700 1 -- 172.21.15.27:0/2575977052 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf180f5a20 con 0x7fbf180f9690 2021-11-03T17:36:16.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.772+0000 7fbf0f7fe700 1 -- 172.21.15.27:0/2575977052 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fbf00013af0 con 0x7fbf180f9690 2021-11-03T17:36:16.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.772+0000 7fbf0f7fe700 1 -- 172.21.15.27:0/2575977052 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbf0000db10 con 0x7fbf180f9690 2021-11-03T17:36:16.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.772+0000 7fbf1c859700 1 -- 172.21.15.27:0/2575977052 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf1810ef00 con 0x7fbf180f9690 2021-11-03T17:36:16.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.773+0000 7fbf1c859700 1 -- 172.21.15.27:0/2575977052 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf18045ad0 con 0x7fbf180f9690 2021-11-03T17:36:16.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.774+0000 7fbf0f7fe700 1 -- 172.21.15.27:0/2575977052 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fbf0001a060 con 0x7fbf180f9690 2021-11-03T17:36:16.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.774+0000 7fbf0f7fe700 1 --2- 172.21.15.27:0/2575977052 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbf04064100 0x7fbf040665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:36:16.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.774+0000 7fbf0f7fe700 1 -- 172.21.15.27:0/2575977052 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fbf00080970 con 0x7fbf180f9690 2021-11-03T17:36:16.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.774+0000 7fbf16d9d700 1 --2- 172.21.15.27:0/2575977052 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbf04064100 0x7fbf040665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:36:16.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.775+0000 7fbf16d9d700 1 --2- 172.21.15.27:0/2575977052 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbf04064100 0x7fbf040665c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fbf10009f50 tx=0x7fbf10004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:36:16.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.778+0000 7fbf0f7fe700 1 -- 172.21.15.27:0/2575977052 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fbf00052b80 con 0x7fbf180f9690 2021-11-03T17:36:16.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.981+0000 7fbf1c859700 1 -- 172.21.15.27:0/2575977052 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbf18027a00 con 0x7fbf180f9690 2021-11-03T17:36:16.977 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.981+0000 7fbf0f7fe700 1 -- 172.21.15.27:0/2575977052 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7fbf00018710 con 0x7fbf180f9690 2021-11-03T17:36:16.977 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:36:16.977 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:36:16.978 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:36:16.978 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:36:16.978 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:36:16.978 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:36:16.979 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:36:16.979 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:36:16.979 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:36:16.979 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:36:16.980 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:36:16.980 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:36:16.980 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:36:16.980 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:36:16.981 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:36:16.981 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:36:16.981 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:36:16.981 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:36:16.982 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:36:16.982 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:36:16.982 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:36:16.982 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:36:16.983 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:36:16.983 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:36:16.983 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:36:16.983 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:36:16.983 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:36:16.984 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:36:16.984 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:36:16.984 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:36:16.984 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:36:16.985 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:36:16.985 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:36:16.985 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:36:16.985 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:36:16.986 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:36:16.986 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:36:16.986 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:36:16.986 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:36:16.987 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.983+0000 7fbf1c859700 1 -- 172.21.15.27:0/2575977052 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbf04064100 msgr2=0x7fbf040665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:36:16.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.983+0000 7fbf1c859700 1 --2- 172.21.15.27:0/2575977052 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbf04064100 0x7fbf040665c0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7fbf10009f50 tx=0x7fbf10004d10).stop 2021-11-03T17:36:16.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.983+0000 7fbf1c859700 1 -- 172.21.15.27:0/2575977052 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf180f9690 msgr2=0x7fbf180f5010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:36:16.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.983+0000 7fbf1c859700 1 --2- 172.21.15.27:0/2575977052 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf180f9690 0x7fbf180f5010 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7fbf000093f0 tx=0x7fbf00006ec0).stop 2021-11-03T17:36:16.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.983+0000 7fbf1c859700 1 -- 172.21.15.27:0/2575977052 shutdown_connections 2021-11-03T17:36:16.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.983+0000 7fbf1c859700 1 --2- 172.21.15.27:0/2575977052 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbf04064100 0x7fbf040665c0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:16.989 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.983+0000 7fbf1c859700 1 --2- 172.21.15.27:0/2575977052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbf18060310 0x7fbf180f6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:16.989 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.983+0000 7fbf1c859700 1 --2- 172.21.15.27:0/2575977052 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbf180f9690 0x7fbf180f5010 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:16.989 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.983+0000 7fbf1c859700 1 -- 172.21.15.27:0/2575977052 >> 172.21.15.27:0/2575977052 conn(0x7fbf180f0b90 msgr2=0x7fbf180f2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:36:16.989 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.984+0000 7fbf1c859700 1 -- 172.21.15.27:0/2575977052 shutdown_connections 2021-11-03T17:36:16.990 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:16.984+0000 7fbf1c859700 1 -- 172.21.15.27:0/2575977052 wait complete. 2021-11-03T17:36:16.990 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:36:17.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:17 smithi115 ceph-mon[65453]: from='client.34780 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:36:17.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:17 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/752741792' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:17.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:17 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2575977052' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:36:17.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:17 smithi027 ceph-mon[97244]: from='client.34780 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:36:17.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:17 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/752741792' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:17.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:17 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2575977052' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:36:18.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:18 smithi115 ceph-mon[65453]: pgmap v650: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:18.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:18 smithi027 ceph-mon[97244]: pgmap v650: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:19.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:19 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1222.387928 secs 2021-11-03T17:36:20.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:19 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1222.387928 secs 2021-11-03T17:36:20.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:20 smithi115 ceph-mon[65453]: pgmap v651: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:21.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:20 smithi027 ceph-mon[97244]: pgmap v651: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:22.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:22 smithi115 ceph-mon[65453]: pgmap v652: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:22.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:22 smithi027 ceph-mon[97244]: pgmap v652: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:23.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:36:23.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:36:23.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:36:23.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:23.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:23.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:23.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:23.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:23 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:36:23.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:36:23.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:36:23.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:36:23.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:23.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:23.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:23.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:23.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:23 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:36:24.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:24 smithi115 ceph-mon[65453]: pgmap v653: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:24.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:24 smithi027 ceph-mon[97244]: pgmap v653: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:25.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1227.388054 secs 2021-11-03T17:36:25.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1227.388054 secs 2021-11-03T17:36:26.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:26 smithi115 ceph-mon[65453]: pgmap v654: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:26.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:26 smithi027 ceph-mon[97244]: pgmap v654: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:28.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:28 smithi115 ceph-mon[65453]: pgmap v655: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:28.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:28 smithi027 ceph-mon[97244]: pgmap v655: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:29.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:29 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1232.388187 secs 2021-11-03T17:36:30.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:29 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1232.388187 secs 2021-11-03T17:36:30.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:30 smithi115 ceph-mon[65453]: pgmap v656: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:31.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:30 smithi027 ceph-mon[97244]: pgmap v656: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:32.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:32 smithi115 ceph-mon[65453]: pgmap v657: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:32.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:32 smithi027 ceph-mon[97244]: pgmap v657: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:33.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:36:33.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:36:33.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:36:33.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:33.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:33.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:33.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:33.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:33 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:36:33.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:36:33.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:36:33.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:36:33.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:33.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:33.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:33.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:33.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:33 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:36:34.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:34 smithi115 ceph-mon[65453]: pgmap v658: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:34.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:34 smithi027 ceph-mon[97244]: pgmap v658: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:35.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1237.388316 secs 2021-11-03T17:36:35.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1237.388316 secs 2021-11-03T17:36:36.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:36 smithi115 ceph-mon[65453]: pgmap v659: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:36.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:36 smithi027 ceph-mon[97244]: pgmap v659: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:38.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:38 smithi115 ceph-mon[65453]: pgmap v660: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:38.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:38 smithi027 ceph-mon[97244]: pgmap v660: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:39.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:39 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1242.388452 secs 2021-11-03T17:36:40.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:39 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1242.388452 secs 2021-11-03T17:36:40.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:40 smithi115 ceph-mon[65453]: pgmap v661: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:41.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:40 smithi027 ceph-mon[97244]: pgmap v661: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:42.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:42 smithi115 ceph-mon[65453]: pgmap v662: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:42.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:42 smithi027 ceph-mon[97244]: pgmap v662: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:43.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:36:43.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:36:43.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:36:43.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:43.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:43.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:43.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:43.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:43 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:36:43.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:36:43.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:36:43.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:36:43.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:43.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:43.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:43.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:43.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:43 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:36:44.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:44 smithi115 ceph-mon[65453]: pgmap v663: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:44.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:44 smithi027 ceph-mon[97244]: pgmap v663: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:45.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1247.388575 secs 2021-11-03T17:36:45.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1247.388575 secs 2021-11-03T17:36:46.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:46 smithi115 ceph-mon[65453]: pgmap v664: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:46.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:46 smithi027 ceph-mon[97244]: pgmap v664: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:47.095 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.098+0000 7f6288e93700 1 -- 172.21.15.27:0/1387598344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62840f66e0 msgr2=0x7f62840f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:36:47.096 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.098+0000 7f6288e93700 1 --2- 172.21.15.27:0/1387598344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62840f66e0 0x7f62840f6b00 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f6274004660 tx=0x7f6274009b30).stop 2021-11-03T17:36:47.096 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.099+0000 7f6288e93700 1 -- 172.21.15.27:0/1387598344 shutdown_connections 2021-11-03T17:36:47.096 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.099+0000 7f6288e93700 1 --2- 172.21.15.27:0/1387598344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62840f6ff0 0x7f62840f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:47.096 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.099+0000 7f6288e93700 1 --2- 172.21.15.27:0/1387598344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62840f66e0 0x7f62840f6b00 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:47.097 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.099+0000 7f6288e93700 1 -- 172.21.15.27:0/1387598344 >> 172.21.15.27:0/1387598344 conn(0x7f62840f1f00 msgr2=0x7f62840f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:36:47.097 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.099+0000 7f6288e93700 1 -- 172.21.15.27:0/1387598344 shutdown_connections 2021-11-03T17:36:47.097 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.099+0000 7f6288e93700 1 -- 172.21.15.27:0/1387598344 wait complete. 2021-11-03T17:36:47.097 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.099+0000 7f6288e93700 1 Processor -- start 2021-11-03T17:36:47.098 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.099+0000 7f6288e93700 1 -- start start 2021-11-03T17:36:47.098 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.099+0000 7f6288e93700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62840f66e0 0x7f6284106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:36:47.098 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.099+0000 7f6288e93700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62840f6ff0 0x7f6284107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:36:47.098 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.099+0000 7f6288e93700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6284107a00 con 0x7f62840f6ff0 2021-11-03T17:36:47.099 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.099+0000 7f6288e93700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6284107b40 con 0x7f62840f66e0 2021-11-03T17:36:47.099 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.100+0000 7f6281d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62840f6ff0 0x7f6284107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:36:47.099 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.100+0000 7f6281d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62840f6ff0 0x7f6284107450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58406/0 (socket says 172.21.15.27:58406) 2021-11-03T17:36:47.099 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.100+0000 7f6281d9b700 1 -- 172.21.15.27:0/1087007479 learned_addr learned my addr 172.21.15.27:0/1087007479 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:36:47.100 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.100+0000 7f628259c700 1 --2- 172.21.15.27:0/1087007479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62840f66e0 0x7f6284106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:36:47.100 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.100+0000 7f6281d9b700 1 -- 172.21.15.27:0/1087007479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62840f66e0 msgr2=0x7f6284106f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:36:47.100 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.100+0000 7f6281d9b700 1 --2- 172.21.15.27:0/1087007479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62840f66e0 0x7f6284106f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:47.100 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.100+0000 7f6281d9b700 1 -- 172.21.15.27:0/1087007479 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6274005040 con 0x7f62840f6ff0 2021-11-03T17:36:47.101 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.100+0000 7f6281d9b700 1 --2- 172.21.15.27:0/1087007479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62840f6ff0 0x7f6284107450 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7f626c00c790 tx=0x7f626c007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:36:47.101 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.100+0000 7f627affd700 1 -- 172.21.15.27:0/1087007479 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f626c002590 con 0x7f62840f6ff0 2021-11-03T17:36:47.101 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.100+0000 7f627affd700 1 -- 172.21.15.27:0/1087007479 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f626c002ba0 con 0x7f62840f6ff0 2021-11-03T17:36:47.101 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.100+0000 7f627affd700 1 -- 172.21.15.27:0/1087007479 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f626c013d90 con 0x7f62840f6ff0 2021-11-03T17:36:47.102 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.100+0000 7f6288e93700 1 -- 172.21.15.27:0/1087007479 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62841084e0 con 0x7f62840f6ff0 2021-11-03T17:36:47.102 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.100+0000 7f6288e93700 1 -- 172.21.15.27:0/1087007479 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6284108aa0 con 0x7f62840f6ff0 2021-11-03T17:36:47.103 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.101+0000 7f6288e93700 1 -- 172.21.15.27:0/1087007479 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6284045ad0 con 0x7f62840f6ff0 2021-11-03T17:36:47.103 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.102+0000 7f627affd700 1 -- 172.21.15.27:0/1087007479 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f626c01a060 con 0x7f62840f6ff0 2021-11-03T17:36:47.103 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.102+0000 7f627affd700 1 --2- 172.21.15.27:0/1087007479 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f62700640b0 0x7f6270066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:36:47.104 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.102+0000 7f627affd700 1 -- 172.21.15.27:0/1087007479 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f626c080ae0 con 0x7f62840f6ff0 2021-11-03T17:36:47.104 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.102+0000 7f628259c700 1 --2- 172.21.15.27:0/1087007479 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f62700640b0 0x7f6270066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:36:47.104 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.103+0000 7f628259c700 1 --2- 172.21.15.27:0/1087007479 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f62700640b0 0x7f6270066570 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f6274004630 tx=0x7f6274009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:36:47.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.106+0000 7f627affd700 1 -- 172.21.15.27:0/1087007479 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f626c052c90 con 0x7f62840f6ff0 2021-11-03T17:36:47.281 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.286+0000 7f6288e93700 1 -- 172.21.15.27:0/1087007479 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f62840ff010 con 0x7f62700640b0 2021-11-03T17:36:47.283 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.288+0000 7f627affd700 1 -- 172.21.15.27:0/1087007479 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f62840ff010 con 0x7f62700640b0 2021-11-03T17:36:47.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.290+0000 7f6288e93700 1 -- 172.21.15.27:0/1087007479 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f62700640b0 msgr2=0x7f6270066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:36:47.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.290+0000 7f6288e93700 1 --2- 172.21.15.27:0/1087007479 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f62700640b0 0x7f6270066570 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f6274004630 tx=0x7f6274009b30).stop 2021-11-03T17:36:47.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.290+0000 7f6288e93700 1 -- 172.21.15.27:0/1087007479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62840f6ff0 msgr2=0x7f6284107450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:36:47.286 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.290+0000 7f6288e93700 1 --2- 172.21.15.27:0/1087007479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62840f6ff0 0x7f6284107450 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7f626c00c790 tx=0x7f626c007d70).stop 2021-11-03T17:36:47.286 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.290+0000 7f6288e93700 1 -- 172.21.15.27:0/1087007479 shutdown_connections 2021-11-03T17:36:47.286 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.290+0000 7f6288e93700 1 --2- 172.21.15.27:0/1087007479 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f62700640b0 0x7f6270066570 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:47.286 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.290+0000 7f6288e93700 1 --2- 172.21.15.27:0/1087007479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f62840f66e0 0x7f6284106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:47.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.290+0000 7f6288e93700 1 --2- 172.21.15.27:0/1087007479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f62840f6ff0 0x7f6284107450 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:47.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.290+0000 7f6288e93700 1 -- 172.21.15.27:0/1087007479 >> 172.21.15.27:0/1087007479 conn(0x7f62840f1f00 msgr2=0x7f62840fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:36:47.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.290+0000 7f6288e93700 1 -- 172.21.15.27:0/1087007479 shutdown_connections 2021-11-03T17:36:47.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.290+0000 7f6288e93700 1 -- 172.21.15.27:0/1087007479 wait complete. 2021-11-03T17:36:47.295 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:36:47.389 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.390+0000 7f64e128f700 1 -- 172.21.15.27:0/983562452 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64dc0f77d0 msgr2=0x7f64dc0f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:36:47.389 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.390+0000 7f64e128f700 1 --2- 172.21.15.27:0/983562452 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64dc0f77d0 0x7f64dc0f7c30 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7f64d0004660 tx=0x7f64d0009b30).stop 2021-11-03T17:36:47.389 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.390+0000 7f64e128f700 1 -- 172.21.15.27:0/983562452 shutdown_connections 2021-11-03T17:36:47.390 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.390+0000 7f64e128f700 1 --2- 172.21.15.27:0/983562452 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64dc0f77d0 0x7f64dc0f7c30 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:47.390 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.390+0000 7f64e128f700 1 --2- 172.21.15.27:0/983562452 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64dc0f6530 0x7f64dc0f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:47.390 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.390+0000 7f64e128f700 1 -- 172.21.15.27:0/983562452 >> 172.21.15.27:0/983562452 conn(0x7f64dc0f1c90 msgr2=0x7f64dc0f40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:36:47.391 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.390+0000 7f64e128f700 1 -- 172.21.15.27:0/983562452 shutdown_connections 2021-11-03T17:36:47.391 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.390+0000 7f64e128f700 1 -- 172.21.15.27:0/983562452 wait complete. 2021-11-03T17:36:47.391 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.391+0000 7f64e128f700 1 Processor -- start 2021-11-03T17:36:47.391 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.391+0000 7f64e128f700 1 -- start start 2021-11-03T17:36:47.391 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.391+0000 7f64e128f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64dc0f6530 0x7f64dc106d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:36:47.392 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.391+0000 7f64e128f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64dc0f77d0 0x7f64dc107280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:36:47.392 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.391+0000 7f64e128f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64dc107880 con 0x7f64dc0f6530 2021-11-03T17:36:47.392 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.391+0000 7f64e128f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64dc108090 con 0x7f64dc0f77d0 2021-11-03T17:36:47.392 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.391+0000 7f64dbfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64dc0f6530 0x7f64dc106d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:36:47.393 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.391+0000 7f64dbfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64dc0f6530 0x7f64dc106d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58414/0 (socket says 172.21.15.27:58414) 2021-11-03T17:36:47.393 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.391+0000 7f64dbfff700 1 -- 172.21.15.27:0/2428242145 learned_addr learned my addr 172.21.15.27:0/2428242145 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:36:47.393 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.392+0000 7f64db7fe700 1 --2- 172.21.15.27:0/2428242145 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64dc0f77d0 0x7f64dc107280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:36:47.393 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.392+0000 7f64dbfff700 1 -- 172.21.15.27:0/2428242145 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64dc0f77d0 msgr2=0x7f64dc107280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:36:47.394 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.392+0000 7f64dbfff700 1 --2- 172.21.15.27:0/2428242145 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64dc0f77d0 0x7f64dc107280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:47.394 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.392+0000 7f64dbfff700 1 -- 172.21.15.27:0/2428242145 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d0005040 con 0x7f64dc0f6530 2021-11-03T17:36:47.394 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.392+0000 7f64dbfff700 1 --2- 172.21.15.27:0/2428242145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64dc0f6530 0x7f64dc106d90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f64cc00c790 tx=0x7f64cc007d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:36:47.394 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.392+0000 7f64d8ff9700 1 -- 172.21.15.27:0/2428242145 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64cc00e680 con 0x7f64dc0f6530 2021-11-03T17:36:47.394 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.392+0000 7f64e128f700 1 -- 172.21.15.27:0/2428242145 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64dc108310 con 0x7f64dc0f6530 2021-11-03T17:36:47.395 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.392+0000 7f64e128f700 1 -- 172.21.15.27:0/2428242145 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64dc1088d0 con 0x7f64dc0f6530 2021-11-03T17:36:47.395 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.392+0000 7f64d8ff9700 1 -- 172.21.15.27:0/2428242145 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f64cc00ec90 con 0x7f64dc0f6530 2021-11-03T17:36:47.395 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.392+0000 7f64d8ff9700 1 -- 172.21.15.27:0/2428242145 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f64cc012c40 con 0x7f64dc0f6530 2021-11-03T17:36:47.395 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.393+0000 7f64e128f700 1 -- 172.21.15.27:0/2428242145 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64dc059100 con 0x7f64dc0f6530 2021-11-03T17:36:47.396 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.393+0000 7f64d8ff9700 1 -- 172.21.15.27:0/2428242145 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f64cc010070 con 0x7f64dc0f6530 2021-11-03T17:36:47.396 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.394+0000 7f64d8ff9700 1 --2- 172.21.15.27:0/2428242145 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f64c4064100 0x7f64c40665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:36:47.396 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.394+0000 7f64d8ff9700 1 -- 172.21.15.27:0/2428242145 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f64cc080a60 con 0x7f64dc0f6530 2021-11-03T17:36:47.396 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.394+0000 7f64db7fe700 1 --2- 172.21.15.27:0/2428242145 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f64c4064100 0x7f64c40665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:36:47.397 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.394+0000 7f64db7fe700 1 --2- 172.21.15.27:0/2428242145 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f64c4064100 0x7f64c40665c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f64d0004e30 tx=0x7f64d0009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:36:47.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.398+0000 7f64d8ff9700 1 -- 172.21.15.27:0/2428242145 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f64cc052c10 con 0x7f64dc0f6530 2021-11-03T17:36:47.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.570+0000 7f64e128f700 1 -- 172.21.15.27:0/2428242145 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f64dc0008d0 con 0x7f64c4064100 2021-11-03T17:36:47.580 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.585+0000 7f64d8ff9700 1 -- 172.21.15.27:0/2428242145 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f64dc0008d0 con 0x7f64c4064100 2021-11-03T17:36:47.580 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:36:47.581 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (28m) 8m ago 29m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:36:47.582 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (20m) 8m ago 29m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:36:47.583 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (19m) 7m ago 28m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:36:47.583 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (28m) 8m ago 29m 40.2M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:36:47.583 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (21m) 8m ago 24m 18.2M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:36:47.583 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (24m) 8m ago 24m 3841M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:36:47.584 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (20m) 7m ago 24m 17.6M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:36:47.584 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (22m) 7m ago 24m 19.3M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:36:47.584 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (22m) 8m ago 30m 497M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:36:47.584 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (21m) 7m ago 28m 415M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:36:47.585 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (21m) 8m ago 31m 119M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:36:47.585 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (20m) 7m ago 28m 71.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:36:47.585 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (29m) 8m ago 29m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:36:47.585 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (28m) 7m ago 28m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:36:47.586 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (19m) 8m ago 28m 290M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:36:47.586 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (19m) 8m ago 27m 317M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:36:47.586 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (18m) 8m ago 27m 229M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:36:47.586 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (18m) 7m ago 26m 329M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:36:47.587 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (17m) 7m ago 26m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:36:47.587 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (17m) 7m ago 26m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:36:47.587 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (28m) 8m ago 28m 48.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:36:47.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.586+0000 7f64e128f700 1 -- 172.21.15.27:0/2428242145 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f64c4064100 msgr2=0x7f64c40665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:36:47.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.586+0000 7f64e128f700 1 --2- 172.21.15.27:0/2428242145 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f64c4064100 0x7f64c40665c0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 rx=0x7f64d0004e30 tx=0x7f64d0009b30).stop 2021-11-03T17:36:47.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.586+0000 7f64e128f700 1 -- 172.21.15.27:0/2428242145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64dc0f6530 msgr2=0x7f64dc106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:36:47.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.586+0000 7f64e128f700 1 --2- 172.21.15.27:0/2428242145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64dc0f6530 0x7f64dc106d90 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f64cc00c790 tx=0x7f64cc007d90).stop 2021-11-03T17:36:47.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.587+0000 7f64e128f700 1 -- 172.21.15.27:0/2428242145 shutdown_connections 2021-11-03T17:36:47.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.587+0000 7f64e128f700 1 --2- 172.21.15.27:0/2428242145 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f64c4064100 0x7f64c40665c0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:47.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.587+0000 7f64e128f700 1 --2- 172.21.15.27:0/2428242145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f64dc0f6530 0x7f64dc106d90 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:47.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.587+0000 7f64e128f700 1 --2- 172.21.15.27:0/2428242145 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f64dc0f77d0 0x7f64dc107280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:47.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.587+0000 7f64e128f700 1 -- 172.21.15.27:0/2428242145 >> 172.21.15.27:0/2428242145 conn(0x7f64dc0f1c90 msgr2=0x7f64dc0fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:36:47.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.587+0000 7f64e128f700 1 -- 172.21.15.27:0/2428242145 shutdown_connections 2021-11-03T17:36:47.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.587+0000 7f64e128f700 1 -- 172.21.15.27:0/2428242145 wait complete. 2021-11-03T17:36:47.682 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.687+0000 7f0f09853700 1 -- 172.21.15.27:0/3912713971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f04060250 msgr2=0x7f0f040f9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:36:47.683 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.687+0000 7f0f09853700 1 --2- 172.21.15.27:0/3912713971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f04060250 0x7f0f040f9360 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f0efc0099e0 tx=0x7f0efc004d10).stop 2021-11-03T17:36:47.683 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.687+0000 7f0f09853700 1 -- 172.21.15.27:0/3912713971 shutdown_connections 2021-11-03T17:36:47.683 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.687+0000 7f0f09853700 1 --2- 172.21.15.27:0/3912713971 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f040f9850 0x7f0f040fbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:47.683 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.687+0000 7f0f09853700 1 --2- 172.21.15.27:0/3912713971 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f04060250 0x7f0f040f9360 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:47.684 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.687+0000 7f0f09853700 1 -- 172.21.15.27:0/3912713971 >> 172.21.15.27:0/3912713971 conn(0x7f0f040f0d40 msgr2=0x7f0f040f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:36:47.684 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.687+0000 7f0f09853700 1 -- 172.21.15.27:0/3912713971 shutdown_connections 2021-11-03T17:36:47.684 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.687+0000 7f0f09853700 1 -- 172.21.15.27:0/3912713971 wait complete. 2021-11-03T17:36:47.687 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.688+0000 7f0f09853700 1 Processor -- start 2021-11-03T17:36:47.687 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.688+0000 7f0f09853700 1 -- start start 2021-11-03T17:36:47.688 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.688+0000 7f0f09853700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f04060250 0x7f0f040f6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:36:47.688 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.688+0000 7f0f09853700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f040f9850 0x7f0f040f5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:36:47.688 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.688+0000 7f0f09853700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f040f57c0 con 0x7f0f040f9850 2021-11-03T17:36:47.688 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.688+0000 7f0f09853700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f040f5900 con 0x7f0f04060250 2021-11-03T17:36:47.689 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.688+0000 7f0f027fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f040f9850 0x7f0f040f5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:36:47.689 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.688+0000 7f0f027fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f040f9850 0x7f0f040f5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58426/0 (socket says 172.21.15.27:58426) 2021-11-03T17:36:47.689 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.688+0000 7f0f027fc700 1 -- 172.21.15.27:0/3012499418 learned_addr learned my addr 172.21.15.27:0/3012499418 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:36:47.689 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.688+0000 7f0f02ffd700 1 --2- 172.21.15.27:0/3012499418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f04060250 0x7f0f040f6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:36:47.690 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.688+0000 7f0f027fc700 1 -- 172.21.15.27:0/3012499418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f04060250 msgr2=0x7f0f040f6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:36:47.690 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.688+0000 7f0f027fc700 1 --2- 172.21.15.27:0/3012499418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f04060250 0x7f0f040f6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:47.690 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.688+0000 7f0f027fc700 1 -- 172.21.15.27:0/3012499418 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0efc009640 con 0x7f0f040f9850 2021-11-03T17:36:47.690 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.688+0000 7f0f02ffd700 1 --2- 172.21.15.27:0/3012499418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f04060250 0x7f0f040f6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:36:47.691 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.688+0000 7f0f027fc700 1 --2- 172.21.15.27:0/3012499418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f040f9850 0x7f0f040f5260 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f0eec0026d0 tx=0x7f0eec009d40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:36:47.691 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.689+0000 7f0efb7fe700 1 -- 172.21.15.27:0/3012499418 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0eec009320 con 0x7f0f040f9850 2021-11-03T17:36:47.691 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.689+0000 7f0efb7fe700 1 -- 172.21.15.27:0/3012499418 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f0eec00f040 con 0x7f0f040f9850 2021-11-03T17:36:47.691 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.689+0000 7f0f09853700 1 -- 172.21.15.27:0/3012499418 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f040f5be0 con 0x7f0f040f9850 2021-11-03T17:36:47.692 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.689+0000 7f0efb7fe700 1 -- 172.21.15.27:0/3012499418 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0eec014760 con 0x7f0f040f9850 2021-11-03T17:36:47.692 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.689+0000 7f0f09853700 1 -- 172.21.15.27:0/3012499418 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f0410f090 con 0x7f0f040f9850 2021-11-03T17:36:47.692 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.690+0000 7f0f09853700 1 -- 172.21.15.27:0/3012499418 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f04045ad0 con 0x7f0f040f9850 2021-11-03T17:36:47.692 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.690+0000 7f0efb7fe700 1 -- 172.21.15.27:0/3012499418 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f0eec009480 con 0x7f0f040f9850 2021-11-03T17:36:47.693 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.691+0000 7f0efb7fe700 1 --2- 172.21.15.27:0/3012499418 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0ef00640b0 0x7f0ef0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:36:47.693 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.691+0000 7f0efb7fe700 1 -- 172.21.15.27:0/3012499418 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f0eec080f70 con 0x7f0f040f9850 2021-11-03T17:36:47.693 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.691+0000 7f0f02ffd700 1 --2- 172.21.15.27:0/3012499418 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0ef00640b0 0x7f0ef0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:36:47.693 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.691+0000 7f0f02ffd700 1 --2- 172.21.15.27:0/3012499418 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0ef00640b0 0x7f0ef0066570 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f0efc009f50 tx=0x7f0efc004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:36:47.694 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.695+0000 7f0efb7fe700 1 -- 172.21.15.27:0/3012499418 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f0eec053150 con 0x7f0f040f9850 2021-11-03T17:36:47.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.922+0000 7f0f09853700 1 -- 172.21.15.27:0/3012499418 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0f04027a00 con 0x7f0f040f9850 2021-11-03T17:36:47.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.923+0000 7f0efb7fe700 1 -- 172.21.15.27:0/3012499418 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f0eec052d40 con 0x7f0f040f9850 2021-11-03T17:36:47.918 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:36:47.919 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:36:47.919 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:36:47.919 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:36:47.919 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:36:47.920 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:36:47.920 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:36:47.920 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:36:47.920 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:36:47.920 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:36:47.921 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:36:47.921 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:36:47.921 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:36:47.921 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:36:47.922 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:36:47.922 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:36:47.922 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:36:47.922 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:36:47.923 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.925+0000 7f0f09853700 1 -- 172.21.15.27:0/3012499418 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0ef00640b0 msgr2=0x7f0ef0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:36:47.923 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.925+0000 7f0f09853700 1 --2- 172.21.15.27:0/3012499418 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0ef00640b0 0x7f0ef0066570 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f0efc009f50 tx=0x7f0efc004d10).stop 2021-11-03T17:36:47.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.925+0000 7f0f09853700 1 -- 172.21.15.27:0/3012499418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f040f9850 msgr2=0x7f0f040f5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:36:47.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.925+0000 7f0f09853700 1 --2- 172.21.15.27:0/3012499418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f040f9850 0x7f0f040f5260 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f0eec0026d0 tx=0x7f0eec009d40).stop 2021-11-03T17:36:47.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.925+0000 7f0f09853700 1 -- 172.21.15.27:0/3012499418 shutdown_connections 2021-11-03T17:36:47.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.925+0000 7f0f09853700 1 --2- 172.21.15.27:0/3012499418 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0ef00640b0 0x7f0ef0066570 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:47.925 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.925+0000 7f0f09853700 1 --2- 172.21.15.27:0/3012499418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f04060250 0x7f0f040f6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:47.926 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.925+0000 7f0f09853700 1 --2- 172.21.15.27:0/3012499418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f040f9850 0x7f0f040f5260 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:47.926 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.925+0000 7f0f09853700 1 -- 172.21.15.27:0/3012499418 >> 172.21.15.27:0/3012499418 conn(0x7f0f040f0d40 msgr2=0x7f0f040f3130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:36:47.926 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.925+0000 7f0f09853700 1 -- 172.21.15.27:0/3012499418 shutdown_connections 2021-11-03T17:36:47.926 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:47.925+0000 7f0f09853700 1 -- 172.21.15.27:0/3012499418 wait complete. 2021-11-03T17:36:48.021 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.026+0000 7f9c5af07700 1 -- 172.21.15.27:0/136581646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c540f6860 msgr2=0x7f9c540f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:36:48.022 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.026+0000 7f9c5af07700 1 --2- 172.21.15.27:0/136581646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c540f6860 0x7f9c540f6cc0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f9c44004660 tx=0x7f9c44009b30).stop 2021-11-03T17:36:48.022 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.026+0000 7f9c5af07700 1 -- 172.21.15.27:0/136581646 shutdown_connections 2021-11-03T17:36:48.022 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.026+0000 7f9c5af07700 1 --2- 172.21.15.27:0/136581646 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c540f6860 0x7f9c540f6cc0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:48.022 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.026+0000 7f9c5af07700 1 --2- 172.21.15.27:0/136581646 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c540f55c0 0x7f9c540f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:48.023 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.026+0000 7f9c5af07700 1 -- 172.21.15.27:0/136581646 >> 172.21.15.27:0/136581646 conn(0x7f9c540f0d40 msgr2=0x7f9c540f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:36:48.023 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.026+0000 7f9c5af07700 1 -- 172.21.15.27:0/136581646 shutdown_connections 2021-11-03T17:36:48.023 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.026+0000 7f9c5af07700 1 -- 172.21.15.27:0/136581646 wait complete. 2021-11-03T17:36:48.023 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.026+0000 7f9c5af07700 1 Processor -- start 2021-11-03T17:36:48.023 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.026+0000 7f9c5af07700 1 -- start start 2021-11-03T17:36:48.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.027+0000 7f9c5af07700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c540f55c0 0x7f9c54109be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:36:48.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.027+0000 7f9c5af07700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c540f6860 0x7f9c5410c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:36:48.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.027+0000 7f9c5af07700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c540603c0 con 0x7f9c540f55c0 2021-11-03T17:36:48.027 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.027+0000 7f9c5af07700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c54060500 con 0x7f9c540f6860 2021-11-03T17:36:48.027 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.027+0000 7f9c58ca3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c540f55c0 0x7f9c54109be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:36:48.027 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.027+0000 7f9c58ca3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c540f55c0 0x7f9c54109be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58436/0 (socket says 172.21.15.27:58436) 2021-11-03T17:36:48.027 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.027+0000 7f9c58ca3700 1 -- 172.21.15.27:0/4065088790 learned_addr learned my addr 172.21.15.27:0/4065088790 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:36:48.027 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.027+0000 7f9c53fff700 1 --2- 172.21.15.27:0/4065088790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c540f6860 0x7f9c5410c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:36:48.028 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.027+0000 7f9c58ca3700 1 -- 172.21.15.27:0/4065088790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c540f6860 msgr2=0x7f9c5410c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:36:48.028 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.027+0000 7f9c58ca3700 1 --2- 172.21.15.27:0/4065088790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c540f6860 0x7f9c5410c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:48.028 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.027+0000 7f9c58ca3700 1 -- 172.21.15.27:0/4065088790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c44005040 con 0x7f9c540f55c0 2021-11-03T17:36:48.029 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.027+0000 7f9c53fff700 1 --2- 172.21.15.27:0/4065088790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c540f6860 0x7f9c5410c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:36:48.029 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.027+0000 7f9c58ca3700 1 --2- 172.21.15.27:0/4065088790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c540f55c0 0x7f9c54109be0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7f9c3c00e7c0 tx=0x7f9c3c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:36:48.029 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.028+0000 7f9c517fa700 1 -- 172.21.15.27:0/4065088790 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c3c0092e0 con 0x7f9c540f55c0 2021-11-03T17:36:48.030 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.028+0000 7f9c5af07700 1 -- 172.21.15.27:0/4065088790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c5410c690 con 0x7f9c540f55c0 2021-11-03T17:36:48.030 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.028+0000 7f9c517fa700 1 -- 172.21.15.27:0/4065088790 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f9c3c00f040 con 0x7f9c540f55c0 2021-11-03T17:36:48.030 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.028+0000 7f9c517fa700 1 -- 172.21.15.27:0/4065088790 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c3c014750 con 0x7f9c540f55c0 2021-11-03T17:36:48.030 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.028+0000 7f9c5af07700 1 -- 172.21.15.27:0/4065088790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c5410cc20 con 0x7f9c540f55c0 2021-11-03T17:36:48.031 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.029+0000 7f9c5af07700 1 -- 172.21.15.27:0/4065088790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c540f49b0 con 0x7f9c540f55c0 2021-11-03T17:36:48.031 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.029+0000 7f9c517fa700 1 -- 172.21.15.27:0/4065088790 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f9c3c01b030 con 0x7f9c540f55c0 2021-11-03T17:36:48.031 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.029+0000 7f9c517fa700 1 --2- 172.21.15.27:0/4065088790 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9c400640b0 0x7f9c40066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:36:48.031 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.030+0000 7f9c517fa700 1 -- 172.21.15.27:0/4065088790 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f9c3c0808c0 con 0x7f9c540f55c0 2021-11-03T17:36:48.031 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.030+0000 7f9c53fff700 1 --2- 172.21.15.27:0/4065088790 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9c400640b0 0x7f9c40066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:36:48.032 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.030+0000 7f9c53fff700 1 --2- 172.21.15.27:0/4065088790 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9c400640b0 0x7f9c40066570 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f9c44004d80 tx=0x7f9c4400b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:36:48.032 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.034+0000 7f9c517fa700 1 -- 172.21.15.27:0/4065088790 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f9c3c052a70 con 0x7f9c540f55c0 2021-11-03T17:36:48.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.238+0000 7f9c5af07700 1 -- 172.21.15.27:0/4065088790 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9c540586e0 con 0x7f9c540f55c0 2021-11-03T17:36:48.234 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.238+0000 7f9c517fa700 1 -- 172.21.15.27:0/4065088790 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f9c3c019710 con 0x7f9c540f55c0 2021-11-03T17:36:48.234 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:36:48.234 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:36:48.235 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:36:48.235 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:36:48.235 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:36:48.235 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:36:48.236 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:36:48.236 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:36:48.236 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:36:48.236 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:36:48.237 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:36:48.237 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:36:48.237 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:36:48.237 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:36:48.238 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:36:48.238 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:36:48.238 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:36:48.238 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:36:48.238 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:36:48.239 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:36:48.239 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:36:48.239 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:36:48.239 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:36:48.240 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:36:48.240 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:36:48.240 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:36:48.240 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:36:48.241 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:36:48.241 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:36:48.241 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:36:48.241 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:36:48.242 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:36:48.242 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:36:48.242 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:36:48.242 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:36:48.242 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:36:48.243 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:36:48.243 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:36:48.243 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:36:48.244 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:47 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3012499418' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:48.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.240+0000 7f9c5af07700 1 -- 172.21.15.27:0/4065088790 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9c400640b0 msgr2=0x7f9c40066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:36:48.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.240+0000 7f9c5af07700 1 --2- 172.21.15.27:0/4065088790 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9c400640b0 0x7f9c40066570 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f9c44004d80 tx=0x7f9c4400b040).stop 2021-11-03T17:36:48.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.240+0000 7f9c5af07700 1 -- 172.21.15.27:0/4065088790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c540f55c0 msgr2=0x7f9c54109be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:36:48.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.240+0000 7f9c5af07700 1 --2- 172.21.15.27:0/4065088790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c540f55c0 0x7f9c54109be0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7f9c3c00e7c0 tx=0x7f9c3c00b040).stop 2021-11-03T17:36:48.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.240+0000 7f9c5af07700 1 -- 172.21.15.27:0/4065088790 shutdown_connections 2021-11-03T17:36:48.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.240+0000 7f9c5af07700 1 --2- 172.21.15.27:0/4065088790 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9c400640b0 0x7f9c40066570 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:48.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.240+0000 7f9c5af07700 1 --2- 172.21.15.27:0/4065088790 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c540f55c0 0x7f9c54109be0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:48.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.240+0000 7f9c5af07700 1 --2- 172.21.15.27:0/4065088790 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c540f6860 0x7f9c5410c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:36:48.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.240+0000 7f9c5af07700 1 -- 172.21.15.27:0/4065088790 >> 172.21.15.27:0/4065088790 conn(0x7f9c540f0d40 msgr2=0x7f9c540fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:36:48.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.240+0000 7f9c5af07700 1 -- 172.21.15.27:0/4065088790 shutdown_connections 2021-11-03T17:36:48.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:36:48.240+0000 7f9c5af07700 1 -- 172.21.15.27:0/4065088790 wait complete. 2021-11-03T17:36:48.248 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:36:48.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:47 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3012499418' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:49.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:48 smithi115 ceph-mon[65453]: from='client.34792 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:36:49.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:48 smithi115 ceph-mon[65453]: from='client.34796 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:36:49.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:48 smithi115 ceph-mon[65453]: pgmap v665: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:49.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:48 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/4065088790' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:36:49.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:48 smithi027 ceph-mon[97244]: from='client.34792 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:36:49.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:48 smithi027 ceph-mon[97244]: from='client.34796 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:36:49.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:48 smithi027 ceph-mon[97244]: pgmap v665: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:49.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:48 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/4065088790' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:36:50.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:49 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1252.388653 secs 2021-11-03T17:36:50.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:49 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1252.388653 secs 2021-11-03T17:36:51.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:50 smithi027 ceph-mon[97244]: pgmap v666: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:51.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:50 smithi115 ceph-mon[65453]: pgmap v666: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:52.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:52 smithi115 ceph-mon[65453]: pgmap v667: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:52.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:52 smithi027 ceph-mon[97244]: pgmap v667: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:53.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:53 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:36:53.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:53 smithi115 ceph-mon[65453]: poll took 50136.605 ms: 1 events 2021-11-03T17:36:53.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:53 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:36:53.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:53 smithi027 ceph-mon[97244]: poll took 50136.605 ms: 1 events 2021-11-03T17:36:54.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:36:54.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:36:54.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:36:54.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:54.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:54.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:54.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:54.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:54 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:36:54.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:54 smithi115 ceph-mon[65453]: pgmap v668: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:54.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:36:54.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:36:54.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:36:54.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:54.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:54.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:54.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:36:54.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:54 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:36:54.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:54 smithi027 ceph-mon[97244]: pgmap v668: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:55.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1257.388771 secs 2021-11-03T17:36:55.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1257.388771 secs 2021-11-03T17:36:56.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:36:56.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:36:56.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:56 smithi115 ceph-mon[65453]: pgmap v669: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:56.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:36:56.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:36:56.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:56 smithi027 ceph-mon[97244]: pgmap v669: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:58.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:58 smithi115 ceph-mon[65453]: pgmap v670: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:58.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:58 smithi027 ceph-mon[97244]: pgmap v670: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:36:59.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:36:59 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1262.388895 secs 2021-11-03T17:37:00.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:36:59 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1262.388895 secs 2021-11-03T17:37:00.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:00 smithi115 ceph-mon[65453]: pgmap v671: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:01.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:00 smithi027 ceph-mon[97244]: pgmap v671: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:02.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:02 smithi115 ceph-mon[65453]: pgmap v672: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:02.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:02 smithi027 ceph-mon[97244]: pgmap v672: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:03.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:37:03.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:37:05.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:04 smithi115 ceph-mon[65453]: poll took 10030.403 ms: 1 events 2021-11-03T17:37:05.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:37:05.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:37:05.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:05.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:05.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:05.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:05.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:04 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:37:05.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:04 smithi115 ceph-mon[65453]: pgmap v673: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:05.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:04 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1267.388984 secs 2021-11-03T17:37:05.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:04 smithi027 ceph-mon[97244]: poll took 10030.403 ms: 1 events 2021-11-03T17:37:05.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:37:05.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:37:05.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:05.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:05.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:05.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:05.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:04 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:37:05.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:04 smithi027 ceph-mon[97244]: pgmap v673: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:05.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:04 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1267.388984 secs 2021-11-03T17:37:06.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:06 smithi115 ceph-mon[65453]: pgmap v674: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:06.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:06 smithi027 ceph-mon[97244]: pgmap v674: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:08.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:08 smithi115 ceph-mon[65453]: pgmap v675: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:08.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:08 smithi027 ceph-mon[97244]: pgmap v675: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:09.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:09 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1272.389129 secs 2021-11-03T17:37:10.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:09 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1272.389129 secs 2021-11-03T17:37:10.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:10 smithi115 ceph-mon[65453]: pgmap v676: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:11.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:10 smithi027 ceph-mon[97244]: pgmap v676: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:12.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:12 smithi115 ceph-mon[65453]: pgmap v677: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:12.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:12 smithi027 ceph-mon[97244]: pgmap v677: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:14.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:37:14.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:37:14.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:37:14.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:14.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:14.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:14.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:14.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:37:14.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:37:14.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:37:14.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:14.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:14.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:14.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:15.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:14 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:37:15.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:14 smithi115 ceph-mon[65453]: pgmap v678: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:15.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:14 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1277.389270 secs 2021-11-03T17:37:15.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:14 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:37:15.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:14 smithi027 ceph-mon[97244]: pgmap v678: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:15.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:14 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1277.389270 secs 2021-11-03T17:37:16.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:16 smithi115 ceph-mon[65453]: pgmap v679: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:16.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:16 smithi027 ceph-mon[97244]: pgmap v679: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:18.352 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.357+0000 7f6e8d3c5700 1 -- 172.21.15.27:0/2520726985 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e880f6860 msgr2=0x7f6e880f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:37:18.352 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.357+0000 7f6e8d3c5700 1 --2- 172.21.15.27:0/2520726985 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e880f6860 0x7f6e880f6cc0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f6e70004660 tx=0x7f6e70009b30).stop 2021-11-03T17:37:18.352 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.357+0000 7f6e8d3c5700 1 -- 172.21.15.27:0/2520726985 shutdown_connections 2021-11-03T17:37:18.352 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.357+0000 7f6e8d3c5700 1 --2- 172.21.15.27:0/2520726985 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e880f6860 0x7f6e880f6cc0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:18.353 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.357+0000 7f6e8d3c5700 1 --2- 172.21.15.27:0/2520726985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e880f55c0 0x7f6e880f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:18.353 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.357+0000 7f6e8d3c5700 1 -- 172.21.15.27:0/2520726985 >> 172.21.15.27:0/2520726985 conn(0x7f6e880f0d40 msgr2=0x7f6e880f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:37:18.353 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.357+0000 7f6e8d3c5700 1 -- 172.21.15.27:0/2520726985 shutdown_connections 2021-11-03T17:37:18.353 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.357+0000 7f6e8d3c5700 1 -- 172.21.15.27:0/2520726985 wait complete. 2021-11-03T17:37:18.354 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.358+0000 7f6e8d3c5700 1 Processor -- start 2021-11-03T17:37:18.354 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.358+0000 7f6e8d3c5700 1 -- start start 2021-11-03T17:37:18.354 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.358+0000 7f6e8d3c5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e880f55c0 0x7f6e88106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:37:18.354 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.358+0000 7f6e8d3c5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e880f6860 0x7f6e88107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:37:18.355 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.358+0000 7f6e8d3c5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e88107a10 con 0x7f6e880f55c0 2021-11-03T17:37:18.355 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.358+0000 7f6e8d3c5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e88107b50 con 0x7f6e880f6860 2021-11-03T17:37:18.357 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.358+0000 7f6e86ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e880f55c0 0x7f6e88106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:37:18.358 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.358+0000 7f6e86ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e880f55c0 0x7f6e88106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58454/0 (socket says 172.21.15.27:58454) 2021-11-03T17:37:18.358 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.358+0000 7f6e86ffd700 1 -- 172.21.15.27:0/2433115642 learned_addr learned my addr 172.21.15.27:0/2433115642 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:37:18.358 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.358+0000 7f6e867fc700 1 --2- 172.21.15.27:0/2433115642 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e880f6860 0x7f6e88107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:37:18.359 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.358+0000 7f6e86ffd700 1 -- 172.21.15.27:0/2433115642 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e880f6860 msgr2=0x7f6e88107460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:37:18.359 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.358+0000 7f6e86ffd700 1 --2- 172.21.15.27:0/2433115642 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e880f6860 0x7f6e88107460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:18.359 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.358+0000 7f6e86ffd700 1 -- 172.21.15.27:0/2433115642 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e70005040 con 0x7f6e880f55c0 2021-11-03T17:37:18.359 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.359+0000 7f6e86ffd700 1 --2- 172.21.15.27:0/2433115642 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e880f55c0 0x7f6e88106f70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f6e8000d7f0 tx=0x7f6e80004e80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:37:18.359 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.359+0000 7f6e7f7fe700 1 -- 172.21.15.27:0/2433115642 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e8000b4f0 con 0x7f6e880f55c0 2021-11-03T17:37:18.360 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.359+0000 7f6e7f7fe700 1 -- 172.21.15.27:0/2433115642 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f6e8000f040 con 0x7f6e880f55c0 2021-11-03T17:37:18.360 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.359+0000 7f6e8d3c5700 1 -- 172.21.15.27:0/2433115642 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e881084f0 con 0x7f6e880f55c0 2021-11-03T17:37:18.360 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.359+0000 7f6e7f7fe700 1 -- 172.21.15.27:0/2433115642 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e80013980 con 0x7f6e880f55c0 2021-11-03T17:37:18.360 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.359+0000 7f6e8d3c5700 1 -- 172.21.15.27:0/2433115642 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e88108ab0 con 0x7f6e880f55c0 2021-11-03T17:37:18.361 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.360+0000 7f6e8d3c5700 1 -- 172.21.15.27:0/2433115642 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e88045ad0 con 0x7f6e880f55c0 2021-11-03T17:37:18.361 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.360+0000 7f6e7f7fe700 1 -- 172.21.15.27:0/2433115642 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f6e8001a030 con 0x7f6e880f55c0 2021-11-03T17:37:18.361 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.361+0000 7f6e7f7fe700 1 --2- 172.21.15.27:0/2433115642 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6e740640b0 0x7f6e74066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:37:18.361 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.361+0000 7f6e7f7fe700 1 -- 172.21.15.27:0/2433115642 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f6e80080890 con 0x7f6e880f55c0 2021-11-03T17:37:18.362 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.361+0000 7f6e867fc700 1 --2- 172.21.15.27:0/2433115642 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6e740640b0 0x7f6e74066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:37:18.362 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.361+0000 7f6e867fc700 1 --2- 172.21.15.27:0/2433115642 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6e740640b0 0x7f6e74066570 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f6e70009da0 tx=0x7f6e70009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:37:18.362 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.365+0000 7f6e7f7fe700 1 -- 172.21.15.27:0/2433115642 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f6e80052aa0 con 0x7f6e880f55c0 2021-11-03T17:37:18.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:18 smithi115 ceph-mon[65453]: pgmap v680: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:18.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:18 smithi027 ceph-mon[97244]: pgmap v680: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:18.539 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.545+0000 7f6e8d3c5700 1 -- 172.21.15.27:0/2433115642 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e8805fd30 con 0x7f6e740640b0 2021-11-03T17:37:18.540 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.546+0000 7f6e7f7fe700 1 -- 172.21.15.27:0/2433115642 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f6e8805fd30 con 0x7f6e740640b0 2021-11-03T17:37:18.542 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.548+0000 7f6e8d3c5700 1 -- 172.21.15.27:0/2433115642 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6e740640b0 msgr2=0x7f6e74066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:37:18.543 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.548+0000 7f6e8d3c5700 1 --2- 172.21.15.27:0/2433115642 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6e740640b0 0x7f6e74066570 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f6e70009da0 tx=0x7f6e70009b30).stop 2021-11-03T17:37:18.543 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.548+0000 7f6e8d3c5700 1 -- 172.21.15.27:0/2433115642 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e880f55c0 msgr2=0x7f6e88106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:37:18.544 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.548+0000 7f6e8d3c5700 1 --2- 172.21.15.27:0/2433115642 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e880f55c0 0x7f6e88106f70 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f6e8000d7f0 tx=0x7f6e80004e80).stop 2021-11-03T17:37:18.544 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.548+0000 7f6e8d3c5700 1 -- 172.21.15.27:0/2433115642 shutdown_connections 2021-11-03T17:37:18.544 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.548+0000 7f6e8d3c5700 1 --2- 172.21.15.27:0/2433115642 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6e740640b0 0x7f6e74066570 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:18.544 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.548+0000 7f6e8d3c5700 1 --2- 172.21.15.27:0/2433115642 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e880f55c0 0x7f6e88106f70 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:18.544 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.548+0000 7f6e8d3c5700 1 --2- 172.21.15.27:0/2433115642 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e880f6860 0x7f6e88107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:18.545 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.548+0000 7f6e8d3c5700 1 -- 172.21.15.27:0/2433115642 >> 172.21.15.27:0/2433115642 conn(0x7f6e880f0d40 msgr2=0x7f6e8805f620 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:37:18.545 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.548+0000 7f6e8d3c5700 1 -- 172.21.15.27:0/2433115642 shutdown_connections 2021-11-03T17:37:18.545 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.548+0000 7f6e8d3c5700 1 -- 172.21.15.27:0/2433115642 wait complete. 2021-11-03T17:37:18.552 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:37:18.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.649+0000 7faa88774700 1 -- 172.21.15.27:0/3140517322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa800f66e0 msgr2=0x7faa800f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:37:18.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.649+0000 7faa88774700 1 --2- 172.21.15.27:0/3140517322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa800f66e0 0x7faa800f6b00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7faa780099e0 tx=0x7faa78004d10).stop 2021-11-03T17:37:18.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.649+0000 7faa88774700 1 -- 172.21.15.27:0/3140517322 shutdown_connections 2021-11-03T17:37:18.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.649+0000 7faa88774700 1 --2- 172.21.15.27:0/3140517322 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa800f7980 0x7faa800f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:18.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.649+0000 7faa88774700 1 --2- 172.21.15.27:0/3140517322 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa800f66e0 0x7faa800f6b00 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:18.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.649+0000 7faa88774700 1 -- 172.21.15.27:0/3140517322 >> 172.21.15.27:0/3140517322 conn(0x7faa800f1e40 msgr2=0x7faa800f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:37:18.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.649+0000 7faa88774700 1 -- 172.21.15.27:0/3140517322 shutdown_connections 2021-11-03T17:37:18.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.649+0000 7faa88774700 1 -- 172.21.15.27:0/3140517322 wait complete. 2021-11-03T17:37:18.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.649+0000 7faa88774700 1 Processor -- start 2021-11-03T17:37:18.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.649+0000 7faa88774700 1 -- start start 2021-11-03T17:37:18.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.650+0000 7faa88774700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa800f66e0 0x7faa8006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:37:18.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.650+0000 7faa88774700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa800f7980 0x7faa80069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:37:18.648 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.650+0000 7faa88774700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8006a380 con 0x7faa800f66e0 2021-11-03T17:37:18.648 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.650+0000 7faa88774700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa8006a4c0 con 0x7faa800f7980 2021-11-03T17:37:18.648 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.650+0000 7faa86510700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa800f66e0 0x7faa8006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:37:18.648 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.650+0000 7faa86510700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa800f66e0 0x7faa8006b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58464/0 (socket says 172.21.15.27:58464) 2021-11-03T17:37:18.649 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.650+0000 7faa86510700 1 -- 172.21.15.27:0/939915523 learned_addr learned my addr 172.21.15.27:0/939915523 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:37:18.649 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.650+0000 7faa85d0f700 1 --2- 172.21.15.27:0/939915523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa800f7980 0x7faa80069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:37:18.649 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.650+0000 7faa86510700 1 -- 172.21.15.27:0/939915523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa800f7980 msgr2=0x7faa80069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:37:18.649 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.650+0000 7faa86510700 1 --2- 172.21.15.27:0/939915523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa800f7980 0x7faa80069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:18.649 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.650+0000 7faa86510700 1 -- 172.21.15.27:0/939915523 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa78009640 con 0x7faa800f66e0 2021-11-03T17:37:18.650 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.650+0000 7faa86510700 1 --2- 172.21.15.27:0/939915523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa800f66e0 0x7faa8006b9a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7faa7800baa0 tx=0x7faa78004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:37:18.650 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.650+0000 7faa7effd700 1 -- 172.21.15.27:0/939915523 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa7800e070 con 0x7faa800f66e0 2021-11-03T17:37:18.650 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.650+0000 7faa7effd700 1 -- 172.21.15.27:0/939915523 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7faa78007d40 con 0x7faa800f66e0 2021-11-03T17:37:18.650 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.650+0000 7faa88774700 1 -- 172.21.15.27:0/939915523 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa8006a740 con 0x7faa800f66e0 2021-11-03T17:37:18.651 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.650+0000 7faa7effd700 1 -- 172.21.15.27:0/939915523 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa78018070 con 0x7faa800f66e0 2021-11-03T17:37:18.651 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.650+0000 7faa88774700 1 -- 172.21.15.27:0/939915523 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa8006aca0 con 0x7faa800f66e0 2021-11-03T17:37:18.652 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.651+0000 7faa88774700 1 -- 172.21.15.27:0/939915523 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa80045ad0 con 0x7faa800f66e0 2021-11-03T17:37:18.652 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.652+0000 7faa7effd700 1 -- 172.21.15.27:0/939915523 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7faa78024060 con 0x7faa800f66e0 2021-11-03T17:37:18.653 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.652+0000 7faa7effd700 1 --2- 172.21.15.27:0/939915523 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7faa680640b0 0x7faa68066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:37:18.653 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.652+0000 7faa7effd700 1 -- 172.21.15.27:0/939915523 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7faa78081f10 con 0x7faa800f66e0 2021-11-03T17:37:18.653 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.652+0000 7faa85d0f700 1 --2- 172.21.15.27:0/939915523 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7faa680640b0 0x7faa68066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:37:18.653 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.653+0000 7faa85d0f700 1 --2- 172.21.15.27:0/939915523 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7faa680640b0 0x7faa68066570 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7faa74009fd0 tx=0x7faa7400b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:37:18.654 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.656+0000 7faa7effd700 1 -- 172.21.15.27:0/939915523 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7faa78054210 con 0x7faa800f66e0 2021-11-03T17:37:18.827 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.833+0000 7faa88774700 1 -- 172.21.15.27:0/939915523 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7faa800fefe0 con 0x7faa680640b0 2021-11-03T17:37:18.841 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.847+0000 7faa7effd700 1 -- 172.21.15.27:0/939915523 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7faa800fefe0 con 0x7faa680640b0 2021-11-03T17:37:18.842 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:37:18.842 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (28m) 8m ago 30m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:37:18.842 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (20m) 8m ago 30m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:37:18.842 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (20m) 7m ago 29m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:37:18.842 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (28m) 8m ago 29m 40.2M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:37:18.843 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (22m) 8m ago 25m 18.2M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:37:18.843 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (25m) 8m ago 25m 3841M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:37:18.843 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (21m) 7m ago 25m 17.6M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:37:18.843 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (22m) 7m ago 25m 19.3M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:37:18.844 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (22m) 8m ago 31m 497M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:37:18.844 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (21m) 7m ago 29m 415M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:37:18.844 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (21m) 8m ago 31m 119M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:37:18.844 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (21m) 7m ago 29m 71.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:37:18.845 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (29m) 8m ago 29m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:37:18.845 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (29m) 7m ago 29m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:37:18.845 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (20m) 8m ago 28m 290M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:37:18.845 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (19m) 8m ago 28m 317M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:37:18.845 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (19m) 8m ago 27m 229M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:37:18.846 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (18m) 7m ago 27m 329M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:37:18.846 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (18m) 7m ago 27m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:37:18.846 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (17m) 7m ago 26m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:37:18.846 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (28m) 8m ago 29m 48.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:37:18.847 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.849+0000 7faa88774700 1 -- 172.21.15.27:0/939915523 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7faa680640b0 msgr2=0x7faa68066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:37:18.847 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.849+0000 7faa88774700 1 --2- 172.21.15.27:0/939915523 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7faa680640b0 0x7faa68066570 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7faa74009fd0 tx=0x7faa7400b040).stop 2021-11-03T17:37:18.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.849+0000 7faa88774700 1 -- 172.21.15.27:0/939915523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa800f66e0 msgr2=0x7faa8006b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:37:18.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.849+0000 7faa88774700 1 --2- 172.21.15.27:0/939915523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa800f66e0 0x7faa8006b9a0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7faa7800baa0 tx=0x7faa78004d10).stop 2021-11-03T17:37:18.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.849+0000 7faa88774700 1 -- 172.21.15.27:0/939915523 shutdown_connections 2021-11-03T17:37:18.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.849+0000 7faa88774700 1 --2- 172.21.15.27:0/939915523 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7faa680640b0 0x7faa68066570 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:18.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.849+0000 7faa88774700 1 --2- 172.21.15.27:0/939915523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa800f66e0 0x7faa8006b9a0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:18.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.849+0000 7faa88774700 1 --2- 172.21.15.27:0/939915523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa800f7980 0x7faa80069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:18.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.849+0000 7faa88774700 1 -- 172.21.15.27:0/939915523 >> 172.21.15.27:0/939915523 conn(0x7faa800f1e40 msgr2=0x7faa800fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:37:18.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.849+0000 7faa88774700 1 -- 172.21.15.27:0/939915523 shutdown_connections 2021-11-03T17:37:18.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.850+0000 7faa88774700 1 -- 172.21.15.27:0/939915523 wait complete. 2021-11-03T17:37:18.943 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.948+0000 7fc1c15f4700 1 -- 172.21.15.27:0/2920591458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1bc0f66c0 msgr2=0x7fc1bc0f6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:37:18.943 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.948+0000 7fc1c15f4700 1 --2- 172.21.15.27:0/2920591458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1bc0f66c0 0x7fc1bc0f6ae0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7fc1ac004660 tx=0x7fc1ac009b30).stop 2021-11-03T17:37:18.943 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.948+0000 7fc1c15f4700 1 -- 172.21.15.27:0/2920591458 shutdown_connections 2021-11-03T17:37:18.943 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.948+0000 7fc1c15f4700 1 --2- 172.21.15.27:0/2920591458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1bc0f7960 0x7fc1bc0f7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:18.944 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.948+0000 7fc1c15f4700 1 --2- 172.21.15.27:0/2920591458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1bc0f66c0 0x7fc1bc0f6ae0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:18.944 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.948+0000 7fc1c15f4700 1 -- 172.21.15.27:0/2920591458 >> 172.21.15.27:0/2920591458 conn(0x7fc1bc0f1e20 msgr2=0x7fc1bc0f4240 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:37:18.944 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.948+0000 7fc1c15f4700 1 -- 172.21.15.27:0/2920591458 shutdown_connections 2021-11-03T17:37:18.944 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.949+0000 7fc1c15f4700 1 -- 172.21.15.27:0/2920591458 wait complete. 2021-11-03T17:37:18.947 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.949+0000 7fc1c15f4700 1 Processor -- start 2021-11-03T17:37:18.947 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.949+0000 7fc1c15f4700 1 -- start start 2021-11-03T17:37:18.947 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.949+0000 7fc1c15f4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1bc0f66c0 0x7fc1bc106f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:37:18.948 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.949+0000 7fc1c15f4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1bc0f7960 0x7fc1bc107400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:37:18.948 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.949+0000 7fc1c15f4700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc1079b0 con 0x7fc1bc0f7960 2021-11-03T17:37:18.948 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.949+0000 7fc1c15f4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1bc107af0 con 0x7fc1bc0f66c0 2021-11-03T17:37:18.948 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.949+0000 7fc1ba7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1bc0f7960 0x7fc1bc107400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:37:18.949 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.950+0000 7fc1ba7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1bc0f7960 0x7fc1bc107400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58476/0 (socket says 172.21.15.27:58476) 2021-11-03T17:37:18.949 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.950+0000 7fc1ba7fc700 1 -- 172.21.15.27:0/238244441 learned_addr learned my addr 172.21.15.27:0/238244441 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:37:18.949 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.950+0000 7fc1baffd700 1 --2- 172.21.15.27:0/238244441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1bc0f66c0 0x7fc1bc106f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:37:18.949 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.950+0000 7fc1ba7fc700 1 -- 172.21.15.27:0/238244441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1bc0f66c0 msgr2=0x7fc1bc106f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:37:18.949 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.950+0000 7fc1ba7fc700 1 --2- 172.21.15.27:0/238244441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1bc0f66c0 0x7fc1bc106f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:18.950 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.950+0000 7fc1ba7fc700 1 -- 172.21.15.27:0/238244441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ac005040 con 0x7fc1bc0f7960 2021-11-03T17:37:18.950 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.950+0000 7fc1ba7fc700 1 --2- 172.21.15.27:0/238244441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1bc0f7960 0x7fc1bc107400 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7fc1b000e7f0 tx=0x7fc1b000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:37:18.950 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.950+0000 7fc1a37fe700 1 -- 172.21.15.27:0/238244441 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1b00092e0 con 0x7fc1bc0f7960 2021-11-03T17:37:18.950 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.950+0000 7fc1a37fe700 1 -- 172.21.15.27:0/238244441 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fc1b000f040 con 0x7fc1bc0f7960 2021-11-03T17:37:18.951 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.950+0000 7fc1c15f4700 1 -- 172.21.15.27:0/238244441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1bc108490 con 0x7fc1bc0f7960 2021-11-03T17:37:18.951 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.950+0000 7fc1a37fe700 1 -- 172.21.15.27:0/238244441 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1b0014750 con 0x7fc1bc0f7960 2021-11-03T17:37:18.951 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.950+0000 7fc1c15f4700 1 -- 172.21.15.27:0/238244441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1bc108a50 con 0x7fc1bc0f7960 2021-11-03T17:37:18.951 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.951+0000 7fc1c15f4700 1 -- 172.21.15.27:0/238244441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1bc045ad0 con 0x7fc1bc0f7960 2021-11-03T17:37:18.951 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.952+0000 7fc1a37fe700 1 -- 172.21.15.27:0/238244441 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fc1b001b030 con 0x7fc1bc0f7960 2021-11-03T17:37:18.952 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.952+0000 7fc1a37fe700 1 --2- 172.21.15.27:0/238244441 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc1a40640b0 0x7fc1a4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:37:18.952 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.953+0000 7fc1a37fe700 1 -- 172.21.15.27:0/238244441 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fc1b0080750 con 0x7fc1bc0f7960 2021-11-03T17:37:18.952 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.953+0000 7fc1baffd700 1 --2- 172.21.15.27:0/238244441 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc1a40640b0 0x7fc1a4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:37:18.952 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.953+0000 7fc1baffd700 1 --2- 172.21.15.27:0/238244441 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc1a40640b0 0x7fc1a4066570 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7fc1ac004d80 tx=0x7fc1ac016040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:37:18.953 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:18.956+0000 7fc1a37fe700 1 -- 172.21.15.27:0/238244441 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fc1b0048df0 con 0x7fc1bc0f7960 2021-11-03T17:37:19.179 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.185+0000 7fc1c15f4700 1 -- 172.21.15.27:0/238244441 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc1bc027a00 con 0x7fc1bc0f7960 2021-11-03T17:37:19.180 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.186+0000 7fc1a37fe700 1 -- 172.21.15.27:0/238244441 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fc1b00192e0 con 0x7fc1bc0f7960 2021-11-03T17:37:19.181 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:37:19.181 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:37:19.181 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:37:19.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:37:19.182 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:37:19.183 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:37:19.183 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:37:19.183 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:37:19.184 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:37:19.184 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:37:19.185 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:37:19.185 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:37:19.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:37:19.186 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:37:19.186 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:37:19.187 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:37:19.187 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:37:19.187 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:37:19.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.188+0000 7fc1c15f4700 1 -- 172.21.15.27:0/238244441 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc1a40640b0 msgr2=0x7fc1a4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:37:19.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.188+0000 7fc1c15f4700 1 --2- 172.21.15.27:0/238244441 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc1a40640b0 0x7fc1a4066570 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7fc1ac004d80 tx=0x7fc1ac016040).stop 2021-11-03T17:37:19.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.188+0000 7fc1c15f4700 1 -- 172.21.15.27:0/238244441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1bc0f7960 msgr2=0x7fc1bc107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:37:19.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.188+0000 7fc1c15f4700 1 --2- 172.21.15.27:0/238244441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1bc0f7960 0x7fc1bc107400 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7fc1b000e7f0 tx=0x7fc1b000b040).stop 2021-11-03T17:37:19.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.188+0000 7fc1c15f4700 1 -- 172.21.15.27:0/238244441 shutdown_connections 2021-11-03T17:37:19.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.188+0000 7fc1c15f4700 1 --2- 172.21.15.27:0/238244441 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc1a40640b0 0x7fc1a4066570 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:19.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.188+0000 7fc1c15f4700 1 --2- 172.21.15.27:0/238244441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1bc0f66c0 0x7fc1bc106f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:19.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.188+0000 7fc1c15f4700 1 --2- 172.21.15.27:0/238244441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1bc0f7960 0x7fc1bc107400 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:19.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.188+0000 7fc1c15f4700 1 -- 172.21.15.27:0/238244441 >> 172.21.15.27:0/238244441 conn(0x7fc1bc0f1e20 msgr2=0x7fc1bc0fb3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:37:19.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.188+0000 7fc1c15f4700 1 -- 172.21.15.27:0/238244441 shutdown_connections 2021-11-03T17:37:19.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.188+0000 7fc1c15f4700 1 -- 172.21.15.27:0/238244441 wait complete. 2021-11-03T17:37:19.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.290+0000 7f2f78665700 1 -- 172.21.15.27:0/204403378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f700f55e0 msgr2=0x7f2f700f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:37:19.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.290+0000 7f2f78665700 1 --2- 172.21.15.27:0/204403378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f700f55e0 0x7f2f700f5a00 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7f2f6c004660 tx=0x7f2f6c009b30).stop 2021-11-03T17:37:19.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.290+0000 7f2f78665700 1 -- 172.21.15.27:0/204403378 shutdown_connections 2021-11-03T17:37:19.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.290+0000 7f2f78665700 1 --2- 172.21.15.27:0/204403378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f700f5ef0 0x7f2f700f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:19.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.290+0000 7f2f78665700 1 --2- 172.21.15.27:0/204403378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f700f55e0 0x7f2f700f5a00 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:19.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.290+0000 7f2f78665700 1 -- 172.21.15.27:0/204403378 >> 172.21.15.27:0/204403378 conn(0x7f2f700f0de0 msgr2=0x7f2f700f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:37:19.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.290+0000 7f2f78665700 1 -- 172.21.15.27:0/204403378 shutdown_connections 2021-11-03T17:37:19.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.290+0000 7f2f78665700 1 -- 172.21.15.27:0/204403378 wait complete. 2021-11-03T17:37:19.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.290+0000 7f2f78665700 1 Processor -- start 2021-11-03T17:37:19.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.290+0000 7f2f78665700 1 -- start start 2021-11-03T17:37:19.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.291+0000 7f2f78665700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f700f55e0 0x7f2f70109c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:37:19.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.291+0000 7f2f78665700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f700f5ef0 0x7f2f7010c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:37:19.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.291+0000 7f2f78665700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f700604e0 con 0x7f2f700f55e0 2021-11-03T17:37:19.290 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.291+0000 7f2f78665700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f70060620 con 0x7f2f700f5ef0 2021-11-03T17:37:19.290 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.291+0000 7f2f76401700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f700f55e0 0x7f2f70109c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:37:19.290 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.291+0000 7f2f76401700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f700f55e0 0x7f2f70109c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58482/0 (socket says 172.21.15.27:58482) 2021-11-03T17:37:19.290 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.291+0000 7f2f76401700 1 -- 172.21.15.27:0/3774870894 learned_addr learned my addr 172.21.15.27:0/3774870894 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:37:19.291 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.291+0000 7f2f75c00700 1 --2- 172.21.15.27:0/3774870894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f700f5ef0 0x7f2f7010c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:37:19.291 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.291+0000 7f2f76401700 1 -- 172.21.15.27:0/3774870894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f700f5ef0 msgr2=0x7f2f7010c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:37:19.291 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.291+0000 7f2f76401700 1 --2- 172.21.15.27:0/3774870894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f700f5ef0 0x7f2f7010c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:19.291 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.291+0000 7f2f76401700 1 -- 172.21.15.27:0/3774870894 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f6c005040 con 0x7f2f700f55e0 2021-11-03T17:37:19.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.291+0000 7f2f76401700 1 --2- 172.21.15.27:0/3774870894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f700f55e0 0x7f2f70109c10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f2f6c009990 tx=0x7f2f6c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:37:19.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.291+0000 7f2f66ffd700 1 -- 172.21.15.27:0/3774870894 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f6c018070 con 0x7f2f700f55e0 2021-11-03T17:37:19.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.291+0000 7f2f66ffd700 1 -- 172.21.15.27:0/3774870894 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f2f6c007ba0 con 0x7f2f700f55e0 2021-11-03T17:37:19.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.291+0000 7f2f78665700 1 -- 172.21.15.27:0/3774870894 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f7010c660 con 0x7f2f700f55e0 2021-11-03T17:37:19.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.291+0000 7f2f66ffd700 1 -- 172.21.15.27:0/3774870894 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f6c01c3c0 con 0x7f2f700f55e0 2021-11-03T17:37:19.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.291+0000 7f2f78665700 1 -- 172.21.15.27:0/3774870894 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f7010cb90 con 0x7f2f700f55e0 2021-11-03T17:37:19.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.292+0000 7f2f78665700 1 -- 172.21.15.27:0/3774870894 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f70045ad0 con 0x7f2f700f55e0 2021-11-03T17:37:19.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.293+0000 7f2f66ffd700 1 -- 172.21.15.27:0/3774870894 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f2f6c01d420 con 0x7f2f700f55e0 2021-11-03T17:37:19.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.293+0000 7f2f66ffd700 1 --2- 172.21.15.27:0/3774870894 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2f5c0640b0 0x7f2f5c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:37:19.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.293+0000 7f2f66ffd700 1 -- 172.21.15.27:0/3774870894 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f2f6c082470 con 0x7f2f700f55e0 2021-11-03T17:37:19.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.293+0000 7f2f75c00700 1 --2- 172.21.15.27:0/3774870894 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2f5c0640b0 0x7f2f5c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:37:19.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.294+0000 7f2f75c00700 1 --2- 172.21.15.27:0/3774870894 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2f5c0640b0 0x7f2f5c066570 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f2f60004ce0 tx=0x7f2f6000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:37:19.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.297+0000 7f2f66ffd700 1 -- 172.21.15.27:0/3774870894 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f2f6c054830 con 0x7f2f700f55e0 2021-11-03T17:37:19.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:19 smithi115 ceph-mon[65453]: from='client.34808 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:37:19.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:19 smithi115 ceph-mon[65453]: from='client.34812 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:37:19.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:19 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/238244441' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:19.496 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.502+0000 7f2f78665700 1 -- 172.21.15.27:0/3774870894 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2f700fde70 con 0x7f2f700f55e0 2021-11-03T17:37:19.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.503+0000 7f2f66ffd700 1 -- 172.21.15.27:0/3774870894 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f2f6c054420 con 0x7f2f700f55e0 2021-11-03T17:37:19.497 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:37:19.498 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:37:19.498 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:37:19.498 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:37:19.499 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:37:19.499 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:37:19.499 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:37:19.499 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:37:19.499 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:37:19.500 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:37:19.500 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:37:19.500 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:37:19.501 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:37:19.501 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:37:19.501 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:37:19.501 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:37:19.501 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:37:19.502 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:37:19.502 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:37:19.503 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:37:19.503 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:37:19.503 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:37:19.503 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:37:19.504 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:37:19.504 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:37:19.504 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:37:19.504 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:37:19.505 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:37:19.505 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:37:19.505 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:37:19.505 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:37:19.505 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:37:19.506 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:37:19.506 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:37:19.506 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:37:19.506 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:37:19.507 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:37:19.507 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:37:19.507 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:37:19.509 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.505+0000 7f2f78665700 1 -- 172.21.15.27:0/3774870894 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2f5c0640b0 msgr2=0x7f2f5c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:37:19.510 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.505+0000 7f2f78665700 1 --2- 172.21.15.27:0/3774870894 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2f5c0640b0 0x7f2f5c066570 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f2f60004ce0 tx=0x7f2f6000b040).stop 2021-11-03T17:37:19.510 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.505+0000 7f2f78665700 1 -- 172.21.15.27:0/3774870894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f700f55e0 msgr2=0x7f2f70109c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:37:19.510 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.505+0000 7f2f78665700 1 --2- 172.21.15.27:0/3774870894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f700f55e0 0x7f2f70109c10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f2f6c009990 tx=0x7f2f6c009b30).stop 2021-11-03T17:37:19.510 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.505+0000 7f2f78665700 1 -- 172.21.15.27:0/3774870894 shutdown_connections 2021-11-03T17:37:19.511 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.505+0000 7f2f78665700 1 --2- 172.21.15.27:0/3774870894 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2f5c0640b0 0x7f2f5c066570 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:19.511 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.505+0000 7f2f78665700 1 --2- 172.21.15.27:0/3774870894 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f700f55e0 0x7f2f70109c10 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:19.511 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.505+0000 7f2f78665700 1 --2- 172.21.15.27:0/3774870894 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f700f5ef0 0x7f2f7010c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:19.512 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.505+0000 7f2f78665700 1 -- 172.21.15.27:0/3774870894 >> 172.21.15.27:0/3774870894 conn(0x7f2f700f0de0 msgr2=0x7f2f700fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:37:19.512 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.505+0000 7f2f78665700 1 -- 172.21.15.27:0/3774870894 shutdown_connections 2021-11-03T17:37:19.512 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:19.505+0000 7f2f78665700 1 -- 172.21.15.27:0/3774870894 wait complete. 2021-11-03T17:37:19.512 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:37:19.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:19 smithi027 ceph-mon[97244]: from='client.34808 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:37:19.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:19 smithi027 ceph-mon[97244]: from='client.34812 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:37:19.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:19 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/238244441' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:20.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1282.389354 secs 2021-11-03T17:37:20.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:20 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3774870894' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:37:20.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:20 smithi115 ceph-mon[65453]: pgmap v681: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:20.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1282.389354 secs 2021-11-03T17:37:20.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:20 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3774870894' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:37:20.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:20 smithi027 ceph-mon[97244]: pgmap v681: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:22.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:22 smithi115 ceph-mon[65453]: pgmap v682: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:22.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:22 smithi027 ceph-mon[97244]: pgmap v682: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:24.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:37:24.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:37:24.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:37:24.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:24.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:24.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:24.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:24.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:37:24.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:37:24.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:37:24.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:24.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:24.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:24.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:25.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:24 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:37:25.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:24 smithi115 ceph-mon[65453]: pgmap v683: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:25.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:24 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1287.389489 secs 2021-11-03T17:37:25.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:24 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:37:25.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:24 smithi027 ceph-mon[97244]: pgmap v683: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:25.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:24 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1287.389489 secs 2021-11-03T17:37:26.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:26 smithi115 ceph-mon[65453]: pgmap v684: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:26.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:26 smithi027 ceph-mon[97244]: pgmap v684: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:28.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:28 smithi115 ceph-mon[65453]: pgmap v685: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:28.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:28 smithi027 ceph-mon[97244]: pgmap v685: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:29.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:29 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1292.389581 secs 2021-11-03T17:37:30.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:29 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1292.389581 secs 2021-11-03T17:37:30.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:30 smithi115 ceph-mon[65453]: pgmap v686: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:31.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:30 smithi027 ceph-mon[97244]: pgmap v686: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:32.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:32 smithi115 ceph-mon[65453]: pgmap v687: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:32.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:32 smithi027 ceph-mon[97244]: pgmap v687: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:34.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:37:34.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:37:34.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:37:34.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:34.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:34.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:34.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:34.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:37:34.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:37:34.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:37:34.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:34.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:34.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:34.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:35.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:34 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:37:35.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:34 smithi115 ceph-mon[65453]: pgmap v688: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:35.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:34 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1297.389720 secs 2021-11-03T17:37:35.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:34 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:37:35.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:34 smithi027 ceph-mon[97244]: pgmap v688: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:35.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:34 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1297.389720 secs 2021-11-03T17:37:36.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:36 smithi115 ceph-mon[65453]: pgmap v689: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:36.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:36 smithi027 ceph-mon[97244]: pgmap v689: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:38.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:38 smithi115 ceph-mon[65453]: pgmap v690: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:38.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:38 smithi027 ceph-mon[97244]: pgmap v690: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:39.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:39 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1302.389858 secs 2021-11-03T17:37:40.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:39 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1302.389858 secs 2021-11-03T17:37:40.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:40 smithi115 ceph-mon[65453]: pgmap v691: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:41.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:40 smithi027 ceph-mon[97244]: pgmap v691: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:42.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:42 smithi115 ceph-mon[65453]: pgmap v692: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:42.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:42 smithi027 ceph-mon[97244]: pgmap v692: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:44.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:37:44.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:37:44.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:37:44.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:44.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:44.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:44.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:44.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:37:44.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:37:44.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:37:44.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:44.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:44.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:44.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:45.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:44 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:37:45.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:44 smithi115 ceph-mon[65453]: pgmap v693: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:45.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:44 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1307.389999 secs 2021-11-03T17:37:45.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:44 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:37:45.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:44 smithi027 ceph-mon[97244]: pgmap v693: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:45.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:44 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1307.389999 secs 2021-11-03T17:37:46.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:46 smithi115 ceph-mon[65453]: pgmap v694: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:46.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:46 smithi027 ceph-mon[97244]: pgmap v694: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:48.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:48 smithi115 ceph-mon[65453]: pgmap v695: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:48.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:48 smithi027 ceph-mon[97244]: pgmap v695: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:49.613 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.617+0000 7faa98b8e700 1 -- 172.21.15.27:0/901800435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa940f66e0 msgr2=0x7faa940f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:37:49.614 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.617+0000 7faa98b8e700 1 --2- 172.21.15.27:0/901800435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa940f66e0 0x7faa940f6b00 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7faa8c004660 tx=0x7faa8c009b30).stop 2021-11-03T17:37:49.614 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.617+0000 7faa98b8e700 1 -- 172.21.15.27:0/901800435 shutdown_connections 2021-11-03T17:37:49.615 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.617+0000 7faa98b8e700 1 --2- 172.21.15.27:0/901800435 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa940f6ff0 0x7faa940f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:37:49.615 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.617+0000 7faa98b8e700 1 --2- 172.21.15.27:0/901800435 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa940f66e0 0x7faa940f6b00 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:49.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.617+0000 7faa98b8e700 1 -- 172.21.15.27:0/901800435 >> 172.21.15.27:0/901800435 conn(0x7faa940f1f00 msgr2=0x7faa940f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:37:49.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.617+0000 7faa98b8e700 1 -- 172.21.15.27:0/901800435 shutdown_connections 2021-11-03T17:37:49.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.617+0000 7faa98b8e700 1 -- 172.21.15.27:0/901800435 wait complete. 2021-11-03T17:37:49.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.618+0000 7faa98b8e700 1 Processor -- start 2021-11-03T17:37:49.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.618+0000 7faa98b8e700 1 -- start start 2021-11-03T17:37:49.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.618+0000 7faa98b8e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa940f66e0 0x7faa9406b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:37:49.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.618+0000 7faa98b8e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa940f6ff0 0x7faa94069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:37:49.618 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.618+0000 7faa98b8e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9406a410 con 0x7faa940f6ff0 2021-11-03T17:37:49.618 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.618+0000 7faa98b8e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9406a550 con 0x7faa940f66e0 2021-11-03T17:37:49.619 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.618+0000 7faa91d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa940f6ff0 0x7faa94069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:37:49.619 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.618+0000 7faa91d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa940f6ff0 0x7faa94069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58502/0 (socket says 172.21.15.27:58502) 2021-11-03T17:37:49.619 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.618+0000 7faa91d9b700 1 -- 172.21.15.27:0/3099566701 learned_addr learned my addr 172.21.15.27:0/3099566701 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:37:49.620 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.618+0000 7faa9259c700 1 --2- 172.21.15.27:0/3099566701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa940f66e0 0x7faa9406b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:37:49.620 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.619+0000 7faa91d9b700 1 -- 172.21.15.27:0/3099566701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa940f66e0 msgr2=0x7faa9406b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:37:49.620 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.619+0000 7faa91d9b700 1 --2- 172.21.15.27:0/3099566701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa940f66e0 0x7faa9406b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:49.620 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.619+0000 7faa91d9b700 1 -- 172.21.15.27:0/3099566701 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa8c005040 con 0x7faa940f6ff0 2021-11-03T17:37:49.620 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.619+0000 7faa9259c700 1 --2- 172.21.15.27:0/3099566701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa940f66e0 0x7faa9406b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:37:49.621 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.619+0000 7faa91d9b700 1 --2- 172.21.15.27:0/3099566701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa940f6ff0 0x7faa94069e20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7faa7c00b410 tx=0x7faa7c006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:37:49.621 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.619+0000 7faa8affd700 1 -- 172.21.15.27:0/3099566701 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa7c0125f0 con 0x7faa940f6ff0 2021-11-03T17:37:49.621 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.619+0000 7faa8affd700 1 -- 172.21.15.27:0/3099566701 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7faa7c012c00 con 0x7faa940f6ff0 2021-11-03T17:37:49.621 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.619+0000 7faa98b8e700 1 -- 172.21.15.27:0/3099566701 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa9406a830 con 0x7faa940f6ff0 2021-11-03T17:37:49.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.619+0000 7faa8affd700 1 -- 172.21.15.27:0/3099566701 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faa7c002ae0 con 0x7faa940f6ff0 2021-11-03T17:37:49.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.619+0000 7faa98b8e700 1 -- 172.21.15.27:0/3099566701 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa9410efb0 con 0x7faa940f6ff0 2021-11-03T17:37:49.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.620+0000 7faa98b8e700 1 -- 172.21.15.27:0/3099566701 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa94045ad0 con 0x7faa940f6ff0 2021-11-03T17:37:49.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.620+0000 7faa8affd700 1 -- 172.21.15.27:0/3099566701 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7faa7c00f030 con 0x7faa940f6ff0 2021-11-03T17:37:49.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.621+0000 7faa8affd700 1 --2- 172.21.15.27:0/3099566701 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7faa80064090 0x7faa80066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:37:49.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.621+0000 7faa8affd700 1 -- 172.21.15.27:0/3099566701 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7faa7c080cc0 con 0x7faa940f6ff0 2021-11-03T17:37:49.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.621+0000 7faa9259c700 1 --2- 172.21.15.27:0/3099566701 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7faa80064090 0x7faa80066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:37:49.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.621+0000 7faa9259c700 1 --2- 172.21.15.27:0/3099566701 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7faa80064090 0x7faa80066550 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7faa8c009830 tx=0x7faa8c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:37:49.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.625+0000 7faa8affd700 1 -- 172.21.15.27:0/3099566701 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7faa7c052fc0 con 0x7faa940f6ff0 2021-11-03T17:37:49.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.804+0000 7faa98b8e700 1 -- 172.21.15.27:0/3099566701 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faa940ff0c0 con 0x7faa80064090 2021-11-03T17:37:49.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.806+0000 7faa8affd700 1 -- 172.21.15.27:0/3099566701 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7faa940ff0c0 con 0x7faa80064090 2021-11-03T17:37:49.802 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.807+0000 7faa98b8e700 1 -- 172.21.15.27:0/3099566701 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7faa80064090 msgr2=0x7faa80066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:37:49.802 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.808+0000 7faa98b8e700 1 --2- 172.21.15.27:0/3099566701 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7faa80064090 0x7faa80066550 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7faa8c009830 tx=0x7faa8c009b30).stop 2021-11-03T17:37:49.802 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.808+0000 7faa98b8e700 1 -- 172.21.15.27:0/3099566701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa940f6ff0 msgr2=0x7faa94069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:37:49.803 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.808+0000 7faa98b8e700 1 --2- 172.21.15.27:0/3099566701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa940f6ff0 0x7faa94069e20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7faa7c00b410 tx=0x7faa7c006ef0).stop 2021-11-03T17:37:49.803 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.808+0000 7faa98b8e700 1 -- 172.21.15.27:0/3099566701 shutdown_connections 2021-11-03T17:37:49.803 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.808+0000 7faa98b8e700 1 --2- 172.21.15.27:0/3099566701 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7faa80064090 0x7faa80066550 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:49.803 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.808+0000 7faa98b8e700 1 --2- 172.21.15.27:0/3099566701 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faa940f66e0 0x7faa9406b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:49.804 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.808+0000 7faa98b8e700 1 --2- 172.21.15.27:0/3099566701 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faa940f6ff0 0x7faa94069e20 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:49.804 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.808+0000 7faa98b8e700 1 -- 172.21.15.27:0/3099566701 >> 172.21.15.27:0/3099566701 conn(0x7faa940f1f00 msgr2=0x7faa940fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:37:49.804 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.808+0000 7faa98b8e700 1 -- 172.21.15.27:0/3099566701 shutdown_connections 2021-11-03T17:37:49.804 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.808+0000 7faa98b8e700 1 -- 172.21.15.27:0/3099566701 wait complete. 2021-11-03T17:37:49.811 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:37:49.902 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.908+0000 7f6e4943d700 1 -- 172.21.15.27:0/3576230653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e440f6550 msgr2=0x7f6e440f6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:37:49.903 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.908+0000 7f6e4943d700 1 --2- 172.21.15.27:0/3576230653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e440f6550 0x7f6e440f6970 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f6e34004660 tx=0x7f6e34009b30).stop 2021-11-03T17:37:49.903 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.908+0000 7f6e4943d700 1 -- 172.21.15.27:0/3576230653 shutdown_connections 2021-11-03T17:37:49.903 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.908+0000 7f6e4943d700 1 --2- 172.21.15.27:0/3576230653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e440f77f0 0x7f6e440f7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:37:49.904 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.908+0000 7f6e4943d700 1 --2- 172.21.15.27:0/3576230653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e440f6550 0x7f6e440f6970 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:49.904 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.908+0000 7f6e4943d700 1 -- 172.21.15.27:0/3576230653 >> 172.21.15.27:0/3576230653 conn(0x7f6e440f1cd0 msgr2=0x7f6e440f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:37:49.904 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.908+0000 7f6e4943d700 1 -- 172.21.15.27:0/3576230653 shutdown_connections 2021-11-03T17:37:49.904 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.909+0000 7f6e4943d700 1 -- 172.21.15.27:0/3576230653 wait complete. 2021-11-03T17:37:49.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.909+0000 7f6e4943d700 1 Processor -- start 2021-11-03T17:37:49.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.909+0000 7f6e4943d700 1 -- start start 2021-11-03T17:37:49.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.909+0000 7f6e4943d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e440f6550 0x7f6e4406b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:37:49.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.909+0000 7f6e4943d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e440f77f0 0x7f6e44069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:37:49.908 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.909+0000 7f6e4943d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4406a410 con 0x7f6e440f77f0 2021-11-03T17:37:49.908 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.909+0000 7f6e4943d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4406a550 con 0x7f6e440f6550 2021-11-03T17:37:49.908 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.910+0000 7f6e427fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e440f77f0 0x7f6e44069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:37:49.908 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.910+0000 7f6e427fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e440f77f0 0x7f6e44069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58512/0 (socket says 172.21.15.27:58512) 2021-11-03T17:37:49.908 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.910+0000 7f6e427fc700 1 -- 172.21.15.27:0/1533590918 learned_addr learned my addr 172.21.15.27:0/1533590918 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:37:49.909 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.910+0000 7f6e42ffd700 1 --2- 172.21.15.27:0/1533590918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e440f6550 0x7f6e4406b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:37:49.909 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.910+0000 7f6e427fc700 1 -- 172.21.15.27:0/1533590918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e440f6550 msgr2=0x7f6e4406b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:37:49.909 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.910+0000 7f6e427fc700 1 --2- 172.21.15.27:0/1533590918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e440f6550 0x7f6e4406b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:49.909 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.910+0000 7f6e427fc700 1 -- 172.21.15.27:0/1533590918 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e34005040 con 0x7f6e440f77f0 2021-11-03T17:37:49.910 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.910+0000 7f6e427fc700 1 --2- 172.21.15.27:0/1533590918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e440f77f0 0x7f6e44069e20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f6e3800b410 tx=0x7f6e38006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:37:49.910 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.910+0000 7f6e2b7fe700 1 -- 172.21.15.27:0/1533590918 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e380125f0 con 0x7f6e440f77f0 2021-11-03T17:37:49.910 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.910+0000 7f6e4943d700 1 -- 172.21.15.27:0/1533590918 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e4406a830 con 0x7f6e440f77f0 2021-11-03T17:37:49.910 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.910+0000 7f6e2b7fe700 1 -- 172.21.15.27:0/1533590918 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f6e38012c00 con 0x7f6e440f77f0 2021-11-03T17:37:49.911 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.910+0000 7f6e2b7fe700 1 -- 172.21.15.27:0/1533590918 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e38002ae0 con 0x7f6e440f77f0 2021-11-03T17:37:49.911 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.910+0000 7f6e4943d700 1 -- 172.21.15.27:0/1533590918 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e4410edc0 con 0x7f6e440f77f0 2021-11-03T17:37:49.911 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.911+0000 7f6e4943d700 1 -- 172.21.15.27:0/1533590918 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e44045ad0 con 0x7f6e440f77f0 2021-11-03T17:37:49.911 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.912+0000 7f6e2b7fe700 1 -- 172.21.15.27:0/1533590918 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f6e3800f030 con 0x7f6e440f77f0 2021-11-03T17:37:49.911 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.912+0000 7f6e2b7fe700 1 --2- 172.21.15.27:0/1533590918 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6e2c064100 0x7f6e2c0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:37:49.912 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.912+0000 7f6e2b7fe700 1 -- 172.21.15.27:0/1533590918 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f6e38080cc0 con 0x7f6e440f77f0 2021-11-03T17:37:49.912 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.912+0000 7f6e42ffd700 1 --2- 172.21.15.27:0/1533590918 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6e2c064100 0x7f6e2c0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:37:49.912 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.913+0000 7f6e42ffd700 1 --2- 172.21.15.27:0/1533590918 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6e2c064100 0x7f6e2c0665c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f6e34004e30 tx=0x7f6e34009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:37:49.913 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:49.916+0000 7f6e2b7fe700 1 -- 172.21.15.27:0/1533590918 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f6e38052fc0 con 0x7f6e440f77f0 2021-11-03T17:37:49.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:49 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1312.390147 secs 2021-11-03T17:37:50.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:49 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1312.390147 secs 2021-11-03T17:37:50.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.090+0000 7f6e4943d700 1 -- 172.21.15.27:0/1533590918 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e440feea0 con 0x7f6e2c064100 2021-11-03T17:37:50.099 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.104+0000 7f6e2b7fe700 1 -- 172.21.15.27:0/1533590918 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f6e440feea0 con 0x7f6e2c064100 2021-11-03T17:37:50.101 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:37:50.101 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (29m) 9m ago 30m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:37:50.101 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (21m) 9m ago 30m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:37:50.102 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (20m) 8m ago 29m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:37:50.102 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (29m) 9m ago 30m 40.2M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:37:50.103 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (22m) 9m ago 25m 18.2M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:37:50.103 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (25m) 9m ago 25m 3841M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:37:50.103 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (21m) 8m ago 25m 17.6M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:37:50.104 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (23m) 8m ago 25m 19.3M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:37:50.104 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (23m) 9m ago 31m 497M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:37:50.104 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (22m) 8m ago 29m 415M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:37:50.105 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (22m) 9m ago 32m 119M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:37:50.105 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (21m) 8m ago 29m 71.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:37:50.106 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (30m) 9m ago 30m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:37:50.106 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (29m) 8m ago 29m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:37:50.106 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (20m) 9m ago 29m 290M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:37:50.106 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (20m) 9m ago 28m 317M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:37:50.107 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (19m) 9m ago 28m 229M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:37:50.107 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (19m) 8m ago 28m 329M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:37:50.107 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (18m) 8m ago 27m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:37:50.108 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (18m) 8m ago 27m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:37:50.108 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (29m) 9m ago 30m 48.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:37:50.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.106+0000 7f6e4943d700 1 -- 172.21.15.27:0/1533590918 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6e2c064100 msgr2=0x7f6e2c0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:37:50.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.106+0000 7f6e4943d700 1 --2- 172.21.15.27:0/1533590918 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6e2c064100 0x7f6e2c0665c0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 rx=0x7f6e34004e30 tx=0x7f6e34009b30).stop 2021-11-03T17:37:50.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.106+0000 7f6e4943d700 1 -- 172.21.15.27:0/1533590918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e440f77f0 msgr2=0x7f6e44069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:37:50.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.106+0000 7f6e4943d700 1 --2- 172.21.15.27:0/1533590918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e440f77f0 0x7f6e44069e20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f6e3800b410 tx=0x7f6e38006ef0).stop 2021-11-03T17:37:50.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.106+0000 7f6e4943d700 1 -- 172.21.15.27:0/1533590918 shutdown_connections 2021-11-03T17:37:50.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.106+0000 7f6e4943d700 1 --2- 172.21.15.27:0/1533590918 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6e2c064100 0x7f6e2c0665c0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:50.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.106+0000 7f6e4943d700 1 --2- 172.21.15.27:0/1533590918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e440f6550 0x7f6e4406b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:50.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.106+0000 7f6e4943d700 1 --2- 172.21.15.27:0/1533590918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e440f77f0 0x7f6e44069e20 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:50.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.106+0000 7f6e4943d700 1 -- 172.21.15.27:0/1533590918 >> 172.21.15.27:0/1533590918 conn(0x7f6e440f1cd0 msgr2=0x7f6e440fb240 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:37:50.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.106+0000 7f6e4943d700 1 -- 172.21.15.27:0/1533590918 shutdown_connections 2021-11-03T17:37:50.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.106+0000 7f6e4943d700 1 -- 172.21.15.27:0/1533590918 wait complete. 2021-11-03T17:37:50.204 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.206+0000 7fd2da127700 1 -- 172.21.15.27:0/3022825000 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2d40f7960 msgr2=0x7fd2d40f7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:37:50.204 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.206+0000 7fd2da127700 1 --2- 172.21.15.27:0/3022825000 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2d40f7960 0x7fd2d40f7dc0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7fd2c8004660 tx=0x7fd2c8009b30).stop 2021-11-03T17:37:50.205 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.206+0000 7fd2da127700 1 -- 172.21.15.27:0/3022825000 shutdown_connections 2021-11-03T17:37:50.205 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.206+0000 7fd2da127700 1 --2- 172.21.15.27:0/3022825000 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2d40f7960 0x7fd2d40f7dc0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:50.205 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.206+0000 7fd2da127700 1 --2- 172.21.15.27:0/3022825000 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2d40f66c0 0x7fd2d40f6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:50.205 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.206+0000 7fd2da127700 1 -- 172.21.15.27:0/3022825000 >> 172.21.15.27:0/3022825000 conn(0x7fd2d40f1e20 msgr2=0x7fd2d40f4240 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:37:50.206 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.206+0000 7fd2da127700 1 -- 172.21.15.27:0/3022825000 shutdown_connections 2021-11-03T17:37:50.206 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.206+0000 7fd2da127700 1 -- 172.21.15.27:0/3022825000 wait complete. 2021-11-03T17:37:50.206 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.206+0000 7fd2da127700 1 Processor -- start 2021-11-03T17:37:50.206 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.206+0000 7fd2da127700 1 -- start start 2021-11-03T17:37:50.207 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.207+0000 7fd2da127700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2d40f66c0 0x7fd2d4106f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:37:50.207 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.207+0000 7fd2da127700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2d40f7960 0x7fd2d4107440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:37:50.207 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.207+0000 7fd2da127700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d41079f0 con 0x7fd2d40f66c0 2021-11-03T17:37:50.207 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.207+0000 7fd2da127700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2d4107b30 con 0x7fd2d40f7960 2021-11-03T17:37:50.207 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.207+0000 7fd2d37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2d40f66c0 0x7fd2d4106f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:37:50.208 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.207+0000 7fd2d37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2d40f66c0 0x7fd2d4106f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58520/0 (socket says 172.21.15.27:58520) 2021-11-03T17:37:50.208 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.207+0000 7fd2d37fe700 1 -- 172.21.15.27:0/1664222660 learned_addr learned my addr 172.21.15.27:0/1664222660 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:37:50.208 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.207+0000 7fd2d2ffd700 1 --2- 172.21.15.27:0/1664222660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2d40f7960 0x7fd2d4107440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:37:50.208 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.207+0000 7fd2d37fe700 1 -- 172.21.15.27:0/1664222660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2d40f7960 msgr2=0x7fd2d4107440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:37:50.209 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.207+0000 7fd2d37fe700 1 --2- 172.21.15.27:0/1664222660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2d40f7960 0x7fd2d4107440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:50.209 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.207+0000 7fd2d37fe700 1 -- 172.21.15.27:0/1664222660 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c8005040 con 0x7fd2d40f66c0 2021-11-03T17:37:50.210 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.207+0000 7fd2d37fe700 1 --2- 172.21.15.27:0/1664222660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2d40f66c0 0x7fd2d4106f50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7fd2c400c790 tx=0x7fd2c4007d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:37:50.210 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.207+0000 7fd2bbfff700 1 -- 172.21.15.27:0/1664222660 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2c400e680 con 0x7fd2d40f66c0 2021-11-03T17:37:50.210 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.207+0000 7fd2bbfff700 1 -- 172.21.15.27:0/1664222660 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fd2c400ec90 con 0x7fd2d40f66c0 2021-11-03T17:37:50.210 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.207+0000 7fd2da127700 1 -- 172.21.15.27:0/1664222660 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2d41084d0 con 0x7fd2d40f66c0 2021-11-03T17:37:50.211 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.207+0000 7fd2bbfff700 1 -- 172.21.15.27:0/1664222660 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2c4012d00 con 0x7fd2d40f66c0 2021-11-03T17:37:50.211 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.207+0000 7fd2da127700 1 -- 172.21.15.27:0/1664222660 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2d4108a90 con 0x7fd2d40f66c0 2021-11-03T17:37:50.212 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.208+0000 7fd2da127700 1 -- 172.21.15.27:0/1664222660 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2d4045ad0 con 0x7fd2d40f66c0 2021-11-03T17:37:50.212 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.209+0000 7fd2bbfff700 1 -- 172.21.15.27:0/1664222660 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fd2c4010070 con 0x7fd2d40f66c0 2021-11-03T17:37:50.213 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.209+0000 7fd2bbfff700 1 --2- 172.21.15.27:0/1664222660 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd2bc0640b0 0x7fd2bc066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:37:50.213 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.209+0000 7fd2bbfff700 1 -- 172.21.15.27:0/1664222660 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fd2c40809f0 con 0x7fd2d40f66c0 2021-11-03T17:37:50.214 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.209+0000 7fd2d2ffd700 1 --2- 172.21.15.27:0/1664222660 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd2bc0640b0 0x7fd2bc066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:37:50.214 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.210+0000 7fd2d2ffd700 1 --2- 172.21.15.27:0/1664222660 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd2bc0640b0 0x7fd2bc066570 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7fd2c8009da0 tx=0x7fd2c8009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:37:50.215 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.213+0000 7fd2bbfff700 1 -- 172.21.15.27:0/1664222660 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fd2c4082e60 con 0x7fd2d40f66c0 2021-11-03T17:37:50.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.443+0000 7fd2da127700 1 -- 172.21.15.27:0/1664222660 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd2d40fefd0 con 0x7fd2d40f66c0 2021-11-03T17:37:50.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.443+0000 7fd2bbfff700 1 -- 172.21.15.27:0/1664222660 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fd2c4052c00 con 0x7fd2d40f66c0 2021-11-03T17:37:50.438 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:37:50.438 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:37:50.439 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:37:50.439 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:37:50.439 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:37:50.439 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:37:50.439 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:37:50.440 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:37:50.440 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:37:50.440 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:37:50.440 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:37:50.441 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:37:50.441 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:37:50.442 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:37:50.442 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:37:50.442 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:37:50.442 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:37:50.443 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:37:50.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.445+0000 7fd2da127700 1 -- 172.21.15.27:0/1664222660 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd2bc0640b0 msgr2=0x7fd2bc066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:37:50.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.445+0000 7fd2da127700 1 --2- 172.21.15.27:0/1664222660 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd2bc0640b0 0x7fd2bc066570 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7fd2c8009da0 tx=0x7fd2c8009b30).stop 2021-11-03T17:37:50.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.445+0000 7fd2da127700 1 -- 172.21.15.27:0/1664222660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2d40f66c0 msgr2=0x7fd2d4106f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:37:50.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.445+0000 7fd2da127700 1 --2- 172.21.15.27:0/1664222660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2d40f66c0 0x7fd2d4106f50 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7fd2c400c790 tx=0x7fd2c4007d90).stop 2021-11-03T17:37:50.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.445+0000 7fd2da127700 1 -- 172.21.15.27:0/1664222660 shutdown_connections 2021-11-03T17:37:50.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.445+0000 7fd2da127700 1 --2- 172.21.15.27:0/1664222660 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd2bc0640b0 0x7fd2bc066570 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:50.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.445+0000 7fd2da127700 1 --2- 172.21.15.27:0/1664222660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd2d40f66c0 0x7fd2d4106f50 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:50.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.445+0000 7fd2da127700 1 --2- 172.21.15.27:0/1664222660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd2d40f7960 0x7fd2d4107440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:50.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.445+0000 7fd2da127700 1 -- 172.21.15.27:0/1664222660 >> 172.21.15.27:0/1664222660 conn(0x7fd2d40f1e20 msgr2=0x7fd2d40fb3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:37:50.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.446+0000 7fd2da127700 1 -- 172.21.15.27:0/1664222660 shutdown_connections 2021-11-03T17:37:50.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.446+0000 7fd2da127700 1 -- 172.21.15.27:0/1664222660 wait complete. 2021-11-03T17:37:50.542 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.546+0000 7fb74517b700 1 -- 172.21.15.27:0/942170288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7400f6860 msgr2=0x7fb7400f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:37:50.543 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.546+0000 7fb74517b700 1 --2- 172.21.15.27:0/942170288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7400f6860 0x7fb7400f6cc0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7fb728004660 tx=0x7fb728009b30).stop 2021-11-03T17:37:50.543 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.547+0000 7fb74517b700 1 -- 172.21.15.27:0/942170288 shutdown_connections 2021-11-03T17:37:50.543 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.547+0000 7fb74517b700 1 --2- 172.21.15.27:0/942170288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7400f6860 0x7fb7400f6cc0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:50.543 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.547+0000 7fb74517b700 1 --2- 172.21.15.27:0/942170288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7400f55c0 0x7fb7400f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:50.544 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.547+0000 7fb74517b700 1 -- 172.21.15.27:0/942170288 >> 172.21.15.27:0/942170288 conn(0x7fb7400f0d40 msgr2=0x7fb7400f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:37:50.544 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.547+0000 7fb74517b700 1 -- 172.21.15.27:0/942170288 shutdown_connections 2021-11-03T17:37:50.544 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.547+0000 7fb74517b700 1 -- 172.21.15.27:0/942170288 wait complete. 2021-11-03T17:37:50.544 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.547+0000 7fb74517b700 1 Processor -- start 2021-11-03T17:37:50.545 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.547+0000 7fb74517b700 1 -- start start 2021-11-03T17:37:50.545 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.547+0000 7fb74517b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7400f55c0 0x7fb740106f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:37:50.545 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.547+0000 7fb74517b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7400f6860 0x7fb740107420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:37:50.546 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.548+0000 7fb74517b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7401079d0 con 0x7fb7400f6860 2021-11-03T17:37:50.546 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.548+0000 7fb74517b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb740107b10 con 0x7fb7400f55c0 2021-11-03T17:37:50.546 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.548+0000 7fb73e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7400f6860 0x7fb740107420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:37:50.547 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.548+0000 7fb73e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7400f6860 0x7fb740107420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58532/0 (socket says 172.21.15.27:58532) 2021-11-03T17:37:50.547 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.548+0000 7fb73e59c700 1 -- 172.21.15.27:0/3635147399 learned_addr learned my addr 172.21.15.27:0/3635147399 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:37:50.547 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.548+0000 7fb73ed9d700 1 --2- 172.21.15.27:0/3635147399 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7400f55c0 0x7fb740106f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:37:50.548 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.548+0000 7fb73e59c700 1 -- 172.21.15.27:0/3635147399 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7400f55c0 msgr2=0x7fb740106f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:37:50.548 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.548+0000 7fb73e59c700 1 --2- 172.21.15.27:0/3635147399 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7400f55c0 0x7fb740106f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:50.548 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.548+0000 7fb73e59c700 1 -- 172.21.15.27:0/3635147399 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb728005040 con 0x7fb7400f6860 2021-11-03T17:37:50.548 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.548+0000 7fb73e59c700 1 --2- 172.21.15.27:0/3635147399 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7400f6860 0x7fb740107420 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7fb728009990 tx=0x7fb728009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:37:50.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.548+0000 7fb7377fe700 1 -- 172.21.15.27:0/3635147399 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb728018070 con 0x7fb7400f6860 2021-11-03T17:37:50.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.548+0000 7fb7377fe700 1 -- 172.21.15.27:0/3635147399 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fb728007ba0 con 0x7fb7400f6860 2021-11-03T17:37:50.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.548+0000 7fb74517b700 1 -- 172.21.15.27:0/3635147399 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb740108450 con 0x7fb7400f6860 2021-11-03T17:37:50.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.548+0000 7fb7377fe700 1 -- 172.21.15.27:0/3635147399 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb72801c3c0 con 0x7fb7400f6860 2021-11-03T17:37:50.550 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.548+0000 7fb74517b700 1 -- 172.21.15.27:0/3635147399 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7401089b0 con 0x7fb7400f6860 2021-11-03T17:37:50.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.549+0000 7fb74517b700 1 -- 172.21.15.27:0/3635147399 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb740045ad0 con 0x7fb7400f6860 2021-11-03T17:37:50.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.550+0000 7fb7377fe700 1 -- 172.21.15.27:0/3635147399 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fb72801d420 con 0x7fb7400f6860 2021-11-03T17:37:50.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.550+0000 7fb7377fe700 1 --2- 172.21.15.27:0/3635147399 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb72c0640b0 0x7fb72c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:37:50.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.550+0000 7fb7377fe700 1 -- 172.21.15.27:0/3635147399 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fb728082470 con 0x7fb7400f6860 2021-11-03T17:37:50.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.550+0000 7fb73ed9d700 1 --2- 172.21.15.27:0/3635147399 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb72c0640b0 0x7fb72c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:37:50.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.551+0000 7fb73ed9d700 1 --2- 172.21.15.27:0/3635147399 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb72c0640b0 0x7fb72c066570 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fb7400f66c0 tx=0x7fb730009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:37:50.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.554+0000 7fb7377fe700 1 -- 172.21.15.27:0/3635147399 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fb728054830 con 0x7fb7400f6860 2021-11-03T17:37:50.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.758+0000 7fb74517b700 1 -- 172.21.15.27:0/3635147399 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb7400fdd80 con 0x7fb7400f6860 2021-11-03T17:37:50.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.758+0000 7fb7377fe700 1 -- 172.21.15.27:0/3635147399 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7fb728054420 con 0x7fb7400f6860 2021-11-03T17:37:50.753 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:37:50.753 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:37:50.753 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:37:50.754 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:37:50.754 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:37:50.754 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:37:50.754 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:37:50.755 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:37:50.755 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:37:50.755 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:37:50.755 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:37:50.756 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:37:50.756 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:37:50.756 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:37:50.756 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:37:50.756 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:37:50.757 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:37:50.757 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:37:50.757 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:37:50.757 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:37:50.758 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:37:50.758 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:37:50.758 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:37:50.758 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:37:50.759 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:37:50.759 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:37:50.759 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:37:50.759 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:37:50.759 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:37:50.760 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:37:50.760 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:37:50.760 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:37:50.760 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:37:50.761 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:37:50.761 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:37:50.761 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:37:50.761 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:37:50.761 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:37:50.762 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:37:50.763 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.760+0000 7fb74517b700 1 -- 172.21.15.27:0/3635147399 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb72c0640b0 msgr2=0x7fb72c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:37:50.763 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.760+0000 7fb74517b700 1 --2- 172.21.15.27:0/3635147399 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb72c0640b0 0x7fb72c066570 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fb7400f66c0 tx=0x7fb730009040).stop 2021-11-03T17:37:50.763 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.760+0000 7fb74517b700 1 -- 172.21.15.27:0/3635147399 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7400f6860 msgr2=0x7fb740107420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:37:50.763 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.760+0000 7fb74517b700 1 --2- 172.21.15.27:0/3635147399 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7400f6860 0x7fb740107420 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7fb728009990 tx=0x7fb728009b30).stop 2021-11-03T17:37:50.764 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.760+0000 7fb74517b700 1 -- 172.21.15.27:0/3635147399 shutdown_connections 2021-11-03T17:37:50.764 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.760+0000 7fb74517b700 1 --2- 172.21.15.27:0/3635147399 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb72c0640b0 0x7fb72c066570 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:50.764 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.760+0000 7fb74517b700 1 --2- 172.21.15.27:0/3635147399 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7400f55c0 0x7fb740106f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:50.764 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.760+0000 7fb74517b700 1 --2- 172.21.15.27:0/3635147399 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7400f6860 0x7fb740107420 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:37:50.765 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.760+0000 7fb74517b700 1 -- 172.21.15.27:0/3635147399 >> 172.21.15.27:0/3635147399 conn(0x7fb7400f0d40 msgr2=0x7fb74005ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:37:50.765 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.761+0000 7fb74517b700 1 -- 172.21.15.27:0/3635147399 shutdown_connections 2021-11-03T17:37:50.765 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:37:50.761+0000 7fb74517b700 1 -- 172.21.15.27:0/3635147399 wait complete. 2021-11-03T17:37:50.765 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:37:50.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:50 smithi115 ceph-mon[65453]: from='client.34824 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:37:50.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:50 smithi115 ceph-mon[65453]: from='client.34828 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:37:50.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:50 smithi115 ceph-mon[65453]: pgmap v696: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:50.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:50 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1664222660' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:51.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:50 smithi027 ceph-mon[97244]: from='client.34824 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:37:51.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:50 smithi027 ceph-mon[97244]: from='client.34828 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:37:51.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:50 smithi027 ceph-mon[97244]: pgmap v696: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:51.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:50 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1664222660' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:51.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:51 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3635147399' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:37:52.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:51 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3635147399' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:37:52.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:52 smithi115 ceph-mon[65453]: pgmap v697: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:53.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:52 smithi027 ceph-mon[97244]: pgmap v697: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:54.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:37:54.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:37:54.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:37:54.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:54.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:54.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:54.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:54.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:37:54.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:37:54.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:37:54.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:54.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:54.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:54.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:37:55.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:55 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:37:55.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:55 smithi027 ceph-mon[97244]: pgmap v698: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:55.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1317.390271 secs 2021-11-03T17:37:55.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:55 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:37:55.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:55 smithi115 ceph-mon[65453]: pgmap v698: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:55.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1317.390271 secs 2021-11-03T17:37:56.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:37:56.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:37:56.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:37:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:37:57.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:57 smithi027 ceph-mon[97244]: pgmap v699: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:57.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:57 smithi115 ceph-mon[65453]: pgmap v699: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:58.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:58 smithi115 ceph-mon[65453]: pgmap v700: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:58.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:58 smithi027 ceph-mon[97244]: pgmap v700: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:37:59.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:37:59 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1322.390404 secs 2021-11-03T17:38:00.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:37:59 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1322.390404 secs 2021-11-03T17:38:00.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:00 smithi115 ceph-mon[65453]: pgmap v701: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:01.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:00 smithi027 ceph-mon[97244]: pgmap v701: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:02.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:02 smithi115 ceph-mon[65453]: pgmap v702: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:02.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:02 smithi027 ceph-mon[97244]: pgmap v702: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:04.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:38:04.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:38:05.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:05 smithi115 ceph-mon[65453]: poll took 60184.584 ms: 1 events 2021-11-03T17:38:05.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:05 smithi115 ceph-mon[65453]: pgmap v703: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:05.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1327.390459 secs 2021-11-03T17:38:05.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:38:05.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:38:05.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:05.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:05.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:05.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:05.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:05 smithi027 ceph-mon[97244]: poll took 60184.584 ms: 1 events 2021-11-03T17:38:05.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:05 smithi027 ceph-mon[97244]: pgmap v703: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:05.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1327.390459 secs 2021-11-03T17:38:05.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:38:05.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:38:05.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:05.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:05.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:05.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:06.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:06 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:38:06.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:06 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:38:07.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:07 smithi115 ceph-mon[65453]: pgmap v704: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:07.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:07 smithi027 ceph-mon[97244]: pgmap v704: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:08.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:08 smithi115 ceph-mon[65453]: pgmap v705: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:08.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:08 smithi027 ceph-mon[97244]: pgmap v705: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:09.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:09 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1332.390582 secs 2021-11-03T17:38:10.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:09 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1332.390582 secs 2021-11-03T17:38:10.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:10 smithi115 ceph-mon[65453]: pgmap v706: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:11.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:10 smithi027 ceph-mon[97244]: pgmap v706: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:12.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:12 smithi115 ceph-mon[65453]: pgmap v707: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:12.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:12 smithi027 ceph-mon[97244]: pgmap v707: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:14.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:14 smithi115 ceph-mon[65453]: pgmap v708: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:14.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:14 smithi027 ceph-mon[97244]: pgmap v708: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:15.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1337.390699 secs 2021-11-03T17:38:15.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:38:15.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:15 smithi115 ceph-mon[65453]: poll took 10026.157 ms: 1 events 2021-11-03T17:38:15.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:38:15.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:38:15.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:38:15.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:15.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:15.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:15.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:15.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:15 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:38:15.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1337.390699 secs 2021-11-03T17:38:15.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:38:15.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:15 smithi027 ceph-mon[97244]: poll took 10026.157 ms: 1 events 2021-11-03T17:38:15.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:38:15.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:38:15.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:38:15.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:15.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:15.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:15.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:15.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:15 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:38:16.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:16 smithi115 ceph-mon[65453]: pgmap v709: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:16.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:16 smithi027 ceph-mon[97244]: pgmap v709: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:18.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:18 smithi115 ceph-mon[65453]: pgmap v710: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:18.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:18 smithi027 ceph-mon[97244]: pgmap v710: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:19.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:19 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1342.390813 secs 2021-11-03T17:38:20.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:19 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1342.390813 secs 2021-11-03T17:38:20.874 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:20.874+0000 7fb545039700 1 -- 172.21.15.27:0/2758944347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5400f6860 msgr2=0x7fb5400f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:38:20.875 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:20.874+0000 7fb545039700 1 --2- 172.21.15.27:0/2758944347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5400f6860 0x7fb5400f6cc0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7fb528004660 tx=0x7fb528009b30).stop 2021-11-03T17:38:20.875 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:20.874+0000 7fb545039700 1 -- 172.21.15.27:0/2758944347 shutdown_connections 2021-11-03T17:38:20.876 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:20.874+0000 7fb545039700 1 --2- 172.21.15.27:0/2758944347 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5400f6860 0x7fb5400f6cc0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:20.876 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:20.874+0000 7fb545039700 1 --2- 172.21.15.27:0/2758944347 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5400f55c0 0x7fb5400f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:20.877 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:20.874+0000 7fb545039700 1 -- 172.21.15.27:0/2758944347 >> 172.21.15.27:0/2758944347 conn(0x7fb5400f0d40 msgr2=0x7fb5400f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:38:20.877 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:20.874+0000 7fb545039700 1 -- 172.21.15.27:0/2758944347 shutdown_connections 2021-11-03T17:38:20.877 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:20.874+0000 7fb545039700 1 -- 172.21.15.27:0/2758944347 wait complete. 2021-11-03T17:38:20.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:20.874+0000 7fb545039700 1 Processor -- start 2021-11-03T17:38:20.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:20.875+0000 7fb545039700 1 -- start start 2021-11-03T17:38:20.879 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:20.875+0000 7fb545039700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5400f55c0 0x7fb540106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:38:20.879 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:20.875+0000 7fb545039700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5400f6860 0x7fb540107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:38:20.880 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:20.875+0000 7fb545039700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb540107a10 con 0x7fb5400f6860 2021-11-03T17:38:20.880 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:20.875+0000 7fb545039700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb540107b50 con 0x7fb5400f55c0 2021-11-03T17:38:20.880 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:20.875+0000 7fb53e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5400f6860 0x7fb540107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:38:20.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:20.875+0000 7fb53e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5400f6860 0x7fb540107460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58550/0 (socket says 172.21.15.27:58550) 2021-11-03T17:38:20.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:20.875+0000 7fb53e59c700 1 -- 172.21.15.27:0/3649581074 learned_addr learned my addr 172.21.15.27:0/3649581074 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:38:20.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:20.875+0000 7fb53ed9d700 1 --2- 172.21.15.27:0/3649581074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5400f55c0 0x7fb540106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:38:20.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:20.875+0000 7fb53e59c700 1 -- 172.21.15.27:0/3649581074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5400f55c0 msgr2=0x7fb540106f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:38:20.882 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:20.875+0000 7fb53e59c700 1 --2- 172.21.15.27:0/3649581074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5400f55c0 0x7fb540106f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:20.882 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:20.875+0000 7fb53e59c700 1 -- 172.21.15.27:0/3649581074 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb528005040 con 0x7fb5400f6860 2021-11-03T17:38:20.882 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:20.875+0000 7fb53e59c700 1 --2- 172.21.15.27:0/3649581074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5400f6860 0x7fb540107460 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7fb528009990 tx=0x7fb528009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:38:20.882 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:20.876+0000 7fb5377fe700 1 -- 172.21.15.27:0/3649581074 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb528018070 con 0x7fb5400f6860 2021-11-03T17:38:20.883 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:20.876+0000 7fb5377fe700 1 -- 172.21.15.27:0/3649581074 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fb528007ba0 con 0x7fb5400f6860 2021-11-03T17:38:20.883 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:20.876+0000 7fb545039700 1 -- 172.21.15.27:0/3649581074 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb540108490 con 0x7fb5400f6860 2021-11-03T17:38:20.883 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:20.876+0000 7fb5377fe700 1 -- 172.21.15.27:0/3649581074 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb52801c3c0 con 0x7fb5400f6860 2021-11-03T17:38:20.883 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:20.876+0000 7fb545039700 1 -- 172.21.15.27:0/3649581074 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5401089f0 con 0x7fb5400f6860 2021-11-03T17:38:20.884 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:20.877+0000 7fb545039700 1 -- 172.21.15.27:0/3649581074 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb540045ad0 con 0x7fb5400f6860 2021-11-03T17:38:20.884 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:20.877+0000 7fb5377fe700 1 -- 172.21.15.27:0/3649581074 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fb52801d420 con 0x7fb5400f6860 2021-11-03T17:38:20.885 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:20.877+0000 7fb5377fe700 1 --2- 172.21.15.27:0/3649581074 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb52c0640b0 0x7fb52c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:38:20.885 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:20.878+0000 7fb5377fe700 1 -- 172.21.15.27:0/3649581074 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fb528082470 con 0x7fb5400f6860 2021-11-03T17:38:20.885 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:20.878+0000 7fb53ed9d700 1 --2- 172.21.15.27:0/3649581074 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb52c0640b0 0x7fb52c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:38:20.885 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:20.878+0000 7fb53ed9d700 1 --2- 172.21.15.27:0/3649581074 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb52c0640b0 0x7fb52c066570 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7fb5400f66c0 tx=0x7fb530009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:38:20.886 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:20.882+0000 7fb5377fe700 1 -- 172.21.15.27:0/3649581074 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fb528054830 con 0x7fb5400f6860 2021-11-03T17:38:20.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:20 smithi115 ceph-mon[65453]: pgmap v711: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:21.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:20 smithi027 ceph-mon[97244]: pgmap v711: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:21.056 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.060+0000 7fb545039700 1 -- 172.21.15.27:0/3649581074 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb5400faca0 con 0x7fb52c0640b0 2021-11-03T17:38:21.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.062+0000 7fb5377fe700 1 -- 172.21.15.27:0/3649581074 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fb5400faca0 con 0x7fb52c0640b0 2021-11-03T17:38:21.059 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.063+0000 7fb545039700 1 -- 172.21.15.27:0/3649581074 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb52c0640b0 msgr2=0x7fb52c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:38:21.059 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.063+0000 7fb545039700 1 --2- 172.21.15.27:0/3649581074 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb52c0640b0 0x7fb52c066570 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 rx=0x7fb5400f66c0 tx=0x7fb530009040).stop 2021-11-03T17:38:21.060 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.063+0000 7fb545039700 1 -- 172.21.15.27:0/3649581074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5400f6860 msgr2=0x7fb540107460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:38:21.060 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.064+0000 7fb545039700 1 --2- 172.21.15.27:0/3649581074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5400f6860 0x7fb540107460 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7fb528009990 tx=0x7fb528009b30).stop 2021-11-03T17:38:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.064+0000 7fb545039700 1 -- 172.21.15.27:0/3649581074 shutdown_connections 2021-11-03T17:38:21.061 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.064+0000 7fb545039700 1 --2- 172.21.15.27:0/3649581074 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb52c0640b0 0x7fb52c066570 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:21.062 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.064+0000 7fb545039700 1 --2- 172.21.15.27:0/3649581074 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb5400f55c0 0x7fb540106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:21.062 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.064+0000 7fb545039700 1 --2- 172.21.15.27:0/3649581074 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb5400f6860 0x7fb540107460 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:21.063 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.064+0000 7fb545039700 1 -- 172.21.15.27:0/3649581074 >> 172.21.15.27:0/3649581074 conn(0x7fb5400f0d40 msgr2=0x7fb54005fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:38:21.063 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.064+0000 7fb545039700 1 -- 172.21.15.27:0/3649581074 shutdown_connections 2021-11-03T17:38:21.064 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.064+0000 7fb545039700 1 -- 172.21.15.27:0/3649581074 wait complete. 2021-11-03T17:38:21.068 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:38:21.164 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.164+0000 7fbcb59e3700 1 -- 172.21.15.27:0/1259208415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcb00f6ff0 msgr2=0x7fbcb00f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:38:21.164 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.164+0000 7fbcb59e3700 1 --2- 172.21.15.27:0/1259208415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcb00f6ff0 0x7fbcb00f7a20 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7fbca4004660 tx=0x7fbca4009b30).stop 2021-11-03T17:38:21.165 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.164+0000 7fbcb59e3700 1 -- 172.21.15.27:0/1259208415 shutdown_connections 2021-11-03T17:38:21.165 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.164+0000 7fbcb59e3700 1 --2- 172.21.15.27:0/1259208415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcb00f6ff0 0x7fbcb00f7a20 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:21.166 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.164+0000 7fbcb59e3700 1 --2- 172.21.15.27:0/1259208415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcb00f66e0 0x7fbcb00f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:38:21.166 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.164+0000 7fbcb59e3700 1 -- 172.21.15.27:0/1259208415 >> 172.21.15.27:0/1259208415 conn(0x7fbcb00f1f00 msgr2=0x7fbcb00f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:38:21.166 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.164+0000 7fbcb59e3700 1 -- 172.21.15.27:0/1259208415 shutdown_connections 2021-11-03T17:38:21.167 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.164+0000 7fbcb59e3700 1 -- 172.21.15.27:0/1259208415 wait complete. 2021-11-03T17:38:21.167 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.165+0000 7fbcb59e3700 1 Processor -- start 2021-11-03T17:38:21.168 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.165+0000 7fbcb59e3700 1 -- start start 2021-11-03T17:38:21.168 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.165+0000 7fbcb59e3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcb00f66e0 0x7fbcb006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:38:21.169 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.165+0000 7fbcb59e3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcb00f6ff0 0x7fbcb0069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:38:21.169 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.165+0000 7fbcb59e3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb006a410 con 0x7fbcb00f66e0 2021-11-03T17:38:21.170 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.165+0000 7fbcb59e3700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb006a550 con 0x7fbcb00f6ff0 2021-11-03T17:38:21.170 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.165+0000 7fbcaeffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcb00f66e0 0x7fbcb006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:38:21.170 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.165+0000 7fbcaeffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcb00f66e0 0x7fbcb006b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58558/0 (socket says 172.21.15.27:58558) 2021-11-03T17:38:21.171 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.165+0000 7fbcaeffd700 1 -- 172.21.15.27:0/2896408912 learned_addr learned my addr 172.21.15.27:0/2896408912 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:38:21.171 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.165+0000 7fbcae7fc700 1 --2- 172.21.15.27:0/2896408912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcb00f6ff0 0x7fbcb0069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:38:21.172 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.165+0000 7fbcaeffd700 1 -- 172.21.15.27:0/2896408912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcb00f6ff0 msgr2=0x7fbcb0069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:38:21.172 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.165+0000 7fbcaeffd700 1 --2- 172.21.15.27:0/2896408912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcb00f6ff0 0x7fbcb0069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:21.172 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.165+0000 7fbcaeffd700 1 -- 172.21.15.27:0/2896408912 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca4005040 con 0x7fbcb00f66e0 2021-11-03T17:38:21.173 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.165+0000 7fbcaeffd700 1 --2- 172.21.15.27:0/2896408912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcb00f66e0 0x7fbcb006b9a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7fbca000b410 tx=0x7fbca0006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:38:21.173 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.166+0000 7fbc977fe700 1 -- 172.21.15.27:0/2896408912 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbca00125f0 con 0x7fbcb00f66e0 2021-11-03T17:38:21.174 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.166+0000 7fbcb59e3700 1 -- 172.21.15.27:0/2896408912 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcb006a830 con 0x7fbcb00f66e0 2021-11-03T17:38:21.174 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.166+0000 7fbc977fe700 1 -- 172.21.15.27:0/2896408912 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fbca0012c00 con 0x7fbcb00f66e0 2021-11-03T17:38:21.174 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.166+0000 7fbc977fe700 1 -- 172.21.15.27:0/2896408912 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbca0002ae0 con 0x7fbcb00f66e0 2021-11-03T17:38:21.175 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.166+0000 7fbcb59e3700 1 -- 172.21.15.27:0/2896408912 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcb010efd0 con 0x7fbcb00f66e0 2021-11-03T17:38:21.175 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.167+0000 7fbcb59e3700 1 -- 172.21.15.27:0/2896408912 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcb0045ad0 con 0x7fbcb00f66e0 2021-11-03T17:38:21.176 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.167+0000 7fbc977fe700 1 -- 172.21.15.27:0/2896408912 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fbca000f030 con 0x7fbcb00f66e0 2021-11-03T17:38:21.176 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.167+0000 7fbc977fe700 1 --2- 172.21.15.27:0/2896408912 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbc980640b0 0x7fbc98066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:38:21.177 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.168+0000 7fbc977fe700 1 -- 172.21.15.27:0/2896408912 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fbca0080cc0 con 0x7fbcb00f66e0 2021-11-03T17:38:21.177 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.168+0000 7fbcae7fc700 1 --2- 172.21.15.27:0/2896408912 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbc980640b0 0x7fbc98066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:38:21.177 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.168+0000 7fbcae7fc700 1 --2- 172.21.15.27:0/2896408912 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbc980640b0 0x7fbc98066570 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7fbca4004e30 tx=0x7fbca4009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:38:21.178 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.172+0000 7fbc977fe700 1 -- 172.21.15.27:0/2896408912 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fbca0052fc0 con 0x7fbcb00f66e0 2021-11-03T17:38:21.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.346+0000 7fbcb59e3700 1 -- 172.21.15.27:0/2896408912 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbcb0057f10 con 0x7fbc980640b0 2021-11-03T17:38:21.355 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.360+0000 7fbc977fe700 1 -- 172.21.15.27:0/2896408912 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fbcb0057f10 con 0x7fbc980640b0 2021-11-03T17:38:21.356 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:38:21.356 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (29m) 9m ago 31m 19.7M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:38:21.357 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (21m) 9m ago 31m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:38:21.357 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (21m) 8m ago 30m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:38:21.357 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (29m) 9m ago 30m 40.2M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:38:21.357 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (23m) 9m ago 26m 18.2M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:38:21.357 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (26m) 9m ago 26m 3841M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:38:21.358 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (22m) 8m ago 26m 17.6M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:38:21.358 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (23m) 8m ago 26m 19.3M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:38:21.358 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (23m) 9m ago 32m 497M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:38:21.359 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (22m) 8m ago 30m 415M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:38:21.359 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (22m) 9m ago 32m 119M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:38:21.359 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (22m) 8m ago 30m 71.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:38:21.360 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (30m) 9m ago 30m 18.0M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:38:21.360 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (30m) 8m ago 30m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:38:21.360 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (21m) 9m ago 29m 290M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:38:21.360 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (20m) 9m ago 29m 317M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:38:21.361 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (20m) 9m ago 28m 229M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:38:21.361 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (19m) 8m ago 28m 329M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:38:21.361 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (19m) 8m ago 28m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:38:21.361 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (18m) 8m ago 27m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:38:21.362 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (29m) 9m ago 30m 48.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:38:21.363 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.362+0000 7fbcb59e3700 1 -- 172.21.15.27:0/2896408912 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbc980640b0 msgr2=0x7fbc98066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:38:21.363 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.362+0000 7fbcb59e3700 1 --2- 172.21.15.27:0/2896408912 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbc980640b0 0x7fbc98066570 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7fbca4004e30 tx=0x7fbca4009b30).stop 2021-11-03T17:38:21.363 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.362+0000 7fbcb59e3700 1 -- 172.21.15.27:0/2896408912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcb00f66e0 msgr2=0x7fbcb006b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:38:21.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.362+0000 7fbcb59e3700 1 --2- 172.21.15.27:0/2896408912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcb00f66e0 0x7fbcb006b9a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7fbca000b410 tx=0x7fbca0006ef0).stop 2021-11-03T17:38:21.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.362+0000 7fbcb59e3700 1 -- 172.21.15.27:0/2896408912 shutdown_connections 2021-11-03T17:38:21.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.362+0000 7fbcb59e3700 1 --2- 172.21.15.27:0/2896408912 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbc980640b0 0x7fbc98066570 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:21.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.362+0000 7fbcb59e3700 1 --2- 172.21.15.27:0/2896408912 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcb00f66e0 0x7fbcb006b9a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:21.365 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.362+0000 7fbcb59e3700 1 --2- 172.21.15.27:0/2896408912 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcb00f6ff0 0x7fbcb0069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:21.365 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.362+0000 7fbcb59e3700 1 -- 172.21.15.27:0/2896408912 >> 172.21.15.27:0/2896408912 conn(0x7fbcb00f1f00 msgr2=0x7fbcb00fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:38:21.365 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.362+0000 7fbcb59e3700 1 -- 172.21.15.27:0/2896408912 shutdown_connections 2021-11-03T17:38:21.366 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.362+0000 7fbcb59e3700 1 -- 172.21.15.27:0/2896408912 wait complete. 2021-11-03T17:38:21.459 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.462+0000 7f5ec4a7c700 1 -- 172.21.15.27:0/1267821816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ec00f5ef0 msgr2=0x7f5ec00f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:38:21.459 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.462+0000 7f5ec4a7c700 1 --2- 172.21.15.27:0/1267821816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ec00f5ef0 0x7f5ec00f6920 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f5ea8004660 tx=0x7f5ea8009b30).stop 2021-11-03T17:38:21.459 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.462+0000 7f5ec4a7c700 1 -- 172.21.15.27:0/1267821816 shutdown_connections 2021-11-03T17:38:21.460 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.462+0000 7f5ec4a7c700 1 --2- 172.21.15.27:0/1267821816 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ec00f5ef0 0x7f5ec00f6920 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:21.460 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.462+0000 7f5ec4a7c700 1 --2- 172.21.15.27:0/1267821816 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5ec00f55e0 0x7f5ec00f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:21.460 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.462+0000 7f5ec4a7c700 1 -- 172.21.15.27:0/1267821816 >> 172.21.15.27:0/1267821816 conn(0x7f5ec00f0de0 msgr2=0x7f5ec00f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:38:21.460 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.462+0000 7f5ec4a7c700 1 -- 172.21.15.27:0/1267821816 shutdown_connections 2021-11-03T17:38:21.460 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.462+0000 7f5ec4a7c700 1 -- 172.21.15.27:0/1267821816 wait complete. 2021-11-03T17:38:21.461 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.462+0000 7f5ec4a7c700 1 Processor -- start 2021-11-03T17:38:21.461 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.463+0000 7f5ec4a7c700 1 -- start start 2021-11-03T17:38:21.461 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.463+0000 7f5ec4a7c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ec00f55e0 0x7f5ec00fa000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:38:21.461 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.463+0000 7f5ec4a7c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5ec00f5ef0 0x7f5ec00f8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:38:21.462 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.463+0000 7f5ec4a7c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec00f89e0 con 0x7f5ec00f55e0 2021-11-03T17:38:21.462 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.463+0000 7f5ec4a7c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec00f8b20 con 0x7f5ec00f5ef0 2021-11-03T17:38:21.462 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.463+0000 7f5ebe59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ec00f55e0 0x7f5ec00fa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:38:21.462 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.463+0000 7f5ebe59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ec00f55e0 0x7f5ec00fa000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58568/0 (socket says 172.21.15.27:58568) 2021-11-03T17:38:21.462 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.463+0000 7f5ebe59c700 1 -- 172.21.15.27:0/3216406888 learned_addr learned my addr 172.21.15.27:0/3216406888 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:38:21.463 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.463+0000 7f5ebdd9b700 1 --2- 172.21.15.27:0/3216406888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5ec00f5ef0 0x7f5ec00f8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:38:21.463 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.463+0000 7f5ebe59c700 1 -- 172.21.15.27:0/3216406888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5ec00f5ef0 msgr2=0x7f5ec00f8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:38:21.463 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.463+0000 7f5ebe59c700 1 --2- 172.21.15.27:0/3216406888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5ec00f5ef0 0x7f5ec00f8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:21.463 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.463+0000 7f5ebe59c700 1 -- 172.21.15.27:0/3216406888 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ea8005040 con 0x7f5ec00f55e0 2021-11-03T17:38:21.464 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.463+0000 7f5ebdd9b700 1 --2- 172.21.15.27:0/3216406888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5ec00f5ef0 0x7f5ec00f8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:38:21.464 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.463+0000 7f5ebe59c700 1 --2- 172.21.15.27:0/3216406888 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ec00f55e0 0x7f5ec00fa000 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f5eb8007fd0 tx=0x7f5eb8004ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:38:21.464 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.463+0000 7f5eb6ffd700 1 -- 172.21.15.27:0/3216406888 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5eb800b5a0 con 0x7f5ec00f55e0 2021-11-03T17:38:21.464 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.463+0000 7f5eb6ffd700 1 -- 172.21.15.27:0/3216406888 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f5eb800f040 con 0x7f5ec00f55e0 2021-11-03T17:38:21.465 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.463+0000 7f5ec4a7c700 1 -- 172.21.15.27:0/3216406888 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ec00f8e00 con 0x7f5ec00f55e0 2021-11-03T17:38:21.465 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.463+0000 7f5eb6ffd700 1 -- 172.21.15.27:0/3216406888 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5eb8013b60 con 0x7f5ec00f55e0 2021-11-03T17:38:21.465 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.464+0000 7f5ec4a7c700 1 -- 172.21.15.27:0/3216406888 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec010efa0 con 0x7f5ec00f55e0 2021-11-03T17:38:21.466 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.464+0000 7f5ec4a7c700 1 -- 172.21.15.27:0/3216406888 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ec0045ad0 con 0x7f5ec00f55e0 2021-11-03T17:38:21.467 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.465+0000 7f5eb6ffd700 1 -- 172.21.15.27:0/3216406888 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f5eb801a030 con 0x7f5ec00f55e0 2021-11-03T17:38:21.467 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.465+0000 7f5eb6ffd700 1 --2- 172.21.15.27:0/3216406888 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5eac0640b0 0x7f5eac066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:38:21.467 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.465+0000 7f5eb6ffd700 1 -- 172.21.15.27:0/3216406888 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f5eb80807a0 con 0x7f5ec00f55e0 2021-11-03T17:38:21.467 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.465+0000 7f5ebdd9b700 1 --2- 172.21.15.27:0/3216406888 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5eac0640b0 0x7f5eac066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:38:21.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.466+0000 7f5ebdd9b700 1 --2- 172.21.15.27:0/3216406888 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5eac0640b0 0x7f5eac066570 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f5ea8004630 tx=0x7f5ea8009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:38:21.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.470+0000 7f5eb6ffd700 1 -- 172.21.15.27:0/3216406888 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f5eb8052a60 con 0x7f5ec00f55e0 2021-11-03T17:38:21.690 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.696+0000 7f5ec4a7c700 1 -- 172.21.15.27:0/3216406888 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5ec010f790 con 0x7f5ec00f55e0 2021-11-03T17:38:21.691 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.697+0000 7f5eb6ffd700 1 -- 172.21.15.27:0/3216406888 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f5eb8018070 con 0x7f5ec00f55e0 2021-11-03T17:38:21.691 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:38:21.692 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:38:21.692 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:38:21.692 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:38:21.692 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:38:21.693 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:38:21.693 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:38:21.693 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:38:21.693 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:38:21.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:38:21.694 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:38:21.694 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:38:21.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:38:21.695 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:38:21.695 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:38:21.695 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:38:21.695 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:38:21.695 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:38:21.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.698+0000 7f5ec4a7c700 1 -- 172.21.15.27:0/3216406888 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5eac0640b0 msgr2=0x7f5eac066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:38:21.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.698+0000 7f5ec4a7c700 1 --2- 172.21.15.27:0/3216406888 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5eac0640b0 0x7f5eac066570 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7f5ea8004630 tx=0x7f5ea8009b30).stop 2021-11-03T17:38:21.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.698+0000 7f5ec4a7c700 1 -- 172.21.15.27:0/3216406888 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ec00f55e0 msgr2=0x7f5ec00fa000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:38:21.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.698+0000 7f5ec4a7c700 1 --2- 172.21.15.27:0/3216406888 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ec00f55e0 0x7f5ec00fa000 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f5eb8007fd0 tx=0x7f5eb8004ee0).stop 2021-11-03T17:38:21.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.699+0000 7f5ec4a7c700 1 -- 172.21.15.27:0/3216406888 shutdown_connections 2021-11-03T17:38:21.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.699+0000 7f5ec4a7c700 1 --2- 172.21.15.27:0/3216406888 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5eac0640b0 0x7f5eac066570 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:21.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.699+0000 7f5ec4a7c700 1 --2- 172.21.15.27:0/3216406888 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5ec00f55e0 0x7f5ec00fa000 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:21.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.699+0000 7f5ec4a7c700 1 --2- 172.21.15.27:0/3216406888 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5ec00f5ef0 0x7f5ec00f8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:21.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.699+0000 7f5ec4a7c700 1 -- 172.21.15.27:0/3216406888 >> 172.21.15.27:0/3216406888 conn(0x7f5ec00f0de0 msgr2=0x7f5ec005f740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:38:21.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.699+0000 7f5ec4a7c700 1 -- 172.21.15.27:0/3216406888 shutdown_connections 2021-11-03T17:38:21.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.699+0000 7f5ec4a7c700 1 -- 172.21.15.27:0/3216406888 wait complete. 2021-11-03T17:38:21.801 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.805+0000 7f575446a700 1 -- 172.21.15.27:0/443812513 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f574c0f66e0 msgr2=0x7f574c0f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:38:21.801 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.805+0000 7f575446a700 1 --2- 172.21.15.27:0/443812513 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f574c0f66e0 0x7f574c0f6b00 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7f57440099e0 tx=0x7f5744004d10).stop 2021-11-03T17:38:21.802 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.805+0000 7f575446a700 1 -- 172.21.15.27:0/443812513 shutdown_connections 2021-11-03T17:38:21.802 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.805+0000 7f575446a700 1 --2- 172.21.15.27:0/443812513 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f574c0f7980 0x7f574c0f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:38:21.802 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.805+0000 7f575446a700 1 --2- 172.21.15.27:0/443812513 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f574c0f66e0 0x7f574c0f6b00 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:21.803 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.805+0000 7f575446a700 1 -- 172.21.15.27:0/443812513 >> 172.21.15.27:0/443812513 conn(0x7f574c0f1e40 msgr2=0x7f574c0f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:38:21.803 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.805+0000 7f575446a700 1 -- 172.21.15.27:0/443812513 shutdown_connections 2021-11-03T17:38:21.803 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.805+0000 7f575446a700 1 -- 172.21.15.27:0/443812513 wait complete. 2021-11-03T17:38:21.804 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.805+0000 7f575446a700 1 Processor -- start 2021-11-03T17:38:21.804 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.806+0000 7f575446a700 1 -- start start 2021-11-03T17:38:21.804 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.806+0000 7f575446a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f574c0f66e0 0x7f574c06b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:38:21.804 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.806+0000 7f575446a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f574c0f7980 0x7f574c069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:38:21.805 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.806+0000 7f575446a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c06a410 con 0x7f574c0f7980 2021-11-03T17:38:21.805 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.806+0000 7f575446a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c06a550 con 0x7f574c0f66e0 2021-11-03T17:38:21.805 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.806+0000 7f5751a05700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f574c0f7980 0x7f574c069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:38:21.805 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.806+0000 7f5751a05700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f574c0f7980 0x7f574c069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58578/0 (socket says 172.21.15.27:58578) 2021-11-03T17:38:21.806 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.806+0000 7f5751a05700 1 -- 172.21.15.27:0/218024926 learned_addr learned my addr 172.21.15.27:0/218024926 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:38:21.806 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.806+0000 7f5752206700 1 --2- 172.21.15.27:0/218024926 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f574c0f66e0 0x7f574c06b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:38:21.806 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.806+0000 7f5751a05700 1 -- 172.21.15.27:0/218024926 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f574c0f66e0 msgr2=0x7f574c06b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:38:21.806 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.806+0000 7f5751a05700 1 --2- 172.21.15.27:0/218024926 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f574c0f66e0 0x7f574c06b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:21.807 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.806+0000 7f5751a05700 1 -- 172.21.15.27:0/218024926 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5744009640 con 0x7f574c0f7980 2021-11-03T17:38:21.807 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.806+0000 7f5751a05700 1 --2- 172.21.15.27:0/218024926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f574c0f7980 0x7f574c069e20 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f573c00b410 tx=0x7f573c006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:38:21.807 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.807+0000 7f5742ffd700 1 -- 172.21.15.27:0/218024926 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f573c0125f0 con 0x7f574c0f7980 2021-11-03T17:38:21.807 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.807+0000 7f5742ffd700 1 -- 172.21.15.27:0/218024926 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f573c00be30 con 0x7f574c0f7980 2021-11-03T17:38:21.807 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.807+0000 7f575446a700 1 -- 172.21.15.27:0/218024926 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f574c06a830 con 0x7f574c0f7980 2021-11-03T17:38:21.808 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.807+0000 7f5742ffd700 1 -- 172.21.15.27:0/218024926 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f573c002a70 con 0x7f574c0f7980 2021-11-03T17:38:21.808 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.807+0000 7f575446a700 1 -- 172.21.15.27:0/218024926 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f574c10f150 con 0x7f574c0f7980 2021-11-03T17:38:21.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.808+0000 7f575446a700 1 -- 172.21.15.27:0/218024926 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f574c045ad0 con 0x7f574c0f7980 2021-11-03T17:38:21.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.808+0000 7f5742ffd700 1 -- 172.21.15.27:0/218024926 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f573c00f030 con 0x7f574c0f7980 2021-11-03T17:38:21.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.808+0000 7f5742ffd700 1 --2- 172.21.15.27:0/218024926 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f57380640b0 0x7f5738066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:38:21.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.808+0000 7f5742ffd700 1 -- 172.21.15.27:0/218024926 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f573c080c70 con 0x7f574c0f7980 2021-11-03T17:38:21.811 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.808+0000 7f5752206700 1 --2- 172.21.15.27:0/218024926 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f57380640b0 0x7f5738066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:38:21.811 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.809+0000 7f5752206700 1 --2- 172.21.15.27:0/218024926 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f57380640b0 0x7f5738066570 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f57440099b0 tx=0x7f5744004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:38:21.811 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:21.812+0000 7f5742ffd700 1 -- 172.21.15.27:0/218024926 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f573c052f70 con 0x7f574c0f7980 2021-11-03T17:38:21.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:21 smithi115 ceph-mon[65453]: from='client.34840 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:38:21.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:21 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3216406888' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:22.012 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:22.018+0000 7f575446a700 1 -- 172.21.15.27:0/218024926 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f574c0fefc0 con 0x7f574c0f7980 2021-11-03T17:38:22.013 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:22.019+0000 7f5742ffd700 1 -- 172.21.15.27:0/218024926 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f573c052b60 con 0x7f574c0f7980 2021-11-03T17:38:22.013 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:38:22.013 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:38:22.014 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:38:22.014 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:38:22.015 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:38:22.015 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:38:22.015 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:38:22.015 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:38:22.016 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:38:22.016 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:38:22.016 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:38:22.016 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:38:22.017 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:38:22.017 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:38:22.017 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:38:22.017 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:38:22.017 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:38:22.018 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:38:22.018 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:38:22.018 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:38:22.018 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:38:22.019 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:38:22.019 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:38:22.019 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:38:22.019 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:38:22.020 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:38:22.020 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:38:22.020 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:38:22.020 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:38:22.021 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:38:22.021 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:38:22.021 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:38:22.021 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:38:22.021 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:38:22.022 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:38:22.022 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:38:22.022 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:38:22.022 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:38:22.023 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:38:22.023 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:22.021+0000 7f575446a700 1 -- 172.21.15.27:0/218024926 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f57380640b0 msgr2=0x7f5738066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:38:22.024 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:22.021+0000 7f575446a700 1 --2- 172.21.15.27:0/218024926 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f57380640b0 0x7f5738066570 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f57440099b0 tx=0x7f5744004d10).stop 2021-11-03T17:38:22.024 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:22.021+0000 7f575446a700 1 -- 172.21.15.27:0/218024926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f574c0f7980 msgr2=0x7f574c069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:38:22.024 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:22.021+0000 7f575446a700 1 --2- 172.21.15.27:0/218024926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f574c0f7980 0x7f574c069e20 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f573c00b410 tx=0x7f573c006ef0).stop 2021-11-03T17:38:22.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:22.021+0000 7f575446a700 1 -- 172.21.15.27:0/218024926 shutdown_connections 2021-11-03T17:38:22.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:22.021+0000 7f575446a700 1 --2- 172.21.15.27:0/218024926 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f57380640b0 0x7f5738066570 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:22.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:22.021+0000 7f575446a700 1 --2- 172.21.15.27:0/218024926 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f574c0f66e0 0x7f574c06b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:22.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:22.021+0000 7f575446a700 1 --2- 172.21.15.27:0/218024926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f574c0f7980 0x7f574c069e20 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:22.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:22.021+0000 7f575446a700 1 -- 172.21.15.27:0/218024926 >> 172.21.15.27:0/218024926 conn(0x7f574c0f1e40 msgr2=0x7f574c0fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:38:22.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:22.021+0000 7f575446a700 1 -- 172.21.15.27:0/218024926 shutdown_connections 2021-11-03T17:38:22.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:22.021+0000 7f575446a700 1 -- 172.21.15.27:0/218024926 wait complete. 2021-11-03T17:38:22.026 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:38:22.027 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:21 smithi027 ceph-mon[97244]: from='client.34840 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:38:22.027 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:21 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3216406888' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:22.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:22 smithi115 ceph-mon[65453]: from='client.34844 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:38:22.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:22 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/218024926' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:38:22.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:22 smithi115 ceph-mon[65453]: pgmap v712: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:23.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:22 smithi027 ceph-mon[97244]: from='client.34844 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:38:23.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:22 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/218024926' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:38:23.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:22 smithi027 ceph-mon[97244]: pgmap v712: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:24.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:24 smithi115 ceph-mon[65453]: pgmap v713: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:24.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:24 smithi027 ceph-mon[97244]: pgmap v713: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:25.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1347.390952 secs 2021-11-03T17:38:25.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:38:25.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:38:25.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:38:25.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:25.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:25.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:25.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:25.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:25 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:38:25.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1347.390952 secs 2021-11-03T17:38:25.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:38:25.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:38:25.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:38:25.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:25.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:25.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:25.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:25.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:25 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:38:26.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:26 smithi115 ceph-mon[65453]: pgmap v714: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:26.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:26 smithi027 ceph-mon[97244]: pgmap v714: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:28.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:28 smithi115 ceph-mon[65453]: pgmap v715: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:28.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:28 smithi027 ceph-mon[97244]: pgmap v715: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:29.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:29 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1352.391066 secs 2021-11-03T17:38:30.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:29 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1352.391066 secs 2021-11-03T17:38:30.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:30 smithi115 ceph-mon[65453]: pgmap v716: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:31.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:30 smithi027 ceph-mon[97244]: pgmap v716: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:32.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:32 smithi115 ceph-mon[65453]: pgmap v717: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:32.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:32 smithi027 ceph-mon[97244]: pgmap v717: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:34.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:34 smithi115 ceph-mon[65453]: pgmap v718: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:34.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:34 smithi027 ceph-mon[97244]: pgmap v718: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:35.213 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1357.391193 secs 2021-11-03T17:38:35.214 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:38:35.214 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:35 smithi027 ceph-mon[97244]: poll took 20064.102 ms: 1 events 2021-11-03T17:38:35.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1357.391193 secs 2021-11-03T17:38:35.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:38:35.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:35 smithi115 ceph-mon[65453]: poll took 20064.102 ms: 1 events 2021-11-03T17:38:36.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:36 smithi115 ceph-mon[65453]: pgmap v719: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:36.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:36 smithi027 ceph-mon[97244]: pgmap v719: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:38.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:38 smithi115 ceph-mon[65453]: pgmap v720: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:38.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:38 smithi027 ceph-mon[97244]: pgmap v720: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:39.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:39 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1362.391254 secs 2021-11-03T17:38:40.004 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:39 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1362.391254 secs 2021-11-03T17:38:41.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:41 smithi027 ceph-mon[97244]: poll took 5205.993 ms: 1 events 2021-11-03T17:38:41.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:38:41.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:38:41.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:38:41.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:41.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:41.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:41.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:41.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:41 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:38:41.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:41 smithi027 ceph-mon[97244]: pgmap v721: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:41.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:41 smithi115 ceph-mon[65453]: poll took 5205.993 ms: 1 events 2021-11-03T17:38:41.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:38:41.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:38:41.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:38:41.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:41.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:41.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:41.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:41.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:41 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:38:41.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:41 smithi115 ceph-mon[65453]: pgmap v721: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:42.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:42 smithi115 ceph-mon[65453]: pgmap v722: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:42.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:42 smithi027 ceph-mon[97244]: pgmap v722: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:44.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:44 smithi115 ceph-mon[65453]: pgmap v723: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:44.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:44 smithi027 ceph-mon[97244]: pgmap v723: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:45.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1367.391370 secs 2021-11-03T17:38:45.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1367.391370 secs 2021-11-03T17:38:46.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:46 smithi115 ceph-mon[65453]: pgmap v724: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:46.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:46 smithi027 ceph-mon[97244]: pgmap v724: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:48.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:48 smithi115 ceph-mon[65453]: pgmap v725: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:48.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:48 smithi027 ceph-mon[97244]: pgmap v725: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:49.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:49 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1372.391496 secs 2021-11-03T17:38:50.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:49 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1372.391496 secs 2021-11-03T17:38:50.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:38:50.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:38:50.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:38:50.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:50.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:50.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:50.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:50.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:50 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:38:50.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:50 smithi115 ceph-mon[65453]: pgmap v726: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:51.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:38:51.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:38:51.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:38:51.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:51.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:51.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:51.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:51.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:50 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:38:51.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:50 smithi027 ceph-mon[97244]: pgmap v726: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:52.132 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.137+0000 7f114b578700 1 -- 172.21.15.27:0/2632996205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11440602d0 msgr2=0x7f11440f9420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:38:52.133 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.137+0000 7f114b578700 1 --2- 172.21.15.27:0/2632996205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11440602d0 0x7f11440f9420 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f1140004690 tx=0x7f1140009b30).stop 2021-11-03T17:38:52.133 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.137+0000 7f114b578700 1 -- 172.21.15.27:0/2632996205 shutdown_connections 2021-11-03T17:38:52.133 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.137+0000 7f114b578700 1 --2- 172.21.15.27:0/2632996205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11440f9910 0x7f11440fbd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:52.133 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.137+0000 7f114b578700 1 --2- 172.21.15.27:0/2632996205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11440602d0 0x7f11440f9420 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:52.134 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.137+0000 7f114b578700 1 -- 172.21.15.27:0/2632996205 >> 172.21.15.27:0/2632996205 conn(0x7f11440f0de0 msgr2=0x7f11440f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:38:52.134 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.137+0000 7f114b578700 1 -- 172.21.15.27:0/2632996205 shutdown_connections 2021-11-03T17:38:52.134 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.137+0000 7f114b578700 1 -- 172.21.15.27:0/2632996205 wait complete. 2021-11-03T17:38:52.137 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.138+0000 7f114b578700 1 Processor -- start 2021-11-03T17:38:52.137 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.138+0000 7f114b578700 1 -- start start 2021-11-03T17:38:52.137 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.138+0000 7f114b578700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11440602d0 0x7f1144106f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:38:52.137 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.138+0000 7f114b578700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11440f9910 0x7f1144107420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:38:52.138 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.138+0000 7f114b578700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11441079d0 con 0x7f11440f9910 2021-11-03T17:38:52.138 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.138+0000 7f114b578700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1144107b10 con 0x7f11440602d0 2021-11-03T17:38:52.138 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.138+0000 7f1148b13700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11440f9910 0x7f1144107420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:38:52.138 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.138+0000 7f1148b13700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11440f9910 0x7f1144107420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58598/0 (socket says 172.21.15.27:58598) 2021-11-03T17:38:52.138 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.138+0000 7f1148b13700 1 -- 172.21.15.27:0/1709041050 learned_addr learned my addr 172.21.15.27:0/1709041050 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:38:52.139 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.138+0000 7f1149314700 1 --2- 172.21.15.27:0/1709041050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11440602d0 0x7f1144106f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:38:52.139 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.139+0000 7f1148b13700 1 -- 172.21.15.27:0/1709041050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11440602d0 msgr2=0x7f1144106f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:38:52.139 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.139+0000 7f1148b13700 1 --2- 172.21.15.27:0/1709041050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11440602d0 0x7f1144106f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:52.139 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.139+0000 7f1148b13700 1 -- 172.21.15.27:0/1709041050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1140005040 con 0x7f11440f9910 2021-11-03T17:38:52.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.139+0000 7f1148b13700 1 --2- 172.21.15.27:0/1709041050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11440f9910 0x7f1144107420 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f113400c790 tx=0x7f1134007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:38:52.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.139+0000 7f1139ffb700 1 -- 172.21.15.27:0/1709041050 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1134002590 con 0x7f11440f9910 2021-11-03T17:38:52.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.139+0000 7f1139ffb700 1 -- 172.21.15.27:0/1709041050 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f1134002ba0 con 0x7f11440f9910 2021-11-03T17:38:52.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.139+0000 7f114b578700 1 -- 172.21.15.27:0/1709041050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11441084b0 con 0x7f11440f9910 2021-11-03T17:38:52.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.139+0000 7f1139ffb700 1 -- 172.21.15.27:0/1709041050 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1134013d90 con 0x7f11440f9910 2021-11-03T17:38:52.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.139+0000 7f114b578700 1 -- 172.21.15.27:0/1709041050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1144108a70 con 0x7f11440f9910 2021-11-03T17:38:52.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.140+0000 7f114b578700 1 -- 172.21.15.27:0/1709041050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1144045ad0 con 0x7f11440f9910 2021-11-03T17:38:52.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.141+0000 7f1139ffb700 1 -- 172.21.15.27:0/1709041050 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f113401a060 con 0x7f11440f9910 2021-11-03T17:38:52.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.141+0000 7f1139ffb700 1 --2- 172.21.15.27:0/1709041050 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f11300640b0 0x7f1130066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:38:52.142 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.141+0000 7f1139ffb700 1 -- 172.21.15.27:0/1709041050 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f1134080ae0 con 0x7f11440f9910 2021-11-03T17:38:52.142 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.141+0000 7f1149314700 1 --2- 172.21.15.27:0/1709041050 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f11300640b0 0x7f1130066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:38:52.142 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.142+0000 7f1149314700 1 --2- 172.21.15.27:0/1709041050 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f11300640b0 0x7f1130066570 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f1140000c00 tx=0x7f1140009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:38:52.143 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.145+0000 7f1139ffb700 1 -- 172.21.15.27:0/1709041050 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f1134052cf0 con 0x7f11440f9910 2021-11-03T17:38:52.320 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.325+0000 7f114b578700 1 -- 172.21.15.27:0/1709041050 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f11440ff030 con 0x7f11300640b0 2021-11-03T17:38:52.321 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.326+0000 7f1139ffb700 1 -- 172.21.15.27:0/1709041050 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f11440ff030 con 0x7f11300640b0 2021-11-03T17:38:52.324 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.328+0000 7f114b578700 1 -- 172.21.15.27:0/1709041050 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f11300640b0 msgr2=0x7f1130066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:38:52.324 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.328+0000 7f114b578700 1 --2- 172.21.15.27:0/1709041050 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f11300640b0 0x7f1130066570 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f1140000c00 tx=0x7f1140009b30).stop 2021-11-03T17:38:52.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.328+0000 7f114b578700 1 -- 172.21.15.27:0/1709041050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11440f9910 msgr2=0x7f1144107420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:38:52.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.328+0000 7f114b578700 1 --2- 172.21.15.27:0/1709041050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11440f9910 0x7f1144107420 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f113400c790 tx=0x7f1134007d70).stop 2021-11-03T17:38:52.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.328+0000 7f114b578700 1 -- 172.21.15.27:0/1709041050 shutdown_connections 2021-11-03T17:38:52.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.328+0000 7f114b578700 1 --2- 172.21.15.27:0/1709041050 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f11300640b0 0x7f1130066570 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:52.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.328+0000 7f114b578700 1 --2- 172.21.15.27:0/1709041050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11440602d0 0x7f1144106f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:52.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.328+0000 7f114b578700 1 --2- 172.21.15.27:0/1709041050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11440f9910 0x7f1144107420 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:52.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.328+0000 7f114b578700 1 -- 172.21.15.27:0/1709041050 >> 172.21.15.27:0/1709041050 conn(0x7f11440f0de0 msgr2=0x7f11440f31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:38:52.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.328+0000 7f114b578700 1 -- 172.21.15.27:0/1709041050 shutdown_connections 2021-11-03T17:38:52.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.328+0000 7f114b578700 1 -- 172.21.15.27:0/1709041050 wait complete. 2021-11-03T17:38:52.333 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:38:52.426 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.430+0000 7fc97cfdf700 1 -- 172.21.15.27:0/2357239363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9780f66e0 msgr2=0x7fc9780f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:38:52.427 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.430+0000 7fc97cfdf700 1 --2- 172.21.15.27:0/2357239363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9780f66e0 0x7fc9780f6b00 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7fc960004660 tx=0x7fc960009b30).stop 2021-11-03T17:38:52.427 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.430+0000 7fc97cfdf700 1 -- 172.21.15.27:0/2357239363 shutdown_connections 2021-11-03T17:38:52.427 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.430+0000 7fc97cfdf700 1 --2- 172.21.15.27:0/2357239363 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9780f7980 0x7fc9780f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:52.427 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.430+0000 7fc97cfdf700 1 --2- 172.21.15.27:0/2357239363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9780f66e0 0x7fc9780f6b00 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:52.428 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.430+0000 7fc97cfdf700 1 -- 172.21.15.27:0/2357239363 >> 172.21.15.27:0/2357239363 conn(0x7fc9780f1e40 msgr2=0x7fc9780f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:38:52.428 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.430+0000 7fc97cfdf700 1 -- 172.21.15.27:0/2357239363 shutdown_connections 2021-11-03T17:38:52.428 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.430+0000 7fc97cfdf700 1 -- 172.21.15.27:0/2357239363 wait complete. 2021-11-03T17:38:52.428 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.430+0000 7fc97cfdf700 1 Processor -- start 2021-11-03T17:38:52.429 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.430+0000 7fc97cfdf700 1 -- start start 2021-11-03T17:38:52.429 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.431+0000 7fc97cfdf700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9780f66e0 0x7fc978106fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:38:52.429 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.431+0000 7fc97cfdf700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9780f7980 0x7fc9781074a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:38:52.429 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.431+0000 7fc97cfdf700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc978107a50 con 0x7fc9780f66e0 2021-11-03T17:38:52.429 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.431+0000 7fc97cfdf700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc978107b90 con 0x7fc9780f7980 2021-11-03T17:38:52.430 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.431+0000 7fc97659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9780f66e0 0x7fc978106fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:38:52.430 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.431+0000 7fc97659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9780f66e0 0x7fc978106fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58606/0 (socket says 172.21.15.27:58606) 2021-11-03T17:38:52.430 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.431+0000 7fc97659c700 1 -- 172.21.15.27:0/747342787 learned_addr learned my addr 172.21.15.27:0/747342787 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:38:52.430 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.431+0000 7fc975d9b700 1 --2- 172.21.15.27:0/747342787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9780f7980 0x7fc9781074a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:38:52.431 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.431+0000 7fc97659c700 1 -- 172.21.15.27:0/747342787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9780f7980 msgr2=0x7fc9781074a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:38:52.431 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.431+0000 7fc97659c700 1 --2- 172.21.15.27:0/747342787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9780f7980 0x7fc9781074a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:52.431 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.431+0000 7fc97659c700 1 -- 172.21.15.27:0/747342787 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc960005040 con 0x7fc9780f66e0 2021-11-03T17:38:52.431 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.431+0000 7fc97659c700 1 --2- 172.21.15.27:0/747342787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9780f66e0 0x7fc978106fb0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7fc960013a70 tx=0x7fc960009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:38:52.432 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.431+0000 7fc96effd700 1 -- 172.21.15.27:0/747342787 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc960018070 con 0x7fc9780f66e0 2021-11-03T17:38:52.432 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.431+0000 7fc96effd700 1 -- 172.21.15.27:0/747342787 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fc9600097d0 con 0x7fc9780f66e0 2021-11-03T17:38:52.432 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.431+0000 7fc97cfdf700 1 -- 172.21.15.27:0/747342787 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9781084d0 con 0x7fc9780f66e0 2021-11-03T17:38:52.432 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.431+0000 7fc96effd700 1 -- 172.21.15.27:0/747342787 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc96001de80 con 0x7fc9780f66e0 2021-11-03T17:38:52.433 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.431+0000 7fc97cfdf700 1 -- 172.21.15.27:0/747342787 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc978108a30 con 0x7fc9780f66e0 2021-11-03T17:38:52.434 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.432+0000 7fc97cfdf700 1 -- 172.21.15.27:0/747342787 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc978045ad0 con 0x7fc9780f66e0 2021-11-03T17:38:52.434 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.433+0000 7fc96effd700 1 -- 172.21.15.27:0/747342787 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fc960007ba0 con 0x7fc9780f66e0 2021-11-03T17:38:52.434 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.433+0000 7fc96effd700 1 --2- 172.21.15.27:0/747342787 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc9640640b0 0x7fc964066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:38:52.435 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.433+0000 7fc96effd700 1 -- 172.21.15.27:0/747342787 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fc9600824f0 con 0x7fc9780f66e0 2021-11-03T17:38:52.435 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.433+0000 7fc975d9b700 1 --2- 172.21.15.27:0/747342787 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc9640640b0 0x7fc964066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:38:52.435 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.434+0000 7fc975d9b700 1 --2- 172.21.15.27:0/747342787 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc9640640b0 0x7fc964066570 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7fc968004c40 tx=0x7fc96800b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:38:52.435 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.437+0000 7fc96effd700 1 -- 172.21.15.27:0/747342787 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fc9600547f0 con 0x7fc9780f66e0 2021-11-03T17:38:52.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:52 smithi115 ceph-mon[65453]: pgmap v727: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:52.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:52 smithi027 ceph-mon[97244]: pgmap v727: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:52.604 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.609+0000 7fc97cfdf700 1 -- 172.21.15.27:0/747342787 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9780008d0 con 0x7fc9640640b0 2021-11-03T17:38:52.618 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.623+0000 7fc96effd700 1 -- 172.21.15.27:0/747342787 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fc9780008d0 con 0x7fc9640640b0 2021-11-03T17:38:52.619 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:38:52.619 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (30m) 12s ago 31m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:38:52.619 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (22m) 12s ago 31m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:38:52.619 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (21m) 9m ago 30m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:38:52.620 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (30m) 12s ago 31m 41.3M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:38:52.620 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (23m) 12s ago 26m 21.1M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:38:52.620 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (26m) 12s ago 26m 3843M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:38:52.620 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (22m) 9m ago 26m 17.6M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:38:52.620 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (24m) 9m ago 26m 19.3M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:38:52.621 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (24m) 12s ago 32m 517M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:38:52.621 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (23m) 9m ago 30m 415M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:38:52.621 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (23m) 12s ago 33m 159M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:38:52.621 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (22m) 9m ago 30m 71.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:38:52.622 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (31m) 12s ago 31m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:38:52.622 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (30m) 9m ago 30m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:38:52.622 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (21m) 12s ago 30m 311M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:38:52.622 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (21m) 12s ago 29m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:38:52.623 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (20m) 12s ago 29m 248M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:38:52.623 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (20m) 9m ago 29m 329M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:38:52.623 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (19m) 9m ago 28m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:38:52.623 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (19m) 9m ago 28m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:38:52.623 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (30m) 12s ago 31m 50.5M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:38:52.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.625+0000 7fc97cfdf700 1 -- 172.21.15.27:0/747342787 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc9640640b0 msgr2=0x7fc964066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:38:52.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.625+0000 7fc97cfdf700 1 --2- 172.21.15.27:0/747342787 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc9640640b0 0x7fc964066570 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 rx=0x7fc968004c40 tx=0x7fc96800b040).stop 2021-11-03T17:38:52.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.625+0000 7fc97cfdf700 1 -- 172.21.15.27:0/747342787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9780f66e0 msgr2=0x7fc978106fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:38:52.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.625+0000 7fc97cfdf700 1 --2- 172.21.15.27:0/747342787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9780f66e0 0x7fc978106fb0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7fc960013a70 tx=0x7fc960009b30).stop 2021-11-03T17:38:52.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.625+0000 7fc97cfdf700 1 -- 172.21.15.27:0/747342787 shutdown_connections 2021-11-03T17:38:52.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.625+0000 7fc97cfdf700 1 --2- 172.21.15.27:0/747342787 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc9640640b0 0x7fc964066570 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:52.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.625+0000 7fc97cfdf700 1 --2- 172.21.15.27:0/747342787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9780f66e0 0x7fc978106fb0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:52.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.625+0000 7fc97cfdf700 1 --2- 172.21.15.27:0/747342787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9780f7980 0x7fc9781074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:52.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.625+0000 7fc97cfdf700 1 -- 172.21.15.27:0/747342787 >> 172.21.15.27:0/747342787 conn(0x7fc9780f1e40 msgr2=0x7fc9780fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:38:52.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.625+0000 7fc97cfdf700 1 -- 172.21.15.27:0/747342787 shutdown_connections 2021-11-03T17:38:52.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.625+0000 7fc97cfdf700 1 -- 172.21.15.27:0/747342787 wait complete. 2021-11-03T17:38:52.721 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.724+0000 7ffadf016700 1 -- 172.21.15.27:0/777179479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffad80f6530 msgr2=0x7ffad80f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:38:52.721 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.724+0000 7ffadf016700 1 --2- 172.21.15.27:0/777179479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffad80f6530 0x7ffad80f6950 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7ffad0004660 tx=0x7ffad0009b30).stop 2021-11-03T17:38:52.721 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.724+0000 7ffadf016700 1 -- 172.21.15.27:0/777179479 shutdown_connections 2021-11-03T17:38:52.722 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.724+0000 7ffadf016700 1 --2- 172.21.15.27:0/777179479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffad80f77d0 0x7ffad80f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:52.722 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.724+0000 7ffadf016700 1 --2- 172.21.15.27:0/777179479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffad80f6530 0x7ffad80f6950 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:52.722 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.724+0000 7ffadf016700 1 -- 172.21.15.27:0/777179479 >> 172.21.15.27:0/777179479 conn(0x7ffad80f1cd0 msgr2=0x7ffad80f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:38:52.723 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.725+0000 7ffadf016700 1 -- 172.21.15.27:0/777179479 shutdown_connections 2021-11-03T17:38:52.723 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.725+0000 7ffadf016700 1 -- 172.21.15.27:0/777179479 wait complete. 2021-11-03T17:38:52.723 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.725+0000 7ffadf016700 1 Processor -- start 2021-11-03T17:38:52.723 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.725+0000 7ffadf016700 1 -- start start 2021-11-03T17:38:52.723 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.725+0000 7ffadf016700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffad80f6530 0x7ffad8106d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:38:52.724 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.725+0000 7ffadf016700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffad80f77d0 0x7ffad8107260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:38:52.724 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.725+0000 7ffadf016700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad8107860 con 0x7ffad80f77d0 2021-11-03T17:38:52.724 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.725+0000 7ffadf016700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad8108070 con 0x7ffad80f6530 2021-11-03T17:38:52.724 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.725+0000 7ffadd813700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffad80f77d0 0x7ffad8107260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:38:52.725 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.725+0000 7ffadd813700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffad80f77d0 0x7ffad8107260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58620/0 (socket says 172.21.15.27:58620) 2021-11-03T17:38:52.725 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.725+0000 7ffadd813700 1 -- 172.21.15.27:0/3935725843 learned_addr learned my addr 172.21.15.27:0/3935725843 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:38:52.725 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.725+0000 7ffade014700 1 --2- 172.21.15.27:0/3935725843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffad80f6530 0x7ffad8106d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:38:52.725 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.726+0000 7ffadd813700 1 -- 172.21.15.27:0/3935725843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffad80f6530 msgr2=0x7ffad8106d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:38:52.725 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.726+0000 7ffadd813700 1 --2- 172.21.15.27:0/3935725843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffad80f6530 0x7ffad8106d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:52.726 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.726+0000 7ffadd813700 1 -- 172.21.15.27:0/3935725843 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffad0005040 con 0x7ffad80f77d0 2021-11-03T17:38:52.726 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.726+0000 7ffadd813700 1 --2- 172.21.15.27:0/3935725843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffad80f77d0 0x7ffad8107260 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7ffacc002700 tx=0x7ffacc009e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:38:52.726 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.726+0000 7ffad6ffd700 1 -- 172.21.15.27:0/3935725843 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffacc009320 con 0x7ffad80f77d0 2021-11-03T17:38:52.727 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.726+0000 7ffad6ffd700 1 -- 172.21.15.27:0/3935725843 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7ffacc00f040 con 0x7ffad80f77d0 2021-11-03T17:38:52.728 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.726+0000 7ffadf016700 1 -- 172.21.15.27:0/3935725843 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad81082f0 con 0x7ffad80f77d0 2021-11-03T17:38:52.728 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.726+0000 7ffad6ffd700 1 -- 172.21.15.27:0/3935725843 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffacc014870 con 0x7ffad80f77d0 2021-11-03T17:38:52.728 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.726+0000 7ffadf016700 1 -- 172.21.15.27:0/3935725843 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad81088b0 con 0x7ffad80f77d0 2021-11-03T17:38:52.728 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.727+0000 7ffadf016700 1 -- 172.21.15.27:0/3935725843 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffad8059100 con 0x7ffad80f77d0 2021-11-03T17:38:52.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.727+0000 7ffad6ffd700 1 -- 172.21.15.27:0/3935725843 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7ffacc009480 con 0x7ffad80f77d0 2021-11-03T17:38:52.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.728+0000 7ffad6ffd700 1 --2- 172.21.15.27:0/3935725843 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffac0064100 0x7ffac00665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:38:52.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.728+0000 7ffad6ffd700 1 -- 172.21.15.27:0/3935725843 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7ffacc080a10 con 0x7ffad80f77d0 2021-11-03T17:38:52.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.728+0000 7ffade014700 1 --2- 172.21.15.27:0/3935725843 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffac0064100 0x7ffac00665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:38:52.730 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.728+0000 7ffade014700 1 --2- 172.21.15.27:0/3935725843 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffac0064100 0x7ffac00665c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7ffad0013690 tx=0x7ffad0009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:38:52.730 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.732+0000 7ffad6ffd700 1 -- 172.21.15.27:0/3935725843 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7ffacc052d10 con 0x7ffad80f77d0 2021-11-03T17:38:52.956 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.961+0000 7ffadf016700 1 -- 172.21.15.27:0/3935725843 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ffad8027a00 con 0x7ffad80f77d0 2021-11-03T17:38:52.957 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.962+0000 7ffad6ffd700 1 -- 172.21.15.27:0/3935725843 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7ffacc052900 con 0x7ffad80f77d0 2021-11-03T17:38:52.957 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:38:52.958 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:38:52.958 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:38:52.958 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:38:52.958 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:38:52.959 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:38:52.959 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:38:52.959 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:38:52.960 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:38:52.960 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:38:52.960 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:38:52.960 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:38:52.960 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:38:52.961 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:38:52.961 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:38:52.961 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:38:52.961 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:38:52.962 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:38:52.963 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.963+0000 7ffadf016700 1 -- 172.21.15.27:0/3935725843 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffac0064100 msgr2=0x7ffac00665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:38:52.963 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.963+0000 7ffadf016700 1 --2- 172.21.15.27:0/3935725843 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffac0064100 0x7ffac00665c0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7ffad0013690 tx=0x7ffad0009b30).stop 2021-11-03T17:38:52.963 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.963+0000 7ffadf016700 1 -- 172.21.15.27:0/3935725843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffad80f77d0 msgr2=0x7ffad8107260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:38:52.963 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.963+0000 7ffadf016700 1 --2- 172.21.15.27:0/3935725843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffad80f77d0 0x7ffad8107260 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7ffacc002700 tx=0x7ffacc009e00).stop 2021-11-03T17:38:52.964 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.964+0000 7ffadf016700 1 -- 172.21.15.27:0/3935725843 shutdown_connections 2021-11-03T17:38:52.964 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.964+0000 7ffadf016700 1 --2- 172.21.15.27:0/3935725843 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffac0064100 0x7ffac00665c0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:52.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.964+0000 7ffadf016700 1 --2- 172.21.15.27:0/3935725843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffad80f6530 0x7ffad8106d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:52.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.964+0000 7ffadf016700 1 --2- 172.21.15.27:0/3935725843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffad80f77d0 0x7ffad8107260 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:52.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.964+0000 7ffadf016700 1 -- 172.21.15.27:0/3935725843 >> 172.21.15.27:0/3935725843 conn(0x7ffad80f1cd0 msgr2=0x7ffad80fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:38:52.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.964+0000 7ffadf016700 1 -- 172.21.15.27:0/3935725843 shutdown_connections 2021-11-03T17:38:52.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:52.964+0000 7ffadf016700 1 -- 172.21.15.27:0/3935725843 wait complete. 2021-11-03T17:38:53.059 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.064+0000 7f0eed2b1700 1 -- 172.21.15.27:0/616350145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ee80f55c0 msgr2=0x7f0ee80f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:38:53.059 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.064+0000 7f0eed2b1700 1 --2- 172.21.15.27:0/616350145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ee80f55c0 0x7f0ee80f59e0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f0ee0004660 tx=0x7f0ee0009b30).stop 2021-11-03T17:38:53.059 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.064+0000 7f0eed2b1700 1 -- 172.21.15.27:0/616350145 shutdown_connections 2021-11-03T17:38:53.060 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.064+0000 7f0eed2b1700 1 --2- 172.21.15.27:0/616350145 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ee80f6860 0x7f0ee80f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:53.060 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.064+0000 7f0eed2b1700 1 --2- 172.21.15.27:0/616350145 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ee80f55c0 0x7f0ee80f59e0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:53.060 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.064+0000 7f0eed2b1700 1 -- 172.21.15.27:0/616350145 >> 172.21.15.27:0/616350145 conn(0x7f0ee80f0d40 msgr2=0x7f0ee80f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:38:53.061 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.064+0000 7f0eed2b1700 1 -- 172.21.15.27:0/616350145 shutdown_connections 2021-11-03T17:38:53.061 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.064+0000 7f0eed2b1700 1 -- 172.21.15.27:0/616350145 wait complete. 2021-11-03T17:38:53.065 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.064+0000 7f0eed2b1700 1 Processor -- start 2021-11-03T17:38:53.065 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.064+0000 7f0eed2b1700 1 -- start start 2021-11-03T17:38:53.065 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.065+0000 7f0eed2b1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ee80f55c0 0x7f0ee8105e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:38:53.065 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.065+0000 7f0eed2b1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ee80f6860 0x7f0ee8106360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:38:53.066 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.065+0000 7f0eed2b1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee8106910 con 0x7f0ee80f55c0 2021-11-03T17:38:53.066 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.065+0000 7f0eed2b1700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ee8106a50 con 0x7f0ee80f6860 2021-11-03T17:38:53.066 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.065+0000 7f0ee6ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ee80f55c0 0x7f0ee8105e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:38:53.066 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.065+0000 7f0ee6ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ee80f55c0 0x7f0ee8105e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58628/0 (socket says 172.21.15.27:58628) 2021-11-03T17:38:53.067 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.065+0000 7f0ee6ffd700 1 -- 172.21.15.27:0/3937704401 learned_addr learned my addr 172.21.15.27:0/3937704401 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:38:53.067 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.065+0000 7f0ee67fc700 1 --2- 172.21.15.27:0/3937704401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ee80f6860 0x7f0ee8106360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:38:53.067 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.065+0000 7f0ee6ffd700 1 -- 172.21.15.27:0/3937704401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ee80f6860 msgr2=0x7f0ee8106360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:38:53.068 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.065+0000 7f0ee6ffd700 1 --2- 172.21.15.27:0/3937704401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ee80f6860 0x7f0ee8106360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:53.068 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.065+0000 7f0ee6ffd700 1 -- 172.21.15.27:0/3937704401 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee0005040 con 0x7f0ee80f55c0 2021-11-03T17:38:53.068 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.065+0000 7f0ee6ffd700 1 --2- 172.21.15.27:0/3937704401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ee80f55c0 0x7f0ee8105e70 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f0ee0013a70 tx=0x7f0ee0009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:38:53.068 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.065+0000 7f0ecf7fe700 1 -- 172.21.15.27:0/3937704401 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ee0018070 con 0x7f0ee80f55c0 2021-11-03T17:38:53.069 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.065+0000 7f0eed2b1700 1 -- 172.21.15.27:0/3937704401 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ee8107390 con 0x7f0ee80f55c0 2021-11-03T17:38:53.069 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.065+0000 7f0ecf7fe700 1 -- 172.21.15.27:0/3937704401 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f0ee00097d0 con 0x7f0ee80f55c0 2021-11-03T17:38:53.069 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.065+0000 7f0ecf7fe700 1 -- 172.21.15.27:0/3937704401 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ee001c3c0 con 0x7f0ee80f55c0 2021-11-03T17:38:53.069 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.065+0000 7f0eed2b1700 1 -- 172.21.15.27:0/3937704401 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ee81078f0 con 0x7f0ee80f55c0 2021-11-03T17:38:53.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.066+0000 7f0eed2b1700 1 -- 172.21.15.27:0/3937704401 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ee8045ad0 con 0x7f0ee80f55c0 2021-11-03T17:38:53.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.067+0000 7f0ecf7fe700 1 -- 172.21.15.27:0/3937704401 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f0ee0007ba0 con 0x7f0ee80f55c0 2021-11-03T17:38:53.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.067+0000 7f0ecf7fe700 1 --2- 172.21.15.27:0/3937704401 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0ed00640b0 0x7f0ed0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:38:53.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.067+0000 7f0ecf7fe700 1 -- 172.21.15.27:0/3937704401 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f0ee00824d0 con 0x7f0ee80f55c0 2021-11-03T17:38:53.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.067+0000 7f0ee67fc700 1 --2- 172.21.15.27:0/3937704401 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0ed00640b0 0x7f0ed0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:38:53.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.068+0000 7f0ee67fc700 1 --2- 172.21.15.27:0/3937704401 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0ed00640b0 0x7f0ed0066570 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f0edc0096c0 tx=0x7f0edc009be0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:38:53.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.071+0000 7f0ecf7fe700 1 -- 172.21.15.27:0/3937704401 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f0ee00547d0 con 0x7f0ee80f55c0 2021-11-03T17:38:53.271 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.276+0000 7f0eed2b1700 1 -- 172.21.15.27:0/3937704401 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0ee80fde50 con 0x7f0ee80f55c0 2021-11-03T17:38:53.272 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.276+0000 7f0ecf7fe700 1 -- 172.21.15.27:0/3937704401 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f0ee00543c0 con 0x7f0ee80f55c0 2021-11-03T17:38:53.272 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:38:53.272 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:38:53.273 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:38:53.273 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:38:53.273 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:38:53.273 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:38:53.273 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:38:53.274 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:38:53.274 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:38:53.274 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:38:53.274 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:38:53.275 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:38:53.275 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:38:53.275 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:38:53.275 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:38:53.276 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:38:53.276 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:38:53.276 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:38:53.276 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:38:53.276 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:38:53.277 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:38:53.277 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:38:53.277 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:38:53.277 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:38:53.278 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:38:53.278 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:38:53.278 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:38:53.278 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:38:53.278 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:38:53.279 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:38:53.279 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:38:53.279 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:38:53.279 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:38:53.280 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:38:53.280 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:38:53.280 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:38:53.280 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:38:53.280 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:38:53.281 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:38:53.282 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.278+0000 7f0eed2b1700 1 -- 172.21.15.27:0/3937704401 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0ed00640b0 msgr2=0x7f0ed0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:38:53.282 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.278+0000 7f0eed2b1700 1 --2- 172.21.15.27:0/3937704401 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0ed00640b0 0x7f0ed0066570 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7f0edc0096c0 tx=0x7f0edc009be0).stop 2021-11-03T17:38:53.282 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.278+0000 7f0eed2b1700 1 -- 172.21.15.27:0/3937704401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ee80f55c0 msgr2=0x7f0ee8105e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:38:53.282 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.278+0000 7f0eed2b1700 1 --2- 172.21.15.27:0/3937704401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ee80f55c0 0x7f0ee8105e70 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f0ee0013a70 tx=0x7f0ee0009b30).stop 2021-11-03T17:38:53.283 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.278+0000 7f0eed2b1700 1 -- 172.21.15.27:0/3937704401 shutdown_connections 2021-11-03T17:38:53.283 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.278+0000 7f0eed2b1700 1 --2- 172.21.15.27:0/3937704401 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0ed00640b0 0x7f0ed0066570 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:53.283 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.278+0000 7f0eed2b1700 1 --2- 172.21.15.27:0/3937704401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ee80f55c0 0x7f0ee8105e70 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:53.283 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.278+0000 7f0eed2b1700 1 --2- 172.21.15.27:0/3937704401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ee80f6860 0x7f0ee8106360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:38:53.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.278+0000 7f0eed2b1700 1 -- 172.21.15.27:0/3937704401 >> 172.21.15.27:0/3937704401 conn(0x7f0ee80f0d40 msgr2=0x7f0ee80fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:38:53.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.278+0000 7f0eed2b1700 1 -- 172.21.15.27:0/3937704401 shutdown_connections 2021-11-03T17:38:53.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:38:53.278+0000 7f0eed2b1700 1 -- 172.21.15.27:0/3937704401 wait complete. 2021-11-03T17:38:53.284 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:38:53.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:53 smithi115 ceph-mon[65453]: from='client.34856 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:38:53.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:53 smithi115 ceph-mon[65453]: from='client.34860 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:38:53.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:53 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3935725843' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:53.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:53 smithi027 ceph-mon[97244]: from='client.34856 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:38:53.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:53 smithi027 ceph-mon[97244]: from='client.34860 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:38:53.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:53 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3935725843' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:38:54.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:54 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3937704401' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:38:54.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:54 smithi115 ceph-mon[65453]: pgmap v728: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:54.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:54 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3937704401' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:38:54.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:54 smithi027 ceph-mon[97244]: pgmap v728: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:55.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1377.391621 secs 2021-11-03T17:38:55.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1377.391621 secs 2021-11-03T17:38:56.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:38:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:38:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:56 smithi115 ceph-mon[65453]: pgmap v729: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:38:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:38:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:56 smithi027 ceph-mon[97244]: pgmap v729: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:58.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:58 smithi115 ceph-mon[65453]: pgmap v730: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:58.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:58 smithi027 ceph-mon[97244]: pgmap v730: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:38:59.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:38:59 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1382.391755 secs 2021-11-03T17:39:00.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:38:59 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1382.391755 secs 2021-11-03T17:39:00.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:39:00.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:39:00.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:39:00.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:00.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:00.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:00.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:00.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:00 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:39:00.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:00 smithi115 ceph-mon[65453]: pgmap v731: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:01.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:39:01.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:39:01.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:39:01.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:01.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:01.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:01.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:01.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:00 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:39:01.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:00 smithi027 ceph-mon[97244]: pgmap v731: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:02.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:02 smithi115 ceph-mon[65453]: pgmap v732: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:02.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:02 smithi027 ceph-mon[97244]: pgmap v732: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:04.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:04 smithi115 ceph-mon[65453]: pgmap v733: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:04.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:04 smithi027 ceph-mon[97244]: pgmap v733: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:05.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1387.391895 secs 2021-11-03T17:39:05.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1387.391895 secs 2021-11-03T17:39:06.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:06 smithi115 ceph-mon[65453]: pgmap v734: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:06.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:06 smithi027 ceph-mon[97244]: pgmap v734: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:08.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:08 smithi115 ceph-mon[65453]: pgmap v735: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:08.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:08 smithi027 ceph-mon[97244]: pgmap v735: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:09.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:09 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1392.392042 secs 2021-11-03T17:39:10.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:09 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1392.392042 secs 2021-11-03T17:39:10.717 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:39:10.717 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:10 smithi027 ceph-mon[97244]: poll took 30095.964 ms: 1 events 2021-11-03T17:39:10.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:39:10.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:10 smithi115 ceph-mon[65453]: poll took 30095.964 ms: 1 events 2021-11-03T17:39:10.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:10 smithi115 ceph-mon[65453]: pgmap v736: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:10.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:39:10.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:39:10.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:10.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:10.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:10.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:11.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:10 smithi027 ceph-mon[97244]: pgmap v736: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:11.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:39:11.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:39:11.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:11.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:11.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:11.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:11.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:11 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:39:12.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:11 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:39:12.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:12 smithi115 ceph-mon[65453]: pgmap v737: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:13.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:12 smithi027 ceph-mon[97244]: pgmap v737: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:14.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:14 smithi115 ceph-mon[65453]: pgmap v738: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:14.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:14 smithi027 ceph-mon[97244]: pgmap v738: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:15.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1397.392170 secs 2021-11-03T17:39:15.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1397.392170 secs 2021-11-03T17:39:16.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:16 smithi115 ceph-mon[65453]: pgmap v739: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:16.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:16 smithi027 ceph-mon[97244]: pgmap v739: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:18.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:18 smithi115 ceph-mon[65453]: pgmap v740: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:18.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:18 smithi027 ceph-mon[97244]: pgmap v740: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:19.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:19 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1402.392308 secs 2021-11-03T17:39:20.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:19 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1402.392308 secs 2021-11-03T17:39:20.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:20 smithi115 ceph-mon[65453]: pgmap v741: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:20.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:39:20.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:39:20.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:39:20.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:20.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:20.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:20.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:21.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:20 smithi027 ceph-mon[97244]: pgmap v741: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:21.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:39:21.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:39:21.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:39:21.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:21.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:21.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:21.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:21.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:21 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:39:22.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:21 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:39:22.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:22 smithi115 ceph-mon[65453]: pgmap v742: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:23.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:22 smithi027 ceph-mon[97244]: pgmap v742: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:23.393 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.393+0000 7f0ea9d85700 1 -- 172.21.15.27:0/4076235509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ea40f55e0 msgr2=0x7f0ea40f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:39:23.393 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.393+0000 7f0ea9d85700 1 --2- 172.21.15.27:0/4076235509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ea40f55e0 0x7f0ea40f5a00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f0e94004660 tx=0x7f0e94009b30).stop 2021-11-03T17:39:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.394+0000 7f0ea9d85700 1 -- 172.21.15.27:0/4076235509 shutdown_connections 2021-11-03T17:39:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.394+0000 7f0ea9d85700 1 --2- 172.21.15.27:0/4076235509 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ea40f5ef0 0x7f0ea40f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:23.394 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.394+0000 7f0ea9d85700 1 --2- 172.21.15.27:0/4076235509 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ea40f55e0 0x7f0ea40f5a00 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:23.395 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.394+0000 7f0ea9d85700 1 -- 172.21.15.27:0/4076235509 >> 172.21.15.27:0/4076235509 conn(0x7f0ea40f0de0 msgr2=0x7f0ea40f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:39:23.395 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.394+0000 7f0ea9d85700 1 -- 172.21.15.27:0/4076235509 shutdown_connections 2021-11-03T17:39:23.395 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.394+0000 7f0ea9d85700 1 -- 172.21.15.27:0/4076235509 wait complete. 2021-11-03T17:39:23.395 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.394+0000 7f0ea9d85700 1 Processor -- start 2021-11-03T17:39:23.396 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.394+0000 7f0ea9d85700 1 -- start start 2021-11-03T17:39:23.396 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.395+0000 7f0ea9d85700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ea40f55e0 0x7f0ea4106ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:39:23.396 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.395+0000 7f0ea9d85700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ea40f5ef0 0x7f0ea41073e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:39:23.396 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.395+0000 7f0ea9d85700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea4107990 con 0x7f0ea40f55e0 2021-11-03T17:39:23.397 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.395+0000 7f0ea9d85700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ea4107ad0 con 0x7f0ea40f5ef0 2021-11-03T17:39:23.397 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.395+0000 7f0ea37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ea40f55e0 0x7f0ea4106ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:39:23.397 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.395+0000 7f0ea37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ea40f55e0 0x7f0ea4106ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58646/0 (socket says 172.21.15.27:58646) 2021-11-03T17:39:23.397 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.395+0000 7f0ea37fe700 1 -- 172.21.15.27:0/3746088900 learned_addr learned my addr 172.21.15.27:0/3746088900 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:39:23.397 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.395+0000 7f0ea2ffd700 1 --2- 172.21.15.27:0/3746088900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ea40f5ef0 0x7f0ea41073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:39:23.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.395+0000 7f0ea37fe700 1 -- 172.21.15.27:0/3746088900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ea40f5ef0 msgr2=0x7f0ea41073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:39:23.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.395+0000 7f0ea37fe700 1 --2- 172.21.15.27:0/3746088900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ea40f5ef0 0x7f0ea41073e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:23.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.395+0000 7f0ea37fe700 1 -- 172.21.15.27:0/3746088900 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e94005040 con 0x7f0ea40f55e0 2021-11-03T17:39:23.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.395+0000 7f0ea37fe700 1 --2- 172.21.15.27:0/3746088900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ea40f55e0 0x7f0ea4106ef0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f0e94009990 tx=0x7f0e94009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:39:23.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.395+0000 7f0e9bfff700 1 -- 172.21.15.27:0/3746088900 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e94018070 con 0x7f0ea40f55e0 2021-11-03T17:39:23.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.395+0000 7f0ea9d85700 1 -- 172.21.15.27:0/3746088900 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ea4108410 con 0x7f0ea40f55e0 2021-11-03T17:39:23.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.395+0000 7f0e9bfff700 1 -- 172.21.15.27:0/3746088900 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f0e94007ba0 con 0x7f0ea40f55e0 2021-11-03T17:39:23.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.395+0000 7f0e9bfff700 1 -- 172.21.15.27:0/3746088900 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e9401c3c0 con 0x7f0ea40f55e0 2021-11-03T17:39:23.400 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.395+0000 7f0ea9d85700 1 -- 172.21.15.27:0/3746088900 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ea4108970 con 0x7f0ea40f55e0 2021-11-03T17:39:23.400 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.396+0000 7f0ea9d85700 1 -- 172.21.15.27:0/3746088900 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ea4045ad0 con 0x7f0ea40f55e0 2021-11-03T17:39:23.400 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.397+0000 7f0e9bfff700 1 -- 172.21.15.27:0/3746088900 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f0e9401d420 con 0x7f0ea40f55e0 2021-11-03T17:39:23.400 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.397+0000 7f0e9bfff700 1 --2- 172.21.15.27:0/3746088900 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0e900640b0 0x7f0e90066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:39:23.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.397+0000 7f0e9bfff700 1 -- 172.21.15.27:0/3746088900 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f0e94082470 con 0x7f0ea40f55e0 2021-11-03T17:39:23.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.397+0000 7f0ea2ffd700 1 --2- 172.21.15.27:0/3746088900 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0e900640b0 0x7f0e90066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:39:23.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.398+0000 7f0ea2ffd700 1 --2- 172.21.15.27:0/3746088900 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0e900640b0 0x7f0e90066570 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f0e8c004ce0 tx=0x7f0e8c00b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:39:23.402 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.401+0000 7f0e9bfff700 1 -- 172.21.15.27:0/3746088900 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f0e94054830 con 0x7f0ea40f55e0 2021-11-03T17:39:23.577 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.582+0000 7f0ea9d85700 1 -- 172.21.15.27:0/3746088900 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ea40fad60 con 0x7f0e900640b0 2021-11-03T17:39:23.578 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.584+0000 7f0e9bfff700 1 -- 172.21.15.27:0/3746088900 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f0ea40fad60 con 0x7f0e900640b0 2021-11-03T17:39:23.580 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.585+0000 7f0ea9d85700 1 -- 172.21.15.27:0/3746088900 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0e900640b0 msgr2=0x7f0e90066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:39:23.581 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.585+0000 7f0ea9d85700 1 --2- 172.21.15.27:0/3746088900 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0e900640b0 0x7f0e90066570 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 rx=0x7f0e8c004ce0 tx=0x7f0e8c00b040).stop 2021-11-03T17:39:23.581 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.585+0000 7f0ea9d85700 1 -- 172.21.15.27:0/3746088900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ea40f55e0 msgr2=0x7f0ea4106ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:39:23.581 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.585+0000 7f0ea9d85700 1 --2- 172.21.15.27:0/3746088900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ea40f55e0 0x7f0ea4106ef0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f0e94009990 tx=0x7f0e94009b30).stop 2021-11-03T17:39:23.581 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.585+0000 7f0ea9d85700 1 -- 172.21.15.27:0/3746088900 shutdown_connections 2021-11-03T17:39:23.582 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.586+0000 7f0ea9d85700 1 --2- 172.21.15.27:0/3746088900 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0e900640b0 0x7f0e90066570 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:23.582 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.586+0000 7f0ea9d85700 1 --2- 172.21.15.27:0/3746088900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ea40f55e0 0x7f0ea4106ef0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:23.582 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.586+0000 7f0ea9d85700 1 --2- 172.21.15.27:0/3746088900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ea40f5ef0 0x7f0ea41073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:23.582 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.586+0000 7f0ea9d85700 1 -- 172.21.15.27:0/3746088900 >> 172.21.15.27:0/3746088900 conn(0x7f0ea40f0de0 msgr2=0x7f0ea405ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:39:23.583 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.586+0000 7f0ea9d85700 1 -- 172.21.15.27:0/3746088900 shutdown_connections 2021-11-03T17:39:23.583 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.586+0000 7f0ea9d85700 1 -- 172.21.15.27:0/3746088900 wait complete. 2021-11-03T17:39:23.590 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:39:23.683 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.686+0000 7ff9c17f7700 1 -- 172.21.15.27:0/1064726285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9bc0f55c0 msgr2=0x7ff9bc0f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:39:23.683 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.686+0000 7ff9c17f7700 1 --2- 172.21.15.27:0/1064726285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9bc0f55c0 0x7ff9bc0f59e0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7ff9b4004660 tx=0x7ff9b4009b30).stop 2021-11-03T17:39:23.683 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.686+0000 7ff9c17f7700 1 -- 172.21.15.27:0/1064726285 shutdown_connections 2021-11-03T17:39:23.684 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.686+0000 7ff9c17f7700 1 --2- 172.21.15.27:0/1064726285 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9bc0f6860 0x7ff9bc0f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:23.684 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.686+0000 7ff9c17f7700 1 --2- 172.21.15.27:0/1064726285 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9bc0f55c0 0x7ff9bc0f59e0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:23.684 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.686+0000 7ff9c17f7700 1 -- 172.21.15.27:0/1064726285 >> 172.21.15.27:0/1064726285 conn(0x7ff9bc0f0d40 msgr2=0x7ff9bc0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:39:23.684 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.686+0000 7ff9c17f7700 1 -- 172.21.15.27:0/1064726285 shutdown_connections 2021-11-03T17:39:23.684 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.686+0000 7ff9c17f7700 1 -- 172.21.15.27:0/1064726285 wait complete. 2021-11-03T17:39:23.685 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.686+0000 7ff9c17f7700 1 Processor -- start 2021-11-03T17:39:23.685 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.687+0000 7ff9c17f7700 1 -- start start 2021-11-03T17:39:23.685 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.687+0000 7ff9c17f7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9bc0f55c0 0x7ff9bc109c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:39:23.685 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.687+0000 7ff9c17f7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9bc0f6860 0x7ff9bc10c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:39:23.686 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.687+0000 7ff9c17f7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9bc0603c0 con 0x7ff9bc0f6860 2021-11-03T17:39:23.686 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.687+0000 7ff9c17f7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9bc060500 con 0x7ff9bc0f55c0 2021-11-03T17:39:23.686 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.687+0000 7ff9ba7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9bc0f6860 0x7ff9bc10c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:39:23.686 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.687+0000 7ff9ba7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9bc0f6860 0x7ff9bc10c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58658/0 (socket says 172.21.15.27:58658) 2021-11-03T17:39:23.687 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.687+0000 7ff9ba7fc700 1 -- 172.21.15.27:0/2251574901 learned_addr learned my addr 172.21.15.27:0/2251574901 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:39:23.687 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.687+0000 7ff9baffd700 1 --2- 172.21.15.27:0/2251574901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9bc0f55c0 0x7ff9bc109c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:39:23.687 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.687+0000 7ff9ba7fc700 1 -- 172.21.15.27:0/2251574901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9bc0f55c0 msgr2=0x7ff9bc109c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:39:23.687 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.687+0000 7ff9ba7fc700 1 --2- 172.21.15.27:0/2251574901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9bc0f55c0 0x7ff9bc109c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:23.688 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.687+0000 7ff9ba7fc700 1 -- 172.21.15.27:0/2251574901 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b4005040 con 0x7ff9bc0f6860 2021-11-03T17:39:23.688 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.687+0000 7ff9baffd700 1 --2- 172.21.15.27:0/2251574901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9bc0f55c0 0x7ff9bc109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:39:23.688 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.687+0000 7ff9ba7fc700 1 --2- 172.21.15.27:0/2251574901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9bc0f6860 0x7ff9bc10c190 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7ff9b00026d0 tx=0x7ff9b0009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:39:23.688 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.687+0000 7ff9a37fe700 1 -- 172.21.15.27:0/2251574901 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9b0009320 con 0x7ff9bc0f6860 2021-11-03T17:39:23.689 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.687+0000 7ff9a37fe700 1 -- 172.21.15.27:0/2251574901 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7ff9b000f040 con 0x7ff9bc0f6860 2021-11-03T17:39:23.689 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.688+0000 7ff9c17f7700 1 -- 172.21.15.27:0/2251574901 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9bc10c740 con 0x7ff9bc0f6860 2021-11-03T17:39:23.689 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.688+0000 7ff9a37fe700 1 -- 172.21.15.27:0/2251574901 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9b00147e0 con 0x7ff9bc0f6860 2021-11-03T17:39:23.689 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.688+0000 7ff9c17f7700 1 -- 172.21.15.27:0/2251574901 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9bc10ccd0 con 0x7ff9bc0f6860 2021-11-03T17:39:23.690 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.689+0000 7ff9c17f7700 1 -- 172.21.15.27:0/2251574901 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9bc045ad0 con 0x7ff9bc0f6860 2021-11-03T17:39:23.691 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.689+0000 7ff9a37fe700 1 -- 172.21.15.27:0/2251574901 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7ff9b0009480 con 0x7ff9bc0f6860 2021-11-03T17:39:23.691 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.690+0000 7ff9a37fe700 1 --2- 172.21.15.27:0/2251574901 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff9a40640b0 0x7ff9a4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:39:23.691 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.690+0000 7ff9a37fe700 1 -- 172.21.15.27:0/2251574901 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7ff9b00811e0 con 0x7ff9bc0f6860 2021-11-03T17:39:23.691 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.690+0000 7ff9baffd700 1 --2- 172.21.15.27:0/2251574901 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff9a40640b0 0x7ff9a4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:39:23.692 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.690+0000 7ff9baffd700 1 --2- 172.21.15.27:0/2251574901 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff9a40640b0 0x7ff9a4066570 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7ff9b4004e30 tx=0x7ff9b4009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:39:23.692 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.693+0000 7ff9a37fe700 1 -- 172.21.15.27:0/2251574901 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7ff9b00534e0 con 0x7ff9bc0f6860 2021-11-03T17:39:23.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.868+0000 7ff9c17f7700 1 -- 172.21.15.27:0/2251574901 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9bc057f10 con 0x7ff9a40640b0 2021-11-03T17:39:23.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.883+0000 7ff9a37fe700 1 -- 172.21.15.27:0/2251574901 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7ff9bc057f10 con 0x7ff9a40640b0 2021-11-03T17:39:23.879 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:39:23.879 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (30m) 43s ago 32m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:39:23.879 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (22m) 43s ago 32m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:39:23.879 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (22m) 9m ago 31m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:39:23.880 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (30m) 43s ago 31m 41.3M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:39:23.880 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (24m) 43s ago 27m 21.1M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:39:23.880 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (27m) 43s ago 27m 3843M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:39:23.880 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (23m) 9m ago 27m 17.6M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:39:23.881 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (24m) 9m ago 27m 19.3M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:39:23.881 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (24m) 43s ago 33m 517M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:39:23.881 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (24m) 9m ago 31m 415M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:39:23.881 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (23m) 43s ago 33m 159M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:39:23.881 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (23m) 9m ago 31m 71.0M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:39:23.882 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (31m) 43s ago 31m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:39:23.882 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (31m) 9m ago 31m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:39:23.882 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (22m) 43s ago 30m 311M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:39:23.882 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (21m) 43s ago 30m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:39:23.882 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (21m) 43s ago 29m 248M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:39:23.883 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (20m) 9m ago 29m 329M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:39:23.883 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (20m) 9m ago 29m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:39:23.883 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (19m) 9m ago 28m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:39:23.883 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (30m) 43s ago 31m 50.5M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:39:23.884 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.885+0000 7ff9c17f7700 1 -- 172.21.15.27:0/2251574901 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff9a40640b0 msgr2=0x7ff9a4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:39:23.884 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.885+0000 7ff9c17f7700 1 --2- 172.21.15.27:0/2251574901 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff9a40640b0 0x7ff9a4066570 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7ff9b4004e30 tx=0x7ff9b4009b30).stop 2021-11-03T17:39:23.885 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.885+0000 7ff9c17f7700 1 -- 172.21.15.27:0/2251574901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9bc0f6860 msgr2=0x7ff9bc10c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:39:23.885 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.885+0000 7ff9c17f7700 1 --2- 172.21.15.27:0/2251574901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9bc0f6860 0x7ff9bc10c190 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7ff9b00026d0 tx=0x7ff9b0009de0).stop 2021-11-03T17:39:23.885 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.885+0000 7ff9c17f7700 1 -- 172.21.15.27:0/2251574901 shutdown_connections 2021-11-03T17:39:23.885 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.885+0000 7ff9c17f7700 1 --2- 172.21.15.27:0/2251574901 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff9a40640b0 0x7ff9a4066570 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:23.885 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.885+0000 7ff9c17f7700 1 --2- 172.21.15.27:0/2251574901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9bc0f55c0 0x7ff9bc109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:23.886 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.885+0000 7ff9c17f7700 1 --2- 172.21.15.27:0/2251574901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9bc0f6860 0x7ff9bc10c190 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:23.886 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.885+0000 7ff9c17f7700 1 -- 172.21.15.27:0/2251574901 >> 172.21.15.27:0/2251574901 conn(0x7ff9bc0f0d40 msgr2=0x7ff9bc0fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:39:23.886 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.885+0000 7ff9c17f7700 1 -- 172.21.15.27:0/2251574901 shutdown_connections 2021-11-03T17:39:23.886 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.885+0000 7ff9c17f7700 1 -- 172.21.15.27:0/2251574901 wait complete. 2021-11-03T17:39:23.981 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.984+0000 7f366359e700 1 -- 172.21.15.27:0/2365272076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36640f4800 msgr2=0x7f36640f4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:39:23.981 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.984+0000 7f366359e700 1 --2- 172.21.15.27:0/2365272076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36640f4800 0x7f36640f4c20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f3654004660 tx=0x7f3654009b30).stop 2021-11-03T17:39:23.982 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.985+0000 7f366359e700 1 -- 172.21.15.27:0/2365272076 shutdown_connections 2021-11-03T17:39:23.982 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.985+0000 7f366359e700 1 --2- 172.21.15.27:0/2365272076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36640f51a0 0x7f36640f7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:39:23.982 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.985+0000 7f366359e700 1 --2- 172.21.15.27:0/2365272076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36640f4800 0x7f36640f4c20 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:23.983 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.985+0000 7f366359e700 1 -- 172.21.15.27:0/2365272076 >> 172.21.15.27:0/2365272076 conn(0x7f36640f0320 msgr2=0x7f36640f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:39:23.983 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.985+0000 7f366359e700 1 -- 172.21.15.27:0/2365272076 shutdown_connections 2021-11-03T17:39:23.983 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.985+0000 7f366359e700 1 -- 172.21.15.27:0/2365272076 wait complete. 2021-11-03T17:39:23.983 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.985+0000 7f366359e700 1 Processor -- start 2021-11-03T17:39:23.983 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.985+0000 7f366359e700 1 -- start start 2021-11-03T17:39:23.984 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.985+0000 7f366359e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36640f4800 0x7f3664106e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:39:23.984 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.985+0000 7f366359e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36640f51a0 0x7f3664107340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:39:23.984 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.985+0000 7f366359e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36641079d0 con 0x7f36640f51a0 2021-11-03T17:39:23.984 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.985+0000 7f366359e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3664108150 con 0x7f36640f4800 2021-11-03T17:39:23.985 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.985+0000 7f3661d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36640f51a0 0x7f3664107340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:39:23.985 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.985+0000 7f3661d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36640f51a0 0x7f3664107340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58668/0 (socket says 172.21.15.27:58668) 2021-11-03T17:39:23.985 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.985+0000 7f3661d9b700 1 -- 172.21.15.27:0/1569413876 learned_addr learned my addr 172.21.15.27:0/1569413876 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:39:23.985 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.986+0000 7f3661d9b700 1 -- 172.21.15.27:0/1569413876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36640f4800 msgr2=0x7f3664106e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:39:23.985 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.986+0000 7f366259c700 1 --2- 172.21.15.27:0/1569413876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36640f4800 0x7f3664106e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:39:23.986 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.986+0000 7f3661d9b700 1 --2- 172.21.15.27:0/1569413876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36640f4800 0x7f3664106e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:23.986 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.986+0000 7f3661d9b700 1 -- 172.21.15.27:0/1569413876 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3654005040 con 0x7f36640f51a0 2021-11-03T17:39:23.986 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.986+0000 7f3661d9b700 1 --2- 172.21.15.27:0/1569413876 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36640f51a0 0x7f3664107340 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f36580093f0 tx=0x7f3658006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:39:23.986 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.986+0000 7f3652ffd700 1 -- 172.21.15.27:0/1569413876 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3658009ea0 con 0x7f36640f51a0 2021-11-03T17:39:23.986 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.986+0000 7f366359e700 1 -- 172.21.15.27:0/1569413876 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36641083d0 con 0x7f36640f51a0 2021-11-03T17:39:23.987 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.986+0000 7f366359e700 1 -- 172.21.15.27:0/1569413876 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3664108990 con 0x7f36640f51a0 2021-11-03T17:39:23.987 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.986+0000 7f3652ffd700 1 -- 172.21.15.27:0/1569413876 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f3658013af0 con 0x7f36640f51a0 2021-11-03T17:39:23.987 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.986+0000 7f3652ffd700 1 -- 172.21.15.27:0/1569413876 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f365800da70 con 0x7f36640f51a0 2021-11-03T17:39:23.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.987+0000 7f366359e700 1 -- 172.21.15.27:0/1569413876 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3664045ad0 con 0x7f36640f51a0 2021-11-03T17:39:23.989 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.987+0000 7f3652ffd700 1 -- 172.21.15.27:0/1569413876 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f365801a060 con 0x7f36640f51a0 2021-11-03T17:39:23.989 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.988+0000 7f3652ffd700 1 --2- 172.21.15.27:0/1569413876 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f364c064100 0x7f364c0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:39:23.989 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.988+0000 7f3652ffd700 1 -- 172.21.15.27:0/1569413876 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f36580809a0 con 0x7f36640f51a0 2021-11-03T17:39:23.989 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.988+0000 7f366259c700 1 --2- 172.21.15.27:0/1569413876 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f364c064100 0x7f364c0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:39:23.990 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.988+0000 7f366259c700 1 --2- 172.21.15.27:0/1569413876 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f364c064100 0x7f364c0665c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f3654009da0 tx=0x7f3654009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:39:23.990 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:23.992+0000 7f3652ffd700 1 -- 172.21.15.27:0/1569413876 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f3658052bb0 con 0x7f36640f51a0 2021-11-03T17:39:24.217 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.222+0000 7f366359e700 1 -- 172.21.15.27:0/1569413876 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3664027a00 con 0x7f36640f51a0 2021-11-03T17:39:24.218 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.223+0000 7f3652ffd700 1 -- 172.21.15.27:0/1569413876 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f36580527a0 con 0x7f36640f51a0 2021-11-03T17:39:24.218 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:39:24.219 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:39:24.219 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:39:24.219 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:39:24.219 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:39:24.220 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:39:24.220 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:39:24.220 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:39:24.220 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:39:24.220 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:39:24.221 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:39:24.221 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:39:24.222 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:39:24.222 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:39:24.223 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:39:24.223 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:39:24.223 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:39:24.223 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:39:24.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.225+0000 7f366359e700 1 -- 172.21.15.27:0/1569413876 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f364c064100 msgr2=0x7f364c0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:39:24.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.225+0000 7f366359e700 1 --2- 172.21.15.27:0/1569413876 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f364c064100 0x7f364c0665c0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f3654009da0 tx=0x7f3654009b30).stop 2021-11-03T17:39:24.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.225+0000 7f366359e700 1 -- 172.21.15.27:0/1569413876 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36640f51a0 msgr2=0x7f3664107340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:39:24.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.225+0000 7f366359e700 1 --2- 172.21.15.27:0/1569413876 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36640f51a0 0x7f3664107340 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7f36580093f0 tx=0x7f3658006ec0).stop 2021-11-03T17:39:24.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.225+0000 7f366359e700 1 -- 172.21.15.27:0/1569413876 shutdown_connections 2021-11-03T17:39:24.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.225+0000 7f366359e700 1 --2- 172.21.15.27:0/1569413876 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f364c064100 0x7f364c0665c0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:24.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.225+0000 7f366359e700 1 --2- 172.21.15.27:0/1569413876 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36640f4800 0x7f3664106e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:24.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.225+0000 7f366359e700 1 --2- 172.21.15.27:0/1569413876 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36640f51a0 0x7f3664107340 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:24.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.225+0000 7f366359e700 1 -- 172.21.15.27:0/1569413876 >> 172.21.15.27:0/1569413876 conn(0x7f36640f0320 msgr2=0x7f36640fb370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:39:24.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.225+0000 7f366359e700 1 -- 172.21.15.27:0/1569413876 shutdown_connections 2021-11-03T17:39:24.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.225+0000 7f366359e700 1 -- 172.21.15.27:0/1569413876 wait complete. 2021-11-03T17:39:24.321 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.326+0000 7f554f350700 1 -- 172.21.15.27:0/3798487058 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55480f7980 msgr2=0x7f55480f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:39:24.322 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.326+0000 7f554f350700 1 --2- 172.21.15.27:0/3798487058 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55480f7980 0x7f55480f7de0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f5544004660 tx=0x7f5544009b30).stop 2021-11-03T17:39:24.322 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.326+0000 7f554f350700 1 -- 172.21.15.27:0/3798487058 shutdown_connections 2021-11-03T17:39:24.323 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.326+0000 7f554f350700 1 --2- 172.21.15.27:0/3798487058 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55480f7980 0x7f55480f7de0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:24.323 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.326+0000 7f554f350700 1 --2- 172.21.15.27:0/3798487058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55480f66e0 0x7f55480f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:24.323 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.326+0000 7f554f350700 1 -- 172.21.15.27:0/3798487058 >> 172.21.15.27:0/3798487058 conn(0x7f55480f1e40 msgr2=0x7f55480f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:39:24.324 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.326+0000 7f554f350700 1 -- 172.21.15.27:0/3798487058 shutdown_connections 2021-11-03T17:39:24.324 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.326+0000 7f554f350700 1 -- 172.21.15.27:0/3798487058 wait complete. 2021-11-03T17:39:24.324 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.326+0000 7f554f350700 1 Processor -- start 2021-11-03T17:39:24.324 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.326+0000 7f554f350700 1 -- start start 2021-11-03T17:39:24.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.327+0000 7f554f350700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55480f66e0 0x7f5548106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:39:24.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.327+0000 7f554f350700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55480f7980 0x7f5548107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:39:24.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.327+0000 7f554f350700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5548107a00 con 0x7f55480f7980 2021-11-03T17:39:24.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.327+0000 7f554f350700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5548107b40 con 0x7f55480f66e0 2021-11-03T17:39:24.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.327+0000 7f554c8eb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55480f7980 0x7f5548107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:39:24.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.327+0000 7f554c8eb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55480f7980 0x7f5548107450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58678/0 (socket says 172.21.15.27:58678) 2021-11-03T17:39:24.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.327+0000 7f554c8eb700 1 -- 172.21.15.27:0/2092508714 learned_addr learned my addr 172.21.15.27:0/2092508714 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:39:24.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.327+0000 7f554c8eb700 1 -- 172.21.15.27:0/2092508714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55480f66e0 msgr2=0x7f5548106f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:39:24.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.327+0000 7f554d0ec700 1 --2- 172.21.15.27:0/2092508714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55480f66e0 0x7f5548106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:39:24.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.327+0000 7f554c8eb700 1 --2- 172.21.15.27:0/2092508714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55480f66e0 0x7f5548106f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:24.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.327+0000 7f554c8eb700 1 -- 172.21.15.27:0/2092508714 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5544005040 con 0x7f55480f7980 2021-11-03T17:39:24.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.327+0000 7f554d0ec700 1 --2- 172.21.15.27:0/2092508714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55480f66e0 0x7f5548106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:39:24.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.327+0000 7f554c8eb700 1 --2- 172.21.15.27:0/2092508714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55480f7980 0x7f5548107450 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f5544009990 tx=0x7f5544009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:39:24.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.327+0000 7f5539ffb700 1 -- 172.21.15.27:0/2092508714 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5544018070 con 0x7f55480f7980 2021-11-03T17:39:24.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.327+0000 7f5539ffb700 1 -- 172.21.15.27:0/2092508714 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f5544007ca0 con 0x7f55480f7980 2021-11-03T17:39:24.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.327+0000 7f554f350700 1 -- 172.21.15.27:0/2092508714 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5548108480 con 0x7f55480f7980 2021-11-03T17:39:24.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.327+0000 7f5539ffb700 1 -- 172.21.15.27:0/2092508714 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f554401c620 con 0x7f55480f7980 2021-11-03T17:39:24.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.328+0000 7f554f350700 1 -- 172.21.15.27:0/2092508714 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55481089e0 con 0x7f55480f7980 2021-11-03T17:39:24.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.328+0000 7f554f350700 1 -- 172.21.15.27:0/2092508714 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5548045ad0 con 0x7f55480f7980 2021-11-03T17:39:24.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.329+0000 7f5539ffb700 1 -- 172.21.15.27:0/2092508714 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f554401d960 con 0x7f55480f7980 2021-11-03T17:39:24.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.329+0000 7f5539ffb700 1 --2- 172.21.15.27:0/2092508714 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f55340640b0 0x7f5534066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:39:24.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.329+0000 7f5539ffb700 1 -- 172.21.15.27:0/2092508714 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f5544082660 con 0x7f55480f7980 2021-11-03T17:39:24.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.329+0000 7f554d0ec700 1 --2- 172.21.15.27:0/2092508714 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f55340640b0 0x7f5534066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:39:24.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.330+0000 7f554d0ec700 1 --2- 172.21.15.27:0/2092508714 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f55340640b0 0x7f5534066570 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f55480f77e0 tx=0x7f553c009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:39:24.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.333+0000 7f5539ffb700 1 -- 172.21.15.27:0/2092508714 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f5544054960 con 0x7f55480f7980 2021-11-03T17:39:24.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:24 smithi115 ceph-mon[65453]: from='client.34872 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:39:24.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:24 smithi115 ceph-mon[65453]: from='client.34876 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:39:24.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:24 smithi115 ceph-mon[65453]: pgmap v743: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:24.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:24 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1569413876' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:24.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:24 smithi027 ceph-mon[97244]: from='client.34872 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:39:24.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:24 smithi027 ceph-mon[97244]: from='client.34876 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:39:24.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:24 smithi027 ceph-mon[97244]: pgmap v743: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:24.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:24 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1569413876' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:24.532 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.537+0000 7f554f350700 1 -- 172.21.15.27:0/2092508714 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5548027a00 con 0x7f55480f7980 2021-11-03T17:39:24.534 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.538+0000 7f5539ffb700 1 -- 172.21.15.27:0/2092508714 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f5544054550 con 0x7f55480f7980 2021-11-03T17:39:24.534 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:39:24.535 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:39:24.535 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:39:24.536 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:39:24.536 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:39:24.536 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:39:24.536 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:39:24.537 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:39:24.537 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:39:24.537 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:39:24.537 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:39:24.537 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:39:24.538 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:39:24.538 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:39:24.538 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:39:24.538 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:39:24.539 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:39:24.539 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:39:24.539 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:39:24.540 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:39:24.540 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:39:24.540 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:39:24.541 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:39:24.541 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:39:24.542 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:39:24.542 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:39:24.543 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:39:24.543 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:39:24.543 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:39:24.544 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:39:24.544 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:39:24.544 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:39:24.544 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:39:24.545 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:39:24.545 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:39:24.545 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:39:24.545 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:39:24.545 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:39:24.546 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:39:24.547 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.540+0000 7f554f350700 1 -- 172.21.15.27:0/2092508714 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f55340640b0 msgr2=0x7f5534066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:39:24.547 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.540+0000 7f554f350700 1 --2- 172.21.15.27:0/2092508714 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f55340640b0 0x7f5534066570 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f55480f77e0 tx=0x7f553c009040).stop 2021-11-03T17:39:24.548 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.540+0000 7f554f350700 1 -- 172.21.15.27:0/2092508714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55480f7980 msgr2=0x7f5548107450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:39:24.548 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.540+0000 7f554f350700 1 --2- 172.21.15.27:0/2092508714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55480f7980 0x7f5548107450 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f5544009990 tx=0x7f5544009b30).stop 2021-11-03T17:39:24.548 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.540+0000 7f554f350700 1 -- 172.21.15.27:0/2092508714 shutdown_connections 2021-11-03T17:39:24.548 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.540+0000 7f554f350700 1 --2- 172.21.15.27:0/2092508714 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f55340640b0 0x7f5534066570 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:24.548 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.540+0000 7f554f350700 1 --2- 172.21.15.27:0/2092508714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55480f66e0 0x7f5548106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:24.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.540+0000 7f554f350700 1 --2- 172.21.15.27:0/2092508714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55480f7980 0x7f5548107450 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:24.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.540+0000 7f554f350700 1 -- 172.21.15.27:0/2092508714 >> 172.21.15.27:0/2092508714 conn(0x7f55480f1e40 msgr2=0x7f55480fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:39:24.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.540+0000 7f554f350700 1 -- 172.21.15.27:0/2092508714 shutdown_connections 2021-11-03T17:39:24.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:24.540+0000 7f554f350700 1 -- 172.21.15.27:0/2092508714 wait complete. 2021-11-03T17:39:24.550 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:39:25.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1407.392391 secs 2021-11-03T17:39:25.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:25 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2092508714' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:39:25.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1407.392391 secs 2021-11-03T17:39:25.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:25 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2092508714' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:39:26.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:26 smithi115 ceph-mon[65453]: pgmap v744: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:26.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:26 smithi027 ceph-mon[97244]: pgmap v744: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:28.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:28 smithi115 ceph-mon[65453]: pgmap v745: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:28.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:28 smithi027 ceph-mon[97244]: pgmap v745: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:29.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:29 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1412.392522 secs 2021-11-03T17:39:30.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:29 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1412.392522 secs 2021-11-03T17:39:30.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:30 smithi115 ceph-mon[65453]: pgmap v746: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:30.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:39:30.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:39:30.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:39:30.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:30.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:30.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:30.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:31.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:30 smithi027 ceph-mon[97244]: pgmap v746: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:31.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:39:31.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:39:31.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:39:31.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:31.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:31.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:31.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:31.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:31 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:39:32.017 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:31 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:39:32.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:32 smithi115 ceph-mon[65453]: pgmap v747: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:33.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:32 smithi027 ceph-mon[97244]: pgmap v747: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:34.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:34 smithi115 ceph-mon[65453]: pgmap v748: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:34.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:34 smithi027 ceph-mon[97244]: pgmap v748: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:35.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1417.392668 secs 2021-11-03T17:39:35.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1417.392668 secs 2021-11-03T17:39:36.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:36 smithi115 ceph-mon[65453]: pgmap v749: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:36.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:36 smithi027 ceph-mon[97244]: pgmap v749: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:38.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:38 smithi115 ceph-mon[65453]: pgmap v750: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:38.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:38 smithi027 ceph-mon[97244]: pgmap v750: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:39.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:39 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1422.392755 secs 2021-11-03T17:39:40.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:39 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1422.392755 secs 2021-11-03T17:39:40.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:40 smithi115 ceph-mon[65453]: pgmap v751: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:40.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:39:41.017 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:40 smithi027 ceph-mon[97244]: pgmap v751: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:41.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:39:41.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:41 smithi027 ceph-mon[97244]: poll took 30094.517 ms: 1 events 2021-11-03T17:39:41.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:41 smithi115 ceph-mon[65453]: poll took 30094.517 ms: 1 events 2021-11-03T17:39:42.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:42 smithi027 ceph-mon[97244]: pgmap v752: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:42.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:42 smithi115 ceph-mon[65453]: pgmap v752: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:44.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:44 smithi115 ceph-mon[65453]: poll took 2601.620 ms: 1 events 2021-11-03T17:39:44.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:39:44.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:44 smithi115 ceph-mon[65453]: Detected new or changed devices on smithi027 2021-11-03T17:39:44.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:39:44.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:39:44.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:39:44.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:44.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:44.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:44.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:44.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:44 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:39:44.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:44 smithi115 ceph-mon[65453]: pgmap v753: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:45.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:44 smithi027 ceph-mon[97244]: poll took 2601.620 ms: 1 events 2021-11-03T17:39:45.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:39:45.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:44 smithi027 ceph-mon[97244]: Detected new or changed devices on smithi027 2021-11-03T17:39:45.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:39:45.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:39:45.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:39:45.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:45.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:45.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:45.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:45.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:44 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:39:45.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:44 smithi027 ceph-mon[97244]: pgmap v753: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:45.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1427.392849 secs 2021-11-03T17:39:46.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1427.392849 secs 2021-11-03T17:39:46.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:46 smithi115 ceph-mon[65453]: pgmap v754: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:47.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:46 smithi027 ceph-mon[97244]: pgmap v754: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:48.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:48 smithi115 ceph-mon[65453]: pgmap v755: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:48.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:48 smithi027 ceph-mon[97244]: pgmap v755: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:49.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:49 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1432.392988 secs 2021-11-03T17:39:50.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:49 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1432.392988 secs 2021-11-03T17:39:50.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:50 smithi115 ceph-mon[65453]: pgmap v756: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:51.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:50 smithi027 ceph-mon[97244]: pgmap v756: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:52.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:52 smithi115 ceph-mon[65453]: pgmap v757: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:52.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:52 smithi027 ceph-mon[97244]: pgmap v757: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:54.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:39:54.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:39:54.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:39:54.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:54.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:54.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:54.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:54.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:39:54.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:39:54.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:39:54.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:54.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:54.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:54.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:54.650 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.656+0000 7f04e880b700 1 -- 172.21.15.27:0/1596851352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04e40f6530 msgr2=0x7f04e40f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:39:54.650 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.656+0000 7f04e880b700 1 --2- 172.21.15.27:0/1596851352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04e40f6530 0x7f04e40f6950 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f04cc004660 tx=0x7f04cc009b30).stop 2021-11-03T17:39:54.651 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.656+0000 7f04e880b700 1 -- 172.21.15.27:0/1596851352 shutdown_connections 2021-11-03T17:39:54.651 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.656+0000 7f04e880b700 1 --2- 172.21.15.27:0/1596851352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04e40f77d0 0x7f04e40f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:54.651 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.656+0000 7f04e880b700 1 --2- 172.21.15.27:0/1596851352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04e40f6530 0x7f04e40f6950 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:54.651 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.656+0000 7f04e880b700 1 -- 172.21.15.27:0/1596851352 >> 172.21.15.27:0/1596851352 conn(0x7f04e40f1cd0 msgr2=0x7f04e40f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:39:54.652 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.656+0000 7f04e880b700 1 -- 172.21.15.27:0/1596851352 shutdown_connections 2021-11-03T17:39:54.652 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.656+0000 7f04e880b700 1 -- 172.21.15.27:0/1596851352 wait complete. 2021-11-03T17:39:54.654 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.657+0000 7f04e880b700 1 Processor -- start 2021-11-03T17:39:54.654 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.657+0000 7f04e880b700 1 -- start start 2021-11-03T17:39:54.655 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.657+0000 7f04e880b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04e40f6530 0x7f04e4106db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:39:54.655 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.657+0000 7f04e880b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04e40f77d0 0x7f04e41072a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:39:54.655 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.657+0000 7f04e880b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e41078a0 con 0x7f04e40f6530 2021-11-03T17:39:54.655 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.657+0000 7f04e880b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04e41080e0 con 0x7f04e40f77d0 2021-11-03T17:39:54.656 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.657+0000 7f04e2d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04e40f6530 0x7f04e4106db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:39:54.656 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.657+0000 7f04e2d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04e40f6530 0x7f04e4106db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58694/0 (socket says 172.21.15.27:58694) 2021-11-03T17:39:54.656 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.657+0000 7f04e2d9d700 1 -- 172.21.15.27:0/2177281858 learned_addr learned my addr 172.21.15.27:0/2177281858 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:39:54.656 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.657+0000 7f04e259c700 1 --2- 172.21.15.27:0/2177281858 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04e40f77d0 0x7f04e41072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:39:54.657 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.657+0000 7f04e2d9d700 1 -- 172.21.15.27:0/2177281858 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04e40f77d0 msgr2=0x7f04e41072a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:39:54.657 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.657+0000 7f04e2d9d700 1 --2- 172.21.15.27:0/2177281858 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04e40f77d0 0x7f04e41072a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:54.657 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.657+0000 7f04e2d9d700 1 -- 172.21.15.27:0/2177281858 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04cc005040 con 0x7f04e40f6530 2021-11-03T17:39:54.657 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.657+0000 7f04e2d9d700 1 --2- 172.21.15.27:0/2177281858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04e40f6530 0x7f04e4106db0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f04cc013a70 tx=0x7f04cc009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:39:54.657 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.658+0000 7f04db7fe700 1 -- 172.21.15.27:0/2177281858 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04cc018070 con 0x7f04e40f6530 2021-11-03T17:39:54.658 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.658+0000 7f04e880b700 1 -- 172.21.15.27:0/2177281858 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04e4108360 con 0x7f04e40f6530 2021-11-03T17:39:54.658 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.658+0000 7f04db7fe700 1 -- 172.21.15.27:0/2177281858 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f04cc0097d0 con 0x7f04e40f6530 2021-11-03T17:39:54.658 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.658+0000 7f04db7fe700 1 -- 172.21.15.27:0/2177281858 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04cc01c3c0 con 0x7f04e40f6530 2021-11-03T17:39:54.658 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.658+0000 7f04e880b700 1 -- 172.21.15.27:0/2177281858 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04e4108840 con 0x7f04e40f6530 2021-11-03T17:39:54.659 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.659+0000 7f04e880b700 1 -- 172.21.15.27:0/2177281858 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04e4059100 con 0x7f04e40f6530 2021-11-03T17:39:54.659 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.659+0000 7f04db7fe700 1 -- 172.21.15.27:0/2177281858 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f04cc007ba0 con 0x7f04e40f6530 2021-11-03T17:39:54.659 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.660+0000 7f04db7fe700 1 --2- 172.21.15.27:0/2177281858 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f04d0064100 0x7f04d00665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:39:54.659 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.660+0000 7f04db7fe700 1 -- 172.21.15.27:0/2177281858 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f04cc082510 con 0x7f04e40f6530 2021-11-03T17:39:54.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.660+0000 7f04e259c700 1 --2- 172.21.15.27:0/2177281858 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f04d0064100 0x7f04d00665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:39:54.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.660+0000 7f04e259c700 1 --2- 172.21.15.27:0/2177281858 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f04d0064100 0x7f04d00665c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f04e4106b40 tx=0x7f04d4009be0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:39:54.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.664+0000 7f04db7fe700 1 -- 172.21.15.27:0/2177281858 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f04cc054810 con 0x7f04e40f6530 2021-11-03T17:39:54.836 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.842+0000 7f04e880b700 1 -- 172.21.15.27:0/2177281858 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f04e40fee00 con 0x7f04d0064100 2021-11-03T17:39:54.837 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.844+0000 7f04db7fe700 1 -- 172.21.15.27:0/2177281858 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f04e40fee00 con 0x7f04d0064100 2021-11-03T17:39:54.839 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.845+0000 7f04e880b700 1 -- 172.21.15.27:0/2177281858 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f04d0064100 msgr2=0x7f04d00665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:39:54.840 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.845+0000 7f04e880b700 1 --2- 172.21.15.27:0/2177281858 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f04d0064100 0x7f04d00665c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f04e4106b40 tx=0x7f04d4009be0).stop 2021-11-03T17:39:54.840 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.845+0000 7f04e880b700 1 -- 172.21.15.27:0/2177281858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04e40f6530 msgr2=0x7f04e4106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:39:54.840 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.845+0000 7f04e880b700 1 --2- 172.21.15.27:0/2177281858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04e40f6530 0x7f04e4106db0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 rx=0x7f04cc013a70 tx=0x7f04cc009b30).stop 2021-11-03T17:39:54.840 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.845+0000 7f04e880b700 1 -- 172.21.15.27:0/2177281858 shutdown_connections 2021-11-03T17:39:54.841 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.845+0000 7f04e880b700 1 --2- 172.21.15.27:0/2177281858 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f04d0064100 0x7f04d00665c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:54.841 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.845+0000 7f04e880b700 1 --2- 172.21.15.27:0/2177281858 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04e40f6530 0x7f04e4106db0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:54.841 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.845+0000 7f04e880b700 1 --2- 172.21.15.27:0/2177281858 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04e40f77d0 0x7f04e41072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:54.841 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.845+0000 7f04e880b700 1 -- 172.21.15.27:0/2177281858 >> 172.21.15.27:0/2177281858 conn(0x7f04e40f1cd0 msgr2=0x7f04e40fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:39:54.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.845+0000 7f04e880b700 1 -- 172.21.15.27:0/2177281858 shutdown_connections 2021-11-03T17:39:54.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.845+0000 7f04e880b700 1 -- 172.21.15.27:0/2177281858 wait complete. 2021-11-03T17:39:54.849 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:39:54.941 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.945+0000 7f4a167b1700 1 -- 172.21.15.27:0/2412579361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a100f66e0 msgr2=0x7f4a100f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:39:54.942 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.945+0000 7f4a167b1700 1 --2- 172.21.15.27:0/2412579361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a100f66e0 0x7f4a100f6b00 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f4a00004660 tx=0x7f4a00009b30).stop 2021-11-03T17:39:54.942 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.946+0000 7f4a167b1700 1 -- 172.21.15.27:0/2412579361 shutdown_connections 2021-11-03T17:39:54.942 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.946+0000 7f4a167b1700 1 --2- 172.21.15.27:0/2412579361 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a100f6ff0 0x7f4a100f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:54.942 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.946+0000 7f4a167b1700 1 --2- 172.21.15.27:0/2412579361 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a100f66e0 0x7f4a100f6b00 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:54.943 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.946+0000 7f4a167b1700 1 -- 172.21.15.27:0/2412579361 >> 172.21.15.27:0/2412579361 conn(0x7f4a100f1f00 msgr2=0x7f4a100f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:39:54.943 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.946+0000 7f4a167b1700 1 -- 172.21.15.27:0/2412579361 shutdown_connections 2021-11-03T17:39:54.943 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.946+0000 7f4a167b1700 1 -- 172.21.15.27:0/2412579361 wait complete. 2021-11-03T17:39:54.943 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.946+0000 7f4a167b1700 1 Processor -- start 2021-11-03T17:39:54.944 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.946+0000 7f4a167b1700 1 -- start start 2021-11-03T17:39:54.944 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.946+0000 7f4a167b1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a100f66e0 0x7f4a1006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:39:54.944 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.946+0000 7f4a167b1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a100f6ff0 0x7f4a10069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:39:54.944 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.946+0000 7f4a167b1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1006a380 con 0x7f4a100f66e0 2021-11-03T17:39:54.944 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.947+0000 7f4a167b1700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a1006a4c0 con 0x7f4a100f6ff0 2021-11-03T17:39:54.945 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.947+0000 7f4a0ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a100f66e0 0x7f4a1006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:39:54.945 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.947+0000 7f4a0ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a100f66e0 0x7f4a1006b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58704/0 (socket says 172.21.15.27:58704) 2021-11-03T17:39:54.945 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.947+0000 7f4a0ffff700 1 -- 172.21.15.27:0/1789796979 learned_addr learned my addr 172.21.15.27:0/1789796979 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:39:54.945 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.947+0000 7f4a0f7fe700 1 --2- 172.21.15.27:0/1789796979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a100f6ff0 0x7f4a10069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:39:54.946 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.947+0000 7f4a0ffff700 1 -- 172.21.15.27:0/1789796979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a100f6ff0 msgr2=0x7f4a10069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:39:54.946 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.947+0000 7f4a0ffff700 1 --2- 172.21.15.27:0/1789796979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a100f6ff0 0x7f4a10069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:54.946 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.947+0000 7f4a0ffff700 1 -- 172.21.15.27:0/1789796979 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a00005040 con 0x7f4a100f66e0 2021-11-03T17:39:54.946 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.947+0000 7f4a0ffff700 1 --2- 172.21.15.27:0/1789796979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a100f66e0 0x7f4a1006b9a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f4a00009830 tx=0x7f4a00009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:39:54.946 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.947+0000 7f4a0cff9700 1 -- 172.21.15.27:0/1789796979 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a00018070 con 0x7f4a100f66e0 2021-11-03T17:39:54.947 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.947+0000 7f4a167b1700 1 -- 172.21.15.27:0/1789796979 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a1006a740 con 0x7f4a100f66e0 2021-11-03T17:39:54.947 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.947+0000 7f4a0cff9700 1 -- 172.21.15.27:0/1789796979 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f4a00007ba0 con 0x7f4a100f66e0 2021-11-03T17:39:54.947 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.947+0000 7f4a0cff9700 1 -- 172.21.15.27:0/1789796979 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a0001c620 con 0x7f4a100f66e0 2021-11-03T17:39:54.947 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.947+0000 7f4a167b1700 1 -- 172.21.15.27:0/1789796979 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a1006aca0 con 0x7f4a100f66e0 2021-11-03T17:39:54.949 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.948+0000 7f4a167b1700 1 -- 172.21.15.27:0/1789796979 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a10045ad0 con 0x7f4a100f66e0 2021-11-03T17:39:54.949 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.949+0000 7f4a0cff9700 1 -- 172.21.15.27:0/1789796979 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f4a0001c850 con 0x7f4a100f66e0 2021-11-03T17:39:54.949 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.949+0000 7f4a0cff9700 1 --2- 172.21.15.27:0/1789796979 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f49f8064090 0x7f49f8066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:39:54.949 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.949+0000 7f4a0cff9700 1 -- 172.21.15.27:0/1789796979 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f4a000824a0 con 0x7f4a100f66e0 2021-11-03T17:39:54.950 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.949+0000 7f4a0f7fe700 1 --2- 172.21.15.27:0/1789796979 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f49f8064090 0x7f49f8066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:39:54.950 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.950+0000 7f4a0f7fe700 1 --2- 172.21.15.27:0/1789796979 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f49f8064090 0x7f49f8066550 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f4a04009760 tx=0x7f4a04009c60).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:39:54.950 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:54.953+0000 7f4a0cff9700 1 -- 172.21.15.27:0/1789796979 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f4a00054860 con 0x7f4a100f66e0 2021-11-03T17:39:55.122 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.128+0000 7f4a167b1700 1 -- 172.21.15.27:0/1789796979 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a100ff010 con 0x7f49f8064090 2021-11-03T17:39:55.137 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.143+0000 7f4a0cff9700 1 -- 172.21.15.27:0/1789796979 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f4a100ff010 con 0x7f49f8064090 2021-11-03T17:39:55.137 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:39:55.138 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (31m) 75s ago 32m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:39:55.138 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (23m) 75s ago 32m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:39:55.138 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (23m) 11s ago 31m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:39:55.138 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (31m) 75s ago 32m 41.3M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:39:55.139 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (24m) 75s ago 27m 21.1M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:39:55.139 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (27m) 75s ago 27m 3843M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:39:55.139 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (23m) 11s ago 27m 20.4M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:39:55.139 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (25m) 11s ago 27m 22.1M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:39:55.140 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (25m) 75s ago 34m 517M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:39:55.140 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (24m) 11s ago 31m 419M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:39:55.140 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (24m) 75s ago 34m 159M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:39:55.140 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (23m) 11s ago 31m 82.5M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:39:55.140 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (32m) 75s ago 32m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:39:55.141 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (31m) 11s ago 31m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:39:55.141 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (22m) 75s ago 31m 311M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:39:55.141 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (22m) 75s ago 30m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:39:55.141 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (21m) 75s ago 30m 248M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:39:55.141 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (21m) 11s ago 30m 354M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:39:55.142 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (20m) 11s ago 29m 285M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:39:55.142 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (20m) 11s ago 29m 287M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:39:55.142 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (31m) 75s ago 32m 50.5M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:39:55.143 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.145+0000 7f4a167b1700 1 -- 172.21.15.27:0/1789796979 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f49f8064090 msgr2=0x7f49f8066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:39:55.143 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.145+0000 7f4a167b1700 1 --2- 172.21.15.27:0/1789796979 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f49f8064090 0x7f49f8066550 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f4a04009760 tx=0x7f4a04009c60).stop 2021-11-03T17:39:55.144 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.145+0000 7f4a167b1700 1 -- 172.21.15.27:0/1789796979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a100f66e0 msgr2=0x7f4a1006b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:39:55.144 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.145+0000 7f4a167b1700 1 --2- 172.21.15.27:0/1789796979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a100f66e0 0x7f4a1006b9a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f4a00009830 tx=0x7f4a00009b30).stop 2021-11-03T17:39:55.144 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.145+0000 7f4a167b1700 1 -- 172.21.15.27:0/1789796979 shutdown_connections 2021-11-03T17:39:55.144 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.145+0000 7f4a167b1700 1 --2- 172.21.15.27:0/1789796979 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f49f8064090 0x7f49f8066550 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:55.145 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.145+0000 7f4a167b1700 1 --2- 172.21.15.27:0/1789796979 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a100f66e0 0x7f4a1006b9a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:55.145 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.145+0000 7f4a167b1700 1 --2- 172.21.15.27:0/1789796979 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a100f6ff0 0x7f4a10069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:55.145 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.145+0000 7f4a167b1700 1 -- 172.21.15.27:0/1789796979 >> 172.21.15.27:0/1789796979 conn(0x7f4a100f1f00 msgr2=0x7f4a100fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:39:55.145 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.145+0000 7f4a167b1700 1 -- 172.21.15.27:0/1789796979 shutdown_connections 2021-11-03T17:39:55.145 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.145+0000 7f4a167b1700 1 -- 172.21.15.27:0/1789796979 wait complete. 2021-11-03T17:39:55.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.244+0000 7f6c721b5700 1 -- 172.21.15.27:0/3769144691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c6c0f55e0 msgr2=0x7f6c6c0f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:39:55.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.244+0000 7f6c721b5700 1 --2- 172.21.15.27:0/3769144691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c6c0f55e0 0x7f6c6c0f5a00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7f6c5c004660 tx=0x7f6c5c009b30).stop 2021-11-03T17:39:55.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.244+0000 7f6c721b5700 1 -- 172.21.15.27:0/3769144691 shutdown_connections 2021-11-03T17:39:55.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.244+0000 7f6c721b5700 1 --2- 172.21.15.27:0/3769144691 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c6c0f5ef0 0x7f6c6c0f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:55.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.244+0000 7f6c721b5700 1 --2- 172.21.15.27:0/3769144691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c6c0f55e0 0x7f6c6c0f5a00 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:55.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.244+0000 7f6c721b5700 1 -- 172.21.15.27:0/3769144691 >> 172.21.15.27:0/3769144691 conn(0x7f6c6c0f0de0 msgr2=0x7f6c6c0f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:39:55.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.244+0000 7f6c721b5700 1 -- 172.21.15.27:0/3769144691 shutdown_connections 2021-11-03T17:39:55.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.244+0000 7f6c721b5700 1 -- 172.21.15.27:0/3769144691 wait complete. 2021-11-03T17:39:55.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.244+0000 7f6c721b5700 1 Processor -- start 2021-11-03T17:39:55.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.244+0000 7f6c721b5700 1 -- start start 2021-11-03T17:39:55.243 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.245+0000 7f6c721b5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c6c0f55e0 0x7f6c6c106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:39:55.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.245+0000 7f6c721b5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c6c0f5ef0 0x7f6c6c107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:39:55.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.245+0000 7f6c721b5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6c107a00 con 0x7f6c6c0f5ef0 2021-11-03T17:39:55.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.245+0000 7f6c721b5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c6c107b40 con 0x7f6c6c0f55e0 2021-11-03T17:39:55.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.245+0000 7f6c6affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c6c0f5ef0 0x7f6c6c107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:39:55.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.245+0000 7f6c6affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c6c0f5ef0 0x7f6c6c107450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58716/0 (socket says 172.21.15.27:58716) 2021-11-03T17:39:55.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.245+0000 7f6c6affd700 1 -- 172.21.15.27:0/1187706204 learned_addr learned my addr 172.21.15.27:0/1187706204 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:39:55.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.245+0000 7f6c6b7fe700 1 --2- 172.21.15.27:0/1187706204 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c6c0f55e0 0x7f6c6c106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:39:55.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.245+0000 7f6c6affd700 1 -- 172.21.15.27:0/1187706204 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c6c0f55e0 msgr2=0x7f6c6c106f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:39:55.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.245+0000 7f6c6affd700 1 --2- 172.21.15.27:0/1187706204 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c6c0f55e0 0x7f6c6c106f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:55.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.245+0000 7f6c6affd700 1 -- 172.21.15.27:0/1187706204 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c5c005040 con 0x7f6c6c0f5ef0 2021-11-03T17:39:55.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.245+0000 7f6c6affd700 1 --2- 172.21.15.27:0/1187706204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c6c0f5ef0 0x7f6c6c107450 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f6c6000c530 tx=0x7f6c60007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:39:55.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.245+0000 7f6c53fff700 1 -- 172.21.15.27:0/1187706204 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c6000f650 con 0x7f6c6c0f5ef0 2021-11-03T17:39:55.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.245+0000 7f6c721b5700 1 -- 172.21.15.27:0/1187706204 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c6c1084e0 con 0x7f6c6c0f5ef0 2021-11-03T17:39:55.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.245+0000 7f6c53fff700 1 -- 172.21.15.27:0/1187706204 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f6c6000fc60 con 0x7f6c6c0f5ef0 2021-11-03T17:39:55.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.246+0000 7f6c53fff700 1 -- 172.21.15.27:0/1187706204 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c60012c20 con 0x7f6c6c0f5ef0 2021-11-03T17:39:55.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.246+0000 7f6c721b5700 1 -- 172.21.15.27:0/1187706204 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c6c108aa0 con 0x7f6c6c0f5ef0 2021-11-03T17:39:55.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.246+0000 7f6c721b5700 1 -- 172.21.15.27:0/1187706204 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c6c045ad0 con 0x7f6c6c0f5ef0 2021-11-03T17:39:55.248 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.247+0000 7f6c53fff700 1 -- 172.21.15.27:0/1187706204 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f6c60010070 con 0x7f6c6c0f5ef0 2021-11-03T17:39:55.248 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.247+0000 7f6c53fff700 1 --2- 172.21.15.27:0/1187706204 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6c540640b0 0x7f6c54066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:39:55.248 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.247+0000 7f6c53fff700 1 -- 172.21.15.27:0/1187706204 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f6c60080760 con 0x7f6c6c0f5ef0 2021-11-03T17:39:55.248 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.247+0000 7f6c6b7fe700 1 --2- 172.21.15.27:0/1187706204 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6c540640b0 0x7f6c54066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:39:55.248 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.248+0000 7f6c6b7fe700 1 --2- 172.21.15.27:0/1187706204 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6c540640b0 0x7f6c54066570 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f6c5c004630 tx=0x7f6c5c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:39:55.249 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.251+0000 7f6c53fff700 1 -- 172.21.15.27:0/1187706204 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f6c600528b0 con 0x7f6c6c0f5ef0 2021-11-03T17:39:55.476 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.481+0000 7f6c721b5700 1 -- 172.21.15.27:0/1187706204 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6c6c10d580 con 0x7f6c6c0f5ef0 2021-11-03T17:39:55.477 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.481+0000 7f6c53fff700 1 -- 172.21.15.27:0/1187706204 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f6c600524a0 con 0x7f6c6c0f5ef0 2021-11-03T17:39:55.477 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:39:55.477 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:39:55.477 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:39:55.478 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:39:55.478 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:39:55.478 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:39:55.479 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:39:55.479 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:39:55.479 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:39:55.479 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:39:55.479 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:39:55.480 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:39:55.480 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:39:55.480 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:39:55.480 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:39:55.481 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:39:55.481 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:39:55.482 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:39:55.482 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.483+0000 7f6c721b5700 1 -- 172.21.15.27:0/1187706204 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6c540640b0 msgr2=0x7f6c54066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:39:55.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.483+0000 7f6c721b5700 1 --2- 172.21.15.27:0/1187706204 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6c540640b0 0x7f6c54066570 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f6c5c004630 tx=0x7f6c5c009b30).stop 2021-11-03T17:39:55.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.483+0000 7f6c721b5700 1 -- 172.21.15.27:0/1187706204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c6c0f5ef0 msgr2=0x7f6c6c107450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:39:55.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.483+0000 7f6c721b5700 1 --2- 172.21.15.27:0/1187706204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c6c0f5ef0 0x7f6c6c107450 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7f6c6000c530 tx=0x7f6c60007e20).stop 2021-11-03T17:39:55.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.483+0000 7f6c721b5700 1 -- 172.21.15.27:0/1187706204 shutdown_connections 2021-11-03T17:39:55.484 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.483+0000 7f6c721b5700 1 --2- 172.21.15.27:0/1187706204 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6c540640b0 0x7f6c54066570 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:55.484 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.483+0000 7f6c721b5700 1 --2- 172.21.15.27:0/1187706204 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c6c0f55e0 0x7f6c6c106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:55.484 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.483+0000 7f6c721b5700 1 --2- 172.21.15.27:0/1187706204 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c6c0f5ef0 0x7f6c6c107450 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:55.484 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.483+0000 7f6c721b5700 1 -- 172.21.15.27:0/1187706204 >> 172.21.15.27:0/1187706204 conn(0x7f6c6c0f0de0 msgr2=0x7f6c6c05ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:39:55.485 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.483+0000 7f6c721b5700 1 -- 172.21.15.27:0/1187706204 shutdown_connections 2021-11-03T17:39:55.485 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.483+0000 7f6c721b5700 1 -- 172.21.15.27:0/1187706204 wait complete. 2021-11-03T17:39:55.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:55 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:39:55.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:55 smithi115 ceph-mon[65453]: pgmap v758: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:55.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1437.393078 secs 2021-11-03T17:39:55.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:55 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:39:55.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:55 smithi027 ceph-mon[97244]: pgmap v758: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:55.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1437.393078 secs 2021-11-03T17:39:55.581 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.585+0000 7f4f7d834700 1 -- 172.21.15.27:0/3438619754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f78060310 msgr2=0x7f4f780f9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:39:55.581 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.585+0000 7f4f7d834700 1 --2- 172.21.15.27:0/3438619754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f78060310 0x7f4f780f9140 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 rx=0x7f4f68004660 tx=0x7f4f68009b30).stop 2021-11-03T17:39:55.581 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.585+0000 7f4f7d834700 1 -- 172.21.15.27:0/3438619754 shutdown_connections 2021-11-03T17:39:55.582 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.585+0000 7f4f7d834700 1 --2- 172.21.15.27:0/3438619754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f780f96c0 0x7f4f780fbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:39:55.583 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.585+0000 7f4f7d834700 1 --2- 172.21.15.27:0/3438619754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f78060310 0x7f4f780f9140 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:55.583 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.585+0000 7f4f7d834700 1 -- 172.21.15.27:0/3438619754 >> 172.21.15.27:0/3438619754 conn(0x7f4f780f0bc0 msgr2=0x7f4f780f2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:39:55.583 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.586+0000 7f4f7d834700 1 -- 172.21.15.27:0/3438619754 shutdown_connections 2021-11-03T17:39:55.583 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.586+0000 7f4f7d834700 1 -- 172.21.15.27:0/3438619754 wait complete. 2021-11-03T17:39:55.585 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.586+0000 7f4f7d834700 1 Processor -- start 2021-11-03T17:39:55.586 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.586+0000 7f4f7d834700 1 -- start start 2021-11-03T17:39:55.586 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.586+0000 7f4f7d834700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f78060310 0x7f4f78106e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:39:55.586 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.587+0000 7f4f7d834700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f780f96c0 0x7f4f781072f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:39:55.586 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.587+0000 7f4f7d834700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f78107980 con 0x7f4f780f96c0 2021-11-03T17:39:55.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.587+0000 7f4f7d834700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f78108100 con 0x7f4f78060310 2021-11-03T17:39:55.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.587+0000 7f4f767fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f780f96c0 0x7f4f781072f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:39:55.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.587+0000 7f4f767fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f780f96c0 0x7f4f781072f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58726/0 (socket says 172.21.15.27:58726) 2021-11-03T17:39:55.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.587+0000 7f4f767fc700 1 -- 172.21.15.27:0/1751739688 learned_addr learned my addr 172.21.15.27:0/1751739688 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:39:55.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.587+0000 7f4f76ffd700 1 --2- 172.21.15.27:0/1751739688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f78060310 0x7f4f78106e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:39:55.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.587+0000 7f4f767fc700 1 -- 172.21.15.27:0/1751739688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f78060310 msgr2=0x7f4f78106e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:39:55.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.587+0000 7f4f767fc700 1 --2- 172.21.15.27:0/1751739688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f78060310 0x7f4f78106e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:55.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.587+0000 7f4f767fc700 1 -- 172.21.15.27:0/1751739688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f68005040 con 0x7f4f780f96c0 2021-11-03T17:39:55.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.587+0000 7f4f767fc700 1 --2- 172.21.15.27:0/1751739688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f780f96c0 0x7f4f781072f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f4f6c0093f0 tx=0x7f4f6c006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:39:55.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.587+0000 7f4f5f7fe700 1 -- 172.21.15.27:0/1751739688 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f6c013640 con 0x7f4f780f96c0 2021-11-03T17:39:55.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.587+0000 7f4f5f7fe700 1 -- 172.21.15.27:0/1751739688 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f4f6c013c50 con 0x7f4f780f96c0 2021-11-03T17:39:55.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.587+0000 7f4f7d834700 1 -- 172.21.15.27:0/1751739688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f78108380 con 0x7f4f780f96c0 2021-11-03T17:39:55.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.587+0000 7f4f5f7fe700 1 -- 172.21.15.27:0/1751739688 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f6c012c70 con 0x7f4f780f96c0 2021-11-03T17:39:55.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.587+0000 7f4f7d834700 1 -- 172.21.15.27:0/1751739688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f78108940 con 0x7f4f780f96c0 2021-11-03T17:39:55.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.589+0000 7f4f5f7fe700 1 -- 172.21.15.27:0/1751739688 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f4f6c01a060 con 0x7f4f780f96c0 2021-11-03T17:39:55.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.589+0000 7f4f7d834700 1 -- 172.21.15.27:0/1751739688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f78045ad0 con 0x7f4f780f96c0 2021-11-03T17:39:55.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.589+0000 7f4f5f7fe700 1 --2- 172.21.15.27:0/1751739688 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4f60064100 0x7f4f600665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:39:55.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.589+0000 7f4f5f7fe700 1 -- 172.21.15.27:0/1751739688 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f4f6c080870 con 0x7f4f780f96c0 2021-11-03T17:39:55.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.589+0000 7f4f76ffd700 1 --2- 172.21.15.27:0/1751739688 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4f60064100 0x7f4f600665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:39:55.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.590+0000 7f4f76ffd700 1 --2- 172.21.15.27:0/1751739688 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4f60064100 0x7f4f600665c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f4f68004630 tx=0x7f4f68009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:39:55.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.594+0000 7f4f5f7fe700 1 -- 172.21.15.27:0/1751739688 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f4f6c052a80 con 0x7f4f780f96c0 2021-11-03T17:39:55.795 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.800+0000 7f4f7d834700 1 -- 172.21.15.27:0/1751739688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4f78027a00 con 0x7f4f780f96c0 2021-11-03T17:39:55.795 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.800+0000 7f4f5f7fe700 1 -- 172.21.15.27:0/1751739688 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f4f6c018710 con 0x7f4f780f96c0 2021-11-03T17:39:55.795 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:39:55.796 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:39:55.796 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:39:55.796 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:39:55.796 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:39:55.797 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:39:55.797 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:39:55.797 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:39:55.797 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:39:55.798 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:39:55.798 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:39:55.798 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:39:55.798 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:39:55.798 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:39:55.799 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:39:55.799 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:39:55.799 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:39:55.799 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:39:55.800 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:39:55.800 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:39:55.800 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:39:55.800 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:39:55.800 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:39:55.801 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:39:55.801 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:39:55.801 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:39:55.801 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:39:55.802 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:39:55.802 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:39:55.802 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:39:55.802 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:39:55.802 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:39:55.803 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:39:55.803 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:39:55.803 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:39:55.803 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:39:55.803 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:39:55.804 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:39:55.804 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:39:55.805 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.802+0000 7f4f7d834700 1 -- 172.21.15.27:0/1751739688 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4f60064100 msgr2=0x7f4f600665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:39:55.805 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.802+0000 7f4f7d834700 1 --2- 172.21.15.27:0/1751739688 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4f60064100 0x7f4f600665c0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 rx=0x7f4f68004630 tx=0x7f4f68009b30).stop 2021-11-03T17:39:55.805 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.802+0000 7f4f7d834700 1 -- 172.21.15.27:0/1751739688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f780f96c0 msgr2=0x7f4f781072f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:39:55.806 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.802+0000 7f4f7d834700 1 --2- 172.21.15.27:0/1751739688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f780f96c0 0x7f4f781072f0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f4f6c0093f0 tx=0x7f4f6c006ec0).stop 2021-11-03T17:39:55.806 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.802+0000 7f4f7d834700 1 -- 172.21.15.27:0/1751739688 shutdown_connections 2021-11-03T17:39:55.806 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.802+0000 7f4f7d834700 1 --2- 172.21.15.27:0/1751739688 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4f60064100 0x7f4f600665c0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:55.806 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.802+0000 7f4f7d834700 1 --2- 172.21.15.27:0/1751739688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f78060310 0x7f4f78106e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:55.806 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.802+0000 7f4f7d834700 1 --2- 172.21.15.27:0/1751739688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f780f96c0 0x7f4f781072f0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:39:55.807 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.802+0000 7f4f7d834700 1 -- 172.21.15.27:0/1751739688 >> 172.21.15.27:0/1751739688 conn(0x7f4f780f0bc0 msgr2=0x7f4f780f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:39:55.807 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.802+0000 7f4f7d834700 1 -- 172.21.15.27:0/1751739688 shutdown_connections 2021-11-03T17:39:55.807 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:39:55.802+0000 7f4f7d834700 1 -- 172.21.15.27:0/1751739688 wait complete. 2021-11-03T17:39:55.807 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:39:56.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:56 smithi115 ceph-mon[65453]: from='client.34888 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:39:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:56 smithi115 ceph-mon[65453]: from='client.34892 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:39:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:56 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1187706204' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:39:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:39:56.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:56 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1751739688' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:39:56.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:56 smithi027 ceph-mon[97244]: from='client.34888 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:39:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:56 smithi027 ceph-mon[97244]: from='client.34892 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:39:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:56 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1187706204' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:39:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:39:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:39:56.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:56 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1751739688' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:39:57.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:57 smithi115 ceph-mon[65453]: pgmap v759: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:57.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:57 smithi027 ceph-mon[97244]: pgmap v759: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:58.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:58 smithi115 ceph-mon[65453]: pgmap v760: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:58.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:58 smithi027 ceph-mon[97244]: pgmap v760: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:39:59.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:59 smithi115 ceph-mon[65453]: 1 slow requests, 1 included below; oldest blocked for > 1442.393207 secs 2021-11-03T17:39:59.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:39:59 smithi115 ceph-mon[65453]: slow request 1442.393207 seconds old, received at 2021-11-03T17:15:56.956064+0000: client_request(client.24331:21043 rmdir #0x10000003957/db1 2021-11-03T17:15:56.954824+0000 caller_uid=1000, caller_gid=1254{6,36,1000,1254,}) currently cleaned up request 2021-11-03T17:40:00.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:59 smithi027 ceph-mon[97244]: 1 slow requests, 1 included below; oldest blocked for > 1442.393207 secs 2021-11-03T17:40:00.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:39:59 smithi027 ceph-mon[97244]: slow request 1442.393207 seconds old, received at 2021-11-03T17:15:56.956064+0000: client_request(client.24331:21043 rmdir #0x10000003957/db1 2021-11-03T17:15:56.954824+0000 caller_uid=1000, caller_gid=1254{6,36,1000,1254,}) currently cleaned up request 2021-11-03T17:40:00.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:00 smithi115 ceph-mon[65453]: overall HEALTH_WARN 1 filesystem is degraded; 1 filesystem has a failed mds daemon; 1 MDSs report slow requests; 3 daemons have recently crashed 2021-11-03T17:40:00.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:00 smithi115 ceph-mon[65453]: pgmap v761: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:01.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:00 smithi027 ceph-mon[97244]: overall HEALTH_WARN 1 filesystem is degraded; 1 filesystem has a failed mds daemon; 1 MDSs report slow requests; 3 daemons have recently crashed 2021-11-03T17:40:01.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:00 smithi027 ceph-mon[97244]: pgmap v761: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:02.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:02 smithi115 ceph-mon[65453]: pgmap v762: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:02.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:02 smithi027 ceph-mon[97244]: pgmap v762: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:04.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:40:04.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:40:04.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:40:04.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:04.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:04.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:04.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:04.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:40:04.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:40:04.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:40:04.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:04.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:04.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:04.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:05.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:05 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:40:05.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:05 smithi115 ceph-mon[65453]: pgmap v763: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:05.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1447.393272 secs 2021-11-03T17:40:05.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:05 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:40:05.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:05 smithi027 ceph-mon[97244]: pgmap v763: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:05.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1447.393272 secs 2021-11-03T17:40:06.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:06 smithi115 ceph-mon[65453]: pgmap v764: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:06.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:06 smithi027 ceph-mon[97244]: pgmap v764: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:08.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:08 smithi115 ceph-mon[65453]: pgmap v765: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:08.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:08 smithi027 ceph-mon[97244]: pgmap v765: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:09.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:09 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1452.393400 secs 2021-11-03T17:40:10.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:09 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1452.393400 secs 2021-11-03T17:40:11.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:10 smithi027 ceph-mon[97244]: pgmap v766: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:11.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:10 smithi115 ceph-mon[65453]: pgmap v766: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:12.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:12 smithi115 ceph-mon[65453]: pgmap v767: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:12.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:12 smithi027 ceph-mon[97244]: pgmap v767: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:14.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:40:14.419 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:40:15.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:15 smithi115 ceph-mon[65453]: poll took 30096.555 ms: 1 events 2021-11-03T17:40:15.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:15 smithi115 ceph-mon[65453]: pgmap v768: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:15.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1457.393460 secs 2021-11-03T17:40:15.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:40:15.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:40:15.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:15.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:15.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:15.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:15.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:15 smithi027 ceph-mon[97244]: poll took 30096.555 ms: 1 events 2021-11-03T17:40:15.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:15 smithi027 ceph-mon[97244]: pgmap v768: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:15.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1457.393460 secs 2021-11-03T17:40:15.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:40:15.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:40:15.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:15.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:15.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:15.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:16.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:16 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:40:16.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:16 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:40:17.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:17 smithi115 ceph-mon[65453]: pgmap v769: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:17.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:17 smithi027 ceph-mon[97244]: pgmap v769: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:18.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:18 smithi115 ceph-mon[65453]: pgmap v770: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:18.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:18 smithi027 ceph-mon[97244]: pgmap v770: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:19.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:19 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1462.393584 secs 2021-11-03T17:40:20.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:19 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1462.393584 secs 2021-11-03T17:40:21.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:20 smithi027 ceph-mon[97244]: pgmap v771: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:21.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:20 smithi115 ceph-mon[65453]: pgmap v771: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:22.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:22 smithi027 ceph-mon[97244]: pgmap v772: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:22.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:22 smithi115 ceph-mon[65453]: pgmap v772: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:24.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:24 smithi027 ceph-mon[97244]: pgmap v773: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:24.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:24 smithi115 ceph-mon[65453]: pgmap v773: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:25.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1467.393718 secs 2021-11-03T17:40:25.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:40:25.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:40:25.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:40:25.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:25.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:25.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:25.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:25.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:25 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:40:25.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1467.393718 secs 2021-11-03T17:40:25.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:40:25.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:40:25.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:40:25.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:25.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:25.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:25.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:25.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:25 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:40:25.916 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:25.920+0000 7f5b54b2e700 1 -- 172.21.15.27:0/3180825863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b500f6530 msgr2=0x7f5b500f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:40:25.916 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:25.920+0000 7f5b54b2e700 1 --2- 172.21.15.27:0/3180825863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b500f6530 0x7f5b500f6950 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f5b40004660 tx=0x7f5b40009b30).stop 2021-11-03T17:40:25.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:25.920+0000 7f5b54b2e700 1 -- 172.21.15.27:0/3180825863 shutdown_connections 2021-11-03T17:40:25.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:25.920+0000 7f5b54b2e700 1 --2- 172.21.15.27:0/3180825863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b500f77d0 0x7f5b500f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:25.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:25.920+0000 7f5b54b2e700 1 --2- 172.21.15.27:0/3180825863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b500f6530 0x7f5b500f6950 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:25.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:25.920+0000 7f5b54b2e700 1 -- 172.21.15.27:0/3180825863 >> 172.21.15.27:0/3180825863 conn(0x7f5b500f1cd0 msgr2=0x7f5b500f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:40:25.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:25.920+0000 7f5b54b2e700 1 -- 172.21.15.27:0/3180825863 shutdown_connections 2021-11-03T17:40:25.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:25.920+0000 7f5b54b2e700 1 -- 172.21.15.27:0/3180825863 wait complete. 2021-11-03T17:40:25.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:25.920+0000 7f5b54b2e700 1 Processor -- start 2021-11-03T17:40:25.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:25.921+0000 7f5b54b2e700 1 -- start start 2021-11-03T17:40:25.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:25.921+0000 7f5b54b2e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b500f6530 0x7f5b50106db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:40:25.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:25.921+0000 7f5b54b2e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b500f77d0 0x7f5b501072a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:40:25.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:25.921+0000 7f5b54b2e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b501078a0 con 0x7f5b500f6530 2021-11-03T17:40:25.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:25.921+0000 7f5b54b2e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b501080e0 con 0x7f5b500f77d0 2021-11-03T17:40:25.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:25.921+0000 7f5b4f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b500f6530 0x7f5b50106db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:40:25.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:25.921+0000 7f5b4f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b500f6530 0x7f5b50106db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58742/0 (socket says 172.21.15.27:58742) 2021-11-03T17:40:25.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:25.921+0000 7f5b4f7fe700 1 -- 172.21.15.27:0/3751119372 learned_addr learned my addr 172.21.15.27:0/3751119372 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:40:25.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:25.921+0000 7f5b4effd700 1 --2- 172.21.15.27:0/3751119372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b500f77d0 0x7f5b501072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:40:25.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:25.921+0000 7f5b4f7fe700 1 -- 172.21.15.27:0/3751119372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b500f77d0 msgr2=0x7f5b501072a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:40:25.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:25.921+0000 7f5b4f7fe700 1 --2- 172.21.15.27:0/3751119372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b500f77d0 0x7f5b501072a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:25.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:25.921+0000 7f5b4f7fe700 1 -- 172.21.15.27:0/3751119372 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b40005040 con 0x7f5b500f6530 2021-11-03T17:40:25.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:25.921+0000 7f5b4effd700 1 --2- 172.21.15.27:0/3751119372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b500f77d0 0x7f5b501072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:40:25.922 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:25.921+0000 7f5b4f7fe700 1 --2- 172.21.15.27:0/3751119372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b500f6530 0x7f5b50106db0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f5b40013a70 tx=0x7f5b40009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:40:25.922 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:25.922+0000 7f5b37fff700 1 -- 172.21.15.27:0/3751119372 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b40018070 con 0x7f5b500f6530 2021-11-03T17:40:25.922 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:25.922+0000 7f5b37fff700 1 -- 172.21.15.27:0/3751119372 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f5b400097d0 con 0x7f5b500f6530 2021-11-03T17:40:25.922 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:25.922+0000 7f5b37fff700 1 -- 172.21.15.27:0/3751119372 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b4001c3c0 con 0x7f5b500f6530 2021-11-03T17:40:25.922 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:25.922+0000 7f5b54b2e700 1 -- 172.21.15.27:0/3751119372 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b50108360 con 0x7f5b500f6530 2021-11-03T17:40:25.923 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:25.922+0000 7f5b54b2e700 1 -- 172.21.15.27:0/3751119372 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b50108840 con 0x7f5b500f6530 2021-11-03T17:40:25.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:25.923+0000 7f5b54b2e700 1 -- 172.21.15.27:0/3751119372 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b50059100 con 0x7f5b500f6530 2021-11-03T17:40:25.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:25.923+0000 7f5b37fff700 1 -- 172.21.15.27:0/3751119372 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f5b40007ba0 con 0x7f5b500f6530 2021-11-03T17:40:25.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:25.924+0000 7f5b37fff700 1 --2- 172.21.15.27:0/3751119372 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5b38064100 0x7f5b380665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:40:25.925 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:25.924+0000 7f5b37fff700 1 -- 172.21.15.27:0/3751119372 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f5b40082400 con 0x7f5b500f6530 2021-11-03T17:40:25.925 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:25.924+0000 7f5b4effd700 1 --2- 172.21.15.27:0/3751119372 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5b38064100 0x7f5b380665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:40:25.925 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:25.924+0000 7f5b4effd700 1 --2- 172.21.15.27:0/3751119372 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5b38064100 0x7f5b380665c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f5b44009fd0 tx=0x7f5b4400b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:40:25.925 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:25.928+0000 7f5b37fff700 1 -- 172.21.15.27:0/3751119372 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f5b400547c0 con 0x7f5b500f6530 2021-11-03T17:40:26.102 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.108+0000 7f5b54b2e700 1 -- 172.21.15.27:0/3751119372 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b500fedd0 con 0x7f5b38064100 2021-11-03T17:40:26.103 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.109+0000 7f5b37fff700 1 -- 172.21.15.27:0/3751119372 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f5b500fedd0 con 0x7f5b38064100 2021-11-03T17:40:26.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.111+0000 7f5b54b2e700 1 -- 172.21.15.27:0/3751119372 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5b38064100 msgr2=0x7f5b380665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:40:26.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.111+0000 7f5b54b2e700 1 --2- 172.21.15.27:0/3751119372 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5b38064100 0x7f5b380665c0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f5b44009fd0 tx=0x7f5b4400b040).stop 2021-11-03T17:40:26.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.111+0000 7f5b54b2e700 1 -- 172.21.15.27:0/3751119372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b500f6530 msgr2=0x7f5b50106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:40:26.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.111+0000 7f5b54b2e700 1 --2- 172.21.15.27:0/3751119372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b500f6530 0x7f5b50106db0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f5b40013a70 tx=0x7f5b40009b30).stop 2021-11-03T17:40:26.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.111+0000 7f5b54b2e700 1 -- 172.21.15.27:0/3751119372 shutdown_connections 2021-11-03T17:40:26.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.111+0000 7f5b54b2e700 1 --2- 172.21.15.27:0/3751119372 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5b38064100 0x7f5b380665c0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:26.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.111+0000 7f5b54b2e700 1 --2- 172.21.15.27:0/3751119372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5b500f6530 0x7f5b50106db0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:26.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.111+0000 7f5b54b2e700 1 --2- 172.21.15.27:0/3751119372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5b500f77d0 0x7f5b501072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:26.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.111+0000 7f5b54b2e700 1 -- 172.21.15.27:0/3751119372 >> 172.21.15.27:0/3751119372 conn(0x7f5b500f1cd0 msgr2=0x7f5b500fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:40:26.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.111+0000 7f5b54b2e700 1 -- 172.21.15.27:0/3751119372 shutdown_connections 2021-11-03T17:40:26.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.111+0000 7f5b54b2e700 1 -- 172.21.15.27:0/3751119372 wait complete. 2021-11-03T17:40:26.115 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:40:26.206 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.212+0000 7f5905c22700 1 -- 172.21.15.27:0/1596105525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59000f55e0 msgr2=0x7f59000f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:40:26.207 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.212+0000 7f5905c22700 1 --2- 172.21.15.27:0/1596105525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59000f55e0 0x7f59000f5a00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f58f0004660 tx=0x7f58f0009b30).stop 2021-11-03T17:40:26.207 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.212+0000 7f5905c22700 1 -- 172.21.15.27:0/1596105525 shutdown_connections 2021-11-03T17:40:26.207 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.212+0000 7f5905c22700 1 --2- 172.21.15.27:0/1596105525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59000f5ef0 0x7f59000f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:26.207 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.212+0000 7f5905c22700 1 --2- 172.21.15.27:0/1596105525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59000f55e0 0x7f59000f5a00 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:26.208 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.212+0000 7f5905c22700 1 -- 172.21.15.27:0/1596105525 >> 172.21.15.27:0/1596105525 conn(0x7f59000f0de0 msgr2=0x7f59000f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:40:26.208 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.212+0000 7f5905c22700 1 -- 172.21.15.27:0/1596105525 shutdown_connections 2021-11-03T17:40:26.208 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.212+0000 7f5905c22700 1 -- 172.21.15.27:0/1596105525 wait complete. 2021-11-03T17:40:26.208 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.213+0000 7f5905c22700 1 Processor -- start 2021-11-03T17:40:26.211 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.213+0000 7f5905c22700 1 -- start start 2021-11-03T17:40:26.211 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.213+0000 7f5905c22700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59000f55e0 0x7f5900109cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:40:26.211 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.213+0000 7f5905c22700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59000f5ef0 0x7f590010c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:40:26.211 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.213+0000 7f5905c22700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59000604e0 con 0x7f59000f55e0 2021-11-03T17:40:26.211 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.213+0000 7f5905c22700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5900060620 con 0x7f59000f5ef0 2021-11-03T17:40:26.212 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.213+0000 7f58ff7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59000f55e0 0x7f5900109cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:40:26.212 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.213+0000 7f58ff7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59000f55e0 0x7f5900109cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58752/0 (socket says 172.21.15.27:58752) 2021-11-03T17:40:26.212 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.213+0000 7f58ff7fe700 1 -- 172.21.15.27:0/1496519191 learned_addr learned my addr 172.21.15.27:0/1496519191 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:40:26.212 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.213+0000 7f58feffd700 1 --2- 172.21.15.27:0/1496519191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59000f5ef0 0x7f590010c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:40:26.213 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.213+0000 7f58ff7fe700 1 -- 172.21.15.27:0/1496519191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59000f5ef0 msgr2=0x7f590010c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:40:26.213 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.213+0000 7f58ff7fe700 1 --2- 172.21.15.27:0/1496519191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59000f5ef0 0x7f590010c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:26.213 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.213+0000 7f58ff7fe700 1 -- 172.21.15.27:0/1496519191 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f0005040 con 0x7f59000f55e0 2021-11-03T17:40:26.213 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.214+0000 7f58ff7fe700 1 --2- 172.21.15.27:0/1496519191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59000f55e0 0x7f5900109cb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f58f0009990 tx=0x7f58f0009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:40:26.214 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.214+0000 7f58e7fff700 1 -- 172.21.15.27:0/1496519191 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58f0018070 con 0x7f59000f55e0 2021-11-03T17:40:26.214 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.214+0000 7f5905c22700 1 -- 172.21.15.27:0/1496519191 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f590010c700 con 0x7f59000f55e0 2021-11-03T17:40:26.214 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.214+0000 7f58e7fff700 1 -- 172.21.15.27:0/1496519191 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f58f0007ba0 con 0x7f59000f55e0 2021-11-03T17:40:26.214 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.214+0000 7f58e7fff700 1 -- 172.21.15.27:0/1496519191 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58f001c3c0 con 0x7f59000f55e0 2021-11-03T17:40:26.214 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.214+0000 7f5905c22700 1 -- 172.21.15.27:0/1496519191 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f590010cc30 con 0x7f59000f55e0 2021-11-03T17:40:26.215 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.215+0000 7f5905c22700 1 -- 172.21.15.27:0/1496519191 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59000f4a70 con 0x7f59000f55e0 2021-11-03T17:40:26.215 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.215+0000 7f58e7fff700 1 -- 172.21.15.27:0/1496519191 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f58f001c680 con 0x7f59000f55e0 2021-11-03T17:40:26.215 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.216+0000 7f58e7fff700 1 --2- 172.21.15.27:0/1496519191 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f58e80640b0 0x7f58e8066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:40:26.215 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.216+0000 7f58e7fff700 1 -- 172.21.15.27:0/1496519191 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f58f00825a0 con 0x7f59000f55e0 2021-11-03T17:40:26.216 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.216+0000 7f58feffd700 1 --2- 172.21.15.27:0/1496519191 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f58e80640b0 0x7f58e8066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:40:26.216 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.216+0000 7f58feffd700 1 --2- 172.21.15.27:0/1496519191 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f58e80640b0 0x7f58e8066570 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f58f40096c0 tx=0x7f58f4009be0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:40:26.216 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.220+0000 7f58e7fff700 1 -- 172.21.15.27:0/1496519191 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f58f00549f0 con 0x7f59000f55e0 2021-11-03T17:40:26.389 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.395+0000 7f5905c22700 1 -- 172.21.15.27:0/1496519191 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f59000fdf10 con 0x7f58e80640b0 2021-11-03T17:40:26.404 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.410+0000 7f58e7fff700 1 -- 172.21.15.27:0/1496519191 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f59000fdf10 con 0x7f58e80640b0 2021-11-03T17:40:26.404 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:40:26.405 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (32m) 106s ago 33m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:40:26.405 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (23m) 106s ago 33m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:40:26.405 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (23m) 42s ago 32m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:40:26.405 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (31m) 106s ago 32m 41.3M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:40:26.406 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (25m) 106s ago 28m 21.1M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:40:26.406 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (28m) 106s ago 28m 3843M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:40:26.406 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (24m) 42s ago 28m 20.4M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:40:26.406 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (25m) 42s ago 28m 22.1M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:40:26.406 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (25m) 106s ago 34m 517M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:40:26.407 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (25m) 42s ago 32m 419M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:40:26.407 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (24m) 106s ago 34m 159M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:40:26.407 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (24m) 42s ago 32m 82.5M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:40:26.407 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (32m) 106s ago 32m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:40:26.408 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (32m) 42s ago 32m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:40:26.408 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (23m) 106s ago 31m 311M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:40:26.408 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (22m) 106s ago 31m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:40:26.408 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (22m) 106s ago 30m 248M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:40:26.408 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (21m) 42s ago 30m 354M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:40:26.409 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (21m) 42s ago 30m 285M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:40:26.409 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (20m) 42s ago 29m 287M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:40:26.409 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (31m) 106s ago 32m 50.5M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:40:26.410 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.412+0000 7f5905c22700 1 -- 172.21.15.27:0/1496519191 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f58e80640b0 msgr2=0x7f58e8066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:40:26.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.412+0000 7f5905c22700 1 --2- 172.21.15.27:0/1496519191 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f58e80640b0 0x7f58e8066570 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f58f40096c0 tx=0x7f58f4009be0).stop 2021-11-03T17:40:26.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.412+0000 7f5905c22700 1 -- 172.21.15.27:0/1496519191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59000f55e0 msgr2=0x7f5900109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:40:26.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.412+0000 7f5905c22700 1 --2- 172.21.15.27:0/1496519191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59000f55e0 0x7f5900109cb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7f58f0009990 tx=0x7f58f0009b30).stop 2021-11-03T17:40:26.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.412+0000 7f5905c22700 1 -- 172.21.15.27:0/1496519191 shutdown_connections 2021-11-03T17:40:26.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.412+0000 7f5905c22700 1 --2- 172.21.15.27:0/1496519191 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f58e80640b0 0x7f58e8066570 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:26.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.412+0000 7f5905c22700 1 --2- 172.21.15.27:0/1496519191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59000f55e0 0x7f5900109cb0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:26.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.412+0000 7f5905c22700 1 --2- 172.21.15.27:0/1496519191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59000f5ef0 0x7f590010c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:26.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.412+0000 7f5905c22700 1 -- 172.21.15.27:0/1496519191 >> 172.21.15.27:0/1496519191 conn(0x7f59000f0de0 msgr2=0x7f59000fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:40:26.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.412+0000 7f5905c22700 1 -- 172.21.15.27:0/1496519191 shutdown_connections 2021-11-03T17:40:26.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.412+0000 7f5905c22700 1 -- 172.21.15.27:0/1496519191 wait complete. 2021-11-03T17:40:26.506 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.512+0000 7fdaad0a9700 1 -- 172.21.15.27:0/518486022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaa80f7980 msgr2=0x7fdaa80f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:40:26.506 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.512+0000 7fdaad0a9700 1 --2- 172.21.15.27:0/518486022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaa80f7980 0x7fdaa80f7de0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7fda90004660 tx=0x7fda90009b30).stop 2021-11-03T17:40:26.507 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.512+0000 7fdaad0a9700 1 -- 172.21.15.27:0/518486022 shutdown_connections 2021-11-03T17:40:26.507 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.512+0000 7fdaad0a9700 1 --2- 172.21.15.27:0/518486022 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaa80f7980 0x7fdaa80f7de0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:26.507 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.512+0000 7fdaad0a9700 1 --2- 172.21.15.27:0/518486022 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdaa80f66e0 0x7fdaa80f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:26.507 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.512+0000 7fdaad0a9700 1 -- 172.21.15.27:0/518486022 >> 172.21.15.27:0/518486022 conn(0x7fdaa80f1e40 msgr2=0x7fdaa80f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:40:26.507 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.512+0000 7fdaad0a9700 1 -- 172.21.15.27:0/518486022 shutdown_connections 2021-11-03T17:40:26.508 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.512+0000 7fdaad0a9700 1 -- 172.21.15.27:0/518486022 wait complete. 2021-11-03T17:40:26.508 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.512+0000 7fdaad0a9700 1 Processor -- start 2021-11-03T17:40:26.508 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.512+0000 7fdaad0a9700 1 -- start start 2021-11-03T17:40:26.510 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.513+0000 7fdaad0a9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdaa80f66e0 0x7fdaa8106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:40:26.510 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.513+0000 7fdaad0a9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaa80f7980 0x7fdaa8107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:40:26.511 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.513+0000 7fdaad0a9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa8107a00 con 0x7fdaa80f7980 2021-11-03T17:40:26.511 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.513+0000 7fdaad0a9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa8107b40 con 0x7fdaa80f66e0 2021-11-03T17:40:26.511 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.513+0000 7fdaa659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaa80f7980 0x7fdaa8107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:40:26.511 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.513+0000 7fdaa659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaa80f7980 0x7fdaa8107450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58764/0 (socket says 172.21.15.27:58764) 2021-11-03T17:40:26.512 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.513+0000 7fdaa659c700 1 -- 172.21.15.27:0/3107665353 learned_addr learned my addr 172.21.15.27:0/3107665353 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:40:26.512 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.513+0000 7fdaa6d9d700 1 --2- 172.21.15.27:0/3107665353 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdaa80f66e0 0x7fdaa8106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:40:26.512 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.513+0000 7fdaa659c700 1 -- 172.21.15.27:0/3107665353 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdaa80f66e0 msgr2=0x7fdaa8106f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:40:26.512 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.513+0000 7fdaa659c700 1 --2- 172.21.15.27:0/3107665353 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdaa80f66e0 0x7fdaa8106f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:26.513 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.513+0000 7fdaa659c700 1 -- 172.21.15.27:0/3107665353 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda90005040 con 0x7fdaa80f7980 2021-11-03T17:40:26.513 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.513+0000 7fdaa659c700 1 --2- 172.21.15.27:0/3107665353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaa80f7980 0x7fdaa8107450 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7fda90009990 tx=0x7fda90009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:40:26.513 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.513+0000 7fda9f7fe700 1 -- 172.21.15.27:0/3107665353 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda90018070 con 0x7fdaa80f7980 2021-11-03T17:40:26.513 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.513+0000 7fdaad0a9700 1 -- 172.21.15.27:0/3107665353 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaa8108480 con 0x7fdaa80f7980 2021-11-03T17:40:26.513 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.513+0000 7fda9f7fe700 1 -- 172.21.15.27:0/3107665353 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fda90007ca0 con 0x7fdaa80f7980 2021-11-03T17:40:26.514 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.513+0000 7fda9f7fe700 1 -- 172.21.15.27:0/3107665353 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda9001c620 con 0x7fdaa80f7980 2021-11-03T17:40:26.514 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.513+0000 7fdaad0a9700 1 -- 172.21.15.27:0/3107665353 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa81089e0 con 0x7fdaa80f7980 2021-11-03T17:40:26.514 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.514+0000 7fdaad0a9700 1 -- 172.21.15.27:0/3107665353 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaa8045ad0 con 0x7fdaa80f7980 2021-11-03T17:40:26.514 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.515+0000 7fda9f7fe700 1 -- 172.21.15.27:0/3107665353 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fda9001d960 con 0x7fdaa80f7980 2021-11-03T17:40:26.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.515+0000 7fda9f7fe700 1 --2- 172.21.15.27:0/3107665353 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fda940640b0 0x7fda94066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:40:26.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.515+0000 7fda9f7fe700 1 -- 172.21.15.27:0/3107665353 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fda900824a0 con 0x7fdaa80f7980 2021-11-03T17:40:26.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.515+0000 7fdaa6d9d700 1 --2- 172.21.15.27:0/3107665353 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fda940640b0 0x7fda94066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:40:26.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.515+0000 7fdaa6d9d700 1 --2- 172.21.15.27:0/3107665353 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fda940640b0 0x7fda94066570 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7fdaa0004fd0 tx=0x7fdaa0009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:40:26.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.519+0000 7fda9f7fe700 1 -- 172.21.15.27:0/3107665353 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fda900547a0 con 0x7fdaa80f7980 2021-11-03T17:40:26.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:26 smithi027 ceph-mon[97244]: from='client.34904 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:40:26.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:26 smithi027 ceph-mon[97244]: pgmap v774: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:26.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:26 smithi115 ceph-mon[65453]: from='client.34904 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:40:26.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:26 smithi115 ceph-mon[65453]: pgmap v774: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:26.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.749+0000 7fdaad0a9700 1 -- 172.21.15.27:0/3107665353 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdaa8027a00 con 0x7fdaa80f7980 2021-11-03T17:40:26.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.749+0000 7fda9f7fe700 1 -- 172.21.15.27:0/3107665353 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fda90054390 con 0x7fdaa80f7980 2021-11-03T17:40:26.744 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:40:26.744 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:40:26.744 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:40:26.744 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:40:26.745 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:40:26.745 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:40:26.745 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:40:26.745 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:40:26.745 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:40:26.746 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:40:26.746 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:40:26.746 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:40:26.746 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:40:26.747 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:40:26.747 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:40:26.747 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:40:26.747 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:40:26.748 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:40:26.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.751+0000 7fdaad0a9700 1 -- 172.21.15.27:0/3107665353 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fda940640b0 msgr2=0x7fda94066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:40:26.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.751+0000 7fdaad0a9700 1 --2- 172.21.15.27:0/3107665353 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fda940640b0 0x7fda94066570 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 rx=0x7fdaa0004fd0 tx=0x7fdaa0009040).stop 2021-11-03T17:40:26.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.751+0000 7fdaad0a9700 1 -- 172.21.15.27:0/3107665353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaa80f7980 msgr2=0x7fdaa8107450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:40:26.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.751+0000 7fdaad0a9700 1 --2- 172.21.15.27:0/3107665353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaa80f7980 0x7fdaa8107450 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7fda90009990 tx=0x7fda90009b30).stop 2021-11-03T17:40:26.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.751+0000 7fdaad0a9700 1 -- 172.21.15.27:0/3107665353 shutdown_connections 2021-11-03T17:40:26.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.751+0000 7fdaad0a9700 1 --2- 172.21.15.27:0/3107665353 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fda940640b0 0x7fda94066570 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:26.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.751+0000 7fdaad0a9700 1 --2- 172.21.15.27:0/3107665353 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdaa80f66e0 0x7fdaa8106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:26.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.751+0000 7fdaad0a9700 1 --2- 172.21.15.27:0/3107665353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdaa80f7980 0x7fdaa8107450 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:26.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.751+0000 7fdaad0a9700 1 -- 172.21.15.27:0/3107665353 >> 172.21.15.27:0/3107665353 conn(0x7fdaa80f1e40 msgr2=0x7fdaa80fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:40:26.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.751+0000 7fdaad0a9700 1 -- 172.21.15.27:0/3107665353 shutdown_connections 2021-11-03T17:40:26.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.751+0000 7fdaad0a9700 1 -- 172.21.15.27:0/3107665353 wait complete. 2021-11-03T17:40:26.847 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.851+0000 7f2a11fe1700 1 -- 172.21.15.27:0/4204232932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a0c0f55e0 msgr2=0x7f2a0c0f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:40:26.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.851+0000 7f2a11fe1700 1 --2- 172.21.15.27:0/4204232932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a0c0f55e0 0x7f2a0c0f5a00 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f29fc004660 tx=0x7f29fc009b30).stop 2021-11-03T17:40:26.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.851+0000 7f2a11fe1700 1 -- 172.21.15.27:0/4204232932 shutdown_connections 2021-11-03T17:40:26.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.851+0000 7f2a11fe1700 1 --2- 172.21.15.27:0/4204232932 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a0c0f5ef0 0x7f2a0c0f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:40:26.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.851+0000 7f2a11fe1700 1 --2- 172.21.15.27:0/4204232932 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a0c0f55e0 0x7f2a0c0f5a00 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:26.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.851+0000 7f2a11fe1700 1 -- 172.21.15.27:0/4204232932 >> 172.21.15.27:0/4204232932 conn(0x7f2a0c0f0de0 msgr2=0x7f2a0c0f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:40:26.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.851+0000 7f2a11fe1700 1 -- 172.21.15.27:0/4204232932 shutdown_connections 2021-11-03T17:40:26.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.851+0000 7f2a11fe1700 1 -- 172.21.15.27:0/4204232932 wait complete. 2021-11-03T17:40:26.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.852+0000 7f2a11fe1700 1 Processor -- start 2021-11-03T17:40:26.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.852+0000 7f2a11fe1700 1 -- start start 2021-11-03T17:40:26.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.852+0000 7f2a11fe1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a0c0f55e0 0x7f2a0c109c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:40:26.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.852+0000 7f2a11fe1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a0c0f5ef0 0x7f2a0c10c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:40:26.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.852+0000 7f2a11fe1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c0605d0 con 0x7f2a0c0f5ef0 2021-11-03T17:40:26.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.852+0000 7f2a11fe1700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a0c060710 con 0x7f2a0c0f55e0 2021-11-03T17:40:26.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.852+0000 7f2a0affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a0c0f5ef0 0x7f2a0c10c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:40:26.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.852+0000 7f2a0affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a0c0f5ef0 0x7f2a0c10c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58774/0 (socket says 172.21.15.27:58774) 2021-11-03T17:40:26.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.852+0000 7f2a0affd700 1 -- 172.21.15.27:0/213814984 learned_addr learned my addr 172.21.15.27:0/213814984 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:40:26.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.852+0000 7f2a0affd700 1 -- 172.21.15.27:0/213814984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a0c0f55e0 msgr2=0x7f2a0c109c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:40:26.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.852+0000 7f2a0affd700 1 --2- 172.21.15.27:0/213814984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a0c0f55e0 0x7f2a0c109c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:40:26.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.852+0000 7f2a0affd700 1 -- 172.21.15.27:0/213814984 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29fc005040 con 0x7f2a0c0f5ef0 2021-11-03T17:40:26.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.853+0000 7f2a0affd700 1 --2- 172.21.15.27:0/213814984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a0c0f5ef0 0x7f2a0c10c130 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f29f400b410 tx=0x7f29f4006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:40:26.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.853+0000 7f2a03fff700 1 -- 172.21.15.27:0/213814984 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29f40125f0 con 0x7f2a0c0f5ef0 2021-11-03T17:40:26.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.853+0000 7f2a11fe1700 1 -- 172.21.15.27:0/213814984 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a0c10c790 con 0x7f2a0c0f5ef0 2021-11-03T17:40:26.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.853+0000 7f2a03fff700 1 -- 172.21.15.27:0/213814984 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f29f4012c00 con 0x7f2a0c0f5ef0 2021-11-03T17:40:26.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.853+0000 7f2a03fff700 1 -- 172.21.15.27:0/213814984 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29f4002ae0 con 0x7f2a0c0f5ef0 2021-11-03T17:40:26.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.853+0000 7f2a11fe1700 1 -- 172.21.15.27:0/213814984 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a0c10cd50 con 0x7f2a0c0f5ef0 2021-11-03T17:40:26.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.854+0000 7f2a11fe1700 1 -- 172.21.15.27:0/213814984 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a0c045ad0 con 0x7f2a0c0f5ef0 2021-11-03T17:40:26.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.854+0000 7f2a03fff700 1 -- 172.21.15.27:0/213814984 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f29f400f030 con 0x7f2a0c0f5ef0 2021-11-03T17:40:26.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.855+0000 7f2a03fff700 1 --2- 172.21.15.27:0/213814984 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f29f80640b0 0x7f29f8066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:40:26.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.855+0000 7f2a03fff700 1 -- 172.21.15.27:0/213814984 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f29f4080d10 con 0x7f2a0c0f5ef0 2021-11-03T17:40:26.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.855+0000 7f2a0b7fe700 1 --2- 172.21.15.27:0/213814984 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f29f80640b0 0x7f29f8066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:40:26.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.855+0000 7f2a0b7fe700 1 --2- 172.21.15.27:0/213814984 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f29f80640b0 0x7f29f8066570 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f29fc004e30 tx=0x7f29fc009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:40:26.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:26.859+0000 7f2a03fff700 1 -- 172.21.15.27:0/213814984 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f29f4053010 con 0x7f2a0c0f5ef0 2021-11-03T17:40:27.055 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:27.061+0000 7f2a11fe1700 1 -- 172.21.15.27:0/213814984 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2a0c0fdfe0 con 0x7f2a0c0f5ef0 2021-11-03T17:40:27.056 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:27.062+0000 7f2a03fff700 1 -- 172.21.15.27:0/213814984 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f2a0c0fdfe0 con 0x7f2a0c0f5ef0 2021-11-03T17:40:27.056 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:40:27.056 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:40:27.057 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:40:27.057 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:40:27.057 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:40:27.057 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:40:27.058 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:40:27.058 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:40:27.058 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:40:27.058 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:40:27.059 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:40:27.059 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:40:27.059 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:40:27.059 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:40:27.060 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:40:27.060 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:40:27.060 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:40:27.060 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:40:27.061 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:40:27.061 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:40:27.061 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:40:27.061 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:40:27.061 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:40:27.062 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:40:27.062 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:40:27.062 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:40:27.062 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:40:27.063 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:40:27.063 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:40:27.063 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:40:27.063 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:40:27.064 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:40:27.064 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:40:27.064 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:40:27.064 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:40:27.065 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:40:27.065 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:40:27.065 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:40:27.065 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:40:27.066 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:27.063+0000 7f2a11fe1700 1 -- 172.21.15.27:0/213814984 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f29f80640b0 msgr2=0x7f29f8066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:40:27.066 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:27.063+0000 7f2a11fe1700 1 --2- 172.21.15.27:0/213814984 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f29f80640b0 0x7f29f8066570 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f29fc004e30 tx=0x7f29fc009b30).stop 2021-11-03T17:40:27.067 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:27.063+0000 7f2a11fe1700 1 -- 172.21.15.27:0/213814984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a0c0f5ef0 msgr2=0x7f2a0c10c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:40:27.067 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:27.063+0000 7f2a11fe1700 1 --2- 172.21.15.27:0/213814984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a0c0f5ef0 0x7f2a0c10c130 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f29f400b410 tx=0x7f29f4006ef0).stop 2021-11-03T17:40:27.067 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:27.063+0000 7f2a11fe1700 1 -- 172.21.15.27:0/213814984 shutdown_connections 2021-11-03T17:40:27.067 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:27.063+0000 7f2a11fe1700 1 --2- 172.21.15.27:0/213814984 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f29f80640b0 0x7f29f8066570 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:27.068 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:27.063+0000 7f2a11fe1700 1 --2- 172.21.15.27:0/213814984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a0c0f55e0 0x7f2a0c109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:40:27.068 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:27.063+0000 7f2a11fe1700 1 --2- 172.21.15.27:0/213814984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a0c0f5ef0 0x7f2a0c10c130 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:27.068 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:27.063+0000 7f2a11fe1700 1 -- 172.21.15.27:0/213814984 >> 172.21.15.27:0/213814984 conn(0x7f2a0c0f0de0 msgr2=0x7f2a0c0fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:40:27.068 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:27.064+0000 7f2a11fe1700 1 -- 172.21.15.27:0/213814984 shutdown_connections 2021-11-03T17:40:27.069 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:27.064+0000 7f2a11fe1700 1 -- 172.21.15.27:0/213814984 wait complete. 2021-11-03T17:40:27.069 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:40:27.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:27 smithi027 ceph-mon[97244]: from='client.34908 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:40:27.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:27 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3107665353' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:27.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:27 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/213814984' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:40:27.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:27 smithi115 ceph-mon[65453]: from='client.34908 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:40:27.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:27 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3107665353' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:27.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:27 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/213814984' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:40:28.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:28 smithi027 ceph-mon[97244]: pgmap v775: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:28.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:28 smithi115 ceph-mon[65453]: pgmap v775: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:29.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:29 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1472.393839 secs 2021-11-03T17:40:30.017 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:29 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1472.393839 secs 2021-11-03T17:40:31.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:30 smithi027 ceph-mon[97244]: pgmap v776: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:31.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:30 smithi115 ceph-mon[65453]: pgmap v776: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:32.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:32 smithi027 ceph-mon[97244]: pgmap v777: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:32.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:32 smithi115 ceph-mon[65453]: pgmap v777: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:34.506 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:34 smithi115 ceph-mon[65453]: pgmap v778: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:34.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:34 smithi027 ceph-mon[97244]: pgmap v778: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:35.355 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1477.393999 secs 2021-11-03T17:40:35.355 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:40:35.355 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:35 smithi115 ceph-mon[65453]: poll took 20055.791 ms: 1 events 2021-11-03T17:40:35.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1477.393999 secs 2021-11-03T17:40:35.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:40:35.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:35 smithi027 ceph-mon[97244]: poll took 20055.791 ms: 1 events 2021-11-03T17:40:36.436 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:36 smithi115 ceph-mon[65453]: pgmap v779: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:36.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:36 smithi027 ceph-mon[97244]: pgmap v779: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:38.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:38 smithi115 ceph-mon[65453]: poll took 2771.550 ms: 1 events 2021-11-03T17:40:38.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:38 smithi115 ceph-mon[65453]: Detected new or changed devices on smithi115 2021-11-03T17:40:38.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:40:38.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:40:38.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:40:38.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:38.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:38.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:38.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:38.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:38 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:40:38.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:38 smithi115 ceph-mon[65453]: pgmap v780: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:39.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:38 smithi027 ceph-mon[97244]: poll took 2771.550 ms: 1 events 2021-11-03T17:40:39.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:38 smithi027 ceph-mon[97244]: Detected new or changed devices on smithi115 2021-11-03T17:40:39.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:40:39.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:40:39.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:40:39.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:39.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:39.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:39.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:39.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:38 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:40:39.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:38 smithi027 ceph-mon[97244]: pgmap v780: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:39.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:39 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1482.394118 secs 2021-11-03T17:40:40.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:39 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1482.394118 secs 2021-11-03T17:40:41.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:40 smithi027 ceph-mon[97244]: pgmap v781: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:41.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:40 smithi115 ceph-mon[65453]: pgmap v781: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:42.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:42 smithi027 ceph-mon[97244]: pgmap v782: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:42.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:42 smithi115 ceph-mon[65453]: pgmap v782: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:44.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:44 smithi027 ceph-mon[97244]: pgmap v783: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:44.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:44 smithi115 ceph-mon[65453]: pgmap v783: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:45.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1487.394263 secs 2021-11-03T17:40:45.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1487.394263 secs 2021-11-03T17:40:46.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:46 smithi027 ceph-mon[97244]: pgmap v784: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:46.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:46 smithi115 ceph-mon[65453]: pgmap v784: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:47.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:40:47.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:40:47.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:40:47.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:47.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:47.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:47.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:48.017 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:40:48.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:40:48.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:40:48.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:48.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:48.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:48.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:48.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:48 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:40:48.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:48 smithi115 ceph-mon[65453]: pgmap v785: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:49.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:48 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:40:49.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:48 smithi027 ceph-mon[97244]: pgmap v785: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:49.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:49 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1492.394401 secs 2021-11-03T17:40:50.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:49 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1492.394401 secs 2021-11-03T17:40:51.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:50 smithi027 ceph-mon[97244]: pgmap v786: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:51.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:50 smithi115 ceph-mon[65453]: pgmap v786: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:52.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:52 smithi027 ceph-mon[97244]: pgmap v787: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:52.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:52 smithi115 ceph-mon[65453]: pgmap v787: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:54.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:54 smithi027 ceph-mon[97244]: pgmap v788: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:54.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:54 smithi115 ceph-mon[65453]: pgmap v788: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:55.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1497.394529 secs 2021-11-03T17:40:55.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1497.394529 secs 2021-11-03T17:40:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:40:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:40:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:56 smithi027 ceph-mon[97244]: pgmap v789: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:56.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:40:56.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:40:56.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:56 smithi115 ceph-mon[65453]: pgmap v789: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:57.173 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.178+0000 7f22557b6700 1 -- 172.21.15.27:0/3398218997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22500f6ff0 msgr2=0x7f22500f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:40:57.173 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.178+0000 7f22557b6700 1 --2- 172.21.15.27:0/3398218997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22500f6ff0 0x7f22500f7a20 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f2244004660 tx=0x7f2244009b30).stop 2021-11-03T17:40:57.173 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.178+0000 7f22557b6700 1 -- 172.21.15.27:0/3398218997 shutdown_connections 2021-11-03T17:40:57.173 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.178+0000 7f22557b6700 1 --2- 172.21.15.27:0/3398218997 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22500f6ff0 0x7f22500f7a20 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:57.174 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.178+0000 7f22557b6700 1 --2- 172.21.15.27:0/3398218997 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22500f66e0 0x7f22500f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:57.174 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.178+0000 7f22557b6700 1 -- 172.21.15.27:0/3398218997 >> 172.21.15.27:0/3398218997 conn(0x7f22500f1f00 msgr2=0x7f22500f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:40:57.174 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.179+0000 7f22557b6700 1 -- 172.21.15.27:0/3398218997 shutdown_connections 2021-11-03T17:40:57.174 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.179+0000 7f22557b6700 1 -- 172.21.15.27:0/3398218997 wait complete. 2021-11-03T17:40:57.177 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.179+0000 7f22557b6700 1 Processor -- start 2021-11-03T17:40:57.177 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.179+0000 7f22557b6700 1 -- start start 2021-11-03T17:40:57.177 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.179+0000 7f22557b6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22500f66e0 0x7f2250106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:40:57.178 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.179+0000 7f22557b6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22500f6ff0 0x7f2250107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:40:57.178 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.180+0000 7f22557b6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2250107a10 con 0x7f22500f6ff0 2021-11-03T17:40:57.178 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.180+0000 7f22557b6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2250107b50 con 0x7f22500f66e0 2021-11-03T17:40:57.178 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.180+0000 7f224e7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22500f6ff0 0x7f2250107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:40:57.179 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.180+0000 7f224e7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22500f6ff0 0x7f2250107460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58792/0 (socket says 172.21.15.27:58792) 2021-11-03T17:40:57.179 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.180+0000 7f224e7fc700 1 -- 172.21.15.27:0/155961168 learned_addr learned my addr 172.21.15.27:0/155961168 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:40:57.179 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.180+0000 7f224effd700 1 --2- 172.21.15.27:0/155961168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22500f66e0 0x7f2250106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:40:57.179 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.180+0000 7f224e7fc700 1 -- 172.21.15.27:0/155961168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22500f66e0 msgr2=0x7f2250106f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:40:57.180 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.180+0000 7f224e7fc700 1 --2- 172.21.15.27:0/155961168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22500f66e0 0x7f2250106f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:57.180 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.180+0000 7f224e7fc700 1 -- 172.21.15.27:0/155961168 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2244005040 con 0x7f22500f6ff0 2021-11-03T17:40:57.180 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.180+0000 7f224e7fc700 1 --2- 172.21.15.27:0/155961168 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22500f6ff0 0x7f2250107460 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f2244009990 tx=0x7f2244009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:40:57.180 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.180+0000 7f22377fe700 1 -- 172.21.15.27:0/155961168 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2244018070 con 0x7f22500f6ff0 2021-11-03T17:40:57.181 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.180+0000 7f22557b6700 1 -- 172.21.15.27:0/155961168 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2250108490 con 0x7f22500f6ff0 2021-11-03T17:40:57.181 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.180+0000 7f22377fe700 1 -- 172.21.15.27:0/155961168 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f2244007ba0 con 0x7f22500f6ff0 2021-11-03T17:40:57.181 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.180+0000 7f22377fe700 1 -- 172.21.15.27:0/155961168 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f224401c3c0 con 0x7f22500f6ff0 2021-11-03T17:40:57.181 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.180+0000 7f22557b6700 1 -- 172.21.15.27:0/155961168 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22501089f0 con 0x7f22500f6ff0 2021-11-03T17:40:57.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.181+0000 7f22557b6700 1 -- 172.21.15.27:0/155961168 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2250045ad0 con 0x7f22500f6ff0 2021-11-03T17:40:57.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.182+0000 7f22377fe700 1 -- 172.21.15.27:0/155961168 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f224401d420 con 0x7f22500f6ff0 2021-11-03T17:40:57.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.182+0000 7f22377fe700 1 --2- 172.21.15.27:0/155961168 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f22380640b0 0x7f2238066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:40:57.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.182+0000 7f22377fe700 1 -- 172.21.15.27:0/155961168 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f2244082530 con 0x7f22500f6ff0 2021-11-03T17:40:57.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.182+0000 7f224effd700 1 --2- 172.21.15.27:0/155961168 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f22380640b0 0x7f2238066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:40:57.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.183+0000 7f224effd700 1 --2- 172.21.15.27:0/155961168 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f22380640b0 0x7f2238066570 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f2240004c40 tx=0x7f224000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:40:57.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.186+0000 7f22377fe700 1 -- 172.21.15.27:0/155961168 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f2244054710 con 0x7f22500f6ff0 2021-11-03T17:40:57.358 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.365+0000 7f22557b6700 1 -- 172.21.15.27:0/155961168 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f22500fefe0 con 0x7f22380640b0 2021-11-03T17:40:57.360 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.366+0000 7f22377fe700 1 -- 172.21.15.27:0/155961168 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f22500fefe0 con 0x7f22380640b0 2021-11-03T17:40:57.362 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.368+0000 7f22557b6700 1 -- 172.21.15.27:0/155961168 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f22380640b0 msgr2=0x7f2238066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:40:57.362 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.368+0000 7f22557b6700 1 --2- 172.21.15.27:0/155961168 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f22380640b0 0x7f2238066570 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f2240004c40 tx=0x7f224000b040).stop 2021-11-03T17:40:57.363 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.368+0000 7f22557b6700 1 -- 172.21.15.27:0/155961168 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22500f6ff0 msgr2=0x7f2250107460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:40:57.363 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.368+0000 7f22557b6700 1 --2- 172.21.15.27:0/155961168 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22500f6ff0 0x7f2250107460 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7f2244009990 tx=0x7f2244009b30).stop 2021-11-03T17:40:57.363 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.368+0000 7f22557b6700 1 -- 172.21.15.27:0/155961168 shutdown_connections 2021-11-03T17:40:57.363 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.368+0000 7f22557b6700 1 --2- 172.21.15.27:0/155961168 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f22380640b0 0x7f2238066570 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:57.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.368+0000 7f22557b6700 1 --2- 172.21.15.27:0/155961168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22500f66e0 0x7f2250106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:57.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.368+0000 7f22557b6700 1 --2- 172.21.15.27:0/155961168 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22500f6ff0 0x7f2250107460 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:57.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.368+0000 7f22557b6700 1 -- 172.21.15.27:0/155961168 >> 172.21.15.27:0/155961168 conn(0x7f22500f1f00 msgr2=0x7f22500fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:40:57.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.368+0000 7f22557b6700 1 -- 172.21.15.27:0/155961168 shutdown_connections 2021-11-03T17:40:57.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.368+0000 7f22557b6700 1 -- 172.21.15.27:0/155961168 wait complete. 2021-11-03T17:40:57.372 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:40:57.462 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.468+0000 7fa6fadd3700 1 -- 172.21.15.27:0/45324139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6f40f6ff0 msgr2=0x7fa6f40f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:40:57.462 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.468+0000 7fa6fadd3700 1 --2- 172.21.15.27:0/45324139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6f40f6ff0 0x7fa6f40f7a20 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7fa6e0004660 tx=0x7fa6e0009b30).stop 2021-11-03T17:40:57.463 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.468+0000 7fa6fadd3700 1 -- 172.21.15.27:0/45324139 shutdown_connections 2021-11-03T17:40:57.463 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.468+0000 7fa6fadd3700 1 --2- 172.21.15.27:0/45324139 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6f40f6ff0 0x7fa6f40f7a20 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:57.463 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.468+0000 7fa6fadd3700 1 --2- 172.21.15.27:0/45324139 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6f40f66e0 0x7fa6f40f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:57.463 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.468+0000 7fa6fadd3700 1 -- 172.21.15.27:0/45324139 >> 172.21.15.27:0/45324139 conn(0x7fa6f40f1f00 msgr2=0x7fa6f40f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:40:57.463 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.468+0000 7fa6fadd3700 1 -- 172.21.15.27:0/45324139 shutdown_connections 2021-11-03T17:40:57.464 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.468+0000 7fa6fadd3700 1 -- 172.21.15.27:0/45324139 wait complete. 2021-11-03T17:40:57.466 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.469+0000 7fa6fadd3700 1 Processor -- start 2021-11-03T17:40:57.466 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.469+0000 7fa6fadd3700 1 -- start start 2021-11-03T17:40:57.466 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.469+0000 7fa6fadd3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6f40f66e0 0x7fa6f4106ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:40:57.467 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.469+0000 7fa6fadd3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6f40f6ff0 0x7fa6f41073e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:40:57.467 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.469+0000 7fa6fadd3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f4107990 con 0x7fa6f40f66e0 2021-11-03T17:40:57.467 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.469+0000 7fa6fadd3700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f4107ad0 con 0x7fa6f40f6ff0 2021-11-03T17:40:57.467 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.469+0000 7fa6f8b6f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6f40f66e0 0x7fa6f4106ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:40:57.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.469+0000 7fa6f8b6f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6f40f66e0 0x7fa6f4106ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58800/0 (socket says 172.21.15.27:58800) 2021-11-03T17:40:57.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.469+0000 7fa6f8b6f700 1 -- 172.21.15.27:0/3976838158 learned_addr learned my addr 172.21.15.27:0/3976838158 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:40:57.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.469+0000 7fa6f8b6f700 1 -- 172.21.15.27:0/3976838158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6f40f6ff0 msgr2=0x7fa6f41073e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:40:57.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.469+0000 7fa6f8b6f700 1 --2- 172.21.15.27:0/3976838158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6f40f6ff0 0x7fa6f41073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:40:57.469 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.469+0000 7fa6f8b6f700 1 -- 172.21.15.27:0/3976838158 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e0005040 con 0x7fa6f40f66e0 2021-11-03T17:40:57.469 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.469+0000 7fa6f8b6f700 1 --2- 172.21.15.27:0/3976838158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6f40f66e0 0x7fa6f4106ef0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7fa6e8002700 tx=0x7fa6e8009e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:40:57.469 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.470+0000 7fa6f17fa700 1 -- 172.21.15.27:0/3976838158 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6e8009320 con 0x7fa6f40f66e0 2021-11-03T17:40:57.469 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.470+0000 7fa6f17fa700 1 -- 172.21.15.27:0/3976838158 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fa6e800f040 con 0x7fa6f40f66e0 2021-11-03T17:40:57.469 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.470+0000 7fa6fadd3700 1 -- 172.21.15.27:0/3976838158 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f4108470 con 0x7fa6f40f66e0 2021-11-03T17:40:57.470 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.470+0000 7fa6f17fa700 1 -- 172.21.15.27:0/3976838158 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6e8014820 con 0x7fa6f40f66e0 2021-11-03T17:40:57.470 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.470+0000 7fa6fadd3700 1 -- 172.21.15.27:0/3976838158 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f4108a30 con 0x7fa6f40f66e0 2021-11-03T17:40:57.470 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.471+0000 7fa6fadd3700 1 -- 172.21.15.27:0/3976838158 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6f4045ad0 con 0x7fa6f40f66e0 2021-11-03T17:40:57.470 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.471+0000 7fa6f17fa700 1 -- 172.21.15.27:0/3976838158 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fa6e8009480 con 0x7fa6f40f66e0 2021-11-03T17:40:57.471 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.471+0000 7fa6f17fa700 1 --2- 172.21.15.27:0/3976838158 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa6e40640b0 0x7fa6e4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:40:57.471 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.472+0000 7fa6f17fa700 1 -- 172.21.15.27:0/3976838158 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fa6e8080a90 con 0x7fa6f40f66e0 2021-11-03T17:40:57.472 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.472+0000 7fa6f3fff700 1 --2- 172.21.15.27:0/3976838158 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa6e40640b0 0x7fa6e4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:40:57.472 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.472+0000 7fa6f3fff700 1 --2- 172.21.15.27:0/3976838158 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa6e40640b0 0x7fa6e4066570 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7fa6e0004e30 tx=0x7fa6e0009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:40:57.473 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.476+0000 7fa6f17fa700 1 -- 172.21.15.27:0/3976838158 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fa6e8052d90 con 0x7fa6f40f66e0 2021-11-03T17:40:57.641 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.647+0000 7fa6fadd3700 1 -- 172.21.15.27:0/3976838158 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6f4108d00 con 0x7fa6e40640b0 2021-11-03T17:40:57.656 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.662+0000 7fa6f17fa700 1 -- 172.21.15.27:0/3976838158 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fa6f4108d00 con 0x7fa6e40640b0 2021-11-03T17:40:57.656 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:40:57.657 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (32m) 2m ago 33m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:40:57.657 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (24m) 2m ago 33m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:40:57.657 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (24m) 74s ago 33m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:40:57.658 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (32m) 2m ago 33m 41.3M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:40:57.658 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (26m) 2m ago 28m 21.1M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:40:57.659 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (28m) 2m ago 28m 3843M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:40:57.659 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (24m) 74s ago 28m 20.4M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:40:57.659 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (26m) 74s ago 29m 22.1M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:40:57.660 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (26m) 2m ago 35m 517M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:40:57.660 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (25m) 74s ago 32m 419M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:40:57.661 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (25m) 2m ago 35m 159M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:40:57.661 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (24m) 74s ago 32m 82.5M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:40:57.661 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (33m) 2m ago 33m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:40:57.662 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (32m) 74s ago 32m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:40:57.662 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (23m) 2m ago 32m 311M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:40:57.663 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (23m) 2m ago 31m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:40:57.663 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (22m) 2m ago 31m 248M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:40:57.663 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (22m) 74s ago 31m 354M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:40:57.664 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (21m) 74s ago 30m 285M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:40:57.664 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (21m) 74s ago 30m 287M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:40:57.664 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (32m) 2m ago 33m 50.5M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:40:57.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.664+0000 7fa6fadd3700 1 -- 172.21.15.27:0/3976838158 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa6e40640b0 msgr2=0x7fa6e4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:40:57.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.664+0000 7fa6fadd3700 1 --2- 172.21.15.27:0/3976838158 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa6e40640b0 0x7fa6e4066570 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7fa6e0004e30 tx=0x7fa6e0009b30).stop 2021-11-03T17:40:57.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.664+0000 7fa6fadd3700 1 -- 172.21.15.27:0/3976838158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6f40f66e0 msgr2=0x7fa6f4106ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:40:57.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.664+0000 7fa6fadd3700 1 --2- 172.21.15.27:0/3976838158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6f40f66e0 0x7fa6f4106ef0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7fa6e8002700 tx=0x7fa6e8009e00).stop 2021-11-03T17:40:57.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.664+0000 7fa6fadd3700 1 -- 172.21.15.27:0/3976838158 shutdown_connections 2021-11-03T17:40:57.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.664+0000 7fa6fadd3700 1 --2- 172.21.15.27:0/3976838158 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa6e40640b0 0x7fa6e4066570 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:57.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.664+0000 7fa6fadd3700 1 --2- 172.21.15.27:0/3976838158 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6f40f66e0 0x7fa6f4106ef0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:57.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.664+0000 7fa6fadd3700 1 --2- 172.21.15.27:0/3976838158 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6f40f6ff0 0x7fa6f41073e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:40:57.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.664+0000 7fa6fadd3700 1 -- 172.21.15.27:0/3976838158 >> 172.21.15.27:0/3976838158 conn(0x7fa6f40f1f00 msgr2=0x7fa6f40fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:40:57.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.664+0000 7fa6fadd3700 1 -- 172.21.15.27:0/3976838158 shutdown_connections 2021-11-03T17:40:57.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.664+0000 7fa6fadd3700 1 -- 172.21.15.27:0/3976838158 wait complete. 2021-11-03T17:40:57.764 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.770+0000 7f85ba541700 1 -- 172.21.15.27:0/3902039938 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85b40f5340 msgr2=0x7f85b40f7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:40:57.765 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.770+0000 7f85ba541700 1 --2- 172.21.15.27:0/3902039938 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85b40f5340 0x7f85b40f7ac0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f85a8004660 tx=0x7f85a8009b30).stop 2021-11-03T17:40:57.765 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.770+0000 7f85ba541700 1 -- 172.21.15.27:0/3902039938 shutdown_connections 2021-11-03T17:40:57.765 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.770+0000 7f85ba541700 1 --2- 172.21.15.27:0/3902039938 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85b40f5340 0x7f85b40f7ac0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:57.765 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.770+0000 7f85ba541700 1 --2- 172.21.15.27:0/3902039938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85b40f49a0 0x7f85b40f4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:57.766 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.770+0000 7f85ba541700 1 -- 172.21.15.27:0/3902039938 >> 172.21.15.27:0/3902039938 conn(0x7f85b40f04c0 msgr2=0x7f85b40f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:40:57.766 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.770+0000 7f85ba541700 1 -- 172.21.15.27:0/3902039938 shutdown_connections 2021-11-03T17:40:57.766 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.770+0000 7f85ba541700 1 -- 172.21.15.27:0/3902039938 wait complete. 2021-11-03T17:40:57.766 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.771+0000 7f85ba541700 1 Processor -- start 2021-11-03T17:40:57.767 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.771+0000 7f85ba541700 1 -- start start 2021-11-03T17:40:57.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.771+0000 7f85ba541700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85b40f49a0 0x7f85b4106fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:40:57.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.771+0000 7f85ba541700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85b40f5340 0x7f85b41074c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:40:57.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.771+0000 7f85ba541700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b4107a70 con 0x7f85b40f49a0 2021-11-03T17:40:57.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.771+0000 7f85ba541700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b4107bb0 con 0x7f85b40f5340 2021-11-03T17:40:57.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.771+0000 7f85b3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85b40f49a0 0x7f85b4106fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:40:57.770 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.771+0000 7f85b3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85b40f49a0 0x7f85b4106fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58810/0 (socket says 172.21.15.27:58810) 2021-11-03T17:40:57.770 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.771+0000 7f85b3fff700 1 -- 172.21.15.27:0/3276061254 learned_addr learned my addr 172.21.15.27:0/3276061254 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:40:57.770 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.771+0000 7f85b37fe700 1 --2- 172.21.15.27:0/3276061254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85b40f5340 0x7f85b41074c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:40:57.770 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.771+0000 7f85b3fff700 1 -- 172.21.15.27:0/3276061254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85b40f5340 msgr2=0x7f85b41074c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:40:57.771 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.771+0000 7f85b3fff700 1 --2- 172.21.15.27:0/3276061254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85b40f5340 0x7f85b41074c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:57.771 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.771+0000 7f85b3fff700 1 -- 172.21.15.27:0/3276061254 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a8005040 con 0x7f85b40f49a0 2021-11-03T17:40:57.771 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.772+0000 7f85b3fff700 1 --2- 172.21.15.27:0/3276061254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85b40f49a0 0x7f85b4106fd0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f85a400c530 tx=0x7f85a4007e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:40:57.771 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.772+0000 7f85b0ff9700 1 -- 172.21.15.27:0/3276061254 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85a400f650 con 0x7f85b40f49a0 2021-11-03T17:40:57.771 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.772+0000 7f85b0ff9700 1 -- 172.21.15.27:0/3276061254 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f85a400fc60 con 0x7f85b40f49a0 2021-11-03T17:40:57.772 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.772+0000 7f85b0ff9700 1 -- 172.21.15.27:0/3276061254 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f85a4012d20 con 0x7f85b40f49a0 2021-11-03T17:40:57.772 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.772+0000 7f85ba541700 1 -- 172.21.15.27:0/3276061254 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85b4108550 con 0x7f85b40f49a0 2021-11-03T17:40:57.772 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.772+0000 7f85ba541700 1 -- 172.21.15.27:0/3276061254 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b4108b10 con 0x7f85b40f49a0 2021-11-03T17:40:57.772 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.773+0000 7f85ba541700 1 -- 172.21.15.27:0/3276061254 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85b4045ad0 con 0x7f85b40f49a0 2021-11-03T17:40:57.773 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.773+0000 7f85b0ff9700 1 -- 172.21.15.27:0/3276061254 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f85a4019030 con 0x7f85b40f49a0 2021-11-03T17:40:57.773 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.773+0000 7f85b0ff9700 1 --2- 172.21.15.27:0/3276061254 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f859c0640b0 0x7f859c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:40:57.773 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.774+0000 7f85b0ff9700 1 -- 172.21.15.27:0/3276061254 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f85a4080680 con 0x7f85b40f49a0 2021-11-03T17:40:57.773 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.774+0000 7f85b37fe700 1 --2- 172.21.15.27:0/3276061254 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f859c0640b0 0x7f859c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:40:57.774 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.774+0000 7f85b37fe700 1 --2- 172.21.15.27:0/3276061254 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f859c0640b0 0x7f859c066570 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f85a8004e30 tx=0x7f85a8009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:40:57.774 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:57.778+0000 7f85b0ff9700 1 -- 172.21.15.27:0/3276061254 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f85a40529a0 con 0x7f85b40f49a0 2021-11-03T17:40:57.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:40:57.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:40:57.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:40:57.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:57.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:57.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:57.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:58.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.007+0000 7f85ba541700 1 -- 172.21.15.27:0/3276061254 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f85b40ff040 con 0x7f85b40f49a0 2021-11-03T17:40:58.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.008+0000 7f85b0ff9700 1 -- 172.21.15.27:0/3276061254 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f85a4052590 con 0x7f85b40f49a0 2021-11-03T17:40:58.002 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:40:58.003 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:40:58.003 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:40:58.003 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:40:58.003 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:40:58.004 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:40:58.004 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:40:58.004 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:40:58.004 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:40:58.004 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:40:58.005 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:40:58.005 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:40:58.005 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:40:58.005 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:40:58.006 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:40:58.006 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:40:58.006 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:40:58.006 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:40:58.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.010+0000 7f85ba541700 1 -- 172.21.15.27:0/3276061254 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f859c0640b0 msgr2=0x7f859c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:40:58.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.010+0000 7f85ba541700 1 --2- 172.21.15.27:0/3276061254 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f859c0640b0 0x7f859c066570 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f85a8004e30 tx=0x7f85a8009b30).stop 2021-11-03T17:40:58.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.010+0000 7f85ba541700 1 -- 172.21.15.27:0/3276061254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85b40f49a0 msgr2=0x7f85b4106fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:40:58.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.010+0000 7f85ba541700 1 --2- 172.21.15.27:0/3276061254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85b40f49a0 0x7f85b4106fd0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f85a400c530 tx=0x7f85a4007e10).stop 2021-11-03T17:40:58.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.010+0000 7f85ba541700 1 -- 172.21.15.27:0/3276061254 shutdown_connections 2021-11-03T17:40:58.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.010+0000 7f85ba541700 1 --2- 172.21.15.27:0/3276061254 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f859c0640b0 0x7f859c066570 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:58.009 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.010+0000 7f85ba541700 1 --2- 172.21.15.27:0/3276061254 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f85b40f49a0 0x7f85b4106fd0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:58.009 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.010+0000 7f85ba541700 1 --2- 172.21.15.27:0/3276061254 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f85b40f5340 0x7f85b41074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:58.009 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.010+0000 7f85ba541700 1 -- 172.21.15.27:0/3276061254 >> 172.21.15.27:0/3276061254 conn(0x7f85b40f04c0 msgr2=0x7f85b40fb510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:40:58.009 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.010+0000 7f85ba541700 1 -- 172.21.15.27:0/3276061254 shutdown_connections 2021-11-03T17:40:58.009 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.010+0000 7f85ba541700 1 -- 172.21.15.27:0/3276061254 wait complete. 2021-11-03T17:40:58.017 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:40:58.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:40:58.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:40:58.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:58.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:58.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:58.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:58.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.110+0000 7f07f95db700 1 -- 172.21.15.27:0/3640986720 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07f40f77d0 msgr2=0x7f07f40f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:40:58.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.110+0000 7f07f95db700 1 --2- 172.21.15.27:0/3640986720 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07f40f77d0 0x7f07f40f7c30 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f07e8004660 tx=0x7f07e8009b30).stop 2021-11-03T17:40:58.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.110+0000 7f07f95db700 1 -- 172.21.15.27:0/3640986720 shutdown_connections 2021-11-03T17:40:58.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.110+0000 7f07f95db700 1 --2- 172.21.15.27:0/3640986720 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07f40f77d0 0x7f07f40f7c30 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:58.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.110+0000 7f07f95db700 1 --2- 172.21.15.27:0/3640986720 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07f40f6530 0x7f07f40f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:58.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.110+0000 7f07f95db700 1 -- 172.21.15.27:0/3640986720 >> 172.21.15.27:0/3640986720 conn(0x7f07f40f1cd0 msgr2=0x7f07f40f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:40:58.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.110+0000 7f07f95db700 1 -- 172.21.15.27:0/3640986720 shutdown_connections 2021-11-03T17:40:58.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.110+0000 7f07f95db700 1 -- 172.21.15.27:0/3640986720 wait complete. 2021-11-03T17:40:58.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.111+0000 7f07f95db700 1 Processor -- start 2021-11-03T17:40:58.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.111+0000 7f07f95db700 1 -- start start 2021-11-03T17:40:58.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.111+0000 7f07f95db700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07f40f6530 0x7f07f406b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:40:58.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.111+0000 7f07f95db700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07f40f77d0 0x7f07f4069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:40:58.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.111+0000 7f07f95db700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f406a380 con 0x7f07f40f77d0 2021-11-03T17:40:58.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.111+0000 7f07f95db700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f406a4c0 con 0x7f07f40f6530 2021-11-03T17:40:58.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.111+0000 7f07f37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07f40f77d0 0x7f07f4069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:40:58.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.111+0000 7f07f37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07f40f77d0 0x7f07f4069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58822/0 (socket says 172.21.15.27:58822) 2021-11-03T17:40:58.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.111+0000 7f07f37fe700 1 -- 172.21.15.27:0/2876248122 learned_addr learned my addr 172.21.15.27:0/2876248122 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:40:58.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.112+0000 7f07f3fff700 1 --2- 172.21.15.27:0/2876248122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07f40f6530 0x7f07f406b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:40:58.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.112+0000 7f07f37fe700 1 -- 172.21.15.27:0/2876248122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07f40f6530 msgr2=0x7f07f406b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:40:58.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.112+0000 7f07f37fe700 1 --2- 172.21.15.27:0/2876248122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07f40f6530 0x7f07f406b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:58.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.112+0000 7f07f37fe700 1 -- 172.21.15.27:0/2876248122 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e8005040 con 0x7f07f40f77d0 2021-11-03T17:40:58.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.112+0000 7f07f37fe700 1 --2- 172.21.15.27:0/2876248122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07f40f77d0 0x7f07f4069e20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f07e8009990 tx=0x7f07e8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:40:58.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.112+0000 7f07f0ff9700 1 -- 172.21.15.27:0/2876248122 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07e8018070 con 0x7f07f40f77d0 2021-11-03T17:40:58.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.112+0000 7f07f95db700 1 -- 172.21.15.27:0/2876248122 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07f406a740 con 0x7f07f40f77d0 2021-11-03T17:40:58.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.112+0000 7f07f0ff9700 1 -- 172.21.15.27:0/2876248122 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f07e8007ca0 con 0x7f07f40f77d0 2021-11-03T17:40:58.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.112+0000 7f07f0ff9700 1 -- 172.21.15.27:0/2876248122 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07e801c620 con 0x7f07f40f77d0 2021-11-03T17:40:58.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.112+0000 7f07f95db700 1 -- 172.21.15.27:0/2876248122 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07f406aca0 con 0x7f07f40f77d0 2021-11-03T17:40:58.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.113+0000 7f07f95db700 1 -- 172.21.15.27:0/2876248122 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07f4045ad0 con 0x7f07f40f77d0 2021-11-03T17:40:58.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.113+0000 7f07f0ff9700 1 -- 172.21.15.27:0/2876248122 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f07e801d960 con 0x7f07f40f77d0 2021-11-03T17:40:58.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.114+0000 7f07f0ff9700 1 --2- 172.21.15.27:0/2876248122 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f07dc064100 0x7f07dc0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:40:58.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.114+0000 7f07f0ff9700 1 -- 172.21.15.27:0/2876248122 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f07e8082580 con 0x7f07f40f77d0 2021-11-03T17:40:58.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.114+0000 7f07f3fff700 1 --2- 172.21.15.27:0/2876248122 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f07dc064100 0x7f07dc0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:40:58.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.114+0000 7f07f3fff700 1 --2- 172.21.15.27:0/2876248122 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f07dc064100 0x7f07dc0665c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f07f406b730 tx=0x7f07e4009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:40:58.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.118+0000 7f07f0ff9700 1 -- 172.21.15.27:0/2876248122 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f07e8054940 con 0x7f07f40f77d0 2021-11-03T17:40:58.324 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.330+0000 7f07f95db700 1 -- 172.21.15.27:0/2876248122 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f07f4027a00 con 0x7f07f40f77d0 2021-11-03T17:40:58.324 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.330+0000 7f07f0ff9700 1 -- 172.21.15.27:0/2876248122 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f07e8054530 con 0x7f07f40f77d0 2021-11-03T17:40:58.325 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:40:58.325 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:40:58.325 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:40:58.326 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:40:58.326 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:40:58.326 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:40:58.326 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:40:58.326 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:40:58.327 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:40:58.327 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:40:58.327 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:40:58.327 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:40:58.328 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:40:58.328 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:40:58.328 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:40:58.328 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:40:58.329 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:40:58.329 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:40:58.329 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:40:58.329 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:40:58.329 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:40:58.330 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:40:58.330 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:40:58.330 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:40:58.330 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:40:58.331 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:40:58.331 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:40:58.331 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:40:58.331 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:40:58.331 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:40:58.332 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:40:58.332 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:40:58.332 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:40:58.332 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:40:58.333 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:40:58.333 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:40:58.333 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:40:58.333 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:40:58.334 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:40:58.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.332+0000 7f07f95db700 1 -- 172.21.15.27:0/2876248122 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f07dc064100 msgr2=0x7f07dc0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:40:58.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.332+0000 7f07f95db700 1 --2- 172.21.15.27:0/2876248122 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f07dc064100 0x7f07dc0665c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f07f406b730 tx=0x7f07e4009040).stop 2021-11-03T17:40:58.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.332+0000 7f07f95db700 1 -- 172.21.15.27:0/2876248122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07f40f77d0 msgr2=0x7f07f4069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:40:58.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.332+0000 7f07f95db700 1 --2- 172.21.15.27:0/2876248122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07f40f77d0 0x7f07f4069e20 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 rx=0x7f07e8009990 tx=0x7f07e8009b30).stop 2021-11-03T17:40:58.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.332+0000 7f07f95db700 1 -- 172.21.15.27:0/2876248122 shutdown_connections 2021-11-03T17:40:58.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.332+0000 7f07f95db700 1 --2- 172.21.15.27:0/2876248122 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f07dc064100 0x7f07dc0665c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:58.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.332+0000 7f07f95db700 1 --2- 172.21.15.27:0/2876248122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07f40f6530 0x7f07f406b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:58.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.332+0000 7f07f95db700 1 --2- 172.21.15.27:0/2876248122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07f40f77d0 0x7f07f4069e20 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:40:58.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.332+0000 7f07f95db700 1 -- 172.21.15.27:0/2876248122 >> 172.21.15.27:0/2876248122 conn(0x7f07f40f1cd0 msgr2=0x7f07f40fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:40:58.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.332+0000 7f07f95db700 1 -- 172.21.15.27:0/2876248122 shutdown_connections 2021-11-03T17:40:58.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:40:58.332+0000 7f07f95db700 1 -- 172.21.15.27:0/2876248122 wait complete. 2021-11-03T17:40:58.337 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:40:58.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:58 smithi115 ceph-mon[65453]: from='client.34920 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:40:58.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:58 smithi115 ceph-mon[65453]: from='client.34924 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:40:58.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:58 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:40:58.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:58 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3276061254' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:58.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:58 smithi115 ceph-mon[65453]: pgmap v790: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:58.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:58 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2876248122' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:40:59.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:58 smithi027 ceph-mon[97244]: from='client.34920 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:40:59.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:58 smithi027 ceph-mon[97244]: from='client.34924 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:40:59.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:58 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:40:59.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:58 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3276061254' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:40:59.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:58 smithi027 ceph-mon[97244]: pgmap v790: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:40:59.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:58 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2876248122' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:40:59.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:40:59 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1502.394673 secs 2021-11-03T17:41:00.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:40:59 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1502.394673 secs 2021-11-03T17:41:01.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:00 smithi027 ceph-mon[97244]: pgmap v791: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:01.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:00 smithi115 ceph-mon[65453]: pgmap v791: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:02.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:02 smithi027 ceph-mon[97244]: pgmap v792: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:02.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:02 smithi115 ceph-mon[65453]: pgmap v792: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:04.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:04 smithi027 ceph-mon[97244]: pgmap v793: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:04.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:04 smithi115 ceph-mon[65453]: pgmap v793: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:05.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1507.394767 secs 2021-11-03T17:41:05.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1507.394767 secs 2021-11-03T17:41:06.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:06 smithi027 ceph-mon[97244]: pgmap v794: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:06.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:06 smithi115 ceph-mon[65453]: pgmap v794: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:08.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:41:08.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:41:08.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:41:08.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:08.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:08.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:08.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:08.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:41:08.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:41:08.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:41:08.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:08.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:08.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:08.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:09.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:08 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:41:09.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:08 smithi027 ceph-mon[97244]: pgmap v795: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:09.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:08 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:41:09.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:08 smithi115 ceph-mon[65453]: pgmap v795: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:10.017 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:09 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1512.394907 secs 2021-11-03T17:41:10.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:09 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1512.394907 secs 2021-11-03T17:41:11.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:10 smithi027 ceph-mon[97244]: pgmap v796: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:11.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:10 smithi115 ceph-mon[65453]: pgmap v796: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:12.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:12 smithi027 ceph-mon[97244]: pgmap v797: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:12.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:12 smithi115 ceph-mon[65453]: pgmap v797: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:14.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:14 smithi027 ceph-mon[97244]: pgmap v798: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:14.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:14 smithi115 ceph-mon[65453]: pgmap v798: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:15.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1517.395038 secs 2021-11-03T17:41:15.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1517.395038 secs 2021-11-03T17:41:16.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:16 smithi027 ceph-mon[97244]: pgmap v799: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:16.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:16 smithi115 ceph-mon[65453]: pgmap v799: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:18.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:41:18.073 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:41:19.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:19 smithi115 ceph-mon[65453]: poll took 40124.792 ms: 1 events 2021-11-03T17:41:19.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:41:19.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:41:19.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:19.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:19.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:19.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:19.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:19 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:41:19.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:19 smithi115 ceph-mon[65453]: pgmap v800: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:19.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:19 smithi027 ceph-mon[97244]: poll took 40124.792 ms: 1 events 2021-11-03T17:41:19.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:41:19.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:41:19.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:19.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:19.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:19.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:19.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:19 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:41:19.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:19 smithi027 ceph-mon[97244]: pgmap v800: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:20.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1522.395154 secs 2021-11-03T17:41:20.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1522.395154 secs 2021-11-03T17:41:21.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:21 smithi115 ceph-mon[65453]: pgmap v801: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:21.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:21 smithi027 ceph-mon[97244]: pgmap v801: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:22.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:22 smithi027 ceph-mon[97244]: pgmap v802: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:22.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:22 smithi115 ceph-mon[65453]: pgmap v802: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:24.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:24 smithi027 ceph-mon[97244]: pgmap v803: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:24.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:24 smithi115 ceph-mon[65453]: pgmap v803: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:25.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1527.395298 secs 2021-11-03T17:41:25.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1527.395298 secs 2021-11-03T17:41:26.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:26 smithi027 ceph-mon[97244]: pgmap v804: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:26.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:26 smithi115 ceph-mon[65453]: pgmap v804: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:28.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.447+0000 7f1635ce6700 1 -- 172.21.15.27:0/894914368 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16300f55c0 msgr2=0x7f16300f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:41:28.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.447+0000 7f1635ce6700 1 --2- 172.21.15.27:0/894914368 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16300f55c0 0x7f16300f59e0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f1620004660 tx=0x7f1620009b30).stop 2021-11-03T17:41:28.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.447+0000 7f1635ce6700 1 -- 172.21.15.27:0/894914368 shutdown_connections 2021-11-03T17:41:28.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.447+0000 7f1635ce6700 1 --2- 172.21.15.27:0/894914368 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16300f5ed0 0x7f16300f6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:41:28.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.447+0000 7f1635ce6700 1 --2- 172.21.15.27:0/894914368 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16300f55c0 0x7f16300f59e0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:41:28.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.447+0000 7f1635ce6700 1 -- 172.21.15.27:0/894914368 >> 172.21.15.27:0/894914368 conn(0x7f16300f0de0 msgr2=0x7f16300f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:41:28.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.447+0000 7f1635ce6700 1 -- 172.21.15.27:0/894914368 shutdown_connections 2021-11-03T17:41:28.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.447+0000 7f1635ce6700 1 -- 172.21.15.27:0/894914368 wait complete. 2021-11-03T17:41:28.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.447+0000 7f1635ce6700 1 Processor -- start 2021-11-03T17:41:28.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.448+0000 7f1635ce6700 1 -- start start 2021-11-03T17:41:28.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.448+0000 7f1635ce6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16300f55c0 0x7f1630109c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:41:28.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.448+0000 7f1635ce6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16300f5ed0 0x7f163010c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:41:28.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.448+0000 7f1635ce6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1630060530 con 0x7f16300f55c0 2021-11-03T17:41:28.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.448+0000 7f1635ce6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1630060670 con 0x7f16300f5ed0 2021-11-03T17:41:28.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.448+0000 7f162f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16300f55c0 0x7f1630109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:41:28.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.448+0000 7f162f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16300f55c0 0x7f1630109c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58838/0 (socket says 172.21.15.27:58838) 2021-11-03T17:41:28.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.448+0000 7f162f7fe700 1 -- 172.21.15.27:0/3846422440 learned_addr learned my addr 172.21.15.27:0/3846422440 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:41:28.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.448+0000 7f162effd700 1 --2- 172.21.15.27:0/3846422440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16300f5ed0 0x7f163010c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:41:28.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.448+0000 7f162f7fe700 1 -- 172.21.15.27:0/3846422440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16300f5ed0 msgr2=0x7f163010c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:41:28.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.448+0000 7f162f7fe700 1 --2- 172.21.15.27:0/3846422440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16300f5ed0 0x7f163010c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:41:28.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.448+0000 7f162f7fe700 1 -- 172.21.15.27:0/3846422440 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1620005040 con 0x7f16300f55c0 2021-11-03T17:41:28.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.449+0000 7f162f7fe700 1 --2- 172.21.15.27:0/3846422440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16300f55c0 0x7f1630109c70 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f1620009990 tx=0x7f1620009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:41:28.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.449+0000 7f1617fff700 1 -- 172.21.15.27:0/3846422440 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1620018070 con 0x7f16300f55c0 2021-11-03T17:41:28.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.449+0000 7f1635ce6700 1 -- 172.21.15.27:0/3846422440 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f163010c720 con 0x7f16300f55c0 2021-11-03T17:41:28.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.449+0000 7f1617fff700 1 -- 172.21.15.27:0/3846422440 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f1620007ca0 con 0x7f16300f55c0 2021-11-03T17:41:28.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.449+0000 7f1635ce6700 1 -- 172.21.15.27:0/3846422440 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f163010cc80 con 0x7f16300f55c0 2021-11-03T17:41:28.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.449+0000 7f1617fff700 1 -- 172.21.15.27:0/3846422440 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f162001c620 con 0x7f16300f55c0 2021-11-03T17:41:28.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.450+0000 7f1635ce6700 1 -- 172.21.15.27:0/3846422440 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1630045ad0 con 0x7f16300f55c0 2021-11-03T17:41:28.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.450+0000 7f1617fff700 1 -- 172.21.15.27:0/3846422440 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f162001d960 con 0x7f16300f55c0 2021-11-03T17:41:28.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.451+0000 7f1617fff700 1 --2- 172.21.15.27:0/3846422440 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f16180640b0 0x7f1618066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:41:28.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.451+0000 7f1617fff700 1 -- 172.21.15.27:0/3846422440 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f16200825f0 con 0x7f16300f55c0 2021-11-03T17:41:28.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.451+0000 7f162effd700 1 --2- 172.21.15.27:0/3846422440 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f16180640b0 0x7f1618066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:41:28.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.451+0000 7f162effd700 1 --2- 172.21.15.27:0/3846422440 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f16180640b0 0x7f1618066570 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f162400b790 tx=0x7f1624008040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:41:28.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.455+0000 7f1617fff700 1 -- 172.21.15.27:0/3846422440 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f16200549b0 con 0x7f16300f55c0 2021-11-03T17:41:28.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:41:28.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:41:28.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:41:28.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:28.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:28.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:28.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:28.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:41:28.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:41:28.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:41:28.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:28.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:28.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:28.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:28.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.635+0000 7f1635ce6700 1 -- 172.21.15.27:0/3846422440 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f16300fdea0 con 0x7f16180640b0 2021-11-03T17:41:28.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.637+0000 7f1617fff700 1 -- 172.21.15.27:0/3846422440 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f16300fdea0 con 0x7f16180640b0 2021-11-03T17:41:28.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.639+0000 7f1635ce6700 1 -- 172.21.15.27:0/3846422440 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f16180640b0 msgr2=0x7f1618066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:41:28.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.639+0000 7f1635ce6700 1 --2- 172.21.15.27:0/3846422440 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f16180640b0 0x7f1618066570 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 rx=0x7f162400b790 tx=0x7f1624008040).stop 2021-11-03T17:41:28.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.639+0000 7f1635ce6700 1 -- 172.21.15.27:0/3846422440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16300f55c0 msgr2=0x7f1630109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:41:28.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.639+0000 7f1635ce6700 1 --2- 172.21.15.27:0/3846422440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16300f55c0 0x7f1630109c70 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f1620009990 tx=0x7f1620009b30).stop 2021-11-03T17:41:28.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.639+0000 7f1635ce6700 1 -- 172.21.15.27:0/3846422440 shutdown_connections 2021-11-03T17:41:28.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.639+0000 7f1635ce6700 1 --2- 172.21.15.27:0/3846422440 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f16180640b0 0x7f1618066570 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:41:28.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.639+0000 7f1635ce6700 1 --2- 172.21.15.27:0/3846422440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16300f55c0 0x7f1630109c70 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:41:28.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.639+0000 7f1635ce6700 1 --2- 172.21.15.27:0/3846422440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16300f5ed0 0x7f163010c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:41:28.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.639+0000 7f1635ce6700 1 -- 172.21.15.27:0/3846422440 >> 172.21.15.27:0/3846422440 conn(0x7f16300f0de0 msgr2=0x7f16300fa350 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:41:28.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.639+0000 7f1635ce6700 1 -- 172.21.15.27:0/3846422440 shutdown_connections 2021-11-03T17:41:28.637 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.639+0000 7f1635ce6700 1 -- 172.21.15.27:0/3846422440 wait complete. 2021-11-03T17:41:28.643 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:41:28.735 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.740+0000 7f46dad87700 1 -- 172.21.15.27:0/2466786548 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46d40f6bf0 msgr2=0x7f46d40f7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:41:28.735 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.740+0000 7f46dad87700 1 --2- 172.21.15.27:0/2466786548 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46d40f6bf0 0x7f46d40f7010 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f46c8004660 tx=0x7f46c8009b30).stop 2021-11-03T17:41:28.735 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.740+0000 7f46dad87700 1 -- 172.21.15.27:0/2466786548 shutdown_connections 2021-11-03T17:41:28.736 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.740+0000 7f46dad87700 1 --2- 172.21.15.27:0/2466786548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46d40f7590 0x7f46d40f59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:41:28.736 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.740+0000 7f46dad87700 1 --2- 172.21.15.27:0/2466786548 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46d40f6bf0 0x7f46d40f7010 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:41:28.736 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.740+0000 7f46dad87700 1 -- 172.21.15.27:0/2466786548 >> 172.21.15.27:0/2466786548 conn(0x7f46d40f04c0 msgr2=0x7f46d40f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:41:28.736 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.740+0000 7f46dad87700 1 -- 172.21.15.27:0/2466786548 shutdown_connections 2021-11-03T17:41:28.737 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.740+0000 7f46dad87700 1 -- 172.21.15.27:0/2466786548 wait complete. 2021-11-03T17:41:28.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.740+0000 7f46dad87700 1 Processor -- start 2021-11-03T17:41:28.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.740+0000 7f46dad87700 1 -- start start 2021-11-03T17:41:28.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.741+0000 7f46dad87700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46d40f6bf0 0x7f46d4109cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:41:28.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.741+0000 7f46dad87700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46d40f7590 0x7f46d410c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:41:28.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.741+0000 7f46dad87700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d40f56b0 con 0x7f46d40f6bf0 2021-11-03T17:41:28.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.741+0000 7f46dad87700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46d40f57f0 con 0x7f46d40f7590 2021-11-03T17:41:28.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.741+0000 7f46d8b23700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46d40f6bf0 0x7f46d4109cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:41:28.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.741+0000 7f46d8b23700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46d40f6bf0 0x7f46d4109cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58848/0 (socket says 172.21.15.27:58848) 2021-11-03T17:41:28.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.741+0000 7f46d8b23700 1 -- 172.21.15.27:0/1738257642 learned_addr learned my addr 172.21.15.27:0/1738257642 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:41:28.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.741+0000 7f46d8b23700 1 -- 172.21.15.27:0/1738257642 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46d40f7590 msgr2=0x7f46d410c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:41:28.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.741+0000 7f46d8b23700 1 --2- 172.21.15.27:0/1738257642 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46d40f7590 0x7f46d410c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:41:28.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.741+0000 7f46d8b23700 1 -- 172.21.15.27:0/1738257642 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c8005040 con 0x7f46d40f6bf0 2021-11-03T17:41:28.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.741+0000 7f46d8b23700 1 --2- 172.21.15.27:0/1738257642 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46d40f6bf0 0x7f46d4109cc0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f46c8004f60 tx=0x7f46c8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:41:28.744 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.741+0000 7f46d17fa700 1 -- 172.21.15.27:0/1738257642 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46c8018070 con 0x7f46d40f6bf0 2021-11-03T17:41:28.744 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.741+0000 7f46dad87700 1 -- 172.21.15.27:0/1738257642 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46d410c770 con 0x7f46d40f6bf0 2021-11-03T17:41:28.744 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.741+0000 7f46d17fa700 1 -- 172.21.15.27:0/1738257642 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f46c8007ba0 con 0x7f46d40f6bf0 2021-11-03T17:41:28.745 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.742+0000 7f46d17fa700 1 -- 172.21.15.27:0/1738257642 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46c801dea0 con 0x7f46d40f6bf0 2021-11-03T17:41:28.745 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.742+0000 7f46dad87700 1 -- 172.21.15.27:0/1738257642 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46d410ccd0 con 0x7f46d40f6bf0 2021-11-03T17:41:28.745 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.743+0000 7f46dad87700 1 -- 172.21.15.27:0/1738257642 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46d4045ad0 con 0x7f46d40f6bf0 2021-11-03T17:41:28.746 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.743+0000 7f46d17fa700 1 -- 172.21.15.27:0/1738257642 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f46c801d420 con 0x7f46d40f6bf0 2021-11-03T17:41:28.746 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.743+0000 7f46d17fa700 1 --2- 172.21.15.27:0/1738257642 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f46c40640b0 0x7f46c4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:41:28.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.743+0000 7f46d17fa700 1 -- 172.21.15.27:0/1738257642 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f46c8082820 con 0x7f46d40f6bf0 2021-11-03T17:41:28.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.743+0000 7f46d3fff700 1 --2- 172.21.15.27:0/1738257642 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f46c40640b0 0x7f46c4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:41:28.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.744+0000 7f46d3fff700 1 --2- 172.21.15.27:0/1738257642 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f46c40640b0 0x7f46c4066570 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f46c0009610 tx=0x7f46c0006ab0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:41:28.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.747+0000 7f46d17fa700 1 -- 172.21.15.27:0/1738257642 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f46c8054be0 con 0x7f46d40f6bf0 2021-11-03T17:41:28.916 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.920+0000 7f46dad87700 1 -- 172.21.15.27:0/1738257642 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f46d40fdf70 con 0x7f46c40640b0 2021-11-03T17:41:28.930 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.935+0000 7f46d17fa700 1 -- 172.21.15.27:0/1738257642 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f46d40fdf70 con 0x7f46c40640b0 2021-11-03T17:41:28.930 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:41:28.930 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (33m) 2m ago 34m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:41:28.931 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (24m) 2m ago 34m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:41:28.931 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (24m) 105s ago 33m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:41:28.931 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (32m) 2m ago 33m 41.3M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:41:28.931 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (26m) 2m ago 29m 21.1M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:41:28.932 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (29m) 2m ago 29m 3843M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:41:28.932 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (25m) 105s ago 29m 20.4M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:41:28.932 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (26m) 105s ago 29m 22.1M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:41:28.932 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (26m) 2m ago 35m 517M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:41:28.932 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (26m) 105s ago 33m 419M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:41:28.933 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (25m) 2m ago 35m 159M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:41:28.933 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (25m) 105s ago 33m 82.5M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:41:28.933 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (33m) 2m ago 33m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:41:28.933 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (33m) 105s ago 33m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:41:28.934 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (24m) 2m ago 32m 311M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:41:28.934 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (23m) 2m ago 32m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:41:28.934 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (23m) 2m ago 31m 248M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:41:28.934 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (22m) 105s ago 31m 354M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:41:28.934 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (22m) 105s ago 31m 285M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:41:28.935 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (21m) 105s ago 31m 287M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:41:28.935 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (32m) 2m ago 33m 50.5M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:41:28.936 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.937+0000 7f46dad87700 1 -- 172.21.15.27:0/1738257642 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f46c40640b0 msgr2=0x7f46c4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:41:28.936 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.937+0000 7f46dad87700 1 --2- 172.21.15.27:0/1738257642 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f46c40640b0 0x7f46c4066570 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f46c0009610 tx=0x7f46c0006ab0).stop 2021-11-03T17:41:28.936 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.937+0000 7f46dad87700 1 -- 172.21.15.27:0/1738257642 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46d40f6bf0 msgr2=0x7f46d4109cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:41:28.937 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.937+0000 7f46dad87700 1 --2- 172.21.15.27:0/1738257642 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46d40f6bf0 0x7f46d4109cc0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f46c8004f60 tx=0x7f46c8009b30).stop 2021-11-03T17:41:28.937 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.937+0000 7f46dad87700 1 -- 172.21.15.27:0/1738257642 shutdown_connections 2021-11-03T17:41:28.937 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.937+0000 7f46dad87700 1 --2- 172.21.15.27:0/1738257642 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f46c40640b0 0x7f46c4066570 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:41:28.937 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.937+0000 7f46dad87700 1 --2- 172.21.15.27:0/1738257642 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46d40f6bf0 0x7f46d4109cc0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:41:28.938 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.937+0000 7f46dad87700 1 --2- 172.21.15.27:0/1738257642 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46d40f7590 0x7f46d410c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:41:28.938 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.937+0000 7f46dad87700 1 -- 172.21.15.27:0/1738257642 >> 172.21.15.27:0/1738257642 conn(0x7f46d40f04c0 msgr2=0x7f46d40fa3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:41:28.938 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.937+0000 7f46dad87700 1 -- 172.21.15.27:0/1738257642 shutdown_connections 2021-11-03T17:41:28.938 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:28.937+0000 7f46dad87700 1 -- 172.21.15.27:0/1738257642 wait complete. 2021-11-03T17:41:29.032 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.037+0000 7f5136294700 1 -- 172.21.15.27:0/1697175559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51300f6860 msgr2=0x7f51300f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:41:29.032 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.037+0000 7f5136294700 1 --2- 172.21.15.27:0/1697175559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51300f6860 0x7f51300f6cc0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f5124004660 tx=0x7f5124009b30).stop 2021-11-03T17:41:29.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.037+0000 7f5136294700 1 -- 172.21.15.27:0/1697175559 shutdown_connections 2021-11-03T17:41:29.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.037+0000 7f5136294700 1 --2- 172.21.15.27:0/1697175559 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51300f6860 0x7f51300f6cc0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:41:29.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.037+0000 7f5136294700 1 --2- 172.21.15.27:0/1697175559 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51300f55c0 0x7f51300f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:41:29.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.037+0000 7f5136294700 1 -- 172.21.15.27:0/1697175559 >> 172.21.15.27:0/1697175559 conn(0x7f51300f0d40 msgr2=0x7f51300f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:41:29.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.037+0000 7f5136294700 1 -- 172.21.15.27:0/1697175559 shutdown_connections 2021-11-03T17:41:29.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.037+0000 7f5136294700 1 -- 172.21.15.27:0/1697175559 wait complete. 2021-11-03T17:41:29.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.037+0000 7f5136294700 1 Processor -- start 2021-11-03T17:41:29.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.038+0000 7f5136294700 1 -- start start 2021-11-03T17:41:29.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.038+0000 7f5136294700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51300f55c0 0x7f5130106f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:41:29.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.038+0000 7f5136294700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51300f6860 0x7f5130107440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:41:29.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.038+0000 7f5136294700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51301079f0 con 0x7f51300f6860 2021-11-03T17:41:29.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.038+0000 7f5136294700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5130107b30 con 0x7f51300f55c0 2021-11-03T17:41:29.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.038+0000 7f512f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51300f6860 0x7f5130107440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:41:29.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.038+0000 7f512f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51300f6860 0x7f5130107440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58860/0 (socket says 172.21.15.27:58860) 2021-11-03T17:41:29.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.038+0000 7f512f7fe700 1 -- 172.21.15.27:0/1601272895 learned_addr learned my addr 172.21.15.27:0/1601272895 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:41:29.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.038+0000 7f512ffff700 1 --2- 172.21.15.27:0/1601272895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51300f55c0 0x7f5130106f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:41:29.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.038+0000 7f512f7fe700 1 -- 172.21.15.27:0/1601272895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51300f55c0 msgr2=0x7f5130106f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:41:29.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.038+0000 7f512f7fe700 1 --2- 172.21.15.27:0/1601272895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51300f55c0 0x7f5130106f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:41:29.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.038+0000 7f512f7fe700 1 -- 172.21.15.27:0/1601272895 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5124005040 con 0x7f51300f6860 2021-11-03T17:41:29.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.038+0000 7f512f7fe700 1 --2- 172.21.15.27:0/1601272895 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51300f6860 0x7f5130107440 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f5124009990 tx=0x7f5124009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:41:29.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.039+0000 7f512cff9700 1 -- 172.21.15.27:0/1601272895 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5124018070 con 0x7f51300f6860 2021-11-03T17:41:29.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.039+0000 7f512cff9700 1 -- 172.21.15.27:0/1601272895 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f5124007ba0 con 0x7f51300f6860 2021-11-03T17:41:29.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.039+0000 7f5136294700 1 -- 172.21.15.27:0/1601272895 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5130108470 con 0x7f51300f6860 2021-11-03T17:41:29.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.039+0000 7f512cff9700 1 -- 172.21.15.27:0/1601272895 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f512401c3c0 con 0x7f51300f6860 2021-11-03T17:41:29.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.039+0000 7f5136294700 1 -- 172.21.15.27:0/1601272895 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51301089d0 con 0x7f51300f6860 2021-11-03T17:41:29.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.040+0000 7f5136294700 1 -- 172.21.15.27:0/1601272895 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5130045ad0 con 0x7f51300f6860 2021-11-03T17:41:29.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.040+0000 7f512cff9700 1 -- 172.21.15.27:0/1601272895 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f512401d420 con 0x7f51300f6860 2021-11-03T17:41:29.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.040+0000 7f512cff9700 1 --2- 172.21.15.27:0/1601272895 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f51180640b0 0x7f5118066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:41:29.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.040+0000 7f512cff9700 1 -- 172.21.15.27:0/1601272895 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f5124082580 con 0x7f51300f6860 2021-11-03T17:41:29.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.041+0000 7f512ffff700 1 --2- 172.21.15.27:0/1601272895 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f51180640b0 0x7f5118066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:41:29.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.041+0000 7f512ffff700 1 --2- 172.21.15.27:0/1601272895 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f51180640b0 0x7f5118066570 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f51300f66c0 tx=0x7f512000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:41:29.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.044+0000 7f512cff9700 1 -- 172.21.15.27:0/1601272895 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f5124054940 con 0x7f51300f6860 2021-11-03T17:41:29.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.272+0000 7f5136294700 1 -- 172.21.15.27:0/1601272895 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5130027a00 con 0x7f51300f6860 2021-11-03T17:41:29.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.272+0000 7f512cff9700 1 -- 172.21.15.27:0/1601272895 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f5124054530 con 0x7f51300f6860 2021-11-03T17:41:29.268 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:41:29.268 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:41:29.268 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:41:29.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:41:29.269 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:41:29.269 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:41:29.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:41:29.270 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:41:29.270 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:41:29.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:41:29.270 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:41:29.271 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:41:29.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:41:29.271 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:41:29.272 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:41:29.272 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:41:29.273 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:41:29.273 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:41:29.274 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.274+0000 7f5136294700 1 -- 172.21.15.27:0/1601272895 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f51180640b0 msgr2=0x7f5118066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:41:29.275 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.274+0000 7f5136294700 1 --2- 172.21.15.27:0/1601272895 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f51180640b0 0x7f5118066570 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f51300f66c0 tx=0x7f512000b040).stop 2021-11-03T17:41:29.275 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.274+0000 7f5136294700 1 -- 172.21.15.27:0/1601272895 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51300f6860 msgr2=0x7f5130107440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:41:29.275 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.274+0000 7f5136294700 1 --2- 172.21.15.27:0/1601272895 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51300f6860 0x7f5130107440 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7f5124009990 tx=0x7f5124009b30).stop 2021-11-03T17:41:29.276 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.274+0000 7f5136294700 1 -- 172.21.15.27:0/1601272895 shutdown_connections 2021-11-03T17:41:29.276 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.274+0000 7f5136294700 1 --2- 172.21.15.27:0/1601272895 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f51180640b0 0x7f5118066570 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:41:29.276 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.274+0000 7f5136294700 1 --2- 172.21.15.27:0/1601272895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51300f55c0 0x7f5130106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:41:29.276 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.274+0000 7f5136294700 1 --2- 172.21.15.27:0/1601272895 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51300f6860 0x7f5130107440 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:41:29.277 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.274+0000 7f5136294700 1 -- 172.21.15.27:0/1601272895 >> 172.21.15.27:0/1601272895 conn(0x7f51300f0d40 msgr2=0x7f51300fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:41:29.277 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.274+0000 7f5136294700 1 -- 172.21.15.27:0/1601272895 shutdown_connections 2021-11-03T17:41:29.277 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.274+0000 7f5136294700 1 -- 172.21.15.27:0/1601272895 wait complete. 2021-11-03T17:41:29.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.374+0000 7f0f86c91700 1 -- 172.21.15.27:0/3105788941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f800f55c0 msgr2=0x7f0f800f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:41:29.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.374+0000 7f0f86c91700 1 --2- 172.21.15.27:0/3105788941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f800f55c0 0x7f0f800f59e0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f0f78004660 tx=0x7f0f78009b30).stop 2021-11-03T17:41:29.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.375+0000 7f0f86c91700 1 -- 172.21.15.27:0/3105788941 shutdown_connections 2021-11-03T17:41:29.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.375+0000 7f0f86c91700 1 --2- 172.21.15.27:0/3105788941 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f800f5ed0 0x7f0f800f6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:41:29.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.375+0000 7f0f86c91700 1 --2- 172.21.15.27:0/3105788941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f800f55c0 0x7f0f800f59e0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:41:29.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.375+0000 7f0f86c91700 1 -- 172.21.15.27:0/3105788941 >> 172.21.15.27:0/3105788941 conn(0x7f0f800f0dc0 msgr2=0x7f0f800f31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:41:29.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.375+0000 7f0f86c91700 1 -- 172.21.15.27:0/3105788941 shutdown_connections 2021-11-03T17:41:29.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.375+0000 7f0f86c91700 1 -- 172.21.15.27:0/3105788941 wait complete. 2021-11-03T17:41:29.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.375+0000 7f0f86c91700 1 Processor -- start 2021-11-03T17:41:29.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.375+0000 7f0f86c91700 1 -- start start 2021-11-03T17:41:29.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.375+0000 7f0f86c91700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f800f55c0 0x7f0f80109c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:41:29.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.376+0000 7f0f86c91700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f800f5ed0 0x7f0f8010c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:41:29.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.376+0000 7f0f86c91700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f800605a0 con 0x7f0f800f5ed0 2021-11-03T17:41:29.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.376+0000 7f0f86c91700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f800606e0 con 0x7f0f800f55c0 2021-11-03T17:41:29.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.376+0000 7f0f7ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f800f5ed0 0x7f0f8010c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:41:29.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.376+0000 7f0f7ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f800f5ed0 0x7f0f8010c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58870/0 (socket says 172.21.15.27:58870) 2021-11-03T17:41:29.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.376+0000 7f0f7ffff700 1 -- 172.21.15.27:0/4033023311 learned_addr learned my addr 172.21.15.27:0/4033023311 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:41:29.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.376+0000 7f0f84a2d700 1 --2- 172.21.15.27:0/4033023311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f800f55c0 0x7f0f80109c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:41:29.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.376+0000 7f0f7ffff700 1 -- 172.21.15.27:0/4033023311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f800f55c0 msgr2=0x7f0f80109c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:41:29.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.376+0000 7f0f7ffff700 1 --2- 172.21.15.27:0/4033023311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f800f55c0 0x7f0f80109c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:41:29.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.376+0000 7f0f7ffff700 1 -- 172.21.15.27:0/4033023311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f78005040 con 0x7f0f800f5ed0 2021-11-03T17:41:29.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.376+0000 7f0f7ffff700 1 --2- 172.21.15.27:0/4033023311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f800f5ed0 0x7f0f8010c130 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f0f70007fd0 tx=0x7f0f70004ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:41:29.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.376+0000 7f0f7d7fa700 1 -- 172.21.15.27:0/4033023311 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f7000b4f0 con 0x7f0f800f5ed0 2021-11-03T17:41:29.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.376+0000 7f0f7d7fa700 1 -- 172.21.15.27:0/4033023311 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f0f7000f040 con 0x7f0f800f5ed0 2021-11-03T17:41:29.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.376+0000 7f0f86c91700 1 -- 172.21.15.27:0/4033023311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f8010c740 con 0x7f0f800f5ed0 2021-11-03T17:41:29.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.376+0000 7f0f7d7fa700 1 -- 172.21.15.27:0/4033023311 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f70013a50 con 0x7f0f800f5ed0 2021-11-03T17:41:29.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.376+0000 7f0f86c91700 1 -- 172.21.15.27:0/4033023311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f8010ccd0 con 0x7f0f800f5ed0 2021-11-03T17:41:29.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.377+0000 7f0f86c91700 1 -- 172.21.15.27:0/4033023311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f80045ad0 con 0x7f0f800f5ed0 2021-11-03T17:41:29.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.379+0000 7f0f7d7fa700 1 -- 172.21.15.27:0/4033023311 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f0f7001a030 con 0x7f0f800f5ed0 2021-11-03T17:41:29.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.379+0000 7f0f7d7fa700 1 --2- 172.21.15.27:0/4033023311 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0f6c0640b0 0x7f0f6c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:41:29.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.380+0000 7f0f7d7fa700 1 -- 172.21.15.27:0/4033023311 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f0f70080f60 con 0x7f0f800f5ed0 2021-11-03T17:41:29.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.380+0000 7f0f84a2d700 1 --2- 172.21.15.27:0/4033023311 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0f6c0640b0 0x7f0f6c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:41:29.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.380+0000 7f0f84a2d700 1 --2- 172.21.15.27:0/4033023311 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0f6c0640b0 0x7f0f6c066570 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f0f78004630 tx=0x7f0f78009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:41:29.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.382+0000 7f0f7d7fa700 1 -- 172.21.15.27:0/4033023311 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f0f70053220 con 0x7f0f800f5ed0 2021-11-03T17:41:29.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:29 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:41:29.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:29 smithi115 ceph-mon[65453]: pgmap v805: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:29.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:29 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:41:29.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:29 smithi027 ceph-mon[97244]: pgmap v805: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:29.581 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.586+0000 7f0f86c91700 1 -- 172.21.15.27:0/4033023311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0f80027a00 con 0x7f0f800f5ed0 2021-11-03T17:41:29.582 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.587+0000 7f0f7d7fa700 1 -- 172.21.15.27:0/4033023311 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f0f70022020 con 0x7f0f800f5ed0 2021-11-03T17:41:29.583 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:41:29.583 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:41:29.583 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:41:29.584 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:41:29.584 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:41:29.584 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:41:29.584 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:41:29.584 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:41:29.585 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:41:29.585 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:41:29.585 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:41:29.585 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:41:29.586 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:41:29.586 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:41:29.586 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:41:29.586 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:41:29.587 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:41:29.587 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:41:29.587 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:41:29.587 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:41:29.587 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:41:29.588 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:41:29.588 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:41:29.588 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:41:29.589 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:41:29.589 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:41:29.589 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:41:29.590 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:41:29.590 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:41:29.590 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:41:29.590 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:41:29.591 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:41:29.591 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:41:29.591 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:41:29.591 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:41:29.591 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:41:29.592 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:41:29.592 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:41:29.592 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:41:29.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.589+0000 7f0f86c91700 1 -- 172.21.15.27:0/4033023311 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0f6c0640b0 msgr2=0x7f0f6c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:41:29.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.589+0000 7f0f86c91700 1 --2- 172.21.15.27:0/4033023311 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0f6c0640b0 0x7f0f6c066570 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f0f78004630 tx=0x7f0f78009b30).stop 2021-11-03T17:41:29.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.589+0000 7f0f86c91700 1 -- 172.21.15.27:0/4033023311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f800f5ed0 msgr2=0x7f0f8010c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:41:29.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.589+0000 7f0f86c91700 1 --2- 172.21.15.27:0/4033023311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f800f5ed0 0x7f0f8010c130 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f0f70007fd0 tx=0x7f0f70004ee0).stop 2021-11-03T17:41:29.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.589+0000 7f0f86c91700 1 -- 172.21.15.27:0/4033023311 shutdown_connections 2021-11-03T17:41:29.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.589+0000 7f0f86c91700 1 --2- 172.21.15.27:0/4033023311 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0f6c0640b0 0x7f0f6c066570 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:41:29.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.589+0000 7f0f86c91700 1 --2- 172.21.15.27:0/4033023311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f800f55c0 0x7f0f80109c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:41:29.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.589+0000 7f0f86c91700 1 --2- 172.21.15.27:0/4033023311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f800f5ed0 0x7f0f8010c130 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:41:29.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.589+0000 7f0f86c91700 1 -- 172.21.15.27:0/4033023311 >> 172.21.15.27:0/4033023311 conn(0x7f0f800f0dc0 msgr2=0x7f0f800fa350 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:41:29.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.589+0000 7f0f86c91700 1 -- 172.21.15.27:0/4033023311 shutdown_connections 2021-11-03T17:41:29.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:29.589+0000 7f0f86c91700 1 -- 172.21.15.27:0/4033023311 wait complete. 2021-11-03T17:41:29.596 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:41:30.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:30 smithi115 ceph-mon[65453]: from='client.34936 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:41:30.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:30 smithi115 ceph-mon[65453]: from='client.34940 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:41:30.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:30 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1601272895' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:30.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1532.395382 secs 2021-11-03T17:41:30.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:30 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/4033023311' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:41:30.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:30 smithi027 ceph-mon[97244]: from='client.34936 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:41:30.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:30 smithi027 ceph-mon[97244]: from='client.34940 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:41:30.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:30 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1601272895' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:30.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1532.395382 secs 2021-11-03T17:41:30.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:30 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/4033023311' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:41:31.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:31 smithi115 ceph-mon[65453]: pgmap v806: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:31.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:31 smithi027 ceph-mon[97244]: pgmap v806: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:33.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:33 smithi027 ceph-mon[97244]: pgmap v807: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:33.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:33 smithi115 ceph-mon[65453]: pgmap v807: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:35.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:35 smithi115 ceph-mon[65453]: pgmap v808: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:35.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1537.395500 secs 2021-11-03T17:41:35.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:35 smithi027 ceph-mon[97244]: pgmap v808: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:35.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1537.395500 secs 2021-11-03T17:41:37.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:37 smithi115 ceph-mon[65453]: pgmap v809: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:37.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:37 smithi027 ceph-mon[97244]: pgmap v809: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:38.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:41:38.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:41:38.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:41:38.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:38.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:38.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:38.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:38.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:41:38.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:41:38.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:41:38.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:38.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:38.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:38.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:39.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:39 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:41:39.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:39 smithi115 ceph-mon[65453]: pgmap v810: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:39.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:39 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:41:39.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:39 smithi027 ceph-mon[97244]: pgmap v810: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:40.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1542.395617 secs 2021-11-03T17:41:40.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1542.395617 secs 2021-11-03T17:41:41.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:41 smithi115 ceph-mon[65453]: pgmap v811: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:41.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:41 smithi027 ceph-mon[97244]: pgmap v811: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:42.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:42 smithi115 ceph-mon[65453]: pgmap v812: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:42.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:42 smithi027 ceph-mon[97244]: pgmap v812: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:44.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:44 smithi027 ceph-mon[97244]: pgmap v813: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:44.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:44 smithi115 ceph-mon[65453]: pgmap v813: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:45.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1547.395749 secs 2021-11-03T17:41:45.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1547.395749 secs 2021-11-03T17:41:46.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:46 smithi027 ceph-mon[97244]: pgmap v814: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:46.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:46 smithi115 ceph-mon[65453]: pgmap v814: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:48.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:41:48.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:41:48.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:48 smithi115 ceph-mon[65453]: pgmap v815: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:48.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:41:48.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:48.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:48.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:48.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:48.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:41:48.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:41:48.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:48 smithi027 ceph-mon[97244]: pgmap v815: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:48.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:41:48.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:48.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:48.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:48.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:50.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:50 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:41:50.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1552.395871 secs 2021-11-03T17:41:50.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:50 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:41:50.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1552.395871 secs 2021-11-03T17:41:51.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:51 smithi115 ceph-mon[65453]: pgmap v816: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:51.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:51 smithi027 ceph-mon[97244]: pgmap v816: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:53.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:53 smithi115 ceph-mon[65453]: pgmap v817: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:53.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:53 smithi027 ceph-mon[97244]: pgmap v817: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:55.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:55 smithi115 ceph-mon[65453]: pgmap v818: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:55.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1557.396026 secs 2021-11-03T17:41:55.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:55 smithi027 ceph-mon[97244]: pgmap v818: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:55.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1557.396026 secs 2021-11-03T17:41:56.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:41:56.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:41:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:41:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:41:57.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:57 smithi115 ceph-mon[65453]: pgmap v819: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:57.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:57 smithi027 ceph-mon[97244]: pgmap v819: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:58.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:58 smithi115 ceph-mon[65453]: pgmap v820: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:58.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:58 smithi027 ceph-mon[97244]: pgmap v820: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:41:59.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:41:59.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:41:59.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:41:59.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:59.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:59.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:59.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:59.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:41:59 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:41:59.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:41:59.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:41:59.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:41:59.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:59.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:59.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:59.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:41:59.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:41:59 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:41:59.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.703+0000 7f63663a5700 1 -- 172.21.15.27:0/3109970998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63600f98b0 msgr2=0x7f63600fbca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:41:59.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.703+0000 7f63663a5700 1 --2- 172.21.15.27:0/3109970998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63600f98b0 0x7f63600fbca0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f6350004660 tx=0x7f6350009b30).stop 2021-11-03T17:41:59.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.704+0000 7f63663a5700 1 -- 172.21.15.27:0/3109970998 shutdown_connections 2021-11-03T17:41:59.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.704+0000 7f63663a5700 1 --2- 172.21.15.27:0/3109970998 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63600f98b0 0x7f63600fbca0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:41:59.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.704+0000 7f63663a5700 1 --2- 172.21.15.27:0/3109970998 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6360060390 0x7f63600f9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:41:59.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.704+0000 7f63663a5700 1 -- 172.21.15.27:0/3109970998 >> 172.21.15.27:0/3109970998 conn(0x7f63600f0d30 msgr2=0x7f63600f3150 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:41:59.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.704+0000 7f63663a5700 1 -- 172.21.15.27:0/3109970998 shutdown_connections 2021-11-03T17:41:59.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.704+0000 7f63663a5700 1 -- 172.21.15.27:0/3109970998 wait complete. 2021-11-03T17:41:59.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.704+0000 7f63663a5700 1 Processor -- start 2021-11-03T17:41:59.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.704+0000 7f63663a5700 1 -- start start 2021-11-03T17:41:59.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.705+0000 7f63663a5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6360060390 0x7f6360106f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:41:59.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.705+0000 7f63663a5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63600f98b0 0x7f6360107430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:41:59.704 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.705+0000 7f63663a5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63601079e0 con 0x7f63600f98b0 2021-11-03T17:41:59.704 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.705+0000 7f63663a5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6360107b20 con 0x7f6360060390 2021-11-03T17:41:59.704 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.705+0000 7f635f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63600f98b0 0x7f6360107430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:41:59.704 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.705+0000 7f635f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63600f98b0 0x7f6360107430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58888/0 (socket says 172.21.15.27:58888) 2021-11-03T17:41:59.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.705+0000 7f635f7fe700 1 -- 172.21.15.27:0/3113396865 learned_addr learned my addr 172.21.15.27:0/3113396865 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:41:59.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.705+0000 7f635ffff700 1 --2- 172.21.15.27:0/3113396865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6360060390 0x7f6360106f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:41:59.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.705+0000 7f635f7fe700 1 -- 172.21.15.27:0/3113396865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6360060390 msgr2=0x7f6360106f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:41:59.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.705+0000 7f635f7fe700 1 --2- 172.21.15.27:0/3113396865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6360060390 0x7f6360106f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:41:59.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.705+0000 7f635f7fe700 1 -- 172.21.15.27:0/3113396865 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6350005040 con 0x7f63600f98b0 2021-11-03T17:41:59.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.705+0000 7f635ffff700 1 --2- 172.21.15.27:0/3113396865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6360060390 0x7f6360106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:41:59.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.705+0000 7f635f7fe700 1 --2- 172.21.15.27:0/3113396865 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63600f98b0 0x7f6360107430 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f6350013a70 tx=0x7f6350009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:41:59.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.705+0000 7f635cff9700 1 -- 172.21.15.27:0/3113396865 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6350018070 con 0x7f63600f98b0 2021-11-03T17:41:59.707 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.705+0000 7f635cff9700 1 -- 172.21.15.27:0/3113396865 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f63500097d0 con 0x7f63600f98b0 2021-11-03T17:41:59.707 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.705+0000 7f63663a5700 1 -- 172.21.15.27:0/3113396865 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6360108460 con 0x7f63600f98b0 2021-11-03T17:41:59.707 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.706+0000 7f63663a5700 1 -- 172.21.15.27:0/3113396865 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63601089c0 con 0x7f63600f98b0 2021-11-03T17:41:59.707 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.706+0000 7f635cff9700 1 -- 172.21.15.27:0/3113396865 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f635001c3c0 con 0x7f63600f98b0 2021-11-03T17:41:59.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.707+0000 7f63663a5700 1 -- 172.21.15.27:0/3113396865 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6360045ad0 con 0x7f63600f98b0 2021-11-03T17:41:59.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.707+0000 7f635cff9700 1 -- 172.21.15.27:0/3113396865 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f6350007ba0 con 0x7f63600f98b0 2021-11-03T17:41:59.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.707+0000 7f635cff9700 1 --2- 172.21.15.27:0/3113396865 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f63480640b0 0x7f6348066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:41:59.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.708+0000 7f635cff9700 1 -- 172.21.15.27:0/3113396865 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f63500824d0 con 0x7f63600f98b0 2021-11-03T17:41:59.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.708+0000 7f635ffff700 1 --2- 172.21.15.27:0/3113396865 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f63480640b0 0x7f6348066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:41:59.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.708+0000 7f635ffff700 1 --2- 172.21.15.27:0/3113396865 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f63480640b0 0x7f6348066570 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f6354009740 tx=0x7f6354009be0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:41:59.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.712+0000 7f635cff9700 1 -- 172.21.15.27:0/3113396865 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f63500547d0 con 0x7f63600f98b0 2021-11-03T17:41:59.884 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.889+0000 7f63663a5700 1 -- 172.21.15.27:0/3113396865 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f63600fefa0 con 0x7f63480640b0 2021-11-03T17:41:59.886 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.891+0000 7f635cff9700 1 -- 172.21.15.27:0/3113396865 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f63600fefa0 con 0x7f63480640b0 2021-11-03T17:41:59.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.892+0000 7f63663a5700 1 -- 172.21.15.27:0/3113396865 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f63480640b0 msgr2=0x7f6348066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:41:59.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.892+0000 7f63663a5700 1 --2- 172.21.15.27:0/3113396865 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f63480640b0 0x7f6348066570 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 rx=0x7f6354009740 tx=0x7f6354009be0).stop 2021-11-03T17:41:59.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.892+0000 7f63663a5700 1 -- 172.21.15.27:0/3113396865 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63600f98b0 msgr2=0x7f6360107430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:41:59.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.892+0000 7f63663a5700 1 --2- 172.21.15.27:0/3113396865 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63600f98b0 0x7f6360107430 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 rx=0x7f6350013a70 tx=0x7f6350009b30).stop 2021-11-03T17:41:59.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.893+0000 7f63663a5700 1 -- 172.21.15.27:0/3113396865 shutdown_connections 2021-11-03T17:41:59.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.893+0000 7f63663a5700 1 --2- 172.21.15.27:0/3113396865 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f63480640b0 0x7f6348066570 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:41:59.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.893+0000 7f63663a5700 1 --2- 172.21.15.27:0/3113396865 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6360060390 0x7f6360106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:41:59.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.893+0000 7f63663a5700 1 --2- 172.21.15.27:0/3113396865 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f63600f98b0 0x7f6360107430 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:41:59.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.893+0000 7f63663a5700 1 -- 172.21.15.27:0/3113396865 >> 172.21.15.27:0/3113396865 conn(0x7f63600f0d30 msgr2=0x7f63600f3120 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:41:59.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.893+0000 7f63663a5700 1 -- 172.21.15.27:0/3113396865 shutdown_connections 2021-11-03T17:41:59.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.893+0000 7f63663a5700 1 -- 172.21.15.27:0/3113396865 wait complete. 2021-11-03T17:41:59.897 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:41:59.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.993+0000 7f5476862700 1 -- 172.21.15.27:0/924958851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54700f66e0 msgr2=0x7f54700f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:41:59.989 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.993+0000 7f5476862700 1 --2- 172.21.15.27:0/924958851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54700f66e0 0x7f54700f6b00 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f5460004660 tx=0x7f5460009b30).stop 2021-11-03T17:41:59.989 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.993+0000 7f5476862700 1 -- 172.21.15.27:0/924958851 shutdown_connections 2021-11-03T17:41:59.989 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.993+0000 7f5476862700 1 --2- 172.21.15.27:0/924958851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54700f6ff0 0x7f54700f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:41:59.990 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.993+0000 7f5476862700 1 --2- 172.21.15.27:0/924958851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54700f66e0 0x7f54700f6b00 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:41:59.990 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.993+0000 7f5476862700 1 -- 172.21.15.27:0/924958851 >> 172.21.15.27:0/924958851 conn(0x7f54700f1f00 msgr2=0x7f54700f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:41:59.990 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.993+0000 7f5476862700 1 -- 172.21.15.27:0/924958851 shutdown_connections 2021-11-03T17:41:59.990 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.993+0000 7f5476862700 1 -- 172.21.15.27:0/924958851 wait complete. 2021-11-03T17:41:59.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.994+0000 7f5476862700 1 Processor -- start 2021-11-03T17:41:59.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.994+0000 7f5476862700 1 -- start start 2021-11-03T17:41:59.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.994+0000 7f5476862700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54700f66e0 0x7f5470106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:41:59.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.994+0000 7f5476862700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54700f6ff0 0x7f5470107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:41:59.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.994+0000 7f5476862700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5470107a00 con 0x7f54700f66e0 2021-11-03T17:41:59.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.994+0000 7f5476862700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5470107b40 con 0x7f54700f6ff0 2021-11-03T17:41:59.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.994+0000 7f546ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54700f66e0 0x7f5470106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:41:59.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.994+0000 7f546ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54700f66e0 0x7f5470106f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58896/0 (socket says 172.21.15.27:58896) 2021-11-03T17:41:59.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.994+0000 7f546ffff700 1 -- 172.21.15.27:0/200512277 learned_addr learned my addr 172.21.15.27:0/200512277 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:41:59.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.995+0000 7f546ffff700 1 -- 172.21.15.27:0/200512277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54700f6ff0 msgr2=0x7f5470107450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:41:59.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.995+0000 7f546f7fe700 1 --2- 172.21.15.27:0/200512277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54700f6ff0 0x7f5470107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:41:59.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.995+0000 7f546ffff700 1 --2- 172.21.15.27:0/200512277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54700f6ff0 0x7f5470107450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:41:59.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.995+0000 7f546ffff700 1 -- 172.21.15.27:0/200512277 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5460005040 con 0x7f54700f66e0 2021-11-03T17:41:59.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.995+0000 7f546f7fe700 1 --2- 172.21.15.27:0/200512277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54700f6ff0 0x7f5470107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:41:59.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.995+0000 7f546ffff700 1 --2- 172.21.15.27:0/200512277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54700f66e0 0x7f5470106f60 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f5460009990 tx=0x7f5460009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:41:59.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.995+0000 7f546cff9700 1 -- 172.21.15.27:0/200512277 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5460018070 con 0x7f54700f66e0 2021-11-03T17:41:59.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.995+0000 7f5476862700 1 -- 172.21.15.27:0/200512277 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5470108480 con 0x7f54700f66e0 2021-11-03T17:41:59.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.995+0000 7f546cff9700 1 -- 172.21.15.27:0/200512277 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f5460007ca0 con 0x7f54700f66e0 2021-11-03T17:41:59.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.995+0000 7f546cff9700 1 -- 172.21.15.27:0/200512277 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f546001c620 con 0x7f54700f66e0 2021-11-03T17:41:59.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.995+0000 7f5476862700 1 -- 172.21.15.27:0/200512277 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54701089e0 con 0x7f54700f66e0 2021-11-03T17:41:59.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.996+0000 7f5476862700 1 -- 172.21.15.27:0/200512277 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5470045ad0 con 0x7f54700f66e0 2021-11-03T17:41:59.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.996+0000 7f546cff9700 1 -- 172.21.15.27:0/200512277 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f546001d960 con 0x7f54700f66e0 2021-11-03T17:41:59.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.997+0000 7f546cff9700 1 --2- 172.21.15.27:0/200512277 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f54580640b0 0x7f5458066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:41:59.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.997+0000 7f546cff9700 1 -- 172.21.15.27:0/200512277 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f5460082640 con 0x7f54700f66e0 2021-11-03T17:41:59.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.997+0000 7f546f7fe700 1 --2- 172.21.15.27:0/200512277 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f54580640b0 0x7f5458066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:41:59.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:41:59.997+0000 7f546f7fe700 1 --2- 172.21.15.27:0/200512277 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f54580640b0 0x7f5458066570 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f5464004a70 tx=0x7f5464009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:42:00.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.001+0000 7f546cff9700 1 -- 172.21.15.27:0/200512277 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f5460054940 con 0x7f54700f66e0 2021-11-03T17:42:00.168 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.173+0000 7f5476862700 1 -- 172.21.15.27:0/200512277 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f54700fef40 con 0x7f54580640b0 2021-11-03T17:42:00.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.187+0000 7f546cff9700 1 -- 172.21.15.27:0/200512277 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f54700fef40 con 0x7f54580640b0 2021-11-03T17:42:00.183 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:42:00.183 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (33m) 3m ago 34m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:42:00.183 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (25m) 3m ago 34m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:42:00.184 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (25m) 2m ago 34m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:42:00.184 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (33m) 3m ago 34m 41.3M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:42:00.184 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (27m) 3m ago 29m 21.1M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:42:00.184 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (30m) 3m ago 30m 3843M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:42:00.184 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (25m) 2m ago 29m 20.4M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:42:00.185 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (27m) 2m ago 30m 22.1M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:42:00.185 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (27m) 3m ago 36m 517M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:42:00.185 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (26m) 2m ago 34m 419M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:42:00.185 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (26m) 3m ago 36m 159M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:42:00.186 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (25m) 2m ago 33m 82.5M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:42:00.186 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (34m) 3m ago 34m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:42:00.186 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (33m) 2m ago 33m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:42:00.186 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (24m) 3m ago 33m 311M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:42:00.186 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (24m) 3m ago 32m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:42:00.187 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (23m) 3m ago 32m 248M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:42:00.187 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (23m) 2m ago 32m 354M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:42:00.187 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (22m) 2m ago 31m 285M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:42:00.187 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (22m) 2m ago 31m 287M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:42:00.188 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (33m) 3m ago 34m 50.5M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:42:00.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.189+0000 7f5476862700 1 -- 172.21.15.27:0/200512277 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f54580640b0 msgr2=0x7f5458066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:42:00.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.189+0000 7f5476862700 1 --2- 172.21.15.27:0/200512277 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f54580640b0 0x7f5458066570 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7f5464004a70 tx=0x7f5464009040).stop 2021-11-03T17:42:00.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.189+0000 7f5476862700 1 -- 172.21.15.27:0/200512277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54700f66e0 msgr2=0x7f5470106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:42:00.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.189+0000 7f5476862700 1 --2- 172.21.15.27:0/200512277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54700f66e0 0x7f5470106f60 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f5460009990 tx=0x7f5460009b30).stop 2021-11-03T17:42:00.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.189+0000 7f5476862700 1 -- 172.21.15.27:0/200512277 shutdown_connections 2021-11-03T17:42:00.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.189+0000 7f5476862700 1 --2- 172.21.15.27:0/200512277 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f54580640b0 0x7f5458066570 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:42:00.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.189+0000 7f5476862700 1 --2- 172.21.15.27:0/200512277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54700f66e0 0x7f5470106f60 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:42:00.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.189+0000 7f5476862700 1 --2- 172.21.15.27:0/200512277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54700f6ff0 0x7f5470107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:42:00.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.189+0000 7f5476862700 1 -- 172.21.15.27:0/200512277 >> 172.21.15.27:0/200512277 conn(0x7f54700f1f00 msgr2=0x7f54700fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:42:00.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.189+0000 7f5476862700 1 -- 172.21.15.27:0/200512277 shutdown_connections 2021-11-03T17:42:00.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.189+0000 7f5476862700 1 -- 172.21.15.27:0/200512277 wait complete. 2021-11-03T17:42:00.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.290+0000 7f7260032700 1 -- 172.21.15.27:0/389753985 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72580f55e0 msgr2=0x7f72580f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:42:00.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.290+0000 7f7260032700 1 --2- 172.21.15.27:0/389753985 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72580f55e0 0x7f72580f5a00 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f72500099e0 tx=0x7f7250004d10).stop 2021-11-03T17:42:00.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.290+0000 7f7260032700 1 -- 172.21.15.27:0/389753985 shutdown_connections 2021-11-03T17:42:00.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.290+0000 7f7260032700 1 --2- 172.21.15.27:0/389753985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72580f5ef0 0x7f72580f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:42:00.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.290+0000 7f7260032700 1 --2- 172.21.15.27:0/389753985 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72580f55e0 0x7f72580f5a00 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:42:00.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.290+0000 7f7260032700 1 -- 172.21.15.27:0/389753985 >> 172.21.15.27:0/389753985 conn(0x7f72580f0de0 msgr2=0x7f72580f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:42:00.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.290+0000 7f7260032700 1 -- 172.21.15.27:0/389753985 shutdown_connections 2021-11-03T17:42:00.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.290+0000 7f7260032700 1 -- 172.21.15.27:0/389753985 wait complete. 2021-11-03T17:42:00.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.291+0000 7f7260032700 1 Processor -- start 2021-11-03T17:42:00.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.291+0000 7f7260032700 1 -- start start 2021-11-03T17:42:00.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.291+0000 7f7260032700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72580f55e0 0x7f7258109d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:42:00.290 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.291+0000 7f7260032700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72580f5ef0 0x7f725810c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:42:00.290 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.291+0000 7f7260032700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7258060540 con 0x7f72580f5ef0 2021-11-03T17:42:00.290 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.291+0000 7f7260032700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7258060680 con 0x7f72580f55e0 2021-11-03T17:42:00.290 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.291+0000 7f725d5cd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72580f5ef0 0x7f725810c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:42:00.291 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.291+0000 7f725d5cd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72580f5ef0 0x7f725810c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58908/0 (socket says 172.21.15.27:58908) 2021-11-03T17:42:00.291 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.291+0000 7f725d5cd700 1 -- 172.21.15.27:0/1887834010 learned_addr learned my addr 172.21.15.27:0/1887834010 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:42:00.291 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.291+0000 7f725ddce700 1 --2- 172.21.15.27:0/1887834010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72580f55e0 0x7f7258109d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:42:00.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.291+0000 7f725d5cd700 1 -- 172.21.15.27:0/1887834010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72580f55e0 msgr2=0x7f7258109d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:42:00.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.291+0000 7f725d5cd700 1 --2- 172.21.15.27:0/1887834010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72580f55e0 0x7f7258109d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:42:00.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.291+0000 7f725d5cd700 1 -- 172.21.15.27:0/1887834010 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7250009640 con 0x7f72580f5ef0 2021-11-03T17:42:00.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.291+0000 7f725d5cd700 1 --2- 172.21.15.27:0/1887834010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72580f5ef0 0x7f725810c220 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7f724c00c530 tx=0x7f724c007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:42:00.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.292+0000 7f72567fc700 1 -- 172.21.15.27:0/1887834010 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f724c00f650 con 0x7f72580f5ef0 2021-11-03T17:42:00.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.292+0000 7f7260032700 1 -- 172.21.15.27:0/1887834010 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f725810c830 con 0x7f72580f5ef0 2021-11-03T17:42:00.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.292+0000 7f72567fc700 1 -- 172.21.15.27:0/1887834010 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f724c00fc60 con 0x7f72580f5ef0 2021-11-03T17:42:00.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.292+0000 7f72567fc700 1 -- 172.21.15.27:0/1887834010 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f724c012cd0 con 0x7f72580f5ef0 2021-11-03T17:42:00.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.292+0000 7f7260032700 1 -- 172.21.15.27:0/1887834010 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f725810cd40 con 0x7f72580f5ef0 2021-11-03T17:42:00.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.293+0000 7f7260032700 1 -- 172.21.15.27:0/1887834010 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72580f4a70 con 0x7f72580f5ef0 2021-11-03T17:42:00.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.293+0000 7f72567fc700 1 -- 172.21.15.27:0/1887834010 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f724c010070 con 0x7f72580f5ef0 2021-11-03T17:42:00.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.293+0000 7f72567fc700 1 --2- 172.21.15.27:0/1887834010 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f72400640b0 0x7f7240066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:42:00.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.294+0000 7f72567fc700 1 -- 172.21.15.27:0/1887834010 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f724c080670 con 0x7f72580f5ef0 2021-11-03T17:42:00.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.294+0000 7f725ddce700 1 --2- 172.21.15.27:0/1887834010 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f72400640b0 0x7f7240066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:42:00.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.294+0000 7f725ddce700 1 --2- 172.21.15.27:0/1887834010 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f72400640b0 0x7f7240066570 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f7250009f50 tx=0x7f7250004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:42:00.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.298+0000 7f72567fc700 1 -- 172.21.15.27:0/1887834010 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f724c052880 con 0x7f72580f5ef0 2021-11-03T17:42:00.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1562.396149 secs 2021-11-03T17:42:00.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:00 smithi115 ceph-mon[65453]: from='client.34952 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:42:00.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:00 smithi115 ceph-mon[65453]: from='client.34956 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:42:00.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:00 smithi115 ceph-mon[65453]: pgmap v821: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:00.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1562.396149 secs 2021-11-03T17:42:00.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:00 smithi027 ceph-mon[97244]: from='client.34952 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:42:00.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:00 smithi027 ceph-mon[97244]: from='client.34956 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:42:00.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:00 smithi027 ceph-mon[97244]: pgmap v821: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:00.524 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.529+0000 7f7260032700 1 -- 172.21.15.27:0/1887834010 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7258045ad0 con 0x7f72580f5ef0 2021-11-03T17:42:00.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.530+0000 7f72567fc700 1 -- 172.21.15.27:0/1887834010 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f724c052470 con 0x7f72580f5ef0 2021-11-03T17:42:00.525 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:42:00.525 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:42:00.525 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:42:00.526 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:42:00.526 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:42:00.526 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:42:00.527 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:42:00.527 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:42:00.527 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:42:00.527 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:42:00.528 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:42:00.528 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:42:00.528 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:42:00.528 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:42:00.529 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:42:00.529 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:42:00.529 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:42:00.529 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:42:00.530 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.532+0000 7f7260032700 1 -- 172.21.15.27:0/1887834010 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f72400640b0 msgr2=0x7f7240066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:42:00.530 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.532+0000 7f7260032700 1 --2- 172.21.15.27:0/1887834010 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f72400640b0 0x7f7240066570 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7f7250009f50 tx=0x7f7250004d10).stop 2021-11-03T17:42:00.531 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.532+0000 7f7260032700 1 -- 172.21.15.27:0/1887834010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72580f5ef0 msgr2=0x7f725810c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:42:00.531 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.532+0000 7f7260032700 1 --2- 172.21.15.27:0/1887834010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72580f5ef0 0x7f725810c220 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7f724c00c530 tx=0x7f724c007e20).stop 2021-11-03T17:42:00.531 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.532+0000 7f7260032700 1 -- 172.21.15.27:0/1887834010 shutdown_connections 2021-11-03T17:42:00.531 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.532+0000 7f7260032700 1 --2- 172.21.15.27:0/1887834010 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f72400640b0 0x7f7240066570 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:42:00.532 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.532+0000 7f7260032700 1 --2- 172.21.15.27:0/1887834010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72580f55e0 0x7f7258109d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:42:00.532 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.532+0000 7f7260032700 1 --2- 172.21.15.27:0/1887834010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72580f5ef0 0x7f725810c220 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:42:00.532 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.532+0000 7f7260032700 1 -- 172.21.15.27:0/1887834010 >> 172.21.15.27:0/1887834010 conn(0x7f72580f0de0 msgr2=0x7f72580fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:42:00.532 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.532+0000 7f7260032700 1 -- 172.21.15.27:0/1887834010 shutdown_connections 2021-11-03T17:42:00.533 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.532+0000 7f7260032700 1 -- 172.21.15.27:0/1887834010 wait complete. 2021-11-03T17:42:00.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.633+0000 7f33c6741700 1 -- 172.21.15.27:0/3800847388 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33c00f5440 msgr2=0x7f33c00f5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:42:00.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.633+0000 7f33c6741700 1 --2- 172.21.15.27:0/3800847388 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33c00f5440 0x7f33c00f5860 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f33bc004660 tx=0x7f33bc009b30).stop 2021-11-03T17:42:00.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.633+0000 7f33c6741700 1 -- 172.21.15.27:0/3800847388 shutdown_connections 2021-11-03T17:42:00.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.633+0000 7f33c6741700 1 --2- 172.21.15.27:0/3800847388 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33c00f66e0 0x7f33c00f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:42:00.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.633+0000 7f33c6741700 1 --2- 172.21.15.27:0/3800847388 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33c00f5440 0x7f33c00f5860 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:42:00.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.633+0000 7f33c6741700 1 -- 172.21.15.27:0/3800847388 >> 172.21.15.27:0/3800847388 conn(0x7f33c00f0ba0 msgr2=0x7f33c00f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:42:00.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.633+0000 7f33c6741700 1 -- 172.21.15.27:0/3800847388 shutdown_connections 2021-11-03T17:42:00.637 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.633+0000 7f33c6741700 1 -- 172.21.15.27:0/3800847388 wait complete. 2021-11-03T17:42:00.637 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.633+0000 7f33c6741700 1 Processor -- start 2021-11-03T17:42:00.638 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.633+0000 7f33c6741700 1 -- start start 2021-11-03T17:42:00.638 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.634+0000 7f33c6741700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33c00f5440 0x7f33c00fd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:42:00.638 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.634+0000 7f33c6741700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33c00f66e0 0x7f33c00fba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:42:00.639 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.634+0000 7f33c6741700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c00fbfa0 con 0x7f33c00f5440 2021-11-03T17:42:00.639 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.634+0000 7f33c6741700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33c00fc0e0 con 0x7f33c00f66e0 2021-11-03T17:42:00.640 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.634+0000 7f33c573f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33c00f5440 0x7f33c00fd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:42:00.640 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.634+0000 7f33c573f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33c00f5440 0x7f33c00fd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58914/0 (socket says 172.21.15.27:58914) 2021-11-03T17:42:00.640 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.634+0000 7f33c573f700 1 -- 172.21.15.27:0/1910853794 learned_addr learned my addr 172.21.15.27:0/1910853794 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:42:00.641 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.634+0000 7f33c4f3e700 1 --2- 172.21.15.27:0/1910853794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33c00f66e0 0x7f33c00fba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:42:00.641 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.634+0000 7f33c573f700 1 -- 172.21.15.27:0/1910853794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33c00f66e0 msgr2=0x7f33c00fba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:42:00.642 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.634+0000 7f33c573f700 1 --2- 172.21.15.27:0/1910853794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33c00f66e0 0x7f33c00fba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:42:00.642 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.634+0000 7f33c573f700 1 -- 172.21.15.27:0/1910853794 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33bc005040 con 0x7f33c00f5440 2021-11-03T17:42:00.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.634+0000 7f33c4f3e700 1 --2- 172.21.15.27:0/1910853794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33c00f66e0 0x7f33c00fba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:42:00.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.634+0000 7f33c573f700 1 --2- 172.21.15.27:0/1910853794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33c00f5440 0x7f33c00fd5c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7f33bc014040 tx=0x7f33bc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:42:00.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.634+0000 7f33b5ffb700 1 -- 172.21.15.27:0/1910853794 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33bc019070 con 0x7f33c00f5440 2021-11-03T17:42:00.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.634+0000 7f33b5ffb700 1 -- 172.21.15.27:0/1910853794 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f33bc007e70 con 0x7f33c00f5440 2021-11-03T17:42:00.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.634+0000 7f33c6741700 1 -- 172.21.15.27:0/1910853794 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33c00fc360 con 0x7f33c00f5440 2021-11-03T17:42:00.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.634+0000 7f33b5ffb700 1 -- 172.21.15.27:0/1910853794 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33bc014d50 con 0x7f33c00f5440 2021-11-03T17:42:00.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.635+0000 7f33c6741700 1 -- 172.21.15.27:0/1910853794 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33c00fc8c0 con 0x7f33c00f5440 2021-11-03T17:42:00.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.636+0000 7f33c6741700 1 -- 172.21.15.27:0/1910853794 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33c0045ad0 con 0x7f33c00f5440 2021-11-03T17:42:00.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.636+0000 7f33b5ffb700 1 -- 172.21.15.27:0/1910853794 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f33bc025070 con 0x7f33c00f5440 2021-11-03T17:42:00.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.636+0000 7f33b5ffb700 1 --2- 172.21.15.27:0/1910853794 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f33ac064100 0x7f33ac0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:42:00.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.636+0000 7f33b5ffb700 1 -- 172.21.15.27:0/1910853794 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f33bc081820 con 0x7f33c00f5440 2021-11-03T17:42:00.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.636+0000 7f33c4f3e700 1 --2- 172.21.15.27:0/1910853794 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f33ac064100 0x7f33ac0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:42:00.648 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.637+0000 7f33c4f3e700 1 --2- 172.21.15.27:0/1910853794 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f33ac064100 0x7f33ac0665c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f33b0009fd0 tx=0x7f33b000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:42:00.649 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.640+0000 7f33b5ffb700 1 -- 172.21.15.27:0/1910853794 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f33bc053b20 con 0x7f33c00f5440 2021-11-03T17:42:00.840 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.845+0000 7f33c6741700 1 -- 172.21.15.27:0/1910853794 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f33c0027a00 con 0x7f33c00f5440 2021-11-03T17:42:00.841 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.846+0000 7f33b5ffb700 1 -- 172.21.15.27:0/1910853794 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f33bc023070 con 0x7f33c00f5440 2021-11-03T17:42:00.841 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:42:00.842 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:42:00.842 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:42:00.842 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:42:00.842 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:42:00.843 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:42:00.843 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:42:00.843 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:42:00.843 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:42:00.843 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:42:00.844 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:42:00.844 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:42:00.844 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:42:00.844 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:42:00.844 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:42:00.845 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:42:00.845 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:42:00.845 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:42:00.845 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:42:00.846 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:42:00.846 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:42:00.846 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:42:00.846 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:42:00.847 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:42:00.847 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:42:00.847 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:42:00.847 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:42:00.847 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:42:00.848 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:42:00.848 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:42:00.848 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:42:00.848 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:42:00.849 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:42:00.849 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:42:00.849 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:42:00.849 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:42:00.849 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:42:00.850 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:42:00.850 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:42:00.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.847+0000 7f33c6741700 1 -- 172.21.15.27:0/1910853794 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f33ac064100 msgr2=0x7f33ac0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:42:00.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.847+0000 7f33c6741700 1 --2- 172.21.15.27:0/1910853794 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f33ac064100 0x7f33ac0665c0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 rx=0x7f33b0009fd0 tx=0x7f33b000b040).stop 2021-11-03T17:42:00.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.847+0000 7f33c6741700 1 -- 172.21.15.27:0/1910853794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33c00f5440 msgr2=0x7f33c00fd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:42:00.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.847+0000 7f33c6741700 1 --2- 172.21.15.27:0/1910853794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33c00f5440 0x7f33c00fd5c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7f33bc014040 tx=0x7f33bc00b040).stop 2021-11-03T17:42:00.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.847+0000 7f33c6741700 1 -- 172.21.15.27:0/1910853794 shutdown_connections 2021-11-03T17:42:00.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.847+0000 7f33c6741700 1 --2- 172.21.15.27:0/1910853794 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f33ac064100 0x7f33ac0665c0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:42:00.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.847+0000 7f33c6741700 1 --2- 172.21.15.27:0/1910853794 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33c00f5440 0x7f33c00fd5c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:42:00.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.847+0000 7f33c6741700 1 --2- 172.21.15.27:0/1910853794 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33c00f66e0 0x7f33c00fba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:42:00.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.847+0000 7f33c6741700 1 -- 172.21.15.27:0/1910853794 >> 172.21.15.27:0/1910853794 conn(0x7f33c00f0ba0 msgr2=0x7f33c00fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:42:00.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.847+0000 7f33c6741700 1 -- 172.21.15.27:0/1910853794 shutdown_connections 2021-11-03T17:42:00.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:00.847+0000 7f33c6741700 1 -- 172.21.15.27:0/1910853794 wait complete. 2021-11-03T17:42:00.854 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:42:01.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:01 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1887834010' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:01.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:01 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1910853794' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:42:01.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:01 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1887834010' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:01.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:01 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1910853794' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:42:02.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:02 smithi115 ceph-mon[65453]: pgmap v822: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:02.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:02 smithi027 ceph-mon[97244]: pgmap v822: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:04.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:04 smithi027 ceph-mon[97244]: pgmap v823: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:04.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:04 smithi115 ceph-mon[65453]: pgmap v823: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:05.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1567.396290 secs 2021-11-03T17:42:05.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1567.396290 secs 2021-11-03T17:42:06.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:06 smithi027 ceph-mon[97244]: pgmap v824: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:06.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:06 smithi115 ceph-mon[65453]: pgmap v824: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:08.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:08 smithi027 ceph-mon[97244]: pgmap v825: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:08.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:42:08.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:42:08.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:08 smithi115 ceph-mon[65453]: pgmap v825: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:08.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:42:08.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:42:09.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:42:09.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:09.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:09.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:09.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:09.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:09 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:42:09.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:42:09.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:09.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:09.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:09.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:09.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:09 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:42:10.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1572.396378 secs 2021-11-03T17:42:10.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:10 smithi027 ceph-mon[97244]: pgmap v826: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:10.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1572.396378 secs 2021-11-03T17:42:10.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:10 smithi115 ceph-mon[65453]: pgmap v826: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:12.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:12 smithi027 ceph-mon[97244]: pgmap v827: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:12.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:12 smithi115 ceph-mon[65453]: pgmap v827: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:14.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:14 smithi027 ceph-mon[97244]: pgmap v828: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:14.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:14 smithi115 ceph-mon[65453]: pgmap v828: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:15.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1577.396469 secs 2021-11-03T17:42:15.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1577.396469 secs 2021-11-03T17:42:16.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:16 smithi027 ceph-mon[97244]: pgmap v829: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:16.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:16 smithi115 ceph-mon[65453]: pgmap v829: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:18.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:18 smithi027 ceph-mon[97244]: pgmap v830: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:18.631 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:18 smithi115 ceph-mon[65453]: pgmap v830: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:19.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:42:19.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:19 smithi027 ceph-mon[97244]: poll took 60190.127 ms: 1 events 2021-11-03T17:42:19.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:42:19.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:42:19.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:19.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:19.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:19.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:19.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:19 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:42:19.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:42:19.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:19 smithi115 ceph-mon[65453]: poll took 60190.127 ms: 1 events 2021-11-03T17:42:19.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:42:19.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:42:19.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:19.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:19.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:19.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:19.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:19 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:42:20.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1582.396613 secs 2021-11-03T17:42:20.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:20 smithi027 ceph-mon[97244]: pgmap v831: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:20.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1582.396613 secs 2021-11-03T17:42:20.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:20 smithi115 ceph-mon[65453]: pgmap v831: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:22.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:22 smithi027 ceph-mon[97244]: pgmap v832: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:22.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:22 smithi115 ceph-mon[65453]: pgmap v832: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:24.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:24 smithi027 ceph-mon[97244]: pgmap v833: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:24.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:24 smithi115 ceph-mon[65453]: pgmap v833: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:25.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1587.396728 secs 2021-11-03T17:42:25.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1587.396728 secs 2021-11-03T17:42:26.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:26 smithi027 ceph-mon[97244]: pgmap v834: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:26.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:26 smithi115 ceph-mon[65453]: pgmap v834: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:28.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:28 smithi027 ceph-mon[97244]: pgmap v835: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:28.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:28 smithi115 ceph-mon[65453]: pgmap v835: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:29.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:42:29.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:42:29.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:42:29.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:29.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:29.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:29.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:29.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:29 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:42:29.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:42:29.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:42:29.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:42:29.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:29.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:29.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:29.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:29.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:29 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:42:30.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1592.396871 secs 2021-11-03T17:42:30.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:30 smithi027 ceph-mon[97244]: pgmap v836: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:30.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1592.396871 secs 2021-11-03T17:42:30.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:30 smithi115 ceph-mon[65453]: pgmap v836: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:30.959 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:30.962+0000 7f71f9294700 1 -- 172.21.15.27:0/3849387911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71f40f5ef0 msgr2=0x7f71f40f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:42:30.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:30.962+0000 7f71f9294700 1 --2- 172.21.15.27:0/3849387911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71f40f5ef0 0x7f71f40f6920 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f71dc004660 tx=0x7f71dc009b30).stop 2021-11-03T17:42:30.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:30.962+0000 7f71f9294700 1 -- 172.21.15.27:0/3849387911 shutdown_connections 2021-11-03T17:42:30.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:30.962+0000 7f71f9294700 1 --2- 172.21.15.27:0/3849387911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71f40f5ef0 0x7f71f40f6920 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:42:30.961 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:30.962+0000 7f71f9294700 1 --2- 172.21.15.27:0/3849387911 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71f40f55e0 0x7f71f40f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:42:30.961 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:30.962+0000 7f71f9294700 1 -- 172.21.15.27:0/3849387911 >> 172.21.15.27:0/3849387911 conn(0x7f71f40f0de0 msgr2=0x7f71f40f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:42:30.961 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:30.962+0000 7f71f9294700 1 -- 172.21.15.27:0/3849387911 shutdown_connections 2021-11-03T17:42:30.961 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:30.962+0000 7f71f9294700 1 -- 172.21.15.27:0/3849387911 wait complete. 2021-11-03T17:42:30.962 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:30.963+0000 7f71f9294700 1 Processor -- start 2021-11-03T17:42:30.962 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:30.963+0000 7f71f9294700 1 -- start start 2021-11-03T17:42:30.962 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:30.963+0000 7f71f9294700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71f40f55e0 0x7f71f4109c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:42:30.962 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:30.963+0000 7f71f9294700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71f40f5ef0 0x7f71f410c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:42:30.963 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:30.963+0000 7f71f9294700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f40604e0 con 0x7f71f40f55e0 2021-11-03T17:42:30.963 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:30.963+0000 7f71f9294700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f4060620 con 0x7f71f40f5ef0 2021-11-03T17:42:30.963 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:30.963+0000 7f71f2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71f40f55e0 0x7f71f4109c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:42:30.963 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:30.963+0000 7f71f2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71f40f55e0 0x7f71f4109c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58932/0 (socket says 172.21.15.27:58932) 2021-11-03T17:42:30.963 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:30.963+0000 7f71f2ffd700 1 -- 172.21.15.27:0/882059714 learned_addr learned my addr 172.21.15.27:0/882059714 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:42:30.964 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:30.964+0000 7f71f27fc700 1 --2- 172.21.15.27:0/882059714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71f40f5ef0 0x7f71f410c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:42:30.964 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:30.964+0000 7f71f2ffd700 1 -- 172.21.15.27:0/882059714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71f40f5ef0 msgr2=0x7f71f410c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:42:30.964 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:30.964+0000 7f71f2ffd700 1 --2- 172.21.15.27:0/882059714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71f40f5ef0 0x7f71f410c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:42:30.964 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:30.964+0000 7f71f2ffd700 1 -- 172.21.15.27:0/882059714 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71dc005040 con 0x7f71f40f55e0 2021-11-03T17:42:30.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:30.964+0000 7f71f2ffd700 1 --2- 172.21.15.27:0/882059714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71f40f55e0 0x7f71f4109c90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f71ec007fd0 tx=0x7f71ec004ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:42:30.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:30.964+0000 7f71eb7fe700 1 -- 172.21.15.27:0/882059714 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71ec00b4f0 con 0x7f71f40f55e0 2021-11-03T17:42:30.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:30.964+0000 7f71eb7fe700 1 -- 172.21.15.27:0/882059714 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f71ec00f040 con 0x7f71f40f55e0 2021-11-03T17:42:30.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:30.964+0000 7f71f9294700 1 -- 172.21.15.27:0/882059714 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f410c740 con 0x7f71f40f55e0 2021-11-03T17:42:30.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:30.964+0000 7f71eb7fe700 1 -- 172.21.15.27:0/882059714 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71ec013a50 con 0x7f71f40f55e0 2021-11-03T17:42:30.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:30.964+0000 7f71f9294700 1 -- 172.21.15.27:0/882059714 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f410cd00 con 0x7f71f40f55e0 2021-11-03T17:42:30.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:30.965+0000 7f71f9294700 1 -- 172.21.15.27:0/882059714 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71f4045ad0 con 0x7f71f40f55e0 2021-11-03T17:42:30.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:30.966+0000 7f71eb7fe700 1 -- 172.21.15.27:0/882059714 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f71ec01a030 con 0x7f71f40f55e0 2021-11-03T17:42:30.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:30.966+0000 7f71eb7fe700 1 --2- 172.21.15.27:0/882059714 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f71e00640b0 0x7f71e0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:42:30.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:30.966+0000 7f71eb7fe700 1 -- 172.21.15.27:0/882059714 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f71ec0807f0 con 0x7f71f40f55e0 2021-11-03T17:42:30.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:30.966+0000 7f71f27fc700 1 --2- 172.21.15.27:0/882059714 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f71e00640b0 0x7f71e0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:42:30.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:30.967+0000 7f71f27fc700 1 --2- 172.21.15.27:0/882059714 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f71e00640b0 0x7f71e0066570 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f71dc004630 tx=0x7f71dc009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:42:30.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:30.970+0000 7f71eb7fe700 1 -- 172.21.15.27:0/882059714 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f71ec052af0 con 0x7f71f40f55e0 2021-11-03T17:42:31.148 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.153+0000 7f71f9294700 1 -- 172.21.15.27:0/882059714 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f71f40fdf80 con 0x7f71e00640b0 2021-11-03T17:42:31.149 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.154+0000 7f71eb7fe700 1 -- 172.21.15.27:0/882059714 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f71dc019040 con 0x7f71e00640b0 2021-11-03T17:42:31.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.156+0000 7f71f9294700 1 -- 172.21.15.27:0/882059714 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f71e00640b0 msgr2=0x7f71e0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:42:31.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.156+0000 7f71f9294700 1 --2- 172.21.15.27:0/882059714 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f71e00640b0 0x7f71e0066570 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f71dc004630 tx=0x7f71dc009b30).stop 2021-11-03T17:42:31.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.156+0000 7f71f9294700 1 -- 172.21.15.27:0/882059714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71f40f55e0 msgr2=0x7f71f4109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:42:31.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.156+0000 7f71f9294700 1 --2- 172.21.15.27:0/882059714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71f40f55e0 0x7f71f4109c90 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f71ec007fd0 tx=0x7f71ec004ee0).stop 2021-11-03T17:42:31.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.156+0000 7f71f9294700 1 -- 172.21.15.27:0/882059714 shutdown_connections 2021-11-03T17:42:31.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.156+0000 7f71f9294700 1 --2- 172.21.15.27:0/882059714 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f71e00640b0 0x7f71e0066570 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:42:31.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.156+0000 7f71f9294700 1 --2- 172.21.15.27:0/882059714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71f40f55e0 0x7f71f4109c90 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:42:31.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.156+0000 7f71f9294700 1 --2- 172.21.15.27:0/882059714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71f40f5ef0 0x7f71f410c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:42:31.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.156+0000 7f71f9294700 1 -- 172.21.15.27:0/882059714 >> 172.21.15.27:0/882059714 conn(0x7f71f40f0de0 msgr2=0x7f71f40fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:42:31.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.156+0000 7f71f9294700 1 -- 172.21.15.27:0/882059714 shutdown_connections 2021-11-03T17:42:31.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.156+0000 7f71f9294700 1 -- 172.21.15.27:0/882059714 wait complete. 2021-11-03T17:42:31.160 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:42:31.253 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.256+0000 7fafaa302700 1 -- 172.21.15.27:0/1858162036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafa406b920 msgr2=0x7fafa4069ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:42:31.253 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.256+0000 7fafaa302700 1 --2- 172.21.15.27:0/1858162036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafa406b920 0x7fafa4069ec0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7faf98004660 tx=0x7faf98009b30).stop 2021-11-03T17:42:31.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.256+0000 7fafaa302700 1 -- 172.21.15.27:0/1858162036 shutdown_connections 2021-11-03T17:42:31.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.256+0000 7fafaa302700 1 --2- 172.21.15.27:0/1858162036 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafa406a3b0 0x7fafa406a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:42:31.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.256+0000 7fafaa302700 1 --2- 172.21.15.27:0/1858162036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafa406b920 0x7fafa4069ec0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:42:31.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.256+0000 7fafaa302700 1 -- 172.21.15.27:0/1858162036 >> 172.21.15.27:0/1858162036 conn(0x7fafa40f1f00 msgr2=0x7fafa40f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:42:31.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.256+0000 7fafaa302700 1 -- 172.21.15.27:0/1858162036 shutdown_connections 2021-11-03T17:42:31.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.256+0000 7fafaa302700 1 -- 172.21.15.27:0/1858162036 wait complete. 2021-11-03T17:42:31.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.256+0000 7fafaa302700 1 Processor -- start 2021-11-03T17:42:31.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.256+0000 7fafaa302700 1 -- start start 2021-11-03T17:42:31.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.257+0000 7fafaa302700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafa406a3b0 0x7fafa410d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:42:31.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.257+0000 7fafaa302700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafa406b920 0x7fafa410dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:42:31.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.257+0000 7fafaa302700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa410e070 con 0x7fafa406b920 2021-11-03T17:42:31.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.257+0000 7fafaa302700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafa410e1b0 con 0x7fafa406a3b0 2021-11-03T17:42:31.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.257+0000 7fafa37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafa406b920 0x7fafa410dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:42:31.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.257+0000 7fafa37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafa406b920 0x7fafa410dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58942/0 (socket says 172.21.15.27:58942) 2021-11-03T17:42:31.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.257+0000 7fafa37fe700 1 -- 172.21.15.27:0/1220370076 learned_addr learned my addr 172.21.15.27:0/1220370076 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:42:31.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.257+0000 7fafa3fff700 1 --2- 172.21.15.27:0/1220370076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafa406a3b0 0x7fafa410d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:42:31.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.257+0000 7fafa37fe700 1 -- 172.21.15.27:0/1220370076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafa406a3b0 msgr2=0x7fafa410d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:42:31.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.257+0000 7fafa37fe700 1 --2- 172.21.15.27:0/1220370076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafa406a3b0 0x7fafa410d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:42:31.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.257+0000 7fafa37fe700 1 -- 172.21.15.27:0/1220370076 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf98005040 con 0x7fafa406b920 2021-11-03T17:42:31.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.257+0000 7fafa3fff700 1 --2- 172.21.15.27:0/1220370076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafa406a3b0 0x7fafa410d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:42:31.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.257+0000 7fafa37fe700 1 --2- 172.21.15.27:0/1220370076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafa406b920 0x7fafa410dac0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7faf940026d0 tx=0x7faf94009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:42:31.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.257+0000 7fafa0ff9700 1 -- 172.21.15.27:0/1220370076 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf94009320 con 0x7fafa406b920 2021-11-03T17:42:31.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.257+0000 7fafaa302700 1 -- 172.21.15.27:0/1220370076 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafa410eb50 con 0x7fafa406b920 2021-11-03T17:42:31.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.257+0000 7fafa0ff9700 1 -- 172.21.15.27:0/1220370076 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7faf9400f040 con 0x7fafa406b920 2021-11-03T17:42:31.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.257+0000 7fafa0ff9700 1 -- 172.21.15.27:0/1220370076 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf940147e0 con 0x7fafa406b920 2021-11-03T17:42:31.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.257+0000 7fafaa302700 1 -- 172.21.15.27:0/1220370076 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafa410f110 con 0x7fafa406b920 2021-11-03T17:42:31.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.258+0000 7fafaa302700 1 -- 172.21.15.27:0/1220370076 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafa4045ad0 con 0x7fafa406b920 2021-11-03T17:42:31.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.259+0000 7fafa0ff9700 1 -- 172.21.15.27:0/1220370076 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7faf94009480 con 0x7fafa406b920 2021-11-03T17:42:31.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.259+0000 7fafa0ff9700 1 --2- 172.21.15.27:0/1220370076 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7faf8c0640b0 0x7faf8c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:42:31.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.259+0000 7fafa0ff9700 1 -- 172.21.15.27:0/1220370076 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7faf94080fd0 con 0x7fafa406b920 2021-11-03T17:42:31.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.259+0000 7fafa3fff700 1 --2- 172.21.15.27:0/1220370076 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7faf8c0640b0 0x7faf8c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:42:31.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.260+0000 7fafa3fff700 1 --2- 172.21.15.27:0/1220370076 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7faf8c0640b0 0x7faf8c066570 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7faf98005010 tx=0x7faf98009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:42:31.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.263+0000 7fafa0ff9700 1 -- 172.21.15.27:0/1220370076 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7faf94053210 con 0x7fafa406b920 2021-11-03T17:42:31.432 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.437+0000 7fafaa302700 1 -- 172.21.15.27:0/1220370076 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fafa410f390 con 0x7faf8c0640b0 2021-11-03T17:42:31.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.451+0000 7fafa0ff9700 1 -- 172.21.15.27:0/1220370076 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fafa410f390 con 0x7faf8c0640b0 2021-11-03T17:42:31.446 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:42:31.447 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (34m) 3m ago 35m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:42:31.447 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (25m) 3m ago 35m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:42:31.447 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (25m) 2m ago 34m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:42:31.448 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (33m) 3m ago 35m 41.3M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:42:31.448 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (27m) 3m ago 30m 21.1M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:42:31.448 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (30m) 3m ago 30m 3843M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:42:31.448 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (26m) 2m ago 30m 20.4M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:42:31.449 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (27m) 2m ago 30m 22.1M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:42:31.449 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (27m) 3m ago 36m 517M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:42:31.449 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (27m) 2m ago 34m 419M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:42:31.449 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (26m) 3m ago 36m 159M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:42:31.450 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (26m) 2m ago 34m 82.5M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:42:31.450 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (34m) 3m ago 34m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:42:31.450 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (34m) 2m ago 34m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:42:31.450 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (25m) 3m ago 33m 311M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:42:31.451 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (24m) 3m ago 33m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:42:31.451 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (24m) 3m ago 33m 248M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:42:31.451 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (23m) 2m ago 32m 354M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:42:31.451 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (23m) 2m ago 32m 285M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:42:31.452 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (23m) 2m ago 32m 287M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:42:31.452 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (33m) 3m ago 34m 50.5M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:42:31.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.453+0000 7fafaa302700 1 -- 172.21.15.27:0/1220370076 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7faf8c0640b0 msgr2=0x7faf8c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:42:31.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.453+0000 7fafaa302700 1 --2- 172.21.15.27:0/1220370076 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7faf8c0640b0 0x7faf8c066570 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7faf98005010 tx=0x7faf98009b30).stop 2021-11-03T17:42:31.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.453+0000 7fafaa302700 1 -- 172.21.15.27:0/1220370076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafa406b920 msgr2=0x7fafa410dac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:42:31.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.453+0000 7fafaa302700 1 --2- 172.21.15.27:0/1220370076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafa406b920 0x7fafa410dac0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7faf940026d0 tx=0x7faf94009de0).stop 2021-11-03T17:42:31.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.453+0000 7fafaa302700 1 -- 172.21.15.27:0/1220370076 shutdown_connections 2021-11-03T17:42:31.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.453+0000 7fafaa302700 1 --2- 172.21.15.27:0/1220370076 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7faf8c0640b0 0x7faf8c066570 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:42:31.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.453+0000 7fafaa302700 1 --2- 172.21.15.27:0/1220370076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafa406a3b0 0x7fafa410d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:42:31.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.453+0000 7fafaa302700 1 --2- 172.21.15.27:0/1220370076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafa406b920 0x7fafa410dac0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:42:31.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.453+0000 7fafaa302700 1 -- 172.21.15.27:0/1220370076 >> 172.21.15.27:0/1220370076 conn(0x7fafa40f1f00 msgr2=0x7fafa40f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:42:31.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.453+0000 7fafaa302700 1 -- 172.21.15.27:0/1220370076 shutdown_connections 2021-11-03T17:42:31.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.453+0000 7fafaa302700 1 -- 172.21.15.27:0/1220370076 wait complete. 2021-11-03T17:42:31.550 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.553+0000 7f0c4dbaa700 1 -- 172.21.15.27:0/2847351954 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c480f55c0 msgr2=0x7f0c480f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:42:31.550 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.553+0000 7f0c4dbaa700 1 --2- 172.21.15.27:0/2847351954 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c480f55c0 0x7f0c480f59e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f0c400099e0 tx=0x7f0c40004d10).stop 2021-11-03T17:42:31.550 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.553+0000 7f0c4dbaa700 1 -- 172.21.15.27:0/2847351954 shutdown_connections 2021-11-03T17:42:31.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.553+0000 7f0c4dbaa700 1 --2- 172.21.15.27:0/2847351954 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c480f6860 0x7f0c480f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:42:31.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.553+0000 7f0c4dbaa700 1 --2- 172.21.15.27:0/2847351954 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c480f55c0 0x7f0c480f59e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:42:31.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.553+0000 7f0c4dbaa700 1 -- 172.21.15.27:0/2847351954 >> 172.21.15.27:0/2847351954 conn(0x7f0c480f0d40 msgr2=0x7f0c480f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:42:31.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.553+0000 7f0c4dbaa700 1 -- 172.21.15.27:0/2847351954 shutdown_connections 2021-11-03T17:42:31.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.553+0000 7f0c4dbaa700 1 -- 172.21.15.27:0/2847351954 wait complete. 2021-11-03T17:42:31.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.553+0000 7f0c4dbaa700 1 Processor -- start 2021-11-03T17:42:31.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.553+0000 7f0c4dbaa700 1 -- start start 2021-11-03T17:42:31.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.554+0000 7f0c4dbaa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c480f55c0 0x7f0c48109d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:42:31.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.554+0000 7f0c4dbaa700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c480f6860 0x7f0c4810c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:42:31.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.554+0000 7f0c4dbaa700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c480603e0 con 0x7f0c480f55c0 2021-11-03T17:42:31.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.554+0000 7f0c4dbaa700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c48060520 con 0x7f0c480f6860 2021-11-03T17:42:31.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.554+0000 7f0c477fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c480f55c0 0x7f0c48109d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:42:31.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.554+0000 7f0c477fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c480f55c0 0x7f0c48109d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58950/0 (socket says 172.21.15.27:58950) 2021-11-03T17:42:31.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.554+0000 7f0c477fe700 1 -- 172.21.15.27:0/1033397853 learned_addr learned my addr 172.21.15.27:0/1033397853 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:42:31.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.554+0000 7f0c46ffd700 1 --2- 172.21.15.27:0/1033397853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c480f6860 0x7f0c4810c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:42:31.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.554+0000 7f0c477fe700 1 -- 172.21.15.27:0/1033397853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c480f6860 msgr2=0x7f0c4810c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:42:31.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.554+0000 7f0c477fe700 1 --2- 172.21.15.27:0/1033397853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c480f6860 0x7f0c4810c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:42:31.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.554+0000 7f0c477fe700 1 -- 172.21.15.27:0/1033397853 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c40009640 con 0x7f0c480f55c0 2021-11-03T17:42:31.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.554+0000 7f0c477fe700 1 --2- 172.21.15.27:0/1033397853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c480f55c0 0x7f0c48109d60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f0c4000c010 tx=0x7f0c40004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:42:31.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.554+0000 7f0c3ffff700 1 -- 172.21.15.27:0/1033397853 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c4000e070 con 0x7f0c480f55c0 2021-11-03T17:42:31.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.554+0000 7f0c4dbaa700 1 -- 172.21.15.27:0/1033397853 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c4810c7b0 con 0x7f0c480f55c0 2021-11-03T17:42:31.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.554+0000 7f0c3ffff700 1 -- 172.21.15.27:0/1033397853 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f0c40007dc0 con 0x7f0c480f55c0 2021-11-03T17:42:31.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.554+0000 7f0c3ffff700 1 -- 172.21.15.27:0/1033397853 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c40018070 con 0x7f0c480f55c0 2021-11-03T17:42:31.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.554+0000 7f0c4dbaa700 1 -- 172.21.15.27:0/1033397853 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c4810cc90 con 0x7f0c480f55c0 2021-11-03T17:42:31.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.555+0000 7f0c4dbaa700 1 -- 172.21.15.27:0/1033397853 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c48045ad0 con 0x7f0c480f55c0 2021-11-03T17:42:31.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.556+0000 7f0c3ffff700 1 -- 172.21.15.27:0/1033397853 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f0c40024060 con 0x7f0c480f55c0 2021-11-03T17:42:31.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.556+0000 7f0c3ffff700 1 --2- 172.21.15.27:0/1033397853 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0c340640b0 0x7f0c34066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:42:31.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.556+0000 7f0c3ffff700 1 -- 172.21.15.27:0/1033397853 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f0c40082060 con 0x7f0c480f55c0 2021-11-03T17:42:31.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.556+0000 7f0c46ffd700 1 --2- 172.21.15.27:0/1033397853 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0c340640b0 0x7f0c34066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:42:31.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.557+0000 7f0c46ffd700 1 --2- 172.21.15.27:0/1033397853 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0c340640b0 0x7f0c34066570 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f0c30009740 tx=0x7f0c30009c60).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:42:31.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.560+0000 7f0c3ffff700 1 -- 172.21.15.27:0/1033397853 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f0c40054360 con 0x7f0c480f55c0 2021-11-03T17:42:31.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.790+0000 7f0c4dbaa700 1 -- 172.21.15.27:0/1033397853 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0c480fdec0 con 0x7f0c480f55c0 2021-11-03T17:42:31.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.791+0000 7f0c3ffff700 1 -- 172.21.15.27:0/1033397853 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f0c400222e0 con 0x7f0c480f55c0 2021-11-03T17:42:31.786 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:42:31.787 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:42:31.787 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:42:31.787 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:42:31.787 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:42:31.788 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:42:31.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:42:31.788 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:42:31.788 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:42:31.788 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:42:31.789 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:42:31.789 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:42:31.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:42:31.789 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:42:31.790 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:42:31.790 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:42:31.790 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:42:31.790 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:42:31.791 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.793+0000 7f0c4dbaa700 1 -- 172.21.15.27:0/1033397853 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0c340640b0 msgr2=0x7f0c34066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:42:31.792 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.793+0000 7f0c4dbaa700 1 --2- 172.21.15.27:0/1033397853 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0c340640b0 0x7f0c34066570 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7f0c30009740 tx=0x7f0c30009c60).stop 2021-11-03T17:42:31.792 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.793+0000 7f0c4dbaa700 1 -- 172.21.15.27:0/1033397853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c480f55c0 msgr2=0x7f0c48109d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:42:31.792 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.793+0000 7f0c4dbaa700 1 --2- 172.21.15.27:0/1033397853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c480f55c0 0x7f0c48109d60 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7f0c4000c010 tx=0x7f0c40004d10).stop 2021-11-03T17:42:31.792 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.793+0000 7f0c4dbaa700 1 -- 172.21.15.27:0/1033397853 shutdown_connections 2021-11-03T17:42:31.792 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.793+0000 7f0c4dbaa700 1 --2- 172.21.15.27:0/1033397853 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0c340640b0 0x7f0c34066570 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:42:31.793 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.793+0000 7f0c4dbaa700 1 --2- 172.21.15.27:0/1033397853 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c480f55c0 0x7f0c48109d60 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:42:31.793 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.793+0000 7f0c4dbaa700 1 --2- 172.21.15.27:0/1033397853 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c480f6860 0x7f0c4810c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:42:31.793 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.793+0000 7f0c4dbaa700 1 -- 172.21.15.27:0/1033397853 >> 172.21.15.27:0/1033397853 conn(0x7f0c480f0d40 msgr2=0x7f0c480fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:42:31.793 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.793+0000 7f0c4dbaa700 1 -- 172.21.15.27:0/1033397853 shutdown_connections 2021-11-03T17:42:31.794 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.793+0000 7f0c4dbaa700 1 -- 172.21.15.27:0/1033397853 wait complete. 2021-11-03T17:42:31.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.893+0000 7fc73c74f700 1 -- 172.21.15.27:0/3592130953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7340f66e0 msgr2=0x7fc7340f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:42:31.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.893+0000 7fc73c74f700 1 --2- 172.21.15.27:0/3592130953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7340f66e0 0x7fc7340f6b00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7fc724004660 tx=0x7fc724009b30).stop 2021-11-03T17:42:31.891 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.893+0000 7fc73c74f700 1 -- 172.21.15.27:0/3592130953 shutdown_connections 2021-11-03T17:42:31.891 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.893+0000 7fc73c74f700 1 --2- 172.21.15.27:0/3592130953 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7340f7980 0x7fc7340f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:42:31.891 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.893+0000 7fc73c74f700 1 --2- 172.21.15.27:0/3592130953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7340f66e0 0x7fc7340f6b00 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:42:31.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.893+0000 7fc73c74f700 1 -- 172.21.15.27:0/3592130953 >> 172.21.15.27:0/3592130953 conn(0x7fc7340f1e40 msgr2=0x7fc7340f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:42:31.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.893+0000 7fc73c74f700 1 -- 172.21.15.27:0/3592130953 shutdown_connections 2021-11-03T17:42:31.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.893+0000 7fc73c74f700 1 -- 172.21.15.27:0/3592130953 wait complete. 2021-11-03T17:42:31.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.894+0000 7fc73c74f700 1 Processor -- start 2021-11-03T17:42:31.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.894+0000 7fc73c74f700 1 -- start start 2021-11-03T17:42:31.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.894+0000 7fc73c74f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7340f66e0 0x7fc734106fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:42:31.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.894+0000 7fc73c74f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7340f7980 0x7fc7341074a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:42:31.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.894+0000 7fc73c74f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc734107a50 con 0x7fc7340f7980 2021-11-03T17:42:31.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.894+0000 7fc73c74f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc734107b90 con 0x7fc7340f66e0 2021-11-03T17:42:31.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.894+0000 7fc739cea700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7340f7980 0x7fc7341074a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:42:31.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.894+0000 7fc739cea700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7340f7980 0x7fc7341074a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58960/0 (socket says 172.21.15.27:58960) 2021-11-03T17:42:31.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.894+0000 7fc739cea700 1 -- 172.21.15.27:0/1710810498 learned_addr learned my addr 172.21.15.27:0/1710810498 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:42:31.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.894+0000 7fc73a4eb700 1 --2- 172.21.15.27:0/1710810498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7340f66e0 0x7fc734106fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:42:31.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.894+0000 7fc739cea700 1 -- 172.21.15.27:0/1710810498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7340f66e0 msgr2=0x7fc734106fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:42:31.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.894+0000 7fc739cea700 1 --2- 172.21.15.27:0/1710810498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7340f66e0 0x7fc734106fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:42:31.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.894+0000 7fc739cea700 1 -- 172.21.15.27:0/1710810498 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc724005040 con 0x7fc7340f7980 2021-11-03T17:42:31.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.895+0000 7fc739cea700 1 --2- 172.21.15.27:0/1710810498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7340f7980 0x7fc7341074a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7fc7300026d0 tx=0x7fc730009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:42:31.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.895+0000 7fc72affd700 1 -- 172.21.15.27:0/1710810498 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7300079c0 con 0x7fc7340f7980 2021-11-03T17:42:31.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.895+0000 7fc73c74f700 1 -- 172.21.15.27:0/1710810498 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc734108530 con 0x7fc7340f7980 2021-11-03T17:42:31.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.895+0000 7fc72affd700 1 -- 172.21.15.27:0/1710810498 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fc73000f040 con 0x7fc7340f7980 2021-11-03T17:42:31.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.895+0000 7fc72affd700 1 -- 172.21.15.27:0/1710810498 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7300148b0 con 0x7fc7340f7980 2021-11-03T17:42:31.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.895+0000 7fc73c74f700 1 -- 172.21.15.27:0/1710810498 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc734108af0 con 0x7fc7340f7980 2021-11-03T17:42:31.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.896+0000 7fc73c74f700 1 -- 172.21.15.27:0/1710810498 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc734045ad0 con 0x7fc7340f7980 2021-11-03T17:42:31.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.896+0000 7fc72affd700 1 -- 172.21.15.27:0/1710810498 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fc730007b20 con 0x7fc7340f7980 2021-11-03T17:42:31.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.896+0000 7fc72affd700 1 --2- 172.21.15.27:0/1710810498 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc7200640b0 0x7fc720066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:42:31.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.897+0000 7fc72affd700 1 -- 172.21.15.27:0/1710810498 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fc730080820 con 0x7fc7340f7980 2021-11-03T17:42:31.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.897+0000 7fc73a4eb700 1 --2- 172.21.15.27:0/1710810498 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc7200640b0 0x7fc720066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:42:31.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.897+0000 7fc73a4eb700 1 --2- 172.21.15.27:0/1710810498 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc7200640b0 0x7fc720066570 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7fc724004630 tx=0x7fc724009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:42:31.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:31.900+0000 7fc72affd700 1 -- 172.21.15.27:0/1710810498 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fc730052bd0 con 0x7fc7340f7980 2021-11-03T17:42:32.017 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:31 smithi027 ceph-mon[97244]: from='client.34968 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:42:32.099 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:32.104+0000 7fc73c74f700 1 -- 172.21.15.27:0/1710810498 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc734027a00 con 0x7fc7340f7980 2021-11-03T17:42:32.099 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:32.104+0000 7fc72affd700 1 -- 172.21.15.27:0/1710810498 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7fc7300527c0 con 0x7fc7340f7980 2021-11-03T17:42:32.100 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:42:32.100 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:42:32.100 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:42:32.101 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:42:32.101 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:42:32.101 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:42:32.101 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:42:32.102 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:42:32.102 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:42:32.102 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:42:32.102 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:42:32.103 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:42:32.103 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:42:32.103 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:42:32.103 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:42:32.104 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:42:32.104 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:42:32.104 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:42:32.104 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:42:32.104 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:42:32.105 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:42:32.105 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:42:32.105 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:42:32.105 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:42:32.106 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:42:32.106 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:42:32.106 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:42:32.106 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:42:32.106 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:42:32.107 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:42:32.107 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:42:32.107 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:42:32.107 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:42:32.108 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:42:32.108 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:42:32.108 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:42:32.108 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:42:32.109 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:42:32.109 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:42:32.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:32.106+0000 7fc73c74f700 1 -- 172.21.15.27:0/1710810498 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc7200640b0 msgr2=0x7fc720066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:42:32.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:32.106+0000 7fc73c74f700 1 --2- 172.21.15.27:0/1710810498 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc7200640b0 0x7fc720066570 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7fc724004630 tx=0x7fc724009b30).stop 2021-11-03T17:42:32.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:32.106+0000 7fc73c74f700 1 -- 172.21.15.27:0/1710810498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7340f7980 msgr2=0x7fc7341074a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:42:32.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:32.106+0000 7fc73c74f700 1 --2- 172.21.15.27:0/1710810498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7340f7980 0x7fc7341074a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7fc7300026d0 tx=0x7fc730009de0).stop 2021-11-03T17:42:32.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:32.106+0000 7fc73c74f700 1 -- 172.21.15.27:0/1710810498 shutdown_connections 2021-11-03T17:42:32.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:32.106+0000 7fc73c74f700 1 --2- 172.21.15.27:0/1710810498 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc7200640b0 0x7fc720066570 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:42:32.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:32.106+0000 7fc73c74f700 1 --2- 172.21.15.27:0/1710810498 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc7340f66e0 0x7fc734106fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:42:32.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:32.106+0000 7fc73c74f700 1 --2- 172.21.15.27:0/1710810498 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc7340f7980 0x7fc7341074a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:42:32.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:32.106+0000 7fc73c74f700 1 -- 172.21.15.27:0/1710810498 >> 172.21.15.27:0/1710810498 conn(0x7fc7340f1e40 msgr2=0x7fc7340fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:42:32.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:32.106+0000 7fc73c74f700 1 -- 172.21.15.27:0/1710810498 shutdown_connections 2021-11-03T17:42:32.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:42:32.106+0000 7fc73c74f700 1 -- 172.21.15.27:0/1710810498 wait complete. 2021-11-03T17:42:32.113 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:42:32.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:31 smithi115 ceph-mon[65453]: from='client.34968 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:42:33.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:32 smithi115 ceph-mon[65453]: from='client.34972 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:42:33.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:32 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1033397853' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:33.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:32 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1710810498' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:42:33.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:32 smithi115 ceph-mon[65453]: pgmap v837: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:33.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:32 smithi027 ceph-mon[97244]: from='client.34972 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:42:33.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:32 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1033397853' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:33.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:32 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1710810498' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:42:33.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:32 smithi027 ceph-mon[97244]: pgmap v837: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:34.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:34 smithi027 ceph-mon[97244]: pgmap v838: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:34.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:34 smithi115 ceph-mon[65453]: pgmap v838: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:35.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1597.396988 secs 2021-11-03T17:42:35.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1597.396988 secs 2021-11-03T17:42:36.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:36 smithi027 ceph-mon[97244]: pgmap v839: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:36.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:36 smithi115 ceph-mon[65453]: pgmap v839: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:38.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:38 smithi027 ceph-mon[97244]: pgmap v840: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:38.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:38 smithi115 ceph-mon[65453]: pgmap v840: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:39.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:42:39.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:42:39.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:42:39.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:39.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:39.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:39.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:39.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:39 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:42:39.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:42:39.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:42:39.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:42:39.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:39.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:39.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:39.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:39.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:39 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:42:40.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1602.397101 secs 2021-11-03T17:42:40.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:40 smithi027 ceph-mon[97244]: pgmap v841: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:40.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1602.397101 secs 2021-11-03T17:42:40.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:40 smithi115 ceph-mon[65453]: pgmap v841: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:42.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:42 smithi027 ceph-mon[97244]: pgmap v842: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:42.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:42 smithi115 ceph-mon[65453]: pgmap v842: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:44.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:44 smithi027 ceph-mon[97244]: pgmap v843: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:44.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:44 smithi115 ceph-mon[65453]: pgmap v843: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:45.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1607.397214 secs 2021-11-03T17:42:45.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1607.397214 secs 2021-11-03T17:42:46.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:46 smithi027 ceph-mon[97244]: pgmap v844: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:46.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:46 smithi115 ceph-mon[65453]: pgmap v844: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:48.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:48 smithi027 ceph-mon[97244]: pgmap v845: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:48.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:48 smithi115 ceph-mon[65453]: pgmap v845: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:49.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:42:49.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:42:49.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:42:49.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:49.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:49.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:49.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:49.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:49 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:42:49.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:42:49.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:42:49.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:42:49.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:49.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:49.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:49.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:49.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:49 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:42:50.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1612.397354 secs 2021-11-03T17:42:50.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:50 smithi027 ceph-mon[97244]: pgmap v846: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:50.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1612.397354 secs 2021-11-03T17:42:50.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:50 smithi115 ceph-mon[65453]: pgmap v846: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:52.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:52 smithi027 ceph-mon[97244]: pgmap v847: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:52.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:52 smithi115 ceph-mon[65453]: pgmap v847: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:54.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:54 smithi027 ceph-mon[97244]: pgmap v848: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:54.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:54 smithi115 ceph-mon[65453]: pgmap v848: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:55.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1617.397486 secs 2021-11-03T17:42:55.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1617.397486 secs 2021-11-03T17:42:56.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:42:56.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:42:56.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:56 smithi115 ceph-mon[65453]: pgmap v849: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:56.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:42:56.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:42:56.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:56 smithi027 ceph-mon[97244]: pgmap v849: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:58.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:58 smithi027 ceph-mon[97244]: pgmap v850: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:58.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:58 smithi115 ceph-mon[65453]: pgmap v850: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:42:59.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:42:59.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:42:59.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:42:59.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:59.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:59.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:59.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:59.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:42:59 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:42:59.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:42:59.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:42:59.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:42:59.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:59.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:59.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:59.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:42:59.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:42:59 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:43:00.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1622.397626 secs 2021-11-03T17:43:00.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:00 smithi027 ceph-mon[97244]: pgmap v851: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:00.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1622.397626 secs 2021-11-03T17:43:00.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:00 smithi115 ceph-mon[65453]: pgmap v851: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:02.218 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.221+0000 7f67ed4c4700 1 -- 172.21.15.27:0/2201856337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67e80f9850 msgr2=0x7f67e80fbc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:43:02.219 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.221+0000 7f67ed4c4700 1 --2- 172.21.15.27:0/2201856337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67e80f9850 0x7f67e80fbc40 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f67d0004660 tx=0x7f67d0009b30).stop 2021-11-03T17:43:02.219 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.222+0000 7f67ed4c4700 1 -- 172.21.15.27:0/2201856337 shutdown_connections 2021-11-03T17:43:02.219 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.222+0000 7f67ed4c4700 1 --2- 172.21.15.27:0/2201856337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67e80f9850 0x7f67e80fbc40 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:02.219 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.222+0000 7f67ed4c4700 1 --2- 172.21.15.27:0/2201856337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67e8060370 0x7f67e80f9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:02.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.222+0000 7f67ed4c4700 1 -- 172.21.15.27:0/2201856337 >> 172.21.15.27:0/2201856337 conn(0x7f67e80f0de0 msgr2=0x7f67e80f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:43:02.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.222+0000 7f67ed4c4700 1 -- 172.21.15.27:0/2201856337 shutdown_connections 2021-11-03T17:43:02.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.222+0000 7f67ed4c4700 1 -- 172.21.15.27:0/2201856337 wait complete. 2021-11-03T17:43:02.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.222+0000 7f67ed4c4700 1 Processor -- start 2021-11-03T17:43:02.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.222+0000 7f67ed4c4700 1 -- start start 2021-11-03T17:43:02.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.223+0000 7f67ed4c4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67e8060370 0x7f67e8106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:43:02.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.223+0000 7f67ed4c4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67e80f9850 0x7f67e8107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:43:02.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.223+0000 7f67ed4c4700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e8107a00 con 0x7f67e8060370 2021-11-03T17:43:02.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.223+0000 7f67ed4c4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67e8107b40 con 0x7f67e80f9850 2021-11-03T17:43:02.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.223+0000 7f67e6ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67e8060370 0x7f67e8106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:43:02.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.223+0000 7f67e6ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67e8060370 0x7f67e8106f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58976/0 (socket says 172.21.15.27:58976) 2021-11-03T17:43:02.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.223+0000 7f67e6ffd700 1 -- 172.21.15.27:0/3081316957 learned_addr learned my addr 172.21.15.27:0/3081316957 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:43:02.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.223+0000 7f67e67fc700 1 --2- 172.21.15.27:0/3081316957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67e80f9850 0x7f67e8107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:43:02.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.223+0000 7f67e6ffd700 1 -- 172.21.15.27:0/3081316957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67e80f9850 msgr2=0x7f67e8107450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:43:02.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.223+0000 7f67e6ffd700 1 --2- 172.21.15.27:0/3081316957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67e80f9850 0x7f67e8107450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:02.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.223+0000 7f67e6ffd700 1 -- 172.21.15.27:0/3081316957 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d0005040 con 0x7f67e8060370 2021-11-03T17:43:02.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.223+0000 7f67e6ffd700 1 --2- 172.21.15.27:0/3081316957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67e8060370 0x7f67e8106f60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f67e000c530 tx=0x7f67e0007e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:43:02.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.223+0000 7f67df7fe700 1 -- 172.21.15.27:0/3081316957 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67e000f650 con 0x7f67e8060370 2021-11-03T17:43:02.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.223+0000 7f67df7fe700 1 -- 172.21.15.27:0/3081316957 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f67e000fc60 con 0x7f67e8060370 2021-11-03T17:43:02.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.223+0000 7f67ed4c4700 1 -- 172.21.15.27:0/3081316957 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67e81084e0 con 0x7f67e8060370 2021-11-03T17:43:02.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.223+0000 7f67df7fe700 1 -- 172.21.15.27:0/3081316957 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f67e0012d20 con 0x7f67e8060370 2021-11-03T17:43:02.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.224+0000 7f67ed4c4700 1 -- 172.21.15.27:0/3081316957 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67e8108aa0 con 0x7f67e8060370 2021-11-03T17:43:02.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.224+0000 7f67ed4c4700 1 -- 172.21.15.27:0/3081316957 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67e8045ad0 con 0x7f67e8060370 2021-11-03T17:43:02.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.225+0000 7f67df7fe700 1 -- 172.21.15.27:0/3081316957 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f67e0019030 con 0x7f67e8060370 2021-11-03T17:43:02.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.225+0000 7f67df7fe700 1 --2- 172.21.15.27:0/3081316957 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f67d40640b0 0x7f67d4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:43:02.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.226+0000 7f67df7fe700 1 -- 172.21.15.27:0/3081316957 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f67e0080870 con 0x7f67e8060370 2021-11-03T17:43:02.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.227+0000 7f67e67fc700 1 --2- 172.21.15.27:0/3081316957 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f67d40640b0 0x7f67d4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:43:02.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.227+0000 7f67e67fc700 1 --2- 172.21.15.27:0/3081316957 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f67d40640b0 0x7f67d4066570 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f67d0004e30 tx=0x7f67d0009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:43:02.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.230+0000 7f67df7fe700 1 -- 172.21.15.27:0/3081316957 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f67e00529c0 con 0x7f67e8060370 2021-11-03T17:43:02.408 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.414+0000 7f67ed4c4700 1 -- 172.21.15.27:0/3081316957 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f67e8108d70 con 0x7f67d40640b0 2021-11-03T17:43:02.409 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.416+0000 7f67df7fe700 1 -- 172.21.15.27:0/3081316957 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f67e8108d70 con 0x7f67d40640b0 2021-11-03T17:43:02.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.418+0000 7f67ed4c4700 1 -- 172.21.15.27:0/3081316957 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f67d40640b0 msgr2=0x7f67d4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:43:02.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.418+0000 7f67ed4c4700 1 --2- 172.21.15.27:0/3081316957 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f67d40640b0 0x7f67d4066570 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 rx=0x7f67d0004e30 tx=0x7f67d0009b30).stop 2021-11-03T17:43:02.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.418+0000 7f67ed4c4700 1 -- 172.21.15.27:0/3081316957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67e8060370 msgr2=0x7f67e8106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:43:02.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.418+0000 7f67ed4c4700 1 --2- 172.21.15.27:0/3081316957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67e8060370 0x7f67e8106f60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f67e000c530 tx=0x7f67e0007e10).stop 2021-11-03T17:43:02.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.418+0000 7f67ed4c4700 1 -- 172.21.15.27:0/3081316957 shutdown_connections 2021-11-03T17:43:02.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.418+0000 7f67ed4c4700 1 --2- 172.21.15.27:0/3081316957 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f67d40640b0 0x7f67d4066570 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:02.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.418+0000 7f67ed4c4700 1 --2- 172.21.15.27:0/3081316957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67e8060370 0x7f67e8106f60 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:02.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.418+0000 7f67ed4c4700 1 --2- 172.21.15.27:0/3081316957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67e80f9850 0x7f67e8107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:02.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.418+0000 7f67ed4c4700 1 -- 172.21.15.27:0/3081316957 >> 172.21.15.27:0/3081316957 conn(0x7f67e80f0de0 msgr2=0x7f67e80f31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:43:02.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.418+0000 7f67ed4c4700 1 -- 172.21.15.27:0/3081316957 shutdown_connections 2021-11-03T17:43:02.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.418+0000 7f67ed4c4700 1 -- 172.21.15.27:0/3081316957 wait complete. 2021-11-03T17:43:02.421 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:43:02.514 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.518+0000 7f541f7ee700 1 -- 172.21.15.27:0/4191815401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f541806a3b0 msgr2=0x7f541806a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:43:02.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.518+0000 7f541f7ee700 1 --2- 172.21.15.27:0/4191815401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f541806a3b0 0x7f541806a830 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7f5408004660 tx=0x7f5408009b30).stop 2021-11-03T17:43:02.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.518+0000 7f541f7ee700 1 -- 172.21.15.27:0/4191815401 shutdown_connections 2021-11-03T17:43:02.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.518+0000 7f541f7ee700 1 --2- 172.21.15.27:0/4191815401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f541806a3b0 0x7f541806a830 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:02.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.518+0000 7f541f7ee700 1 --2- 172.21.15.27:0/4191815401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f541806b920 0x7f5418069ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:02.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.519+0000 7f541f7ee700 1 -- 172.21.15.27:0/4191815401 >> 172.21.15.27:0/4191815401 conn(0x7f54180f1f00 msgr2=0x7f54180f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:43:02.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.519+0000 7f541f7ee700 1 -- 172.21.15.27:0/4191815401 shutdown_connections 2021-11-03T17:43:02.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.519+0000 7f541f7ee700 1 -- 172.21.15.27:0/4191815401 wait complete. 2021-11-03T17:43:02.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.519+0000 7f541f7ee700 1 Processor -- start 2021-11-03T17:43:02.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.519+0000 7f541f7ee700 1 -- start start 2021-11-03T17:43:02.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.519+0000 7f541f7ee700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f541806a3b0 0x7f541810d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:43:02.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.519+0000 7f541f7ee700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f541806b920 0x7f541810da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:43:02.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.519+0000 7f541f7ee700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541810e030 con 0x7f541806b920 2021-11-03T17:43:02.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.519+0000 7f541f7ee700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f541810e170 con 0x7f541806a3b0 2021-11-03T17:43:02.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.519+0000 7f541cd89700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f541806b920 0x7f541810da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:43:02.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.519+0000 7f541cd89700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f541806b920 0x7f541810da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58988/0 (socket says 172.21.15.27:58988) 2021-11-03T17:43:02.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.519+0000 7f541cd89700 1 -- 172.21.15.27:0/1261525401 learned_addr learned my addr 172.21.15.27:0/1261525401 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:43:02.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.520+0000 7f541cd89700 1 -- 172.21.15.27:0/1261525401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f541806a3b0 msgr2=0x7f541810d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:43:02.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.520+0000 7f541cd89700 1 --2- 172.21.15.27:0/1261525401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f541806a3b0 0x7f541810d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:43:02.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.520+0000 7f541cd89700 1 -- 172.21.15.27:0/1261525401 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5408005040 con 0x7f541806b920 2021-11-03T17:43:02.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.520+0000 7f541cd89700 1 --2- 172.21.15.27:0/1261525401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f541806b920 0x7f541810da80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f5408009990 tx=0x7f5408009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:43:02.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.520+0000 7f540dffb700 1 -- 172.21.15.27:0/1261525401 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5408018070 con 0x7f541806b920 2021-11-03T17:43:02.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.520+0000 7f540dffb700 1 -- 172.21.15.27:0/1261525401 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f5408007ca0 con 0x7f541806b920 2021-11-03T17:43:02.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.520+0000 7f540dffb700 1 -- 172.21.15.27:0/1261525401 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f540801c600 con 0x7f541806b920 2021-11-03T17:43:02.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.520+0000 7f541f7ee700 1 -- 172.21.15.27:0/1261525401 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f541810eab0 con 0x7f541806b920 2021-11-03T17:43:02.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.520+0000 7f541f7ee700 1 -- 172.21.15.27:0/1261525401 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f541810f010 con 0x7f541806b920 2021-11-03T17:43:02.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.521+0000 7f541f7ee700 1 -- 172.21.15.27:0/1261525401 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5418045ad0 con 0x7f541806b920 2021-11-03T17:43:02.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.522+0000 7f540dffb700 1 -- 172.21.15.27:0/1261525401 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f540801d9d0 con 0x7f541806b920 2021-11-03T17:43:02.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.522+0000 7f540dffb700 1 --2- 172.21.15.27:0/1261525401 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f54040640b0 0x7f5404066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:43:02.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.522+0000 7f540dffb700 1 -- 172.21.15.27:0/1261525401 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f54080825f0 con 0x7f541806b920 2021-11-03T17:43:02.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.522+0000 7f541d58a700 1 --2- 172.21.15.27:0/1261525401 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f54040640b0 0x7f5404066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:43:02.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.523+0000 7f541d58a700 1 --2- 172.21.15.27:0/1261525401 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f54040640b0 0x7f5404066570 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f541806b6f0 tx=0x7f5414007a60).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:43:02.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:02 smithi027 ceph-mon[97244]: pgmap v852: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:02.524 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.526+0000 7f540dffb700 1 -- 172.21.15.27:0/1261525401 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f54080549b0 con 0x7f541806b920 2021-11-03T17:43:02.693 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.699+0000 7f541f7ee700 1 -- 172.21.15.27:0/1261525401 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f541810f280 con 0x7f54040640b0 2021-11-03T17:43:02.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.714+0000 7f540dffb700 1 -- 172.21.15.27:0/1261525401 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f541810f280 con 0x7f54040640b0 2021-11-03T17:43:02.708 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:43:02.708 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (34m) 4m ago 35m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:43:02.709 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (26m) 4m ago 35m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:43:02.709 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (26m) 3m ago 35m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:43:02.709 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (34m) 4m ago 35m 41.3M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:43:02.709 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (28m) 4m ago 30m 21.1M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:43:02.709 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (31m) 4m ago 31m 3843M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:43:02.710 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (26m) 3m ago 31m 20.4M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:43:02.710 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (28m) 3m ago 31m 22.1M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:43:02.710 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (28m) 4m ago 37m 517M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:43:02.710 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (27m) 3m ago 35m 419M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:43:02.711 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (27m) 4m ago 37m 159M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:43:02.711 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (26m) 3m ago 34m 82.5M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:43:02.711 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (35m) 4m ago 35m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:43:02.711 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (34m) 3m ago 34m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:43:02.712 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (25m) 4m ago 34m 311M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:43:02.712 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (25m) 4m ago 33m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:43:02.712 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (24m) 4m ago 33m 248M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:43:02.712 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (24m) 3m ago 33m 354M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:43:02.713 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (23m) 3m ago 32m 285M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:43:02.713 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (23m) 3m ago 32m 287M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:43:02.713 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (34m) 4m ago 35m 50.5M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:43:02.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.716+0000 7f541f7ee700 1 -- 172.21.15.27:0/1261525401 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f54040640b0 msgr2=0x7f5404066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:43:02.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.716+0000 7f541f7ee700 1 --2- 172.21.15.27:0/1261525401 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f54040640b0 0x7f5404066570 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7f541806b6f0 tx=0x7f5414007a60).stop 2021-11-03T17:43:02.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.716+0000 7f541f7ee700 1 -- 172.21.15.27:0/1261525401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f541806b920 msgr2=0x7f541810da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:43:02.715 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.716+0000 7f541f7ee700 1 --2- 172.21.15.27:0/1261525401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f541806b920 0x7f541810da80 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7f5408009990 tx=0x7f5408009b30).stop 2021-11-03T17:43:02.715 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.716+0000 7f541f7ee700 1 -- 172.21.15.27:0/1261525401 shutdown_connections 2021-11-03T17:43:02.715 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.716+0000 7f541f7ee700 1 --2- 172.21.15.27:0/1261525401 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f54040640b0 0x7f5404066570 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:02.715 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.716+0000 7f541f7ee700 1 --2- 172.21.15.27:0/1261525401 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f541806a3b0 0x7f541810d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:43:02.716 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.716+0000 7f541f7ee700 1 --2- 172.21.15.27:0/1261525401 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f541806b920 0x7f541810da80 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:02.716 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.716+0000 7f541f7ee700 1 -- 172.21.15.27:0/1261525401 >> 172.21.15.27:0/1261525401 conn(0x7f54180f1f00 msgr2=0x7f54180f8960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:43:02.716 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.716+0000 7f541f7ee700 1 -- 172.21.15.27:0/1261525401 shutdown_connections 2021-11-03T17:43:02.716 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.716+0000 7f541f7ee700 1 -- 172.21.15.27:0/1261525401 wait complete. 2021-11-03T17:43:02.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:02 smithi115 ceph-mon[65453]: pgmap v852: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:02.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.815+0000 7f27319b2700 1 -- 172.21.15.27:0/3462835179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f272c0f66e0 msgr2=0x7f272c0f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:43:02.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.815+0000 7f27319b2700 1 --2- 172.21.15.27:0/3462835179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f272c0f66e0 0x7f272c0f6b40 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7f271c004660 tx=0x7f271c009b30).stop 2021-11-03T17:43:02.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.815+0000 7f27319b2700 1 -- 172.21.15.27:0/3462835179 shutdown_connections 2021-11-03T17:43:02.811 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.815+0000 7f27319b2700 1 --2- 172.21.15.27:0/3462835179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f272c0f66e0 0x7f272c0f6b40 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:02.811 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.815+0000 7f27319b2700 1 --2- 172.21.15.27:0/3462835179 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f272c0f5440 0x7f272c0f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:02.811 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.815+0000 7f27319b2700 1 -- 172.21.15.27:0/3462835179 >> 172.21.15.27:0/3462835179 conn(0x7f272c0f0ba0 msgr2=0x7f272c0f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:43:02.812 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.815+0000 7f27319b2700 1 -- 172.21.15.27:0/3462835179 shutdown_connections 2021-11-03T17:43:02.812 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.815+0000 7f27319b2700 1 -- 172.21.15.27:0/3462835179 wait complete. 2021-11-03T17:43:02.812 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.816+0000 7f27319b2700 1 Processor -- start 2021-11-03T17:43:02.812 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.816+0000 7f27319b2700 1 -- start start 2021-11-03T17:43:02.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.816+0000 7f27319b2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f272c0f5440 0x7f272c109ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:43:02.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.816+0000 7f27319b2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f272c0f66e0 0x7f272c10bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:43:02.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.816+0000 7f27319b2700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272c0603c0 con 0x7f272c0f66e0 2021-11-03T17:43:02.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.816+0000 7f27319b2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f272c060500 con 0x7f272c0f5440 2021-11-03T17:43:02.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.816+0000 7f272bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f272c0f66e0 0x7f272c10bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:43:02.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.816+0000 7f272bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f272c0f66e0 0x7f272c10bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:58998/0 (socket says 172.21.15.27:58998) 2021-11-03T17:43:02.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.816+0000 7f272bfff700 1 -- 172.21.15.27:0/2577267805 learned_addr learned my addr 172.21.15.27:0/2577267805 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:43:02.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.817+0000 7f27309b0700 1 --2- 172.21.15.27:0/2577267805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f272c0f5440 0x7f272c109ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:43:02.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.817+0000 7f272bfff700 1 -- 172.21.15.27:0/2577267805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f272c0f5440 msgr2=0x7f272c109ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:43:02.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.817+0000 7f272bfff700 1 --2- 172.21.15.27:0/2577267805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f272c0f5440 0x7f272c109ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:02.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.817+0000 7f272bfff700 1 -- 172.21.15.27:0/2577267805 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f271c005040 con 0x7f272c0f66e0 2021-11-03T17:43:02.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.817+0000 7f27309b0700 1 --2- 172.21.15.27:0/2577267805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f272c0f5440 0x7f272c109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:43:02.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.817+0000 7f272bfff700 1 --2- 172.21.15.27:0/2577267805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f272c0f66e0 0x7f272c10bfb0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f271c009990 tx=0x7f271c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:43:02.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.817+0000 7f27297fa700 1 -- 172.21.15.27:0/2577267805 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f271c018070 con 0x7f272c0f66e0 2021-11-03T17:43:02.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.817+0000 7f27319b2700 1 -- 172.21.15.27:0/2577267805 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f272c10c500 con 0x7f272c0f66e0 2021-11-03T17:43:02.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.817+0000 7f27297fa700 1 -- 172.21.15.27:0/2577267805 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f271c007ca0 con 0x7f272c0f66e0 2021-11-03T17:43:02.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.817+0000 7f27297fa700 1 -- 172.21.15.27:0/2577267805 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f271c01c620 con 0x7f272c0f66e0 2021-11-03T17:43:02.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.817+0000 7f27319b2700 1 -- 172.21.15.27:0/2577267805 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f272c10ca30 con 0x7f272c0f66e0 2021-11-03T17:43:02.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.818+0000 7f27319b2700 1 -- 172.21.15.27:0/2577267805 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f272c059100 con 0x7f272c0f66e0 2021-11-03T17:43:02.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.819+0000 7f27297fa700 1 -- 172.21.15.27:0/2577267805 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f271c01d960 con 0x7f272c0f66e0 2021-11-03T17:43:02.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.819+0000 7f27297fa700 1 --2- 172.21.15.27:0/2577267805 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2718064100 0x7f27180665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:43:02.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.819+0000 7f27297fa700 1 -- 172.21.15.27:0/2577267805 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f271c0825a0 con 0x7f272c0f66e0 2021-11-03T17:43:02.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.819+0000 7f27309b0700 1 --2- 172.21.15.27:0/2577267805 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2718064100 0x7f27180665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:43:02.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.819+0000 7f27309b0700 1 --2- 172.21.15.27:0/2577267805 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2718064100 0x7f27180665c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f272c060040 tx=0x7f2724009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:43:02.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:02.823+0000 7f27297fa700 1 -- 172.21.15.27:0/2577267805 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f271c054960 con 0x7f272c0f66e0 2021-11-03T17:43:03.047 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.053+0000 7f27319b2700 1 -- 172.21.15.27:0/2577267805 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f272c045ad0 con 0x7f272c0f66e0 2021-11-03T17:43:03.048 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.054+0000 7f27297fa700 1 -- 172.21.15.27:0/2577267805 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f271c054550 con 0x7f272c0f66e0 2021-11-03T17:43:03.049 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:43:03.049 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:43:03.049 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:43:03.049 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:43:03.050 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:43:03.050 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:43:03.050 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:43:03.050 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:43:03.051 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:43:03.051 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:43:03.051 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:43:03.051 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:43:03.052 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:43:03.052 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:43:03.052 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:43:03.052 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:43:03.052 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:43:03.053 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:43:03.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.056+0000 7f27319b2700 1 -- 172.21.15.27:0/2577267805 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2718064100 msgr2=0x7f27180665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:43:03.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.056+0000 7f27319b2700 1 --2- 172.21.15.27:0/2577267805 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2718064100 0x7f27180665c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 rx=0x7f272c060040 tx=0x7f2724009040).stop 2021-11-03T17:43:03.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.056+0000 7f27319b2700 1 -- 172.21.15.27:0/2577267805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f272c0f66e0 msgr2=0x7f272c10bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:43:03.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.056+0000 7f27319b2700 1 --2- 172.21.15.27:0/2577267805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f272c0f66e0 0x7f272c10bfb0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f271c009990 tx=0x7f271c009b30).stop 2021-11-03T17:43:03.055 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.056+0000 7f27319b2700 1 -- 172.21.15.27:0/2577267805 shutdown_connections 2021-11-03T17:43:03.055 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.056+0000 7f27319b2700 1 --2- 172.21.15.27:0/2577267805 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2718064100 0x7f27180665c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:03.055 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.056+0000 7f27319b2700 1 --2- 172.21.15.27:0/2577267805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f272c0f5440 0x7f272c109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:03.055 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.056+0000 7f27319b2700 1 --2- 172.21.15.27:0/2577267805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f272c0f66e0 0x7f272c10bfb0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:03.056 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.056+0000 7f27319b2700 1 -- 172.21.15.27:0/2577267805 >> 172.21.15.27:0/2577267805 conn(0x7f272c0f0ba0 msgr2=0x7f272c0fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:43:03.056 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.056+0000 7f27319b2700 1 -- 172.21.15.27:0/2577267805 shutdown_connections 2021-11-03T17:43:03.056 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.056+0000 7f27319b2700 1 -- 172.21.15.27:0/2577267805 wait complete. 2021-11-03T17:43:03.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.158+0000 7f2f438e1700 1 -- 172.21.15.27:0/1786813119 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f3c0f6860 msgr2=0x7f2f3c0f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:43:03.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.158+0000 7f2f438e1700 1 --2- 172.21.15.27:0/1786813119 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f3c0f6860 0x7f2f3c0f6cc0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f2f38004660 tx=0x7f2f38009b30).stop 2021-11-03T17:43:03.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.158+0000 7f2f438e1700 1 -- 172.21.15.27:0/1786813119 shutdown_connections 2021-11-03T17:43:03.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.158+0000 7f2f438e1700 1 --2- 172.21.15.27:0/1786813119 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f3c0f6860 0x7f2f3c0f6cc0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:03.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.158+0000 7f2f438e1700 1 --2- 172.21.15.27:0/1786813119 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f3c0f55c0 0x7f2f3c0f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:03.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.158+0000 7f2f438e1700 1 -- 172.21.15.27:0/1786813119 >> 172.21.15.27:0/1786813119 conn(0x7f2f3c0f0d40 msgr2=0x7f2f3c0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:43:03.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.158+0000 7f2f438e1700 1 -- 172.21.15.27:0/1786813119 shutdown_connections 2021-11-03T17:43:03.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.158+0000 7f2f438e1700 1 -- 172.21.15.27:0/1786813119 wait complete. 2021-11-03T17:43:03.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.158+0000 7f2f438e1700 1 Processor -- start 2021-11-03T17:43:03.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.158+0000 7f2f438e1700 1 -- start start 2021-11-03T17:43:03.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.159+0000 7f2f438e1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f3c0f55c0 0x7f2f3c105ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:43:03.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.159+0000 7f2f438e1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f3c0f6860 0x7f2f3c1063b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:43:03.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.159+0000 7f2f438e1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c1069b0 con 0x7f2f3c0f6860 2021-11-03T17:43:03.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.159+0000 7f2f438e1700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c1071c0 con 0x7f2f3c0f55c0 2021-11-03T17:43:03.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.159+0000 7f2f40e7c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f3c0f6860 0x7f2f3c1063b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:43:03.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.159+0000 7f2f40e7c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f3c0f6860 0x7f2f3c1063b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59010/0 (socket says 172.21.15.27:59010) 2021-11-03T17:43:03.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.159+0000 7f2f40e7c700 1 -- 172.21.15.27:0/1420427416 learned_addr learned my addr 172.21.15.27:0/1420427416 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:43:03.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.159+0000 7f2f4167d700 1 --2- 172.21.15.27:0/1420427416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f3c0f55c0 0x7f2f3c105ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:43:03.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.159+0000 7f2f40e7c700 1 -- 172.21.15.27:0/1420427416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f3c0f55c0 msgr2=0x7f2f3c105ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:43:03.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.159+0000 7f2f40e7c700 1 --2- 172.21.15.27:0/1420427416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f3c0f55c0 0x7f2f3c105ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:03.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.159+0000 7f2f40e7c700 1 -- 172.21.15.27:0/1420427416 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f38005040 con 0x7f2f3c0f6860 2021-11-03T17:43:03.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.159+0000 7f2f40e7c700 1 --2- 172.21.15.27:0/1420427416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f3c0f6860 0x7f2f3c1063b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f2f38009da0 tx=0x7f2f38009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:43:03.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.159+0000 7f2f31ffb700 1 -- 172.21.15.27:0/1420427416 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f38018070 con 0x7f2f3c0f6860 2021-11-03T17:43:03.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.159+0000 7f2f31ffb700 1 -- 172.21.15.27:0/1420427416 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f2f38007cf0 con 0x7f2f3c0f6860 2021-11-03T17:43:03.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.159+0000 7f2f438e1700 1 -- 172.21.15.27:0/1420427416 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f3c1073e0 con 0x7f2f3c0f6860 2021-11-03T17:43:03.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.159+0000 7f2f31ffb700 1 -- 172.21.15.27:0/1420427416 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2f3801c620 con 0x7f2f3c0f6860 2021-11-03T17:43:03.161 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.160+0000 7f2f438e1700 1 -- 172.21.15.27:0/1420427416 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f3c107940 con 0x7f2f3c0f6860 2021-11-03T17:43:03.161 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.160+0000 7f2f438e1700 1 -- 172.21.15.27:0/1420427416 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f3c0f49b0 con 0x7f2f3c0f6860 2021-11-03T17:43:03.161 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.161+0000 7f2f31ffb700 1 -- 172.21.15.27:0/1420427416 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f2f3801d420 con 0x7f2f3c0f6860 2021-11-03T17:43:03.161 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.161+0000 7f2f31ffb700 1 --2- 172.21.15.27:0/1420427416 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2f28064090 0x7f2f28066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:43:03.162 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.161+0000 7f2f31ffb700 1 -- 172.21.15.27:0/1420427416 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f2f38082770 con 0x7f2f3c0f6860 2021-11-03T17:43:03.162 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.161+0000 7f2f4167d700 1 --2- 172.21.15.27:0/1420427416 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2f28064090 0x7f2f28066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:43:03.162 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.162+0000 7f2f4167d700 1 --2- 172.21.15.27:0/1420427416 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2f28064090 0x7f2f28066550 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f2f3c105c30 tx=0x7f2f2c009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:43:03.162 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.165+0000 7f2f31ffb700 1 -- 172.21.15.27:0/1420427416 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f2f38054b30 con 0x7f2f3c0f6860 2021-11-03T17:43:03.365 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.371+0000 7f2f438e1700 1 -- 172.21.15.27:0/1420427416 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2f3c027a00 con 0x7f2f3c0f6860 2021-11-03T17:43:03.366 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.372+0000 7f2f31ffb700 1 -- 172.21.15.27:0/1420427416 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f2f38054720 con 0x7f2f3c0f6860 2021-11-03T17:43:03.366 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:43:03.367 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:43:03.367 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:43:03.367 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:43:03.367 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:43:03.368 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:43:03.368 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:43:03.368 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:43:03.369 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:43:03.369 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:43:03.369 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:43:03.369 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:43:03.370 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:43:03.370 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:43:03.370 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:43:03.370 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:43:03.371 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:43:03.371 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:43:03.371 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:43:03.371 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:43:03.371 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:43:03.372 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:43:03.372 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:43:03.372 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:43:03.372 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:43:03.372 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:43:03.373 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:43:03.373 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:43:03.373 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:43:03.373 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:43:03.374 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:43:03.374 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:43:03.374 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:43:03.374 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:43:03.374 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:43:03.375 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:43:03.375 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:43:03.375 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:43:03.375 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:43:03.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.374+0000 7f2f438e1700 1 -- 172.21.15.27:0/1420427416 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2f28064090 msgr2=0x7f2f28066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:43:03.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.374+0000 7f2f438e1700 1 --2- 172.21.15.27:0/1420427416 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2f28064090 0x7f2f28066550 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f2f3c105c30 tx=0x7f2f2c009040).stop 2021-11-03T17:43:03.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.374+0000 7f2f438e1700 1 -- 172.21.15.27:0/1420427416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f3c0f6860 msgr2=0x7f2f3c1063b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:43:03.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.374+0000 7f2f438e1700 1 --2- 172.21.15.27:0/1420427416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f3c0f6860 0x7f2f3c1063b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7f2f38009da0 tx=0x7f2f38009b30).stop 2021-11-03T17:43:03.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.374+0000 7f2f438e1700 1 -- 172.21.15.27:0/1420427416 shutdown_connections 2021-11-03T17:43:03.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.374+0000 7f2f438e1700 1 --2- 172.21.15.27:0/1420427416 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2f28064090 0x7f2f28066550 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:03.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.374+0000 7f2f438e1700 1 --2- 172.21.15.27:0/1420427416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2f3c0f55c0 0x7f2f3c105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:03.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.374+0000 7f2f438e1700 1 --2- 172.21.15.27:0/1420427416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2f3c0f6860 0x7f2f3c1063b0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:03.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.374+0000 7f2f438e1700 1 -- 172.21.15.27:0/1420427416 >> 172.21.15.27:0/1420427416 conn(0x7f2f3c0f0d40 msgr2=0x7f2f3c0fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:43:03.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.374+0000 7f2f438e1700 1 -- 172.21.15.27:0/1420427416 shutdown_connections 2021-11-03T17:43:03.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:03.374+0000 7f2f438e1700 1 -- 172.21.15.27:0/1420427416 wait complete. 2021-11-03T17:43:03.379 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:43:03.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:03 smithi027 ceph-mon[97244]: from='client.34984 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:43:03.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:03 smithi027 ceph-mon[97244]: from='client.34988 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:43:03.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:03 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2577267805' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:03.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:03 smithi115 ceph-mon[65453]: from='client.34984 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:43:03.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:03 smithi115 ceph-mon[65453]: from='client.34988 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:43:03.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:03 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2577267805' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:04.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:04 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1420427416' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:43:04.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:04 smithi115 ceph-mon[65453]: pgmap v853: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:04.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:04 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1420427416' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:43:04.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:04 smithi027 ceph-mon[97244]: pgmap v853: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:05.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1627.397704 secs 2021-11-03T17:43:05.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1627.397704 secs 2021-11-03T17:43:06.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:06 smithi115 ceph-mon[65453]: pgmap v854: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:06.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:06 smithi027 ceph-mon[97244]: pgmap v854: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:08.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:08 smithi027 ceph-mon[97244]: pgmap v855: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:08.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:08 smithi115 ceph-mon[65453]: pgmap v855: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:09.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:43:09.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:43:09.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:43:09.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:09.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:09.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:09.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:09.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:09 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:43:09.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:43:09.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:43:09.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:43:09.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:09.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:09.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:09.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:09.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:09 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:43:10.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1632.397844 secs 2021-11-03T17:43:10.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:10 smithi027 ceph-mon[97244]: pgmap v856: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:10.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1632.397844 secs 2021-11-03T17:43:10.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:10 smithi115 ceph-mon[65453]: pgmap v856: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:12.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:12 smithi027 ceph-mon[97244]: pgmap v857: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:12.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:12 smithi115 ceph-mon[65453]: pgmap v857: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:14.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:14 smithi027 ceph-mon[97244]: pgmap v858: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:14.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:14 smithi115 ceph-mon[65453]: pgmap v858: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:15.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1637.397970 secs 2021-11-03T17:43:15.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1637.397970 secs 2021-11-03T17:43:16.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:16 smithi115 ceph-mon[65453]: pgmap v859: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:16.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:16 smithi027 ceph-mon[97244]: pgmap v859: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:18.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:18 smithi027 ceph-mon[97244]: pgmap v860: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:18.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:18 smithi115 ceph-mon[65453]: pgmap v860: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:19.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:43:19.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:19 smithi115 ceph-mon[65453]: poll took 60163.688 ms: 1 events 2021-11-03T17:43:19.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:43:19.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:43:19.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:19.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:19.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:19.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:19.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:19 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:43:19.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:43:19.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:19 smithi027 ceph-mon[97244]: poll took 60163.688 ms: 1 events 2021-11-03T17:43:19.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:43:19.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:43:19.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:19.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:19.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:19.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:19.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:19 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:43:20.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1642.398094 secs 2021-11-03T17:43:20.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:20 smithi115 ceph-mon[65453]: pgmap v861: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:20.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1642.398094 secs 2021-11-03T17:43:20.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:20 smithi027 ceph-mon[97244]: pgmap v861: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:22.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:22 smithi115 ceph-mon[65453]: pgmap v862: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:22.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:22 smithi027 ceph-mon[97244]: pgmap v862: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:24.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:24 smithi115 ceph-mon[65453]: pgmap v863: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:24.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:24 smithi027 ceph-mon[97244]: pgmap v863: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:25.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1647.398220 secs 2021-11-03T17:43:25.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1647.398220 secs 2021-11-03T17:43:26.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:26 smithi115 ceph-mon[65453]: pgmap v864: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:26.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:26 smithi027 ceph-mon[97244]: pgmap v864: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:28.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:28 smithi115 ceph-mon[65453]: pgmap v865: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:28.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:28 smithi027 ceph-mon[97244]: pgmap v865: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:29.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:43:29.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:43:29.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:43:29.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:29.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:29.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:29.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:29.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:43:29.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:43:29.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:43:29.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:29.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:29.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:29.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:30.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:30 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:43:30.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1652.398338 secs 2021-11-03T17:43:30.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:30 smithi115 ceph-mon[65453]: pgmap v866: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:30.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:30 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:43:30.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1652.398338 secs 2021-11-03T17:43:30.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:30 smithi027 ceph-mon[97244]: pgmap v866: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:32.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:32 smithi115 ceph-mon[65453]: pgmap v867: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:32.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:32 smithi027 ceph-mon[97244]: pgmap v867: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:33.485 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.489+0000 7f21eb331700 1 -- 172.21.15.27:0/566372193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21e40f7980 msgr2=0x7f21e40f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:43:33.485 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.489+0000 7f21eb331700 1 --2- 172.21.15.27:0/566372193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21e40f7980 0x7f21e40f7de0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f21d4004660 tx=0x7f21d4009b30).stop 2021-11-03T17:43:33.486 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.489+0000 7f21eb331700 1 -- 172.21.15.27:0/566372193 shutdown_connections 2021-11-03T17:43:33.486 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.489+0000 7f21eb331700 1 --2- 172.21.15.27:0/566372193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21e40f7980 0x7f21e40f7de0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:33.486 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.489+0000 7f21eb331700 1 --2- 172.21.15.27:0/566372193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21e40f66e0 0x7f21e40f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:33.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.489+0000 7f21eb331700 1 -- 172.21.15.27:0/566372193 >> 172.21.15.27:0/566372193 conn(0x7f21e40f1e40 msgr2=0x7f21e40f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:43:33.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.489+0000 7f21eb331700 1 -- 172.21.15.27:0/566372193 shutdown_connections 2021-11-03T17:43:33.488 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.489+0000 7f21eb331700 1 -- 172.21.15.27:0/566372193 wait complete. 2021-11-03T17:43:33.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.490+0000 7f21eb331700 1 Processor -- start 2021-11-03T17:43:33.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.490+0000 7f21eb331700 1 -- start start 2021-11-03T17:43:33.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.490+0000 7f21eb331700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21e40f66e0 0x7f21e4106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:43:33.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.490+0000 7f21eb331700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21e40f7980 0x7f21e4107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:43:33.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.490+0000 7f21eb331700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e4107a00 con 0x7f21e40f7980 2021-11-03T17:43:33.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.490+0000 7f21eb331700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e4107b40 con 0x7f21e40f66e0 2021-11-03T17:43:33.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.490+0000 7f21e88cc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21e40f7980 0x7f21e4107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:43:33.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.490+0000 7f21e88cc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21e40f7980 0x7f21e4107450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59028/0 (socket says 172.21.15.27:59028) 2021-11-03T17:43:33.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.490+0000 7f21e88cc700 1 -- 172.21.15.27:0/1756683124 learned_addr learned my addr 172.21.15.27:0/1756683124 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:43:33.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.490+0000 7f21e90cd700 1 --2- 172.21.15.27:0/1756683124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21e40f66e0 0x7f21e4106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:43:33.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.490+0000 7f21e88cc700 1 -- 172.21.15.27:0/1756683124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21e40f66e0 msgr2=0x7f21e4106f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:43:33.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.490+0000 7f21e88cc700 1 --2- 172.21.15.27:0/1756683124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21e40f66e0 0x7f21e4106f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:33.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.490+0000 7f21e88cc700 1 -- 172.21.15.27:0/1756683124 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d4005040 con 0x7f21e40f7980 2021-11-03T17:43:33.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.491+0000 7f21e88cc700 1 --2- 172.21.15.27:0/1756683124 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21e40f7980 0x7f21e4107450 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7f21d4009990 tx=0x7f21d4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:43:33.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.491+0000 7f21d9ffb700 1 -- 172.21.15.27:0/1756683124 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21d4018070 con 0x7f21e40f7980 2021-11-03T17:43:33.494 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.491+0000 7f21d9ffb700 1 -- 172.21.15.27:0/1756683124 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f21d4007ba0 con 0x7f21e40f7980 2021-11-03T17:43:33.494 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.491+0000 7f21eb331700 1 -- 172.21.15.27:0/1756683124 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e4108480 con 0x7f21e40f7980 2021-11-03T17:43:33.494 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.491+0000 7f21d9ffb700 1 -- 172.21.15.27:0/1756683124 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21d401c3c0 con 0x7f21e40f7980 2021-11-03T17:43:33.494 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.491+0000 7f21eb331700 1 -- 172.21.15.27:0/1756683124 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e41089e0 con 0x7f21e40f7980 2021-11-03T17:43:33.496 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.492+0000 7f21eb331700 1 -- 172.21.15.27:0/1756683124 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e4045ad0 con 0x7f21e40f7980 2021-11-03T17:43:33.496 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.493+0000 7f21d9ffb700 1 -- 172.21.15.27:0/1756683124 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f21d401d420 con 0x7f21e40f7980 2021-11-03T17:43:33.496 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.493+0000 7f21d9ffb700 1 --2- 172.21.15.27:0/1756683124 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f21d00640b0 0x7f21d0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:43:33.496 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.493+0000 7f21d9ffb700 1 -- 172.21.15.27:0/1756683124 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f21d40823c0 con 0x7f21e40f7980 2021-11-03T17:43:33.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.493+0000 7f21e90cd700 1 --2- 172.21.15.27:0/1756683124 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f21d00640b0 0x7f21d0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:43:33.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.494+0000 7f21e90cd700 1 --2- 172.21.15.27:0/1756683124 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f21d00640b0 0x7f21d0066570 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f21e40f77e0 tx=0x7f21e000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:43:33.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.497+0000 7f21d9ffb700 1 -- 172.21.15.27:0/1756683124 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f21d4054720 con 0x7f21e40f7980 2021-11-03T17:43:33.670 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.676+0000 7f21eb331700 1 -- 172.21.15.27:0/1756683124 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f21e40fefb0 con 0x7f21d00640b0 2021-11-03T17:43:33.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.677+0000 7f21d9ffb700 1 -- 172.21.15.27:0/1756683124 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f21e40fefb0 con 0x7f21d00640b0 2021-11-03T17:43:33.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.679+0000 7f21eb331700 1 -- 172.21.15.27:0/1756683124 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f21d00640b0 msgr2=0x7f21d0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:43:33.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.679+0000 7f21eb331700 1 --2- 172.21.15.27:0/1756683124 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f21d00640b0 0x7f21d0066570 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f21e40f77e0 tx=0x7f21e000b040).stop 2021-11-03T17:43:33.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.679+0000 7f21eb331700 1 -- 172.21.15.27:0/1756683124 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21e40f7980 msgr2=0x7f21e4107450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:43:33.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.679+0000 7f21eb331700 1 --2- 172.21.15.27:0/1756683124 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21e40f7980 0x7f21e4107450 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7f21d4009990 tx=0x7f21d4009b30).stop 2021-11-03T17:43:33.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.679+0000 7f21eb331700 1 -- 172.21.15.27:0/1756683124 shutdown_connections 2021-11-03T17:43:33.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.679+0000 7f21eb331700 1 --2- 172.21.15.27:0/1756683124 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f21d00640b0 0x7f21d0066570 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:33.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.679+0000 7f21eb331700 1 --2- 172.21.15.27:0/1756683124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21e40f66e0 0x7f21e4106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:33.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.679+0000 7f21eb331700 1 --2- 172.21.15.27:0/1756683124 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21e40f7980 0x7f21e4107450 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:33.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.679+0000 7f21eb331700 1 -- 172.21.15.27:0/1756683124 >> 172.21.15.27:0/1756683124 conn(0x7f21e40f1e40 msgr2=0x7f21e40fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:43:33.676 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.679+0000 7f21eb331700 1 -- 172.21.15.27:0/1756683124 shutdown_connections 2021-11-03T17:43:33.676 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.679+0000 7f21eb331700 1 -- 172.21.15.27:0/1756683124 wait complete. 2021-11-03T17:43:33.682 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:43:33.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.779+0000 7fea96ac9700 1 -- 172.21.15.27:0/595319980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea900f6860 msgr2=0x7fea900f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:43:33.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.779+0000 7fea96ac9700 1 --2- 172.21.15.27:0/595319980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea900f6860 0x7fea900f6cc0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7fea7c004660 tx=0x7fea7c009b30).stop 2021-11-03T17:43:33.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.780+0000 7fea96ac9700 1 -- 172.21.15.27:0/595319980 shutdown_connections 2021-11-03T17:43:33.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.780+0000 7fea96ac9700 1 --2- 172.21.15.27:0/595319980 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea900f6860 0x7fea900f6cc0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:33.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.780+0000 7fea96ac9700 1 --2- 172.21.15.27:0/595319980 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea900f55c0 0x7fea900f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:33.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.780+0000 7fea96ac9700 1 -- 172.21.15.27:0/595319980 >> 172.21.15.27:0/595319980 conn(0x7fea900f0d20 msgr2=0x7fea900f3140 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:43:33.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.780+0000 7fea96ac9700 1 -- 172.21.15.27:0/595319980 shutdown_connections 2021-11-03T17:43:33.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.780+0000 7fea96ac9700 1 -- 172.21.15.27:0/595319980 wait complete. 2021-11-03T17:43:33.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.780+0000 7fea96ac9700 1 Processor -- start 2021-11-03T17:43:33.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.780+0000 7fea96ac9700 1 -- start start 2021-11-03T17:43:33.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.780+0000 7fea96ac9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea900f55c0 0x7fea90106f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:43:33.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.780+0000 7fea96ac9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea900f6860 0x7fea90107420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:43:33.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.780+0000 7fea96ac9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea901079d0 con 0x7fea900f55c0 2021-11-03T17:43:33.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.780+0000 7fea96ac9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea90107b10 con 0x7fea900f6860 2021-11-03T17:43:33.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.781+0000 7fea94865700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea900f55c0 0x7fea90106f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:43:33.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.781+0000 7fea94865700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea900f55c0 0x7fea90106f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59036/0 (socket says 172.21.15.27:59036) 2021-11-03T17:43:33.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.781+0000 7fea94865700 1 -- 172.21.15.27:0/1018275660 learned_addr learned my addr 172.21.15.27:0/1018275660 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:43:33.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.781+0000 7fea8ffff700 1 --2- 172.21.15.27:0/1018275660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea900f6860 0x7fea90107420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:43:33.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.781+0000 7fea94865700 1 -- 172.21.15.27:0/1018275660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea900f6860 msgr2=0x7fea90107420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:43:33.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.781+0000 7fea94865700 1 --2- 172.21.15.27:0/1018275660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea900f6860 0x7fea90107420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:33.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.781+0000 7fea94865700 1 -- 172.21.15.27:0/1018275660 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea7c005040 con 0x7fea900f55c0 2021-11-03T17:43:33.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.781+0000 7fea8ffff700 1 --2- 172.21.15.27:0/1018275660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea900f6860 0x7fea90107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:43:33.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.781+0000 7fea94865700 1 --2- 172.21.15.27:0/1018275660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea900f55c0 0x7fea90106f30 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7fea8400c530 tx=0x7fea84007e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:43:33.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.781+0000 7fea8d7fa700 1 -- 172.21.15.27:0/1018275660 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea8400f650 con 0x7fea900f55c0 2021-11-03T17:43:33.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.781+0000 7fea8d7fa700 1 -- 172.21.15.27:0/1018275660 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fea8400fc60 con 0x7fea900f55c0 2021-11-03T17:43:33.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.781+0000 7fea96ac9700 1 -- 172.21.15.27:0/1018275660 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea901084b0 con 0x7fea900f55c0 2021-11-03T17:43:33.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.781+0000 7fea8d7fa700 1 -- 172.21.15.27:0/1018275660 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea84012d20 con 0x7fea900f55c0 2021-11-03T17:43:33.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.781+0000 7fea96ac9700 1 -- 172.21.15.27:0/1018275660 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea90108a70 con 0x7fea900f55c0 2021-11-03T17:43:33.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.782+0000 7fea96ac9700 1 -- 172.21.15.27:0/1018275660 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea90045ad0 con 0x7fea900f55c0 2021-11-03T17:43:33.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.783+0000 7fea8d7fa700 1 -- 172.21.15.27:0/1018275660 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fea84019030 con 0x7fea900f55c0 2021-11-03T17:43:33.787 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.783+0000 7fea8d7fa700 1 --2- 172.21.15.27:0/1018275660 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fea800640b0 0x7fea80066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:43:33.787 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.783+0000 7fea8d7fa700 1 -- 172.21.15.27:0/1018275660 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fea84080680 con 0x7fea900f55c0 2021-11-03T17:43:33.787 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.783+0000 7fea8ffff700 1 --2- 172.21.15.27:0/1018275660 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fea800640b0 0x7fea80066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:43:33.788 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.783+0000 7fea8ffff700 1 --2- 172.21.15.27:0/1018275660 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fea800640b0 0x7fea80066570 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7fea7c009da0 tx=0x7fea7c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:43:33.789 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.787+0000 7fea8d7fa700 1 -- 172.21.15.27:0/1018275660 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fea840528e0 con 0x7fea900f55c0 2021-11-03T17:43:33.954 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.960+0000 7fea96ac9700 1 -- 172.21.15.27:0/1018275660 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fea90108cf0 con 0x7fea800640b0 2021-11-03T17:43:33.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.975+0000 7fea8d7fa700 1 -- 172.21.15.27:0/1018275660 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fea90108cf0 con 0x7fea800640b0 2021-11-03T17:43:33.969 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:43:33.970 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (35m) 4m ago 36m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:43:33.970 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (26m) 4m ago 36m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:43:33.971 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (26m) 3m ago 35m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:43:33.971 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (34m) 4m ago 36m 41.3M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:43:33.971 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (28m) 4m ago 31m 21.1M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:43:33.971 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (31m) 4m ago 31m 3843M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:43:33.971 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (27m) 3m ago 31m 20.4M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:43:33.972 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (28m) 3m ago 31m 22.1M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:43:33.972 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (28m) 4m ago 37m 517M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:43:33.972 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (28m) 3m ago 35m 419M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:43:33.972 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (27m) 4m ago 37m 159M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:43:33.973 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (27m) 3m ago 35m 82.5M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:43:33.973 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (35m) 4m ago 35m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:43:33.973 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (35m) 3m ago 35m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:43:33.973 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (26m) 4m ago 34m 311M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:43:33.974 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (25m) 4m ago 34m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:43:33.974 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (25m) 4m ago 34m 248M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:43:33.974 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (24m) 3m ago 33m 354M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:43:33.974 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (24m) 3m ago 33m 285M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:43:33.974 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (24m) 3m ago 33m 287M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:43:33.975 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (34m) 4m ago 35m 50.5M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:43:33.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.977+0000 7fea96ac9700 1 -- 172.21.15.27:0/1018275660 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fea800640b0 msgr2=0x7fea80066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:43:33.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.977+0000 7fea96ac9700 1 --2- 172.21.15.27:0/1018275660 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fea800640b0 0x7fea80066570 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 rx=0x7fea7c009da0 tx=0x7fea7c009b30).stop 2021-11-03T17:43:33.977 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.977+0000 7fea96ac9700 1 -- 172.21.15.27:0/1018275660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea900f55c0 msgr2=0x7fea90106f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:43:33.977 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.977+0000 7fea96ac9700 1 --2- 172.21.15.27:0/1018275660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea900f55c0 0x7fea90106f30 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7fea8400c530 tx=0x7fea84007e10).stop 2021-11-03T17:43:33.978 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.977+0000 7fea96ac9700 1 -- 172.21.15.27:0/1018275660 shutdown_connections 2021-11-03T17:43:33.978 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.977+0000 7fea96ac9700 1 --2- 172.21.15.27:0/1018275660 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fea800640b0 0x7fea80066570 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:33.978 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.977+0000 7fea96ac9700 1 --2- 172.21.15.27:0/1018275660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea900f55c0 0x7fea90106f30 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:33.978 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.977+0000 7fea96ac9700 1 --2- 172.21.15.27:0/1018275660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea900f6860 0x7fea90107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:33.979 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.977+0000 7fea96ac9700 1 -- 172.21.15.27:0/1018275660 >> 172.21.15.27:0/1018275660 conn(0x7fea900f0d20 msgr2=0x7fea900fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:43:33.979 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.977+0000 7fea96ac9700 1 -- 172.21.15.27:0/1018275660 shutdown_connections 2021-11-03T17:43:33.979 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:33.977+0000 7fea96ac9700 1 -- 172.21.15.27:0/1018275660 wait complete. 2021-11-03T17:43:34.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.076+0000 7f8e44f5c700 1 -- 172.21.15.27:0/3590857291 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e40060310 msgr2=0x7f8e400f9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:43:34.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.076+0000 7f8e44f5c700 1 --2- 172.21.15.27:0/3590857291 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e40060310 0x7f8e400f9120 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f8e30004660 tx=0x7f8e30009b30).stop 2021-11-03T17:43:34.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.076+0000 7f8e44f5c700 1 -- 172.21.15.27:0/3590857291 shutdown_connections 2021-11-03T17:43:34.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.076+0000 7f8e44f5c700 1 --2- 172.21.15.27:0/3590857291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e400f96a0 0x7f8e400fbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:34.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.076+0000 7f8e44f5c700 1 --2- 172.21.15.27:0/3590857291 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e40060310 0x7f8e400f9120 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:34.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.076+0000 7f8e44f5c700 1 -- 172.21.15.27:0/3590857291 >> 172.21.15.27:0/3590857291 conn(0x7f8e400f0ba0 msgr2=0x7f8e400f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:43:34.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.076+0000 7f8e44f5c700 1 -- 172.21.15.27:0/3590857291 shutdown_connections 2021-11-03T17:43:34.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.076+0000 7f8e44f5c700 1 -- 172.21.15.27:0/3590857291 wait complete. 2021-11-03T17:43:34.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.077+0000 7f8e44f5c700 1 Processor -- start 2021-11-03T17:43:34.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.077+0000 7f8e44f5c700 1 -- start start 2021-11-03T17:43:34.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.077+0000 7f8e44f5c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e40060310 0x7f8e40106da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:43:34.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.077+0000 7f8e44f5c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e400f96a0 0x7f8e40107290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:43:34.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.077+0000 7f8e44f5c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e40107890 con 0x7f8e400f96a0 2021-11-03T17:43:34.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.077+0000 7f8e44f5c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e401080d0 con 0x7f8e40060310 2021-11-03T17:43:34.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.077+0000 7f8e3effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e400f96a0 0x7f8e40107290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:43:34.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.077+0000 7f8e3effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e400f96a0 0x7f8e40107290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59048/0 (socket says 172.21.15.27:59048) 2021-11-03T17:43:34.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.077+0000 7f8e3effd700 1 -- 172.21.15.27:0/2266933615 learned_addr learned my addr 172.21.15.27:0/2266933615 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:43:34.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.078+0000 7f8e3effd700 1 -- 172.21.15.27:0/2266933615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e40060310 msgr2=0x7f8e40106da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:43:34.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.078+0000 7f8e3effd700 1 --2- 172.21.15.27:0/2266933615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e40060310 0x7f8e40106da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:43:34.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.078+0000 7f8e3effd700 1 -- 172.21.15.27:0/2266933615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e30005040 con 0x7f8e400f96a0 2021-11-03T17:43:34.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.078+0000 7f8e3effd700 1 --2- 172.21.15.27:0/2266933615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e400f96a0 0x7f8e40107290 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f8e34002700 tx=0x7f8e34009d60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:43:34.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.078+0000 7f8e27fff700 1 -- 172.21.15.27:0/2266933615 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e34009320 con 0x7f8e400f96a0 2021-11-03T17:43:34.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.078+0000 7f8e27fff700 1 -- 172.21.15.27:0/2266933615 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f8e3400f040 con 0x7f8e400f96a0 2021-11-03T17:43:34.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.078+0000 7f8e44f5c700 1 -- 172.21.15.27:0/2266933615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e401083b0 con 0x7f8e400f96a0 2021-11-03T17:43:34.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.078+0000 7f8e27fff700 1 -- 172.21.15.27:0/2266933615 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e34014810 con 0x7f8e400f96a0 2021-11-03T17:43:34.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.078+0000 7f8e44f5c700 1 -- 172.21.15.27:0/2266933615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e401088f0 con 0x7f8e400f96a0 2021-11-03T17:43:34.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.079+0000 7f8e44f5c700 1 -- 172.21.15.27:0/2266933615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e40059100 con 0x7f8e400f96a0 2021-11-03T17:43:34.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.079+0000 7f8e27fff700 1 -- 172.21.15.27:0/2266933615 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f8e340077b0 con 0x7f8e400f96a0 2021-11-03T17:43:34.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.080+0000 7f8e27fff700 1 --2- 172.21.15.27:0/2266933615 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8e28064100 0x7f8e280665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:43:34.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.080+0000 7f8e3f7fe700 1 --2- 172.21.15.27:0/2266933615 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8e28064100 0x7f8e280665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:43:34.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.080+0000 7f8e27fff700 1 -- 172.21.15.27:0/2266933615 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f8e34080df0 con 0x7f8e400f96a0 2021-11-03T17:43:34.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.080+0000 7f8e3f7fe700 1 --2- 172.21.15.27:0/2266933615 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8e28064100 0x7f8e280665c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f8e30004e30 tx=0x7f8e30009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:43:34.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.084+0000 7f8e27fff700 1 -- 172.21.15.27:0/2266933615 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f8e340530f0 con 0x7f8e400f96a0 2021-11-03T17:43:34.308 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.314+0000 7f8e44f5c700 1 -- 172.21.15.27:0/2266933615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8e400fee60 con 0x7f8e400f96a0 2021-11-03T17:43:34.309 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.315+0000 7f8e27fff700 1 -- 172.21.15.27:0/2266933615 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f8e34052ce0 con 0x7f8e400f96a0 2021-11-03T17:43:34.309 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:43:34.310 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:43:34.310 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:43:34.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:43:34.311 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:43:34.311 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:43:34.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:43:34.312 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:43:34.312 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:43:34.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:43:34.313 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:43:34.313 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:43:34.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:43:34.314 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:43:34.314 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:43:34.315 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:43:34.315 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:43:34.319 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:43:34.320 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.316+0000 7f8e44f5c700 1 -- 172.21.15.27:0/2266933615 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8e28064100 msgr2=0x7f8e280665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:43:34.321 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.316+0000 7f8e44f5c700 1 --2- 172.21.15.27:0/2266933615 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8e28064100 0x7f8e280665c0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f8e30004e30 tx=0x7f8e30009b30).stop 2021-11-03T17:43:34.321 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.316+0000 7f8e44f5c700 1 -- 172.21.15.27:0/2266933615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e400f96a0 msgr2=0x7f8e40107290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:43:34.321 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.316+0000 7f8e44f5c700 1 --2- 172.21.15.27:0/2266933615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e400f96a0 0x7f8e40107290 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f8e34002700 tx=0x7f8e34009d60).stop 2021-11-03T17:43:34.321 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.316+0000 7f8e44f5c700 1 -- 172.21.15.27:0/2266933615 shutdown_connections 2021-11-03T17:43:34.322 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.316+0000 7f8e44f5c700 1 --2- 172.21.15.27:0/2266933615 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8e28064100 0x7f8e280665c0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:34.322 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.316+0000 7f8e44f5c700 1 --2- 172.21.15.27:0/2266933615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e40060310 0x7f8e40106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:43:34.322 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.316+0000 7f8e44f5c700 1 --2- 172.21.15.27:0/2266933615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e400f96a0 0x7f8e40107290 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:34.322 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.316+0000 7f8e44f5c700 1 -- 172.21.15.27:0/2266933615 >> 172.21.15.27:0/2266933615 conn(0x7f8e400f0ba0 msgr2=0x7f8e400f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:43:34.323 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.317+0000 7f8e44f5c700 1 -- 172.21.15.27:0/2266933615 shutdown_connections 2021-11-03T17:43:34.324 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.317+0000 7f8e44f5c700 1 -- 172.21.15.27:0/2266933615 wait complete. 2021-11-03T17:43:34.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.416+0000 7fb366f20700 1 -- 172.21.15.27:0/463928154 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3600f55e0 msgr2=0x7fb3600f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:43:34.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.416+0000 7fb366f20700 1 --2- 172.21.15.27:0/463928154 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3600f55e0 0x7fb3600f5a00 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7fb354004660 tx=0x7fb354009b30).stop 2021-11-03T17:43:34.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.416+0000 7fb366f20700 1 -- 172.21.15.27:0/463928154 shutdown_connections 2021-11-03T17:43:34.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.416+0000 7fb366f20700 1 --2- 172.21.15.27:0/463928154 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3600f5ef0 0x7fb3600f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:34.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.416+0000 7fb366f20700 1 --2- 172.21.15.27:0/463928154 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3600f55e0 0x7fb3600f5a00 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:34.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.416+0000 7fb366f20700 1 -- 172.21.15.27:0/463928154 >> 172.21.15.27:0/463928154 conn(0x7fb3600f0de0 msgr2=0x7fb3600f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:43:34.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.416+0000 7fb366f20700 1 -- 172.21.15.27:0/463928154 shutdown_connections 2021-11-03T17:43:34.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.416+0000 7fb366f20700 1 -- 172.21.15.27:0/463928154 wait complete. 2021-11-03T17:43:34.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.417+0000 7fb366f20700 1 Processor -- start 2021-11-03T17:43:34.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.417+0000 7fb366f20700 1 -- start start 2021-11-03T17:43:34.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.417+0000 7fb366f20700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3600f55e0 0x7fb360109cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:43:34.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.417+0000 7fb366f20700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3600f5ef0 0x7fb36010c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:43:34.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.417+0000 7fb366f20700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3600604e0 con 0x7fb3600f55e0 2021-11-03T17:43:34.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.417+0000 7fb366f20700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb360060620 con 0x7fb3600f5ef0 2021-11-03T17:43:34.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.417+0000 7fb364cbc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3600f55e0 0x7fb360109cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:43:34.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.417+0000 7fb364cbc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3600f55e0 0x7fb360109cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59056/0 (socket says 172.21.15.27:59056) 2021-11-03T17:43:34.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.417+0000 7fb364cbc700 1 -- 172.21.15.27:0/3291283850 learned_addr learned my addr 172.21.15.27:0/3291283850 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:43:34.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.418+0000 7fb364cbc700 1 -- 172.21.15.27:0/3291283850 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3600f5ef0 msgr2=0x7fb36010c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:43:34.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.418+0000 7fb364cbc700 1 --2- 172.21.15.27:0/3291283850 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3600f5ef0 0x7fb36010c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:43:34.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.418+0000 7fb364cbc700 1 -- 172.21.15.27:0/3291283850 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb354005040 con 0x7fb3600f55e0 2021-11-03T17:43:34.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.418+0000 7fb364cbc700 1 --2- 172.21.15.27:0/3291283850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3600f55e0 0x7fb360109cb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7fb354009990 tx=0x7fb354009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:43:34.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.418+0000 7fb35d7fa700 1 -- 172.21.15.27:0/3291283850 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb354018070 con 0x7fb3600f55e0 2021-11-03T17:43:34.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.418+0000 7fb35d7fa700 1 -- 172.21.15.27:0/3291283850 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fb354007ca0 con 0x7fb3600f55e0 2021-11-03T17:43:34.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.418+0000 7fb366f20700 1 -- 172.21.15.27:0/3291283850 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb36010c700 con 0x7fb3600f55e0 2021-11-03T17:43:34.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.418+0000 7fb35d7fa700 1 -- 172.21.15.27:0/3291283850 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb35401c620 con 0x7fb3600f55e0 2021-11-03T17:43:34.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.418+0000 7fb366f20700 1 -- 172.21.15.27:0/3291283850 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb36010cc30 con 0x7fb3600f55e0 2021-11-03T17:43:34.419 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.419+0000 7fb366f20700 1 -- 172.21.15.27:0/3291283850 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb360045ad0 con 0x7fb3600f55e0 2021-11-03T17:43:34.420 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.419+0000 7fb35d7fa700 1 -- 172.21.15.27:0/3291283850 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fb35401d960 con 0x7fb3600f55e0 2021-11-03T17:43:34.420 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.420+0000 7fb35d7fa700 1 --2- 172.21.15.27:0/3291283850 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb3500640b0 0x7fb350066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:43:34.420 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.420+0000 7fb35d7fa700 1 -- 172.21.15.27:0/3291283850 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fb354082580 con 0x7fb3600f55e0 2021-11-03T17:43:34.420 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.420+0000 7fb35ffff700 1 --2- 172.21.15.27:0/3291283850 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb3500640b0 0x7fb350066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:43:34.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.420+0000 7fb35ffff700 1 --2- 172.21.15.27:0/3291283850 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb3500640b0 0x7fb350066570 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7fb34c0047d0 tx=0x7fb34c007a40).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:43:34.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.424+0000 7fb35d7fa700 1 -- 172.21.15.27:0/3291283850 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fb354054940 con 0x7fb3600f55e0 2021-11-03T17:43:34.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.628+0000 7fb366f20700 1 -- 172.21.15.27:0/3291283850 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb3600586e0 con 0x7fb3600f55e0 2021-11-03T17:43:34.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.629+0000 7fb35d7fa700 1 -- 172.21.15.27:0/3291283850 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7fb354054530 con 0x7fb3600f55e0 2021-11-03T17:43:34.623 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:43:34.623 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:43:34.624 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:43:34.624 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:43:34.624 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:43:34.624 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:43:34.625 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:43:34.625 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:43:34.625 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:43:34.625 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:43:34.626 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:43:34.626 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:43:34.626 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:43:34.626 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:43:34.626 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:43:34.627 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:43:34.627 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:43:34.627 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:43:34.628 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:43:34.628 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:43:34.628 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:43:34.628 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:43:34.629 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:43:34.629 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:43:34.629 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:43:34.629 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:43:34.630 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:43:34.630 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:43:34.630 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:43:34.630 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:43:34.630 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:43:34.631 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:43:34.631 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:43:34.631 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:43:34.631 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:43:34.632 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:43:34.632 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:43:34.632 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:43:34.632 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:43:34.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.630+0000 7fb366f20700 1 -- 172.21.15.27:0/3291283850 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb3500640b0 msgr2=0x7fb350066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:43:34.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.630+0000 7fb366f20700 1 --2- 172.21.15.27:0/3291283850 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb3500640b0 0x7fb350066570 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7fb34c0047d0 tx=0x7fb34c007a40).stop 2021-11-03T17:43:34.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.630+0000 7fb366f20700 1 -- 172.21.15.27:0/3291283850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3600f55e0 msgr2=0x7fb360109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:43:34.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.630+0000 7fb366f20700 1 --2- 172.21.15.27:0/3291283850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3600f55e0 0x7fb360109cb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7fb354009990 tx=0x7fb354009b30).stop 2021-11-03T17:43:34.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.630+0000 7fb366f20700 1 -- 172.21.15.27:0/3291283850 shutdown_connections 2021-11-03T17:43:34.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.630+0000 7fb366f20700 1 --2- 172.21.15.27:0/3291283850 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb3500640b0 0x7fb350066570 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:34.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.630+0000 7fb366f20700 1 --2- 172.21.15.27:0/3291283850 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3600f55e0 0x7fb360109cb0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:43:34.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.630+0000 7fb366f20700 1 --2- 172.21.15.27:0/3291283850 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3600f5ef0 0x7fb36010c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:43:34.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.630+0000 7fb366f20700 1 -- 172.21.15.27:0/3291283850 >> 172.21.15.27:0/3291283850 conn(0x7fb3600f0de0 msgr2=0x7fb3600fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:43:34.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.631+0000 7fb366f20700 1 -- 172.21.15.27:0/3291283850 shutdown_connections 2021-11-03T17:43:34.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:43:34.631+0000 7fb366f20700 1 -- 172.21.15.27:0/3291283850 wait complete. 2021-11-03T17:43:34.636 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:43:34.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:34 smithi115 ceph-mon[65453]: from='client.35000 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:43:34.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:34 smithi115 ceph-mon[65453]: from='client.35004 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:43:34.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:34 smithi115 ceph-mon[65453]: pgmap v868: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:34.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:34 smithi027 ceph-mon[97244]: from='client.35000 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:43:34.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:34 smithi027 ceph-mon[97244]: from='client.35004 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:43:34.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:34 smithi027 ceph-mon[97244]: pgmap v868: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:35.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:35 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2266933615' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:35.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1657.398431 secs 2021-11-03T17:43:35.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:35 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3291283850' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:43:35.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:35 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2266933615' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:35.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1657.398431 secs 2021-11-03T17:43:35.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:35 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3291283850' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:43:36.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:36 smithi115 ceph-mon[65453]: pgmap v869: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:36.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:36 smithi027 ceph-mon[97244]: pgmap v869: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:38.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:38 smithi115 ceph-mon[65453]: pgmap v870: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:38.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:38 smithi027 ceph-mon[97244]: pgmap v870: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:39.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:43:39.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:43:39.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:43:39.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:39.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:39.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:39.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:39.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:43:39.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:43:39.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:43:39.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:39.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:39.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:39.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:40.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:40 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:43:40.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1662.398521 secs 2021-11-03T17:43:40.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:40 smithi115 ceph-mon[65453]: pgmap v871: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:40.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:40 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:43:40.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1662.398521 secs 2021-11-03T17:43:40.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:40 smithi027 ceph-mon[97244]: pgmap v871: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:43.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:43 smithi115 ceph-mon[65453]: pgmap v872: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:43.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:43 smithi027 ceph-mon[97244]: pgmap v872: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:45.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:45 smithi115 ceph-mon[65453]: pgmap v873: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:45.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1667.398643 secs 2021-11-03T17:43:45.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:45 smithi027 ceph-mon[97244]: pgmap v873: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:45.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1667.398643 secs 2021-11-03T17:43:47.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:47 smithi115 ceph-mon[65453]: pgmap v874: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:47.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:47 smithi027 ceph-mon[97244]: pgmap v874: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:49.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:49 smithi115 ceph-mon[65453]: pgmap v875: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:49.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:43:49.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:43:49.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:43:49.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:49.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:49.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:49 smithi027 ceph-mon[97244]: pgmap v875: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:49.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:43:49.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:43:49.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:43:49.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:49.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:50.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:50.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:50.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:50 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:43:50.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1672.398793 secs 2021-11-03T17:43:50.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:50.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:43:50.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:50 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:43:50.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1672.398793 secs 2021-11-03T17:43:52.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:51 smithi115 ceph-mon[65453]: pgmap v876: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:52.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:51 smithi027 ceph-mon[97244]: pgmap v876: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:54.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:53 smithi115 ceph-mon[65453]: pgmap v877: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:54.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:53 smithi027 ceph-mon[97244]: pgmap v877: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:55.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:54 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1677.398918 secs 2021-11-03T17:43:55.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:54 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1677.398918 secs 2021-11-03T17:43:56.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:55 smithi115 ceph-mon[65453]: pgmap v878: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:56.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:43:56.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:43:56.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:55 smithi027 ceph-mon[97244]: pgmap v878: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:56.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:43:56.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:43:58.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:57 smithi115 ceph-mon[65453]: pgmap v879: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:43:58.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:57 smithi027 ceph-mon[97244]: pgmap v879: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:00.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:59 smithi115 ceph-mon[65453]: pgmap v880: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:00.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:44:00.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:44:00.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:44:00.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:00.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:00.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:00.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:00.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:43:59 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1682.399060 secs 2021-11-03T17:44:00.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:59 smithi027 ceph-mon[97244]: pgmap v880: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:00.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:44:00.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:44:00.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:44:00.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:00.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:00.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:00.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:00.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:43:59 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1682.399060 secs 2021-11-03T17:44:01.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:00 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:44:01.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:00 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:44:02.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:01 smithi115 ceph-mon[65453]: pgmap v881: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:02.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:01 smithi027 ceph-mon[97244]: pgmap v881: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:04.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:03 smithi115 ceph-mon[65453]: pgmap v882: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:04.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:03 smithi027 ceph-mon[97244]: pgmap v882: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:04.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.745+0000 7f1fa1c79700 1 -- 172.21.15.27:0/304091365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f9c0f55c0 msgr2=0x7f1f9c0f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:44:04.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.745+0000 7f1fa1c79700 1 --2- 172.21.15.27:0/304091365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f9c0f55c0 0x7f1f9c0f59e0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7f1f8c0099e0 tx=0x7f1f8c004d10).stop 2021-11-03T17:44:04.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.745+0000 7f1fa1c79700 1 -- 172.21.15.27:0/304091365 shutdown_connections 2021-11-03T17:44:04.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.745+0000 7f1fa1c79700 1 --2- 172.21.15.27:0/304091365 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f9c0f6860 0x7f1f9c0f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:04.744 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.745+0000 7f1fa1c79700 1 --2- 172.21.15.27:0/304091365 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f9c0f55c0 0x7f1f9c0f59e0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:04.744 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.745+0000 7f1fa1c79700 1 -- 172.21.15.27:0/304091365 >> 172.21.15.27:0/304091365 conn(0x7f1f9c0f0d40 msgr2=0x7f1f9c0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:44:04.744 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.745+0000 7f1fa1c79700 1 -- 172.21.15.27:0/304091365 shutdown_connections 2021-11-03T17:44:04.744 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.745+0000 7f1fa1c79700 1 -- 172.21.15.27:0/304091365 wait complete. 2021-11-03T17:44:04.745 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.746+0000 7f1fa1c79700 1 Processor -- start 2021-11-03T17:44:04.745 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.746+0000 7f1fa1c79700 1 -- start start 2021-11-03T17:44:04.745 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.746+0000 7f1fa1c79700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f9c0f55c0 0x7f1f9c0f9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:44:04.745 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.746+0000 7f1fa1c79700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f9c0f6860 0x7f1f9c0f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:44:04.746 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.746+0000 7f1fa1c79700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c0f8100 con 0x7f1f9c0f6860 2021-11-03T17:44:04.746 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.746+0000 7f1fa1c79700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9c0f8240 con 0x7f1f9c0f55c0 2021-11-03T17:44:04.746 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.746+0000 7f1f9affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f9c0f6860 0x7f1f9c0f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:44:04.746 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.746+0000 7f1f9affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f9c0f6860 0x7f1f9c0f7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59076/0 (socket says 172.21.15.27:59076) 2021-11-03T17:44:04.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.746+0000 7f1f9affd700 1 -- 172.21.15.27:0/58206588 learned_addr learned my addr 172.21.15.27:0/58206588 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:44:04.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.746+0000 7f1f9b7fe700 1 --2- 172.21.15.27:0/58206588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f9c0f55c0 0x7f1f9c0f9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:44:04.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.747+0000 7f1f9affd700 1 -- 172.21.15.27:0/58206588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f9c0f55c0 msgr2=0x7f1f9c0f9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:44:04.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.747+0000 7f1f9affd700 1 --2- 172.21.15.27:0/58206588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f9c0f55c0 0x7f1f9c0f9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:04.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.747+0000 7f1f9affd700 1 -- 172.21.15.27:0/58206588 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f8c009640 con 0x7f1f9c0f6860 2021-11-03T17:44:04.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.747+0000 7f1f9affd700 1 --2- 172.21.15.27:0/58206588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f9c0f6860 0x7f1f9c0f7ba0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7f1f900026d0 tx=0x7f1f90009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:44:04.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.747+0000 7f1f83fff700 1 -- 172.21.15.27:0/58206588 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f900079c0 con 0x7f1f9c0f6860 2021-11-03T17:44:04.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.747+0000 7f1f83fff700 1 -- 172.21.15.27:0/58206588 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f1f9000f040 con 0x7f1f9c0f6860 2021-11-03T17:44:04.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.747+0000 7f1fa1c79700 1 -- 172.21.15.27:0/58206588 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f9c0f8520 con 0x7f1f9c0f6860 2021-11-03T17:44:04.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.747+0000 7f1f83fff700 1 -- 172.21.15.27:0/58206588 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f900148b0 con 0x7f1f9c0f6860 2021-11-03T17:44:04.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.747+0000 7f1fa1c79700 1 -- 172.21.15.27:0/58206588 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f9c10f090 con 0x7f1f9c0f6860 2021-11-03T17:44:04.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.748+0000 7f1fa1c79700 1 -- 172.21.15.27:0/58206588 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f9c045ad0 con 0x7f1f9c0f6860 2021-11-03T17:44:04.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.748+0000 7f1f83fff700 1 -- 172.21.15.27:0/58206588 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f1f90007b20 con 0x7f1f9c0f6860 2021-11-03T17:44:04.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.749+0000 7f1f83fff700 1 --2- 172.21.15.27:0/58206588 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1f840640b0 0x7f1f84066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:44:04.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.749+0000 7f1f83fff700 1 -- 172.21.15.27:0/58206588 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f1f900806c0 con 0x7f1f9c0f6860 2021-11-03T17:44:04.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.749+0000 7f1f9b7fe700 1 --2- 172.21.15.27:0/58206588 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1f840640b0 0x7f1f84066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:44:04.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.749+0000 7f1f9b7fe700 1 --2- 172.21.15.27:0/58206588 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1f840640b0 0x7f1f84066570 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f1f8c0099b0 tx=0x7f1f8c004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:44:04.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.753+0000 7f1f83fff700 1 -- 172.21.15.27:0/58206588 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f1f900529c0 con 0x7f1f9c0f6860 2021-11-03T17:44:04.927 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.933+0000 7f1fa1c79700 1 -- 172.21.15.27:0/58206588 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f9c0faca0 con 0x7f1f840640b0 2021-11-03T17:44:04.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.935+0000 7f1f83fff700 1 -- 172.21.15.27:0/58206588 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f1f9c0faca0 con 0x7f1f840640b0 2021-11-03T17:44:04.930 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.936+0000 7f1fa1c79700 1 -- 172.21.15.27:0/58206588 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1f840640b0 msgr2=0x7f1f84066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:44:04.931 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.936+0000 7f1fa1c79700 1 --2- 172.21.15.27:0/58206588 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1f840640b0 0x7f1f84066570 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f1f8c0099b0 tx=0x7f1f8c004d10).stop 2021-11-03T17:44:04.931 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.936+0000 7f1fa1c79700 1 -- 172.21.15.27:0/58206588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f9c0f6860 msgr2=0x7f1f9c0f7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:44:04.931 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.936+0000 7f1fa1c79700 1 --2- 172.21.15.27:0/58206588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f9c0f6860 0x7f1f9c0f7ba0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7f1f900026d0 tx=0x7f1f90009de0).stop 2021-11-03T17:44:04.931 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.936+0000 7f1fa1c79700 1 -- 172.21.15.27:0/58206588 shutdown_connections 2021-11-03T17:44:04.932 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.936+0000 7f1fa1c79700 1 --2- 172.21.15.27:0/58206588 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1f840640b0 0x7f1f84066570 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:04.932 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.936+0000 7f1fa1c79700 1 --2- 172.21.15.27:0/58206588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f9c0f55c0 0x7f1f9c0f9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:04.932 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.936+0000 7f1fa1c79700 1 --2- 172.21.15.27:0/58206588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f9c0f6860 0x7f1f9c0f7ba0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:04.932 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.936+0000 7f1fa1c79700 1 -- 172.21.15.27:0/58206588 >> 172.21.15.27:0/58206588 conn(0x7f1f9c0f0d40 msgr2=0x7f1f9c05fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:44:04.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.937+0000 7f1fa1c79700 1 -- 172.21.15.27:0/58206588 shutdown_connections 2021-11-03T17:44:04.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:04.937+0000 7f1fa1c79700 1 -- 172.21.15.27:0/58206588 wait complete. 2021-11-03T17:44:04.940 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:44:05.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.037+0000 7f1b4e8bf700 1 -- 172.21.15.27:0/787287068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b480f55c0 msgr2=0x7f1b480f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:44:05.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.037+0000 7f1b4e8bf700 1 --2- 172.21.15.27:0/787287068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b480f55c0 0x7f1b480f59e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f1b30004660 tx=0x7f1b30009b30).stop 2021-11-03T17:44:05.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.037+0000 7f1b4e8bf700 1 -- 172.21.15.27:0/787287068 shutdown_connections 2021-11-03T17:44:05.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.037+0000 7f1b4e8bf700 1 --2- 172.21.15.27:0/787287068 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b480f5ed0 0x7f1b480f6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:05.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.037+0000 7f1b4e8bf700 1 --2- 172.21.15.27:0/787287068 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b480f55c0 0x7f1b480f59e0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:05.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.037+0000 7f1b4e8bf700 1 -- 172.21.15.27:0/787287068 >> 172.21.15.27:0/787287068 conn(0x7f1b480f0dc0 msgr2=0x7f1b480f31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:44:05.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.037+0000 7f1b4e8bf700 1 -- 172.21.15.27:0/787287068 shutdown_connections 2021-11-03T17:44:05.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.037+0000 7f1b4e8bf700 1 -- 172.21.15.27:0/787287068 wait complete. 2021-11-03T17:44:05.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.037+0000 7f1b4e8bf700 1 Processor -- start 2021-11-03T17:44:05.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.037+0000 7f1b4e8bf700 1 -- start start 2021-11-03T17:44:05.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.038+0000 7f1b4e8bf700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b480f55c0 0x7f1b48106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:44:05.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.038+0000 7f1b4e8bf700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b480f5ed0 0x7f1b48107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:44:05.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.038+0000 7f1b4e8bf700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b48107a10 con 0x7f1b480f55c0 2021-11-03T17:44:05.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.038+0000 7f1b4e8bf700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b48107b50 con 0x7f1b480f5ed0 2021-11-03T17:44:05.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.038+0000 7f1b47fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b480f55c0 0x7f1b48106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:44:05.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.038+0000 7f1b47fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b480f55c0 0x7f1b48106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59084/0 (socket says 172.21.15.27:59084) 2021-11-03T17:44:05.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.038+0000 7f1b47fff700 1 -- 172.21.15.27:0/354146666 learned_addr learned my addr 172.21.15.27:0/354146666 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:44:05.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.038+0000 7f1b477fe700 1 --2- 172.21.15.27:0/354146666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b480f5ed0 0x7f1b48107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:44:05.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.038+0000 7f1b47fff700 1 -- 172.21.15.27:0/354146666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b480f5ed0 msgr2=0x7f1b48107460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:44:05.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.038+0000 7f1b47fff700 1 --2- 172.21.15.27:0/354146666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b480f5ed0 0x7f1b48107460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:05.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.038+0000 7f1b47fff700 1 -- 172.21.15.27:0/354146666 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b30005040 con 0x7f1b480f55c0 2021-11-03T17:44:05.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.038+0000 7f1b47fff700 1 --2- 172.21.15.27:0/354146666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b480f55c0 0x7f1b48106f70 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f1b30009990 tx=0x7f1b30009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:44:05.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.038+0000 7f1b44ff9700 1 -- 172.21.15.27:0/354146666 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b30018070 con 0x7f1b480f55c0 2021-11-03T17:44:05.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.038+0000 7f1b44ff9700 1 -- 172.21.15.27:0/354146666 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f1b30007ba0 con 0x7f1b480f55c0 2021-11-03T17:44:05.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.038+0000 7f1b4e8bf700 1 -- 172.21.15.27:0/354146666 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b48108490 con 0x7f1b480f55c0 2021-11-03T17:44:05.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.038+0000 7f1b44ff9700 1 -- 172.21.15.27:0/354146666 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b3001c3c0 con 0x7f1b480f55c0 2021-11-03T17:44:05.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.038+0000 7f1b4e8bf700 1 -- 172.21.15.27:0/354146666 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b481089f0 con 0x7f1b480f55c0 2021-11-03T17:44:05.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.039+0000 7f1b4e8bf700 1 -- 172.21.15.27:0/354146666 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b48045ad0 con 0x7f1b480f55c0 2021-11-03T17:44:05.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.040+0000 7f1b44ff9700 1 -- 172.21.15.27:0/354146666 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f1b3001d420 con 0x7f1b480f55c0 2021-11-03T17:44:05.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.040+0000 7f1b44ff9700 1 --2- 172.21.15.27:0/354146666 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1b340640b0 0x7f1b34066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:44:05.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.040+0000 7f1b44ff9700 1 -- 172.21.15.27:0/354146666 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f1b300826f0 con 0x7f1b480f55c0 2021-11-03T17:44:05.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.040+0000 7f1b477fe700 1 --2- 172.21.15.27:0/354146666 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1b340640b0 0x7f1b34066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:44:05.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.041+0000 7f1b477fe700 1 --2- 172.21.15.27:0/354146666 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1b340640b0 0x7f1b34066570 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f1b380096c0 tx=0x7f1b38009be0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:44:05.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.044+0000 7f1b44ff9700 1 -- 172.21.15.27:0/354146666 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f1b30054990 con 0x7f1b480f55c0 2021-11-03T17:44:05.209 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.215+0000 7f1b4e8bf700 1 -- 172.21.15.27:0/354146666 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b480ff030 con 0x7f1b340640b0 2021-11-03T17:44:05.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.229+0000 7f1b44ff9700 1 -- 172.21.15.27:0/354146666 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f1b480ff030 con 0x7f1b340640b0 2021-11-03T17:44:05.224 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:44:05.224 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (35m) 5m ago 36m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:44:05.224 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (27m) 5m ago 36m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:44:05.224 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (27m) 4m ago 36m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:44:05.225 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (35m) 5m ago 36m 41.3M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:44:05.225 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (29m) 5m ago 31m 21.1M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:44:05.225 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (32m) 5m ago 32m 3843M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:44:05.226 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (27m) 4m ago 32m 20.4M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:44:05.226 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (29m) 4m ago 32m 22.1M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:44:05.226 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (29m) 5m ago 38m 517M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:44:05.226 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (28m) 4m ago 36m 419M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:44:05.227 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (28m) 5m ago 38m 159M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:44:05.227 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (27m) 4m ago 36m 82.5M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:44:05.227 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (36m) 5m ago 36m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:44:05.227 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (35m) 4m ago 35m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:44:05.228 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (26m) 5m ago 35m 311M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:44:05.228 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (26m) 5m ago 34m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:44:05.228 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (25m) 5m ago 34m 248M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:44:05.228 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (25m) 4m ago 34m 354M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:44:05.229 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (25m) 4m ago 33m 285M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:44:05.229 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (24m) 4m ago 33m 287M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:44:05.229 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (35m) 5m ago 36m 50.5M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:44:05.230 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.231+0000 7f1b4e8bf700 1 -- 172.21.15.27:0/354146666 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1b340640b0 msgr2=0x7f1b34066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:44:05.230 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.231+0000 7f1b4e8bf700 1 --2- 172.21.15.27:0/354146666 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1b340640b0 0x7f1b34066570 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7f1b380096c0 tx=0x7f1b38009be0).stop 2021-11-03T17:44:05.231 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.231+0000 7f1b4e8bf700 1 -- 172.21.15.27:0/354146666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b480f55c0 msgr2=0x7f1b48106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:44:05.231 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.231+0000 7f1b4e8bf700 1 --2- 172.21.15.27:0/354146666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b480f55c0 0x7f1b48106f70 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7f1b30009990 tx=0x7f1b30009b30).stop 2021-11-03T17:44:05.231 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.231+0000 7f1b4e8bf700 1 -- 172.21.15.27:0/354146666 shutdown_connections 2021-11-03T17:44:05.231 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.231+0000 7f1b4e8bf700 1 --2- 172.21.15.27:0/354146666 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1b340640b0 0x7f1b34066570 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:05.232 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.231+0000 7f1b4e8bf700 1 --2- 172.21.15.27:0/354146666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1b480f55c0 0x7f1b48106f70 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:05.232 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.231+0000 7f1b4e8bf700 1 --2- 172.21.15.27:0/354146666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1b480f5ed0 0x7f1b48107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:05.232 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.231+0000 7f1b4e8bf700 1 -- 172.21.15.27:0/354146666 >> 172.21.15.27:0/354146666 conn(0x7f1b480f0dc0 msgr2=0x7f1b480fa350 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:44:05.232 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.231+0000 7f1b4e8bf700 1 -- 172.21.15.27:0/354146666 shutdown_connections 2021-11-03T17:44:05.232 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.231+0000 7f1b4e8bf700 1 -- 172.21.15.27:0/354146666 wait complete. 2021-11-03T17:44:05.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:04 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1687.399194 secs 2021-11-03T17:44:05.267 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:04 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1687.399194 secs 2021-11-03T17:44:05.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.331+0000 7fb8346c2700 1 -- 172.21.15.27:0/4093002567 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb82c0f6540 msgr2=0x7fb82c0f69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:44:05.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.331+0000 7fb8346c2700 1 --2- 172.21.15.27:0/4093002567 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb82c0f6540 0x7fb82c0f69c0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7fb828004660 tx=0x7fb828009b30).stop 2021-11-03T17:44:05.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.331+0000 7fb8346c2700 1 -- 172.21.15.27:0/4093002567 shutdown_connections 2021-11-03T17:44:05.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.331+0000 7fb8346c2700 1 --2- 172.21.15.27:0/4093002567 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb82c0f6540 0x7fb82c0f69c0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:05.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.331+0000 7fb8346c2700 1 --2- 172.21.15.27:0/4093002567 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb82c0f5c30 0x7fb82c0f6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:05.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.331+0000 7fb8346c2700 1 -- 172.21.15.27:0/4093002567 >> 172.21.15.27:0/4093002567 conn(0x7fb82c0f0540 msgr2=0x7fb82c0f2960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:44:05.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.331+0000 7fb8346c2700 1 -- 172.21.15.27:0/4093002567 shutdown_connections 2021-11-03T17:44:05.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.331+0000 7fb8346c2700 1 -- 172.21.15.27:0/4093002567 wait complete. 2021-11-03T17:44:05.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.331+0000 7fb8346c2700 1 Processor -- start 2021-11-03T17:44:05.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.331+0000 7fb8346c2700 1 -- start start 2021-11-03T17:44:05.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.331+0000 7fb8346c2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb82c0f5c30 0x7fb82c106ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:44:05.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.331+0000 7fb8346c2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb82c0f6540 0x7fb82c1073e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:44:05.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.331+0000 7fb8346c2700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82c107990 con 0x7fb82c0f6540 2021-11-03T17:44:05.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.331+0000 7fb8346c2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82c107ad0 con 0x7fb82c0f5c30 2021-11-03T17:44:05.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.332+0000 7fb831c5d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb82c0f6540 0x7fb82c1073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:44:05.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.332+0000 7fb831c5d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb82c0f6540 0x7fb82c1073e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59096/0 (socket says 172.21.15.27:59096) 2021-11-03T17:44:05.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.332+0000 7fb831c5d700 1 -- 172.21.15.27:0/726473323 learned_addr learned my addr 172.21.15.27:0/726473323 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:44:05.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.332+0000 7fb83245e700 1 --2- 172.21.15.27:0/726473323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb82c0f5c30 0x7fb82c106ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:44:05.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.332+0000 7fb831c5d700 1 -- 172.21.15.27:0/726473323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb82c0f5c30 msgr2=0x7fb82c106ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:44:05.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.332+0000 7fb831c5d700 1 --2- 172.21.15.27:0/726473323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb82c0f5c30 0x7fb82c106ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:05.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.332+0000 7fb831c5d700 1 -- 172.21.15.27:0/726473323 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb828005040 con 0x7fb82c0f6540 2021-11-03T17:44:05.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.332+0000 7fb831c5d700 1 --2- 172.21.15.27:0/726473323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb82c0f6540 0x7fb82c1073e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7fb828009990 tx=0x7fb828009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:44:05.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.332+0000 7fb822ffd700 1 -- 172.21.15.27:0/726473323 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb828018070 con 0x7fb82c0f6540 2021-11-03T17:44:05.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.332+0000 7fb822ffd700 1 -- 172.21.15.27:0/726473323 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fb828007ba0 con 0x7fb82c0f6540 2021-11-03T17:44:05.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.332+0000 7fb8346c2700 1 -- 172.21.15.27:0/726473323 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb82c108410 con 0x7fb82c0f6540 2021-11-03T17:44:05.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.332+0000 7fb822ffd700 1 -- 172.21.15.27:0/726473323 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb82801c3c0 con 0x7fb82c0f6540 2021-11-03T17:44:05.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.332+0000 7fb8346c2700 1 -- 172.21.15.27:0/726473323 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb82c108970 con 0x7fb82c0f6540 2021-11-03T17:44:05.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.333+0000 7fb8346c2700 1 -- 172.21.15.27:0/726473323 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb82c045ad0 con 0x7fb82c0f6540 2021-11-03T17:44:05.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.334+0000 7fb822ffd700 1 -- 172.21.15.27:0/726473323 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fb82801d420 con 0x7fb82c0f6540 2021-11-03T17:44:05.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.334+0000 7fb822ffd700 1 --2- 172.21.15.27:0/726473323 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb8180640b0 0x7fb818066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:44:05.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.334+0000 7fb822ffd700 1 -- 172.21.15.27:0/726473323 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fb828082550 con 0x7fb82c0f6540 2021-11-03T17:44:05.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.334+0000 7fb83245e700 1 --2- 172.21.15.27:0/726473323 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb8180640b0 0x7fb818066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:44:05.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.335+0000 7fb83245e700 1 --2- 172.21.15.27:0/726473323 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb8180640b0 0x7fb818066570 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7fb81c004a70 tx=0x7fb81c00b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:44:05.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.338+0000 7fb822ffd700 1 -- 172.21.15.27:0/726473323 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fb828054910 con 0x7fb82c0f6540 2021-11-03T17:44:05.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.565+0000 7fb8346c2700 1 -- 172.21.15.27:0/726473323 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb82c1090c0 con 0x7fb82c0f6540 2021-11-03T17:44:05.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.565+0000 7fb822ffd700 1 -- 172.21.15.27:0/726473323 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fb828054500 con 0x7fb82c0f6540 2021-11-03T17:44:05.560 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:44:05.560 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:44:05.560 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:44:05.560 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:44:05.561 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:44:05.561 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:44:05.561 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:44:05.561 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:44:05.562 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:44:05.562 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:44:05.562 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:44:05.562 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:44:05.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:44:05.563 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:44:05.563 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:44:05.563 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:44:05.563 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:44:05.564 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:44:05.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.567+0000 7fb8346c2700 1 -- 172.21.15.27:0/726473323 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb8180640b0 msgr2=0x7fb818066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:44:05.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.567+0000 7fb8346c2700 1 --2- 172.21.15.27:0/726473323 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb8180640b0 0x7fb818066570 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 rx=0x7fb81c004a70 tx=0x7fb81c00b040).stop 2021-11-03T17:44:05.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.567+0000 7fb8346c2700 1 -- 172.21.15.27:0/726473323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb82c0f6540 msgr2=0x7fb82c1073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:44:05.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.567+0000 7fb8346c2700 1 --2- 172.21.15.27:0/726473323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb82c0f6540 0x7fb82c1073e0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7fb828009990 tx=0x7fb828009b30).stop 2021-11-03T17:44:05.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.567+0000 7fb8346c2700 1 -- 172.21.15.27:0/726473323 shutdown_connections 2021-11-03T17:44:05.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.567+0000 7fb8346c2700 1 --2- 172.21.15.27:0/726473323 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb8180640b0 0x7fb818066570 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:05.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.567+0000 7fb8346c2700 1 --2- 172.21.15.27:0/726473323 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb82c0f5c30 0x7fb82c106ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:05.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.567+0000 7fb8346c2700 1 --2- 172.21.15.27:0/726473323 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb82c0f6540 0x7fb82c1073e0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:05.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.567+0000 7fb8346c2700 1 -- 172.21.15.27:0/726473323 >> 172.21.15.27:0/726473323 conn(0x7fb82c0f0540 msgr2=0x7fb82c0f5180 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:44:05.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.567+0000 7fb8346c2700 1 -- 172.21.15.27:0/726473323 shutdown_connections 2021-11-03T17:44:05.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.567+0000 7fb8346c2700 1 -- 172.21.15.27:0/726473323 wait complete. 2021-11-03T17:44:05.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.667+0000 7f0e0dd27700 1 -- 172.21.15.27:0/2360865829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e080f6860 msgr2=0x7f0e080f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:44:05.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.667+0000 7f0e0dd27700 1 --2- 172.21.15.27:0/2360865829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e080f6860 0x7f0e080f6cc0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f0df8004660 tx=0x7f0df8009b30).stop 2021-11-03T17:44:05.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.667+0000 7f0e0dd27700 1 -- 172.21.15.27:0/2360865829 shutdown_connections 2021-11-03T17:44:05.665 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.667+0000 7f0e0dd27700 1 --2- 172.21.15.27:0/2360865829 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e080f6860 0x7f0e080f6cc0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:05.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.667+0000 7f0e0dd27700 1 --2- 172.21.15.27:0/2360865829 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e080f55c0 0x7f0e080f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:05.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.667+0000 7f0e0dd27700 1 -- 172.21.15.27:0/2360865829 >> 172.21.15.27:0/2360865829 conn(0x7f0e080f0d40 msgr2=0x7f0e080f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:44:05.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.667+0000 7f0e0dd27700 1 -- 172.21.15.27:0/2360865829 shutdown_connections 2021-11-03T17:44:05.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.667+0000 7f0e0dd27700 1 -- 172.21.15.27:0/2360865829 wait complete. 2021-11-03T17:44:05.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.667+0000 7f0e0dd27700 1 Processor -- start 2021-11-03T17:44:05.670 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.668+0000 7f0e0dd27700 1 -- start start 2021-11-03T17:44:05.670 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.668+0000 7f0e0dd27700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e080f55c0 0x7f0e08105ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:44:05.670 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.668+0000 7f0e0dd27700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e080f6860 0x7f0e081063b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:44:05.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.668+0000 7f0e0dd27700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e081069b0 con 0x7f0e080f6860 2021-11-03T17:44:05.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.668+0000 7f0e0dd27700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e081071c0 con 0x7f0e080f55c0 2021-11-03T17:44:05.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.668+0000 7f0e06ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e080f6860 0x7f0e081063b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:44:05.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.668+0000 7f0e06ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e080f6860 0x7f0e081063b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59106/0 (socket says 172.21.15.27:59106) 2021-11-03T17:44:05.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.668+0000 7f0e06ffd700 1 -- 172.21.15.27:0/3772443921 learned_addr learned my addr 172.21.15.27:0/3772443921 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:44:05.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.668+0000 7f0e077fe700 1 --2- 172.21.15.27:0/3772443921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e080f55c0 0x7f0e08105ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:44:05.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.668+0000 7f0e06ffd700 1 -- 172.21.15.27:0/3772443921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e080f55c0 msgr2=0x7f0e08105ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:44:05.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.668+0000 7f0e06ffd700 1 --2- 172.21.15.27:0/3772443921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e080f55c0 0x7f0e08105ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:05.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.668+0000 7f0e06ffd700 1 -- 172.21.15.27:0/3772443921 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df8005040 con 0x7f0e080f6860 2021-11-03T17:44:05.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.668+0000 7f0e06ffd700 1 --2- 172.21.15.27:0/3772443921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e080f6860 0x7f0e081063b0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f0df8009990 tx=0x7f0df8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:44:05.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.669+0000 7f0deffff700 1 -- 172.21.15.27:0/3772443921 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0df801a070 con 0x7f0e080f6860 2021-11-03T17:44:05.676 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.669+0000 7f0e0dd27700 1 -- 172.21.15.27:0/3772443921 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e081073e0 con 0x7f0e080f6860 2021-11-03T17:44:05.676 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.669+0000 7f0deffff700 1 -- 172.21.15.27:0/3772443921 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f0df8009630 con 0x7f0e080f6860 2021-11-03T17:44:05.676 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.669+0000 7f0deffff700 1 -- 172.21.15.27:0/3772443921 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0df801f820 con 0x7f0e080f6860 2021-11-03T17:44:05.677 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.669+0000 7f0e0dd27700 1 -- 172.21.15.27:0/3772443921 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e08107940 con 0x7f0e080f6860 2021-11-03T17:44:05.679 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.670+0000 7f0e0dd27700 1 -- 172.21.15.27:0/3772443921 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e080f49b0 con 0x7f0e080f6860 2021-11-03T17:44:05.680 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.670+0000 7f0deffff700 1 -- 172.21.15.27:0/3772443921 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f0df8012b80 con 0x7f0e080f6860 2021-11-03T17:44:05.680 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.670+0000 7f0deffff700 1 --2- 172.21.15.27:0/3772443921 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0df00640b0 0x7f0df0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:44:05.681 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.671+0000 7f0deffff700 1 -- 172.21.15.27:0/3772443921 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f0df8084a20 con 0x7f0e080f6860 2021-11-03T17:44:05.681 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.671+0000 7f0e077fe700 1 --2- 172.21.15.27:0/3772443921 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0df00640b0 0x7f0df0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:44:05.682 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.671+0000 7f0e077fe700 1 --2- 172.21.15.27:0/3772443921 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0df00640b0 0x7f0df0066570 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f0e08105c30 tx=0x7f0dfc00b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:44:05.682 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.675+0000 7f0deffff700 1 -- 172.21.15.27:0/3772443921 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f0df8056c60 con 0x7f0e080f6860 2021-11-03T17:44:05.874 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.879+0000 7f0e0dd27700 1 -- 172.21.15.27:0/3772443921 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0e08045b20 con 0x7f0e080f6860 2021-11-03T17:44:05.874 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.880+0000 7f0deffff700 1 -- 172.21.15.27:0/3772443921 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f0df8056850 con 0x7f0e080f6860 2021-11-03T17:44:05.875 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:44:05.875 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:44:05.875 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:44:05.876 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:44:05.876 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:44:05.876 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:44:05.876 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:44:05.877 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:44:05.877 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:44:05.877 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:44:05.877 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:44:05.878 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:44:05.878 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:44:05.878 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:44:05.878 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:44:05.878 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:44:05.879 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:44:05.879 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:44:05.879 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:44:05.879 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:44:05.880 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:44:05.880 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:44:05.880 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:44:05.880 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:44:05.881 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:44:05.881 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:44:05.881 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:44:05.881 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:44:05.882 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:44:05.882 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:44:05.882 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:44:05.883 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:44:05.883 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:44:05.883 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:44:05.883 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:44:05.884 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:44:05.884 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:44:05.884 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:44:05.884 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:44:05.885 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.882+0000 7f0e0dd27700 1 -- 172.21.15.27:0/3772443921 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0df00640b0 msgr2=0x7f0df0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:44:05.885 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.882+0000 7f0e0dd27700 1 --2- 172.21.15.27:0/3772443921 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0df00640b0 0x7f0df0066570 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f0e08105c30 tx=0x7f0dfc00b040).stop 2021-11-03T17:44:05.886 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.882+0000 7f0e0dd27700 1 -- 172.21.15.27:0/3772443921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e080f6860 msgr2=0x7f0e081063b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:44:05.886 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.882+0000 7f0e0dd27700 1 --2- 172.21.15.27:0/3772443921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e080f6860 0x7f0e081063b0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f0df8009990 tx=0x7f0df8009b30).stop 2021-11-03T17:44:05.886 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.882+0000 7f0e0dd27700 1 -- 172.21.15.27:0/3772443921 shutdown_connections 2021-11-03T17:44:05.886 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.882+0000 7f0e0dd27700 1 --2- 172.21.15.27:0/3772443921 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0df00640b0 0x7f0df0066570 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:05.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.882+0000 7f0e0dd27700 1 --2- 172.21.15.27:0/3772443921 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e080f55c0 0x7f0e08105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:05.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.882+0000 7f0e0dd27700 1 --2- 172.21.15.27:0/3772443921 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e080f6860 0x7f0e081063b0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:05.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.882+0000 7f0e0dd27700 1 -- 172.21.15.27:0/3772443921 >> 172.21.15.27:0/3772443921 conn(0x7f0e080f0d40 msgr2=0x7f0e080fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:44:05.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.882+0000 7f0e0dd27700 1 -- 172.21.15.27:0/3772443921 shutdown_connections 2021-11-03T17:44:05.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:05.882+0000 7f0e0dd27700 1 -- 172.21.15.27:0/3772443921 wait complete. 2021-11-03T17:44:05.888 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:44:06.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:05 smithi115 ceph-mon[65453]: pgmap v883: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:06.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:05 smithi115 ceph-mon[65453]: from='client.35016 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:44:06.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:05 smithi115 ceph-mon[65453]: from='client.35020 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:44:06.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:05 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/726473323' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:06.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:05 smithi027 ceph-mon[97244]: pgmap v883: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:06.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:05 smithi027 ceph-mon[97244]: from='client.35016 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:44:06.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:05 smithi027 ceph-mon[97244]: from='client.35020 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:44:06.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:05 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/726473323' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:07.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:06 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3772443921' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:44:07.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:06 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3772443921' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:44:08.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:07 smithi115 ceph-mon[65453]: pgmap v884: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:08.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:07 smithi027 ceph-mon[97244]: pgmap v884: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:10.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:09 smithi115 ceph-mon[65453]: pgmap v885: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:10.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:44:10.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:44:10.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:44:10.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:10.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:10.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:10.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:10.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:09 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1692.399337 secs 2021-11-03T17:44:10.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:09 smithi027 ceph-mon[97244]: pgmap v885: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:10.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:44:10.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:44:10.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:44:10.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:10.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:10.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:10.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:10.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:09 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1692.399337 secs 2021-11-03T17:44:11.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:10 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:44:11.267 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:10 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:44:12.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:11 smithi115 ceph-mon[65453]: pgmap v886: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:12.267 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:11 smithi027 ceph-mon[97244]: pgmap v886: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:14.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:13 smithi115 ceph-mon[65453]: pgmap v887: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:14.267 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:13 smithi027 ceph-mon[97244]: pgmap v887: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:15.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:14 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1697.399477 secs 2021-11-03T17:44:15.267 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:14 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1697.399477 secs 2021-11-03T17:44:16.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:15 smithi115 ceph-mon[65453]: pgmap v888: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:16.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:15 smithi027 ceph-mon[97244]: pgmap v888: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:18.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:17 smithi115 ceph-mon[65453]: pgmap v889: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:18.267 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:17 smithi027 ceph-mon[97244]: pgmap v889: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:19.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:19 smithi115 ceph-mon[65453]: pgmap v890: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:19.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:19 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1702.399605 secs 2021-11-03T17:44:19.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:44:19.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:44:19.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:44:19.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:19.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:19.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:19.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:20.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:19 smithi027 ceph-mon[97244]: pgmap v890: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:20.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:19 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1702.399605 secs 2021-11-03T17:44:20.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:44:20.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:44:20.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:44:20.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:20.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:20.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:20.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:21.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:20 smithi115 ceph-mon[65453]: poll took 60178.300 ms: 1 events 2021-11-03T17:44:21.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:20 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:44:21.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:20 smithi027 ceph-mon[97244]: poll took 60178.300 ms: 1 events 2021-11-03T17:44:21.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:20 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:44:22.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:21 smithi115 ceph-mon[65453]: pgmap v891: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:22.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:21 smithi027 ceph-mon[97244]: pgmap v891: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:24.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:23 smithi115 ceph-mon[65453]: pgmap v892: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:24.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:23 smithi027 ceph-mon[97244]: pgmap v892: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:25.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:24 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1707.399751 secs 2021-11-03T17:44:25.267 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:24 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1707.399751 secs 2021-11-03T17:44:26.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:25 smithi115 ceph-mon[65453]: pgmap v893: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:26.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:25 smithi027 ceph-mon[97244]: pgmap v893: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:28.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:27 smithi115 ceph-mon[65453]: pgmap v894: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:28.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:27 smithi027 ceph-mon[97244]: pgmap v894: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:30.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:29 smithi115 ceph-mon[65453]: pgmap v895: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:30.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:29 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1712.399883 secs 2021-11-03T17:44:30.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:44:30.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:44:30.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:44:30.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:30.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:30.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:30.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:30.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:29 smithi027 ceph-mon[97244]: pgmap v895: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:30.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:29 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1712.399883 secs 2021-11-03T17:44:30.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:44:30.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:44:30.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:44:30.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:30.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:30.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:30.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:31.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:30 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:44:31.267 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:30 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:44:32.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:31 smithi115 ceph-mon[65453]: pgmap v896: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:32.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:31 smithi027 ceph-mon[97244]: pgmap v896: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:34.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:33 smithi115 ceph-mon[65453]: pgmap v897: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:34.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:33 smithi027 ceph-mon[97244]: pgmap v897: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:35.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:34 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1717.400014 secs 2021-11-03T17:44:35.267 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:34 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1717.400014 secs 2021-11-03T17:44:35.990 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:35.992+0000 7f173cae0700 1 -- 172.21.15.27:0/543669327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17380f66e0 msgr2=0x7f17380f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:44:35.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:35.992+0000 7f173cae0700 1 --2- 172.21.15.27:0/543669327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17380f66e0 0x7f17380f6b00 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7f17300099e0 tx=0x7f1730004d10).stop 2021-11-03T17:44:35.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:35.992+0000 7f173cae0700 1 -- 172.21.15.27:0/543669327 shutdown_connections 2021-11-03T17:44:35.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:35.992+0000 7f173cae0700 1 --2- 172.21.15.27:0/543669327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17380f6ff0 0x7f17380f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:44:35.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:35.992+0000 7f173cae0700 1 --2- 172.21.15.27:0/543669327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17380f66e0 0x7f17380f6b00 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:35.992 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:35.992+0000 7f173cae0700 1 -- 172.21.15.27:0/543669327 >> 172.21.15.27:0/543669327 conn(0x7f17380f1f00 msgr2=0x7f17380f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:44:35.992 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:35.992+0000 7f173cae0700 1 -- 172.21.15.27:0/543669327 shutdown_connections 2021-11-03T17:44:35.992 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:35.992+0000 7f173cae0700 1 -- 172.21.15.27:0/543669327 wait complete. 2021-11-03T17:44:35.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:35.993+0000 7f173cae0700 1 Processor -- start 2021-11-03T17:44:35.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:35.993+0000 7f173cae0700 1 -- start start 2021-11-03T17:44:35.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:35.993+0000 7f173cae0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17380f66e0 0x7f17381070f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:44:35.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:35.993+0000 7f173cae0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17380f6ff0 0x7f17381075e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:44:35.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:35.993+0000 7f173cae0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1738107c20 con 0x7f17380f66e0 2021-11-03T17:44:35.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:35.993+0000 7f173cae0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17381084d0 con 0x7f17380f6ff0 2021-11-03T17:44:35.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:35.993+0000 7f173659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17380f66e0 0x7f17381070f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:44:35.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:35.993+0000 7f173659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17380f66e0 0x7f17381070f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59122/0 (socket says 172.21.15.27:59122) 2021-11-03T17:44:35.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:35.993+0000 7f173659c700 1 -- 172.21.15.27:0/1039771293 learned_addr learned my addr 172.21.15.27:0/1039771293 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:44:35.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:35.994+0000 7f173659c700 1 -- 172.21.15.27:0/1039771293 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17380f6ff0 msgr2=0x7f17381075e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:44:35.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:35.994+0000 7f173659c700 1 --2- 172.21.15.27:0/1039771293 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17380f6ff0 0x7f17381075e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:44:35.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:35.994+0000 7f173659c700 1 -- 172.21.15.27:0/1039771293 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1730009640 con 0x7f17380f66e0 2021-11-03T17:44:35.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:35.994+0000 7f173659c700 1 --2- 172.21.15.27:0/1039771293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17380f66e0 0x7f17381070f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7f1730004b70 tx=0x7f1730004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:44:35.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:35.994+0000 7f172effd700 1 -- 172.21.15.27:0/1039771293 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f173000e030 con 0x7f17380f66e0 2021-11-03T17:44:35.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:35.994+0000 7f172effd700 1 -- 172.21.15.27:0/1039771293 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f17300049b0 con 0x7f17380f66e0 2021-11-03T17:44:35.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:35.994+0000 7f173cae0700 1 -- 172.21.15.27:0/1039771293 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1738108750 con 0x7f17380f66e0 2021-11-03T17:44:35.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:35.994+0000 7f172effd700 1 -- 172.21.15.27:0/1039771293 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1730018070 con 0x7f17380f66e0 2021-11-03T17:44:35.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:35.994+0000 7f173cae0700 1 -- 172.21.15.27:0/1039771293 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1738108c30 con 0x7f17380f66e0 2021-11-03T17:44:35.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:35.995+0000 7f173cae0700 1 -- 172.21.15.27:0/1039771293 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17380f5b70 con 0x7f17380f66e0 2021-11-03T17:44:35.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:35.995+0000 7f172effd700 1 -- 172.21.15.27:0/1039771293 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f173001db40 con 0x7f17380f66e0 2021-11-03T17:44:35.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:35.996+0000 7f172effd700 1 --2- 172.21.15.27:0/1039771293 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f17240640b0 0x7f1724066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:44:35.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:35.996+0000 7f172effd700 1 -- 172.21.15.27:0/1039771293 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f17300827b0 con 0x7f17380f66e0 2021-11-03T17:44:35.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:35.996+0000 7f1735d9b700 1 --2- 172.21.15.27:0/1039771293 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f17240640b0 0x7f1724066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:44:35.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:35.996+0000 7f1735d9b700 1 --2- 172.21.15.27:0/1039771293 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f17240640b0 0x7f1724066570 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f1720009610 tx=0x7f1720006ab0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:44:36.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.000+0000 7f172effd700 1 -- 172.21.15.27:0/1039771293 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f1730054a50 con 0x7f17380f66e0 2021-11-03T17:44:36.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.179+0000 7f173cae0700 1 -- 172.21.15.27:0/1039771293 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f17380fefc0 con 0x7f17240640b0 2021-11-03T17:44:36.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.180+0000 7f172effd700 1 -- 172.21.15.27:0/1039771293 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f17380fefc0 con 0x7f17240640b0 2021-11-03T17:44:36.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.182+0000 7f173cae0700 1 -- 172.21.15.27:0/1039771293 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f17240640b0 msgr2=0x7f1724066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:44:36.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.182+0000 7f173cae0700 1 --2- 172.21.15.27:0/1039771293 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f17240640b0 0x7f1724066570 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f1720009610 tx=0x7f1720006ab0).stop 2021-11-03T17:44:36.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.182+0000 7f173cae0700 1 -- 172.21.15.27:0/1039771293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17380f66e0 msgr2=0x7f17381070f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:44:36.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.182+0000 7f173cae0700 1 --2- 172.21.15.27:0/1039771293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17380f66e0 0x7f17381070f0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7f1730004b70 tx=0x7f1730004d10).stop 2021-11-03T17:44:36.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.182+0000 7f173cae0700 1 -- 172.21.15.27:0/1039771293 shutdown_connections 2021-11-03T17:44:36.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.182+0000 7f173cae0700 1 --2- 172.21.15.27:0/1039771293 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f17240640b0 0x7f1724066570 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:36.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.182+0000 7f173cae0700 1 --2- 172.21.15.27:0/1039771293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17380f66e0 0x7f17381070f0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:36.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.182+0000 7f173cae0700 1 --2- 172.21.15.27:0/1039771293 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17380f6ff0 0x7f17381075e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:44:36.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.182+0000 7f173cae0700 1 -- 172.21.15.27:0/1039771293 >> 172.21.15.27:0/1039771293 conn(0x7f17380f1f00 msgr2=0x7f17380fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:44:36.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.182+0000 7f173cae0700 1 -- 172.21.15.27:0/1039771293 shutdown_connections 2021-11-03T17:44:36.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.182+0000 7f173cae0700 1 -- 172.21.15.27:0/1039771293 wait complete. 2021-11-03T17:44:36.193 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:44:36.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:35 smithi115 ceph-mon[65453]: pgmap v898: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:36.267 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:35 smithi027 ceph-mon[97244]: pgmap v898: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:36.283 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.283+0000 7f8086da7700 1 -- 172.21.15.27:0/1411838434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80800f6a10 msgr2=0x7f80800f6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:44:36.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.283+0000 7f8086da7700 1 --2- 172.21.15.27:0/1411838434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80800f6a10 0x7f80800f6e30 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7f807c004660 tx=0x7f807c009b30).stop 2021-11-03T17:44:36.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.283+0000 7f8086da7700 1 -- 172.21.15.27:0/1411838434 shutdown_connections 2021-11-03T17:44:36.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.283+0000 7f8086da7700 1 --2- 172.21.15.27:0/1411838434 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80800f5710 0x7f80800f5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:36.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.283+0000 7f8086da7700 1 --2- 172.21.15.27:0/1411838434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80800f6a10 0x7f80800f6e30 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:36.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.283+0000 7f8086da7700 1 -- 172.21.15.27:0/1411838434 >> 172.21.15.27:0/1411838434 conn(0x7f80800f0320 msgr2=0x7f80800f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:44:36.286 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.283+0000 7f8086da7700 1 -- 172.21.15.27:0/1411838434 shutdown_connections 2021-11-03T17:44:36.286 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.283+0000 7f8086da7700 1 -- 172.21.15.27:0/1411838434 wait complete. 2021-11-03T17:44:36.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.284+0000 7f8086da7700 1 Processor -- start 2021-11-03T17:44:36.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.284+0000 7f8086da7700 1 -- start start 2021-11-03T17:44:36.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.284+0000 7f8086da7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80800f5710 0x7f80800f95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:44:36.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.284+0000 7f8086da7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80800f6a10 0x7f80800f7a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:44:36.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.284+0000 7f8086da7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80800f7fc0 con 0x7f80800f6a10 2021-11-03T17:44:36.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.284+0000 7f8086da7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80800f8100 con 0x7f80800f5710 2021-11-03T17:44:36.290 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.284+0000 7f80855a4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80800f6a10 0x7f80800f7a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:44:36.290 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.284+0000 7f80855a4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80800f6a10 0x7f80800f7a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59134/0 (socket says 172.21.15.27:59134) 2021-11-03T17:44:36.291 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.284+0000 7f80855a4700 1 -- 172.21.15.27:0/2264083061 learned_addr learned my addr 172.21.15.27:0/2264083061 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:44:36.291 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.284+0000 7f8085da5700 1 --2- 172.21.15.27:0/2264083061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80800f5710 0x7f80800f95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:44:36.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.284+0000 7f80855a4700 1 -- 172.21.15.27:0/2264083061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80800f5710 msgr2=0x7f80800f95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:44:36.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.284+0000 7f80855a4700 1 --2- 172.21.15.27:0/2264083061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80800f5710 0x7f80800f95e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:36.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.285+0000 7f80855a4700 1 -- 172.21.15.27:0/2264083061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f807c005040 con 0x7f80800f6a10 2021-11-03T17:44:36.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.285+0000 7f80855a4700 1 --2- 172.21.15.27:0/2264083061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80800f6a10 0x7f80800f7a60 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7f8070002700 tx=0x7f8070009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:44:36.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.285+0000 7f80767fc700 1 -- 172.21.15.27:0/2264083061 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f80700079c0 con 0x7f80800f6a10 2021-11-03T17:44:36.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.285+0000 7f8086da7700 1 -- 172.21.15.27:0/2264083061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80800f83e0 con 0x7f80800f6a10 2021-11-03T17:44:36.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.285+0000 7f80767fc700 1 -- 172.21.15.27:0/2264083061 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f807000f040 con 0x7f80800f6a10 2021-11-03T17:44:36.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.285+0000 7f80767fc700 1 -- 172.21.15.27:0/2264083061 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8070014860 con 0x7f80800f6a10 2021-11-03T17:44:36.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.285+0000 7f8086da7700 1 -- 172.21.15.27:0/2264083061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f808010ed70 con 0x7f80800f6a10 2021-11-03T17:44:36.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.286+0000 7f8086da7700 1 -- 172.21.15.27:0/2264083061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8080045ad0 con 0x7f80800f6a10 2021-11-03T17:44:36.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.286+0000 7f80767fc700 1 -- 172.21.15.27:0/2264083061 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f807001b060 con 0x7f80800f6a10 2021-11-03T17:44:36.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.287+0000 7f80767fc700 1 --2- 172.21.15.27:0/2264083061 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f806c064100 0x7f806c0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:44:36.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.287+0000 7f80767fc700 1 -- 172.21.15.27:0/2264083061 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f80700806d0 con 0x7f80800f6a10 2021-11-03T17:44:36.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.287+0000 7f8085da5700 1 --2- 172.21.15.27:0/2264083061 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f806c064100 0x7f806c0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:44:36.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.287+0000 7f8085da5700 1 --2- 172.21.15.27:0/2264083061 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f806c064100 0x7f806c0665c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f807c013690 tx=0x7f807c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:44:36.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.291+0000 7f80767fc700 1 -- 172.21.15.27:0/2264083061 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f80700529d0 con 0x7f80800f6a10 2021-11-03T17:44:36.460 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.464+0000 7f8086da7700 1 -- 172.21.15.27:0/2264083061 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f80800fab60 con 0x7f806c064100 2021-11-03T17:44:36.475 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.479+0000 7f80767fc700 1 -- 172.21.15.27:0/2264083061 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f80800fab60 con 0x7f806c064100 2021-11-03T17:44:36.476 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:44:36.477 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (36m) 5m ago 37m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:44:36.477 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (27m) 5m ago 37m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:44:36.477 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (27m) 4m ago 36m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:44:36.479 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (35m) 5m ago 37m 41.3M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:44:36.480 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (29m) 5m ago 32m 21.1M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:44:36.480 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (32m) 5m ago 32m 3843M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:44:36.481 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (28m) 4m ago 32m 20.4M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:44:36.481 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (30m) 4m ago 32m 22.1M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:44:36.482 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (29m) 5m ago 38m 517M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:44:36.482 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (29m) 4m ago 36m 419M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:44:36.482 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (28m) 5m ago 38m 159M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:44:36.483 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (28m) 4m ago 36m 82.5M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:44:36.483 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (37m) 5m ago 37m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:44:36.484 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (36m) 4m ago 36m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:44:36.484 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (27m) 5m ago 35m 311M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:44:36.485 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (26m) 5m ago 35m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:44:36.485 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (26m) 5m ago 35m 248M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:44:36.485 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (26m) 4m ago 34m 354M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:44:36.486 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (25m) 4m ago 34m 285M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:44:36.486 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (25m) 4m ago 34m 287M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:44:36.487 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (35m) 5m ago 36m 50.5M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:44:36.488 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.481+0000 7f8086da7700 1 -- 172.21.15.27:0/2264083061 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f806c064100 msgr2=0x7f806c0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:44:36.488 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.481+0000 7f8086da7700 1 --2- 172.21.15.27:0/2264083061 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f806c064100 0x7f806c0665c0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7f807c013690 tx=0x7f807c009b30).stop 2021-11-03T17:44:36.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.481+0000 7f8086da7700 1 -- 172.21.15.27:0/2264083061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80800f6a10 msgr2=0x7f80800f7a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:44:36.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.481+0000 7f8086da7700 1 --2- 172.21.15.27:0/2264083061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80800f6a10 0x7f80800f7a60 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7f8070002700 tx=0x7f8070009d80).stop 2021-11-03T17:44:36.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.481+0000 7f8086da7700 1 -- 172.21.15.27:0/2264083061 shutdown_connections 2021-11-03T17:44:36.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.481+0000 7f8086da7700 1 --2- 172.21.15.27:0/2264083061 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f806c064100 0x7f806c0665c0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:36.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.481+0000 7f8086da7700 1 --2- 172.21.15.27:0/2264083061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f80800f5710 0x7f80800f95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:36.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.481+0000 7f8086da7700 1 --2- 172.21.15.27:0/2264083061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f80800f6a10 0x7f80800f7a60 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:36.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.481+0000 7f8086da7700 1 -- 172.21.15.27:0/2264083061 >> 172.21.15.27:0/2264083061 conn(0x7f80800f0320 msgr2=0x7f80800f4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:44:36.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.481+0000 7f8086da7700 1 -- 172.21.15.27:0/2264083061 shutdown_connections 2021-11-03T17:44:36.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.481+0000 7f8086da7700 1 -- 172.21.15.27:0/2264083061 wait complete. 2021-11-03T17:44:36.582 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.581+0000 7f23c84e9700 1 -- 172.21.15.27:0/146262120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23c00f55e0 msgr2=0x7f23c00f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:44:36.582 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.581+0000 7f23c84e9700 1 --2- 172.21.15.27:0/146262120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23c00f55e0 0x7f23c00f5a00 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f23b8004660 tx=0x7f23b8009b30).stop 2021-11-03T17:44:36.583 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.581+0000 7f23c84e9700 1 -- 172.21.15.27:0/146262120 shutdown_connections 2021-11-03T17:44:36.584 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.581+0000 7f23c84e9700 1 --2- 172.21.15.27:0/146262120 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23c00f5ef0 0x7f23c00f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:44:36.584 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.581+0000 7f23c84e9700 1 --2- 172.21.15.27:0/146262120 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23c00f55e0 0x7f23c00f5a00 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:36.585 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.581+0000 7f23c84e9700 1 -- 172.21.15.27:0/146262120 >> 172.21.15.27:0/146262120 conn(0x7f23c00f0de0 msgr2=0x7f23c00f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:44:36.585 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.581+0000 7f23c84e9700 1 -- 172.21.15.27:0/146262120 shutdown_connections 2021-11-03T17:44:36.586 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.581+0000 7f23c84e9700 1 -- 172.21.15.27:0/146262120 wait complete. 2021-11-03T17:44:36.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.582+0000 7f23c84e9700 1 Processor -- start 2021-11-03T17:44:36.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.582+0000 7f23c84e9700 1 -- start start 2021-11-03T17:44:36.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.582+0000 7f23c84e9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23c00f55e0 0x7f23c0106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:44:36.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.582+0000 7f23c84e9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23c00f5ef0 0x7f23c0107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:44:36.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.582+0000 7f23c84e9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c0107aa0 con 0x7f23c00f5ef0 2021-11-03T17:44:36.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.582+0000 7f23c84e9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23c0108350 con 0x7f23c00f55e0 2021-11-03T17:44:36.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.582+0000 7f23c5a84700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23c00f5ef0 0x7f23c0107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:44:36.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.582+0000 7f23c5a84700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23c00f5ef0 0x7f23c0107460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59144/0 (socket says 172.21.15.27:59144) 2021-11-03T17:44:36.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.582+0000 7f23c5a84700 1 -- 172.21.15.27:0/3332729900 learned_addr learned my addr 172.21.15.27:0/3332729900 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:44:36.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.582+0000 7f23c6285700 1 --2- 172.21.15.27:0/3332729900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23c00f55e0 0x7f23c0106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:44:36.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.582+0000 7f23c5a84700 1 -- 172.21.15.27:0/3332729900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23c00f55e0 msgr2=0x7f23c0106f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:44:36.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.582+0000 7f23c5a84700 1 --2- 172.21.15.27:0/3332729900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23c00f55e0 0x7f23c0106f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:36.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.582+0000 7f23c5a84700 1 -- 172.21.15.27:0/3332729900 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b8005040 con 0x7f23c00f5ef0 2021-11-03T17:44:36.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.582+0000 7f23c6285700 1 --2- 172.21.15.27:0/3332729900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23c00f55e0 0x7f23c0106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:44:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.582+0000 7f23c5a84700 1 --2- 172.21.15.27:0/3332729900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23c00f5ef0 0x7f23c0107460 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f23b400b410 tx=0x7f23b4006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:44:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.583+0000 7f23beffd700 1 -- 172.21.15.27:0/3332729900 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23b40125f0 con 0x7f23c00f5ef0 2021-11-03T17:44:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.583+0000 7f23beffd700 1 -- 172.21.15.27:0/3332729900 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f23b4012c00 con 0x7f23c00f5ef0 2021-11-03T17:44:36.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.583+0000 7f23c84e9700 1 -- 172.21.15.27:0/3332729900 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23c0108630 con 0x7f23c00f5ef0 2021-11-03T17:44:36.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.583+0000 7f23beffd700 1 -- 172.21.15.27:0/3332729900 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23b4002ae0 con 0x7f23c00f5ef0 2021-11-03T17:44:36.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.583+0000 7f23c84e9700 1 -- 172.21.15.27:0/3332729900 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23c0108bf0 con 0x7f23c00f5ef0 2021-11-03T17:44:36.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.584+0000 7f23c84e9700 1 -- 172.21.15.27:0/3332729900 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23c00f4a70 con 0x7f23c00f5ef0 2021-11-03T17:44:36.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.584+0000 7f23beffd700 1 -- 172.21.15.27:0/3332729900 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f23b400f030 con 0x7f23c00f5ef0 2021-11-03T17:44:36.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.585+0000 7f23beffd700 1 --2- 172.21.15.27:0/3332729900 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f23a80640b0 0x7f23a8066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:44:36.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.585+0000 7f23beffd700 1 -- 172.21.15.27:0/3332729900 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f23b4080c50 con 0x7f23c00f5ef0 2021-11-03T17:44:36.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.585+0000 7f23c6285700 1 --2- 172.21.15.27:0/3332729900 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f23a80640b0 0x7f23a8066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:44:36.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.585+0000 7f23c6285700 1 --2- 172.21.15.27:0/3332729900 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f23a80640b0 0x7f23a8066570 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7f23b8009da0 tx=0x7f23b8009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:44:36.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.589+0000 7f23beffd700 1 -- 172.21.15.27:0/3332729900 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f23b4052f50 con 0x7f23c00f5ef0 2021-11-03T17:44:36.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.820+0000 7f23c84e9700 1 -- 172.21.15.27:0/3332729900 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f23c00fefc0 con 0x7f23c00f5ef0 2021-11-03T17:44:36.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.821+0000 7f23beffd700 1 -- 172.21.15.27:0/3332729900 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f23b4052b40 con 0x7f23c00f5ef0 2021-11-03T17:44:36.817 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:44:36.817 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:44:36.818 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:44:36.818 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:44:36.818 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:44:36.818 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:44:36.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:44:36.819 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:44:36.819 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:44:36.819 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:44:36.820 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:44:36.820 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:44:36.820 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:44:36.820 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:44:36.821 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:44:36.821 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:44:36.821 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:44:36.821 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:44:36.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.823+0000 7f23c84e9700 1 -- 172.21.15.27:0/3332729900 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f23a80640b0 msgr2=0x7f23a8066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:44:36.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.823+0000 7f23c84e9700 1 --2- 172.21.15.27:0/3332729900 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f23a80640b0 0x7f23a8066570 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7f23b8009da0 tx=0x7f23b8009b30).stop 2021-11-03T17:44:36.823 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.823+0000 7f23c84e9700 1 -- 172.21.15.27:0/3332729900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23c00f5ef0 msgr2=0x7f23c0107460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:44:36.823 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.823+0000 7f23c84e9700 1 --2- 172.21.15.27:0/3332729900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23c00f5ef0 0x7f23c0107460 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f23b400b410 tx=0x7f23b4006ef0).stop 2021-11-03T17:44:36.823 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.823+0000 7f23c84e9700 1 -- 172.21.15.27:0/3332729900 shutdown_connections 2021-11-03T17:44:36.824 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.823+0000 7f23c84e9700 1 --2- 172.21.15.27:0/3332729900 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f23a80640b0 0x7f23a8066570 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:36.824 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.823+0000 7f23c84e9700 1 --2- 172.21.15.27:0/3332729900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f23c00f55e0 0x7f23c0106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:36.824 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.823+0000 7f23c84e9700 1 --2- 172.21.15.27:0/3332729900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f23c00f5ef0 0x7f23c0107460 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:36.825 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.823+0000 7f23c84e9700 1 -- 172.21.15.27:0/3332729900 >> 172.21.15.27:0/3332729900 conn(0x7f23c00f0de0 msgr2=0x7f23c00fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:44:36.825 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.823+0000 7f23c84e9700 1 -- 172.21.15.27:0/3332729900 shutdown_connections 2021-11-03T17:44:36.825 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.823+0000 7f23c84e9700 1 -- 172.21.15.27:0/3332729900 wait complete. 2021-11-03T17:44:36.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.924+0000 7fa303171700 1 -- 172.21.15.27:0/4111356605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2fc060310 msgr2=0x7fa2fc0f9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:44:36.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.924+0000 7fa303171700 1 --2- 172.21.15.27:0/4111356605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2fc060310 0x7fa2fc0f9120 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7fa2f8004660 tx=0x7fa2f8009b30).stop 2021-11-03T17:44:36.922 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.924+0000 7fa303171700 1 -- 172.21.15.27:0/4111356605 shutdown_connections 2021-11-03T17:44:36.922 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.924+0000 7fa303171700 1 --2- 172.21.15.27:0/4111356605 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2fc0f96a0 0x7fa2fc0fbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:36.922 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.924+0000 7fa303171700 1 --2- 172.21.15.27:0/4111356605 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2fc060310 0x7fa2fc0f9120 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:36.923 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.924+0000 7fa303171700 1 -- 172.21.15.27:0/4111356605 >> 172.21.15.27:0/4111356605 conn(0x7fa2fc0f0ba0 msgr2=0x7fa2fc0f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:44:36.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.925+0000 7fa303171700 1 -- 172.21.15.27:0/4111356605 shutdown_connections 2021-11-03T17:44:36.925 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.925+0000 7fa303171700 1 -- 172.21.15.27:0/4111356605 wait complete. 2021-11-03T17:44:36.925 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.925+0000 7fa303171700 1 Processor -- start 2021-11-03T17:44:36.925 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.925+0000 7fa303171700 1 -- start start 2021-11-03T17:44:36.925 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.925+0000 7fa303171700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2fc060310 0x7fa2fc0f6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:44:36.926 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.925+0000 7fa303171700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2fc0f96a0 0x7fa2fc0f5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:44:36.926 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.925+0000 7fa303171700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2fc0f5580 con 0x7fa2fc0f96a0 2021-11-03T17:44:36.926 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.925+0000 7fa303171700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2fc0f56c0 con 0x7fa2fc060310 2021-11-03T17:44:36.926 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.925+0000 7fa30196e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2fc0f96a0 0x7fa2fc0f5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:44:36.927 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.925+0000 7fa30196e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2fc0f96a0 0x7fa2fc0f5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59154/0 (socket says 172.21.15.27:59154) 2021-11-03T17:44:36.927 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.925+0000 7fa30196e700 1 -- 172.21.15.27:0/3158124834 learned_addr learned my addr 172.21.15.27:0/3158124834 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:44:36.927 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.926+0000 7fa30216f700 1 --2- 172.21.15.27:0/3158124834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2fc060310 0x7fa2fc0f6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:44:36.927 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.926+0000 7fa30196e700 1 -- 172.21.15.27:0/3158124834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2fc060310 msgr2=0x7fa2fc0f6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:44:36.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.926+0000 7fa30196e700 1 --2- 172.21.15.27:0/3158124834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2fc060310 0x7fa2fc0f6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:36.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.926+0000 7fa30196e700 1 -- 172.21.15.27:0/3158124834 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f8005040 con 0x7fa2fc0f96a0 2021-11-03T17:44:36.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.926+0000 7fa30216f700 1 --2- 172.21.15.27:0/3158124834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2fc060310 0x7fa2fc0f6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:44:36.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.926+0000 7fa30196e700 1 --2- 172.21.15.27:0/3158124834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2fc0f96a0 0x7fa2fc0f5020 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7fa2ec00c790 tx=0x7fa2ec007d40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:44:36.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.926+0000 7fa2f2ffd700 1 -- 172.21.15.27:0/3158124834 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2ec00e680 con 0x7fa2fc0f96a0 2021-11-03T17:44:36.929 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.926+0000 7fa2f2ffd700 1 -- 172.21.15.27:0/3158124834 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fa2ec00ec90 con 0x7fa2fc0f96a0 2021-11-03T17:44:36.929 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.926+0000 7fa303171700 1 -- 172.21.15.27:0/3158124834 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2fc0f59a0 con 0x7fa2fc0f96a0 2021-11-03T17:44:36.929 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.926+0000 7fa303171700 1 -- 172.21.15.27:0/3158124834 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2fc10ed80 con 0x7fa2fc0f96a0 2021-11-03T17:44:36.929 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.926+0000 7fa2f2ffd700 1 -- 172.21.15.27:0/3158124834 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2ec013cd0 con 0x7fa2fc0f96a0 2021-11-03T17:44:36.932 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.927+0000 7fa303171700 1 -- 172.21.15.27:0/3158124834 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2e4004fa0 con 0x7fa2fc0f96a0 2021-11-03T17:44:36.932 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.927+0000 7fa2f2ffd700 1 -- 172.21.15.27:0/3158124834 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fa2ec01a060 con 0x7fa2fc0f96a0 2021-11-03T17:44:36.932 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.928+0000 7fa2f2ffd700 1 --2- 172.21.15.27:0/3158124834 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa2e8064100 0x7fa2e80665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:44:36.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.928+0000 7fa2f2ffd700 1 -- 172.21.15.27:0/3158124834 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fa2ec080a20 con 0x7fa2fc0f96a0 2021-11-03T17:44:36.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.928+0000 7fa30216f700 1 --2- 172.21.15.27:0/3158124834 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa2e8064100 0x7fa2e80665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:44:36.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.928+0000 7fa30216f700 1 --2- 172.21.15.27:0/3158124834 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa2e8064100 0x7fa2e80665c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7fa2f8004e30 tx=0x7fa2f8009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:44:36.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:36.932+0000 7fa2f2ffd700 1 -- 172.21.15.27:0/3158124834 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fa2ec052bd0 con 0x7fa2fc0f96a0 2021-11-03T17:44:37.129 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:37.134+0000 7fa303171700 1 -- 172.21.15.27:0/3158124834 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa2e4005e80 con 0x7fa2fc0f96a0 2021-11-03T17:44:37.129 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:37.134+0000 7fa2f2ffd700 1 -- 172.21.15.27:0/3158124834 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7fa2ec018070 con 0x7fa2fc0f96a0 2021-11-03T17:44:37.130 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:44:37.130 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:44:37.130 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:44:37.130 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:44:37.131 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:44:37.131 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:44:37.131 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:44:37.131 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:44:37.132 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:44:37.132 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:44:37.132 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:44:37.132 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:44:37.132 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:44:37.133 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:44:37.133 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:44:37.133 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:44:37.133 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:44:37.134 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:44:37.134 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:44:37.134 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:44:37.134 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:44:37.134 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:44:37.135 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:44:37.135 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:44:37.135 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:44:37.135 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:44:37.136 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:44:37.136 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:44:37.136 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:44:37.136 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:44:37.136 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:44:37.137 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:44:37.137 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:44:37.137 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:44:37.137 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:44:37.138 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:44:37.138 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:44:37.138 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:44:37.138 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:44:37.139 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:37.136+0000 7fa303171700 1 -- 172.21.15.27:0/3158124834 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa2e8064100 msgr2=0x7fa2e80665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:44:37.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:37.136+0000 7fa303171700 1 --2- 172.21.15.27:0/3158124834 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa2e8064100 0x7fa2e80665c0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 rx=0x7fa2f8004e30 tx=0x7fa2f8009b30).stop 2021-11-03T17:44:37.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:37.136+0000 7fa303171700 1 -- 172.21.15.27:0/3158124834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2fc0f96a0 msgr2=0x7fa2fc0f5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:44:37.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:37.136+0000 7fa303171700 1 --2- 172.21.15.27:0/3158124834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2fc0f96a0 0x7fa2fc0f5020 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7fa2ec00c790 tx=0x7fa2ec007d40).stop 2021-11-03T17:44:37.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:37.136+0000 7fa303171700 1 -- 172.21.15.27:0/3158124834 shutdown_connections 2021-11-03T17:44:37.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:37.136+0000 7fa303171700 1 --2- 172.21.15.27:0/3158124834 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa2e8064100 0x7fa2e80665c0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:37.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:37.136+0000 7fa303171700 1 --2- 172.21.15.27:0/3158124834 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2fc060310 0x7fa2fc0f6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:37.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:37.136+0000 7fa303171700 1 --2- 172.21.15.27:0/3158124834 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2fc0f96a0 0x7fa2fc0f5020 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:44:37.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:37.136+0000 7fa303171700 1 -- 172.21.15.27:0/3158124834 >> 172.21.15.27:0/3158124834 conn(0x7fa2fc0f0ba0 msgr2=0x7fa2fc0f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:44:37.142 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:37.136+0000 7fa303171700 1 -- 172.21.15.27:0/3158124834 shutdown_connections 2021-11-03T17:44:37.142 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:44:37.136+0000 7fa303171700 1 -- 172.21.15.27:0/3158124834 wait complete. 2021-11-03T17:44:37.142 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:44:37.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:36 smithi115 ceph-mon[65453]: from='client.35032 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:44:37.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:36 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3332729900' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:37.267 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:36 smithi027 ceph-mon[97244]: from='client.35032 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:44:37.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:36 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3332729900' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:38.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:37 smithi115 ceph-mon[65453]: pgmap v899: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:38.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:37 smithi115 ceph-mon[65453]: from='client.35036 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:44:38.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:37 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3158124834' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:44:38.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:37 smithi027 ceph-mon[97244]: pgmap v899: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:38.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:37 smithi027 ceph-mon[97244]: from='client.35036 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:44:38.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:37 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3158124834' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:44:40.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:39 smithi115 ceph-mon[65453]: pgmap v900: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:40.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:39 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1722.400143 secs 2021-11-03T17:44:40.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:44:40.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:44:40.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:44:40.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:40.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:40.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:40.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:40.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:39 smithi027 ceph-mon[97244]: pgmap v900: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:40.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:39 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1722.400143 secs 2021-11-03T17:44:40.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:44:40.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:44:40.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:44:40.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:40.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:40.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:40.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:41.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:40 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:44:41.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:40 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:44:42.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:41 smithi115 ceph-mon[65453]: pgmap v901: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:42.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:41 smithi027 ceph-mon[97244]: pgmap v901: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:44.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:43 smithi115 ceph-mon[65453]: pgmap v902: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:44.267 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:43 smithi027 ceph-mon[97244]: pgmap v902: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:45.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:44 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1727.400256 secs 2021-11-03T17:44:45.267 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:44 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1727.400256 secs 2021-11-03T17:44:46.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:45 smithi115 ceph-mon[65453]: pgmap v903: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:46.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:45 smithi027 ceph-mon[97244]: pgmap v903: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:48.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:47 smithi115 ceph-mon[65453]: pgmap v904: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:48.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:47 smithi027 ceph-mon[97244]: pgmap v904: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:50.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:49 smithi115 ceph-mon[65453]: pgmap v905: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:50.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:49 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1732.400398 secs 2021-11-03T17:44:50.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:44:50.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:44:50.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:44:50.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:50.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:50.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:50.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:50.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:49 smithi027 ceph-mon[97244]: pgmap v905: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:50.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:49 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1732.400398 secs 2021-11-03T17:44:50.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:44:50.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:44:50.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:44:50.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:50.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:50.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:50.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:44:51.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:50 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:44:51.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:50 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:44:52.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:51 smithi115 ceph-mon[65453]: pgmap v906: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:52.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:51 smithi027 ceph-mon[97244]: pgmap v906: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:54.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:53 smithi115 ceph-mon[65453]: pgmap v907: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:54.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:53 smithi027 ceph-mon[97244]: pgmap v907: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:55.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:54 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1737.400491 secs 2021-11-03T17:44:55.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:54 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1737.400491 secs 2021-11-03T17:44:56.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:55 smithi115 ceph-mon[65453]: pgmap v908: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:56.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:44:56.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:44:56.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:55 smithi027 ceph-mon[97244]: pgmap v908: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:56.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:44:56.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:44:58.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:57 smithi115 ceph-mon[65453]: pgmap v909: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:44:58.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:57 smithi027 ceph-mon[97244]: pgmap v909: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:00.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:59 smithi115 ceph-mon[65453]: pgmap v910: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:00.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:59 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1742.400627 secs 2021-11-03T17:45:00.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:45:00.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:45:00.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:45:00.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:00.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:00.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:00.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:44:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:00.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:59 smithi027 ceph-mon[97244]: pgmap v910: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:00.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:59 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1742.400627 secs 2021-11-03T17:45:00.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:45:00.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:45:00.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:45:00.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:00.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:00.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:00.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:44:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:01.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:00 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:45:01.267 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:00 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:45:02.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:01 smithi115 ceph-mon[65453]: pgmap v911: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:02.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:01 smithi027 ceph-mon[97244]: pgmap v911: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:04.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:03 smithi115 ceph-mon[65453]: pgmap v912: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:04.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:03 smithi027 ceph-mon[97244]: pgmap v912: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:05.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:04 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1747.400717 secs 2021-11-03T17:45:05.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:04 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1747.400717 secs 2021-11-03T17:45:06.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:05 smithi115 ceph-mon[65453]: pgmap v913: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:06.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:05 smithi027 ceph-mon[97244]: pgmap v913: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:07.248 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.251+0000 7f8fecd29700 1 -- 172.21.15.27:0/4111547787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fe80f55c0 msgr2=0x7f8fe80f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:45:07.248 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.251+0000 7f8fecd29700 1 --2- 172.21.15.27:0/4111547787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fe80f55c0 0x7f8fe80f59e0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f8fe00099e0 tx=0x7f8fe0004d10).stop 2021-11-03T17:45:07.248 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.251+0000 7f8fecd29700 1 -- 172.21.15.27:0/4111547787 shutdown_connections 2021-11-03T17:45:07.249 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.251+0000 7f8fecd29700 1 --2- 172.21.15.27:0/4111547787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8fe80f6860 0x7f8fe80f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:45:07.249 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.251+0000 7f8fecd29700 1 --2- 172.21.15.27:0/4111547787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fe80f55c0 0x7f8fe80f59e0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:07.249 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.251+0000 7f8fecd29700 1 -- 172.21.15.27:0/4111547787 >> 172.21.15.27:0/4111547787 conn(0x7f8fe80f0d40 msgr2=0x7f8fe80f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:45:07.250 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.251+0000 7f8fecd29700 1 -- 172.21.15.27:0/4111547787 shutdown_connections 2021-11-03T17:45:07.250 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.251+0000 7f8fecd29700 1 -- 172.21.15.27:0/4111547787 wait complete. 2021-11-03T17:45:07.250 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.252+0000 7f8fecd29700 1 Processor -- start 2021-11-03T17:45:07.250 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.252+0000 7f8fecd29700 1 -- start start 2021-11-03T17:45:07.251 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.252+0000 7f8fecd29700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8fe80f55c0 0x7f8fe8109e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:45:07.251 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.252+0000 7f8fecd29700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fe80f6860 0x7f8fe810c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:45:07.251 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.252+0000 7f8fecd29700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe8060450 con 0x7f8fe80f6860 2021-11-03T17:45:07.251 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.252+0000 7f8fecd29700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fe8060590 con 0x7f8fe80f55c0 2021-11-03T17:45:07.252 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.252+0000 7f8fe5d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fe80f6860 0x7f8fe810c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:45:07.252 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.252+0000 7f8fe5d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fe80f6860 0x7f8fe810c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59172/0 (socket says 172.21.15.27:59172) 2021-11-03T17:45:07.252 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.252+0000 7f8fe5d9b700 1 -- 172.21.15.27:0/1584706227 learned_addr learned my addr 172.21.15.27:0/1584706227 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:45:07.252 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.252+0000 7f8fe659c700 1 --2- 172.21.15.27:0/1584706227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8fe80f55c0 0x7f8fe8109e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:45:07.253 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.252+0000 7f8fe5d9b700 1 -- 172.21.15.27:0/1584706227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8fe80f55c0 msgr2=0x7f8fe8109e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:45:07.253 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.252+0000 7f8fe5d9b700 1 --2- 172.21.15.27:0/1584706227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8fe80f55c0 0x7f8fe8109e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:07.253 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.252+0000 7f8fe5d9b700 1 -- 172.21.15.27:0/1584706227 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe0009640 con 0x7f8fe80f6860 2021-11-03T17:45:07.253 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.252+0000 7f8fe5d9b700 1 --2- 172.21.15.27:0/1584706227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fe80f6860 0x7f8fe810c350 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7f8fdc00b410 tx=0x7f8fdc006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:45:07.253 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.253+0000 7f8fd6ffd700 1 -- 172.21.15.27:0/1584706227 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fdc0125f0 con 0x7f8fe80f6860 2021-11-03T17:45:07.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.253+0000 7f8fecd29700 1 -- 172.21.15.27:0/1584706227 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fe810c960 con 0x7f8fe80f6860 2021-11-03T17:45:07.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.253+0000 7f8fd6ffd700 1 -- 172.21.15.27:0/1584706227 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f8fdc012c00 con 0x7f8fe80f6860 2021-11-03T17:45:07.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.253+0000 7f8fd6ffd700 1 -- 172.21.15.27:0/1584706227 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fdc011d20 con 0x7f8fe80f6860 2021-11-03T17:45:07.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.253+0000 7f8fecd29700 1 -- 172.21.15.27:0/1584706227 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fe810cea0 con 0x7f8fe80f6860 2021-11-03T17:45:07.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.254+0000 7f8fecd29700 1 -- 172.21.15.27:0/1584706227 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fe8045ad0 con 0x7f8fe80f6860 2021-11-03T17:45:07.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.254+0000 7f8fd6ffd700 1 -- 172.21.15.27:0/1584706227 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f8fdc00f030 con 0x7f8fe80f6860 2021-11-03T17:45:07.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.255+0000 7f8fd6ffd700 1 --2- 172.21.15.27:0/1584706227 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8fd00640b0 0x7f8fd0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:45:07.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.255+0000 7f8fd6ffd700 1 -- 172.21.15.27:0/1584706227 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f8fdc080bc0 con 0x7f8fe80f6860 2021-11-03T17:45:07.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.255+0000 7f8fe659c700 1 --2- 172.21.15.27:0/1584706227 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8fd00640b0 0x7f8fd0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:45:07.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.255+0000 7f8fe659c700 1 --2- 172.21.15.27:0/1584706227 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8fd00640b0 0x7f8fd0066570 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f8fe00099b0 tx=0x7f8fe0004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:45:07.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.259+0000 7f8fd6ffd700 1 -- 172.21.15.27:0/1584706227 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f8fdc052ec0 con 0x7f8fe80f6860 2021-11-03T17:45:07.433 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.438+0000 7f8fecd29700 1 -- 172.21.15.27:0/1584706227 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8fe80fdf70 con 0x7f8fd00640b0 2021-11-03T17:45:07.434 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.439+0000 7f8fd6ffd700 1 -- 172.21.15.27:0/1584706227 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f8fe80fdf70 con 0x7f8fd00640b0 2021-11-03T17:45:07.436 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.441+0000 7f8fecd29700 1 -- 172.21.15.27:0/1584706227 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8fd00640b0 msgr2=0x7f8fd0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:45:07.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.441+0000 7f8fecd29700 1 --2- 172.21.15.27:0/1584706227 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8fd00640b0 0x7f8fd0066570 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7f8fe00099b0 tx=0x7f8fe0004d10).stop 2021-11-03T17:45:07.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.441+0000 7f8fecd29700 1 -- 172.21.15.27:0/1584706227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fe80f6860 msgr2=0x7f8fe810c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:45:07.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.441+0000 7f8fecd29700 1 --2- 172.21.15.27:0/1584706227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fe80f6860 0x7f8fe810c350 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7f8fdc00b410 tx=0x7f8fdc006ef0).stop 2021-11-03T17:45:07.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.441+0000 7f8fecd29700 1 -- 172.21.15.27:0/1584706227 shutdown_connections 2021-11-03T17:45:07.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.441+0000 7f8fecd29700 1 --2- 172.21.15.27:0/1584706227 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8fd00640b0 0x7f8fd0066570 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:07.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.441+0000 7f8fecd29700 1 --2- 172.21.15.27:0/1584706227 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8fe80f55c0 0x7f8fe8109e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:07.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.441+0000 7f8fecd29700 1 --2- 172.21.15.27:0/1584706227 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fe80f6860 0x7f8fe810c350 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:07.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.441+0000 7f8fecd29700 1 -- 172.21.15.27:0/1584706227 >> 172.21.15.27:0/1584706227 conn(0x7f8fe80f0d40 msgr2=0x7f8fe80fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:45:07.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.441+0000 7f8fecd29700 1 -- 172.21.15.27:0/1584706227 shutdown_connections 2021-11-03T17:45:07.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.441+0000 7f8fecd29700 1 -- 172.21.15.27:0/1584706227 wait complete. 2021-11-03T17:45:07.446 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:45:07.538 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.542+0000 7f6c7cd56700 1 -- 172.21.15.27:0/714023553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c780f55e0 msgr2=0x7f6c780f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:45:07.538 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.542+0000 7f6c7cd56700 1 --2- 172.21.15.27:0/714023553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c780f55e0 0x7f6c780f5a00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7f6c700099e0 tx=0x7f6c70004d10).stop 2021-11-03T17:45:07.539 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.542+0000 7f6c7cd56700 1 -- 172.21.15.27:0/714023553 shutdown_connections 2021-11-03T17:45:07.539 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.542+0000 7f6c7cd56700 1 --2- 172.21.15.27:0/714023553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c780f5ef0 0x7f6c780f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:07.539 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.542+0000 7f6c7cd56700 1 --2- 172.21.15.27:0/714023553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c780f55e0 0x7f6c780f5a00 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:07.540 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.542+0000 7f6c7cd56700 1 -- 172.21.15.27:0/714023553 >> 172.21.15.27:0/714023553 conn(0x7f6c780f0de0 msgr2=0x7f6c780f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:45:07.540 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.542+0000 7f6c7cd56700 1 -- 172.21.15.27:0/714023553 shutdown_connections 2021-11-03T17:45:07.541 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.542+0000 7f6c7cd56700 1 -- 172.21.15.27:0/714023553 wait complete. 2021-11-03T17:45:07.541 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.542+0000 7f6c7cd56700 1 Processor -- start 2021-11-03T17:45:07.542 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.542+0000 7f6c7cd56700 1 -- start start 2021-11-03T17:45:07.542 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.543+0000 7f6c7cd56700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c780f55e0 0x7f6c78109d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:45:07.543 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.543+0000 7f6c7cd56700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c780f5ef0 0x7f6c7810c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:45:07.543 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.543+0000 7f6c7cd56700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c78060540 con 0x7f6c780f55e0 2021-11-03T17:45:07.543 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.543+0000 7f6c7cd56700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c78060680 con 0x7f6c780f5ef0 2021-11-03T17:45:07.544 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.543+0000 7f6c7659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c780f55e0 0x7f6c78109d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:45:07.544 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.543+0000 7f6c7659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c780f55e0 0x7f6c78109d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59182/0 (socket says 172.21.15.27:59182) 2021-11-03T17:45:07.545 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.543+0000 7f6c7659c700 1 -- 172.21.15.27:0/661847468 learned_addr learned my addr 172.21.15.27:0/661847468 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:45:07.545 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.543+0000 7f6c75d9b700 1 --2- 172.21.15.27:0/661847468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c780f5ef0 0x7f6c7810c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:45:07.546 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.543+0000 7f6c7659c700 1 -- 172.21.15.27:0/661847468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c780f5ef0 msgr2=0x7f6c7810c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:45:07.546 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.543+0000 7f6c7659c700 1 --2- 172.21.15.27:0/661847468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c780f5ef0 0x7f6c7810c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:07.547 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.543+0000 7f6c7659c700 1 -- 172.21.15.27:0/661847468 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c70009640 con 0x7f6c780f55e0 2021-11-03T17:45:07.547 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.543+0000 7f6c7659c700 1 --2- 172.21.15.27:0/661847468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c780f55e0 0x7f6c78109d20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7f6c7000b3b0 tx=0x7f6c70004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:45:07.547 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.543+0000 7f6c66ffd700 1 -- 172.21.15.27:0/661847468 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c7000e030 con 0x7f6c780f55e0 2021-11-03T17:45:07.548 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.543+0000 7f6c7cd56700 1 -- 172.21.15.27:0/661847468 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c7810c7d0 con 0x7f6c780f55e0 2021-11-03T17:45:07.548 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.543+0000 7f6c66ffd700 1 -- 172.21.15.27:0/661847468 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f6c700049b0 con 0x7f6c780f55e0 2021-11-03T17:45:07.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.543+0000 7f6c66ffd700 1 -- 172.21.15.27:0/661847468 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c70018070 con 0x7f6c780f55e0 2021-11-03T17:45:07.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.543+0000 7f6c7cd56700 1 -- 172.21.15.27:0/661847468 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c7810cc80 con 0x7f6c780f55e0 2021-11-03T17:45:07.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.544+0000 7f6c7cd56700 1 -- 172.21.15.27:0/661847468 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c78045ad0 con 0x7f6c780f55e0 2021-11-03T17:45:07.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.545+0000 7f6c66ffd700 1 -- 172.21.15.27:0/661847468 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f6c70007c90 con 0x7f6c780f55e0 2021-11-03T17:45:07.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.545+0000 7f6c66ffd700 1 --2- 172.21.15.27:0/661847468 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6c600640b0 0x7f6c60066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:45:07.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.545+0000 7f6c66ffd700 1 -- 172.21.15.27:0/661847468 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f6c700824f0 con 0x7f6c780f55e0 2021-11-03T17:45:07.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.545+0000 7f6c75d9b700 1 --2- 172.21.15.27:0/661847468 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6c600640b0 0x7f6c60066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:45:07.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.546+0000 7f6c75d9b700 1 --2- 172.21.15.27:0/661847468 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6c600640b0 0x7f6c60066570 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f6c6c004c40 tx=0x7f6c6c00b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:45:07.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.549+0000 7f6c66ffd700 1 -- 172.21.15.27:0/661847468 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f6c700547f0 con 0x7f6c780f55e0 2021-11-03T17:45:07.715 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.720+0000 7f6c7cd56700 1 -- 172.21.15.27:0/661847468 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c780fdf80 con 0x7f6c600640b0 2021-11-03T17:45:07.731 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.735+0000 7f6c66ffd700 1 -- 172.21.15.27:0/661847468 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f6c6c00f040 con 0x7f6c600640b0 2021-11-03T17:45:07.731 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:45:07.731 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (36m) 6m ago 37m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:45:07.732 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (28m) 6m ago 37m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:45:07.732 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (28m) 5m ago 37m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:45:07.732 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (36m) 6m ago 37m 41.3M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:45:07.732 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (30m) 6m ago 33m 21.1M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:45:07.732 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (33m) 6m ago 33m 3843M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:45:07.733 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (28m) 5m ago 33m 20.4M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:45:07.733 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (30m) 5m ago 33m 22.1M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:45:07.733 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (30m) 6m ago 39m 517M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:45:07.733 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (29m) 5m ago 37m 419M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:45:07.734 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (29m) 6m ago 39m 159M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:45:07.734 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (28m) 5m ago 37m 82.5M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:45:07.734 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (37m) 6m ago 37m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:45:07.734 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (36m) 5m ago 36m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:45:07.734 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (28m) 6m ago 36m 311M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:45:07.735 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (27m) 6m ago 36m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:45:07.735 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (27m) 6m ago 35m 248M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:45:07.735 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (26m) 5m ago 35m 354M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:45:07.735 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (26m) 5m ago 34m 285M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:45:07.736 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (25m) 5m ago 34m 287M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:45:07.736 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (36m) 6m ago 37m 50.5M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:45:07.738 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.737+0000 7f6c7cd56700 1 -- 172.21.15.27:0/661847468 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6c600640b0 msgr2=0x7f6c60066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:45:07.738 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.737+0000 7f6c7cd56700 1 --2- 172.21.15.27:0/661847468 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6c600640b0 0x7f6c60066570 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7f6c6c004c40 tx=0x7f6c6c00b040).stop 2021-11-03T17:45:07.738 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.737+0000 7f6c7cd56700 1 -- 172.21.15.27:0/661847468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c780f55e0 msgr2=0x7f6c78109d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:45:07.739 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.737+0000 7f6c7cd56700 1 --2- 172.21.15.27:0/661847468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c780f55e0 0x7f6c78109d20 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7f6c7000b3b0 tx=0x7f6c70004d10).stop 2021-11-03T17:45:07.739 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.737+0000 7f6c7cd56700 1 -- 172.21.15.27:0/661847468 shutdown_connections 2021-11-03T17:45:07.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.737+0000 7f6c7cd56700 1 --2- 172.21.15.27:0/661847468 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6c600640b0 0x7f6c60066570 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:07.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.737+0000 7f6c7cd56700 1 --2- 172.21.15.27:0/661847468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c780f55e0 0x7f6c78109d20 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:07.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.737+0000 7f6c7cd56700 1 --2- 172.21.15.27:0/661847468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c780f5ef0 0x7f6c7810c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:07.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.737+0000 7f6c7cd56700 1 -- 172.21.15.27:0/661847468 >> 172.21.15.27:0/661847468 conn(0x7f6c780f0de0 msgr2=0x7f6c780fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:45:07.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.737+0000 7f6c7cd56700 1 -- 172.21.15.27:0/661847468 shutdown_connections 2021-11-03T17:45:07.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.737+0000 7f6c7cd56700 1 -- 172.21.15.27:0/661847468 wait complete. 2021-11-03T17:45:07.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.837+0000 7fd5b88e0700 1 -- 172.21.15.27:0/3573306451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5b4060310 msgr2=0x7fd5b40f9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:45:07.833 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.837+0000 7fd5b88e0700 1 --2- 172.21.15.27:0/3573306451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5b4060310 0x7fd5b40f9120 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7fd5a4004660 tx=0x7fd5a4009b30).stop 2021-11-03T17:45:07.833 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.837+0000 7fd5b88e0700 1 -- 172.21.15.27:0/3573306451 shutdown_connections 2021-11-03T17:45:07.833 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.837+0000 7fd5b88e0700 1 --2- 172.21.15.27:0/3573306451 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5b40f96a0 0x7fd5b40fbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:07.833 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.837+0000 7fd5b88e0700 1 --2- 172.21.15.27:0/3573306451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5b4060310 0x7fd5b40f9120 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:07.834 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.837+0000 7fd5b88e0700 1 -- 172.21.15.27:0/3573306451 >> 172.21.15.27:0/3573306451 conn(0x7fd5b40f0ba0 msgr2=0x7fd5b40f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:45:07.834 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.837+0000 7fd5b88e0700 1 -- 172.21.15.27:0/3573306451 shutdown_connections 2021-11-03T17:45:07.834 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.837+0000 7fd5b88e0700 1 -- 172.21.15.27:0/3573306451 wait complete. 2021-11-03T17:45:07.837 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.838+0000 7fd5b88e0700 1 Processor -- start 2021-11-03T17:45:07.837 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.838+0000 7fd5b88e0700 1 -- start start 2021-11-03T17:45:07.838 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.838+0000 7fd5b88e0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5b4060310 0x7fd5b4106d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:45:07.838 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.838+0000 7fd5b88e0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5b40f96a0 0x7fd5b4107240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:45:07.838 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.838+0000 7fd5b88e0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b4107840 con 0x7fd5b40f96a0 2021-11-03T17:45:07.838 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.838+0000 7fd5b88e0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5b4108050 con 0x7fd5b4060310 2021-11-03T17:45:07.838 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.838+0000 7fd5b259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5b40f96a0 0x7fd5b4107240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:45:07.839 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.838+0000 7fd5b259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5b40f96a0 0x7fd5b4107240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59194/0 (socket says 172.21.15.27:59194) 2021-11-03T17:45:07.839 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.838+0000 7fd5b259c700 1 -- 172.21.15.27:0/4075466918 learned_addr learned my addr 172.21.15.27:0/4075466918 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:45:07.839 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.838+0000 7fd5b2d9d700 1 --2- 172.21.15.27:0/4075466918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5b4060310 0x7fd5b4106d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:45:07.839 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.838+0000 7fd5b259c700 1 -- 172.21.15.27:0/4075466918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5b4060310 msgr2=0x7fd5b4106d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:45:07.840 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.838+0000 7fd5b259c700 1 --2- 172.21.15.27:0/4075466918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5b4060310 0x7fd5b4106d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:07.840 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.839+0000 7fd5b259c700 1 -- 172.21.15.27:0/4075466918 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5a4005040 con 0x7fd5b40f96a0 2021-11-03T17:45:07.840 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.839+0000 7fd5b259c700 1 --2- 172.21.15.27:0/4075466918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5b40f96a0 0x7fd5b4107240 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7fd5a80026d0 tx=0x7fd5a8009e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:45:07.840 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.839+0000 7fd59b7fe700 1 -- 172.21.15.27:0/4075466918 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5a8009320 con 0x7fd5b40f96a0 2021-11-03T17:45:07.841 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.839+0000 7fd59b7fe700 1 -- 172.21.15.27:0/4075466918 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fd5a800f040 con 0x7fd5b40f96a0 2021-11-03T17:45:07.841 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.839+0000 7fd5b88e0700 1 -- 172.21.15.27:0/4075466918 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5b41082d0 con 0x7fd5b40f96a0 2021-11-03T17:45:07.841 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.839+0000 7fd59b7fe700 1 -- 172.21.15.27:0/4075466918 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd5a8014870 con 0x7fd5b40f96a0 2021-11-03T17:45:07.841 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.839+0000 7fd5b88e0700 1 -- 172.21.15.27:0/4075466918 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5b4108890 con 0x7fd5b40f96a0 2021-11-03T17:45:07.841 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.840+0000 7fd5b88e0700 1 -- 172.21.15.27:0/4075466918 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5b4059100 con 0x7fd5b40f96a0 2021-11-03T17:45:07.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.841+0000 7fd59b7fe700 1 -- 172.21.15.27:0/4075466918 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fd5a8009480 con 0x7fd5b40f96a0 2021-11-03T17:45:07.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.841+0000 7fd59b7fe700 1 --2- 172.21.15.27:0/4075466918 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd59c064100 0x7fd59c0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:45:07.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.841+0000 7fd59b7fe700 1 -- 172.21.15.27:0/4075466918 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fd5a8080b60 con 0x7fd5b40f96a0 2021-11-03T17:45:07.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.841+0000 7fd5b2d9d700 1 --2- 172.21.15.27:0/4075466918 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd59c064100 0x7fd59c0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:45:07.843 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.841+0000 7fd5b2d9d700 1 --2- 172.21.15.27:0/4075466918 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd59c064100 0x7fd59c0665c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7fd5a4009da0 tx=0x7fd5a4009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:45:07.843 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:07.845+0000 7fd59b7fe700 1 -- 172.21.15.27:0/4075466918 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fd5a8052e60 con 0x7fd5b40f96a0 2021-11-03T17:45:08.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.076+0000 7fd5b88e0700 1 -- 172.21.15.27:0/4075466918 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd5b40fee20 con 0x7fd5b40f96a0 2021-11-03T17:45:08.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.077+0000 7fd59b7fe700 1 -- 172.21.15.27:0/4075466918 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fd5a8052a50 con 0x7fd5b40f96a0 2021-11-03T17:45:08.072 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:45:08.073 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:45:08.073 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:45:08.073 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:45:08.073 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:45:08.074 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:45:08.074 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:45:08.074 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:45:08.075 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:45:08.075 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:45:08.075 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:45:08.076 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:45:08.076 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:45:08.076 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:45:08.076 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:45:08.076 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:45:08.077 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:45:08.077 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:45:08.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.078+0000 7fd5b88e0700 1 -- 172.21.15.27:0/4075466918 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd59c064100 msgr2=0x7fd59c0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:45:08.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.078+0000 7fd5b88e0700 1 --2- 172.21.15.27:0/4075466918 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd59c064100 0x7fd59c0665c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7fd5a4009da0 tx=0x7fd5a4009b30).stop 2021-11-03T17:45:08.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.078+0000 7fd5b88e0700 1 -- 172.21.15.27:0/4075466918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5b40f96a0 msgr2=0x7fd5b4107240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:45:08.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.078+0000 7fd5b88e0700 1 --2- 172.21.15.27:0/4075466918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5b40f96a0 0x7fd5b4107240 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7fd5a80026d0 tx=0x7fd5a8009e00).stop 2021-11-03T17:45:08.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.078+0000 7fd5b88e0700 1 -- 172.21.15.27:0/4075466918 shutdown_connections 2021-11-03T17:45:08.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.078+0000 7fd5b88e0700 1 --2- 172.21.15.27:0/4075466918 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd59c064100 0x7fd59c0665c0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:08.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.078+0000 7fd5b88e0700 1 --2- 172.21.15.27:0/4075466918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd5b4060310 0x7fd5b4106d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:08.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.078+0000 7fd5b88e0700 1 --2- 172.21.15.27:0/4075466918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd5b40f96a0 0x7fd5b4107240 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:08.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.078+0000 7fd5b88e0700 1 -- 172.21.15.27:0/4075466918 >> 172.21.15.27:0/4075466918 conn(0x7fd5b40f0ba0 msgr2=0x7fd5b40f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:45:08.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.079+0000 7fd5b88e0700 1 -- 172.21.15.27:0/4075466918 shutdown_connections 2021-11-03T17:45:08.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.079+0000 7fd5b88e0700 1 -- 172.21.15.27:0/4075466918 wait complete. 2021-11-03T17:45:08.174 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.178+0000 7f7f1e5f5700 1 -- 172.21.15.27:0/3845693513 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f180f6ff0 msgr2=0x7f7f180f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:45:08.174 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.178+0000 7f7f1e5f5700 1 --2- 172.21.15.27:0/3845693513 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f180f6ff0 0x7f7f180f7a20 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7f7f08004660 tx=0x7f7f08009b30).stop 2021-11-03T17:45:08.174 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.178+0000 7f7f1e5f5700 1 -- 172.21.15.27:0/3845693513 shutdown_connections 2021-11-03T17:45:08.175 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.178+0000 7f7f1e5f5700 1 --2- 172.21.15.27:0/3845693513 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f180f6ff0 0x7f7f180f7a20 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:08.175 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.178+0000 7f7f1e5f5700 1 --2- 172.21.15.27:0/3845693513 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f180f66e0 0x7f7f180f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:45:08.175 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.178+0000 7f7f1e5f5700 1 -- 172.21.15.27:0/3845693513 >> 172.21.15.27:0/3845693513 conn(0x7f7f180f1f00 msgr2=0x7f7f180f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:45:08.176 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.178+0000 7f7f1e5f5700 1 -- 172.21.15.27:0/3845693513 shutdown_connections 2021-11-03T17:45:08.176 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.179+0000 7f7f1e5f5700 1 -- 172.21.15.27:0/3845693513 wait complete. 2021-11-03T17:45:08.176 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.179+0000 7f7f1e5f5700 1 Processor -- start 2021-11-03T17:45:08.179 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.179+0000 7f7f1e5f5700 1 -- start start 2021-11-03T17:45:08.179 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.179+0000 7f7f1e5f5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f180f66e0 0x7f7f1806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:45:08.179 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.179+0000 7f7f1e5f5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f180f6ff0 0x7f7f18069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:45:08.180 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.179+0000 7f7f1e5f5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1806a410 con 0x7f7f180f6ff0 2021-11-03T17:45:08.180 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.179+0000 7f7f1e5f5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1806a550 con 0x7f7f180f66e0 2021-11-03T17:45:08.180 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.180+0000 7f7f177fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f180f6ff0 0x7f7f18069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:45:08.181 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.180+0000 7f7f177fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f180f6ff0 0x7f7f18069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59204/0 (socket says 172.21.15.27:59204) 2021-11-03T17:45:08.181 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.180+0000 7f7f177fe700 1 -- 172.21.15.27:0/3622586391 learned_addr learned my addr 172.21.15.27:0/3622586391 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:45:08.181 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.180+0000 7f7f17fff700 1 --2- 172.21.15.27:0/3622586391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f180f66e0 0x7f7f1806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:45:08.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.180+0000 7f7f177fe700 1 -- 172.21.15.27:0/3622586391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f180f66e0 msgr2=0x7f7f1806b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:45:08.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.180+0000 7f7f177fe700 1 --2- 172.21.15.27:0/3622586391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f180f66e0 0x7f7f1806b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:08.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.180+0000 7f7f177fe700 1 -- 172.21.15.27:0/3622586391 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f08005040 con 0x7f7f180f6ff0 2021-11-03T17:45:08.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.180+0000 7f7f17fff700 1 --2- 172.21.15.27:0/3622586391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f180f66e0 0x7f7f1806b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:45:08.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.180+0000 7f7f177fe700 1 --2- 172.21.15.27:0/3622586391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f180f6ff0 0x7f7f18069e20 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f7f08009990 tx=0x7f7f08009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:45:08.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.180+0000 7f7f14ff9700 1 -- 172.21.15.27:0/3622586391 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f08018070 con 0x7f7f180f6ff0 2021-11-03T17:45:08.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.180+0000 7f7f14ff9700 1 -- 172.21.15.27:0/3622586391 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f7f08007ca0 con 0x7f7f180f6ff0 2021-11-03T17:45:08.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.180+0000 7f7f1e5f5700 1 -- 172.21.15.27:0/3622586391 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f1806a7d0 con 0x7f7f180f6ff0 2021-11-03T17:45:08.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.180+0000 7f7f14ff9700 1 -- 172.21.15.27:0/3622586391 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f0801c620 con 0x7f7f180f6ff0 2021-11-03T17:45:08.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.180+0000 7f7f1e5f5700 1 -- 172.21.15.27:0/3622586391 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f1810efd0 con 0x7f7f180f6ff0 2021-11-03T17:45:08.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.181+0000 7f7f1e5f5700 1 -- 172.21.15.27:0/3622586391 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f18045ad0 con 0x7f7f180f6ff0 2021-11-03T17:45:08.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.182+0000 7f7f14ff9700 1 -- 172.21.15.27:0/3622586391 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f7f0801d960 con 0x7f7f180f6ff0 2021-11-03T17:45:08.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.182+0000 7f7f14ff9700 1 --2- 172.21.15.27:0/3622586391 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7f000640b0 0x7f7f00066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:45:08.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.182+0000 7f7f14ff9700 1 -- 172.21.15.27:0/3622586391 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f7f080824e0 con 0x7f7f180f6ff0 2021-11-03T17:45:08.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.182+0000 7f7f17fff700 1 --2- 172.21.15.27:0/3622586391 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7f000640b0 0x7f7f00066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:45:08.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.183+0000 7f7f17fff700 1 --2- 172.21.15.27:0/3622586391 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7f000640b0 0x7f7f00066570 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f7f1806b730 tx=0x7f7f0c008040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:45:08.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.186+0000 7f7f14ff9700 1 -- 172.21.15.27:0/3622586391 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f7f080548a0 con 0x7f7f180f6ff0 2021-11-03T17:45:08.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:07 smithi115 ceph-mon[65453]: pgmap v914: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:08.267 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:07 smithi027 ceph-mon[97244]: pgmap v914: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:08.386 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.391+0000 7f7f1e5f5700 1 -- 172.21.15.27:0/3622586391 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7f180fef50 con 0x7f7f180f6ff0 2021-11-03T17:45:08.386 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.391+0000 7f7f14ff9700 1 -- 172.21.15.27:0/3622586391 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f7f08054490 con 0x7f7f180f6ff0 2021-11-03T17:45:08.386 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:45:08.387 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:45:08.387 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:45:08.387 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:45:08.388 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:45:08.388 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:45:08.388 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:45:08.388 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:45:08.389 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:45:08.389 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:45:08.389 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:45:08.389 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:45:08.390 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:45:08.390 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:45:08.390 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:45:08.390 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:45:08.390 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:45:08.391 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:45:08.391 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:45:08.391 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:45:08.391 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:45:08.392 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:45:08.392 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:45:08.392 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:45:08.392 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:45:08.392 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:45:08.393 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:45:08.393 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:45:08.393 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:45:08.393 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:45:08.394 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:45:08.394 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:45:08.395 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:45:08.395 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:45:08.395 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:45:08.395 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:45:08.396 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:45:08.396 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:45:08.396 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:45:08.397 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.393+0000 7f7f1e5f5700 1 -- 172.21.15.27:0/3622586391 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7f000640b0 msgr2=0x7f7f00066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:45:08.397 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.393+0000 7f7f1e5f5700 1 --2- 172.21.15.27:0/3622586391 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7f000640b0 0x7f7f00066570 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f7f1806b730 tx=0x7f7f0c008040).stop 2021-11-03T17:45:08.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.393+0000 7f7f1e5f5700 1 -- 172.21.15.27:0/3622586391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f180f6ff0 msgr2=0x7f7f18069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:45:08.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.393+0000 7f7f1e5f5700 1 --2- 172.21.15.27:0/3622586391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f180f6ff0 0x7f7f18069e20 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f7f08009990 tx=0x7f7f08009b30).stop 2021-11-03T17:45:08.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.393+0000 7f7f1e5f5700 1 -- 172.21.15.27:0/3622586391 shutdown_connections 2021-11-03T17:45:08.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.393+0000 7f7f1e5f5700 1 --2- 172.21.15.27:0/3622586391 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7f000640b0 0x7f7f00066570 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:08.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.393+0000 7f7f1e5f5700 1 --2- 172.21.15.27:0/3622586391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f180f66e0 0x7f7f1806b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:08.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.393+0000 7f7f1e5f5700 1 --2- 172.21.15.27:0/3622586391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f180f6ff0 0x7f7f18069e20 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:08.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.393+0000 7f7f1e5f5700 1 -- 172.21.15.27:0/3622586391 >> 172.21.15.27:0/3622586391 conn(0x7f7f180f1f00 msgr2=0x7f7f180fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:45:08.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.393+0000 7f7f1e5f5700 1 -- 172.21.15.27:0/3622586391 shutdown_connections 2021-11-03T17:45:08.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:08.393+0000 7f7f1e5f5700 1 -- 172.21.15.27:0/3622586391 wait complete. 2021-11-03T17:45:08.400 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:45:09.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:08 smithi115 ceph-mon[65453]: from='client.35048 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:45:09.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:08 smithi115 ceph-mon[65453]: from='client.35052 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:45:09.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:08 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/4075466918' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:09.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:08 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3622586391' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:45:09.267 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:08 smithi027 ceph-mon[97244]: from='client.35048 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:45:09.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:08 smithi027 ceph-mon[97244]: from='client.35052 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:45:09.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:08 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/4075466918' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:09.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:08 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3622586391' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:45:10.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:09 smithi115 ceph-mon[65453]: pgmap v915: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:10.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:09 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1752.400856 secs 2021-11-03T17:45:10.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:45:10.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:45:10.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:45:10.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:10.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:10.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:10.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:10.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:09 smithi027 ceph-mon[97244]: pgmap v915: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:10.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:09 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1752.400856 secs 2021-11-03T17:45:10.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:45:10.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:45:10.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:45:10.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:10.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:10.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:10.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:11.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:10 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:45:11.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:10 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:45:12.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:11 smithi115 ceph-mon[65453]: pgmap v916: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:12.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:11 smithi027 ceph-mon[97244]: pgmap v916: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:14.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:13 smithi115 ceph-mon[65453]: pgmap v917: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:14.267 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:13 smithi027 ceph-mon[97244]: pgmap v917: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:15.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:14 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1757.400988 secs 2021-11-03T17:45:15.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:14 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1757.400988 secs 2021-11-03T17:45:16.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:15 smithi027 ceph-mon[97244]: pgmap v918: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:16.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:15 smithi115 ceph-mon[65453]: pgmap v918: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:18.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:17 smithi027 ceph-mon[97244]: pgmap v919: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:18.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:17 smithi115 ceph-mon[65453]: pgmap v919: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:20.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:19 smithi115 ceph-mon[65453]: pgmap v920: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:20.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:19 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1762.401127 secs 2021-11-03T17:45:20.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:45:20.266 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:19 smithi027 ceph-mon[97244]: pgmap v920: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:20.267 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:19 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1762.401127 secs 2021-11-03T17:45:20.267 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:45:21.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:21 smithi027 ceph-mon[97244]: poll took 60185.675 ms: 1 events 2021-11-03T17:45:21.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:45:21.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:45:21.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:21.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:21.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:21.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:21.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:21 smithi115 ceph-mon[65453]: poll took 60185.675 ms: 1 events 2021-11-03T17:45:21.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:45:21.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:45:21.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:21.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:21.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:21.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:22.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:22 smithi027 ceph-mon[97244]: pgmap v921: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:22.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:22 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:45:22.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:22 smithi115 ceph-mon[65453]: pgmap v921: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:22.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:22 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:45:24.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:24 smithi115 ceph-mon[65453]: pgmap v922: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:24.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:24 smithi027 ceph-mon[97244]: pgmap v922: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:25.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1767.401220 secs 2021-11-03T17:45:25.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1767.401220 secs 2021-11-03T17:45:26.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:26 smithi027 ceph-mon[97244]: pgmap v923: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:26.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:26 smithi115 ceph-mon[65453]: pgmap v923: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:28.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:28 smithi027 ceph-mon[97244]: pgmap v924: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:28.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:28 smithi115 ceph-mon[65453]: pgmap v924: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:30.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:30 smithi027 ceph-mon[97244]: pgmap v925: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:30.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1772.401345 secs 2021-11-03T17:45:30.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:30 smithi115 ceph-mon[65453]: pgmap v925: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:30.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1772.401345 secs 2021-11-03T17:45:31.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:45:31.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:45:31.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:45:31.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:31.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:31.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:31.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:31.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:45:31.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:45:31.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:45:31.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:31.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:31.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:31.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:32.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:32 smithi115 ceph-mon[65453]: pgmap v926: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:32.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:32 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:45:32.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:32 smithi027 ceph-mon[97244]: pgmap v926: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:32.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:32 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:45:34.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:34 smithi115 ceph-mon[65453]: pgmap v927: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:34.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:34 smithi027 ceph-mon[97244]: pgmap v927: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:35.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1777.401440 secs 2021-11-03T17:45:35.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1777.401440 secs 2021-11-03T17:45:36.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:36 smithi115 ceph-mon[65453]: pgmap v928: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:36.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:36 smithi027 ceph-mon[97244]: pgmap v928: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:38.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:38 smithi115 ceph-mon[65453]: pgmap v929: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:38.504 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.508+0000 7f34f5973700 1 -- 172.21.15.27:0/4148229444 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34f00f66e0 msgr2=0x7f34f00f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:45:38.505 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.508+0000 7f34f5973700 1 --2- 172.21.15.27:0/4148229444 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34f00f66e0 0x7f34f00f6b00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7f34e0004660 tx=0x7f34e0009b30).stop 2021-11-03T17:45:38.505 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.508+0000 7f34f5973700 1 -- 172.21.15.27:0/4148229444 shutdown_connections 2021-11-03T17:45:38.505 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.508+0000 7f34f5973700 1 --2- 172.21.15.27:0/4148229444 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34f00f6ff0 0x7f34f00f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:38.505 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.508+0000 7f34f5973700 1 --2- 172.21.15.27:0/4148229444 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34f00f66e0 0x7f34f00f6b00 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:38.506 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.508+0000 7f34f5973700 1 -- 172.21.15.27:0/4148229444 >> 172.21.15.27:0/4148229444 conn(0x7f34f00f1f00 msgr2=0x7f34f00f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:45:38.506 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.508+0000 7f34f5973700 1 -- 172.21.15.27:0/4148229444 shutdown_connections 2021-11-03T17:45:38.506 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.508+0000 7f34f5973700 1 -- 172.21.15.27:0/4148229444 wait complete. 2021-11-03T17:45:38.506 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.508+0000 7f34f5973700 1 Processor -- start 2021-11-03T17:45:38.507 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.509+0000 7f34f5973700 1 -- start start 2021-11-03T17:45:38.507 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.509+0000 7f34f5973700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34f00f66e0 0x7f34f0106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:45:38.507 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.509+0000 7f34f5973700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34f00f6ff0 0x7f34f0107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:45:38.507 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.509+0000 7f34f5973700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f0107a00 con 0x7f34f00f6ff0 2021-11-03T17:45:38.508 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.509+0000 7f34f5973700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34f0107b40 con 0x7f34f00f66e0 2021-11-03T17:45:38.508 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.509+0000 7f34e7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34f00f6ff0 0x7f34f0107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:45:38.508 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.509+0000 7f34e7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34f00f6ff0 0x7f34f0107450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59222/0 (socket says 172.21.15.27:59222) 2021-11-03T17:45:38.508 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.509+0000 7f34e7fff700 1 -- 172.21.15.27:0/3328862196 learned_addr learned my addr 172.21.15.27:0/3328862196 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:45:38.508 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.509+0000 7f34eeffd700 1 --2- 172.21.15.27:0/3328862196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34f00f66e0 0x7f34f0106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:45:38.509 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.509+0000 7f34e7fff700 1 -- 172.21.15.27:0/3328862196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34f00f66e0 msgr2=0x7f34f0106f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:45:38.509 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.509+0000 7f34e7fff700 1 --2- 172.21.15.27:0/3328862196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34f00f66e0 0x7f34f0106f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:38.509 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.509+0000 7f34e7fff700 1 -- 172.21.15.27:0/3328862196 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34e0005040 con 0x7f34f00f6ff0 2021-11-03T17:45:38.509 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.509+0000 7f34eeffd700 1 --2- 172.21.15.27:0/3328862196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34f00f66e0 0x7f34f0106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:45:38.510 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.509+0000 7f34e7fff700 1 --2- 172.21.15.27:0/3328862196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34f00f6ff0 0x7f34f0107450 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f34d800c530 tx=0x7f34d8007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:45:38.510 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.509+0000 7f34e77fe700 1 -- 172.21.15.27:0/3328862196 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34d800f650 con 0x7f34f00f6ff0 2021-11-03T17:45:38.510 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.509+0000 7f34f5973700 1 -- 172.21.15.27:0/3328862196 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34f01084e0 con 0x7f34f00f6ff0 2021-11-03T17:45:38.510 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.509+0000 7f34e77fe700 1 -- 172.21.15.27:0/3328862196 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f34d800fc60 con 0x7f34f00f6ff0 2021-11-03T17:45:38.511 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.510+0000 7f34e77fe700 1 -- 172.21.15.27:0/3328862196 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f34d8012cd0 con 0x7f34f00f6ff0 2021-11-03T17:45:38.511 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.510+0000 7f34f5973700 1 -- 172.21.15.27:0/3328862196 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34f0108aa0 con 0x7f34f00f6ff0 2021-11-03T17:45:38.512 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.511+0000 7f34f5973700 1 -- 172.21.15.27:0/3328862196 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34f0045ad0 con 0x7f34f00f6ff0 2021-11-03T17:45:38.512 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.511+0000 7f34e77fe700 1 -- 172.21.15.27:0/3328862196 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f34d8010070 con 0x7f34f00f6ff0 2021-11-03T17:45:38.513 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.511+0000 7f34e77fe700 1 --2- 172.21.15.27:0/3328862196 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f34dc0640b0 0x7f34dc066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:45:38.513 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.512+0000 7f34e77fe700 1 -- 172.21.15.27:0/3328862196 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f34d8080740 con 0x7f34f00f6ff0 2021-11-03T17:45:38.513 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.512+0000 7f34eeffd700 1 --2- 172.21.15.27:0/3328862196 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f34dc0640b0 0x7f34dc066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:45:38.513 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.512+0000 7f34eeffd700 1 --2- 172.21.15.27:0/3328862196 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f34dc0640b0 0x7f34dc066570 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f34e0004e30 tx=0x7f34e0009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:45:38.513 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.516+0000 7f34e77fe700 1 -- 172.21.15.27:0/3328862196 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f34d8052950 con 0x7f34f00f6ff0 2021-11-03T17:45:38.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:38 smithi027 ceph-mon[97244]: pgmap v929: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:38.691 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.696+0000 7f34f5973700 1 -- 172.21.15.27:0/3328862196 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f34f00ff080 con 0x7f34dc0640b0 2021-11-03T17:45:38.692 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.697+0000 7f34e77fe700 1 -- 172.21.15.27:0/3328862196 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f34e0019040 con 0x7f34dc0640b0 2021-11-03T17:45:38.694 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.699+0000 7f34f5973700 1 -- 172.21.15.27:0/3328862196 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f34dc0640b0 msgr2=0x7f34dc066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:45:38.694 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.699+0000 7f34f5973700 1 --2- 172.21.15.27:0/3328862196 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f34dc0640b0 0x7f34dc066570 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 rx=0x7f34e0004e30 tx=0x7f34e0009b30).stop 2021-11-03T17:45:38.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.699+0000 7f34f5973700 1 -- 172.21.15.27:0/3328862196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34f00f6ff0 msgr2=0x7f34f0107450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:45:38.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.699+0000 7f34f5973700 1 --2- 172.21.15.27:0/3328862196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34f00f6ff0 0x7f34f0107450 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f34d800c530 tx=0x7f34d8007e20).stop 2021-11-03T17:45:38.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.699+0000 7f34f5973700 1 -- 172.21.15.27:0/3328862196 shutdown_connections 2021-11-03T17:45:38.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.699+0000 7f34f5973700 1 --2- 172.21.15.27:0/3328862196 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f34dc0640b0 0x7f34dc066570 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:38.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.699+0000 7f34f5973700 1 --2- 172.21.15.27:0/3328862196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34f00f66e0 0x7f34f0106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:38.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.699+0000 7f34f5973700 1 --2- 172.21.15.27:0/3328862196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34f00f6ff0 0x7f34f0107450 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:38.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.699+0000 7f34f5973700 1 -- 172.21.15.27:0/3328862196 >> 172.21.15.27:0/3328862196 conn(0x7f34f00f1f00 msgr2=0x7f34f00fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:45:38.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.699+0000 7f34f5973700 1 -- 172.21.15.27:0/3328862196 shutdown_connections 2021-11-03T17:45:38.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.699+0000 7f34f5973700 1 -- 172.21.15.27:0/3328862196 wait complete. 2021-11-03T17:45:38.705 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:45:38.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.799+0000 7f1594ab2700 1 -- 172.21.15.27:0/4236902429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15900f6bf0 msgr2=0x7f15900f7010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:45:38.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.799+0000 7f1594ab2700 1 --2- 172.21.15.27:0/4236902429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15900f6bf0 0x7f15900f7010 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f1584004660 tx=0x7f1584009b30).stop 2021-11-03T17:45:38.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.799+0000 7f1594ab2700 1 -- 172.21.15.27:0/4236902429 shutdown_connections 2021-11-03T17:45:38.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.799+0000 7f1594ab2700 1 --2- 172.21.15.27:0/4236902429 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15900f7590 0x7f15900f59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:38.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.799+0000 7f1594ab2700 1 --2- 172.21.15.27:0/4236902429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15900f6bf0 0x7f15900f7010 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:38.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.799+0000 7f1594ab2700 1 -- 172.21.15.27:0/4236902429 >> 172.21.15.27:0/4236902429 conn(0x7f15900f04c0 msgr2=0x7f15900f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:45:38.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.799+0000 7f1594ab2700 1 -- 172.21.15.27:0/4236902429 shutdown_connections 2021-11-03T17:45:38.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.799+0000 7f1594ab2700 1 -- 172.21.15.27:0/4236902429 wait complete. 2021-11-03T17:45:38.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.800+0000 7f1594ab2700 1 Processor -- start 2021-11-03T17:45:38.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.800+0000 7f1594ab2700 1 -- start start 2021-11-03T17:45:38.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.800+0000 7f1594ab2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15900f6bf0 0x7f1590109ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:45:38.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.800+0000 7f1594ab2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15900f7590 0x7f159010c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:45:38.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.800+0000 7f1594ab2700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15900f5640 con 0x7f15900f7590 2021-11-03T17:45:38.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.800+0000 7f1594ab2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15900f5780 con 0x7f15900f6bf0 2021-11-03T17:45:38.800 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.800+0000 7f158dd9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15900f7590 0x7f159010c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:45:38.800 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.800+0000 7f158dd9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15900f7590 0x7f159010c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59232/0 (socket says 172.21.15.27:59232) 2021-11-03T17:45:38.800 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.800+0000 7f158dd9b700 1 -- 172.21.15.27:0/4189832089 learned_addr learned my addr 172.21.15.27:0/4189832089 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:45:38.800 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.800+0000 7f158e59c700 1 --2- 172.21.15.27:0/4189832089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15900f6bf0 0x7f1590109ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:45:38.800 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.800+0000 7f158dd9b700 1 -- 172.21.15.27:0/4189832089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15900f6bf0 msgr2=0x7f1590109ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:45:38.801 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.800+0000 7f158dd9b700 1 --2- 172.21.15.27:0/4189832089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15900f6bf0 0x7f1590109ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:38.801 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.800+0000 7f158dd9b700 1 -- 172.21.15.27:0/4189832089 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1584005040 con 0x7f15900f7590 2021-11-03T17:45:38.801 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.800+0000 7f158e59c700 1 --2- 172.21.15.27:0/4189832089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15900f6bf0 0x7f1590109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:45:38.801 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.800+0000 7f158dd9b700 1 --2- 172.21.15.27:0/4189832089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15900f7590 0x7f159010c1a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7f15800026d0 tx=0x7f1580009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:45:38.802 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.801+0000 7f157effd700 1 -- 172.21.15.27:0/4189832089 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1580009320 con 0x7f15900f7590 2021-11-03T17:45:38.802 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.801+0000 7f157effd700 1 -- 172.21.15.27:0/4189832089 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f158000f040 con 0x7f15900f7590 2021-11-03T17:45:38.802 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.801+0000 7f1594ab2700 1 -- 172.21.15.27:0/4189832089 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f159010c750 con 0x7f15900f7590 2021-11-03T17:45:38.802 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.801+0000 7f157effd700 1 -- 172.21.15.27:0/4189832089 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15800147e0 con 0x7f15900f7590 2021-11-03T17:45:38.803 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.801+0000 7f1594ab2700 1 -- 172.21.15.27:0/4189832089 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f159010cd10 con 0x7f15900f7590 2021-11-03T17:45:38.803 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.801+0000 7f1594ab2700 1 -- 172.21.15.27:0/4189832089 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1590045ad0 con 0x7f15900f7590 2021-11-03T17:45:38.804 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.802+0000 7f157effd700 1 -- 172.21.15.27:0/4189832089 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f1580009480 con 0x7f15900f7590 2021-11-03T17:45:38.804 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.803+0000 7f157effd700 1 --2- 172.21.15.27:0/4189832089 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f15780640b0 0x7f1578066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:45:38.804 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.803+0000 7f157effd700 1 -- 172.21.15.27:0/4189832089 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f1580081140 con 0x7f15900f7590 2021-11-03T17:45:38.805 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.803+0000 7f158e59c700 1 --2- 172.21.15.27:0/4189832089 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f15780640b0 0x7f1578066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:45:38.805 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.803+0000 7f158e59c700 1 --2- 172.21.15.27:0/4189832089 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f15780640b0 0x7f1578066570 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f1584004e30 tx=0x7f1584009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:45:38.805 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.806+0000 7f157effd700 1 -- 172.21.15.27:0/4189832089 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f15800534c0 con 0x7f15900f7590 2021-11-03T17:45:38.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.977+0000 7f1594ab2700 1 -- 172.21.15.27:0/4189832089 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1590057f10 con 0x7f15780640b0 2021-11-03T17:45:38.987 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.992+0000 7f157effd700 1 -- 172.21.15.27:0/4189832089 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f1590057f10 con 0x7f15780640b0 2021-11-03T17:45:38.987 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:45:38.987 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (37m) 6m ago 38m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:45:38.988 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (28m) 6m ago 38m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:45:38.988 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (28m) 5m ago 37m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:45:38.988 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (37m) 6m ago 38m 41.3M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:45:38.988 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (30m) 6m ago 33m 21.1M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:45:38.989 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (33m) 6m ago 33m 3843M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:45:38.989 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (29m) 5m ago 33m 20.4M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:45:38.989 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (31m) 5m ago 33m 22.1M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:45:38.989 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (30m) 6m ago 39m 517M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:45:38.990 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (30m) 5m ago 37m 419M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:45:38.990 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (29m) 6m ago 39m 159M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:45:38.990 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (29m) 5m ago 37m 82.5M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:45:38.990 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (38m) 6m ago 38m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:45:38.990 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (37m) 5m ago 37m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:45:38.991 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (28m) 6m ago 36m 311M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:45:38.991 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (28m) 6m ago 36m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:45:38.991 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (27m) 6m ago 36m 248M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:45:38.991 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (27m) 5m ago 35m 354M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:45:38.991 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (26m) 5m ago 35m 285M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:45:38.992 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (26m) 5m ago 35m 287M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:45:38.992 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (36m) 6m ago 37m 50.5M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:45:38.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.994+0000 7f1594ab2700 1 -- 172.21.15.27:0/4189832089 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f15780640b0 msgr2=0x7f1578066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:45:38.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.994+0000 7f1594ab2700 1 --2- 172.21.15.27:0/4189832089 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f15780640b0 0x7f1578066570 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f1584004e30 tx=0x7f1584009b30).stop 2021-11-03T17:45:38.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.994+0000 7f1594ab2700 1 -- 172.21.15.27:0/4189832089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15900f7590 msgr2=0x7f159010c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:45:38.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.994+0000 7f1594ab2700 1 --2- 172.21.15.27:0/4189832089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15900f7590 0x7f159010c1a0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7f15800026d0 tx=0x7f1580009de0).stop 2021-11-03T17:45:38.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.994+0000 7f1594ab2700 1 -- 172.21.15.27:0/4189832089 shutdown_connections 2021-11-03T17:45:38.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.994+0000 7f1594ab2700 1 --2- 172.21.15.27:0/4189832089 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f15780640b0 0x7f1578066570 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:38.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.994+0000 7f1594ab2700 1 --2- 172.21.15.27:0/4189832089 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15900f6bf0 0x7f1590109ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:38.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.994+0000 7f1594ab2700 1 --2- 172.21.15.27:0/4189832089 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15900f7590 0x7f159010c1a0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:38.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.994+0000 7f1594ab2700 1 -- 172.21.15.27:0/4189832089 >> 172.21.15.27:0/4189832089 conn(0x7f15900f04c0 msgr2=0x7f15900fa3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:45:38.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.994+0000 7f1594ab2700 1 -- 172.21.15.27:0/4189832089 shutdown_connections 2021-11-03T17:45:38.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:38.994+0000 7f1594ab2700 1 -- 172.21.15.27:0/4189832089 wait complete. 2021-11-03T17:45:39.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.093+0000 7fd08199a700 1 -- 172.21.15.27:0/3888093183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd07c0f6550 msgr2=0x7fd07c0f6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:45:39.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.093+0000 7fd08199a700 1 --2- 172.21.15.27:0/3888093183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd07c0f6550 0x7fd07c0f6970 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7fd06c004660 tx=0x7fd06c009b30).stop 2021-11-03T17:45:39.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.093+0000 7fd08199a700 1 -- 172.21.15.27:0/3888093183 shutdown_connections 2021-11-03T17:45:39.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.093+0000 7fd08199a700 1 --2- 172.21.15.27:0/3888093183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd07c0f77f0 0x7fd07c0f7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:39.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.093+0000 7fd08199a700 1 --2- 172.21.15.27:0/3888093183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd07c0f6550 0x7fd07c0f6970 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:39.090 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.093+0000 7fd08199a700 1 -- 172.21.15.27:0/3888093183 >> 172.21.15.27:0/3888093183 conn(0x7fd07c0f1cd0 msgr2=0x7fd07c0f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:45:39.090 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.094+0000 7fd08199a700 1 -- 172.21.15.27:0/3888093183 shutdown_connections 2021-11-03T17:45:39.090 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.094+0000 7fd08199a700 1 -- 172.21.15.27:0/3888093183 wait complete. 2021-11-03T17:45:39.093 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.094+0000 7fd08199a700 1 Processor -- start 2021-11-03T17:45:39.093 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.094+0000 7fd08199a700 1 -- start start 2021-11-03T17:45:39.093 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.094+0000 7fd08199a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd07c0f6550 0x7fd07c106e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:45:39.093 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.094+0000 7fd08199a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd07c0f77f0 0x7fd07c107300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:45:39.094 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.094+0000 7fd08199a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07c107900 con 0x7fd07c0f77f0 2021-11-03T17:45:39.094 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.094+0000 7fd08199a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd07c108110 con 0x7fd07c0f6550 2021-11-03T17:45:39.094 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.095+0000 7fd07a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd07c0f77f0 0x7fd07c107300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:45:39.094 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.095+0000 7fd07a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd07c0f77f0 0x7fd07c107300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59242/0 (socket says 172.21.15.27:59242) 2021-11-03T17:45:39.094 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.095+0000 7fd07a7fc700 1 -- 172.21.15.27:0/4194827440 learned_addr learned my addr 172.21.15.27:0/4194827440 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:45:39.095 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.095+0000 7fd07affd700 1 --2- 172.21.15.27:0/4194827440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd07c0f6550 0x7fd07c106e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:45:39.095 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.095+0000 7fd07a7fc700 1 -- 172.21.15.27:0/4194827440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd07c0f6550 msgr2=0x7fd07c106e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:45:39.095 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.095+0000 7fd07a7fc700 1 --2- 172.21.15.27:0/4194827440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd07c0f6550 0x7fd07c106e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:39.095 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.095+0000 7fd07a7fc700 1 -- 172.21.15.27:0/4194827440 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd06c005040 con 0x7fd07c0f77f0 2021-11-03T17:45:39.096 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.095+0000 7fd07affd700 1 --2- 172.21.15.27:0/4194827440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd07c0f6550 0x7fd07c106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:45:39.096 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.095+0000 7fd07a7fc700 1 --2- 172.21.15.27:0/4194827440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd07c0f77f0 0x7fd07c107300 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7fd0700026d0 tx=0x7fd070009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:45:39.096 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.095+0000 7fd0637fe700 1 -- 172.21.15.27:0/4194827440 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd070009320 con 0x7fd07c0f77f0 2021-11-03T17:45:39.096 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.095+0000 7fd08199a700 1 -- 172.21.15.27:0/4194827440 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd07c108390 con 0x7fd07c0f77f0 2021-11-03T17:45:39.097 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.095+0000 7fd0637fe700 1 -- 172.21.15.27:0/4194827440 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fd07000f040 con 0x7fd07c0f77f0 2021-11-03T17:45:39.097 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.095+0000 7fd0637fe700 1 -- 172.21.15.27:0/4194827440 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd0700147e0 con 0x7fd07c0f77f0 2021-11-03T17:45:39.097 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.095+0000 7fd08199a700 1 -- 172.21.15.27:0/4194827440 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd07c108950 con 0x7fd07c0f77f0 2021-11-03T17:45:39.097 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.096+0000 7fd08199a700 1 -- 172.21.15.27:0/4194827440 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd07c059100 con 0x7fd07c0f77f0 2021-11-03T17:45:39.097 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.096+0000 7fd0637fe700 1 -- 172.21.15.27:0/4194827440 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fd070009480 con 0x7fd07c0f77f0 2021-11-03T17:45:39.098 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.097+0000 7fd0637fe700 1 --2- 172.21.15.27:0/4194827440 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd064064100 0x7fd0640665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:45:39.098 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.097+0000 7fd0637fe700 1 -- 172.21.15.27:0/4194827440 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fd070080f10 con 0x7fd07c0f77f0 2021-11-03T17:45:39.098 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.097+0000 7fd07affd700 1 --2- 172.21.15.27:0/4194827440 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd064064100 0x7fd0640665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:45:39.098 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.097+0000 7fd07affd700 1 --2- 172.21.15.27:0/4194827440 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd064064100 0x7fd0640665c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7fd06c004e30 tx=0x7fd06c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:45:39.099 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.101+0000 7fd0637fe700 1 -- 172.21.15.27:0/4194827440 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fd070053210 con 0x7fd07c0f77f0 2021-11-03T17:45:39.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.333+0000 7fd08199a700 1 -- 172.21.15.27:0/4194827440 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd07c0fed10 con 0x7fd07c0f77f0 2021-11-03T17:45:39.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.333+0000 7fd0637fe700 1 -- 172.21.15.27:0/4194827440 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fd070052e00 con 0x7fd07c0f77f0 2021-11-03T17:45:39.329 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:45:39.329 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:45:39.329 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:45:39.330 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:45:39.330 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:45:39.330 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:45:39.331 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:45:39.331 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:45:39.331 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:45:39.332 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:45:39.332 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:45:39.332 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:45:39.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:45:39.333 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:45:39.333 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:45:39.334 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:45:39.334 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:45:39.334 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:45:39.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.335+0000 7fd08199a700 1 -- 172.21.15.27:0/4194827440 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd064064100 msgr2=0x7fd0640665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:45:39.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.335+0000 7fd08199a700 1 --2- 172.21.15.27:0/4194827440 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd064064100 0x7fd0640665c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7fd06c004e30 tx=0x7fd06c009b30).stop 2021-11-03T17:45:39.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.335+0000 7fd08199a700 1 -- 172.21.15.27:0/4194827440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd07c0f77f0 msgr2=0x7fd07c107300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:45:39.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.335+0000 7fd08199a700 1 --2- 172.21.15.27:0/4194827440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd07c0f77f0 0x7fd07c107300 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7fd0700026d0 tx=0x7fd070009de0).stop 2021-11-03T17:45:39.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.335+0000 7fd08199a700 1 -- 172.21.15.27:0/4194827440 shutdown_connections 2021-11-03T17:45:39.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.335+0000 7fd08199a700 1 --2- 172.21.15.27:0/4194827440 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd064064100 0x7fd0640665c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:39.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.335+0000 7fd08199a700 1 --2- 172.21.15.27:0/4194827440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd07c0f6550 0x7fd07c106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:39.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.335+0000 7fd08199a700 1 --2- 172.21.15.27:0/4194827440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd07c0f77f0 0x7fd07c107300 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:39.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.335+0000 7fd08199a700 1 -- 172.21.15.27:0/4194827440 >> 172.21.15.27:0/4194827440 conn(0x7fd07c0f1cd0 msgr2=0x7fd07c0fb240 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:45:39.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.335+0000 7fd08199a700 1 -- 172.21.15.27:0/4194827440 shutdown_connections 2021-11-03T17:45:39.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.335+0000 7fd08199a700 1 -- 172.21.15.27:0/4194827440 wait complete. 2021-11-03T17:45:39.434 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.436+0000 7fc352cf9700 1 -- 172.21.15.27:0/502988589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc34c0f55c0 msgr2=0x7fc34c0f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:45:39.434 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.436+0000 7fc352cf9700 1 --2- 172.21.15.27:0/502988589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc34c0f55c0 0x7fc34c0f59e0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7fc3440099e0 tx=0x7fc344004d10).stop 2021-11-03T17:45:39.434 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.436+0000 7fc352cf9700 1 -- 172.21.15.27:0/502988589 shutdown_connections 2021-11-03T17:45:39.435 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.436+0000 7fc352cf9700 1 --2- 172.21.15.27:0/502988589 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc34c0f6860 0x7fc34c0f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:39.435 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.436+0000 7fc352cf9700 1 --2- 172.21.15.27:0/502988589 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc34c0f55c0 0x7fc34c0f59e0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:39.435 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.436+0000 7fc352cf9700 1 -- 172.21.15.27:0/502988589 >> 172.21.15.27:0/502988589 conn(0x7fc34c0f0d40 msgr2=0x7fc34c0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:45:39.435 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.436+0000 7fc352cf9700 1 -- 172.21.15.27:0/502988589 shutdown_connections 2021-11-03T17:45:39.436 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.436+0000 7fc352cf9700 1 -- 172.21.15.27:0/502988589 wait complete. 2021-11-03T17:45:39.436 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.436+0000 7fc352cf9700 1 Processor -- start 2021-11-03T17:45:39.436 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.436+0000 7fc352cf9700 1 -- start start 2021-11-03T17:45:39.436 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.437+0000 7fc352cf9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc34c0f55c0 0x7fc34c109d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:45:39.436 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.437+0000 7fc352cf9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc34c0f6860 0x7fc34c10c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:45:39.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.437+0000 7fc352cf9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc34c0603e0 con 0x7fc34c0f55c0 2021-11-03T17:45:39.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.437+0000 7fc352cf9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc34c060520 con 0x7fc34c0f6860 2021-11-03T17:45:39.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.437+0000 7fc350a95700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc34c0f55c0 0x7fc34c109d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:45:39.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.437+0000 7fc350a95700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc34c0f55c0 0x7fc34c109d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59250/0 (socket says 172.21.15.27:59250) 2021-11-03T17:45:39.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.437+0000 7fc350a95700 1 -- 172.21.15.27:0/1516916647 learned_addr learned my addr 172.21.15.27:0/1516916647 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:45:39.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.437+0000 7fc34bfff700 1 --2- 172.21.15.27:0/1516916647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc34c0f6860 0x7fc34c10c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:45:39.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.437+0000 7fc350a95700 1 -- 172.21.15.27:0/1516916647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc34c0f6860 msgr2=0x7fc34c10c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:45:39.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.437+0000 7fc350a95700 1 --2- 172.21.15.27:0/1516916647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc34c0f6860 0x7fc34c10c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:39.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.437+0000 7fc350a95700 1 -- 172.21.15.27:0/1516916647 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc344009640 con 0x7fc34c0f55c0 2021-11-03T17:45:39.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.437+0000 7fc34bfff700 1 --2- 172.21.15.27:0/1516916647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc34c0f6860 0x7fc34c10c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:45:39.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.437+0000 7fc350a95700 1 --2- 172.21.15.27:0/1516916647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc34c0f55c0 0x7fc34c109d60 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7fc34400c010 tx=0x7fc344004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:45:39.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.437+0000 7fc3497fa700 1 -- 172.21.15.27:0/1516916647 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc34400e070 con 0x7fc34c0f55c0 2021-11-03T17:45:39.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.437+0000 7fc352cf9700 1 -- 172.21.15.27:0/1516916647 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc34c10c7b0 con 0x7fc34c0f55c0 2021-11-03T17:45:39.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.437+0000 7fc3497fa700 1 -- 172.21.15.27:0/1516916647 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fc344007dc0 con 0x7fc34c0f55c0 2021-11-03T17:45:39.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.437+0000 7fc3497fa700 1 -- 172.21.15.27:0/1516916647 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc344018070 con 0x7fc34c0f55c0 2021-11-03T17:45:39.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.438+0000 7fc352cf9700 1 -- 172.21.15.27:0/1516916647 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc34c10cc90 con 0x7fc34c0f55c0 2021-11-03T17:45:39.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.438+0000 7fc352cf9700 1 -- 172.21.15.27:0/1516916647 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc34c045ad0 con 0x7fc34c0f55c0 2021-11-03T17:45:39.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.439+0000 7fc3497fa700 1 -- 172.21.15.27:0/1516916647 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fc344024060 con 0x7fc34c0f55c0 2021-11-03T17:45:39.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.439+0000 7fc3497fa700 1 --2- 172.21.15.27:0/1516916647 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc3340640b0 0x7fc334066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:45:39.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.439+0000 7fc3497fa700 1 -- 172.21.15.27:0/1516916647 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fc344082130 con 0x7fc34c0f55c0 2021-11-03T17:45:39.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.439+0000 7fc34bfff700 1 --2- 172.21.15.27:0/1516916647 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc3340640b0 0x7fc334066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:45:39.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.440+0000 7fc34bfff700 1 --2- 172.21.15.27:0/1516916647 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc3340640b0 0x7fc334066570 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7fc340009740 tx=0x7fc340009be0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:45:39.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.443+0000 7fc3497fa700 1 -- 172.21.15.27:0/1516916647 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fc344054430 con 0x7fc34c0f55c0 2021-11-03T17:45:39.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.648+0000 7fc352cf9700 1 -- 172.21.15.27:0/1516916647 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc34c0fdec0 con 0x7fc34c0f55c0 2021-11-03T17:45:39.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.649+0000 7fc3497fa700 1 -- 172.21.15.27:0/1516916647 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7fc344022710 con 0x7fc34c0f55c0 2021-11-03T17:45:39.644 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:45:39.644 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:45:39.645 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:45:39.645 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:45:39.645 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:45:39.646 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:45:39.646 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:45:39.646 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:45:39.647 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:45:39.647 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:45:39.647 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:45:39.647 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:45:39.647 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:45:39.648 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:45:39.648 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:45:39.648 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:45:39.648 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:45:39.649 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:45:39.649 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:45:39.649 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:45:39.649 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:45:39.649 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:45:39.650 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:45:39.650 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:45:39.650 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:45:39.650 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:45:39.651 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:45:39.651 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:45:39.651 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:45:39.652 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:45:39.652 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:45:39.652 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:45:39.652 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:45:39.653 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:45:39.653 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:45:39.653 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:45:39.653 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:45:39.653 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:45:39.654 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:45:39.655 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.650+0000 7fc352cf9700 1 -- 172.21.15.27:0/1516916647 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc3340640b0 msgr2=0x7fc334066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:45:39.655 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.650+0000 7fc352cf9700 1 --2- 172.21.15.27:0/1516916647 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc3340640b0 0x7fc334066570 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 rx=0x7fc340009740 tx=0x7fc340009be0).stop 2021-11-03T17:45:39.655 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.650+0000 7fc352cf9700 1 -- 172.21.15.27:0/1516916647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc34c0f55c0 msgr2=0x7fc34c109d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:45:39.656 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.650+0000 7fc352cf9700 1 --2- 172.21.15.27:0/1516916647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc34c0f55c0 0x7fc34c109d60 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7fc34400c010 tx=0x7fc344004d10).stop 2021-11-03T17:45:39.656 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.651+0000 7fc352cf9700 1 -- 172.21.15.27:0/1516916647 shutdown_connections 2021-11-03T17:45:39.656 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.651+0000 7fc352cf9700 1 --2- 172.21.15.27:0/1516916647 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc3340640b0 0x7fc334066570 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:39.656 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.651+0000 7fc352cf9700 1 --2- 172.21.15.27:0/1516916647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc34c0f55c0 0x7fc34c109d60 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:39.657 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.651+0000 7fc352cf9700 1 --2- 172.21.15.27:0/1516916647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc34c0f6860 0x7fc34c10c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:45:39.657 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.651+0000 7fc352cf9700 1 -- 172.21.15.27:0/1516916647 >> 172.21.15.27:0/1516916647 conn(0x7fc34c0f0d40 msgr2=0x7fc34c0fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:45:39.657 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.651+0000 7fc352cf9700 1 -- 172.21.15.27:0/1516916647 shutdown_connections 2021-11-03T17:45:39.657 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:45:39.651+0000 7fc352cf9700 1 -- 172.21.15.27:0/1516916647 wait complete. 2021-11-03T17:45:39.657 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:45:40.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:40 smithi115 ceph-mon[65453]: pgmap v930: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:40.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:40 smithi115 ceph-mon[65453]: from='client.35064 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:45:40.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:40 smithi115 ceph-mon[65453]: from='client.35068 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:45:40.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:40 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/4194827440' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:40.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1782.401509 secs 2021-11-03T17:45:40.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:40 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1516916647' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:45:40.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:40 smithi027 ceph-mon[97244]: pgmap v930: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:40.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:40 smithi027 ceph-mon[97244]: from='client.35064 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:45:40.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:40 smithi027 ceph-mon[97244]: from='client.35068 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:45:40.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:40 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/4194827440' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:40.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1782.401509 secs 2021-11-03T17:45:40.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:40 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1516916647' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:45:41.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:45:41.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:45:41.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:45:41.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:41.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:41.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:41.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:41.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:45:41.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:45:41.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:45:41.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:41.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:41.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:41.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:42.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:42 smithi115 ceph-mon[65453]: pgmap v931: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:42.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:42 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:45:42.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:42 smithi027 ceph-mon[97244]: pgmap v931: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:42.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:42 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:45:44.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:44 smithi115 ceph-mon[65453]: pgmap v932: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:44.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:44 smithi027 ceph-mon[97244]: pgmap v932: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:45.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1787.401631 secs 2021-11-03T17:45:45.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1787.401631 secs 2021-11-03T17:45:46.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:46 smithi115 ceph-mon[65453]: pgmap v933: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:46.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:46 smithi027 ceph-mon[97244]: pgmap v933: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:48.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:48 smithi115 ceph-mon[65453]: pgmap v934: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:48.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:48 smithi027 ceph-mon[97244]: pgmap v934: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:50.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:50 smithi115 ceph-mon[65453]: pgmap v935: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:50.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1792.401772 secs 2021-11-03T17:45:50.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:50 smithi027 ceph-mon[97244]: pgmap v935: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:50.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1792.401772 secs 2021-11-03T17:45:51.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:45:51.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:45:51.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:45:51.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:51.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:51.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:51.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:51.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:45:51.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:45:51.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:45:51.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:51.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:51.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:51.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:45:52.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:52 smithi115 ceph-mon[65453]: pgmap v936: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:52.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:52 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:45:52.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:52 smithi027 ceph-mon[97244]: pgmap v936: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:52.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:52 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:45:54.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:54 smithi115 ceph-mon[65453]: pgmap v937: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:54.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:54 smithi027 ceph-mon[97244]: pgmap v937: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:55.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1797.401881 secs 2021-11-03T17:45:55.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1797.401881 secs 2021-11-03T17:45:56.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:56 smithi115 ceph-mon[65453]: pgmap v938: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:45:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:45:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:56 smithi027 ceph-mon[97244]: pgmap v938: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:45:56.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:45:58.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:45:58 smithi115 ceph-mon[65453]: pgmap v939: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:45:58.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:45:58 smithi027 ceph-mon[97244]: pgmap v939: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:00.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:00 smithi115 ceph-mon[65453]: pgmap v940: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:00.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1802.402003 secs 2021-11-03T17:46:00.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:00 smithi027 ceph-mon[97244]: pgmap v940: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:00.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1802.402003 secs 2021-11-03T17:46:01.447 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:46:01.448 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:46:01.448 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:46:01.448 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:01.448 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:01.449 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:01.449 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:01.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:46:01.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:46:01.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:46:01.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:01.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:01.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:01.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:02.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:02 smithi115 ceph-mon[65453]: pgmap v941: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:02.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:02 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:46:02.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:02 smithi027 ceph-mon[97244]: pgmap v941: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:02.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:02 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:46:04.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:04 smithi115 ceph-mon[65453]: pgmap v942: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:04.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:04 smithi027 ceph-mon[97244]: pgmap v942: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:05.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1807.402143 secs 2021-11-03T17:46:05.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1807.402143 secs 2021-11-03T17:46:06.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:06 smithi115 ceph-mon[65453]: pgmap v943: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:06.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:06 smithi027 ceph-mon[97244]: pgmap v943: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:08.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:08 smithi115 ceph-mon[65453]: pgmap v944: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:08.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:08 smithi027 ceph-mon[97244]: pgmap v944: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:09.761 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.766+0000 7f3996faf700 1 -- 172.21.15.27:0/2378821821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39900f5ef0 msgr2=0x7f39900f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:46:09.761 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.766+0000 7f3996faf700 1 --2- 172.21.15.27:0/2378821821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39900f5ef0 0x7f39900f6920 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f39800099e0 tx=0x7f3980004d10).stop 2021-11-03T17:46:09.761 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.766+0000 7f3996faf700 1 -- 172.21.15.27:0/2378821821 shutdown_connections 2021-11-03T17:46:09.762 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.766+0000 7f3996faf700 1 --2- 172.21.15.27:0/2378821821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39900f5ef0 0x7f39900f6920 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:09.762 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.766+0000 7f3996faf700 1 --2- 172.21.15.27:0/2378821821 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39900f55e0 0x7f39900f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:09.762 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.766+0000 7f3996faf700 1 -- 172.21.15.27:0/2378821821 >> 172.21.15.27:0/2378821821 conn(0x7f39900f0de0 msgr2=0x7f39900f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:46:09.762 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.766+0000 7f3996faf700 1 -- 172.21.15.27:0/2378821821 shutdown_connections 2021-11-03T17:46:09.763 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.767+0000 7f3996faf700 1 -- 172.21.15.27:0/2378821821 wait complete. 2021-11-03T17:46:09.763 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.767+0000 7f3996faf700 1 Processor -- start 2021-11-03T17:46:09.763 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.767+0000 7f3996faf700 1 -- start start 2021-11-03T17:46:09.765 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.767+0000 7f3996faf700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39900f55e0 0x7f3990109d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:46:09.766 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.767+0000 7f3996faf700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39900f5ef0 0x7f399010c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:46:09.766 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.767+0000 7f3996faf700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39900604e0 con 0x7f39900f55e0 2021-11-03T17:46:09.766 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.767+0000 7f3996faf700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3990060620 con 0x7f39900f5ef0 2021-11-03T17:46:09.767 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.768+0000 7f3994d4b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39900f55e0 0x7f3990109d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:46:09.767 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.768+0000 7f3994d4b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39900f55e0 0x7f3990109d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59282/0 (socket says 172.21.15.27:59282) 2021-11-03T17:46:09.767 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.768+0000 7f3994d4b700 1 -- 172.21.15.27:0/1727481464 learned_addr learned my addr 172.21.15.27:0/1727481464 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:46:09.767 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.768+0000 7f398ffff700 1 --2- 172.21.15.27:0/1727481464 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39900f5ef0 0x7f399010c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:46:09.767 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.768+0000 7f3994d4b700 1 -- 172.21.15.27:0/1727481464 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39900f5ef0 msgr2=0x7f399010c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:46:09.768 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.768+0000 7f3994d4b700 1 --2- 172.21.15.27:0/1727481464 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39900f5ef0 0x7f399010c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:09.768 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.768+0000 7f3994d4b700 1 -- 172.21.15.27:0/1727481464 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3980009640 con 0x7f39900f55e0 2021-11-03T17:46:09.768 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.768+0000 7f3994d4b700 1 --2- 172.21.15.27:0/1727481464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39900f55e0 0x7f3990109d60 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f39880026d0 tx=0x7f3988009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:46:09.768 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.768+0000 7f398d7fa700 1 -- 172.21.15.27:0/1727481464 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39880079c0 con 0x7f39900f55e0 2021-11-03T17:46:09.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.768+0000 7f3996faf700 1 -- 172.21.15.27:0/1727481464 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f399010c810 con 0x7f39900f55e0 2021-11-03T17:46:09.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.768+0000 7f3996faf700 1 -- 172.21.15.27:0/1727481464 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f399010cd50 con 0x7f39900f55e0 2021-11-03T17:46:09.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.769+0000 7f398d7fa700 1 -- 172.21.15.27:0/1727481464 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f398800f040 con 0x7f39900f55e0 2021-11-03T17:46:09.770 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.769+0000 7f398d7fa700 1 -- 172.21.15.27:0/1727481464 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3988014800 con 0x7f39900f55e0 2021-11-03T17:46:09.770 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.769+0000 7f3996faf700 1 -- 172.21.15.27:0/1727481464 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3974004fa0 con 0x7f39900f55e0 2021-11-03T17:46:09.770 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.770+0000 7f398d7fa700 1 -- 172.21.15.27:0/1727481464 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f398801d3f0 con 0x7f39900f55e0 2021-11-03T17:46:09.770 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.770+0000 7f398d7fa700 1 --2- 172.21.15.27:0/1727481464 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f397c064090 0x7f397c066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:46:09.771 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.770+0000 7f398d7fa700 1 -- 172.21.15.27:0/1727481464 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f3988080680 con 0x7f39900f55e0 2021-11-03T17:46:09.771 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.770+0000 7f398ffff700 1 --2- 172.21.15.27:0/1727481464 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f397c064090 0x7f397c066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:46:09.771 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.771+0000 7f398ffff700 1 --2- 172.21.15.27:0/1727481464 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f397c064090 0x7f397c066550 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f3980003380 tx=0x7f3980004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:46:09.772 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.774+0000 7f398d7fa700 1 -- 172.21.15.27:0/1727481464 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f3988052a60 con 0x7f39900f55e0 2021-11-03T17:46:09.948 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.954+0000 7f3996faf700 1 -- 172.21.15.27:0/1727481464 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3974000bc0 con 0x7f397c064090 2021-11-03T17:46:09.949 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.955+0000 7f398d7fa700 1 -- 172.21.15.27:0/1727481464 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f3974000bc0 con 0x7f397c064090 2021-11-03T17:46:09.951 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.957+0000 7f3996faf700 1 -- 172.21.15.27:0/1727481464 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f397c064090 msgr2=0x7f397c066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:46:09.951 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.957+0000 7f3996faf700 1 --2- 172.21.15.27:0/1727481464 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f397c064090 0x7f397c066550 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f3980003380 tx=0x7f3980004d10).stop 2021-11-03T17:46:09.952 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.957+0000 7f3996faf700 1 -- 172.21.15.27:0/1727481464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39900f55e0 msgr2=0x7f3990109d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:46:09.952 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.957+0000 7f3996faf700 1 --2- 172.21.15.27:0/1727481464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39900f55e0 0x7f3990109d60 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f39880026d0 tx=0x7f3988009de0).stop 2021-11-03T17:46:09.952 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.957+0000 7f3996faf700 1 -- 172.21.15.27:0/1727481464 shutdown_connections 2021-11-03T17:46:09.952 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.957+0000 7f3996faf700 1 --2- 172.21.15.27:0/1727481464 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f397c064090 0x7f397c066550 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:09.953 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.957+0000 7f3996faf700 1 --2- 172.21.15.27:0/1727481464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39900f55e0 0x7f3990109d60 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:09.953 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.957+0000 7f3996faf700 1 --2- 172.21.15.27:0/1727481464 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39900f5ef0 0x7f399010c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:09.953 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.957+0000 7f3996faf700 1 -- 172.21.15.27:0/1727481464 >> 172.21.15.27:0/1727481464 conn(0x7f39900f0de0 msgr2=0x7f39900fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:46:09.953 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.957+0000 7f3996faf700 1 -- 172.21.15.27:0/1727481464 shutdown_connections 2021-11-03T17:46:09.954 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:09.957+0000 7f3996faf700 1 -- 172.21.15.27:0/1727481464 wait complete. 2021-11-03T17:46:09.961 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:46:10.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.057+0000 7f81bd09f700 1 -- 172.21.15.27:0/1323687237 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81b80f55e0 msgr2=0x7f81b80f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:46:10.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.057+0000 7f81bd09f700 1 --2- 172.21.15.27:0/1323687237 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81b80f55e0 0x7f81b80f5a00 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f81a8004660 tx=0x7f81a8009b30).stop 2021-11-03T17:46:10.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.057+0000 7f81bd09f700 1 -- 172.21.15.27:0/1323687237 shutdown_connections 2021-11-03T17:46:10.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.057+0000 7f81bd09f700 1 --2- 172.21.15.27:0/1323687237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81b80f5ef0 0x7f81b80f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:10.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.057+0000 7f81bd09f700 1 --2- 172.21.15.27:0/1323687237 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81b80f55e0 0x7f81b80f5a00 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:10.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.057+0000 7f81bd09f700 1 -- 172.21.15.27:0/1323687237 >> 172.21.15.27:0/1323687237 conn(0x7f81b80f0de0 msgr2=0x7f81b80f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:46:10.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.057+0000 7f81bd09f700 1 -- 172.21.15.27:0/1323687237 shutdown_connections 2021-11-03T17:46:10.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.057+0000 7f81bd09f700 1 -- 172.21.15.27:0/1323687237 wait complete. 2021-11-03T17:46:10.056 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.058+0000 7f81bd09f700 1 Processor -- start 2021-11-03T17:46:10.056 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.058+0000 7f81bd09f700 1 -- start start 2021-11-03T17:46:10.056 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.058+0000 7f81bd09f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81b80f55e0 0x7f81b8109c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:46:10.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.058+0000 7f81bd09f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81b80f5ef0 0x7f81b810c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:46:10.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.058+0000 7f81bd09f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b8060520 con 0x7f81b80f5ef0 2021-11-03T17:46:10.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.058+0000 7f81bd09f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81b8060660 con 0x7f81b80f55e0 2021-11-03T17:46:10.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.058+0000 7f81b659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81b80f5ef0 0x7f81b810c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:46:10.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.058+0000 7f81b659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81b80f5ef0 0x7f81b810c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59294/0 (socket says 172.21.15.27:59294) 2021-11-03T17:46:10.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.058+0000 7f81b659c700 1 -- 172.21.15.27:0/553264561 learned_addr learned my addr 172.21.15.27:0/553264561 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:46:10.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.058+0000 7f81b6d9d700 1 --2- 172.21.15.27:0/553264561 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81b80f55e0 0x7f81b8109c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:46:10.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.059+0000 7f81b659c700 1 -- 172.21.15.27:0/553264561 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81b80f55e0 msgr2=0x7f81b8109c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:46:10.059 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.059+0000 7f81b659c700 1 --2- 172.21.15.27:0/553264561 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81b80f55e0 0x7f81b8109c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:10.059 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.059+0000 7f81b659c700 1 -- 172.21.15.27:0/553264561 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a8005040 con 0x7f81b80f5ef0 2021-11-03T17:46:10.059 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.059+0000 7f81b659c700 1 --2- 172.21.15.27:0/553264561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81b80f5ef0 0x7f81b810c150 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f81ac0026d0 tx=0x7f81ac009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:46:10.059 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.059+0000 7f819f7fe700 1 -- 172.21.15.27:0/553264561 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81ac0079c0 con 0x7f81b80f5ef0 2021-11-03T17:46:10.060 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.059+0000 7f819f7fe700 1 -- 172.21.15.27:0/553264561 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f81ac00f040 con 0x7f81b80f5ef0 2021-11-03T17:46:10.060 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.059+0000 7f81bd09f700 1 -- 172.21.15.27:0/553264561 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81b810c760 con 0x7f81b80f5ef0 2021-11-03T17:46:10.060 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.059+0000 7f819f7fe700 1 -- 172.21.15.27:0/553264561 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81ac0148b0 con 0x7f81b80f5ef0 2021-11-03T17:46:10.060 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.059+0000 7f81bd09f700 1 -- 172.21.15.27:0/553264561 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81b810ccf0 con 0x7f81b80f5ef0 2021-11-03T17:46:10.061 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.060+0000 7f81bd09f700 1 -- 172.21.15.27:0/553264561 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81b8045ad0 con 0x7f81b80f5ef0 2021-11-03T17:46:10.061 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.061+0000 7f819f7fe700 1 -- 172.21.15.27:0/553264561 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f81ac007b20 con 0x7f81b80f5ef0 2021-11-03T17:46:10.061 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.061+0000 7f819f7fe700 1 --2- 172.21.15.27:0/553264561 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f81a00640b0 0x7f81a0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:46:10.062 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.061+0000 7f819f7fe700 1 -- 172.21.15.27:0/553264561 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f81ac0806f0 con 0x7f81b80f5ef0 2021-11-03T17:46:10.062 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.061+0000 7f81b6d9d700 1 --2- 172.21.15.27:0/553264561 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f81a00640b0 0x7f81a0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:46:10.062 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.061+0000 7f81b6d9d700 1 --2- 172.21.15.27:0/553264561 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f81a00640b0 0x7f81a0066570 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f81a8004630 tx=0x7f81a8009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:46:10.063 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.065+0000 7f819f7fe700 1 -- 172.21.15.27:0/553264561 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f81ac0529f0 con 0x7f81b80f5ef0 2021-11-03T17:46:10.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.239+0000 7f81bd09f700 1 -- 172.21.15.27:0/553264561 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f81b8057f10 con 0x7f81a00640b0 2021-11-03T17:46:10.248 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.254+0000 7f819f7fe700 1 -- 172.21.15.27:0/553264561 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f81b8057f10 con 0x7f81a00640b0 2021-11-03T17:46:10.248 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:46:10.249 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (37m) 7m ago 38m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:46:10.249 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (29m) 7m ago 38m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:46:10.249 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (29m) 6m ago 38m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:46:10.250 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (37m) 7m ago 38m 41.3M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:46:10.250 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (31m) 7m ago 34m 21.1M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:46:10.250 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (34m) 7m ago 34m 3843M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:46:10.250 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (29m) 6m ago 34m 20.4M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:46:10.250 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (31m) 6m ago 34m 22.1M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:46:10.251 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (31m) 7m ago 40m 517M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:46:10.251 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (30m) 6m ago 38m 419M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:46:10.251 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (30m) 7m ago 40m 159M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:46:10.251 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (30m) 6m ago 38m 82.5M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:46:10.252 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (38m) 7m ago 38m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:46:10.252 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (37m) 6m ago 37m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:46:10.252 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (29m) 7m ago 37m 311M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:46:10.252 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (28m) 7m ago 37m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:46:10.252 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (28m) 7m ago 36m 248M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:46:10.253 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (27m) 6m ago 36m 354M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:46:10.253 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (27m) 6m ago 36m 285M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:46:10.253 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (26m) 6m ago 35m 287M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:46:10.253 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (37m) 7m ago 38m 50.5M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:46:10.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.256+0000 7f81bd09f700 1 -- 172.21.15.27:0/553264561 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f81a00640b0 msgr2=0x7f81a0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:46:10.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.256+0000 7f81bd09f700 1 --2- 172.21.15.27:0/553264561 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f81a00640b0 0x7f81a0066570 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f81a8004630 tx=0x7f81a8009b30).stop 2021-11-03T17:46:10.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.256+0000 7f81bd09f700 1 -- 172.21.15.27:0/553264561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81b80f5ef0 msgr2=0x7f81b810c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:46:10.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.256+0000 7f81bd09f700 1 --2- 172.21.15.27:0/553264561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81b80f5ef0 0x7f81b810c150 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f81ac0026d0 tx=0x7f81ac009de0).stop 2021-11-03T17:46:10.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.256+0000 7f81bd09f700 1 -- 172.21.15.27:0/553264561 shutdown_connections 2021-11-03T17:46:10.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.256+0000 7f81bd09f700 1 --2- 172.21.15.27:0/553264561 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f81a00640b0 0x7f81a0066570 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:10.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.256+0000 7f81bd09f700 1 --2- 172.21.15.27:0/553264561 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81b80f55e0 0x7f81b8109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:10.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.256+0000 7f81bd09f700 1 --2- 172.21.15.27:0/553264561 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81b80f5ef0 0x7f81b810c150 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:10.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.256+0000 7f81bd09f700 1 -- 172.21.15.27:0/553264561 >> 172.21.15.27:0/553264561 conn(0x7f81b80f0de0 msgr2=0x7f81b80fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:46:10.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.256+0000 7f81bd09f700 1 -- 172.21.15.27:0/553264561 shutdown_connections 2021-11-03T17:46:10.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.256+0000 7f81bd09f700 1 -- 172.21.15.27:0/553264561 wait complete. 2021-11-03T17:46:10.352 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.355+0000 7f9c7143b700 1 -- 172.21.15.27:0/3728049871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c6c060310 msgr2=0x7f9c6c0f9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:46:10.353 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.355+0000 7f9c7143b700 1 --2- 172.21.15.27:0/3728049871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c6c060310 0x7f9c6c0f9120 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7f9c60004660 tx=0x7f9c60009b30).stop 2021-11-03T17:46:10.353 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.355+0000 7f9c7143b700 1 -- 172.21.15.27:0/3728049871 shutdown_connections 2021-11-03T17:46:10.353 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.355+0000 7f9c7143b700 1 --2- 172.21.15.27:0/3728049871 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c6c0f96a0 0x7f9c6c0fbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:10.353 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.355+0000 7f9c7143b700 1 --2- 172.21.15.27:0/3728049871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c6c060310 0x7f9c6c0f9120 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:10.354 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.355+0000 7f9c7143b700 1 -- 172.21.15.27:0/3728049871 >> 172.21.15.27:0/3728049871 conn(0x7f9c6c0f0ba0 msgr2=0x7f9c6c0f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:46:10.354 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.355+0000 7f9c7143b700 1 -- 172.21.15.27:0/3728049871 shutdown_connections 2021-11-03T17:46:10.354 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.355+0000 7f9c7143b700 1 -- 172.21.15.27:0/3728049871 wait complete. 2021-11-03T17:46:10.355 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.356+0000 7f9c7143b700 1 Processor -- start 2021-11-03T17:46:10.355 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.356+0000 7f9c7143b700 1 -- start start 2021-11-03T17:46:10.355 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.356+0000 7f9c7143b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c6c060310 0x7f9c6c106da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:46:10.355 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.356+0000 7f9c7143b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c6c0f96a0 0x7f9c6c107290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:46:10.356 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.356+0000 7f9c7143b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6c107890 con 0x7f9c6c060310 2021-11-03T17:46:10.356 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.356+0000 7f9c7143b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6c1080d0 con 0x7f9c6c0f96a0 2021-11-03T17:46:10.356 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.356+0000 7f9c6bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c6c060310 0x7f9c6c106da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:46:10.356 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.356+0000 7f9c6bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c6c060310 0x7f9c6c106da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59302/0 (socket says 172.21.15.27:59302) 2021-11-03T17:46:10.357 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.356+0000 7f9c6bfff700 1 -- 172.21.15.27:0/3536625983 learned_addr learned my addr 172.21.15.27:0/3536625983 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:46:10.357 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.357+0000 7f9c6b7fe700 1 --2- 172.21.15.27:0/3536625983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c6c0f96a0 0x7f9c6c107290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:46:10.358 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.357+0000 7f9c6bfff700 1 -- 172.21.15.27:0/3536625983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c6c0f96a0 msgr2=0x7f9c6c107290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:46:10.358 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.357+0000 7f9c6bfff700 1 --2- 172.21.15.27:0/3536625983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c6c0f96a0 0x7f9c6c107290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:10.358 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.357+0000 7f9c6bfff700 1 -- 172.21.15.27:0/3536625983 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c60005040 con 0x7f9c6c060310 2021-11-03T17:46:10.358 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.357+0000 7f9c6bfff700 1 --2- 172.21.15.27:0/3536625983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c6c060310 0x7f9c6c106da0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f9c60009990 tx=0x7f9c60009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:46:10.359 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.357+0000 7f9c68ff9700 1 -- 172.21.15.27:0/3536625983 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c60018070 con 0x7f9c6c060310 2021-11-03T17:46:10.359 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.357+0000 7f9c7143b700 1 -- 172.21.15.27:0/3536625983 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c6c108350 con 0x7f9c6c060310 2021-11-03T17:46:10.359 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.357+0000 7f9c68ff9700 1 -- 172.21.15.27:0/3536625983 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f9c60007ca0 con 0x7f9c6c060310 2021-11-03T17:46:10.360 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.357+0000 7f9c68ff9700 1 -- 172.21.15.27:0/3536625983 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c6001c620 con 0x7f9c6c060310 2021-11-03T17:46:10.360 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.357+0000 7f9c7143b700 1 -- 172.21.15.27:0/3536625983 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c6c108830 con 0x7f9c6c060310 2021-11-03T17:46:10.360 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.358+0000 7f9c7143b700 1 -- 172.21.15.27:0/3536625983 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c6c059100 con 0x7f9c6c060310 2021-11-03T17:46:10.360 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.359+0000 7f9c68ff9700 1 -- 172.21.15.27:0/3536625983 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f9c6001d960 con 0x7f9c6c060310 2021-11-03T17:46:10.361 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.359+0000 7f9c68ff9700 1 --2- 172.21.15.27:0/3536625983 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9c54064100 0x7f9c540665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:46:10.362 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.359+0000 7f9c68ff9700 1 -- 172.21.15.27:0/3536625983 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f9c600824d0 con 0x7f9c6c060310 2021-11-03T17:46:10.362 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.359+0000 7f9c6b7fe700 1 --2- 172.21.15.27:0/3536625983 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9c54064100 0x7f9c540665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:46:10.362 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.359+0000 7f9c6b7fe700 1 --2- 172.21.15.27:0/3536625983 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9c54064100 0x7f9c540665c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f9c6c106b30 tx=0x7f9c5c009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:46:10.362 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.363+0000 7f9c68ff9700 1 -- 172.21.15.27:0/3536625983 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f9c60054890 con 0x7f9c6c060310 2021-11-03T17:46:10.438 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:10 smithi027 ceph-mon[97244]: pgmap v945: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:10.439 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1812.402283 secs 2021-11-03T17:46:10.441 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:10 smithi115 ceph-mon[65453]: pgmap v945: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:10.441 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1812.402283 secs 2021-11-03T17:46:10.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.595+0000 7f9c7143b700 1 -- 172.21.15.27:0/3536625983 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9c6c0fee20 con 0x7f9c6c060310 2021-11-03T17:46:10.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.596+0000 7f9c68ff9700 1 -- 172.21.15.27:0/3536625983 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f9c60054480 con 0x7f9c6c060310 2021-11-03T17:46:10.590 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:46:10.590 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:46:10.591 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:46:10.591 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:46:10.591 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:46:10.591 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:46:10.592 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:46:10.592 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:46:10.592 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:46:10.593 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:46:10.593 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:46:10.593 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:46:10.594 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:46:10.594 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:46:10.594 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:46:10.594 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:46:10.595 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:46:10.595 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:46:10.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.598+0000 7f9c7143b700 1 -- 172.21.15.27:0/3536625983 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9c54064100 msgr2=0x7f9c540665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:46:10.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.598+0000 7f9c7143b700 1 --2- 172.21.15.27:0/3536625983 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9c54064100 0x7f9c540665c0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f9c6c106b30 tx=0x7f9c5c009040).stop 2021-11-03T17:46:10.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.598+0000 7f9c7143b700 1 -- 172.21.15.27:0/3536625983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c6c060310 msgr2=0x7f9c6c106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:46:10.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.598+0000 7f9c7143b700 1 --2- 172.21.15.27:0/3536625983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c6c060310 0x7f9c6c106da0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f9c60009990 tx=0x7f9c60009b30).stop 2021-11-03T17:46:10.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.598+0000 7f9c7143b700 1 -- 172.21.15.27:0/3536625983 shutdown_connections 2021-11-03T17:46:10.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.598+0000 7f9c7143b700 1 --2- 172.21.15.27:0/3536625983 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9c54064100 0x7f9c540665c0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:10.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.598+0000 7f9c7143b700 1 --2- 172.21.15.27:0/3536625983 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c6c060310 0x7f9c6c106da0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:10.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.598+0000 7f9c7143b700 1 --2- 172.21.15.27:0/3536625983 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c6c0f96a0 0x7f9c6c107290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:10.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.598+0000 7f9c7143b700 1 -- 172.21.15.27:0/3536625983 >> 172.21.15.27:0/3536625983 conn(0x7f9c6c0f0ba0 msgr2=0x7f9c6c0f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:46:10.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.598+0000 7f9c7143b700 1 -- 172.21.15.27:0/3536625983 shutdown_connections 2021-11-03T17:46:10.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.598+0000 7f9c7143b700 1 -- 172.21.15.27:0/3536625983 wait complete. 2021-11-03T17:46:10.704 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.709+0000 7ffb447a6700 1 -- 172.21.15.27:0/1144355352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb3c0f7980 msgr2=0x7ffb3c0f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:46:10.704 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.709+0000 7ffb447a6700 1 --2- 172.21.15.27:0/1144355352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb3c0f7980 0x7ffb3c0f7de0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7ffb2c004660 tx=0x7ffb2c009b30).stop 2021-11-03T17:46:10.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.709+0000 7ffb447a6700 1 -- 172.21.15.27:0/1144355352 shutdown_connections 2021-11-03T17:46:10.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.709+0000 7ffb447a6700 1 --2- 172.21.15.27:0/1144355352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb3c0f7980 0x7ffb3c0f7de0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:10.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.709+0000 7ffb447a6700 1 --2- 172.21.15.27:0/1144355352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb3c0f66e0 0x7ffb3c0f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:10.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.709+0000 7ffb447a6700 1 -- 172.21.15.27:0/1144355352 >> 172.21.15.27:0/1144355352 conn(0x7ffb3c0f1e40 msgr2=0x7ffb3c0f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:46:10.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.709+0000 7ffb447a6700 1 -- 172.21.15.27:0/1144355352 shutdown_connections 2021-11-03T17:46:10.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.709+0000 7ffb447a6700 1 -- 172.21.15.27:0/1144355352 wait complete. 2021-11-03T17:46:10.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.710+0000 7ffb447a6700 1 Processor -- start 2021-11-03T17:46:10.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.710+0000 7ffb447a6700 1 -- start start 2021-11-03T17:46:10.707 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.710+0000 7ffb447a6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb3c0f66e0 0x7ffb3c109110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:46:10.707 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.710+0000 7ffb447a6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb3c0f7980 0x7ffb3c109600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:46:10.707 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.710+0000 7ffb447a6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3c109bb0 con 0x7ffb3c0f66e0 2021-11-03T17:46:10.707 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.710+0000 7ffb447a6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3c109cf0 con 0x7ffb3c0f7980 2021-11-03T17:46:10.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.710+0000 7ffb42542700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb3c0f66e0 0x7ffb3c109110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:46:10.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.710+0000 7ffb42542700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb3c0f66e0 0x7ffb3c109110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59310/0 (socket says 172.21.15.27:59310) 2021-11-03T17:46:10.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.710+0000 7ffb42542700 1 -- 172.21.15.27:0/1098141355 learned_addr learned my addr 172.21.15.27:0/1098141355 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:46:10.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.710+0000 7ffb41d41700 1 --2- 172.21.15.27:0/1098141355 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb3c0f7980 0x7ffb3c109600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:46:10.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.710+0000 7ffb42542700 1 -- 172.21.15.27:0/1098141355 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb3c0f7980 msgr2=0x7ffb3c109600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:46:10.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.710+0000 7ffb42542700 1 --2- 172.21.15.27:0/1098141355 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb3c0f7980 0x7ffb3c109600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:10.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.710+0000 7ffb42542700 1 -- 172.21.15.27:0/1098141355 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb2c005040 con 0x7ffb3c0f66e0 2021-11-03T17:46:10.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.710+0000 7ffb42542700 1 --2- 172.21.15.27:0/1098141355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb3c0f66e0 0x7ffb3c109110 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7ffb38002700 tx=0x7ffb38009e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:46:10.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.711+0000 7ffb32ffd700 1 -- 172.21.15.27:0/1098141355 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb380079c0 con 0x7ffb3c0f66e0 2021-11-03T17:46:10.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.712+0000 7ffb447a6700 1 -- 172.21.15.27:0/1098141355 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb3c10a690 con 0x7ffb3c0f66e0 2021-11-03T17:46:10.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.712+0000 7ffb32ffd700 1 -- 172.21.15.27:0/1098141355 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7ffb3800f040 con 0x7ffb3c0f66e0 2021-11-03T17:46:10.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.712+0000 7ffb32ffd700 1 -- 172.21.15.27:0/1098141355 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb38014850 con 0x7ffb3c0f66e0 2021-11-03T17:46:10.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.712+0000 7ffb447a6700 1 -- 172.21.15.27:0/1098141355 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb3c10ac50 con 0x7ffb3c0f66e0 2021-11-03T17:46:10.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.713+0000 7ffb447a6700 1 -- 172.21.15.27:0/1098141355 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb3c045ad0 con 0x7ffb3c0f66e0 2021-11-03T17:46:10.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.713+0000 7ffb32ffd700 1 -- 172.21.15.27:0/1098141355 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7ffb3801b060 con 0x7ffb3c0f66e0 2021-11-03T17:46:10.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.713+0000 7ffb32ffd700 1 --2- 172.21.15.27:0/1098141355 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffb28064090 0x7ffb28066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:46:10.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.713+0000 7ffb32ffd700 1 -- 172.21.15.27:0/1098141355 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7ffb38080680 con 0x7ffb3c0f66e0 2021-11-03T17:46:10.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.714+0000 7ffb41d41700 1 --2- 172.21.15.27:0/1098141355 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffb28064090 0x7ffb28066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:46:10.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.714+0000 7ffb41d41700 1 --2- 172.21.15.27:0/1098141355 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffb28064090 0x7ffb28066550 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7ffb2c004e30 tx=0x7ffb2c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:46:10.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.717+0000 7ffb32ffd700 1 -- 172.21.15.27:0/1098141355 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7ffb38052ad0 con 0x7ffb3c0f66e0 2021-11-03T17:46:10.914 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.920+0000 7ffb447a6700 1 -- 172.21.15.27:0/1098141355 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ffb3c0ff020 con 0x7ffb3c0f66e0 2021-11-03T17:46:10.915 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.921+0000 7ffb32ffd700 1 -- 172.21.15.27:0/1098141355 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7ffb38019710 con 0x7ffb3c0f66e0 2021-11-03T17:46:10.915 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:46:10.915 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:46:10.915 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:46:10.916 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:46:10.916 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:46:10.916 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:46:10.916 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:46:10.917 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:46:10.917 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:46:10.918 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:46:10.918 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:46:10.920 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:46:10.920 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:46:10.921 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:46:10.921 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:46:10.922 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:46:10.922 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:46:10.923 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:46:10.923 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:46:10.924 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:46:10.924 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:46:10.925 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:46:10.925 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:46:10.926 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:46:10.926 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:46:10.926 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:46:10.927 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:46:10.927 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:46:10.928 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:46:10.928 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:46:10.929 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:46:10.929 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:46:10.929 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:46:10.930 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:46:10.930 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:46:10.931 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:46:10.931 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:46:10.932 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:46:10.932 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:46:10.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.922+0000 7ffb447a6700 1 -- 172.21.15.27:0/1098141355 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffb28064090 msgr2=0x7ffb28066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:46:10.935 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.922+0000 7ffb447a6700 1 --2- 172.21.15.27:0/1098141355 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffb28064090 0x7ffb28066550 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7ffb2c004e30 tx=0x7ffb2c009b30).stop 2021-11-03T17:46:10.935 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.922+0000 7ffb447a6700 1 -- 172.21.15.27:0/1098141355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb3c0f66e0 msgr2=0x7ffb3c109110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:46:10.936 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.922+0000 7ffb447a6700 1 --2- 172.21.15.27:0/1098141355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb3c0f66e0 0x7ffb3c109110 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7ffb38002700 tx=0x7ffb38009e00).stop 2021-11-03T17:46:10.936 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.922+0000 7ffb447a6700 1 -- 172.21.15.27:0/1098141355 shutdown_connections 2021-11-03T17:46:10.936 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.922+0000 7ffb447a6700 1 --2- 172.21.15.27:0/1098141355 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffb28064090 0x7ffb28066550 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:10.937 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.922+0000 7ffb447a6700 1 --2- 172.21.15.27:0/1098141355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb3c0f66e0 0x7ffb3c109110 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:10.937 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.922+0000 7ffb447a6700 1 --2- 172.21.15.27:0/1098141355 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb3c0f7980 0x7ffb3c109600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:10.938 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.922+0000 7ffb447a6700 1 -- 172.21.15.27:0/1098141355 >> 172.21.15.27:0/1098141355 conn(0x7ffb3c0f1e40 msgr2=0x7ffb3c0f93c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:46:10.938 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.922+0000 7ffb447a6700 1 -- 172.21.15.27:0/1098141355 shutdown_connections 2021-11-03T17:46:10.939 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:10.922+0000 7ffb447a6700 1 -- 172.21.15.27:0/1098141355 wait complete. 2021-11-03T17:46:10.939 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:46:11.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:11 smithi115 ceph-mon[65453]: from='client.35080 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:46:11.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:11 smithi115 ceph-mon[65453]: from='client.35084 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:46:11.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:46:11.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:11 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3536625983' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:11.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:46:11.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:46:11.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:46:11.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:46:11.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:11.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:11.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:11.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:11.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:11 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1098141355' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:46:11.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:11 smithi027 ceph-mon[97244]: from='client.35080 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:46:11.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:11 smithi027 ceph-mon[97244]: from='client.35084 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:46:11.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:46:11.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:11 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3536625983' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:11.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:46:11.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:46:11.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:46:11.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:46:11.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:11.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:11.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:11.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:11.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:11 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1098141355' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:46:12.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:12 smithi115 ceph-mon[65453]: pgmap v946: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:12.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:12 smithi115 ceph-mon[65453]: poll took 50143.404 ms: 1 events 2021-11-03T17:46:12.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:12 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:46:12.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:12 smithi027 ceph-mon[97244]: pgmap v946: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:12.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:12 smithi027 ceph-mon[97244]: poll took 50143.404 ms: 1 events 2021-11-03T17:46:12.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:12 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:46:14.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:14 smithi115 ceph-mon[65453]: pgmap v947: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:14.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:14 smithi027 ceph-mon[97244]: pgmap v947: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:15.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1817.402395 secs 2021-11-03T17:46:15.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1817.402395 secs 2021-11-03T17:46:16.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:16 smithi115 ceph-mon[65453]: pgmap v948: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:16.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:16 smithi027 ceph-mon[97244]: pgmap v948: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:18.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:18 smithi115 ceph-mon[65453]: pgmap v949: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:18.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:18 smithi027 ceph-mon[97244]: pgmap v949: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:20.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:20 smithi115 ceph-mon[65453]: pgmap v950: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:20.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1822.402539 secs 2021-11-03T17:46:20.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:20 smithi027 ceph-mon[97244]: pgmap v950: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:20.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1822.402539 secs 2021-11-03T17:46:21.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:46:21.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2021-11-03T17:46:21.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"}]: dispatch 2021-11-03T17:46:21.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:46:21.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:46:21.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:21.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:21.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:21.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:46:21.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2021-11-03T17:46:21.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"}]: dispatch 2021-11-03T17:46:21.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:46:21.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:46:21.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:21.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:21.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:22.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:22 smithi115 ceph-mon[65453]: pgmap v951: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:22.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:22 smithi115 ceph-mon[65453]: poll took 10040.296 ms: 1 events 2021-11-03T17:46:22.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:22.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:22 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:46:22.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:22 smithi027 ceph-mon[97244]: pgmap v951: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:22.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:22 smithi027 ceph-mon[97244]: poll took 10040.296 ms: 1 events 2021-11-03T17:46:22.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:22.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:22 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:46:24.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:24 smithi115 ceph-mon[65453]: pgmap v952: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:24.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:24 smithi027 ceph-mon[97244]: pgmap v952: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:25.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1827.402618 secs 2021-11-03T17:46:25.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1827.402618 secs 2021-11-03T17:46:26.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:26 smithi115 ceph-mon[65453]: pgmap v953: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:26.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:26 smithi027 ceph-mon[97244]: pgmap v953: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:28.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:28 smithi115 ceph-mon[65453]: pgmap v954: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:28.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:28 smithi027 ceph-mon[97244]: pgmap v954: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:30.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:30 smithi115 ceph-mon[65453]: pgmap v955: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:30.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1832.402760 secs 2021-11-03T17:46:30.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:30 smithi027 ceph-mon[97244]: pgmap v955: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:30.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1832.402760 secs 2021-11-03T17:46:31.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:46:31.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:46:31.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:46:31.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:31.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:46:31.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:46:31.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:46:31.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:32.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:32 smithi115 ceph-mon[65453]: pgmap v956: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:32.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:32.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:32.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:32.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:32 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:46:32.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:32 smithi027 ceph-mon[97244]: pgmap v956: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:32.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:32.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:32.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:32.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:32 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:46:34.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:34 smithi115 ceph-mon[65453]: pgmap v957: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:34.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:34 smithi027 ceph-mon[97244]: pgmap v957: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:35.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1837.402886 secs 2021-11-03T17:46:35.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1837.402886 secs 2021-11-03T17:46:36.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:36 smithi115 ceph-mon[65453]: pgmap v958: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:36.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:36 smithi027 ceph-mon[97244]: pgmap v958: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:38.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:38 smithi115 ceph-mon[65453]: pgmap v959: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:38.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:38 smithi027 ceph-mon[97244]: pgmap v959: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:40.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:40 smithi115 ceph-mon[65453]: pgmap v960: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:40.500 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1842.403029 secs 2021-11-03T17:46:40.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:40 smithi027 ceph-mon[97244]: pgmap v960: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:40.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1842.403029 secs 2021-11-03T17:46:41.032 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.037+0000 7f5358920700 1 -- 172.21.15.27:0/1940205132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5354060310 msgr2=0x7f53540f9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:46:41.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.037+0000 7f5358920700 1 --2- 172.21.15.27:0/1940205132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5354060310 0x7f53540f9120 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f5344004660 tx=0x7f5344009b30).stop 2021-11-03T17:46:41.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.037+0000 7f5358920700 1 -- 172.21.15.27:0/1940205132 shutdown_connections 2021-11-03T17:46:41.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.037+0000 7f5358920700 1 --2- 172.21.15.27:0/1940205132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53540f96a0 0x7f53540fbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:41.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.037+0000 7f5358920700 1 --2- 172.21.15.27:0/1940205132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5354060310 0x7f53540f9120 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:41.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.037+0000 7f5358920700 1 -- 172.21.15.27:0/1940205132 >> 172.21.15.27:0/1940205132 conn(0x7f53540f0ba0 msgr2=0x7f53540f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:46:41.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.037+0000 7f5358920700 1 -- 172.21.15.27:0/1940205132 shutdown_connections 2021-11-03T17:46:41.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.037+0000 7f5358920700 1 -- 172.21.15.27:0/1940205132 wait complete. 2021-11-03T17:46:41.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.037+0000 7f5358920700 1 Processor -- start 2021-11-03T17:46:41.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.038+0000 7f5358920700 1 -- start start 2021-11-03T17:46:41.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.038+0000 7f5358920700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5354060310 0x7f5354106da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:46:41.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.038+0000 7f5358920700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53540f96a0 0x7f5354107290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:46:41.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.038+0000 7f5358920700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5354107890 con 0x7f5354060310 2021-11-03T17:46:41.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.038+0000 7f5358920700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53541080d0 con 0x7f53540f96a0 2021-11-03T17:46:41.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.038+0000 7f53537fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5354060310 0x7f5354106da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:46:41.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.038+0000 7f53537fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5354060310 0x7f5354106da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59328/0 (socket says 172.21.15.27:59328) 2021-11-03T17:46:41.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.038+0000 7f53537fe700 1 -- 172.21.15.27:0/3484294385 learned_addr learned my addr 172.21.15.27:0/3484294385 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:46:41.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.038+0000 7f5352ffd700 1 --2- 172.21.15.27:0/3484294385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53540f96a0 0x7f5354107290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:46:41.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.038+0000 7f53537fe700 1 -- 172.21.15.27:0/3484294385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53540f96a0 msgr2=0x7f5354107290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:46:41.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.038+0000 7f53537fe700 1 --2- 172.21.15.27:0/3484294385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53540f96a0 0x7f5354107290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:41.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.038+0000 7f53537fe700 1 -- 172.21.15.27:0/3484294385 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5344005040 con 0x7f5354060310 2021-11-03T17:46:41.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.038+0000 7f53537fe700 1 --2- 172.21.15.27:0/3484294385 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5354060310 0x7f5354106da0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f5344009990 tx=0x7f5344009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:46:41.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.038+0000 7f533bfff700 1 -- 172.21.15.27:0/3484294385 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5344018070 con 0x7f5354060310 2021-11-03T17:46:41.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.039+0000 7f5358920700 1 -- 172.21.15.27:0/3484294385 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5354108350 con 0x7f5354060310 2021-11-03T17:46:41.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.039+0000 7f533bfff700 1 -- 172.21.15.27:0/3484294385 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f5344007ba0 con 0x7f5354060310 2021-11-03T17:46:41.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.039+0000 7f533bfff700 1 -- 172.21.15.27:0/3484294385 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f534401c3c0 con 0x7f5354060310 2021-11-03T17:46:41.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.039+0000 7f5358920700 1 -- 172.21.15.27:0/3484294385 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5354108830 con 0x7f5354060310 2021-11-03T17:46:41.046 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.040+0000 7f5358920700 1 -- 172.21.15.27:0/3484294385 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5354059100 con 0x7f5354060310 2021-11-03T17:46:41.046 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.040+0000 7f533bfff700 1 -- 172.21.15.27:0/3484294385 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f534401d420 con 0x7f5354060310 2021-11-03T17:46:41.047 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.040+0000 7f533bfff700 1 --2- 172.21.15.27:0/3484294385 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f533c064100 0x7f533c0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:46:41.047 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.041+0000 7f533bfff700 1 -- 172.21.15.27:0/3484294385 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f5344082470 con 0x7f5354060310 2021-11-03T17:46:41.048 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.041+0000 7f5352ffd700 1 --2- 172.21.15.27:0/3484294385 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f533c064100 0x7f533c0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:46:41.048 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.041+0000 7f5352ffd700 1 --2- 172.21.15.27:0/3484294385 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f533c064100 0x7f533c0665c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f5354106b30 tx=0x7f534800b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:46:41.048 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.045+0000 7f533bfff700 1 -- 172.21.15.27:0/3484294385 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f5344054830 con 0x7f5354060310 2021-11-03T17:46:41.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.227+0000 7f5358920700 1 -- 172.21.15.27:0/3484294385 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f53540fee60 con 0x7f533c064100 2021-11-03T17:46:41.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.228+0000 7f533bfff700 1 -- 172.21.15.27:0/3484294385 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f53540fee60 con 0x7f533c064100 2021-11-03T17:46:41.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.230+0000 7f5358920700 1 -- 172.21.15.27:0/3484294385 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f533c064100 msgr2=0x7f533c0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:46:41.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.230+0000 7f5358920700 1 --2- 172.21.15.27:0/3484294385 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f533c064100 0x7f533c0665c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7f5354106b30 tx=0x7f534800b040).stop 2021-11-03T17:46:41.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.230+0000 7f5358920700 1 -- 172.21.15.27:0/3484294385 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5354060310 msgr2=0x7f5354106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:46:41.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.230+0000 7f5358920700 1 --2- 172.21.15.27:0/3484294385 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5354060310 0x7f5354106da0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7f5344009990 tx=0x7f5344009b30).stop 2021-11-03T17:46:41.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.230+0000 7f5358920700 1 -- 172.21.15.27:0/3484294385 shutdown_connections 2021-11-03T17:46:41.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.230+0000 7f5358920700 1 --2- 172.21.15.27:0/3484294385 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f533c064100 0x7f533c0665c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:41.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.230+0000 7f5358920700 1 --2- 172.21.15.27:0/3484294385 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5354060310 0x7f5354106da0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:41.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.230+0000 7f5358920700 1 --2- 172.21.15.27:0/3484294385 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53540f96a0 0x7f5354107290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:41.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.230+0000 7f5358920700 1 -- 172.21.15.27:0/3484294385 >> 172.21.15.27:0/3484294385 conn(0x7f53540f0ba0 msgr2=0x7f53540f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:46:41.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.230+0000 7f5358920700 1 -- 172.21.15.27:0/3484294385 shutdown_connections 2021-11-03T17:46:41.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.231+0000 7f5358920700 1 -- 172.21.15.27:0/3484294385 wait complete. 2021-11-03T17:46:41.234 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:46:41.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.330+0000 7fcf7d92c700 1 -- 172.21.15.27:0/1791835277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf780f96a0 msgr2=0x7fcf780fbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:46:41.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.330+0000 7fcf7d92c700 1 --2- 172.21.15.27:0/1791835277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf780f96a0 0x7fcf780fbaf0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7fcf64004660 tx=0x7fcf64009b30).stop 2021-11-03T17:46:41.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.330+0000 7fcf7d92c700 1 -- 172.21.15.27:0/1791835277 shutdown_connections 2021-11-03T17:46:41.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.330+0000 7fcf7d92c700 1 --2- 172.21.15.27:0/1791835277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf780f96a0 0x7fcf780fbaf0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:41.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.330+0000 7fcf7d92c700 1 --2- 172.21.15.27:0/1791835277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf78060310 0x7fcf780f9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:41.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.330+0000 7fcf7d92c700 1 -- 172.21.15.27:0/1791835277 >> 172.21.15.27:0/1791835277 conn(0x7fcf780f0ba0 msgr2=0x7fcf780f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:46:41.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.331+0000 7fcf7d92c700 1 -- 172.21.15.27:0/1791835277 shutdown_connections 2021-11-03T17:46:41.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.331+0000 7fcf7d92c700 1 -- 172.21.15.27:0/1791835277 wait complete. 2021-11-03T17:46:41.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.331+0000 7fcf7d92c700 1 Processor -- start 2021-11-03T17:46:41.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.331+0000 7fcf7d92c700 1 -- start start 2021-11-03T17:46:41.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.331+0000 7fcf7d92c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf78060310 0x7fcf78106d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:46:41.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.331+0000 7fcf7d92c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf780f96a0 0x7fcf78107280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:46:41.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.331+0000 7fcf7d92c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf78107880 con 0x7fcf78060310 2021-11-03T17:46:41.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.331+0000 7fcf7d92c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf781080c0 con 0x7fcf780f96a0 2021-11-03T17:46:41.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.332+0000 7fcf7c92a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf78060310 0x7fcf78106d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:46:41.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.332+0000 7fcf7c92a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf78060310 0x7fcf78106d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59338/0 (socket says 172.21.15.27:59338) 2021-11-03T17:46:41.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.332+0000 7fcf7c92a700 1 -- 172.21.15.27:0/2618935990 learned_addr learned my addr 172.21.15.27:0/2618935990 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:46:41.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.332+0000 7fcf77fff700 1 --2- 172.21.15.27:0/2618935990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf780f96a0 0x7fcf78107280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:46:41.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.332+0000 7fcf7c92a700 1 -- 172.21.15.27:0/2618935990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf780f96a0 msgr2=0x7fcf78107280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:46:41.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.332+0000 7fcf7c92a700 1 --2- 172.21.15.27:0/2618935990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf780f96a0 0x7fcf78107280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:41.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.332+0000 7fcf7c92a700 1 -- 172.21.15.27:0/2618935990 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf64005040 con 0x7fcf78060310 2021-11-03T17:46:41.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.332+0000 7fcf7c92a700 1 --2- 172.21.15.27:0/2618935990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf78060310 0x7fcf78106d90 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7fcf6c00c790 tx=0x7fcf6c007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:46:41.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.332+0000 7fcf757fa700 1 -- 172.21.15.27:0/2618935990 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf6c002590 con 0x7fcf78060310 2021-11-03T17:46:41.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.332+0000 7fcf757fa700 1 -- 172.21.15.27:0/2618935990 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fcf6c002ba0 con 0x7fcf78060310 2021-11-03T17:46:41.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.332+0000 7fcf7d92c700 1 -- 172.21.15.27:0/2618935990 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf781083a0 con 0x7fcf78060310 2021-11-03T17:46:41.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.332+0000 7fcf757fa700 1 -- 172.21.15.27:0/2618935990 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf6c013d90 con 0x7fcf78060310 2021-11-03T17:46:41.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.332+0000 7fcf7d92c700 1 -- 172.21.15.27:0/2618935990 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf781088e0 con 0x7fcf78060310 2021-11-03T17:46:41.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.333+0000 7fcf7d92c700 1 -- 172.21.15.27:0/2618935990 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf78059100 con 0x7fcf78060310 2021-11-03T17:46:41.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.334+0000 7fcf757fa700 1 -- 172.21.15.27:0/2618935990 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fcf6c01a060 con 0x7fcf78060310 2021-11-03T17:46:41.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.334+0000 7fcf757fa700 1 --2- 172.21.15.27:0/2618935990 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcf68064100 0x7fcf680665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:46:41.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.334+0000 7fcf757fa700 1 -- 172.21.15.27:0/2618935990 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fcf6c080c10 con 0x7fcf78060310 2021-11-03T17:46:41.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.334+0000 7fcf77fff700 1 --2- 172.21.15.27:0/2618935990 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcf68064100 0x7fcf680665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:46:41.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.335+0000 7fcf77fff700 1 --2- 172.21.15.27:0/2618935990 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcf68064100 0x7fcf680665c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7fcf78106b20 tx=0x7fcf64009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:46:41.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.338+0000 7fcf757fa700 1 -- 172.21.15.27:0/2618935990 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fcf6c052d40 con 0x7fcf78060310 2021-11-03T17:46:41.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:46:41.506 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.512+0000 7fcf7d92c700 1 -- 172.21.15.27:0/2618935990 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf780fee10 con 0x7fcf68064100 2021-11-03T17:46:41.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:46:41.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.526+0000 7fcf757fa700 1 -- 172.21.15.27:0/2618935990 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fcf780fee10 con 0x7fcf68064100 2021-11-03T17:46:41.521 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:46:41.521 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (38m) 8m ago 39m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:46:41.521 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (29m) 8m ago 39m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:46:41.521 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (29m) 6m ago 38m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:46:41.522 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (38m) 8m ago 39m 41.3M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:46:41.522 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (31m) 8m ago 34m 21.1M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:46:41.522 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (34m) 8m ago 34m 3843M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:46:41.522 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (30m) 6m ago 34m 20.4M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:46:41.523 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (32m) 6m ago 34m 22.1M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:46:41.523 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (31m) 8m ago 40m 517M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:46:41.523 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (31m) 6m ago 38m 419M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:46:41.523 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (30m) 8m ago 40m 159M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:46:41.524 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (30m) 6m ago 38m 82.5M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:46:41.524 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (39m) 8m ago 39m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:46:41.524 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (38m) 6m ago 38m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:46:41.524 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (29m) 8m ago 37m 311M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:46:41.525 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (29m) 8m ago 37m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:46:41.525 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (28m) 8m ago 37m 248M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:46:41.525 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (28m) 6m ago 36m 354M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:46:41.525 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (27m) 6m ago 36m 285M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:46:41.526 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (27m) 6m ago 36m 287M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:46:41.526 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (37m) 8m ago 38m 50.5M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:46:41.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.528+0000 7fcf7d92c700 1 -- 172.21.15.27:0/2618935990 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcf68064100 msgr2=0x7fcf680665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:46:41.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.528+0000 7fcf7d92c700 1 --2- 172.21.15.27:0/2618935990 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcf68064100 0x7fcf680665c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7fcf78106b20 tx=0x7fcf64009b30).stop 2021-11-03T17:46:41.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.528+0000 7fcf7d92c700 1 -- 172.21.15.27:0/2618935990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf78060310 msgr2=0x7fcf78106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:46:41.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.528+0000 7fcf7d92c700 1 --2- 172.21.15.27:0/2618935990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf78060310 0x7fcf78106d90 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7fcf6c00c790 tx=0x7fcf6c007d70).stop 2021-11-03T17:46:41.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.528+0000 7fcf7d92c700 1 -- 172.21.15.27:0/2618935990 shutdown_connections 2021-11-03T17:46:41.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.528+0000 7fcf7d92c700 1 --2- 172.21.15.27:0/2618935990 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcf68064100 0x7fcf680665c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:41.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.528+0000 7fcf7d92c700 1 --2- 172.21.15.27:0/2618935990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf78060310 0x7fcf78106d90 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:41.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.528+0000 7fcf7d92c700 1 --2- 172.21.15.27:0/2618935990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf780f96a0 0x7fcf78107280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:41.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.528+0000 7fcf7d92c700 1 -- 172.21.15.27:0/2618935990 >> 172.21.15.27:0/2618935990 conn(0x7fcf780f0ba0 msgr2=0x7fcf780f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:46:41.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.528+0000 7fcf7d92c700 1 -- 172.21.15.27:0/2618935990 shutdown_connections 2021-11-03T17:46:41.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.528+0000 7fcf7d92c700 1 -- 172.21.15.27:0/2618935990 wait complete. 2021-11-03T17:46:41.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.628+0000 7fc037d12700 1 -- 172.21.15.27:0/1331917694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0300f6ff0 msgr2=0x7fc0300f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:46:41.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.628+0000 7fc037d12700 1 --2- 172.21.15.27:0/1331917694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0300f6ff0 0x7fc0300f7a20 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7fc024004660 tx=0x7fc024009b30).stop 2021-11-03T17:46:41.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.628+0000 7fc037d12700 1 -- 172.21.15.27:0/1331917694 shutdown_connections 2021-11-03T17:46:41.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.628+0000 7fc037d12700 1 --2- 172.21.15.27:0/1331917694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0300f6ff0 0x7fc0300f7a20 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:41.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.628+0000 7fc037d12700 1 --2- 172.21.15.27:0/1331917694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0300f66e0 0x7fc0300f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:41.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.628+0000 7fc037d12700 1 -- 172.21.15.27:0/1331917694 >> 172.21.15.27:0/1331917694 conn(0x7fc0300f1f00 msgr2=0x7fc0300f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:46:41.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.628+0000 7fc037d12700 1 -- 172.21.15.27:0/1331917694 shutdown_connections 2021-11-03T17:46:41.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.628+0000 7fc037d12700 1 -- 172.21.15.27:0/1331917694 wait complete. 2021-11-03T17:46:41.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.629+0000 7fc037d12700 1 Processor -- start 2021-11-03T17:46:41.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.629+0000 7fc037d12700 1 -- start start 2021-11-03T17:46:41.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.629+0000 7fc037d12700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0300f66e0 0x7fc03006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:46:41.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.629+0000 7fc037d12700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0300f6ff0 0x7fc030069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:46:41.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.629+0000 7fc037d12700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03006a380 con 0x7fc0300f66e0 2021-11-03T17:46:41.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.629+0000 7fc037d12700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03006a4c0 con 0x7fc0300f6ff0 2021-11-03T17:46:41.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.629+0000 7fc035aae700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0300f66e0 0x7fc03006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:46:41.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.629+0000 7fc035aae700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0300f66e0 0x7fc03006b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59348/0 (socket says 172.21.15.27:59348) 2021-11-03T17:46:41.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.629+0000 7fc035aae700 1 -- 172.21.15.27:0/3819307714 learned_addr learned my addr 172.21.15.27:0/3819307714 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:46:41.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.629+0000 7fc0352ad700 1 --2- 172.21.15.27:0/3819307714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0300f6ff0 0x7fc030069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:46:41.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.629+0000 7fc035aae700 1 -- 172.21.15.27:0/3819307714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0300f6ff0 msgr2=0x7fc030069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:46:41.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.629+0000 7fc035aae700 1 --2- 172.21.15.27:0/3819307714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0300f6ff0 0x7fc030069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:41.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.629+0000 7fc035aae700 1 -- 172.21.15.27:0/3819307714 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc024005040 con 0x7fc0300f66e0 2021-11-03T17:46:41.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.629+0000 7fc0352ad700 1 --2- 172.21.15.27:0/3819307714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0300f6ff0 0x7fc030069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:46:41.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.629+0000 7fc035aae700 1 --2- 172.21.15.27:0/3819307714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0300f66e0 0x7fc03006b9a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7fc028007fd0 tx=0x7fc028004ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:46:41.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.630+0000 7fc02e7fc700 1 -- 172.21.15.27:0/3819307714 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc02800b5a0 con 0x7fc0300f66e0 2021-11-03T17:46:41.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.630+0000 7fc037d12700 1 -- 172.21.15.27:0/3819307714 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc03006a7a0 con 0x7fc0300f66e0 2021-11-03T17:46:41.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.630+0000 7fc02e7fc700 1 -- 172.21.15.27:0/3819307714 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fc02800f040 con 0x7fc0300f66e0 2021-11-03T17:46:41.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.630+0000 7fc02e7fc700 1 -- 172.21.15.27:0/3819307714 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc028013b60 con 0x7fc0300f66e0 2021-11-03T17:46:41.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.630+0000 7fc037d12700 1 -- 172.21.15.27:0/3819307714 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc03010efa0 con 0x7fc0300f66e0 2021-11-03T17:46:41.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.631+0000 7fc037d12700 1 -- 172.21.15.27:0/3819307714 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc030045ad0 con 0x7fc0300f66e0 2021-11-03T17:46:41.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.631+0000 7fc02e7fc700 1 -- 172.21.15.27:0/3819307714 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fc02801a030 con 0x7fc0300f66e0 2021-11-03T17:46:41.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.631+0000 7fc02e7fc700 1 --2- 172.21.15.27:0/3819307714 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc0180640b0 0x7fc018066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:46:41.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.631+0000 7fc02e7fc700 1 -- 172.21.15.27:0/3819307714 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fc028080780 con 0x7fc0300f66e0 2021-11-03T17:46:41.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.632+0000 7fc0352ad700 1 --2- 172.21.15.27:0/3819307714 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc0180640b0 0x7fc018066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:46:41.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.632+0000 7fc0352ad700 1 --2- 172.21.15.27:0/3819307714 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc0180640b0 0x7fc018066570 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7fc024004630 tx=0x7fc024009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:46:41.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.635+0000 7fc02e7fc700 1 -- 172.21.15.27:0/3819307714 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fc028052a40 con 0x7fc0300f66e0 2021-11-03T17:46:41.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.866+0000 7fc037d12700 1 -- 172.21.15.27:0/3819307714 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc0300008d0 con 0x7fc0300f66e0 2021-11-03T17:46:41.860 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.866+0000 7fc02e7fc700 1 -- 172.21.15.27:0/3819307714 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fc028018070 con 0x7fc0300f66e0 2021-11-03T17:46:41.861 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:46:41.861 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:46:41.861 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:46:41.861 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:46:41.862 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:46:41.862 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:46:41.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:46:41.862 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:46:41.862 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:46:41.863 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:46:41.863 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:46:41.863 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:46:41.863 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:46:41.864 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:46:41.864 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:46:41.864 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:46:41.864 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:46:41.864 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:46:41.865 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.868+0000 7fc037d12700 1 -- 172.21.15.27:0/3819307714 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc0180640b0 msgr2=0x7fc018066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:46:41.866 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.868+0000 7fc037d12700 1 --2- 172.21.15.27:0/3819307714 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc0180640b0 0x7fc018066570 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7fc024004630 tx=0x7fc024009b30).stop 2021-11-03T17:46:41.866 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.868+0000 7fc037d12700 1 -- 172.21.15.27:0/3819307714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0300f66e0 msgr2=0x7fc03006b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:46:41.866 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.868+0000 7fc037d12700 1 --2- 172.21.15.27:0/3819307714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0300f66e0 0x7fc03006b9a0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7fc028007fd0 tx=0x7fc028004ee0).stop 2021-11-03T17:46:41.866 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.868+0000 7fc037d12700 1 -- 172.21.15.27:0/3819307714 shutdown_connections 2021-11-03T17:46:41.867 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.868+0000 7fc037d12700 1 --2- 172.21.15.27:0/3819307714 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc0180640b0 0x7fc018066570 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:41.867 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.868+0000 7fc037d12700 1 --2- 172.21.15.27:0/3819307714 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0300f66e0 0x7fc03006b9a0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:41.867 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.868+0000 7fc037d12700 1 --2- 172.21.15.27:0/3819307714 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0300f6ff0 0x7fc030069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:41.867 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.868+0000 7fc037d12700 1 -- 172.21.15.27:0/3819307714 >> 172.21.15.27:0/3819307714 conn(0x7fc0300f1f00 msgr2=0x7fc0300fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:46:41.867 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.868+0000 7fc037d12700 1 -- 172.21.15.27:0/3819307714 shutdown_connections 2021-11-03T17:46:41.868 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.868+0000 7fc037d12700 1 -- 172.21.15.27:0/3819307714 wait complete. 2021-11-03T17:46:41.964 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.968+0000 7f0abaee1700 1 -- 172.21.15.27:0/4107132066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ab40f77d0 msgr2=0x7f0ab40f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:46:41.964 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.968+0000 7f0abaee1700 1 --2- 172.21.15.27:0/4107132066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ab40f77d0 0x7f0ab40f7c30 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f0aa4004660 tx=0x7f0aa4009b30).stop 2021-11-03T17:46:41.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.968+0000 7f0abaee1700 1 -- 172.21.15.27:0/4107132066 shutdown_connections 2021-11-03T17:46:41.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.968+0000 7f0abaee1700 1 --2- 172.21.15.27:0/4107132066 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ab40f77d0 0x7f0ab40f7c30 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:41.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.968+0000 7f0abaee1700 1 --2- 172.21.15.27:0/4107132066 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ab40f6530 0x7f0ab40f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:46:41.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.968+0000 7f0abaee1700 1 -- 172.21.15.27:0/4107132066 >> 172.21.15.27:0/4107132066 conn(0x7f0ab40f1cd0 msgr2=0x7f0ab40f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:46:41.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.968+0000 7f0abaee1700 1 -- 172.21.15.27:0/4107132066 shutdown_connections 2021-11-03T17:46:41.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.968+0000 7f0abaee1700 1 -- 172.21.15.27:0/4107132066 wait complete. 2021-11-03T17:46:41.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.969+0000 7f0abaee1700 1 Processor -- start 2021-11-03T17:46:41.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.969+0000 7f0abaee1700 1 -- start start 2021-11-03T17:46:41.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.969+0000 7f0abaee1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ab40f6530 0x7f0ab4106d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:46:41.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.969+0000 7f0abaee1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ab40f77d0 0x7f0ab4107270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:46:41.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.969+0000 7f0abaee1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab4107900 con 0x7f0ab40f77d0 2021-11-03T17:46:41.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.969+0000 7f0abaee1700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab41080b0 con 0x7f0ab40f6530 2021-11-03T17:46:41.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.969+0000 7f0ab96de700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ab40f77d0 0x7f0ab4107270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:46:41.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.969+0000 7f0ab96de700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ab40f77d0 0x7f0ab4107270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59360/0 (socket says 172.21.15.27:59360) 2021-11-03T17:46:41.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.969+0000 7f0ab96de700 1 -- 172.21.15.27:0/629538360 learned_addr learned my addr 172.21.15.27:0/629538360 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:46:41.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.969+0000 7f0ab9edf700 1 --2- 172.21.15.27:0/629538360 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ab40f6530 0x7f0ab4106d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:46:41.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.969+0000 7f0ab96de700 1 -- 172.21.15.27:0/629538360 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ab40f6530 msgr2=0x7f0ab4106d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:46:41.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.970+0000 7f0ab96de700 1 --2- 172.21.15.27:0/629538360 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ab40f6530 0x7f0ab4106d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:41.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.970+0000 7f0ab96de700 1 -- 172.21.15.27:0/629538360 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa4005040 con 0x7f0ab40f77d0 2021-11-03T17:46:41.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.970+0000 7f0ab9edf700 1 --2- 172.21.15.27:0/629538360 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ab40f6530 0x7f0ab4106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:46:41.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.970+0000 7f0ab96de700 1 --2- 172.21.15.27:0/629538360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ab40f77d0 0x7f0ab4107270 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7f0aa4009990 tx=0x7f0aa4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:46:41.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.970+0000 7f0aaa7fc700 1 -- 172.21.15.27:0/629538360 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0aa4018070 con 0x7f0ab40f77d0 2021-11-03T17:46:41.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.970+0000 7f0aaa7fc700 1 -- 172.21.15.27:0/629538360 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f0aa4007ca0 con 0x7f0ab40f77d0 2021-11-03T17:46:41.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.970+0000 7f0abaee1700 1 -- 172.21.15.27:0/629538360 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab4108330 con 0x7f0ab40f77d0 2021-11-03T17:46:41.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.970+0000 7f0aaa7fc700 1 -- 172.21.15.27:0/629538360 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0aa401c620 con 0x7f0ab40f77d0 2021-11-03T17:46:41.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.970+0000 7f0abaee1700 1 -- 172.21.15.27:0/629538360 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab4108890 con 0x7f0ab40f77d0 2021-11-03T17:46:41.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.971+0000 7f0abaee1700 1 -- 172.21.15.27:0/629538360 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab4059100 con 0x7f0ab40f77d0 2021-11-03T17:46:41.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.971+0000 7f0aaa7fc700 1 -- 172.21.15.27:0/629538360 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f0aa401d960 con 0x7f0ab40f77d0 2021-11-03T17:46:41.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.972+0000 7f0aaa7fc700 1 --2- 172.21.15.27:0/629538360 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0aa0064100 0x7f0aa00665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:46:41.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.972+0000 7f0aaa7fc700 1 -- 172.21.15.27:0/629538360 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f0aa40825a0 con 0x7f0ab40f77d0 2021-11-03T17:46:41.973 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.972+0000 7f0ab9edf700 1 --2- 172.21.15.27:0/629538360 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0aa0064100 0x7f0aa00665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:46:41.973 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.972+0000 7f0ab9edf700 1 --2- 172.21.15.27:0/629538360 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0aa0064100 0x7f0aa00665c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f0ab4107b40 tx=0x7f0ab0008040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:46:41.973 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:41.976+0000 7f0aaa7fc700 1 -- 172.21.15.27:0/629538360 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f0aa4054960 con 0x7f0ab40f77d0 2021-11-03T17:46:42.173 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:42.179+0000 7f0abaee1700 1 -- 172.21.15.27:0/629538360 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0ab40fed90 con 0x7f0ab40f77d0 2021-11-03T17:46:42.176 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:42.182+0000 7f0aaa7fc700 1 -- 172.21.15.27:0/629538360 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f0aa4054550 con 0x7f0ab40f77d0 2021-11-03T17:46:42.177 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:46:42.177 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:46:42.177 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:46:42.177 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:46:42.178 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:46:42.178 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:46:42.178 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:46:42.178 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:46:42.179 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:46:42.179 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:46:42.179 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:46:42.179 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:46:42.180 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:46:42.180 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:46:42.180 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:46:42.180 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:46:42.181 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:46:42.181 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:46:42.181 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:46:42.181 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:46:42.181 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:46:42.182 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:46:42.182 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:46:42.182 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:46:42.182 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:46:42.183 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:46:42.183 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:46:42.183 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:46:42.183 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:46:42.184 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:46:42.184 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:46:42.184 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:46:42.184 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:46:42.185 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:46:42.185 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:46:42.185 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:46:42.185 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:46:42.186 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:46:42.186 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:46:42.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:42.184+0000 7f0abaee1700 1 -- 172.21.15.27:0/629538360 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0aa0064100 msgr2=0x7f0aa00665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:46:42.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:42.184+0000 7f0abaee1700 1 --2- 172.21.15.27:0/629538360 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0aa0064100 0x7f0aa00665c0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f0ab4107b40 tx=0x7f0ab0008040).stop 2021-11-03T17:46:42.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:42.184+0000 7f0abaee1700 1 -- 172.21.15.27:0/629538360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ab40f77d0 msgr2=0x7f0ab4107270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:46:42.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:42.184+0000 7f0abaee1700 1 --2- 172.21.15.27:0/629538360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ab40f77d0 0x7f0ab4107270 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7f0aa4009990 tx=0x7f0aa4009b30).stop 2021-11-03T17:46:42.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:42.184+0000 7f0abaee1700 1 -- 172.21.15.27:0/629538360 shutdown_connections 2021-11-03T17:46:42.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:42.184+0000 7f0abaee1700 1 --2- 172.21.15.27:0/629538360 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0aa0064100 0x7f0aa00665c0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:42.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:42.184+0000 7f0abaee1700 1 --2- 172.21.15.27:0/629538360 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ab40f6530 0x7f0ab4106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:42.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:42.184+0000 7f0abaee1700 1 --2- 172.21.15.27:0/629538360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ab40f77d0 0x7f0ab4107270 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:46:42.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:42.184+0000 7f0abaee1700 1 -- 172.21.15.27:0/629538360 >> 172.21.15.27:0/629538360 conn(0x7f0ab40f1cd0 msgr2=0x7f0ab40fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:46:42.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:42.185+0000 7f0abaee1700 1 -- 172.21.15.27:0/629538360 shutdown_connections 2021-11-03T17:46:42.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:46:42.185+0000 7f0abaee1700 1 -- 172.21.15.27:0/629538360 wait complete. 2021-11-03T17:46:42.190 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:46:42.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:42 smithi115 ceph-mon[65453]: pgmap v961: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:42.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:46:42.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:46:42.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:42.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:42.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:42.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:42.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:42 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:46:42.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:42 smithi115 ceph-mon[65453]: from='client.35096 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:46:42.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:42 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3819307714' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:42.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:42 smithi027 ceph-mon[97244]: pgmap v961: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:42.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:46:42.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:46:42.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:42.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:42.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:42.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:42.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:42 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:46:42.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:42 smithi027 ceph-mon[97244]: from='client.35096 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:46:42.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:42 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3819307714' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:43.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:43 smithi027 ceph-mon[97244]: from='client.35100 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:46:43.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:43 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/629538360' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:46:43.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:43 smithi115 ceph-mon[65453]: from='client.35100 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:46:43.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:43 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/629538360' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:46:44.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:44 smithi115 ceph-mon[65453]: pgmap v962: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:44.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:44 smithi027 ceph-mon[97244]: pgmap v962: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:45.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1847.403152 secs 2021-11-03T17:46:45.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1847.403152 secs 2021-11-03T17:46:46.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:46 smithi115 ceph-mon[65453]: pgmap v963: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:46.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:46 smithi027 ceph-mon[97244]: pgmap v963: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:48.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:48 smithi115 ceph-mon[65453]: pgmap v964: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:48.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:48 smithi027 ceph-mon[97244]: pgmap v964: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:50.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:50 smithi115 ceph-mon[65453]: pgmap v965: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:50.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1852.403290 secs 2021-11-03T17:46:50.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:50 smithi027 ceph-mon[97244]: pgmap v965: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:50.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1852.403290 secs 2021-11-03T17:46:52.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:52 smithi115 ceph-mon[65453]: pgmap v966: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:52.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:46:52.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:46:52.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:46:52.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:52.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:52.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:52.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:52.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:52 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:46:52.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:52 smithi027 ceph-mon[97244]: pgmap v966: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:52.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:46:52.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:46:52.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:46:52.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:52.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:52.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:52.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:46:52.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:52 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:46:54.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:54 smithi115 ceph-mon[65453]: pgmap v967: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:54.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:54 smithi027 ceph-mon[97244]: pgmap v967: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:55.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1857.403440 secs 2021-11-03T17:46:55.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1857.403440 secs 2021-11-03T17:46:56.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:56 smithi115 ceph-mon[65453]: pgmap v968: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:56.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:46:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:46:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:56 smithi027 ceph-mon[97244]: pgmap v968: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:46:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:46:58.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:46:58 smithi115 ceph-mon[65453]: pgmap v969: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:46:58.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:46:58 smithi027 ceph-mon[97244]: pgmap v969: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:00.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:00 smithi115 ceph-mon[65453]: pgmap v970: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:00.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1862.403562 secs 2021-11-03T17:47:00.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:00 smithi027 ceph-mon[97244]: pgmap v970: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:00.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1862.403562 secs 2021-11-03T17:47:02.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:02 smithi115 ceph-mon[65453]: pgmap v971: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:02.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:47:02.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:02 smithi115 ceph-mon[65453]: poll took 40113.626 ms: 1 events 2021-11-03T17:47:02.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:47:02.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:47:02.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:47:02.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:02.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:02.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:02.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:02.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:02 smithi027 ceph-mon[97244]: pgmap v971: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:02.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:47:02.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:02 smithi027 ceph-mon[97244]: poll took 40113.626 ms: 1 events 2021-11-03T17:47:02.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:47:02.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:47:02.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:47:02.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:02.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:02.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:02.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:03.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:03 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:47:03.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:03 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:47:04.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:04 smithi115 ceph-mon[65453]: pgmap v972: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:04.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:04 smithi027 ceph-mon[97244]: pgmap v972: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:05.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1867.403692 secs 2021-11-03T17:47:05.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1867.403692 secs 2021-11-03T17:47:06.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:06 smithi115 ceph-mon[65453]: pgmap v973: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:06.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:06 smithi027 ceph-mon[97244]: pgmap v973: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:08.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:08 smithi027 ceph-mon[97244]: pgmap v974: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:08.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:08 smithi115 ceph-mon[65453]: pgmap v974: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:10.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:10 smithi027 ceph-mon[97244]: pgmap v975: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:10.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1872.403831 secs 2021-11-03T17:47:10.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:10 smithi115 ceph-mon[65453]: pgmap v975: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:10.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1872.403831 secs 2021-11-03T17:47:12.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.300+0000 7f3f1e699700 1 -- 172.21.15.27:0/3537313186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f180f66e0 msgr2=0x7f3f180f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:47:12.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.300+0000 7f3f1e699700 1 --2- 172.21.15.27:0/3537313186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f180f66e0 0x7f3f180f6b00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7f3f00004660 tx=0x7f3f00009b30).stop 2021-11-03T17:47:12.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.300+0000 7f3f1e699700 1 -- 172.21.15.27:0/3537313186 shutdown_connections 2021-11-03T17:47:12.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.300+0000 7f3f1e699700 1 --2- 172.21.15.27:0/3537313186 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f180f6ff0 0x7f3f180f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:12.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.300+0000 7f3f1e699700 1 --2- 172.21.15.27:0/3537313186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f180f66e0 0x7f3f180f6b00 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:12.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.300+0000 7f3f1e699700 1 -- 172.21.15.27:0/3537313186 >> 172.21.15.27:0/3537313186 conn(0x7f3f180f1f00 msgr2=0x7f3f180f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:47:12.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.300+0000 7f3f1e699700 1 -- 172.21.15.27:0/3537313186 shutdown_connections 2021-11-03T17:47:12.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.300+0000 7f3f1e699700 1 -- 172.21.15.27:0/3537313186 wait complete. 2021-11-03T17:47:12.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.301+0000 7f3f1e699700 1 Processor -- start 2021-11-03T17:47:12.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.301+0000 7f3f1e699700 1 -- start start 2021-11-03T17:47:12.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.301+0000 7f3f1e699700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f180f66e0 0x7f3f18106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:47:12.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.301+0000 7f3f1e699700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f180f6ff0 0x7f3f18107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:47:12.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.301+0000 7f3f1e699700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f18107a00 con 0x7f3f180f66e0 2021-11-03T17:47:12.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.301+0000 7f3f1e699700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f18107b40 con 0x7f3f180f6ff0 2021-11-03T17:47:12.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.301+0000 7f3f17fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f180f66e0 0x7f3f18106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:47:12.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.301+0000 7f3f17fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f180f66e0 0x7f3f18106f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59376/0 (socket says 172.21.15.27:59376) 2021-11-03T17:47:12.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.302+0000 7f3f17fff700 1 -- 172.21.15.27:0/1122241333 learned_addr learned my addr 172.21.15.27:0/1122241333 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:47:12.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.302+0000 7f3f177fe700 1 --2- 172.21.15.27:0/1122241333 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f180f6ff0 0x7f3f18107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:47:12.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.302+0000 7f3f17fff700 1 -- 172.21.15.27:0/1122241333 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f180f6ff0 msgr2=0x7f3f18107450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:47:12.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.302+0000 7f3f17fff700 1 --2- 172.21.15.27:0/1122241333 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f180f6ff0 0x7f3f18107450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:12.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.302+0000 7f3f17fff700 1 -- 172.21.15.27:0/1122241333 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f00005040 con 0x7f3f180f66e0 2021-11-03T17:47:12.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.302+0000 7f3f17fff700 1 --2- 172.21.15.27:0/1122241333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f180f66e0 0x7f3f18106f60 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7f3f00009990 tx=0x7f3f00009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:47:12.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.302+0000 7f3f14ff9700 1 -- 172.21.15.27:0/1122241333 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f00018070 con 0x7f3f180f66e0 2021-11-03T17:47:12.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.302+0000 7f3f14ff9700 1 -- 172.21.15.27:0/1122241333 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f3f00007ba0 con 0x7f3f180f66e0 2021-11-03T17:47:12.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.302+0000 7f3f1e699700 1 -- 172.21.15.27:0/1122241333 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f18108480 con 0x7f3f180f66e0 2021-11-03T17:47:12.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.302+0000 7f3f14ff9700 1 -- 172.21.15.27:0/1122241333 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f0001c3c0 con 0x7f3f180f66e0 2021-11-03T17:47:12.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.302+0000 7f3f1e699700 1 -- 172.21.15.27:0/1122241333 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f181089e0 con 0x7f3f180f66e0 2021-11-03T17:47:12.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.303+0000 7f3f1e699700 1 -- 172.21.15.27:0/1122241333 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f18045ad0 con 0x7f3f180f66e0 2021-11-03T17:47:12.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.304+0000 7f3f14ff9700 1 -- 172.21.15.27:0/1122241333 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f3f0001d420 con 0x7f3f180f66e0 2021-11-03T17:47:12.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.304+0000 7f3f14ff9700 1 --2- 172.21.15.27:0/1122241333 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3f040640b0 0x7f3f04066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:47:12.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.304+0000 7f3f14ff9700 1 -- 172.21.15.27:0/1122241333 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f3f000823c0 con 0x7f3f180f66e0 2021-11-03T17:47:12.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.304+0000 7f3f177fe700 1 --2- 172.21.15.27:0/1122241333 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3f040640b0 0x7f3f04066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:47:12.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.305+0000 7f3f177fe700 1 --2- 172.21.15.27:0/1122241333 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3f040640b0 0x7f3f04066570 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f3f08004c40 tx=0x7f3f0800b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:47:12.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.308+0000 7f3f14ff9700 1 -- 172.21.15.27:0/1122241333 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f3f00054720 con 0x7f3f180f66e0 2021-11-03T17:47:12.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.493+0000 7f3f1e699700 1 -- 172.21.15.27:0/1122241333 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f180fef40 con 0x7f3f040640b0 2021-11-03T17:47:12.488 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.494+0000 7f3f14ff9700 1 -- 172.21.15.27:0/1122241333 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f3f180fef40 con 0x7f3f040640b0 2021-11-03T17:47:12.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.496+0000 7f3f1e699700 1 -- 172.21.15.27:0/1122241333 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3f040640b0 msgr2=0x7f3f04066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:47:12.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.496+0000 7f3f1e699700 1 --2- 172.21.15.27:0/1122241333 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3f040640b0 0x7f3f04066570 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f3f08004c40 tx=0x7f3f0800b040).stop 2021-11-03T17:47:12.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.496+0000 7f3f1e699700 1 -- 172.21.15.27:0/1122241333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f180f66e0 msgr2=0x7f3f18106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:47:12.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.496+0000 7f3f1e699700 1 --2- 172.21.15.27:0/1122241333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f180f66e0 0x7f3f18106f60 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7f3f00009990 tx=0x7f3f00009b30).stop 2021-11-03T17:47:12.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.496+0000 7f3f1e699700 1 -- 172.21.15.27:0/1122241333 shutdown_connections 2021-11-03T17:47:12.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.496+0000 7f3f1e699700 1 --2- 172.21.15.27:0/1122241333 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3f040640b0 0x7f3f04066570 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:12.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.496+0000 7f3f1e699700 1 --2- 172.21.15.27:0/1122241333 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f180f66e0 0x7f3f18106f60 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:12.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.496+0000 7f3f1e699700 1 --2- 172.21.15.27:0/1122241333 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f180f6ff0 0x7f3f18107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:12.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.496+0000 7f3f1e699700 1 -- 172.21.15.27:0/1122241333 >> 172.21.15.27:0/1122241333 conn(0x7f3f180f1f00 msgr2=0x7f3f180fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:47:12.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.497+0000 7f3f1e699700 1 -- 172.21.15.27:0/1122241333 shutdown_connections 2021-11-03T17:47:12.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.497+0000 7f3f1e699700 1 -- 172.21.15.27:0/1122241333 wait complete. 2021-11-03T17:47:12.500 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:47:12.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:12 smithi027 ceph-mon[97244]: pgmap v976: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:12.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:47:12.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:47:12.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:47:12.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:12.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:12.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:12.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:12.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.598+0000 7f8a6e341700 1 -- 172.21.15.27:0/654477611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a680f7d20 msgr2=0x7f8a680f8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:47:12.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.598+0000 7f8a6e341700 1 --2- 172.21.15.27:0/654477611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a680f7d20 0x7f8a680f8140 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f8a5c004660 tx=0x7f8a5c009b30).stop 2021-11-03T17:47:12.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.598+0000 7f8a6e341700 1 -- 172.21.15.27:0/654477611 shutdown_connections 2021-11-03T17:47:12.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.598+0000 7f8a6e341700 1 --2- 172.21.15.27:0/654477611 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a680f86c0 0x7f8a680f4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:12.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.598+0000 7f8a6e341700 1 --2- 172.21.15.27:0/654477611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a680f7d20 0x7f8a680f8140 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:12.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.598+0000 7f8a6e341700 1 -- 172.21.15.27:0/654477611 >> 172.21.15.27:0/654477611 conn(0x7f8a680f04c0 msgr2=0x7f8a680f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:47:12.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.598+0000 7f8a6e341700 1 -- 172.21.15.27:0/654477611 shutdown_connections 2021-11-03T17:47:12.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.598+0000 7f8a6e341700 1 -- 172.21.15.27:0/654477611 wait complete. 2021-11-03T17:47:12.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.599+0000 7f8a6e341700 1 Processor -- start 2021-11-03T17:47:12.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.599+0000 7f8a6e341700 1 -- start start 2021-11-03T17:47:12.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.599+0000 7f8a6e341700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a680f7d20 0x7f8a68106fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:47:12.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.599+0000 7f8a6e341700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a680f86c0 0x7f8a681074b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:47:12.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.599+0000 7f8a6e341700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a68107a60 con 0x7f8a680f7d20 2021-11-03T17:47:12.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.599+0000 7f8a6e341700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a68107ba0 con 0x7f8a680f86c0 2021-11-03T17:47:12.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.599+0000 7f8a67fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a680f7d20 0x7f8a68106fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:47:12.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.599+0000 7f8a67fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a680f7d20 0x7f8a68106fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59386/0 (socket says 172.21.15.27:59386) 2021-11-03T17:47:12.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.599+0000 7f8a67fff700 1 -- 172.21.15.27:0/3951055027 learned_addr learned my addr 172.21.15.27:0/3951055027 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:47:12.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.599+0000 7f8a677fe700 1 --2- 172.21.15.27:0/3951055027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a680f86c0 0x7f8a681074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:47:12.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.599+0000 7f8a67fff700 1 -- 172.21.15.27:0/3951055027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a680f86c0 msgr2=0x7f8a681074b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:47:12.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.599+0000 7f8a67fff700 1 --2- 172.21.15.27:0/3951055027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a680f86c0 0x7f8a681074b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:12.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.599+0000 7f8a67fff700 1 -- 172.21.15.27:0/3951055027 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a5c005040 con 0x7f8a680f7d20 2021-11-03T17:47:12.600 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.599+0000 7f8a677fe700 1 --2- 172.21.15.27:0/3951055027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a680f86c0 0x7f8a681074b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:47:12.600 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.599+0000 7f8a67fff700 1 --2- 172.21.15.27:0/3951055027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a680f7d20 0x7f8a68106fc0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f8a5c004f60 tx=0x7f8a5c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:47:12.600 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.600+0000 7f8a64ff9700 1 -- 172.21.15.27:0/3951055027 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a5c018070 con 0x7f8a680f7d20 2021-11-03T17:47:12.600 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.600+0000 7f8a64ff9700 1 -- 172.21.15.27:0/3951055027 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f8a5c007ba0 con 0x7f8a680f7d20 2021-11-03T17:47:12.601 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.600+0000 7f8a6e341700 1 -- 172.21.15.27:0/3951055027 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a681084e0 con 0x7f8a680f7d20 2021-11-03T17:47:12.601 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.600+0000 7f8a64ff9700 1 -- 172.21.15.27:0/3951055027 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a5c01c620 con 0x7f8a680f7d20 2021-11-03T17:47:12.601 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.600+0000 7f8a6e341700 1 -- 172.21.15.27:0/3951055027 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a68108a40 con 0x7f8a680f7d20 2021-11-03T17:47:12.601 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.601+0000 7f8a6e341700 1 -- 172.21.15.27:0/3951055027 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a68045ad0 con 0x7f8a680f7d20 2021-11-03T17:47:12.602 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.601+0000 7f8a64ff9700 1 -- 172.21.15.27:0/3951055027 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f8a5c01c8e0 con 0x7f8a680f7d20 2021-11-03T17:47:12.602 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.601+0000 7f8a64ff9700 1 --2- 172.21.15.27:0/3951055027 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8a500640b0 0x7f8a50066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:47:12.602 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.602+0000 7f8a64ff9700 1 -- 172.21.15.27:0/3951055027 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f8a5c082740 con 0x7f8a680f7d20 2021-11-03T17:47:12.602 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.602+0000 7f8a677fe700 1 --2- 172.21.15.27:0/3951055027 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8a500640b0 0x7f8a50066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:47:12.602 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.602+0000 7f8a677fe700 1 --2- 172.21.15.27:0/3951055027 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8a500640b0 0x7f8a50066570 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f8a580041d0 tx=0x7f8a58009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:47:12.603 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.606+0000 7f8a64ff9700 1 -- 172.21.15.27:0/3951055027 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f8a5c054a40 con 0x7f8a680f7d20 2021-11-03T17:47:12.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:12 smithi115 ceph-mon[65453]: pgmap v976: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:12.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:47:12.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:47:12.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:47:12.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:12.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:12.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:12.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:12.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.781+0000 7f8a6e341700 1 -- 172.21.15.27:0/3951055027 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a680ff050 con 0x7f8a500640b0 2021-11-03T17:47:12.790 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.795+0000 7f8a64ff9700 1 -- 172.21.15.27:0/3951055027 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f8a58004600 con 0x7f8a500640b0 2021-11-03T17:47:12.790 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:47:12.790 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (38m) 8m ago 40m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:47:12.791 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (30m) 8m ago 39m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:47:12.791 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (30m) 7m ago 39m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:47:12.791 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (38m) 8m ago 39m 41.3M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:47:12.792 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (32m) 8m ago 35m 21.1M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:47:12.792 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (35m) 8m ago 35m 3843M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:47:12.792 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (31m) 7m ago 35m 20.4M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:47:12.792 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (32m) 7m ago 35m 22.1M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:47:12.793 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (32m) 8m ago 41m 517M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:47:12.793 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (31m) 7m ago 39m 419M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:47:12.793 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (31m) 8m ago 41m 159M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:47:12.793 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (31m) 7m ago 39m 82.5M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:47:12.794 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (39m) 8m ago 39m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:47:12.794 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (39m) 7m ago 38m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:47:12.794 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (30m) 8m ago 38m 311M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:47:12.794 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (29m) 8m ago 38m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:47:12.795 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (29m) 8m ago 37m 248M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:47:12.795 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (28m) 7m ago 37m 354M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:47:12.795 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (28m) 7m ago 37m 285M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:47:12.795 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (27m) 7m ago 36m 287M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:47:12.795 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (38m) 8m ago 39m 50.5M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:47:12.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.797+0000 7f8a6e341700 1 -- 172.21.15.27:0/3951055027 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8a500640b0 msgr2=0x7f8a50066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:47:12.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.797+0000 7f8a6e341700 1 --2- 172.21.15.27:0/3951055027 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8a500640b0 0x7f8a50066570 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 rx=0x7f8a580041d0 tx=0x7f8a58009040).stop 2021-11-03T17:47:12.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.797+0000 7f8a6e341700 1 -- 172.21.15.27:0/3951055027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a680f7d20 msgr2=0x7f8a68106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:47:12.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.797+0000 7f8a6e341700 1 --2- 172.21.15.27:0/3951055027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a680f7d20 0x7f8a68106fc0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7f8a5c004f60 tx=0x7f8a5c009b30).stop 2021-11-03T17:47:12.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.797+0000 7f8a6e341700 1 -- 172.21.15.27:0/3951055027 shutdown_connections 2021-11-03T17:47:12.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.797+0000 7f8a6e341700 1 --2- 172.21.15.27:0/3951055027 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8a500640b0 0x7f8a50066570 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:12.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.797+0000 7f8a6e341700 1 --2- 172.21.15.27:0/3951055027 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a680f7d20 0x7f8a68106fc0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:12.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.797+0000 7f8a6e341700 1 --2- 172.21.15.27:0/3951055027 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a680f86c0 0x7f8a681074b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:12.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.797+0000 7f8a6e341700 1 -- 172.21.15.27:0/3951055027 >> 172.21.15.27:0/3951055027 conn(0x7f8a680f04c0 msgr2=0x7f8a680fb5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:47:12.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.798+0000 7f8a6e341700 1 -- 172.21.15.27:0/3951055027 shutdown_connections 2021-11-03T17:47:12.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.798+0000 7f8a6e341700 1 -- 172.21.15.27:0/3951055027 wait complete. 2021-11-03T17:47:12.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.896+0000 7f8411333700 1 -- 172.21.15.27:0/2177029763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f840c0f6530 msgr2=0x7f840c0f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:47:12.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.896+0000 7f8411333700 1 --2- 172.21.15.27:0/2177029763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f840c0f6530 0x7f840c0f6950 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7f83fc004660 tx=0x7f83fc009b30).stop 2021-11-03T17:47:12.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.896+0000 7f8411333700 1 -- 172.21.15.27:0/2177029763 shutdown_connections 2021-11-03T17:47:12.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.896+0000 7f8411333700 1 --2- 172.21.15.27:0/2177029763 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f840c0f77d0 0x7f840c0f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:12.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.896+0000 7f8411333700 1 --2- 172.21.15.27:0/2177029763 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f840c0f6530 0x7f840c0f6950 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:12.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.896+0000 7f8411333700 1 -- 172.21.15.27:0/2177029763 >> 172.21.15.27:0/2177029763 conn(0x7f840c0f1c90 msgr2=0x7f840c0f40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:47:12.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.897+0000 7f8411333700 1 -- 172.21.15.27:0/2177029763 shutdown_connections 2021-11-03T17:47:12.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.897+0000 7f8411333700 1 -- 172.21.15.27:0/2177029763 wait complete. 2021-11-03T17:47:12.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.897+0000 7f8411333700 1 Processor -- start 2021-11-03T17:47:12.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.897+0000 7f8411333700 1 -- start start 2021-11-03T17:47:12.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.897+0000 7f8411333700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f840c0f6530 0x7f840c106d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:47:12.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.897+0000 7f8411333700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f840c0f77d0 0x7f840c107280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:47:12.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.897+0000 7f8411333700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840c107880 con 0x7f840c0f6530 2021-11-03T17:47:12.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.897+0000 7f8411333700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f840c108090 con 0x7f840c0f77d0 2021-11-03T17:47:12.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.898+0000 7f840bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f840c0f6530 0x7f840c106d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:47:12.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.898+0000 7f840bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f840c0f6530 0x7f840c106d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59396/0 (socket says 172.21.15.27:59396) 2021-11-03T17:47:12.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.898+0000 7f840bfff700 1 -- 172.21.15.27:0/4222296226 learned_addr learned my addr 172.21.15.27:0/4222296226 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:47:12.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.898+0000 7f840b7fe700 1 --2- 172.21.15.27:0/4222296226 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f840c0f77d0 0x7f840c107280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:47:12.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.898+0000 7f840bfff700 1 -- 172.21.15.27:0/4222296226 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f840c0f77d0 msgr2=0x7f840c107280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:47:12.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.898+0000 7f840bfff700 1 --2- 172.21.15.27:0/4222296226 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f840c0f77d0 0x7f840c107280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:12.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.898+0000 7f840bfff700 1 -- 172.21.15.27:0/4222296226 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83fc005040 con 0x7f840c0f6530 2021-11-03T17:47:12.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.898+0000 7f840bfff700 1 --2- 172.21.15.27:0/4222296226 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f840c0f6530 0x7f840c106d90 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f83fc013a70 tx=0x7f83fc009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:47:12.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.898+0000 7f8408ff9700 1 -- 172.21.15.27:0/4222296226 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83fc018070 con 0x7f840c0f6530 2021-11-03T17:47:12.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.898+0000 7f8408ff9700 1 -- 172.21.15.27:0/4222296226 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f83fc0097d0 con 0x7f840c0f6530 2021-11-03T17:47:12.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.898+0000 7f8411333700 1 -- 172.21.15.27:0/4222296226 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f840c1082b0 con 0x7f840c0f6530 2021-11-03T17:47:12.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.898+0000 7f8408ff9700 1 -- 172.21.15.27:0/4222296226 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83fc01c3c0 con 0x7f840c0f6530 2021-11-03T17:47:12.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.898+0000 7f8411333700 1 -- 172.21.15.27:0/4222296226 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f840c108810 con 0x7f840c0f6530 2021-11-03T17:47:12.900 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.899+0000 7f8411333700 1 -- 172.21.15.27:0/4222296226 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f840c059100 con 0x7f840c0f6530 2021-11-03T17:47:12.900 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.900+0000 7f8408ff9700 1 -- 172.21.15.27:0/4222296226 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f83fc007ba0 con 0x7f840c0f6530 2021-11-03T17:47:12.900 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.900+0000 7f8408ff9700 1 --2- 172.21.15.27:0/4222296226 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f83f4064100 0x7f83f40665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:47:12.901 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.900+0000 7f8408ff9700 1 -- 172.21.15.27:0/4222296226 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f83fc082510 con 0x7f840c0f6530 2021-11-03T17:47:12.901 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.900+0000 7f840b7fe700 1 --2- 172.21.15.27:0/4222296226 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f83f4064100 0x7f83f40665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:47:12.901 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.900+0000 7f840b7fe700 1 --2- 172.21.15.27:0/4222296226 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f83f4064100 0x7f83f40665c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f8400009740 tx=0x7f8400009c60).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:47:12.902 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:12.904+0000 7f8408ff9700 1 -- 172.21.15.27:0/4222296226 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f83fc054810 con 0x7f840c0f6530 2021-11-03T17:47:13.132 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.139+0000 7f8411333700 1 -- 172.21.15.27:0/4222296226 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f840c0008d0 con 0x7f840c0f6530 2021-11-03T17:47:13.133 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.139+0000 7f8408ff9700 1 -- 172.21.15.27:0/4222296226 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f83fc054400 con 0x7f840c0f6530 2021-11-03T17:47:13.134 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:47:13.134 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:47:13.134 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:47:13.134 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:47:13.135 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:47:13.135 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:47:13.135 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:47:13.135 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:47:13.136 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:47:13.136 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:47:13.136 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:47:13.136 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:47:13.136 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:47:13.137 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:47:13.137 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:47:13.137 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:47:13.137 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:47:13.138 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:47:13.139 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.141+0000 7f8411333700 1 -- 172.21.15.27:0/4222296226 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f83f4064100 msgr2=0x7f83f40665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:47:13.139 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.141+0000 7f8411333700 1 --2- 172.21.15.27:0/4222296226 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f83f4064100 0x7f83f40665c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 rx=0x7f8400009740 tx=0x7f8400009c60).stop 2021-11-03T17:47:13.139 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.141+0000 7f8411333700 1 -- 172.21.15.27:0/4222296226 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f840c0f6530 msgr2=0x7f840c106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:47:13.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.141+0000 7f8411333700 1 --2- 172.21.15.27:0/4222296226 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f840c0f6530 0x7f840c106d90 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f83fc013a70 tx=0x7f83fc009b30).stop 2021-11-03T17:47:13.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.141+0000 7f8411333700 1 -- 172.21.15.27:0/4222296226 shutdown_connections 2021-11-03T17:47:13.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.141+0000 7f8411333700 1 --2- 172.21.15.27:0/4222296226 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f83f4064100 0x7f83f40665c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:13.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.141+0000 7f8411333700 1 --2- 172.21.15.27:0/4222296226 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f840c0f6530 0x7f840c106d90 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:13.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.141+0000 7f8411333700 1 --2- 172.21.15.27:0/4222296226 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f840c0f77d0 0x7f840c107280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:13.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.141+0000 7f8411333700 1 -- 172.21.15.27:0/4222296226 >> 172.21.15.27:0/4222296226 conn(0x7f840c0f1c90 msgr2=0x7f840c0fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:47:13.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.141+0000 7f8411333700 1 -- 172.21.15.27:0/4222296226 shutdown_connections 2021-11-03T17:47:13.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.141+0000 7f8411333700 1 -- 172.21.15.27:0/4222296226 wait complete. 2021-11-03T17:47:13.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.242+0000 7f7d329e3700 1 -- 172.21.15.27:0/1621584077 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d2c0f77d0 msgr2=0x7f7d2c0f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:47:13.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.242+0000 7f7d329e3700 1 --2- 172.21.15.27:0/1621584077 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d2c0f77d0 0x7f7d2c0f7c30 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7f7d28004660 tx=0x7f7d28009b30).stop 2021-11-03T17:47:13.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.242+0000 7f7d329e3700 1 -- 172.21.15.27:0/1621584077 shutdown_connections 2021-11-03T17:47:13.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.242+0000 7f7d329e3700 1 --2- 172.21.15.27:0/1621584077 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d2c0f77d0 0x7f7d2c0f7c30 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:13.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.242+0000 7f7d329e3700 1 --2- 172.21.15.27:0/1621584077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d2c0f6530 0x7f7d2c0f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:13.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.242+0000 7f7d329e3700 1 -- 172.21.15.27:0/1621584077 >> 172.21.15.27:0/1621584077 conn(0x7f7d2c0f1cd0 msgr2=0x7f7d2c0f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:47:13.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.242+0000 7f7d329e3700 1 -- 172.21.15.27:0/1621584077 shutdown_connections 2021-11-03T17:47:13.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.242+0000 7f7d329e3700 1 -- 172.21.15.27:0/1621584077 wait complete. 2021-11-03T17:47:13.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.243+0000 7f7d329e3700 1 Processor -- start 2021-11-03T17:47:13.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.243+0000 7f7d329e3700 1 -- start start 2021-11-03T17:47:13.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.243+0000 7f7d329e3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d2c0f6530 0x7f7d2c106ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:47:13.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.243+0000 7f7d329e3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d2c1073d0 0x7f7d2c1082c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:47:13.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.243+0000 7f7d329e3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c1078a0 con 0x7f7d2c1073d0 2021-11-03T17:47:13.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.243+0000 7f7d329e3700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d2c1079e0 con 0x7f7d2c0f6530 2021-11-03T17:47:13.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.243+0000 7f7d311e0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d2c1073d0 0x7f7d2c1082c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:47:13.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.243+0000 7f7d311e0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d2c1073d0 0x7f7d2c1082c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59408/0 (socket says 172.21.15.27:59408) 2021-11-03T17:47:13.242 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.243+0000 7f7d311e0700 1 -- 172.21.15.27:0/2983812692 learned_addr learned my addr 172.21.15.27:0/2983812692 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:47:13.242 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.243+0000 7f7d319e1700 1 --2- 172.21.15.27:0/2983812692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d2c0f6530 0x7f7d2c106ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:47:13.242 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.243+0000 7f7d311e0700 1 -- 172.21.15.27:0/2983812692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d2c0f6530 msgr2=0x7f7d2c106ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:47:13.242 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.243+0000 7f7d311e0700 1 --2- 172.21.15.27:0/2983812692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d2c0f6530 0x7f7d2c106ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:13.242 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.243+0000 7f7d311e0700 1 -- 172.21.15.27:0/2983812692 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d28005040 con 0x7f7d2c1073d0 2021-11-03T17:47:13.243 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.243+0000 7f7d311e0700 1 --2- 172.21.15.27:0/2983812692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d2c1073d0 0x7f7d2c1082c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f7d28000c00 tx=0x7f7d28009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:47:13.243 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.244+0000 7f7d1e7fc700 1 -- 172.21.15.27:0/2983812692 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d28019070 con 0x7f7d2c1073d0 2021-11-03T17:47:13.243 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.244+0000 7f7d1e7fc700 1 -- 172.21.15.27:0/2983812692 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f7d280097a0 con 0x7f7d2c1073d0 2021-11-03T17:47:13.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.244+0000 7f7d329e3700 1 -- 172.21.15.27:0/2983812692 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d2c108810 con 0x7f7d2c1073d0 2021-11-03T17:47:13.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.244+0000 7f7d1e7fc700 1 -- 172.21.15.27:0/2983812692 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d28014490 con 0x7f7d2c1073d0 2021-11-03T17:47:13.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.244+0000 7f7d329e3700 1 -- 172.21.15.27:0/2983812692 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d2c108d10 con 0x7f7d2c1073d0 2021-11-03T17:47:13.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.245+0000 7f7d329e3700 1 -- 172.21.15.27:0/2983812692 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d2c045ad0 con 0x7f7d2c1073d0 2021-11-03T17:47:13.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.245+0000 7f7d1e7fc700 1 -- 172.21.15.27:0/2983812692 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f7d28013c60 con 0x7f7d2c1073d0 2021-11-03T17:47:13.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.245+0000 7f7d1e7fc700 1 --2- 172.21.15.27:0/2983812692 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7d18064100 0x7f7d180665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:47:13.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.246+0000 7f7d1e7fc700 1 -- 172.21.15.27:0/2983812692 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f7d28081ab0 con 0x7f7d2c1073d0 2021-11-03T17:47:13.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.246+0000 7f7d319e1700 1 --2- 172.21.15.27:0/2983812692 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7d18064100 0x7f7d180665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:47:13.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.246+0000 7f7d319e1700 1 --2- 172.21.15.27:0/2983812692 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7d18064100 0x7f7d180665c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f7d2c0f7630 tx=0x7f7d20009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:47:13.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.249+0000 7f7d1e7fc700 1 -- 172.21.15.27:0/2983812692 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f7d28053db0 con 0x7f7d2c1073d0 2021-11-03T17:47:13.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.456+0000 7f7d329e3700 1 -- 172.21.15.27:0/2983812692 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7d2c108f90 con 0x7f7d2c1073d0 2021-11-03T17:47:13.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.457+0000 7f7d1e7fc700 1 -- 172.21.15.27:0/2983812692 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f7d280539a0 con 0x7f7d2c1073d0 2021-11-03T17:47:13.451 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:47:13.451 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:47:13.452 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:47:13.452 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:47:13.452 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:47:13.453 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:47:13.453 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:47:13.453 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:47:13.453 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:47:13.454 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:47:13.454 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:47:13.454 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:47:13.454 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:47:13.455 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:47:13.455 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:47:13.455 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:47:13.455 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:47:13.455 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:47:13.456 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:47:13.456 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:47:13.456 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:47:13.456 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:47:13.457 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:47:13.457 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:47:13.457 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:47:13.457 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:47:13.457 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:47:13.458 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:47:13.458 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:47:13.458 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:47:13.458 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:47:13.459 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:47:13.459 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:47:13.459 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:47:13.459 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:47:13.460 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:47:13.460 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:47:13.460 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:47:13.460 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:47:13.461 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.459+0000 7f7d329e3700 1 -- 172.21.15.27:0/2983812692 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7d18064100 msgr2=0x7f7d180665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:47:13.461 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.459+0000 7f7d329e3700 1 --2- 172.21.15.27:0/2983812692 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7d18064100 0x7f7d180665c0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 rx=0x7f7d2c0f7630 tx=0x7f7d20009040).stop 2021-11-03T17:47:13.462 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.459+0000 7f7d329e3700 1 -- 172.21.15.27:0/2983812692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d2c1073d0 msgr2=0x7f7d2c1082c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:47:13.462 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.459+0000 7f7d329e3700 1 --2- 172.21.15.27:0/2983812692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d2c1073d0 0x7f7d2c1082c0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f7d28000c00 tx=0x7f7d28009b30).stop 2021-11-03T17:47:13.462 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.459+0000 7f7d329e3700 1 -- 172.21.15.27:0/2983812692 shutdown_connections 2021-11-03T17:47:13.462 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.459+0000 7f7d329e3700 1 --2- 172.21.15.27:0/2983812692 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7d18064100 0x7f7d180665c0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:13.462 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.459+0000 7f7d329e3700 1 --2- 172.21.15.27:0/2983812692 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d2c0f6530 0x7f7d2c106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:13.463 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.459+0000 7f7d329e3700 1 --2- 172.21.15.27:0/2983812692 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d2c1073d0 0x7f7d2c1082c0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:13.463 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.459+0000 7f7d329e3700 1 -- 172.21.15.27:0/2983812692 >> 172.21.15.27:0/2983812692 conn(0x7f7d2c0f1cd0 msgr2=0x7f7d2c0fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:47:13.463 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.459+0000 7f7d329e3700 1 -- 172.21.15.27:0/2983812692 shutdown_connections 2021-11-03T17:47:13.464 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:13.459+0000 7f7d329e3700 1 -- 172.21.15.27:0/2983812692 wait complete. 2021-11-03T17:47:13.464 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:47:13.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:13 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:47:13.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:13 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/4222296226' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:13.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:13 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:47:13.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:13 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/4222296226' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:14.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:14 smithi027 ceph-mon[97244]: pgmap v977: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:14.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:14 smithi027 ceph-mon[97244]: from='client.35112 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:47:14.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:14 smithi027 ceph-mon[97244]: from='client.35116 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:47:14.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:14 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2983812692' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:47:14.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:14 smithi115 ceph-mon[65453]: pgmap v977: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:14.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:14 smithi115 ceph-mon[65453]: from='client.35112 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:47:14.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:14 smithi115 ceph-mon[65453]: from='client.35116 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:47:14.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:14 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2983812692' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:47:15.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1877.403975 secs 2021-11-03T17:47:15.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1877.403975 secs 2021-11-03T17:47:16.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:16 smithi027 ceph-mon[97244]: pgmap v978: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:16.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:16 smithi115 ceph-mon[65453]: pgmap v978: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:18.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:18 smithi027 ceph-mon[97244]: pgmap v979: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:18.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:18 smithi115 ceph-mon[65453]: pgmap v979: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:19.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:19 smithi115 ceph-mon[65453]: pgmap v980: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:19.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:19 smithi027 ceph-mon[97244]: pgmap v980: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:20.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1882.404103 secs 2021-11-03T17:47:20.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1882.404103 secs 2021-11-03T17:47:21.943 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:21 smithi115 ceph-mon[65453]: pgmap v981: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:21.943 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:47:21.964 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:21 smithi027 ceph-mon[97244]: pgmap v981: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:21.964 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:47:23.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:22 smithi115 ceph-mon[65453]: poll took 20062.724 ms: 1 events 2021-11-03T17:47:23.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:47:23.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:47:23.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:23.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:23.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:23.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:23.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:22 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:47:23.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:22 smithi027 ceph-mon[97244]: poll took 20062.724 ms: 1 events 2021-11-03T17:47:23.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:47:23.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:47:23.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:23.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:23.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:23.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:23.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:22 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:47:24.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:23 smithi115 ceph-mon[65453]: pgmap v982: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:24.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:23 smithi027 ceph-mon[97244]: pgmap v982: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:25.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:24 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1887.404232 secs 2021-11-03T17:47:25.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:24 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1887.404232 secs 2021-11-03T17:47:26.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:25 smithi115 ceph-mon[65453]: pgmap v983: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:26.267 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:25 smithi027 ceph-mon[97244]: pgmap v983: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:28.267 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:27 smithi027 ceph-mon[97244]: pgmap v984: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:28.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:27 smithi115 ceph-mon[65453]: pgmap v984: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:30.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:29 smithi027 ceph-mon[97244]: pgmap v985: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:30.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:29 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1892.404360 secs 2021-11-03T17:47:30.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:29 smithi115 ceph-mon[65453]: pgmap v985: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:30.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:29 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1892.404360 secs 2021-11-03T17:47:32.267 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:31 smithi027 ceph-mon[97244]: pgmap v986: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:32.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:31 smithi115 ceph-mon[65453]: pgmap v986: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:33.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:47:33.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:47:33.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:47:33.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:33.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:33.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:33.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:33.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:32 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:47:33.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:47:33.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:47:33.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:47:33.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:33.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:33.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:33.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:33.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:32 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:47:34.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:34 smithi115 ceph-mon[65453]: pgmap v987: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:34.267 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:34 smithi027 ceph-mon[97244]: pgmap v987: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:35.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1897.404818 secs 2021-11-03T17:47:35.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1897.404818 secs 2021-11-03T17:47:36.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:36 smithi027 ceph-mon[97244]: pgmap v988: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:36.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:36 smithi115 ceph-mon[65453]: pgmap v988: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:38.267 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:38 smithi027 ceph-mon[97244]: pgmap v989: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:38.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:38 smithi115 ceph-mon[65453]: pgmap v989: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:40.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:40 smithi027 ceph-mon[97244]: pgmap v990: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:40.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1902.404941 secs 2021-11-03T17:47:40.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:40 smithi115 ceph-mon[65453]: pgmap v990: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:40.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1902.404941 secs 2021-11-03T17:47:42.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:42 smithi027 ceph-mon[97244]: pgmap v991: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:42.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:42 smithi115 ceph-mon[65453]: pgmap v991: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:43.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:47:43.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:47:43.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:47:43.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:43.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:43.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:43.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:43.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:43 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:47:43.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:47:43.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:47:43.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:47:43.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:43.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:43.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:43.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:43.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:43 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:47:43.568 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.572+0000 7f844ddc9700 1 -- 172.21.15.27:0/11165191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84480f55e0 msgr2=0x7f84480f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:47:43.568 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.572+0000 7f844ddc9700 1 --2- 172.21.15.27:0/11165191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84480f55e0 0x7f84480f5a00 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f8438004660 tx=0x7f8438009b30).stop 2021-11-03T17:47:43.569 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.573+0000 7f844ddc9700 1 -- 172.21.15.27:0/11165191 shutdown_connections 2021-11-03T17:47:43.569 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.573+0000 7f844ddc9700 1 --2- 172.21.15.27:0/11165191 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84480f5ef0 0x7f84480f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:43.569 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.573+0000 7f844ddc9700 1 --2- 172.21.15.27:0/11165191 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84480f55e0 0x7f84480f5a00 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:43.569 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.573+0000 7f844ddc9700 1 -- 172.21.15.27:0/11165191 >> 172.21.15.27:0/11165191 conn(0x7f84480f0de0 msgr2=0x7f84480f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:47:43.570 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.573+0000 7f844ddc9700 1 -- 172.21.15.27:0/11165191 shutdown_connections 2021-11-03T17:47:43.570 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.573+0000 7f844ddc9700 1 -- 172.21.15.27:0/11165191 wait complete. 2021-11-03T17:47:43.570 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.573+0000 7f844ddc9700 1 Processor -- start 2021-11-03T17:47:43.570 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.573+0000 7f844ddc9700 1 -- start start 2021-11-03T17:47:43.571 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.574+0000 7f844ddc9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84480f55e0 0x7f8448105e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:47:43.571 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.574+0000 7f844ddc9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84480f5ef0 0x7f8448106380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:47:43.571 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.574+0000 7f844ddc9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8448106930 con 0x7f84480f5ef0 2021-11-03T17:47:43.571 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.574+0000 7f844ddc9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8448106a70 con 0x7f84480f55e0 2021-11-03T17:47:43.574 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.574+0000 7f8446ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84480f5ef0 0x7f8448106380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:47:43.574 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.574+0000 7f8446ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84480f5ef0 0x7f8448106380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59428/0 (socket says 172.21.15.27:59428) 2021-11-03T17:47:43.574 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.574+0000 7f8446ffd700 1 -- 172.21.15.27:0/2596935135 learned_addr learned my addr 172.21.15.27:0/2596935135 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:47:43.574 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.574+0000 7f84477fe700 1 --2- 172.21.15.27:0/2596935135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84480f55e0 0x7f8448105e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:47:43.575 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.574+0000 7f8446ffd700 1 -- 172.21.15.27:0/2596935135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84480f55e0 msgr2=0x7f8448105e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:47:43.575 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.574+0000 7f8446ffd700 1 --2- 172.21.15.27:0/2596935135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84480f55e0 0x7f8448105e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:43.575 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.574+0000 7f8446ffd700 1 -- 172.21.15.27:0/2596935135 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8438005040 con 0x7f84480f5ef0 2021-11-03T17:47:43.575 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.574+0000 7f8446ffd700 1 --2- 172.21.15.27:0/2596935135 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84480f5ef0 0x7f8448106380 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7f843c0026d0 tx=0x7f843c009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:47:43.576 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.574+0000 7f842ffff700 1 -- 172.21.15.27:0/2596935135 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f843c0079c0 con 0x7f84480f5ef0 2021-11-03T17:47:43.576 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.574+0000 7f842ffff700 1 -- 172.21.15.27:0/2596935135 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f843c00f040 con 0x7f84480f5ef0 2021-11-03T17:47:43.576 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.574+0000 7f844ddc9700 1 -- 172.21.15.27:0/2596935135 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8448107410 con 0x7f84480f5ef0 2021-11-03T17:47:43.576 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.574+0000 7f842ffff700 1 -- 172.21.15.27:0/2596935135 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f843c0148b0 con 0x7f84480f5ef0 2021-11-03T17:47:43.577 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.574+0000 7f844ddc9700 1 -- 172.21.15.27:0/2596935135 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84481079d0 con 0x7f84480f5ef0 2021-11-03T17:47:43.577 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.575+0000 7f844ddc9700 1 -- 172.21.15.27:0/2596935135 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8448045ad0 con 0x7f84480f5ef0 2021-11-03T17:47:43.577 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.576+0000 7f842ffff700 1 -- 172.21.15.27:0/2596935135 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f843c007b20 con 0x7f84480f5ef0 2021-11-03T17:47:43.577 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.576+0000 7f842ffff700 1 --2- 172.21.15.27:0/2596935135 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f84300640b0 0x7f8430066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:47:43.578 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.576+0000 7f842ffff700 1 -- 172.21.15.27:0/2596935135 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f843c0806c0 con 0x7f84480f5ef0 2021-11-03T17:47:43.578 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.577+0000 7f84477fe700 1 --2- 172.21.15.27:0/2596935135 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f84300640b0 0x7f8430066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:47:43.578 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.577+0000 7f84477fe700 1 --2- 172.21.15.27:0/2596935135 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f84300640b0 0x7f8430066570 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f8438004630 tx=0x7f8438009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:47:43.579 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.581+0000 7f842ffff700 1 -- 172.21.15.27:0/2596935135 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f843c0529c0 con 0x7f84480f5ef0 2021-11-03T17:47:43.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.758+0000 7f844ddc9700 1 -- 172.21.15.27:0/2596935135 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f84480fdf80 con 0x7f84300640b0 2021-11-03T17:47:43.754 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.759+0000 7f842ffff700 1 -- 172.21.15.27:0/2596935135 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f8438019040 con 0x7f84300640b0 2021-11-03T17:47:43.756 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.761+0000 7f844ddc9700 1 -- 172.21.15.27:0/2596935135 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f84300640b0 msgr2=0x7f8430066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:47:43.756 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.761+0000 7f844ddc9700 1 --2- 172.21.15.27:0/2596935135 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f84300640b0 0x7f8430066570 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 rx=0x7f8438004630 tx=0x7f8438009b30).stop 2021-11-03T17:47:43.756 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.761+0000 7f844ddc9700 1 -- 172.21.15.27:0/2596935135 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84480f5ef0 msgr2=0x7f8448106380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:47:43.756 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.761+0000 7f844ddc9700 1 --2- 172.21.15.27:0/2596935135 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84480f5ef0 0x7f8448106380 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7f843c0026d0 tx=0x7f843c009de0).stop 2021-11-03T17:47:43.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.761+0000 7f844ddc9700 1 -- 172.21.15.27:0/2596935135 shutdown_connections 2021-11-03T17:47:43.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.761+0000 7f844ddc9700 1 --2- 172.21.15.27:0/2596935135 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f84300640b0 0x7f8430066570 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:43.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.761+0000 7f844ddc9700 1 --2- 172.21.15.27:0/2596935135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84480f55e0 0x7f8448105e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:43.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.761+0000 7f844ddc9700 1 --2- 172.21.15.27:0/2596935135 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84480f5ef0 0x7f8448106380 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:43.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.761+0000 7f844ddc9700 1 -- 172.21.15.27:0/2596935135 >> 172.21.15.27:0/2596935135 conn(0x7f84480f0de0 msgr2=0x7f84480fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:47:43.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.761+0000 7f844ddc9700 1 -- 172.21.15.27:0/2596935135 shutdown_connections 2021-11-03T17:47:43.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.761+0000 7f844ddc9700 1 -- 172.21.15.27:0/2596935135 wait complete. 2021-11-03T17:47:43.766 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:47:43.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.861+0000 7f0071876700 1 -- 172.21.15.27:0/3527994032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f006c0f5ef0 msgr2=0x7f006c0f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:47:43.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.861+0000 7f0071876700 1 --2- 172.21.15.27:0/3527994032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f006c0f5ef0 0x7f006c0f6920 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7f0060004660 tx=0x7f0060009b30).stop 2021-11-03T17:47:43.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.861+0000 7f0071876700 1 -- 172.21.15.27:0/3527994032 shutdown_connections 2021-11-03T17:47:43.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.861+0000 7f0071876700 1 --2- 172.21.15.27:0/3527994032 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f006c0f5ef0 0x7f006c0f6920 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:43.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.861+0000 7f0071876700 1 --2- 172.21.15.27:0/3527994032 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f006c0f55e0 0x7f006c0f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:43.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.861+0000 7f0071876700 1 -- 172.21.15.27:0/3527994032 >> 172.21.15.27:0/3527994032 conn(0x7f006c0f0de0 msgr2=0x7f006c0f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:47:43.860 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.861+0000 7f0071876700 1 -- 172.21.15.27:0/3527994032 shutdown_connections 2021-11-03T17:47:43.860 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.861+0000 7f0071876700 1 -- 172.21.15.27:0/3527994032 wait complete. 2021-11-03T17:47:43.860 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.862+0000 7f0071876700 1 Processor -- start 2021-11-03T17:47:43.860 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.862+0000 7f0071876700 1 -- start start 2021-11-03T17:47:43.861 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.862+0000 7f0071876700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f006c0f55e0 0x7f006c10c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:47:43.861 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.862+0000 7f0071876700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f006c0f5ef0 0x7f006c10e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:47:43.861 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.862+0000 7f0071876700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006c0603b0 con 0x7f006c0f5ef0 2021-11-03T17:47:43.861 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.862+0000 7f0071876700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006c0604f0 con 0x7f006c0f55e0 2021-11-03T17:47:43.862 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.862+0000 7f006a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f006c0f5ef0 0x7f006c10e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:47:43.862 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.862+0000 7f006a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f006c0f5ef0 0x7f006c10e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59438/0 (socket says 172.21.15.27:59438) 2021-11-03T17:47:43.862 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.862+0000 7f006a7fc700 1 -- 172.21.15.27:0/1445342085 learned_addr learned my addr 172.21.15.27:0/1445342085 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:47:43.862 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.862+0000 7f006affd700 1 --2- 172.21.15.27:0/1445342085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f006c0f55e0 0x7f006c10c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:47:43.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.862+0000 7f006a7fc700 1 -- 172.21.15.27:0/1445342085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f006c0f55e0 msgr2=0x7f006c10c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:47:43.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.862+0000 7f006a7fc700 1 --2- 172.21.15.27:0/1445342085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f006c0f55e0 0x7f006c10c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:43.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.862+0000 7f006a7fc700 1 -- 172.21.15.27:0/1445342085 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0060005040 con 0x7f006c0f5ef0 2021-11-03T17:47:43.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.863+0000 7f006a7fc700 1 --2- 172.21.15.27:0/1445342085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f006c0f5ef0 0x7f006c10e5b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7f0060009990 tx=0x7f0060009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:47:43.864 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.863+0000 7f00537fe700 1 -- 172.21.15.27:0/1445342085 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0060018070 con 0x7f006c0f5ef0 2021-11-03T17:47:43.864 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.863+0000 7f00537fe700 1 -- 172.21.15.27:0/1445342085 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f0060007ba0 con 0x7f006c0f5ef0 2021-11-03T17:47:43.864 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.863+0000 7f0071876700 1 -- 172.21.15.27:0/1445342085 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f006c10eaa0 con 0x7f006c0f5ef0 2021-11-03T17:47:43.864 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.863+0000 7f00537fe700 1 -- 172.21.15.27:0/1445342085 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f006001c3c0 con 0x7f006c0f5ef0 2021-11-03T17:47:43.864 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.863+0000 7f0071876700 1 -- 172.21.15.27:0/1445342085 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f006c10ee70 con 0x7f006c0f5ef0 2021-11-03T17:47:43.866 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.864+0000 7f0071876700 1 -- 172.21.15.27:0/1445342085 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f006c0f4a70 con 0x7f006c0f5ef0 2021-11-03T17:47:43.866 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.864+0000 7f00537fe700 1 -- 172.21.15.27:0/1445342085 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f006001d420 con 0x7f006c0f5ef0 2021-11-03T17:47:43.866 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.865+0000 7f00537fe700 1 --2- 172.21.15.27:0/1445342085 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f00540640b0 0x7f0054066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:47:43.866 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.865+0000 7f00537fe700 1 -- 172.21.15.27:0/1445342085 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f0060082450 con 0x7f006c0f5ef0 2021-11-03T17:47:43.867 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.865+0000 7f006affd700 1 --2- 172.21.15.27:0/1445342085 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f00540640b0 0x7f0054066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:47:43.867 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.865+0000 7f006affd700 1 --2- 172.21.15.27:0/1445342085 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f00540640b0 0x7f0054066570 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f006c060140 tx=0x7f005c00b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:47:43.867 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:43.869+0000 7f00537fe700 1 -- 172.21.15.27:0/1445342085 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f00600546f0 con 0x7f006c0f5ef0 2021-11-03T17:47:44.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.041+0000 7f0071876700 1 -- 172.21.15.27:0/1445342085 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f006c0fded0 con 0x7f00540640b0 2021-11-03T17:47:44.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.056+0000 7f00537fe700 1 -- 172.21.15.27:0/1445342085 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f006c0fded0 con 0x7f00540640b0 2021-11-03T17:47:44.051 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:47:44.051 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (39m) 9m ago 40m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:47:44.052 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (30m) 9m ago 40m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:47:44.052 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (30m) 8m ago 39m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:47:44.052 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (39m) 9m ago 40m 41.3M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:47:44.052 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (32m) 9m ago 35m 21.1M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:47:44.053 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (35m) 9m ago 35m 3843M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:47:44.053 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (31m) 8m ago 35m 20.4M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:47:44.053 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (33m) 8m ago 35m 22.1M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:47:44.053 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (33m) 9m ago 41m 517M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:47:44.054 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (32m) 8m ago 39m 419M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:47:44.054 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (31m) 9m ago 41m 159M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:47:44.054 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (31m) 8m ago 39m 82.5M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:47:44.054 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (40m) 9m ago 40m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:47:44.054 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (39m) 8m ago 39m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:47:44.055 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (30m) 9m ago 38m 311M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:47:44.055 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (30m) 9m ago 38m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:47:44.055 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (29m) 9m ago 38m 248M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:47:44.055 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (29m) 8m ago 37m 354M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:47:44.056 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (28m) 8m ago 37m 285M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:47:44.056 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (28m) 8m ago 37m 287M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:47:44.056 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (39m) 9m ago 39m 50.5M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:47:44.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.058+0000 7f0071876700 1 -- 172.21.15.27:0/1445342085 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f00540640b0 msgr2=0x7f0054066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:47:44.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.058+0000 7f0071876700 1 --2- 172.21.15.27:0/1445342085 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f00540640b0 0x7f0054066570 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 rx=0x7f006c060140 tx=0x7f005c00b040).stop 2021-11-03T17:47:44.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.058+0000 7f0071876700 1 -- 172.21.15.27:0/1445342085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f006c0f5ef0 msgr2=0x7f006c10e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:47:44.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.058+0000 7f0071876700 1 --2- 172.21.15.27:0/1445342085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f006c0f5ef0 0x7f006c10e5b0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7f0060009990 tx=0x7f0060009b30).stop 2021-11-03T17:47:44.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.058+0000 7f0071876700 1 -- 172.21.15.27:0/1445342085 shutdown_connections 2021-11-03T17:47:44.059 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.058+0000 7f0071876700 1 --2- 172.21.15.27:0/1445342085 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f00540640b0 0x7f0054066570 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:44.059 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.058+0000 7f0071876700 1 --2- 172.21.15.27:0/1445342085 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f006c0f55e0 0x7f006c10c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:44.060 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.058+0000 7f0071876700 1 --2- 172.21.15.27:0/1445342085 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f006c0f5ef0 0x7f006c10e5b0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:44.060 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.058+0000 7f0071876700 1 -- 172.21.15.27:0/1445342085 >> 172.21.15.27:0/1445342085 conn(0x7f006c0f0de0 msgr2=0x7f006c0fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:47:44.060 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.058+0000 7f0071876700 1 -- 172.21.15.27:0/1445342085 shutdown_connections 2021-11-03T17:47:44.060 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.058+0000 7f0071876700 1 -- 172.21.15.27:0/1445342085 wait complete. 2021-11-03T17:47:44.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.157+0000 7f9b015dd700 1 -- 172.21.15.27:0/1808688693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9afc0f6530 msgr2=0x7f9afc0f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:47:44.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.157+0000 7f9b015dd700 1 --2- 172.21.15.27:0/1808688693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9afc0f6530 0x7f9afc0f6950 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7f9aec004660 tx=0x7f9aec009b30).stop 2021-11-03T17:47:44.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.157+0000 7f9b015dd700 1 -- 172.21.15.27:0/1808688693 shutdown_connections 2021-11-03T17:47:44.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.157+0000 7f9b015dd700 1 --2- 172.21.15.27:0/1808688693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9afc0f77d0 0x7f9afc0f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:44.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.157+0000 7f9b015dd700 1 --2- 172.21.15.27:0/1808688693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9afc0f6530 0x7f9afc0f6950 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:44.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.157+0000 7f9b015dd700 1 -- 172.21.15.27:0/1808688693 >> 172.21.15.27:0/1808688693 conn(0x7f9afc0f1cd0 msgr2=0x7f9afc0f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:47:44.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.158+0000 7f9b015dd700 1 -- 172.21.15.27:0/1808688693 shutdown_connections 2021-11-03T17:47:44.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.158+0000 7f9b015dd700 1 -- 172.21.15.27:0/1808688693 wait complete. 2021-11-03T17:47:44.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.158+0000 7f9b015dd700 1 Processor -- start 2021-11-03T17:47:44.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.158+0000 7f9b015dd700 1 -- start start 2021-11-03T17:47:44.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.158+0000 7f9b015dd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9afc0f6530 0x7f9afc106db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:47:44.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.158+0000 7f9b015dd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9afc0f77d0 0x7f9afc1072a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:47:44.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.158+0000 7f9b015dd700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc1078a0 con 0x7f9afc0f6530 2021-11-03T17:47:44.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.158+0000 7f9b015dd700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc1080e0 con 0x7f9afc0f77d0 2021-11-03T17:47:44.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.159+0000 7f9afbfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9afc0f6530 0x7f9afc106db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:47:44.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.159+0000 7f9afbfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9afc0f6530 0x7f9afc106db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59446/0 (socket says 172.21.15.27:59446) 2021-11-03T17:47:44.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.159+0000 7f9afbfff700 1 -- 172.21.15.27:0/2295070144 learned_addr learned my addr 172.21.15.27:0/2295070144 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:47:44.160 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.159+0000 7f9afb7fe700 1 --2- 172.21.15.27:0/2295070144 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9afc0f77d0 0x7f9afc1072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:47:44.160 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.159+0000 7f9afbfff700 1 -- 172.21.15.27:0/2295070144 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9afc0f77d0 msgr2=0x7f9afc1072a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:47:44.160 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.159+0000 7f9afbfff700 1 --2- 172.21.15.27:0/2295070144 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9afc0f77d0 0x7f9afc1072a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:44.160 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.159+0000 7f9afbfff700 1 -- 172.21.15.27:0/2295070144 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aec005040 con 0x7f9afc0f6530 2021-11-03T17:47:44.161 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.159+0000 7f9afbfff700 1 --2- 172.21.15.27:0/2295070144 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9afc0f6530 0x7f9afc106db0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7f9aec004f60 tx=0x7f9aec009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:47:44.161 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.159+0000 7f9af8ff9700 1 -- 172.21.15.27:0/2295070144 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9aec018070 con 0x7f9afc0f6530 2021-11-03T17:47:44.161 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.159+0000 7f9af8ff9700 1 -- 172.21.15.27:0/2295070144 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f9aec009770 con 0x7f9afc0f6530 2021-11-03T17:47:44.161 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.159+0000 7f9b015dd700 1 -- 172.21.15.27:0/2295070144 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9afc108360 con 0x7f9afc0f6530 2021-11-03T17:47:44.162 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.159+0000 7f9af8ff9700 1 -- 172.21.15.27:0/2295070144 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9aec01c3c0 con 0x7f9afc0f6530 2021-11-03T17:47:44.162 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.159+0000 7f9b015dd700 1 -- 172.21.15.27:0/2295070144 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9afc108840 con 0x7f9afc0f6530 2021-11-03T17:47:44.162 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.160+0000 7f9b015dd700 1 -- 172.21.15.27:0/2295070144 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9afc059100 con 0x7f9afc0f6530 2021-11-03T17:47:44.162 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.161+0000 7f9af8ff9700 1 -- 172.21.15.27:0/2295070144 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f9aec01d420 con 0x7f9afc0f6530 2021-11-03T17:47:44.163 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.162+0000 7f9af8ff9700 1 --2- 172.21.15.27:0/2295070144 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9ae4064100 0x7f9ae40665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:47:44.163 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.162+0000 7f9af8ff9700 1 -- 172.21.15.27:0/2295070144 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f9aec082960 con 0x7f9afc0f6530 2021-11-03T17:47:44.163 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.162+0000 7f9afb7fe700 1 --2- 172.21.15.27:0/2295070144 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9ae4064100 0x7f9ae40665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:47:44.163 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.162+0000 7f9afb7fe700 1 --2- 172.21.15.27:0/2295070144 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9ae4064100 0x7f9ae40665c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f9afc106b40 tx=0x7f9af0009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:47:44.164 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.165+0000 7f9af8ff9700 1 -- 172.21.15.27:0/2295070144 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f9aec054c60 con 0x7f9afc0f6530 2021-11-03T17:47:44.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:44 smithi115 ceph-mon[65453]: pgmap v992: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:44.389 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.394+0000 7f9b015dd700 1 -- 172.21.15.27:0/2295070144 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9afc0fedf0 con 0x7f9afc0f6530 2021-11-03T17:47:44.389 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.394+0000 7f9af8ff9700 1 -- 172.21.15.27:0/2295070144 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f9aec054850 con 0x7f9afc0f6530 2021-11-03T17:47:44.390 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:47:44.390 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:47:44.390 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:47:44.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:47:44.391 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:47:44.391 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:47:44.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:47:44.392 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:47:44.392 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:47:44.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:47:44.392 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:47:44.393 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:47:44.393 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:47:44.393 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:47:44.393 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:47:44.393 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:47:44.394 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:47:44.394 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:47:44.395 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.396+0000 7f9b015dd700 1 -- 172.21.15.27:0/2295070144 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9ae4064100 msgr2=0x7f9ae40665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:47:44.395 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.396+0000 7f9b015dd700 1 --2- 172.21.15.27:0/2295070144 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9ae4064100 0x7f9ae40665c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 rx=0x7f9afc106b40 tx=0x7f9af0009040).stop 2021-11-03T17:47:44.395 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.396+0000 7f9b015dd700 1 -- 172.21.15.27:0/2295070144 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9afc0f6530 msgr2=0x7f9afc106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:47:44.395 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.396+0000 7f9b015dd700 1 --2- 172.21.15.27:0/2295070144 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9afc0f6530 0x7f9afc106db0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7f9aec004f60 tx=0x7f9aec009b30).stop 2021-11-03T17:47:44.396 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.396+0000 7f9b015dd700 1 -- 172.21.15.27:0/2295070144 shutdown_connections 2021-11-03T17:47:44.396 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.396+0000 7f9b015dd700 1 --2- 172.21.15.27:0/2295070144 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9ae4064100 0x7f9ae40665c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:44.396 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.396+0000 7f9b015dd700 1 --2- 172.21.15.27:0/2295070144 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9afc0f6530 0x7f9afc106db0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:44.396 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.396+0000 7f9b015dd700 1 --2- 172.21.15.27:0/2295070144 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9afc0f77d0 0x7f9afc1072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:44.397 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.396+0000 7f9b015dd700 1 -- 172.21.15.27:0/2295070144 >> 172.21.15.27:0/2295070144 conn(0x7f9afc0f1cd0 msgr2=0x7f9afc0fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:47:44.397 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.396+0000 7f9b015dd700 1 -- 172.21.15.27:0/2295070144 shutdown_connections 2021-11-03T17:47:44.397 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.396+0000 7f9b015dd700 1 -- 172.21.15.27:0/2295070144 wait complete. 2021-11-03T17:47:44.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.497+0000 7f3b1342a700 1 -- 172.21.15.27:0/1214607528 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b0c0f6ff0 msgr2=0x7f3b0c0f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:47:44.496 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.497+0000 7f3b1342a700 1 --2- 172.21.15.27:0/1214607528 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b0c0f6ff0 0x7f3b0c0f7a20 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7f3b08004660 tx=0x7f3b08009b30).stop 2021-11-03T17:47:44.496 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.497+0000 7f3b1342a700 1 -- 172.21.15.27:0/1214607528 shutdown_connections 2021-11-03T17:47:44.496 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.497+0000 7f3b1342a700 1 --2- 172.21.15.27:0/1214607528 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b0c0f6ff0 0x7f3b0c0f7a20 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:44.496 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.497+0000 7f3b1342a700 1 --2- 172.21.15.27:0/1214607528 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b0c0f66e0 0x7f3b0c0f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:44.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.497+0000 7f3b1342a700 1 -- 172.21.15.27:0/1214607528 >> 172.21.15.27:0/1214607528 conn(0x7f3b0c0f1f00 msgr2=0x7f3b0c0f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:47:44.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.497+0000 7f3b1342a700 1 -- 172.21.15.27:0/1214607528 shutdown_connections 2021-11-03T17:47:44.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.497+0000 7f3b1342a700 1 -- 172.21.15.27:0/1214607528 wait complete. 2021-11-03T17:47:44.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.497+0000 7f3b1342a700 1 Processor -- start 2021-11-03T17:47:44.498 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.498+0000 7f3b1342a700 1 -- start start 2021-11-03T17:47:44.498 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.498+0000 7f3b1342a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b0c0f66e0 0x7f3b0c06b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:47:44.498 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.498+0000 7f3b1342a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b0c0f6ff0 0x7f3b0c069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:47:44.498 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.498+0000 7f3b1342a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c06a380 con 0x7f3b0c0f66e0 2021-11-03T17:47:44.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.498+0000 7f3b1342a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b0c06a4c0 con 0x7f3b0c0f6ff0 2021-11-03T17:47:44.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.498+0000 7f3b111c6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b0c0f66e0 0x7f3b0c06b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:47:44.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.498+0000 7f3b111c6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b0c0f66e0 0x7f3b0c06b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59456/0 (socket says 172.21.15.27:59456) 2021-11-03T17:47:44.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.498+0000 7f3b111c6700 1 -- 172.21.15.27:0/3772970362 learned_addr learned my addr 172.21.15.27:0/3772970362 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:47:44.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.498+0000 7f3b109c5700 1 --2- 172.21.15.27:0/3772970362 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b0c0f6ff0 0x7f3b0c069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:47:44.500 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.498+0000 7f3b111c6700 1 -- 172.21.15.27:0/3772970362 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b0c0f6ff0 msgr2=0x7f3b0c069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:47:44.500 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.498+0000 7f3b111c6700 1 --2- 172.21.15.27:0/3772970362 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b0c0f6ff0 0x7f3b0c069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:44.500 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.498+0000 7f3b111c6700 1 -- 172.21.15.27:0/3772970362 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b08005040 con 0x7f3b0c0f66e0 2021-11-03T17:47:44.500 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.498+0000 7f3b111c6700 1 --2- 172.21.15.27:0/3772970362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b0c0f66e0 0x7f3b0c06b9a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f3afc0026d0 tx=0x7f3afc009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:47:44.501 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.498+0000 7f3b01ffb700 1 -- 172.21.15.27:0/3772970362 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3afc009320 con 0x7f3b0c0f66e0 2021-11-03T17:47:44.501 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.498+0000 7f3b01ffb700 1 -- 172.21.15.27:0/3772970362 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f3afc00f040 con 0x7f3b0c0f66e0 2021-11-03T17:47:44.501 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.498+0000 7f3b1342a700 1 -- 172.21.15.27:0/3772970362 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b0c06a7a0 con 0x7f3b0c0f66e0 2021-11-03T17:47:44.501 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.498+0000 7f3b01ffb700 1 -- 172.21.15.27:0/3772970362 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3afc0148f0 con 0x7f3b0c0f66e0 2021-11-03T17:47:44.502 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.499+0000 7f3b1342a700 1 -- 172.21.15.27:0/3772970362 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b0c10efc0 con 0x7f3b0c0f66e0 2021-11-03T17:47:44.502 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.500+0000 7f3b1342a700 1 -- 172.21.15.27:0/3772970362 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b0c045ad0 con 0x7f3b0c0f66e0 2021-11-03T17:47:44.502 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.500+0000 7f3b01ffb700 1 -- 172.21.15.27:0/3772970362 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f3afc01b060 con 0x7f3b0c0f66e0 2021-11-03T17:47:44.502 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.500+0000 7f3b01ffb700 1 --2- 172.21.15.27:0/3772970362 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3af80640b0 0x7f3af8066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:47:44.503 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.500+0000 7f3b01ffb700 1 -- 172.21.15.27:0/3772970362 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f3afc080790 con 0x7f3b0c0f66e0 2021-11-03T17:47:44.503 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.500+0000 7f3b109c5700 1 --2- 172.21.15.27:0/3772970362 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3af80640b0 0x7f3af8066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:47:44.503 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.501+0000 7f3b109c5700 1 --2- 172.21.15.27:0/3772970362 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3af80640b0 0x7f3af8066570 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f3b08004630 tx=0x7f3b08009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:47:44.504 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.504+0000 7f3b01ffb700 1 -- 172.21.15.27:0/3772970362 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f3afc0529a0 con 0x7f3b0c0f66e0 2021-11-03T17:47:44.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:44 smithi027 ceph-mon[97244]: pgmap v992: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:44.707 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.712+0000 7f3b1342a700 1 -- 172.21.15.27:0/3772970362 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3b0c0fef90 con 0x7f3b0c0f66e0 2021-11-03T17:47:44.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.713+0000 7f3b01ffb700 1 -- 172.21.15.27:0/3772970362 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f3afc052590 con 0x7f3b0c0f66e0 2021-11-03T17:47:44.708 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:47:44.709 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:47:44.709 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:47:44.709 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:47:44.709 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:47:44.710 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:47:44.710 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:47:44.710 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:47:44.710 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:47:44.711 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:47:44.711 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:47:44.711 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:47:44.711 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:47:44.711 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:47:44.712 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:47:44.712 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:47:44.712 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:47:44.712 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:47:44.713 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:47:44.713 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:47:44.713 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:47:44.713 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:47:44.713 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:47:44.714 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:47:44.714 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:47:44.714 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:47:44.714 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:47:44.715 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:47:44.715 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:47:44.715 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:47:44.715 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:47:44.716 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:47:44.716 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:47:44.716 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:47:44.717 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:47:44.717 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:47:44.717 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:47:44.717 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:47:44.718 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:47:44.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.714+0000 7f3b1342a700 1 -- 172.21.15.27:0/3772970362 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3af80640b0 msgr2=0x7f3af8066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:47:44.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.714+0000 7f3b1342a700 1 --2- 172.21.15.27:0/3772970362 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3af80640b0 0x7f3af8066570 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f3b08004630 tx=0x7f3b08009b30).stop 2021-11-03T17:47:44.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.714+0000 7f3b1342a700 1 -- 172.21.15.27:0/3772970362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b0c0f66e0 msgr2=0x7f3b0c06b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:47:44.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.714+0000 7f3b1342a700 1 --2- 172.21.15.27:0/3772970362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b0c0f66e0 0x7f3b0c06b9a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f3afc0026d0 tx=0x7f3afc009de0).stop 2021-11-03T17:47:44.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.714+0000 7f3b1342a700 1 -- 172.21.15.27:0/3772970362 shutdown_connections 2021-11-03T17:47:44.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.714+0000 7f3b1342a700 1 --2- 172.21.15.27:0/3772970362 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3af80640b0 0x7f3af8066570 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:44.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.714+0000 7f3b1342a700 1 --2- 172.21.15.27:0/3772970362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b0c0f66e0 0x7f3b0c06b9a0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:44.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.714+0000 7f3b1342a700 1 --2- 172.21.15.27:0/3772970362 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b0c0f6ff0 0x7f3b0c069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:47:44.721 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.714+0000 7f3b1342a700 1 -- 172.21.15.27:0/3772970362 >> 172.21.15.27:0/3772970362 conn(0x7f3b0c0f1f00 msgr2=0x7f3b0c0fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:47:44.721 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.715+0000 7f3b1342a700 1 -- 172.21.15.27:0/3772970362 shutdown_connections 2021-11-03T17:47:44.721 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:47:44.715+0000 7f3b1342a700 1 -- 172.21.15.27:0/3772970362 wait complete. 2021-11-03T17:47:44.721 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:47:45.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:45 smithi115 ceph-mon[65453]: from='client.35128 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:47:45.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:45 smithi115 ceph-mon[65453]: from='client.35132 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:47:45.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1907.405006 secs 2021-11-03T17:47:45.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:45 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2295070144' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:45.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:45 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3772970362' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:47:45.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:45 smithi027 ceph-mon[97244]: from='client.35128 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:47:45.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:45 smithi027 ceph-mon[97244]: from='client.35132 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:47:45.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1907.405006 secs 2021-11-03T17:47:45.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:45 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2295070144' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:45.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:45 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3772970362' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:47:46.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:46 smithi115 ceph-mon[65453]: pgmap v993: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:46.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:46 smithi027 ceph-mon[97244]: pgmap v993: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:48.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:48 smithi115 ceph-mon[65453]: pgmap v994: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:48.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:48 smithi027 ceph-mon[97244]: pgmap v994: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:50.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:50 smithi115 ceph-mon[65453]: pgmap v995: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:50.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1912.405148 secs 2021-11-03T17:47:50.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:50 smithi027 ceph-mon[97244]: pgmap v995: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:50.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1912.405148 secs 2021-11-03T17:47:52.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:52 smithi115 ceph-mon[65453]: pgmap v996: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:52.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:52 smithi027 ceph-mon[97244]: pgmap v996: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:53.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:53 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:47:53.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:53 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:47:53.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:53 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:47:53.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:53 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:53.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:53 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:53.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:53 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:53.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:53 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:53.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:53 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:47:53.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:53 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:47:53.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:53 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:47:53.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:53 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:47:53.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:53 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:53.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:53 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:53.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:53 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:53.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:53 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:47:53.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:53 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:47:54.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:54 smithi115 ceph-mon[65453]: pgmap v997: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:54.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:54 smithi027 ceph-mon[97244]: pgmap v997: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:55.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1917.405282 secs 2021-11-03T17:47:55.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1917.405282 secs 2021-11-03T17:47:56.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:56 smithi115 ceph-mon[65453]: pgmap v998: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:56.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:47:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:47:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:56 smithi027 ceph-mon[97244]: pgmap v998: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:47:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:47:58.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:47:58 smithi115 ceph-mon[65453]: pgmap v999: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:47:58.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:47:58 smithi027 ceph-mon[97244]: pgmap v999: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:00.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:00 smithi115 ceph-mon[65453]: pgmap v1000: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:00.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1922.405396 secs 2021-11-03T17:48:00.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:00 smithi027 ceph-mon[97244]: pgmap v1000: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:00.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1922.405396 secs 2021-11-03T17:48:02.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:02 smithi115 ceph-mon[65453]: pgmap v1001: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:02.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:02 smithi027 ceph-mon[97244]: pgmap v1001: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:03.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:48:03.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:48:03.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:48:03.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:03.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:03.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:03.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:03.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:03 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:48:03.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:48:03.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:48:03.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:48:03.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:03.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:03.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:03.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:03.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:03 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:48:04.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:04 smithi115 ceph-mon[65453]: pgmap v1002: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:04.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:04 smithi027 ceph-mon[97244]: pgmap v1002: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:05.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1927.405537 secs 2021-11-03T17:48:05.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1927.405537 secs 2021-11-03T17:48:06.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:06 smithi115 ceph-mon[65453]: pgmap v1003: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:06.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:06 smithi027 ceph-mon[97244]: pgmap v1003: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:08.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:08 smithi115 ceph-mon[65453]: pgmap v1004: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:08.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:08 smithi027 ceph-mon[97244]: pgmap v1004: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:10.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:10 smithi115 ceph-mon[65453]: pgmap v1005: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:10.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1932.405662 secs 2021-11-03T17:48:10.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:10 smithi027 ceph-mon[97244]: pgmap v1005: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:10.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1932.405662 secs 2021-11-03T17:48:12.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:12 smithi115 ceph-mon[65453]: pgmap v1006: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:12.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:12 smithi027 ceph-mon[97244]: pgmap v1006: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:13.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:48:13.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:48:13.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:48:13.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:13.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:13.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:13.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:13.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:13 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:48:13.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:48:13.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:48:13.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:48:13.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:13.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:13.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:13.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:13.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:13 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:48:14.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:14 smithi115 ceph-mon[65453]: pgmap v1007: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:14.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:14 smithi027 ceph-mon[97244]: pgmap v1007: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:14.827 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:14.830+0000 7f428eab7700 1 -- 172.21.15.27:0/2901641737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42880f66e0 msgr2=0x7f42880f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:48:14.827 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:14.830+0000 7f428eab7700 1 --2- 172.21.15.27:0/2901641737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42880f66e0 0x7f42880f6b40 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7f4284004660 tx=0x7f4284009b30).stop 2021-11-03T17:48:14.827 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:14.830+0000 7f428eab7700 1 -- 172.21.15.27:0/2901641737 shutdown_connections 2021-11-03T17:48:14.828 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:14.830+0000 7f428eab7700 1 --2- 172.21.15.27:0/2901641737 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42880f66e0 0x7f42880f6b40 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:14.828 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:14.830+0000 7f428eab7700 1 --2- 172.21.15.27:0/2901641737 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42880f5440 0x7f42880f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:14.828 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:14.830+0000 7f428eab7700 1 -- 172.21.15.27:0/2901641737 >> 172.21.15.27:0/2901641737 conn(0x7f42880f0ba0 msgr2=0x7f42880f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:48:14.829 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:14.830+0000 7f428eab7700 1 -- 172.21.15.27:0/2901641737 shutdown_connections 2021-11-03T17:48:14.829 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:14.830+0000 7f428eab7700 1 -- 172.21.15.27:0/2901641737 wait complete. 2021-11-03T17:48:14.829 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:14.830+0000 7f428eab7700 1 Processor -- start 2021-11-03T17:48:14.829 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:14.830+0000 7f428eab7700 1 -- start start 2021-11-03T17:48:14.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:14.831+0000 7f428eab7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42880f5440 0x7f4288109a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:48:14.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:14.831+0000 7f428eab7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42880f66e0 0x7f428810bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:48:14.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:14.831+0000 7f428eab7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4288060400 con 0x7f42880f66e0 2021-11-03T17:48:14.831 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:14.831+0000 7f428eab7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4288060540 con 0x7f42880f5440 2021-11-03T17:48:14.831 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:14.831+0000 7f428d2b4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42880f66e0 0x7f428810bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:48:14.831 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:14.831+0000 7f428d2b4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42880f66e0 0x7f428810bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59478/0 (socket says 172.21.15.27:59478) 2021-11-03T17:48:14.831 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:14.831+0000 7f428d2b4700 1 -- 172.21.15.27:0/2199855716 learned_addr learned my addr 172.21.15.27:0/2199855716 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:48:14.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:14.831+0000 7f428dab5700 1 --2- 172.21.15.27:0/2199855716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42880f5440 0x7f4288109a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:48:14.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:14.831+0000 7f428d2b4700 1 -- 172.21.15.27:0/2199855716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42880f5440 msgr2=0x7f4288109a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:48:14.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:14.831+0000 7f428d2b4700 1 --2- 172.21.15.27:0/2199855716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42880f5440 0x7f4288109a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:14.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:14.831+0000 7f428d2b4700 1 -- 172.21.15.27:0/2199855716 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4284005040 con 0x7f42880f66e0 2021-11-03T17:48:14.833 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:14.831+0000 7f428dab5700 1 --2- 172.21.15.27:0/2199855716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42880f5440 0x7f4288109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:48:14.833 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:14.831+0000 7f428d2b4700 1 --2- 172.21.15.27:0/2199855716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42880f66e0 0x7f428810bf30 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f4284009990 tx=0x7f4284009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:48:14.833 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:14.831+0000 7f427e7fc700 1 -- 172.21.15.27:0/2199855716 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4284018070 con 0x7f42880f66e0 2021-11-03T17:48:14.833 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:14.832+0000 7f427e7fc700 1 -- 172.21.15.27:0/2199855716 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f4284007ca0 con 0x7f42880f66e0 2021-11-03T17:48:14.834 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:14.832+0000 7f428eab7700 1 -- 172.21.15.27:0/2199855716 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f428810c480 con 0x7f42880f66e0 2021-11-03T17:48:14.834 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:14.832+0000 7f427e7fc700 1 -- 172.21.15.27:0/2199855716 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f428401c620 con 0x7f42880f66e0 2021-11-03T17:48:14.834 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:14.832+0000 7f428eab7700 1 -- 172.21.15.27:0/2199855716 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f428810c9e0 con 0x7f42880f66e0 2021-11-03T17:48:14.835 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:14.832+0000 7f428eab7700 1 -- 172.21.15.27:0/2199855716 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4288045ad0 con 0x7f42880f66e0 2021-11-03T17:48:14.836 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:14.833+0000 7f427e7fc700 1 -- 172.21.15.27:0/2199855716 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f428401d960 con 0x7f42880f66e0 2021-11-03T17:48:14.836 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:14.833+0000 7f427e7fc700 1 --2- 172.21.15.27:0/2199855716 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4274064100 0x7f42740665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:48:14.836 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:14.833+0000 7f427e7fc700 1 -- 172.21.15.27:0/2199855716 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f4284082580 con 0x7f42880f66e0 2021-11-03T17:48:14.836 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:14.833+0000 7f428dab5700 1 --2- 172.21.15.27:0/2199855716 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4274064100 0x7f42740665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:48:14.837 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:14.834+0000 7f428dab5700 1 --2- 172.21.15.27:0/2199855716 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4274064100 0x7f42740665c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f4288060080 tx=0x7f4278009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:48:14.837 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:14.838+0000 7f427e7fc700 1 -- 172.21.15.27:0/2199855716 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f4284054940 con 0x7f42880f66e0 2021-11-03T17:48:15.012 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.017+0000 7f428eab7700 1 -- 172.21.15.27:0/2199855716 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4288057f10 con 0x7f4274064100 2021-11-03T17:48:15.013 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.019+0000 7f427e7fc700 1 -- 172.21.15.27:0/2199855716 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f4288057f10 con 0x7f4274064100 2021-11-03T17:48:15.015 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.020+0000 7f428eab7700 1 -- 172.21.15.27:0/2199855716 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4274064100 msgr2=0x7f42740665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:48:15.015 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.020+0000 7f428eab7700 1 --2- 172.21.15.27:0/2199855716 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4274064100 0x7f42740665c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f4288060080 tx=0x7f4278009040).stop 2021-11-03T17:48:15.016 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.020+0000 7f428eab7700 1 -- 172.21.15.27:0/2199855716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42880f66e0 msgr2=0x7f428810bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:48:15.016 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.020+0000 7f428eab7700 1 --2- 172.21.15.27:0/2199855716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42880f66e0 0x7f428810bf30 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f4284009990 tx=0x7f4284009b30).stop 2021-11-03T17:48:15.016 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.020+0000 7f428eab7700 1 -- 172.21.15.27:0/2199855716 shutdown_connections 2021-11-03T17:48:15.016 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.020+0000 7f428eab7700 1 --2- 172.21.15.27:0/2199855716 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4274064100 0x7f42740665c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:15.016 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.020+0000 7f428eab7700 1 --2- 172.21.15.27:0/2199855716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f42880f5440 0x7f4288109a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:15.017 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.020+0000 7f428eab7700 1 --2- 172.21.15.27:0/2199855716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f42880f66e0 0x7f428810bf30 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:15.017 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.020+0000 7f428eab7700 1 -- 172.21.15.27:0/2199855716 >> 172.21.15.27:0/2199855716 conn(0x7f42880f0ba0 msgr2=0x7f42880fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:48:15.017 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.021+0000 7f428eab7700 1 -- 172.21.15.27:0/2199855716 shutdown_connections 2021-11-03T17:48:15.017 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.021+0000 7f428eab7700 1 -- 172.21.15.27:0/2199855716 wait complete. 2021-11-03T17:48:15.025 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:48:15.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.121+0000 7f7ebc2dc700 1 -- 172.21.15.27:0/2197960940 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7eb40f5ef0 msgr2=0x7f7eb40f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:48:15.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.121+0000 7f7ebc2dc700 1 --2- 172.21.15.27:0/2197960940 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7eb40f5ef0 0x7f7eb40f6920 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f7eb0004660 tx=0x7f7eb0009b30).stop 2021-11-03T17:48:15.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.121+0000 7f7ebc2dc700 1 -- 172.21.15.27:0/2197960940 shutdown_connections 2021-11-03T17:48:15.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.121+0000 7f7ebc2dc700 1 --2- 172.21.15.27:0/2197960940 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7eb40f5ef0 0x7f7eb40f6920 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:15.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.121+0000 7f7ebc2dc700 1 --2- 172.21.15.27:0/2197960940 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7eb40f55e0 0x7f7eb40f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:15.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.121+0000 7f7ebc2dc700 1 -- 172.21.15.27:0/2197960940 >> 172.21.15.27:0/2197960940 conn(0x7f7eb40f0de0 msgr2=0x7f7eb40f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:48:15.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.121+0000 7f7ebc2dc700 1 -- 172.21.15.27:0/2197960940 shutdown_connections 2021-11-03T17:48:15.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.121+0000 7f7ebc2dc700 1 -- 172.21.15.27:0/2197960940 wait complete. 2021-11-03T17:48:15.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.122+0000 7f7ebc2dc700 1 Processor -- start 2021-11-03T17:48:15.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.122+0000 7f7ebc2dc700 1 -- start start 2021-11-03T17:48:15.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.122+0000 7f7ebc2dc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7eb40f55e0 0x7f7eb4105e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:48:15.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.122+0000 7f7ebc2dc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7eb40f5ef0 0x7f7eb4106380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:48:15.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.122+0000 7f7ebc2dc700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb4106930 con 0x7f7eb40f5ef0 2021-11-03T17:48:15.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.122+0000 7f7ebc2dc700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb4106a70 con 0x7f7eb40f55e0 2021-11-03T17:48:15.121 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.122+0000 7f7eb9877700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7eb40f5ef0 0x7f7eb4106380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:48:15.121 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.122+0000 7f7eb9877700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7eb40f5ef0 0x7f7eb4106380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59488/0 (socket says 172.21.15.27:59488) 2021-11-03T17:48:15.121 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.122+0000 7f7eb9877700 1 -- 172.21.15.27:0/2044718122 learned_addr learned my addr 172.21.15.27:0/2044718122 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:48:15.121 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.122+0000 7f7eba078700 1 --2- 172.21.15.27:0/2044718122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7eb40f55e0 0x7f7eb4105e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:48:15.122 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.122+0000 7f7eb9877700 1 -- 172.21.15.27:0/2044718122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7eb40f55e0 msgr2=0x7f7eb4105e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:48:15.122 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.122+0000 7f7eb9877700 1 --2- 172.21.15.27:0/2044718122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7eb40f55e0 0x7f7eb4105e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:15.122 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.122+0000 7f7eb9877700 1 -- 172.21.15.27:0/2044718122 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb0005040 con 0x7f7eb40f5ef0 2021-11-03T17:48:15.122 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.122+0000 7f7eb9877700 1 --2- 172.21.15.27:0/2044718122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7eb40f5ef0 0x7f7eb4106380 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7f7eb0009990 tx=0x7f7eb0009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:48:15.124 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.125+0000 7f7eaaffd700 1 -- 172.21.15.27:0/2044718122 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7eb0018070 con 0x7f7eb40f5ef0 2021-11-03T17:48:15.124 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.125+0000 7f7ebc2dc700 1 -- 172.21.15.27:0/2044718122 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eb41073b0 con 0x7f7eb40f5ef0 2021-11-03T17:48:15.124 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.125+0000 7f7eaaffd700 1 -- 172.21.15.27:0/2044718122 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f7eb0007ba0 con 0x7f7eb40f5ef0 2021-11-03T17:48:15.124 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.125+0000 7f7eaaffd700 1 -- 172.21.15.27:0/2044718122 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7eb001c620 con 0x7f7eb40f5ef0 2021-11-03T17:48:15.125 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.125+0000 7f7ebc2dc700 1 -- 172.21.15.27:0/2044718122 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eb4107910 con 0x7f7eb40f5ef0 2021-11-03T17:48:15.125 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.126+0000 7f7ebc2dc700 1 -- 172.21.15.27:0/2044718122 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eb4045ad0 con 0x7f7eb40f5ef0 2021-11-03T17:48:15.125 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.126+0000 7f7eaaffd700 1 -- 172.21.15.27:0/2044718122 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f7eb001c890 con 0x7f7eb40f5ef0 2021-11-03T17:48:15.125 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.126+0000 7f7eaaffd700 1 --2- 172.21.15.27:0/2044718122 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7ea00640b0 0x7f7ea0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:48:15.126 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.127+0000 7f7eaaffd700 1 -- 172.21.15.27:0/2044718122 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f7eb00825a0 con 0x7f7eb40f5ef0 2021-11-03T17:48:15.126 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.127+0000 7f7eba078700 1 --2- 172.21.15.27:0/2044718122 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7ea00640b0 0x7f7ea0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:48:15.126 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.127+0000 7f7eba078700 1 --2- 172.21.15.27:0/2044718122 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7ea00640b0 0x7f7ea0066570 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f7ea40041d0 tx=0x7f7ea4009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:48:15.127 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.131+0000 7f7eaaffd700 1 -- 172.21.15.27:0/2044718122 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f7eb0054960 con 0x7f7eb40f5ef0 2021-11-03T17:48:15.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.306+0000 7f7ebc2dc700 1 -- 172.21.15.27:0/2044718122 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7eb405f720 con 0x7f7ea00640b0 2021-11-03T17:48:15.316 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.321+0000 7f7eaaffd700 1 -- 172.21.15.27:0/2044718122 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f7eb405f720 con 0x7f7ea00640b0 2021-11-03T17:48:15.317 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:48:15.317 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (39m) 9m ago 41m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:48:15.318 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (31m) 9m ago 41m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:48:15.318 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (31m) 8m ago 40m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:48:15.319 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (39m) 9m ago 40m 41.3M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:48:15.319 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (33m) 9m ago 36m 21.1M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:48:15.319 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (36m) 9m ago 36m 3843M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:48:15.320 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (32m) 8m ago 36m 20.4M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:48:15.320 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (33m) 8m ago 36m 22.1M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:48:15.321 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (33m) 9m ago 42m 517M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:48:15.321 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (32m) 8m ago 40m 419M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:48:15.322 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (32m) 9m ago 42m 159M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:48:15.322 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (32m) 8m ago 40m 82.5M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:48:15.323 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (40m) 9m ago 40m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:48:15.323 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (40m) 8m ago 40m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:48:15.324 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (31m) 9m ago 39m 311M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:48:15.324 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (30m) 9m ago 39m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:48:15.324 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (30m) 9m ago 38m 248M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:48:15.325 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (29m) 8m ago 38m 354M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:48:15.325 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (29m) 8m ago 38m 285M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:48:15.326 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (28m) 8m ago 37m 287M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:48:15.326 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (39m) 9m ago 40m 50.5M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:48:15.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.323+0000 7f7ebc2dc700 1 -- 172.21.15.27:0/2044718122 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7ea00640b0 msgr2=0x7f7ea0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:48:15.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.323+0000 7f7ebc2dc700 1 --2- 172.21.15.27:0/2044718122 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7ea00640b0 0x7f7ea0066570 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 rx=0x7f7ea40041d0 tx=0x7f7ea4009040).stop 2021-11-03T17:48:15.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.323+0000 7f7ebc2dc700 1 -- 172.21.15.27:0/2044718122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7eb40f5ef0 msgr2=0x7f7eb4106380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:48:15.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.323+0000 7f7ebc2dc700 1 --2- 172.21.15.27:0/2044718122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7eb40f5ef0 0x7f7eb4106380 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7f7eb0009990 tx=0x7f7eb0009b30).stop 2021-11-03T17:48:15.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.323+0000 7f7ebc2dc700 1 -- 172.21.15.27:0/2044718122 shutdown_connections 2021-11-03T17:48:15.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.323+0000 7f7ebc2dc700 1 --2- 172.21.15.27:0/2044718122 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7ea00640b0 0x7f7ea0066570 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:15.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.323+0000 7f7ebc2dc700 1 --2- 172.21.15.27:0/2044718122 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7eb40f55e0 0x7f7eb4105e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:15.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.323+0000 7f7ebc2dc700 1 --2- 172.21.15.27:0/2044718122 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7eb40f5ef0 0x7f7eb4106380 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:15.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.323+0000 7f7ebc2dc700 1 -- 172.21.15.27:0/2044718122 >> 172.21.15.27:0/2044718122 conn(0x7f7eb40f0de0 msgr2=0x7f7eb40fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:48:15.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.323+0000 7f7ebc2dc700 1 -- 172.21.15.27:0/2044718122 shutdown_connections 2021-11-03T17:48:15.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.323+0000 7f7ebc2dc700 1 -- 172.21.15.27:0/2044718122 wait complete. 2021-11-03T17:48:15.419 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.422+0000 7f3f8bd02700 1 -- 172.21.15.27:0/2629401242 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f840f5ef0 msgr2=0x7f3f840f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:48:15.420 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.422+0000 7f3f8bd02700 1 --2- 172.21.15.27:0/2629401242 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f840f5ef0 0x7f3f840f6920 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7f3f74004660 tx=0x7f3f74009b30).stop 2021-11-03T17:48:15.420 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.422+0000 7f3f8bd02700 1 -- 172.21.15.27:0/2629401242 shutdown_connections 2021-11-03T17:48:15.420 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.422+0000 7f3f8bd02700 1 --2- 172.21.15.27:0/2629401242 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f840f5ef0 0x7f3f840f6920 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:15.420 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.422+0000 7f3f8bd02700 1 --2- 172.21.15.27:0/2629401242 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f840f55e0 0x7f3f840f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:15.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.422+0000 7f3f8bd02700 1 -- 172.21.15.27:0/2629401242 >> 172.21.15.27:0/2629401242 conn(0x7f3f840f0de0 msgr2=0x7f3f840f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:48:15.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.422+0000 7f3f8bd02700 1 -- 172.21.15.27:0/2629401242 shutdown_connections 2021-11-03T17:48:15.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.422+0000 7f3f8bd02700 1 -- 172.21.15.27:0/2629401242 wait complete. 2021-11-03T17:48:15.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.423+0000 7f3f8bd02700 1 Processor -- start 2021-11-03T17:48:15.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.423+0000 7f3f8bd02700 1 -- start start 2021-11-03T17:48:15.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.423+0000 7f3f8bd02700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f840f55e0 0x7f3f84109c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:48:15.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.423+0000 7f3f8bd02700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f840f5ef0 0x7f3f8410c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:48:15.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.423+0000 7f3f8bd02700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f84060500 con 0x7f3f840f5ef0 2021-11-03T17:48:15.423 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.423+0000 7f3f8bd02700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f84060640 con 0x7f3f840f55e0 2021-11-03T17:48:15.423 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.423+0000 7f3f8929d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f840f5ef0 0x7f3f8410c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:48:15.423 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.423+0000 7f3f8929d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f840f5ef0 0x7f3f8410c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59498/0 (socket says 172.21.15.27:59498) 2021-11-03T17:48:15.423 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.423+0000 7f3f8929d700 1 -- 172.21.15.27:0/2427128736 learned_addr learned my addr 172.21.15.27:0/2427128736 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:48:15.423 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.423+0000 7f3f89a9e700 1 --2- 172.21.15.27:0/2427128736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f840f55e0 0x7f3f84109c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:48:15.424 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.423+0000 7f3f8929d700 1 -- 172.21.15.27:0/2427128736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f840f55e0 msgr2=0x7f3f84109c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:48:15.424 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.423+0000 7f3f8929d700 1 --2- 172.21.15.27:0/2427128736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f840f55e0 0x7f3f84109c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:15.424 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.423+0000 7f3f8929d700 1 -- 172.21.15.27:0/2427128736 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f74005040 con 0x7f3f840f5ef0 2021-11-03T17:48:15.424 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.423+0000 7f3f89a9e700 1 --2- 172.21.15.27:0/2427128736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f840f55e0 0x7f3f84109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:48:15.424 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.424+0000 7f3f8929d700 1 --2- 172.21.15.27:0/2427128736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f840f5ef0 0x7f3f8410c190 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7f3f74009990 tx=0x7f3f74009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:48:15.425 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.424+0000 7f3f7a7fc700 1 -- 172.21.15.27:0/2427128736 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f74018070 con 0x7f3f840f5ef0 2021-11-03T17:48:15.425 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.424+0000 7f3f7a7fc700 1 -- 172.21.15.27:0/2427128736 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f3f74007ba0 con 0x7f3f840f5ef0 2021-11-03T17:48:15.425 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.424+0000 7f3f8bd02700 1 -- 172.21.15.27:0/2427128736 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f8410c6e0 con 0x7f3f840f5ef0 2021-11-03T17:48:15.425 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.424+0000 7f3f7a7fc700 1 -- 172.21.15.27:0/2427128736 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f7401c3c0 con 0x7f3f840f5ef0 2021-11-03T17:48:15.425 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.424+0000 7f3f8bd02700 1 -- 172.21.15.27:0/2427128736 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f8410cc40 con 0x7f3f840f5ef0 2021-11-03T17:48:15.426 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.425+0000 7f3f8bd02700 1 -- 172.21.15.27:0/2427128736 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f84045ad0 con 0x7f3f840f5ef0 2021-11-03T17:48:15.427 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.425+0000 7f3f7a7fc700 1 -- 172.21.15.27:0/2427128736 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f3f7401d420 con 0x7f3f840f5ef0 2021-11-03T17:48:15.427 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.425+0000 7f3f7a7fc700 1 --2- 172.21.15.27:0/2427128736 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3f700640b0 0x7f3f70066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:48:15.427 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.426+0000 7f3f7a7fc700 1 -- 172.21.15.27:0/2427128736 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f3f74082440 con 0x7f3f840f5ef0 2021-11-03T17:48:15.428 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.426+0000 7f3f89a9e700 1 --2- 172.21.15.27:0/2427128736 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3f700640b0 0x7f3f70066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:48:15.428 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.426+0000 7f3f89a9e700 1 --2- 172.21.15.27:0/2427128736 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3f700640b0 0x7f3f70066570 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f3f84060180 tx=0x7f3f80009c00).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:48:15.428 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.429+0000 7f3f7a7fc700 1 -- 172.21.15.27:0/2427128736 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f3f740547a0 con 0x7f3f840f5ef0 2021-11-03T17:48:15.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1937.405813 secs 2021-11-03T17:48:15.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1937.405813 secs 2021-11-03T17:48:15.653 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.658+0000 7f3f8bd02700 1 -- 172.21.15.27:0/2427128736 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3f84027a00 con 0x7f3f840f5ef0 2021-11-03T17:48:15.653 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.658+0000 7f3f7a7fc700 1 -- 172.21.15.27:0/2427128736 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f3f74054390 con 0x7f3f840f5ef0 2021-11-03T17:48:15.654 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:48:15.654 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:48:15.654 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:48:15.654 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:48:15.655 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:48:15.655 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:48:15.655 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:48:15.655 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:48:15.655 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:48:15.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:48:15.656 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:48:15.656 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:48:15.656 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:48:15.657 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:48:15.657 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:48:15.657 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:48:15.657 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:48:15.658 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:48:15.658 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.660+0000 7f3f8bd02700 1 -- 172.21.15.27:0/2427128736 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3f700640b0 msgr2=0x7f3f70066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:48:15.659 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.660+0000 7f3f8bd02700 1 --2- 172.21.15.27:0/2427128736 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3f700640b0 0x7f3f70066570 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f3f84060180 tx=0x7f3f80009c00).stop 2021-11-03T17:48:15.659 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.660+0000 7f3f8bd02700 1 -- 172.21.15.27:0/2427128736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f840f5ef0 msgr2=0x7f3f8410c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:48:15.659 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.660+0000 7f3f8bd02700 1 --2- 172.21.15.27:0/2427128736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f840f5ef0 0x7f3f8410c190 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 rx=0x7f3f74009990 tx=0x7f3f74009b30).stop 2021-11-03T17:48:15.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.660+0000 7f3f8bd02700 1 -- 172.21.15.27:0/2427128736 shutdown_connections 2021-11-03T17:48:15.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.660+0000 7f3f8bd02700 1 --2- 172.21.15.27:0/2427128736 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3f700640b0 0x7f3f70066570 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:15.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.660+0000 7f3f8bd02700 1 --2- 172.21.15.27:0/2427128736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f840f55e0 0x7f3f84109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:15.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.660+0000 7f3f8bd02700 1 --2- 172.21.15.27:0/2427128736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f840f5ef0 0x7f3f8410c190 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:15.661 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.660+0000 7f3f8bd02700 1 -- 172.21.15.27:0/2427128736 >> 172.21.15.27:0/2427128736 conn(0x7f3f840f0de0 msgr2=0x7f3f840fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:48:15.661 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.660+0000 7f3f8bd02700 1 -- 172.21.15.27:0/2427128736 shutdown_connections 2021-11-03T17:48:15.662 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.660+0000 7f3f8bd02700 1 -- 172.21.15.27:0/2427128736 wait complete. 2021-11-03T17:48:15.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.760+0000 7f13e8cef700 1 -- 172.21.15.27:0/24444795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13e40f96f0 msgr2=0x7f13e40fbae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:48:15.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.760+0000 7f13e8cef700 1 --2- 172.21.15.27:0/24444795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13e40f96f0 0x7f13e40fbae0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7f13d8004660 tx=0x7f13d8009b30).stop 2021-11-03T17:48:15.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.760+0000 7f13e8cef700 1 -- 172.21.15.27:0/24444795 shutdown_connections 2021-11-03T17:48:15.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.760+0000 7f13e8cef700 1 --2- 172.21.15.27:0/24444795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13e40f96f0 0x7f13e40fbae0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:15.759 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.760+0000 7f13e8cef700 1 --2- 172.21.15.27:0/24444795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13e40602b0 0x7f13e40f9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:15.759 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.760+0000 7f13e8cef700 1 -- 172.21.15.27:0/24444795 >> 172.21.15.27:0/24444795 conn(0x7f13e40f0b90 msgr2=0x7f13e40f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:48:15.759 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.760+0000 7f13e8cef700 1 -- 172.21.15.27:0/24444795 shutdown_connections 2021-11-03T17:48:15.759 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.760+0000 7f13e8cef700 1 -- 172.21.15.27:0/24444795 wait complete. 2021-11-03T17:48:15.760 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.760+0000 7f13e8cef700 1 Processor -- start 2021-11-03T17:48:15.760 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.760+0000 7f13e8cef700 1 -- start start 2021-11-03T17:48:15.760 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.761+0000 7f13e8cef700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13e40602b0 0x7f13e40f6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:48:15.760 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.761+0000 7f13e8cef700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13e40f96f0 0x7f13e40f5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:48:15.761 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.761+0000 7f13e8cef700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e40f55d0 con 0x7f13e40f96f0 2021-11-03T17:48:15.761 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.761+0000 7f13e8cef700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13e40f5710 con 0x7f13e40602b0 2021-11-03T17:48:15.761 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.761+0000 7f13e2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13e40f96f0 0x7f13e40f5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:48:15.762 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.761+0000 7f13e2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13e40f96f0 0x7f13e40f5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59508/0 (socket says 172.21.15.27:59508) 2021-11-03T17:48:15.763 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.761+0000 7f13e2ffd700 1 -- 172.21.15.27:0/1435237648 learned_addr learned my addr 172.21.15.27:0/1435237648 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:48:15.763 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.761+0000 7f13e37fe700 1 --2- 172.21.15.27:0/1435237648 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13e40602b0 0x7f13e40f6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:48:15.763 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.761+0000 7f13e2ffd700 1 -- 172.21.15.27:0/1435237648 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13e40602b0 msgr2=0x7f13e40f6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:48:15.764 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.761+0000 7f13e2ffd700 1 --2- 172.21.15.27:0/1435237648 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13e40602b0 0x7f13e40f6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:15.764 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.761+0000 7f13e2ffd700 1 -- 172.21.15.27:0/1435237648 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d8005040 con 0x7f13e40f96f0 2021-11-03T17:48:15.765 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.761+0000 7f13e37fe700 1 --2- 172.21.15.27:0/1435237648 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13e40602b0 0x7f13e40f6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:48:15.765 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.761+0000 7f13e2ffd700 1 --2- 172.21.15.27:0/1435237648 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13e40f96f0 0x7f13e40f5070 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f13d8014040 tx=0x7f13d800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:48:15.766 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.761+0000 7f13cbfff700 1 -- 172.21.15.27:0/1435237648 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13d8019070 con 0x7f13e40f96f0 2021-11-03T17:48:15.766 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.761+0000 7f13e8cef700 1 -- 172.21.15.27:0/1435237648 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13e40f5990 con 0x7f13e40f96f0 2021-11-03T17:48:15.767 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.761+0000 7f13cbfff700 1 -- 172.21.15.27:0/1435237648 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f13d8007e70 con 0x7f13e40f96f0 2021-11-03T17:48:15.767 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.761+0000 7f13cbfff700 1 -- 172.21.15.27:0/1435237648 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13d8014d50 con 0x7f13e40f96f0 2021-11-03T17:48:15.768 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.761+0000 7f13e8cef700 1 -- 172.21.15.27:0/1435237648 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13e40f5ef0 con 0x7f13e40f96f0 2021-11-03T17:48:15.768 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.762+0000 7f13e8cef700 1 -- 172.21.15.27:0/1435237648 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13e4045ad0 con 0x7f13e40f96f0 2021-11-03T17:48:15.770 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.763+0000 7f13cbfff700 1 -- 172.21.15.27:0/1435237648 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f13d8025070 con 0x7f13e40f96f0 2021-11-03T17:48:15.770 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.763+0000 7f13cbfff700 1 --2- 172.21.15.27:0/1435237648 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f13cc064100 0x7f13cc0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:48:15.771 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.763+0000 7f13cbfff700 1 -- 172.21.15.27:0/1435237648 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f13d8081820 con 0x7f13e40f96f0 2021-11-03T17:48:15.771 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.763+0000 7f13e37fe700 1 --2- 172.21.15.27:0/1435237648 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f13cc064100 0x7f13cc0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:48:15.772 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.764+0000 7f13e37fe700 1 --2- 172.21.15.27:0/1435237648 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f13cc064100 0x7f13cc0665c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f13d4009da0 tx=0x7f13d400b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:48:15.772 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.767+0000 7f13cbfff700 1 -- 172.21.15.27:0/1435237648 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f13d8053b20 con 0x7f13e40f96f0 2021-11-03T17:48:15.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.973+0000 7f13e8cef700 1 -- 172.21.15.27:0/1435237648 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f13e4027a00 con 0x7f13e40f96f0 2021-11-03T17:48:15.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.973+0000 7f13cbfff700 1 -- 172.21.15.27:0/1435237648 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f13d8023070 con 0x7f13e40f96f0 2021-11-03T17:48:15.970 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:48:15.970 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:48:15.971 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:48:15.971 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:48:15.971 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:48:15.972 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:48:15.972 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:48:15.973 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:48:15.973 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:48:15.974 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:48:15.974 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:48:15.974 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:48:15.975 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:48:15.976 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:48:15.976 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:48:15.977 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:48:15.977 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:48:15.978 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:48:15.978 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:48:15.979 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:48:15.979 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:48:15.979 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:48:15.980 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:48:15.980 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:48:15.981 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:48:15.981 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:48:15.981 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:48:15.982 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:48:15.982 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:48:15.982 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:48:15.983 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:48:15.983 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:48:15.984 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:48:15.984 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:48:15.985 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:48:15.985 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:48:15.985 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:48:15.986 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:48:15.986 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:48:15.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.975+0000 7f13e8cef700 1 -- 172.21.15.27:0/1435237648 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f13cc064100 msgr2=0x7f13cc0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:48:15.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.975+0000 7f13e8cef700 1 --2- 172.21.15.27:0/1435237648 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f13cc064100 0x7f13cc0665c0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f13d4009da0 tx=0x7f13d400b040).stop 2021-11-03T17:48:15.989 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.975+0000 7f13e8cef700 1 -- 172.21.15.27:0/1435237648 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13e40f96f0 msgr2=0x7f13e40f5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:48:15.989 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.975+0000 7f13e8cef700 1 --2- 172.21.15.27:0/1435237648 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13e40f96f0 0x7f13e40f5070 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f13d8014040 tx=0x7f13d800b040).stop 2021-11-03T17:48:15.990 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.975+0000 7f13e8cef700 1 -- 172.21.15.27:0/1435237648 shutdown_connections 2021-11-03T17:48:15.990 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.975+0000 7f13e8cef700 1 --2- 172.21.15.27:0/1435237648 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f13cc064100 0x7f13cc0665c0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:15.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.975+0000 7f13e8cef700 1 --2- 172.21.15.27:0/1435237648 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13e40602b0 0x7f13e40f6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:15.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.975+0000 7f13e8cef700 1 --2- 172.21.15.27:0/1435237648 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13e40f96f0 0x7f13e40f5070 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:15.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.975+0000 7f13e8cef700 1 -- 172.21.15.27:0/1435237648 >> 172.21.15.27:0/1435237648 conn(0x7f13e40f0b90 msgr2=0x7f13e40f2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:48:15.992 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.975+0000 7f13e8cef700 1 -- 172.21.15.27:0/1435237648 shutdown_connections 2021-11-03T17:48:15.992 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:15.975+0000 7f13e8cef700 1 -- 172.21.15.27:0/1435237648 wait complete. 2021-11-03T17:48:15.992 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:48:16.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:16 smithi115 ceph-mon[65453]: pgmap v1008: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:16.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:16 smithi115 ceph-mon[65453]: from='client.35144 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:48:16.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:16 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2427128736' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:16.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:16 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1435237648' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:48:16.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:16 smithi027 ceph-mon[97244]: pgmap v1008: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:16.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:16 smithi027 ceph-mon[97244]: from='client.35144 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:48:16.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:16 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2427128736' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:16.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:16 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1435237648' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:48:17.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:17 smithi115 ceph-mon[65453]: from='client.35148 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:48:17.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:17 smithi027 ceph-mon[97244]: from='client.35148 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:48:18.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:18 smithi115 ceph-mon[65453]: pgmap v1009: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:18.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:18 smithi027 ceph-mon[97244]: pgmap v1009: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:20.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:20 smithi115 ceph-mon[65453]: pgmap v1010: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:20.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1942.405934 secs 2021-11-03T17:48:20.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:20 smithi027 ceph-mon[97244]: pgmap v1010: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:20.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1942.405934 secs 2021-11-03T17:48:22.442 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:22 smithi115 ceph-mon[65453]: pgmap v1011: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:22.502 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:22 smithi027 ceph-mon[97244]: pgmap v1011: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:23.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:48:23.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:23 smithi027 ceph-mon[97244]: poll took 60177.286 ms: 1 events 2021-11-03T17:48:23.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:48:23.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:48:23.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:48:23.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:23.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:23.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:23.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:23.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:48:23.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:23 smithi115 ceph-mon[65453]: poll took 60177.286 ms: 1 events 2021-11-03T17:48:23.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:48:23.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:48:23.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:48:23.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:23.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:23.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:23.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:24.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:24 smithi115 ceph-mon[65453]: pgmap v1012: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:24.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:24 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:48:24.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:24 smithi027 ceph-mon[97244]: pgmap v1012: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:24.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:24 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:48:25.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1947.406071 secs 2021-11-03T17:48:25.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1947.406071 secs 2021-11-03T17:48:26.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:26 smithi115 ceph-mon[65453]: pgmap v1013: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:26.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:26 smithi027 ceph-mon[97244]: pgmap v1013: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:28.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:28 smithi115 ceph-mon[65453]: pgmap v1014: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:28.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:28 smithi027 ceph-mon[97244]: pgmap v1014: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:30.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:30 smithi115 ceph-mon[65453]: pgmap v1015: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:30.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1952.406213 secs 2021-11-03T17:48:30.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:30 smithi027 ceph-mon[97244]: pgmap v1015: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:30.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1952.406213 secs 2021-11-03T17:48:32.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:32 smithi115 ceph-mon[65453]: pgmap v1016: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:32.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:32 smithi027 ceph-mon[97244]: pgmap v1016: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:33.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:48:33.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:48:33.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:48:33.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:33.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:33.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:33.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:33.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:48:33.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:48:33.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:48:33.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:33.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:33.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:33.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:34.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:34 smithi115 ceph-mon[65453]: pgmap v1017: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:34.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:34 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:48:34.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:34 smithi027 ceph-mon[97244]: pgmap v1017: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:34.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:34 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:48:35.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1957.406359 secs 2021-11-03T17:48:35.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1957.406359 secs 2021-11-03T17:48:36.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:36 smithi115 ceph-mon[65453]: pgmap v1018: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:36.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:36 smithi027 ceph-mon[97244]: pgmap v1018: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:38.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:38 smithi115 ceph-mon[65453]: pgmap v1019: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:38.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:38 smithi027 ceph-mon[97244]: pgmap v1019: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:40.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:40 smithi115 ceph-mon[65453]: pgmap v1020: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:40.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1962.406481 secs 2021-11-03T17:48:40.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:40 smithi027 ceph-mon[97244]: pgmap v1020: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:40.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1962.406481 secs 2021-11-03T17:48:42.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:42 smithi115 ceph-mon[65453]: pgmap v1021: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:42.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:42 smithi027 ceph-mon[97244]: pgmap v1021: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:43.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:48:43.541 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:48:44.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:44 smithi115 ceph-mon[65453]: pgmap v1022: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:44.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:44 smithi115 ceph-mon[65453]: poll took 20055.656 ms: 1 events 2021-11-03T17:48:44.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:44 smithi027 ceph-mon[97244]: pgmap v1022: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:44.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:44 smithi027 ceph-mon[97244]: poll took 20055.656 ms: 1 events 2021-11-03T17:48:45.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1967.406589 secs 2021-11-03T17:48:45.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1967.406589 secs 2021-11-03T17:48:46.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.079+0000 7f1abbf84700 1 -- 172.21.15.27:0/146373821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ab40602b0 msgr2=0x7f1ab40f9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:48:46.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.079+0000 7f1abbf84700 1 --2- 172.21.15.27:0/146373821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ab40602b0 0x7f1ab40f9300 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 rx=0x7f1ab0004660 tx=0x7f1ab0009b30).stop 2021-11-03T17:48:46.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.079+0000 7f1abbf84700 1 -- 172.21.15.27:0/146373821 shutdown_connections 2021-11-03T17:48:46.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.079+0000 7f1abbf84700 1 --2- 172.21.15.27:0/146373821 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ab40f9880 0x7f1ab40fbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:46.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.079+0000 7f1abbf84700 1 --2- 172.21.15.27:0/146373821 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ab40602b0 0x7f1ab40f9300 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:46.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.079+0000 7f1abbf84700 1 -- 172.21.15.27:0/146373821 >> 172.21.15.27:0/146373821 conn(0x7f1ab40f0d40 msgr2=0x7f1ab40f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:48:46.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.079+0000 7f1abbf84700 1 -- 172.21.15.27:0/146373821 shutdown_connections 2021-11-03T17:48:46.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.079+0000 7f1abbf84700 1 -- 172.21.15.27:0/146373821 wait complete. 2021-11-03T17:48:46.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.079+0000 7f1abbf84700 1 Processor -- start 2021-11-03T17:48:46.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.080+0000 7f1abbf84700 1 -- start start 2021-11-03T17:48:46.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.080+0000 7f1abbf84700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ab40602b0 0x7f1ab4106f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:48:46.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.080+0000 7f1abbf84700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ab40f9880 0x7f1ab4107410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:48:46.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.080+0000 7f1abbf84700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab41079c0 con 0x7f1ab40f9880 2021-11-03T17:48:46.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.080+0000 7f1abbf84700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab4107b00 con 0x7f1ab40602b0 2021-11-03T17:48:46.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.080+0000 7f1ab951f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ab40f9880 0x7f1ab4107410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:48:46.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.080+0000 7f1ab951f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ab40f9880 0x7f1ab4107410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59530/0 (socket says 172.21.15.27:59530) 2021-11-03T17:48:46.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.080+0000 7f1ab951f700 1 -- 172.21.15.27:0/3578593545 learned_addr learned my addr 172.21.15.27:0/3578593545 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:48:46.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.080+0000 7f1ab9d20700 1 --2- 172.21.15.27:0/3578593545 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ab40602b0 0x7f1ab4106f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:48:46.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.080+0000 7f1ab951f700 1 -- 172.21.15.27:0/3578593545 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ab40602b0 msgr2=0x7f1ab4106f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:48:46.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.080+0000 7f1ab951f700 1 --2- 172.21.15.27:0/3578593545 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ab40602b0 0x7f1ab4106f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:46.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.080+0000 7f1ab951f700 1 -- 172.21.15.27:0/3578593545 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab0005040 con 0x7f1ab40f9880 2021-11-03T17:48:46.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.080+0000 7f1ab951f700 1 --2- 172.21.15.27:0/3578593545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ab40f9880 0x7f1ab4107410 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f1aa400c790 tx=0x7f1aa4007d40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:48:46.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.080+0000 7f1aaa7fc700 1 -- 172.21.15.27:0/3578593545 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1aa400e680 con 0x7f1ab40f9880 2021-11-03T17:48:46.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.080+0000 7f1aaa7fc700 1 -- 172.21.15.27:0/3578593545 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f1aa400ec90 con 0x7f1ab40f9880 2021-11-03T17:48:46.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.080+0000 7f1abbf84700 1 -- 172.21.15.27:0/3578593545 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ab41084a0 con 0x7f1ab40f9880 2021-11-03T17:48:46.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.081+0000 7f1aaa7fc700 1 -- 172.21.15.27:0/3578593545 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1aa4013cd0 con 0x7f1ab40f9880 2021-11-03T17:48:46.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.081+0000 7f1abbf84700 1 -- 172.21.15.27:0/3578593545 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ab4108a60 con 0x7f1ab40f9880 2021-11-03T17:48:46.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.081+0000 7f1abbf84700 1 -- 172.21.15.27:0/3578593545 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ab4045ad0 con 0x7f1ab40f9880 2021-11-03T17:48:46.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.082+0000 7f1aaa7fc700 1 -- 172.21.15.27:0/3578593545 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f1aa401a060 con 0x7f1ab40f9880 2021-11-03T17:48:46.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.082+0000 7f1aaa7fc700 1 --2- 172.21.15.27:0/3578593545 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1aa0064180 0x7f1aa0066640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:48:46.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.082+0000 7f1aaa7fc700 1 -- 172.21.15.27:0/3578593545 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f1aa40809c0 con 0x7f1ab40f9880 2021-11-03T17:48:46.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.083+0000 7f1ab9d20700 1 --2- 172.21.15.27:0/3578593545 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1aa0064180 0x7f1aa0066640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:48:46.086 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.083+0000 7f1ab9d20700 1 --2- 172.21.15.27:0/3578593545 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1aa0064180 0x7f1aa0066640 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f1ab0004e30 tx=0x7f1ab0009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:48:46.086 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.087+0000 7f1aaa7fc700 1 -- 172.21.15.27:0/3578593545 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f1aa4052bf0 con 0x7f1ab40f9880 2021-11-03T17:48:46.282 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.287+0000 7f1abbf84700 1 -- 172.21.15.27:0/3578593545 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ab40ff050 con 0x7f1aa0064180 2021-11-03T17:48:46.283 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.288+0000 7f1aaa7fc700 1 -- 172.21.15.27:0/3578593545 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f1ab0019040 con 0x7f1aa0064180 2021-11-03T17:48:46.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.290+0000 7f1abbf84700 1 -- 172.21.15.27:0/3578593545 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1aa0064180 msgr2=0x7f1aa0066640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:48:46.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.290+0000 7f1abbf84700 1 --2- 172.21.15.27:0/3578593545 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1aa0064180 0x7f1aa0066640 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f1ab0004e30 tx=0x7f1ab0009b30).stop 2021-11-03T17:48:46.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.290+0000 7f1abbf84700 1 -- 172.21.15.27:0/3578593545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ab40f9880 msgr2=0x7f1ab4107410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:48:46.286 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.290+0000 7f1abbf84700 1 --2- 172.21.15.27:0/3578593545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ab40f9880 0x7f1ab4107410 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 rx=0x7f1aa400c790 tx=0x7f1aa4007d40).stop 2021-11-03T17:48:46.286 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.290+0000 7f1abbf84700 1 -- 172.21.15.27:0/3578593545 shutdown_connections 2021-11-03T17:48:46.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.290+0000 7f1abbf84700 1 --2- 172.21.15.27:0/3578593545 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1aa0064180 0x7f1aa0066640 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:46.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.290+0000 7f1abbf84700 1 --2- 172.21.15.27:0/3578593545 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ab40602b0 0x7f1ab4106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:46.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.290+0000 7f1abbf84700 1 --2- 172.21.15.27:0/3578593545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ab40f9880 0x7f1ab4107410 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:46.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.290+0000 7f1abbf84700 1 -- 172.21.15.27:0/3578593545 >> 172.21.15.27:0/3578593545 conn(0x7f1ab40f0d40 msgr2=0x7f1ab40f3130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:48:46.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.290+0000 7f1abbf84700 1 -- 172.21.15.27:0/3578593545 shutdown_connections 2021-11-03T17:48:46.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.290+0000 7f1abbf84700 1 -- 172.21.15.27:0/3578593545 wait complete. 2021-11-03T17:48:46.295 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:48:46.397 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.400+0000 7f744759e700 1 -- 172.21.15.27:0/234562147 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74480f5430 msgr2=0x7f74480f5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:48:46.397 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.400+0000 7f744759e700 1 --2- 172.21.15.27:0/234562147 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74480f5430 0x7f74480f5850 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 rx=0x7f7438004660 tx=0x7f7438009b30).stop 2021-11-03T17:48:46.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.400+0000 7f744759e700 1 -- 172.21.15.27:0/234562147 shutdown_connections 2021-11-03T17:48:46.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.400+0000 7f744759e700 1 --2- 172.21.15.27:0/234562147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74480f66d0 0x7f74480f6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:46.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.400+0000 7f744759e700 1 --2- 172.21.15.27:0/234562147 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74480f5430 0x7f74480f5850 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:46.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.400+0000 7f744759e700 1 -- 172.21.15.27:0/234562147 >> 172.21.15.27:0/234562147 conn(0x7f74480f0b90 msgr2=0x7f74480f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:48:46.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.400+0000 7f744759e700 1 -- 172.21.15.27:0/234562147 shutdown_connections 2021-11-03T17:48:46.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.400+0000 7f744759e700 1 -- 172.21.15.27:0/234562147 wait complete. 2021-11-03T17:48:46.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.401+0000 7f744759e700 1 Processor -- start 2021-11-03T17:48:46.400 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.401+0000 7f744759e700 1 -- start start 2021-11-03T17:48:46.400 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.401+0000 7f744759e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74480f5430 0x7f7448109a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:48:46.400 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.401+0000 7f744759e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74480f66d0 0x7f744810bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:48:46.400 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.401+0000 7f744759e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74480603e0 con 0x7f74480f5430 2021-11-03T17:48:46.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.401+0000 7f744759e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7448060520 con 0x7f74480f66d0 2021-11-03T17:48:46.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.401+0000 7f744659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74480f5430 0x7f7448109a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:48:46.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.401+0000 7f744659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74480f5430 0x7f7448109a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59538/0 (socket says 172.21.15.27:59538) 2021-11-03T17:48:46.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.401+0000 7f744659c700 1 -- 172.21.15.27:0/1891218282 learned_addr learned my addr 172.21.15.27:0/1891218282 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:48:46.402 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.401+0000 7f7445d9b700 1 --2- 172.21.15.27:0/1891218282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74480f66d0 0x7f744810bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:48:46.402 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.401+0000 7f744659c700 1 -- 172.21.15.27:0/1891218282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74480f66d0 msgr2=0x7f744810bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:48:46.402 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.401+0000 7f744659c700 1 --2- 172.21.15.27:0/1891218282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74480f66d0 0x7f744810bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:46.402 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.401+0000 7f744659c700 1 -- 172.21.15.27:0/1891218282 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7438005040 con 0x7f74480f5430 2021-11-03T17:48:46.403 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.401+0000 7f744659c700 1 --2- 172.21.15.27:0/1891218282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74480f5430 0x7f7448109a90 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f7438000c00 tx=0x7f7438009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:48:46.403 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.402+0000 7f7436ffd700 1 -- 172.21.15.27:0/1891218282 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7438018070 con 0x7f74480f5430 2021-11-03T17:48:46.403 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.402+0000 7f7436ffd700 1 -- 172.21.15.27:0/1891218282 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f7438007d70 con 0x7f74480f5430 2021-11-03T17:48:46.403 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.402+0000 7f744759e700 1 -- 172.21.15.27:0/1891218282 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f744810c4e0 con 0x7f74480f5430 2021-11-03T17:48:46.404 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.402+0000 7f7436ffd700 1 -- 172.21.15.27:0/1891218282 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f743801c620 con 0x7f74480f5430 2021-11-03T17:48:46.404 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.402+0000 7f744759e700 1 -- 172.21.15.27:0/1891218282 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f744810ca40 con 0x7f74480f5430 2021-11-03T17:48:46.404 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.403+0000 7f744759e700 1 -- 172.21.15.27:0/1891218282 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7448045ad0 con 0x7f74480f5430 2021-11-03T17:48:46.405 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.403+0000 7f7436ffd700 1 -- 172.21.15.27:0/1891218282 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f743801c860 con 0x7f74480f5430 2021-11-03T17:48:46.406 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.403+0000 7f7436ffd700 1 --2- 172.21.15.27:0/1891218282 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7430064100 0x7f74300665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:48:46.406 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.403+0000 7f7436ffd700 1 -- 172.21.15.27:0/1891218282 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f74380826f0 con 0x7f74480f5430 2021-11-03T17:48:46.406 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.404+0000 7f7445d9b700 1 --2- 172.21.15.27:0/1891218282 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7430064100 0x7f74300665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:48:46.407 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.404+0000 7f7445d9b700 1 --2- 172.21.15.27:0/1891218282 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7430064100 0x7f74300665c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f743c0041d0 tx=0x7f743c009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:48:46.407 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.408+0000 7f7436ffd700 1 -- 172.21.15.27:0/1891218282 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f7438054ab0 con 0x7f74480f5430 2021-11-03T17:48:46.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:46 smithi115 ceph-mon[65453]: pgmap v1023: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:46.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:46 smithi027 ceph-mon[97244]: pgmap v1023: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:46.579 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.584+0000 7f744759e700 1 -- 172.21.15.27:0/1891218282 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f74480fdd70 con 0x7f7430064100 2021-11-03T17:48:46.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.599+0000 7f7436ffd700 1 -- 172.21.15.27:0/1891218282 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f74480fdd70 con 0x7f7430064100 2021-11-03T17:48:46.594 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:48:46.594 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (40m) 10m ago 41m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:48:46.594 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (32m) 10m ago 41m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:48:46.595 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (31m) 9m ago 40m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:48:46.595 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (40m) 10m ago 41m 41.3M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:48:46.595 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (33m) 10m ago 36m 21.1M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:48:46.596 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (36m) 10m ago 36m 3843M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:48:46.596 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (32m) 9m ago 36m 20.4M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:48:46.596 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (34m) 9m ago 36m 22.1M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:48:46.596 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (34m) 10m ago 42m 517M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:48:46.597 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (33m) 9m ago 40m 419M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:48:46.597 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (33m) 10m ago 43m 159M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:48:46.597 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (32m) 9m ago 40m 82.5M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:48:46.597 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (41m) 10m ago 41m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:48:46.598 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (40m) 9m ago 40m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:48:46.598 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (31m) 10m ago 40m 311M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:48:46.598 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (31m) 10m ago 39m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:48:46.598 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (30m) 10m ago 39m 248M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:48:46.598 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (30m) 9m ago 38m 354M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:48:46.599 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (29m) 9m ago 38m 285M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:48:46.599 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (29m) 9m ago 38m 287M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:48:46.599 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (40m) 10m ago 40m 50.5M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:48:46.600 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.600+0000 7f744759e700 1 -- 172.21.15.27:0/1891218282 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7430064100 msgr2=0x7f74300665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:48:46.600 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.600+0000 7f744759e700 1 --2- 172.21.15.27:0/1891218282 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7430064100 0x7f74300665c0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f743c0041d0 tx=0x7f743c009040).stop 2021-11-03T17:48:46.601 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.600+0000 7f744759e700 1 -- 172.21.15.27:0/1891218282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74480f5430 msgr2=0x7f7448109a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:48:46.601 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.600+0000 7f744759e700 1 --2- 172.21.15.27:0/1891218282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74480f5430 0x7f7448109a90 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 rx=0x7f7438000c00 tx=0x7f7438009b30).stop 2021-11-03T17:48:46.601 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.601+0000 7f744759e700 1 -- 172.21.15.27:0/1891218282 shutdown_connections 2021-11-03T17:48:46.601 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.601+0000 7f744759e700 1 --2- 172.21.15.27:0/1891218282 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7430064100 0x7f74300665c0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:46.602 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.601+0000 7f744759e700 1 --2- 172.21.15.27:0/1891218282 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74480f5430 0x7f7448109a90 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:46.602 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.601+0000 7f744759e700 1 --2- 172.21.15.27:0/1891218282 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74480f66d0 0x7f744810bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:46.602 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.601+0000 7f744759e700 1 -- 172.21.15.27:0/1891218282 >> 172.21.15.27:0/1891218282 conn(0x7f74480f0b90 msgr2=0x7f74480fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:48:46.602 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.601+0000 7f744759e700 1 -- 172.21.15.27:0/1891218282 shutdown_connections 2021-11-03T17:48:46.602 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.601+0000 7f744759e700 1 -- 172.21.15.27:0/1891218282 wait complete. 2021-11-03T17:48:46.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.701+0000 7f6594fe6700 1 -- 172.21.15.27:0/1652648994 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65900f66e0 msgr2=0x7f65900f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:48:46.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.701+0000 7f6594fe6700 1 --2- 172.21.15.27:0/1652648994 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65900f66e0 0x7f65900f6b00 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 rx=0x7f6580004660 tx=0x7f6580009b30).stop 2021-11-03T17:48:46.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.702+0000 7f6594fe6700 1 -- 172.21.15.27:0/1652648994 shutdown_connections 2021-11-03T17:48:46.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.702+0000 7f6594fe6700 1 --2- 172.21.15.27:0/1652648994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65900f6ff0 0x7f65900f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:46.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.702+0000 7f6594fe6700 1 --2- 172.21.15.27:0/1652648994 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65900f66e0 0x7f65900f6b00 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:46.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.702+0000 7f6594fe6700 1 -- 172.21.15.27:0/1652648994 >> 172.21.15.27:0/1652648994 conn(0x7f65900f1f00 msgr2=0x7f65900f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:48:46.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.702+0000 7f6594fe6700 1 -- 172.21.15.27:0/1652648994 shutdown_connections 2021-11-03T17:48:46.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.702+0000 7f6594fe6700 1 -- 172.21.15.27:0/1652648994 wait complete. 2021-11-03T17:48:46.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.702+0000 7f6594fe6700 1 Processor -- start 2021-11-03T17:48:46.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.702+0000 7f6594fe6700 1 -- start start 2021-11-03T17:48:46.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.702+0000 7f6594fe6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65900f66e0 0x7f659006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:48:46.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.702+0000 7f6594fe6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65900f6ff0 0x7f6590069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:48:46.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.702+0000 7f6594fe6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659006a380 con 0x7f65900f6ff0 2021-11-03T17:48:46.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.702+0000 7f6594fe6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f659006a4c0 con 0x7f65900f66e0 2021-11-03T17:48:46.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.702+0000 7f658dd9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65900f6ff0 0x7f6590069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:48:46.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.703+0000 7f658dd9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65900f6ff0 0x7f6590069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59550/0 (socket says 172.21.15.27:59550) 2021-11-03T17:48:46.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.703+0000 7f658dd9b700 1 -- 172.21.15.27:0/2689911614 learned_addr learned my addr 172.21.15.27:0/2689911614 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:48:46.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.703+0000 7f658e59c700 1 --2- 172.21.15.27:0/2689911614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65900f66e0 0x7f659006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:48:46.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.703+0000 7f658dd9b700 1 -- 172.21.15.27:0/2689911614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65900f66e0 msgr2=0x7f659006b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:48:46.704 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.703+0000 7f658dd9b700 1 --2- 172.21.15.27:0/2689911614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65900f66e0 0x7f659006b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:46.704 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.703+0000 7f658dd9b700 1 -- 172.21.15.27:0/2689911614 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6580005040 con 0x7f65900f6ff0 2021-11-03T17:48:46.704 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.703+0000 7f658dd9b700 1 --2- 172.21.15.27:0/2689911614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65900f6ff0 0x7f6590069e20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7f65840026d0 tx=0x7f6584009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:48:46.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.703+0000 7f657effd700 1 -- 172.21.15.27:0/2689911614 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65840079c0 con 0x7f65900f6ff0 2021-11-03T17:48:46.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.703+0000 7f657effd700 1 -- 172.21.15.27:0/2689911614 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f658400f040 con 0x7f65900f6ff0 2021-11-03T17:48:46.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.703+0000 7f657effd700 1 -- 172.21.15.27:0/2689911614 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f65840148b0 con 0x7f65900f6ff0 2021-11-03T17:48:46.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.703+0000 7f6594fe6700 1 -- 172.21.15.27:0/2689911614 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f659006a7a0 con 0x7f65900f6ff0 2021-11-03T17:48:46.707 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.703+0000 7f6594fe6700 1 -- 172.21.15.27:0/2689911614 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f659010ef90 con 0x7f65900f6ff0 2021-11-03T17:48:46.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.704+0000 7f6594fe6700 1 -- 172.21.15.27:0/2689911614 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6590045ad0 con 0x7f65900f6ff0 2021-11-03T17:48:46.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.705+0000 7f657effd700 1 -- 172.21.15.27:0/2689911614 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f6584007b20 con 0x7f65900f6ff0 2021-11-03T17:48:46.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.705+0000 7f657effd700 1 --2- 172.21.15.27:0/2689911614 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f65780640b0 0x7f6578066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:48:46.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.705+0000 7f657effd700 1 -- 172.21.15.27:0/2689911614 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f6584080bf0 con 0x7f65900f6ff0 2021-11-03T17:48:46.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.707+0000 7f658e59c700 1 --2- 172.21.15.27:0/2689911614 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f65780640b0 0x7f6578066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:48:46.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.707+0000 7f658e59c700 1 --2- 172.21.15.27:0/2689911614 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f65780640b0 0x7f6578066570 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f6580004630 tx=0x7f6580009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:48:46.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.709+0000 7f657effd700 1 -- 172.21.15.27:0/2689911614 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f6584052ef0 con 0x7f65900f6ff0 2021-11-03T17:48:46.946 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.951+0000 7f6594fe6700 1 -- 172.21.15.27:0/2689911614 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6590027a00 con 0x7f65900f6ff0 2021-11-03T17:48:46.947 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.952+0000 7f657effd700 1 -- 172.21.15.27:0/2689911614 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f6584052ae0 con 0x7f65900f6ff0 2021-11-03T17:48:46.947 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:48:46.948 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:48:46.948 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:48:46.948 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:48:46.948 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:48:46.949 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:48:46.949 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:48:46.949 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:48:46.949 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:48:46.950 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:48:46.950 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:48:46.950 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:48:46.950 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:48:46.951 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:48:46.951 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:48:46.951 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:48:46.951 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:48:46.952 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:48:46.953 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.954+0000 7f6594fe6700 1 -- 172.21.15.27:0/2689911614 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f65780640b0 msgr2=0x7f6578066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:48:46.954 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.954+0000 7f6594fe6700 1 --2- 172.21.15.27:0/2689911614 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f65780640b0 0x7f6578066570 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 rx=0x7f6580004630 tx=0x7f6580009b30).stop 2021-11-03T17:48:46.954 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.954+0000 7f6594fe6700 1 -- 172.21.15.27:0/2689911614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65900f6ff0 msgr2=0x7f6590069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:48:46.954 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.954+0000 7f6594fe6700 1 --2- 172.21.15.27:0/2689911614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65900f6ff0 0x7f6590069e20 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7f65840026d0 tx=0x7f6584009de0).stop 2021-11-03T17:48:46.955 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.954+0000 7f6594fe6700 1 -- 172.21.15.27:0/2689911614 shutdown_connections 2021-11-03T17:48:46.955 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.954+0000 7f6594fe6700 1 --2- 172.21.15.27:0/2689911614 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f65780640b0 0x7f6578066570 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:46.955 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.954+0000 7f6594fe6700 1 --2- 172.21.15.27:0/2689911614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f65900f66e0 0x7f659006b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:46.956 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.954+0000 7f6594fe6700 1 --2- 172.21.15.27:0/2689911614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f65900f6ff0 0x7f6590069e20 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:46.956 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.954+0000 7f6594fe6700 1 -- 172.21.15.27:0/2689911614 >> 172.21.15.27:0/2689911614 conn(0x7f65900f1f00 msgr2=0x7f65900fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:48:46.957 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.954+0000 7f6594fe6700 1 -- 172.21.15.27:0/2689911614 shutdown_connections 2021-11-03T17:48:46.957 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:46.954+0000 7f6594fe6700 1 -- 172.21.15.27:0/2689911614 wait complete. 2021-11-03T17:48:47.063 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.067+0000 7f9664e23700 1 -- 172.21.15.27:0/1238349487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96600f7980 msgr2=0x7f96600f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:48:47.063 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.068+0000 7f9664e23700 1 --2- 172.21.15.27:0/1238349487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96600f7980 0x7f96600f7de0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 rx=0x7f9654004660 tx=0x7f9654009b30).stop 2021-11-03T17:48:47.063 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.068+0000 7f9664e23700 1 -- 172.21.15.27:0/1238349487 shutdown_connections 2021-11-03T17:48:47.064 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.068+0000 7f9664e23700 1 --2- 172.21.15.27:0/1238349487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96600f7980 0x7f96600f7de0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:47.064 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.068+0000 7f9664e23700 1 --2- 172.21.15.27:0/1238349487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96600f66e0 0x7f96600f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:47.064 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.068+0000 7f9664e23700 1 -- 172.21.15.27:0/1238349487 >> 172.21.15.27:0/1238349487 conn(0x7f96600f1e40 msgr2=0x7f96600f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:48:47.064 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.068+0000 7f9664e23700 1 -- 172.21.15.27:0/1238349487 shutdown_connections 2021-11-03T17:48:47.065 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.068+0000 7f9664e23700 1 -- 172.21.15.27:0/1238349487 wait complete. 2021-11-03T17:48:47.065 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.068+0000 7f9664e23700 1 Processor -- start 2021-11-03T17:48:47.065 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.068+0000 7f9664e23700 1 -- start start 2021-11-03T17:48:47.068 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.068+0000 7f9664e23700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96600f66e0 0x7f9660106f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:48:47.068 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.068+0000 7f9664e23700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96600f7980 0x7f9660107430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:48:47.068 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.069+0000 7f9664e23700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96601079e0 con 0x7f96600f66e0 2021-11-03T17:48:47.068 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.069+0000 7f9664e23700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9660107b20 con 0x7f96600f7980 2021-11-03T17:48:47.069 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.069+0000 7f965e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96600f66e0 0x7f9660106f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:48:47.069 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.069+0000 7f965e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96600f66e0 0x7f9660106f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59558/0 (socket says 172.21.15.27:59558) 2021-11-03T17:48:47.069 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.069+0000 7f965e59c700 1 -- 172.21.15.27:0/318382496 learned_addr learned my addr 172.21.15.27:0/318382496 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:48:47.069 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.069+0000 7f965dd9b700 1 --2- 172.21.15.27:0/318382496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96600f7980 0x7f9660107430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:48:47.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.069+0000 7f965e59c700 1 -- 172.21.15.27:0/318382496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96600f7980 msgr2=0x7f9660107430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:48:47.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.069+0000 7f965e59c700 1 --2- 172.21.15.27:0/318382496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96600f7980 0x7f9660107430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:47.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.069+0000 7f965e59c700 1 -- 172.21.15.27:0/318382496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9654005040 con 0x7f96600f66e0 2021-11-03T17:48:47.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.069+0000 7f965e59c700 1 --2- 172.21.15.27:0/318382496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96600f66e0 0x7f9660106f40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f9650002700 tx=0x7f9650009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:48:47.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.069+0000 7f964effd700 1 -- 172.21.15.27:0/318382496 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96500079c0 con 0x7f96600f66e0 2021-11-03T17:48:47.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.069+0000 7f9664e23700 1 -- 172.21.15.27:0/318382496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96601084c0 con 0x7f96600f66e0 2021-11-03T17:48:47.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.069+0000 7f964effd700 1 -- 172.21.15.27:0/318382496 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f965000f040 con 0x7f96600f66e0 2021-11-03T17:48:47.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.069+0000 7f964effd700 1 -- 172.21.15.27:0/318382496 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9650014860 con 0x7f96600f66e0 2021-11-03T17:48:47.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.070+0000 7f9664e23700 1 -- 172.21.15.27:0/318382496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9660108a80 con 0x7f96600f66e0 2021-11-03T17:48:47.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.071+0000 7f9664e23700 1 -- 172.21.15.27:0/318382496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9660045ad0 con 0x7f96600f66e0 2021-11-03T17:48:47.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.071+0000 7f964effd700 1 -- 172.21.15.27:0/318382496 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f965001b060 con 0x7f96600f66e0 2021-11-03T17:48:47.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.072+0000 7f964effd700 1 --2- 172.21.15.27:0/318382496 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f96480640b0 0x7f9648066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:48:47.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.072+0000 7f964effd700 1 -- 172.21.15.27:0/318382496 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f96500806d0 con 0x7f96600f66e0 2021-11-03T17:48:47.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.072+0000 7f965dd9b700 1 --2- 172.21.15.27:0/318382496 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f96480640b0 0x7f9648066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:48:47.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.072+0000 7f965dd9b700 1 --2- 172.21.15.27:0/318382496 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f96480640b0 0x7f9648066570 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f9654004630 tx=0x7f9654009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:48:47.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.079+0000 7f964effd700 1 -- 172.21.15.27:0/318382496 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f96500529d0 con 0x7f96600f66e0 2021-11-03T17:48:47.321 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.324+0000 7f9664e23700 1 -- 172.21.15.27:0/318382496 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f96600ff020 con 0x7f96600f66e0 2021-11-03T17:48:47.321 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.325+0000 7f964effd700 1 -- 172.21.15.27:0/318382496 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f9650019070 con 0x7f96600f66e0 2021-11-03T17:48:47.322 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:48:47.322 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:48:47.322 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:48:47.322 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:48:47.323 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:48:47.323 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:48:47.323 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:48:47.323 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:48:47.324 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:48:47.324 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:48:47.324 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:48:47.324 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:48:47.325 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:48:47.325 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:48:47.325 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:48:47.325 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:48:47.326 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:48:47.326 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:48:47.326 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:48:47.326 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:48:47.327 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:48:47.327 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:48:47.327 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:48:47.327 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:48:47.327 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:48:47.328 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:48:47.328 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:48:47.328 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:48:47.328 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:48:47.329 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:48:47.329 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:48:47.329 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:48:47.329 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:48:47.330 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:48:47.330 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:48:47.330 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:48:47.330 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:48:47.331 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:48:47.331 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:48:47.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.327+0000 7f9664e23700 1 -- 172.21.15.27:0/318382496 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f96480640b0 msgr2=0x7f9648066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:48:47.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.327+0000 7f9664e23700 1 --2- 172.21.15.27:0/318382496 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f96480640b0 0x7f9648066570 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f9654004630 tx=0x7f9654009b30).stop 2021-11-03T17:48:47.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.327+0000 7f9664e23700 1 -- 172.21.15.27:0/318382496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96600f66e0 msgr2=0x7f9660106f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:48:47.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.327+0000 7f9664e23700 1 --2- 172.21.15.27:0/318382496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96600f66e0 0x7f9660106f40 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f9650002700 tx=0x7f9650009d80).stop 2021-11-03T17:48:47.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.327+0000 7f9664e23700 1 -- 172.21.15.27:0/318382496 shutdown_connections 2021-11-03T17:48:47.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.327+0000 7f9664e23700 1 --2- 172.21.15.27:0/318382496 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f96480640b0 0x7f9648066570 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:47.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.327+0000 7f9664e23700 1 --2- 172.21.15.27:0/318382496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96600f66e0 0x7f9660106f40 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:47.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.327+0000 7f9664e23700 1 --2- 172.21.15.27:0/318382496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96600f7980 0x7f9660107430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:48:47.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.327+0000 7f9664e23700 1 -- 172.21.15.27:0/318382496 >> 172.21.15.27:0/318382496 conn(0x7f96600f1e40 msgr2=0x7f96600fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:48:47.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.327+0000 7f9664e23700 1 -- 172.21.15.27:0/318382496 shutdown_connections 2021-11-03T17:48:47.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:48:47.327+0000 7f9664e23700 1 -- 172.21.15.27:0/318382496 wait complete. 2021-11-03T17:48:47.334 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:48:47.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:47 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2689911614' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:47.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:47 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2689911614' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:48.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:48 smithi115 ceph-mon[65453]: from='client.35160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:48:48.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:48 smithi115 ceph-mon[65453]: pgmap v1024: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:48.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:48 smithi115 ceph-mon[65453]: from='client.35164 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:48:48.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:48 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/318382496' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:48:48.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:48:48.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:48:48.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:48:48.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:48.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:48.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:48.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:48.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:48 smithi027 ceph-mon[97244]: from='client.35160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:48:48.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:48 smithi027 ceph-mon[97244]: pgmap v1024: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:48.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:48 smithi027 ceph-mon[97244]: from='client.35164 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:48:48.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:48 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/318382496' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:48:48.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:48:48.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:48:48.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:48:48.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:48.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:48.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:48.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:49.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:49 smithi115 ceph-mon[65453]: poll took 5272.243 ms: 1 events 2021-11-03T17:48:49.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:49 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:48:49.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:49 smithi027 ceph-mon[97244]: poll took 5272.243 ms: 1 events 2021-11-03T17:48:49.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:49 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:48:50.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:50 smithi115 ceph-mon[65453]: pgmap v1025: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:50.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1972.406728 secs 2021-11-03T17:48:50.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:50 smithi027 ceph-mon[97244]: pgmap v1025: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:50.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1972.406728 secs 2021-11-03T17:48:52.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:52 smithi115 ceph-mon[65453]: pgmap v1026: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:52.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:52 smithi027 ceph-mon[97244]: pgmap v1026: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:54.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:54 smithi115 ceph-mon[65453]: pgmap v1027: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:54.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:54 smithi027 ceph-mon[97244]: pgmap v1027: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:55.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1977.406808 secs 2021-11-03T17:48:55.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1977.406808 secs 2021-11-03T17:48:56.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:56 smithi115 ceph-mon[65453]: pgmap v1028: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:56.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:48:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:48:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:56 smithi027 ceph-mon[97244]: pgmap v1028: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:48:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:48:58.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:58 smithi115 ceph-mon[65453]: pgmap v1029: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:58.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:48:58.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:48:58.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:48:58.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:58.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:58 smithi027 ceph-mon[97244]: pgmap v1029: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:48:58.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:48:58.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:48:58.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:48:58.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:59.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:59.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:59.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:59.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:48:59 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:48:59.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:59.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:59.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:48:59.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:48:59 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:49:00.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:00 smithi027 ceph-mon[97244]: pgmap v1030: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:00.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1982.406923 secs 2021-11-03T17:49:00.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:00 smithi115 ceph-mon[65453]: pgmap v1030: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:00.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1982.406923 secs 2021-11-03T17:49:02.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:02 smithi027 ceph-mon[97244]: pgmap v1031: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:02.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:02 smithi115 ceph-mon[65453]: pgmap v1031: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:04.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:04 smithi027 ceph-mon[97244]: pgmap v1032: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:04.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:04 smithi115 ceph-mon[65453]: pgmap v1032: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:05.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1987.407065 secs 2021-11-03T17:49:05.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1987.407065 secs 2021-11-03T17:49:06.517 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:06 smithi027 ceph-mon[97244]: pgmap v1033: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:06.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:06 smithi115 ceph-mon[65453]: pgmap v1033: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:08.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:08 smithi027 ceph-mon[97244]: pgmap v1034: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:08.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:49:08.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:49:08.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:08 smithi115 ceph-mon[65453]: pgmap v1034: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:08.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:49:08.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:49:09.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:49:09.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:09.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:09.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:09.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:09.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:09 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:49:09.767 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:49:09.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:09.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:09.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:09.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:09.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:09 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:49:10.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:10 smithi115 ceph-mon[65453]: pgmap v1035: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:10.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1992.407193 secs 2021-11-03T17:49:10.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:10 smithi027 ceph-mon[97244]: pgmap v1035: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:10.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1992.407193 secs 2021-11-03T17:49:12.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:12 smithi115 ceph-mon[65453]: pgmap v1036: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:12.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:12 smithi027 ceph-mon[97244]: pgmap v1036: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:14.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:14 smithi115 ceph-mon[65453]: pgmap v1037: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:14.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:14 smithi027 ceph-mon[97244]: pgmap v1037: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:15.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 1997.407312 secs 2021-11-03T17:49:15.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 1997.407312 secs 2021-11-03T17:49:16.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:16 smithi115 ceph-mon[65453]: pgmap v1038: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:16.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:16 smithi027 ceph-mon[97244]: pgmap v1038: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:17.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.444+0000 7f51764b4700 1 -- 172.21.15.27:0/357089312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51700f66e0 msgr2=0x7f51700f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:49:17.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.444+0000 7f51764b4700 1 --2- 172.21.15.27:0/357089312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51700f66e0 0x7f51700f6b00 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f5160004660 tx=0x7f5160009b30).stop 2021-11-03T17:49:17.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.444+0000 7f51764b4700 1 -- 172.21.15.27:0/357089312 shutdown_connections 2021-11-03T17:49:17.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.444+0000 7f51764b4700 1 --2- 172.21.15.27:0/357089312 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51700f6ff0 0x7f51700f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:17.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.444+0000 7f51764b4700 1 --2- 172.21.15.27:0/357089312 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51700f66e0 0x7f51700f6b00 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:17.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.444+0000 7f51764b4700 1 -- 172.21.15.27:0/357089312 >> 172.21.15.27:0/357089312 conn(0x7f51700f1f00 msgr2=0x7f51700f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:49:17.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.444+0000 7f51764b4700 1 -- 172.21.15.27:0/357089312 shutdown_connections 2021-11-03T17:49:17.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.444+0000 7f51764b4700 1 -- 172.21.15.27:0/357089312 wait complete. 2021-11-03T17:49:17.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.444+0000 7f51764b4700 1 Processor -- start 2021-11-03T17:49:17.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.444+0000 7f51764b4700 1 -- start start 2021-11-03T17:49:17.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.445+0000 7f51764b4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51700f66e0 0x7f517006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:49:17.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.445+0000 7f51764b4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51700f6ff0 0x7f5170069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:49:17.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.445+0000 7f51764b4700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517006a380 con 0x7f51700f6ff0 2021-11-03T17:49:17.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.445+0000 7f51764b4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517006a4c0 con 0x7f51700f66e0 2021-11-03T17:49:17.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.445+0000 7f516f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51700f6ff0 0x7f5170069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:49:17.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.445+0000 7f516f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51700f6ff0 0x7f5170069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59578/0 (socket says 172.21.15.27:59578) 2021-11-03T17:49:17.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.445+0000 7f516f7fe700 1 -- 172.21.15.27:0/584423958 learned_addr learned my addr 172.21.15.27:0/584423958 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:49:17.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.445+0000 7f516ffff700 1 --2- 172.21.15.27:0/584423958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51700f66e0 0x7f517006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:49:17.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.445+0000 7f516f7fe700 1 -- 172.21.15.27:0/584423958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51700f66e0 msgr2=0x7f517006b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:49:17.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.445+0000 7f516f7fe700 1 --2- 172.21.15.27:0/584423958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51700f66e0 0x7f517006b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:17.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.445+0000 7f516f7fe700 1 -- 172.21.15.27:0/584423958 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5160005040 con 0x7f51700f6ff0 2021-11-03T17:49:17.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.445+0000 7f516f7fe700 1 --2- 172.21.15.27:0/584423958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51700f6ff0 0x7f5170069e20 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f51580026d0 tx=0x7f5158009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:49:17.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.445+0000 7f516cff9700 1 -- 172.21.15.27:0/584423958 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5158009320 con 0x7f51700f6ff0 2021-11-03T17:49:17.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.445+0000 7f516cff9700 1 -- 172.21.15.27:0/584423958 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f515800f040 con 0x7f51700f6ff0 2021-11-03T17:49:17.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.445+0000 7f51764b4700 1 -- 172.21.15.27:0/584423958 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f517006a7a0 con 0x7f51700f6ff0 2021-11-03T17:49:17.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.446+0000 7f516cff9700 1 -- 172.21.15.27:0/584423958 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51580147e0 con 0x7f51700f6ff0 2021-11-03T17:49:17.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.446+0000 7f51764b4700 1 -- 172.21.15.27:0/584423958 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f517010ef90 con 0x7f51700f6ff0 2021-11-03T17:49:17.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.446+0000 7f51764b4700 1 -- 172.21.15.27:0/584423958 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5170045ad0 con 0x7f51700f6ff0 2021-11-03T17:49:17.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.447+0000 7f516cff9700 1 -- 172.21.15.27:0/584423958 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f5158009480 con 0x7f51700f6ff0 2021-11-03T17:49:17.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.447+0000 7f516cff9700 1 --2- 172.21.15.27:0/584423958 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f515c0640b0 0x7f515c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:49:17.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.447+0000 7f516cff9700 1 -- 172.21.15.27:0/584423958 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f5158080f10 con 0x7f51700f6ff0 2021-11-03T17:49:17.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.448+0000 7f516ffff700 1 --2- 172.21.15.27:0/584423958 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f515c0640b0 0x7f515c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:49:17.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.448+0000 7f516ffff700 1 --2- 172.21.15.27:0/584423958 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f515c0640b0 0x7f515c066570 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f5160004630 tx=0x7f5160009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:49:17.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.452+0000 7f516cff9700 1 -- 172.21.15.27:0/584423958 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f5158053210 con 0x7f51700f6ff0 2021-11-03T17:49:17.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.634+0000 7f51764b4700 1 -- 172.21.15.27:0/584423958 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f51700fefb0 con 0x7f515c0640b0 2021-11-03T17:49:17.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.635+0000 7f516cff9700 1 -- 172.21.15.27:0/584423958 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f51700fefb0 con 0x7f515c0640b0 2021-11-03T17:49:17.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.637+0000 7f51764b4700 1 -- 172.21.15.27:0/584423958 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f515c0640b0 msgr2=0x7f515c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:49:17.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.637+0000 7f51764b4700 1 --2- 172.21.15.27:0/584423958 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f515c0640b0 0x7f515c066570 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 rx=0x7f5160004630 tx=0x7f5160009b30).stop 2021-11-03T17:49:17.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.637+0000 7f51764b4700 1 -- 172.21.15.27:0/584423958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51700f6ff0 msgr2=0x7f5170069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:49:17.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.637+0000 7f51764b4700 1 --2- 172.21.15.27:0/584423958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51700f6ff0 0x7f5170069e20 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f51580026d0 tx=0x7f5158009de0).stop 2021-11-03T17:49:17.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.637+0000 7f51764b4700 1 -- 172.21.15.27:0/584423958 shutdown_connections 2021-11-03T17:49:17.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.637+0000 7f51764b4700 1 --2- 172.21.15.27:0/584423958 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f515c0640b0 0x7f515c066570 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:17.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.637+0000 7f51764b4700 1 --2- 172.21.15.27:0/584423958 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51700f66e0 0x7f517006b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:17.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.637+0000 7f51764b4700 1 --2- 172.21.15.27:0/584423958 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51700f6ff0 0x7f5170069e20 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:17.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.637+0000 7f51764b4700 1 -- 172.21.15.27:0/584423958 >> 172.21.15.27:0/584423958 conn(0x7f51700f1f00 msgr2=0x7f51700fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:49:17.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.637+0000 7f51764b4700 1 -- 172.21.15.27:0/584423958 shutdown_connections 2021-11-03T17:49:17.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.637+0000 7f51764b4700 1 -- 172.21.15.27:0/584423958 wait complete. 2021-11-03T17:49:17.641 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:49:17.733 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.737+0000 7ff829b39700 1 -- 172.21.15.27:0/1760004521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8240f66e0 msgr2=0x7ff8240f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:49:17.733 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.737+0000 7ff829b39700 1 --2- 172.21.15.27:0/1760004521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8240f66e0 0x7ff8240f6b40 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7ff810004660 tx=0x7ff810009b30).stop 2021-11-03T17:49:17.734 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.737+0000 7ff829b39700 1 -- 172.21.15.27:0/1760004521 shutdown_connections 2021-11-03T17:49:17.734 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.737+0000 7ff829b39700 1 --2- 172.21.15.27:0/1760004521 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8240f66e0 0x7ff8240f6b40 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:17.734 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.737+0000 7ff829b39700 1 --2- 172.21.15.27:0/1760004521 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8240f5440 0x7ff8240f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:17.734 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.737+0000 7ff829b39700 1 -- 172.21.15.27:0/1760004521 >> 172.21.15.27:0/1760004521 conn(0x7ff8240f0ba0 msgr2=0x7ff8240f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:49:17.735 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.737+0000 7ff829b39700 1 -- 172.21.15.27:0/1760004521 shutdown_connections 2021-11-03T17:49:17.735 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.737+0000 7ff829b39700 1 -- 172.21.15.27:0/1760004521 wait complete. 2021-11-03T17:49:17.735 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.738+0000 7ff829b39700 1 Processor -- start 2021-11-03T17:49:17.735 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.738+0000 7ff829b39700 1 -- start start 2021-11-03T17:49:17.736 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.738+0000 7ff829b39700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8240f5440 0x7ff824109a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:49:17.736 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.738+0000 7ff829b39700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8240f66e0 0x7ff82410bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:49:17.736 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.738+0000 7ff829b39700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8240603e0 con 0x7ff8240f66e0 2021-11-03T17:49:17.736 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.738+0000 7ff829b39700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff824060520 con 0x7ff8240f5440 2021-11-03T17:49:17.737 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.738+0000 7ff823fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8240f66e0 0x7ff82410bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:49:17.737 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.738+0000 7ff823fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8240f66e0 0x7ff82410bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59588/0 (socket says 172.21.15.27:59588) 2021-11-03T17:49:17.737 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.738+0000 7ff823fff700 1 -- 172.21.15.27:0/1237907541 learned_addr learned my addr 172.21.15.27:0/1237907541 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:49:17.737 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.739+0000 7ff828b37700 1 --2- 172.21.15.27:0/1237907541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8240f5440 0x7ff824109a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:49:17.737 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.739+0000 7ff823fff700 1 -- 172.21.15.27:0/1237907541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8240f5440 msgr2=0x7ff824109a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:49:17.738 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.739+0000 7ff823fff700 1 --2- 172.21.15.27:0/1237907541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8240f5440 0x7ff824109a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:17.738 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.739+0000 7ff823fff700 1 -- 172.21.15.27:0/1237907541 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff810005040 con 0x7ff8240f66e0 2021-11-03T17:49:17.738 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.739+0000 7ff823fff700 1 --2- 172.21.15.27:0/1237907541 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8240f66e0 0x7ff82410bf90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7ff810009990 tx=0x7ff810009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:49:17.738 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.739+0000 7ff8217fa700 1 -- 172.21.15.27:0/1237907541 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff810018070 con 0x7ff8240f66e0 2021-11-03T17:49:17.739 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.739+0000 7ff829b39700 1 -- 172.21.15.27:0/1237907541 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff82410c4e0 con 0x7ff8240f66e0 2021-11-03T17:49:17.739 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.739+0000 7ff8217fa700 1 -- 172.21.15.27:0/1237907541 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7ff810007ba0 con 0x7ff8240f66e0 2021-11-03T17:49:17.739 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.739+0000 7ff8217fa700 1 -- 172.21.15.27:0/1237907541 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff81001c3c0 con 0x7ff8240f66e0 2021-11-03T17:49:17.739 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.739+0000 7ff829b39700 1 -- 172.21.15.27:0/1237907541 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff82410ca40 con 0x7ff8240f66e0 2021-11-03T17:49:17.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:17 smithi115 ceph-mon[65453]: pgmap v1039: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:17.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.740+0000 7ff829b39700 1 -- 172.21.15.27:0/1237907541 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff824045ad0 con 0x7ff8240f66e0 2021-11-03T17:49:17.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.741+0000 7ff8217fa700 1 -- 172.21.15.27:0/1237907541 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7ff81001d420 con 0x7ff8240f66e0 2021-11-03T17:49:17.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.741+0000 7ff8217fa700 1 --2- 172.21.15.27:0/1237907541 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff814064100 0x7ff8140665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:49:17.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.741+0000 7ff8217fa700 1 -- 172.21.15.27:0/1237907541 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7ff810082560 con 0x7ff8240f66e0 2021-11-03T17:49:17.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.741+0000 7ff828b37700 1 --2- 172.21.15.27:0/1237907541 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff814064100 0x7ff8140665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:49:17.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.741+0000 7ff828b37700 1 --2- 172.21.15.27:0/1237907541 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff814064100 0x7ff8140665c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7ff824060060 tx=0x7ff81800b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:49:17.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.745+0000 7ff8217fa700 1 -- 172.21.15.27:0/1237907541 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7ff8100548c0 con 0x7ff8240f66e0 2021-11-03T17:49:17.767 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:17 smithi027 ceph-mon[97244]: pgmap v1039: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:17.912 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.918+0000 7ff829b39700 1 -- 172.21.15.27:0/1237907541 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff8240fdd40 con 0x7ff814064100 2021-11-03T17:49:17.927 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.933+0000 7ff8217fa700 1 -- 172.21.15.27:0/1237907541 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7ff8240fdd40 con 0x7ff814064100 2021-11-03T17:49:17.927 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:49:17.928 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (40m) 29s ago 42m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:49:17.928 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (32m) 29s ago 42m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:49:17.928 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (32m) 9m ago 41m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:49:17.928 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (40m) 29s ago 41m 41.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:49:17.929 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (34m) 29s ago 37m 23.8M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:49:17.929 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (37m) 29s ago 37m 3844M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:49:17.929 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (33m) 9m ago 37m 20.4M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:49:17.929 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (34m) 9m ago 37m 22.1M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:49:17.930 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (34m) 29s ago 43m 535M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:49:17.930 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (33m) 9m ago 41m 419M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:49:17.930 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (33m) 29s ago 43m 198M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:49:17.930 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (33m) 9m ago 41m 82.5M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:49:17.931 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (41m) 29s ago 41m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:49:17.931 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (41m) 9m ago 41m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:49:17.931 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (32m) 29s ago 40m 331M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:49:17.931 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (31m) 29s ago 40m 361M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:49:17.931 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (31m) 29s ago 39m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:49:17.932 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (30m) 9m ago 39m 354M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:49:17.932 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (30m) 9m ago 39m 285M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:49:17.932 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (29m) 9m ago 38m 287M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:49:17.932 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (40m) 29s ago 41m 52.9M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:49:17.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.935+0000 7ff829b39700 1 -- 172.21.15.27:0/1237907541 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff814064100 msgr2=0x7ff8140665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:49:17.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.935+0000 7ff829b39700 1 --2- 172.21.15.27:0/1237907541 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff814064100 0x7ff8140665c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7ff824060060 tx=0x7ff81800b040).stop 2021-11-03T17:49:17.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.935+0000 7ff829b39700 1 -- 172.21.15.27:0/1237907541 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8240f66e0 msgr2=0x7ff82410bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:49:17.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.935+0000 7ff829b39700 1 --2- 172.21.15.27:0/1237907541 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8240f66e0 0x7ff82410bf90 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7ff810009990 tx=0x7ff810009b30).stop 2021-11-03T17:49:17.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.935+0000 7ff829b39700 1 -- 172.21.15.27:0/1237907541 shutdown_connections 2021-11-03T17:49:17.935 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.935+0000 7ff829b39700 1 --2- 172.21.15.27:0/1237907541 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff814064100 0x7ff8140665c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:17.935 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.935+0000 7ff829b39700 1 --2- 172.21.15.27:0/1237907541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8240f5440 0x7ff824109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:17.935 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.935+0000 7ff829b39700 1 --2- 172.21.15.27:0/1237907541 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8240f66e0 0x7ff82410bf90 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:17.935 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.935+0000 7ff829b39700 1 -- 172.21.15.27:0/1237907541 >> 172.21.15.27:0/1237907541 conn(0x7ff8240f0ba0 msgr2=0x7ff8240fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:49:17.936 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.935+0000 7ff829b39700 1 -- 172.21.15.27:0/1237907541 shutdown_connections 2021-11-03T17:49:17.936 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:17.935+0000 7ff829b39700 1 -- 172.21.15.27:0/1237907541 wait complete. 2021-11-03T17:49:18.031 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.035+0000 7fb316a23700 1 -- 172.21.15.27:0/984015629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb310060250 msgr2=0x7fb3100f9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:49:18.031 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.035+0000 7fb316a23700 1 --2- 172.21.15.27:0/984015629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb310060250 0x7fb3100f9360 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7fb2f8004660 tx=0x7fb2f8009b30).stop 2021-11-03T17:49:18.032 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.035+0000 7fb316a23700 1 -- 172.21.15.27:0/984015629 shutdown_connections 2021-11-03T17:49:18.032 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.035+0000 7fb316a23700 1 --2- 172.21.15.27:0/984015629 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3100f9850 0x7fb3100fbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:18.032 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.035+0000 7fb316a23700 1 --2- 172.21.15.27:0/984015629 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb310060250 0x7fb3100f9360 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:18.032 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.035+0000 7fb316a23700 1 -- 172.21.15.27:0/984015629 >> 172.21.15.27:0/984015629 conn(0x7fb3100f0d40 msgr2=0x7fb3100f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:49:18.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.035+0000 7fb316a23700 1 -- 172.21.15.27:0/984015629 shutdown_connections 2021-11-03T17:49:18.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.035+0000 7fb316a23700 1 -- 172.21.15.27:0/984015629 wait complete. 2021-11-03T17:49:18.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.035+0000 7fb316a23700 1 Processor -- start 2021-11-03T17:49:18.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.036+0000 7fb316a23700 1 -- start start 2021-11-03T17:49:18.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.036+0000 7fb316a23700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb310060250 0x7fb310106fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:49:18.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.036+0000 7fb316a23700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3100f9850 0x7fb310107490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:49:18.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.036+0000 7fb316a23700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb310107a40 con 0x7fb3100f9850 2021-11-03T17:49:18.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.036+0000 7fb316a23700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb310107b80 con 0x7fb310060250 2021-11-03T17:49:18.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.036+0000 7fb30f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3100f9850 0x7fb310107490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:49:18.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.036+0000 7fb30f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3100f9850 0x7fb310107490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59598/0 (socket says 172.21.15.27:59598) 2021-11-03T17:49:18.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.036+0000 7fb30f7fe700 1 -- 172.21.15.27:0/1802683808 learned_addr learned my addr 172.21.15.27:0/1802683808 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:49:18.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.036+0000 7fb30ffff700 1 --2- 172.21.15.27:0/1802683808 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb310060250 0x7fb310106fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:49:18.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.036+0000 7fb30f7fe700 1 -- 172.21.15.27:0/1802683808 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb310060250 msgr2=0x7fb310106fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:49:18.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.036+0000 7fb30f7fe700 1 --2- 172.21.15.27:0/1802683808 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb310060250 0x7fb310106fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:18.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.036+0000 7fb30f7fe700 1 -- 172.21.15.27:0/1802683808 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2f8005040 con 0x7fb3100f9850 2021-11-03T17:49:18.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.036+0000 7fb30f7fe700 1 --2- 172.21.15.27:0/1802683808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3100f9850 0x7fb310107490 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7fb30000c560 tx=0x7fb300007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:49:18.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.037+0000 7fb30cff9700 1 -- 172.21.15.27:0/1802683808 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb30000f650 con 0x7fb3100f9850 2021-11-03T17:49:18.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.037+0000 7fb30cff9700 1 -- 172.21.15.27:0/1802683808 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fb30000fc60 con 0x7fb3100f9850 2021-11-03T17:49:18.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.037+0000 7fb30cff9700 1 -- 172.21.15.27:0/1802683808 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb300012ca0 con 0x7fb3100f9850 2021-11-03T17:49:18.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.037+0000 7fb316a23700 1 -- 172.21.15.27:0/1802683808 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb310108520 con 0x7fb3100f9850 2021-11-03T17:49:18.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.037+0000 7fb316a23700 1 -- 172.21.15.27:0/1802683808 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb310108ae0 con 0x7fb3100f9850 2021-11-03T17:49:18.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.038+0000 7fb316a23700 1 -- 172.21.15.27:0/1802683808 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb310045ad0 con 0x7fb3100f9850 2021-11-03T17:49:18.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.038+0000 7fb30cff9700 1 -- 172.21.15.27:0/1802683808 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fb300010070 con 0x7fb3100f9850 2021-11-03T17:49:18.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.039+0000 7fb30cff9700 1 --2- 172.21.15.27:0/1802683808 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb2fc0640b0 0x7fb2fc066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:49:18.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.039+0000 7fb30cff9700 1 -- 172.21.15.27:0/1802683808 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fb300080620 con 0x7fb3100f9850 2021-11-03T17:49:18.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.039+0000 7fb30ffff700 1 --2- 172.21.15.27:0/1802683808 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb2fc0640b0 0x7fb2fc066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:49:18.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.039+0000 7fb30ffff700 1 --2- 172.21.15.27:0/1802683808 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb2fc0640b0 0x7fb2fc066570 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7fb2f8004e30 tx=0x7fb2f8009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:49:18.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.043+0000 7fb30cff9700 1 -- 172.21.15.27:0/1802683808 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fb300052830 con 0x7fb3100f9850 2021-11-03T17:49:18.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.273+0000 7fb316a23700 1 -- 172.21.15.27:0/1802683808 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb310027a00 con 0x7fb3100f9850 2021-11-03T17:49:18.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.273+0000 7fb30cff9700 1 -- 172.21.15.27:0/1802683808 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fb300052420 con 0x7fb3100f9850 2021-11-03T17:49:18.268 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:49:18.268 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:49:18.268 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:49:18.268 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:49:18.269 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:49:18.269 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:49:18.269 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:49:18.269 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:49:18.270 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:49:18.270 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:49:18.270 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:49:18.270 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:49:18.271 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:49:18.271 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:49:18.271 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:49:18.271 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:49:18.272 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:49:18.272 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:49:18.273 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.275+0000 7fb316a23700 1 -- 172.21.15.27:0/1802683808 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb2fc0640b0 msgr2=0x7fb2fc066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:49:18.273 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.275+0000 7fb316a23700 1 --2- 172.21.15.27:0/1802683808 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb2fc0640b0 0x7fb2fc066570 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7fb2f8004e30 tx=0x7fb2f8009b30).stop 2021-11-03T17:49:18.273 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.275+0000 7fb316a23700 1 -- 172.21.15.27:0/1802683808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3100f9850 msgr2=0x7fb310107490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:49:18.273 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.275+0000 7fb316a23700 1 --2- 172.21.15.27:0/1802683808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3100f9850 0x7fb310107490 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7fb30000c560 tx=0x7fb300007dc0).stop 2021-11-03T17:49:18.274 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.275+0000 7fb316a23700 1 -- 172.21.15.27:0/1802683808 shutdown_connections 2021-11-03T17:49:18.274 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.275+0000 7fb316a23700 1 --2- 172.21.15.27:0/1802683808 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb2fc0640b0 0x7fb2fc066570 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:18.274 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.275+0000 7fb316a23700 1 --2- 172.21.15.27:0/1802683808 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb310060250 0x7fb310106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:18.274 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.275+0000 7fb316a23700 1 --2- 172.21.15.27:0/1802683808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3100f9850 0x7fb310107490 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:18.275 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.275+0000 7fb316a23700 1 -- 172.21.15.27:0/1802683808 >> 172.21.15.27:0/1802683808 conn(0x7fb3100f0d40 msgr2=0x7fb3100f3130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:49:18.275 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.275+0000 7fb316a23700 1 -- 172.21.15.27:0/1802683808 shutdown_connections 2021-11-03T17:49:18.275 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.275+0000 7fb316a23700 1 -- 172.21.15.27:0/1802683808 wait complete. 2021-11-03T17:49:18.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.378+0000 7f69e6549700 1 -- 172.21.15.27:0/2723455820 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69e00602d0 msgr2=0x7f69e00f92e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:49:18.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.378+0000 7f69e6549700 1 --2- 172.21.15.27:0/2723455820 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69e00602d0 0x7f69e00f92e0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7f69d0004660 tx=0x7f69d0009b30).stop 2021-11-03T17:49:18.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.378+0000 7f69e6549700 1 -- 172.21.15.27:0/2723455820 shutdown_connections 2021-11-03T17:49:18.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.378+0000 7f69e6549700 1 --2- 172.21.15.27:0/2723455820 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69e00f9860 0x7f69e00fbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:18.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.378+0000 7f69e6549700 1 --2- 172.21.15.27:0/2723455820 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69e00602d0 0x7f69e00f92e0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:18.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.378+0000 7f69e6549700 1 -- 172.21.15.27:0/2723455820 >> 172.21.15.27:0/2723455820 conn(0x7f69e00f0d40 msgr2=0x7f69e00f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:49:18.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.378+0000 7f69e6549700 1 -- 172.21.15.27:0/2723455820 shutdown_connections 2021-11-03T17:49:18.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.378+0000 7f69e6549700 1 -- 172.21.15.27:0/2723455820 wait complete. 2021-11-03T17:49:18.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.378+0000 7f69e6549700 1 Processor -- start 2021-11-03T17:49:18.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.379+0000 7f69e6549700 1 -- start start 2021-11-03T17:49:18.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.379+0000 7f69e6549700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69e00602d0 0x7f69e0106f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:49:18.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.379+0000 7f69e6549700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69e00f9860 0x7f69e0107430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:49:18.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.379+0000 7f69e6549700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e01079e0 con 0x7f69e00f9860 2021-11-03T17:49:18.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.379+0000 7f69e6549700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69e0107b20 con 0x7f69e00602d0 2021-11-03T17:49:18.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.379+0000 7f69df7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69e00f9860 0x7f69e0107430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:49:18.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.379+0000 7f69df7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69e00f9860 0x7f69e0107430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59610/0 (socket says 172.21.15.27:59610) 2021-11-03T17:49:18.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.379+0000 7f69df7fe700 1 -- 172.21.15.27:0/3959589999 learned_addr learned my addr 172.21.15.27:0/3959589999 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:49:18.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.379+0000 7f69dffff700 1 --2- 172.21.15.27:0/3959589999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69e00602d0 0x7f69e0106f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:49:18.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.379+0000 7f69df7fe700 1 -- 172.21.15.27:0/3959589999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69e00602d0 msgr2=0x7f69e0106f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:49:18.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.379+0000 7f69df7fe700 1 --2- 172.21.15.27:0/3959589999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69e00602d0 0x7f69e0106f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:18.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.379+0000 7f69df7fe700 1 -- 172.21.15.27:0/3959589999 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69d0005040 con 0x7f69e00f9860 2021-11-03T17:49:18.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.379+0000 7f69df7fe700 1 --2- 172.21.15.27:0/3959589999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69e00f9860 0x7f69e0107430 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f69d400c790 tx=0x7f69d4007d40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:49:18.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.379+0000 7f69dcff9700 1 -- 172.21.15.27:0/3959589999 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69d4013630 con 0x7f69e00f9860 2021-11-03T17:49:18.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.380+0000 7f69dcff9700 1 -- 172.21.15.27:0/3959589999 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f69d4013c40 con 0x7f69e00f9860 2021-11-03T17:49:18.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.380+0000 7f69dcff9700 1 -- 172.21.15.27:0/3959589999 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f69d4002b80 con 0x7f69e00f9860 2021-11-03T17:49:18.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.380+0000 7f69e6549700 1 -- 172.21.15.27:0/3959589999 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69e01084c0 con 0x7f69e00f9860 2021-11-03T17:49:18.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.380+0000 7f69e6549700 1 -- 172.21.15.27:0/3959589999 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69e0108a30 con 0x7f69e00f9860 2021-11-03T17:49:18.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.381+0000 7f69e6549700 1 -- 172.21.15.27:0/3959589999 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69e0045ad0 con 0x7f69e00f9860 2021-11-03T17:49:18.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.381+0000 7f69dcff9700 1 -- 172.21.15.27:0/3959589999 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f69d401a060 con 0x7f69e00f9860 2021-11-03T17:49:18.382 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.381+0000 7f69dcff9700 1 --2- 172.21.15.27:0/3959589999 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f69c80640b0 0x7f69c8066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:49:18.382 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.381+0000 7f69dcff9700 1 -- 172.21.15.27:0/3959589999 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f69d4080ab0 con 0x7f69e00f9860 2021-11-03T17:49:18.382 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.382+0000 7f69dffff700 1 --2- 172.21.15.27:0/3959589999 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f69c80640b0 0x7f69c8066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:49:18.382 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.382+0000 7f69dffff700 1 --2- 172.21.15.27:0/3959589999 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f69c80640b0 0x7f69c8066570 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f69d0004630 tx=0x7f69d0009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:49:18.383 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.386+0000 7f69dcff9700 1 -- 172.21.15.27:0/3959589999 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f69d4052cc0 con 0x7f69e00f9860 2021-11-03T17:49:18.582 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.587+0000 7f69e6549700 1 -- 172.21.15.27:0/3959589999 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f69e0027a00 con 0x7f69e00f9860 2021-11-03T17:49:18.582 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.588+0000 7f69dcff9700 1 -- 172.21.15.27:0/3959589999 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f69d4018070 con 0x7f69e00f9860 2021-11-03T17:49:18.582 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:49:18.583 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:49:18.583 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:49:18.583 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:49:18.583 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:49:18.583 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:49:18.584 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:49:18.584 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:49:18.584 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:49:18.584 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:49:18.585 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:49:18.585 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:49:18.585 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:49:18.585 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:49:18.585 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:49:18.586 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:49:18.586 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:49:18.586 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:49:18.586 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:49:18.587 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:49:18.587 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:49:18.587 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:49:18.587 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:49:18.587 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:49:18.588 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:49:18.588 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:49:18.588 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:49:18.588 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:49:18.588 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:49:18.589 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:49:18.589 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:49:18.589 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:49:18.589 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:49:18.590 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:49:18.590 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:49:18.590 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:49:18.590 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:49:18.590 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:49:18.591 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:49:18.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.590+0000 7f69e6549700 1 -- 172.21.15.27:0/3959589999 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f69c80640b0 msgr2=0x7f69c8066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:49:18.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.590+0000 7f69e6549700 1 --2- 172.21.15.27:0/3959589999 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f69c80640b0 0x7f69c8066570 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 rx=0x7f69d0004630 tx=0x7f69d0009b30).stop 2021-11-03T17:49:18.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.590+0000 7f69e6549700 1 -- 172.21.15.27:0/3959589999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69e00f9860 msgr2=0x7f69e0107430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:49:18.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.590+0000 7f69e6549700 1 --2- 172.21.15.27:0/3959589999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69e00f9860 0x7f69e0107430 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7f69d400c790 tx=0x7f69d4007d40).stop 2021-11-03T17:49:18.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.590+0000 7f69e6549700 1 -- 172.21.15.27:0/3959589999 shutdown_connections 2021-11-03T17:49:18.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.590+0000 7f69e6549700 1 --2- 172.21.15.27:0/3959589999 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f69c80640b0 0x7f69c8066570 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:18.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.590+0000 7f69e6549700 1 --2- 172.21.15.27:0/3959589999 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69e00602d0 0x7f69e0106f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:18.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.590+0000 7f69e6549700 1 --2- 172.21.15.27:0/3959589999 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69e00f9860 0x7f69e0107430 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:18.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.590+0000 7f69e6549700 1 -- 172.21.15.27:0/3959589999 >> 172.21.15.27:0/3959589999 conn(0x7f69e00f0d40 msgr2=0x7f69e00f3130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:49:18.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.590+0000 7f69e6549700 1 -- 172.21.15.27:0/3959589999 shutdown_connections 2021-11-03T17:49:18.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:18.590+0000 7f69e6549700 1 -- 172.21.15.27:0/3959589999 wait complete. 2021-11-03T17:49:18.594 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:49:18.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:18 smithi115 ceph-mon[65453]: from='client.35176 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:49:18.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:18 smithi115 ceph-mon[65453]: from='client.35180 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:49:18.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:18 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1802683808' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:18.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:49:18.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:49:18.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:18 smithi027 ceph-mon[97244]: from='client.35176 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:49:18.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:18 smithi027 ceph-mon[97244]: from='client.35180 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:49:18.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:18 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1802683808' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:18.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:49:18.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:49:19.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:49:19.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:19.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:19.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:19.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:19.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:19 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:49:19.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:19 smithi115 ceph-mon[65453]: pgmap v1040: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:19.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:19 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3959589999' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:49:19.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:49:19.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:19.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:19.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:19.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:19.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:19 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:49:19.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:19 smithi027 ceph-mon[97244]: pgmap v1040: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:19.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:19 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3959589999' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:49:20.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2002.407447 secs 2021-11-03T17:49:20.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2002.407447 secs 2021-11-03T17:49:21.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:21 smithi115 ceph-mon[65453]: pgmap v1041: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:21.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:21 smithi027 ceph-mon[97244]: pgmap v1041: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:23.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:23 smithi115 ceph-mon[65453]: pgmap v1042: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:23.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:23 smithi027 ceph-mon[97244]: pgmap v1042: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:25.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:25 smithi115 ceph-mon[65453]: pgmap v1043: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:25.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2007.407536 secs 2021-11-03T17:49:25.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:25 smithi027 ceph-mon[97244]: pgmap v1043: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:25.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2007.407536 secs 2021-11-03T17:49:27.733 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:27 smithi115 ceph-mon[65453]: pgmap v1044: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:27.767 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:27 smithi027 ceph-mon[97244]: pgmap v1044: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:28.659 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:49:28.691 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:49:29.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:29 smithi115 ceph-mon[65453]: poll took 40137.322 ms: 1 events 2021-11-03T17:49:29.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:29 smithi115 ceph-mon[65453]: pgmap v1045: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:29.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:49:29.984 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:49:29.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:29.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:29.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:29.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:29.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:29 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:49:30.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:29 smithi027 ceph-mon[97244]: poll took 40137.322 ms: 1 events 2021-11-03T17:49:30.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:29 smithi027 ceph-mon[97244]: pgmap v1045: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:30.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:49:30.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:49:30.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:30.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:30.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:30.019 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:30.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:29 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:49:30.983 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2012.407637 secs 2021-11-03T17:49:31.017 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2012.407637 secs 2021-11-03T17:49:32.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:31 smithi115 ceph-mon[65453]: pgmap v1046: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:32.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:31 smithi027 ceph-mon[97244]: pgmap v1046: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:34.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:33 smithi115 ceph-mon[65453]: pgmap v1047: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:34.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:33 smithi027 ceph-mon[97244]: pgmap v1047: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:35.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:34 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2017.407738 secs 2021-11-03T17:49:35.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:34 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2017.407738 secs 2021-11-03T17:49:36.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:35 smithi115 ceph-mon[65453]: pgmap v1048: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:36.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:35 smithi027 ceph-mon[97244]: pgmap v1048: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:38.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:37 smithi115 ceph-mon[65453]: pgmap v1049: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:38.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:37 smithi027 ceph-mon[97244]: pgmap v1049: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:39.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:49:39.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:49:39.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:49:39.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:39.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:39.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:39.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:39.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:49:39.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:49:39.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:49:39.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:39.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:39.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:39.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:40.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:39 smithi115 ceph-mon[65453]: pgmap v1050: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:40.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:39 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:49:40.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:39 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2022.407869 secs 2021-11-03T17:49:40.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:39 smithi027 ceph-mon[97244]: pgmap v1050: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:40.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:39 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:49:40.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:39 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2022.407869 secs 2021-11-03T17:49:42.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:41 smithi115 ceph-mon[65453]: pgmap v1051: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:42.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:41 smithi027 ceph-mon[97244]: pgmap v1051: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:44.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:43 smithi115 ceph-mon[65453]: pgmap v1052: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:44.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:43 smithi027 ceph-mon[97244]: pgmap v1052: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:45.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:44 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2027.407999 secs 2021-11-03T17:49:45.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:44 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2027.407999 secs 2021-11-03T17:49:46.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:45 smithi115 ceph-mon[65453]: pgmap v1053: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:46.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:45 smithi027 ceph-mon[97244]: pgmap v1053: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:48.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:47 smithi115 ceph-mon[65453]: pgmap v1054: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:48.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:47 smithi027 ceph-mon[97244]: pgmap v1054: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:48.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.705+0000 7f60df351700 1 -- 172.21.15.27:0/167142669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60d80f77d0 msgr2=0x7f60d80f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:49:48.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.705+0000 7f60df351700 1 --2- 172.21.15.27:0/167142669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60d80f77d0 0x7f60d80f7c30 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f60c8004660 tx=0x7f60c8009b30).stop 2021-11-03T17:49:48.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.705+0000 7f60df351700 1 -- 172.21.15.27:0/167142669 shutdown_connections 2021-11-03T17:49:48.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.705+0000 7f60df351700 1 --2- 172.21.15.27:0/167142669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60d80f77d0 0x7f60d80f7c30 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:48.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.705+0000 7f60df351700 1 --2- 172.21.15.27:0/167142669 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60d80f6530 0x7f60d80f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:48.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.705+0000 7f60df351700 1 -- 172.21.15.27:0/167142669 >> 172.21.15.27:0/167142669 conn(0x7f60d80f1cd0 msgr2=0x7f60d80f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:49:48.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.705+0000 7f60df351700 1 -- 172.21.15.27:0/167142669 shutdown_connections 2021-11-03T17:49:48.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.705+0000 7f60df351700 1 -- 172.21.15.27:0/167142669 wait complete. 2021-11-03T17:49:48.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.706+0000 7f60df351700 1 Processor -- start 2021-11-03T17:49:48.704 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.706+0000 7f60df351700 1 -- start start 2021-11-03T17:49:48.704 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.706+0000 7f60df351700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60d80f6530 0x7f60d8106d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:49:48.704 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.706+0000 7f60df351700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60d80f77d0 0x7f60d8107270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:49:48.704 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.706+0000 7f60df351700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d8107870 con 0x7f60d80f6530 2021-11-03T17:49:48.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.706+0000 7f60df351700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60d8108080 con 0x7f60d80f77d0 2021-11-03T17:49:48.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.706+0000 7f60de34f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60d80f6530 0x7f60d8106d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:49:48.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.706+0000 7f60de34f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60d80f6530 0x7f60d8106d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59624/0 (socket says 172.21.15.27:59624) 2021-11-03T17:49:48.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.706+0000 7f60de34f700 1 -- 172.21.15.27:0/1612915168 learned_addr learned my addr 172.21.15.27:0/1612915168 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:49:48.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.706+0000 7f60ddb4e700 1 --2- 172.21.15.27:0/1612915168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60d80f77d0 0x7f60d8107270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:49:48.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.706+0000 7f60de34f700 1 -- 172.21.15.27:0/1612915168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60d80f77d0 msgr2=0x7f60d8107270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:49:48.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.706+0000 7f60de34f700 1 --2- 172.21.15.27:0/1612915168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60d80f77d0 0x7f60d8107270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:48.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.706+0000 7f60de34f700 1 -- 172.21.15.27:0/1612915168 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60c8005040 con 0x7f60d80f6530 2021-11-03T17:49:48.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.706+0000 7f60de34f700 1 --2- 172.21.15.27:0/1612915168 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60d80f6530 0x7f60d8106d80 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7f60d4002700 tx=0x7f60d4009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:49:48.707 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.707+0000 7f60ceffd700 1 -- 172.21.15.27:0/1612915168 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60d4009320 con 0x7f60d80f6530 2021-11-03T17:49:48.707 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.707+0000 7f60ceffd700 1 -- 172.21.15.27:0/1612915168 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f60d400f040 con 0x7f60d80f6530 2021-11-03T17:49:48.707 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.707+0000 7f60df351700 1 -- 172.21.15.27:0/1612915168 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60d8108300 con 0x7f60d80f6530 2021-11-03T17:49:48.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.707+0000 7f60ceffd700 1 -- 172.21.15.27:0/1612915168 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60d4014900 con 0x7f60d80f6530 2021-11-03T17:49:48.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.707+0000 7f60df351700 1 -- 172.21.15.27:0/1612915168 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60d81088c0 con 0x7f60d80f6530 2021-11-03T17:49:48.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.708+0000 7f60df351700 1 -- 172.21.15.27:0/1612915168 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60d8059100 con 0x7f60d80f6530 2021-11-03T17:49:48.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.708+0000 7f60ceffd700 1 -- 172.21.15.27:0/1612915168 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f60d4009480 con 0x7f60d80f6530 2021-11-03T17:49:48.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.708+0000 7f60ceffd700 1 --2- 172.21.15.27:0/1612915168 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f60c4064100 0x7f60c40665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:49:48.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.709+0000 7f60ceffd700 1 -- 172.21.15.27:0/1612915168 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f60d4080a10 con 0x7f60d80f6530 2021-11-03T17:49:48.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.709+0000 7f60ddb4e700 1 --2- 172.21.15.27:0/1612915168 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f60c4064100 0x7f60c40665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:49:48.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.709+0000 7f60ddb4e700 1 --2- 172.21.15.27:0/1612915168 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f60c4064100 0x7f60c40665c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f60c8004630 tx=0x7f60c8009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:49:48.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.713+0000 7f60ceffd700 1 -- 172.21.15.27:0/1612915168 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f60d4052d10 con 0x7f60d80f6530 2021-11-03T17:49:48.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.893+0000 7f60df351700 1 -- 172.21.15.27:0/1612915168 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f60d80fee60 con 0x7f60c4064100 2021-11-03T17:49:48.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.894+0000 7f60ceffd700 1 -- 172.21.15.27:0/1612915168 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f60d80fee60 con 0x7f60c4064100 2021-11-03T17:49:48.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.896+0000 7f60df351700 1 -- 172.21.15.27:0/1612915168 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f60c4064100 msgr2=0x7f60c40665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:49:48.891 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.896+0000 7f60df351700 1 --2- 172.21.15.27:0/1612915168 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f60c4064100 0x7f60c40665c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f60c8004630 tx=0x7f60c8009b30).stop 2021-11-03T17:49:48.891 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.896+0000 7f60df351700 1 -- 172.21.15.27:0/1612915168 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60d80f6530 msgr2=0x7f60d8106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:49:48.891 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.896+0000 7f60df351700 1 --2- 172.21.15.27:0/1612915168 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60d80f6530 0x7f60d8106d80 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7f60d4002700 tx=0x7f60d4009d80).stop 2021-11-03T17:49:48.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.896+0000 7f60df351700 1 -- 172.21.15.27:0/1612915168 shutdown_connections 2021-11-03T17:49:48.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.896+0000 7f60df351700 1 --2- 172.21.15.27:0/1612915168 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f60c4064100 0x7f60c40665c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:48.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.896+0000 7f60df351700 1 --2- 172.21.15.27:0/1612915168 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60d80f6530 0x7f60d8106d80 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:48.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.896+0000 7f60df351700 1 --2- 172.21.15.27:0/1612915168 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60d80f77d0 0x7f60d8107270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:48.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.896+0000 7f60df351700 1 -- 172.21.15.27:0/1612915168 >> 172.21.15.27:0/1612915168 conn(0x7f60d80f1cd0 msgr2=0x7f60d80fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:49:48.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.896+0000 7f60df351700 1 -- 172.21.15.27:0/1612915168 shutdown_connections 2021-11-03T17:49:48.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.896+0000 7f60df351700 1 -- 172.21.15.27:0/1612915168 wait complete. 2021-11-03T17:49:48.901 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:49:48.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.998+0000 7f27ecbf9700 1 -- 172.21.15.27:0/866013964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27e80f6ff0 msgr2=0x7f27e80f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:49:48.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.998+0000 7f27ecbf9700 1 --2- 172.21.15.27:0/866013964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27e80f6ff0 0x7f27e80f7a20 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f27d0004660 tx=0x7f27d0009b30).stop 2021-11-03T17:49:48.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.998+0000 7f27ecbf9700 1 -- 172.21.15.27:0/866013964 shutdown_connections 2021-11-03T17:49:48.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.998+0000 7f27ecbf9700 1 --2- 172.21.15.27:0/866013964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27e80f6ff0 0x7f27e80f7a20 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:48.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.998+0000 7f27ecbf9700 1 --2- 172.21.15.27:0/866013964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27e80f66e0 0x7f27e80f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:48.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.998+0000 7f27ecbf9700 1 -- 172.21.15.27:0/866013964 >> 172.21.15.27:0/866013964 conn(0x7f27e80f1f00 msgr2=0x7f27e80f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:49:48.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.998+0000 7f27ecbf9700 1 -- 172.21.15.27:0/866013964 shutdown_connections 2021-11-03T17:49:48.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.998+0000 7f27ecbf9700 1 -- 172.21.15.27:0/866013964 wait complete. 2021-11-03T17:49:48.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.999+0000 7f27ecbf9700 1 Processor -- start 2021-11-03T17:49:48.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.999+0000 7f27ecbf9700 1 -- start start 2021-11-03T17:49:48.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.999+0000 7f27ecbf9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27e80f66e0 0x7f27e806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:49:48.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.999+0000 7f27ecbf9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27e80f6ff0 0x7f27e8069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:49:48.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.999+0000 7f27ecbf9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e806a380 con 0x7f27e80f6ff0 2021-11-03T17:49:48.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.999+0000 7f27ecbf9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e806a4c0 con 0x7f27e80f66e0 2021-11-03T17:49:48.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.999+0000 7f27e5d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27e80f6ff0 0x7f27e8069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:49:48.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.999+0000 7f27e5d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27e80f6ff0 0x7f27e8069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59636/0 (socket says 172.21.15.27:59636) 2021-11-03T17:49:48.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.999+0000 7f27e5d9b700 1 -- 172.21.15.27:0/3206558181 learned_addr learned my addr 172.21.15.27:0/3206558181 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:49:48.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.999+0000 7f27e659c700 1 --2- 172.21.15.27:0/3206558181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27e80f66e0 0x7f27e806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:49:48.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.999+0000 7f27e5d9b700 1 -- 172.21.15.27:0/3206558181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27e80f66e0 msgr2=0x7f27e806b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:49:48.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.999+0000 7f27e5d9b700 1 --2- 172.21.15.27:0/3206558181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27e80f66e0 0x7f27e806b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:48.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.999+0000 7f27e5d9b700 1 -- 172.21.15.27:0/3206558181 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27d0005040 con 0x7f27e80f6ff0 2021-11-03T17:49:48.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:48.999+0000 7f27e5d9b700 1 --2- 172.21.15.27:0/3206558181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27e80f6ff0 0x7f27e8069e20 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f27d0000c00 tx=0x7f27d000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:49:48.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.000+0000 7f27deffd700 1 -- 172.21.15.27:0/3206558181 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27d0019070 con 0x7f27e80f6ff0 2021-11-03T17:49:49.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.000+0000 7f27deffd700 1 -- 172.21.15.27:0/3206558181 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f27d0007e70 con 0x7f27e80f6ff0 2021-11-03T17:49:49.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.000+0000 7f27ecbf9700 1 -- 172.21.15.27:0/3206558181 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e806a740 con 0x7f27e80f6ff0 2021-11-03T17:49:49.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.000+0000 7f27deffd700 1 -- 172.21.15.27:0/3206558181 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27d0014ce0 con 0x7f27e80f6ff0 2021-11-03T17:49:49.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.000+0000 7f27ecbf9700 1 -- 172.21.15.27:0/3206558181 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e806aca0 con 0x7f27e80f6ff0 2021-11-03T17:49:49.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.001+0000 7f27ecbf9700 1 -- 172.21.15.27:0/3206558181 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27e8045ad0 con 0x7f27e80f6ff0 2021-11-03T17:49:49.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.001+0000 7f27deffd700 1 -- 172.21.15.27:0/3206558181 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f27d0025070 con 0x7f27e80f6ff0 2021-11-03T17:49:49.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.001+0000 7f27deffd700 1 --2- 172.21.15.27:0/3206558181 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f27d40640b0 0x7f27d4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:49:49.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.002+0000 7f27deffd700 1 -- 172.21.15.27:0/3206558181 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f27d00817d0 con 0x7f27e80f6ff0 2021-11-03T17:49:49.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.002+0000 7f27e659c700 1 --2- 172.21.15.27:0/3206558181 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f27d40640b0 0x7f27d4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:49:49.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.002+0000 7f27e659c700 1 --2- 172.21.15.27:0/3206558181 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f27d40640b0 0x7f27d4066570 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f27e806b730 tx=0x7f27e000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:49:49.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.006+0000 7f27deffd700 1 -- 172.21.15.27:0/3206558181 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f27d0053ad0 con 0x7f27e80f6ff0 2021-11-03T17:49:49.171 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.177+0000 7f27ecbf9700 1 -- 172.21.15.27:0/3206558181 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f27e80ff040 con 0x7f27d40640b0 2021-11-03T17:49:49.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.192+0000 7f27deffd700 1 -- 172.21.15.27:0/3206558181 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f27e80ff040 con 0x7f27d40640b0 2021-11-03T17:49:49.186 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:49:49.186 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (41m) 60s ago 42m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:49:49.187 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (33m) 60s ago 42m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:49:49.187 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (32m) 10m ago 41m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:49:49.187 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (41m) 60s ago 42m 41.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:49:49.187 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (34m) 60s ago 37m 23.8M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:49:49.188 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (37m) 60s ago 37m 3844M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:49:49.188 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (33m) 10m ago 37m 20.4M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:49:49.188 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (35m) 10m ago 37m 22.1M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:49:49.188 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (35m) 60s ago 43m 535M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:49:49.189 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (34m) 10m ago 41m 419M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:49:49.189 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (34m) 60s ago 44m 198M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:49:49.190 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (33m) 10m ago 41m 82.5M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:49:49.190 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (42m) 60s ago 42m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:49:49.190 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (41m) 10m ago 41m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:49:49.190 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (32m) 60s ago 41m 331M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:49:49.191 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (32m) 60s ago 40m 361M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:49:49.191 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (31m) 60s ago 40m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:49:49.191 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (31m) 10m ago 40m 354M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:49:49.191 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (30m) 10m ago 39m 285M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:49:49.192 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (30m) 10m ago 39m 287M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:49:49.192 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (41m) 60s ago 42m 52.9M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:49:49.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.193+0000 7f27ecbf9700 1 -- 172.21.15.27:0/3206558181 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f27d40640b0 msgr2=0x7f27d4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:49:49.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.193+0000 7f27ecbf9700 1 --2- 172.21.15.27:0/3206558181 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f27d40640b0 0x7f27d4066570 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f27e806b730 tx=0x7f27e000b040).stop 2021-11-03T17:49:49.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.193+0000 7f27ecbf9700 1 -- 172.21.15.27:0/3206558181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27e80f6ff0 msgr2=0x7f27e8069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:49:49.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.193+0000 7f27ecbf9700 1 --2- 172.21.15.27:0/3206558181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27e80f6ff0 0x7f27e8069e20 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f27d0000c00 tx=0x7f27d000b040).stop 2021-11-03T17:49:49.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.193+0000 7f27ecbf9700 1 -- 172.21.15.27:0/3206558181 shutdown_connections 2021-11-03T17:49:49.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.193+0000 7f27ecbf9700 1 --2- 172.21.15.27:0/3206558181 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f27d40640b0 0x7f27d4066570 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:49.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.193+0000 7f27ecbf9700 1 --2- 172.21.15.27:0/3206558181 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27e80f66e0 0x7f27e806b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:49.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.193+0000 7f27ecbf9700 1 --2- 172.21.15.27:0/3206558181 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27e80f6ff0 0x7f27e8069e20 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:49.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.193+0000 7f27ecbf9700 1 -- 172.21.15.27:0/3206558181 >> 172.21.15.27:0/3206558181 conn(0x7f27e80f1f00 msgr2=0x7f27e80fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:49:49.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.194+0000 7f27ecbf9700 1 -- 172.21.15.27:0/3206558181 shutdown_connections 2021-11-03T17:49:49.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.194+0000 7f27ecbf9700 1 -- 172.21.15.27:0/3206558181 wait complete. 2021-11-03T17:49:49.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:49:49.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:49:49.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.293+0000 7fc1d410f700 1 -- 172.21.15.27:0/1729676995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1cc0603f0 msgr2=0x7fc1cc0f7130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:49:49.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.293+0000 7fc1d410f700 1 --2- 172.21.15.27:0/1729676995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1cc0603f0 0x7fc1cc0f7130 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7fc1c40099e0 tx=0x7fc1c4004d10).stop 2021-11-03T17:49:49.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.293+0000 7fc1d410f700 1 -- 172.21.15.27:0/1729676995 shutdown_connections 2021-11-03T17:49:49.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.293+0000 7fc1d410f700 1 --2- 172.21.15.27:0/1729676995 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1cc0f7620 0x7fc1cc0f9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:49.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.293+0000 7fc1d410f700 1 --2- 172.21.15.27:0/1729676995 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1cc0603f0 0x7fc1cc0f7130 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:49.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.293+0000 7fc1d410f700 1 -- 172.21.15.27:0/1729676995 >> 172.21.15.27:0/1729676995 conn(0x7fc1cc0f0dc0 msgr2=0x7fc1cc0f31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:49:49.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.293+0000 7fc1d410f700 1 -- 172.21.15.27:0/1729676995 shutdown_connections 2021-11-03T17:49:49.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.293+0000 7fc1d410f700 1 -- 172.21.15.27:0/1729676995 wait complete. 2021-11-03T17:49:49.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.294+0000 7fc1d410f700 1 Processor -- start 2021-11-03T17:49:49.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.294+0000 7fc1d410f700 1 -- start start 2021-11-03T17:49:49.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.294+0000 7fc1d410f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1cc0603f0 0x7fc1cc10d660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:49:49.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.294+0000 7fc1d410f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1cc0f7620 0x7fc1cc10db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:49:49.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.294+0000 7fc1d410f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc10e100 con 0x7fc1cc0f7620 2021-11-03T17:49:49.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.294+0000 7fc1d410f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc10e240 con 0x7fc1cc0603f0 2021-11-03T17:49:49.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.294+0000 7fc1d16aa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1cc0f7620 0x7fc1cc10db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:49:49.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.294+0000 7fc1d16aa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1cc0f7620 0x7fc1cc10db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59644/0 (socket says 172.21.15.27:59644) 2021-11-03T17:49:49.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.294+0000 7fc1d16aa700 1 -- 172.21.15.27:0/1168786566 learned_addr learned my addr 172.21.15.27:0/1168786566 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:49:49.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.294+0000 7fc1d1eab700 1 --2- 172.21.15.27:0/1168786566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1cc0603f0 0x7fc1cc10d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:49:49.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.294+0000 7fc1d16aa700 1 -- 172.21.15.27:0/1168786566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1cc0603f0 msgr2=0x7fc1cc10d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:49:49.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.294+0000 7fc1d16aa700 1 --2- 172.21.15.27:0/1168786566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1cc0603f0 0x7fc1cc10d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:49.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.294+0000 7fc1d16aa700 1 -- 172.21.15.27:0/1168786566 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c4009640 con 0x7fc1cc0f7620 2021-11-03T17:49:49.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.294+0000 7fc1d16aa700 1 --2- 172.21.15.27:0/1168786566 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1cc0f7620 0x7fc1cc10db50 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7fc1bc00c530 tx=0x7fc1bc007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:49:49.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.295+0000 7fc1c27fc700 1 -- 172.21.15.27:0/1168786566 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1bc00f650 con 0x7fc1cc0f7620 2021-11-03T17:49:49.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.295+0000 7fc1c27fc700 1 -- 172.21.15.27:0/1168786566 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fc1bc00fc60 con 0x7fc1cc0f7620 2021-11-03T17:49:49.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.295+0000 7fc1d410f700 1 -- 172.21.15.27:0/1168786566 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1cc10ebe0 con 0x7fc1cc0f7620 2021-11-03T17:49:49.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.295+0000 7fc1c27fc700 1 -- 172.21.15.27:0/1168786566 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc1bc012c20 con 0x7fc1cc0f7620 2021-11-03T17:49:49.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.295+0000 7fc1d410f700 1 -- 172.21.15.27:0/1168786566 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1cc10f120 con 0x7fc1cc0f7620 2021-11-03T17:49:49.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.295+0000 7fc1d410f700 1 -- 172.21.15.27:0/1168786566 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1cc045ad0 con 0x7fc1cc0f7620 2021-11-03T17:49:49.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.296+0000 7fc1c27fc700 1 -- 172.21.15.27:0/1168786566 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fc1bc010070 con 0x7fc1cc0f7620 2021-11-03T17:49:49.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.296+0000 7fc1c27fc700 1 --2- 172.21.15.27:0/1168786566 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc1b80640b0 0x7fc1b8066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:49:49.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.296+0000 7fc1c27fc700 1 -- 172.21.15.27:0/1168786566 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fc1bc0806a0 con 0x7fc1cc0f7620 2021-11-03T17:49:49.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.296+0000 7fc1d1eab700 1 --2- 172.21.15.27:0/1168786566 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc1b80640b0 0x7fc1b8066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:49:49.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.297+0000 7fc1d1eab700 1 --2- 172.21.15.27:0/1168786566 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc1b80640b0 0x7fc1b8066570 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7fc1c400b730 tx=0x7fc1c4004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:49:49.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.301+0000 7fc1c27fc700 1 -- 172.21.15.27:0/1168786566 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fc1bc0528b0 con 0x7fc1cc0f7620 2021-11-03T17:49:49.526 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.532+0000 7fc1d410f700 1 -- 172.21.15.27:0/1168786566 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc1cc027a00 con 0x7fc1cc0f7620 2021-11-03T17:49:49.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.533+0000 7fc1c27fc700 1 -- 172.21.15.27:0/1168786566 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fc1bc0524a0 con 0x7fc1cc0f7620 2021-11-03T17:49:49.528 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:49:49.528 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:49:49.528 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:49:49.528 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:49:49.528 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:49:49.529 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:49:49.529 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:49:49.529 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:49:49.529 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:49:49.530 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:49:49.530 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:49:49.530 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:49:49.530 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:49:49.531 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:49:49.531 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:49:49.531 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:49:49.531 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:49:49.531 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:49:49.532 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.534+0000 7fc1d410f700 1 -- 172.21.15.27:0/1168786566 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc1b80640b0 msgr2=0x7fc1b8066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:49:49.533 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.534+0000 7fc1d410f700 1 --2- 172.21.15.27:0/1168786566 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc1b80640b0 0x7fc1b8066570 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7fc1c400b730 tx=0x7fc1c4004d10).stop 2021-11-03T17:49:49.533 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.534+0000 7fc1d410f700 1 -- 172.21.15.27:0/1168786566 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1cc0f7620 msgr2=0x7fc1cc10db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:49:49.533 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.534+0000 7fc1d410f700 1 --2- 172.21.15.27:0/1168786566 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1cc0f7620 0x7fc1cc10db50 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7fc1bc00c530 tx=0x7fc1bc007e20).stop 2021-11-03T17:49:49.533 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.535+0000 7fc1d410f700 1 -- 172.21.15.27:0/1168786566 shutdown_connections 2021-11-03T17:49:49.533 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.535+0000 7fc1d410f700 1 --2- 172.21.15.27:0/1168786566 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc1b80640b0 0x7fc1b8066570 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:49.534 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.535+0000 7fc1d410f700 1 --2- 172.21.15.27:0/1168786566 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1cc0603f0 0x7fc1cc10d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:49.534 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.535+0000 7fc1d410f700 1 --2- 172.21.15.27:0/1168786566 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1cc0f7620 0x7fc1cc10db50 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:49.534 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.535+0000 7fc1d410f700 1 -- 172.21.15.27:0/1168786566 >> 172.21.15.27:0/1168786566 conn(0x7fc1cc0f0dc0 msgr2=0x7fc1cc0f31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:49:49.534 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.535+0000 7fc1d410f700 1 -- 172.21.15.27:0/1168786566 shutdown_connections 2021-11-03T17:49:49.535 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.535+0000 7fc1d410f700 1 -- 172.21.15.27:0/1168786566 wait complete. 2021-11-03T17:49:49.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.634+0000 7fe160b5c700 1 -- 172.21.15.27:0/3345177871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe15c0f6ff0 msgr2=0x7fe15c0f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:49:49.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.635+0000 7fe160b5c700 1 --2- 172.21.15.27:0/3345177871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe15c0f6ff0 0x7fe15c0f7a20 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7fe150004660 tx=0x7fe150009b30).stop 2021-11-03T17:49:49.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.635+0000 7fe160b5c700 1 -- 172.21.15.27:0/3345177871 shutdown_connections 2021-11-03T17:49:49.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.635+0000 7fe160b5c700 1 --2- 172.21.15.27:0/3345177871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe15c0f6ff0 0x7fe15c0f7a20 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:49.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.635+0000 7fe160b5c700 1 --2- 172.21.15.27:0/3345177871 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe15c0f66e0 0x7fe15c0f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:49:49.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.635+0000 7fe160b5c700 1 -- 172.21.15.27:0/3345177871 >> 172.21.15.27:0/3345177871 conn(0x7fe15c0f1f00 msgr2=0x7fe15c0f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:49:49.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.635+0000 7fe160b5c700 1 -- 172.21.15.27:0/3345177871 shutdown_connections 2021-11-03T17:49:49.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.635+0000 7fe160b5c700 1 -- 172.21.15.27:0/3345177871 wait complete. 2021-11-03T17:49:49.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.635+0000 7fe160b5c700 1 Processor -- start 2021-11-03T17:49:49.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.635+0000 7fe160b5c700 1 -- start start 2021-11-03T17:49:49.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.635+0000 7fe160b5c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe15c0f66e0 0x7fe15c106fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:49:49.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.635+0000 7fe160b5c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe15c0f6ff0 0x7fe15c107490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:49:49.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.635+0000 7fe160b5c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15c107ad0 con 0x7fe15c0f6ff0 2021-11-03T17:49:49.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.635+0000 7fe160b5c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15c108380 con 0x7fe15c0f66e0 2021-11-03T17:49:49.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.636+0000 7fe159d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe15c0f6ff0 0x7fe15c107490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:49:49.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.636+0000 7fe159d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe15c0f6ff0 0x7fe15c107490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59654/0 (socket says 172.21.15.27:59654) 2021-11-03T17:49:49.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.636+0000 7fe159d9b700 1 -- 172.21.15.27:0/548942179 learned_addr learned my addr 172.21.15.27:0/548942179 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:49:49.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.636+0000 7fe15a59c700 1 --2- 172.21.15.27:0/548942179 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe15c0f66e0 0x7fe15c106fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:49:49.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.636+0000 7fe159d9b700 1 -- 172.21.15.27:0/548942179 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe15c0f66e0 msgr2=0x7fe15c106fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:49:49.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.636+0000 7fe159d9b700 1 --2- 172.21.15.27:0/548942179 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe15c0f66e0 0x7fe15c106fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:49.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.636+0000 7fe159d9b700 1 -- 172.21.15.27:0/548942179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe150005040 con 0x7fe15c0f6ff0 2021-11-03T17:49:49.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.636+0000 7fe159d9b700 1 --2- 172.21.15.27:0/548942179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe15c0f6ff0 0x7fe15c107490 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7fe150009990 tx=0x7fe150009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:49:49.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.636+0000 7fe14affd700 1 -- 172.21.15.27:0/548942179 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe150018070 con 0x7fe15c0f6ff0 2021-11-03T17:49:49.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.636+0000 7fe14affd700 1 -- 172.21.15.27:0/548942179 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fe150007ba0 con 0x7fe15c0f6ff0 2021-11-03T17:49:49.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.636+0000 7fe160b5c700 1 -- 172.21.15.27:0/548942179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe15c108600 con 0x7fe15c0f6ff0 2021-11-03T17:49:49.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.636+0000 7fe14affd700 1 -- 172.21.15.27:0/548942179 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe15001c620 con 0x7fe15c0f6ff0 2021-11-03T17:49:49.637 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.636+0000 7fe160b5c700 1 -- 172.21.15.27:0/548942179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe15c108b60 con 0x7fe15c0f6ff0 2021-11-03T17:49:49.638 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.637+0000 7fe160b5c700 1 -- 172.21.15.27:0/548942179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe15c0f5b70 con 0x7fe15c0f6ff0 2021-11-03T17:49:49.638 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.638+0000 7fe14affd700 1 -- 172.21.15.27:0/548942179 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fe15001c890 con 0x7fe15c0f6ff0 2021-11-03T17:49:49.639 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.638+0000 7fe14affd700 1 --2- 172.21.15.27:0/548942179 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe1440640b0 0x7fe144066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:49:49.639 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.638+0000 7fe14affd700 1 -- 172.21.15.27:0/548942179 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fe1500825a0 con 0x7fe15c0f6ff0 2021-11-03T17:49:49.639 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.638+0000 7fe15a59c700 1 --2- 172.21.15.27:0/548942179 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe1440640b0 0x7fe144066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:49:49.639 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.639+0000 7fe15a59c700 1 --2- 172.21.15.27:0/548942179 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe1440640b0 0x7fe144066570 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7fe154009c70 tx=0x7fe154009220).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:49:49.639 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.642+0000 7fe14affd700 1 -- 172.21.15.27:0/548942179 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fe150054960 con 0x7fe15c0f6ff0 2021-11-03T17:49:49.838 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.844+0000 7fe160b5c700 1 -- 172.21.15.27:0/548942179 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe15c045ad0 con 0x7fe15c0f6ff0 2021-11-03T17:49:49.839 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.844+0000 7fe14affd700 1 -- 172.21.15.27:0/548942179 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7fe150054550 con 0x7fe15c0f6ff0 2021-11-03T17:49:49.839 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:49:49.840 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:49:49.840 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:49:49.840 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:49:49.840 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:49:49.841 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:49:49.841 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:49:49.841 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:49:49.841 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:49:49.841 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:49:49.842 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:49:49.842 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:49:49.842 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:49:49.842 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:49:49.843 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:49:49.843 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:49:49.843 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:49:49.843 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:49:49.844 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:49:49.844 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:49:49.844 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:49:49.844 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:49:49.844 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:49:49.845 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:49:49.845 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:49:49.845 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:49:49.845 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:49:49.846 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:49:49.846 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:49:49.846 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:49:49.846 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:49:49.847 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:49:49.847 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:49:49.847 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:49:49.847 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:49:49.847 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:49:49.848 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:49:49.848 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:49:49.848 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:49:49.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.846+0000 7fe160b5c700 1 -- 172.21.15.27:0/548942179 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe1440640b0 msgr2=0x7fe144066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:49:49.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.846+0000 7fe160b5c700 1 --2- 172.21.15.27:0/548942179 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe1440640b0 0x7fe144066570 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7fe154009c70 tx=0x7fe154009220).stop 2021-11-03T17:49:49.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.846+0000 7fe160b5c700 1 -- 172.21.15.27:0/548942179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe15c0f6ff0 msgr2=0x7fe15c107490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:49:49.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.846+0000 7fe160b5c700 1 --2- 172.21.15.27:0/548942179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe15c0f6ff0 0x7fe15c107490 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7fe150009990 tx=0x7fe150009b30).stop 2021-11-03T17:49:49.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.846+0000 7fe160b5c700 1 -- 172.21.15.27:0/548942179 shutdown_connections 2021-11-03T17:49:49.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.846+0000 7fe160b5c700 1 --2- 172.21.15.27:0/548942179 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe1440640b0 0x7fe144066570 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:49.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.846+0000 7fe160b5c700 1 --2- 172.21.15.27:0/548942179 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe15c0f66e0 0x7fe15c106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:49.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.846+0000 7fe160b5c700 1 --2- 172.21.15.27:0/548942179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe15c0f6ff0 0x7fe15c107490 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:49:49.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.846+0000 7fe160b5c700 1 -- 172.21.15.27:0/548942179 >> 172.21.15.27:0/548942179 conn(0x7fe15c0f1f00 msgr2=0x7fe15c0fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:49:49.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.846+0000 7fe160b5c700 1 -- 172.21.15.27:0/548942179 shutdown_connections 2021-11-03T17:49:49.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:49:49.846+0000 7fe160b5c700 1 -- 172.21.15.27:0/548942179 wait complete. 2021-11-03T17:49:49.852 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:49:50.201 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:49 smithi115 ceph-mon[65453]: pgmap v1055: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:50.202 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:49 smithi115 ceph-mon[65453]: poll took 20058.195 ms: 1 events 2021-11-03T17:49:50.202 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:49 smithi115 ceph-mon[65453]: from='client.35192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:49:50.202 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:49 smithi115 ceph-mon[65453]: from='client.35196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:49:50.202 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:49 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2032.408066 secs 2021-11-03T17:49:50.203 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:49 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1168786566' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:50.203 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:49 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/548942179' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:49:50.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:49 smithi027 ceph-mon[97244]: pgmap v1055: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:50.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:49 smithi027 ceph-mon[97244]: poll took 20058.195 ms: 1 events 2021-11-03T17:49:50.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:49 smithi027 ceph-mon[97244]: from='client.35192 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:49:50.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:49 smithi027 ceph-mon[97244]: from='client.35196 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:49:50.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:49 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2032.408066 secs 2021-11-03T17:49:50.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:49 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1168786566' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:50.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:49 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/548942179' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:49:52.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:51 smithi115 ceph-mon[65453]: pgmap v1056: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:52.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:49:52.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:49:52.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:49:52.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:52.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:52.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:52.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:52.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:51 smithi027 ceph-mon[97244]: pgmap v1056: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:52.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:49:52.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:49:52.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:49:52.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:52.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:52.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:52.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:49:53.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:52 smithi115 ceph-mon[65453]: poll took 2975.701 ms: 1 events 2021-11-03T17:49:53.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:52 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:49:53.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:52 smithi027 ceph-mon[97244]: poll took 2975.701 ms: 1 events 2021-11-03T17:49:53.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:52 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:49:54.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:53 smithi115 ceph-mon[65453]: pgmap v1057: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:54.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:53 smithi027 ceph-mon[97244]: pgmap v1057: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:55.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:54 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2037.408193 secs 2021-11-03T17:49:55.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:54 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2037.408193 secs 2021-11-03T17:49:56.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:55 smithi115 ceph-mon[65453]: pgmap v1058: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:56.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:49:56.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:49:56.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:55 smithi027 ceph-mon[97244]: pgmap v1058: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:56.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:49:56.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:49:58.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:57 smithi115 ceph-mon[65453]: pgmap v1059: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:49:58.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:57 smithi027 ceph-mon[97244]: pgmap v1059: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:00.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:59 smithi115 ceph-mon[65453]: pgmap v1060: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:00.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:49:59 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2042.408319 secs 2021-11-03T17:50:00.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:59 smithi027 ceph-mon[97244]: pgmap v1060: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:00.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:49:59 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2042.408319 secs 2021-11-03T17:50:01.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:00 smithi115 ceph-mon[65453]: overall HEALTH_WARN 1 filesystem is degraded; 1 filesystem has a failed mds daemon; 1 MDSs report slow requests; 3 daemons have recently crashed 2021-11-03T17:50:01.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:00 smithi027 ceph-mon[97244]: overall HEALTH_WARN 1 filesystem is degraded; 1 filesystem has a failed mds daemon; 1 MDSs report slow requests; 3 daemons have recently crashed 2021-11-03T17:50:02.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:01 smithi115 ceph-mon[65453]: pgmap v1061: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:02.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:50:02.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:50:02.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:50:02.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:02.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:02.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:02.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:02.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:01 smithi027 ceph-mon[97244]: pgmap v1061: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:02.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:50:02.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:50:02.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:50:02.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:02.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:02.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:02.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:03.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:02 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:50:03.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:02 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:50:04.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:03 smithi115 ceph-mon[65453]: pgmap v1062: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:04.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:03 smithi027 ceph-mon[97244]: pgmap v1062: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:05.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:04 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2047.408444 secs 2021-11-03T17:50:05.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:04 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2047.408444 secs 2021-11-03T17:50:06.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:05 smithi115 ceph-mon[65453]: pgmap v1063: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:06.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:05 smithi027 ceph-mon[97244]: pgmap v1063: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:08.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:07 smithi115 ceph-mon[65453]: pgmap v1064: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:08.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:07 smithi027 ceph-mon[97244]: pgmap v1064: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:10.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:09 smithi115 ceph-mon[65453]: pgmap v1065: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:10.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:09 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2052.408566 secs 2021-11-03T17:50:10.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:09 smithi027 ceph-mon[97244]: pgmap v1065: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:10.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:09 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2052.408566 secs 2021-11-03T17:50:12.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:11 smithi115 ceph-mon[65453]: pgmap v1066: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:12.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:50:12.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:50:12.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:50:12.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:12.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:12.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:12.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:12.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:11 smithi027 ceph-mon[97244]: pgmap v1066: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:12.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:50:12.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:50:12.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:50:12.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:12.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:12.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:12.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:13.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:12 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:50:13.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:12 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:50:14.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:13 smithi115 ceph-mon[65453]: pgmap v1067: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:14.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:13 smithi027 ceph-mon[97244]: pgmap v1067: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:15.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:14 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2057.408707 secs 2021-11-03T17:50:15.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:14 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2057.408707 secs 2021-11-03T17:50:16.018 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:15 smithi027 ceph-mon[97244]: pgmap v1068: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:16.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:15 smithi115 ceph-mon[65453]: pgmap v1068: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:18.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:17 smithi115 ceph-mon[65453]: pgmap v1069: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:18.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:17 smithi027 ceph-mon[97244]: pgmap v1069: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:19.957 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:19.961+0000 7f3692d69700 1 -- 172.21.15.27:0/3073257484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f368c0f6870 msgr2=0x7f368c0f6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:50:19.958 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:19.961+0000 7f3692d69700 1 --2- 172.21.15.27:0/3073257484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f368c0f6870 0x7f368c0f6cd0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f3678004660 tx=0x7f3678009b30).stop 2021-11-03T17:50:19.958 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:19.961+0000 7f3692d69700 1 -- 172.21.15.27:0/3073257484 shutdown_connections 2021-11-03T17:50:19.958 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:19.961+0000 7f3692d69700 1 --2- 172.21.15.27:0/3073257484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f368c0f6870 0x7f368c0f6cd0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:19.958 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:19.961+0000 7f3692d69700 1 --2- 172.21.15.27:0/3073257484 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f368c0f55d0 0x7f368c0f59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:19.959 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:19.961+0000 7f3692d69700 1 -- 172.21.15.27:0/3073257484 >> 172.21.15.27:0/3073257484 conn(0x7f368c0f0d30 msgr2=0x7f368c0f3150 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:50:19.959 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:19.961+0000 7f3692d69700 1 -- 172.21.15.27:0/3073257484 shutdown_connections 2021-11-03T17:50:19.959 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:19.962+0000 7f3692d69700 1 -- 172.21.15.27:0/3073257484 wait complete. 2021-11-03T17:50:19.959 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:19.962+0000 7f3692d69700 1 Processor -- start 2021-11-03T17:50:19.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:19.962+0000 7f3692d69700 1 -- start start 2021-11-03T17:50:19.963 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:19.962+0000 7f3692d69700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f368c0f55d0 0x7f368c0f9f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:50:19.963 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:19.962+0000 7f3692d69700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f368c0f6870 0x7f368c0f83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:50:19.963 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:19.962+0000 7f3692d69700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368c0f8930 con 0x7f368c0f6870 2021-11-03T17:50:19.963 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:19.962+0000 7f3692d69700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368c0f8a70 con 0x7f368c0f55d0 2021-11-03T17:50:19.964 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:19.963+0000 7f368bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f368c0f6870 0x7f368c0f83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:50:19.964 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:19.963+0000 7f368bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f368c0f6870 0x7f368c0f83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59672/0 (socket says 172.21.15.27:59672) 2021-11-03T17:50:19.964 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:19.963+0000 7f368bfff700 1 -- 172.21.15.27:0/2832215924 learned_addr learned my addr 172.21.15.27:0/2832215924 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:50:19.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:19.963+0000 7f3690b05700 1 --2- 172.21.15.27:0/2832215924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f368c0f55d0 0x7f368c0f9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:50:19.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:19.963+0000 7f368bfff700 1 -- 172.21.15.27:0/2832215924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f368c0f55d0 msgr2=0x7f368c0f9f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:50:19.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:19.963+0000 7f368bfff700 1 --2- 172.21.15.27:0/2832215924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f368c0f55d0 0x7f368c0f9f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:19.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:19.963+0000 7f368bfff700 1 -- 172.21.15.27:0/2832215924 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3678005040 con 0x7f368c0f6870 2021-11-03T17:50:19.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:19.963+0000 7f368bfff700 1 --2- 172.21.15.27:0/2832215924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f368c0f6870 0x7f368c0f83d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7f3678009990 tx=0x7f3678009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:50:19.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:19.963+0000 7f36897fa700 1 -- 172.21.15.27:0/2832215924 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3678018070 con 0x7f368c0f6870 2021-11-03T17:50:19.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:19.963+0000 7f36897fa700 1 -- 172.21.15.27:0/2832215924 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f3678007ba0 con 0x7f368c0f6870 2021-11-03T17:50:19.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:19.963+0000 7f3692d69700 1 -- 172.21.15.27:0/2832215924 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f368c0f8cf0 con 0x7f368c0f6870 2021-11-03T17:50:19.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:19.963+0000 7f36897fa700 1 -- 172.21.15.27:0/2832215924 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f367801c3c0 con 0x7f368c0f6870 2021-11-03T17:50:19.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:19.963+0000 7f3692d69700 1 -- 172.21.15.27:0/2832215924 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f368c0f9250 con 0x7f368c0f6870 2021-11-03T17:50:19.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:19.964+0000 7f3692d69700 1 -- 172.21.15.27:0/2832215924 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f368c045ad0 con 0x7f368c0f6870 2021-11-03T17:50:19.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:19.965+0000 7f36897fa700 1 -- 172.21.15.27:0/2832215924 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f367801d420 con 0x7f368c0f6870 2021-11-03T17:50:19.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:19.965+0000 7f36897fa700 1 --2- 172.21.15.27:0/2832215924 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f367c0640b0 0x7f367c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:50:19.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:19.965+0000 7f36897fa700 1 -- 172.21.15.27:0/2832215924 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f3678082470 con 0x7f368c0f6870 2021-11-03T17:50:19.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:19.965+0000 7f3690b05700 1 --2- 172.21.15.27:0/2832215924 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f367c0640b0 0x7f367c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:50:19.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:19.966+0000 7f3690b05700 1 --2- 172.21.15.27:0/2832215924 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f367c0640b0 0x7f367c066570 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f368c0f9ce0 tx=0x7f368000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:50:19.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:19.970+0000 7f36897fa700 1 -- 172.21.15.27:0/2832215924 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f3678054830 con 0x7f368c0f6870 2021-11-03T17:50:20.147 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.151+0000 7f3692d69700 1 -- 172.21.15.27:0/2832215924 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f368c05fe30 con 0x7f367c0640b0 2021-11-03T17:50:20.148 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.153+0000 7f36897fa700 1 -- 172.21.15.27:0/2832215924 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f368c05fe30 con 0x7f367c0640b0 2021-11-03T17:50:20.150 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.154+0000 7f3692d69700 1 -- 172.21.15.27:0/2832215924 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f367c0640b0 msgr2=0x7f367c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:50:20.150 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.154+0000 7f3692d69700 1 --2- 172.21.15.27:0/2832215924 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f367c0640b0 0x7f367c066570 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7f368c0f9ce0 tx=0x7f368000b040).stop 2021-11-03T17:50:20.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.154+0000 7f3692d69700 1 -- 172.21.15.27:0/2832215924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f368c0f6870 msgr2=0x7f368c0f83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:50:20.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.154+0000 7f3692d69700 1 --2- 172.21.15.27:0/2832215924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f368c0f6870 0x7f368c0f83d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7f3678009990 tx=0x7f3678009b30).stop 2021-11-03T17:50:20.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.155+0000 7f3692d69700 1 -- 172.21.15.27:0/2832215924 shutdown_connections 2021-11-03T17:50:20.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.155+0000 7f3692d69700 1 --2- 172.21.15.27:0/2832215924 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f367c0640b0 0x7f367c066570 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:20.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.155+0000 7f3692d69700 1 --2- 172.21.15.27:0/2832215924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f368c0f55d0 0x7f368c0f9f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:20.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.155+0000 7f3692d69700 1 --2- 172.21.15.27:0/2832215924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f368c0f6870 0x7f368c0f83d0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:20.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.155+0000 7f3692d69700 1 -- 172.21.15.27:0/2832215924 >> 172.21.15.27:0/2832215924 conn(0x7f368c0f0d30 msgr2=0x7f368c05f720 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:50:20.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.155+0000 7f3692d69700 1 -- 172.21.15.27:0/2832215924 shutdown_connections 2021-11-03T17:50:20.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.155+0000 7f3692d69700 1 -- 172.21.15.27:0/2832215924 wait complete. 2021-11-03T17:50:20.160 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:50:20.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:19 smithi115 ceph-mon[65453]: pgmap v1070: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:20.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:19 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2062.408826 secs 2021-11-03T17:50:20.252 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.255+0000 7f25a4558700 1 -- 172.21.15.27:0/1245066731 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f259c0f55d0 msgr2=0x7f259c0f59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:50:20.252 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.255+0000 7f25a4558700 1 --2- 172.21.15.27:0/1245066731 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f259c0f55d0 0x7f259c0f59f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7f258c004660 tx=0x7f258c009b30).stop 2021-11-03T17:50:20.253 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.255+0000 7f25a4558700 1 -- 172.21.15.27:0/1245066731 shutdown_connections 2021-11-03T17:50:20.253 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.255+0000 7f25a4558700 1 --2- 172.21.15.27:0/1245066731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f259c0f6870 0x7f259c0f6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:20.253 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.255+0000 7f25a4558700 1 --2- 172.21.15.27:0/1245066731 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f259c0f55d0 0x7f259c0f59f0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:20.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.255+0000 7f25a4558700 1 -- 172.21.15.27:0/1245066731 >> 172.21.15.27:0/1245066731 conn(0x7f259c0f0d30 msgr2=0x7f259c0f3150 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:50:20.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.255+0000 7f25a4558700 1 -- 172.21.15.27:0/1245066731 shutdown_connections 2021-11-03T17:50:20.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.255+0000 7f25a4558700 1 -- 172.21.15.27:0/1245066731 wait complete. 2021-11-03T17:50:20.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.256+0000 7f25a4558700 1 Processor -- start 2021-11-03T17:50:20.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.256+0000 7f25a4558700 1 -- start start 2021-11-03T17:50:20.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.256+0000 7f25a4558700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f259c0f55d0 0x7f259c109c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:50:20.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.256+0000 7f25a4558700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f259c0f6870 0x7f259c10c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:50:20.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.256+0000 7f25a4558700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259c0604e0 con 0x7f259c0f6870 2021-11-03T17:50:20.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.256+0000 7f25a4558700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f259c060620 con 0x7f259c0f55d0 2021-11-03T17:50:20.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.256+0000 7f25a1af3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f259c0f6870 0x7f259c10c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:50:20.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.256+0000 7f25a1af3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f259c0f6870 0x7f259c10c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59682/0 (socket says 172.21.15.27:59682) 2021-11-03T17:50:20.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.256+0000 7f25a1af3700 1 -- 172.21.15.27:0/732287764 learned_addr learned my addr 172.21.15.27:0/732287764 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:50:20.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.256+0000 7f25a22f4700 1 --2- 172.21.15.27:0/732287764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f259c0f55d0 0x7f259c109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:50:20.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.256+0000 7f25a1af3700 1 -- 172.21.15.27:0/732287764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f259c0f55d0 msgr2=0x7f259c109c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:50:20.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.256+0000 7f25a1af3700 1 --2- 172.21.15.27:0/732287764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f259c0f55d0 0x7f259c109c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:20.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.256+0000 7f25a1af3700 1 -- 172.21.15.27:0/732287764 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f258c005040 con 0x7f259c0f6870 2021-11-03T17:50:20.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.256+0000 7f25a1af3700 1 --2- 172.21.15.27:0/732287764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f259c0f6870 0x7f259c10c170 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f25980026d0 tx=0x7f2598009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:50:20.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.257+0000 7f2592ffd700 1 -- 172.21.15.27:0/732287764 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2598009320 con 0x7f259c0f6870 2021-11-03T17:50:20.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.257+0000 7f2592ffd700 1 -- 172.21.15.27:0/732287764 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f259800f040 con 0x7f259c0f6870 2021-11-03T17:50:20.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.257+0000 7f25a4558700 1 -- 172.21.15.27:0/732287764 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f259c10c720 con 0x7f259c0f6870 2021-11-03T17:50:20.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.257+0000 7f2592ffd700 1 -- 172.21.15.27:0/732287764 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25980147e0 con 0x7f259c0f6870 2021-11-03T17:50:20.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.257+0000 7f25a4558700 1 -- 172.21.15.27:0/732287764 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f259c10cce0 con 0x7f259c0f6870 2021-11-03T17:50:20.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.258+0000 7f25a4558700 1 -- 172.21.15.27:0/732287764 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f259c045ad0 con 0x7f259c0f6870 2021-11-03T17:50:20.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.258+0000 7f2592ffd700 1 -- 172.21.15.27:0/732287764 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f2598009480 con 0x7f259c0f6870 2021-11-03T17:50:20.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.258+0000 7f2592ffd700 1 --2- 172.21.15.27:0/732287764 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f25880640b0 0x7f2588066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:50:20.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.259+0000 7f2592ffd700 1 -- 172.21.15.27:0/732287764 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f2598080e50 con 0x7f259c0f6870 2021-11-03T17:50:20.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.259+0000 7f25a22f4700 1 --2- 172.21.15.27:0/732287764 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f25880640b0 0x7f2588066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:50:20.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.259+0000 7f25a22f4700 1 --2- 172.21.15.27:0/732287764 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f25880640b0 0x7f2588066570 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f258c009da0 tx=0x7f258c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:50:20.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.263+0000 7f2592ffd700 1 -- 172.21.15.27:0/732287764 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f2598053210 con 0x7f259c0f6870 2021-11-03T17:50:20.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:19 smithi027 ceph-mon[97244]: pgmap v1070: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:20.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:19 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2062.408826 secs 2021-11-03T17:50:20.431 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.435+0000 7f25a4558700 1 -- 172.21.15.27:0/732287764 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f259c057f10 con 0x7f25880640b0 2021-11-03T17:50:20.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.450+0000 7f2592ffd700 1 -- 172.21.15.27:0/732287764 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f259c057f10 con 0x7f25880640b0 2021-11-03T17:50:20.447 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:50:20.447 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (41m) 92s ago 43m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:50:20.447 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (33m) 92s ago 43m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:50:20.448 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (33m) 28s ago 42m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:50:20.448 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (41m) 92s ago 42m 41.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:50:20.448 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (35m) 92s ago 38m 23.8M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:50:20.448 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (38m) 92s ago 38m 3844M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:50:20.449 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (34m) 28s ago 38m 23.2M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:50:20.449 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (35m) 28s ago 38m 24.9M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:50:20.449 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (35m) 92s ago 44m 535M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:50:20.449 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (34m) 28s ago 42m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:50:20.449 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (34m) 92s ago 44m 198M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:50:20.450 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (34m) 28s ago 42m 95.6M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:50:20.450 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (42m) 92s ago 42m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:50:20.450 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (42m) 28s ago 42m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:50:20.450 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (33m) 92s ago 41m 331M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:50:20.451 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (32m) 92s ago 41m 361M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:50:20.451 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (32m) 92s ago 40m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:50:20.451 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (31m) 28s ago 40m 379M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:50:20.451 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (31m) 28s ago 40m 304M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:50:20.451 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (30m) 28s ago 39m 307M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:50:20.452 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (41m) 92s ago 42m 52.9M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:50:20.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.452+0000 7f25a4558700 1 -- 172.21.15.27:0/732287764 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f25880640b0 msgr2=0x7f2588066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:50:20.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.452+0000 7f25a4558700 1 --2- 172.21.15.27:0/732287764 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f25880640b0 0x7f2588066570 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 rx=0x7f258c009da0 tx=0x7f258c009b30).stop 2021-11-03T17:50:20.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.452+0000 7f25a4558700 1 -- 172.21.15.27:0/732287764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f259c0f6870 msgr2=0x7f259c10c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:50:20.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.452+0000 7f25a4558700 1 --2- 172.21.15.27:0/732287764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f259c0f6870 0x7f259c10c170 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f25980026d0 tx=0x7f2598009de0).stop 2021-11-03T17:50:20.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.452+0000 7f25a4558700 1 -- 172.21.15.27:0/732287764 shutdown_connections 2021-11-03T17:50:20.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.452+0000 7f25a4558700 1 --2- 172.21.15.27:0/732287764 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f25880640b0 0x7f2588066570 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:20.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.452+0000 7f25a4558700 1 --2- 172.21.15.27:0/732287764 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f259c0f55d0 0x7f259c109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:20.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.452+0000 7f25a4558700 1 --2- 172.21.15.27:0/732287764 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f259c0f6870 0x7f259c10c170 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:20.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.452+0000 7f25a4558700 1 -- 172.21.15.27:0/732287764 >> 172.21.15.27:0/732287764 conn(0x7f259c0f0d30 msgr2=0x7f259c0fa2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:50:20.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.452+0000 7f25a4558700 1 -- 172.21.15.27:0/732287764 shutdown_connections 2021-11-03T17:50:20.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.452+0000 7f25a4558700 1 -- 172.21.15.27:0/732287764 wait complete. 2021-11-03T17:50:20.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.551+0000 7ff18359e700 1 -- 172.21.15.27:0/3327249295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1840f4800 msgr2=0x7ff1840f4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:50:20.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.551+0000 7ff18359e700 1 --2- 172.21.15.27:0/3327249295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1840f4800 0x7ff1840f4c20 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7ff174004660 tx=0x7ff174009b30).stop 2021-11-03T17:50:20.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.551+0000 7ff18359e700 1 -- 172.21.15.27:0/3327249295 shutdown_connections 2021-11-03T17:50:20.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.551+0000 7ff18359e700 1 --2- 172.21.15.27:0/3327249295 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1840f51a0 0x7ff1840f7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:50:20.550 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.551+0000 7ff18359e700 1 --2- 172.21.15.27:0/3327249295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1840f4800 0x7ff1840f4c20 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:20.550 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.551+0000 7ff18359e700 1 -- 172.21.15.27:0/3327249295 >> 172.21.15.27:0/3327249295 conn(0x7ff1840f0320 msgr2=0x7ff1840f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:50:20.550 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.551+0000 7ff18359e700 1 -- 172.21.15.27:0/3327249295 shutdown_connections 2021-11-03T17:50:20.550 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.551+0000 7ff18359e700 1 -- 172.21.15.27:0/3327249295 wait complete. 2021-11-03T17:50:20.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.552+0000 7ff18359e700 1 Processor -- start 2021-11-03T17:50:20.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.552+0000 7ff18359e700 1 -- start start 2021-11-03T17:50:20.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.552+0000 7ff18359e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1840f4800 0x7ff1840f73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:50:20.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.552+0000 7ff18359e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1840f51a0 0x7ff1840f5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:50:20.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.552+0000 7ff18359e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1840f5e20 con 0x7ff1840f51a0 2021-11-03T17:50:20.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.552+0000 7ff18359e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1840f5f60 con 0x7ff1840f4800 2021-11-03T17:50:20.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.552+0000 7ff181d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1840f51a0 0x7ff1840f5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:50:20.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.552+0000 7ff181d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1840f51a0 0x7ff1840f5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59692/0 (socket says 172.21.15.27:59692) 2021-11-03T17:50:20.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.552+0000 7ff181d9b700 1 -- 172.21.15.27:0/3469697403 learned_addr learned my addr 172.21.15.27:0/3469697403 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:50:20.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.553+0000 7ff18259c700 1 --2- 172.21.15.27:0/3469697403 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1840f4800 0x7ff1840f73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:50:20.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.553+0000 7ff181d9b700 1 -- 172.21.15.27:0/3469697403 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1840f4800 msgr2=0x7ff1840f73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:50:20.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.553+0000 7ff181d9b700 1 --2- 172.21.15.27:0/3469697403 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1840f4800 0x7ff1840f73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:20.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.553+0000 7ff181d9b700 1 -- 172.21.15.27:0/3469697403 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff174005040 con 0x7ff1840f51a0 2021-11-03T17:50:20.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.553+0000 7ff181d9b700 1 --2- 172.21.15.27:0/3469697403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1840f51a0 0x7ff1840f5830 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7ff1780093f0 tx=0x7ff178006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:50:20.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.553+0000 7ff172ffd700 1 -- 172.21.15.27:0/3469697403 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff178009ea0 con 0x7ff1840f51a0 2021-11-03T17:50:20.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.553+0000 7ff172ffd700 1 -- 172.21.15.27:0/3469697403 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7ff178013af0 con 0x7ff1840f51a0 2021-11-03T17:50:20.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.553+0000 7ff18359e700 1 -- 172.21.15.27:0/3469697403 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1840f6240 con 0x7ff1840f51a0 2021-11-03T17:50:20.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.553+0000 7ff172ffd700 1 -- 172.21.15.27:0/3469697403 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff17800db10 con 0x7ff1840f51a0 2021-11-03T17:50:20.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.553+0000 7ff18359e700 1 -- 172.21.15.27:0/3469697403 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff18410ee10 con 0x7ff1840f51a0 2021-11-03T17:50:20.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.554+0000 7ff18359e700 1 -- 172.21.15.27:0/3469697403 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff184045ad0 con 0x7ff1840f51a0 2021-11-03T17:50:20.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.554+0000 7ff172ffd700 1 -- 172.21.15.27:0/3469697403 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7ff17801a060 con 0x7ff1840f51a0 2021-11-03T17:50:20.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.555+0000 7ff172ffd700 1 --2- 172.21.15.27:0/3469697403 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff16c064100 0x7ff16c0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:50:20.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.555+0000 7ff172ffd700 1 -- 172.21.15.27:0/3469697403 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7ff178080880 con 0x7ff1840f51a0 2021-11-03T17:50:20.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.555+0000 7ff18259c700 1 --2- 172.21.15.27:0/3469697403 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff16c064100 0x7ff16c0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:50:20.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.555+0000 7ff18259c700 1 --2- 172.21.15.27:0/3469697403 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff16c064100 0x7ff16c0665c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7ff174009da0 tx=0x7ff174009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:50:20.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.559+0000 7ff172ffd700 1 -- 172.21.15.27:0/3469697403 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7ff178052a90 con 0x7ff1840f51a0 2021-11-03T17:50:20.787 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.792+0000 7ff18359e700 1 -- 172.21.15.27:0/3469697403 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff184027a00 con 0x7ff1840f51a0 2021-11-03T17:50:20.788 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.792+0000 7ff172ffd700 1 -- 172.21.15.27:0/3469697403 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7ff1780182e0 con 0x7ff1840f51a0 2021-11-03T17:50:20.788 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:50:20.789 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:50:20.789 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:50:20.789 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:50:20.789 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:50:20.790 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:50:20.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:50:20.790 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:50:20.790 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:50:20.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:50:20.791 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:50:20.791 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:50:20.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:50:20.792 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:50:20.792 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:50:20.792 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:50:20.792 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:50:20.793 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:50:20.793 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.794+0000 7ff18359e700 1 -- 172.21.15.27:0/3469697403 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff16c064100 msgr2=0x7ff16c0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:50:20.794 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.794+0000 7ff18359e700 1 --2- 172.21.15.27:0/3469697403 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff16c064100 0x7ff16c0665c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7ff174009da0 tx=0x7ff174009b30).stop 2021-11-03T17:50:20.794 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.794+0000 7ff18359e700 1 -- 172.21.15.27:0/3469697403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1840f51a0 msgr2=0x7ff1840f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:50:20.794 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.794+0000 7ff18359e700 1 --2- 172.21.15.27:0/3469697403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1840f51a0 0x7ff1840f5830 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7ff1780093f0 tx=0x7ff178006ec0).stop 2021-11-03T17:50:20.794 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.794+0000 7ff18359e700 1 -- 172.21.15.27:0/3469697403 shutdown_connections 2021-11-03T17:50:20.795 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.794+0000 7ff18359e700 1 --2- 172.21.15.27:0/3469697403 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff16c064100 0x7ff16c0665c0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:20.795 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.794+0000 7ff18359e700 1 --2- 172.21.15.27:0/3469697403 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1840f4800 0x7ff1840f73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:20.795 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.794+0000 7ff18359e700 1 --2- 172.21.15.27:0/3469697403 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1840f51a0 0x7ff1840f5830 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:20.795 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.794+0000 7ff18359e700 1 -- 172.21.15.27:0/3469697403 >> 172.21.15.27:0/3469697403 conn(0x7ff1840f0320 msgr2=0x7ff1840fb370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:50:20.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.794+0000 7ff18359e700 1 -- 172.21.15.27:0/3469697403 shutdown_connections 2021-11-03T17:50:20.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.795+0000 7ff18359e700 1 -- 172.21.15.27:0/3469697403 wait complete. 2021-11-03T17:50:20.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.895+0000 7f40613a8700 1 -- 172.21.15.27:0/631435338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f405c0f6980 msgr2=0x7f405c0f6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:50:20.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.895+0000 7f40613a8700 1 --2- 172.21.15.27:0/631435338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f405c0f6980 0x7f405c0f6da0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7f404c004660 tx=0x7f404c009b30).stop 2021-11-03T17:50:20.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.895+0000 7f40613a8700 1 -- 172.21.15.27:0/631435338 shutdown_connections 2021-11-03T17:50:20.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.895+0000 7f40613a8700 1 --2- 172.21.15.27:0/631435338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f405c0f5680 0x7f405c0f5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:50:20.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.895+0000 7f40613a8700 1 --2- 172.21.15.27:0/631435338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f405c0f6980 0x7f405c0f6da0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:20.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.895+0000 7f40613a8700 1 -- 172.21.15.27:0/631435338 >> 172.21.15.27:0/631435338 conn(0x7f405c0f0290 msgr2=0x7f405c0f26b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:50:20.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.895+0000 7f40613a8700 1 -- 172.21.15.27:0/631435338 shutdown_connections 2021-11-03T17:50:20.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.895+0000 7f40613a8700 1 -- 172.21.15.27:0/631435338 wait complete. 2021-11-03T17:50:20.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.896+0000 7f40613a8700 1 Processor -- start 2021-11-03T17:50:20.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.896+0000 7f40613a8700 1 -- start start 2021-11-03T17:50:20.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.896+0000 7f40613a8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f405c0f5680 0x7f405c106d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:50:20.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.896+0000 7f40613a8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f405c0f6980 0x7f405c107280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:50:20.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.896+0000 7f40613a8700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405c107910 con 0x7f405c0f5680 2021-11-03T17:50:20.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.896+0000 7f40613a8700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f405c108090 con 0x7f405c0f6980 2021-11-03T17:50:20.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.896+0000 7f405bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f405c0f5680 0x7f405c106d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:50:20.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.896+0000 7f405bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f405c0f5680 0x7f405c106d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59700/0 (socket says 172.21.15.27:59700) 2021-11-03T17:50:20.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.896+0000 7f405bfff700 1 -- 172.21.15.27:0/1033342891 learned_addr learned my addr 172.21.15.27:0/1033342891 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:50:20.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.896+0000 7f405bfff700 1 -- 172.21.15.27:0/1033342891 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f405c0f6980 msgr2=0x7f405c107280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:50:20.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.896+0000 7f405b7fe700 1 --2- 172.21.15.27:0/1033342891 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f405c0f6980 0x7f405c107280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:50:20.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.896+0000 7f405bfff700 1 --2- 172.21.15.27:0/1033342891 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f405c0f6980 0x7f405c107280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:20.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.896+0000 7f405bfff700 1 -- 172.21.15.27:0/1033342891 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f404c005040 con 0x7f405c0f5680 2021-11-03T17:50:20.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.896+0000 7f405b7fe700 1 --2- 172.21.15.27:0/1033342891 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f405c0f6980 0x7f405c107280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:50:20.900 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.897+0000 7f405bfff700 1 --2- 172.21.15.27:0/1033342891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f405c0f5680 0x7f405c106d90 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7f404c004f60 tx=0x7f404c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:50:20.900 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.897+0000 7f4058ff9700 1 -- 172.21.15.27:0/1033342891 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f404c018070 con 0x7f405c0f5680 2021-11-03T17:50:20.900 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.897+0000 7f40613a8700 1 -- 172.21.15.27:0/1033342891 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f405c1082b0 con 0x7f405c0f5680 2021-11-03T17:50:20.901 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.897+0000 7f4058ff9700 1 -- 172.21.15.27:0/1033342891 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f404c007ba0 con 0x7f405c0f5680 2021-11-03T17:50:20.901 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.897+0000 7f4058ff9700 1 -- 172.21.15.27:0/1033342891 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f404c01c620 con 0x7f405c0f5680 2021-11-03T17:50:20.902 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.897+0000 7f40613a8700 1 -- 172.21.15.27:0/1033342891 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f405c108810 con 0x7f405c0f5680 2021-11-03T17:50:20.902 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.898+0000 7f40613a8700 1 -- 172.21.15.27:0/1033342891 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f405c059100 con 0x7f405c0f5680 2021-11-03T17:50:20.904 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.898+0000 7f4058ff9700 1 -- 172.21.15.27:0/1033342891 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f404c01c8e0 con 0x7f405c0f5680 2021-11-03T17:50:20.905 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.899+0000 7f4058ff9700 1 --2- 172.21.15.27:0/1033342891 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4044064100 0x7f40440665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:50:20.905 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.899+0000 7f4058ff9700 1 -- 172.21.15.27:0/1033342891 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f404c082740 con 0x7f405c0f5680 2021-11-03T17:50:20.905 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.899+0000 7f405b7fe700 1 --2- 172.21.15.27:0/1033342891 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4044064100 0x7f40440665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:50:20.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.899+0000 7f405b7fe700 1 --2- 172.21.15.27:0/1033342891 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4044064100 0x7f40440665c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f405c0f6750 tx=0x7f4050008040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:50:20.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:20.904+0000 7f4058ff9700 1 -- 172.21.15.27:0/1033342891 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f404c054a40 con 0x7f405c0f5680 2021-11-03T17:50:21.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:21.109+0000 7f40613a8700 1 -- 172.21.15.27:0/1033342891 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f405c10d2d0 con 0x7f405c0f5680 2021-11-03T17:50:21.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:21.110+0000 7f4058ff9700 1 -- 172.21.15.27:0/1033342891 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f404c054630 con 0x7f405c0f5680 2021-11-03T17:50:21.106 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:50:21.106 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:50:21.107 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:50:21.107 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:50:21.107 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:50:21.107 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:50:21.108 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:50:21.108 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:50:21.108 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:50:21.108 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:50:21.108 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:50:21.109 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:50:21.109 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:50:21.109 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:50:21.109 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:50:21.110 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:50:21.110 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:50:21.110 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:50:21.110 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:50:21.110 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:50:21.111 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:50:21.111 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:50:21.111 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:50:21.111 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:50:21.112 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:50:21.112 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:50:21.112 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:50:21.112 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:50:21.112 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:50:21.113 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:50:21.113 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:50:21.113 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:50:21.114 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:50:21.114 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:50:21.114 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:50:21.114 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:50:21.115 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:50:21.115 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:50:21.115 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:50:21.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:21.112+0000 7f40613a8700 1 -- 172.21.15.27:0/1033342891 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4044064100 msgr2=0x7f40440665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:50:21.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:21.112+0000 7f40613a8700 1 --2- 172.21.15.27:0/1033342891 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4044064100 0x7f40440665c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f405c0f6750 tx=0x7f4050008040).stop 2021-11-03T17:50:21.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:21.112+0000 7f40613a8700 1 -- 172.21.15.27:0/1033342891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f405c0f5680 msgr2=0x7f405c106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:50:21.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:21.112+0000 7f40613a8700 1 --2- 172.21.15.27:0/1033342891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f405c0f5680 0x7f405c106d90 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7f404c004f60 tx=0x7f404c009b30).stop 2021-11-03T17:50:21.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:21.112+0000 7f40613a8700 1 -- 172.21.15.27:0/1033342891 shutdown_connections 2021-11-03T17:50:21.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:21.112+0000 7f40613a8700 1 --2- 172.21.15.27:0/1033342891 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4044064100 0x7f40440665c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:21.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:21.112+0000 7f40613a8700 1 --2- 172.21.15.27:0/1033342891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f405c0f5680 0x7f405c106d90 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:21.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:21.112+0000 7f40613a8700 1 --2- 172.21.15.27:0/1033342891 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f405c0f6980 0x7f405c107280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:21.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:21.112+0000 7f40613a8700 1 -- 172.21.15.27:0/1033342891 >> 172.21.15.27:0/1033342891 conn(0x7f405c0f0290 msgr2=0x7f405c0f4570 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:50:21.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:21.112+0000 7f40613a8700 1 -- 172.21.15.27:0/1033342891 shutdown_connections 2021-11-03T17:50:21.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:21.112+0000 7f40613a8700 1 -- 172.21.15.27:0/1033342891 wait complete. 2021-11-03T17:50:21.119 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:50:21.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:20 smithi115 ceph-mon[65453]: from='client.35208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:50:21.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:20 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3469697403' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:21.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:20 smithi027 ceph-mon[97244]: from='client.35208 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:50:21.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:20 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3469697403' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:22.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:21 smithi115 ceph-mon[65453]: pgmap v1071: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:22.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:21 smithi115 ceph-mon[65453]: from='client.35212 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:50:22.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:21 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1033342891' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:50:22.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:50:22.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:50:22.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:50:22.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:22.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:22.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:22.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:22.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:21 smithi027 ceph-mon[97244]: pgmap v1071: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:22.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:21 smithi027 ceph-mon[97244]: from='client.35212 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:50:22.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:21 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1033342891' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:50:22.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:50:22.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:50:22.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:50:22.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:22.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:22.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:22.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:23.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:22 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:50:23.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:22 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:50:24.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:23 smithi115 ceph-mon[65453]: pgmap v1072: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:24.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:23 smithi027 ceph-mon[97244]: pgmap v1072: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:25.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:24 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2067.408966 secs 2021-11-03T17:50:25.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:24 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2067.408966 secs 2021-11-03T17:50:26.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:25 smithi027 ceph-mon[97244]: pgmap v1073: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:26.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:25 smithi115 ceph-mon[65453]: pgmap v1073: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:28.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:28 smithi027 ceph-mon[97244]: pgmap v1074: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:28.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:28 smithi115 ceph-mon[65453]: pgmap v1074: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:30.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:30 smithi027 ceph-mon[97244]: pgmap v1075: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:30.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2072.409122 secs 2021-11-03T17:50:30.483 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:30 smithi115 ceph-mon[65453]: pgmap v1075: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:30.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2072.409122 secs 2021-11-03T17:50:32.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:32 smithi115 ceph-mon[65453]: pgmap v1076: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:32.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:50:32.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:32 smithi027 ceph-mon[97244]: pgmap v1076: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:32.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:50:33.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:33 smithi115 ceph-mon[65453]: poll took 40126.024 ms: 1 events 2021-11-03T17:50:33.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:50:33.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:50:33.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:33.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:33.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:33.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:33.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:33 smithi027 ceph-mon[97244]: poll took 40126.024 ms: 1 events 2021-11-03T17:50:33.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:50:33.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:50:33.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:33.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:33.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:33.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:34.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:34 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:50:34.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:34 smithi115 ceph-mon[65453]: pgmap v1077: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:34.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:34 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:50:34.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:34 smithi027 ceph-mon[97244]: pgmap v1077: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:35.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:35 smithi115 ceph-mon[65453]: pgmap v1078: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:35.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2077.409265 secs 2021-11-03T17:50:35.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:35 smithi027 ceph-mon[97244]: pgmap v1078: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:35.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2077.409265 secs 2021-11-03T17:50:37.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:37 smithi115 ceph-mon[65453]: pgmap v1079: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:37.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:37 smithi027 ceph-mon[97244]: pgmap v1079: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:39.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:39 smithi115 ceph-mon[65453]: pgmap v1080: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:39.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:39 smithi027 ceph-mon[97244]: pgmap v1080: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:40.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2082.409384 secs 2021-11-03T17:50:40.768 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2082.409384 secs 2021-11-03T17:50:42.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:41 smithi115 ceph-mon[65453]: pgmap v1081: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:42.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:41 smithi027 ceph-mon[97244]: pgmap v1081: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:43.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:50:43.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:50:43.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:50:43.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:43.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:43.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:43.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:43.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:50:43.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:50:43.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:50:43.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:43.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:43.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:43.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:44.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:43 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:50:44.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:43 smithi115 ceph-mon[65453]: pgmap v1082: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:44.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:43 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:50:44.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:43 smithi027 ceph-mon[97244]: pgmap v1082: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:45.233 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:44 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2087.409524 secs 2021-11-03T17:50:45.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:44 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2087.409524 secs 2021-11-03T17:50:46.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:45 smithi115 ceph-mon[65453]: pgmap v1083: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:46.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:45 smithi027 ceph-mon[97244]: pgmap v1083: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:48.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:47 smithi115 ceph-mon[65453]: pgmap v1084: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:48.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:47 smithi027 ceph-mon[97244]: pgmap v1084: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:50.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:49 smithi115 ceph-mon[65453]: pgmap v1085: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:50.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:49 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2092.409671 secs 2021-11-03T17:50:50.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:49 smithi027 ceph-mon[97244]: pgmap v1085: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:50.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:49 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2092.409671 secs 2021-11-03T17:50:51.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.227+0000 7fce88fbe700 1 -- 172.21.15.27:0/735192749 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce840f6860 msgr2=0x7fce840f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:50:51.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.227+0000 7fce88fbe700 1 --2- 172.21.15.27:0/735192749 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce840f6860 0x7fce840f6cc0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7fce6c004660 tx=0x7fce6c009b30).stop 2021-11-03T17:50:51.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.227+0000 7fce88fbe700 1 -- 172.21.15.27:0/735192749 shutdown_connections 2021-11-03T17:50:51.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.227+0000 7fce88fbe700 1 --2- 172.21.15.27:0/735192749 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce840f6860 0x7fce840f6cc0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:51.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.227+0000 7fce88fbe700 1 --2- 172.21.15.27:0/735192749 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce840f55c0 0x7fce840f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:51.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.227+0000 7fce88fbe700 1 -- 172.21.15.27:0/735192749 >> 172.21.15.27:0/735192749 conn(0x7fce840f0d40 msgr2=0x7fce840f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:50:51.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.227+0000 7fce88fbe700 1 -- 172.21.15.27:0/735192749 shutdown_connections 2021-11-03T17:50:51.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.227+0000 7fce88fbe700 1 -- 172.21.15.27:0/735192749 wait complete. 2021-11-03T17:50:51.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.228+0000 7fce88fbe700 1 Processor -- start 2021-11-03T17:50:51.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.228+0000 7fce88fbe700 1 -- start start 2021-11-03T17:50:51.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.228+0000 7fce88fbe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce840f55c0 0x7fce84109c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:50:51.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.228+0000 7fce88fbe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce840f6860 0x7fce8410c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:50:51.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.228+0000 7fce88fbe700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce840603e0 con 0x7fce840f6860 2021-11-03T17:50:51.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.228+0000 7fce88fbe700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce84060520 con 0x7fce840f55c0 2021-11-03T17:50:51.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.228+0000 7fce81d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce840f6860 0x7fce8410c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:50:51.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.228+0000 7fce81d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce840f6860 0x7fce8410c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59720/0 (socket says 172.21.15.27:59720) 2021-11-03T17:50:51.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.228+0000 7fce81d9b700 1 -- 172.21.15.27:0/1984734996 learned_addr learned my addr 172.21.15.27:0/1984734996 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:50:51.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.228+0000 7fce8259c700 1 --2- 172.21.15.27:0/1984734996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce840f55c0 0x7fce84109c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:50:51.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.229+0000 7fce81d9b700 1 -- 172.21.15.27:0/1984734996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce840f55c0 msgr2=0x7fce84109c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:50:51.230 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.229+0000 7fce81d9b700 1 --2- 172.21.15.27:0/1984734996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce840f55c0 0x7fce84109c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:51.230 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.229+0000 7fce81d9b700 1 -- 172.21.15.27:0/1984734996 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce6c005040 con 0x7fce840f6860 2021-11-03T17:50:51.230 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.229+0000 7fce81d9b700 1 --2- 172.21.15.27:0/1984734996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce840f6860 0x7fce8410c150 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7fce6c009990 tx=0x7fce6c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:50:51.230 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.229+0000 7fce7affd700 1 -- 172.21.15.27:0/1984734996 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce6c018070 con 0x7fce840f6860 2021-11-03T17:50:51.231 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.229+0000 7fce7affd700 1 -- 172.21.15.27:0/1984734996 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fce6c007ba0 con 0x7fce840f6860 2021-11-03T17:50:51.231 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.229+0000 7fce88fbe700 1 -- 172.21.15.27:0/1984734996 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce8410c6a0 con 0x7fce840f6860 2021-11-03T17:50:51.231 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.229+0000 7fce7affd700 1 -- 172.21.15.27:0/1984734996 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce6c01c3c0 con 0x7fce840f6860 2021-11-03T17:50:51.231 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.229+0000 7fce88fbe700 1 -- 172.21.15.27:0/1984734996 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce8410cc00 con 0x7fce840f6860 2021-11-03T17:50:51.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.230+0000 7fce88fbe700 1 -- 172.21.15.27:0/1984734996 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce84045ad0 con 0x7fce840f6860 2021-11-03T17:50:51.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.230+0000 7fce7affd700 1 -- 172.21.15.27:0/1984734996 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fce6c01d420 con 0x7fce840f6860 2021-11-03T17:50:51.234 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.231+0000 7fce7affd700 1 --2- 172.21.15.27:0/1984734996 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fce700640b0 0x7fce70066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:50:51.234 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.231+0000 7fce7affd700 1 -- 172.21.15.27:0/1984734996 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fce6c082470 con 0x7fce840f6860 2021-11-03T17:50:51.234 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.231+0000 7fce8259c700 1 --2- 172.21.15.27:0/1984734996 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fce700640b0 0x7fce70066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:50:51.235 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.231+0000 7fce8259c700 1 --2- 172.21.15.27:0/1984734996 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fce700640b0 0x7fce70066570 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7fce84060060 tx=0x7fce7c00b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:50:51.235 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.235+0000 7fce7affd700 1 -- 172.21.15.27:0/1984734996 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fce6c054830 con 0x7fce840f6860 2021-11-03T17:50:51.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.416+0000 7fce88fbe700 1 -- 172.21.15.27:0/1984734996 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fce84057f10 con 0x7fce700640b0 2021-11-03T17:50:51.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.417+0000 7fce7affd700 1 -- 172.21.15.27:0/1984734996 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fce84057f10 con 0x7fce700640b0 2021-11-03T17:50:51.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.419+0000 7fce88fbe700 1 -- 172.21.15.27:0/1984734996 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fce700640b0 msgr2=0x7fce70066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:50:51.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.419+0000 7fce88fbe700 1 --2- 172.21.15.27:0/1984734996 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fce700640b0 0x7fce70066570 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 rx=0x7fce84060060 tx=0x7fce7c00b040).stop 2021-11-03T17:50:51.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.419+0000 7fce88fbe700 1 -- 172.21.15.27:0/1984734996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce840f6860 msgr2=0x7fce8410c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:50:51.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.419+0000 7fce88fbe700 1 --2- 172.21.15.27:0/1984734996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce840f6860 0x7fce8410c150 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 rx=0x7fce6c009990 tx=0x7fce6c009b30).stop 2021-11-03T17:50:51.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.419+0000 7fce88fbe700 1 -- 172.21.15.27:0/1984734996 shutdown_connections 2021-11-03T17:50:51.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.419+0000 7fce88fbe700 1 --2- 172.21.15.27:0/1984734996 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fce700640b0 0x7fce70066570 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:51.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.419+0000 7fce88fbe700 1 --2- 172.21.15.27:0/1984734996 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce840f55c0 0x7fce84109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:51.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.419+0000 7fce88fbe700 1 --2- 172.21.15.27:0/1984734996 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce840f6860 0x7fce8410c150 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:51.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.419+0000 7fce88fbe700 1 -- 172.21.15.27:0/1984734996 >> 172.21.15.27:0/1984734996 conn(0x7fce840f0d40 msgr2=0x7fce840fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:50:51.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.419+0000 7fce88fbe700 1 -- 172.21.15.27:0/1984734996 shutdown_connections 2021-11-03T17:50:51.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.419+0000 7fce88fbe700 1 -- 172.21.15.27:0/1984734996 wait complete. 2021-11-03T17:50:51.424 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:50:51.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.519+0000 7f6c78f61700 1 -- 172.21.15.27:0/2940614056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c740f6860 msgr2=0x7f6c740f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:50:51.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.519+0000 7f6c78f61700 1 --2- 172.21.15.27:0/2940614056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c740f6860 0x7f6c740f6cc0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f6c5c004660 tx=0x7f6c5c009b30).stop 2021-11-03T17:50:51.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.519+0000 7f6c78f61700 1 -- 172.21.15.27:0/2940614056 shutdown_connections 2021-11-03T17:50:51.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.519+0000 7f6c78f61700 1 --2- 172.21.15.27:0/2940614056 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c740f6860 0x7f6c740f6cc0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:51.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.519+0000 7f6c78f61700 1 --2- 172.21.15.27:0/2940614056 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c740f55c0 0x7f6c740f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:51.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.519+0000 7f6c78f61700 1 -- 172.21.15.27:0/2940614056 >> 172.21.15.27:0/2940614056 conn(0x7f6c740f0d40 msgr2=0x7f6c740f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:50:51.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.519+0000 7f6c78f61700 1 -- 172.21.15.27:0/2940614056 shutdown_connections 2021-11-03T17:50:51.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.519+0000 7f6c78f61700 1 -- 172.21.15.27:0/2940614056 wait complete. 2021-11-03T17:50:51.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.519+0000 7f6c78f61700 1 Processor -- start 2021-11-03T17:50:51.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.520+0000 7f6c78f61700 1 -- start start 2021-11-03T17:50:51.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.520+0000 7f6c78f61700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c740f55c0 0x7f6c74109c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:50:51.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.520+0000 7f6c78f61700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c740f6860 0x7f6c7410c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:50:51.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.520+0000 7f6c78f61700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c740603c0 con 0x7f6c740f6860 2021-11-03T17:50:51.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.520+0000 7f6c78f61700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c74060500 con 0x7f6c740f55c0 2021-11-03T17:50:51.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.520+0000 7f6c71d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c740f6860 0x7f6c7410c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:50:51.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.520+0000 7f6c71d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c740f6860 0x7f6c7410c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59730/0 (socket says 172.21.15.27:59730) 2021-11-03T17:50:51.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.520+0000 7f6c71d9b700 1 -- 172.21.15.27:0/312326436 learned_addr learned my addr 172.21.15.27:0/312326436 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:50:51.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.520+0000 7f6c7259c700 1 --2- 172.21.15.27:0/312326436 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c740f55c0 0x7f6c74109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:50:51.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.520+0000 7f6c71d9b700 1 -- 172.21.15.27:0/312326436 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c740f55c0 msgr2=0x7f6c74109c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:50:51.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.520+0000 7f6c71d9b700 1 --2- 172.21.15.27:0/312326436 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c740f55c0 0x7f6c74109c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:51.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.520+0000 7f6c71d9b700 1 -- 172.21.15.27:0/312326436 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c5c005040 con 0x7f6c740f6860 2021-11-03T17:50:51.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.520+0000 7f6c71d9b700 1 --2- 172.21.15.27:0/312326436 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c740f6860 0x7f6c7410c170 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7f6c5c009830 tx=0x7f6c5c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:50:51.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.520+0000 7f6c6affd700 1 -- 172.21.15.27:0/312326436 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c5c018070 con 0x7f6c740f6860 2021-11-03T17:50:51.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.520+0000 7f6c6affd700 1 -- 172.21.15.27:0/312326436 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f6c5c007ba0 con 0x7f6c740f6860 2021-11-03T17:50:51.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.520+0000 7f6c78f61700 1 -- 172.21.15.27:0/312326436 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c7410c6c0 con 0x7f6c740f6860 2021-11-03T17:50:51.524 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.520+0000 7f6c6affd700 1 -- 172.21.15.27:0/312326436 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c5c01c620 con 0x7f6c740f6860 2021-11-03T17:50:51.524 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.521+0000 7f6c78f61700 1 -- 172.21.15.27:0/312326436 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c7410cbf0 con 0x7f6c740f6860 2021-11-03T17:50:51.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.522+0000 7f6c78f61700 1 -- 172.21.15.27:0/312326436 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c740f49b0 con 0x7f6c740f6860 2021-11-03T17:50:51.526 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.522+0000 7f6c6affd700 1 -- 172.21.15.27:0/312326436 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f6c5c01c850 con 0x7f6c740f6860 2021-11-03T17:50:51.526 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.522+0000 7f6c6affd700 1 --2- 172.21.15.27:0/312326436 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6c60064090 0x7f6c60066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:50:51.526 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.522+0000 7f6c6affd700 1 -- 172.21.15.27:0/312326436 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f6c5c082490 con 0x7f6c740f6860 2021-11-03T17:50:51.526 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.522+0000 7f6c7259c700 1 --2- 172.21.15.27:0/312326436 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6c60064090 0x7f6c60066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:50:51.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.523+0000 7f6c7259c700 1 --2- 172.21.15.27:0/312326436 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6c60064090 0x7f6c60066550 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f6c74060040 tx=0x7f6c6c004d00).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:50:51.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.526+0000 7f6c6affd700 1 -- 172.21.15.27:0/312326436 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f6c5c054850 con 0x7f6c740f6860 2021-11-03T17:50:51.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.702+0000 7f6c78f61700 1 -- 172.21.15.27:0/312326436 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c740fdf00 con 0x7f6c60064090 2021-11-03T17:50:51.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.717+0000 7f6c6affd700 1 -- 172.21.15.27:0/312326436 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f6c740fdf00 con 0x7f6c60064090 2021-11-03T17:50:51.713 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:50:51.713 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (42m) 2m ago 43m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:50:51.713 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (34m) 2m ago 43m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:50:51.714 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (33m) 59s ago 42m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:50:51.714 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (42m) 2m ago 43m 41.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:50:51.714 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (35m) 2m ago 38m 23.8M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:50:51.714 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (38m) 2m ago 38m 3844M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:50:51.715 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (34m) 59s ago 38m 23.2M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:50:51.715 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (36m) 59s ago 38m 24.9M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:50:51.715 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (36m) 2m ago 44m 535M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:50:51.715 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (35m) 59s ago 42m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:50:51.716 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (35m) 2m ago 45m 198M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:50:51.716 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (34m) 59s ago 42m 95.6M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:50:51.716 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (43m) 2m ago 43m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:50:51.716 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (42m) 59s ago 42m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:50:51.716 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (33m) 2m ago 42m 331M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:50:51.717 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (33m) 2m ago 41m 361M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:50:51.717 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (32m) 2m ago 41m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:50:51.717 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (32m) 59s ago 41m 379M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:50:51.717 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (31m) 59s ago 40m 304M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:50:51.718 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (31m) 59s ago 40m 307M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:50:51.718 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (42m) 2m ago 43m 52.9M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:50:51.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.718+0000 7f6c78f61700 1 -- 172.21.15.27:0/312326436 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6c60064090 msgr2=0x7f6c60066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:50:51.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.718+0000 7f6c78f61700 1 --2- 172.21.15.27:0/312326436 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6c60064090 0x7f6c60066550 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7f6c74060040 tx=0x7f6c6c004d00).stop 2021-11-03T17:50:51.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.718+0000 7f6c78f61700 1 -- 172.21.15.27:0/312326436 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c740f6860 msgr2=0x7f6c7410c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:50:51.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.718+0000 7f6c78f61700 1 --2- 172.21.15.27:0/312326436 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c740f6860 0x7f6c7410c170 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7f6c5c009830 tx=0x7f6c5c009b30).stop 2021-11-03T17:50:51.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.718+0000 7f6c78f61700 1 -- 172.21.15.27:0/312326436 shutdown_connections 2021-11-03T17:50:51.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.718+0000 7f6c78f61700 1 --2- 172.21.15.27:0/312326436 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6c60064090 0x7f6c60066550 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:51.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.718+0000 7f6c78f61700 1 --2- 172.21.15.27:0/312326436 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c740f55c0 0x7f6c74109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:51.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.718+0000 7f6c78f61700 1 --2- 172.21.15.27:0/312326436 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c740f6860 0x7f6c7410c170 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:51.721 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.718+0000 7f6c78f61700 1 -- 172.21.15.27:0/312326436 >> 172.21.15.27:0/312326436 conn(0x7f6c740f0d40 msgr2=0x7f6c740fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:50:51.721 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.719+0000 7f6c78f61700 1 -- 172.21.15.27:0/312326436 shutdown_connections 2021-11-03T17:50:51.721 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.719+0000 7f6c78f61700 1 -- 172.21.15.27:0/312326436 wait complete. 2021-11-03T17:50:51.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.819+0000 7ff2a040b700 1 -- 172.21.15.27:0/3779726956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2980f98f0 msgr2=0x7ff2980fbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:50:51.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.819+0000 7ff2a040b700 1 --2- 172.21.15.27:0/3779726956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2980f98f0 0x7ff2980fbce0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7ff288004660 tx=0x7ff288009b30).stop 2021-11-03T17:50:51.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.819+0000 7ff2a040b700 1 -- 172.21.15.27:0/3779726956 shutdown_connections 2021-11-03T17:50:51.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.819+0000 7ff2a040b700 1 --2- 172.21.15.27:0/3779726956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2980f98f0 0x7ff2980fbce0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:51.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.819+0000 7ff2a040b700 1 --2- 172.21.15.27:0/3779726956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2980602f0 0x7ff2980f9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:51.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.819+0000 7ff2a040b700 1 -- 172.21.15.27:0/3779726956 >> 172.21.15.27:0/3779726956 conn(0x7ff2980f0de0 msgr2=0x7ff2980f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:50:51.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.819+0000 7ff2a040b700 1 -- 172.21.15.27:0/3779726956 shutdown_connections 2021-11-03T17:50:51.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.819+0000 7ff2a040b700 1 -- 172.21.15.27:0/3779726956 wait complete. 2021-11-03T17:50:51.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.819+0000 7ff2a040b700 1 Processor -- start 2021-11-03T17:50:51.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.820+0000 7ff2a040b700 1 -- start start 2021-11-03T17:50:51.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.820+0000 7ff2a040b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2980602f0 0x7ff298106eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:50:51.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.820+0000 7ff2a040b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2980f98f0 0x7ff2981073a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:50:51.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.820+0000 7ff2a040b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff298107950 con 0x7ff2980f98f0 2021-11-03T17:50:51.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.820+0000 7ff2a040b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff298107a90 con 0x7ff2980602f0 2021-11-03T17:50:51.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.820+0000 7ff29d9a6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2980f98f0 0x7ff2981073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:50:51.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.820+0000 7ff29d9a6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2980f98f0 0x7ff2981073a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59740/0 (socket says 172.21.15.27:59740) 2021-11-03T17:50:51.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.820+0000 7ff29d9a6700 1 -- 172.21.15.27:0/2856944209 learned_addr learned my addr 172.21.15.27:0/2856944209 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:50:51.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.820+0000 7ff29e1a7700 1 --2- 172.21.15.27:0/2856944209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2980602f0 0x7ff298106eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:50:51.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.820+0000 7ff29d9a6700 1 -- 172.21.15.27:0/2856944209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2980602f0 msgr2=0x7ff298106eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:50:51.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.820+0000 7ff29d9a6700 1 --2- 172.21.15.27:0/2856944209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2980602f0 0x7ff298106eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:51.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.820+0000 7ff29d9a6700 1 -- 172.21.15.27:0/2856944209 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff288005040 con 0x7ff2980f98f0 2021-11-03T17:50:51.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.820+0000 7ff29d9a6700 1 --2- 172.21.15.27:0/2856944209 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2980f98f0 0x7ff2981073a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7ff288000c00 tx=0x7ff28800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:50:51.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.820+0000 7ff28effd700 1 -- 172.21.15.27:0/2856944209 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff288019070 con 0x7ff2980f98f0 2021-11-03T17:50:51.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.820+0000 7ff28effd700 1 -- 172.21.15.27:0/2856944209 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7ff288007e70 con 0x7ff2980f98f0 2021-11-03T17:50:51.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.820+0000 7ff2a040b700 1 -- 172.21.15.27:0/2856944209 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2981083d0 con 0x7ff2980f98f0 2021-11-03T17:50:51.823 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.820+0000 7ff28effd700 1 -- 172.21.15.27:0/2856944209 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff288014ce0 con 0x7ff2980f98f0 2021-11-03T17:50:51.823 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.821+0000 7ff2a040b700 1 -- 172.21.15.27:0/2856944209 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff298108930 con 0x7ff2980f98f0 2021-11-03T17:50:51.823 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.821+0000 7ff2a040b700 1 -- 172.21.15.27:0/2856944209 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff298045ad0 con 0x7ff2980f98f0 2021-11-03T17:50:51.825 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.822+0000 7ff28effd700 1 -- 172.21.15.27:0/2856944209 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7ff288025070 con 0x7ff2980f98f0 2021-11-03T17:50:51.825 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.822+0000 7ff28effd700 1 --2- 172.21.15.27:0/2856944209 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff2840640b0 0x7ff284066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:50:51.826 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.822+0000 7ff28effd700 1 -- 172.21.15.27:0/2856944209 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7ff288081770 con 0x7ff2980f98f0 2021-11-03T17:50:51.826 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.822+0000 7ff29e1a7700 1 --2- 172.21.15.27:0/2856944209 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff2840640b0 0x7ff284066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:50:51.826 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.823+0000 7ff29e1a7700 1 --2- 172.21.15.27:0/2856944209 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff2840640b0 0x7ff284066570 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7ff294009fd0 tx=0x7ff29400b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:50:51.827 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:51.826+0000 7ff28effd700 1 -- 172.21.15.27:0/2856944209 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7ff2880539b0 con 0x7ff2980f98f0 2021-11-03T17:50:52.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.055+0000 7ff2a040b700 1 -- 172.21.15.27:0/2856944209 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff298027a00 con 0x7ff2980f98f0 2021-11-03T17:50:52.230 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.056+0000 7ff28effd700 1 -- 172.21.15.27:0/2856944209 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7ff288023070 con 0x7ff2980f98f0 2021-11-03T17:50:52.231 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:50:52.231 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:50:52.231 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:50:52.231 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:50:52.232 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:50:52.232 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:50:52.232 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:50:52.232 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:50:52.233 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:50:52.233 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:50:52.233 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:50:52.233 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:50:52.233 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:50:52.234 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:50:52.234 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:50:52.234 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:50:52.234 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:50:52.235 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:50:52.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:51 smithi115 ceph-mon[65453]: pgmap v1086: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:52.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.058+0000 7ff2a040b700 1 -- 172.21.15.27:0/2856944209 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff2840640b0 msgr2=0x7ff284066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:50:52.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.058+0000 7ff2a040b700 1 --2- 172.21.15.27:0/2856944209 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff2840640b0 0x7ff284066570 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7ff294009fd0 tx=0x7ff29400b040).stop 2021-11-03T17:50:52.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.058+0000 7ff2a040b700 1 -- 172.21.15.27:0/2856944209 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2980f98f0 msgr2=0x7ff2981073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:50:52.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.058+0000 7ff2a040b700 1 --2- 172.21.15.27:0/2856944209 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2980f98f0 0x7ff2981073a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7ff288000c00 tx=0x7ff28800b040).stop 2021-11-03T17:50:52.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.058+0000 7ff2a040b700 1 -- 172.21.15.27:0/2856944209 shutdown_connections 2021-11-03T17:50:52.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.058+0000 7ff2a040b700 1 --2- 172.21.15.27:0/2856944209 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff2840640b0 0x7ff284066570 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:52.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.058+0000 7ff2a040b700 1 --2- 172.21.15.27:0/2856944209 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2980602f0 0x7ff298106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:52.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.058+0000 7ff2a040b700 1 --2- 172.21.15.27:0/2856944209 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2980f98f0 0x7ff2981073a0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:52.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.058+0000 7ff2a040b700 1 -- 172.21.15.27:0/2856944209 >> 172.21.15.27:0/2856944209 conn(0x7ff2980f0de0 msgr2=0x7ff2980f31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:50:52.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.058+0000 7ff2a040b700 1 -- 172.21.15.27:0/2856944209 shutdown_connections 2021-11-03T17:50:52.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.058+0000 7ff2a040b700 1 -- 172.21.15.27:0/2856944209 wait complete. 2021-11-03T17:50:52.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.159+0000 7fc117d0f700 1 -- 172.21.15.27:0/3239606049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1100f7980 msgr2=0x7fc1100f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:50:52.242 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.159+0000 7fc117d0f700 1 --2- 172.21.15.27:0/3239606049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1100f7980 0x7fc1100f7de0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7fc104004660 tx=0x7fc104009b30).stop 2021-11-03T17:50:52.242 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.159+0000 7fc117d0f700 1 -- 172.21.15.27:0/3239606049 shutdown_connections 2021-11-03T17:50:52.242 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.159+0000 7fc117d0f700 1 --2- 172.21.15.27:0/3239606049 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1100f7980 0x7fc1100f7de0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:52.242 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.159+0000 7fc117d0f700 1 --2- 172.21.15.27:0/3239606049 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1100f66e0 0x7fc1100f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:50:52.243 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.159+0000 7fc117d0f700 1 -- 172.21.15.27:0/3239606049 >> 172.21.15.27:0/3239606049 conn(0x7fc1100f1e40 msgr2=0x7fc1100f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:50:52.243 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.159+0000 7fc117d0f700 1 -- 172.21.15.27:0/3239606049 shutdown_connections 2021-11-03T17:50:52.243 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.159+0000 7fc117d0f700 1 -- 172.21.15.27:0/3239606049 wait complete. 2021-11-03T17:50:52.243 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.159+0000 7fc117d0f700 1 Processor -- start 2021-11-03T17:50:52.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.159+0000 7fc117d0f700 1 -- start start 2021-11-03T17:50:52.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.160+0000 7fc117d0f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1100f66e0 0x7fc110106fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:50:52.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.160+0000 7fc117d0f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1100f7980 0x7fc1101074a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:50:52.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.160+0000 7fc117d0f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc110107ae0 con 0x7fc1100f66e0 2021-11-03T17:50:52.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.160+0000 7fc117d0f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc110108390 con 0x7fc1100f7980 2021-11-03T17:50:52.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.160+0000 7fc115aab700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1100f66e0 0x7fc110106fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:50:52.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.160+0000 7fc115aab700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1100f66e0 0x7fc110106fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59748/0 (socket says 172.21.15.27:59748) 2021-11-03T17:50:52.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.160+0000 7fc115aab700 1 -- 172.21.15.27:0/363016872 learned_addr learned my addr 172.21.15.27:0/363016872 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:50:52.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.160+0000 7fc1152aa700 1 --2- 172.21.15.27:0/363016872 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1100f7980 0x7fc1101074a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:50:52.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.160+0000 7fc115aab700 1 -- 172.21.15.27:0/363016872 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1100f7980 msgr2=0x7fc1101074a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:50:52.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.160+0000 7fc115aab700 1 --2- 172.21.15.27:0/363016872 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1100f7980 0x7fc1101074a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:52.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.160+0000 7fc115aab700 1 -- 172.21.15.27:0/363016872 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc104005040 con 0x7fc1100f66e0 2021-11-03T17:50:52.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.160+0000 7fc115aab700 1 --2- 172.21.15.27:0/363016872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1100f66e0 0x7fc110106fb0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7fc10800b790 tx=0x7fc1080092b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:50:52.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.160+0000 7fc10e7fc700 1 -- 172.21.15.27:0/363016872 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc10800d680 con 0x7fc1100f66e0 2021-11-03T17:50:52.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.160+0000 7fc10e7fc700 1 -- 172.21.15.27:0/363016872 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fc10800dc90 con 0x7fc1100f66e0 2021-11-03T17:50:52.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.160+0000 7fc10e7fc700 1 -- 172.21.15.27:0/363016872 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc108010cf0 con 0x7fc1100f66e0 2021-11-03T17:50:52.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.160+0000 7fc117d0f700 1 -- 172.21.15.27:0/363016872 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc110108670 con 0x7fc1100f66e0 2021-11-03T17:50:52.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.160+0000 7fc117d0f700 1 -- 172.21.15.27:0/363016872 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc110108c30 con 0x7fc1100f66e0 2021-11-03T17:50:52.248 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.161+0000 7fc117d0f700 1 -- 172.21.15.27:0/363016872 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1100f5ad0 con 0x7fc1100f66e0 2021-11-03T17:50:52.248 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.162+0000 7fc10e7fc700 1 -- 172.21.15.27:0/363016872 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fc10800f070 con 0x7fc1100f66e0 2021-11-03T17:50:52.248 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.162+0000 7fc10e7fc700 1 --2- 172.21.15.27:0/363016872 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc0f8064180 0x7fc0f8066640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:50:52.249 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.162+0000 7fc10e7fc700 1 -- 172.21.15.27:0/363016872 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fc108080600 con 0x7fc1100f66e0 2021-11-03T17:50:52.249 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.162+0000 7fc1152aa700 1 --2- 172.21.15.27:0/363016872 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc0f8064180 0x7fc0f8066640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:50:52.249 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.163+0000 7fc1152aa700 1 --2- 172.21.15.27:0/363016872 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc0f8064180 0x7fc0f8066640 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7fc104004630 tx=0x7fc104009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:50:52.249 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.166+0000 7fc10e7fc700 1 -- 172.21.15.27:0/363016872 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fc1080527d0 con 0x7fc1100f66e0 2021-11-03T17:50:52.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:51 smithi027 ceph-mon[97244]: pgmap v1086: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:52.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.377+0000 7fc117d0f700 1 -- 172.21.15.27:0/363016872 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc110045ad0 con 0x7fc1100f66e0 2021-11-03T17:50:52.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.377+0000 7fc10e7fc700 1 -- 172.21.15.27:0/363016872 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7fc108016070 con 0x7fc1100f66e0 2021-11-03T17:50:52.373 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:50:52.374 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:50:52.374 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:50:52.374 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:50:52.375 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:50:52.375 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:50:52.375 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:50:52.375 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:50:52.375 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:50:52.376 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:50:52.376 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:50:52.376 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:50:52.376 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:50:52.377 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:50:52.377 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:50:52.377 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:50:52.377 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:50:52.378 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:50:52.378 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:50:52.378 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:50:52.378 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:50:52.379 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:50:52.379 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:50:52.379 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:50:52.380 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:50:52.380 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:50:52.380 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:50:52.380 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:50:52.381 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:50:52.381 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:50:52.381 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:50:52.381 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:50:52.382 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:50:52.382 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:50:52.382 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:50:52.382 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:50:52.382 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:50:52.383 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:50:52.383 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:50:52.384 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.379+0000 7fc117d0f700 1 -- 172.21.15.27:0/363016872 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc0f8064180 msgr2=0x7fc0f8066640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:50:52.384 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.379+0000 7fc117d0f700 1 --2- 172.21.15.27:0/363016872 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc0f8064180 0x7fc0f8066640 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7fc104004630 tx=0x7fc104009b30).stop 2021-11-03T17:50:52.385 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.379+0000 7fc117d0f700 1 -- 172.21.15.27:0/363016872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1100f66e0 msgr2=0x7fc110106fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:50:52.385 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.379+0000 7fc117d0f700 1 --2- 172.21.15.27:0/363016872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1100f66e0 0x7fc110106fb0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7fc10800b790 tx=0x7fc1080092b0).stop 2021-11-03T17:50:52.385 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.379+0000 7fc117d0f700 1 -- 172.21.15.27:0/363016872 shutdown_connections 2021-11-03T17:50:52.385 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.379+0000 7fc117d0f700 1 --2- 172.21.15.27:0/363016872 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc0f8064180 0x7fc0f8066640 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:52.386 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.379+0000 7fc117d0f700 1 --2- 172.21.15.27:0/363016872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc1100f66e0 0x7fc110106fb0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:52.386 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.379+0000 7fc117d0f700 1 --2- 172.21.15.27:0/363016872 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc1100f7980 0x7fc1101074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:50:52.386 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.379+0000 7fc117d0f700 1 -- 172.21.15.27:0/363016872 >> 172.21.15.27:0/363016872 conn(0x7fc1100f1e40 msgr2=0x7fc1100fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:50:52.386 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.379+0000 7fc117d0f700 1 -- 172.21.15.27:0/363016872 shutdown_connections 2021-11-03T17:50:52.387 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:50:52.379+0000 7fc117d0f700 1 -- 172.21.15.27:0/363016872 wait complete. 2021-11-03T17:50:52.387 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:50:53.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:52 smithi115 ceph-mon[65453]: from='client.35224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:50:53.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:52 smithi115 ceph-mon[65453]: from='client.35228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:50:53.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:52 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2856944209' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:53.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:50:53.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:50:53.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:50:53.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:53.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:53.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:53.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:53.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:52 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/363016872' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:50:53.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:52 smithi027 ceph-mon[97244]: from='client.35224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:50:53.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:52 smithi027 ceph-mon[97244]: from='client.35228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:50:53.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:52 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2856944209' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:53.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:50:53.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:50:53.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:50:53.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:53.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:53.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:53.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:50:53.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:52 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/363016872' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:50:54.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:53 smithi115 ceph-mon[65453]: pgmap v1087: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:54.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:53 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:50:54.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:53 smithi027 ceph-mon[97244]: pgmap v1087: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:54.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:53 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:50:55.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:54 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2097.409811 secs 2021-11-03T17:50:55.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:54 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2097.409811 secs 2021-11-03T17:50:56.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:55 smithi115 ceph-mon[65453]: pgmap v1088: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:56.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:50:56.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:50:56.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:55 smithi027 ceph-mon[97244]: pgmap v1088: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:56.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:50:56.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:50:58.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:57 smithi115 ceph-mon[65453]: pgmap v1089: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:50:58.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:57 smithi027 ceph-mon[97244]: pgmap v1089: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:00.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:59 smithi115 ceph-mon[65453]: pgmap v1090: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:00.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:50:59 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2102.409948 secs 2021-11-03T17:51:00.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:59 smithi027 ceph-mon[97244]: pgmap v1090: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:00.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:50:59 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2102.409948 secs 2021-11-03T17:51:02.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:01 smithi115 ceph-mon[65453]: pgmap v1091: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:02.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:01 smithi027 ceph-mon[97244]: pgmap v1091: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:03.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:51:03.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:51:03.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:51:03.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:03.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:03.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:03.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:03.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:51:03.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:51:03.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:51:03.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:03.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:03.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:03.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:04.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:03 smithi115 ceph-mon[65453]: pgmap v1092: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:04.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:03 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:51:04.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:03 smithi027 ceph-mon[97244]: pgmap v1092: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:04.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:03 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:51:05.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:04 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2107.410090 secs 2021-11-03T17:51:05.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:04 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2107.410090 secs 2021-11-03T17:51:06.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:05 smithi115 ceph-mon[65453]: pgmap v1093: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:06.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:05 smithi027 ceph-mon[97244]: pgmap v1093: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:08.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:07 smithi115 ceph-mon[65453]: pgmap v1094: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:08.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:07 smithi027 ceph-mon[97244]: pgmap v1094: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:10.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:09 smithi115 ceph-mon[65453]: pgmap v1095: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:10.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:09 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2112.410234 secs 2021-11-03T17:51:10.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:09 smithi027 ceph-mon[97244]: pgmap v1095: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:10.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:09 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2112.410234 secs 2021-11-03T17:51:12.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:11 smithi115 ceph-mon[65453]: pgmap v1096: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:12.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:11 smithi027 ceph-mon[97244]: pgmap v1096: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:13.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:51:13.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:51:13.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:51:13.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:13.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:13.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:13.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:13.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:51:13.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:51:13.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:51:13.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:13.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:13.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:13.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:14.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:13 smithi115 ceph-mon[65453]: pgmap v1097: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:14.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:13 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:51:14.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:13 smithi027 ceph-mon[97244]: pgmap v1097: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:14.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:13 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:51:15.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:14 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2117.410325 secs 2021-11-03T17:51:15.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:14 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2117.410325 secs 2021-11-03T17:51:16.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:15 smithi115 ceph-mon[65453]: pgmap v1098: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:16.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:15 smithi027 ceph-mon[97244]: pgmap v1098: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:18.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:17 smithi115 ceph-mon[65453]: pgmap v1099: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:18.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:17 smithi027 ceph-mon[97244]: pgmap v1099: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:20.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:19 smithi115 ceph-mon[65453]: pgmap v1100: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:20.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:19 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2122.410456 secs 2021-11-03T17:51:20.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:19 smithi027 ceph-mon[97244]: pgmap v1100: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:20.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:19 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2122.410456 secs 2021-11-03T17:51:22.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:21 smithi115 ceph-mon[65453]: pgmap v1101: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:22.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:21 smithi027 ceph-mon[97244]: pgmap v1101: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:22.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.494+0000 7f1012bba700 1 -- 172.21.15.27:0/1505995447 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f100c0f6810 msgr2=0x7f100c0f6c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:51:22.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.494+0000 7f1012bba700 1 --2- 172.21.15.27:0/1505995447 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f100c0f6810 0x7f100c0f6c70 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7f0ffc0099e0 tx=0x7f0ffc004d10).stop 2021-11-03T17:51:22.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.494+0000 7f1012bba700 1 -- 172.21.15.27:0/1505995447 shutdown_connections 2021-11-03T17:51:22.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.494+0000 7f1012bba700 1 --2- 172.21.15.27:0/1505995447 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f100c0f6810 0x7f100c0f6c70 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:22.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.494+0000 7f1012bba700 1 --2- 172.21.15.27:0/1505995447 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f100c0f5570 0x7f100c0f5990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:22.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.494+0000 7f1012bba700 1 -- 172.21.15.27:0/1505995447 >> 172.21.15.27:0/1505995447 conn(0x7f100c0f0cd0 msgr2=0x7f100c0f30f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:51:22.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.494+0000 7f1012bba700 1 -- 172.21.15.27:0/1505995447 shutdown_connections 2021-11-03T17:51:22.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.494+0000 7f1012bba700 1 -- 172.21.15.27:0/1505995447 wait complete. 2021-11-03T17:51:22.494 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.495+0000 7f1012bba700 1 Processor -- start 2021-11-03T17:51:22.494 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.495+0000 7f1012bba700 1 -- start start 2021-11-03T17:51:22.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.495+0000 7f1012bba700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f100c0f5570 0x7f100c10c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:51:22.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.495+0000 7f1012bba700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f100c0f6810 0x7f100c10e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:51:22.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.495+0000 7f1012bba700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100c0602b0 con 0x7f100c0f5570 2021-11-03T17:51:22.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.495+0000 7f1012bba700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100c0603f0 con 0x7f100c0f6810 2021-11-03T17:51:22.496 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.495+0000 7f1010956700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f100c0f5570 0x7f100c10c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:51:22.496 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.495+0000 7f1010956700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f100c0f5570 0x7f100c10c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59766/0 (socket says 172.21.15.27:59766) 2021-11-03T17:51:22.496 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.495+0000 7f1010956700 1 -- 172.21.15.27:0/807912502 learned_addr learned my addr 172.21.15.27:0/807912502 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:51:22.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.495+0000 7f100bfff700 1 --2- 172.21.15.27:0/807912502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f100c0f6810 0x7f100c10e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:51:22.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.496+0000 7f1010956700 1 -- 172.21.15.27:0/807912502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f100c0f6810 msgr2=0x7f100c10e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:51:22.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.496+0000 7f1010956700 1 --2- 172.21.15.27:0/807912502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f100c0f6810 0x7f100c10e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:22.498 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.496+0000 7f1010956700 1 -- 172.21.15.27:0/807912502 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ffc009640 con 0x7f100c0f5570 2021-11-03T17:51:22.498 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.496+0000 7f100bfff700 1 --2- 172.21.15.27:0/807912502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f100c0f6810 0x7f100c10e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:51:22.498 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.496+0000 7f1010956700 1 --2- 172.21.15.27:0/807912502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f100c0f5570 0x7f100c10c090 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f100400e7c0 tx=0x7f100400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:51:22.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.496+0000 7f10097fa700 1 -- 172.21.15.27:0/807912502 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f10040092e0 con 0x7f100c0f5570 2021-11-03T17:51:22.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.496+0000 7f1012bba700 1 -- 172.21.15.27:0/807912502 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f100c10ea80 con 0x7f100c0f5570 2021-11-03T17:51:22.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.496+0000 7f10097fa700 1 -- 172.21.15.27:0/807912502 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f100400f040 con 0x7f100c0f5570 2021-11-03T17:51:22.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.496+0000 7f10097fa700 1 -- 172.21.15.27:0/807912502 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1004014750 con 0x7f100c0f5570 2021-11-03T17:51:22.500 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.496+0000 7f1012bba700 1 -- 172.21.15.27:0/807912502 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f100c10ef00 con 0x7f100c0f5570 2021-11-03T17:51:22.502 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.497+0000 7f1012bba700 1 -- 172.21.15.27:0/807912502 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f100c045ad0 con 0x7f100c0f5570 2021-11-03T17:51:22.502 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.497+0000 7f10097fa700 1 -- 172.21.15.27:0/807912502 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f100401b030 con 0x7f100c0f5570 2021-11-03T17:51:22.502 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.498+0000 7f10097fa700 1 --2- 172.21.15.27:0/807912502 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0ff8064100 0x7f0ff80665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:51:22.503 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.498+0000 7f10097fa700 1 -- 172.21.15.27:0/807912502 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f10040809a0 con 0x7f100c0f5570 2021-11-03T17:51:22.503 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.498+0000 7f100bfff700 1 --2- 172.21.15.27:0/807912502 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0ff8064100 0x7f0ff80665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:51:22.504 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.498+0000 7f100bfff700 1 --2- 172.21.15.27:0/807912502 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0ff8064100 0x7f0ff80665c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f0ffc00b3b0 tx=0x7f0ffc004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:51:22.504 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.502+0000 7f10097fa700 1 -- 172.21.15.27:0/807912502 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f1004052bb0 con 0x7f100c0f5570 2021-11-03T17:51:22.676 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.680+0000 7f1012bba700 1 -- 172.21.15.27:0/807912502 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f100c057f10 con 0x7f0ff8064100 2021-11-03T17:51:22.677 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.682+0000 7f10097fa700 1 -- 172.21.15.27:0/807912502 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f100c057f10 con 0x7f0ff8064100 2021-11-03T17:51:22.679 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.684+0000 7f1012bba700 1 -- 172.21.15.27:0/807912502 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0ff8064100 msgr2=0x7f0ff80665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:51:22.680 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.684+0000 7f1012bba700 1 --2- 172.21.15.27:0/807912502 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0ff8064100 0x7f0ff80665c0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 rx=0x7f0ffc00b3b0 tx=0x7f0ffc004d10).stop 2021-11-03T17:51:22.680 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.684+0000 7f1012bba700 1 -- 172.21.15.27:0/807912502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f100c0f5570 msgr2=0x7f100c10c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:51:22.680 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.684+0000 7f1012bba700 1 --2- 172.21.15.27:0/807912502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f100c0f5570 0x7f100c10c090 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f100400e7c0 tx=0x7f100400b040).stop 2021-11-03T17:51:22.680 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.684+0000 7f1012bba700 1 -- 172.21.15.27:0/807912502 shutdown_connections 2021-11-03T17:51:22.681 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.684+0000 7f1012bba700 1 --2- 172.21.15.27:0/807912502 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0ff8064100 0x7f0ff80665c0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:22.681 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.684+0000 7f1012bba700 1 --2- 172.21.15.27:0/807912502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f100c0f5570 0x7f100c10c090 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:22.681 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.684+0000 7f1012bba700 1 --2- 172.21.15.27:0/807912502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f100c0f6810 0x7f100c10e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:22.681 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.684+0000 7f1012bba700 1 -- 172.21.15.27:0/807912502 >> 172.21.15.27:0/807912502 conn(0x7f100c0f0cd0 msgr2=0x7f100c0fa260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:51:22.682 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.684+0000 7f1012bba700 1 -- 172.21.15.27:0/807912502 shutdown_connections 2021-11-03T17:51:22.682 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.684+0000 7f1012bba700 1 -- 172.21.15.27:0/807912502 wait complete. 2021-11-03T17:51:22.689 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:51:22.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.784+0000 7fe3cdb67700 1 -- 172.21.15.27:0/382625471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3c80f5ed0 msgr2=0x7fe3c80f6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:51:22.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.784+0000 7fe3cdb67700 1 --2- 172.21.15.27:0/382625471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3c80f5ed0 0x7fe3c80f6900 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7fe3bc004660 tx=0x7fe3bc009b30).stop 2021-11-03T17:51:22.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.784+0000 7fe3cdb67700 1 -- 172.21.15.27:0/382625471 shutdown_connections 2021-11-03T17:51:22.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.784+0000 7fe3cdb67700 1 --2- 172.21.15.27:0/382625471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3c80f5ed0 0x7fe3c80f6900 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:22.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.784+0000 7fe3cdb67700 1 --2- 172.21.15.27:0/382625471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3c80f55c0 0x7fe3c80f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:22.787 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.784+0000 7fe3cdb67700 1 -- 172.21.15.27:0/382625471 >> 172.21.15.27:0/382625471 conn(0x7fe3c80f0de0 msgr2=0x7fe3c80f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:51:22.787 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.784+0000 7fe3cdb67700 1 -- 172.21.15.27:0/382625471 shutdown_connections 2021-11-03T17:51:22.787 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.784+0000 7fe3cdb67700 1 -- 172.21.15.27:0/382625471 wait complete. 2021-11-03T17:51:22.788 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.785+0000 7fe3cdb67700 1 Processor -- start 2021-11-03T17:51:22.788 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.785+0000 7fe3cdb67700 1 -- start start 2021-11-03T17:51:22.789 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.785+0000 7fe3cdb67700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3c80f55c0 0x7fe3c8109c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:51:22.789 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.785+0000 7fe3cdb67700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3c80f5ed0 0x7fe3c810c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:51:22.790 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.785+0000 7fe3cdb67700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c80604c0 con 0x7fe3c80f55c0 2021-11-03T17:51:22.790 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.785+0000 7fe3cdb67700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3c8060600 con 0x7fe3c80f5ed0 2021-11-03T17:51:22.791 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.785+0000 7fe3c77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3c80f55c0 0x7fe3c8109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:51:22.791 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.785+0000 7fe3c77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3c80f55c0 0x7fe3c8109c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59778/0 (socket says 172.21.15.27:59778) 2021-11-03T17:51:22.791 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.785+0000 7fe3c77fe700 1 -- 172.21.15.27:0/1062002495 learned_addr learned my addr 172.21.15.27:0/1062002495 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:51:22.792 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.785+0000 7fe3c6ffd700 1 --2- 172.21.15.27:0/1062002495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3c80f5ed0 0x7fe3c810c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:51:22.792 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.785+0000 7fe3c77fe700 1 -- 172.21.15.27:0/1062002495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3c80f5ed0 msgr2=0x7fe3c810c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:51:22.793 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.785+0000 7fe3c77fe700 1 --2- 172.21.15.27:0/1062002495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3c80f5ed0 0x7fe3c810c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:22.793 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.785+0000 7fe3c77fe700 1 -- 172.21.15.27:0/1062002495 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3bc005040 con 0x7fe3c80f55c0 2021-11-03T17:51:22.794 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.786+0000 7fe3c77fe700 1 --2- 172.21.15.27:0/1062002495 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3c80f55c0 0x7fe3c8109c70 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7fe3b80026d0 tx=0x7fe3b8009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:51:22.794 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.786+0000 7fe3affff700 1 -- 172.21.15.27:0/1062002495 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3b80079c0 con 0x7fe3c80f55c0 2021-11-03T17:51:22.794 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.786+0000 7fe3cdb67700 1 -- 172.21.15.27:0/1062002495 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3c810c720 con 0x7fe3c80f55c0 2021-11-03T17:51:22.795 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.786+0000 7fe3affff700 1 -- 172.21.15.27:0/1062002495 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fe3b800f040 con 0x7fe3c80f55c0 2021-11-03T17:51:22.795 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.786+0000 7fe3affff700 1 -- 172.21.15.27:0/1062002495 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe3b80148b0 con 0x7fe3c80f55c0 2021-11-03T17:51:22.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.786+0000 7fe3cdb67700 1 -- 172.21.15.27:0/1062002495 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3c810cce0 con 0x7fe3c80f55c0 2021-11-03T17:51:22.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.787+0000 7fe3cdb67700 1 -- 172.21.15.27:0/1062002495 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3c8045ad0 con 0x7fe3c80f55c0 2021-11-03T17:51:22.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.787+0000 7fe3affff700 1 -- 172.21.15.27:0/1062002495 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fe3b8007b20 con 0x7fe3c80f55c0 2021-11-03T17:51:22.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.788+0000 7fe3affff700 1 --2- 172.21.15.27:0/1062002495 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe3b00640b0 0x7fe3b0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:51:22.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.788+0000 7fe3affff700 1 -- 172.21.15.27:0/1062002495 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fe3b8080790 con 0x7fe3c80f55c0 2021-11-03T17:51:22.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.788+0000 7fe3c6ffd700 1 --2- 172.21.15.27:0/1062002495 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe3b00640b0 0x7fe3b0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:51:22.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.788+0000 7fe3c6ffd700 1 --2- 172.21.15.27:0/1062002495 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe3b00640b0 0x7fe3b0066570 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7fe3bc004630 tx=0x7fe3bc009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:51:22.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.792+0000 7fe3affff700 1 -- 172.21.15.27:0/1062002495 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fe3b8052a90 con 0x7fe3c80f55c0 2021-11-03T17:51:22.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.964+0000 7fe3cdb67700 1 -- 172.21.15.27:0/1062002495 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe3c8057f10 con 0x7fe3b00640b0 2021-11-03T17:51:22.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.979+0000 7fe3affff700 1 -- 172.21.15.27:0/1062002495 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fe3c8057f10 con 0x7fe3b00640b0 2021-11-03T17:51:22.976 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:51:22.976 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (42m) 2m ago 44m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:51:22.977 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (34m) 2m ago 44m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:51:22.977 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (34m) 91s ago 43m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:51:22.977 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (42m) 2m ago 43m 41.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:51:22.978 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (36m) 2m ago 39m 23.8M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:51:22.978 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (39m) 2m ago 39m 3844M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:51:22.979 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (35m) 91s ago 39m 23.2M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:51:22.979 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (36m) 91s ago 39m 24.9M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:51:22.980 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (36m) 2m ago 45m 535M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:51:22.980 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (35m) 91s ago 43m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:51:22.981 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (35m) 2m ago 45m 198M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:51:22.981 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (35m) 91s ago 43m 95.6M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:51:22.981 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (43m) 2m ago 43m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:51:22.982 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (43m) 91s ago 43m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:51:22.982 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (34m) 2m ago 42m 331M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:51:22.983 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (33m) 2m ago 42m 361M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:51:22.983 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (33m) 2m ago 41m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:51:22.983 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (32m) 91s ago 41m 379M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:51:22.984 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (32m) 91s ago 41m 304M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:51:22.984 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (31m) 91s ago 40m 307M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:51:22.984 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (42m) 2m ago 43m 52.9M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:51:22.986 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.981+0000 7fe3cdb67700 1 -- 172.21.15.27:0/1062002495 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe3b00640b0 msgr2=0x7fe3b0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:51:22.986 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.981+0000 7fe3cdb67700 1 --2- 172.21.15.27:0/1062002495 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe3b00640b0 0x7fe3b0066570 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7fe3bc004630 tx=0x7fe3bc009b30).stop 2021-11-03T17:51:22.986 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.981+0000 7fe3cdb67700 1 -- 172.21.15.27:0/1062002495 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3c80f55c0 msgr2=0x7fe3c8109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:51:22.987 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.981+0000 7fe3cdb67700 1 --2- 172.21.15.27:0/1062002495 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3c80f55c0 0x7fe3c8109c70 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7fe3b80026d0 tx=0x7fe3b8009de0).stop 2021-11-03T17:51:22.987 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.981+0000 7fe3cdb67700 1 -- 172.21.15.27:0/1062002495 shutdown_connections 2021-11-03T17:51:22.987 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.981+0000 7fe3cdb67700 1 --2- 172.21.15.27:0/1062002495 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe3b00640b0 0x7fe3b0066570 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:22.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.981+0000 7fe3cdb67700 1 --2- 172.21.15.27:0/1062002495 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe3c80f55c0 0x7fe3c8109c70 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:22.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.981+0000 7fe3cdb67700 1 --2- 172.21.15.27:0/1062002495 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe3c80f5ed0 0x7fe3c810c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:22.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.981+0000 7fe3cdb67700 1 -- 172.21.15.27:0/1062002495 >> 172.21.15.27:0/1062002495 conn(0x7fe3c80f0de0 msgr2=0x7fe3c80fa350 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:51:22.989 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.981+0000 7fe3cdb67700 1 -- 172.21.15.27:0/1062002495 shutdown_connections 2021-11-03T17:51:22.989 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:22.981+0000 7fe3cdb67700 1 -- 172.21.15.27:0/1062002495 wait complete. 2021-11-03T17:51:23.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.080+0000 7fd8785bd700 1 -- 172.21.15.27:0/2623262314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8700f7980 msgr2=0x7fd8700f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:51:23.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.080+0000 7fd8785bd700 1 --2- 172.21.15.27:0/2623262314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8700f7980 0x7fd8700f7de0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7fd864004660 tx=0x7fd864009b30).stop 2021-11-03T17:51:23.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.080+0000 7fd8785bd700 1 -- 172.21.15.27:0/2623262314 shutdown_connections 2021-11-03T17:51:23.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.080+0000 7fd8785bd700 1 --2- 172.21.15.27:0/2623262314 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8700f7980 0x7fd8700f7de0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:23.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.080+0000 7fd8785bd700 1 --2- 172.21.15.27:0/2623262314 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8700f66e0 0x7fd8700f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:23.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.080+0000 7fd8785bd700 1 -- 172.21.15.27:0/2623262314 >> 172.21.15.27:0/2623262314 conn(0x7fd8700f1e40 msgr2=0x7fd8700f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:51:23.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.080+0000 7fd8785bd700 1 -- 172.21.15.27:0/2623262314 shutdown_connections 2021-11-03T17:51:23.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.080+0000 7fd8785bd700 1 -- 172.21.15.27:0/2623262314 wait complete. 2021-11-03T17:51:23.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.080+0000 7fd8785bd700 1 Processor -- start 2021-11-03T17:51:23.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.081+0000 7fd8785bd700 1 -- start start 2021-11-03T17:51:23.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.081+0000 7fd8785bd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8700f66e0 0x7fd87006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:51:23.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.081+0000 7fd8785bd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8700f7980 0x7fd870069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:51:23.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.081+0000 7fd8785bd700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd87006a380 con 0x7fd8700f7980 2021-11-03T17:51:23.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.081+0000 7fd8785bd700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd87006a4c0 con 0x7fd8700f66e0 2021-11-03T17:51:23.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.081+0000 7fd875b58700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8700f7980 0x7fd870069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:51:23.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.081+0000 7fd875b58700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8700f7980 0x7fd870069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59790/0 (socket says 172.21.15.27:59790) 2021-11-03T17:51:23.086 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.081+0000 7fd875b58700 1 -- 172.21.15.27:0/3565681602 learned_addr learned my addr 172.21.15.27:0/3565681602 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:51:23.086 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.081+0000 7fd876359700 1 --2- 172.21.15.27:0/3565681602 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8700f66e0 0x7fd87006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:51:23.086 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.081+0000 7fd875b58700 1 -- 172.21.15.27:0/3565681602 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8700f66e0 msgr2=0x7fd87006b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:51:23.086 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.081+0000 7fd875b58700 1 --2- 172.21.15.27:0/3565681602 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8700f66e0 0x7fd87006b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:23.087 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.081+0000 7fd875b58700 1 -- 172.21.15.27:0/3565681602 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd864005040 con 0x7fd8700f7980 2021-11-03T17:51:23.087 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.081+0000 7fd875b58700 1 --2- 172.21.15.27:0/3565681602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8700f7980 0x7fd870069e20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7fd864009990 tx=0x7fd864009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:51:23.087 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.081+0000 7fd86effd700 1 -- 172.21.15.27:0/3565681602 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd864018070 con 0x7fd8700f7980 2021-11-03T17:51:23.087 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.081+0000 7fd86effd700 1 -- 172.21.15.27:0/3565681602 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fd864007ba0 con 0x7fd8700f7980 2021-11-03T17:51:23.087 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.081+0000 7fd8785bd700 1 -- 172.21.15.27:0/3565681602 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd87006a740 con 0x7fd8700f7980 2021-11-03T17:51:23.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.081+0000 7fd86effd700 1 -- 172.21.15.27:0/3565681602 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd86401c3c0 con 0x7fd8700f7980 2021-11-03T17:51:23.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.082+0000 7fd8785bd700 1 -- 172.21.15.27:0/3565681602 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd87006aca0 con 0x7fd8700f7980 2021-11-03T17:51:23.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.082+0000 7fd8785bd700 1 -- 172.21.15.27:0/3565681602 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd870045ad0 con 0x7fd8700f7980 2021-11-03T17:51:23.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.083+0000 7fd86effd700 1 -- 172.21.15.27:0/3565681602 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fd86401d420 con 0x7fd8700f7980 2021-11-03T17:51:23.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.083+0000 7fd86effd700 1 --2- 172.21.15.27:0/3565681602 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd8580640b0 0x7fd858066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:51:23.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.083+0000 7fd86effd700 1 -- 172.21.15.27:0/3565681602 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fd864082490 con 0x7fd8700f7980 2021-11-03T17:51:23.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.083+0000 7fd876359700 1 --2- 172.21.15.27:0/3565681602 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd8580640b0 0x7fd858066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:51:23.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.084+0000 7fd876359700 1 --2- 172.21.15.27:0/3565681602 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd8580640b0 0x7fd858066570 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7fd87006b730 tx=0x7fd868004d00).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:51:23.090 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.087+0000 7fd86effd700 1 -- 172.21.15.27:0/3565681602 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fd8640547f0 con 0x7fd8700f7980 2021-11-03T17:51:23.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:51:23.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:51:23.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:51:23.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:23.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:23.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:23.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:23.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:51:23.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:51:23.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:51:23.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:23.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:23.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:23.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:23.312 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.317+0000 7fd8785bd700 1 -- 172.21.15.27:0/3565681602 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd870027a00 con 0x7fd8700f7980 2021-11-03T17:51:23.313 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.317+0000 7fd86effd700 1 -- 172.21.15.27:0/3565681602 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fd8640543e0 con 0x7fd8700f7980 2021-11-03T17:51:23.314 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:51:23.314 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:51:23.314 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:51:23.314 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:51:23.315 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:51:23.315 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:51:23.315 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:51:23.315 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:51:23.316 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:51:23.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:51:23.316 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:51:23.316 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:51:23.316 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:51:23.317 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:51:23.317 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:51:23.317 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:51:23.317 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:51:23.317 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:51:23.318 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.319+0000 7fd8785bd700 1 -- 172.21.15.27:0/3565681602 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd8580640b0 msgr2=0x7fd858066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:51:23.319 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.319+0000 7fd8785bd700 1 --2- 172.21.15.27:0/3565681602 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd8580640b0 0x7fd858066570 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7fd87006b730 tx=0x7fd868004d00).stop 2021-11-03T17:51:23.319 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.319+0000 7fd8785bd700 1 -- 172.21.15.27:0/3565681602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8700f7980 msgr2=0x7fd870069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:51:23.319 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.319+0000 7fd8785bd700 1 --2- 172.21.15.27:0/3565681602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8700f7980 0x7fd870069e20 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7fd864009990 tx=0x7fd864009b30).stop 2021-11-03T17:51:23.319 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.319+0000 7fd8785bd700 1 -- 172.21.15.27:0/3565681602 shutdown_connections 2021-11-03T17:51:23.320 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.319+0000 7fd8785bd700 1 --2- 172.21.15.27:0/3565681602 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd8580640b0 0x7fd858066570 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:23.320 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.319+0000 7fd8785bd700 1 --2- 172.21.15.27:0/3565681602 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8700f66e0 0x7fd87006b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:23.320 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.319+0000 7fd8785bd700 1 --2- 172.21.15.27:0/3565681602 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8700f7980 0x7fd870069e20 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:23.320 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.319+0000 7fd8785bd700 1 -- 172.21.15.27:0/3565681602 >> 172.21.15.27:0/3565681602 conn(0x7fd8700f1e40 msgr2=0x7fd8700fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:51:23.321 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.319+0000 7fd8785bd700 1 -- 172.21.15.27:0/3565681602 shutdown_connections 2021-11-03T17:51:23.321 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.319+0000 7fd8785bd700 1 -- 172.21.15.27:0/3565681602 wait complete. 2021-11-03T17:51:23.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.419+0000 7f16b1e0d700 1 -- 172.21.15.27:0/853831203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16ac0f6bb0 msgr2=0x7f16ac0f6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:51:23.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.419+0000 7f16b1e0d700 1 --2- 172.21.15.27:0/853831203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16ac0f6bb0 0x7f16ac0f6fd0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f169c004660 tx=0x7f169c009b30).stop 2021-11-03T17:51:23.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.419+0000 7f16b1e0d700 1 -- 172.21.15.27:0/853831203 shutdown_connections 2021-11-03T17:51:23.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.419+0000 7f16b1e0d700 1 --2- 172.21.15.27:0/853831203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16ac0f58b0 0x7f16ac0f5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:51:23.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.419+0000 7f16b1e0d700 1 --2- 172.21.15.27:0/853831203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16ac0f6bb0 0x7f16ac0f6fd0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:23.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.419+0000 7f16b1e0d700 1 -- 172.21.15.27:0/853831203 >> 172.21.15.27:0/853831203 conn(0x7f16ac0f04c0 msgr2=0x7f16ac0f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:51:23.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.419+0000 7f16b1e0d700 1 -- 172.21.15.27:0/853831203 shutdown_connections 2021-11-03T17:51:23.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.419+0000 7f16b1e0d700 1 -- 172.21.15.27:0/853831203 wait complete. 2021-11-03T17:51:23.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.420+0000 7f16b1e0d700 1 Processor -- start 2021-11-03T17:51:23.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.420+0000 7f16b1e0d700 1 -- start start 2021-11-03T17:51:23.419 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.420+0000 7f16b1e0d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16ac0f58b0 0x7f16ac105eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:51:23.419 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.420+0000 7f16b1e0d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16ac0f6bb0 0x7f16ac1063a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:51:23.419 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.420+0000 7f16b1e0d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac1069e0 con 0x7f16ac0f6bb0 2021-11-03T17:51:23.419 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.420+0000 7f16b1e0d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16ac107290 con 0x7f16ac0f58b0 2021-11-03T17:51:23.420 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.420+0000 7f16aaffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16ac0f6bb0 0x7f16ac1063a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:51:23.420 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.420+0000 7f16aaffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16ac0f6bb0 0x7f16ac1063a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59800/0 (socket says 172.21.15.27:59800) 2021-11-03T17:51:23.420 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.420+0000 7f16aaffd700 1 -- 172.21.15.27:0/1475180828 learned_addr learned my addr 172.21.15.27:0/1475180828 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:51:23.420 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.420+0000 7f16aaffd700 1 -- 172.21.15.27:0/1475180828 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16ac0f58b0 msgr2=0x7f16ac105eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:51:23.420 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.420+0000 7f16aaffd700 1 --2- 172.21.15.27:0/1475180828 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16ac0f58b0 0x7f16ac105eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:51:23.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.420+0000 7f16aaffd700 1 -- 172.21.15.27:0/1475180828 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f169c005040 con 0x7f16ac0f6bb0 2021-11-03T17:51:23.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.420+0000 7f16aaffd700 1 --2- 172.21.15.27:0/1475180828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16ac0f6bb0 0x7f16ac1063a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7f16a00093f0 tx=0x7f16a0006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:51:23.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.421+0000 7f1693fff700 1 -- 172.21.15.27:0/1475180828 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16a0009ea0 con 0x7f16ac0f6bb0 2021-11-03T17:51:23.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.421+0000 7f1693fff700 1 -- 172.21.15.27:0/1475180828 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f16a0013af0 con 0x7f16ac0f6bb0 2021-11-03T17:51:23.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.421+0000 7f16b1e0d700 1 -- 172.21.15.27:0/1475180828 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16ac107570 con 0x7f16ac0f6bb0 2021-11-03T17:51:23.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.421+0000 7f1693fff700 1 -- 172.21.15.27:0/1475180828 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f16a000db10 con 0x7f16ac0f6bb0 2021-11-03T17:51:23.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.421+0000 7f16b1e0d700 1 -- 172.21.15.27:0/1475180828 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16ac107b30 con 0x7f16ac0f6bb0 2021-11-03T17:51:23.423 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.422+0000 7f16b1e0d700 1 -- 172.21.15.27:0/1475180828 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16ac0f4150 con 0x7f16ac0f6bb0 2021-11-03T17:51:23.424 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.422+0000 7f1693fff700 1 -- 172.21.15.27:0/1475180828 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f16a001a060 con 0x7f16ac0f6bb0 2021-11-03T17:51:23.424 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.422+0000 7f1693fff700 1 --2- 172.21.15.27:0/1475180828 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f16940640b0 0x7f1694066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:51:23.424 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.422+0000 7f1693fff700 1 -- 172.21.15.27:0/1475180828 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f16a00808c0 con 0x7f16ac0f6bb0 2021-11-03T17:51:23.424 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.423+0000 7f16ab7fe700 1 --2- 172.21.15.27:0/1475180828 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f16940640b0 0x7f1694066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:51:23.425 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.423+0000 7f16ab7fe700 1 --2- 172.21.15.27:0/1475180828 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f16940640b0 0x7f1694066570 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f169c004e30 tx=0x7f169c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:51:23.425 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.427+0000 7f1693fff700 1 -- 172.21.15.27:0/1475180828 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f16a0052ad0 con 0x7f16ac0f6bb0 2021-11-03T17:51:23.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.633+0000 7f16b1e0d700 1 -- 172.21.15.27:0/1475180828 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f16ac0fdf70 con 0x7f16ac0f6bb0 2021-11-03T17:51:23.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.633+0000 7f1693fff700 1 -- 172.21.15.27:0/1475180828 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f16a0018710 con 0x7f16ac0f6bb0 2021-11-03T17:51:23.630 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:51:23.630 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:51:23.630 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:51:23.631 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:51:23.631 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:51:23.631 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:51:23.631 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:51:23.632 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:51:23.632 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:51:23.632 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:51:23.632 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:51:23.633 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:51:23.633 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:51:23.633 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:51:23.633 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:51:23.633 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:51:23.634 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:51:23.634 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:51:23.634 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:51:23.634 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:51:23.635 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:51:23.635 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:51:23.635 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:51:23.635 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:51:23.636 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:51:23.636 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:51:23.636 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:51:23.636 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:51:23.636 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:51:23.637 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:51:23.637 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:51:23.637 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:51:23.637 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:51:23.638 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:51:23.638 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:51:23.638 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:51:23.638 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:51:23.639 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:51:23.639 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:51:23.640 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.635+0000 7f16b1e0d700 1 -- 172.21.15.27:0/1475180828 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f16940640b0 msgr2=0x7f1694066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:51:23.641 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.635+0000 7f16b1e0d700 1 --2- 172.21.15.27:0/1475180828 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f16940640b0 0x7f1694066570 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 rx=0x7f169c004e30 tx=0x7f169c009b30).stop 2021-11-03T17:51:23.641 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.635+0000 7f16b1e0d700 1 -- 172.21.15.27:0/1475180828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16ac0f6bb0 msgr2=0x7f16ac1063a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:51:23.641 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.635+0000 7f16b1e0d700 1 --2- 172.21.15.27:0/1475180828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16ac0f6bb0 0x7f16ac1063a0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7f16a00093f0 tx=0x7f16a0006ec0).stop 2021-11-03T17:51:23.641 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.635+0000 7f16b1e0d700 1 -- 172.21.15.27:0/1475180828 shutdown_connections 2021-11-03T17:51:23.642 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.635+0000 7f16b1e0d700 1 --2- 172.21.15.27:0/1475180828 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f16940640b0 0x7f1694066570 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:23.642 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.635+0000 7f16b1e0d700 1 --2- 172.21.15.27:0/1475180828 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f16ac0f58b0 0x7f16ac105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:51:23.642 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.635+0000 7f16b1e0d700 1 --2- 172.21.15.27:0/1475180828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f16ac0f6bb0 0x7f16ac1063a0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:23.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.635+0000 7f16b1e0d700 1 -- 172.21.15.27:0/1475180828 >> 172.21.15.27:0/1475180828 conn(0x7f16ac0f04c0 msgr2=0x7f16ac0fa310 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:51:23.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.636+0000 7f16b1e0d700 1 -- 172.21.15.27:0/1475180828 shutdown_connections 2021-11-03T17:51:23.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:23.636+0000 7f16b1e0d700 1 -- 172.21.15.27:0/1475180828 wait complete. 2021-11-03T17:51:23.643 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:51:24.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:23 smithi115 ceph-mon[65453]: pgmap v1102: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:24.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:23 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:51:24.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:23 smithi115 ceph-mon[65453]: from='client.35240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:51:24.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:23 smithi115 ceph-mon[65453]: from='client.35244 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:51:24.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:23 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3565681602' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:24.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:23 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1475180828' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:51:24.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:23 smithi027 ceph-mon[97244]: pgmap v1102: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:24.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:23 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:51:24.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:23 smithi027 ceph-mon[97244]: from='client.35240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:51:24.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:23 smithi027 ceph-mon[97244]: from='client.35244 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:51:24.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:23 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3565681602' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:24.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:23 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1475180828' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:51:25.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:24 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2127.410517 secs 2021-11-03T17:51:25.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:24 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2127.410517 secs 2021-11-03T17:51:26.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:25 smithi115 ceph-mon[65453]: pgmap v1103: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:26.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:25 smithi027 ceph-mon[97244]: pgmap v1103: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:28.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:27 smithi115 ceph-mon[65453]: pgmap v1104: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:28.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:27 smithi027 ceph-mon[97244]: pgmap v1104: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:30.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:29 smithi115 ceph-mon[65453]: pgmap v1105: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:30.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:29 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2132.410657 secs 2021-11-03T17:51:30.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:29 smithi027 ceph-mon[97244]: pgmap v1105: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:30.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:29 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2132.410657 secs 2021-11-03T17:51:32.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:31 smithi115 ceph-mon[65453]: pgmap v1106: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:32.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:31 smithi027 ceph-mon[97244]: pgmap v1106: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:33.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:51:33.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:51:33.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:51:33.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:33.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:33.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:33.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:33.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:51:33.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:51:33.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:51:33.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:33.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:33.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:33.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:34.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:33 smithi115 ceph-mon[65453]: pgmap v1107: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:34.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:33 smithi115 ceph-mon[65453]: poll took 60159.962 ms: 1 events 2021-11-03T17:51:34.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:33 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:51:34.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:33 smithi027 ceph-mon[97244]: pgmap v1107: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:34.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:33 smithi027 ceph-mon[97244]: poll took 60159.962 ms: 1 events 2021-11-03T17:51:34.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:33 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:51:35.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:34 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2137.410782 secs 2021-11-03T17:51:35.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:34 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2137.410782 secs 2021-11-03T17:51:36.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:36 smithi027 ceph-mon[97244]: pgmap v1108: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:36.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:36 smithi115 ceph-mon[65453]: pgmap v1108: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:38.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:38 smithi027 ceph-mon[97244]: pgmap v1109: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:38.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:38 smithi115 ceph-mon[65453]: pgmap v1109: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:40.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:40 smithi027 ceph-mon[97244]: pgmap v1110: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:40.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2142.410899 secs 2021-11-03T17:51:40.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:40 smithi115 ceph-mon[65453]: pgmap v1110: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:40.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2142.410899 secs 2021-11-03T17:51:42.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:42 smithi027 ceph-mon[97244]: pgmap v1111: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:42.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:42 smithi115 ceph-mon[65453]: pgmap v1111: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:43.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:51:43.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:51:43.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:51:43.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:43.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:43.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:43.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:43.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:51:43.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:51:43.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:51:43.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:43.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:43.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:43.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:44.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:44 smithi115 ceph-mon[65453]: pgmap v1112: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:44.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:44 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:51:44.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:44 smithi027 ceph-mon[97244]: pgmap v1112: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:44.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:44 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:51:45.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2147.411037 secs 2021-11-03T17:51:45.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2147.411037 secs 2021-11-03T17:51:46.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:46 smithi027 ceph-mon[97244]: pgmap v1113: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:46.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:46 smithi115 ceph-mon[65453]: pgmap v1113: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:48.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:48 smithi115 ceph-mon[65453]: pgmap v1114: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:48.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:48 smithi027 ceph-mon[97244]: pgmap v1114: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:50.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:50 smithi115 ceph-mon[65453]: pgmap v1115: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:50.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2152.411177 secs 2021-11-03T17:51:50.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:50 smithi027 ceph-mon[97244]: pgmap v1115: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:50.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2152.411177 secs 2021-11-03T17:51:52.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:52 smithi115 ceph-mon[65453]: pgmap v1116: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:52.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:52 smithi027 ceph-mon[97244]: pgmap v1116: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:53.268 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:53 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:51:53.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:53 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:51:53.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:53 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:51:53.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:53 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:53.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:53 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:53.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:53 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:53.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:53 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:53.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:53 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:51:53.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:53 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:51:53.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:53 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:51:53.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:53 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:53.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:53 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:53.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:53 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:53.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:53 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:53.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.751+0000 7fa145e08700 1 -- 172.21.15.27:0/2620629639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1400f5ef0 msgr2=0x7fa1400f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:51:53.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.751+0000 7fa145e08700 1 --2- 172.21.15.27:0/2620629639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1400f5ef0 0x7fa1400f6920 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7fa134004660 tx=0x7fa134009b30).stop 2021-11-03T17:51:53.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.752+0000 7fa145e08700 1 -- 172.21.15.27:0/2620629639 shutdown_connections 2021-11-03T17:51:53.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.752+0000 7fa145e08700 1 --2- 172.21.15.27:0/2620629639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1400f5ef0 0x7fa1400f6920 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:53.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.752+0000 7fa145e08700 1 --2- 172.21.15.27:0/2620629639 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1400f55e0 0x7fa1400f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:53.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.752+0000 7fa145e08700 1 -- 172.21.15.27:0/2620629639 >> 172.21.15.27:0/2620629639 conn(0x7fa1400f0de0 msgr2=0x7fa1400f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:51:53.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.752+0000 7fa145e08700 1 -- 172.21.15.27:0/2620629639 shutdown_connections 2021-11-03T17:51:53.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.752+0000 7fa145e08700 1 -- 172.21.15.27:0/2620629639 wait complete. 2021-11-03T17:51:53.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.753+0000 7fa145e08700 1 Processor -- start 2021-11-03T17:51:53.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.753+0000 7fa145e08700 1 -- start start 2021-11-03T17:51:53.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.753+0000 7fa145e08700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1400f55e0 0x7fa140105e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:51:53.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.753+0000 7fa145e08700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1400f5ef0 0x7fa140106380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:51:53.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.753+0000 7fa145e08700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa140106930 con 0x7fa1400f55e0 2021-11-03T17:51:53.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.753+0000 7fa145e08700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa140106a70 con 0x7fa1400f5ef0 2021-11-03T17:51:53.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.753+0000 7fa13f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1400f55e0 0x7fa140105e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:51:53.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.753+0000 7fa13f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1400f55e0 0x7fa140105e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59816/0 (socket says 172.21.15.27:59816) 2021-11-03T17:51:53.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.753+0000 7fa13f7fe700 1 -- 172.21.15.27:0/422660618 learned_addr learned my addr 172.21.15.27:0/422660618 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:51:53.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.753+0000 7fa13effd700 1 --2- 172.21.15.27:0/422660618 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1400f5ef0 0x7fa140106380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:51:53.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.754+0000 7fa13f7fe700 1 -- 172.21.15.27:0/422660618 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1400f5ef0 msgr2=0x7fa140106380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:51:53.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.754+0000 7fa13f7fe700 1 --2- 172.21.15.27:0/422660618 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1400f5ef0 0x7fa140106380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:53.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.754+0000 7fa13f7fe700 1 -- 172.21.15.27:0/422660618 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa134005040 con 0x7fa1400f55e0 2021-11-03T17:51:53.754 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.754+0000 7fa13f7fe700 1 --2- 172.21.15.27:0/422660618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1400f55e0 0x7fa140105e90 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7fa13000c530 tx=0x7fa130007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:51:53.754 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.754+0000 7fa127fff700 1 -- 172.21.15.27:0/422660618 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa13000f650 con 0x7fa1400f55e0 2021-11-03T17:51:53.754 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.754+0000 7fa145e08700 1 -- 172.21.15.27:0/422660618 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa140107410 con 0x7fa1400f55e0 2021-11-03T17:51:53.755 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.754+0000 7fa127fff700 1 -- 172.21.15.27:0/422660618 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fa13000fc60 con 0x7fa1400f55e0 2021-11-03T17:51:53.755 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.754+0000 7fa127fff700 1 -- 172.21.15.27:0/422660618 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa130012c20 con 0x7fa1400f55e0 2021-11-03T17:51:53.755 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.754+0000 7fa145e08700 1 -- 172.21.15.27:0/422660618 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1401079d0 con 0x7fa1400f55e0 2021-11-03T17:51:53.756 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.755+0000 7fa145e08700 1 -- 172.21.15.27:0/422660618 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa140045ad0 con 0x7fa1400f55e0 2021-11-03T17:51:53.756 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.755+0000 7fa127fff700 1 -- 172.21.15.27:0/422660618 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fa130010070 con 0x7fa1400f55e0 2021-11-03T17:51:53.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.756+0000 7fa127fff700 1 --2- 172.21.15.27:0/422660618 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa1280640b0 0x7fa128066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:51:53.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.756+0000 7fa127fff700 1 -- 172.21.15.27:0/422660618 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fa1300806a0 con 0x7fa1400f55e0 2021-11-03T17:51:53.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.756+0000 7fa13effd700 1 --2- 172.21.15.27:0/422660618 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa1280640b0 0x7fa128066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:51:53.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.756+0000 7fa13effd700 1 --2- 172.21.15.27:0/422660618 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa1280640b0 0x7fa128066570 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7fa134004630 tx=0x7fa134009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:51:53.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.760+0000 7fa127fff700 1 -- 172.21.15.27:0/422660618 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fa1300528b0 con 0x7fa1400f55e0 2021-11-03T17:51:53.931 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.936+0000 7fa145e08700 1 -- 172.21.15.27:0/422660618 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa1400008d0 con 0x7fa1280640b0 2021-11-03T17:51:53.944 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.938+0000 7fa127fff700 1 -- 172.21.15.27:0/422660618 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fa1400008d0 con 0x7fa1280640b0 2021-11-03T17:51:53.944 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.939+0000 7fa145e08700 1 -- 172.21.15.27:0/422660618 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa1280640b0 msgr2=0x7fa128066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:51:53.945 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.939+0000 7fa145e08700 1 --2- 172.21.15.27:0/422660618 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa1280640b0 0x7fa128066570 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7fa134004630 tx=0x7fa134009b30).stop 2021-11-03T17:51:53.945 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.939+0000 7fa145e08700 1 -- 172.21.15.27:0/422660618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1400f55e0 msgr2=0x7fa140105e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:51:53.945 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.939+0000 7fa145e08700 1 --2- 172.21.15.27:0/422660618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1400f55e0 0x7fa140105e90 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7fa13000c530 tx=0x7fa130007e20).stop 2021-11-03T17:51:53.945 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.940+0000 7fa145e08700 1 -- 172.21.15.27:0/422660618 shutdown_connections 2021-11-03T17:51:53.946 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.940+0000 7fa145e08700 1 --2- 172.21.15.27:0/422660618 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa1280640b0 0x7fa128066570 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:53.946 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.940+0000 7fa145e08700 1 --2- 172.21.15.27:0/422660618 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1400f55e0 0x7fa140105e90 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:53.946 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.940+0000 7fa145e08700 1 --2- 172.21.15.27:0/422660618 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1400f5ef0 0x7fa140106380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:53.946 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.940+0000 7fa145e08700 1 -- 172.21.15.27:0/422660618 >> 172.21.15.27:0/422660618 conn(0x7fa1400f0de0 msgr2=0x7fa1400fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:51:53.947 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.940+0000 7fa145e08700 1 -- 172.21.15.27:0/422660618 shutdown_connections 2021-11-03T17:51:53.947 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:53.940+0000 7fa145e08700 1 -- 172.21.15.27:0/422660618 wait complete. 2021-11-03T17:51:53.947 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:51:54.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.040+0000 7f263481d700 1 -- 172.21.15.27:0/113255653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26300f66e0 msgr2=0x7f26300f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:51:54.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.040+0000 7f263481d700 1 --2- 172.21.15.27:0/113255653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26300f66e0 0x7f26300f6b00 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f2624004660 tx=0x7f2624009b30).stop 2021-11-03T17:51:54.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.040+0000 7f263481d700 1 -- 172.21.15.27:0/113255653 shutdown_connections 2021-11-03T17:51:54.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.040+0000 7f263481d700 1 --2- 172.21.15.27:0/113255653 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26300f6ff0 0x7f26300f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:51:54.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.040+0000 7f263481d700 1 --2- 172.21.15.27:0/113255653 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26300f66e0 0x7f26300f6b00 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:54.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.040+0000 7f263481d700 1 -- 172.21.15.27:0/113255653 >> 172.21.15.27:0/113255653 conn(0x7f26300f1f00 msgr2=0x7f26300f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:51:54.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.040+0000 7f263481d700 1 -- 172.21.15.27:0/113255653 shutdown_connections 2021-11-03T17:51:54.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.040+0000 7f263481d700 1 -- 172.21.15.27:0/113255653 wait complete. 2021-11-03T17:51:54.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.040+0000 7f263481d700 1 Processor -- start 2021-11-03T17:51:54.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.040+0000 7f263481d700 1 -- start start 2021-11-03T17:51:54.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.041+0000 7f263481d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26300f66e0 0x7f2630106f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:51:54.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.041+0000 7f263481d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26300f6ff0 0x7f2630107470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:51:54.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.041+0000 7f263481d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2630107ab0 con 0x7f26300f6ff0 2021-11-03T17:51:54.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.041+0000 7f263481d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2630108360 con 0x7f26300f66e0 2021-11-03T17:51:54.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.041+0000 7f262dd9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26300f6ff0 0x7f2630107470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:51:54.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.041+0000 7f262dd9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26300f6ff0 0x7f2630107470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59828/0 (socket says 172.21.15.27:59828) 2021-11-03T17:51:54.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.041+0000 7f262dd9b700 1 -- 172.21.15.27:0/2801578784 learned_addr learned my addr 172.21.15.27:0/2801578784 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:51:54.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.041+0000 7f262e59c700 1 --2- 172.21.15.27:0/2801578784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26300f66e0 0x7f2630106f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:51:54.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.041+0000 7f262dd9b700 1 -- 172.21.15.27:0/2801578784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26300f66e0 msgr2=0x7f2630106f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:51:54.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.041+0000 7f262dd9b700 1 --2- 172.21.15.27:0/2801578784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26300f66e0 0x7f2630106f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:54.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.041+0000 7f262dd9b700 1 -- 172.21.15.27:0/2801578784 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2624005040 con 0x7f26300f6ff0 2021-11-03T17:51:54.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.041+0000 7f262dd9b700 1 --2- 172.21.15.27:0/2801578784 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26300f6ff0 0x7f2630107470 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f262000b410 tx=0x7f2620006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:51:54.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.041+0000 7f261effd700 1 -- 172.21.15.27:0/2801578784 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26200125f0 con 0x7f26300f6ff0 2021-11-03T17:51:54.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.041+0000 7f261effd700 1 -- 172.21.15.27:0/2801578784 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f2620012c00 con 0x7f26300f6ff0 2021-11-03T17:51:54.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.041+0000 7f263481d700 1 -- 172.21.15.27:0/2801578784 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2630108640 con 0x7f26300f6ff0 2021-11-03T17:51:54.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.041+0000 7f261effd700 1 -- 172.21.15.27:0/2801578784 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2620002ae0 con 0x7f26300f6ff0 2021-11-03T17:51:54.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.041+0000 7f263481d700 1 -- 172.21.15.27:0/2801578784 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2630108c00 con 0x7f26300f6ff0 2021-11-03T17:51:54.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.042+0000 7f263481d700 1 -- 172.21.15.27:0/2801578784 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26300f5b70 con 0x7f26300f6ff0 2021-11-03T17:51:54.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.043+0000 7f261effd700 1 -- 172.21.15.27:0/2801578784 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f262000f030 con 0x7f26300f6ff0 2021-11-03T17:51:54.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.043+0000 7f261effd700 1 --2- 172.21.15.27:0/2801578784 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f26180640b0 0x7f2618066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:51:54.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.043+0000 7f261effd700 1 -- 172.21.15.27:0/2801578784 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f2620080ce0 con 0x7f26300f6ff0 2021-11-03T17:51:54.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.043+0000 7f262e59c700 1 --2- 172.21.15.27:0/2801578784 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f26180640b0 0x7f2618066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:51:54.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.044+0000 7f262e59c700 1 --2- 172.21.15.27:0/2801578784 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f26180640b0 0x7f2618066570 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f2624004e30 tx=0x7f2624009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:51:54.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.047+0000 7f261effd700 1 -- 172.21.15.27:0/2801578784 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f2620052fe0 con 0x7f26300f6ff0 2021-11-03T17:51:54.215 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.220+0000 7f263481d700 1 -- 172.21.15.27:0/2801578784 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f26300fefc0 con 0x7f26180640b0 2021-11-03T17:51:54.230 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.235+0000 7f261effd700 1 -- 172.21.15.27:0/2801578784 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f26300fefc0 con 0x7f26180640b0 2021-11-03T17:51:54.230 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:51:54.230 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (43m) 3m ago 44m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:51:54.231 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (35m) 3m ago 44m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:51:54.231 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (34m) 2m ago 43m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:51:54.231 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (43m) 3m ago 44m 41.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:51:54.231 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (36m) 3m ago 39m 23.8M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:51:54.232 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (39m) 3m ago 39m 3844M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:51:54.232 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (35m) 2m ago 39m 23.2M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:51:54.232 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (37m) 2m ago 39m 24.9M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:51:54.232 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (37m) 3m ago 46m 535M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:51:54.232 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (36m) 2m ago 43m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:51:54.233 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (36m) 3m ago 46m 198M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:51:54.233 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (35m) 2m ago 43m 95.6M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:51:54.233 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (44m) 3m ago 44m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:51:54.233 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (43m) 2m ago 43m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:51:54.233 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (34m) 3m ago 43m 331M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:51:54.234 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (34m) 3m ago 42m 361M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:51:54.234 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (33m) 3m ago 42m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:51:54.234 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (33m) 2m ago 42m 379M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:51:54.234 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (32m) 2m ago 41m 304M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:51:54.235 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (32m) 2m ago 41m 307M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:51:54.235 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (43m) 3m ago 44m 52.9M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:51:54.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:54 smithi115 ceph-mon[65453]: pgmap v1117: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:54.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:54 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:51:54.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.237+0000 7f263481d700 1 -- 172.21.15.27:0/2801578784 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f26180640b0 msgr2=0x7f2618066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:51:54.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.237+0000 7f263481d700 1 --2- 172.21.15.27:0/2801578784 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f26180640b0 0x7f2618066570 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7f2624004e30 tx=0x7f2624009b30).stop 2021-11-03T17:51:54.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.237+0000 7f263481d700 1 -- 172.21.15.27:0/2801578784 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26300f6ff0 msgr2=0x7f2630107470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:51:54.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.237+0000 7f263481d700 1 --2- 172.21.15.27:0/2801578784 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26300f6ff0 0x7f2630107470 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f262000b410 tx=0x7f2620006ef0).stop 2021-11-03T17:51:54.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.237+0000 7f263481d700 1 -- 172.21.15.27:0/2801578784 shutdown_connections 2021-11-03T17:51:54.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.237+0000 7f263481d700 1 --2- 172.21.15.27:0/2801578784 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f26180640b0 0x7f2618066570 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:54.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.237+0000 7f263481d700 1 --2- 172.21.15.27:0/2801578784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26300f66e0 0x7f2630106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:54.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.237+0000 7f263481d700 1 --2- 172.21.15.27:0/2801578784 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26300f6ff0 0x7f2630107470 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:54.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.237+0000 7f263481d700 1 -- 172.21.15.27:0/2801578784 >> 172.21.15.27:0/2801578784 conn(0x7f26300f1f00 msgr2=0x7f26300fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:51:54.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.237+0000 7f263481d700 1 -- 172.21.15.27:0/2801578784 shutdown_connections 2021-11-03T17:51:54.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.237+0000 7f263481d700 1 -- 172.21.15.27:0/2801578784 wait complete. 2021-11-03T17:51:54.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.336+0000 7f2968f9e700 1 -- 172.21.15.27:0/3350974287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29640f5ef0 msgr2=0x7f29640f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:51:54.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.336+0000 7f2968f9e700 1 --2- 172.21.15.27:0/3350974287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29640f5ef0 0x7f29640f6920 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7f294c004660 tx=0x7f294c009b30).stop 2021-11-03T17:51:54.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.336+0000 7f2968f9e700 1 -- 172.21.15.27:0/3350974287 shutdown_connections 2021-11-03T17:51:54.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.336+0000 7f2968f9e700 1 --2- 172.21.15.27:0/3350974287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29640f5ef0 0x7f29640f6920 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:54.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.336+0000 7f2968f9e700 1 --2- 172.21.15.27:0/3350974287 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29640f55e0 0x7f29640f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:54.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.336+0000 7f2968f9e700 1 -- 172.21.15.27:0/3350974287 >> 172.21.15.27:0/3350974287 conn(0x7f29640f0de0 msgr2=0x7f29640f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:51:54.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.336+0000 7f2968f9e700 1 -- 172.21.15.27:0/3350974287 shutdown_connections 2021-11-03T17:51:54.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.336+0000 7f2968f9e700 1 -- 172.21.15.27:0/3350974287 wait complete. 2021-11-03T17:51:54.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.336+0000 7f2968f9e700 1 Processor -- start 2021-11-03T17:51:54.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.337+0000 7f2968f9e700 1 -- start start 2021-11-03T17:51:54.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.337+0000 7f2968f9e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29640f55e0 0x7f2964109c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:51:54.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.337+0000 7f2968f9e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29640f5ef0 0x7f296410c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:51:54.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.337+0000 7f2968f9e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2964060540 con 0x7f29640f55e0 2021-11-03T17:51:54.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.337+0000 7f2968f9e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2964060680 con 0x7f29640f5ef0 2021-11-03T17:51:54.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.337+0000 7f296259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29640f55e0 0x7f2964109c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:51:54.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.337+0000 7f296259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29640f55e0 0x7f2964109c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59836/0 (socket says 172.21.15.27:59836) 2021-11-03T17:51:54.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.337+0000 7f296259c700 1 -- 172.21.15.27:0/991765744 learned_addr learned my addr 172.21.15.27:0/991765744 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:51:54.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.337+0000 7f2961d9b700 1 --2- 172.21.15.27:0/991765744 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29640f5ef0 0x7f296410c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:51:54.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.337+0000 7f296259c700 1 -- 172.21.15.27:0/991765744 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29640f5ef0 msgr2=0x7f296410c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:51:54.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.337+0000 7f296259c700 1 --2- 172.21.15.27:0/991765744 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29640f5ef0 0x7f296410c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:54.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.337+0000 7f296259c700 1 -- 172.21.15.27:0/991765744 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f294c005040 con 0x7f29640f55e0 2021-11-03T17:51:54.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.337+0000 7f2961d9b700 1 --2- 172.21.15.27:0/991765744 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29640f5ef0 0x7f296410c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:51:54.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.337+0000 7f296259c700 1 --2- 172.21.15.27:0/991765744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29640f55e0 0x7f2964109c20 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7f295c00c530 tx=0x7f295c007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:51:54.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.337+0000 7f295affd700 1 -- 172.21.15.27:0/991765744 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f295c00f650 con 0x7f29640f55e0 2021-11-03T17:51:54.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.337+0000 7f2968f9e700 1 -- 172.21.15.27:0/991765744 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f296410c730 con 0x7f29640f55e0 2021-11-03T17:51:54.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.337+0000 7f295affd700 1 -- 172.21.15.27:0/991765744 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f295c00fc60 con 0x7f29640f55e0 2021-11-03T17:51:54.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.337+0000 7f295affd700 1 -- 172.21.15.27:0/991765744 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f295c012cd0 con 0x7f29640f55e0 2021-11-03T17:51:54.342 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.337+0000 7f2968f9e700 1 -- 172.21.15.27:0/991765744 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f296410ccc0 con 0x7f29640f55e0 2021-11-03T17:51:54.342 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.339+0000 7f2968f9e700 1 -- 172.21.15.27:0/991765744 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29640f4a70 con 0x7f29640f55e0 2021-11-03T17:51:54.342 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.339+0000 7f295affd700 1 -- 172.21.15.27:0/991765744 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f295c010070 con 0x7f29640f55e0 2021-11-03T17:51:54.342 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.339+0000 7f295affd700 1 --2- 172.21.15.27:0/991765744 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f29500640b0 0x7f2950066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:51:54.343 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.339+0000 7f295affd700 1 -- 172.21.15.27:0/991765744 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f295c080670 con 0x7f29640f55e0 2021-11-03T17:51:54.343 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.339+0000 7f2961d9b700 1 --2- 172.21.15.27:0/991765744 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f29500640b0 0x7f2950066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:51:54.343 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.340+0000 7f2961d9b700 1 --2- 172.21.15.27:0/991765744 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f29500640b0 0x7f2950066570 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f294c004e30 tx=0x7f294c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:51:54.344 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.343+0000 7f295affd700 1 -- 172.21.15.27:0/991765744 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f295c052880 con 0x7f29640f55e0 2021-11-03T17:51:54.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:54 smithi027 ceph-mon[97244]: pgmap v1117: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:54.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:54 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:51:54.568 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.573+0000 7f2968f9e700 1 -- 172.21.15.27:0/991765744 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2964027a00 con 0x7f29640f55e0 2021-11-03T17:51:54.569 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.574+0000 7f295affd700 1 -- 172.21.15.27:0/991765744 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f295c052470 con 0x7f29640f55e0 2021-11-03T17:51:54.570 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:51:54.570 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:51:54.570 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:51:54.571 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:51:54.571 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:51:54.571 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:51:54.571 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:51:54.572 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:51:54.572 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:51:54.572 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:51:54.572 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:51:54.572 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:51:54.573 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:51:54.573 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:51:54.573 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:51:54.573 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:51:54.574 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:51:54.574 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:51:54.575 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.576+0000 7f2968f9e700 1 -- 172.21.15.27:0/991765744 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f29500640b0 msgr2=0x7f2950066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:51:54.575 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.576+0000 7f2968f9e700 1 --2- 172.21.15.27:0/991765744 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f29500640b0 0x7f2950066570 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 rx=0x7f294c004e30 tx=0x7f294c009b30).stop 2021-11-03T17:51:54.575 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.576+0000 7f2968f9e700 1 -- 172.21.15.27:0/991765744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29640f55e0 msgr2=0x7f2964109c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:51:54.576 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.576+0000 7f2968f9e700 1 --2- 172.21.15.27:0/991765744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29640f55e0 0x7f2964109c20 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7f295c00c530 tx=0x7f295c007e20).stop 2021-11-03T17:51:54.576 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.576+0000 7f2968f9e700 1 -- 172.21.15.27:0/991765744 shutdown_connections 2021-11-03T17:51:54.576 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.576+0000 7f2968f9e700 1 --2- 172.21.15.27:0/991765744 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f29500640b0 0x7f2950066570 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:54.576 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.576+0000 7f2968f9e700 1 --2- 172.21.15.27:0/991765744 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29640f55e0 0x7f2964109c20 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:54.576 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.576+0000 7f2968f9e700 1 --2- 172.21.15.27:0/991765744 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29640f5ef0 0x7f296410c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:54.577 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.576+0000 7f2968f9e700 1 -- 172.21.15.27:0/991765744 >> 172.21.15.27:0/991765744 conn(0x7f29640f0de0 msgr2=0x7f29640fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:51:54.577 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.576+0000 7f2968f9e700 1 -- 172.21.15.27:0/991765744 shutdown_connections 2021-11-03T17:51:54.577 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.576+0000 7f2968f9e700 1 -- 172.21.15.27:0/991765744 wait complete. 2021-11-03T17:51:54.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.676+0000 7f720bc9a700 1 -- 172.21.15.27:0/4059055194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72040f55d0 msgr2=0x7f72040f59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:51:54.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.676+0000 7f720bc9a700 1 --2- 172.21.15.27:0/4059055194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72040f55d0 0x7f72040f59f0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7f71f4004660 tx=0x7f71f4009b30).stop 2021-11-03T17:51:54.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.676+0000 7f720bc9a700 1 -- 172.21.15.27:0/4059055194 shutdown_connections 2021-11-03T17:51:54.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.676+0000 7f720bc9a700 1 --2- 172.21.15.27:0/4059055194 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72040f6870 0x7f72040f6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:51:54.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.676+0000 7f720bc9a700 1 --2- 172.21.15.27:0/4059055194 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72040f55d0 0x7f72040f59f0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:54.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.676+0000 7f720bc9a700 1 -- 172.21.15.27:0/4059055194 >> 172.21.15.27:0/4059055194 conn(0x7f72040f0d70 msgr2=0x7f72040f3190 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:51:54.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.676+0000 7f720bc9a700 1 -- 172.21.15.27:0/4059055194 shutdown_connections 2021-11-03T17:51:54.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.676+0000 7f720bc9a700 1 -- 172.21.15.27:0/4059055194 wait complete. 2021-11-03T17:51:54.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.676+0000 7f720bc9a700 1 Processor -- start 2021-11-03T17:51:54.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.676+0000 7f720bc9a700 1 -- start start 2021-11-03T17:51:54.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.677+0000 7f720bc9a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72040f55d0 0x7f7204106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:51:54.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.677+0000 7f720bc9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72040f6870 0x7f7204107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:51:54.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.677+0000 7f720bc9a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7204107aa0 con 0x7f72040f6870 2021-11-03T17:51:54.676 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.677+0000 7f720bc9a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7204108350 con 0x7f72040f55d0 2021-11-03T17:51:54.676 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.677+0000 7f7209235700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72040f6870 0x7f7204107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:51:54.676 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.677+0000 7f7209235700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72040f6870 0x7f7204107460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59846/0 (socket says 172.21.15.27:59846) 2021-11-03T17:51:54.676 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.677+0000 7f7209235700 1 -- 172.21.15.27:0/1034069065 learned_addr learned my addr 172.21.15.27:0/1034069065 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:51:54.676 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.677+0000 7f7209a36700 1 --2- 172.21.15.27:0/1034069065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72040f55d0 0x7f7204106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:51:54.677 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.677+0000 7f7209235700 1 -- 172.21.15.27:0/1034069065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72040f55d0 msgr2=0x7f7204106f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:51:54.677 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.677+0000 7f7209235700 1 --2- 172.21.15.27:0/1034069065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72040f55d0 0x7f7204106f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:54.677 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.677+0000 7f7209235700 1 -- 172.21.15.27:0/1034069065 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f4005040 con 0x7f72040f6870 2021-11-03T17:51:54.678 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.677+0000 7f7209a36700 1 --2- 172.21.15.27:0/1034069065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72040f55d0 0x7f7204106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:51:54.678 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.677+0000 7f7209235700 1 --2- 172.21.15.27:0/1034069065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72040f6870 0x7f7204107460 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f720000b410 tx=0x7f7200006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:51:54.678 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.677+0000 7f71fa7fc700 1 -- 172.21.15.27:0/1034069065 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f72000125f0 con 0x7f72040f6870 2021-11-03T17:51:54.678 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.677+0000 7f720bc9a700 1 -- 172.21.15.27:0/1034069065 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7204108630 con 0x7f72040f6870 2021-11-03T17:51:54.679 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.677+0000 7f71fa7fc700 1 -- 172.21.15.27:0/1034069065 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f7200012c00 con 0x7f72040f6870 2021-11-03T17:51:54.679 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.677+0000 7f71fa7fc700 1 -- 172.21.15.27:0/1034069065 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7200002ae0 con 0x7f72040f6870 2021-11-03T17:51:54.679 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.677+0000 7f720bc9a700 1 -- 172.21.15.27:0/1034069065 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7204108bf0 con 0x7f72040f6870 2021-11-03T17:51:54.680 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.678+0000 7f720bc9a700 1 -- 172.21.15.27:0/1034069065 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72040f49c0 con 0x7f72040f6870 2021-11-03T17:51:54.681 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.679+0000 7f71fa7fc700 1 -- 172.21.15.27:0/1034069065 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f720000f030 con 0x7f72040f6870 2021-11-03T17:51:54.681 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.679+0000 7f71fa7fc700 1 --2- 172.21.15.27:0/1034069065 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f71f00640b0 0x7f71f0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:51:54.681 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.679+0000 7f71fa7fc700 1 -- 172.21.15.27:0/1034069065 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f7200080d10 con 0x7f72040f6870 2021-11-03T17:51:54.682 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.679+0000 7f7209a36700 1 --2- 172.21.15.27:0/1034069065 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f71f00640b0 0x7f71f0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:51:54.682 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.680+0000 7f7209a36700 1 --2- 172.21.15.27:0/1034069065 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f71f00640b0 0x7f71f0066570 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f71f4004e30 tx=0x7f71f4009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:51:54.682 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.683+0000 7f71fa7fc700 1 -- 172.21.15.27:0/1034069065 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f7200053010 con 0x7f72040f6870 2021-11-03T17:51:54.883 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.888+0000 7f720bc9a700 1 -- 172.21.15.27:0/1034069065 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f720410d590 con 0x7f72040f6870 2021-11-03T17:51:54.884 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.889+0000 7f71fa7fc700 1 -- 172.21.15.27:0/1034069065 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f7200052c00 con 0x7f72040f6870 2021-11-03T17:51:54.884 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:51:54.884 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:51:54.884 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:51:54.885 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:51:54.885 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:51:54.885 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:51:54.885 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:51:54.886 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:51:54.886 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:51:54.886 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:51:54.886 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:51:54.887 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:51:54.887 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:51:54.887 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:51:54.887 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:51:54.888 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:51:54.888 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:51:54.888 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:51:54.888 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:51:54.889 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:51:54.889 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:51:54.889 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:51:54.890 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:51:54.890 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:51:54.890 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:51:54.890 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:51:54.891 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:51:54.891 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:51:54.891 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:51:54.891 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:51:54.892 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:51:54.892 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:51:54.892 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:51:54.892 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:51:54.892 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:51:54.893 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:51:54.893 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:51:54.893 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:51:54.893 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:51:54.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.890+0000 7f720bc9a700 1 -- 172.21.15.27:0/1034069065 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f71f00640b0 msgr2=0x7f71f0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:51:54.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.890+0000 7f720bc9a700 1 --2- 172.21.15.27:0/1034069065 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f71f00640b0 0x7f71f0066570 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f71f4004e30 tx=0x7f71f4009b30).stop 2021-11-03T17:51:54.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.890+0000 7f720bc9a700 1 -- 172.21.15.27:0/1034069065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72040f6870 msgr2=0x7f7204107460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:51:54.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.890+0000 7f720bc9a700 1 --2- 172.21.15.27:0/1034069065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72040f6870 0x7f7204107460 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f720000b410 tx=0x7f7200006ef0).stop 2021-11-03T17:51:54.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.891+0000 7f720bc9a700 1 -- 172.21.15.27:0/1034069065 shutdown_connections 2021-11-03T17:51:54.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.891+0000 7f720bc9a700 1 --2- 172.21.15.27:0/1034069065 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f71f00640b0 0x7f71f0066570 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:54.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.891+0000 7f720bc9a700 1 --2- 172.21.15.27:0/1034069065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f72040f55d0 0x7f7204106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:54.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.891+0000 7f720bc9a700 1 --2- 172.21.15.27:0/1034069065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f72040f6870 0x7f7204107460 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:51:54.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.891+0000 7f720bc9a700 1 -- 172.21.15.27:0/1034069065 >> 172.21.15.27:0/1034069065 conn(0x7f72040f0d70 msgr2=0x7f720405f760 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:51:54.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.891+0000 7f720bc9a700 1 -- 172.21.15.27:0/1034069065 shutdown_connections 2021-11-03T17:51:54.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:51:54.891+0000 7f720bc9a700 1 -- 172.21.15.27:0/1034069065 wait complete. 2021-11-03T17:51:54.897 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:51:55.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:55 smithi115 ceph-mon[65453]: from='client.35256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:51:55.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:55 smithi115 ceph-mon[65453]: from='client.35260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:51:55.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2157.411245 secs 2021-11-03T17:51:55.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:55 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/991765744' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:55.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:55 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1034069065' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:51:55.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:55 smithi027 ceph-mon[97244]: from='client.35256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:51:55.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:55 smithi027 ceph-mon[97244]: from='client.35260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:51:55.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2157.411245 secs 2021-11-03T17:51:55.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:55 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/991765744' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:51:55.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:55 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1034069065' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:51:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:56 smithi115 ceph-mon[65453]: pgmap v1118: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:56.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:51:56.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:51:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:56 smithi027 ceph-mon[97244]: pgmap v1118: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:56.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:51:56.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:51:58.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:51:58 smithi115 ceph-mon[65453]: pgmap v1119: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:51:58.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:51:58 smithi027 ceph-mon[97244]: pgmap v1119: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:00.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:00 smithi115 ceph-mon[65453]: pgmap v1120: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:00.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2162.411374 secs 2021-11-03T17:52:00.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:00 smithi027 ceph-mon[97244]: pgmap v1120: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:00.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2162.411374 secs 2021-11-03T17:52:02.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:02 smithi115 ceph-mon[65453]: pgmap v1121: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:02.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:02 smithi027 ceph-mon[97244]: pgmap v1121: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:03.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:52:03.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:52:03.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:52:03.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:03.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:03.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:03.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:03.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:52:03.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:52:03.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:52:03.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:03.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:03.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:03.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:04.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:04 smithi115 ceph-mon[65453]: pgmap v1122: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:04.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:04 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:52:04.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:04 smithi027 ceph-mon[97244]: pgmap v1122: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:04.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:04 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:52:05.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2167.411496 secs 2021-11-03T17:52:05.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2167.411496 secs 2021-11-03T17:52:06.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:06 smithi115 ceph-mon[65453]: pgmap v1123: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:06.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:06 smithi027 ceph-mon[97244]: pgmap v1123: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:08.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:08 smithi115 ceph-mon[65453]: pgmap v1124: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:08.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:08 smithi027 ceph-mon[97244]: pgmap v1124: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:10.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:10 smithi115 ceph-mon[65453]: pgmap v1125: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:10.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2172.411599 secs 2021-11-03T17:52:10.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:10 smithi027 ceph-mon[97244]: pgmap v1125: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:10.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2172.411599 secs 2021-11-03T17:52:12.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:12 smithi115 ceph-mon[65453]: pgmap v1126: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:12.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:12 smithi027 ceph-mon[97244]: pgmap v1126: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:13.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:52:13.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:52:13.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:52:13.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:13.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:13.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:13.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:13.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:52:13.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:52:13.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:52:13.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:13.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:13.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:13.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:14.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:14 smithi115 ceph-mon[65453]: pgmap v1127: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:14.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:14 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:52:14.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:14 smithi027 ceph-mon[97244]: pgmap v1127: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:14.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:14 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:52:15.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2177.411755 secs 2021-11-03T17:52:15.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2177.411755 secs 2021-11-03T17:52:16.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:16 smithi115 ceph-mon[65453]: pgmap v1128: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:16.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:16 smithi027 ceph-mon[97244]: pgmap v1128: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:18.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:18 smithi115 ceph-mon[65453]: pgmap v1129: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:18.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:18 smithi027 ceph-mon[97244]: pgmap v1129: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:20.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:20 smithi115 ceph-mon[65453]: pgmap v1130: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:20.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2182.411896 secs 2021-11-03T17:52:20.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:20 smithi027 ceph-mon[97244]: pgmap v1130: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:20.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2182.411896 secs 2021-11-03T17:52:22.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:22 smithi115 ceph-mon[65453]: pgmap v1131: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:22.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:22 smithi027 ceph-mon[97244]: pgmap v1131: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:23.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:52:23.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:52:23.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:52:23.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:23.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:23.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:23.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:23.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:52:23.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:52:23.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:52:23.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:23.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:23.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:23.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:24.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:24 smithi115 ceph-mon[65453]: pgmap v1132: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:24.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:24 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:52:24.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:24 smithi027 ceph-mon[97244]: pgmap v1132: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:24.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:24 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:52:25.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.005+0000 7fb44909f700 1 -- 172.21.15.27:0/4282181001 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4440f4800 msgr2=0x7fb4440f4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:52:25.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.005+0000 7fb44909f700 1 --2- 172.21.15.27:0/4282181001 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4440f4800 0x7fb4440f4c20 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7fb434004660 tx=0x7fb434009b30).stop 2021-11-03T17:52:25.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.005+0000 7fb44909f700 1 -- 172.21.15.27:0/4282181001 shutdown_connections 2021-11-03T17:52:25.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.005+0000 7fb44909f700 1 --2- 172.21.15.27:0/4282181001 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4440f51a0 0x7fb4440f7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:25.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.005+0000 7fb44909f700 1 --2- 172.21.15.27:0/4282181001 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4440f4800 0x7fb4440f4c20 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:25.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.005+0000 7fb44909f700 1 -- 172.21.15.27:0/4282181001 >> 172.21.15.27:0/4282181001 conn(0x7fb4440f0320 msgr2=0x7fb4440f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:52:25.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.005+0000 7fb44909f700 1 -- 172.21.15.27:0/4282181001 shutdown_connections 2021-11-03T17:52:25.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.005+0000 7fb44909f700 1 -- 172.21.15.27:0/4282181001 wait complete. 2021-11-03T17:52:25.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.005+0000 7fb44909f700 1 Processor -- start 2021-11-03T17:52:25.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.005+0000 7fb44909f700 1 -- start start 2021-11-03T17:52:25.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.006+0000 7fb44909f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4440f4800 0x7fb444106dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:52:25.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.006+0000 7fb44909f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4440f51a0 0x7fb4441072b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:52:25.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.006+0000 7fb44909f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4441078b0 con 0x7fb4440f51a0 2021-11-03T17:52:25.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.006+0000 7fb44909f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4441080f0 con 0x7fb4440f4800 2021-11-03T17:52:25.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.006+0000 7fb4437fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4440f51a0 0x7fb4441072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:52:25.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.006+0000 7fb4437fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4440f51a0 0x7fb4441072b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59864/0 (socket says 172.21.15.27:59864) 2021-11-03T17:52:25.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.006+0000 7fb4437fe700 1 -- 172.21.15.27:0/1025147809 learned_addr learned my addr 172.21.15.27:0/1025147809 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:52:25.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.006+0000 7fb443fff700 1 --2- 172.21.15.27:0/1025147809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4440f4800 0x7fb444106dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:52:25.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.006+0000 7fb4437fe700 1 -- 172.21.15.27:0/1025147809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4440f4800 msgr2=0x7fb444106dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:52:25.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.006+0000 7fb4437fe700 1 --2- 172.21.15.27:0/1025147809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4440f4800 0x7fb444106dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:25.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.006+0000 7fb4437fe700 1 -- 172.21.15.27:0/1025147809 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb434005040 con 0x7fb4440f51a0 2021-11-03T17:52:25.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.006+0000 7fb4437fe700 1 --2- 172.21.15.27:0/1025147809 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4440f51a0 0x7fb4441072b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7fb43800c510 tx=0x7fb438007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:52:25.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.006+0000 7fb440ff9700 1 -- 172.21.15.27:0/1025147809 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb43800f650 con 0x7fb4440f51a0 2021-11-03T17:52:25.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.006+0000 7fb440ff9700 1 -- 172.21.15.27:0/1025147809 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fb43800fc60 con 0x7fb4440f51a0 2021-11-03T17:52:25.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.006+0000 7fb44909f700 1 -- 172.21.15.27:0/1025147809 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4441083d0 con 0x7fb4440f51a0 2021-11-03T17:52:25.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.006+0000 7fb440ff9700 1 -- 172.21.15.27:0/1025147809 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb438012c20 con 0x7fb4440f51a0 2021-11-03T17:52:25.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.006+0000 7fb44909f700 1 -- 172.21.15.27:0/1025147809 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb444108910 con 0x7fb4440f51a0 2021-11-03T17:52:25.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.007+0000 7fb44909f700 1 -- 172.21.15.27:0/1025147809 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb444059100 con 0x7fb4440f51a0 2021-11-03T17:52:25.009 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.008+0000 7fb440ff9700 1 -- 172.21.15.27:0/1025147809 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fb438010070 con 0x7fb4440f51a0 2021-11-03T17:52:25.009 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.008+0000 7fb440ff9700 1 --2- 172.21.15.27:0/1025147809 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb42c064100 0x7fb42c0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:52:25.009 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.008+0000 7fb440ff9700 1 -- 172.21.15.27:0/1025147809 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fb438080590 con 0x7fb4440f51a0 2021-11-03T17:52:25.009 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.008+0000 7fb443fff700 1 --2- 172.21.15.27:0/1025147809 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb42c064100 0x7fb42c0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:52:25.010 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.009+0000 7fb443fff700 1 --2- 172.21.15.27:0/1025147809 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb42c064100 0x7fb42c0665c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7fb434013690 tx=0x7fb434009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:52:25.010 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.012+0000 7fb440ff9700 1 -- 172.21.15.27:0/1025147809 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fb4380527a0 con 0x7fb4440f51a0 2021-11-03T17:52:25.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.195+0000 7fb44909f700 1 -- 172.21.15.27:0/1025147809 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb4440feed0 con 0x7fb42c064100 2021-11-03T17:52:25.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.197+0000 7fb440ff9700 1 -- 172.21.15.27:0/1025147809 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fb4440feed0 con 0x7fb42c064100 2021-11-03T17:52:25.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.198+0000 7fb44909f700 1 -- 172.21.15.27:0/1025147809 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb42c064100 msgr2=0x7fb42c0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:52:25.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.198+0000 7fb44909f700 1 --2- 172.21.15.27:0/1025147809 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb42c064100 0x7fb42c0665c0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7fb434013690 tx=0x7fb434009b30).stop 2021-11-03T17:52:25.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.198+0000 7fb44909f700 1 -- 172.21.15.27:0/1025147809 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4440f51a0 msgr2=0x7fb4441072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:52:25.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.198+0000 7fb44909f700 1 --2- 172.21.15.27:0/1025147809 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4440f51a0 0x7fb4441072b0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7fb43800c510 tx=0x7fb438007df0).stop 2021-11-03T17:52:25.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.199+0000 7fb44909f700 1 -- 172.21.15.27:0/1025147809 shutdown_connections 2021-11-03T17:52:25.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.199+0000 7fb44909f700 1 --2- 172.21.15.27:0/1025147809 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb42c064100 0x7fb42c0665c0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:25.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.199+0000 7fb44909f700 1 --2- 172.21.15.27:0/1025147809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4440f4800 0x7fb444106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:25.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.199+0000 7fb44909f700 1 --2- 172.21.15.27:0/1025147809 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4440f51a0 0x7fb4441072b0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:25.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.199+0000 7fb44909f700 1 -- 172.21.15.27:0/1025147809 >> 172.21.15.27:0/1025147809 conn(0x7fb4440f0320 msgr2=0x7fb4440fb370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:52:25.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.199+0000 7fb44909f700 1 -- 172.21.15.27:0/1025147809 shutdown_connections 2021-11-03T17:52:25.196 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.199+0000 7fb44909f700 1 -- 172.21.15.27:0/1025147809 wait complete. 2021-11-03T17:52:25.203 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:52:25.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.299+0000 7faba3cc1700 1 -- 172.21.15.27:0/1553279315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab9c0f49a0 msgr2=0x7fab9c0f4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:52:25.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.299+0000 7faba3cc1700 1 --2- 172.21.15.27:0/1553279315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab9c0f49a0 0x7fab9c0f4dc0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7fab940099e0 tx=0x7fab94004d10).stop 2021-11-03T17:52:25.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.299+0000 7faba3cc1700 1 -- 172.21.15.27:0/1553279315 shutdown_connections 2021-11-03T17:52:25.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.299+0000 7faba3cc1700 1 --2- 172.21.15.27:0/1553279315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab9c0f5340 0x7fab9c0f7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:25.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.299+0000 7faba3cc1700 1 --2- 172.21.15.27:0/1553279315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab9c0f49a0 0x7fab9c0f4dc0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:25.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.299+0000 7faba3cc1700 1 -- 172.21.15.27:0/1553279315 >> 172.21.15.27:0/1553279315 conn(0x7fab9c0f04c0 msgr2=0x7fab9c0f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:52:25.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.299+0000 7faba3cc1700 1 -- 172.21.15.27:0/1553279315 shutdown_connections 2021-11-03T17:52:25.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.299+0000 7faba3cc1700 1 -- 172.21.15.27:0/1553279315 wait complete. 2021-11-03T17:52:25.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.299+0000 7faba3cc1700 1 Processor -- start 2021-11-03T17:52:25.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.299+0000 7faba3cc1700 1 -- start start 2021-11-03T17:52:25.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.299+0000 7faba3cc1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab9c0f49a0 0x7fab9c0f7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:52:25.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.299+0000 7faba3cc1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab9c0f5340 0x7fab9c0f59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:52:25.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.299+0000 7faba3cc1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c0f5f30 con 0x7fab9c0f49a0 2021-11-03T17:52:25.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.299+0000 7faba3cc1700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab9c0f6070 con 0x7fab9c0f5340 2021-11-03T17:52:25.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.299+0000 7faba1a5d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab9c0f49a0 0x7fab9c0f7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:52:25.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.300+0000 7faba1a5d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab9c0f49a0 0x7fab9c0f7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59872/0 (socket says 172.21.15.27:59872) 2021-11-03T17:52:25.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.300+0000 7faba1a5d700 1 -- 172.21.15.27:0/3727652755 learned_addr learned my addr 172.21.15.27:0/3727652755 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:52:25.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.300+0000 7faba125c700 1 --2- 172.21.15.27:0/3727652755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab9c0f5340 0x7fab9c0f59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:52:25.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.300+0000 7faba1a5d700 1 -- 172.21.15.27:0/3727652755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab9c0f5340 msgr2=0x7fab9c0f59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:52:25.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.300+0000 7faba1a5d700 1 --2- 172.21.15.27:0/3727652755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab9c0f5340 0x7fab9c0f59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:25.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.300+0000 7faba1a5d700 1 -- 172.21.15.27:0/3727652755 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab94009640 con 0x7fab9c0f49a0 2021-11-03T17:52:25.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.300+0000 7faba125c700 1 --2- 172.21.15.27:0/3727652755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab9c0f5340 0x7fab9c0f59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:52:25.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.300+0000 7faba1a5d700 1 --2- 172.21.15.27:0/3727652755 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab9c0f49a0 0x7fab9c0f7550 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7fab9400c010 tx=0x7fab94004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:52:25.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.300+0000 7fab9a7fc700 1 -- 172.21.15.27:0/3727652755 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab9400e070 con 0x7fab9c0f49a0 2021-11-03T17:52:25.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.300+0000 7fab9a7fc700 1 -- 172.21.15.27:0/3727652755 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fab94007dc0 con 0x7fab9c0f49a0 2021-11-03T17:52:25.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.300+0000 7fab9a7fc700 1 -- 172.21.15.27:0/3727652755 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab94018070 con 0x7fab9c0f49a0 2021-11-03T17:52:25.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.300+0000 7faba3cc1700 1 -- 172.21.15.27:0/3727652755 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab9c0f62f0 con 0x7fab9c0f49a0 2021-11-03T17:52:25.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.300+0000 7faba3cc1700 1 -- 172.21.15.27:0/3727652755 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab9c0f67d0 con 0x7fab9c0f49a0 2021-11-03T17:52:25.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.301+0000 7faba3cc1700 1 -- 172.21.15.27:0/3727652755 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab9c045ad0 con 0x7fab9c0f49a0 2021-11-03T17:52:25.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.302+0000 7fab9a7fc700 1 -- 172.21.15.27:0/3727652755 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fab94024060 con 0x7fab9c0f49a0 2021-11-03T17:52:25.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.302+0000 7fab9a7fc700 1 --2- 172.21.15.27:0/3727652755 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fab840640b0 0x7fab84066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:52:25.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.302+0000 7fab9a7fc700 1 -- 172.21.15.27:0/3727652755 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fab94082230 con 0x7fab9c0f49a0 2021-11-03T17:52:25.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.303+0000 7faba125c700 1 --2- 172.21.15.27:0/3727652755 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fab840640b0 0x7fab84066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:52:25.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.303+0000 7faba125c700 1 --2- 172.21.15.27:0/3727652755 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fab840640b0 0x7fab84066570 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7fab9c0f5780 tx=0x7fab9000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:52:25.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.306+0000 7fab9a7fc700 1 -- 172.21.15.27:0/3727652755 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fab940545b0 con 0x7fab9c0f49a0 2021-11-03T17:52:25.475 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.480+0000 7faba3cc1700 1 -- 172.21.15.27:0/3727652755 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fab9c0ff090 con 0x7fab840640b0 2021-11-03T17:52:25.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2187.412027 secs 2021-11-03T17:52:25.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.494+0000 7fab9a7fc700 1 -- 172.21.15.27:0/3727652755 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fab9000f040 con 0x7fab840640b0 2021-11-03T17:52:25.489 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:52:25.490 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (43m) 3m ago 45m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:52:25.490 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (35m) 3m ago 45m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:52:25.490 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (35m) 2m ago 44m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:52:25.490 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (43m) 3m ago 44m 41.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:52:25.491 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (37m) 3m ago 40m 23.8M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:52:25.491 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (40m) 3m ago 40m 3844M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:52:25.491 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (36m) 2m ago 40m 23.2M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:52:25.491 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (37m) 2m ago 40m 24.9M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:52:25.492 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (37m) 3m ago 46m 535M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:52:25.492 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (37m) 2m ago 44m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:52:25.492 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (36m) 3m ago 46m 198M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:52:25.492 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (36m) 2m ago 44m 95.6M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:52:25.493 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (44m) 3m ago 44m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:52:25.493 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (44m) 2m ago 44m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:52:25.493 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (35m) 3m ago 43m 331M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:52:25.493 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (34m) 3m ago 43m 361M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:52:25.493 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (34m) 3m ago 42m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:52:25.494 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (33m) 2m ago 42m 379M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:52:25.494 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (33m) 2m ago 42m 304M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:52:25.494 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (32m) 2m ago 41m 307M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:52:25.494 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (43m) 3m ago 44m 52.9M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:52:25.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.496+0000 7faba3cc1700 1 -- 172.21.15.27:0/3727652755 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fab840640b0 msgr2=0x7fab84066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:52:25.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.496+0000 7faba3cc1700 1 --2- 172.21.15.27:0/3727652755 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fab840640b0 0x7fab84066570 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7fab9c0f5780 tx=0x7fab9000b040).stop 2021-11-03T17:52:25.496 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.496+0000 7faba3cc1700 1 -- 172.21.15.27:0/3727652755 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab9c0f49a0 msgr2=0x7fab9c0f7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:52:25.496 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.496+0000 7faba3cc1700 1 --2- 172.21.15.27:0/3727652755 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab9c0f49a0 0x7fab9c0f7550 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7fab9400c010 tx=0x7fab94004d10).stop 2021-11-03T17:52:25.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.496+0000 7faba3cc1700 1 -- 172.21.15.27:0/3727652755 shutdown_connections 2021-11-03T17:52:25.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.496+0000 7faba3cc1700 1 --2- 172.21.15.27:0/3727652755 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fab840640b0 0x7fab84066570 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:25.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.496+0000 7faba3cc1700 1 --2- 172.21.15.27:0/3727652755 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab9c0f49a0 0x7fab9c0f7550 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:25.498 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.497+0000 7faba3cc1700 1 --2- 172.21.15.27:0/3727652755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab9c0f5340 0x7fab9c0f59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:25.498 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.497+0000 7faba3cc1700 1 -- 172.21.15.27:0/3727652755 >> 172.21.15.27:0/3727652755 conn(0x7fab9c0f04c0 msgr2=0x7fab9c0fb510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:52:25.498 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.497+0000 7faba3cc1700 1 -- 172.21.15.27:0/3727652755 shutdown_connections 2021-11-03T17:52:25.498 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.497+0000 7faba3cc1700 1 -- 172.21.15.27:0/3727652755 wait complete. 2021-11-03T17:52:25.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2187.412027 secs 2021-11-03T17:52:25.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.596+0000 7f817dcbc700 1 -- 172.21.15.27:0/1858321465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81780f77d0 msgr2=0x7f81780f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:52:25.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.596+0000 7f817dcbc700 1 --2- 172.21.15.27:0/1858321465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81780f77d0 0x7f81780f7c30 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7f8164004660 tx=0x7f8164009b30).stop 2021-11-03T17:52:25.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.596+0000 7f817dcbc700 1 -- 172.21.15.27:0/1858321465 shutdown_connections 2021-11-03T17:52:25.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.596+0000 7f817dcbc700 1 --2- 172.21.15.27:0/1858321465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81780f77d0 0x7f81780f7c30 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:25.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.596+0000 7f817dcbc700 1 --2- 172.21.15.27:0/1858321465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81780f6530 0x7f81780f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:25.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.596+0000 7f817dcbc700 1 -- 172.21.15.27:0/1858321465 >> 172.21.15.27:0/1858321465 conn(0x7f81780f1c90 msgr2=0x7f81780f40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:52:25.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.596+0000 7f817dcbc700 1 -- 172.21.15.27:0/1858321465 shutdown_connections 2021-11-03T17:52:25.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.596+0000 7f817dcbc700 1 -- 172.21.15.27:0/1858321465 wait complete. 2021-11-03T17:52:25.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.597+0000 7f817dcbc700 1 Processor -- start 2021-11-03T17:52:25.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.597+0000 7f817dcbc700 1 -- start start 2021-11-03T17:52:25.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.597+0000 7f817dcbc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81780f6530 0x7f8178106d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:52:25.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.597+0000 7f817dcbc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81780f77d0 0x7f8178107280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:52:25.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.597+0000 7f817dcbc700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8178107880 con 0x7f81780f6530 2021-11-03T17:52:25.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.597+0000 7f817dcbc700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8178108090 con 0x7f81780f77d0 2021-11-03T17:52:25.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.597+0000 7f817ccba700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81780f6530 0x7f8178106d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:52:25.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.597+0000 7f817ccba700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81780f6530 0x7f8178106d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59882/0 (socket says 172.21.15.27:59882) 2021-11-03T17:52:25.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.597+0000 7f817ccba700 1 -- 172.21.15.27:0/3703972615 learned_addr learned my addr 172.21.15.27:0/3703972615 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:52:25.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.597+0000 7f8177fff700 1 --2- 172.21.15.27:0/3703972615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81780f77d0 0x7f8178107280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:52:25.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.597+0000 7f817ccba700 1 -- 172.21.15.27:0/3703972615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81780f77d0 msgr2=0x7f8178107280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:52:25.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.597+0000 7f817ccba700 1 --2- 172.21.15.27:0/3703972615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81780f77d0 0x7f8178107280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:25.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.597+0000 7f817ccba700 1 -- 172.21.15.27:0/3703972615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8164005040 con 0x7f81780f6530 2021-11-03T17:52:25.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.598+0000 7f817ccba700 1 --2- 172.21.15.27:0/3703972615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81780f6530 0x7f8178106d90 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f816c002700 tx=0x7f816c009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:52:25.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.598+0000 7f81757fa700 1 -- 172.21.15.27:0/3703972615 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f816c009320 con 0x7f81780f6530 2021-11-03T17:52:25.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.598+0000 7f81757fa700 1 -- 172.21.15.27:0/3703972615 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f816c00f040 con 0x7f81780f6530 2021-11-03T17:52:25.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.598+0000 7f817dcbc700 1 -- 172.21.15.27:0/3703972615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8178108310 con 0x7f81780f6530 2021-11-03T17:52:25.600 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.598+0000 7f81757fa700 1 -- 172.21.15.27:0/3703972615 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f816c014900 con 0x7f81780f6530 2021-11-03T17:52:25.600 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.598+0000 7f817dcbc700 1 -- 172.21.15.27:0/3703972615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81781088d0 con 0x7f81780f6530 2021-11-03T17:52:25.600 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.599+0000 7f817dcbc700 1 -- 172.21.15.27:0/3703972615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8178059100 con 0x7f81780f6530 2021-11-03T17:52:25.600 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.600+0000 7f81757fa700 1 -- 172.21.15.27:0/3703972615 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f816c009480 con 0x7f81780f6530 2021-11-03T17:52:25.601 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.600+0000 7f81757fa700 1 --2- 172.21.15.27:0/3703972615 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8168064100 0x7f81680665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:52:25.601 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.600+0000 7f81757fa700 1 -- 172.21.15.27:0/3703972615 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f816c080ab0 con 0x7f81780f6530 2021-11-03T17:52:25.601 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.600+0000 7f8177fff700 1 --2- 172.21.15.27:0/3703972615 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8168064100 0x7f81680665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:52:25.601 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.601+0000 7f8177fff700 1 --2- 172.21.15.27:0/3703972615 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8168064100 0x7f81680665c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7f8164009da0 tx=0x7f8164009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:52:25.602 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.604+0000 7f81757fa700 1 -- 172.21.15.27:0/3703972615 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f816c052db0 con 0x7f81780f6530 2021-11-03T17:52:25.829 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.834+0000 7f817dcbc700 1 -- 172.21.15.27:0/3703972615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f81780fee30 con 0x7f81780f6530 2021-11-03T17:52:25.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.835+0000 7f81757fa700 1 -- 172.21.15.27:0/3703972615 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f816c0529a0 con 0x7f81780f6530 2021-11-03T17:52:25.831 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:52:25.831 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:52:25.831 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:52:25.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:52:25.832 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:52:25.832 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:52:25.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:52:25.832 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:52:25.833 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:52:25.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:52:25.833 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:52:25.833 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:52:25.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:52:25.834 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:52:25.834 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:52:25.834 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:52:25.834 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:52:25.835 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:52:25.836 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.837+0000 7f817dcbc700 1 -- 172.21.15.27:0/3703972615 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8168064100 msgr2=0x7f81680665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:52:25.836 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.837+0000 7f817dcbc700 1 --2- 172.21.15.27:0/3703972615 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8168064100 0x7f81680665c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7f8164009da0 tx=0x7f8164009b30).stop 2021-11-03T17:52:25.836 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.837+0000 7f817dcbc700 1 -- 172.21.15.27:0/3703972615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81780f6530 msgr2=0x7f8178106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:52:25.837 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.837+0000 7f817dcbc700 1 --2- 172.21.15.27:0/3703972615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81780f6530 0x7f8178106d90 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7f816c002700 tx=0x7f816c009d80).stop 2021-11-03T17:52:25.837 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.837+0000 7f817dcbc700 1 -- 172.21.15.27:0/3703972615 shutdown_connections 2021-11-03T17:52:25.837 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.837+0000 7f817dcbc700 1 --2- 172.21.15.27:0/3703972615 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8168064100 0x7f81680665c0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:25.837 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.837+0000 7f817dcbc700 1 --2- 172.21.15.27:0/3703972615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81780f6530 0x7f8178106d90 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:25.837 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.837+0000 7f817dcbc700 1 --2- 172.21.15.27:0/3703972615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81780f77d0 0x7f8178107280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:25.838 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.837+0000 7f817dcbc700 1 -- 172.21.15.27:0/3703972615 >> 172.21.15.27:0/3703972615 conn(0x7f81780f1c90 msgr2=0x7f81780fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:52:25.838 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.837+0000 7f817dcbc700 1 -- 172.21.15.27:0/3703972615 shutdown_connections 2021-11-03T17:52:25.838 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.837+0000 7f817dcbc700 1 -- 172.21.15.27:0/3703972615 wait complete. 2021-11-03T17:52:25.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.937+0000 7f7efe9fe700 1 -- 172.21.15.27:0/2540544542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ef80f6880 msgr2=0x7f7ef80f6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:52:25.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.937+0000 7f7efe9fe700 1 --2- 172.21.15.27:0/2540544542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ef80f6880 0x7f7ef80f6ce0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7f7eec004660 tx=0x7f7eec009b30).stop 2021-11-03T17:52:25.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.937+0000 7f7efe9fe700 1 -- 172.21.15.27:0/2540544542 shutdown_connections 2021-11-03T17:52:25.935 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.937+0000 7f7efe9fe700 1 --2- 172.21.15.27:0/2540544542 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ef80f6880 0x7f7ef80f6ce0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:25.935 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.937+0000 7f7efe9fe700 1 --2- 172.21.15.27:0/2540544542 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ef80f55e0 0x7f7ef80f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:25.935 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.937+0000 7f7efe9fe700 1 -- 172.21.15.27:0/2540544542 >> 172.21.15.27:0/2540544542 conn(0x7f7ef80f0d40 msgr2=0x7f7ef80f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:52:25.936 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.937+0000 7f7efe9fe700 1 -- 172.21.15.27:0/2540544542 shutdown_connections 2021-11-03T17:52:25.936 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.937+0000 7f7efe9fe700 1 -- 172.21.15.27:0/2540544542 wait complete. 2021-11-03T17:52:25.936 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.938+0000 7f7efe9fe700 1 Processor -- start 2021-11-03T17:52:25.936 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.938+0000 7f7efe9fe700 1 -- start start 2021-11-03T17:52:25.937 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.938+0000 7f7efe9fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ef80f55e0 0x7f7ef810c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:52:25.937 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.938+0000 7f7efe9fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ef80f6880 0x7f7ef810e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:52:25.937 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.938+0000 7f7efe9fe700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef80603d0 con 0x7f7ef80f6880 2021-11-03T17:52:25.937 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.938+0000 7f7efe9fe700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef8060510 con 0x7f7ef80f55e0 2021-11-03T17:52:25.938 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.938+0000 7f7ef77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ef80f6880 0x7f7ef810e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:52:25.938 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.938+0000 7f7ef77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ef80f6880 0x7f7ef810e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59894/0 (socket says 172.21.15.27:59894) 2021-11-03T17:52:25.938 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.938+0000 7f7ef77fe700 1 -- 172.21.15.27:0/2022773184 learned_addr learned my addr 172.21.15.27:0/2022773184 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:52:25.938 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.938+0000 7f7ef7fff700 1 --2- 172.21.15.27:0/2022773184 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ef80f55e0 0x7f7ef810c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:52:25.938 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.938+0000 7f7ef77fe700 1 -- 172.21.15.27:0/2022773184 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ef80f55e0 msgr2=0x7f7ef810c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:52:25.939 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.938+0000 7f7ef77fe700 1 --2- 172.21.15.27:0/2022773184 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ef80f55e0 0x7f7ef810c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:25.939 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.938+0000 7f7ef77fe700 1 -- 172.21.15.27:0/2022773184 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eec005040 con 0x7f7ef80f6880 2021-11-03T17:52:25.939 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.939+0000 7f7ef77fe700 1 --2- 172.21.15.27:0/2022773184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ef80f6880 0x7f7ef810e570 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f7eec009990 tx=0x7f7eec009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:52:25.939 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.939+0000 7f7ef4ff9700 1 -- 172.21.15.27:0/2022773184 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7eec018070 con 0x7f7ef80f6880 2021-11-03T17:52:25.940 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.939+0000 7f7ef4ff9700 1 -- 172.21.15.27:0/2022773184 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f7eec007ba0 con 0x7f7ef80f6880 2021-11-03T17:52:25.940 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.939+0000 7f7efe9fe700 1 -- 172.21.15.27:0/2022773184 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ef810ea60 con 0x7f7ef80f6880 2021-11-03T17:52:25.940 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.939+0000 7f7ef4ff9700 1 -- 172.21.15.27:0/2022773184 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7eec01c620 con 0x7f7ef80f6880 2021-11-03T17:52:25.940 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.939+0000 7f7efe9fe700 1 -- 172.21.15.27:0/2022773184 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ef810ee30 con 0x7f7ef80f6880 2021-11-03T17:52:25.942 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.940+0000 7f7efe9fe700 1 -- 172.21.15.27:0/2022773184 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ef80f49d0 con 0x7f7ef80f6880 2021-11-03T17:52:25.942 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.940+0000 7f7ef4ff9700 1 -- 172.21.15.27:0/2022773184 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f7eec01c890 con 0x7f7ef80f6880 2021-11-03T17:52:25.942 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.940+0000 7f7ef4ff9700 1 --2- 172.21.15.27:0/2022773184 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7ee00640b0 0x7f7ee0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:52:25.942 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.941+0000 7f7ef4ff9700 1 -- 172.21.15.27:0/2022773184 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f7eec082580 con 0x7f7ef80f6880 2021-11-03T17:52:25.943 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.941+0000 7f7ef7fff700 1 --2- 172.21.15.27:0/2022773184 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7ee00640b0 0x7f7ee0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:52:25.943 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.941+0000 7f7ef7fff700 1 --2- 172.21.15.27:0/2022773184 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7ee00640b0 0x7f7ee0066570 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f7ef8060160 tx=0x7f7ee8009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:52:25.943 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:25.945+0000 7f7ef4ff9700 1 -- 172.21.15.27:0/2022773184 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f7eec054940 con 0x7f7ef80f6880 2021-11-03T17:52:26.144 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:26.148+0000 7f7efe9fe700 1 -- 172.21.15.27:0/2022773184 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7ef80fde60 con 0x7f7ef80f6880 2021-11-03T17:52:26.144 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:26.149+0000 7f7ef4ff9700 1 -- 172.21.15.27:0/2022773184 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f7eec054530 con 0x7f7ef80f6880 2021-11-03T17:52:26.144 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:52:26.145 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:52:26.145 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:52:26.145 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:52:26.145 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:52:26.146 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:52:26.146 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:52:26.146 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:52:26.146 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:52:26.147 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:52:26.147 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:52:26.147 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:52:26.147 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:52:26.148 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:52:26.148 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:52:26.148 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:52:26.148 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:52:26.149 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:52:26.149 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:52:26.149 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:52:26.149 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:52:26.149 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:52:26.150 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:52:26.150 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:52:26.150 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:52:26.150 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:52:26.151 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:52:26.151 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:52:26.151 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:52:26.151 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:52:26.152 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:52:26.152 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:52:26.152 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:52:26.152 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:52:26.153 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:52:26.153 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:52:26.153 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:52:26.153 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:52:26.153 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:52:26.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:26.151+0000 7f7efe9fe700 1 -- 172.21.15.27:0/2022773184 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7ee00640b0 msgr2=0x7f7ee0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:52:26.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:26.151+0000 7f7efe9fe700 1 --2- 172.21.15.27:0/2022773184 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7ee00640b0 0x7f7ee0066570 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7f7ef8060160 tx=0x7f7ee8009040).stop 2021-11-03T17:52:26.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:26.151+0000 7f7efe9fe700 1 -- 172.21.15.27:0/2022773184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ef80f6880 msgr2=0x7f7ef810e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:52:26.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:26.151+0000 7f7efe9fe700 1 --2- 172.21.15.27:0/2022773184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ef80f6880 0x7f7ef810e570 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f7eec009990 tx=0x7f7eec009b30).stop 2021-11-03T17:52:26.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:26.151+0000 7f7efe9fe700 1 -- 172.21.15.27:0/2022773184 shutdown_connections 2021-11-03T17:52:26.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:26.151+0000 7f7efe9fe700 1 --2- 172.21.15.27:0/2022773184 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7ee00640b0 0x7f7ee0066570 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:26.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:26.151+0000 7f7efe9fe700 1 --2- 172.21.15.27:0/2022773184 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ef80f55e0 0x7f7ef810c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:26.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:26.151+0000 7f7efe9fe700 1 --2- 172.21.15.27:0/2022773184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ef80f6880 0x7f7ef810e570 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:26.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:26.151+0000 7f7efe9fe700 1 -- 172.21.15.27:0/2022773184 >> 172.21.15.27:0/2022773184 conn(0x7f7ef80f0d40 msgr2=0x7f7ef80fa2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:52:26.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:26.151+0000 7f7efe9fe700 1 -- 172.21.15.27:0/2022773184 shutdown_connections 2021-11-03T17:52:26.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:26.151+0000 7f7efe9fe700 1 -- 172.21.15.27:0/2022773184 wait complete. 2021-11-03T17:52:26.157 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:52:26.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:26 smithi115 ceph-mon[65453]: pgmap v1133: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:26.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:26 smithi115 ceph-mon[65453]: from='client.35272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:52:26.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:26 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3703972615' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:26.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:26 smithi027 ceph-mon[97244]: pgmap v1133: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:26.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:26 smithi027 ceph-mon[97244]: from='client.35272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:52:26.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:26 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3703972615' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:27.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:27 smithi115 ceph-mon[65453]: from='client.35276 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:52:27.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:27 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2022773184' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:52:27.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:27 smithi027 ceph-mon[97244]: from='client.35276 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:52:27.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:27 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2022773184' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:52:28.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:28 smithi115 ceph-mon[65453]: pgmap v1134: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:28.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:28 smithi027 ceph-mon[97244]: pgmap v1134: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:30.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:30 smithi115 ceph-mon[65453]: pgmap v1135: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:30.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2192.412273 secs 2021-11-03T17:52:30.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:30 smithi027 ceph-mon[97244]: pgmap v1135: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:30.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2192.412273 secs 2021-11-03T17:52:32.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:32 smithi115 ceph-mon[65453]: pgmap v1136: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:32.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:32 smithi027 ceph-mon[97244]: pgmap v1136: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:33.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:52:33.340 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:52:34.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:34 smithi115 ceph-mon[65453]: pgmap v1137: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:34.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:34 smithi115 ceph-mon[65453]: poll took 60179.961 ms: 1 events 2021-11-03T17:52:34.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:52:34.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:52:34.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:34.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:34.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:34.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:34.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:34 smithi027 ceph-mon[97244]: pgmap v1137: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:34.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:34 smithi027 ceph-mon[97244]: poll took 60179.961 ms: 1 events 2021-11-03T17:52:34.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:52:34.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:52:34.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:34.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:34.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:34.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:35.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:35 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:52:35.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2197.412396 secs 2021-11-03T17:52:35.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:35 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:52:35.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2197.412396 secs 2021-11-03T17:52:36.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:36 smithi115 ceph-mon[65453]: pgmap v1138: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:36.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:36 smithi027 ceph-mon[97244]: pgmap v1138: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:38.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:38 smithi115 ceph-mon[65453]: pgmap v1139: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:38.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:38 smithi027 ceph-mon[97244]: pgmap v1139: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:40.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:40 smithi115 ceph-mon[65453]: pgmap v1140: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:40.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2202.412465 secs 2021-11-03T17:52:40.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:40 smithi027 ceph-mon[97244]: pgmap v1140: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:40.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2202.412465 secs 2021-11-03T17:52:42.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:42 smithi115 ceph-mon[65453]: pgmap v1141: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:42.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:42 smithi027 ceph-mon[97244]: pgmap v1141: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:44.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:44 smithi115 ceph-mon[65453]: pgmap v1142: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:44.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:52:44.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:52:44.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:52:44.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:44.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:44.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:44.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:44.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:44 smithi027 ceph-mon[97244]: pgmap v1142: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:44.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:52:44.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:52:44.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:52:44.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:44.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:44.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:44.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:45.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:45 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:52:45.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2207.412608 secs 2021-11-03T17:52:45.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:45 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:52:45.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2207.412608 secs 2021-11-03T17:52:46.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:46 smithi115 ceph-mon[65453]: pgmap v1143: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:46.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:46 smithi027 ceph-mon[97244]: pgmap v1143: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:48.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:48 smithi115 ceph-mon[65453]: pgmap v1144: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:48.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:48 smithi027 ceph-mon[97244]: pgmap v1144: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:50.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:50 smithi115 ceph-mon[65453]: pgmap v1145: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:50.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2212.412676 secs 2021-11-03T17:52:50.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:50 smithi027 ceph-mon[97244]: pgmap v1145: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:50.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2212.412676 secs 2021-11-03T17:52:52.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:52 smithi115 ceph-mon[65453]: pgmap v1146: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:52.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:52 smithi027 ceph-mon[97244]: pgmap v1146: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:54.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:54 smithi115 ceph-mon[65453]: pgmap v1147: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:54.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:52:54.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:52:54.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:52:54.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:54.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:54.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:54.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:54.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:54 smithi027 ceph-mon[97244]: pgmap v1147: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:54.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:52:54.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:52:54.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:52:54.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:54.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:54.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:54.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:55.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:55 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:52:55.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2217.412814 secs 2021-11-03T17:52:55.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:55 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:52:55.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2217.412814 secs 2021-11-03T17:52:56.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.267+0000 7f2025559700 1 -- 172.21.15.27:0/1120141915 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2020060310 msgr2=0x7f20200f9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:52:56.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.267+0000 7f2025559700 1 --2- 172.21.15.27:0/1120141915 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2020060310 0x7f20200f9120 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7f2008004660 tx=0x7f2008009b30).stop 2021-11-03T17:52:56.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.267+0000 7f2025559700 1 -- 172.21.15.27:0/1120141915 shutdown_connections 2021-11-03T17:52:56.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.267+0000 7f2025559700 1 --2- 172.21.15.27:0/1120141915 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20200f96a0 0x7f20200fbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:56.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.267+0000 7f2025559700 1 --2- 172.21.15.27:0/1120141915 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2020060310 0x7f20200f9120 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:56.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.267+0000 7f2025559700 1 -- 172.21.15.27:0/1120141915 >> 172.21.15.27:0/1120141915 conn(0x7f20200f0ba0 msgr2=0x7f20200f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:52:56.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.267+0000 7f2025559700 1 -- 172.21.15.27:0/1120141915 shutdown_connections 2021-11-03T17:52:56.265 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.267+0000 7f2025559700 1 -- 172.21.15.27:0/1120141915 wait complete. 2021-11-03T17:52:56.265 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.268+0000 7f2025559700 1 Processor -- start 2021-11-03T17:52:56.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.268+0000 7f2025559700 1 -- start start 2021-11-03T17:52:56.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.268+0000 7f2025559700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2020060310 0x7f2020106da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:52:56.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.268+0000 7f2025559700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20200f96a0 0x7f2020107290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:52:56.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.268+0000 7f2025559700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2020107890 con 0x7f2020060310 2021-11-03T17:52:56.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.268+0000 7f2025559700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20201080d0 con 0x7f20200f96a0 2021-11-03T17:52:56.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.268+0000 7f201ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2020060310 0x7f2020106da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:52:56.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.269+0000 7f201ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2020060310 0x7f2020106da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59910/0 (socket says 172.21.15.27:59910) 2021-11-03T17:52:56.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.269+0000 7f201ffff700 1 -- 172.21.15.27:0/68039383 learned_addr learned my addr 172.21.15.27:0/68039383 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:52:56.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.269+0000 7f201f7fe700 1 --2- 172.21.15.27:0/68039383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20200f96a0 0x7f2020107290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:52:56.270 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.269+0000 7f201ffff700 1 -- 172.21.15.27:0/68039383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20200f96a0 msgr2=0x7f2020107290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:52:56.270 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.269+0000 7f201ffff700 1 --2- 172.21.15.27:0/68039383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20200f96a0 0x7f2020107290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:56.270 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.269+0000 7f201ffff700 1 -- 172.21.15.27:0/68039383 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2008005040 con 0x7f2020060310 2021-11-03T17:52:56.270 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.269+0000 7f201f7fe700 1 --2- 172.21.15.27:0/68039383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20200f96a0 0x7f2020107290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:52:56.271 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.269+0000 7f201ffff700 1 --2- 172.21.15.27:0/68039383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2020060310 0x7f2020106da0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f2008009990 tx=0x7f2008009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:52:56.271 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.269+0000 7f201cff9700 1 -- 172.21.15.27:0/68039383 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2008018070 con 0x7f2020060310 2021-11-03T17:52:56.271 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.269+0000 7f201cff9700 1 -- 172.21.15.27:0/68039383 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f2008007ba0 con 0x7f2020060310 2021-11-03T17:52:56.272 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.269+0000 7f2025559700 1 -- 172.21.15.27:0/68039383 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2020108350 con 0x7f2020060310 2021-11-03T17:52:56.272 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.269+0000 7f201cff9700 1 -- 172.21.15.27:0/68039383 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f200801c3c0 con 0x7f2020060310 2021-11-03T17:52:56.272 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.269+0000 7f2025559700 1 -- 172.21.15.27:0/68039383 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2020108830 con 0x7f2020060310 2021-11-03T17:52:56.272 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.270+0000 7f2025559700 1 -- 172.21.15.27:0/68039383 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2020059100 con 0x7f2020060310 2021-11-03T17:52:56.273 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.271+0000 7f201cff9700 1 -- 172.21.15.27:0/68039383 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f200801d420 con 0x7f2020060310 2021-11-03T17:52:56.273 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.271+0000 7f201cff9700 1 --2- 172.21.15.27:0/68039383 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f200c064100 0x7f200c0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:52:56.273 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.271+0000 7f201cff9700 1 -- 172.21.15.27:0/68039383 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f2008082560 con 0x7f2020060310 2021-11-03T17:52:56.273 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.271+0000 7f201f7fe700 1 --2- 172.21.15.27:0/68039383 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f200c064100 0x7f200c0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:52:56.273 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.272+0000 7f201f7fe700 1 --2- 172.21.15.27:0/68039383 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f200c064100 0x7f200c0665c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f2020106b30 tx=0x7f2010009be0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:52:56.274 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.275+0000 7f201cff9700 1 -- 172.21.15.27:0/68039383 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f20080548c0 con 0x7f2020060310 2021-11-03T17:52:56.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.456+0000 7f2025559700 1 -- 172.21.15.27:0/68039383 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f20200fee30 con 0x7f200c064100 2021-11-03T17:52:56.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.457+0000 7f201cff9700 1 -- 172.21.15.27:0/68039383 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f20200fee30 con 0x7f200c064100 2021-11-03T17:52:56.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.459+0000 7f2025559700 1 -- 172.21.15.27:0/68039383 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f200c064100 msgr2=0x7f200c0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:52:56.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.459+0000 7f2025559700 1 --2- 172.21.15.27:0/68039383 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f200c064100 0x7f200c0665c0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7f2020106b30 tx=0x7f2010009be0).stop 2021-11-03T17:52:56.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.459+0000 7f2025559700 1 -- 172.21.15.27:0/68039383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2020060310 msgr2=0x7f2020106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:52:56.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.459+0000 7f2025559700 1 --2- 172.21.15.27:0/68039383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2020060310 0x7f2020106da0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f2008009990 tx=0x7f2008009b30).stop 2021-11-03T17:52:56.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.459+0000 7f2025559700 1 -- 172.21.15.27:0/68039383 shutdown_connections 2021-11-03T17:52:56.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.459+0000 7f2025559700 1 --2- 172.21.15.27:0/68039383 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f200c064100 0x7f200c0665c0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:56.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.459+0000 7f2025559700 1 --2- 172.21.15.27:0/68039383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2020060310 0x7f2020106da0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:56.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.459+0000 7f2025559700 1 --2- 172.21.15.27:0/68039383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20200f96a0 0x7f2020107290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:56.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.459+0000 7f2025559700 1 -- 172.21.15.27:0/68039383 >> 172.21.15.27:0/68039383 conn(0x7f20200f0ba0 msgr2=0x7f20200f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:52:56.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.459+0000 7f2025559700 1 -- 172.21.15.27:0/68039383 shutdown_connections 2021-11-03T17:52:56.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.459+0000 7f2025559700 1 -- 172.21.15.27:0/68039383 wait complete. 2021-11-03T17:52:56.464 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:52:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:56 smithi115 ceph-mon[65453]: pgmap v1148: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:52:56.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:52:56.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:56 smithi027 ceph-mon[97244]: pgmap v1148: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:56.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:52:56.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:52:56.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.560+0000 7fd339c51700 1 -- 172.21.15.27:0/270305286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd33406a3b0 msgr2=0x7fd33406a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:52:56.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.560+0000 7fd339c51700 1 --2- 172.21.15.27:0/270305286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd33406a3b0 0x7fd33406a830 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7fd320004660 tx=0x7fd320009b30).stop 2021-11-03T17:52:56.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.560+0000 7fd339c51700 1 -- 172.21.15.27:0/270305286 shutdown_connections 2021-11-03T17:52:56.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.560+0000 7fd339c51700 1 --2- 172.21.15.27:0/270305286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd33406a3b0 0x7fd33406a830 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:56.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.560+0000 7fd339c51700 1 --2- 172.21.15.27:0/270305286 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd33406b9c0 0x7fd334069ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:52:56.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.560+0000 7fd339c51700 1 -- 172.21.15.27:0/270305286 >> 172.21.15.27:0/270305286 conn(0x7fd3340f1c90 msgr2=0x7fd3340f40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:52:56.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.561+0000 7fd339c51700 1 -- 172.21.15.27:0/270305286 shutdown_connections 2021-11-03T17:52:56.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.561+0000 7fd339c51700 1 -- 172.21.15.27:0/270305286 wait complete. 2021-11-03T17:52:56.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.561+0000 7fd339c51700 1 Processor -- start 2021-11-03T17:52:56.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.561+0000 7fd339c51700 1 -- start start 2021-11-03T17:52:56.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.561+0000 7fd339c51700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd33406a3b0 0x7fd33410d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:52:56.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.561+0000 7fd339c51700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd33406b9c0 0x7fd33410d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:52:56.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.561+0000 7fd339c51700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33410de80 con 0x7fd33406b9c0 2021-11-03T17:52:56.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.561+0000 7fd339c51700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd33410e730 con 0x7fd33406a3b0 2021-11-03T17:52:56.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.562+0000 7fd333fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd33406b9c0 0x7fd33410d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:52:56.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.562+0000 7fd333fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd33406b9c0 0x7fd33410d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59922/0 (socket says 172.21.15.27:59922) 2021-11-03T17:52:56.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.562+0000 7fd333fff700 1 -- 172.21.15.27:0/4120167936 learned_addr learned my addr 172.21.15.27:0/4120167936 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:52:56.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.562+0000 7fd338c4f700 1 --2- 172.21.15.27:0/4120167936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd33406a3b0 0x7fd33410d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:52:56.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.562+0000 7fd333fff700 1 -- 172.21.15.27:0/4120167936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd33406a3b0 msgr2=0x7fd33410d350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:52:56.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.562+0000 7fd333fff700 1 --2- 172.21.15.27:0/4120167936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd33406a3b0 0x7fd33410d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:56.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.562+0000 7fd333fff700 1 -- 172.21.15.27:0/4120167936 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd320005040 con 0x7fd33406b9c0 2021-11-03T17:52:56.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.562+0000 7fd333fff700 1 --2- 172.21.15.27:0/4120167936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd33406b9c0 0x7fd33410d840 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7fd320009da0 tx=0x7fd320009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:52:56.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.562+0000 7fd3317fa700 1 -- 172.21.15.27:0/4120167936 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd320018070 con 0x7fd33406b9c0 2021-11-03T17:52:56.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.562+0000 7fd3317fa700 1 -- 172.21.15.27:0/4120167936 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fd320007d70 con 0x7fd33406b9c0 2021-11-03T17:52:56.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.562+0000 7fd3317fa700 1 -- 172.21.15.27:0/4120167936 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd32001dea0 con 0x7fd33406b9c0 2021-11-03T17:52:56.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.562+0000 7fd339c51700 1 -- 172.21.15.27:0/4120167936 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd33410e9b0 con 0x7fd33406b9c0 2021-11-03T17:52:56.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.562+0000 7fd339c51700 1 -- 172.21.15.27:0/4120167936 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd33410ef10 con 0x7fd33406b9c0 2021-11-03T17:52:56.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.563+0000 7fd339c51700 1 -- 172.21.15.27:0/4120167936 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd334059100 con 0x7fd33406b9c0 2021-11-03T17:52:56.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.564+0000 7fd3317fa700 1 -- 172.21.15.27:0/4120167936 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fd32001d420 con 0x7fd33406b9c0 2021-11-03T17:52:56.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.564+0000 7fd3317fa700 1 --2- 172.21.15.27:0/4120167936 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd324064100 0x7fd3240665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:52:56.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.564+0000 7fd3317fa700 1 -- 172.21.15.27:0/4120167936 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fd3200826d0 con 0x7fd33406b9c0 2021-11-03T17:52:56.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.564+0000 7fd338c4f700 1 --2- 172.21.15.27:0/4120167936 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd324064100 0x7fd3240665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:52:56.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.565+0000 7fd338c4f700 1 --2- 172.21.15.27:0/4120167936 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd324064100 0x7fd3240665c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7fd32800a6f0 tx=0x7fd328008040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:52:56.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.568+0000 7fd3317fa700 1 -- 172.21.15.27:0/4120167936 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fd320054a90 con 0x7fd33406b9c0 2021-11-03T17:52:56.735 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.740+0000 7fd339c51700 1 -- 172.21.15.27:0/4120167936 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd3340008d0 con 0x7fd324064100 2021-11-03T17:52:56.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.754+0000 7fd3317fa700 1 -- 172.21.15.27:0/4120167936 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fd3340008d0 con 0x7fd324064100 2021-11-03T17:52:56.750 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:52:56.750 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (44m) 4m ago 45m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:52:56.750 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (36m) 4m ago 45m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:52:56.751 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (36m) 3m ago 44m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:52:56.751 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (44m) 4m ago 45m 41.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:52:56.751 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (38m) 4m ago 40m 23.8M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:52:56.751 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (40m) 4m ago 40m 3844M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:52:56.752 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (36m) 3m ago 40m 23.2M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:52:56.752 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (38m) 3m ago 41m 24.9M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:52:56.752 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (38m) 4m ago 47m 535M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:52:56.752 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (37m) 3m ago 44m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:52:56.753 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (37m) 4m ago 47m 198M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:52:56.753 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (36m) 3m ago 44m 95.6M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:52:56.753 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (45m) 4m ago 45m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:52:56.754 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (44m) 3m ago 44m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:52:56.754 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (35m) 4m ago 44m 331M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:52:56.754 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (35m) 4m ago 43m 361M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:52:56.754 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (34m) 4m ago 43m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:52:56.755 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (34m) 3m ago 43m 379M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:52:56.755 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (33m) 3m ago 42m 304M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:52:56.755 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (33m) 3m ago 42m 307M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:52:56.755 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (44m) 4m ago 45m 52.9M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:52:56.756 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.756+0000 7fd339c51700 1 -- 172.21.15.27:0/4120167936 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd324064100 msgr2=0x7fd3240665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:52:56.756 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.756+0000 7fd339c51700 1 --2- 172.21.15.27:0/4120167936 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd324064100 0x7fd3240665c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7fd32800a6f0 tx=0x7fd328008040).stop 2021-11-03T17:52:56.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.756+0000 7fd339c51700 1 -- 172.21.15.27:0/4120167936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd33406b9c0 msgr2=0x7fd33410d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:52:56.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.756+0000 7fd339c51700 1 --2- 172.21.15.27:0/4120167936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd33406b9c0 0x7fd33410d840 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7fd320009da0 tx=0x7fd320009b30).stop 2021-11-03T17:52:56.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.756+0000 7fd339c51700 1 -- 172.21.15.27:0/4120167936 shutdown_connections 2021-11-03T17:52:56.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.756+0000 7fd339c51700 1 --2- 172.21.15.27:0/4120167936 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd324064100 0x7fd3240665c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:56.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.756+0000 7fd339c51700 1 --2- 172.21.15.27:0/4120167936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd33406a3b0 0x7fd33410d350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:56.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.756+0000 7fd339c51700 1 --2- 172.21.15.27:0/4120167936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd33406b9c0 0x7fd33410d840 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:56.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.756+0000 7fd339c51700 1 -- 172.21.15.27:0/4120167936 >> 172.21.15.27:0/4120167936 conn(0x7fd3340f1c90 msgr2=0x7fd3340f8820 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:52:56.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.756+0000 7fd339c51700 1 -- 172.21.15.27:0/4120167936 shutdown_connections 2021-11-03T17:52:56.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.757+0000 7fd339c51700 1 -- 172.21.15.27:0/4120167936 wait complete. 2021-11-03T17:52:56.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.855+0000 7f57a2d4c700 1 -- 172.21.15.27:0/1058522153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f579c0f5ef0 msgr2=0x7f579c0f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:52:56.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.855+0000 7f57a2d4c700 1 --2- 172.21.15.27:0/1058522153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f579c0f5ef0 0x7f579c0f6920 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f578c0099e0 tx=0x7f578c004d10).stop 2021-11-03T17:52:56.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.855+0000 7f57a2d4c700 1 -- 172.21.15.27:0/1058522153 shutdown_connections 2021-11-03T17:52:56.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.855+0000 7f57a2d4c700 1 --2- 172.21.15.27:0/1058522153 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f579c0f5ef0 0x7f579c0f6920 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:56.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.855+0000 7f57a2d4c700 1 --2- 172.21.15.27:0/1058522153 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f579c0f55e0 0x7f579c0f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:56.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.855+0000 7f57a2d4c700 1 -- 172.21.15.27:0/1058522153 >> 172.21.15.27:0/1058522153 conn(0x7f579c0f0de0 msgr2=0x7f579c0f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:52:56.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.855+0000 7f57a2d4c700 1 -- 172.21.15.27:0/1058522153 shutdown_connections 2021-11-03T17:52:56.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.855+0000 7f57a2d4c700 1 -- 172.21.15.27:0/1058522153 wait complete. 2021-11-03T17:52:56.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.856+0000 7f57a2d4c700 1 Processor -- start 2021-11-03T17:52:56.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.856+0000 7f57a2d4c700 1 -- start start 2021-11-03T17:52:56.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.856+0000 7f57a2d4c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f579c0f55e0 0x7f579c109d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:52:56.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.856+0000 7f57a2d4c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f579c0f5ef0 0x7f579c10c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:52:56.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.856+0000 7f57a2d4c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579c060500 con 0x7f579c0f55e0 2021-11-03T17:52:56.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.856+0000 7f57a2d4c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f579c060640 con 0x7f579c0f5ef0 2021-11-03T17:52:56.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.856+0000 7f57a0ae8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f579c0f55e0 0x7f579c109d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:52:56.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.856+0000 7f57a0ae8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f579c0f55e0 0x7f579c109d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59930/0 (socket says 172.21.15.27:59930) 2021-11-03T17:52:56.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.856+0000 7f57a0ae8700 1 -- 172.21.15.27:0/2422054558 learned_addr learned my addr 172.21.15.27:0/2422054558 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:52:56.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.856+0000 7f579bfff700 1 --2- 172.21.15.27:0/2422054558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f579c0f5ef0 0x7f579c10c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:52:56.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.857+0000 7f57a0ae8700 1 -- 172.21.15.27:0/2422054558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f579c0f5ef0 msgr2=0x7f579c10c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:52:56.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.857+0000 7f57a0ae8700 1 --2- 172.21.15.27:0/2422054558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f579c0f5ef0 0x7f579c10c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:56.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.857+0000 7f57a0ae8700 1 -- 172.21.15.27:0/2422054558 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578c009640 con 0x7f579c0f55e0 2021-11-03T17:52:56.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.857+0000 7f57a0ae8700 1 --2- 172.21.15.27:0/2422054558 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f579c0f55e0 0x7f579c109d60 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f57940026d0 tx=0x7f5794009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:52:56.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.857+0000 7f57997fa700 1 -- 172.21.15.27:0/2422054558 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57940079c0 con 0x7f579c0f55e0 2021-11-03T17:52:56.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.857+0000 7f57997fa700 1 -- 172.21.15.27:0/2422054558 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f579400f040 con 0x7f579c0f55e0 2021-11-03T17:52:56.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.857+0000 7f57a2d4c700 1 -- 172.21.15.27:0/2422054558 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f579c10c810 con 0x7f579c0f55e0 2021-11-03T17:52:56.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.857+0000 7f57997fa700 1 -- 172.21.15.27:0/2422054558 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57940148b0 con 0x7f579c0f55e0 2021-11-03T17:52:56.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.857+0000 7f57a2d4c700 1 -- 172.21.15.27:0/2422054558 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f579c10cd50 con 0x7f579c0f55e0 2021-11-03T17:52:56.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.858+0000 7f57a2d4c700 1 -- 172.21.15.27:0/2422054558 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f579c045ad0 con 0x7f579c0f55e0 2021-11-03T17:52:56.860 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.858+0000 7f57997fa700 1 -- 172.21.15.27:0/2422054558 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f5794007b20 con 0x7f579c0f55e0 2021-11-03T17:52:56.860 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.859+0000 7f57997fa700 1 --2- 172.21.15.27:0/2422054558 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f57880640b0 0x7f5788066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:52:56.860 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.859+0000 7f57997fa700 1 -- 172.21.15.27:0/2422054558 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f5794080790 con 0x7f579c0f55e0 2021-11-03T17:52:56.860 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.859+0000 7f579bfff700 1 --2- 172.21.15.27:0/2422054558 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f57880640b0 0x7f5788066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:52:56.861 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.859+0000 7f579bfff700 1 --2- 172.21.15.27:0/2422054558 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f57880640b0 0x7f5788066570 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f578c003380 tx=0x7f578c004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:52:56.862 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:56.863+0000 7f57997fa700 1 -- 172.21.15.27:0/2422054558 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f5794052a90 con 0x7f579c0f55e0 2021-11-03T17:52:57.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.094+0000 7f57a2d4c700 1 -- 172.21.15.27:0/2422054558 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f579c0fded0 con 0x7f579c0f55e0 2021-11-03T17:52:57.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.094+0000 7f57997fa700 1 -- 172.21.15.27:0/2422054558 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f5794052680 con 0x7f579c0f55e0 2021-11-03T17:52:57.090 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:52:57.090 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:52:57.090 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:52:57.090 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:52:57.091 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:52:57.091 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:52:57.091 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:52:57.091 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:52:57.092 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:52:57.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:52:57.092 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:52:57.092 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:52:57.093 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:52:57.093 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:52:57.093 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:52:57.093 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:52:57.093 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:52:57.094 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:52:57.095 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.096+0000 7f57a2d4c700 1 -- 172.21.15.27:0/2422054558 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f57880640b0 msgr2=0x7f5788066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:52:57.095 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.096+0000 7f57a2d4c700 1 --2- 172.21.15.27:0/2422054558 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f57880640b0 0x7f5788066570 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f578c003380 tx=0x7f578c004d10).stop 2021-11-03T17:52:57.095 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.096+0000 7f57a2d4c700 1 -- 172.21.15.27:0/2422054558 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f579c0f55e0 msgr2=0x7f579c109d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:52:57.096 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.096+0000 7f57a2d4c700 1 --2- 172.21.15.27:0/2422054558 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f579c0f55e0 0x7f579c109d60 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f57940026d0 tx=0x7f5794009de0).stop 2021-11-03T17:52:57.096 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.096+0000 7f57a2d4c700 1 -- 172.21.15.27:0/2422054558 shutdown_connections 2021-11-03T17:52:57.096 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.096+0000 7f57a2d4c700 1 --2- 172.21.15.27:0/2422054558 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f57880640b0 0x7f5788066570 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:57.096 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.096+0000 7f57a2d4c700 1 --2- 172.21.15.27:0/2422054558 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f579c0f55e0 0x7f579c109d60 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:57.097 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.096+0000 7f57a2d4c700 1 --2- 172.21.15.27:0/2422054558 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f579c0f5ef0 0x7f579c10c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:57.097 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.096+0000 7f57a2d4c700 1 -- 172.21.15.27:0/2422054558 >> 172.21.15.27:0/2422054558 conn(0x7f579c0f0de0 msgr2=0x7f579c0fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:52:57.097 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.096+0000 7f57a2d4c700 1 -- 172.21.15.27:0/2422054558 shutdown_connections 2021-11-03T17:52:57.098 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.096+0000 7f57a2d4c700 1 -- 172.21.15.27:0/2422054558 wait complete. 2021-11-03T17:52:57.197 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.198+0000 7f20794a5700 1 -- 172.21.15.27:0/2620645929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20740f5460 msgr2=0x7f20740f5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:52:57.198 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.198+0000 7f20794a5700 1 --2- 172.21.15.27:0/2620645929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20740f5460 0x7f20740f5880 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7f2064004660 tx=0x7f2064009b30).stop 2021-11-03T17:52:57.198 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.198+0000 7f20794a5700 1 -- 172.21.15.27:0/2620645929 shutdown_connections 2021-11-03T17:52:57.199 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.198+0000 7f20794a5700 1 --2- 172.21.15.27:0/2620645929 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20740f6700 0x7f20740f6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:57.199 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.198+0000 7f20794a5700 1 --2- 172.21.15.27:0/2620645929 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20740f5460 0x7f20740f5880 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:57.199 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.198+0000 7f20794a5700 1 -- 172.21.15.27:0/2620645929 >> 172.21.15.27:0/2620645929 conn(0x7f20740f0bc0 msgr2=0x7f20740f2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:52:57.199 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.198+0000 7f20794a5700 1 -- 172.21.15.27:0/2620645929 shutdown_connections 2021-11-03T17:52:57.199 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.198+0000 7f20794a5700 1 -- 172.21.15.27:0/2620645929 wait complete. 2021-11-03T17:52:57.200 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.199+0000 7f20794a5700 1 Processor -- start 2021-11-03T17:52:57.200 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.199+0000 7f20794a5700 1 -- start start 2021-11-03T17:52:57.200 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.199+0000 7f20794a5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20740f5460 0x7f20740fd5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:52:57.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.199+0000 7f20794a5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20740f6700 0x7f20740fba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:52:57.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.199+0000 7f20794a5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20740fbfc0 con 0x7f20740f6700 2021-11-03T17:52:57.202 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.199+0000 7f20794a5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20740fc100 con 0x7f20740f5460 2021-11-03T17:52:57.202 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.199+0000 7f206bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20740f6700 0x7f20740fba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:52:57.203 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.199+0000 7f206bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20740f6700 0x7f20740fba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59942/0 (socket says 172.21.15.27:59942) 2021-11-03T17:52:57.203 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.199+0000 7f206bfff700 1 -- 172.21.15.27:0/4262956800 learned_addr learned my addr 172.21.15.27:0/4262956800 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:52:57.203 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.199+0000 7f2072ffd700 1 --2- 172.21.15.27:0/4262956800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20740f5460 0x7f20740fd5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:52:57.204 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.199+0000 7f206bfff700 1 -- 172.21.15.27:0/4262956800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20740f5460 msgr2=0x7f20740fd5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:52:57.204 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.199+0000 7f206bfff700 1 --2- 172.21.15.27:0/4262956800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20740f5460 0x7f20740fd5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:57.204 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.199+0000 7f206bfff700 1 -- 172.21.15.27:0/4262956800 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2064005040 con 0x7f20740f6700 2021-11-03T17:52:57.204 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.199+0000 7f206bfff700 1 --2- 172.21.15.27:0/4262956800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20740f6700 0x7f20740fba60 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7f205c00e7f0 tx=0x7f205c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:52:57.205 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.200+0000 7f206b7fe700 1 -- 172.21.15.27:0/4262956800 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f205c0092e0 con 0x7f20740f6700 2021-11-03T17:52:57.205 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.200+0000 7f206b7fe700 1 -- 172.21.15.27:0/4262956800 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f205c00f040 con 0x7f20740f6700 2021-11-03T17:52:57.205 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.200+0000 7f20794a5700 1 -- 172.21.15.27:0/4262956800 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20740fc3e0 con 0x7f20740f6700 2021-11-03T17:52:57.205 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.200+0000 7f206b7fe700 1 -- 172.21.15.27:0/4262956800 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f205c014750 con 0x7f20740f6700 2021-11-03T17:52:57.206 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.200+0000 7f20794a5700 1 -- 172.21.15.27:0/4262956800 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f207410ed60 con 0x7f20740f6700 2021-11-03T17:52:57.206 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.201+0000 7f20794a5700 1 -- 172.21.15.27:0/4262956800 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2074045ad0 con 0x7f20740f6700 2021-11-03T17:52:57.206 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.201+0000 7f206b7fe700 1 -- 172.21.15.27:0/4262956800 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f205c01b030 con 0x7f20740f6700 2021-11-03T17:52:57.206 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.201+0000 7f206b7fe700 1 --2- 172.21.15.27:0/4262956800 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2060064100 0x7f20600665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:52:57.207 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.202+0000 7f206b7fe700 1 -- 172.21.15.27:0/4262956800 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f205c080630 con 0x7f20740f6700 2021-11-03T17:52:57.207 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.202+0000 7f2072ffd700 1 --2- 172.21.15.27:0/4262956800 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2060064100 0x7f20600665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:52:57.207 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.202+0000 7f2072ffd700 1 --2- 172.21.15.27:0/4262956800 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2060064100 0x7f20600665c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f2064000c00 tx=0x7f2064016040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:52:57.208 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.205+0000 7f206b7fe700 1 -- 172.21.15.27:0/4262956800 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f205c0528f0 con 0x7f20740f6700 2021-11-03T17:52:57.402 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.407+0000 7f20794a5700 1 -- 172.21.15.27:0/4262956800 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f20740fed30 con 0x7f20740f6700 2021-11-03T17:52:57.402 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.407+0000 7f206b7fe700 1 -- 172.21.15.27:0/4262956800 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f205c019070 con 0x7f20740f6700 2021-11-03T17:52:57.403 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:52:57.403 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:52:57.403 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:52:57.404 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:52:57.404 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:52:57.404 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:52:57.404 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:52:57.405 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:52:57.405 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:52:57.405 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:52:57.405 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:52:57.406 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:52:57.406 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:52:57.406 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:52:57.406 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:52:57.406 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:52:57.407 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:52:57.407 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:52:57.407 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:52:57.407 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:52:57.408 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:52:57.408 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:52:57.408 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:52:57.408 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:52:57.409 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:52:57.409 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:52:57.409 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:52:57.410 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:52:57.410 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:52:57.411 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:52:57.411 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:52:57.411 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:52:57.412 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:52:57.412 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:52:57.412 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:52:57.412 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:52:57.413 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:52:57.413 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:52:57.413 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:52:57.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.409+0000 7f20794a5700 1 -- 172.21.15.27:0/4262956800 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2060064100 msgr2=0x7f20600665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:52:57.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.409+0000 7f20794a5700 1 --2- 172.21.15.27:0/4262956800 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2060064100 0x7f20600665c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f2064000c00 tx=0x7f2064016040).stop 2021-11-03T17:52:57.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.409+0000 7f20794a5700 1 -- 172.21.15.27:0/4262956800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20740f6700 msgr2=0x7f20740fba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:52:57.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.409+0000 7f20794a5700 1 --2- 172.21.15.27:0/4262956800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20740f6700 0x7f20740fba60 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7f205c00e7f0 tx=0x7f205c00b040).stop 2021-11-03T17:52:57.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.409+0000 7f20794a5700 1 -- 172.21.15.27:0/4262956800 shutdown_connections 2021-11-03T17:52:57.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.409+0000 7f20794a5700 1 --2- 172.21.15.27:0/4262956800 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2060064100 0x7f20600665c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:57.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.409+0000 7f20794a5700 1 --2- 172.21.15.27:0/4262956800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f20740f5460 0x7f20740fd5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:57.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.409+0000 7f20794a5700 1 --2- 172.21.15.27:0/4262956800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f20740f6700 0x7f20740fba60 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:52:57.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.409+0000 7f20794a5700 1 -- 172.21.15.27:0/4262956800 >> 172.21.15.27:0/4262956800 conn(0x7f20740f0bc0 msgr2=0x7f20740fa150 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:52:57.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.409+0000 7f20794a5700 1 -- 172.21.15.27:0/4262956800 shutdown_connections 2021-11-03T17:52:57.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:52:57.409+0000 7f20794a5700 1 -- 172.21.15.27:0/4262956800 wait complete. 2021-11-03T17:52:57.417 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:52:57.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:57 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2422054558' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:57.518 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:57 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2422054558' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:52:58.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:58 smithi115 ceph-mon[65453]: pgmap v1149: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:58.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:58 smithi115 ceph-mon[65453]: from='client.35288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:52:58.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:58 smithi115 ceph-mon[65453]: from='client.35292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:52:58.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:52:58 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/4262956800' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:52:58.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:58 smithi027 ceph-mon[97244]: pgmap v1149: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:52:58.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:58 smithi027 ceph-mon[97244]: from='client.35288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:52:58.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:58 smithi027 ceph-mon[97244]: from='client.35292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:52:58.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:52:58 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/4262956800' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:53:00.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:00 smithi115 ceph-mon[65453]: pgmap v1150: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:00.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2222.412903 secs 2021-11-03T17:53:00.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:00 smithi027 ceph-mon[97244]: pgmap v1150: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:00.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2222.412903 secs 2021-11-03T17:53:02.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:02 smithi115 ceph-mon[65453]: pgmap v1151: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:02.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:02 smithi027 ceph-mon[97244]: pgmap v1151: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:04.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:04 smithi115 ceph-mon[65453]: pgmap v1152: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:04.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:53:04.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:53:04.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:53:04.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:04.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:04.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:04.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:04.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:04 smithi027 ceph-mon[97244]: pgmap v1152: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:04.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:53:04.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:53:04.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:53:04.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:04.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:04.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:04.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:05.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:05 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:53:05.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2227.413018 secs 2021-11-03T17:53:05.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:05 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:53:05.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2227.413018 secs 2021-11-03T17:53:06.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:06 smithi115 ceph-mon[65453]: pgmap v1153: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:06.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:06 smithi027 ceph-mon[97244]: pgmap v1153: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:08.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:08 smithi115 ceph-mon[65453]: pgmap v1154: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:08.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:08 smithi027 ceph-mon[97244]: pgmap v1154: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:10.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:10 smithi115 ceph-mon[65453]: pgmap v1155: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:10.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2232.413143 secs 2021-11-03T17:53:10.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:10 smithi027 ceph-mon[97244]: pgmap v1155: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:10.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2232.413143 secs 2021-11-03T17:53:12.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:12 smithi115 ceph-mon[65453]: pgmap v1156: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:12.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:12 smithi027 ceph-mon[97244]: pgmap v1156: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:14.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:14 smithi115 ceph-mon[65453]: pgmap v1157: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:14.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:53:14.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:53:14.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:53:14.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:14.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:14.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:14.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:14.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:14 smithi027 ceph-mon[97244]: pgmap v1157: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:14.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:53:14.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:53:14.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:53:14.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:14.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:14.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:14.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:15.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:15 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:53:15.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2237.413290 secs 2021-11-03T17:53:15.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:15 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:53:15.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2237.413290 secs 2021-11-03T17:53:16.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:16 smithi115 ceph-mon[65453]: pgmap v1158: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:16.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:16 smithi027 ceph-mon[97244]: pgmap v1158: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:18.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:18 smithi027 ceph-mon[97244]: pgmap v1159: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:18.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:18 smithi115 ceph-mon[65453]: pgmap v1159: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:20.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:20 smithi027 ceph-mon[97244]: pgmap v1160: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:20.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2242.413457 secs 2021-11-03T17:53:20.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:20 smithi115 ceph-mon[65453]: pgmap v1160: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:20.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2242.413457 secs 2021-11-03T17:53:22.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:22 smithi027 ceph-mon[97244]: pgmap v1161: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:22.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:22 smithi115 ceph-mon[65453]: pgmap v1161: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:24.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:24 smithi027 ceph-mon[97244]: pgmap v1162: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:24.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:24 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:53:24.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:24 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:53:24.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:24 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:53:24.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:24 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:24.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:24 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:24.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:24 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:24.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:24 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:24.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:24 smithi115 ceph-mon[65453]: pgmap v1162: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:24.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:24 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:53:24.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:24 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:53:24.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:24 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:53:24.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:24 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:24.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:24 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:24.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:24 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:24.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:24 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:25.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:25 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:53:25.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2247.413586 secs 2021-11-03T17:53:25.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:25 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:53:25.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2247.413586 secs 2021-11-03T17:53:26.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:26 smithi027 ceph-mon[97244]: pgmap v1163: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:26.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:26 smithi115 ceph-mon[65453]: pgmap v1163: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:27.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.522+0000 7f463055f700 1 -- 172.21.15.27:0/680690036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46280f66e0 msgr2=0x7f46280f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:53:27.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.522+0000 7f463055f700 1 --2- 172.21.15.27:0/680690036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46280f66e0 0x7f46280f6b00 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7f46200099e0 tx=0x7f4620004d10).stop 2021-11-03T17:53:27.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.522+0000 7f463055f700 1 -- 172.21.15.27:0/680690036 shutdown_connections 2021-11-03T17:53:27.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.522+0000 7f463055f700 1 --2- 172.21.15.27:0/680690036 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46280f6ff0 0x7f46280f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:27.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.522+0000 7f463055f700 1 --2- 172.21.15.27:0/680690036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46280f66e0 0x7f46280f6b00 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:27.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.522+0000 7f463055f700 1 -- 172.21.15.27:0/680690036 >> 172.21.15.27:0/680690036 conn(0x7f46280f1f00 msgr2=0x7f46280f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:53:27.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.522+0000 7f463055f700 1 -- 172.21.15.27:0/680690036 shutdown_connections 2021-11-03T17:53:27.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.522+0000 7f463055f700 1 -- 172.21.15.27:0/680690036 wait complete. 2021-11-03T17:53:27.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.523+0000 7f463055f700 1 Processor -- start 2021-11-03T17:53:27.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.523+0000 7f463055f700 1 -- start start 2021-11-03T17:53:27.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.523+0000 7f463055f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46280f66e0 0x7f4628107070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:53:27.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.523+0000 7f463055f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46280f6ff0 0x7f4628107560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:53:27.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.523+0000 7f463055f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4628107b10 con 0x7f46280f6ff0 2021-11-03T17:53:27.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.523+0000 7f463055f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4628107c50 con 0x7f46280f66e0 2021-11-03T17:53:27.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.523+0000 7f462dafa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46280f6ff0 0x7f4628107560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:53:27.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.523+0000 7f462dafa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46280f6ff0 0x7f4628107560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59960/0 (socket says 172.21.15.27:59960) 2021-11-03T17:53:27.524 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.523+0000 7f462dafa700 1 -- 172.21.15.27:0/188411470 learned_addr learned my addr 172.21.15.27:0/188411470 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:53:27.524 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.523+0000 7f462e2fb700 1 --2- 172.21.15.27:0/188411470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46280f66e0 0x7f4628107070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:53:27.524 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.524+0000 7f462dafa700 1 -- 172.21.15.27:0/188411470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46280f66e0 msgr2=0x7f4628107070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:53:27.524 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.524+0000 7f462dafa700 1 --2- 172.21.15.27:0/188411470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46280f66e0 0x7f4628107070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:27.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.524+0000 7f462dafa700 1 -- 172.21.15.27:0/188411470 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4620009640 con 0x7f46280f6ff0 2021-11-03T17:53:27.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.524+0000 7f462dafa700 1 --2- 172.21.15.27:0/188411470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46280f6ff0 0x7f4628107560 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f461c0026d0 tx=0x7f461c009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:53:27.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.524+0000 7f4626ffd700 1 -- 172.21.15.27:0/188411470 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f461c0079c0 con 0x7f46280f6ff0 2021-11-03T17:53:27.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.524+0000 7f4626ffd700 1 -- 172.21.15.27:0/188411470 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f461c00f040 con 0x7f46280f6ff0 2021-11-03T17:53:27.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.524+0000 7f463055f700 1 -- 172.21.15.27:0/188411470 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46281085f0 con 0x7f46280f6ff0 2021-11-03T17:53:27.526 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.524+0000 7f4626ffd700 1 -- 172.21.15.27:0/188411470 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f461c0148b0 con 0x7f46280f6ff0 2021-11-03T17:53:27.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.524+0000 7f463055f700 1 -- 172.21.15.27:0/188411470 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4628108b30 con 0x7f46280f6ff0 2021-11-03T17:53:27.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.525+0000 7f463055f700 1 -- 172.21.15.27:0/188411470 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4628045ad0 con 0x7f46280f6ff0 2021-11-03T17:53:27.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.526+0000 7f4626ffd700 1 -- 172.21.15.27:0/188411470 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f461c007b20 con 0x7f46280f6ff0 2021-11-03T17:53:27.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.526+0000 7f4626ffd700 1 --2- 172.21.15.27:0/188411470 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f46100640b0 0x7f4610066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:53:27.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.526+0000 7f4626ffd700 1 -- 172.21.15.27:0/188411470 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f461c080790 con 0x7f46280f6ff0 2021-11-03T17:53:27.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.526+0000 7f462e2fb700 1 --2- 172.21.15.27:0/188411470 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f46100640b0 0x7f4610066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:53:27.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.527+0000 7f462e2fb700 1 --2- 172.21.15.27:0/188411470 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f46100640b0 0x7f4610066570 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f4620003380 tx=0x7f4620004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:53:27.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.530+0000 7f4626ffd700 1 -- 172.21.15.27:0/188411470 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f461c052a90 con 0x7f46280f6ff0 2021-11-03T17:53:27.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.709+0000 7f463055f700 1 -- 172.21.15.27:0/188411470 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f46280ff080 con 0x7f46100640b0 2021-11-03T17:53:27.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.710+0000 7f4626ffd700 1 -- 172.21.15.27:0/188411470 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f4620019040 con 0x7f46100640b0 2021-11-03T17:53:27.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.712+0000 7f463055f700 1 -- 172.21.15.27:0/188411470 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f46100640b0 msgr2=0x7f4610066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:53:27.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.712+0000 7f463055f700 1 --2- 172.21.15.27:0/188411470 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f46100640b0 0x7f4610066570 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f4620003380 tx=0x7f4620004d10).stop 2021-11-03T17:53:27.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.712+0000 7f463055f700 1 -- 172.21.15.27:0/188411470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46280f6ff0 msgr2=0x7f4628107560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:53:27.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.712+0000 7f463055f700 1 --2- 172.21.15.27:0/188411470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46280f6ff0 0x7f4628107560 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f461c0026d0 tx=0x7f461c009de0).stop 2021-11-03T17:53:27.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.712+0000 7f463055f700 1 -- 172.21.15.27:0/188411470 shutdown_connections 2021-11-03T17:53:27.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.712+0000 7f463055f700 1 --2- 172.21.15.27:0/188411470 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f46100640b0 0x7f4610066570 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:27.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.712+0000 7f463055f700 1 --2- 172.21.15.27:0/188411470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f46280f66e0 0x7f4628107070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:27.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.712+0000 7f463055f700 1 --2- 172.21.15.27:0/188411470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f46280f6ff0 0x7f4628107560 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:27.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.712+0000 7f463055f700 1 -- 172.21.15.27:0/188411470 >> 172.21.15.27:0/188411470 conn(0x7f46280f1f00 msgr2=0x7f46280fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:53:27.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.712+0000 7f463055f700 1 -- 172.21.15.27:0/188411470 shutdown_connections 2021-11-03T17:53:27.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.712+0000 7f463055f700 1 -- 172.21.15.27:0/188411470 wait complete. 2021-11-03T17:53:27.718 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:53:27.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.812+0000 7f818ff81700 1 -- 172.21.15.27:0/2633395225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81880f7960 msgr2=0x7f81880f7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:53:27.811 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.812+0000 7f818ff81700 1 --2- 172.21.15.27:0/2633395225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81880f7960 0x7f81880f7dc0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f8178004660 tx=0x7f8178009b30).stop 2021-11-03T17:53:27.811 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.813+0000 7f818ff81700 1 -- 172.21.15.27:0/2633395225 shutdown_connections 2021-11-03T17:53:27.811 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.813+0000 7f818ff81700 1 --2- 172.21.15.27:0/2633395225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81880f7960 0x7f81880f7dc0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:27.812 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.813+0000 7f818ff81700 1 --2- 172.21.15.27:0/2633395225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81880f66c0 0x7f81880f6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:27.812 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.813+0000 7f818ff81700 1 -- 172.21.15.27:0/2633395225 >> 172.21.15.27:0/2633395225 conn(0x7f81880f1e20 msgr2=0x7f81880f4240 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:53:27.812 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.813+0000 7f818ff81700 1 -- 172.21.15.27:0/2633395225 shutdown_connections 2021-11-03T17:53:27.812 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.813+0000 7f818ff81700 1 -- 172.21.15.27:0/2633395225 wait complete. 2021-11-03T17:53:27.813 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.813+0000 7f818ff81700 1 Processor -- start 2021-11-03T17:53:27.813 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.813+0000 7f818ff81700 1 -- start start 2021-11-03T17:53:27.813 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.813+0000 7f818ff81700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81880f66c0 0x7f8188106ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:53:27.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.813+0000 7f818ff81700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81880f7960 0x7f81881073b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:53:27.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.813+0000 7f818ff81700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8188107960 con 0x7f81880f7960 2021-11-03T17:53:27.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.813+0000 7f818ff81700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8188107aa0 con 0x7f81880f66c0 2021-11-03T17:53:27.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.813+0000 7f818d51c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81880f7960 0x7f81881073b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:53:27.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.813+0000 7f818d51c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81880f7960 0x7f81881073b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59970/0 (socket says 172.21.15.27:59970) 2021-11-03T17:53:27.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.813+0000 7f818d51c700 1 -- 172.21.15.27:0/4025484969 learned_addr learned my addr 172.21.15.27:0/4025484969 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:53:27.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.814+0000 7f818dd1d700 1 --2- 172.21.15.27:0/4025484969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81880f66c0 0x7f8188106ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:53:27.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.814+0000 7f818d51c700 1 -- 172.21.15.27:0/4025484969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81880f66c0 msgr2=0x7f8188106ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:53:27.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.814+0000 7f818d51c700 1 --2- 172.21.15.27:0/4025484969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81880f66c0 0x7f8188106ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:27.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.814+0000 7f818d51c700 1 -- 172.21.15.27:0/4025484969 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8178005040 con 0x7f81880f7960 2021-11-03T17:53:27.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.814+0000 7f818dd1d700 1 --2- 172.21.15.27:0/4025484969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81880f66c0 0x7f8188106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:53:27.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.814+0000 7f818d51c700 1 --2- 172.21.15.27:0/4025484969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81880f7960 0x7f81881073b0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f8178000c00 tx=0x7f817800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:53:27.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.814+0000 7f817e7fc700 1 -- 172.21.15.27:0/4025484969 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8178019070 con 0x7f81880f7960 2021-11-03T17:53:27.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.814+0000 7f818ff81700 1 -- 172.21.15.27:0/4025484969 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81881083e0 con 0x7f81880f7960 2021-11-03T17:53:27.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.814+0000 7f817e7fc700 1 -- 172.21.15.27:0/4025484969 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f8178007e70 con 0x7f81880f7960 2021-11-03T17:53:27.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.814+0000 7f817e7fc700 1 -- 172.21.15.27:0/4025484969 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8178014d90 con 0x7f81880f7960 2021-11-03T17:53:27.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.814+0000 7f818ff81700 1 -- 172.21.15.27:0/4025484969 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8188108940 con 0x7f81880f7960 2021-11-03T17:53:27.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.815+0000 7f818ff81700 1 -- 172.21.15.27:0/4025484969 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8188045ad0 con 0x7f81880f7960 2021-11-03T17:53:27.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.815+0000 7f817e7fc700 1 -- 172.21.15.27:0/4025484969 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f8178025070 con 0x7f81880f7960 2021-11-03T17:53:27.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.816+0000 7f817e7fc700 1 --2- 172.21.15.27:0/4025484969 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f81740640b0 0x7f8174066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:53:27.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.816+0000 7f817e7fc700 1 -- 172.21.15.27:0/4025484969 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f8178081800 con 0x7f81880f7960 2021-11-03T17:53:27.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.816+0000 7f818dd1d700 1 --2- 172.21.15.27:0/4025484969 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f81740640b0 0x7f8174066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:53:27.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.816+0000 7f818dd1d700 1 --2- 172.21.15.27:0/4025484969 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f81740640b0 0x7f8174066570 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f81880f77c0 tx=0x7f818400b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:53:27.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.820+0000 7f817e7fc700 1 -- 172.21.15.27:0/4025484969 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f8178053b00 con 0x7f81880f7960 2021-11-03T17:53:27.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:27.995+0000 7f818ff81700 1 -- 172.21.15.27:0/4025484969 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8188108b60 con 0x7f81740640b0 2021-11-03T17:53:28.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.009+0000 7f817e7fc700 1 -- 172.21.15.27:0/4025484969 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f8188108b60 con 0x7f81740640b0 2021-11-03T17:53:28.006 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:53:28.006 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (45m) 4m ago 46m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:53:28.006 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (36m) 4m ago 46m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:53:28.007 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (36m) 3m ago 45m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:53:28.007 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (44m) 4m ago 45m 41.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:53:28.007 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (38m) 4m ago 41m 23.8M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:53:28.007 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (41m) 4m ago 41m 3844M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:53:28.008 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (37m) 3m ago 41m 23.2M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:53:28.008 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (38m) 3m ago 41m 24.9M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:53:28.008 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (38m) 4m ago 47m 535M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:53:28.008 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (38m) 3m ago 45m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:53:28.008 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (37m) 4m ago 47m 198M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:53:28.009 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (37m) 3m ago 45m 95.6M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:53:28.009 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (45m) 4m ago 45m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:53:28.009 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (45m) 3m ago 45m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:53:28.009 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (36m) 4m ago 44m 331M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:53:28.009 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (35m) 4m ago 44m 361M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:53:28.010 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (35m) 4m ago 43m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:53:28.010 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (34m) 3m ago 43m 379M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:53:28.010 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (34m) 3m ago 43m 304M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:53:28.010 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (33m) 3m ago 43m 307M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:53:28.011 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (44m) 4m ago 45m 52.9M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:53:28.011 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.011+0000 7f818ff81700 1 -- 172.21.15.27:0/4025484969 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f81740640b0 msgr2=0x7f8174066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:53:28.012 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.011+0000 7f818ff81700 1 --2- 172.21.15.27:0/4025484969 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f81740640b0 0x7f8174066570 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f81880f77c0 tx=0x7f818400b040).stop 2021-11-03T17:53:28.012 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.011+0000 7f818ff81700 1 -- 172.21.15.27:0/4025484969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81880f7960 msgr2=0x7f81881073b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:53:28.012 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.011+0000 7f818ff81700 1 --2- 172.21.15.27:0/4025484969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81880f7960 0x7f81881073b0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f8178000c00 tx=0x7f817800b040).stop 2021-11-03T17:53:28.012 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.012+0000 7f818ff81700 1 -- 172.21.15.27:0/4025484969 shutdown_connections 2021-11-03T17:53:28.013 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.012+0000 7f818ff81700 1 --2- 172.21.15.27:0/4025484969 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f81740640b0 0x7f8174066570 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:28.013 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.012+0000 7f818ff81700 1 --2- 172.21.15.27:0/4025484969 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81880f66c0 0x7f8188106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:28.013 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.012+0000 7f818ff81700 1 --2- 172.21.15.27:0/4025484969 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81880f7960 0x7f81881073b0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:28.014 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.012+0000 7f818ff81700 1 -- 172.21.15.27:0/4025484969 >> 172.21.15.27:0/4025484969 conn(0x7f81880f1e20 msgr2=0x7f81880fb3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:53:28.014 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.012+0000 7f818ff81700 1 -- 172.21.15.27:0/4025484969 shutdown_connections 2021-11-03T17:53:28.014 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.012+0000 7f818ff81700 1 -- 172.21.15.27:0/4025484969 wait complete. 2021-11-03T17:53:28.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.111+0000 7fea906d3700 1 -- 172.21.15.27:0/2315955327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea880f5ed0 msgr2=0x7fea880f6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:53:28.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.111+0000 7fea906d3700 1 --2- 172.21.15.27:0/2315955327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea880f5ed0 0x7fea880f6900 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7fea7c004660 tx=0x7fea7c009b30).stop 2021-11-03T17:53:28.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.111+0000 7fea906d3700 1 -- 172.21.15.27:0/2315955327 shutdown_connections 2021-11-03T17:53:28.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.111+0000 7fea906d3700 1 --2- 172.21.15.27:0/2315955327 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea880f5ed0 0x7fea880f6900 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:28.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.111+0000 7fea906d3700 1 --2- 172.21.15.27:0/2315955327 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea880f55c0 0x7fea880f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:28.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.111+0000 7fea906d3700 1 -- 172.21.15.27:0/2315955327 >> 172.21.15.27:0/2315955327 conn(0x7fea880f0dc0 msgr2=0x7fea880f31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:53:28.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.111+0000 7fea906d3700 1 -- 172.21.15.27:0/2315955327 shutdown_connections 2021-11-03T17:53:28.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.111+0000 7fea906d3700 1 -- 172.21.15.27:0/2315955327 wait complete. 2021-11-03T17:53:28.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.111+0000 7fea906d3700 1 Processor -- start 2021-11-03T17:53:28.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.111+0000 7fea906d3700 1 -- start start 2021-11-03T17:53:28.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.112+0000 7fea906d3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea880f55c0 0x7fea88109bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:53:28.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.112+0000 7fea906d3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea880f5ed0 0x7fea8810c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:53:28.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.112+0000 7fea906d3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea88060560 con 0x7fea880f55c0 2021-11-03T17:53:28.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.112+0000 7fea906d3700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea880606a0 con 0x7fea880f5ed0 2021-11-03T17:53:28.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.112+0000 7fea8e46f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea880f55c0 0x7fea88109bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:53:28.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.112+0000 7fea8e46f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea880f55c0 0x7fea88109bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59978/0 (socket says 172.21.15.27:59978) 2021-11-03T17:53:28.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.112+0000 7fea8e46f700 1 -- 172.21.15.27:0/1675729957 learned_addr learned my addr 172.21.15.27:0/1675729957 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:53:28.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.112+0000 7fea8dc6e700 1 --2- 172.21.15.27:0/1675729957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea880f5ed0 0x7fea8810c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:53:28.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.112+0000 7fea8e46f700 1 -- 172.21.15.27:0/1675729957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea880f5ed0 msgr2=0x7fea8810c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:53:28.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.112+0000 7fea8e46f700 1 --2- 172.21.15.27:0/1675729957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea880f5ed0 0x7fea8810c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:28.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.112+0000 7fea8e46f700 1 -- 172.21.15.27:0/1675729957 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea7c005040 con 0x7fea880f55c0 2021-11-03T17:53:28.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.112+0000 7fea8dc6e700 1 --2- 172.21.15.27:0/1675729957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea880f5ed0 0x7fea8810c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:53:28.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.112+0000 7fea8e46f700 1 --2- 172.21.15.27:0/1675729957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea880f55c0 0x7fea88109bd0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7fea8000eb70 tx=0x7fea8000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:53:28.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.112+0000 7fea86ffd700 1 -- 172.21.15.27:0/1675729957 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea800092e0 con 0x7fea880f55c0 2021-11-03T17:53:28.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.112+0000 7fea86ffd700 1 -- 172.21.15.27:0/1675729957 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fea8000f040 con 0x7fea880f55c0 2021-11-03T17:53:28.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.112+0000 7fea906d3700 1 -- 172.21.15.27:0/1675729957 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea8810c680 con 0x7fea880f55c0 2021-11-03T17:53:28.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.112+0000 7fea86ffd700 1 -- 172.21.15.27:0/1675729957 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fea8000c9a0 con 0x7fea880f55c0 2021-11-03T17:53:28.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.112+0000 7fea906d3700 1 -- 172.21.15.27:0/1675729957 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea8810cc40 con 0x7fea880f55c0 2021-11-03T17:53:28.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.113+0000 7fea906d3700 1 -- 172.21.15.27:0/1675729957 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea88045ad0 con 0x7fea880f55c0 2021-11-03T17:53:28.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.114+0000 7fea86ffd700 1 -- 172.21.15.27:0/1675729957 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fea8001b030 con 0x7fea880f55c0 2021-11-03T17:53:28.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.114+0000 7fea86ffd700 1 --2- 172.21.15.27:0/1675729957 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fea700640b0 0x7fea70066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:53:28.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.114+0000 7fea86ffd700 1 -- 172.21.15.27:0/1675729957 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fea80080da0 con 0x7fea880f55c0 2021-11-03T17:53:28.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.114+0000 7fea8dc6e700 1 --2- 172.21.15.27:0/1675729957 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fea700640b0 0x7fea70066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:53:28.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.115+0000 7fea8dc6e700 1 --2- 172.21.15.27:0/1675729957 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fea700640b0 0x7fea70066570 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7fea7c004d80 tx=0x7fea7c00b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:53:28.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.118+0000 7fea86ffd700 1 -- 172.21.15.27:0/1675729957 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fea80052fb0 con 0x7fea880f55c0 2021-11-03T17:53:28.347 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.350+0000 7fea906d3700 1 -- 172.21.15.27:0/1675729957 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fea880fde50 con 0x7fea880f55c0 2021-11-03T17:53:28.347 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.351+0000 7fea86ffd700 1 -- 172.21.15.27:0/1675729957 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fea800192e0 con 0x7fea880f55c0 2021-11-03T17:53:28.348 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:53:28.348 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:53:28.348 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:53:28.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:53:28.349 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:53:28.349 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:53:28.349 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:53:28.349 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:53:28.350 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:53:28.350 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:53:28.350 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:53:28.350 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:53:28.351 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:53:28.351 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:53:28.351 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:53:28.351 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:53:28.352 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:53:28.352 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:53:28.353 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.352+0000 7fea906d3700 1 -- 172.21.15.27:0/1675729957 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fea700640b0 msgr2=0x7fea70066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:53:28.353 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.352+0000 7fea906d3700 1 --2- 172.21.15.27:0/1675729957 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fea700640b0 0x7fea70066570 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7fea7c004d80 tx=0x7fea7c00b040).stop 2021-11-03T17:53:28.353 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.352+0000 7fea906d3700 1 -- 172.21.15.27:0/1675729957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea880f55c0 msgr2=0x7fea88109bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:53:28.353 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.352+0000 7fea906d3700 1 --2- 172.21.15.27:0/1675729957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea880f55c0 0x7fea88109bd0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7fea8000eb70 tx=0x7fea8000b040).stop 2021-11-03T17:53:28.354 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.353+0000 7fea906d3700 1 -- 172.21.15.27:0/1675729957 shutdown_connections 2021-11-03T17:53:28.354 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.353+0000 7fea906d3700 1 --2- 172.21.15.27:0/1675729957 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fea700640b0 0x7fea70066570 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:28.354 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.353+0000 7fea906d3700 1 --2- 172.21.15.27:0/1675729957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fea880f55c0 0x7fea88109bd0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:28.354 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.353+0000 7fea906d3700 1 --2- 172.21.15.27:0/1675729957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fea880f5ed0 0x7fea8810c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:28.355 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.353+0000 7fea906d3700 1 -- 172.21.15.27:0/1675729957 >> 172.21.15.27:0/1675729957 conn(0x7fea880f0dc0 msgr2=0x7fea880fa350 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:53:28.355 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.353+0000 7fea906d3700 1 -- 172.21.15.27:0/1675729957 shutdown_connections 2021-11-03T17:53:28.355 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.353+0000 7fea906d3700 1 -- 172.21.15.27:0/1675729957 wait complete. 2021-11-03T17:53:28.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.453+0000 7f1ec5d50700 1 -- 172.21.15.27:0/3964810255 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec00f66e0 msgr2=0x7f1ec00f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:53:28.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.453+0000 7f1ec5d50700 1 --2- 172.21.15.27:0/3964810255 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec00f66e0 0x7f1ec00f6b00 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f1eb4004660 tx=0x7f1eb4009b30).stop 2021-11-03T17:53:28.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.453+0000 7f1ec5d50700 1 -- 172.21.15.27:0/3964810255 shutdown_connections 2021-11-03T17:53:28.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.453+0000 7f1ec5d50700 1 --2- 172.21.15.27:0/3964810255 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ec00f6ff0 0x7f1ec00f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:28.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.453+0000 7f1ec5d50700 1 --2- 172.21.15.27:0/3964810255 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec00f66e0 0x7f1ec00f6b00 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:28.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.453+0000 7f1ec5d50700 1 -- 172.21.15.27:0/3964810255 >> 172.21.15.27:0/3964810255 conn(0x7f1ec00f1f00 msgr2=0x7f1ec00f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:53:28.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.453+0000 7f1ec5d50700 1 -- 172.21.15.27:0/3964810255 shutdown_connections 2021-11-03T17:53:28.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.453+0000 7f1ec5d50700 1 -- 172.21.15.27:0/3964810255 wait complete. 2021-11-03T17:53:28.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.453+0000 7f1ec5d50700 1 Processor -- start 2021-11-03T17:53:28.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.454+0000 7f1ec5d50700 1 -- start start 2021-11-03T17:53:28.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.454+0000 7f1ec5d50700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ec00f66e0 0x7f1ec0106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:53:28.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.454+0000 7f1ec5d50700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec00f6ff0 0x7f1ec0107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:53:28.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.454+0000 7f1ec5d50700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec0107a00 con 0x7f1ec00f6ff0 2021-11-03T17:53:28.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.454+0000 7f1ec5d50700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec0107b40 con 0x7f1ec00f66e0 2021-11-03T17:53:28.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.454+0000 7f1ebeffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec00f6ff0 0x7f1ec0107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:53:28.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.454+0000 7f1ebeffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec00f6ff0 0x7f1ec0107450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:59990/0 (socket says 172.21.15.27:59990) 2021-11-03T17:53:28.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.454+0000 7f1ebeffd700 1 -- 172.21.15.27:0/760754069 learned_addr learned my addr 172.21.15.27:0/760754069 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:53:28.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.454+0000 7f1ebf7fe700 1 --2- 172.21.15.27:0/760754069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ec00f66e0 0x7f1ec0106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:53:28.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.454+0000 7f1ebeffd700 1 -- 172.21.15.27:0/760754069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ec00f66e0 msgr2=0x7f1ec0106f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:53:28.457 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.454+0000 7f1ebeffd700 1 --2- 172.21.15.27:0/760754069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ec00f66e0 0x7f1ec0106f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:28.457 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.454+0000 7f1ebeffd700 1 -- 172.21.15.27:0/760754069 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb4005040 con 0x7f1ec00f6ff0 2021-11-03T17:53:28.457 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.454+0000 7f1ebf7fe700 1 --2- 172.21.15.27:0/760754069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ec00f66e0 0x7f1ec0106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:53:28.457 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.454+0000 7f1ebeffd700 1 --2- 172.21.15.27:0/760754069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec00f6ff0 0x7f1ec0107450 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f1eb000c790 tx=0x7f1eb0007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:53:28.457 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.455+0000 7f1ea7fff700 1 -- 172.21.15.27:0/760754069 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1eb000e680 con 0x7f1ec00f6ff0 2021-11-03T17:53:28.458 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.455+0000 7f1ec5d50700 1 -- 172.21.15.27:0/760754069 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ec01084e0 con 0x7f1ec00f6ff0 2021-11-03T17:53:28.458 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.455+0000 7f1ea7fff700 1 -- 172.21.15.27:0/760754069 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f1eb000ec90 con 0x7f1ec00f6ff0 2021-11-03T17:53:28.458 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.455+0000 7f1ea7fff700 1 -- 172.21.15.27:0/760754069 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1eb0013d60 con 0x7f1ec00f6ff0 2021-11-03T17:53:28.458 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.455+0000 7f1ec5d50700 1 -- 172.21.15.27:0/760754069 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ec0108aa0 con 0x7f1ec00f6ff0 2021-11-03T17:53:28.459 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.456+0000 7f1ec5d50700 1 -- 172.21.15.27:0/760754069 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ec0045ad0 con 0x7f1ec00f6ff0 2021-11-03T17:53:28.459 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.456+0000 7f1ea7fff700 1 -- 172.21.15.27:0/760754069 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f1eb001a060 con 0x7f1ec00f6ff0 2021-11-03T17:53:28.459 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.456+0000 7f1ea7fff700 1 --2- 172.21.15.27:0/760754069 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1ea80640b0 0x7f1ea8066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:53:28.459 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.457+0000 7f1ea7fff700 1 -- 172.21.15.27:0/760754069 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f1eb0080b30 con 0x7f1ec00f6ff0 2021-11-03T17:53:28.460 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.457+0000 7f1ebf7fe700 1 --2- 172.21.15.27:0/760754069 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1ea80640b0 0x7f1ea8066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:53:28.460 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.457+0000 7f1ebf7fe700 1 --2- 172.21.15.27:0/760754069 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1ea80640b0 0x7f1ea8066570 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f1eb4004e30 tx=0x7f1eb4009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:53:28.460 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.461+0000 7f1ea7fff700 1 -- 172.21.15.27:0/760754069 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f1eb0052d40 con 0x7f1ec00f6ff0 2021-11-03T17:53:28.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:28 smithi027 ceph-mon[97244]: pgmap v1164: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:28.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.664+0000 7f1ec5d50700 1 -- 172.21.15.27:0/760754069 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1ec00feed0 con 0x7f1ec00f6ff0 2021-11-03T17:53:28.661 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.664+0000 7f1ea7fff700 1 -- 172.21.15.27:0/760754069 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f1eb0018070 con 0x7f1ec00f6ff0 2021-11-03T17:53:28.661 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:53:28.661 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:53:28.662 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:53:28.662 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:53:28.662 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:53:28.662 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:53:28.663 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:53:28.663 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:53:28.663 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:53:28.663 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:53:28.664 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:53:28.664 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:53:28.664 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:53:28.665 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:53:28.665 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:53:28.665 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:53:28.665 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:53:28.665 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:53:28.666 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:53:28.666 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:53:28.666 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:53:28.666 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:53:28.667 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:53:28.667 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:53:28.667 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:53:28.667 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:53:28.667 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:53:28.668 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:53:28.668 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:53:28.668 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:53:28.668 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:53:28.669 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:53:28.669 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:53:28.669 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:53:28.669 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:53:28.669 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:53:28.670 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:53:28.670 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:53:28.670 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:53:28.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.666+0000 7f1ec5d50700 1 -- 172.21.15.27:0/760754069 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1ea80640b0 msgr2=0x7f1ea8066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:53:28.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.666+0000 7f1ec5d50700 1 --2- 172.21.15.27:0/760754069 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1ea80640b0 0x7f1ea8066570 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 rx=0x7f1eb4004e30 tx=0x7f1eb4009b30).stop 2021-11-03T17:53:28.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.666+0000 7f1ec5d50700 1 -- 172.21.15.27:0/760754069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec00f6ff0 msgr2=0x7f1ec0107450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:53:28.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.666+0000 7f1ec5d50700 1 --2- 172.21.15.27:0/760754069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec00f6ff0 0x7f1ec0107450 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f1eb000c790 tx=0x7f1eb0007d70).stop 2021-11-03T17:53:28.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.666+0000 7f1ec5d50700 1 -- 172.21.15.27:0/760754069 shutdown_connections 2021-11-03T17:53:28.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.666+0000 7f1ec5d50700 1 --2- 172.21.15.27:0/760754069 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1ea80640b0 0x7f1ea8066570 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:28.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.666+0000 7f1ec5d50700 1 --2- 172.21.15.27:0/760754069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ec00f66e0 0x7f1ec0106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:28.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.666+0000 7f1ec5d50700 1 --2- 172.21.15.27:0/760754069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ec00f6ff0 0x7f1ec0107450 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:28.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.666+0000 7f1ec5d50700 1 -- 172.21.15.27:0/760754069 >> 172.21.15.27:0/760754069 conn(0x7f1ec00f1f00 msgr2=0x7f1ec00fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:53:28.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.666+0000 7f1ec5d50700 1 -- 172.21.15.27:0/760754069 shutdown_connections 2021-11-03T17:53:28.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:28.666+0000 7f1ec5d50700 1 -- 172.21.15.27:0/760754069 wait complete. 2021-11-03T17:53:28.674 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:53:28.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:28 smithi115 ceph-mon[65453]: pgmap v1164: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:29.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:29 smithi027 ceph-mon[97244]: from='client.35304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:53:29.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:29 smithi027 ceph-mon[97244]: from='client.35308 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:53:29.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:29 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1675729957' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:29.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:29 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/760754069' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:53:29.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:29 smithi115 ceph-mon[65453]: from='client.35304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:53:29.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:29 smithi115 ceph-mon[65453]: from='client.35308 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:53:29.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:29 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1675729957' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:29.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:29 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/760754069' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:53:30.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:30 smithi027 ceph-mon[97244]: pgmap v1165: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:30.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2252.413676 secs 2021-11-03T17:53:30.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:30 smithi115 ceph-mon[65453]: pgmap v1165: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:30.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2252.413676 secs 2021-11-03T17:53:32.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:32 smithi027 ceph-mon[97244]: pgmap v1166: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:32.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:32 smithi115 ceph-mon[65453]: pgmap v1166: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:34.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:34 smithi027 ceph-mon[97244]: pgmap v1167: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:34.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:53:34.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:53:34.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:53:34.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:34.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:34.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:34.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:34.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:34 smithi115 ceph-mon[65453]: pgmap v1167: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:34.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:53:34.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:53:34.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:53:34.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:34.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:34.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:34.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:35.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:35 smithi027 ceph-mon[97244]: poll took 60173.257 ms: 1 events 2021-11-03T17:53:35.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:35 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:53:35.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2257.413754 secs 2021-11-03T17:53:35.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:35 smithi115 ceph-mon[65453]: poll took 60173.257 ms: 1 events 2021-11-03T17:53:35.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:35 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:53:35.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2257.413754 secs 2021-11-03T17:53:36.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:36 smithi115 ceph-mon[65453]: pgmap v1168: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:36.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:36 smithi027 ceph-mon[97244]: pgmap v1168: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:38.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:38 smithi115 ceph-mon[65453]: pgmap v1169: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:38.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:38 smithi027 ceph-mon[97244]: pgmap v1169: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:40.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:40 smithi115 ceph-mon[65453]: pgmap v1170: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:40.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2262.413867 secs 2021-11-03T17:53:40.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:40 smithi027 ceph-mon[97244]: pgmap v1170: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:40.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2262.413867 secs 2021-11-03T17:53:42.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:42 smithi115 ceph-mon[65453]: pgmap v1171: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:42.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:42 smithi027 ceph-mon[97244]: pgmap v1171: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:44.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:44 smithi115 ceph-mon[65453]: pgmap v1172: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:44.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:53:44.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:53:44.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:53:44.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:44.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:44.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:44.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:44.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:44 smithi027 ceph-mon[97244]: pgmap v1172: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:44.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:53:44.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:53:44.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:53:44.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:44.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:44.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:44.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:45.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:45 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:53:45.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2267.414020 secs 2021-11-03T17:53:45.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:45 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:53:45.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2267.414020 secs 2021-11-03T17:53:46.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:46 smithi115 ceph-mon[65453]: pgmap v1173: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:46.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:46 smithi027 ceph-mon[97244]: pgmap v1173: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:48.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:48 smithi115 ceph-mon[65453]: pgmap v1174: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:48.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:48 smithi027 ceph-mon[97244]: pgmap v1174: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:50.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:50 smithi115 ceph-mon[65453]: pgmap v1175: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:50.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2272.414134 secs 2021-11-03T17:53:50.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:50 smithi027 ceph-mon[97244]: pgmap v1175: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:50.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2272.414134 secs 2021-11-03T17:53:52.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:52 smithi115 ceph-mon[65453]: pgmap v1176: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:52.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:52 smithi027 ceph-mon[97244]: pgmap v1176: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:54.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:54 smithi115 ceph-mon[65453]: pgmap v1177: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:54.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:53:54.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:53:54.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:53:54.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:54.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:54.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:54.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:54.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:54 smithi027 ceph-mon[97244]: pgmap v1177: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:54.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:53:54.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:53:54.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:53:54.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:54.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:54.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:54.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:53:55.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:55 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:53:55.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2277.414264 secs 2021-11-03T17:53:55.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:55 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:53:55.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2277.414264 secs 2021-11-03T17:53:56.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:56 smithi115 ceph-mon[65453]: pgmap v1178: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:56.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:53:56.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:53:56.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:56 smithi027 ceph-mon[97244]: pgmap v1178: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:56.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:53:56.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:53:58.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:53:58 smithi115 ceph-mon[65453]: pgmap v1179: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:58.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:53:58 smithi027 ceph-mon[97244]: pgmap v1179: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:53:58.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.783+0000 7f9f9f463700 1 -- 172.21.15.27:0/55122471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f98060310 msgr2=0x7f9f980f9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:53:58.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.783+0000 7f9f9f463700 1 --2- 172.21.15.27:0/55122471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f98060310 0x7f9f980f9120 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7f9f94004660 tx=0x7f9f94009b30).stop 2021-11-03T17:53:58.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.783+0000 7f9f9f463700 1 -- 172.21.15.27:0/55122471 shutdown_connections 2021-11-03T17:53:58.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.783+0000 7f9f9f463700 1 --2- 172.21.15.27:0/55122471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f980f96a0 0x7f9f980fbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:58.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.783+0000 7f9f9f463700 1 --2- 172.21.15.27:0/55122471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f98060310 0x7f9f980f9120 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:58.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.783+0000 7f9f9f463700 1 -- 172.21.15.27:0/55122471 >> 172.21.15.27:0/55122471 conn(0x7f9f980f0ba0 msgr2=0x7f9f980f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:53:58.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.783+0000 7f9f9f463700 1 -- 172.21.15.27:0/55122471 shutdown_connections 2021-11-03T17:53:58.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.783+0000 7f9f9f463700 1 -- 172.21.15.27:0/55122471 wait complete. 2021-11-03T17:53:58.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.784+0000 7f9f9f463700 1 Processor -- start 2021-11-03T17:53:58.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.784+0000 7f9f9f463700 1 -- start start 2021-11-03T17:53:58.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.784+0000 7f9f9f463700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f98060310 0x7f9f98106d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:53:58.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.784+0000 7f9f9f463700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f980f96a0 0x7f9f98107240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:53:58.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.784+0000 7f9f9f463700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f98107840 con 0x7f9f980f96a0 2021-11-03T17:53:58.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.784+0000 7f9f9f463700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f98108050 con 0x7f9f98060310 2021-11-03T17:53:58.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.784+0000 7f9f9dc60700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f980f96a0 0x7f9f98107240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:53:58.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.784+0000 7f9f9dc60700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f980f96a0 0x7f9f98107240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60008/0 (socket says 172.21.15.27:60008) 2021-11-03T17:53:58.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.784+0000 7f9f9dc60700 1 -- 172.21.15.27:0/1512979659 learned_addr learned my addr 172.21.15.27:0/1512979659 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:53:58.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.784+0000 7f9f9dc60700 1 -- 172.21.15.27:0/1512979659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f98060310 msgr2=0x7f9f98106d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:53:58.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.784+0000 7f9f9e461700 1 --2- 172.21.15.27:0/1512979659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f98060310 0x7f9f98106d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:53:58.787 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.784+0000 7f9f9dc60700 1 --2- 172.21.15.27:0/1512979659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f98060310 0x7f9f98106d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:58.787 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.784+0000 7f9f9dc60700 1 -- 172.21.15.27:0/1512979659 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f94005040 con 0x7f9f980f96a0 2021-11-03T17:53:58.787 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.784+0000 7f9f9dc60700 1 --2- 172.21.15.27:0/1512979659 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f980f96a0 0x7f9f98107240 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7f9f880026d0 tx=0x7f9f88009e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:53:58.787 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.785+0000 7f9f8effd700 1 -- 172.21.15.27:0/1512979659 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f88009320 con 0x7f9f980f96a0 2021-11-03T17:53:58.788 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.785+0000 7f9f8effd700 1 -- 172.21.15.27:0/1512979659 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f9f8800f040 con 0x7f9f980f96a0 2021-11-03T17:53:58.788 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.785+0000 7f9f8effd700 1 -- 172.21.15.27:0/1512979659 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f88014870 con 0x7f9f980f96a0 2021-11-03T17:53:58.788 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.785+0000 7f9f9f463700 1 -- 172.21.15.27:0/1512979659 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f981082d0 con 0x7f9f980f96a0 2021-11-03T17:53:58.788 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.785+0000 7f9f9f463700 1 -- 172.21.15.27:0/1512979659 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f98108890 con 0x7f9f980f96a0 2021-11-03T17:53:58.789 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.786+0000 7f9f9f463700 1 -- 172.21.15.27:0/1512979659 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f98059100 con 0x7f9f980f96a0 2021-11-03T17:53:58.789 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.786+0000 7f9f8effd700 1 -- 172.21.15.27:0/1512979659 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f9f88009480 con 0x7f9f980f96a0 2021-11-03T17:53:58.789 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.786+0000 7f9f8effd700 1 --2- 172.21.15.27:0/1512979659 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9f84064100 0x7f9f840665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:53:58.789 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.787+0000 7f9f8effd700 1 -- 172.21.15.27:0/1512979659 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f9f88080a90 con 0x7f9f980f96a0 2021-11-03T17:53:58.790 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.787+0000 7f9f9e461700 1 --2- 172.21.15.27:0/1512979659 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9f84064100 0x7f9f840665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:53:58.790 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.787+0000 7f9f9e461700 1 --2- 172.21.15.27:0/1512979659 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9f84064100 0x7f9f840665c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f9f94009da0 tx=0x7f9f94009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:53:58.791 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.791+0000 7f9f8effd700 1 -- 172.21.15.27:0/1512979659 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f9f88052d90 con 0x7f9f980f96a0 2021-11-03T17:53:58.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.968+0000 7f9f9f463700 1 -- 172.21.15.27:0/1512979659 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f980fee50 con 0x7f9f84064100 2021-11-03T17:53:58.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.970+0000 7f9f8effd700 1 -- 172.21.15.27:0/1512979659 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f9f980fee50 con 0x7f9f84064100 2021-11-03T17:53:58.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.971+0000 7f9f9f463700 1 -- 172.21.15.27:0/1512979659 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9f84064100 msgr2=0x7f9f840665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:53:58.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.971+0000 7f9f9f463700 1 --2- 172.21.15.27:0/1512979659 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9f84064100 0x7f9f840665c0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f9f94009da0 tx=0x7f9f94009b30).stop 2021-11-03T17:53:58.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.971+0000 7f9f9f463700 1 -- 172.21.15.27:0/1512979659 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f980f96a0 msgr2=0x7f9f98107240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:53:58.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.971+0000 7f9f9f463700 1 --2- 172.21.15.27:0/1512979659 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f980f96a0 0x7f9f98107240 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7f9f880026d0 tx=0x7f9f88009e00).stop 2021-11-03T17:53:58.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.972+0000 7f9f9f463700 1 -- 172.21.15.27:0/1512979659 shutdown_connections 2021-11-03T17:53:58.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.972+0000 7f9f9f463700 1 --2- 172.21.15.27:0/1512979659 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9f84064100 0x7f9f840665c0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:58.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.972+0000 7f9f9f463700 1 --2- 172.21.15.27:0/1512979659 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f98060310 0x7f9f98106d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:58.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.972+0000 7f9f9f463700 1 --2- 172.21.15.27:0/1512979659 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f980f96a0 0x7f9f98107240 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:58.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.972+0000 7f9f9f463700 1 -- 172.21.15.27:0/1512979659 >> 172.21.15.27:0/1512979659 conn(0x7f9f980f0ba0 msgr2=0x7f9f980f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:53:58.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.972+0000 7f9f9f463700 1 -- 172.21.15.27:0/1512979659 shutdown_connections 2021-11-03T17:53:58.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:58.972+0000 7f9f9f463700 1 -- 172.21.15.27:0/1512979659 wait complete. 2021-11-03T17:53:58.977 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:53:59.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.072+0000 7f8bfeb1d700 1 -- 172.21.15.27:0/1414245331 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf80f6ff0 msgr2=0x7f8bf80f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:53:59.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.072+0000 7f8bfeb1d700 1 --2- 172.21.15.27:0/1414245331 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf80f6ff0 0x7f8bf80f7a20 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7f8be4004660 tx=0x7f8be4009b30).stop 2021-11-03T17:53:59.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.072+0000 7f8bfeb1d700 1 -- 172.21.15.27:0/1414245331 shutdown_connections 2021-11-03T17:53:59.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.072+0000 7f8bfeb1d700 1 --2- 172.21.15.27:0/1414245331 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf80f6ff0 0x7f8bf80f7a20 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:59.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.072+0000 7f8bfeb1d700 1 --2- 172.21.15.27:0/1414245331 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8bf80f66e0 0x7f8bf80f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:59.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.072+0000 7f8bfeb1d700 1 -- 172.21.15.27:0/1414245331 >> 172.21.15.27:0/1414245331 conn(0x7f8bf80f1f00 msgr2=0x7f8bf80f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:53:59.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.072+0000 7f8bfeb1d700 1 -- 172.21.15.27:0/1414245331 shutdown_connections 2021-11-03T17:53:59.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.072+0000 7f8bfeb1d700 1 -- 172.21.15.27:0/1414245331 wait complete. 2021-11-03T17:53:59.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.073+0000 7f8bfeb1d700 1 Processor -- start 2021-11-03T17:53:59.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.073+0000 7f8bfeb1d700 1 -- start start 2021-11-03T17:53:59.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.073+0000 7f8bfeb1d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8bf80f66e0 0x7f8bf806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:53:59.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.073+0000 7f8bfeb1d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf80f6ff0 0x7f8bf8069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:53:59.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.073+0000 7f8bfeb1d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf806a380 con 0x7f8bf80f6ff0 2021-11-03T17:53:59.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.073+0000 7f8bfeb1d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf806a4c0 con 0x7f8bf80f66e0 2021-11-03T17:53:59.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.073+0000 7f8bf7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf80f6ff0 0x7f8bf8069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:53:59.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.073+0000 7f8bf7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf80f6ff0 0x7f8bf8069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60018/0 (socket says 172.21.15.27:60018) 2021-11-03T17:53:59.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.073+0000 7f8bf7fff700 1 -- 172.21.15.27:0/1415184393 learned_addr learned my addr 172.21.15.27:0/1415184393 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:53:59.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.073+0000 7f8bfc8b9700 1 --2- 172.21.15.27:0/1415184393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8bf80f66e0 0x7f8bf806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:53:59.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.073+0000 7f8bf7fff700 1 -- 172.21.15.27:0/1415184393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8bf80f66e0 msgr2=0x7f8bf806b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:53:59.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.073+0000 7f8bf7fff700 1 --2- 172.21.15.27:0/1415184393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8bf80f66e0 0x7f8bf806b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:59.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.073+0000 7f8bf7fff700 1 -- 172.21.15.27:0/1415184393 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be4005040 con 0x7f8bf80f6ff0 2021-11-03T17:53:59.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.073+0000 7f8bf7fff700 1 --2- 172.21.15.27:0/1415184393 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf80f6ff0 0x7f8bf8069e20 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f8be4000c00 tx=0x7f8be400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:53:59.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.074+0000 7f8bf57fa700 1 -- 172.21.15.27:0/1415184393 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8be4019070 con 0x7f8bf80f6ff0 2021-11-03T17:53:59.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.074+0000 7f8bf57fa700 1 -- 172.21.15.27:0/1415184393 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f8be4007e70 con 0x7f8bf80f6ff0 2021-11-03T17:53:59.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.074+0000 7f8bfeb1d700 1 -- 172.21.15.27:0/1415184393 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf806a740 con 0x7f8bf80f6ff0 2021-11-03T17:53:59.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.074+0000 7f8bf57fa700 1 -- 172.21.15.27:0/1415184393 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8be4014ce0 con 0x7f8bf80f6ff0 2021-11-03T17:53:59.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.074+0000 7f8bfeb1d700 1 -- 172.21.15.27:0/1415184393 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf806aca0 con 0x7f8bf80f6ff0 2021-11-03T17:53:59.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.075+0000 7f8bfeb1d700 1 -- 172.21.15.27:0/1415184393 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf8045ad0 con 0x7f8bf80f6ff0 2021-11-03T17:53:59.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.075+0000 7f8bf57fa700 1 -- 172.21.15.27:0/1415184393 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f8be4025070 con 0x7f8bf80f6ff0 2021-11-03T17:53:59.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.076+0000 7f8bf57fa700 1 --2- 172.21.15.27:0/1415184393 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8be80640b0 0x7f8be8066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:53:59.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.076+0000 7f8bfc8b9700 1 --2- 172.21.15.27:0/1415184393 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8be80640b0 0x7f8be8066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:53:59.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.076+0000 7f8bf57fa700 1 -- 172.21.15.27:0/1415184393 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f8be40817d0 con 0x7f8bf80f6ff0 2021-11-03T17:53:59.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.076+0000 7f8bfc8b9700 1 --2- 172.21.15.27:0/1415184393 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8be80640b0 0x7f8be8066570 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f8bf806b730 tx=0x7f8bec00b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:53:59.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.080+0000 7f8bf57fa700 1 -- 172.21.15.27:0/1415184393 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f8be4053ad0 con 0x7f8bf80f6ff0 2021-11-03T17:53:59.248 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.251+0000 7f8bfeb1d700 1 -- 172.21.15.27:0/1415184393 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8bf80ff040 con 0x7f8be80640b0 2021-11-03T17:53:59.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.266+0000 7f8bf57fa700 1 -- 172.21.15.27:0/1415184393 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f8bf80ff040 con 0x7f8be80640b0 2021-11-03T17:53:59.263 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:53:59.264 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (45m) 5m ago 46m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:53:59.264 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (37m) 5m ago 46m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:53:59.264 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (37m) 4m ago 46m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:53:59.264 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (45m) 5m ago 46m 41.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:53:59.265 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (39m) 5m ago 41m 23.8M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:53:59.265 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (42m) 5m ago 41m 3844M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:53:59.265 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (37m) 4m ago 41m 23.2M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:53:59.265 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (39m) 4m ago 42m 24.9M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:53:59.266 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (39m) 5m ago 48m 535M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:53:59.266 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (38m) 4m ago 45m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:53:59.266 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (38m) 5m ago 48m 198M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:53:59.266 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (37m) 4m ago 45m 95.6M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:53:59.267 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (46m) 5m ago 46m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:53:59.267 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (45m) 4m ago 45m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:53:59.267 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (36m) 5m ago 45m 331M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:53:59.267 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (36m) 5m ago 44m 361M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:53:59.268 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (35m) 5m ago 44m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:53:59.268 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (35m) 4m ago 44m 379M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:53:59.268 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (34m) 4m ago 43m 304M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:53:59.268 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (34m) 4m ago 43m 307M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:53:59.269 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (45m) 5m ago 46m 52.9M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:53:59.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.267+0000 7f8bfeb1d700 1 -- 172.21.15.27:0/1415184393 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8be80640b0 msgr2=0x7f8be8066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:53:59.270 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.267+0000 7f8bfeb1d700 1 --2- 172.21.15.27:0/1415184393 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8be80640b0 0x7f8be8066570 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f8bf806b730 tx=0x7f8bec00b040).stop 2021-11-03T17:53:59.270 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.267+0000 7f8bfeb1d700 1 -- 172.21.15.27:0/1415184393 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf80f6ff0 msgr2=0x7f8bf8069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:53:59.270 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.267+0000 7f8bfeb1d700 1 --2- 172.21.15.27:0/1415184393 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf80f6ff0 0x7f8bf8069e20 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f8be4000c00 tx=0x7f8be400b040).stop 2021-11-03T17:53:59.270 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.268+0000 7f8bfeb1d700 1 -- 172.21.15.27:0/1415184393 shutdown_connections 2021-11-03T17:53:59.271 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.268+0000 7f8bfeb1d700 1 --2- 172.21.15.27:0/1415184393 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8be80640b0 0x7f8be8066570 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:59.271 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.268+0000 7f8bfeb1d700 1 --2- 172.21.15.27:0/1415184393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8bf80f66e0 0x7f8bf806b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:59.271 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.268+0000 7f8bfeb1d700 1 --2- 172.21.15.27:0/1415184393 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf80f6ff0 0x7f8bf8069e20 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:59.271 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.268+0000 7f8bfeb1d700 1 -- 172.21.15.27:0/1415184393 >> 172.21.15.27:0/1415184393 conn(0x7f8bf80f1f00 msgr2=0x7f8bf80fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:53:59.272 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.268+0000 7f8bfeb1d700 1 -- 172.21.15.27:0/1415184393 shutdown_connections 2021-11-03T17:53:59.272 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.268+0000 7f8bfeb1d700 1 -- 172.21.15.27:0/1415184393 wait complete. 2021-11-03T17:53:59.363 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.367+0000 7f2b81e7d700 1 -- 172.21.15.27:0/3451733183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b7c0f66e0 msgr2=0x7f2b7c0f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:53:59.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.367+0000 7f2b81e7d700 1 --2- 172.21.15.27:0/3451733183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b7c0f66e0 0x7f2b7c0f6b00 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7f2b6c004660 tx=0x7f2b6c009b30).stop 2021-11-03T17:53:59.365 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.367+0000 7f2b81e7d700 1 -- 172.21.15.27:0/3451733183 shutdown_connections 2021-11-03T17:53:59.365 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.367+0000 7f2b81e7d700 1 --2- 172.21.15.27:0/3451733183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b7c0f6ff0 0x7f2b7c0f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:59.365 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.367+0000 7f2b81e7d700 1 --2- 172.21.15.27:0/3451733183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b7c0f66e0 0x7f2b7c0f6b00 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:59.365 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.367+0000 7f2b81e7d700 1 -- 172.21.15.27:0/3451733183 >> 172.21.15.27:0/3451733183 conn(0x7f2b7c0f1f00 msgr2=0x7f2b7c0f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:53:59.366 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.367+0000 7f2b81e7d700 1 -- 172.21.15.27:0/3451733183 shutdown_connections 2021-11-03T17:53:59.366 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.367+0000 7f2b81e7d700 1 -- 172.21.15.27:0/3451733183 wait complete. 2021-11-03T17:53:59.366 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.368+0000 7f2b81e7d700 1 Processor -- start 2021-11-03T17:53:59.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.368+0000 7f2b81e7d700 1 -- start start 2021-11-03T17:53:59.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.368+0000 7f2b81e7d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b7c0f66e0 0x7f2b7c06b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:53:59.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.368+0000 7f2b81e7d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b7c0f6ff0 0x7f2b7c069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:53:59.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.368+0000 7f2b81e7d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c06a380 con 0x7f2b7c0f6ff0 2021-11-03T17:53:59.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.368+0000 7f2b81e7d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7c06a4c0 con 0x7f2b7c0f66e0 2021-11-03T17:53:59.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.368+0000 7f2b7affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b7c0f6ff0 0x7f2b7c069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:53:59.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.368+0000 7f2b7affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b7c0f6ff0 0x7f2b7c069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60028/0 (socket says 172.21.15.27:60028) 2021-11-03T17:53:59.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.368+0000 7f2b7affd700 1 -- 172.21.15.27:0/1251479036 learned_addr learned my addr 172.21.15.27:0/1251479036 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:53:59.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.368+0000 7f2b7affd700 1 -- 172.21.15.27:0/1251479036 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b7c0f66e0 msgr2=0x7f2b7c06b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:53:59.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.368+0000 7f2b7affd700 1 --2- 172.21.15.27:0/1251479036 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b7c0f66e0 0x7f2b7c06b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:53:59.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.368+0000 7f2b7affd700 1 -- 172.21.15.27:0/1251479036 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b6c005040 con 0x7f2b7c0f6ff0 2021-11-03T17:53:59.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.368+0000 7f2b7affd700 1 --2- 172.21.15.27:0/1251479036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b7c0f6ff0 0x7f2b7c069e20 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f2b7000c530 tx=0x7f2b70007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:53:59.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.369+0000 7f2b63fff700 1 -- 172.21.15.27:0/1251479036 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b7000f650 con 0x7f2b7c0f6ff0 2021-11-03T17:53:59.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.369+0000 7f2b63fff700 1 -- 172.21.15.27:0/1251479036 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f2b7000fc60 con 0x7f2b7c0f6ff0 2021-11-03T17:53:59.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.369+0000 7f2b81e7d700 1 -- 172.21.15.27:0/1251479036 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b7c06a7a0 con 0x7f2b7c0f6ff0 2021-11-03T17:53:59.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.369+0000 7f2b63fff700 1 -- 172.21.15.27:0/1251479036 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b70012cd0 con 0x7f2b7c0f6ff0 2021-11-03T17:53:59.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.369+0000 7f2b81e7d700 1 -- 172.21.15.27:0/1251479036 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b7c10ef90 con 0x7f2b7c0f6ff0 2021-11-03T17:53:59.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.370+0000 7f2b81e7d700 1 -- 172.21.15.27:0/1251479036 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b7c045ad0 con 0x7f2b7c0f6ff0 2021-11-03T17:53:59.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.370+0000 7f2b63fff700 1 -- 172.21.15.27:0/1251479036 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f2b70010070 con 0x7f2b7c0f6ff0 2021-11-03T17:53:59.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.370+0000 7f2b63fff700 1 --2- 172.21.15.27:0/1251479036 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2b640640b0 0x7f2b64066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:53:59.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.371+0000 7f2b63fff700 1 -- 172.21.15.27:0/1251479036 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f2b700806a0 con 0x7f2b7c0f6ff0 2021-11-03T17:53:59.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.371+0000 7f2b7b7fe700 1 --2- 172.21.15.27:0/1251479036 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2b640640b0 0x7f2b64066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:53:59.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.371+0000 7f2b7b7fe700 1 --2- 172.21.15.27:0/1251479036 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2b640640b0 0x7f2b64066570 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f2b6c004e30 tx=0x7f2b6c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:53:59.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.374+0000 7f2b63fff700 1 -- 172.21.15.27:0/1251479036 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f2b700528b0 con 0x7f2b7c0f6ff0 2021-11-03T17:53:59.603 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.607+0000 7f2b81e7d700 1 -- 172.21.15.27:0/1251479036 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2b7c027a00 con 0x7f2b7c0f6ff0 2021-11-03T17:53:59.604 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.607+0000 7f2b63fff700 1 -- 172.21.15.27:0/1251479036 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f2b700524a0 con 0x7f2b7c0f6ff0 2021-11-03T17:53:59.604 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:53:59.604 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:53:59.605 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:53:59.605 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:53:59.605 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:53:59.605 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:53:59.606 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:53:59.606 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:53:59.606 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:53:59.606 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:53:59.606 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:53:59.607 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:53:59.607 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:53:59.607 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:53:59.608 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:53:59.608 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:53:59.608 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:53:59.608 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:53:59.609 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.609+0000 7f2b81e7d700 1 -- 172.21.15.27:0/1251479036 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2b640640b0 msgr2=0x7f2b64066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:53:59.609 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.609+0000 7f2b81e7d700 1 --2- 172.21.15.27:0/1251479036 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2b640640b0 0x7f2b64066570 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 rx=0x7f2b6c004e30 tx=0x7f2b6c009b30).stop 2021-11-03T17:53:59.610 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.609+0000 7f2b81e7d700 1 -- 172.21.15.27:0/1251479036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b7c0f6ff0 msgr2=0x7f2b7c069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:53:59.610 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.609+0000 7f2b81e7d700 1 --2- 172.21.15.27:0/1251479036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b7c0f6ff0 0x7f2b7c069e20 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f2b7000c530 tx=0x7f2b70007e20).stop 2021-11-03T17:53:59.610 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.609+0000 7f2b81e7d700 1 -- 172.21.15.27:0/1251479036 shutdown_connections 2021-11-03T17:53:59.610 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.609+0000 7f2b81e7d700 1 --2- 172.21.15.27:0/1251479036 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2b640640b0 0x7f2b64066570 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:59.611 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.609+0000 7f2b81e7d700 1 --2- 172.21.15.27:0/1251479036 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2b7c0f66e0 0x7f2b7c06b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:53:59.611 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.609+0000 7f2b81e7d700 1 --2- 172.21.15.27:0/1251479036 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2b7c0f6ff0 0x7f2b7c069e20 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:59.611 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.609+0000 7f2b81e7d700 1 -- 172.21.15.27:0/1251479036 >> 172.21.15.27:0/1251479036 conn(0x7f2b7c0f1f00 msgr2=0x7f2b7c0fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:53:59.611 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.609+0000 7f2b81e7d700 1 -- 172.21.15.27:0/1251479036 shutdown_connections 2021-11-03T17:53:59.612 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.609+0000 7f2b81e7d700 1 -- 172.21.15.27:0/1251479036 wait complete. 2021-11-03T17:53:59.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.710+0000 7fc945570700 1 -- 172.21.15.27:0/3741955965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9400f6540 msgr2=0x7fc9400f69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:53:59.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.710+0000 7fc945570700 1 --2- 172.21.15.27:0/3741955965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9400f6540 0x7fc9400f69c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7fc930004660 tx=0x7fc930009b30).stop 2021-11-03T17:53:59.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.710+0000 7fc945570700 1 -- 172.21.15.27:0/3741955965 shutdown_connections 2021-11-03T17:53:59.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.710+0000 7fc945570700 1 --2- 172.21.15.27:0/3741955965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9400f6540 0x7fc9400f69c0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:59.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.710+0000 7fc945570700 1 --2- 172.21.15.27:0/3741955965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9400f5c30 0x7fc9400f6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:59.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.710+0000 7fc945570700 1 -- 172.21.15.27:0/3741955965 >> 172.21.15.27:0/3741955965 conn(0x7fc9400f0540 msgr2=0x7fc9400f2960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:53:59.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.710+0000 7fc945570700 1 -- 172.21.15.27:0/3741955965 shutdown_connections 2021-11-03T17:53:59.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.710+0000 7fc945570700 1 -- 172.21.15.27:0/3741955965 wait complete. 2021-11-03T17:53:59.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.711+0000 7fc945570700 1 Processor -- start 2021-11-03T17:53:59.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.711+0000 7fc945570700 1 -- start start 2021-11-03T17:53:59.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.711+0000 7fc945570700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9400f5c30 0x7fc94010c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:53:59.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.711+0000 7fc945570700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9400f6540 0x7fc94010e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:53:59.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.711+0000 7fc945570700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9400f55f0 con 0x7fc9400f5c30 2021-11-03T17:53:59.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.711+0000 7fc945570700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9400f5730 con 0x7fc9400f6540 2021-11-03T17:53:59.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.711+0000 7fc93effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9400f5c30 0x7fc94010c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:53:59.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.711+0000 7fc93effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9400f5c30 0x7fc94010c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60036/0 (socket says 172.21.15.27:60036) 2021-11-03T17:53:59.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.711+0000 7fc93effd700 1 -- 172.21.15.27:0/2757829057 learned_addr learned my addr 172.21.15.27:0/2757829057 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:53:59.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.711+0000 7fc93e7fc700 1 --2- 172.21.15.27:0/2757829057 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9400f6540 0x7fc94010e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:53:59.715 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.711+0000 7fc93effd700 1 -- 172.21.15.27:0/2757829057 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9400f6540 msgr2=0x7fc94010e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:53:59.716 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.711+0000 7fc93effd700 1 --2- 172.21.15.27:0/2757829057 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9400f6540 0x7fc94010e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:59.716 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.711+0000 7fc93effd700 1 -- 172.21.15.27:0/2757829057 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc930005040 con 0x7fc9400f5c30 2021-11-03T17:53:59.717 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.711+0000 7fc93e7fc700 1 --2- 172.21.15.27:0/2757829057 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9400f6540 0x7fc94010e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:53:59.717 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.711+0000 7fc93effd700 1 --2- 172.21.15.27:0/2757829057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9400f5c30 0x7fc94010c040 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7fc92800c530 tx=0x7fc928007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:53:59.718 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.711+0000 7fc9377fe700 1 -- 172.21.15.27:0/2757829057 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc92800f650 con 0x7fc9400f5c30 2021-11-03T17:53:59.718 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.712+0000 7fc9377fe700 1 -- 172.21.15.27:0/2757829057 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fc92800fc60 con 0x7fc9400f5c30 2021-11-03T17:53:59.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.712+0000 7fc945570700 1 -- 172.21.15.27:0/2757829057 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc94010ea30 con 0x7fc9400f5c30 2021-11-03T17:53:59.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.712+0000 7fc9377fe700 1 -- 172.21.15.27:0/2757829057 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc928012cd0 con 0x7fc9400f5c30 2021-11-03T17:53:59.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.712+0000 7fc945570700 1 -- 172.21.15.27:0/2757829057 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc94010eeb0 con 0x7fc9400f5c30 2021-11-03T17:53:59.722 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.712+0000 7fc945570700 1 -- 172.21.15.27:0/2757829057 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc940045ad0 con 0x7fc9400f5c30 2021-11-03T17:53:59.722 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.713+0000 7fc9377fe700 1 -- 172.21.15.27:0/2757829057 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fc928010070 con 0x7fc9400f5c30 2021-11-03T17:53:59.722 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.713+0000 7fc9377fe700 1 --2- 172.21.15.27:0/2757829057 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc92c0640b0 0x7fc92c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:53:59.723 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.713+0000 7fc9377fe700 1 -- 172.21.15.27:0/2757829057 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fc928080770 con 0x7fc9400f5c30 2021-11-03T17:53:59.723 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.713+0000 7fc93e7fc700 1 --2- 172.21.15.27:0/2757829057 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc92c0640b0 0x7fc92c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:53:59.723 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.714+0000 7fc93e7fc700 1 --2- 172.21.15.27:0/2757829057 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc92c0640b0 0x7fc92c066570 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fc930009da0 tx=0x7fc930009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:53:59.724 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.717+0000 7fc9377fe700 1 -- 172.21.15.27:0/2757829057 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fc928052980 con 0x7fc9400f5c30 2021-11-03T17:53:59.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.924+0000 7fc945570700 1 -- 172.21.15.27:0/2757829057 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc9400586e0 con 0x7fc9400f5c30 2021-11-03T17:53:59.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.925+0000 7fc9377fe700 1 -- 172.21.15.27:0/2757829057 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7fc928052570 con 0x7fc9400f5c30 2021-11-03T17:53:59.921 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:53:59.922 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:53:59.922 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:53:59.922 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:53:59.922 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:53:59.923 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:53:59.923 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:53:59.923 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:53:59.923 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:53:59.923 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:53:59.924 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:53:59.924 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:53:59.924 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:53:59.924 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:53:59.925 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:53:59.925 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:53:59.925 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:53:59.925 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:53:59.926 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:53:59.926 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:53:59.926 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:53:59.926 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:53:59.926 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:53:59.927 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:53:59.927 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:53:59.927 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:53:59.927 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:53:59.928 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:53:59.928 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:53:59.928 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:53:59.928 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:53:59.929 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:53:59.929 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:53:59.929 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:53:59.930 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:53:59.930 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:53:59.930 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:53:59.930 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:53:59.930 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:53:59.932 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.926+0000 7fc945570700 1 -- 172.21.15.27:0/2757829057 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc92c0640b0 msgr2=0x7fc92c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:53:59.932 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.926+0000 7fc945570700 1 --2- 172.21.15.27:0/2757829057 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc92c0640b0 0x7fc92c066570 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fc930009da0 tx=0x7fc930009b30).stop 2021-11-03T17:53:59.932 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.926+0000 7fc945570700 1 -- 172.21.15.27:0/2757829057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9400f5c30 msgr2=0x7fc94010c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:53:59.932 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.926+0000 7fc945570700 1 --2- 172.21.15.27:0/2757829057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9400f5c30 0x7fc94010c040 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7fc92800c530 tx=0x7fc928007e20).stop 2021-11-03T17:53:59.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.926+0000 7fc945570700 1 -- 172.21.15.27:0/2757829057 shutdown_connections 2021-11-03T17:53:59.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.926+0000 7fc945570700 1 --2- 172.21.15.27:0/2757829057 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc92c0640b0 0x7fc92c066570 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:59.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.926+0000 7fc945570700 1 --2- 172.21.15.27:0/2757829057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9400f5c30 0x7fc94010c040 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:59.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.926+0000 7fc945570700 1 --2- 172.21.15.27:0/2757829057 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9400f6540 0x7fc94010e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:53:59.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.926+0000 7fc945570700 1 -- 172.21.15.27:0/2757829057 >> 172.21.15.27:0/2757829057 conn(0x7fc9400f0540 msgr2=0x7fc9400fa4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:53:59.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.926+0000 7fc945570700 1 -- 172.21.15.27:0/2757829057 shutdown_connections 2021-11-03T17:53:59.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:53:59.927+0000 7fc945570700 1 -- 172.21.15.27:0/2757829057 wait complete. 2021-11-03T17:53:59.934 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:54:00.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:00 smithi115 ceph-mon[65453]: pgmap v1180: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:00.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:00 smithi115 ceph-mon[65453]: from='client.35320 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:54:00.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:00 smithi115 ceph-mon[65453]: from='client.35324 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:54:00.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2282.414327 secs 2021-11-03T17:54:00.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:00 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1251479036' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:00.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:00 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2757829057' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:54:00.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:00 smithi027 ceph-mon[97244]: pgmap v1180: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:00.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:00 smithi027 ceph-mon[97244]: from='client.35320 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:54:00.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:00 smithi027 ceph-mon[97244]: from='client.35324 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:54:00.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2282.414327 secs 2021-11-03T17:54:00.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:00 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1251479036' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:00.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:00 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2757829057' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:54:02.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:02 smithi115 ceph-mon[65453]: pgmap v1181: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:02.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:02 smithi027 ceph-mon[97244]: pgmap v1181: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:04.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:04 smithi115 ceph-mon[65453]: pgmap v1182: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:04.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:54:04.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:54:04.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:54:04.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:04.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:04.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:04.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:04.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:04 smithi027 ceph-mon[97244]: pgmap v1182: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:04.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:54:04.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:54:04.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:54:04.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:04.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:04.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:04.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:05.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:05 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:54:05.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2287.414482 secs 2021-11-03T17:54:05.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:05 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:54:05.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2287.414482 secs 2021-11-03T17:54:06.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:06 smithi115 ceph-mon[65453]: pgmap v1183: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:06.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:06 smithi027 ceph-mon[97244]: pgmap v1183: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:08.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:08 smithi115 ceph-mon[65453]: pgmap v1184: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:08.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:08 smithi027 ceph-mon[97244]: pgmap v1184: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:10.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:10 smithi115 ceph-mon[65453]: pgmap v1185: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:10.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2292.414617 secs 2021-11-03T17:54:10.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:10 smithi027 ceph-mon[97244]: pgmap v1185: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:10.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2292.414617 secs 2021-11-03T17:54:12.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:12 smithi115 ceph-mon[65453]: pgmap v1186: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:12.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:12 smithi027 ceph-mon[97244]: pgmap v1186: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:14.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:14 smithi115 ceph-mon[65453]: pgmap v1187: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:14.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:54:14.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:54:14.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:54:14.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:14.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:14.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:14.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:14.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:14 smithi027 ceph-mon[97244]: pgmap v1187: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:14.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:54:14.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:54:14.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:54:14.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:14.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:14.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:14.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:15.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:15 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:54:15.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2297.414733 secs 2021-11-03T17:54:15.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:15 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:54:15.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2297.414733 secs 2021-11-03T17:54:16.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:16 smithi115 ceph-mon[65453]: pgmap v1188: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:16.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:16 smithi027 ceph-mon[97244]: pgmap v1188: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:17.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:17 smithi115 ceph-mon[65453]: pgmap v1189: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:17.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:17 smithi027 ceph-mon[97244]: pgmap v1189: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:19.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:19 smithi115 ceph-mon[65453]: pgmap v1190: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:19.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:19 smithi027 ceph-mon[97244]: pgmap v1190: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:20.734 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2302.414875 secs 2021-11-03T17:54:20.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2302.414875 secs 2021-11-03T17:54:22.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:21 smithi115 ceph-mon[65453]: pgmap v1191: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:22.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:21 smithi027 ceph-mon[97244]: pgmap v1191: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:24.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:23 smithi115 ceph-mon[65453]: pgmap v1192: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:24.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:23 smithi027 ceph-mon[97244]: pgmap v1192: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:25.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:24 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:54:25.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:24 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:54:25.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:24 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:54:25.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:24 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:25.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:24 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:25.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:24 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:25.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:24 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:25.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:24 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:54:25.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:24 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2307.415003 secs 2021-11-03T17:54:25.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:24 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:54:25.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:24 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:54:25.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:24 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:54:25.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:24 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:25.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:24 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:25.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:24 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:25.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:24 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:25.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:24 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:54:25.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:24 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2307.415003 secs 2021-11-03T17:54:26.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:26 smithi115 ceph-mon[65453]: pgmap v1193: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:26.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:26 smithi027 ceph-mon[97244]: pgmap v1193: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:28.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:28 smithi115 ceph-mon[65453]: pgmap v1194: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:28.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:28 smithi027 ceph-mon[97244]: pgmap v1194: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:30.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.040+0000 7fe949b0e700 1 -- 172.21.15.27:0/1441307819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9440f9700 msgr2=0x7fe9440fbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:54:30.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.040+0000 7fe949b0e700 1 --2- 172.21.15.27:0/1441307819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9440f9700 0x7fe9440fbaf0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7fe930004660 tx=0x7fe930009b30).stop 2021-11-03T17:54:30.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.040+0000 7fe949b0e700 1 -- 172.21.15.27:0/1441307819 shutdown_connections 2021-11-03T17:54:30.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.040+0000 7fe949b0e700 1 --2- 172.21.15.27:0/1441307819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9440f9700 0x7fe9440fbaf0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:54:30.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.041+0000 7fe949b0e700 1 --2- 172.21.15.27:0/1441307819 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9440602b0 0x7fe9440f9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:54:30.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.041+0000 7fe949b0e700 1 -- 172.21.15.27:0/1441307819 >> 172.21.15.27:0/1441307819 conn(0x7fe9440f0ba0 msgr2=0x7fe9440f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:54:30.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.041+0000 7fe949b0e700 1 -- 172.21.15.27:0/1441307819 shutdown_connections 2021-11-03T17:54:30.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.041+0000 7fe949b0e700 1 -- 172.21.15.27:0/1441307819 wait complete. 2021-11-03T17:54:30.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.041+0000 7fe949b0e700 1 Processor -- start 2021-11-03T17:54:30.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.041+0000 7fe949b0e700 1 -- start start 2021-11-03T17:54:30.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.042+0000 7fe949b0e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9440602b0 0x7fe944106dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:54:30.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.042+0000 7fe949b0e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9440f9700 0x7fe9441072b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:54:30.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.042+0000 7fe949b0e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe944107940 con 0x7fe9440f9700 2021-11-03T17:54:30.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.042+0000 7fe949b0e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9441080f0 con 0x7fe9440602b0 2021-11-03T17:54:30.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.042+0000 7fe943fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9440f9700 0x7fe9441072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:54:30.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.042+0000 7fe943fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9440f9700 0x7fe9441072b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60068/0 (socket says 172.21.15.27:60068) 2021-11-03T17:54:30.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.042+0000 7fe943fff700 1 -- 172.21.15.27:0/2458328518 learned_addr learned my addr 172.21.15.27:0/2458328518 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:54:30.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.042+0000 7fe948b0c700 1 --2- 172.21.15.27:0/2458328518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9440602b0 0x7fe944106dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:54:30.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.042+0000 7fe943fff700 1 -- 172.21.15.27:0/2458328518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9440602b0 msgr2=0x7fe944106dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:54:30.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.042+0000 7fe943fff700 1 --2- 172.21.15.27:0/2458328518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9440602b0 0x7fe944106dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:54:30.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.042+0000 7fe943fff700 1 -- 172.21.15.27:0/2458328518 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe930005040 con 0x7fe9440f9700 2021-11-03T17:54:30.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.042+0000 7fe943fff700 1 --2- 172.21.15.27:0/2458328518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9440f9700 0x7fe9441072b0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7fe930004f60 tx=0x7fe930009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:54:30.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.042+0000 7fe9417fa700 1 -- 172.21.15.27:0/2458328518 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe930018070 con 0x7fe9440f9700 2021-11-03T17:54:30.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.042+0000 7fe949b0e700 1 -- 172.21.15.27:0/2458328518 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe944108370 con 0x7fe9440f9700 2021-11-03T17:54:30.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.042+0000 7fe9417fa700 1 -- 172.21.15.27:0/2458328518 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fe930007ba0 con 0x7fe9440f9700 2021-11-03T17:54:30.046 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.042+0000 7fe9417fa700 1 -- 172.21.15.27:0/2458328518 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe93001c620 con 0x7fe9440f9700 2021-11-03T17:54:30.046 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.042+0000 7fe949b0e700 1 -- 172.21.15.27:0/2458328518 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9441088d0 con 0x7fe9440f9700 2021-11-03T17:54:30.046 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.043+0000 7fe949b0e700 1 -- 172.21.15.27:0/2458328518 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe944059100 con 0x7fe9440f9700 2021-11-03T17:54:30.048 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.044+0000 7fe9417fa700 1 -- 172.21.15.27:0/2458328518 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fe93001c8e0 con 0x7fe9440f9700 2021-11-03T17:54:30.048 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.044+0000 7fe9417fa700 1 --2- 172.21.15.27:0/2458328518 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe934064100 0x7fe9340665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:54:30.049 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.045+0000 7fe9417fa700 1 -- 172.21.15.27:0/2458328518 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fe930082c80 con 0x7fe9440f9700 2021-11-03T17:54:30.049 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.045+0000 7fe948b0c700 1 --2- 172.21.15.27:0/2458328518 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe934064100 0x7fe9340665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:54:30.050 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.045+0000 7fe948b0c700 1 --2- 172.21.15.27:0/2458328518 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe934064100 0x7fe9340665c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7fe944107b80 tx=0x7fe938008040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:54:30.050 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.048+0000 7fe9417fa700 1 -- 172.21.15.27:0/2458328518 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fe930054ec0 con 0x7fe9440f9700 2021-11-03T17:54:30.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.228+0000 7fe949b0e700 1 -- 172.21.15.27:0/2458328518 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe9440008d0 con 0x7fe934064100 2021-11-03T17:54:30.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.229+0000 7fe9417fa700 1 -- 172.21.15.27:0/2458328518 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fe9440008d0 con 0x7fe934064100 2021-11-03T17:54:30.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.231+0000 7fe949b0e700 1 -- 172.21.15.27:0/2458328518 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe934064100 msgr2=0x7fe9340665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:54:30.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.231+0000 7fe949b0e700 1 --2- 172.21.15.27:0/2458328518 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe934064100 0x7fe9340665c0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7fe944107b80 tx=0x7fe938008040).stop 2021-11-03T17:54:30.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.231+0000 7fe949b0e700 1 -- 172.21.15.27:0/2458328518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9440f9700 msgr2=0x7fe9441072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:54:30.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.231+0000 7fe949b0e700 1 --2- 172.21.15.27:0/2458328518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9440f9700 0x7fe9441072b0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7fe930004f60 tx=0x7fe930009b30).stop 2021-11-03T17:54:30.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.231+0000 7fe949b0e700 1 -- 172.21.15.27:0/2458328518 shutdown_connections 2021-11-03T17:54:30.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.231+0000 7fe949b0e700 1 --2- 172.21.15.27:0/2458328518 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe934064100 0x7fe9340665c0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:54:30.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.231+0000 7fe949b0e700 1 --2- 172.21.15.27:0/2458328518 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9440602b0 0x7fe944106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:54:30.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.231+0000 7fe949b0e700 1 --2- 172.21.15.27:0/2458328518 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9440f9700 0x7fe9441072b0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:54:30.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.231+0000 7fe949b0e700 1 -- 172.21.15.27:0/2458328518 >> 172.21.15.27:0/2458328518 conn(0x7fe9440f0ba0 msgr2=0x7fe9440f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:54:30.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.231+0000 7fe949b0e700 1 -- 172.21.15.27:0/2458328518 shutdown_connections 2021-11-03T17:54:30.230 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.231+0000 7fe949b0e700 1 -- 172.21.15.27:0/2458328518 wait complete. 2021-11-03T17:54:30.237 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:54:30.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.331+0000 7fb18bc56700 1 -- 172.21.15.27:0/4037322784 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1840f7980 msgr2=0x7fb1840f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:54:30.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.331+0000 7fb18bc56700 1 --2- 172.21.15.27:0/4037322784 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1840f7980 0x7fb1840f7de0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7fb180004660 tx=0x7fb180009b30).stop 2021-11-03T17:54:30.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.331+0000 7fb18bc56700 1 -- 172.21.15.27:0/4037322784 shutdown_connections 2021-11-03T17:54:30.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.331+0000 7fb18bc56700 1 --2- 172.21.15.27:0/4037322784 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1840f7980 0x7fb1840f7de0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:54:30.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.331+0000 7fb18bc56700 1 --2- 172.21.15.27:0/4037322784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1840f66e0 0x7fb1840f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:54:30.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.331+0000 7fb18bc56700 1 -- 172.21.15.27:0/4037322784 >> 172.21.15.27:0/4037322784 conn(0x7fb1840f1e40 msgr2=0x7fb1840f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:54:30.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.331+0000 7fb18bc56700 1 -- 172.21.15.27:0/4037322784 shutdown_connections 2021-11-03T17:54:30.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.331+0000 7fb18bc56700 1 -- 172.21.15.27:0/4037322784 wait complete. 2021-11-03T17:54:30.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.332+0000 7fb18bc56700 1 Processor -- start 2021-11-03T17:54:30.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.332+0000 7fb18bc56700 1 -- start start 2021-11-03T17:54:30.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.332+0000 7fb18bc56700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1840f66e0 0x7fb184106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:54:30.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.332+0000 7fb18bc56700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1840f7980 0x7fb184107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:54:30.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.332+0000 7fb18bc56700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb184107a00 con 0x7fb1840f66e0 2021-11-03T17:54:30.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.332+0000 7fb18bc56700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb184107b40 con 0x7fb1840f7980 2021-11-03T17:54:30.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.332+0000 7fb1899f2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1840f66e0 0x7fb184106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:54:30.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.332+0000 7fb1899f2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1840f66e0 0x7fb184106f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60076/0 (socket says 172.21.15.27:60076) 2021-11-03T17:54:30.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.332+0000 7fb1899f2700 1 -- 172.21.15.27:0/4171282275 learned_addr learned my addr 172.21.15.27:0/4171282275 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:54:30.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.332+0000 7fb1891f1700 1 --2- 172.21.15.27:0/4171282275 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1840f7980 0x7fb184107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:54:30.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.332+0000 7fb1899f2700 1 -- 172.21.15.27:0/4171282275 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1840f7980 msgr2=0x7fb184107450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:54:30.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.332+0000 7fb1899f2700 1 --2- 172.21.15.27:0/4171282275 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1840f7980 0x7fb184107450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:54:30.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.332+0000 7fb1899f2700 1 -- 172.21.15.27:0/4171282275 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb180005040 con 0x7fb1840f66e0 2021-11-03T17:54:30.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.332+0000 7fb1899f2700 1 --2- 172.21.15.27:0/4171282275 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1840f66e0 0x7fb184106f60 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7fb17400c530 tx=0x7fb174007e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:54:30.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.332+0000 7fb17a7fc700 1 -- 172.21.15.27:0/4171282275 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb17400f650 con 0x7fb1840f66e0 2021-11-03T17:54:30.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.333+0000 7fb17a7fc700 1 -- 172.21.15.27:0/4171282275 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fb17400fc60 con 0x7fb1840f66e0 2021-11-03T17:54:30.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.333+0000 7fb18bc56700 1 -- 172.21.15.27:0/4171282275 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1841084e0 con 0x7fb1840f66e0 2021-11-03T17:54:30.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.333+0000 7fb17a7fc700 1 -- 172.21.15.27:0/4171282275 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb174012d20 con 0x7fb1840f66e0 2021-11-03T17:54:30.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.333+0000 7fb18bc56700 1 -- 172.21.15.27:0/4171282275 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb184108aa0 con 0x7fb1840f66e0 2021-11-03T17:54:30.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.334+0000 7fb18bc56700 1 -- 172.21.15.27:0/4171282275 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb184045ad0 con 0x7fb1840f66e0 2021-11-03T17:54:30.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.334+0000 7fb17a7fc700 1 -- 172.21.15.27:0/4171282275 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fb174019030 con 0x7fb1840f66e0 2021-11-03T17:54:30.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.334+0000 7fb17a7fc700 1 --2- 172.21.15.27:0/4171282275 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb1700640b0 0x7fb170066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:54:30.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.334+0000 7fb17a7fc700 1 -- 172.21.15.27:0/4171282275 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fb174080680 con 0x7fb1840f66e0 2021-11-03T17:54:30.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.334+0000 7fb1891f1700 1 --2- 172.21.15.27:0/4171282275 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb1700640b0 0x7fb170066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:54:30.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.335+0000 7fb1891f1700 1 --2- 172.21.15.27:0/4171282275 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb1700640b0 0x7fb170066570 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7fb180004e30 tx=0x7fb180009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:54:30.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.338+0000 7fb17a7fc700 1 -- 172.21.15.27:0/4171282275 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fb1740529a0 con 0x7fb1840f66e0 2021-11-03T17:54:30.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:30 smithi115 ceph-mon[65453]: pgmap v1195: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:30.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2312.415147 secs 2021-11-03T17:54:30.507 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.511+0000 7fb18bc56700 1 -- 172.21.15.27:0/4171282275 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb184108d70 con 0x7fb1700640b0 2021-11-03T17:54:30.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:30 smithi027 ceph-mon[97244]: pgmap v1195: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:30.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2312.415147 secs 2021-11-03T17:54:30.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.525+0000 7fb17a7fc700 1 -- 172.21.15.27:0/4171282275 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fb184108d70 con 0x7fb1700640b0 2021-11-03T17:54:30.522 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:54:30.522 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (46m) 5m ago 47m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:54:30.523 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (37m) 5m ago 47m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:54:30.523 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (37m) 4m ago 46m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:54:30.523 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (45m) 5m ago 46m 41.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:54:30.523 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (39m) 5m ago 42m 23.8M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:54:30.524 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (42m) 5m ago 42m 3844M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:54:30.524 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (38m) 4m ago 42m 23.2M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:54:30.524 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (39m) 4m ago 42m 24.9M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:54:30.524 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (39m) 5m ago 48m 535M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:54:30.524 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (39m) 4m ago 46m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:54:30.525 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (38m) 5m ago 48m 198M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:54:30.525 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (38m) 4m ago 46m 95.6M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:54:30.525 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (46m) 5m ago 46m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:54:30.525 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (46m) 4m ago 46m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:54:30.526 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (37m) 5m ago 45m 331M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:54:30.526 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (36m) 5m ago 45m 361M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:54:30.526 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (36m) 5m ago 45m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:54:30.526 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (35m) 4m ago 44m 379M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:54:30.527 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (35m) 4m ago 44m 304M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:54:30.527 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (35m) 4m ago 44m 307M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:54:30.527 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (45m) 5m ago 46m 52.9M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:54:30.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.527+0000 7fb18bc56700 1 -- 172.21.15.27:0/4171282275 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb1700640b0 msgr2=0x7fb170066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:54:30.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.527+0000 7fb18bc56700 1 --2- 172.21.15.27:0/4171282275 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb1700640b0 0x7fb170066570 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 rx=0x7fb180004e30 tx=0x7fb180009b30).stop 2021-11-03T17:54:30.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.527+0000 7fb18bc56700 1 -- 172.21.15.27:0/4171282275 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1840f66e0 msgr2=0x7fb184106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:54:30.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.527+0000 7fb18bc56700 1 --2- 172.21.15.27:0/4171282275 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1840f66e0 0x7fb184106f60 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7fb17400c530 tx=0x7fb174007e10).stop 2021-11-03T17:54:30.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.527+0000 7fb18bc56700 1 -- 172.21.15.27:0/4171282275 shutdown_connections 2021-11-03T17:54:30.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.527+0000 7fb18bc56700 1 --2- 172.21.15.27:0/4171282275 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb1700640b0 0x7fb170066570 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:54:30.530 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.527+0000 7fb18bc56700 1 --2- 172.21.15.27:0/4171282275 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb1840f66e0 0x7fb184106f60 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:54:30.530 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.527+0000 7fb18bc56700 1 --2- 172.21.15.27:0/4171282275 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb1840f7980 0x7fb184107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:54:30.530 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.527+0000 7fb18bc56700 1 -- 172.21.15.27:0/4171282275 >> 172.21.15.27:0/4171282275 conn(0x7fb1840f1e40 msgr2=0x7fb1840fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:54:30.530 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.528+0000 7fb18bc56700 1 -- 172.21.15.27:0/4171282275 shutdown_connections 2021-11-03T17:54:30.531 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.528+0000 7fb18bc56700 1 -- 172.21.15.27:0/4171282275 wait complete. 2021-11-03T17:54:30.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.627+0000 7f52ede58700 1 -- 172.21.15.27:0/4163944910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52e80f6ff0 msgr2=0x7f52e80f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:54:30.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.627+0000 7f52ede58700 1 --2- 172.21.15.27:0/4163944910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52e80f6ff0 0x7f52e80f7a20 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7f52dc004660 tx=0x7f52dc009b30).stop 2021-11-03T17:54:30.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.627+0000 7f52ede58700 1 -- 172.21.15.27:0/4163944910 shutdown_connections 2021-11-03T17:54:30.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.627+0000 7f52ede58700 1 --2- 172.21.15.27:0/4163944910 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52e80f6ff0 0x7f52e80f7a20 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:54:30.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.627+0000 7f52ede58700 1 --2- 172.21.15.27:0/4163944910 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52e80f66e0 0x7f52e80f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:54:30.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.627+0000 7f52ede58700 1 -- 172.21.15.27:0/4163944910 >> 172.21.15.27:0/4163944910 conn(0x7f52e80f1f00 msgr2=0x7f52e80f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:54:30.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.627+0000 7f52ede58700 1 -- 172.21.15.27:0/4163944910 shutdown_connections 2021-11-03T17:54:30.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.627+0000 7f52ede58700 1 -- 172.21.15.27:0/4163944910 wait complete. 2021-11-03T17:54:30.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.628+0000 7f52ede58700 1 Processor -- start 2021-11-03T17:54:30.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.628+0000 7f52ede58700 1 -- start start 2021-11-03T17:54:30.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.628+0000 7f52ede58700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52e80f66e0 0x7f52e8106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:54:30.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.628+0000 7f52ede58700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52e80f6ff0 0x7f52e8107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:54:30.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.628+0000 7f52ede58700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e8107a10 con 0x7f52e80f66e0 2021-11-03T17:54:30.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.628+0000 7f52ede58700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e8107b50 con 0x7f52e80f6ff0 2021-11-03T17:54:30.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.628+0000 7f52e77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52e80f66e0 0x7f52e8106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:54:30.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.628+0000 7f52e77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52e80f66e0 0x7f52e8106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60086/0 (socket says 172.21.15.27:60086) 2021-11-03T17:54:30.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.628+0000 7f52e77fe700 1 -- 172.21.15.27:0/549014445 learned_addr learned my addr 172.21.15.27:0/549014445 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:54:30.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.629+0000 7f52e6ffd700 1 --2- 172.21.15.27:0/549014445 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52e80f6ff0 0x7f52e8107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:54:30.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.629+0000 7f52e77fe700 1 -- 172.21.15.27:0/549014445 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52e80f6ff0 msgr2=0x7f52e8107460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:54:30.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.629+0000 7f52e77fe700 1 --2- 172.21.15.27:0/549014445 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52e80f6ff0 0x7f52e8107460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:54:30.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.629+0000 7f52e77fe700 1 -- 172.21.15.27:0/549014445 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52dc005040 con 0x7f52e80f66e0 2021-11-03T17:54:30.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.629+0000 7f52e77fe700 1 --2- 172.21.15.27:0/549014445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52e80f66e0 0x7f52e8106f70 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f52d800c790 tx=0x7f52d8007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:54:30.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.629+0000 7f52cffff700 1 -- 172.21.15.27:0/549014445 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52d800e680 con 0x7f52e80f66e0 2021-11-03T17:54:30.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.629+0000 7f52cffff700 1 -- 172.21.15.27:0/549014445 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f52d800ec90 con 0x7f52e80f66e0 2021-11-03T17:54:30.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.629+0000 7f52ede58700 1 -- 172.21.15.27:0/549014445 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52e81084f0 con 0x7f52e80f66e0 2021-11-03T17:54:30.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.629+0000 7f52cffff700 1 -- 172.21.15.27:0/549014445 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52d8013d60 con 0x7f52e80f66e0 2021-11-03T17:54:30.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.629+0000 7f52ede58700 1 -- 172.21.15.27:0/549014445 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e8108ab0 con 0x7f52e80f66e0 2021-11-03T17:54:30.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.630+0000 7f52ede58700 1 -- 172.21.15.27:0/549014445 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52e8045ad0 con 0x7f52e80f66e0 2021-11-03T17:54:30.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.630+0000 7f52cffff700 1 -- 172.21.15.27:0/549014445 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f52d801a060 con 0x7f52e80f66e0 2021-11-03T17:54:30.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.631+0000 7f52cffff700 1 --2- 172.21.15.27:0/549014445 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f52d00640b0 0x7f52d0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:54:30.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.631+0000 7f52cffff700 1 -- 172.21.15.27:0/549014445 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f52d8080a80 con 0x7f52e80f66e0 2021-11-03T17:54:30.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.631+0000 7f52e6ffd700 1 --2- 172.21.15.27:0/549014445 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f52d00640b0 0x7f52d0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:54:30.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.631+0000 7f52e6ffd700 1 --2- 172.21.15.27:0/549014445 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f52d00640b0 0x7f52d0066570 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f52dc004e30 tx=0x7f52dc009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:54:30.637 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.635+0000 7f52cffff700 1 -- 172.21.15.27:0/549014445 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f52d8052c30 con 0x7f52e80f66e0 2021-11-03T17:54:30.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.862+0000 7f52ede58700 1 -- 172.21.15.27:0/549014445 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f52e80fef70 con 0x7f52e80f66e0 2021-11-03T17:54:30.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.863+0000 7f52cffff700 1 -- 172.21.15.27:0/549014445 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f52d8018070 con 0x7f52e80f66e0 2021-11-03T17:54:30.860 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:54:30.860 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:54:30.860 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:54:30.861 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:54:30.861 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:54:30.861 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:54:30.861 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:54:30.861 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:54:30.862 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:54:30.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:54:30.862 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:54:30.862 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:54:30.863 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:54:30.863 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:54:30.863 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:54:30.863 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:54:30.863 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:54:30.864 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:54:30.864 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.864+0000 7f52ede58700 1 -- 172.21.15.27:0/549014445 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f52d00640b0 msgr2=0x7f52d0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:54:30.865 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.864+0000 7f52ede58700 1 --2- 172.21.15.27:0/549014445 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f52d00640b0 0x7f52d0066570 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f52dc004e30 tx=0x7f52dc009b30).stop 2021-11-03T17:54:30.865 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.864+0000 7f52ede58700 1 -- 172.21.15.27:0/549014445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52e80f66e0 msgr2=0x7f52e8106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:54:30.865 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.864+0000 7f52ede58700 1 --2- 172.21.15.27:0/549014445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52e80f66e0 0x7f52e8106f70 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f52d800c790 tx=0x7f52d8007d70).stop 2021-11-03T17:54:30.865 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.865+0000 7f52ede58700 1 -- 172.21.15.27:0/549014445 shutdown_connections 2021-11-03T17:54:30.866 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.865+0000 7f52ede58700 1 --2- 172.21.15.27:0/549014445 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f52d00640b0 0x7f52d0066570 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:54:30.866 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.865+0000 7f52ede58700 1 --2- 172.21.15.27:0/549014445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52e80f66e0 0x7f52e8106f70 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:54:30.866 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.865+0000 7f52ede58700 1 --2- 172.21.15.27:0/549014445 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52e80f6ff0 0x7f52e8107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:54:30.866 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.865+0000 7f52ede58700 1 -- 172.21.15.27:0/549014445 >> 172.21.15.27:0/549014445 conn(0x7f52e80f1f00 msgr2=0x7f52e80fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:54:30.867 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.865+0000 7f52ede58700 1 -- 172.21.15.27:0/549014445 shutdown_connections 2021-11-03T17:54:30.867 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.865+0000 7f52ede58700 1 -- 172.21.15.27:0/549014445 wait complete. 2021-11-03T17:54:30.961 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.965+0000 7f3525b9d700 1 -- 172.21.15.27:0/3210187693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35200f66d0 msgr2=0x7f35200f6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:54:30.962 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.965+0000 7f3525b9d700 1 --2- 172.21.15.27:0/3210187693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35200f66d0 0x7f35200f6b30 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f350c004660 tx=0x7f350c009b30).stop 2021-11-03T17:54:30.962 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.965+0000 7f3525b9d700 1 -- 172.21.15.27:0/3210187693 shutdown_connections 2021-11-03T17:54:30.962 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.965+0000 7f3525b9d700 1 --2- 172.21.15.27:0/3210187693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35200f66d0 0x7f35200f6b30 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:54:30.963 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.965+0000 7f3525b9d700 1 --2- 172.21.15.27:0/3210187693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35200f5430 0x7f35200f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:54:30.963 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.965+0000 7f3525b9d700 1 -- 172.21.15.27:0/3210187693 >> 172.21.15.27:0/3210187693 conn(0x7f35200f0b90 msgr2=0x7f35200f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:54:30.963 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.965+0000 7f3525b9d700 1 -- 172.21.15.27:0/3210187693 shutdown_connections 2021-11-03T17:54:30.963 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.965+0000 7f3525b9d700 1 -- 172.21.15.27:0/3210187693 wait complete. 2021-11-03T17:54:30.963 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.965+0000 7f3525b9d700 1 Processor -- start 2021-11-03T17:54:30.964 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.965+0000 7f3525b9d700 1 -- start start 2021-11-03T17:54:30.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.966+0000 7f3525b9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35200f5430 0x7f3520105cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:54:30.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.966+0000 7f3525b9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35200f66d0 0x7f35201061b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:54:30.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.966+0000 7f3525b9d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3520106760 con 0x7f35200f5430 2021-11-03T17:54:30.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.966+0000 7f3525b9d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3520106f60 con 0x7f35200f66d0 2021-11-03T17:54:30.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.966+0000 7f3524b9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35200f5430 0x7f3520105cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:54:30.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.966+0000 7f3524b9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35200f5430 0x7f3520105cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60096/0 (socket says 172.21.15.27:60096) 2021-11-03T17:54:30.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.966+0000 7f3524b9b700 1 -- 172.21.15.27:0/3836604427 learned_addr learned my addr 172.21.15.27:0/3836604427 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:54:30.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.966+0000 7f351ffff700 1 --2- 172.21.15.27:0/3836604427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35200f66d0 0x7f35201061b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:54:30.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.966+0000 7f3524b9b700 1 -- 172.21.15.27:0/3836604427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35200f66d0 msgr2=0x7f35201061b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:54:30.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.966+0000 7f3524b9b700 1 --2- 172.21.15.27:0/3836604427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35200f66d0 0x7f35201061b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:54:30.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.966+0000 7f3524b9b700 1 -- 172.21.15.27:0/3836604427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f350c005040 con 0x7f35200f5430 2021-11-03T17:54:30.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.966+0000 7f351ffff700 1 --2- 172.21.15.27:0/3836604427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35200f66d0 0x7f35201061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:54:30.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.966+0000 7f3524b9b700 1 --2- 172.21.15.27:0/3836604427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35200f5430 0x7f3520105cc0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f3514002700 tx=0x7f3514009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:54:30.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.966+0000 7f351d7fa700 1 -- 172.21.15.27:0/3836604427 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3514009320 con 0x7f35200f5430 2021-11-03T17:54:30.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.966+0000 7f3525b9d700 1 -- 172.21.15.27:0/3836604427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3520107240 con 0x7f35200f5430 2021-11-03T17:54:30.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.967+0000 7f351d7fa700 1 -- 172.21.15.27:0/3836604427 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f351400f040 con 0x7f35200f5430 2021-11-03T17:54:30.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.967+0000 7f351d7fa700 1 -- 172.21.15.27:0/3836604427 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3514014900 con 0x7f35200f5430 2021-11-03T17:54:30.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.967+0000 7f3525b9d700 1 -- 172.21.15.27:0/3836604427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3520107800 con 0x7f35200f5430 2021-11-03T17:54:30.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.968+0000 7f3525b9d700 1 -- 172.21.15.27:0/3836604427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3520059100 con 0x7f35200f5430 2021-11-03T17:54:30.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.968+0000 7f351d7fa700 1 -- 172.21.15.27:0/3836604427 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f35140077b0 con 0x7f35200f5430 2021-11-03T17:54:30.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.968+0000 7f351d7fa700 1 --2- 172.21.15.27:0/3836604427 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3510064100 0x7f35100665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:54:30.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.968+0000 7f351d7fa700 1 -- 172.21.15.27:0/3836604427 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f3514080ac0 con 0x7f35200f5430 2021-11-03T17:54:30.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.968+0000 7f351ffff700 1 --2- 172.21.15.27:0/3836604427 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3510064100 0x7f35100665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:54:30.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.969+0000 7f351ffff700 1 --2- 172.21.15.27:0/3836604427 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3510064100 0x7f35100665c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7f350c004630 tx=0x7f350c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:54:30.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:30.972+0000 7f351d7fa700 1 -- 172.21.15.27:0/3836604427 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f3514052dc0 con 0x7f35200f5430 2021-11-03T17:54:31.171 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:31.175+0000 7f3525b9d700 1 -- 172.21.15.27:0/3836604427 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f35200fdc90 con 0x7f35200f5430 2021-11-03T17:54:31.172 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:31.175+0000 7f351d7fa700 1 -- 172.21.15.27:0/3836604427 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f35140529b0 con 0x7f35200f5430 2021-11-03T17:54:31.172 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:54:31.172 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:54:31.173 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:54:31.173 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:54:31.174 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:54:31.175 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:54:31.176 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:54:31.176 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:54:31.176 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:54:31.177 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:54:31.177 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:54:31.178 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:54:31.178 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:54:31.179 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:54:31.179 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:54:31.179 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:54:31.180 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:54:31.180 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:54:31.181 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:54:31.181 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:54:31.182 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:54:31.182 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:54:31.182 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:54:31.183 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:54:31.184 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:54:31.185 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:54:31.185 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:54:31.185 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:54:31.185 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:54:31.186 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:54:31.186 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:54:31.186 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:54:31.187 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:54:31.187 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:54:31.187 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:54:31.187 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:54:31.187 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:54:31.188 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:54:31.188 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:54:31.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:31.177+0000 7f3525b9d700 1 -- 172.21.15.27:0/3836604427 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3510064100 msgr2=0x7f35100665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:54:31.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:31.177+0000 7f3525b9d700 1 --2- 172.21.15.27:0/3836604427 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3510064100 0x7f35100665c0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7f350c004630 tx=0x7f350c009b30).stop 2021-11-03T17:54:31.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:31.177+0000 7f3525b9d700 1 -- 172.21.15.27:0/3836604427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35200f5430 msgr2=0x7f3520105cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:54:31.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:31.177+0000 7f3525b9d700 1 --2- 172.21.15.27:0/3836604427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35200f5430 0x7f3520105cc0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f3514002700 tx=0x7f3514009d80).stop 2021-11-03T17:54:31.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:31.177+0000 7f3525b9d700 1 -- 172.21.15.27:0/3836604427 shutdown_connections 2021-11-03T17:54:31.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:31.177+0000 7f3525b9d700 1 --2- 172.21.15.27:0/3836604427 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3510064100 0x7f35100665c0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:54:31.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:31.177+0000 7f3525b9d700 1 --2- 172.21.15.27:0/3836604427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35200f5430 0x7f3520105cc0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:54:31.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:31.177+0000 7f3525b9d700 1 --2- 172.21.15.27:0/3836604427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35200f66d0 0x7f35201061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:54:31.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:31.177+0000 7f3525b9d700 1 -- 172.21.15.27:0/3836604427 >> 172.21.15.27:0/3836604427 conn(0x7f35200f0b90 msgr2=0x7f35200fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:54:31.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:31.177+0000 7f3525b9d700 1 -- 172.21.15.27:0/3836604427 shutdown_connections 2021-11-03T17:54:31.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:54:31.177+0000 7f3525b9d700 1 -- 172.21.15.27:0/3836604427 wait complete. 2021-11-03T17:54:31.192 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:54:31.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:31 smithi115 ceph-mon[65453]: from='client.35336 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:54:31.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:31 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/549014445' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:31.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:31 smithi027 ceph-mon[97244]: from='client.35336 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:54:31.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:31 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/549014445' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:32.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:32 smithi115 ceph-mon[65453]: pgmap v1196: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:32.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:32 smithi115 ceph-mon[65453]: from='client.35340 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:54:32.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:32 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3836604427' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:54:32.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:32 smithi027 ceph-mon[97244]: pgmap v1196: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:32.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:32 smithi027 ceph-mon[97244]: from='client.35340 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:54:32.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:32 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3836604427' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:54:34.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:34 smithi115 ceph-mon[65453]: pgmap v1197: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:34.453 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:34 smithi027 ceph-mon[97244]: pgmap v1197: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:35.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:54:35.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:35 smithi115 ceph-mon[65453]: poll took 60185.967 ms: 1 events 2021-11-03T17:54:35.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2317.415277 secs 2021-11-03T17:54:35.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:54:35.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:54:35.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:35.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:35.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:35.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:35.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:54:35.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:35 smithi027 ceph-mon[97244]: poll took 60185.967 ms: 1 events 2021-11-03T17:54:35.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2317.415277 secs 2021-11-03T17:54:35.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:54:35.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:54:35.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:35.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:35.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:35.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:36.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:36 smithi115 ceph-mon[65453]: pgmap v1198: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:36.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:36 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:54:36.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:36 smithi027 ceph-mon[97244]: pgmap v1198: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:36.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:36 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:54:38.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:38 smithi115 ceph-mon[65453]: pgmap v1199: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:38.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:38 smithi027 ceph-mon[97244]: pgmap v1199: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:40.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:40 smithi115 ceph-mon[65453]: pgmap v1200: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:40.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2322.415401 secs 2021-11-03T17:54:40.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:40 smithi027 ceph-mon[97244]: pgmap v1200: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:40.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2322.415401 secs 2021-11-03T17:54:42.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:42 smithi115 ceph-mon[65453]: pgmap v1201: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:42.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:42 smithi027 ceph-mon[97244]: pgmap v1201: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:44.234 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:44 smithi115 ceph-mon[65453]: pgmap v1202: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:44.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:44 smithi027 ceph-mon[97244]: pgmap v1202: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:45.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2327.415526 secs 2021-11-03T17:54:45.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:54:45.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:54:45.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:54:45.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:45.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:45.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:45.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:45.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2327.415526 secs 2021-11-03T17:54:45.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:54:45.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:54:45.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:54:45.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:45.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:45.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:45.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:46.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:46 smithi115 ceph-mon[65453]: pgmap v1203: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:46.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:46 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:54:46.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:46 smithi027 ceph-mon[97244]: pgmap v1203: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:46.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:46 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:54:48.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:48 smithi115 ceph-mon[65453]: pgmap v1204: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:48.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:48 smithi027 ceph-mon[97244]: pgmap v1204: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:50.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:50 smithi115 ceph-mon[65453]: pgmap v1205: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:50.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2332.415653 secs 2021-11-03T17:54:50.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:50 smithi027 ceph-mon[97244]: pgmap v1205: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:50.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2332.415653 secs 2021-11-03T17:54:52.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:52 smithi115 ceph-mon[65453]: pgmap v1206: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:52.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:52 smithi027 ceph-mon[97244]: pgmap v1206: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:54.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:54 smithi115 ceph-mon[65453]: pgmap v1207: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:54.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:54 smithi027 ceph-mon[97244]: pgmap v1207: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:55.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2337.415786 secs 2021-11-03T17:54:55.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:54:55.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:54:55.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:54:55.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:55.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:55.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:55.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:55.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2337.415786 secs 2021-11-03T17:54:55.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:54:55.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:54:55.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:54:55.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:55.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:55.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:55.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:54:56.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:56 smithi115 ceph-mon[65453]: pgmap v1208: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:56.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:56 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:54:56.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:54:56.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:54:56.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:56 smithi027 ceph-mon[97244]: pgmap v1208: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:56.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:56 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:54:56.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:54:56.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:54:58.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:54:58 smithi115 ceph-mon[65453]: pgmap v1209: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:54:58.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:54:58 smithi027 ceph-mon[97244]: pgmap v1209: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:00.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:00 smithi115 ceph-mon[65453]: pgmap v1210: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:00.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2342.415918 secs 2021-11-03T17:55:00.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:00 smithi027 ceph-mon[97244]: pgmap v1210: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:00.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2342.415918 secs 2021-11-03T17:55:01.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.294+0000 7f4305fad700 1 -- 172.21.15.27:0/2596463004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43000f7970 msgr2=0x7f43000f7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:55:01.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.294+0000 7f4305fad700 1 --2- 172.21.15.27:0/2596463004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43000f7970 0x7f43000f7dd0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f42f4004660 tx=0x7f42f4009b30).stop 2021-11-03T17:55:01.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.294+0000 7f4305fad700 1 -- 172.21.15.27:0/2596463004 shutdown_connections 2021-11-03T17:55:01.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.295+0000 7f4305fad700 1 --2- 172.21.15.27:0/2596463004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43000f7970 0x7f43000f7dd0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:01.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.295+0000 7f4305fad700 1 --2- 172.21.15.27:0/2596463004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43000f66d0 0x7f43000f6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:01.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.295+0000 7f4305fad700 1 -- 172.21.15.27:0/2596463004 >> 172.21.15.27:0/2596463004 conn(0x7f43000f1e10 msgr2=0x7f43000f4250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:55:01.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.295+0000 7f4305fad700 1 -- 172.21.15.27:0/2596463004 shutdown_connections 2021-11-03T17:55:01.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.295+0000 7f4305fad700 1 -- 172.21.15.27:0/2596463004 wait complete. 2021-11-03T17:55:01.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.295+0000 7f4305fad700 1 Processor -- start 2021-11-03T17:55:01.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.295+0000 7f4305fad700 1 -- start start 2021-11-03T17:55:01.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.295+0000 7f4305fad700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43000f66d0 0x7f430006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:55:01.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.296+0000 7f4305fad700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43000f7970 0x7f4300069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:55:01.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.296+0000 7f4305fad700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430006a380 con 0x7f43000f7970 2021-11-03T17:55:01.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.296+0000 7f4305fad700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f430006a4c0 con 0x7f43000f66d0 2021-11-03T17:55:01.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.296+0000 7f42feffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43000f7970 0x7f4300069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:55:01.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.296+0000 7f42feffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43000f7970 0x7f4300069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60116/0 (socket says 172.21.15.27:60116) 2021-11-03T17:55:01.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.296+0000 7f42feffd700 1 -- 172.21.15.27:0/3040120866 learned_addr learned my addr 172.21.15.27:0/3040120866 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:55:01.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.296+0000 7f42ff7fe700 1 --2- 172.21.15.27:0/3040120866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43000f66d0 0x7f430006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:55:01.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.296+0000 7f42feffd700 1 -- 172.21.15.27:0/3040120866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43000f66d0 msgr2=0x7f430006b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:55:01.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.296+0000 7f42feffd700 1 --2- 172.21.15.27:0/3040120866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43000f66d0 0x7f430006b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:01.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.296+0000 7f42feffd700 1 -- 172.21.15.27:0/3040120866 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f4005040 con 0x7f43000f7970 2021-11-03T17:55:01.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.296+0000 7f42feffd700 1 --2- 172.21.15.27:0/3040120866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43000f7970 0x7f4300069e20 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f42f4009990 tx=0x7f42f4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:55:01.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.296+0000 7f42e7fff700 1 -- 172.21.15.27:0/3040120866 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42f4018070 con 0x7f43000f7970 2021-11-03T17:55:01.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.296+0000 7f4305fad700 1 -- 172.21.15.27:0/3040120866 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f430006a740 con 0x7f43000f7970 2021-11-03T17:55:01.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.296+0000 7f42e7fff700 1 -- 172.21.15.27:0/3040120866 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f42f4007ba0 con 0x7f43000f7970 2021-11-03T17:55:01.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.296+0000 7f42e7fff700 1 -- 172.21.15.27:0/3040120866 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f42f401c3c0 con 0x7f43000f7970 2021-11-03T17:55:01.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.296+0000 7f4305fad700 1 -- 172.21.15.27:0/3040120866 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f430006aca0 con 0x7f43000f7970 2021-11-03T17:55:01.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.297+0000 7f4305fad700 1 -- 172.21.15.27:0/3040120866 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4300045ad0 con 0x7f43000f7970 2021-11-03T17:55:01.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.298+0000 7f42e7fff700 1 -- 172.21.15.27:0/3040120866 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f42f401d420 con 0x7f43000f7970 2021-11-03T17:55:01.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.298+0000 7f42e7fff700 1 --2- 172.21.15.27:0/3040120866 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f42e80640b0 0x7f42e8066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:55:01.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.299+0000 7f42e7fff700 1 -- 172.21.15.27:0/3040120866 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f42f4082470 con 0x7f43000f7970 2021-11-03T17:55:01.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.299+0000 7f42ff7fe700 1 --2- 172.21.15.27:0/3040120866 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f42e80640b0 0x7f42e8066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:55:01.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.299+0000 7f42ff7fe700 1 --2- 172.21.15.27:0/3040120866 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f42e80640b0 0x7f42e8066570 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7f430006b730 tx=0x7f42f000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:55:01.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.303+0000 7f42e7fff700 1 -- 172.21.15.27:0/3040120866 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f42f4054830 con 0x7f43000f7970 2021-11-03T17:55:01.475 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.480+0000 7f4305fad700 1 -- 172.21.15.27:0/3040120866 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4300057f10 con 0x7f42e80640b0 2021-11-03T17:55:01.477 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.481+0000 7f42e7fff700 1 -- 172.21.15.27:0/3040120866 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f4300057f10 con 0x7f42e80640b0 2021-11-03T17:55:01.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.483+0000 7f4305fad700 1 -- 172.21.15.27:0/3040120866 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f42e80640b0 msgr2=0x7f42e8066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:55:01.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.483+0000 7f4305fad700 1 --2- 172.21.15.27:0/3040120866 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f42e80640b0 0x7f42e8066570 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 rx=0x7f430006b730 tx=0x7f42f000b040).stop 2021-11-03T17:55:01.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.483+0000 7f4305fad700 1 -- 172.21.15.27:0/3040120866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43000f7970 msgr2=0x7f4300069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:55:01.480 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.483+0000 7f4305fad700 1 --2- 172.21.15.27:0/3040120866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43000f7970 0x7f4300069e20 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f42f4009990 tx=0x7f42f4009b30).stop 2021-11-03T17:55:01.480 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.483+0000 7f4305fad700 1 -- 172.21.15.27:0/3040120866 shutdown_connections 2021-11-03T17:55:01.480 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.483+0000 7f4305fad700 1 --2- 172.21.15.27:0/3040120866 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f42e80640b0 0x7f42e8066570 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:01.480 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.483+0000 7f4305fad700 1 --2- 172.21.15.27:0/3040120866 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43000f66d0 0x7f430006b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:01.480 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.483+0000 7f4305fad700 1 --2- 172.21.15.27:0/3040120866 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43000f7970 0x7f4300069e20 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:01.481 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.483+0000 7f4305fad700 1 -- 172.21.15.27:0/3040120866 >> 172.21.15.27:0/3040120866 conn(0x7f43000f1e10 msgr2=0x7f43000fb3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:55:01.481 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.483+0000 7f4305fad700 1 -- 172.21.15.27:0/3040120866 shutdown_connections 2021-11-03T17:55:01.481 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.484+0000 7f4305fad700 1 -- 172.21.15.27:0/3040120866 wait complete. 2021-11-03T17:55:01.488 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:55:01.581 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.584+0000 7f814b347700 1 -- 172.21.15.27:0/2765323441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81440f9880 msgr2=0x7f81440fbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:55:01.582 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.584+0000 7f814b347700 1 --2- 172.21.15.27:0/2765323441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81440f9880 0x7f81440fbc70 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7f8140004660 tx=0x7f8140009b30).stop 2021-11-03T17:55:01.582 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.584+0000 7f814b347700 1 -- 172.21.15.27:0/2765323441 shutdown_connections 2021-11-03T17:55:01.582 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.584+0000 7f814b347700 1 --2- 172.21.15.27:0/2765323441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81440f9880 0x7f81440fbc70 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:01.582 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.584+0000 7f814b347700 1 --2- 172.21.15.27:0/2765323441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81440602b0 0x7f81440f9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:01.583 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.584+0000 7f814b347700 1 -- 172.21.15.27:0/2765323441 >> 172.21.15.27:0/2765323441 conn(0x7f81440f0d40 msgr2=0x7f81440f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:55:01.583 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.584+0000 7f814b347700 1 -- 172.21.15.27:0/2765323441 shutdown_connections 2021-11-03T17:55:01.583 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.584+0000 7f814b347700 1 -- 172.21.15.27:0/2765323441 wait complete. 2021-11-03T17:55:01.583 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.584+0000 7f814b347700 1 Processor -- start 2021-11-03T17:55:01.584 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.584+0000 7f814b347700 1 -- start start 2021-11-03T17:55:01.584 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.585+0000 7f814b347700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81440602b0 0x7f8144106ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:55:01.584 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.585+0000 7f814b347700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81440f9880 0x7f81441073b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:55:01.584 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.585+0000 7f814b347700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8144107960 con 0x7f81440602b0 2021-11-03T17:55:01.584 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.585+0000 7f814b347700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8144107aa0 con 0x7f81440f9880 2021-11-03T17:55:01.585 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.585+0000 7f81490e3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81440602b0 0x7f8144106ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:55:01.585 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.585+0000 7f81490e3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81440602b0 0x7f8144106ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60124/0 (socket says 172.21.15.27:60124) 2021-11-03T17:55:01.585 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.585+0000 7f81490e3700 1 -- 172.21.15.27:0/4188765265 learned_addr learned my addr 172.21.15.27:0/4188765265 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:55:01.585 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.585+0000 7f81488e2700 1 --2- 172.21.15.27:0/4188765265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81440f9880 0x7f81441073b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:55:01.586 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.585+0000 7f81490e3700 1 -- 172.21.15.27:0/4188765265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81440f9880 msgr2=0x7f81441073b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:55:01.586 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.585+0000 7f81490e3700 1 --2- 172.21.15.27:0/4188765265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81440f9880 0x7f81441073b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:01.586 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.585+0000 7f81490e3700 1 -- 172.21.15.27:0/4188765265 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8140005040 con 0x7f81440602b0 2021-11-03T17:55:01.586 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.585+0000 7f81490e3700 1 --2- 172.21.15.27:0/4188765265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81440602b0 0x7f8144106ec0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f8134002700 tx=0x7f8134009e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:55:01.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.585+0000 7f8139ffb700 1 -- 172.21.15.27:0/4188765265 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8134009320 con 0x7f81440602b0 2021-11-03T17:55:01.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.585+0000 7f814b347700 1 -- 172.21.15.27:0/4188765265 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8144108440 con 0x7f81440602b0 2021-11-03T17:55:01.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.585+0000 7f8139ffb700 1 -- 172.21.15.27:0/4188765265 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f813400f040 con 0x7f81440602b0 2021-11-03T17:55:01.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.585+0000 7f8139ffb700 1 -- 172.21.15.27:0/4188765265 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8134014820 con 0x7f81440602b0 2021-11-03T17:55:01.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.586+0000 7f814b347700 1 -- 172.21.15.27:0/4188765265 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8144108a00 con 0x7f81440602b0 2021-11-03T17:55:01.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.587+0000 7f814b347700 1 -- 172.21.15.27:0/4188765265 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8144045ad0 con 0x7f81440602b0 2021-11-03T17:55:01.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.587+0000 7f8139ffb700 1 -- 172.21.15.27:0/4188765265 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f8134009480 con 0x7f81440602b0 2021-11-03T17:55:01.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.587+0000 7f8139ffb700 1 --2- 172.21.15.27:0/4188765265 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f81300640b0 0x7f8130066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:55:01.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.587+0000 7f8139ffb700 1 -- 172.21.15.27:0/4188765265 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f8134080c30 con 0x7f81440602b0 2021-11-03T17:55:01.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.587+0000 7f81488e2700 1 --2- 172.21.15.27:0/4188765265 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f81300640b0 0x7f8130066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:55:01.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.588+0000 7f81488e2700 1 --2- 172.21.15.27:0/4188765265 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f81300640b0 0x7f8130066570 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f8140004630 tx=0x7f8140009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:55:01.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.591+0000 7f8139ffb700 1 -- 172.21.15.27:0/4188765265 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f8134052e70 con 0x7f81440602b0 2021-11-03T17:55:01.761 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.764+0000 7f814b347700 1 -- 172.21.15.27:0/4188765265 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f81440ff030 con 0x7f81300640b0 2021-11-03T17:55:01.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.779+0000 7f8139ffb700 1 -- 172.21.15.27:0/4188765265 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f8140019040 con 0x7f81300640b0 2021-11-03T17:55:01.778 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:55:01.779 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (46m) 6m ago 47m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:55:01.779 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (38m) 6m ago 47m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:55:01.779 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (38m) 5m ago 47m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:55:01.780 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (46m) 6m ago 47m 41.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:55:01.780 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (40m) 6m ago 42m 23.8M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:55:01.781 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (43m) 6m ago 43m 3844M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:55:01.781 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (38m) 5m ago 43m 23.2M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:55:01.782 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (40m) 5m ago 43m 24.9M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:55:01.782 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (40m) 6m ago 49m 535M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:55:01.784 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (39m) 5m ago 47m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:55:01.784 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (39m) 6m ago 49m 198M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:55:01.785 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (38m) 5m ago 46m 95.6M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:55:01.785 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (47m) 6m ago 47m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:55:01.785 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (46m) 5m ago 46m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:55:01.786 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (37m) 6m ago 46m 331M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:55:01.786 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (37m) 6m ago 45m 361M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:55:01.787 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (36m) 6m ago 45m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:55:01.787 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (36m) 5m ago 45m 379M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:55:01.788 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (35m) 5m ago 44m 304M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:55:01.788 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (35m) 5m ago 44m 307M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:55:01.789 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (46m) 6m ago 47m 52.9M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:55:01.790 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.781+0000 7f814b347700 1 -- 172.21.15.27:0/4188765265 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f81300640b0 msgr2=0x7f8130066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:55:01.790 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.781+0000 7f814b347700 1 --2- 172.21.15.27:0/4188765265 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f81300640b0 0x7f8130066570 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f8140004630 tx=0x7f8140009b30).stop 2021-11-03T17:55:01.791 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.781+0000 7f814b347700 1 -- 172.21.15.27:0/4188765265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81440602b0 msgr2=0x7f8144106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:55:01.791 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.781+0000 7f814b347700 1 --2- 172.21.15.27:0/4188765265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81440602b0 0x7f8144106ec0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f8134002700 tx=0x7f8134009e00).stop 2021-11-03T17:55:01.792 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.781+0000 7f814b347700 1 -- 172.21.15.27:0/4188765265 shutdown_connections 2021-11-03T17:55:01.792 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.781+0000 7f814b347700 1 --2- 172.21.15.27:0/4188765265 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f81300640b0 0x7f8130066570 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:01.793 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.781+0000 7f814b347700 1 --2- 172.21.15.27:0/4188765265 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81440602b0 0x7f8144106ec0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:01.793 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.781+0000 7f814b347700 1 --2- 172.21.15.27:0/4188765265 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81440f9880 0x7f81441073b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:01.793 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.781+0000 7f814b347700 1 -- 172.21.15.27:0/4188765265 >> 172.21.15.27:0/4188765265 conn(0x7f81440f0d40 msgr2=0x7f81440f3130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:55:01.794 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.781+0000 7f814b347700 1 -- 172.21.15.27:0/4188765265 shutdown_connections 2021-11-03T17:55:01.794 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.781+0000 7f814b347700 1 -- 172.21.15.27:0/4188765265 wait complete. 2021-11-03T17:55:01.880 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.880+0000 7fdada70e700 1 -- 172.21.15.27:0/446533023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdad40f66e0 msgr2=0x7fdad40f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:55:01.880 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.880+0000 7fdada70e700 1 --2- 172.21.15.27:0/446533023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdad40f66e0 0x7fdad40f6b00 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7fdac8004660 tx=0x7fdac8009b30).stop 2021-11-03T17:55:01.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.880+0000 7fdada70e700 1 -- 172.21.15.27:0/446533023 shutdown_connections 2021-11-03T17:55:01.882 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.880+0000 7fdada70e700 1 --2- 172.21.15.27:0/446533023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdad40f6ff0 0x7fdad40f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:55:01.882 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.880+0000 7fdada70e700 1 --2- 172.21.15.27:0/446533023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdad40f66e0 0x7fdad40f6b00 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:01.883 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.880+0000 7fdada70e700 1 -- 172.21.15.27:0/446533023 >> 172.21.15.27:0/446533023 conn(0x7fdad40f1f00 msgr2=0x7fdad40f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:55:01.884 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.880+0000 7fdada70e700 1 -- 172.21.15.27:0/446533023 shutdown_connections 2021-11-03T17:55:01.884 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.880+0000 7fdada70e700 1 -- 172.21.15.27:0/446533023 wait complete. 2021-11-03T17:55:01.884 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.880+0000 7fdada70e700 1 Processor -- start 2021-11-03T17:55:01.885 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.881+0000 7fdada70e700 1 -- start start 2021-11-03T17:55:01.885 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.881+0000 7fdada70e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdad40f66e0 0x7fdad406b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:55:01.886 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.881+0000 7fdada70e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdad40f6ff0 0x7fdad4069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:55:01.886 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.881+0000 7fdada70e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad406a410 con 0x7fdad40f66e0 2021-11-03T17:55:01.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.881+0000 7fdada70e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad406a550 con 0x7fdad40f6ff0 2021-11-03T17:55:01.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.881+0000 7fdad3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdad40f66e0 0x7fdad406b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:55:01.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.881+0000 7fdad3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdad40f66e0 0x7fdad406b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60134/0 (socket says 172.21.15.27:60134) 2021-11-03T17:55:01.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.881+0000 7fdad3fff700 1 -- 172.21.15.27:0/4158839549 learned_addr learned my addr 172.21.15.27:0/4158839549 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:55:01.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.881+0000 7fdad37fe700 1 --2- 172.21.15.27:0/4158839549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdad40f6ff0 0x7fdad4069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:55:01.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.881+0000 7fdad3fff700 1 -- 172.21.15.27:0/4158839549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdad40f6ff0 msgr2=0x7fdad4069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:55:01.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.881+0000 7fdad3fff700 1 --2- 172.21.15.27:0/4158839549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdad40f6ff0 0x7fdad4069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:01.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.881+0000 7fdad3fff700 1 -- 172.21.15.27:0/4158839549 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac8005040 con 0x7fdad40f66e0 2021-11-03T17:55:01.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.881+0000 7fdad3fff700 1 --2- 172.21.15.27:0/4158839549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdad40f66e0 0x7fdad406b9a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7fdac8009990 tx=0x7fdac8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:55:01.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.881+0000 7fdad0ff9700 1 -- 172.21.15.27:0/4158839549 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdac8018070 con 0x7fdad40f66e0 2021-11-03T17:55:01.891 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.882+0000 7fdad0ff9700 1 -- 172.21.15.27:0/4158839549 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fdac8007ca0 con 0x7fdad40f66e0 2021-11-03T17:55:01.891 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.882+0000 7fdad0ff9700 1 -- 172.21.15.27:0/4158839549 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdac801c620 con 0x7fdad40f66e0 2021-11-03T17:55:01.891 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.882+0000 7fdada70e700 1 -- 172.21.15.27:0/4158839549 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdad406a7d0 con 0x7fdad40f66e0 2021-11-03T17:55:01.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.882+0000 7fdada70e700 1 -- 172.21.15.27:0/4158839549 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad410efb0 con 0x7fdad40f66e0 2021-11-03T17:55:01.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.883+0000 7fdada70e700 1 -- 172.21.15.27:0/4158839549 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdad40f5b70 con 0x7fdad40f66e0 2021-11-03T17:55:01.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.883+0000 7fdad0ff9700 1 -- 172.21.15.27:0/4158839549 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fdac801d960 con 0x7fdad40f66e0 2021-11-03T17:55:01.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.884+0000 7fdad0ff9700 1 --2- 172.21.15.27:0/4158839549 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdabc0640b0 0x7fdabc066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:55:01.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.884+0000 7fdad0ff9700 1 -- 172.21.15.27:0/4158839549 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fdac8082050 con 0x7fdad40f66e0 2021-11-03T17:55:01.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.884+0000 7fdad37fe700 1 --2- 172.21.15.27:0/4158839549 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdabc0640b0 0x7fdabc066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:55:01.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.884+0000 7fdad37fe700 1 --2- 172.21.15.27:0/4158839549 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdabc0640b0 0x7fdabc066570 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7fdac400b830 tx=0x7fdac4008040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:55:01.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:01.888+0000 7fdad0ff9700 1 -- 172.21.15.27:0/4158839549 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fdac8054350 con 0x7fdad40f66e0 2021-11-03T17:55:02.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.118+0000 7fdada70e700 1 -- 172.21.15.27:0/4158839549 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdad4045ad0 con 0x7fdad40f66e0 2021-11-03T17:55:02.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.118+0000 7fdad0ff9700 1 -- 172.21.15.27:0/4158839549 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fdac8053f40 con 0x7fdad40f66e0 2021-11-03T17:55:02.115 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:55:02.115 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:55:02.115 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:55:02.115 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:55:02.116 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:55:02.116 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:55:02.116 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:55:02.116 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:55:02.117 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:55:02.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:55:02.117 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:55:02.117 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:55:02.117 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:55:02.118 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:55:02.118 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:55:02.118 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:55:02.118 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:55:02.119 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:55:02.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.120+0000 7fdada70e700 1 -- 172.21.15.27:0/4158839549 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdabc0640b0 msgr2=0x7fdabc066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:55:02.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.120+0000 7fdada70e700 1 --2- 172.21.15.27:0/4158839549 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdabc0640b0 0x7fdabc066570 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7fdac400b830 tx=0x7fdac4008040).stop 2021-11-03T17:55:02.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.120+0000 7fdada70e700 1 -- 172.21.15.27:0/4158839549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdad40f66e0 msgr2=0x7fdad406b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:55:02.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.120+0000 7fdada70e700 1 --2- 172.21.15.27:0/4158839549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdad40f66e0 0x7fdad406b9a0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7fdac8009990 tx=0x7fdac8009b30).stop 2021-11-03T17:55:02.121 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.120+0000 7fdada70e700 1 -- 172.21.15.27:0/4158839549 shutdown_connections 2021-11-03T17:55:02.121 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.120+0000 7fdada70e700 1 --2- 172.21.15.27:0/4158839549 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdabc0640b0 0x7fdabc066570 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:02.121 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.120+0000 7fdada70e700 1 --2- 172.21.15.27:0/4158839549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdad40f66e0 0x7fdad406b9a0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:02.121 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.120+0000 7fdada70e700 1 --2- 172.21.15.27:0/4158839549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdad40f6ff0 0x7fdad4069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:02.122 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.120+0000 7fdada70e700 1 -- 172.21.15.27:0/4158839549 >> 172.21.15.27:0/4158839549 conn(0x7fdad40f1f00 msgr2=0x7fdad40fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:55:02.122 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.120+0000 7fdada70e700 1 -- 172.21.15.27:0/4158839549 shutdown_connections 2021-11-03T17:55:02.122 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.120+0000 7fdada70e700 1 -- 172.21.15.27:0/4158839549 wait complete. 2021-11-03T17:55:02.218 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.221+0000 7fdd9359e700 1 -- 172.21.15.27:0/494545030 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd940f77d0 msgr2=0x7fdd940f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:55:02.219 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.221+0000 7fdd9359e700 1 --2- 172.21.15.27:0/494545030 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd940f77d0 0x7fdd940f7c30 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7fdd88004660 tx=0x7fdd88009b30).stop 2021-11-03T17:55:02.219 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.221+0000 7fdd9359e700 1 -- 172.21.15.27:0/494545030 shutdown_connections 2021-11-03T17:55:02.219 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.221+0000 7fdd9359e700 1 --2- 172.21.15.27:0/494545030 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd940f77d0 0x7fdd940f7c30 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:02.219 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.221+0000 7fdd9359e700 1 --2- 172.21.15.27:0/494545030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd940f6530 0x7fdd940f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:02.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.221+0000 7fdd9359e700 1 -- 172.21.15.27:0/494545030 >> 172.21.15.27:0/494545030 conn(0x7fdd940f1cd0 msgr2=0x7fdd940f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:55:02.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.221+0000 7fdd9359e700 1 -- 172.21.15.27:0/494545030 shutdown_connections 2021-11-03T17:55:02.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.221+0000 7fdd9359e700 1 -- 172.21.15.27:0/494545030 wait complete. 2021-11-03T17:55:02.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.222+0000 7fdd9359e700 1 Processor -- start 2021-11-03T17:55:02.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.222+0000 7fdd9359e700 1 -- start start 2021-11-03T17:55:02.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.222+0000 7fdd9359e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd940f6530 0x7fdd9406b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:55:02.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.222+0000 7fdd9359e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd940f77d0 0x7fdd94069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:55:02.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.222+0000 7fdd9359e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9406a380 con 0x7fdd940f6530 2021-11-03T17:55:02.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.222+0000 7fdd9359e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9406a4c0 con 0x7fdd940f77d0 2021-11-03T17:55:02.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.222+0000 7fdd9259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd940f6530 0x7fdd9406b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:55:02.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.222+0000 7fdd9259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd940f6530 0x7fdd9406b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60144/0 (socket says 172.21.15.27:60144) 2021-11-03T17:55:02.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.222+0000 7fdd9259c700 1 -- 172.21.15.27:0/3148826095 learned_addr learned my addr 172.21.15.27:0/3148826095 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:55:02.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.222+0000 7fdd9259c700 1 -- 172.21.15.27:0/3148826095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd940f77d0 msgr2=0x7fdd94069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:55:02.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.222+0000 7fdd9259c700 1 --2- 172.21.15.27:0/3148826095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd940f77d0 0x7fdd94069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:55:02.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.223+0000 7fdd9259c700 1 -- 172.21.15.27:0/3148826095 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd88005040 con 0x7fdd940f6530 2021-11-03T17:55:02.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.223+0000 7fdd9259c700 1 --2- 172.21.15.27:0/3148826095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd940f6530 0x7fdd9406b9a0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7fdd84002700 tx=0x7fdd84009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:55:02.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.223+0000 7fdd82ffd700 1 -- 172.21.15.27:0/3148826095 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd84009320 con 0x7fdd940f6530 2021-11-03T17:55:02.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.223+0000 7fdd82ffd700 1 -- 172.21.15.27:0/3148826095 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fdd8400f040 con 0x7fdd940f6530 2021-11-03T17:55:02.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.223+0000 7fdd9359e700 1 -- 172.21.15.27:0/3148826095 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd9406a7a0 con 0x7fdd940f6530 2021-11-03T17:55:02.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.223+0000 7fdd82ffd700 1 -- 172.21.15.27:0/3148826095 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd84014900 con 0x7fdd940f6530 2021-11-03T17:55:02.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.223+0000 7fdd9359e700 1 -- 172.21.15.27:0/3148826095 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd9410ed40 con 0x7fdd940f6530 2021-11-03T17:55:02.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.224+0000 7fdd9359e700 1 -- 172.21.15.27:0/3148826095 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd94045ad0 con 0x7fdd940f6530 2021-11-03T17:55:02.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.224+0000 7fdd82ffd700 1 -- 172.21.15.27:0/3148826095 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fdd84009480 con 0x7fdd940f6530 2021-11-03T17:55:02.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.225+0000 7fdd82ffd700 1 --2- 172.21.15.27:0/3148826095 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdd7c064100 0x7fdd7c0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:55:02.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.225+0000 7fdd82ffd700 1 -- 172.21.15.27:0/3148826095 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fdd84080b70 con 0x7fdd940f6530 2021-11-03T17:55:02.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.225+0000 7fdd91d9b700 1 --2- 172.21.15.27:0/3148826095 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdd7c064100 0x7fdd7c0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:55:02.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.225+0000 7fdd91d9b700 1 --2- 172.21.15.27:0/3148826095 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdd7c064100 0x7fdd7c0665c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7fdd88004e30 tx=0x7fdd88009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:55:02.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.229+0000 7fdd82ffd700 1 -- 172.21.15.27:0/3148826095 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fdd84052db0 con 0x7fdd940f6530 2021-11-03T17:55:02.433 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.437+0000 7fdd9359e700 1 -- 172.21.15.27:0/3148826095 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fdd940fee30 con 0x7fdd940f6530 2021-11-03T17:55:02.433 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.438+0000 7fdd82ffd700 1 -- 172.21.15.27:0/3148826095 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7fdd840529a0 con 0x7fdd940f6530 2021-11-03T17:55:02.434 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:55:02.434 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:55:02.434 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:55:02.435 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:55:02.435 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:55:02.435 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:55:02.436 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:55:02.436 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:55:02.436 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:55:02.436 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:55:02.437 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:55:02.437 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:55:02.437 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:55:02.437 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:55:02.438 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:55:02.438 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:55:02.438 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:55:02.438 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:55:02.439 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:55:02.439 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:55:02.439 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:55:02.439 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:55:02.439 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:55:02.440 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:55:02.440 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:55:02.440 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:55:02.440 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:55:02.441 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:55:02.441 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:55:02.441 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:55:02.441 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:55:02.442 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:55:02.442 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:55:02.442 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:55:02.442 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:55:02.443 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:55:02.443 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:55:02.443 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:55:02.443 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:55:02.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.440+0000 7fdd9359e700 1 -- 172.21.15.27:0/3148826095 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdd7c064100 msgr2=0x7fdd7c0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:55:02.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.440+0000 7fdd9359e700 1 --2- 172.21.15.27:0/3148826095 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdd7c064100 0x7fdd7c0665c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7fdd88004e30 tx=0x7fdd88009b30).stop 2021-11-03T17:55:02.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.440+0000 7fdd9359e700 1 -- 172.21.15.27:0/3148826095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd940f6530 msgr2=0x7fdd9406b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:55:02.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.440+0000 7fdd9359e700 1 --2- 172.21.15.27:0/3148826095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd940f6530 0x7fdd9406b9a0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7fdd84002700 tx=0x7fdd84009d80).stop 2021-11-03T17:55:02.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.440+0000 7fdd9359e700 1 -- 172.21.15.27:0/3148826095 shutdown_connections 2021-11-03T17:55:02.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.440+0000 7fdd9359e700 1 --2- 172.21.15.27:0/3148826095 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdd7c064100 0x7fdd7c0665c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:02.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.440+0000 7fdd9359e700 1 --2- 172.21.15.27:0/3148826095 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd940f6530 0x7fdd9406b9a0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:02.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.440+0000 7fdd9359e700 1 --2- 172.21.15.27:0/3148826095 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd940f77d0 0x7fdd94069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T17:55:02.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.440+0000 7fdd9359e700 1 -- 172.21.15.27:0/3148826095 >> 172.21.15.27:0/3148826095 conn(0x7fdd940f1cd0 msgr2=0x7fdd940fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:55:02.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.440+0000 7fdd9359e700 1 -- 172.21.15.27:0/3148826095 shutdown_connections 2021-11-03T17:55:02.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:02.440+0000 7fdd9359e700 1 -- 172.21.15.27:0/3148826095 wait complete. 2021-11-03T17:55:02.447 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:55:02.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:02 smithi115 ceph-mon[65453]: pgmap v1211: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:02.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:02 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/4158839549' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:02.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:02 smithi027 ceph-mon[97244]: pgmap v1211: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:02.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:02 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/4158839549' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:03.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:03 smithi027 ceph-mon[97244]: from='client.35352 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:55:03.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:03 smithi027 ceph-mon[97244]: from='client.35356 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:55:03.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:03 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3148826095' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:55:03.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:03 smithi115 ceph-mon[65453]: from='client.35352 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:55:03.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:03 smithi115 ceph-mon[65453]: from='client.35356 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:55:03.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:03 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3148826095' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:55:04.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:04 smithi115 ceph-mon[65453]: pgmap v1212: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:04.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:04 smithi027 ceph-mon[97244]: pgmap v1212: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:05.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2347.416059 secs 2021-11-03T17:55:05.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:55:05.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:55:05.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:55:05.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:05.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:05.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:05.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:05.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2347.416059 secs 2021-11-03T17:55:05.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:55:05.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:55:05.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:55:05.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:05.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:05.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:05.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:06.484 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:06 smithi115 ceph-mon[65453]: pgmap v1213: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:06.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:06 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:55:06.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:06 smithi027 ceph-mon[97244]: pgmap v1213: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:06.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:06 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:55:08.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:08 smithi115 ceph-mon[65453]: pgmap v1214: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:08.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:08 smithi027 ceph-mon[97244]: pgmap v1214: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:10.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:10 smithi115 ceph-mon[65453]: pgmap v1215: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:10.560 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2352.416184 secs 2021-11-03T17:55:10.561 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:10 smithi027 ceph-mon[97244]: pgmap v1215: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:10.562 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2352.416184 secs 2021-11-03T17:55:12.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:12 smithi115 ceph-mon[65453]: pgmap v1216: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:12.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:12 smithi027 ceph-mon[97244]: pgmap v1216: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:14.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:14 smithi115 ceph-mon[65453]: pgmap v1217: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:14.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:14 smithi027 ceph-mon[97244]: pgmap v1217: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:15.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2357.416329 secs 2021-11-03T17:55:15.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:55:15.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:55:15.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:55:15.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:15.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:15.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:15.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:15.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2357.416329 secs 2021-11-03T17:55:15.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:55:15.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:55:15.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:55:15.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:15.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:15.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:15.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:16.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:16 smithi115 ceph-mon[65453]: pgmap v1218: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:16.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:16 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:55:16.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:16 smithi027 ceph-mon[97244]: pgmap v1218: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:16.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:16 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:55:18.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:18 smithi115 ceph-mon[65453]: pgmap v1219: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:18.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:18 smithi027 ceph-mon[97244]: pgmap v1219: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:20.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:20 smithi115 ceph-mon[65453]: pgmap v1220: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:20.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2362.416459 secs 2021-11-03T17:55:20.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:20 smithi027 ceph-mon[97244]: pgmap v1220: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:20.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2362.416459 secs 2021-11-03T17:55:22.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:22 smithi115 ceph-mon[65453]: pgmap v1221: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:22.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:22 smithi027 ceph-mon[97244]: pgmap v1221: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:24.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:24 smithi115 ceph-mon[65453]: pgmap v1222: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:24.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:24 smithi027 ceph-mon[97244]: pgmap v1222: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:25.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2367.416587 secs 2021-11-03T17:55:25.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:55:25.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:55:25.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:55:25.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:25.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:25.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:25.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:25.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2367.416587 secs 2021-11-03T17:55:25.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:55:25.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:55:25.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:55:25.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:25.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:25.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:25.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:26.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:26 smithi115 ceph-mon[65453]: pgmap v1223: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:26.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:26 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:55:26.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:26 smithi027 ceph-mon[97244]: pgmap v1223: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:26.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:26 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:55:28.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:28 smithi115 ceph-mon[65453]: pgmap v1224: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:28.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:28 smithi027 ceph-mon[97244]: pgmap v1224: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:30.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:30 smithi115 ceph-mon[65453]: pgmap v1225: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:30.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2372.416709 secs 2021-11-03T17:55:30.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:30 smithi027 ceph-mon[97244]: pgmap v1225: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:30.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2372.416709 secs 2021-11-03T17:55:32.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:32 smithi115 ceph-mon[65453]: pgmap v1226: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:32.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:32 smithi027 ceph-mon[97244]: pgmap v1226: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:32.550 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.554+0000 7f49517c5700 1 -- 172.21.15.27:0/1590509115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f494c0f66e0 msgr2=0x7f494c0f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:55:32.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.554+0000 7f49517c5700 1 --2- 172.21.15.27:0/1590509115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f494c0f66e0 0x7f494c0f6b00 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f493c004660 tx=0x7f493c009b30).stop 2021-11-03T17:55:32.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.554+0000 7f49517c5700 1 -- 172.21.15.27:0/1590509115 shutdown_connections 2021-11-03T17:55:32.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.554+0000 7f49517c5700 1 --2- 172.21.15.27:0/1590509115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f494c0f6ff0 0x7f494c0f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:32.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.554+0000 7f49517c5700 1 --2- 172.21.15.27:0/1590509115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f494c0f66e0 0x7f494c0f6b00 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:32.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.554+0000 7f49517c5700 1 -- 172.21.15.27:0/1590509115 >> 172.21.15.27:0/1590509115 conn(0x7f494c0f1f00 msgr2=0x7f494c0f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:55:32.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.555+0000 7f49517c5700 1 -- 172.21.15.27:0/1590509115 shutdown_connections 2021-11-03T17:55:32.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.555+0000 7f49517c5700 1 -- 172.21.15.27:0/1590509115 wait complete. 2021-11-03T17:55:32.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.555+0000 7f49517c5700 1 Processor -- start 2021-11-03T17:55:32.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.555+0000 7f49517c5700 1 -- start start 2021-11-03T17:55:32.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.555+0000 7f49517c5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f494c0f66e0 0x7f494c06b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:55:32.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.555+0000 7f49517c5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f494c0f6ff0 0x7f494c069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:55:32.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.555+0000 7f49517c5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494c06a380 con 0x7f494c0f6ff0 2021-11-03T17:55:32.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.556+0000 7f49517c5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494c06a4c0 con 0x7f494c0f66e0 2021-11-03T17:55:32.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.556+0000 7f494a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f494c0f6ff0 0x7f494c069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:55:32.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.556+0000 7f494a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f494c0f6ff0 0x7f494c069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60164/0 (socket says 172.21.15.27:60164) 2021-11-03T17:55:32.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.556+0000 7f494a7fc700 1 -- 172.21.15.27:0/1680409202 learned_addr learned my addr 172.21.15.27:0/1680409202 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:55:32.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.556+0000 7f494affd700 1 --2- 172.21.15.27:0/1680409202 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f494c0f66e0 0x7f494c06b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:55:32.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.556+0000 7f494a7fc700 1 -- 172.21.15.27:0/1680409202 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f494c0f66e0 msgr2=0x7f494c06b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:55:32.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.556+0000 7f494a7fc700 1 --2- 172.21.15.27:0/1680409202 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f494c0f66e0 0x7f494c06b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:32.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.556+0000 7f494a7fc700 1 -- 172.21.15.27:0/1680409202 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f493c005040 con 0x7f494c0f6ff0 2021-11-03T17:55:32.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.556+0000 7f494a7fc700 1 --2- 172.21.15.27:0/1680409202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f494c0f6ff0 0x7f494c069e20 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7f49400026d0 tx=0x7f4940009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:55:32.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.556+0000 7f49337fe700 1 -- 172.21.15.27:0/1680409202 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49400079c0 con 0x7f494c0f6ff0 2021-11-03T17:55:32.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.556+0000 7f49337fe700 1 -- 172.21.15.27:0/1680409202 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f494000f040 con 0x7f494c0f6ff0 2021-11-03T17:55:32.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.556+0000 7f49517c5700 1 -- 172.21.15.27:0/1680409202 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f494c06a7a0 con 0x7f494c0f6ff0 2021-11-03T17:55:32.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.556+0000 7f49337fe700 1 -- 172.21.15.27:0/1680409202 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49400148b0 con 0x7f494c0f6ff0 2021-11-03T17:55:32.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.556+0000 7f49517c5700 1 -- 172.21.15.27:0/1680409202 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f494c10ef90 con 0x7f494c0f6ff0 2021-11-03T17:55:32.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.557+0000 7f49517c5700 1 -- 172.21.15.27:0/1680409202 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f494c045ad0 con 0x7f494c0f6ff0 2021-11-03T17:55:32.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.558+0000 7f49337fe700 1 -- 172.21.15.27:0/1680409202 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f4940007b20 con 0x7f494c0f6ff0 2021-11-03T17:55:32.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.558+0000 7f49337fe700 1 --2- 172.21.15.27:0/1680409202 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f49340640b0 0x7f4934066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:55:32.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.558+0000 7f49337fe700 1 -- 172.21.15.27:0/1680409202 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f49400806d0 con 0x7f494c0f6ff0 2021-11-03T17:55:32.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.558+0000 7f494affd700 1 --2- 172.21.15.27:0/1680409202 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f49340640b0 0x7f4934066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:55:32.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.559+0000 7f494affd700 1 --2- 172.21.15.27:0/1680409202 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f49340640b0 0x7f4934066570 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f493c004630 tx=0x7f493c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:55:32.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.563+0000 7f49337fe700 1 -- 172.21.15.27:0/1680409202 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f49400529d0 con 0x7f494c0f6ff0 2021-11-03T17:55:32.739 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.744+0000 7f49517c5700 1 -- 172.21.15.27:0/1680409202 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f494c057f10 con 0x7f49340640b0 2021-11-03T17:55:32.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.745+0000 7f49337fe700 1 -- 172.21.15.27:0/1680409202 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f494c057f10 con 0x7f49340640b0 2021-11-03T17:55:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.747+0000 7f49517c5700 1 -- 172.21.15.27:0/1680409202 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f49340640b0 msgr2=0x7f4934066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:55:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.747+0000 7f49517c5700 1 --2- 172.21.15.27:0/1680409202 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f49340640b0 0x7f4934066570 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 rx=0x7f493c004630 tx=0x7f493c009b30).stop 2021-11-03T17:55:32.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.747+0000 7f49517c5700 1 -- 172.21.15.27:0/1680409202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f494c0f6ff0 msgr2=0x7f494c069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:55:32.744 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.747+0000 7f49517c5700 1 --2- 172.21.15.27:0/1680409202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f494c0f6ff0 0x7f494c069e20 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7f49400026d0 tx=0x7f4940009de0).stop 2021-11-03T17:55:32.744 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.747+0000 7f49517c5700 1 -- 172.21.15.27:0/1680409202 shutdown_connections 2021-11-03T17:55:32.744 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.747+0000 7f49517c5700 1 --2- 172.21.15.27:0/1680409202 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f49340640b0 0x7f4934066570 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:32.744 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.747+0000 7f49517c5700 1 --2- 172.21.15.27:0/1680409202 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f494c0f66e0 0x7f494c06b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:32.745 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.747+0000 7f49517c5700 1 --2- 172.21.15.27:0/1680409202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f494c0f6ff0 0x7f494c069e20 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:32.745 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.747+0000 7f49517c5700 1 -- 172.21.15.27:0/1680409202 >> 172.21.15.27:0/1680409202 conn(0x7f494c0f1f00 msgr2=0x7f494c0fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:55:32.745 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.747+0000 7f49517c5700 1 -- 172.21.15.27:0/1680409202 shutdown_connections 2021-11-03T17:55:32.745 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.747+0000 7f49517c5700 1 -- 172.21.15.27:0/1680409202 wait complete. 2021-11-03T17:55:32.752 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:55:32.844 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.847+0000 7ff6d7f78700 1 -- 172.21.15.27:0/721901571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6d00f66e0 msgr2=0x7ff6d00f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:55:32.845 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.847+0000 7ff6d7f78700 1 --2- 172.21.15.27:0/721901571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6d00f66e0 0x7ff6d00f6b00 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7ff6c0004660 tx=0x7ff6c0009b30).stop 2021-11-03T17:55:32.845 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.847+0000 7ff6d7f78700 1 -- 172.21.15.27:0/721901571 shutdown_connections 2021-11-03T17:55:32.845 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.847+0000 7ff6d7f78700 1 --2- 172.21.15.27:0/721901571 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6d00f7980 0x7ff6d00f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:55:32.845 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.847+0000 7ff6d7f78700 1 --2- 172.21.15.27:0/721901571 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6d00f66e0 0x7ff6d00f6b00 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:32.845 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.847+0000 7ff6d7f78700 1 -- 172.21.15.27:0/721901571 >> 172.21.15.27:0/721901571 conn(0x7ff6d00f1e40 msgr2=0x7ff6d00f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:55:32.846 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.847+0000 7ff6d7f78700 1 -- 172.21.15.27:0/721901571 shutdown_connections 2021-11-03T17:55:32.846 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.847+0000 7ff6d7f78700 1 -- 172.21.15.27:0/721901571 wait complete. 2021-11-03T17:55:32.846 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.847+0000 7ff6d7f78700 1 Processor -- start 2021-11-03T17:55:32.847 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.847+0000 7ff6d7f78700 1 -- start start 2021-11-03T17:55:32.847 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.848+0000 7ff6d7f78700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6d00f66e0 0x7ff6d0106fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:55:32.847 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.848+0000 7ff6d7f78700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6d00f7980 0x7ff6d01074a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:55:32.847 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.848+0000 7ff6d7f78700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d0107ae0 con 0x7ff6d00f66e0 2021-11-03T17:55:32.847 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.848+0000 7ff6d7f78700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d0108390 con 0x7ff6d00f7980 2021-11-03T17:55:32.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.848+0000 7ff6d5d14700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6d00f66e0 0x7ff6d0106fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:55:32.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.848+0000 7ff6d5d14700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6d00f66e0 0x7ff6d0106fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60172/0 (socket says 172.21.15.27:60172) 2021-11-03T17:55:32.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.848+0000 7ff6d5d14700 1 -- 172.21.15.27:0/2943188716 learned_addr learned my addr 172.21.15.27:0/2943188716 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:55:32.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.848+0000 7ff6d5513700 1 --2- 172.21.15.27:0/2943188716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6d00f7980 0x7ff6d01074a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:55:32.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.848+0000 7ff6d5d14700 1 -- 172.21.15.27:0/2943188716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6d00f7980 msgr2=0x7ff6d01074a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:55:32.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.848+0000 7ff6d5d14700 1 --2- 172.21.15.27:0/2943188716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6d00f7980 0x7ff6d01074a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:32.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.848+0000 7ff6d5d14700 1 -- 172.21.15.27:0/2943188716 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c0005040 con 0x7ff6d00f66e0 2021-11-03T17:55:32.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.848+0000 7ff6d5d14700 1 --2- 172.21.15.27:0/2943188716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6d00f66e0 0x7ff6d0106fb0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7ff6c0004f60 tx=0x7ff6c0009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:55:32.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.848+0000 7ff6c67fc700 1 -- 172.21.15.27:0/2943188716 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6c0018070 con 0x7ff6d00f66e0 2021-11-03T17:55:32.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.848+0000 7ff6c67fc700 1 -- 172.21.15.27:0/2943188716 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7ff6c0007ba0 con 0x7ff6d00f66e0 2021-11-03T17:55:32.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.848+0000 7ff6d7f78700 1 -- 172.21.15.27:0/2943188716 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6d0108610 con 0x7ff6d00f66e0 2021-11-03T17:55:32.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.848+0000 7ff6c67fc700 1 -- 172.21.15.27:0/2943188716 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff6c001c620 con 0x7ff6d00f66e0 2021-11-03T17:55:32.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.848+0000 7ff6d7f78700 1 -- 172.21.15.27:0/2943188716 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d0108b70 con 0x7ff6d00f66e0 2021-11-03T17:55:32.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.849+0000 7ff6d7f78700 1 -- 172.21.15.27:0/2943188716 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6d00f5ad0 con 0x7ff6d00f66e0 2021-11-03T17:55:32.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.850+0000 7ff6c67fc700 1 -- 172.21.15.27:0/2943188716 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7ff6c001c8e0 con 0x7ff6d00f66e0 2021-11-03T17:55:32.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.850+0000 7ff6c67fc700 1 --2- 172.21.15.27:0/2943188716 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff6bc0640b0 0x7ff6bc066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:55:32.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.850+0000 7ff6c67fc700 1 -- 172.21.15.27:0/2943188716 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7ff6c0082740 con 0x7ff6d00f66e0 2021-11-03T17:55:32.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.850+0000 7ff6d5513700 1 --2- 172.21.15.27:0/2943188716 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff6bc0640b0 0x7ff6bc066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:55:32.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.851+0000 7ff6d5513700 1 --2- 172.21.15.27:0/2943188716 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff6bc0640b0 0x7ff6bc066570 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7ff6cc00b790 tx=0x7ff6cc008040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:55:32.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:32.855+0000 7ff6c67fc700 1 -- 172.21.15.27:0/2943188716 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7ff6c0054a40 con 0x7ff6d00f66e0 2021-11-03T17:55:33.024 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.028+0000 7ff6d7f78700 1 -- 172.21.15.27:0/2943188716 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff6d00fef90 con 0x7ff6bc0640b0 2021-11-03T17:55:33.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.043+0000 7ff6c67fc700 1 -- 172.21.15.27:0/2943188716 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7ff6d00fef90 con 0x7ff6bc0640b0 2021-11-03T17:55:33.039 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:55:33.039 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (47m) 6m ago 48m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:55:33.039 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (38m) 6m ago 48m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:55:33.040 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (38m) 5m ago 47m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:55:33.040 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (46m) 6m ago 48m 41.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:55:33.040 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (40m) 6m ago 43m 23.8M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:55:33.040 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (43m) 6m ago 43m 3844M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:55:33.040 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (39m) 5m ago 43m 23.2M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:55:33.041 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (40m) 5m ago 43m 24.9M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:55:33.041 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (40m) 6m ago 49m 535M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:55:33.041 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (40m) 5m ago 47m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:55:33.041 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (39m) 6m ago 49m 198M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:55:33.041 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (39m) 5m ago 47m 95.6M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:55:33.042 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (47m) 6m ago 47m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:55:33.042 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (47m) 5m ago 47m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:55:33.042 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (38m) 6m ago 46m 331M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:55:33.042 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (37m) 6m ago 46m 361M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:55:33.043 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (37m) 6m ago 46m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:55:33.043 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (36m) 5m ago 45m 379M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:55:33.043 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (36m) 5m ago 45m 304M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:55:33.043 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (36m) 5m ago 45m 307M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:55:33.044 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (46m) 6m ago 47m 52.9M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:55:33.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.045+0000 7ff6d7f78700 1 -- 172.21.15.27:0/2943188716 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff6bc0640b0 msgr2=0x7ff6bc066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:55:33.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.045+0000 7ff6d7f78700 1 --2- 172.21.15.27:0/2943188716 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff6bc0640b0 0x7ff6bc066570 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7ff6cc00b790 tx=0x7ff6cc008040).stop 2021-11-03T17:55:33.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.045+0000 7ff6d7f78700 1 -- 172.21.15.27:0/2943188716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6d00f66e0 msgr2=0x7ff6d0106fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:55:33.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.045+0000 7ff6d7f78700 1 --2- 172.21.15.27:0/2943188716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6d00f66e0 0x7ff6d0106fb0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7ff6c0004f60 tx=0x7ff6c0009b30).stop 2021-11-03T17:55:33.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.045+0000 7ff6d7f78700 1 -- 172.21.15.27:0/2943188716 shutdown_connections 2021-11-03T17:55:33.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.045+0000 7ff6d7f78700 1 --2- 172.21.15.27:0/2943188716 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff6bc0640b0 0x7ff6bc066570 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:33.046 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.045+0000 7ff6d7f78700 1 --2- 172.21.15.27:0/2943188716 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6d00f66e0 0x7ff6d0106fb0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:33.046 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.045+0000 7ff6d7f78700 1 --2- 172.21.15.27:0/2943188716 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6d00f7980 0x7ff6d01074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:33.046 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.045+0000 7ff6d7f78700 1 -- 172.21.15.27:0/2943188716 >> 172.21.15.27:0/2943188716 conn(0x7ff6d00f1e40 msgr2=0x7ff6d00fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:55:33.046 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.045+0000 7ff6d7f78700 1 -- 172.21.15.27:0/2943188716 shutdown_connections 2021-11-03T17:55:33.047 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.045+0000 7ff6d7f78700 1 -- 172.21.15.27:0/2943188716 wait complete. 2021-11-03T17:55:33.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.144+0000 7f7b9da3a700 1 -- 172.21.15.27:0/2626817484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b980f55e0 msgr2=0x7f7b980f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:55:33.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.144+0000 7f7b9da3a700 1 --2- 172.21.15.27:0/2626817484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b980f55e0 0x7f7b980f5a00 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f7b900099e0 tx=0x7f7b90004d10).stop 2021-11-03T17:55:33.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.144+0000 7f7b9da3a700 1 -- 172.21.15.27:0/2626817484 shutdown_connections 2021-11-03T17:55:33.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.144+0000 7f7b9da3a700 1 --2- 172.21.15.27:0/2626817484 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b980f5ef0 0x7f7b980f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:33.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.144+0000 7f7b9da3a700 1 --2- 172.21.15.27:0/2626817484 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b980f55e0 0x7f7b980f5a00 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:33.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.144+0000 7f7b9da3a700 1 -- 172.21.15.27:0/2626817484 >> 172.21.15.27:0/2626817484 conn(0x7f7b980f0de0 msgr2=0x7f7b980f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:55:33.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.144+0000 7f7b9da3a700 1 -- 172.21.15.27:0/2626817484 shutdown_connections 2021-11-03T17:55:33.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.144+0000 7f7b9da3a700 1 -- 172.21.15.27:0/2626817484 wait complete. 2021-11-03T17:55:33.144 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.145+0000 7f7b9da3a700 1 Processor -- start 2021-11-03T17:55:33.144 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.145+0000 7f7b9da3a700 1 -- start start 2021-11-03T17:55:33.144 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.145+0000 7f7b9da3a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b980f55e0 0x7f7b98109d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:55:33.145 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.145+0000 7f7b9da3a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b980f5ef0 0x7f7b9810c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:55:33.145 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.145+0000 7f7b9da3a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b980604e0 con 0x7f7b980f55e0 2021-11-03T17:55:33.145 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.145+0000 7f7b9da3a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b98060620 con 0x7f7b980f5ef0 2021-11-03T17:55:33.145 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.145+0000 7f7b96ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b980f55e0 0x7f7b98109d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:55:33.145 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.145+0000 7f7b96ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b980f55e0 0x7f7b98109d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60184/0 (socket says 172.21.15.27:60184) 2021-11-03T17:55:33.146 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.145+0000 7f7b96ffd700 1 -- 172.21.15.27:0/3538125762 learned_addr learned my addr 172.21.15.27:0/3538125762 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:55:33.146 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.145+0000 7f7b967fc700 1 --2- 172.21.15.27:0/3538125762 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b980f5ef0 0x7f7b9810c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:55:33.146 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.145+0000 7f7b96ffd700 1 -- 172.21.15.27:0/3538125762 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b980f5ef0 msgr2=0x7f7b9810c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:55:33.146 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.145+0000 7f7b96ffd700 1 --2- 172.21.15.27:0/3538125762 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b980f5ef0 0x7f7b9810c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:33.147 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.145+0000 7f7b96ffd700 1 -- 172.21.15.27:0/3538125762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b90009640 con 0x7f7b980f55e0 2021-11-03T17:55:33.147 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.145+0000 7f7b96ffd700 1 --2- 172.21.15.27:0/3538125762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b980f55e0 0x7f7b98109d60 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f7b90003380 tx=0x7f7b90004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:55:33.147 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.146+0000 7f7b8f7fe700 1 -- 172.21.15.27:0/3538125762 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b9000e030 con 0x7f7b980f55e0 2021-11-03T17:55:33.147 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.146+0000 7f7b8f7fe700 1 -- 172.21.15.27:0/3538125762 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f7b90007dd0 con 0x7f7b980f55e0 2021-11-03T17:55:33.148 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.146+0000 7f7b9da3a700 1 -- 172.21.15.27:0/3538125762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b9810c7b0 con 0x7f7b980f55e0 2021-11-03T17:55:33.148 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.146+0000 7f7b8f7fe700 1 -- 172.21.15.27:0/3538125762 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b90018070 con 0x7f7b980f55e0 2021-11-03T17:55:33.148 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.146+0000 7f7b9da3a700 1 -- 172.21.15.27:0/3538125762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b9810cd10 con 0x7f7b980f55e0 2021-11-03T17:55:33.148 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.147+0000 7f7b9da3a700 1 -- 172.21.15.27:0/3538125762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b98045ad0 con 0x7f7b980f55e0 2021-11-03T17:55:33.148 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.147+0000 7f7b8f7fe700 1 -- 172.21.15.27:0/3538125762 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f7b90024060 con 0x7f7b980f55e0 2021-11-03T17:55:33.149 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.148+0000 7f7b8f7fe700 1 --2- 172.21.15.27:0/3538125762 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7b84064090 0x7f7b84066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:55:33.149 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.148+0000 7f7b8f7fe700 1 -- 172.21.15.27:0/3538125762 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f7b90082290 con 0x7f7b980f55e0 2021-11-03T17:55:33.149 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.148+0000 7f7b967fc700 1 --2- 172.21.15.27:0/3538125762 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7b84064090 0x7f7b84066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:55:33.149 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.148+0000 7f7b967fc700 1 --2- 172.21.15.27:0/3538125762 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7b84064090 0x7f7b84066550 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f7b80007f80 tx=0x7f7b8000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:55:33.150 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.152+0000 7f7b8f7fe700 1 -- 172.21.15.27:0/3538125762 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f7b90054530 con 0x7f7b980f55e0 2021-11-03T17:55:33.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.379+0000 7f7b9da3a700 1 -- 172.21.15.27:0/3538125762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7b980fded0 con 0x7f7b980f55e0 2021-11-03T17:55:33.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.380+0000 7f7b8f7fe700 1 -- 172.21.15.27:0/3538125762 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f7b90054120 con 0x7f7b980f55e0 2021-11-03T17:55:33.376 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:55:33.376 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:55:33.376 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:55:33.377 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:55:33.377 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:55:33.377 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:55:33.377 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:55:33.378 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:55:33.378 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:55:33.378 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:55:33.378 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:55:33.379 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:55:33.379 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:55:33.379 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:55:33.379 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:55:33.380 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:55:33.380 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:55:33.380 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:55:33.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.381+0000 7f7b9da3a700 1 -- 172.21.15.27:0/3538125762 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7b84064090 msgr2=0x7f7b84066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:55:33.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.381+0000 7f7b9da3a700 1 --2- 172.21.15.27:0/3538125762 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7b84064090 0x7f7b84066550 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f7b80007f80 tx=0x7f7b8000b040).stop 2021-11-03T17:55:33.382 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.381+0000 7f7b9da3a700 1 -- 172.21.15.27:0/3538125762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b980f55e0 msgr2=0x7f7b98109d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:55:33.382 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.381+0000 7f7b9da3a700 1 --2- 172.21.15.27:0/3538125762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b980f55e0 0x7f7b98109d60 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f7b90003380 tx=0x7f7b90004d10).stop 2021-11-03T17:55:33.382 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.381+0000 7f7b9da3a700 1 -- 172.21.15.27:0/3538125762 shutdown_connections 2021-11-03T17:55:33.382 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.381+0000 7f7b9da3a700 1 --2- 172.21.15.27:0/3538125762 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7b84064090 0x7f7b84066550 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:33.383 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.381+0000 7f7b9da3a700 1 --2- 172.21.15.27:0/3538125762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7b980f55e0 0x7f7b98109d60 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:33.383 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.381+0000 7f7b9da3a700 1 --2- 172.21.15.27:0/3538125762 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7b980f5ef0 0x7f7b9810c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:33.383 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.381+0000 7f7b9da3a700 1 -- 172.21.15.27:0/3538125762 >> 172.21.15.27:0/3538125762 conn(0x7f7b980f0de0 msgr2=0x7f7b980fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:55:33.383 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.382+0000 7f7b9da3a700 1 -- 172.21.15.27:0/3538125762 shutdown_connections 2021-11-03T17:55:33.384 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.382+0000 7f7b9da3a700 1 -- 172.21.15.27:0/3538125762 wait complete. 2021-11-03T17:55:33.477 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.481+0000 7fe441c56700 1 -- 172.21.15.27:0/219316891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe43c0f5440 msgr2=0x7fe43c0f5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:55:33.478 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.481+0000 7fe441c56700 1 --2- 172.21.15.27:0/219316891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe43c0f5440 0x7fe43c0f5860 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7fe430004660 tx=0x7fe430009b30).stop 2021-11-03T17:55:33.478 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.481+0000 7fe441c56700 1 -- 172.21.15.27:0/219316891 shutdown_connections 2021-11-03T17:55:33.478 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.481+0000 7fe441c56700 1 --2- 172.21.15.27:0/219316891 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe43c0f66e0 0x7fe43c0f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:33.478 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.481+0000 7fe441c56700 1 --2- 172.21.15.27:0/219316891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe43c0f5440 0x7fe43c0f5860 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:33.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.481+0000 7fe441c56700 1 -- 172.21.15.27:0/219316891 >> 172.21.15.27:0/219316891 conn(0x7fe43c0f0ba0 msgr2=0x7fe43c0f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:55:33.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.481+0000 7fe441c56700 1 -- 172.21.15.27:0/219316891 shutdown_connections 2021-11-03T17:55:33.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.481+0000 7fe441c56700 1 -- 172.21.15.27:0/219316891 wait complete. 2021-11-03T17:55:33.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.482+0000 7fe441c56700 1 Processor -- start 2021-11-03T17:55:33.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.482+0000 7fe441c56700 1 -- start start 2021-11-03T17:55:33.482 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.482+0000 7fe441c56700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe43c0f5440 0x7fe43c109aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:55:33.482 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.482+0000 7fe441c56700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe43c0f66e0 0x7fe43c10bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:55:33.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.482+0000 7fe441c56700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43c0603e0 con 0x7fe43c0f66e0 2021-11-03T17:55:33.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.482+0000 7fe441c56700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe43c060520 con 0x7fe43c0f5440 2021-11-03T17:55:33.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.482+0000 7fe43bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe43c0f66e0 0x7fe43c10bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:55:33.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.482+0000 7fe43bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe43c0f66e0 0x7fe43c10bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60196/0 (socket says 172.21.15.27:60196) 2021-11-03T17:55:33.484 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.482+0000 7fe43bfff700 1 -- 172.21.15.27:0/4164598751 learned_addr learned my addr 172.21.15.27:0/4164598751 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:55:33.484 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.482+0000 7fe440c54700 1 --2- 172.21.15.27:0/4164598751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe43c0f5440 0x7fe43c109aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:55:33.484 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.483+0000 7fe43bfff700 1 -- 172.21.15.27:0/4164598751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe43c0f5440 msgr2=0x7fe43c109aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:55:33.484 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.483+0000 7fe43bfff700 1 --2- 172.21.15.27:0/4164598751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe43c0f5440 0x7fe43c109aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:33.485 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.483+0000 7fe43bfff700 1 -- 172.21.15.27:0/4164598751 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe430005040 con 0x7fe43c0f66e0 2021-11-03T17:55:33.485 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.483+0000 7fe43bfff700 1 --2- 172.21.15.27:0/4164598751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe43c0f66e0 0x7fe43c10bfa0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7fe4280026d0 tx=0x7fe428009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:55:33.486 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.483+0000 7fe4397fa700 1 -- 172.21.15.27:0/4164598751 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4280079c0 con 0x7fe43c0f66e0 2021-11-03T17:55:33.486 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.483+0000 7fe441c56700 1 -- 172.21.15.27:0/4164598751 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe43c10c550 con 0x7fe43c0f66e0 2021-11-03T17:55:33.486 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.483+0000 7fe4397fa700 1 -- 172.21.15.27:0/4164598751 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fe42800f040 con 0x7fe43c0f66e0 2021-11-03T17:55:33.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.483+0000 7fe4397fa700 1 -- 172.21.15.27:0/4164598751 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4280148b0 con 0x7fe43c0f66e0 2021-11-03T17:55:33.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.483+0000 7fe441c56700 1 -- 172.21.15.27:0/4164598751 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe43c10cb10 con 0x7fe43c0f66e0 2021-11-03T17:55:33.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.484+0000 7fe441c56700 1 -- 172.21.15.27:0/4164598751 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe43c045ad0 con 0x7fe43c0f66e0 2021-11-03T17:55:33.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.484+0000 7fe4397fa700 1 -- 172.21.15.27:0/4164598751 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fe428007b20 con 0x7fe43c0f66e0 2021-11-03T17:55:33.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.485+0000 7fe4397fa700 1 --2- 172.21.15.27:0/4164598751 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe42c064100 0x7fe42c0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:55:33.488 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.485+0000 7fe4397fa700 1 -- 172.21.15.27:0/4164598751 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fe428080790 con 0x7fe43c0f66e0 2021-11-03T17:55:33.488 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.485+0000 7fe440c54700 1 --2- 172.21.15.27:0/4164598751 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe42c064100 0x7fe42c0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:55:33.488 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.485+0000 7fe440c54700 1 --2- 172.21.15.27:0/4164598751 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe42c064100 0x7fe42c0665c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7fe430013690 tx=0x7fe430009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:55:33.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.489+0000 7fe4397fa700 1 -- 172.21.15.27:0/4164598751 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fe428052a90 con 0x7fe43c0f66e0 2021-11-03T17:55:33.689 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.693+0000 7fe441c56700 1 -- 172.21.15.27:0/4164598751 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe43c027a00 con 0x7fe43c0f66e0 2021-11-03T17:55:33.689 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.693+0000 7fe4397fa700 1 -- 172.21.15.27:0/4164598751 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7fe428052680 con 0x7fe43c0f66e0 2021-11-03T17:55:33.690 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:55:33.690 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:55:33.690 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:55:33.690 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:55:33.691 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:55:33.691 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:55:33.691 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:55:33.691 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:55:33.691 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:55:33.692 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:55:33.692 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:55:33.692 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:55:33.692 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:55:33.693 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:55:33.693 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:55:33.693 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:55:33.693 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:55:33.693 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:55:33.694 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:55:33.694 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:55:33.694 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:55:33.695 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:55:33.695 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:55:33.695 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:55:33.695 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:55:33.696 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:55:33.696 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:55:33.696 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:55:33.696 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:55:33.696 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:55:33.697 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:55:33.697 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:55:33.697 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:55:33.697 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:55:33.698 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:55:33.698 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:55:33.698 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:55:33.698 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:55:33.699 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:55:33.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.695+0000 7fe441c56700 1 -- 172.21.15.27:0/4164598751 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe42c064100 msgr2=0x7fe42c0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:55:33.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.695+0000 7fe441c56700 1 --2- 172.21.15.27:0/4164598751 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe42c064100 0x7fe42c0665c0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7fe430013690 tx=0x7fe430009b30).stop 2021-11-03T17:55:33.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.695+0000 7fe441c56700 1 -- 172.21.15.27:0/4164598751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe43c0f66e0 msgr2=0x7fe43c10bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:55:33.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.695+0000 7fe441c56700 1 --2- 172.21.15.27:0/4164598751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe43c0f66e0 0x7fe43c10bfa0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7fe4280026d0 tx=0x7fe428009de0).stop 2021-11-03T17:55:33.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.695+0000 7fe441c56700 1 -- 172.21.15.27:0/4164598751 shutdown_connections 2021-11-03T17:55:33.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.695+0000 7fe441c56700 1 --2- 172.21.15.27:0/4164598751 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe42c064100 0x7fe42c0665c0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:33.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.695+0000 7fe441c56700 1 --2- 172.21.15.27:0/4164598751 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe43c0f5440 0x7fe43c109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:33.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.695+0000 7fe441c56700 1 --2- 172.21.15.27:0/4164598751 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe43c0f66e0 0x7fe43c10bfa0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:55:33.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.695+0000 7fe441c56700 1 -- 172.21.15.27:0/4164598751 >> 172.21.15.27:0/4164598751 conn(0x7fe43c0f0ba0 msgr2=0x7fe43c0fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:55:33.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.696+0000 7fe441c56700 1 -- 172.21.15.27:0/4164598751 shutdown_connections 2021-11-03T17:55:33.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:55:33.696+0000 7fe441c56700 1 -- 172.21.15.27:0/4164598751 wait complete. 2021-11-03T17:55:33.702 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:55:34.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:34 smithi115 ceph-mon[65453]: pgmap v1227: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:34.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:34 smithi115 ceph-mon[65453]: from='client.35368 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:55:34.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:34 smithi115 ceph-mon[65453]: from='client.35372 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:55:34.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:34 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3538125762' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:34.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:34 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/4164598751' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:55:34.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:34 smithi027 ceph-mon[97244]: pgmap v1227: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:34.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:34 smithi027 ceph-mon[97244]: from='client.35368 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:55:34.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:34 smithi027 ceph-mon[97244]: from='client.35372 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:55:34.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:34 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3538125762' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:34.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:34 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/4164598751' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:55:35.269 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2377.416841 secs 2021-11-03T17:55:35.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:55:35.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:55:35.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:55:35.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:35.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:35.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:35.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:35.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2377.416841 secs 2021-11-03T17:55:35.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:55:35.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:55:35.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:55:35.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:35.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:35.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:35.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:36.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:36 smithi115 ceph-mon[65453]: pgmap v1228: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:36.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:36 smithi115 ceph-mon[65453]: poll took 60187.503 ms: 1 events 2021-11-03T17:55:36.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:36 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:55:36.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:36 smithi027 ceph-mon[97244]: pgmap v1228: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:36.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:36 smithi027 ceph-mon[97244]: poll took 60187.503 ms: 1 events 2021-11-03T17:55:36.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:36 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:55:38.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:38 smithi115 ceph-mon[65453]: pgmap v1229: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:38.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:38 smithi027 ceph-mon[97244]: pgmap v1229: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:40.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:40 smithi115 ceph-mon[65453]: pgmap v1230: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:40.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2382.416966 secs 2021-11-03T17:55:40.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:40 smithi027 ceph-mon[97244]: pgmap v1230: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:40.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2382.416966 secs 2021-11-03T17:55:42.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:42 smithi115 ceph-mon[65453]: pgmap v1231: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:42.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:42 smithi027 ceph-mon[97244]: pgmap v1231: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:44.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:44 smithi115 ceph-mon[65453]: pgmap v1232: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:44.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:44 smithi027 ceph-mon[97244]: pgmap v1232: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:45.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2387.417111 secs 2021-11-03T17:55:45.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:55:45.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:55:45.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:55:45.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:45.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:45.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:45.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:45.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2387.417111 secs 2021-11-03T17:55:45.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:55:45.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:55:45.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:55:45.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:45.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:45.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:45.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:46.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:46 smithi115 ceph-mon[65453]: pgmap v1233: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:46.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:46 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:55:46.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:46 smithi027 ceph-mon[97244]: pgmap v1233: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:46.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:46 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:55:48.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:48 smithi115 ceph-mon[65453]: pgmap v1234: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:48.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:48 smithi027 ceph-mon[97244]: pgmap v1234: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:50.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:50 smithi115 ceph-mon[65453]: pgmap v1235: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:50.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2392.417244 secs 2021-11-03T17:55:50.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:50 smithi027 ceph-mon[97244]: pgmap v1235: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:50.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2392.417244 secs 2021-11-03T17:55:52.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:52 smithi115 ceph-mon[65453]: pgmap v1236: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:52.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:52 smithi027 ceph-mon[97244]: pgmap v1236: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:54.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:54 smithi115 ceph-mon[65453]: pgmap v1237: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:54.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:54 smithi027 ceph-mon[97244]: pgmap v1237: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:55.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2397.417373 secs 2021-11-03T17:55:55.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:55:55.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:55:55.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:55:55.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:55.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:55.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:55.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:55.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2397.417373 secs 2021-11-03T17:55:55.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:55:55.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:55:55.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:55:55.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:55.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:55.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:55.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:55:56.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:56 smithi027 ceph-mon[97244]: pgmap v1238: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:56.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:56 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:55:56.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:55:56.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:55:56.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:56 smithi115 ceph-mon[65453]: pgmap v1238: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:56.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:56 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:55:56.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:55:56.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:55:58.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:55:58 smithi027 ceph-mon[97244]: pgmap v1239: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:55:58.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:55:58 smithi115 ceph-mon[65453]: pgmap v1239: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:00.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:00 smithi027 ceph-mon[97244]: pgmap v1240: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:00.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2402.417441 secs 2021-11-03T17:56:00.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:00 smithi115 ceph-mon[65453]: pgmap v1240: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:00.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2402.417441 secs 2021-11-03T17:56:02.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:02 smithi027 ceph-mon[97244]: pgmap v1241: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:02.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:02 smithi115 ceph-mon[65453]: pgmap v1241: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:03.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:03.811+0000 7f06b083d700 1 -- 172.21.15.27:0/1448971046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06ac0f55d0 msgr2=0x7f06ac0f59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:56:03.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:03.811+0000 7f06b083d700 1 --2- 172.21.15.27:0/1448971046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06ac0f55d0 0x7f06ac0f59f0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f06a4004660 tx=0x7f06a4009b30).stop 2021-11-03T17:56:03.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:03.811+0000 7f06b083d700 1 -- 172.21.15.27:0/1448971046 shutdown_connections 2021-11-03T17:56:03.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:03.811+0000 7f06b083d700 1 --2- 172.21.15.27:0/1448971046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06ac0f6870 0x7f06ac0f6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:03.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:03.811+0000 7f06b083d700 1 --2- 172.21.15.27:0/1448971046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06ac0f55d0 0x7f06ac0f59f0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:03.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:03.811+0000 7f06b083d700 1 -- 172.21.15.27:0/1448971046 >> 172.21.15.27:0/1448971046 conn(0x7f06ac0f0d30 msgr2=0x7f06ac0f3150 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:56:03.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:03.811+0000 7f06b083d700 1 -- 172.21.15.27:0/1448971046 shutdown_connections 2021-11-03T17:56:03.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:03.811+0000 7f06b083d700 1 -- 172.21.15.27:0/1448971046 wait complete. 2021-11-03T17:56:03.811 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:03.812+0000 7f06b083d700 1 Processor -- start 2021-11-03T17:56:03.811 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:03.812+0000 7f06b083d700 1 -- start start 2021-11-03T17:56:03.811 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:03.812+0000 7f06b083d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06ac0f55d0 0x7f06ac10c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:56:03.812 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:03.812+0000 7f06b083d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06ac0f6870 0x7f06ac10e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:56:03.812 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:03.812+0000 7f06b083d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac0603b0 con 0x7f06ac0f55d0 2021-11-03T17:56:03.812 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:03.812+0000 7f06b083d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06ac0604f0 con 0x7f06ac0f6870 2021-11-03T17:56:03.812 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:03.812+0000 7f06aa59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06ac0f55d0 0x7f06ac10c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:56:03.813 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:03.812+0000 7f06aa59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06ac0f55d0 0x7f06ac10c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60212/0 (socket says 172.21.15.27:60212) 2021-11-03T17:56:03.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:03.812+0000 7f06aa59c700 1 -- 172.21.15.27:0/3162676548 learned_addr learned my addr 172.21.15.27:0/3162676548 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:56:03.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:03.812+0000 7f06a9d9b700 1 --2- 172.21.15.27:0/3162676548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06ac0f6870 0x7f06ac10e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:56:03.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:03.812+0000 7f06aa59c700 1 -- 172.21.15.27:0/3162676548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06ac0f6870 msgr2=0x7f06ac10e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:56:03.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:03.812+0000 7f06aa59c700 1 --2- 172.21.15.27:0/3162676548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06ac0f6870 0x7f06ac10e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:03.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:03.812+0000 7f06aa59c700 1 -- 172.21.15.27:0/3162676548 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a4005040 con 0x7f06ac0f55d0 2021-11-03T17:56:03.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:03.812+0000 7f06a9d9b700 1 --2- 172.21.15.27:0/3162676548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06ac0f6870 0x7f06ac10e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:56:03.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:03.812+0000 7f06aa59c700 1 --2- 172.21.15.27:0/3162676548 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06ac0f55d0 0x7f06ac10c090 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f06a4013a70 tx=0x7f06a4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:56:03.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:03.813+0000 7f06a2ffd700 1 -- 172.21.15.27:0/3162676548 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06a4018070 con 0x7f06ac0f55d0 2021-11-03T17:56:03.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:03.813+0000 7f06a2ffd700 1 -- 172.21.15.27:0/3162676548 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f06a40097d0 con 0x7f06ac0f55d0 2021-11-03T17:56:03.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:03.813+0000 7f06b083d700 1 -- 172.21.15.27:0/3162676548 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06ac10ea80 con 0x7f06ac0f55d0 2021-11-03T17:56:03.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:03.813+0000 7f06a2ffd700 1 -- 172.21.15.27:0/3162676548 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06a401c3c0 con 0x7f06ac0f55d0 2021-11-03T17:56:03.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:03.813+0000 7f06b083d700 1 -- 172.21.15.27:0/3162676548 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06ac10ee50 con 0x7f06ac0f55d0 2021-11-03T17:56:03.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:03.814+0000 7f06b083d700 1 -- 172.21.15.27:0/3162676548 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06ac045ad0 con 0x7f06ac0f55d0 2021-11-03T17:56:03.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:03.814+0000 7f06a2ffd700 1 -- 172.21.15.27:0/3162676548 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f06a4007ba0 con 0x7f06ac0f55d0 2021-11-03T17:56:03.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:03.814+0000 7f06a2ffd700 1 --2- 172.21.15.27:0/3162676548 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f06980640b0 0x7f0698066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:56:03.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:03.815+0000 7f06a2ffd700 1 -- 172.21.15.27:0/3162676548 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f06a40824c0 con 0x7f06ac0f55d0 2021-11-03T17:56:03.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:03.815+0000 7f06a9d9b700 1 --2- 172.21.15.27:0/3162676548 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f06980640b0 0x7f0698066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:56:03.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:03.815+0000 7f06a9d9b700 1 --2- 172.21.15.27:0/3162676548 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f06980640b0 0x7f0698066570 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7f0694009fd0 tx=0x7f069400b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:56:03.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:03.819+0000 7f06a2ffd700 1 -- 172.21.15.27:0/3162676548 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f06a4054880 con 0x7f06ac0f55d0 2021-11-03T17:56:03.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:03.999+0000 7f06b083d700 1 -- 172.21.15.27:0/3162676548 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f06ac057f10 con 0x7f06980640b0 2021-11-03T17:56:03.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.001+0000 7f06a2ffd700 1 -- 172.21.15.27:0/3162676548 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f06ac057f10 con 0x7f06980640b0 2021-11-03T17:56:03.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.002+0000 7f06b083d700 1 -- 172.21.15.27:0/3162676548 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f06980640b0 msgr2=0x7f0698066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:56:03.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.002+0000 7f06b083d700 1 --2- 172.21.15.27:0/3162676548 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f06980640b0 0x7f0698066570 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7f0694009fd0 tx=0x7f069400b040).stop 2021-11-03T17:56:03.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.003+0000 7f06b083d700 1 -- 172.21.15.27:0/3162676548 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06ac0f55d0 msgr2=0x7f06ac10c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:56:03.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.003+0000 7f06b083d700 1 --2- 172.21.15.27:0/3162676548 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06ac0f55d0 0x7f06ac10c090 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f06a4013a70 tx=0x7f06a4009b30).stop 2021-11-03T17:56:03.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.003+0000 7f06b083d700 1 -- 172.21.15.27:0/3162676548 shutdown_connections 2021-11-03T17:56:04.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.003+0000 7f06b083d700 1 --2- 172.21.15.27:0/3162676548 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f06980640b0 0x7f0698066570 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:04.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.003+0000 7f06b083d700 1 --2- 172.21.15.27:0/3162676548 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06ac0f55d0 0x7f06ac10c090 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:04.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.003+0000 7f06b083d700 1 --2- 172.21.15.27:0/3162676548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06ac0f6870 0x7f06ac10e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:04.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.003+0000 7f06b083d700 1 -- 172.21.15.27:0/3162676548 >> 172.21.15.27:0/3162676548 conn(0x7f06ac0f0d30 msgr2=0x7f06ac0fa2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:56:04.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.003+0000 7f06b083d700 1 -- 172.21.15.27:0/3162676548 shutdown_connections 2021-11-03T17:56:04.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.003+0000 7f06b083d700 1 -- 172.21.15.27:0/3162676548 wait complete. 2021-11-03T17:56:04.008 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:56:04.100 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.103+0000 7f4fb289f700 1 -- 172.21.15.27:0/1678511630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fac0f5440 msgr2=0x7f4fac0f5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:56:04.101 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.103+0000 7f4fb289f700 1 --2- 172.21.15.27:0/1678511630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fac0f5440 0x7f4fac0f5860 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f4f9c004660 tx=0x7f4f9c009b30).stop 2021-11-03T17:56:04.101 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.103+0000 7f4fb289f700 1 -- 172.21.15.27:0/1678511630 shutdown_connections 2021-11-03T17:56:04.101 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.103+0000 7f4fb289f700 1 --2- 172.21.15.27:0/1678511630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4fac0f66e0 0x7f4fac0f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:04.101 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.103+0000 7f4fb289f700 1 --2- 172.21.15.27:0/1678511630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fac0f5440 0x7f4fac0f5860 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:04.102 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.103+0000 7f4fb289f700 1 -- 172.21.15.27:0/1678511630 >> 172.21.15.27:0/1678511630 conn(0x7f4fac0f0ba0 msgr2=0x7f4fac0f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:56:04.102 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.103+0000 7f4fb289f700 1 -- 172.21.15.27:0/1678511630 shutdown_connections 2021-11-03T17:56:04.102 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.103+0000 7f4fb289f700 1 -- 172.21.15.27:0/1678511630 wait complete. 2021-11-03T17:56:04.102 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.104+0000 7f4fb289f700 1 Processor -- start 2021-11-03T17:56:04.103 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.104+0000 7f4fb289f700 1 -- start start 2021-11-03T17:56:04.103 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.104+0000 7f4fb289f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4fac0f5440 0x7f4fac109ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:56:04.103 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.104+0000 7f4fb289f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fac0f66e0 0x7f4fac10bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:56:04.103 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.104+0000 7f4fb289f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac0603c0 con 0x7f4fac0f66e0 2021-11-03T17:56:04.103 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.104+0000 7f4fb289f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac060500 con 0x7f4fac0f5440 2021-11-03T17:56:04.104 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.104+0000 7f4fb109c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fac0f66e0 0x7f4fac10bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:56:04.104 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.104+0000 7f4fb109c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fac0f66e0 0x7f4fac10bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60224/0 (socket says 172.21.15.27:60224) 2021-11-03T17:56:04.104 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.104+0000 7f4fb109c700 1 -- 172.21.15.27:0/3761585553 learned_addr learned my addr 172.21.15.27:0/3761585553 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:56:04.104 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.104+0000 7f4fb189d700 1 --2- 172.21.15.27:0/3761585553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4fac0f5440 0x7f4fac109ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:56:04.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.104+0000 7f4fb109c700 1 -- 172.21.15.27:0/3761585553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4fac0f5440 msgr2=0x7f4fac109ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:56:04.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.104+0000 7f4fb109c700 1 --2- 172.21.15.27:0/3761585553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4fac0f5440 0x7f4fac109ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:04.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.104+0000 7f4fb109c700 1 -- 172.21.15.27:0/3761585553 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f9c005040 con 0x7f4fac0f66e0 2021-11-03T17:56:04.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.104+0000 7f4fb189d700 1 --2- 172.21.15.27:0/3761585553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4fac0f5440 0x7f4fac109ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:56:04.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.104+0000 7f4fb109c700 1 --2- 172.21.15.27:0/3761585553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fac0f66e0 0x7f4fac10bfc0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f4fa800c530 tx=0x7f4fa8007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:56:04.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.105+0000 7f4fa27fc700 1 -- 172.21.15.27:0/3761585553 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fa800f650 con 0x7f4fac0f66e0 2021-11-03T17:56:04.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.105+0000 7f4fb289f700 1 -- 172.21.15.27:0/3761585553 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fac10c570 con 0x7f4fac0f66e0 2021-11-03T17:56:04.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.105+0000 7f4fa27fc700 1 -- 172.21.15.27:0/3761585553 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f4fa800fc60 con 0x7f4fac0f66e0 2021-11-03T17:56:04.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.105+0000 7f4fa27fc700 1 -- 172.21.15.27:0/3761585553 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fa8012cd0 con 0x7f4fac0f66e0 2021-11-03T17:56:04.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.105+0000 7f4fb289f700 1 -- 172.21.15.27:0/3761585553 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fac10cb00 con 0x7f4fac0f66e0 2021-11-03T17:56:04.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.105+0000 7f4fb289f700 1 -- 172.21.15.27:0/3761585553 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fac059100 con 0x7f4fac0f66e0 2021-11-03T17:56:04.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.106+0000 7f4fa27fc700 1 -- 172.21.15.27:0/3761585553 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f4fa8010070 con 0x7f4fac0f66e0 2021-11-03T17:56:04.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.106+0000 7f4fa27fc700 1 --2- 172.21.15.27:0/3761585553 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4f98064100 0x7f4f980665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:56:04.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.106+0000 7f4fa27fc700 1 -- 172.21.15.27:0/3761585553 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f4fa8080770 con 0x7f4fac0f66e0 2021-11-03T17:56:04.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.106+0000 7f4fb189d700 1 --2- 172.21.15.27:0/3761585553 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4f98064100 0x7f4f980665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:56:04.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.107+0000 7f4fb189d700 1 --2- 172.21.15.27:0/3761585553 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4f98064100 0x7f4f980665c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f4f9c004e30 tx=0x7f4f9c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:56:04.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.110+0000 7f4fa27fc700 1 -- 172.21.15.27:0/3761585553 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f4fa8052980 con 0x7f4fac0f66e0 2021-11-03T17:56:04.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.288+0000 7f4fb289f700 1 -- 172.21.15.27:0/3761585553 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4fac0fddb0 con 0x7f4f98064100 2021-11-03T17:56:04.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.303+0000 7f4fa27fc700 1 -- 172.21.15.27:0/3761585553 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f4f9c013410 con 0x7f4f98064100 2021-11-03T17:56:04.299 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:56:04.299 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (47m) 7m ago 48m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:56:04.300 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (39m) 7m ago 48m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:56:04.300 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (39m) 6m ago 48m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:56:04.300 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (47m) 7m ago 48m 41.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:56:04.301 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (41m) 7m ago 43m 23.8M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:56:04.301 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (44m) 7m ago 44m 3844M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:56:04.301 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (39m) 6m ago 44m 23.2M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:56:04.301 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (41m) 6m ago 44m 24.9M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:56:04.301 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (41m) 7m ago 50m 535M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:56:04.302 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (40m) 6m ago 48m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:56:04.302 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (40m) 7m ago 50m 198M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:56:04.302 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (39m) 6m ago 47m 95.6M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:56:04.302 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (48m) 7m ago 48m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:56:04.303 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (47m) 6m ago 47m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:56:04.303 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (38m) 7m ago 47m 331M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:56:04.303 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (38m) 7m ago 46m 361M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:56:04.303 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (37m) 7m ago 46m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:56:04.303 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (37m) 6m ago 46m 379M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:56:04.304 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (37m) 6m ago 45m 304M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:56:04.304 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (36m) 6m ago 45m 307M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:56:04.304 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (47m) 7m ago 48m 52.9M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:56:04.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.305+0000 7f4fb289f700 1 -- 172.21.15.27:0/3761585553 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4f98064100 msgr2=0x7f4f980665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:56:04.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.305+0000 7f4fb289f700 1 --2- 172.21.15.27:0/3761585553 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4f98064100 0x7f4f980665c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f4f9c004e30 tx=0x7f4f9c009b30).stop 2021-11-03T17:56:04.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.305+0000 7f4fb289f700 1 -- 172.21.15.27:0/3761585553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fac0f66e0 msgr2=0x7f4fac10bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:56:04.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.305+0000 7f4fb289f700 1 --2- 172.21.15.27:0/3761585553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fac0f66e0 0x7f4fac10bfc0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f4fa800c530 tx=0x7f4fa8007e20).stop 2021-11-03T17:56:04.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.305+0000 7f4fb289f700 1 -- 172.21.15.27:0/3761585553 shutdown_connections 2021-11-03T17:56:04.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.305+0000 7f4fb289f700 1 --2- 172.21.15.27:0/3761585553 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4f98064100 0x7f4f980665c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:04.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.305+0000 7f4fb289f700 1 --2- 172.21.15.27:0/3761585553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4fac0f5440 0x7f4fac109ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:04.307 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.305+0000 7f4fb289f700 1 --2- 172.21.15.27:0/3761585553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fac0f66e0 0x7f4fac10bfc0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:04.307 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.305+0000 7f4fb289f700 1 -- 172.21.15.27:0/3761585553 >> 172.21.15.27:0/3761585553 conn(0x7f4fac0f0ba0 msgr2=0x7f4fac0fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:56:04.307 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.305+0000 7f4fb289f700 1 -- 172.21.15.27:0/3761585553 shutdown_connections 2021-11-03T17:56:04.307 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.305+0000 7f4fb289f700 1 -- 172.21.15.27:0/3761585553 wait complete. 2021-11-03T17:56:04.402 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.404+0000 7f458cc45700 1 -- 172.21.15.27:0/600201793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45880f7970 msgr2=0x7f45880f7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:56:04.403 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.404+0000 7f458cc45700 1 --2- 172.21.15.27:0/600201793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45880f7970 0x7f45880f7dd0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7f4570004660 tx=0x7f4570009b30).stop 2021-11-03T17:56:04.403 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.404+0000 7f458cc45700 1 -- 172.21.15.27:0/600201793 shutdown_connections 2021-11-03T17:56:04.403 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.404+0000 7f458cc45700 1 --2- 172.21.15.27:0/600201793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45880f7970 0x7f45880f7dd0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:04.403 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.404+0000 7f458cc45700 1 --2- 172.21.15.27:0/600201793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45880f66d0 0x7f45880f6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:04.404 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.404+0000 7f458cc45700 1 -- 172.21.15.27:0/600201793 >> 172.21.15.27:0/600201793 conn(0x7f45880f1e50 msgr2=0x7f45880f4270 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:56:04.404 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.404+0000 7f458cc45700 1 -- 172.21.15.27:0/600201793 shutdown_connections 2021-11-03T17:56:04.404 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.405+0000 7f458cc45700 1 -- 172.21.15.27:0/600201793 wait complete. 2021-11-03T17:56:04.404 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.405+0000 7f458cc45700 1 Processor -- start 2021-11-03T17:56:04.405 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.405+0000 7f458cc45700 1 -- start start 2021-11-03T17:56:04.405 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.405+0000 7f458cc45700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45880f66d0 0x7f4588106f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:56:04.405 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.405+0000 7f458cc45700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45880f7970 0x7f4588107440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:56:04.405 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.405+0000 7f458cc45700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45881079f0 con 0x7f45880f7970 2021-11-03T17:56:04.406 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.405+0000 7f458cc45700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4588107b30 con 0x7f45880f66d0 2021-11-03T17:56:04.406 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.405+0000 7f4585d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45880f7970 0x7f4588107440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:56:04.406 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.405+0000 7f4585d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45880f7970 0x7f4588107440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60234/0 (socket says 172.21.15.27:60234) 2021-11-03T17:56:04.406 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.405+0000 7f4585d9b700 1 -- 172.21.15.27:0/1391933845 learned_addr learned my addr 172.21.15.27:0/1391933845 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:56:04.407 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.405+0000 7f458659c700 1 --2- 172.21.15.27:0/1391933845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45880f66d0 0x7f4588106f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:56:04.407 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.406+0000 7f4585d9b700 1 -- 172.21.15.27:0/1391933845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45880f66d0 msgr2=0x7f4588106f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:56:04.407 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.406+0000 7f4585d9b700 1 --2- 172.21.15.27:0/1391933845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45880f66d0 0x7f4588106f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:04.408 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.406+0000 7f4585d9b700 1 -- 172.21.15.27:0/1391933845 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4570005040 con 0x7f45880f7970 2021-11-03T17:56:04.408 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.406+0000 7f458659c700 1 --2- 172.21.15.27:0/1391933845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45880f66d0 0x7f4588106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:56:04.408 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.406+0000 7f4585d9b700 1 --2- 172.21.15.27:0/1391933845 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45880f7970 0x7f4588107440 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7f4570009990 tx=0x7f4570009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:56:04.409 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.406+0000 7f457effd700 1 -- 172.21.15.27:0/1391933845 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4570018070 con 0x7f45880f7970 2021-11-03T17:56:04.409 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.406+0000 7f457effd700 1 -- 172.21.15.27:0/1391933845 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f4570007ba0 con 0x7f45880f7970 2021-11-03T17:56:04.409 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.406+0000 7f457effd700 1 -- 172.21.15.27:0/1391933845 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f457001c3c0 con 0x7f45880f7970 2021-11-03T17:56:04.409 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.406+0000 7f458cc45700 1 -- 172.21.15.27:0/1391933845 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4588108470 con 0x7f45880f7970 2021-11-03T17:56:04.410 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.406+0000 7f458cc45700 1 -- 172.21.15.27:0/1391933845 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45881089d0 con 0x7f45880f7970 2021-11-03T17:56:04.410 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.407+0000 7f458cc45700 1 -- 172.21.15.27:0/1391933845 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4588045ad0 con 0x7f45880f7970 2021-11-03T17:56:04.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.407+0000 7f457effd700 1 -- 172.21.15.27:0/1391933845 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f457001d420 con 0x7f45880f7970 2021-11-03T17:56:04.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.408+0000 7f457effd700 1 --2- 172.21.15.27:0/1391933845 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f45740640b0 0x7f4574066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:56:04.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.408+0000 7f457effd700 1 -- 172.21.15.27:0/1391933845 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f4570082630 con 0x7f45880f7970 2021-11-03T17:56:04.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.408+0000 7f458659c700 1 --2- 172.21.15.27:0/1391933845 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f45740640b0 0x7f4574066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:56:04.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.408+0000 7f458659c700 1 --2- 172.21.15.27:0/1391933845 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f45740640b0 0x7f4574066570 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f45880f77d0 tx=0x7f4578009c00).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:56:04.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.412+0000 7f457effd700 1 -- 172.21.15.27:0/1391933845 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f4570054990 con 0x7f45880f7970 2021-11-03T17:56:04.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:04 smithi027 ceph-mon[97244]: pgmap v1242: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:04.639 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.643+0000 7f458cc45700 1 -- 172.21.15.27:0/1391933845 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f45880fef90 con 0x7f45880f7970 2021-11-03T17:56:04.640 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.644+0000 7f457effd700 1 -- 172.21.15.27:0/1391933845 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f4570054580 con 0x7f45880f7970 2021-11-03T17:56:04.640 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:56:04.640 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:56:04.640 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:56:04.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:56:04.641 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:56:04.641 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:56:04.641 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:56:04.642 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:56:04.642 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:56:04.642 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:56:04.642 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:56:04.642 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:56:04.643 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:56:04.643 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:56:04.643 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:56:04.643 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:56:04.644 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:56:04.644 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:56:04.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.645+0000 7f458cc45700 1 -- 172.21.15.27:0/1391933845 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f45740640b0 msgr2=0x7f4574066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:56:04.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.645+0000 7f458cc45700 1 --2- 172.21.15.27:0/1391933845 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f45740640b0 0x7f4574066570 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f45880f77d0 tx=0x7f4578009c00).stop 2021-11-03T17:56:04.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.645+0000 7f458cc45700 1 -- 172.21.15.27:0/1391933845 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45880f7970 msgr2=0x7f4588107440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:56:04.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.645+0000 7f458cc45700 1 --2- 172.21.15.27:0/1391933845 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45880f7970 0x7f4588107440 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7f4570009990 tx=0x7f4570009b30).stop 2021-11-03T17:56:04.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.645+0000 7f458cc45700 1 -- 172.21.15.27:0/1391933845 shutdown_connections 2021-11-03T17:56:04.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.645+0000 7f458cc45700 1 --2- 172.21.15.27:0/1391933845 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f45740640b0 0x7f4574066570 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:04.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.645+0000 7f458cc45700 1 --2- 172.21.15.27:0/1391933845 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f45880f66d0 0x7f4588106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:04.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.645+0000 7f458cc45700 1 --2- 172.21.15.27:0/1391933845 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f45880f7970 0x7f4588107440 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:04.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.645+0000 7f458cc45700 1 -- 172.21.15.27:0/1391933845 >> 172.21.15.27:0/1391933845 conn(0x7f45880f1e50 msgr2=0x7f45880fb3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:56:04.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.646+0000 7f458cc45700 1 -- 172.21.15.27:0/1391933845 shutdown_connections 2021-11-03T17:56:04.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.646+0000 7f458cc45700 1 -- 172.21.15.27:0/1391933845 wait complete. 2021-11-03T17:56:04.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:04 smithi115 ceph-mon[65453]: pgmap v1242: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:04.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.745+0000 7f1a1183a700 1 -- 172.21.15.27:0/581540846 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a0c0f6860 msgr2=0x7f1a0c0f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:56:04.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.745+0000 7f1a1183a700 1 --2- 172.21.15.27:0/581540846 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a0c0f6860 0x7f1a0c0f6cc0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7f1a00004660 tx=0x7f1a00009b30).stop 2021-11-03T17:56:04.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.745+0000 7f1a1183a700 1 -- 172.21.15.27:0/581540846 shutdown_connections 2021-11-03T17:56:04.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.745+0000 7f1a1183a700 1 --2- 172.21.15.27:0/581540846 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a0c0f6860 0x7f1a0c0f6cc0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:04.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.745+0000 7f1a1183a700 1 --2- 172.21.15.27:0/581540846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a0c0f55c0 0x7f1a0c0f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:04.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.745+0000 7f1a1183a700 1 -- 172.21.15.27:0/581540846 >> 172.21.15.27:0/581540846 conn(0x7f1a0c0f0d40 msgr2=0x7f1a0c0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:56:04.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.745+0000 7f1a1183a700 1 -- 172.21.15.27:0/581540846 shutdown_connections 2021-11-03T17:56:04.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.745+0000 7f1a1183a700 1 -- 172.21.15.27:0/581540846 wait complete. 2021-11-03T17:56:04.745 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.746+0000 7f1a1183a700 1 Processor -- start 2021-11-03T17:56:04.745 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.746+0000 7f1a1183a700 1 -- start start 2021-11-03T17:56:04.745 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.746+0000 7f1a1183a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a0c0f55c0 0x7f1a0c1090f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:56:04.746 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.746+0000 7f1a1183a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a0c0f6860 0x7f1a0c1095e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:56:04.746 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.746+0000 7f1a1183a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c109b90 con 0x7f1a0c0f55c0 2021-11-03T17:56:04.746 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.746+0000 7f1a1183a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a0c109cd0 con 0x7f1a0c0f6860 2021-11-03T17:56:04.746 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.746+0000 7f1a0affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a0c0f55c0 0x7f1a0c1090f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:56:04.746 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.746+0000 7f1a0affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a0c0f55c0 0x7f1a0c1090f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60242/0 (socket says 172.21.15.27:60242) 2021-11-03T17:56:04.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.746+0000 7f1a0affd700 1 -- 172.21.15.27:0/1622099585 learned_addr learned my addr 172.21.15.27:0/1622099585 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:56:04.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.746+0000 7f1a0a7fc700 1 --2- 172.21.15.27:0/1622099585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a0c0f6860 0x7f1a0c1095e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:56:04.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.746+0000 7f1a0affd700 1 -- 172.21.15.27:0/1622099585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a0c0f6860 msgr2=0x7f1a0c1095e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:56:04.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.746+0000 7f1a0affd700 1 --2- 172.21.15.27:0/1622099585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a0c0f6860 0x7f1a0c1095e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:04.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.746+0000 7f1a0affd700 1 -- 172.21.15.27:0/1622099585 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a00005040 con 0x7f1a0c0f55c0 2021-11-03T17:56:04.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.746+0000 7f1a0a7fc700 1 --2- 172.21.15.27:0/1622099585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a0c0f6860 0x7f1a0c1095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:56:04.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.746+0000 7f1a0affd700 1 --2- 172.21.15.27:0/1622099585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a0c0f55c0 0x7f1a0c1090f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7f1a0400eb70 tx=0x7f1a0400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:56:04.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.747+0000 7f19f37fe700 1 -- 172.21.15.27:0/1622099585 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a040092e0 con 0x7f1a0c0f55c0 2021-11-03T17:56:04.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.747+0000 7f19f37fe700 1 -- 172.21.15.27:0/1622099585 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f1a0400f040 con 0x7f1a0c0f55c0 2021-11-03T17:56:04.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.747+0000 7f1a1183a700 1 -- 172.21.15.27:0/1622099585 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a0c10a670 con 0x7f1a0c0f55c0 2021-11-03T17:56:04.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.747+0000 7f19f37fe700 1 -- 172.21.15.27:0/1622099585 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1a0400c9a0 con 0x7f1a0c0f55c0 2021-11-03T17:56:04.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.747+0000 7f1a1183a700 1 -- 172.21.15.27:0/1622099585 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a0c10ac30 con 0x7f1a0c0f55c0 2021-11-03T17:56:04.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.748+0000 7f1a1183a700 1 -- 172.21.15.27:0/1622099585 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a0c045ad0 con 0x7f1a0c0f55c0 2021-11-03T17:56:04.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.748+0000 7f19f37fe700 1 -- 172.21.15.27:0/1622099585 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f1a0401b030 con 0x7f1a0c0f55c0 2021-11-03T17:56:04.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.749+0000 7f19f37fe700 1 --2- 172.21.15.27:0/1622099585 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f19f4064090 0x7f19f4066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:56:04.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.749+0000 7f19f37fe700 1 -- 172.21.15.27:0/1622099585 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f1a04080b30 con 0x7f1a0c0f55c0 2021-11-03T17:56:04.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.749+0000 7f1a0a7fc700 1 --2- 172.21.15.27:0/1622099585 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f19f4064090 0x7f19f4066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:56:04.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.749+0000 7f1a0a7fc700 1 --2- 172.21.15.27:0/1622099585 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f19f4064090 0x7f19f4066550 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f1a000043d0 tx=0x7f1a0000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:56:04.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.753+0000 7f19f37fe700 1 -- 172.21.15.27:0/1622099585 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f1a04052d40 con 0x7f1a0c0f55c0 2021-11-03T17:56:04.953 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.957+0000 7f1a1183a700 1 -- 172.21.15.27:0/1622099585 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1a0c10f710 con 0x7f1a0c0f55c0 2021-11-03T17:56:04.954 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.958+0000 7f19f37fe700 1 -- 172.21.15.27:0/1622099585 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f1a04019710 con 0x7f1a0c0f55c0 2021-11-03T17:56:04.954 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:56:04.955 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:56:04.955 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:56:04.955 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:56:04.955 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:56:04.956 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:56:04.956 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:56:04.956 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:56:04.956 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:56:04.957 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:56:04.957 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:56:04.957 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:56:04.957 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:56:04.958 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:56:04.958 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:56:04.958 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:56:04.959 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:56:04.959 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:56:04.959 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:56:04.959 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:56:04.960 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:56:04.960 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:56:04.960 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:56:04.960 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:56:04.961 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:56:04.961 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:56:04.961 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:56:04.961 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:56:04.962 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:56:04.962 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:56:04.962 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:56:04.962 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:56:04.963 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:56:04.963 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:56:04.964 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:56:04.964 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:56:04.964 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:56:04.964 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:56:04.965 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:56:04.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.960+0000 7f1a1183a700 1 -- 172.21.15.27:0/1622099585 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f19f4064090 msgr2=0x7f19f4066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:56:04.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.960+0000 7f1a1183a700 1 --2- 172.21.15.27:0/1622099585 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f19f4064090 0x7f19f4066550 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f1a000043d0 tx=0x7f1a0000b040).stop 2021-11-03T17:56:04.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.960+0000 7f1a1183a700 1 -- 172.21.15.27:0/1622099585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a0c0f55c0 msgr2=0x7f1a0c1090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:56:04.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.960+0000 7f1a1183a700 1 --2- 172.21.15.27:0/1622099585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a0c0f55c0 0x7f1a0c1090f0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7f1a0400eb70 tx=0x7f1a0400b040).stop 2021-11-03T17:56:04.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.960+0000 7f1a1183a700 1 -- 172.21.15.27:0/1622099585 shutdown_connections 2021-11-03T17:56:04.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.960+0000 7f1a1183a700 1 --2- 172.21.15.27:0/1622099585 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f19f4064090 0x7f19f4066550 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:04.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.960+0000 7f1a1183a700 1 --2- 172.21.15.27:0/1622099585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1a0c0f55c0 0x7f1a0c1090f0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:04.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.960+0000 7f1a1183a700 1 --2- 172.21.15.27:0/1622099585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1a0c0f6860 0x7f1a0c1095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:04.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.960+0000 7f1a1183a700 1 -- 172.21.15.27:0/1622099585 >> 172.21.15.27:0/1622099585 conn(0x7f1a0c0f0d40 msgr2=0x7f1a0c05f620 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:56:04.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.960+0000 7f1a1183a700 1 -- 172.21.15.27:0/1622099585 shutdown_connections 2021-11-03T17:56:04.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:04.960+0000 7f1a1183a700 1 -- 172.21.15.27:0/1622099585 wait complete. 2021-11-03T17:56:04.968 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:56:05.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:05 smithi027 ceph-mon[97244]: from='client.35384 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:56:05.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:05 smithi027 ceph-mon[97244]: from='client.35388 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:56:05.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2407.417529 secs 2021-11-03T17:56:05.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:05 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1391933845' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:05.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:05 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1622099585' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:56:05.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:56:05.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:56:05.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:56:05.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:05.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:05.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:05.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:05.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:05 smithi115 ceph-mon[65453]: from='client.35384 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:56:05.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:05 smithi115 ceph-mon[65453]: from='client.35388 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:56:05.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2407.417529 secs 2021-11-03T17:56:05.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:05 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1391933845' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:05.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:05 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1622099585' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:56:05.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:56:05.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:56:05.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:56:05.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:05.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:05.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:05.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:06.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:06 smithi027 ceph-mon[97244]: pgmap v1243: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:06.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:06 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:56:06.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:06 smithi115 ceph-mon[65453]: pgmap v1243: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:06.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:06 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:56:08.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:08 smithi027 ceph-mon[97244]: pgmap v1244: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:08.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:08 smithi115 ceph-mon[65453]: pgmap v1244: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:10.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:10 smithi027 ceph-mon[97244]: pgmap v1245: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:10.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2412.417675 secs 2021-11-03T17:56:10.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:10 smithi115 ceph-mon[65453]: pgmap v1245: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:10.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2412.417675 secs 2021-11-03T17:56:12.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:12 smithi027 ceph-mon[97244]: pgmap v1246: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:12.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:12 smithi115 ceph-mon[65453]: pgmap v1246: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:14.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:14 smithi027 ceph-mon[97244]: pgmap v1247: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:14.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:14 smithi115 ceph-mon[65453]: pgmap v1247: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:15.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2417.417790 secs 2021-11-03T17:56:15.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:56:15.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:56:15.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:56:15.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:15.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:15.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:15.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:15.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2417.417790 secs 2021-11-03T17:56:15.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:56:15.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:56:15.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:56:15.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:15.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:15.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:15.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:16.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:16 smithi115 ceph-mon[65453]: pgmap v1248: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:16.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:16 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:56:16.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:16 smithi027 ceph-mon[97244]: pgmap v1248: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:16.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:16 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:56:18.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:18 smithi115 ceph-mon[65453]: pgmap v1249: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:18.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:18 smithi027 ceph-mon[97244]: pgmap v1249: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:20.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:20 smithi115 ceph-mon[65453]: pgmap v1250: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:20.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2422.417932 secs 2021-11-03T17:56:20.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:20 smithi027 ceph-mon[97244]: pgmap v1250: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:20.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2422.417932 secs 2021-11-03T17:56:22.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:22 smithi115 ceph-mon[65453]: pgmap v1251: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:22.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:22 smithi027 ceph-mon[97244]: pgmap v1251: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:24.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:24 smithi115 ceph-mon[65453]: pgmap v1252: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:24.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:24 smithi027 ceph-mon[97244]: pgmap v1252: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:25.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2427.418070 secs 2021-11-03T17:56:25.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:56:25.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"}]: dispatch 2021-11-03T17:56:25.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2021-11-03T17:56:25.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:56:25.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:56:25.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:25.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:25.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:25.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:25.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2427.418070 secs 2021-11-03T17:56:25.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:56:25.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"}]: dispatch 2021-11-03T17:56:25.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2021-11-03T17:56:25.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:56:25.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:56:25.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:25.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:25.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:25.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:26.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:26 smithi115 ceph-mon[65453]: pgmap v1253: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:26.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:26 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:56:26.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:26 smithi027 ceph-mon[97244]: pgmap v1253: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:26.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:26 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:56:28.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:28 smithi115 ceph-mon[65453]: pgmap v1254: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:28.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:28 smithi027 ceph-mon[97244]: pgmap v1254: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:30.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:30 smithi115 ceph-mon[65453]: pgmap v1255: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:30.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2432.418214 secs 2021-11-03T17:56:30.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:30 smithi027 ceph-mon[97244]: pgmap v1255: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:30.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2432.418214 secs 2021-11-03T17:56:32.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:32 smithi115 ceph-mon[65453]: pgmap v1256: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:32.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:32 smithi027 ceph-mon[97244]: pgmap v1256: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:34.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:34 smithi115 ceph-mon[65453]: pgmap v1257: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:34.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:34 smithi027 ceph-mon[97244]: pgmap v1257: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:35.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.073+0000 7ffbcb59e700 1 -- 172.21.15.27:0/67547451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbcc0fba90 msgr2=0x7ffbcc0fde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:56:35.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.073+0000 7ffbcb59e700 1 --2- 172.21.15.27:0/67547451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbcc0fba90 0x7ffbcc0fde80 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7ffbc0004690 tx=0x7ffbc0009b30).stop 2021-11-03T17:56:35.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.073+0000 7ffbcb59e700 1 -- 172.21.15.27:0/67547451 shutdown_connections 2021-11-03T17:56:35.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.073+0000 7ffbcb59e700 1 --2- 172.21.15.27:0/67547451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbcc0fba90 0x7ffbcc0fde80 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:35.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.073+0000 7ffbcb59e700 1 --2- 172.21.15.27:0/67547451 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffbcc060290 0x7ffbcc0fb5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:35.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.073+0000 7ffbcb59e700 1 -- 172.21.15.27:0/67547451 >> 172.21.15.27:0/67547451 conn(0x7ffbcc0f0ba0 msgr2=0x7ffbcc0f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:56:35.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.073+0000 7ffbcb59e700 1 -- 172.21.15.27:0/67547451 shutdown_connections 2021-11-03T17:56:35.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.073+0000 7ffbcb59e700 1 -- 172.21.15.27:0/67547451 wait complete. 2021-11-03T17:56:35.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.073+0000 7ffbcb59e700 1 Processor -- start 2021-11-03T17:56:35.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.074+0000 7ffbcb59e700 1 -- start start 2021-11-03T17:56:35.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.074+0000 7ffbcb59e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbcc060290 0x7ffbcc108ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:56:35.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.074+0000 7ffbcb59e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffbcc0fba90 0x7ffbcc1093e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:56:35.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.074+0000 7ffbcb59e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbcc109990 con 0x7ffbcc060290 2021-11-03T17:56:35.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.074+0000 7ffbcb59e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbcc109ad0 con 0x7ffbcc0fba90 2021-11-03T17:56:35.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.074+0000 7ffbca59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbcc060290 0x7ffbcc108ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:56:35.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.074+0000 7ffbca59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbcc060290 0x7ffbcc108ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60260/0 (socket says 172.21.15.27:60260) 2021-11-03T17:56:35.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.074+0000 7ffbca59c700 1 -- 172.21.15.27:0/4171865560 learned_addr learned my addr 172.21.15.27:0/4171865560 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:56:35.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.074+0000 7ffbc9d9b700 1 --2- 172.21.15.27:0/4171865560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffbcc0fba90 0x7ffbcc1093e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:56:35.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.074+0000 7ffbca59c700 1 -- 172.21.15.27:0/4171865560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffbcc0fba90 msgr2=0x7ffbcc1093e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:56:35.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.074+0000 7ffbca59c700 1 --2- 172.21.15.27:0/4171865560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffbcc0fba90 0x7ffbcc1093e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:35.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.074+0000 7ffbca59c700 1 -- 172.21.15.27:0/4171865560 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc0005040 con 0x7ffbcc060290 2021-11-03T17:56:35.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.074+0000 7ffbca59c700 1 --2- 172.21.15.27:0/4171865560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbcc060290 0x7ffbcc108ef0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7ffbbc00c530 tx=0x7ffbbc007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:56:35.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.075+0000 7ffbbaffd700 1 -- 172.21.15.27:0/4171865560 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffbbc00f650 con 0x7ffbcc060290 2021-11-03T17:56:35.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.075+0000 7ffbbaffd700 1 -- 172.21.15.27:0/4171865560 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7ffbbc00fc60 con 0x7ffbcc060290 2021-11-03T17:56:35.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.075+0000 7ffbcb59e700 1 -- 172.21.15.27:0/4171865560 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbcc10a470 con 0x7ffbcc060290 2021-11-03T17:56:35.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.075+0000 7ffbbaffd700 1 -- 172.21.15.27:0/4171865560 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffbbc012cd0 con 0x7ffbcc060290 2021-11-03T17:56:35.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.075+0000 7ffbcb59e700 1 -- 172.21.15.27:0/4171865560 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbcc10aa30 con 0x7ffbcc060290 2021-11-03T17:56:35.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.076+0000 7ffbcb59e700 1 -- 172.21.15.27:0/4171865560 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbcc045ad0 con 0x7ffbcc060290 2021-11-03T17:56:35.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.076+0000 7ffbbaffd700 1 -- 172.21.15.27:0/4171865560 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7ffbbc010070 con 0x7ffbcc060290 2021-11-03T17:56:35.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.076+0000 7ffbbaffd700 1 --2- 172.21.15.27:0/4171865560 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffbb4064100 0x7ffbb40665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:56:35.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.077+0000 7ffbbaffd700 1 -- 172.21.15.27:0/4171865560 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7ffbbc080740 con 0x7ffbcc060290 2021-11-03T17:56:35.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.077+0000 7ffbc9d9b700 1 --2- 172.21.15.27:0/4171865560 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffbb4064100 0x7ffbb40665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:56:35.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.077+0000 7ffbc9d9b700 1 --2- 172.21.15.27:0/4171865560 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffbb4064100 0x7ffbb40665c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7ffbc0000c00 tx=0x7ffbc0009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:56:35.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.081+0000 7ffbbaffd700 1 -- 172.21.15.27:0/4171865560 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7ffbbc052950 con 0x7ffbcc060290 2021-11-03T17:56:35.281 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.284+0000 7ffbcb59e700 1 -- 172.21.15.27:0/4171865560 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffbcc100f80 con 0x7ffbb4064100 2021-11-03T17:56:35.282 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.286+0000 7ffbbaffd700 1 -- 172.21.15.27:0/4171865560 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7ffbcc100f80 con 0x7ffbb4064100 2021-11-03T17:56:35.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.288+0000 7ffbcb59e700 1 -- 172.21.15.27:0/4171865560 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffbb4064100 msgr2=0x7ffbb40665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:56:35.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.288+0000 7ffbcb59e700 1 --2- 172.21.15.27:0/4171865560 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffbb4064100 0x7ffbb40665c0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7ffbc0000c00 tx=0x7ffbc0009b30).stop 2021-11-03T17:56:35.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.288+0000 7ffbcb59e700 1 -- 172.21.15.27:0/4171865560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbcc060290 msgr2=0x7ffbcc108ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:56:35.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.288+0000 7ffbcb59e700 1 --2- 172.21.15.27:0/4171865560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbcc060290 0x7ffbcc108ef0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7ffbbc00c530 tx=0x7ffbbc007e20).stop 2021-11-03T17:56:35.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.288+0000 7ffbcb59e700 1 -- 172.21.15.27:0/4171865560 shutdown_connections 2021-11-03T17:56:35.286 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.288+0000 7ffbcb59e700 1 --2- 172.21.15.27:0/4171865560 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffbb4064100 0x7ffbb40665c0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:35.286 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.288+0000 7ffbcb59e700 1 --2- 172.21.15.27:0/4171865560 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffbcc060290 0x7ffbcc108ef0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:35.286 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.288+0000 7ffbcb59e700 1 --2- 172.21.15.27:0/4171865560 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffbcc0fba90 0x7ffbcc1093e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:35.286 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.288+0000 7ffbcb59e700 1 -- 172.21.15.27:0/4171865560 >> 172.21.15.27:0/4171865560 conn(0x7ffbcc0f0ba0 msgr2=0x7ffbcc0f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:56:35.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.288+0000 7ffbcb59e700 1 -- 172.21.15.27:0/4171865560 shutdown_connections 2021-11-03T17:56:35.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.288+0000 7ffbcb59e700 1 -- 172.21.15.27:0/4171865560 wait complete. 2021-11-03T17:56:35.295 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:56:35.390 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.391+0000 7f07ef59e700 1 -- 172.21.15.27:0/3805379346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07f00f5430 msgr2=0x7f07f00f5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:56:35.390 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.391+0000 7f07ef59e700 1 --2- 172.21.15.27:0/3805379346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07f00f5430 0x7f07f00f5850 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7f07e0004660 tx=0x7f07e0009b30).stop 2021-11-03T17:56:35.391 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.391+0000 7f07ef59e700 1 -- 172.21.15.27:0/3805379346 shutdown_connections 2021-11-03T17:56:35.391 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.391+0000 7f07ef59e700 1 --2- 172.21.15.27:0/3805379346 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07f00f66d0 0x7f07f00f6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:35.391 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.391+0000 7f07ef59e700 1 --2- 172.21.15.27:0/3805379346 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07f00f5430 0x7f07f00f5850 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:35.391 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.391+0000 7f07ef59e700 1 -- 172.21.15.27:0/3805379346 >> 172.21.15.27:0/3805379346 conn(0x7f07f00f0b90 msgr2=0x7f07f00f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:56:35.392 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.391+0000 7f07ef59e700 1 -- 172.21.15.27:0/3805379346 shutdown_connections 2021-11-03T17:56:35.392 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.392+0000 7f07ef59e700 1 -- 172.21.15.27:0/3805379346 wait complete. 2021-11-03T17:56:35.392 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.392+0000 7f07ef59e700 1 Processor -- start 2021-11-03T17:56:35.392 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.392+0000 7f07ef59e700 1 -- start start 2021-11-03T17:56:35.392 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.392+0000 7f07ef59e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07f00f5430 0x7f07f0109a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:56:35.393 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.392+0000 7f07ef59e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07f00f66d0 0x7f07f010bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:56:35.393 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.392+0000 7f07ef59e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f00603c0 con 0x7f07f00f66d0 2021-11-03T17:56:35.393 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.392+0000 7f07ef59e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07f0060500 con 0x7f07f00f5430 2021-11-03T17:56:35.393 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.392+0000 7f07edd9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07f00f66d0 0x7f07f010bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:56:35.394 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.392+0000 7f07edd9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07f00f66d0 0x7f07f010bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60272/0 (socket says 172.21.15.27:60272) 2021-11-03T17:56:35.394 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.392+0000 7f07edd9b700 1 -- 172.21.15.27:0/1163590105 learned_addr learned my addr 172.21.15.27:0/1163590105 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:56:35.394 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.392+0000 7f07ee59c700 1 --2- 172.21.15.27:0/1163590105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07f00f5430 0x7f07f0109a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:56:35.394 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.393+0000 7f07ee59c700 1 -- 172.21.15.27:0/1163590105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07f00f66d0 msgr2=0x7f07f010bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:56:35.395 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.393+0000 7f07ee59c700 1 --2- 172.21.15.27:0/1163590105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07f00f66d0 0x7f07f010bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:35.395 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.393+0000 7f07ee59c700 1 -- 172.21.15.27:0/1163590105 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e0005040 con 0x7f07f00f5430 2021-11-03T17:56:35.395 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.393+0000 7f07edd9b700 1 --2- 172.21.15.27:0/1163590105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07f00f66d0 0x7f07f010bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-03T17:56:35.395 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.393+0000 7f07ee59c700 1 --2- 172.21.15.27:0/1163590105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07f00f5430 0x7f07f0109a40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f07f00f6530 tx=0x7f07e000b040).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:56:35.396 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.393+0000 7f07e6ffd700 1 -- 172.21.15.27:0/1163590105 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07e0019070 con 0x7f07f00f5430 2021-11-03T17:56:35.396 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.393+0000 7f07e6ffd700 1 -- 172.21.15.27:0/1163590105 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f07e0007e70 con 0x7f07f00f5430 2021-11-03T17:56:35.396 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.393+0000 7f07ef59e700 1 -- 172.21.15.27:0/1163590105 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07f010c490 con 0x7f07f00f5430 2021-11-03T17:56:35.396 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.393+0000 7f07e6ffd700 1 -- 172.21.15.27:0/1163590105 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07e0014c20 con 0x7f07f00f5430 2021-11-03T17:56:35.397 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.393+0000 7f07ef59e700 1 -- 172.21.15.27:0/1163590105 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07f010c9c0 con 0x7f07f00f5430 2021-11-03T17:56:35.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.394+0000 7f07ef59e700 1 -- 172.21.15.27:0/1163590105 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07f0059100 con 0x7f07f00f5430 2021-11-03T17:56:35.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.394+0000 7f07e6ffd700 1 -- 172.21.15.27:0/1163590105 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f07e0025070 con 0x7f07f00f5430 2021-11-03T17:56:35.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.395+0000 7f07e6ffd700 1 --2- 172.21.15.27:0/1163590105 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f07dc064100 0x7f07dc0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:56:35.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.395+0000 7f07e6ffd700 1 -- 172.21.15.27:0/1163590105 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f07e0081710 con 0x7f07f00f5430 2021-11-03T17:56:35.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.395+0000 7f07edd9b700 1 --2- 172.21.15.27:0/1163590105 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f07dc064100 0x7f07dc0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:56:35.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.395+0000 7f07edd9b700 1 --2- 172.21.15.27:0/1163590105 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f07dc064100 0x7f07dc0665c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f07d8007a30 tx=0x7f07d800b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:56:35.400 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.399+0000 7f07e6ffd700 1 -- 172.21.15.27:0/1163590105 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f07e0053a10 con 0x7f07f00f5430 2021-11-03T17:56:35.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2437.418345 secs 2021-11-03T17:56:35.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:56:35.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2437.418345 secs 2021-11-03T17:56:35.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:56:35.576 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.580+0000 7f07ef59e700 1 -- 172.21.15.27:0/1163590105 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f07f00fdd70 con 0x7f07dc064100 2021-11-03T17:56:35.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.594+0000 7f07e6ffd700 1 -- 172.21.15.27:0/1163590105 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f07f00fdd70 con 0x7f07dc064100 2021-11-03T17:56:35.591 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:56:35.592 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (48m) 7m ago 49m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:56:35.592 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (39m) 7m ago 49m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:56:35.592 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (39m) 6m ago 48m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:56:35.592 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (47m) 7m ago 49m 41.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:56:35.593 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (41m) 7m ago 44m 23.8M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:56:35.593 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (44m) 7m ago 44m 3844M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:56:35.593 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (40m) 6m ago 44m 23.2M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:56:35.593 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (41m) 6m ago 44m 24.9M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:56:35.594 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (41m) 7m ago 50m 535M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:56:35.594 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (41m) 6m ago 48m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:56:35.594 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (40m) 7m ago 50m 198M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:56:35.594 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (40m) 6m ago 48m 95.6M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:56:35.595 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (49m) 7m ago 49m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:56:35.595 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (48m) 6m ago 48m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:56:35.595 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (39m) 7m ago 47m 331M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:56:35.595 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (38m) 7m ago 47m 361M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:56:35.595 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (38m) 7m ago 47m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:56:35.596 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (37m) 6m ago 46m 379M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:56:35.596 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (37m) 6m ago 46m 304M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:56:35.596 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (37m) 6m ago 46m 307M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:56:35.596 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (47m) 7m ago 48m 52.9M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:56:35.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.596+0000 7f07ef59e700 1 -- 172.21.15.27:0/1163590105 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f07dc064100 msgr2=0x7f07dc0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:56:35.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.596+0000 7f07ef59e700 1 --2- 172.21.15.27:0/1163590105 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f07dc064100 0x7f07dc0665c0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f07d8007a30 tx=0x7f07d800b040).stop 2021-11-03T17:56:35.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.596+0000 7f07ef59e700 1 -- 172.21.15.27:0/1163590105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07f00f5430 msgr2=0x7f07f0109a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:56:35.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.596+0000 7f07ef59e700 1 --2- 172.21.15.27:0/1163590105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07f00f5430 0x7f07f0109a40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f07f00f6530 tx=0x7f07e000b040).stop 2021-11-03T17:56:35.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.596+0000 7f07ef59e700 1 -- 172.21.15.27:0/1163590105 shutdown_connections 2021-11-03T17:56:35.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.596+0000 7f07ef59e700 1 --2- 172.21.15.27:0/1163590105 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f07dc064100 0x7f07dc0665c0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:35.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.596+0000 7f07ef59e700 1 --2- 172.21.15.27:0/1163590105 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07f00f5430 0x7f07f0109a40 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:35.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.596+0000 7f07ef59e700 1 --2- 172.21.15.27:0/1163590105 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07f00f66d0 0x7f07f010bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:35.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.596+0000 7f07ef59e700 1 -- 172.21.15.27:0/1163590105 >> 172.21.15.27:0/1163590105 conn(0x7f07f00f0b90 msgr2=0x7f07f00fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:56:35.600 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.596+0000 7f07ef59e700 1 -- 172.21.15.27:0/1163590105 shutdown_connections 2021-11-03T17:56:35.600 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.596+0000 7f07ef59e700 1 -- 172.21.15.27:0/1163590105 wait complete. 2021-11-03T17:56:35.694 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.695+0000 7f340447f700 1 -- 172.21.15.27:0/1977801138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33fc0f66c0 msgr2=0x7f33fc0f6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:56:35.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.695+0000 7f340447f700 1 --2- 172.21.15.27:0/1977801138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33fc0f66c0 0x7f33fc0f6ae0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7f33ec0099e0 tx=0x7f33ec004d10).stop 2021-11-03T17:56:35.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.695+0000 7f340447f700 1 -- 172.21.15.27:0/1977801138 shutdown_connections 2021-11-03T17:56:35.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.695+0000 7f340447f700 1 --2- 172.21.15.27:0/1977801138 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33fc0f7960 0x7f33fc0f7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:35.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.695+0000 7f340447f700 1 --2- 172.21.15.27:0/1977801138 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33fc0f66c0 0x7f33fc0f6ae0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:35.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.695+0000 7f340447f700 1 -- 172.21.15.27:0/1977801138 >> 172.21.15.27:0/1977801138 conn(0x7f33fc0f1e20 msgr2=0x7f33fc0f4240 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:56:35.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.695+0000 7f340447f700 1 -- 172.21.15.27:0/1977801138 shutdown_connections 2021-11-03T17:56:35.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.695+0000 7f340447f700 1 -- 172.21.15.27:0/1977801138 wait complete. 2021-11-03T17:56:35.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.696+0000 7f340447f700 1 Processor -- start 2021-11-03T17:56:35.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.696+0000 7f340447f700 1 -- start start 2021-11-03T17:56:35.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.696+0000 7f340447f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33fc0f66c0 0x7f33fc107080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:56:35.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.696+0000 7f340447f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33fc0f7960 0x7f33fc107570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:56:35.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.696+0000 7f340447f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc107b20 con 0x7f33fc0f66c0 2021-11-03T17:56:35.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.696+0000 7f340447f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33fc107c60 con 0x7f33fc0f7960 2021-11-03T17:56:35.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.696+0000 7f340221b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33fc0f66c0 0x7f33fc107080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:56:35.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.696+0000 7f340221b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33fc0f66c0 0x7f33fc107080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60280/0 (socket says 172.21.15.27:60280) 2021-11-03T17:56:35.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.696+0000 7f340221b700 1 -- 172.21.15.27:0/3446674174 learned_addr learned my addr 172.21.15.27:0/3446674174 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:56:35.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.696+0000 7f3401a1a700 1 --2- 172.21.15.27:0/3446674174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33fc0f7960 0x7f33fc107570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:56:35.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.696+0000 7f340221b700 1 -- 172.21.15.27:0/3446674174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33fc0f7960 msgr2=0x7f33fc107570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:56:35.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.696+0000 7f340221b700 1 --2- 172.21.15.27:0/3446674174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33fc0f7960 0x7f33fc107570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:35.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.696+0000 7f340221b700 1 -- 172.21.15.27:0/3446674174 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33ec009640 con 0x7f33fc0f66c0 2021-11-03T17:56:35.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.697+0000 7f340221b700 1 --2- 172.21.15.27:0/3446674174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33fc0f66c0 0x7f33fc107080 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7f33ec00c010 tx=0x7f33ec004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:56:35.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.697+0000 7f33faffd700 1 -- 172.21.15.27:0/3446674174 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33ec00e070 con 0x7f33fc0f66c0 2021-11-03T17:56:35.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.697+0000 7f340447f700 1 -- 172.21.15.27:0/3446674174 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33fc1085a0 con 0x7f33fc0f66c0 2021-11-03T17:56:35.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.697+0000 7f33faffd700 1 -- 172.21.15.27:0/3446674174 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f33ec007d10 con 0x7f33fc0f66c0 2021-11-03T17:56:35.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.697+0000 7f33faffd700 1 -- 172.21.15.27:0/3446674174 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33ec018070 con 0x7f33fc0f66c0 2021-11-03T17:56:35.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.697+0000 7f340447f700 1 -- 172.21.15.27:0/3446674174 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33fc108a80 con 0x7f33fc0f66c0 2021-11-03T17:56:35.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.698+0000 7f340447f700 1 -- 172.21.15.27:0/3446674174 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33fc045ad0 con 0x7f33fc0f66c0 2021-11-03T17:56:35.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.698+0000 7f33faffd700 1 -- 172.21.15.27:0/3446674174 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f33ec024060 con 0x7f33fc0f66c0 2021-11-03T17:56:35.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.698+0000 7f33faffd700 1 --2- 172.21.15.27:0/3446674174 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f33e40640b0 0x7f33e4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:56:35.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.699+0000 7f33faffd700 1 -- 172.21.15.27:0/3446674174 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f33ec082060 con 0x7f33fc0f66c0 2021-11-03T17:56:35.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.699+0000 7f3401a1a700 1 --2- 172.21.15.27:0/3446674174 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f33e40640b0 0x7f33e4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:56:35.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.699+0000 7f3401a1a700 1 --2- 172.21.15.27:0/3446674174 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f33e40640b0 0x7f33e4066570 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f33f0004c40 tx=0x7f33f000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:56:35.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.702+0000 7f33faffd700 1 -- 172.21.15.27:0/3446674174 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f33ec054420 con 0x7f33fc0f66c0 2021-11-03T17:56:35.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.935+0000 7f340447f700 1 -- 172.21.15.27:0/3446674174 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f33fc100e30 con 0x7f33fc0f66c0 2021-11-03T17:56:35.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.936+0000 7f33faffd700 1 -- 172.21.15.27:0/3446674174 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f33ec0222e0 con 0x7f33fc0f66c0 2021-11-03T17:56:35.934 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:56:35.934 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:56:35.934 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:56:35.934 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:56:35.935 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:56:35.935 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:56:35.935 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:56:35.935 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:56:35.936 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:56:35.936 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:56:35.936 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:56:35.936 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:56:35.937 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:56:35.937 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:56:35.937 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:56:35.937 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:56:35.937 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:56:35.938 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:56:35.938 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.938+0000 7f340447f700 1 -- 172.21.15.27:0/3446674174 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f33e40640b0 msgr2=0x7f33e4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:56:35.939 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.938+0000 7f340447f700 1 --2- 172.21.15.27:0/3446674174 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f33e40640b0 0x7f33e4066570 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f33f0004c40 tx=0x7f33f000b040).stop 2021-11-03T17:56:35.939 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.938+0000 7f340447f700 1 -- 172.21.15.27:0/3446674174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33fc0f66c0 msgr2=0x7f33fc107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:56:35.939 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.938+0000 7f340447f700 1 --2- 172.21.15.27:0/3446674174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33fc0f66c0 0x7f33fc107080 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7f33ec00c010 tx=0x7f33ec004d10).stop 2021-11-03T17:56:35.939 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.938+0000 7f340447f700 1 -- 172.21.15.27:0/3446674174 shutdown_connections 2021-11-03T17:56:35.940 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.938+0000 7f340447f700 1 --2- 172.21.15.27:0/3446674174 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f33e40640b0 0x7f33e4066570 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:35.940 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.938+0000 7f340447f700 1 --2- 172.21.15.27:0/3446674174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33fc0f66c0 0x7f33fc107080 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:35.940 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.938+0000 7f340447f700 1 --2- 172.21.15.27:0/3446674174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33fc0f7960 0x7f33fc107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:35.940 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.938+0000 7f340447f700 1 -- 172.21.15.27:0/3446674174 >> 172.21.15.27:0/3446674174 conn(0x7f33fc0f1e20 msgr2=0x7f33fc0fb3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:56:35.940 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.938+0000 7f340447f700 1 -- 172.21.15.27:0/3446674174 shutdown_connections 2021-11-03T17:56:35.941 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:35.938+0000 7f340447f700 1 -- 172.21.15.27:0/3446674174 wait complete. 2021-11-03T17:56:36.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.038+0000 7ff34b038700 1 -- 172.21.15.27:0/3359372023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3440f6ff0 msgr2=0x7ff3440f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:56:36.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.038+0000 7ff34b038700 1 --2- 172.21.15.27:0/3359372023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3440f6ff0 0x7ff3440f7a20 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 rx=0x7ff330004660 tx=0x7ff330009b30).stop 2021-11-03T17:56:36.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.038+0000 7ff34b038700 1 -- 172.21.15.27:0/3359372023 shutdown_connections 2021-11-03T17:56:36.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.038+0000 7ff34b038700 1 --2- 172.21.15.27:0/3359372023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3440f6ff0 0x7ff3440f7a20 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:36.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.038+0000 7ff34b038700 1 --2- 172.21.15.27:0/3359372023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3440f66e0 0x7ff3440f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:56:36.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.038+0000 7ff34b038700 1 -- 172.21.15.27:0/3359372023 >> 172.21.15.27:0/3359372023 conn(0x7ff3440f1f00 msgr2=0x7ff3440f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:56:36.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.038+0000 7ff34b038700 1 -- 172.21.15.27:0/3359372023 shutdown_connections 2021-11-03T17:56:36.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.038+0000 7ff34b038700 1 -- 172.21.15.27:0/3359372023 wait complete. 2021-11-03T17:56:36.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.039+0000 7ff34b038700 1 Processor -- start 2021-11-03T17:56:36.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.039+0000 7ff34b038700 1 -- start start 2021-11-03T17:56:36.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.039+0000 7ff34b038700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3440f66e0 0x7ff344106fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:56:36.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.039+0000 7ff34b038700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3440f6ff0 0x7ff344107490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:56:36.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.039+0000 7ff34b038700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff344107ad0 con 0x7ff3440f6ff0 2021-11-03T17:56:36.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.039+0000 7ff34b038700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff344108380 con 0x7ff3440f66e0 2021-11-03T17:56:36.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.039+0000 7ff343fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3440f6ff0 0x7ff344107490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:56:36.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.039+0000 7ff343fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3440f6ff0 0x7ff344107490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60292/0 (socket says 172.21.15.27:60292) 2021-11-03T17:56:36.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.039+0000 7ff343fff700 1 -- 172.21.15.27:0/1589639021 learned_addr learned my addr 172.21.15.27:0/1589639021 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:56:36.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.039+0000 7ff348dd4700 1 --2- 172.21.15.27:0/1589639021 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3440f66e0 0x7ff344106fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:56:36.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.039+0000 7ff343fff700 1 -- 172.21.15.27:0/1589639021 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3440f66e0 msgr2=0x7ff344106fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:56:36.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.039+0000 7ff343fff700 1 --2- 172.21.15.27:0/1589639021 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3440f66e0 0x7ff344106fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:36.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.039+0000 7ff343fff700 1 -- 172.21.15.27:0/1589639021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff330005040 con 0x7ff3440f6ff0 2021-11-03T17:56:36.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.040+0000 7ff343fff700 1 --2- 172.21.15.27:0/1589639021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3440f6ff0 0x7ff344107490 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7ff330009990 tx=0x7ff330009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:56:36.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.040+0000 7ff3417fa700 1 -- 172.21.15.27:0/1589639021 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff330018070 con 0x7ff3440f6ff0 2021-11-03T17:56:36.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.040+0000 7ff34b038700 1 -- 172.21.15.27:0/1589639021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff344108600 con 0x7ff3440f6ff0 2021-11-03T17:56:36.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.040+0000 7ff3417fa700 1 -- 172.21.15.27:0/1589639021 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7ff330007ca0 con 0x7ff3440f6ff0 2021-11-03T17:56:36.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.040+0000 7ff3417fa700 1 -- 172.21.15.27:0/1589639021 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff33001c620 con 0x7ff3440f6ff0 2021-11-03T17:56:36.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.040+0000 7ff34b038700 1 -- 172.21.15.27:0/1589639021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff344108b60 con 0x7ff3440f6ff0 2021-11-03T17:56:36.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.041+0000 7ff34b038700 1 -- 172.21.15.27:0/1589639021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3440f5b70 con 0x7ff3440f6ff0 2021-11-03T17:56:36.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.041+0000 7ff3417fa700 1 -- 172.21.15.27:0/1589639021 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7ff33001d960 con 0x7ff3440f6ff0 2021-11-03T17:56:36.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.042+0000 7ff3417fa700 1 --2- 172.21.15.27:0/1589639021 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff3340640b0 0x7ff334066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:56:36.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.042+0000 7ff3417fa700 1 -- 172.21.15.27:0/1589639021 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7ff330082640 con 0x7ff3440f6ff0 2021-11-03T17:56:36.046 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.042+0000 7ff348dd4700 1 --2- 172.21.15.27:0/1589639021 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff3340640b0 0x7ff334066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:56:36.046 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.042+0000 7ff348dd4700 1 --2- 172.21.15.27:0/1589639021 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff3340640b0 0x7ff334066570 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7ff33800b790 tx=0x7ff338008040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:56:36.046 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.046+0000 7ff3417fa700 1 -- 172.21.15.27:0/1589639021 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7ff330054940 con 0x7ff3440f6ff0 2021-11-03T17:56:36.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.248+0000 7ff34b038700 1 -- 172.21.15.27:0/1589639021 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff3440fef50 con 0x7ff3440f6ff0 2021-11-03T17:56:36.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.248+0000 7ff3417fa700 1 -- 172.21.15.27:0/1589639021 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7ff330054530 con 0x7ff3440f6ff0 2021-11-03T17:56:36.246 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:56:36.246 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:56:36.247 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:56:36.247 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:56:36.247 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:56:36.248 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:56:36.248 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:56:36.248 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:56:36.248 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:56:36.249 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:56:36.249 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:56:36.249 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:56:36.249 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:56:36.250 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:56:36.250 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:56:36.250 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:56:36.250 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:56:36.251 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:56:36.251 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:56:36.251 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:56:36.251 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:56:36.252 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:56:36.252 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:56:36.252 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:56:36.252 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:56:36.253 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:56:36.253 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:56:36.253 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:56:36.253 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:56:36.253 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:56:36.254 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:56:36.254 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:56:36.254 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:56:36.254 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:56:36.255 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:56:36.255 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:56:36.255 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:56:36.255 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:56:36.256 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:56:36.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.250+0000 7ff34b038700 1 -- 172.21.15.27:0/1589639021 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff3340640b0 msgr2=0x7ff334066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:56:36.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.250+0000 7ff34b038700 1 --2- 172.21.15.27:0/1589639021 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff3340640b0 0x7ff334066570 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7ff33800b790 tx=0x7ff338008040).stop 2021-11-03T17:56:36.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.250+0000 7ff34b038700 1 -- 172.21.15.27:0/1589639021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3440f6ff0 msgr2=0x7ff344107490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:56:36.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.250+0000 7ff34b038700 1 --2- 172.21.15.27:0/1589639021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3440f6ff0 0x7ff344107490 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 rx=0x7ff330009990 tx=0x7ff330009b30).stop 2021-11-03T17:56:36.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.250+0000 7ff34b038700 1 -- 172.21.15.27:0/1589639021 shutdown_connections 2021-11-03T17:56:36.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.250+0000 7ff34b038700 1 --2- 172.21.15.27:0/1589639021 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff3340640b0 0x7ff334066570 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:36.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.250+0000 7ff34b038700 1 --2- 172.21.15.27:0/1589639021 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff3440f66e0 0x7ff344106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:36.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.250+0000 7ff34b038700 1 --2- 172.21.15.27:0/1589639021 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff3440f6ff0 0x7ff344107490 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:56:36.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.250+0000 7ff34b038700 1 -- 172.21.15.27:0/1589639021 >> 172.21.15.27:0/1589639021 conn(0x7ff3440f1f00 msgr2=0x7ff3440fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:56:36.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.250+0000 7ff34b038700 1 -- 172.21.15.27:0/1589639021 shutdown_connections 2021-11-03T17:56:36.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:56:36.250+0000 7ff34b038700 1 -- 172.21.15.27:0/1589639021 wait complete. 2021-11-03T17:56:36.260 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:56:36.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:36 smithi115 ceph-mon[65453]: pgmap v1258: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:36.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:36 smithi115 ceph-mon[65453]: poll took 60175.375 ms: 1 events 2021-11-03T17:56:36.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:36 smithi115 ceph-mon[65453]: from='client.35400 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:56:36.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:56:36.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:56:36.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:36.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:36.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:36.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:36.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:36 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3446674174' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:36.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:36 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1589639021' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:56:36.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:36 smithi027 ceph-mon[97244]: pgmap v1258: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:36.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:36 smithi027 ceph-mon[97244]: poll took 60175.375 ms: 1 events 2021-11-03T17:56:36.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:36 smithi027 ceph-mon[97244]: from='client.35400 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:56:36.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:56:36.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:56:36.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:36.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:36.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:36.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:36.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:36 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3446674174' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:36.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:36 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1589639021' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:56:37.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:37 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:56:37.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:37 smithi115 ceph-mon[65453]: from='client.44919 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:56:37.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:37 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:56:37.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:37 smithi027 ceph-mon[97244]: from='client.44919 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:56:38.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:38 smithi115 ceph-mon[65453]: pgmap v1259: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:38.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:38 smithi027 ceph-mon[97244]: pgmap v1259: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:40.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:40 smithi115 ceph-mon[65453]: pgmap v1260: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:40.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2442.418468 secs 2021-11-03T17:56:40.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:40 smithi027 ceph-mon[97244]: pgmap v1260: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:40.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2442.418468 secs 2021-11-03T17:56:42.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:42 smithi115 ceph-mon[65453]: pgmap v1261: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:42.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:42 smithi027 ceph-mon[97244]: pgmap v1261: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:44.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:44 smithi115 ceph-mon[65453]: pgmap v1262: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:44.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:44 smithi027 ceph-mon[97244]: pgmap v1262: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:45.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2447.418607 secs 2021-11-03T17:56:45.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2447.418607 secs 2021-11-03T17:56:46.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:46 smithi115 ceph-mon[65453]: pgmap v1263: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:46.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:56:46.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:56:46.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:56:46.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:46.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:46.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:46.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:46.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:46 smithi027 ceph-mon[97244]: pgmap v1263: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:46.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:56:46.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:56:46.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:56:46.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:46.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:46.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:46.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:47.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:47 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:56:47.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:47 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:56:48.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:48 smithi115 ceph-mon[65453]: pgmap v1264: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:48.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:48 smithi027 ceph-mon[97244]: pgmap v1264: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:50.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:50 smithi115 ceph-mon[65453]: pgmap v1265: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:50.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2452.418739 secs 2021-11-03T17:56:50.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:50 smithi027 ceph-mon[97244]: pgmap v1265: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:50.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2452.418739 secs 2021-11-03T17:56:52.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:52 smithi115 ceph-mon[65453]: pgmap v1266: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:52.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:52 smithi027 ceph-mon[97244]: pgmap v1266: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:54.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:54 smithi115 ceph-mon[65453]: pgmap v1267: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:54.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:54 smithi027 ceph-mon[97244]: pgmap v1267: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:55.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2457.418856 secs 2021-11-03T17:56:55.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2457.418856 secs 2021-11-03T17:56:56.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:56 smithi115 ceph-mon[65453]: pgmap v1268: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:56.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:56:56.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:56:56.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:56:56.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:56.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:56.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:56.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:56.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:56:56.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:56:56.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:56 smithi027 ceph-mon[97244]: pgmap v1268: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:56.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:56:56.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:56:56.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:56:56.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:56.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:56.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:56.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:56:56.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:56:56.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:56:57.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:57 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:56:57.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:57 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:56:58.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:56:58 smithi115 ceph-mon[65453]: pgmap v1269: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:56:58.769 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:56:58 smithi027 ceph-mon[97244]: pgmap v1269: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:00.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:00 smithi115 ceph-mon[65453]: pgmap v1270: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:00.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2462.418921 secs 2021-11-03T17:57:00.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:00 smithi027 ceph-mon[97244]: pgmap v1270: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:00.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2462.418921 secs 2021-11-03T17:57:02.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:02 smithi115 ceph-mon[65453]: pgmap v1271: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:02.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:02 smithi027 ceph-mon[97244]: pgmap v1271: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:04.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:04 smithi115 ceph-mon[65453]: pgmap v1272: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:04.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:04 smithi027 ceph-mon[97244]: pgmap v1272: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:05.678 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2467.419039 secs 2021-11-03T17:57:05.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2467.419039 secs 2021-11-03T17:57:06.356 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.357+0000 7f0b2a520700 1 -- 172.21.15.27:0/3712046476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b240f7d00 msgr2=0x7f0b240f8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:57:06.356 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.357+0000 7f0b2a520700 1 --2- 172.21.15.27:0/3712046476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b240f7d00 0x7f0b240f8120 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7f0b18004660 tx=0x7f0b18009b30).stop 2021-11-03T17:57:06.356 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.357+0000 7f0b2a520700 1 -- 172.21.15.27:0/3712046476 shutdown_connections 2021-11-03T17:57:06.357 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.357+0000 7f0b2a520700 1 --2- 172.21.15.27:0/3712046476 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b240f8610 0x7f0b240f4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:06.357 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.357+0000 7f0b2a520700 1 --2- 172.21.15.27:0/3712046476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b240f7d00 0x7f0b240f8120 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:06.357 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.357+0000 7f0b2a520700 1 -- 172.21.15.27:0/3712046476 >> 172.21.15.27:0/3712046476 conn(0x7f0b240f0540 msgr2=0x7f0b240f2960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:57:06.357 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.357+0000 7f0b2a520700 1 -- 172.21.15.27:0/3712046476 shutdown_connections 2021-11-03T17:57:06.358 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.357+0000 7f0b2a520700 1 -- 172.21.15.27:0/3712046476 wait complete. 2021-11-03T17:57:06.358 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.357+0000 7f0b2a520700 1 Processor -- start 2021-11-03T17:57:06.358 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.357+0000 7f0b2a520700 1 -- start start 2021-11-03T17:57:06.358 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.358+0000 7f0b2a520700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b240f7d00 0x7f0b240f75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:57:06.359 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.358+0000 7f0b2a520700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b240f8610 0x7f0b240f5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:57:06.359 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.358+0000 7f0b2a520700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b240f5fb0 con 0x7f0b240f7d00 2021-11-03T17:57:06.359 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.358+0000 7f0b2a520700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b240f60f0 con 0x7f0b240f8610 2021-11-03T17:57:06.359 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.358+0000 7f0b23fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b240f7d00 0x7f0b240f75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:57:06.360 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.358+0000 7f0b23fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b240f7d00 0x7f0b240f75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60308/0 (socket says 172.21.15.27:60308) 2021-11-03T17:57:06.360 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.358+0000 7f0b23fff700 1 -- 172.21.15.27:0/2650213245 learned_addr learned my addr 172.21.15.27:0/2650213245 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:57:06.360 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.358+0000 7f0b237fe700 1 --2- 172.21.15.27:0/2650213245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b240f8610 0x7f0b240f5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:57:06.360 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.358+0000 7f0b23fff700 1 -- 172.21.15.27:0/2650213245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b240f8610 msgr2=0x7f0b240f5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:57:06.361 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.358+0000 7f0b23fff700 1 --2- 172.21.15.27:0/2650213245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b240f8610 0x7f0b240f5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:06.361 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.358+0000 7f0b23fff700 1 -- 172.21.15.27:0/2650213245 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b18005040 con 0x7f0b240f7d00 2021-11-03T17:57:06.361 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.358+0000 7f0b23fff700 1 --2- 172.21.15.27:0/2650213245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b240f7d00 0x7f0b240f75d0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f0b18009990 tx=0x7f0b18009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:57:06.361 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.358+0000 7f0b20ff9700 1 -- 172.21.15.27:0/2650213245 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b18018070 con 0x7f0b240f7d00 2021-11-03T17:57:06.361 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.358+0000 7f0b20ff9700 1 -- 172.21.15.27:0/2650213245 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f0b18007ba0 con 0x7f0b240f7d00 2021-11-03T17:57:06.362 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.358+0000 7f0b2a520700 1 -- 172.21.15.27:0/2650213245 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b240f6370 con 0x7f0b240f7d00 2021-11-03T17:57:06.362 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.358+0000 7f0b20ff9700 1 -- 172.21.15.27:0/2650213245 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b1801c3c0 con 0x7f0b240f7d00 2021-11-03T17:57:06.362 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.359+0000 7f0b2a520700 1 -- 172.21.15.27:0/2650213245 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b240f68d0 con 0x7f0b240f7d00 2021-11-03T17:57:06.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.360+0000 7f0b2a520700 1 -- 172.21.15.27:0/2650213245 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b24045ad0 con 0x7f0b240f7d00 2021-11-03T17:57:06.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.360+0000 7f0b20ff9700 1 -- 172.21.15.27:0/2650213245 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f0b1801d420 con 0x7f0b240f7d00 2021-11-03T17:57:06.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.360+0000 7f0b20ff9700 1 --2- 172.21.15.27:0/2650213245 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0b0c0640b0 0x7f0b0c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:57:06.365 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.360+0000 7f0b20ff9700 1 -- 172.21.15.27:0/2650213245 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f0b180823c0 con 0x7f0b240f7d00 2021-11-03T17:57:06.365 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.360+0000 7f0b237fe700 1 --2- 172.21.15.27:0/2650213245 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0b0c0640b0 0x7f0b0c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:57:06.365 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.361+0000 7f0b237fe700 1 --2- 172.21.15.27:0/2650213245 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0b0c0640b0 0x7f0b0c066570 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f0b14004e20 tx=0x7f0b1400b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:57:06.365 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.364+0000 7f0b20ff9700 1 -- 172.21.15.27:0/2650213245 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f0b18054720 con 0x7f0b240f7d00 2021-11-03T17:57:06.540 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.543+0000 7f0b2a520700 1 -- 172.21.15.27:0/2650213245 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b240feff0 con 0x7f0b0c0640b0 2021-11-03T17:57:06.541 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.544+0000 7f0b20ff9700 1 -- 172.21.15.27:0/2650213245 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f0b240feff0 con 0x7f0b0c0640b0 2021-11-03T17:57:06.544 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.546+0000 7f0b2a520700 1 -- 172.21.15.27:0/2650213245 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0b0c0640b0 msgr2=0x7f0b0c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:57:06.544 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.546+0000 7f0b2a520700 1 --2- 172.21.15.27:0/2650213245 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0b0c0640b0 0x7f0b0c066570 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7f0b14004e20 tx=0x7f0b1400b040).stop 2021-11-03T17:57:06.545 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.546+0000 7f0b2a520700 1 -- 172.21.15.27:0/2650213245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b240f7d00 msgr2=0x7f0b240f75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:57:06.545 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.546+0000 7f0b2a520700 1 --2- 172.21.15.27:0/2650213245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b240f7d00 0x7f0b240f75d0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7f0b18009990 tx=0x7f0b18009b30).stop 2021-11-03T17:57:06.546 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.546+0000 7f0b2a520700 1 -- 172.21.15.27:0/2650213245 shutdown_connections 2021-11-03T17:57:06.546 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.546+0000 7f0b2a520700 1 --2- 172.21.15.27:0/2650213245 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0b0c0640b0 0x7f0b0c066570 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:06.546 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.546+0000 7f0b2a520700 1 --2- 172.21.15.27:0/2650213245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0b240f7d00 0x7f0b240f75d0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:06.547 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.546+0000 7f0b2a520700 1 --2- 172.21.15.27:0/2650213245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0b240f8610 0x7f0b240f5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:06.548 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.546+0000 7f0b2a520700 1 -- 172.21.15.27:0/2650213245 >> 172.21.15.27:0/2650213245 conn(0x7f0b240f0540 msgr2=0x7f0b240fb590 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:57:06.548 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.546+0000 7f0b2a520700 1 -- 172.21.15.27:0/2650213245 shutdown_connections 2021-11-03T17:57:06.548 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.546+0000 7f0b2a520700 1 -- 172.21.15.27:0/2650213245 wait complete. 2021-11-03T17:57:06.553 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:57:06.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.647+0000 7f1eea22d700 1 -- 172.21.15.27:0/1474376081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ee40f6860 msgr2=0x7f1ee40f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:57:06.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.647+0000 7f1eea22d700 1 --2- 172.21.15.27:0/1474376081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ee40f6860 0x7f1ee40f6cc0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7f1ecc004660 tx=0x7f1ecc009b30).stop 2021-11-03T17:57:06.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.647+0000 7f1eea22d700 1 -- 172.21.15.27:0/1474376081 shutdown_connections 2021-11-03T17:57:06.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.647+0000 7f1eea22d700 1 --2- 172.21.15.27:0/1474376081 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ee40f6860 0x7f1ee40f6cc0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:06.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.647+0000 7f1eea22d700 1 --2- 172.21.15.27:0/1474376081 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ee40f55c0 0x7f1ee40f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:06.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.647+0000 7f1eea22d700 1 -- 172.21.15.27:0/1474376081 >> 172.21.15.27:0/1474376081 conn(0x7f1ee40f0d40 msgr2=0x7f1ee40f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:57:06.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.647+0000 7f1eea22d700 1 -- 172.21.15.27:0/1474376081 shutdown_connections 2021-11-03T17:57:06.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.647+0000 7f1eea22d700 1 -- 172.21.15.27:0/1474376081 wait complete. 2021-11-03T17:57:06.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.647+0000 7f1eea22d700 1 Processor -- start 2021-11-03T17:57:06.649 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.648+0000 7f1eea22d700 1 -- start start 2021-11-03T17:57:06.649 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.648+0000 7f1eea22d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ee40f55c0 0x7f1ee4109c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:57:06.649 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.648+0000 7f1eea22d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ee40f6860 0x7f1ee410c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:57:06.649 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.648+0000 7f1eea22d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee40603c0 con 0x7f1ee40f55c0 2021-11-03T17:57:06.650 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.648+0000 7f1eea22d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee4060500 con 0x7f1ee40f6860 2021-11-03T17:57:06.650 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.648+0000 7f1ee37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ee40f55c0 0x7f1ee4109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:57:06.650 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.648+0000 7f1ee37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ee40f55c0 0x7f1ee4109c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60318/0 (socket says 172.21.15.27:60318) 2021-11-03T17:57:06.650 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.648+0000 7f1ee37fe700 1 -- 172.21.15.27:0/353480510 learned_addr learned my addr 172.21.15.27:0/353480510 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:57:06.651 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.648+0000 7f1ee2ffd700 1 --2- 172.21.15.27:0/353480510 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ee40f6860 0x7f1ee410c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:57:06.651 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.648+0000 7f1ee37fe700 1 -- 172.21.15.27:0/353480510 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ee40f6860 msgr2=0x7f1ee410c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:57:06.651 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.648+0000 7f1ee37fe700 1 --2- 172.21.15.27:0/353480510 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ee40f6860 0x7f1ee410c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:06.651 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.648+0000 7f1ee37fe700 1 -- 172.21.15.27:0/353480510 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ecc005040 con 0x7f1ee40f55c0 2021-11-03T17:57:06.652 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.648+0000 7f1ee37fe700 1 --2- 172.21.15.27:0/353480510 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ee40f55c0 0x7f1ee4109c70 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7f1edc00d7f0 tx=0x7f1edc004e80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:57:06.652 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.649+0000 7f1edbfff700 1 -- 172.21.15.27:0/353480510 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1edc00f490 con 0x7f1ee40f55c0 2021-11-03T17:57:06.652 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.649+0000 7f1edbfff700 1 -- 172.21.15.27:0/353480510 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f1edc00f040 con 0x7f1ee40f55c0 2021-11-03T17:57:06.652 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.649+0000 7f1eea22d700 1 -- 172.21.15.27:0/353480510 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ee410c720 con 0x7f1ee40f55c0 2021-11-03T17:57:06.652 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.649+0000 7f1edbfff700 1 -- 172.21.15.27:0/353480510 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1edc0139b0 con 0x7f1ee40f55c0 2021-11-03T17:57:06.653 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.649+0000 7f1eea22d700 1 -- 172.21.15.27:0/353480510 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ee410ccb0 con 0x7f1ee40f55c0 2021-11-03T17:57:06.653 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.650+0000 7f1eea22d700 1 -- 172.21.15.27:0/353480510 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ee4045ad0 con 0x7f1ee40f55c0 2021-11-03T17:57:06.653 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.651+0000 7f1edbfff700 1 -- 172.21.15.27:0/353480510 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f1edc01a030 con 0x7f1ee40f55c0 2021-11-03T17:57:06.653 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.651+0000 7f1edbfff700 1 --2- 172.21.15.27:0/353480510 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1ed00640b0 0x7f1ed0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:57:06.654 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.652+0000 7f1edbfff700 1 -- 172.21.15.27:0/353480510 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f1edc080fd0 con 0x7f1ee40f55c0 2021-11-03T17:57:06.654 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.652+0000 7f1ee2ffd700 1 --2- 172.21.15.27:0/353480510 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1ed00640b0 0x7f1ed0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:57:06.654 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.652+0000 7f1ee2ffd700 1 --2- 172.21.15.27:0/353480510 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1ed00640b0 0x7f1ed0066570 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f1ecc004630 tx=0x7f1ecc009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:57:06.655 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.655+0000 7f1edbfff700 1 -- 172.21.15.27:0/353480510 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f1edc053120 con 0x7f1ee40f55c0 2021-11-03T17:57:06.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:06 smithi115 ceph-mon[65453]: pgmap v1273: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:06.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:57:06.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:06 smithi115 ceph-mon[65453]: poll took 30102.352 ms: 1 events 2021-11-03T17:57:06.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:57:06.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:57:06.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:57:06.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:06.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:06.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:06.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:06.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:06 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:57:06.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:06 smithi027 ceph-mon[97244]: pgmap v1273: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:06.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:57:06.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:06 smithi027 ceph-mon[97244]: poll took 30102.352 ms: 1 events 2021-11-03T17:57:06.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:57:06.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:57:06.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:57:06.776 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:06.776 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:06.776 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:06.776 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:06.776 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:06 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:57:06.825 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.828+0000 7f1eea22d700 1 -- 172.21.15.27:0/353480510 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ee4057f10 con 0x7f1ed00640b0 2021-11-03T17:57:06.839 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.843+0000 7f1edbfff700 1 -- 172.21.15.27:0/353480510 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f1ee4057f10 con 0x7f1ed00640b0 2021-11-03T17:57:06.840 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:57:06.840 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (48m) 8m ago 49m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:57:06.840 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (40m) 8m ago 49m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:57:06.841 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (40m) 7m ago 49m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:57:06.841 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (48m) 8m ago 49m 41.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:57:06.841 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (42m) 8m ago 45m 23.8M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:57:06.841 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (45m) 8m ago 45m 3844M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:57:06.842 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (40m) 7m ago 45m 23.2M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:57:06.842 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (42m) 7m ago 45m 24.9M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:57:06.842 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (42m) 8m ago 51m 535M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:57:06.842 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (41m) 7m ago 49m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:57:06.842 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (41m) 8m ago 51m 198M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:57:06.843 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (40m) 7m ago 49m 95.6M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:57:06.843 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (49m) 8m ago 49m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:57:06.843 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (48m) 7m ago 48m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:57:06.843 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (39m) 8m ago 48m 331M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:57:06.844 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (39m) 8m ago 47m 361M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:57:06.844 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (38m) 8m ago 47m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:57:06.844 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (38m) 7m ago 47m 379M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:57:06.844 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (38m) 7m ago 46m 304M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:57:06.844 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (37m) 7m ago 46m 307M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:57:06.845 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (48m) 8m ago 49m 52.9M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:57:06.846 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.845+0000 7f1eea22d700 1 -- 172.21.15.27:0/353480510 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1ed00640b0 msgr2=0x7f1ed0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:57:06.846 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.845+0000 7f1eea22d700 1 --2- 172.21.15.27:0/353480510 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1ed00640b0 0x7f1ed0066570 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 rx=0x7f1ecc004630 tx=0x7f1ecc009b30).stop 2021-11-03T17:57:06.846 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.845+0000 7f1eea22d700 1 -- 172.21.15.27:0/353480510 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ee40f55c0 msgr2=0x7f1ee4109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:57:06.846 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.845+0000 7f1eea22d700 1 --2- 172.21.15.27:0/353480510 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ee40f55c0 0x7f1ee4109c70 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7f1edc00d7f0 tx=0x7f1edc004e80).stop 2021-11-03T17:57:06.847 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.845+0000 7f1eea22d700 1 -- 172.21.15.27:0/353480510 shutdown_connections 2021-11-03T17:57:06.847 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.845+0000 7f1eea22d700 1 --2- 172.21.15.27:0/353480510 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1ed00640b0 0x7f1ed0066570 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:06.847 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.845+0000 7f1eea22d700 1 --2- 172.21.15.27:0/353480510 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1ee40f55c0 0x7f1ee4109c70 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:06.847 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.845+0000 7f1eea22d700 1 --2- 172.21.15.27:0/353480510 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1ee40f6860 0x7f1ee410c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:06.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.845+0000 7f1eea22d700 1 -- 172.21.15.27:0/353480510 >> 172.21.15.27:0/353480510 conn(0x7f1ee40f0d40 msgr2=0x7f1ee40fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:57:06.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.845+0000 7f1eea22d700 1 -- 172.21.15.27:0/353480510 shutdown_connections 2021-11-03T17:57:06.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.845+0000 7f1eea22d700 1 -- 172.21.15.27:0/353480510 wait complete. 2021-11-03T17:57:06.941 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.944+0000 7f00d7072700 1 -- 172.21.15.27:0/1017933592 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00d00f86d0 msgr2=0x7f00d00f48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:57:06.942 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.944+0000 7f00d7072700 1 --2- 172.21.15.27:0/1017933592 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00d00f86d0 0x7f00d00f48e0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7f00c00099e0 tx=0x7f00c0004d10).stop 2021-11-03T17:57:06.943 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.944+0000 7f00d7072700 1 -- 172.21.15.27:0/1017933592 shutdown_connections 2021-11-03T17:57:06.943 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.944+0000 7f00d7072700 1 --2- 172.21.15.27:0/1017933592 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00d00f86d0 0x7f00d00f48e0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:06.944 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.944+0000 7f00d7072700 1 --2- 172.21.15.27:0/1017933592 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00d00f7dc0 0x7f00d00f81e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:06.944 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.944+0000 7f00d7072700 1 -- 172.21.15.27:0/1017933592 >> 172.21.15.27:0/1017933592 conn(0x7f00d00f0540 msgr2=0x7f00d00f2960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:57:06.944 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.944+0000 7f00d7072700 1 -- 172.21.15.27:0/1017933592 shutdown_connections 2021-11-03T17:57:06.944 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.944+0000 7f00d7072700 1 -- 172.21.15.27:0/1017933592 wait complete. 2021-11-03T17:57:06.945 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.945+0000 7f00d7072700 1 Processor -- start 2021-11-03T17:57:06.945 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.945+0000 7f00d7072700 1 -- start start 2021-11-03T17:57:06.947 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.945+0000 7f00d7072700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00d00f7dc0 0x7f00d00f7690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:57:06.948 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.945+0000 7f00d7072700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00d00f86d0 0x7f00d00f5b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:57:06.948 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.945+0000 7f00d7072700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d00f6070 con 0x7f00d00f86d0 2021-11-03T17:57:06.948 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.945+0000 7f00d7072700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00d00f61b0 con 0x7f00d00f7dc0 2021-11-03T17:57:06.948 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.945+0000 7f00cffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00d00f86d0 0x7f00d00f5b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:57:06.949 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.945+0000 7f00cffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00d00f86d0 0x7f00d00f5b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60330/0 (socket says 172.21.15.27:60330) 2021-11-03T17:57:06.949 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.945+0000 7f00cffff700 1 -- 172.21.15.27:0/430042788 learned_addr learned my addr 172.21.15.27:0/430042788 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:57:06.949 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.945+0000 7f00d4e0e700 1 --2- 172.21.15.27:0/430042788 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00d00f7dc0 0x7f00d00f7690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:57:06.949 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.945+0000 7f00cffff700 1 -- 172.21.15.27:0/430042788 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00d00f7dc0 msgr2=0x7f00d00f7690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:57:06.950 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.945+0000 7f00cffff700 1 --2- 172.21.15.27:0/430042788 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00d00f7dc0 0x7f00d00f7690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:06.950 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.945+0000 7f00cffff700 1 -- 172.21.15.27:0/430042788 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00c0009640 con 0x7f00d00f86d0 2021-11-03T17:57:06.950 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.946+0000 7f00cffff700 1 --2- 172.21.15.27:0/430042788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00d00f86d0 0x7f00d00f5b10 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f00c0004b70 tx=0x7f00c0004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:57:06.950 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.946+0000 7f00cd7fa700 1 -- 172.21.15.27:0/430042788 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00c000e030 con 0x7f00d00f86d0 2021-11-03T17:57:06.951 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.946+0000 7f00cd7fa700 1 -- 172.21.15.27:0/430042788 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f00c00049b0 con 0x7f00d00f86d0 2021-11-03T17:57:06.951 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.946+0000 7f00cd7fa700 1 -- 172.21.15.27:0/430042788 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00c0018070 con 0x7f00d00f86d0 2021-11-03T17:57:06.951 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.946+0000 7f00d7072700 1 -- 172.21.15.27:0/430042788 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00d00f6430 con 0x7f00d00f86d0 2021-11-03T17:57:06.951 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.946+0000 7f00d7072700 1 -- 172.21.15.27:0/430042788 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00d00f6910 con 0x7f00d00f86d0 2021-11-03T17:57:06.952 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.947+0000 7f00d7072700 1 -- 172.21.15.27:0/430042788 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00d0045ad0 con 0x7f00d00f86d0 2021-11-03T17:57:06.952 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.948+0000 7f00cd7fa700 1 -- 172.21.15.27:0/430042788 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f00c0023020 con 0x7f00d00f86d0 2021-11-03T17:57:06.952 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.948+0000 7f00cd7fa700 1 --2- 172.21.15.27:0/430042788 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f00bc064050 0x7f00bc066510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:57:06.952 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.948+0000 7f00cd7fa700 1 -- 172.21.15.27:0/430042788 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f00c0082790 con 0x7f00d00f86d0 2021-11-03T17:57:06.952 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.948+0000 7f00d4e0e700 1 --2- 172.21.15.27:0/430042788 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f00bc064050 0x7f00bc066510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:57:06.953 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.948+0000 7f00d4e0e700 1 --2- 172.21.15.27:0/430042788 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f00bc064050 0x7f00bc066510 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f00c80041d0 tx=0x7f00c8009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:57:06.953 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:06.952+0000 7f00cd7fa700 1 -- 172.21.15.27:0/430042788 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f00c0054a10 con 0x7f00d00f86d0 2021-11-03T17:57:07.178 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.181+0000 7f00d7072700 1 -- 172.21.15.27:0/430042788 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f00d0027a00 con 0x7f00d00f86d0 2021-11-03T17:57:07.179 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.182+0000 7f00cd7fa700 1 -- 172.21.15.27:0/430042788 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f00c0054600 con 0x7f00d00f86d0 2021-11-03T17:57:07.180 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:57:07.180 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:57:07.180 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:57:07.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:57:07.181 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:57:07.181 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:57:07.181 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:57:07.182 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:57:07.182 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:57:07.182 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:57:07.182 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:57:07.182 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:57:07.183 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:57:07.183 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:57:07.183 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:57:07.183 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:57:07.184 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:57:07.184 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:57:07.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.184+0000 7f00d7072700 1 -- 172.21.15.27:0/430042788 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f00bc064050 msgr2=0x7f00bc066510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:57:07.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.184+0000 7f00d7072700 1 --2- 172.21.15.27:0/430042788 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f00bc064050 0x7f00bc066510 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f00c80041d0 tx=0x7f00c8009040).stop 2021-11-03T17:57:07.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.184+0000 7f00d7072700 1 -- 172.21.15.27:0/430042788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00d00f86d0 msgr2=0x7f00d00f5b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:57:07.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.184+0000 7f00d7072700 1 --2- 172.21.15.27:0/430042788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00d00f86d0 0x7f00d00f5b10 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f00c0004b70 tx=0x7f00c0004d10).stop 2021-11-03T17:57:07.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.184+0000 7f00d7072700 1 -- 172.21.15.27:0/430042788 shutdown_connections 2021-11-03T17:57:07.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.184+0000 7f00d7072700 1 --2- 172.21.15.27:0/430042788 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f00bc064050 0x7f00bc066510 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:07.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.184+0000 7f00d7072700 1 --2- 172.21.15.27:0/430042788 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00d00f7dc0 0x7f00d00f7690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:07.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.184+0000 7f00d7072700 1 --2- 172.21.15.27:0/430042788 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00d00f86d0 0x7f00d00f5b10 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:07.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.184+0000 7f00d7072700 1 -- 172.21.15.27:0/430042788 >> 172.21.15.27:0/430042788 conn(0x7f00d00f0540 msgr2=0x7f00d00fb650 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:57:07.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.184+0000 7f00d7072700 1 -- 172.21.15.27:0/430042788 shutdown_connections 2021-11-03T17:57:07.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.184+0000 7f00d7072700 1 -- 172.21.15.27:0/430042788 wait complete. 2021-11-03T17:57:07.283 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.284+0000 7f949359e700 1 -- 172.21.15.27:0/2333826015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94940f6650 msgr2=0x7f94940f6ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:57:07.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.284+0000 7f949359e700 1 --2- 172.21.15.27:0/2333826015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94940f6650 0x7f94940f6ab0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7f9484004660 tx=0x7f9484009b30).stop 2021-11-03T17:57:07.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.284+0000 7f949359e700 1 -- 172.21.15.27:0/2333826015 shutdown_connections 2021-11-03T17:57:07.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.284+0000 7f949359e700 1 --2- 172.21.15.27:0/2333826015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94940f6650 0x7f94940f6ab0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:07.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.284+0000 7f949359e700 1 --2- 172.21.15.27:0/2333826015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94940f53b0 0x7f94940f57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:07.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.284+0000 7f949359e700 1 -- 172.21.15.27:0/2333826015 >> 172.21.15.27:0/2333826015 conn(0x7f94940f0b30 msgr2=0x7f94940f2f50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:57:07.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.284+0000 7f949359e700 1 -- 172.21.15.27:0/2333826015 shutdown_connections 2021-11-03T17:57:07.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.284+0000 7f949359e700 1 -- 172.21.15.27:0/2333826015 wait complete. 2021-11-03T17:57:07.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.285+0000 7f949359e700 1 Processor -- start 2021-11-03T17:57:07.286 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.285+0000 7f949359e700 1 -- start start 2021-11-03T17:57:07.286 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.285+0000 7f949359e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94940f53b0 0x7f94941099c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:57:07.286 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.285+0000 7f949359e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94940f6650 0x7f949410bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:57:07.286 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.285+0000 7f949359e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94940603e0 con 0x7f94940f53b0 2021-11-03T17:57:07.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.285+0000 7f949359e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9494060520 con 0x7f94940f6650 2021-11-03T17:57:07.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.285+0000 7f949259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94940f53b0 0x7f94941099c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:57:07.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.285+0000 7f949259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94940f53b0 0x7f94941099c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60336/0 (socket says 172.21.15.27:60336) 2021-11-03T17:57:07.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.285+0000 7f949259c700 1 -- 172.21.15.27:0/3601642797 learned_addr learned my addr 172.21.15.27:0/3601642797 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:57:07.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.285+0000 7f9491d9b700 1 --2- 172.21.15.27:0/3601642797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94940f6650 0x7f949410bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:57:07.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.285+0000 7f949259c700 1 -- 172.21.15.27:0/3601642797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94940f6650 msgr2=0x7f949410bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:57:07.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.285+0000 7f949259c700 1 --2- 172.21.15.27:0/3601642797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94940f6650 0x7f949410bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:07.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.286+0000 7f949259c700 1 -- 172.21.15.27:0/3601642797 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9484005040 con 0x7f94940f53b0 2021-11-03T17:57:07.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.286+0000 7f9491d9b700 1 --2- 172.21.15.27:0/3601642797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94940f6650 0x7f949410bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:57:07.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.286+0000 7f949259c700 1 --2- 172.21.15.27:0/3601642797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94940f53b0 0x7f94941099c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f947c002700 tx=0x7f947c009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:57:07.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.286+0000 7f948affd700 1 -- 172.21.15.27:0/3601642797 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f947c009320 con 0x7f94940f53b0 2021-11-03T17:57:07.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.286+0000 7f949359e700 1 -- 172.21.15.27:0/3601642797 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f949410c470 con 0x7f94940f53b0 2021-11-03T17:57:07.290 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.286+0000 7f948affd700 1 -- 172.21.15.27:0/3601642797 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f947c00f040 con 0x7f94940f53b0 2021-11-03T17:57:07.290 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.286+0000 7f948affd700 1 -- 172.21.15.27:0/3601642797 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f947c014900 con 0x7f94940f53b0 2021-11-03T17:57:07.290 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.286+0000 7f949359e700 1 -- 172.21.15.27:0/3601642797 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f949410ca30 con 0x7f94940f53b0 2021-11-03T17:57:07.291 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.287+0000 7f949359e700 1 -- 172.21.15.27:0/3601642797 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9494045ad0 con 0x7f94940f53b0 2021-11-03T17:57:07.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.288+0000 7f948affd700 1 -- 172.21.15.27:0/3601642797 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f947c009480 con 0x7f94940f53b0 2021-11-03T17:57:07.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.288+0000 7f948affd700 1 --2- 172.21.15.27:0/3601642797 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9480064100 0x7f94800665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:57:07.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.288+0000 7f948affd700 1 -- 172.21.15.27:0/3601642797 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f947c080ef0 con 0x7f94940f53b0 2021-11-03T17:57:07.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.288+0000 7f9491d9b700 1 --2- 172.21.15.27:0/3601642797 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9480064100 0x7f94800665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:57:07.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.289+0000 7f9491d9b700 1 --2- 172.21.15.27:0/3601642797 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9480064100 0x7f94800665c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f9484004630 tx=0x7f9484009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:57:07.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.291+0000 7f948affd700 1 -- 172.21.15.27:0/3601642797 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f947c0531e0 con 0x7f94940f53b0 2021-11-03T17:57:07.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.497+0000 7f949359e700 1 -- 172.21.15.27:0/3601642797 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f949410c600 con 0x7f94940f53b0 2021-11-03T17:57:07.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.498+0000 7f948affd700 1 -- 172.21.15.27:0/3601642797 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f949410c600 con 0x7f94940f53b0 2021-11-03T17:57:07.495 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:57:07.496 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:57:07.496 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:57:07.496 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:57:07.496 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:57:07.497 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:57:07.497 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:57:07.497 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:57:07.497 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:57:07.497 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:57:07.498 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:57:07.498 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:57:07.498 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:57:07.498 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:57:07.499 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:57:07.499 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:57:07.499 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:57:07.499 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:57:07.500 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:57:07.500 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:57:07.500 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:57:07.500 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:57:07.500 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:57:07.501 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:57:07.501 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:57:07.501 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:57:07.501 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:57:07.502 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:57:07.502 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:57:07.502 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:57:07.502 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:57:07.502 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:57:07.503 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:57:07.503 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:57:07.503 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:57:07.503 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:57:07.504 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:57:07.504 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:57:07.504 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:57:07.505 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.499+0000 7f949359e700 1 -- 172.21.15.27:0/3601642797 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9480064100 msgr2=0x7f94800665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:57:07.506 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.499+0000 7f949359e700 1 --2- 172.21.15.27:0/3601642797 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9480064100 0x7f94800665c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f9484004630 tx=0x7f9484009b30).stop 2021-11-03T17:57:07.506 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.499+0000 7f949359e700 1 -- 172.21.15.27:0/3601642797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94940f53b0 msgr2=0x7f94941099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:57:07.506 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.499+0000 7f949359e700 1 --2- 172.21.15.27:0/3601642797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94940f53b0 0x7f94941099c0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f947c002700 tx=0x7f947c009d80).stop 2021-11-03T17:57:07.506 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.499+0000 7f949359e700 1 -- 172.21.15.27:0/3601642797 shutdown_connections 2021-11-03T17:57:07.506 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.499+0000 7f949359e700 1 --2- 172.21.15.27:0/3601642797 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9480064100 0x7f94800665c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:07.507 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.499+0000 7f949359e700 1 --2- 172.21.15.27:0/3601642797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94940f53b0 0x7f94941099c0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:07.507 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.499+0000 7f949359e700 1 --2- 172.21.15.27:0/3601642797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94940f6650 0x7f949410bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:07.507 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.499+0000 7f949359e700 1 -- 172.21.15.27:0/3601642797 >> 172.21.15.27:0/3601642797 conn(0x7f94940f0b30 msgr2=0x7f94940fa0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:57:07.507 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.500+0000 7f949359e700 1 -- 172.21.15.27:0/3601642797 shutdown_connections 2021-11-03T17:57:07.508 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:07.500+0000 7f949359e700 1 -- 172.21.15.27:0/3601642797 wait complete. 2021-11-03T17:57:07.508 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:57:07.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:07 smithi115 ceph-mon[65453]: pgmap v1274: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:07.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:07 smithi115 ceph-mon[65453]: from='client.35416 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:57:07.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:07 smithi115 ceph-mon[65453]: from='client.35420 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:57:07.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:07 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/430042788' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:07.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:07 smithi027 ceph-mon[97244]: pgmap v1274: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:07.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:07 smithi027 ceph-mon[97244]: from='client.35416 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:57:07.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:07 smithi027 ceph-mon[97244]: from='client.35420 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:57:07.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:07 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/430042788' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:08.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:08 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3601642797' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:57:08.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:08 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3601642797' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:57:09.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:09 smithi115 ceph-mon[65453]: pgmap v1275: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:09.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:09 smithi027 ceph-mon[97244]: pgmap v1275: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:10.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2472.419150 secs 2021-11-03T17:57:10.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2472.419150 secs 2021-11-03T17:57:11.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:11 smithi115 ceph-mon[65453]: pgmap v1276: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:11.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:11 smithi027 ceph-mon[97244]: pgmap v1276: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:13.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:13 smithi027 ceph-mon[97244]: pgmap v1277: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:13.956 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:13 smithi115 ceph-mon[65453]: pgmap v1277: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:15.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2477.419271 secs 2021-11-03T17:57:15.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:15 smithi027 ceph-mon[97244]: pgmap v1278: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:15.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2477.419271 secs 2021-11-03T17:57:15.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:15 smithi115 ceph-mon[65453]: pgmap v1278: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:16.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:57:16.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:57:16.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:57:16.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:16.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:16.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:16.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:16.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:16 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:57:17.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:57:17.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:57:17.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:57:17.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:17.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:17.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:17.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:17.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:16 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:57:18.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:18 smithi027 ceph-mon[97244]: pgmap v1279: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:18.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:18 smithi115 ceph-mon[65453]: pgmap v1279: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:20.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:20 smithi027 ceph-mon[97244]: pgmap v1280: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:20.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2482.419406 secs 2021-11-03T17:57:20.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:20 smithi115 ceph-mon[65453]: pgmap v1280: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:20.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2482.419406 secs 2021-11-03T17:57:22.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:22 smithi027 ceph-mon[97244]: pgmap v1281: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:22.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:22 smithi115 ceph-mon[65453]: pgmap v1281: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:24.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:24 smithi115 ceph-mon[65453]: pgmap v1282: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:24.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:24 smithi027 ceph-mon[97244]: pgmap v1282: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:25.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2487.419531 secs 2021-11-03T17:57:25.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2487.419531 secs 2021-11-03T17:57:26.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:26 smithi027 ceph-mon[97244]: pgmap v1283: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:26.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:26 smithi115 ceph-mon[65453]: pgmap v1283: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:27.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:57:27.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:57:27.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:57:27.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:27.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:27.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:27.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:27.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:27 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:57:27.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:57:27.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:57:27.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:57:27.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:27.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:27.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:27.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:27.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:27 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:57:28.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:28 smithi115 ceph-mon[65453]: pgmap v1284: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:28.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:28 smithi027 ceph-mon[97244]: pgmap v1284: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:30.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:30 smithi115 ceph-mon[65453]: pgmap v1285: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:30.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2492.419654 secs 2021-11-03T17:57:30.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:30 smithi027 ceph-mon[97244]: pgmap v1285: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:30.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2492.419654 secs 2021-11-03T17:57:32.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:32 smithi115 ceph-mon[65453]: pgmap v1286: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:32.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:32 smithi027 ceph-mon[97244]: pgmap v1286: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:34.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:34 smithi115 ceph-mon[65453]: pgmap v1287: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:34.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:34 smithi027 ceph-mon[97244]: pgmap v1287: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:35.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2497.419787 secs 2021-11-03T17:57:35.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2497.419787 secs 2021-11-03T17:57:36.398 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:36 smithi115 ceph-mon[65453]: pgmap v1288: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:36.411 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:36 smithi027 ceph-mon[97244]: pgmap v1288: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:37.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:57:37.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:37 smithi115 ceph-mon[65453]: poll took 30096.523 ms: 1 events 2021-11-03T17:57:37.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:57:37.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:57:37.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:37.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:37.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:37.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:37.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:57:37.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:37 smithi027 ceph-mon[97244]: poll took 30096.523 ms: 1 events 2021-11-03T17:57:37.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:57:37.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:57:37.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:37.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:37.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:37.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:37.614 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.614+0000 7f76a0cfa700 1 -- 172.21.15.27:0/2754138785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f769c0f5ef0 msgr2=0x7f769c0f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:57:37.615 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.614+0000 7f76a0cfa700 1 --2- 172.21.15.27:0/2754138785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f769c0f5ef0 0x7f769c0f6920 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7f7690004660 tx=0x7f7690009b30).stop 2021-11-03T17:57:37.615 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.614+0000 7f76a0cfa700 1 -- 172.21.15.27:0/2754138785 shutdown_connections 2021-11-03T17:57:37.615 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.614+0000 7f76a0cfa700 1 --2- 172.21.15.27:0/2754138785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f769c0f5ef0 0x7f769c0f6920 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:37.615 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.614+0000 7f76a0cfa700 1 --2- 172.21.15.27:0/2754138785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f769c0f55e0 0x7f769c0f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:37.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.614+0000 7f76a0cfa700 1 -- 172.21.15.27:0/2754138785 >> 172.21.15.27:0/2754138785 conn(0x7f769c0f0de0 msgr2=0x7f769c0f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:57:37.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.615+0000 7f76a0cfa700 1 -- 172.21.15.27:0/2754138785 shutdown_connections 2021-11-03T17:57:37.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.615+0000 7f76a0cfa700 1 -- 172.21.15.27:0/2754138785 wait complete. 2021-11-03T17:57:37.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.615+0000 7f76a0cfa700 1 Processor -- start 2021-11-03T17:57:37.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.615+0000 7f76a0cfa700 1 -- start start 2021-11-03T17:57:37.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.615+0000 7f76a0cfa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f769c0f55e0 0x7f769c109c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:57:37.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.615+0000 7f76a0cfa700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f769c0f5ef0 0x7f769c10c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:57:37.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.615+0000 7f76a0cfa700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769c0604e0 con 0x7f769c0f55e0 2021-11-03T17:57:37.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.615+0000 7f76a0cfa700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f769c060620 con 0x7f769c0f5ef0 2021-11-03T17:57:37.618 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.615+0000 7f769a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f769c0f55e0 0x7f769c109c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:57:37.618 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.615+0000 7f769a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f769c0f55e0 0x7f769c109c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60354/0 (socket says 172.21.15.27:60354) 2021-11-03T17:57:37.619 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.615+0000 7f769a59c700 1 -- 172.21.15.27:0/3073667229 learned_addr learned my addr 172.21.15.27:0/3073667229 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:57:37.621 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.616+0000 7f7699d9b700 1 --2- 172.21.15.27:0/3073667229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f769c0f5ef0 0x7f769c10c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:57:37.621 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.616+0000 7f769a59c700 1 -- 172.21.15.27:0/3073667229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f769c0f5ef0 msgr2=0x7f769c10c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:57:37.621 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.616+0000 7f769a59c700 1 --2- 172.21.15.27:0/3073667229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f769c0f5ef0 0x7f769c10c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:37.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.616+0000 7f769a59c700 1 -- 172.21.15.27:0/3073667229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7690005040 con 0x7f769c0f55e0 2021-11-03T17:57:37.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.616+0000 7f769a59c700 1 --2- 172.21.15.27:0/3073667229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f769c0f55e0 0x7f769c109c90 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7f768c0026d0 tx=0x7f768c009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:57:37.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.616+0000 7f768affd700 1 -- 172.21.15.27:0/3073667229 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f768c009320 con 0x7f769c0f55e0 2021-11-03T17:57:37.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.616+0000 7f76a0cfa700 1 -- 172.21.15.27:0/3073667229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f769c10c740 con 0x7f769c0f55e0 2021-11-03T17:57:37.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.616+0000 7f768affd700 1 -- 172.21.15.27:0/3073667229 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f768c00f040 con 0x7f769c0f55e0 2021-11-03T17:57:37.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.616+0000 7f768affd700 1 -- 172.21.15.27:0/3073667229 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f768c0147e0 con 0x7f769c0f55e0 2021-11-03T17:57:37.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.616+0000 7f76a0cfa700 1 -- 172.21.15.27:0/3073667229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f769c10cd00 con 0x7f769c0f55e0 2021-11-03T17:57:37.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.617+0000 7f76a0cfa700 1 -- 172.21.15.27:0/3073667229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f769c045ad0 con 0x7f769c0f55e0 2021-11-03T17:57:37.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.618+0000 7f768affd700 1 -- 172.21.15.27:0/3073667229 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f768c009480 con 0x7f769c0f55e0 2021-11-03T17:57:37.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.618+0000 7f768affd700 1 --2- 172.21.15.27:0/3073667229 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f76840640b0 0x7f7684066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:57:37.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.618+0000 7f768affd700 1 -- 172.21.15.27:0/3073667229 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f768c080f10 con 0x7f769c0f55e0 2021-11-03T17:57:37.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.618+0000 7f7699d9b700 1 --2- 172.21.15.27:0/3073667229 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f76840640b0 0x7f7684066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:57:37.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.618+0000 7f7699d9b700 1 --2- 172.21.15.27:0/3073667229 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f76840640b0 0x7f7684066570 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f7690004630 tx=0x7f7690009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:57:37.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.622+0000 7f768affd700 1 -- 172.21.15.27:0/3073667229 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f768c053210 con 0x7f769c0f55e0 2021-11-03T17:57:37.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.801+0000 7f76a0cfa700 1 -- 172.21.15.27:0/3073667229 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f769c0fdf40 con 0x7f76840640b0 2021-11-03T17:57:37.800 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.803+0000 7f768affd700 1 -- 172.21.15.27:0/3073667229 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f769c0fdf40 con 0x7f76840640b0 2021-11-03T17:57:37.802 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.805+0000 7f76a0cfa700 1 -- 172.21.15.27:0/3073667229 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f76840640b0 msgr2=0x7f7684066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:57:37.802 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.805+0000 7f76a0cfa700 1 --2- 172.21.15.27:0/3073667229 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f76840640b0 0x7f7684066570 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 rx=0x7f7690004630 tx=0x7f7690009b30).stop 2021-11-03T17:57:37.802 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.805+0000 7f76a0cfa700 1 -- 172.21.15.27:0/3073667229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f769c0f55e0 msgr2=0x7f769c109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:57:37.803 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.805+0000 7f76a0cfa700 1 --2- 172.21.15.27:0/3073667229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f769c0f55e0 0x7f769c109c90 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 rx=0x7f768c0026d0 tx=0x7f768c009de0).stop 2021-11-03T17:57:37.803 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.805+0000 7f76a0cfa700 1 -- 172.21.15.27:0/3073667229 shutdown_connections 2021-11-03T17:57:37.803 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.805+0000 7f76a0cfa700 1 --2- 172.21.15.27:0/3073667229 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f76840640b0 0x7f7684066570 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:37.803 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.805+0000 7f76a0cfa700 1 --2- 172.21.15.27:0/3073667229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f769c0f55e0 0x7f769c109c90 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:37.803 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.805+0000 7f76a0cfa700 1 --2- 172.21.15.27:0/3073667229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f769c0f5ef0 0x7f769c10c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:37.804 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.805+0000 7f76a0cfa700 1 -- 172.21.15.27:0/3073667229 >> 172.21.15.27:0/3073667229 conn(0x7f769c0f0de0 msgr2=0x7f769c0fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:57:37.804 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.805+0000 7f76a0cfa700 1 -- 172.21.15.27:0/3073667229 shutdown_connections 2021-11-03T17:57:37.804 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.805+0000 7f76a0cfa700 1 -- 172.21.15.27:0/3073667229 wait complete. 2021-11-03T17:57:37.811 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:57:37.904 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.905+0000 7f60b8a99700 1 -- 172.21.15.27:0/248658539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60b40f55c0 msgr2=0x7f60b40f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:57:37.905 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.905+0000 7f60b8a99700 1 --2- 172.21.15.27:0/248658539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60b40f55c0 0x7f60b40f59e0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7f60a8004660 tx=0x7f60a8009b30).stop 2021-11-03T17:57:37.905 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.905+0000 7f60b8a99700 1 -- 172.21.15.27:0/248658539 shutdown_connections 2021-11-03T17:57:37.905 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.905+0000 7f60b8a99700 1 --2- 172.21.15.27:0/248658539 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60b40f6860 0x7f60b40f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:37.905 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.905+0000 7f60b8a99700 1 --2- 172.21.15.27:0/248658539 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60b40f55c0 0x7f60b40f59e0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:37.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.905+0000 7f60b8a99700 1 -- 172.21.15.27:0/248658539 >> 172.21.15.27:0/248658539 conn(0x7f60b40f0d40 msgr2=0x7f60b40f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:57:37.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.905+0000 7f60b8a99700 1 -- 172.21.15.27:0/248658539 shutdown_connections 2021-11-03T17:57:37.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.905+0000 7f60b8a99700 1 -- 172.21.15.27:0/248658539 wait complete. 2021-11-03T17:57:37.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.906+0000 7f60b8a99700 1 Processor -- start 2021-11-03T17:57:37.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.906+0000 7f60b8a99700 1 -- start start 2021-11-03T17:57:37.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.906+0000 7f60b8a99700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60b40f55c0 0x7f60b4106fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:57:37.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.906+0000 7f60b8a99700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60b40f6860 0x7f60b4107490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:57:37.908 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.906+0000 7f60b8a99700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b4107a40 con 0x7f60b40f6860 2021-11-03T17:57:37.908 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.906+0000 7f60b8a99700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b4107b80 con 0x7f60b40f55c0 2021-11-03T17:57:37.908 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.906+0000 7f60b1d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60b40f6860 0x7f60b4107490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:57:37.908 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.906+0000 7f60b1d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60b40f6860 0x7f60b4107490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60366/0 (socket says 172.21.15.27:60366) 2021-11-03T17:57:37.909 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.906+0000 7f60b1d9b700 1 -- 172.21.15.27:0/4126578843 learned_addr learned my addr 172.21.15.27:0/4126578843 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:57:37.909 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.906+0000 7f60b259c700 1 --2- 172.21.15.27:0/4126578843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60b40f55c0 0x7f60b4106fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:57:37.909 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.906+0000 7f60b1d9b700 1 -- 172.21.15.27:0/4126578843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60b40f55c0 msgr2=0x7f60b4106fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:57:37.909 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.906+0000 7f60b1d9b700 1 --2- 172.21.15.27:0/4126578843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60b40f55c0 0x7f60b4106fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:37.910 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.906+0000 7f60b1d9b700 1 -- 172.21.15.27:0/4126578843 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a8005040 con 0x7f60b40f6860 2021-11-03T17:57:37.910 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.907+0000 7f60b1d9b700 1 --2- 172.21.15.27:0/4126578843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60b40f6860 0x7f60b4107490 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7f60a40026d0 tx=0x7f60a4009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:57:37.910 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.907+0000 7f60a2ffd700 1 -- 172.21.15.27:0/4126578843 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60a4009320 con 0x7f60b40f6860 2021-11-03T17:57:37.910 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.907+0000 7f60a2ffd700 1 -- 172.21.15.27:0/4126578843 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f60a400f040 con 0x7f60b40f6860 2021-11-03T17:57:37.911 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.907+0000 7f60b8a99700 1 -- 172.21.15.27:0/4126578843 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60b4108520 con 0x7f60b40f6860 2021-11-03T17:57:37.911 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.907+0000 7f60a2ffd700 1 -- 172.21.15.27:0/4126578843 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f60a40147e0 con 0x7f60b40f6860 2021-11-03T17:57:37.911 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.907+0000 7f60b8a99700 1 -- 172.21.15.27:0/4126578843 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60b4108ae0 con 0x7f60b40f6860 2021-11-03T17:57:37.912 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.908+0000 7f60b8a99700 1 -- 172.21.15.27:0/4126578843 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60b4045ad0 con 0x7f60b40f6860 2021-11-03T17:57:37.913 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.908+0000 7f60a2ffd700 1 -- 172.21.15.27:0/4126578843 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f60a4009480 con 0x7f60b40f6860 2021-11-03T17:57:37.913 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.909+0000 7f60a2ffd700 1 --2- 172.21.15.27:0/4126578843 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f609c0640b0 0x7f609c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:57:37.913 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.909+0000 7f60a2ffd700 1 -- 172.21.15.27:0/4126578843 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f60a4080e50 con 0x7f60b40f6860 2021-11-03T17:57:37.913 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.909+0000 7f60b259c700 1 --2- 172.21.15.27:0/4126578843 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f609c0640b0 0x7f609c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:57:37.914 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.909+0000 7f60b259c700 1 --2- 172.21.15.27:0/4126578843 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f609c0640b0 0x7f609c066570 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f60a8009da0 tx=0x7f60a8009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:57:37.914 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:37.913+0000 7f60a2ffd700 1 -- 172.21.15.27:0/4126578843 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f60a4053210 con 0x7f60b40f6860 2021-11-03T17:57:38.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.086+0000 7f60b8a99700 1 -- 172.21.15.27:0/4126578843 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f60b40faca0 con 0x7f609c0640b0 2021-11-03T17:57:38.099 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.100+0000 7f60a2ffd700 1 -- 172.21.15.27:0/4126578843 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f60b40faca0 con 0x7f609c0640b0 2021-11-03T17:57:38.099 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:57:38.100 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (49m) 8m ago 50m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:57:38.100 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (40m) 8m ago 50m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:57:38.100 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (40m) 7m ago 49m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:57:38.100 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (49m) 8m ago 50m 41.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:57:38.101 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (42m) 8m ago 45m 23.8M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:57:38.101 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (45m) 8m ago 45m 3844M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:57:38.101 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (41m) 7m ago 45m 23.2M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:57:38.101 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (43m) 7m ago 45m 24.9M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:57:38.102 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (42m) 8m ago 51m 535M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:57:38.102 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (42m) 7m ago 49m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:57:38.102 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (41m) 8m ago 51m 198M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:57:38.102 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (41m) 7m ago 49m 95.6M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:57:38.102 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (50m) 8m ago 50m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:57:38.103 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (49m) 7m ago 49m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:57:38.103 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (40m) 8m ago 48m 331M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:57:38.103 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (40m) 8m ago 48m 361M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:57:38.103 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (39m) 8m ago 48m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:57:38.104 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (39m) 7m ago 47m 379M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:57:38.104 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (38m) 7m ago 47m 304M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:57:38.104 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (38m) 7m ago 47m 307M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:57:38.104 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (48m) 8m ago 49m 52.9M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:57:38.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.102+0000 7f60b8a99700 1 -- 172.21.15.27:0/4126578843 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f609c0640b0 msgr2=0x7f609c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:57:38.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.102+0000 7f60b8a99700 1 --2- 172.21.15.27:0/4126578843 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f609c0640b0 0x7f609c066570 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 rx=0x7f60a8009da0 tx=0x7f60a8009b30).stop 2021-11-03T17:57:38.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.102+0000 7f60b8a99700 1 -- 172.21.15.27:0/4126578843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60b40f6860 msgr2=0x7f60b4107490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:57:38.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.102+0000 7f60b8a99700 1 --2- 172.21.15.27:0/4126578843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60b40f6860 0x7f60b4107490 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7f60a40026d0 tx=0x7f60a4009de0).stop 2021-11-03T17:57:38.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.102+0000 7f60b8a99700 1 -- 172.21.15.27:0/4126578843 shutdown_connections 2021-11-03T17:57:38.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.102+0000 7f60b8a99700 1 --2- 172.21.15.27:0/4126578843 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f609c0640b0 0x7f609c066570 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:38.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.102+0000 7f60b8a99700 1 --2- 172.21.15.27:0/4126578843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f60b40f55c0 0x7f60b4106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:38.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.102+0000 7f60b8a99700 1 --2- 172.21.15.27:0/4126578843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f60b40f6860 0x7f60b4107490 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:38.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.102+0000 7f60b8a99700 1 -- 172.21.15.27:0/4126578843 >> 172.21.15.27:0/4126578843 conn(0x7f60b40f0d40 msgr2=0x7f60b405ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:57:38.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.102+0000 7f60b8a99700 1 -- 172.21.15.27:0/4126578843 shutdown_connections 2021-11-03T17:57:38.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.102+0000 7f60b8a99700 1 -- 172.21.15.27:0/4126578843 wait complete. 2021-11-03T17:57:38.200 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.201+0000 7fdf63237700 1 -- 172.21.15.27:0/4196464097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf5c0f6530 msgr2=0x7fdf5c0f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:57:38.200 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.201+0000 7fdf63237700 1 --2- 172.21.15.27:0/4196464097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf5c0f6530 0x7fdf5c0f6950 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7fdf4c0099e0 tx=0x7fdf4c004d10).stop 2021-11-03T17:57:38.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.201+0000 7fdf63237700 1 -- 172.21.15.27:0/4196464097 shutdown_connections 2021-11-03T17:57:38.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.201+0000 7fdf63237700 1 --2- 172.21.15.27:0/4196464097 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf5c0f77d0 0x7fdf5c0f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:38.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.201+0000 7fdf63237700 1 --2- 172.21.15.27:0/4196464097 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf5c0f6530 0x7fdf5c0f6950 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:38.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.201+0000 7fdf63237700 1 -- 172.21.15.27:0/4196464097 >> 172.21.15.27:0/4196464097 conn(0x7fdf5c0f1cd0 msgr2=0x7fdf5c0f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:57:38.202 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.201+0000 7fdf63237700 1 -- 172.21.15.27:0/4196464097 shutdown_connections 2021-11-03T17:57:38.202 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.201+0000 7fdf63237700 1 -- 172.21.15.27:0/4196464097 wait complete. 2021-11-03T17:57:38.202 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.202+0000 7fdf63237700 1 Processor -- start 2021-11-03T17:57:38.202 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.202+0000 7fdf63237700 1 -- start start 2021-11-03T17:57:38.202 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.202+0000 7fdf63237700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf5c0f6530 0x7fdf5c106e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:57:38.203 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.202+0000 7fdf63237700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf5c0f77d0 0x7fdf5c107360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:57:38.203 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.202+0000 7fdf63237700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c107910 con 0x7fdf5c0f77d0 2021-11-03T17:57:38.203 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.202+0000 7fdf63237700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf5c107a50 con 0x7fdf5c0f6530 2021-11-03T17:57:38.203 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.202+0000 7fdf61a34700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf5c0f77d0 0x7fdf5c107360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:57:38.204 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.202+0000 7fdf61a34700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf5c0f77d0 0x7fdf5c107360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60376/0 (socket says 172.21.15.27:60376) 2021-11-03T17:57:38.204 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.202+0000 7fdf61a34700 1 -- 172.21.15.27:0/3922933371 learned_addr learned my addr 172.21.15.27:0/3922933371 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:57:38.204 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.202+0000 7fdf62235700 1 --2- 172.21.15.27:0/3922933371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf5c0f6530 0x7fdf5c106e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:57:38.204 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.203+0000 7fdf61a34700 1 -- 172.21.15.27:0/3922933371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf5c0f6530 msgr2=0x7fdf5c106e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:57:38.205 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.203+0000 7fdf61a34700 1 --2- 172.21.15.27:0/3922933371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf5c0f6530 0x7fdf5c106e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:38.205 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.203+0000 7fdf61a34700 1 -- 172.21.15.27:0/3922933371 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf4c009640 con 0x7fdf5c0f77d0 2021-11-03T17:57:38.205 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.203+0000 7fdf61a34700 1 --2- 172.21.15.27:0/3922933371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf5c0f77d0 0x7fdf5c107360 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7fdf500026d0 tx=0x7fdf50009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:57:38.205 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.203+0000 7fdf5affd700 1 -- 172.21.15.27:0/3922933371 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf500079c0 con 0x7fdf5c0f77d0 2021-11-03T17:57:38.205 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.203+0000 7fdf5affd700 1 -- 172.21.15.27:0/3922933371 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fdf5000f040 con 0x7fdf5c0f77d0 2021-11-03T17:57:38.206 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.203+0000 7fdf5affd700 1 -- 172.21.15.27:0/3922933371 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf500148b0 con 0x7fdf5c0f77d0 2021-11-03T17:57:38.206 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.203+0000 7fdf63237700 1 -- 172.21.15.27:0/3922933371 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf5c1083f0 con 0x7fdf5c0f77d0 2021-11-03T17:57:38.206 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.203+0000 7fdf63237700 1 -- 172.21.15.27:0/3922933371 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf5c1088e0 con 0x7fdf5c0f77d0 2021-11-03T17:57:38.207 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.204+0000 7fdf63237700 1 -- 172.21.15.27:0/3922933371 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf5c045ad0 con 0x7fdf5c0f77d0 2021-11-03T17:57:38.208 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.204+0000 7fdf5affd700 1 -- 172.21.15.27:0/3922933371 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fdf50007b20 con 0x7fdf5c0f77d0 2021-11-03T17:57:38.208 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.205+0000 7fdf5affd700 1 --2- 172.21.15.27:0/3922933371 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdf44064100 0x7fdf440665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:57:38.208 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.205+0000 7fdf5affd700 1 -- 172.21.15.27:0/3922933371 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fdf50080790 con 0x7fdf5c0f77d0 2021-11-03T17:57:38.208 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.205+0000 7fdf62235700 1 --2- 172.21.15.27:0/3922933371 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdf44064100 0x7fdf440665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:57:38.209 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.205+0000 7fdf62235700 1 --2- 172.21.15.27:0/3922933371 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdf44064100 0x7fdf440665c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7fdf4c00b730 tx=0x7fdf4c004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:57:38.209 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.209+0000 7fdf5affd700 1 -- 172.21.15.27:0/3922933371 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fdf50052a90 con 0x7fdf5c0f77d0 2021-11-03T17:57:38.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.441+0000 7fdf63237700 1 -- 172.21.15.27:0/3922933371 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdf5c0fedf0 con 0x7fdf5c0f77d0 2021-11-03T17:57:38.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.441+0000 7fdf5affd700 1 -- 172.21.15.27:0/3922933371 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fdf50052680 con 0x7fdf5c0f77d0 2021-11-03T17:57:38.439 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:57:38.439 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:57:38.439 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:57:38.440 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:57:38.440 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:57:38.440 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:57:38.441 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:57:38.441 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:57:38.441 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:57:38.441 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:57:38.442 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:57:38.442 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:57:38.442 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:57:38.442 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:57:38.443 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:57:38.443 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:57:38.443 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:57:38.443 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:57:38.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.443+0000 7fdf63237700 1 -- 172.21.15.27:0/3922933371 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdf44064100 msgr2=0x7fdf440665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:57:38.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.443+0000 7fdf63237700 1 --2- 172.21.15.27:0/3922933371 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdf44064100 0x7fdf440665c0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 rx=0x7fdf4c00b730 tx=0x7fdf4c004d10).stop 2021-11-03T17:57:38.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.443+0000 7fdf63237700 1 -- 172.21.15.27:0/3922933371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf5c0f77d0 msgr2=0x7fdf5c107360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:57:38.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.443+0000 7fdf63237700 1 --2- 172.21.15.27:0/3922933371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf5c0f77d0 0x7fdf5c107360 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7fdf500026d0 tx=0x7fdf50009de0).stop 2021-11-03T17:57:38.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.443+0000 7fdf63237700 1 -- 172.21.15.27:0/3922933371 shutdown_connections 2021-11-03T17:57:38.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.443+0000 7fdf63237700 1 --2- 172.21.15.27:0/3922933371 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdf44064100 0x7fdf440665c0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:38.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.443+0000 7fdf63237700 1 --2- 172.21.15.27:0/3922933371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf5c0f6530 0x7fdf5c106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:38.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.443+0000 7fdf63237700 1 --2- 172.21.15.27:0/3922933371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf5c0f77d0 0x7fdf5c107360 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:38.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.443+0000 7fdf63237700 1 -- 172.21.15.27:0/3922933371 >> 172.21.15.27:0/3922933371 conn(0x7fdf5c0f1cd0 msgr2=0x7fdf5c0fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:57:38.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.443+0000 7fdf63237700 1 -- 172.21.15.27:0/3922933371 shutdown_connections 2021-11-03T17:57:38.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.443+0000 7fdf63237700 1 -- 172.21.15.27:0/3922933371 wait complete. 2021-11-03T17:57:38.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:38 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:57:38.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:38 smithi115 ceph-mon[65453]: pgmap v1289: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:38.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:38 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:57:38.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:38 smithi027 ceph-mon[97244]: pgmap v1289: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:38.541 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.544+0000 7f91e6209700 1 -- 172.21.15.27:0/1835426840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91e00f66e0 msgr2=0x7f91e00f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:57:38.541 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.544+0000 7f91e6209700 1 --2- 172.21.15.27:0/1835426840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91e00f66e0 0x7f91e00f6b00 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7f91d80099e0 tx=0x7f91d8004d10).stop 2021-11-03T17:57:38.541 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.544+0000 7f91e6209700 1 -- 172.21.15.27:0/1835426840 shutdown_connections 2021-11-03T17:57:38.542 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.544+0000 7f91e6209700 1 --2- 172.21.15.27:0/1835426840 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91e00f7980 0x7f91e00f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:38.542 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.544+0000 7f91e6209700 1 --2- 172.21.15.27:0/1835426840 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91e00f66e0 0x7f91e00f6b00 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:38.542 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.544+0000 7f91e6209700 1 -- 172.21.15.27:0/1835426840 >> 172.21.15.27:0/1835426840 conn(0x7f91e00f1e40 msgr2=0x7f91e00f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:57:38.542 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.544+0000 7f91e6209700 1 -- 172.21.15.27:0/1835426840 shutdown_connections 2021-11-03T17:57:38.543 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.544+0000 7f91e6209700 1 -- 172.21.15.27:0/1835426840 wait complete. 2021-11-03T17:57:38.543 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.544+0000 7f91e6209700 1 Processor -- start 2021-11-03T17:57:38.545 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.544+0000 7f91e6209700 1 -- start start 2021-11-03T17:57:38.546 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.545+0000 7f91e6209700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91e00f66e0 0x7f91e01070d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:57:38.546 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.545+0000 7f91e6209700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91e00f7980 0x7f91e01075c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:57:38.546 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.545+0000 7f91e6209700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e0107b70 con 0x7f91e00f66e0 2021-11-03T17:57:38.546 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.545+0000 7f91e6209700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91e0107cb0 con 0x7f91e00f7980 2021-11-03T17:57:38.547 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.545+0000 7f91df7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91e00f66e0 0x7f91e01070d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:57:38.547 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.545+0000 7f91df7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91e00f66e0 0x7f91e01070d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60384/0 (socket says 172.21.15.27:60384) 2021-11-03T17:57:38.547 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.545+0000 7f91df7fe700 1 -- 172.21.15.27:0/2727260796 learned_addr learned my addr 172.21.15.27:0/2727260796 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:57:38.547 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.545+0000 7f91deffd700 1 --2- 172.21.15.27:0/2727260796 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91e00f7980 0x7f91e01075c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:57:38.548 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.545+0000 7f91df7fe700 1 -- 172.21.15.27:0/2727260796 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91e00f7980 msgr2=0x7f91e01075c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:57:38.548 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.545+0000 7f91df7fe700 1 --2- 172.21.15.27:0/2727260796 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91e00f7980 0x7f91e01075c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:38.548 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.545+0000 7f91df7fe700 1 -- 172.21.15.27:0/2727260796 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d8009640 con 0x7f91e00f66e0 2021-11-03T17:57:38.548 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.545+0000 7f91deffd700 1 --2- 172.21.15.27:0/2727260796 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91e00f7980 0x7f91e01075c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:57:38.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.545+0000 7f91df7fe700 1 --2- 172.21.15.27:0/2727260796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91e00f66e0 0x7f91e01070d0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7f91d800c010 tx=0x7f91d8004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:57:38.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.545+0000 7f91d7fff700 1 -- 172.21.15.27:0/2727260796 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91d800e070 con 0x7f91e00f66e0 2021-11-03T17:57:38.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.545+0000 7f91d7fff700 1 -- 172.21.15.27:0/2727260796 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f91d8007dc0 con 0x7f91e00f66e0 2021-11-03T17:57:38.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.545+0000 7f91e6209700 1 -- 172.21.15.27:0/2727260796 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91e01085f0 con 0x7f91e00f66e0 2021-11-03T17:57:38.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.545+0000 7f91d7fff700 1 -- 172.21.15.27:0/2727260796 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f91d8018070 con 0x7f91e00f66e0 2021-11-03T17:57:38.550 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.545+0000 7f91e6209700 1 -- 172.21.15.27:0/2727260796 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91e0108ad0 con 0x7f91e00f66e0 2021-11-03T17:57:38.550 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.546+0000 7f91e6209700 1 -- 172.21.15.27:0/2727260796 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91e0045ad0 con 0x7f91e00f66e0 2021-11-03T17:57:38.550 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.547+0000 7f91d7fff700 1 -- 172.21.15.27:0/2727260796 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f91d8024060 con 0x7f91e00f66e0 2021-11-03T17:57:38.550 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.547+0000 7f91d7fff700 1 --2- 172.21.15.27:0/2727260796 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f91cc0640b0 0x7f91cc066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:57:38.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.547+0000 7f91d7fff700 1 -- 172.21.15.27:0/2727260796 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f91d8082320 con 0x7f91e00f66e0 2021-11-03T17:57:38.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.547+0000 7f91deffd700 1 --2- 172.21.15.27:0/2727260796 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f91cc0640b0 0x7f91cc066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:57:38.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.548+0000 7f91deffd700 1 --2- 172.21.15.27:0/2727260796 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f91cc0640b0 0x7f91cc066570 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f91c80041d0 tx=0x7f91c8009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:57:38.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.551+0000 7f91d7fff700 1 -- 172.21.15.27:0/2727260796 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f91d8054620 con 0x7f91e00f66e0 2021-11-03T17:57:38.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.756+0000 7f91e6209700 1 -- 172.21.15.27:0/2727260796 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f91e00feea0 con 0x7f91e00f66e0 2021-11-03T17:57:38.754 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.757+0000 7f91d7fff700 1 -- 172.21.15.27:0/2727260796 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f91d8022710 con 0x7f91e00f66e0 2021-11-03T17:57:38.754 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:57:38.755 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:57:38.755 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:57:38.755 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:57:38.755 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:57:38.756 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:57:38.756 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:57:38.756 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:57:38.756 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:57:38.756 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:57:38.757 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:57:38.757 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:57:38.757 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:57:38.757 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:57:38.758 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:57:38.758 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:57:38.758 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:57:38.758 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:57:38.759 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:57:38.759 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:57:38.759 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:57:38.759 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:57:38.760 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:57:38.760 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:57:38.760 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:57:38.760 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:57:38.761 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:57:38.761 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:57:38.761 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:57:38.761 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:57:38.762 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:57:38.762 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:57:38.762 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:57:38.762 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:57:38.762 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:57:38.763 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:57:38.763 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:57:38.763 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:57:38.763 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:57:38.764 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.758+0000 7f91e6209700 1 -- 172.21.15.27:0/2727260796 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f91cc0640b0 msgr2=0x7f91cc066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:57:38.765 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.758+0000 7f91e6209700 1 --2- 172.21.15.27:0/2727260796 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f91cc0640b0 0x7f91cc066570 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f91c80041d0 tx=0x7f91c8009040).stop 2021-11-03T17:57:38.765 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.758+0000 7f91e6209700 1 -- 172.21.15.27:0/2727260796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91e00f66e0 msgr2=0x7f91e01070d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:57:38.765 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.758+0000 7f91e6209700 1 --2- 172.21.15.27:0/2727260796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91e00f66e0 0x7f91e01070d0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7f91d800c010 tx=0x7f91d8004d10).stop 2021-11-03T17:57:38.765 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.758+0000 7f91e6209700 1 -- 172.21.15.27:0/2727260796 shutdown_connections 2021-11-03T17:57:38.766 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.758+0000 7f91e6209700 1 --2- 172.21.15.27:0/2727260796 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f91cc0640b0 0x7f91cc066570 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:38.766 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.758+0000 7f91e6209700 1 --2- 172.21.15.27:0/2727260796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f91e00f66e0 0x7f91e01070d0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:38.766 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.758+0000 7f91e6209700 1 --2- 172.21.15.27:0/2727260796 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f91e00f7980 0x7f91e01075c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:57:38.766 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.758+0000 7f91e6209700 1 -- 172.21.15.27:0/2727260796 >> 172.21.15.27:0/2727260796 conn(0x7f91e00f1e40 msgr2=0x7f91e00fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:57:38.766 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.759+0000 7f91e6209700 1 -- 172.21.15.27:0/2727260796 shutdown_connections 2021-11-03T17:57:38.767 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:57:38.759+0000 7f91e6209700 1 -- 172.21.15.27:0/2727260796 wait complete. 2021-11-03T17:57:38.767 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:57:39.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:39 smithi115 ceph-mon[65453]: from='client.35432 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:57:39.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:39 smithi115 ceph-mon[65453]: from='client.35436 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:57:39.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:39 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3922933371' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:39.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:39 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2727260796' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:57:39.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:39 smithi027 ceph-mon[97244]: from='client.35432 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:57:39.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:39 smithi027 ceph-mon[97244]: from='client.35436 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:57:39.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:39 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3922933371' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:39.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:39 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2727260796' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:57:40.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:40 smithi115 ceph-mon[65453]: pgmap v1290: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:40.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2502.419914 secs 2021-11-03T17:57:40.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:40 smithi027 ceph-mon[97244]: pgmap v1290: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:40.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2502.419914 secs 2021-11-03T17:57:42.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:42 smithi115 ceph-mon[65453]: pgmap v1291: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:42.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:42 smithi027 ceph-mon[97244]: pgmap v1291: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:44.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:44 smithi115 ceph-mon[65453]: pgmap v1292: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:44.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:44 smithi027 ceph-mon[97244]: pgmap v1292: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:45.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2507.420018 secs 2021-11-03T17:57:45.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2507.420018 secs 2021-11-03T17:57:46.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:46 smithi115 ceph-mon[65453]: pgmap v1293: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:46.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:46 smithi027 ceph-mon[97244]: pgmap v1293: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:47.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:57:47.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:57:47.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:57:47.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:47.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:47.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:47.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:47.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:57:47.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:57:47.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:57:47.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:47.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:47.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:47.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:48.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:48 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:57:48.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:48 smithi115 ceph-mon[65453]: pgmap v1294: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:48.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:48 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:57:48.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:48 smithi027 ceph-mon[97244]: pgmap v1294: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:50.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:50 smithi115 ceph-mon[65453]: pgmap v1295: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:50.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2512.420159 secs 2021-11-03T17:57:50.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:50 smithi027 ceph-mon[97244]: pgmap v1295: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:50.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2512.420159 secs 2021-11-03T17:57:52.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:52 smithi115 ceph-mon[65453]: pgmap v1296: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:52.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:52 smithi027 ceph-mon[97244]: pgmap v1296: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:54.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:54 smithi115 ceph-mon[65453]: pgmap v1297: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:54.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:54 smithi027 ceph-mon[97244]: pgmap v1297: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:55.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2517.420301 secs 2021-11-03T17:57:55.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2517.420301 secs 2021-11-03T17:57:56.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:56 smithi115 ceph-mon[65453]: pgmap v1298: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:56.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:57:56.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:57:56.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:56 smithi027 ceph-mon[97244]: pgmap v1298: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:56.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:57:56.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:57:57.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:57:57.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:57:57.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:57:57.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:57.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:57.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:57.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:57.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:57:57.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:57:57.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:57:57.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:57.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:57.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:57.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:57:58.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:58 smithi115 ceph-mon[65453]: pgmap v1299: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:58.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:57:58 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:57:58.519 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:58 smithi027 ceph-mon[97244]: pgmap v1299: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:57:58.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:57:58 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:58:00.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:00 smithi115 ceph-mon[65453]: pgmap v1300: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:00.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2522.420439 secs 2021-11-03T17:58:00.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:00 smithi027 ceph-mon[97244]: pgmap v1300: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:00.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2522.420439 secs 2021-11-03T17:58:02.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:02 smithi115 ceph-mon[65453]: pgmap v1301: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:02.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:02 smithi027 ceph-mon[97244]: pgmap v1301: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:04.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:04 smithi115 ceph-mon[65453]: pgmap v1302: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:04.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:04 smithi027 ceph-mon[97244]: pgmap v1302: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:05.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2527.420563 secs 2021-11-03T17:58:05.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2527.420563 secs 2021-11-03T17:58:06.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:06 smithi115 ceph-mon[65453]: pgmap v1303: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:06.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:06 smithi027 ceph-mon[97244]: pgmap v1303: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:07.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:58:07.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:58:07.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:58:07.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:07.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:07.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:07.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:07.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:58:07.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:58:07.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:58:07.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:07.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:07.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:07.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:08.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:08 smithi115 ceph-mon[65453]: pgmap v1304: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:08.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:08 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:58:08.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:08 smithi027 ceph-mon[97244]: pgmap v1304: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:08.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:08 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:58:08.872 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:08.873+0000 7efff54d5700 1 -- 172.21.15.27:0/1818146502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efff00f5ef0 msgr2=0x7efff00f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:58:08.872 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:08.873+0000 7efff54d5700 1 --2- 172.21.15.27:0/1818146502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efff00f5ef0 0x7efff00f6920 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7effe0004660 tx=0x7effe0009b30).stop 2021-11-03T17:58:08.873 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:08.873+0000 7efff54d5700 1 -- 172.21.15.27:0/1818146502 shutdown_connections 2021-11-03T17:58:08.873 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:08.873+0000 7efff54d5700 1 --2- 172.21.15.27:0/1818146502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efff00f5ef0 0x7efff00f6920 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:08.873 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:08.873+0000 7efff54d5700 1 --2- 172.21.15.27:0/1818146502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efff00f55e0 0x7efff00f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:58:08.873 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:08.873+0000 7efff54d5700 1 -- 172.21.15.27:0/1818146502 >> 172.21.15.27:0/1818146502 conn(0x7efff00f0de0 msgr2=0x7efff00f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:58:08.874 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:08.873+0000 7efff54d5700 1 -- 172.21.15.27:0/1818146502 shutdown_connections 2021-11-03T17:58:08.874 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:08.873+0000 7efff54d5700 1 -- 172.21.15.27:0/1818146502 wait complete. 2021-11-03T17:58:08.874 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:08.874+0000 7efff54d5700 1 Processor -- start 2021-11-03T17:58:08.874 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:08.874+0000 7efff54d5700 1 -- start start 2021-11-03T17:58:08.875 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:08.874+0000 7efff54d5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efff00f55e0 0x7efff00f97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:58:08.875 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:08.874+0000 7efff54d5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efff00f5ef0 0x7efff00f7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:58:08.875 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:08.874+0000 7efff54d5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff00f8250 con 0x7efff00f5ef0 2021-11-03T17:58:08.875 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:08.874+0000 7efff54d5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff00f8390 con 0x7efff00f55e0 2021-11-03T17:58:08.875 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:08.874+0000 7effee7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efff00f5ef0 0x7efff00f7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:58:08.876 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:08.875+0000 7effee7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efff00f5ef0 0x7efff00f7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60404/0 (socket says 172.21.15.27:60404) 2021-11-03T17:58:08.876 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:08.875+0000 7effee7fc700 1 -- 172.21.15.27:0/230645743 learned_addr learned my addr 172.21.15.27:0/230645743 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:58:08.876 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:08.875+0000 7effeeffd700 1 --2- 172.21.15.27:0/230645743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efff00f55e0 0x7efff00f97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:58:08.876 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:08.875+0000 7effee7fc700 1 -- 172.21.15.27:0/230645743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efff00f55e0 msgr2=0x7efff00f97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:58:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:08.875+0000 7effee7fc700 1 --2- 172.21.15.27:0/230645743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efff00f55e0 0x7efff00f97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:08.875+0000 7effee7fc700 1 -- 172.21.15.27:0/230645743 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe0005040 con 0x7efff00f5ef0 2021-11-03T17:58:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:08.875+0000 7effeeffd700 1 --2- 172.21.15.27:0/230645743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efff00f55e0 0x7efff00f97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:58:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:08.875+0000 7effee7fc700 1 --2- 172.21.15.27:0/230645743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efff00f5ef0 0x7efff00f7c60 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7effe0009990 tx=0x7effe0009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:58:08.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:08.875+0000 7effd77fe700 1 -- 172.21.15.27:0/230645743 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7effe0018070 con 0x7efff00f5ef0 2021-11-03T17:58:08.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:08.875+0000 7effd77fe700 1 -- 172.21.15.27:0/230645743 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7effe0007ca0 con 0x7efff00f5ef0 2021-11-03T17:58:08.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:08.875+0000 7efff54d5700 1 -- 172.21.15.27:0/230645743 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efff00f8610 con 0x7efff00f5ef0 2021-11-03T17:58:08.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:08.875+0000 7effd77fe700 1 -- 172.21.15.27:0/230645743 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7effe001c620 con 0x7efff00f5ef0 2021-11-03T17:58:08.879 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:08.875+0000 7efff54d5700 1 -- 172.21.15.27:0/230645743 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efff010efd0 con 0x7efff00f5ef0 2021-11-03T17:58:08.879 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:08.876+0000 7efff54d5700 1 -- 172.21.15.27:0/230645743 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efff0045ad0 con 0x7efff00f5ef0 2021-11-03T17:58:08.880 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:08.877+0000 7effd77fe700 1 -- 172.21.15.27:0/230645743 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7effe001d960 con 0x7efff00f5ef0 2021-11-03T17:58:08.880 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:08.877+0000 7effd77fe700 1 --2- 172.21.15.27:0/230645743 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7effd80640b0 0x7effd8066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:58:08.880 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:08.877+0000 7effd77fe700 1 -- 172.21.15.27:0/230645743 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7effe0082660 con 0x7efff00f5ef0 2021-11-03T17:58:08.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:08.877+0000 7effeeffd700 1 --2- 172.21.15.27:0/230645743 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7effd80640b0 0x7effd8066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:58:08.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:08.878+0000 7effeeffd700 1 --2- 172.21.15.27:0/230645743 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7effd80640b0 0x7effd8066570 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7efff00f9570 tx=0x7effe4008040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:58:08.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:08.881+0000 7effd77fe700 1 -- 172.21.15.27:0/230645743 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7effe0054a20 con 0x7efff00f5ef0 2021-11-03T17:58:09.056 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.060+0000 7efff54d5700 1 -- 172.21.15.27:0/230645743 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efff00fad60 con 0x7effd80640b0 2021-11-03T17:58:09.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.061+0000 7effd77fe700 1 -- 172.21.15.27:0/230645743 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7efff00fad60 con 0x7effd80640b0 2021-11-03T17:58:09.059 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.063+0000 7efff54d5700 1 -- 172.21.15.27:0/230645743 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7effd80640b0 msgr2=0x7effd8066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:58:09.059 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.063+0000 7efff54d5700 1 --2- 172.21.15.27:0/230645743 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7effd80640b0 0x7effd8066570 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7efff00f9570 tx=0x7effe4008040).stop 2021-11-03T17:58:09.060 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.063+0000 7efff54d5700 1 -- 172.21.15.27:0/230645743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efff00f5ef0 msgr2=0x7efff00f7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:58:09.060 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.063+0000 7efff54d5700 1 --2- 172.21.15.27:0/230645743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efff00f5ef0 0x7efff00f7c60 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7effe0009990 tx=0x7effe0009b30).stop 2021-11-03T17:58:09.060 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.063+0000 7efff54d5700 1 -- 172.21.15.27:0/230645743 shutdown_connections 2021-11-03T17:58:09.060 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.063+0000 7efff54d5700 1 --2- 172.21.15.27:0/230645743 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7effd80640b0 0x7effd8066570 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:09.061 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.063+0000 7efff54d5700 1 --2- 172.21.15.27:0/230645743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efff00f55e0 0x7efff00f97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:09.061 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.063+0000 7efff54d5700 1 --2- 172.21.15.27:0/230645743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efff00f5ef0 0x7efff00f7c60 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:09.061 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.063+0000 7efff54d5700 1 -- 172.21.15.27:0/230645743 >> 172.21.15.27:0/230645743 conn(0x7efff00f0de0 msgr2=0x7efff005ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:58:09.061 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.063+0000 7efff54d5700 1 -- 172.21.15.27:0/230645743 shutdown_connections 2021-11-03T17:58:09.062 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.063+0000 7efff54d5700 1 -- 172.21.15.27:0/230645743 wait complete. 2021-11-03T17:58:09.069 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:58:09.161 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.163+0000 7ff25e61a700 1 -- 172.21.15.27:0/2749138588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2580f6ff0 msgr2=0x7ff2580f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:58:09.161 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.163+0000 7ff25e61a700 1 --2- 172.21.15.27:0/2749138588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2580f6ff0 0x7ff2580f7a20 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7ff248004660 tx=0x7ff248009b30).stop 2021-11-03T17:58:09.162 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.164+0000 7ff25e61a700 1 -- 172.21.15.27:0/2749138588 shutdown_connections 2021-11-03T17:58:09.162 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.164+0000 7ff25e61a700 1 --2- 172.21.15.27:0/2749138588 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2580f6ff0 0x7ff2580f7a20 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:09.162 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.164+0000 7ff25e61a700 1 --2- 172.21.15.27:0/2749138588 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2580f66e0 0x7ff2580f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:09.162 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.164+0000 7ff25e61a700 1 -- 172.21.15.27:0/2749138588 >> 172.21.15.27:0/2749138588 conn(0x7ff2580f1f00 msgr2=0x7ff2580f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:58:09.163 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.164+0000 7ff25e61a700 1 -- 172.21.15.27:0/2749138588 shutdown_connections 2021-11-03T17:58:09.163 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.164+0000 7ff25e61a700 1 -- 172.21.15.27:0/2749138588 wait complete. 2021-11-03T17:58:09.163 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.164+0000 7ff25e61a700 1 Processor -- start 2021-11-03T17:58:09.163 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.164+0000 7ff25e61a700 1 -- start start 2021-11-03T17:58:09.164 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.164+0000 7ff25e61a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2580f66e0 0x7ff258106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:58:09.164 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.164+0000 7ff25e61a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2580f6ff0 0x7ff258107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:58:09.164 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.164+0000 7ff25e61a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff258107a10 con 0x7ff2580f66e0 2021-11-03T17:58:09.164 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.164+0000 7ff25e61a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff258107b50 con 0x7ff2580f6ff0 2021-11-03T17:58:09.165 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.165+0000 7ff257fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2580f66e0 0x7ff258106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:58:09.165 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.165+0000 7ff257fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2580f66e0 0x7ff258106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60412/0 (socket says 172.21.15.27:60412) 2021-11-03T17:58:09.165 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.165+0000 7ff257fff700 1 -- 172.21.15.27:0/23422124 learned_addr learned my addr 172.21.15.27:0/23422124 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:58:09.165 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.165+0000 7ff2577fe700 1 --2- 172.21.15.27:0/23422124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2580f6ff0 0x7ff258107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:58:09.165 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.165+0000 7ff257fff700 1 -- 172.21.15.27:0/23422124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2580f6ff0 msgr2=0x7ff258107460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:58:09.166 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.165+0000 7ff257fff700 1 --2- 172.21.15.27:0/23422124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2580f6ff0 0x7ff258107460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:09.166 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.165+0000 7ff257fff700 1 -- 172.21.15.27:0/23422124 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff248005040 con 0x7ff2580f66e0 2021-11-03T17:58:09.166 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.165+0000 7ff257fff700 1 --2- 172.21.15.27:0/23422124 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2580f66e0 0x7ff258106f70 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7ff2400026d0 tx=0x7ff240009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:58:09.166 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.165+0000 7ff254ff9700 1 -- 172.21.15.27:0/23422124 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2400079c0 con 0x7ff2580f66e0 2021-11-03T17:58:09.167 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.165+0000 7ff254ff9700 1 -- 172.21.15.27:0/23422124 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7ff24000f040 con 0x7ff2580f66e0 2021-11-03T17:58:09.167 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.165+0000 7ff25e61a700 1 -- 172.21.15.27:0/23422124 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2581084f0 con 0x7ff2580f66e0 2021-11-03T17:58:09.167 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.165+0000 7ff254ff9700 1 -- 172.21.15.27:0/23422124 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2400148b0 con 0x7ff2580f66e0 2021-11-03T17:58:09.167 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.165+0000 7ff25e61a700 1 -- 172.21.15.27:0/23422124 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff258108ab0 con 0x7ff2580f66e0 2021-11-03T17:58:09.169 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.166+0000 7ff25e61a700 1 -- 172.21.15.27:0/23422124 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff258045ad0 con 0x7ff2580f66e0 2021-11-03T17:58:09.169 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.167+0000 7ff254ff9700 1 -- 172.21.15.27:0/23422124 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7ff240007b20 con 0x7ff2580f66e0 2021-11-03T17:58:09.169 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.167+0000 7ff254ff9700 1 --2- 172.21.15.27:0/23422124 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff2440640b0 0x7ff244066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:58:09.169 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.167+0000 7ff254ff9700 1 -- 172.21.15.27:0/23422124 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7ff240080bf0 con 0x7ff2580f66e0 2021-11-03T17:58:09.170 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.167+0000 7ff2577fe700 1 --2- 172.21.15.27:0/23422124 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff2440640b0 0x7ff244066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:58:09.170 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.168+0000 7ff2577fe700 1 --2- 172.21.15.27:0/23422124 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff2440640b0 0x7ff244066570 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7ff248004630 tx=0x7ff248009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:58:09.170 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.171+0000 7ff254ff9700 1 -- 172.21.15.27:0/23422124 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7ff240052ef0 con 0x7ff2580f66e0 2021-11-03T17:58:09.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.343+0000 7ff25e61a700 1 -- 172.21.15.27:0/23422124 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff258108d30 con 0x7ff2440640b0 2021-11-03T17:58:09.354 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.358+0000 7ff254ff9700 1 -- 172.21.15.27:0/23422124 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7ff258108d30 con 0x7ff2440640b0 2021-11-03T17:58:09.355 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:58:09.355 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (49m) 9m ago 50m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:58:09.355 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (41m) 9m ago 50m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:58:09.355 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (41m) 8m ago 50m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:58:09.356 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (49m) 9m ago 50m 41.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:58:09.356 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (43m) 9m ago 46m 23.8M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:58:09.356 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (46m) 9m ago 46m 3844M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:58:09.356 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (41m) 8m ago 46m 23.2M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:58:09.356 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (43m) 8m ago 46m 24.9M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:58:09.357 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (43m) 9m ago 52m 535M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:58:09.357 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (42m) 8m ago 50m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:58:09.357 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (42m) 9m ago 52m 198M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:58:09.357 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (41m) 8m ago 50m 95.6M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:58:09.358 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (50m) 9m ago 50m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:58:09.358 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (49m) 8m ago 49m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:58:09.358 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (41m) 9m ago 49m 331M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:58:09.358 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (40m) 9m ago 49m 361M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:58:09.359 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (40m) 9m ago 48m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:58:09.359 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (39m) 8m ago 48m 379M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:58:09.359 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (39m) 8m ago 48m 304M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:58:09.359 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (38m) 8m ago 47m 307M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:58:09.360 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (49m) 9m ago 50m 52.9M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:58:09.360 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.360+0000 7ff25e61a700 1 -- 172.21.15.27:0/23422124 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff2440640b0 msgr2=0x7ff244066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:58:09.361 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.360+0000 7ff25e61a700 1 --2- 172.21.15.27:0/23422124 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff2440640b0 0x7ff244066570 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 rx=0x7ff248004630 tx=0x7ff248009b30).stop 2021-11-03T17:58:09.361 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.360+0000 7ff25e61a700 1 -- 172.21.15.27:0/23422124 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2580f66e0 msgr2=0x7ff258106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:58:09.361 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.360+0000 7ff25e61a700 1 --2- 172.21.15.27:0/23422124 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2580f66e0 0x7ff258106f70 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7ff2400026d0 tx=0x7ff240009de0).stop 2021-11-03T17:58:09.361 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.361+0000 7ff25e61a700 1 -- 172.21.15.27:0/23422124 shutdown_connections 2021-11-03T17:58:09.362 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.361+0000 7ff25e61a700 1 --2- 172.21.15.27:0/23422124 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff2440640b0 0x7ff244066570 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:09.362 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.361+0000 7ff25e61a700 1 --2- 172.21.15.27:0/23422124 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2580f66e0 0x7ff258106f70 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:09.362 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.361+0000 7ff25e61a700 1 --2- 172.21.15.27:0/23422124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2580f6ff0 0x7ff258107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:09.362 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.361+0000 7ff25e61a700 1 -- 172.21.15.27:0/23422124 >> 172.21.15.27:0/23422124 conn(0x7ff2580f1f00 msgr2=0x7ff2580fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:58:09.362 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.361+0000 7ff25e61a700 1 -- 172.21.15.27:0/23422124 shutdown_connections 2021-11-03T17:58:09.363 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.361+0000 7ff25e61a700 1 -- 172.21.15.27:0/23422124 wait complete. 2021-11-03T17:58:09.457 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.459+0000 7f1f3f3c0700 1 -- 172.21.15.27:0/3744454784 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f380f66e0 msgr2=0x7f1f380f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:58:09.457 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.459+0000 7f1f3f3c0700 1 --2- 172.21.15.27:0/3744454784 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f380f66e0 0x7f1f380f6b40 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7f1f28004660 tx=0x7f1f28009b30).stop 2021-11-03T17:58:09.458 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.459+0000 7f1f3f3c0700 1 -- 172.21.15.27:0/3744454784 shutdown_connections 2021-11-03T17:58:09.458 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.459+0000 7f1f3f3c0700 1 --2- 172.21.15.27:0/3744454784 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f380f66e0 0x7f1f380f6b40 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:09.458 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.459+0000 7f1f3f3c0700 1 --2- 172.21.15.27:0/3744454784 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f380f5440 0x7f1f380f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:09.458 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.459+0000 7f1f3f3c0700 1 -- 172.21.15.27:0/3744454784 >> 172.21.15.27:0/3744454784 conn(0x7f1f380f0ba0 msgr2=0x7f1f380f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:58:09.459 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.460+0000 7f1f3f3c0700 1 -- 172.21.15.27:0/3744454784 shutdown_connections 2021-11-03T17:58:09.459 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.460+0000 7f1f3f3c0700 1 -- 172.21.15.27:0/3744454784 wait complete. 2021-11-03T17:58:09.459 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.460+0000 7f1f3f3c0700 1 Processor -- start 2021-11-03T17:58:09.459 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.460+0000 7f1f3f3c0700 1 -- start start 2021-11-03T17:58:09.459 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.460+0000 7f1f3f3c0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f380f5440 0x7f1f38105d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:58:09.460 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.460+0000 7f1f3f3c0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f380f66e0 0x7f1f38106220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:58:09.460 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.460+0000 7f1f3f3c0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f381067d0 con 0x7f1f380f5440 2021-11-03T17:58:09.460 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.460+0000 7f1f3f3c0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f38106fd0 con 0x7f1f380f66e0 2021-11-03T17:58:09.460 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.460+0000 7f1f3e3be700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f380f5440 0x7f1f38105d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:58:09.461 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.460+0000 7f1f3e3be700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f380f5440 0x7f1f38105d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60422/0 (socket says 172.21.15.27:60422) 2021-11-03T17:58:09.461 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.460+0000 7f1f3e3be700 1 -- 172.21.15.27:0/520201251 learned_addr learned my addr 172.21.15.27:0/520201251 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:58:09.461 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.461+0000 7f1f3dbbd700 1 --2- 172.21.15.27:0/520201251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f380f66e0 0x7f1f38106220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:58:09.461 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.461+0000 7f1f3e3be700 1 -- 172.21.15.27:0/520201251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f380f66e0 msgr2=0x7f1f38106220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:58:09.462 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.461+0000 7f1f3e3be700 1 --2- 172.21.15.27:0/520201251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f380f66e0 0x7f1f38106220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:09.462 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.461+0000 7f1f3e3be700 1 -- 172.21.15.27:0/520201251 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f28005040 con 0x7f1f380f5440 2021-11-03T17:58:09.462 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.461+0000 7f1f3e3be700 1 --2- 172.21.15.27:0/520201251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f380f5440 0x7f1f38105d30 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f1f3400e9a0 tx=0x7f1f34007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:58:09.462 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.461+0000 7f1f2effd700 1 -- 172.21.15.27:0/520201251 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f3400c980 con 0x7f1f380f5440 2021-11-03T17:58:09.462 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.461+0000 7f1f2effd700 1 -- 172.21.15.27:0/520201251 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f1f34011470 con 0x7f1f380f5440 2021-11-03T17:58:09.463 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.461+0000 7f1f3f3c0700 1 -- 172.21.15.27:0/520201251 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f38107250 con 0x7f1f380f5440 2021-11-03T17:58:09.463 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.461+0000 7f1f2effd700 1 -- 172.21.15.27:0/520201251 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f34010630 con 0x7f1f380f5440 2021-11-03T17:58:09.463 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.461+0000 7f1f3f3c0700 1 -- 172.21.15.27:0/520201251 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f381077c0 con 0x7f1f380f5440 2021-11-03T17:58:09.464 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.462+0000 7f1f3f3c0700 1 -- 172.21.15.27:0/520201251 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f38059100 con 0x7f1f380f5440 2021-11-03T17:58:09.464 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.463+0000 7f1f2effd700 1 -- 172.21.15.27:0/520201251 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f1f340115e0 con 0x7f1f380f5440 2021-11-03T17:58:09.465 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.463+0000 7f1f2effd700 1 --2- 172.21.15.27:0/520201251 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1f240640e0 0x7f1f240665a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:58:09.465 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.463+0000 7f1f2effd700 1 -- 172.21.15.27:0/520201251 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f1f34083800 con 0x7f1f380f5440 2021-11-03T17:58:09.465 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.463+0000 7f1f3dbbd700 1 --2- 172.21.15.27:0/520201251 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1f240640e0 0x7f1f240665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:58:09.466 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.464+0000 7f1f3dbbd700 1 --2- 172.21.15.27:0/520201251 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1f240640e0 0x7f1f240665a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f1f28004630 tx=0x7f1f28009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:58:09.466 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.467+0000 7f1f2effd700 1 -- 172.21.15.27:0/520201251 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f1f340559b0 con 0x7f1f380f5440 2021-11-03T17:58:09.692 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.696+0000 7f1f3f3c0700 1 -- 172.21.15.27:0/520201251 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1f380ff610 con 0x7f1f380f5440 2021-11-03T17:58:09.692 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.696+0000 7f1f2effd700 1 -- 172.21.15.27:0/520201251 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f1f340555a0 con 0x7f1f380f5440 2021-11-03T17:58:09.693 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:58:09.693 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:58:09.693 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:58:09.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:58:09.694 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:58:09.694 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:58:09.694 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:58:09.695 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:58:09.695 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:58:09.695 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:58:09.695 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:58:09.695 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:58:09.696 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:58:09.696 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:58:09.696 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:58:09.696 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:58:09.697 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:58:09.697 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:58:09.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.698+0000 7f1f3f3c0700 1 -- 172.21.15.27:0/520201251 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1f240640e0 msgr2=0x7f1f240665a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:58:09.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.698+0000 7f1f3f3c0700 1 --2- 172.21.15.27:0/520201251 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1f240640e0 0x7f1f240665a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 rx=0x7f1f28004630 tx=0x7f1f28009b30).stop 2021-11-03T17:58:09.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.698+0000 7f1f3f3c0700 1 -- 172.21.15.27:0/520201251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f380f5440 msgr2=0x7f1f38105d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:58:09.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.698+0000 7f1f3f3c0700 1 --2- 172.21.15.27:0/520201251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f380f5440 0x7f1f38105d30 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f1f3400e9a0 tx=0x7f1f34007e20).stop 2021-11-03T17:58:09.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.698+0000 7f1f3f3c0700 1 -- 172.21.15.27:0/520201251 shutdown_connections 2021-11-03T17:58:09.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.698+0000 7f1f3f3c0700 1 --2- 172.21.15.27:0/520201251 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1f240640e0 0x7f1f240665a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:09.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.698+0000 7f1f3f3c0700 1 --2- 172.21.15.27:0/520201251 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f380f5440 0x7f1f38105d30 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:09.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.698+0000 7f1f3f3c0700 1 --2- 172.21.15.27:0/520201251 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f380f66e0 0x7f1f38106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:09.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.698+0000 7f1f3f3c0700 1 -- 172.21.15.27:0/520201251 >> 172.21.15.27:0/520201251 conn(0x7f1f380f0ba0 msgr2=0x7f1f380fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:58:09.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.698+0000 7f1f3f3c0700 1 -- 172.21.15.27:0/520201251 shutdown_connections 2021-11-03T17:58:09.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.698+0000 7f1f3f3c0700 1 -- 172.21.15.27:0/520201251 wait complete. 2021-11-03T17:58:09.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.798+0000 7ff10f59e700 1 -- 172.21.15.27:0/1605748871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1100f66e0 msgr2=0x7ff1100f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:58:09.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.798+0000 7ff10f59e700 1 --2- 172.21.15.27:0/1605748871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1100f66e0 0x7ff1100f6b40 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 rx=0x7ff104004660 tx=0x7ff104009b30).stop 2021-11-03T17:58:09.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.798+0000 7ff10f59e700 1 -- 172.21.15.27:0/1605748871 shutdown_connections 2021-11-03T17:58:09.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.798+0000 7ff10f59e700 1 --2- 172.21.15.27:0/1605748871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1100f66e0 0x7ff1100f6b40 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:09.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.798+0000 7ff10f59e700 1 --2- 172.21.15.27:0/1605748871 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1100f5440 0x7ff1100f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:58:09.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.798+0000 7ff10f59e700 1 -- 172.21.15.27:0/1605748871 >> 172.21.15.27:0/1605748871 conn(0x7ff1100f0ba0 msgr2=0x7ff1100f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:58:09.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.798+0000 7ff10f59e700 1 -- 172.21.15.27:0/1605748871 shutdown_connections 2021-11-03T17:58:09.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.798+0000 7ff10f59e700 1 -- 172.21.15.27:0/1605748871 wait complete. 2021-11-03T17:58:09.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.799+0000 7ff10f59e700 1 Processor -- start 2021-11-03T17:58:09.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.799+0000 7ff10f59e700 1 -- start start 2021-11-03T17:58:09.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.799+0000 7ff10f59e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1100f5440 0x7ff1100fd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:58:09.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.799+0000 7ff10f59e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1100f66e0 0x7ff1100fba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:58:09.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.799+0000 7ff10f59e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1100fc030 con 0x7ff1100f66e0 2021-11-03T17:58:09.800 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.799+0000 7ff10f59e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1100fc170 con 0x7ff1100f5440 2021-11-03T17:58:09.800 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.799+0000 7ff10dd9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1100f66e0 0x7ff1100fba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:58:09.800 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.799+0000 7ff10dd9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1100f66e0 0x7ff1100fba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60434/0 (socket says 172.21.15.27:60434) 2021-11-03T17:58:09.800 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.799+0000 7ff10dd9b700 1 -- 172.21.15.27:0/1604231686 learned_addr learned my addr 172.21.15.27:0/1604231686 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:58:09.801 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.799+0000 7ff10e59c700 1 --2- 172.21.15.27:0/1604231686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1100f5440 0x7ff1100fd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:58:09.801 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.800+0000 7ff10dd9b700 1 -- 172.21.15.27:0/1604231686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1100f5440 msgr2=0x7ff1100fd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:58:09.801 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.800+0000 7ff10dd9b700 1 --2- 172.21.15.27:0/1604231686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1100f5440 0x7ff1100fd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:09.801 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.800+0000 7ff10dd9b700 1 -- 172.21.15.27:0/1604231686 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff104005040 con 0x7ff1100f66e0 2021-11-03T17:58:09.801 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.800+0000 7ff10e59c700 1 --2- 172.21.15.27:0/1604231686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1100f5440 0x7ff1100fd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:58:09.802 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.800+0000 7ff10dd9b700 1 --2- 172.21.15.27:0/1604231686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1100f66e0 0x7ff1100fba40 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7ff104009990 tx=0x7ff104009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:58:09.802 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.800+0000 7ff0feffd700 1 -- 172.21.15.27:0/1604231686 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff104018070 con 0x7ff1100f66e0 2021-11-03T17:58:09.802 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.800+0000 7ff0feffd700 1 -- 172.21.15.27:0/1604231686 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7ff104007ca0 con 0x7ff1100f66e0 2021-11-03T17:58:09.802 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.800+0000 7ff0feffd700 1 -- 172.21.15.27:0/1604231686 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff10401c620 con 0x7ff1100f66e0 2021-11-03T17:58:09.803 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.800+0000 7ff10f59e700 1 -- 172.21.15.27:0/1604231686 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1100fc3f0 con 0x7ff1100f66e0 2021-11-03T17:58:09.803 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.800+0000 7ff10f59e700 1 -- 172.21.15.27:0/1604231686 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff11010edb0 con 0x7ff1100f66e0 2021-11-03T17:58:09.804 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.801+0000 7ff10f59e700 1 -- 172.21.15.27:0/1604231686 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff110045ad0 con 0x7ff1100f66e0 2021-11-03T17:58:09.804 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.801+0000 7ff0feffd700 1 -- 172.21.15.27:0/1604231686 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7ff10401d960 con 0x7ff1100f66e0 2021-11-03T17:58:09.805 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.802+0000 7ff0feffd700 1 --2- 172.21.15.27:0/1604231686 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff0f8064100 0x7ff0f80665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:58:09.805 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.802+0000 7ff0feffd700 1 -- 172.21.15.27:0/1604231686 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7ff1040824d0 con 0x7ff1100f66e0 2021-11-03T17:58:09.805 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.802+0000 7ff10e59c700 1 --2- 172.21.15.27:0/1604231686 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff0f8064100 0x7ff0f80665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:58:09.805 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.802+0000 7ff10e59c700 1 --2- 172.21.15.27:0/1604231686 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff0f8064100 0x7ff0f80665c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7ff1100fd350 tx=0x7ff100008040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:58:09.806 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:09.806+0000 7ff0feffd700 1 -- 172.21.15.27:0/1604231686 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7ff104054890 con 0x7ff1100f66e0 2021-11-03T17:58:10.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:10.009+0000 7ff10f59e700 1 -- 172.21.15.27:0/1604231686 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff1100feeb0 con 0x7ff1100f66e0 2021-11-03T17:58:10.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:10.009+0000 7ff0feffd700 1 -- 172.21.15.27:0/1604231686 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7ff104054480 con 0x7ff1100f66e0 2021-11-03T17:58:10.006 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:58:10.007 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:58:10.007 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:58:10.007 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:58:10.007 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:58:10.008 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:58:10.008 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:58:10.008 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:58:10.008 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:58:10.009 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:58:10.009 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:58:10.009 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:58:10.010 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:58:10.010 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:58:10.010 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:58:10.010 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:58:10.011 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:58:10.011 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:58:10.011 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:58:10.011 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:58:10.011 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:58:10.012 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:58:10.012 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:58:10.012 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:58:10.012 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:58:10.013 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:58:10.013 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:58:10.013 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:58:10.013 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:58:10.014 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:58:10.014 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:58:10.014 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:58:10.015 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:58:10.015 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:58:10.015 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:58:10.015 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:58:10.016 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:58:10.016 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:58:10.016 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:58:10.017 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:10.011+0000 7ff10f59e700 1 -- 172.21.15.27:0/1604231686 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff0f8064100 msgr2=0x7ff0f80665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:58:10.018 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:10.011+0000 7ff10f59e700 1 --2- 172.21.15.27:0/1604231686 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff0f8064100 0x7ff0f80665c0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7ff1100fd350 tx=0x7ff100008040).stop 2021-11-03T17:58:10.018 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:10.011+0000 7ff10f59e700 1 -- 172.21.15.27:0/1604231686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1100f66e0 msgr2=0x7ff1100fba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:58:10.018 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:10.011+0000 7ff10f59e700 1 --2- 172.21.15.27:0/1604231686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1100f66e0 0x7ff1100fba40 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7ff104009990 tx=0x7ff104009b30).stop 2021-11-03T17:58:10.018 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:10.011+0000 7ff10f59e700 1 -- 172.21.15.27:0/1604231686 shutdown_connections 2021-11-03T17:58:10.019 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:10.011+0000 7ff10f59e700 1 --2- 172.21.15.27:0/1604231686 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff0f8064100 0x7ff0f80665c0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:10.019 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:10.011+0000 7ff10f59e700 1 --2- 172.21.15.27:0/1604231686 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1100f5440 0x7ff1100fd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:10.019 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:10.011+0000 7ff10f59e700 1 --2- 172.21.15.27:0/1604231686 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1100f66e0 0x7ff1100fba40 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:10.019 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:10.011+0000 7ff10f59e700 1 -- 172.21.15.27:0/1604231686 >> 172.21.15.27:0/1604231686 conn(0x7ff1100f0ba0 msgr2=0x7ff1100fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:58:10.020 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:10.011+0000 7ff10f59e700 1 -- 172.21.15.27:0/1604231686 shutdown_connections 2021-11-03T17:58:10.020 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:10.011+0000 7ff10f59e700 1 -- 172.21.15.27:0/1604231686 wait complete. 2021-11-03T17:58:10.020 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:58:10.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:10 smithi115 ceph-mon[65453]: pgmap v1305: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:10.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:10 smithi115 ceph-mon[65453]: from='client.35448 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:58:10.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:10 smithi115 ceph-mon[65453]: from='client.35452 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:58:10.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2532.420626 secs 2021-11-03T17:58:10.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:10 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/520201251' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:10.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:10 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1604231686' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:58:10.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:10 smithi027 ceph-mon[97244]: pgmap v1305: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:10.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:10 smithi027 ceph-mon[97244]: from='client.35448 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:58:10.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:10 smithi027 ceph-mon[97244]: from='client.35452 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:58:10.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2532.420626 secs 2021-11-03T17:58:10.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:10 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/520201251' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:10.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:10 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1604231686' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:58:12.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:12 smithi115 ceph-mon[65453]: pgmap v1306: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:12.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:12 smithi027 ceph-mon[97244]: pgmap v1306: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:14.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:14 smithi115 ceph-mon[65453]: pgmap v1307: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:14.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:14 smithi027 ceph-mon[97244]: pgmap v1307: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:15.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2537.420774 secs 2021-11-03T17:58:15.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2537.420774 secs 2021-11-03T17:58:16.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:16 smithi115 ceph-mon[65453]: pgmap v1308: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:16.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:16 smithi027 ceph-mon[97244]: pgmap v1308: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:17.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:58:17.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:58:17.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:58:17.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:17.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:17.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:17.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:17.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:58:17.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:58:17.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:58:17.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:17.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:17.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:17.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:18.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:18 smithi115 ceph-mon[65453]: pgmap v1309: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:18.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:18 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:58:18.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:18 smithi027 ceph-mon[97244]: pgmap v1309: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:18.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:18 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:58:20.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:20 smithi115 ceph-mon[65453]: pgmap v1310: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:20.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2542.420913 secs 2021-11-03T17:58:20.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:20 smithi027 ceph-mon[97244]: pgmap v1310: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:20.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2542.420913 secs 2021-11-03T17:58:22.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:22 smithi115 ceph-mon[65453]: pgmap v1311: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:22.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:22 smithi027 ceph-mon[97244]: pgmap v1311: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:24.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:24 smithi115 ceph-mon[65453]: pgmap v1312: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:24.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:24 smithi027 ceph-mon[97244]: pgmap v1312: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:25.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2547.421043 secs 2021-11-03T17:58:25.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2547.421043 secs 2021-11-03T17:58:26.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:26 smithi115 ceph-mon[65453]: pgmap v1313: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:26.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:26 smithi027 ceph-mon[97244]: pgmap v1313: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:27.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:58:27.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:58:27.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:58:27.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:58:27.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:27.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:27.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:27.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:27.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:58:27.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:58:27.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:58:27.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:58:27.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:27.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:27.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:27.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:28.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:28 smithi115 ceph-mon[65453]: pgmap v1314: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:28.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:28 smithi115 ceph-mon[65453]: poll took 50140.740 ms: 1 events 2021-11-03T17:58:28.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:28 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:58:28.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:28 smithi027 ceph-mon[97244]: pgmap v1314: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:28.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:28 smithi027 ceph-mon[97244]: poll took 50140.740 ms: 1 events 2021-11-03T17:58:28.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:28 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:58:30.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:30 smithi115 ceph-mon[65453]: pgmap v1315: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:30.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2552.421129 secs 2021-11-03T17:58:30.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:30 smithi027 ceph-mon[97244]: pgmap v1315: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:30.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2552.421129 secs 2021-11-03T17:58:32.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:32 smithi115 ceph-mon[65453]: pgmap v1316: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:32.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:32 smithi027 ceph-mon[97244]: pgmap v1316: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:34.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:34 smithi115 ceph-mon[65453]: pgmap v1317: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:34.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:34 smithi027 ceph-mon[97244]: pgmap v1317: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:35.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2557.421236 secs 2021-11-03T17:58:35.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2557.421236 secs 2021-11-03T17:58:36.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:36 smithi115 ceph-mon[65453]: pgmap v1318: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:36.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:36 smithi027 ceph-mon[97244]: pgmap v1318: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:37.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:58:37.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:58:38.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:38 smithi115 ceph-mon[65453]: pgmap v1319: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:38.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:38 smithi115 ceph-mon[65453]: poll took 10033.998 ms: 1 events 2021-11-03T17:58:38.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:58:38.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:58:38.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:38.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:38.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:38.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:38.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:38 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:58:38.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:38 smithi027 ceph-mon[97244]: pgmap v1319: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:38.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:38 smithi027 ceph-mon[97244]: poll took 10033.998 ms: 1 events 2021-11-03T17:58:38.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:58:38.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:58:38.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:38.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:38.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:38.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:38.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:38 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:58:40.124 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.126+0000 7f547e078700 1 -- 172.21.15.27:0/1793081630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54780f5ef0 msgr2=0x7f54780f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:58:40.125 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.127+0000 7f547e078700 1 --2- 172.21.15.27:0/1793081630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54780f5ef0 0x7f54780f6920 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7f5460004660 tx=0x7f5460009b30).stop 2021-11-03T17:58:40.125 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.127+0000 7f547e078700 1 -- 172.21.15.27:0/1793081630 shutdown_connections 2021-11-03T17:58:40.125 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.127+0000 7f547e078700 1 --2- 172.21.15.27:0/1793081630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54780f5ef0 0x7f54780f6920 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:40.126 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.127+0000 7f547e078700 1 --2- 172.21.15.27:0/1793081630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54780f55e0 0x7f54780f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:40.126 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.127+0000 7f547e078700 1 -- 172.21.15.27:0/1793081630 >> 172.21.15.27:0/1793081630 conn(0x7f54780f0de0 msgr2=0x7f54780f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:58:40.126 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.127+0000 7f547e078700 1 -- 172.21.15.27:0/1793081630 shutdown_connections 2021-11-03T17:58:40.126 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.127+0000 7f547e078700 1 -- 172.21.15.27:0/1793081630 wait complete. 2021-11-03T17:58:40.126 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.127+0000 7f547e078700 1 Processor -- start 2021-11-03T17:58:40.127 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.128+0000 7f547e078700 1 -- start start 2021-11-03T17:58:40.127 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.128+0000 7f547e078700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54780f55e0 0x7f5478106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:58:40.127 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.128+0000 7f547e078700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54780f5ef0 0x7f5478107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:58:40.127 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.128+0000 7f547e078700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5478107a00 con 0x7f54780f55e0 2021-11-03T17:58:40.128 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.128+0000 7f547e078700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5478107b40 con 0x7f54780f5ef0 2021-11-03T17:58:40.128 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.128+0000 7f54777fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54780f55e0 0x7f5478106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:58:40.128 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.128+0000 7f54777fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54780f55e0 0x7f5478106f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60454/0 (socket says 172.21.15.27:60454) 2021-11-03T17:58:40.128 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.128+0000 7f54777fe700 1 -- 172.21.15.27:0/867744235 learned_addr learned my addr 172.21.15.27:0/867744235 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:58:40.128 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.128+0000 7f546effd700 1 --2- 172.21.15.27:0/867744235 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54780f5ef0 0x7f5478107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:58:40.129 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.128+0000 7f54777fe700 1 -- 172.21.15.27:0/867744235 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54780f5ef0 msgr2=0x7f5478107450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:58:40.129 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.128+0000 7f54777fe700 1 --2- 172.21.15.27:0/867744235 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54780f5ef0 0x7f5478107450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:40.129 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.128+0000 7f54777fe700 1 -- 172.21.15.27:0/867744235 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5460005040 con 0x7f54780f55e0 2021-11-03T17:58:40.129 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.128+0000 7f54777fe700 1 --2- 172.21.15.27:0/867744235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54780f55e0 0x7f5478106f60 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7f546800c530 tx=0x7f5468007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:58:40.130 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.129+0000 7f5474ff9700 1 -- 172.21.15.27:0/867744235 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f546800f650 con 0x7f54780f55e0 2021-11-03T17:58:40.130 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.129+0000 7f5474ff9700 1 -- 172.21.15.27:0/867744235 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f546800fc60 con 0x7f54780f55e0 2021-11-03T17:58:40.130 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.129+0000 7f547e078700 1 -- 172.21.15.27:0/867744235 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54781084e0 con 0x7f54780f55e0 2021-11-03T17:58:40.131 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.129+0000 7f5474ff9700 1 -- 172.21.15.27:0/867744235 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5468012c20 con 0x7f54780f55e0 2021-11-03T17:58:40.131 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.129+0000 7f547e078700 1 -- 172.21.15.27:0/867744235 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5478108aa0 con 0x7f54780f55e0 2021-11-03T17:58:40.132 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.130+0000 7f547e078700 1 -- 172.21.15.27:0/867744235 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5478045ad0 con 0x7f54780f55e0 2021-11-03T17:58:40.132 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.132+0000 7f5474ff9700 1 -- 172.21.15.27:0/867744235 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f5468010070 con 0x7f54780f55e0 2021-11-03T17:58:40.132 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.132+0000 7f5474ff9700 1 --2- 172.21.15.27:0/867744235 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f54640640b0 0x7f5464066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:58:40.132 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.132+0000 7f5474ff9700 1 -- 172.21.15.27:0/867744235 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f5468080780 con 0x7f54780f55e0 2021-11-03T17:58:40.133 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.132+0000 7f546effd700 1 --2- 172.21.15.27:0/867744235 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f54640640b0 0x7f5464066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:58:40.133 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.133+0000 7f546effd700 1 --2- 172.21.15.27:0/867744235 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f54640640b0 0x7f5464066570 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f5460004630 tx=0x7f5460009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:58:40.133 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.137+0000 7f5474ff9700 1 -- 172.21.15.27:0/867744235 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f54680528d0 con 0x7f54780f55e0 2021-11-03T17:58:40.312 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.316+0000 7f547e078700 1 -- 172.21.15.27:0/867744235 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f54780fad60 con 0x7f54640640b0 2021-11-03T17:58:40.313 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.317+0000 7f5474ff9700 1 -- 172.21.15.27:0/867744235 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f54780fad60 con 0x7f54640640b0 2021-11-03T17:58:40.316 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.319+0000 7f547e078700 1 -- 172.21.15.27:0/867744235 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f54640640b0 msgr2=0x7f5464066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:58:40.316 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.319+0000 7f547e078700 1 --2- 172.21.15.27:0/867744235 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f54640640b0 0x7f5464066570 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 rx=0x7f5460004630 tx=0x7f5460009b30).stop 2021-11-03T17:58:40.316 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.319+0000 7f547e078700 1 -- 172.21.15.27:0/867744235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54780f55e0 msgr2=0x7f5478106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:58:40.316 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.319+0000 7f547e078700 1 --2- 172.21.15.27:0/867744235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54780f55e0 0x7f5478106f60 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7f546800c530 tx=0x7f5468007e20).stop 2021-11-03T17:58:40.317 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.319+0000 7f547e078700 1 -- 172.21.15.27:0/867744235 shutdown_connections 2021-11-03T17:58:40.317 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.319+0000 7f547e078700 1 --2- 172.21.15.27:0/867744235 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f54640640b0 0x7f5464066570 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:40.317 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.319+0000 7f547e078700 1 --2- 172.21.15.27:0/867744235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54780f55e0 0x7f5478106f60 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:40.317 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.319+0000 7f547e078700 1 --2- 172.21.15.27:0/867744235 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54780f5ef0 0x7f5478107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:40.317 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.319+0000 7f547e078700 1 -- 172.21.15.27:0/867744235 >> 172.21.15.27:0/867744235 conn(0x7f54780f0de0 msgr2=0x7f547805ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:58:40.318 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.320+0000 7f547e078700 1 -- 172.21.15.27:0/867744235 shutdown_connections 2021-11-03T17:58:40.318 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.320+0000 7f547e078700 1 -- 172.21.15.27:0/867744235 wait complete. 2021-11-03T17:58:40.325 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:58:40.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.420+0000 7f9afbcf8700 1 -- 172.21.15.27:0/3800064160 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9af40f66e0 msgr2=0x7f9af40f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:58:40.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.420+0000 7f9afbcf8700 1 --2- 172.21.15.27:0/3800064160 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9af40f66e0 0x7f9af40f6b00 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7f9ae4004660 tx=0x7f9ae4009b30).stop 2021-11-03T17:58:40.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.420+0000 7f9afbcf8700 1 -- 172.21.15.27:0/3800064160 shutdown_connections 2021-11-03T17:58:40.419 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.420+0000 7f9afbcf8700 1 --2- 172.21.15.27:0/3800064160 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9af40f7980 0x7f9af40f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T17:58:40.419 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.420+0000 7f9afbcf8700 1 --2- 172.21.15.27:0/3800064160 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9af40f66e0 0x7f9af40f6b00 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:40.419 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.420+0000 7f9afbcf8700 1 -- 172.21.15.27:0/3800064160 >> 172.21.15.27:0/3800064160 conn(0x7f9af40f1e40 msgr2=0x7f9af40f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:58:40.419 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.420+0000 7f9afbcf8700 1 -- 172.21.15.27:0/3800064160 shutdown_connections 2021-11-03T17:58:40.420 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.420+0000 7f9afbcf8700 1 -- 172.21.15.27:0/3800064160 wait complete. 2021-11-03T17:58:40.420 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.420+0000 7f9afbcf8700 1 Processor -- start 2021-11-03T17:58:40.420 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.420+0000 7f9afbcf8700 1 -- start start 2021-11-03T17:58:40.420 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.421+0000 7f9afbcf8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9af40f66e0 0x7f9af406b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:58:40.420 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.421+0000 7f9afbcf8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9af40f7980 0x7f9af4069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:58:40.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.421+0000 7f9afbcf8700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af406a410 con 0x7f9af40f7980 2021-11-03T17:58:40.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.421+0000 7f9afbcf8700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af406a550 con 0x7f9af40f66e0 2021-11-03T17:58:40.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.421+0000 7f9af9293700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9af40f7980 0x7f9af4069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:58:40.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.421+0000 7f9af9293700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9af40f7980 0x7f9af4069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60466/0 (socket says 172.21.15.27:60466) 2021-11-03T17:58:40.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.421+0000 7f9af9293700 1 -- 172.21.15.27:0/3676024568 learned_addr learned my addr 172.21.15.27:0/3676024568 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:58:40.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.421+0000 7f9af9a94700 1 --2- 172.21.15.27:0/3676024568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9af40f66e0 0x7f9af406b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:58:40.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.421+0000 7f9af9293700 1 -- 172.21.15.27:0/3676024568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9af40f66e0 msgr2=0x7f9af406b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:58:40.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.421+0000 7f9af9293700 1 --2- 172.21.15.27:0/3676024568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9af40f66e0 0x7f9af406b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:40.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.421+0000 7f9af9293700 1 -- 172.21.15.27:0/3676024568 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae4005040 con 0x7f9af40f7980 2021-11-03T17:58:40.423 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.421+0000 7f9af9293700 1 --2- 172.21.15.27:0/3676024568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9af40f7980 0x7f9af4069e20 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7f9af000b410 tx=0x7f9af0006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:58:40.423 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.421+0000 7f9aea7fc700 1 -- 172.21.15.27:0/3676024568 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9af00125f0 con 0x7f9af40f7980 2021-11-03T17:58:40.423 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.421+0000 7f9aea7fc700 1 -- 172.21.15.27:0/3676024568 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f9af000be30 con 0x7f9af40f7980 2021-11-03T17:58:40.423 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.421+0000 7f9aea7fc700 1 -- 172.21.15.27:0/3676024568 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9af0002a70 con 0x7f9af40f7980 2021-11-03T17:58:40.424 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.421+0000 7f9afbcf8700 1 -- 172.21.15.27:0/3676024568 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9af406a830 con 0x7f9af40f7980 2021-11-03T17:58:40.424 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.421+0000 7f9afbcf8700 1 -- 172.21.15.27:0/3676024568 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9af410efe0 con 0x7f9af40f7980 2021-11-03T17:58:40.425 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.422+0000 7f9afbcf8700 1 -- 172.21.15.27:0/3676024568 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9af4045ad0 con 0x7f9af40f7980 2021-11-03T17:58:40.426 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.423+0000 7f9aea7fc700 1 -- 172.21.15.27:0/3676024568 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f9af000f030 con 0x7f9af40f7980 2021-11-03T17:58:40.426 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.423+0000 7f9aea7fc700 1 --2- 172.21.15.27:0/3676024568 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9ae00640b0 0x7f9ae0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:58:40.426 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.423+0000 7f9aea7fc700 1 -- 172.21.15.27:0/3676024568 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f9af0080c50 con 0x7f9af40f7980 2021-11-03T17:58:40.426 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.423+0000 7f9af9a94700 1 --2- 172.21.15.27:0/3676024568 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9ae00640b0 0x7f9ae0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:58:40.427 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.424+0000 7f9af9a94700 1 --2- 172.21.15.27:0/3676024568 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9ae00640b0 0x7f9ae0066570 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f9ae4013690 tx=0x7f9ae4009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:58:40.427 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.427+0000 7f9aea7fc700 1 -- 172.21.15.27:0/3676024568 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f9af0052f50 con 0x7f9af40f7980 2021-11-03T17:58:40.600 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.603+0000 7f9afbcf8700 1 -- 172.21.15.27:0/3676024568 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9af4057f10 con 0x7f9ae00640b0 2021-11-03T17:58:40.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.632+0000 7f9aea7fc700 1 -- 172.21.15.27:0/3676024568 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f9af4057f10 con 0x7f9ae00640b0 2021-11-03T17:58:40.629 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:58:40.629 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (50m) 9m ago 51m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:58:40.630 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (41m) 9m ago 51m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:58:40.630 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (41m) 8m ago 50m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:58:40.630 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (50m) 9m ago 51m 41.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:58:40.630 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (43m) 9m ago 46m 23.8M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:58:40.631 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (46m) 9m ago 46m 3844M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:58:40.631 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (42m) 8m ago 46m 23.2M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:58:40.631 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (44m) 8m ago 46m 24.9M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:58:40.631 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (43m) 9m ago 52m 535M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:58:40.631 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (43m) 8m ago 50m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:58:40.632 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (42m) 9m ago 52m 198M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:58:40.632 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (42m) 8m ago 50m 95.6M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:58:40.632 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (51m) 9m ago 51m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:58:40.632 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (50m) 8m ago 50m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:58:40.633 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (41m) 9m ago 49m 331M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:58:40.633 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (41m) 9m ago 49m 361M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:58:40.633 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (40m) 9m ago 49m 266M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:58:40.633 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (40m) 8m ago 48m 379M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:58:40.633 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (39m) 8m ago 48m 304M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:58:40.634 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (39m) 8m ago 48m 307M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:58:40.634 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (49m) 9m ago 50m 52.9M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:58:40.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.634+0000 7f9afbcf8700 1 -- 172.21.15.27:0/3676024568 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9ae00640b0 msgr2=0x7f9ae0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:58:40.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.634+0000 7f9afbcf8700 1 --2- 172.21.15.27:0/3676024568 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9ae00640b0 0x7f9ae0066570 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 rx=0x7f9ae4013690 tx=0x7f9ae4009b30).stop 2021-11-03T17:58:40.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.634+0000 7f9afbcf8700 1 -- 172.21.15.27:0/3676024568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9af40f7980 msgr2=0x7f9af4069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:58:40.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.634+0000 7f9afbcf8700 1 --2- 172.21.15.27:0/3676024568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9af40f7980 0x7f9af4069e20 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7f9af000b410 tx=0x7f9af0006ef0).stop 2021-11-03T17:58:40.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.634+0000 7f9afbcf8700 1 -- 172.21.15.27:0/3676024568 shutdown_connections 2021-11-03T17:58:40.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.634+0000 7f9afbcf8700 1 --2- 172.21.15.27:0/3676024568 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9ae00640b0 0x7f9ae0066570 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:40.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.634+0000 7f9afbcf8700 1 --2- 172.21.15.27:0/3676024568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9af40f66e0 0x7f9af406b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:40.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.634+0000 7f9afbcf8700 1 --2- 172.21.15.27:0/3676024568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9af40f7980 0x7f9af4069e20 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:40.637 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.634+0000 7f9afbcf8700 1 -- 172.21.15.27:0/3676024568 >> 172.21.15.27:0/3676024568 conn(0x7f9af40f1e40 msgr2=0x7f9af40fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:58:40.637 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.634+0000 7f9afbcf8700 1 -- 172.21.15.27:0/3676024568 shutdown_connections 2021-11-03T17:58:40.637 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.634+0000 7f9afbcf8700 1 -- 172.21.15.27:0/3676024568 wait complete. 2021-11-03T17:58:40.732 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.734+0000 7f87b810d700 1 -- 172.21.15.27:0/1862806309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87b00f55c0 msgr2=0x7f87b00f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:58:40.733 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.734+0000 7f87b810d700 1 --2- 172.21.15.27:0/1862806309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87b00f55c0 0x7f87b00f59e0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7f87a80099e0 tx=0x7f87a8004d10).stop 2021-11-03T17:58:40.733 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.734+0000 7f87b810d700 1 -- 172.21.15.27:0/1862806309 shutdown_connections 2021-11-03T17:58:40.733 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.734+0000 7f87b810d700 1 --2- 172.21.15.27:0/1862806309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87b00f6860 0x7f87b00f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:40.733 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.734+0000 7f87b810d700 1 --2- 172.21.15.27:0/1862806309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87b00f55c0 0x7f87b00f59e0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:40.734 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.734+0000 7f87b810d700 1 -- 172.21.15.27:0/1862806309 >> 172.21.15.27:0/1862806309 conn(0x7f87b00f0d40 msgr2=0x7f87b00f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:58:40.734 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.734+0000 7f87b810d700 1 -- 172.21.15.27:0/1862806309 shutdown_connections 2021-11-03T17:58:40.734 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.734+0000 7f87b810d700 1 -- 172.21.15.27:0/1862806309 wait complete. 2021-11-03T17:58:40.735 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.734+0000 7f87b810d700 1 Processor -- start 2021-11-03T17:58:40.735 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.734+0000 7f87b810d700 1 -- start start 2021-11-03T17:58:40.735 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.735+0000 7f87b810d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87b00f55c0 0x7f87b0109d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:58:40.735 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.735+0000 7f87b810d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87b00f6860 0x7f87b010c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:58:40.736 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.735+0000 7f87b810d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b0060420 con 0x7f87b00f55c0 2021-11-03T17:58:40.736 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.735+0000 7f87b810d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87b0060560 con 0x7f87b00f6860 2021-11-03T17:58:40.736 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.735+0000 7f87b5ea9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87b00f55c0 0x7f87b0109d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:58:40.736 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.735+0000 7f87b5ea9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87b00f55c0 0x7f87b0109d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60474/0 (socket says 172.21.15.27:60474) 2021-11-03T17:58:40.737 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.735+0000 7f87b5ea9700 1 -- 172.21.15.27:0/4267537668 learned_addr learned my addr 172.21.15.27:0/4267537668 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:58:40.737 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.735+0000 7f87b56a8700 1 --2- 172.21.15.27:0/4267537668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87b00f6860 0x7f87b010c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:58:40.737 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.735+0000 7f87b5ea9700 1 -- 172.21.15.27:0/4267537668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87b00f6860 msgr2=0x7f87b010c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:58:40.737 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.735+0000 7f87b5ea9700 1 --2- 172.21.15.27:0/4267537668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87b00f6860 0x7f87b010c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:40.738 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.735+0000 7f87b5ea9700 1 -- 172.21.15.27:0/4267537668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a8009640 con 0x7f87b00f55c0 2021-11-03T17:58:40.738 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.735+0000 7f87b56a8700 1 --2- 172.21.15.27:0/4267537668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87b00f6860 0x7f87b010c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:58:40.738 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.735+0000 7f87b5ea9700 1 --2- 172.21.15.27:0/4267537668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87b00f55c0 0x7f87b0109d20 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f87a800baa0 tx=0x7f87a8004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:58:40.738 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.735+0000 7f87ae7fc700 1 -- 172.21.15.27:0/4267537668 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87a800e070 con 0x7f87b00f55c0 2021-11-03T17:58:40.739 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.735+0000 7f87b810d700 1 -- 172.21.15.27:0/4267537668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87b010c7d0 con 0x7f87b00f55c0 2021-11-03T17:58:40.739 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.735+0000 7f87ae7fc700 1 -- 172.21.15.27:0/4267537668 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f87a8007df0 con 0x7f87b00f55c0 2021-11-03T17:58:40.739 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.735+0000 7f87ae7fc700 1 -- 172.21.15.27:0/4267537668 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87a8018070 con 0x7f87b00f55c0 2021-11-03T17:58:40.739 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.735+0000 7f87b810d700 1 -- 172.21.15.27:0/4267537668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87b010cd00 con 0x7f87b00f55c0 2021-11-03T17:58:40.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.736+0000 7f87b810d700 1 -- 172.21.15.27:0/4267537668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87b0045ad0 con 0x7f87b00f55c0 2021-11-03T17:58:40.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:40 smithi115 ceph-mon[65453]: pgmap v1320: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:40.741 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2562.421381 secs 2021-11-03T17:58:40.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.737+0000 7f87ae7fc700 1 -- 172.21.15.27:0/4267537668 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f87a8024060 con 0x7f87b00f55c0 2021-11-03T17:58:40.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.737+0000 7f87ae7fc700 1 --2- 172.21.15.27:0/4267537668 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f87980640b0 0x7f8798066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:58:40.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.737+0000 7f87ae7fc700 1 -- 172.21.15.27:0/4267537668 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f87a8081f90 con 0x7f87b00f55c0 2021-11-03T17:58:40.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.737+0000 7f87b56a8700 1 --2- 172.21.15.27:0/4267537668 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f87980640b0 0x7f8798066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:58:40.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.738+0000 7f87b56a8700 1 --2- 172.21.15.27:0/4267537668 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f87980640b0 0x7f8798066570 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f87a4009da0 tx=0x7f87a400b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:58:40.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.741+0000 7f87ae7fc700 1 -- 172.21.15.27:0/4267537668 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f87a8054290 con 0x7f87b00f55c0 2021-11-03T17:58:40.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:40 smithi027 ceph-mon[97244]: pgmap v1320: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:40.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2562.421381 secs 2021-11-03T17:58:40.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.971+0000 7f87b810d700 1 -- 172.21.15.27:0/4267537668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f87b00fde50 con 0x7f87b00f55c0 2021-11-03T17:58:40.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.971+0000 7f87ae7fc700 1 -- 172.21.15.27:0/4267537668 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f87a80222e0 con 0x7f87b00f55c0 2021-11-03T17:58:40.968 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:58:40.968 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:58:40.968 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:58:40.969 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:58:40.969 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:58:40.969 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:58:40.969 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:58:40.969 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:58:40.970 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:58:40.970 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:58:40.970 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:58:40.970 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:58:40.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:58:40.971 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:58:40.971 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:58:40.971 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:58:40.972 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:58:40.972 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:58:40.973 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.973+0000 7f87b810d700 1 -- 172.21.15.27:0/4267537668 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f87980640b0 msgr2=0x7f8798066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:58:40.973 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.973+0000 7f87b810d700 1 --2- 172.21.15.27:0/4267537668 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f87980640b0 0x7f8798066570 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 rx=0x7f87a4009da0 tx=0x7f87a400b040).stop 2021-11-03T17:58:40.973 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.973+0000 7f87b810d700 1 -- 172.21.15.27:0/4267537668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87b00f55c0 msgr2=0x7f87b0109d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:58:40.973 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.973+0000 7f87b810d700 1 --2- 172.21.15.27:0/4267537668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87b00f55c0 0x7f87b0109d20 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f87a800baa0 tx=0x7f87a8004d10).stop 2021-11-03T17:58:40.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.973+0000 7f87b810d700 1 -- 172.21.15.27:0/4267537668 shutdown_connections 2021-11-03T17:58:40.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.973+0000 7f87b810d700 1 --2- 172.21.15.27:0/4267537668 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f87980640b0 0x7f8798066570 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:40.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.973+0000 7f87b810d700 1 --2- 172.21.15.27:0/4267537668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87b00f55c0 0x7f87b0109d20 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:40.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.973+0000 7f87b810d700 1 --2- 172.21.15.27:0/4267537668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87b00f6860 0x7f87b010c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:40.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.973+0000 7f87b810d700 1 -- 172.21.15.27:0/4267537668 >> 172.21.15.27:0/4267537668 conn(0x7f87b00f0d40 msgr2=0x7f87b00fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:58:40.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.973+0000 7f87b810d700 1 -- 172.21.15.27:0/4267537668 shutdown_connections 2021-11-03T17:58:40.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:40.973+0000 7f87b810d700 1 -- 172.21.15.27:0/4267537668 wait complete. 2021-11-03T17:58:41.069 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.072+0000 7f4be295b700 1 -- 172.21.15.27:0/2335322093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bdc0f6ff0 msgr2=0x7f4bdc0f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:58:41.069 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.072+0000 7f4be295b700 1 --2- 172.21.15.27:0/2335322093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bdc0f6ff0 0x7f4bdc0f7a20 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7f4bd0004660 tx=0x7f4bd0009b30).stop 2021-11-03T17:58:41.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.073+0000 7f4be295b700 1 -- 172.21.15.27:0/2335322093 shutdown_connections 2021-11-03T17:58:41.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.073+0000 7f4be295b700 1 --2- 172.21.15.27:0/2335322093 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bdc0f6ff0 0x7f4bdc0f7a20 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:41.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.073+0000 7f4be295b700 1 --2- 172.21.15.27:0/2335322093 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bdc0f66e0 0x7f4bdc0f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:41.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.073+0000 7f4be295b700 1 -- 172.21.15.27:0/2335322093 >> 172.21.15.27:0/2335322093 conn(0x7f4bdc0f1f00 msgr2=0x7f4bdc0f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:58:41.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.073+0000 7f4be295b700 1 -- 172.21.15.27:0/2335322093 shutdown_connections 2021-11-03T17:58:41.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.073+0000 7f4be295b700 1 -- 172.21.15.27:0/2335322093 wait complete. 2021-11-03T17:58:41.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.073+0000 7f4be295b700 1 Processor -- start 2021-11-03T17:58:41.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.073+0000 7f4be295b700 1 -- start start 2021-11-03T17:58:41.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.073+0000 7f4be295b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bdc0f66e0 0x7f4bdc106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:58:41.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.073+0000 7f4be295b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bdc0f6ff0 0x7f4bdc107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:58:41.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.074+0000 7f4be295b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bdc107a10 con 0x7f4bdc0f66e0 2021-11-03T17:58:41.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.074+0000 7f4be295b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bdc107b50 con 0x7f4bdc0f6ff0 2021-11-03T17:58:41.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.074+0000 7f4bdbfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bdc0f66e0 0x7f4bdc106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:58:41.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.074+0000 7f4bdbfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bdc0f66e0 0x7f4bdc106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60484/0 (socket says 172.21.15.27:60484) 2021-11-03T17:58:41.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.074+0000 7f4bdbfff700 1 -- 172.21.15.27:0/2049539343 learned_addr learned my addr 172.21.15.27:0/2049539343 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:58:41.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.074+0000 7f4bdb7fe700 1 --2- 172.21.15.27:0/2049539343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bdc0f6ff0 0x7f4bdc107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:58:41.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.074+0000 7f4bdbfff700 1 -- 172.21.15.27:0/2049539343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bdc0f6ff0 msgr2=0x7f4bdc107460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:58:41.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.074+0000 7f4bdbfff700 1 --2- 172.21.15.27:0/2049539343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bdc0f6ff0 0x7f4bdc107460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:41.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.074+0000 7f4bdbfff700 1 -- 172.21.15.27:0/2049539343 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd0005040 con 0x7f4bdc0f66e0 2021-11-03T17:58:41.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.074+0000 7f4bdbfff700 1 --2- 172.21.15.27:0/2049539343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bdc0f66e0 0x7f4bdc106f70 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7f4bcc0026d0 tx=0x7f4bcc009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:58:41.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.074+0000 7f4bd8ff9700 1 -- 172.21.15.27:0/2049539343 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bcc0079c0 con 0x7f4bdc0f66e0 2021-11-03T17:58:41.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.074+0000 7f4bd8ff9700 1 -- 172.21.15.27:0/2049539343 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f4bcc00f040 con 0x7f4bdc0f66e0 2021-11-03T17:58:41.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.074+0000 7f4be295b700 1 -- 172.21.15.27:0/2049539343 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bdc1084f0 con 0x7f4bdc0f66e0 2021-11-03T17:58:41.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.074+0000 7f4bd8ff9700 1 -- 172.21.15.27:0/2049539343 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bcc0148b0 con 0x7f4bdc0f66e0 2021-11-03T17:58:41.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.074+0000 7f4be295b700 1 -- 172.21.15.27:0/2049539343 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bdc108ab0 con 0x7f4bdc0f66e0 2021-11-03T17:58:41.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.076+0000 7f4be295b700 1 -- 172.21.15.27:0/2049539343 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bdc045ad0 con 0x7f4bdc0f66e0 2021-11-03T17:58:41.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.076+0000 7f4bd8ff9700 1 -- 172.21.15.27:0/2049539343 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f4bcc007b20 con 0x7f4bdc0f66e0 2021-11-03T17:58:41.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.076+0000 7f4bd8ff9700 1 --2- 172.21.15.27:0/2049539343 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4bc4064090 0x7f4bc4066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:58:41.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.076+0000 7f4bd8ff9700 1 -- 172.21.15.27:0/2049539343 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f4bcc0806c0 con 0x7f4bdc0f66e0 2021-11-03T17:58:41.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.076+0000 7f4bdb7fe700 1 --2- 172.21.15.27:0/2049539343 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4bc4064090 0x7f4bc4066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:58:41.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.077+0000 7f4bdb7fe700 1 --2- 172.21.15.27:0/2049539343 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4bc4064090 0x7f4bc4066550 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7f4bd0004e30 tx=0x7f4bd0009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:58:41.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.080+0000 7f4bd8ff9700 1 -- 172.21.15.27:0/2049539343 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f4bcc0529c0 con 0x7f4bdc0f66e0 2021-11-03T17:58:41.280 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.284+0000 7f4be295b700 1 -- 172.21.15.27:0/2049539343 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4bdc0fefe0 con 0x7f4bdc0f66e0 2021-11-03T17:58:41.281 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.284+0000 7f4bd8ff9700 1 -- 172.21.15.27:0/2049539343 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f4bcc0525b0 con 0x7f4bdc0f66e0 2021-11-03T17:58:41.282 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:58:41.282 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:58:41.282 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:58:41.282 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:58:41.283 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:58:41.283 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:58:41.283 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:58:41.284 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:58:41.284 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:58:41.284 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:58:41.284 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:58:41.284 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:58:41.285 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:58:41.285 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:58:41.285 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:58:41.285 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:58:41.286 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:58:41.286 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:58:41.286 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:58:41.286 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:58:41.286 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:58:41.287 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:58:41.287 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:58:41.287 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:58:41.287 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:58:41.288 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:58:41.288 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:58:41.288 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:58:41.288 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:58:41.289 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:58:41.289 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:58:41.289 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:58:41.289 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:58:41.290 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:58:41.290 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:58:41.290 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:58:41.290 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:58:41.291 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:58:41.291 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:58:41.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.286+0000 7f4be295b700 1 -- 172.21.15.27:0/2049539343 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4bc4064090 msgr2=0x7f4bc4066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:58:41.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.286+0000 7f4be295b700 1 --2- 172.21.15.27:0/2049539343 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4bc4064090 0x7f4bc4066550 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 rx=0x7f4bd0004e30 tx=0x7f4bd0009b30).stop 2021-11-03T17:58:41.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.286+0000 7f4be295b700 1 -- 172.21.15.27:0/2049539343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bdc0f66e0 msgr2=0x7f4bdc106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:58:41.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.286+0000 7f4be295b700 1 --2- 172.21.15.27:0/2049539343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bdc0f66e0 0x7f4bdc106f70 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7f4bcc0026d0 tx=0x7f4bcc009de0).stop 2021-11-03T17:58:41.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.286+0000 7f4be295b700 1 -- 172.21.15.27:0/2049539343 shutdown_connections 2021-11-03T17:58:41.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.286+0000 7f4be295b700 1 --2- 172.21.15.27:0/2049539343 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4bc4064090 0x7f4bc4066550 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:41.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.286+0000 7f4be295b700 1 --2- 172.21.15.27:0/2049539343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4bdc0f66e0 0x7f4bdc106f70 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:41.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.286+0000 7f4be295b700 1 --2- 172.21.15.27:0/2049539343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4bdc0f6ff0 0x7f4bdc107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:58:41.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.286+0000 7f4be295b700 1 -- 172.21.15.27:0/2049539343 >> 172.21.15.27:0/2049539343 conn(0x7f4bdc0f1f00 msgr2=0x7f4bdc0fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:58:41.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.286+0000 7f4be295b700 1 -- 172.21.15.27:0/2049539343 shutdown_connections 2021-11-03T17:58:41.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:58:41.287+0000 7f4be295b700 1 -- 172.21.15.27:0/2049539343 wait complete. 2021-11-03T17:58:41.295 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:58:41.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:41 smithi115 ceph-mon[65453]: from='client.35464 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:58:41.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:41 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/4267537668' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:41.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:41 smithi027 ceph-mon[97244]: from='client.35464 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:58:41.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:41 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/4267537668' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:42.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:42 smithi115 ceph-mon[65453]: pgmap v1321: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:42.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:42 smithi115 ceph-mon[65453]: from='client.35468 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:58:42.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:42 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2049539343' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:58:42.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:42 smithi027 ceph-mon[97244]: pgmap v1321: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:42.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:42 smithi027 ceph-mon[97244]: from='client.35468 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:58:42.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:42 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2049539343' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:58:44.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:44 smithi115 ceph-mon[65453]: pgmap v1322: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:44.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:44 smithi027 ceph-mon[97244]: pgmap v1322: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:45.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2567.421522 secs 2021-11-03T17:58:45.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2567.421522 secs 2021-11-03T17:58:46.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:46 smithi115 ceph-mon[65453]: pgmap v1323: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:46.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:46 smithi027 ceph-mon[97244]: pgmap v1323: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:47.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:58:47.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:58:47.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:58:47.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:58:48.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:48 smithi115 ceph-mon[65453]: pgmap v1324: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:48.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:58:48.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:48.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:48.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:48.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:48.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:48 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:58:48.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:48 smithi027 ceph-mon[97244]: pgmap v1324: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:48.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:58:48.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:48.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:48.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:48.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:58:48.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:48 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:58:50.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:50 smithi115 ceph-mon[65453]: pgmap v1325: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:50.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2572.421651 secs 2021-11-03T17:58:50.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:50 smithi027 ceph-mon[97244]: pgmap v1325: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:50.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2572.421651 secs 2021-11-03T17:58:52.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:52 smithi115 ceph-mon[65453]: pgmap v1326: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:52.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:52 smithi027 ceph-mon[97244]: pgmap v1326: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:54.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:54 smithi115 ceph-mon[65453]: pgmap v1327: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:54.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:54 smithi027 ceph-mon[97244]: pgmap v1327: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:55.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2577.421793 secs 2021-11-03T17:58:55.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2577.421793 secs 2021-11-03T17:58:56.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:56 smithi115 ceph-mon[65453]: pgmap v1328: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:56.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:58:56.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:58:56.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:56 smithi027 ceph-mon[97244]: pgmap v1328: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:56.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:58:56.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:58:57.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:58:57.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:58:58.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:58 smithi115 ceph-mon[65453]: pgmap v1329: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:58.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:58:58 smithi115 ceph-mon[65453]: poll took 20050.716 ms: 1 events 2021-11-03T17:58:58.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:58 smithi027 ceph-mon[97244]: pgmap v1329: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:58:58.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:58:58 smithi027 ceph-mon[97244]: poll took 20050.716 ms: 1 events 2021-11-03T17:59:00.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:00 smithi115 ceph-mon[65453]: pgmap v1330: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:00.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2582.421875 secs 2021-11-03T17:59:00.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:00 smithi027 ceph-mon[97244]: pgmap v1330: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:00.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2582.421875 secs 2021-11-03T17:59:02.665 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:02 smithi027 ceph-mon[97244]: pgmap v1331: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:02.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:02 smithi115 ceph-mon[65453]: pgmap v1331: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:03.957 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:03 smithi115 ceph-mon[65453]: pgmap v1332: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:03.957 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:03 smithi115 ceph-mon[65453]: poll took 5287.883 ms: 1 events 2021-11-03T17:59:03.957 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:59:03.958 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:59:03.958 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:59:03.958 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:03.958 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:03.959 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:03.959 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:03.959 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:03 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:59:04.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:03 smithi027 ceph-mon[97244]: pgmap v1332: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:04.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:03 smithi027 ceph-mon[97244]: poll took 5287.883 ms: 1 events 2021-11-03T17:59:04.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:59:04.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:59:04.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:59:04.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:04.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:04.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:04.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:04.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:03 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:59:05.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2587.422011 secs 2021-11-03T17:59:05.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:05 smithi115 ceph-mon[65453]: pgmap v1333: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:06.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2587.422011 secs 2021-11-03T17:59:06.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:05 smithi027 ceph-mon[97244]: pgmap v1333: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:07.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:07 smithi115 ceph-mon[65453]: pgmap v1334: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:08.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:07 smithi027 ceph-mon[97244]: pgmap v1334: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:09.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:09 smithi115 ceph-mon[65453]: pgmap v1335: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:10.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:09 smithi027 ceph-mon[97244]: pgmap v1335: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:10.985 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2592.422128 secs 2021-11-03T17:59:11.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2592.422128 secs 2021-11-03T17:59:11.400 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.401+0000 7f3eccdc4700 1 -- 172.21.15.27:0/3424363299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ec80f5ed0 msgr2=0x7f3ec80f6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:59:11.400 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.401+0000 7f3eccdc4700 1 --2- 172.21.15.27:0/3424363299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ec80f5ed0 0x7f3ec80f6900 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7f3ebc004660 tx=0x7f3ebc009b30).stop 2021-11-03T17:59:11.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.402+0000 7f3eccdc4700 1 -- 172.21.15.27:0/3424363299 shutdown_connections 2021-11-03T17:59:11.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.402+0000 7f3eccdc4700 1 --2- 172.21.15.27:0/3424363299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ec80f5ed0 0x7f3ec80f6900 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:11.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.402+0000 7f3eccdc4700 1 --2- 172.21.15.27:0/3424363299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ec80f55c0 0x7f3ec80f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:11.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.402+0000 7f3eccdc4700 1 -- 172.21.15.27:0/3424363299 >> 172.21.15.27:0/3424363299 conn(0x7f3ec80f0dc0 msgr2=0x7f3ec80f31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:59:11.402 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.402+0000 7f3eccdc4700 1 -- 172.21.15.27:0/3424363299 shutdown_connections 2021-11-03T17:59:11.402 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.402+0000 7f3eccdc4700 1 -- 172.21.15.27:0/3424363299 wait complete. 2021-11-03T17:59:11.402 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.402+0000 7f3eccdc4700 1 Processor -- start 2021-11-03T17:59:11.403 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.402+0000 7f3eccdc4700 1 -- start start 2021-11-03T17:59:11.403 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.402+0000 7f3eccdc4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ec80f55c0 0x7f3ec8109c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:59:11.403 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.402+0000 7f3eccdc4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ec80f5ed0 0x7f3ec810c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:59:11.403 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.402+0000 7f3eccdc4700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec8060560 con 0x7f3ec80f55c0 2021-11-03T17:59:11.404 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.402+0000 7f3eccdc4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ec80606a0 con 0x7f3ec80f5ed0 2021-11-03T17:59:11.404 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.403+0000 7f3ec659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ec80f55c0 0x7f3ec8109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:59:11.404 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.403+0000 7f3ec659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ec80f55c0 0x7f3ec8109c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60502/0 (socket says 172.21.15.27:60502) 2021-11-03T17:59:11.404 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.403+0000 7f3ec659c700 1 -- 172.21.15.27:0/3359748150 learned_addr learned my addr 172.21.15.27:0/3359748150 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:59:11.405 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.403+0000 7f3ec5d9b700 1 --2- 172.21.15.27:0/3359748150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ec80f5ed0 0x7f3ec810c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:59:11.405 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.403+0000 7f3ec659c700 1 -- 172.21.15.27:0/3359748150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ec80f5ed0 msgr2=0x7f3ec810c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:59:11.405 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.403+0000 7f3ec659c700 1 --2- 172.21.15.27:0/3359748150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ec80f5ed0 0x7f3ec810c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:11.405 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.403+0000 7f3ec659c700 1 -- 172.21.15.27:0/3359748150 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ebc005040 con 0x7f3ec80f55c0 2021-11-03T17:59:11.405 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.403+0000 7f3ec659c700 1 --2- 172.21.15.27:0/3359748150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ec80f55c0 0x7f3ec8109c70 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f3eb800c790 tx=0x7f3eb8007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:59:11.406 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.403+0000 7f3eb6ffd700 1 -- 172.21.15.27:0/3359748150 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3eb800e680 con 0x7f3ec80f55c0 2021-11-03T17:59:11.406 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.403+0000 7f3eccdc4700 1 -- 172.21.15.27:0/3359748150 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ec810c720 con 0x7f3ec80f55c0 2021-11-03T17:59:11.406 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.403+0000 7f3eb6ffd700 1 -- 172.21.15.27:0/3359748150 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f3eb800ec90 con 0x7f3ec80f55c0 2021-11-03T17:59:11.406 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.403+0000 7f3eb6ffd700 1 -- 172.21.15.27:0/3359748150 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3eb8013d60 con 0x7f3ec80f55c0 2021-11-03T17:59:11.407 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.403+0000 7f3eccdc4700 1 -- 172.21.15.27:0/3359748150 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ec810cce0 con 0x7f3ec80f55c0 2021-11-03T17:59:11.408 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.404+0000 7f3eccdc4700 1 -- 172.21.15.27:0/3359748150 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ec8045ad0 con 0x7f3ec80f55c0 2021-11-03T17:59:11.408 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.405+0000 7f3eb6ffd700 1 -- 172.21.15.27:0/3359748150 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f3eb801a060 con 0x7f3ec80f55c0 2021-11-03T17:59:11.408 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.405+0000 7f3eb6ffd700 1 --2- 172.21.15.27:0/3359748150 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3eb00640b0 0x7f3eb0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:59:11.409 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.405+0000 7f3eb6ffd700 1 -- 172.21.15.27:0/3359748150 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f3eb8080b60 con 0x7f3ec80f55c0 2021-11-03T17:59:11.409 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.405+0000 7f3ec5d9b700 1 --2- 172.21.15.27:0/3359748150 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3eb00640b0 0x7f3eb0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:59:11.409 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.406+0000 7f3ec5d9b700 1 --2- 172.21.15.27:0/3359748150 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3eb00640b0 0x7f3eb0066570 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f3ebc004e30 tx=0x7f3ebc009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:59:11.409 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.409+0000 7f3eb6ffd700 1 -- 172.21.15.27:0/3359748150 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f3eb8052d70 con 0x7f3ec80f55c0 2021-11-03T17:59:11.586 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.589+0000 7f3eccdc4700 1 -- 172.21.15.27:0/3359748150 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ec80fdec0 con 0x7f3eb00640b0 2021-11-03T17:59:11.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.591+0000 7f3eb6ffd700 1 -- 172.21.15.27:0/3359748150 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f3ec80fdec0 con 0x7f3eb00640b0 2021-11-03T17:59:11.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.592+0000 7f3eccdc4700 1 -- 172.21.15.27:0/3359748150 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3eb00640b0 msgr2=0x7f3eb0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:59:11.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.592+0000 7f3eccdc4700 1 --2- 172.21.15.27:0/3359748150 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3eb00640b0 0x7f3eb0066570 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 rx=0x7f3ebc004e30 tx=0x7f3ebc009b30).stop 2021-11-03T17:59:11.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.592+0000 7f3eccdc4700 1 -- 172.21.15.27:0/3359748150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ec80f55c0 msgr2=0x7f3ec8109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:59:11.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.592+0000 7f3eccdc4700 1 --2- 172.21.15.27:0/3359748150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ec80f55c0 0x7f3ec8109c70 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f3eb800c790 tx=0x7f3eb8007d70).stop 2021-11-03T17:59:11.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.593+0000 7f3eccdc4700 1 -- 172.21.15.27:0/3359748150 shutdown_connections 2021-11-03T17:59:11.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.593+0000 7f3eccdc4700 1 --2- 172.21.15.27:0/3359748150 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3eb00640b0 0x7f3eb0066570 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:11.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.593+0000 7f3eccdc4700 1 --2- 172.21.15.27:0/3359748150 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ec80f55c0 0x7f3ec8109c70 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:11.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.593+0000 7f3eccdc4700 1 --2- 172.21.15.27:0/3359748150 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ec80f5ed0 0x7f3ec810c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:11.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.593+0000 7f3eccdc4700 1 -- 172.21.15.27:0/3359748150 >> 172.21.15.27:0/3359748150 conn(0x7f3ec80f0dc0 msgr2=0x7f3ec80fa350 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:59:11.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.593+0000 7f3eccdc4700 1 -- 172.21.15.27:0/3359748150 shutdown_connections 2021-11-03T17:59:11.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.593+0000 7f3eccdc4700 1 -- 172.21.15.27:0/3359748150 wait complete. 2021-11-03T17:59:11.598 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:59:11.691 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.693+0000 7f7bf37ab700 1 -- 172.21.15.27:0/126469952 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bec0f55c0 msgr2=0x7f7bec0f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:59:11.691 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.693+0000 7f7bf37ab700 1 --2- 172.21.15.27:0/126469952 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bec0f55c0 0x7f7bec0f59e0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7f7bdc004660 tx=0x7f7bdc009b30).stop 2021-11-03T17:59:11.691 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.693+0000 7f7bf37ab700 1 -- 172.21.15.27:0/126469952 shutdown_connections 2021-11-03T17:59:11.692 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.693+0000 7f7bf37ab700 1 --2- 172.21.15.27:0/126469952 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bec0f6860 0x7f7bec0f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:11.692 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.693+0000 7f7bf37ab700 1 --2- 172.21.15.27:0/126469952 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bec0f55c0 0x7f7bec0f59e0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:11.692 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.693+0000 7f7bf37ab700 1 -- 172.21.15.27:0/126469952 >> 172.21.15.27:0/126469952 conn(0x7f7bec0f0d40 msgr2=0x7f7bec0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:59:11.692 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.693+0000 7f7bf37ab700 1 -- 172.21.15.27:0/126469952 shutdown_connections 2021-11-03T17:59:11.693 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.693+0000 7f7bf37ab700 1 -- 172.21.15.27:0/126469952 wait complete. 2021-11-03T17:59:11.693 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.693+0000 7f7bf37ab700 1 Processor -- start 2021-11-03T17:59:11.693 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.693+0000 7f7bf37ab700 1 -- start start 2021-11-03T17:59:11.693 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.694+0000 7f7bf37ab700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bec0f55c0 0x7f7bec10c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:59:11.693 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.694+0000 7f7bf37ab700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bec0f6860 0x7f7bec10e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:59:11.694 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.694+0000 7f7bf37ab700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec0602b0 con 0x7f7bec0f55c0 2021-11-03T17:59:11.694 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.694+0000 7f7bf37ab700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec0603f0 con 0x7f7bec0f6860 2021-11-03T17:59:11.694 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.694+0000 7f7bf1547700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bec0f55c0 0x7f7bec10c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:59:11.694 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.694+0000 7f7bf1547700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bec0f55c0 0x7f7bec10c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60512/0 (socket says 172.21.15.27:60512) 2021-11-03T17:59:11.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.694+0000 7f7bf1547700 1 -- 172.21.15.27:0/2256841638 learned_addr learned my addr 172.21.15.27:0/2256841638 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:59:11.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.694+0000 7f7bf0d46700 1 --2- 172.21.15.27:0/2256841638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bec0f6860 0x7f7bec10e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:59:11.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.694+0000 7f7bf1547700 1 -- 172.21.15.27:0/2256841638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bec0f6860 msgr2=0x7f7bec10e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:59:11.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.694+0000 7f7bf1547700 1 --2- 172.21.15.27:0/2256841638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bec0f6860 0x7f7bec10e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:11.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.694+0000 7f7bf1547700 1 -- 172.21.15.27:0/2256841638 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bdc005040 con 0x7f7bec0f55c0 2021-11-03T17:59:11.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.694+0000 7f7bf0d46700 1 --2- 172.21.15.27:0/2256841638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bec0f6860 0x7f7bec10e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:59:11.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.694+0000 7f7bf1547700 1 --2- 172.21.15.27:0/2256841638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bec0f55c0 0x7f7bec10c0d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7f7bdc004f60 tx=0x7f7bdc009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:59:11.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.694+0000 7f7be1ffb700 1 -- 172.21.15.27:0/2256841638 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bdc018070 con 0x7f7bec0f55c0 2021-11-03T17:59:11.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.694+0000 7f7bf37ab700 1 -- 172.21.15.27:0/2256841638 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bec10eac0 con 0x7f7bec0f55c0 2021-11-03T17:59:11.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.694+0000 7f7be1ffb700 1 -- 172.21.15.27:0/2256841638 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f7bdc007ba0 con 0x7f7bec0f55c0 2021-11-03T17:59:11.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.694+0000 7f7be1ffb700 1 -- 172.21.15.27:0/2256841638 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bdc01c620 con 0x7f7bec0f55c0 2021-11-03T17:59:11.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.694+0000 7f7bf37ab700 1 -- 172.21.15.27:0/2256841638 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bec10ee90 con 0x7f7bec0f55c0 2021-11-03T17:59:11.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.695+0000 7f7bf37ab700 1 -- 172.21.15.27:0/2256841638 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bec045ad0 con 0x7f7bec0f55c0 2021-11-03T17:59:11.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.696+0000 7f7be1ffb700 1 -- 172.21.15.27:0/2256841638 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f7bdc01c8e0 con 0x7f7bec0f55c0 2021-11-03T17:59:11.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.696+0000 7f7be1ffb700 1 --2- 172.21.15.27:0/2256841638 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7bd80640b0 0x7f7bd8066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:59:11.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.696+0000 7f7be1ffb700 1 -- 172.21.15.27:0/2256841638 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f7bdc082720 con 0x7f7bec0f55c0 2021-11-03T17:59:11.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.696+0000 7f7bf0d46700 1 --2- 172.21.15.27:0/2256841638 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7bd80640b0 0x7f7bd8066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:59:11.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.697+0000 7f7bf0d46700 1 --2- 172.21.15.27:0/2256841638 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7bd80640b0 0x7f7bd8066570 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f7be80041d0 tx=0x7f7be8009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:59:11.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.700+0000 7f7be1ffb700 1 -- 172.21.15.27:0/2256841638 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f7bdc054a20 con 0x7f7bec0f55c0 2021-11-03T17:59:11.870 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.874+0000 7f7bf37ab700 1 -- 172.21.15.27:0/2256841638 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7bec057f10 con 0x7f7bd80640b0 2021-11-03T17:59:11.885 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.889+0000 7f7be1ffb700 1 -- 172.21.15.27:0/2256841638 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f7bec057f10 con 0x7f7bd80640b0 2021-11-03T17:59:11.886 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:59:11.886 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (50m) 9s ago 51m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:59:11.886 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (42m) 9s ago 51m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:59:11.886 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (42m) 9m ago 51m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:59:11.887 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (50m) 9s ago 51m 41.8M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:59:11.887 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (44m) 9s ago 47m 26.0M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:59:11.887 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (47m) 9s ago 47m 3845M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:59:11.887 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (42m) 9m ago 47m 23.2M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:59:11.887 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (44m) 9m ago 47m 24.9M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:59:11.888 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (44m) 9s ago 53m 554M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:59:11.888 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (43m) 9m ago 51m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:59:11.888 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (43m) 9s ago 53m 235M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:59:11.888 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (43m) 9m ago 51m 95.6M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:59:11.889 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (51m) 9s ago 51m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:59:11.889 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (50m) 9m ago 50m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:59:11.889 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (42m) 9s ago 50m 351M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:59:11.889 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (41m) 9s ago 50m 383M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:59:11.889 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (41m) 9s ago 49m 284M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:59:11.890 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (40m) 9m ago 49m 379M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:59:11.890 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (40m) 9m ago 49m 304M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:59:11.890 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (39m) 9m ago 48m 307M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:59:11.891 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (50m) 9s ago 51m 55.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:59:11.891 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.891+0000 7f7bf37ab700 1 -- 172.21.15.27:0/2256841638 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7bd80640b0 msgr2=0x7f7bd8066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:59:11.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.891+0000 7f7bf37ab700 1 --2- 172.21.15.27:0/2256841638 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7bd80640b0 0x7f7bd8066570 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7f7be80041d0 tx=0x7f7be8009040).stop 2021-11-03T17:59:11.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.891+0000 7f7bf37ab700 1 -- 172.21.15.27:0/2256841638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bec0f55c0 msgr2=0x7f7bec10c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:59:11.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.891+0000 7f7bf37ab700 1 --2- 172.21.15.27:0/2256841638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bec0f55c0 0x7f7bec10c0d0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7f7bdc004f60 tx=0x7f7bdc009b30).stop 2021-11-03T17:59:11.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.891+0000 7f7bf37ab700 1 -- 172.21.15.27:0/2256841638 shutdown_connections 2021-11-03T17:59:11.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.891+0000 7f7bf37ab700 1 --2- 172.21.15.27:0/2256841638 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7bd80640b0 0x7f7bd8066570 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:11.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.891+0000 7f7bf37ab700 1 --2- 172.21.15.27:0/2256841638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bec0f55c0 0x7f7bec10c0d0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:11.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.891+0000 7f7bf37ab700 1 --2- 172.21.15.27:0/2256841638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bec0f6860 0x7f7bec10e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:11.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.891+0000 7f7bf37ab700 1 -- 172.21.15.27:0/2256841638 >> 172.21.15.27:0/2256841638 conn(0x7f7bec0f0d40 msgr2=0x7f7bec0fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:59:11.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.891+0000 7f7bf37ab700 1 -- 172.21.15.27:0/2256841638 shutdown_connections 2021-11-03T17:59:11.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.891+0000 7f7bf37ab700 1 -- 172.21.15.27:0/2256841638 wait complete. 2021-11-03T17:59:11.986 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.990+0000 7f7002ec9700 1 -- 172.21.15.27:0/4095677380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ffc0f66d0 msgr2=0x7f6ffc0f6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:59:11.987 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.990+0000 7f7002ec9700 1 --2- 172.21.15.27:0/4095677380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ffc0f66d0 0x7f6ffc0f6af0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7f6ff40099e0 tx=0x7f6ff4004d10).stop 2021-11-03T17:59:11.987 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.990+0000 7f7002ec9700 1 -- 172.21.15.27:0/4095677380 shutdown_connections 2021-11-03T17:59:11.987 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.990+0000 7f7002ec9700 1 --2- 172.21.15.27:0/4095677380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ffc0f7970 0x7f6ffc0f7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:11.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.990+0000 7f7002ec9700 1 --2- 172.21.15.27:0/4095677380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ffc0f66d0 0x7f6ffc0f6af0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:11.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.990+0000 7f7002ec9700 1 -- 172.21.15.27:0/4095677380 >> 172.21.15.27:0/4095677380 conn(0x7f6ffc0f1e10 msgr2=0x7f6ffc0f4250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:59:11.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.990+0000 7f7002ec9700 1 -- 172.21.15.27:0/4095677380 shutdown_connections 2021-11-03T17:59:11.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.990+0000 7f7002ec9700 1 -- 172.21.15.27:0/4095677380 wait complete. 2021-11-03T17:59:11.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.990+0000 7f7002ec9700 1 Processor -- start 2021-11-03T17:59:11.989 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.990+0000 7f7002ec9700 1 -- start start 2021-11-03T17:59:11.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.991+0000 7f7002ec9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ffc0f66d0 0x7f6ffc107070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:59:11.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.991+0000 7f7002ec9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ffc0f7970 0x7f6ffc107560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:59:11.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.991+0000 7f7002ec9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc107b10 con 0x7f6ffc0f7970 2021-11-03T17:59:11.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.991+0000 7f7002ec9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ffc107c50 con 0x7f6ffc0f66d0 2021-11-03T17:59:11.992 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.991+0000 7f6ffbfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ffc0f7970 0x7f6ffc107560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:59:11.992 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.991+0000 7f6ffbfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ffc0f7970 0x7f6ffc107560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60524/0 (socket says 172.21.15.27:60524) 2021-11-03T17:59:11.992 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.991+0000 7f6ffbfff700 1 -- 172.21.15.27:0/2025167851 learned_addr learned my addr 172.21.15.27:0/2025167851 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:59:11.992 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.991+0000 7f7000c65700 1 --2- 172.21.15.27:0/2025167851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ffc0f66d0 0x7f6ffc107070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:59:11.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.991+0000 7f6ffbfff700 1 -- 172.21.15.27:0/2025167851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ffc0f66d0 msgr2=0x7f6ffc107070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:59:11.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.991+0000 7f6ffbfff700 1 --2- 172.21.15.27:0/2025167851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ffc0f66d0 0x7f6ffc107070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:11.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.991+0000 7f6ffbfff700 1 -- 172.21.15.27:0/2025167851 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff4009640 con 0x7f6ffc0f7970 2021-11-03T17:59:11.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.991+0000 7f6ffbfff700 1 --2- 172.21.15.27:0/2025167851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ffc0f7970 0x7f6ffc107560 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7f6ff000e7f0 tx=0x7f6ff000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:59:11.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.993+0000 7f6ff97fa700 1 -- 172.21.15.27:0/2025167851 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ff00092e0 con 0x7f6ffc0f7970 2021-11-03T17:59:11.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.993+0000 7f6ff97fa700 1 -- 172.21.15.27:0/2025167851 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f6ff000f040 con 0x7f6ffc0f7970 2021-11-03T17:59:11.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.993+0000 7f7002ec9700 1 -- 172.21.15.27:0/2025167851 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ffc1085f0 con 0x7f6ffc0f7970 2021-11-03T17:59:11.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.993+0000 7f7002ec9700 1 -- 172.21.15.27:0/2025167851 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ffc108bb0 con 0x7f6ffc0f7970 2021-11-03T17:59:11.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.993+0000 7f6ff97fa700 1 -- 172.21.15.27:0/2025167851 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ff0014750 con 0x7f6ffc0f7970 2021-11-03T17:59:11.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.994+0000 7f7002ec9700 1 -- 172.21.15.27:0/2025167851 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ffc045ad0 con 0x7f6ffc0f7970 2021-11-03T17:59:11.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.995+0000 7f6ff97fa700 1 -- 172.21.15.27:0/2025167851 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f6ff001b030 con 0x7f6ffc0f7970 2021-11-03T17:59:11.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.995+0000 7f6ff97fa700 1 --2- 172.21.15.27:0/2025167851 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6fe40640b0 0x7f6fe4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:59:11.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.995+0000 7f6ff97fa700 1 -- 172.21.15.27:0/2025167851 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f6ff00806f0 con 0x7f6ffc0f7970 2021-11-03T17:59:11.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.995+0000 7f7000c65700 1 --2- 172.21.15.27:0/2025167851 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6fe40640b0 0x7f6fe4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:59:11.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.996+0000 7f7000c65700 1 --2- 172.21.15.27:0/2025167851 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6fe40640b0 0x7f6fe4066570 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f6ff400b620 tx=0x7f6ff4015040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:59:11.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:11.999+0000 7f6ff97fa700 1 -- 172.21.15.27:0/2025167851 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f6ff00528f0 con 0x7f6ffc0f7970 2021-11-03T17:59:12.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.230+0000 7f7002ec9700 1 -- 172.21.15.27:0/2025167851 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6ffc027a00 con 0x7f6ffc0f7970 2021-11-03T17:59:12.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.231+0000 7f6ff97fa700 1 -- 172.21.15.27:0/2025167851 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f6ff00192e0 con 0x7f6ffc0f7970 2021-11-03T17:59:12.228 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:59:12.228 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:59:12.228 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:59:12.228 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:59:12.229 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:59:12.229 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:59:12.229 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:59:12.229 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:59:12.229 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:59:12.230 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:59:12.230 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:59:12.230 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:59:12.230 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:59:12.231 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:59:12.231 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:59:12.231 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:59:12.231 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:59:12.232 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:59:12.232 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.232+0000 7f7002ec9700 1 -- 172.21.15.27:0/2025167851 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6fe40640b0 msgr2=0x7f6fe4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:59:12.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.232+0000 7f7002ec9700 1 --2- 172.21.15.27:0/2025167851 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6fe40640b0 0x7f6fe4066570 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 rx=0x7f6ff400b620 tx=0x7f6ff4015040).stop 2021-11-03T17:59:12.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.232+0000 7f7002ec9700 1 -- 172.21.15.27:0/2025167851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ffc0f7970 msgr2=0x7f6ffc107560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:59:12.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.232+0000 7f7002ec9700 1 --2- 172.21.15.27:0/2025167851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ffc0f7970 0x7f6ffc107560 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 rx=0x7f6ff000e7f0 tx=0x7f6ff000b040).stop 2021-11-03T17:59:12.234 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.232+0000 7f7002ec9700 1 -- 172.21.15.27:0/2025167851 shutdown_connections 2021-11-03T17:59:12.234 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.233+0000 7f7002ec9700 1 --2- 172.21.15.27:0/2025167851 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6fe40640b0 0x7f6fe4066570 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:12.234 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.233+0000 7f7002ec9700 1 --2- 172.21.15.27:0/2025167851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ffc0f66d0 0x7f6ffc107070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:12.234 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.233+0000 7f7002ec9700 1 --2- 172.21.15.27:0/2025167851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ffc0f7970 0x7f6ffc107560 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:12.235 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.233+0000 7f7002ec9700 1 -- 172.21.15.27:0/2025167851 >> 172.21.15.27:0/2025167851 conn(0x7f6ffc0f1e10 msgr2=0x7f6ffc0fb3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:59:12.235 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.233+0000 7f7002ec9700 1 -- 172.21.15.27:0/2025167851 shutdown_connections 2021-11-03T17:59:12.235 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.233+0000 7f7002ec9700 1 -- 172.21.15.27:0/2025167851 wait complete. 2021-11-03T17:59:12.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:11 smithi027 ceph-mon[97244]: pgmap v1336: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:12.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.333+0000 7f44cc250700 1 -- 172.21.15.27:0/1049942165 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44c40f66e0 msgr2=0x7f44c40f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:59:12.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.333+0000 7f44cc250700 1 --2- 172.21.15.27:0/1049942165 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44c40f66e0 0x7f44c40f6b00 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7f44b4004660 tx=0x7f44b4009b30).stop 2021-11-03T17:59:12.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.333+0000 7f44cc250700 1 -- 172.21.15.27:0/1049942165 shutdown_connections 2021-11-03T17:59:12.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.333+0000 7f44cc250700 1 --2- 172.21.15.27:0/1049942165 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44c40f6ff0 0x7f44c40f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:12.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.333+0000 7f44cc250700 1 --2- 172.21.15.27:0/1049942165 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44c40f66e0 0x7f44c40f6b00 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:12.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.333+0000 7f44cc250700 1 -- 172.21.15.27:0/1049942165 >> 172.21.15.27:0/1049942165 conn(0x7f44c40f1f00 msgr2=0x7f44c40f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:59:12.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.334+0000 7f44cc250700 1 -- 172.21.15.27:0/1049942165 shutdown_connections 2021-11-03T17:59:12.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.334+0000 7f44cc250700 1 -- 172.21.15.27:0/1049942165 wait complete. 2021-11-03T17:59:12.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.334+0000 7f44cc250700 1 Processor -- start 2021-11-03T17:59:12.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.334+0000 7f44cc250700 1 -- start start 2021-11-03T17:59:12.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.334+0000 7f44cc250700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44c40f66e0 0x7f44c406b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:59:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.334+0000 7f44cc250700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44c40f6ff0 0x7f44c4069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:59:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.334+0000 7f44cc250700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c406a380 con 0x7f44c40f6ff0 2021-11-03T17:59:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.334+0000 7f44cc250700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44c406a4c0 con 0x7f44c40f66e0 2021-11-03T17:59:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.334+0000 7f44c97eb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44c40f6ff0 0x7f44c4069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:59:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.334+0000 7f44c97eb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44c40f6ff0 0x7f44c4069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60534/0 (socket says 172.21.15.27:60534) 2021-11-03T17:59:12.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.334+0000 7f44c97eb700 1 -- 172.21.15.27:0/2920073311 learned_addr learned my addr 172.21.15.27:0/2920073311 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:59:12.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.335+0000 7f44c9fec700 1 --2- 172.21.15.27:0/2920073311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44c40f66e0 0x7f44c406b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:59:12.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.335+0000 7f44c97eb700 1 -- 172.21.15.27:0/2920073311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44c40f66e0 msgr2=0x7f44c406b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:59:12.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.335+0000 7f44c97eb700 1 --2- 172.21.15.27:0/2920073311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44c40f66e0 0x7f44c406b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:12.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.335+0000 7f44c97eb700 1 -- 172.21.15.27:0/2920073311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44b4005040 con 0x7f44c40f6ff0 2021-11-03T17:59:12.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.335+0000 7f44c9fec700 1 --2- 172.21.15.27:0/2920073311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44c40f66e0 0x7f44c406b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T17:59:12.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.335+0000 7f44c97eb700 1 --2- 172.21.15.27:0/2920073311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44c40f6ff0 0x7f44c4069e20 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f44c000c790 tx=0x7f44c0007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:59:12.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.335+0000 7f44ba7fc700 1 -- 172.21.15.27:0/2920073311 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44c000e680 con 0x7f44c40f6ff0 2021-11-03T17:59:12.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.335+0000 7f44ba7fc700 1 -- 172.21.15.27:0/2920073311 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f44c000ec90 con 0x7f44c40f6ff0 2021-11-03T17:59:12.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.335+0000 7f44cc250700 1 -- 172.21.15.27:0/2920073311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44c406a7a0 con 0x7f44c40f6ff0 2021-11-03T17:59:12.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.335+0000 7f44ba7fc700 1 -- 172.21.15.27:0/2920073311 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44c0013d60 con 0x7f44c40f6ff0 2021-11-03T17:59:12.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.335+0000 7f44cc250700 1 -- 172.21.15.27:0/2920073311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44c410ef90 con 0x7f44c40f6ff0 2021-11-03T17:59:12.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.336+0000 7f44cc250700 1 -- 172.21.15.27:0/2920073311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44c4045ad0 con 0x7f44c40f6ff0 2021-11-03T17:59:12.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.337+0000 7f44ba7fc700 1 -- 172.21.15.27:0/2920073311 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f44c001a060 con 0x7f44c40f6ff0 2021-11-03T17:59:12.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.337+0000 7f44ba7fc700 1 --2- 172.21.15.27:0/2920073311 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f44b0064180 0x7f44b0066640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:59:12.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.337+0000 7f44ba7fc700 1 -- 172.21.15.27:0/2920073311 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f44c00809d0 con 0x7f44c40f6ff0 2021-11-03T17:59:12.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.337+0000 7f44c9fec700 1 --2- 172.21.15.27:0/2920073311 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f44b0064180 0x7f44b0066640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:59:12.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.337+0000 7f44c9fec700 1 --2- 172.21.15.27:0/2920073311 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f44b0064180 0x7f44b0066640 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f44b4004e30 tx=0x7f44b4009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:59:12.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.341+0000 7f44ba7fc700 1 -- 172.21.15.27:0/2920073311 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f44c0052c60 con 0x7f44c40f6ff0 2021-11-03T17:59:12.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:11 smithi115 ceph-mon[65453]: pgmap v1336: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:12.546 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.549+0000 7f44cc250700 1 -- 172.21.15.27:0/2920073311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f44c4027a00 con 0x7f44c40f6ff0 2021-11-03T17:59:12.546 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.550+0000 7f44ba7fc700 1 -- 172.21.15.27:0/2920073311 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f44c0018070 con 0x7f44c40f6ff0 2021-11-03T17:59:12.547 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:59:12.547 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:59:12.547 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:59:12.547 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:59:12.548 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:59:12.548 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:59:12.548 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:59:12.548 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:59:12.549 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:59:12.549 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:59:12.549 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:59:12.549 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:59:12.550 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:59:12.550 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:59:12.550 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:59:12.550 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:59:12.551 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:59:12.551 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:59:12.551 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:59:12.551 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:59:12.552 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:59:12.552 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:59:12.552 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:59:12.552 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:59:12.553 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:59:12.553 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:59:12.553 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:59:12.554 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:59:12.554 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:59:12.554 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:59:12.555 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:59:12.555 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:59:12.556 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:59:12.556 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:59:12.557 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:59:12.557 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:59:12.558 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:59:12.559 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:59:12.559 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:59:12.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.551+0000 7f44cc250700 1 -- 172.21.15.27:0/2920073311 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f44b0064180 msgr2=0x7f44b0066640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:59:12.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.551+0000 7f44cc250700 1 --2- 172.21.15.27:0/2920073311 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f44b0064180 0x7f44b0066640 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7f44b4004e30 tx=0x7f44b4009b30).stop 2021-11-03T17:59:12.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.551+0000 7f44cc250700 1 -- 172.21.15.27:0/2920073311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44c40f6ff0 msgr2=0x7f44c4069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:59:12.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.551+0000 7f44cc250700 1 --2- 172.21.15.27:0/2920073311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44c40f6ff0 0x7f44c4069e20 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f44c000c790 tx=0x7f44c0007d70).stop 2021-11-03T17:59:12.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.551+0000 7f44cc250700 1 -- 172.21.15.27:0/2920073311 shutdown_connections 2021-11-03T17:59:12.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.551+0000 7f44cc250700 1 --2- 172.21.15.27:0/2920073311 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f44b0064180 0x7f44b0066640 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:12.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.551+0000 7f44cc250700 1 --2- 172.21.15.27:0/2920073311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44c40f66e0 0x7f44c406b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:12.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.551+0000 7f44cc250700 1 --2- 172.21.15.27:0/2920073311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44c40f6ff0 0x7f44c4069e20 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:12.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.551+0000 7f44cc250700 1 -- 172.21.15.27:0/2920073311 >> 172.21.15.27:0/2920073311 conn(0x7f44c40f1f00 msgr2=0x7f44c40fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:59:12.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.551+0000 7f44cc250700 1 -- 172.21.15.27:0/2920073311 shutdown_connections 2021-11-03T17:59:12.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:12.551+0000 7f44cc250700 1 -- 172.21.15.27:0/2920073311 wait complete. 2021-11-03T17:59:12.567 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:59:13.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:12 smithi027 ceph-mon[97244]: from='client.35480 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:59:13.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:12 smithi027 ceph-mon[97244]: from='client.35484 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:59:13.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:12 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2025167851' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:13.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:12 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2920073311' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:59:13.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:59:13.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:59:13.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:59:13.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:13.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:13.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:13.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:13.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:12 smithi115 ceph-mon[65453]: from='client.35480 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:59:13.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:12 smithi115 ceph-mon[65453]: from='client.35484 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:59:13.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:12 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2025167851' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:13.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:12 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2920073311' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:59:13.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:59:13.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:59:13.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:59:13.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:13.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:13.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:13.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:14.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:13 smithi115 ceph-mon[65453]: pgmap v1337: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:14.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:13 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:59:14.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:13 smithi027 ceph-mon[97244]: pgmap v1337: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:14.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:13 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:59:15.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2597.422271 secs 2021-11-03T17:59:15.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2597.422271 secs 2021-11-03T17:59:16.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:16 smithi027 ceph-mon[97244]: pgmap v1338: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:16.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:16 smithi115 ceph-mon[65453]: pgmap v1338: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:18.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:18 smithi027 ceph-mon[97244]: pgmap v1339: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:18.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:18 smithi115 ceph-mon[65453]: pgmap v1339: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:20.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:20 smithi027 ceph-mon[97244]: pgmap v1340: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:20.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2602.422299 secs 2021-11-03T17:59:20.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:20 smithi115 ceph-mon[65453]: pgmap v1340: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:20.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2602.422299 secs 2021-11-03T17:59:22.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:22 smithi027 ceph-mon[97244]: pgmap v1341: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:22.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:22 smithi115 ceph-mon[65453]: pgmap v1341: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:23.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:59:23.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:59:23.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:59:23.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:23.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:23.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:23.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:23.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:59:23.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:59:23.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:59:23.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:23.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:23.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:23.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:24.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:24 smithi115 ceph-mon[65453]: pgmap v1342: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:24.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:24 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:59:24.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:24 smithi027 ceph-mon[97244]: pgmap v1342: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:24.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:24 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:59:25.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2607.422445 secs 2021-11-03T17:59:25.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2607.422445 secs 2021-11-03T17:59:26.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:26 smithi115 ceph-mon[65453]: pgmap v1343: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:26.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:26 smithi027 ceph-mon[97244]: pgmap v1343: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:28.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:28 smithi115 ceph-mon[65453]: pgmap v1344: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:28.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:28 smithi027 ceph-mon[97244]: pgmap v1344: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:30.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:30 smithi115 ceph-mon[65453]: pgmap v1345: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:30.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2612.422516 secs 2021-11-03T17:59:30.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:30 smithi027 ceph-mon[97244]: pgmap v1345: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:30.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2612.422516 secs 2021-11-03T17:59:32.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:32 smithi115 ceph-mon[65453]: pgmap v1346: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:32.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:32 smithi027 ceph-mon[97244]: pgmap v1346: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:33.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:59:33.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:59:33.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:59:33.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:33.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:33.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:33.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:33.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:59:33.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:59:33.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:59:33.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:33.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:33.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:33.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:34.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:34 smithi115 ceph-mon[65453]: pgmap v1347: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:34.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:34 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:59:34.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:34 smithi027 ceph-mon[97244]: pgmap v1347: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:34.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:34 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:59:35.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2617.422658 secs 2021-11-03T17:59:35.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2617.422658 secs 2021-11-03T17:59:36.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:36 smithi115 ceph-mon[65453]: pgmap v1348: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:36.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:36 smithi027 ceph-mon[97244]: pgmap v1348: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:38.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:38 smithi115 ceph-mon[65453]: pgmap v1349: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:38.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:38 smithi027 ceph-mon[97244]: pgmap v1349: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:40.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:40 smithi115 ceph-mon[65453]: pgmap v1350: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:40.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2622.422772 secs 2021-11-03T17:59:40.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:40 smithi027 ceph-mon[97244]: pgmap v1350: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:40.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2622.422772 secs 2021-11-03T17:59:42.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:42 smithi115 ceph-mon[65453]: pgmap v1351: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:42.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:42 smithi027 ceph-mon[97244]: pgmap v1351: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:42.662 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.664+0000 7efd2d171700 1 -- 172.21.15.27:0/2945302481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd280f4800 msgr2=0x7efd280f4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:59:42.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.664+0000 7efd2d171700 1 --2- 172.21.15.27:0/2945302481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd280f4800 0x7efd280f4c20 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7efd1c004660 tx=0x7efd1c009b30).stop 2021-11-03T17:59:42.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.665+0000 7efd2d171700 1 -- 172.21.15.27:0/2945302481 shutdown_connections 2021-11-03T17:59:42.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.665+0000 7efd2d171700 1 --2- 172.21.15.27:0/2945302481 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd280f51a0 0x7efd280f7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:42.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.665+0000 7efd2d171700 1 --2- 172.21.15.27:0/2945302481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd280f4800 0x7efd280f4c20 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:42.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.665+0000 7efd2d171700 1 -- 172.21.15.27:0/2945302481 >> 172.21.15.27:0/2945302481 conn(0x7efd280f0320 msgr2=0x7efd280f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:59:42.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.665+0000 7efd2d171700 1 -- 172.21.15.27:0/2945302481 shutdown_connections 2021-11-03T17:59:42.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.665+0000 7efd2d171700 1 -- 172.21.15.27:0/2945302481 wait complete. 2021-11-03T17:59:42.665 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.665+0000 7efd2d171700 1 Processor -- start 2021-11-03T17:59:42.665 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.665+0000 7efd2d171700 1 -- start start 2021-11-03T17:59:42.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.666+0000 7efd2d171700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd280f4800 0x7efd28106dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:59:42.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.666+0000 7efd2d171700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd280f51a0 0x7efd281072b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:59:42.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.666+0000 7efd2d171700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd281078b0 con 0x7efd280f51a0 2021-11-03T17:59:42.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.666+0000 7efd2d171700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd281080f0 con 0x7efd280f4800 2021-11-03T17:59:42.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.666+0000 7efd277fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd280f51a0 0x7efd281072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:59:42.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.666+0000 7efd277fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd280f51a0 0x7efd281072b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60552/0 (socket says 172.21.15.27:60552) 2021-11-03T17:59:42.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.666+0000 7efd277fe700 1 -- 172.21.15.27:0/3402051749 learned_addr learned my addr 172.21.15.27:0/3402051749 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:59:42.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.666+0000 7efd27fff700 1 --2- 172.21.15.27:0/3402051749 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd280f4800 0x7efd28106dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:59:42.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.666+0000 7efd277fe700 1 -- 172.21.15.27:0/3402051749 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd280f4800 msgr2=0x7efd28106dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:59:42.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.666+0000 7efd277fe700 1 --2- 172.21.15.27:0/3402051749 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd280f4800 0x7efd28106dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:42.670 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.666+0000 7efd277fe700 1 -- 172.21.15.27:0/3402051749 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd1c005040 con 0x7efd280f51a0 2021-11-03T17:59:42.670 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.666+0000 7efd277fe700 1 --2- 172.21.15.27:0/3402051749 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd280f51a0 0x7efd281072b0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7efd18002700 tx=0x7efd18009d60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:59:42.670 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.666+0000 7efd24ff9700 1 -- 172.21.15.27:0/3402051749 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd180079c0 con 0x7efd280f51a0 2021-11-03T17:59:42.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.667+0000 7efd2d171700 1 -- 172.21.15.27:0/3402051749 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd281083d0 con 0x7efd280f51a0 2021-11-03T17:59:42.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.667+0000 7efd24ff9700 1 -- 172.21.15.27:0/3402051749 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7efd1800f040 con 0x7efd280f51a0 2021-11-03T17:59:42.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.667+0000 7efd24ff9700 1 -- 172.21.15.27:0/3402051749 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd18014740 con 0x7efd280f51a0 2021-11-03T17:59:42.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.667+0000 7efd2d171700 1 -- 172.21.15.27:0/3402051749 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd28108910 con 0x7efd280f51a0 2021-11-03T17:59:42.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.667+0000 7efd2d171700 1 -- 172.21.15.27:0/3402051749 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd28059100 con 0x7efd280f51a0 2021-11-03T17:59:42.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.668+0000 7efd24ff9700 1 -- 172.21.15.27:0/3402051749 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7efd1801b060 con 0x7efd280f51a0 2021-11-03T17:59:42.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.668+0000 7efd24ff9700 1 --2- 172.21.15.27:0/3402051749 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7efd10064100 0x7efd100665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:59:42.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.668+0000 7efd24ff9700 1 -- 172.21.15.27:0/3402051749 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7efd180806d0 con 0x7efd280f51a0 2021-11-03T17:59:42.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.669+0000 7efd27fff700 1 --2- 172.21.15.27:0/3402051749 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7efd10064100 0x7efd100665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:59:42.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.669+0000 7efd27fff700 1 --2- 172.21.15.27:0/3402051749 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7efd10064100 0x7efd100665c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7efd1c013690 tx=0x7efd1c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:59:42.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.673+0000 7efd24ff9700 1 -- 172.21.15.27:0/3402051749 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7efd18052a80 con 0x7efd280f51a0 2021-11-03T17:59:42.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.856+0000 7efd2d171700 1 -- 172.21.15.27:0/3402051749 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7efd280fee70 con 0x7efd10064100 2021-11-03T17:59:42.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.857+0000 7efd24ff9700 1 -- 172.21.15.27:0/3402051749 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7efd280fee70 con 0x7efd10064100 2021-11-03T17:59:42.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.859+0000 7efd2d171700 1 -- 172.21.15.27:0/3402051749 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7efd10064100 msgr2=0x7efd100665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:59:42.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.859+0000 7efd2d171700 1 --2- 172.21.15.27:0/3402051749 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7efd10064100 0x7efd100665c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7efd1c013690 tx=0x7efd1c009b30).stop 2021-11-03T17:59:42.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.859+0000 7efd2d171700 1 -- 172.21.15.27:0/3402051749 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd280f51a0 msgr2=0x7efd281072b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:59:42.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.859+0000 7efd2d171700 1 --2- 172.21.15.27:0/3402051749 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd280f51a0 0x7efd281072b0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 rx=0x7efd18002700 tx=0x7efd18009d60).stop 2021-11-03T17:59:42.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.859+0000 7efd2d171700 1 -- 172.21.15.27:0/3402051749 shutdown_connections 2021-11-03T17:59:42.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.859+0000 7efd2d171700 1 --2- 172.21.15.27:0/3402051749 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7efd10064100 0x7efd100665c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:42.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.859+0000 7efd2d171700 1 --2- 172.21.15.27:0/3402051749 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd280f4800 0x7efd28106dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:42.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.859+0000 7efd2d171700 1 --2- 172.21.15.27:0/3402051749 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd280f51a0 0x7efd281072b0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:42.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.859+0000 7efd2d171700 1 -- 172.21.15.27:0/3402051749 >> 172.21.15.27:0/3402051749 conn(0x7efd280f0320 msgr2=0x7efd280fb370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:59:42.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.859+0000 7efd2d171700 1 -- 172.21.15.27:0/3402051749 shutdown_connections 2021-11-03T17:59:42.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.859+0000 7efd2d171700 1 -- 172.21.15.27:0/3402051749 wait complete. 2021-11-03T17:59:42.866 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T17:59:42.964 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.964+0000 7f7bdf628700 1 -- 172.21.15.27:0/971959187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bd80f6bc0 msgr2=0x7f7bd80f6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:59:42.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.964+0000 7f7bdf628700 1 --2- 172.21.15.27:0/971959187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bd80f6bc0 0x7f7bd80f6fe0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7f7bcc004660 tx=0x7f7bcc009b30).stop 2021-11-03T17:59:42.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.964+0000 7f7bdf628700 1 -- 172.21.15.27:0/971959187 shutdown_connections 2021-11-03T17:59:42.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.964+0000 7f7bdf628700 1 --2- 172.21.15.27:0/971959187 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bd80f7560 0x7f7bd80f59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:42.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.964+0000 7f7bdf628700 1 --2- 172.21.15.27:0/971959187 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bd80f6bc0 0x7f7bd80f6fe0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:42.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.964+0000 7f7bdf628700 1 -- 172.21.15.27:0/971959187 >> 172.21.15.27:0/971959187 conn(0x7f7bd80f04c0 msgr2=0x7f7bd80f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:59:42.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.964+0000 7f7bdf628700 1 -- 172.21.15.27:0/971959187 shutdown_connections 2021-11-03T17:59:42.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.964+0000 7f7bdf628700 1 -- 172.21.15.27:0/971959187 wait complete. 2021-11-03T17:59:42.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.965+0000 7f7bdf628700 1 Processor -- start 2021-11-03T17:59:42.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.965+0000 7f7bdf628700 1 -- start start 2021-11-03T17:59:42.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.965+0000 7f7bdf628700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bd80f6bc0 0x7f7bd810c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:59:42.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.965+0000 7f7bdf628700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bd80f7560 0x7f7bd810e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:59:42.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.965+0000 7f7bdf628700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd80f5530 con 0x7f7bd80f6bc0 2021-11-03T17:59:42.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.965+0000 7f7bdf628700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bd80f5670 con 0x7f7bd80f7560 2021-11-03T17:59:42.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.965+0000 7f7bdd3c4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bd80f6bc0 0x7f7bd810c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:59:42.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.965+0000 7f7bdd3c4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bd80f6bc0 0x7f7bd810c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60560/0 (socket says 172.21.15.27:60560) 2021-11-03T17:59:42.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.965+0000 7f7bdd3c4700 1 -- 172.21.15.27:0/161239606 learned_addr learned my addr 172.21.15.27:0/161239606 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:59:42.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.965+0000 7f7bdcbc3700 1 --2- 172.21.15.27:0/161239606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bd80f7560 0x7f7bd810e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:59:42.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.965+0000 7f7bdd3c4700 1 -- 172.21.15.27:0/161239606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bd80f7560 msgr2=0x7f7bd810e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:59:42.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.965+0000 7f7bdd3c4700 1 --2- 172.21.15.27:0/161239606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bd80f7560 0x7f7bd810e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:42.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.965+0000 7f7bdd3c4700 1 -- 172.21.15.27:0/161239606 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bcc005040 con 0x7f7bd80f6bc0 2021-11-03T17:59:42.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.965+0000 7f7bdd3c4700 1 --2- 172.21.15.27:0/161239606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bd80f6bc0 0x7f7bd810c0e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7f7bcc013a70 tx=0x7f7bcc009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:59:42.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.966+0000 7f7bc9ffb700 1 -- 172.21.15.27:0/161239606 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bcc018070 con 0x7f7bd80f6bc0 2021-11-03T17:59:42.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.966+0000 7f7bc9ffb700 1 -- 172.21.15.27:0/161239606 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f7bcc0097d0 con 0x7f7bd80f6bc0 2021-11-03T17:59:42.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.966+0000 7f7bdf628700 1 -- 172.21.15.27:0/161239606 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bd810ead0 con 0x7f7bd80f6bc0 2021-11-03T17:59:42.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.966+0000 7f7bc9ffb700 1 -- 172.21.15.27:0/161239606 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bcc01de80 con 0x7f7bd80f6bc0 2021-11-03T17:59:42.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.966+0000 7f7bdf628700 1 -- 172.21.15.27:0/161239606 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bd810eea0 con 0x7f7bd80f6bc0 2021-11-03T17:59:42.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.967+0000 7f7bdf628700 1 -- 172.21.15.27:0/161239606 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bd8045ad0 con 0x7f7bd80f6bc0 2021-11-03T17:59:42.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.967+0000 7f7bc9ffb700 1 -- 172.21.15.27:0/161239606 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f7bcc007ba0 con 0x7f7bd80f6bc0 2021-11-03T17:59:42.973 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.968+0000 7f7bc9ffb700 1 --2- 172.21.15.27:0/161239606 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7bc40640b0 0x7f7bc4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:59:42.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.968+0000 7f7bc9ffb700 1 -- 172.21.15.27:0/161239606 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f7bcc082410 con 0x7f7bd80f6bc0 2021-11-03T17:59:42.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.968+0000 7f7bdcbc3700 1 --2- 172.21.15.27:0/161239606 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7bc40640b0 0x7f7bc4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:59:42.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.968+0000 7f7bdcbc3700 1 --2- 172.21.15.27:0/161239606 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7bc40640b0 0x7f7bc4066570 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f7bd80f79a0 tx=0x7f7bd400b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:59:42.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:42.972+0000 7f7bc9ffb700 1 -- 172.21.15.27:0/161239606 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f7bcc0547d0 con 0x7f7bd80f6bc0 2021-11-03T17:59:43.128 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:59:43.148 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:59:43.150 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.153+0000 7f7bdf628700 1 -- 172.21.15.27:0/161239606 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7bd8057f10 con 0x7f7bc40640b0 2021-11-03T17:59:43.168 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.170+0000 7f7bc9ffb700 1 -- 172.21.15.27:0/161239606 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f7bd8057f10 con 0x7f7bc40640b0 2021-11-03T17:59:43.168 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T17:59:43.168 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (51m) 40s ago 52m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T17:59:43.169 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (42m) 40s ago 52m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T17:59:43.169 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (42m) 9m ago 51m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T17:59:43.170 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (51m) 40s ago 52m 41.8M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T17:59:43.170 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (44m) 40s ago 47m 26.0M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T17:59:43.170 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (47m) 40s ago 47m 3845M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T17:59:43.170 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (43m) 9m ago 47m 23.2M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T17:59:43.171 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (45m) 9m ago 47m 24.9M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T17:59:43.171 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (44m) 40s ago 53m 554M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T17:59:43.171 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (44m) 9m ago 51m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T17:59:43.171 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (43m) 40s ago 53m 235M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T17:59:43.172 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (43m) 9m ago 51m 95.6M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T17:59:43.172 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (52m) 40s ago 52m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T17:59:43.172 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (51m) 9m ago 51m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T17:59:43.172 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (42m) 40s ago 50m 351M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T17:59:43.172 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (42m) 40s ago 50m 383M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T17:59:43.173 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (41m) 40s ago 50m 284M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T17:59:43.173 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (41m) 9m ago 49m 379M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T17:59:43.174 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (40m) 9m ago 49m 304M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T17:59:43.174 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (40m) 9m ago 49m 307M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T17:59:43.174 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (51m) 40s ago 51m 55.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T17:59:43.175 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.174+0000 7f7bdf628700 1 -- 172.21.15.27:0/161239606 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7bc40640b0 msgr2=0x7f7bc4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:59:43.175 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.174+0000 7f7bdf628700 1 --2- 172.21.15.27:0/161239606 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7bc40640b0 0x7f7bc4066570 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 rx=0x7f7bd80f79a0 tx=0x7f7bd400b040).stop 2021-11-03T17:59:43.176 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.174+0000 7f7bdf628700 1 -- 172.21.15.27:0/161239606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bd80f6bc0 msgr2=0x7f7bd810c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:59:43.176 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.174+0000 7f7bdf628700 1 --2- 172.21.15.27:0/161239606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bd80f6bc0 0x7f7bd810c0e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7f7bcc013a70 tx=0x7f7bcc009b30).stop 2021-11-03T17:59:43.176 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.174+0000 7f7bdf628700 1 -- 172.21.15.27:0/161239606 shutdown_connections 2021-11-03T17:59:43.176 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.174+0000 7f7bdf628700 1 --2- 172.21.15.27:0/161239606 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7bc40640b0 0x7f7bc4066570 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:43.177 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.174+0000 7f7bdf628700 1 --2- 172.21.15.27:0/161239606 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7bd80f6bc0 0x7f7bd810c0e0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:43.177 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.174+0000 7f7bdf628700 1 --2- 172.21.15.27:0/161239606 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7bd80f7560 0x7f7bd810e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:43.177 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.174+0000 7f7bdf628700 1 -- 172.21.15.27:0/161239606 >> 172.21.15.27:0/161239606 conn(0x7f7bd80f04c0 msgr2=0x7f7bd80fa410 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:59:43.177 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.174+0000 7f7bdf628700 1 -- 172.21.15.27:0/161239606 shutdown_connections 2021-11-03T17:59:43.178 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.174+0000 7f7bdf628700 1 -- 172.21.15.27:0/161239606 wait complete. 2021-11-03T17:59:43.275 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.276+0000 7f94865a1700 1 -- 172.21.15.27:0/1050274479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94800f66d0 msgr2=0x7f94800f6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:59:43.276 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.276+0000 7f94865a1700 1 --2- 172.21.15.27:0/1050274479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94800f66d0 0x7f94800f6b30 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7f9474004660 tx=0x7f9474009b30).stop 2021-11-03T17:59:43.276 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.276+0000 7f94865a1700 1 -- 172.21.15.27:0/1050274479 shutdown_connections 2021-11-03T17:59:43.276 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.276+0000 7f94865a1700 1 --2- 172.21.15.27:0/1050274479 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94800f66d0 0x7f94800f6b30 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:43.277 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.276+0000 7f94865a1700 1 --2- 172.21.15.27:0/1050274479 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94800f5430 0x7f94800f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:43.277 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.276+0000 7f94865a1700 1 -- 172.21.15.27:0/1050274479 >> 172.21.15.27:0/1050274479 conn(0x7f94800f0b90 msgr2=0x7f94800f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:59:43.277 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.276+0000 7f94865a1700 1 -- 172.21.15.27:0/1050274479 shutdown_connections 2021-11-03T17:59:43.277 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.276+0000 7f94865a1700 1 -- 172.21.15.27:0/1050274479 wait complete. 2021-11-03T17:59:43.278 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.277+0000 7f94865a1700 1 Processor -- start 2021-11-03T17:59:43.278 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.277+0000 7f94865a1700 1 -- start start 2021-11-03T17:59:43.278 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.277+0000 7f94865a1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94800f5430 0x7f9480109a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:59:43.278 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.277+0000 7f94865a1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94800f66d0 0x7f948010bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:59:43.279 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.277+0000 7f94865a1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94800603e0 con 0x7f94800f66d0 2021-11-03T17:59:43.279 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.277+0000 7f94865a1700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9480060520 con 0x7f94800f5430 2021-11-03T17:59:43.279 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.277+0000 7f9484d9e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94800f66d0 0x7f948010bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:59:43.279 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.277+0000 7f9484d9e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94800f66d0 0x7f948010bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60572/0 (socket says 172.21.15.27:60572) 2021-11-03T17:59:43.280 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.277+0000 7f9484d9e700 1 -- 172.21.15.27:0/361997245 learned_addr learned my addr 172.21.15.27:0/361997245 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:59:43.280 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.277+0000 7f948559f700 1 --2- 172.21.15.27:0/361997245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94800f5430 0x7f9480109a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:59:43.280 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.277+0000 7f9484d9e700 1 -- 172.21.15.27:0/361997245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94800f5430 msgr2=0x7f9480109a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:59:43.280 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.277+0000 7f9484d9e700 1 --2- 172.21.15.27:0/361997245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94800f5430 0x7f9480109a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:43.281 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.277+0000 7f9484d9e700 1 -- 172.21.15.27:0/361997245 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9474005040 con 0x7f94800f66d0 2021-11-03T17:59:43.281 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.277+0000 7f9484d9e700 1 --2- 172.21.15.27:0/361997245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94800f66d0 0x7f948010bf20 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7f9474009990 tx=0x7f9474009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:59:43.281 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.278+0000 7f947dffb700 1 -- 172.21.15.27:0/361997245 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9474018070 con 0x7f94800f66d0 2021-11-03T17:59:43.281 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.278+0000 7f94865a1700 1 -- 172.21.15.27:0/361997245 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f948010c470 con 0x7f94800f66d0 2021-11-03T17:59:43.282 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.278+0000 7f947dffb700 1 -- 172.21.15.27:0/361997245 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f9474007ba0 con 0x7f94800f66d0 2021-11-03T17:59:43.282 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.278+0000 7f947dffb700 1 -- 172.21.15.27:0/361997245 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f947401c3c0 con 0x7f94800f66d0 2021-11-03T17:59:43.282 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.278+0000 7f94865a1700 1 -- 172.21.15.27:0/361997245 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f948010c9d0 con 0x7f94800f66d0 2021-11-03T17:59:43.283 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.279+0000 7f94865a1700 1 -- 172.21.15.27:0/361997245 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9480045ad0 con 0x7f94800f66d0 2021-11-03T17:59:43.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.279+0000 7f947dffb700 1 -- 172.21.15.27:0/361997245 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f947401d420 con 0x7f94800f66d0 2021-11-03T17:59:43.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.280+0000 7f947dffb700 1 --2- 172.21.15.27:0/361997245 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9468064100 0x7f94680665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:59:43.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.280+0000 7f947dffb700 1 -- 172.21.15.27:0/361997245 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f9474082470 con 0x7f94800f66d0 2021-11-03T17:59:43.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.280+0000 7f948559f700 1 --2- 172.21.15.27:0/361997245 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9468064100 0x7f94680665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:59:43.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.280+0000 7f948559f700 1 --2- 172.21.15.27:0/361997245 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9468064100 0x7f94680665c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f9480060060 tx=0x7f9478009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:59:43.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.284+0000 7f947dffb700 1 -- 172.21.15.27:0/361997245 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f9474054710 con 0x7f94800f66d0 2021-11-03T17:59:43.512 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.515+0000 7f94865a1700 1 -- 172.21.15.27:0/361997245 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f94800fdc90 con 0x7f94800f66d0 2021-11-03T17:59:43.513 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.515+0000 7f947dffb700 1 -- 172.21.15.27:0/361997245 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f9474054300 con 0x7f94800f66d0 2021-11-03T17:59:43.513 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T17:59:43.514 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T17:59:43.514 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:59:43.514 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:59:43.514 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T17:59:43.515 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T17:59:43.515 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:59:43.515 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T17:59:43.515 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T17:59:43.516 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:59:43.516 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T17:59:43.516 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T17:59:43.516 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T17:59:43.517 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T17:59:43.517 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T17:59:43.517 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T17:59:43.518 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T17:59:43.518 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T17:59:43.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.517+0000 7f94865a1700 1 -- 172.21.15.27:0/361997245 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9468064100 msgr2=0x7f94680665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:59:43.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.517+0000 7f94865a1700 1 --2- 172.21.15.27:0/361997245 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9468064100 0x7f94680665c0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7f9480060060 tx=0x7f9478009040).stop 2021-11-03T17:59:43.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.517+0000 7f94865a1700 1 -- 172.21.15.27:0/361997245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94800f66d0 msgr2=0x7f948010bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:59:43.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.517+0000 7f94865a1700 1 --2- 172.21.15.27:0/361997245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94800f66d0 0x7f948010bf20 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7f9474009990 tx=0x7f9474009b30).stop 2021-11-03T17:59:43.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.517+0000 7f94865a1700 1 -- 172.21.15.27:0/361997245 shutdown_connections 2021-11-03T17:59:43.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.517+0000 7f94865a1700 1 --2- 172.21.15.27:0/361997245 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9468064100 0x7f94680665c0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:43.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.517+0000 7f94865a1700 1 --2- 172.21.15.27:0/361997245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94800f5430 0x7f9480109a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:43.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.517+0000 7f94865a1700 1 --2- 172.21.15.27:0/361997245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94800f66d0 0x7f948010bf20 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:43.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.517+0000 7f94865a1700 1 -- 172.21.15.27:0/361997245 >> 172.21.15.27:0/361997245 conn(0x7f94800f0b90 msgr2=0x7f94800fa120 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:59:43.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.517+0000 7f94865a1700 1 -- 172.21.15.27:0/361997245 shutdown_connections 2021-11-03T17:59:43.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.517+0000 7f94865a1700 1 -- 172.21.15.27:0/361997245 wait complete. 2021-11-03T17:59:43.614 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.617+0000 7f0c1ec4d700 1 -- 172.21.15.27:0/3155344223 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c180f66e0 msgr2=0x7f0c180f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:59:43.615 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.617+0000 7f0c1ec4d700 1 --2- 172.21.15.27:0/3155344223 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c180f66e0 0x7f0c180f6b00 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7f0c0c004660 tx=0x7f0c0c009b30).stop 2021-11-03T17:59:43.615 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.617+0000 7f0c1ec4d700 1 -- 172.21.15.27:0/3155344223 shutdown_connections 2021-11-03T17:59:43.615 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.617+0000 7f0c1ec4d700 1 --2- 172.21.15.27:0/3155344223 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c180f7980 0x7f0c180f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:43.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.617+0000 7f0c1ec4d700 1 --2- 172.21.15.27:0/3155344223 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c180f66e0 0x7f0c180f6b00 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:43.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.617+0000 7f0c1ec4d700 1 -- 172.21.15.27:0/3155344223 >> 172.21.15.27:0/3155344223 conn(0x7f0c180f1e40 msgr2=0x7f0c180f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:59:43.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.617+0000 7f0c1ec4d700 1 -- 172.21.15.27:0/3155344223 shutdown_connections 2021-11-03T17:59:43.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.617+0000 7f0c1ec4d700 1 -- 172.21.15.27:0/3155344223 wait complete. 2021-11-03T17:59:43.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.617+0000 7f0c1ec4d700 1 Processor -- start 2021-11-03T17:59:43.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.618+0000 7f0c1ec4d700 1 -- start start 2021-11-03T17:59:43.619 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.618+0000 7f0c1ec4d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c180f66e0 0x7f0c18106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:59:43.620 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.618+0000 7f0c1ec4d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c180f7980 0x7f0c18107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:59:43.620 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.618+0000 7f0c1ec4d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c18107a00 con 0x7f0c180f66e0 2021-11-03T17:59:43.620 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.618+0000 7f0c1ec4d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c18107b40 con 0x7f0c180f7980 2021-11-03T17:59:43.620 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.618+0000 7f0c1c9e9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c180f66e0 0x7f0c18106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:59:43.621 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.618+0000 7f0c1c9e9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c180f66e0 0x7f0c18106f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60580/0 (socket says 172.21.15.27:60580) 2021-11-03T17:59:43.621 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.618+0000 7f0c1c9e9700 1 -- 172.21.15.27:0/2520836380 learned_addr learned my addr 172.21.15.27:0/2520836380 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T17:59:43.621 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.618+0000 7f0c17fff700 1 --2- 172.21.15.27:0/2520836380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c180f7980 0x7f0c18107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:59:43.621 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.618+0000 7f0c1c9e9700 1 -- 172.21.15.27:0/2520836380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c180f7980 msgr2=0x7f0c18107450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:59:43.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.618+0000 7f0c1c9e9700 1 --2- 172.21.15.27:0/2520836380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c180f7980 0x7f0c18107450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:43.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.618+0000 7f0c1c9e9700 1 -- 172.21.15.27:0/2520836380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c0c005040 con 0x7f0c180f66e0 2021-11-03T17:59:43.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.618+0000 7f0c17fff700 1 --2- 172.21.15.27:0/2520836380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c180f7980 0x7f0c18107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T17:59:43.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.618+0000 7f0c1c9e9700 1 --2- 172.21.15.27:0/2520836380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c180f66e0 0x7f0c18106f60 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7f0c0c014040 tx=0x7f0c0c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:59:43.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.619+0000 7f0c157fa700 1 -- 172.21.15.27:0/2520836380 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c0c019070 con 0x7f0c180f66e0 2021-11-03T17:59:43.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.619+0000 7f0c157fa700 1 -- 172.21.15.27:0/2520836380 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f0c0c007e70 con 0x7f0c180f66e0 2021-11-03T17:59:43.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.619+0000 7f0c1ec4d700 1 -- 172.21.15.27:0/2520836380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c18108480 con 0x7f0c180f66e0 2021-11-03T17:59:43.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.619+0000 7f0c157fa700 1 -- 172.21.15.27:0/2520836380 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c0c014d50 con 0x7f0c180f66e0 2021-11-03T17:59:43.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.619+0000 7f0c1ec4d700 1 -- 172.21.15.27:0/2520836380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c181089e0 con 0x7f0c180f66e0 2021-11-03T17:59:43.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.620+0000 7f0c1ec4d700 1 -- 172.21.15.27:0/2520836380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c18045ad0 con 0x7f0c180f66e0 2021-11-03T17:59:43.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.620+0000 7f0c157fa700 1 -- 172.21.15.27:0/2520836380 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f0c0c025070 con 0x7f0c180f66e0 2021-11-03T17:59:43.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.620+0000 7f0c157fa700 1 --2- 172.21.15.27:0/2520836380 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0c080640b0 0x7f0c08066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T17:59:43.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.620+0000 7f0c157fa700 1 -- 172.21.15.27:0/2520836380 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f0c0c081820 con 0x7f0c180f66e0 2021-11-03T17:59:43.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.620+0000 7f0c17fff700 1 --2- 172.21.15.27:0/2520836380 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0c080640b0 0x7f0c08066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T17:59:43.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.621+0000 7f0c17fff700 1 --2- 172.21.15.27:0/2520836380 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0c080640b0 0x7f0c08066570 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f0c04009da0 tx=0x7f0c0400b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T17:59:43.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.624+0000 7f0c157fa700 1 -- 172.21.15.27:0/2520836380 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f0c0c053b20 con 0x7f0c180f66e0 2021-11-03T17:59:43.825 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.827+0000 7f0c1ec4d700 1 -- 172.21.15.27:0/2520836380 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0c18027a00 con 0x7f0c180f66e0 2021-11-03T17:59:43.825 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.828+0000 7f0c157fa700 1 -- 172.21.15.27:0/2520836380 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f0c0c023020 con 0x7f0c180f66e0 2021-11-03T17:59:43.826 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T17:59:43.826 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T17:59:43.826 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:59:43.827 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T17:59:43.827 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:59:43.827 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T17:59:43.827 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T17:59:43.827 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T17:59:43.828 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T17:59:43.828 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T17:59:43.828 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T17:59:43.828 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T17:59:43.829 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T17:59:43.829 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T17:59:43.829 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T17:59:43.829 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T17:59:43.829 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T17:59:43.830 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T17:59:43.830 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T17:59:43.830 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T17:59:43.830 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T17:59:43.831 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T17:59:43.831 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T17:59:43.831 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T17:59:43.831 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T17:59:43.832 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T17:59:43.833 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T17:59:43.833 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T17:59:43.833 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T17:59:43.833 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T17:59:43.834 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T17:59:43.834 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T17:59:43.834 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:59:43.834 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:59:43.835 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T17:59:43.835 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T17:59:43.835 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:59:43.835 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:59:43.835 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T17:59:43.836 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.830+0000 7f0c1ec4d700 1 -- 172.21.15.27:0/2520836380 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0c080640b0 msgr2=0x7f0c08066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:59:43.837 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.830+0000 7f0c1ec4d700 1 --2- 172.21.15.27:0/2520836380 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0c080640b0 0x7f0c08066570 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7f0c04009da0 tx=0x7f0c0400b040).stop 2021-11-03T17:59:43.837 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.830+0000 7f0c1ec4d700 1 -- 172.21.15.27:0/2520836380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c180f66e0 msgr2=0x7f0c18106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T17:59:43.837 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.830+0000 7f0c1ec4d700 1 --2- 172.21.15.27:0/2520836380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c180f66e0 0x7f0c18106f60 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7f0c0c014040 tx=0x7f0c0c00b040).stop 2021-11-03T17:59:43.837 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.830+0000 7f0c1ec4d700 1 -- 172.21.15.27:0/2520836380 shutdown_connections 2021-11-03T17:59:43.838 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.830+0000 7f0c1ec4d700 1 --2- 172.21.15.27:0/2520836380 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0c080640b0 0x7f0c08066570 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:43.838 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.830+0000 7f0c1ec4d700 1 --2- 172.21.15.27:0/2520836380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c180f66e0 0x7f0c18106f60 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:43.838 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.830+0000 7f0c1ec4d700 1 --2- 172.21.15.27:0/2520836380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c180f7980 0x7f0c18107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T17:59:43.838 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.830+0000 7f0c1ec4d700 1 -- 172.21.15.27:0/2520836380 >> 172.21.15.27:0/2520836380 conn(0x7f0c180f1e40 msgr2=0x7f0c180fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T17:59:43.839 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.830+0000 7f0c1ec4d700 1 -- 172.21.15.27:0/2520836380 shutdown_connections 2021-11-03T17:59:43.839 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T17:59:43.830+0000 7f0c1ec4d700 1 -- 172.21.15.27:0/2520836380 wait complete. 2021-11-03T17:59:43.839 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T17:59:44.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:44 smithi115 ceph-mon[65453]: pgmap v1352: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:44.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:44 smithi115 ceph-mon[65453]: poll took 40119.622 ms: 1 events 2021-11-03T17:59:44.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:44 smithi115 ceph-mon[65453]: from='client.35496 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:59:44.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:44 smithi115 ceph-mon[65453]: from='client.35500 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:59:44.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:59:44.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:59:44.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:44.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:44.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:44.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:44.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:44 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:59:44.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:44 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/361997245' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:44.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:44 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2520836380' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:59:44.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:44 smithi027 ceph-mon[97244]: pgmap v1352: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:44.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:44 smithi027 ceph-mon[97244]: poll took 40119.622 ms: 1 events 2021-11-03T17:59:44.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:44 smithi027 ceph-mon[97244]: from='client.35496 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:59:44.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:44 smithi027 ceph-mon[97244]: from='client.35500 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T17:59:44.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:59:44.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:59:44.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:44.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:44.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:44.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:44.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:44 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:59:44.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:44 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/361997245' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:44.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:44 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2520836380' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T17:59:45.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2627.422866 secs 2021-11-03T17:59:45.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2627.422866 secs 2021-11-03T17:59:46.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:46 smithi115 ceph-mon[65453]: pgmap v1353: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:46.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:46 smithi027 ceph-mon[97244]: pgmap v1353: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:48.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:48 smithi115 ceph-mon[65453]: pgmap v1354: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:48.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:48 smithi027 ceph-mon[97244]: pgmap v1354: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:50.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:50 smithi115 ceph-mon[65453]: pgmap v1355: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:50.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2632.422936 secs 2021-11-03T17:59:50.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:50 smithi027 ceph-mon[97244]: pgmap v1355: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:50.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2632.422936 secs 2021-11-03T17:59:52.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:52 smithi115 ceph-mon[65453]: pgmap v1356: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:52.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:52 smithi027 ceph-mon[97244]: pgmap v1356: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:54.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:54 smithi115 ceph-mon[65453]: pgmap v1357: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:54.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:59:54.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:54 smithi115 ceph-mon[65453]: poll took 10043.680 ms: 1 events 2021-11-03T17:59:54.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:54 smithi027 ceph-mon[97244]: pgmap v1357: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:54.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:59:54.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:54 smithi027 ceph-mon[97244]: poll took 10043.680 ms: 1 events 2021-11-03T17:59:55.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2637.423067 secs 2021-11-03T17:59:55.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2637.423067 secs 2021-11-03T17:59:56.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:56 smithi115 ceph-mon[65453]: pgmap v1358: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:56.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:59:56.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:59:56.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:59:56.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:59:56.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:59:56.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:56.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:56 smithi027 ceph-mon[97244]: pgmap v1358: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:56.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T17:59:56.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T17:59:56.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:59:56.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T17:59:56.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T17:59:56.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:57.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:57 smithi115 ceph-mon[65453]: poll took 2904.785 ms: 1 events 2021-11-03T17:59:57.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:57.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:57.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:57.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:57 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:59:57.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:57 smithi027 ceph-mon[97244]: poll took 2904.785 ms: 1 events 2021-11-03T17:59:57.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:57.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:57.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T17:59:57.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:57 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T17:59:58.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 17:59:58 smithi115 ceph-mon[65453]: pgmap v1359: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T17:59:58.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 17:59:58 smithi027 ceph-mon[97244]: pgmap v1359: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:00.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:00 smithi115 ceph-mon[65453]: pgmap v1360: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:00.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2642.423154 secs 2021-11-03T18:00:00.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:00 smithi115 ceph-mon[65453]: overall HEALTH_WARN 1 filesystem is degraded; 1 filesystem has a failed mds daemon; 1 MDSs report slow requests; 3 daemons have recently crashed 2021-11-03T18:00:00.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:00 smithi027 ceph-mon[97244]: pgmap v1360: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:00.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2642.423154 secs 2021-11-03T18:00:00.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:00 smithi027 ceph-mon[97244]: overall HEALTH_WARN 1 filesystem is degraded; 1 filesystem has a failed mds daemon; 1 MDSs report slow requests; 3 daemons have recently crashed 2021-11-03T18:00:02.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:02 smithi115 ceph-mon[65453]: pgmap v1361: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:02.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:02 smithi027 ceph-mon[97244]: pgmap v1361: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:04.235 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:04 smithi115 ceph-mon[65453]: pgmap v1362: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:04.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:04 smithi027 ceph-mon[97244]: pgmap v1362: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:05.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2647.423222 secs 2021-11-03T18:00:05.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2647.423222 secs 2021-11-03T18:00:06.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:06 smithi115 ceph-mon[65453]: pgmap v1363: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:06.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:00:06.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:00:06.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:00:06.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:06.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:06 smithi027 ceph-mon[97244]: pgmap v1363: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:06.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:00:06.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:00:06.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:00:06.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:07.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:07.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:07.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:07.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:07 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:00:07.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:07.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:07.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:07.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:07 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:00:08.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:08 smithi115 ceph-mon[65453]: pgmap v1364: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:08.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:08 smithi027 ceph-mon[97244]: pgmap v1364: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:10.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:10 smithi115 ceph-mon[65453]: pgmap v1365: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:10.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2652.423297 secs 2021-11-03T18:00:10.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:10 smithi027 ceph-mon[97244]: pgmap v1365: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:10.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2652.423297 secs 2021-11-03T18:00:12.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:12 smithi115 ceph-mon[65453]: pgmap v1366: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:12.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:12 smithi027 ceph-mon[97244]: pgmap v1366: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:13.944 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:13.945+0000 7f40ade83700 1 -- 172.21.15.27:0/722882248 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40a80f5340 msgr2=0x7f40a80f7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:00:13.945 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:13.945+0000 7f40ade83700 1 --2- 172.21.15.27:0/722882248 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40a80f5340 0x7f40a80f7ac0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f409c004660 tx=0x7f409c009b30).stop 2021-11-03T18:00:13.945 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:13.945+0000 7f40ade83700 1 -- 172.21.15.27:0/722882248 shutdown_connections 2021-11-03T18:00:13.945 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:13.945+0000 7f40ade83700 1 --2- 172.21.15.27:0/722882248 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40a80f5340 0x7f40a80f7ac0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:13.946 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:13.945+0000 7f40ade83700 1 --2- 172.21.15.27:0/722882248 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40a80f49a0 0x7f40a80f4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:13.946 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:13.945+0000 7f40ade83700 1 -- 172.21.15.27:0/722882248 >> 172.21.15.27:0/722882248 conn(0x7f40a80f04c0 msgr2=0x7f40a80f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:00:13.946 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:13.945+0000 7f40ade83700 1 -- 172.21.15.27:0/722882248 shutdown_connections 2021-11-03T18:00:13.946 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:13.945+0000 7f40ade83700 1 -- 172.21.15.27:0/722882248 wait complete. 2021-11-03T18:00:13.947 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:13.946+0000 7f40ade83700 1 Processor -- start 2021-11-03T18:00:13.947 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:13.946+0000 7f40ade83700 1 -- start start 2021-11-03T18:00:13.947 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:13.946+0000 7f40ade83700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40a80f49a0 0x7f40a8106f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:00:13.947 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:13.946+0000 7f40ade83700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40a80f5340 0x7f40a8107410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:00:13.948 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:13.946+0000 7f40ade83700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a81079c0 con 0x7f40a80f5340 2021-11-03T18:00:13.948 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:13.946+0000 7f40ade83700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a8107b00 con 0x7f40a80f49a0 2021-11-03T18:00:13.948 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:13.947+0000 7f40a6ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40a80f5340 0x7f40a8107410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:00:13.948 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:13.947+0000 7f40a6ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40a80f5340 0x7f40a8107410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60598/0 (socket says 172.21.15.27:60598) 2021-11-03T18:00:13.948 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:13.947+0000 7f40a6ffd700 1 -- 172.21.15.27:0/185200548 learned_addr learned my addr 172.21.15.27:0/185200548 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:00:13.949 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:13.947+0000 7f40a77fe700 1 --2- 172.21.15.27:0/185200548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40a80f49a0 0x7f40a8106f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:00:13.949 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:13.947+0000 7f40a6ffd700 1 -- 172.21.15.27:0/185200548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40a80f49a0 msgr2=0x7f40a8106f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:00:13.949 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:13.947+0000 7f40a6ffd700 1 --2- 172.21.15.27:0/185200548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40a80f49a0 0x7f40a8106f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:13.950 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:13.947+0000 7f40a6ffd700 1 -- 172.21.15.27:0/185200548 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409c005040 con 0x7f40a80f5340 2021-11-03T18:00:13.950 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:13.947+0000 7f40a6ffd700 1 --2- 172.21.15.27:0/185200548 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40a80f5340 0x7f40a8107410 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f409c009830 tx=0x7f409c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:00:13.950 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:13.947+0000 7f408ffff700 1 -- 172.21.15.27:0/185200548 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f409c018070 con 0x7f40a80f5340 2021-11-03T18:00:13.950 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:13.947+0000 7f408ffff700 1 -- 172.21.15.27:0/185200548 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f409c007ba0 con 0x7f40a80f5340 2021-11-03T18:00:13.951 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:13.947+0000 7f40ade83700 1 -- 172.21.15.27:0/185200548 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a8108440 con 0x7f40a80f5340 2021-11-03T18:00:13.951 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:13.947+0000 7f408ffff700 1 -- 172.21.15.27:0/185200548 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f409c01c620 con 0x7f40a80f5340 2021-11-03T18:00:13.951 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:13.947+0000 7f40ade83700 1 -- 172.21.15.27:0/185200548 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a81089a0 con 0x7f40a80f5340 2021-11-03T18:00:13.952 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:13.948+0000 7f40ade83700 1 -- 172.21.15.27:0/185200548 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40a8045ad0 con 0x7f40a80f5340 2021-11-03T18:00:13.953 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:13.949+0000 7f408ffff700 1 -- 172.21.15.27:0/185200548 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f409c007d10 con 0x7f40a80f5340 2021-11-03T18:00:13.953 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:13.949+0000 7f408ffff700 1 --2- 172.21.15.27:0/185200548 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4090064090 0x7f4090066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:00:13.953 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:13.949+0000 7f408ffff700 1 -- 172.21.15.27:0/185200548 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f409c082770 con 0x7f40a80f5340 2021-11-03T18:00:13.953 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:13.949+0000 7f40a77fe700 1 --2- 172.21.15.27:0/185200548 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4090064090 0x7f4090066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:00:13.954 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:13.950+0000 7f40a77fe700 1 --2- 172.21.15.27:0/185200548 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4090064090 0x7f4090066550 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f40a805f480 tx=0x7f4098009c80).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:00:13.954 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:13.953+0000 7f408ffff700 1 -- 172.21.15.27:0/185200548 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f409c054b30 con 0x7f40a80f5340 2021-11-03T18:00:14.131 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.133+0000 7f40ade83700 1 -- 172.21.15.27:0/185200548 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f40a8108bc0 con 0x7f4090064090 2021-11-03T18:00:14.132 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.135+0000 7f408ffff700 1 -- 172.21.15.27:0/185200548 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f40a8108bc0 con 0x7f4090064090 2021-11-03T18:00:14.134 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.136+0000 7f40ade83700 1 -- 172.21.15.27:0/185200548 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4090064090 msgr2=0x7f4090066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:00:14.134 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.136+0000 7f40ade83700 1 --2- 172.21.15.27:0/185200548 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4090064090 0x7f4090066550 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 rx=0x7f40a805f480 tx=0x7f4098009c80).stop 2021-11-03T18:00:14.135 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.136+0000 7f40ade83700 1 -- 172.21.15.27:0/185200548 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40a80f5340 msgr2=0x7f40a8107410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:00:14.135 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.136+0000 7f40ade83700 1 --2- 172.21.15.27:0/185200548 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40a80f5340 0x7f40a8107410 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f409c009830 tx=0x7f409c009b30).stop 2021-11-03T18:00:14.135 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.137+0000 7f40ade83700 1 -- 172.21.15.27:0/185200548 shutdown_connections 2021-11-03T18:00:14.136 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.137+0000 7f40ade83700 1 --2- 172.21.15.27:0/185200548 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4090064090 0x7f4090066550 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:14.136 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.137+0000 7f40ade83700 1 --2- 172.21.15.27:0/185200548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40a80f49a0 0x7f40a8106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:14.136 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.137+0000 7f40ade83700 1 --2- 172.21.15.27:0/185200548 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40a80f5340 0x7f40a8107410 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:14.136 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.137+0000 7f40ade83700 1 -- 172.21.15.27:0/185200548 >> 172.21.15.27:0/185200548 conn(0x7f40a80f04c0 msgr2=0x7f40a80fb510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:00:14.137 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.137+0000 7f40ade83700 1 -- 172.21.15.27:0/185200548 shutdown_connections 2021-11-03T18:00:14.137 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.137+0000 7f40ade83700 1 -- 172.21.15.27:0/185200548 wait complete. 2021-11-03T18:00:14.144 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:00:14.236 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.237+0000 7f1572a22700 1 -- 172.21.15.27:0/293145650 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f156c0f5e60 msgr2=0x7f156c0f62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:00:14.236 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.237+0000 7f1572a22700 1 --2- 172.21.15.27:0/293145650 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f156c0f5e60 0x7f156c0f62c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7f1560004660 tx=0x7f1560009b30).stop 2021-11-03T18:00:14.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.237+0000 7f1572a22700 1 -- 172.21.15.27:0/293145650 shutdown_connections 2021-11-03T18:00:14.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.237+0000 7f1572a22700 1 --2- 172.21.15.27:0/293145650 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f156c0f5e60 0x7f156c0f62c0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:14.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.237+0000 7f1572a22700 1 --2- 172.21.15.27:0/293145650 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f156c0f4bc0 0x7f156c0f4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T18:00:14.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.237+0000 7f1572a22700 1 -- 172.21.15.27:0/293145650 >> 172.21.15.27:0/293145650 conn(0x7f156c0f0320 msgr2=0x7f156c0f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:00:14.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.237+0000 7f1572a22700 1 -- 172.21.15.27:0/293145650 shutdown_connections 2021-11-03T18:00:14.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.237+0000 7f1572a22700 1 -- 172.21.15.27:0/293145650 wait complete. 2021-11-03T18:00:14.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.238+0000 7f1572a22700 1 Processor -- start 2021-11-03T18:00:14.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.238+0000 7f1572a22700 1 -- start start 2021-11-03T18:00:14.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.238+0000 7f1572a22700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f156c0f4bc0 0x7f156c109a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:00:14.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.238+0000 7f1572a22700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f156c0f5e60 0x7f156c10bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:00:14.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.238+0000 7f1572a22700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c0fbef0 con 0x7f156c0f4bc0 2021-11-03T18:00:14.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.238+0000 7f1572a22700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f156c0fc030 con 0x7f156c0f5e60 2021-11-03T18:00:14.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.238+0000 7f1571a20700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f156c0f4bc0 0x7f156c109a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:00:14.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.238+0000 7f1571a20700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f156c0f4bc0 0x7f156c109a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60606/0 (socket says 172.21.15.27:60606) 2021-11-03T18:00:14.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.238+0000 7f1571a20700 1 -- 172.21.15.27:0/1685110042 learned_addr learned my addr 172.21.15.27:0/1685110042 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:00:14.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.238+0000 7f157121f700 1 --2- 172.21.15.27:0/1685110042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f156c0f5e60 0x7f156c10bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:00:14.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.239+0000 7f1571a20700 1 -- 172.21.15.27:0/1685110042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f156c0f5e60 msgr2=0x7f156c10bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:00:14.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.239+0000 7f1571a20700 1 --2- 172.21.15.27:0/1685110042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f156c0f5e60 0x7f156c10bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:14.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.239+0000 7f1571a20700 1 -- 172.21.15.27:0/1685110042 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1560005040 con 0x7f156c0f4bc0 2021-11-03T18:00:14.242 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.239+0000 7f1571a20700 1 --2- 172.21.15.27:0/1685110042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f156c0f4bc0 0x7f156c109a80 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7f15640093f0 tx=0x7f1564006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:00:14.242 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.239+0000 7f156a7fc700 1 -- 172.21.15.27:0/1685110042 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1564013640 con 0x7f156c0f4bc0 2021-11-03T18:00:14.242 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.239+0000 7f156a7fc700 1 -- 172.21.15.27:0/1685110042 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f1564013c50 con 0x7f156c0f4bc0 2021-11-03T18:00:14.242 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.239+0000 7f1572a22700 1 -- 172.21.15.27:0/1685110042 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f156c10c5e0 con 0x7f156c0f4bc0 2021-11-03T18:00:14.243 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.239+0000 7f156a7fc700 1 -- 172.21.15.27:0/1685110042 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1564012c70 con 0x7f156c0f4bc0 2021-11-03T18:00:14.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.239+0000 7f1572a22700 1 -- 172.21.15.27:0/1685110042 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f156c10cb70 con 0x7f156c0f4bc0 2021-11-03T18:00:14.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.240+0000 7f1572a22700 1 -- 172.21.15.27:0/1685110042 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f156c059100 con 0x7f156c0f4bc0 2021-11-03T18:00:14.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.240+0000 7f156a7fc700 1 -- 172.21.15.27:0/1685110042 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f156401a060 con 0x7f156c0f4bc0 2021-11-03T18:00:14.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.241+0000 7f156a7fc700 1 --2- 172.21.15.27:0/1685110042 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1554064100 0x7f15540665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:00:14.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.241+0000 7f156a7fc700 1 -- 172.21.15.27:0/1685110042 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f1564080840 con 0x7f156c0f4bc0 2021-11-03T18:00:14.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.241+0000 7f157121f700 1 --2- 172.21.15.27:0/1685110042 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1554064100 0x7f15540665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:00:14.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.241+0000 7f157121f700 1 --2- 172.21.15.27:0/1685110042 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1554064100 0x7f15540665c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f1560004630 tx=0x7f1560009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:00:14.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.245+0000 7f156a7fc700 1 -- 172.21.15.27:0/1685110042 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f15640529f0 con 0x7f156c0f4bc0 2021-11-03T18:00:14.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.418+0000 7f1572a22700 1 -- 172.21.15.27:0/1685110042 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f156c0fddf0 con 0x7f1554064100 2021-11-03T18:00:14.430 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.433+0000 7f156a7fc700 1 -- 172.21.15.27:0/1685110042 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f156c0fddf0 con 0x7f1554064100 2021-11-03T18:00:14.430 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:00:14.431 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (51m) 71s ago 53m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:00:14.431 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (43m) 71s ago 52m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:00:14.431 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (43m) 18s ago 52m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:00:14.431 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (51m) 71s ago 52m 41.8M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:00:14.432 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (45m) 71s ago 48m 26.0M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:00:14.432 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (48m) 71s ago 48m 3845M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:00:14.432 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (44m) 18s ago 48m 25.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:00:14.432 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (45m) 18s ago 48m 26.8M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:00:14.432 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (45m) 71s ago 54m 554M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:00:14.433 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (44m) 18s ago 52m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:00:14.433 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (44m) 71s ago 54m 235M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:00:14.433 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (44m) 18s ago 52m 112M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:00:14.433 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (52m) 71s ago 52m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:00:14.434 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (52m) 18s ago 52m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:00:14.434 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (43m) 71s ago 51m 351M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:00:14.434 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (42m) 71s ago 51m 383M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:00:14.434 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (42m) 71s ago 50m 284M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:00:14.435 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (41m) 18s ago 50m 404M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:00:14.435 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (41m) 18s ago 50m 322M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:00:14.435 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (40m) 18s ago 49m 327M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:00:14.435 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (51m) 71s ago 52m 55.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:00:14.436 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.434+0000 7f1572a22700 1 -- 172.21.15.27:0/1685110042 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1554064100 msgr2=0x7f15540665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:00:14.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.434+0000 7f1572a22700 1 --2- 172.21.15.27:0/1685110042 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1554064100 0x7f15540665c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f1560004630 tx=0x7f1560009b30).stop 2021-11-03T18:00:14.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.434+0000 7f1572a22700 1 -- 172.21.15.27:0/1685110042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f156c0f4bc0 msgr2=0x7f156c109a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:00:14.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.434+0000 7f1572a22700 1 --2- 172.21.15.27:0/1685110042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f156c0f4bc0 0x7f156c109a80 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7f15640093f0 tx=0x7f1564006ec0).stop 2021-11-03T18:00:14.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.434+0000 7f1572a22700 1 -- 172.21.15.27:0/1685110042 shutdown_connections 2021-11-03T18:00:14.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.435+0000 7f1572a22700 1 --2- 172.21.15.27:0/1685110042 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1554064100 0x7f15540665c0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:14.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.435+0000 7f1572a22700 1 --2- 172.21.15.27:0/1685110042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f156c0f4bc0 0x7f156c109a80 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:14.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.435+0000 7f1572a22700 1 --2- 172.21.15.27:0/1685110042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f156c0f5e60 0x7f156c10bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:14.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.435+0000 7f1572a22700 1 -- 172.21.15.27:0/1685110042 >> 172.21.15.27:0/1685110042 conn(0x7f156c0f0320 msgr2=0x7f156c0f98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:00:14.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.435+0000 7f1572a22700 1 -- 172.21.15.27:0/1685110042 shutdown_connections 2021-11-03T18:00:14.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.435+0000 7f1572a22700 1 -- 172.21.15.27:0/1685110042 wait complete. 2021-11-03T18:00:14.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:14 smithi115 ceph-mon[65453]: pgmap v1367: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:14.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:14 smithi027 ceph-mon[97244]: pgmap v1367: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:14.531 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.534+0000 7f28a19bb700 1 -- 172.21.15.27:0/3899355792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f289c0f5ed0 msgr2=0x7f289c0f6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:00:14.531 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.534+0000 7f28a19bb700 1 --2- 172.21.15.27:0/3899355792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f289c0f5ed0 0x7f289c0f6900 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7f2884004660 tx=0x7f2884009b30).stop 2021-11-03T18:00:14.532 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.534+0000 7f28a19bb700 1 -- 172.21.15.27:0/3899355792 shutdown_connections 2021-11-03T18:00:14.532 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.534+0000 7f28a19bb700 1 --2- 172.21.15.27:0/3899355792 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f289c0f5ed0 0x7f289c0f6900 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:14.532 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.534+0000 7f28a19bb700 1 --2- 172.21.15.27:0/3899355792 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f289c0f55c0 0x7f289c0f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:14.532 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.534+0000 7f28a19bb700 1 -- 172.21.15.27:0/3899355792 >> 172.21.15.27:0/3899355792 conn(0x7f289c0f0de0 msgr2=0x7f289c0f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:00:14.533 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.534+0000 7f28a19bb700 1 -- 172.21.15.27:0/3899355792 shutdown_connections 2021-11-03T18:00:14.533 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.534+0000 7f28a19bb700 1 -- 172.21.15.27:0/3899355792 wait complete. 2021-11-03T18:00:14.535 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.535+0000 7f28a19bb700 1 Processor -- start 2021-11-03T18:00:14.535 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.535+0000 7f28a19bb700 1 -- start start 2021-11-03T18:00:14.536 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.535+0000 7f28a19bb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f289c0f55c0 0x7f289c109c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:00:14.536 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.535+0000 7f28a19bb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f289c0f5ed0 0x7f289c10c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:00:14.536 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.535+0000 7f28a19bb700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289c060520 con 0x7f289c0f5ed0 2021-11-03T18:00:14.536 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.535+0000 7f28a19bb700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289c060660 con 0x7f289c0f55c0 2021-11-03T18:00:14.537 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.535+0000 7f289a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f289c0f5ed0 0x7f289c10c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:00:14.537 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.535+0000 7f289a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f289c0f5ed0 0x7f289c10c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60618/0 (socket says 172.21.15.27:60618) 2021-11-03T18:00:14.537 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.535+0000 7f289a7fc700 1 -- 172.21.15.27:0/2648167609 learned_addr learned my addr 172.21.15.27:0/2648167609 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:00:14.537 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.535+0000 7f289a7fc700 1 -- 172.21.15.27:0/2648167609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f289c0f55c0 msgr2=0x7f289c109c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:00:14.537 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.535+0000 7f289a7fc700 1 --2- 172.21.15.27:0/2648167609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f289c0f55c0 0x7f289c109c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:00:14.538 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.535+0000 7f289a7fc700 1 -- 172.21.15.27:0/2648167609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2884005040 con 0x7f289c0f5ed0 2021-11-03T18:00:14.538 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.535+0000 7f289a7fc700 1 --2- 172.21.15.27:0/2648167609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f289c0f5ed0 0x7f289c10c160 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7f2884009990 tx=0x7f2884009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:00:14.538 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.535+0000 7f28937fe700 1 -- 172.21.15.27:0/2648167609 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2884018070 con 0x7f289c0f5ed0 2021-11-03T18:00:14.538 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.535+0000 7f28a19bb700 1 -- 172.21.15.27:0/2648167609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f289c10c6b0 con 0x7f289c0f5ed0 2021-11-03T18:00:14.539 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.535+0000 7f28937fe700 1 -- 172.21.15.27:0/2648167609 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f2884007ca0 con 0x7f289c0f5ed0 2021-11-03T18:00:14.539 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.535+0000 7f28937fe700 1 -- 172.21.15.27:0/2648167609 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f288401c620 con 0x7f289c0f5ed0 2021-11-03T18:00:14.539 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.536+0000 7f28a19bb700 1 -- 172.21.15.27:0/2648167609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f289c10cbe0 con 0x7f289c0f5ed0 2021-11-03T18:00:14.539 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.536+0000 7f28a19bb700 1 -- 172.21.15.27:0/2648167609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f289c0f4a50 con 0x7f289c0f5ed0 2021-11-03T18:00:14.540 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.537+0000 7f28937fe700 1 -- 172.21.15.27:0/2648167609 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f288401d960 con 0x7f289c0f5ed0 2021-11-03T18:00:14.540 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.537+0000 7f28937fe700 1 --2- 172.21.15.27:0/2648167609 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f28880640b0 0x7f2888066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:00:14.540 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.537+0000 7f28937fe700 1 -- 172.21.15.27:0/2648167609 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f2884082580 con 0x7f289c0f5ed0 2021-11-03T18:00:14.540 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.537+0000 7f289affd700 1 --2- 172.21.15.27:0/2648167609 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f28880640b0 0x7f2888066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:00:14.541 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.538+0000 7f289affd700 1 --2- 172.21.15.27:0/2648167609 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f28880640b0 0x7f2888066570 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f289c0601a0 tx=0x7f2894007a40).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:00:14.541 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.541+0000 7f28937fe700 1 -- 172.21.15.27:0/2648167609 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f2884054940 con 0x7f289c0f5ed0 2021-11-03T18:00:14.770 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.773+0000 7f28a19bb700 1 -- 172.21.15.27:0/2648167609 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f289c045ad0 con 0x7f289c0f5ed0 2021-11-03T18:00:14.771 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.773+0000 7f28937fe700 1 -- 172.21.15.27:0/2648167609 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f2884054530 con 0x7f289c0f5ed0 2021-11-03T18:00:14.771 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:00:14.771 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:00:14.771 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:00:14.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:00:14.772 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:00:14.772 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:00:14.772 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:00:14.773 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:00:14.773 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:00:14.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:00:14.773 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:00:14.774 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:00:14.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:00:14.774 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:00:14.774 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:00:14.775 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:00:14.775 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:00:14.775 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:00:14.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.775+0000 7f28a19bb700 1 -- 172.21.15.27:0/2648167609 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f28880640b0 msgr2=0x7f2888066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:00:14.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.775+0000 7f28a19bb700 1 --2- 172.21.15.27:0/2648167609 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f28880640b0 0x7f2888066570 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f289c0601a0 tx=0x7f2894007a40).stop 2021-11-03T18:00:14.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.775+0000 7f28a19bb700 1 -- 172.21.15.27:0/2648167609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f289c0f5ed0 msgr2=0x7f289c10c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:00:14.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.775+0000 7f28a19bb700 1 --2- 172.21.15.27:0/2648167609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f289c0f5ed0 0x7f289c10c160 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7f2884009990 tx=0x7f2884009b30).stop 2021-11-03T18:00:14.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.775+0000 7f28a19bb700 1 -- 172.21.15.27:0/2648167609 shutdown_connections 2021-11-03T18:00:14.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.775+0000 7f28a19bb700 1 --2- 172.21.15.27:0/2648167609 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f28880640b0 0x7f2888066570 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:14.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.775+0000 7f28a19bb700 1 --2- 172.21.15.27:0/2648167609 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f289c0f55c0 0x7f289c109c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:00:14.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.775+0000 7f28a19bb700 1 --2- 172.21.15.27:0/2648167609 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f289c0f5ed0 0x7f289c10c160 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:14.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.775+0000 7f28a19bb700 1 -- 172.21.15.27:0/2648167609 >> 172.21.15.27:0/2648167609 conn(0x7f289c0f0de0 msgr2=0x7f289c0fa350 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:00:14.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.775+0000 7f28a19bb700 1 -- 172.21.15.27:0/2648167609 shutdown_connections 2021-11-03T18:00:14.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.775+0000 7f28a19bb700 1 -- 172.21.15.27:0/2648167609 wait complete. 2021-11-03T18:00:14.872 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.875+0000 7f5636eec700 1 -- 172.21.15.27:0/3203221550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56300f66e0 msgr2=0x7f56300f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:00:14.873 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.875+0000 7f5636eec700 1 --2- 172.21.15.27:0/3203221550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56300f66e0 0x7f56300f6b00 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7f5624004660 tx=0x7f5624009b30).stop 2021-11-03T18:00:14.873 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.875+0000 7f5636eec700 1 -- 172.21.15.27:0/3203221550 shutdown_connections 2021-11-03T18:00:14.873 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.875+0000 7f5636eec700 1 --2- 172.21.15.27:0/3203221550 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56300f6ff0 0x7f56300f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:14.873 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.875+0000 7f5636eec700 1 --2- 172.21.15.27:0/3203221550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56300f66e0 0x7f56300f6b00 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:14.874 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.875+0000 7f5636eec700 1 -- 172.21.15.27:0/3203221550 >> 172.21.15.27:0/3203221550 conn(0x7f56300f1f00 msgr2=0x7f56300f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:00:14.874 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.875+0000 7f5636eec700 1 -- 172.21.15.27:0/3203221550 shutdown_connections 2021-11-03T18:00:14.874 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.875+0000 7f5636eec700 1 -- 172.21.15.27:0/3203221550 wait complete. 2021-11-03T18:00:14.875 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.875+0000 7f5636eec700 1 Processor -- start 2021-11-03T18:00:14.875 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.875+0000 7f5636eec700 1 -- start start 2021-11-03T18:00:14.877 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.876+0000 7f5636eec700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56300f66e0 0x7f5630106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:00:14.877 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.876+0000 7f5636eec700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56300f6ff0 0x7f5630107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:00:14.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.876+0000 7f5636eec700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5630107a00 con 0x7f56300f6ff0 2021-11-03T18:00:14.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.876+0000 7f5636eec700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5630107b40 con 0x7f56300f66e0 2021-11-03T18:00:14.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.876+0000 7f562ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56300f6ff0 0x7f5630107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:00:14.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.876+0000 7f562ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56300f6ff0 0x7f5630107450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60626/0 (socket says 172.21.15.27:60626) 2021-11-03T18:00:14.879 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.876+0000 7f562ffff700 1 -- 172.21.15.27:0/2706127012 learned_addr learned my addr 172.21.15.27:0/2706127012 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:00:14.879 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.876+0000 7f5634c88700 1 --2- 172.21.15.27:0/2706127012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56300f66e0 0x7f5630106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:00:14.879 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.876+0000 7f562ffff700 1 -- 172.21.15.27:0/2706127012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56300f66e0 msgr2=0x7f5630106f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:00:14.879 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.876+0000 7f562ffff700 1 --2- 172.21.15.27:0/2706127012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56300f66e0 0x7f5630106f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:14.879 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.876+0000 7f562ffff700 1 -- 172.21.15.27:0/2706127012 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5624005040 con 0x7f56300f6ff0 2021-11-03T18:00:14.880 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.876+0000 7f562ffff700 1 --2- 172.21.15.27:0/2706127012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56300f6ff0 0x7f5630107450 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7f561c00c530 tx=0x7f561c007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:00:14.880 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.877+0000 7f562d7fa700 1 -- 172.21.15.27:0/2706127012 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f561c00f650 con 0x7f56300f6ff0 2021-11-03T18:00:14.880 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.877+0000 7f5636eec700 1 -- 172.21.15.27:0/2706127012 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56301084e0 con 0x7f56300f6ff0 2021-11-03T18:00:14.880 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.877+0000 7f562d7fa700 1 -- 172.21.15.27:0/2706127012 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f561c00fc60 con 0x7f56300f6ff0 2021-11-03T18:00:14.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.877+0000 7f562d7fa700 1 -- 172.21.15.27:0/2706127012 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f561c012c20 con 0x7f56300f6ff0 2021-11-03T18:00:14.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.877+0000 7f5636eec700 1 -- 172.21.15.27:0/2706127012 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5630108aa0 con 0x7f56300f6ff0 2021-11-03T18:00:14.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.878+0000 7f5636eec700 1 -- 172.21.15.27:0/2706127012 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5630045ad0 con 0x7f56300f6ff0 2021-11-03T18:00:14.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.878+0000 7f562d7fa700 1 -- 172.21.15.27:0/2706127012 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f561c010070 con 0x7f56300f6ff0 2021-11-03T18:00:14.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.879+0000 7f562d7fa700 1 --2- 172.21.15.27:0/2706127012 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f56200640b0 0x7f5620066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:00:14.882 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.879+0000 7f562d7fa700 1 -- 172.21.15.27:0/2706127012 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f561c0806c0 con 0x7f56300f6ff0 2021-11-03T18:00:14.882 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.879+0000 7f5634c88700 1 --2- 172.21.15.27:0/2706127012 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f56200640b0 0x7f5620066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:00:14.882 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.879+0000 7f5634c88700 1 --2- 172.21.15.27:0/2706127012 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f56200640b0 0x7f5620066570 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f5624004630 tx=0x7f5624009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:00:14.883 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:14.883+0000 7f562d7fa700 1 -- 172.21.15.27:0/2706127012 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f561c0528d0 con 0x7f56300f6ff0 2021-11-03T18:00:15.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:15.085+0000 7f5636eec700 1 -- 172.21.15.27:0/2706127012 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5630027a00 con 0x7f56300f6ff0 2021-11-03T18:00:15.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:15.085+0000 7f562d7fa700 1 -- 172.21.15.27:0/2706127012 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f561c0524c0 con 0x7f56300f6ff0 2021-11-03T18:00:15.083 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:00:15.084 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:00:15.084 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:00:15.084 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:00:15.084 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:00:15.084 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:00:15.085 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:00:15.085 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:00:15.085 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:00:15.085 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:00:15.086 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:00:15.086 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:00:15.086 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:00:15.086 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:00:15.087 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:00:15.087 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:00:15.087 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:00:15.088 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:00:15.088 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:00:15.088 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:00:15.088 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:00:15.089 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:00:15.089 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:00:15.089 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:00:15.089 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:00:15.089 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:00:15.090 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:00:15.090 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:00:15.090 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:00:15.090 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:00:15.091 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:00:15.091 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:00:15.091 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:00:15.091 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:00:15.091 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:00:15.092 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:00:15.092 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:00:15.092 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:00:15.092 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:00:15.093 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:15.087+0000 7f5636eec700 1 -- 172.21.15.27:0/2706127012 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f56200640b0 msgr2=0x7f5620066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:00:15.094 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:15.087+0000 7f5636eec700 1 --2- 172.21.15.27:0/2706127012 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f56200640b0 0x7f5620066570 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 rx=0x7f5624004630 tx=0x7f5624009b30).stop 2021-11-03T18:00:15.094 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:15.087+0000 7f5636eec700 1 -- 172.21.15.27:0/2706127012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56300f6ff0 msgr2=0x7f5630107450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:00:15.094 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:15.087+0000 7f5636eec700 1 --2- 172.21.15.27:0/2706127012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56300f6ff0 0x7f5630107450 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7f561c00c530 tx=0x7f561c007e20).stop 2021-11-03T18:00:15.094 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:15.087+0000 7f5636eec700 1 -- 172.21.15.27:0/2706127012 shutdown_connections 2021-11-03T18:00:15.095 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:15.087+0000 7f5636eec700 1 --2- 172.21.15.27:0/2706127012 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f56200640b0 0x7f5620066570 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:15.095 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:15.087+0000 7f5636eec700 1 --2- 172.21.15.27:0/2706127012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56300f66e0 0x7f5630106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:15.095 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:15.087+0000 7f5636eec700 1 --2- 172.21.15.27:0/2706127012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56300f6ff0 0x7f5630107450 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:15.095 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:15.087+0000 7f5636eec700 1 -- 172.21.15.27:0/2706127012 >> 172.21.15.27:0/2706127012 conn(0x7f56300f1f00 msgr2=0x7f56300fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:00:15.096 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:15.087+0000 7f5636eec700 1 -- 172.21.15.27:0/2706127012 shutdown_connections 2021-11-03T18:00:15.096 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:15.087+0000 7f5636eec700 1 -- 172.21.15.27:0/2706127012 wait complete. 2021-11-03T18:00:15.096 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:00:15.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:15 smithi115 ceph-mon[65453]: from='client.35512 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:00:15.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2657.423379 secs 2021-11-03T18:00:15.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:15 smithi115 ceph-mon[65453]: from='client.35516 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:00:15.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:15 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2648167609' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:15.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:15 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2706127012' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:00:15.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:15 smithi027 ceph-mon[97244]: from='client.35512 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:00:15.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2657.423379 secs 2021-11-03T18:00:15.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:15 smithi027 ceph-mon[97244]: from='client.35516 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:00:15.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:15 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2648167609' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:15.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:15 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2706127012' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:00:16.485 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:16 smithi115 ceph-mon[65453]: pgmap v1368: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:16.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:00:16.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:16 smithi027 ceph-mon[97244]: pgmap v1368: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:16.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:00:17.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:00:17.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:00:17.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:17.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:17.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:17.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:17.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:17 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:00:17.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:00:17.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:00:17.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:17.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:17.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:17.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:17.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:17 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:00:18.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:18 smithi027 ceph-mon[97244]: pgmap v1369: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:18.707 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:18 smithi115 ceph-mon[65453]: pgmap v1369: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:20.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:20 smithi027 ceph-mon[97244]: pgmap v1370: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:20.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2662.423470 secs 2021-11-03T18:00:20.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:20 smithi115 ceph-mon[65453]: pgmap v1370: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:20.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2662.423470 secs 2021-11-03T18:00:22.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:22 smithi027 ceph-mon[97244]: pgmap v1371: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:22.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:22 smithi115 ceph-mon[65453]: pgmap v1371: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:24.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:24 smithi027 ceph-mon[97244]: pgmap v1372: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:24.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:24 smithi115 ceph-mon[65453]: pgmap v1372: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:25.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2667.423602 secs 2021-11-03T18:00:25.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2667.423602 secs 2021-11-03T18:00:26.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:26 smithi027 ceph-mon[97244]: pgmap v1373: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:26.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:26 smithi115 ceph-mon[65453]: pgmap v1373: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:27.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:00:27.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:00:27.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:00:27.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:27.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:27.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:27.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:27.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:27 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:00:27.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:00:27.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:00:27.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:00:27.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:27.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:27.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:27.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:27.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:27 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:00:28.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:28 smithi027 ceph-mon[97244]: pgmap v1374: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:28.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:28 smithi115 ceph-mon[65453]: pgmap v1374: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:30.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:30 smithi027 ceph-mon[97244]: pgmap v1375: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:30.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2672.423676 secs 2021-11-03T18:00:30.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:30 smithi115 ceph-mon[65453]: pgmap v1375: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:30.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2672.423676 secs 2021-11-03T18:00:32.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:32 smithi027 ceph-mon[97244]: pgmap v1376: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:32.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:32 smithi115 ceph-mon[65453]: pgmap v1376: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:34.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:34 smithi115 ceph-mon[65453]: pgmap v1377: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:34.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:34 smithi027 ceph-mon[97244]: pgmap v1377: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:35.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2677.423818 secs 2021-11-03T18:00:35.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2677.423818 secs 2021-11-03T18:00:36.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:36 smithi115 ceph-mon[65453]: pgmap v1378: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:36.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:36 smithi027 ceph-mon[97244]: pgmap v1378: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:37.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:00:37.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:00:37.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:00:37.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:37.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:37.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:37.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:37.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:37 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:00:37.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:00:37.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:00:37.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:00:37.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:37.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:37.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:37.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:37.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:37 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:00:38.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:38 smithi115 ceph-mon[65453]: pgmap v1379: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:38.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:38 smithi027 ceph-mon[97244]: pgmap v1379: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:40.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:40 smithi115 ceph-mon[65453]: pgmap v1380: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:40.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2682.423882 secs 2021-11-03T18:00:40.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:40 smithi027 ceph-mon[97244]: pgmap v1380: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:40.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2682.423882 secs 2021-11-03T18:00:42.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:42 smithi115 ceph-mon[65453]: pgmap v1381: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:42.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:42 smithi027 ceph-mon[97244]: pgmap v1381: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:44.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:44 smithi115 ceph-mon[65453]: pgmap v1382: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:44.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:44 smithi027 ceph-mon[97244]: pgmap v1382: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:45.200 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.203+0000 7fa6e5e1b700 1 -- 172.21.15.27:0/1488403527 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6e00f66e0 msgr2=0x7fa6e00f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:00:45.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.203+0000 7fa6e5e1b700 1 --2- 172.21.15.27:0/1488403527 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6e00f66e0 0x7fa6e00f6b00 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7fa6d0004660 tx=0x7fa6d0009b30).stop 2021-11-03T18:00:45.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.203+0000 7fa6e5e1b700 1 -- 172.21.15.27:0/1488403527 shutdown_connections 2021-11-03T18:00:45.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.203+0000 7fa6e5e1b700 1 --2- 172.21.15.27:0/1488403527 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6e00f6ff0 0x7fa6e00f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:45.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.203+0000 7fa6e5e1b700 1 --2- 172.21.15.27:0/1488403527 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6e00f66e0 0x7fa6e00f6b00 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:45.202 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.203+0000 7fa6e5e1b700 1 -- 172.21.15.27:0/1488403527 >> 172.21.15.27:0/1488403527 conn(0x7fa6e00f1f00 msgr2=0x7fa6e00f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:00:45.202 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.203+0000 7fa6e5e1b700 1 -- 172.21.15.27:0/1488403527 shutdown_connections 2021-11-03T18:00:45.202 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.203+0000 7fa6e5e1b700 1 -- 172.21.15.27:0/1488403527 wait complete. 2021-11-03T18:00:45.205 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.203+0000 7fa6e5e1b700 1 Processor -- start 2021-11-03T18:00:45.205 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.204+0000 7fa6e5e1b700 1 -- start start 2021-11-03T18:00:45.205 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.204+0000 7fa6e5e1b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6e00f66e0 0x7fa6e0106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:00:45.206 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.204+0000 7fa6e5e1b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6e00f6ff0 0x7fa6e0107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:00:45.206 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.204+0000 7fa6e5e1b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e0107a00 con 0x7fa6e00f66e0 2021-11-03T18:00:45.206 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.204+0000 7fa6e5e1b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e0107b40 con 0x7fa6e00f6ff0 2021-11-03T18:00:45.206 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.204+0000 7fa6df7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6e00f66e0 0x7fa6e0106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:00:45.206 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.204+0000 7fa6df7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6e00f66e0 0x7fa6e0106f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60644/0 (socket says 172.21.15.27:60644) 2021-11-03T18:00:45.207 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.204+0000 7fa6df7fe700 1 -- 172.21.15.27:0/2668915148 learned_addr learned my addr 172.21.15.27:0/2668915148 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:00:45.207 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.204+0000 7fa6df7fe700 1 -- 172.21.15.27:0/2668915148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6e00f6ff0 msgr2=0x7fa6e0107450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:00:45.207 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.204+0000 7fa6df7fe700 1 --2- 172.21.15.27:0/2668915148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6e00f6ff0 0x7fa6e0107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:00:45.207 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.204+0000 7fa6df7fe700 1 -- 172.21.15.27:0/2668915148 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d0005040 con 0x7fa6e00f66e0 2021-11-03T18:00:45.208 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.205+0000 7fa6df7fe700 1 --2- 172.21.15.27:0/2668915148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6e00f66e0 0x7fa6e0106f60 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7fa6d0009990 tx=0x7fa6d0009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:00:45.208 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.205+0000 7fa6c7fff700 1 -- 172.21.15.27:0/2668915148 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6d0018070 con 0x7fa6e00f66e0 2021-11-03T18:00:45.208 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.205+0000 7fa6c7fff700 1 -- 172.21.15.27:0/2668915148 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fa6d0007ba0 con 0x7fa6e00f66e0 2021-11-03T18:00:45.208 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.205+0000 7fa6e5e1b700 1 -- 172.21.15.27:0/2668915148 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6e0108480 con 0x7fa6e00f66e0 2021-11-03T18:00:45.208 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.205+0000 7fa6c7fff700 1 -- 172.21.15.27:0/2668915148 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6d001c3c0 con 0x7fa6e00f66e0 2021-11-03T18:00:45.209 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.205+0000 7fa6e5e1b700 1 -- 172.21.15.27:0/2668915148 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6e01089e0 con 0x7fa6e00f66e0 2021-11-03T18:00:45.209 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.206+0000 7fa6e5e1b700 1 -- 172.21.15.27:0/2668915148 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6e0045ad0 con 0x7fa6e00f66e0 2021-11-03T18:00:45.209 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.206+0000 7fa6c7fff700 1 -- 172.21.15.27:0/2668915148 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fa6d001d420 con 0x7fa6e00f66e0 2021-11-03T18:00:45.209 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.207+0000 7fa6c7fff700 1 --2- 172.21.15.27:0/2668915148 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa6c80640b0 0x7fa6c8066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:00:45.209 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.207+0000 7fa6c7fff700 1 -- 172.21.15.27:0/2668915148 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fa6d0082630 con 0x7fa6e00f66e0 2021-11-03T18:00:45.210 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.207+0000 7fa6deffd700 1 --2- 172.21.15.27:0/2668915148 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa6c80640b0 0x7fa6c8066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:00:45.210 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.207+0000 7fa6deffd700 1 --2- 172.21.15.27:0/2668915148 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa6c80640b0 0x7fa6c8066570 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7fa6d4004a10 tx=0x7fa6d40099e0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:00:45.210 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.211+0000 7fa6c7fff700 1 -- 172.21.15.27:0/2668915148 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fa6d0054990 con 0x7fa6e00f66e0 2021-11-03T18:00:45.390 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.392+0000 7fa6e5e1b700 1 -- 172.21.15.27:0/2668915148 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6e0108c00 con 0x7fa6c80640b0 2021-11-03T18:00:45.390 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.393+0000 7fa6c7fff700 1 -- 172.21.15.27:0/2668915148 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fa6e0108c00 con 0x7fa6c80640b0 2021-11-03T18:00:45.393 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.395+0000 7fa6e5e1b700 1 -- 172.21.15.27:0/2668915148 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa6c80640b0 msgr2=0x7fa6c8066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:00:45.393 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.395+0000 7fa6e5e1b700 1 --2- 172.21.15.27:0/2668915148 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa6c80640b0 0x7fa6c8066570 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7fa6d4004a10 tx=0x7fa6d40099e0).stop 2021-11-03T18:00:45.393 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.395+0000 7fa6e5e1b700 1 -- 172.21.15.27:0/2668915148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6e00f66e0 msgr2=0x7fa6e0106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:00:45.394 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.395+0000 7fa6e5e1b700 1 --2- 172.21.15.27:0/2668915148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6e00f66e0 0x7fa6e0106f60 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7fa6d0009990 tx=0x7fa6d0009b30).stop 2021-11-03T18:00:45.394 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.395+0000 7fa6e5e1b700 1 -- 172.21.15.27:0/2668915148 shutdown_connections 2021-11-03T18:00:45.394 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.395+0000 7fa6e5e1b700 1 --2- 172.21.15.27:0/2668915148 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa6c80640b0 0x7fa6c8066570 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:45.394 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.395+0000 7fa6e5e1b700 1 --2- 172.21.15.27:0/2668915148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6e00f66e0 0x7fa6e0106f60 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:45.395 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.395+0000 7fa6e5e1b700 1 --2- 172.21.15.27:0/2668915148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6e00f6ff0 0x7fa6e0107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:00:45.395 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.395+0000 7fa6e5e1b700 1 -- 172.21.15.27:0/2668915148 >> 172.21.15.27:0/2668915148 conn(0x7fa6e00f1f00 msgr2=0x7fa6e00fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:00:45.395 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.395+0000 7fa6e5e1b700 1 -- 172.21.15.27:0/2668915148 shutdown_connections 2021-11-03T18:00:45.395 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.395+0000 7fa6e5e1b700 1 -- 172.21.15.27:0/2668915148 wait complete. 2021-11-03T18:00:45.402 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:00:45.494 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.495+0000 7f9f6370e700 1 -- 172.21.15.27:0/3474306728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c0f7d00 msgr2=0x7f9f5c0f8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:00:45.494 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.495+0000 7f9f6370e700 1 --2- 172.21.15.27:0/3474306728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c0f7d00 0x7f9f5c0f8120 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7f9f4c004660 tx=0x7f9f4c009b30).stop 2021-11-03T18:00:45.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.495+0000 7f9f6370e700 1 -- 172.21.15.27:0/3474306728 shutdown_connections 2021-11-03T18:00:45.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.495+0000 7f9f6370e700 1 --2- 172.21.15.27:0/3474306728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f5c0f8610 0x7f9f5c0f4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:45.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.495+0000 7f9f6370e700 1 --2- 172.21.15.27:0/3474306728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c0f7d00 0x7f9f5c0f8120 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:45.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.495+0000 7f9f6370e700 1 -- 172.21.15.27:0/3474306728 >> 172.21.15.27:0/3474306728 conn(0x7f9f5c0f0540 msgr2=0x7f9f5c0f2960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:00:45.496 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.495+0000 7f9f6370e700 1 -- 172.21.15.27:0/3474306728 shutdown_connections 2021-11-03T18:00:45.496 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.495+0000 7f9f6370e700 1 -- 172.21.15.27:0/3474306728 wait complete. 2021-11-03T18:00:45.496 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.496+0000 7f9f6370e700 1 Processor -- start 2021-11-03T18:00:45.496 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.496+0000 7f9f6370e700 1 -- start start 2021-11-03T18:00:45.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.496+0000 7f9f6370e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f5c0f7d00 0x7f9f5c0f75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:00:45.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.496+0000 7f9f6370e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c0f8610 0x7f9f5c0f5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:00:45.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.496+0000 7f9f6370e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c0f5fb0 con 0x7f9f5c0f8610 2021-11-03T18:00:45.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.496+0000 7f9f6370e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c0f60f0 con 0x7f9f5c0f7d00 2021-11-03T18:00:45.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.496+0000 7f9f60ca9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c0f8610 0x7f9f5c0f5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:00:45.498 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.496+0000 7f9f60ca9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c0f8610 0x7f9f5c0f5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60656/0 (socket says 172.21.15.27:60656) 2021-11-03T18:00:45.498 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.496+0000 7f9f60ca9700 1 -- 172.21.15.27:0/2894798742 learned_addr learned my addr 172.21.15.27:0/2894798742 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:00:45.498 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.496+0000 7f9f614aa700 1 --2- 172.21.15.27:0/2894798742 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f5c0f7d00 0x7f9f5c0f75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:00:45.498 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.496+0000 7f9f60ca9700 1 -- 172.21.15.27:0/2894798742 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f5c0f7d00 msgr2=0x7f9f5c0f75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:00:45.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.496+0000 7f9f60ca9700 1 --2- 172.21.15.27:0/2894798742 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f5c0f7d00 0x7f9f5c0f75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:45.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.496+0000 7f9f60ca9700 1 -- 172.21.15.27:0/2894798742 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f4c005040 con 0x7f9f5c0f8610 2021-11-03T18:00:45.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.496+0000 7f9f614aa700 1 --2- 172.21.15.27:0/2894798742 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f5c0f7d00 0x7f9f5c0f75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:00:45.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.496+0000 7f9f60ca9700 1 --2- 172.21.15.27:0/2894798742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c0f8610 0x7f9f5c0f5a50 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7f9f580026d0 tx=0x7f9f58009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:00:45.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.497+0000 7f9f51ffb700 1 -- 172.21.15.27:0/2894798742 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f58009320 con 0x7f9f5c0f8610 2021-11-03T18:00:45.500 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.497+0000 7f9f51ffb700 1 -- 172.21.15.27:0/2894798742 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f9f5800f040 con 0x7f9f5c0f8610 2021-11-03T18:00:45.500 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.497+0000 7f9f51ffb700 1 -- 172.21.15.27:0/2894798742 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f580147e0 con 0x7f9f5c0f8610 2021-11-03T18:00:45.500 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.497+0000 7f9f6370e700 1 -- 172.21.15.27:0/2894798742 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f5c0f63d0 con 0x7f9f5c0f8610 2021-11-03T18:00:45.500 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.497+0000 7f9f6370e700 1 -- 172.21.15.27:0/2894798742 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f5c10efa0 con 0x7f9f5c0f8610 2021-11-03T18:00:45.501 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.498+0000 7f9f6370e700 1 -- 172.21.15.27:0/2894798742 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f5c045ad0 con 0x7f9f5c0f8610 2021-11-03T18:00:45.502 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.498+0000 7f9f51ffb700 1 -- 172.21.15.27:0/2894798742 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f9f58009480 con 0x7f9f5c0f8610 2021-11-03T18:00:45.502 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.498+0000 7f9f51ffb700 1 --2- 172.21.15.27:0/2894798742 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9f480640b0 0x7f9f48066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:00:45.502 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.499+0000 7f9f51ffb700 1 -- 172.21.15.27:0/2894798742 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f9f58080f10 con 0x7f9f5c0f8610 2021-11-03T18:00:45.502 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.499+0000 7f9f614aa700 1 --2- 172.21.15.27:0/2894798742 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9f480640b0 0x7f9f48066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:00:45.503 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.499+0000 7f9f614aa700 1 --2- 172.21.15.27:0/2894798742 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9f480640b0 0x7f9f48066570 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f9f4c005010 tx=0x7f9f4c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:00:45.503 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.503+0000 7f9f51ffb700 1 -- 172.21.15.27:0/2894798742 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f9f58053210 con 0x7f9f5c0f8610 2021-11-03T18:00:45.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.677+0000 7f9f6370e700 1 -- 172.21.15.27:0/2894798742 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f5c0ff060 con 0x7f9f480640b0 2021-11-03T18:00:45.689 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.691+0000 7f9f51ffb700 1 -- 172.21.15.27:0/2894798742 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f9f4c019040 con 0x7f9f480640b0 2021-11-03T18:00:45.689 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:00:45.689 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (52m) 103s ago 53m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:00:45.690 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (44m) 103s ago 53m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:00:45.690 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (43m) 49s ago 52m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:00:45.690 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (52m) 103s ago 53m 41.8M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:00:45.690 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (45m) 103s ago 48m 26.0M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:00:45.691 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (48m) 103s ago 48m 3845M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:00:45.691 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (44m) 49s ago 48m 25.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:00:45.691 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (46m) 49s ago 48m 26.8M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:00:45.691 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (46m) 103s ago 54m 554M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:00:45.692 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (45m) 49s ago 52m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:00:45.692 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (45m) 103s ago 54m 235M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:00:45.692 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (44m) 49s ago 52m 112M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:00:45.692 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (53m) 103s ago 53m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:00:45.693 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (52m) 49s ago 52m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:00:45.693 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (43m) 103s ago 51m 351M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:00:45.693 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (43m) 103s ago 51m 383M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:00:45.693 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (42m) 103s ago 51m 284M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:00:45.693 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (42m) 49s ago 50m 404M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:00:45.694 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (41m) 49s ago 50m 322M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:00:45.694 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (41m) 49s ago 50m 327M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:00:45.694 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (52m) 103s ago 52m 55.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:00:45.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.693+0000 7f9f6370e700 1 -- 172.21.15.27:0/2894798742 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9f480640b0 msgr2=0x7f9f48066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:00:45.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.693+0000 7f9f6370e700 1 --2- 172.21.15.27:0/2894798742 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9f480640b0 0x7f9f48066570 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7f9f4c005010 tx=0x7f9f4c009b30).stop 2021-11-03T18:00:45.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.693+0000 7f9f6370e700 1 -- 172.21.15.27:0/2894798742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c0f8610 msgr2=0x7f9f5c0f5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:00:45.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.693+0000 7f9f6370e700 1 --2- 172.21.15.27:0/2894798742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c0f8610 0x7f9f5c0f5a50 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7f9f580026d0 tx=0x7f9f58009de0).stop 2021-11-03T18:00:45.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.693+0000 7f9f6370e700 1 -- 172.21.15.27:0/2894798742 shutdown_connections 2021-11-03T18:00:45.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.693+0000 7f9f6370e700 1 --2- 172.21.15.27:0/2894798742 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9f480640b0 0x7f9f48066570 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:45.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.693+0000 7f9f6370e700 1 --2- 172.21.15.27:0/2894798742 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f5c0f7d00 0x7f9f5c0f75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:45.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.693+0000 7f9f6370e700 1 --2- 172.21.15.27:0/2894798742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f5c0f8610 0x7f9f5c0f5a50 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:45.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.693+0000 7f9f6370e700 1 -- 172.21.15.27:0/2894798742 >> 172.21.15.27:0/2894798742 conn(0x7f9f5c0f0540 msgr2=0x7f9f5c0fb590 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:00:45.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.693+0000 7f9f6370e700 1 -- 172.21.15.27:0/2894798742 shutdown_connections 2021-11-03T18:00:45.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.693+0000 7f9f6370e700 1 -- 172.21.15.27:0/2894798742 wait complete. 2021-11-03T18:00:45.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2687.424024 secs 2021-11-03T18:00:45.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2687.424024 secs 2021-11-03T18:00:45.792 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.793+0000 7fa808dec700 1 -- 172.21.15.27:0/1179741487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8040f66e0 msgr2=0x7fa8040f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:00:45.792 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.793+0000 7fa808dec700 1 --2- 172.21.15.27:0/1179741487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8040f66e0 0x7fa8040f6b00 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7fa7fc004660 tx=0x7fa7fc009b30).stop 2021-11-03T18:00:45.792 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.793+0000 7fa808dec700 1 -- 172.21.15.27:0/1179741487 shutdown_connections 2021-11-03T18:00:45.792 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.793+0000 7fa808dec700 1 --2- 172.21.15.27:0/1179741487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8040f6ff0 0x7fa8040f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:45.793 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.793+0000 7fa808dec700 1 --2- 172.21.15.27:0/1179741487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8040f66e0 0x7fa8040f6b00 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:45.793 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.793+0000 7fa808dec700 1 -- 172.21.15.27:0/1179741487 >> 172.21.15.27:0/1179741487 conn(0x7fa8040f1f00 msgr2=0x7fa8040f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:00:45.793 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.793+0000 7fa808dec700 1 -- 172.21.15.27:0/1179741487 shutdown_connections 2021-11-03T18:00:45.793 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.793+0000 7fa808dec700 1 -- 172.21.15.27:0/1179741487 wait complete. 2021-11-03T18:00:45.794 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.793+0000 7fa808dec700 1 Processor -- start 2021-11-03T18:00:45.794 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.793+0000 7fa808dec700 1 -- start start 2021-11-03T18:00:45.794 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.793+0000 7fa808dec700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8040f66e0 0x7fa804106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:00:45.794 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.793+0000 7fa808dec700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8040f6ff0 0x7fa804107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:00:45.795 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.793+0000 7fa808dec700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa804107a00 con 0x7fa8040f6ff0 2021-11-03T18:00:45.795 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.793+0000 7fa808dec700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa804107b40 con 0x7fa8040f66e0 2021-11-03T18:00:45.795 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.794+0000 7fa801d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8040f6ff0 0x7fa804107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:00:45.795 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.794+0000 7fa801d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8040f6ff0 0x7fa804107450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60666/0 (socket says 172.21.15.27:60666) 2021-11-03T18:00:45.795 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.794+0000 7fa801d9b700 1 -- 172.21.15.27:0/3416721019 learned_addr learned my addr 172.21.15.27:0/3416721019 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:00:45.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.794+0000 7fa80259c700 1 --2- 172.21.15.27:0/3416721019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8040f66e0 0x7fa804106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:00:45.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.794+0000 7fa801d9b700 1 -- 172.21.15.27:0/3416721019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8040f66e0 msgr2=0x7fa804106f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:00:45.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.794+0000 7fa801d9b700 1 --2- 172.21.15.27:0/3416721019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8040f66e0 0x7fa804106f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:45.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.794+0000 7fa801d9b700 1 -- 172.21.15.27:0/3416721019 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7fc005040 con 0x7fa8040f6ff0 2021-11-03T18:00:45.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.794+0000 7fa801d9b700 1 --2- 172.21.15.27:0/3416721019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8040f6ff0 0x7fa804107450 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7fa7ec00c790 tx=0x7fa7ec007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:00:45.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.794+0000 7fa7faffd700 1 -- 172.21.15.27:0/3416721019 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7ec00e680 con 0x7fa8040f6ff0 2021-11-03T18:00:45.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.794+0000 7fa808dec700 1 -- 172.21.15.27:0/3416721019 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8041084e0 con 0x7fa8040f6ff0 2021-11-03T18:00:45.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.794+0000 7fa7faffd700 1 -- 172.21.15.27:0/3416721019 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fa7ec00ec90 con 0x7fa8040f6ff0 2021-11-03T18:00:45.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.794+0000 7fa808dec700 1 -- 172.21.15.27:0/3416721019 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa804108aa0 con 0x7fa8040f6ff0 2021-11-03T18:00:45.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.794+0000 7fa7faffd700 1 -- 172.21.15.27:0/3416721019 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7ec013d60 con 0x7fa8040f6ff0 2021-11-03T18:00:45.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.795+0000 7fa808dec700 1 -- 172.21.15.27:0/3416721019 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa804045ad0 con 0x7fa8040f6ff0 2021-11-03T18:00:45.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.796+0000 7fa7faffd700 1 -- 172.21.15.27:0/3416721019 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fa7ec01a060 con 0x7fa8040f6ff0 2021-11-03T18:00:45.800 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.796+0000 7fa7faffd700 1 --2- 172.21.15.27:0/3416721019 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa7f00640b0 0x7fa7f0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:00:45.800 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.796+0000 7fa7faffd700 1 -- 172.21.15.27:0/3416721019 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fa7ec080b30 con 0x7fa8040f6ff0 2021-11-03T18:00:45.800 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.796+0000 7fa80259c700 1 --2- 172.21.15.27:0/3416721019 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa7f00640b0 0x7fa7f0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:00:45.800 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.797+0000 7fa80259c700 1 --2- 172.21.15.27:0/3416721019 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa7f00640b0 0x7fa7f0066570 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7fa7fc004e30 tx=0x7fa7fc009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:00:45.801 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:45.800+0000 7fa7faffd700 1 -- 172.21.15.27:0/3416721019 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fa7ec052d40 con 0x7fa8040f6ff0 2021-11-03T18:00:46.028 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.030+0000 7fa808dec700 1 -- 172.21.15.27:0/3416721019 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa804027a00 con 0x7fa8040f6ff0 2021-11-03T18:00:46.028 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.031+0000 7fa7faffd700 1 -- 172.21.15.27:0/3416721019 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fa7ec018070 con 0x7fa8040f6ff0 2021-11-03T18:00:46.029 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:00:46.029 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:00:46.029 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:00:46.029 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:00:46.030 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:00:46.030 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:00:46.030 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:00:46.030 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:00:46.031 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:00:46.031 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:00:46.031 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:00:46.031 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:00:46.032 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:00:46.032 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:00:46.032 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:00:46.032 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:00:46.033 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:00:46.033 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:00:46.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.032+0000 7fa808dec700 1 -- 172.21.15.27:0/3416721019 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa7f00640b0 msgr2=0x7fa7f0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:00:46.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.032+0000 7fa808dec700 1 --2- 172.21.15.27:0/3416721019 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa7f00640b0 0x7fa7f0066570 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 rx=0x7fa7fc004e30 tx=0x7fa7fc009b30).stop 2021-11-03T18:00:46.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.032+0000 7fa808dec700 1 -- 172.21.15.27:0/3416721019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8040f6ff0 msgr2=0x7fa804107450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:00:46.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.032+0000 7fa808dec700 1 --2- 172.21.15.27:0/3416721019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8040f6ff0 0x7fa804107450 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7fa7ec00c790 tx=0x7fa7ec007d70).stop 2021-11-03T18:00:46.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.032+0000 7fa808dec700 1 -- 172.21.15.27:0/3416721019 shutdown_connections 2021-11-03T18:00:46.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.032+0000 7fa808dec700 1 --2- 172.21.15.27:0/3416721019 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa7f00640b0 0x7fa7f0066570 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:46.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.032+0000 7fa808dec700 1 --2- 172.21.15.27:0/3416721019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8040f66e0 0x7fa804106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:46.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.032+0000 7fa808dec700 1 --2- 172.21.15.27:0/3416721019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8040f6ff0 0x7fa804107450 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:46.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.032+0000 7fa808dec700 1 -- 172.21.15.27:0/3416721019 >> 172.21.15.27:0/3416721019 conn(0x7fa8040f1f00 msgr2=0x7fa8040fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:00:46.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.033+0000 7fa808dec700 1 -- 172.21.15.27:0/3416721019 shutdown_connections 2021-11-03T18:00:46.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.033+0000 7fa808dec700 1 -- 172.21.15.27:0/3416721019 wait complete. 2021-11-03T18:00:46.132 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.133+0000 7f4d4af52700 1 -- 172.21.15.27:0/353100871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d440f66c0 msgr2=0x7f4d440f6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:00:46.133 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.133+0000 7f4d4af52700 1 --2- 172.21.15.27:0/353100871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d440f66c0 0x7f4d440f6ae0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7f4d3c0099e0 tx=0x7f4d3c004d10).stop 2021-11-03T18:00:46.133 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.133+0000 7f4d4af52700 1 -- 172.21.15.27:0/353100871 shutdown_connections 2021-11-03T18:00:46.133 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.133+0000 7f4d4af52700 1 --2- 172.21.15.27:0/353100871 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d440f7960 0x7f4d440f7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:46.134 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.133+0000 7f4d4af52700 1 --2- 172.21.15.27:0/353100871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d440f66c0 0x7f4d440f6ae0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:46.134 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.133+0000 7f4d4af52700 1 -- 172.21.15.27:0/353100871 >> 172.21.15.27:0/353100871 conn(0x7f4d440f1e20 msgr2=0x7f4d440f4240 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:00:46.134 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.133+0000 7f4d4af52700 1 -- 172.21.15.27:0/353100871 shutdown_connections 2021-11-03T18:00:46.134 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.133+0000 7f4d4af52700 1 -- 172.21.15.27:0/353100871 wait complete. 2021-11-03T18:00:46.134 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.134+0000 7f4d4af52700 1 Processor -- start 2021-11-03T18:00:46.135 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.134+0000 7f4d4af52700 1 -- start start 2021-11-03T18:00:46.135 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.134+0000 7f4d4af52700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d440f66c0 0x7f4d44107080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:00:46.135 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.134+0000 7f4d4af52700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d440f7960 0x7f4d44107570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:00:46.135 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.134+0000 7f4d4af52700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d44107b20 con 0x7f4d440f66c0 2021-11-03T18:00:46.136 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.134+0000 7f4d4af52700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d44107c60 con 0x7f4d440f7960 2021-11-03T18:00:46.136 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.134+0000 7f4d48cee700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d440f66c0 0x7f4d44107080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:00:46.136 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.134+0000 7f4d48cee700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d440f66c0 0x7f4d44107080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60674/0 (socket says 172.21.15.27:60674) 2021-11-03T18:00:46.136 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.134+0000 7f4d48cee700 1 -- 172.21.15.27:0/3531594094 learned_addr learned my addr 172.21.15.27:0/3531594094 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:00:46.137 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.134+0000 7f4d43fff700 1 --2- 172.21.15.27:0/3531594094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d440f7960 0x7f4d44107570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:00:46.137 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.135+0000 7f4d48cee700 1 -- 172.21.15.27:0/3531594094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d440f7960 msgr2=0x7f4d44107570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:00:46.137 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.135+0000 7f4d48cee700 1 --2- 172.21.15.27:0/3531594094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d440f7960 0x7f4d44107570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:46.137 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.135+0000 7f4d48cee700 1 -- 172.21.15.27:0/3531594094 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d3c009640 con 0x7f4d440f66c0 2021-11-03T18:00:46.137 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.135+0000 7f4d43fff700 1 --2- 172.21.15.27:0/3531594094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d440f7960 0x7f4d44107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:00:46.138 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.135+0000 7f4d48cee700 1 --2- 172.21.15.27:0/3531594094 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d440f66c0 0x7f4d44107080 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7f4d3c00baa0 tx=0x7f4d3c004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:00:46.138 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.135+0000 7f4d417fa700 1 -- 172.21.15.27:0/3531594094 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d3c00e070 con 0x7f4d440f66c0 2021-11-03T18:00:46.138 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.135+0000 7f4d417fa700 1 -- 172.21.15.27:0/3531594094 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f4d3c007df0 con 0x7f4d440f66c0 2021-11-03T18:00:46.138 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.135+0000 7f4d4af52700 1 -- 172.21.15.27:0/3531594094 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d441085a0 con 0x7f4d440f66c0 2021-11-03T18:00:46.139 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.135+0000 7f4d417fa700 1 -- 172.21.15.27:0/3531594094 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d3c018070 con 0x7f4d440f66c0 2021-11-03T18:00:46.139 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.135+0000 7f4d4af52700 1 -- 172.21.15.27:0/3531594094 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d44108b00 con 0x7f4d440f66c0 2021-11-03T18:00:46.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.136+0000 7f4d4af52700 1 -- 172.21.15.27:0/3531594094 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d44045ad0 con 0x7f4d440f66c0 2021-11-03T18:00:46.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.136+0000 7f4d417fa700 1 -- 172.21.15.27:0/3531594094 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f4d3c024060 con 0x7f4d440f66c0 2021-11-03T18:00:46.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.137+0000 7f4d417fa700 1 --2- 172.21.15.27:0/3531594094 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4d300640b0 0x7f4d30066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:00:46.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.137+0000 7f4d417fa700 1 -- 172.21.15.27:0/3531594094 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f4d3c081fe0 con 0x7f4d440f66c0 2021-11-03T18:00:46.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.137+0000 7f4d43fff700 1 --2- 172.21.15.27:0/3531594094 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4d300640b0 0x7f4d30066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:00:46.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.137+0000 7f4d43fff700 1 --2- 172.21.15.27:0/3531594094 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4d300640b0 0x7f4d30066570 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f4d2c007fc0 tx=0x7f4d2c00b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:00:46.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.141+0000 7f4d417fa700 1 -- 172.21.15.27:0/3531594094 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f4d3c0542e0 con 0x7f4d440f66c0 2021-11-03T18:00:46.344 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.346+0000 7f4d4af52700 1 -- 172.21.15.27:0/3531594094 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4d440fef90 con 0x7f4d440f66c0 2021-11-03T18:00:46.344 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.347+0000 7f4d417fa700 1 -- 172.21.15.27:0/3531594094 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f4d3c022070 con 0x7f4d440f66c0 2021-11-03T18:00:46.345 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:00:46.345 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:00:46.345 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:00:46.345 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:00:46.346 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:00:46.346 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:00:46.346 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:00:46.346 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:00:46.347 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:00:46.347 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:00:46.347 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:00:46.347 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:00:46.348 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:00:46.348 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:00:46.348 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:00:46.348 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:00:46.348 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:00:46.349 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:00:46.349 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:00:46.349 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:00:46.349 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:00:46.349 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:00:46.350 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:00:46.350 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:00:46.350 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:00:46.350 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:00:46.351 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:00:46.351 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:00:46.351 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:00:46.351 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:00:46.352 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:00:46.352 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:00:46.352 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:00:46.352 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:00:46.352 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:00:46.353 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:00:46.353 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:00:46.353 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:00:46.353 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:00:46.354 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.349+0000 7f4d4af52700 1 -- 172.21.15.27:0/3531594094 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4d300640b0 msgr2=0x7f4d30066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:00:46.355 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.349+0000 7f4d4af52700 1 --2- 172.21.15.27:0/3531594094 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4d300640b0 0x7f4d30066570 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f4d2c007fc0 tx=0x7f4d2c00b040).stop 2021-11-03T18:00:46.355 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.349+0000 7f4d4af52700 1 -- 172.21.15.27:0/3531594094 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d440f66c0 msgr2=0x7f4d44107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:00:46.355 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.349+0000 7f4d4af52700 1 --2- 172.21.15.27:0/3531594094 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d440f66c0 0x7f4d44107080 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7f4d3c00baa0 tx=0x7f4d3c004d10).stop 2021-11-03T18:00:46.355 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.349+0000 7f4d4af52700 1 -- 172.21.15.27:0/3531594094 shutdown_connections 2021-11-03T18:00:46.356 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.349+0000 7f4d4af52700 1 --2- 172.21.15.27:0/3531594094 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4d300640b0 0x7f4d30066570 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:46.356 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.349+0000 7f4d4af52700 1 --2- 172.21.15.27:0/3531594094 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d440f66c0 0x7f4d44107080 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:46.356 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.349+0000 7f4d4af52700 1 --2- 172.21.15.27:0/3531594094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d440f7960 0x7f4d44107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:00:46.356 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.349+0000 7f4d4af52700 1 -- 172.21.15.27:0/3531594094 >> 172.21.15.27:0/3531594094 conn(0x7f4d440f1e20 msgr2=0x7f4d440fb3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:00:46.356 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.349+0000 7f4d4af52700 1 -- 172.21.15.27:0/3531594094 shutdown_connections 2021-11-03T18:00:46.357 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:00:46.349+0000 7f4d4af52700 1 -- 172.21.15.27:0/3531594094 wait complete. 2021-11-03T18:00:46.357 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:00:46.677 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:46 smithi115 ceph-mon[65453]: pgmap v1383: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:46.678 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:46 smithi115 ceph-mon[65453]: from='client.35528 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:00:46.678 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:46 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3416721019' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:46.687 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:46 smithi027 ceph-mon[97244]: pgmap v1383: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:46.687 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:46 smithi027 ceph-mon[97244]: from='client.35528 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:00:46.688 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:46 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3416721019' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:47.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:47 smithi115 ceph-mon[65453]: from='client.35532 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:00:47.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:00:47.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:47 smithi115 ceph-mon[65453]: poll took 50157.599 ms: 1 events 2021-11-03T18:00:47.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:47 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3531594094' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:00:47.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:00:47.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:00:47.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:47.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:47.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:47.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:47.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:47 smithi027 ceph-mon[97244]: from='client.35532 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:00:47.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:00:47.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:47 smithi027 ceph-mon[97244]: poll took 50157.599 ms: 1 events 2021-11-03T18:00:47.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:47 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3531594094' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:00:47.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:00:47.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:00:47.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:47.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:47.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:47.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:48.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:48 smithi115 ceph-mon[65453]: pgmap v1384: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:48.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:48 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:00:48.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:48 smithi027 ceph-mon[97244]: pgmap v1384: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:48.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:48 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:00:50.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:50 smithi115 ceph-mon[65453]: pgmap v1385: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:50.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2692.424142 secs 2021-11-03T18:00:50.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:50 smithi027 ceph-mon[97244]: pgmap v1385: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:50.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2692.424142 secs 2021-11-03T18:00:52.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:52 smithi115 ceph-mon[65453]: pgmap v1386: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:52.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:52 smithi027 ceph-mon[97244]: pgmap v1386: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:54.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:54 smithi115 ceph-mon[65453]: pgmap v1387: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:54.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:54 smithi027 ceph-mon[97244]: pgmap v1387: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:55.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2697.424285 secs 2021-11-03T18:00:55.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2697.424285 secs 2021-11-03T18:00:56.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:56 smithi115 ceph-mon[65453]: pgmap v1388: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:56.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:00:56.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:00:56.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:56 smithi027 ceph-mon[97244]: pgmap v1388: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:56.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:00:56.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:00:57.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:00:57.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:00:57.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:00:57.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:57.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:57.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:57.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:57.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:00:57.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:00:57.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:00:57.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:57.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:57.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:57.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:00:58.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:58 smithi115 ceph-mon[65453]: pgmap v1389: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:58.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:00:58 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:00:58.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:58 smithi027 ceph-mon[97244]: pgmap v1389: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:00:58.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:00:58 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:01:00.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:00 smithi115 ceph-mon[65453]: pgmap v1390: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:00.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2702.424401 secs 2021-11-03T18:01:00.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:00 smithi027 ceph-mon[97244]: pgmap v1390: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:00.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2702.424401 secs 2021-11-03T18:01:02.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:02 smithi115 ceph-mon[65453]: pgmap v1391: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:02.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:02 smithi027 ceph-mon[97244]: pgmap v1391: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:04.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:04 smithi115 ceph-mon[65453]: pgmap v1392: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:04.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:04 smithi027 ceph-mon[97244]: pgmap v1392: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:05.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2707.424549 secs 2021-11-03T18:01:05.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2707.424549 secs 2021-11-03T18:01:06.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:06 smithi115 ceph-mon[65453]: pgmap v1393: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:06.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:06 smithi027 ceph-mon[97244]: pgmap v1393: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:07.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:01:07.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:01:07.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:01:07.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:07.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:07.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:07.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:07.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:01:07.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:01:07.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:01:07.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:07.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:07.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:07.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:08.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:08 smithi115 ceph-mon[65453]: pgmap v1394: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:08.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:08 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:01:08.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:08 smithi027 ceph-mon[97244]: pgmap v1394: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:08.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:08 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:01:10.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:10 smithi115 ceph-mon[65453]: pgmap v1395: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:10.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2712.424674 secs 2021-11-03T18:01:10.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:10 smithi027 ceph-mon[97244]: pgmap v1395: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:10.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2712.424674 secs 2021-11-03T18:01:12.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:12 smithi115 ceph-mon[65453]: pgmap v1396: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:12.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:12 smithi027 ceph-mon[97244]: pgmap v1396: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:14.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:14 smithi115 ceph-mon[65453]: pgmap v1397: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:14.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:14 smithi027 ceph-mon[97244]: pgmap v1397: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:15.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2717.424758 secs 2021-11-03T18:01:15.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2717.424758 secs 2021-11-03T18:01:16.462 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.463+0000 7f70e4950700 1 -- 172.21.15.27:0/249789042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70e00f6870 msgr2=0x7f70e00f6cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:01:16.462 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.463+0000 7f70e4950700 1 --2- 172.21.15.27:0/249789042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70e00f6870 0x7f70e00f6cd0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 rx=0x7f70c8004660 tx=0x7f70c8009b30).stop 2021-11-03T18:01:16.463 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.464+0000 7f70e4950700 1 -- 172.21.15.27:0/249789042 shutdown_connections 2021-11-03T18:01:16.463 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.464+0000 7f70e4950700 1 --2- 172.21.15.27:0/249789042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70e00f6870 0x7f70e00f6cd0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:16.463 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.464+0000 7f70e4950700 1 --2- 172.21.15.27:0/249789042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70e00f55d0 0x7f70e00f59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:16.463 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.464+0000 7f70e4950700 1 -- 172.21.15.27:0/249789042 >> 172.21.15.27:0/249789042 conn(0x7f70e00f0d70 msgr2=0x7f70e00f3190 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:01:16.464 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.464+0000 7f70e4950700 1 -- 172.21.15.27:0/249789042 shutdown_connections 2021-11-03T18:01:16.464 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.464+0000 7f70e4950700 1 -- 172.21.15.27:0/249789042 wait complete. 2021-11-03T18:01:16.464 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.464+0000 7f70e4950700 1 Processor -- start 2021-11-03T18:01:16.464 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.464+0000 7f70e4950700 1 -- start start 2021-11-03T18:01:16.465 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.464+0000 7f70e4950700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70e00f55d0 0x7f70e00f9730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:01:16.465 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.464+0000 7f70e4950700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70e00f6870 0x7f70e00f7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:01:16.465 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.464+0000 7f70e4950700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e00f8110 con 0x7f70e00f55d0 2021-11-03T18:01:16.465 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.464+0000 7f70e4950700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70e00f8250 con 0x7f70e00f6870 2021-11-03T18:01:16.466 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.464+0000 7f70de59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70e00f55d0 0x7f70e00f9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:01:16.466 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.465+0000 7f70de59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70e00f55d0 0x7f70e00f9730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60706/0 (socket says 172.21.15.27:60706) 2021-11-03T18:01:16.466 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.465+0000 7f70de59c700 1 -- 172.21.15.27:0/2206682803 learned_addr learned my addr 172.21.15.27:0/2206682803 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:01:16.466 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.465+0000 7f70ddd9b700 1 --2- 172.21.15.27:0/2206682803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70e00f6870 0x7f70e00f7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:01:16.467 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.465+0000 7f70de59c700 1 -- 172.21.15.27:0/2206682803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70e00f6870 msgr2=0x7f70e00f7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:01:16.467 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.465+0000 7f70de59c700 1 --2- 172.21.15.27:0/2206682803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70e00f6870 0x7f70e00f7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:16.467 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.465+0000 7f70de59c700 1 -- 172.21.15.27:0/2206682803 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c8005040 con 0x7f70e00f55d0 2021-11-03T18:01:16.467 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.465+0000 7f70de59c700 1 --2- 172.21.15.27:0/2206682803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70e00f55d0 0x7f70e00f9730 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7f70d800c810 tx=0x7f70d8004d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:01:16.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.465+0000 7f70d6ffd700 1 -- 172.21.15.27:0/2206682803 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70d800e680 con 0x7f70e00f55d0 2021-11-03T18:01:16.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.465+0000 7f70d6ffd700 1 -- 172.21.15.27:0/2206682803 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f70d800ec90 con 0x7f70e00f55d0 2021-11-03T18:01:16.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.465+0000 7f70e4950700 1 -- 172.21.15.27:0/2206682803 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70e00f8530 con 0x7f70e00f55d0 2021-11-03T18:01:16.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.465+0000 7f70d6ffd700 1 -- 172.21.15.27:0/2206682803 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70d8012df0 con 0x7f70e00f55d0 2021-11-03T18:01:16.469 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.465+0000 7f70e4950700 1 -- 172.21.15.27:0/2206682803 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70e010ef70 con 0x7f70e00f55d0 2021-11-03T18:01:16.470 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.466+0000 7f70e4950700 1 -- 172.21.15.27:0/2206682803 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70e0045ad0 con 0x7f70e00f55d0 2021-11-03T18:01:16.470 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.467+0000 7f70d6ffd700 1 -- 172.21.15.27:0/2206682803 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f70d8019030 con 0x7f70e00f55d0 2021-11-03T18:01:16.470 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.467+0000 7f70d6ffd700 1 --2- 172.21.15.27:0/2206682803 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f70cc064050 0x7f70cc066510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:01:16.471 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.467+0000 7f70d6ffd700 1 -- 172.21.15.27:0/2206682803 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f70d8080a00 con 0x7f70e00f55d0 2021-11-03T18:01:16.471 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.467+0000 7f70ddd9b700 1 --2- 172.21.15.27:0/2206682803 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f70cc064050 0x7f70cc066510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:01:16.471 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.468+0000 7f70ddd9b700 1 --2- 172.21.15.27:0/2206682803 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f70cc064050 0x7f70cc066510 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f70c8004630 tx=0x7f70c8009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:01:16.472 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.471+0000 7f70d6ffd700 1 -- 172.21.15.27:0/2206682803 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f70d8052cc0 con 0x7f70e00f55d0 2021-11-03T18:01:16.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.649+0000 7f70e4950700 1 -- 172.21.15.27:0/2206682803 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f70e00facb0 con 0x7f70cc064050 2021-11-03T18:01:16.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.650+0000 7f70d6ffd700 1 -- 172.21.15.27:0/2206682803 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f70e00facb0 con 0x7f70cc064050 2021-11-03T18:01:16.649 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.652+0000 7f70e4950700 1 -- 172.21.15.27:0/2206682803 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f70cc064050 msgr2=0x7f70cc066510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:01:16.649 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.652+0000 7f70e4950700 1 --2- 172.21.15.27:0/2206682803 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f70cc064050 0x7f70cc066510 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f70c8004630 tx=0x7f70c8009b30).stop 2021-11-03T18:01:16.649 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.652+0000 7f70e4950700 1 -- 172.21.15.27:0/2206682803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70e00f55d0 msgr2=0x7f70e00f9730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:01:16.650 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.652+0000 7f70e4950700 1 --2- 172.21.15.27:0/2206682803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70e00f55d0 0x7f70e00f9730 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7f70d800c810 tx=0x7f70d8004d90).stop 2021-11-03T18:01:16.650 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.652+0000 7f70e4950700 1 -- 172.21.15.27:0/2206682803 shutdown_connections 2021-11-03T18:01:16.650 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.652+0000 7f70e4950700 1 --2- 172.21.15.27:0/2206682803 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f70cc064050 0x7f70cc066510 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:16.650 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.652+0000 7f70e4950700 1 --2- 172.21.15.27:0/2206682803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70e00f55d0 0x7f70e00f9730 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:16.650 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.652+0000 7f70e4950700 1 --2- 172.21.15.27:0/2206682803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70e00f6870 0x7f70e00f7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:16.651 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.652+0000 7f70e4950700 1 -- 172.21.15.27:0/2206682803 >> 172.21.15.27:0/2206682803 conn(0x7f70e00f0d70 msgr2=0x7f70e005ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:01:16.651 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.652+0000 7f70e4950700 1 -- 172.21.15.27:0/2206682803 shutdown_connections 2021-11-03T18:01:16.651 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.652+0000 7f70e4950700 1 -- 172.21.15.27:0/2206682803 wait complete. 2021-11-03T18:01:16.658 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:01:16.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:16 smithi115 ceph-mon[65453]: pgmap v1398: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:16.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.752+0000 7fb378c40700 1 -- 172.21.15.27:0/2619037964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3740f66e0 msgr2=0x7fb3740f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:01:16.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.752+0000 7fb378c40700 1 --2- 172.21.15.27:0/2619037964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3740f66e0 0x7fb3740f6b00 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 rx=0x7fb36c0099e0 tx=0x7fb36c004d10).stop 2021-11-03T18:01:16.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.752+0000 7fb378c40700 1 -- 172.21.15.27:0/2619037964 shutdown_connections 2021-11-03T18:01:16.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.752+0000 7fb378c40700 1 --2- 172.21.15.27:0/2619037964 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3740f6ff0 0x7fb3740f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:16.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.752+0000 7fb378c40700 1 --2- 172.21.15.27:0/2619037964 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3740f66e0 0x7fb3740f6b00 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:16.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.752+0000 7fb378c40700 1 -- 172.21.15.27:0/2619037964 >> 172.21.15.27:0/2619037964 conn(0x7fb3740f1f00 msgr2=0x7fb3740f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:01:16.754 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.752+0000 7fb378c40700 1 -- 172.21.15.27:0/2619037964 shutdown_connections 2021-11-03T18:01:16.754 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.752+0000 7fb378c40700 1 -- 172.21.15.27:0/2619037964 wait complete. 2021-11-03T18:01:16.754 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.752+0000 7fb378c40700 1 Processor -- start 2021-11-03T18:01:16.754 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.753+0000 7fb378c40700 1 -- start start 2021-11-03T18:01:16.755 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.753+0000 7fb378c40700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3740f66e0 0x7fb37406b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:01:16.755 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.753+0000 7fb378c40700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3740f6ff0 0x7fb374069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:01:16.755 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.753+0000 7fb378c40700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37406a380 con 0x7fb3740f66e0 2021-11-03T18:01:16.756 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.753+0000 7fb378c40700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37406a4c0 con 0x7fb3740f6ff0 2021-11-03T18:01:16.756 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.753+0000 7fb37259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3740f66e0 0x7fb37406b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:01:16.756 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.753+0000 7fb37259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3740f66e0 0x7fb37406b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60716/0 (socket says 172.21.15.27:60716) 2021-11-03T18:01:16.756 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.753+0000 7fb37259c700 1 -- 172.21.15.27:0/2877642225 learned_addr learned my addr 172.21.15.27:0/2877642225 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:01:16.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.753+0000 7fb371d9b700 1 --2- 172.21.15.27:0/2877642225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3740f6ff0 0x7fb374069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:01:16.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.753+0000 7fb37259c700 1 -- 172.21.15.27:0/2877642225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3740f6ff0 msgr2=0x7fb374069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:01:16.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.753+0000 7fb37259c700 1 --2- 172.21.15.27:0/2877642225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3740f6ff0 0x7fb374069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:16.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.753+0000 7fb37259c700 1 -- 172.21.15.27:0/2877642225 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb36c009640 con 0x7fb3740f66e0 2021-11-03T18:01:16.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.753+0000 7fb37259c700 1 --2- 172.21.15.27:0/2877642225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3740f66e0 0x7fb37406b9a0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7fb36c004b70 tx=0x7fb36c004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:01:16.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.753+0000 7fb36affd700 1 -- 172.21.15.27:0/2877642225 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb36c00e030 con 0x7fb3740f66e0 2021-11-03T18:01:16.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.753+0000 7fb378c40700 1 -- 172.21.15.27:0/2877642225 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb37406a740 con 0x7fb3740f66e0 2021-11-03T18:01:16.759 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.753+0000 7fb36affd700 1 -- 172.21.15.27:0/2877642225 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fb36c0049b0 con 0x7fb3740f66e0 2021-11-03T18:01:16.759 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.754+0000 7fb36affd700 1 -- 172.21.15.27:0/2877642225 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb36c018070 con 0x7fb3740f66e0 2021-11-03T18:01:16.759 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.754+0000 7fb378c40700 1 -- 172.21.15.27:0/2877642225 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb37406ac20 con 0x7fb3740f66e0 2021-11-03T18:01:16.759 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.754+0000 7fb378c40700 1 -- 172.21.15.27:0/2877642225 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb374045ad0 con 0x7fb3740f66e0 2021-11-03T18:01:16.760 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.755+0000 7fb36affd700 1 -- 172.21.15.27:0/2877642225 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fb36c01db40 con 0x7fb3740f66e0 2021-11-03T18:01:16.760 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.755+0000 7fb36affd700 1 --2- 172.21.15.27:0/2877642225 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb3600640b0 0x7fb360066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:01:16.760 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.755+0000 7fb36affd700 1 -- 172.21.15.27:0/2877642225 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fb36c082820 con 0x7fb3740f66e0 2021-11-03T18:01:16.760 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.755+0000 7fb371d9b700 1 --2- 172.21.15.27:0/2877642225 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb3600640b0 0x7fb360066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:01:16.761 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.756+0000 7fb371d9b700 1 --2- 172.21.15.27:0/2877642225 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb3600640b0 0x7fb360066570 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7fb35c0049e0 tx=0x7fb35c009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:01:16.761 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.759+0000 7fb36affd700 1 -- 172.21.15.27:0/2877642225 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fb36c054ac0 con 0x7fb3740f66e0 2021-11-03T18:01:16.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:16 smithi027 ceph-mon[97244]: pgmap v1398: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:16.931 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.934+0000 7fb378c40700 1 -- 172.21.15.27:0/2877642225 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb3740008d0 con 0x7fb3600640b0 2021-11-03T18:01:16.945 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.949+0000 7fb36affd700 1 -- 172.21.15.27:0/2877642225 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fb3740008d0 con 0x7fb3600640b0 2021-11-03T18:01:16.946 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:01:16.946 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (52m) 2m ago 54m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:01:16.947 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (44m) 2m ago 54m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:01:16.947 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (44m) 80s ago 53m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:01:16.947 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (52m) 2m ago 53m 41.8M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:01:16.947 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (46m) 2m ago 49m 26.0M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:01:16.948 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (49m) 2m ago 49m 3845M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:01:16.948 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (45m) 80s ago 49m 25.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:01:16.948 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (46m) 80s ago 49m 26.8M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:01:16.948 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (46m) 2m ago 55m 554M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:01:16.948 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (45m) 80s ago 53m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:01:16.949 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (45m) 2m ago 55m 235M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:01:16.949 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (45m) 80s ago 53m 112M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:01:16.949 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (53m) 2m ago 53m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:01:16.949 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (53m) 80s ago 53m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:01:16.950 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (44m) 2m ago 52m 351M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:01:16.950 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (43m) 2m ago 52m 383M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:01:16.950 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (43m) 2m ago 51m 284M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:01:16.950 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (42m) 80s ago 51m 404M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:01:16.950 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (42m) 80s ago 51m 322M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:01:16.951 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (41m) 80s ago 50m 327M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:01:16.951 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (52m) 2m ago 53m 55.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:01:16.952 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.951+0000 7fb378c40700 1 -- 172.21.15.27:0/2877642225 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb3600640b0 msgr2=0x7fb360066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:01:16.952 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.951+0000 7fb378c40700 1 --2- 172.21.15.27:0/2877642225 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb3600640b0 0x7fb360066570 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 rx=0x7fb35c0049e0 tx=0x7fb35c009040).stop 2021-11-03T18:01:16.952 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.951+0000 7fb378c40700 1 -- 172.21.15.27:0/2877642225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3740f66e0 msgr2=0x7fb37406b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:01:16.952 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.951+0000 7fb378c40700 1 --2- 172.21.15.27:0/2877642225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3740f66e0 0x7fb37406b9a0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7fb36c004b70 tx=0x7fb36c004d10).stop 2021-11-03T18:01:16.953 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.951+0000 7fb378c40700 1 -- 172.21.15.27:0/2877642225 shutdown_connections 2021-11-03T18:01:16.953 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.951+0000 7fb378c40700 1 --2- 172.21.15.27:0/2877642225 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb3600640b0 0x7fb360066570 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:16.953 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.951+0000 7fb378c40700 1 --2- 172.21.15.27:0/2877642225 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb3740f66e0 0x7fb37406b9a0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:16.953 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.951+0000 7fb378c40700 1 --2- 172.21.15.27:0/2877642225 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb3740f6ff0 0x7fb374069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:16.954 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.951+0000 7fb378c40700 1 -- 172.21.15.27:0/2877642225 >> 172.21.15.27:0/2877642225 conn(0x7fb3740f1f00 msgr2=0x7fb3740fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:01:16.954 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.951+0000 7fb378c40700 1 -- 172.21.15.27:0/2877642225 shutdown_connections 2021-11-03T18:01:16.954 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:16.951+0000 7fb378c40700 1 -- 172.21.15.27:0/2877642225 wait complete. 2021-11-03T18:01:17.046 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.050+0000 7f3a835c7700 1 -- 172.21.15.27:0/1161244353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a7c0f66d0 msgr2=0x7f3a7c0f6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:01:17.047 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.050+0000 7f3a835c7700 1 --2- 172.21.15.27:0/1161244353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a7c0f66d0 0x7f3a7c0f6af0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0x7f3a740099e0 tx=0x7f3a74004d10).stop 2021-11-03T18:01:17.047 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.050+0000 7f3a835c7700 1 -- 172.21.15.27:0/1161244353 shutdown_connections 2021-11-03T18:01:17.047 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.050+0000 7f3a835c7700 1 --2- 172.21.15.27:0/1161244353 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a7c0f7970 0x7f3a7c0f7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:17.047 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.050+0000 7f3a835c7700 1 --2- 172.21.15.27:0/1161244353 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a7c0f66d0 0x7f3a7c0f6af0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:17.048 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.050+0000 7f3a835c7700 1 -- 172.21.15.27:0/1161244353 >> 172.21.15.27:0/1161244353 conn(0x7f3a7c0f1e10 msgr2=0x7f3a7c0f4250 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:01:17.048 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.050+0000 7f3a835c7700 1 -- 172.21.15.27:0/1161244353 shutdown_connections 2021-11-03T18:01:17.048 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.050+0000 7f3a835c7700 1 -- 172.21.15.27:0/1161244353 wait complete. 2021-11-03T18:01:17.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.050+0000 7f3a835c7700 1 Processor -- start 2021-11-03T18:01:17.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.050+0000 7f3a835c7700 1 -- start start 2021-11-03T18:01:17.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.051+0000 7f3a835c7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a7c0f66d0 0x7f3a7c1070c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:01:17.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.051+0000 7f3a835c7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a7c0f7970 0x7f3a7c1075b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:01:17.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.051+0000 7f3a835c7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c107b60 con 0x7f3a7c0f7970 2021-11-03T18:01:17.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.051+0000 7f3a835c7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7c107ca0 con 0x7f3a7c0f66d0 2021-11-03T18:01:17.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.051+0000 7f3a80b62700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a7c0f7970 0x7f3a7c1075b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:01:17.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.051+0000 7f3a80b62700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a7c0f7970 0x7f3a7c1075b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60728/0 (socket says 172.21.15.27:60728) 2021-11-03T18:01:17.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.051+0000 7f3a80b62700 1 -- 172.21.15.27:0/443435283 learned_addr learned my addr 172.21.15.27:0/443435283 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:01:17.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.051+0000 7f3a81363700 1 --2- 172.21.15.27:0/443435283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a7c0f66d0 0x7f3a7c1070c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:01:17.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.051+0000 7f3a80b62700 1 -- 172.21.15.27:0/443435283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a7c0f66d0 msgr2=0x7f3a7c1070c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:01:17.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.051+0000 7f3a80b62700 1 --2- 172.21.15.27:0/443435283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a7c0f66d0 0x7f3a7c1070c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:17.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.051+0000 7f3a80b62700 1 -- 172.21.15.27:0/443435283 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a74009640 con 0x7f3a7c0f7970 2021-11-03T18:01:17.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.051+0000 7f3a80b62700 1 --2- 172.21.15.27:0/443435283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a7c0f7970 0x7f3a7c1075b0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 rx=0x7f3a6c00c790 tx=0x7f3a6c007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:01:17.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.051+0000 7f3a71ffb700 1 -- 172.21.15.27:0/443435283 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a6c002590 con 0x7f3a7c0f7970 2021-11-03T18:01:17.055 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.051+0000 7f3a71ffb700 1 -- 172.21.15.27:0/443435283 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f3a6c002ba0 con 0x7f3a7c0f7970 2021-11-03T18:01:17.055 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.051+0000 7f3a835c7700 1 -- 172.21.15.27:0/443435283 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a7c108640 con 0x7f3a7c0f7970 2021-11-03T18:01:17.055 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.051+0000 7f3a71ffb700 1 -- 172.21.15.27:0/443435283 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a6c013d90 con 0x7f3a7c0f7970 2021-11-03T18:01:17.055 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.052+0000 7f3a835c7700 1 -- 172.21.15.27:0/443435283 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a7c108b80 con 0x7f3a7c0f7970 2021-11-03T18:01:17.055 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.052+0000 7f3a835c7700 1 -- 172.21.15.27:0/443435283 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a7c045ad0 con 0x7f3a7c0f7970 2021-11-03T18:01:17.056 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.053+0000 7f3a71ffb700 1 -- 172.21.15.27:0/443435283 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f3a6c01a060 con 0x7f3a7c0f7970 2021-11-03T18:01:17.056 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.053+0000 7f3a71ffb700 1 --2- 172.21.15.27:0/443435283 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3a680640b0 0x7f3a68066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:01:17.056 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.053+0000 7f3a71ffb700 1 -- 172.21.15.27:0/443435283 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f3a6c080ae0 con 0x7f3a7c0f7970 2021-11-03T18:01:17.056 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.053+0000 7f3a81363700 1 --2- 172.21.15.27:0/443435283 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3a680640b0 0x7f3a68066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:01:17.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.054+0000 7f3a81363700 1 --2- 172.21.15.27:0/443435283 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3a680640b0 0x7f3a68066570 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f3a7400b730 tx=0x7f3a74004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:01:17.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.057+0000 7f3a71ffb700 1 -- 172.21.15.27:0/443435283 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f3a6c052c90 con 0x7f3a7c0f7970 2021-11-03T18:01:17.282 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.286+0000 7f3a835c7700 1 -- 172.21.15.27:0/443435283 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3a7c027a00 con 0x7f3a7c0f7970 2021-11-03T18:01:17.283 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.286+0000 7f3a71ffb700 1 -- 172.21.15.27:0/443435283 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f3a6c0182e0 con 0x7f3a7c0f7970 2021-11-03T18:01:17.284 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:01:17.284 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:01:17.284 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:01:17.284 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:01:17.285 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:01:17.286 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:01:17.286 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:01:17.287 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:01:17.287 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:01:17.287 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:01:17.288 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:01:17.288 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:01:17.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:01:17.289 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:01:17.289 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:01:17.290 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:01:17.290 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:01:17.290 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:01:17.291 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.288+0000 7f3a835c7700 1 -- 172.21.15.27:0/443435283 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3a680640b0 msgr2=0x7f3a68066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:01:17.291 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.288+0000 7f3a835c7700 1 --2- 172.21.15.27:0/443435283 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3a680640b0 0x7f3a68066570 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f3a7400b730 tx=0x7f3a74004d10).stop 2021-11-03T18:01:17.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.288+0000 7f3a835c7700 1 -- 172.21.15.27:0/443435283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a7c0f7970 msgr2=0x7f3a7c1075b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:01:17.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.288+0000 7f3a835c7700 1 --2- 172.21.15.27:0/443435283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a7c0f7970 0x7f3a7c1075b0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 rx=0x7f3a6c00c790 tx=0x7f3a6c007d70).stop 2021-11-03T18:01:17.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.288+0000 7f3a835c7700 1 -- 172.21.15.27:0/443435283 shutdown_connections 2021-11-03T18:01:17.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.288+0000 7f3a835c7700 1 --2- 172.21.15.27:0/443435283 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3a680640b0 0x7f3a68066570 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:17.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.288+0000 7f3a835c7700 1 --2- 172.21.15.27:0/443435283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a7c0f66d0 0x7f3a7c1070c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:17.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.288+0000 7f3a835c7700 1 --2- 172.21.15.27:0/443435283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a7c0f7970 0x7f3a7c1075b0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:17.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.288+0000 7f3a835c7700 1 -- 172.21.15.27:0/443435283 >> 172.21.15.27:0/443435283 conn(0x7f3a7c0f1e10 msgr2=0x7f3a7c0fb3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:01:17.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.288+0000 7f3a835c7700 1 -- 172.21.15.27:0/443435283 shutdown_connections 2021-11-03T18:01:17.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.288+0000 7f3a835c7700 1 -- 172.21.15.27:0/443435283 wait complete. 2021-11-03T18:01:17.387 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.388+0000 7f5f94e17700 1 -- 172.21.15.27:0/3140188456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f900f66e0 msgr2=0x7f5f900f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:01:17.387 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.388+0000 7f5f94e17700 1 --2- 172.21.15.27:0/3140188456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f900f66e0 0x7f5f900f6b00 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7f5f80004660 tx=0x7f5f80009b30).stop 2021-11-03T18:01:17.387 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.389+0000 7f5f94e17700 1 -- 172.21.15.27:0/3140188456 shutdown_connections 2021-11-03T18:01:17.387 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.389+0000 7f5f94e17700 1 --2- 172.21.15.27:0/3140188456 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f900f7980 0x7f5f900f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:17.388 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.389+0000 7f5f94e17700 1 --2- 172.21.15.27:0/3140188456 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f900f66e0 0x7f5f900f6b00 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:17.388 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.389+0000 7f5f94e17700 1 -- 172.21.15.27:0/3140188456 >> 172.21.15.27:0/3140188456 conn(0x7f5f900f1e40 msgr2=0x7f5f900f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:01:17.388 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.389+0000 7f5f94e17700 1 -- 172.21.15.27:0/3140188456 shutdown_connections 2021-11-03T18:01:17.388 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.389+0000 7f5f94e17700 1 -- 172.21.15.27:0/3140188456 wait complete. 2021-11-03T18:01:17.389 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.389+0000 7f5f94e17700 1 Processor -- start 2021-11-03T18:01:17.389 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.389+0000 7f5f94e17700 1 -- start start 2021-11-03T18:01:17.389 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.389+0000 7f5f94e17700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f900f66e0 0x7f5f9006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:01:17.389 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.389+0000 7f5f94e17700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f900f7980 0x7f5f90069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:01:17.390 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.389+0000 7f5f94e17700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9006a380 con 0x7f5f900f7980 2021-11-03T18:01:17.390 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.389+0000 7f5f94e17700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f9006a4c0 con 0x7f5f900f66e0 2021-11-03T18:01:17.390 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.390+0000 7f5f8dd9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f900f7980 0x7f5f90069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:01:17.390 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.390+0000 7f5f8dd9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f900f7980 0x7f5f90069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60738/0 (socket says 172.21.15.27:60738) 2021-11-03T18:01:17.391 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.390+0000 7f5f8dd9b700 1 -- 172.21.15.27:0/3779070151 learned_addr learned my addr 172.21.15.27:0/3779070151 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:01:17.391 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.390+0000 7f5f8e59c700 1 --2- 172.21.15.27:0/3779070151 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f900f66e0 0x7f5f9006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:01:17.391 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.390+0000 7f5f8dd9b700 1 -- 172.21.15.27:0/3779070151 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f900f66e0 msgr2=0x7f5f9006b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:01:17.391 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.390+0000 7f5f8dd9b700 1 --2- 172.21.15.27:0/3779070151 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f900f66e0 0x7f5f9006b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:17.391 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.390+0000 7f5f8dd9b700 1 -- 172.21.15.27:0/3779070151 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f80005040 con 0x7f5f900f7980 2021-11-03T18:01:17.392 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.390+0000 7f5f8dd9b700 1 --2- 172.21.15.27:0/3779070151 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f900f7980 0x7f5f90069e20 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 rx=0x7f5f8400e7f0 tx=0x7f5f8400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:01:17.392 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.390+0000 7f5f7effd700 1 -- 172.21.15.27:0/3779070151 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f840092e0 con 0x7f5f900f7980 2021-11-03T18:01:17.392 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.390+0000 7f5f94e17700 1 -- 172.21.15.27:0/3779070151 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f9006a7a0 con 0x7f5f900f7980 2021-11-03T18:01:17.392 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.390+0000 7f5f7effd700 1 -- 172.21.15.27:0/3779070151 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f5f8400f040 con 0x7f5f900f7980 2021-11-03T18:01:17.393 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.390+0000 7f5f7effd700 1 -- 172.21.15.27:0/3779070151 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f84014750 con 0x7f5f900f7980 2021-11-03T18:01:17.393 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.390+0000 7f5f94e17700 1 -- 172.21.15.27:0/3779070151 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f9010ef90 con 0x7f5f900f7980 2021-11-03T18:01:17.394 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.391+0000 7f5f94e17700 1 -- 172.21.15.27:0/3779070151 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f90045ad0 con 0x7f5f900f7980 2021-11-03T18:01:17.394 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.391+0000 7f5f7effd700 1 -- 172.21.15.27:0/3779070151 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f5f8401b030 con 0x7f5f900f7980 2021-11-03T18:01:17.395 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.392+0000 7f5f7effd700 1 --2- 172.21.15.27:0/3779070151 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5f780640b0 0x7f5f78066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:01:17.395 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.392+0000 7f5f7effd700 1 -- 172.21.15.27:0/3779070151 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f5f840806e0 con 0x7f5f900f7980 2021-11-03T18:01:17.395 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.392+0000 7f5f8e59c700 1 --2- 172.21.15.27:0/3779070151 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5f780640b0 0x7f5f78066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:01:17.395 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.392+0000 7f5f8e59c700 1 --2- 172.21.15.27:0/3779070151 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5f780640b0 0x7f5f78066570 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f5f80000c00 tx=0x7f5f80016040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:01:17.396 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.396+0000 7f5f7effd700 1 -- 172.21.15.27:0/3779070151 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f5f840529a0 con 0x7f5f900f7980 2021-11-03T18:01:17.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.600+0000 7f5f94e17700 1 -- 172.21.15.27:0/3779070151 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5f90027a00 con 0x7f5f900f7980 2021-11-03T18:01:17.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.600+0000 7f5f7effd700 1 -- 172.21.15.27:0/3779070151 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f5f84019710 con 0x7f5f900f7980 2021-11-03T18:01:17.598 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:01:17.598 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:01:17.599 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:01:17.599 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:01:17.600 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:01:17.601 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:01:17.601 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:01:17.602 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:01:17.602 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:01:17.603 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:01:17.603 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:01:17.603 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:01:17.604 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:01:17.604 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:01:17.605 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:01:17.605 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:01:17.606 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:01:17.606 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:01:17.607 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:01:17.607 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:01:17.607 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:01:17.608 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:01:17.608 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:01:17.609 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:01:17.609 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:01:17.610 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:01:17.610 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:01:17.611 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:01:17.611 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:01:17.611 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:01:17.612 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:01:17.612 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:01:17.613 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:01:17.613 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:01:17.614 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:01:17.614 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:01:17.614 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:01:17.615 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:01:17.615 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:01:17.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.602+0000 7f5f94e17700 1 -- 172.21.15.27:0/3779070151 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5f780640b0 msgr2=0x7f5f78066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:01:17.618 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.602+0000 7f5f94e17700 1 --2- 172.21.15.27:0/3779070151 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5f780640b0 0x7f5f78066570 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f5f80000c00 tx=0x7f5f80016040).stop 2021-11-03T18:01:17.618 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.602+0000 7f5f94e17700 1 -- 172.21.15.27:0/3779070151 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f900f7980 msgr2=0x7f5f90069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:01:17.619 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.602+0000 7f5f94e17700 1 --2- 172.21.15.27:0/3779070151 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f900f7980 0x7f5f90069e20 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 rx=0x7f5f8400e7f0 tx=0x7f5f8400b040).stop 2021-11-03T18:01:17.619 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.602+0000 7f5f94e17700 1 -- 172.21.15.27:0/3779070151 shutdown_connections 2021-11-03T18:01:17.620 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.602+0000 7f5f94e17700 1 --2- 172.21.15.27:0/3779070151 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5f780640b0 0x7f5f78066570 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:17.620 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.602+0000 7f5f94e17700 1 --2- 172.21.15.27:0/3779070151 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f900f66e0 0x7f5f9006b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:17.620 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.602+0000 7f5f94e17700 1 --2- 172.21.15.27:0/3779070151 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f900f7980 0x7f5f90069e20 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:17.621 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.602+0000 7f5f94e17700 1 -- 172.21.15.27:0/3779070151 >> 172.21.15.27:0/3779070151 conn(0x7f5f900f1e40 msgr2=0x7f5f900fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:01:17.621 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.602+0000 7f5f94e17700 1 -- 172.21.15.27:0/3779070151 shutdown_connections 2021-11-03T18:01:17.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:17.602+0000 7f5f94e17700 1 -- 172.21.15.27:0/3779070151 wait complete. 2021-11-03T18:01:17.622 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:01:17.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:01:17.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:01:17.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:01:17.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:17.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:17.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:17.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:17.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:17 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/443435283' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:17.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:01:17.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:01:17.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:01:17.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:17.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:17.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:17.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:17.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:17 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/443435283' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:18.735 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:18 smithi115 ceph-mon[65453]: pgmap v1399: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:18.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:18 smithi115 ceph-mon[65453]: from='client.35544 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:01:18.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:18 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:01:18.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:18 smithi115 ceph-mon[65453]: from='client.35548 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:01:18.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:18 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3779070151' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:01:18.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:18 smithi027 ceph-mon[97244]: pgmap v1399: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:18.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:18 smithi027 ceph-mon[97244]: from='client.35544 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:01:18.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:18 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:01:18.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:18 smithi027 ceph-mon[97244]: from='client.35548 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:01:18.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:18 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3779070151' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:01:20.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:20 smithi115 ceph-mon[65453]: pgmap v1400: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:20.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2722.424849 secs 2021-11-03T18:01:20.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:20 smithi027 ceph-mon[97244]: pgmap v1400: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:20.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2722.424849 secs 2021-11-03T18:01:22.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:22 smithi115 ceph-mon[65453]: pgmap v1401: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:22.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:22 smithi027 ceph-mon[97244]: pgmap v1401: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:24.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:24 smithi115 ceph-mon[65453]: pgmap v1402: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:24.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:24 smithi027 ceph-mon[97244]: pgmap v1402: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:25.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2727.424974 secs 2021-11-03T18:01:25.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2727.424974 secs 2021-11-03T18:01:26.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:26 smithi115 ceph-mon[65453]: pgmap v1403: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:26.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:26 smithi027 ceph-mon[97244]: pgmap v1403: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:27.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:01:27.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:01:27.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:01:27.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:27.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:27.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:27.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:27.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:01:27.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:01:27.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:01:27.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:27.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:27.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:27.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:28.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:28 smithi115 ceph-mon[65453]: pgmap v1404: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:28.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:28 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:01:28.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:28 smithi027 ceph-mon[97244]: pgmap v1404: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:28.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:28 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:01:30.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:30 smithi115 ceph-mon[65453]: pgmap v1405: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:30.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2732.425119 secs 2021-11-03T18:01:30.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:30 smithi027 ceph-mon[97244]: pgmap v1405: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:30.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2732.425119 secs 2021-11-03T18:01:32.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:32 smithi115 ceph-mon[65453]: pgmap v1406: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:32.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:32 smithi027 ceph-mon[97244]: pgmap v1406: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:34.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:34 smithi115 ceph-mon[65453]: pgmap v1407: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:34.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:34 smithi027 ceph-mon[97244]: pgmap v1407: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:35.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2737.425264 secs 2021-11-03T18:01:35.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2737.425264 secs 2021-11-03T18:01:36.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:36 smithi115 ceph-mon[65453]: pgmap v1408: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:36.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:36 smithi027 ceph-mon[97244]: pgmap v1408: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:37.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:01:37.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:01:37.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:01:37.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:37.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:37.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:37.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:37.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:01:37.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:01:37.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:01:37.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:37.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:37.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:37.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:38.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:38 smithi115 ceph-mon[65453]: pgmap v1409: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:38.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:38 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:01:38.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:38 smithi027 ceph-mon[97244]: pgmap v1409: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:38.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:38 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:01:40.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:40 smithi115 ceph-mon[65453]: pgmap v1410: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:40.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2742.425380 secs 2021-11-03T18:01:40.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:40 smithi027 ceph-mon[97244]: pgmap v1410: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:40.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2742.425380 secs 2021-11-03T18:01:41.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:41 smithi027 ceph-mon[97244]: pgmap v1411: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:41.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:41 smithi115 ceph-mon[65453]: pgmap v1411: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:43.957 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:43 smithi115 ceph-mon[65453]: pgmap v1412: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:44.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:43 smithi027 ceph-mon[97244]: pgmap v1412: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:45.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2747.425516 secs 2021-11-03T18:01:45.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:45 smithi115 ceph-mon[65453]: pgmap v1413: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:46.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2747.425516 secs 2021-11-03T18:01:46.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:45 smithi027 ceph-mon[97244]: pgmap v1413: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:47.704 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.707+0000 7f5405004700 1 -- 172.21.15.27:0/2265201925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54000f77d0 msgr2=0x7f54000f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:01:47.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.707+0000 7f5405004700 1 --2- 172.21.15.27:0/2265201925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54000f77d0 0x7f54000f7c30 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x7f53f4004660 tx=0x7f53f4009b30).stop 2021-11-03T18:01:47.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.707+0000 7f5405004700 1 -- 172.21.15.27:0/2265201925 shutdown_connections 2021-11-03T18:01:47.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.707+0000 7f5405004700 1 --2- 172.21.15.27:0/2265201925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54000f77d0 0x7f54000f7c30 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:47.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.707+0000 7f5405004700 1 --2- 172.21.15.27:0/2265201925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54000f6530 0x7f54000f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:47.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.707+0000 7f5405004700 1 -- 172.21.15.27:0/2265201925 >> 172.21.15.27:0/2265201925 conn(0x7f54000f1cd0 msgr2=0x7f54000f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:01:47.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.707+0000 7f5405004700 1 -- 172.21.15.27:0/2265201925 shutdown_connections 2021-11-03T18:01:47.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.707+0000 7f5405004700 1 -- 172.21.15.27:0/2265201925 wait complete. 2021-11-03T18:01:47.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.708+0000 7f5405004700 1 Processor -- start 2021-11-03T18:01:47.707 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.708+0000 7f5405004700 1 -- start start 2021-11-03T18:01:47.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.708+0000 7f5405004700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54000f6530 0x7f540006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:01:47.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.708+0000 7f5405004700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54000f77d0 0x7f5400069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:01:47.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.708+0000 7f5405004700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540006a380 con 0x7f54000f6530 2021-11-03T18:01:47.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.708+0000 7f5405004700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f540006a4c0 con 0x7f54000f77d0 2021-11-03T18:01:47.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.708+0000 7f53fffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54000f6530 0x7f540006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:01:47.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.709+0000 7f53fffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54000f6530 0x7f540006b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60754/0 (socket says 172.21.15.27:60754) 2021-11-03T18:01:47.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.709+0000 7f53fffff700 1 -- 172.21.15.27:0/3082294802 learned_addr learned my addr 172.21.15.27:0/3082294802 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:01:47.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.709+0000 7f53ff7fe700 1 --2- 172.21.15.27:0/3082294802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54000f77d0 0x7f5400069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:01:47.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.709+0000 7f53fffff700 1 -- 172.21.15.27:0/3082294802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54000f77d0 msgr2=0x7f5400069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:01:47.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.709+0000 7f53fffff700 1 --2- 172.21.15.27:0/3082294802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54000f77d0 0x7f5400069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:47.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.709+0000 7f53fffff700 1 -- 172.21.15.27:0/3082294802 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f4005040 con 0x7f54000f6530 2021-11-03T18:01:47.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.709+0000 7f53fffff700 1 --2- 172.21.15.27:0/3082294802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54000f6530 0x7f540006b9a0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0x7f53f0002700 tx=0x7f53f0009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:01:47.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.709+0000 7f53fcff9700 1 -- 172.21.15.27:0/3082294802 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53f00079c0 con 0x7f54000f6530 2021-11-03T18:01:47.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.709+0000 7f5405004700 1 -- 172.21.15.27:0/3082294802 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f540006a7a0 con 0x7f54000f6530 2021-11-03T18:01:47.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.709+0000 7f53fcff9700 1 -- 172.21.15.27:0/3082294802 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f53f000f040 con 0x7f54000f6530 2021-11-03T18:01:47.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.709+0000 7f53fcff9700 1 -- 172.21.15.27:0/3082294802 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53f0014860 con 0x7f54000f6530 2021-11-03T18:01:47.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.709+0000 7f5405004700 1 -- 172.21.15.27:0/3082294802 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f540010ed40 con 0x7f54000f6530 2021-11-03T18:01:47.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.710+0000 7f5405004700 1 -- 172.21.15.27:0/3082294802 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5400045ad0 con 0x7f54000f6530 2021-11-03T18:01:47.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.711+0000 7f53fcff9700 1 -- 172.21.15.27:0/3082294802 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f53f001b060 con 0x7f54000f6530 2021-11-03T18:01:47.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.711+0000 7f53fcff9700 1 --2- 172.21.15.27:0/3082294802 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f53e8064100 0x7f53e80665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:01:47.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.711+0000 7f53fcff9700 1 -- 172.21.15.27:0/3082294802 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f53f00806c0 con 0x7f54000f6530 2021-11-03T18:01:47.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.711+0000 7f53ff7fe700 1 --2- 172.21.15.27:0/3082294802 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f53e8064100 0x7f53e80665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:01:47.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.711+0000 7f53ff7fe700 1 --2- 172.21.15.27:0/3082294802 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f53e8064100 0x7f53e80665c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f53f4004630 tx=0x7f53f4009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:01:47.715 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.715+0000 7f53fcff9700 1 -- 172.21.15.27:0/3082294802 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f53f00529c0 con 0x7f54000f6530 2021-11-03T18:01:47.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.893+0000 7f5405004700 1 -- 172.21.15.27:0/3082294802 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f54000fee30 con 0x7f53e8064100 2021-11-03T18:01:47.891 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.895+0000 7f53fcff9700 1 -- 172.21.15.27:0/3082294802 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f54000fee30 con 0x7f53e8064100 2021-11-03T18:01:47.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.897+0000 7f5405004700 1 -- 172.21.15.27:0/3082294802 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f53e8064100 msgr2=0x7f53e80665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:01:47.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.897+0000 7f5405004700 1 --2- 172.21.15.27:0/3082294802 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f53e8064100 0x7f53e80665c0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 rx=0x7f53f4004630 tx=0x7f53f4009b30).stop 2021-11-03T18:01:47.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.897+0000 7f5405004700 1 -- 172.21.15.27:0/3082294802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54000f6530 msgr2=0x7f540006b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:01:47.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.897+0000 7f5405004700 1 --2- 172.21.15.27:0/3082294802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54000f6530 0x7f540006b9a0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0x7f53f0002700 tx=0x7f53f0009d80).stop 2021-11-03T18:01:47.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.897+0000 7f5405004700 1 -- 172.21.15.27:0/3082294802 shutdown_connections 2021-11-03T18:01:47.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.897+0000 7f5405004700 1 --2- 172.21.15.27:0/3082294802 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f53e8064100 0x7f53e80665c0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:47.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.897+0000 7f5405004700 1 --2- 172.21.15.27:0/3082294802 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54000f6530 0x7f540006b9a0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:47.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.897+0000 7f5405004700 1 --2- 172.21.15.27:0/3082294802 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54000f77d0 0x7f5400069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:47.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.897+0000 7f5405004700 1 -- 172.21.15.27:0/3082294802 >> 172.21.15.27:0/3082294802 conn(0x7f54000f1cd0 msgr2=0x7f54000fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:01:47.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.897+0000 7f5405004700 1 -- 172.21.15.27:0/3082294802 shutdown_connections 2021-11-03T18:01:47.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.897+0000 7f5405004700 1 -- 172.21.15.27:0/3082294802 wait complete. 2021-11-03T18:01:47.903 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:01:47.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:47 smithi115 ceph-mon[65453]: pgmap v1414: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:47.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:01:47.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:47 smithi115 ceph-mon[65453]: poll took 60180.107 ms: 1 events 2021-11-03T18:01:47.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:01:47.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:01:47.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:47.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:47.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:47.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:47.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:47 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:01:47.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.998+0000 7f138a87b700 1 -- 172.21.15.27:0/4163005253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13840fbcb0 msgr2=0x7f13840fe0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:01:47.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.998+0000 7f138a87b700 1 --2- 172.21.15.27:0/4163005253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13840fbcb0 0x7f13840fe0a0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 rx=0x7f1374004660 tx=0x7f1374009b30).stop 2021-11-03T18:01:47.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.998+0000 7f138a87b700 1 -- 172.21.15.27:0/4163005253 shutdown_connections 2021-11-03T18:01:47.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.998+0000 7f138a87b700 1 --2- 172.21.15.27:0/4163005253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13840fbcb0 0x7f13840fe0a0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:47.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.998+0000 7f138a87b700 1 --2- 172.21.15.27:0/4163005253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1384060330 0x7f13840fb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:47.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.998+0000 7f138a87b700 1 -- 172.21.15.27:0/4163005253 >> 172.21.15.27:0/4163005253 conn(0x7f13840f0de0 msgr2=0x7f13840f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:01:47.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.998+0000 7f138a87b700 1 -- 172.21.15.27:0/4163005253 shutdown_connections 2021-11-03T18:01:47.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.998+0000 7f138a87b700 1 -- 172.21.15.27:0/4163005253 wait complete. 2021-11-03T18:01:47.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.998+0000 7f138a87b700 1 Processor -- start 2021-11-03T18:01:47.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.998+0000 7f138a87b700 1 -- start start 2021-11-03T18:01:47.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.998+0000 7f138a87b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1384060330 0x7f1384109170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:01:47.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.999+0000 7f138a87b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13840fbcb0 0x7f1384109660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:01:47.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.999+0000 7f138a87b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1384109c10 con 0x7f1384060330 2021-11-03T18:01:47.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.999+0000 7f138a87b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1384109d50 con 0x7f13840fbcb0 2021-11-03T18:01:48.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.999+0000 7f1383fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1384060330 0x7f1384109170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:01:48.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.999+0000 7f1383fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1384060330 0x7f1384109170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60764/0 (socket says 172.21.15.27:60764) 2021-11-03T18:01:48.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.999+0000 7f1383fff700 1 -- 172.21.15.27:0/2488076673 learned_addr learned my addr 172.21.15.27:0/2488076673 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:01:48.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.999+0000 7f13837fe700 1 --2- 172.21.15.27:0/2488076673 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13840fbcb0 0x7f1384109660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:01:48.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.999+0000 7f1383fff700 1 -- 172.21.15.27:0/2488076673 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13840fbcb0 msgr2=0x7f1384109660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:01:48.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.999+0000 7f1383fff700 1 --2- 172.21.15.27:0/2488076673 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13840fbcb0 0x7f1384109660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:48.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.999+0000 7f1383fff700 1 -- 172.21.15.27:0/2488076673 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1374005040 con 0x7f1384060330 2021-11-03T18:01:48.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.999+0000 7f13837fe700 1 --2- 172.21.15.27:0/2488076673 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13840fbcb0 0x7f1384109660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T18:01:48.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.999+0000 7f1383fff700 1 --2- 172.21.15.27:0/2488076673 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1384060330 0x7f1384109170 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0x7f1378002700 tx=0x7f1378009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:01:48.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.999+0000 7f1380ff9700 1 -- 172.21.15.27:0/2488076673 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1378009320 con 0x7f1384060330 2021-11-03T18:01:48.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.999+0000 7f138a87b700 1 -- 172.21.15.27:0/2488076673 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f138410a6f0 con 0x7f1384060330 2021-11-03T18:01:48.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.999+0000 7f1380ff9700 1 -- 172.21.15.27:0/2488076673 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f137800f040 con 0x7f1384060330 2021-11-03T18:01:48.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.999+0000 7f1380ff9700 1 -- 172.21.15.27:0/2488076673 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1378014900 con 0x7f1384060330 2021-11-03T18:01:48.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:47.999+0000 7f138a87b700 1 -- 172.21.15.27:0/2488076673 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f138410acb0 con 0x7f1384060330 2021-11-03T18:01:48.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.000+0000 7f138a87b700 1 -- 172.21.15.27:0/2488076673 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1384045ad0 con 0x7f1384060330 2021-11-03T18:01:48.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.001+0000 7f1380ff9700 1 -- 172.21.15.27:0/2488076673 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f1378009480 con 0x7f1384060330 2021-11-03T18:01:48.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.001+0000 7f1380ff9700 1 --2- 172.21.15.27:0/2488076673 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f136c0640b0 0x7f136c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:01:48.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.001+0000 7f1380ff9700 1 -- 172.21.15.27:0/2488076673 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f1378080ab0 con 0x7f1384060330 2021-11-03T18:01:48.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.001+0000 7f13837fe700 1 --2- 172.21.15.27:0/2488076673 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f136c0640b0 0x7f136c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:01:48.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.001+0000 7f13837fe700 1 --2- 172.21.15.27:0/2488076673 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f136c0640b0 0x7f136c066570 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f1374009da0 tx=0x7f1374009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:01:48.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.005+0000 7f1380ff9700 1 -- 172.21.15.27:0/2488076673 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f1378052db0 con 0x7f1384060330 2021-11-03T18:01:48.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:47 smithi027 ceph-mon[97244]: pgmap v1414: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:48.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:01:48.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:47 smithi027 ceph-mon[97244]: poll took 60180.107 ms: 1 events 2021-11-03T18:01:48.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:01:48.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:01:48.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:48.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:48.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:48.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:48.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:47 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:01:48.176 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.179+0000 7f138a87b700 1 -- 172.21.15.27:0/2488076673 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f138410af30 con 0x7f136c0640b0 2021-11-03T18:01:48.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.194+0000 7f1380ff9700 1 -- 172.21.15.27:0/2488076673 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f138410af30 con 0x7f136c0640b0 2021-11-03T18:01:48.191 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:01:48.191 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (53m) 2m ago 54m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:01:48.191 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (45m) 2m ago 54m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:01:48.191 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (44m) 112s ago 53m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:01:48.192 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (53m) 2m ago 54m 41.8M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:01:48.192 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (46m) 2m ago 49m 26.0M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:01:48.192 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (49m) 2m ago 49m 3845M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:01:48.192 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (45m) 112s ago 49m 25.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:01:48.192 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (47m) 112s ago 49m 26.8M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:01:48.193 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (47m) 2m ago 55m 554M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:01:48.193 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (46m) 112s ago 53m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:01:48.193 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (46m) 2m ago 56m 235M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:01:48.193 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (45m) 112s ago 53m 112M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:01:48.194 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (54m) 2m ago 54m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:01:48.194 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (53m) 112s ago 53m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:01:48.194 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (44m) 2m ago 53m 351M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:01:48.194 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (44m) 2m ago 52m 383M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:01:48.194 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (43m) 2m ago 52m 284M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:01:48.195 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (43m) 112s ago 52m 404M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:01:48.195 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (42m) 112s ago 51m 322M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:01:48.195 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (42m) 112s ago 51m 327M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:01:48.195 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (53m) 2m ago 53m 55.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:01:48.196 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.196+0000 7f138a87b700 1 -- 172.21.15.27:0/2488076673 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f136c0640b0 msgr2=0x7f136c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:01:48.196 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.196+0000 7f138a87b700 1 --2- 172.21.15.27:0/2488076673 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f136c0640b0 0x7f136c066570 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f1374009da0 tx=0x7f1374009b30).stop 2021-11-03T18:01:48.197 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.196+0000 7f138a87b700 1 -- 172.21.15.27:0/2488076673 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1384060330 msgr2=0x7f1384109170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:01:48.197 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.196+0000 7f138a87b700 1 --2- 172.21.15.27:0/2488076673 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1384060330 0x7f1384109170 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0x7f1378002700 tx=0x7f1378009d80).stop 2021-11-03T18:01:48.197 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.196+0000 7f138a87b700 1 -- 172.21.15.27:0/2488076673 shutdown_connections 2021-11-03T18:01:48.197 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.196+0000 7f138a87b700 1 --2- 172.21.15.27:0/2488076673 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f136c0640b0 0x7f136c066570 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:48.198 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.196+0000 7f138a87b700 1 --2- 172.21.15.27:0/2488076673 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1384060330 0x7f1384109170 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:48.198 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.196+0000 7f138a87b700 1 --2- 172.21.15.27:0/2488076673 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13840fbcb0 0x7f1384109660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:48.198 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.196+0000 7f138a87b700 1 -- 172.21.15.27:0/2488076673 >> 172.21.15.27:0/2488076673 conn(0x7f13840f0de0 msgr2=0x7f13840f31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:01:48.198 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.196+0000 7f138a87b700 1 -- 172.21.15.27:0/2488076673 shutdown_connections 2021-11-03T18:01:48.199 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.196+0000 7f138a87b700 1 -- 172.21.15.27:0/2488076673 wait complete. 2021-11-03T18:01:48.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.295+0000 7f4c6805f700 1 -- 172.21.15.27:0/2841800990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c600f7960 msgr2=0x7f4c600f7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:01:48.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.295+0000 7f4c6805f700 1 --2- 172.21.15.27:0/2841800990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c600f7960 0x7f4c600f7dc0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7f4c54004660 tx=0x7f4c54009b30).stop 2021-11-03T18:01:48.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.295+0000 7f4c6805f700 1 -- 172.21.15.27:0/2841800990 shutdown_connections 2021-11-03T18:01:48.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.295+0000 7f4c6805f700 1 --2- 172.21.15.27:0/2841800990 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c600f7960 0x7f4c600f7dc0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:48.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.295+0000 7f4c6805f700 1 --2- 172.21.15.27:0/2841800990 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c600f66c0 0x7f4c600f6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:48.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.295+0000 7f4c6805f700 1 -- 172.21.15.27:0/2841800990 >> 172.21.15.27:0/2841800990 conn(0x7f4c600f1e20 msgr2=0x7f4c600f4240 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:01:48.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.295+0000 7f4c6805f700 1 -- 172.21.15.27:0/2841800990 shutdown_connections 2021-11-03T18:01:48.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.295+0000 7f4c6805f700 1 -- 172.21.15.27:0/2841800990 wait complete. 2021-11-03T18:01:48.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.296+0000 7f4c6805f700 1 Processor -- start 2021-11-03T18:01:48.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.296+0000 7f4c6805f700 1 -- start start 2021-11-03T18:01:48.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.296+0000 7f4c6805f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c600f66c0 0x7f4c6006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:01:48.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.296+0000 7f4c6805f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c600f7960 0x7f4c60069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:01:48.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.296+0000 7f4c6805f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6006a380 con 0x7f4c600f7960 2021-11-03T18:01:48.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.296+0000 7f4c6805f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6006a4c0 con 0x7f4c600f66c0 2021-11-03T18:01:48.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.296+0000 7f4c655fa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c600f7960 0x7f4c60069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:01:48.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.296+0000 7f4c655fa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c600f7960 0x7f4c60069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60778/0 (socket says 172.21.15.27:60778) 2021-11-03T18:01:48.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.296+0000 7f4c655fa700 1 -- 172.21.15.27:0/3523361630 learned_addr learned my addr 172.21.15.27:0/3523361630 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:01:48.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.296+0000 7f4c655fa700 1 -- 172.21.15.27:0/3523361630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c600f66c0 msgr2=0x7f4c6006b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:01:48.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.296+0000 7f4c655fa700 1 --2- 172.21.15.27:0/3523361630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c600f66c0 0x7f4c6006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:01:48.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.296+0000 7f4c655fa700 1 -- 172.21.15.27:0/3523361630 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c54005040 con 0x7f4c600f7960 2021-11-03T18:01:48.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.297+0000 7f4c655fa700 1 --2- 172.21.15.27:0/3523361630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c600f7960 0x7f4c60069e20 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 rx=0x7f4c54009830 tx=0x7f4c54009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:01:48.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.297+0000 7f4c5e7fc700 1 -- 172.21.15.27:0/3523361630 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c54018070 con 0x7f4c600f7960 2021-11-03T18:01:48.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.297+0000 7f4c5e7fc700 1 -- 172.21.15.27:0/3523361630 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f4c54007ba0 con 0x7f4c600f7960 2021-11-03T18:01:48.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.297+0000 7f4c5e7fc700 1 -- 172.21.15.27:0/3523361630 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4c5401c620 con 0x7f4c600f7960 2021-11-03T18:01:48.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.297+0000 7f4c6805f700 1 -- 172.21.15.27:0/3523361630 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c6006a740 con 0x7f4c600f7960 2021-11-03T18:01:48.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.297+0000 7f4c6805f700 1 -- 172.21.15.27:0/3523361630 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c6006aca0 con 0x7f4c600f7960 2021-11-03T18:01:48.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.298+0000 7f4c6805f700 1 -- 172.21.15.27:0/3523361630 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c60045ad0 con 0x7f4c600f7960 2021-11-03T18:01:48.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.298+0000 7f4c5e7fc700 1 -- 172.21.15.27:0/3523361630 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f4c54007d10 con 0x7f4c600f7960 2021-11-03T18:01:48.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.298+0000 7f4c5e7fc700 1 --2- 172.21.15.27:0/3523361630 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4c48064090 0x7f4c48066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:01:48.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.299+0000 7f4c5e7fc700 1 -- 172.21.15.27:0/3523361630 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f4c54082560 con 0x7f4c600f7960 2021-11-03T18:01:48.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.299+0000 7f4c65dfb700 1 --2- 172.21.15.27:0/3523361630 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4c48064090 0x7f4c48066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:01:48.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.299+0000 7f4c65dfb700 1 --2- 172.21.15.27:0/3523361630 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4c48064090 0x7f4c48066550 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f4c6006b730 tx=0x7f4c58004b00).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:01:48.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.303+0000 7f4c5e7fc700 1 -- 172.21.15.27:0/3523361630 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f4c54054920 con 0x7f4c600f7960 2021-11-03T18:01:48.533 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.537+0000 7f4c6805f700 1 -- 172.21.15.27:0/3523361630 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f4c60027a00 con 0x7f4c600f7960 2021-11-03T18:01:48.534 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.537+0000 7f4c5e7fc700 1 -- 172.21.15.27:0/3523361630 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f4c54054510 con 0x7f4c600f7960 2021-11-03T18:01:48.534 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:01:48.535 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:01:48.535 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:01:48.535 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:01:48.535 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:01:48.536 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:01:48.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:01:48.536 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:01:48.536 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:01:48.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:01:48.537 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:01:48.537 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:01:48.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:01:48.537 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:01:48.537 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:01:48.538 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:01:48.538 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:01:48.538 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:01:48.539 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.539+0000 7f4c6805f700 1 -- 172.21.15.27:0/3523361630 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4c48064090 msgr2=0x7f4c48066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:01:48.539 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.539+0000 7f4c6805f700 1 --2- 172.21.15.27:0/3523361630 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4c48064090 0x7f4c48066550 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f4c6006b730 tx=0x7f4c58004b00).stop 2021-11-03T18:01:48.539 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.539+0000 7f4c6805f700 1 -- 172.21.15.27:0/3523361630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c600f7960 msgr2=0x7f4c60069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:01:48.540 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.539+0000 7f4c6805f700 1 --2- 172.21.15.27:0/3523361630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c600f7960 0x7f4c60069e20 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 rx=0x7f4c54009830 tx=0x7f4c54009b30).stop 2021-11-03T18:01:48.540 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.539+0000 7f4c6805f700 1 -- 172.21.15.27:0/3523361630 shutdown_connections 2021-11-03T18:01:48.540 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.539+0000 7f4c6805f700 1 --2- 172.21.15.27:0/3523361630 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4c48064090 0x7f4c48066550 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:48.540 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.539+0000 7f4c6805f700 1 --2- 172.21.15.27:0/3523361630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4c600f66c0 0x7f4c6006b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:01:48.541 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.539+0000 7f4c6805f700 1 --2- 172.21.15.27:0/3523361630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4c600f7960 0x7f4c60069e20 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:48.541 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.539+0000 7f4c6805f700 1 -- 172.21.15.27:0/3523361630 >> 172.21.15.27:0/3523361630 conn(0x7f4c600f1e20 msgr2=0x7f4c600fb3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:01:48.541 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.539+0000 7f4c6805f700 1 -- 172.21.15.27:0/3523361630 shutdown_connections 2021-11-03T18:01:48.541 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.539+0000 7f4c6805f700 1 -- 172.21.15.27:0/3523361630 wait complete. 2021-11-03T18:01:48.639 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.640+0000 7fb8adc8f700 1 -- 172.21.15.27:0/3085855640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a8060390 msgr2=0x7fb8a80f93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:01:48.639 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.640+0000 7fb8adc8f700 1 --2- 172.21.15.27:0/3085855640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a8060390 0x7fb8a80f93a0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7fb898004660 tx=0x7fb898009b30).stop 2021-11-03T18:01:48.639 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.641+0000 7fb8adc8f700 1 -- 172.21.15.27:0/3085855640 shutdown_connections 2021-11-03T18:01:48.640 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.641+0000 7fb8adc8f700 1 --2- 172.21.15.27:0/3085855640 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8a80f9890 0x7fb8a80fbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:48.640 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.641+0000 7fb8adc8f700 1 --2- 172.21.15.27:0/3085855640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a8060390 0x7fb8a80f93a0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:48.640 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.641+0000 7fb8adc8f700 1 -- 172.21.15.27:0/3085855640 >> 172.21.15.27:0/3085855640 conn(0x7fb8a80f0dc0 msgr2=0x7fb8a80f31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:01:48.640 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.641+0000 7fb8adc8f700 1 -- 172.21.15.27:0/3085855640 shutdown_connections 2021-11-03T18:01:48.641 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.641+0000 7fb8adc8f700 1 -- 172.21.15.27:0/3085855640 wait complete. 2021-11-03T18:01:48.641 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.641+0000 7fb8adc8f700 1 Processor -- start 2021-11-03T18:01:48.641 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.641+0000 7fb8adc8f700 1 -- start start 2021-11-03T18:01:48.641 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.641+0000 7fb8adc8f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a8060390 0x7fb8a80f6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:01:48.642 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.641+0000 7fb8adc8f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8a80f9890 0x7fb8a80f52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:01:48.642 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.641+0000 7fb8adc8f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a80f5800 con 0x7fb8a8060390 2021-11-03T18:01:48.642 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.641+0000 7fb8adc8f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a80f5940 con 0x7fb8a80f9890 2021-11-03T18:01:48.642 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.642+0000 7fb8a77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a8060390 0x7fb8a80f6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:01:48.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.642+0000 7fb8a77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a8060390 0x7fb8a80f6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60786/0 (socket says 172.21.15.27:60786) 2021-11-03T18:01:48.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.642+0000 7fb8a77fe700 1 -- 172.21.15.27:0/1713481336 learned_addr learned my addr 172.21.15.27:0/1713481336 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:01:48.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.642+0000 7fb8a6ffd700 1 --2- 172.21.15.27:0/1713481336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8a80f9890 0x7fb8a80f52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:01:48.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.642+0000 7fb8a77fe700 1 -- 172.21.15.27:0/1713481336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8a80f9890 msgr2=0x7fb8a80f52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:01:48.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.642+0000 7fb8a77fe700 1 --2- 172.21.15.27:0/1713481336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8a80f9890 0x7fb8a80f52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:48.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.642+0000 7fb8a77fe700 1 -- 172.21.15.27:0/1713481336 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb898005040 con 0x7fb8a8060390 2021-11-03T18:01:48.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.642+0000 7fb8a77fe700 1 --2- 172.21.15.27:0/1713481336 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a8060390 0x7fb8a80f6e20 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7fb898004f60 tx=0x7fb898009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:01:48.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.642+0000 7fb88ffff700 1 -- 172.21.15.27:0/1713481336 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb898018070 con 0x7fb8a8060390 2021-11-03T18:01:48.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.642+0000 7fb8adc8f700 1 -- 172.21.15.27:0/1713481336 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8a80f5bc0 con 0x7fb8a8060390 2021-11-03T18:01:48.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.642+0000 7fb88ffff700 1 -- 172.21.15.27:0/1713481336 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fb898007ba0 con 0x7fb8a8060390 2021-11-03T18:01:48.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.642+0000 7fb88ffff700 1 -- 172.21.15.27:0/1713481336 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb89801c620 con 0x7fb8a8060390 2021-11-03T18:01:48.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.642+0000 7fb8adc8f700 1 -- 172.21.15.27:0/1713481336 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8a80f6120 con 0x7fb8a8060390 2021-11-03T18:01:48.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.643+0000 7fb8adc8f700 1 -- 172.21.15.27:0/1713481336 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8a8045ad0 con 0x7fb8a8060390 2021-11-03T18:01:48.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.644+0000 7fb88ffff700 1 -- 172.21.15.27:0/1713481336 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fb89801c8e0 con 0x7fb8a8060390 2021-11-03T18:01:48.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.644+0000 7fb88ffff700 1 --2- 172.21.15.27:0/1713481336 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb8900640b0 0x7fb890066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:01:48.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.644+0000 7fb88ffff700 1 -- 172.21.15.27:0/1713481336 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fb898082c00 con 0x7fb8a8060390 2021-11-03T18:01:48.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.644+0000 7fb8a6ffd700 1 --2- 172.21.15.27:0/1713481336 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb8900640b0 0x7fb890066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:01:48.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.645+0000 7fb8a6ffd700 1 --2- 172.21.15.27:0/1713481336 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb8900640b0 0x7fb890066570 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7fb89c0049e0 tx=0x7fb89c009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:01:48.648 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.648+0000 7fb88ffff700 1 -- 172.21.15.27:0/1713481336 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fb898054f00 con 0x7fb8a8060390 2021-11-03T18:01:48.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.851+0000 7fb8adc8f700 1 -- 172.21.15.27:0/1713481336 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb8a8027a00 con 0x7fb8a8060390 2021-11-03T18:01:48.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.852+0000 7fb88ffff700 1 -- 172.21.15.27:0/1713481336 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7fb898054af0 con 0x7fb8a8060390 2021-11-03T18:01:48.849 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:01:48.849 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:01:48.849 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:01:48.850 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:01:48.850 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:01:48.850 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:01:48.850 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:01:48.850 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:01:48.851 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:01:48.851 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:01:48.851 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:01:48.851 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:01:48.852 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:01:48.852 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:01:48.852 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:01:48.852 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:01:48.852 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:01:48.853 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:01:48.853 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:01:48.853 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:01:48.853 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:01:48.854 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:01:48.854 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:01:48.854 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:01:48.854 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:01:48.854 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:01:48.855 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:01:48.855 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:01:48.855 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:01:48.855 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:01:48.856 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:01:48.856 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:01:48.856 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:01:48.856 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:01:48.856 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:01:48.857 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:01:48.857 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:01:48.857 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:01:48.857 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:01:48.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.853+0000 7fb8adc8f700 1 -- 172.21.15.27:0/1713481336 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb8900640b0 msgr2=0x7fb890066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:01:48.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.853+0000 7fb8adc8f700 1 --2- 172.21.15.27:0/1713481336 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb8900640b0 0x7fb890066570 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 rx=0x7fb89c0049e0 tx=0x7fb89c009040).stop 2021-11-03T18:01:48.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.853+0000 7fb8adc8f700 1 -- 172.21.15.27:0/1713481336 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a8060390 msgr2=0x7fb8a80f6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:01:48.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.853+0000 7fb8adc8f700 1 --2- 172.21.15.27:0/1713481336 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a8060390 0x7fb8a80f6e20 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7fb898004f60 tx=0x7fb898009b30).stop 2021-11-03T18:01:48.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.853+0000 7fb8adc8f700 1 -- 172.21.15.27:0/1713481336 shutdown_connections 2021-11-03T18:01:48.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.853+0000 7fb8adc8f700 1 --2- 172.21.15.27:0/1713481336 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb8900640b0 0x7fb890066570 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:48.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.853+0000 7fb8adc8f700 1 --2- 172.21.15.27:0/1713481336 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb8a8060390 0x7fb8a80f6e20 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:48.860 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.853+0000 7fb8adc8f700 1 --2- 172.21.15.27:0/1713481336 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb8a80f9890 0x7fb8a80f52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:01:48.860 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.853+0000 7fb8adc8f700 1 -- 172.21.15.27:0/1713481336 >> 172.21.15.27:0/1713481336 conn(0x7fb8a80f0dc0 msgr2=0x7fb8a80f31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:01:48.860 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.854+0000 7fb8adc8f700 1 -- 172.21.15.27:0/1713481336 shutdown_connections 2021-11-03T18:01:48.860 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:01:48.854+0000 7fb8adc8f700 1 -- 172.21.15.27:0/1713481336 wait complete. 2021-11-03T18:01:48.860 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:01:48.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:48 smithi115 ceph-mon[65453]: from='client.35560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:01:48.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:48 smithi115 ceph-mon[65453]: from='client.35564 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:01:48.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:48 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3523361630' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:49.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:48 smithi027 ceph-mon[97244]: from='client.35560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:01:49.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:48 smithi027 ceph-mon[97244]: from='client.35564 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:01:49.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:48 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3523361630' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:49.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:49 smithi115 ceph-mon[65453]: pgmap v1415: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:49.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:49 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1713481336' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:01:50.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:49 smithi027 ceph-mon[97244]: pgmap v1415: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:50.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:49 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1713481336' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:01:50.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2752.425653 secs 2021-11-03T18:01:51.020 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2752.425653 secs 2021-11-03T18:01:52.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:52 smithi115 ceph-mon[65453]: pgmap v1416: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:52.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:52 smithi027 ceph-mon[97244]: pgmap v1416: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:54.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:54 smithi115 ceph-mon[65453]: pgmap v1417: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:54.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:54 smithi027 ceph-mon[97244]: pgmap v1417: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:55.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2757.425744 secs 2021-11-03T18:01:55.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2757.425744 secs 2021-11-03T18:01:56.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:56 smithi115 ceph-mon[65453]: pgmap v1418: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:56.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:01:56.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:01:56.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:56 smithi027 ceph-mon[97244]: pgmap v1418: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:56.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:01:56.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:01:58.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:58 smithi115 ceph-mon[65453]: pgmap v1419: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:58.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:01:58.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:01:58.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:01:58.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:58.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:58.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:58.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:58.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:01:58 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:01:58.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:58 smithi027 ceph-mon[97244]: pgmap v1419: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:01:58.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:01:58.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:01:58.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:01:58.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:58.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:58.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:58.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:01:58.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:01:58 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:02:00.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:00 smithi115 ceph-mon[65453]: pgmap v1420: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:00.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2762.425878 secs 2021-11-03T18:02:00.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:00 smithi027 ceph-mon[97244]: pgmap v1420: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:00.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2762.425878 secs 2021-11-03T18:02:02.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:02 smithi115 ceph-mon[65453]: pgmap v1421: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:02.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:02 smithi027 ceph-mon[97244]: pgmap v1421: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:04.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:04 smithi115 ceph-mon[65453]: pgmap v1422: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:04.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:04 smithi027 ceph-mon[97244]: pgmap v1422: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:05.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2767.426020 secs 2021-11-03T18:02:05.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2767.426020 secs 2021-11-03T18:02:06.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:06 smithi115 ceph-mon[65453]: pgmap v1423: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:06.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:06 smithi027 ceph-mon[97244]: pgmap v1423: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:08.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:08 smithi115 ceph-mon[65453]: pgmap v1424: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:08.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:02:08.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:02:08.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:02:08.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:08.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:08.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:08.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:08.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:08 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:02:08.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:08 smithi027 ceph-mon[97244]: pgmap v1424: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:08.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:02:08.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:02:08.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:02:08.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:08.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:08.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:08.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:08.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:08 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:02:10.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:10 smithi115 ceph-mon[65453]: pgmap v1425: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:10.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2772.426134 secs 2021-11-03T18:02:10.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:10 smithi027 ceph-mon[97244]: pgmap v1425: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:10.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2772.426134 secs 2021-11-03T18:02:12.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:12 smithi115 ceph-mon[65453]: pgmap v1426: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:12.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:12 smithi027 ceph-mon[97244]: pgmap v1426: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:14.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:14 smithi115 ceph-mon[65453]: pgmap v1427: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:14.524 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:14 smithi027 ceph-mon[97244]: pgmap v1427: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:15.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2777.426261 secs 2021-11-03T18:02:15.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2777.426261 secs 2021-11-03T18:02:16.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:16 smithi115 ceph-mon[65453]: pgmap v1428: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:16.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:16 smithi027 ceph-mon[97244]: pgmap v1428: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:18.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:18 smithi115 ceph-mon[65453]: pgmap v1429: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:18.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:02:18.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:02:18.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:02:18.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:18.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:18.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:18.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:18.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:18 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:02:18.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:18 smithi027 ceph-mon[97244]: pgmap v1429: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:18.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:02:18.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:02:18.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:02:18.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:18.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:18.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:18.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:18.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:18 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:02:18.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:18.967+0000 7f83c2a8d700 1 -- 172.21.15.27:0/1320368037 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83bc0f6530 msgr2=0x7f83bc0f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:02:18.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:18.967+0000 7f83c2a8d700 1 --2- 172.21.15.27:0/1320368037 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83bc0f6530 0x7f83bc0f6950 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 rx=0x7f83ac004660 tx=0x7f83ac009b30).stop 2021-11-03T18:02:18.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:18.968+0000 7f83c2a8d700 1 -- 172.21.15.27:0/1320368037 shutdown_connections 2021-11-03T18:02:18.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:18.968+0000 7f83c2a8d700 1 --2- 172.21.15.27:0/1320368037 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83bc0f77d0 0x7f83bc0f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:18.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:18.968+0000 7f83c2a8d700 1 --2- 172.21.15.27:0/1320368037 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83bc0f6530 0x7f83bc0f6950 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:18.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:18.968+0000 7f83c2a8d700 1 -- 172.21.15.27:0/1320368037 >> 172.21.15.27:0/1320368037 conn(0x7f83bc0f1cd0 msgr2=0x7f83bc0f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:02:18.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:18.968+0000 7f83c2a8d700 1 -- 172.21.15.27:0/1320368037 shutdown_connections 2021-11-03T18:02:18.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:18.968+0000 7f83c2a8d700 1 -- 172.21.15.27:0/1320368037 wait complete. 2021-11-03T18:02:18.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:18.968+0000 7f83c2a8d700 1 Processor -- start 2021-11-03T18:02:18.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:18.968+0000 7f83c2a8d700 1 -- start start 2021-11-03T18:02:18.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:18.968+0000 7f83c2a8d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83bc0f6530 0x7f83bc106d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:02:18.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:18.968+0000 7f83c2a8d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83bc0f77d0 0x7f83bc107260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:02:18.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:18.968+0000 7f83c2a8d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc107860 con 0x7f83bc0f6530 2021-11-03T18:02:18.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:18.968+0000 7f83c2a8d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc108070 con 0x7f83bc0f77d0 2021-11-03T18:02:18.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:18.968+0000 7f83c1a8b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83bc0f6530 0x7f83bc106d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:02:18.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:18.970+0000 7f83c1a8b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83bc0f6530 0x7f83bc106d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60806/0 (socket says 172.21.15.27:60806) 2021-11-03T18:02:18.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:18.970+0000 7f83c1a8b700 1 -- 172.21.15.27:0/505478881 learned_addr learned my addr 172.21.15.27:0/505478881 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:02:18.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:18.970+0000 7f83c128a700 1 --2- 172.21.15.27:0/505478881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83bc0f77d0 0x7f83bc107260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:02:18.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:18.970+0000 7f83c1a8b700 1 -- 172.21.15.27:0/505478881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83bc0f77d0 msgr2=0x7f83bc107260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:02:18.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:18.970+0000 7f83c1a8b700 1 --2- 172.21.15.27:0/505478881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83bc0f77d0 0x7f83bc107260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:18.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:18.970+0000 7f83c1a8b700 1 -- 172.21.15.27:0/505478881 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83ac005040 con 0x7f83bc0f6530 2021-11-03T18:02:18.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:18.970+0000 7f83c1a8b700 1 --2- 172.21.15.27:0/505478881 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83bc0f6530 0x7f83bc106d70 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0x7f83ac013a70 tx=0x7f83ac009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:02:18.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:18.970+0000 7f83b27fc700 1 -- 172.21.15.27:0/505478881 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83ac018070 con 0x7f83bc0f6530 2021-11-03T18:02:18.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:18.970+0000 7f83b27fc700 1 -- 172.21.15.27:0/505478881 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f83ac0097d0 con 0x7f83bc0f6530 2021-11-03T18:02:18.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:18.970+0000 7f83c2a8d700 1 -- 172.21.15.27:0/505478881 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83bc108290 con 0x7f83bc0f6530 2021-11-03T18:02:18.973 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:18.970+0000 7f83b27fc700 1 -- 172.21.15.27:0/505478881 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f83ac01de80 con 0x7f83bc0f6530 2021-11-03T18:02:18.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:18.970+0000 7f83c2a8d700 1 -- 172.21.15.27:0/505478881 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83bc1087f0 con 0x7f83bc0f6530 2021-11-03T18:02:18.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:18.970+0000 7f83c2a8d700 1 -- 172.21.15.27:0/505478881 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83bc059100 con 0x7f83bc0f6530 2021-11-03T18:02:18.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:18.971+0000 7f83b27fc700 1 -- 172.21.15.27:0/505478881 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f83ac007ba0 con 0x7f83bc0f6530 2021-11-03T18:02:18.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:18.971+0000 7f83b27fc700 1 --2- 172.21.15.27:0/505478881 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f83a8064100 0x7f83a80665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:02:18.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:18.971+0000 7f83b27fc700 1 -- 172.21.15.27:0/505478881 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f83ac082400 con 0x7f83bc0f6530 2021-11-03T18:02:18.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:18.971+0000 7f83c128a700 1 --2- 172.21.15.27:0/505478881 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f83a8064100 0x7f83a80665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:02:18.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:18.972+0000 7f83c128a700 1 --2- 172.21.15.27:0/505478881 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f83a8064100 0x7f83a80665c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f83b8004ce0 tx=0x7f83b800b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:02:18.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:18.976+0000 7f83b27fc700 1 -- 172.21.15.27:0/505478881 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f83ac054700 con 0x7f83bc0f6530 2021-11-03T18:02:19.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.158+0000 7f83c2a8d700 1 -- 172.21.15.27:0/505478881 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f83bc0fee60 con 0x7f83a8064100 2021-11-03T18:02:19.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.160+0000 7f83b27fc700 1 -- 172.21.15.27:0/505478881 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f83bc0fee60 con 0x7f83a8064100 2021-11-03T18:02:19.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.161+0000 7f83c2a8d700 1 -- 172.21.15.27:0/505478881 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f83a8064100 msgr2=0x7f83a80665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:02:19.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.161+0000 7f83c2a8d700 1 --2- 172.21.15.27:0/505478881 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f83a8064100 0x7f83a80665c0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f83b8004ce0 tx=0x7f83b800b040).stop 2021-11-03T18:02:19.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.161+0000 7f83c2a8d700 1 -- 172.21.15.27:0/505478881 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83bc0f6530 msgr2=0x7f83bc106d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:02:19.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.161+0000 7f83c2a8d700 1 --2- 172.21.15.27:0/505478881 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83bc0f6530 0x7f83bc106d70 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0x7f83ac013a70 tx=0x7f83ac009b30).stop 2021-11-03T18:02:19.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.162+0000 7f83c2a8d700 1 -- 172.21.15.27:0/505478881 shutdown_connections 2021-11-03T18:02:19.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.162+0000 7f83c2a8d700 1 --2- 172.21.15.27:0/505478881 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f83a8064100 0x7f83a80665c0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:19.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.162+0000 7f83c2a8d700 1 --2- 172.21.15.27:0/505478881 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f83bc0f6530 0x7f83bc106d70 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:19.160 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.162+0000 7f83c2a8d700 1 --2- 172.21.15.27:0/505478881 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f83bc0f77d0 0x7f83bc107260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:19.160 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.162+0000 7f83c2a8d700 1 -- 172.21.15.27:0/505478881 >> 172.21.15.27:0/505478881 conn(0x7f83bc0f1cd0 msgr2=0x7f83bc0fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:02:19.160 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.162+0000 7f83c2a8d700 1 -- 172.21.15.27:0/505478881 shutdown_connections 2021-11-03T18:02:19.160 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.162+0000 7f83c2a8d700 1 -- 172.21.15.27:0/505478881 wait complete. 2021-11-03T18:02:19.168 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:02:19.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.261+0000 7f3f14829700 1 -- 172.21.15.27:0/615193665 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f100f77d0 msgr2=0x7f3f100f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:02:19.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.261+0000 7f3f14829700 1 --2- 172.21.15.27:0/615193665 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f100f77d0 0x7f3f100f7c30 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7f3f04004660 tx=0x7f3f04009b30).stop 2021-11-03T18:02:19.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.261+0000 7f3f14829700 1 -- 172.21.15.27:0/615193665 shutdown_connections 2021-11-03T18:02:19.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.261+0000 7f3f14829700 1 --2- 172.21.15.27:0/615193665 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f100f77d0 0x7f3f100f7c30 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:19.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.261+0000 7f3f14829700 1 --2- 172.21.15.27:0/615193665 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f100f6530 0x7f3f100f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:19.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.261+0000 7f3f14829700 1 -- 172.21.15.27:0/615193665 >> 172.21.15.27:0/615193665 conn(0x7f3f100f1cd0 msgr2=0x7f3f100f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:02:19.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.261+0000 7f3f14829700 1 -- 172.21.15.27:0/615193665 shutdown_connections 2021-11-03T18:02:19.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.261+0000 7f3f14829700 1 -- 172.21.15.27:0/615193665 wait complete. 2021-11-03T18:02:19.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.262+0000 7f3f14829700 1 Processor -- start 2021-11-03T18:02:19.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.262+0000 7f3f14829700 1 -- start start 2021-11-03T18:02:19.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.262+0000 7f3f14829700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f100f6530 0x7f3f1006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:02:19.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.262+0000 7f3f14829700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f100f77d0 0x7f3f10069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:02:19.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.262+0000 7f3f14829700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1006a380 con 0x7f3f100f6530 2021-11-03T18:02:19.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.262+0000 7f3f14829700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1006a4c0 con 0x7f3f100f77d0 2021-11-03T18:02:19.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.262+0000 7f3f0ed9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f100f6530 0x7f3f1006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:02:19.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.262+0000 7f3f0ed9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f100f6530 0x7f3f1006b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60816/0 (socket says 172.21.15.27:60816) 2021-11-03T18:02:19.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.262+0000 7f3f0ed9d700 1 -- 172.21.15.27:0/1881611918 learned_addr learned my addr 172.21.15.27:0/1881611918 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:02:19.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.262+0000 7f3f0e59c700 1 --2- 172.21.15.27:0/1881611918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f100f77d0 0x7f3f10069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:02:19.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.263+0000 7f3f0ed9d700 1 -- 172.21.15.27:0/1881611918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f100f77d0 msgr2=0x7f3f10069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:02:19.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.263+0000 7f3f0ed9d700 1 --2- 172.21.15.27:0/1881611918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f100f77d0 0x7f3f10069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:19.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.263+0000 7f3f0ed9d700 1 -- 172.21.15.27:0/1881611918 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f04005040 con 0x7f3f100f6530 2021-11-03T18:02:19.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.263+0000 7f3f0ed9d700 1 --2- 172.21.15.27:0/1881611918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f100f6530 0x7f3f1006b9a0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 rx=0x7f3f0000c790 tx=0x7f3f00007d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:02:19.265 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.263+0000 7f3ef77fe700 1 -- 172.21.15.27:0/1881611918 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f00013630 con 0x7f3f100f6530 2021-11-03T18:02:19.265 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.263+0000 7f3ef77fe700 1 -- 172.21.15.27:0/1881611918 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f3f00012860 con 0x7f3f100f6530 2021-11-03T18:02:19.265 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.263+0000 7f3f14829700 1 -- 172.21.15.27:0/1881611918 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f1006a7a0 con 0x7f3f100f6530 2021-11-03T18:02:19.265 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.263+0000 7f3ef77fe700 1 -- 172.21.15.27:0/1881611918 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f00002780 con 0x7f3f100f6530 2021-11-03T18:02:19.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.263+0000 7f3f14829700 1 -- 172.21.15.27:0/1881611918 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f1010ed40 con 0x7f3f100f6530 2021-11-03T18:02:19.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.264+0000 7f3f14829700 1 -- 172.21.15.27:0/1881611918 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f10045ad0 con 0x7f3f100f6530 2021-11-03T18:02:19.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.264+0000 7f3ef77fe700 1 -- 172.21.15.27:0/1881611918 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f3f00010070 con 0x7f3f100f6530 2021-11-03T18:02:19.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.265+0000 7f3ef77fe700 1 --2- 172.21.15.27:0/1881611918 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3ef8064100 0x7f3ef80665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:02:19.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.265+0000 7f3ef77fe700 1 -- 172.21.15.27:0/1881611918 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f3f000809c0 con 0x7f3f100f6530 2021-11-03T18:02:19.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.265+0000 7f3f0e59c700 1 --2- 172.21.15.27:0/1881611918 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3ef8064100 0x7f3ef80665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:02:19.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.265+0000 7f3f0e59c700 1 --2- 172.21.15.27:0/1881611918 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3ef8064100 0x7f3ef80665c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f3f04004630 tx=0x7f3f04009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:02:19.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.269+0000 7f3ef77fe700 1 -- 172.21.15.27:0/1881611918 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f3f00052b70 con 0x7f3f100f6530 2021-11-03T18:02:19.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.441+0000 7f3f14829700 1 -- 172.21.15.27:0/1881611918 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f100fee00 con 0x7f3ef8064100 2021-11-03T18:02:19.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.456+0000 7f3ef77fe700 1 -- 172.21.15.27:0/1881611918 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f3f100fee00 con 0x7f3ef8064100 2021-11-03T18:02:19.452 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:02:19.453 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (53m) 3m ago 55m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:02:19.453 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (45m) 3m ago 55m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:02:19.453 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (45m) 2m ago 54m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:02:19.453 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (53m) 3m ago 54m 41.8M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:02:19.454 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (47m) 3m ago 50m 26.0M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:02:19.454 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (50m) 3m ago 50m 3845M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:02:19.454 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (46m) 2m ago 50m 25.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:02:19.454 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (47m) 2m ago 50m 26.8M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:02:19.455 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (47m) 3m ago 56m 554M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:02:19.455 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (46m) 2m ago 54m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:02:19.455 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (46m) 3m ago 56m 235M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:02:19.455 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (46m) 2m ago 54m 112M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:02:19.456 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (54m) 3m ago 54m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:02:19.456 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (54m) 2m ago 54m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:02:19.456 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (45m) 3m ago 53m 351M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:02:19.456 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (44m) 3m ago 53m 383M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:02:19.457 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (44m) 3m ago 52m 284M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:02:19.457 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (43m) 2m ago 52m 404M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:02:19.457 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (43m) 2m ago 52m 322M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:02:19.457 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (42m) 2m ago 51m 327M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:02:19.457 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (53m) 3m ago 54m 55.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:02:19.458 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.457+0000 7f3f14829700 1 -- 172.21.15.27:0/1881611918 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3ef8064100 msgr2=0x7f3ef80665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:02:19.458 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.457+0000 7f3f14829700 1 --2- 172.21.15.27:0/1881611918 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3ef8064100 0x7f3ef80665c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 rx=0x7f3f04004630 tx=0x7f3f04009b30).stop 2021-11-03T18:02:19.459 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.457+0000 7f3f14829700 1 -- 172.21.15.27:0/1881611918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f100f6530 msgr2=0x7f3f1006b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:02:19.459 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.457+0000 7f3f14829700 1 --2- 172.21.15.27:0/1881611918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f100f6530 0x7f3f1006b9a0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 rx=0x7f3f0000c790 tx=0x7f3f00007d90).stop 2021-11-03T18:02:19.459 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.457+0000 7f3f14829700 1 -- 172.21.15.27:0/1881611918 shutdown_connections 2021-11-03T18:02:19.459 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.457+0000 7f3f14829700 1 --2- 172.21.15.27:0/1881611918 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3ef8064100 0x7f3ef80665c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:19.460 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.457+0000 7f3f14829700 1 --2- 172.21.15.27:0/1881611918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f100f6530 0x7f3f1006b9a0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:19.460 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.457+0000 7f3f14829700 1 --2- 172.21.15.27:0/1881611918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f100f77d0 0x7f3f10069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:19.460 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.457+0000 7f3f14829700 1 -- 172.21.15.27:0/1881611918 >> 172.21.15.27:0/1881611918 conn(0x7f3f100f1cd0 msgr2=0x7f3f100fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:02:19.460 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.458+0000 7f3f14829700 1 -- 172.21.15.27:0/1881611918 shutdown_connections 2021-11-03T18:02:19.461 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.458+0000 7f3f14829700 1 -- 172.21.15.27:0/1881611918 wait complete. 2021-11-03T18:02:19.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.556+0000 7f549704a700 1 -- 172.21.15.27:0/421281366 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54900f55e0 msgr2=0x7f54900f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:02:19.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.556+0000 7f549704a700 1 --2- 172.21.15.27:0/421281366 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54900f55e0 0x7f54900f5a00 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0x7f54880099e0 tx=0x7f5488004d10).stop 2021-11-03T18:02:19.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.556+0000 7f549704a700 1 -- 172.21.15.27:0/421281366 shutdown_connections 2021-11-03T18:02:19.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.556+0000 7f549704a700 1 --2- 172.21.15.27:0/421281366 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54900f5ef0 0x7f54900f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:19.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.556+0000 7f549704a700 1 --2- 172.21.15.27:0/421281366 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54900f55e0 0x7f54900f5a00 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:19.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.556+0000 7f549704a700 1 -- 172.21.15.27:0/421281366 >> 172.21.15.27:0/421281366 conn(0x7f54900f0de0 msgr2=0x7f54900f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:02:19.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.556+0000 7f549704a700 1 -- 172.21.15.27:0/421281366 shutdown_connections 2021-11-03T18:02:19.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.556+0000 7f549704a700 1 -- 172.21.15.27:0/421281366 wait complete. 2021-11-03T18:02:19.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.557+0000 7f549704a700 1 Processor -- start 2021-11-03T18:02:19.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.557+0000 7f549704a700 1 -- start start 2021-11-03T18:02:19.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.557+0000 7f549704a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54900f55e0 0x7f5490107070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:02:19.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.557+0000 7f549704a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54900f5ef0 0x7f5490107560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:02:19.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.557+0000 7f549704a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5490107b10 con 0x7f54900f55e0 2021-11-03T18:02:19.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.557+0000 7f549704a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5490107c50 con 0x7f54900f5ef0 2021-11-03T18:02:19.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.557+0000 7f5494de6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54900f55e0 0x7f5490107070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:02:19.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.557+0000 7f5494de6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54900f55e0 0x7f5490107070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60826/0 (socket says 172.21.15.27:60826) 2021-11-03T18:02:19.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.557+0000 7f5494de6700 1 -- 172.21.15.27:0/3297650967 learned_addr learned my addr 172.21.15.27:0/3297650967 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:02:19.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.557+0000 7f5494de6700 1 -- 172.21.15.27:0/3297650967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54900f5ef0 msgr2=0x7f5490107560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:02:19.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.557+0000 7f5494de6700 1 --2- 172.21.15.27:0/3297650967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54900f5ef0 0x7f5490107560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:02:19.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.557+0000 7f5494de6700 1 -- 172.21.15.27:0/3297650967 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5488009640 con 0x7f54900f55e0 2021-11-03T18:02:19.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.557+0000 7f5494de6700 1 --2- 172.21.15.27:0/3297650967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54900f55e0 0x7f5490107070 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 rx=0x7f5488004b70 tx=0x7f5488004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:02:19.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.558+0000 7f548d7fa700 1 -- 172.21.15.27:0/3297650967 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f548800e030 con 0x7f54900f55e0 2021-11-03T18:02:19.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.558+0000 7f549704a700 1 -- 172.21.15.27:0/3297650967 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5490108590 con 0x7f54900f55e0 2021-11-03T18:02:19.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.558+0000 7f548d7fa700 1 -- 172.21.15.27:0/3297650967 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f54880049b0 con 0x7f54900f55e0 2021-11-03T18:02:19.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.558+0000 7f548d7fa700 1 -- 172.21.15.27:0/3297650967 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5488018070 con 0x7f54900f55e0 2021-11-03T18:02:19.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.558+0000 7f549704a700 1 -- 172.21.15.27:0/3297650967 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5490108a70 con 0x7f54900f55e0 2021-11-03T18:02:19.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.559+0000 7f549704a700 1 -- 172.21.15.27:0/3297650967 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5490045ad0 con 0x7f54900f55e0 2021-11-03T18:02:19.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.559+0000 7f548d7fa700 1 -- 172.21.15.27:0/3297650967 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f548801db40 con 0x7f54900f55e0 2021-11-03T18:02:19.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.559+0000 7f548d7fa700 1 --2- 172.21.15.27:0/3297650967 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f54780640b0 0x7f5478066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:02:19.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.559+0000 7f548d7fa700 1 -- 172.21.15.27:0/3297650967 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f5488082820 con 0x7f54900f55e0 2021-11-03T18:02:19.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.560+0000 7f548ffff700 1 --2- 172.21.15.27:0/3297650967 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f54780640b0 0x7f5478066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:02:19.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.560+0000 7f548ffff700 1 --2- 172.21.15.27:0/3297650967 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f54780640b0 0x7f5478066570 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f5484004870 tx=0x7f54840079c0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:02:19.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.563+0000 7f548d7fa700 1 -- 172.21.15.27:0/3297650967 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f5488054ac0 con 0x7f54900f55e0 2021-11-03T18:02:19.788 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.791+0000 7f549704a700 1 -- 172.21.15.27:0/3297650967 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f54900008d0 con 0x7f54900f55e0 2021-11-03T18:02:19.788 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.792+0000 7f548d7fa700 1 -- 172.21.15.27:0/3297650967 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f54880546b0 con 0x7f54900f55e0 2021-11-03T18:02:19.789 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:02:19.789 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:02:19.789 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:02:19.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:02:19.790 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:02:19.790 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:02:19.790 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:02:19.790 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:02:19.791 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:02:19.791 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:02:19.791 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:02:19.791 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:02:19.792 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:02:19.792 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:02:19.792 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:02:19.792 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:02:19.792 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:02:19.793 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:02:19.794 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.794+0000 7f549704a700 1 -- 172.21.15.27:0/3297650967 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f54780640b0 msgr2=0x7f5478066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:02:19.794 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.794+0000 7f549704a700 1 --2- 172.21.15.27:0/3297650967 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f54780640b0 0x7f5478066570 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f5484004870 tx=0x7f54840079c0).stop 2021-11-03T18:02:19.794 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.794+0000 7f549704a700 1 -- 172.21.15.27:0/3297650967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54900f55e0 msgr2=0x7f5490107070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:02:19.794 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.794+0000 7f549704a700 1 --2- 172.21.15.27:0/3297650967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54900f55e0 0x7f5490107070 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 rx=0x7f5488004b70 tx=0x7f5488004d10).stop 2021-11-03T18:02:19.795 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.794+0000 7f549704a700 1 -- 172.21.15.27:0/3297650967 shutdown_connections 2021-11-03T18:02:19.795 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.794+0000 7f549704a700 1 --2- 172.21.15.27:0/3297650967 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f54780640b0 0x7f5478066570 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:19.795 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.794+0000 7f549704a700 1 --2- 172.21.15.27:0/3297650967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54900f55e0 0x7f5490107070 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:19.795 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.794+0000 7f549704a700 1 --2- 172.21.15.27:0/3297650967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54900f5ef0 0x7f5490107560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:02:19.795 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.794+0000 7f549704a700 1 -- 172.21.15.27:0/3297650967 >> 172.21.15.27:0/3297650967 conn(0x7f54900f0de0 msgr2=0x7f54900fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:02:19.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.794+0000 7f549704a700 1 -- 172.21.15.27:0/3297650967 shutdown_connections 2021-11-03T18:02:19.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.794+0000 7f549704a700 1 -- 172.21.15.27:0/3297650967 wait complete. 2021-11-03T18:02:19.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.895+0000 7f4e1ed93700 1 -- 172.21.15.27:0/2757612697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e180f75a0 msgr2=0x7f4e180fdfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:02:19.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.895+0000 7f4e1ed93700 1 --2- 172.21.15.27:0/2757612697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e180f75a0 0x7f4e180fdfc0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0x7f4e08009a10 tx=0x7f4e08004d10).stop 2021-11-03T18:02:19.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.895+0000 7f4e1ed93700 1 -- 172.21.15.27:0/2757612697 shutdown_connections 2021-11-03T18:02:19.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.895+0000 7f4e1ed93700 1 --2- 172.21.15.27:0/2757612697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e180f75a0 0x7f4e180fdfc0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:19.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.895+0000 7f4e1ed93700 1 --2- 172.21.15.27:0/2757612697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e180f6c90 0x7f4e180f70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:19.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.895+0000 7f4e1ed93700 1 -- 172.21.15.27:0/2757612697 >> 172.21.15.27:0/2757612697 conn(0x7f4e180f04c0 msgr2=0x7f4e180f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:02:19.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.895+0000 7f4e1ed93700 1 -- 172.21.15.27:0/2757612697 shutdown_connections 2021-11-03T18:02:19.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.895+0000 7f4e1ed93700 1 -- 172.21.15.27:0/2757612697 wait complete. 2021-11-03T18:02:19.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.895+0000 7f4e1ed93700 1 Processor -- start 2021-11-03T18:02:19.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.895+0000 7f4e1ed93700 1 -- start start 2021-11-03T18:02:19.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.896+0000 7f4e1ed93700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e180f6c90 0x7f4e18109200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:02:19.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.896+0000 7f4e1ed93700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e180f75a0 0x7f4e181096f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:02:19.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.896+0000 7f4e1ed93700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e18109ca0 con 0x7f4e180f6c90 2021-11-03T18:02:19.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.896+0000 7f4e1ed93700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e18109de0 con 0x7f4e180f75a0 2021-11-03T18:02:19.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.896+0000 7f4e1cb2f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e180f6c90 0x7f4e18109200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:02:19.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.896+0000 7f4e1cb2f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e180f6c90 0x7f4e18109200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60836/0 (socket says 172.21.15.27:60836) 2021-11-03T18:02:19.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.896+0000 7f4e1cb2f700 1 -- 172.21.15.27:0/250315608 learned_addr learned my addr 172.21.15.27:0/250315608 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:02:19.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.896+0000 7f4e17fff700 1 --2- 172.21.15.27:0/250315608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e180f75a0 0x7f4e181096f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:02:19.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.896+0000 7f4e1cb2f700 1 -- 172.21.15.27:0/250315608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e180f75a0 msgr2=0x7f4e181096f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:02:19.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.896+0000 7f4e1cb2f700 1 --2- 172.21.15.27:0/250315608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e180f75a0 0x7f4e181096f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:19.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.896+0000 7f4e1cb2f700 1 -- 172.21.15.27:0/250315608 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e08009640 con 0x7f4e180f6c90 2021-11-03T18:02:19.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.896+0000 7f4e17fff700 1 --2- 172.21.15.27:0/250315608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e180f75a0 0x7f4e181096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:02:19.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.896+0000 7f4e1cb2f700 1 --2- 172.21.15.27:0/250315608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e180f6c90 0x7f4e18109200 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 rx=0x7f4e1000c530 tx=0x7f4e10007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:02:19.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.896+0000 7f4e157fa700 1 -- 172.21.15.27:0/250315608 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e1000f650 con 0x7f4e180f6c90 2021-11-03T18:02:19.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.896+0000 7f4e1ed93700 1 -- 172.21.15.27:0/250315608 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e1810a780 con 0x7f4e180f6c90 2021-11-03T18:02:19.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.896+0000 7f4e157fa700 1 -- 172.21.15.27:0/250315608 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f4e1000fc60 con 0x7f4e180f6c90 2021-11-03T18:02:19.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.896+0000 7f4e157fa700 1 -- 172.21.15.27:0/250315608 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4e10012cd0 con 0x7f4e180f6c90 2021-11-03T18:02:19.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.896+0000 7f4e1ed93700 1 -- 172.21.15.27:0/250315608 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e1810ad40 con 0x7f4e180f6c90 2021-11-03T18:02:19.900 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.897+0000 7f4e1ed93700 1 -- 172.21.15.27:0/250315608 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e18045ad0 con 0x7f4e180f6c90 2021-11-03T18:02:19.901 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.898+0000 7f4e157fa700 1 -- 172.21.15.27:0/250315608 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f4e10010070 con 0x7f4e180f6c90 2021-11-03T18:02:19.901 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.898+0000 7f4e157fa700 1 --2- 172.21.15.27:0/250315608 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4e040640b0 0x7f4e04066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:02:19.901 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.898+0000 7f4e157fa700 1 -- 172.21.15.27:0/250315608 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f4e10080740 con 0x7f4e180f6c90 2021-11-03T18:02:19.901 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.898+0000 7f4e17fff700 1 --2- 172.21.15.27:0/250315608 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4e040640b0 0x7f4e04066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:02:19.902 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.899+0000 7f4e17fff700 1 --2- 172.21.15.27:0/250315608 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4e040640b0 0x7f4e04066570 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f4e08005010 tx=0x7f4e08004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:02:19.902 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:19.902+0000 7f4e157fa700 1 -- 172.21.15.27:0/250315608 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f4e10052950 con 0x7f4e180f6c90 2021-11-03T18:02:20.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:20.111+0000 7f4e1ed93700 1 -- 172.21.15.27:0/250315608 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4e181011d0 con 0x7f4e180f6c90 2021-11-03T18:02:20.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:20.112+0000 7f4e157fa700 1 -- 172.21.15.27:0/250315608 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f4e10052540 con 0x7f4e180f6c90 2021-11-03T18:02:20.109 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:02:20.109 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:02:20.110 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:02:20.110 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:02:20.110 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:02:20.110 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:02:20.111 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:02:20.111 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:02:20.111 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:02:20.111 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:02:20.112 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:02:20.112 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:02:20.112 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:02:20.112 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:02:20.112 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:02:20.113 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:02:20.113 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:02:20.113 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:02:20.113 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:02:20.114 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:02:20.114 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:02:20.114 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:02:20.114 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:02:20.114 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:02:20.115 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:02:20.115 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:02:20.115 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:02:20.115 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:02:20.116 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:02:20.116 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:02:20.116 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:02:20.116 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:02:20.116 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:02:20.117 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:02:20.117 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:02:20.117 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:02:20.117 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:02:20.118 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:02:20.118 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:02:20.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:20.114+0000 7f4e1ed93700 1 -- 172.21.15.27:0/250315608 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4e040640b0 msgr2=0x7f4e04066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:02:20.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:20.114+0000 7f4e1ed93700 1 --2- 172.21.15.27:0/250315608 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4e040640b0 0x7f4e04066570 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f4e08005010 tx=0x7f4e08004d10).stop 2021-11-03T18:02:20.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:20.114+0000 7f4e1ed93700 1 -- 172.21.15.27:0/250315608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e180f6c90 msgr2=0x7f4e18109200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:02:20.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:20.114+0000 7f4e1ed93700 1 --2- 172.21.15.27:0/250315608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e180f6c90 0x7f4e18109200 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 rx=0x7f4e1000c530 tx=0x7f4e10007e20).stop 2021-11-03T18:02:20.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:20.114+0000 7f4e1ed93700 1 -- 172.21.15.27:0/250315608 shutdown_connections 2021-11-03T18:02:20.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:20.114+0000 7f4e1ed93700 1 --2- 172.21.15.27:0/250315608 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4e040640b0 0x7f4e04066570 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:20.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:20.114+0000 7f4e1ed93700 1 --2- 172.21.15.27:0/250315608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4e180f6c90 0x7f4e18109200 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:20.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:20.114+0000 7f4e1ed93700 1 --2- 172.21.15.27:0/250315608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4e180f75a0 0x7f4e181096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:20.121 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:20.114+0000 7f4e1ed93700 1 -- 172.21.15.27:0/250315608 >> 172.21.15.27:0/250315608 conn(0x7f4e180f04c0 msgr2=0x7f4e180f28b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:02:20.121 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:20.114+0000 7f4e1ed93700 1 -- 172.21.15.27:0/250315608 shutdown_connections 2021-11-03T18:02:20.121 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:20.114+0000 7f4e1ed93700 1 -- 172.21.15.27:0/250315608 wait complete. 2021-11-03T18:02:20.121 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:02:20.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:20 smithi115 ceph-mon[65453]: pgmap v1430: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:20.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:20 smithi115 ceph-mon[65453]: from='client.35576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:02:20.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2782.426342 secs 2021-11-03T18:02:20.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:20 smithi115 ceph-mon[65453]: from='client.35580 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:02:20.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:20 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3297650967' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:20.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:20 smithi027 ceph-mon[97244]: pgmap v1430: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:20.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:20 smithi027 ceph-mon[97244]: from='client.35576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:02:20.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2782.426342 secs 2021-11-03T18:02:20.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:20 smithi027 ceph-mon[97244]: from='client.35580 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:02:20.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:20 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3297650967' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:21.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:21 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/250315608' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:02:21.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:21 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/250315608' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:02:22.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:22 smithi115 ceph-mon[65453]: pgmap v1431: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:22.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:22 smithi027 ceph-mon[97244]: pgmap v1431: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:24.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:24 smithi115 ceph-mon[65453]: pgmap v1432: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:24.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:24 smithi027 ceph-mon[97244]: pgmap v1432: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:25.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2787.426483 secs 2021-11-03T18:02:25.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2787.426483 secs 2021-11-03T18:02:26.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:26 smithi115 ceph-mon[65453]: pgmap v1433: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:26.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:26 smithi027 ceph-mon[97244]: pgmap v1433: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:28.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:28 smithi115 ceph-mon[65453]: pgmap v1434: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:28.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:02:28.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:02:28.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:02:28.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:28.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:28.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:28.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:28.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:28 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:02:28.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:28 smithi027 ceph-mon[97244]: pgmap v1434: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:28.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:02:28.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:02:28.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:02:28.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:28.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:28.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:28.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:28.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:28 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:02:30.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:30 smithi115 ceph-mon[65453]: pgmap v1435: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:30.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2792.426608 secs 2021-11-03T18:02:30.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:30 smithi027 ceph-mon[97244]: pgmap v1435: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:30.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2792.426608 secs 2021-11-03T18:02:32.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:32 smithi115 ceph-mon[65453]: pgmap v1436: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:32.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:32 smithi027 ceph-mon[97244]: pgmap v1436: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:34.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:34 smithi115 ceph-mon[65453]: pgmap v1437: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:34.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:34 smithi027 ceph-mon[97244]: pgmap v1437: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:35.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2797.426733 secs 2021-11-03T18:02:35.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2797.426733 secs 2021-11-03T18:02:36.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:36 smithi115 ceph-mon[65453]: pgmap v1438: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:36.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:36 smithi027 ceph-mon[97244]: pgmap v1438: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:38.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:38 smithi115 ceph-mon[65453]: pgmap v1439: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:38.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:02:38.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:02:38.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:02:38.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:38.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:38.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:38.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:38.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:38 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:02:38.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:38 smithi027 ceph-mon[97244]: pgmap v1439: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:38.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:02:38.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:02:38.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:02:38.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:38.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:38.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:38.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:38.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:38 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:02:40.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:40 smithi115 ceph-mon[65453]: pgmap v1440: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:40.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2802.426858 secs 2021-11-03T18:02:40.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:40 smithi027 ceph-mon[97244]: pgmap v1440: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:40.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2802.426858 secs 2021-11-03T18:02:42.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:42 smithi115 ceph-mon[65453]: pgmap v1441: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:42.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:42 smithi027 ceph-mon[97244]: pgmap v1441: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:44.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:44 smithi115 ceph-mon[65453]: pgmap v1442: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:44.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:44 smithi027 ceph-mon[97244]: pgmap v1442: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:45.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2807.426921 secs 2021-11-03T18:02:45.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2807.426921 secs 2021-11-03T18:02:46.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:46 smithi115 ceph-mon[65453]: pgmap v1443: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:46.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:46 smithi027 ceph-mon[97244]: pgmap v1443: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:48.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:48 smithi115 ceph-mon[65453]: pgmap v1444: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:48.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:02:48.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:48 smithi115 ceph-mon[65453]: poll took 60170.501 ms: 1 events 2021-11-03T18:02:48.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:02:48.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:02:48.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:48.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:48.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:48.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:48.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:48 smithi027 ceph-mon[97244]: pgmap v1444: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:48.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:02:48.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:48 smithi027 ceph-mon[97244]: poll took 60170.501 ms: 1 events 2021-11-03T18:02:48.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:02:48.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:02:48.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:48.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:48.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:48.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:49.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:49 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:02:49.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:49 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:02:50.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.228+0000 7f6cd9255700 1 -- 172.21.15.27:0/1229440483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cd406a3b0 msgr2=0x7f6cd406a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:02:50.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.228+0000 7f6cd9255700 1 --2- 172.21.15.27:0/1229440483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cd406a3b0 0x7f6cd406a830 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 rx=0x7f6cc8004660 tx=0x7f6cc8009b30).stop 2021-11-03T18:02:50.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.228+0000 7f6cd9255700 1 -- 172.21.15.27:0/1229440483 shutdown_connections 2021-11-03T18:02:50.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.228+0000 7f6cd9255700 1 --2- 172.21.15.27:0/1229440483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cd406a3b0 0x7f6cd406a830 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:50.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.228+0000 7f6cd9255700 1 --2- 172.21.15.27:0/1229440483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cd406b920 0x7f6cd4069ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:50.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.228+0000 7f6cd9255700 1 -- 172.21.15.27:0/1229440483 >> 172.21.15.27:0/1229440483 conn(0x7f6cd40f1f00 msgr2=0x7f6cd40f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:02:50.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.228+0000 7f6cd9255700 1 -- 172.21.15.27:0/1229440483 shutdown_connections 2021-11-03T18:02:50.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.228+0000 7f6cd9255700 1 -- 172.21.15.27:0/1229440483 wait complete. 2021-11-03T18:02:50.230 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.228+0000 7f6cd9255700 1 Processor -- start 2021-11-03T18:02:50.230 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.229+0000 7f6cd9255700 1 -- start start 2021-11-03T18:02:50.230 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.229+0000 7f6cd9255700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cd406a3b0 0x7f6cd410d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:02:50.230 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.229+0000 7f6cd9255700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cd406b920 0x7f6cd410da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:02:50.231 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.229+0000 7f6cd9255700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd410e030 con 0x7f6cd406a3b0 2021-11-03T18:02:50.231 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.229+0000 7f6cd9255700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cd410e170 con 0x7f6cd406b920 2021-11-03T18:02:50.231 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.229+0000 7f6cd2d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cd406a3b0 0x7f6cd410d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:02:50.231 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.229+0000 7f6cd2d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cd406a3b0 0x7f6cd410d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60856/0 (socket says 172.21.15.27:60856) 2021-11-03T18:02:50.232 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.229+0000 7f6cd2d9d700 1 -- 172.21.15.27:0/4268327797 learned_addr learned my addr 172.21.15.27:0/4268327797 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:02:50.232 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.229+0000 7f6cd259c700 1 --2- 172.21.15.27:0/4268327797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cd406b920 0x7f6cd410da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:02:50.232 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.229+0000 7f6cd2d9d700 1 -- 172.21.15.27:0/4268327797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cd406b920 msgr2=0x7f6cd410da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:02:50.232 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.229+0000 7f6cd2d9d700 1 --2- 172.21.15.27:0/4268327797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cd406b920 0x7f6cd410da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:50.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.229+0000 7f6cd2d9d700 1 -- 172.21.15.27:0/4268327797 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc8005040 con 0x7f6cd406a3b0 2021-11-03T18:02:50.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.229+0000 7f6cd2d9d700 1 --2- 172.21.15.27:0/4268327797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cd406a3b0 0x7f6cd410d590 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 rx=0x7f6cc400c530 tx=0x7f6cc4007e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:02:50.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.230+0000 7f6cbb7fe700 1 -- 172.21.15.27:0/4268327797 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cc400f650 con 0x7f6cd406a3b0 2021-11-03T18:02:50.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.230+0000 7f6cbb7fe700 1 -- 172.21.15.27:0/4268327797 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f6cc400fc60 con 0x7f6cd406a3b0 2021-11-03T18:02:50.234 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.230+0000 7f6cd9255700 1 -- 172.21.15.27:0/4268327797 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cd410eb10 con 0x7f6cd406a3b0 2021-11-03T18:02:50.234 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.230+0000 7f6cbb7fe700 1 -- 172.21.15.27:0/4268327797 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6cc4012cb0 con 0x7f6cd406a3b0 2021-11-03T18:02:50.234 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.230+0000 7f6cd9255700 1 -- 172.21.15.27:0/4268327797 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cd410f0d0 con 0x7f6cd406a3b0 2021-11-03T18:02:50.235 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.231+0000 7f6cd9255700 1 -- 172.21.15.27:0/4268327797 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cd4045ad0 con 0x7f6cd406a3b0 2021-11-03T18:02:50.236 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.231+0000 7f6cbb7fe700 1 -- 172.21.15.27:0/4268327797 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f6cc4019030 con 0x7f6cd406a3b0 2021-11-03T18:02:50.236 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.231+0000 7f6cbb7fe700 1 --2- 172.21.15.27:0/4268327797 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6cbc0640b0 0x7f6cbc066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:02:50.236 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.232+0000 7f6cbb7fe700 1 -- 172.21.15.27:0/4268327797 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f6cc40805e0 con 0x7f6cd406a3b0 2021-11-03T18:02:50.236 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.232+0000 7f6cd259c700 1 --2- 172.21.15.27:0/4268327797 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6cbc0640b0 0x7f6cbc066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:02:50.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.232+0000 7f6cd259c700 1 --2- 172.21.15.27:0/4268327797 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6cbc0640b0 0x7f6cbc066570 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f6cc8004630 tx=0x7f6cc8009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:02:50.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.236+0000 7f6cbb7fe700 1 -- 172.21.15.27:0/4268327797 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f6cc4052840 con 0x7f6cd406a3b0 2021-11-03T18:02:50.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.413+0000 7f6cd9255700 1 -- 172.21.15.27:0/4268327797 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6cd4105640 con 0x7f6cbc0640b0 2021-11-03T18:02:50.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.415+0000 7f6cbb7fe700 1 -- 172.21.15.27:0/4268327797 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f6cd4105640 con 0x7f6cbc0640b0 2021-11-03T18:02:50.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.417+0000 7f6cd9255700 1 -- 172.21.15.27:0/4268327797 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6cbc0640b0 msgr2=0x7f6cbc066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:02:50.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.417+0000 7f6cd9255700 1 --2- 172.21.15.27:0/4268327797 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6cbc0640b0 0x7f6cbc066570 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7f6cc8004630 tx=0x7f6cc8009b30).stop 2021-11-03T18:02:50.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.417+0000 7f6cd9255700 1 -- 172.21.15.27:0/4268327797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cd406a3b0 msgr2=0x7f6cd410d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:02:50.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.417+0000 7f6cd9255700 1 --2- 172.21.15.27:0/4268327797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cd406a3b0 0x7f6cd410d590 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 rx=0x7f6cc400c530 tx=0x7f6cc4007e10).stop 2021-11-03T18:02:50.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.417+0000 7f6cd9255700 1 -- 172.21.15.27:0/4268327797 shutdown_connections 2021-11-03T18:02:50.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.417+0000 7f6cd9255700 1 --2- 172.21.15.27:0/4268327797 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6cbc0640b0 0x7f6cbc066570 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:50.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.417+0000 7f6cd9255700 1 --2- 172.21.15.27:0/4268327797 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6cd406a3b0 0x7f6cd410d590 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:50.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.417+0000 7f6cd9255700 1 --2- 172.21.15.27:0/4268327797 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6cd406b920 0x7f6cd410da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:50.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.417+0000 7f6cd9255700 1 -- 172.21.15.27:0/4268327797 >> 172.21.15.27:0/4268327797 conn(0x7f6cd40f1f00 msgr2=0x7f6cd40f8960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:02:50.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.417+0000 7f6cd9255700 1 -- 172.21.15.27:0/4268327797 shutdown_connections 2021-11-03T18:02:50.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.417+0000 7f6cd9255700 1 -- 172.21.15.27:0/4268327797 wait complete. 2021-11-03T18:02:50.424 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:02:50.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:50 smithi115 ceph-mon[65453]: pgmap v1445: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:50.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2812.427065 secs 2021-11-03T18:02:50.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.517+0000 7f9f13376700 1 -- 172.21.15.27:0/2212266817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f0c0f6ff0 msgr2=0x7f9f0c0f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:02:50.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.517+0000 7f9f13376700 1 --2- 172.21.15.27:0/2212266817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f0c0f6ff0 0x7f9f0c0f7a20 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 rx=0x7f9f08004660 tx=0x7f9f08009b30).stop 2021-11-03T18:02:50.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.517+0000 7f9f13376700 1 -- 172.21.15.27:0/2212266817 shutdown_connections 2021-11-03T18:02:50.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.517+0000 7f9f13376700 1 --2- 172.21.15.27:0/2212266817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f0c0f6ff0 0x7f9f0c0f7a20 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:50.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.517+0000 7f9f13376700 1 --2- 172.21.15.27:0/2212266817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f0c0f66e0 0x7f9f0c0f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T18:02:50.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.517+0000 7f9f13376700 1 -- 172.21.15.27:0/2212266817 >> 172.21.15.27:0/2212266817 conn(0x7f9f0c0f1f00 msgr2=0x7f9f0c0f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:02:50.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.517+0000 7f9f13376700 1 -- 172.21.15.27:0/2212266817 shutdown_connections 2021-11-03T18:02:50.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.517+0000 7f9f13376700 1 -- 172.21.15.27:0/2212266817 wait complete. 2021-11-03T18:02:50.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.518+0000 7f9f13376700 1 Processor -- start 2021-11-03T18:02:50.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.518+0000 7f9f13376700 1 -- start start 2021-11-03T18:02:50.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.518+0000 7f9f13376700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f0c0f66e0 0x7f9f0c106fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:02:50.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.518+0000 7f9f13376700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f0c0f6ff0 0x7f9f0c107490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:02:50.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.518+0000 7f9f13376700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0c107ad0 con 0x7f9f0c0f6ff0 2021-11-03T18:02:50.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.518+0000 7f9f13376700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f0c108380 con 0x7f9f0c0f66e0 2021-11-03T18:02:50.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.518+0000 7f9f10911700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f0c0f6ff0 0x7f9f0c107490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:02:50.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.518+0000 7f9f10911700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f0c0f6ff0 0x7f9f0c107490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60868/0 (socket says 172.21.15.27:60868) 2021-11-03T18:02:50.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.518+0000 7f9f10911700 1 -- 172.21.15.27:0/3713014415 learned_addr learned my addr 172.21.15.27:0/3713014415 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:02:50.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.518+0000 7f9f11112700 1 --2- 172.21.15.27:0/3713014415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f0c0f66e0 0x7f9f0c106fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:02:50.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.518+0000 7f9f10911700 1 -- 172.21.15.27:0/3713014415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f0c0f66e0 msgr2=0x7f9f0c106fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:02:50.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.518+0000 7f9f10911700 1 --2- 172.21.15.27:0/3713014415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f0c0f66e0 0x7f9f0c106fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:50.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.518+0000 7f9f10911700 1 -- 172.21.15.27:0/3713014415 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f08005040 con 0x7f9f0c0f6ff0 2021-11-03T18:02:50.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.518+0000 7f9f10911700 1 --2- 172.21.15.27:0/3713014415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f0c0f6ff0 0x7f9f0c107490 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7f9f08009990 tx=0x7f9f08009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:02:50.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.518+0000 7f9efdffb700 1 -- 172.21.15.27:0/3713014415 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f08018070 con 0x7f9f0c0f6ff0 2021-11-03T18:02:50.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.519+0000 7f9efdffb700 1 -- 172.21.15.27:0/3713014415 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f9f08007ca0 con 0x7f9f0c0f6ff0 2021-11-03T18:02:50.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.519+0000 7f9f13376700 1 -- 172.21.15.27:0/3713014415 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f0c108600 con 0x7f9f0c0f6ff0 2021-11-03T18:02:50.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.519+0000 7f9efdffb700 1 -- 172.21.15.27:0/3713014415 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f0801c620 con 0x7f9f0c0f6ff0 2021-11-03T18:02:50.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.519+0000 7f9f13376700 1 -- 172.21.15.27:0/3713014415 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f0c108b60 con 0x7f9f0c0f6ff0 2021-11-03T18:02:50.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.520+0000 7f9f13376700 1 -- 172.21.15.27:0/3713014415 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f0c0f5b70 con 0x7f9f0c0f6ff0 2021-11-03T18:02:50.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.520+0000 7f9efdffb700 1 -- 172.21.15.27:0/3713014415 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f9f0801d960 con 0x7f9f0c0f6ff0 2021-11-03T18:02:50.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.521+0000 7f9efdffb700 1 --2- 172.21.15.27:0/3713014415 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9ef80640b0 0x7f9ef8066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:02:50.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.521+0000 7f9efdffb700 1 -- 172.21.15.27:0/3713014415 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f9f08082580 con 0x7f9f0c0f6ff0 2021-11-03T18:02:50.526 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.521+0000 7f9f11112700 1 --2- 172.21.15.27:0/3713014415 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9ef80640b0 0x7f9ef8066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:02:50.526 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.521+0000 7f9f11112700 1 --2- 172.21.15.27:0/3713014415 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9ef80640b0 0x7f9ef8066570 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f9f0000b790 tx=0x7f9f00008040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:02:50.526 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:50 smithi027 ceph-mon[97244]: pgmap v1445: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:50.527 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2812.427065 secs 2021-11-03T18:02:50.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.525+0000 7f9efdffb700 1 -- 172.21.15.27:0/3713014415 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f9f08054940 con 0x7f9f0c0f6ff0 2021-11-03T18:02:50.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.699+0000 7f9f13376700 1 -- 172.21.15.27:0/3713014415 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f0c0fef50 con 0x7f9ef80640b0 2021-11-03T18:02:50.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.714+0000 7f9efdffb700 1 -- 172.21.15.27:0/3713014415 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f9f0c0fef50 con 0x7f9ef80640b0 2021-11-03T18:02:50.712 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:02:50.712 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (54m) 3m ago 55m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:02:50.712 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (46m) 3m ago 55m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:02:50.713 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (45m) 2m ago 54m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:02:50.713 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (54m) 3m ago 55m 41.8M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:02:50.713 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (47m) 3m ago 50m 26.0M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:02:50.713 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (50m) 3m ago 50m 3845M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:02:50.714 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (46m) 2m ago 50m 25.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:02:50.714 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (48m) 2m ago 50m 26.8M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:02:50.714 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (48m) 3m ago 56m 554M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:02:50.714 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (47m) 2m ago 54m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:02:50.715 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (47m) 3m ago 57m 235M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:02:50.715 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (46m) 2m ago 54m 112M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:02:50.715 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (55m) 3m ago 55m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:02:50.715 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (54m) 2m ago 54m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:02:50.715 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (45m) 3m ago 54m 351M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:02:50.716 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (45m) 3m ago 53m 383M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:02:50.716 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (44m) 3m ago 53m 284M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:02:50.716 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (44m) 2m ago 53m 404M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:02:50.716 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (43m) 2m ago 52m 322M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:02:50.717 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (43m) 2m ago 52m 327M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:02:50.717 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (54m) 3m ago 55m 55.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:02:50.717 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.716+0000 7f9f13376700 1 -- 172.21.15.27:0/3713014415 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9ef80640b0 msgr2=0x7f9ef8066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:02:50.718 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.716+0000 7f9f13376700 1 --2- 172.21.15.27:0/3713014415 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9ef80640b0 0x7f9ef8066570 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f9f0000b790 tx=0x7f9f00008040).stop 2021-11-03T18:02:50.718 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.716+0000 7f9f13376700 1 -- 172.21.15.27:0/3713014415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f0c0f6ff0 msgr2=0x7f9f0c107490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:02:50.718 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.716+0000 7f9f13376700 1 --2- 172.21.15.27:0/3713014415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f0c0f6ff0 0x7f9f0c107490 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7f9f08009990 tx=0x7f9f08009b30).stop 2021-11-03T18:02:50.718 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.716+0000 7f9f13376700 1 -- 172.21.15.27:0/3713014415 shutdown_connections 2021-11-03T18:02:50.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.716+0000 7f9f13376700 1 --2- 172.21.15.27:0/3713014415 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9ef80640b0 0x7f9ef8066570 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:50.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.716+0000 7f9f13376700 1 --2- 172.21.15.27:0/3713014415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9f0c0f66e0 0x7f9f0c106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:50.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.716+0000 7f9f13376700 1 --2- 172.21.15.27:0/3713014415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9f0c0f6ff0 0x7f9f0c107490 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:50.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.716+0000 7f9f13376700 1 -- 172.21.15.27:0/3713014415 >> 172.21.15.27:0/3713014415 conn(0x7f9f0c0f1f00 msgr2=0x7f9f0c0fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:02:50.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.716+0000 7f9f13376700 1 -- 172.21.15.27:0/3713014415 shutdown_connections 2021-11-03T18:02:50.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.716+0000 7f9f13376700 1 -- 172.21.15.27:0/3713014415 wait complete. 2021-11-03T18:02:50.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.815+0000 7fd1f3d6e700 1 -- 172.21.15.27:0/3356133448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1ec0f6ff0 msgr2=0x7fd1ec0f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:02:50.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.815+0000 7fd1f3d6e700 1 --2- 172.21.15.27:0/3356133448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1ec0f6ff0 0x7fd1ec0f7a20 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 rx=0x7fd1e0004660 tx=0x7fd1e0009b30).stop 2021-11-03T18:02:50.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.815+0000 7fd1f3d6e700 1 -- 172.21.15.27:0/3356133448 shutdown_connections 2021-11-03T18:02:50.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.815+0000 7fd1f3d6e700 1 --2- 172.21.15.27:0/3356133448 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1ec0f6ff0 0x7fd1ec0f7a20 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:50.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.815+0000 7fd1f3d6e700 1 --2- 172.21.15.27:0/3356133448 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1ec0f66e0 0x7fd1ec0f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:50.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.815+0000 7fd1f3d6e700 1 -- 172.21.15.27:0/3356133448 >> 172.21.15.27:0/3356133448 conn(0x7fd1ec0f1f00 msgr2=0x7fd1ec0f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:02:50.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.815+0000 7fd1f3d6e700 1 -- 172.21.15.27:0/3356133448 shutdown_connections 2021-11-03T18:02:50.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.815+0000 7fd1f3d6e700 1 -- 172.21.15.27:0/3356133448 wait complete. 2021-11-03T18:02:50.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.815+0000 7fd1f3d6e700 1 Processor -- start 2021-11-03T18:02:50.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.815+0000 7fd1f3d6e700 1 -- start start 2021-11-03T18:02:50.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.816+0000 7fd1f3d6e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1ec0f66e0 0x7fd1ec06b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:02:50.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.816+0000 7fd1f3d6e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1ec0f6ff0 0x7fd1ec069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:02:50.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.816+0000 7fd1f3d6e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec06a380 con 0x7fd1ec0f6ff0 2021-11-03T18:02:50.823 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.816+0000 7fd1f3d6e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1ec06a4c0 con 0x7fd1ec0f66e0 2021-11-03T18:02:50.823 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.816+0000 7fd1f1309700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1ec0f6ff0 0x7fd1ec069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:02:50.823 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.816+0000 7fd1f1309700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1ec0f6ff0 0x7fd1ec069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60878/0 (socket says 172.21.15.27:60878) 2021-11-03T18:02:50.824 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.816+0000 7fd1f1309700 1 -- 172.21.15.27:0/3472692540 learned_addr learned my addr 172.21.15.27:0/3472692540 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:02:50.824 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.816+0000 7fd1f1b0a700 1 --2- 172.21.15.27:0/3472692540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1ec0f66e0 0x7fd1ec06b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:02:50.825 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.816+0000 7fd1f1309700 1 -- 172.21.15.27:0/3472692540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1ec0f66e0 msgr2=0x7fd1ec06b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:02:50.825 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.816+0000 7fd1f1309700 1 --2- 172.21.15.27:0/3472692540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1ec0f66e0 0x7fd1ec06b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:50.826 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.816+0000 7fd1f1309700 1 -- 172.21.15.27:0/3472692540 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e0005040 con 0x7fd1ec0f6ff0 2021-11-03T18:02:50.826 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.816+0000 7fd1f1309700 1 --2- 172.21.15.27:0/3472692540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1ec0f6ff0 0x7fd1ec069e20 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 rx=0x7fd1e0009990 tx=0x7fd1e0009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:02:50.826 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.816+0000 7fd1ea7fc700 1 -- 172.21.15.27:0/3472692540 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1e0018070 con 0x7fd1ec0f6ff0 2021-11-03T18:02:50.827 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.816+0000 7fd1f3d6e700 1 -- 172.21.15.27:0/3472692540 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1ec06a740 con 0x7fd1ec0f6ff0 2021-11-03T18:02:50.827 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.816+0000 7fd1ea7fc700 1 -- 172.21.15.27:0/3472692540 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fd1e0007ba0 con 0x7fd1ec0f6ff0 2021-11-03T18:02:50.828 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.816+0000 7fd1ea7fc700 1 -- 172.21.15.27:0/3472692540 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd1e001c3c0 con 0x7fd1ec0f6ff0 2021-11-03T18:02:50.828 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.816+0000 7fd1f3d6e700 1 -- 172.21.15.27:0/3472692540 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1ec06aca0 con 0x7fd1ec0f6ff0 2021-11-03T18:02:50.829 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.817+0000 7fd1f3d6e700 1 -- 172.21.15.27:0/3472692540 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1ec045ad0 con 0x7fd1ec0f6ff0 2021-11-03T18:02:50.829 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.818+0000 7fd1ea7fc700 1 -- 172.21.15.27:0/3472692540 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fd1e001d420 con 0x7fd1ec0f6ff0 2021-11-03T18:02:50.829 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.818+0000 7fd1ea7fc700 1 --2- 172.21.15.27:0/3472692540 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd1d40640b0 0x7fd1d4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:02:50.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.818+0000 7fd1ea7fc700 1 -- 172.21.15.27:0/3472692540 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fd1e0082470 con 0x7fd1ec0f6ff0 2021-11-03T18:02:50.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.818+0000 7fd1f1b0a700 1 --2- 172.21.15.27:0/3472692540 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd1d40640b0 0x7fd1d4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:02:50.831 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.819+0000 7fd1f1b0a700 1 --2- 172.21.15.27:0/3472692540 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd1d40640b0 0x7fd1d4066570 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7fd1ec06b730 tx=0x7fd1e4009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:02:50.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:50.822+0000 7fd1ea7fc700 1 -- 172.21.15.27:0/3472692540 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fd1e0054710 con 0x7fd1ec0f6ff0 2021-11-03T18:02:51.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.053+0000 7fd1f3d6e700 1 -- 172.21.15.27:0/3472692540 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd1ec0fef70 con 0x7fd1ec0f6ff0 2021-11-03T18:02:51.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.054+0000 7fd1ea7fc700 1 -- 172.21.15.27:0/3472692540 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fd1e0054300 con 0x7fd1ec0f6ff0 2021-11-03T18:02:51.052 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:02:51.052 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:02:51.052 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:02:51.052 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:02:51.053 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:02:51.053 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:02:51.053 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:02:51.053 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:02:51.054 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:02:51.054 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:02:51.054 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:02:51.054 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:02:51.054 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:02:51.055 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:02:51.055 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:02:51.055 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:02:51.055 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:02:51.055 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:02:51.056 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.055+0000 7fd1f3d6e700 1 -- 172.21.15.27:0/3472692540 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd1d40640b0 msgr2=0x7fd1d4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:02:51.056 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.055+0000 7fd1f3d6e700 1 --2- 172.21.15.27:0/3472692540 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd1d40640b0 0x7fd1d4066570 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7fd1ec06b730 tx=0x7fd1e4009040).stop 2021-11-03T18:02:51.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.055+0000 7fd1f3d6e700 1 -- 172.21.15.27:0/3472692540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1ec0f6ff0 msgr2=0x7fd1ec069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:02:51.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.055+0000 7fd1f3d6e700 1 --2- 172.21.15.27:0/3472692540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1ec0f6ff0 0x7fd1ec069e20 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 rx=0x7fd1e0009990 tx=0x7fd1e0009b30).stop 2021-11-03T18:02:51.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.055+0000 7fd1f3d6e700 1 -- 172.21.15.27:0/3472692540 shutdown_connections 2021-11-03T18:02:51.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.055+0000 7fd1f3d6e700 1 --2- 172.21.15.27:0/3472692540 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd1d40640b0 0x7fd1d4066570 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:51.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.055+0000 7fd1f3d6e700 1 --2- 172.21.15.27:0/3472692540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd1ec0f66e0 0x7fd1ec06b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:51.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.055+0000 7fd1f3d6e700 1 --2- 172.21.15.27:0/3472692540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd1ec0f6ff0 0x7fd1ec069e20 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:51.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.055+0000 7fd1f3d6e700 1 -- 172.21.15.27:0/3472692540 >> 172.21.15.27:0/3472692540 conn(0x7fd1ec0f1f00 msgr2=0x7fd1ec0fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:02:51.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.055+0000 7fd1f3d6e700 1 -- 172.21.15.27:0/3472692540 shutdown_connections 2021-11-03T18:02:51.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.056+0000 7fd1f3d6e700 1 -- 172.21.15.27:0/3472692540 wait complete. 2021-11-03T18:02:51.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.155+0000 7f2ce132f700 1 -- 172.21.15.27:0/2690805533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cdc0f55c0 msgr2=0x7f2cdc0f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:02:51.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.155+0000 7f2ce132f700 1 --2- 172.21.15.27:0/2690805533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cdc0f55c0 0x7f2cdc0f59e0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 rx=0x7f2cd0004660 tx=0x7f2cd0009b30).stop 2021-11-03T18:02:51.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.155+0000 7f2ce132f700 1 -- 172.21.15.27:0/2690805533 shutdown_connections 2021-11-03T18:02:51.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.155+0000 7f2ce132f700 1 --2- 172.21.15.27:0/2690805533 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cdc0f6860 0x7f2cdc0f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:51.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.155+0000 7f2ce132f700 1 --2- 172.21.15.27:0/2690805533 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cdc0f55c0 0x7f2cdc0f59e0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:51.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.155+0000 7f2ce132f700 1 -- 172.21.15.27:0/2690805533 >> 172.21.15.27:0/2690805533 conn(0x7f2cdc0f0d40 msgr2=0x7f2cdc0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:02:51.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.155+0000 7f2ce132f700 1 -- 172.21.15.27:0/2690805533 shutdown_connections 2021-11-03T18:02:51.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.155+0000 7f2ce132f700 1 -- 172.21.15.27:0/2690805533 wait complete. 2021-11-03T18:02:51.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.156+0000 7f2ce132f700 1 Processor -- start 2021-11-03T18:02:51.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.156+0000 7f2ce132f700 1 -- start start 2021-11-03T18:02:51.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.156+0000 7f2ce132f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cdc0f55c0 0x7f2cdc105ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:02:51.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.156+0000 7f2ce132f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cdc0f6860 0x7f2cdc1063b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:02:51.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.156+0000 7f2ce132f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc106960 con 0x7f2cdc0f55c0 2021-11-03T18:02:51.160 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.156+0000 7f2ce132f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cdc106aa0 con 0x7f2cdc0f6860 2021-11-03T18:02:51.160 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.156+0000 7f2cdaffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cdc0f55c0 0x7f2cdc105ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:02:51.161 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.156+0000 7f2cdaffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cdc0f55c0 0x7f2cdc105ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60886/0 (socket says 172.21.15.27:60886) 2021-11-03T18:02:51.161 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.156+0000 7f2cdaffd700 1 -- 172.21.15.27:0/3038656229 learned_addr learned my addr 172.21.15.27:0/3038656229 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:02:51.161 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.156+0000 7f2cda7fc700 1 --2- 172.21.15.27:0/3038656229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cdc0f6860 0x7f2cdc1063b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:02:51.161 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.157+0000 7f2cdaffd700 1 -- 172.21.15.27:0/3038656229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cdc0f6860 msgr2=0x7f2cdc1063b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:02:51.162 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.157+0000 7f2cdaffd700 1 --2- 172.21.15.27:0/3038656229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cdc0f6860 0x7f2cdc1063b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:51.162 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.157+0000 7f2cdaffd700 1 -- 172.21.15.27:0/3038656229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cd0005040 con 0x7f2cdc0f55c0 2021-11-03T18:02:51.162 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.157+0000 7f2cdaffd700 1 --2- 172.21.15.27:0/3038656229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cdc0f55c0 0x7f2cdc105ec0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7f2cd0000c00 tx=0x7f2cd0009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:02:51.162 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.159+0000 7f2cc37fe700 1 -- 172.21.15.27:0/3038656229 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cd001a070 con 0x7f2cdc0f55c0 2021-11-03T18:02:51.163 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.159+0000 7f2cc37fe700 1 -- 172.21.15.27:0/3038656229 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f2cd0012a80 con 0x7f2cdc0f55c0 2021-11-03T18:02:51.163 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.159+0000 7f2ce132f700 1 -- 172.21.15.27:0/3038656229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cdc1073e0 con 0x7f2cdc0f55c0 2021-11-03T18:02:51.163 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.159+0000 7f2cc37fe700 1 -- 172.21.15.27:0/3038656229 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2cd001f8c0 con 0x7f2cdc0f55c0 2021-11-03T18:02:51.163 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.159+0000 7f2ce132f700 1 -- 172.21.15.27:0/3038656229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cdc107940 con 0x7f2cdc0f55c0 2021-11-03T18:02:51.164 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.160+0000 7f2ce132f700 1 -- 172.21.15.27:0/3038656229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cdc045ad0 con 0x7f2cdc0f55c0 2021-11-03T18:02:51.164 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.160+0000 7f2cc37fe700 1 -- 172.21.15.27:0/3038656229 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f2cd0012bf0 con 0x7f2cdc0f55c0 2021-11-03T18:02:51.164 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.161+0000 7f2cc37fe700 1 --2- 172.21.15.27:0/3038656229 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2cc4064090 0x7f2cc4066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:02:51.164 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.161+0000 7f2cc37fe700 1 -- 172.21.15.27:0/3038656229 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f2cd0084a80 con 0x7f2cdc0f55c0 2021-11-03T18:02:51.164 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.161+0000 7f2cda7fc700 1 --2- 172.21.15.27:0/3038656229 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2cc4064090 0x7f2cc4066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:02:51.165 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.161+0000 7f2cda7fc700 1 --2- 172.21.15.27:0/3038656229 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2cc4064090 0x7f2cc4066550 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f2ccc0041d0 tx=0x7f2ccc009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:02:51.166 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.165+0000 7f2cc37fe700 1 -- 172.21.15.27:0/3038656229 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f2cd0056d20 con 0x7f2cdc0f55c0 2021-11-03T18:02:51.367 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.369+0000 7f2ce132f700 1 -- 172.21.15.27:0/3038656229 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2cdc0ff790 con 0x7f2cdc0f55c0 2021-11-03T18:02:51.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.370+0000 7f2cc37fe700 1 -- 172.21.15.27:0/3038656229 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f2cd0056910 con 0x7f2cdc0f55c0 2021-11-03T18:02:51.368 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:02:51.368 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:02:51.369 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:02:51.369 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:02:51.369 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:02:51.369 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:02:51.369 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:02:51.370 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:02:51.370 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:02:51.370 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:02:51.370 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:02:51.371 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:02:51.371 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:02:51.371 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:02:51.371 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:02:51.372 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:02:51.372 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:02:51.372 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:02:51.372 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:02:51.372 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:02:51.373 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:02:51.373 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:02:51.373 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:02:51.373 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:02:51.374 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:02:51.374 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:02:51.374 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:02:51.374 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:02:51.374 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:02:51.375 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:02:51.375 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:02:51.375 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:02:51.375 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:02:51.376 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:02:51.376 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:02:51.376 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:02:51.376 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:02:51.376 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:02:51.377 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:02:51.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.372+0000 7f2ce132f700 1 -- 172.21.15.27:0/3038656229 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2cc4064090 msgr2=0x7f2cc4066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:02:51.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.372+0000 7f2ce132f700 1 --2- 172.21.15.27:0/3038656229 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2cc4064090 0x7f2cc4066550 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f2ccc0041d0 tx=0x7f2ccc009040).stop 2021-11-03T18:02:51.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.372+0000 7f2ce132f700 1 -- 172.21.15.27:0/3038656229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cdc0f55c0 msgr2=0x7f2cdc105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:02:51.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.372+0000 7f2ce132f700 1 --2- 172.21.15.27:0/3038656229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cdc0f55c0 0x7f2cdc105ec0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7f2cd0000c00 tx=0x7f2cd0009b30).stop 2021-11-03T18:02:51.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.372+0000 7f2ce132f700 1 -- 172.21.15.27:0/3038656229 shutdown_connections 2021-11-03T18:02:51.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.372+0000 7f2ce132f700 1 --2- 172.21.15.27:0/3038656229 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2cc4064090 0x7f2cc4066550 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:51.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.372+0000 7f2ce132f700 1 --2- 172.21.15.27:0/3038656229 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2cdc0f55c0 0x7f2cdc105ec0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:51.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.372+0000 7f2ce132f700 1 --2- 172.21.15.27:0/3038656229 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2cdc0f6860 0x7f2cdc1063b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:02:51.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.372+0000 7f2ce132f700 1 -- 172.21.15.27:0/3038656229 >> 172.21.15.27:0/3038656229 conn(0x7f2cdc0f0d40 msgr2=0x7f2cdc0fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:02:51.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.372+0000 7f2ce132f700 1 -- 172.21.15.27:0/3038656229 shutdown_connections 2021-11-03T18:02:51.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:02:51.372+0000 7f2ce132f700 1 -- 172.21.15.27:0/3038656229 wait complete. 2021-11-03T18:02:51.380 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:02:51.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:51 smithi115 ceph-mon[65453]: from='client.35592 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:02:51.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:51 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3472692540' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:51.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:51 smithi027 ceph-mon[97244]: from='client.35592 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:02:51.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:51 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3472692540' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:52.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:52 smithi115 ceph-mon[65453]: pgmap v1446: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:52.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:52 smithi115 ceph-mon[65453]: from='client.35596 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:02:52.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:52 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3038656229' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:02:52.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:52 smithi027 ceph-mon[97244]: pgmap v1446: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:52.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:52 smithi027 ceph-mon[97244]: from='client.35596 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:02:52.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:52 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3038656229' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:02:54.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:54 smithi115 ceph-mon[65453]: pgmap v1447: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:54.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:54 smithi027 ceph-mon[97244]: pgmap v1447: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:55.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2817.427203 secs 2021-11-03T18:02:55.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2817.427203 secs 2021-11-03T18:02:56.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:56 smithi115 ceph-mon[65453]: pgmap v1448: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:56.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:02:56.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:02:56.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:56 smithi027 ceph-mon[97244]: pgmap v1448: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:56.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:02:56.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:02:58.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:58 smithi115 ceph-mon[65453]: pgmap v1449: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:58.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:02:58.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:02:58.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:02:58.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:58.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:58.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:58.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:58.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:58 smithi027 ceph-mon[97244]: pgmap v1449: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:02:58.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:02:58.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:02:58.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:02:58.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:58.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:58.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:58.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:02:59.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:02:59 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:02:59.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:02:59 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:03:00.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:00 smithi115 ceph-mon[65453]: pgmap v1450: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:00.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2822.427321 secs 2021-11-03T18:03:00.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:00 smithi027 ceph-mon[97244]: pgmap v1450: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:00.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2822.427321 secs 2021-11-03T18:03:02.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:02 smithi115 ceph-mon[65453]: pgmap v1451: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:02.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:02 smithi027 ceph-mon[97244]: pgmap v1451: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:04.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:04 smithi115 ceph-mon[65453]: pgmap v1452: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:04.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:04 smithi027 ceph-mon[97244]: pgmap v1452: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:05.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2827.427432 secs 2021-11-03T18:03:05.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2827.427432 secs 2021-11-03T18:03:06.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:06 smithi115 ceph-mon[65453]: pgmap v1453: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:06.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:06 smithi027 ceph-mon[97244]: pgmap v1453: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:08.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:08 smithi115 ceph-mon[65453]: pgmap v1454: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:08.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:03:08.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:03:08.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:03:08.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:08.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:08.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:08.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:08.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:08 smithi027 ceph-mon[97244]: pgmap v1454: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:08.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:03:08.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:03:08.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:03:08.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:08.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:08.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:08.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:09.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:09 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:03:09.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:09 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:03:10.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:10 smithi115 ceph-mon[65453]: pgmap v1455: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:10.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2832.427573 secs 2021-11-03T18:03:10.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:10 smithi027 ceph-mon[97244]: pgmap v1455: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:10.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2832.427573 secs 2021-11-03T18:03:12.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:12 smithi115 ceph-mon[65453]: pgmap v1456: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:12.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:12 smithi027 ceph-mon[97244]: pgmap v1456: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:14.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:14 smithi115 ceph-mon[65453]: pgmap v1457: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:14.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:14 smithi027 ceph-mon[97244]: pgmap v1457: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:15.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2837.427638 secs 2021-11-03T18:03:15.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2837.427638 secs 2021-11-03T18:03:16.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:16 smithi115 ceph-mon[65453]: pgmap v1458: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:16.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:16 smithi027 ceph-mon[97244]: pgmap v1458: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:18.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:18 smithi115 ceph-mon[65453]: pgmap v1459: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:18.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:03:18.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:03:18.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:03:18.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:18.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:18.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:18.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:18.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:18 smithi027 ceph-mon[97244]: pgmap v1459: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:18.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:03:18.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:03:18.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:03:18.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:18.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:18.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:18.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:19.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:19 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:03:19.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:19 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:03:20.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:20 smithi115 ceph-mon[65453]: pgmap v1460: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:20.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2842.427780 secs 2021-11-03T18:03:20.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:20 smithi027 ceph-mon[97244]: pgmap v1460: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:20.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2842.427780 secs 2021-11-03T18:03:21.486 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.487+0000 7faeeb4c1700 1 -- 172.21.15.27:0/2071390267 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faee40f6ff0 msgr2=0x7faee40f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:03:21.486 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.487+0000 7faeeb4c1700 1 --2- 172.21.15.27:0/2071390267 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faee40f6ff0 0x7faee40f7a20 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 rx=0x7faed8004660 tx=0x7faed8009b30).stop 2021-11-03T18:03:21.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.487+0000 7faeeb4c1700 1 -- 172.21.15.27:0/2071390267 shutdown_connections 2021-11-03T18:03:21.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.487+0000 7faeeb4c1700 1 --2- 172.21.15.27:0/2071390267 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faee40f6ff0 0x7faee40f7a20 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:21.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.487+0000 7faeeb4c1700 1 --2- 172.21.15.27:0/2071390267 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faee40f66e0 0x7faee40f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:21.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.487+0000 7faeeb4c1700 1 -- 172.21.15.27:0/2071390267 >> 172.21.15.27:0/2071390267 conn(0x7faee40f1f00 msgr2=0x7faee40f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:03:21.488 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.487+0000 7faeeb4c1700 1 -- 172.21.15.27:0/2071390267 shutdown_connections 2021-11-03T18:03:21.488 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.487+0000 7faeeb4c1700 1 -- 172.21.15.27:0/2071390267 wait complete. 2021-11-03T18:03:21.488 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.488+0000 7faeeb4c1700 1 Processor -- start 2021-11-03T18:03:21.488 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.488+0000 7faeeb4c1700 1 -- start start 2021-11-03T18:03:21.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.488+0000 7faeeb4c1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faee40f66e0 0x7faee4106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:03:21.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.488+0000 7faeeb4c1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faee40f6ff0 0x7faee4107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:03:21.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.488+0000 7faeeb4c1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee4107a10 con 0x7faee40f6ff0 2021-11-03T18:03:21.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.488+0000 7faeeb4c1700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faee4107b50 con 0x7faee40f66e0 2021-11-03T18:03:21.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.488+0000 7faee8a5c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faee40f6ff0 0x7faee4107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:03:21.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.488+0000 7faee8a5c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faee40f6ff0 0x7faee4107460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60908/0 (socket says 172.21.15.27:60908) 2021-11-03T18:03:21.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.488+0000 7faee8a5c700 1 -- 172.21.15.27:0/3911372905 learned_addr learned my addr 172.21.15.27:0/3911372905 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:03:21.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.488+0000 7faee8a5c700 1 -- 172.21.15.27:0/3911372905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faee40f66e0 msgr2=0x7faee4106f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:03:21.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.488+0000 7faee8a5c700 1 --2- 172.21.15.27:0/3911372905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faee40f66e0 0x7faee4106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:03:21.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.488+0000 7faee8a5c700 1 -- 172.21.15.27:0/3911372905 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faed8005040 con 0x7faee40f6ff0 2021-11-03T18:03:21.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.489+0000 7faee8a5c700 1 --2- 172.21.15.27:0/3911372905 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faee40f6ff0 0x7faee4107460 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 rx=0x7faed8009990 tx=0x7faed8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:03:21.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.489+0000 7faee1ffb700 1 -- 172.21.15.27:0/3911372905 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faed8018070 con 0x7faee40f6ff0 2021-11-03T18:03:21.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.489+0000 7faee1ffb700 1 -- 172.21.15.27:0/3911372905 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7faed8007ca0 con 0x7faee40f6ff0 2021-11-03T18:03:21.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.489+0000 7faeeb4c1700 1 -- 172.21.15.27:0/3911372905 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faee4108490 con 0x7faee40f6ff0 2021-11-03T18:03:21.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.489+0000 7faee1ffb700 1 -- 172.21.15.27:0/3911372905 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faed801c620 con 0x7faee40f6ff0 2021-11-03T18:03:21.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.489+0000 7faeeb4c1700 1 -- 172.21.15.27:0/3911372905 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faee41089f0 con 0x7faee40f6ff0 2021-11-03T18:03:21.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.490+0000 7faeeb4c1700 1 -- 172.21.15.27:0/3911372905 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faee4045ad0 con 0x7faee40f6ff0 2021-11-03T18:03:21.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.491+0000 7faee1ffb700 1 -- 172.21.15.27:0/3911372905 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7faed801d960 con 0x7faee40f6ff0 2021-11-03T18:03:21.494 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.491+0000 7faee1ffb700 1 --2- 172.21.15.27:0/3911372905 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7faecc0640b0 0x7faecc066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:03:21.494 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.491+0000 7faee1ffb700 1 -- 172.21.15.27:0/3911372905 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7faed80824e0 con 0x7faee40f6ff0 2021-11-03T18:03:21.494 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.491+0000 7faee925d700 1 --2- 172.21.15.27:0/3911372905 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7faecc0640b0 0x7faecc066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:03:21.494 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.492+0000 7faee925d700 1 --2- 172.21.15.27:0/3911372905 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7faecc0640b0 0x7faecc066570 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7faedc0079c0 tx=0x7faedc004940).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:03:21.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.495+0000 7faee1ffb700 1 -- 172.21.15.27:0/3911372905 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7faed80548a0 con 0x7faee40f6ff0 2021-11-03T18:03:21.670 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.672+0000 7faeeb4c1700 1 -- 172.21.15.27:0/3911372905 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faee40ff050 con 0x7faecc0640b0 2021-11-03T18:03:21.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.674+0000 7faee1ffb700 1 -- 172.21.15.27:0/3911372905 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7faee40ff050 con 0x7faecc0640b0 2021-11-03T18:03:21.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.676+0000 7faeeb4c1700 1 -- 172.21.15.27:0/3911372905 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7faecc0640b0 msgr2=0x7faecc066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:03:21.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.676+0000 7faeeb4c1700 1 --2- 172.21.15.27:0/3911372905 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7faecc0640b0 0x7faecc066570 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7faedc0079c0 tx=0x7faedc004940).stop 2021-11-03T18:03:21.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.676+0000 7faeeb4c1700 1 -- 172.21.15.27:0/3911372905 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faee40f6ff0 msgr2=0x7faee4107460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:03:21.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.676+0000 7faeeb4c1700 1 --2- 172.21.15.27:0/3911372905 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faee40f6ff0 0x7faee4107460 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 rx=0x7faed8009990 tx=0x7faed8009b30).stop 2021-11-03T18:03:21.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.676+0000 7faeeb4c1700 1 -- 172.21.15.27:0/3911372905 shutdown_connections 2021-11-03T18:03:21.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.676+0000 7faeeb4c1700 1 --2- 172.21.15.27:0/3911372905 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7faecc0640b0 0x7faecc066570 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:21.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.676+0000 7faeeb4c1700 1 --2- 172.21.15.27:0/3911372905 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faee40f66e0 0x7faee4106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:03:21.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.676+0000 7faeeb4c1700 1 --2- 172.21.15.27:0/3911372905 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faee40f6ff0 0x7faee4107460 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:21.676 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.676+0000 7faeeb4c1700 1 -- 172.21.15.27:0/3911372905 >> 172.21.15.27:0/3911372905 conn(0x7faee40f1f00 msgr2=0x7faee40fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:03:21.676 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.676+0000 7faeeb4c1700 1 -- 172.21.15.27:0/3911372905 shutdown_connections 2021-11-03T18:03:21.676 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.676+0000 7faeeb4c1700 1 -- 172.21.15.27:0/3911372905 wait complete. 2021-11-03T18:03:21.683 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:03:21.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.776+0000 7f56cb386700 1 -- 172.21.15.27:0/2678388550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56c40f66e0 msgr2=0x7f56c40f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:03:21.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.776+0000 7f56cb386700 1 --2- 172.21.15.27:0/2678388550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56c40f66e0 0x7f56c40f6b00 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7f56b4004660 tx=0x7f56b4009b30).stop 2021-11-03T18:03:21.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.776+0000 7f56cb386700 1 -- 172.21.15.27:0/2678388550 shutdown_connections 2021-11-03T18:03:21.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.776+0000 7f56cb386700 1 --2- 172.21.15.27:0/2678388550 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56c40f7980 0x7f56c40f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:21.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.776+0000 7f56cb386700 1 --2- 172.21.15.27:0/2678388550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56c40f66e0 0x7f56c40f6b00 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:21.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.776+0000 7f56cb386700 1 -- 172.21.15.27:0/2678388550 >> 172.21.15.27:0/2678388550 conn(0x7f56c40f1e40 msgr2=0x7f56c40f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:03:21.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.776+0000 7f56cb386700 1 -- 172.21.15.27:0/2678388550 shutdown_connections 2021-11-03T18:03:21.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.776+0000 7f56cb386700 1 -- 172.21.15.27:0/2678388550 wait complete. 2021-11-03T18:03:21.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.776+0000 7f56cb386700 1 Processor -- start 2021-11-03T18:03:21.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.776+0000 7f56cb386700 1 -- start start 2021-11-03T18:03:21.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.777+0000 7f56cb386700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56c40f66e0 0x7f56c4106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:03:21.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.777+0000 7f56cb386700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56c40f7980 0x7f56c4107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:03:21.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.777+0000 7f56cb386700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c4107a10 con 0x7f56c40f66e0 2021-11-03T18:03:21.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.777+0000 7f56cb386700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c4107b50 con 0x7f56c40f7980 2021-11-03T18:03:21.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.777+0000 7f56c9122700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56c40f66e0 0x7f56c4106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:03:21.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.777+0000 7f56c9122700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56c40f66e0 0x7f56c4106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60916/0 (socket says 172.21.15.27:60916) 2021-11-03T18:03:21.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.777+0000 7f56c9122700 1 -- 172.21.15.27:0/1664358287 learned_addr learned my addr 172.21.15.27:0/1664358287 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:03:21.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.777+0000 7f56c8921700 1 --2- 172.21.15.27:0/1664358287 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56c40f7980 0x7f56c4107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:03:21.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.777+0000 7f56c9122700 1 -- 172.21.15.27:0/1664358287 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56c40f7980 msgr2=0x7f56c4107460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:03:21.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.777+0000 7f56c9122700 1 --2- 172.21.15.27:0/1664358287 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56c40f7980 0x7f56c4107460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:21.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.777+0000 7f56c9122700 1 -- 172.21.15.27:0/1664358287 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b4005040 con 0x7f56c40f66e0 2021-11-03T18:03:21.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.777+0000 7f56c8921700 1 --2- 172.21.15.27:0/1664358287 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56c40f7980 0x7f56c4107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:03:21.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.777+0000 7f56c9122700 1 --2- 172.21.15.27:0/1664358287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56c40f66e0 0x7f56c4106f70 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 rx=0x7f56b4013a70 tx=0x7f56b4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:03:21.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.777+0000 7f56b9ffb700 1 -- 172.21.15.27:0/1664358287 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56b4018070 con 0x7f56c40f66e0 2021-11-03T18:03:21.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.777+0000 7f56b9ffb700 1 -- 172.21.15.27:0/1664358287 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f56b40097d0 con 0x7f56c40f66e0 2021-11-03T18:03:21.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.777+0000 7f56cb386700 1 -- 172.21.15.27:0/1664358287 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56c4108490 con 0x7f56c40f66e0 2021-11-03T18:03:21.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.777+0000 7f56b9ffb700 1 -- 172.21.15.27:0/1664358287 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f56b401c3c0 con 0x7f56c40f66e0 2021-11-03T18:03:21.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.778+0000 7f56cb386700 1 -- 172.21.15.27:0/1664358287 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c41089f0 con 0x7f56c40f66e0 2021-11-03T18:03:21.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.779+0000 7f56cb386700 1 -- 172.21.15.27:0/1664358287 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56c4045ad0 con 0x7f56c40f66e0 2021-11-03T18:03:21.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.779+0000 7f56b9ffb700 1 -- 172.21.15.27:0/1664358287 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f56b4007ba0 con 0x7f56c40f66e0 2021-11-03T18:03:21.787 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.779+0000 7f56b9ffb700 1 --2- 172.21.15.27:0/1664358287 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f56b00640b0 0x7f56b0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:03:21.787 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.779+0000 7f56b9ffb700 1 -- 172.21.15.27:0/1664358287 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f56b4082400 con 0x7f56c40f66e0 2021-11-03T18:03:21.787 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.780+0000 7f56c8921700 1 --2- 172.21.15.27:0/1664358287 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f56b00640b0 0x7f56b0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:03:21.787 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.780+0000 7f56c8921700 1 --2- 172.21.15.27:0/1664358287 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f56b00640b0 0x7f56b0066570 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f56c0009740 tx=0x7f56c0009be0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:03:21.788 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.783+0000 7f56b9ffb700 1 -- 172.21.15.27:0/1664358287 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f56b40547c0 con 0x7f56c40f66e0 2021-11-03T18:03:21.952 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.955+0000 7f56cb386700 1 -- 172.21.15.27:0/1664358287 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f56c40fefb0 con 0x7f56b00640b0 2021-11-03T18:03:21.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.969+0000 7f56b9ffb700 1 -- 172.21.15.27:0/1664358287 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f56c40fefb0 con 0x7f56b00640b0 2021-11-03T18:03:21.967 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:03:21.967 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (54m) 4m ago 56m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:03:21.968 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (46m) 4m ago 56m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:03:21.968 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (46m) 3m ago 55m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:03:21.968 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (54m) 4m ago 55m 41.8M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:03:21.968 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (48m) 4m ago 51m 26.0M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:03:21.969 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (51m) 4m ago 51m 3845M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:03:21.969 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (47m) 3m ago 51m 25.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:03:21.969 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (48m) 3m ago 51m 26.8M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:03:21.969 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (48m) 4m ago 57m 554M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:03:21.970 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (47m) 3m ago 55m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:03:21.970 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (47m) 4m ago 57m 235M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:03:21.970 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (47m) 3m ago 55m 112M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:03:21.970 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (55m) 4m ago 55m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:03:21.971 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (55m) 3m ago 55m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:03:21.971 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (46m) 4m ago 54m 351M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:03:21.971 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (45m) 4m ago 54m 383M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:03:21.971 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (45m) 4m ago 53m 284M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:03:21.971 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (44m) 3m ago 53m 404M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:03:21.972 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (44m) 3m ago 53m 322M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:03:21.972 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (43m) 3m ago 52m 327M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:03:21.972 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (54m) 4m ago 55m 55.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:03:21.973 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.971+0000 7f56cb386700 1 -- 172.21.15.27:0/1664358287 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f56b00640b0 msgr2=0x7f56b0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:03:21.973 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.971+0000 7f56cb386700 1 --2- 172.21.15.27:0/1664358287 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f56b00640b0 0x7f56b0066570 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7f56c0009740 tx=0x7f56c0009be0).stop 2021-11-03T18:03:21.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.971+0000 7f56cb386700 1 -- 172.21.15.27:0/1664358287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56c40f66e0 msgr2=0x7f56c4106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:03:21.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.971+0000 7f56cb386700 1 --2- 172.21.15.27:0/1664358287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56c40f66e0 0x7f56c4106f70 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 rx=0x7f56b4013a70 tx=0x7f56b4009b30).stop 2021-11-03T18:03:21.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.971+0000 7f56cb386700 1 -- 172.21.15.27:0/1664358287 shutdown_connections 2021-11-03T18:03:21.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.971+0000 7f56cb386700 1 --2- 172.21.15.27:0/1664358287 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f56b00640b0 0x7f56b0066570 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:21.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.971+0000 7f56cb386700 1 --2- 172.21.15.27:0/1664358287 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f56c40f66e0 0x7f56c4106f70 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:21.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.971+0000 7f56cb386700 1 --2- 172.21.15.27:0/1664358287 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f56c40f7980 0x7f56c4107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:21.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.971+0000 7f56cb386700 1 -- 172.21.15.27:0/1664358287 >> 172.21.15.27:0/1664358287 conn(0x7f56c40f1e40 msgr2=0x7f56c40fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:03:21.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.971+0000 7f56cb386700 1 -- 172.21.15.27:0/1664358287 shutdown_connections 2021-11-03T18:03:21.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:21.971+0000 7f56cb386700 1 -- 172.21.15.27:0/1664358287 wait complete. 2021-11-03T18:03:22.069 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.071+0000 7f98d207e700 1 -- 172.21.15.27:0/1568745584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98cc0f66e0 msgr2=0x7f98cc0f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:03:22.069 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.071+0000 7f98d207e700 1 --2- 172.21.15.27:0/1568745584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98cc0f66e0 0x7f98cc0f6b00 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7f98c40099e0 tx=0x7f98c4004d10).stop 2021-11-03T18:03:22.069 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.071+0000 7f98d207e700 1 -- 172.21.15.27:0/1568745584 shutdown_connections 2021-11-03T18:03:22.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.071+0000 7f98d207e700 1 --2- 172.21.15.27:0/1568745584 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98cc0f7980 0x7f98cc0f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:22.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.071+0000 7f98d207e700 1 --2- 172.21.15.27:0/1568745584 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98cc0f66e0 0x7f98cc0f6b00 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:22.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.071+0000 7f98d207e700 1 -- 172.21.15.27:0/1568745584 >> 172.21.15.27:0/1568745584 conn(0x7f98cc0f1e40 msgr2=0x7f98cc0f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:03:22.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.071+0000 7f98d207e700 1 -- 172.21.15.27:0/1568745584 shutdown_connections 2021-11-03T18:03:22.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.071+0000 7f98d207e700 1 -- 172.21.15.27:0/1568745584 wait complete. 2021-11-03T18:03:22.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.071+0000 7f98d207e700 1 Processor -- start 2021-11-03T18:03:22.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.072+0000 7f98d207e700 1 -- start start 2021-11-03T18:03:22.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.072+0000 7f98d207e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98cc0f66e0 0x7f98cc06b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:03:22.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.072+0000 7f98d207e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98cc0f7980 0x7f98cc069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:03:22.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.072+0000 7f98d207e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc06a380 con 0x7f98cc0f66e0 2021-11-03T18:03:22.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.072+0000 7f98d207e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98cc06a4c0 con 0x7f98cc0f7980 2021-11-03T18:03:22.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.072+0000 7f98cb7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98cc0f66e0 0x7f98cc06b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:03:22.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.072+0000 7f98cb7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98cc0f66e0 0x7f98cc06b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60926/0 (socket says 172.21.15.27:60926) 2021-11-03T18:03:22.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.072+0000 7f98cb7fe700 1 -- 172.21.15.27:0/4046508468 learned_addr learned my addr 172.21.15.27:0/4046508468 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:03:22.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.072+0000 7f98caffd700 1 --2- 172.21.15.27:0/4046508468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98cc0f7980 0x7f98cc069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:03:22.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.072+0000 7f98cb7fe700 1 -- 172.21.15.27:0/4046508468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98cc0f7980 msgr2=0x7f98cc069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:03:22.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.072+0000 7f98cb7fe700 1 --2- 172.21.15.27:0/4046508468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98cc0f7980 0x7f98cc069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:22.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.072+0000 7f98cb7fe700 1 -- 172.21.15.27:0/4046508468 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c4009640 con 0x7f98cc0f66e0 2021-11-03T18:03:22.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.072+0000 7f98cb7fe700 1 --2- 172.21.15.27:0/4046508468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98cc0f66e0 0x7f98cc06b9a0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 rx=0x7f98c400bb10 tx=0x7f98c4004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:03:22.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.072+0000 7f98c3fff700 1 -- 172.21.15.27:0/4046508468 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98c400e070 con 0x7f98cc0f66e0 2021-11-03T18:03:22.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.072+0000 7f98c3fff700 1 -- 172.21.15.27:0/4046508468 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f98c4007d40 con 0x7f98cc0f66e0 2021-11-03T18:03:22.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.072+0000 7f98d207e700 1 -- 172.21.15.27:0/4046508468 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98cc06a740 con 0x7f98cc0f66e0 2021-11-03T18:03:22.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.073+0000 7f98c3fff700 1 -- 172.21.15.27:0/4046508468 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f98c4018070 con 0x7f98cc0f66e0 2021-11-03T18:03:22.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.073+0000 7f98d207e700 1 -- 172.21.15.27:0/4046508468 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98cc06ac20 con 0x7f98cc0f66e0 2021-11-03T18:03:22.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.073+0000 7f98d207e700 1 -- 172.21.15.27:0/4046508468 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98cc045ad0 con 0x7f98cc0f66e0 2021-11-03T18:03:22.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.074+0000 7f98c3fff700 1 -- 172.21.15.27:0/4046508468 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f98c4024060 con 0x7f98cc0f66e0 2021-11-03T18:03:22.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.074+0000 7f98c3fff700 1 --2- 172.21.15.27:0/4046508468 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f98b8064090 0x7f98b8066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:03:22.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.074+0000 7f98c3fff700 1 -- 172.21.15.27:0/4046508468 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f98c4082020 con 0x7f98cc0f66e0 2021-11-03T18:03:22.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.074+0000 7f98caffd700 1 --2- 172.21.15.27:0/4046508468 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f98b8064090 0x7f98b8066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:03:22.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.075+0000 7f98caffd700 1 --2- 172.21.15.27:0/4046508468 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f98b8064090 0x7f98b8066550 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f98b4004e20 tx=0x7f98b400b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:03:22.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.078+0000 7f98c3fff700 1 -- 172.21.15.27:0/4046508468 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f98c4054320 con 0x7f98cc0f66e0 2021-11-03T18:03:22.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.308+0000 7f98d207e700 1 -- 172.21.15.27:0/4046508468 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f98cc0fefb0 con 0x7f98cc0f66e0 2021-11-03T18:03:22.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.308+0000 7f98c3fff700 1 -- 172.21.15.27:0/4046508468 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f98c40222e0 con 0x7f98cc0f66e0 2021-11-03T18:03:22.307 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:03:22.307 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:03:22.307 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:03:22.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:03:22.308 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:03:22.308 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:03:22.308 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:03:22.308 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:03:22.309 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:03:22.309 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:03:22.309 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:03:22.309 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:03:22.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:03:22.310 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:03:22.310 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:03:22.310 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:03:22.311 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:03:22.311 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:03:22.312 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.310+0000 7f98d207e700 1 -- 172.21.15.27:0/4046508468 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f98b8064090 msgr2=0x7f98b8066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:03:22.312 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.310+0000 7f98d207e700 1 --2- 172.21.15.27:0/4046508468 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f98b8064090 0x7f98b8066550 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7f98b4004e20 tx=0x7f98b400b040).stop 2021-11-03T18:03:22.312 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.310+0000 7f98d207e700 1 -- 172.21.15.27:0/4046508468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98cc0f66e0 msgr2=0x7f98cc06b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:03:22.312 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.310+0000 7f98d207e700 1 --2- 172.21.15.27:0/4046508468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98cc0f66e0 0x7f98cc06b9a0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 rx=0x7f98c400bb10 tx=0x7f98c4004d10).stop 2021-11-03T18:03:22.313 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.310+0000 7f98d207e700 1 -- 172.21.15.27:0/4046508468 shutdown_connections 2021-11-03T18:03:22.313 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.310+0000 7f98d207e700 1 --2- 172.21.15.27:0/4046508468 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f98b8064090 0x7f98b8066550 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:22.313 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.310+0000 7f98d207e700 1 --2- 172.21.15.27:0/4046508468 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f98cc0f66e0 0x7f98cc06b9a0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:22.313 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.310+0000 7f98d207e700 1 --2- 172.21.15.27:0/4046508468 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f98cc0f7980 0x7f98cc069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:22.314 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.310+0000 7f98d207e700 1 -- 172.21.15.27:0/4046508468 >> 172.21.15.27:0/4046508468 conn(0x7f98cc0f1e40 msgr2=0x7f98cc0fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:03:22.314 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.310+0000 7f98d207e700 1 -- 172.21.15.27:0/4046508468 shutdown_connections 2021-11-03T18:03:22.314 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.310+0000 7f98d207e700 1 -- 172.21.15.27:0/4046508468 wait complete. 2021-11-03T18:03:22.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.412+0000 7f9cb91e0700 1 -- 172.21.15.27:0/1150474140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9cb40f48a0 msgr2=0x7f9cb40f4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:03:22.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.412+0000 7f9cb91e0700 1 --2- 172.21.15.27:0/1150474140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9cb40f48a0 0x7f9cb40f4cc0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0x7f9ca4004660 tx=0x7f9ca4009b30).stop 2021-11-03T18:03:22.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.412+0000 7f9cb91e0700 1 -- 172.21.15.27:0/1150474140 shutdown_connections 2021-11-03T18:03:22.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.412+0000 7f9cb91e0700 1 --2- 172.21.15.27:0/1150474140 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9cb40f5240 0x7f9cb40f7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:22.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.412+0000 7f9cb91e0700 1 --2- 172.21.15.27:0/1150474140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9cb40f48a0 0x7f9cb40f4cc0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:22.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.412+0000 7f9cb91e0700 1 -- 172.21.15.27:0/1150474140 >> 172.21.15.27:0/1150474140 conn(0x7f9cb40f0320 msgr2=0x7f9cb40f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:03:22.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.413+0000 7f9cb91e0700 1 -- 172.21.15.27:0/1150474140 shutdown_connections 2021-11-03T18:03:22.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.413+0000 7f9cb91e0700 1 -- 172.21.15.27:0/1150474140 wait complete. 2021-11-03T18:03:22.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.413+0000 7f9cb91e0700 1 Processor -- start 2021-11-03T18:03:22.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.413+0000 7f9cb91e0700 1 -- start start 2021-11-03T18:03:22.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.413+0000 7f9cb91e0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9cb40f48a0 0x7f9cb4106f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:03:22.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.413+0000 7f9cb91e0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9cb40f5240 0x7f9cb4107470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:03:22.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.413+0000 7f9cb91e0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb4107a20 con 0x7f9cb40f48a0 2021-11-03T18:03:22.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.413+0000 7f9cb91e0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cb4107b60 con 0x7f9cb40f5240 2021-11-03T18:03:22.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.414+0000 7f9cb3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9cb40f48a0 0x7f9cb4106f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:03:22.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.414+0000 7f9cb3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9cb40f48a0 0x7f9cb4106f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60936/0 (socket says 172.21.15.27:60936) 2021-11-03T18:03:22.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.414+0000 7f9cb3fff700 1 -- 172.21.15.27:0/905167803 learned_addr learned my addr 172.21.15.27:0/905167803 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:03:22.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.414+0000 7f9cb37fe700 1 --2- 172.21.15.27:0/905167803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9cb40f5240 0x7f9cb4107470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:03:22.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.414+0000 7f9cb3fff700 1 -- 172.21.15.27:0/905167803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9cb40f5240 msgr2=0x7f9cb4107470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:03:22.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.414+0000 7f9cb3fff700 1 --2- 172.21.15.27:0/905167803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9cb40f5240 0x7f9cb4107470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:22.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.414+0000 7f9cb3fff700 1 -- 172.21.15.27:0/905167803 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ca4005040 con 0x7f9cb40f48a0 2021-11-03T18:03:22.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.414+0000 7f9cb37fe700 1 --2- 172.21.15.27:0/905167803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9cb40f5240 0x7f9cb4107470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:03:22.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.414+0000 7f9cb3fff700 1 --2- 172.21.15.27:0/905167803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9cb40f48a0 0x7f9cb4106f80 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x7f9ca4009990 tx=0x7f9ca4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:03:22.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.414+0000 7f9cb0ff9700 1 -- 172.21.15.27:0/905167803 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ca4018070 con 0x7f9cb40f48a0 2021-11-03T18:03:22.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.414+0000 7f9cb0ff9700 1 -- 172.21.15.27:0/905167803 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f9ca4007ba0 con 0x7f9cb40f48a0 2021-11-03T18:03:22.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.414+0000 7f9cb91e0700 1 -- 172.21.15.27:0/905167803 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cb41084a0 con 0x7f9cb40f48a0 2021-11-03T18:03:22.419 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.414+0000 7f9cb0ff9700 1 -- 172.21.15.27:0/905167803 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ca401c3c0 con 0x7f9cb40f48a0 2021-11-03T18:03:22.419 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.414+0000 7f9cb91e0700 1 -- 172.21.15.27:0/905167803 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cb4108a00 con 0x7f9cb40f48a0 2021-11-03T18:03:22.419 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.415+0000 7f9cb91e0700 1 -- 172.21.15.27:0/905167803 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cb4045ad0 con 0x7f9cb40f48a0 2021-11-03T18:03:22.420 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.416+0000 7f9cb0ff9700 1 -- 172.21.15.27:0/905167803 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f9ca401d420 con 0x7f9cb40f48a0 2021-11-03T18:03:22.420 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.416+0000 7f9cb0ff9700 1 --2- 172.21.15.27:0/905167803 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9c9c064100 0x7f9c9c0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:03:22.420 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.416+0000 7f9cb0ff9700 1 -- 172.21.15.27:0/905167803 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f9ca4082560 con 0x7f9cb40f48a0 2021-11-03T18:03:22.420 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.416+0000 7f9cb37fe700 1 --2- 172.21.15.27:0/905167803 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9c9c064100 0x7f9c9c0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:03:22.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.417+0000 7f9cb37fe700 1 --2- 172.21.15.27:0/905167803 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9c9c064100 0x7f9c9c0665c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f9cb40f56c0 tx=0x7f9ca8009c00).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:03:22.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.420+0000 7f9cb0ff9700 1 -- 172.21.15.27:0/905167803 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f9ca40548c0 con 0x7f9cb40f48a0 2021-11-03T18:03:22.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:22 smithi115 ceph-mon[65453]: pgmap v1461: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:22.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:22 smithi027 ceph-mon[97244]: pgmap v1461: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:22.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.625+0000 7f9cb91e0700 1 -- 172.21.15.27:0/905167803 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9cb405f400 con 0x7f9cb40f48a0 2021-11-03T18:03:22.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.625+0000 7f9cb0ff9700 1 -- 172.21.15.27:0/905167803 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f9ca40544b0 con 0x7f9cb40f48a0 2021-11-03T18:03:22.623 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:03:22.624 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:03:22.624 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:03:22.624 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:03:22.624 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:03:22.625 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:03:22.625 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:03:22.625 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:03:22.625 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:03:22.626 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:03:22.626 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:03:22.626 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:03:22.626 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:03:22.627 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:03:22.627 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:03:22.627 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:03:22.627 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:03:22.627 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:03:22.628 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:03:22.628 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:03:22.628 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:03:22.628 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:03:22.629 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:03:22.629 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:03:22.629 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:03:22.629 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:03:22.630 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:03:22.630 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:03:22.630 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:03:22.630 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:03:22.630 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:03:22.631 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:03:22.631 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:03:22.631 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:03:22.631 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:03:22.632 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:03:22.632 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:03:22.632 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:03:22.632 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:03:22.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.627+0000 7f9cb91e0700 1 -- 172.21.15.27:0/905167803 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9c9c064100 msgr2=0x7f9c9c0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:03:22.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.627+0000 7f9cb91e0700 1 --2- 172.21.15.27:0/905167803 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9c9c064100 0x7f9c9c0665c0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7f9cb40f56c0 tx=0x7f9ca8009c00).stop 2021-11-03T18:03:22.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.627+0000 7f9cb91e0700 1 -- 172.21.15.27:0/905167803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9cb40f48a0 msgr2=0x7f9cb4106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:03:22.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.627+0000 7f9cb91e0700 1 --2- 172.21.15.27:0/905167803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9cb40f48a0 0x7f9cb4106f80 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x7f9ca4009990 tx=0x7f9ca4009b30).stop 2021-11-03T18:03:22.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.627+0000 7f9cb91e0700 1 -- 172.21.15.27:0/905167803 shutdown_connections 2021-11-03T18:03:22.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.627+0000 7f9cb91e0700 1 --2- 172.21.15.27:0/905167803 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9c9c064100 0x7f9c9c0665c0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:22.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.627+0000 7f9cb91e0700 1 --2- 172.21.15.27:0/905167803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9cb40f48a0 0x7f9cb4106f80 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:22.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.627+0000 7f9cb91e0700 1 --2- 172.21.15.27:0/905167803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9cb40f5240 0x7f9cb4107470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:22.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.627+0000 7f9cb91e0700 1 -- 172.21.15.27:0/905167803 >> 172.21.15.27:0/905167803 conn(0x7f9cb40f0320 msgr2=0x7f9cb40fb5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:03:22.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.627+0000 7f9cb91e0700 1 -- 172.21.15.27:0/905167803 shutdown_connections 2021-11-03T18:03:22.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:22.627+0000 7f9cb91e0700 1 -- 172.21.15.27:0/905167803 wait complete. 2021-11-03T18:03:22.636 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:03:23.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:23 smithi027 ceph-mon[97244]: from='client.35608 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:03:23.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:23 smithi027 ceph-mon[97244]: from='client.35612 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:03:23.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:23 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/4046508468' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:23.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:23 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/905167803' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:03:23.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:23 smithi115 ceph-mon[65453]: from='client.35608 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:03:23.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:23 smithi115 ceph-mon[65453]: from='client.35612 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:03:23.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:23 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/4046508468' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:23.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:23 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/905167803' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:03:24.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:24 smithi027 ceph-mon[97244]: pgmap v1462: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:24.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:24 smithi115 ceph-mon[65453]: pgmap v1462: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:25.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2847.427899 secs 2021-11-03T18:03:25.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2847.427899 secs 2021-11-03T18:03:26.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:26 smithi027 ceph-mon[97244]: pgmap v1463: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:26.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:26 smithi115 ceph-mon[65453]: pgmap v1463: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:28.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:28 smithi027 ceph-mon[97244]: pgmap v1464: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:28.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:03:28.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:03:28.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:03:28.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:28.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:28.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:28.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:28.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:28 smithi115 ceph-mon[65453]: pgmap v1464: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:28.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:03:28.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:03:28.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:03:28.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:28.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:28.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:28.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:29.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:29 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:03:29.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:29 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:03:30.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:30 smithi027 ceph-mon[97244]: pgmap v1465: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:30.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2852.428040 secs 2021-11-03T18:03:30.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:30 smithi115 ceph-mon[65453]: pgmap v1465: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:30.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2852.428040 secs 2021-11-03T18:03:32.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:32 smithi027 ceph-mon[97244]: pgmap v1466: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:32.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:32 smithi115 ceph-mon[65453]: pgmap v1466: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:34.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:34 smithi027 ceph-mon[97244]: pgmap v1467: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:34.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:34 smithi115 ceph-mon[65453]: pgmap v1467: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:35.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2857.428171 secs 2021-11-03T18:03:35.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2857.428171 secs 2021-11-03T18:03:36.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:36 smithi027 ceph-mon[97244]: pgmap v1468: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:36.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:36 smithi115 ceph-mon[65453]: pgmap v1468: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:38.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:38 smithi115 ceph-mon[65453]: pgmap v1469: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:38.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:03:38.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:03:38.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:03:38.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:38.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:38.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:38.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:38.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:38 smithi027 ceph-mon[97244]: pgmap v1469: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:38.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:03:38.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:03:38.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:03:38.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:38.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:38.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:38.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:39.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:39 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:03:39.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:39 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:03:40.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:40 smithi115 ceph-mon[65453]: pgmap v1470: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:40.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2862.428307 secs 2021-11-03T18:03:40.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:40 smithi027 ceph-mon[97244]: pgmap v1470: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:40.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2862.428307 secs 2021-11-03T18:03:42.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:42 smithi115 ceph-mon[65453]: pgmap v1471: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:42.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:42 smithi027 ceph-mon[97244]: pgmap v1471: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:44.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:44 smithi115 ceph-mon[65453]: pgmap v1472: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:44.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:44 smithi027 ceph-mon[97244]: pgmap v1472: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:45.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2867.428459 secs 2021-11-03T18:03:45.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2867.428459 secs 2021-11-03T18:03:46.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:46 smithi115 ceph-mon[65453]: pgmap v1473: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:46.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:46 smithi027 ceph-mon[97244]: pgmap v1473: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:48.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:48 smithi115 ceph-mon[65453]: pgmap v1474: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:48.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:03:48.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:48 smithi027 ceph-mon[97244]: pgmap v1474: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:48.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:03:49.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:49 smithi115 ceph-mon[65453]: poll took 60172.085 ms: 1 events 2021-11-03T18:03:49.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:03:49.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:03:49.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:49.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:49.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:49.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:49.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:49 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:03:49.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:49 smithi027 ceph-mon[97244]: poll took 60172.085 ms: 1 events 2021-11-03T18:03:49.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:03:49.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:03:49.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:49.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:49.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:49.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:49.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:49 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:03:50.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:50 smithi115 ceph-mon[65453]: pgmap v1475: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:50.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2872.428524 secs 2021-11-03T18:03:50.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:50 smithi027 ceph-mon[97244]: pgmap v1475: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:50.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2872.428524 secs 2021-11-03T18:03:52.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:52 smithi115 ceph-mon[65453]: pgmap v1476: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:52.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.742+0000 7f594e411700 1 -- 172.21.15.27:0/174653489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59480f6530 msgr2=0x7f59480f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:03:52.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.742+0000 7f594e411700 1 --2- 172.21.15.27:0/174653489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59480f6530 0x7f59480f6950 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 rx=0x7f5944004660 tx=0x7f5944009b30).stop 2021-11-03T18:03:52.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.743+0000 7f594e411700 1 -- 172.21.15.27:0/174653489 shutdown_connections 2021-11-03T18:03:52.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.743+0000 7f594e411700 1 --2- 172.21.15.27:0/174653489 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59480f77d0 0x7f59480f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:52.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.743+0000 7f594e411700 1 --2- 172.21.15.27:0/174653489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59480f6530 0x7f59480f6950 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:52.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.743+0000 7f594e411700 1 -- 172.21.15.27:0/174653489 >> 172.21.15.27:0/174653489 conn(0x7f59480f1cd0 msgr2=0x7f59480f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:03:52.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.743+0000 7f594e411700 1 -- 172.21.15.27:0/174653489 shutdown_connections 2021-11-03T18:03:52.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.743+0000 7f594e411700 1 -- 172.21.15.27:0/174653489 wait complete. 2021-11-03T18:03:52.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.743+0000 7f594e411700 1 Processor -- start 2021-11-03T18:03:52.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.743+0000 7f594e411700 1 -- start start 2021-11-03T18:03:52.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.743+0000 7f594e411700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59480f6530 0x7f5948106db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:03:52.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.743+0000 7f594e411700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59480f77d0 0x7f59481072a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:03:52.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.743+0000 7f594e411700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59481078a0 con 0x7f59480f77d0 2021-11-03T18:03:52.744 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.743+0000 7f594e411700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59481080e0 con 0x7f59480f6530 2021-11-03T18:03:52.746 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.744+0000 7f594cc0e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59480f77d0 0x7f59481072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:03:52.746 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.744+0000 7f594cc0e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59480f77d0 0x7f59481072a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60958/0 (socket says 172.21.15.27:60958) 2021-11-03T18:03:52.746 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.744+0000 7f594cc0e700 1 -- 172.21.15.27:0/2719652196 learned_addr learned my addr 172.21.15.27:0/2719652196 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:03:52.746 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.744+0000 7f594d40f700 1 --2- 172.21.15.27:0/2719652196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59480f6530 0x7f5948106db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:03:52.746 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.744+0000 7f594cc0e700 1 -- 172.21.15.27:0/2719652196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59480f6530 msgr2=0x7f5948106db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:03:52.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.744+0000 7f594cc0e700 1 --2- 172.21.15.27:0/2719652196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59480f6530 0x7f5948106db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:52.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.744+0000 7f594cc0e700 1 -- 172.21.15.27:0/2719652196 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5944005040 con 0x7f59480f77d0 2021-11-03T18:03:52.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.744+0000 7f594cc0e700 1 --2- 172.21.15.27:0/2719652196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59480f77d0 0x7f59481072a0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7f593800c530 tx=0x7f5938007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:03:52.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.744+0000 7f593dffb700 1 -- 172.21.15.27:0/2719652196 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f593800f650 con 0x7f59480f77d0 2021-11-03T18:03:52.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.744+0000 7f593dffb700 1 -- 172.21.15.27:0/2719652196 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f593800fc60 con 0x7f59480f77d0 2021-11-03T18:03:52.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.744+0000 7f594e411700 1 -- 172.21.15.27:0/2719652196 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59481083c0 con 0x7f59480f77d0 2021-11-03T18:03:52.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.744+0000 7f593dffb700 1 -- 172.21.15.27:0/2719652196 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5938012c20 con 0x7f59480f77d0 2021-11-03T18:03:52.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.745+0000 7f594e411700 1 -- 172.21.15.27:0/2719652196 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5948108900 con 0x7f59480f77d0 2021-11-03T18:03:52.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.745+0000 7f594e411700 1 -- 172.21.15.27:0/2719652196 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5948059100 con 0x7f59480f77d0 2021-11-03T18:03:52.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.746+0000 7f593dffb700 1 -- 172.21.15.27:0/2719652196 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f5938010070 con 0x7f59480f77d0 2021-11-03T18:03:52.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.746+0000 7f593dffb700 1 --2- 172.21.15.27:0/2719652196 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5934064100 0x7f59340665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:03:52.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.746+0000 7f593dffb700 1 -- 172.21.15.27:0/2719652196 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f59380805b0 con 0x7f59480f77d0 2021-11-03T18:03:52.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.747+0000 7f594d40f700 1 --2- 172.21.15.27:0/2719652196 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5934064100 0x7f59340665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:03:52.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.747+0000 7f594d40f700 1 --2- 172.21.15.27:0/2719652196 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5934064100 0x7f59340665c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f5944013690 tx=0x7f5944009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:03:52.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.751+0000 7f593dffb700 1 -- 172.21.15.27:0/2719652196 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f59380527c0 con 0x7f59480f77d0 2021-11-03T18:03:52.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:52 smithi027 ceph-mon[97244]: pgmap v1476: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:52.931 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.932+0000 7f594e411700 1 -- 172.21.15.27:0/2719652196 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f59480fee80 con 0x7f5934064100 2021-11-03T18:03:52.931 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.934+0000 7f593dffb700 1 -- 172.21.15.27:0/2719652196 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f59480fee80 con 0x7f5934064100 2021-11-03T18:03:52.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.936+0000 7f594e411700 1 -- 172.21.15.27:0/2719652196 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5934064100 msgr2=0x7f59340665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:03:52.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.936+0000 7f594e411700 1 --2- 172.21.15.27:0/2719652196 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5934064100 0x7f59340665c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7f5944013690 tx=0x7f5944009b30).stop 2021-11-03T18:03:52.935 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.936+0000 7f594e411700 1 -- 172.21.15.27:0/2719652196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59480f77d0 msgr2=0x7f59481072a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:03:52.935 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.936+0000 7f594e411700 1 --2- 172.21.15.27:0/2719652196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59480f77d0 0x7f59481072a0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7f593800c530 tx=0x7f5938007e20).stop 2021-11-03T18:03:52.936 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.936+0000 7f594e411700 1 -- 172.21.15.27:0/2719652196 shutdown_connections 2021-11-03T18:03:52.936 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.936+0000 7f594e411700 1 --2- 172.21.15.27:0/2719652196 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5934064100 0x7f59340665c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:52.937 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.936+0000 7f594e411700 1 --2- 172.21.15.27:0/2719652196 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59480f6530 0x7f5948106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:52.937 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.936+0000 7f594e411700 1 --2- 172.21.15.27:0/2719652196 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59480f77d0 0x7f59481072a0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:52.937 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.936+0000 7f594e411700 1 -- 172.21.15.27:0/2719652196 >> 172.21.15.27:0/2719652196 conn(0x7f59480f1cd0 msgr2=0x7f59480fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:03:52.938 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.936+0000 7f594e411700 1 -- 172.21.15.27:0/2719652196 shutdown_connections 2021-11-03T18:03:52.938 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:52.936+0000 7f594e411700 1 -- 172.21.15.27:0/2719652196 wait complete. 2021-11-03T18:03:52.944 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:03:53.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.037+0000 7f9ac6b29700 1 -- 172.21.15.27:0/3670150680 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ac00f55c0 msgr2=0x7f9ac00f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:03:53.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.037+0000 7f9ac6b29700 1 --2- 172.21.15.27:0/3670150680 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ac00f55c0 0x7f9ac00f59e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 rx=0x7f9ab80099e0 tx=0x7f9ab8004d10).stop 2021-11-03T18:03:53.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.037+0000 7f9ac6b29700 1 -- 172.21.15.27:0/3670150680 shutdown_connections 2021-11-03T18:03:53.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.037+0000 7f9ac6b29700 1 --2- 172.21.15.27:0/3670150680 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ac00f6860 0x7f9ac00f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:53.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.037+0000 7f9ac6b29700 1 --2- 172.21.15.27:0/3670150680 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ac00f55c0 0x7f9ac00f59e0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:53.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.037+0000 7f9ac6b29700 1 -- 172.21.15.27:0/3670150680 >> 172.21.15.27:0/3670150680 conn(0x7f9ac00f0d40 msgr2=0x7f9ac00f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:03:53.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.037+0000 7f9ac6b29700 1 -- 172.21.15.27:0/3670150680 shutdown_connections 2021-11-03T18:03:53.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.037+0000 7f9ac6b29700 1 -- 172.21.15.27:0/3670150680 wait complete. 2021-11-03T18:03:53.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.037+0000 7f9ac6b29700 1 Processor -- start 2021-11-03T18:03:53.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.037+0000 7f9ac6b29700 1 -- start start 2021-11-03T18:03:53.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.038+0000 7f9ac6b29700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ac00f55c0 0x7f9ac0109d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:03:53.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.038+0000 7f9ac6b29700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ac00f6860 0x7f9ac010c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:03:53.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.038+0000 7f9ac6b29700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac00603c0 con 0x7f9ac00f6860 2021-11-03T18:03:53.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.038+0000 7f9ac6b29700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac0060500 con 0x7f9ac00f55c0 2021-11-03T18:03:53.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.038+0000 7f9abffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ac00f6860 0x7f9ac010c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:03:53.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.038+0000 7f9abffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ac00f6860 0x7f9ac010c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60968/0 (socket says 172.21.15.27:60968) 2021-11-03T18:03:53.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.038+0000 7f9abffff700 1 -- 172.21.15.27:0/4243503149 learned_addr learned my addr 172.21.15.27:0/4243503149 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:03:53.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.038+0000 7f9ac48c5700 1 --2- 172.21.15.27:0/4243503149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ac00f55c0 0x7f9ac0109d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:03:53.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.038+0000 7f9abffff700 1 -- 172.21.15.27:0/4243503149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ac00f55c0 msgr2=0x7f9ac0109d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:03:53.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.038+0000 7f9abffff700 1 --2- 172.21.15.27:0/4243503149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ac00f55c0 0x7f9ac0109d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:53.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.038+0000 7f9abffff700 1 -- 172.21.15.27:0/4243503149 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab8009640 con 0x7f9ac00f6860 2021-11-03T18:03:53.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.038+0000 7f9ac48c5700 1 --2- 172.21.15.27:0/4243503149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ac00f55c0 0x7f9ac0109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:03:53.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.038+0000 7f9abffff700 1 --2- 172.21.15.27:0/4243503149 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ac00f6860 0x7f9ac010c280 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 rx=0x7f9ab40026d0 tx=0x7f9ab4009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:03:53.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.038+0000 7f9abd7fa700 1 -- 172.21.15.27:0/4243503149 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ab4009320 con 0x7f9ac00f6860 2021-11-03T18:03:53.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.038+0000 7f9abd7fa700 1 -- 172.21.15.27:0/4243503149 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f9ab400f040 con 0x7f9ac00f6860 2021-11-03T18:03:53.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.038+0000 7f9abd7fa700 1 -- 172.21.15.27:0/4243503149 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9ab40147e0 con 0x7f9ac00f6860 2021-11-03T18:03:53.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.039+0000 7f9ac6b29700 1 -- 172.21.15.27:0/4243503149 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ac010c830 con 0x7f9ac00f6860 2021-11-03T18:03:53.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.039+0000 7f9ac6b29700 1 -- 172.21.15.27:0/4243503149 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac010ccf0 con 0x7f9ac00f6860 2021-11-03T18:03:53.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.039+0000 7f9ac6b29700 1 -- 172.21.15.27:0/4243503149 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ac0045ad0 con 0x7f9ac00f6860 2021-11-03T18:03:53.046 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.040+0000 7f9abd7fa700 1 -- 172.21.15.27:0/4243503149 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f9ab4009480 con 0x7f9ac00f6860 2021-11-03T18:03:53.046 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.040+0000 7f9abd7fa700 1 --2- 172.21.15.27:0/4243503149 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9aa80640b0 0x7f9aa8066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:03:53.046 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.040+0000 7f9abd7fa700 1 -- 172.21.15.27:0/4243503149 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f9ab4080f10 con 0x7f9ac00f6860 2021-11-03T18:03:53.047 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.041+0000 7f9ac48c5700 1 --2- 172.21.15.27:0/4243503149 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9aa80640b0 0x7f9aa8066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:03:53.047 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.041+0000 7f9ac48c5700 1 --2- 172.21.15.27:0/4243503149 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9aa80640b0 0x7f9aa8066570 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f9ab800b730 tx=0x7f9ab8004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:03:53.048 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.045+0000 7f9abd7fa700 1 -- 172.21.15.27:0/4243503149 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f9ab4053210 con 0x7f9ac00f6860 2021-11-03T18:03:53.216 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.219+0000 7f9ac6b29700 1 -- 172.21.15.27:0/4243503149 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ac0057f10 con 0x7f9aa80640b0 2021-11-03T18:03:53.231 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.233+0000 7f9abd7fa700 1 -- 172.21.15.27:0/4243503149 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f9ac0057f10 con 0x7f9aa80640b0 2021-11-03T18:03:53.231 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:03:53.231 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (55m) 4m ago 56m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:03:53.232 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (47m) 4m ago 56m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:03:53.232 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (46m) 3m ago 55m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:03:53.232 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (55m) 4m ago 56m 41.8M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:03:53.232 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (48m) 4m ago 51m 26.0M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:03:53.233 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (51m) 4m ago 51m 3845M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:03:53.233 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (47m) 3m ago 51m 25.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:03:53.233 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (49m) 3m ago 51m 26.8M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:03:53.233 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (49m) 4m ago 57m 554M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:03:53.234 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (48m) 3m ago 55m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:03:53.234 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (48m) 4m ago 58m 235M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:03:53.234 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (47m) 3m ago 55m 112M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:03:53.234 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (56m) 4m ago 56m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:03:53.235 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (55m) 3m ago 55m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:03:53.235 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (46m) 4m ago 55m 351M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:03:53.235 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (46m) 4m ago 54m 383M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:03:53.235 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (45m) 4m ago 54m 284M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:03:53.235 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (45m) 3m ago 54m 404M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:03:53.236 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (44m) 3m ago 53m 322M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:03:53.236 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (44m) 3m ago 53m 327M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:03:53.236 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (55m) 4m ago 56m 55.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:03:53.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.235+0000 7f9ac6b29700 1 -- 172.21.15.27:0/4243503149 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9aa80640b0 msgr2=0x7f9aa8066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:03:53.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.235+0000 7f9ac6b29700 1 --2- 172.21.15.27:0/4243503149 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9aa80640b0 0x7f9aa8066570 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 rx=0x7f9ab800b730 tx=0x7f9ab8004d10).stop 2021-11-03T18:03:53.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.235+0000 7f9ac6b29700 1 -- 172.21.15.27:0/4243503149 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ac00f6860 msgr2=0x7f9ac010c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:03:53.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.235+0000 7f9ac6b29700 1 --2- 172.21.15.27:0/4243503149 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ac00f6860 0x7f9ac010c280 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 rx=0x7f9ab40026d0 tx=0x7f9ab4009de0).stop 2021-11-03T18:03:53.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.235+0000 7f9ac6b29700 1 -- 172.21.15.27:0/4243503149 shutdown_connections 2021-11-03T18:03:53.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.235+0000 7f9ac6b29700 1 --2- 172.21.15.27:0/4243503149 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9aa80640b0 0x7f9aa8066570 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:53.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.235+0000 7f9ac6b29700 1 --2- 172.21.15.27:0/4243503149 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9ac00f55c0 0x7f9ac0109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:53.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.235+0000 7f9ac6b29700 1 --2- 172.21.15.27:0/4243503149 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9ac00f6860 0x7f9ac010c280 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:53.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.235+0000 7f9ac6b29700 1 -- 172.21.15.27:0/4243503149 >> 172.21.15.27:0/4243503149 conn(0x7f9ac00f0d40 msgr2=0x7f9ac00fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:03:53.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.235+0000 7f9ac6b29700 1 -- 172.21.15.27:0/4243503149 shutdown_connections 2021-11-03T18:03:53.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.235+0000 7f9ac6b29700 1 -- 172.21.15.27:0/4243503149 wait complete. 2021-11-03T18:03:53.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.334+0000 7fbcdc8b7700 1 -- 172.21.15.27:0/722309140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcd80f4a80 msgr2=0x7fbcd80f4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:03:53.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.334+0000 7fbcdc8b7700 1 --2- 172.21.15.27:0/722309140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcd80f4a80 0x7fbcd80f4ea0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7fbcc80099e0 tx=0x7fbcc8004d10).stop 2021-11-03T18:03:53.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.334+0000 7fbcdc8b7700 1 -- 172.21.15.27:0/722309140 shutdown_connections 2021-11-03T18:03:53.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.334+0000 7fbcdc8b7700 1 --2- 172.21.15.27:0/722309140 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcd80f5420 0x7fbcd80fbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T18:03:53.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.334+0000 7fbcdc8b7700 1 --2- 172.21.15.27:0/722309140 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcd80f4a80 0x7fbcd80f4ea0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:53.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.334+0000 7fbcdc8b7700 1 -- 172.21.15.27:0/722309140 >> 172.21.15.27:0/722309140 conn(0x7fbcd80f04c0 msgr2=0x7fbcd80f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:03:53.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.334+0000 7fbcdc8b7700 1 -- 172.21.15.27:0/722309140 shutdown_connections 2021-11-03T18:03:53.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.334+0000 7fbcdc8b7700 1 -- 172.21.15.27:0/722309140 wait complete. 2021-11-03T18:03:53.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.335+0000 7fbcdc8b7700 1 Processor -- start 2021-11-03T18:03:53.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.335+0000 7fbcdc8b7700 1 -- start start 2021-11-03T18:03:53.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.335+0000 7fbcdc8b7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcd80f4a80 0x7fbcd8107070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:03:53.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.335+0000 7fbcdc8b7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcd80f5420 0x7fbcd8107560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:03:53.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.335+0000 7fbcdc8b7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd8107ba0 con 0x7fbcd80f5420 2021-11-03T18:03:53.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.335+0000 7fbcdc8b7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd8108450 con 0x7fbcd80f4a80 2021-11-03T18:03:53.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.335+0000 7fbcd5d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcd80f5420 0x7fbcd8107560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:03:53.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.335+0000 7fbcd5d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcd80f5420 0x7fbcd8107560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60978/0 (socket says 172.21.15.27:60978) 2021-11-03T18:03:53.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.335+0000 7fbcd5d9b700 1 -- 172.21.15.27:0/2549710339 learned_addr learned my addr 172.21.15.27:0/2549710339 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:03:53.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.335+0000 7fbcd659c700 1 --2- 172.21.15.27:0/2549710339 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcd80f4a80 0x7fbcd8107070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:03:53.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.336+0000 7fbcd5d9b700 1 -- 172.21.15.27:0/2549710339 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcd80f4a80 msgr2=0x7fbcd8107070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:03:53.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.336+0000 7fbcd5d9b700 1 --2- 172.21.15.27:0/2549710339 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcd80f4a80 0x7fbcd8107070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:53.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.336+0000 7fbcd5d9b700 1 -- 172.21.15.27:0/2549710339 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc8009640 con 0x7fbcd80f5420 2021-11-03T18:03:53.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.336+0000 7fbcd5d9b700 1 --2- 172.21.15.27:0/2549710339 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcd80f5420 0x7fbcd8107560 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 rx=0x7fbcc000b410 tx=0x7fbcc0006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:03:53.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.336+0000 7fbcceffd700 1 -- 172.21.15.27:0/2549710339 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcc00125f0 con 0x7fbcd80f5420 2021-11-03T18:03:53.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.336+0000 7fbcdc8b7700 1 -- 172.21.15.27:0/2549710339 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcd8108730 con 0x7fbcd80f5420 2021-11-03T18:03:53.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.336+0000 7fbcceffd700 1 -- 172.21.15.27:0/2549710339 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fbcc000be30 con 0x7fbcd80f5420 2021-11-03T18:03:53.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.336+0000 7fbcceffd700 1 -- 172.21.15.27:0/2549710339 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcc0002a70 con 0x7fbcd80f5420 2021-11-03T18:03:53.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.336+0000 7fbcdc8b7700 1 -- 172.21.15.27:0/2549710339 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcd8108c70 con 0x7fbcd80f5420 2021-11-03T18:03:53.342 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.337+0000 7fbcdc8b7700 1 -- 172.21.15.27:0/2549710339 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcd80f4150 con 0x7fbcd80f5420 2021-11-03T18:03:53.342 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.338+0000 7fbcceffd700 1 -- 172.21.15.27:0/2549710339 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fbcc000f030 con 0x7fbcd80f5420 2021-11-03T18:03:53.342 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.338+0000 7fbcceffd700 1 --2- 172.21.15.27:0/2549710339 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbcc40640b0 0x7fbcc4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:03:53.342 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.338+0000 7fbcceffd700 1 -- 172.21.15.27:0/2549710339 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fbcc0080fd0 con 0x7fbcd80f5420 2021-11-03T18:03:53.343 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.338+0000 7fbcd659c700 1 --2- 172.21.15.27:0/2549710339 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbcc40640b0 0x7fbcc4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:03:53.343 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.339+0000 7fbcd659c700 1 --2- 172.21.15.27:0/2549710339 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbcc40640b0 0x7fbcc4066570 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7fbcc800b730 tx=0x7fbcc8004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:03:53.343 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.342+0000 7fbcceffd700 1 -- 172.21.15.27:0/2549710339 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fbcc00532d0 con 0x7fbcd80f5420 2021-11-03T18:03:53.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.568+0000 7fbcdc8b7700 1 -- 172.21.15.27:0/2549710339 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbcd8045ad0 con 0x7fbcd80f5420 2021-11-03T18:03:53.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.569+0000 7fbcceffd700 1 -- 172.21.15.27:0/2549710339 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fbcc0052ec0 con 0x7fbcd80f5420 2021-11-03T18:03:53.567 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:03:53.568 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:03:53.568 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:03:53.568 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:03:53.568 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:03:53.569 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:03:53.569 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:03:53.569 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:03:53.569 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:03:53.570 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:03:53.570 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:03:53.570 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:03:53.570 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:03:53.571 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:03:53.571 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:03:53.571 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:03:53.571 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:03:53.572 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:03:53.572 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.571+0000 7fbcdc8b7700 1 -- 172.21.15.27:0/2549710339 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbcc40640b0 msgr2=0x7fbcc4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:03:53.573 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.571+0000 7fbcdc8b7700 1 --2- 172.21.15.27:0/2549710339 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbcc40640b0 0x7fbcc4066570 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7fbcc800b730 tx=0x7fbcc8004d10).stop 2021-11-03T18:03:53.573 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.571+0000 7fbcdc8b7700 1 -- 172.21.15.27:0/2549710339 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcd80f5420 msgr2=0x7fbcd8107560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:03:53.573 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.571+0000 7fbcdc8b7700 1 --2- 172.21.15.27:0/2549710339 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcd80f5420 0x7fbcd8107560 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 rx=0x7fbcc000b410 tx=0x7fbcc0006ef0).stop 2021-11-03T18:03:53.573 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.571+0000 7fbcdc8b7700 1 -- 172.21.15.27:0/2549710339 shutdown_connections 2021-11-03T18:03:53.574 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.571+0000 7fbcdc8b7700 1 --2- 172.21.15.27:0/2549710339 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbcc40640b0 0x7fbcc4066570 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:53.574 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.571+0000 7fbcdc8b7700 1 --2- 172.21.15.27:0/2549710339 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcd80f4a80 0x7fbcd8107070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:53.574 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.571+0000 7fbcdc8b7700 1 --2- 172.21.15.27:0/2549710339 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcd80f5420 0x7fbcd8107560 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:53.574 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.571+0000 7fbcdc8b7700 1 -- 172.21.15.27:0/2549710339 >> 172.21.15.27:0/2549710339 conn(0x7fbcd80f04c0 msgr2=0x7fbcd80f8530 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:03:53.575 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.571+0000 7fbcdc8b7700 1 -- 172.21.15.27:0/2549710339 shutdown_connections 2021-11-03T18:03:53.575 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.571+0000 7fbcdc8b7700 1 -- 172.21.15.27:0/2549710339 wait complete. 2021-11-03T18:03:53.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.671+0000 7f082dd5f700 1 -- 172.21.15.27:0/603680060 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08280f98a0 msgr2=0x7f08280fbc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:03:53.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.671+0000 7f082dd5f700 1 --2- 172.21.15.27:0/603680060 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08280f98a0 0x7f08280fbc90 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7f0818004660 tx=0x7f0818009b30).stop 2021-11-03T18:03:53.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.671+0000 7f082dd5f700 1 -- 172.21.15.27:0/603680060 shutdown_connections 2021-11-03T18:03:53.670 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.671+0000 7f082dd5f700 1 --2- 172.21.15.27:0/603680060 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08280f98a0 0x7f08280fbc90 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:53.670 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.671+0000 7f082dd5f700 1 --2- 172.21.15.27:0/603680060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0828060290 0x7f08280f9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:53.670 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.671+0000 7f082dd5f700 1 -- 172.21.15.27:0/603680060 >> 172.21.15.27:0/603680060 conn(0x7f08280f0d40 msgr2=0x7f08280f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:03:53.670 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.671+0000 7f082dd5f700 1 -- 172.21.15.27:0/603680060 shutdown_connections 2021-11-03T18:03:53.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.671+0000 7f082dd5f700 1 -- 172.21.15.27:0/603680060 wait complete. 2021-11-03T18:03:53.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.671+0000 7f082dd5f700 1 Processor -- start 2021-11-03T18:03:53.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.672+0000 7f082dd5f700 1 -- start start 2021-11-03T18:03:53.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.672+0000 7f082dd5f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0828060290 0x7f08280ffac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:03:53.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.672+0000 7f082dd5f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08280f98a0 0x7f08280fffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:03:53.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.672+0000 7f082dd5f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08281005b0 con 0x7f0828060290 2021-11-03T18:03:53.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.672+0000 7f082dd5f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08281006f0 con 0x7f08280f98a0 2021-11-03T18:03:53.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.672+0000 7f08277fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0828060290 0x7f08280ffac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:03:53.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.672+0000 7f08277fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0828060290 0x7f08280ffac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:60986/0 (socket says 172.21.15.27:60986) 2021-11-03T18:03:53.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.672+0000 7f08277fe700 1 -- 172.21.15.27:0/3565987835 learned_addr learned my addr 172.21.15.27:0/3565987835 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:03:53.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.672+0000 7f0826ffd700 1 --2- 172.21.15.27:0/3565987835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08280f98a0 0x7f08280fffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:03:53.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.672+0000 7f08277fe700 1 -- 172.21.15.27:0/3565987835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08280f98a0 msgr2=0x7f08280fffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:03:53.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.672+0000 7f08277fe700 1 --2- 172.21.15.27:0/3565987835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08280f98a0 0x7f08280fffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:53.676 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.672+0000 7f08277fe700 1 -- 172.21.15.27:0/3565987835 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0818005040 con 0x7f0828060290 2021-11-03T18:03:53.676 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.672+0000 7f08277fe700 1 --2- 172.21.15.27:0/3565987835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0828060290 0x7f08280ffac0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7f081c0026d0 tx=0x7f081c009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:03:53.676 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.672+0000 7f080ffff700 1 -- 172.21.15.27:0/3565987835 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f081c009320 con 0x7f0828060290 2021-11-03T18:03:53.676 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.672+0000 7f082dd5f700 1 -- 172.21.15.27:0/3565987835 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0828101060 con 0x7f0828060290 2021-11-03T18:03:53.677 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.672+0000 7f080ffff700 1 -- 172.21.15.27:0/3565987835 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f081c00f040 con 0x7f0828060290 2021-11-03T18:03:53.677 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.672+0000 7f080ffff700 1 -- 172.21.15.27:0/3565987835 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f081c0147e0 con 0x7f0828060290 2021-11-03T18:03:53.677 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.672+0000 7f082dd5f700 1 -- 172.21.15.27:0/3565987835 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0828101620 con 0x7f0828060290 2021-11-03T18:03:53.677 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.673+0000 7f082dd5f700 1 -- 172.21.15.27:0/3565987835 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0828045ad0 con 0x7f0828060290 2021-11-03T18:03:53.677 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.674+0000 7f080ffff700 1 -- 172.21.15.27:0/3565987835 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f081c009480 con 0x7f0828060290 2021-11-03T18:03:53.678 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.674+0000 7f080ffff700 1 --2- 172.21.15.27:0/3565987835 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f08100640b0 0x7f0810066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:03:53.678 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.674+0000 7f080ffff700 1 -- 172.21.15.27:0/3565987835 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f081c080f10 con 0x7f0828060290 2021-11-03T18:03:53.678 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.674+0000 7f0826ffd700 1 --2- 172.21.15.27:0/3565987835 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f08100640b0 0x7f0810066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:03:53.678 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.675+0000 7f0826ffd700 1 --2- 172.21.15.27:0/3565987835 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f08100640b0 0x7f0810066570 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7f0818004630 tx=0x7f0818009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:03:53.679 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.678+0000 7f080ffff700 1 -- 172.21.15.27:0/3565987835 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f081c053210 con 0x7f0828060290 2021-11-03T18:03:53.885 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.887+0000 7f082dd5f700 1 -- 172.21.15.27:0/3565987835 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0828101cc0 con 0x7f0828060290 2021-11-03T18:03:53.885 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.887+0000 7f080ffff700 1 -- 172.21.15.27:0/3565987835 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f081c052e00 con 0x7f0828060290 2021-11-03T18:03:53.886 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:03:53.886 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:03:53.886 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:03:53.886 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:03:53.887 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:03:53.887 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:03:53.887 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:03:53.887 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:03:53.887 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:03:53.888 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:03:53.888 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:03:53.889 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:03:53.889 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:03:53.890 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:03:53.890 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:03:53.890 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:03:53.890 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:03:53.891 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:03:53.891 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:03:53.891 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:03:53.891 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:03:53.891 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:03:53.892 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:03:53.892 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:03:53.892 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:03:53.892 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:03:53.893 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:03:53.893 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:03:53.893 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:03:53.893 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:03:53.893 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:03:53.894 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:03:53.894 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:03:53.894 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:03:53.895 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:03:53.895 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:03:53.895 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:03:53.895 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:03:53.896 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:03:53.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.889+0000 7f082dd5f700 1 -- 172.21.15.27:0/3565987835 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f08100640b0 msgr2=0x7f0810066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:03:53.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.889+0000 7f082dd5f700 1 --2- 172.21.15.27:0/3565987835 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f08100640b0 0x7f0810066570 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 rx=0x7f0818004630 tx=0x7f0818009b30).stop 2021-11-03T18:03:53.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.889+0000 7f082dd5f700 1 -- 172.21.15.27:0/3565987835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0828060290 msgr2=0x7f08280ffac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:03:53.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.889+0000 7f082dd5f700 1 --2- 172.21.15.27:0/3565987835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0828060290 0x7f08280ffac0 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7f081c0026d0 tx=0x7f081c009de0).stop 2021-11-03T18:03:53.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.889+0000 7f082dd5f700 1 -- 172.21.15.27:0/3565987835 shutdown_connections 2021-11-03T18:03:53.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.889+0000 7f082dd5f700 1 --2- 172.21.15.27:0/3565987835 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f08100640b0 0x7f0810066570 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:53.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.889+0000 7f082dd5f700 1 --2- 172.21.15.27:0/3565987835 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0828060290 0x7f08280ffac0 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:53.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.889+0000 7f082dd5f700 1 --2- 172.21.15.27:0/3565987835 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08280f98a0 0x7f08280fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:03:53.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.889+0000 7f082dd5f700 1 -- 172.21.15.27:0/3565987835 >> 172.21.15.27:0/3565987835 conn(0x7f08280f0d40 msgr2=0x7f08280f51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:03:53.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.889+0000 7f082dd5f700 1 -- 172.21.15.27:0/3565987835 shutdown_connections 2021-11-03T18:03:53.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:03:53.889+0000 7f082dd5f700 1 -- 172.21.15.27:0/3565987835 wait complete. 2021-11-03T18:03:53.899 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:03:54.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:54 smithi115 ceph-mon[65453]: pgmap v1477: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:54.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:54 smithi115 ceph-mon[65453]: from='client.35624 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:03:54.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:54 smithi115 ceph-mon[65453]: from='client.35628 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:03:54.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:54 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2549710339' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:54.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:54 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3565987835' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:03:54.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:54 smithi027 ceph-mon[97244]: pgmap v1477: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:54.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:54 smithi027 ceph-mon[97244]: from='client.35624 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:03:54.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:54 smithi027 ceph-mon[97244]: from='client.35628 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:03:54.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:54 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2549710339' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:54.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:54 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3565987835' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:03:55.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2877.428662 secs 2021-11-03T18:03:55.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2877.428662 secs 2021-11-03T18:03:56.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:56 smithi115 ceph-mon[65453]: pgmap v1478: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:56.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:03:56.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:03:56.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:56 smithi027 ceph-mon[97244]: pgmap v1478: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:56.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:03:56.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:03:58.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:58 smithi115 ceph-mon[65453]: pgmap v1479: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:58.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:58 smithi027 ceph-mon[97244]: pgmap v1479: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:03:59.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:03:59.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:03:59.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:03:59.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:59.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:59.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:59.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:59.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:03:59 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:03:59.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:03:59.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:03:59.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:03:59.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:59.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:59.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:59.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:03:59.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:03:59 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:04:00.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:00 smithi115 ceph-mon[65453]: pgmap v1480: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:00.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:00 smithi115 ceph-mon[65453]: 1 slow requests, 1 included below; oldest blocked for > 2882.428808 secs 2021-11-03T18:04:00.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:00 smithi115 ceph-mon[65453]: slow request 2882.428807 seconds old, received at 2021-11-03T17:15:56.956064+0000: client_request(client.24331:21043 rmdir #0x10000003957/db1 2021-11-03T17:15:56.954824+0000 caller_uid=1000, caller_gid=1254{6,36,1000,1254,}) currently cleaned up request 2021-11-03T18:04:00.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:00 smithi027 ceph-mon[97244]: pgmap v1480: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:00.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:00 smithi027 ceph-mon[97244]: 1 slow requests, 1 included below; oldest blocked for > 2882.428808 secs 2021-11-03T18:04:00.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:00 smithi027 ceph-mon[97244]: slow request 2882.428807 seconds old, received at 2021-11-03T17:15:56.956064+0000: client_request(client.24331:21043 rmdir #0x10000003957/db1 2021-11-03T17:15:56.954824+0000 caller_uid=1000, caller_gid=1254{6,36,1000,1254,}) currently cleaned up request 2021-11-03T18:04:02.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:02 smithi115 ceph-mon[65453]: pgmap v1481: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:02.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:02 smithi027 ceph-mon[97244]: pgmap v1481: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:04.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:04 smithi115 ceph-mon[65453]: pgmap v1482: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:04.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:04 smithi027 ceph-mon[97244]: pgmap v1482: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:05.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2887.428945 secs 2021-11-03T18:04:05.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2887.428945 secs 2021-11-03T18:04:06.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:06 smithi115 ceph-mon[65453]: pgmap v1483: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:06.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:06 smithi027 ceph-mon[97244]: pgmap v1483: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:08.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:08 smithi115 ceph-mon[65453]: pgmap v1484: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:08.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:08 smithi027 ceph-mon[97244]: pgmap v1484: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:09.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:04:09.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:04:09.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:04:09.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:09.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:09.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:09.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:09.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:09 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:04:09.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:04:09.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:04:09.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:04:09.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:09.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:09.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:09.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:09.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:09 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:04:10.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:10 smithi115 ceph-mon[65453]: pgmap v1485: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:10.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2892.429091 secs 2021-11-03T18:04:10.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:10 smithi027 ceph-mon[97244]: pgmap v1485: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:10.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2892.429091 secs 2021-11-03T18:04:12.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:12 smithi115 ceph-mon[65453]: pgmap v1486: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:12.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:12 smithi027 ceph-mon[97244]: pgmap v1486: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:14.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:14 smithi115 ceph-mon[65453]: pgmap v1487: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:14.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:14 smithi027 ceph-mon[97244]: pgmap v1487: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:15.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2897.429181 secs 2021-11-03T18:04:15.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2897.429181 secs 2021-11-03T18:04:16.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:16 smithi115 ceph-mon[65453]: pgmap v1488: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:16.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:16 smithi027 ceph-mon[97244]: pgmap v1488: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:18.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:18 smithi115 ceph-mon[65453]: pgmap v1489: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:18.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:18 smithi027 ceph-mon[97244]: pgmap v1489: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:19.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:04:19.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:04:19.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:04:19.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:19.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:19.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:19.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:19.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:19 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:04:19.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:04:19.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:04:19.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:04:19.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:19.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:19.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:19.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:19.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:19 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:04:20.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:20 smithi115 ceph-mon[65453]: pgmap v1490: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:20.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2902.429329 secs 2021-11-03T18:04:20.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:20 smithi027 ceph-mon[97244]: pgmap v1490: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:20.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2902.429329 secs 2021-11-03T18:04:22.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:22 smithi115 ceph-mon[65453]: pgmap v1491: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:22.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:22 smithi027 ceph-mon[97244]: pgmap v1491: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:24.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.004+0000 7f036307c700 1 -- 172.21.15.27:0/1225089006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f035c0f96a0 msgr2=0x7f035c0fbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:04:24.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.004+0000 7f036307c700 1 --2- 172.21.15.27:0/1225089006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f035c0f96a0 0x7f035c0fbaf0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 rx=0x7f0358004660 tx=0x7f0358009b30).stop 2021-11-03T18:04:24.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.004+0000 7f036307c700 1 -- 172.21.15.27:0/1225089006 shutdown_connections 2021-11-03T18:04:24.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.004+0000 7f036307c700 1 --2- 172.21.15.27:0/1225089006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f035c0f96a0 0x7f035c0fbaf0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:24.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.004+0000 7f036307c700 1 --2- 172.21.15.27:0/1225089006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f035c060310 0x7f035c0f9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:24.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.004+0000 7f036307c700 1 -- 172.21.15.27:0/1225089006 >> 172.21.15.27:0/1225089006 conn(0x7f035c0f0ba0 msgr2=0x7f035c0f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:04:24.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.004+0000 7f036307c700 1 -- 172.21.15.27:0/1225089006 shutdown_connections 2021-11-03T18:04:24.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.004+0000 7f036307c700 1 -- 172.21.15.27:0/1225089006 wait complete. 2021-11-03T18:04:24.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.004+0000 7f036307c700 1 Processor -- start 2021-11-03T18:04:24.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.004+0000 7f036307c700 1 -- start start 2021-11-03T18:04:24.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.005+0000 7f036307c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f035c060310 0x7f035c0f6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:04:24.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.005+0000 7f036307c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f035c0f96a0 0x7f035c0f5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:04:24.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.005+0000 7f036307c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035c0f5580 con 0x7f035c060310 2021-11-03T18:04:24.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.005+0000 7f036307c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f035c0f56c0 con 0x7f035c0f96a0 2021-11-03T18:04:24.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.005+0000 7f036207a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f035c060310 0x7f035c0f6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:04:24.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.005+0000 7f036207a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f035c060310 0x7f035c0f6ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32772/0 (socket says 172.21.15.27:32772) 2021-11-03T18:04:24.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.005+0000 7f036207a700 1 -- 172.21.15.27:0/731470895 learned_addr learned my addr 172.21.15.27:0/731470895 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:04:24.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.005+0000 7f0361879700 1 --2- 172.21.15.27:0/731470895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f035c0f96a0 0x7f035c0f5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:04:24.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.005+0000 7f036207a700 1 -- 172.21.15.27:0/731470895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f035c0f96a0 msgr2=0x7f035c0f5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:04:24.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.005+0000 7f036207a700 1 --2- 172.21.15.27:0/731470895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f035c0f96a0 0x7f035c0f5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:24.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.005+0000 7f036207a700 1 -- 172.21.15.27:0/731470895 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0358005040 con 0x7f035c060310 2021-11-03T18:04:24.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.005+0000 7f036207a700 1 --2- 172.21.15.27:0/731470895 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f035c060310 0x7f035c0f6ba0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7f034c00c530 tx=0x7f034c007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:04:24.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.005+0000 7f0352ffd700 1 -- 172.21.15.27:0/731470895 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f034c00f650 con 0x7f035c060310 2021-11-03T18:04:24.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.006+0000 7f0352ffd700 1 -- 172.21.15.27:0/731470895 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f034c00fc60 con 0x7f035c060310 2021-11-03T18:04:24.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.006+0000 7f036307c700 1 -- 172.21.15.27:0/731470895 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f035c0f59a0 con 0x7f035c060310 2021-11-03T18:04:24.009 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.006+0000 7f0352ffd700 1 -- 172.21.15.27:0/731470895 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f034c012c20 con 0x7f035c060310 2021-11-03T18:04:24.010 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.006+0000 7f036307c700 1 -- 172.21.15.27:0/731470895 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f035c10ed70 con 0x7f035c060310 2021-11-03T18:04:24.010 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.007+0000 7f036307c700 1 -- 172.21.15.27:0/731470895 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f035c045ad0 con 0x7f035c060310 2021-11-03T18:04:24.011 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.007+0000 7f0352ffd700 1 -- 172.21.15.27:0/731470895 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f034c010070 con 0x7f035c060310 2021-11-03T18:04:24.011 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.008+0000 7f0352ffd700 1 --2- 172.21.15.27:0/731470895 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0348064100 0x7f03480665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:04:24.011 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.008+0000 7f0352ffd700 1 -- 172.21.15.27:0/731470895 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f034c0805d0 con 0x7f035c060310 2021-11-03T18:04:24.012 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.008+0000 7f0361879700 1 --2- 172.21.15.27:0/731470895 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0348064100 0x7f03480665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:04:24.012 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.008+0000 7f0361879700 1 --2- 172.21.15.27:0/731470895 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0348064100 0x7f03480665c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f0358004630 tx=0x7f0358009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:04:24.012 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.012+0000 7f0352ffd700 1 -- 172.21.15.27:0/731470895 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f034c0528a0 con 0x7f035c060310 2021-11-03T18:04:24.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.189+0000 7f036307c700 1 -- 172.21.15.27:0/731470895 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f035c0fee10 con 0x7f0348064100 2021-11-03T18:04:24.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.191+0000 7f0352ffd700 1 -- 172.21.15.27:0/731470895 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f035c0fee10 con 0x7f0348064100 2021-11-03T18:04:24.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.192+0000 7f036307c700 1 -- 172.21.15.27:0/731470895 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0348064100 msgr2=0x7f03480665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:04:24.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.192+0000 7f036307c700 1 --2- 172.21.15.27:0/731470895 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0348064100 0x7f03480665c0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7f0358004630 tx=0x7f0358009b30).stop 2021-11-03T18:04:24.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.192+0000 7f036307c700 1 -- 172.21.15.27:0/731470895 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f035c060310 msgr2=0x7f035c0f6ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:04:24.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.192+0000 7f036307c700 1 --2- 172.21.15.27:0/731470895 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f035c060310 0x7f035c0f6ba0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7f034c00c530 tx=0x7f034c007e20).stop 2021-11-03T18:04:24.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.193+0000 7f036307c700 1 -- 172.21.15.27:0/731470895 shutdown_connections 2021-11-03T18:04:24.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.193+0000 7f036307c700 1 --2- 172.21.15.27:0/731470895 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0348064100 0x7f03480665c0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:24.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.193+0000 7f036307c700 1 --2- 172.21.15.27:0/731470895 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f035c060310 0x7f035c0f6ba0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:24.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.193+0000 7f036307c700 1 --2- 172.21.15.27:0/731470895 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f035c0f96a0 0x7f035c0f5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:24.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.193+0000 7f036307c700 1 -- 172.21.15.27:0/731470895 >> 172.21.15.27:0/731470895 conn(0x7f035c0f0ba0 msgr2=0x7f035c0f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:04:24.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.193+0000 7f036307c700 1 -- 172.21.15.27:0/731470895 shutdown_connections 2021-11-03T18:04:24.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.193+0000 7f036307c700 1 -- 172.21.15.27:0/731470895 wait complete. 2021-11-03T18:04:24.199 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:04:24.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.292+0000 7fdc57cb0700 1 -- 172.21.15.27:0/3052711046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc500f6b90 msgr2=0x7fdc500f6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:04:24.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.292+0000 7fdc57cb0700 1 --2- 172.21.15.27:0/3052711046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc500f6b90 0x7fdc500f6fb0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7fdc4c004660 tx=0x7fdc4c009b30).stop 2021-11-03T18:04:24.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.293+0000 7fdc57cb0700 1 -- 172.21.15.27:0/3052711046 shutdown_connections 2021-11-03T18:04:24.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.293+0000 7fdc57cb0700 1 --2- 172.21.15.27:0/3052711046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc500f74a0 0x7fdc500f5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:24.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.293+0000 7fdc57cb0700 1 --2- 172.21.15.27:0/3052711046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc500f6b90 0x7fdc500f6fb0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:24.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.293+0000 7fdc57cb0700 1 -- 172.21.15.27:0/3052711046 >> 172.21.15.27:0/3052711046 conn(0x7fdc500f0540 msgr2=0x7fdc500f2960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:04:24.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.293+0000 7fdc57cb0700 1 -- 172.21.15.27:0/3052711046 shutdown_connections 2021-11-03T18:04:24.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.293+0000 7fdc57cb0700 1 -- 172.21.15.27:0/3052711046 wait complete. 2021-11-03T18:04:24.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.293+0000 7fdc57cb0700 1 Processor -- start 2021-11-03T18:04:24.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.293+0000 7fdc57cb0700 1 -- start start 2021-11-03T18:04:24.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.293+0000 7fdc57cb0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc500f6b90 0x7fdc50104d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:04:24.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.293+0000 7fdc57cb0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc500f74a0 0x7fdc50105260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:04:24.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.293+0000 7fdc57cb0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc50105860 con 0x7fdc500f74a0 2021-11-03T18:04:24.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.293+0000 7fdc57cb0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc50106070 con 0x7fdc500f6b90 2021-11-03T18:04:24.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.293+0000 7fdc5524b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc500f74a0 0x7fdc50105260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:04:24.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.293+0000 7fdc5524b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc500f74a0 0x7fdc50105260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32784/0 (socket says 172.21.15.27:32784) 2021-11-03T18:04:24.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.293+0000 7fdc5524b700 1 -- 172.21.15.27:0/364016422 learned_addr learned my addr 172.21.15.27:0/364016422 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:04:24.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.294+0000 7fdc55a4c700 1 --2- 172.21.15.27:0/364016422 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc500f6b90 0x7fdc50104d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:04:24.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.294+0000 7fdc5524b700 1 -- 172.21.15.27:0/364016422 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc500f6b90 msgr2=0x7fdc50104d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:04:24.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.294+0000 7fdc5524b700 1 --2- 172.21.15.27:0/364016422 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc500f6b90 0x7fdc50104d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:24.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.294+0000 7fdc5524b700 1 -- 172.21.15.27:0/364016422 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc4c005040 con 0x7fdc500f74a0 2021-11-03T18:04:24.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.294+0000 7fdc55a4c700 1 --2- 172.21.15.27:0/364016422 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc500f6b90 0x7fdc50104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:04:24.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.294+0000 7fdc5524b700 1 --2- 172.21.15.27:0/364016422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc500f74a0 0x7fdc50105260 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7fdc40002700 tx=0x7fdc40009e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:04:24.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.294+0000 7fdc467fc700 1 -- 172.21.15.27:0/364016422 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc40009320 con 0x7fdc500f74a0 2021-11-03T18:04:24.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.294+0000 7fdc467fc700 1 -- 172.21.15.27:0/364016422 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fdc4000f040 con 0x7fdc500f74a0 2021-11-03T18:04:24.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.294+0000 7fdc467fc700 1 -- 172.21.15.27:0/364016422 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc40014820 con 0x7fdc500f74a0 2021-11-03T18:04:24.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.294+0000 7fdc57cb0700 1 -- 172.21.15.27:0/364016422 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc501062f0 con 0x7fdc500f74a0 2021-11-03T18:04:24.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.294+0000 7fdc57cb0700 1 -- 172.21.15.27:0/364016422 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc501068b0 con 0x7fdc500f74a0 2021-11-03T18:04:24.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.295+0000 7fdc57cb0700 1 -- 172.21.15.27:0/364016422 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc500f41d0 con 0x7fdc500f74a0 2021-11-03T18:04:24.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.296+0000 7fdc467fc700 1 -- 172.21.15.27:0/364016422 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fdc40009480 con 0x7fdc500f74a0 2021-11-03T18:04:24.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.296+0000 7fdc467fc700 1 --2- 172.21.15.27:0/364016422 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdc3c0662c0 0x7fdc3c068780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:04:24.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.296+0000 7fdc467fc700 1 -- 172.21.15.27:0/364016422 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fdc40080a90 con 0x7fdc500f74a0 2021-11-03T18:04:24.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.296+0000 7fdc55a4c700 1 --2- 172.21.15.27:0/364016422 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdc3c0662c0 0x7fdc3c068780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:04:24.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.297+0000 7fdc55a4c700 1 --2- 172.21.15.27:0/364016422 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdc3c0662c0 0x7fdc3c068780 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7fdc4c004e30 tx=0x7fdc4c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:04:24.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.300+0000 7fdc467fc700 1 -- 172.21.15.27:0/364016422 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fdc40052d90 con 0x7fdc500f74a0 2021-11-03T18:04:24.472 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.475+0000 7fdc57cb0700 1 -- 172.21.15.27:0/364016422 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc500fce50 con 0x7fdc3c0662c0 2021-11-03T18:04:24.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.490+0000 7fdc467fc700 1 -- 172.21.15.27:0/364016422 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fdc500fce50 con 0x7fdc3c0662c0 2021-11-03T18:04:24.488 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:04:24.488 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (55m) 5m ago 57m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:04:24.489 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (47m) 5m ago 57m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:04:24.489 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (47m) 4m ago 56m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:04:24.490 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (55m) 5m ago 56m 41.8M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:04:24.490 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (49m) 5m ago 52m 26.0M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:04:24.490 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (52m) 5m ago 52m 3845M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:04:24.491 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (48m) 4m ago 52m 25.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:04:24.491 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (49m) 4m ago 52m 26.8M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:04:24.492 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (49m) 5m ago 58m 554M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:04:24.492 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (49m) 4m ago 56m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:04:24.492 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (48m) 5m ago 58m 235M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:04:24.493 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (48m) 4m ago 56m 112M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:04:24.493 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (56m) 5m ago 56m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:04:24.493 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (56m) 4m ago 56m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:04:24.494 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (47m) 5m ago 55m 351M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:04:24.494 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (46m) 5m ago 55m 383M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:04:24.495 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (46m) 5m ago 54m 284M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:04:24.495 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (45m) 4m ago 54m 404M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:04:24.495 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (45m) 4m ago 54m 322M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:04:24.496 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (44m) 4m ago 53m 327M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:04:24.496 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (55m) 5m ago 56m 55.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:04:24.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.492+0000 7fdc57cb0700 1 -- 172.21.15.27:0/364016422 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdc3c0662c0 msgr2=0x7fdc3c068780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:04:24.498 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.492+0000 7fdc57cb0700 1 --2- 172.21.15.27:0/364016422 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdc3c0662c0 0x7fdc3c068780 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7fdc4c004e30 tx=0x7fdc4c009b30).stop 2021-11-03T18:04:24.498 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.492+0000 7fdc57cb0700 1 -- 172.21.15.27:0/364016422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc500f74a0 msgr2=0x7fdc50105260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:04:24.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.492+0000 7fdc57cb0700 1 --2- 172.21.15.27:0/364016422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc500f74a0 0x7fdc50105260 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7fdc40002700 tx=0x7fdc40009e00).stop 2021-11-03T18:04:24.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.492+0000 7fdc57cb0700 1 -- 172.21.15.27:0/364016422 shutdown_connections 2021-11-03T18:04:24.500 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.492+0000 7fdc57cb0700 1 --2- 172.21.15.27:0/364016422 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdc3c0662c0 0x7fdc3c068780 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:24.500 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.492+0000 7fdc57cb0700 1 --2- 172.21.15.27:0/364016422 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdc500f6b90 0x7fdc50104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:24.500 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.492+0000 7fdc57cb0700 1 --2- 172.21.15.27:0/364016422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdc500f74a0 0x7fdc50105260 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:24.501 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.492+0000 7fdc57cb0700 1 -- 172.21.15.27:0/364016422 >> 172.21.15.27:0/364016422 conn(0x7fdc500f0540 msgr2=0x7fdc500fa390 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:04:24.501 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.492+0000 7fdc57cb0700 1 -- 172.21.15.27:0/364016422 shutdown_connections 2021-11-03T18:04:24.502 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.492+0000 7fdc57cb0700 1 -- 172.21.15.27:0/364016422 wait complete. 2021-11-03T18:04:24.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.592+0000 7f18f6fb7700 1 -- 172.21.15.27:0/3965462832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18f00f7960 msgr2=0x7f18f00f7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:04:24.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.592+0000 7f18f6fb7700 1 --2- 172.21.15.27:0/3965462832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18f00f7960 0x7f18f00f7dc0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7f18dc004660 tx=0x7f18dc009b30).stop 2021-11-03T18:04:24.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.592+0000 7f18f6fb7700 1 -- 172.21.15.27:0/3965462832 shutdown_connections 2021-11-03T18:04:24.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.592+0000 7f18f6fb7700 1 --2- 172.21.15.27:0/3965462832 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18f00f7960 0x7f18f00f7dc0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:24.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.592+0000 7f18f6fb7700 1 --2- 172.21.15.27:0/3965462832 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18f00f66c0 0x7f18f00f6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:24.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.592+0000 7f18f6fb7700 1 -- 172.21.15.27:0/3965462832 >> 172.21.15.27:0/3965462832 conn(0x7f18f00f1e20 msgr2=0x7f18f00f4240 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:04:24.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.592+0000 7f18f6fb7700 1 -- 172.21.15.27:0/3965462832 shutdown_connections 2021-11-03T18:04:24.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.592+0000 7f18f6fb7700 1 -- 172.21.15.27:0/3965462832 wait complete. 2021-11-03T18:04:24.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.592+0000 7f18f6fb7700 1 Processor -- start 2021-11-03T18:04:24.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.592+0000 7f18f6fb7700 1 -- start start 2021-11-03T18:04:24.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.593+0000 7f18f6fb7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18f00f66c0 0x7f18f006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:04:24.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.593+0000 7f18f6fb7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18f00f7960 0x7f18f0069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:04:24.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.593+0000 7f18f6fb7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f006a380 con 0x7f18f00f66c0 2021-11-03T18:04:24.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.593+0000 7f18f6fb7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18f006a4c0 con 0x7f18f00f7960 2021-11-03T18:04:24.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.593+0000 7f18f4d53700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18f00f66c0 0x7f18f006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:04:24.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.593+0000 7f18f4d53700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18f00f66c0 0x7f18f006b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32792/0 (socket says 172.21.15.27:32792) 2021-11-03T18:04:24.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.593+0000 7f18f4d53700 1 -- 172.21.15.27:0/1677069505 learned_addr learned my addr 172.21.15.27:0/1677069505 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:04:24.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.593+0000 7f18effff700 1 --2- 172.21.15.27:0/1677069505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18f00f7960 0x7f18f0069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:04:24.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.593+0000 7f18f4d53700 1 -- 172.21.15.27:0/1677069505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18f00f7960 msgr2=0x7f18f0069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:04:24.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.593+0000 7f18f4d53700 1 --2- 172.21.15.27:0/1677069505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18f00f7960 0x7f18f0069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:24.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.593+0000 7f18f4d53700 1 -- 172.21.15.27:0/1677069505 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18dc005040 con 0x7f18f00f66c0 2021-11-03T18:04:24.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.593+0000 7f18f4d53700 1 --2- 172.21.15.27:0/1677069505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18f00f66c0 0x7f18f006b9a0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7f18e4002700 tx=0x7f18e4009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:04:24.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.593+0000 7f18ed7fa700 1 -- 172.21.15.27:0/1677069505 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18e4009320 con 0x7f18f00f66c0 2021-11-03T18:04:24.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.593+0000 7f18ed7fa700 1 -- 172.21.15.27:0/1677069505 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f18e400f040 con 0x7f18f00f66c0 2021-11-03T18:04:24.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.593+0000 7f18f6fb7700 1 -- 172.21.15.27:0/1677069505 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18f006a7a0 con 0x7f18f00f66c0 2021-11-03T18:04:24.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.594+0000 7f18ed7fa700 1 -- 172.21.15.27:0/1677069505 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f18e4014900 con 0x7f18f00f66c0 2021-11-03T18:04:24.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.594+0000 7f18f6fb7700 1 -- 172.21.15.27:0/1677069505 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18f010ef80 con 0x7f18f00f66c0 2021-11-03T18:04:24.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.595+0000 7f18f6fb7700 1 -- 172.21.15.27:0/1677069505 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18f0045ad0 con 0x7f18f00f66c0 2021-11-03T18:04:24.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.595+0000 7f18ed7fa700 1 -- 172.21.15.27:0/1677069505 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f18e4009480 con 0x7f18f00f66c0 2021-11-03T18:04:24.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.595+0000 7f18ed7fa700 1 --2- 172.21.15.27:0/1677069505 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f18e00640b0 0x7f18e0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:04:24.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.596+0000 7f18ed7fa700 1 -- 172.21.15.27:0/1677069505 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f18e4080ab0 con 0x7f18f00f66c0 2021-11-03T18:04:24.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.596+0000 7f18effff700 1 --2- 172.21.15.27:0/1677069505 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f18e00640b0 0x7f18e0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:04:24.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.596+0000 7f18effff700 1 --2- 172.21.15.27:0/1677069505 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f18e00640b0 0x7f18e0066570 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f18dc004630 tx=0x7f18dc009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:04:24.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.600+0000 7f18ed7fa700 1 -- 172.21.15.27:0/1677069505 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f18e4052db0 con 0x7f18f00f66c0 2021-11-03T18:04:24.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:24 smithi115 ceph-mon[65453]: pgmap v1492: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:24.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:24 smithi027 ceph-mon[97244]: pgmap v1492: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:24.823 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.827+0000 7f18f6fb7700 1 -- 172.21.15.27:0/1677069505 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f18f00ff000 con 0x7f18f00f66c0 2021-11-03T18:04:24.824 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.827+0000 7f18ed7fa700 1 -- 172.21.15.27:0/1677069505 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f18e40529a0 con 0x7f18f00f66c0 2021-11-03T18:04:24.824 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:04:24.825 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:04:24.825 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:04:24.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:04:24.825 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:04:24.826 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:04:24.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:04:24.826 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:04:24.826 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:04:24.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:04:24.827 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:04:24.827 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:04:24.827 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:04:24.827 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:04:24.828 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:04:24.828 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:04:24.828 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:04:24.828 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:04:24.829 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.829+0000 7f18f6fb7700 1 -- 172.21.15.27:0/1677069505 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f18e00640b0 msgr2=0x7f18e0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:04:24.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.829+0000 7f18f6fb7700 1 --2- 172.21.15.27:0/1677069505 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f18e00640b0 0x7f18e0066570 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 rx=0x7f18dc004630 tx=0x7f18dc009b30).stop 2021-11-03T18:04:24.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.829+0000 7f18f6fb7700 1 -- 172.21.15.27:0/1677069505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18f00f66c0 msgr2=0x7f18f006b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:04:24.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.829+0000 7f18f6fb7700 1 --2- 172.21.15.27:0/1677069505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18f00f66c0 0x7f18f006b9a0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7f18e4002700 tx=0x7f18e4009d80).stop 2021-11-03T18:04:24.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.829+0000 7f18f6fb7700 1 -- 172.21.15.27:0/1677069505 shutdown_connections 2021-11-03T18:04:24.831 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.829+0000 7f18f6fb7700 1 --2- 172.21.15.27:0/1677069505 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f18e00640b0 0x7f18e0066570 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:24.831 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.829+0000 7f18f6fb7700 1 --2- 172.21.15.27:0/1677069505 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18f00f66c0 0x7f18f006b9a0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:24.831 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.829+0000 7f18f6fb7700 1 --2- 172.21.15.27:0/1677069505 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18f00f7960 0x7f18f0069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:24.831 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.829+0000 7f18f6fb7700 1 -- 172.21.15.27:0/1677069505 >> 172.21.15.27:0/1677069505 conn(0x7f18f00f1e20 msgr2=0x7f18f00fb3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:04:24.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.829+0000 7f18f6fb7700 1 -- 172.21.15.27:0/1677069505 shutdown_connections 2021-11-03T18:04:24.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.829+0000 7f18f6fb7700 1 -- 172.21.15.27:0/1677069505 wait complete. 2021-11-03T18:04:24.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.929+0000 7f27f799e700 1 -- 172.21.15.27:0/3532590405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27f00f55c0 msgr2=0x7f27f00f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:04:24.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.929+0000 7f27f799e700 1 --2- 172.21.15.27:0/3532590405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27f00f55c0 0x7f27f00f59e0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7f27e0004660 tx=0x7f27e0009b30).stop 2021-11-03T18:04:24.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.929+0000 7f27f799e700 1 -- 172.21.15.27:0/3532590405 shutdown_connections 2021-11-03T18:04:24.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.929+0000 7f27f799e700 1 --2- 172.21.15.27:0/3532590405 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27f00f6860 0x7f27f00f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:24.929 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.929+0000 7f27f799e700 1 --2- 172.21.15.27:0/3532590405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27f00f55c0 0x7f27f00f59e0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:24.929 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.929+0000 7f27f799e700 1 -- 172.21.15.27:0/3532590405 >> 172.21.15.27:0/3532590405 conn(0x7f27f00f0d20 msgr2=0x7f27f00f3140 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:04:24.929 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.929+0000 7f27f799e700 1 -- 172.21.15.27:0/3532590405 shutdown_connections 2021-11-03T18:04:24.929 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.929+0000 7f27f799e700 1 -- 172.21.15.27:0/3532590405 wait complete. 2021-11-03T18:04:24.930 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.929+0000 7f27f799e700 1 Processor -- start 2021-11-03T18:04:24.930 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.930+0000 7f27f799e700 1 -- start start 2021-11-03T18:04:24.930 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.930+0000 7f27f799e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27f00f55c0 0x7f27f0109cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:04:24.930 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.930+0000 7f27f799e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27f00f6860 0x7f27f010c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:04:24.931 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.930+0000 7f27f799e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f0060540 con 0x7f27f00f6860 2021-11-03T18:04:24.931 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.930+0000 7f27f799e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f0060680 con 0x7f27f00f55c0 2021-11-03T18:04:24.931 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.930+0000 7f27f4f39700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27f00f6860 0x7f27f010c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:04:24.931 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.930+0000 7f27f4f39700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27f00f6860 0x7f27f010c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32804/0 (socket says 172.21.15.27:32804) 2021-11-03T18:04:24.931 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.930+0000 7f27f4f39700 1 -- 172.21.15.27:0/258012451 learned_addr learned my addr 172.21.15.27:0/258012451 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:04:24.932 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.930+0000 7f27f573a700 1 --2- 172.21.15.27:0/258012451 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27f00f55c0 0x7f27f0109cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:04:24.932 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.930+0000 7f27f4f39700 1 -- 172.21.15.27:0/258012451 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27f00f55c0 msgr2=0x7f27f0109cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:04:24.932 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.930+0000 7f27f4f39700 1 --2- 172.21.15.27:0/258012451 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27f00f55c0 0x7f27f0109cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:24.932 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.930+0000 7f27f4f39700 1 -- 172.21.15.27:0/258012451 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e0005040 con 0x7f27f00f6860 2021-11-03T18:04:24.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.930+0000 7f27f573a700 1 --2- 172.21.15.27:0/258012451 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27f00f55c0 0x7f27f0109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T18:04:24.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.930+0000 7f27f4f39700 1 --2- 172.21.15.27:0/258012451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27f00f6860 0x7f27f010c1c0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7f27ec0026d0 tx=0x7f27ec009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:04:24.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.930+0000 7f27e5ffb700 1 -- 172.21.15.27:0/258012451 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27ec009320 con 0x7f27f00f6860 2021-11-03T18:04:24.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.931+0000 7f27e5ffb700 1 -- 172.21.15.27:0/258012451 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f27ec00f040 con 0x7f27f00f6860 2021-11-03T18:04:24.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.931+0000 7f27f799e700 1 -- 172.21.15.27:0/258012451 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27f010c770 con 0x7f27f00f6860 2021-11-03T18:04:24.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.931+0000 7f27e5ffb700 1 -- 172.21.15.27:0/258012451 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f27ec0147e0 con 0x7f27f00f6860 2021-11-03T18:04:24.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.931+0000 7f27f799e700 1 -- 172.21.15.27:0/258012451 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27f010cd30 con 0x7f27f00f6860 2021-11-03T18:04:24.935 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.932+0000 7f27f799e700 1 -- 172.21.15.27:0/258012451 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27f0045ad0 con 0x7f27f00f6860 2021-11-03T18:04:24.935 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.932+0000 7f27e5ffb700 1 -- 172.21.15.27:0/258012451 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f27ec009480 con 0x7f27f00f6860 2021-11-03T18:04:24.936 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.933+0000 7f27e5ffb700 1 --2- 172.21.15.27:0/258012451 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f27dc0640b0 0x7f27dc066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:04:24.936 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.933+0000 7f27e5ffb700 1 -- 172.21.15.27:0/258012451 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f27ec081000 con 0x7f27f00f6860 2021-11-03T18:04:24.936 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.933+0000 7f27f573a700 1 --2- 172.21.15.27:0/258012451 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f27dc0640b0 0x7f27dc066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:04:24.936 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.933+0000 7f27f573a700 1 --2- 172.21.15.27:0/258012451 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f27dc0640b0 0x7f27dc066570 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f27e0004e30 tx=0x7f27e0009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:04:24.937 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:24.937+0000 7f27e5ffb700 1 -- 172.21.15.27:0/258012451 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f27ec0addf0 con 0x7f27f00f6860 2021-11-03T18:04:25.143 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:25.146+0000 7f27f799e700 1 -- 172.21.15.27:0/258012451 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f27f0027a00 con 0x7f27f00f6860 2021-11-03T18:04:25.143 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:25.146+0000 7f27e5ffb700 1 -- 172.21.15.27:0/258012451 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f27ec053240 con 0x7f27f00f6860 2021-11-03T18:04:25.143 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:04:25.144 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:04:25.144 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:04:25.144 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:04:25.144 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:04:25.145 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:04:25.145 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:04:25.145 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:04:25.145 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:04:25.146 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:04:25.146 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:04:25.146 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:04:25.146 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:04:25.147 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:04:25.147 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:04:25.147 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:04:25.147 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:04:25.147 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:04:25.148 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:04:25.148 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:04:25.148 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:04:25.148 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:04:25.149 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:04:25.149 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:04:25.149 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:04:25.149 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:04:25.149 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:04:25.150 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:04:25.150 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:04:25.150 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:04:25.150 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:04:25.151 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:04:25.151 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:04:25.151 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:04:25.151 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:04:25.152 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:04:25.152 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:04:25.152 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:04:25.152 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:04:25.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:25.148+0000 7f27f799e700 1 -- 172.21.15.27:0/258012451 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f27dc0640b0 msgr2=0x7f27dc066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:04:25.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:25.148+0000 7f27f799e700 1 --2- 172.21.15.27:0/258012451 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f27dc0640b0 0x7f27dc066570 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f27e0004e30 tx=0x7f27e0009b30).stop 2021-11-03T18:04:25.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:25.148+0000 7f27f799e700 1 -- 172.21.15.27:0/258012451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27f00f6860 msgr2=0x7f27f010c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:04:25.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:25.148+0000 7f27f799e700 1 --2- 172.21.15.27:0/258012451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27f00f6860 0x7f27f010c1c0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7f27ec0026d0 tx=0x7f27ec009de0).stop 2021-11-03T18:04:25.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:25.148+0000 7f27f799e700 1 -- 172.21.15.27:0/258012451 shutdown_connections 2021-11-03T18:04:25.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:25.148+0000 7f27f799e700 1 --2- 172.21.15.27:0/258012451 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f27dc0640b0 0x7f27dc066570 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:25.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:25.148+0000 7f27f799e700 1 --2- 172.21.15.27:0/258012451 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f27f00f55c0 0x7f27f0109cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:25.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:25.148+0000 7f27f799e700 1 --2- 172.21.15.27:0/258012451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f27f00f6860 0x7f27f010c1c0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:25.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:25.148+0000 7f27f799e700 1 -- 172.21.15.27:0/258012451 >> 172.21.15.27:0/258012451 conn(0x7f27f00f0d20 msgr2=0x7f27f00fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:04:25.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:25.148+0000 7f27f799e700 1 -- 172.21.15.27:0/258012451 shutdown_connections 2021-11-03T18:04:25.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:25.148+0000 7f27f799e700 1 -- 172.21.15.27:0/258012451 wait complete. 2021-11-03T18:04:25.156 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:04:25.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:25 smithi115 ceph-mon[65453]: from='client.35640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:04:25.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2907.429424 secs 2021-11-03T18:04:25.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:25 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1677069505' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:25.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:25 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/258012451' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:04:25.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:25 smithi027 ceph-mon[97244]: from='client.35640 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:04:25.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2907.429424 secs 2021-11-03T18:04:25.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:25 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1677069505' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:25.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:25 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/258012451' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:04:26.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:26 smithi115 ceph-mon[65453]: from='client.35644 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:04:26.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:26 smithi115 ceph-mon[65453]: pgmap v1493: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:26.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:26 smithi027 ceph-mon[97244]: from='client.35644 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:04:26.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:26 smithi027 ceph-mon[97244]: pgmap v1493: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:28.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:28 smithi115 ceph-mon[65453]: pgmap v1494: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:28.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:28 smithi027 ceph-mon[97244]: pgmap v1494: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:29.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:04:29.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:04:29.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:04:29.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:29.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:29.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:29.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:29.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:29 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:04:29.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:04:29.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:04:29.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:04:29.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:29.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:29.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:29.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:29.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:29 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:04:30.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:30 smithi115 ceph-mon[65453]: pgmap v1495: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:30.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2912.429568 secs 2021-11-03T18:04:30.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:30 smithi027 ceph-mon[97244]: pgmap v1495: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:30.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2912.429568 secs 2021-11-03T18:04:32.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:32 smithi115 ceph-mon[65453]: pgmap v1496: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:32.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:32 smithi027 ceph-mon[97244]: pgmap v1496: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:34.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:34 smithi115 ceph-mon[65453]: pgmap v1497: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:34.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:34 smithi027 ceph-mon[97244]: pgmap v1497: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:35.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2917.429693 secs 2021-11-03T18:04:35.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2917.429693 secs 2021-11-03T18:04:36.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:36 smithi115 ceph-mon[65453]: pgmap v1498: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:36.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:36 smithi027 ceph-mon[97244]: pgmap v1498: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:38.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:38 smithi115 ceph-mon[65453]: pgmap v1499: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:38.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:38 smithi027 ceph-mon[97244]: pgmap v1499: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:39.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:04:39.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:04:39.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:04:39.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:39.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:39.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:39.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:39.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:04:39.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:04:39.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:04:39.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:39.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:39.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:39.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:40.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:40 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:04:40.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:40 smithi115 ceph-mon[65453]: pgmap v1500: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:40.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2922.429831 secs 2021-11-03T18:04:40.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:40 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:04:40.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:40 smithi027 ceph-mon[97244]: pgmap v1500: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:40.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2922.429831 secs 2021-11-03T18:04:42.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:42 smithi115 ceph-mon[65453]: pgmap v1501: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:42.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:42 smithi027 ceph-mon[97244]: pgmap v1501: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:44.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:44 smithi115 ceph-mon[65453]: pgmap v1502: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:44.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:44 smithi027 ceph-mon[97244]: pgmap v1502: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:45.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2927.429949 secs 2021-11-03T18:04:45.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2927.429949 secs 2021-11-03T18:04:46.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:46 smithi115 ceph-mon[65453]: pgmap v1503: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:46.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:46 smithi027 ceph-mon[97244]: pgmap v1503: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:47.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:47 smithi115 ceph-mon[65453]: pgmap v1504: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:48.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:47 smithi027 ceph-mon[97244]: pgmap v1504: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:48.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:04:48.829 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:04:50.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:49 smithi115 ceph-mon[65453]: poll took 60173.760 ms: 1 events 2021-11-03T18:04:50.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:49 smithi115 ceph-mon[65453]: pgmap v1505: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:50.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:04:50.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:04:50.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:50.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:50.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:50.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:50.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:49 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:04:50.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:49 smithi027 ceph-mon[97244]: poll took 60173.760 ms: 1 events 2021-11-03T18:04:50.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:49 smithi027 ceph-mon[97244]: pgmap v1505: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:50.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:04:50.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:04:50.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:50.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:50.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:50.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:50.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:49 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:04:51.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2932.430090 secs 2021-11-03T18:04:51.270 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2932.430090 secs 2021-11-03T18:04:52.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:52 smithi115 ceph-mon[65453]: pgmap v1506: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:52.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:52 smithi027 ceph-mon[97244]: pgmap v1506: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:54.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:54 smithi115 ceph-mon[65453]: pgmap v1507: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:54.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:54 smithi027 ceph-mon[97244]: pgmap v1507: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:55.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.263+0000 7ff4a18c3700 1 -- 172.21.15.27:0/4163924666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff49c0f66e0 msgr2=0x7ff49c0f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:04:55.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.263+0000 7ff4a18c3700 1 --2- 172.21.15.27:0/4163924666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff49c0f66e0 0x7ff49c0f6b00 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 rx=0x7ff490004660 tx=0x7ff490009b30).stop 2021-11-03T18:04:55.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.263+0000 7ff4a18c3700 1 -- 172.21.15.27:0/4163924666 shutdown_connections 2021-11-03T18:04:55.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.263+0000 7ff4a18c3700 1 --2- 172.21.15.27:0/4163924666 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff49c0f6ff0 0x7ff49c0f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:55.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.263+0000 7ff4a18c3700 1 --2- 172.21.15.27:0/4163924666 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff49c0f66e0 0x7ff49c0f6b00 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:55.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.263+0000 7ff4a18c3700 1 -- 172.21.15.27:0/4163924666 >> 172.21.15.27:0/4163924666 conn(0x7ff49c0f1f00 msgr2=0x7ff49c0f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:04:55.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.263+0000 7ff4a18c3700 1 -- 172.21.15.27:0/4163924666 shutdown_connections 2021-11-03T18:04:55.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.263+0000 7ff4a18c3700 1 -- 172.21.15.27:0/4163924666 wait complete. 2021-11-03T18:04:55.265 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.264+0000 7ff4a18c3700 1 Processor -- start 2021-11-03T18:04:55.265 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.264+0000 7ff4a18c3700 1 -- start start 2021-11-03T18:04:55.265 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.264+0000 7ff4a18c3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff49c0f66e0 0x7ff49c106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:04:55.265 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.264+0000 7ff4a18c3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff49c0f6ff0 0x7ff49c107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:04:55.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.264+0000 7ff4a18c3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c107a00 con 0x7ff49c0f6ff0 2021-11-03T18:04:55.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.264+0000 7ff4a18c3700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c107b40 con 0x7ff49c0f66e0 2021-11-03T18:04:55.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.265+0000 7ff49a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff49c0f6ff0 0x7ff49c107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:04:55.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.265+0000 7ff49a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff49c0f6ff0 0x7ff49c107450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32822/0 (socket says 172.21.15.27:32822) 2021-11-03T18:04:55.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.265+0000 7ff49a7fc700 1 -- 172.21.15.27:0/1793302454 learned_addr learned my addr 172.21.15.27:0/1793302454 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:04:55.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.265+0000 7ff49affd700 1 --2- 172.21.15.27:0/1793302454 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff49c0f66e0 0x7ff49c106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:04:55.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.265+0000 7ff49a7fc700 1 -- 172.21.15.27:0/1793302454 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff49c0f66e0 msgr2=0x7ff49c106f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:04:55.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.265+0000 7ff49a7fc700 1 --2- 172.21.15.27:0/1793302454 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff49c0f66e0 0x7ff49c106f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:55.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.265+0000 7ff49a7fc700 1 -- 172.21.15.27:0/1793302454 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff490005040 con 0x7ff49c0f6ff0 2021-11-03T18:04:55.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.265+0000 7ff49a7fc700 1 --2- 172.21.15.27:0/1793302454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff49c0f6ff0 0x7ff49c107450 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7ff48c0026d0 tx=0x7ff48c009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:04:55.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.265+0000 7ff4837fe700 1 -- 172.21.15.27:0/1793302454 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff48c0079c0 con 0x7ff49c0f6ff0 2021-11-03T18:04:55.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.265+0000 7ff4a18c3700 1 -- 172.21.15.27:0/1793302454 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff49c1084e0 con 0x7ff49c0f6ff0 2021-11-03T18:04:55.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.265+0000 7ff4837fe700 1 -- 172.21.15.27:0/1793302454 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7ff48c00f040 con 0x7ff49c0f6ff0 2021-11-03T18:04:55.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.265+0000 7ff4837fe700 1 -- 172.21.15.27:0/1793302454 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff48c0148b0 con 0x7ff49c0f6ff0 2021-11-03T18:04:55.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.265+0000 7ff4a18c3700 1 -- 172.21.15.27:0/1793302454 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff49c108aa0 con 0x7ff49c0f6ff0 2021-11-03T18:04:55.270 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.266+0000 7ff4a18c3700 1 -- 172.21.15.27:0/1793302454 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff49c045ad0 con 0x7ff49c0f6ff0 2021-11-03T18:04:55.270 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.267+0000 7ff4837fe700 1 -- 172.21.15.27:0/1793302454 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7ff48c007b20 con 0x7ff49c0f6ff0 2021-11-03T18:04:55.270 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.267+0000 7ff4837fe700 1 --2- 172.21.15.27:0/1793302454 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff4840640b0 0x7ff484066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:04:55.271 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.267+0000 7ff4837fe700 1 -- 172.21.15.27:0/1793302454 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7ff48c0807a0 con 0x7ff49c0f6ff0 2021-11-03T18:04:55.271 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.267+0000 7ff49affd700 1 --2- 172.21.15.27:0/1793302454 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff4840640b0 0x7ff484066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:04:55.271 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.268+0000 7ff49affd700 1 --2- 172.21.15.27:0/1793302454 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff4840640b0 0x7ff484066570 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7ff490004630 tx=0x7ff490009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:04:55.271 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.271+0000 7ff4837fe700 1 -- 172.21.15.27:0/1793302454 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7ff48c052aa0 con 0x7ff49c0f6ff0 2021-11-03T18:04:55.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.448+0000 7ff4a18c3700 1 -- 172.21.15.27:0/1793302454 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff49c0ff010 con 0x7ff4840640b0 2021-11-03T18:04:55.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.449+0000 7ff4837fe700 1 -- 172.21.15.27:0/1793302454 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7ff49c0ff010 con 0x7ff4840640b0 2021-11-03T18:04:55.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.451+0000 7ff4a18c3700 1 -- 172.21.15.27:0/1793302454 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff4840640b0 msgr2=0x7ff484066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:04:55.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.451+0000 7ff4a18c3700 1 --2- 172.21.15.27:0/1793302454 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff4840640b0 0x7ff484066570 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7ff490004630 tx=0x7ff490009b30).stop 2021-11-03T18:04:55.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.451+0000 7ff4a18c3700 1 -- 172.21.15.27:0/1793302454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff49c0f6ff0 msgr2=0x7ff49c107450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:04:55.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.451+0000 7ff4a18c3700 1 --2- 172.21.15.27:0/1793302454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff49c0f6ff0 0x7ff49c107450 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7ff48c0026d0 tx=0x7ff48c009de0).stop 2021-11-03T18:04:55.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.451+0000 7ff4a18c3700 1 -- 172.21.15.27:0/1793302454 shutdown_connections 2021-11-03T18:04:55.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.451+0000 7ff4a18c3700 1 --2- 172.21.15.27:0/1793302454 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff4840640b0 0x7ff484066570 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:55.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.451+0000 7ff4a18c3700 1 --2- 172.21.15.27:0/1793302454 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff49c0f66e0 0x7ff49c106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:55.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.451+0000 7ff4a18c3700 1 --2- 172.21.15.27:0/1793302454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff49c0f6ff0 0x7ff49c107450 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:55.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.451+0000 7ff4a18c3700 1 -- 172.21.15.27:0/1793302454 >> 172.21.15.27:0/1793302454 conn(0x7ff49c0f1f00 msgr2=0x7ff49c0fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:04:55.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.451+0000 7ff4a18c3700 1 -- 172.21.15.27:0/1793302454 shutdown_connections 2021-11-03T18:04:55.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.451+0000 7ff4a18c3700 1 -- 172.21.15.27:0/1793302454 wait complete. 2021-11-03T18:04:55.457 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:04:55.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2937.430222 secs 2021-11-03T18:04:55.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2937.430222 secs 2021-11-03T18:04:55.548 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.551+0000 7f0966c20700 1 -- 172.21.15.27:0/4107843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f096006a3b0 msgr2=0x7f096006a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:04:55.548 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.551+0000 7f0966c20700 1 --2- 172.21.15.27:0/4107843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f096006a3b0 0x7f096006a830 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7f094c004660 tx=0x7f094c009b30).stop 2021-11-03T18:04:55.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.551+0000 7f0966c20700 1 -- 172.21.15.27:0/4107843 shutdown_connections 2021-11-03T18:04:55.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.551+0000 7f0966c20700 1 --2- 172.21.15.27:0/4107843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f096006a3b0 0x7f096006a830 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:55.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.551+0000 7f0966c20700 1 --2- 172.21.15.27:0/4107843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f096006b920 0x7f0960069ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:55.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.551+0000 7f0966c20700 1 -- 172.21.15.27:0/4107843 >> 172.21.15.27:0/4107843 conn(0x7f09600f1f00 msgr2=0x7f09600f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:04:55.550 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.552+0000 7f0966c20700 1 -- 172.21.15.27:0/4107843 shutdown_connections 2021-11-03T18:04:55.550 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.552+0000 7f0966c20700 1 -- 172.21.15.27:0/4107843 wait complete. 2021-11-03T18:04:55.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.552+0000 7f0966c20700 1 Processor -- start 2021-11-03T18:04:55.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.552+0000 7f0966c20700 1 -- start start 2021-11-03T18:04:55.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.552+0000 7f0966c20700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f096006a3b0 0x7f096010d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:04:55.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.552+0000 7f0966c20700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f096006b920 0x7f096010da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:04:55.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.552+0000 7f0966c20700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096010e030 con 0x7f096006b920 2021-11-03T18:04:55.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.552+0000 7f0966c20700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096010e170 con 0x7f096006a3b0 2021-11-03T18:04:55.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.553+0000 7f095ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f096006b920 0x7f096010da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:04:55.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.553+0000 7f095ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f096006b920 0x7f096010da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32832/0 (socket says 172.21.15.27:32832) 2021-11-03T18:04:55.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.553+0000 7f095ffff700 1 -- 172.21.15.27:0/596863639 learned_addr learned my addr 172.21.15.27:0/596863639 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:04:55.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.553+0000 7f09649bc700 1 --2- 172.21.15.27:0/596863639 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f096006a3b0 0x7f096010d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:04:55.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.553+0000 7f095ffff700 1 -- 172.21.15.27:0/596863639 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f096006a3b0 msgr2=0x7f096010d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:04:55.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.553+0000 7f095ffff700 1 --2- 172.21.15.27:0/596863639 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f096006a3b0 0x7f096010d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:55.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.553+0000 7f095ffff700 1 -- 172.21.15.27:0/596863639 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f094c005040 con 0x7f096006b920 2021-11-03T18:04:55.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.553+0000 7f09649bc700 1 --2- 172.21.15.27:0/596863639 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f096006a3b0 0x7f096010d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:04:55.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.553+0000 7f095ffff700 1 --2- 172.21.15.27:0/596863639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f096006b920 0x7f096010da80 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7f094c009990 tx=0x7f094c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:04:55.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.553+0000 7f095d7fa700 1 -- 172.21.15.27:0/596863639 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f094c018070 con 0x7f096006b920 2021-11-03T18:04:55.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.553+0000 7f095d7fa700 1 -- 172.21.15.27:0/596863639 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f094c007ca0 con 0x7f096006b920 2021-11-03T18:04:55.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.553+0000 7f0966c20700 1 -- 172.21.15.27:0/596863639 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f096010eab0 con 0x7f096006b920 2021-11-03T18:04:55.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.553+0000 7f095d7fa700 1 -- 172.21.15.27:0/596863639 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f094c01c600 con 0x7f096006b920 2021-11-03T18:04:55.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.553+0000 7f0966c20700 1 -- 172.21.15.27:0/596863639 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f096010f010 con 0x7f096006b920 2021-11-03T18:04:55.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.554+0000 7f0966c20700 1 -- 172.21.15.27:0/596863639 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0960045ad0 con 0x7f096006b920 2021-11-03T18:04:55.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.555+0000 7f095d7fa700 1 -- 172.21.15.27:0/596863639 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f094c01d9d0 con 0x7f096006b920 2021-11-03T18:04:55.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.555+0000 7f095d7fa700 1 --2- 172.21.15.27:0/596863639 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f09500640b0 0x7f0950066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:04:55.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.555+0000 7f095d7fa700 1 -- 172.21.15.27:0/596863639 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f094c0825f0 con 0x7f096006b920 2021-11-03T18:04:55.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.555+0000 7f09649bc700 1 --2- 172.21.15.27:0/596863639 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f09500640b0 0x7f0950066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:04:55.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.556+0000 7f09649bc700 1 --2- 172.21.15.27:0/596863639 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f09500640b0 0x7f0950066570 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f096006b6f0 tx=0x7f0954009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:04:55.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.559+0000 7f095d7fa700 1 -- 172.21.15.27:0/596863639 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f094c0549b0 con 0x7f096006b920 2021-11-03T18:04:55.728 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.731+0000 7f0966c20700 1 -- 172.21.15.27:0/596863639 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0960105610 con 0x7f09500640b0 2021-11-03T18:04:55.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.746+0000 7f095d7fa700 1 -- 172.21.15.27:0/596863639 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f0960105610 con 0x7f09500640b0 2021-11-03T18:04:55.743 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:04:55.744 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (56m) 5m ago 57m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:04:55.744 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (48m) 5m ago 57m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:04:55.744 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (48m) 4m ago 56m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:04:55.744 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (56m) 5m ago 57m 41.8M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:04:55.745 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (49m) 5m ago 52m 26.0M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:04:55.745 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (52m) 5m ago 52m 3845M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:04:55.745 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (48m) 4m ago 52m 25.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:04:55.745 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (50m) 4m ago 53m 26.8M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:04:55.746 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (50m) 5m ago 59m 554M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:04:55.746 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (49m) 4m ago 56m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:04:55.746 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (49m) 5m ago 59m 235M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:04:55.746 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (48m) 4m ago 56m 112M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:04:55.747 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (57m) 5m ago 57m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:04:55.748 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (56m) 4m ago 56m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:04:55.748 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (47m) 5m ago 56m 351M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:04:55.748 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (47m) 5m ago 55m 383M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:04:55.748 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (46m) 5m ago 55m 284M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:04:55.749 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (46m) 4m ago 55m 404M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:04:55.750 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (45m) 4m ago 54m 322M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:04:55.750 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (45m) 4m ago 54m 327M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:04:55.750 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (56m) 5m ago 57m 55.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:04:55.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.748+0000 7f0966c20700 1 -- 172.21.15.27:0/596863639 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f09500640b0 msgr2=0x7f0950066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:04:55.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.748+0000 7f0966c20700 1 --2- 172.21.15.27:0/596863639 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f09500640b0 0x7f0950066570 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7f096006b6f0 tx=0x7f0954009040).stop 2021-11-03T18:04:55.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.748+0000 7f0966c20700 1 -- 172.21.15.27:0/596863639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f096006b920 msgr2=0x7f096010da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:04:55.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.748+0000 7f0966c20700 1 --2- 172.21.15.27:0/596863639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f096006b920 0x7f096010da80 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7f094c009990 tx=0x7f094c009b30).stop 2021-11-03T18:04:55.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.748+0000 7f0966c20700 1 -- 172.21.15.27:0/596863639 shutdown_connections 2021-11-03T18:04:55.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.748+0000 7f0966c20700 1 --2- 172.21.15.27:0/596863639 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f09500640b0 0x7f0950066570 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:55.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.748+0000 7f0966c20700 1 --2- 172.21.15.27:0/596863639 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f096006a3b0 0x7f096010d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:55.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.748+0000 7f0966c20700 1 --2- 172.21.15.27:0/596863639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f096006b920 0x7f096010da80 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:55.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.748+0000 7f0966c20700 1 -- 172.21.15.27:0/596863639 >> 172.21.15.27:0/596863639 conn(0x7f09600f1f00 msgr2=0x7f09600f8960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:04:55.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.748+0000 7f0966c20700 1 -- 172.21.15.27:0/596863639 shutdown_connections 2021-11-03T18:04:55.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.748+0000 7f0966c20700 1 -- 172.21.15.27:0/596863639 wait complete. 2021-11-03T18:04:55.847 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.849+0000 7ff9c6e33700 1 -- 172.21.15.27:0/2362225449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9c00fba00 msgr2=0x7ff9c00fddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:04:55.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.849+0000 7ff9c6e33700 1 --2- 172.21.15.27:0/2362225449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9c00fba00 0x7ff9c00fddf0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 rx=0x7ff9b0004660 tx=0x7ff9b0009b30).stop 2021-11-03T18:04:55.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.849+0000 7ff9c6e33700 1 -- 172.21.15.27:0/2362225449 shutdown_connections 2021-11-03T18:04:55.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.849+0000 7ff9c6e33700 1 --2- 172.21.15.27:0/2362225449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9c00fba00 0x7ff9c00fddf0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:55.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.849+0000 7ff9c6e33700 1 --2- 172.21.15.27:0/2362225449 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9c0060310 0x7ff9c00fb510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:55.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.849+0000 7ff9c6e33700 1 -- 172.21.15.27:0/2362225449 >> 172.21.15.27:0/2362225449 conn(0x7ff9c00f0b90 msgr2=0x7ff9c00f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:04:55.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.849+0000 7ff9c6e33700 1 -- 172.21.15.27:0/2362225449 shutdown_connections 2021-11-03T18:04:55.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.849+0000 7ff9c6e33700 1 -- 172.21.15.27:0/2362225449 wait complete. 2021-11-03T18:04:55.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.849+0000 7ff9c6e33700 1 Processor -- start 2021-11-03T18:04:55.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.849+0000 7ff9c6e33700 1 -- start start 2021-11-03T18:04:55.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.849+0000 7ff9c6e33700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9c0060310 0x7ff9c0108f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:04:55.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.849+0000 7ff9c6e33700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9c00fba00 0x7ff9c0109410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:04:55.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.849+0000 7ff9c6e33700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c01099c0 con 0x7ff9c0060310 2021-11-03T18:04:55.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.849+0000 7ff9c6e33700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c0109b00 con 0x7ff9c00fba00 2021-11-03T18:04:55.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.850+0000 7ff9c5e31700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9c0060310 0x7ff9c0108f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:04:55.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.850+0000 7ff9c5e31700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9c0060310 0x7ff9c0108f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32840/0 (socket says 172.21.15.27:32840) 2021-11-03T18:04:55.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.850+0000 7ff9c5e31700 1 -- 172.21.15.27:0/380734864 learned_addr learned my addr 172.21.15.27:0/380734864 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:04:55.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.850+0000 7ff9c5630700 1 --2- 172.21.15.27:0/380734864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9c00fba00 0x7ff9c0109410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:04:55.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.850+0000 7ff9c5e31700 1 -- 172.21.15.27:0/380734864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9c00fba00 msgr2=0x7ff9c0109410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:04:55.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.850+0000 7ff9c5e31700 1 --2- 172.21.15.27:0/380734864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9c00fba00 0x7ff9c0109410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:55.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.850+0000 7ff9c5e31700 1 -- 172.21.15.27:0/380734864 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b0005040 con 0x7ff9c0060310 2021-11-03T18:04:55.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.850+0000 7ff9c5e31700 1 --2- 172.21.15.27:0/380734864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9c0060310 0x7ff9c0108f20 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7ff9bc0026d0 tx=0x7ff9bc009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:04:55.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.850+0000 7ff9b67fc700 1 -- 172.21.15.27:0/380734864 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9bc0079c0 con 0x7ff9c0060310 2021-11-03T18:04:55.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.850+0000 7ff9b67fc700 1 -- 172.21.15.27:0/380734864 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7ff9bc00f040 con 0x7ff9c0060310 2021-11-03T18:04:55.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.850+0000 7ff9c6e33700 1 -- 172.21.15.27:0/380734864 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9c010a4a0 con 0x7ff9c0060310 2021-11-03T18:04:55.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.850+0000 7ff9b67fc700 1 -- 172.21.15.27:0/380734864 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9bc0148b0 con 0x7ff9c0060310 2021-11-03T18:04:55.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.850+0000 7ff9c6e33700 1 -- 172.21.15.27:0/380734864 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9c010aa60 con 0x7ff9c0060310 2021-11-03T18:04:55.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.851+0000 7ff9c6e33700 1 -- 172.21.15.27:0/380734864 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9c0045ad0 con 0x7ff9c0060310 2021-11-03T18:04:55.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.852+0000 7ff9b67fc700 1 -- 172.21.15.27:0/380734864 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7ff9bc007b20 con 0x7ff9c0060310 2021-11-03T18:04:55.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.852+0000 7ff9b67fc700 1 --2- 172.21.15.27:0/380734864 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff9ac064100 0x7ff9ac0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:04:55.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.852+0000 7ff9b67fc700 1 -- 172.21.15.27:0/380734864 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7ff9bc080790 con 0x7ff9c0060310 2021-11-03T18:04:55.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.852+0000 7ff9c5630700 1 --2- 172.21.15.27:0/380734864 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff9ac064100 0x7ff9ac0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:04:55.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.852+0000 7ff9c5630700 1 --2- 172.21.15.27:0/380734864 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff9ac064100 0x7ff9ac0665c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7ff9b0005010 tx=0x7ff9b0009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:04:55.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:55.856+0000 7ff9b67fc700 1 -- 172.21.15.27:0/380734864 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7ff9bc052a90 con 0x7ff9c0060310 2021-11-03T18:04:56.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.091+0000 7ff9c6e33700 1 -- 172.21.15.27:0/380734864 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff9c0027a00 con 0x7ff9c0060310 2021-11-03T18:04:56.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.092+0000 7ff9b67fc700 1 -- 172.21.15.27:0/380734864 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7ff9bc052680 con 0x7ff9c0060310 2021-11-03T18:04:56.089 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:04:56.089 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:04:56.089 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:04:56.090 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:04:56.090 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:04:56.090 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:04:56.090 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:04:56.091 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:04:56.091 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:04:56.091 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:04:56.091 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:04:56.092 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:04:56.092 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:04:56.093 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:04:56.093 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:04:56.094 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:04:56.094 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:04:56.094 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:04:56.096 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.094+0000 7ff9c6e33700 1 -- 172.21.15.27:0/380734864 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff9ac064100 msgr2=0x7ff9ac0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:04:56.097 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.094+0000 7ff9c6e33700 1 --2- 172.21.15.27:0/380734864 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff9ac064100 0x7ff9ac0665c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 rx=0x7ff9b0005010 tx=0x7ff9b0009b30).stop 2021-11-03T18:04:56.097 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.094+0000 7ff9c6e33700 1 -- 172.21.15.27:0/380734864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9c0060310 msgr2=0x7ff9c0108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:04:56.098 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.094+0000 7ff9c6e33700 1 --2- 172.21.15.27:0/380734864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9c0060310 0x7ff9c0108f20 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7ff9bc0026d0 tx=0x7ff9bc009de0).stop 2021-11-03T18:04:56.098 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.094+0000 7ff9c6e33700 1 -- 172.21.15.27:0/380734864 shutdown_connections 2021-11-03T18:04:56.099 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.094+0000 7ff9c6e33700 1 --2- 172.21.15.27:0/380734864 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff9ac064100 0x7ff9ac0665c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:56.099 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.094+0000 7ff9c6e33700 1 --2- 172.21.15.27:0/380734864 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9c0060310 0x7ff9c0108f20 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:56.100 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.094+0000 7ff9c6e33700 1 --2- 172.21.15.27:0/380734864 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9c00fba00 0x7ff9c0109410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:56.100 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.094+0000 7ff9c6e33700 1 -- 172.21.15.27:0/380734864 >> 172.21.15.27:0/380734864 conn(0x7ff9c00f0b90 msgr2=0x7ff9c00f2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:04:56.100 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.094+0000 7ff9c6e33700 1 -- 172.21.15.27:0/380734864 shutdown_connections 2021-11-03T18:04:56.101 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.094+0000 7ff9c6e33700 1 -- 172.21.15.27:0/380734864 wait complete. 2021-11-03T18:04:56.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.193+0000 7f695c878700 1 -- 172.21.15.27:0/3814980380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69580f6530 msgr2=0x7f69580f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:04:56.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.193+0000 7f695c878700 1 --2- 172.21.15.27:0/3814980380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69580f6530 0x7f69580f6950 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0x7f6948004660 tx=0x7f6948009b30).stop 2021-11-03T18:04:56.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.193+0000 7f695c878700 1 -- 172.21.15.27:0/3814980380 shutdown_connections 2021-11-03T18:04:56.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.193+0000 7f695c878700 1 --2- 172.21.15.27:0/3814980380 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69580f77d0 0x7f69580f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:56.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.193+0000 7f695c878700 1 --2- 172.21.15.27:0/3814980380 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69580f6530 0x7f69580f6950 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:56.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.193+0000 7f695c878700 1 -- 172.21.15.27:0/3814980380 >> 172.21.15.27:0/3814980380 conn(0x7f69580f1c90 msgr2=0x7f69580f40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:04:56.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.193+0000 7f695c878700 1 -- 172.21.15.27:0/3814980380 shutdown_connections 2021-11-03T18:04:56.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.193+0000 7f695c878700 1 -- 172.21.15.27:0/3814980380 wait complete. 2021-11-03T18:04:56.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.194+0000 7f695c878700 1 Processor -- start 2021-11-03T18:04:56.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.194+0000 7f695c878700 1 -- start start 2021-11-03T18:04:56.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.194+0000 7f695c878700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69580f6530 0x7f6958106e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:04:56.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.194+0000 7f695c878700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69580f77d0 0x7f69581072f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:04:56.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.194+0000 7f695c878700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69581078f0 con 0x7f69580f77d0 2021-11-03T18:04:56.196 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.194+0000 7f695c878700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6958108100 con 0x7f69580f6530 2021-11-03T18:04:56.196 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.194+0000 7f695659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69580f77d0 0x7f69581072f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:04:56.196 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.194+0000 7f695659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69580f77d0 0x7f69581072f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32852/0 (socket says 172.21.15.27:32852) 2021-11-03T18:04:56.196 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.194+0000 7f695659c700 1 -- 172.21.15.27:0/2427443431 learned_addr learned my addr 172.21.15.27:0/2427443431 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:04:56.197 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.194+0000 7f6956d9d700 1 --2- 172.21.15.27:0/2427443431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69580f6530 0x7f6958106e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:04:56.197 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.195+0000 7f695659c700 1 -- 172.21.15.27:0/2427443431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69580f6530 msgr2=0x7f6958106e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:04:56.197 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.195+0000 7f695659c700 1 --2- 172.21.15.27:0/2427443431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69580f6530 0x7f6958106e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:56.197 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.195+0000 7f695659c700 1 -- 172.21.15.27:0/2427443431 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6948005040 con 0x7f69580f77d0 2021-11-03T18:04:56.197 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.195+0000 7f695659c700 1 --2- 172.21.15.27:0/2427443431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69580f77d0 0x7f69581072f0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0x7f694c00c530 tx=0x7f694c007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:04:56.198 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.195+0000 7f693f7fe700 1 -- 172.21.15.27:0/2427443431 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f694c00f650 con 0x7f69580f77d0 2021-11-03T18:04:56.198 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.195+0000 7f695c878700 1 -- 172.21.15.27:0/2427443431 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6958108380 con 0x7f69580f77d0 2021-11-03T18:04:56.198 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.195+0000 7f693f7fe700 1 -- 172.21.15.27:0/2427443431 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f694c00fc60 con 0x7f69580f77d0 2021-11-03T18:04:56.198 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.195+0000 7f693f7fe700 1 -- 172.21.15.27:0/2427443431 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f694c012c20 con 0x7f69580f77d0 2021-11-03T18:04:56.199 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.195+0000 7f695c878700 1 -- 172.21.15.27:0/2427443431 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6958108940 con 0x7f69580f77d0 2021-11-03T18:04:56.199 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.196+0000 7f695c878700 1 -- 172.21.15.27:0/2427443431 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6958059100 con 0x7f69580f77d0 2021-11-03T18:04:56.200 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.196+0000 7f693f7fe700 1 -- 172.21.15.27:0/2427443431 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f694c010070 con 0x7f69580f77d0 2021-11-03T18:04:56.200 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.197+0000 7f693f7fe700 1 --2- 172.21.15.27:0/2427443431 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6940064100 0x7f69400665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:04:56.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.197+0000 7f693f7fe700 1 -- 172.21.15.27:0/2427443431 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f694c0806c0 con 0x7f69580f77d0 2021-11-03T18:04:56.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.197+0000 7f6956d9d700 1 --2- 172.21.15.27:0/2427443431 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6940064100 0x7f69400665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:04:56.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.197+0000 7f6956d9d700 1 --2- 172.21.15.27:0/2427443431 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6940064100 0x7f69400665c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7f6948013690 tx=0x7f6948009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:04:56.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.201+0000 7f693f7fe700 1 -- 172.21.15.27:0/2427443431 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f694c0528d0 con 0x7f69580f77d0 2021-11-03T18:04:56.404 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.407+0000 7f695c878700 1 -- 172.21.15.27:0/2427443431 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f69580fee80 con 0x7f69580f77d0 2021-11-03T18:04:56.404 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.407+0000 7f693f7fe700 1 -- 172.21.15.27:0/2427443431 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f69580fee80 con 0x7f69580f77d0 2021-11-03T18:04:56.404 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:04:56.405 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:04:56.405 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:04:56.405 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:04:56.405 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:04:56.406 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:04:56.406 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:04:56.406 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:04:56.406 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:04:56.407 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:04:56.407 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:04:56.407 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:04:56.407 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:04:56.407 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:04:56.408 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:04:56.408 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:04:56.408 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:04:56.408 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:04:56.409 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:04:56.409 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:04:56.409 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:04:56.409 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:04:56.410 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:04:56.410 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:04:56.410 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:04:56.410 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:04:56.410 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:04:56.411 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:04:56.411 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:04:56.411 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:04:56.411 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:04:56.411 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:04:56.412 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:04:56.412 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:04:56.412 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:04:56.412 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:04:56.413 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:04:56.413 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:04:56.413 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:04:56.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.409+0000 7f695c878700 1 -- 172.21.15.27:0/2427443431 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6940064100 msgr2=0x7f69400665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:04:56.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.409+0000 7f695c878700 1 --2- 172.21.15.27:0/2427443431 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6940064100 0x7f69400665c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7f6948013690 tx=0x7f6948009b30).stop 2021-11-03T18:04:56.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.409+0000 7f695c878700 1 -- 172.21.15.27:0/2427443431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69580f77d0 msgr2=0x7f69581072f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:04:56.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.409+0000 7f695c878700 1 --2- 172.21.15.27:0/2427443431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69580f77d0 0x7f69581072f0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0x7f694c00c530 tx=0x7f694c007e20).stop 2021-11-03T18:04:56.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.409+0000 7f695c878700 1 -- 172.21.15.27:0/2427443431 shutdown_connections 2021-11-03T18:04:56.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.409+0000 7f695c878700 1 --2- 172.21.15.27:0/2427443431 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6940064100 0x7f69400665c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:56.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.409+0000 7f695c878700 1 --2- 172.21.15.27:0/2427443431 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f69580f6530 0x7f6958106e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:56.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.409+0000 7f695c878700 1 --2- 172.21.15.27:0/2427443431 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f69580f77d0 0x7f69581072f0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:04:56.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.409+0000 7f695c878700 1 -- 172.21.15.27:0/2427443431 >> 172.21.15.27:0/2427443431 conn(0x7f69580f1c90 msgr2=0x7f69580fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:04:56.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.409+0000 7f695c878700 1 -- 172.21.15.27:0/2427443431 shutdown_connections 2021-11-03T18:04:56.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:04:56.409+0000 7f695c878700 1 -- 172.21.15.27:0/2427443431 wait complete. 2021-11-03T18:04:56.417 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:04:56.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:56 smithi115 ceph-mon[65453]: pgmap v1508: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:56.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:56 smithi115 ceph-mon[65453]: from='client.35656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:04:56.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:04:56.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:04:56.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:56 smithi027 ceph-mon[97244]: pgmap v1508: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:56.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:56 smithi027 ceph-mon[97244]: from='client.35656 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:04:56.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:04:56.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:04:57.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:57 smithi115 ceph-mon[65453]: from='client.35660 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:04:57.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:57 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/380734864' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:57.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:57 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2427443431' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:04:57.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:57 smithi027 ceph-mon[97244]: from='client.35660 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:04:57.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:57 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/380734864' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:57.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:57 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2427443431' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:04:58.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:58 smithi115 ceph-mon[65453]: pgmap v1509: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:58.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:58 smithi027 ceph-mon[97244]: pgmap v1509: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:04:59.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:04:59.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:04:59.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:04:59.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:59.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:59.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:59.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:04:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:59.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:04:59.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:04:59.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:04:59.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:59.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:59.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:04:59.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:04:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:00.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:00 smithi115 ceph-mon[65453]: pgmap v1510: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:00.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:00 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:05:00.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2942.430367 secs 2021-11-03T18:05:00.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:00 smithi027 ceph-mon[97244]: pgmap v1510: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:00.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:00 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:05:00.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2942.430367 secs 2021-11-03T18:05:02.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:02 smithi115 ceph-mon[65453]: pgmap v1511: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:02.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:02 smithi027 ceph-mon[97244]: pgmap v1511: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:04.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:04 smithi115 ceph-mon[65453]: pgmap v1512: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:04.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:04 smithi027 ceph-mon[97244]: pgmap v1512: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:05.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2947.430490 secs 2021-11-03T18:05:05.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2947.430490 secs 2021-11-03T18:05:06.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:06 smithi115 ceph-mon[65453]: pgmap v1513: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:06.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:06 smithi027 ceph-mon[97244]: pgmap v1513: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:08.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:08 smithi115 ceph-mon[65453]: pgmap v1514: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:08.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:08 smithi027 ceph-mon[97244]: pgmap v1514: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:09.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:05:09.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:05:09.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:05:09.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:09.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:09.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:09.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:09.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:05:09.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:05:09.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:05:09.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:09.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:09.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:09.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:10.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:10 smithi115 ceph-mon[65453]: pgmap v1515: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:10.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:10 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:05:10.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2952.430557 secs 2021-11-03T18:05:10.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:10 smithi027 ceph-mon[97244]: pgmap v1515: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:10.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:10 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:05:10.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2952.430557 secs 2021-11-03T18:05:12.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:12 smithi115 ceph-mon[65453]: pgmap v1516: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:12.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:12 smithi027 ceph-mon[97244]: pgmap v1516: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:14.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:14 smithi115 ceph-mon[65453]: pgmap v1517: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:14.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:14 smithi027 ceph-mon[97244]: pgmap v1517: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:15.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2957.430622 secs 2021-11-03T18:05:15.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2957.430622 secs 2021-11-03T18:05:16.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:16 smithi115 ceph-mon[65453]: pgmap v1518: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:16.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:16 smithi027 ceph-mon[97244]: pgmap v1518: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:18.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:18 smithi115 ceph-mon[65453]: pgmap v1519: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:18.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:18 smithi027 ceph-mon[97244]: pgmap v1519: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:19.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:05:19.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:05:19.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:05:19.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:19.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:19.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:19.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:19.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:05:19.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:05:19.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:05:19.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:19.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:19.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:19.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:20.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:20 smithi115 ceph-mon[65453]: pgmap v1520: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:20.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:20 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:05:20.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2962.430739 secs 2021-11-03T18:05:20.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:20 smithi027 ceph-mon[97244]: pgmap v1520: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:20.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:20 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:05:20.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2962.430739 secs 2021-11-03T18:05:22.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:22 smithi115 ceph-mon[65453]: pgmap v1521: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:22.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:22 smithi027 ceph-mon[97244]: pgmap v1521: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:24.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:24 smithi115 ceph-mon[65453]: pgmap v1522: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:24.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:24 smithi027 ceph-mon[97244]: pgmap v1522: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:25.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2967.430883 secs 2021-11-03T18:05:25.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2967.430883 secs 2021-11-03T18:05:26.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:26 smithi115 ceph-mon[65453]: pgmap v1523: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:26.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:26 smithi027 ceph-mon[97244]: pgmap v1523: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:26.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.523+0000 7fc37df86700 1 -- 172.21.15.27:0/3447254286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3780f6860 msgr2=0x7fc3780f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:05:26.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.523+0000 7fc37df86700 1 --2- 172.21.15.27:0/3447254286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3780f6860 0x7fc3780f6cc0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 rx=0x7fc360004660 tx=0x7fc360009b30).stop 2021-11-03T18:05:26.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.523+0000 7fc37df86700 1 -- 172.21.15.27:0/3447254286 shutdown_connections 2021-11-03T18:05:26.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.523+0000 7fc37df86700 1 --2- 172.21.15.27:0/3447254286 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3780f6860 0x7fc3780f6cc0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:26.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.523+0000 7fc37df86700 1 --2- 172.21.15.27:0/3447254286 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3780f55c0 0x7fc3780f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:26.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.523+0000 7fc37df86700 1 -- 172.21.15.27:0/3447254286 >> 172.21.15.27:0/3447254286 conn(0x7fc3780f0d40 msgr2=0x7fc3780f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:05:26.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.523+0000 7fc37df86700 1 -- 172.21.15.27:0/3447254286 shutdown_connections 2021-11-03T18:05:26.524 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.523+0000 7fc37df86700 1 -- 172.21.15.27:0/3447254286 wait complete. 2021-11-03T18:05:26.524 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.524+0000 7fc37df86700 1 Processor -- start 2021-11-03T18:05:26.524 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.524+0000 7fc37df86700 1 -- start start 2021-11-03T18:05:26.524 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.524+0000 7fc37df86700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3780f55c0 0x7fc378109bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:05:26.524 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.524+0000 7fc37df86700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3780f6860 0x7fc37810c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:05:26.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.524+0000 7fc37df86700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3780603e0 con 0x7fc3780f6860 2021-11-03T18:05:26.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.524+0000 7fc37df86700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc378060520 con 0x7fc3780f55c0 2021-11-03T18:05:26.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.524+0000 7fc376ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3780f6860 0x7fc37810c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:05:26.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.524+0000 7fc376ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3780f6860 0x7fc37810c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32870/0 (socket says 172.21.15.27:32870) 2021-11-03T18:05:26.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.524+0000 7fc376ffd700 1 -- 172.21.15.27:0/417520773 learned_addr learned my addr 172.21.15.27:0/417520773 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:05:26.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.524+0000 7fc3777fe700 1 --2- 172.21.15.27:0/417520773 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3780f55c0 0x7fc378109bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:05:26.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.525+0000 7fc376ffd700 1 -- 172.21.15.27:0/417520773 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3780f55c0 msgr2=0x7fc378109bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:05:26.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.525+0000 7fc376ffd700 1 --2- 172.21.15.27:0/417520773 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3780f55c0 0x7fc378109bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:26.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.525+0000 7fc376ffd700 1 -- 172.21.15.27:0/417520773 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc360005040 con 0x7fc3780f6860 2021-11-03T18:05:26.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.525+0000 7fc376ffd700 1 --2- 172.21.15.27:0/417520773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3780f6860 0x7fc37810c0c0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7fc360009990 tx=0x7fc360009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:05:26.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.525+0000 7fc36ffff700 1 -- 172.21.15.27:0/417520773 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc360018070 con 0x7fc3780f6860 2021-11-03T18:05:26.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.525+0000 7fc37df86700 1 -- 172.21.15.27:0/417520773 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc37810c610 con 0x7fc3780f6860 2021-11-03T18:05:26.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.525+0000 7fc36ffff700 1 -- 172.21.15.27:0/417520773 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fc360007ba0 con 0x7fc3780f6860 2021-11-03T18:05:26.530 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.525+0000 7fc36ffff700 1 -- 172.21.15.27:0/417520773 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc36001c3c0 con 0x7fc3780f6860 2021-11-03T18:05:26.530 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.525+0000 7fc37df86700 1 -- 172.21.15.27:0/417520773 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc37810cb70 con 0x7fc3780f6860 2021-11-03T18:05:26.530 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.526+0000 7fc37df86700 1 -- 172.21.15.27:0/417520773 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc378045ad0 con 0x7fc3780f6860 2021-11-03T18:05:26.530 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.526+0000 7fc36ffff700 1 -- 172.21.15.27:0/417520773 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fc36001d420 con 0x7fc3780f6860 2021-11-03T18:05:26.530 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.527+0000 7fc36ffff700 1 --2- 172.21.15.27:0/417520773 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc3640640b0 0x7fc364066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:05:26.531 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.527+0000 7fc36ffff700 1 -- 172.21.15.27:0/417520773 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fc360082630 con 0x7fc3780f6860 2021-11-03T18:05:26.531 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.527+0000 7fc3777fe700 1 --2- 172.21.15.27:0/417520773 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc3640640b0 0x7fc364066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:05:26.531 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.527+0000 7fc3777fe700 1 --2- 172.21.15.27:0/417520773 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc3640640b0 0x7fc364066570 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7fc378060060 tx=0x7fc370004d80).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:05:26.532 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.531+0000 7fc36ffff700 1 -- 172.21.15.27:0/417520773 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fc360054990 con 0x7fc3780f6860 2021-11-03T18:05:26.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.711+0000 7fc37df86700 1 -- 172.21.15.27:0/417520773 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc3780fdec0 con 0x7fc3640640b0 2021-11-03T18:05:26.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.712+0000 7fc36ffff700 1 -- 172.21.15.27:0/417520773 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fc3780fdec0 con 0x7fc3640640b0 2021-11-03T18:05:26.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.714+0000 7fc37df86700 1 -- 172.21.15.27:0/417520773 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc3640640b0 msgr2=0x7fc364066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:05:26.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.714+0000 7fc37df86700 1 --2- 172.21.15.27:0/417520773 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc3640640b0 0x7fc364066570 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7fc378060060 tx=0x7fc370004d80).stop 2021-11-03T18:05:26.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.714+0000 7fc37df86700 1 -- 172.21.15.27:0/417520773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3780f6860 msgr2=0x7fc37810c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:05:26.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.714+0000 7fc37df86700 1 --2- 172.21.15.27:0/417520773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3780f6860 0x7fc37810c0c0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 rx=0x7fc360009990 tx=0x7fc360009b30).stop 2021-11-03T18:05:26.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.714+0000 7fc37df86700 1 -- 172.21.15.27:0/417520773 shutdown_connections 2021-11-03T18:05:26.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.714+0000 7fc37df86700 1 --2- 172.21.15.27:0/417520773 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc3640640b0 0x7fc364066570 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:26.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.714+0000 7fc37df86700 1 --2- 172.21.15.27:0/417520773 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3780f55c0 0x7fc378109bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:26.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.714+0000 7fc37df86700 1 --2- 172.21.15.27:0/417520773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3780f6860 0x7fc37810c0c0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:26.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.714+0000 7fc37df86700 1 -- 172.21.15.27:0/417520773 >> 172.21.15.27:0/417520773 conn(0x7fc3780f0d40 msgr2=0x7fc3780fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:05:26.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.714+0000 7fc37df86700 1 -- 172.21.15.27:0/417520773 shutdown_connections 2021-11-03T18:05:26.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.714+0000 7fc37df86700 1 -- 172.21.15.27:0/417520773 wait complete. 2021-11-03T18:05:26.721 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:05:26.813 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.814+0000 7f7d8fa43700 1 -- 172.21.15.27:0/3128784203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d880f66e0 msgr2=0x7f7d880f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:05:26.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.814+0000 7f7d8fa43700 1 --2- 172.21.15.27:0/3128784203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d880f66e0 0x7f7d880f6b00 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 rx=0x7f7d78004660 tx=0x7f7d78009b30).stop 2021-11-03T18:05:26.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.814+0000 7f7d8fa43700 1 -- 172.21.15.27:0/3128784203 shutdown_connections 2021-11-03T18:05:26.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.814+0000 7f7d8fa43700 1 --2- 172.21.15.27:0/3128784203 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d880f6ff0 0x7f7d880f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T18:05:26.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.814+0000 7f7d8fa43700 1 --2- 172.21.15.27:0/3128784203 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d880f66e0 0x7f7d880f6b00 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:26.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.814+0000 7f7d8fa43700 1 -- 172.21.15.27:0/3128784203 >> 172.21.15.27:0/3128784203 conn(0x7f7d880f1f00 msgr2=0x7f7d880f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:05:26.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.814+0000 7f7d8fa43700 1 -- 172.21.15.27:0/3128784203 shutdown_connections 2021-11-03T18:05:26.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.814+0000 7f7d8fa43700 1 -- 172.21.15.27:0/3128784203 wait complete. 2021-11-03T18:05:26.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.815+0000 7f7d8fa43700 1 Processor -- start 2021-11-03T18:05:26.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.815+0000 7f7d8fa43700 1 -- start start 2021-11-03T18:05:26.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.815+0000 7f7d8fa43700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d880f66e0 0x7f7d8806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:05:26.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.815+0000 7f7d8fa43700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d880f6ff0 0x7f7d88069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:05:26.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.815+0000 7f7d8fa43700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8806a410 con 0x7f7d880f6ff0 2021-11-03T18:05:26.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.815+0000 7f7d8fa43700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d8806a550 con 0x7f7d880f66e0 2021-11-03T18:05:26.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.815+0000 7f7d8cfde700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d880f6ff0 0x7f7d88069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:05:26.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.815+0000 7f7d8cfde700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d880f6ff0 0x7f7d88069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32880/0 (socket says 172.21.15.27:32880) 2021-11-03T18:05:26.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.815+0000 7f7d8cfde700 1 -- 172.21.15.27:0/3643979311 learned_addr learned my addr 172.21.15.27:0/3643979311 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:05:26.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.815+0000 7f7d8cfde700 1 -- 172.21.15.27:0/3643979311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d880f66e0 msgr2=0x7f7d8806b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:05:26.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.815+0000 7f7d8cfde700 1 --2- 172.21.15.27:0/3643979311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d880f66e0 0x7f7d8806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:05:26.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.815+0000 7f7d8cfde700 1 -- 172.21.15.27:0/3643979311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d78005040 con 0x7f7d880f6ff0 2021-11-03T18:05:26.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.815+0000 7f7d8cfde700 1 --2- 172.21.15.27:0/3643979311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d880f6ff0 0x7f7d88069e20 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 rx=0x7f7d8400b410 tx=0x7f7d84006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:05:26.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.815+0000 7f7d7dffb700 1 -- 172.21.15.27:0/3643979311 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d840125f0 con 0x7f7d880f6ff0 2021-11-03T18:05:26.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.816+0000 7f7d8fa43700 1 -- 172.21.15.27:0/3643979311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d8806a830 con 0x7f7d880f6ff0 2021-11-03T18:05:26.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.816+0000 7f7d7dffb700 1 -- 172.21.15.27:0/3643979311 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f7d84012c00 con 0x7f7d880f6ff0 2021-11-03T18:05:26.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.816+0000 7f7d7dffb700 1 -- 172.21.15.27:0/3643979311 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d84002ae0 con 0x7f7d880f6ff0 2021-11-03T18:05:26.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.816+0000 7f7d8fa43700 1 -- 172.21.15.27:0/3643979311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d8810efb0 con 0x7f7d880f6ff0 2021-11-03T18:05:26.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.817+0000 7f7d8fa43700 1 -- 172.21.15.27:0/3643979311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d88045ad0 con 0x7f7d880f6ff0 2021-11-03T18:05:26.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.817+0000 7f7d7dffb700 1 -- 172.21.15.27:0/3643979311 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f7d8400f030 con 0x7f7d880f6ff0 2021-11-03T18:05:26.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.817+0000 7f7d7dffb700 1 --2- 172.21.15.27:0/3643979311 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7d740640b0 0x7f7d74066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:05:26.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.818+0000 7f7d7dffb700 1 -- 172.21.15.27:0/3643979311 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f7d84080cc0 con 0x7f7d880f6ff0 2021-11-03T18:05:26.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.818+0000 7f7d8d7df700 1 --2- 172.21.15.27:0/3643979311 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7d740640b0 0x7f7d74066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:05:26.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.818+0000 7f7d8d7df700 1 --2- 172.21.15.27:0/3643979311 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7d740640b0 0x7f7d74066570 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f7d78004e30 tx=0x7f7d78009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:05:26.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.822+0000 7f7d7dffb700 1 -- 172.21.15.27:0/3643979311 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f7d84052fc0 con 0x7f7d880f6ff0 2021-11-03T18:05:26.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:26.995+0000 7f7d8fa43700 1 -- 172.21.15.27:0/3643979311 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d880ff0d0 con 0x7f7d740640b0 2021-11-03T18:05:27.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.009+0000 7f7d7dffb700 1 -- 172.21.15.27:0/3643979311 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f7d880ff0d0 con 0x7f7d740640b0 2021-11-03T18:05:27.007 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:05:27.008 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (57m) 6m ago 58m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:05:27.008 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (48m) 6m ago 58m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:05:27.008 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (48m) 5m ago 57m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:05:27.008 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (56m) 6m ago 57m 41.8M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:05:27.009 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (50m) 6m ago 53m 26.0M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:05:27.009 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (53m) 6m ago 53m 3845M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:05:27.009 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (49m) 5m ago 53m 25.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:05:27.009 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (50m) 5m ago 53m 26.8M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:05:27.010 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (50m) 6m ago 59m 554M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:05:27.010 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (50m) 5m ago 57m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:05:27.010 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (49m) 6m ago 59m 235M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:05:27.010 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (49m) 5m ago 57m 112M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:05:27.010 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (57m) 6m ago 57m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:05:27.011 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (57m) 5m ago 57m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:05:27.011 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (48m) 6m ago 56m 351M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:05:27.011 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (47m) 6m ago 56m 383M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:05:27.011 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (47m) 6m ago 55m 284M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:05:27.012 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (46m) 5m ago 55m 404M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:05:27.012 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (46m) 5m ago 55m 322M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:05:27.012 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (45m) 5m ago 54m 327M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:05:27.012 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (56m) 6m ago 57m 55.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:05:27.013 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.011+0000 7f7d8fa43700 1 -- 172.21.15.27:0/3643979311 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7d740640b0 msgr2=0x7f7d74066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:05:27.013 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.011+0000 7f7d8fa43700 1 --2- 172.21.15.27:0/3643979311 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7d740640b0 0x7f7d74066570 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f7d78004e30 tx=0x7f7d78009b30).stop 2021-11-03T18:05:27.014 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.011+0000 7f7d8fa43700 1 -- 172.21.15.27:0/3643979311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d880f6ff0 msgr2=0x7f7d88069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:05:27.014 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.011+0000 7f7d8fa43700 1 --2- 172.21.15.27:0/3643979311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d880f6ff0 0x7f7d88069e20 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 rx=0x7f7d8400b410 tx=0x7f7d84006ef0).stop 2021-11-03T18:05:27.014 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.011+0000 7f7d8fa43700 1 -- 172.21.15.27:0/3643979311 shutdown_connections 2021-11-03T18:05:27.014 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.011+0000 7f7d8fa43700 1 --2- 172.21.15.27:0/3643979311 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7d740640b0 0x7f7d74066570 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:27.015 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.011+0000 7f7d8fa43700 1 --2- 172.21.15.27:0/3643979311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d880f66e0 0x7f7d8806b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:05:27.015 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.011+0000 7f7d8fa43700 1 --2- 172.21.15.27:0/3643979311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d880f6ff0 0x7f7d88069e20 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:27.015 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.011+0000 7f7d8fa43700 1 -- 172.21.15.27:0/3643979311 >> 172.21.15.27:0/3643979311 conn(0x7f7d880f1f00 msgr2=0x7f7d880fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:05:27.015 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.011+0000 7f7d8fa43700 1 -- 172.21.15.27:0/3643979311 shutdown_connections 2021-11-03T18:05:27.015 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.011+0000 7f7d8fa43700 1 -- 172.21.15.27:0/3643979311 wait complete. 2021-11-03T18:05:27.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.110+0000 7f01ad365700 1 -- 172.21.15.27:0/3315246147 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01a80f96a0 msgr2=0x7f01a80fbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:05:27.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.110+0000 7f01ad365700 1 --2- 172.21.15.27:0/3315246147 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01a80f96a0 0x7f01a80fbaf0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7f019c004660 tx=0x7f019c009b30).stop 2021-11-03T18:05:27.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.110+0000 7f01ad365700 1 -- 172.21.15.27:0/3315246147 shutdown_connections 2021-11-03T18:05:27.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.110+0000 7f01ad365700 1 --2- 172.21.15.27:0/3315246147 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01a80f96a0 0x7f01a80fbaf0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:27.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.110+0000 7f01ad365700 1 --2- 172.21.15.27:0/3315246147 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01a8060310 0x7f01a80f9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:27.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.110+0000 7f01ad365700 1 -- 172.21.15.27:0/3315246147 >> 172.21.15.27:0/3315246147 conn(0x7f01a80f0ba0 msgr2=0x7f01a80f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:05:27.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.110+0000 7f01ad365700 1 -- 172.21.15.27:0/3315246147 shutdown_connections 2021-11-03T18:05:27.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.111+0000 7f01ad365700 1 -- 172.21.15.27:0/3315246147 wait complete. 2021-11-03T18:05:27.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.111+0000 7f01ad365700 1 Processor -- start 2021-11-03T18:05:27.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.111+0000 7f01ad365700 1 -- start start 2021-11-03T18:05:27.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.111+0000 7f01ad365700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01a8060310 0x7f01a80f6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:05:27.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.111+0000 7f01ad365700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01a80f96a0 0x7f01a80f5020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:05:27.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.111+0000 7f01ad365700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a80f5580 con 0x7f01a80f96a0 2021-11-03T18:05:27.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.111+0000 7f01ad365700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01a80f56c0 con 0x7f01a8060310 2021-11-03T18:05:27.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.112+0000 7f01a77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01a80f96a0 0x7f01a80f5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:05:27.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.112+0000 7f01a77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01a80f96a0 0x7f01a80f5020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32890/0 (socket says 172.21.15.27:32890) 2021-11-03T18:05:27.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.112+0000 7f01a77fe700 1 -- 172.21.15.27:0/4278890424 learned_addr learned my addr 172.21.15.27:0/4278890424 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:05:27.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.112+0000 7f01a7fff700 1 --2- 172.21.15.27:0/4278890424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01a8060310 0x7f01a80f6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:05:27.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.112+0000 7f01a77fe700 1 -- 172.21.15.27:0/4278890424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01a8060310 msgr2=0x7f01a80f6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:05:27.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.112+0000 7f01a77fe700 1 --2- 172.21.15.27:0/4278890424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01a8060310 0x7f01a80f6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:27.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.112+0000 7f01a77fe700 1 -- 172.21.15.27:0/4278890424 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f019c005040 con 0x7f01a80f96a0 2021-11-03T18:05:27.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.112+0000 7f01a77fe700 1 --2- 172.21.15.27:0/4278890424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01a80f96a0 0x7f01a80f5020 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7f019c013a70 tx=0x7f019c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:05:27.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.112+0000 7f01a4ff9700 1 -- 172.21.15.27:0/4278890424 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f019c018070 con 0x7f01a80f96a0 2021-11-03T18:05:27.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.112+0000 7f01ad365700 1 -- 172.21.15.27:0/4278890424 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01a80f5940 con 0x7f01a80f96a0 2021-11-03T18:05:27.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.112+0000 7f01a4ff9700 1 -- 172.21.15.27:0/4278890424 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f019c0097d0 con 0x7f01a80f96a0 2021-11-03T18:05:27.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.112+0000 7f01a4ff9700 1 -- 172.21.15.27:0/4278890424 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f019c01de80 con 0x7f01a80f96a0 2021-11-03T18:05:27.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.112+0000 7f01ad365700 1 -- 172.21.15.27:0/4278890424 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a80f5ea0 con 0x7f01a80f96a0 2021-11-03T18:05:27.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.113+0000 7f01ad365700 1 -- 172.21.15.27:0/4278890424 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01a8045ad0 con 0x7f01a80f96a0 2021-11-03T18:05:27.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.114+0000 7f01a4ff9700 1 -- 172.21.15.27:0/4278890424 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f019c007ba0 con 0x7f01a80f96a0 2021-11-03T18:05:27.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.114+0000 7f01a4ff9700 1 --2- 172.21.15.27:0/4278890424 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0190064100 0x7f01900665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:05:27.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.114+0000 7f01a4ff9700 1 -- 172.21.15.27:0/4278890424 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f019c0824c0 con 0x7f01a80f96a0 2021-11-03T18:05:27.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.114+0000 7f01a7fff700 1 --2- 172.21.15.27:0/4278890424 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0190064100 0x7f01900665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:05:27.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.115+0000 7f01a7fff700 1 --2- 172.21.15.27:0/4278890424 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0190064100 0x7f01900665c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f01a80f6930 tx=0x7f019800b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:05:27.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.119+0000 7f01a4ff9700 1 -- 172.21.15.27:0/4278890424 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f019c0547c0 con 0x7f01a80f96a0 2021-11-03T18:05:27.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.348+0000 7f01ad365700 1 -- 172.21.15.27:0/4278890424 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f01a80feda0 con 0x7f01a80f96a0 2021-11-03T18:05:27.347 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.349+0000 7f01a4ff9700 1 -- 172.21.15.27:0/4278890424 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f019c0543b0 con 0x7f01a80f96a0 2021-11-03T18:05:27.347 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:05:27.348 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:05:27.348 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:05:27.348 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:05:27.348 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:05:27.349 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:05:27.349 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:05:27.349 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:05:27.349 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:05:27.349 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:05:27.350 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:05:27.350 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:05:27.350 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:05:27.350 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:05:27.351 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:05:27.351 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:05:27.351 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:05:27.351 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:05:27.352 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.350+0000 7f01ad365700 1 -- 172.21.15.27:0/4278890424 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0190064100 msgr2=0x7f01900665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:05:27.352 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.350+0000 7f01ad365700 1 --2- 172.21.15.27:0/4278890424 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0190064100 0x7f01900665c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f01a80f6930 tx=0x7f019800b040).stop 2021-11-03T18:05:27.353 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.350+0000 7f01ad365700 1 -- 172.21.15.27:0/4278890424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01a80f96a0 msgr2=0x7f01a80f5020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:05:27.353 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.350+0000 7f01ad365700 1 --2- 172.21.15.27:0/4278890424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01a80f96a0 0x7f01a80f5020 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7f019c013a70 tx=0x7f019c009b30).stop 2021-11-03T18:05:27.353 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.350+0000 7f01ad365700 1 -- 172.21.15.27:0/4278890424 shutdown_connections 2021-11-03T18:05:27.353 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.351+0000 7f01ad365700 1 --2- 172.21.15.27:0/4278890424 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0190064100 0x7f01900665c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:27.354 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.351+0000 7f01ad365700 1 --2- 172.21.15.27:0/4278890424 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01a8060310 0x7f01a80f6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:27.354 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.351+0000 7f01ad365700 1 --2- 172.21.15.27:0/4278890424 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01a80f96a0 0x7f01a80f5020 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:27.354 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.351+0000 7f01ad365700 1 -- 172.21.15.27:0/4278890424 >> 172.21.15.27:0/4278890424 conn(0x7f01a80f0ba0 msgr2=0x7f01a80f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:05:27.354 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.351+0000 7f01ad365700 1 -- 172.21.15.27:0/4278890424 shutdown_connections 2021-11-03T18:05:27.354 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.351+0000 7f01ad365700 1 -- 172.21.15.27:0/4278890424 wait complete. 2021-11-03T18:05:27.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.451+0000 7f7fa0735700 1 -- 172.21.15.27:0/4181502749 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f980f6ff0 msgr2=0x7f7f980f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:05:27.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.451+0000 7f7fa0735700 1 --2- 172.21.15.27:0/4181502749 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f980f6ff0 0x7f7f980f7a20 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 rx=0x7f7f8c004660 tx=0x7f7f8c009b30).stop 2021-11-03T18:05:27.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.451+0000 7f7fa0735700 1 -- 172.21.15.27:0/4181502749 shutdown_connections 2021-11-03T18:05:27.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.451+0000 7f7fa0735700 1 --2- 172.21.15.27:0/4181502749 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f980f6ff0 0x7f7f980f7a20 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:27.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.451+0000 7f7fa0735700 1 --2- 172.21.15.27:0/4181502749 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f980f66e0 0x7f7f980f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:27.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.451+0000 7f7fa0735700 1 -- 172.21.15.27:0/4181502749 >> 172.21.15.27:0/4181502749 conn(0x7f7f980f1f00 msgr2=0x7f7f980f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:05:27.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.451+0000 7f7fa0735700 1 -- 172.21.15.27:0/4181502749 shutdown_connections 2021-11-03T18:05:27.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.451+0000 7f7fa0735700 1 -- 172.21.15.27:0/4181502749 wait complete. 2021-11-03T18:05:27.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.451+0000 7f7fa0735700 1 Processor -- start 2021-11-03T18:05:27.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.451+0000 7f7fa0735700 1 -- start start 2021-11-03T18:05:27.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.452+0000 7f7fa0735700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f980f66e0 0x7f7f98106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:05:27.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.452+0000 7f7fa0735700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f980f6ff0 0x7f7f98107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:05:27.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.452+0000 7f7fa0735700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f98107a10 con 0x7f7f980f66e0 2021-11-03T18:05:27.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.452+0000 7f7fa0735700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f98107b50 con 0x7f7f980f6ff0 2021-11-03T18:05:27.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.452+0000 7f7f9e4d1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f980f66e0 0x7f7f98106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:05:27.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.452+0000 7f7f9e4d1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f980f66e0 0x7f7f98106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32898/0 (socket says 172.21.15.27:32898) 2021-11-03T18:05:27.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.452+0000 7f7f9e4d1700 1 -- 172.21.15.27:0/1349317457 learned_addr learned my addr 172.21.15.27:0/1349317457 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:05:27.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.452+0000 7f7f9dcd0700 1 --2- 172.21.15.27:0/1349317457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f980f6ff0 0x7f7f98107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:05:27.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.452+0000 7f7f9e4d1700 1 -- 172.21.15.27:0/1349317457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f980f6ff0 msgr2=0x7f7f98107460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:05:27.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.452+0000 7f7f9e4d1700 1 --2- 172.21.15.27:0/1349317457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f980f6ff0 0x7f7f98107460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:27.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.452+0000 7f7f9e4d1700 1 -- 172.21.15.27:0/1349317457 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f8c005040 con 0x7f7f980f66e0 2021-11-03T18:05:27.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.452+0000 7f7f9e4d1700 1 --2- 172.21.15.27:0/1349317457 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f980f66e0 0x7f7f98106f70 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0x7f7f90007fd0 tx=0x7f7f90004ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:05:27.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.452+0000 7f7f96ffd700 1 -- 172.21.15.27:0/1349317457 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f9000b4f0 con 0x7f7f980f66e0 2021-11-03T18:05:27.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.452+0000 7f7f96ffd700 1 -- 172.21.15.27:0/1349317457 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f7f9000f040 con 0x7f7f980f66e0 2021-11-03T18:05:27.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.452+0000 7f7fa0735700 1 -- 172.21.15.27:0/1349317457 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f981084f0 con 0x7f7f980f66e0 2021-11-03T18:05:27.457 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.452+0000 7f7f96ffd700 1 -- 172.21.15.27:0/1349317457 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7f90013a50 con 0x7f7f980f66e0 2021-11-03T18:05:27.457 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.452+0000 7f7fa0735700 1 -- 172.21.15.27:0/1349317457 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f98108ab0 con 0x7f7f980f66e0 2021-11-03T18:05:27.457 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.453+0000 7f7fa0735700 1 -- 172.21.15.27:0/1349317457 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f98045ad0 con 0x7f7f980f66e0 2021-11-03T18:05:27.458 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.454+0000 7f7f96ffd700 1 -- 172.21.15.27:0/1349317457 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f7f9001a030 con 0x7f7f980f66e0 2021-11-03T18:05:27.458 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.454+0000 7f7f96ffd700 1 --2- 172.21.15.27:0/1349317457 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7f800640b0 0x7f7f80066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:05:27.459 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.454+0000 7f7f96ffd700 1 -- 172.21.15.27:0/1349317457 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f7f90080740 con 0x7f7f980f66e0 2021-11-03T18:05:27.459 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.454+0000 7f7f9dcd0700 1 --2- 172.21.15.27:0/1349317457 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7f800640b0 0x7f7f80066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:05:27.459 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.455+0000 7f7f9dcd0700 1 --2- 172.21.15.27:0/1349317457 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7f800640b0 0x7f7f80066570 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7f7f8c004630 tx=0x7f7f8c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:05:27.459 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.458+0000 7f7f96ffd700 1 -- 172.21.15.27:0/1349317457 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f7f90052a00 con 0x7f7f980f66e0 2021-11-03T18:05:27.658 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.661+0000 7f7fa0735700 1 -- 172.21.15.27:0/1349317457 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7f98108680 con 0x7f7f980f66e0 2021-11-03T18:05:27.659 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.661+0000 7f7f96ffd700 1 -- 172.21.15.27:0/1349317457 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f7f98108680 con 0x7f7f980f66e0 2021-11-03T18:05:27.659 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:05:27.659 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:05:27.660 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:05:27.660 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:05:27.660 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:05:27.660 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:05:27.660 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:05:27.661 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:05:27.661 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:05:27.661 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:05:27.661 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:05:27.662 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:05:27.662 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:05:27.662 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:05:27.662 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:05:27.662 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:05:27.663 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:05:27.663 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:05:27.663 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:05:27.663 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:05:27.664 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:05:27.664 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:05:27.664 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:05:27.664 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:05:27.664 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:05:27.665 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:05:27.665 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:05:27.665 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:05:27.665 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:05:27.665 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:05:27.666 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:05:27.666 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:05:27.666 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:05:27.666 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:05:27.667 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:05:27.667 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:05:27.667 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:05:27.667 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:05:27.668 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:05:27.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.663+0000 7f7fa0735700 1 -- 172.21.15.27:0/1349317457 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7f800640b0 msgr2=0x7f7f80066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:05:27.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.663+0000 7f7fa0735700 1 --2- 172.21.15.27:0/1349317457 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7f800640b0 0x7f7f80066570 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 rx=0x7f7f8c004630 tx=0x7f7f8c009b30).stop 2021-11-03T18:05:27.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.663+0000 7f7fa0735700 1 -- 172.21.15.27:0/1349317457 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f980f66e0 msgr2=0x7f7f98106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:05:27.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.663+0000 7f7fa0735700 1 --2- 172.21.15.27:0/1349317457 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f980f66e0 0x7f7f98106f70 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 rx=0x7f7f90007fd0 tx=0x7f7f90004ee0).stop 2021-11-03T18:05:27.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.663+0000 7f7fa0735700 1 -- 172.21.15.27:0/1349317457 shutdown_connections 2021-11-03T18:05:27.670 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.663+0000 7f7fa0735700 1 --2- 172.21.15.27:0/1349317457 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7f800640b0 0x7f7f80066570 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:27.670 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.663+0000 7f7fa0735700 1 --2- 172.21.15.27:0/1349317457 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f980f66e0 0x7f7f98106f70 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:27.670 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.663+0000 7f7fa0735700 1 --2- 172.21.15.27:0/1349317457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f980f6ff0 0x7f7f98107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:27.670 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.663+0000 7f7fa0735700 1 -- 172.21.15.27:0/1349317457 >> 172.21.15.27:0/1349317457 conn(0x7f7f980f1f00 msgr2=0x7f7f980fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:05:27.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.663+0000 7f7fa0735700 1 -- 172.21.15.27:0/1349317457 shutdown_connections 2021-11-03T18:05:27.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:27.663+0000 7f7fa0735700 1 -- 172.21.15.27:0/1349317457 wait complete. 2021-11-03T18:05:27.671 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:05:28.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:28 smithi115 ceph-mon[65453]: pgmap v1524: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:28.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:28 smithi115 ceph-mon[65453]: from='client.35672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:05:28.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:28 smithi115 ceph-mon[65453]: from='client.35676 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:05:28.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:28 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/4278890424' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:28.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:28 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1349317457' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:05:28.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:28 smithi027 ceph-mon[97244]: pgmap v1524: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:28.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:28 smithi027 ceph-mon[97244]: from='client.35672 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:05:28.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:28 smithi027 ceph-mon[97244]: from='client.35676 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:05:28.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:28 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/4278890424' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:28.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:28 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1349317457' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:05:29.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:05:29.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:05:29.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:05:29.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:29.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:29.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:29.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:29.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:05:29.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:05:29.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:05:29.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:29.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:29.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:29.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:30.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:30 smithi115 ceph-mon[65453]: pgmap v1525: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:30.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:30 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:05:30.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2972.431010 secs 2021-11-03T18:05:30.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:30 smithi027 ceph-mon[97244]: pgmap v1525: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:30.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:30 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:05:30.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2972.431010 secs 2021-11-03T18:05:32.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:32 smithi115 ceph-mon[65453]: pgmap v1526: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:32.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:32 smithi027 ceph-mon[97244]: pgmap v1526: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:34.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:34 smithi115 ceph-mon[65453]: pgmap v1527: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:34.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:34 smithi027 ceph-mon[97244]: pgmap v1527: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:35.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2977.431143 secs 2021-11-03T18:05:35.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2977.431143 secs 2021-11-03T18:05:36.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:36 smithi115 ceph-mon[65453]: pgmap v1528: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:36.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:36 smithi027 ceph-mon[97244]: pgmap v1528: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:38.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:38 smithi115 ceph-mon[65453]: pgmap v1529: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:38.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:38 smithi027 ceph-mon[97244]: pgmap v1529: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:39.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:05:39.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:05:39.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:05:39.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:39.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:39.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:39.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:39.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:05:39.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:05:39.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:05:39.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:39.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:39.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:39.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:40.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:40 smithi115 ceph-mon[65453]: pgmap v1530: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:40.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:40 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:05:40.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2982.431265 secs 2021-11-03T18:05:40.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:40 smithi027 ceph-mon[97244]: pgmap v1530: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:40.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:40 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:05:40.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2982.431265 secs 2021-11-03T18:05:42.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:42 smithi115 ceph-mon[65453]: pgmap v1531: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:42.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:42 smithi027 ceph-mon[97244]: pgmap v1531: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:44.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:44 smithi115 ceph-mon[65453]: pgmap v1532: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:44.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:44 smithi027 ceph-mon[97244]: pgmap v1532: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:45.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2987.431391 secs 2021-11-03T18:05:45.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2987.431391 secs 2021-11-03T18:05:46.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:46 smithi115 ceph-mon[65453]: pgmap v1533: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:46.520 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:46 smithi027 ceph-mon[97244]: pgmap v1533: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:48.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:48 smithi115 ceph-mon[65453]: pgmap v1534: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:48.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:48 smithi027 ceph-mon[97244]: pgmap v1534: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:49.359 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:05:49.394 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:05:50.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:50 smithi115 ceph-mon[65453]: pgmap v1535: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:50.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:50 smithi115 ceph-mon[65453]: poll took 60179.604 ms: 1 events 2021-11-03T18:05:50.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2992.431456 secs 2021-11-03T18:05:50.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:05:50.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:05:50.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:50.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:50.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:50.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:50.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:50 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:05:50.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:50 smithi027 ceph-mon[97244]: pgmap v1535: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:50.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:50 smithi027 ceph-mon[97244]: poll took 60179.604 ms: 1 events 2021-11-03T18:05:50.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2992.431456 secs 2021-11-03T18:05:50.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:05:50.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:05:50.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:50.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:50.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:50.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:50.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:50 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:05:52.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:52 smithi115 ceph-mon[65453]: pgmap v1536: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:52.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:52 smithi027 ceph-mon[97244]: pgmap v1536: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:54.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:54 smithi115 ceph-mon[65453]: pgmap v1537: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:54.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:54 smithi027 ceph-mon[97244]: pgmap v1537: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:55.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 2997.431587 secs 2021-11-03T18:05:55.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 2997.431587 secs 2021-11-03T18:05:56.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:56 smithi115 ceph-mon[65453]: pgmap v1538: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:56.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:05:56.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:05:56.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:56 smithi027 ceph-mon[97244]: pgmap v1538: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:56.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:05:56.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:05:57.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.777+0000 7fd84f59e700 1 -- 172.21.15.27:0/1776243573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8500602b0 msgr2=0x7fd8500f9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:05:57.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.777+0000 7fd84f59e700 1 --2- 172.21.15.27:0/1776243573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8500602b0 0x7fd8500f9180 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0x7fd8480099e0 tx=0x7fd848004d10).stop 2021-11-03T18:05:57.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.777+0000 7fd84f59e700 1 -- 172.21.15.27:0/1776243573 shutdown_connections 2021-11-03T18:05:57.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.777+0000 7fd84f59e700 1 --2- 172.21.15.27:0/1776243573 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8500f9700 0x7fd8500fbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:57.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.777+0000 7fd84f59e700 1 --2- 172.21.15.27:0/1776243573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8500602b0 0x7fd8500f9180 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:57.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.777+0000 7fd84f59e700 1 -- 172.21.15.27:0/1776243573 >> 172.21.15.27:0/1776243573 conn(0x7fd8500f0ba0 msgr2=0x7fd8500f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:05:57.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.777+0000 7fd84f59e700 1 -- 172.21.15.27:0/1776243573 shutdown_connections 2021-11-03T18:05:57.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.777+0000 7fd84f59e700 1 -- 172.21.15.27:0/1776243573 wait complete. 2021-11-03T18:05:57.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.777+0000 7fd84f59e700 1 Processor -- start 2021-11-03T18:05:57.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.777+0000 7fd84f59e700 1 -- start start 2021-11-03T18:05:57.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.778+0000 7fd84f59e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8500602b0 0x7fd850106eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:05:57.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.778+0000 7fd84f59e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8500f9700 0x7fd8501073a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:05:57.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.778+0000 7fd84f59e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8501079a0 con 0x7fd8500f9700 2021-11-03T18:05:57.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.778+0000 7fd84f59e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8501081b0 con 0x7fd8500602b0 2021-11-03T18:05:57.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.778+0000 7fd84dd9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8500f9700 0x7fd8501073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:05:57.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.778+0000 7fd84dd9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8500f9700 0x7fd8501073a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32918/0 (socket says 172.21.15.27:32918) 2021-11-03T18:05:57.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.778+0000 7fd84dd9b700 1 -- 172.21.15.27:0/1112217649 learned_addr learned my addr 172.21.15.27:0/1112217649 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:05:57.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.778+0000 7fd84e59c700 1 --2- 172.21.15.27:0/1112217649 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8500602b0 0x7fd850106eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:05:57.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.778+0000 7fd84dd9b700 1 -- 172.21.15.27:0/1112217649 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8500602b0 msgr2=0x7fd850106eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:05:57.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.778+0000 7fd84dd9b700 1 --2- 172.21.15.27:0/1112217649 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8500602b0 0x7fd850106eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:57.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.778+0000 7fd84dd9b700 1 -- 172.21.15.27:0/1112217649 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd848009640 con 0x7fd8500f9700 2021-11-03T18:05:57.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.778+0000 7fd84dd9b700 1 --2- 172.21.15.27:0/1112217649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8500f9700 0x7fd8501073a0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0x7fd838002700 tx=0x7fd838009d60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:05:57.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.778+0000 7fd846ffd700 1 -- 172.21.15.27:0/1112217649 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd8380079c0 con 0x7fd8500f9700 2021-11-03T18:05:57.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.778+0000 7fd846ffd700 1 -- 172.21.15.27:0/1112217649 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fd83800f040 con 0x7fd8500f9700 2021-11-03T18:05:57.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.778+0000 7fd84f59e700 1 -- 172.21.15.27:0/1112217649 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd850108430 con 0x7fd8500f9700 2021-11-03T18:05:57.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.778+0000 7fd846ffd700 1 -- 172.21.15.27:0/1112217649 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd838014740 con 0x7fd8500f9700 2021-11-03T18:05:57.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.779+0000 7fd84f59e700 1 -- 172.21.15.27:0/1112217649 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd850108970 con 0x7fd8500f9700 2021-11-03T18:05:57.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.780+0000 7fd84f59e700 1 -- 172.21.15.27:0/1112217649 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd850059100 con 0x7fd8500f9700 2021-11-03T18:05:57.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.780+0000 7fd846ffd700 1 -- 172.21.15.27:0/1112217649 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fd83801b060 con 0x7fd8500f9700 2021-11-03T18:05:57.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.780+0000 7fd846ffd700 1 --2- 172.21.15.27:0/1112217649 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd83c064100 0x7fd83c0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:05:57.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.781+0000 7fd846ffd700 1 -- 172.21.15.27:0/1112217649 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fd838080600 con 0x7fd8500f9700 2021-11-03T18:05:57.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.781+0000 7fd84e59c700 1 --2- 172.21.15.27:0/1112217649 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd83c064100 0x7fd83c0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:05:57.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.781+0000 7fd84e59c700 1 --2- 172.21.15.27:0/1112217649 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd83c064100 0x7fd83c0665c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7fd84800b730 tx=0x7fd848004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:05:57.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.785+0000 7fd846ffd700 1 -- 172.21.15.27:0/1112217649 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fd8380528a0 con 0x7fd8500f9700 2021-11-03T18:05:57.961 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.963+0000 7fd84f59e700 1 -- 172.21.15.27:0/1112217649 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8500feeb0 con 0x7fd83c064100 2021-11-03T18:05:57.962 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.964+0000 7fd846ffd700 1 -- 172.21.15.27:0/1112217649 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fd8500feeb0 con 0x7fd83c064100 2021-11-03T18:05:57.964 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.966+0000 7fd84f59e700 1 -- 172.21.15.27:0/1112217649 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd83c064100 msgr2=0x7fd83c0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:05:57.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.966+0000 7fd84f59e700 1 --2- 172.21.15.27:0/1112217649 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd83c064100 0x7fd83c0665c0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7fd84800b730 tx=0x7fd848004d10).stop 2021-11-03T18:05:57.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.966+0000 7fd84f59e700 1 -- 172.21.15.27:0/1112217649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8500f9700 msgr2=0x7fd8501073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:05:57.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.966+0000 7fd84f59e700 1 --2- 172.21.15.27:0/1112217649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8500f9700 0x7fd8501073a0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0x7fd838002700 tx=0x7fd838009d60).stop 2021-11-03T18:05:57.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.967+0000 7fd84f59e700 1 -- 172.21.15.27:0/1112217649 shutdown_connections 2021-11-03T18:05:57.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.967+0000 7fd84f59e700 1 --2- 172.21.15.27:0/1112217649 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd83c064100 0x7fd83c0665c0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:57.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.967+0000 7fd84f59e700 1 --2- 172.21.15.27:0/1112217649 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd8500602b0 0x7fd850106eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:57.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.967+0000 7fd84f59e700 1 --2- 172.21.15.27:0/1112217649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd8500f9700 0x7fd8501073a0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:57.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.967+0000 7fd84f59e700 1 -- 172.21.15.27:0/1112217649 >> 172.21.15.27:0/1112217649 conn(0x7fd8500f0ba0 msgr2=0x7fd8500f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:05:57.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.967+0000 7fd84f59e700 1 -- 172.21.15.27:0/1112217649 shutdown_connections 2021-11-03T18:05:57.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:57.967+0000 7fd84f59e700 1 -- 172.21.15.27:0/1112217649 wait complete. 2021-11-03T18:05:57.974 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:05:58.067 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.067+0000 7ff938499700 1 -- 172.21.15.27:0/2804617984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9300f66e0 msgr2=0x7ff9300f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:05:58.067 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.067+0000 7ff938499700 1 --2- 172.21.15.27:0/2804617984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9300f66e0 0x7ff9300f6b00 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7ff920004660 tx=0x7ff920009b30).stop 2021-11-03T18:05:58.067 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.067+0000 7ff938499700 1 -- 172.21.15.27:0/2804617984 shutdown_connections 2021-11-03T18:05:58.068 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.067+0000 7ff938499700 1 --2- 172.21.15.27:0/2804617984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9300f7980 0x7ff9300f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:58.068 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.067+0000 7ff938499700 1 --2- 172.21.15.27:0/2804617984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9300f66e0 0x7ff9300f6b00 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:58.068 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.067+0000 7ff938499700 1 -- 172.21.15.27:0/2804617984 >> 172.21.15.27:0/2804617984 conn(0x7ff9300f1e40 msgr2=0x7ff9300f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:05:58.068 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.067+0000 7ff938499700 1 -- 172.21.15.27:0/2804617984 shutdown_connections 2021-11-03T18:05:58.068 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.067+0000 7ff938499700 1 -- 172.21.15.27:0/2804617984 wait complete. 2021-11-03T18:05:58.069 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.068+0000 7ff938499700 1 Processor -- start 2021-11-03T18:05:58.069 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.068+0000 7ff938499700 1 -- start start 2021-11-03T18:05:58.069 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.068+0000 7ff938499700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9300f66e0 0x7ff930106fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:05:58.069 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.068+0000 7ff938499700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9300f7980 0x7ff9301074a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:05:58.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.068+0000 7ff938499700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff930107a50 con 0x7ff9300f66e0 2021-11-03T18:05:58.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.068+0000 7ff938499700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff930107b90 con 0x7ff9300f7980 2021-11-03T18:05:58.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.068+0000 7ff936235700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9300f66e0 0x7ff930106fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:05:58.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.068+0000 7ff936235700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9300f66e0 0x7ff930106fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32924/0 (socket says 172.21.15.27:32924) 2021-11-03T18:05:58.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.068+0000 7ff936235700 1 -- 172.21.15.27:0/836363212 learned_addr learned my addr 172.21.15.27:0/836363212 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:05:58.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.068+0000 7ff935a34700 1 --2- 172.21.15.27:0/836363212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9300f7980 0x7ff9301074a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:05:58.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.068+0000 7ff936235700 1 -- 172.21.15.27:0/836363212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9300f7980 msgr2=0x7ff9301074a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:05:58.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.068+0000 7ff936235700 1 --2- 172.21.15.27:0/836363212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9300f7980 0x7ff9301074a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:58.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.068+0000 7ff936235700 1 -- 172.21.15.27:0/836363212 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff920005040 con 0x7ff9300f66e0 2021-11-03T18:05:58.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.068+0000 7ff935a34700 1 --2- 172.21.15.27:0/836363212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9300f7980 0x7ff9301074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:05:58.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.068+0000 7ff936235700 1 --2- 172.21.15.27:0/836363212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9300f66e0 0x7ff930106fb0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7ff920000c00 tx=0x7ff920009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:05:58.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.069+0000 7ff926ffd700 1 -- 172.21.15.27:0/836363212 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff920018070 con 0x7ff9300f66e0 2021-11-03T18:05:58.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.069+0000 7ff926ffd700 1 -- 172.21.15.27:0/836363212 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7ff920007d70 con 0x7ff9300f66e0 2021-11-03T18:05:58.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.069+0000 7ff926ffd700 1 -- 172.21.15.27:0/836363212 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff92001c620 con 0x7ff9300f66e0 2021-11-03T18:05:58.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.069+0000 7ff938499700 1 -- 172.21.15.27:0/836363212 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9301084d0 con 0x7ff9300f66e0 2021-11-03T18:05:58.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.069+0000 7ff938499700 1 -- 172.21.15.27:0/836363212 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff930108a30 con 0x7ff9300f66e0 2021-11-03T18:05:58.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.069+0000 7ff938499700 1 -- 172.21.15.27:0/836363212 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff930045ad0 con 0x7ff9300f66e0 2021-11-03T18:05:58.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.070+0000 7ff926ffd700 1 -- 172.21.15.27:0/836363212 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7ff92001d420 con 0x7ff9300f66e0 2021-11-03T18:05:58.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.070+0000 7ff926ffd700 1 --2- 172.21.15.27:0/836363212 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff91c064090 0x7ff91c066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:05:58.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.071+0000 7ff926ffd700 1 -- 172.21.15.27:0/836363212 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7ff9200825c0 con 0x7ff9300f66e0 2021-11-03T18:05:58.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.071+0000 7ff935a34700 1 --2- 172.21.15.27:0/836363212 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff91c064090 0x7ff91c066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:05:58.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.071+0000 7ff935a34700 1 --2- 172.21.15.27:0/836363212 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff91c064090 0x7ff91c066550 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7ff92c0041d0 tx=0x7ff92c009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:05:58.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.075+0000 7ff926ffd700 1 -- 172.21.15.27:0/836363212 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7ff920054980 con 0x7ff9300f66e0 2021-11-03T18:05:58.249 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.251+0000 7ff938499700 1 -- 172.21.15.27:0/836363212 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9300fefe0 con 0x7ff91c064090 2021-11-03T18:05:58.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.266+0000 7ff926ffd700 1 -- 172.21.15.27:0/836363212 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7ff9300fefe0 con 0x7ff91c064090 2021-11-03T18:05:58.264 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:05:58.265 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (57m) 6m ago 58m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:05:58.265 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (49m) 6m ago 58m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:05:58.265 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (49m) 6m ago 58m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:05:58.265 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (57m) 6m ago 58m 41.8M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:05:58.266 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (51m) 6m ago 53m 26.0M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:05:58.266 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (53m) 6m ago 53m 3845M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:05:58.266 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (49m) 6m ago 53m 25.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:05:58.266 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (51m) 6m ago 54m 26.8M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:05:58.267 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (51m) 6m ago 60m 554M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:05:58.267 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (50m) 6m ago 57m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:05:58.267 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (50m) 6m ago 60m 235M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:05:58.267 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (49m) 6m ago 57m 112M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:05:58.268 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (58m) 6m ago 58m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:05:58.268 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (57m) 6m ago 57m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:05:58.268 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (48m) 6m ago 57m 351M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:05:58.268 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (48m) 6m ago 56m 383M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:05:58.269 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (47m) 6m ago 56m 284M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:05:58.269 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (47m) 6m ago 56m 404M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:05:58.269 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (46m) 6m ago 55m 322M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:05:58.269 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (46m) 6m ago 55m 327M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:05:58.270 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (57m) 6m ago 58m 55.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:05:58.271 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.268+0000 7ff938499700 1 -- 172.21.15.27:0/836363212 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff91c064090 msgr2=0x7ff91c066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:05:58.271 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.268+0000 7ff938499700 1 --2- 172.21.15.27:0/836363212 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff91c064090 0x7ff91c066550 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7ff92c0041d0 tx=0x7ff92c009040).stop 2021-11-03T18:05:58.271 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.268+0000 7ff938499700 1 -- 172.21.15.27:0/836363212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9300f66e0 msgr2=0x7ff930106fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:05:58.271 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.268+0000 7ff938499700 1 --2- 172.21.15.27:0/836363212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9300f66e0 0x7ff930106fb0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7ff920000c00 tx=0x7ff920009b30).stop 2021-11-03T18:05:58.272 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.268+0000 7ff938499700 1 -- 172.21.15.27:0/836363212 shutdown_connections 2021-11-03T18:05:58.272 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.268+0000 7ff938499700 1 --2- 172.21.15.27:0/836363212 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff91c064090 0x7ff91c066550 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:58.272 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.268+0000 7ff938499700 1 --2- 172.21.15.27:0/836363212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9300f66e0 0x7ff930106fb0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:58.273 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.268+0000 7ff938499700 1 --2- 172.21.15.27:0/836363212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9300f7980 0x7ff9301074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:58.273 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.268+0000 7ff938499700 1 -- 172.21.15.27:0/836363212 >> 172.21.15.27:0/836363212 conn(0x7ff9300f1e40 msgr2=0x7ff9300fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:05:58.273 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.268+0000 7ff938499700 1 -- 172.21.15.27:0/836363212 shutdown_connections 2021-11-03T18:05:58.273 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.268+0000 7ff938499700 1 -- 172.21.15.27:0/836363212 wait complete. 2021-11-03T18:05:58.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.368+0000 7f73b0490700 1 -- 172.21.15.27:0/569596488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73a80f58b0 msgr2=0x7f73a80f5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:05:58.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.368+0000 7f73b0490700 1 --2- 172.21.15.27:0/569596488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73a80f58b0 0x7f73a80f5d30 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 rx=0x7f739c004660 tx=0x7f739c009b30).stop 2021-11-03T18:05:58.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.368+0000 7f73b0490700 1 -- 172.21.15.27:0/569596488 shutdown_connections 2021-11-03T18:05:58.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.368+0000 7f73b0490700 1 --2- 172.21.15.27:0/569596488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73a80f58b0 0x7f73a80f5d30 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:58.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.368+0000 7f73b0490700 1 --2- 172.21.15.27:0/569596488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73a80f6bb0 0x7f73a80f6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:58.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.368+0000 7f73b0490700 1 -- 172.21.15.27:0/569596488 >> 172.21.15.27:0/569596488 conn(0x7f73a80f04c0 msgr2=0x7f73a80f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:05:58.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.368+0000 7f73b0490700 1 -- 172.21.15.27:0/569596488 shutdown_connections 2021-11-03T18:05:58.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.368+0000 7f73b0490700 1 -- 172.21.15.27:0/569596488 wait complete. 2021-11-03T18:05:58.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.369+0000 7f73b0490700 1 Processor -- start 2021-11-03T18:05:58.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.369+0000 7f73b0490700 1 -- start start 2021-11-03T18:05:58.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.369+0000 7f73b0490700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73a80f58b0 0x7f73a8109c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:05:58.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.369+0000 7f73b0490700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73a80f6bb0 0x7f73a810c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:05:58.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.369+0000 7f73b0490700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a80f5560 con 0x7f73a80f6bb0 2021-11-03T18:05:58.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.369+0000 7f73b0490700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73a80f56a0 con 0x7f73a80f58b0 2021-11-03T18:05:58.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.369+0000 7f73ada2b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73a80f6bb0 0x7f73a810c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:05:58.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.369+0000 7f73ada2b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73a80f6bb0 0x7f73a810c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32936/0 (socket says 172.21.15.27:32936) 2021-11-03T18:05:58.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.369+0000 7f73ada2b700 1 -- 172.21.15.27:0/2320987573 learned_addr learned my addr 172.21.15.27:0/2320987573 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:05:58.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.369+0000 7f73ada2b700 1 -- 172.21.15.27:0/2320987573 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73a80f58b0 msgr2=0x7f73a8109c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:05:58.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.369+0000 7f73ada2b700 1 --2- 172.21.15.27:0/2320987573 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73a80f58b0 0x7f73a8109c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:05:58.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.369+0000 7f73ada2b700 1 -- 172.21.15.27:0/2320987573 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f739c005040 con 0x7f73a80f6bb0 2021-11-03T18:05:58.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.369+0000 7f73ada2b700 1 --2- 172.21.15.27:0/2320987573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73a80f6bb0 0x7f73a810c190 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7f739c004f60 tx=0x7f739c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:05:58.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.369+0000 7f73a6ffd700 1 -- 172.21.15.27:0/2320987573 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f739c018070 con 0x7f73a80f6bb0 2021-11-03T18:05:58.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.369+0000 7f73a6ffd700 1 -- 172.21.15.27:0/2320987573 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f739c007ba0 con 0x7f73a80f6bb0 2021-11-03T18:05:58.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.370+0000 7f73b0490700 1 -- 172.21.15.27:0/2320987573 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73a810c6e0 con 0x7f73a80f6bb0 2021-11-03T18:05:58.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.370+0000 7f73a6ffd700 1 -- 172.21.15.27:0/2320987573 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f739c01c620 con 0x7f73a80f6bb0 2021-11-03T18:05:58.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.370+0000 7f73b0490700 1 -- 172.21.15.27:0/2320987573 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73a810cc40 con 0x7f73a80f6bb0 2021-11-03T18:05:58.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.371+0000 7f73b0490700 1 -- 172.21.15.27:0/2320987573 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73a8045ad0 con 0x7f73a80f6bb0 2021-11-03T18:05:58.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.371+0000 7f73a6ffd700 1 -- 172.21.15.27:0/2320987573 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f739c01c8e0 con 0x7f73a80f6bb0 2021-11-03T18:05:58.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.371+0000 7f73a6ffd700 1 --2- 172.21.15.27:0/2320987573 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f73900640b0 0x7f7390066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:05:58.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.371+0000 7f73a6ffd700 1 -- 172.21.15.27:0/2320987573 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f739c082720 con 0x7f73a80f6bb0 2021-11-03T18:05:58.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.372+0000 7f73ae22c700 1 --2- 172.21.15.27:0/2320987573 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f73900640b0 0x7f7390066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:05:58.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.372+0000 7f73ae22c700 1 --2- 172.21.15.27:0/2320987573 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f73900640b0 0x7f7390066570 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f73a80f51e0 tx=0x7f73a0004960).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:05:58.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.375+0000 7f73a6ffd700 1 -- 172.21.15.27:0/2320987573 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f739c054a20 con 0x7f73a80f6bb0 2021-11-03T18:05:58.606 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.608+0000 7f73b0490700 1 -- 172.21.15.27:0/2320987573 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f73a80586e0 con 0x7f73a80f6bb0 2021-11-03T18:05:58.607 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.608+0000 7f73a6ffd700 1 -- 172.21.15.27:0/2320987573 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f739c054610 con 0x7f73a80f6bb0 2021-11-03T18:05:58.607 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:05:58.608 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:05:58.608 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:05:58.608 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:05:58.608 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:05:58.608 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:05:58.609 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:05:58.609 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:05:58.609 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:05:58.609 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:05:58.610 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:05:58.610 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:05:58.610 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:05:58.610 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:05:58.611 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:05:58.611 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:05:58.611 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:05:58.611 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:05:58.612 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.610+0000 7f73b0490700 1 -- 172.21.15.27:0/2320987573 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f73900640b0 msgr2=0x7f7390066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:05:58.612 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.610+0000 7f73b0490700 1 --2- 172.21.15.27:0/2320987573 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f73900640b0 0x7f7390066570 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 rx=0x7f73a80f51e0 tx=0x7f73a0004960).stop 2021-11-03T18:05:58.613 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.610+0000 7f73b0490700 1 -- 172.21.15.27:0/2320987573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73a80f6bb0 msgr2=0x7f73a810c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:05:58.613 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.610+0000 7f73b0490700 1 --2- 172.21.15.27:0/2320987573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73a80f6bb0 0x7f73a810c190 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7f739c004f60 tx=0x7f739c009b30).stop 2021-11-03T18:05:58.613 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.610+0000 7f73b0490700 1 -- 172.21.15.27:0/2320987573 shutdown_connections 2021-11-03T18:05:58.613 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.610+0000 7f73b0490700 1 --2- 172.21.15.27:0/2320987573 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f73900640b0 0x7f7390066570 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:58.614 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.610+0000 7f73b0490700 1 --2- 172.21.15.27:0/2320987573 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73a80f58b0 0x7f73a8109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:05:58.614 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.610+0000 7f73b0490700 1 --2- 172.21.15.27:0/2320987573 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73a80f6bb0 0x7f73a810c190 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:58.614 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.610+0000 7f73b0490700 1 -- 172.21.15.27:0/2320987573 >> 172.21.15.27:0/2320987573 conn(0x7f73a80f04c0 msgr2=0x7f73a80fa310 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:05:58.614 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.610+0000 7f73b0490700 1 -- 172.21.15.27:0/2320987573 shutdown_connections 2021-11-03T18:05:58.615 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.610+0000 7f73b0490700 1 -- 172.21.15.27:0/2320987573 wait complete. 2021-11-03T18:05:58.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.710+0000 7fe7e1556700 1 -- 172.21.15.27:0/3971616440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7dc060310 msgr2=0x7fe7dc0f9140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:05:58.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.710+0000 7fe7e1556700 1 --2- 172.21.15.27:0/3971616440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7dc060310 0x7fe7dc0f9140 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 rx=0x7fe7cc004660 tx=0x7fe7cc009b30).stop 2021-11-03T18:05:58.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.710+0000 7fe7e1556700 1 -- 172.21.15.27:0/3971616440 shutdown_connections 2021-11-03T18:05:58.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.710+0000 7fe7e1556700 1 --2- 172.21.15.27:0/3971616440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7dc0f96c0 0x7fe7dc0fbb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:58.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.710+0000 7fe7e1556700 1 --2- 172.21.15.27:0/3971616440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7dc060310 0x7fe7dc0f9140 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:58.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.710+0000 7fe7e1556700 1 -- 172.21.15.27:0/3971616440 >> 172.21.15.27:0/3971616440 conn(0x7fe7dc0f0bc0 msgr2=0x7fe7dc0f2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:05:58.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.710+0000 7fe7e1556700 1 -- 172.21.15.27:0/3971616440 shutdown_connections 2021-11-03T18:05:58.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.710+0000 7fe7e1556700 1 -- 172.21.15.27:0/3971616440 wait complete. 2021-11-03T18:05:58.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.711+0000 7fe7e1556700 1 Processor -- start 2021-11-03T18:05:58.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.711+0000 7fe7e1556700 1 -- start start 2021-11-03T18:05:58.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.711+0000 7fe7e1556700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7dc060310 0x7fe7dc0f6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:05:58.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.711+0000 7fe7e1556700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7dc0f96c0 0x7fe7dc0f5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:05:58.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.711+0000 7fe7e1556700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7dc0f55a0 con 0x7fe7dc060310 2021-11-03T18:05:58.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.711+0000 7fe7e1556700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7dc0f56e0 con 0x7fe7dc0f96c0 2021-11-03T18:05:58.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.711+0000 7fe7daffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7dc060310 0x7fe7dc0f6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:05:58.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.711+0000 7fe7daffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7dc060310 0x7fe7dc0f6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32944/0 (socket says 172.21.15.27:32944) 2021-11-03T18:05:58.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.711+0000 7fe7daffd700 1 -- 172.21.15.27:0/435544781 learned_addr learned my addr 172.21.15.27:0/435544781 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:05:58.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.711+0000 7fe7da7fc700 1 --2- 172.21.15.27:0/435544781 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7dc0f96c0 0x7fe7dc0f5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:05:58.715 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.712+0000 7fe7daffd700 1 -- 172.21.15.27:0/435544781 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7dc0f96c0 msgr2=0x7fe7dc0f5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:05:58.715 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.712+0000 7fe7daffd700 1 --2- 172.21.15.27:0/435544781 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7dc0f96c0 0x7fe7dc0f5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:58.715 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.712+0000 7fe7daffd700 1 -- 172.21.15.27:0/435544781 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7cc005040 con 0x7fe7dc060310 2021-11-03T18:05:58.715 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.712+0000 7fe7daffd700 1 --2- 172.21.15.27:0/435544781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7dc060310 0x7fe7dc0f6bc0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7fe7cc009990 tx=0x7fe7cc009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:05:58.716 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.712+0000 7fe7c37fe700 1 -- 172.21.15.27:0/435544781 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7cc018070 con 0x7fe7dc060310 2021-11-03T18:05:58.716 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.712+0000 7fe7c37fe700 1 -- 172.21.15.27:0/435544781 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fe7cc007ba0 con 0x7fe7dc060310 2021-11-03T18:05:58.716 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.712+0000 7fe7e1556700 1 -- 172.21.15.27:0/435544781 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7dc0f5960 con 0x7fe7dc060310 2021-11-03T18:05:58.716 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.712+0000 7fe7c37fe700 1 -- 172.21.15.27:0/435544781 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7cc01c3c0 con 0x7fe7dc060310 2021-11-03T18:05:58.716 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.712+0000 7fe7e1556700 1 -- 172.21.15.27:0/435544781 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7dc0f5ec0 con 0x7fe7dc060310 2021-11-03T18:05:58.717 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.713+0000 7fe7e1556700 1 -- 172.21.15.27:0/435544781 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7dc045ad0 con 0x7fe7dc060310 2021-11-03T18:05:58.718 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.713+0000 7fe7c37fe700 1 -- 172.21.15.27:0/435544781 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fe7cc01d420 con 0x7fe7dc060310 2021-11-03T18:05:58.718 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.714+0000 7fe7c37fe700 1 --2- 172.21.15.27:0/435544781 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe7c4064100 0x7fe7c40665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:05:58.718 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.714+0000 7fe7c37fe700 1 -- 172.21.15.27:0/435544781 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fe7cc082470 con 0x7fe7dc060310 2021-11-03T18:05:58.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.714+0000 7fe7da7fc700 1 --2- 172.21.15.27:0/435544781 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe7c4064100 0x7fe7c40665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:05:58.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.714+0000 7fe7da7fc700 1 --2- 172.21.15.27:0/435544781 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe7c4064100 0x7fe7c40665c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7fe7dc0600e0 tx=0x7fe7d000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:05:58.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.718+0000 7fe7c37fe700 1 -- 172.21.15.27:0/435544781 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fe7cc054710 con 0x7fe7dc060310 2021-11-03T18:05:58.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:58 smithi115 ceph-mon[65453]: pgmap v1539: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:58.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:58 smithi027 ceph-mon[97244]: pgmap v1539: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:05:58.916 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.918+0000 7fe7e1556700 1 -- 172.21.15.27:0/435544781 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe7dc0fede0 con 0x7fe7dc060310 2021-11-03T18:05:58.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.919+0000 7fe7c37fe700 1 -- 172.21.15.27:0/435544781 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7fe7cc054300 con 0x7fe7dc060310 2021-11-03T18:05:58.917 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:05:58.917 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:05:58.918 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:05:58.918 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:05:58.918 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:05:58.918 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:05:58.919 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:05:58.919 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:05:58.919 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:05:58.919 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:05:58.920 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:05:58.920 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:05:58.920 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:05:58.920 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:05:58.920 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:05:58.921 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:05:58.921 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:05:58.921 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:05:58.921 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:05:58.922 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:05:58.922 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:05:58.922 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:05:58.922 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:05:58.922 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:05:58.923 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:05:58.923 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:05:58.923 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:05:58.923 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:05:58.924 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:05:58.924 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:05:58.924 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:05:58.924 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:05:58.924 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:05:58.925 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:05:58.925 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:05:58.925 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:05:58.925 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:05:58.925 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:05:58.926 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:05:58.927 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.920+0000 7fe7e1556700 1 -- 172.21.15.27:0/435544781 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe7c4064100 msgr2=0x7fe7c40665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:05:58.927 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.920+0000 7fe7e1556700 1 --2- 172.21.15.27:0/435544781 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe7c4064100 0x7fe7c40665c0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7fe7dc0600e0 tx=0x7fe7d000b040).stop 2021-11-03T18:05:58.927 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.920+0000 7fe7e1556700 1 -- 172.21.15.27:0/435544781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7dc060310 msgr2=0x7fe7dc0f6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:05:58.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.920+0000 7fe7e1556700 1 --2- 172.21.15.27:0/435544781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7dc060310 0x7fe7dc0f6bc0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7fe7cc009990 tx=0x7fe7cc009b30).stop 2021-11-03T18:05:58.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.920+0000 7fe7e1556700 1 -- 172.21.15.27:0/435544781 shutdown_connections 2021-11-03T18:05:58.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.920+0000 7fe7e1556700 1 --2- 172.21.15.27:0/435544781 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe7c4064100 0x7fe7c40665c0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:58.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.920+0000 7fe7e1556700 1 --2- 172.21.15.27:0/435544781 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7dc060310 0x7fe7dc0f6bc0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:58.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.920+0000 7fe7e1556700 1 --2- 172.21.15.27:0/435544781 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7dc0f96c0 0x7fe7dc0f5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:05:58.929 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.920+0000 7fe7e1556700 1 -- 172.21.15.27:0/435544781 >> 172.21.15.27:0/435544781 conn(0x7fe7dc0f0bc0 msgr2=0x7fe7dc0f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:05:58.929 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.921+0000 7fe7e1556700 1 -- 172.21.15.27:0/435544781 shutdown_connections 2021-11-03T18:05:58.929 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:05:58.921+0000 7fe7e1556700 1 -- 172.21.15.27:0/435544781 wait complete. 2021-11-03T18:05:58.929 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:05:59.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:59 smithi115 ceph-mon[65453]: from='client.35688 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:05:59.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:59 smithi115 ceph-mon[65453]: from='client.35692 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:05:59.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:59 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2320987573' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:59.741 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:05:59 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/435544781' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:05:59.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:59 smithi027 ceph-mon[97244]: from='client.35688 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:05:59.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:59 smithi027 ceph-mon[97244]: from='client.35692 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:05:59.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:59 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2320987573' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:05:59.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:05:59 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/435544781' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:06:00.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:00 smithi115 ceph-mon[65453]: pgmap v1540: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:00.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3002.431708 secs 2021-11-03T18:06:00.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:06:00.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:06:00.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:06:00.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:00.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:00.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:00.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:00.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:00 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:06:00.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:00 smithi027 ceph-mon[97244]: pgmap v1540: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:00.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3002.431708 secs 2021-11-03T18:06:00.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:06:00.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:06:00.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:06:00.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:00.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:00.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:00.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:00.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:00 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:06:02.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:02 smithi115 ceph-mon[65453]: pgmap v1541: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:02.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:02 smithi027 ceph-mon[97244]: pgmap v1541: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:04.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:04 smithi115 ceph-mon[65453]: pgmap v1542: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:04.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:04 smithi027 ceph-mon[97244]: pgmap v1542: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:05.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3007.431847 secs 2021-11-03T18:06:05.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3007.431847 secs 2021-11-03T18:06:06.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:06 smithi115 ceph-mon[65453]: pgmap v1543: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:06.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:06 smithi027 ceph-mon[97244]: pgmap v1543: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:08.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:08 smithi115 ceph-mon[65453]: pgmap v1544: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:08.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:08 smithi027 ceph-mon[97244]: pgmap v1544: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:10.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:10 smithi115 ceph-mon[65453]: pgmap v1545: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:10.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3012.431990 secs 2021-11-03T18:06:10.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:06:10.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:06:10.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:06:10.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:10.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:10.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:10.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:10.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:10 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:06:10.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:10 smithi027 ceph-mon[97244]: pgmap v1545: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:10.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3012.431990 secs 2021-11-03T18:06:10.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:06:10.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:06:10.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:06:10.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:10.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:10.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:10.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:10.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:10 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:06:12.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:12 smithi115 ceph-mon[65453]: pgmap v1546: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:12.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:12 smithi027 ceph-mon[97244]: pgmap v1546: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:14.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:14 smithi115 ceph-mon[65453]: pgmap v1547: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:14.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:14 smithi027 ceph-mon[97244]: pgmap v1547: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:15.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3017.432124 secs 2021-11-03T18:06:15.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3017.432124 secs 2021-11-03T18:06:16.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:16 smithi115 ceph-mon[65453]: pgmap v1548: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:16.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:16 smithi027 ceph-mon[97244]: pgmap v1548: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:18.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:18 smithi115 ceph-mon[65453]: pgmap v1549: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:18.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:18 smithi027 ceph-mon[97244]: pgmap v1549: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:20.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:20 smithi115 ceph-mon[65453]: pgmap v1550: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:20.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3022.432266 secs 2021-11-03T18:06:20.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:06:20.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:06:20.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:06:20.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:20.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:20.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:20.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:20.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:20 smithi027 ceph-mon[97244]: pgmap v1550: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:20.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3022.432266 secs 2021-11-03T18:06:20.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:06:20.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:06:20.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:06:20.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:20.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:20.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:20.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:21.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:21 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:06:21.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:21 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:06:22.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:22 smithi115 ceph-mon[65453]: pgmap v1551: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:22.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:22 smithi027 ceph-mon[97244]: pgmap v1551: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:24.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:24 smithi115 ceph-mon[65453]: pgmap v1552: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:24.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:24 smithi027 ceph-mon[97244]: pgmap v1552: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:25.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3027.432307 secs 2021-11-03T18:06:25.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3027.432307 secs 2021-11-03T18:06:26.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:26 smithi027 ceph-mon[97244]: pgmap v1553: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:26.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:26 smithi115 ceph-mon[65453]: pgmap v1553: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:28.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:28 smithi027 ceph-mon[97244]: pgmap v1554: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:28.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:28 smithi115 ceph-mon[65453]: pgmap v1554: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:29.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.035+0000 7fc0b3de7700 1 -- 172.21.15.27:0/1716013451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0ac0f7d00 msgr2=0x7fc0ac0f8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:06:29.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.035+0000 7fc0b3de7700 1 --2- 172.21.15.27:0/1716013451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0ac0f7d00 0x7fc0ac0f8120 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7fc0a8004660 tx=0x7fc0a8009b30).stop 2021-11-03T18:06:29.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.035+0000 7fc0b3de7700 1 -- 172.21.15.27:0/1716013451 shutdown_connections 2021-11-03T18:06:29.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.035+0000 7fc0b3de7700 1 --2- 172.21.15.27:0/1716013451 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0ac0f8610 0x7fc0ac0f4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:06:29.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.035+0000 7fc0b3de7700 1 --2- 172.21.15.27:0/1716013451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0ac0f7d00 0x7fc0ac0f8120 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:06:29.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.035+0000 7fc0b3de7700 1 -- 172.21.15.27:0/1716013451 >> 172.21.15.27:0/1716013451 conn(0x7fc0ac0f0540 msgr2=0x7fc0ac0f2960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:06:29.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.035+0000 7fc0b3de7700 1 -- 172.21.15.27:0/1716013451 shutdown_connections 2021-11-03T18:06:29.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.035+0000 7fc0b3de7700 1 -- 172.21.15.27:0/1716013451 wait complete. 2021-11-03T18:06:29.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.036+0000 7fc0b3de7700 1 Processor -- start 2021-11-03T18:06:29.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.036+0000 7fc0b3de7700 1 -- start start 2021-11-03T18:06:29.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.036+0000 7fc0b3de7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0ac0f7d00 0x7fc0ac106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:06:29.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.036+0000 7fc0b3de7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0ac0f8610 0x7fc0ac107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:06:29.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.036+0000 7fc0b3de7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ac107a10 con 0x7fc0ac0f8610 2021-11-03T18:06:29.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.036+0000 7fc0b3de7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0ac107b50 con 0x7fc0ac0f7d00 2021-11-03T18:06:29.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.036+0000 7fc0b1382700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0ac0f8610 0x7fc0ac107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:06:29.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.036+0000 7fc0b1382700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0ac0f8610 0x7fc0ac107460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32964/0 (socket says 172.21.15.27:32964) 2021-11-03T18:06:29.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.036+0000 7fc0b1382700 1 -- 172.21.15.27:0/48651718 learned_addr learned my addr 172.21.15.27:0/48651718 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:06:29.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.037+0000 7fc0b1b83700 1 --2- 172.21.15.27:0/48651718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0ac0f7d00 0x7fc0ac106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:06:29.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.037+0000 7fc0b1382700 1 -- 172.21.15.27:0/48651718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0ac0f7d00 msgr2=0x7fc0ac106f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:06:29.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.037+0000 7fc0b1382700 1 --2- 172.21.15.27:0/48651718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0ac0f7d00 0x7fc0ac106f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:06:29.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.037+0000 7fc0b1382700 1 -- 172.21.15.27:0/48651718 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a8005040 con 0x7fc0ac0f8610 2021-11-03T18:06:29.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.037+0000 7fc0b1b83700 1 --2- 172.21.15.27:0/48651718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0ac0f7d00 0x7fc0ac106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T18:06:29.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.037+0000 7fc0b1382700 1 --2- 172.21.15.27:0/48651718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0ac0f8610 0x7fc0ac107460 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0x7fc0a00026d0 tx=0x7fc0a0009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:06:29.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.037+0000 7fc09e7fc700 1 -- 172.21.15.27:0/48651718 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0a0009320 con 0x7fc0ac0f8610 2021-11-03T18:06:29.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.037+0000 7fc09e7fc700 1 -- 172.21.15.27:0/48651718 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fc0a000f040 con 0x7fc0ac0f8610 2021-11-03T18:06:29.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.037+0000 7fc0b3de7700 1 -- 172.21.15.27:0/48651718 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0ac1084f0 con 0x7fc0ac0f8610 2021-11-03T18:06:29.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.037+0000 7fc09e7fc700 1 -- 172.21.15.27:0/48651718 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0a00147e0 con 0x7fc0ac0f8610 2021-11-03T18:06:29.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.037+0000 7fc0b3de7700 1 -- 172.21.15.27:0/48651718 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0ac108ab0 con 0x7fc0ac0f8610 2021-11-03T18:06:29.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.038+0000 7fc0b3de7700 1 -- 172.21.15.27:0/48651718 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0ac045ad0 con 0x7fc0ac0f8610 2021-11-03T18:06:29.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.038+0000 7fc09e7fc700 1 -- 172.21.15.27:0/48651718 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fc0a0009480 con 0x7fc0ac0f8610 2021-11-03T18:06:29.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.039+0000 7fc09e7fc700 1 --2- 172.21.15.27:0/48651718 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc0980640b0 0x7fc098066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:06:29.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.039+0000 7fc09e7fc700 1 -- 172.21.15.27:0/48651718 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fc0a0080f20 con 0x7fc0ac0f8610 2021-11-03T18:06:29.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.039+0000 7fc0b1b83700 1 --2- 172.21.15.27:0/48651718 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc0980640b0 0x7fc098066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:06:29.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.039+0000 7fc0b1b83700 1 --2- 172.21.15.27:0/48651718 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc0980640b0 0x7fc098066570 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7fc0a8009da0 tx=0x7fc0a8009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:06:29.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.043+0000 7fc09e7fc700 1 -- 172.21.15.27:0/48651718 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fc0a00addf0 con 0x7fc0ac0f8610 2021-11-03T18:06:29.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.222+0000 7fc0b3de7700 1 -- 172.21.15.27:0/48651718 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc0ac108d30 con 0x7fc0980640b0 2021-11-03T18:06:29.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.223+0000 7fc09e7fc700 1 -- 172.21.15.27:0/48651718 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fc0ac108d30 con 0x7fc0980640b0 2021-11-03T18:06:29.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.225+0000 7fc0b3de7700 1 -- 172.21.15.27:0/48651718 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc0980640b0 msgr2=0x7fc098066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:06:29.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.225+0000 7fc0b3de7700 1 --2- 172.21.15.27:0/48651718 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc0980640b0 0x7fc098066570 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7fc0a8009da0 tx=0x7fc0a8009b30).stop 2021-11-03T18:06:29.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.225+0000 7fc0b3de7700 1 -- 172.21.15.27:0/48651718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0ac0f8610 msgr2=0x7fc0ac107460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:06:29.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.225+0000 7fc0b3de7700 1 --2- 172.21.15.27:0/48651718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0ac0f8610 0x7fc0ac107460 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0x7fc0a00026d0 tx=0x7fc0a0009de0).stop 2021-11-03T18:06:29.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.225+0000 7fc0b3de7700 1 -- 172.21.15.27:0/48651718 shutdown_connections 2021-11-03T18:06:29.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.225+0000 7fc0b3de7700 1 --2- 172.21.15.27:0/48651718 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc0980640b0 0x7fc098066570 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:06:29.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.225+0000 7fc0b3de7700 1 --2- 172.21.15.27:0/48651718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0ac0f7d00 0x7fc0ac106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:06:29.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.225+0000 7fc0b3de7700 1 --2- 172.21.15.27:0/48651718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0ac0f8610 0x7fc0ac107460 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:06:29.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.225+0000 7fc0b3de7700 1 -- 172.21.15.27:0/48651718 >> 172.21.15.27:0/48651718 conn(0x7fc0ac0f0540 msgr2=0x7fc0ac0fb590 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:06:29.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.225+0000 7fc0b3de7700 1 -- 172.21.15.27:0/48651718 shutdown_connections 2021-11-03T18:06:29.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.225+0000 7fc0b3de7700 1 -- 172.21.15.27:0/48651718 wait complete. 2021-11-03T18:06:29.232 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:06:29.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.325+0000 7f11a088b700 1 -- 172.21.15.27:0/478993715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f119c0f55c0 msgr2=0x7f119c0f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:06:29.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.325+0000 7f11a088b700 1 --2- 172.21.15.27:0/478993715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f119c0f55c0 0x7f119c0f59e0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7f1184004660 tx=0x7f1184009b30).stop 2021-11-03T18:06:29.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.325+0000 7f11a088b700 1 -- 172.21.15.27:0/478993715 shutdown_connections 2021-11-03T18:06:29.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.325+0000 7f11a088b700 1 --2- 172.21.15.27:0/478993715 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f119c0f5ed0 0x7f119c0f6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:06:29.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.325+0000 7f11a088b700 1 --2- 172.21.15.27:0/478993715 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f119c0f55c0 0x7f119c0f59e0 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:06:29.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.325+0000 7f11a088b700 1 -- 172.21.15.27:0/478993715 >> 172.21.15.27:0/478993715 conn(0x7f119c0f0de0 msgr2=0x7f119c0f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:06:29.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.325+0000 7f11a088b700 1 -- 172.21.15.27:0/478993715 shutdown_connections 2021-11-03T18:06:29.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.325+0000 7f11a088b700 1 -- 172.21.15.27:0/478993715 wait complete. 2021-11-03T18:06:29.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.326+0000 7f11a088b700 1 Processor -- start 2021-11-03T18:06:29.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.326+0000 7f11a088b700 1 -- start start 2021-11-03T18:06:29.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.326+0000 7f11a088b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f119c0f55c0 0x7f119c109bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:06:29.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.326+0000 7f11a088b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f119c0f5ed0 0x7f119c10c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:06:29.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.326+0000 7f11a088b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119c060540 con 0x7f119c0f5ed0 2021-11-03T18:06:29.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.326+0000 7f11a088b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f119c060680 con 0x7f119c0f55c0 2021-11-03T18:06:29.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.326+0000 7f1199d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f119c0f5ed0 0x7f119c10c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:06:29.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.326+0000 7f1199d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f119c0f5ed0 0x7f119c10c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32974/0 (socket says 172.21.15.27:32974) 2021-11-03T18:06:29.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.326+0000 7f1199d9b700 1 -- 172.21.15.27:0/2604484164 learned_addr learned my addr 172.21.15.27:0/2604484164 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:06:29.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.326+0000 7f1199d9b700 1 -- 172.21.15.27:0/2604484164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f119c0f55c0 msgr2=0x7f119c109bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:06:29.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.326+0000 7f1199d9b700 1 --2- 172.21.15.27:0/2604484164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f119c0f55c0 0x7f119c109bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:06:29.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.326+0000 7f1199d9b700 1 -- 172.21.15.27:0/2604484164 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1184005040 con 0x7f119c0f5ed0 2021-11-03T18:06:29.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.327+0000 7f1199d9b700 1 --2- 172.21.15.27:0/2604484164 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f119c0f5ed0 0x7f119c10c0d0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 rx=0x7f118c00e7f0 tx=0x7f118c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:06:29.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.327+0000 7f1192ffd700 1 -- 172.21.15.27:0/2604484164 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f118c0092e0 con 0x7f119c0f5ed0 2021-11-03T18:06:29.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.327+0000 7f1192ffd700 1 -- 172.21.15.27:0/2604484164 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f118c00f040 con 0x7f119c0f5ed0 2021-11-03T18:06:29.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.327+0000 7f1192ffd700 1 -- 172.21.15.27:0/2604484164 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f118c014750 con 0x7f119c0f5ed0 2021-11-03T18:06:29.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.327+0000 7f11a088b700 1 -- 172.21.15.27:0/2604484164 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f119c10c680 con 0x7f119c0f5ed0 2021-11-03T18:06:29.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.327+0000 7f11a088b700 1 -- 172.21.15.27:0/2604484164 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f119c10cc40 con 0x7f119c0f5ed0 2021-11-03T18:06:29.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.328+0000 7f11a088b700 1 -- 172.21.15.27:0/2604484164 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f119c045ad0 con 0x7f119c0f5ed0 2021-11-03T18:06:29.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.328+0000 7f1192ffd700 1 -- 172.21.15.27:0/2604484164 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f118c01b030 con 0x7f119c0f5ed0 2021-11-03T18:06:29.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.329+0000 7f1192ffd700 1 --2- 172.21.15.27:0/2604484164 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f11880640b0 0x7f1188066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:06:29.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.329+0000 7f1192ffd700 1 -- 172.21.15.27:0/2604484164 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f118c0806d0 con 0x7f119c0f5ed0 2021-11-03T18:06:29.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.329+0000 7f119a59c700 1 --2- 172.21.15.27:0/2604484164 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f11880640b0 0x7f1188066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:06:29.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.329+0000 7f119a59c700 1 --2- 172.21.15.27:0/2604484164 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f11880640b0 0x7f1188066570 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f1184004d80 tx=0x7f118400b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:06:29.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.333+0000 7f1192ffd700 1 -- 172.21.15.27:0/2604484164 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f118c052990 con 0x7f119c0f5ed0 2021-11-03T18:06:29.505 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.507+0000 7f11a088b700 1 -- 172.21.15.27:0/2604484164 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f119c057f10 con 0x7f11880640b0 2021-11-03T18:06:29.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.521+0000 7f1192ffd700 1 -- 172.21.15.27:0/2604484164 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f119c057f10 con 0x7f11880640b0 2021-11-03T18:06:29.520 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:06:29.522 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (58m) 7m ago 59m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:06:29.522 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (49m) 7m ago 59m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:06:29.523 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (49m) 6m ago 58m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:06:29.523 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (57m) 7m ago 58m 41.8M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:06:29.523 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (51m) 7m ago 54m 26.0M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:06:29.524 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (54m) 7m ago 54m 3845M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:06:29.524 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (50m) 6m ago 54m 25.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:06:29.525 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (51m) 6m ago 54m 26.8M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:06:29.525 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (51m) 7m ago 60m 554M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:06:29.525 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (51m) 6m ago 58m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:06:29.526 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (50m) 7m ago 60m 235M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:06:29.526 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (50m) 6m ago 58m 112M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:06:29.527 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (58m) 7m ago 58m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:06:29.527 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (58m) 6m ago 58m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:06:29.528 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (49m) 7m ago 57m 351M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:06:29.528 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (48m) 7m ago 57m 383M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:06:29.528 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (48m) 7m ago 57m 284M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:06:29.529 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (47m) 6m ago 56m 404M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:06:29.529 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (47m) 6m ago 56m 322M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:06:29.529 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (47m) 6m ago 56m 327M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:06:29.529 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (57m) 7m ago 58m 55.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:06:29.530 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.523+0000 7f11a088b700 1 -- 172.21.15.27:0/2604484164 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f11880640b0 msgr2=0x7f1188066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:06:29.530 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.523+0000 7f11a088b700 1 --2- 172.21.15.27:0/2604484164 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f11880640b0 0x7f1188066570 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7f1184004d80 tx=0x7f118400b040).stop 2021-11-03T18:06:29.531 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.523+0000 7f11a088b700 1 -- 172.21.15.27:0/2604484164 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f119c0f5ed0 msgr2=0x7f119c10c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:06:29.531 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.523+0000 7f11a088b700 1 --2- 172.21.15.27:0/2604484164 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f119c0f5ed0 0x7f119c10c0d0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 rx=0x7f118c00e7f0 tx=0x7f118c00b040).stop 2021-11-03T18:06:29.531 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.523+0000 7f11a088b700 1 -- 172.21.15.27:0/2604484164 shutdown_connections 2021-11-03T18:06:29.531 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.523+0000 7f11a088b700 1 --2- 172.21.15.27:0/2604484164 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f11880640b0 0x7f1188066570 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:06:29.532 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.523+0000 7f11a088b700 1 --2- 172.21.15.27:0/2604484164 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f119c0f55c0 0x7f119c109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:06:29.532 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.523+0000 7f11a088b700 1 --2- 172.21.15.27:0/2604484164 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f119c0f5ed0 0x7f119c10c0d0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:06:29.532 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.523+0000 7f11a088b700 1 -- 172.21.15.27:0/2604484164 >> 172.21.15.27:0/2604484164 conn(0x7f119c0f0de0 msgr2=0x7f119c0fa350 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:06:29.532 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.524+0000 7f11a088b700 1 -- 172.21.15.27:0/2604484164 shutdown_connections 2021-11-03T18:06:29.533 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.524+0000 7f11a088b700 1 -- 172.21.15.27:0/2604484164 wait complete. 2021-11-03T18:06:29.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.625+0000 7f3a1084a700 1 -- 172.21.15.27:0/3829799029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a0c0f8610 msgr2=0x7f3a0c0f4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:06:29.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.625+0000 7f3a1084a700 1 --2- 172.21.15.27:0/3829799029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a0c0f8610 0x7f3a0c0f4820 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 rx=0x7f39fc004660 tx=0x7f39fc009b30).stop 2021-11-03T18:06:29.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.625+0000 7f3a1084a700 1 -- 172.21.15.27:0/3829799029 shutdown_connections 2021-11-03T18:06:29.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.625+0000 7f3a1084a700 1 --2- 172.21.15.27:0/3829799029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a0c0f8610 0x7f3a0c0f4820 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:06:29.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.625+0000 7f3a1084a700 1 --2- 172.21.15.27:0/3829799029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a0c0f7d00 0x7f3a0c0f8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:06:29.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.625+0000 7f3a1084a700 1 -- 172.21.15.27:0/3829799029 >> 172.21.15.27:0/3829799029 conn(0x7f3a0c0f0540 msgr2=0x7f3a0c0f2960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:06:29.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.625+0000 7f3a1084a700 1 -- 172.21.15.27:0/3829799029 shutdown_connections 2021-11-03T18:06:29.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.625+0000 7f3a1084a700 1 -- 172.21.15.27:0/3829799029 wait complete. 2021-11-03T18:06:29.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.626+0000 7f3a1084a700 1 Processor -- start 2021-11-03T18:06:29.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.626+0000 7f3a1084a700 1 -- start start 2021-11-03T18:06:29.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.626+0000 7f3a1084a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a0c0f7d00 0x7f3a0c0f75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:06:29.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.626+0000 7f3a1084a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a0c0f8610 0x7f3a0c0f5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:06:29.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.626+0000 7f3a1084a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c0f5fb0 con 0x7f3a0c0f8610 2021-11-03T18:06:29.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.626+0000 7f3a1084a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0c0f60f0 con 0x7f3a0c0f7d00 2021-11-03T18:06:29.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.626+0000 7f3a09d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a0c0f8610 0x7f3a0c0f5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:06:29.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.626+0000 7f3a09d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a0c0f8610 0x7f3a0c0f5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32984/0 (socket says 172.21.15.27:32984) 2021-11-03T18:06:29.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.626+0000 7f3a09d9b700 1 -- 172.21.15.27:0/1966430526 learned_addr learned my addr 172.21.15.27:0/1966430526 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:06:29.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.626+0000 7f3a0a59c700 1 --2- 172.21.15.27:0/1966430526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a0c0f7d00 0x7f3a0c0f75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:06:29.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.626+0000 7f3a09d9b700 1 -- 172.21.15.27:0/1966430526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a0c0f7d00 msgr2=0x7f3a0c0f75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:06:29.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.626+0000 7f3a09d9b700 1 --2- 172.21.15.27:0/1966430526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a0c0f7d00 0x7f3a0c0f75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:06:29.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.626+0000 7f3a09d9b700 1 -- 172.21.15.27:0/1966430526 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39fc005040 con 0x7f3a0c0f8610 2021-11-03T18:06:29.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.626+0000 7f3a09d9b700 1 --2- 172.21.15.27:0/1966430526 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a0c0f8610 0x7f3a0c0f5a50 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 rx=0x7f39fc009990 tx=0x7f39fc009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:06:29.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.627+0000 7f3a02ffd700 1 -- 172.21.15.27:0/1966430526 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39fc018070 con 0x7f3a0c0f8610 2021-11-03T18:06:29.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.627+0000 7f3a02ffd700 1 -- 172.21.15.27:0/1966430526 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f39fc007ba0 con 0x7f3a0c0f8610 2021-11-03T18:06:29.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.627+0000 7f3a1084a700 1 -- 172.21.15.27:0/1966430526 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a0c0f6370 con 0x7f3a0c0f8610 2021-11-03T18:06:29.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.627+0000 7f3a02ffd700 1 -- 172.21.15.27:0/1966430526 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39fc01c3c0 con 0x7f3a0c0f8610 2021-11-03T18:06:29.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.627+0000 7f3a1084a700 1 -- 172.21.15.27:0/1966430526 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a0c0f68d0 con 0x7f3a0c0f8610 2021-11-03T18:06:29.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.627+0000 7f3a1084a700 1 -- 172.21.15.27:0/1966430526 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a0c045ad0 con 0x7f3a0c0f8610 2021-11-03T18:06:29.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.628+0000 7f3a02ffd700 1 -- 172.21.15.27:0/1966430526 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f39fc01d420 con 0x7f3a0c0f8610 2021-11-03T18:06:29.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.628+0000 7f3a02ffd700 1 --2- 172.21.15.27:0/1966430526 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f39f80640b0 0x7f39f8066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:06:29.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.629+0000 7f3a02ffd700 1 -- 172.21.15.27:0/1966430526 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f39fc0823a0 con 0x7f3a0c0f8610 2021-11-03T18:06:29.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.629+0000 7f3a0a59c700 1 --2- 172.21.15.27:0/1966430526 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f39f80640b0 0x7f39f8066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:06:29.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.629+0000 7f3a0a59c700 1 --2- 172.21.15.27:0/1966430526 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f39f80640b0 0x7f39f8066570 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7f3a0c0f7360 tx=0x7f3a04009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:06:29.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.633+0000 7f3a02ffd700 1 -- 172.21.15.27:0/1966430526 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f39fc054760 con 0x7f3a0c0f8610 2021-11-03T18:06:29.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:29 smithi027 ceph-mon[97244]: pgmap v1555: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:29.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:29 smithi027 ceph-mon[97244]: from='client.35704 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:06:29.861 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.862+0000 7f3a1084a700 1 -- 172.21.15.27:0/1966430526 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3a0c027a00 con 0x7f3a0c0f8610 2021-11-03T18:06:29.861 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.863+0000 7f3a02ffd700 1 -- 172.21.15.27:0/1966430526 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f39fc054350 con 0x7f3a0c0f8610 2021-11-03T18:06:29.862 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:06:29.862 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:06:29.862 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:06:29.862 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:06:29.863 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:06:29.863 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:06:29.863 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:06:29.863 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:06:29.864 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:06:29.864 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:06:29.864 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:06:29.864 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:06:29.865 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:06:29.865 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:06:29.865 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:06:29.865 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:06:29.866 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:06:29.866 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:06:29.867 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.864+0000 7f3a1084a700 1 -- 172.21.15.27:0/1966430526 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f39f80640b0 msgr2=0x7f39f8066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:06:29.868 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.864+0000 7f3a1084a700 1 --2- 172.21.15.27:0/1966430526 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f39f80640b0 0x7f39f8066570 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7f3a0c0f7360 tx=0x7f3a04009040).stop 2021-11-03T18:06:29.868 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.865+0000 7f3a1084a700 1 -- 172.21.15.27:0/1966430526 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a0c0f8610 msgr2=0x7f3a0c0f5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:06:29.868 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.865+0000 7f3a1084a700 1 --2- 172.21.15.27:0/1966430526 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a0c0f8610 0x7f3a0c0f5a50 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 rx=0x7f39fc009990 tx=0x7f39fc009b30).stop 2021-11-03T18:06:29.868 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.865+0000 7f3a1084a700 1 -- 172.21.15.27:0/1966430526 shutdown_connections 2021-11-03T18:06:29.869 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.865+0000 7f3a1084a700 1 --2- 172.21.15.27:0/1966430526 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f39f80640b0 0x7f39f8066570 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:06:29.869 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.865+0000 7f3a1084a700 1 --2- 172.21.15.27:0/1966430526 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a0c0f7d00 0x7f3a0c0f75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:06:29.869 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.865+0000 7f3a1084a700 1 --2- 172.21.15.27:0/1966430526 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a0c0f8610 0x7f3a0c0f5a50 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:06:29.869 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.865+0000 7f3a1084a700 1 -- 172.21.15.27:0/1966430526 >> 172.21.15.27:0/1966430526 conn(0x7f3a0c0f0540 msgr2=0x7f3a0c0fb590 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:06:29.870 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.865+0000 7f3a1084a700 1 -- 172.21.15.27:0/1966430526 shutdown_connections 2021-11-03T18:06:29.870 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.865+0000 7f3a1084a700 1 -- 172.21.15.27:0/1966430526 wait complete. 2021-11-03T18:06:29.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.965+0000 7fa054843700 1 -- 172.21.15.27:0/1973160295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0500f5440 msgr2=0x7fa0500f5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:06:29.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.965+0000 7fa054843700 1 --2- 172.21.15.27:0/1973160295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0500f5440 0x7fa0500f5860 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 rx=0x7fa044004660 tx=0x7fa044009b30).stop 2021-11-03T18:06:29.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.965+0000 7fa054843700 1 -- 172.21.15.27:0/1973160295 shutdown_connections 2021-11-03T18:06:29.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.965+0000 7fa054843700 1 --2- 172.21.15.27:0/1973160295 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0500f66e0 0x7fa0500f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:06:29.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.965+0000 7fa054843700 1 --2- 172.21.15.27:0/1973160295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0500f5440 0x7fa0500f5860 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:06:29.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.965+0000 7fa054843700 1 -- 172.21.15.27:0/1973160295 >> 172.21.15.27:0/1973160295 conn(0x7fa0500f0ba0 msgr2=0x7fa0500f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:06:29.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.965+0000 7fa054843700 1 -- 172.21.15.27:0/1973160295 shutdown_connections 2021-11-03T18:06:29.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.965+0000 7fa054843700 1 -- 172.21.15.27:0/1973160295 wait complete. 2021-11-03T18:06:29.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.966+0000 7fa054843700 1 Processor -- start 2021-11-03T18:06:29.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.966+0000 7fa054843700 1 -- start start 2021-11-03T18:06:29.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.966+0000 7fa054843700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0500f5440 0x7fa050109aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:06:29.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.966+0000 7fa054843700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0500f66e0 0x7fa05010bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:06:29.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.966+0000 7fa054843700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0500603e0 con 0x7fa0500f66e0 2021-11-03T18:06:29.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.966+0000 7fa054843700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa050060520 con 0x7fa0500f5440 2021-11-03T18:06:29.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.966+0000 7fa04e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0500f66e0 0x7fa05010bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:06:29.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.966+0000 7fa04e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0500f66e0 0x7fa05010bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:32994/0 (socket says 172.21.15.27:32994) 2021-11-03T18:06:29.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.966+0000 7fa04e59c700 1 -- 172.21.15.27:0/384626875 learned_addr learned my addr 172.21.15.27:0/384626875 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:06:29.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.966+0000 7fa04ed9d700 1 --2- 172.21.15.27:0/384626875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0500f5440 0x7fa050109aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:06:29.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.966+0000 7fa04e59c700 1 -- 172.21.15.27:0/384626875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0500f5440 msgr2=0x7fa050109aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:06:29.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.966+0000 7fa04e59c700 1 --2- 172.21.15.27:0/384626875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0500f5440 0x7fa050109aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:06:29.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.966+0000 7fa04e59c700 1 -- 172.21.15.27:0/384626875 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa044005040 con 0x7fa0500f66e0 2021-11-03T18:06:29.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.967+0000 7fa04e59c700 1 --2- 172.21.15.27:0/384626875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0500f66e0 0x7fa05010bfa0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x7fa04000c530 tx=0x7fa040007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:06:29.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.967+0000 7fa0377fe700 1 -- 172.21.15.27:0/384626875 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa04000f650 con 0x7fa0500f66e0 2021-11-03T18:06:29.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.967+0000 7fa054843700 1 -- 172.21.15.27:0/384626875 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa05010c550 con 0x7fa0500f66e0 2021-11-03T18:06:29.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.967+0000 7fa0377fe700 1 -- 172.21.15.27:0/384626875 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fa04000fc60 con 0x7fa0500f66e0 2021-11-03T18:06:29.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.967+0000 7fa0377fe700 1 -- 172.21.15.27:0/384626875 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa040012c20 con 0x7fa0500f66e0 2021-11-03T18:06:29.973 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.967+0000 7fa054843700 1 -- 172.21.15.27:0/384626875 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa05010cb10 con 0x7fa0500f66e0 2021-11-03T18:06:29.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.968+0000 7fa054843700 1 -- 172.21.15.27:0/384626875 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa050045ad0 con 0x7fa0500f66e0 2021-11-03T18:06:29.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.968+0000 7fa0377fe700 1 -- 172.21.15.27:0/384626875 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fa040010070 con 0x7fa0500f66e0 2021-11-03T18:06:29.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.969+0000 7fa0377fe700 1 --2- 172.21.15.27:0/384626875 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa038064100 0x7fa0380665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:06:29.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.969+0000 7fa0377fe700 1 -- 172.21.15.27:0/384626875 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fa0400806c0 con 0x7fa0500f66e0 2021-11-03T18:06:29.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.969+0000 7fa04ed9d700 1 --2- 172.21.15.27:0/384626875 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa038064100 0x7fa0380665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:06:29.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.969+0000 7fa04ed9d700 1 --2- 172.21.15.27:0/384626875 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa038064100 0x7fa0380665c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7fa044013690 tx=0x7fa044009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:06:29.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:29.973+0000 7fa0377fe700 1 -- 172.21.15.27:0/384626875 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fa0400528d0 con 0x7fa0500f66e0 2021-11-03T18:06:29.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:29 smithi115 ceph-mon[65453]: pgmap v1555: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:29.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:29 smithi115 ceph-mon[65453]: from='client.35704 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:06:30.178 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:30.180+0000 7fa054843700 1 -- 172.21.15.27:0/384626875 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa050027a00 con 0x7fa0500f66e0 2021-11-03T18:06:30.179 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:30.181+0000 7fa0377fe700 1 -- 172.21.15.27:0/384626875 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7fa0400524c0 con 0x7fa0500f66e0 2021-11-03T18:06:30.179 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:06:30.179 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:06:30.180 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:06:30.180 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:06:30.180 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:06:30.180 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:06:30.181 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:06:30.181 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:06:30.181 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:06:30.181 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:06:30.182 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:06:30.182 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:06:30.182 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:06:30.182 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:06:30.183 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:06:30.183 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:06:30.183 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:06:30.184 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:06:30.184 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:06:30.184 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:06:30.184 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:06:30.184 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:06:30.185 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:06:30.185 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:06:30.185 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:06:30.185 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:06:30.185 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:06:30.186 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:06:30.186 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:06:30.186 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:06:30.186 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:06:30.187 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:06:30.187 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:06:30.187 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:06:30.187 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:06:30.187 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:06:30.188 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:06:30.188 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:06:30.188 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:06:30.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:30.182+0000 7fa054843700 1 -- 172.21.15.27:0/384626875 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa038064100 msgr2=0x7fa0380665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:06:30.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:30.182+0000 7fa054843700 1 --2- 172.21.15.27:0/384626875 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa038064100 0x7fa0380665c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7fa044013690 tx=0x7fa044009b30).stop 2021-11-03T18:06:30.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:30.182+0000 7fa054843700 1 -- 172.21.15.27:0/384626875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0500f66e0 msgr2=0x7fa05010bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:06:30.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:30.182+0000 7fa054843700 1 --2- 172.21.15.27:0/384626875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0500f66e0 0x7fa05010bfa0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x7fa04000c530 tx=0x7fa040007e20).stop 2021-11-03T18:06:30.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:30.182+0000 7fa054843700 1 -- 172.21.15.27:0/384626875 shutdown_connections 2021-11-03T18:06:30.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:30.182+0000 7fa054843700 1 --2- 172.21.15.27:0/384626875 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa038064100 0x7fa0380665c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:06:30.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:30.182+0000 7fa054843700 1 --2- 172.21.15.27:0/384626875 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa0500f5440 0x7fa050109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:06:30.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:30.182+0000 7fa054843700 1 --2- 172.21.15.27:0/384626875 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa0500f66e0 0x7fa05010bfa0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:06:30.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:30.182+0000 7fa054843700 1 -- 172.21.15.27:0/384626875 >> 172.21.15.27:0/384626875 conn(0x7fa0500f0ba0 msgr2=0x7fa0500fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:06:30.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:30.182+0000 7fa054843700 1 -- 172.21.15.27:0/384626875 shutdown_connections 2021-11-03T18:06:30.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:06:30.182+0000 7fa054843700 1 -- 172.21.15.27:0/384626875 wait complete. 2021-11-03T18:06:30.192 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:06:30.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3032.432380 secs 2021-11-03T18:06:30.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:30 smithi027 ceph-mon[97244]: from='client.35708 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:06:30.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:06:30.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"}]: dispatch 2021-11-03T18:06:30.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2021-11-03T18:06:30.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:06:30.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:06:30.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:30.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:30.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:30.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:30.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:30 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:06:30.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:30 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1966430526' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:30.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:30 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/384626875' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:06:30.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3032.432380 secs 2021-11-03T18:06:30.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:30 smithi115 ceph-mon[65453]: from='client.35708 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:06:30.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:06:30.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"}]: dispatch 2021-11-03T18:06:30.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2021-11-03T18:06:30.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:06:30.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:06:30.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:30.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:30.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:30.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:30.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:30 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:06:30.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:30 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1966430526' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:30.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:30 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/384626875' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:06:31.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:31 smithi027 ceph-mon[97244]: pgmap v1556: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:31.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:31 smithi115 ceph-mon[65453]: pgmap v1556: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:33.958 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:33 smithi115 ceph-mon[65453]: pgmap v1557: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:34.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:33 smithi027 ceph-mon[97244]: pgmap v1557: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:35.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3037.432503 secs 2021-11-03T18:06:35.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:35 smithi115 ceph-mon[65453]: pgmap v1558: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:36.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3037.432503 secs 2021-11-03T18:06:36.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:35 smithi027 ceph-mon[97244]: pgmap v1558: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:37.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:37 smithi115 ceph-mon[65453]: pgmap v1559: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:38.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:37 smithi027 ceph-mon[97244]: pgmap v1559: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:39.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:39 smithi115 ceph-mon[65453]: pgmap v1560: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:39.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:06:39.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:06:39.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:06:39.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:39.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:39.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:39.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:40.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:39 smithi027 ceph-mon[97244]: pgmap v1560: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:40.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:06:40.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:06:40.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:06:40.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:40.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:40.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:40.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:40.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3042.432650 secs 2021-11-03T18:06:40.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:40 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:06:41.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3042.432650 secs 2021-11-03T18:06:41.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:40 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:06:42.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:42 smithi115 ceph-mon[65453]: pgmap v1561: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:42.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:42 smithi027 ceph-mon[97244]: pgmap v1561: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:44.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:44 smithi115 ceph-mon[65453]: pgmap v1562: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:44.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:44 smithi027 ceph-mon[97244]: pgmap v1562: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:45.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3047.432794 secs 2021-11-03T18:06:45.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3047.432794 secs 2021-11-03T18:06:46.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:46 smithi115 ceph-mon[65453]: pgmap v1563: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:46.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:46 smithi027 ceph-mon[97244]: pgmap v1563: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:48.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:48 smithi115 ceph-mon[65453]: pgmap v1564: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:48.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:48 smithi027 ceph-mon[97244]: pgmap v1564: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:50.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:50 smithi115 ceph-mon[65453]: pgmap v1565: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:50.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3052.432917 secs 2021-11-03T18:06:50.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:06:50.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:06:50.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:06:50.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:50.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:50.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:50.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:50.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:50 smithi027 ceph-mon[97244]: pgmap v1565: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:50.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3052.432917 secs 2021-11-03T18:06:50.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:06:50.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:06:50.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:06:50.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:50.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:50.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:50.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:06:51.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:51 smithi115 ceph-mon[65453]: poll took 60186.880 ms: 1 events 2021-11-03T18:06:51.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:51 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:06:51.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:51 smithi027 ceph-mon[97244]: poll took 60186.880 ms: 1 events 2021-11-03T18:06:51.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:51 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:06:52.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:52 smithi115 ceph-mon[65453]: pgmap v1566: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:52.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:52 smithi027 ceph-mon[97244]: pgmap v1566: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:54.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:54 smithi115 ceph-mon[65453]: pgmap v1567: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:54.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:54 smithi027 ceph-mon[97244]: pgmap v1567: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:55.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3057.433057 secs 2021-11-03T18:06:55.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3057.433057 secs 2021-11-03T18:06:56.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:56 smithi115 ceph-mon[65453]: pgmap v1568: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:56.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:06:56.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:06:56.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:56 smithi027 ceph-mon[97244]: pgmap v1568: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:56.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:06:56.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:06:58.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:06:58 smithi115 ceph-mon[65453]: pgmap v1569: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:06:58.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:06:58 smithi027 ceph-mon[97244]: pgmap v1569: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:00.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.298+0000 7f66772b9700 1 -- 172.21.15.27:0/3435541777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66700f55c0 msgr2=0x7f66700f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:07:00.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.298+0000 7f66772b9700 1 --2- 172.21.15.27:0/3435541777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66700f55c0 0x7f66700f59e0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0x7f66680099e0 tx=0x7f6668004d10).stop 2021-11-03T18:07:00.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.298+0000 7f66772b9700 1 -- 172.21.15.27:0/3435541777 shutdown_connections 2021-11-03T18:07:00.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.298+0000 7f66772b9700 1 --2- 172.21.15.27:0/3435541777 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66700f6860 0x7f66700f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:00.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.298+0000 7f66772b9700 1 --2- 172.21.15.27:0/3435541777 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66700f55c0 0x7f66700f59e0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:00.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.298+0000 7f66772b9700 1 -- 172.21.15.27:0/3435541777 >> 172.21.15.27:0/3435541777 conn(0x7f66700f0d40 msgr2=0x7f66700f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:07:00.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.298+0000 7f66772b9700 1 -- 172.21.15.27:0/3435541777 shutdown_connections 2021-11-03T18:07:00.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.298+0000 7f66772b9700 1 -- 172.21.15.27:0/3435541777 wait complete. 2021-11-03T18:07:00.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.299+0000 7f66772b9700 1 Processor -- start 2021-11-03T18:07:00.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.299+0000 7f66772b9700 1 -- start start 2021-11-03T18:07:00.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.299+0000 7f66772b9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66700f55c0 0x7f6670109d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:07:00.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.299+0000 7f66772b9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66700f6860 0x7f667010c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:07:00.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.299+0000 7f66772b9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66700603c0 con 0x7f66700f6860 2021-11-03T18:07:00.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.299+0000 7f66772b9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6670060500 con 0x7f66700f55c0 2021-11-03T18:07:00.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.299+0000 7f6674854700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66700f6860 0x7f667010c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:07:00.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.299+0000 7f6674854700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66700f6860 0x7f667010c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33012/0 (socket says 172.21.15.27:33012) 2021-11-03T18:07:00.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.299+0000 7f6674854700 1 -- 172.21.15.27:0/124051608 learned_addr learned my addr 172.21.15.27:0/124051608 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:07:00.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.299+0000 7f6675055700 1 --2- 172.21.15.27:0/124051608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66700f55c0 0x7f6670109d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:07:00.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.300+0000 7f6674854700 1 -- 172.21.15.27:0/124051608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66700f55c0 msgr2=0x7f6670109d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:07:00.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.300+0000 7f6674854700 1 --2- 172.21.15.27:0/124051608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66700f55c0 0x7f6670109d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:00.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.300+0000 7f6674854700 1 -- 172.21.15.27:0/124051608 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6668009640 con 0x7f66700f6860 2021-11-03T18:07:00.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.300+0000 7f6675055700 1 --2- 172.21.15.27:0/124051608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66700f55c0 0x7f6670109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:07:00.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.300+0000 7f6674854700 1 --2- 172.21.15.27:0/124051608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66700f6860 0x7f667010c280 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 rx=0x7f666400c790 tx=0x7f6664007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:07:00.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.300+0000 7f666dffb700 1 -- 172.21.15.27:0/124051608 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f666400e680 con 0x7f66700f6860 2021-11-03T18:07:00.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.300+0000 7f666dffb700 1 -- 172.21.15.27:0/124051608 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f666400ec90 con 0x7f66700f6860 2021-11-03T18:07:00.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.300+0000 7f66772b9700 1 -- 172.21.15.27:0/124051608 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f667010c830 con 0x7f66700f6860 2021-11-03T18:07:00.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.300+0000 7f666dffb700 1 -- 172.21.15.27:0/124051608 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6664013d60 con 0x7f66700f6860 2021-11-03T18:07:00.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.300+0000 7f66772b9700 1 -- 172.21.15.27:0/124051608 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f667010cd40 con 0x7f66700f6860 2021-11-03T18:07:00.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.301+0000 7f66772b9700 1 -- 172.21.15.27:0/124051608 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66700f49b0 con 0x7f66700f6860 2021-11-03T18:07:00.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.301+0000 7f666dffb700 1 -- 172.21.15.27:0/124051608 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f666401a060 con 0x7f66700f6860 2021-11-03T18:07:00.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.302+0000 7f666dffb700 1 --2- 172.21.15.27:0/124051608 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f66580640b0 0x7f6658066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:07:00.307 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.302+0000 7f666dffb700 1 -- 172.21.15.27:0/124051608 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f6664080a50 con 0x7f66700f6860 2021-11-03T18:07:00.307 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.302+0000 7f6675055700 1 --2- 172.21.15.27:0/124051608 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f66580640b0 0x7f6658066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:07:00.307 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.302+0000 7f6675055700 1 --2- 172.21.15.27:0/124051608 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f66580640b0 0x7f6658066570 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f666800b730 tx=0x7f6668004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:07:00.308 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.306+0000 7f666dffb700 1 -- 172.21.15.27:0/124051608 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f6664052c00 con 0x7f66700f6860 2021-11-03T18:07:00.484 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.486+0000 7f66772b9700 1 -- 172.21.15.27:0/124051608 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f66700fdf10 con 0x7f66580640b0 2021-11-03T18:07:00.484 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.488+0000 7f666dffb700 1 -- 172.21.15.27:0/124051608 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f66700fdf10 con 0x7f66580640b0 2021-11-03T18:07:00.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:00 smithi115 ceph-mon[65453]: pgmap v1570: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:00.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3062.433098 secs 2021-11-03T18:07:00.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:07:00.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:07:00.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:07:00.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:00.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:00.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:00.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:00.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.489+0000 7f66772b9700 1 -- 172.21.15.27:0/124051608 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f66580640b0 msgr2=0x7f6658066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:07:00.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.489+0000 7f66772b9700 1 --2- 172.21.15.27:0/124051608 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f66580640b0 0x7f6658066570 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7f666800b730 tx=0x7f6668004d10).stop 2021-11-03T18:07:00.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.489+0000 7f66772b9700 1 -- 172.21.15.27:0/124051608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66700f6860 msgr2=0x7f667010c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:07:00.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.489+0000 7f66772b9700 1 --2- 172.21.15.27:0/124051608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66700f6860 0x7f667010c280 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 rx=0x7f666400c790 tx=0x7f6664007d70).stop 2021-11-03T18:07:00.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.489+0000 7f66772b9700 1 -- 172.21.15.27:0/124051608 shutdown_connections 2021-11-03T18:07:00.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.489+0000 7f66772b9700 1 --2- 172.21.15.27:0/124051608 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f66580640b0 0x7f6658066570 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:00.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.489+0000 7f66772b9700 1 --2- 172.21.15.27:0/124051608 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f66700f55c0 0x7f6670109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:00.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.489+0000 7f66772b9700 1 --2- 172.21.15.27:0/124051608 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f66700f6860 0x7f667010c280 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:00.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.489+0000 7f66772b9700 1 -- 172.21.15.27:0/124051608 >> 172.21.15.27:0/124051608 conn(0x7f66700f0d40 msgr2=0x7f66700fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:07:00.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.490+0000 7f66772b9700 1 -- 172.21.15.27:0/124051608 shutdown_connections 2021-11-03T18:07:00.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.490+0000 7f66772b9700 1 -- 172.21.15.27:0/124051608 wait complete. 2021-11-03T18:07:00.496 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:07:00.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:00 smithi027 ceph-mon[97244]: pgmap v1570: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:00.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3062.433098 secs 2021-11-03T18:07:00.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:07:00.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:07:00.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:07:00.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:00.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:00.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:00.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:00.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.590+0000 7f92d715a700 1 -- 172.21.15.27:0/1562685188 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92d00f55c0 msgr2=0x7f92d00f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:07:00.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.590+0000 7f92d715a700 1 --2- 172.21.15.27:0/1562685188 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92d00f55c0 0x7f92d00f59e0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 rx=0x7f92c80099e0 tx=0x7f92c8004d10).stop 2021-11-03T18:07:00.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.590+0000 7f92d715a700 1 -- 172.21.15.27:0/1562685188 shutdown_connections 2021-11-03T18:07:00.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.590+0000 7f92d715a700 1 --2- 172.21.15.27:0/1562685188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92d00f6860 0x7f92d00f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:00.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.590+0000 7f92d715a700 1 --2- 172.21.15.27:0/1562685188 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92d00f55c0 0x7f92d00f59e0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:00.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.590+0000 7f92d715a700 1 -- 172.21.15.27:0/1562685188 >> 172.21.15.27:0/1562685188 conn(0x7f92d00f0d40 msgr2=0x7f92d00f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:07:00.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.590+0000 7f92d715a700 1 -- 172.21.15.27:0/1562685188 shutdown_connections 2021-11-03T18:07:00.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.590+0000 7f92d715a700 1 -- 172.21.15.27:0/1562685188 wait complete. 2021-11-03T18:07:00.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.590+0000 7f92d715a700 1 Processor -- start 2021-11-03T18:07:00.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.590+0000 7f92d715a700 1 -- start start 2021-11-03T18:07:00.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.591+0000 7f92d715a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92d00f55c0 0x7f92d0105fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:07:00.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.591+0000 7f92d715a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92d00f6860 0x7f92d01064c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:07:00.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.591+0000 7f92d715a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d0106a70 con 0x7f92d00f55c0 2021-11-03T18:07:00.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.591+0000 7f92d715a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d0106bb0 con 0x7f92d00f6860 2021-11-03T18:07:00.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.591+0000 7f92d4ef6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92d00f55c0 0x7f92d0105fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:07:00.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.591+0000 7f92d4ef6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92d00f55c0 0x7f92d0105fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33020/0 (socket says 172.21.15.27:33020) 2021-11-03T18:07:00.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.591+0000 7f92d4ef6700 1 -- 172.21.15.27:0/3546041842 learned_addr learned my addr 172.21.15.27:0/3546041842 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:07:00.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.591+0000 7f92d4ef6700 1 -- 172.21.15.27:0/3546041842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92d00f6860 msgr2=0x7f92d01064c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:07:00.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.591+0000 7f92d4ef6700 1 --2- 172.21.15.27:0/3546041842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92d00f6860 0x7f92d01064c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:07:00.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.591+0000 7f92d4ef6700 1 -- 172.21.15.27:0/3546041842 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c8009640 con 0x7f92d00f55c0 2021-11-03T18:07:00.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.591+0000 7f92d4ef6700 1 --2- 172.21.15.27:0/3546041842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92d00f55c0 0x7f92d0105fd0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 rx=0x7f92c800c010 tx=0x7f92c8004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:07:00.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.591+0000 7f92cd7fa700 1 -- 172.21.15.27:0/3546041842 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92c800e070 con 0x7f92d00f55c0 2021-11-03T18:07:00.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.591+0000 7f92cd7fa700 1 -- 172.21.15.27:0/3546041842 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f92c8007dc0 con 0x7f92d00f55c0 2021-11-03T18:07:00.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.591+0000 7f92d715a700 1 -- 172.21.15.27:0/3546041842 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92d01074f0 con 0x7f92d00f55c0 2021-11-03T18:07:00.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.591+0000 7f92cd7fa700 1 -- 172.21.15.27:0/3546041842 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92c8018070 con 0x7f92d00f55c0 2021-11-03T18:07:00.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.592+0000 7f92d715a700 1 -- 172.21.15.27:0/3546041842 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92d01079d0 con 0x7f92d00f55c0 2021-11-03T18:07:00.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.592+0000 7f92d715a700 1 -- 172.21.15.27:0/3546041842 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92d0045ad0 con 0x7f92d00f55c0 2021-11-03T18:07:00.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.593+0000 7f92cd7fa700 1 -- 172.21.15.27:0/3546041842 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f92c8024060 con 0x7f92d00f55c0 2021-11-03T18:07:00.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.593+0000 7f92cd7fa700 1 --2- 172.21.15.27:0/3546041842 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f92b80662c0 0x7f92b8068780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:07:00.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.593+0000 7f92cd7fa700 1 -- 172.21.15.27:0/3546041842 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f92c8082330 con 0x7f92d00f55c0 2021-11-03T18:07:00.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.593+0000 7f92cffff700 1 --2- 172.21.15.27:0/3546041842 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f92b80662c0 0x7f92b8068780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:07:00.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.594+0000 7f92cffff700 1 --2- 172.21.15.27:0/3546041842 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f92b80662c0 0x7f92b8068780 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f92c40047d0 tx=0x7f92c4007a40).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:07:00.600 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.597+0000 7f92cd7fa700 1 -- 172.21.15.27:0/3546041842 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f92c8054630 con 0x7f92d00f55c0 2021-11-03T18:07:00.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.772+0000 7f92d715a700 1 -- 172.21.15.27:0/3546041842 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f92d005f620 con 0x7f92b80662c0 2021-11-03T18:07:00.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.787+0000 7f92cd7fa700 1 -- 172.21.15.27:0/3546041842 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f92d005f620 con 0x7f92b80662c0 2021-11-03T18:07:00.784 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:07:00.784 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (58m) 7m ago 59m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:07:00.784 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (50m) 7m ago 59m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:07:00.785 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (50m) 7m ago 59m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:07:00.785 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (58m) 7m ago 59m 41.8M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:07:00.785 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (52m) 7m ago 54m 26.0M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:07:00.785 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (55m) 7m ago 55m 3845M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:07:00.786 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (50m) 7m ago 54m 25.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:07:00.786 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (52m) 7m ago 55m 26.8M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:07:00.786 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (52m) 7m ago 61m 554M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:07:00.786 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (51m) 7m ago 59m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:07:00.787 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (51m) 7m ago 61m 235M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:07:00.787 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (50m) 7m ago 58m 112M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:07:00.787 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (59m) 7m ago 59m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:07:00.787 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (58m) 7m ago 58m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:07:00.787 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (49m) 7m ago 58m 351M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:07:00.788 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (49m) 7m ago 57m 383M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:07:00.788 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (48m) 7m ago 57m 284M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:07:00.788 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (48m) 7m ago 57m 404M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:07:00.788 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (47m) 7m ago 56m 322M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:07:00.789 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (47m) 7m ago 56m 327M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:07:00.789 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (58m) 7m ago 59m 55.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:07:00.790 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.788+0000 7f92d715a700 1 -- 172.21.15.27:0/3546041842 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f92b80662c0 msgr2=0x7f92b8068780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:07:00.790 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.788+0000 7f92d715a700 1 --2- 172.21.15.27:0/3546041842 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f92b80662c0 0x7f92b8068780 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7f92c40047d0 tx=0x7f92c4007a40).stop 2021-11-03T18:07:00.790 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.788+0000 7f92d715a700 1 -- 172.21.15.27:0/3546041842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92d00f55c0 msgr2=0x7f92d0105fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:07:00.790 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.788+0000 7f92d715a700 1 --2- 172.21.15.27:0/3546041842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92d00f55c0 0x7f92d0105fd0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 rx=0x7f92c800c010 tx=0x7f92c8004d10).stop 2021-11-03T18:07:00.791 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.789+0000 7f92d715a700 1 -- 172.21.15.27:0/3546041842 shutdown_connections 2021-11-03T18:07:00.791 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.789+0000 7f92d715a700 1 --2- 172.21.15.27:0/3546041842 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f92b80662c0 0x7f92b8068780 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:00.791 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.789+0000 7f92d715a700 1 --2- 172.21.15.27:0/3546041842 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92d00f55c0 0x7f92d0105fd0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:00.791 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.789+0000 7f92d715a700 1 --2- 172.21.15.27:0/3546041842 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92d00f6860 0x7f92d01064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:07:00.792 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.789+0000 7f92d715a700 1 -- 172.21.15.27:0/3546041842 >> 172.21.15.27:0/3546041842 conn(0x7f92d00f0d40 msgr2=0x7f92d00fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:07:00.792 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.789+0000 7f92d715a700 1 -- 172.21.15.27:0/3546041842 shutdown_connections 2021-11-03T18:07:00.792 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.789+0000 7f92d715a700 1 -- 172.21.15.27:0/3546041842 wait complete. 2021-11-03T18:07:00.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.889+0000 7f380f7da700 1 -- 172.21.15.27:0/193290931 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38080f6880 msgr2=0x7f38080f6ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:07:00.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.889+0000 7f380f7da700 1 --2- 172.21.15.27:0/193290931 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38080f6880 0x7f38080f6ce0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 rx=0x7f3804004660 tx=0x7f3804009b30).stop 2021-11-03T18:07:00.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.889+0000 7f380f7da700 1 -- 172.21.15.27:0/193290931 shutdown_connections 2021-11-03T18:07:00.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.889+0000 7f380f7da700 1 --2- 172.21.15.27:0/193290931 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38080f6880 0x7f38080f6ce0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:00.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.889+0000 7f380f7da700 1 --2- 172.21.15.27:0/193290931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38080f55e0 0x7f38080f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:00.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.889+0000 7f380f7da700 1 -- 172.21.15.27:0/193290931 >> 172.21.15.27:0/193290931 conn(0x7f38080f0d40 msgr2=0x7f38080f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:07:00.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.889+0000 7f380f7da700 1 -- 172.21.15.27:0/193290931 shutdown_connections 2021-11-03T18:07:00.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.889+0000 7f380f7da700 1 -- 172.21.15.27:0/193290931 wait complete. 2021-11-03T18:07:00.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.889+0000 7f380f7da700 1 Processor -- start 2021-11-03T18:07:00.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.889+0000 7f380f7da700 1 -- start start 2021-11-03T18:07:00.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.890+0000 7f380f7da700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38080f55e0 0x7f3808106f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:07:00.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.890+0000 7f380f7da700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38080f6880 0x7f3808107440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:07:00.891 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.890+0000 7f380f7da700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38081079f0 con 0x7f38080f6880 2021-11-03T18:07:00.891 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.890+0000 7f380f7da700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3808107b30 con 0x7f38080f55e0 2021-11-03T18:07:00.891 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.890+0000 7f380cd75700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38080f6880 0x7f3808107440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:07:00.891 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.890+0000 7f380cd75700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38080f6880 0x7f3808107440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33032/0 (socket says 172.21.15.27:33032) 2021-11-03T18:07:00.891 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.890+0000 7f380cd75700 1 -- 172.21.15.27:0/3092435755 learned_addr learned my addr 172.21.15.27:0/3092435755 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:07:00.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.890+0000 7f380d576700 1 --2- 172.21.15.27:0/3092435755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38080f55e0 0x7f3808106f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:07:00.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.890+0000 7f380cd75700 1 -- 172.21.15.27:0/3092435755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38080f55e0 msgr2=0x7f3808106f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:07:00.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.890+0000 7f380cd75700 1 --2- 172.21.15.27:0/3092435755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38080f55e0 0x7f3808106f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:00.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.890+0000 7f380cd75700 1 -- 172.21.15.27:0/3092435755 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3804005040 con 0x7f38080f6880 2021-11-03T18:07:00.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.890+0000 7f380cd75700 1 --2- 172.21.15.27:0/3092435755 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38080f6880 0x7f3808107440 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0x7f3804009990 tx=0x7f3804009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:07:00.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.890+0000 7f37fdffb700 1 -- 172.21.15.27:0/3092435755 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3804018070 con 0x7f38080f6880 2021-11-03T18:07:00.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.890+0000 7f37fdffb700 1 -- 172.21.15.27:0/3092435755 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f3804007ba0 con 0x7f38080f6880 2021-11-03T18:07:00.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.890+0000 7f37fdffb700 1 -- 172.21.15.27:0/3092435755 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f380401c3c0 con 0x7f38080f6880 2021-11-03T18:07:00.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.890+0000 7f380f7da700 1 -- 172.21.15.27:0/3092435755 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3808108470 con 0x7f38080f6880 2021-11-03T18:07:00.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.891+0000 7f380f7da700 1 -- 172.21.15.27:0/3092435755 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38081089d0 con 0x7f38080f6880 2021-11-03T18:07:00.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.891+0000 7f380f7da700 1 -- 172.21.15.27:0/3092435755 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3808045ad0 con 0x7f38080f6880 2021-11-03T18:07:00.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.892+0000 7f37fdffb700 1 -- 172.21.15.27:0/3092435755 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f380401d420 con 0x7f38080f6880 2021-11-03T18:07:00.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.892+0000 7f37fdffb700 1 --2- 172.21.15.27:0/3092435755 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f37f40640b0 0x7f37f4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:07:00.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.892+0000 7f37fdffb700 1 -- 172.21.15.27:0/3092435755 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f3804082530 con 0x7f38080f6880 2021-11-03T18:07:00.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.892+0000 7f380d576700 1 --2- 172.21.15.27:0/3092435755 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f37f40640b0 0x7f37f4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:07:00.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.893+0000 7f380d576700 1 --2- 172.21.15.27:0/3092435755 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f37f40640b0 0x7f37f4066570 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f38080f66e0 tx=0x7f37f800b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:07:00.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:00.896+0000 7f37fdffb700 1 -- 172.21.15.27:0/3092435755 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f3804054710 con 0x7f38080f6880 2021-11-03T18:07:01.127 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.129+0000 7f380f7da700 1 -- 172.21.15.27:0/3092435755 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3808027a00 con 0x7f38080f6880 2021-11-03T18:07:01.128 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.130+0000 7f37fdffb700 1 -- 172.21.15.27:0/3092435755 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f3804054300 con 0x7f38080f6880 2021-11-03T18:07:01.128 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:07:01.128 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:07:01.129 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:07:01.129 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:07:01.129 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:07:01.130 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:07:01.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:07:01.130 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:07:01.130 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:07:01.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:07:01.131 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:07:01.131 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:07:01.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:07:01.131 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:07:01.132 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:07:01.132 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:07:01.132 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:07:01.133 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:07:01.133 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.132+0000 7f380f7da700 1 -- 172.21.15.27:0/3092435755 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f37f40640b0 msgr2=0x7f37f4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:07:01.134 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.132+0000 7f380f7da700 1 --2- 172.21.15.27:0/3092435755 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f37f40640b0 0x7f37f4066570 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f38080f66e0 tx=0x7f37f800b040).stop 2021-11-03T18:07:01.134 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.132+0000 7f380f7da700 1 -- 172.21.15.27:0/3092435755 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38080f6880 msgr2=0x7f3808107440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:07:01.134 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.132+0000 7f380f7da700 1 --2- 172.21.15.27:0/3092435755 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38080f6880 0x7f3808107440 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0x7f3804009990 tx=0x7f3804009b30).stop 2021-11-03T18:07:01.135 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.132+0000 7f380f7da700 1 -- 172.21.15.27:0/3092435755 shutdown_connections 2021-11-03T18:07:01.135 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.132+0000 7f380f7da700 1 --2- 172.21.15.27:0/3092435755 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f37f40640b0 0x7f37f4066570 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:01.135 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.132+0000 7f380f7da700 1 --2- 172.21.15.27:0/3092435755 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38080f55e0 0x7f3808106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:01.135 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.132+0000 7f380f7da700 1 --2- 172.21.15.27:0/3092435755 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38080f6880 0x7f3808107440 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:01.135 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.132+0000 7f380f7da700 1 -- 172.21.15.27:0/3092435755 >> 172.21.15.27:0/3092435755 conn(0x7f38080f0d40 msgr2=0x7f38080fa2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:07:01.136 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.132+0000 7f380f7da700 1 -- 172.21.15.27:0/3092435755 shutdown_connections 2021-11-03T18:07:01.136 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.132+0000 7f380f7da700 1 -- 172.21.15.27:0/3092435755 wait complete. 2021-11-03T18:07:01.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.233+0000 7ff932445700 1 -- 172.21.15.27:0/560920809 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff92c0f66e0 msgr2=0x7ff92c0f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:07:01.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.233+0000 7ff932445700 1 --2- 172.21.15.27:0/560920809 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff92c0f66e0 0x7ff92c0f6b00 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 rx=0x7ff920004660 tx=0x7ff920009b30).stop 2021-11-03T18:07:01.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.233+0000 7ff932445700 1 -- 172.21.15.27:0/560920809 shutdown_connections 2021-11-03T18:07:01.234 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.233+0000 7ff932445700 1 --2- 172.21.15.27:0/560920809 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff92c0f6ff0 0x7ff92c0f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:01.234 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.233+0000 7ff932445700 1 --2- 172.21.15.27:0/560920809 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff92c0f66e0 0x7ff92c0f6b00 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:01.234 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.233+0000 7ff932445700 1 -- 172.21.15.27:0/560920809 >> 172.21.15.27:0/560920809 conn(0x7ff92c0f1f00 msgr2=0x7ff92c0f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:07:01.234 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.233+0000 7ff932445700 1 -- 172.21.15.27:0/560920809 shutdown_connections 2021-11-03T18:07:01.235 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.233+0000 7ff932445700 1 -- 172.21.15.27:0/560920809 wait complete. 2021-11-03T18:07:01.235 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.233+0000 7ff932445700 1 Processor -- start 2021-11-03T18:07:01.236 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.233+0000 7ff932445700 1 -- start start 2021-11-03T18:07:01.236 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.234+0000 7ff932445700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff92c0f66e0 0x7ff92c106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:07:01.236 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.234+0000 7ff932445700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff92c0f6ff0 0x7ff92c107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:07:01.236 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.234+0000 7ff932445700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c107a00 con 0x7ff92c0f66e0 2021-11-03T18:07:01.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.234+0000 7ff932445700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92c107b40 con 0x7ff92c0f6ff0 2021-11-03T18:07:01.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.234+0000 7ff92bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff92c0f66e0 0x7ff92c106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:07:01.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.234+0000 7ff92bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff92c0f66e0 0x7ff92c106f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33040/0 (socket says 172.21.15.27:33040) 2021-11-03T18:07:01.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.234+0000 7ff92bfff700 1 -- 172.21.15.27:0/116587902 learned_addr learned my addr 172.21.15.27:0/116587902 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:07:01.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.234+0000 7ff92b7fe700 1 --2- 172.21.15.27:0/116587902 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff92c0f6ff0 0x7ff92c107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:07:01.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.234+0000 7ff92bfff700 1 -- 172.21.15.27:0/116587902 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff92c0f6ff0 msgr2=0x7ff92c107450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:07:01.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.234+0000 7ff92bfff700 1 --2- 172.21.15.27:0/116587902 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff92c0f6ff0 0x7ff92c107450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:01.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.234+0000 7ff92bfff700 1 -- 172.21.15.27:0/116587902 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff920005040 con 0x7ff92c0f66e0 2021-11-03T18:07:01.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.234+0000 7ff92b7fe700 1 --2- 172.21.15.27:0/116587902 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff92c0f6ff0 0x7ff92c107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:07:01.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.234+0000 7ff92bfff700 1 --2- 172.21.15.27:0/116587902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff92c0f66e0 0x7ff92c106f60 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 rx=0x7ff920009990 tx=0x7ff920009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:07:01.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.234+0000 7ff928ff9700 1 -- 172.21.15.27:0/116587902 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff920018070 con 0x7ff92c0f66e0 2021-11-03T18:07:01.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.234+0000 7ff928ff9700 1 -- 172.21.15.27:0/116587902 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7ff920007ba0 con 0x7ff92c0f66e0 2021-11-03T18:07:01.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.234+0000 7ff932445700 1 -- 172.21.15.27:0/116587902 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff92c108480 con 0x7ff92c0f66e0 2021-11-03T18:07:01.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.234+0000 7ff928ff9700 1 -- 172.21.15.27:0/116587902 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff92001c3c0 con 0x7ff92c0f66e0 2021-11-03T18:07:01.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.234+0000 7ff932445700 1 -- 172.21.15.27:0/116587902 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff92c1089e0 con 0x7ff92c0f66e0 2021-11-03T18:07:01.242 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.235+0000 7ff932445700 1 -- 172.21.15.27:0/116587902 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff92c045ad0 con 0x7ff92c0f66e0 2021-11-03T18:07:01.242 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.236+0000 7ff928ff9700 1 -- 172.21.15.27:0/116587902 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7ff92001d420 con 0x7ff92c0f66e0 2021-11-03T18:07:01.242 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.236+0000 7ff928ff9700 1 --2- 172.21.15.27:0/116587902 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff9140640b0 0x7ff914066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:07:01.243 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.236+0000 7ff928ff9700 1 -- 172.21.15.27:0/116587902 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7ff920082490 con 0x7ff92c0f66e0 2021-11-03T18:07:01.243 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.236+0000 7ff92b7fe700 1 --2- 172.21.15.27:0/116587902 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff9140640b0 0x7ff914066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:07:01.243 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.237+0000 7ff92b7fe700 1 --2- 172.21.15.27:0/116587902 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff9140640b0 0x7ff914066570 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7ff91c009740 tx=0x7ff91c009be0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:07:01.243 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.240+0000 7ff928ff9700 1 -- 172.21.15.27:0/116587902 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7ff9200547f0 con 0x7ff92c0f66e0 2021-11-03T18:07:01.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.444+0000 7ff932445700 1 -- 172.21.15.27:0/116587902 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff92c0feed0 con 0x7ff92c0f66e0 2021-11-03T18:07:01.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.445+0000 7ff928ff9700 1 -- 172.21.15.27:0/116587902 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7ff9200543e0 con 0x7ff92c0f66e0 2021-11-03T18:07:01.443 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:07:01.444 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:07:01.444 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:07:01.444 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:07:01.444 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:07:01.445 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:07:01.445 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:07:01.445 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:07:01.445 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:07:01.446 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:07:01.446 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:07:01.446 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:07:01.446 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:07:01.446 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:07:01.447 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:07:01.447 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:07:01.447 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:07:01.447 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:07:01.448 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:07:01.448 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:07:01.448 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:07:01.448 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:07:01.448 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:07:01.449 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:07:01.449 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:07:01.449 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:07:01.449 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:07:01.450 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:07:01.450 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:07:01.450 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:07:01.450 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:07:01.450 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:07:01.451 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:07:01.451 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:07:01.451 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:07:01.451 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:07:01.452 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:07:01.452 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:07:01.452 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:07:01.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.446+0000 7ff932445700 1 -- 172.21.15.27:0/116587902 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff9140640b0 msgr2=0x7ff914066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:07:01.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.446+0000 7ff932445700 1 --2- 172.21.15.27:0/116587902 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff9140640b0 0x7ff914066570 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7ff91c009740 tx=0x7ff91c009be0).stop 2021-11-03T18:07:01.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.446+0000 7ff932445700 1 -- 172.21.15.27:0/116587902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff92c0f66e0 msgr2=0x7ff92c106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:07:01.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.446+0000 7ff932445700 1 --2- 172.21.15.27:0/116587902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff92c0f66e0 0x7ff92c106f60 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 rx=0x7ff920009990 tx=0x7ff920009b30).stop 2021-11-03T18:07:01.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.446+0000 7ff932445700 1 -- 172.21.15.27:0/116587902 shutdown_connections 2021-11-03T18:07:01.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.446+0000 7ff932445700 1 --2- 172.21.15.27:0/116587902 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff9140640b0 0x7ff914066570 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:01.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.446+0000 7ff932445700 1 --2- 172.21.15.27:0/116587902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff92c0f66e0 0x7ff92c106f60 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:01.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.446+0000 7ff932445700 1 --2- 172.21.15.27:0/116587902 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff92c0f6ff0 0x7ff92c107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:01.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.446+0000 7ff932445700 1 -- 172.21.15.27:0/116587902 >> 172.21.15.27:0/116587902 conn(0x7ff92c0f1f00 msgr2=0x7ff92c0fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:07:01.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.447+0000 7ff932445700 1 -- 172.21.15.27:0/116587902 shutdown_connections 2021-11-03T18:07:01.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:01.447+0000 7ff932445700 1 -- 172.21.15.27:0/116587902 wait complete. 2021-11-03T18:07:01.456 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:07:01.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:01 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:07:01.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:01 smithi115 ceph-mon[65453]: from='client.35720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:07:01.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:01 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3092435755' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:01.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:01 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:07:01.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:01 smithi027 ceph-mon[97244]: from='client.35720 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:07:01.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:01 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3092435755' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:02.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:02 smithi115 ceph-mon[65453]: pgmap v1571: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:02.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:02 smithi115 ceph-mon[65453]: from='client.35724 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:07:02.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:02 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/116587902' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:07:02.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:02 smithi027 ceph-mon[97244]: pgmap v1571: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:02.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:02 smithi027 ceph-mon[97244]: from='client.35724 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:07:02.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:02 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/116587902' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:07:04.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:04 smithi115 ceph-mon[65453]: pgmap v1572: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:04.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:04 smithi027 ceph-mon[97244]: pgmap v1572: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:05.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3067.433238 secs 2021-11-03T18:07:05.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3067.433238 secs 2021-11-03T18:07:06.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:06 smithi115 ceph-mon[65453]: pgmap v1573: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:06.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:06 smithi027 ceph-mon[97244]: pgmap v1573: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:08.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:08 smithi115 ceph-mon[65453]: pgmap v1574: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:08.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:08 smithi027 ceph-mon[97244]: pgmap v1574: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:10.321 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:10 smithi027 ceph-mon[97244]: pgmap v1575: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:10.322 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3072.433305 secs 2021-11-03T18:07:10.322 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:07:10.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:10 smithi115 ceph-mon[65453]: pgmap v1575: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:10.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3072.433305 secs 2021-11-03T18:07:10.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:07:11.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:11 smithi115 ceph-mon[65453]: poll took 20061.458 ms: 1 events 2021-11-03T18:07:11.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:07:11.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:07:11.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:07:11.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:11.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:11.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:11.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:11.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:11 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:07:11.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:11 smithi027 ceph-mon[97244]: poll took 20061.458 ms: 1 events 2021-11-03T18:07:11.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:07:11.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:07:11.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:07:11.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:11.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:11.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:11.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:11.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:11 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:07:12.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:12 smithi115 ceph-mon[65453]: pgmap v1576: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:12.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:12 smithi027 ceph-mon[97244]: pgmap v1576: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:14.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:14 smithi115 ceph-mon[65453]: pgmap v1577: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:14.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:14 smithi027 ceph-mon[97244]: pgmap v1577: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:15.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3077.433455 secs 2021-11-03T18:07:15.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3077.433455 secs 2021-11-03T18:07:16.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:16 smithi115 ceph-mon[65453]: pgmap v1578: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:16.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:16 smithi027 ceph-mon[97244]: pgmap v1578: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:18.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:18 smithi115 ceph-mon[65453]: pgmap v1579: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:18.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:18 smithi027 ceph-mon[97244]: pgmap v1579: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:20.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:20 smithi115 ceph-mon[65453]: pgmap v1580: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:20.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3082.433547 secs 2021-11-03T18:07:20.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:20 smithi027 ceph-mon[97244]: pgmap v1580: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:20.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3082.433547 secs 2021-11-03T18:07:21.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:07:21.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:07:21.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:07:21.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:21.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:21.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:21.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:21.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:21 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:07:21.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:07:21.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:07:21.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:07:21.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:21.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:21.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:21.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:21.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:21 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:07:22.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:22 smithi115 ceph-mon[65453]: pgmap v1581: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:22.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:22 smithi027 ceph-mon[97244]: pgmap v1581: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:24.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:24 smithi115 ceph-mon[65453]: pgmap v1582: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:24.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:24 smithi027 ceph-mon[97244]: pgmap v1582: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:25.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3087.433669 secs 2021-11-03T18:07:25.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3087.433669 secs 2021-11-03T18:07:26.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:26 smithi115 ceph-mon[65453]: pgmap v1583: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:26.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:26 smithi027 ceph-mon[97244]: pgmap v1583: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:28.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:28 smithi115 ceph-mon[65453]: pgmap v1584: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:28.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:28 smithi027 ceph-mon[97244]: pgmap v1584: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:30.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:30 smithi115 ceph-mon[65453]: pgmap v1585: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:30.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3092.433739 secs 2021-11-03T18:07:30.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:30 smithi027 ceph-mon[97244]: pgmap v1585: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:30.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3092.433739 secs 2021-11-03T18:07:31.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.563+0000 7f692b59e700 1 -- 172.21.15.27:0/1662538508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f692c0f77d0 msgr2=0x7f692c0f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:07:31.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.563+0000 7f692b59e700 1 --2- 172.21.15.27:0/1662538508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f692c0f77d0 0x7f692c0f7c30 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 rx=0x7f6920004660 tx=0x7f6920009b30).stop 2021-11-03T18:07:31.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.563+0000 7f692b59e700 1 -- 172.21.15.27:0/1662538508 shutdown_connections 2021-11-03T18:07:31.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.563+0000 7f692b59e700 1 --2- 172.21.15.27:0/1662538508 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f692c0f77d0 0x7f692c0f7c30 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:31.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.563+0000 7f692b59e700 1 --2- 172.21.15.27:0/1662538508 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f692c0f6530 0x7f692c0f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:31.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.563+0000 7f692b59e700 1 -- 172.21.15.27:0/1662538508 >> 172.21.15.27:0/1662538508 conn(0x7f692c0f1cd0 msgr2=0x7f692c0f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:07:31.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.563+0000 7f692b59e700 1 -- 172.21.15.27:0/1662538508 shutdown_connections 2021-11-03T18:07:31.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.563+0000 7f692b59e700 1 -- 172.21.15.27:0/1662538508 wait complete. 2021-11-03T18:07:31.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.563+0000 7f692b59e700 1 Processor -- start 2021-11-03T18:07:31.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.563+0000 7f692b59e700 1 -- start start 2021-11-03T18:07:31.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.564+0000 7f692b59e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f692c0f6530 0x7f692c106c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:07:31.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.564+0000 7f692b59e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f692c0f77d0 0x7f692c107100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:07:31.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.564+0000 7f692b59e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692c1076b0 con 0x7f692c0f6530 2021-11-03T18:07:31.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.564+0000 7f692b59e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692c1077f0 con 0x7f692c0f77d0 2021-11-03T18:07:31.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.564+0000 7f692a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f692c0f6530 0x7f692c106c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:07:31.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.564+0000 7f692a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f692c0f6530 0x7f692c106c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33058/0 (socket says 172.21.15.27:33058) 2021-11-03T18:07:31.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.564+0000 7f692a59c700 1 -- 172.21.15.27:0/3696575402 learned_addr learned my addr 172.21.15.27:0/3696575402 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:07:31.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.564+0000 7f6929d9b700 1 --2- 172.21.15.27:0/3696575402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f692c0f77d0 0x7f692c107100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:07:31.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.564+0000 7f692a59c700 1 -- 172.21.15.27:0/3696575402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f692c0f77d0 msgr2=0x7f692c107100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:07:31.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.564+0000 7f692a59c700 1 --2- 172.21.15.27:0/3696575402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f692c0f77d0 0x7f692c107100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:31.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.564+0000 7f692a59c700 1 -- 172.21.15.27:0/3696575402 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6920005040 con 0x7f692c0f6530 2021-11-03T18:07:31.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.564+0000 7f6929d9b700 1 --2- 172.21.15.27:0/3696575402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f692c0f77d0 0x7f692c107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:07:31.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.564+0000 7f692a59c700 1 --2- 172.21.15.27:0/3696575402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f692c0f6530 0x7f692c106c10 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 rx=0x7f691c002700 tx=0x7f691c009e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:07:31.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.564+0000 7f691affd700 1 -- 172.21.15.27:0/3696575402 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f691c009320 con 0x7f692c0f6530 2021-11-03T18:07:31.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.564+0000 7f692b59e700 1 -- 172.21.15.27:0/3696575402 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f692c108190 con 0x7f692c0f6530 2021-11-03T18:07:31.568 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.564+0000 7f691affd700 1 -- 172.21.15.27:0/3696575402 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f691c00f040 con 0x7f692c0f6530 2021-11-03T18:07:31.568 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.564+0000 7f691affd700 1 -- 172.21.15.27:0/3696575402 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f691c0148e0 con 0x7f692c0f6530 2021-11-03T18:07:31.568 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.565+0000 7f692b59e700 1 -- 172.21.15.27:0/3696575402 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f692c108750 con 0x7f692c0f6530 2021-11-03T18:07:31.569 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.565+0000 7f692b59e700 1 -- 172.21.15.27:0/3696575402 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f692c045ad0 con 0x7f692c0f6530 2021-11-03T18:07:31.569 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.566+0000 7f691affd700 1 -- 172.21.15.27:0/3696575402 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f691c009480 con 0x7f692c0f6530 2021-11-03T18:07:31.570 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.566+0000 7f691affd700 1 --2- 172.21.15.27:0/3696575402 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6914064100 0x7f69140665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:07:31.570 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.566+0000 7f691affd700 1 -- 172.21.15.27:0/3696575402 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f691c080b50 con 0x7f692c0f6530 2021-11-03T18:07:31.570 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.566+0000 7f6929d9b700 1 --2- 172.21.15.27:0/3696575402 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6914064100 0x7f69140665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:07:31.570 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.567+0000 7f6929d9b700 1 --2- 172.21.15.27:0/3696575402 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6914064100 0x7f69140665c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7f6920004e30 tx=0x7f6920009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:07:31.571 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.571+0000 7f691affd700 1 -- 172.21.15.27:0/3696575402 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f691c052e50 con 0x7f692c0f6530 2021-11-03T18:07:31.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:07:31.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:07:31.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:07:31.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:31.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:31.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:31.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:31.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:31 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:07:31.746 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.749+0000 7f692b59e700 1 -- 172.21.15.27:0/3696575402 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f692c1089d0 con 0x7f6914064100 2021-11-03T18:07:31.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.750+0000 7f691affd700 1 -- 172.21.15.27:0/3696575402 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f692c1089d0 con 0x7f6914064100 2021-11-03T18:07:31.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.752+0000 7f692b59e700 1 -- 172.21.15.27:0/3696575402 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6914064100 msgr2=0x7f69140665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:07:31.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.752+0000 7f692b59e700 1 --2- 172.21.15.27:0/3696575402 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6914064100 0x7f69140665c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 rx=0x7f6920004e30 tx=0x7f6920009b30).stop 2021-11-03T18:07:31.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.752+0000 7f692b59e700 1 -- 172.21.15.27:0/3696575402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f692c0f6530 msgr2=0x7f692c106c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:07:31.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.752+0000 7f692b59e700 1 --2- 172.21.15.27:0/3696575402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f692c0f6530 0x7f692c106c10 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 rx=0x7f691c002700 tx=0x7f691c009e00).stop 2021-11-03T18:07:31.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.752+0000 7f692b59e700 1 -- 172.21.15.27:0/3696575402 shutdown_connections 2021-11-03T18:07:31.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.752+0000 7f692b59e700 1 --2- 172.21.15.27:0/3696575402 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6914064100 0x7f69140665c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:31.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.752+0000 7f692b59e700 1 --2- 172.21.15.27:0/3696575402 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f692c0f6530 0x7f692c106c10 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:31.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.752+0000 7f692b59e700 1 --2- 172.21.15.27:0/3696575402 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f692c0f77d0 0x7f692c107100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:31.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.752+0000 7f692b59e700 1 -- 172.21.15.27:0/3696575402 >> 172.21.15.27:0/3696575402 conn(0x7f692c0f1cd0 msgr2=0x7f692c0fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:07:31.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.752+0000 7f692b59e700 1 -- 172.21.15.27:0/3696575402 shutdown_connections 2021-11-03T18:07:31.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.752+0000 7f692b59e700 1 -- 172.21.15.27:0/3696575402 wait complete. 2021-11-03T18:07:31.759 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:07:31.770 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:07:31.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:07:31.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:07:31.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:31.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:31.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:31.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:31.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:31 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:07:31.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.852+0000 7fbaa9337700 1 -- 172.21.15.27:0/3461916806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaa40f48a0 msgr2=0x7fbaa40f4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:07:31.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.852+0000 7fbaa9337700 1 --2- 172.21.15.27:0/3461916806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaa40f48a0 0x7fbaa40f4cc0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 rx=0x7fba94004660 tx=0x7fba94009b30).stop 2021-11-03T18:07:31.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.852+0000 7fbaa9337700 1 -- 172.21.15.27:0/3461916806 shutdown_connections 2021-11-03T18:07:31.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.852+0000 7fbaa9337700 1 --2- 172.21.15.27:0/3461916806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbaa40f5240 0x7fbaa40f7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:31.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.852+0000 7fbaa9337700 1 --2- 172.21.15.27:0/3461916806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaa40f48a0 0x7fbaa40f4cc0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:31.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.852+0000 7fbaa9337700 1 -- 172.21.15.27:0/3461916806 >> 172.21.15.27:0/3461916806 conn(0x7fbaa40f0320 msgr2=0x7fbaa40f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:07:31.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.852+0000 7fbaa9337700 1 -- 172.21.15.27:0/3461916806 shutdown_connections 2021-11-03T18:07:31.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.852+0000 7fbaa9337700 1 -- 172.21.15.27:0/3461916806 wait complete. 2021-11-03T18:07:31.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.853+0000 7fbaa9337700 1 Processor -- start 2021-11-03T18:07:31.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.853+0000 7fbaa9337700 1 -- start start 2021-11-03T18:07:31.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.853+0000 7fbaa9337700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaa40f48a0 0x7fbaa40f7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:07:31.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.853+0000 7fbaa9337700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbaa40f5240 0x7fbaa40f5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:07:31.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.853+0000 7fbaa9337700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa40f5e70 con 0x7fbaa40f48a0 2021-11-03T18:07:31.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.853+0000 7fbaa9337700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbaa40f5fb0 con 0x7fbaa40f5240 2021-11-03T18:07:31.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.853+0000 7fbaa3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaa40f48a0 0x7fbaa40f7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:07:31.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.853+0000 7fbaa3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaa40f48a0 0x7fbaa40f7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33068/0 (socket says 172.21.15.27:33068) 2021-11-03T18:07:31.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.853+0000 7fbaa3fff700 1 -- 172.21.15.27:0/3409364487 learned_addr learned my addr 172.21.15.27:0/3409364487 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:07:31.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.854+0000 7fbaa37fe700 1 --2- 172.21.15.27:0/3409364487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbaa40f5240 0x7fbaa40f5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:07:31.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.854+0000 7fbaa3fff700 1 -- 172.21.15.27:0/3409364487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbaa40f5240 msgr2=0x7fbaa40f5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:07:31.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.854+0000 7fbaa3fff700 1 --2- 172.21.15.27:0/3409364487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbaa40f5240 0x7fbaa40f5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:31.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.854+0000 7fbaa3fff700 1 -- 172.21.15.27:0/3409364487 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba94005040 con 0x7fbaa40f48a0 2021-11-03T18:07:31.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.854+0000 7fbaa3fff700 1 --2- 172.21.15.27:0/3409364487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaa40f48a0 0x7fbaa40f7490 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 rx=0x7fba94009990 tx=0x7fba94009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:07:31.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.854+0000 7fbaa0ff9700 1 -- 172.21.15.27:0/3409364487 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba94018070 con 0x7fbaa40f48a0 2021-11-03T18:07:31.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.854+0000 7fbaa9337700 1 -- 172.21.15.27:0/3409364487 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbaa40f6230 con 0x7fbaa40f48a0 2021-11-03T18:07:31.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.854+0000 7fbaa0ff9700 1 -- 172.21.15.27:0/3409364487 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fba94007ba0 con 0x7fbaa40f48a0 2021-11-03T18:07:31.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.854+0000 7fbaa0ff9700 1 -- 172.21.15.27:0/3409364487 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba9401c3c0 con 0x7fbaa40f48a0 2021-11-03T18:07:31.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.854+0000 7fbaa9337700 1 -- 172.21.15.27:0/3409364487 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaa40f6790 con 0x7fbaa40f48a0 2021-11-03T18:07:31.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.855+0000 7fbaa9337700 1 -- 172.21.15.27:0/3409364487 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbaa4045ad0 con 0x7fbaa40f48a0 2021-11-03T18:07:31.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.856+0000 7fbaa0ff9700 1 -- 172.21.15.27:0/3409364487 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fba9401d420 con 0x7fbaa40f48a0 2021-11-03T18:07:31.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.856+0000 7fbaa0ff9700 1 --2- 172.21.15.27:0/3409364487 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fba8c064100 0x7fba8c0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:07:31.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.856+0000 7fbaa0ff9700 1 -- 172.21.15.27:0/3409364487 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fba940823a0 con 0x7fbaa40f48a0 2021-11-03T18:07:31.860 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.856+0000 7fbaa37fe700 1 --2- 172.21.15.27:0/3409364487 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fba8c064100 0x7fba8c0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:07:31.860 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.856+0000 7fbaa37fe700 1 --2- 172.21.15.27:0/3409364487 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fba8c064100 0x7fba8c0665c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7fbaa40f56c0 tx=0x7fba9800b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:07:31.860 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:31.860+0000 7fbaa0ff9700 1 -- 172.21.15.27:0/3409364487 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fba94054760 con 0x7fbaa40f48a0 2021-11-03T18:07:32.030 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.033+0000 7fbaa9337700 1 -- 172.21.15.27:0/3409364487 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbaa40fef50 con 0x7fba8c064100 2021-11-03T18:07:32.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.047+0000 7fbaa0ff9700 1 -- 172.21.15.27:0/3409364487 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fbaa40fef50 con 0x7fba8c064100 2021-11-03T18:07:32.045 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:07:32.045 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (59m) 8m ago 60m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:07:32.045 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (50m) 8m ago 60m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:07:32.046 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (50m) 7m ago 59m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:07:32.046 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (58m) 8m ago 60m 41.8M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:07:32.046 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (52m) 8m ago 55m 26.0M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:07:32.046 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (55m) 8m ago 55m 3845M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:07:32.047 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (51m) 7m ago 55m 25.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:07:32.047 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (52m) 7m ago 55m 26.8M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:07:32.047 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (52m) 8m ago 61m 554M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:07:32.047 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (52m) 7m ago 59m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:07:32.048 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (51m) 8m ago 61m 235M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:07:32.048 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (51m) 7m ago 59m 112M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:07:32.048 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (59m) 8m ago 59m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:07:32.048 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (59m) 7m ago 59m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:07:32.048 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (50m) 8m ago 58m 351M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:07:32.049 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (49m) 8m ago 58m 383M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:07:32.049 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (49m) 8m ago 58m 284M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:07:32.049 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (48m) 7m ago 57m 404M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:07:32.049 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (48m) 7m ago 57m 322M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:07:32.049 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (48m) 7m ago 57m 327M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:07:32.050 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (58m) 8m ago 59m 55.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:07:32.050 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.049+0000 7fbaa9337700 1 -- 172.21.15.27:0/3409364487 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fba8c064100 msgr2=0x7fba8c0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:07:32.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.049+0000 7fbaa9337700 1 --2- 172.21.15.27:0/3409364487 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fba8c064100 0x7fba8c0665c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7fbaa40f56c0 tx=0x7fba9800b040).stop 2021-11-03T18:07:32.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.049+0000 7fbaa9337700 1 -- 172.21.15.27:0/3409364487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaa40f48a0 msgr2=0x7fbaa40f7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:07:32.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.049+0000 7fbaa9337700 1 --2- 172.21.15.27:0/3409364487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaa40f48a0 0x7fbaa40f7490 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 rx=0x7fba94009990 tx=0x7fba94009b30).stop 2021-11-03T18:07:32.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.049+0000 7fbaa9337700 1 -- 172.21.15.27:0/3409364487 shutdown_connections 2021-11-03T18:07:32.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.049+0000 7fbaa9337700 1 --2- 172.21.15.27:0/3409364487 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fba8c064100 0x7fba8c0665c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:32.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.049+0000 7fbaa9337700 1 --2- 172.21.15.27:0/3409364487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbaa40f48a0 0x7fbaa40f7490 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:32.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.049+0000 7fbaa9337700 1 --2- 172.21.15.27:0/3409364487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbaa40f5240 0x7fbaa40f5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:32.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.049+0000 7fbaa9337700 1 -- 172.21.15.27:0/3409364487 >> 172.21.15.27:0/3409364487 conn(0x7fbaa40f0320 msgr2=0x7fbaa40fb5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:07:32.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.049+0000 7fbaa9337700 1 -- 172.21.15.27:0/3409364487 shutdown_connections 2021-11-03T18:07:32.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.049+0000 7fbaa9337700 1 -- 172.21.15.27:0/3409364487 wait complete. 2021-11-03T18:07:32.147 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.149+0000 7f9688ea3700 1 -- 172.21.15.27:0/678072238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9684060270 msgr2=0x7f96840f91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:07:32.147 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.149+0000 7f9688ea3700 1 --2- 172.21.15.27:0/678072238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9684060270 0x7f96840f91c0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 rx=0x7f9674004660 tx=0x7f9674009b30).stop 2021-11-03T18:07:32.147 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.149+0000 7f9688ea3700 1 -- 172.21.15.27:0/678072238 shutdown_connections 2021-11-03T18:07:32.148 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.149+0000 7f9688ea3700 1 --2- 172.21.15.27:0/678072238 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96840f96b0 0x7f96840fbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:32.148 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.149+0000 7f9688ea3700 1 --2- 172.21.15.27:0/678072238 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9684060270 0x7f96840f91c0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:32.148 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.149+0000 7f9688ea3700 1 -- 172.21.15.27:0/678072238 >> 172.21.15.27:0/678072238 conn(0x7f96840f0ba0 msgr2=0x7f96840f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:07:32.149 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.150+0000 7f9688ea3700 1 -- 172.21.15.27:0/678072238 shutdown_connections 2021-11-03T18:07:32.149 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.150+0000 7f9688ea3700 1 -- 172.21.15.27:0/678072238 wait complete. 2021-11-03T18:07:32.149 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.150+0000 7f9688ea3700 1 Processor -- start 2021-11-03T18:07:32.149 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.150+0000 7f9688ea3700 1 -- start start 2021-11-03T18:07:32.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.150+0000 7f9688ea3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9684060270 0x7f9684106e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:07:32.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.150+0000 7f9688ea3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96840f96b0 0x7f9684107310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:07:32.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.150+0000 7f9688ea3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9684107910 con 0x7f96840f96b0 2021-11-03T18:07:32.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.150+0000 7f9688ea3700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9684108120 con 0x7f9684060270 2021-11-03T18:07:32.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.151+0000 7f9682ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96840f96b0 0x7f9684107310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:07:32.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.151+0000 7f9682ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96840f96b0 0x7f9684107310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33080/0 (socket says 172.21.15.27:33080) 2021-11-03T18:07:32.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.151+0000 7f9682ffd700 1 -- 172.21.15.27:0/2335325434 learned_addr learned my addr 172.21.15.27:0/2335325434 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:07:32.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.151+0000 7f96837fe700 1 --2- 172.21.15.27:0/2335325434 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9684060270 0x7f9684106e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:07:32.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.151+0000 7f9682ffd700 1 -- 172.21.15.27:0/2335325434 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9684060270 msgr2=0x7f9684106e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:07:32.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.151+0000 7f9682ffd700 1 --2- 172.21.15.27:0/2335325434 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9684060270 0x7f9684106e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:32.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.151+0000 7f9682ffd700 1 -- 172.21.15.27:0/2335325434 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9674005040 con 0x7f96840f96b0 2021-11-03T18:07:32.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.151+0000 7f9682ffd700 1 --2- 172.21.15.27:0/2335325434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96840f96b0 0x7f9684107310 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 rx=0x7f9678002700 tx=0x7f9678009d60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:07:32.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.151+0000 7f966bfff700 1 -- 172.21.15.27:0/2335325434 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96780079c0 con 0x7f96840f96b0 2021-11-03T18:07:32.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.151+0000 7f966bfff700 1 -- 172.21.15.27:0/2335325434 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f967800f040 con 0x7f96840f96b0 2021-11-03T18:07:32.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.151+0000 7f9688ea3700 1 -- 172.21.15.27:0/2335325434 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96841083a0 con 0x7f96840f96b0 2021-11-03T18:07:32.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.151+0000 7f966bfff700 1 -- 172.21.15.27:0/2335325434 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9678014740 con 0x7f96840f96b0 2021-11-03T18:07:32.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.151+0000 7f9688ea3700 1 -- 172.21.15.27:0/2335325434 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9684108960 con 0x7f96840f96b0 2021-11-03T18:07:32.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.152+0000 7f9688ea3700 1 -- 172.21.15.27:0/2335325434 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9684059100 con 0x7f96840f96b0 2021-11-03T18:07:32.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.153+0000 7f966bfff700 1 -- 172.21.15.27:0/2335325434 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f967801b060 con 0x7f96840f96b0 2021-11-03T18:07:32.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.153+0000 7f966bfff700 1 --2- 172.21.15.27:0/2335325434 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f966c064100 0x7f966c0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:07:32.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.153+0000 7f966bfff700 1 -- 172.21.15.27:0/2335325434 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f96780806f0 con 0x7f96840f96b0 2021-11-03T18:07:32.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.153+0000 7f96837fe700 1 --2- 172.21.15.27:0/2335325434 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f966c064100 0x7f966c0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:07:32.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.153+0000 7f96837fe700 1 --2- 172.21.15.27:0/2335325434 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f966c064100 0x7f966c0665c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7f9674005010 tx=0x7f9674009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:07:32.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.157+0000 7f966bfff700 1 -- 172.21.15.27:0/2335325434 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f9678052990 con 0x7f96840f96b0 2021-11-03T18:07:32.388 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.390+0000 7f9688ea3700 1 -- 172.21.15.27:0/2335325434 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f96840feec0 con 0x7f96840f96b0 2021-11-03T18:07:32.389 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.391+0000 7f966bfff700 1 -- 172.21.15.27:0/2335325434 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f96780192e0 con 0x7f96840f96b0 2021-11-03T18:07:32.389 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:07:32.389 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:07:32.390 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:07:32.390 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:07:32.390 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:07:32.390 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:07:32.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:07:32.391 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:07:32.391 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:07:32.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:07:32.392 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:07:32.392 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:07:32.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:07:32.392 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:07:32.393 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:07:32.393 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:07:32.393 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:07:32.393 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:07:32.394 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.392+0000 7f9688ea3700 1 -- 172.21.15.27:0/2335325434 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f966c064100 msgr2=0x7f966c0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:07:32.394 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.392+0000 7f9688ea3700 1 --2- 172.21.15.27:0/2335325434 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f966c064100 0x7f966c0665c0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7f9674005010 tx=0x7f9674009b30).stop 2021-11-03T18:07:32.395 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.392+0000 7f9688ea3700 1 -- 172.21.15.27:0/2335325434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96840f96b0 msgr2=0x7f9684107310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:07:32.395 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.392+0000 7f9688ea3700 1 --2- 172.21.15.27:0/2335325434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96840f96b0 0x7f9684107310 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 rx=0x7f9678002700 tx=0x7f9678009d60).stop 2021-11-03T18:07:32.395 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.392+0000 7f9688ea3700 1 -- 172.21.15.27:0/2335325434 shutdown_connections 2021-11-03T18:07:32.395 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.392+0000 7f9688ea3700 1 --2- 172.21.15.27:0/2335325434 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f966c064100 0x7f966c0665c0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:32.396 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.392+0000 7f9688ea3700 1 --2- 172.21.15.27:0/2335325434 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9684060270 0x7f9684106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:32.396 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.393+0000 7f9688ea3700 1 --2- 172.21.15.27:0/2335325434 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96840f96b0 0x7f9684107310 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:32.396 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.393+0000 7f9688ea3700 1 -- 172.21.15.27:0/2335325434 >> 172.21.15.27:0/2335325434 conn(0x7f96840f0ba0 msgr2=0x7f96840f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:07:32.397 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.393+0000 7f9688ea3700 1 -- 172.21.15.27:0/2335325434 shutdown_connections 2021-11-03T18:07:32.397 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.393+0000 7f9688ea3700 1 -- 172.21.15.27:0/2335325434 wait complete. 2021-11-03T18:07:32.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.493+0000 7f39f92bb700 1 -- 172.21.15.27:0/3904199567 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39f40f5340 msgr2=0x7f39f40f7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:07:32.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.493+0000 7f39f92bb700 1 --2- 172.21.15.27:0/3904199567 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39f40f5340 0x7f39f40f7ac0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 rx=0x7f39dc004660 tx=0x7f39dc009b30).stop 2021-11-03T18:07:32.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.494+0000 7f39f92bb700 1 -- 172.21.15.27:0/3904199567 shutdown_connections 2021-11-03T18:07:32.494 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.494+0000 7f39f92bb700 1 --2- 172.21.15.27:0/3904199567 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39f40f5340 0x7f39f40f7ac0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:32.494 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.494+0000 7f39f92bb700 1 --2- 172.21.15.27:0/3904199567 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39f40f49a0 0x7f39f40f4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:32.494 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.494+0000 7f39f92bb700 1 -- 172.21.15.27:0/3904199567 >> 172.21.15.27:0/3904199567 conn(0x7f39f40f04c0 msgr2=0x7f39f40f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:07:32.494 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.494+0000 7f39f92bb700 1 -- 172.21.15.27:0/3904199567 shutdown_connections 2021-11-03T18:07:32.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.494+0000 7f39f92bb700 1 -- 172.21.15.27:0/3904199567 wait complete. 2021-11-03T18:07:32.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.494+0000 7f39f92bb700 1 Processor -- start 2021-11-03T18:07:32.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.494+0000 7f39f92bb700 1 -- start start 2021-11-03T18:07:32.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.494+0000 7f39f92bb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39f40f49a0 0x7f39f4106fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:07:32.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.494+0000 7f39f92bb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39f40f5340 0x7f39f41074b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:07:32.496 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.494+0000 7f39f92bb700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f4107a60 con 0x7f39f40f5340 2021-11-03T18:07:32.496 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.494+0000 7f39f92bb700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39f4107ba0 con 0x7f39f40f49a0 2021-11-03T18:07:32.496 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.495+0000 7f39f27fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39f40f5340 0x7f39f41074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:07:32.496 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.495+0000 7f39f27fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39f40f5340 0x7f39f41074b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33090/0 (socket says 172.21.15.27:33090) 2021-11-03T18:07:32.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.495+0000 7f39f27fc700 1 -- 172.21.15.27:0/1485847462 learned_addr learned my addr 172.21.15.27:0/1485847462 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:07:32.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.495+0000 7f39f2ffd700 1 --2- 172.21.15.27:0/1485847462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39f40f49a0 0x7f39f4106fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:07:32.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.495+0000 7f39f27fc700 1 -- 172.21.15.27:0/1485847462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39f40f49a0 msgr2=0x7f39f4106fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:07:32.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.495+0000 7f39f27fc700 1 --2- 172.21.15.27:0/1485847462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39f40f49a0 0x7f39f4106fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:32.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.495+0000 7f39f27fc700 1 -- 172.21.15.27:0/1485847462 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39dc005040 con 0x7f39f40f5340 2021-11-03T18:07:32.498 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.495+0000 7f39f27fc700 1 --2- 172.21.15.27:0/1485847462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39f40f5340 0x7f39f41074b0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 rx=0x7f39dc013a70 tx=0x7f39dc009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:07:32.498 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.495+0000 7f39eb7fe700 1 -- 172.21.15.27:0/1485847462 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39dc018070 con 0x7f39f40f5340 2021-11-03T18:07:32.498 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.495+0000 7f39f92bb700 1 -- 172.21.15.27:0/1485847462 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39f41084e0 con 0x7f39f40f5340 2021-11-03T18:07:32.498 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.495+0000 7f39eb7fe700 1 -- 172.21.15.27:0/1485847462 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f39dc0097d0 con 0x7f39f40f5340 2021-11-03T18:07:32.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.495+0000 7f39eb7fe700 1 -- 172.21.15.27:0/1485847462 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39dc01de80 con 0x7f39f40f5340 2021-11-03T18:07:32.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.495+0000 7f39f92bb700 1 -- 172.21.15.27:0/1485847462 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39f4108a40 con 0x7f39f40f5340 2021-11-03T18:07:32.500 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.497+0000 7f39f92bb700 1 -- 172.21.15.27:0/1485847462 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39f4045ad0 con 0x7f39f40f5340 2021-11-03T18:07:32.500 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.497+0000 7f39eb7fe700 1 -- 172.21.15.27:0/1485847462 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f39dc007ba0 con 0x7f39f40f5340 2021-11-03T18:07:32.501 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.497+0000 7f39eb7fe700 1 --2- 172.21.15.27:0/1485847462 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f39e00640b0 0x7f39e0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:07:32.501 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.497+0000 7f39eb7fe700 1 -- 172.21.15.27:0/1485847462 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f39dc082430 con 0x7f39f40f5340 2021-11-03T18:07:32.501 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.497+0000 7f39f2ffd700 1 --2- 172.21.15.27:0/1485847462 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f39e00640b0 0x7f39e0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:07:32.501 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.498+0000 7f39f2ffd700 1 --2- 172.21.15.27:0/1485847462 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f39e00640b0 0x7f39e0066570 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f39f405f4c0 tx=0x7f39e400b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:07:32.501 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.501+0000 7f39eb7fe700 1 -- 172.21.15.27:0/1485847462 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f39dc0547f0 con 0x7f39f40f5340 2021-11-03T18:07:32.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.703+0000 7f39f92bb700 1 -- 172.21.15.27:0/1485847462 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f39f4027a00 con 0x7f39f40f5340 2021-11-03T18:07:32.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.704+0000 7f39eb7fe700 1 -- 172.21.15.27:0/1485847462 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f39dc0543e0 con 0x7f39f40f5340 2021-11-03T18:07:32.702 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:07:32.702 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:07:32.702 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:07:32.702 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:07:32.703 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:07:32.703 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:07:32.703 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:07:32.703 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:07:32.704 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:07:32.704 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:07:32.704 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:07:32.704 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:07:32.704 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:07:32.705 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:07:32.705 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:07:32.705 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:07:32.705 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:07:32.705 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:07:32.706 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:07:32.706 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:07:32.706 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:07:32.706 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:07:32.707 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:07:32.707 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:07:32.707 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:07:32.707 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:07:32.708 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:07:32.708 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:07:32.708 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:07:32.708 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:07:32.708 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:07:32.709 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:07:32.709 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:07:32.709 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:07:32.709 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:07:32.710 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:07:32.710 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:07:32.710 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:07:32.710 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:07:32.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.705+0000 7f39f92bb700 1 -- 172.21.15.27:0/1485847462 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f39e00640b0 msgr2=0x7f39e0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:07:32.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.705+0000 7f39f92bb700 1 --2- 172.21.15.27:0/1485847462 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f39e00640b0 0x7f39e0066570 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7f39f405f4c0 tx=0x7f39e400b040).stop 2021-11-03T18:07:32.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.705+0000 7f39f92bb700 1 -- 172.21.15.27:0/1485847462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39f40f5340 msgr2=0x7f39f41074b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:07:32.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.705+0000 7f39f92bb700 1 --2- 172.21.15.27:0/1485847462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39f40f5340 0x7f39f41074b0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 rx=0x7f39dc013a70 tx=0x7f39dc009b30).stop 2021-11-03T18:07:32.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.706+0000 7f39f92bb700 1 -- 172.21.15.27:0/1485847462 shutdown_connections 2021-11-03T18:07:32.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.706+0000 7f39f92bb700 1 --2- 172.21.15.27:0/1485847462 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f39e00640b0 0x7f39e0066570 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:32.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.706+0000 7f39f92bb700 1 --2- 172.21.15.27:0/1485847462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39f40f49a0 0x7f39f4106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:32.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.706+0000 7f39f92bb700 1 --2- 172.21.15.27:0/1485847462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39f40f5340 0x7f39f41074b0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:07:32.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.706+0000 7f39f92bb700 1 -- 172.21.15.27:0/1485847462 >> 172.21.15.27:0/1485847462 conn(0x7f39f40f04c0 msgr2=0x7f39f40fb510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:07:32.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.706+0000 7f39f92bb700 1 -- 172.21.15.27:0/1485847462 shutdown_connections 2021-11-03T18:07:32.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:07:32.706+0000 7f39f92bb700 1 -- 172.21.15.27:0/1485847462 wait complete. 2021-11-03T18:07:32.714 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:07:32.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:32 smithi115 ceph-mon[65453]: pgmap v1586: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:32.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:32 smithi027 ceph-mon[97244]: pgmap v1586: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:33.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:33 smithi115 ceph-mon[65453]: from='client.35736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:07:33.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:33 smithi115 ceph-mon[65453]: from='client.35740 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:07:33.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:33 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2335325434' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:33.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:33 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1485847462' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:07:33.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:33 smithi027 ceph-mon[97244]: from='client.35736 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:07:33.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:33 smithi027 ceph-mon[97244]: from='client.35740 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:07:33.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:33 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2335325434' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:33.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:33 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1485847462' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:07:34.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:34 smithi115 ceph-mon[65453]: pgmap v1587: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:34.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:34 smithi027 ceph-mon[97244]: pgmap v1587: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:35.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3097.433862 secs 2021-11-03T18:07:35.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3097.433862 secs 2021-11-03T18:07:36.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:36 smithi115 ceph-mon[65453]: pgmap v1588: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:36.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:36 smithi027 ceph-mon[97244]: pgmap v1588: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:38.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:38 smithi115 ceph-mon[65453]: pgmap v1589: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:38.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:38 smithi027 ceph-mon[97244]: pgmap v1589: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:40.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:40 smithi115 ceph-mon[65453]: pgmap v1590: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:40.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3102.433931 secs 2021-11-03T18:07:40.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:07:40.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:40 smithi027 ceph-mon[97244]: pgmap v1590: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:40.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3102.433931 secs 2021-11-03T18:07:40.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:07:41.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:07:41.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:07:41.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:41.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:41.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:41.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:41.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:41 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:07:41.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:07:41.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:07:41.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:41.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:41.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:41.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:41.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:41 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:07:42.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:42 smithi115 ceph-mon[65453]: pgmap v1591: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:42.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:42 smithi027 ceph-mon[97244]: pgmap v1591: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:44.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:44 smithi115 ceph-mon[65453]: pgmap v1592: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:45.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3107.434046 secs 2021-11-03T18:07:45.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:44 smithi027 ceph-mon[97244]: pgmap v1592: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:45.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3107.434046 secs 2021-11-03T18:07:46.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:46 smithi115 ceph-mon[65453]: pgmap v1593: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:46.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:46 smithi027 ceph-mon[97244]: pgmap v1593: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:48.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:48 smithi115 ceph-mon[65453]: pgmap v1594: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:48.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:48 smithi027 ceph-mon[97244]: pgmap v1594: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:50.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:50 smithi115 ceph-mon[65453]: pgmap v1595: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:50.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3112.434164 secs 2021-11-03T18:07:50.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:50 smithi027 ceph-mon[97244]: pgmap v1595: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:50.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3112.434164 secs 2021-11-03T18:07:51.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:07:51.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:51 smithi115 ceph-mon[65453]: poll took 40120.669 ms: 1 events 2021-11-03T18:07:51.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:07:51.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:07:51.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:51.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:51.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:51.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:51.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:07:51.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:51 smithi027 ceph-mon[97244]: poll took 40120.669 ms: 1 events 2021-11-03T18:07:51.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:07:51.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:07:51.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:51.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:51.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:51.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:07:52.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:52 smithi115 ceph-mon[65453]: pgmap v1596: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:52.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:52 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:07:52.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:52 smithi027 ceph-mon[97244]: pgmap v1596: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:52.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:52 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:07:54.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:54 smithi115 ceph-mon[65453]: pgmap v1597: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:54.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:54 smithi027 ceph-mon[97244]: pgmap v1597: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:55.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3117.434275 secs 2021-11-03T18:07:55.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3117.434275 secs 2021-11-03T18:07:56.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:56 smithi115 ceph-mon[65453]: pgmap v1598: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:56.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:07:56.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:07:56.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:56 smithi027 ceph-mon[97244]: pgmap v1598: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:56.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:07:56.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:07:58.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:07:58 smithi115 ceph-mon[65453]: pgmap v1599: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:07:58.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:07:58 smithi027 ceph-mon[97244]: pgmap v1599: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:00.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:00 smithi115 ceph-mon[65453]: pgmap v1600: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:00.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3122.434388 secs 2021-11-03T18:08:00.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:00 smithi027 ceph-mon[97244]: pgmap v1600: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:00.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3122.434388 secs 2021-11-03T18:08:01.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:08:01.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:08:01.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:08:01.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:01.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:01.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:01.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:01.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:08:01.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:08:01.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:08:01.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:01.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:01.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:01.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:02.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:02 smithi115 ceph-mon[65453]: pgmap v1601: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:02.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:02 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:08:02.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:02 smithi027 ceph-mon[97244]: pgmap v1601: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:02.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:02 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:08:02.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:02.820+0000 7ff145bda700 1 -- 172.21.15.27:0/246836494 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1400f6ff0 msgr2=0x7ff1400f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:08:02.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:02.820+0000 7ff145bda700 1 --2- 172.21.15.27:0/246836494 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1400f6ff0 0x7ff1400f7a20 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 rx=0x7ff134004660 tx=0x7ff134009b30).stop 2021-11-03T18:08:02.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:02.820+0000 7ff145bda700 1 -- 172.21.15.27:0/246836494 shutdown_connections 2021-11-03T18:08:02.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:02.820+0000 7ff145bda700 1 --2- 172.21.15.27:0/246836494 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1400f6ff0 0x7ff1400f7a20 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:02.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:02.820+0000 7ff145bda700 1 --2- 172.21.15.27:0/246836494 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1400f66e0 0x7ff1400f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:02.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:02.820+0000 7ff145bda700 1 -- 172.21.15.27:0/246836494 >> 172.21.15.27:0/246836494 conn(0x7ff1400f1f00 msgr2=0x7ff1400f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:08:02.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:02.820+0000 7ff145bda700 1 -- 172.21.15.27:0/246836494 shutdown_connections 2021-11-03T18:08:02.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:02.820+0000 7ff145bda700 1 -- 172.21.15.27:0/246836494 wait complete. 2021-11-03T18:08:02.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:02.821+0000 7ff145bda700 1 Processor -- start 2021-11-03T18:08:02.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:02.821+0000 7ff145bda700 1 -- start start 2021-11-03T18:08:02.823 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:02.821+0000 7ff145bda700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1400f66e0 0x7ff14006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:08:02.823 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:02.821+0000 7ff145bda700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1400f6ff0 0x7ff140069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:08:02.823 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:02.821+0000 7ff145bda700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14006a380 con 0x7ff1400f6ff0 2021-11-03T18:08:02.823 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:02.821+0000 7ff145bda700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14006a4c0 con 0x7ff1400f66e0 2021-11-03T18:08:02.824 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:02.821+0000 7ff13effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1400f6ff0 0x7ff140069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:08:02.824 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:02.822+0000 7ff13effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1400f6ff0 0x7ff140069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33108/0 (socket says 172.21.15.27:33108) 2021-11-03T18:08:02.824 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:02.822+0000 7ff13effd700 1 -- 172.21.15.27:0/3769892949 learned_addr learned my addr 172.21.15.27:0/3769892949 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:08:02.825 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:02.822+0000 7ff13f7fe700 1 --2- 172.21.15.27:0/3769892949 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1400f66e0 0x7ff14006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:08:02.825 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:02.822+0000 7ff13effd700 1 -- 172.21.15.27:0/3769892949 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1400f66e0 msgr2=0x7ff14006b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:08:02.825 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:02.822+0000 7ff13effd700 1 --2- 172.21.15.27:0/3769892949 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1400f66e0 0x7ff14006b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:02.825 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:02.822+0000 7ff13effd700 1 -- 172.21.15.27:0/3769892949 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff134005040 con 0x7ff1400f6ff0 2021-11-03T18:08:02.826 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:02.822+0000 7ff13effd700 1 --2- 172.21.15.27:0/3769892949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1400f6ff0 0x7ff140069e20 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 rx=0x7ff134000c00 tx=0x7ff13400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:08:02.826 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:02.822+0000 7ff127fff700 1 -- 172.21.15.27:0/3769892949 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff134019070 con 0x7ff1400f6ff0 2021-11-03T18:08:02.826 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:02.822+0000 7ff145bda700 1 -- 172.21.15.27:0/3769892949 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff14006a740 con 0x7ff1400f6ff0 2021-11-03T18:08:02.826 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:02.822+0000 7ff127fff700 1 -- 172.21.15.27:0/3769892949 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7ff134007e70 con 0x7ff1400f6ff0 2021-11-03T18:08:02.827 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:02.822+0000 7ff127fff700 1 -- 172.21.15.27:0/3769892949 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff134014ce0 con 0x7ff1400f6ff0 2021-11-03T18:08:02.827 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:02.822+0000 7ff145bda700 1 -- 172.21.15.27:0/3769892949 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff14006aca0 con 0x7ff1400f6ff0 2021-11-03T18:08:02.827 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:02.824+0000 7ff145bda700 1 -- 172.21.15.27:0/3769892949 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff140045ad0 con 0x7ff1400f6ff0 2021-11-03T18:08:02.827 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:02.824+0000 7ff127fff700 1 -- 172.21.15.27:0/3769892949 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7ff134025070 con 0x7ff1400f6ff0 2021-11-03T18:08:02.827 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:02.824+0000 7ff127fff700 1 --2- 172.21.15.27:0/3769892949 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff1280640b0 0x7ff128066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:08:02.828 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:02.824+0000 7ff127fff700 1 -- 172.21.15.27:0/3769892949 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7ff1340817d0 con 0x7ff1400f6ff0 2021-11-03T18:08:02.828 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:02.824+0000 7ff13f7fe700 1 --2- 172.21.15.27:0/3769892949 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff1280640b0 0x7ff128066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:08:02.828 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:02.825+0000 7ff13f7fe700 1 --2- 172.21.15.27:0/3769892949 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff1280640b0 0x7ff128066570 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7ff14006b730 tx=0x7ff13000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:08:02.829 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:02.828+0000 7ff127fff700 1 -- 172.21.15.27:0/3769892949 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7ff134053ad0 con 0x7ff1400f6ff0 2021-11-03T18:08:03.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.007+0000 7ff145bda700 1 -- 172.21.15.27:0/3769892949 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1400ff040 con 0x7ff1280640b0 2021-11-03T18:08:03.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.008+0000 7ff127fff700 1 -- 172.21.15.27:0/3769892949 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7ff1400ff040 con 0x7ff1280640b0 2021-11-03T18:08:03.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.010+0000 7ff145bda700 1 -- 172.21.15.27:0/3769892949 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff1280640b0 msgr2=0x7ff128066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:08:03.009 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.010+0000 7ff145bda700 1 --2- 172.21.15.27:0/3769892949 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff1280640b0 0x7ff128066570 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7ff14006b730 tx=0x7ff13000b040).stop 2021-11-03T18:08:03.009 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.010+0000 7ff145bda700 1 -- 172.21.15.27:0/3769892949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1400f6ff0 msgr2=0x7ff140069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:08:03.009 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.010+0000 7ff145bda700 1 --2- 172.21.15.27:0/3769892949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1400f6ff0 0x7ff140069e20 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 rx=0x7ff134000c00 tx=0x7ff13400b040).stop 2021-11-03T18:08:03.010 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.010+0000 7ff145bda700 1 -- 172.21.15.27:0/3769892949 shutdown_connections 2021-11-03T18:08:03.010 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.010+0000 7ff145bda700 1 --2- 172.21.15.27:0/3769892949 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff1280640b0 0x7ff128066570 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:03.010 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.010+0000 7ff145bda700 1 --2- 172.21.15.27:0/3769892949 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1400f66e0 0x7ff14006b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:03.010 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.010+0000 7ff145bda700 1 --2- 172.21.15.27:0/3769892949 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1400f6ff0 0x7ff140069e20 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:03.011 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.010+0000 7ff145bda700 1 -- 172.21.15.27:0/3769892949 >> 172.21.15.27:0/3769892949 conn(0x7ff1400f1f00 msgr2=0x7ff1400fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:08:03.011 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.010+0000 7ff145bda700 1 -- 172.21.15.27:0/3769892949 shutdown_connections 2021-11-03T18:08:03.011 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.010+0000 7ff145bda700 1 -- 172.21.15.27:0/3769892949 wait complete. 2021-11-03T18:08:03.017 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:08:03.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.110+0000 7f8d1a48f700 1 -- 172.21.15.27:0/3374536822 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d140f55c0 msgr2=0x7f8d140f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:08:03.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.110+0000 7f8d1a48f700 1 --2- 172.21.15.27:0/3374536822 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d140f55c0 0x7f8d140f59e0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 rx=0x7f8d04004660 tx=0x7f8d04009b30).stop 2021-11-03T18:08:03.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.110+0000 7f8d1a48f700 1 -- 172.21.15.27:0/3374536822 shutdown_connections 2021-11-03T18:08:03.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.110+0000 7f8d1a48f700 1 --2- 172.21.15.27:0/3374536822 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d140f6860 0x7f8d140f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:03.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.110+0000 7f8d1a48f700 1 --2- 172.21.15.27:0/3374536822 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d140f55c0 0x7f8d140f59e0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:03.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.110+0000 7f8d1a48f700 1 -- 172.21.15.27:0/3374536822 >> 172.21.15.27:0/3374536822 conn(0x7f8d140f0d40 msgr2=0x7f8d140f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:08:03.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.110+0000 7f8d1a48f700 1 -- 172.21.15.27:0/3374536822 shutdown_connections 2021-11-03T18:08:03.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.110+0000 7f8d1a48f700 1 -- 172.21.15.27:0/3374536822 wait complete. 2021-11-03T18:08:03.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.111+0000 7f8d1a48f700 1 Processor -- start 2021-11-03T18:08:03.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.111+0000 7f8d1a48f700 1 -- start start 2021-11-03T18:08:03.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.111+0000 7f8d1a48f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d140f55c0 0x7f8d14109c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:08:03.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.111+0000 7f8d1a48f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d140f6860 0x7f8d1410c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:08:03.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.111+0000 7f8d1a48f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d140603e0 con 0x7f8d140f6860 2021-11-03T18:08:03.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.111+0000 7f8d1a48f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d14060520 con 0x7f8d140f55c0 2021-11-03T18:08:03.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.111+0000 7f8d137fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d140f6860 0x7f8d1410c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:08:03.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.111+0000 7f8d137fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d140f6860 0x7f8d1410c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33118/0 (socket says 172.21.15.27:33118) 2021-11-03T18:08:03.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.111+0000 7f8d137fe700 1 -- 172.21.15.27:0/193832457 learned_addr learned my addr 172.21.15.27:0/193832457 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:08:03.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.111+0000 7f8d13fff700 1 --2- 172.21.15.27:0/193832457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d140f55c0 0x7f8d14109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:08:03.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.111+0000 7f8d137fe700 1 -- 172.21.15.27:0/193832457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d140f55c0 msgr2=0x7f8d14109c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:08:03.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.111+0000 7f8d137fe700 1 --2- 172.21.15.27:0/193832457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d140f55c0 0x7f8d14109c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:03.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.111+0000 7f8d137fe700 1 -- 172.21.15.27:0/193832457 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d04005040 con 0x7f8d140f6860 2021-11-03T18:08:03.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.112+0000 7f8d137fe700 1 --2- 172.21.15.27:0/193832457 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d140f6860 0x7f8d1410c170 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 rx=0x7f8d080026d0 tx=0x7f8d08009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:08:03.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.112+0000 7f8d10ff9700 1 -- 172.21.15.27:0/193832457 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d08009320 con 0x7f8d140f6860 2021-11-03T18:08:03.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.112+0000 7f8d10ff9700 1 -- 172.21.15.27:0/193832457 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f8d0800f040 con 0x7f8d140f6860 2021-11-03T18:08:03.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.112+0000 7f8d1a48f700 1 -- 172.21.15.27:0/193832457 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d1410c720 con 0x7f8d140f6860 2021-11-03T18:08:03.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.112+0000 7f8d10ff9700 1 -- 172.21.15.27:0/193832457 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d080147e0 con 0x7f8d140f6860 2021-11-03T18:08:03.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.112+0000 7f8d1a48f700 1 -- 172.21.15.27:0/193832457 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d1410cce0 con 0x7f8d140f6860 2021-11-03T18:08:03.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.113+0000 7f8d1a48f700 1 -- 172.21.15.27:0/193832457 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d14045ad0 con 0x7f8d140f6860 2021-11-03T18:08:03.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.113+0000 7f8d10ff9700 1 -- 172.21.15.27:0/193832457 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f8d08009480 con 0x7f8d140f6860 2021-11-03T18:08:03.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.114+0000 7f8d10ff9700 1 --2- 172.21.15.27:0/193832457 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8cfc0640b0 0x7f8cfc066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:08:03.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.114+0000 7f8d10ff9700 1 -- 172.21.15.27:0/193832457 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f8d08080f10 con 0x7f8d140f6860 2021-11-03T18:08:03.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.114+0000 7f8d13fff700 1 --2- 172.21.15.27:0/193832457 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8cfc0640b0 0x7f8cfc066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:08:03.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.114+0000 7f8d13fff700 1 --2- 172.21.15.27:0/193832457 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8cfc0640b0 0x7f8cfc066570 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7f8d04009da0 tx=0x7f8d04009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:08:03.121 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.118+0000 7f8d10ff9700 1 -- 172.21.15.27:0/193832457 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f8d08053210 con 0x7f8d140f6860 2021-11-03T18:08:03.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.290+0000 7f8d1a48f700 1 -- 172.21.15.27:0/193832457 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8d140fdf00 con 0x7f8cfc0640b0 2021-11-03T18:08:03.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.304+0000 7f8d10ff9700 1 -- 172.21.15.27:0/193832457 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f8d140fdf00 con 0x7f8cfc0640b0 2021-11-03T18:08:03.302 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:08:03.302 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (59m) 9m ago 60m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:08:03.302 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (51m) 9m ago 60m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:08:03.303 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (51m) 8m ago 60m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:08:03.303 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (59m) 9m ago 60m 41.8M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:08:03.303 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (53m) 9m ago 55m 26.0M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:08:03.303 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (56m) 9m ago 56m 3845M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:08:03.303 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (51m) 8m ago 56m 25.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:08:03.304 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (53m) 8m ago 56m 26.8M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:08:03.304 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (53m) 9m ago 62m 554M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:08:03.304 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (52m) 8m ago 60m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:08:03.304 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (52m) 9m ago 62m 235M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:08:03.305 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (51m) 8m ago 59m 112M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:08:03.305 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (60m) 9m ago 60m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:08:03.305 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (59m) 8m ago 59m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:08:03.305 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (50m) 9m ago 59m 351M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:08:03.305 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (50m) 9m ago 58m 383M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:08:03.306 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (49m) 9m ago 58m 284M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:08:03.307 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (49m) 8m ago 58m 404M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:08:03.307 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (49m) 8m ago 57m 322M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:08:03.308 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (48m) 8m ago 57m 327M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:08:03.309 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (59m) 9m ago 60m 55.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:08:03.310 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.306+0000 7f8d1a48f700 1 -- 172.21.15.27:0/193832457 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8cfc0640b0 msgr2=0x7f8cfc066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:08:03.311 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.306+0000 7f8d1a48f700 1 --2- 172.21.15.27:0/193832457 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8cfc0640b0 0x7f8cfc066570 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7f8d04009da0 tx=0x7f8d04009b30).stop 2021-11-03T18:08:03.312 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.306+0000 7f8d1a48f700 1 -- 172.21.15.27:0/193832457 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d140f6860 msgr2=0x7f8d1410c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:08:03.313 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.306+0000 7f8d1a48f700 1 --2- 172.21.15.27:0/193832457 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d140f6860 0x7f8d1410c170 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 rx=0x7f8d080026d0 tx=0x7f8d08009de0).stop 2021-11-03T18:08:03.313 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.306+0000 7f8d1a48f700 1 -- 172.21.15.27:0/193832457 shutdown_connections 2021-11-03T18:08:03.314 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.306+0000 7f8d1a48f700 1 --2- 172.21.15.27:0/193832457 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8cfc0640b0 0x7f8cfc066570 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:03.314 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.306+0000 7f8d1a48f700 1 --2- 172.21.15.27:0/193832457 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d140f55c0 0x7f8d14109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:03.314 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.306+0000 7f8d1a48f700 1 --2- 172.21.15.27:0/193832457 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d140f6860 0x7f8d1410c170 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:03.315 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.306+0000 7f8d1a48f700 1 -- 172.21.15.27:0/193832457 >> 172.21.15.27:0/193832457 conn(0x7f8d140f0d40 msgr2=0x7f8d140fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:08:03.315 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.306+0000 7f8d1a48f700 1 -- 172.21.15.27:0/193832457 shutdown_connections 2021-11-03T18:08:03.316 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.306+0000 7f8d1a48f700 1 -- 172.21.15.27:0/193832457 wait complete. 2021-11-03T18:08:03.406 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.405+0000 7f1399d88700 1 -- 172.21.15.27:0/9180647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13940f5ed0 msgr2=0x7f13940f6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:08:03.407 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.405+0000 7f1399d88700 1 --2- 172.21.15.27:0/9180647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13940f5ed0 0x7f13940f6900 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7f1388004660 tx=0x7f1388009b30).stop 2021-11-03T18:08:03.407 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.405+0000 7f1399d88700 1 -- 172.21.15.27:0/9180647 shutdown_connections 2021-11-03T18:08:03.408 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.405+0000 7f1399d88700 1 --2- 172.21.15.27:0/9180647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13940f5ed0 0x7f13940f6900 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:03.408 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.405+0000 7f1399d88700 1 --2- 172.21.15.27:0/9180647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13940f55c0 0x7f13940f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:03.409 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.405+0000 7f1399d88700 1 -- 172.21.15.27:0/9180647 >> 172.21.15.27:0/9180647 conn(0x7f13940f0de0 msgr2=0x7f13940f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:08:03.409 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.405+0000 7f1399d88700 1 -- 172.21.15.27:0/9180647 shutdown_connections 2021-11-03T18:08:03.409 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.405+0000 7f1399d88700 1 -- 172.21.15.27:0/9180647 wait complete. 2021-11-03T18:08:03.409 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.406+0000 7f1399d88700 1 Processor -- start 2021-11-03T18:08:03.410 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.406+0000 7f1399d88700 1 -- start start 2021-11-03T18:08:03.410 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.406+0000 7f1399d88700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13940f55c0 0x7f1394105e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:08:03.410 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.406+0000 7f1399d88700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13940f5ed0 0x7f1394106330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:08:03.410 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.406+0000 7f1399d88700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1394106930 con 0x7f13940f55c0 2021-11-03T18:08:03.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.406+0000 7f1399d88700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1394107140 con 0x7f13940f5ed0 2021-11-03T18:08:03.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.406+0000 7f13937fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13940f55c0 0x7f1394105e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:08:03.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.406+0000 7f13937fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13940f55c0 0x7f1394105e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33128/0 (socket says 172.21.15.27:33128) 2021-11-03T18:08:03.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.406+0000 7f13937fe700 1 -- 172.21.15.27:0/2005633699 learned_addr learned my addr 172.21.15.27:0/2005633699 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:08:03.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.406+0000 7f1392ffd700 1 --2- 172.21.15.27:0/2005633699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13940f5ed0 0x7f1394106330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:08:03.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.407+0000 7f13937fe700 1 -- 172.21.15.27:0/2005633699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13940f5ed0 msgr2=0x7f1394106330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:08:03.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.407+0000 7f13937fe700 1 --2- 172.21.15.27:0/2005633699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13940f5ed0 0x7f1394106330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:03.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.407+0000 7f13937fe700 1 -- 172.21.15.27:0/2005633699 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1388005040 con 0x7f13940f55c0 2021-11-03T18:08:03.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.407+0000 7f13937fe700 1 --2- 172.21.15.27:0/2005633699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13940f55c0 0x7f1394105e40 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0x7f1384010a00 tx=0x7f138400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:08:03.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.407+0000 7f137bfff700 1 -- 172.21.15.27:0/2005633699 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f13840094c0 con 0x7f13940f55c0 2021-11-03T18:08:03.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.407+0000 7f137bfff700 1 -- 172.21.15.27:0/2005633699 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f1384010e90 con 0x7f13940f55c0 2021-11-03T18:08:03.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.407+0000 7f1399d88700 1 -- 172.21.15.27:0/2005633699 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1394107360 con 0x7f13940f55c0 2021-11-03T18:08:03.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.407+0000 7f137bfff700 1 -- 172.21.15.27:0/2005633699 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1384007db0 con 0x7f13940f55c0 2021-11-03T18:08:03.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.407+0000 7f1399d88700 1 -- 172.21.15.27:0/2005633699 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13941078f0 con 0x7f13940f55c0 2021-11-03T18:08:03.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.408+0000 7f1399d88700 1 -- 172.21.15.27:0/2005633699 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13940f4a50 con 0x7f13940f55c0 2021-11-03T18:08:03.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.408+0000 7f137bfff700 1 -- 172.21.15.27:0/2005633699 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f1384022070 con 0x7f13940f55c0 2021-11-03T18:08:03.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.409+0000 7f137bfff700 1 --2- 172.21.15.27:0/2005633699 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f137c0640b0 0x7f137c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:08:03.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.409+0000 7f137bfff700 1 -- 172.21.15.27:0/2005633699 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f13840831e0 con 0x7f13940f55c0 2021-11-03T18:08:03.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.409+0000 7f1392ffd700 1 --2- 172.21.15.27:0/2005633699 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f137c0640b0 0x7f137c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:08:03.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.409+0000 7f1392ffd700 1 --2- 172.21.15.27:0/2005633699 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f137c0640b0 0x7f137c066570 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f1388005010 tx=0x7f1388016040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:08:03.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.413+0000 7f137bfff700 1 -- 172.21.15.27:0/2005633699 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f13840553f0 con 0x7f13940f55c0 2021-11-03T18:08:03.642 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.645+0000 7f1399d88700 1 -- 172.21.15.27:0/2005633699 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1394045b50 con 0x7f13940f55c0 2021-11-03T18:08:03.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.646+0000 7f137bfff700 1 -- 172.21.15.27:0/2005633699 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f1394045b50 con 0x7f13940f55c0 2021-11-03T18:08:03.643 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:08:03.644 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:08:03.645 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:08:03.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:08:03.645 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:08:03.645 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:08:03.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:08:03.646 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:08:03.646 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:08:03.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:08:03.648 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:08:03.648 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:08:03.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:08:03.649 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:08:03.649 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:08:03.649 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:08:03.649 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:08:03.650 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:08:03.651 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.647+0000 7f1399d88700 1 -- 172.21.15.27:0/2005633699 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f137c0640b0 msgr2=0x7f137c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:08:03.651 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.647+0000 7f1399d88700 1 --2- 172.21.15.27:0/2005633699 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f137c0640b0 0x7f137c066570 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f1388005010 tx=0x7f1388016040).stop 2021-11-03T18:08:03.651 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.647+0000 7f1399d88700 1 -- 172.21.15.27:0/2005633699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13940f55c0 msgr2=0x7f1394105e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:08:03.651 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.647+0000 7f1399d88700 1 --2- 172.21.15.27:0/2005633699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13940f55c0 0x7f1394105e40 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0x7f1384010a00 tx=0x7f138400b040).stop 2021-11-03T18:08:03.652 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.647+0000 7f1399d88700 1 -- 172.21.15.27:0/2005633699 shutdown_connections 2021-11-03T18:08:03.652 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.647+0000 7f1399d88700 1 --2- 172.21.15.27:0/2005633699 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f137c0640b0 0x7f137c066570 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:03.652 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.647+0000 7f1399d88700 1 --2- 172.21.15.27:0/2005633699 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f13940f55c0 0x7f1394105e40 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:03.652 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.647+0000 7f1399d88700 1 --2- 172.21.15.27:0/2005633699 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f13940f5ed0 0x7f1394106330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:03.653 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.647+0000 7f1399d88700 1 -- 172.21.15.27:0/2005633699 >> 172.21.15.27:0/2005633699 conn(0x7f13940f0de0 msgr2=0x7f13940fa350 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:08:03.653 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.647+0000 7f1399d88700 1 -- 172.21.15.27:0/2005633699 shutdown_connections 2021-11-03T18:08:03.653 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.647+0000 7f1399d88700 1 -- 172.21.15.27:0/2005633699 wait complete. 2021-11-03T18:08:03.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.748+0000 7f79fa594700 1 -- 172.21.15.27:0/3860779636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79f40f48a0 msgr2=0x7f79f40f4cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:08:03.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.748+0000 7f79fa594700 1 --2- 172.21.15.27:0/3860779636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79f40f48a0 0x7f79f40f4cc0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 rx=0x7f79ec0099e0 tx=0x7f79ec004d10).stop 2021-11-03T18:08:03.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.748+0000 7f79fa594700 1 -- 172.21.15.27:0/3860779636 shutdown_connections 2021-11-03T18:08:03.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.748+0000 7f79fa594700 1 --2- 172.21.15.27:0/3860779636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79f40f5240 0x7f79f40f7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:03.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.748+0000 7f79fa594700 1 --2- 172.21.15.27:0/3860779636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79f40f48a0 0x7f79f40f4cc0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:03.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.748+0000 7f79fa594700 1 -- 172.21.15.27:0/3860779636 >> 172.21.15.27:0/3860779636 conn(0x7f79f40f0320 msgr2=0x7f79f40f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:08:03.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.748+0000 7f79fa594700 1 -- 172.21.15.27:0/3860779636 shutdown_connections 2021-11-03T18:08:03.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.748+0000 7f79fa594700 1 -- 172.21.15.27:0/3860779636 wait complete. 2021-11-03T18:08:03.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.749+0000 7f79fa594700 1 Processor -- start 2021-11-03T18:08:03.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.749+0000 7f79fa594700 1 -- start start 2021-11-03T18:08:03.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.749+0000 7f79fa594700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79f40f48a0 0x7f79f40f7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:08:03.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.749+0000 7f79fa594700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79f40f5240 0x7f79f40f5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:08:03.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.749+0000 7f79fa594700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f40f5e70 con 0x7f79f40f48a0 2021-11-03T18:08:03.754 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.749+0000 7f79fa594700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f40f5fb0 con 0x7f79f40f5240 2021-11-03T18:08:03.754 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.749+0000 7f79f9592700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79f40f48a0 0x7f79f40f7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:08:03.755 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.749+0000 7f79f9592700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79f40f48a0 0x7f79f40f7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33138/0 (socket says 172.21.15.27:33138) 2021-11-03T18:08:03.755 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.749+0000 7f79f9592700 1 -- 172.21.15.27:0/1110221428 learned_addr learned my addr 172.21.15.27:0/1110221428 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:08:03.756 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.749+0000 7f79f8d91700 1 --2- 172.21.15.27:0/1110221428 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79f40f5240 0x7f79f40f5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:08:03.756 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.749+0000 7f79f9592700 1 -- 172.21.15.27:0/1110221428 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79f40f5240 msgr2=0x7f79f40f5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:08:03.756 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.749+0000 7f79f9592700 1 --2- 172.21.15.27:0/1110221428 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79f40f5240 0x7f79f40f5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:03.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.750+0000 7f79f9592700 1 -- 172.21.15.27:0/1110221428 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ec009640 con 0x7f79f40f48a0 2021-11-03T18:08:03.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.750+0000 7f79f9592700 1 --2- 172.21.15.27:0/1110221428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79f40f48a0 0x7f79f40f7490 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 rx=0x7f79ec00b3b0 tx=0x7f79ec004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:08:03.759 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.750+0000 7f79f1ffb700 1 -- 172.21.15.27:0/1110221428 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79ec00e030 con 0x7f79f40f48a0 2021-11-03T18:08:03.759 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.750+0000 7f79f1ffb700 1 -- 172.21.15.27:0/1110221428 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f79ec0049b0 con 0x7f79f40f48a0 2021-11-03T18:08:03.760 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.750+0000 7f79fa594700 1 -- 172.21.15.27:0/1110221428 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f40f6230 con 0x7f79f40f48a0 2021-11-03T18:08:03.761 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.750+0000 7f79f1ffb700 1 -- 172.21.15.27:0/1110221428 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79ec018070 con 0x7f79f40f48a0 2021-11-03T18:08:03.761 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.750+0000 7f79fa594700 1 -- 172.21.15.27:0/1110221428 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f40f6710 con 0x7f79f40f48a0 2021-11-03T18:08:03.762 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.751+0000 7f79fa594700 1 -- 172.21.15.27:0/1110221428 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79f4045ad0 con 0x7f79f40f48a0 2021-11-03T18:08:03.765 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.751+0000 7f79f1ffb700 1 -- 172.21.15.27:0/1110221428 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f79ec007c90 con 0x7f79f40f48a0 2021-11-03T18:08:03.765 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.752+0000 7f79f1ffb700 1 --2- 172.21.15.27:0/1110221428 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f79dc064100 0x7f79dc0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:08:03.765 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.752+0000 7f79f1ffb700 1 -- 172.21.15.27:0/1110221428 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f79ec0824e0 con 0x7f79f40f48a0 2021-11-03T18:08:03.766 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.752+0000 7f79f8d91700 1 --2- 172.21.15.27:0/1110221428 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f79dc064100 0x7f79dc0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:08:03.766 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.752+0000 7f79f8d91700 1 --2- 172.21.15.27:0/1110221428 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f79dc064100 0x7f79dc0665c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f79f40f56c0 tx=0x7f79e800b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:08:03.767 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.756+0000 7f79f1ffb700 1 -- 172.21.15.27:0/1110221428 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f79ec0547e0 con 0x7f79f40f48a0 2021-11-03T18:08:03.958 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.961+0000 7f79fa594700 1 -- 172.21.15.27:0/1110221428 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f79f405f400 con 0x7f79f40f48a0 2021-11-03T18:08:03.959 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.961+0000 7f79f1ffb700 1 -- 172.21.15.27:0/1110221428 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f79ec0543d0 con 0x7f79f40f48a0 2021-11-03T18:08:03.959 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:08:03.960 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:08:03.960 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:08:03.960 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:08:03.960 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:08:03.961 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:08:03.961 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:08:03.961 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:08:03.961 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:08:03.962 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:08:03.962 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:08:03.962 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:08:03.963 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:08:03.963 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:08:03.963 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:08:03.963 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:08:03.964 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:08:03.964 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:08:03.964 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:08:03.964 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:08:03.965 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:08:03.965 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:08:03.965 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:08:03.965 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:08:03.965 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:08:03.966 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:08:03.966 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:08:03.967 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:08:03.967 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:08:03.967 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:08:03.968 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:08:03.968 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:08:03.969 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:08:03.969 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:08:03.970 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:08:03.970 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:08:03.971 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:08:03.971 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:08:03.972 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:08:03.973 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.963+0000 7f79fa594700 1 -- 172.21.15.27:0/1110221428 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f79dc064100 msgr2=0x7f79dc0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:08:03.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.963+0000 7f79fa594700 1 --2- 172.21.15.27:0/1110221428 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f79dc064100 0x7f79dc0665c0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f79f40f56c0 tx=0x7f79e800b040).stop 2021-11-03T18:08:03.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.963+0000 7f79fa594700 1 -- 172.21.15.27:0/1110221428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79f40f48a0 msgr2=0x7f79f40f7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:08:03.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.963+0000 7f79fa594700 1 --2- 172.21.15.27:0/1110221428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79f40f48a0 0x7f79f40f7490 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 rx=0x7f79ec00b3b0 tx=0x7f79ec004d10).stop 2021-11-03T18:08:03.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.963+0000 7f79fa594700 1 -- 172.21.15.27:0/1110221428 shutdown_connections 2021-11-03T18:08:03.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.963+0000 7f79fa594700 1 --2- 172.21.15.27:0/1110221428 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f79dc064100 0x7f79dc0665c0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:03.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.963+0000 7f79fa594700 1 --2- 172.21.15.27:0/1110221428 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f79f40f48a0 0x7f79f40f7490 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:03.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.963+0000 7f79fa594700 1 --2- 172.21.15.27:0/1110221428 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f79f40f5240 0x7f79f40f5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:03.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.963+0000 7f79fa594700 1 -- 172.21.15.27:0/1110221428 >> 172.21.15.27:0/1110221428 conn(0x7f79f40f0320 msgr2=0x7f79f40fb5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:08:03.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.963+0000 7f79fa594700 1 -- 172.21.15.27:0/1110221428 shutdown_connections 2021-11-03T18:08:03.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:03.963+0000 7f79fa594700 1 -- 172.21.15.27:0/1110221428 wait complete. 2021-11-03T18:08:03.976 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:08:04.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:04 smithi115 ceph-mon[65453]: pgmap v1602: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:04.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:04 smithi115 ceph-mon[65453]: from='client.35752 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:08:04.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:04 smithi115 ceph-mon[65453]: from='client.35756 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:08:04.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:04 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2005633699' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:04.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:04 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1110221428' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:08:04.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:04 smithi027 ceph-mon[97244]: pgmap v1602: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:04.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:04 smithi027 ceph-mon[97244]: from='client.35752 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:08:04.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:04 smithi027 ceph-mon[97244]: from='client.35756 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:08:04.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:04 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2005633699' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:04.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:04 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1110221428' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:08:05.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3127.434517 secs 2021-11-03T18:08:05.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3127.434517 secs 2021-11-03T18:08:06.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:06 smithi115 ceph-mon[65453]: pgmap v1603: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:06.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:06 smithi027 ceph-mon[97244]: pgmap v1603: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:08.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:08 smithi027 ceph-mon[97244]: pgmap v1604: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:08.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:08 smithi115 ceph-mon[65453]: pgmap v1604: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:10.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:10 smithi027 ceph-mon[97244]: pgmap v1605: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:10.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3132.434632 secs 2021-11-03T18:08:10.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:10 smithi115 ceph-mon[65453]: pgmap v1605: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:10.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3132.434632 secs 2021-11-03T18:08:11.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:08:11.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:08:11.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:08:11.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:11.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:11.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:11.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:11.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:08:11.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:08:11.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:08:11.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:11.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:11.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:11.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:12.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:12 smithi027 ceph-mon[97244]: pgmap v1606: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:12.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:12 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:08:12.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:12 smithi115 ceph-mon[65453]: pgmap v1606: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:12.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:12 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:08:14.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:14 smithi027 ceph-mon[97244]: pgmap v1607: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:14.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:14 smithi115 ceph-mon[65453]: pgmap v1607: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:15.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3137.434772 secs 2021-11-03T18:08:15.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3137.434772 secs 2021-11-03T18:08:16.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:16 smithi027 ceph-mon[97244]: pgmap v1608: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:16.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:16 smithi115 ceph-mon[65453]: pgmap v1608: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:18.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:18 smithi027 ceph-mon[97244]: pgmap v1609: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:18.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:18 smithi115 ceph-mon[65453]: pgmap v1609: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:19.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:19 smithi115 ceph-mon[65453]: pgmap v1610: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:20.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:19 smithi027 ceph-mon[97244]: pgmap v1610: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:20.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3142.434904 secs 2021-11-03T18:08:21.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3142.434904 secs 2021-11-03T18:08:21.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:21 smithi115 ceph-mon[65453]: pgmap v1611: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:21.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:08:21.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:08:21.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:08:21.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:21.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:21.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:21.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:21.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:21 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:08:22.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:21 smithi027 ceph-mon[97244]: pgmap v1611: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:22.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:08:22.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:08:22.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:08:22.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:22.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:22.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:22.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:22.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:21 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:08:23.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:23 smithi115 ceph-mon[65453]: pgmap v1612: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:24.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:23 smithi027 ceph-mon[97244]: pgmap v1612: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:25.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3147.435015 secs 2021-11-03T18:08:25.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:25 smithi115 ceph-mon[65453]: pgmap v1613: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:26.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3147.435015 secs 2021-11-03T18:08:26.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:25 smithi027 ceph-mon[97244]: pgmap v1613: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:27.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:27 smithi115 ceph-mon[65453]: pgmap v1614: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:28.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:27 smithi027 ceph-mon[97244]: pgmap v1614: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:29.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:29 smithi115 ceph-mon[65453]: pgmap v1615: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:30.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:29 smithi027 ceph-mon[97244]: pgmap v1615: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:30.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3152.435121 secs 2021-11-03T18:08:31.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3152.435121 secs 2021-11-03T18:08:31.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:31 smithi115 ceph-mon[65453]: pgmap v1616: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:31.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:08:31.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:31 smithi115 ceph-mon[65453]: poll took 40119.784 ms: 1 events 2021-11-03T18:08:31.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:08:31.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:08:31.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:08:31.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:31.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:31.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:31.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:31.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:31 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:08:32.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:31 smithi027 ceph-mon[97244]: pgmap v1616: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:32.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:08:32.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:31 smithi027 ceph-mon[97244]: poll took 40119.784 ms: 1 events 2021-11-03T18:08:32.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:08:32.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:08:32.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:08:32.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:32.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:32.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:32.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:32.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:31 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:08:34.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:33 smithi027 ceph-mon[97244]: pgmap v1617: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:34.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.076+0000 7f57e7124700 1 -- 172.21.15.27:0/3494677359 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57e00f66e0 msgr2=0x7f57e00f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:08:34.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.076+0000 7f57e7124700 1 --2- 172.21.15.27:0/3494677359 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57e00f66e0 0x7f57e00f6b40 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 rx=0x7f57d4004660 tx=0x7f57d4009b30).stop 2021-11-03T18:08:34.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.076+0000 7f57e7124700 1 -- 172.21.15.27:0/3494677359 shutdown_connections 2021-11-03T18:08:34.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.076+0000 7f57e7124700 1 --2- 172.21.15.27:0/3494677359 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57e00f66e0 0x7f57e00f6b40 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:34.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.076+0000 7f57e7124700 1 --2- 172.21.15.27:0/3494677359 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57e00f5440 0x7f57e00f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:34.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.076+0000 7f57e7124700 1 -- 172.21.15.27:0/3494677359 >> 172.21.15.27:0/3494677359 conn(0x7f57e00f0ba0 msgr2=0x7f57e00f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:08:34.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.076+0000 7f57e7124700 1 -- 172.21.15.27:0/3494677359 shutdown_connections 2021-11-03T18:08:34.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.076+0000 7f57e7124700 1 -- 172.21.15.27:0/3494677359 wait complete. 2021-11-03T18:08:34.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.077+0000 7f57e7124700 1 Processor -- start 2021-11-03T18:08:34.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.077+0000 7f57e7124700 1 -- start start 2021-11-03T18:08:34.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.077+0000 7f57e7124700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57e00f5440 0x7f57e0106dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:08:34.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.077+0000 7f57e7124700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57e00f66e0 0x7f57e01072c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:08:34.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.077+0000 7f57e7124700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e01078c0 con 0x7f57e00f66e0 2021-11-03T18:08:34.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.077+0000 7f57e7124700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e0108100 con 0x7f57e00f5440 2021-11-03T18:08:34.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.077+0000 7f57e5921700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57e00f66e0 0x7f57e01072c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:08:34.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.077+0000 7f57e5921700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57e00f66e0 0x7f57e01072c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33158/0 (socket says 172.21.15.27:33158) 2021-11-03T18:08:34.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.077+0000 7f57e5921700 1 -- 172.21.15.27:0/1080665878 learned_addr learned my addr 172.21.15.27:0/1080665878 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:08:34.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.077+0000 7f57e6122700 1 --2- 172.21.15.27:0/1080665878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57e00f5440 0x7f57e0106dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:08:34.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.078+0000 7f57e5921700 1 -- 172.21.15.27:0/1080665878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57e00f5440 msgr2=0x7f57e0106dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:08:34.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.078+0000 7f57e5921700 1 --2- 172.21.15.27:0/1080665878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57e00f5440 0x7f57e0106dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:34.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.078+0000 7f57e5921700 1 -- 172.21.15.27:0/1080665878 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d4005040 con 0x7f57e00f66e0 2021-11-03T18:08:34.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.078+0000 7f57e5921700 1 --2- 172.21.15.27:0/1080665878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57e00f66e0 0x7f57e01072c0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 rx=0x7f57d4009990 tx=0x7f57d4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:08:34.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.078+0000 7f57deffd700 1 -- 172.21.15.27:0/1080665878 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57d4018070 con 0x7f57e00f66e0 2021-11-03T18:08:34.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.078+0000 7f57deffd700 1 -- 172.21.15.27:0/1080665878 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f57d4007ba0 con 0x7f57e00f66e0 2021-11-03T18:08:34.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.078+0000 7f57e7124700 1 -- 172.21.15.27:0/1080665878 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57e0108380 con 0x7f57e00f66e0 2021-11-03T18:08:34.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.078+0000 7f57deffd700 1 -- 172.21.15.27:0/1080665878 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57d401c3c0 con 0x7f57e00f66e0 2021-11-03T18:08:34.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.078+0000 7f57e7124700 1 -- 172.21.15.27:0/1080665878 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e0108860 con 0x7f57e00f66e0 2021-11-03T18:08:34.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.079+0000 7f57e7124700 1 -- 172.21.15.27:0/1080665878 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57e0059100 con 0x7f57e00f66e0 2021-11-03T18:08:34.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.079+0000 7f57deffd700 1 -- 172.21.15.27:0/1080665878 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f57d401d420 con 0x7f57e00f66e0 2021-11-03T18:08:34.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.080+0000 7f57deffd700 1 --2- 172.21.15.27:0/1080665878 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f57c8064100 0x7f57c80665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:08:34.086 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.080+0000 7f57deffd700 1 -- 172.21.15.27:0/1080665878 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f57d4082470 con 0x7f57e00f66e0 2021-11-03T18:08:34.086 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.080+0000 7f57e6122700 1 --2- 172.21.15.27:0/1080665878 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f57c8064100 0x7f57c80665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:08:34.086 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.080+0000 7f57e6122700 1 --2- 172.21.15.27:0/1080665878 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f57c8064100 0x7f57c80665c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f57e0107b90 tx=0x7f57d8009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:08:34.087 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.084+0000 7f57deffd700 1 -- 172.21.15.27:0/1080665878 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f57d4054710 con 0x7f57e00f66e0 2021-11-03T18:08:34.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:33 smithi115 ceph-mon[65453]: pgmap v1617: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:34.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.265+0000 7f57e7124700 1 -- 172.21.15.27:0/1080665878 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f57e005fd30 con 0x7f57c8064100 2021-11-03T18:08:34.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.266+0000 7f57deffd700 1 -- 172.21.15.27:0/1080665878 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f57e005fd30 con 0x7f57c8064100 2021-11-03T18:08:34.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.268+0000 7f57e7124700 1 -- 172.21.15.27:0/1080665878 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f57c8064100 msgr2=0x7f57c80665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:08:34.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.268+0000 7f57e7124700 1 --2- 172.21.15.27:0/1080665878 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f57c8064100 0x7f57c80665c0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f57e0107b90 tx=0x7f57d8009040).stop 2021-11-03T18:08:34.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.268+0000 7f57e7124700 1 -- 172.21.15.27:0/1080665878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57e00f66e0 msgr2=0x7f57e01072c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:08:34.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.268+0000 7f57e7124700 1 --2- 172.21.15.27:0/1080665878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57e00f66e0 0x7f57e01072c0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 rx=0x7f57d4009990 tx=0x7f57d4009b30).stop 2021-11-03T18:08:34.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.268+0000 7f57e7124700 1 -- 172.21.15.27:0/1080665878 shutdown_connections 2021-11-03T18:08:34.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.268+0000 7f57e7124700 1 --2- 172.21.15.27:0/1080665878 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f57c8064100 0x7f57c80665c0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:34.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.268+0000 7f57e7124700 1 --2- 172.21.15.27:0/1080665878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57e00f5440 0x7f57e0106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:34.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.268+0000 7f57e7124700 1 --2- 172.21.15.27:0/1080665878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57e00f66e0 0x7f57e01072c0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:34.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.268+0000 7f57e7124700 1 -- 172.21.15.27:0/1080665878 >> 172.21.15.27:0/1080665878 conn(0x7f57e00f0ba0 msgr2=0x7f57e005f620 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:08:34.270 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.269+0000 7f57e7124700 1 -- 172.21.15.27:0/1080665878 shutdown_connections 2021-11-03T18:08:34.270 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.269+0000 7f57e7124700 1 -- 172.21.15.27:0/1080665878 wait complete. 2021-11-03T18:08:34.276 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:08:34.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.368+0000 7feab6c59700 1 -- 172.21.15.27:0/808213612 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feab00f66e0 msgr2=0x7feab00f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:08:34.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.368+0000 7feab6c59700 1 --2- 172.21.15.27:0/808213612 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feab00f66e0 0x7feab00f6b40 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 rx=0x7feaa0004660 tx=0x7feaa0009b30).stop 2021-11-03T18:08:34.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.368+0000 7feab6c59700 1 -- 172.21.15.27:0/808213612 shutdown_connections 2021-11-03T18:08:34.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.368+0000 7feab6c59700 1 --2- 172.21.15.27:0/808213612 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feab00f66e0 0x7feab00f6b40 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:34.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.368+0000 7feab6c59700 1 --2- 172.21.15.27:0/808213612 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feab00f5440 0x7feab00f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:34.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.368+0000 7feab6c59700 1 -- 172.21.15.27:0/808213612 >> 172.21.15.27:0/808213612 conn(0x7feab00f0ba0 msgr2=0x7feab00f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:08:34.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.369+0000 7feab6c59700 1 -- 172.21.15.27:0/808213612 shutdown_connections 2021-11-03T18:08:34.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.369+0000 7feab6c59700 1 -- 172.21.15.27:0/808213612 wait complete. 2021-11-03T18:08:34.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.369+0000 7feab6c59700 1 Processor -- start 2021-11-03T18:08:34.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.369+0000 7feab6c59700 1 -- start start 2021-11-03T18:08:34.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.369+0000 7feab6c59700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feab00f5440 0x7feab01099d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:08:34.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.369+0000 7feab6c59700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feab00f66e0 0x7feab010bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:08:34.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.369+0000 7feab6c59700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab00603c0 con 0x7feab00f5440 2021-11-03T18:08:34.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.369+0000 7feab6c59700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feab0060500 con 0x7feab00f66e0 2021-11-03T18:08:34.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.369+0000 7feab5c57700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feab00f5440 0x7feab01099d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:08:34.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.369+0000 7feab5c57700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feab00f5440 0x7feab01099d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33166/0 (socket says 172.21.15.27:33166) 2021-11-03T18:08:34.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.369+0000 7feab5c57700 1 -- 172.21.15.27:0/400421800 learned_addr learned my addr 172.21.15.27:0/400421800 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:08:34.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.370+0000 7feab5c57700 1 -- 172.21.15.27:0/400421800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feab00f66e0 msgr2=0x7feab010bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:08:34.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.370+0000 7feab5456700 1 --2- 172.21.15.27:0/400421800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feab00f66e0 0x7feab010bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:08:34.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.370+0000 7feab5c57700 1 --2- 172.21.15.27:0/400421800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feab00f66e0 0x7feab010bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:34.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.370+0000 7feab5c57700 1 -- 172.21.15.27:0/400421800 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa0005040 con 0x7feab00f5440 2021-11-03T18:08:34.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.370+0000 7feab5456700 1 --2- 172.21.15.27:0/400421800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feab00f66e0 0x7feab010bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T18:08:34.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.370+0000 7feab5c57700 1 --2- 172.21.15.27:0/400421800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feab00f5440 0x7feab01099d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 rx=0x7feaac002700 tx=0x7feaac009e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:08:34.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.370+0000 7feaa67fc700 1 -- 172.21.15.27:0/400421800 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feaac009320 con 0x7feab00f5440 2021-11-03T18:08:34.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.370+0000 7feaa67fc700 1 -- 172.21.15.27:0/400421800 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7feaac00f040 con 0x7feab00f5440 2021-11-03T18:08:34.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.370+0000 7feab6c59700 1 -- 172.21.15.27:0/400421800 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feab010c480 con 0x7feab00f5440 2021-11-03T18:08:34.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.370+0000 7feaa67fc700 1 -- 172.21.15.27:0/400421800 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feaac0148e0 con 0x7feab00f5440 2021-11-03T18:08:34.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.370+0000 7feab6c59700 1 -- 172.21.15.27:0/400421800 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feab010ca10 con 0x7feab00f5440 2021-11-03T18:08:34.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.371+0000 7feab6c59700 1 -- 172.21.15.27:0/400421800 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feab0059100 con 0x7feab00f5440 2021-11-03T18:08:34.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.371+0000 7feaa67fc700 1 -- 172.21.15.27:0/400421800 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7feaac009480 con 0x7feab00f5440 2021-11-03T18:08:34.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.372+0000 7feaa67fc700 1 --2- 172.21.15.27:0/400421800 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fea9c064100 0x7fea9c0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:08:34.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.372+0000 7feaa67fc700 1 -- 172.21.15.27:0/400421800 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7feaac080b40 con 0x7feab00f5440 2021-11-03T18:08:34.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.372+0000 7feab5456700 1 --2- 172.21.15.27:0/400421800 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fea9c064100 0x7fea9c0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:08:34.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.372+0000 7feab5456700 1 --2- 172.21.15.27:0/400421800 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fea9c064100 0x7fea9c0665c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7feaa0009da0 tx=0x7feaa0009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:08:34.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.376+0000 7feaa67fc700 1 -- 172.21.15.27:0/400421800 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7feaac052e40 con 0x7feab00f5440 2021-11-03T18:08:34.545 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.547+0000 7feab6c59700 1 -- 172.21.15.27:0/400421800 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7feab0057f10 con 0x7fea9c064100 2021-11-03T18:08:34.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.562+0000 7feaa67fc700 1 -- 172.21.15.27:0/400421800 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7feab0057f10 con 0x7fea9c064100 2021-11-03T18:08:34.560 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:08:34.561 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (60m) 9m ago 61m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:08:34.561 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (51m) 9m ago 61m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:08:34.561 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (51m) 8m ago 60m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:08:34.561 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (59m) 9m ago 61m 41.8M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:08:34.562 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (53m) 9m ago 56m 26.0M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:08:34.562 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (56m) 9m ago 56m 3845M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:08:34.562 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (52m) 8m ago 56m 25.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:08:34.563 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (53m) 8m ago 56m 26.8M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:08:34.564 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (53m) 9m ago 62m 554M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:08:34.564 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (53m) 8m ago 60m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:08:34.564 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (52m) 9m ago 62m 235M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:08:34.564 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (52m) 8m ago 60m 112M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:08:34.565 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (61m) 9m ago 60m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:08:34.565 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (60m) 8m ago 60m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:08:34.565 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (51m) 9m ago 59m 351M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:08:34.565 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (50m) 9m ago 59m 383M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:08:34.565 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (50m) 9m ago 59m 284M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:08:34.566 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (49m) 8m ago 58m 404M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:08:34.566 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (49m) 8m ago 58m 322M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:08:34.566 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (49m) 8m ago 58m 327M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:08:34.566 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (59m) 9m ago 60m 55.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:08:34.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.563+0000 7feab6c59700 1 -- 172.21.15.27:0/400421800 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fea9c064100 msgr2=0x7fea9c0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:08:34.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.563+0000 7feab6c59700 1 --2- 172.21.15.27:0/400421800 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fea9c064100 0x7fea9c0665c0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 rx=0x7feaa0009da0 tx=0x7feaa0009b30).stop 2021-11-03T18:08:34.568 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.563+0000 7feab6c59700 1 -- 172.21.15.27:0/400421800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feab00f5440 msgr2=0x7feab01099d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:08:34.568 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.563+0000 7feab6c59700 1 --2- 172.21.15.27:0/400421800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feab00f5440 0x7feab01099d0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 rx=0x7feaac002700 tx=0x7feaac009e00).stop 2021-11-03T18:08:34.569 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.563+0000 7feab6c59700 1 -- 172.21.15.27:0/400421800 shutdown_connections 2021-11-03T18:08:34.569 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.563+0000 7feab6c59700 1 --2- 172.21.15.27:0/400421800 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fea9c064100 0x7fea9c0665c0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:34.569 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.563+0000 7feab6c59700 1 --2- 172.21.15.27:0/400421800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feab00f5440 0x7feab01099d0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:34.569 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.563+0000 7feab6c59700 1 --2- 172.21.15.27:0/400421800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feab00f66e0 0x7feab010bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:34.570 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.563+0000 7feab6c59700 1 -- 172.21.15.27:0/400421800 >> 172.21.15.27:0/400421800 conn(0x7feab00f0ba0 msgr2=0x7feab00fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:08:34.570 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.564+0000 7feab6c59700 1 -- 172.21.15.27:0/400421800 shutdown_connections 2021-11-03T18:08:34.570 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.564+0000 7feab6c59700 1 -- 172.21.15.27:0/400421800 wait complete. 2021-11-03T18:08:34.662 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.662+0000 7f8f2c59e700 1 -- 172.21.15.27:0/701561046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f240f6860 msgr2=0x7f8f240f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:08:34.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.663+0000 7f8f2c59e700 1 --2- 172.21.15.27:0/701561046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f240f6860 0x7f8f240f6cc0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 rx=0x7f8f14004660 tx=0x7f8f14009b30).stop 2021-11-03T18:08:34.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.663+0000 7f8f2c59e700 1 -- 172.21.15.27:0/701561046 shutdown_connections 2021-11-03T18:08:34.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.663+0000 7f8f2c59e700 1 --2- 172.21.15.27:0/701561046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f240f6860 0x7f8f240f6cc0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:34.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.663+0000 7f8f2c59e700 1 --2- 172.21.15.27:0/701561046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f240f55c0 0x7f8f240f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:34.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.663+0000 7f8f2c59e700 1 -- 172.21.15.27:0/701561046 >> 172.21.15.27:0/701561046 conn(0x7f8f240f0d40 msgr2=0x7f8f240f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:08:34.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.663+0000 7f8f2c59e700 1 -- 172.21.15.27:0/701561046 shutdown_connections 2021-11-03T18:08:34.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.663+0000 7f8f2c59e700 1 -- 172.21.15.27:0/701561046 wait complete. 2021-11-03T18:08:34.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.663+0000 7f8f2c59e700 1 Processor -- start 2021-11-03T18:08:34.665 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.663+0000 7f8f2c59e700 1 -- start start 2021-11-03T18:08:34.665 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.663+0000 7f8f2c59e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f240f55c0 0x7f8f240f9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:08:34.665 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.663+0000 7f8f2c59e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f240f6860 0x7f8f240f83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:08:34.665 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.663+0000 7f8f2c59e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f240f8920 con 0x7f8f240f6860 2021-11-03T18:08:34.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.663+0000 7f8f2c59e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f240f8a60 con 0x7f8f240f55c0 2021-11-03T18:08:34.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.663+0000 7f8f29b39700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f240f6860 0x7f8f240f83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:08:34.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.664+0000 7f8f29b39700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f240f6860 0x7f8f240f83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33178/0 (socket says 172.21.15.27:33178) 2021-11-03T18:08:34.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.664+0000 7f8f29b39700 1 -- 172.21.15.27:0/2765695440 learned_addr learned my addr 172.21.15.27:0/2765695440 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:08:34.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.664+0000 7f8f2a33a700 1 --2- 172.21.15.27:0/2765695440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f240f55c0 0x7f8f240f9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:08:34.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.664+0000 7f8f29b39700 1 -- 172.21.15.27:0/2765695440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f240f55c0 msgr2=0x7f8f240f9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:08:34.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.664+0000 7f8f29b39700 1 --2- 172.21.15.27:0/2765695440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f240f55c0 0x7f8f240f9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:34.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.664+0000 7f8f29b39700 1 -- 172.21.15.27:0/2765695440 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f14005040 con 0x7f8f240f6860 2021-11-03T18:08:34.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.664+0000 7f8f2a33a700 1 --2- 172.21.15.27:0/2765695440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f240f55c0 0x7f8f240f9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:08:34.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.664+0000 7f8f29b39700 1 --2- 172.21.15.27:0/2765695440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f240f6860 0x7f8f240f83c0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 rx=0x7f8f14009990 tx=0x7f8f14009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:08:34.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.664+0000 7f8f1affd700 1 -- 172.21.15.27:0/2765695440 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f14018070 con 0x7f8f240f6860 2021-11-03T18:08:34.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.664+0000 7f8f1affd700 1 -- 172.21.15.27:0/2765695440 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f8f14007ba0 con 0x7f8f240f6860 2021-11-03T18:08:34.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.664+0000 7f8f2c59e700 1 -- 172.21.15.27:0/2765695440 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f240f8ce0 con 0x7f8f240f6860 2021-11-03T18:08:34.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.664+0000 7f8f1affd700 1 -- 172.21.15.27:0/2765695440 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f1401c3c0 con 0x7f8f240f6860 2021-11-03T18:08:34.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.664+0000 7f8f2c59e700 1 -- 172.21.15.27:0/2765695440 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f240f9240 con 0x7f8f240f6860 2021-11-03T18:08:34.670 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.665+0000 7f8f2c59e700 1 -- 172.21.15.27:0/2765695440 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f24045ad0 con 0x7f8f240f6860 2021-11-03T18:08:34.670 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.666+0000 7f8f1affd700 1 -- 172.21.15.27:0/2765695440 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f8f1401d420 con 0x7f8f240f6860 2021-11-03T18:08:34.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.666+0000 7f8f1affd700 1 --2- 172.21.15.27:0/2765695440 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8f100640b0 0x7f8f10066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:08:34.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.666+0000 7f8f1affd700 1 -- 172.21.15.27:0/2765695440 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f8f14082630 con 0x7f8f240f6860 2021-11-03T18:08:34.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.666+0000 7f8f2a33a700 1 --2- 172.21.15.27:0/2765695440 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8f100640b0 0x7f8f10066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:08:34.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.666+0000 7f8f2a33a700 1 --2- 172.21.15.27:0/2765695440 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8f100640b0 0x7f8f10066570 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7f8f240f9cd0 tx=0x7f8f20009c00).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:08:34.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.670+0000 7f8f1affd700 1 -- 172.21.15.27:0/2765695440 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f8f14054990 con 0x7f8f240f6860 2021-11-03T18:08:34.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.899+0000 7f8f2c59e700 1 -- 172.21.15.27:0/2765695440 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8f240fdd80 con 0x7f8f240f6860 2021-11-03T18:08:34.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.900+0000 7f8f1affd700 1 -- 172.21.15.27:0/2765695440 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f8f14054580 con 0x7f8f240f6860 2021-11-03T18:08:34.899 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:08:34.899 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:08:34.899 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:08:34.900 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:08:34.900 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:08:34.900 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:08:34.900 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:08:34.901 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:08:34.901 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:08:34.901 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:08:34.901 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:08:34.901 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:08:34.902 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:08:34.902 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:08:34.902 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:08:34.902 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:08:34.903 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:08:34.903 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:08:34.904 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.902+0000 7f8f2c59e700 1 -- 172.21.15.27:0/2765695440 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8f100640b0 msgr2=0x7f8f10066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:08:34.904 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.902+0000 7f8f2c59e700 1 --2- 172.21.15.27:0/2765695440 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8f100640b0 0x7f8f10066570 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7f8f240f9cd0 tx=0x7f8f20009c00).stop 2021-11-03T18:08:34.904 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.902+0000 7f8f2c59e700 1 -- 172.21.15.27:0/2765695440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f240f6860 msgr2=0x7f8f240f83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:08:34.905 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.902+0000 7f8f2c59e700 1 --2- 172.21.15.27:0/2765695440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f240f6860 0x7f8f240f83c0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 rx=0x7f8f14009990 tx=0x7f8f14009b30).stop 2021-11-03T18:08:34.905 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.902+0000 7f8f2c59e700 1 -- 172.21.15.27:0/2765695440 shutdown_connections 2021-11-03T18:08:34.905 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.902+0000 7f8f2c59e700 1 --2- 172.21.15.27:0/2765695440 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8f100640b0 0x7f8f10066570 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:34.905 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.902+0000 7f8f2c59e700 1 --2- 172.21.15.27:0/2765695440 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f240f55c0 0x7f8f240f9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:34.905 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.902+0000 7f8f2c59e700 1 --2- 172.21.15.27:0/2765695440 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f240f6860 0x7f8f240f83c0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:34.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.902+0000 7f8f2c59e700 1 -- 172.21.15.27:0/2765695440 >> 172.21.15.27:0/2765695440 conn(0x7f8f240f0d40 msgr2=0x7f8f2405f740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:08:34.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.902+0000 7f8f2c59e700 1 -- 172.21.15.27:0/2765695440 shutdown_connections 2021-11-03T18:08:34.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:34.902+0000 7f8f2c59e700 1 -- 172.21.15.27:0/2765695440 wait complete. 2021-11-03T18:08:35.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.002+0000 7fcbdcea0700 1 -- 172.21.15.27:0/2361000019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbd80f55c0 msgr2=0x7fcbd80f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:08:35.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.002+0000 7fcbdcea0700 1 --2- 172.21.15.27:0/2361000019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbd80f55c0 0x7fcbd80f59e0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 rx=0x7fcbd00099e0 tx=0x7fcbd0004d10).stop 2021-11-03T18:08:35.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.002+0000 7fcbdcea0700 1 -- 172.21.15.27:0/2361000019 shutdown_connections 2021-11-03T18:08:35.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.002+0000 7fcbdcea0700 1 --2- 172.21.15.27:0/2361000019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbd80f5ed0 0x7fcbd80f6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:35.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.002+0000 7fcbdcea0700 1 --2- 172.21.15.27:0/2361000019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbd80f55c0 0x7fcbd80f59e0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:35.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.002+0000 7fcbdcea0700 1 -- 172.21.15.27:0/2361000019 >> 172.21.15.27:0/2361000019 conn(0x7fcbd80f0de0 msgr2=0x7fcbd80f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:08:35.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.002+0000 7fcbdcea0700 1 -- 172.21.15.27:0/2361000019 shutdown_connections 2021-11-03T18:08:35.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.002+0000 7fcbdcea0700 1 -- 172.21.15.27:0/2361000019 wait complete. 2021-11-03T18:08:35.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.002+0000 7fcbdcea0700 1 Processor -- start 2021-11-03T18:08:35.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.002+0000 7fcbdcea0700 1 -- start start 2021-11-03T18:08:35.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.002+0000 7fcbdcea0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbd80f55c0 0x7fcbd8109d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:08:35.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.002+0000 7fcbdcea0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbd80f5ed0 0x7fcbd810c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:08:35.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.002+0000 7fcbdcea0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd80604a0 con 0x7fcbd80f55c0 2021-11-03T18:08:35.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.002+0000 7fcbdcea0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbd80605e0 con 0x7fcbd80f5ed0 2021-11-03T18:08:35.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.002+0000 7fcbd659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbd80f55c0 0x7fcbd8109d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:08:35.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.003+0000 7fcbd659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbd80f55c0 0x7fcbd8109d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33186/0 (socket says 172.21.15.27:33186) 2021-11-03T18:08:35.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.003+0000 7fcbd659c700 1 -- 172.21.15.27:0/1982854076 learned_addr learned my addr 172.21.15.27:0/1982854076 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:08:35.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.003+0000 7fcbd5d9b700 1 --2- 172.21.15.27:0/1982854076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbd80f5ed0 0x7fcbd810c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:08:35.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.003+0000 7fcbd659c700 1 -- 172.21.15.27:0/1982854076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbd80f5ed0 msgr2=0x7fcbd810c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:08:35.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.003+0000 7fcbd659c700 1 --2- 172.21.15.27:0/1982854076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbd80f5ed0 0x7fcbd810c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:35.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.003+0000 7fcbd659c700 1 -- 172.21.15.27:0/1982854076 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbd0009640 con 0x7fcbd80f55c0 2021-11-03T18:08:35.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.003+0000 7fcbd5d9b700 1 --2- 172.21.15.27:0/1982854076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbd80f5ed0 0x7fcbd810c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:08:35.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.003+0000 7fcbd659c700 1 --2- 172.21.15.27:0/1982854076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbd80f55c0 0x7fcbd8109d60 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 rx=0x7fcbd000b3b0 tx=0x7fcbd0004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:08:35.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.003+0000 7fcbceffd700 1 -- 172.21.15.27:0/1982854076 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbd000e030 con 0x7fcbd80f55c0 2021-11-03T18:08:35.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.003+0000 7fcbceffd700 1 -- 172.21.15.27:0/1982854076 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fcbd00049b0 con 0x7fcbd80f55c0 2021-11-03T18:08:35.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.003+0000 7fcbdcea0700 1 -- 172.21.15.27:0/1982854076 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbd810c7b0 con 0x7fcbd80f55c0 2021-11-03T18:08:35.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.003+0000 7fcbceffd700 1 -- 172.21.15.27:0/1982854076 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbd0018070 con 0x7fcbd80f55c0 2021-11-03T18:08:35.009 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.003+0000 7fcbdcea0700 1 -- 172.21.15.27:0/1982854076 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbd810cc60 con 0x7fcbd80f55c0 2021-11-03T18:08:35.009 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.004+0000 7fcbdcea0700 1 -- 172.21.15.27:0/1982854076 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbd80f4a50 con 0x7fcbd80f55c0 2021-11-03T18:08:35.009 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.004+0000 7fcbceffd700 1 -- 172.21.15.27:0/1982854076 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fcbd0007d40 con 0x7fcbd80f55c0 2021-11-03T18:08:35.010 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.005+0000 7fcbceffd700 1 --2- 172.21.15.27:0/1982854076 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcbc40640b0 0x7fcbc4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:08:35.010 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.005+0000 7fcbceffd700 1 -- 172.21.15.27:0/1982854076 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fcbd0082590 con 0x7fcbd80f55c0 2021-11-03T18:08:35.010 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.005+0000 7fcbd5d9b700 1 --2- 172.21.15.27:0/1982854076 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcbc40640b0 0x7fcbc4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:08:35.010 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.005+0000 7fcbd5d9b700 1 --2- 172.21.15.27:0/1982854076 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcbc40640b0 0x7fcbc4066570 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7fcbc0009740 tx=0x7fcbc0009be0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:08:35.011 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.009+0000 7fcbceffd700 1 -- 172.21.15.27:0/1982854076 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fcbd0054890 con 0x7fcbd80f55c0 2021-11-03T18:08:35.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:34 smithi027 ceph-mon[97244]: from='client.35768 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:08:35.212 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.214+0000 7fcbdcea0700 1 -- 172.21.15.27:0/1982854076 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fcbd8045ad0 con 0x7fcbd80f55c0 2021-11-03T18:08:35.213 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.214+0000 7fcbceffd700 1 -- 172.21.15.27:0/1982854076 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7fcbd0054480 con 0x7fcbd80f55c0 2021-11-03T18:08:35.213 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:08:35.213 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:08:35.214 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:08:35.214 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:08:35.214 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:08:35.214 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:08:35.215 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:08:35.215 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:08:35.215 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:08:35.216 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:08:35.216 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:08:35.216 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:08:35.217 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:08:35.217 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:08:35.217 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:08:35.217 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:08:35.217 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:08:35.218 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:08:35.218 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:08:35.218 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:08:35.219 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:08:35.219 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:08:35.219 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:08:35.219 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:08:35.219 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:08:35.220 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:08:35.220 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:08:35.220 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:08:35.220 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:08:35.221 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:08:35.221 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:08:35.221 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:08:35.221 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:08:35.222 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:08:35.222 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:08:35.222 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:08:35.222 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:08:35.222 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:08:35.223 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:08:35.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.216+0000 7fcbdcea0700 1 -- 172.21.15.27:0/1982854076 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcbc40640b0 msgr2=0x7fcbc4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:08:35.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.216+0000 7fcbdcea0700 1 --2- 172.21.15.27:0/1982854076 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcbc40640b0 0x7fcbc4066570 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7fcbc0009740 tx=0x7fcbc0009be0).stop 2021-11-03T18:08:35.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.216+0000 7fcbdcea0700 1 -- 172.21.15.27:0/1982854076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbd80f55c0 msgr2=0x7fcbd8109d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:08:35.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.216+0000 7fcbdcea0700 1 --2- 172.21.15.27:0/1982854076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbd80f55c0 0x7fcbd8109d60 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 rx=0x7fcbd000b3b0 tx=0x7fcbd0004d10).stop 2021-11-03T18:08:35.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.216+0000 7fcbdcea0700 1 -- 172.21.15.27:0/1982854076 shutdown_connections 2021-11-03T18:08:35.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.216+0000 7fcbdcea0700 1 --2- 172.21.15.27:0/1982854076 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcbc40640b0 0x7fcbc4066570 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:35.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.216+0000 7fcbdcea0700 1 --2- 172.21.15.27:0/1982854076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbd80f55c0 0x7fcbd8109d60 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:35.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.216+0000 7fcbdcea0700 1 --2- 172.21.15.27:0/1982854076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbd80f5ed0 0x7fcbd810c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:08:35.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.216+0000 7fcbdcea0700 1 -- 172.21.15.27:0/1982854076 >> 172.21.15.27:0/1982854076 conn(0x7fcbd80f0de0 msgr2=0x7fcbd80fa350 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:08:35.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.216+0000 7fcbdcea0700 1 -- 172.21.15.27:0/1982854076 shutdown_connections 2021-11-03T18:08:35.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:08:35.216+0000 7fcbdcea0700 1 -- 172.21.15.27:0/1982854076 wait complete. 2021-11-03T18:08:35.226 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:08:35.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:34 smithi115 ceph-mon[65453]: from='client.35768 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:08:36.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3157.435205 secs 2021-11-03T18:08:36.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:35 smithi027 ceph-mon[97244]: from='client.35772 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:08:36.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:35 smithi027 ceph-mon[97244]: pgmap v1618: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:36.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:35 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2765695440' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:36.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:35 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1982854076' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:08:36.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3157.435205 secs 2021-11-03T18:08:36.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:35 smithi115 ceph-mon[65453]: from='client.35772 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:08:36.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:35 smithi115 ceph-mon[65453]: pgmap v1618: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:36.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:35 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2765695440' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:36.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:35 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1982854076' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:08:38.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:37 smithi027 ceph-mon[97244]: pgmap v1619: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:38.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:37 smithi115 ceph-mon[65453]: pgmap v1619: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:40.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:39 smithi027 ceph-mon[97244]: pgmap v1620: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:40.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:39 smithi115 ceph-mon[65453]: pgmap v1620: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:41.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3162.435325 secs 2021-11-03T18:08:41.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3162.435325 secs 2021-11-03T18:08:42.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:42 smithi115 ceph-mon[65453]: pgmap v1621: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:42.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:08:42.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:08:42.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:08:42.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:42.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:42.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:42.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:42.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:42 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:08:42.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:42 smithi027 ceph-mon[97244]: pgmap v1621: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:42.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:08:42.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:08:42.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:08:42.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:42.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:42.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:42.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:42.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:42 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:08:44.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:44 smithi115 ceph-mon[65453]: pgmap v1622: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:44.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:44 smithi027 ceph-mon[97244]: pgmap v1622: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:45.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3167.435457 secs 2021-11-03T18:08:45.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3167.435457 secs 2021-11-03T18:08:46.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:46 smithi115 ceph-mon[65453]: pgmap v1623: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:46.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:46 smithi027 ceph-mon[97244]: pgmap v1623: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:48.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:48 smithi115 ceph-mon[65453]: pgmap v1624: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:48.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:48 smithi027 ceph-mon[97244]: pgmap v1624: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:50.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:50 smithi115 ceph-mon[65453]: pgmap v1625: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:50.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3172.435550 secs 2021-11-03T18:08:50.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:50 smithi027 ceph-mon[97244]: pgmap v1625: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:50.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3172.435550 secs 2021-11-03T18:08:52.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:52 smithi115 ceph-mon[65453]: pgmap v1626: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:52.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:08:52.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:52 smithi115 ceph-mon[65453]: poll took 20059.923 ms: 1 events 2021-11-03T18:08:52.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:08:52.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:08:52.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:52.490 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:52.490 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:52.491 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:52.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:52 smithi027 ceph-mon[97244]: pgmap v1626: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:52.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:08:52.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:52 smithi027 ceph-mon[97244]: poll took 20059.923 ms: 1 events 2021-11-03T18:08:52.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:08:52.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:08:52.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:52.524 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:52.524 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:52.524 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:08:53.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:53 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:08:53.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:53 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:08:54.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:54 smithi115 ceph-mon[65453]: pgmap v1627: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:54.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:54 smithi027 ceph-mon[97244]: pgmap v1627: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:55.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3177.435673 secs 2021-11-03T18:08:55.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3177.435673 secs 2021-11-03T18:08:56.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:56 smithi115 ceph-mon[65453]: pgmap v1628: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:56.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:08:56.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:08:56.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:56 smithi027 ceph-mon[97244]: pgmap v1628: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:56.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:08:56.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:08:58.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:08:58 smithi115 ceph-mon[65453]: pgmap v1629: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:08:58.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:08:58 smithi027 ceph-mon[97244]: pgmap v1629: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:00.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:00 smithi115 ceph-mon[65453]: pgmap v1630: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:00.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3182.435787 secs 2021-11-03T18:09:00.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:00 smithi027 ceph-mon[97244]: pgmap v1630: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:00.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3182.435787 secs 2021-11-03T18:09:02.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:02 smithi115 ceph-mon[65453]: pgmap v1631: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:02.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:09:02.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:09:02.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:09:02.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:02.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:02.490 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:02.490 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:02.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:02 smithi027 ceph-mon[97244]: pgmap v1631: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:02.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:09:02.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:09:02.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:09:02.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:02.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:02.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:02.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:03.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:03 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:09:03.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:03 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:09:04.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:04 smithi115 ceph-mon[65453]: pgmap v1632: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:04.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:04 smithi027 ceph-mon[97244]: pgmap v1632: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:05.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.331+0000 7fdca7fff700 1 -- 172.21.15.27:0/333223552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdca80f6530 msgr2=0x7fdca80f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:09:05.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.331+0000 7fdca7fff700 1 --2- 172.21.15.27:0/333223552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdca80f6530 0x7fdca80f6950 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 rx=0x7fdc98004660 tx=0x7fdc98009b30).stop 2021-11-03T18:09:05.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.331+0000 7fdca7fff700 1 -- 172.21.15.27:0/333223552 shutdown_connections 2021-11-03T18:09:05.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.331+0000 7fdca7fff700 1 --2- 172.21.15.27:0/333223552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdca80f77d0 0x7fdca80f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:05.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.331+0000 7fdca7fff700 1 --2- 172.21.15.27:0/333223552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdca80f6530 0x7fdca80f6950 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:05.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.331+0000 7fdca7fff700 1 -- 172.21.15.27:0/333223552 >> 172.21.15.27:0/333223552 conn(0x7fdca80f1cd0 msgr2=0x7fdca80f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:09:05.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.331+0000 7fdca7fff700 1 -- 172.21.15.27:0/333223552 shutdown_connections 2021-11-03T18:09:05.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.331+0000 7fdca7fff700 1 -- 172.21.15.27:0/333223552 wait complete. 2021-11-03T18:09:05.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.332+0000 7fdca7fff700 1 Processor -- start 2021-11-03T18:09:05.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.332+0000 7fdca7fff700 1 -- start start 2021-11-03T18:09:05.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.332+0000 7fdca7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdca80f6530 0x7fdca806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:09:05.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.332+0000 7fdca7fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdca80f77d0 0x7fdca8069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:09:05.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.332+0000 7fdca7fff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca806a380 con 0x7fdca80f6530 2021-11-03T18:09:05.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.332+0000 7fdca7fff700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdca806a4c0 con 0x7fdca80f77d0 2021-11-03T18:09:05.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.332+0000 7fdca6ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdca80f6530 0x7fdca806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:09:05.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.332+0000 7fdca6ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdca80f6530 0x7fdca806b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33208/0 (socket says 172.21.15.27:33208) 2021-11-03T18:09:05.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.332+0000 7fdca6ffd700 1 -- 172.21.15.27:0/3947722372 learned_addr learned my addr 172.21.15.27:0/3947722372 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:09:05.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.333+0000 7fdca6ffd700 1 -- 172.21.15.27:0/3947722372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdca80f77d0 msgr2=0x7fdca8069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:09:05.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.333+0000 7fdca67fc700 1 --2- 172.21.15.27:0/3947722372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdca80f77d0 0x7fdca8069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:09:05.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.333+0000 7fdca6ffd700 1 --2- 172.21.15.27:0/3947722372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdca80f77d0 0x7fdca8069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:05.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.333+0000 7fdca6ffd700 1 -- 172.21.15.27:0/3947722372 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc98005040 con 0x7fdca80f6530 2021-11-03T18:09:05.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.333+0000 7fdca6ffd700 1 --2- 172.21.15.27:0/3947722372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdca80f6530 0x7fdca806b9a0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 rx=0x7fdc98004f60 tx=0x7fdc98009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:09:05.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.333+0000 7fdc8f7fe700 1 -- 172.21.15.27:0/3947722372 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc98018070 con 0x7fdca80f6530 2021-11-03T18:09:05.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.333+0000 7fdca7fff700 1 -- 172.21.15.27:0/3947722372 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdca806a740 con 0x7fdca80f6530 2021-11-03T18:09:05.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.333+0000 7fdc8f7fe700 1 -- 172.21.15.27:0/3947722372 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fdc98007ba0 con 0x7fdca80f6530 2021-11-03T18:09:05.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.333+0000 7fdc8f7fe700 1 -- 172.21.15.27:0/3947722372 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdc9801c620 con 0x7fdca80f6530 2021-11-03T18:09:05.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.333+0000 7fdca7fff700 1 -- 172.21.15.27:0/3947722372 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdca806aca0 con 0x7fdca80f6530 2021-11-03T18:09:05.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.334+0000 7fdca7fff700 1 -- 172.21.15.27:0/3947722372 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdca8045ad0 con 0x7fdca80f6530 2021-11-03T18:09:05.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.335+0000 7fdc8f7fe700 1 -- 172.21.15.27:0/3947722372 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fdc9801c8e0 con 0x7fdca80f6530 2021-11-03T18:09:05.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.335+0000 7fdc8f7fe700 1 --2- 172.21.15.27:0/3947722372 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdc90064100 0x7fdc900665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:09:05.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.335+0000 7fdc8f7fe700 1 -- 172.21.15.27:0/3947722372 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fdc98082750 con 0x7fdca80f6530 2021-11-03T18:09:05.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.335+0000 7fdca67fc700 1 --2- 172.21.15.27:0/3947722372 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdc90064100 0x7fdc900665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:09:05.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.336+0000 7fdca67fc700 1 --2- 172.21.15.27:0/3947722372 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdc90064100 0x7fdc900665c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7fdc9c0041d0 tx=0x7fdc9c009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:09:05.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.339+0000 7fdc8f7fe700 1 -- 172.21.15.27:0/3947722372 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fdc98054a50 con 0x7fdca80f6530 2021-11-03T18:09:05.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3187.435928 secs 2021-11-03T18:09:05.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.517+0000 7fdca7fff700 1 -- 172.21.15.27:0/3947722372 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdca80fee00 con 0x7fdc90064100 2021-11-03T18:09:05.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.518+0000 7fdc8f7fe700 1 -- 172.21.15.27:0/3947722372 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fdca80fee00 con 0x7fdc90064100 2021-11-03T18:09:05.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.520+0000 7fdca7fff700 1 -- 172.21.15.27:0/3947722372 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdc90064100 msgr2=0x7fdc900665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:09:05.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.520+0000 7fdca7fff700 1 --2- 172.21.15.27:0/3947722372 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdc90064100 0x7fdc900665c0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7fdc9c0041d0 tx=0x7fdc9c009040).stop 2021-11-03T18:09:05.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.520+0000 7fdca7fff700 1 -- 172.21.15.27:0/3947722372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdca80f6530 msgr2=0x7fdca806b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:09:05.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.520+0000 7fdca7fff700 1 --2- 172.21.15.27:0/3947722372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdca80f6530 0x7fdca806b9a0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 rx=0x7fdc98004f60 tx=0x7fdc98009b30).stop 2021-11-03T18:09:05.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.520+0000 7fdca7fff700 1 -- 172.21.15.27:0/3947722372 shutdown_connections 2021-11-03T18:09:05.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.520+0000 7fdca7fff700 1 --2- 172.21.15.27:0/3947722372 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdc90064100 0x7fdc900665c0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:05.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.520+0000 7fdca7fff700 1 --2- 172.21.15.27:0/3947722372 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdca80f6530 0x7fdca806b9a0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:05.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.520+0000 7fdca7fff700 1 --2- 172.21.15.27:0/3947722372 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdca80f77d0 0x7fdca8069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:05.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.520+0000 7fdca7fff700 1 -- 172.21.15.27:0/3947722372 >> 172.21.15.27:0/3947722372 conn(0x7fdca80f1cd0 msgr2=0x7fdca80fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:09:05.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.520+0000 7fdca7fff700 1 -- 172.21.15.27:0/3947722372 shutdown_connections 2021-11-03T18:09:05.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.520+0000 7fdca7fff700 1 -- 172.21.15.27:0/3947722372 wait complete. 2021-11-03T18:09:05.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3187.435928 secs 2021-11-03T18:09:05.528 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:09:05.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.621+0000 7f00295c4700 1 -- 172.21.15.27:0/1754606449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00240f55c0 msgr2=0x7f00240f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:09:05.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.621+0000 7f00295c4700 1 --2- 172.21.15.27:0/1754606449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00240f55c0 0x7f00240f59e0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 rx=0x7f001c0099e0 tx=0x7f001c004d10).stop 2021-11-03T18:09:05.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.621+0000 7f00295c4700 1 -- 172.21.15.27:0/1754606449 shutdown_connections 2021-11-03T18:09:05.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.621+0000 7f00295c4700 1 --2- 172.21.15.27:0/1754606449 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00240f6860 0x7f00240f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:05.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.621+0000 7f00295c4700 1 --2- 172.21.15.27:0/1754606449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00240f55c0 0x7f00240f59e0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:05.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.621+0000 7f00295c4700 1 -- 172.21.15.27:0/1754606449 >> 172.21.15.27:0/1754606449 conn(0x7f00240f0d40 msgr2=0x7f00240f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:09:05.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.621+0000 7f00295c4700 1 -- 172.21.15.27:0/1754606449 shutdown_connections 2021-11-03T18:09:05.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.621+0000 7f00295c4700 1 -- 172.21.15.27:0/1754606449 wait complete. 2021-11-03T18:09:05.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.622+0000 7f00295c4700 1 Processor -- start 2021-11-03T18:09:05.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.622+0000 7f00295c4700 1 -- start start 2021-11-03T18:09:05.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.622+0000 7f00295c4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00240f55c0 0x7f0024109d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:09:05.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.622+0000 7f00295c4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00240f6860 0x7f002410c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:09:05.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.622+0000 7f00295c4700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00240603c0 con 0x7f00240f6860 2021-11-03T18:09:05.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.622+0000 7f00295c4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0024060500 con 0x7f00240f55c0 2021-11-03T18:09:05.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.622+0000 7f00227fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00240f6860 0x7f002410c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:09:05.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.622+0000 7f00227fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00240f6860 0x7f002410c280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33220/0 (socket says 172.21.15.27:33220) 2021-11-03T18:09:05.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.622+0000 7f00227fc700 1 -- 172.21.15.27:0/3577868640 learned_addr learned my addr 172.21.15.27:0/3577868640 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:09:05.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.622+0000 7f0022ffd700 1 --2- 172.21.15.27:0/3577868640 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00240f55c0 0x7f0024109d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:09:05.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.622+0000 7f00227fc700 1 -- 172.21.15.27:0/3577868640 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00240f55c0 msgr2=0x7f0024109d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:09:05.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.622+0000 7f00227fc700 1 --2- 172.21.15.27:0/3577868640 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00240f55c0 0x7f0024109d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:05.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.622+0000 7f00227fc700 1 -- 172.21.15.27:0/3577868640 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f001c009640 con 0x7f00240f6860 2021-11-03T18:09:05.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.622+0000 7f00227fc700 1 --2- 172.21.15.27:0/3577868640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00240f6860 0x7f002410c280 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 rx=0x7f000c0026d0 tx=0x7f000c009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:09:05.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.623+0000 7f001b7fe700 1 -- 172.21.15.27:0/3577868640 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f000c0079c0 con 0x7f00240f6860 2021-11-03T18:09:05.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.623+0000 7f001b7fe700 1 -- 172.21.15.27:0/3577868640 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f000c00f040 con 0x7f00240f6860 2021-11-03T18:09:05.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.623+0000 7f001b7fe700 1 -- 172.21.15.27:0/3577868640 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f000c0148b0 con 0x7f00240f6860 2021-11-03T18:09:05.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.623+0000 7f00295c4700 1 -- 172.21.15.27:0/3577868640 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f002410c830 con 0x7f00240f6860 2021-11-03T18:09:05.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.623+0000 7f00295c4700 1 -- 172.21.15.27:0/3577868640 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f002410cd40 con 0x7f00240f6860 2021-11-03T18:09:05.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.624+0000 7f00295c4700 1 -- 172.21.15.27:0/3577868640 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00240f49b0 con 0x7f00240f6860 2021-11-03T18:09:05.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.624+0000 7f001b7fe700 1 -- 172.21.15.27:0/3577868640 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f000c007b20 con 0x7f00240f6860 2021-11-03T18:09:05.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.624+0000 7f001b7fe700 1 --2- 172.21.15.27:0/3577868640 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f00100640b0 0x7f0010066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:09:05.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.624+0000 7f001b7fe700 1 -- 172.21.15.27:0/3577868640 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f000c0806f0 con 0x7f00240f6860 2021-11-03T18:09:05.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.624+0000 7f0022ffd700 1 --2- 172.21.15.27:0/3577868640 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f00100640b0 0x7f0010066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:09:05.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.625+0000 7f0022ffd700 1 --2- 172.21.15.27:0/3577868640 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f00100640b0 0x7f0010066570 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f001c0099b0 tx=0x7f001c004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:09:05.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.628+0000 7f001b7fe700 1 -- 172.21.15.27:0/3577868640 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f000c0529f0 con 0x7f00240f6860 2021-11-03T18:09:05.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.801+0000 7f00295c4700 1 -- 172.21.15.27:0/3577868640 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f00240fdf00 con 0x7f00100640b0 2021-11-03T18:09:05.813 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.815+0000 7f001b7fe700 1 -- 172.21.15.27:0/3577868640 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f00240fdf00 con 0x7f00100640b0 2021-11-03T18:09:05.814 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:09:05.814 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (60m) 10m ago 61m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:09:05.814 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (52m) 10m ago 61m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:09:05.815 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (52m) 9m ago 61m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:09:05.815 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (60m) 10m ago 61m 41.8M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:09:05.815 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (54m) 10m ago 57m 26.0M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:09:05.815 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (57m) 10m ago 57m 3845M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:09:05.816 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (52m) 9m ago 57m 25.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:09:05.816 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (54m) 9m ago 57m 26.8M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:09:05.816 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (54m) 10m ago 63m 554M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:09:05.816 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (53m) 9m ago 61m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:09:05.817 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (53m) 10m ago 63m 235M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:09:05.817 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (52m) 9m ago 61m 112M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:09:05.817 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (61m) 10m ago 61m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:09:05.817 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (60m) 9m ago 60m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:09:05.817 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (51m) 10m ago 60m 351M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:09:05.818 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (51m) 10m ago 59m 383M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:09:05.818 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (50m) 10m ago 59m 284M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:09:05.818 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (50m) 9m ago 59m 404M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:09:05.818 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (50m) 9m ago 58m 322M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:09:05.819 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (49m) 9m ago 58m 327M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:09:05.819 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (60m) 10m ago 61m 55.2M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:09:05.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.817+0000 7f00295c4700 1 -- 172.21.15.27:0/3577868640 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f00100640b0 msgr2=0x7f0010066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:09:05.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.817+0000 7f00295c4700 1 --2- 172.21.15.27:0/3577868640 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f00100640b0 0x7f0010066570 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f001c0099b0 tx=0x7f001c004d10).stop 2021-11-03T18:09:05.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.817+0000 7f00295c4700 1 -- 172.21.15.27:0/3577868640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00240f6860 msgr2=0x7f002410c280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:09:05.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.817+0000 7f00295c4700 1 --2- 172.21.15.27:0/3577868640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00240f6860 0x7f002410c280 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 rx=0x7f000c0026d0 tx=0x7f000c009de0).stop 2021-11-03T18:09:05.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.817+0000 7f00295c4700 1 -- 172.21.15.27:0/3577868640 shutdown_connections 2021-11-03T18:09:05.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.817+0000 7f00295c4700 1 --2- 172.21.15.27:0/3577868640 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f00100640b0 0x7f0010066570 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:05.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.817+0000 7f00295c4700 1 --2- 172.21.15.27:0/3577868640 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00240f55c0 0x7f0024109d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:05.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.817+0000 7f00295c4700 1 --2- 172.21.15.27:0/3577868640 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00240f6860 0x7f002410c280 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:05.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.817+0000 7f00295c4700 1 -- 172.21.15.27:0/3577868640 >> 172.21.15.27:0/3577868640 conn(0x7f00240f0d40 msgr2=0x7f00240fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:09:05.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.818+0000 7f00295c4700 1 -- 172.21.15.27:0/3577868640 shutdown_connections 2021-11-03T18:09:05.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.818+0000 7f00295c4700 1 -- 172.21.15.27:0/3577868640 wait complete. 2021-11-03T18:09:05.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.916+0000 7fc8ca678700 1 -- 172.21.15.27:0/2254163335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8c40f77d0 msgr2=0x7fc8c40f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:09:05.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.916+0000 7fc8ca678700 1 --2- 172.21.15.27:0/2254163335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8c40f77d0 0x7fc8c40f7c30 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 rx=0x7fc8b4004660 tx=0x7fc8b4009b30).stop 2021-11-03T18:09:05.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.916+0000 7fc8ca678700 1 -- 172.21.15.27:0/2254163335 shutdown_connections 2021-11-03T18:09:05.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.916+0000 7fc8ca678700 1 --2- 172.21.15.27:0/2254163335 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8c40f77d0 0x7fc8c40f7c30 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:05.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.916+0000 7fc8ca678700 1 --2- 172.21.15.27:0/2254163335 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8c40f6530 0x7fc8c40f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:05.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.916+0000 7fc8ca678700 1 -- 172.21.15.27:0/2254163335 >> 172.21.15.27:0/2254163335 conn(0x7fc8c40f1cd0 msgr2=0x7fc8c40f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:09:05.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.917+0000 7fc8ca678700 1 -- 172.21.15.27:0/2254163335 shutdown_connections 2021-11-03T18:09:05.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.917+0000 7fc8ca678700 1 -- 172.21.15.27:0/2254163335 wait complete. 2021-11-03T18:09:05.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.917+0000 7fc8ca678700 1 Processor -- start 2021-11-03T18:09:05.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.917+0000 7fc8ca678700 1 -- start start 2021-11-03T18:09:05.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.917+0000 7fc8ca678700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8c40f6530 0x7fc8c4106d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:09:05.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.917+0000 7fc8ca678700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8c40f77d0 0x7fc8c4107270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:09:05.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.917+0000 7fc8ca678700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c4107870 con 0x7fc8c40f6530 2021-11-03T18:09:05.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.917+0000 7fc8ca678700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c4108080 con 0x7fc8c40f77d0 2021-11-03T18:09:05.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.917+0000 7fc8c9676700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8c40f6530 0x7fc8c4106d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:09:05.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.917+0000 7fc8c9676700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8c40f6530 0x7fc8c4106d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33228/0 (socket says 172.21.15.27:33228) 2021-11-03T18:09:05.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.917+0000 7fc8c9676700 1 -- 172.21.15.27:0/1443632101 learned_addr learned my addr 172.21.15.27:0/1443632101 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:09:05.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.918+0000 7fc8c8e75700 1 --2- 172.21.15.27:0/1443632101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8c40f77d0 0x7fc8c4107270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:09:05.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.918+0000 7fc8c9676700 1 -- 172.21.15.27:0/1443632101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8c40f77d0 msgr2=0x7fc8c4107270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:09:05.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.918+0000 7fc8c9676700 1 --2- 172.21.15.27:0/1443632101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8c40f77d0 0x7fc8c4107270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:05.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.918+0000 7fc8c9676700 1 -- 172.21.15.27:0/1443632101 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8b4005040 con 0x7fc8c40f6530 2021-11-03T18:09:05.922 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.918+0000 7fc8c9676700 1 --2- 172.21.15.27:0/1443632101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8c40f6530 0x7fc8c4106d80 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 rx=0x7fc8c000c530 tx=0x7fc8c0007e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:09:05.922 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.918+0000 7fc8b9ffb700 1 -- 172.21.15.27:0/1443632101 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8c000f650 con 0x7fc8c40f6530 2021-11-03T18:09:05.922 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.918+0000 7fc8b9ffb700 1 -- 172.21.15.27:0/1443632101 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fc8c000fc60 con 0x7fc8c40f6530 2021-11-03T18:09:05.922 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.918+0000 7fc8ca678700 1 -- 172.21.15.27:0/1443632101 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8c4108300 con 0x7fc8c40f6530 2021-11-03T18:09:05.923 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.918+0000 7fc8b9ffb700 1 -- 172.21.15.27:0/1443632101 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8c0012d20 con 0x7fc8c40f6530 2021-11-03T18:09:05.923 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.918+0000 7fc8ca678700 1 -- 172.21.15.27:0/1443632101 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8c41088c0 con 0x7fc8c40f6530 2021-11-03T18:09:05.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.919+0000 7fc8ca678700 1 -- 172.21.15.27:0/1443632101 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8c4059100 con 0x7fc8c40f6530 2021-11-03T18:09:05.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.919+0000 7fc8b9ffb700 1 -- 172.21.15.27:0/1443632101 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fc8c0019030 con 0x7fc8c40f6530 2021-11-03T18:09:05.925 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.920+0000 7fc8b9ffb700 1 --2- 172.21.15.27:0/1443632101 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc8b0064100 0x7fc8b00665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:09:05.925 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.920+0000 7fc8b9ffb700 1 -- 172.21.15.27:0/1443632101 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fc8c0080680 con 0x7fc8c40f6530 2021-11-03T18:09:05.925 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.920+0000 7fc8c8e75700 1 --2- 172.21.15.27:0/1443632101 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc8b0064100 0x7fc8b00665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:09:05.925 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.920+0000 7fc8c8e75700 1 --2- 172.21.15.27:0/1443632101 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc8b0064100 0x7fc8b00665c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7fc8b4004e30 tx=0x7fc8b4009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:09:05.925 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:05.924+0000 7fc8b9ffb700 1 -- 172.21.15.27:0/1443632101 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fc8c00529a0 con 0x7fc8c40f6530 2021-11-03T18:09:06.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.153+0000 7fc8ca678700 1 -- 172.21.15.27:0/1443632101 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc8c4045ad0 con 0x7fc8c40f6530 2021-11-03T18:09:06.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.153+0000 7fc8b9ffb700 1 -- 172.21.15.27:0/1443632101 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fc8c0052590 con 0x7fc8c40f6530 2021-11-03T18:09:06.152 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:09:06.153 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:09:06.153 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:09:06.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:09:06.153 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:09:06.153 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:09:06.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:09:06.154 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:09:06.154 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:09:06.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:09:06.155 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:09:06.155 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:09:06.155 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:09:06.155 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:09:06.156 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:09:06.156 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:09:06.156 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:09:06.157 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:09:06.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.155+0000 7fc8ca678700 1 -- 172.21.15.27:0/1443632101 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc8b0064100 msgr2=0x7fc8b00665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:09:06.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.155+0000 7fc8ca678700 1 --2- 172.21.15.27:0/1443632101 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc8b0064100 0x7fc8b00665c0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7fc8b4004e30 tx=0x7fc8b4009b30).stop 2021-11-03T18:09:06.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.155+0000 7fc8ca678700 1 -- 172.21.15.27:0/1443632101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8c40f6530 msgr2=0x7fc8c4106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:09:06.160 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.155+0000 7fc8ca678700 1 --2- 172.21.15.27:0/1443632101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8c40f6530 0x7fc8c4106d80 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 rx=0x7fc8c000c530 tx=0x7fc8c0007e10).stop 2021-11-03T18:09:06.160 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.155+0000 7fc8ca678700 1 -- 172.21.15.27:0/1443632101 shutdown_connections 2021-11-03T18:09:06.160 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.155+0000 7fc8ca678700 1 --2- 172.21.15.27:0/1443632101 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc8b0064100 0x7fc8b00665c0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:06.160 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.155+0000 7fc8ca678700 1 --2- 172.21.15.27:0/1443632101 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8c40f6530 0x7fc8c4106d80 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:06.161 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.155+0000 7fc8ca678700 1 --2- 172.21.15.27:0/1443632101 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8c40f77d0 0x7fc8c4107270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:06.161 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.155+0000 7fc8ca678700 1 -- 172.21.15.27:0/1443632101 >> 172.21.15.27:0/1443632101 conn(0x7fc8c40f1cd0 msgr2=0x7fc8c40fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:09:06.161 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.155+0000 7fc8ca678700 1 -- 172.21.15.27:0/1443632101 shutdown_connections 2021-11-03T18:09:06.161 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.155+0000 7fc8ca678700 1 -- 172.21.15.27:0/1443632101 wait complete. 2021-11-03T18:09:06.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.255+0000 7f8c005f2700 1 -- 172.21.15.27:0/1564284250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf80f6ff0 msgr2=0x7f8bf80f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:09:06.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.255+0000 7f8c005f2700 1 --2- 172.21.15.27:0/1564284250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf80f6ff0 0x7f8bf80f7a20 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 rx=0x7f8be8004660 tx=0x7f8be8009b30).stop 2021-11-03T18:09:06.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.256+0000 7f8c005f2700 1 -- 172.21.15.27:0/1564284250 shutdown_connections 2021-11-03T18:09:06.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.256+0000 7f8c005f2700 1 --2- 172.21.15.27:0/1564284250 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf80f6ff0 0x7f8bf80f7a20 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:06.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.256+0000 7f8c005f2700 1 --2- 172.21.15.27:0/1564284250 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8bf80f66e0 0x7f8bf80f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:06.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.256+0000 7f8c005f2700 1 -- 172.21.15.27:0/1564284250 >> 172.21.15.27:0/1564284250 conn(0x7f8bf80f1f00 msgr2=0x7f8bf80f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:09:06.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.256+0000 7f8c005f2700 1 -- 172.21.15.27:0/1564284250 shutdown_connections 2021-11-03T18:09:06.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.256+0000 7f8c005f2700 1 -- 172.21.15.27:0/1564284250 wait complete. 2021-11-03T18:09:06.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.256+0000 7f8c005f2700 1 Processor -- start 2021-11-03T18:09:06.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.256+0000 7f8c005f2700 1 -- start start 2021-11-03T18:09:06.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.256+0000 7f8c005f2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf80f66e0 0x7f8bf8106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:09:06.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.256+0000 7f8c005f2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8bf80f6ff0 0x7f8bf8107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:09:06.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.256+0000 7f8c005f2700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf8107a10 con 0x7f8bf80f66e0 2021-11-03T18:09:06.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.256+0000 7f8c005f2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf8107b50 con 0x7f8bf80f6ff0 2021-11-03T18:09:06.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.256+0000 7f8bfe38e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf80f66e0 0x7f8bf8106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:09:06.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.257+0000 7f8bfe38e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf80f66e0 0x7f8bf8106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33238/0 (socket says 172.21.15.27:33238) 2021-11-03T18:09:06.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.257+0000 7f8bfe38e700 1 -- 172.21.15.27:0/316520615 learned_addr learned my addr 172.21.15.27:0/316520615 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:09:06.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.257+0000 7f8bfdb8d700 1 --2- 172.21.15.27:0/316520615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8bf80f6ff0 0x7f8bf8107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:09:06.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.257+0000 7f8bfe38e700 1 -- 172.21.15.27:0/316520615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8bf80f6ff0 msgr2=0x7f8bf8107460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:09:06.265 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.257+0000 7f8bfe38e700 1 --2- 172.21.15.27:0/316520615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8bf80f6ff0 0x7f8bf8107460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:06.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.257+0000 7f8bfe38e700 1 -- 172.21.15.27:0/316520615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be8005040 con 0x7f8bf80f66e0 2021-11-03T18:09:06.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.257+0000 7f8bfe38e700 1 --2- 172.21.15.27:0/316520615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf80f66e0 0x7f8bf8106f70 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 rx=0x7f8bf400c530 tx=0x7f8bf4007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:09:06.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.257+0000 7f8beeffd700 1 -- 172.21.15.27:0/316520615 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bf400f650 con 0x7f8bf80f66e0 2021-11-03T18:09:06.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.257+0000 7f8beeffd700 1 -- 172.21.15.27:0/316520615 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f8bf400fc60 con 0x7f8bf80f66e0 2021-11-03T18:09:06.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.257+0000 7f8c005f2700 1 -- 172.21.15.27:0/316520615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf81084f0 con 0x7f8bf80f66e0 2021-11-03T18:09:06.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.257+0000 7f8beeffd700 1 -- 172.21.15.27:0/316520615 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8bf4012c20 con 0x7f8bf80f66e0 2021-11-03T18:09:06.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.257+0000 7f8c005f2700 1 -- 172.21.15.27:0/316520615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf8108ab0 con 0x7f8bf80f66e0 2021-11-03T18:09:06.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.258+0000 7f8c005f2700 1 -- 172.21.15.27:0/316520615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf8045ad0 con 0x7f8bf80f66e0 2021-11-03T18:09:06.270 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.259+0000 7f8beeffd700 1 -- 172.21.15.27:0/316520615 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f8bf4010070 con 0x7f8bf80f66e0 2021-11-03T18:09:06.271 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.259+0000 7f8beeffd700 1 --2- 172.21.15.27:0/316520615 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8be40640b0 0x7f8be4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:09:06.271 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.259+0000 7f8beeffd700 1 -- 172.21.15.27:0/316520615 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f8bf40805f0 con 0x7f8bf80f66e0 2021-11-03T18:09:06.272 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.259+0000 7f8bfdb8d700 1 --2- 172.21.15.27:0/316520615 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8be40640b0 0x7f8be4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:09:06.273 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.259+0000 7f8bfdb8d700 1 --2- 172.21.15.27:0/316520615 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8be40640b0 0x7f8be4066570 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f8be8004630 tx=0x7f8be8009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:09:06.273 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.263+0000 7f8beeffd700 1 -- 172.21.15.27:0/316520615 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f8bf4052800 con 0x7f8bf80f66e0 2021-11-03T18:09:06.466 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.468+0000 7f8c005f2700 1 -- 172.21.15.27:0/316520615 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8bf80fefa0 con 0x7f8bf80f66e0 2021-11-03T18:09:06.467 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.468+0000 7f8beeffd700 1 -- 172.21.15.27:0/316520615 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f8bf40523f0 con 0x7f8bf80f66e0 2021-11-03T18:09:06.467 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:09:06.467 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:09:06.468 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:09:06.468 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:09:06.468 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:09:06.468 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:09:06.469 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:09:06.469 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:09:06.469 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:09:06.469 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:09:06.470 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:09:06.470 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:09:06.470 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:09:06.470 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:09:06.470 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:09:06.471 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:09:06.471 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:09:06.471 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:09:06.471 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:09:06.472 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:09:06.472 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:09:06.472 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:09:06.473 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:09:06.473 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:09:06.473 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:09:06.473 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:09:06.473 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:09:06.474 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:09:06.474 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:09:06.474 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:09:06.474 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:09:06.475 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:09:06.475 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:09:06.475 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:09:06.475 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:09:06.476 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:09:06.476 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:09:06.476 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:09:06.476 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:09:06.477 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.470+0000 7f8c005f2700 1 -- 172.21.15.27:0/316520615 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8be40640b0 msgr2=0x7f8be4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:09:06.478 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.470+0000 7f8c005f2700 1 --2- 172.21.15.27:0/316520615 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8be40640b0 0x7f8be4066570 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7f8be8004630 tx=0x7f8be8009b30).stop 2021-11-03T18:09:06.478 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.470+0000 7f8c005f2700 1 -- 172.21.15.27:0/316520615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf80f66e0 msgr2=0x7f8bf8106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:09:06.478 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.470+0000 7f8c005f2700 1 --2- 172.21.15.27:0/316520615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf80f66e0 0x7f8bf8106f70 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 rx=0x7f8bf400c530 tx=0x7f8bf4007e20).stop 2021-11-03T18:09:06.478 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.470+0000 7f8c005f2700 1 -- 172.21.15.27:0/316520615 shutdown_connections 2021-11-03T18:09:06.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.470+0000 7f8c005f2700 1 --2- 172.21.15.27:0/316520615 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8be40640b0 0x7f8be4066570 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:06.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.470+0000 7f8c005f2700 1 --2- 172.21.15.27:0/316520615 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8bf80f66e0 0x7f8bf8106f70 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:06.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.470+0000 7f8c005f2700 1 --2- 172.21.15.27:0/316520615 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8bf80f6ff0 0x7f8bf8107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:06.480 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.470+0000 7f8c005f2700 1 -- 172.21.15.27:0/316520615 >> 172.21.15.27:0/316520615 conn(0x7f8bf80f1f00 msgr2=0x7f8bf80fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:09:06.480 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.470+0000 7f8c005f2700 1 -- 172.21.15.27:0/316520615 shutdown_connections 2021-11-03T18:09:06.480 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:06.470+0000 7f8c005f2700 1 -- 172.21.15.27:0/316520615 wait complete. 2021-11-03T18:09:06.480 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:09:06.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:06 smithi115 ceph-mon[65453]: pgmap v1633: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:06.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:06 smithi115 ceph-mon[65453]: from='client.35784 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:09:06.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:06 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1443632101' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:06.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:06 smithi027 ceph-mon[97244]: pgmap v1633: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:06.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:06 smithi027 ceph-mon[97244]: from='client.35784 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:09:06.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:06 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1443632101' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:07.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:07 smithi115 ceph-mon[65453]: from='client.35788 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:09:07.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:07 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/316520615' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:09:07.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:07 smithi027 ceph-mon[97244]: from='client.35788 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:09:07.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:07 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/316520615' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:09:08.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:08 smithi115 ceph-mon[65453]: pgmap v1634: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:08.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:08 smithi027 ceph-mon[97244]: pgmap v1634: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:10.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:10 smithi115 ceph-mon[65453]: pgmap v1635: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:10.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3192.436071 secs 2021-11-03T18:09:10.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:10 smithi027 ceph-mon[97244]: pgmap v1635: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:10.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3192.436071 secs 2021-11-03T18:09:12.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:12 smithi115 ceph-mon[65453]: pgmap v1636: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:12.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:09:12.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:12 smithi027 ceph-mon[97244]: pgmap v1636: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:12.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:09:13.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:13 smithi027 ceph-mon[97244]: poll took 20059.657 ms: 1 events 2021-11-03T18:09:13.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:13 smithi115 ceph-mon[65453]: poll took 20059.657 ms: 1 events 2021-11-03T18:09:14.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:14 smithi115 ceph-mon[65453]: pgmap v1637: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:14.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:14 smithi027 ceph-mon[97244]: pgmap v1637: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:15.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3197.436178 secs 2021-11-03T18:09:15.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3197.436178 secs 2021-11-03T18:09:16.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:16 smithi115 ceph-mon[65453]: pgmap v1638: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:16.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:16 smithi027 ceph-mon[97244]: pgmap v1638: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:18.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:17 smithi115 ceph-mon[65453]: pgmap v1639: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:18.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:17 smithi115 ceph-mon[65453]: poll took 5051.617 ms: 1 events 2021-11-03T18:09:18.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:09:18.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:09:18.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:09:18.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:18.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:18.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:18.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:18.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:17 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:09:18.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:17 smithi027 ceph-mon[97244]: pgmap v1639: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:18.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:17 smithi027 ceph-mon[97244]: poll took 5051.617 ms: 1 events 2021-11-03T18:09:18.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:09:18.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:09:18.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:09:18.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:18.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:18.275 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:18.275 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:18.276 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:17 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:09:20.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:19 smithi115 ceph-mon[65453]: pgmap v1640: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:20.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:19 smithi027 ceph-mon[97244]: pgmap v1640: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:21.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3202.436301 secs 2021-11-03T18:09:21.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3202.436301 secs 2021-11-03T18:09:22.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:22 smithi115 ceph-mon[65453]: pgmap v1641: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:22.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:22 smithi027 ceph-mon[97244]: pgmap v1641: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:24.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:24 smithi115 ceph-mon[65453]: pgmap v1642: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:24.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:24 smithi027 ceph-mon[97244]: pgmap v1642: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:25.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3207.436443 secs 2021-11-03T18:09:25.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3207.436443 secs 2021-11-03T18:09:26.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:26 smithi115 ceph-mon[65453]: pgmap v1643: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:26.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:26 smithi027 ceph-mon[97244]: pgmap v1643: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:27.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:09:27.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:09:27.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:09:27.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:27.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:27.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:27.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:27.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:09:27.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:09:27.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:09:27.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:27.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:27.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:27.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:28.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:28 smithi115 ceph-mon[65453]: pgmap v1644: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:28.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:28 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:09:28.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:28 smithi027 ceph-mon[97244]: pgmap v1644: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:28.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:28 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:09:30.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:30 smithi115 ceph-mon[65453]: pgmap v1645: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:30.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3212.436525 secs 2021-11-03T18:09:30.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:30 smithi027 ceph-mon[97244]: pgmap v1645: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:30.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3212.436525 secs 2021-11-03T18:09:32.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:32 smithi115 ceph-mon[65453]: pgmap v1646: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:32.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:32 smithi027 ceph-mon[97244]: pgmap v1646: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:34.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:34 smithi115 ceph-mon[65453]: pgmap v1647: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:34.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:34 smithi027 ceph-mon[97244]: pgmap v1647: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:35.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3217.436643 secs 2021-11-03T18:09:35.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3217.436643 secs 2021-11-03T18:09:36.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:36 smithi115 ceph-mon[65453]: pgmap v1648: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:36.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:36 smithi027 ceph-mon[97244]: pgmap v1648: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:36.586 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.585+0000 7f70c5c05700 1 -- 172.21.15.27:0/2706560454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70c00f9830 msgr2=0x7f70c00fbc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:09:36.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.585+0000 7f70c5c05700 1 --2- 172.21.15.27:0/2706560454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70c00f9830 0x7f70c00fbc20 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 rx=0x7f70b4004660 tx=0x7f70b4009b30).stop 2021-11-03T18:09:36.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.585+0000 7f70c5c05700 1 -- 172.21.15.27:0/2706560454 shutdown_connections 2021-11-03T18:09:36.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.585+0000 7f70c5c05700 1 --2- 172.21.15.27:0/2706560454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70c00f9830 0x7f70c00fbc20 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:36.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.585+0000 7f70c5c05700 1 --2- 172.21.15.27:0/2706560454 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70c0060350 0x7f70c00f9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:36.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.585+0000 7f70c5c05700 1 -- 172.21.15.27:0/2706560454 >> 172.21.15.27:0/2706560454 conn(0x7f70c00f0de0 msgr2=0x7f70c00f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:09:36.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.586+0000 7f70c5c05700 1 -- 172.21.15.27:0/2706560454 shutdown_connections 2021-11-03T18:09:36.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.586+0000 7f70c5c05700 1 -- 172.21.15.27:0/2706560454 wait complete. 2021-11-03T18:09:36.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.586+0000 7f70c5c05700 1 Processor -- start 2021-11-03T18:09:36.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.586+0000 7f70c5c05700 1 -- start start 2021-11-03T18:09:36.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.586+0000 7f70c5c05700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70c0060350 0x7f70c0106f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:09:36.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.587+0000 7f70c5c05700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70c00f9830 0x7f70c0107430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:09:36.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.587+0000 7f70c5c05700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c01079e0 con 0x7f70c0060350 2021-11-03T18:09:36.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.587+0000 7f70c5c05700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c0107b20 con 0x7f70c00f9830 2021-11-03T18:09:36.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.587+0000 7f70bf7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70c0060350 0x7f70c0106f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:09:36.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.587+0000 7f70bf7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70c0060350 0x7f70c0106f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33268/0 (socket says 172.21.15.27:33268) 2021-11-03T18:09:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.587+0000 7f70bf7fe700 1 -- 172.21.15.27:0/3426886024 learned_addr learned my addr 172.21.15.27:0/3426886024 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:09:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.587+0000 7f70beffd700 1 --2- 172.21.15.27:0/3426886024 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70c00f9830 0x7f70c0107430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:09:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.587+0000 7f70bf7fe700 1 -- 172.21.15.27:0/3426886024 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70c00f9830 msgr2=0x7f70c0107430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:09:36.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.587+0000 7f70bf7fe700 1 --2- 172.21.15.27:0/3426886024 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70c00f9830 0x7f70c0107430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:36.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.587+0000 7f70bf7fe700 1 -- 172.21.15.27:0/3426886024 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70b4005040 con 0x7f70c0060350 2021-11-03T18:09:36.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.587+0000 7f70bf7fe700 1 --2- 172.21.15.27:0/3426886024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70c0060350 0x7f70c0106f40 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 rx=0x7f70b0002700 tx=0x7f70b0009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:09:36.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.587+0000 7f70a7fff700 1 -- 172.21.15.27:0/3426886024 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70b00079c0 con 0x7f70c0060350 2021-11-03T18:09:36.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.587+0000 7f70c5c05700 1 -- 172.21.15.27:0/3426886024 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70c01084c0 con 0x7f70c0060350 2021-11-03T18:09:36.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.587+0000 7f70a7fff700 1 -- 172.21.15.27:0/3426886024 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f70b000f040 con 0x7f70c0060350 2021-11-03T18:09:36.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.587+0000 7f70a7fff700 1 -- 172.21.15.27:0/3426886024 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f70b0014860 con 0x7f70c0060350 2021-11-03T18:09:36.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.587+0000 7f70c5c05700 1 -- 172.21.15.27:0/3426886024 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70c0108a80 con 0x7f70c0060350 2021-11-03T18:09:36.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.589+0000 7f70c5c05700 1 -- 172.21.15.27:0/3426886024 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70c0045ad0 con 0x7f70c0060350 2021-11-03T18:09:36.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.589+0000 7f70a7fff700 1 -- 172.21.15.27:0/3426886024 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f70b001b060 con 0x7f70c0060350 2021-11-03T18:09:36.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.589+0000 7f70a7fff700 1 --2- 172.21.15.27:0/3426886024 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f70a80640b0 0x7f70a8066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:09:36.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.589+0000 7f70a7fff700 1 -- 172.21.15.27:0/3426886024 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f70b0080600 con 0x7f70c0060350 2021-11-03T18:09:36.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.589+0000 7f70beffd700 1 --2- 172.21.15.27:0/3426886024 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f70a80640b0 0x7f70a8066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:09:36.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.590+0000 7f70beffd700 1 --2- 172.21.15.27:0/3426886024 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f70a80640b0 0x7f70a8066570 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f70b4004630 tx=0x7f70b4009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:09:36.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.593+0000 7f70a7fff700 1 -- 172.21.15.27:0/3426886024 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f70b0052900 con 0x7f70c0060350 2021-11-03T18:09:36.772 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.774+0000 7f70c5c05700 1 -- 172.21.15.27:0/3426886024 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f70c00ff040 con 0x7f70a80640b0 2021-11-03T18:09:36.773 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.776+0000 7f70a7fff700 1 -- 172.21.15.27:0/3426886024 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f70b4019040 con 0x7f70a80640b0 2021-11-03T18:09:36.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.778+0000 7f70c5c05700 1 -- 172.21.15.27:0/3426886024 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f70a80640b0 msgr2=0x7f70a8066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:09:36.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.778+0000 7f70c5c05700 1 --2- 172.21.15.27:0/3426886024 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f70a80640b0 0x7f70a8066570 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f70b4004630 tx=0x7f70b4009b30).stop 2021-11-03T18:09:36.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.778+0000 7f70c5c05700 1 -- 172.21.15.27:0/3426886024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70c0060350 msgr2=0x7f70c0106f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:09:36.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.778+0000 7f70c5c05700 1 --2- 172.21.15.27:0/3426886024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70c0060350 0x7f70c0106f40 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 rx=0x7f70b0002700 tx=0x7f70b0009d80).stop 2021-11-03T18:09:36.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.778+0000 7f70c5c05700 1 -- 172.21.15.27:0/3426886024 shutdown_connections 2021-11-03T18:09:36.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.778+0000 7f70c5c05700 1 --2- 172.21.15.27:0/3426886024 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f70a80640b0 0x7f70a8066570 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:36.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.778+0000 7f70c5c05700 1 --2- 172.21.15.27:0/3426886024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70c0060350 0x7f70c0106f40 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:36.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.778+0000 7f70c5c05700 1 --2- 172.21.15.27:0/3426886024 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70c00f9830 0x7f70c0107430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:36.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.778+0000 7f70c5c05700 1 -- 172.21.15.27:0/3426886024 >> 172.21.15.27:0/3426886024 conn(0x7f70c00f0de0 msgr2=0x7f70c00f31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:09:36.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.778+0000 7f70c5c05700 1 -- 172.21.15.27:0/3426886024 shutdown_connections 2021-11-03T18:09:36.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.778+0000 7f70c5c05700 1 -- 172.21.15.27:0/3426886024 wait complete. 2021-11-03T18:09:36.786 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:09:36.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.878+0000 7f49fedcd700 1 -- 172.21.15.27:0/777763124 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49f80f7980 msgr2=0x7f49f80f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:09:36.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.878+0000 7f49fedcd700 1 --2- 172.21.15.27:0/777763124 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49f80f7980 0x7f49f80f7de0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 rx=0x7f49e4004660 tx=0x7f49e4009b30).stop 2021-11-03T18:09:36.879 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.878+0000 7f49fedcd700 1 -- 172.21.15.27:0/777763124 shutdown_connections 2021-11-03T18:09:36.879 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.878+0000 7f49fedcd700 1 --2- 172.21.15.27:0/777763124 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49f80f7980 0x7f49f80f7de0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:36.879 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.878+0000 7f49fedcd700 1 --2- 172.21.15.27:0/777763124 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49f80f66e0 0x7f49f80f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:36.880 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.878+0000 7f49fedcd700 1 -- 172.21.15.27:0/777763124 >> 172.21.15.27:0/777763124 conn(0x7f49f80f1e40 msgr2=0x7f49f80f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:09:36.880 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.878+0000 7f49fedcd700 1 -- 172.21.15.27:0/777763124 shutdown_connections 2021-11-03T18:09:36.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.878+0000 7f49fedcd700 1 -- 172.21.15.27:0/777763124 wait complete. 2021-11-03T18:09:36.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.878+0000 7f49fedcd700 1 Processor -- start 2021-11-03T18:09:36.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.879+0000 7f49fedcd700 1 -- start start 2021-11-03T18:09:36.882 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.879+0000 7f49fedcd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49f80f66e0 0x7f49f8106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:09:36.882 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.879+0000 7f49fedcd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49f80f7980 0x7f49f8107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:09:36.883 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.879+0000 7f49fedcd700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f8107a00 con 0x7f49f80f7980 2021-11-03T18:09:36.883 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.879+0000 7f49fedcd700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f8107b40 con 0x7f49f80f66e0 2021-11-03T18:09:36.883 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.879+0000 7f49f7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49f80f7980 0x7f49f8107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:09:36.883 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.879+0000 7f49f7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49f80f7980 0x7f49f8107450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33280/0 (socket says 172.21.15.27:33280) 2021-11-03T18:09:36.883 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.879+0000 7f49f7fff700 1 -- 172.21.15.27:0/2889186616 learned_addr learned my addr 172.21.15.27:0/2889186616 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:09:36.884 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.879+0000 7f49fcb69700 1 --2- 172.21.15.27:0/2889186616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49f80f66e0 0x7f49f8106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:09:36.884 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.879+0000 7f49f7fff700 1 -- 172.21.15.27:0/2889186616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49f80f66e0 msgr2=0x7f49f8106f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:09:36.884 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.879+0000 7f49f7fff700 1 --2- 172.21.15.27:0/2889186616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49f80f66e0 0x7f49f8106f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:36.885 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.879+0000 7f49f7fff700 1 -- 172.21.15.27:0/2889186616 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e4005040 con 0x7f49f80f7980 2021-11-03T18:09:36.885 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.879+0000 7f49f7fff700 1 --2- 172.21.15.27:0/2889186616 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49f80f7980 0x7f49f8107450 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 rx=0x7f49e4009990 tx=0x7f49e4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:09:36.885 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.880+0000 7f49f57fa700 1 -- 172.21.15.27:0/2889186616 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49e4018070 con 0x7f49f80f7980 2021-11-03T18:09:36.886 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.880+0000 7f49f57fa700 1 -- 172.21.15.27:0/2889186616 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f49e4007ba0 con 0x7f49f80f7980 2021-11-03T18:09:36.886 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.880+0000 7f49fedcd700 1 -- 172.21.15.27:0/2889186616 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49f8108480 con 0x7f49f80f7980 2021-11-03T18:09:36.886 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.880+0000 7f49f57fa700 1 -- 172.21.15.27:0/2889186616 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49e401c3c0 con 0x7f49f80f7980 2021-11-03T18:09:36.886 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.880+0000 7f49fedcd700 1 -- 172.21.15.27:0/2889186616 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49f81089e0 con 0x7f49f80f7980 2021-11-03T18:09:36.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.881+0000 7f49fedcd700 1 -- 172.21.15.27:0/2889186616 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49f8045ad0 con 0x7f49f80f7980 2021-11-03T18:09:36.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.881+0000 7f49f57fa700 1 -- 172.21.15.27:0/2889186616 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f49e401d420 con 0x7f49f80f7980 2021-11-03T18:09:36.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.881+0000 7f49f57fa700 1 --2- 172.21.15.27:0/2889186616 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f49e80640b0 0x7f49e8066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:09:36.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.882+0000 7f49f57fa700 1 -- 172.21.15.27:0/2889186616 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f49e40823a0 con 0x7f49f80f7980 2021-11-03T18:09:36.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.882+0000 7f49fcb69700 1 --2- 172.21.15.27:0/2889186616 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f49e80640b0 0x7f49e8066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:09:36.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.882+0000 7f49fcb69700 1 --2- 172.21.15.27:0/2889186616 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f49e80640b0 0x7f49e8066570 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f49f80f77e0 tx=0x7f49ec00b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:09:36.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:36.886+0000 7f49f57fa700 1 -- 172.21.15.27:0/2889186616 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f49e4054760 con 0x7f49f80f7980 2021-11-03T18:09:37.060 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.062+0000 7f49fedcd700 1 -- 172.21.15.27:0/2889186616 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f49f8108c50 con 0x7f49e80640b0 2021-11-03T18:09:37.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.077+0000 7f49f57fa700 1 -- 172.21.15.27:0/2889186616 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f49f8108c50 con 0x7f49e80640b0 2021-11-03T18:09:37.075 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:09:37.076 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (61m) 20s ago 62m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:09:37.076 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (52m) 20s ago 62m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:09:37.076 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (52m) 9m ago 61m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:09:37.076 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (61m) 20s ago 62m 42.1M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:09:37.076 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (54m) 20s ago 57m 26.3M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:09:37.077 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (57m) 20s ago 57m 3847M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:09:37.077 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (53m) 9m ago 57m 25.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:09:37.077 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (55m) 9m ago 57m 26.8M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:09:37.077 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (54m) 20s ago 63m 572M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:09:37.078 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (54m) 9m ago 61m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:09:37.078 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (53m) 20s ago 63m 273M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:09:37.078 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (53m) 9m ago 61m 112M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:09:37.078 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (62m) 20s ago 62m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:09:37.078 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (61m) 9m ago 61m 18.3M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:09:37.079 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (52m) 20s ago 60m 371M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:09:37.079 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (51m) 20s ago 60m 405M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:09:37.079 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (51m) 20s ago 60m 302M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:09:37.079 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (51m) 9m ago 59m 404M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:09:37.080 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (50m) 9m ago 59m 322M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:09:37.080 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (50m) 9m ago 59m 327M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:09:37.080 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (60m) 20s ago 61m 57.6M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:09:37.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.079+0000 7f49fedcd700 1 -- 172.21.15.27:0/2889186616 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f49e80640b0 msgr2=0x7f49e8066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:09:37.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.079+0000 7f49fedcd700 1 --2- 172.21.15.27:0/2889186616 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f49e80640b0 0x7f49e8066570 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f49f80f77e0 tx=0x7f49ec00b040).stop 2021-11-03T18:09:37.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.079+0000 7f49fedcd700 1 -- 172.21.15.27:0/2889186616 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49f80f7980 msgr2=0x7f49f8107450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:09:37.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.079+0000 7f49fedcd700 1 --2- 172.21.15.27:0/2889186616 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49f80f7980 0x7f49f8107450 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 rx=0x7f49e4009990 tx=0x7f49e4009b30).stop 2021-11-03T18:09:37.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.079+0000 7f49fedcd700 1 -- 172.21.15.27:0/2889186616 shutdown_connections 2021-11-03T18:09:37.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.079+0000 7f49fedcd700 1 --2- 172.21.15.27:0/2889186616 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f49e80640b0 0x7f49e8066570 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:37.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.079+0000 7f49fedcd700 1 --2- 172.21.15.27:0/2889186616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f49f80f66e0 0x7f49f8106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:37.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.079+0000 7f49fedcd700 1 --2- 172.21.15.27:0/2889186616 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f49f80f7980 0x7f49f8107450 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:37.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.079+0000 7f49fedcd700 1 -- 172.21.15.27:0/2889186616 >> 172.21.15.27:0/2889186616 conn(0x7f49f80f1e40 msgr2=0x7f49f80fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:09:37.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.079+0000 7f49fedcd700 1 -- 172.21.15.27:0/2889186616 shutdown_connections 2021-11-03T18:09:37.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.079+0000 7f49fedcd700 1 -- 172.21.15.27:0/2889186616 wait complete. 2021-11-03T18:09:37.181 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.180+0000 7ffad359e700 1 -- 172.21.15.27:0/621777541 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffad40f5440 msgr2=0x7ffad40f5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:09:37.181 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.180+0000 7ffad359e700 1 --2- 172.21.15.27:0/621777541 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffad40f5440 0x7ffad40f5860 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 rx=0x7ffac4004660 tx=0x7ffac4009b30).stop 2021-11-03T18:09:37.181 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.180+0000 7ffad359e700 1 -- 172.21.15.27:0/621777541 shutdown_connections 2021-11-03T18:09:37.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.180+0000 7ffad359e700 1 --2- 172.21.15.27:0/621777541 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffad40f66e0 0x7ffad40f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:37.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.180+0000 7ffad359e700 1 --2- 172.21.15.27:0/621777541 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffad40f5440 0x7ffad40f5860 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:37.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.180+0000 7ffad359e700 1 -- 172.21.15.27:0/621777541 >> 172.21.15.27:0/621777541 conn(0x7ffad40f0ba0 msgr2=0x7ffad40f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:09:37.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.181+0000 7ffad359e700 1 -- 172.21.15.27:0/621777541 shutdown_connections 2021-11-03T18:09:37.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.181+0000 7ffad359e700 1 -- 172.21.15.27:0/621777541 wait complete. 2021-11-03T18:09:37.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.181+0000 7ffad359e700 1 Processor -- start 2021-11-03T18:09:37.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.181+0000 7ffad359e700 1 -- start start 2021-11-03T18:09:37.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.181+0000 7ffad359e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffad40f5440 0x7ffad4106e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:09:37.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.181+0000 7ffad359e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffad40f66e0 0x7ffad4107300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:09:37.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.181+0000 7ffad359e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad4107900 con 0x7ffad40f5440 2021-11-03T18:09:37.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.181+0000 7ffad359e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffad4108110 con 0x7ffad40f66e0 2021-11-03T18:09:37.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.181+0000 7ffad259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffad40f5440 0x7ffad4106e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:09:37.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.181+0000 7ffad259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffad40f5440 0x7ffad4106e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33288/0 (socket says 172.21.15.27:33288) 2021-11-03T18:09:37.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.181+0000 7ffad259c700 1 -- 172.21.15.27:0/3628667205 learned_addr learned my addr 172.21.15.27:0/3628667205 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:09:37.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.182+0000 7ffad1d9b700 1 --2- 172.21.15.27:0/3628667205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffad40f66e0 0x7ffad4107300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:09:37.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.182+0000 7ffad259c700 1 -- 172.21.15.27:0/3628667205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffad40f66e0 msgr2=0x7ffad4107300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:09:37.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.182+0000 7ffad259c700 1 --2- 172.21.15.27:0/3628667205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffad40f66e0 0x7ffad4107300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:37.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.182+0000 7ffad259c700 1 -- 172.21.15.27:0/3628667205 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffac4005040 con 0x7ffad40f5440 2021-11-03T18:09:37.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.182+0000 7ffad1d9b700 1 --2- 172.21.15.27:0/3628667205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffad40f66e0 0x7ffad4107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T18:09:37.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.182+0000 7ffad259c700 1 --2- 172.21.15.27:0/3628667205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffad40f5440 0x7ffad4106e10 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 rx=0x7ffac4004f60 tx=0x7ffac4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:09:37.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.182+0000 7ffac2ffd700 1 -- 172.21.15.27:0/3628667205 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffac4018070 con 0x7ffad40f5440 2021-11-03T18:09:37.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.182+0000 7ffac2ffd700 1 -- 172.21.15.27:0/3628667205 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7ffac4007ba0 con 0x7ffad40f5440 2021-11-03T18:09:37.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.182+0000 7ffac2ffd700 1 -- 172.21.15.27:0/3628667205 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffac401c620 con 0x7ffad40f5440 2021-11-03T18:09:37.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.182+0000 7ffad359e700 1 -- 172.21.15.27:0/3628667205 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffad4108330 con 0x7ffad40f5440 2021-11-03T18:09:37.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.182+0000 7ffad359e700 1 -- 172.21.15.27:0/3628667205 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffad4108890 con 0x7ffad40f5440 2021-11-03T18:09:37.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.183+0000 7ffad359e700 1 -- 172.21.15.27:0/3628667205 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffad4059100 con 0x7ffad40f5440 2021-11-03T18:09:37.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.183+0000 7ffac2ffd700 1 -- 172.21.15.27:0/3628667205 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7ffac401d420 con 0x7ffad40f5440 2021-11-03T18:09:37.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.184+0000 7ffac2ffd700 1 --2- 172.21.15.27:0/3628667205 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffabc064100 0x7ffabc0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:09:37.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.184+0000 7ffac2ffd700 1 -- 172.21.15.27:0/3628667205 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7ffac4082670 con 0x7ffad40f5440 2021-11-03T18:09:37.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.184+0000 7ffad1d9b700 1 --2- 172.21.15.27:0/3628667205 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffabc064100 0x7ffabc0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:09:37.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.184+0000 7ffad1d9b700 1 --2- 172.21.15.27:0/3628667205 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffabc064100 0x7ffabc0665c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7ffac8004a70 tx=0x7ffac8009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:09:37.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.188+0000 7ffac2ffd700 1 -- 172.21.15.27:0/3628667205 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7ffac4054a30 con 0x7ffad40f5440 2021-11-03T18:09:37.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.416+0000 7ffad359e700 1 -- 172.21.15.27:0/3628667205 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ffad4045ad0 con 0x7ffad40f5440 2021-11-03T18:09:37.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.417+0000 7ffac2ffd700 1 -- 172.21.15.27:0/3628667205 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7ffac4054620 con 0x7ffad40f5440 2021-11-03T18:09:37.415 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:09:37.416 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:09:37.416 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:09:37.416 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:09:37.417 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:09:37.417 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:09:37.417 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:09:37.418 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:09:37.418 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:09:37.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:09:37.419 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:09:37.419 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:09:37.419 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:09:37.419 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:09:37.420 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:09:37.420 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:09:37.420 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:09:37.420 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:09:37.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.418+0000 7ffad359e700 1 -- 172.21.15.27:0/3628667205 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffabc064100 msgr2=0x7ffabc0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:09:37.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.418+0000 7ffad359e700 1 --2- 172.21.15.27:0/3628667205 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffabc064100 0x7ffabc0665c0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 rx=0x7ffac8004a70 tx=0x7ffac8009040).stop 2021-11-03T18:09:37.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.418+0000 7ffad359e700 1 -- 172.21.15.27:0/3628667205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffad40f5440 msgr2=0x7ffad4106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:09:37.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.418+0000 7ffad359e700 1 --2- 172.21.15.27:0/3628667205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffad40f5440 0x7ffad4106e10 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 rx=0x7ffac4004f60 tx=0x7ffac4009b30).stop 2021-11-03T18:09:37.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.418+0000 7ffad359e700 1 -- 172.21.15.27:0/3628667205 shutdown_connections 2021-11-03T18:09:37.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.418+0000 7ffad359e700 1 --2- 172.21.15.27:0/3628667205 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffabc064100 0x7ffabc0665c0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:37.423 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.418+0000 7ffad359e700 1 --2- 172.21.15.27:0/3628667205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffad40f5440 0x7ffad4106e10 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:37.423 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.418+0000 7ffad359e700 1 --2- 172.21.15.27:0/3628667205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffad40f66e0 0x7ffad4107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:37.423 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.418+0000 7ffad359e700 1 -- 172.21.15.27:0/3628667205 >> 172.21.15.27:0/3628667205 conn(0x7ffad40f0ba0 msgr2=0x7ffad40fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:09:37.423 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.418+0000 7ffad359e700 1 -- 172.21.15.27:0/3628667205 shutdown_connections 2021-11-03T18:09:37.424 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.418+0000 7ffad359e700 1 -- 172.21.15.27:0/3628667205 wait complete. 2021-11-03T18:09:37.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:09:37.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:09:37.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:09:37.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:37.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:37.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:37.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:37.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.519+0000 7f25aabdb700 1 -- 172.21.15.27:0/730149060 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25a40f55c0 msgr2=0x7f25a40f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:09:37.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.519+0000 7f25aabdb700 1 --2- 172.21.15.27:0/730149060 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25a40f55c0 0x7f25a40f59e0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 rx=0x7f2598004660 tx=0x7f2598009b30).stop 2021-11-03T18:09:37.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.519+0000 7f25aabdb700 1 -- 172.21.15.27:0/730149060 shutdown_connections 2021-11-03T18:09:37.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.519+0000 7f25aabdb700 1 --2- 172.21.15.27:0/730149060 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25a40f6860 0x7f25a40f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:37.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.519+0000 7f25aabdb700 1 --2- 172.21.15.27:0/730149060 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25a40f55c0 0x7f25a40f59e0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:37.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.519+0000 7f25aabdb700 1 -- 172.21.15.27:0/730149060 >> 172.21.15.27:0/730149060 conn(0x7f25a40f0d40 msgr2=0x7f25a40f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:09:37.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.519+0000 7f25aabdb700 1 -- 172.21.15.27:0/730149060 shutdown_connections 2021-11-03T18:09:37.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.519+0000 7f25aabdb700 1 -- 172.21.15.27:0/730149060 wait complete. 2021-11-03T18:09:37.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.519+0000 7f25aabdb700 1 Processor -- start 2021-11-03T18:09:37.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.519+0000 7f25aabdb700 1 -- start start 2021-11-03T18:09:37.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.520+0000 7f25aabdb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25a40f55c0 0x7f25a4109c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:09:37.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.520+0000 7f25aabdb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25a40f6860 0x7f25a410c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:09:37.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.520+0000 7f25aabdb700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a40603c0 con 0x7f25a40f55c0 2021-11-03T18:09:37.524 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.520+0000 7f25aabdb700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a4060500 con 0x7f25a40f6860 2021-11-03T18:09:37.524 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.520+0000 7f25a8977700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25a40f55c0 0x7f25a4109c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:09:37.524 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.520+0000 7f25a8977700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25a40f55c0 0x7f25a4109c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33298/0 (socket says 172.21.15.27:33298) 2021-11-03T18:09:37.524 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.520+0000 7f25a8977700 1 -- 172.21.15.27:0/329050628 learned_addr learned my addr 172.21.15.27:0/329050628 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:09:37.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.520+0000 7f25a3fff700 1 --2- 172.21.15.27:0/329050628 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25a40f6860 0x7f25a410c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:09:37.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.520+0000 7f25a8977700 1 -- 172.21.15.27:0/329050628 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25a40f6860 msgr2=0x7f25a410c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:09:37.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.520+0000 7f25a8977700 1 --2- 172.21.15.27:0/329050628 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25a40f6860 0x7f25a410c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:37.526 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.520+0000 7f25a8977700 1 -- 172.21.15.27:0/329050628 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2598005040 con 0x7f25a40f55c0 2021-11-03T18:09:37.526 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.520+0000 7f25a8977700 1 --2- 172.21.15.27:0/329050628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25a40f55c0 0x7f25a4109c90 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 rx=0x7f2598013a70 tx=0x7f2598009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:09:37.526 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.520+0000 7f25a17fa700 1 -- 172.21.15.27:0/329050628 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2598018070 con 0x7f25a40f55c0 2021-11-03T18:09:37.526 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.520+0000 7f25aabdb700 1 -- 172.21.15.27:0/329050628 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25a410c6e0 con 0x7f25a40f55c0 2021-11-03T18:09:37.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.521+0000 7f25a17fa700 1 -- 172.21.15.27:0/329050628 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f25980097d0 con 0x7f25a40f55c0 2021-11-03T18:09:37.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.521+0000 7f25a17fa700 1 -- 172.21.15.27:0/329050628 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f259801de80 con 0x7f25a40f55c0 2021-11-03T18:09:37.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.521+0000 7f25aabdb700 1 -- 172.21.15.27:0/329050628 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a410cc10 con 0x7f25a40f55c0 2021-11-03T18:09:37.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.522+0000 7f25aabdb700 1 -- 172.21.15.27:0/329050628 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25a40f49b0 con 0x7f25a40f55c0 2021-11-03T18:09:37.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.522+0000 7f25a17fa700 1 -- 172.21.15.27:0/329050628 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f2598007ba0 con 0x7f25a40f55c0 2021-11-03T18:09:37.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.522+0000 7f25a17fa700 1 --2- 172.21.15.27:0/329050628 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f25940640b0 0x7f2594066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:09:37.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.522+0000 7f25a17fa700 1 -- 172.21.15.27:0/329050628 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f2598082410 con 0x7f25a40f55c0 2021-11-03T18:09:37.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.522+0000 7f25a3fff700 1 --2- 172.21.15.27:0/329050628 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f25940640b0 0x7f2594066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:09:37.529 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:09:37.529 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:09:37.529 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:09:37.530 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:37.530 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:37.530 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:37.530 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:37.531 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.523+0000 7f25a3fff700 1 --2- 172.21.15.27:0/329050628 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f25940640b0 0x7f2594066570 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7f2590004c40 tx=0x7f259000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:09:37.532 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.526+0000 7f25a17fa700 1 -- 172.21.15.27:0/329050628 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f25980547d0 con 0x7f25a40f55c0 2021-11-03T18:09:37.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.730+0000 7f25aabdb700 1 -- 172.21.15.27:0/329050628 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f25a40fde90 con 0x7f25a40f55c0 2021-11-03T18:09:37.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.731+0000 7f25a17fa700 1 -- 172.21.15.27:0/329050628 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f25980543c0 con 0x7f25a40f55c0 2021-11-03T18:09:37.730 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:09:37.730 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:09:37.730 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:09:37.730 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:09:37.731 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:09:37.731 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:09:37.731 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:09:37.731 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:09:37.732 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:09:37.732 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:09:37.732 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:09:37.732 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:09:37.732 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:09:37.733 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:09:37.733 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:09:37.733 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:09:37.734 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:09:37.734 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:09:37.734 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:09:37.734 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:09:37.734 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:09:37.735 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:09:37.735 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:09:37.735 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:09:37.735 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:09:37.736 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:09:37.736 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:09:37.736 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:09:37.736 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:09:37.737 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:09:37.737 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:09:37.737 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:09:37.737 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:09:37.738 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:09:37.738 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:09:37.738 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:09:37.738 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:09:37.739 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:09:37.739 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:09:37.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.733+0000 7f25aabdb700 1 -- 172.21.15.27:0/329050628 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f25940640b0 msgr2=0x7f2594066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:09:37.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.733+0000 7f25aabdb700 1 --2- 172.21.15.27:0/329050628 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f25940640b0 0x7f2594066570 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7f2590004c40 tx=0x7f259000b040).stop 2021-11-03T18:09:37.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.733+0000 7f25aabdb700 1 -- 172.21.15.27:0/329050628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25a40f55c0 msgr2=0x7f25a4109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:09:37.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.733+0000 7f25aabdb700 1 --2- 172.21.15.27:0/329050628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25a40f55c0 0x7f25a4109c90 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 rx=0x7f2598013a70 tx=0x7f2598009b30).stop 2021-11-03T18:09:37.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.733+0000 7f25aabdb700 1 -- 172.21.15.27:0/329050628 shutdown_connections 2021-11-03T18:09:37.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.733+0000 7f25aabdb700 1 --2- 172.21.15.27:0/329050628 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f25940640b0 0x7f2594066570 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:37.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.733+0000 7f25aabdb700 1 --2- 172.21.15.27:0/329050628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25a40f55c0 0x7f25a4109c90 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:37.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.733+0000 7f25aabdb700 1 --2- 172.21.15.27:0/329050628 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25a40f6860 0x7f25a410c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:09:37.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.733+0000 7f25aabdb700 1 -- 172.21.15.27:0/329050628 >> 172.21.15.27:0/329050628 conn(0x7f25a40f0d40 msgr2=0x7f25a40fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:09:37.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.733+0000 7f25aabdb700 1 -- 172.21.15.27:0/329050628 shutdown_connections 2021-11-03T18:09:37.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:09:37.733+0000 7f25aabdb700 1 -- 172.21.15.27:0/329050628 wait complete. 2021-11-03T18:09:37.743 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:09:38.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:38 smithi115 ceph-mon[65453]: pgmap v1649: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:38.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:38 smithi115 ceph-mon[65453]: from='client.35800 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:09:38.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:38 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:09:38.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:38 smithi115 ceph-mon[65453]: from='client.35804 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:09:38.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:38 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3628667205' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:38.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:38 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/329050628' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:09:38.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:38 smithi027 ceph-mon[97244]: pgmap v1649: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:38.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:38 smithi027 ceph-mon[97244]: from='client.35800 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:09:38.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:38 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:09:38.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:38 smithi027 ceph-mon[97244]: from='client.35804 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:09:38.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:38 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3628667205' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:38.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:38 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/329050628' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:09:40.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:40 smithi115 ceph-mon[65453]: pgmap v1650: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:40.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3222.436780 secs 2021-11-03T18:09:40.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:40 smithi027 ceph-mon[97244]: pgmap v1650: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:40.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3222.436780 secs 2021-11-03T18:09:42.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:42 smithi115 ceph-mon[65453]: pgmap v1651: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:42.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:42 smithi027 ceph-mon[97244]: pgmap v1651: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:44.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:44 smithi115 ceph-mon[65453]: pgmap v1652: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:44.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:44 smithi027 ceph-mon[97244]: pgmap v1652: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:45.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3227.436922 secs 2021-11-03T18:09:45.574 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3227.436922 secs 2021-11-03T18:09:46.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:46 smithi115 ceph-mon[65453]: pgmap v1653: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:46.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:46 smithi027 ceph-mon[97244]: pgmap v1653: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:47.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:09:47.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:09:48.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:48 smithi115 ceph-mon[65453]: pgmap v1654: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:48.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:48 smithi115 ceph-mon[65453]: poll took 30093.340 ms: 1 events 2021-11-03T18:09:48.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:48 smithi027 ceph-mon[97244]: pgmap v1654: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:48.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:48 smithi027 ceph-mon[97244]: poll took 30093.340 ms: 1 events 2021-11-03T18:09:50.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:50 smithi115 ceph-mon[65453]: pgmap v1655: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:50.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3232.436991 secs 2021-11-03T18:09:50.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:09:50.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:09:50.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:09:50.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:50.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:50.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:50.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:50.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:50 smithi027 ceph-mon[97244]: pgmap v1655: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:50.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3232.436991 secs 2021-11-03T18:09:50.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:09:50.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:09:50.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:09:50.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:50.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:50.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:50.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:09:51.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:51 smithi115 ceph-mon[65453]: poll took 2877.884 ms: 1 events 2021-11-03T18:09:51.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:51 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:09:51.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:51 smithi027 ceph-mon[97244]: poll took 2877.884 ms: 1 events 2021-11-03T18:09:51.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:51 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:09:52.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:52 smithi115 ceph-mon[65453]: pgmap v1656: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:52.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:52 smithi027 ceph-mon[97244]: pgmap v1656: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:54.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:54 smithi115 ceph-mon[65453]: pgmap v1657: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:54.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:54 smithi027 ceph-mon[97244]: pgmap v1657: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:55.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3237.437134 secs 2021-11-03T18:09:55.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3237.437134 secs 2021-11-03T18:09:56.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:56 smithi115 ceph-mon[65453]: pgmap v1658: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:56.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:09:56.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:09:56.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:56 smithi027 ceph-mon[97244]: pgmap v1658: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:56.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:09:56.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:09:58.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:09:58 smithi115 ceph-mon[65453]: pgmap v1659: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:09:58.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:09:58 smithi027 ceph-mon[97244]: pgmap v1659: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:00.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:00 smithi115 ceph-mon[65453]: pgmap v1660: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:00.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3242.437283 secs 2021-11-03T18:10:00.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:00 smithi115 ceph-mon[65453]: overall HEALTH_WARN 1 filesystem is degraded; 1 filesystem has a failed mds daemon; 1 MDSs report slow requests; 3 daemons have recently crashed 2021-11-03T18:10:00.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:10:00.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:00 smithi027 ceph-mon[97244]: pgmap v1660: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:00.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3242.437283 secs 2021-11-03T18:10:00.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:00 smithi027 ceph-mon[97244]: overall HEALTH_WARN 1 filesystem is degraded; 1 filesystem has a failed mds daemon; 1 MDSs report slow requests; 3 daemons have recently crashed 2021-11-03T18:10:00.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:10:01.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:01 smithi115 ceph-mon[65453]: poll took 10031.001 ms: 1 events 2021-11-03T18:10:01.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:01 smithi027 ceph-mon[97244]: poll took 10031.001 ms: 1 events 2021-11-03T18:10:02.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:02 smithi115 ceph-mon[65453]: pgmap v1661: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:02.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:02 smithi027 ceph-mon[97244]: pgmap v1661: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:04.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:04 smithi115 ceph-mon[65453]: pgmap v1662: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:04.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:04 smithi115 ceph-mon[65453]: poll took 2632.381 ms: 1 events 2021-11-03T18:10:04.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:10:04.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:10:04.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:10:04.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:04.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:04.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:04.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:04.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:04 smithi027 ceph-mon[97244]: pgmap v1662: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:04.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:04 smithi027 ceph-mon[97244]: poll took 2632.381 ms: 1 events 2021-11-03T18:10:04.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:10:04.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:10:04.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:10:04.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:04.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:04.524 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:04.524 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:05.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:05 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:10:05.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3247.437399 secs 2021-11-03T18:10:05.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:05 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:10:05.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3247.437399 secs 2021-11-03T18:10:06.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:06 smithi115 ceph-mon[65453]: pgmap v1663: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:06.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:06 smithi027 ceph-mon[97244]: pgmap v1663: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:07.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:07.850+0000 7f7d126ed700 1 -- 172.21.15.27:0/2857611887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d0c0f66e0 msgr2=0x7f7d0c0f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:10:07.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:07.850+0000 7f7d126ed700 1 --2- 172.21.15.27:0/2857611887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d0c0f66e0 0x7f7d0c0f6b40 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 rx=0x7f7cfc004660 tx=0x7f7cfc009b30).stop 2021-11-03T18:10:07.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:07.850+0000 7f7d126ed700 1 -- 172.21.15.27:0/2857611887 shutdown_connections 2021-11-03T18:10:07.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:07.850+0000 7f7d126ed700 1 --2- 172.21.15.27:0/2857611887 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d0c0f66e0 0x7f7d0c0f6b40 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:07.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:07.850+0000 7f7d126ed700 1 --2- 172.21.15.27:0/2857611887 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d0c0f5440 0x7f7d0c0f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:07.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:07.850+0000 7f7d126ed700 1 -- 172.21.15.27:0/2857611887 >> 172.21.15.27:0/2857611887 conn(0x7f7d0c0f0ba0 msgr2=0x7f7d0c0f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:10:07.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:07.850+0000 7f7d126ed700 1 -- 172.21.15.27:0/2857611887 shutdown_connections 2021-11-03T18:10:07.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:07.850+0000 7f7d126ed700 1 -- 172.21.15.27:0/2857611887 wait complete. 2021-11-03T18:10:07.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:07.850+0000 7f7d126ed700 1 Processor -- start 2021-11-03T18:10:07.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:07.850+0000 7f7d126ed700 1 -- start start 2021-11-03T18:10:07.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:07.850+0000 7f7d126ed700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d0c0f5440 0x7f7d0c105d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:10:07.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:07.850+0000 7f7d126ed700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d0c0f66e0 0x7f7d0c106220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:10:07.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:07.850+0000 7f7d126ed700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0c1067d0 con 0x7f7d0c0f5440 2021-11-03T18:10:07.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:07.850+0000 7f7d126ed700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0c106fd0 con 0x7f7d0c0f66e0 2021-11-03T18:10:07.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:07.851+0000 7f7d116eb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d0c0f5440 0x7f7d0c105d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:10:07.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:07.851+0000 7f7d116eb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d0c0f5440 0x7f7d0c105d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33320/0 (socket says 172.21.15.27:33320) 2021-11-03T18:10:07.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:07.851+0000 7f7d116eb700 1 -- 172.21.15.27:0/158444016 learned_addr learned my addr 172.21.15.27:0/158444016 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:10:07.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:07.851+0000 7f7d10eea700 1 --2- 172.21.15.27:0/158444016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d0c0f66e0 0x7f7d0c106220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:10:07.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:07.851+0000 7f7d116eb700 1 -- 172.21.15.27:0/158444016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d0c0f66e0 msgr2=0x7f7d0c106220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:10:07.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:07.851+0000 7f7d116eb700 1 --2- 172.21.15.27:0/158444016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d0c0f66e0 0x7f7d0c106220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:07.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:07.851+0000 7f7d116eb700 1 -- 172.21.15.27:0/158444016 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cfc005040 con 0x7f7d0c0f5440 2021-11-03T18:10:07.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:07.851+0000 7f7d116eb700 1 --2- 172.21.15.27:0/158444016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d0c0f5440 0x7f7d0c105d30 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 rx=0x7f7d0800c530 tx=0x7f7d08007e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:10:07.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:07.851+0000 7f7d01ffb700 1 -- 172.21.15.27:0/158444016 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d0800f650 con 0x7f7d0c0f5440 2021-11-03T18:10:07.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:07.851+0000 7f7d01ffb700 1 -- 172.21.15.27:0/158444016 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f7d0800fc60 con 0x7f7d0c0f5440 2021-11-03T18:10:07.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:07.851+0000 7f7d01ffb700 1 -- 172.21.15.27:0/158444016 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d08012cb0 con 0x7f7d0c0f5440 2021-11-03T18:10:07.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:07.851+0000 7f7d126ed700 1 -- 172.21.15.27:0/158444016 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d0c107250 con 0x7f7d0c0f5440 2021-11-03T18:10:07.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:07.852+0000 7f7d126ed700 1 -- 172.21.15.27:0/158444016 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d0c107770 con 0x7f7d0c0f5440 2021-11-03T18:10:07.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:07.853+0000 7f7d126ed700 1 -- 172.21.15.27:0/158444016 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d0c045ad0 con 0x7f7d0c0f5440 2021-11-03T18:10:07.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:07.853+0000 7f7d01ffb700 1 -- 172.21.15.27:0/158444016 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f7d08019030 con 0x7f7d0c0f5440 2021-11-03T18:10:07.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:07.853+0000 7f7d01ffb700 1 --2- 172.21.15.27:0/158444016 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7cf8064100 0x7f7cf80665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:10:07.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:07.853+0000 7f7d01ffb700 1 -- 172.21.15.27:0/158444016 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f7d08080610 con 0x7f7d0c0f5440 2021-11-03T18:10:07.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:07.853+0000 7f7d10eea700 1 --2- 172.21.15.27:0/158444016 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7cf8064100 0x7f7cf80665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:10:07.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:07.854+0000 7f7d10eea700 1 --2- 172.21.15.27:0/158444016 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7cf8064100 0x7f7cf80665c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7f7cfc004630 tx=0x7f7cfc009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:10:07.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:07.857+0000 7f7d01ffb700 1 -- 172.21.15.27:0/158444016 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f7d080528d0 con 0x7f7d0c0f5440 2021-11-03T18:10:08.032 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.035+0000 7f7d126ed700 1 -- 172.21.15.27:0/158444016 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d0c0ff720 con 0x7f7cf8064100 2021-11-03T18:10:08.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.037+0000 7f7d01ffb700 1 -- 172.21.15.27:0/158444016 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f7d0c0ff720 con 0x7f7cf8064100 2021-11-03T18:10:08.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.038+0000 7f7d126ed700 1 -- 172.21.15.27:0/158444016 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7cf8064100 msgr2=0x7f7cf80665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:10:08.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.038+0000 7f7d126ed700 1 --2- 172.21.15.27:0/158444016 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7cf8064100 0x7f7cf80665c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 rx=0x7f7cfc004630 tx=0x7f7cfc009b30).stop 2021-11-03T18:10:08.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.038+0000 7f7d126ed700 1 -- 172.21.15.27:0/158444016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d0c0f5440 msgr2=0x7f7d0c105d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:10:08.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.038+0000 7f7d126ed700 1 --2- 172.21.15.27:0/158444016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d0c0f5440 0x7f7d0c105d30 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 rx=0x7f7d0800c530 tx=0x7f7d08007e10).stop 2021-11-03T18:10:08.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.038+0000 7f7d126ed700 1 -- 172.21.15.27:0/158444016 shutdown_connections 2021-11-03T18:10:08.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.038+0000 7f7d126ed700 1 --2- 172.21.15.27:0/158444016 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7cf8064100 0x7f7cf80665c0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:08.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.038+0000 7f7d126ed700 1 --2- 172.21.15.27:0/158444016 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7d0c0f5440 0x7f7d0c105d30 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:08.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.038+0000 7f7d126ed700 1 --2- 172.21.15.27:0/158444016 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7d0c0f66e0 0x7f7d0c106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:08.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.038+0000 7f7d126ed700 1 -- 172.21.15.27:0/158444016 >> 172.21.15.27:0/158444016 conn(0x7f7d0c0f0ba0 msgr2=0x7f7d0c0fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:10:08.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.039+0000 7f7d126ed700 1 -- 172.21.15.27:0/158444016 shutdown_connections 2021-11-03T18:10:08.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.039+0000 7f7d126ed700 1 -- 172.21.15.27:0/158444016 wait complete. 2021-11-03T18:10:08.046 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:10:08.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.139+0000 7f4aa1e09700 1 -- 172.21.15.27:0/676553317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a9c0f5ef0 msgr2=0x7f4a9c0f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:10:08.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.139+0000 7f4aa1e09700 1 --2- 172.21.15.27:0/676553317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a9c0f5ef0 0x7f4a9c0f6920 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 rx=0x7f4a90004660 tx=0x7f4a90009b30).stop 2021-11-03T18:10:08.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.139+0000 7f4aa1e09700 1 -- 172.21.15.27:0/676553317 shutdown_connections 2021-11-03T18:10:08.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.139+0000 7f4aa1e09700 1 --2- 172.21.15.27:0/676553317 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a9c0f5ef0 0x7f4a9c0f6920 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:08.142 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.139+0000 7f4aa1e09700 1 --2- 172.21.15.27:0/676553317 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a9c0f55e0 0x7f4a9c0f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T18:10:08.143 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.139+0000 7f4aa1e09700 1 -- 172.21.15.27:0/676553317 >> 172.21.15.27:0/676553317 conn(0x7f4a9c0f0de0 msgr2=0x7f4a9c0f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:10:08.143 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.139+0000 7f4aa1e09700 1 -- 172.21.15.27:0/676553317 shutdown_connections 2021-11-03T18:10:08.144 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.139+0000 7f4aa1e09700 1 -- 172.21.15.27:0/676553317 wait complete. 2021-11-03T18:10:08.145 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.140+0000 7f4aa1e09700 1 Processor -- start 2021-11-03T18:10:08.146 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.140+0000 7f4aa1e09700 1 -- start start 2021-11-03T18:10:08.147 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.140+0000 7f4aa1e09700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a9c0f55e0 0x7f4a9c0f97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:10:08.147 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.140+0000 7f4aa1e09700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a9c0f5ef0 0x7f4a9c0f7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:10:08.147 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.140+0000 7f4aa1e09700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9c0f8250 con 0x7f4a9c0f55e0 2021-11-03T18:10:08.148 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.140+0000 7f4aa1e09700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9c0f8390 con 0x7f4a9c0f5ef0 2021-11-03T18:10:08.148 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.140+0000 7f4a9b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a9c0f55e0 0x7f4a9c0f97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:10:08.149 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.140+0000 7f4a9b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a9c0f55e0 0x7f4a9c0f97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33328/0 (socket says 172.21.15.27:33328) 2021-11-03T18:10:08.149 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.140+0000 7f4a9b7fe700 1 -- 172.21.15.27:0/1112074909 learned_addr learned my addr 172.21.15.27:0/1112074909 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:10:08.150 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.140+0000 7f4a9affd700 1 --2- 172.21.15.27:0/1112074909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a9c0f5ef0 0x7f4a9c0f7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:10:08.150 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.140+0000 7f4a9b7fe700 1 -- 172.21.15.27:0/1112074909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a9c0f5ef0 msgr2=0x7f4a9c0f7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:10:08.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.140+0000 7f4a9b7fe700 1 --2- 172.21.15.27:0/1112074909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a9c0f5ef0 0x7f4a9c0f7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:08.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.140+0000 7f4a9b7fe700 1 -- 172.21.15.27:0/1112074909 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a90005040 con 0x7f4a9c0f55e0 2021-11-03T18:10:08.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.140+0000 7f4a9affd700 1 --2- 172.21.15.27:0/1112074909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a9c0f5ef0 0x7f4a9c0f7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:10:08.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.140+0000 7f4a9b7fe700 1 --2- 172.21.15.27:0/1112074909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a9c0f55e0 0x7f4a9c0f97e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 rx=0x7f4a8c00b410 tx=0x7f4a8c006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:10:08.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.141+0000 7f4a83fff700 1 -- 172.21.15.27:0/1112074909 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a8c0125f0 con 0x7f4a9c0f55e0 2021-11-03T18:10:08.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.141+0000 7f4aa1e09700 1 -- 172.21.15.27:0/1112074909 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a9c0f8670 con 0x7f4a9c0f55e0 2021-11-03T18:10:08.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.141+0000 7f4a83fff700 1 -- 172.21.15.27:0/1112074909 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f4a8c012c00 con 0x7f4a9c0f55e0 2021-11-03T18:10:08.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.141+0000 7f4a83fff700 1 -- 172.21.15.27:0/1112074909 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4a8c002ae0 con 0x7f4a9c0f55e0 2021-11-03T18:10:08.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.141+0000 7f4aa1e09700 1 -- 172.21.15.27:0/1112074909 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a9c10efd0 con 0x7f4a9c0f55e0 2021-11-03T18:10:08.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.142+0000 7f4aa1e09700 1 -- 172.21.15.27:0/1112074909 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a9c045ad0 con 0x7f4a9c0f55e0 2021-11-03T18:10:08.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.142+0000 7f4a83fff700 1 -- 172.21.15.27:0/1112074909 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f4a8c00f030 con 0x7f4a9c0f55e0 2021-11-03T18:10:08.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.142+0000 7f4a83fff700 1 --2- 172.21.15.27:0/1112074909 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4a840640b0 0x7f4a84066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:10:08.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.143+0000 7f4a83fff700 1 -- 172.21.15.27:0/1112074909 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f4a8c080c20 con 0x7f4a9c0f55e0 2021-11-03T18:10:08.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.143+0000 7f4a9affd700 1 --2- 172.21.15.27:0/1112074909 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4a840640b0 0x7f4a84066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:10:08.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.143+0000 7f4a9affd700 1 --2- 172.21.15.27:0/1112074909 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4a840640b0 0x7f4a84066570 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f4a90004e30 tx=0x7f4a90009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:10:08.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.146+0000 7f4a83fff700 1 -- 172.21.15.27:0/1112074909 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f4a8c052f20 con 0x7f4a9c0f55e0 2021-11-03T18:10:08.318 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.321+0000 7f4aa1e09700 1 -- 172.21.15.27:0/1112074909 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a9c0fad60 con 0x7f4a840640b0 2021-11-03T18:10:08.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.336+0000 7f4a83fff700 1 -- 172.21.15.27:0/1112074909 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f4a9c0fad60 con 0x7f4a840640b0 2021-11-03T18:10:08.333 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:10:08.334 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (61m) 51s ago 62m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:10:08.334 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (53m) 51s ago 62m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:10:08.334 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (53m) 5s ago 62m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:10:08.334 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (61m) 51s ago 62m 42.1M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:10:08.335 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (55m) 51s ago 58m 26.3M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:10:08.335 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (58m) 51s ago 58m 3847M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:10:08.335 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (53m) 5s ago 58m 25.9M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:10:08.335 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (55m) 5s ago 58m 27.0M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:10:08.339 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (55m) 51s ago 64m 572M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:10:08.339 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (54m) 5s ago 62m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:10:08.339 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (54m) 51s ago 64m 273M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:10:08.340 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (53m) 5s ago 62m 129M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:10:08.340 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (62m) 51s ago 62m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:10:08.340 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (61m) 5s ago 61m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:10:08.340 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (53m) 51s ago 61m 371M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:10:08.340 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (52m) 51s ago 61m 405M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:10:08.341 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (52m) 51s ago 60m 302M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:10:08.341 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (51m) 5s ago 60m 429M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:10:08.341 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (51m) 5s ago 60m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:10:08.341 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (50m) 5s ago 59m 348M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:10:08.342 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (61m) 51s ago 62m 57.6M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:10:08.342 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.337+0000 7f4aa1e09700 1 -- 172.21.15.27:0/1112074909 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4a840640b0 msgr2=0x7f4a84066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:10:08.343 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.337+0000 7f4aa1e09700 1 --2- 172.21.15.27:0/1112074909 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4a840640b0 0x7f4a84066570 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f4a90004e30 tx=0x7f4a90009b30).stop 2021-11-03T18:10:08.343 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.337+0000 7f4aa1e09700 1 -- 172.21.15.27:0/1112074909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a9c0f55e0 msgr2=0x7f4a9c0f97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:10:08.343 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.337+0000 7f4aa1e09700 1 --2- 172.21.15.27:0/1112074909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a9c0f55e0 0x7f4a9c0f97e0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 rx=0x7f4a8c00b410 tx=0x7f4a8c006ef0).stop 2021-11-03T18:10:08.344 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.337+0000 7f4aa1e09700 1 -- 172.21.15.27:0/1112074909 shutdown_connections 2021-11-03T18:10:08.344 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.337+0000 7f4aa1e09700 1 --2- 172.21.15.27:0/1112074909 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4a840640b0 0x7f4a84066570 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:08.344 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.337+0000 7f4aa1e09700 1 --2- 172.21.15.27:0/1112074909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a9c0f55e0 0x7f4a9c0f97e0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:08.344 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.337+0000 7f4aa1e09700 1 --2- 172.21.15.27:0/1112074909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a9c0f5ef0 0x7f4a9c0f7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:08.345 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.337+0000 7f4aa1e09700 1 -- 172.21.15.27:0/1112074909 >> 172.21.15.27:0/1112074909 conn(0x7f4a9c0f0de0 msgr2=0x7f4a9c05ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:10:08.345 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.338+0000 7f4aa1e09700 1 -- 172.21.15.27:0/1112074909 shutdown_connections 2021-11-03T18:10:08.345 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.338+0000 7f4aa1e09700 1 -- 172.21.15.27:0/1112074909 wait complete. 2021-11-03T18:10:08.436 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.437+0000 7f36c35a3700 1 -- 172.21.15.27:0/335800684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36bc0f6ff0 msgr2=0x7f36bc0f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:10:08.436 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.437+0000 7f36c35a3700 1 --2- 172.21.15.27:0/335800684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36bc0f6ff0 0x7f36bc0f7a20 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 rx=0x7f36b0004660 tx=0x7f36b0009b30).stop 2021-11-03T18:10:08.436 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.437+0000 7f36c35a3700 1 -- 172.21.15.27:0/335800684 shutdown_connections 2021-11-03T18:10:08.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.437+0000 7f36c35a3700 1 --2- 172.21.15.27:0/335800684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36bc0f6ff0 0x7f36bc0f7a20 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:08.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.437+0000 7f36c35a3700 1 --2- 172.21.15.27:0/335800684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36bc0f66e0 0x7f36bc0f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:08.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.437+0000 7f36c35a3700 1 -- 172.21.15.27:0/335800684 >> 172.21.15.27:0/335800684 conn(0x7f36bc0f1f00 msgr2=0x7f36bc0f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:10:08.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.437+0000 7f36c35a3700 1 -- 172.21.15.27:0/335800684 shutdown_connections 2021-11-03T18:10:08.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.437+0000 7f36c35a3700 1 -- 172.21.15.27:0/335800684 wait complete. 2021-11-03T18:10:08.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.437+0000 7f36c35a3700 1 Processor -- start 2021-11-03T18:10:08.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.437+0000 7f36c35a3700 1 -- start start 2021-11-03T18:10:08.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.438+0000 7f36c35a3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36bc0f66e0 0x7f36bc06b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:10:08.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.438+0000 7f36c35a3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36bc0f6ff0 0x7f36bc069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:10:08.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.438+0000 7f36c35a3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36bc06a380 con 0x7f36bc0f66e0 2021-11-03T18:10:08.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.438+0000 7f36c35a3700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36bc06a4c0 con 0x7f36bc0f6ff0 2021-11-03T18:10:08.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.438+0000 7f36c133f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36bc0f66e0 0x7f36bc06b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:10:08.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.438+0000 7f36c133f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36bc0f66e0 0x7f36bc06b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33338/0 (socket says 172.21.15.27:33338) 2021-11-03T18:10:08.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.438+0000 7f36c133f700 1 -- 172.21.15.27:0/438652210 learned_addr learned my addr 172.21.15.27:0/438652210 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:10:08.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.438+0000 7f36c0b3e700 1 --2- 172.21.15.27:0/438652210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36bc0f6ff0 0x7f36bc069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:10:08.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.438+0000 7f36c133f700 1 -- 172.21.15.27:0/438652210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36bc0f6ff0 msgr2=0x7f36bc069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:10:08.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.438+0000 7f36c133f700 1 --2- 172.21.15.27:0/438652210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36bc0f6ff0 0x7f36bc069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:08.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.438+0000 7f36c133f700 1 -- 172.21.15.27:0/438652210 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b0005040 con 0x7f36bc0f66e0 2021-11-03T18:10:08.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.438+0000 7f36c0b3e700 1 --2- 172.21.15.27:0/438652210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36bc0f6ff0 0x7f36bc069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:10:08.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.438+0000 7f36c133f700 1 --2- 172.21.15.27:0/438652210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36bc0f66e0 0x7f36bc06b9a0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 rx=0x7f36b400c530 tx=0x7f36b4007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:10:08.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.438+0000 7f36b9ffb700 1 -- 172.21.15.27:0/438652210 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36b400f650 con 0x7f36bc0f66e0 2021-11-03T18:10:08.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.438+0000 7f36b9ffb700 1 -- 172.21.15.27:0/438652210 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f36b400fc60 con 0x7f36bc0f66e0 2021-11-03T18:10:08.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.438+0000 7f36b9ffb700 1 -- 172.21.15.27:0/438652210 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36b4012cd0 con 0x7f36bc0f66e0 2021-11-03T18:10:08.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.438+0000 7f36c35a3700 1 -- 172.21.15.27:0/438652210 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36bc06a7a0 con 0x7f36bc0f66e0 2021-11-03T18:10:08.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.439+0000 7f36c35a3700 1 -- 172.21.15.27:0/438652210 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36bc10efa0 con 0x7f36bc0f66e0 2021-11-03T18:10:08.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.439+0000 7f36c35a3700 1 -- 172.21.15.27:0/438652210 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36bc045ad0 con 0x7f36bc0f66e0 2021-11-03T18:10:08.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.440+0000 7f36b9ffb700 1 -- 172.21.15.27:0/438652210 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f36b4010070 con 0x7f36bc0f66e0 2021-11-03T18:10:08.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.440+0000 7f36b9ffb700 1 --2- 172.21.15.27:0/438652210 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f36a40640b0 0x7f36a4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:10:08.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.440+0000 7f36b9ffb700 1 -- 172.21.15.27:0/438652210 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f36b4080680 con 0x7f36bc0f66e0 2021-11-03T18:10:08.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.440+0000 7f36c0b3e700 1 --2- 172.21.15.27:0/438652210 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f36a40640b0 0x7f36a4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:10:08.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.441+0000 7f36c0b3e700 1 --2- 172.21.15.27:0/438652210 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f36a40640b0 0x7f36a4066570 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f36b0004e30 tx=0x7f36b0009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:10:08.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.444+0000 7f36b9ffb700 1 -- 172.21.15.27:0/438652210 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f36b4052890 con 0x7f36bc0f66e0 2021-11-03T18:10:08.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:08 smithi115 ceph-mon[65453]: pgmap v1664: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:08.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:08 smithi027 ceph-mon[97244]: pgmap v1664: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:08.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.674+0000 7f36c35a3700 1 -- 172.21.15.27:0/438652210 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f36bc0fef70 con 0x7f36bc0f66e0 2021-11-03T18:10:08.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.675+0000 7f36b9ffb700 1 -- 172.21.15.27:0/438652210 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f36b4052480 con 0x7f36bc0f66e0 2021-11-03T18:10:08.674 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:10:08.674 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:10:08.674 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:10:08.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:10:08.675 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:10:08.675 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:10:08.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:10:08.675 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:10:08.676 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:10:08.676 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:10:08.676 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:10:08.676 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:10:08.677 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:10:08.677 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:10:08.677 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:10:08.677 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:10:08.678 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:10:08.678 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:10:08.679 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.677+0000 7f36c35a3700 1 -- 172.21.15.27:0/438652210 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f36a40640b0 msgr2=0x7f36a4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:10:08.679 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.677+0000 7f36c35a3700 1 --2- 172.21.15.27:0/438652210 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f36a40640b0 0x7f36a4066570 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f36b0004e30 tx=0x7f36b0009b30).stop 2021-11-03T18:10:08.679 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.677+0000 7f36c35a3700 1 -- 172.21.15.27:0/438652210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36bc0f66e0 msgr2=0x7f36bc06b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:10:08.679 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.677+0000 7f36c35a3700 1 --2- 172.21.15.27:0/438652210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36bc0f66e0 0x7f36bc06b9a0 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 rx=0x7f36b400c530 tx=0x7f36b4007e20).stop 2021-11-03T18:10:08.680 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.677+0000 7f36c35a3700 1 -- 172.21.15.27:0/438652210 shutdown_connections 2021-11-03T18:10:08.680 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.677+0000 7f36c35a3700 1 --2- 172.21.15.27:0/438652210 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f36a40640b0 0x7f36a4066570 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:08.680 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.677+0000 7f36c35a3700 1 --2- 172.21.15.27:0/438652210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36bc0f66e0 0x7f36bc06b9a0 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:08.680 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.677+0000 7f36c35a3700 1 --2- 172.21.15.27:0/438652210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36bc0f6ff0 0x7f36bc069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:08.681 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.677+0000 7f36c35a3700 1 -- 172.21.15.27:0/438652210 >> 172.21.15.27:0/438652210 conn(0x7f36bc0f1f00 msgr2=0x7f36bc0fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:10:08.681 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.677+0000 7f36c35a3700 1 -- 172.21.15.27:0/438652210 shutdown_connections 2021-11-03T18:10:08.681 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.677+0000 7f36c35a3700 1 -- 172.21.15.27:0/438652210 wait complete. 2021-11-03T18:10:08.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.777+0000 7fcfc10d7700 1 -- 172.21.15.27:0/889645808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfbc0f5440 msgr2=0x7fcfbc0f5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:10:08.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.777+0000 7fcfc10d7700 1 --2- 172.21.15.27:0/889645808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfbc0f5440 0x7fcfbc0f5860 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 rx=0x7fcfac004660 tx=0x7fcfac009b30).stop 2021-11-03T18:10:08.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.777+0000 7fcfc10d7700 1 -- 172.21.15.27:0/889645808 shutdown_connections 2021-11-03T18:10:08.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.777+0000 7fcfc10d7700 1 --2- 172.21.15.27:0/889645808 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfbc0f66e0 0x7fcfbc0f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:08.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.777+0000 7fcfc10d7700 1 --2- 172.21.15.27:0/889645808 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfbc0f5440 0x7fcfbc0f5860 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:08.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.777+0000 7fcfc10d7700 1 -- 172.21.15.27:0/889645808 >> 172.21.15.27:0/889645808 conn(0x7fcfbc0f0ba0 msgr2=0x7fcfbc0f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:10:08.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.777+0000 7fcfc10d7700 1 -- 172.21.15.27:0/889645808 shutdown_connections 2021-11-03T18:10:08.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.777+0000 7fcfc10d7700 1 -- 172.21.15.27:0/889645808 wait complete. 2021-11-03T18:10:08.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.778+0000 7fcfc10d7700 1 Processor -- start 2021-11-03T18:10:08.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.778+0000 7fcfc10d7700 1 -- start start 2021-11-03T18:10:08.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.778+0000 7fcfc10d7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfbc0f5440 0x7fcfbc109aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:10:08.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.778+0000 7fcfc10d7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfbc0f66e0 0x7fcfbc10bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:10:08.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.778+0000 7fcfc10d7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc0603e0 con 0x7fcfbc0f66e0 2021-11-03T18:10:08.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.778+0000 7fcfc10d7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfbc060520 con 0x7fcfbc0f5440 2021-11-03T18:10:08.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.778+0000 7fcfbb7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfbc0f66e0 0x7fcfbc10bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:10:08.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.778+0000 7fcfbb7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfbc0f66e0 0x7fcfbc10bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33350/0 (socket says 172.21.15.27:33350) 2021-11-03T18:10:08.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.778+0000 7fcfbb7fe700 1 -- 172.21.15.27:0/691315184 learned_addr learned my addr 172.21.15.27:0/691315184 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:10:08.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.778+0000 7fcfbbfff700 1 --2- 172.21.15.27:0/691315184 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfbc0f5440 0x7fcfbc109aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:10:08.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.779+0000 7fcfbb7fe700 1 -- 172.21.15.27:0/691315184 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfbc0f5440 msgr2=0x7fcfbc109aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:10:08.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.779+0000 7fcfbb7fe700 1 --2- 172.21.15.27:0/691315184 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfbc0f5440 0x7fcfbc109aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:08.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.779+0000 7fcfbb7fe700 1 -- 172.21.15.27:0/691315184 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfac005040 con 0x7fcfbc0f66e0 2021-11-03T18:10:08.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.779+0000 7fcfbb7fe700 1 --2- 172.21.15.27:0/691315184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfbc0f66e0 0x7fcfbc10bfa0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 rx=0x7fcfb000c790 tx=0x7fcfb0007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:10:08.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.779+0000 7fcfb8ff9700 1 -- 172.21.15.27:0/691315184 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfb0002590 con 0x7fcfbc0f66e0 2021-11-03T18:10:08.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.779+0000 7fcfb8ff9700 1 -- 172.21.15.27:0/691315184 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fcfb0002ba0 con 0x7fcfbc0f66e0 2021-11-03T18:10:08.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.779+0000 7fcfc10d7700 1 -- 172.21.15.27:0/691315184 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfbc10c550 con 0x7fcfbc0f66e0 2021-11-03T18:10:08.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.779+0000 7fcfb8ff9700 1 -- 172.21.15.27:0/691315184 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfb0013d90 con 0x7fcfbc0f66e0 2021-11-03T18:10:08.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.779+0000 7fcfc10d7700 1 -- 172.21.15.27:0/691315184 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfbc10cb10 con 0x7fcfbc0f66e0 2021-11-03T18:10:08.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.780+0000 7fcfc10d7700 1 -- 172.21.15.27:0/691315184 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfbc045ad0 con 0x7fcfbc0f66e0 2021-11-03T18:10:08.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.780+0000 7fcfb8ff9700 1 -- 172.21.15.27:0/691315184 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fcfb001a060 con 0x7fcfbc0f66e0 2021-11-03T18:10:08.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.781+0000 7fcfb8ff9700 1 --2- 172.21.15.27:0/691315184 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcfa4064100 0x7fcfa40665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:10:08.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.781+0000 7fcfb8ff9700 1 -- 172.21.15.27:0/691315184 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fcfb0080b10 con 0x7fcfbc0f66e0 2021-11-03T18:10:08.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.781+0000 7fcfbbfff700 1 --2- 172.21.15.27:0/691315184 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcfa4064100 0x7fcfa40665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:10:08.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.781+0000 7fcfbbfff700 1 --2- 172.21.15.27:0/691315184 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcfa4064100 0x7fcfa40665c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7fcfac013690 tx=0x7fcfac009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:10:08.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.785+0000 7fcfb8ff9700 1 -- 172.21.15.27:0/691315184 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fcfb0052d20 con 0x7fcfbc0f66e0 2021-11-03T18:10:08.987 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.989+0000 7fcfc10d7700 1 -- 172.21.15.27:0/691315184 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fcfbc0fdc00 con 0x7fcfbc0f66e0 2021-11-03T18:10:08.987 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.989+0000 7fcfb8ff9700 1 -- 172.21.15.27:0/691315184 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7fcfb0018020 con 0x7fcfbc0f66e0 2021-11-03T18:10:08.988 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:10:08.989 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:10:08.989 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:10:08.989 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:10:08.990 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:10:08.990 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:10:08.990 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:10:08.990 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:10:08.991 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:10:08.991 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:10:08.991 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:10:08.992 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:10:08.992 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:10:08.992 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:10:08.992 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:10:08.993 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:10:08.993 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:10:08.994 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:10:08.994 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:10:08.994 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:10:08.994 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:10:08.995 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:10:08.995 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:10:08.995 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:10:08.995 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:10:08.995 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:10:08.996 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:10:08.996 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:10:08.996 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:10:08.996 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:10:08.997 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:10:08.997 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:10:08.997 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:10:08.997 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:10:08.998 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:10:08.998 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:10:08.999 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:10:08.999 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:10:09.000 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:10:09.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.991+0000 7fcfc10d7700 1 -- 172.21.15.27:0/691315184 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcfa4064100 msgr2=0x7fcfa40665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:10:09.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.991+0000 7fcfc10d7700 1 --2- 172.21.15.27:0/691315184 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcfa4064100 0x7fcfa40665c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7fcfac013690 tx=0x7fcfac009b30).stop 2021-11-03T18:10:09.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.991+0000 7fcfc10d7700 1 -- 172.21.15.27:0/691315184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfbc0f66e0 msgr2=0x7fcfbc10bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:10:09.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.991+0000 7fcfc10d7700 1 --2- 172.21.15.27:0/691315184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfbc0f66e0 0x7fcfbc10bfa0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 rx=0x7fcfb000c790 tx=0x7fcfb0007d70).stop 2021-11-03T18:10:09.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.991+0000 7fcfc10d7700 1 -- 172.21.15.27:0/691315184 shutdown_connections 2021-11-03T18:10:09.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.991+0000 7fcfc10d7700 1 --2- 172.21.15.27:0/691315184 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcfa4064100 0x7fcfa40665c0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:09.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.991+0000 7fcfc10d7700 1 --2- 172.21.15.27:0/691315184 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfbc0f5440 0x7fcfbc109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:09.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.991+0000 7fcfc10d7700 1 --2- 172.21.15.27:0/691315184 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfbc0f66e0 0x7fcfbc10bfa0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:09.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.991+0000 7fcfc10d7700 1 -- 172.21.15.27:0/691315184 >> 172.21.15.27:0/691315184 conn(0x7fcfbc0f0ba0 msgr2=0x7fcfbc0fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:10:09.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.991+0000 7fcfc10d7700 1 -- 172.21.15.27:0/691315184 shutdown_connections 2021-11-03T18:10:09.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:08.991+0000 7fcfc10d7700 1 -- 172.21.15.27:0/691315184 wait complete. 2021-11-03T18:10:09.004 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:10:09.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:09 smithi115 ceph-mon[65453]: from='client.35816 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:10:09.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:09 smithi115 ceph-mon[65453]: from='client.35820 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:10:09.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:09 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/438652210' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:09.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:09 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/691315184' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:10:09.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:09 smithi027 ceph-mon[97244]: from='client.35816 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:10:09.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:09 smithi027 ceph-mon[97244]: from='client.35820 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:10:09.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:09 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/438652210' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:09.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:09 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/691315184' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:10:10.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:10 smithi115 ceph-mon[65453]: pgmap v1665: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:10.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3252.437538 secs 2021-11-03T18:10:10.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:10 smithi027 ceph-mon[97244]: pgmap v1665: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:10.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3252.437538 secs 2021-11-03T18:10:12.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:12 smithi115 ceph-mon[65453]: pgmap v1666: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:12.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:12 smithi027 ceph-mon[97244]: pgmap v1666: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:14.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:14 smithi115 ceph-mon[65453]: pgmap v1667: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:14.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:10:14.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:10:14.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:10:14.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:14.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:14.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:14.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:14.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:14 smithi027 ceph-mon[97244]: pgmap v1667: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:14.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:10:14.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:10:14.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:10:14.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:14.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:14.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:14.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:15.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:15 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:10:15.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3257.437682 secs 2021-11-03T18:10:15.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:15 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:10:15.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3257.437682 secs 2021-11-03T18:10:16.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:16 smithi115 ceph-mon[65453]: pgmap v1668: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:16.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:16 smithi027 ceph-mon[97244]: pgmap v1668: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:18.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:18 smithi027 ceph-mon[97244]: pgmap v1669: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:18.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:18 smithi115 ceph-mon[65453]: pgmap v1669: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:20.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:20 smithi027 ceph-mon[97244]: pgmap v1670: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:20.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3262.437798 secs 2021-11-03T18:10:20.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:20 smithi115 ceph-mon[65453]: pgmap v1670: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:20.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3262.437798 secs 2021-11-03T18:10:22.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:22 smithi027 ceph-mon[97244]: pgmap v1671: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:22.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:22 smithi115 ceph-mon[65453]: pgmap v1671: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:24.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:24 smithi027 ceph-mon[97244]: pgmap v1672: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:24.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:24 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:10:24.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:24 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:10:24.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:24 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:10:24.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:24 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:24.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:24 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:24.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:24 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:24.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:24 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:24.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:24 smithi115 ceph-mon[65453]: pgmap v1672: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:24.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:24 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:10:24.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:24 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:10:24.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:24 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:10:24.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:24 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:24.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:24 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:24.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:24 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:24.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:24 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:25.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:25 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:10:25.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3267.437901 secs 2021-11-03T18:10:25.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:25 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:10:25.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3267.437901 secs 2021-11-03T18:10:26.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:26 smithi027 ceph-mon[97244]: pgmap v1673: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:26.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:26 smithi115 ceph-mon[65453]: pgmap v1673: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:28.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:28 smithi027 ceph-mon[97244]: pgmap v1674: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:28.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:28 smithi115 ceph-mon[65453]: pgmap v1674: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:30.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:30 smithi027 ceph-mon[97244]: pgmap v1675: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:30.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3272.438000 secs 2021-11-03T18:10:30.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:30 smithi115 ceph-mon[65453]: pgmap v1675: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:30.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3272.438000 secs 2021-11-03T18:10:32.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:32 smithi027 ceph-mon[97244]: pgmap v1676: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:32.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:32 smithi115 ceph-mon[65453]: pgmap v1676: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:34.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:34 smithi027 ceph-mon[97244]: pgmap v1677: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:34.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:10:34.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:10:34.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:10:34.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:34.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:34.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:34.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:34.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:34 smithi115 ceph-mon[65453]: pgmap v1677: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:34.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:10:34.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:10:34.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:10:34.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:34.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:34.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:34.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:35.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:35 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:10:35.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3277.438151 secs 2021-11-03T18:10:35.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:35 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:10:35.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3277.438151 secs 2021-11-03T18:10:36.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:36 smithi115 ceph-mon[65453]: pgmap v1678: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:36.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:36 smithi027 ceph-mon[97244]: pgmap v1678: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:38.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:38 smithi115 ceph-mon[65453]: pgmap v1679: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:38.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:38 smithi027 ceph-mon[97244]: pgmap v1679: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:39.104 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.106+0000 7f7ff9408700 1 -- 172.21.15.27:0/3145781350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ff40f6860 msgr2=0x7f7ff40f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:10:39.104 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.106+0000 7f7ff9408700 1 --2- 172.21.15.27:0/3145781350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ff40f6860 0x7f7ff40f6cc0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 rx=0x7f7fe4004660 tx=0x7f7fe4009b30).stop 2021-11-03T18:10:39.104 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.106+0000 7f7ff9408700 1 -- 172.21.15.27:0/3145781350 shutdown_connections 2021-11-03T18:10:39.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.106+0000 7f7ff9408700 1 --2- 172.21.15.27:0/3145781350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ff40f6860 0x7f7ff40f6cc0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:39.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.106+0000 7f7ff9408700 1 --2- 172.21.15.27:0/3145781350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ff40f55c0 0x7f7ff40f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:39.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.106+0000 7f7ff9408700 1 -- 172.21.15.27:0/3145781350 >> 172.21.15.27:0/3145781350 conn(0x7f7ff40f0d20 msgr2=0x7f7ff40f3140 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:10:39.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.106+0000 7f7ff9408700 1 -- 172.21.15.27:0/3145781350 shutdown_connections 2021-11-03T18:10:39.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.106+0000 7f7ff9408700 1 -- 172.21.15.27:0/3145781350 wait complete. 2021-11-03T18:10:39.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.107+0000 7f7ff9408700 1 Processor -- start 2021-11-03T18:10:39.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.107+0000 7f7ff9408700 1 -- start start 2021-11-03T18:10:39.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.107+0000 7f7ff9408700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ff40f55c0 0x7f7ff4105e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:10:39.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.107+0000 7f7ff9408700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ff40f6860 0x7f7ff4106320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:10:39.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.107+0000 7f7ff9408700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff4106920 con 0x7f7ff40f6860 2021-11-03T18:10:39.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.107+0000 7f7ff9408700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ff4107130 con 0x7f7ff40f55c0 2021-11-03T18:10:39.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.107+0000 7f7ff27fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ff40f6860 0x7f7ff4106320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:10:39.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.107+0000 7f7ff27fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ff40f6860 0x7f7ff4106320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33368/0 (socket says 172.21.15.27:33368) 2021-11-03T18:10:39.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.107+0000 7f7ff27fc700 1 -- 172.21.15.27:0/4269722896 learned_addr learned my addr 172.21.15.27:0/4269722896 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:10:39.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.107+0000 7f7ff2ffd700 1 --2- 172.21.15.27:0/4269722896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ff40f55c0 0x7f7ff4105e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:10:39.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.107+0000 7f7ff27fc700 1 -- 172.21.15.27:0/4269722896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ff40f55c0 msgr2=0x7f7ff4105e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:10:39.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.107+0000 7f7ff27fc700 1 --2- 172.21.15.27:0/4269722896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ff40f55c0 0x7f7ff4105e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:39.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.107+0000 7f7ff27fc700 1 -- 172.21.15.27:0/4269722896 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe4005040 con 0x7f7ff40f6860 2021-11-03T18:10:39.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.108+0000 7f7ff27fc700 1 --2- 172.21.15.27:0/4269722896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ff40f6860 0x7f7ff4106320 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 rx=0x7f7fe4000c00 tx=0x7f7fe400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:10:39.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.108+0000 7f7fdb7fe700 1 -- 172.21.15.27:0/4269722896 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fe4019070 con 0x7f7ff40f6860 2021-11-03T18:10:39.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.108+0000 7f7fdb7fe700 1 -- 172.21.15.27:0/4269722896 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f7fe4007e70 con 0x7f7ff40f6860 2021-11-03T18:10:39.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.108+0000 7f7ff9408700 1 -- 172.21.15.27:0/4269722896 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ff4107350 con 0x7f7ff40f6860 2021-11-03T18:10:39.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.108+0000 7f7fdb7fe700 1 -- 172.21.15.27:0/4269722896 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7fe4014ce0 con 0x7f7ff40f6860 2021-11-03T18:10:39.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.108+0000 7f7ff9408700 1 -- 172.21.15.27:0/4269722896 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ff41078b0 con 0x7f7ff40f6860 2021-11-03T18:10:39.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.109+0000 7f7ff9408700 1 -- 172.21.15.27:0/4269722896 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ff40f49b0 con 0x7f7ff40f6860 2021-11-03T18:10:39.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.109+0000 7f7fdb7fe700 1 -- 172.21.15.27:0/4269722896 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f7fe4025070 con 0x7f7ff40f6860 2021-11-03T18:10:39.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.110+0000 7f7fdb7fe700 1 --2- 172.21.15.27:0/4269722896 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7fdc0640b0 0x7f7fdc066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:10:39.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.110+0000 7f7fdb7fe700 1 -- 172.21.15.27:0/4269722896 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f7fe40816b0 con 0x7f7ff40f6860 2021-11-03T18:10:39.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.110+0000 7f7ff2ffd700 1 --2- 172.21.15.27:0/4269722896 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7fdc0640b0 0x7f7fdc066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:10:39.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.110+0000 7f7ff2ffd700 1 --2- 172.21.15.27:0/4269722896 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7fdc0640b0 0x7f7fdc066570 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f7ff4105ba0 tx=0x7f7fe800b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:10:39.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.114+0000 7f7fdb7fe700 1 -- 172.21.15.27:0/4269722896 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f7fe40539b0 con 0x7f7ff40f6860 2021-11-03T18:10:39.290 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.293+0000 7f7ff9408700 1 -- 172.21.15.27:0/4269722896 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7ff40fdef0 con 0x7f7fdc0640b0 2021-11-03T18:10:39.291 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.294+0000 7f7fdb7fe700 1 -- 172.21.15.27:0/4269722896 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f7ff40fdef0 con 0x7f7fdc0640b0 2021-11-03T18:10:39.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.296+0000 7f7ff9408700 1 -- 172.21.15.27:0/4269722896 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7fdc0640b0 msgr2=0x7f7fdc066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:10:39.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.296+0000 7f7ff9408700 1 --2- 172.21.15.27:0/4269722896 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7fdc0640b0 0x7f7fdc066570 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f7ff4105ba0 tx=0x7f7fe800b040).stop 2021-11-03T18:10:39.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.296+0000 7f7ff9408700 1 -- 172.21.15.27:0/4269722896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ff40f6860 msgr2=0x7f7ff4106320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:10:39.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.296+0000 7f7ff9408700 1 --2- 172.21.15.27:0/4269722896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ff40f6860 0x7f7ff4106320 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 rx=0x7f7fe4000c00 tx=0x7f7fe400b040).stop 2021-11-03T18:10:39.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.296+0000 7f7ff9408700 1 -- 172.21.15.27:0/4269722896 shutdown_connections 2021-11-03T18:10:39.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.296+0000 7f7ff9408700 1 --2- 172.21.15.27:0/4269722896 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7fdc0640b0 0x7f7fdc066570 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:39.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.296+0000 7f7ff9408700 1 --2- 172.21.15.27:0/4269722896 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7ff40f55c0 0x7f7ff4105e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:39.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.296+0000 7f7ff9408700 1 --2- 172.21.15.27:0/4269722896 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7ff40f6860 0x7f7ff4106320 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:39.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.296+0000 7f7ff9408700 1 -- 172.21.15.27:0/4269722896 >> 172.21.15.27:0/4269722896 conn(0x7f7ff40f0d20 msgr2=0x7f7ff40fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:10:39.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.296+0000 7f7ff9408700 1 -- 172.21.15.27:0/4269722896 shutdown_connections 2021-11-03T18:10:39.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.296+0000 7f7ff9408700 1 -- 172.21.15.27:0/4269722896 wait complete. 2021-11-03T18:10:39.303 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:10:39.397 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.396+0000 7f8c9d26e700 1 -- 172.21.15.27:0/3283347611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c980f77d0 msgr2=0x7f8c980f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:10:39.397 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.396+0000 7f8c9d26e700 1 --2- 172.21.15.27:0/3283347611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c980f77d0 0x7f8c980f7c30 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 rx=0x7f8c8c004660 tx=0x7f8c8c009b30).stop 2021-11-03T18:10:39.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.396+0000 7f8c9d26e700 1 -- 172.21.15.27:0/3283347611 shutdown_connections 2021-11-03T18:10:39.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.396+0000 7f8c9d26e700 1 --2- 172.21.15.27:0/3283347611 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c980f77d0 0x7f8c980f7c30 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:39.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.396+0000 7f8c9d26e700 1 --2- 172.21.15.27:0/3283347611 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c980f6530 0x7f8c980f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:39.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.396+0000 7f8c9d26e700 1 -- 172.21.15.27:0/3283347611 >> 172.21.15.27:0/3283347611 conn(0x7f8c980f1cd0 msgr2=0x7f8c980f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:10:39.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.396+0000 7f8c9d26e700 1 -- 172.21.15.27:0/3283347611 shutdown_connections 2021-11-03T18:10:39.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.396+0000 7f8c9d26e700 1 -- 172.21.15.27:0/3283347611 wait complete. 2021-11-03T18:10:39.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.397+0000 7f8c9d26e700 1 Processor -- start 2021-11-03T18:10:39.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.397+0000 7f8c9d26e700 1 -- start start 2021-11-03T18:10:39.400 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.397+0000 7f8c9d26e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c980f6530 0x7f8c9806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:10:39.400 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.397+0000 7f8c9d26e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c980f77d0 0x7f8c98069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:10:39.400 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.397+0000 7f8c9d26e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9806a380 con 0x7f8c980f6530 2021-11-03T18:10:39.400 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.397+0000 7f8c9d26e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c9806a4c0 con 0x7f8c980f77d0 2021-11-03T18:10:39.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.397+0000 7f8c97fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c980f6530 0x7f8c9806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:10:39.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.397+0000 7f8c97fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c980f6530 0x7f8c9806b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33376/0 (socket says 172.21.15.27:33376) 2021-11-03T18:10:39.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.397+0000 7f8c97fff700 1 -- 172.21.15.27:0/426727274 learned_addr learned my addr 172.21.15.27:0/426727274 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:10:39.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.398+0000 7f8c977fe700 1 --2- 172.21.15.27:0/426727274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c980f77d0 0x7f8c98069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:10:39.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.398+0000 7f8c97fff700 1 -- 172.21.15.27:0/426727274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c980f77d0 msgr2=0x7f8c98069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:10:39.402 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.398+0000 7f8c97fff700 1 --2- 172.21.15.27:0/426727274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c980f77d0 0x7f8c98069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:39.402 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.398+0000 7f8c97fff700 1 -- 172.21.15.27:0/426727274 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c8c005040 con 0x7f8c980f6530 2021-11-03T18:10:39.402 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.398+0000 7f8c97fff700 1 --2- 172.21.15.27:0/426727274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c980f6530 0x7f8c9806b9a0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 rx=0x7f8c88002700 tx=0x7f8c88009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:10:39.402 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.398+0000 7f8c94ff9700 1 -- 172.21.15.27:0/426727274 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c880079c0 con 0x7f8c980f6530 2021-11-03T18:10:39.403 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.398+0000 7f8c9d26e700 1 -- 172.21.15.27:0/426727274 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c9806a7a0 con 0x7f8c980f6530 2021-11-03T18:10:39.403 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.398+0000 7f8c94ff9700 1 -- 172.21.15.27:0/426727274 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f8c8800f040 con 0x7f8c980f6530 2021-11-03T18:10:39.403 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.398+0000 7f8c94ff9700 1 -- 172.21.15.27:0/426727274 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c88014860 con 0x7f8c980f6530 2021-11-03T18:10:39.404 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.398+0000 7f8c9d26e700 1 -- 172.21.15.27:0/426727274 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c9810ed40 con 0x7f8c980f6530 2021-11-03T18:10:39.404 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.399+0000 7f8c9d26e700 1 -- 172.21.15.27:0/426727274 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c98045ad0 con 0x7f8c980f6530 2021-11-03T18:10:39.404 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.399+0000 7f8c94ff9700 1 -- 172.21.15.27:0/426727274 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f8c8801b060 con 0x7f8c980f6530 2021-11-03T18:10:39.404 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.400+0000 7f8c94ff9700 1 --2- 172.21.15.27:0/426727274 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8c80064100 0x7f8c800665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:10:39.405 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.400+0000 7f8c94ff9700 1 -- 172.21.15.27:0/426727274 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f8c880806d0 con 0x7f8c980f6530 2021-11-03T18:10:39.405 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.400+0000 7f8c977fe700 1 --2- 172.21.15.27:0/426727274 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8c80064100 0x7f8c800665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:10:39.406 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.400+0000 7f8c977fe700 1 --2- 172.21.15.27:0/426727274 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8c80064100 0x7f8c800665c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7f8c8c004e30 tx=0x7f8c8c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:10:39.406 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.404+0000 7f8c94ff9700 1 -- 172.21.15.27:0/426727274 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f8c880529d0 con 0x7f8c980f6530 2021-11-03T18:10:39.576 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.577+0000 7f8c9d26e700 1 -- 172.21.15.27:0/426727274 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c980feea0 con 0x7f8c80064100 2021-11-03T18:10:39.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.592+0000 7f8c94ff9700 1 -- 172.21.15.27:0/426727274 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f8c8c019040 con 0x7f8c80064100 2021-11-03T18:10:39.590 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:10:39.590 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (62m) 82s ago 63m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:10:39.590 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (53m) 82s ago 63m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:10:39.590 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (53m) 36s ago 62m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:10:39.591 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (62m) 82s ago 63m 42.1M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:10:39.591 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (55m) 82s ago 58m 26.3M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:10:39.591 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (58m) 82s ago 58m 3847M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:10:39.592 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (54m) 36s ago 58m 25.9M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:10:39.592 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (56m) 36s ago 58m 27.0M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:10:39.592 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (55m) 82s ago 64m 572M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:10:39.592 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (55m) 36s ago 62m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:10:39.593 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (54m) 82s ago 64m 273M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:10:39.593 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (54m) 36s ago 62m 129M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:10:39.593 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (63m) 82s ago 63m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:10:39.593 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (62m) 36s ago 62m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:10:39.593 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (53m) 82s ago 61m 371M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:10:39.594 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (53m) 82s ago 61m 405M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:10:39.594 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (52m) 82s ago 61m 302M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:10:39.594 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (52m) 36s ago 60m 429M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:10:39.594 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (51m) 36s ago 60m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:10:39.595 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (51m) 36s ago 60m 348M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:10:39.595 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (61m) 82s ago 62m 57.6M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:10:39.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.594+0000 7f8c9d26e700 1 -- 172.21.15.27:0/426727274 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8c80064100 msgr2=0x7f8c800665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:10:39.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.594+0000 7f8c9d26e700 1 --2- 172.21.15.27:0/426727274 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8c80064100 0x7f8c800665c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 rx=0x7f8c8c004e30 tx=0x7f8c8c009b30).stop 2021-11-03T18:10:39.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.594+0000 7f8c9d26e700 1 -- 172.21.15.27:0/426727274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c980f6530 msgr2=0x7f8c9806b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:10:39.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.594+0000 7f8c9d26e700 1 --2- 172.21.15.27:0/426727274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c980f6530 0x7f8c9806b9a0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 rx=0x7f8c88002700 tx=0x7f8c88009d80).stop 2021-11-03T18:10:39.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.594+0000 7f8c9d26e700 1 -- 172.21.15.27:0/426727274 shutdown_connections 2021-11-03T18:10:39.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.594+0000 7f8c9d26e700 1 --2- 172.21.15.27:0/426727274 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8c80064100 0x7f8c800665c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:39.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.594+0000 7f8c9d26e700 1 --2- 172.21.15.27:0/426727274 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8c980f6530 0x7f8c9806b9a0 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:39.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.594+0000 7f8c9d26e700 1 --2- 172.21.15.27:0/426727274 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8c980f77d0 0x7f8c98069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:39.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.594+0000 7f8c9d26e700 1 -- 172.21.15.27:0/426727274 >> 172.21.15.27:0/426727274 conn(0x7f8c980f1cd0 msgr2=0x7f8c980fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:10:39.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.594+0000 7f8c9d26e700 1 -- 172.21.15.27:0/426727274 shutdown_connections 2021-11-03T18:10:39.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.594+0000 7f8c9d26e700 1 -- 172.21.15.27:0/426727274 wait complete. 2021-11-03T18:10:39.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.693+0000 7f7c3b59e700 1 -- 172.21.15.27:0/2105444371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c3c0f96a0 msgr2=0x7f7c3c0fbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:10:39.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.693+0000 7f7c3b59e700 1 --2- 172.21.15.27:0/2105444371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c3c0f96a0 0x7f7c3c0fbaf0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 rx=0x7f7c30004660 tx=0x7f7c30009b30).stop 2021-11-03T18:10:39.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.693+0000 7f7c3b59e700 1 -- 172.21.15.27:0/2105444371 shutdown_connections 2021-11-03T18:10:39.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.693+0000 7f7c3b59e700 1 --2- 172.21.15.27:0/2105444371 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c3c0f96a0 0x7f7c3c0fbaf0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:39.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.693+0000 7f7c3b59e700 1 --2- 172.21.15.27:0/2105444371 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c3c060310 0x7f7c3c0f9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:39.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.693+0000 7f7c3b59e700 1 -- 172.21.15.27:0/2105444371 >> 172.21.15.27:0/2105444371 conn(0x7f7c3c0f0ba0 msgr2=0x7f7c3c0f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:10:39.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.693+0000 7f7c3b59e700 1 -- 172.21.15.27:0/2105444371 shutdown_connections 2021-11-03T18:10:39.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.693+0000 7f7c3b59e700 1 -- 172.21.15.27:0/2105444371 wait complete. 2021-11-03T18:10:39.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.694+0000 7f7c3b59e700 1 Processor -- start 2021-11-03T18:10:39.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.694+0000 7f7c3b59e700 1 -- start start 2021-11-03T18:10:39.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.694+0000 7f7c3b59e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c3c060310 0x7f7c3c106d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:10:39.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.694+0000 7f7c3b59e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c3c0f96a0 0x7f7c3c107280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:10:39.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.694+0000 7f7c3b59e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3c107880 con 0x7f7c3c0f96a0 2021-11-03T18:10:39.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.694+0000 7f7c3b59e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3c1080c0 con 0x7f7c3c060310 2021-11-03T18:10:39.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.694+0000 7f7c39d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c3c0f96a0 0x7f7c3c107280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:10:39.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.694+0000 7f7c39d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c3c0f96a0 0x7f7c3c107280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33388/0 (socket says 172.21.15.27:33388) 2021-11-03T18:10:39.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.694+0000 7f7c39d9b700 1 -- 172.21.15.27:0/99553288 learned_addr learned my addr 172.21.15.27:0/99553288 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:10:39.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.694+0000 7f7c3a59c700 1 --2- 172.21.15.27:0/99553288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c3c060310 0x7f7c3c106d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:10:39.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.695+0000 7f7c39d9b700 1 -- 172.21.15.27:0/99553288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c3c060310 msgr2=0x7f7c3c106d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:10:39.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.695+0000 7f7c39d9b700 1 --2- 172.21.15.27:0/99553288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c3c060310 0x7f7c3c106d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:39.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.695+0000 7f7c39d9b700 1 -- 172.21.15.27:0/99553288 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c30005040 con 0x7f7c3c0f96a0 2021-11-03T18:10:39.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.695+0000 7f7c3a59c700 1 --2- 172.21.15.27:0/99553288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c3c060310 0x7f7c3c106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:10:39.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.695+0000 7f7c39d9b700 1 --2- 172.21.15.27:0/99553288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c3c0f96a0 0x7f7c3c107280 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 rx=0x7f7c30013a70 tx=0x7f7c30009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:10:39.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.695+0000 7f7c2affd700 1 -- 172.21.15.27:0/99553288 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c30018070 con 0x7f7c3c0f96a0 2021-11-03T18:10:39.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.695+0000 7f7c2affd700 1 -- 172.21.15.27:0/99553288 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f7c300097d0 con 0x7f7c3c0f96a0 2021-11-03T18:10:39.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.695+0000 7f7c3b59e700 1 -- 172.21.15.27:0/99553288 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c3c108340 con 0x7f7c3c0f96a0 2021-11-03T18:10:39.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.695+0000 7f7c2affd700 1 -- 172.21.15.27:0/99553288 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c3001c3c0 con 0x7f7c3c0f96a0 2021-11-03T18:10:39.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.695+0000 7f7c3b59e700 1 -- 172.21.15.27:0/99553288 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c3c108820 con 0x7f7c3c0f96a0 2021-11-03T18:10:39.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.696+0000 7f7c3b59e700 1 -- 172.21.15.27:0/99553288 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c3c059100 con 0x7f7c3c0f96a0 2021-11-03T18:10:39.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.696+0000 7f7c2affd700 1 -- 172.21.15.27:0/99553288 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f7c30007ba0 con 0x7f7c3c0f96a0 2021-11-03T18:10:39.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.697+0000 7f7c2affd700 1 --2- 172.21.15.27:0/99553288 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7c24064100 0x7f7c240665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:10:39.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.697+0000 7f7c2affd700 1 -- 172.21.15.27:0/99553288 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f7c300824d0 con 0x7f7c3c0f96a0 2021-11-03T18:10:39.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.697+0000 7f7c3a59c700 1 --2- 172.21.15.27:0/99553288 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7c24064100 0x7f7c240665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:10:39.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.697+0000 7f7c3a59c700 1 --2- 172.21.15.27:0/99553288 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7c24064100 0x7f7c240665c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f7c3c107b50 tx=0x7f7c2c009c00).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:10:39.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.701+0000 7f7c2affd700 1 -- 172.21.15.27:0/99553288 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f7c300547d0 con 0x7f7c3c0f96a0 2021-11-03T18:10:39.931 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.933+0000 7f7c3b59e700 1 -- 172.21.15.27:0/99553288 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7c3c0fed00 con 0x7f7c3c0f96a0 2021-11-03T18:10:39.931 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.934+0000 7f7c2affd700 1 -- 172.21.15.27:0/99553288 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f7c300543c0 con 0x7f7c3c0f96a0 2021-11-03T18:10:39.932 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:10:39.932 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:10:39.932 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:10:39.933 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:10:39.933 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:10:39.933 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:10:39.933 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:10:39.934 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:10:39.934 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:10:39.934 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:10:39.934 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:10:39.935 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:10:39.935 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:10:39.935 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:10:39.935 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:10:39.935 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:10:39.936 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:10:39.936 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:10:39.937 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.936+0000 7f7c3b59e700 1 -- 172.21.15.27:0/99553288 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7c24064100 msgr2=0x7f7c240665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:10:39.937 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.936+0000 7f7c3b59e700 1 --2- 172.21.15.27:0/99553288 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7c24064100 0x7f7c240665c0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f7c3c107b50 tx=0x7f7c2c009c00).stop 2021-11-03T18:10:39.938 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.936+0000 7f7c3b59e700 1 -- 172.21.15.27:0/99553288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c3c0f96a0 msgr2=0x7f7c3c107280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:10:39.938 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.936+0000 7f7c3b59e700 1 --2- 172.21.15.27:0/99553288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c3c0f96a0 0x7f7c3c107280 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 rx=0x7f7c30013a70 tx=0x7f7c30009b30).stop 2021-11-03T18:10:39.938 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.936+0000 7f7c3b59e700 1 -- 172.21.15.27:0/99553288 shutdown_connections 2021-11-03T18:10:39.938 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.936+0000 7f7c3b59e700 1 --2- 172.21.15.27:0/99553288 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7c24064100 0x7f7c240665c0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:39.939 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.936+0000 7f7c3b59e700 1 --2- 172.21.15.27:0/99553288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7c3c060310 0x7f7c3c106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:39.939 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.936+0000 7f7c3b59e700 1 --2- 172.21.15.27:0/99553288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7c3c0f96a0 0x7f7c3c107280 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:39.939 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.936+0000 7f7c3b59e700 1 -- 172.21.15.27:0/99553288 >> 172.21.15.27:0/99553288 conn(0x7f7c3c0f0ba0 msgr2=0x7f7c3c0f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:10:39.939 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.936+0000 7f7c3b59e700 1 -- 172.21.15.27:0/99553288 shutdown_connections 2021-11-03T18:10:39.940 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:39.936+0000 7f7c3b59e700 1 -- 172.21.15.27:0/99553288 wait complete. 2021-11-03T18:10:40.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.036+0000 7f1f4d170700 1 -- 172.21.15.27:0/2911725378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f480f55c0 msgr2=0x7f1f480f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:10:40.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.036+0000 7f1f4d170700 1 --2- 172.21.15.27:0/2911725378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f480f55c0 0x7f1f480f59e0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 rx=0x7f1f30004660 tx=0x7f1f30009b30).stop 2021-11-03T18:10:40.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.036+0000 7f1f4d170700 1 -- 172.21.15.27:0/2911725378 shutdown_connections 2021-11-03T18:10:40.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.036+0000 7f1f4d170700 1 --2- 172.21.15.27:0/2911725378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f480f6860 0x7f1f480f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:40.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.036+0000 7f1f4d170700 1 --2- 172.21.15.27:0/2911725378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f480f55c0 0x7f1f480f59e0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:40.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.036+0000 7f1f4d170700 1 -- 172.21.15.27:0/2911725378 >> 172.21.15.27:0/2911725378 conn(0x7f1f480f0d40 msgr2=0x7f1f480f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:10:40.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.036+0000 7f1f4d170700 1 -- 172.21.15.27:0/2911725378 shutdown_connections 2021-11-03T18:10:40.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.036+0000 7f1f4d170700 1 -- 172.21.15.27:0/2911725378 wait complete. 2021-11-03T18:10:40.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.036+0000 7f1f4d170700 1 Processor -- start 2021-11-03T18:10:40.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.037+0000 7f1f4d170700 1 -- start start 2021-11-03T18:10:40.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.037+0000 7f1f4d170700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f480f55c0 0x7f1f48106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:10:40.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.037+0000 7f1f4d170700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f480f6860 0x7f1f48107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:10:40.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.037+0000 7f1f4d170700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f48107a10 con 0x7f1f480f6860 2021-11-03T18:10:40.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.037+0000 7f1f4d170700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f48107b50 con 0x7f1f480f55c0 2021-11-03T18:10:40.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.037+0000 7f1f4659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f480f6860 0x7f1f48107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:10:40.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.037+0000 7f1f4659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f480f6860 0x7f1f48107460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33398/0 (socket says 172.21.15.27:33398) 2021-11-03T18:10:40.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.037+0000 7f1f4659c700 1 -- 172.21.15.27:0/2357738743 learned_addr learned my addr 172.21.15.27:0/2357738743 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:10:40.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.037+0000 7f1f46d9d700 1 --2- 172.21.15.27:0/2357738743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f480f55c0 0x7f1f48106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:10:40.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.037+0000 7f1f4659c700 1 -- 172.21.15.27:0/2357738743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f480f55c0 msgr2=0x7f1f48106f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:10:40.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.037+0000 7f1f4659c700 1 --2- 172.21.15.27:0/2357738743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f480f55c0 0x7f1f48106f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:40.046 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.037+0000 7f1f4659c700 1 -- 172.21.15.27:0/2357738743 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f30005040 con 0x7f1f480f6860 2021-11-03T18:10:40.046 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.037+0000 7f1f4659c700 1 --2- 172.21.15.27:0/2357738743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f480f6860 0x7f1f48107460 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 rx=0x7f1f380026d0 tx=0x7f1f38009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:10:40.049 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.037+0000 7f1f3f7fe700 1 -- 172.21.15.27:0/2357738743 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f380079c0 con 0x7f1f480f6860 2021-11-03T18:10:40.049 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.038+0000 7f1f3f7fe700 1 -- 172.21.15.27:0/2357738743 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f1f3800f040 con 0x7f1f480f6860 2021-11-03T18:10:40.049 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.038+0000 7f1f4d170700 1 -- 172.21.15.27:0/2357738743 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f481084f0 con 0x7f1f480f6860 2021-11-03T18:10:40.049 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.038+0000 7f1f3f7fe700 1 -- 172.21.15.27:0/2357738743 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1f380148b0 con 0x7f1f480f6860 2021-11-03T18:10:40.050 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.038+0000 7f1f4d170700 1 -- 172.21.15.27:0/2357738743 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f48108ab0 con 0x7f1f480f6860 2021-11-03T18:10:40.050 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.038+0000 7f1f4d170700 1 -- 172.21.15.27:0/2357738743 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f48045ad0 con 0x7f1f480f6860 2021-11-03T18:10:40.050 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.039+0000 7f1f3f7fe700 1 -- 172.21.15.27:0/2357738743 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f1f38007b20 con 0x7f1f480f6860 2021-11-03T18:10:40.050 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.039+0000 7f1f3f7fe700 1 --2- 172.21.15.27:0/2357738743 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1f340640b0 0x7f1f34066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:10:40.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.039+0000 7f1f3f7fe700 1 -- 172.21.15.27:0/2357738743 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f1f380806d0 con 0x7f1f480f6860 2021-11-03T18:10:40.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.039+0000 7f1f46d9d700 1 --2- 172.21.15.27:0/2357738743 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1f340640b0 0x7f1f34066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:10:40.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.040+0000 7f1f46d9d700 1 --2- 172.21.15.27:0/2357738743 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1f340640b0 0x7f1f34066570 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f1f30004630 tx=0x7f1f30009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:10:40.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.043+0000 7f1f3f7fe700 1 -- 172.21.15.27:0/2357738743 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f1f380529d0 con 0x7f1f480f6860 2021-11-03T18:10:40.253 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.256+0000 7f1f4d170700 1 -- 172.21.15.27:0/2357738743 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f1f4810d590 con 0x7f1f480f6860 2021-11-03T18:10:40.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.256+0000 7f1f3f7fe700 1 -- 172.21.15.27:0/2357738743 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f1f380525c0 con 0x7f1f480f6860 2021-11-03T18:10:40.254 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:10:40.255 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:10:40.255 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:10:40.255 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:10:40.255 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:10:40.256 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:10:40.256 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:10:40.256 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:10:40.256 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:10:40.257 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:10:40.257 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:10:40.257 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:10:40.257 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:10:40.258 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:10:40.258 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:10:40.258 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:10:40.258 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:10:40.259 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:10:40.259 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:10:40.259 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:10:40.259 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:10:40.260 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:10:40.261 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:10:40.261 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:10:40.262 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:10:40.262 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:10:40.262 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:10:40.262 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:10:40.263 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:10:40.263 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:10:40.263 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:10:40.263 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:10:40.264 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:10:40.264 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:10:40.264 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:10:40.265 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:10:40.265 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:10:40.265 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:10:40.266 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:10:40.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.258+0000 7f1f4d170700 1 -- 172.21.15.27:0/2357738743 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1f340640b0 msgr2=0x7f1f34066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:10:40.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.258+0000 7f1f4d170700 1 --2- 172.21.15.27:0/2357738743 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1f340640b0 0x7f1f34066570 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f1f30004630 tx=0x7f1f30009b30).stop 2021-11-03T18:10:40.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.258+0000 7f1f4d170700 1 -- 172.21.15.27:0/2357738743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f480f6860 msgr2=0x7f1f48107460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:10:40.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.258+0000 7f1f4d170700 1 --2- 172.21.15.27:0/2357738743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f480f6860 0x7f1f48107460 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 rx=0x7f1f380026d0 tx=0x7f1f38009de0).stop 2021-11-03T18:10:40.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.258+0000 7f1f4d170700 1 -- 172.21.15.27:0/2357738743 shutdown_connections 2021-11-03T18:10:40.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.258+0000 7f1f4d170700 1 --2- 172.21.15.27:0/2357738743 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1f340640b0 0x7f1f34066570 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:40.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.258+0000 7f1f4d170700 1 --2- 172.21.15.27:0/2357738743 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1f480f55c0 0x7f1f48106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:40.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.258+0000 7f1f4d170700 1 --2- 172.21.15.27:0/2357738743 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1f480f6860 0x7f1f48107460 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:10:40.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.258+0000 7f1f4d170700 1 -- 172.21.15.27:0/2357738743 >> 172.21.15.27:0/2357738743 conn(0x7f1f480f0d40 msgr2=0x7f1f4805fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:10:40.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.258+0000 7f1f4d170700 1 -- 172.21.15.27:0/2357738743 shutdown_connections 2021-11-03T18:10:40.270 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:10:40.258+0000 7f1f4d170700 1 -- 172.21.15.27:0/2357738743 wait complete. 2021-11-03T18:10:40.270 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:10:40.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:40 smithi115 ceph-mon[65453]: pgmap v1680: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:40.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:40 smithi115 ceph-mon[65453]: from='client.35832 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:10:40.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3282.438232 secs 2021-11-03T18:10:40.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:40 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/99553288' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:40.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:40 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2357738743' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:10:40.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:40 smithi027 ceph-mon[97244]: pgmap v1680: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:40.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:40 smithi027 ceph-mon[97244]: from='client.35832 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:10:40.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3282.438232 secs 2021-11-03T18:10:40.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:40 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/99553288' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:40.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:40 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2357738743' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:10:41.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:41 smithi115 ceph-mon[65453]: from='client.35836 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:10:41.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:41 smithi027 ceph-mon[97244]: from='client.35836 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:10:42.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:42 smithi115 ceph-mon[65453]: pgmap v1681: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:42.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:42 smithi027 ceph-mon[97244]: pgmap v1681: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:44.534 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:44 smithi115 ceph-mon[65453]: pgmap v1682: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:44.535 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:10:44.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:44 smithi027 ceph-mon[97244]: pgmap v1682: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:44.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:10:45.364 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:45 smithi115 ceph-mon[65453]: poll took 40118.863 ms: 1 events 2021-11-03T18:10:45.365 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3287.438359 secs 2021-11-03T18:10:45.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:45 smithi027 ceph-mon[97244]: poll took 40118.863 ms: 1 events 2021-11-03T18:10:45.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3287.438359 secs 2021-11-03T18:10:46.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:46 smithi115 ceph-mon[65453]: pgmap v1683: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:46.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:46 smithi027 ceph-mon[97244]: pgmap v1683: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:47.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:47 smithi115 ceph-mon[65453]: poll took 2686.307 ms: 1 events 2021-11-03T18:10:47.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:10:47.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:10:47.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:10:47.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:47.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:47.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:47.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:47.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:47 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:10:47.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:47 smithi115 ceph-mon[65453]: pgmap v1684: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:48.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:47 smithi027 ceph-mon[97244]: poll took 2686.307 ms: 1 events 2021-11-03T18:10:48.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:10:48.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:10:48.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:10:48.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:48.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:48.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:48.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:48.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:47 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:10:48.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:47 smithi027 ceph-mon[97244]: pgmap v1684: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:50.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:49 smithi027 ceph-mon[97244]: pgmap v1685: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:50.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:49 smithi115 ceph-mon[65453]: pgmap v1685: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:51.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3292.438475 secs 2021-11-03T18:10:51.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3292.438475 secs 2021-11-03T18:10:52.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:52 smithi115 ceph-mon[65453]: pgmap v1686: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:52.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:52 smithi027 ceph-mon[97244]: pgmap v1686: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:54.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:54 smithi115 ceph-mon[65453]: pgmap v1687: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:54.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:54 smithi027 ceph-mon[97244]: pgmap v1687: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:55.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3297.438542 secs 2021-11-03T18:10:55.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3297.438542 secs 2021-11-03T18:10:56.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:56 smithi115 ceph-mon[65453]: pgmap v1688: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:56.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:10:56.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:10:56.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:56 smithi027 ceph-mon[97244]: pgmap v1688: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:56.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:10:56.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:10:57.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:10:57.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:10:57.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:10:57.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:57.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:57.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:57.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:57.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:10:57.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:10:57.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:10:57.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:57.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:57.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:57.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:10:58.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:58 smithi115 ceph-mon[65453]: pgmap v1689: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:58.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:10:58 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:10:58.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:58 smithi027 ceph-mon[97244]: pgmap v1689: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:10:58.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:10:58 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:11:00.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:00 smithi115 ceph-mon[65453]: pgmap v1690: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:00.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3302.438627 secs 2021-11-03T18:11:00.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:00 smithi027 ceph-mon[97244]: pgmap v1690: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:00.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3302.438627 secs 2021-11-03T18:11:02.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:02 smithi115 ceph-mon[65453]: pgmap v1691: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:02.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:02 smithi027 ceph-mon[97244]: pgmap v1691: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:04.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:04 smithi115 ceph-mon[65453]: pgmap v1692: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:04.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:04 smithi027 ceph-mon[97244]: pgmap v1692: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:05.498 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3307.438721 secs 2021-11-03T18:11:05.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3307.438721 secs 2021-11-03T18:11:06.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:06 smithi115 ceph-mon[65453]: pgmap v1693: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:06.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:06 smithi027 ceph-mon[97244]: pgmap v1693: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:07.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:11:07.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:11:07.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:11:07.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:07.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:07.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:07.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:07.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:11:07.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:11:07.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:11:07.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:07.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:07.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:07.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:08.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:08 smithi115 ceph-mon[65453]: pgmap v1694: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:08.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:08 smithi115 ceph-mon[65453]: poll took 20063.707 ms: 1 events 2021-11-03T18:11:08.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:08 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:11:08.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:08 smithi027 ceph-mon[97244]: pgmap v1694: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:08.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:08 smithi027 ceph-mon[97244]: poll took 20063.707 ms: 1 events 2021-11-03T18:11:08.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:08 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:11:10.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.373+0000 7f3a2d3d3700 1 -- 172.21.15.27:0/4193002645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a280f55c0 msgr2=0x7f3a280f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:11:10.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.373+0000 7f3a2d3d3700 1 --2- 172.21.15.27:0/4193002645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a280f55c0 0x7f3a280f59e0 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 rx=0x7f3a200099e0 tx=0x7f3a20004d10).stop 2021-11-03T18:11:10.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.373+0000 7f3a2d3d3700 1 -- 172.21.15.27:0/4193002645 shutdown_connections 2021-11-03T18:11:10.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.373+0000 7f3a2d3d3700 1 --2- 172.21.15.27:0/4193002645 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a280f6860 0x7f3a280f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:10.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.373+0000 7f3a2d3d3700 1 --2- 172.21.15.27:0/4193002645 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a280f55c0 0x7f3a280f59e0 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:10.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.373+0000 7f3a2d3d3700 1 -- 172.21.15.27:0/4193002645 >> 172.21.15.27:0/4193002645 conn(0x7f3a280f0d40 msgr2=0x7f3a280f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:11:10.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.373+0000 7f3a2d3d3700 1 -- 172.21.15.27:0/4193002645 shutdown_connections 2021-11-03T18:11:10.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.373+0000 7f3a2d3d3700 1 -- 172.21.15.27:0/4193002645 wait complete. 2021-11-03T18:11:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.374+0000 7f3a2d3d3700 1 Processor -- start 2021-11-03T18:11:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.374+0000 7f3a2d3d3700 1 -- start start 2021-11-03T18:11:10.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.374+0000 7f3a2d3d3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a280f55c0 0x7f3a28109d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:11:10.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.374+0000 7f3a2d3d3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a280f6860 0x7f3a2810c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:11:10.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.374+0000 7f3a2d3d3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a280603e0 con 0x7f3a280f6860 2021-11-03T18:11:10.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.374+0000 7f3a2d3d3700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a28060520 con 0x7f3a280f55c0 2021-11-03T18:11:10.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.374+0000 7f3a267fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a280f6860 0x7f3a2810c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:11:10.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.374+0000 7f3a267fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a280f6860 0x7f3a2810c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33416/0 (socket says 172.21.15.27:33416) 2021-11-03T18:11:10.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.374+0000 7f3a267fc700 1 -- 172.21.15.27:0/2733823052 learned_addr learned my addr 172.21.15.27:0/2733823052 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:11:10.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.374+0000 7f3a26ffd700 1 --2- 172.21.15.27:0/2733823052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a280f55c0 0x7f3a28109d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:11:10.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.374+0000 7f3a267fc700 1 -- 172.21.15.27:0/2733823052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a280f55c0 msgr2=0x7f3a28109d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:11:10.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.374+0000 7f3a267fc700 1 --2- 172.21.15.27:0/2733823052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a280f55c0 0x7f3a28109d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:10.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.374+0000 7f3a267fc700 1 -- 172.21.15.27:0/2733823052 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a20009640 con 0x7f3a280f6860 2021-11-03T18:11:10.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.375+0000 7f3a267fc700 1 --2- 172.21.15.27:0/2733823052 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a280f6860 0x7f3a2810c260 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 rx=0x7f3a1c0026d0 tx=0x7f3a1c009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:11:10.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.375+0000 7f3a0f7fe700 1 -- 172.21.15.27:0/2733823052 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a1c0079c0 con 0x7f3a280f6860 2021-11-03T18:11:10.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.375+0000 7f3a0f7fe700 1 -- 172.21.15.27:0/2733823052 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f3a1c00f040 con 0x7f3a280f6860 2021-11-03T18:11:10.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.375+0000 7f3a2d3d3700 1 -- 172.21.15.27:0/2733823052 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a2810c810 con 0x7f3a280f6860 2021-11-03T18:11:10.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.375+0000 7f3a0f7fe700 1 -- 172.21.15.27:0/2733823052 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a1c0148b0 con 0x7f3a280f6860 2021-11-03T18:11:10.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.375+0000 7f3a2d3d3700 1 -- 172.21.15.27:0/2733823052 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a2810cd50 con 0x7f3a280f6860 2021-11-03T18:11:10.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.376+0000 7f3a2d3d3700 1 -- 172.21.15.27:0/2733823052 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a28045ad0 con 0x7f3a280f6860 2021-11-03T18:11:10.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.376+0000 7f3a0f7fe700 1 -- 172.21.15.27:0/2733823052 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f3a1c007b20 con 0x7f3a280f6860 2021-11-03T18:11:10.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.377+0000 7f3a0f7fe700 1 --2- 172.21.15.27:0/2733823052 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3a100640b0 0x7f3a10066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:11:10.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.377+0000 7f3a0f7fe700 1 -- 172.21.15.27:0/2733823052 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f3a1c0806d0 con 0x7f3a280f6860 2021-11-03T18:11:10.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.377+0000 7f3a26ffd700 1 --2- 172.21.15.27:0/2733823052 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3a100640b0 0x7f3a10066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:11:10.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.377+0000 7f3a26ffd700 1 --2- 172.21.15.27:0/2733823052 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3a100640b0 0x7f3a10066570 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7f3a200099b0 tx=0x7f3a20004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:11:10.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.381+0000 7f3a0f7fe700 1 -- 172.21.15.27:0/2733823052 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f3a1c0529d0 con 0x7f3a280f6860 2021-11-03T18:11:10.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:10 smithi115 ceph-mon[65453]: pgmap v1695: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:10.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3312.438790 secs 2021-11-03T18:11:10.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:10 smithi027 ceph-mon[97244]: pgmap v1695: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:10.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3312.438790 secs 2021-11-03T18:11:10.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.560+0000 7f3a2d3d3700 1 -- 172.21.15.27:0/2733823052 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a280fdec0 con 0x7f3a100640b0 2021-11-03T18:11:10.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.562+0000 7f3a0f7fe700 1 -- 172.21.15.27:0/2733823052 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f3a280fdec0 con 0x7f3a100640b0 2021-11-03T18:11:10.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.563+0000 7f3a2d3d3700 1 -- 172.21.15.27:0/2733823052 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3a100640b0 msgr2=0x7f3a10066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:11:10.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.563+0000 7f3a2d3d3700 1 --2- 172.21.15.27:0/2733823052 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3a100640b0 0x7f3a10066570 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7f3a200099b0 tx=0x7f3a20004d10).stop 2021-11-03T18:11:10.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.563+0000 7f3a2d3d3700 1 -- 172.21.15.27:0/2733823052 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a280f6860 msgr2=0x7f3a2810c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:11:10.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.563+0000 7f3a2d3d3700 1 --2- 172.21.15.27:0/2733823052 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a280f6860 0x7f3a2810c260 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 rx=0x7f3a1c0026d0 tx=0x7f3a1c009de0).stop 2021-11-03T18:11:10.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.563+0000 7f3a2d3d3700 1 -- 172.21.15.27:0/2733823052 shutdown_connections 2021-11-03T18:11:10.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.563+0000 7f3a2d3d3700 1 --2- 172.21.15.27:0/2733823052 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3a100640b0 0x7f3a10066570 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:10.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.563+0000 7f3a2d3d3700 1 --2- 172.21.15.27:0/2733823052 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3a280f55c0 0x7f3a28109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:10.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.563+0000 7f3a2d3d3700 1 --2- 172.21.15.27:0/2733823052 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3a280f6860 0x7f3a2810c260 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:10.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.563+0000 7f3a2d3d3700 1 -- 172.21.15.27:0/2733823052 >> 172.21.15.27:0/2733823052 conn(0x7f3a280f0d40 msgr2=0x7f3a280fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:11:10.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.564+0000 7f3a2d3d3700 1 -- 172.21.15.27:0/2733823052 shutdown_connections 2021-11-03T18:11:10.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.564+0000 7f3a2d3d3700 1 -- 172.21.15.27:0/2733823052 wait complete. 2021-11-03T18:11:10.571 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:11:10.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.664+0000 7f4da271f700 1 -- 172.21.15.27:0/2917039759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d9c0f66e0 msgr2=0x7f4d9c0f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:11:10.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.664+0000 7f4da271f700 1 --2- 172.21.15.27:0/2917039759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d9c0f66e0 0x7f4d9c0f6b00 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 rx=0x7f4d90004660 tx=0x7f4d90009b30).stop 2021-11-03T18:11:10.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.664+0000 7f4da271f700 1 -- 172.21.15.27:0/2917039759 shutdown_connections 2021-11-03T18:11:10.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.664+0000 7f4da271f700 1 --2- 172.21.15.27:0/2917039759 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d9c0f6ff0 0x7f4d9c0f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:10.665 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.664+0000 7f4da271f700 1 --2- 172.21.15.27:0/2917039759 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d9c0f66e0 0x7f4d9c0f6b00 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:10.665 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.664+0000 7f4da271f700 1 -- 172.21.15.27:0/2917039759 >> 172.21.15.27:0/2917039759 conn(0x7f4d9c0f1f00 msgr2=0x7f4d9c0f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:11:10.665 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.665+0000 7f4da271f700 1 -- 172.21.15.27:0/2917039759 shutdown_connections 2021-11-03T18:11:10.665 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.665+0000 7f4da271f700 1 -- 172.21.15.27:0/2917039759 wait complete. 2021-11-03T18:11:10.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.665+0000 7f4da271f700 1 Processor -- start 2021-11-03T18:11:10.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.665+0000 7f4da271f700 1 -- start start 2021-11-03T18:11:10.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.665+0000 7f4da271f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d9c0f66e0 0x7f4d9c06b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:11:10.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.665+0000 7f4da271f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d9c0f6ff0 0x7f4d9c069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:11:10.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.665+0000 7f4da271f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c06a380 con 0x7f4d9c0f6ff0 2021-11-03T18:11:10.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.665+0000 7f4da271f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9c06a4c0 con 0x7f4d9c0f66e0 2021-11-03T18:11:10.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.665+0000 7f4d9b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d9c0f6ff0 0x7f4d9c069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:11:10.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.666+0000 7f4d9b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d9c0f6ff0 0x7f4d9c069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33426/0 (socket says 172.21.15.27:33426) 2021-11-03T18:11:10.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.666+0000 7f4d9b7fe700 1 -- 172.21.15.27:0/3480721486 learned_addr learned my addr 172.21.15.27:0/3480721486 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:11:10.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.666+0000 7f4d9bfff700 1 --2- 172.21.15.27:0/3480721486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d9c0f66e0 0x7f4d9c06b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:11:10.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.666+0000 7f4d9b7fe700 1 -- 172.21.15.27:0/3480721486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d9c0f66e0 msgr2=0x7f4d9c06b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:11:10.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.666+0000 7f4d9b7fe700 1 --2- 172.21.15.27:0/3480721486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d9c0f66e0 0x7f4d9c06b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:10.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.666+0000 7f4d9b7fe700 1 -- 172.21.15.27:0/3480721486 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d90005040 con 0x7f4d9c0f6ff0 2021-11-03T18:11:10.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.666+0000 7f4d9b7fe700 1 --2- 172.21.15.27:0/3480721486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d9c0f6ff0 0x7f4d9c069e20 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 rx=0x7f4d8c00c530 tx=0x7f4d8c007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:11:10.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.666+0000 7f4d98ff9700 1 -- 172.21.15.27:0/3480721486 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d8c00f650 con 0x7f4d9c0f6ff0 2021-11-03T18:11:10.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.666+0000 7f4d98ff9700 1 -- 172.21.15.27:0/3480721486 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f4d8c00fc60 con 0x7f4d9c0f6ff0 2021-11-03T18:11:10.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.666+0000 7f4da271f700 1 -- 172.21.15.27:0/3480721486 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d9c06a7a0 con 0x7f4d9c0f6ff0 2021-11-03T18:11:10.670 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.666+0000 7f4d98ff9700 1 -- 172.21.15.27:0/3480721486 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d8c012cd0 con 0x7f4d9c0f6ff0 2021-11-03T18:11:10.670 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.666+0000 7f4da271f700 1 -- 172.21.15.27:0/3480721486 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d9c10ef90 con 0x7f4d9c0f6ff0 2021-11-03T18:11:10.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.667+0000 7f4da271f700 1 -- 172.21.15.27:0/3480721486 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d9c045ad0 con 0x7f4d9c0f6ff0 2021-11-03T18:11:10.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.668+0000 7f4d98ff9700 1 -- 172.21.15.27:0/3480721486 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f4d8c010070 con 0x7f4d9c0f6ff0 2021-11-03T18:11:10.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.668+0000 7f4d98ff9700 1 --2- 172.21.15.27:0/3480721486 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4d840640b0 0x7f4d84066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:11:10.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.668+0000 7f4d98ff9700 1 -- 172.21.15.27:0/3480721486 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f4d8c080740 con 0x7f4d9c0f6ff0 2021-11-03T18:11:10.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.668+0000 7f4d9bfff700 1 --2- 172.21.15.27:0/3480721486 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4d840640b0 0x7f4d84066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:11:10.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.668+0000 7f4d9bfff700 1 --2- 172.21.15.27:0/3480721486 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4d840640b0 0x7f4d84066570 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f4d90004e30 tx=0x7f4d90009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:11:10.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.672+0000 7f4d98ff9700 1 -- 172.21.15.27:0/3480721486 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f4d8c052950 con 0x7f4d9c0f6ff0 2021-11-03T18:11:10.841 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.844+0000 7f4da271f700 1 -- 172.21.15.27:0/3480721486 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4d9c0ff010 con 0x7f4d840640b0 2021-11-03T18:11:10.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.859+0000 7f4d98ff9700 1 -- 172.21.15.27:0/3480721486 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f4d9c0ff010 con 0x7f4d840640b0 2021-11-03T18:11:10.856 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:11:10.857 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (62m) 114s ago 63m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:11:10.857 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (54m) 114s ago 63m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:11:10.857 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (54m) 67s ago 63m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:11:10.857 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (62m) 114s ago 63m 42.1M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:11:10.858 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (56m) 114s ago 59m 26.3M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:11:10.858 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (59m) 114s ago 59m 3847M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:11:10.858 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (54m) 67s ago 59m 25.9M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:11:10.858 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (56m) 67s ago 59m 27.0M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:11:10.859 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (56m) 114s ago 65m 572M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:11:10.859 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (55m) 67s ago 63m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:11:10.859 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (55m) 114s ago 65m 273M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:11:10.859 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (55m) 67s ago 63m 129M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:11:10.860 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (63m) 114s ago 63m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:11:10.860 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (62m) 67s ago 62m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:11:10.860 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (54m) 114s ago 62m 371M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:11:10.861 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (53m) 114s ago 62m 405M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:11:10.861 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (53m) 114s ago 61m 302M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:11:10.861 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (52m) 67s ago 61m 429M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:11:10.861 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (52m) 67s ago 61m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:11:10.861 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (51m) 67s ago 60m 348M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:11:10.862 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (62m) 114s ago 63m 57.6M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:11:10.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.860+0000 7f4da271f700 1 -- 172.21.15.27:0/3480721486 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4d840640b0 msgr2=0x7f4d84066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:11:10.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.860+0000 7f4da271f700 1 --2- 172.21.15.27:0/3480721486 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4d840640b0 0x7f4d84066570 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f4d90004e30 tx=0x7f4d90009b30).stop 2021-11-03T18:11:10.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.860+0000 7f4da271f700 1 -- 172.21.15.27:0/3480721486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d9c0f6ff0 msgr2=0x7f4d9c069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:11:10.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.860+0000 7f4da271f700 1 --2- 172.21.15.27:0/3480721486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d9c0f6ff0 0x7f4d9c069e20 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 rx=0x7f4d8c00c530 tx=0x7f4d8c007e20).stop 2021-11-03T18:11:10.864 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.861+0000 7f4da271f700 1 -- 172.21.15.27:0/3480721486 shutdown_connections 2021-11-03T18:11:10.864 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.861+0000 7f4da271f700 1 --2- 172.21.15.27:0/3480721486 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4d840640b0 0x7f4d84066570 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:10.864 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.861+0000 7f4da271f700 1 --2- 172.21.15.27:0/3480721486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d9c0f66e0 0x7f4d9c06b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:10.864 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.861+0000 7f4da271f700 1 --2- 172.21.15.27:0/3480721486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d9c0f6ff0 0x7f4d9c069e20 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:10.864 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.861+0000 7f4da271f700 1 -- 172.21.15.27:0/3480721486 >> 172.21.15.27:0/3480721486 conn(0x7f4d9c0f1f00 msgr2=0x7f4d9c0fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:11:10.865 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.861+0000 7f4da271f700 1 -- 172.21.15.27:0/3480721486 shutdown_connections 2021-11-03T18:11:10.865 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.861+0000 7f4da271f700 1 -- 172.21.15.27:0/3480721486 wait complete. 2021-11-03T18:11:10.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.960+0000 7fc6422d1700 1 -- 172.21.15.27:0/2859854655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc63c0f6c30 msgr2=0x7fc63c0f7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:11:10.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.960+0000 7fc6422d1700 1 --2- 172.21.15.27:0/2859854655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc63c0f6c30 0x7fc63c0f7050 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 rx=0x7fc62c004690 tx=0x7fc62c009b30).stop 2021-11-03T18:11:10.961 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.960+0000 7fc6422d1700 1 -- 172.21.15.27:0/2859854655 shutdown_connections 2021-11-03T18:11:10.961 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.960+0000 7fc6422d1700 1 --2- 172.21.15.27:0/2859854655 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc63c0f7540 0x7fc63c0fdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:10.961 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.961+0000 7fc6422d1700 1 --2- 172.21.15.27:0/2859854655 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc63c0f6c30 0x7fc63c0f7050 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:10.961 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.961+0000 7fc6422d1700 1 -- 172.21.15.27:0/2859854655 >> 172.21.15.27:0/2859854655 conn(0x7fc63c0f0540 msgr2=0x7fc63c0f2960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:11:10.962 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.961+0000 7fc6422d1700 1 -- 172.21.15.27:0/2859854655 shutdown_connections 2021-11-03T18:11:10.962 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.961+0000 7fc6422d1700 1 -- 172.21.15.27:0/2859854655 wait complete. 2021-11-03T18:11:10.962 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.961+0000 7fc6422d1700 1 Processor -- start 2021-11-03T18:11:10.962 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.961+0000 7fc6422d1700 1 -- start start 2021-11-03T18:11:10.963 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.961+0000 7fc6422d1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc63c0f6c30 0x7fc63c1090f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:11:10.963 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.961+0000 7fc6422d1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc63c0f7540 0x7fc63c1095e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:11:10.963 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.961+0000 7fc6422d1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc63c109b90 con 0x7fc63c0f6c30 2021-11-03T18:11:10.963 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.961+0000 7fc6422d1700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc63c109cd0 con 0x7fc63c0f7540 2021-11-03T18:11:10.964 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.962+0000 7fc63bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc63c0f6c30 0x7fc63c1090f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:11:10.964 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.962+0000 7fc63bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc63c0f6c30 0x7fc63c1090f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33434/0 (socket says 172.21.15.27:33434) 2021-11-03T18:11:10.964 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.962+0000 7fc63bfff700 1 -- 172.21.15.27:0/3769745684 learned_addr learned my addr 172.21.15.27:0/3769745684 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:11:10.964 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.962+0000 7fc63b7fe700 1 --2- 172.21.15.27:0/3769745684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc63c0f7540 0x7fc63c1095e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:11:10.964 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.962+0000 7fc63bfff700 1 -- 172.21.15.27:0/3769745684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc63c0f7540 msgr2=0x7fc63c1095e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:11:10.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.962+0000 7fc63bfff700 1 --2- 172.21.15.27:0/3769745684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc63c0f7540 0x7fc63c1095e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:10.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.962+0000 7fc63bfff700 1 -- 172.21.15.27:0/3769745684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc62c005040 con 0x7fc63c0f6c30 2021-11-03T18:11:10.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.962+0000 7fc63b7fe700 1 --2- 172.21.15.27:0/3769745684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc63c0f7540 0x7fc63c1095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:11:10.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.962+0000 7fc63bfff700 1 --2- 172.21.15.27:0/3769745684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc63c0f6c30 0x7fc63c1090f0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 rx=0x7fc62c0079e0 tx=0x7fc62c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:11:10.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.962+0000 7fc638ff9700 1 -- 172.21.15.27:0/3769745684 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc62c018070 con 0x7fc63c0f6c30 2021-11-03T18:11:10.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.962+0000 7fc6422d1700 1 -- 172.21.15.27:0/3769745684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc63c10a610 con 0x7fc63c0f6c30 2021-11-03T18:11:10.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.962+0000 7fc638ff9700 1 -- 172.21.15.27:0/3769745684 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fc62c013e40 con 0x7fc63c0f6c30 2021-11-03T18:11:10.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.962+0000 7fc638ff9700 1 -- 172.21.15.27:0/3769745684 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc62c01c3c0 con 0x7fc63c0f6c30 2021-11-03T18:11:10.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.962+0000 7fc6422d1700 1 -- 172.21.15.27:0/3769745684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc63c10ab70 con 0x7fc63c0f6c30 2021-11-03T18:11:10.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.963+0000 7fc6422d1700 1 -- 172.21.15.27:0/3769745684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc63c045ad0 con 0x7fc63c0f6c30 2021-11-03T18:11:10.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.964+0000 7fc638ff9700 1 -- 172.21.15.27:0/3769745684 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fc62c01d420 con 0x7fc63c0f6c30 2021-11-03T18:11:10.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.964+0000 7fc638ff9700 1 --2- 172.21.15.27:0/3769745684 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc6240640b0 0x7fc624066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:11:10.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.964+0000 7fc638ff9700 1 -- 172.21.15.27:0/3769745684 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fc62c0825a0 con 0x7fc63c0f6c30 2021-11-03T18:11:10.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.964+0000 7fc63b7fe700 1 --2- 172.21.15.27:0/3769745684 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc6240640b0 0x7fc624066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:11:10.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.964+0000 7fc63b7fe700 1 --2- 172.21.15.27:0/3769745684 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc6240640b0 0x7fc624066570 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7fc630009780 tx=0x7fc630009c80).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:11:10.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:10.968+0000 7fc638ff9700 1 -- 172.21.15.27:0/3769745684 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fc62c0548a0 con 0x7fc63c0f6c30 2021-11-03T18:11:11.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.197+0000 7fc6422d1700 1 -- 172.21.15.27:0/3769745684 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc63c101230 con 0x7fc63c0f6c30 2021-11-03T18:11:11.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.198+0000 7fc638ff9700 1 -- 172.21.15.27:0/3769745684 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fc62c054490 con 0x7fc63c0f6c30 2021-11-03T18:11:11.195 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:11:11.196 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:11:11.196 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:11:11.196 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:11:11.196 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:11:11.197 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:11:11.197 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:11:11.197 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:11:11.198 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:11:11.198 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:11:11.198 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:11:11.198 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:11:11.198 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:11:11.199 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:11:11.199 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:11:11.199 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:11:11.199 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:11:11.200 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:11:11.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.199+0000 7fc6422d1700 1 -- 172.21.15.27:0/3769745684 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc6240640b0 msgr2=0x7fc624066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:11:11.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.199+0000 7fc6422d1700 1 --2- 172.21.15.27:0/3769745684 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc6240640b0 0x7fc624066570 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7fc630009780 tx=0x7fc630009c80).stop 2021-11-03T18:11:11.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.199+0000 7fc6422d1700 1 -- 172.21.15.27:0/3769745684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc63c0f6c30 msgr2=0x7fc63c1090f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:11:11.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.199+0000 7fc6422d1700 1 --2- 172.21.15.27:0/3769745684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc63c0f6c30 0x7fc63c1090f0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 rx=0x7fc62c0079e0 tx=0x7fc62c009b30).stop 2021-11-03T18:11:11.202 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.200+0000 7fc6422d1700 1 -- 172.21.15.27:0/3769745684 shutdown_connections 2021-11-03T18:11:11.202 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.200+0000 7fc6422d1700 1 --2- 172.21.15.27:0/3769745684 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc6240640b0 0x7fc624066570 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:11.202 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.200+0000 7fc6422d1700 1 --2- 172.21.15.27:0/3769745684 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc63c0f6c30 0x7fc63c1090f0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:11.202 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.200+0000 7fc6422d1700 1 --2- 172.21.15.27:0/3769745684 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc63c0f7540 0x7fc63c1095e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:11.202 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.200+0000 7fc6422d1700 1 -- 172.21.15.27:0/3769745684 >> 172.21.15.27:0/3769745684 conn(0x7fc63c0f0540 msgr2=0x7fc63c0f2930 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:11:11.203 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.200+0000 7fc6422d1700 1 -- 172.21.15.27:0/3769745684 shutdown_connections 2021-11-03T18:11:11.203 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.200+0000 7fc6422d1700 1 -- 172.21.15.27:0/3769745684 wait complete. 2021-11-03T18:11:11.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.301+0000 7ff23dd09700 1 -- 172.21.15.27:0/1300361669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2380f7980 msgr2=0x7ff2380f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:11:11.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.301+0000 7ff23dd09700 1 --2- 172.21.15.27:0/1300361669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2380f7980 0x7ff2380f7de0 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 rx=0x7ff220004660 tx=0x7ff220009b30).stop 2021-11-03T18:11:11.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.301+0000 7ff23dd09700 1 -- 172.21.15.27:0/1300361669 shutdown_connections 2021-11-03T18:11:11.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.301+0000 7ff23dd09700 1 --2- 172.21.15.27:0/1300361669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2380f7980 0x7ff2380f7de0 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:11.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.301+0000 7ff23dd09700 1 --2- 172.21.15.27:0/1300361669 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2380f66e0 0x7ff2380f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:11.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.301+0000 7ff23dd09700 1 -- 172.21.15.27:0/1300361669 >> 172.21.15.27:0/1300361669 conn(0x7ff2380f1e40 msgr2=0x7ff2380f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:11:11.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.301+0000 7ff23dd09700 1 -- 172.21.15.27:0/1300361669 shutdown_connections 2021-11-03T18:11:11.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.301+0000 7ff23dd09700 1 -- 172.21.15.27:0/1300361669 wait complete. 2021-11-03T18:11:11.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.301+0000 7ff23dd09700 1 Processor -- start 2021-11-03T18:11:11.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.302+0000 7ff23dd09700 1 -- start start 2021-11-03T18:11:11.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.302+0000 7ff23dd09700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2380f66e0 0x7ff23806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:11:11.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.302+0000 7ff23dd09700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2380f7980 0x7ff238069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:11:11.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.302+0000 7ff23dd09700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23806a380 con 0x7ff2380f66e0 2021-11-03T18:11:11.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.302+0000 7ff23dd09700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23806a4c0 con 0x7ff2380f7980 2021-11-03T18:11:11.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.302+0000 7ff2377fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2380f66e0 0x7ff23806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:11:11.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.302+0000 7ff2377fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2380f66e0 0x7ff23806b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33444/0 (socket says 172.21.15.27:33444) 2021-11-03T18:11:11.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.302+0000 7ff2377fe700 1 -- 172.21.15.27:0/2403486955 learned_addr learned my addr 172.21.15.27:0/2403486955 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:11:11.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.302+0000 7ff236ffd700 1 --2- 172.21.15.27:0/2403486955 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2380f7980 0x7ff238069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:11:11.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.302+0000 7ff2377fe700 1 -- 172.21.15.27:0/2403486955 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2380f7980 msgr2=0x7ff238069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:11:11.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.302+0000 7ff2377fe700 1 --2- 172.21.15.27:0/2403486955 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2380f7980 0x7ff238069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:11.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.302+0000 7ff2377fe700 1 -- 172.21.15.27:0/2403486955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff220005040 con 0x7ff2380f66e0 2021-11-03T18:11:11.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.302+0000 7ff2377fe700 1 --2- 172.21.15.27:0/2403486955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2380f66e0 0x7ff23806b9a0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 rx=0x7ff23000e7c0 tx=0x7ff23000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:11:11.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.302+0000 7ff22ffff700 1 -- 172.21.15.27:0/2403486955 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff2300092e0 con 0x7ff2380f66e0 2021-11-03T18:11:11.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.303+0000 7ff22ffff700 1 -- 172.21.15.27:0/2403486955 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7ff23000f040 con 0x7ff2380f66e0 2021-11-03T18:11:11.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.303+0000 7ff23dd09700 1 -- 172.21.15.27:0/2403486955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff23806a7a0 con 0x7ff2380f66e0 2021-11-03T18:11:11.307 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.303+0000 7ff22ffff700 1 -- 172.21.15.27:0/2403486955 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff230014750 con 0x7ff2380f66e0 2021-11-03T18:11:11.307 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.303+0000 7ff23dd09700 1 -- 172.21.15.27:0/2403486955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff23810ef40 con 0x7ff2380f66e0 2021-11-03T18:11:11.307 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.303+0000 7ff23dd09700 1 -- 172.21.15.27:0/2403486955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff238045ad0 con 0x7ff2380f66e0 2021-11-03T18:11:11.307 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.304+0000 7ff22ffff700 1 -- 172.21.15.27:0/2403486955 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7ff23001b030 con 0x7ff2380f66e0 2021-11-03T18:11:11.308 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.304+0000 7ff22ffff700 1 --2- 172.21.15.27:0/2403486955 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff2240640b0 0x7ff224066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:11:11.308 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.304+0000 7ff22ffff700 1 -- 172.21.15.27:0/2403486955 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7ff230080780 con 0x7ff2380f66e0 2021-11-03T18:11:11.308 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.304+0000 7ff236ffd700 1 --2- 172.21.15.27:0/2403486955 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff2240640b0 0x7ff224066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:11:11.308 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.305+0000 7ff236ffd700 1 --2- 172.21.15.27:0/2403486955 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff2240640b0 0x7ff224066570 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7ff2200043d0 tx=0x7ff22000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:11:11.309 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.308+0000 7ff22ffff700 1 -- 172.21.15.27:0/2403486955 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7ff230052a40 con 0x7ff2380f66e0 2021-11-03T18:11:11.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:11 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3769745684' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:11.509 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.511+0000 7ff23dd09700 1 -- 172.21.15.27:0/2403486955 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff2380feea0 con 0x7ff2380f66e0 2021-11-03T18:11:11.509 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.512+0000 7ff22ffff700 1 -- 172.21.15.27:0/2403486955 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7ff230019070 con 0x7ff2380f66e0 2021-11-03T18:11:11.510 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:11:11.510 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:11:11.510 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:11:11.511 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:11:11.511 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:11:11.512 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:11:11.512 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:11:11.512 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:11:11.512 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:11:11.513 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:11:11.513 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:11:11.513 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:11:11.513 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:11:11.513 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:11:11.514 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:11:11.514 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:11:11.514 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:11:11.514 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:11:11.515 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:11:11.515 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:11:11.515 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:11:11.515 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:11:11.516 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:11:11.516 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:11:11.516 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:11:11.516 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:11:11.516 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:11:11.517 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:11:11.517 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:11:11.517 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:11:11.517 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:11:11.518 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:11:11.518 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:11:11.518 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:11:11.519 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:11:11.519 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:11:11.519 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:11:11.519 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:11:11.520 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:11:11.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.514+0000 7ff23dd09700 1 -- 172.21.15.27:0/2403486955 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff2240640b0 msgr2=0x7ff224066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:11:11.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.514+0000 7ff23dd09700 1 --2- 172.21.15.27:0/2403486955 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff2240640b0 0x7ff224066570 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7ff2200043d0 tx=0x7ff22000b040).stop 2021-11-03T18:11:11.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.514+0000 7ff23dd09700 1 -- 172.21.15.27:0/2403486955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2380f66e0 msgr2=0x7ff23806b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:11:11.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.514+0000 7ff23dd09700 1 --2- 172.21.15.27:0/2403486955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2380f66e0 0x7ff23806b9a0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 rx=0x7ff23000e7c0 tx=0x7ff23000b040).stop 2021-11-03T18:11:11.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.514+0000 7ff23dd09700 1 -- 172.21.15.27:0/2403486955 shutdown_connections 2021-11-03T18:11:11.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.514+0000 7ff23dd09700 1 --2- 172.21.15.27:0/2403486955 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff2240640b0 0x7ff224066570 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:11.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.514+0000 7ff23dd09700 1 --2- 172.21.15.27:0/2403486955 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2380f66e0 0x7ff23806b9a0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:11.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.514+0000 7ff23dd09700 1 --2- 172.21.15.27:0/2403486955 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2380f7980 0x7ff238069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:11.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.514+0000 7ff23dd09700 1 -- 172.21.15.27:0/2403486955 >> 172.21.15.27:0/2403486955 conn(0x7ff2380f1e40 msgr2=0x7ff2380fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:11:11.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.514+0000 7ff23dd09700 1 -- 172.21.15.27:0/2403486955 shutdown_connections 2021-11-03T18:11:11.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:11.514+0000 7ff23dd09700 1 -- 172.21.15.27:0/2403486955 wait complete. 2021-11-03T18:11:11.524 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:11:11.524 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:11 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3769745684' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:12.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:12 smithi115 ceph-mon[65453]: from='client.35848 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:11:12.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:12 smithi115 ceph-mon[65453]: pgmap v1696: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:12.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:12 smithi115 ceph-mon[65453]: from='client.35852 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:11:12.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:12 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2403486955' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:11:12.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:12 smithi027 ceph-mon[97244]: from='client.35848 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:11:12.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:12 smithi027 ceph-mon[97244]: pgmap v1696: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:12.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:12 smithi027 ceph-mon[97244]: from='client.35852 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:11:12.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:12 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2403486955' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:11:14.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:14 smithi115 ceph-mon[65453]: pgmap v1697: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:14.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:14 smithi027 ceph-mon[97244]: pgmap v1697: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:15.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3317.438885 secs 2021-11-03T18:11:15.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3317.438885 secs 2021-11-03T18:11:16.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:16 smithi115 ceph-mon[65453]: pgmap v1698: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:16.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:16 smithi027 ceph-mon[97244]: pgmap v1698: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:17.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:11:17.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:11:17.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:11:17.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:17.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:17.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:17.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:17.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:11:17.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:11:17.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:11:17.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:17.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:17.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:17.524 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:18.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:18 smithi115 ceph-mon[65453]: pgmap v1699: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:18.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:18 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:11:18.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:18 smithi027 ceph-mon[97244]: pgmap v1699: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:18.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:18 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:11:20.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:20 smithi115 ceph-mon[65453]: pgmap v1700: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:20.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3322.439030 secs 2021-11-03T18:11:20.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:20 smithi027 ceph-mon[97244]: pgmap v1700: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:20.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3322.439030 secs 2021-11-03T18:11:22.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:22 smithi115 ceph-mon[65453]: pgmap v1701: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:22.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:22 smithi027 ceph-mon[97244]: pgmap v1701: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:24.486 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:24 smithi115 ceph-mon[65453]: pgmap v1702: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:24.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:24 smithi027 ceph-mon[97244]: pgmap v1702: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:25.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3327.439150 secs 2021-11-03T18:11:25.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3327.439150 secs 2021-11-03T18:11:26.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:26 smithi027 ceph-mon[97244]: pgmap v1703: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:26.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:26 smithi115 ceph-mon[65453]: pgmap v1703: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:27.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:11:27.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:11:27.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:11:27.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:27.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:27.524 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:27.524 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:27.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:11:27.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:11:27.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:11:27.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:27.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:27.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:27.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:28.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:28 smithi027 ceph-mon[97244]: pgmap v1704: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:28.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:28 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:11:28.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:28 smithi115 ceph-mon[65453]: pgmap v1704: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:28.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:28 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:11:30.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:30 smithi027 ceph-mon[97244]: pgmap v1705: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:30.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3332.439277 secs 2021-11-03T18:11:30.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:30 smithi115 ceph-mon[65453]: pgmap v1705: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:30.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3332.439277 secs 2021-11-03T18:11:32.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:32 smithi027 ceph-mon[97244]: pgmap v1706: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:32.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:32 smithi115 ceph-mon[65453]: pgmap v1706: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:34.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:34 smithi027 ceph-mon[97244]: pgmap v1707: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:34.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:34 smithi115 ceph-mon[65453]: pgmap v1707: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:35.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3337.439433 secs 2021-11-03T18:11:35.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3337.439433 secs 2021-11-03T18:11:36.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:36 smithi027 ceph-mon[97244]: pgmap v1708: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:36.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:36 smithi115 ceph-mon[65453]: pgmap v1708: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:37.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:11:37.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:11:37.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:11:37.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:37.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:37.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:37.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:37.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:11:37.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:11:37.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:11:37.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:37.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:37.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:37.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:38.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:38 smithi027 ceph-mon[97244]: pgmap v1709: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:38.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:38 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:11:38.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:38 smithi115 ceph-mon[65453]: pgmap v1709: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:38.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:38 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:11:40.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:40 smithi115 ceph-mon[65453]: pgmap v1710: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:40.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3342.439537 secs 2021-11-03T18:11:40.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:40 smithi027 ceph-mon[97244]: pgmap v1710: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:40.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3342.439537 secs 2021-11-03T18:11:41.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.627+0000 7fb684f50700 1 -- 172.21.15.27:0/1736613034 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6800f6ff0 msgr2=0x7fb6800f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:11:41.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.627+0000 7fb684f50700 1 --2- 172.21.15.27:0/1736613034 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6800f6ff0 0x7fb6800f7a20 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 rx=0x7fb674004660 tx=0x7fb674009b30).stop 2021-11-03T18:11:41.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.627+0000 7fb684f50700 1 -- 172.21.15.27:0/1736613034 shutdown_connections 2021-11-03T18:11:41.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.627+0000 7fb684f50700 1 --2- 172.21.15.27:0/1736613034 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6800f6ff0 0x7fb6800f7a20 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:41.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.627+0000 7fb684f50700 1 --2- 172.21.15.27:0/1736613034 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6800f66e0 0x7fb6800f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:41.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.627+0000 7fb684f50700 1 -- 172.21.15.27:0/1736613034 >> 172.21.15.27:0/1736613034 conn(0x7fb6800f1f00 msgr2=0x7fb6800f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:11:41.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.627+0000 7fb684f50700 1 -- 172.21.15.27:0/1736613034 shutdown_connections 2021-11-03T18:11:41.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.627+0000 7fb684f50700 1 -- 172.21.15.27:0/1736613034 wait complete. 2021-11-03T18:11:41.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.628+0000 7fb684f50700 1 Processor -- start 2021-11-03T18:11:41.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.628+0000 7fb684f50700 1 -- start start 2021-11-03T18:11:41.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.628+0000 7fb684f50700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6800f66e0 0x7fb680106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:11:41.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.628+0000 7fb684f50700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6800f6ff0 0x7fb680107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:11:41.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.628+0000 7fb684f50700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb680107a10 con 0x7fb6800f6ff0 2021-11-03T18:11:41.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.628+0000 7fb684f50700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb680107b50 con 0x7fb6800f66e0 2021-11-03T18:11:41.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.628+0000 7fb67dd9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6800f6ff0 0x7fb680107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:11:41.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.628+0000 7fb67dd9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6800f6ff0 0x7fb680107460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33464/0 (socket says 172.21.15.27:33464) 2021-11-03T18:11:41.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.628+0000 7fb67dd9b700 1 -- 172.21.15.27:0/1133035937 learned_addr learned my addr 172.21.15.27:0/1133035937 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:11:41.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.628+0000 7fb67e59c700 1 --2- 172.21.15.27:0/1133035937 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6800f66e0 0x7fb680106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:11:41.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.628+0000 7fb67dd9b700 1 -- 172.21.15.27:0/1133035937 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6800f66e0 msgr2=0x7fb680106f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:11:41.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.629+0000 7fb67dd9b700 1 --2- 172.21.15.27:0/1133035937 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6800f66e0 0x7fb680106f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:41.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.629+0000 7fb67dd9b700 1 -- 172.21.15.27:0/1133035937 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb674005040 con 0x7fb6800f6ff0 2021-11-03T18:11:41.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.629+0000 7fb67dd9b700 1 --2- 172.21.15.27:0/1133035937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6800f6ff0 0x7fb680107460 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 rx=0x7fb674009990 tx=0x7fb674009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:11:41.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.629+0000 7fb66effd700 1 -- 172.21.15.27:0/1133035937 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb674018070 con 0x7fb6800f6ff0 2021-11-03T18:11:41.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.629+0000 7fb66effd700 1 -- 172.21.15.27:0/1133035937 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fb674007ba0 con 0x7fb6800f6ff0 2021-11-03T18:11:41.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.629+0000 7fb684f50700 1 -- 172.21.15.27:0/1133035937 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb680108490 con 0x7fb6800f6ff0 2021-11-03T18:11:41.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.629+0000 7fb66effd700 1 -- 172.21.15.27:0/1133035937 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb67401c3c0 con 0x7fb6800f6ff0 2021-11-03T18:11:41.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.629+0000 7fb684f50700 1 -- 172.21.15.27:0/1133035937 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6801089f0 con 0x7fb6800f6ff0 2021-11-03T18:11:41.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.630+0000 7fb684f50700 1 -- 172.21.15.27:0/1133035937 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb680045ad0 con 0x7fb6800f6ff0 2021-11-03T18:11:41.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.631+0000 7fb66effd700 1 -- 172.21.15.27:0/1133035937 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fb67401d420 con 0x7fb6800f6ff0 2021-11-03T18:11:41.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.631+0000 7fb66effd700 1 --2- 172.21.15.27:0/1133035937 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb6680640b0 0x7fb668066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:11:41.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.631+0000 7fb66effd700 1 -- 172.21.15.27:0/1133035937 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fb674082740 con 0x7fb6800f6ff0 2021-11-03T18:11:41.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.632+0000 7fb67e59c700 1 --2- 172.21.15.27:0/1133035937 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb6680640b0 0x7fb668066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:11:41.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.632+0000 7fb67e59c700 1 --2- 172.21.15.27:0/1133035937 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb6680640b0 0x7fb668066570 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7fb670004c40 tx=0x7fb67000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:11:41.637 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.635+0000 7fb66effd700 1 -- 172.21.15.27:0/1133035937 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fb674054a40 con 0x7fb6800f6ff0 2021-11-03T18:11:41.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.816+0000 7fb684f50700 1 -- 172.21.15.27:0/1133035937 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb6800ff080 con 0x7fb6680640b0 2021-11-03T18:11:41.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.817+0000 7fb66effd700 1 -- 172.21.15.27:0/1133035937 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fb67000f040 con 0x7fb6680640b0 2021-11-03T18:11:41.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.819+0000 7fb684f50700 1 -- 172.21.15.27:0/1133035937 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb6680640b0 msgr2=0x7fb668066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:11:41.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.819+0000 7fb684f50700 1 --2- 172.21.15.27:0/1133035937 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb6680640b0 0x7fb668066570 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 rx=0x7fb670004c40 tx=0x7fb67000b040).stop 2021-11-03T18:11:41.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.819+0000 7fb684f50700 1 -- 172.21.15.27:0/1133035937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6800f6ff0 msgr2=0x7fb680107460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:11:41.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.819+0000 7fb684f50700 1 --2- 172.21.15.27:0/1133035937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6800f6ff0 0x7fb680107460 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 rx=0x7fb674009990 tx=0x7fb674009b30).stop 2021-11-03T18:11:41.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.819+0000 7fb684f50700 1 -- 172.21.15.27:0/1133035937 shutdown_connections 2021-11-03T18:11:41.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.819+0000 7fb684f50700 1 --2- 172.21.15.27:0/1133035937 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb6680640b0 0x7fb668066570 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:41.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.819+0000 7fb684f50700 1 --2- 172.21.15.27:0/1133035937 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6800f66e0 0x7fb680106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:41.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.819+0000 7fb684f50700 1 --2- 172.21.15.27:0/1133035937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6800f6ff0 0x7fb680107460 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:41.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.819+0000 7fb684f50700 1 -- 172.21.15.27:0/1133035937 >> 172.21.15.27:0/1133035937 conn(0x7fb6800f1f00 msgr2=0x7fb6800fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:11:41.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.819+0000 7fb684f50700 1 -- 172.21.15.27:0/1133035937 shutdown_connections 2021-11-03T18:11:41.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.819+0000 7fb684f50700 1 -- 172.21.15.27:0/1133035937 wait complete. 2021-11-03T18:11:41.827 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:11:41.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.919+0000 7f6c8c89e700 1 -- 172.21.15.27:0/3212427331 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c880f6860 msgr2=0x7f6c880f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:11:41.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.919+0000 7f6c8c89e700 1 --2- 172.21.15.27:0/3212427331 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c880f6860 0x7f6c880f6cc0 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 rx=0x7f6c7c004660 tx=0x7f6c7c009b30).stop 2021-11-03T18:11:41.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.919+0000 7f6c8c89e700 1 -- 172.21.15.27:0/3212427331 shutdown_connections 2021-11-03T18:11:41.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.919+0000 7f6c8c89e700 1 --2- 172.21.15.27:0/3212427331 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c880f6860 0x7f6c880f6cc0 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:41.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.919+0000 7f6c8c89e700 1 --2- 172.21.15.27:0/3212427331 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c880f55c0 0x7f6c880f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:41.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.919+0000 7f6c8c89e700 1 -- 172.21.15.27:0/3212427331 >> 172.21.15.27:0/3212427331 conn(0x7f6c880f0d40 msgr2=0x7f6c880f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:11:41.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.919+0000 7f6c8c89e700 1 -- 172.21.15.27:0/3212427331 shutdown_connections 2021-11-03T18:11:41.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.919+0000 7f6c8c89e700 1 -- 172.21.15.27:0/3212427331 wait complete. 2021-11-03T18:11:41.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.919+0000 7f6c8c89e700 1 Processor -- start 2021-11-03T18:11:41.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.919+0000 7f6c8c89e700 1 -- start start 2021-11-03T18:11:41.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.920+0000 7f6c8c89e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c880f55c0 0x7f6c88106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:11:41.922 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.920+0000 7f6c8c89e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c880f6860 0x7f6c88107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:11:41.922 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.920+0000 7f6c8c89e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c88107a10 con 0x7f6c880f55c0 2021-11-03T18:11:41.923 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.920+0000 7f6c8c89e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c88107b50 con 0x7f6c880f6860 2021-11-03T18:11:41.923 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.920+0000 7f6c8659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c880f55c0 0x7f6c88106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:11:41.923 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.920+0000 7f6c8659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c880f55c0 0x7f6c88106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33472/0 (socket says 172.21.15.27:33472) 2021-11-03T18:11:41.923 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.920+0000 7f6c8659c700 1 -- 172.21.15.27:0/1425453486 learned_addr learned my addr 172.21.15.27:0/1425453486 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:11:41.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.920+0000 7f6c85d9b700 1 --2- 172.21.15.27:0/1425453486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c880f6860 0x7f6c88107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:11:41.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.920+0000 7f6c8659c700 1 -- 172.21.15.27:0/1425453486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c880f6860 msgr2=0x7f6c88107460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:11:41.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.920+0000 7f6c8659c700 1 --2- 172.21.15.27:0/1425453486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c880f6860 0x7f6c88107460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:41.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.920+0000 7f6c8659c700 1 -- 172.21.15.27:0/1425453486 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c7c005040 con 0x7f6c880f55c0 2021-11-03T18:11:41.925 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.920+0000 7f6c85d9b700 1 --2- 172.21.15.27:0/1425453486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c880f6860 0x7f6c88107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T18:11:41.925 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.920+0000 7f6c8659c700 1 --2- 172.21.15.27:0/1425453486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c880f55c0 0x7f6c88106f70 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 rx=0x7f6c8000d7f0 tx=0x7f6c80004e80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:11:41.925 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.920+0000 7f6c76ffd700 1 -- 172.21.15.27:0/1425453486 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c8000b4f0 con 0x7f6c880f55c0 2021-11-03T18:11:41.925 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.920+0000 7f6c8c89e700 1 -- 172.21.15.27:0/1425453486 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c881084f0 con 0x7f6c880f55c0 2021-11-03T18:11:41.926 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.920+0000 7f6c76ffd700 1 -- 172.21.15.27:0/1425453486 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f6c8000f040 con 0x7f6c880f55c0 2021-11-03T18:11:41.926 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.920+0000 7f6c76ffd700 1 -- 172.21.15.27:0/1425453486 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6c80013980 con 0x7f6c880f55c0 2021-11-03T18:11:41.926 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.920+0000 7f6c8c89e700 1 -- 172.21.15.27:0/1425453486 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c88108ab0 con 0x7f6c880f55c0 2021-11-03T18:11:41.927 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.921+0000 7f6c8c89e700 1 -- 172.21.15.27:0/1425453486 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c88045ad0 con 0x7f6c880f55c0 2021-11-03T18:11:41.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.922+0000 7f6c76ffd700 1 -- 172.21.15.27:0/1425453486 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f6c8001a030 con 0x7f6c880f55c0 2021-11-03T18:11:41.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.922+0000 7f6c76ffd700 1 --2- 172.21.15.27:0/1425453486 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6c700684c0 0x7f6c7006a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:11:41.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.922+0000 7f6c76ffd700 1 -- 172.21.15.27:0/1425453486 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f6c800808b0 con 0x7f6c880f55c0 2021-11-03T18:11:41.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.922+0000 7f6c85d9b700 1 --2- 172.21.15.27:0/1425453486 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6c700684c0 0x7f6c7006a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:11:41.929 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.923+0000 7f6c85d9b700 1 --2- 172.21.15.27:0/1425453486 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6c700684c0 0x7f6c7006a980 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f6c7c009da0 tx=0x7f6c7c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:11:41.929 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:41.926+0000 7f6c76ffd700 1 -- 172.21.15.27:0/1425453486 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f6c80052ac0 con 0x7f6c880f55c0 2021-11-03T18:11:42.097 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.098+0000 7f6c8c89e700 1 -- 172.21.15.27:0/1425453486 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f6c88108d30 con 0x7f6c700684c0 2021-11-03T18:11:42.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.113+0000 7f6c76ffd700 1 -- 172.21.15.27:0/1425453486 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f6c88108d30 con 0x7f6c700684c0 2021-11-03T18:11:42.112 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:11:42.112 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (63m) 2m ago 64m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:11:42.113 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (54m) 2m ago 64m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:11:42.113 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (54m) 98s ago 63m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:11:42.113 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (63m) 2m ago 64m 42.1M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:11:42.114 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (56m) 2m ago 59m 26.3M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:11:42.114 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (59m) 2m ago 59m 3847M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:11:42.114 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (55m) 98s ago 59m 25.9M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:11:42.114 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (57m) 98s ago 59m 27.0M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:11:42.115 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (56m) 2m ago 65m 572M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:11:42.115 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (56m) 98s ago 63m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:11:42.115 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (55m) 2m ago 65m 273M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:11:42.115 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (55m) 98s ago 63m 129M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:11:42.116 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (64m) 2m ago 64m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:11:42.116 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (63m) 98s ago 63m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:11:42.116 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (54m) 2m ago 62m 371M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:11:42.117 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (54m) 2m ago 62m 405M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:11:42.117 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (53m) 2m ago 62m 302M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:11:42.117 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (53m) 98s ago 61m 429M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:11:42.117 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (52m) 98s ago 61m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:11:42.118 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (52m) 98s ago 61m 348M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:11:42.118 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (63m) 2m ago 63m 57.6M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:11:42.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.115+0000 7f6c8c89e700 1 -- 172.21.15.27:0/1425453486 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6c700684c0 msgr2=0x7f6c7006a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:11:42.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.115+0000 7f6c8c89e700 1 --2- 172.21.15.27:0/1425453486 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6c700684c0 0x7f6c7006a980 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f6c7c009da0 tx=0x7f6c7c009b30).stop 2021-11-03T18:11:42.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.115+0000 7f6c8c89e700 1 -- 172.21.15.27:0/1425453486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c880f55c0 msgr2=0x7f6c88106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:11:42.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.115+0000 7f6c8c89e700 1 --2- 172.21.15.27:0/1425453486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c880f55c0 0x7f6c88106f70 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 rx=0x7f6c8000d7f0 tx=0x7f6c80004e80).stop 2021-11-03T18:11:42.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.115+0000 7f6c8c89e700 1 -- 172.21.15.27:0/1425453486 shutdown_connections 2021-11-03T18:11:42.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.115+0000 7f6c8c89e700 1 --2- 172.21.15.27:0/1425453486 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6c700684c0 0x7f6c7006a980 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:42.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.115+0000 7f6c8c89e700 1 --2- 172.21.15.27:0/1425453486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6c880f55c0 0x7f6c88106f70 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:42.121 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.115+0000 7f6c8c89e700 1 --2- 172.21.15.27:0/1425453486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6c880f6860 0x7f6c88107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:42.122 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.115+0000 7f6c8c89e700 1 -- 172.21.15.27:0/1425453486 >> 172.21.15.27:0/1425453486 conn(0x7f6c880f0d40 msgr2=0x7f6c880f3130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:11:42.122 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.115+0000 7f6c8c89e700 1 -- 172.21.15.27:0/1425453486 shutdown_connections 2021-11-03T18:11:42.122 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.115+0000 7f6c8c89e700 1 -- 172.21.15.27:0/1425453486 wait complete. 2021-11-03T18:11:42.215 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.214+0000 7f535f59e700 1 -- 172.21.15.27:0/2387877879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53600f5240 msgr2=0x7f53600f7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:11:42.215 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.214+0000 7f535f59e700 1 --2- 172.21.15.27:0/2387877879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53600f5240 0x7f53600f7b80 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 rx=0x7f5348004660 tx=0x7f5348009b30).stop 2021-11-03T18:11:42.215 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.214+0000 7f535f59e700 1 -- 172.21.15.27:0/2387877879 shutdown_connections 2021-11-03T18:11:42.216 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.214+0000 7f535f59e700 1 --2- 172.21.15.27:0/2387877879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53600f5240 0x7f53600f7b80 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:42.216 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.214+0000 7f535f59e700 1 --2- 172.21.15.27:0/2387877879 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53600f48a0 0x7f53600f4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:42.216 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.214+0000 7f535f59e700 1 -- 172.21.15.27:0/2387877879 >> 172.21.15.27:0/2387877879 conn(0x7f53600f0320 msgr2=0x7f53600f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:11:42.216 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.214+0000 7f535f59e700 1 -- 172.21.15.27:0/2387877879 shutdown_connections 2021-11-03T18:11:42.217 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.214+0000 7f535f59e700 1 -- 172.21.15.27:0/2387877879 wait complete. 2021-11-03T18:11:42.217 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.215+0000 7f535f59e700 1 Processor -- start 2021-11-03T18:11:42.217 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.215+0000 7f535f59e700 1 -- start start 2021-11-03T18:11:42.217 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.215+0000 7f535f59e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53600f48a0 0x7f5360106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:11:42.217 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.215+0000 7f535f59e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53600f5240 0x7f5360107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:11:42.218 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.215+0000 7f535f59e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5360107a10 con 0x7f53600f48a0 2021-11-03T18:11:42.218 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.215+0000 7f535f59e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5360107b50 con 0x7f53600f5240 2021-11-03T18:11:42.218 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.215+0000 7f535e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53600f48a0 0x7f5360106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:11:42.218 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.215+0000 7f535e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53600f48a0 0x7f5360106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33482/0 (socket says 172.21.15.27:33482) 2021-11-03T18:11:42.219 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.215+0000 7f535e59c700 1 -- 172.21.15.27:0/3940525803 learned_addr learned my addr 172.21.15.27:0/3940525803 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:11:42.219 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.216+0000 7f5355d9b700 1 --2- 172.21.15.27:0/3940525803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53600f5240 0x7f5360107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:11:42.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.216+0000 7f535e59c700 1 -- 172.21.15.27:0/3940525803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53600f5240 msgr2=0x7f5360107460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:11:42.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.216+0000 7f535e59c700 1 --2- 172.21.15.27:0/3940525803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53600f5240 0x7f5360107460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:42.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.216+0000 7f535e59c700 1 -- 172.21.15.27:0/3940525803 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5348005040 con 0x7f53600f48a0 2021-11-03T18:11:42.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.216+0000 7f5355d9b700 1 --2- 172.21.15.27:0/3940525803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53600f5240 0x7f5360107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T18:11:42.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.216+0000 7f535e59c700 1 --2- 172.21.15.27:0/3940525803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53600f48a0 0x7f5360106f70 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 rx=0x7f53500026d0 tx=0x7f5350009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:11:42.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.216+0000 7f53577fe700 1 -- 172.21.15.27:0/3940525803 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5350009320 con 0x7f53600f48a0 2021-11-03T18:11:42.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.216+0000 7f535f59e700 1 -- 172.21.15.27:0/3940525803 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53601084f0 con 0x7f53600f48a0 2021-11-03T18:11:42.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.216+0000 7f535f59e700 1 -- 172.21.15.27:0/3940525803 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5360108ab0 con 0x7f53600f48a0 2021-11-03T18:11:42.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.216+0000 7f53577fe700 1 -- 172.21.15.27:0/3940525803 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f535000f040 con 0x7f53600f48a0 2021-11-03T18:11:42.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.216+0000 7f53577fe700 1 -- 172.21.15.27:0/3940525803 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53500147e0 con 0x7f53600f48a0 2021-11-03T18:11:42.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.217+0000 7f535f59e700 1 -- 172.21.15.27:0/3940525803 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5360045ad0 con 0x7f53600f48a0 2021-11-03T18:11:42.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.218+0000 7f53577fe700 1 -- 172.21.15.27:0/3940525803 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f5350009480 con 0x7f53600f48a0 2021-11-03T18:11:42.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.218+0000 7f53577fe700 1 --2- 172.21.15.27:0/3940525803 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f534c064100 0x7f534c0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:11:42.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.218+0000 7f53577fe700 1 -- 172.21.15.27:0/3940525803 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f5350080f30 con 0x7f53600f48a0 2021-11-03T18:11:42.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.218+0000 7f5355d9b700 1 --2- 172.21.15.27:0/3940525803 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f534c064100 0x7f534c0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:11:42.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.219+0000 7f5355d9b700 1 --2- 172.21.15.27:0/3940525803 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f534c064100 0x7f534c0665c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f5348009da0 tx=0x7f5348009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:11:42.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.222+0000 7f53577fe700 1 -- 172.21.15.27:0/3940525803 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f5350053230 con 0x7f53600f48a0 2021-11-03T18:11:42.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.454+0000 7f535f59e700 1 -- 172.21.15.27:0/3940525803 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f53600fef50 con 0x7f53600f48a0 2021-11-03T18:11:42.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.455+0000 7f53577fe700 1 -- 172.21.15.27:0/3940525803 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f5350052e20 con 0x7f53600f48a0 2021-11-03T18:11:42.454 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:11:42.454 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:11:42.455 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:11:42.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:11:42.455 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:11:42.455 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:11:42.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:11:42.456 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:11:42.456 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:11:42.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:11:42.457 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:11:42.457 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:11:42.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:11:42.457 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:11:42.458 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:11:42.458 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:11:42.458 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:11:42.458 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:11:42.459 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.457+0000 7f535f59e700 1 -- 172.21.15.27:0/3940525803 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f534c064100 msgr2=0x7f534c0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:11:42.459 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.457+0000 7f535f59e700 1 --2- 172.21.15.27:0/3940525803 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f534c064100 0x7f534c0665c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f5348009da0 tx=0x7f5348009b30).stop 2021-11-03T18:11:42.460 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.457+0000 7f535f59e700 1 -- 172.21.15.27:0/3940525803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53600f48a0 msgr2=0x7f5360106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:11:42.460 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.457+0000 7f535f59e700 1 --2- 172.21.15.27:0/3940525803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53600f48a0 0x7f5360106f70 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 rx=0x7f53500026d0 tx=0x7f5350009de0).stop 2021-11-03T18:11:42.460 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.457+0000 7f535f59e700 1 -- 172.21.15.27:0/3940525803 shutdown_connections 2021-11-03T18:11:42.460 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.457+0000 7f535f59e700 1 --2- 172.21.15.27:0/3940525803 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f534c064100 0x7f534c0665c0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:42.461 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.457+0000 7f535f59e700 1 --2- 172.21.15.27:0/3940525803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53600f48a0 0x7f5360106f70 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:42.461 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.457+0000 7f535f59e700 1 --2- 172.21.15.27:0/3940525803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53600f5240 0x7f5360107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:42.461 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.457+0000 7f535f59e700 1 -- 172.21.15.27:0/3940525803 >> 172.21.15.27:0/3940525803 conn(0x7f53600f0320 msgr2=0x7f53600fb5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:11:42.461 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.457+0000 7f535f59e700 1 -- 172.21.15.27:0/3940525803 shutdown_connections 2021-11-03T18:11:42.462 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.457+0000 7f535f59e700 1 -- 172.21.15.27:0/3940525803 wait complete. 2021-11-03T18:11:42.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.557+0000 7f4a05f0a700 1 -- 172.21.15.27:0/3488201585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a000f6530 msgr2=0x7f4a000f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:11:42.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.557+0000 7f4a05f0a700 1 --2- 172.21.15.27:0/3488201585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a000f6530 0x7f4a000f6950 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 rx=0x7f49f4004660 tx=0x7f49f4009b30).stop 2021-11-03T18:11:42.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.557+0000 7f4a05f0a700 1 -- 172.21.15.27:0/3488201585 shutdown_connections 2021-11-03T18:11:42.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.557+0000 7f4a05f0a700 1 --2- 172.21.15.27:0/3488201585 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a000f77d0 0x7f4a000f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:42.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.557+0000 7f4a05f0a700 1 --2- 172.21.15.27:0/3488201585 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a000f6530 0x7f4a000f6950 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:42.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.557+0000 7f4a05f0a700 1 -- 172.21.15.27:0/3488201585 >> 172.21.15.27:0/3488201585 conn(0x7f4a000f1cd0 msgr2=0x7f4a000f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:11:42.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.557+0000 7f4a05f0a700 1 -- 172.21.15.27:0/3488201585 shutdown_connections 2021-11-03T18:11:42.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.557+0000 7f4a05f0a700 1 -- 172.21.15.27:0/3488201585 wait complete. 2021-11-03T18:11:42.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.558+0000 7f4a05f0a700 1 Processor -- start 2021-11-03T18:11:42.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.558+0000 7f4a05f0a700 1 -- start start 2021-11-03T18:11:42.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.558+0000 7f4a05f0a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a000f6530 0x7f4a0006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:11:42.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.558+0000 7f4a05f0a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a000f77d0 0x7f4a00069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:11:42.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.558+0000 7f4a05f0a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0006a380 con 0x7f4a000f6530 2021-11-03T18:11:42.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.558+0000 7f4a05f0a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a0006a4c0 con 0x7f4a000f77d0 2021-11-03T18:11:42.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.558+0000 7f4a04f08700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a000f6530 0x7f4a0006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:11:42.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.558+0000 7f4a04f08700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a000f6530 0x7f4a0006b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33492/0 (socket says 172.21.15.27:33492) 2021-11-03T18:11:42.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.558+0000 7f4a04f08700 1 -- 172.21.15.27:0/388846487 learned_addr learned my addr 172.21.15.27:0/388846487 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:11:42.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.558+0000 7f49fffff700 1 --2- 172.21.15.27:0/388846487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a000f77d0 0x7f4a00069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:11:42.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.558+0000 7f4a04f08700 1 -- 172.21.15.27:0/388846487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a000f77d0 msgr2=0x7f4a00069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:11:42.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.558+0000 7f4a04f08700 1 --2- 172.21.15.27:0/388846487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a000f77d0 0x7f4a00069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:42.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.558+0000 7f4a04f08700 1 -- 172.21.15.27:0/388846487 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f4005040 con 0x7f4a000f6530 2021-11-03T18:11:42.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.558+0000 7f4a04f08700 1 --2- 172.21.15.27:0/388846487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a000f6530 0x7f4a0006b9a0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 rx=0x7f49f4004f60 tx=0x7f49f4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:11:42.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.559+0000 7f49fd7fa700 1 -- 172.21.15.27:0/388846487 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49f4018070 con 0x7f4a000f6530 2021-11-03T18:11:42.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.559+0000 7f49fd7fa700 1 -- 172.21.15.27:0/388846487 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f49f4007ba0 con 0x7f4a000f6530 2021-11-03T18:11:42.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.559+0000 7f4a05f0a700 1 -- 172.21.15.27:0/388846487 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a0006a740 con 0x7f4a000f6530 2021-11-03T18:11:42.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.559+0000 7f49fd7fa700 1 -- 172.21.15.27:0/388846487 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f49f401c620 con 0x7f4a000f6530 2021-11-03T18:11:42.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.559+0000 7f4a05f0a700 1 -- 172.21.15.27:0/388846487 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a0006aca0 con 0x7f4a000f6530 2021-11-03T18:11:42.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.560+0000 7f4a05f0a700 1 -- 172.21.15.27:0/388846487 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a00045ad0 con 0x7f4a000f6530 2021-11-03T18:11:42.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.560+0000 7f49fd7fa700 1 -- 172.21.15.27:0/388846487 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f49f401d420 con 0x7f4a000f6530 2021-11-03T18:11:42.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.561+0000 7f49fd7fa700 1 --2- 172.21.15.27:0/388846487 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f49f0064100 0x7f49f00665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:11:42.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.561+0000 7f49fd7fa700 1 -- 172.21.15.27:0/388846487 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f49f4082680 con 0x7f4a000f6530 2021-11-03T18:11:42.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.561+0000 7f49fffff700 1 --2- 172.21.15.27:0/388846487 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f49f0064100 0x7f49f00665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:11:42.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.561+0000 7f49fffff700 1 --2- 172.21.15.27:0/388846487 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f49f0064100 0x7f49f00665c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f49ec0041d0 tx=0x7f49ec009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:11:42.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.565+0000 7f49fd7fa700 1 -- 172.21.15.27:0/388846487 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f49f4054a40 con 0x7f4a000f6530 2021-11-03T18:11:42.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:42 smithi115 ceph-mon[65453]: pgmap v1711: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:42.771 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.772+0000 7f4a05f0a700 1 -- 172.21.15.27:0/388846487 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4a000fee00 con 0x7f4a000f6530 2021-11-03T18:11:42.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:42 smithi027 ceph-mon[97244]: pgmap v1711: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:42.772 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.773+0000 7f49fd7fa700 1 -- 172.21.15.27:0/388846487 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f49f4054630 con 0x7f4a000f6530 2021-11-03T18:11:42.772 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:11:42.772 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:11:42.773 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:11:42.773 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:11:42.773 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:11:42.773 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:11:42.774 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:11:42.774 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:11:42.774 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:11:42.774 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:11:42.774 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:11:42.775 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:11:42.775 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:11:42.775 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:11:42.775 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:11:42.776 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:11:42.776 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:11:42.776 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:11:42.776 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:11:42.777 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:11:42.777 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:11:42.777 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:11:42.777 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:11:42.778 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:11:42.778 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:11:42.778 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:11:42.778 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:11:42.778 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:11:42.779 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:11:42.779 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:11:42.779 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:11:42.779 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:11:42.780 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:11:42.780 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:11:42.780 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:11:42.780 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:11:42.780 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:11:42.781 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:11:42.781 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:11:42.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.775+0000 7f4a05f0a700 1 -- 172.21.15.27:0/388846487 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f49f0064100 msgr2=0x7f49f00665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:11:42.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.775+0000 7f4a05f0a700 1 --2- 172.21.15.27:0/388846487 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f49f0064100 0x7f49f00665c0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f49ec0041d0 tx=0x7f49ec009040).stop 2021-11-03T18:11:42.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.775+0000 7f4a05f0a700 1 -- 172.21.15.27:0/388846487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a000f6530 msgr2=0x7f4a0006b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:11:42.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.775+0000 7f4a05f0a700 1 --2- 172.21.15.27:0/388846487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a000f6530 0x7f4a0006b9a0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 rx=0x7f49f4004f60 tx=0x7f49f4009b30).stop 2021-11-03T18:11:42.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.775+0000 7f4a05f0a700 1 -- 172.21.15.27:0/388846487 shutdown_connections 2021-11-03T18:11:42.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.775+0000 7f4a05f0a700 1 --2- 172.21.15.27:0/388846487 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f49f0064100 0x7f49f00665c0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:42.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.775+0000 7f4a05f0a700 1 --2- 172.21.15.27:0/388846487 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4a000f6530 0x7f4a0006b9a0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:42.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.775+0000 7f4a05f0a700 1 --2- 172.21.15.27:0/388846487 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4a000f77d0 0x7f4a00069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:11:42.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.775+0000 7f4a05f0a700 1 -- 172.21.15.27:0/388846487 >> 172.21.15.27:0/388846487 conn(0x7f4a000f1cd0 msgr2=0x7f4a000fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:11:42.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.775+0000 7f4a05f0a700 1 -- 172.21.15.27:0/388846487 shutdown_connections 2021-11-03T18:11:42.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:11:42.775+0000 7f4a05f0a700 1 -- 172.21.15.27:0/388846487 wait complete. 2021-11-03T18:11:42.784 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:11:43.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:43 smithi115 ceph-mon[65453]: from='client.35864 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:11:43.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:43 smithi115 ceph-mon[65453]: from='client.35868 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:11:43.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:43 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3940525803' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:43.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:43 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/388846487' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:11:43.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:43 smithi027 ceph-mon[97244]: from='client.35864 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:11:43.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:43 smithi027 ceph-mon[97244]: from='client.35868 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:11:43.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:43 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3940525803' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:43.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:43 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/388846487' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:11:44.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:44 smithi115 ceph-mon[65453]: pgmap v1712: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:44.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:44 smithi027 ceph-mon[97244]: pgmap v1712: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:45.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3347.439641 secs 2021-11-03T18:11:45.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3347.439641 secs 2021-11-03T18:11:46.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:46 smithi115 ceph-mon[65453]: pgmap v1713: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:46.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:46 smithi027 ceph-mon[97244]: pgmap v1713: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:47.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:11:47.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:11:47.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:11:47.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:47.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:47.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:47.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:47.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:11:47.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:11:47.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:11:47.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:47.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:47.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:47.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:48.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:48 smithi115 ceph-mon[65453]: pgmap v1714: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:48.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:48 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:11:48.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:48 smithi027 ceph-mon[97244]: pgmap v1714: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:48.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:48 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:11:50.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:50 smithi115 ceph-mon[65453]: pgmap v1715: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:50.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3352.439776 secs 2021-11-03T18:11:50.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:50 smithi027 ceph-mon[97244]: pgmap v1715: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:50.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3352.439776 secs 2021-11-03T18:11:52.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:52 smithi115 ceph-mon[65453]: pgmap v1716: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:52.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:52 smithi027 ceph-mon[97244]: pgmap v1716: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:54.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:54 smithi115 ceph-mon[65453]: pgmap v1717: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:54.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:54 smithi027 ceph-mon[97244]: pgmap v1717: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:55.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3357.439920 secs 2021-11-03T18:11:55.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3357.439920 secs 2021-11-03T18:11:56.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:56 smithi115 ceph-mon[65453]: pgmap v1718: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:56.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:11:56.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:11:56.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:56 smithi027 ceph-mon[97244]: pgmap v1718: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:56.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:11:56.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:11:57.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:11:57.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:11:57.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:11:57.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:57.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:57.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:57.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:57.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:11:57.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:11:57.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:11:57.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:57.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:57.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:57.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:11:58.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:58 smithi115 ceph-mon[65453]: pgmap v1719: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:58.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:11:58 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:11:58.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:58 smithi027 ceph-mon[97244]: pgmap v1719: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:11:58.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:11:58 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:12:00.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:00 smithi115 ceph-mon[65453]: pgmap v1720: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:00.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3362.440032 secs 2021-11-03T18:12:00.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:00 smithi027 ceph-mon[97244]: pgmap v1720: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:00.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3362.440032 secs 2021-11-03T18:12:02.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:02 smithi115 ceph-mon[65453]: pgmap v1721: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:02.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:02 smithi027 ceph-mon[97244]: pgmap v1721: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:04.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:04 smithi115 ceph-mon[65453]: pgmap v1722: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:04.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:04 smithi027 ceph-mon[97244]: pgmap v1722: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:05.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3367.440161 secs 2021-11-03T18:12:05.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3367.440161 secs 2021-11-03T18:12:06.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:06 smithi115 ceph-mon[65453]: pgmap v1723: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:06.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:06 smithi027 ceph-mon[97244]: pgmap v1723: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:07.631 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:12:07.646 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:12:08.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:08 smithi115 ceph-mon[65453]: pgmap v1724: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:08.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:08 smithi115 ceph-mon[65453]: poll took 60165.931 ms: 1 events 2021-11-03T18:12:08.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:12:08.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:12:08.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:08.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:08.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:08.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:08.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:08 smithi027 ceph-mon[97244]: pgmap v1724: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:08.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:08 smithi027 ceph-mon[97244]: poll took 60165.931 ms: 1 events 2021-11-03T18:12:08.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:12:08.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:12:08.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:08.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:08.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:08.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:09.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:09 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:12:09.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:09 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:12:10.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:10 smithi115 ceph-mon[65453]: pgmap v1725: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:10.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3372.440305 secs 2021-11-03T18:12:10.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:10 smithi027 ceph-mon[97244]: pgmap v1725: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:10.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3372.440305 secs 2021-11-03T18:12:12.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:12 smithi115 ceph-mon[65453]: pgmap v1726: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:12.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:12 smithi027 ceph-mon[97244]: pgmap v1726: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:12.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:12.889+0000 7f1439153700 1 -- 172.21.15.27:0/618364136 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14340f5ef0 msgr2=0x7f14340f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:12:12.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:12.889+0000 7f1439153700 1 --2- 172.21.15.27:0/618364136 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14340f5ef0 0x7f14340f6920 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 rx=0x7f141c004660 tx=0x7f141c009b30).stop 2021-11-03T18:12:12.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:12.890+0000 7f1439153700 1 -- 172.21.15.27:0/618364136 shutdown_connections 2021-11-03T18:12:12.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:12.890+0000 7f1439153700 1 --2- 172.21.15.27:0/618364136 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14340f5ef0 0x7f14340f6920 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:12.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:12.890+0000 7f1439153700 1 --2- 172.21.15.27:0/618364136 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14340f55e0 0x7f14340f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:12.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:12.890+0000 7f1439153700 1 -- 172.21.15.27:0/618364136 >> 172.21.15.27:0/618364136 conn(0x7f14340f0de0 msgr2=0x7f14340f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:12:12.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:12.890+0000 7f1439153700 1 -- 172.21.15.27:0/618364136 shutdown_connections 2021-11-03T18:12:12.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:12.890+0000 7f1439153700 1 -- 172.21.15.27:0/618364136 wait complete. 2021-11-03T18:12:12.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:12.890+0000 7f1439153700 1 Processor -- start 2021-11-03T18:12:12.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:12.890+0000 7f1439153700 1 -- start start 2021-11-03T18:12:12.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:12.891+0000 7f1439153700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14340f55e0 0x7f1434109cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:12:12.900 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:12.891+0000 7f1439153700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14340f5ef0 0x7f143410c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:12:12.900 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:12.891+0000 7f1439153700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14340604c0 con 0x7f14340f55e0 2021-11-03T18:12:12.900 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:12.891+0000 7f1439153700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1434060600 con 0x7f14340f5ef0 2021-11-03T18:12:12.900 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:12.891+0000 7f1432d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14340f55e0 0x7f1434109cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:12:12.901 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:12.891+0000 7f1432d9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14340f55e0 0x7f1434109cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33510/0 (socket says 172.21.15.27:33510) 2021-11-03T18:12:12.901 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:12.891+0000 7f1432d9d700 1 -- 172.21.15.27:0/2992012007 learned_addr learned my addr 172.21.15.27:0/2992012007 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:12:12.901 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:12.891+0000 7f143259c700 1 --2- 172.21.15.27:0/2992012007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14340f5ef0 0x7f143410c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:12:12.901 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:12.891+0000 7f1432d9d700 1 -- 172.21.15.27:0/2992012007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14340f5ef0 msgr2=0x7f143410c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:12:12.902 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:12.891+0000 7f1432d9d700 1 --2- 172.21.15.27:0/2992012007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14340f5ef0 0x7f143410c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:12.902 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:12.891+0000 7f1432d9d700 1 -- 172.21.15.27:0/2992012007 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f141c005040 con 0x7f14340f55e0 2021-11-03T18:12:12.903 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:12.891+0000 7f1432d9d700 1 --2- 172.21.15.27:0/2992012007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14340f55e0 0x7f1434109cb0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 rx=0x7f142400c530 tx=0x7f1424007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:12:12.904 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:12.891+0000 7f142b7fe700 1 -- 172.21.15.27:0/2992012007 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f142400f650 con 0x7f14340f55e0 2021-11-03T18:12:12.904 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:12.891+0000 7f142b7fe700 1 -- 172.21.15.27:0/2992012007 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f142400fc60 con 0x7f14340f55e0 2021-11-03T18:12:12.904 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:12.891+0000 7f1439153700 1 -- 172.21.15.27:0/2992012007 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f143410c760 con 0x7f14340f55e0 2021-11-03T18:12:12.905 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:12.891+0000 7f142b7fe700 1 -- 172.21.15.27:0/2992012007 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1424012c20 con 0x7f14340f55e0 2021-11-03T18:12:12.905 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:12.892+0000 7f1439153700 1 -- 172.21.15.27:0/2992012007 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f143410ccf0 con 0x7f14340f55e0 2021-11-03T18:12:12.905 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:12.892+0000 7f1439153700 1 -- 172.21.15.27:0/2992012007 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14340f4a70 con 0x7f14340f55e0 2021-11-03T18:12:12.905 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:12.893+0000 7f142b7fe700 1 -- 172.21.15.27:0/2992012007 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f1424010070 con 0x7f14340f55e0 2021-11-03T18:12:12.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:12.893+0000 7f142b7fe700 1 --2- 172.21.15.27:0/2992012007 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f14200640b0 0x7f1420066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:12:12.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:12.893+0000 7f142b7fe700 1 -- 172.21.15.27:0/2992012007 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f1424080690 con 0x7f14340f55e0 2021-11-03T18:12:12.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:12.893+0000 7f143259c700 1 --2- 172.21.15.27:0/2992012007 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f14200640b0 0x7f1420066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:12:12.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:12.894+0000 7f143259c700 1 --2- 172.21.15.27:0/2992012007 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f14200640b0 0x7f1420066570 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f141c004630 tx=0x7f141c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:12:12.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:12.897+0000 7f142b7fe700 1 -- 172.21.15.27:0/2992012007 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f14240527e0 con 0x7f14340f55e0 2021-11-03T18:12:13.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.076+0000 7f1439153700 1 -- 172.21.15.27:0/2992012007 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f14340fdf40 con 0x7f14200640b0 2021-11-03T18:12:13.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.077+0000 7f142b7fe700 1 -- 172.21.15.27:0/2992012007 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f14340fdf40 con 0x7f14200640b0 2021-11-03T18:12:13.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.079+0000 7f1439153700 1 -- 172.21.15.27:0/2992012007 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f14200640b0 msgr2=0x7f1420066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:12:13.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.079+0000 7f1439153700 1 --2- 172.21.15.27:0/2992012007 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f14200640b0 0x7f1420066570 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f141c004630 tx=0x7f141c009b30).stop 2021-11-03T18:12:13.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.079+0000 7f1439153700 1 -- 172.21.15.27:0/2992012007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14340f55e0 msgr2=0x7f1434109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:12:13.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.079+0000 7f1439153700 1 --2- 172.21.15.27:0/2992012007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14340f55e0 0x7f1434109cb0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 rx=0x7f142400c530 tx=0x7f1424007e20).stop 2021-11-03T18:12:13.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.079+0000 7f1439153700 1 -- 172.21.15.27:0/2992012007 shutdown_connections 2021-11-03T18:12:13.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.079+0000 7f1439153700 1 --2- 172.21.15.27:0/2992012007 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f14200640b0 0x7f1420066570 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:13.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.079+0000 7f1439153700 1 --2- 172.21.15.27:0/2992012007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14340f55e0 0x7f1434109cb0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:13.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.079+0000 7f1439153700 1 --2- 172.21.15.27:0/2992012007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14340f5ef0 0x7f143410c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:13.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.079+0000 7f1439153700 1 -- 172.21.15.27:0/2992012007 >> 172.21.15.27:0/2992012007 conn(0x7f14340f0de0 msgr2=0x7f14340fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:12:13.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.079+0000 7f1439153700 1 -- 172.21.15.27:0/2992012007 shutdown_connections 2021-11-03T18:12:13.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.079+0000 7f1439153700 1 -- 172.21.15.27:0/2992012007 wait complete. 2021-11-03T18:12:13.087 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:12:13.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.184+0000 7fd15271d700 1 -- 172.21.15.27:0/2720707024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd14c0f55e0 msgr2=0x7fd14c0f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:12:13.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.184+0000 7fd15271d700 1 --2- 172.21.15.27:0/2720707024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd14c0f55e0 0x7fd14c0f5a00 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 rx=0x7fd13c004660 tx=0x7fd13c009b30).stop 2021-11-03T18:12:13.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.184+0000 7fd15271d700 1 -- 172.21.15.27:0/2720707024 shutdown_connections 2021-11-03T18:12:13.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.184+0000 7fd15271d700 1 --2- 172.21.15.27:0/2720707024 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd14c0f5ef0 0x7fd14c0f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:13.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.184+0000 7fd15271d700 1 --2- 172.21.15.27:0/2720707024 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd14c0f55e0 0x7fd14c0f5a00 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:13.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.184+0000 7fd15271d700 1 -- 172.21.15.27:0/2720707024 >> 172.21.15.27:0/2720707024 conn(0x7fd14c0f0de0 msgr2=0x7fd14c0f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:12:13.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.184+0000 7fd15271d700 1 -- 172.21.15.27:0/2720707024 shutdown_connections 2021-11-03T18:12:13.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.184+0000 7fd15271d700 1 -- 172.21.15.27:0/2720707024 wait complete. 2021-11-03T18:12:13.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.184+0000 7fd15271d700 1 Processor -- start 2021-11-03T18:12:13.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.185+0000 7fd15271d700 1 -- start start 2021-11-03T18:12:13.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.185+0000 7fd15271d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd14c0f55e0 0x7fd14c109c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:12:13.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.185+0000 7fd15271d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd14c0f5ef0 0x7fd14c10c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:12:13.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.185+0000 7fd15271d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c0604e0 con 0x7fd14c0f55e0 2021-11-03T18:12:13.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.185+0000 7fd15271d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd14c060620 con 0x7fd14c0f5ef0 2021-11-03T18:12:13.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.185+0000 7fd14bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd14c0f55e0 0x7fd14c109c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:12:13.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.185+0000 7fd14bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd14c0f55e0 0x7fd14c109c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33520/0 (socket says 172.21.15.27:33520) 2021-11-03T18:12:13.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.185+0000 7fd14bfff700 1 -- 172.21.15.27:0/1957982780 learned_addr learned my addr 172.21.15.27:0/1957982780 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:12:13.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.185+0000 7fd14b7fe700 1 --2- 172.21.15.27:0/1957982780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd14c0f5ef0 0x7fd14c10c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:12:13.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.185+0000 7fd14bfff700 1 -- 172.21.15.27:0/1957982780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd14c0f5ef0 msgr2=0x7fd14c10c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:12:13.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.185+0000 7fd14bfff700 1 --2- 172.21.15.27:0/1957982780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd14c0f5ef0 0x7fd14c10c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:13.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.185+0000 7fd14bfff700 1 -- 172.21.15.27:0/1957982780 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd13c005040 con 0x7fd14c0f55e0 2021-11-03T18:12:13.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.185+0000 7fd14b7fe700 1 --2- 172.21.15.27:0/1957982780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd14c0f5ef0 0x7fd14c10c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:12:13.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.185+0000 7fd14bfff700 1 --2- 172.21.15.27:0/1957982780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd14c0f55e0 0x7fd14c109c10 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 rx=0x7fd13c000c00 tx=0x7fd13c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:12:13.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.186+0000 7fd148ff9700 1 -- 172.21.15.27:0/1957982780 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd13c019070 con 0x7fd14c0f55e0 2021-11-03T18:12:13.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.186+0000 7fd15271d700 1 -- 172.21.15.27:0/1957982780 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd14c10c660 con 0x7fd14c0f55e0 2021-11-03T18:12:13.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.186+0000 7fd148ff9700 1 -- 172.21.15.27:0/1957982780 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fd13c007e70 con 0x7fd14c0f55e0 2021-11-03T18:12:13.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.186+0000 7fd148ff9700 1 -- 172.21.15.27:0/1957982780 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd13c014d90 con 0x7fd14c0f55e0 2021-11-03T18:12:13.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.186+0000 7fd15271d700 1 -- 172.21.15.27:0/1957982780 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd14c10cb90 con 0x7fd14c0f55e0 2021-11-03T18:12:13.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.187+0000 7fd15271d700 1 -- 172.21.15.27:0/1957982780 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd14c045ad0 con 0x7fd14c0f55e0 2021-11-03T18:12:13.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.187+0000 7fd148ff9700 1 -- 172.21.15.27:0/1957982780 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fd13c025070 con 0x7fd14c0f55e0 2021-11-03T18:12:13.196 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.188+0000 7fd148ff9700 1 --2- 172.21.15.27:0/1957982780 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd1380640b0 0x7fd138066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:12:13.196 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.188+0000 7fd148ff9700 1 -- 172.21.15.27:0/1957982780 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fd13c081830 con 0x7fd14c0f55e0 2021-11-03T18:12:13.196 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.188+0000 7fd14b7fe700 1 --2- 172.21.15.27:0/1957982780 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd1380640b0 0x7fd138066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:12:13.196 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.188+0000 7fd14b7fe700 1 --2- 172.21.15.27:0/1957982780 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd1380640b0 0x7fd138066570 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7fd134007fc0 tx=0x7fd13400b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:12:13.197 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.192+0000 7fd148ff9700 1 -- 172.21.15.27:0/1957982780 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fd13c053ad0 con 0x7fd14c0f55e0 2021-11-03T18:12:13.362 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.364+0000 7fd15271d700 1 -- 172.21.15.27:0/1957982780 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd14c057f10 con 0x7fd1380640b0 2021-11-03T18:12:13.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.379+0000 7fd148ff9700 1 -- 172.21.15.27:0/1957982780 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fd14c057f10 con 0x7fd1380640b0 2021-11-03T18:12:13.377 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:12:13.378 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (63m) 2m ago 65m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:12:13.378 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (55m) 2m ago 64m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:12:13.378 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (55m) 2m ago 64m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:12:13.378 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (63m) 2m ago 64m 42.1M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:12:13.379 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (57m) 2m ago 60m 26.3M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:12:13.379 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (60m) 2m ago 60m 3847M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:12:13.379 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (56m) 2m ago 60m 25.9M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:12:13.379 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (57m) 2m ago 60m 27.0M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:12:13.380 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (57m) 2m ago 66m 572M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:12:13.380 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (56m) 2m ago 64m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:12:13.380 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (56m) 2m ago 66m 273M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:12:13.380 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (56m) 2m ago 64m 129M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:12:13.381 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (64m) 2m ago 64m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:12:13.381 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (64m) 2m ago 64m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:12:13.381 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (55m) 2m ago 63m 371M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:12:13.381 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (54m) 2m ago 63m 405M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:12:13.382 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (54m) 2m ago 62m 302M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:12:13.382 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (53m) 2m ago 62m 429M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:12:13.382 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (53m) 2m ago 62m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:12:13.382 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (52m) 2m ago 61m 348M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:12:13.382 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (63m) 2m ago 64m 57.6M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:12:13.383 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.380+0000 7fd15271d700 1 -- 172.21.15.27:0/1957982780 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd1380640b0 msgr2=0x7fd138066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:12:13.384 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.380+0000 7fd15271d700 1 --2- 172.21.15.27:0/1957982780 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd1380640b0 0x7fd138066570 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 rx=0x7fd134007fc0 tx=0x7fd13400b040).stop 2021-11-03T18:12:13.384 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.380+0000 7fd15271d700 1 -- 172.21.15.27:0/1957982780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd14c0f55e0 msgr2=0x7fd14c109c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:12:13.384 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.380+0000 7fd15271d700 1 --2- 172.21.15.27:0/1957982780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd14c0f55e0 0x7fd14c109c10 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 rx=0x7fd13c000c00 tx=0x7fd13c00b040).stop 2021-11-03T18:12:13.384 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.380+0000 7fd15271d700 1 -- 172.21.15.27:0/1957982780 shutdown_connections 2021-11-03T18:12:13.385 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.380+0000 7fd15271d700 1 --2- 172.21.15.27:0/1957982780 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd1380640b0 0x7fd138066570 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:13.385 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.380+0000 7fd15271d700 1 --2- 172.21.15.27:0/1957982780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd14c0f55e0 0x7fd14c109c10 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:13.385 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.380+0000 7fd15271d700 1 --2- 172.21.15.27:0/1957982780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd14c0f5ef0 0x7fd14c10c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:13.385 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.380+0000 7fd15271d700 1 -- 172.21.15.27:0/1957982780 >> 172.21.15.27:0/1957982780 conn(0x7fd14c0f0de0 msgr2=0x7fd14c0fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:12:13.385 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.381+0000 7fd15271d700 1 -- 172.21.15.27:0/1957982780 shutdown_connections 2021-11-03T18:12:13.386 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.381+0000 7fd15271d700 1 -- 172.21.15.27:0/1957982780 wait complete. 2021-11-03T18:12:13.480 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.480+0000 7f381a3ae700 1 -- 172.21.15.27:0/2395900183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38140f6ff0 msgr2=0x7f38140f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:12:13.481 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.480+0000 7f381a3ae700 1 --2- 172.21.15.27:0/2395900183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38140f6ff0 0x7f38140f7a20 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 rx=0x7f3804004660 tx=0x7f3804009b30).stop 2021-11-03T18:12:13.481 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.480+0000 7f381a3ae700 1 -- 172.21.15.27:0/2395900183 shutdown_connections 2021-11-03T18:12:13.481 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.480+0000 7f381a3ae700 1 --2- 172.21.15.27:0/2395900183 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38140f6ff0 0x7f38140f7a20 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:13.481 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.480+0000 7f381a3ae700 1 --2- 172.21.15.27:0/2395900183 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38140f66e0 0x7f38140f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:13.482 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.480+0000 7f381a3ae700 1 -- 172.21.15.27:0/2395900183 >> 172.21.15.27:0/2395900183 conn(0x7f38140f1f00 msgr2=0x7f38140f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:12:13.482 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.480+0000 7f381a3ae700 1 -- 172.21.15.27:0/2395900183 shutdown_connections 2021-11-03T18:12:13.482 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.480+0000 7f381a3ae700 1 -- 172.21.15.27:0/2395900183 wait complete. 2021-11-03T18:12:13.482 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.480+0000 7f381a3ae700 1 Processor -- start 2021-11-03T18:12:13.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.481+0000 7f381a3ae700 1 -- start start 2021-11-03T18:12:13.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.481+0000 7f381a3ae700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38140f66e0 0x7f3814106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:12:13.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.481+0000 7f381a3ae700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38140f6ff0 0x7f3814107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:12:13.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.481+0000 7f381a3ae700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3814107a10 con 0x7f38140f66e0 2021-11-03T18:12:13.484 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.481+0000 7f381a3ae700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3814107b50 con 0x7f38140f6ff0 2021-11-03T18:12:13.484 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.481+0000 7f3813fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38140f66e0 0x7f3814106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:12:13.484 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.481+0000 7f3813fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38140f66e0 0x7f3814106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33530/0 (socket says 172.21.15.27:33530) 2021-11-03T18:12:13.484 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.481+0000 7f3813fff700 1 -- 172.21.15.27:0/3126818026 learned_addr learned my addr 172.21.15.27:0/3126818026 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:12:13.485 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.481+0000 7f38137fe700 1 --2- 172.21.15.27:0/3126818026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38140f6ff0 0x7f3814107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:12:13.485 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.481+0000 7f3813fff700 1 -- 172.21.15.27:0/3126818026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38140f6ff0 msgr2=0x7f3814107460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:12:13.485 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.481+0000 7f3813fff700 1 --2- 172.21.15.27:0/3126818026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38140f6ff0 0x7f3814107460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:13.486 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.481+0000 7f3813fff700 1 -- 172.21.15.27:0/3126818026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3804005040 con 0x7f38140f66e0 2021-11-03T18:12:13.486 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.481+0000 7f38137fe700 1 --2- 172.21.15.27:0/3126818026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38140f6ff0 0x7f3814107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:12:13.486 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.481+0000 7f3813fff700 1 --2- 172.21.15.27:0/3126818026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38140f66e0 0x7f3814106f70 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 rx=0x7f38080026d0 tx=0x7f3808009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:12:13.486 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.481+0000 7f3810ff9700 1 -- 172.21.15.27:0/3126818026 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3808009320 con 0x7f38140f66e0 2021-11-03T18:12:13.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.482+0000 7f3810ff9700 1 -- 172.21.15.27:0/3126818026 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f380800f040 con 0x7f38140f66e0 2021-11-03T18:12:13.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.482+0000 7f381a3ae700 1 -- 172.21.15.27:0/3126818026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38141084f0 con 0x7f38140f66e0 2021-11-03T18:12:13.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.482+0000 7f3810ff9700 1 -- 172.21.15.27:0/3126818026 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38080147e0 con 0x7f38140f66e0 2021-11-03T18:12:13.488 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.482+0000 7f381a3ae700 1 -- 172.21.15.27:0/3126818026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3814108ab0 con 0x7f38140f66e0 2021-11-03T18:12:13.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.483+0000 7f381a3ae700 1 -- 172.21.15.27:0/3126818026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3814045ad0 con 0x7f38140f66e0 2021-11-03T18:12:13.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.483+0000 7f3810ff9700 1 -- 172.21.15.27:0/3126818026 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f3808009480 con 0x7f38140f66e0 2021-11-03T18:12:13.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.483+0000 7f3810ff9700 1 --2- 172.21.15.27:0/3126818026 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f37fc0640b0 0x7f37fc066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:12:13.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.483+0000 7f3810ff9700 1 -- 172.21.15.27:0/3126818026 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f3808080f10 con 0x7f38140f66e0 2021-11-03T18:12:13.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.484+0000 7f38137fe700 1 --2- 172.21.15.27:0/3126818026 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f37fc0640b0 0x7f37fc066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:12:13.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.484+0000 7f38137fe700 1 --2- 172.21.15.27:0/3126818026 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f37fc0640b0 0x7f37fc066570 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f3804004630 tx=0x7f3804009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:12:13.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.487+0000 7f3810ff9700 1 -- 172.21.15.27:0/3126818026 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f3808053210 con 0x7f38140f66e0 2021-11-03T18:12:13.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.720+0000 7f381a3ae700 1 -- 172.21.15.27:0/3126818026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f38140fefe0 con 0x7f38140f66e0 2021-11-03T18:12:13.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.720+0000 7f3810ff9700 1 -- 172.21.15.27:0/3126818026 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f3808052e00 con 0x7f38140f66e0 2021-11-03T18:12:13.720 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:12:13.720 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:12:13.720 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:12:13.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:12:13.721 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:12:13.721 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:12:13.721 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:12:13.721 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:12:13.722 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:12:13.722 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:12:13.722 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:12:13.723 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:12:13.723 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:12:13.723 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:12:13.723 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:12:13.724 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:12:13.724 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:12:13.724 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:12:13.725 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.722+0000 7f381a3ae700 1 -- 172.21.15.27:0/3126818026 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f37fc0640b0 msgr2=0x7f37fc066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:12:13.725 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.722+0000 7f381a3ae700 1 --2- 172.21.15.27:0/3126818026 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f37fc0640b0 0x7f37fc066570 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7f3804004630 tx=0x7f3804009b30).stop 2021-11-03T18:12:13.726 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.722+0000 7f381a3ae700 1 -- 172.21.15.27:0/3126818026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38140f66e0 msgr2=0x7f3814106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:12:13.726 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.722+0000 7f381a3ae700 1 --2- 172.21.15.27:0/3126818026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38140f66e0 0x7f3814106f70 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 rx=0x7f38080026d0 tx=0x7f3808009de0).stop 2021-11-03T18:12:13.726 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.722+0000 7f381a3ae700 1 -- 172.21.15.27:0/3126818026 shutdown_connections 2021-11-03T18:12:13.726 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.722+0000 7f381a3ae700 1 --2- 172.21.15.27:0/3126818026 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f37fc0640b0 0x7f37fc066570 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:13.727 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.722+0000 7f381a3ae700 1 --2- 172.21.15.27:0/3126818026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38140f66e0 0x7f3814106f70 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:13.727 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.722+0000 7f381a3ae700 1 --2- 172.21.15.27:0/3126818026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38140f6ff0 0x7f3814107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:13.727 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.722+0000 7f381a3ae700 1 -- 172.21.15.27:0/3126818026 >> 172.21.15.27:0/3126818026 conn(0x7f38140f1f00 msgr2=0x7f38140fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:12:13.727 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.722+0000 7f381a3ae700 1 -- 172.21.15.27:0/3126818026 shutdown_connections 2021-11-03T18:12:13.728 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.722+0000 7f381a3ae700 1 -- 172.21.15.27:0/3126818026 wait complete. 2021-11-03T18:12:13.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.822+0000 7f74284ca700 1 -- 172.21.15.27:0/1648093663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74200f5ef0 msgr2=0x7f74200f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:12:13.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.822+0000 7f74284ca700 1 --2- 172.21.15.27:0/1648093663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74200f5ef0 0x7f74200f6920 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 rx=0x7f7410004660 tx=0x7f7410009b30).stop 2021-11-03T18:12:13.823 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.822+0000 7f74284ca700 1 -- 172.21.15.27:0/1648093663 shutdown_connections 2021-11-03T18:12:13.823 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.822+0000 7f74284ca700 1 --2- 172.21.15.27:0/1648093663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74200f5ef0 0x7f74200f6920 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:13.823 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.822+0000 7f74284ca700 1 --2- 172.21.15.27:0/1648093663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74200f55e0 0x7f74200f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:13.823 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.822+0000 7f74284ca700 1 -- 172.21.15.27:0/1648093663 >> 172.21.15.27:0/1648093663 conn(0x7f74200f0de0 msgr2=0x7f74200f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:12:13.824 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.822+0000 7f74284ca700 1 -- 172.21.15.27:0/1648093663 shutdown_connections 2021-11-03T18:12:13.824 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.822+0000 7f74284ca700 1 -- 172.21.15.27:0/1648093663 wait complete. 2021-11-03T18:12:13.824 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.822+0000 7f74284ca700 1 Processor -- start 2021-11-03T18:12:13.825 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.822+0000 7f74284ca700 1 -- start start 2021-11-03T18:12:13.825 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.822+0000 7f74284ca700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74200f55e0 0x7f7420106f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:12:13.825 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.822+0000 7f74284ca700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74200f5ef0 0x7f7420107470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:12:13.825 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.823+0000 7f74284ca700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7420107a20 con 0x7f74200f55e0 2021-11-03T18:12:13.826 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.823+0000 7f74284ca700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7420107b60 con 0x7f74200f5ef0 2021-11-03T18:12:13.826 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.823+0000 7f7426266700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74200f55e0 0x7f7420106f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:12:13.826 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.823+0000 7f7426266700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74200f55e0 0x7f7420106f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33540/0 (socket says 172.21.15.27:33540) 2021-11-03T18:12:13.826 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.823+0000 7f7426266700 1 -- 172.21.15.27:0/504037833 learned_addr learned my addr 172.21.15.27:0/504037833 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:12:13.827 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.823+0000 7f7425a65700 1 --2- 172.21.15.27:0/504037833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74200f5ef0 0x7f7420107470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:12:13.827 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.823+0000 7f7426266700 1 -- 172.21.15.27:0/504037833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74200f5ef0 msgr2=0x7f7420107470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:12:13.827 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.823+0000 7f7426266700 1 --2- 172.21.15.27:0/504037833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74200f5ef0 0x7f7420107470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:13.827 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.823+0000 7f7426266700 1 -- 172.21.15.27:0/504037833 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7410005040 con 0x7f74200f55e0 2021-11-03T18:12:13.828 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.823+0000 7f7426266700 1 --2- 172.21.15.27:0/504037833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74200f55e0 0x7f7420106f80 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 rx=0x7f741800c7c0 tx=0x7f7418004d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:12:13.828 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.823+0000 7f7416ffd700 1 -- 172.21.15.27:0/504037833 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f741800e680 con 0x7f74200f55e0 2021-11-03T18:12:13.828 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.823+0000 7f74284ca700 1 -- 172.21.15.27:0/504037833 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7420108500 con 0x7f74200f55e0 2021-11-03T18:12:13.828 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.823+0000 7f7416ffd700 1 -- 172.21.15.27:0/504037833 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f741800ec90 con 0x7f74200f55e0 2021-11-03T18:12:13.828 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.823+0000 7f7416ffd700 1 -- 172.21.15.27:0/504037833 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7418012d10 con 0x7f74200f55e0 2021-11-03T18:12:13.829 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.823+0000 7f74284ca700 1 -- 172.21.15.27:0/504037833 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7420108ac0 con 0x7f74200f55e0 2021-11-03T18:12:13.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.824+0000 7f74284ca700 1 -- 172.21.15.27:0/504037833 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7420045ad0 con 0x7f74200f55e0 2021-11-03T18:12:13.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.825+0000 7f7416ffd700 1 -- 172.21.15.27:0/504037833 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f7418019030 con 0x7f74200f55e0 2021-11-03T18:12:13.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.826+0000 7f7416ffd700 1 --2- 172.21.15.27:0/504037833 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f740c0640b0 0x7f740c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:12:13.831 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.826+0000 7f7416ffd700 1 -- 172.21.15.27:0/504037833 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f7418081200 con 0x7f74200f55e0 2021-11-03T18:12:13.831 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.826+0000 7f7425a65700 1 --2- 172.21.15.27:0/504037833 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f740c0640b0 0x7f740c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:12:13.831 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.826+0000 7f7425a65700 1 --2- 172.21.15.27:0/504037833 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f740c0640b0 0x7f740c066570 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7f7410004630 tx=0x7f7410009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:12:13.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:13.829+0000 7f7416ffd700 1 -- 172.21.15.27:0/504037833 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f7418053480 con 0x7f74200f55e0 2021-11-03T18:12:14.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:14.033+0000 7f74284ca700 1 -- 172.21.15.27:0/504037833 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f74200ff010 con 0x7f74200f55e0 2021-11-03T18:12:14.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:14.034+0000 7f7416ffd700 1 -- 172.21.15.27:0/504037833 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f7418021710 con 0x7f74200f55e0 2021-11-03T18:12:14.033 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:12:14.034 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:12:14.034 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:12:14.034 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:12:14.034 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:12:14.035 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:12:14.035 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:12:14.035 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:12:14.035 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:12:14.035 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:12:14.036 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:12:14.037 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:12:14.038 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:12:14.038 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:12:14.038 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:12:14.039 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:12:14.039 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:12:14.039 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:12:14.039 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:12:14.040 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:12:14.040 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:12:14.040 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:12:14.040 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:12:14.041 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:12:14.041 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:12:14.042 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:12:14.043 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:12:14.043 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:12:14.044 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:12:14.044 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:12:14.045 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:12:14.045 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:12:14.045 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:12:14.047 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:12:14.047 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:12:14.048 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:12:14.048 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:12:14.049 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:12:14.049 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:12:14.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:14.036+0000 7f74284ca700 1 -- 172.21.15.27:0/504037833 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f740c0640b0 msgr2=0x7f740c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:12:14.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:14.036+0000 7f74284ca700 1 --2- 172.21.15.27:0/504037833 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f740c0640b0 0x7f740c066570 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7f7410004630 tx=0x7f7410009b30).stop 2021-11-03T18:12:14.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:14.036+0000 7f74284ca700 1 -- 172.21.15.27:0/504037833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74200f55e0 msgr2=0x7f7420106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:12:14.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:14.036+0000 7f74284ca700 1 --2- 172.21.15.27:0/504037833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74200f55e0 0x7f7420106f80 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 rx=0x7f741800c7c0 tx=0x7f7418004d70).stop 2021-11-03T18:12:14.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:14.036+0000 7f74284ca700 1 -- 172.21.15.27:0/504037833 shutdown_connections 2021-11-03T18:12:14.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:14.036+0000 7f74284ca700 1 --2- 172.21.15.27:0/504037833 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f740c0640b0 0x7f740c066570 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:14.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:14.036+0000 7f74284ca700 1 --2- 172.21.15.27:0/504037833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74200f55e0 0x7f7420106f80 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:14.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:14.036+0000 7f74284ca700 1 --2- 172.21.15.27:0/504037833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74200f5ef0 0x7f7420107470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:14.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:14.036+0000 7f74284ca700 1 -- 172.21.15.27:0/504037833 >> 172.21.15.27:0/504037833 conn(0x7f74200f0de0 msgr2=0x7f74200fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:12:14.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:14.036+0000 7f74284ca700 1 -- 172.21.15.27:0/504037833 shutdown_connections 2021-11-03T18:12:14.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:14.036+0000 7f74284ca700 1 -- 172.21.15.27:0/504037833 wait complete. 2021-11-03T18:12:14.054 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:12:14.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:14 smithi115 ceph-mon[65453]: pgmap v1727: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:14.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:14 smithi115 ceph-mon[65453]: from='client.35880 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:12:14.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:14 smithi115 ceph-mon[65453]: from='client.35884 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:12:14.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:14 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3126818026' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:14.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:14 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/504037833' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:12:14.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:14 smithi027 ceph-mon[97244]: pgmap v1727: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:14.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:14 smithi027 ceph-mon[97244]: from='client.35880 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:12:14.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:14 smithi027 ceph-mon[97244]: from='client.35884 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:12:14.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:14 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3126818026' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:14.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:14 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/504037833' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:12:15.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3377.440457 secs 2021-11-03T18:12:15.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3377.440457 secs 2021-11-03T18:12:16.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:16 smithi115 ceph-mon[65453]: pgmap v1728: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:16.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:16 smithi027 ceph-mon[97244]: pgmap v1728: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:18.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:18 smithi115 ceph-mon[65453]: pgmap v1729: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:18.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:12:18.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:12:18.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:12:18.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:18.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:18.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:18.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:18.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:18 smithi027 ceph-mon[97244]: pgmap v1729: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:18.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:12:18.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:12:18.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:12:18.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:18.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:18.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:18.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:19.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:19 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:12:19.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:19 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:12:20.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:20 smithi115 ceph-mon[65453]: pgmap v1730: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:20.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3382.440596 secs 2021-11-03T18:12:20.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:20 smithi027 ceph-mon[97244]: pgmap v1730: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:20.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3382.440596 secs 2021-11-03T18:12:22.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:22 smithi115 ceph-mon[65453]: pgmap v1731: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:22.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:22 smithi027 ceph-mon[97244]: pgmap v1731: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:24.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:24 smithi115 ceph-mon[65453]: pgmap v1732: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:24.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:24 smithi027 ceph-mon[97244]: pgmap v1732: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:25.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3387.440677 secs 2021-11-03T18:12:25.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3387.440677 secs 2021-11-03T18:12:26.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:26 smithi115 ceph-mon[65453]: pgmap v1733: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:26.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:26 smithi027 ceph-mon[97244]: pgmap v1733: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:28.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:28 smithi115 ceph-mon[65453]: pgmap v1734: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:28.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:12:28.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:12:28.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:12:28.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:28.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:28.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:28.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:28.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:28 smithi027 ceph-mon[97244]: pgmap v1734: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:28.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:12:28.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:12:28.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:12:28.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:28.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:28.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:28.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:29.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:29 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:12:29.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:29 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:12:30.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:30 smithi115 ceph-mon[65453]: pgmap v1735: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:30.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3392.440745 secs 2021-11-03T18:12:30.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:30 smithi027 ceph-mon[97244]: pgmap v1735: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:30.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3392.440745 secs 2021-11-03T18:12:32.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:32 smithi115 ceph-mon[65453]: pgmap v1736: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:32.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:32 smithi027 ceph-mon[97244]: pgmap v1736: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:34.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:34 smithi115 ceph-mon[65453]: pgmap v1737: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:34.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:34 smithi027 ceph-mon[97244]: pgmap v1737: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:35.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3397.440885 secs 2021-11-03T18:12:35.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3397.440885 secs 2021-11-03T18:12:36.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:36 smithi115 ceph-mon[65453]: pgmap v1738: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:36.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:36 smithi027 ceph-mon[97244]: pgmap v1738: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:38.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:38 smithi115 ceph-mon[65453]: pgmap v1739: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:38.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:12:38.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:12:38.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:12:38.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:38.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:38.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:38.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:38.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:38 smithi027 ceph-mon[97244]: pgmap v1739: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:38.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:12:38.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:12:38.776 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:12:38.776 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:38.776 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:38.776 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:38.776 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:39.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:39 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:12:39.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:39 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:12:40.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:40 smithi115 ceph-mon[65453]: pgmap v1740: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:40.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3402.441025 secs 2021-11-03T18:12:40.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:40 smithi027 ceph-mon[97244]: pgmap v1740: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:40.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3402.441025 secs 2021-11-03T18:12:42.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:42 smithi115 ceph-mon[65453]: pgmap v1741: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:42.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:42 smithi027 ceph-mon[97244]: pgmap v1741: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:44.149 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.150+0000 7f2992806700 1 -- 172.21.15.27:0/2309295182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f298c0f6ff0 msgr2=0x7f298c0f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:12:44.149 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.150+0000 7f2992806700 1 --2- 172.21.15.27:0/2309295182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f298c0f6ff0 0x7f298c0f7a20 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 rx=0x7f2974004660 tx=0x7f2974009b30).stop 2021-11-03T18:12:44.149 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.150+0000 7f2992806700 1 -- 172.21.15.27:0/2309295182 shutdown_connections 2021-11-03T18:12:44.150 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.150+0000 7f2992806700 1 --2- 172.21.15.27:0/2309295182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f298c0f6ff0 0x7f298c0f7a20 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:44.150 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.150+0000 7f2992806700 1 --2- 172.21.15.27:0/2309295182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f298c0f66e0 0x7f298c0f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:44.150 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.150+0000 7f2992806700 1 -- 172.21.15.27:0/2309295182 >> 172.21.15.27:0/2309295182 conn(0x7f298c0f1f00 msgr2=0x7f298c0f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:12:44.150 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.150+0000 7f2992806700 1 -- 172.21.15.27:0/2309295182 shutdown_connections 2021-11-03T18:12:44.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.150+0000 7f2992806700 1 -- 172.21.15.27:0/2309295182 wait complete. 2021-11-03T18:12:44.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.151+0000 7f2992806700 1 Processor -- start 2021-11-03T18:12:44.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.151+0000 7f2992806700 1 -- start start 2021-11-03T18:12:44.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.151+0000 7f2992806700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f298c0f66e0 0x7f298c06b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:12:44.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.151+0000 7f2992806700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f298c0f6ff0 0x7f298c069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:12:44.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.151+0000 7f2992806700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298c06a380 con 0x7f298c0f6ff0 2021-11-03T18:12:44.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.151+0000 7f2992806700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298c06a4c0 con 0x7f298c0f66e0 2021-11-03T18:12:44.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.151+0000 7f298b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f298c0f6ff0 0x7f298c069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:12:44.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.151+0000 7f298b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f298c0f6ff0 0x7f298c069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33560/0 (socket says 172.21.15.27:33560) 2021-11-03T18:12:44.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.151+0000 7f298b7fe700 1 -- 172.21.15.27:0/2085816581 learned_addr learned my addr 172.21.15.27:0/2085816581 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:12:44.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.151+0000 7f298bfff700 1 --2- 172.21.15.27:0/2085816581 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f298c0f66e0 0x7f298c06b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:12:44.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.151+0000 7f298b7fe700 1 -- 172.21.15.27:0/2085816581 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f298c0f66e0 msgr2=0x7f298c06b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:12:44.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.151+0000 7f298b7fe700 1 --2- 172.21.15.27:0/2085816581 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f298c0f66e0 0x7f298c06b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:44.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.152+0000 7f298b7fe700 1 -- 172.21.15.27:0/2085816581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2974005040 con 0x7f298c0f6ff0 2021-11-03T18:12:44.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.152+0000 7f298b7fe700 1 --2- 172.21.15.27:0/2085816581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f298c0f6ff0 0x7f298c069e20 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 rx=0x7f2974009990 tx=0x7f2974009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:12:44.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.152+0000 7f2988ff9700 1 -- 172.21.15.27:0/2085816581 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2974018070 con 0x7f298c0f6ff0 2021-11-03T18:12:44.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.152+0000 7f2988ff9700 1 -- 172.21.15.27:0/2085816581 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f2974007ba0 con 0x7f298c0f6ff0 2021-11-03T18:12:44.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.152+0000 7f2992806700 1 -- 172.21.15.27:0/2085816581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f298c06a740 con 0x7f298c0f6ff0 2021-11-03T18:12:44.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.152+0000 7f2988ff9700 1 -- 172.21.15.27:0/2085816581 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f297401c3c0 con 0x7f298c0f6ff0 2021-11-03T18:12:44.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.152+0000 7f2992806700 1 -- 172.21.15.27:0/2085816581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f298c06aca0 con 0x7f298c0f6ff0 2021-11-03T18:12:44.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.153+0000 7f2992806700 1 -- 172.21.15.27:0/2085816581 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f298c045ad0 con 0x7f298c0f6ff0 2021-11-03T18:12:44.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.153+0000 7f2988ff9700 1 -- 172.21.15.27:0/2085816581 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f297401d420 con 0x7f298c0f6ff0 2021-11-03T18:12:44.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.154+0000 7f2988ff9700 1 --2- 172.21.15.27:0/2085816581 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f29780640b0 0x7f2978066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:12:44.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.154+0000 7f2988ff9700 1 -- 172.21.15.27:0/2085816581 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f2974082470 con 0x7f298c0f6ff0 2021-11-03T18:12:44.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.154+0000 7f298bfff700 1 --2- 172.21.15.27:0/2085816581 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f29780640b0 0x7f2978066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:12:44.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.154+0000 7f298bfff700 1 --2- 172.21.15.27:0/2085816581 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f29780640b0 0x7f2978066570 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7f298c06b730 tx=0x7f297c00b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:12:44.160 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.158+0000 7f2988ff9700 1 -- 172.21.15.27:0/2085816581 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f2974054830 con 0x7f298c0f6ff0 2021-11-03T18:12:44.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.336+0000 7f2992806700 1 -- 172.21.15.27:0/2085816581 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f298c0ff040 con 0x7f29780640b0 2021-11-03T18:12:44.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.338+0000 7f2988ff9700 1 -- 172.21.15.27:0/2085816581 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f298c0ff040 con 0x7f29780640b0 2021-11-03T18:12:44.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.339+0000 7f2992806700 1 -- 172.21.15.27:0/2085816581 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f29780640b0 msgr2=0x7f2978066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:12:44.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.339+0000 7f2992806700 1 --2- 172.21.15.27:0/2085816581 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f29780640b0 0x7f2978066570 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7f298c06b730 tx=0x7f297c00b040).stop 2021-11-03T18:12:44.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.339+0000 7f2992806700 1 -- 172.21.15.27:0/2085816581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f298c0f6ff0 msgr2=0x7f298c069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:12:44.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.339+0000 7f2992806700 1 --2- 172.21.15.27:0/2085816581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f298c0f6ff0 0x7f298c069e20 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 rx=0x7f2974009990 tx=0x7f2974009b30).stop 2021-11-03T18:12:44.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.340+0000 7f2992806700 1 -- 172.21.15.27:0/2085816581 shutdown_connections 2021-11-03T18:12:44.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.340+0000 7f2992806700 1 --2- 172.21.15.27:0/2085816581 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f29780640b0 0x7f2978066570 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:44.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.340+0000 7f2992806700 1 --2- 172.21.15.27:0/2085816581 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f298c0f66e0 0x7f298c06b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:44.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.340+0000 7f2992806700 1 --2- 172.21.15.27:0/2085816581 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f298c0f6ff0 0x7f298c069e20 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:44.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.340+0000 7f2992806700 1 -- 172.21.15.27:0/2085816581 >> 172.21.15.27:0/2085816581 conn(0x7f298c0f1f00 msgr2=0x7f298c0fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:12:44.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.340+0000 7f2992806700 1 -- 172.21.15.27:0/2085816581 shutdown_connections 2021-11-03T18:12:44.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.340+0000 7f2992806700 1 -- 172.21.15.27:0/2085816581 wait complete. 2021-11-03T18:12:44.348 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:12:44.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.440+0000 7f7dbb27d700 1 -- 172.21.15.27:0/3953581981 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7db40f6a10 msgr2=0x7f7db40f6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:12:44.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.440+0000 7f7dbb27d700 1 --2- 172.21.15.27:0/3953581981 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7db40f6a10 0x7f7db40f6e30 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 rx=0x7f7db0004660 tx=0x7f7db0009b30).stop 2021-11-03T18:12:44.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.440+0000 7f7dbb27d700 1 -- 172.21.15.27:0/3953581981 shutdown_connections 2021-11-03T18:12:44.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.440+0000 7f7dbb27d700 1 --2- 172.21.15.27:0/3953581981 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7db40f5710 0x7f7db40f5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:44.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.440+0000 7f7dbb27d700 1 --2- 172.21.15.27:0/3953581981 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7db40f6a10 0x7f7db40f6e30 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:44.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.440+0000 7f7dbb27d700 1 -- 172.21.15.27:0/3953581981 >> 172.21.15.27:0/3953581981 conn(0x7f7db40f0320 msgr2=0x7f7db40f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:12:44.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.440+0000 7f7dbb27d700 1 -- 172.21.15.27:0/3953581981 shutdown_connections 2021-11-03T18:12:44.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.440+0000 7f7dbb27d700 1 -- 172.21.15.27:0/3953581981 wait complete. 2021-11-03T18:12:44.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.441+0000 7f7dbb27d700 1 Processor -- start 2021-11-03T18:12:44.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.441+0000 7f7dbb27d700 1 -- start start 2021-11-03T18:12:44.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.441+0000 7f7dbb27d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7db40f5710 0x7f7db4109a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:12:44.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.441+0000 7f7dbb27d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7db40f6a10 0x7f7db410bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:12:44.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.441+0000 7f7dbb27d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db40f53c0 con 0x7f7db40f5710 2021-11-03T18:12:44.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.441+0000 7f7dbb27d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db40f5500 con 0x7f7db40f6a10 2021-11-03T18:12:44.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.441+0000 7f7dba27b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7db40f5710 0x7f7db4109a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:12:44.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.441+0000 7f7dba27b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7db40f5710 0x7f7db4109a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33568/0 (socket says 172.21.15.27:33568) 2021-11-03T18:12:44.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.441+0000 7f7dba27b700 1 -- 172.21.15.27:0/3196094729 learned_addr learned my addr 172.21.15.27:0/3196094729 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:12:44.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.441+0000 7f7db9a7a700 1 --2- 172.21.15.27:0/3196094729 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7db40f6a10 0x7f7db410bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:12:44.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.442+0000 7f7dba27b700 1 -- 172.21.15.27:0/3196094729 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7db40f6a10 msgr2=0x7f7db410bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:12:44.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.442+0000 7f7dba27b700 1 --2- 172.21.15.27:0/3196094729 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7db40f6a10 0x7f7db410bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:44.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.442+0000 7f7dba27b700 1 -- 172.21.15.27:0/3196094729 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db0005040 con 0x7f7db40f5710 2021-11-03T18:12:44.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.442+0000 7f7dba27b700 1 --2- 172.21.15.27:0/3196094729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7db40f5710 0x7f7db4109a70 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 rx=0x7f7db0000c00 tx=0x7f7db0009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:12:44.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.442+0000 7f7daaffd700 1 -- 172.21.15.27:0/3196094729 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7db0018070 con 0x7f7db40f5710 2021-11-03T18:12:44.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.442+0000 7f7daaffd700 1 -- 172.21.15.27:0/3196094729 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f7db0007d70 con 0x7f7db40f5710 2021-11-03T18:12:44.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.442+0000 7f7daaffd700 1 -- 172.21.15.27:0/3196094729 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7db001c620 con 0x7f7db40f5710 2021-11-03T18:12:44.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.442+0000 7f7dbb27d700 1 -- 172.21.15.27:0/3196094729 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7db410c4c0 con 0x7f7db40f5710 2021-11-03T18:12:44.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.442+0000 7f7dbb27d700 1 -- 172.21.15.27:0/3196094729 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7db410ca20 con 0x7f7db40f5710 2021-11-03T18:12:44.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.443+0000 7f7dbb27d700 1 -- 172.21.15.27:0/3196094729 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7db4045ad0 con 0x7f7db40f5710 2021-11-03T18:12:44.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.443+0000 7f7daaffd700 1 -- 172.21.15.27:0/3196094729 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f7db001d420 con 0x7f7db40f5710 2021-11-03T18:12:44.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.444+0000 7f7daaffd700 1 --2- 172.21.15.27:0/3196094729 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7da0064100 0x7f7da00665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:12:44.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.444+0000 7f7daaffd700 1 -- 172.21.15.27:0/3196094729 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f7db00825f0 con 0x7f7db40f5710 2021-11-03T18:12:44.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.444+0000 7f7db9a7a700 1 --2- 172.21.15.27:0/3196094729 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7da0064100 0x7f7da00665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:12:44.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.444+0000 7f7db9a7a700 1 --2- 172.21.15.27:0/3196094729 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7da0064100 0x7f7da00665c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f7da4007fd0 tx=0x7f7da4009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:12:44.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.448+0000 7f7daaffd700 1 -- 172.21.15.27:0/3196094729 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f7db00549b0 con 0x7f7db40f5710 2021-11-03T18:12:44.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.624+0000 7f7dbb27d700 1 -- 172.21.15.27:0/3196094729 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7db40fdd40 con 0x7f7da0064100 2021-11-03T18:12:44.637 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.638+0000 7f7daaffd700 1 -- 172.21.15.27:0/3196094729 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f7db40fdd40 con 0x7f7da0064100 2021-11-03T18:12:44.637 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:12:44.637 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (64m) 3m ago 65m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:12:44.637 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (56m) 3m ago 65m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:12:44.638 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (55m) 2m ago 64m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:12:44.638 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (64m) 3m ago 65m 42.1M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:12:44.638 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (57m) 3m ago 60m 26.3M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:12:44.638 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (60m) 3m ago 60m 3847M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:12:44.639 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (56m) 2m ago 60m 25.9M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:12:44.639 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (58m) 2m ago 60m 27.0M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:12:44.639 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (58m) 3m ago 66m 572M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:12:44.640 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (57m) 2m ago 64m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:12:44.640 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (57m) 3m ago 66m 273M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:12:44.640 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (56m) 2m ago 64m 129M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:12:44.640 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (65m) 3m ago 65m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:12:44.641 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (64m) 2m ago 64m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:12:44.641 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (55m) 3m ago 63m 371M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:12:44.641 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (55m) 3m ago 63m 405M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:12:44.641 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (54m) 3m ago 63m 302M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:12:44.642 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (54m) 2m ago 62m 429M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:12:44.642 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (53m) 2m ago 62m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:12:44.642 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (53m) 2m ago 62m 348M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:12:44.642 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (64m) 3m ago 64m 57.6M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:12:44.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.640+0000 7f7dbb27d700 1 -- 172.21.15.27:0/3196094729 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7da0064100 msgr2=0x7f7da00665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:12:44.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.640+0000 7f7dbb27d700 1 --2- 172.21.15.27:0/3196094729 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7da0064100 0x7f7da00665c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7f7da4007fd0 tx=0x7f7da4009040).stop 2021-11-03T18:12:44.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.640+0000 7f7dbb27d700 1 -- 172.21.15.27:0/3196094729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7db40f5710 msgr2=0x7f7db4109a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:12:44.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.640+0000 7f7dbb27d700 1 --2- 172.21.15.27:0/3196094729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7db40f5710 0x7f7db4109a70 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 rx=0x7f7db0000c00 tx=0x7f7db0009b30).stop 2021-11-03T18:12:44.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.640+0000 7f7dbb27d700 1 -- 172.21.15.27:0/3196094729 shutdown_connections 2021-11-03T18:12:44.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.640+0000 7f7dbb27d700 1 --2- 172.21.15.27:0/3196094729 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7da0064100 0x7f7da00665c0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:44.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.640+0000 7f7dbb27d700 1 --2- 172.21.15.27:0/3196094729 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7db40f5710 0x7f7db4109a70 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:44.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.640+0000 7f7dbb27d700 1 --2- 172.21.15.27:0/3196094729 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7db40f6a10 0x7f7db410bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:44.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.640+0000 7f7dbb27d700 1 -- 172.21.15.27:0/3196094729 >> 172.21.15.27:0/3196094729 conn(0x7f7db40f0320 msgr2=0x7f7db40fa170 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:12:44.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.640+0000 7f7dbb27d700 1 -- 172.21.15.27:0/3196094729 shutdown_connections 2021-11-03T18:12:44.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.640+0000 7f7dbb27d700 1 -- 172.21.15.27:0/3196094729 wait complete. 2021-11-03T18:12:44.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:44 smithi115 ceph-mon[65453]: pgmap v1742: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:44.739 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.740+0000 7fd354ca0700 1 -- 172.21.15.27:0/2854804407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3500f7980 msgr2=0x7fd3500f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:12:44.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.740+0000 7fd354ca0700 1 --2- 172.21.15.27:0/2854804407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3500f7980 0x7fd3500f7de0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 rx=0x7fd344004660 tx=0x7fd344009b30).stop 2021-11-03T18:12:44.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.740+0000 7fd354ca0700 1 -- 172.21.15.27:0/2854804407 shutdown_connections 2021-11-03T18:12:44.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.740+0000 7fd354ca0700 1 --2- 172.21.15.27:0/2854804407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3500f7980 0x7fd3500f7de0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:44.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.740+0000 7fd354ca0700 1 --2- 172.21.15.27:0/2854804407 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd3500f66e0 0x7fd3500f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:44.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.740+0000 7fd354ca0700 1 -- 172.21.15.27:0/2854804407 >> 172.21.15.27:0/2854804407 conn(0x7fd3500f1e40 msgr2=0x7fd3500f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:12:44.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.740+0000 7fd354ca0700 1 -- 172.21.15.27:0/2854804407 shutdown_connections 2021-11-03T18:12:44.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.740+0000 7fd354ca0700 1 -- 172.21.15.27:0/2854804407 wait complete. 2021-11-03T18:12:44.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.740+0000 7fd354ca0700 1 Processor -- start 2021-11-03T18:12:44.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.740+0000 7fd354ca0700 1 -- start start 2021-11-03T18:12:44.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.741+0000 7fd354ca0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3500f66e0 0x7fd35006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:12:44.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.741+0000 7fd354ca0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd3500f7980 0x7fd350069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:12:44.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.741+0000 7fd354ca0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35006a380 con 0x7fd3500f66e0 2021-11-03T18:12:44.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.741+0000 7fd354ca0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd35006a4c0 con 0x7fd3500f7980 2021-11-03T18:12:44.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.741+0000 7fd34e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3500f66e0 0x7fd35006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:12:44.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.741+0000 7fd34e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3500f66e0 0x7fd35006b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33578/0 (socket says 172.21.15.27:33578) 2021-11-03T18:12:44.744 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.741+0000 7fd34e59c700 1 -- 172.21.15.27:0/903996006 learned_addr learned my addr 172.21.15.27:0/903996006 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:12:44.744 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.741+0000 7fd34dd9b700 1 --2- 172.21.15.27:0/903996006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd3500f7980 0x7fd350069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:12:44.744 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.741+0000 7fd34e59c700 1 -- 172.21.15.27:0/903996006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd3500f7980 msgr2=0x7fd350069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:12:44.744 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.741+0000 7fd34e59c700 1 --2- 172.21.15.27:0/903996006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd3500f7980 0x7fd350069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:44.745 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.741+0000 7fd34e59c700 1 -- 172.21.15.27:0/903996006 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd344005040 con 0x7fd3500f66e0 2021-11-03T18:12:44.745 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.741+0000 7fd34e59c700 1 --2- 172.21.15.27:0/903996006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3500f66e0 0x7fd35006b9a0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 rx=0x7fd34800c710 tx=0x7fd348004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:12:44.745 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.741+0000 7fd33effd700 1 -- 172.21.15.27:0/903996006 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd3480106f0 con 0x7fd3500f66e0 2021-11-03T18:12:44.745 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.741+0000 7fd33effd700 1 -- 172.21.15.27:0/903996006 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fd348010d00 con 0x7fd3500f66e0 2021-11-03T18:12:44.745 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.741+0000 7fd354ca0700 1 -- 172.21.15.27:0/903996006 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd35006a7a0 con 0x7fd3500f66e0 2021-11-03T18:12:44.746 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.741+0000 7fd33effd700 1 -- 172.21.15.27:0/903996006 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd348012d90 con 0x7fd3500f66e0 2021-11-03T18:12:44.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.741+0000 7fd354ca0700 1 -- 172.21.15.27:0/903996006 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd35010ef90 con 0x7fd3500f66e0 2021-11-03T18:12:44.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.742+0000 7fd354ca0700 1 -- 172.21.15.27:0/903996006 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd350045ad0 con 0x7fd3500f66e0 2021-11-03T18:12:44.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.743+0000 7fd33effd700 1 -- 172.21.15.27:0/903996006 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fd348010070 con 0x7fd3500f66e0 2021-11-03T18:12:44.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.743+0000 7fd33effd700 1 --2- 172.21.15.27:0/903996006 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd3380640b0 0x7fd338066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:12:44.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.743+0000 7fd33effd700 1 -- 172.21.15.27:0/903996006 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fd348080de0 con 0x7fd3500f66e0 2021-11-03T18:12:44.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.743+0000 7fd34dd9b700 1 --2- 172.21.15.27:0/903996006 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd3380640b0 0x7fd338066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:12:44.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.744+0000 7fd34dd9b700 1 --2- 172.21.15.27:0/903996006 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd3380640b0 0x7fd338066570 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7fd344009da0 tx=0x7fd344009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:12:44.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.747+0000 7fd33effd700 1 -- 172.21.15.27:0/903996006 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fd3480530b0 con 0x7fd3500f66e0 2021-11-03T18:12:44.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:44 smithi027 ceph-mon[97244]: pgmap v1742: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:44.978 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.980+0000 7fd354ca0700 1 -- 172.21.15.27:0/903996006 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd3500586e0 con 0x7fd3500f66e0 2021-11-03T18:12:44.979 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.980+0000 7fd33effd700 1 -- 172.21.15.27:0/903996006 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fd348052ca0 con 0x7fd3500f66e0 2021-11-03T18:12:44.979 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:12:44.980 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:12:44.980 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:12:44.980 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:12:44.980 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:12:44.980 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:12:44.981 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:12:44.981 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:12:44.981 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:12:44.981 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:12:44.982 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:12:44.982 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:12:44.982 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:12:44.982 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:12:44.983 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:12:44.983 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:12:44.983 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:12:44.983 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:12:44.984 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.982+0000 7fd354ca0700 1 -- 172.21.15.27:0/903996006 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd3380640b0 msgr2=0x7fd338066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:12:44.984 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.982+0000 7fd354ca0700 1 --2- 172.21.15.27:0/903996006 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd3380640b0 0x7fd338066570 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7fd344009da0 tx=0x7fd344009b30).stop 2021-11-03T18:12:44.985 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.982+0000 7fd354ca0700 1 -- 172.21.15.27:0/903996006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3500f66e0 msgr2=0x7fd35006b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:12:44.985 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.982+0000 7fd354ca0700 1 --2- 172.21.15.27:0/903996006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3500f66e0 0x7fd35006b9a0 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 rx=0x7fd34800c710 tx=0x7fd348004d10).stop 2021-11-03T18:12:44.985 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.982+0000 7fd354ca0700 1 -- 172.21.15.27:0/903996006 shutdown_connections 2021-11-03T18:12:44.985 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.982+0000 7fd354ca0700 1 --2- 172.21.15.27:0/903996006 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd3380640b0 0x7fd338066570 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:44.985 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.982+0000 7fd354ca0700 1 --2- 172.21.15.27:0/903996006 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3500f66e0 0x7fd35006b9a0 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:44.986 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.982+0000 7fd354ca0700 1 --2- 172.21.15.27:0/903996006 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd3500f7980 0x7fd350069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:44.986 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.982+0000 7fd354ca0700 1 -- 172.21.15.27:0/903996006 >> 172.21.15.27:0/903996006 conn(0x7fd3500f1e40 msgr2=0x7fd3500fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:12:44.986 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.982+0000 7fd354ca0700 1 -- 172.21.15.27:0/903996006 shutdown_connections 2021-11-03T18:12:44.986 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:44.982+0000 7fd354ca0700 1 -- 172.21.15.27:0/903996006 wait complete. 2021-11-03T18:12:45.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.082+0000 7fe2309a1700 1 -- 172.21.15.27:0/1646326601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe22c0f6860 msgr2=0x7fe22c0f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:12:45.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.082+0000 7fe2309a1700 1 --2- 172.21.15.27:0/1646326601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe22c0f6860 0x7fe22c0f6cc0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 rx=0x7fe220004660 tx=0x7fe220009b30).stop 2021-11-03T18:12:45.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.082+0000 7fe2309a1700 1 -- 172.21.15.27:0/1646326601 shutdown_connections 2021-11-03T18:12:45.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.082+0000 7fe2309a1700 1 --2- 172.21.15.27:0/1646326601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe22c0f6860 0x7fe22c0f6cc0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:45.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.082+0000 7fe2309a1700 1 --2- 172.21.15.27:0/1646326601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe22c0f55c0 0x7fe22c0f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:45.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.082+0000 7fe2309a1700 1 -- 172.21.15.27:0/1646326601 >> 172.21.15.27:0/1646326601 conn(0x7fe22c0f0d40 msgr2=0x7fe22c0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:12:45.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.082+0000 7fe2309a1700 1 -- 172.21.15.27:0/1646326601 shutdown_connections 2021-11-03T18:12:45.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.082+0000 7fe2309a1700 1 -- 172.21.15.27:0/1646326601 wait complete. 2021-11-03T18:12:45.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.083+0000 7fe2309a1700 1 Processor -- start 2021-11-03T18:12:45.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.083+0000 7fe2309a1700 1 -- start start 2021-11-03T18:12:45.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.083+0000 7fe2309a1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe22c0f55c0 0x7fe22c106f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:12:45.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.083+0000 7fe2309a1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe22c0f6860 0x7fe22c107420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:12:45.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.083+0000 7fe2309a1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c1079d0 con 0x7fe22c0f55c0 2021-11-03T18:12:45.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.083+0000 7fe2309a1700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c107b10 con 0x7fe22c0f6860 2021-11-03T18:12:45.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.083+0000 7fe22a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe22c0f55c0 0x7fe22c106f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:12:45.086 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.083+0000 7fe22a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe22c0f55c0 0x7fe22c106f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33588/0 (socket says 172.21.15.27:33588) 2021-11-03T18:12:45.086 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.083+0000 7fe22a59c700 1 -- 172.21.15.27:0/2739433805 learned_addr learned my addr 172.21.15.27:0/2739433805 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:12:45.086 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.083+0000 7fe229d9b700 1 --2- 172.21.15.27:0/2739433805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe22c0f6860 0x7fe22c107420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:12:45.086 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.083+0000 7fe22a59c700 1 -- 172.21.15.27:0/2739433805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe22c0f6860 msgr2=0x7fe22c107420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:12:45.087 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.083+0000 7fe22a59c700 1 --2- 172.21.15.27:0/2739433805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe22c0f6860 0x7fe22c107420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:45.087 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.083+0000 7fe22a59c700 1 -- 172.21.15.27:0/2739433805 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe220005040 con 0x7fe22c0f55c0 2021-11-03T18:12:45.087 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.083+0000 7fe229d9b700 1 --2- 172.21.15.27:0/2739433805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe22c0f6860 0x7fe22c107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:12:45.087 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.083+0000 7fe22a59c700 1 --2- 172.21.15.27:0/2739433805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe22c0f55c0 0x7fe22c106f30 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 rx=0x7fe21c00c530 tx=0x7fe21c007e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:12:45.087 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.083+0000 7fe21affd700 1 -- 172.21.15.27:0/2739433805 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe21c00f650 con 0x7fe22c0f55c0 2021-11-03T18:12:45.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.084+0000 7fe21affd700 1 -- 172.21.15.27:0/2739433805 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fe21c00fc60 con 0x7fe22c0f55c0 2021-11-03T18:12:45.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.084+0000 7fe2309a1700 1 -- 172.21.15.27:0/2739433805 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe22c1084b0 con 0x7fe22c0f55c0 2021-11-03T18:12:45.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.084+0000 7fe21affd700 1 -- 172.21.15.27:0/2739433805 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe21c012d20 con 0x7fe22c0f55c0 2021-11-03T18:12:45.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.084+0000 7fe2309a1700 1 -- 172.21.15.27:0/2739433805 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe22c108a70 con 0x7fe22c0f55c0 2021-11-03T18:12:45.090 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.084+0000 7fe2309a1700 1 -- 172.21.15.27:0/2739433805 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe22c045ad0 con 0x7fe22c0f55c0 2021-11-03T18:12:45.090 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.085+0000 7fe21affd700 1 -- 172.21.15.27:0/2739433805 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fe21c019030 con 0x7fe22c0f55c0 2021-11-03T18:12:45.090 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.085+0000 7fe21affd700 1 --2- 172.21.15.27:0/2739433805 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe2140640b0 0x7fe214066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:12:45.090 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.085+0000 7fe21affd700 1 -- 172.21.15.27:0/2739433805 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fe21c080680 con 0x7fe22c0f55c0 2021-11-03T18:12:45.091 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.085+0000 7fe229d9b700 1 --2- 172.21.15.27:0/2739433805 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe2140640b0 0x7fe214066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:12:45.091 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.086+0000 7fe229d9b700 1 --2- 172.21.15.27:0/2739433805 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe2140640b0 0x7fe214066570 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7fe220004e30 tx=0x7fe220009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:12:45.091 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.089+0000 7fe21affd700 1 -- 172.21.15.27:0/2739433805 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fe21c0528e0 con 0x7fe22c0f55c0 2021-11-03T18:12:45.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.294+0000 7fe2309a1700 1 -- 172.21.15.27:0/2739433805 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe22c109110 con 0x7fe22c0f55c0 2021-11-03T18:12:45.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.295+0000 7fe21affd700 1 -- 172.21.15.27:0/2739433805 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7fe21c017710 con 0x7fe22c0f55c0 2021-11-03T18:12:45.294 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:12:45.294 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:12:45.294 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:12:45.294 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:12:45.295 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:12:45.295 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:12:45.295 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:12:45.295 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:12:45.296 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:12:45.296 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:12:45.296 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:12:45.296 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:12:45.297 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:12:45.297 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:12:45.297 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:12:45.297 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:12:45.297 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:12:45.298 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:12:45.298 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:12:45.298 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:12:45.298 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:12:45.299 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:12:45.299 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:12:45.299 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:12:45.299 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:12:45.300 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:12:45.300 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:12:45.300 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:12:45.300 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:12:45.301 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:12:45.301 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:12:45.301 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:12:45.301 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:12:45.302 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:12:45.302 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:12:45.302 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:12:45.302 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:12:45.302 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:12:45.303 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:12:45.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.296+0000 7fe2309a1700 1 -- 172.21.15.27:0/2739433805 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe2140640b0 msgr2=0x7fe214066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:12:45.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.296+0000 7fe2309a1700 1 --2- 172.21.15.27:0/2739433805 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe2140640b0 0x7fe214066570 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 rx=0x7fe220004e30 tx=0x7fe220009b30).stop 2021-11-03T18:12:45.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.296+0000 7fe2309a1700 1 -- 172.21.15.27:0/2739433805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe22c0f55c0 msgr2=0x7fe22c106f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:12:45.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.296+0000 7fe2309a1700 1 --2- 172.21.15.27:0/2739433805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe22c0f55c0 0x7fe22c106f30 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 rx=0x7fe21c00c530 tx=0x7fe21c007e10).stop 2021-11-03T18:12:45.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.296+0000 7fe2309a1700 1 -- 172.21.15.27:0/2739433805 shutdown_connections 2021-11-03T18:12:45.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.296+0000 7fe2309a1700 1 --2- 172.21.15.27:0/2739433805 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe2140640b0 0x7fe214066570 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:45.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.296+0000 7fe2309a1700 1 --2- 172.21.15.27:0/2739433805 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe22c0f55c0 0x7fe22c106f30 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:45.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.296+0000 7fe2309a1700 1 --2- 172.21.15.27:0/2739433805 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe22c0f6860 0x7fe22c107420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:12:45.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.296+0000 7fe2309a1700 1 -- 172.21.15.27:0/2739433805 >> 172.21.15.27:0/2739433805 conn(0x7fe22c0f0d40 msgr2=0x7fe22c05ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:12:45.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.297+0000 7fe2309a1700 1 -- 172.21.15.27:0/2739433805 shutdown_connections 2021-11-03T18:12:45.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:12:45.297+0000 7fe2309a1700 1 -- 172.21.15.27:0/2739433805 wait complete. 2021-11-03T18:12:45.306 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:12:45.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:45 smithi115 ceph-mon[65453]: from='client.35896 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:12:45.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3407.441110 secs 2021-11-03T18:12:45.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:45 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/903996006' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:45.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:45 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2739433805' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:12:45.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:45 smithi027 ceph-mon[97244]: from='client.35896 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:12:45.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3407.441110 secs 2021-11-03T18:12:45.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:45 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/903996006' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:45.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:45 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2739433805' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:12:46.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:46 smithi115 ceph-mon[65453]: from='client.35900 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:12:46.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:46 smithi115 ceph-mon[65453]: pgmap v1743: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:46.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:46 smithi027 ceph-mon[97244]: from='client.35900 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:12:46.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:46 smithi027 ceph-mon[97244]: pgmap v1743: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:48.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:48 smithi115 ceph-mon[65453]: pgmap v1744: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:48.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:12:48.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:12:48.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:12:48.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:48.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:48.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:48.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:48.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:48 smithi027 ceph-mon[97244]: pgmap v1744: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:48.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:12:48.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:12:48.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:12:48.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:48.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:48.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:48.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:49.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:49 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:12:49.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:49 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:12:50.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:50 smithi115 ceph-mon[65453]: pgmap v1745: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:50.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3412.441245 secs 2021-11-03T18:12:50.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:50 smithi027 ceph-mon[97244]: pgmap v1745: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:50.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3412.441245 secs 2021-11-03T18:12:52.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:52 smithi115 ceph-mon[65453]: pgmap v1746: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:52.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:52 smithi027 ceph-mon[97244]: pgmap v1746: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:54.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:54 smithi115 ceph-mon[65453]: pgmap v1747: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:54.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:54 smithi027 ceph-mon[97244]: pgmap v1747: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:55.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3417.441370 secs 2021-11-03T18:12:55.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3417.441370 secs 2021-11-03T18:12:56.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:56 smithi115 ceph-mon[65453]: pgmap v1748: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:56.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:12:56.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:12:56.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:56 smithi027 ceph-mon[97244]: pgmap v1748: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:56.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:12:56.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:12:58.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:58 smithi115 ceph-mon[65453]: pgmap v1749: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:58.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:12:58.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:12:58.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:12:58.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:58.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:58.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:58.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:58.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:58 smithi027 ceph-mon[97244]: pgmap v1749: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:12:58.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:12:58.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:12:58.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:12:58.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:58.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:58.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:58.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:12:59.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:12:59 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:12:59.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:12:59 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:13:00.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:00 smithi115 ceph-mon[65453]: pgmap v1750: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:00.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3422.441485 secs 2021-11-03T18:13:00.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:00 smithi027 ceph-mon[97244]: pgmap v1750: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:00.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3422.441485 secs 2021-11-03T18:13:02.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:02 smithi115 ceph-mon[65453]: pgmap v1751: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:02.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:02 smithi027 ceph-mon[97244]: pgmap v1751: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:04.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:04 smithi115 ceph-mon[65453]: pgmap v1752: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:04.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:04 smithi027 ceph-mon[97244]: pgmap v1752: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:05.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3427.441598 secs 2021-11-03T18:13:05.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3427.441598 secs 2021-11-03T18:13:06.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:06 smithi115 ceph-mon[65453]: pgmap v1753: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:06.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:06 smithi027 ceph-mon[97244]: pgmap v1753: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:08.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:08 smithi115 ceph-mon[65453]: pgmap v1754: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:08.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:13:08.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:13:08.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:13:08.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:08.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:08.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:08.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:08.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:08 smithi027 ceph-mon[97244]: pgmap v1754: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:08.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:13:08.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:13:08.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:13:08.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:08.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:08.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:08.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:09.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:09 smithi027 ceph-mon[97244]: poll took 60164.218 ms: 1 events 2021-11-03T18:13:09.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:09 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:13:09.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:09 smithi115 ceph-mon[65453]: poll took 60164.218 ms: 1 events 2021-11-03T18:13:09.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:09 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:13:10.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:10 smithi027 ceph-mon[97244]: pgmap v1755: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:10.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3432.441746 secs 2021-11-03T18:13:10.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:10 smithi115 ceph-mon[65453]: pgmap v1755: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:10.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3432.441746 secs 2021-11-03T18:13:12.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:12 smithi027 ceph-mon[97244]: pgmap v1756: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:12.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:12 smithi115 ceph-mon[65453]: pgmap v1756: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:14.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:14 smithi027 ceph-mon[97244]: pgmap v1757: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:14.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:14 smithi115 ceph-mon[65453]: pgmap v1757: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:15.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.412+0000 7f3ca03ae700 1 -- 172.21.15.27:0/18709030 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c980f66e0 msgr2=0x7f3c980f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:13:15.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.412+0000 7f3ca03ae700 1 --2- 172.21.15.27:0/18709030 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c980f66e0 0x7f3c980f6b00 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 rx=0x7f3c94004660 tx=0x7f3c94009b30).stop 2021-11-03T18:13:15.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.412+0000 7f3ca03ae700 1 -- 172.21.15.27:0/18709030 shutdown_connections 2021-11-03T18:13:15.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.412+0000 7f3ca03ae700 1 --2- 172.21.15.27:0/18709030 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c980f7980 0x7f3c980f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T18:13:15.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.412+0000 7f3ca03ae700 1 --2- 172.21.15.27:0/18709030 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c980f66e0 0x7f3c980f6b00 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:15.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.412+0000 7f3ca03ae700 1 -- 172.21.15.27:0/18709030 >> 172.21.15.27:0/18709030 conn(0x7f3c980f1e40 msgr2=0x7f3c980f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:13:15.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.412+0000 7f3ca03ae700 1 -- 172.21.15.27:0/18709030 shutdown_connections 2021-11-03T18:13:15.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.412+0000 7f3ca03ae700 1 -- 172.21.15.27:0/18709030 wait complete. 2021-11-03T18:13:15.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.413+0000 7f3ca03ae700 1 Processor -- start 2021-11-03T18:13:15.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.413+0000 7f3ca03ae700 1 -- start start 2021-11-03T18:13:15.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.413+0000 7f3ca03ae700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c980f66e0 0x7f3c9806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:13:15.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.413+0000 7f3ca03ae700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c980f7980 0x7f3c98069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:13:15.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.413+0000 7f3ca03ae700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9806a410 con 0x7f3c980f7980 2021-11-03T18:13:15.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.413+0000 7f3ca03ae700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c9806a550 con 0x7f3c980f66e0 2021-11-03T18:13:15.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.413+0000 7f3c9d949700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c980f7980 0x7f3c98069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:13:15.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.413+0000 7f3c9d949700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c980f7980 0x7f3c98069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33608/0 (socket says 172.21.15.27:33608) 2021-11-03T18:13:15.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.413+0000 7f3c9d949700 1 -- 172.21.15.27:0/3952784343 learned_addr learned my addr 172.21.15.27:0/3952784343 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:13:15.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.413+0000 7f3c9e14a700 1 --2- 172.21.15.27:0/3952784343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c980f66e0 0x7f3c9806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:13:15.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.413+0000 7f3c9d949700 1 -- 172.21.15.27:0/3952784343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c980f66e0 msgr2=0x7f3c9806b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:13:15.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.413+0000 7f3c9d949700 1 --2- 172.21.15.27:0/3952784343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c980f66e0 0x7f3c9806b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:15.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.413+0000 7f3c9d949700 1 -- 172.21.15.27:0/3952784343 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c94005040 con 0x7f3c980f7980 2021-11-03T18:13:15.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.414+0000 7f3c9d949700 1 --2- 172.21.15.27:0/3952784343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c980f7980 0x7f3c98069e20 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0x7f3c8c00b410 tx=0x7f3c8c006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:13:15.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.414+0000 7f3c8affd700 1 -- 172.21.15.27:0/3952784343 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c8c0125f0 con 0x7f3c980f7980 2021-11-03T18:13:15.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.414+0000 7f3c8affd700 1 -- 172.21.15.27:0/3952784343 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f3c8c00be30 con 0x7f3c980f7980 2021-11-03T18:13:15.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.414+0000 7f3ca03ae700 1 -- 172.21.15.27:0/3952784343 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c9806a830 con 0x7f3c980f7980 2021-11-03T18:13:15.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.414+0000 7f3c8affd700 1 -- 172.21.15.27:0/3952784343 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3c8c002a70 con 0x7f3c980f7980 2021-11-03T18:13:15.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.414+0000 7f3ca03ae700 1 -- 172.21.15.27:0/3952784343 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c9810efe0 con 0x7f3c980f7980 2021-11-03T18:13:15.419 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.415+0000 7f3ca03ae700 1 -- 172.21.15.27:0/3952784343 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c98045ad0 con 0x7f3c980f7980 2021-11-03T18:13:15.419 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.415+0000 7f3c8affd700 1 -- 172.21.15.27:0/3952784343 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f3c8c00f030 con 0x7f3c980f7980 2021-11-03T18:13:15.420 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.416+0000 7f3c8affd700 1 --2- 172.21.15.27:0/3952784343 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3c840640b0 0x7f3c84066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:13:15.420 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.416+0000 7f3c8affd700 1 -- 172.21.15.27:0/3952784343 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f3c8c080c50 con 0x7f3c980f7980 2021-11-03T18:13:15.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.416+0000 7f3c9e14a700 1 --2- 172.21.15.27:0/3952784343 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3c840640b0 0x7f3c84066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:13:15.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.416+0000 7f3c9e14a700 1 --2- 172.21.15.27:0/3952784343 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3c840640b0 0x7f3c84066570 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7f3c94013690 tx=0x7f3c94009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:13:15.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.420+0000 7f3c8affd700 1 -- 172.21.15.27:0/3952784343 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f3c8c052f50 con 0x7f3c980f7980 2021-11-03T18:13:15.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.599+0000 7f3ca03ae700 1 -- 172.21.15.27:0/3952784343 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3c980fef90 con 0x7f3c840640b0 2021-11-03T18:13:15.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.600+0000 7f3c8affd700 1 -- 172.21.15.27:0/3952784343 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f3c980fef90 con 0x7f3c840640b0 2021-11-03T18:13:15.600 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.602+0000 7f3ca03ae700 1 -- 172.21.15.27:0/3952784343 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3c840640b0 msgr2=0x7f3c84066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:13:15.600 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.602+0000 7f3ca03ae700 1 --2- 172.21.15.27:0/3952784343 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3c840640b0 0x7f3c84066570 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 rx=0x7f3c94013690 tx=0x7f3c94009b30).stop 2021-11-03T18:13:15.600 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.602+0000 7f3ca03ae700 1 -- 172.21.15.27:0/3952784343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c980f7980 msgr2=0x7f3c98069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:13:15.601 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.602+0000 7f3ca03ae700 1 --2- 172.21.15.27:0/3952784343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c980f7980 0x7f3c98069e20 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0x7f3c8c00b410 tx=0x7f3c8c006ef0).stop 2021-11-03T18:13:15.601 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.602+0000 7f3ca03ae700 1 -- 172.21.15.27:0/3952784343 shutdown_connections 2021-11-03T18:13:15.601 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.602+0000 7f3ca03ae700 1 --2- 172.21.15.27:0/3952784343 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3c840640b0 0x7f3c84066570 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:15.601 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.602+0000 7f3ca03ae700 1 --2- 172.21.15.27:0/3952784343 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3c980f66e0 0x7f3c9806b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:15.602 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.602+0000 7f3ca03ae700 1 --2- 172.21.15.27:0/3952784343 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3c980f7980 0x7f3c98069e20 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:15.602 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.602+0000 7f3ca03ae700 1 -- 172.21.15.27:0/3952784343 >> 172.21.15.27:0/3952784343 conn(0x7f3c980f1e40 msgr2=0x7f3c980fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:13:15.602 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.602+0000 7f3ca03ae700 1 -- 172.21.15.27:0/3952784343 shutdown_connections 2021-11-03T18:13:15.602 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.603+0000 7f3ca03ae700 1 -- 172.21.15.27:0/3952784343 wait complete. 2021-11-03T18:13:15.609 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:13:15.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.703+0000 7fc86c51d700 1 -- 172.21.15.27:0/1269556951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8640f6860 msgr2=0x7fc8640f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:13:15.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.703+0000 7fc86c51d700 1 --2- 172.21.15.27:0/1269556951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8640f6860 0x7fc8640f6cc0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 rx=0x7fc858004660 tx=0x7fc858009b30).stop 2021-11-03T18:13:15.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.703+0000 7fc86c51d700 1 -- 172.21.15.27:0/1269556951 shutdown_connections 2021-11-03T18:13:15.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.703+0000 7fc86c51d700 1 --2- 172.21.15.27:0/1269556951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8640f6860 0x7fc8640f6cc0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:15.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.703+0000 7fc86c51d700 1 --2- 172.21.15.27:0/1269556951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8640f55c0 0x7fc8640f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:15.704 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.703+0000 7fc86c51d700 1 -- 172.21.15.27:0/1269556951 >> 172.21.15.27:0/1269556951 conn(0x7fc8640f0d40 msgr2=0x7fc8640f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:13:15.704 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.703+0000 7fc86c51d700 1 -- 172.21.15.27:0/1269556951 shutdown_connections 2021-11-03T18:13:15.704 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.703+0000 7fc86c51d700 1 -- 172.21.15.27:0/1269556951 wait complete. 2021-11-03T18:13:15.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.703+0000 7fc86c51d700 1 Processor -- start 2021-11-03T18:13:15.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.703+0000 7fc86c51d700 1 -- start start 2021-11-03T18:13:15.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.703+0000 7fc86c51d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8640f55c0 0x7fc864106f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:13:15.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.703+0000 7fc86c51d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8640f6860 0x7fc864107440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:13:15.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.703+0000 7fc86c51d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8641079f0 con 0x7fc8640f55c0 2021-11-03T18:13:15.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.703+0000 7fc86c51d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc864107b30 con 0x7fc8640f6860 2021-11-03T18:13:15.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.704+0000 7fc86a2b9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8640f55c0 0x7fc864106f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:13:15.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.704+0000 7fc86a2b9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8640f55c0 0x7fc864106f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33616/0 (socket says 172.21.15.27:33616) 2021-11-03T18:13:15.707 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.704+0000 7fc86a2b9700 1 -- 172.21.15.27:0/3112231643 learned_addr learned my addr 172.21.15.27:0/3112231643 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:13:15.707 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.704+0000 7fc869ab8700 1 --2- 172.21.15.27:0/3112231643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8640f6860 0x7fc864107440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:13:15.707 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.704+0000 7fc86a2b9700 1 -- 172.21.15.27:0/3112231643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8640f6860 msgr2=0x7fc864107440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:13:15.707 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.704+0000 7fc86a2b9700 1 --2- 172.21.15.27:0/3112231643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8640f6860 0x7fc864107440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:15.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.704+0000 7fc86a2b9700 1 -- 172.21.15.27:0/3112231643 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc858005040 con 0x7fc8640f55c0 2021-11-03T18:13:15.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.704+0000 7fc86a2b9700 1 --2- 172.21.15.27:0/3112231643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8640f55c0 0x7fc864106f50 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 rx=0x7fc85c00d7f0 tx=0x7fc85c004e80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:13:15.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.704+0000 7fc862ffd700 1 -- 172.21.15.27:0/3112231643 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc85c00f490 con 0x7fc8640f55c0 2021-11-03T18:13:15.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.704+0000 7fc86c51d700 1 -- 172.21.15.27:0/3112231643 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8641084d0 con 0x7fc8640f55c0 2021-11-03T18:13:15.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.704+0000 7fc862ffd700 1 -- 172.21.15.27:0/3112231643 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fc85c00f040 con 0x7fc8640f55c0 2021-11-03T18:13:15.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.704+0000 7fc862ffd700 1 -- 172.21.15.27:0/3112231643 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc85c0139b0 con 0x7fc8640f55c0 2021-11-03T18:13:15.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.704+0000 7fc86c51d700 1 -- 172.21.15.27:0/3112231643 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc864108a90 con 0x7fc8640f55c0 2021-11-03T18:13:15.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.705+0000 7fc86c51d700 1 -- 172.21.15.27:0/3112231643 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc864045ad0 con 0x7fc8640f55c0 2021-11-03T18:13:15.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.706+0000 7fc862ffd700 1 -- 172.21.15.27:0/3112231643 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fc85c01a030 con 0x7fc8640f55c0 2021-11-03T18:13:15.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.706+0000 7fc862ffd700 1 --2- 172.21.15.27:0/3112231643 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc84c0640b0 0x7fc84c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:13:15.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.706+0000 7fc862ffd700 1 -- 172.21.15.27:0/3112231643 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fc85c080830 con 0x7fc8640f55c0 2021-11-03T18:13:15.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.706+0000 7fc869ab8700 1 --2- 172.21.15.27:0/3112231643 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc84c0640b0 0x7fc84c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:13:15.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.707+0000 7fc869ab8700 1 --2- 172.21.15.27:0/3112231643 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc84c0640b0 0x7fc84c066570 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7fc858004630 tx=0x7fc858009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:13:15.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.710+0000 7fc862ffd700 1 -- 172.21.15.27:0/3112231643 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fc85c052a40 con 0x7fc8640f55c0 2021-11-03T18:13:15.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3437.441889 secs 2021-11-03T18:13:15.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.883+0000 7fc86c51d700 1 -- 172.21.15.27:0/3112231643 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc8640008d0 con 0x7fc84c0640b0 2021-11-03T18:13:15.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.898+0000 7fc862ffd700 1 -- 172.21.15.27:0/3112231643 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fc8640008d0 con 0x7fc84c0640b0 2021-11-03T18:13:15.896 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:13:15.896 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (64m) 3m ago 66m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:13:15.897 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (56m) 3m ago 66m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:13:15.897 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (56m) 3m ago 65m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:13:15.897 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (64m) 3m ago 65m 42.1M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:13:15.897 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (58m) 3m ago 61m 26.3M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:13:15.898 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (61m) 3m ago 61m 3847M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:13:15.898 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (57m) 3m ago 61m 25.9M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:13:15.898 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (58m) 3m ago 61m 27.0M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:13:15.898 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (58m) 3m ago 67m 572M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:13:15.899 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (57m) 3m ago 65m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:13:15.899 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (57m) 3m ago 67m 273M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:13:15.899 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (57m) 3m ago 65m 129M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:13:15.899 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (65m) 3m ago 65m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:13:15.900 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (65m) 3m ago 65m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:13:15.900 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (56m) 3m ago 64m 371M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:13:15.900 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (55m) 3m ago 64m 405M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:13:15.900 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (55m) 3m ago 63m 302M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:13:15.900 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (54m) 3m ago 63m 429M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:13:15.901 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (54m) 3m ago 63m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:13:15.901 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (53m) 3m ago 62m 348M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:13:15.901 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (64m) 3m ago 65m 57.6M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:13:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.900+0000 7fc86c51d700 1 -- 172.21.15.27:0/3112231643 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc84c0640b0 msgr2=0x7fc84c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:13:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.900+0000 7fc86c51d700 1 --2- 172.21.15.27:0/3112231643 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc84c0640b0 0x7fc84c066570 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 rx=0x7fc858004630 tx=0x7fc858009b30).stop 2021-11-03T18:13:15.902 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.900+0000 7fc86c51d700 1 -- 172.21.15.27:0/3112231643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8640f55c0 msgr2=0x7fc864106f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:13:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.900+0000 7fc86c51d700 1 --2- 172.21.15.27:0/3112231643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8640f55c0 0x7fc864106f50 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 rx=0x7fc85c00d7f0 tx=0x7fc85c004e80).stop 2021-11-03T18:13:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.900+0000 7fc86c51d700 1 -- 172.21.15.27:0/3112231643 shutdown_connections 2021-11-03T18:13:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.900+0000 7fc86c51d700 1 --2- 172.21.15.27:0/3112231643 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc84c0640b0 0x7fc84c066570 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:15.903 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.900+0000 7fc86c51d700 1 --2- 172.21.15.27:0/3112231643 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8640f55c0 0x7fc864106f50 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:15.904 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.900+0000 7fc86c51d700 1 --2- 172.21.15.27:0/3112231643 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8640f6860 0x7fc864107440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:15.904 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.900+0000 7fc86c51d700 1 -- 172.21.15.27:0/3112231643 >> 172.21.15.27:0/3112231643 conn(0x7fc8640f0d40 msgr2=0x7fc8640fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:13:15.904 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.900+0000 7fc86c51d700 1 -- 172.21.15.27:0/3112231643 shutdown_connections 2021-11-03T18:13:15.904 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.900+0000 7fc86c51d700 1 -- 172.21.15.27:0/3112231643 wait complete. 2021-11-03T18:13:15.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3437.441889 secs 2021-11-03T18:13:15.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.999+0000 7f52796d7700 1 -- 172.21.15.27:0/3228672028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52740f77f0 msgr2=0x7f52740f7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:13:15.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.999+0000 7f52796d7700 1 --2- 172.21.15.27:0/3228672028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52740f77f0 0x7f52740f7c50 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 rx=0x7f5268004660 tx=0x7f5268009b30).stop 2021-11-03T18:13:15.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.999+0000 7f52796d7700 1 -- 172.21.15.27:0/3228672028 shutdown_connections 2021-11-03T18:13:15.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.999+0000 7f52796d7700 1 --2- 172.21.15.27:0/3228672028 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52740f77f0 0x7f52740f7c50 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:15.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.999+0000 7f52796d7700 1 --2- 172.21.15.27:0/3228672028 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52740f6550 0x7f52740f6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:15.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.999+0000 7f52796d7700 1 -- 172.21.15.27:0/3228672028 >> 172.21.15.27:0/3228672028 conn(0x7f52740f1cd0 msgr2=0x7f52740f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:13:15.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:15.999+0000 7f52796d7700 1 -- 172.21.15.27:0/3228672028 shutdown_connections 2021-11-03T18:13:15.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.000+0000 7f52796d7700 1 -- 172.21.15.27:0/3228672028 wait complete. 2021-11-03T18:13:16.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.000+0000 7f52796d7700 1 Processor -- start 2021-11-03T18:13:16.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.000+0000 7f52796d7700 1 -- start start 2021-11-03T18:13:16.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.000+0000 7f52796d7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52740f6550 0x7f5274106d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:13:16.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.000+0000 7f52796d7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52740f77f0 0x7f5274107270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:13:16.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.000+0000 7f52796d7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5274107870 con 0x7f52740f6550 2021-11-03T18:13:16.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.000+0000 7f52796d7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52741080b0 con 0x7f52740f77f0 2021-11-03T18:13:16.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.000+0000 7f5272ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52740f6550 0x7f5274106d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:13:16.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.001+0000 7f5272ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52740f6550 0x7f5274106d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33626/0 (socket says 172.21.15.27:33626) 2021-11-03T18:13:16.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.001+0000 7f5272ffd700 1 -- 172.21.15.27:0/3628805182 learned_addr learned my addr 172.21.15.27:0/3628805182 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:13:16.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.001+0000 7f52727fc700 1 --2- 172.21.15.27:0/3628805182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52740f77f0 0x7f5274107270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:13:16.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.001+0000 7f5272ffd700 1 -- 172.21.15.27:0/3628805182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52740f77f0 msgr2=0x7f5274107270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:13:16.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.001+0000 7f5272ffd700 1 --2- 172.21.15.27:0/3628805182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52740f77f0 0x7f5274107270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:16.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.001+0000 7f5272ffd700 1 -- 172.21.15.27:0/3628805182 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5268005040 con 0x7f52740f6550 2021-11-03T18:13:16.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.001+0000 7f52727fc700 1 --2- 172.21.15.27:0/3628805182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52740f77f0 0x7f5274107270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:13:16.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.001+0000 7f5272ffd700 1 --2- 172.21.15.27:0/3628805182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52740f6550 0x7f5274106d80 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 rx=0x7f526400c790 tx=0x7f5264007d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:13:16.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.001+0000 7f525b7fe700 1 -- 172.21.15.27:0/3628805182 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f526400e680 con 0x7f52740f6550 2021-11-03T18:13:16.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.001+0000 7f525b7fe700 1 -- 172.21.15.27:0/3628805182 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f526400ec90 con 0x7f52740f6550 2021-11-03T18:13:16.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.001+0000 7f52796d7700 1 -- 172.21.15.27:0/3628805182 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5274108390 con 0x7f52740f6550 2021-11-03T18:13:16.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.001+0000 7f525b7fe700 1 -- 172.21.15.27:0/3628805182 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5264012d00 con 0x7f52740f6550 2021-11-03T18:13:16.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.001+0000 7f52796d7700 1 -- 172.21.15.27:0/3628805182 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52741088d0 con 0x7f52740f6550 2021-11-03T18:13:16.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.002+0000 7f52796d7700 1 -- 172.21.15.27:0/3628805182 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5274059100 con 0x7f52740f6550 2021-11-03T18:13:16.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.003+0000 7f525b7fe700 1 -- 172.21.15.27:0/3628805182 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f5264010070 con 0x7f52740f6550 2021-11-03T18:13:16.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.003+0000 7f525b7fe700 1 --2- 172.21.15.27:0/3628805182 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f525c064100 0x7f525c0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:13:16.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.003+0000 7f525b7fe700 1 -- 172.21.15.27:0/3628805182 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f5264080940 con 0x7f52740f6550 2021-11-03T18:13:16.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.003+0000 7f52727fc700 1 --2- 172.21.15.27:0/3628805182 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f525c064100 0x7f525c0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:13:16.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.003+0000 7f52727fc700 1 --2- 172.21.15.27:0/3628805182 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f525c064100 0x7f525c0665c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7f5268004e30 tx=0x7f5268009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:13:16.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.007+0000 7f525b7fe700 1 -- 172.21.15.27:0/3628805182 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f5264052af0 con 0x7f52740f6550 2021-11-03T18:13:16.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.239+0000 7f52796d7700 1 -- 172.21.15.27:0/3628805182 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5274045ad0 con 0x7f52740f6550 2021-11-03T18:13:16.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.240+0000 7f525b7fe700 1 -- 172.21.15.27:0/3628805182 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f52640526e0 con 0x7f52740f6550 2021-11-03T18:13:16.238 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:13:16.238 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:13:16.238 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:13:16.239 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:13:16.239 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:13:16.239 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:13:16.240 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:13:16.240 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:13:16.240 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:13:16.240 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:13:16.240 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:13:16.241 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:13:16.241 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:13:16.241 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:13:16.241 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:13:16.242 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:13:16.242 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:13:16.242 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:13:16.243 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.241+0000 7f52796d7700 1 -- 172.21.15.27:0/3628805182 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f525c064100 msgr2=0x7f525c0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:13:16.243 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.241+0000 7f52796d7700 1 --2- 172.21.15.27:0/3628805182 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f525c064100 0x7f525c0665c0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 rx=0x7f5268004e30 tx=0x7f5268009b30).stop 2021-11-03T18:13:16.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.241+0000 7f52796d7700 1 -- 172.21.15.27:0/3628805182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52740f6550 msgr2=0x7f5274106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:13:16.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.241+0000 7f52796d7700 1 --2- 172.21.15.27:0/3628805182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52740f6550 0x7f5274106d80 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 rx=0x7f526400c790 tx=0x7f5264007d90).stop 2021-11-03T18:13:16.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.241+0000 7f52796d7700 1 -- 172.21.15.27:0/3628805182 shutdown_connections 2021-11-03T18:13:16.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.241+0000 7f52796d7700 1 --2- 172.21.15.27:0/3628805182 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f525c064100 0x7f525c0665c0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:16.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.241+0000 7f52796d7700 1 --2- 172.21.15.27:0/3628805182 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52740f6550 0x7f5274106d80 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:16.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.241+0000 7f52796d7700 1 --2- 172.21.15.27:0/3628805182 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52740f77f0 0x7f5274107270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:16.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.241+0000 7f52796d7700 1 -- 172.21.15.27:0/3628805182 >> 172.21.15.27:0/3628805182 conn(0x7f52740f1cd0 msgr2=0x7f52740fb240 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:13:16.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.241+0000 7f52796d7700 1 -- 172.21.15.27:0/3628805182 shutdown_connections 2021-11-03T18:13:16.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.241+0000 7f52796d7700 1 -- 172.21.15.27:0/3628805182 wait complete. 2021-11-03T18:13:16.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.341+0000 7f93d4e58700 1 -- 172.21.15.27:0/2311484174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93d00f55e0 msgr2=0x7f93d00f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:13:16.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.341+0000 7f93d4e58700 1 --2- 172.21.15.27:0/2311484174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93d00f55e0 0x7f93d00f5a00 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 rx=0x7f93c0004660 tx=0x7f93c0009b30).stop 2021-11-03T18:13:16.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.342+0000 7f93d4e58700 1 -- 172.21.15.27:0/2311484174 shutdown_connections 2021-11-03T18:13:16.342 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.342+0000 7f93d4e58700 1 --2- 172.21.15.27:0/2311484174 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93d00f5ef0 0x7f93d00f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:16.342 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.342+0000 7f93d4e58700 1 --2- 172.21.15.27:0/2311484174 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93d00f55e0 0x7f93d00f5a00 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:16.342 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.342+0000 7f93d4e58700 1 -- 172.21.15.27:0/2311484174 >> 172.21.15.27:0/2311484174 conn(0x7f93d00f0de0 msgr2=0x7f93d00f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:13:16.342 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.342+0000 7f93d4e58700 1 -- 172.21.15.27:0/2311484174 shutdown_connections 2021-11-03T18:13:16.343 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.342+0000 7f93d4e58700 1 -- 172.21.15.27:0/2311484174 wait complete. 2021-11-03T18:13:16.343 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.342+0000 7f93d4e58700 1 Processor -- start 2021-11-03T18:13:16.343 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.342+0000 7f93d4e58700 1 -- start start 2021-11-03T18:13:16.343 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.342+0000 7f93d4e58700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93d00f55e0 0x7f93d0109cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:13:16.344 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.342+0000 7f93d4e58700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93d00f5ef0 0x7f93d010c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:13:16.344 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.342+0000 7f93d4e58700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d00604c0 con 0x7f93d00f55e0 2021-11-03T18:13:16.344 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.342+0000 7f93d4e58700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93d0060600 con 0x7f93d00f5ef0 2021-11-03T18:13:16.344 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.342+0000 7f93ce59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93d00f55e0 0x7f93d0109cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:13:16.344 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.343+0000 7f93ce59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93d00f55e0 0x7f93d0109cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33636/0 (socket says 172.21.15.27:33636) 2021-11-03T18:13:16.345 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.343+0000 7f93ce59c700 1 -- 172.21.15.27:0/3816452718 learned_addr learned my addr 172.21.15.27:0/3816452718 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:13:16.345 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.343+0000 7f93cdd9b700 1 --2- 172.21.15.27:0/3816452718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93d00f5ef0 0x7f93d010c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:13:16.345 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.343+0000 7f93ce59c700 1 -- 172.21.15.27:0/3816452718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93d00f5ef0 msgr2=0x7f93d010c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:13:16.345 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.343+0000 7f93ce59c700 1 --2- 172.21.15.27:0/3816452718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93d00f5ef0 0x7f93d010c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:16.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.343+0000 7f93ce59c700 1 -- 172.21.15.27:0/3816452718 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c0005040 con 0x7f93d00f55e0 2021-11-03T18:13:16.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.343+0000 7f93ce59c700 1 --2- 172.21.15.27:0/3816452718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93d00f55e0 0x7f93d0109cb0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 rx=0x7f93c0009990 tx=0x7f93c0009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:13:16.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.343+0000 7f93beffd700 1 -- 172.21.15.27:0/3816452718 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93c0018070 con 0x7f93d00f55e0 2021-11-03T18:13:16.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.343+0000 7f93beffd700 1 -- 172.21.15.27:0/3816452718 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f93c0007ca0 con 0x7f93d00f55e0 2021-11-03T18:13:16.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.343+0000 7f93beffd700 1 -- 172.21.15.27:0/3816452718 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f93c001c620 con 0x7f93d00f55e0 2021-11-03T18:13:16.347 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.343+0000 7f93d4e58700 1 -- 172.21.15.27:0/3816452718 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93d010c700 con 0x7f93d00f55e0 2021-11-03T18:13:16.347 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.343+0000 7f93d4e58700 1 -- 172.21.15.27:0/3816452718 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93d010cbe0 con 0x7f93d00f55e0 2021-11-03T18:13:16.348 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.344+0000 7f93d4e58700 1 -- 172.21.15.27:0/3816452718 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93d0045ad0 con 0x7f93d00f55e0 2021-11-03T18:13:16.348 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.345+0000 7f93beffd700 1 -- 172.21.15.27:0/3816452718 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f93c001d960 con 0x7f93d00f55e0 2021-11-03T18:13:16.349 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.345+0000 7f93beffd700 1 --2- 172.21.15.27:0/3816452718 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f93b80640b0 0x7f93b8066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:13:16.349 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.345+0000 7f93beffd700 1 -- 172.21.15.27:0/3816452718 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f93c00825a0 con 0x7f93d00f55e0 2021-11-03T18:13:16.349 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.345+0000 7f93cdd9b700 1 --2- 172.21.15.27:0/3816452718 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f93b80640b0 0x7f93b8066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:13:16.349 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.345+0000 7f93cdd9b700 1 --2- 172.21.15.27:0/3816452718 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f93b80640b0 0x7f93b8066570 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f93c40041d0 tx=0x7f93c4009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:13:16.350 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.349+0000 7f93beffd700 1 -- 172.21.15.27:0/3816452718 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f93c0054960 con 0x7f93d00f55e0 2021-11-03T18:13:16.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.553+0000 7f93d4e58700 1 -- 172.21.15.27:0/3816452718 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f93d00fdf10 con 0x7f93d00f55e0 2021-11-03T18:13:16.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.553+0000 7f93beffd700 1 -- 172.21.15.27:0/3816452718 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f93c0054550 con 0x7f93d00f55e0 2021-11-03T18:13:16.552 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:13:16.552 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:13:16.552 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:13:16.552 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:13:16.553 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:13:16.553 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:13:16.553 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:13:16.553 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:13:16.553 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:13:16.554 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:13:16.554 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:13:16.554 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:13:16.554 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:13:16.555 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:13:16.555 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:13:16.555 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:13:16.555 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:13:16.556 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:13:16.556 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:13:16.556 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:13:16.556 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:13:16.556 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:13:16.557 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:13:16.557 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:13:16.557 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:13:16.557 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:13:16.558 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:13:16.558 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:13:16.558 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:13:16.558 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:13:16.558 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:13:16.559 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:13:16.559 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:13:16.559 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:13:16.559 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:13:16.560 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:13:16.560 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:13:16.560 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:13:16.560 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:13:16.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.555+0000 7f93d4e58700 1 -- 172.21.15.27:0/3816452718 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f93b80640b0 msgr2=0x7f93b8066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:13:16.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.555+0000 7f93d4e58700 1 --2- 172.21.15.27:0/3816452718 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f93b80640b0 0x7f93b8066570 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f93c40041d0 tx=0x7f93c4009040).stop 2021-11-03T18:13:16.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.555+0000 7f93d4e58700 1 -- 172.21.15.27:0/3816452718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93d00f55e0 msgr2=0x7f93d0109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:13:16.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.555+0000 7f93d4e58700 1 --2- 172.21.15.27:0/3816452718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93d00f55e0 0x7f93d0109cb0 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 rx=0x7f93c0009990 tx=0x7f93c0009b30).stop 2021-11-03T18:13:16.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.555+0000 7f93d4e58700 1 -- 172.21.15.27:0/3816452718 shutdown_connections 2021-11-03T18:13:16.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.555+0000 7f93d4e58700 1 --2- 172.21.15.27:0/3816452718 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f93b80640b0 0x7f93b8066570 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:16.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.555+0000 7f93d4e58700 1 --2- 172.21.15.27:0/3816452718 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f93d00f55e0 0x7f93d0109cb0 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:16.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.555+0000 7f93d4e58700 1 --2- 172.21.15.27:0/3816452718 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f93d00f5ef0 0x7f93d010c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:16.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.555+0000 7f93d4e58700 1 -- 172.21.15.27:0/3816452718 >> 172.21.15.27:0/3816452718 conn(0x7f93d00f0de0 msgr2=0x7f93d00fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:13:16.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.555+0000 7f93d4e58700 1 -- 172.21.15.27:0/3816452718 shutdown_connections 2021-11-03T18:13:16.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:16.555+0000 7f93d4e58700 1 -- 172.21.15.27:0/3816452718 wait complete. 2021-11-03T18:13:16.564 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:13:16.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:16 smithi027 ceph-mon[97244]: pgmap v1758: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:16.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:16 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3628805182' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:16.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:16 smithi115 ceph-mon[65453]: pgmap v1758: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:16.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:16 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3628805182' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:17.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:17 smithi027 ceph-mon[97244]: from='client.35912 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:13:17.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:17 smithi027 ceph-mon[97244]: from='client.35916 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:13:17.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:17 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3816452718' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:13:17.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:17 smithi115 ceph-mon[65453]: from='client.35912 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:13:17.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:17 smithi115 ceph-mon[65453]: from='client.35916 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:13:17.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:17 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3816452718' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:13:18.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:18 smithi027 ceph-mon[97244]: pgmap v1759: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:18.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:13:18.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:13:18.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:13:18.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:18.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:18.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:18.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:18.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:18 smithi115 ceph-mon[65453]: pgmap v1759: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:18.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:13:18.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:13:18.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:13:18.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:18.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:18.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:18.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:19.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:19 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:13:19.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:19 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:13:20.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:20 smithi027 ceph-mon[97244]: pgmap v1760: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:20.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3442.442021 secs 2021-11-03T18:13:20.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:20 smithi115 ceph-mon[65453]: pgmap v1760: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:20.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3442.442021 secs 2021-11-03T18:13:22.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:22 smithi027 ceph-mon[97244]: pgmap v1761: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:22.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:22 smithi115 ceph-mon[65453]: pgmap v1761: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:24.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:24 smithi115 ceph-mon[65453]: pgmap v1762: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:25.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:24 smithi027 ceph-mon[97244]: pgmap v1762: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:25.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3447.442136 secs 2021-11-03T18:13:26.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3447.442136 secs 2021-11-03T18:13:26.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:26 smithi115 ceph-mon[65453]: pgmap v1763: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:27.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:26 smithi027 ceph-mon[97244]: pgmap v1763: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:28.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:28 smithi115 ceph-mon[65453]: pgmap v1764: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:28.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:13:28.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:13:28.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:13:28.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:28.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:28.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:28.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:29.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:28 smithi027 ceph-mon[97244]: pgmap v1764: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:29.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:13:29.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:13:29.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:13:29.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:29.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:29.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:29.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:29.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:29 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:13:30.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:29 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:13:30.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:30 smithi115 ceph-mon[65453]: pgmap v1765: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:30.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3452.442252 secs 2021-11-03T18:13:31.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:30 smithi027 ceph-mon[97244]: pgmap v1765: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:31.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3452.442252 secs 2021-11-03T18:13:32.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:32 smithi027 ceph-mon[97244]: pgmap v1766: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:32.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:32 smithi115 ceph-mon[65453]: pgmap v1766: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:34.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:34 smithi115 ceph-mon[65453]: pgmap v1767: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:35.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:34 smithi027 ceph-mon[97244]: pgmap v1767: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:35.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3457.442388 secs 2021-11-03T18:13:36.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3457.442388 secs 2021-11-03T18:13:36.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:36 smithi115 ceph-mon[65453]: pgmap v1768: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:37.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:36 smithi027 ceph-mon[97244]: pgmap v1768: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:38.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:38 smithi115 ceph-mon[65453]: pgmap v1769: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:38.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:13:38.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:13:38.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:13:38.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:38.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:38.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:38.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:39.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:38 smithi027 ceph-mon[97244]: pgmap v1769: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:39.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:13:39.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:13:39.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:13:39.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:39.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:39.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:39.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:39.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:39 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:13:40.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:39 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:13:40.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:40 smithi115 ceph-mon[65453]: pgmap v1770: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:40.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3462.442531 secs 2021-11-03T18:13:41.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:40 smithi027 ceph-mon[97244]: pgmap v1770: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:41.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3462.442531 secs 2021-11-03T18:13:42.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:42 smithi027 ceph-mon[97244]: pgmap v1771: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:42.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:42 smithi115 ceph-mon[65453]: pgmap v1771: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:44.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:44 smithi115 ceph-mon[65453]: pgmap v1772: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:45.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:44 smithi027 ceph-mon[97244]: pgmap v1772: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:45.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3467.442680 secs 2021-11-03T18:13:46.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3467.442680 secs 2021-11-03T18:13:46.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.670+0000 7f92cdcb8700 1 -- 172.21.15.27:0/244792736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92c80f66e0 msgr2=0x7f92c80f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:13:46.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.670+0000 7f92cdcb8700 1 --2- 172.21.15.27:0/244792736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92c80f66e0 0x7f92c80f6b00 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 rx=0x7f92b8004660 tx=0x7f92b8009b30).stop 2021-11-03T18:13:46.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.670+0000 7f92cdcb8700 1 -- 172.21.15.27:0/244792736 shutdown_connections 2021-11-03T18:13:46.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.670+0000 7f92cdcb8700 1 --2- 172.21.15.27:0/244792736 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92c80f6ff0 0x7f92c80f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:46.670 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.670+0000 7f92cdcb8700 1 --2- 172.21.15.27:0/244792736 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92c80f66e0 0x7f92c80f6b00 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:46.670 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.671+0000 7f92cdcb8700 1 -- 172.21.15.27:0/244792736 >> 172.21.15.27:0/244792736 conn(0x7f92c80f1f00 msgr2=0x7f92c80f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:13:46.670 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.671+0000 7f92cdcb8700 1 -- 172.21.15.27:0/244792736 shutdown_connections 2021-11-03T18:13:46.670 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.671+0000 7f92cdcb8700 1 -- 172.21.15.27:0/244792736 wait complete. 2021-11-03T18:13:46.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.671+0000 7f92cdcb8700 1 Processor -- start 2021-11-03T18:13:46.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.671+0000 7f92cdcb8700 1 -- start start 2021-11-03T18:13:46.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.671+0000 7f92cdcb8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92c80f66e0 0x7f92c8106f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:13:46.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.671+0000 7f92cdcb8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92c80f6ff0 0x7f92c81073f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:13:46.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.672+0000 7f92cdcb8700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c81079a0 con 0x7f92c80f66e0 2021-11-03T18:13:46.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.672+0000 7f92cdcb8700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c8107ae0 con 0x7f92c80f6ff0 2021-11-03T18:13:46.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.672+0000 7f92c77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92c80f66e0 0x7f92c8106f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:13:46.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.672+0000 7f92c77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92c80f66e0 0x7f92c8106f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33654/0 (socket says 172.21.15.27:33654) 2021-11-03T18:13:46.676 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.672+0000 7f92c77fe700 1 -- 172.21.15.27:0/2959784583 learned_addr learned my addr 172.21.15.27:0/2959784583 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:13:46.676 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.672+0000 7f92c6ffd700 1 --2- 172.21.15.27:0/2959784583 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92c80f6ff0 0x7f92c81073f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:13:46.676 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.672+0000 7f92c77fe700 1 -- 172.21.15.27:0/2959784583 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92c80f6ff0 msgr2=0x7f92c81073f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:13:46.676 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.672+0000 7f92c77fe700 1 --2- 172.21.15.27:0/2959784583 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92c80f6ff0 0x7f92c81073f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:46.676 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.672+0000 7f92c77fe700 1 -- 172.21.15.27:0/2959784583 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b8005040 con 0x7f92c80f66e0 2021-11-03T18:13:46.677 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.672+0000 7f92c77fe700 1 --2- 172.21.15.27:0/2959784583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92c80f66e0 0x7f92c8106f00 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 rx=0x7f92b8009990 tx=0x7f92b8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:13:46.677 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.672+0000 7f92bffff700 1 -- 172.21.15.27:0/2959784583 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92b8018070 con 0x7f92c80f66e0 2021-11-03T18:13:46.677 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.672+0000 7f92cdcb8700 1 -- 172.21.15.27:0/2959784583 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92c8108420 con 0x7f92c80f66e0 2021-11-03T18:13:46.677 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.672+0000 7f92bffff700 1 -- 172.21.15.27:0/2959784583 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f92b8007ba0 con 0x7f92c80f66e0 2021-11-03T18:13:46.678 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.672+0000 7f92bffff700 1 -- 172.21.15.27:0/2959784583 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f92b801c3c0 con 0x7f92c80f66e0 2021-11-03T18:13:46.678 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.672+0000 7f92cdcb8700 1 -- 172.21.15.27:0/2959784583 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92c8108980 con 0x7f92c80f66e0 2021-11-03T18:13:46.678 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.674+0000 7f92bffff700 1 -- 172.21.15.27:0/2959784583 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f92b801d420 con 0x7f92c80f66e0 2021-11-03T18:13:46.678 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.674+0000 7f92cdcb8700 1 -- 172.21.15.27:0/2959784583 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92c8045ad0 con 0x7f92c80f66e0 2021-11-03T18:13:46.679 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.674+0000 7f92bffff700 1 --2- 172.21.15.27:0/2959784583 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f92b40640b0 0x7f92b4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:13:46.679 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.674+0000 7f92bffff700 1 -- 172.21.15.27:0/2959784583 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f92b8082630 con 0x7f92c80f66e0 2021-11-03T18:13:46.679 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.674+0000 7f92c6ffd700 1 --2- 172.21.15.27:0/2959784583 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f92b40640b0 0x7f92b4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:13:46.679 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.675+0000 7f92c6ffd700 1 --2- 172.21.15.27:0/2959784583 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f92b40640b0 0x7f92b4066570 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7f92b0009720 tx=0x7f92b0009c60).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:13:46.680 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.679+0000 7f92bffff700 1 -- 172.21.15.27:0/2959784583 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f92b8054990 con 0x7f92c80f66e0 2021-11-03T18:13:46.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.859+0000 7f92cdcb8700 1 -- 172.21.15.27:0/2959784583 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f92c80ff050 con 0x7f92b40640b0 2021-11-03T18:13:46.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.860+0000 7f92bffff700 1 -- 172.21.15.27:0/2959784583 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f92c80ff050 con 0x7f92b40640b0 2021-11-03T18:13:46.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.862+0000 7f92cdcb8700 1 -- 172.21.15.27:0/2959784583 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f92b40640b0 msgr2=0x7f92b4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:13:46.860 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.862+0000 7f92cdcb8700 1 --2- 172.21.15.27:0/2959784583 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f92b40640b0 0x7f92b4066570 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7f92b0009720 tx=0x7f92b0009c60).stop 2021-11-03T18:13:46.860 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.862+0000 7f92cdcb8700 1 -- 172.21.15.27:0/2959784583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92c80f66e0 msgr2=0x7f92c8106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:13:46.860 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.862+0000 7f92cdcb8700 1 --2- 172.21.15.27:0/2959784583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92c80f66e0 0x7f92c8106f00 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 rx=0x7f92b8009990 tx=0x7f92b8009b30).stop 2021-11-03T18:13:46.860 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.862+0000 7f92cdcb8700 1 -- 172.21.15.27:0/2959784583 shutdown_connections 2021-11-03T18:13:46.861 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.862+0000 7f92cdcb8700 1 --2- 172.21.15.27:0/2959784583 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f92b40640b0 0x7f92b4066570 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:46.861 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.862+0000 7f92cdcb8700 1 --2- 172.21.15.27:0/2959784583 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92c80f66e0 0x7f92c8106f00 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:46.861 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.862+0000 7f92cdcb8700 1 --2- 172.21.15.27:0/2959784583 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92c80f6ff0 0x7f92c81073f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:46.861 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.862+0000 7f92cdcb8700 1 -- 172.21.15.27:0/2959784583 >> 172.21.15.27:0/2959784583 conn(0x7f92c80f1f00 msgr2=0x7f92c80fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:13:46.861 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.862+0000 7f92cdcb8700 1 -- 172.21.15.27:0/2959784583 shutdown_connections 2021-11-03T18:13:46.862 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.862+0000 7f92cdcb8700 1 -- 172.21.15.27:0/2959784583 wait complete. 2021-11-03T18:13:46.869 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:13:46.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.962+0000 7fd98612a700 1 -- 172.21.15.27:0/2263917451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9800f5580 msgr2=0x7fd9800f59a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:13:46.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.962+0000 7fd98612a700 1 --2- 172.21.15.27:0/2263917451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9800f5580 0x7fd9800f59a0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 rx=0x7fd974004660 tx=0x7fd974009b30).stop 2021-11-03T18:13:46.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.962+0000 7fd98612a700 1 -- 172.21.15.27:0/2263917451 shutdown_connections 2021-11-03T18:13:46.961 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.962+0000 7fd98612a700 1 --2- 172.21.15.27:0/2263917451 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9800f6820 0x7fd9800f6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:46.961 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.962+0000 7fd98612a700 1 --2- 172.21.15.27:0/2263917451 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9800f5580 0x7fd9800f59a0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:46.961 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.962+0000 7fd98612a700 1 -- 172.21.15.27:0/2263917451 >> 172.21.15.27:0/2263917451 conn(0x7fd9800f0ce0 msgr2=0x7fd9800f3100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:13:46.961 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.962+0000 7fd98612a700 1 -- 172.21.15.27:0/2263917451 shutdown_connections 2021-11-03T18:13:46.962 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.962+0000 7fd98612a700 1 -- 172.21.15.27:0/2263917451 wait complete. 2021-11-03T18:13:46.962 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.962+0000 7fd98612a700 1 Processor -- start 2021-11-03T18:13:46.964 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.963+0000 7fd98612a700 1 -- start start 2021-11-03T18:13:46.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.963+0000 7fd98612a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9800f5580 0x7fd980106ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:13:46.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.963+0000 7fd98612a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9800f6820 0x7fd9801073e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:13:46.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.963+0000 7fd98612a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd980107990 con 0x7fd9800f6820 2021-11-03T18:13:46.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.963+0000 7fd98612a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd980107ad0 con 0x7fd9800f5580 2021-11-03T18:13:46.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.963+0000 7fd97effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9800f6820 0x7fd9801073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:13:46.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.963+0000 7fd97effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9800f6820 0x7fd9801073e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33666/0 (socket says 172.21.15.27:33666) 2021-11-03T18:13:46.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.963+0000 7fd97effd700 1 -- 172.21.15.27:0/229697899 learned_addr learned my addr 172.21.15.27:0/229697899 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:13:46.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.963+0000 7fd97f7fe700 1 --2- 172.21.15.27:0/229697899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9800f5580 0x7fd980106ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:13:46.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.963+0000 7fd97effd700 1 -- 172.21.15.27:0/229697899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9800f5580 msgr2=0x7fd980106ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:13:46.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.963+0000 7fd97effd700 1 --2- 172.21.15.27:0/229697899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9800f5580 0x7fd980106ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:46.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.963+0000 7fd97effd700 1 -- 172.21.15.27:0/229697899 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd974005040 con 0x7fd9800f6820 2021-11-03T18:13:46.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.963+0000 7fd97effd700 1 --2- 172.21.15.27:0/229697899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9800f6820 0x7fd9801073e0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 rx=0x7fd9700026d0 tx=0x7fd970009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:13:46.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.964+0000 7fd967fff700 1 -- 172.21.15.27:0/229697899 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9700079c0 con 0x7fd9800f6820 2021-11-03T18:13:46.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.964+0000 7fd967fff700 1 -- 172.21.15.27:0/229697899 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fd97000f040 con 0x7fd9800f6820 2021-11-03T18:13:46.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.964+0000 7fd98612a700 1 -- 172.21.15.27:0/229697899 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd980108470 con 0x7fd9800f6820 2021-11-03T18:13:46.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.964+0000 7fd967fff700 1 -- 172.21.15.27:0/229697899 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd9700148b0 con 0x7fd9800f6820 2021-11-03T18:13:46.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.964+0000 7fd98612a700 1 -- 172.21.15.27:0/229697899 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd980108a30 con 0x7fd9800f6820 2021-11-03T18:13:46.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.965+0000 7fd98612a700 1 -- 172.21.15.27:0/229697899 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd980045ad0 con 0x7fd9800f6820 2021-11-03T18:13:46.973 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.965+0000 7fd967fff700 1 -- 172.21.15.27:0/229697899 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fd970007b20 con 0x7fd9800f6820 2021-11-03T18:13:46.973 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.965+0000 7fd967fff700 1 --2- 172.21.15.27:0/229697899 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd968064100 0x7fd9680665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:13:46.973 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.965+0000 7fd967fff700 1 -- 172.21.15.27:0/229697899 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fd970080790 con 0x7fd9800f6820 2021-11-03T18:13:46.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.965+0000 7fd97f7fe700 1 --2- 172.21.15.27:0/229697899 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd968064100 0x7fd9680665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:13:46.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.966+0000 7fd97f7fe700 1 --2- 172.21.15.27:0/229697899 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd968064100 0x7fd9680665c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7fd974004630 tx=0x7fd974009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:13:46.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:46.969+0000 7fd967fff700 1 -- 172.21.15.27:0/229697899 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fd970052a90 con 0x7fd9800f6820 2021-11-03T18:13:46.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:46 smithi115 ceph-mon[65453]: pgmap v1773: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:47.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:46 smithi027 ceph-mon[97244]: pgmap v1773: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:47.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.143+0000 7fd98612a700 1 -- 172.21.15.27:0/229697899 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9800fac60 con 0x7fd968064100 2021-11-03T18:13:47.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.158+0000 7fd967fff700 1 -- 172.21.15.27:0/229697899 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fd9800fac60 con 0x7fd968064100 2021-11-03T18:13:47.156 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:13:47.156 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (65m) 4m ago 66m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:13:47.157 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (57m) 4m ago 66m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:13:47.157 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (56m) 3m ago 65m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:13:47.157 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (65m) 4m ago 66m 42.1M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:13:47.157 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (58m) 4m ago 61m 26.3M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:13:47.158 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (61m) 4m ago 61m 3847M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:13:47.158 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (57m) 3m ago 61m 25.9M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:13:47.158 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (59m) 3m ago 61m 27.0M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:13:47.158 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (59m) 4m ago 67m 572M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:13:47.159 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (58m) 3m ago 65m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:13:47.159 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (58m) 4m ago 68m 273M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:13:47.159 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (57m) 3m ago 65m 129M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:13:47.159 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (66m) 4m ago 66m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:13:47.159 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (65m) 3m ago 65m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:13:47.160 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (56m) 4m ago 65m 371M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:13:47.160 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (56m) 4m ago 64m 405M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:13:47.160 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (55m) 4m ago 64m 302M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:13:47.160 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (55m) 3m ago 63m 429M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:13:47.161 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (54m) 3m ago 63m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:13:47.161 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (54m) 3m ago 63m 348M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:13:47.161 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (65m) 4m ago 65m 57.6M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:13:47.162 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.160+0000 7fd98612a700 1 -- 172.21.15.27:0/229697899 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd968064100 msgr2=0x7fd9680665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:13:47.162 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.160+0000 7fd98612a700 1 --2- 172.21.15.27:0/229697899 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd968064100 0x7fd9680665c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 rx=0x7fd974004630 tx=0x7fd974009b30).stop 2021-11-03T18:13:47.163 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.160+0000 7fd98612a700 1 -- 172.21.15.27:0/229697899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9800f6820 msgr2=0x7fd9801073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:13:47.163 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.160+0000 7fd98612a700 1 --2- 172.21.15.27:0/229697899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9800f6820 0x7fd9801073e0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 rx=0x7fd9700026d0 tx=0x7fd970009de0).stop 2021-11-03T18:13:47.163 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.160+0000 7fd98612a700 1 -- 172.21.15.27:0/229697899 shutdown_connections 2021-11-03T18:13:47.164 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.160+0000 7fd98612a700 1 --2- 172.21.15.27:0/229697899 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd968064100 0x7fd9680665c0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:47.164 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.160+0000 7fd98612a700 1 --2- 172.21.15.27:0/229697899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9800f5580 0x7fd980106ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:47.164 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.160+0000 7fd98612a700 1 --2- 172.21.15.27:0/229697899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9800f6820 0x7fd9801073e0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:47.164 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.160+0000 7fd98612a700 1 -- 172.21.15.27:0/229697899 >> 172.21.15.27:0/229697899 conn(0x7fd9800f0ce0 msgr2=0x7fd98005ff20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:13:47.165 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.160+0000 7fd98612a700 1 -- 172.21.15.27:0/229697899 shutdown_connections 2021-11-03T18:13:47.165 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.161+0000 7fd98612a700 1 -- 172.21.15.27:0/229697899 wait complete. 2021-11-03T18:13:47.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.259+0000 7fbd4e49b700 1 -- 172.21.15.27:0/3325096397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd48060310 msgr2=0x7fbd480fb520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:13:47.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.259+0000 7fbd4e49b700 1 --2- 172.21.15.27:0/3325096397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd48060310 0x7fbd480fb520 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 rx=0x7fbd44004660 tx=0x7fbd44009b30).stop 2021-11-03T18:13:47.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.259+0000 7fbd4e49b700 1 -- 172.21.15.27:0/3325096397 shutdown_connections 2021-11-03T18:13:47.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.259+0000 7fbd4e49b700 1 --2- 172.21.15.27:0/3325096397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd480fba10 0x7fbd480fde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:47.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.259+0000 7fbd4e49b700 1 --2- 172.21.15.27:0/3325096397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd48060310 0x7fbd480fb520 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:47.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.259+0000 7fbd4e49b700 1 -- 172.21.15.27:0/3325096397 >> 172.21.15.27:0/3325096397 conn(0x7fbd480f0ba0 msgr2=0x7fbd480f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:13:47.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.260+0000 7fbd4e49b700 1 -- 172.21.15.27:0/3325096397 shutdown_connections 2021-11-03T18:13:47.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.260+0000 7fbd4e49b700 1 -- 172.21.15.27:0/3325096397 wait complete. 2021-11-03T18:13:47.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.260+0000 7fbd4e49b700 1 Processor -- start 2021-11-03T18:13:47.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.260+0000 7fbd4e49b700 1 -- start start 2021-11-03T18:13:47.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.260+0000 7fbd4e49b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd48060310 0x7fbd48108f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:13:47.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.260+0000 7fbd4e49b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd480fba10 0x7fbd48109460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:13:47.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.260+0000 7fbd4e49b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd48109a10 con 0x7fbd48060310 2021-11-03T18:13:47.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.260+0000 7fbd4e49b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd48109b50 con 0x7fbd480fba10 2021-11-03T18:13:47.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.260+0000 7fbd4d499700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd48060310 0x7fbd48108f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:13:47.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.260+0000 7fbd4d499700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd48060310 0x7fbd48108f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33674/0 (socket says 172.21.15.27:33674) 2021-11-03T18:13:47.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.260+0000 7fbd4d499700 1 -- 172.21.15.27:0/1875866633 learned_addr learned my addr 172.21.15.27:0/1875866633 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:13:47.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.261+0000 7fbd4cc98700 1 --2- 172.21.15.27:0/1875866633 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd480fba10 0x7fbd48109460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:13:47.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.261+0000 7fbd4d499700 1 -- 172.21.15.27:0/1875866633 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd480fba10 msgr2=0x7fbd48109460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:13:47.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.261+0000 7fbd4d499700 1 --2- 172.21.15.27:0/1875866633 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd480fba10 0x7fbd48109460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:47.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.261+0000 7fbd4d499700 1 -- 172.21.15.27:0/1875866633 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd44005040 con 0x7fbd48060310 2021-11-03T18:13:47.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.261+0000 7fbd4d499700 1 --2- 172.21.15.27:0/1875866633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd48060310 0x7fbd48108f70 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 rx=0x7fbd44009990 tx=0x7fbd44009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:13:47.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.261+0000 7fbd3dffb700 1 -- 172.21.15.27:0/1875866633 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd44018070 con 0x7fbd48060310 2021-11-03T18:13:47.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.261+0000 7fbd3dffb700 1 -- 172.21.15.27:0/1875866633 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fbd44007ba0 con 0x7fbd48060310 2021-11-03T18:13:47.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.261+0000 7fbd4e49b700 1 -- 172.21.15.27:0/1875866633 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd4810a490 con 0x7fbd48060310 2021-11-03T18:13:47.265 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.261+0000 7fbd3dffb700 1 -- 172.21.15.27:0/1875866633 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbd4401c3c0 con 0x7fbd48060310 2021-11-03T18:13:47.265 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.261+0000 7fbd4e49b700 1 -- 172.21.15.27:0/1875866633 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd4810a9f0 con 0x7fbd48060310 2021-11-03T18:13:47.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.262+0000 7fbd4e49b700 1 -- 172.21.15.27:0/1875866633 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd48045ad0 con 0x7fbd48060310 2021-11-03T18:13:47.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.263+0000 7fbd3dffb700 1 -- 172.21.15.27:0/1875866633 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fbd4401d420 con 0x7fbd48060310 2021-11-03T18:13:47.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.263+0000 7fbd3dffb700 1 --2- 172.21.15.27:0/1875866633 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbd34064100 0x7fbd340665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:13:47.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.263+0000 7fbd3dffb700 1 -- 172.21.15.27:0/1875866633 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fbd44082640 con 0x7fbd48060310 2021-11-03T18:13:47.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.263+0000 7fbd4cc98700 1 --2- 172.21.15.27:0/1875866633 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbd34064100 0x7fbd340665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:13:47.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.263+0000 7fbd4cc98700 1 --2- 172.21.15.27:0/1875866633 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbd34064100 0x7fbd340665c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7fbd480600e0 tx=0x7fbd38009be0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:13:47.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.267+0000 7fbd3dffb700 1 -- 172.21.15.27:0/1875866633 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fbd44054940 con 0x7fbd48060310 2021-11-03T18:13:47.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.495+0000 7fbd4e49b700 1 -- 172.21.15.27:0/1875866633 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbd48101000 con 0x7fbd48060310 2021-11-03T18:13:47.494 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.496+0000 7fbd3dffb700 1 -- 172.21.15.27:0/1875866633 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fbd44054530 con 0x7fbd48060310 2021-11-03T18:13:47.495 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:13:47.495 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:13:47.496 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:13:47.496 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:13:47.496 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:13:47.496 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:13:47.496 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:13:47.497 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:13:47.497 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:13:47.497 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:13:47.497 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:13:47.498 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:13:47.498 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:13:47.498 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:13:47.499 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:13:47.499 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:13:47.499 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:13:47.499 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:13:47.500 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.498+0000 7fbd4e49b700 1 -- 172.21.15.27:0/1875866633 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbd34064100 msgr2=0x7fbd340665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:13:47.500 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.498+0000 7fbd4e49b700 1 --2- 172.21.15.27:0/1875866633 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbd34064100 0x7fbd340665c0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7fbd480600e0 tx=0x7fbd38009be0).stop 2021-11-03T18:13:47.501 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.498+0000 7fbd4e49b700 1 -- 172.21.15.27:0/1875866633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd48060310 msgr2=0x7fbd48108f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:13:47.501 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.498+0000 7fbd4e49b700 1 --2- 172.21.15.27:0/1875866633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd48060310 0x7fbd48108f70 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 rx=0x7fbd44009990 tx=0x7fbd44009b30).stop 2021-11-03T18:13:47.501 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.498+0000 7fbd4e49b700 1 -- 172.21.15.27:0/1875866633 shutdown_connections 2021-11-03T18:13:47.501 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.498+0000 7fbd4e49b700 1 --2- 172.21.15.27:0/1875866633 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbd34064100 0x7fbd340665c0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:47.502 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.498+0000 7fbd4e49b700 1 --2- 172.21.15.27:0/1875866633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd48060310 0x7fbd48108f70 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:47.502 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.498+0000 7fbd4e49b700 1 --2- 172.21.15.27:0/1875866633 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd480fba10 0x7fbd48109460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:47.502 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.498+0000 7fbd4e49b700 1 -- 172.21.15.27:0/1875866633 >> 172.21.15.27:0/1875866633 conn(0x7fbd480f0ba0 msgr2=0x7fbd480f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:13:47.502 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.498+0000 7fbd4e49b700 1 -- 172.21.15.27:0/1875866633 shutdown_connections 2021-11-03T18:13:47.503 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.498+0000 7fbd4e49b700 1 -- 172.21.15.27:0/1875866633 wait complete. 2021-11-03T18:13:47.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.599+0000 7f2df2049700 1 -- 172.21.15.27:0/3873703636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2dec0f55e0 msgr2=0x7f2dec0f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:13:47.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.599+0000 7f2df2049700 1 --2- 172.21.15.27:0/3873703636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2dec0f55e0 0x7f2dec0f5a00 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 rx=0x7f2ddc004660 tx=0x7f2ddc009b30).stop 2021-11-03T18:13:47.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.599+0000 7f2df2049700 1 -- 172.21.15.27:0/3873703636 shutdown_connections 2021-11-03T18:13:47.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.599+0000 7f2df2049700 1 --2- 172.21.15.27:0/3873703636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2dec0f6880 0x7f2dec0f6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:47.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.599+0000 7f2df2049700 1 --2- 172.21.15.27:0/3873703636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2dec0f55e0 0x7f2dec0f5a00 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:47.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.599+0000 7f2df2049700 1 -- 172.21.15.27:0/3873703636 >> 172.21.15.27:0/3873703636 conn(0x7f2dec0f0d40 msgr2=0x7f2dec0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:13:47.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.599+0000 7f2df2049700 1 -- 172.21.15.27:0/3873703636 shutdown_connections 2021-11-03T18:13:47.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.599+0000 7f2df2049700 1 -- 172.21.15.27:0/3873703636 wait complete. 2021-11-03T18:13:47.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.599+0000 7f2df2049700 1 Processor -- start 2021-11-03T18:13:47.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.599+0000 7f2df2049700 1 -- start start 2021-11-03T18:13:47.601 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.600+0000 7f2df2049700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2dec0f55e0 0x7f2dec0f9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:13:47.601 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.600+0000 7f2df2049700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2dec0f6880 0x7f2dec0f83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:13:47.602 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.600+0000 7f2df2049700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dec0f8940 con 0x7f2dec0f55e0 2021-11-03T18:13:47.602 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.600+0000 7f2df2049700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dec0f8a80 con 0x7f2dec0f6880 2021-11-03T18:13:47.602 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.600+0000 7f2deb7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2dec0f55e0 0x7f2dec0f9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:13:47.602 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.600+0000 7f2deb7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2dec0f55e0 0x7f2dec0f9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33684/0 (socket says 172.21.15.27:33684) 2021-11-03T18:13:47.603 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.600+0000 7f2deb7fe700 1 -- 172.21.15.27:0/3508498406 learned_addr learned my addr 172.21.15.27:0/3508498406 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:13:47.603 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.600+0000 7f2deaffd700 1 --2- 172.21.15.27:0/3508498406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2dec0f6880 0x7f2dec0f83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:13:47.603 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.600+0000 7f2deb7fe700 1 -- 172.21.15.27:0/3508498406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2dec0f6880 msgr2=0x7f2dec0f83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:13:47.603 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.600+0000 7f2deb7fe700 1 --2- 172.21.15.27:0/3508498406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2dec0f6880 0x7f2dec0f83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:47.604 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.600+0000 7f2deb7fe700 1 -- 172.21.15.27:0/3508498406 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ddc005040 con 0x7f2dec0f55e0 2021-11-03T18:13:47.604 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.600+0000 7f2deb7fe700 1 --2- 172.21.15.27:0/3508498406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2dec0f55e0 0x7f2dec0f9f60 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 rx=0x7f2ddc004f60 tx=0x7f2ddc009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:13:47.604 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.600+0000 7f2dd3fff700 1 -- 172.21.15.27:0/3508498406 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ddc018070 con 0x7f2dec0f55e0 2021-11-03T18:13:47.604 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.600+0000 7f2df2049700 1 -- 172.21.15.27:0/3508498406 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dec0f8d00 con 0x7f2dec0f55e0 2021-11-03T18:13:47.604 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.600+0000 7f2dd3fff700 1 -- 172.21.15.27:0/3508498406 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f2ddc007ba0 con 0x7f2dec0f55e0 2021-11-03T18:13:47.605 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.600+0000 7f2dd3fff700 1 -- 172.21.15.27:0/3508498406 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ddc01c620 con 0x7f2dec0f55e0 2021-11-03T18:13:47.605 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.600+0000 7f2df2049700 1 -- 172.21.15.27:0/3508498406 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dec0f9260 con 0x7f2dec0f55e0 2021-11-03T18:13:47.605 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.601+0000 7f2df2049700 1 -- 172.21.15.27:0/3508498406 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dec045ad0 con 0x7f2dec0f55e0 2021-11-03T18:13:47.605 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.602+0000 7f2dd3fff700 1 -- 172.21.15.27:0/3508498406 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f2ddc01c8e0 con 0x7f2dec0f55e0 2021-11-03T18:13:47.606 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.602+0000 7f2dd3fff700 1 --2- 172.21.15.27:0/3508498406 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2dd40640b0 0x7f2dd4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:13:47.606 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.602+0000 7f2dd3fff700 1 -- 172.21.15.27:0/3508498406 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f2ddc082740 con 0x7f2dec0f55e0 2021-11-03T18:13:47.606 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.602+0000 7f2deaffd700 1 --2- 172.21.15.27:0/3508498406 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2dd40640b0 0x7f2dd4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:13:47.607 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.603+0000 7f2deaffd700 1 --2- 172.21.15.27:0/3508498406 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2dd40640b0 0x7f2dd4066570 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f2de0004a70 tx=0x7f2de0009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:13:47.607 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.606+0000 7f2dd3fff700 1 -- 172.21.15.27:0/3508498406 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f2ddc054a40 con 0x7f2dec0f55e0 2021-11-03T18:13:47.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.812+0000 7f2df2049700 1 -- 172.21.15.27:0/3508498406 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2dec10f660 con 0x7f2dec0f55e0 2021-11-03T18:13:47.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.812+0000 7f2dd3fff700 1 -- 172.21.15.27:0/3508498406 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f2ddc054630 con 0x7f2dec0f55e0 2021-11-03T18:13:47.810 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:13:47.810 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:13:47.811 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:13:47.811 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:13:47.811 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:13:47.812 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:13:47.812 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:13:47.812 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:13:47.812 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:13:47.813 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:13:47.813 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:13:47.813 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:13:47.813 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:13:47.813 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:13:47.814 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:13:47.814 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:13:47.814 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:13:47.814 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:13:47.815 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:13:47.815 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:13:47.815 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:13:47.815 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:13:47.816 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:13:47.816 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:13:47.816 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:13:47.816 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:13:47.816 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:13:47.817 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:13:47.817 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:13:47.817 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:13:47.818 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:13:47.818 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:13:47.818 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:13:47.819 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:13:47.819 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:13:47.819 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:13:47.819 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:13:47.819 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:13:47.820 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:13:47.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.814+0000 7f2df2049700 1 -- 172.21.15.27:0/3508498406 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2dd40640b0 msgr2=0x7f2dd4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:13:47.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.814+0000 7f2df2049700 1 --2- 172.21.15.27:0/3508498406 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2dd40640b0 0x7f2dd4066570 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 rx=0x7f2de0004a70 tx=0x7f2de0009040).stop 2021-11-03T18:13:47.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.814+0000 7f2df2049700 1 -- 172.21.15.27:0/3508498406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2dec0f55e0 msgr2=0x7f2dec0f9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:13:47.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.814+0000 7f2df2049700 1 --2- 172.21.15.27:0/3508498406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2dec0f55e0 0x7f2dec0f9f60 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 rx=0x7f2ddc004f60 tx=0x7f2ddc009b30).stop 2021-11-03T18:13:47.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.814+0000 7f2df2049700 1 -- 172.21.15.27:0/3508498406 shutdown_connections 2021-11-03T18:13:47.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.814+0000 7f2df2049700 1 --2- 172.21.15.27:0/3508498406 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2dd40640b0 0x7f2dd4066570 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:47.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.814+0000 7f2df2049700 1 --2- 172.21.15.27:0/3508498406 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2dec0f55e0 0x7f2dec0f9f60 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:47.823 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.814+0000 7f2df2049700 1 --2- 172.21.15.27:0/3508498406 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2dec0f6880 0x7f2dec0f83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:13:47.823 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.814+0000 7f2df2049700 1 -- 172.21.15.27:0/3508498406 >> 172.21.15.27:0/3508498406 conn(0x7f2dec0f0d40 msgr2=0x7f2dec05f740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:13:47.823 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.814+0000 7f2df2049700 1 -- 172.21.15.27:0/3508498406 shutdown_connections 2021-11-03T18:13:47.823 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:13:47.814+0000 7f2df2049700 1 -- 172.21.15.27:0/3508498406 wait complete. 2021-11-03T18:13:47.823 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:13:47.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:47 smithi115 ceph-mon[65453]: pgmap v1774: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:47.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:47 smithi115 ceph-mon[65453]: from='client.35928 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:13:47.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:47 smithi115 ceph-mon[65453]: from='client.35932 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:13:47.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:47 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1875866633' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:48.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:47 smithi027 ceph-mon[97244]: pgmap v1774: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:48.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:47 smithi027 ceph-mon[97244]: from='client.35928 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:13:48.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:47 smithi027 ceph-mon[97244]: from='client.35932 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:13:48.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:47 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1875866633' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:48.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:48 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3508498406' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:13:48.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:13:48.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:13:48.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:13:48.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:48.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:48.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:48.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:48.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:48 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:13:49.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:48 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3508498406' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:13:49.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:13:49.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:13:49.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:13:49.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:49.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:49.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:49.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:49.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:48 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:13:49.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:49 smithi115 ceph-mon[65453]: pgmap v1775: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:50.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:49 smithi027 ceph-mon[97244]: pgmap v1775: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:50.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3472.442804 secs 2021-11-03T18:13:51.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3472.442804 secs 2021-11-03T18:13:51.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:51 smithi115 ceph-mon[65453]: pgmap v1776: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:52.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:51 smithi027 ceph-mon[97244]: pgmap v1776: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:54.236 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:53 smithi115 ceph-mon[65453]: pgmap v1777: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:54.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:53 smithi027 ceph-mon[97244]: pgmap v1777: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:56.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3477.442929 secs 2021-11-03T18:13:56.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:55 smithi115 ceph-mon[65453]: pgmap v1778: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:56.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:13:56.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:13:56.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3477.442929 secs 2021-11-03T18:13:56.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:55 smithi027 ceph-mon[97244]: pgmap v1778: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:56.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:13:56.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:13:58.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:57 smithi115 ceph-mon[65453]: pgmap v1779: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:58.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:57 smithi027 ceph-mon[97244]: pgmap v1779: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:13:59.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:13:59.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:13:59.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:13:59.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:59.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:59.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:59.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:59.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:13:58 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:13:59.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:13:59.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:13:59.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:13:59.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:59.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:59.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:59.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:13:59.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:13:58 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:14:00.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:00 smithi115 ceph-mon[65453]: pgmap v1780: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:00.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3482.443065 secs 2021-11-03T18:14:00.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:00 smithi027 ceph-mon[97244]: pgmap v1780: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:00.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3482.443065 secs 2021-11-03T18:14:02.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:02 smithi115 ceph-mon[65453]: pgmap v1781: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:02.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:02 smithi027 ceph-mon[97244]: pgmap v1781: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:04.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:04 smithi115 ceph-mon[65453]: pgmap v1782: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:04.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:04 smithi027 ceph-mon[97244]: pgmap v1782: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:05.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3487.443197 secs 2021-11-03T18:14:05.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3487.443197 secs 2021-11-03T18:14:06.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:06 smithi115 ceph-mon[65453]: pgmap v1783: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:06.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:06 smithi027 ceph-mon[97244]: pgmap v1783: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:08.708 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:08 smithi115 ceph-mon[65453]: pgmap v1784: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:08.730 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:08 smithi027 ceph-mon[97244]: pgmap v1784: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:09.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:14:09.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:09 smithi115 ceph-mon[65453]: poll took 60173.040 ms: 1 events 2021-11-03T18:14:09.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:14:09.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:14:09.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:09.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:09.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:09.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:09.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:14:09.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:09 smithi027 ceph-mon[97244]: poll took 60173.040 ms: 1 events 2021-11-03T18:14:09.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:14:09.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:14:09.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:09.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:09.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:09.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:10.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:10 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:14:10.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:10 smithi115 ceph-mon[65453]: pgmap v1785: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:10.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3492.443322 secs 2021-11-03T18:14:10.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:10 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:14:10.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:10 smithi027 ceph-mon[97244]: pgmap v1785: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:10.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3492.443322 secs 2021-11-03T18:14:12.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:12 smithi115 ceph-mon[65453]: pgmap v1786: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:12.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:12 smithi027 ceph-mon[97244]: pgmap v1786: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:14.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:14 smithi115 ceph-mon[65453]: pgmap v1787: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:14.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:14 smithi027 ceph-mon[97244]: pgmap v1787: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:15.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3497.443453 secs 2021-11-03T18:14:15.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3497.443453 secs 2021-11-03T18:14:16.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:16 smithi115 ceph-mon[65453]: pgmap v1788: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:16.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:16 smithi027 ceph-mon[97244]: pgmap v1788: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:17.927 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:17.929+0000 7fb1017bf700 1 -- 172.21.15.27:0/2196659543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0fc0f96d0 msgr2=0x7fb0fc0fbb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:14:17.927 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:17.929+0000 7fb1017bf700 1 --2- 172.21.15.27:0/2196659543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0fc0f96d0 0x7fb0fc0fbb00 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 rx=0x7fb0ec004660 tx=0x7fb0ec009b30).stop 2021-11-03T18:14:17.927 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:17.929+0000 7fb1017bf700 1 -- 172.21.15.27:0/2196659543 shutdown_connections 2021-11-03T18:14:17.927 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:17.929+0000 7fb1017bf700 1 --2- 172.21.15.27:0/2196659543 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0fc0f96d0 0x7fb0fc0fbb00 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:17.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:17.929+0000 7fb1017bf700 1 --2- 172.21.15.27:0/2196659543 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0fc060250 0x7fb0fc0f91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:17.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:17.929+0000 7fb1017bf700 1 -- 172.21.15.27:0/2196659543 >> 172.21.15.27:0/2196659543 conn(0x7fb0fc0f0ba0 msgr2=0x7fb0fc0f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:14:17.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:17.929+0000 7fb1017bf700 1 -- 172.21.15.27:0/2196659543 shutdown_connections 2021-11-03T18:14:17.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:17.929+0000 7fb1017bf700 1 -- 172.21.15.27:0/2196659543 wait complete. 2021-11-03T18:14:17.929 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:17.929+0000 7fb1017bf700 1 Processor -- start 2021-11-03T18:14:17.931 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:17.930+0000 7fb1017bf700 1 -- start start 2021-11-03T18:14:17.931 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:17.930+0000 7fb1017bf700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0fc060250 0x7fb0fc106d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:14:17.931 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:17.930+0000 7fb1017bf700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0fc0f96d0 0x7fb0fc107260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:14:17.932 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:17.930+0000 7fb1017bf700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc107860 con 0x7fb0fc060250 2021-11-03T18:14:17.932 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:17.930+0000 7fb1017bf700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc108070 con 0x7fb0fc0f96d0 2021-11-03T18:14:17.932 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:17.930+0000 7fb0fbfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0fc060250 0x7fb0fc106d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:14:17.932 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:17.930+0000 7fb0fbfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0fc060250 0x7fb0fc106d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33702/0 (socket says 172.21.15.27:33702) 2021-11-03T18:14:17.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:17.930+0000 7fb0fbfff700 1 -- 172.21.15.27:0/3166450641 learned_addr learned my addr 172.21.15.27:0/3166450641 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:14:17.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:17.930+0000 7fb0fb7fe700 1 --2- 172.21.15.27:0/3166450641 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0fc0f96d0 0x7fb0fc107260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:14:17.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:17.930+0000 7fb0fbfff700 1 -- 172.21.15.27:0/3166450641 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0fc0f96d0 msgr2=0x7fb0fc107260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:14:17.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:17.930+0000 7fb0fbfff700 1 --2- 172.21.15.27:0/3166450641 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0fc0f96d0 0x7fb0fc107260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:17.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:17.930+0000 7fb0fbfff700 1 -- 172.21.15.27:0/3166450641 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ec005040 con 0x7fb0fc060250 2021-11-03T18:14:17.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:17.930+0000 7fb0fbfff700 1 --2- 172.21.15.27:0/3166450641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0fc060250 0x7fb0fc106d70 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 rx=0x7fb0e400e7f0 tx=0x7fb0e400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:14:17.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:17.931+0000 7fb0f8ff9700 1 -- 172.21.15.27:0/3166450641 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0e40092e0 con 0x7fb0fc060250 2021-11-03T18:14:17.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:17.931+0000 7fb1017bf700 1 -- 172.21.15.27:0/3166450641 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0fc1082f0 con 0x7fb0fc060250 2021-11-03T18:14:17.935 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:17.931+0000 7fb0f8ff9700 1 -- 172.21.15.27:0/3166450641 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fb0e400f040 con 0x7fb0fc060250 2021-11-03T18:14:17.935 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:17.931+0000 7fb0f8ff9700 1 -- 172.21.15.27:0/3166450641 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0e4014750 con 0x7fb0fc060250 2021-11-03T18:14:17.935 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:17.931+0000 7fb1017bf700 1 -- 172.21.15.27:0/3166450641 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0fc1088b0 con 0x7fb0fc060250 2021-11-03T18:14:17.935 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:17.932+0000 7fb1017bf700 1 -- 172.21.15.27:0/3166450641 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0fc059100 con 0x7fb0fc060250 2021-11-03T18:14:17.935 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:17.932+0000 7fb0f8ff9700 1 -- 172.21.15.27:0/3166450641 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fb0e401b030 con 0x7fb0fc060250 2021-11-03T18:14:17.936 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:17.932+0000 7fb0f8ff9700 1 --2- 172.21.15.27:0/3166450641 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb0e8064100 0x7fb0e80665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:14:17.936 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:17.933+0000 7fb0f8ff9700 1 -- 172.21.15.27:0/3166450641 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fb0e40806e0 con 0x7fb0fc060250 2021-11-03T18:14:17.936 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:17.933+0000 7fb0fb7fe700 1 --2- 172.21.15.27:0/3166450641 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb0e8064100 0x7fb0e80665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:14:17.937 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:17.933+0000 7fb0fb7fe700 1 --2- 172.21.15.27:0/3166450641 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb0e8064100 0x7fb0e80665c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7fb0ec000c00 tx=0x7fb0ec00b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:14:17.937 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:17.937+0000 7fb0f8ff9700 1 -- 172.21.15.27:0/3166450641 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fb0e40529a0 con 0x7fb0fc060250 2021-11-03T18:14:18.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.115+0000 7fb1017bf700 1 -- 172.21.15.27:0/3166450641 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb0fc0fed80 con 0x7fb0e8064100 2021-11-03T18:14:18.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.117+0000 7fb0f8ff9700 1 -- 172.21.15.27:0/3166450641 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fb0fc0fed80 con 0x7fb0e8064100 2021-11-03T18:14:18.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.118+0000 7fb1017bf700 1 -- 172.21.15.27:0/3166450641 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb0e8064100 msgr2=0x7fb0e80665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:14:18.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.118+0000 7fb1017bf700 1 --2- 172.21.15.27:0/3166450641 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb0e8064100 0x7fb0e80665c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7fb0ec000c00 tx=0x7fb0ec00b040).stop 2021-11-03T18:14:18.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.118+0000 7fb1017bf700 1 -- 172.21.15.27:0/3166450641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0fc060250 msgr2=0x7fb0fc106d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:14:18.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.118+0000 7fb1017bf700 1 --2- 172.21.15.27:0/3166450641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0fc060250 0x7fb0fc106d70 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 rx=0x7fb0e400e7f0 tx=0x7fb0e400b040).stop 2021-11-03T18:14:18.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.118+0000 7fb1017bf700 1 -- 172.21.15.27:0/3166450641 shutdown_connections 2021-11-03T18:14:18.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.118+0000 7fb1017bf700 1 --2- 172.21.15.27:0/3166450641 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb0e8064100 0x7fb0e80665c0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:18.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.118+0000 7fb1017bf700 1 --2- 172.21.15.27:0/3166450641 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0fc060250 0x7fb0fc106d70 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:18.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.118+0000 7fb1017bf700 1 --2- 172.21.15.27:0/3166450641 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0fc0f96d0 0x7fb0fc107260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:18.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.118+0000 7fb1017bf700 1 -- 172.21.15.27:0/3166450641 >> 172.21.15.27:0/3166450641 conn(0x7fb0fc0f0ba0 msgr2=0x7fb0fc0f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:14:18.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.119+0000 7fb1017bf700 1 -- 172.21.15.27:0/3166450641 shutdown_connections 2021-11-03T18:14:18.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.119+0000 7fb1017bf700 1 -- 172.21.15.27:0/3166450641 wait complete. 2021-11-03T18:14:18.126 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:14:18.218 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.218+0000 7f5f842db700 1 -- 172.21.15.27:0/1772597180 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f7c0f5ef0 msgr2=0x7f5f7c0f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:14:18.218 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.218+0000 7f5f842db700 1 --2- 172.21.15.27:0/1772597180 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f7c0f5ef0 0x7f5f7c0f6920 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 rx=0x7f5f6c004660 tx=0x7f5f6c009b30).stop 2021-11-03T18:14:18.219 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.219+0000 7f5f842db700 1 -- 172.21.15.27:0/1772597180 shutdown_connections 2021-11-03T18:14:18.219 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.219+0000 7f5f842db700 1 --2- 172.21.15.27:0/1772597180 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f7c0f5ef0 0x7f5f7c0f6920 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:18.219 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.219+0000 7f5f842db700 1 --2- 172.21.15.27:0/1772597180 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f7c0f55e0 0x7f5f7c0f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:18.219 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.219+0000 7f5f842db700 1 -- 172.21.15.27:0/1772597180 >> 172.21.15.27:0/1772597180 conn(0x7f5f7c0f0de0 msgr2=0x7f5f7c0f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:14:18.219 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.219+0000 7f5f842db700 1 -- 172.21.15.27:0/1772597180 shutdown_connections 2021-11-03T18:14:18.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.219+0000 7f5f842db700 1 -- 172.21.15.27:0/1772597180 wait complete. 2021-11-03T18:14:18.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.219+0000 7f5f842db700 1 Processor -- start 2021-11-03T18:14:18.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.219+0000 7f5f842db700 1 -- start start 2021-11-03T18:14:18.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.219+0000 7f5f842db700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f7c0f55e0 0x7f5f7c106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:14:18.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.219+0000 7f5f842db700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f7c0f5ef0 0x7f5f7c107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:14:18.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.219+0000 7f5f842db700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c107a10 con 0x7f5f7c0f55e0 2021-11-03T18:14:18.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.219+0000 7f5f842db700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7c107b50 con 0x7f5f7c0f5ef0 2021-11-03T18:14:18.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.219+0000 7f5f82077700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f7c0f55e0 0x7f5f7c106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:14:18.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.219+0000 7f5f82077700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f7c0f55e0 0x7f5f7c106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33714/0 (socket says 172.21.15.27:33714) 2021-11-03T18:14:18.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.219+0000 7f5f82077700 1 -- 172.21.15.27:0/1583624889 learned_addr learned my addr 172.21.15.27:0/1583624889 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:14:18.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.220+0000 7f5f81876700 1 --2- 172.21.15.27:0/1583624889 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f7c0f5ef0 0x7f5f7c107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:14:18.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.220+0000 7f5f82077700 1 -- 172.21.15.27:0/1583624889 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f7c0f5ef0 msgr2=0x7f5f7c107460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:14:18.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.220+0000 7f5f82077700 1 --2- 172.21.15.27:0/1583624889 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f7c0f5ef0 0x7f5f7c107460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:18.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.220+0000 7f5f82077700 1 -- 172.21.15.27:0/1583624889 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f6c005040 con 0x7f5f7c0f55e0 2021-11-03T18:14:18.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.220+0000 7f5f82077700 1 --2- 172.21.15.27:0/1583624889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f7c0f55e0 0x7f5f7c106f70 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 rx=0x7f5f780026d0 tx=0x7f5f78009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:14:18.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.220+0000 7f5f72ffd700 1 -- 172.21.15.27:0/1583624889 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f780079c0 con 0x7f5f7c0f55e0 2021-11-03T18:14:18.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.220+0000 7f5f842db700 1 -- 172.21.15.27:0/1583624889 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f7c1084f0 con 0x7f5f7c0f55e0 2021-11-03T18:14:18.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.220+0000 7f5f72ffd700 1 -- 172.21.15.27:0/1583624889 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f5f7800f040 con 0x7f5f7c0f55e0 2021-11-03T18:14:18.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.220+0000 7f5f72ffd700 1 -- 172.21.15.27:0/1583624889 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5f780148b0 con 0x7f5f7c0f55e0 2021-11-03T18:14:18.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.220+0000 7f5f842db700 1 -- 172.21.15.27:0/1583624889 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f7c108ab0 con 0x7f5f7c0f55e0 2021-11-03T18:14:18.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.221+0000 7f5f842db700 1 -- 172.21.15.27:0/1583624889 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f7c045ad0 con 0x7f5f7c0f55e0 2021-11-03T18:14:18.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.221+0000 7f5f72ffd700 1 -- 172.21.15.27:0/1583624889 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f5f78007b20 con 0x7f5f7c0f55e0 2021-11-03T18:14:18.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.222+0000 7f5f72ffd700 1 --2- 172.21.15.27:0/1583624889 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5f680640b0 0x7f5f68066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:14:18.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.222+0000 7f5f72ffd700 1 -- 172.21.15.27:0/1583624889 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f5f78080790 con 0x7f5f7c0f55e0 2021-11-03T18:14:18.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.222+0000 7f5f81876700 1 --2- 172.21.15.27:0/1583624889 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5f680640b0 0x7f5f68066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:14:18.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.222+0000 7f5f81876700 1 --2- 172.21.15.27:0/1583624889 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5f680640b0 0x7f5f68066570 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7f5f6c004630 tx=0x7f5f6c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:14:18.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.226+0000 7f5f72ffd700 1 -- 172.21.15.27:0/1583624889 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f5f78052a90 con 0x7f5f7c0f55e0 2021-11-03T18:14:18.397 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.399+0000 7f5f842db700 1 -- 172.21.15.27:0/1583624889 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f7c05fe50 con 0x7f5f680640b0 2021-11-03T18:14:18.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.414+0000 7f5f72ffd700 1 -- 172.21.15.27:0/1583624889 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f5f7c05fe50 con 0x7f5f680640b0 2021-11-03T18:14:18.412 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:14:18.412 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (65m) 5m ago 67m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:14:18.412 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (57m) 5m ago 67m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:14:18.412 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (57m) 4m ago 66m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:14:18.413 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (65m) 5m ago 66m 42.1M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:14:18.413 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (59m) 5m ago 62m 26.3M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:14:18.413 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (62m) 5m ago 62m 3847M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:14:18.413 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (58m) 4m ago 62m 25.9M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:14:18.414 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (59m) 4m ago 62m 27.0M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:14:18.414 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (59m) 5m ago 68m 572M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:14:18.414 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (58m) 4m ago 66m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:14:18.414 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (58m) 5m ago 68m 273M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:14:18.414 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (58m) 4m ago 66m 129M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:14:18.415 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (66m) 5m ago 66m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:14:18.415 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (66m) 4m ago 66m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:14:18.415 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (57m) 5m ago 65m 371M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:14:18.415 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (56m) 5m ago 65m 405M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:14:18.416 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (56m) 5m ago 64m 302M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:14:18.416 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (55m) 4m ago 64m 429M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:14:18.416 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (55m) 4m ago 64m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:14:18.416 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (54m) 4m ago 63m 348M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:14:18.416 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (65m) 5m ago 66m 57.6M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:14:18.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.416+0000 7f5f842db700 1 -- 172.21.15.27:0/1583624889 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5f680640b0 msgr2=0x7f5f68066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:14:18.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.416+0000 7f5f842db700 1 --2- 172.21.15.27:0/1583624889 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5f680640b0 0x7f5f68066570 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 rx=0x7f5f6c004630 tx=0x7f5f6c009b30).stop 2021-11-03T18:14:18.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.416+0000 7f5f842db700 1 -- 172.21.15.27:0/1583624889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f7c0f55e0 msgr2=0x7f5f7c106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:14:18.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.416+0000 7f5f842db700 1 --2- 172.21.15.27:0/1583624889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f7c0f55e0 0x7f5f7c106f70 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 rx=0x7f5f780026d0 tx=0x7f5f78009de0).stop 2021-11-03T18:14:18.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.416+0000 7f5f842db700 1 -- 172.21.15.27:0/1583624889 shutdown_connections 2021-11-03T18:14:18.419 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.416+0000 7f5f842db700 1 --2- 172.21.15.27:0/1583624889 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5f680640b0 0x7f5f68066570 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:18.419 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.416+0000 7f5f842db700 1 --2- 172.21.15.27:0/1583624889 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5f7c0f55e0 0x7f5f7c106f70 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:18.419 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.416+0000 7f5f842db700 1 --2- 172.21.15.27:0/1583624889 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5f7c0f5ef0 0x7f5f7c107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:18.419 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.416+0000 7f5f842db700 1 -- 172.21.15.27:0/1583624889 >> 172.21.15.27:0/1583624889 conn(0x7f5f7c0f0de0 msgr2=0x7f5f7c05f740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:14:18.420 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.416+0000 7f5f842db700 1 -- 172.21.15.27:0/1583624889 shutdown_connections 2021-11-03T18:14:18.420 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.416+0000 7f5f842db700 1 -- 172.21.15.27:0/1583624889 wait complete. 2021-11-03T18:14:18.513 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.515+0000 7f2c064d6700 1 -- 172.21.15.27:0/3463213065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c000f66e0 msgr2=0x7f2c000f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:14:18.513 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.515+0000 7f2c064d6700 1 --2- 172.21.15.27:0/3463213065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c000f66e0 0x7f2c000f6b00 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 rx=0x7f2bf0004660 tx=0x7f2bf0009b30).stop 2021-11-03T18:14:18.513 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.515+0000 7f2c064d6700 1 -- 172.21.15.27:0/3463213065 shutdown_connections 2021-11-03T18:14:18.514 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.515+0000 7f2c064d6700 1 --2- 172.21.15.27:0/3463213065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c000f6ff0 0x7f2c000f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:18.514 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.515+0000 7f2c064d6700 1 --2- 172.21.15.27:0/3463213065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c000f66e0 0x7f2c000f6b00 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:18.514 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.515+0000 7f2c064d6700 1 -- 172.21.15.27:0/3463213065 >> 172.21.15.27:0/3463213065 conn(0x7f2c000f1f00 msgr2=0x7f2c000f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:14:18.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.515+0000 7f2c064d6700 1 -- 172.21.15.27:0/3463213065 shutdown_connections 2021-11-03T18:14:18.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.515+0000 7f2c064d6700 1 -- 172.21.15.27:0/3463213065 wait complete. 2021-11-03T18:14:18.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.515+0000 7f2c064d6700 1 Processor -- start 2021-11-03T18:14:18.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.516+0000 7f2c064d6700 1 -- start start 2021-11-03T18:14:18.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.516+0000 7f2c064d6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c000f66e0 0x7f2c00106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:14:18.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.516+0000 7f2c064d6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c000f6ff0 0x7f2c00107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:14:18.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.516+0000 7f2c064d6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c00107a00 con 0x7f2c000f66e0 2021-11-03T18:14:18.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.516+0000 7f2c064d6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c00107b40 con 0x7f2c000f6ff0 2021-11-03T18:14:18.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.516+0000 7f2bfffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c000f66e0 0x7f2c00106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:14:18.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.516+0000 7f2bfffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c000f66e0 0x7f2c00106f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33724/0 (socket says 172.21.15.27:33724) 2021-11-03T18:14:18.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.516+0000 7f2bfffff700 1 -- 172.21.15.27:0/3693002717 learned_addr learned my addr 172.21.15.27:0/3693002717 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:14:18.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.516+0000 7f2bfffff700 1 -- 172.21.15.27:0/3693002717 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c000f6ff0 msgr2=0x7f2c00107450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:14:18.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.516+0000 7f2bfffff700 1 --2- 172.21.15.27:0/3693002717 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c000f6ff0 0x7f2c00107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:14:18.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.516+0000 7f2bfffff700 1 -- 172.21.15.27:0/3693002717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bf0005040 con 0x7f2c000f66e0 2021-11-03T18:14:18.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.516+0000 7f2bfffff700 1 --2- 172.21.15.27:0/3693002717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c000f66e0 0x7f2c00106f60 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 rx=0x7f2bf0009990 tx=0x7f2bf0009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:14:18.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.517+0000 7f2bfcff9700 1 -- 172.21.15.27:0/3693002717 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bf0018070 con 0x7f2c000f66e0 2021-11-03T18:14:18.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.517+0000 7f2c064d6700 1 -- 172.21.15.27:0/3693002717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c00108480 con 0x7f2c000f66e0 2021-11-03T18:14:18.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.517+0000 7f2c064d6700 1 -- 172.21.15.27:0/3693002717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c001089e0 con 0x7f2c000f66e0 2021-11-03T18:14:18.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.517+0000 7f2bfcff9700 1 -- 172.21.15.27:0/3693002717 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f2bf0007ca0 con 0x7f2c000f66e0 2021-11-03T18:14:18.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.517+0000 7f2bfcff9700 1 -- 172.21.15.27:0/3693002717 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bf001c620 con 0x7f2c000f66e0 2021-11-03T18:14:18.524 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.518+0000 7f2c064d6700 1 -- 172.21.15.27:0/3693002717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c00045ad0 con 0x7f2c000f66e0 2021-11-03T18:14:18.524 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.518+0000 7f2bfcff9700 1 -- 172.21.15.27:0/3693002717 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f2bf001d960 con 0x7f2c000f66e0 2021-11-03T18:14:18.524 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.518+0000 7f2bfcff9700 1 --2- 172.21.15.27:0/3693002717 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2bec0640b0 0x7f2bec066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:14:18.524 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.519+0000 7f2bfcff9700 1 -- 172.21.15.27:0/3693002717 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f2bf0082570 con 0x7f2c000f66e0 2021-11-03T18:14:18.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.519+0000 7f2bff7fe700 1 --2- 172.21.15.27:0/3693002717 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2bec0640b0 0x7f2bec066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:14:18.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.519+0000 7f2bff7fe700 1 --2- 172.21.15.27:0/3693002717 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2bec0640b0 0x7f2bec066570 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f2be80047d0 tx=0x7f2be8007a40).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:14:18.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.523+0000 7f2bfcff9700 1 -- 172.21.15.27:0/3693002717 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f2bf0054930 con 0x7f2c000f66e0 2021-11-03T18:14:18.736 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:18 smithi115 ceph-mon[65453]: pgmap v1789: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:18.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.753+0000 7f2c064d6700 1 -- 172.21.15.27:0/3693002717 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2c000fefe0 con 0x7f2c000f66e0 2021-11-03T18:14:18.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.754+0000 7f2bfcff9700 1 -- 172.21.15.27:0/3693002717 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f2bf0054520 con 0x7f2c000f66e0 2021-11-03T18:14:18.752 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:14:18.752 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:14:18.752 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:14:18.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:14:18.753 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:14:18.753 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:14:18.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:14:18.754 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:14:18.754 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:14:18.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:14:18.754 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:14:18.754 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:14:18.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:14:18.755 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:14:18.755 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:14:18.755 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:14:18.756 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:14:18.756 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:14:18.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.756+0000 7f2c064d6700 1 -- 172.21.15.27:0/3693002717 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2bec0640b0 msgr2=0x7f2bec066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:14:18.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.756+0000 7f2c064d6700 1 --2- 172.21.15.27:0/3693002717 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2bec0640b0 0x7f2bec066570 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f2be80047d0 tx=0x7f2be8007a40).stop 2021-11-03T18:14:18.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.756+0000 7f2c064d6700 1 -- 172.21.15.27:0/3693002717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c000f66e0 msgr2=0x7f2c00106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:14:18.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.756+0000 7f2c064d6700 1 --2- 172.21.15.27:0/3693002717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c000f66e0 0x7f2c00106f60 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 rx=0x7f2bf0009990 tx=0x7f2bf0009b30).stop 2021-11-03T18:14:18.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.756+0000 7f2c064d6700 1 -- 172.21.15.27:0/3693002717 shutdown_connections 2021-11-03T18:14:18.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.756+0000 7f2c064d6700 1 --2- 172.21.15.27:0/3693002717 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2bec0640b0 0x7f2bec066570 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:18.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.756+0000 7f2c064d6700 1 --2- 172.21.15.27:0/3693002717 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2c000f66e0 0x7f2c00106f60 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:18.759 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.756+0000 7f2c064d6700 1 --2- 172.21.15.27:0/3693002717 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2c000f6ff0 0x7f2c00107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:14:18.759 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.756+0000 7f2c064d6700 1 -- 172.21.15.27:0/3693002717 >> 172.21.15.27:0/3693002717 conn(0x7f2c000f1f00 msgr2=0x7f2c000fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:14:18.759 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.756+0000 7f2c064d6700 1 -- 172.21.15.27:0/3693002717 shutdown_connections 2021-11-03T18:14:18.759 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.756+0000 7f2c064d6700 1 -- 172.21.15.27:0/3693002717 wait complete. 2021-11-03T18:14:18.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:18 smithi027 ceph-mon[97244]: pgmap v1789: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:18.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.858+0000 7f9c77fff700 1 -- 172.21.15.27:0/1822291349 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c780f77d0 msgr2=0x7f9c780f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:14:18.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.858+0000 7f9c77fff700 1 --2- 172.21.15.27:0/1822291349 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c780f77d0 0x7f9c780f7c30 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 rx=0x7f9c6c004660 tx=0x7f9c6c009b30).stop 2021-11-03T18:14:18.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.858+0000 7f9c77fff700 1 -- 172.21.15.27:0/1822291349 shutdown_connections 2021-11-03T18:14:18.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.858+0000 7f9c77fff700 1 --2- 172.21.15.27:0/1822291349 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c780f77d0 0x7f9c780f7c30 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:18.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.858+0000 7f9c77fff700 1 --2- 172.21.15.27:0/1822291349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c780f6530 0x7f9c780f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:18.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.858+0000 7f9c77fff700 1 -- 172.21.15.27:0/1822291349 >> 172.21.15.27:0/1822291349 conn(0x7f9c780f1cd0 msgr2=0x7f9c780f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:14:18.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.858+0000 7f9c77fff700 1 -- 172.21.15.27:0/1822291349 shutdown_connections 2021-11-03T18:14:18.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.858+0000 7f9c77fff700 1 -- 172.21.15.27:0/1822291349 wait complete. 2021-11-03T18:14:18.860 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.859+0000 7f9c77fff700 1 Processor -- start 2021-11-03T18:14:18.861 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.859+0000 7f9c77fff700 1 -- start start 2021-11-03T18:14:18.861 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.859+0000 7f9c77fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c780f6530 0x7f9c78106d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:14:18.861 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.859+0000 7f9c77fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c780f77d0 0x7f9c78107270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:14:18.861 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.859+0000 7f9c77fff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c78107870 con 0x7f9c780f6530 2021-11-03T18:14:18.862 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.859+0000 7f9c77fff700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c78108080 con 0x7f9c780f77d0 2021-11-03T18:14:18.862 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.859+0000 7f9c76ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c780f6530 0x7f9c78106d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:14:18.862 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.860+0000 7f9c76ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c780f6530 0x7f9c78106d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33734/0 (socket says 172.21.15.27:33734) 2021-11-03T18:14:18.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.860+0000 7f9c76ffd700 1 -- 172.21.15.27:0/1203206982 learned_addr learned my addr 172.21.15.27:0/1203206982 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:14:18.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.860+0000 7f9c767fc700 1 --2- 172.21.15.27:0/1203206982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c780f77d0 0x7f9c78107270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:14:18.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.860+0000 7f9c76ffd700 1 -- 172.21.15.27:0/1203206982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c780f77d0 msgr2=0x7f9c78107270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:14:18.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.860+0000 7f9c76ffd700 1 --2- 172.21.15.27:0/1203206982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c780f77d0 0x7f9c78107270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:18.864 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.860+0000 7f9c76ffd700 1 -- 172.21.15.27:0/1203206982 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c6c005040 con 0x7f9c780f6530 2021-11-03T18:14:18.864 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.860+0000 7f9c76ffd700 1 --2- 172.21.15.27:0/1203206982 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c780f6530 0x7f9c78106d80 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 rx=0x7f9c6800c530 tx=0x7f9c68007e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:14:18.864 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.860+0000 7f9c5f7fe700 1 -- 172.21.15.27:0/1203206982 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c6800f650 con 0x7f9c780f6530 2021-11-03T18:14:18.864 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.860+0000 7f9c5f7fe700 1 -- 172.21.15.27:0/1203206982 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f9c6800fc60 con 0x7f9c780f6530 2021-11-03T18:14:18.865 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.860+0000 7f9c77fff700 1 -- 172.21.15.27:0/1203206982 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c78108300 con 0x7f9c780f6530 2021-11-03T18:14:18.865 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.860+0000 7f9c5f7fe700 1 -- 172.21.15.27:0/1203206982 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c68012d20 con 0x7f9c780f6530 2021-11-03T18:14:18.865 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.860+0000 7f9c77fff700 1 -- 172.21.15.27:0/1203206982 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c781088c0 con 0x7f9c780f6530 2021-11-03T18:14:18.865 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.861+0000 7f9c77fff700 1 -- 172.21.15.27:0/1203206982 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c78059100 con 0x7f9c780f6530 2021-11-03T18:14:18.865 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.862+0000 7f9c5f7fe700 1 -- 172.21.15.27:0/1203206982 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f9c68019030 con 0x7f9c780f6530 2021-11-03T18:14:18.866 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.862+0000 7f9c5f7fe700 1 --2- 172.21.15.27:0/1203206982 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9c60064100 0x7f9c600665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:14:18.866 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.862+0000 7f9c5f7fe700 1 -- 172.21.15.27:0/1203206982 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f9c680805d0 con 0x7f9c780f6530 2021-11-03T18:14:18.866 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.862+0000 7f9c767fc700 1 --2- 172.21.15.27:0/1203206982 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9c60064100 0x7f9c600665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:14:18.866 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.863+0000 7f9c767fc700 1 --2- 172.21.15.27:0/1203206982 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9c60064100 0x7f9c600665c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7f9c6c004e30 tx=0x7f9c6c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:14:18.868 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:18.866+0000 7f9c5f7fe700 1 -- 172.21.15.27:0/1203206982 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f9c68052830 con 0x7f9c780f6530 2021-11-03T18:14:19.065 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:19.067+0000 7f9c77fff700 1 -- 172.21.15.27:0/1203206982 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9c780fedf0 con 0x7f9c780f6530 2021-11-03T18:14:19.066 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:19.068+0000 7f9c5f7fe700 1 -- 172.21.15.27:0/1203206982 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f9c68017710 con 0x7f9c780f6530 2021-11-03T18:14:19.066 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:14:19.066 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:14:19.067 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:14:19.067 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:14:19.067 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:14:19.067 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:14:19.067 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:14:19.068 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:14:19.068 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:14:19.068 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:14:19.068 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:14:19.069 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:14:19.069 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:14:19.069 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:14:19.069 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:14:19.069 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:14:19.070 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:14:19.070 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:14:19.070 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:14:19.070 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:14:19.071 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:14:19.071 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:14:19.071 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:14:19.071 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:14:19.072 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:14:19.072 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:14:19.072 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:14:19.072 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:14:19.073 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:14:19.073 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:14:19.073 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:14:19.073 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:14:19.074 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:14:19.074 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:14:19.074 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:14:19.074 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:14:19.074 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:14:19.075 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:14:19.075 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:14:19.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:19.070+0000 7f9c77fff700 1 -- 172.21.15.27:0/1203206982 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9c60064100 msgr2=0x7f9c600665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:14:19.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:19.070+0000 7f9c77fff700 1 --2- 172.21.15.27:0/1203206982 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9c60064100 0x7f9c600665c0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7f9c6c004e30 tx=0x7f9c6c009b30).stop 2021-11-03T18:14:19.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:19.070+0000 7f9c77fff700 1 -- 172.21.15.27:0/1203206982 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c780f6530 msgr2=0x7f9c78106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:14:19.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:19.070+0000 7f9c77fff700 1 --2- 172.21.15.27:0/1203206982 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c780f6530 0x7f9c78106d80 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 rx=0x7f9c6800c530 tx=0x7f9c68007e10).stop 2021-11-03T18:14:19.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:19.070+0000 7f9c77fff700 1 -- 172.21.15.27:0/1203206982 shutdown_connections 2021-11-03T18:14:19.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:19.070+0000 7f9c77fff700 1 --2- 172.21.15.27:0/1203206982 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9c60064100 0x7f9c600665c0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:19.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:19.070+0000 7f9c77fff700 1 --2- 172.21.15.27:0/1203206982 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c780f6530 0x7f9c78106d80 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:19.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:19.070+0000 7f9c77fff700 1 --2- 172.21.15.27:0/1203206982 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c780f77d0 0x7f9c78107270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:19.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:19.070+0000 7f9c77fff700 1 -- 172.21.15.27:0/1203206982 >> 172.21.15.27:0/1203206982 conn(0x7f9c780f1cd0 msgr2=0x7f9c780fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:14:19.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:19.070+0000 7f9c77fff700 1 -- 172.21.15.27:0/1203206982 shutdown_connections 2021-11-03T18:14:19.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:19.070+0000 7f9c77fff700 1 -- 172.21.15.27:0/1203206982 wait complete. 2021-11-03T18:14:19.078 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:14:19.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:19 smithi115 ceph-mon[65453]: from='client.35944 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:14:19.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:19 smithi115 ceph-mon[65453]: from='client.35948 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:14:19.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:19 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3693002717' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:19.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:14:19.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:14:19.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:14:19.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:19.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:19.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:19.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:19.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:19 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1203206982' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:14:19.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:19 smithi027 ceph-mon[97244]: from='client.35944 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:14:19.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:19 smithi027 ceph-mon[97244]: from='client.35948 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:14:19.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:19 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3693002717' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:19.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:14:19.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:14:19.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:14:19.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:19.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:19.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:19.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:19.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:19 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1203206982' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:14:20.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:20 smithi115 ceph-mon[65453]: pgmap v1790: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:20.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:20 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:14:20.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3502.443588 secs 2021-11-03T18:14:20.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:20 smithi027 ceph-mon[97244]: pgmap v1790: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:20.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:20 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:14:20.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3502.443588 secs 2021-11-03T18:14:22.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:22 smithi115 ceph-mon[65453]: pgmap v1791: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:22.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:22 smithi027 ceph-mon[97244]: pgmap v1791: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:24.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:24 smithi115 ceph-mon[65453]: pgmap v1792: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:24.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:24 smithi027 ceph-mon[97244]: pgmap v1792: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:25.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3507.443679 secs 2021-11-03T18:14:25.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3507.443679 secs 2021-11-03T18:14:26.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:26 smithi115 ceph-mon[65453]: pgmap v1793: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:26.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:26 smithi027 ceph-mon[97244]: pgmap v1793: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:28.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:28 smithi115 ceph-mon[65453]: pgmap v1794: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:28.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:28 smithi027 ceph-mon[97244]: pgmap v1794: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:29.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:14:29.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:14:29.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:14:29.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:29.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:29.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:29.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:29.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:14:29.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:14:29.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:14:29.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:29.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:29.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:29.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:30.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:30 smithi115 ceph-mon[65453]: pgmap v1795: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:30.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:30 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:14:30.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3512.443771 secs 2021-11-03T18:14:30.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:30 smithi027 ceph-mon[97244]: pgmap v1795: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:30.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:30 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:14:30.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3512.443771 secs 2021-11-03T18:14:32.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:32 smithi115 ceph-mon[65453]: pgmap v1796: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:32.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:32 smithi027 ceph-mon[97244]: pgmap v1796: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:34.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:34 smithi115 ceph-mon[65453]: pgmap v1797: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:34.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:34 smithi027 ceph-mon[97244]: pgmap v1797: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:35.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3517.443928 secs 2021-11-03T18:14:35.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3517.443928 secs 2021-11-03T18:14:36.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:36 smithi115 ceph-mon[65453]: pgmap v1798: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:36.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:36 smithi027 ceph-mon[97244]: pgmap v1798: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:38.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:38 smithi115 ceph-mon[65453]: pgmap v1799: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:38.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:38 smithi027 ceph-mon[97244]: pgmap v1799: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:39.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:14:39.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:14:39.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:14:39.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:39.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:39.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:39.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:39.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:14:39.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:14:39.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:14:39.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:39.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:39.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:39.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:40.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:40 smithi115 ceph-mon[65453]: pgmap v1800: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:40.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:40 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:14:40.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3522.444072 secs 2021-11-03T18:14:40.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:40 smithi027 ceph-mon[97244]: pgmap v1800: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:40.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:40 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:14:40.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3522.444072 secs 2021-11-03T18:14:42.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:42 smithi115 ceph-mon[65453]: pgmap v1801: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:42.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:42 smithi027 ceph-mon[97244]: pgmap v1801: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:44.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:44 smithi115 ceph-mon[65453]: pgmap v1802: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:44.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:44 smithi027 ceph-mon[97244]: pgmap v1802: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:45.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3527.444189 secs 2021-11-03T18:14:45.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3527.444189 secs 2021-11-03T18:14:46.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:46 smithi115 ceph-mon[65453]: pgmap v1803: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:46.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:46 smithi027 ceph-mon[97244]: pgmap v1803: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:48.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:48 smithi115 ceph-mon[65453]: pgmap v1804: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:48.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:48 smithi027 ceph-mon[97244]: pgmap v1804: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:49.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.185+0000 7f11b7883700 1 -- 172.21.15.27:0/3054533410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11b00f66e0 msgr2=0x7f11b00f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:14:49.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.185+0000 7f11b7883700 1 --2- 172.21.15.27:0/3054533410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11b00f66e0 0x7f11b00f6b00 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 rx=0x7f11a00099e0 tx=0x7f11a0004d10).stop 2021-11-03T18:14:49.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.185+0000 7f11b7883700 1 -- 172.21.15.27:0/3054533410 shutdown_connections 2021-11-03T18:14:49.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.185+0000 7f11b7883700 1 --2- 172.21.15.27:0/3054533410 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11b00f6ff0 0x7f11b00f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:49.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.185+0000 7f11b7883700 1 --2- 172.21.15.27:0/3054533410 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11b00f66e0 0x7f11b00f6b00 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:49.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.185+0000 7f11b7883700 1 -- 172.21.15.27:0/3054533410 >> 172.21.15.27:0/3054533410 conn(0x7f11b00f1f00 msgr2=0x7f11b00f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:14:49.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.186+0000 7f11b7883700 1 -- 172.21.15.27:0/3054533410 shutdown_connections 2021-11-03T18:14:49.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.186+0000 7f11b7883700 1 -- 172.21.15.27:0/3054533410 wait complete. 2021-11-03T18:14:49.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.186+0000 7f11b7883700 1 Processor -- start 2021-11-03T18:14:49.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.186+0000 7f11b7883700 1 -- start start 2021-11-03T18:14:49.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.186+0000 7f11b7883700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11b00f66e0 0x7f11b0107070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:14:49.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.186+0000 7f11b7883700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11b00f6ff0 0x7f11b0107560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:14:49.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.186+0000 7f11b7883700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b0107b10 con 0x7f11b00f66e0 2021-11-03T18:14:49.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.186+0000 7f11b7883700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b0107c50 con 0x7f11b00f6ff0 2021-11-03T18:14:49.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.186+0000 7f11b561f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11b00f66e0 0x7f11b0107070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:14:49.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.187+0000 7f11b561f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11b00f66e0 0x7f11b0107070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33752/0 (socket says 172.21.15.27:33752) 2021-11-03T18:14:49.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.187+0000 7f11b561f700 1 -- 172.21.15.27:0/181384418 learned_addr learned my addr 172.21.15.27:0/181384418 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:14:49.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.187+0000 7f11b4e1e700 1 --2- 172.21.15.27:0/181384418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11b00f6ff0 0x7f11b0107560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:14:49.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.187+0000 7f11b561f700 1 -- 172.21.15.27:0/181384418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11b00f6ff0 msgr2=0x7f11b0107560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:14:49.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.187+0000 7f11b561f700 1 --2- 172.21.15.27:0/181384418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11b00f6ff0 0x7f11b0107560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:49.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.187+0000 7f11b561f700 1 -- 172.21.15.27:0/181384418 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a0009640 con 0x7f11b00f66e0 2021-11-03T18:14:49.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.187+0000 7f11b561f700 1 --2- 172.21.15.27:0/181384418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11b00f66e0 0x7f11b0107070 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 rx=0x7f11a000b3b0 tx=0x7f11a0004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:14:49.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.187+0000 7f11adffb700 1 -- 172.21.15.27:0/181384418 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11a000e030 con 0x7f11b00f66e0 2021-11-03T18:14:49.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.187+0000 7f11adffb700 1 -- 172.21.15.27:0/181384418 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f11a00049b0 con 0x7f11b00f66e0 2021-11-03T18:14:49.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.187+0000 7f11adffb700 1 -- 172.21.15.27:0/181384418 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f11a0018070 con 0x7f11b00f66e0 2021-11-03T18:14:49.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.187+0000 7f11b7883700 1 -- 172.21.15.27:0/181384418 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11b0108590 con 0x7f11b00f66e0 2021-11-03T18:14:49.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.188+0000 7f11b7883700 1 -- 172.21.15.27:0/181384418 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11b0108a20 con 0x7f11b00f66e0 2021-11-03T18:14:49.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.189+0000 7f11b7883700 1 -- 172.21.15.27:0/181384418 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11b0045ad0 con 0x7f11b00f66e0 2021-11-03T18:14:49.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.189+0000 7f11adffb700 1 -- 172.21.15.27:0/181384418 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f11a0007c90 con 0x7f11b00f66e0 2021-11-03T18:14:49.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.189+0000 7f11adffb700 1 --2- 172.21.15.27:0/181384418 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f11980640b0 0x7f1198066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:14:49.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.190+0000 7f11adffb700 1 -- 172.21.15.27:0/181384418 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f11a00824e0 con 0x7f11b00f66e0 2021-11-03T18:14:49.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.190+0000 7f11b4e1e700 1 --2- 172.21.15.27:0/181384418 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f11980640b0 0x7f1198066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:14:49.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.190+0000 7f11b4e1e700 1 --2- 172.21.15.27:0/181384418 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f11980640b0 0x7f1198066570 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f11a4004c40 tx=0x7f11a400b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:14:49.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.194+0000 7f11adffb700 1 -- 172.21.15.27:0/181384418 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f11a00547e0 con 0x7f11b00f66e0 2021-11-03T18:14:49.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.373+0000 7f11b7883700 1 -- 172.21.15.27:0/181384418 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f11b00fefb0 con 0x7f11980640b0 2021-11-03T18:14:49.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.375+0000 7f11adffb700 1 -- 172.21.15.27:0/181384418 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f11b00fefb0 con 0x7f11980640b0 2021-11-03T18:14:49.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.377+0000 7f11b7883700 1 -- 172.21.15.27:0/181384418 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f11980640b0 msgr2=0x7f1198066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:14:49.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.377+0000 7f11b7883700 1 --2- 172.21.15.27:0/181384418 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f11980640b0 0x7f1198066570 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7f11a4004c40 tx=0x7f11a400b040).stop 2021-11-03T18:14:49.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.377+0000 7f11b7883700 1 -- 172.21.15.27:0/181384418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11b00f66e0 msgr2=0x7f11b0107070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:14:49.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.377+0000 7f11b7883700 1 --2- 172.21.15.27:0/181384418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11b00f66e0 0x7f11b0107070 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 rx=0x7f11a000b3b0 tx=0x7f11a0004d10).stop 2021-11-03T18:14:49.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.377+0000 7f11b7883700 1 -- 172.21.15.27:0/181384418 shutdown_connections 2021-11-03T18:14:49.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.377+0000 7f11b7883700 1 --2- 172.21.15.27:0/181384418 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f11980640b0 0x7f1198066570 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:49.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.377+0000 7f11b7883700 1 --2- 172.21.15.27:0/181384418 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f11b00f66e0 0x7f11b0107070 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:49.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.377+0000 7f11b7883700 1 --2- 172.21.15.27:0/181384418 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f11b00f6ff0 0x7f11b0107560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:49.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.377+0000 7f11b7883700 1 -- 172.21.15.27:0/181384418 >> 172.21.15.27:0/181384418 conn(0x7f11b00f1f00 msgr2=0x7f11b00fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:14:49.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.377+0000 7f11b7883700 1 -- 172.21.15.27:0/181384418 shutdown_connections 2021-11-03T18:14:49.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.377+0000 7f11b7883700 1 -- 172.21.15.27:0/181384418 wait complete. 2021-11-03T18:14:49.386 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:14:49.481 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.480+0000 7f53d1be4700 1 -- 172.21.15.27:0/4274679299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53cc0f6ff0 msgr2=0x7f53cc0f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:14:49.481 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.480+0000 7f53d1be4700 1 --2- 172.21.15.27:0/4274679299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53cc0f6ff0 0x7f53cc0f7a20 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 rx=0x7f53c0004660 tx=0x7f53c0009b30).stop 2021-11-03T18:14:49.481 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.480+0000 7f53d1be4700 1 -- 172.21.15.27:0/4274679299 shutdown_connections 2021-11-03T18:14:49.481 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.480+0000 7f53d1be4700 1 --2- 172.21.15.27:0/4274679299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53cc0f6ff0 0x7f53cc0f7a20 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.480+0000 7f53d1be4700 1 --2- 172.21.15.27:0/4274679299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53cc0f66e0 0x7f53cc0f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T18:14:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.480+0000 7f53d1be4700 1 -- 172.21.15.27:0/4274679299 >> 172.21.15.27:0/4274679299 conn(0x7f53cc0f1f00 msgr2=0x7f53cc0f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:14:49.482 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.480+0000 7f53d1be4700 1 -- 172.21.15.27:0/4274679299 shutdown_connections 2021-11-03T18:14:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.480+0000 7f53d1be4700 1 -- 172.21.15.27:0/4274679299 wait complete. 2021-11-03T18:14:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.480+0000 7f53d1be4700 1 Processor -- start 2021-11-03T18:14:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.480+0000 7f53d1be4700 1 -- start start 2021-11-03T18:14:49.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.481+0000 7f53d1be4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53cc0f66e0 0x7f53cc106fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:14:49.484 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.481+0000 7f53d1be4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53cc0f6ff0 0x7f53cc107490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:14:49.484 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.481+0000 7f53d1be4700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53cc107ad0 con 0x7f53cc0f66e0 2021-11-03T18:14:49.484 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.481+0000 7f53d1be4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53cc108380 con 0x7f53cc0f6ff0 2021-11-03T18:14:49.485 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.481+0000 7f53cb7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53cc0f66e0 0x7f53cc106fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:14:49.485 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.481+0000 7f53cb7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53cc0f66e0 0x7f53cc106fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33762/0 (socket says 172.21.15.27:33762) 2021-11-03T18:14:49.485 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.481+0000 7f53cb7fe700 1 -- 172.21.15.27:0/3628549938 learned_addr learned my addr 172.21.15.27:0/3628549938 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:14:49.486 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.481+0000 7f53caffd700 1 --2- 172.21.15.27:0/3628549938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53cc0f6ff0 0x7f53cc107490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:14:49.486 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.481+0000 7f53cb7fe700 1 -- 172.21.15.27:0/3628549938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53cc0f6ff0 msgr2=0x7f53cc107490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:14:49.486 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.481+0000 7f53cb7fe700 1 --2- 172.21.15.27:0/3628549938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53cc0f6ff0 0x7f53cc107490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:49.486 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.481+0000 7f53cb7fe700 1 -- 172.21.15.27:0/3628549938 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53c0005040 con 0x7f53cc0f66e0 2021-11-03T18:14:49.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.481+0000 7f53cb7fe700 1 --2- 172.21.15.27:0/3628549938 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53cc0f66e0 0x7f53cc106fa0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 rx=0x7f53bc00b410 tx=0x7f53bc006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:14:49.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.481+0000 7f53b3fff700 1 -- 172.21.15.27:0/3628549938 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53bc0125f0 con 0x7f53cc0f66e0 2021-11-03T18:14:49.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.481+0000 7f53b3fff700 1 -- 172.21.15.27:0/3628549938 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f53bc00be30 con 0x7f53cc0f66e0 2021-11-03T18:14:49.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.481+0000 7f53d1be4700 1 -- 172.21.15.27:0/3628549938 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53cc108660 con 0x7f53cc0f66e0 2021-11-03T18:14:49.488 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.481+0000 7f53b3fff700 1 -- 172.21.15.27:0/3628549938 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53bc002a70 con 0x7f53cc0f66e0 2021-11-03T18:14:49.488 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.482+0000 7f53d1be4700 1 -- 172.21.15.27:0/3628549938 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53cc108c20 con 0x7f53cc0f66e0 2021-11-03T18:14:49.488 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.482+0000 7f53d1be4700 1 -- 172.21.15.27:0/3628549938 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53cc0f5b70 con 0x7f53cc0f66e0 2021-11-03T18:14:49.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.483+0000 7f53b3fff700 1 -- 172.21.15.27:0/3628549938 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f53bc00f030 con 0x7f53cc0f66e0 2021-11-03T18:14:49.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.483+0000 7f53b3fff700 1 --2- 172.21.15.27:0/3628549938 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f53b40640b0 0x7f53b4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:14:49.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.483+0000 7f53b3fff700 1 -- 172.21.15.27:0/3628549938 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f53bc080c50 con 0x7f53cc0f66e0 2021-11-03T18:14:49.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.483+0000 7f53caffd700 1 --2- 172.21.15.27:0/3628549938 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f53b40640b0 0x7f53b4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:14:49.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.484+0000 7f53caffd700 1 --2- 172.21.15.27:0/3628549938 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f53b40640b0 0x7f53b4066570 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f53c0004630 tx=0x7f53c0009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:14:49.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.487+0000 7f53b3fff700 1 -- 172.21.15.27:0/3628549938 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f53bc052f50 con 0x7f53cc0f66e0 2021-11-03T18:14:49.659 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.660+0000 7f53d1be4700 1 -- 172.21.15.27:0/3628549938 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f53cc0fef50 con 0x7f53b40640b0 2021-11-03T18:14:49.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.675+0000 7f53b3fff700 1 -- 172.21.15.27:0/3628549938 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f53cc0fef50 con 0x7f53b40640b0 2021-11-03T18:14:49.674 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:14:49.674 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (66m) 5m ago 67m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:14:49.674 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (58m) 5m ago 67m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:14:49.675 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (57m) 4m ago 66m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:14:49.675 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (66m) 5m ago 67m 42.1M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:14:49.675 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (59m) 5m ago 62m 26.3M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:14:49.675 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (62m) 5m ago 62m 3847M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:14:49.675 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (58m) 4m ago 62m 25.9M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:14:49.676 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (60m) 4m ago 62m 27.0M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:14:49.676 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (60m) 5m ago 68m 572M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:14:49.676 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (59m) 4m ago 66m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:14:49.676 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (59m) 5m ago 69m 273M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:14:49.677 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (58m) 4m ago 66m 129M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:14:49.677 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (67m) 5m ago 67m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:14:49.677 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (66m) 4m ago 66m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:14:49.677 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (57m) 5m ago 66m 371M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:14:49.677 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (57m) 5m ago 65m 405M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:14:49.678 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (56m) 5m ago 65m 302M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:14:49.678 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (56m) 4m ago 65m 429M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:14:49.678 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (55m) 4m ago 64m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:14:49.678 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (55m) 4m ago 64m 348M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:14:49.678 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (66m) 5m ago 67m 57.6M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:14:49.679 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.677+0000 7f53d1be4700 1 -- 172.21.15.27:0/3628549938 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f53b40640b0 msgr2=0x7f53b4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:14:49.680 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.677+0000 7f53d1be4700 1 --2- 172.21.15.27:0/3628549938 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f53b40640b0 0x7f53b4066570 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7f53c0004630 tx=0x7f53c0009b30).stop 2021-11-03T18:14:49.680 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.677+0000 7f53d1be4700 1 -- 172.21.15.27:0/3628549938 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53cc0f66e0 msgr2=0x7f53cc106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:14:49.680 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.677+0000 7f53d1be4700 1 --2- 172.21.15.27:0/3628549938 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53cc0f66e0 0x7f53cc106fa0 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 rx=0x7f53bc00b410 tx=0x7f53bc006ef0).stop 2021-11-03T18:14:49.681 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.677+0000 7f53d1be4700 1 -- 172.21.15.27:0/3628549938 shutdown_connections 2021-11-03T18:14:49.681 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.677+0000 7f53d1be4700 1 --2- 172.21.15.27:0/3628549938 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f53b40640b0 0x7f53b4066570 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:49.681 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.677+0000 7f53d1be4700 1 --2- 172.21.15.27:0/3628549938 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53cc0f66e0 0x7f53cc106fa0 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:49.681 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.677+0000 7f53d1be4700 1 --2- 172.21.15.27:0/3628549938 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53cc0f6ff0 0x7f53cc107490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:49.682 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.677+0000 7f53d1be4700 1 -- 172.21.15.27:0/3628549938 >> 172.21.15.27:0/3628549938 conn(0x7f53cc0f1f00 msgr2=0x7f53cc0fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:14:49.682 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.677+0000 7f53d1be4700 1 -- 172.21.15.27:0/3628549938 shutdown_connections 2021-11-03T18:14:49.682 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.677+0000 7f53d1be4700 1 -- 172.21.15.27:0/3628549938 wait complete. 2021-11-03T18:14:49.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:14:49.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:14:49.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:14:49.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:49.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:49.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:49.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:49.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:14:49.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:14:49.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:14:49.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:49.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:49.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:49.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:49.774 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.776+0000 7f57e11b9700 1 -- 172.21.15.27:0/2641023482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57dc0f55c0 msgr2=0x7f57dc0f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:14:49.774 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.776+0000 7f57e11b9700 1 --2- 172.21.15.27:0/2641023482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57dc0f55c0 0x7f57dc0f59e0 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 rx=0x7f57d40099e0 tx=0x7f57d4004d10).stop 2021-11-03T18:14:49.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.776+0000 7f57e11b9700 1 -- 172.21.15.27:0/2641023482 shutdown_connections 2021-11-03T18:14:49.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.776+0000 7f57e11b9700 1 --2- 172.21.15.27:0/2641023482 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57dc0f6860 0x7f57dc0f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T18:14:49.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.776+0000 7f57e11b9700 1 --2- 172.21.15.27:0/2641023482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57dc0f55c0 0x7f57dc0f59e0 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:49.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.776+0000 7f57e11b9700 1 -- 172.21.15.27:0/2641023482 >> 172.21.15.27:0/2641023482 conn(0x7f57dc0f0d40 msgr2=0x7f57dc0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:14:49.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.776+0000 7f57e11b9700 1 -- 172.21.15.27:0/2641023482 shutdown_connections 2021-11-03T18:14:49.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.776+0000 7f57e11b9700 1 -- 172.21.15.27:0/2641023482 wait complete. 2021-11-03T18:14:49.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.776+0000 7f57e11b9700 1 Processor -- start 2021-11-03T18:14:49.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.776+0000 7f57e11b9700 1 -- start start 2021-11-03T18:14:49.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.776+0000 7f57e11b9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57dc0f55c0 0x7f57dc107100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:14:49.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.776+0000 7f57e11b9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57dc0f6860 0x7f57dc1075f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:14:49.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.776+0000 7f57e11b9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc107c30 con 0x7f57dc0f55c0 2021-11-03T18:14:49.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.776+0000 7f57e11b9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57dc1084e0 con 0x7f57dc0f6860 2021-11-03T18:14:49.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.777+0000 7f57dad9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57dc0f55c0 0x7f57dc107100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:14:49.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.777+0000 7f57dad9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57dc0f55c0 0x7f57dc107100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33772/0 (socket says 172.21.15.27:33772) 2021-11-03T18:14:49.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.777+0000 7f57dad9d700 1 -- 172.21.15.27:0/1663102758 learned_addr learned my addr 172.21.15.27:0/1663102758 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:14:49.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.777+0000 7f57da59c700 1 --2- 172.21.15.27:0/1663102758 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57dc0f6860 0x7f57dc1075f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:14:49.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.777+0000 7f57dad9d700 1 -- 172.21.15.27:0/1663102758 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57dc0f6860 msgr2=0x7f57dc1075f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:14:49.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.777+0000 7f57dad9d700 1 --2- 172.21.15.27:0/1663102758 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57dc0f6860 0x7f57dc1075f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:49.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.777+0000 7f57dad9d700 1 -- 172.21.15.27:0/1663102758 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d4009640 con 0x7f57dc0f55c0 2021-11-03T18:14:49.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.777+0000 7f57dad9d700 1 --2- 172.21.15.27:0/1663102758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57dc0f55c0 0x7f57dc107100 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 rx=0x7f57d400c010 tx=0x7f57d4004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:14:49.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.777+0000 7f57d37fe700 1 -- 172.21.15.27:0/1663102758 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57d400e070 con 0x7f57dc0f55c0 2021-11-03T18:14:49.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.777+0000 7f57d37fe700 1 -- 172.21.15.27:0/1663102758 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f57d4007dc0 con 0x7f57dc0f55c0 2021-11-03T18:14:49.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.777+0000 7f57d37fe700 1 -- 172.21.15.27:0/1663102758 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57d4018070 con 0x7f57dc0f55c0 2021-11-03T18:14:49.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.777+0000 7f57e11b9700 1 -- 172.21.15.27:0/1663102758 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57dc108760 con 0x7f57dc0f55c0 2021-11-03T18:14:49.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.777+0000 7f57e11b9700 1 -- 172.21.15.27:0/1663102758 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57dc108c40 con 0x7f57dc0f55c0 2021-11-03T18:14:49.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.778+0000 7f57e11b9700 1 -- 172.21.15.27:0/1663102758 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57dc0f49b0 con 0x7f57dc0f55c0 2021-11-03T18:14:49.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.779+0000 7f57d37fe700 1 -- 172.21.15.27:0/1663102758 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f57d4024060 con 0x7f57dc0f55c0 2021-11-03T18:14:49.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.779+0000 7f57d37fe700 1 --2- 172.21.15.27:0/1663102758 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f57c80640b0 0x7f57c8066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:14:49.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.779+0000 7f57d37fe700 1 -- 172.21.15.27:0/1663102758 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f57d4082230 con 0x7f57dc0f55c0 2021-11-03T18:14:49.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.779+0000 7f57da59c700 1 --2- 172.21.15.27:0/1663102758 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f57c80640b0 0x7f57c8066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:14:49.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.780+0000 7f57da59c700 1 --2- 172.21.15.27:0/1663102758 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f57c80640b0 0x7f57c8066570 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7f57c400b790 tx=0x7f57c4008040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:14:49.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:49.783+0000 7f57d37fe700 1 -- 172.21.15.27:0/1663102758 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f57d4054530 con 0x7f57dc0f55c0 2021-11-03T18:14:50.014 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.014+0000 7f57e11b9700 1 -- 172.21.15.27:0/1663102758 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f57dc0fdf40 con 0x7f57dc0f55c0 2021-11-03T18:14:50.015 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.015+0000 7f57d37fe700 1 -- 172.21.15.27:0/1663102758 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f57d40222e0 con 0x7f57dc0f55c0 2021-11-03T18:14:50.015 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:14:50.016 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:14:50.016 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:14:50.017 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:14:50.017 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:14:50.017 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:14:50.018 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:14:50.018 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:14:50.019 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:14:50.019 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:14:50.020 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:14:50.020 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:14:50.021 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:14:50.021 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:14:50.021 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:14:50.022 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:14:50.022 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:14:50.023 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:14:50.024 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.017+0000 7f57e11b9700 1 -- 172.21.15.27:0/1663102758 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f57c80640b0 msgr2=0x7f57c8066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:14:50.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.017+0000 7f57e11b9700 1 --2- 172.21.15.27:0/1663102758 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f57c80640b0 0x7f57c8066570 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7f57c400b790 tx=0x7f57c4008040).stop 2021-11-03T18:14:50.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.017+0000 7f57e11b9700 1 -- 172.21.15.27:0/1663102758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57dc0f55c0 msgr2=0x7f57dc107100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:14:50.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.017+0000 7f57e11b9700 1 --2- 172.21.15.27:0/1663102758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57dc0f55c0 0x7f57dc107100 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 rx=0x7f57d400c010 tx=0x7f57d4004d10).stop 2021-11-03T18:14:50.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.017+0000 7f57e11b9700 1 -- 172.21.15.27:0/1663102758 shutdown_connections 2021-11-03T18:14:50.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.017+0000 7f57e11b9700 1 --2- 172.21.15.27:0/1663102758 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f57c80640b0 0x7f57c8066570 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:50.027 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.017+0000 7f57e11b9700 1 --2- 172.21.15.27:0/1663102758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57dc0f55c0 0x7f57dc107100 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:50.027 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.017+0000 7f57e11b9700 1 --2- 172.21.15.27:0/1663102758 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57dc0f6860 0x7f57dc1075f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:50.028 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.017+0000 7f57e11b9700 1 -- 172.21.15.27:0/1663102758 >> 172.21.15.27:0/1663102758 conn(0x7f57dc0f0d40 msgr2=0x7f57dc05fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:14:50.028 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.017+0000 7f57e11b9700 1 -- 172.21.15.27:0/1663102758 shutdown_connections 2021-11-03T18:14:50.029 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.017+0000 7f57e11b9700 1 -- 172.21.15.27:0/1663102758 wait complete. 2021-11-03T18:14:50.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.118+0000 7f946fb0e700 1 -- 172.21.15.27:0/1020476460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94680f6ff0 msgr2=0x7f94680f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:14:50.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.118+0000 7f946fb0e700 1 --2- 172.21.15.27:0/1020476460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94680f6ff0 0x7f94680f7a20 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 rx=0x7f945c004660 tx=0x7f945c009b30).stop 2021-11-03T18:14:50.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.118+0000 7f946fb0e700 1 -- 172.21.15.27:0/1020476460 shutdown_connections 2021-11-03T18:14:50.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.118+0000 7f946fb0e700 1 --2- 172.21.15.27:0/1020476460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94680f6ff0 0x7f94680f7a20 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:50.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.118+0000 7f946fb0e700 1 --2- 172.21.15.27:0/1020476460 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94680f66e0 0x7f94680f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:50.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.118+0000 7f946fb0e700 1 -- 172.21.15.27:0/1020476460 >> 172.21.15.27:0/1020476460 conn(0x7f94680f1f00 msgr2=0x7f94680f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:14:50.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.118+0000 7f946fb0e700 1 -- 172.21.15.27:0/1020476460 shutdown_connections 2021-11-03T18:14:50.121 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.118+0000 7f946fb0e700 1 -- 172.21.15.27:0/1020476460 wait complete. 2021-11-03T18:14:50.121 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.119+0000 7f946fb0e700 1 Processor -- start 2021-11-03T18:14:50.121 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.119+0000 7f946fb0e700 1 -- start start 2021-11-03T18:14:50.121 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.119+0000 7f946fb0e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94680f66e0 0x7f9468106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:14:50.122 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.119+0000 7f946fb0e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94680f6ff0 0x7f9468107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:14:50.122 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.119+0000 7f946fb0e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9468107a10 con 0x7f94680f66e0 2021-11-03T18:14:50.122 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.119+0000 7f946fb0e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9468107b50 con 0x7f94680f6ff0 2021-11-03T18:14:50.122 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.119+0000 7f946d8aa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94680f66e0 0x7f9468106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:14:50.122 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.119+0000 7f946d8aa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94680f66e0 0x7f9468106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33782/0 (socket says 172.21.15.27:33782) 2021-11-03T18:14:50.123 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.119+0000 7f946d8aa700 1 -- 172.21.15.27:0/3425540536 learned_addr learned my addr 172.21.15.27:0/3425540536 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:14:50.123 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.119+0000 7f946d0a9700 1 --2- 172.21.15.27:0/3425540536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94680f6ff0 0x7f9468107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:14:50.123 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.119+0000 7f946d8aa700 1 -- 172.21.15.27:0/3425540536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94680f6ff0 msgr2=0x7f9468107460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:14:50.123 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.119+0000 7f946d8aa700 1 --2- 172.21.15.27:0/3425540536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94680f6ff0 0x7f9468107460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:50.124 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.119+0000 7f946d8aa700 1 -- 172.21.15.27:0/3425540536 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f945c005040 con 0x7f94680f66e0 2021-11-03T18:14:50.124 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.119+0000 7f946d0a9700 1 --2- 172.21.15.27:0/3425540536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94680f6ff0 0x7f9468107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:14:50.124 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.120+0000 7f946d8aa700 1 --2- 172.21.15.27:0/3425540536 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94680f66e0 0x7f9468106f70 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 rx=0x7f9460007fd0 tx=0x7f9460004ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:14:50.124 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.120+0000 7f94667fc700 1 -- 172.21.15.27:0/3425540536 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f946000b5a0 con 0x7f94680f66e0 2021-11-03T18:14:50.124 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.120+0000 7f94667fc700 1 -- 172.21.15.27:0/3425540536 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f946000f040 con 0x7f94680f66e0 2021-11-03T18:14:50.125 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.120+0000 7f946fb0e700 1 -- 172.21.15.27:0/3425540536 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94681084f0 con 0x7f94680f66e0 2021-11-03T18:14:50.125 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.120+0000 7f94667fc700 1 -- 172.21.15.27:0/3425540536 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9460013b60 con 0x7f94680f66e0 2021-11-03T18:14:50.125 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.120+0000 7f946fb0e700 1 -- 172.21.15.27:0/3425540536 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9468108ab0 con 0x7f94680f66e0 2021-11-03T18:14:50.126 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.121+0000 7f946fb0e700 1 -- 172.21.15.27:0/3425540536 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9468045ad0 con 0x7f94680f66e0 2021-11-03T18:14:50.127 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.121+0000 7f94667fc700 1 -- 172.21.15.27:0/3425540536 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f946001a030 con 0x7f94680f66e0 2021-11-03T18:14:50.127 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.122+0000 7f94667fc700 1 --2- 172.21.15.27:0/3425540536 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f94500640b0 0x7f9450066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:14:50.127 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.122+0000 7f94667fc700 1 -- 172.21.15.27:0/3425540536 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f94600806e0 con 0x7f94680f66e0 2021-11-03T18:14:50.127 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.122+0000 7f946d0a9700 1 --2- 172.21.15.27:0/3425540536 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f94500640b0 0x7f9450066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:14:50.128 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.122+0000 7f946d0a9700 1 --2- 172.21.15.27:0/3425540536 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f94500640b0 0x7f9450066570 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7f945c004630 tx=0x7f945c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:14:50.128 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.126+0000 7f94667fc700 1 -- 172.21.15.27:0/3425540536 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f94600529a0 con 0x7f94680f66e0 2021-11-03T18:14:50.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.330+0000 7f946fb0e700 1 -- 172.21.15.27:0/3425540536 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f94680fefe0 con 0x7f94680f66e0 2021-11-03T18:14:50.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.330+0000 7f94667fc700 1 -- 172.21.15.27:0/3425540536 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f9460018020 con 0x7f94680f66e0 2021-11-03T18:14:50.330 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:14:50.330 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:14:50.331 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:14:50.331 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:14:50.331 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:14:50.331 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:14:50.332 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:14:50.332 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:14:50.332 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:14:50.332 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:14:50.333 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:14:50.333 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:14:50.333 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:14:50.333 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:14:50.333 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:14:50.334 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:14:50.334 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:14:50.334 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:14:50.334 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:14:50.335 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:14:50.335 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:14:50.335 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:14:50.335 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:14:50.335 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:14:50.336 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:14:50.336 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:14:50.336 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:14:50.336 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:14:50.337 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:14:50.337 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:14:50.337 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:14:50.337 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:14:50.337 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:14:50.338 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:14:50.338 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:14:50.338 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:14:50.338 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:14:50.338 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:14:50.339 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:14:50.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.332+0000 7f946fb0e700 1 -- 172.21.15.27:0/3425540536 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f94500640b0 msgr2=0x7f9450066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:14:50.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.332+0000 7f946fb0e700 1 --2- 172.21.15.27:0/3425540536 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f94500640b0 0x7f9450066570 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7f945c004630 tx=0x7f945c009b30).stop 2021-11-03T18:14:50.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.332+0000 7f946fb0e700 1 -- 172.21.15.27:0/3425540536 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94680f66e0 msgr2=0x7f9468106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:14:50.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.332+0000 7f946fb0e700 1 --2- 172.21.15.27:0/3425540536 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94680f66e0 0x7f9468106f70 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 rx=0x7f9460007fd0 tx=0x7f9460004ee0).stop 2021-11-03T18:14:50.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.332+0000 7f946fb0e700 1 -- 172.21.15.27:0/3425540536 shutdown_connections 2021-11-03T18:14:50.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.332+0000 7f946fb0e700 1 --2- 172.21.15.27:0/3425540536 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f94500640b0 0x7f9450066570 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:50.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.332+0000 7f946fb0e700 1 --2- 172.21.15.27:0/3425540536 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f94680f66e0 0x7f9468106f70 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:50.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.332+0000 7f946fb0e700 1 --2- 172.21.15.27:0/3425540536 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f94680f6ff0 0x7f9468107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:14:50.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.332+0000 7f946fb0e700 1 -- 172.21.15.27:0/3425540536 >> 172.21.15.27:0/3425540536 conn(0x7f94680f1f00 msgr2=0x7f94680fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:14:50.342 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.332+0000 7f946fb0e700 1 -- 172.21.15.27:0/3425540536 shutdown_connections 2021-11-03T18:14:50.342 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:14:50.332+0000 7f946fb0e700 1 -- 172.21.15.27:0/3425540536 wait complete. 2021-11-03T18:14:50.342 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:14:50.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:50 smithi115 ceph-mon[65453]: pgmap v1805: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:50.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:50 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:14:50.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:50 smithi115 ceph-mon[65453]: from='client.35960 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:14:50.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3532.444275 secs 2021-11-03T18:14:50.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:50 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1663102758' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:50.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:50 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3425540536' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:14:50.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:50 smithi027 ceph-mon[97244]: pgmap v1805: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:50.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:50 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:14:50.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:50 smithi027 ceph-mon[97244]: from='client.35960 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:14:50.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3532.444275 secs 2021-11-03T18:14:50.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:50 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1663102758' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:50.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:50 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3425540536' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:14:51.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:51 smithi115 ceph-mon[65453]: from='client.35964 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:14:51.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:51 smithi027 ceph-mon[97244]: from='client.35964 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:14:52.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:52 smithi115 ceph-mon[65453]: pgmap v1806: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:52.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:52 smithi027 ceph-mon[97244]: pgmap v1806: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:54.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:54 smithi115 ceph-mon[65453]: pgmap v1807: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:54.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:54 smithi027 ceph-mon[97244]: pgmap v1807: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:55.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3537.444379 secs 2021-11-03T18:14:55.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3537.444379 secs 2021-11-03T18:14:56.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:56 smithi115 ceph-mon[65453]: pgmap v1808: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:56.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:14:56.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:14:56.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:56 smithi027 ceph-mon[97244]: pgmap v1808: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:56.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:14:56.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:14:58.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:58 smithi115 ceph-mon[65453]: pgmap v1809: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:58.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:58 smithi027 ceph-mon[97244]: pgmap v1809: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:14:59.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:14:59.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:14:59.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:14:59.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:59.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:59.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:59.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:14:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:59.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:14:59.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:14:59.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:14:59.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:59.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:59.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:14:59.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:14:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:00.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:00 smithi115 ceph-mon[65453]: pgmap v1810: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:00.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:00 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:15:00.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3542.444524 secs 2021-11-03T18:15:00.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:00 smithi027 ceph-mon[97244]: pgmap v1810: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:00.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:00 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:15:00.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3542.444524 secs 2021-11-03T18:15:02.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:02 smithi115 ceph-mon[65453]: pgmap v1811: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:02.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:02 smithi027 ceph-mon[97244]: pgmap v1811: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:04.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:04 smithi115 ceph-mon[65453]: pgmap v1812: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:04.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:04 smithi027 ceph-mon[97244]: pgmap v1812: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:05.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3547.444668 secs 2021-11-03T18:15:05.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3547.444668 secs 2021-11-03T18:15:06.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:06 smithi115 ceph-mon[65453]: pgmap v1813: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:06.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:06 smithi027 ceph-mon[97244]: pgmap v1813: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:08.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:08 smithi115 ceph-mon[65453]: pgmap v1814: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:08.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:08 smithi027 ceph-mon[97244]: pgmap v1814: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:09.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:15:09.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:15:09.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:15:09.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:09.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:09.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:09.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:09.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:15:09.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:15:09.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:15:09.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:09.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:09.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:09.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:10.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:10 smithi027 ceph-mon[97244]: pgmap v1815: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:10.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:10 smithi027 ceph-mon[97244]: poll took 60175.113 ms: 1 events 2021-11-03T18:15:10.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:10 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:15:10.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3552.444800 secs 2021-11-03T18:15:10.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:10 smithi115 ceph-mon[65453]: pgmap v1815: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:10.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:10 smithi115 ceph-mon[65453]: poll took 60175.113 ms: 1 events 2021-11-03T18:15:10.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:10 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:15:10.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3552.444800 secs 2021-11-03T18:15:12.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:12 smithi027 ceph-mon[97244]: pgmap v1816: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:12.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:12 smithi115 ceph-mon[65453]: pgmap v1816: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:14.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:14 smithi027 ceph-mon[97244]: pgmap v1817: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:14.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:14 smithi115 ceph-mon[65453]: pgmap v1817: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:15.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3557.444922 secs 2021-11-03T18:15:15.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3557.444922 secs 2021-11-03T18:15:16.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:16 smithi027 ceph-mon[97244]: pgmap v1818: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:16.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:16 smithi115 ceph-mon[65453]: pgmap v1818: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:18.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:18 smithi027 ceph-mon[97244]: pgmap v1819: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:18.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:18 smithi115 ceph-mon[65453]: pgmap v1819: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:19.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:15:19.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:15:19.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:15:19.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:19.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:19.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:19.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:19.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:15:19.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:15:19.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:15:19.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:19.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:19.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:19.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:20.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.446+0000 7f7409f93700 1 -- 172.21.15.27:0/437704639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74040603f0 msgr2=0x7f74040f9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:15:20.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.446+0000 7f7409f93700 1 --2- 172.21.15.27:0/437704639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74040603f0 0x7f74040f9340 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 rx=0x7f73f4004660 tx=0x7f73f4009b30).stop 2021-11-03T18:15:20.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.446+0000 7f7409f93700 1 -- 172.21.15.27:0/437704639 shutdown_connections 2021-11-03T18:15:20.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.446+0000 7f7409f93700 1 --2- 172.21.15.27:0/437704639 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74040f9830 0x7f74040fbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:20.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.446+0000 7f7409f93700 1 --2- 172.21.15.27:0/437704639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74040603f0 0x7f74040f9340 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:20.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.446+0000 7f7409f93700 1 -- 172.21.15.27:0/437704639 >> 172.21.15.27:0/437704639 conn(0x7f74040f0dc0 msgr2=0x7f74040f31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:15:20.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.446+0000 7f7409f93700 1 -- 172.21.15.27:0/437704639 shutdown_connections 2021-11-03T18:15:20.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.446+0000 7f7409f93700 1 -- 172.21.15.27:0/437704639 wait complete. 2021-11-03T18:15:20.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.447+0000 7f7409f93700 1 Processor -- start 2021-11-03T18:15:20.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.447+0000 7f7409f93700 1 -- start start 2021-11-03T18:15:20.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.447+0000 7f7409f93700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74040603f0 0x7f74040f6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:15:20.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.447+0000 7f7409f93700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74040f9830 0x7f74040f5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:15:20.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.447+0000 7f7409f93700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74040f57a0 con 0x7f74040603f0 2021-11-03T18:15:20.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.447+0000 7f7409f93700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74040f58e0 con 0x7f74040f9830 2021-11-03T18:15:20.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.447+0000 7f74037fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74040603f0 0x7f74040f6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:15:20.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.447+0000 7f74037fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74040603f0 0x7f74040f6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33800/0 (socket says 172.21.15.27:33800) 2021-11-03T18:15:20.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.447+0000 7f74037fe700 1 -- 172.21.15.27:0/4002436011 learned_addr learned my addr 172.21.15.27:0/4002436011 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:15:20.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.448+0000 7f7402ffd700 1 --2- 172.21.15.27:0/4002436011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74040f9830 0x7f74040f5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:15:20.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.448+0000 7f74037fe700 1 -- 172.21.15.27:0/4002436011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74040f9830 msgr2=0x7f74040f5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:15:20.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.448+0000 7f74037fe700 1 --2- 172.21.15.27:0/4002436011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74040f9830 0x7f74040f5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:20.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.448+0000 7f74037fe700 1 -- 172.21.15.27:0/4002436011 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73f4005040 con 0x7f74040603f0 2021-11-03T18:15:20.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.448+0000 7f7402ffd700 1 --2- 172.21.15.27:0/4002436011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74040f9830 0x7f74040f5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T18:15:20.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.448+0000 7f74037fe700 1 --2- 172.21.15.27:0/4002436011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74040603f0 0x7f74040f6dc0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 rx=0x7f73f4004f60 tx=0x7f73f4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:15:20.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.448+0000 7f73ebfff700 1 -- 172.21.15.27:0/4002436011 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73f4018070 con 0x7f74040603f0 2021-11-03T18:15:20.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.448+0000 7f73ebfff700 1 -- 172.21.15.27:0/4002436011 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f73f4007ba0 con 0x7f74040603f0 2021-11-03T18:15:20.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.448+0000 7f7409f93700 1 -- 172.21.15.27:0/4002436011 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74040f5b60 con 0x7f74040603f0 2021-11-03T18:15:20.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.448+0000 7f73ebfff700 1 -- 172.21.15.27:0/4002436011 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73f401c620 con 0x7f74040603f0 2021-11-03T18:15:20.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.448+0000 7f7409f93700 1 -- 172.21.15.27:0/4002436011 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74040f60c0 con 0x7f74040603f0 2021-11-03T18:15:20.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.449+0000 7f7409f93700 1 -- 172.21.15.27:0/4002436011 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7404045ad0 con 0x7f74040603f0 2021-11-03T18:15:20.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.450+0000 7f73ebfff700 1 -- 172.21.15.27:0/4002436011 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f73f401c8e0 con 0x7f74040603f0 2021-11-03T18:15:20.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.450+0000 7f73ebfff700 1 --2- 172.21.15.27:0/4002436011 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f73ec0640b0 0x7f73ec066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:15:20.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.450+0000 7f73ebfff700 1 -- 172.21.15.27:0/4002436011 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f73f4082740 con 0x7f74040603f0 2021-11-03T18:15:20.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.450+0000 7f7402ffd700 1 --2- 172.21.15.27:0/4002436011 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f73ec0640b0 0x7f73ec066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:15:20.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.450+0000 7f7402ffd700 1 --2- 172.21.15.27:0/4002436011 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f73ec0640b0 0x7f73ec066570 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7f73f8004ac0 tx=0x7f73f8009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:15:20.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.454+0000 7f73ebfff700 1 -- 172.21.15.27:0/4002436011 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f73f4054a40 con 0x7f74040603f0 2021-11-03T18:15:20.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.634+0000 7f7409f93700 1 -- 172.21.15.27:0/4002436011 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f74040fef70 con 0x7f73ec0640b0 2021-11-03T18:15:20.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.635+0000 7f73ebfff700 1 -- 172.21.15.27:0/4002436011 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f74040fef70 con 0x7f73ec0640b0 2021-11-03T18:15:20.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.637+0000 7f7409f93700 1 -- 172.21.15.27:0/4002436011 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f73ec0640b0 msgr2=0x7f73ec066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:15:20.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.637+0000 7f7409f93700 1 --2- 172.21.15.27:0/4002436011 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f73ec0640b0 0x7f73ec066570 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 rx=0x7f73f8004ac0 tx=0x7f73f8009040).stop 2021-11-03T18:15:20.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.637+0000 7f7409f93700 1 -- 172.21.15.27:0/4002436011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74040603f0 msgr2=0x7f74040f6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:15:20.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.637+0000 7f7409f93700 1 --2- 172.21.15.27:0/4002436011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74040603f0 0x7f74040f6dc0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 rx=0x7f73f4004f60 tx=0x7f73f4009b30).stop 2021-11-03T18:15:20.637 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.637+0000 7f7409f93700 1 -- 172.21.15.27:0/4002436011 shutdown_connections 2021-11-03T18:15:20.637 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.637+0000 7f7409f93700 1 --2- 172.21.15.27:0/4002436011 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f73ec0640b0 0x7f73ec066570 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:20.637 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.637+0000 7f7409f93700 1 --2- 172.21.15.27:0/4002436011 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74040603f0 0x7f74040f6dc0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:20.638 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.637+0000 7f7409f93700 1 --2- 172.21.15.27:0/4002436011 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74040f9830 0x7f74040f5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:20.638 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.637+0000 7f7409f93700 1 -- 172.21.15.27:0/4002436011 >> 172.21.15.27:0/4002436011 conn(0x7f74040f0dc0 msgr2=0x7f74040f31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:15:20.638 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.637+0000 7f7409f93700 1 -- 172.21.15.27:0/4002436011 shutdown_connections 2021-11-03T18:15:20.638 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.637+0000 7f7409f93700 1 -- 172.21.15.27:0/4002436011 wait complete. 2021-11-03T18:15:20.645 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:15:20.738 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.738+0000 7fa5c96c4700 1 -- 172.21.15.27:0/2205472752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5c40f55c0 msgr2=0x7fa5c40f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:15:20.739 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.738+0000 7fa5c96c4700 1 --2- 172.21.15.27:0/2205472752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5c40f55c0 0x7fa5c40f59e0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 rx=0x7fa5b4004660 tx=0x7fa5b4009b30).stop 2021-11-03T18:15:20.739 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.738+0000 7fa5c96c4700 1 -- 172.21.15.27:0/2205472752 shutdown_connections 2021-11-03T18:15:20.739 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.738+0000 7fa5c96c4700 1 --2- 172.21.15.27:0/2205472752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa5c40f6860 0x7fa5c40f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:20.739 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.738+0000 7fa5c96c4700 1 --2- 172.21.15.27:0/2205472752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5c40f55c0 0x7fa5c40f59e0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:20.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.738+0000 7fa5c96c4700 1 -- 172.21.15.27:0/2205472752 >> 172.21.15.27:0/2205472752 conn(0x7fa5c40f0d40 msgr2=0x7fa5c40f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:15:20.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.738+0000 7fa5c96c4700 1 -- 172.21.15.27:0/2205472752 shutdown_connections 2021-11-03T18:15:20.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.738+0000 7fa5c96c4700 1 -- 172.21.15.27:0/2205472752 wait complete. 2021-11-03T18:15:20.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.739+0000 7fa5c96c4700 1 Processor -- start 2021-11-03T18:15:20.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.739+0000 7fa5c96c4700 1 -- start start 2021-11-03T18:15:20.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.739+0000 7fa5c96c4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5c40f55c0 0x7fa5c40f9f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:15:20.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.739+0000 7fa5c96c4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa5c40f6860 0x7fa5c40f83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:15:20.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.739+0000 7fa5c96c4700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c40f8920 con 0x7fa5c40f55c0 2021-11-03T18:15:20.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.739+0000 7fa5c96c4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5c40f8a60 con 0x7fa5c40f6860 2021-11-03T18:15:20.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.739+0000 7fa5c2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5c40f55c0 0x7fa5c40f9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:15:20.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.739+0000 7fa5c2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5c40f55c0 0x7fa5c40f9f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33810/0 (socket says 172.21.15.27:33810) 2021-11-03T18:15:20.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.739+0000 7fa5c2ffd700 1 -- 172.21.15.27:0/1694046994 learned_addr learned my addr 172.21.15.27:0/1694046994 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:15:20.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.739+0000 7fa5c27fc700 1 --2- 172.21.15.27:0/1694046994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa5c40f6860 0x7fa5c40f83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:15:20.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.739+0000 7fa5c2ffd700 1 -- 172.21.15.27:0/1694046994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa5c40f6860 msgr2=0x7fa5c40f83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:15:20.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.739+0000 7fa5c2ffd700 1 --2- 172.21.15.27:0/1694046994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa5c40f6860 0x7fa5c40f83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:20.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.739+0000 7fa5c2ffd700 1 -- 172.21.15.27:0/1694046994 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b4005040 con 0x7fa5c40f55c0 2021-11-03T18:15:20.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.739+0000 7fa5c2ffd700 1 --2- 172.21.15.27:0/1694046994 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5c40f55c0 0x7fa5c40f9f40 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 rx=0x7fa5b4004f60 tx=0x7fa5b4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:15:20.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.740+0000 7fa5ab7fe700 1 -- 172.21.15.27:0/1694046994 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5b4018070 con 0x7fa5c40f55c0 2021-11-03T18:15:20.744 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.740+0000 7fa5c96c4700 1 -- 172.21.15.27:0/1694046994 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5c40f8ce0 con 0x7fa5c40f55c0 2021-11-03T18:15:20.744 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.740+0000 7fa5ab7fe700 1 -- 172.21.15.27:0/1694046994 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fa5b4009770 con 0x7fa5c40f55c0 2021-11-03T18:15:20.744 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.740+0000 7fa5ab7fe700 1 -- 172.21.15.27:0/1694046994 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa5b401c3c0 con 0x7fa5c40f55c0 2021-11-03T18:15:20.744 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.740+0000 7fa5c96c4700 1 -- 172.21.15.27:0/1694046994 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5c40f9240 con 0x7fa5c40f55c0 2021-11-03T18:15:20.745 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.741+0000 7fa5c96c4700 1 -- 172.21.15.27:0/1694046994 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5c4045ad0 con 0x7fa5c40f55c0 2021-11-03T18:15:20.746 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.741+0000 7fa5ab7fe700 1 -- 172.21.15.27:0/1694046994 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fa5b401d420 con 0x7fa5c40f55c0 2021-11-03T18:15:20.746 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.741+0000 7fa5ab7fe700 1 --2- 172.21.15.27:0/1694046994 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa5ac0640b0 0x7fa5ac066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:15:20.746 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.742+0000 7fa5ab7fe700 1 -- 172.21.15.27:0/1694046994 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fa5b4082650 con 0x7fa5c40f55c0 2021-11-03T18:15:20.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.742+0000 7fa5c27fc700 1 --2- 172.21.15.27:0/1694046994 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa5ac0640b0 0x7fa5ac066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:15:20.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.742+0000 7fa5c27fc700 1 --2- 172.21.15.27:0/1694046994 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa5ac0640b0 0x7fa5ac066570 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7fa5b8007fd0 tx=0x7fa5b8009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:15:20.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.746+0000 7fa5ab7fe700 1 -- 172.21.15.27:0/1694046994 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fa5b4054950 con 0x7fa5c40f55c0 2021-11-03T18:15:20.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:20 smithi027 ceph-mon[97244]: pgmap v1820: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:20.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:20 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:15:20.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3562.445055 secs 2021-11-03T18:15:20.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.919+0000 7fa5c96c4700 1 -- 172.21.15.27:0/1694046994 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa5c405fe50 con 0x7fa5ac0640b0 2021-11-03T18:15:20.932 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.933+0000 7fa5ab7fe700 1 -- 172.21.15.27:0/1694046994 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fa5c405fe50 con 0x7fa5ac0640b0 2021-11-03T18:15:20.932 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:15:20.933 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (66m) 6m ago 68m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:15:20.933 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (58m) 6m ago 68m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:15:20.933 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (58m) 5m ago 67m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:15:20.934 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (66m) 6m ago 67m 42.1M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:15:20.934 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (60m) 6m ago 63m 26.3M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:15:20.935 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (63m) 6m ago 63m 3847M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:15:20.935 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (59m) 5m ago 63m 25.9M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:15:20.935 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (60m) 5m ago 63m 27.0M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:15:20.935 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (60m) 6m ago 69m 572M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:15:20.936 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (59m) 5m ago 67m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:15:20.936 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (59m) 6m ago 69m 273M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:15:20.936 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (59m) 5m ago 67m 129M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:15:20.936 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (67m) 6m ago 67m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:15:20.937 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (67m) 5m ago 67m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:15:20.937 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (58m) 6m ago 66m 371M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:15:20.937 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (57m) 6m ago 66m 405M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:15:20.937 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (57m) 6m ago 65m 302M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:15:20.938 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (56m) 5m ago 65m 429M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:15:20.938 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (56m) 5m ago 65m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:15:20.938 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (55m) 5m ago 64m 348M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:15:20.938 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (66m) 6m ago 67m 57.6M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:15:20.940 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.935+0000 7fa5c96c4700 1 -- 172.21.15.27:0/1694046994 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa5ac0640b0 msgr2=0x7fa5ac066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:15:20.940 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.935+0000 7fa5c96c4700 1 --2- 172.21.15.27:0/1694046994 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa5ac0640b0 0x7fa5ac066570 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7fa5b8007fd0 tx=0x7fa5b8009040).stop 2021-11-03T18:15:20.940 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.935+0000 7fa5c96c4700 1 -- 172.21.15.27:0/1694046994 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5c40f55c0 msgr2=0x7fa5c40f9f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:15:20.940 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.935+0000 7fa5c96c4700 1 --2- 172.21.15.27:0/1694046994 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5c40f55c0 0x7fa5c40f9f40 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 rx=0x7fa5b4004f60 tx=0x7fa5b4009b30).stop 2021-11-03T18:15:20.941 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.935+0000 7fa5c96c4700 1 -- 172.21.15.27:0/1694046994 shutdown_connections 2021-11-03T18:15:20.941 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.935+0000 7fa5c96c4700 1 --2- 172.21.15.27:0/1694046994 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa5ac0640b0 0x7fa5ac066570 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:20.941 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.936+0000 7fa5c96c4700 1 --2- 172.21.15.27:0/1694046994 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa5c40f55c0 0x7fa5c40f9f40 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:20.941 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.936+0000 7fa5c96c4700 1 --2- 172.21.15.27:0/1694046994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa5c40f6860 0x7fa5c40f83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:20.942 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.936+0000 7fa5c96c4700 1 -- 172.21.15.27:0/1694046994 >> 172.21.15.27:0/1694046994 conn(0x7fa5c40f0d40 msgr2=0x7fa5c405f740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:15:20.942 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.936+0000 7fa5c96c4700 1 -- 172.21.15.27:0/1694046994 shutdown_connections 2021-11-03T18:15:20.942 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:20.936+0000 7fa5c96c4700 1 -- 172.21.15.27:0/1694046994 wait complete. 2021-11-03T18:15:20.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:20 smithi115 ceph-mon[65453]: pgmap v1820: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:20.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:20 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:15:20.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3562.445055 secs 2021-11-03T18:15:21.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.034+0000 7fe20ea87700 1 -- 172.21.15.27:0/2634840083 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2080fba10 msgr2=0x7fe2080fde00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:15:21.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.034+0000 7fe20ea87700 1 --2- 172.21.15.27:0/2634840083 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2080fba10 0x7fe2080fde00 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 rx=0x7fe1f8004660 tx=0x7fe1f8009b30).stop 2021-11-03T18:15:21.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.034+0000 7fe20ea87700 1 -- 172.21.15.27:0/2634840083 shutdown_connections 2021-11-03T18:15:21.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.034+0000 7fe20ea87700 1 --2- 172.21.15.27:0/2634840083 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2080fba10 0x7fe2080fde00 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:21.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.035+0000 7fe20ea87700 1 --2- 172.21.15.27:0/2634840083 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe208060310 0x7fe2080fb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:21.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.035+0000 7fe20ea87700 1 -- 172.21.15.27:0/2634840083 >> 172.21.15.27:0/2634840083 conn(0x7fe2080f0ba0 msgr2=0x7fe2080f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:15:21.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.035+0000 7fe20ea87700 1 -- 172.21.15.27:0/2634840083 shutdown_connections 2021-11-03T18:15:21.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.035+0000 7fe20ea87700 1 -- 172.21.15.27:0/2634840083 wait complete. 2021-11-03T18:15:21.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.035+0000 7fe20ea87700 1 Processor -- start 2021-11-03T18:15:21.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.035+0000 7fe20ea87700 1 -- start start 2021-11-03T18:15:21.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.035+0000 7fe20ea87700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe208060310 0x7fe208108f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:15:21.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.035+0000 7fe20ea87700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2080fba10 0x7fe208109410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:15:21.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.035+0000 7fe20ea87700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2081099c0 con 0x7fe208060310 2021-11-03T18:15:21.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.035+0000 7fe20ea87700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe208109b00 con 0x7fe2080fba10 2021-11-03T18:15:21.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.035+0000 7fe20da85700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe208060310 0x7fe208108f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:15:21.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.035+0000 7fe20da85700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe208060310 0x7fe208108f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33820/0 (socket says 172.21.15.27:33820) 2021-11-03T18:15:21.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.035+0000 7fe20da85700 1 -- 172.21.15.27:0/3000443846 learned_addr learned my addr 172.21.15.27:0/3000443846 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:15:21.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.036+0000 7fe20d284700 1 --2- 172.21.15.27:0/3000443846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2080fba10 0x7fe208109410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:15:21.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.036+0000 7fe20da85700 1 -- 172.21.15.27:0/3000443846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2080fba10 msgr2=0x7fe208109410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:15:21.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.036+0000 7fe20da85700 1 --2- 172.21.15.27:0/3000443846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2080fba10 0x7fe208109410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:21.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.036+0000 7fe20da85700 1 -- 172.21.15.27:0/3000443846 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f8005040 con 0x7fe208060310 2021-11-03T18:15:21.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.036+0000 7fe20d284700 1 --2- 172.21.15.27:0/3000443846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2080fba10 0x7fe208109410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:15:21.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.036+0000 7fe20da85700 1 --2- 172.21.15.27:0/3000443846 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe208060310 0x7fe208108f20 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 rx=0x7fe20400c790 tx=0x7fe204007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:15:21.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.036+0000 7fe1fe7fc700 1 -- 172.21.15.27:0/3000443846 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe20400e680 con 0x7fe208060310 2021-11-03T18:15:21.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.036+0000 7fe1fe7fc700 1 -- 172.21.15.27:0/3000443846 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fe20400ec90 con 0x7fe208060310 2021-11-03T18:15:21.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.036+0000 7fe20ea87700 1 -- 172.21.15.27:0/3000443846 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe20810a4a0 con 0x7fe208060310 2021-11-03T18:15:21.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.036+0000 7fe1fe7fc700 1 -- 172.21.15.27:0/3000443846 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe204013d60 con 0x7fe208060310 2021-11-03T18:15:21.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.036+0000 7fe20ea87700 1 -- 172.21.15.27:0/3000443846 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe20810aa60 con 0x7fe208060310 2021-11-03T18:15:21.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.037+0000 7fe20ea87700 1 -- 172.21.15.27:0/3000443846 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe208045ad0 con 0x7fe208060310 2021-11-03T18:15:21.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.037+0000 7fe1fe7fc700 1 -- 172.21.15.27:0/3000443846 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fe20401a060 con 0x7fe208060310 2021-11-03T18:15:21.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.038+0000 7fe1fe7fc700 1 --2- 172.21.15.27:0/3000443846 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe1f4064100 0x7fe1f40665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:15:21.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.038+0000 7fe1fe7fc700 1 -- 172.21.15.27:0/3000443846 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fe204080a80 con 0x7fe208060310 2021-11-03T18:15:21.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.038+0000 7fe20d284700 1 --2- 172.21.15.27:0/3000443846 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe1f4064100 0x7fe1f40665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:15:21.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.038+0000 7fe20d284700 1 --2- 172.21.15.27:0/3000443846 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe1f4064100 0x7fe1f40665c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7fe1f8004e30 tx=0x7fe1f8009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:15:21.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.042+0000 7fe1fe7fc700 1 -- 172.21.15.27:0/3000443846 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fe204052c30 con 0x7fe208060310 2021-11-03T18:15:21.270 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.271+0000 7fe20ea87700 1 -- 172.21.15.27:0/3000443846 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe208027a00 con 0x7fe208060310 2021-11-03T18:15:21.271 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.272+0000 7fe1fe7fc700 1 -- 172.21.15.27:0/3000443846 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fe204018070 con 0x7fe208060310 2021-11-03T18:15:21.271 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:15:21.271 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:15:21.272 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:15:21.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:15:21.272 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:15:21.272 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:15:21.272 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:15:21.273 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:15:21.273 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:15:21.273 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:15:21.273 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:15:21.274 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:15:21.274 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:15:21.274 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:15:21.274 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:15:21.275 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:15:21.275 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:15:21.275 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:15:21.276 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.273+0000 7fe20ea87700 1 -- 172.21.15.27:0/3000443846 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe1f4064100 msgr2=0x7fe1f40665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:15:21.276 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.273+0000 7fe20ea87700 1 --2- 172.21.15.27:0/3000443846 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe1f4064100 0x7fe1f40665c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7fe1f8004e30 tx=0x7fe1f8009b30).stop 2021-11-03T18:15:21.277 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.273+0000 7fe20ea87700 1 -- 172.21.15.27:0/3000443846 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe208060310 msgr2=0x7fe208108f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:15:21.277 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.273+0000 7fe20ea87700 1 --2- 172.21.15.27:0/3000443846 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe208060310 0x7fe208108f20 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 rx=0x7fe20400c790 tx=0x7fe204007d70).stop 2021-11-03T18:15:21.277 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.273+0000 7fe20ea87700 1 -- 172.21.15.27:0/3000443846 shutdown_connections 2021-11-03T18:15:21.277 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.273+0000 7fe20ea87700 1 --2- 172.21.15.27:0/3000443846 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe1f4064100 0x7fe1f40665c0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:21.277 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.273+0000 7fe20ea87700 1 --2- 172.21.15.27:0/3000443846 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe208060310 0x7fe208108f20 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:21.278 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.273+0000 7fe20ea87700 1 --2- 172.21.15.27:0/3000443846 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2080fba10 0x7fe208109410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:21.278 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.273+0000 7fe20ea87700 1 -- 172.21.15.27:0/3000443846 >> 172.21.15.27:0/3000443846 conn(0x7fe2080f0ba0 msgr2=0x7fe2080f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:15:21.278 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.274+0000 7fe20ea87700 1 -- 172.21.15.27:0/3000443846 shutdown_connections 2021-11-03T18:15:21.278 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.274+0000 7fe20ea87700 1 -- 172.21.15.27:0/3000443846 wait complete. 2021-11-03T18:15:21.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.374+0000 7f33b2eb7700 1 -- 172.21.15.27:0/567922703 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33ac0f5ed0 msgr2=0x7f33ac0f6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:15:21.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.374+0000 7f33b2eb7700 1 --2- 172.21.15.27:0/567922703 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33ac0f5ed0 0x7f33ac0f6900 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 rx=0x7f3398004660 tx=0x7f3398009b30).stop 2021-11-03T18:15:21.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.374+0000 7f33b2eb7700 1 -- 172.21.15.27:0/567922703 shutdown_connections 2021-11-03T18:15:21.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.374+0000 7f33b2eb7700 1 --2- 172.21.15.27:0/567922703 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33ac0f5ed0 0x7f33ac0f6900 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:21.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.374+0000 7f33b2eb7700 1 --2- 172.21.15.27:0/567922703 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33ac0f55c0 0x7f33ac0f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:21.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.374+0000 7f33b2eb7700 1 -- 172.21.15.27:0/567922703 >> 172.21.15.27:0/567922703 conn(0x7f33ac0f0dc0 msgr2=0x7f33ac0f31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:15:21.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.374+0000 7f33b2eb7700 1 -- 172.21.15.27:0/567922703 shutdown_connections 2021-11-03T18:15:21.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.374+0000 7f33b2eb7700 1 -- 172.21.15.27:0/567922703 wait complete. 2021-11-03T18:15:21.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.374+0000 7f33b2eb7700 1 Processor -- start 2021-11-03T18:15:21.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.375+0000 7f33b2eb7700 1 -- start start 2021-11-03T18:15:21.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.375+0000 7f33b2eb7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33ac0f55c0 0x7f33ac109c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:15:21.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.375+0000 7f33b2eb7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33ac0f5ed0 0x7f33ac10c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:15:21.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.375+0000 7f33b2eb7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac060560 con 0x7f33ac0f55c0 2021-11-03T18:15:21.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.375+0000 7f33b2eb7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33ac0606a0 con 0x7f33ac0f5ed0 2021-11-03T18:15:21.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.375+0000 7f33b0c53700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33ac0f55c0 0x7f33ac109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:15:21.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.375+0000 7f33b0c53700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33ac0f55c0 0x7f33ac109c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33830/0 (socket says 172.21.15.27:33830) 2021-11-03T18:15:21.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.375+0000 7f33b0c53700 1 -- 172.21.15.27:0/342137043 learned_addr learned my addr 172.21.15.27:0/342137043 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:15:21.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.375+0000 7f33abfff700 1 --2- 172.21.15.27:0/342137043 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33ac0f5ed0 0x7f33ac10c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:15:21.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.375+0000 7f33b0c53700 1 -- 172.21.15.27:0/342137043 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33ac0f5ed0 msgr2=0x7f33ac10c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:15:21.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.375+0000 7f33b0c53700 1 --2- 172.21.15.27:0/342137043 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33ac0f5ed0 0x7f33ac10c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:21.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.375+0000 7f33b0c53700 1 -- 172.21.15.27:0/342137043 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3398005040 con 0x7f33ac0f55c0 2021-11-03T18:15:21.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.375+0000 7f33abfff700 1 --2- 172.21.15.27:0/342137043 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33ac0f5ed0 0x7f33ac10c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:15:21.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.375+0000 7f33b0c53700 1 --2- 172.21.15.27:0/342137043 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33ac0f55c0 0x7f33ac109c70 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 rx=0x7f33a00026d0 tx=0x7f33a0009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:15:21.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.376+0000 7f33a97fa700 1 -- 172.21.15.27:0/342137043 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33a0009320 con 0x7f33ac0f55c0 2021-11-03T18:15:21.382 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.376+0000 7f33a97fa700 1 -- 172.21.15.27:0/342137043 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f33a000f040 con 0x7f33ac0f55c0 2021-11-03T18:15:21.382 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.376+0000 7f33b2eb7700 1 -- 172.21.15.27:0/342137043 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33ac10c720 con 0x7f33ac0f55c0 2021-11-03T18:15:21.382 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.376+0000 7f33a97fa700 1 -- 172.21.15.27:0/342137043 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f33a00147e0 con 0x7f33ac0f55c0 2021-11-03T18:15:21.382 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.376+0000 7f33b2eb7700 1 -- 172.21.15.27:0/342137043 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33ac10cce0 con 0x7f33ac0f55c0 2021-11-03T18:15:21.383 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.377+0000 7f33b2eb7700 1 -- 172.21.15.27:0/342137043 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33ac045ad0 con 0x7f33ac0f55c0 2021-11-03T18:15:21.383 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.377+0000 7f33a97fa700 1 -- 172.21.15.27:0/342137043 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f33a0009480 con 0x7f33ac0f55c0 2021-11-03T18:15:21.383 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.377+0000 7f33a97fa700 1 --2- 172.21.15.27:0/342137043 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f339c0640b0 0x7f339c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:15:21.383 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.377+0000 7f33a97fa700 1 -- 172.21.15.27:0/342137043 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f33a0080f10 con 0x7f33ac0f55c0 2021-11-03T18:15:21.383 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.378+0000 7f33abfff700 1 --2- 172.21.15.27:0/342137043 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f339c0640b0 0x7f339c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:15:21.384 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.378+0000 7f33abfff700 1 --2- 172.21.15.27:0/342137043 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f339c0640b0 0x7f339c066570 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7f3398004630 tx=0x7f3398009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:15:21.384 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.381+0000 7f33a97fa700 1 -- 172.21.15.27:0/342137043 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f33a0053210 con 0x7f33ac0f55c0 2021-11-03T18:15:21.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.588+0000 7f33b2eb7700 1 -- 172.21.15.27:0/342137043 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f33ac0fdeb0 con 0x7f33ac0f55c0 2021-11-03T18:15:21.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.588+0000 7f33a97fa700 1 -- 172.21.15.27:0/342137043 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f33a0052e00 con 0x7f33ac0f55c0 2021-11-03T18:15:21.587 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:15:21.588 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:15:21.588 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:15:21.588 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:15:21.589 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:15:21.589 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:15:21.589 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:15:21.589 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:15:21.589 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:15:21.590 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:15:21.590 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:15:21.590 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:15:21.590 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:15:21.591 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:15:21.591 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:15:21.591 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:15:21.591 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:15:21.592 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:15:21.592 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:15:21.592 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:15:21.593 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:15:21.593 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:15:21.593 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:15:21.593 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:15:21.594 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:15:21.594 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:15:21.594 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:15:21.594 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:15:21.595 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:15:21.595 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:15:21.595 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:15:21.595 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:15:21.595 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:15:21.596 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:15:21.596 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:15:21.596 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:15:21.596 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:15:21.597 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:15:21.597 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:15:21.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.590+0000 7f33b2eb7700 1 -- 172.21.15.27:0/342137043 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f339c0640b0 msgr2=0x7f339c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:15:21.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.590+0000 7f33b2eb7700 1 --2- 172.21.15.27:0/342137043 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f339c0640b0 0x7f339c066570 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 rx=0x7f3398004630 tx=0x7f3398009b30).stop 2021-11-03T18:15:21.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.590+0000 7f33b2eb7700 1 -- 172.21.15.27:0/342137043 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33ac0f55c0 msgr2=0x7f33ac109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:15:21.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.590+0000 7f33b2eb7700 1 --2- 172.21.15.27:0/342137043 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33ac0f55c0 0x7f33ac109c70 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 rx=0x7f33a00026d0 tx=0x7f33a0009de0).stop 2021-11-03T18:15:21.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.590+0000 7f33b2eb7700 1 -- 172.21.15.27:0/342137043 shutdown_connections 2021-11-03T18:15:21.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.590+0000 7f33b2eb7700 1 --2- 172.21.15.27:0/342137043 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f339c0640b0 0x7f339c066570 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:21.600 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.590+0000 7f33b2eb7700 1 --2- 172.21.15.27:0/342137043 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f33ac0f55c0 0x7f33ac109c70 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:21.600 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.590+0000 7f33b2eb7700 1 --2- 172.21.15.27:0/342137043 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f33ac0f5ed0 0x7f33ac10c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:21.600 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.590+0000 7f33b2eb7700 1 -- 172.21.15.27:0/342137043 >> 172.21.15.27:0/342137043 conn(0x7f33ac0f0dc0 msgr2=0x7f33ac0fa350 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:15:21.600 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.590+0000 7f33b2eb7700 1 -- 172.21.15.27:0/342137043 shutdown_connections 2021-11-03T18:15:21.600 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:21.590+0000 7f33b2eb7700 1 -- 172.21.15.27:0/342137043 wait complete. 2021-11-03T18:15:21.601 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:15:21.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:21 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3000443846' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:22.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:21 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3000443846' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:22.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:22 smithi027 ceph-mon[97244]: from='client.35976 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:15:22.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:22 smithi027 ceph-mon[97244]: pgmap v1821: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:22.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:22 smithi027 ceph-mon[97244]: from='client.35980 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:15:22.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:22 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/342137043' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:15:22.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:22 smithi115 ceph-mon[65453]: from='client.35976 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:15:22.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:22 smithi115 ceph-mon[65453]: pgmap v1821: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:22.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:22 smithi115 ceph-mon[65453]: from='client.35980 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:15:22.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:22 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/342137043' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:15:24.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:24 smithi115 ceph-mon[65453]: pgmap v1822: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:25.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:24 smithi027 ceph-mon[97244]: pgmap v1822: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:25.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3567.445202 secs 2021-11-03T18:15:26.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3567.445202 secs 2021-11-03T18:15:26.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:26 smithi115 ceph-mon[65453]: pgmap v1823: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:27.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:26 smithi027 ceph-mon[97244]: pgmap v1823: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:28.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:28 smithi115 ceph-mon[65453]: pgmap v1824: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:29.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:28 smithi027 ceph-mon[97244]: pgmap v1824: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:29.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:15:29.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:15:29.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:15:29.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:29.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:29.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:29.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:30.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:15:30.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:15:30.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:15:30.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:30.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:30.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:30.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:30.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:30 smithi115 ceph-mon[65453]: pgmap v1825: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:30.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:30 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:15:30.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3572.445335 secs 2021-11-03T18:15:31.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:30 smithi027 ceph-mon[97244]: pgmap v1825: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:31.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:30 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:15:31.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3572.445335 secs 2021-11-03T18:15:31.868 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 747, in recv_stderr out = self.in_stderr_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1376, in _read return self.channel.recv_stderr(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 749, in recv_stderr raise socket.timeout() socket.timeout 2021-11-03T18:15:31.870 ERROR:teuthology:Uncaught exception (Hub) Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 699, in recv out = self.in_buffer.read(nbytes, self.timeout) File "/home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/virtualenv/lib/python3.6/site-packages/paramiko/buffered_pipe.py", line 164, in read raise PipeTimeout() paramiko.buffered_pipe.PipeTimeout During handling of the above exception, another exception occurred: Traceback (most recent call last): File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run File "/home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/teuthology/orchestra/run.py", line 323, in copy_file_to copy_to_log(src, logger, capture=stream, quiet=quiet) File "/home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/teuthology/orchestra/run.py", line 276, in copy_to_log for line in f: File "/home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 125, in __next__ line = self.readline() File "/home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/virtualenv/lib/python3.6/site-packages/paramiko/file.py", line 291, in readline new_data = self._read(n) File "/home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 1361, in _read return self.channel.recv(size) File "/home/teuthworker/src/git.ceph.com_git_teuthology_c56135d151713269e811ede3163c9743c2e269de/virtualenv/lib/python3.6/site-packages/paramiko/channel.py", line 701, in recv raise socket.timeout() socket.timeout 2021-11-03T18:15:32.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:32 smithi027 ceph-mon[97244]: pgmap v1826: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:32.986 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:32 smithi115 ceph-mon[65453]: pgmap v1826: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:34.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:34 smithi115 ceph-mon[65453]: pgmap v1827: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:35.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:34 smithi027 ceph-mon[97244]: pgmap v1827: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:35.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3577.445478 secs 2021-11-03T18:15:36.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3577.445478 secs 2021-11-03T18:15:36.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:36 smithi115 ceph-mon[65453]: pgmap v1828: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:37.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:36 smithi027 ceph-mon[97244]: pgmap v1828: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:38.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:38 smithi115 ceph-mon[65453]: pgmap v1829: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:39.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:38 smithi027 ceph-mon[97244]: pgmap v1829: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:39.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:15:39.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:15:39.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:15:39.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:39.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:39.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:39.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:40.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:15:40.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:15:40.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:15:40.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:40.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:40.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:40.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:40.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:40 smithi115 ceph-mon[65453]: pgmap v1830: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:40.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:40 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:15:40.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3582.445618 secs 2021-11-03T18:15:41.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:40 smithi027 ceph-mon[97244]: pgmap v1830: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:41.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:40 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:15:41.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3582.445618 secs 2021-11-03T18:15:42.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:42 smithi027 ceph-mon[97244]: pgmap v1831: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:42.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:42 smithi115 ceph-mon[65453]: pgmap v1831: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:44.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:44 smithi115 ceph-mon[65453]: pgmap v1832: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:45.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:44 smithi027 ceph-mon[97244]: pgmap v1832: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:45.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3587.445760 secs 2021-11-03T18:15:46.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3587.445760 secs 2021-11-03T18:15:46.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:46 smithi115 ceph-mon[65453]: pgmap v1833: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:47.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:46 smithi027 ceph-mon[97244]: pgmap v1833: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:48.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:48 smithi115 ceph-mon[65453]: pgmap v1834: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:49.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:48 smithi027 ceph-mon[97244]: pgmap v1834: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:49.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:15:49.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:15:49.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:15:49.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:49.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:49.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:49.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:50.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:15:50.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:15:50.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:15:50.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:50.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:50.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:50.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:50.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:50 smithi115 ceph-mon[65453]: pgmap v1835: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:50.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3592.445838 secs 2021-11-03T18:15:50.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:50 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:15:51.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:50 smithi027 ceph-mon[97244]: pgmap v1835: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:51.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3592.445838 secs 2021-11-03T18:15:51.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:50 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:15:51.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.704+0000 7fb613fff700 1 -- 172.21.15.27:0/4174649325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6140f77d0 msgr2=0x7fb6140f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:15:51.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.704+0000 7fb613fff700 1 --2- 172.21.15.27:0/4174649325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6140f77d0 0x7fb6140f7c30 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 rx=0x7fb5fc004660 tx=0x7fb5fc009b30).stop 2021-11-03T18:15:51.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.704+0000 7fb613fff700 1 -- 172.21.15.27:0/4174649325 shutdown_connections 2021-11-03T18:15:51.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.704+0000 7fb613fff700 1 --2- 172.21.15.27:0/4174649325 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6140f77d0 0x7fb6140f7c30 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:51.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.704+0000 7fb613fff700 1 --2- 172.21.15.27:0/4174649325 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6140f6530 0x7fb6140f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:51.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.704+0000 7fb613fff700 1 -- 172.21.15.27:0/4174649325 >> 172.21.15.27:0/4174649325 conn(0x7fb6140f1cd0 msgr2=0x7fb6140f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:15:51.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.704+0000 7fb613fff700 1 -- 172.21.15.27:0/4174649325 shutdown_connections 2021-11-03T18:15:51.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.704+0000 7fb613fff700 1 -- 172.21.15.27:0/4174649325 wait complete. 2021-11-03T18:15:51.707 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.705+0000 7fb613fff700 1 Processor -- start 2021-11-03T18:15:51.707 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.705+0000 7fb613fff700 1 -- start start 2021-11-03T18:15:51.707 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.705+0000 7fb613fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6140f6530 0x7fb614106d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:15:51.707 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.705+0000 7fb613fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6140f77d0 0x7fb614107270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:15:51.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.705+0000 7fb613fff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb614107870 con 0x7fb6140f77d0 2021-11-03T18:15:51.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.705+0000 7fb613fff700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb614108080 con 0x7fb6140f6530 2021-11-03T18:15:51.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.705+0000 7fb6127fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6140f77d0 0x7fb614107270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:15:51.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.705+0000 7fb6127fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6140f77d0 0x7fb614107270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33850/0 (socket says 172.21.15.27:33850) 2021-11-03T18:15:51.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.705+0000 7fb6127fc700 1 -- 172.21.15.27:0/527167919 learned_addr learned my addr 172.21.15.27:0/527167919 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:15:51.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.705+0000 7fb6127fc700 1 -- 172.21.15.27:0/527167919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6140f6530 msgr2=0x7fb614106d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:15:51.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.705+0000 7fb6127fc700 1 --2- 172.21.15.27:0/527167919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6140f6530 0x7fb614106d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:15:51.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.705+0000 7fb6127fc700 1 -- 172.21.15.27:0/527167919 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5fc005040 con 0x7fb6140f77d0 2021-11-03T18:15:51.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.706+0000 7fb6127fc700 1 --2- 172.21.15.27:0/527167919 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6140f77d0 0x7fb614107270 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 rx=0x7fb5fc009990 tx=0x7fb5fc009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:15:51.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.706+0000 7fb60b7fe700 1 -- 172.21.15.27:0/527167919 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5fc018070 con 0x7fb6140f77d0 2021-11-03T18:15:51.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.706+0000 7fb60b7fe700 1 -- 172.21.15.27:0/527167919 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fb5fc007ba0 con 0x7fb6140f77d0 2021-11-03T18:15:51.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.706+0000 7fb613fff700 1 -- 172.21.15.27:0/527167919 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6141082a0 con 0x7fb6140f77d0 2021-11-03T18:15:51.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.706+0000 7fb60b7fe700 1 -- 172.21.15.27:0/527167919 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb5fc01c3c0 con 0x7fb6140f77d0 2021-11-03T18:15:51.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.706+0000 7fb613fff700 1 -- 172.21.15.27:0/527167919 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb614108800 con 0x7fb6140f77d0 2021-11-03T18:15:51.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.707+0000 7fb613fff700 1 -- 172.21.15.27:0/527167919 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb614059100 con 0x7fb6140f77d0 2021-11-03T18:15:51.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.707+0000 7fb60b7fe700 1 -- 172.21.15.27:0/527167919 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fb5fc01d420 con 0x7fb6140f77d0 2021-11-03T18:15:51.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.708+0000 7fb60b7fe700 1 --2- 172.21.15.27:0/527167919 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb600064100 0x7fb6000665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:15:51.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.708+0000 7fb60b7fe700 1 -- 172.21.15.27:0/527167919 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fb5fc082630 con 0x7fb6140f77d0 2021-11-03T18:15:51.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.708+0000 7fb612ffd700 1 --2- 172.21.15.27:0/527167919 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb600064100 0x7fb6000665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:15:51.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.708+0000 7fb612ffd700 1 --2- 172.21.15.27:0/527167919 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb600064100 0x7fb6000665c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7fb614106af0 tx=0x7fb60c0099e0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:15:51.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.712+0000 7fb60b7fe700 1 -- 172.21.15.27:0/527167919 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fb5fc054990 con 0x7fb6140f77d0 2021-11-03T18:15:51.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.894+0000 7fb613fff700 1 -- 172.21.15.27:0/527167919 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb6140fee30 con 0x7fb600064100 2021-11-03T18:15:51.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.895+0000 7fb60b7fe700 1 -- 172.21.15.27:0/527167919 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fb6140fee30 con 0x7fb600064100 2021-11-03T18:15:51.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.897+0000 7fb613fff700 1 -- 172.21.15.27:0/527167919 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb600064100 msgr2=0x7fb6000665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:15:51.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.897+0000 7fb613fff700 1 --2- 172.21.15.27:0/527167919 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb600064100 0x7fb6000665c0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7fb614106af0 tx=0x7fb60c0099e0).stop 2021-11-03T18:15:51.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.897+0000 7fb613fff700 1 -- 172.21.15.27:0/527167919 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6140f77d0 msgr2=0x7fb614107270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:15:51.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.897+0000 7fb613fff700 1 --2- 172.21.15.27:0/527167919 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6140f77d0 0x7fb614107270 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 rx=0x7fb5fc009990 tx=0x7fb5fc009b30).stop 2021-11-03T18:15:51.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.897+0000 7fb613fff700 1 -- 172.21.15.27:0/527167919 shutdown_connections 2021-11-03T18:15:51.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.897+0000 7fb613fff700 1 --2- 172.21.15.27:0/527167919 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb600064100 0x7fb6000665c0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:51.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.897+0000 7fb613fff700 1 --2- 172.21.15.27:0/527167919 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb6140f6530 0x7fb614106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:15:51.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.897+0000 7fb613fff700 1 --2- 172.21.15.27:0/527167919 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb6140f77d0 0x7fb614107270 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:51.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.897+0000 7fb613fff700 1 -- 172.21.15.27:0/527167919 >> 172.21.15.27:0/527167919 conn(0x7fb6140f1cd0 msgr2=0x7fb6140fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:15:51.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.897+0000 7fb613fff700 1 -- 172.21.15.27:0/527167919 shutdown_connections 2021-11-03T18:15:51.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.897+0000 7fb613fff700 1 -- 172.21.15.27:0/527167919 wait complete. 2021-11-03T18:15:51.906 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:15:51.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.998+0000 7fe0fa5b8700 1 -- 172.21.15.27:0/508029001 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0f40f66c0 msgr2=0x7fe0f40f6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:15:51.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.998+0000 7fe0fa5b8700 1 --2- 172.21.15.27:0/508029001 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0f40f66c0 0x7fe0f40f6ae0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 rx=0x7fe0e4004660 tx=0x7fe0e4009b30).stop 2021-11-03T18:15:51.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.998+0000 7fe0fa5b8700 1 -- 172.21.15.27:0/508029001 shutdown_connections 2021-11-03T18:15:51.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.998+0000 7fe0fa5b8700 1 --2- 172.21.15.27:0/508029001 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0f40f7960 0x7fe0f40f7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:51.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.998+0000 7fe0fa5b8700 1 --2- 172.21.15.27:0/508029001 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0f40f66c0 0x7fe0f40f6ae0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:51.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.998+0000 7fe0fa5b8700 1 -- 172.21.15.27:0/508029001 >> 172.21.15.27:0/508029001 conn(0x7fe0f40f1e20 msgr2=0x7fe0f40f4240 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:15:51.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.998+0000 7fe0fa5b8700 1 -- 172.21.15.27:0/508029001 shutdown_connections 2021-11-03T18:15:51.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.998+0000 7fe0fa5b8700 1 -- 172.21.15.27:0/508029001 wait complete. 2021-11-03T18:15:52.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.998+0000 7fe0fa5b8700 1 Processor -- start 2021-11-03T18:15:52.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.999+0000 7fe0fa5b8700 1 -- start start 2021-11-03T18:15:52.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.999+0000 7fe0fa5b8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0f40f66c0 0x7fe0f4109190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:15:52.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.999+0000 7fe0fa5b8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0f40f7960 0x7fe0f4109680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:15:52.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.999+0000 7fe0fa5b8700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f4109c30 con 0x7fe0f40f66c0 2021-11-03T18:15:52.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.999+0000 7fe0fa5b8700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0f4109d70 con 0x7fe0f40f7960 2021-11-03T18:15:52.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.999+0000 7fe0f3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0f40f66c0 0x7fe0f4109190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:15:52.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.999+0000 7fe0f3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0f40f66c0 0x7fe0f4109190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33858/0 (socket says 172.21.15.27:33858) 2021-11-03T18:15:52.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.999+0000 7fe0f3fff700 1 -- 172.21.15.27:0/2453823711 learned_addr learned my addr 172.21.15.27:0/2453823711 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:15:52.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.999+0000 7fe0f37fe700 1 --2- 172.21.15.27:0/2453823711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0f40f7960 0x7fe0f4109680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:15:52.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.999+0000 7fe0f3fff700 1 -- 172.21.15.27:0/2453823711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0f40f7960 msgr2=0x7fe0f4109680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:15:52.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.999+0000 7fe0f3fff700 1 --2- 172.21.15.27:0/2453823711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0f40f7960 0x7fe0f4109680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:52.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.999+0000 7fe0f3fff700 1 -- 172.21.15.27:0/2453823711 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0e4005040 con 0x7fe0f40f66c0 2021-11-03T18:15:52.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.999+0000 7fe0f37fe700 1 --2- 172.21.15.27:0/2453823711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0f40f7960 0x7fe0f4109680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T18:15:52.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.999+0000 7fe0f3fff700 1 --2- 172.21.15.27:0/2453823711 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0f40f66c0 0x7fe0f4109190 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 rx=0x7fe0e4000c00 tx=0x7fe0e4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:15:52.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.999+0000 7fe0f0ff9700 1 -- 172.21.15.27:0/2453823711 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0e4018070 con 0x7fe0f40f66c0 2021-11-03T18:15:52.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.999+0000 7fe0fa5b8700 1 -- 172.21.15.27:0/2453823711 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0f410a6b0 con 0x7fe0f40f66c0 2021-11-03T18:15:52.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.999+0000 7fe0f0ff9700 1 -- 172.21.15.27:0/2453823711 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fe0e4007d70 con 0x7fe0f40f66c0 2021-11-03T18:15:52.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.999+0000 7fe0f0ff9700 1 -- 172.21.15.27:0/2453823711 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe0e401c620 con 0x7fe0f40f66c0 2021-11-03T18:15:52.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:51.999+0000 7fe0fa5b8700 1 -- 172.21.15.27:0/2453823711 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0f410ac10 con 0x7fe0f40f66c0 2021-11-03T18:15:52.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.000+0000 7fe0fa5b8700 1 -- 172.21.15.27:0/2453823711 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0f4045ad0 con 0x7fe0f40f66c0 2021-11-03T18:15:52.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.001+0000 7fe0f0ff9700 1 -- 172.21.15.27:0/2453823711 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fe0e401c860 con 0x7fe0f40f66c0 2021-11-03T18:15:52.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.001+0000 7fe0f0ff9700 1 --2- 172.21.15.27:0/2453823711 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe0dc064090 0x7fe0dc066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:15:52.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.001+0000 7fe0f0ff9700 1 -- 172.21.15.27:0/2453823711 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fe0e40826e0 con 0x7fe0f40f66c0 2021-11-03T18:15:52.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.001+0000 7fe0f37fe700 1 --2- 172.21.15.27:0/2453823711 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe0dc064090 0x7fe0dc066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:15:52.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.002+0000 7fe0f37fe700 1 --2- 172.21.15.27:0/2453823711 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe0dc064090 0x7fe0dc066550 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7fe0e80041d0 tx=0x7fe0e8009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:15:52.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.005+0000 7fe0f0ff9700 1 -- 172.21.15.27:0/2453823711 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fe0e4054aa0 con 0x7fe0f40f66c0 2021-11-03T18:15:52.178 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.179+0000 7fe0fa5b8700 1 -- 172.21.15.27:0/2453823711 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe0f40fef90 con 0x7fe0dc064090 2021-11-03T18:15:52.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.194+0000 7fe0f0ff9700 1 -- 172.21.15.27:0/2453823711 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fe0f40fef90 con 0x7fe0dc064090 2021-11-03T18:15:52.193 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:15:52.193 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (67m) 6m ago 68m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:15:52.194 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (59m) 6m ago 68m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:15:52.194 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (58m) 5m ago 67m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:15:52.194 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (67m) 6m ago 68m 42.1M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:15:52.194 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (60m) 6m ago 63m 26.3M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:15:52.195 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (63m) 6m ago 63m 3847M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:15:52.195 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (59m) 5m ago 63m 25.9M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:15:52.195 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (61m) 5m ago 63m 27.0M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:15:52.195 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (61m) 6m ago 69m 572M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:15:52.196 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (60m) 5m ago 67m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:15:52.196 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (60m) 6m ago 70m 273M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:15:52.196 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (59m) 5m ago 67m 129M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:15:52.196 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (68m) 6m ago 68m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:15:52.196 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (67m) 5m ago 67m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:15:52.197 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (58m) 6m ago 67m 371M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:15:52.197 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (58m) 6m ago 66m 405M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:15:52.197 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (57m) 6m ago 66m 302M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:15:52.197 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (57m) 5m ago 66m 429M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:15:52.198 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (56m) 5m ago 65m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:15:52.198 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (56m) 5m ago 65m 348M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:15:52.198 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (67m) 6m ago 68m 57.6M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:15:52.199 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.196+0000 7fe0fa5b8700 1 -- 172.21.15.27:0/2453823711 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe0dc064090 msgr2=0x7fe0dc066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:15:52.199 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.196+0000 7fe0fa5b8700 1 --2- 172.21.15.27:0/2453823711 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe0dc064090 0x7fe0dc066550 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7fe0e80041d0 tx=0x7fe0e8009040).stop 2021-11-03T18:15:52.199 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.196+0000 7fe0fa5b8700 1 -- 172.21.15.27:0/2453823711 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0f40f66c0 msgr2=0x7fe0f4109190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:15:52.200 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.196+0000 7fe0fa5b8700 1 --2- 172.21.15.27:0/2453823711 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0f40f66c0 0x7fe0f4109190 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 rx=0x7fe0e4000c00 tx=0x7fe0e4009b30).stop 2021-11-03T18:15:52.200 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.196+0000 7fe0fa5b8700 1 -- 172.21.15.27:0/2453823711 shutdown_connections 2021-11-03T18:15:52.200 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.196+0000 7fe0fa5b8700 1 --2- 172.21.15.27:0/2453823711 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe0dc064090 0x7fe0dc066550 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:52.200 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.196+0000 7fe0fa5b8700 1 --2- 172.21.15.27:0/2453823711 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe0f40f66c0 0x7fe0f4109190 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:52.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.196+0000 7fe0fa5b8700 1 --2- 172.21.15.27:0/2453823711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe0f40f7960 0x7fe0f4109680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:52.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.196+0000 7fe0fa5b8700 1 -- 172.21.15.27:0/2453823711 >> 172.21.15.27:0/2453823711 conn(0x7fe0f40f1e20 msgr2=0x7fe0f406a520 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:15:52.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.196+0000 7fe0fa5b8700 1 -- 172.21.15.27:0/2453823711 shutdown_connections 2021-11-03T18:15:52.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.196+0000 7fe0fa5b8700 1 -- 172.21.15.27:0/2453823711 wait complete. 2021-11-03T18:15:52.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.295+0000 7f15b400e700 1 -- 172.21.15.27:0/3108693423 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15ac0f6860 msgr2=0x7f15ac0f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:15:52.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.295+0000 7f15b400e700 1 --2- 172.21.15.27:0/3108693423 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15ac0f6860 0x7f15ac0f6cc0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 rx=0x7f159c004660 tx=0x7f159c009b30).stop 2021-11-03T18:15:52.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.295+0000 7f15b400e700 1 -- 172.21.15.27:0/3108693423 shutdown_connections 2021-11-03T18:15:52.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.295+0000 7f15b400e700 1 --2- 172.21.15.27:0/3108693423 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15ac0f6860 0x7f15ac0f6cc0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:52.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.295+0000 7f15b400e700 1 --2- 172.21.15.27:0/3108693423 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15ac0f55c0 0x7f15ac0f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T18:15:52.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.295+0000 7f15b400e700 1 -- 172.21.15.27:0/3108693423 >> 172.21.15.27:0/3108693423 conn(0x7f15ac0f0d40 msgr2=0x7f15ac0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:15:52.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.295+0000 7f15b400e700 1 -- 172.21.15.27:0/3108693423 shutdown_connections 2021-11-03T18:15:52.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.295+0000 7f15b400e700 1 -- 172.21.15.27:0/3108693423 wait complete. 2021-11-03T18:15:52.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.296+0000 7f15b400e700 1 Processor -- start 2021-11-03T18:15:52.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.296+0000 7f15b400e700 1 -- start start 2021-11-03T18:15:52.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.296+0000 7f15b400e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15ac0f55c0 0x7f15ac109cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:15:52.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.296+0000 7f15b400e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15ac0f6860 0x7f15ac10c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:15:52.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.296+0000 7f15b400e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac060450 con 0x7f15ac0f55c0 2021-11-03T18:15:52.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.296+0000 7f15b400e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15ac060590 con 0x7f15ac0f6860 2021-11-03T18:15:52.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.296+0000 7f15b1daa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15ac0f55c0 0x7f15ac109cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:15:52.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.296+0000 7f15b1daa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15ac0f55c0 0x7f15ac109cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33868/0 (socket says 172.21.15.27:33868) 2021-11-03T18:15:52.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.296+0000 7f15b1daa700 1 -- 172.21.15.27:0/3058514616 learned_addr learned my addr 172.21.15.27:0/3058514616 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:15:52.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.296+0000 7f15b15a9700 1 --2- 172.21.15.27:0/3058514616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15ac0f6860 0x7f15ac10c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:15:52.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.296+0000 7f15b1daa700 1 -- 172.21.15.27:0/3058514616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15ac0f6860 msgr2=0x7f15ac10c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:15:52.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.296+0000 7f15b1daa700 1 --2- 172.21.15.27:0/3058514616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15ac0f6860 0x7f15ac10c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:52.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.296+0000 7f15b1daa700 1 -- 172.21.15.27:0/3058514616 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f159c005040 con 0x7f15ac0f55c0 2021-11-03T18:15:52.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.297+0000 7f15b1daa700 1 --2- 172.21.15.27:0/3058514616 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15ac0f55c0 0x7f15ac109cd0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 rx=0x7f15a80093f0 tx=0x7f15a8006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:15:52.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.297+0000 7f15a27fc700 1 -- 172.21.15.27:0/3058514616 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15a8013640 con 0x7f15ac0f55c0 2021-11-03T18:15:52.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.297+0000 7f15a27fc700 1 -- 172.21.15.27:0/3058514616 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f15a8009e40 con 0x7f15ac0f55c0 2021-11-03T18:15:52.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.297+0000 7f15b400e700 1 -- 172.21.15.27:0/3058514616 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15ac10c7e0 con 0x7f15ac0f55c0 2021-11-03T18:15:52.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.297+0000 7f15a27fc700 1 -- 172.21.15.27:0/3058514616 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f15a800dbf0 con 0x7f15ac0f55c0 2021-11-03T18:15:52.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.297+0000 7f15b400e700 1 -- 172.21.15.27:0/3058514616 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15ac10cda0 con 0x7f15ac0f55c0 2021-11-03T18:15:52.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.298+0000 7f15b400e700 1 -- 172.21.15.27:0/3058514616 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15ac045ad0 con 0x7f15ac0f55c0 2021-11-03T18:15:52.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.298+0000 7f15a27fc700 1 -- 172.21.15.27:0/3058514616 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f15a801a060 con 0x7f15ac0f55c0 2021-11-03T18:15:52.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.298+0000 7f15a27fc700 1 --2- 172.21.15.27:0/3058514616 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f15980640b0 0x7f1598066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:15:52.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.299+0000 7f15a27fc700 1 -- 172.21.15.27:0/3058514616 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f15a8080900 con 0x7f15ac0f55c0 2021-11-03T18:15:52.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.299+0000 7f15b15a9700 1 --2- 172.21.15.27:0/3058514616 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f15980640b0 0x7f1598066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:15:52.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.299+0000 7f15b15a9700 1 --2- 172.21.15.27:0/3058514616 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f15980640b0 0x7f1598066570 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f159c009da0 tx=0x7f159c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:15:52.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.303+0000 7f15a27fc700 1 -- 172.21.15.27:0/3058514616 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f15a8052b10 con 0x7f15ac0f55c0 2021-11-03T18:15:52.535 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.536+0000 7f15b400e700 1 -- 172.21.15.27:0/3058514616 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f15ac0586e0 con 0x7f15ac0f55c0 2021-11-03T18:15:52.535 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.537+0000 7f15a27fc700 1 -- 172.21.15.27:0/3058514616 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f15a8052700 con 0x7f15ac0f55c0 2021-11-03T18:15:52.536 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:15:52.536 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:15:52.536 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:15:52.536 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:15:52.537 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:15:52.537 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:15:52.537 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:15:52.537 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:15:52.537 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:15:52.538 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:15:52.538 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:15:52.538 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:15:52.538 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:15:52.539 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:15:52.539 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:15:52.539 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:15:52.539 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:15:52.540 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:15:52.540 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.538+0000 7f15b400e700 1 -- 172.21.15.27:0/3058514616 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f15980640b0 msgr2=0x7f1598066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:15:52.541 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.538+0000 7f15b400e700 1 --2- 172.21.15.27:0/3058514616 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f15980640b0 0x7f1598066570 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f159c009da0 tx=0x7f159c009b30).stop 2021-11-03T18:15:52.541 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.538+0000 7f15b400e700 1 -- 172.21.15.27:0/3058514616 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15ac0f55c0 msgr2=0x7f15ac109cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:15:52.541 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.538+0000 7f15b400e700 1 --2- 172.21.15.27:0/3058514616 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15ac0f55c0 0x7f15ac109cd0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 rx=0x7f15a80093f0 tx=0x7f15a8006ec0).stop 2021-11-03T18:15:52.541 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.538+0000 7f15b400e700 1 -- 172.21.15.27:0/3058514616 shutdown_connections 2021-11-03T18:15:52.541 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.538+0000 7f15b400e700 1 --2- 172.21.15.27:0/3058514616 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f15980640b0 0x7f1598066570 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:52.542 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.538+0000 7f15b400e700 1 --2- 172.21.15.27:0/3058514616 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f15ac0f55c0 0x7f15ac109cd0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:52.542 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.538+0000 7f15b400e700 1 --2- 172.21.15.27:0/3058514616 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f15ac0f6860 0x7f15ac10c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:52.542 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.538+0000 7f15b400e700 1 -- 172.21.15.27:0/3058514616 >> 172.21.15.27:0/3058514616 conn(0x7f15ac0f0d40 msgr2=0x7f15ac0fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:15:52.542 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.539+0000 7f15b400e700 1 -- 172.21.15.27:0/3058514616 shutdown_connections 2021-11-03T18:15:52.543 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.539+0000 7f15b400e700 1 -- 172.21.15.27:0/3058514616 wait complete. 2021-11-03T18:15:52.639 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.640+0000 7f0ca969a700 1 -- 172.21.15.27:0/153564026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ca40f55e0 msgr2=0x7f0ca40f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:15:52.639 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.640+0000 7f0ca969a700 1 --2- 172.21.15.27:0/153564026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ca40f55e0 0x7f0ca40f5a00 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 rx=0x7f0c94004660 tx=0x7f0c94009b30).stop 2021-11-03T18:15:52.640 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.640+0000 7f0ca969a700 1 -- 172.21.15.27:0/153564026 shutdown_connections 2021-11-03T18:15:52.640 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.640+0000 7f0ca969a700 1 --2- 172.21.15.27:0/153564026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ca40f5ef0 0x7f0ca40f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:52.640 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.640+0000 7f0ca969a700 1 --2- 172.21.15.27:0/153564026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ca40f55e0 0x7f0ca40f5a00 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:52.640 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.640+0000 7f0ca969a700 1 -- 172.21.15.27:0/153564026 >> 172.21.15.27:0/153564026 conn(0x7f0ca40f0de0 msgr2=0x7f0ca40f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:15:52.640 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.640+0000 7f0ca969a700 1 -- 172.21.15.27:0/153564026 shutdown_connections 2021-11-03T18:15:52.641 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.640+0000 7f0ca969a700 1 -- 172.21.15.27:0/153564026 wait complete. 2021-11-03T18:15:52.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.641+0000 7f0ca969a700 1 Processor -- start 2021-11-03T18:15:52.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.641+0000 7f0ca969a700 1 -- start start 2021-11-03T18:15:52.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.641+0000 7f0ca969a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ca40f55e0 0x7f0ca4105e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:15:52.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.641+0000 7f0ca969a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ca40f5ef0 0x7f0ca4106380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:15:52.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.641+0000 7f0ca969a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca4106930 con 0x7f0ca40f55e0 2021-11-03T18:15:52.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.641+0000 7f0ca969a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ca4106a70 con 0x7f0ca40f5ef0 2021-11-03T18:15:52.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.641+0000 7f0ca2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ca40f55e0 0x7f0ca4105e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:15:52.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.641+0000 7f0ca2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ca40f55e0 0x7f0ca4105e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33878/0 (socket says 172.21.15.27:33878) 2021-11-03T18:15:52.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.641+0000 7f0ca2ffd700 1 -- 172.21.15.27:0/1165758407 learned_addr learned my addr 172.21.15.27:0/1165758407 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:15:52.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.641+0000 7f0ca27fc700 1 --2- 172.21.15.27:0/1165758407 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ca40f5ef0 0x7f0ca4106380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:15:52.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.642+0000 7f0ca2ffd700 1 -- 172.21.15.27:0/1165758407 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ca40f5ef0 msgr2=0x7f0ca4106380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:15:52.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.642+0000 7f0ca2ffd700 1 --2- 172.21.15.27:0/1165758407 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ca40f5ef0 0x7f0ca4106380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:52.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.642+0000 7f0ca2ffd700 1 -- 172.21.15.27:0/1165758407 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c94005040 con 0x7f0ca40f55e0 2021-11-03T18:15:52.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.642+0000 7f0ca27fc700 1 --2- 172.21.15.27:0/1165758407 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ca40f5ef0 0x7f0ca4106380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:15:52.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.642+0000 7f0ca2ffd700 1 --2- 172.21.15.27:0/1165758407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ca40f55e0 0x7f0ca4105e90 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 rx=0x7f0c94009990 tx=0x7f0c94009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:15:52.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.642+0000 7f0c8b7fe700 1 -- 172.21.15.27:0/1165758407 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c94018070 con 0x7f0ca40f55e0 2021-11-03T18:15:52.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.642+0000 7f0ca969a700 1 -- 172.21.15.27:0/1165758407 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ca41073b0 con 0x7f0ca40f55e0 2021-11-03T18:15:52.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.642+0000 7f0ca969a700 1 -- 172.21.15.27:0/1165758407 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ca4107910 con 0x7f0ca40f55e0 2021-11-03T18:15:52.648 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.642+0000 7f0c8b7fe700 1 -- 172.21.15.27:0/1165758407 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f0c94007ba0 con 0x7f0ca40f55e0 2021-11-03T18:15:52.648 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.642+0000 7f0c8b7fe700 1 -- 172.21.15.27:0/1165758407 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c9401c3c0 con 0x7f0ca40f55e0 2021-11-03T18:15:52.648 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.643+0000 7f0ca969a700 1 -- 172.21.15.27:0/1165758407 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ca4045ad0 con 0x7f0ca40f55e0 2021-11-03T18:15:52.648 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.643+0000 7f0c8b7fe700 1 -- 172.21.15.27:0/1165758407 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f0c9401d420 con 0x7f0ca40f55e0 2021-11-03T18:15:52.648 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.644+0000 7f0c8b7fe700 1 --2- 172.21.15.27:0/1165758407 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0c8c0640b0 0x7f0c8c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:15:52.649 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.644+0000 7f0c8b7fe700 1 -- 172.21.15.27:0/1165758407 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f0c94082440 con 0x7f0ca40f55e0 2021-11-03T18:15:52.649 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.644+0000 7f0ca27fc700 1 --2- 172.21.15.27:0/1165758407 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0c8c0640b0 0x7f0c8c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:15:52.649 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.644+0000 7f0ca27fc700 1 --2- 172.21.15.27:0/1165758407 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0c8c0640b0 0x7f0c8c066570 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f0c98009740 tx=0x7f0c98009be0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:15:52.650 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.648+0000 7f0c8b7fe700 1 -- 172.21.15.27:0/1165758407 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f0c940547a0 con 0x7f0ca40f55e0 2021-11-03T18:15:52.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.852+0000 7f0ca969a700 1 -- 172.21.15.27:0/1165758407 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0ca40fdea0 con 0x7f0ca40f55e0 2021-11-03T18:15:52.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.852+0000 7f0c8b7fe700 1 -- 172.21.15.27:0/1165758407 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f0c94054390 con 0x7f0ca40f55e0 2021-11-03T18:15:52.851 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:15:52.852 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:15:52.852 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:15:52.852 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:15:52.852 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:15:52.853 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:15:52.853 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:15:52.853 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:15:52.853 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:15:52.853 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:15:52.854 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:15:52.854 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:15:52.854 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:15:52.854 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:15:52.855 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:15:52.855 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:15:52.855 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:15:52.855 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:15:52.856 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:15:52.856 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:15:52.856 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:15:52.856 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:15:52.856 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:15:52.857 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:15:52.857 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:15:52.857 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:15:52.857 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:15:52.858 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:15:52.858 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:15:52.858 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:15:52.859 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:15:52.859 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:15:52.859 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:15:52.859 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:15:52.859 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:15:52.860 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:15:52.860 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:15:52.860 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:15:52.860 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:15:52.861 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.854+0000 7f0ca969a700 1 -- 172.21.15.27:0/1165758407 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0c8c0640b0 msgr2=0x7f0c8c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:15:52.862 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.854+0000 7f0ca969a700 1 --2- 172.21.15.27:0/1165758407 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0c8c0640b0 0x7f0c8c066570 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f0c98009740 tx=0x7f0c98009be0).stop 2021-11-03T18:15:52.862 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.854+0000 7f0ca969a700 1 -- 172.21.15.27:0/1165758407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ca40f55e0 msgr2=0x7f0ca4105e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:15:52.862 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.854+0000 7f0ca969a700 1 --2- 172.21.15.27:0/1165758407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ca40f55e0 0x7f0ca4105e90 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 rx=0x7f0c94009990 tx=0x7f0c94009b30).stop 2021-11-03T18:15:52.862 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.854+0000 7f0ca969a700 1 -- 172.21.15.27:0/1165758407 shutdown_connections 2021-11-03T18:15:52.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.854+0000 7f0ca969a700 1 --2- 172.21.15.27:0/1165758407 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0c8c0640b0 0x7f0c8c066570 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:52.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.854+0000 7f0ca969a700 1 --2- 172.21.15.27:0/1165758407 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ca40f55e0 0x7f0ca4105e90 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:52.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.854+0000 7f0ca969a700 1 --2- 172.21.15.27:0/1165758407 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ca40f5ef0 0x7f0ca4106380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:15:52.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.854+0000 7f0ca969a700 1 -- 172.21.15.27:0/1165758407 >> 172.21.15.27:0/1165758407 conn(0x7f0ca40f0de0 msgr2=0x7f0ca40fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:15:52.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.854+0000 7f0ca969a700 1 -- 172.21.15.27:0/1165758407 shutdown_connections 2021-11-03T18:15:52.864 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:15:52.854+0000 7f0ca969a700 1 -- 172.21.15.27:0/1165758407 wait complete. 2021-11-03T18:15:52.864 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:15:52.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:52 smithi027 ceph-mon[97244]: pgmap v1836: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:52.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:52 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3058514616' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:52.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:52 smithi115 ceph-mon[65453]: pgmap v1836: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:52.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:52 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3058514616' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:15:53.958 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:53 smithi115 ceph-mon[65453]: from='client.35992 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:15:53.959 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:53 smithi115 ceph-mon[65453]: from='client.35996 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:15:53.959 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:53 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1165758407' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:15:54.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:53 smithi027 ceph-mon[97244]: from='client.35992 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:15:54.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:53 smithi027 ceph-mon[97244]: from='client.35996 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:15:54.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:53 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1165758407' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:15:54.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:54 smithi115 ceph-mon[65453]: pgmap v1837: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:55.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:54 smithi027 ceph-mon[97244]: pgmap v1837: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:55.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3597.445945 secs 2021-11-03T18:15:56.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3597.445945 secs 2021-11-03T18:15:56.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:56 smithi115 ceph-mon[65453]: pgmap v1838: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:56.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:15:56.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:15:57.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:56 smithi027 ceph-mon[97244]: pgmap v1838: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:57.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:15:57.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:15:57.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:57 smithi115 ceph-mon[65453]: pgmap v1839: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:15:58.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:57 smithi027 ceph-mon[97244]: pgmap v1839: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:00.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:59 smithi115 ceph-mon[65453]: pgmap v1840: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:00.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:16:00.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:16:00.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:16:00.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:00.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:00.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:00.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:00.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:15:59 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:16:00.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:59 smithi027 ceph-mon[97244]: pgmap v1840: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:00.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:16:00.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:16:00.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:16:00.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:00.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:00.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:00.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:00.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:15:59 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:16:01.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3602.446084 secs 2021-11-03T18:16:01.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3602.446084 secs 2021-11-03T18:16:02.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:02 smithi115 ceph-mon[65453]: pgmap v1841: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:02.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:02 smithi027 ceph-mon[97244]: pgmap v1841: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:04.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:04 smithi115 ceph-mon[65453]: pgmap v1842: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:04.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:04 smithi027 ceph-mon[97244]: pgmap v1842: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:05.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3607.446211 secs 2021-11-03T18:16:05.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3607.446211 secs 2021-11-03T18:16:06.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:06 smithi115 ceph-mon[65453]: pgmap v1843: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:06.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:06 smithi027 ceph-mon[97244]: pgmap v1843: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:08.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:08 smithi115 ceph-mon[65453]: pgmap v1844: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:08.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:08 smithi027 ceph-mon[97244]: pgmap v1844: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:10.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:10 smithi115 ceph-mon[65453]: pgmap v1845: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:10.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3612.446353 secs 2021-11-03T18:16:10.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:16:10.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:10 smithi115 ceph-mon[65453]: poll took 60174.074 ms: 1 events 2021-11-03T18:16:10.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:16:10.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:16:10.490 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:10.490 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:10.491 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:10.491 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:10.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:10 smithi027 ceph-mon[97244]: pgmap v1845: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:10.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3612.446353 secs 2021-11-03T18:16:10.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:16:10.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:10 smithi027 ceph-mon[97244]: poll took 60174.074 ms: 1 events 2021-11-03T18:16:10.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:16:10.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:16:10.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:10.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:10.524 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:10.524 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:11.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:11 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:16:11.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:11 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:16:12.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:12 smithi027 ceph-mon[97244]: pgmap v1846: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:12.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:12 smithi115 ceph-mon[65453]: pgmap v1846: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:14.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:14 smithi027 ceph-mon[97244]: pgmap v1847: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:14.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:14 smithi115 ceph-mon[65453]: pgmap v1847: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:15.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3617.446496 secs 2021-11-03T18:16:15.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3617.446496 secs 2021-11-03T18:16:16.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:16 smithi027 ceph-mon[97244]: pgmap v1848: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:16.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:16 smithi115 ceph-mon[65453]: pgmap v1848: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:18.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:18 smithi027 ceph-mon[97244]: pgmap v1849: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:18.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:18 smithi115 ceph-mon[65453]: pgmap v1849: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:20.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:20 smithi115 ceph-mon[65453]: pgmap v1850: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:20.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3622.446588 secs 2021-11-03T18:16:20.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:16:20.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:16:20.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:16:20.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:16:20.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:16:20.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:20.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:20.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:20.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:20.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:20 smithi027 ceph-mon[97244]: pgmap v1850: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:20.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3622.446588 secs 2021-11-03T18:16:20.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:16:20.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:16:20.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:16:20.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:16:20.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:16:20.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:20.524 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:20.524 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:20.524 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:21.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:21 smithi027 ceph-mon[97244]: poll took 10023.782 ms: 1 events 2021-11-03T18:16:21.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:21 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:16:21.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:21 smithi115 ceph-mon[65453]: poll took 10023.782 ms: 1 events 2021-11-03T18:16:21.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:21 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:16:22.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:22 smithi027 ceph-mon[97244]: pgmap v1851: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:22.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:22 smithi115 ceph-mon[65453]: pgmap v1851: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:22.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:22.971+0000 7f080fbb8700 1 -- 172.21.15.27:0/1372455015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08080f98b0 msgr2=0x7f08080fbca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:16:22.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:22.971+0000 7f080fbb8700 1 --2- 172.21.15.27:0/1372455015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08080f98b0 0x7f08080fbca0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 rx=0x7f07f8004660 tx=0x7f07f8009b30).stop 2021-11-03T18:16:22.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:22.971+0000 7f080fbb8700 1 -- 172.21.15.27:0/1372455015 shutdown_connections 2021-11-03T18:16:22.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:22.971+0000 7f080fbb8700 1 --2- 172.21.15.27:0/1372455015 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08080f98b0 0x7f08080fbca0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:22.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:22.971+0000 7f080fbb8700 1 --2- 172.21.15.27:0/1372455015 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0808060310 0x7f08080f93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:22.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:22.971+0000 7f080fbb8700 1 -- 172.21.15.27:0/1372455015 >> 172.21.15.27:0/1372455015 conn(0x7f08080f0de0 msgr2=0x7f08080f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:16:22.973 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:22.972+0000 7f080fbb8700 1 -- 172.21.15.27:0/1372455015 shutdown_connections 2021-11-03T18:16:22.973 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:22.972+0000 7f080fbb8700 1 -- 172.21.15.27:0/1372455015 wait complete. 2021-11-03T18:16:22.973 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:22.972+0000 7f080fbb8700 1 Processor -- start 2021-11-03T18:16:22.973 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:22.972+0000 7f080fbb8700 1 -- start start 2021-11-03T18:16:22.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:22.972+0000 7f080fbb8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0808060310 0x7f0808106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:16:22.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:22.972+0000 7f080fbb8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08080f98b0 0x7f0808107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:16:22.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:22.972+0000 7f080fbb8700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0808107a10 con 0x7f0808060310 2021-11-03T18:16:22.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:22.972+0000 7f080fbb8700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0808107b50 con 0x7f08080f98b0 2021-11-03T18:16:22.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:22.972+0000 7f080d954700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0808060310 0x7f0808106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:16:22.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:22.972+0000 7f080d954700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0808060310 0x7f0808106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33916/0 (socket says 172.21.15.27:33916) 2021-11-03T18:16:22.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:22.972+0000 7f080d954700 1 -- 172.21.15.27:0/635770098 learned_addr learned my addr 172.21.15.27:0/635770098 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:16:22.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:22.973+0000 7f080d153700 1 --2- 172.21.15.27:0/635770098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08080f98b0 0x7f0808107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:16:22.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:22.973+0000 7f080d954700 1 -- 172.21.15.27:0/635770098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08080f98b0 msgr2=0x7f0808107460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:16:22.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:22.973+0000 7f080d954700 1 --2- 172.21.15.27:0/635770098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08080f98b0 0x7f0808107460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:22.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:22.973+0000 7f080d954700 1 -- 172.21.15.27:0/635770098 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07f8005040 con 0x7f0808060310 2021-11-03T18:16:22.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:22.973+0000 7f080d153700 1 --2- 172.21.15.27:0/635770098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08080f98b0 0x7f0808107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T18:16:22.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:22.973+0000 7f080d954700 1 --2- 172.21.15.27:0/635770098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0808060310 0x7f0808106f70 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 rx=0x7f080400c790 tx=0x7f0804007d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:16:22.977 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:22.973+0000 7f07fe7fc700 1 -- 172.21.15.27:0/635770098 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f080400e680 con 0x7f0808060310 2021-11-03T18:16:22.977 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:22.973+0000 7f07fe7fc700 1 -- 172.21.15.27:0/635770098 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f080400ec90 con 0x7f0808060310 2021-11-03T18:16:22.977 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:22.973+0000 7f07fe7fc700 1 -- 172.21.15.27:0/635770098 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0804012d00 con 0x7f0808060310 2021-11-03T18:16:22.977 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:22.973+0000 7f080fbb8700 1 -- 172.21.15.27:0/635770098 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08081084f0 con 0x7f0808060310 2021-11-03T18:16:22.977 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:22.973+0000 7f080fbb8700 1 -- 172.21.15.27:0/635770098 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0808108ab0 con 0x7f0808060310 2021-11-03T18:16:22.979 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:22.974+0000 7f080fbb8700 1 -- 172.21.15.27:0/635770098 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0808045ad0 con 0x7f0808060310 2021-11-03T18:16:22.979 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:22.975+0000 7f07fe7fc700 1 -- 172.21.15.27:0/635770098 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f0804010070 con 0x7f0808060310 2021-11-03T18:16:22.979 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:22.975+0000 7f07fe7fc700 1 --2- 172.21.15.27:0/635770098 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f07f40640b0 0x7f07f4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:16:22.979 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:22.975+0000 7f07fe7fc700 1 -- 172.21.15.27:0/635770098 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f0804080940 con 0x7f0808060310 2021-11-03T18:16:22.980 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:22.975+0000 7f080d153700 1 --2- 172.21.15.27:0/635770098 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f07f40640b0 0x7f07f4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:16:22.980 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:22.976+0000 7f080d153700 1 --2- 172.21.15.27:0/635770098 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f07f40640b0 0x7f07f4066570 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f07f8009da0 tx=0x7f07f8009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:16:22.980 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:22.979+0000 7f07fe7fc700 1 -- 172.21.15.27:0/635770098 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f0804052af0 con 0x7f0808060310 2021-11-03T18:16:23.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.157+0000 7f080fbb8700 1 -- 172.21.15.27:0/635770098 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f08080ff070 con 0x7f07f40640b0 2021-11-03T18:16:23.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.159+0000 7f07fe7fc700 1 -- 172.21.15.27:0/635770098 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f08080ff070 con 0x7f07f40640b0 2021-11-03T18:16:23.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.160+0000 7f080fbb8700 1 -- 172.21.15.27:0/635770098 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f07f40640b0 msgr2=0x7f07f4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:16:23.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.160+0000 7f080fbb8700 1 --2- 172.21.15.27:0/635770098 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f07f40640b0 0x7f07f4066570 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 rx=0x7f07f8009da0 tx=0x7f07f8009b30).stop 2021-11-03T18:16:23.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.160+0000 7f080fbb8700 1 -- 172.21.15.27:0/635770098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0808060310 msgr2=0x7f0808106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:16:23.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.160+0000 7f080fbb8700 1 --2- 172.21.15.27:0/635770098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0808060310 0x7f0808106f70 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 rx=0x7f080400c790 tx=0x7f0804007d90).stop 2021-11-03T18:16:23.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.160+0000 7f080fbb8700 1 -- 172.21.15.27:0/635770098 shutdown_connections 2021-11-03T18:16:23.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.160+0000 7f080fbb8700 1 --2- 172.21.15.27:0/635770098 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f07f40640b0 0x7f07f4066570 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:23.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.160+0000 7f080fbb8700 1 --2- 172.21.15.27:0/635770098 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0808060310 0x7f0808106f70 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:23.160 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.160+0000 7f080fbb8700 1 --2- 172.21.15.27:0/635770098 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08080f98b0 0x7f0808107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:23.160 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.160+0000 7f080fbb8700 1 -- 172.21.15.27:0/635770098 >> 172.21.15.27:0/635770098 conn(0x7f08080f0de0 msgr2=0x7f08080f31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:16:23.160 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.160+0000 7f080fbb8700 1 -- 172.21.15.27:0/635770098 shutdown_connections 2021-11-03T18:16:23.160 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.160+0000 7f080fbb8700 1 -- 172.21.15.27:0/635770098 wait complete. 2021-11-03T18:16:23.167 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:16:23.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.261+0000 7f329a35e700 1 -- 172.21.15.27:0/610102179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32940f6ff0 msgr2=0x7f32940f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:16:23.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.261+0000 7f329a35e700 1 --2- 172.21.15.27:0/610102179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32940f6ff0 0x7f32940f7a20 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 rx=0x7f3288004660 tx=0x7f3288009b30).stop 2021-11-03T18:16:23.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.261+0000 7f329a35e700 1 -- 172.21.15.27:0/610102179 shutdown_connections 2021-11-03T18:16:23.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.261+0000 7f329a35e700 1 --2- 172.21.15.27:0/610102179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32940f6ff0 0x7f32940f7a20 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:23.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.261+0000 7f329a35e700 1 --2- 172.21.15.27:0/610102179 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32940f66e0 0x7f32940f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:23.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.261+0000 7f329a35e700 1 -- 172.21.15.27:0/610102179 >> 172.21.15.27:0/610102179 conn(0x7f32940f1f00 msgr2=0x7f32940f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:16:23.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.261+0000 7f329a35e700 1 -- 172.21.15.27:0/610102179 shutdown_connections 2021-11-03T18:16:23.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.261+0000 7f329a35e700 1 -- 172.21.15.27:0/610102179 wait complete. 2021-11-03T18:16:23.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.261+0000 7f329a35e700 1 Processor -- start 2021-11-03T18:16:23.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.261+0000 7f329a35e700 1 -- start start 2021-11-03T18:16:23.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.262+0000 7f329a35e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32940f66e0 0x7f3294106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:16:23.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.262+0000 7f329a35e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32940f6ff0 0x7f3294107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:16:23.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.262+0000 7f329a35e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3294107a10 con 0x7f32940f66e0 2021-11-03T18:16:23.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.262+0000 7f329a35e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3294107b50 con 0x7f32940f6ff0 2021-11-03T18:16:23.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.262+0000 7f3293fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32940f66e0 0x7f3294106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:16:23.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.262+0000 7f3293fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32940f66e0 0x7f3294106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33926/0 (socket says 172.21.15.27:33926) 2021-11-03T18:16:23.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.262+0000 7f3293fff700 1 -- 172.21.15.27:0/4250157073 learned_addr learned my addr 172.21.15.27:0/4250157073 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:16:23.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.262+0000 7f32937fe700 1 --2- 172.21.15.27:0/4250157073 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32940f6ff0 0x7f3294107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:16:23.265 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.262+0000 7f3293fff700 1 -- 172.21.15.27:0/4250157073 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32940f6ff0 msgr2=0x7f3294107460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:16:23.265 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.262+0000 7f3293fff700 1 --2- 172.21.15.27:0/4250157073 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32940f6ff0 0x7f3294107460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:23.265 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.262+0000 7f3293fff700 1 -- 172.21.15.27:0/4250157073 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3288005040 con 0x7f32940f66e0 2021-11-03T18:16:23.265 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.262+0000 7f3293fff700 1 --2- 172.21.15.27:0/4250157073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32940f66e0 0x7f3294106f70 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 rx=0x7f328400c790 tx=0x7f3284007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:16:23.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.262+0000 7f3290ff9700 1 -- 172.21.15.27:0/4250157073 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f328400e680 con 0x7f32940f66e0 2021-11-03T18:16:23.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.262+0000 7f3290ff9700 1 -- 172.21.15.27:0/4250157073 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f328400ec90 con 0x7f32940f66e0 2021-11-03T18:16:23.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.262+0000 7f329a35e700 1 -- 172.21.15.27:0/4250157073 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32941084f0 con 0x7f32940f66e0 2021-11-03T18:16:23.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.262+0000 7f329a35e700 1 -- 172.21.15.27:0/4250157073 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3294108ab0 con 0x7f32940f66e0 2021-11-03T18:16:23.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.262+0000 7f3290ff9700 1 -- 172.21.15.27:0/4250157073 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3284013d60 con 0x7f32940f66e0 2021-11-03T18:16:23.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.263+0000 7f329a35e700 1 -- 172.21.15.27:0/4250157073 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3294045ad0 con 0x7f32940f66e0 2021-11-03T18:16:23.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.264+0000 7f3290ff9700 1 -- 172.21.15.27:0/4250157073 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f328401a060 con 0x7f32940f66e0 2021-11-03T18:16:23.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.264+0000 7f3290ff9700 1 --2- 172.21.15.27:0/4250157073 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f327c0640b0 0x7f327c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:16:23.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.264+0000 7f3290ff9700 1 -- 172.21.15.27:0/4250157073 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f3284080b90 con 0x7f32940f66e0 2021-11-03T18:16:23.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.264+0000 7f32937fe700 1 --2- 172.21.15.27:0/4250157073 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f327c0640b0 0x7f327c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:16:23.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.265+0000 7f32937fe700 1 --2- 172.21.15.27:0/4250157073 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f327c0640b0 0x7f327c066570 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f3288009da0 tx=0x7f3288009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:16:23.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.268+0000 7f3290ff9700 1 -- 172.21.15.27:0/4250157073 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f3284052da0 con 0x7f32940f66e0 2021-11-03T18:16:23.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.439+0000 7f329a35e700 1 -- 172.21.15.27:0/4250157073 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f32940fefe0 con 0x7f327c0640b0 2021-11-03T18:16:23.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.454+0000 7f3290ff9700 1 -- 172.21.15.27:0/4250157073 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f32940fefe0 con 0x7f327c0640b0 2021-11-03T18:16:23.452 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:16:23.452 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (67m) 7m ago 69m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:16:23.453 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (59m) 7m ago 69m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:16:23.453 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (59m) 6m ago 68m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:16:23.453 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (67m) 7m ago 68m 42.1M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:16:23.453 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (61m) 7m ago 64m 26.3M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:16:23.454 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (64m) 7m ago 64m 3847M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:16:23.454 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (60m) 6m ago 64m 25.9M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:16:23.454 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (61m) 6m ago 64m 27.0M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:16:23.454 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (61m) 7m ago 70m 572M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:16:23.455 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (60m) 6m ago 68m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:16:23.455 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (60m) 7m ago 70m 273M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:16:23.455 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (60m) 6m ago 68m 129M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:16:23.455 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (68m) 7m ago 68m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:16:23.455 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (68m) 6m ago 68m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:16:23.456 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (59m) 7m ago 67m 371M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:16:23.456 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (58m) 7m ago 67m 405M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:16:23.456 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (58m) 7m ago 66m 302M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:16:23.456 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (57m) 6m ago 66m 429M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:16:23.457 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (57m) 6m ago 66m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:16:23.457 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (56m) 6m ago 65m 348M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:16:23.457 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (67m) 7m ago 68m 57.6M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:16:23.458 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.455+0000 7f329a35e700 1 -- 172.21.15.27:0/4250157073 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f327c0640b0 msgr2=0x7f327c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:16:23.458 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.456+0000 7f329a35e700 1 --2- 172.21.15.27:0/4250157073 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f327c0640b0 0x7f327c066570 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7f3288009da0 tx=0x7f3288009b30).stop 2021-11-03T18:16:23.458 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.456+0000 7f329a35e700 1 -- 172.21.15.27:0/4250157073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32940f66e0 msgr2=0x7f3294106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:16:23.459 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.456+0000 7f329a35e700 1 --2- 172.21.15.27:0/4250157073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32940f66e0 0x7f3294106f70 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 rx=0x7f328400c790 tx=0x7f3284007d70).stop 2021-11-03T18:16:23.459 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.456+0000 7f329a35e700 1 -- 172.21.15.27:0/4250157073 shutdown_connections 2021-11-03T18:16:23.459 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.456+0000 7f329a35e700 1 --2- 172.21.15.27:0/4250157073 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f327c0640b0 0x7f327c066570 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:23.459 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.456+0000 7f329a35e700 1 --2- 172.21.15.27:0/4250157073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32940f66e0 0x7f3294106f70 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:23.459 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.456+0000 7f329a35e700 1 --2- 172.21.15.27:0/4250157073 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32940f6ff0 0x7f3294107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:23.460 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.456+0000 7f329a35e700 1 -- 172.21.15.27:0/4250157073 >> 172.21.15.27:0/4250157073 conn(0x7f32940f1f00 msgr2=0x7f32940fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:16:23.460 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.456+0000 7f329a35e700 1 -- 172.21.15.27:0/4250157073 shutdown_connections 2021-11-03T18:16:23.460 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.456+0000 7f329a35e700 1 -- 172.21.15.27:0/4250157073 wait complete. 2021-11-03T18:16:23.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.555+0000 7f2a7a0fc700 1 -- 172.21.15.27:0/4053521769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a740f53e0 msgr2=0x7f2a740f7d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:16:23.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.555+0000 7f2a7a0fc700 1 --2- 172.21.15.27:0/4053521769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a740f53e0 0x7f2a740f7d20 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 rx=0x7f2a68004660 tx=0x7f2a68009b30).stop 2021-11-03T18:16:23.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.555+0000 7f2a7a0fc700 1 -- 172.21.15.27:0/4053521769 shutdown_connections 2021-11-03T18:16:23.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.555+0000 7f2a7a0fc700 1 --2- 172.21.15.27:0/4053521769 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a740f53e0 0x7f2a740f7d20 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:23.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.555+0000 7f2a7a0fc700 1 --2- 172.21.15.27:0/4053521769 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a740f4a40 0x7f2a740f4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:23.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.555+0000 7f2a7a0fc700 1 -- 172.21.15.27:0/4053521769 >> 172.21.15.27:0/4053521769 conn(0x7f2a740f04c0 msgr2=0x7f2a740f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:16:23.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.555+0000 7f2a7a0fc700 1 -- 172.21.15.27:0/4053521769 shutdown_connections 2021-11-03T18:16:23.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.555+0000 7f2a7a0fc700 1 -- 172.21.15.27:0/4053521769 wait complete. 2021-11-03T18:16:23.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.556+0000 7f2a7a0fc700 1 Processor -- start 2021-11-03T18:16:23.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.556+0000 7f2a7a0fc700 1 -- start start 2021-11-03T18:16:23.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.556+0000 7f2a7a0fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a740f4a40 0x7f2a741071a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:16:23.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.556+0000 7f2a7a0fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a740f53e0 0x7f2a74107690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:16:23.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.556+0000 7f2a7a0fc700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a74107c40 con 0x7f2a740f4a40 2021-11-03T18:16:23.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.556+0000 7f2a7a0fc700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a74107d80 con 0x7f2a740f53e0 2021-11-03T18:16:23.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.556+0000 7f2a737fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a740f4a40 0x7f2a741071a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:16:23.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.556+0000 7f2a737fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a740f4a40 0x7f2a741071a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33936/0 (socket says 172.21.15.27:33936) 2021-11-03T18:16:23.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.556+0000 7f2a737fe700 1 -- 172.21.15.27:0/2721731007 learned_addr learned my addr 172.21.15.27:0/2721731007 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:16:23.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.556+0000 7f2a72ffd700 1 --2- 172.21.15.27:0/2721731007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a740f53e0 0x7f2a74107690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:16:23.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.557+0000 7f2a737fe700 1 -- 172.21.15.27:0/2721731007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a740f53e0 msgr2=0x7f2a74107690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:16:23.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.557+0000 7f2a737fe700 1 --2- 172.21.15.27:0/2721731007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a740f53e0 0x7f2a74107690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:23.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.557+0000 7f2a737fe700 1 -- 172.21.15.27:0/2721731007 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a68005040 con 0x7f2a740f4a40 2021-11-03T18:16:23.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.557+0000 7f2a72ffd700 1 --2- 172.21.15.27:0/2721731007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a740f53e0 0x7f2a74107690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:16:23.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.557+0000 7f2a737fe700 1 --2- 172.21.15.27:0/2721731007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a740f4a40 0x7f2a741071a0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 rx=0x7f2a640026d0 tx=0x7f2a64009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:16:23.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.557+0000 7f2a5bfff700 1 -- 172.21.15.27:0/2721731007 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a64009320 con 0x7f2a740f4a40 2021-11-03T18:16:23.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.557+0000 7f2a5bfff700 1 -- 172.21.15.27:0/2721731007 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f2a6400f040 con 0x7f2a740f4a40 2021-11-03T18:16:23.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.557+0000 7f2a7a0fc700 1 -- 172.21.15.27:0/2721731007 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a74108720 con 0x7f2a740f4a40 2021-11-03T18:16:23.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.557+0000 7f2a5bfff700 1 -- 172.21.15.27:0/2721731007 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a640147e0 con 0x7f2a740f4a40 2021-11-03T18:16:23.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.557+0000 7f2a7a0fc700 1 -- 172.21.15.27:0/2721731007 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a74108ce0 con 0x7f2a740f4a40 2021-11-03T18:16:23.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.558+0000 7f2a7a0fc700 1 -- 172.21.15.27:0/2721731007 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a74045ad0 con 0x7f2a740f4a40 2021-11-03T18:16:23.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.558+0000 7f2a5bfff700 1 -- 172.21.15.27:0/2721731007 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f2a64009480 con 0x7f2a740f4a40 2021-11-03T18:16:23.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.559+0000 7f2a5bfff700 1 --2- 172.21.15.27:0/2721731007 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2a5c0640b0 0x7f2a5c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:16:23.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.559+0000 7f2a5bfff700 1 -- 172.21.15.27:0/2721731007 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f2a64080e50 con 0x7f2a740f4a40 2021-11-03T18:16:23.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.559+0000 7f2a72ffd700 1 --2- 172.21.15.27:0/2721731007 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2a5c0640b0 0x7f2a5c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:16:23.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.559+0000 7f2a72ffd700 1 --2- 172.21.15.27:0/2721731007 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2a5c0640b0 0x7f2a5c066570 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f2a68005010 tx=0x7f2a68009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:16:23.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.563+0000 7f2a5bfff700 1 -- 172.21.15.27:0/2721731007 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f2a64053210 con 0x7f2a740f4a40 2021-11-03T18:16:23.792 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.794+0000 7f2a7a0fc700 1 -- 172.21.15.27:0/2721731007 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f2a740ff140 con 0x7f2a740f4a40 2021-11-03T18:16:23.793 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.795+0000 7f2a5bfff700 1 -- 172.21.15.27:0/2721731007 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f2a64052e00 con 0x7f2a740f4a40 2021-11-03T18:16:23.793 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:16:23.793 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:16:23.794 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:16:23.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:16:23.794 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:16:23.794 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:16:23.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:16:23.796 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:16:23.796 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:16:23.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:16:23.796 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:16:23.796 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:16:23.797 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:16:23.798 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:16:23.798 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:16:23.798 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:16:23.798 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:16:23.799 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:16:23.800 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.797+0000 7f2a7a0fc700 1 -- 172.21.15.27:0/2721731007 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2a5c0640b0 msgr2=0x7f2a5c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:16:23.800 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.797+0000 7f2a7a0fc700 1 --2- 172.21.15.27:0/2721731007 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2a5c0640b0 0x7f2a5c066570 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 rx=0x7f2a68005010 tx=0x7f2a68009b30).stop 2021-11-03T18:16:23.800 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.797+0000 7f2a7a0fc700 1 -- 172.21.15.27:0/2721731007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a740f4a40 msgr2=0x7f2a741071a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:16:23.800 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.797+0000 7f2a7a0fc700 1 --2- 172.21.15.27:0/2721731007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a740f4a40 0x7f2a741071a0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 rx=0x7f2a640026d0 tx=0x7f2a64009de0).stop 2021-11-03T18:16:23.801 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.797+0000 7f2a7a0fc700 1 -- 172.21.15.27:0/2721731007 shutdown_connections 2021-11-03T18:16:23.801 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.797+0000 7f2a7a0fc700 1 --2- 172.21.15.27:0/2721731007 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2a5c0640b0 0x7f2a5c066570 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:23.801 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.797+0000 7f2a7a0fc700 1 --2- 172.21.15.27:0/2721731007 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a740f4a40 0x7f2a741071a0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:23.801 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.797+0000 7f2a7a0fc700 1 --2- 172.21.15.27:0/2721731007 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a740f53e0 0x7f2a74107690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:23.801 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.797+0000 7f2a7a0fc700 1 -- 172.21.15.27:0/2721731007 >> 172.21.15.27:0/2721731007 conn(0x7f2a740f04c0 msgr2=0x7f2a740fb770 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:16:23.802 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.797+0000 7f2a7a0fc700 1 -- 172.21.15.27:0/2721731007 shutdown_connections 2021-11-03T18:16:23.802 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.797+0000 7f2a7a0fc700 1 -- 172.21.15.27:0/2721731007 wait complete. 2021-11-03T18:16:23.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.897+0000 7f757d00e700 1 -- 172.21.15.27:0/981788959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75780f77d0 msgr2=0x7f75780f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:16:23.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.897+0000 7f757d00e700 1 --2- 172.21.15.27:0/981788959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75780f77d0 0x7f75780f7c30 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 rx=0x7f756c004660 tx=0x7f756c009b30).stop 2021-11-03T18:16:23.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.897+0000 7f757d00e700 1 -- 172.21.15.27:0/981788959 shutdown_connections 2021-11-03T18:16:23.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.897+0000 7f757d00e700 1 --2- 172.21.15.27:0/981788959 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75780f77d0 0x7f75780f7c30 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:23.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.897+0000 7f757d00e700 1 --2- 172.21.15.27:0/981788959 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75780f6530 0x7f75780f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:23.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.897+0000 7f757d00e700 1 -- 172.21.15.27:0/981788959 >> 172.21.15.27:0/981788959 conn(0x7f75780f1cd0 msgr2=0x7f75780f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:16:23.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.897+0000 7f757d00e700 1 -- 172.21.15.27:0/981788959 shutdown_connections 2021-11-03T18:16:23.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.897+0000 7f757d00e700 1 -- 172.21.15.27:0/981788959 wait complete. 2021-11-03T18:16:23.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.898+0000 7f757d00e700 1 Processor -- start 2021-11-03T18:16:23.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.898+0000 7f757d00e700 1 -- start start 2021-11-03T18:16:23.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.898+0000 7f757d00e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75780f6530 0x7f7578106d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:16:23.900 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.898+0000 7f757d00e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75780f77d0 0x7f7578107210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:16:23.900 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.898+0000 7f757d00e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75781077c0 con 0x7f75780f6530 2021-11-03T18:16:23.900 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.898+0000 7f757d00e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7578107fc0 con 0x7f75780f77d0 2021-11-03T18:16:23.901 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.898+0000 7f7577fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75780f6530 0x7f7578106d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:16:23.901 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.898+0000 7f7577fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75780f6530 0x7f7578106d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33946/0 (socket says 172.21.15.27:33946) 2021-11-03T18:16:23.901 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.898+0000 7f7577fff700 1 -- 172.21.15.27:0/245343042 learned_addr learned my addr 172.21.15.27:0/245343042 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:16:23.901 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.898+0000 7f75777fe700 1 --2- 172.21.15.27:0/245343042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75780f77d0 0x7f7578107210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:16:23.902 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.899+0000 7f7577fff700 1 -- 172.21.15.27:0/245343042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75780f77d0 msgr2=0x7f7578107210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:16:23.902 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.899+0000 7f7577fff700 1 --2- 172.21.15.27:0/245343042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75780f77d0 0x7f7578107210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:23.902 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.899+0000 7f7577fff700 1 -- 172.21.15.27:0/245343042 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f756c005040 con 0x7f75780f6530 2021-11-03T18:16:23.903 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.899+0000 7f7577fff700 1 --2- 172.21.15.27:0/245343042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75780f6530 0x7f7578106d20 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 rx=0x7f756800e7c0 tx=0x7f756800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:16:23.903 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.899+0000 7f7574ff9700 1 -- 172.21.15.27:0/245343042 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75680092e0 con 0x7f75780f6530 2021-11-03T18:16:23.903 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.899+0000 7f7574ff9700 1 -- 172.21.15.27:0/245343042 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f756800f040 con 0x7f75780f6530 2021-11-03T18:16:23.903 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.899+0000 7f757d00e700 1 -- 172.21.15.27:0/245343042 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75781082a0 con 0x7f75780f6530 2021-11-03T18:16:23.904 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.899+0000 7f7574ff9700 1 -- 172.21.15.27:0/245343042 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75680147e0 con 0x7f75780f6530 2021-11-03T18:16:23.904 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.899+0000 7f757d00e700 1 -- 172.21.15.27:0/245343042 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7578108860 con 0x7f75780f6530 2021-11-03T18:16:23.905 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.900+0000 7f757d00e700 1 -- 172.21.15.27:0/245343042 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7578059100 con 0x7f75780f6530 2021-11-03T18:16:23.905 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.900+0000 7f7574ff9700 1 -- 172.21.15.27:0/245343042 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f756801b030 con 0x7f75780f6530 2021-11-03T18:16:23.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.901+0000 7f7574ff9700 1 --2- 172.21.15.27:0/245343042 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7560064100 0x7f75600665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:16:23.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.901+0000 7f7574ff9700 1 -- 172.21.15.27:0/245343042 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f7568080890 con 0x7f75780f6530 2021-11-03T18:16:23.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.901+0000 7f75777fe700 1 --2- 172.21.15.27:0/245343042 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7560064100 0x7f75600665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:16:23.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.901+0000 7f75777fe700 1 --2- 172.21.15.27:0/245343042 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7560064100 0x7f75600665c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f756c005010 tx=0x7f756c016040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:16:23.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:23.905+0000 7f7574ff9700 1 -- 172.21.15.27:0/245343042 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f7568052aa0 con 0x7f75780f6530 2021-11-03T18:16:24.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:24.107+0000 7f757d00e700 1 -- 172.21.15.27:0/245343042 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f75780008d0 con 0x7f75780f6530 2021-11-03T18:16:24.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:24.107+0000 7f7574ff9700 1 -- 172.21.15.27:0/245343042 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f7568019020 con 0x7f75780f6530 2021-11-03T18:16:24.106 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:16:24.106 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:16:24.107 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:16:24.107 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:16:24.107 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:16:24.107 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:16:24.108 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:16:24.108 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:16:24.108 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:16:24.108 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:16:24.108 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:16:24.109 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:16:24.109 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:16:24.109 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:16:24.110 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:16:24.110 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:16:24.110 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:16:24.110 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:16:24.111 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:16:24.111 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:16:24.111 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:16:24.111 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:16:24.111 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:16:24.112 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:16:24.112 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:16:24.112 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:16:24.112 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:16:24.113 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:16:24.113 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:16:24.113 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:16:24.113 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:16:24.113 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:16:24.114 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:16:24.114 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:16:24.114 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:16:24.114 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:16:24.115 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:16:24.115 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:16:24.115 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:16:24.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:24.109+0000 7f757d00e700 1 -- 172.21.15.27:0/245343042 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7560064100 msgr2=0x7f75600665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:16:24.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:24.109+0000 7f757d00e700 1 --2- 172.21.15.27:0/245343042 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7560064100 0x7f75600665c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f756c005010 tx=0x7f756c016040).stop 2021-11-03T18:16:24.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:24.109+0000 7f757d00e700 1 -- 172.21.15.27:0/245343042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75780f6530 msgr2=0x7f7578106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:16:24.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:24.109+0000 7f757d00e700 1 --2- 172.21.15.27:0/245343042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75780f6530 0x7f7578106d20 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 rx=0x7f756800e7c0 tx=0x7f756800b040).stop 2021-11-03T18:16:24.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:24.109+0000 7f757d00e700 1 -- 172.21.15.27:0/245343042 shutdown_connections 2021-11-03T18:16:24.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:24.109+0000 7f757d00e700 1 --2- 172.21.15.27:0/245343042 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7560064100 0x7f75600665c0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:24.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:24.109+0000 7f757d00e700 1 --2- 172.21.15.27:0/245343042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75780f6530 0x7f7578106d20 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:24.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:24.109+0000 7f757d00e700 1 --2- 172.21.15.27:0/245343042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75780f77d0 0x7f7578107210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:24.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:24.109+0000 7f757d00e700 1 -- 172.21.15.27:0/245343042 >> 172.21.15.27:0/245343042 conn(0x7f75780f1cd0 msgr2=0x7f75780fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:16:24.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:24.109+0000 7f757d00e700 1 -- 172.21.15.27:0/245343042 shutdown_connections 2021-11-03T18:16:24.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:24.109+0000 7f757d00e700 1 -- 172.21.15.27:0/245343042 wait complete. 2021-11-03T18:16:24.119 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:16:24.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:24 smithi027 ceph-mon[97244]: pgmap v1852: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:24.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:24 smithi027 ceph-mon[97244]: from='client.36008 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:16:24.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:24 smithi027 ceph-mon[97244]: from='client.36012 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:16:24.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:24 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2721731007' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:24.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:24 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/245343042' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:16:24.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:24 smithi115 ceph-mon[65453]: pgmap v1852: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:24.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:24 smithi115 ceph-mon[65453]: from='client.36008 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:16:24.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:24 smithi115 ceph-mon[65453]: from='client.36012 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:16:24.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:24 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2721731007' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:24.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:24 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/245343042' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:16:25.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3627.446734 secs 2021-11-03T18:16:25.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3627.446734 secs 2021-11-03T18:16:26.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:26 smithi115 ceph-mon[65453]: pgmap v1853: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:26.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:26 smithi027 ceph-mon[97244]: pgmap v1853: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:28.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:28 smithi115 ceph-mon[65453]: pgmap v1854: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:28.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:28 smithi027 ceph-mon[97244]: pgmap v1854: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:30.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:30 smithi115 ceph-mon[65453]: pgmap v1855: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:30.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3632.446860 secs 2021-11-03T18:16:30.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:16:30.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2021-11-03T18:16:30.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"}]: dispatch 2021-11-03T18:16:30.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:16:30.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:16:30.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:30.741 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:30.741 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:30.741 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:30.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:30 smithi027 ceph-mon[97244]: pgmap v1855: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:30.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3632.446860 secs 2021-11-03T18:16:30.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:16:30.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2021-11-03T18:16:30.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"}]: dispatch 2021-11-03T18:16:30.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:16:30.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:16:30.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:30.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:30.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:30.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:31.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:31 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:16:31.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:31 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:16:32.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:32 smithi115 ceph-mon[65453]: pgmap v1856: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:32.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:32 smithi027 ceph-mon[97244]: pgmap v1856: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:34.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:34 smithi115 ceph-mon[65453]: pgmap v1857: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:34.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:34 smithi027 ceph-mon[97244]: pgmap v1857: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:35.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3637.446987 secs 2021-11-03T18:16:35.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3637.446987 secs 2021-11-03T18:16:36.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:36 smithi115 ceph-mon[65453]: pgmap v1858: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:36.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:36 smithi027 ceph-mon[97244]: pgmap v1858: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:38.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:38 smithi115 ceph-mon[65453]: pgmap v1859: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:38.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:38 smithi027 ceph-mon[97244]: pgmap v1859: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:40.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:40 smithi115 ceph-mon[65453]: pgmap v1860: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:40.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3642.447118 secs 2021-11-03T18:16:40.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:16:40.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:16:40.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:16:40.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:40.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:40.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:40.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:40.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:40 smithi027 ceph-mon[97244]: pgmap v1860: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:40.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3642.447118 secs 2021-11-03T18:16:40.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:16:40.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:16:40.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:16:40.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:40.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:40.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:40.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:41.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:41 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:16:41.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:41 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:16:42.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:42 smithi115 ceph-mon[65453]: pgmap v1861: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:42.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:42 smithi027 ceph-mon[97244]: pgmap v1861: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:44.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:44 smithi115 ceph-mon[65453]: pgmap v1862: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:44.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:44 smithi027 ceph-mon[97244]: pgmap v1862: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:45.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3647.447249 secs 2021-11-03T18:16:45.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3647.447249 secs 2021-11-03T18:16:46.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:46 smithi115 ceph-mon[65453]: pgmap v1863: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:46.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:46 smithi027 ceph-mon[97244]: pgmap v1863: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:48.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:48 smithi115 ceph-mon[65453]: pgmap v1864: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:48.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:48 smithi027 ceph-mon[97244]: pgmap v1864: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:50.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:50 smithi115 ceph-mon[65453]: pgmap v1865: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:50.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3652.447391 secs 2021-11-03T18:16:50.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:16:50.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:16:50.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:16:50.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:50.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:50.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:50.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:50.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:50 smithi027 ceph-mon[97244]: pgmap v1865: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:50.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3652.447391 secs 2021-11-03T18:16:50.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:16:50.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:16:50.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:16:50.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:50.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:50.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:50.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:51.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:51 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:16:51.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:51 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:16:52.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:52 smithi115 ceph-mon[65453]: pgmap v1866: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:52.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:52 smithi027 ceph-mon[97244]: pgmap v1866: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:54.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.224+0000 7f84ab070700 1 -- 172.21.15.27:0/3590006360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84a40f96b0 msgr2=0x7f84a40fbb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:16:54.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.224+0000 7f84ab070700 1 --2- 172.21.15.27:0/3590006360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84a40f96b0 0x7f84a40fbb00 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 rx=0x7f8494004660 tx=0x7f8494009b30).stop 2021-11-03T18:16:54.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.224+0000 7f84ab070700 1 -- 172.21.15.27:0/3590006360 shutdown_connections 2021-11-03T18:16:54.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.224+0000 7f84ab070700 1 --2- 172.21.15.27:0/3590006360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84a40f96b0 0x7f84a40fbb00 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:54.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.224+0000 7f84ab070700 1 --2- 172.21.15.27:0/3590006360 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a4060270 0x7f84a40f91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:54.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.224+0000 7f84ab070700 1 -- 172.21.15.27:0/3590006360 >> 172.21.15.27:0/3590006360 conn(0x7f84a40f0ba0 msgr2=0x7f84a40f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:16:54.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.224+0000 7f84ab070700 1 -- 172.21.15.27:0/3590006360 shutdown_connections 2021-11-03T18:16:54.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.224+0000 7f84ab070700 1 -- 172.21.15.27:0/3590006360 wait complete. 2021-11-03T18:16:54.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.224+0000 7f84ab070700 1 Processor -- start 2021-11-03T18:16:54.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.225+0000 7f84ab070700 1 -- start start 2021-11-03T18:16:54.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.225+0000 7f84ab070700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84a4060270 0x7f84a4106db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:16:54.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.225+0000 7f84ab070700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a40f96b0 0x7f84a41072a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:16:54.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.225+0000 7f84ab070700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a41078a0 con 0x7f84a4060270 2021-11-03T18:16:54.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.225+0000 7f84ab070700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84a41080e0 con 0x7f84a40f96b0 2021-11-03T18:16:54.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.225+0000 7f84aa06e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84a4060270 0x7f84a4106db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:16:54.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.225+0000 7f84aa06e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84a4060270 0x7f84a4106db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33964/0 (socket says 172.21.15.27:33964) 2021-11-03T18:16:54.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.225+0000 7f84aa06e700 1 -- 172.21.15.27:0/3857710670 learned_addr learned my addr 172.21.15.27:0/3857710670 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:16:54.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.225+0000 7f84a986d700 1 --2- 172.21.15.27:0/3857710670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a40f96b0 0x7f84a41072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:16:54.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.225+0000 7f84aa06e700 1 -- 172.21.15.27:0/3857710670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a40f96b0 msgr2=0x7f84a41072a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:16:54.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.225+0000 7f84aa06e700 1 --2- 172.21.15.27:0/3857710670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a40f96b0 0x7f84a41072a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:54.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.225+0000 7f84aa06e700 1 -- 172.21.15.27:0/3857710670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8494005040 con 0x7f84a4060270 2021-11-03T18:16:54.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.225+0000 7f84aa06e700 1 --2- 172.21.15.27:0/3857710670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84a4060270 0x7f84a4106db0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 rx=0x7f84a00026d0 tx=0x7f84a0009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:16:54.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.226+0000 7f849affd700 1 -- 172.21.15.27:0/3857710670 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84a00079c0 con 0x7f84a4060270 2021-11-03T18:16:54.230 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.226+0000 7f84ab070700 1 -- 172.21.15.27:0/3857710670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84a41083c0 con 0x7f84a4060270 2021-11-03T18:16:54.230 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.226+0000 7f849affd700 1 -- 172.21.15.27:0/3857710670 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f84a000f040 con 0x7f84a4060270 2021-11-03T18:16:54.230 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.226+0000 7f849affd700 1 -- 172.21.15.27:0/3857710670 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f84a00148b0 con 0x7f84a4060270 2021-11-03T18:16:54.230 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.226+0000 7f84ab070700 1 -- 172.21.15.27:0/3857710670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84a4108900 con 0x7f84a4060270 2021-11-03T18:16:54.231 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.227+0000 7f84ab070700 1 -- 172.21.15.27:0/3857710670 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84a4059100 con 0x7f84a4060270 2021-11-03T18:16:54.231 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.227+0000 7f849affd700 1 -- 172.21.15.27:0/3857710670 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f84a0007b20 con 0x7f84a4060270 2021-11-03T18:16:54.231 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.227+0000 7f849affd700 1 --2- 172.21.15.27:0/3857710670 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8490064100 0x7f84900665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:16:54.231 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.228+0000 7f849affd700 1 -- 172.21.15.27:0/3857710670 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f84a00806c0 con 0x7f84a4060270 2021-11-03T18:16:54.232 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.228+0000 7f84a986d700 1 --2- 172.21.15.27:0/3857710670 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8490064100 0x7f84900665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:16:54.232 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.228+0000 7f84a986d700 1 --2- 172.21.15.27:0/3857710670 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8490064100 0x7f84900665c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f84a4106b40 tx=0x7f8494009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:16:54.232 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.232+0000 7f849affd700 1 -- 172.21.15.27:0/3857710670 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f84a00529c0 con 0x7f84a4060270 2021-11-03T18:16:54.407 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.409+0000 7f84ab070700 1 -- 172.21.15.27:0/3857710670 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f84a40fee80 con 0x7f8490064100 2021-11-03T18:16:54.408 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.411+0000 7f849affd700 1 -- 172.21.15.27:0/3857710670 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f84a40fee80 con 0x7f8490064100 2021-11-03T18:16:54.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.412+0000 7f84ab070700 1 -- 172.21.15.27:0/3857710670 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8490064100 msgr2=0x7f84900665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:16:54.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.412+0000 7f84ab070700 1 --2- 172.21.15.27:0/3857710670 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8490064100 0x7f84900665c0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 rx=0x7f84a4106b40 tx=0x7f8494009b30).stop 2021-11-03T18:16:54.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.412+0000 7f84ab070700 1 -- 172.21.15.27:0/3857710670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84a4060270 msgr2=0x7f84a4106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:16:54.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.412+0000 7f84ab070700 1 --2- 172.21.15.27:0/3857710670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84a4060270 0x7f84a4106db0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 rx=0x7f84a00026d0 tx=0x7f84a0009de0).stop 2021-11-03T18:16:54.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.412+0000 7f84ab070700 1 -- 172.21.15.27:0/3857710670 shutdown_connections 2021-11-03T18:16:54.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.412+0000 7f84ab070700 1 --2- 172.21.15.27:0/3857710670 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8490064100 0x7f84900665c0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:54.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.412+0000 7f84ab070700 1 --2- 172.21.15.27:0/3857710670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84a4060270 0x7f84a4106db0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:54.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.412+0000 7f84ab070700 1 --2- 172.21.15.27:0/3857710670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84a40f96b0 0x7f84a41072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:54.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.412+0000 7f84ab070700 1 -- 172.21.15.27:0/3857710670 >> 172.21.15.27:0/3857710670 conn(0x7f84a40f0ba0 msgr2=0x7f84a40f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:16:54.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.413+0000 7f84ab070700 1 -- 172.21.15.27:0/3857710670 shutdown_connections 2021-11-03T18:16:54.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.413+0000 7f84ab070700 1 -- 172.21.15.27:0/3857710670 wait complete. 2021-11-03T18:16:54.420 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:16:54.513 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.513+0000 7fb4acd87700 1 -- 172.21.15.27:0/3382122851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4a80f55c0 msgr2=0x7fb4a80f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:16:54.513 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.513+0000 7fb4acd87700 1 --2- 172.21.15.27:0/3382122851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4a80f55c0 0x7fb4a80f59e0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 rx=0x7fb498004660 tx=0x7fb498009b30).stop 2021-11-03T18:16:54.513 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.513+0000 7fb4acd87700 1 -- 172.21.15.27:0/3382122851 shutdown_connections 2021-11-03T18:16:54.514 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.513+0000 7fb4acd87700 1 --2- 172.21.15.27:0/3382122851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4a80f6860 0x7fb4a80f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:54.514 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.513+0000 7fb4acd87700 1 --2- 172.21.15.27:0/3382122851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4a80f55c0 0x7fb4a80f59e0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:54.514 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.513+0000 7fb4acd87700 1 -- 172.21.15.27:0/3382122851 >> 172.21.15.27:0/3382122851 conn(0x7fb4a80f0d40 msgr2=0x7fb4a80f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:16:54.514 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.513+0000 7fb4acd87700 1 -- 172.21.15.27:0/3382122851 shutdown_connections 2021-11-03T18:16:54.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.513+0000 7fb4acd87700 1 -- 172.21.15.27:0/3382122851 wait complete. 2021-11-03T18:16:54.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.513+0000 7fb4acd87700 1 Processor -- start 2021-11-03T18:16:54.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.513+0000 7fb4acd87700 1 -- start start 2021-11-03T18:16:54.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.514+0000 7fb4acd87700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4a80f55c0 0x7fb4a8109c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:16:54.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.514+0000 7fb4acd87700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4a80f6860 0x7fb4a810c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:16:54.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.514+0000 7fb4acd87700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a80603e0 con 0x7fb4a80f6860 2021-11-03T18:16:54.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.514+0000 7fb4acd87700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4a8060520 con 0x7fb4a80f55c0 2021-11-03T18:16:54.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.514+0000 7fb4a5d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4a80f6860 0x7fb4a810c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:16:54.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.514+0000 7fb4a5d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4a80f6860 0x7fb4a810c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33976/0 (socket says 172.21.15.27:33976) 2021-11-03T18:16:54.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.514+0000 7fb4a5d9b700 1 -- 172.21.15.27:0/571710378 learned_addr learned my addr 172.21.15.27:0/571710378 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:16:54.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.514+0000 7fb4a659c700 1 --2- 172.21.15.27:0/571710378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4a80f55c0 0x7fb4a8109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:16:54.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.514+0000 7fb4a5d9b700 1 -- 172.21.15.27:0/571710378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4a80f55c0 msgr2=0x7fb4a8109c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:16:54.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.514+0000 7fb4a5d9b700 1 --2- 172.21.15.27:0/571710378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4a80f55c0 0x7fb4a8109c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:54.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.514+0000 7fb4a5d9b700 1 -- 172.21.15.27:0/571710378 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb498005040 con 0x7fb4a80f6860 2021-11-03T18:16:54.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.514+0000 7fb4a5d9b700 1 --2- 172.21.15.27:0/571710378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4a80f6860 0x7fb4a810c170 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 rx=0x7fb49c0026d0 tx=0x7fb49c009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:16:54.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.514+0000 7fb496ffd700 1 -- 172.21.15.27:0/571710378 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb49c0079c0 con 0x7fb4a80f6860 2021-11-03T18:16:54.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.514+0000 7fb496ffd700 1 -- 172.21.15.27:0/571710378 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fb49c00f040 con 0x7fb4a80f6860 2021-11-03T18:16:54.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.514+0000 7fb4acd87700 1 -- 172.21.15.27:0/571710378 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4a810c720 con 0x7fb4a80f6860 2021-11-03T18:16:54.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.514+0000 7fb496ffd700 1 -- 172.21.15.27:0/571710378 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb49c0148b0 con 0x7fb4a80f6860 2021-11-03T18:16:54.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.514+0000 7fb4acd87700 1 -- 172.21.15.27:0/571710378 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4a810cce0 con 0x7fb4a80f6860 2021-11-03T18:16:54.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.515+0000 7fb4acd87700 1 -- 172.21.15.27:0/571710378 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4a8045ad0 con 0x7fb4a80f6860 2021-11-03T18:16:54.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.516+0000 7fb496ffd700 1 -- 172.21.15.27:0/571710378 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fb49c007b20 con 0x7fb4a80f6860 2021-11-03T18:16:54.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.516+0000 7fb496ffd700 1 --2- 172.21.15.27:0/571710378 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb490064090 0x7fb490066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:16:54.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.516+0000 7fb496ffd700 1 -- 172.21.15.27:0/571710378 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fb49c0807a0 con 0x7fb4a80f6860 2021-11-03T18:16:54.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.516+0000 7fb4a659c700 1 --2- 172.21.15.27:0/571710378 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb490064090 0x7fb490066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:16:54.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.517+0000 7fb4a659c700 1 --2- 172.21.15.27:0/571710378 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb490064090 0x7fb490066550 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7fb4a8060060 tx=0x7fb498009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:16:54.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.520+0000 7fb496ffd700 1 -- 172.21.15.27:0/571710378 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fb49c052aa0 con 0x7fb4a80f6860 2021-11-03T18:16:54.690 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.693+0000 7fb4acd87700 1 -- 172.21.15.27:0/571710378 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb4a80fdec0 con 0x7fb490064090 2021-11-03T18:16:54.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.707+0000 7fb496ffd700 1 -- 172.21.15.27:0/571710378 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fb4a80fdec0 con 0x7fb490064090 2021-11-03T18:16:54.705 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:16:54.706 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (68m) 7m ago 69m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:16:54.706 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (60m) 7m ago 69m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:16:54.706 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (60m) 6m ago 68m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:16:54.706 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (68m) 7m ago 69m 42.1M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:16:54.707 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (61m) 7m ago 64m 26.3M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:16:54.707 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (64m) 7m ago 64m 3847M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:16:54.707 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (60m) 6m ago 64m 25.9M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:16:54.707 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (62m) 6m ago 64m 27.0M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:16:54.708 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (62m) 7m ago 71m 572M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:16:54.709 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (61m) 6m ago 68m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:16:54.709 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (61m) 7m ago 71m 273M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:16:54.709 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (60m) 6m ago 68m 129M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:16:54.709 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (69m) 7m ago 69m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:16:54.710 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (68m) 6m ago 68m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:16:54.710 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (59m) 7m ago 68m 371M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:16:54.710 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (59m) 7m ago 67m 405M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:16:54.710 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (58m) 7m ago 67m 302M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:16:54.711 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (58m) 6m ago 67m 429M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:16:54.711 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (57m) 6m ago 66m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:16:54.711 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (57m) 6m ago 66m 348M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:16:54.711 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (68m) 7m ago 69m 57.6M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:16:54.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.709+0000 7fb4acd87700 1 -- 172.21.15.27:0/571710378 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb490064090 msgr2=0x7fb490066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:16:54.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.709+0000 7fb4acd87700 1 --2- 172.21.15.27:0/571710378 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb490064090 0x7fb490066550 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7fb4a8060060 tx=0x7fb498009b30).stop 2021-11-03T18:16:54.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.709+0000 7fb4acd87700 1 -- 172.21.15.27:0/571710378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4a80f6860 msgr2=0x7fb4a810c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:16:54.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.709+0000 7fb4acd87700 1 --2- 172.21.15.27:0/571710378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4a80f6860 0x7fb4a810c170 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 rx=0x7fb49c0026d0 tx=0x7fb49c009de0).stop 2021-11-03T18:16:54.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.709+0000 7fb4acd87700 1 -- 172.21.15.27:0/571710378 shutdown_connections 2021-11-03T18:16:54.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.709+0000 7fb4acd87700 1 --2- 172.21.15.27:0/571710378 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb490064090 0x7fb490066550 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:54.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.709+0000 7fb4acd87700 1 --2- 172.21.15.27:0/571710378 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb4a80f55c0 0x7fb4a8109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:54.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.709+0000 7fb4acd87700 1 --2- 172.21.15.27:0/571710378 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb4a80f6860 0x7fb4a810c170 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:54.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.709+0000 7fb4acd87700 1 -- 172.21.15.27:0/571710378 >> 172.21.15.27:0/571710378 conn(0x7fb4a80f0d40 msgr2=0x7fb4a80fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:16:54.715 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.709+0000 7fb4acd87700 1 -- 172.21.15.27:0/571710378 shutdown_connections 2021-11-03T18:16:54.715 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.709+0000 7fb4acd87700 1 -- 172.21.15.27:0/571710378 wait complete. 2021-11-03T18:16:54.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:54 smithi115 ceph-mon[65453]: pgmap v1867: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:54.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:54 smithi027 ceph-mon[97244]: pgmap v1867: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:54.808 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.808+0000 7f1c0ade9700 1 -- 172.21.15.27:0/3278802136 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c040f6860 msgr2=0x7f1c040f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:16:54.808 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.808+0000 7f1c0ade9700 1 --2- 172.21.15.27:0/3278802136 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c040f6860 0x7f1c040f6cc0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 rx=0x7f1bf0004660 tx=0x7f1bf0009b30).stop 2021-11-03T18:16:54.808 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.808+0000 7f1c0ade9700 1 -- 172.21.15.27:0/3278802136 shutdown_connections 2021-11-03T18:16:54.808 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.808+0000 7f1c0ade9700 1 --2- 172.21.15.27:0/3278802136 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c040f6860 0x7f1c040f6cc0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:54.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.808+0000 7f1c0ade9700 1 --2- 172.21.15.27:0/3278802136 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c040f55c0 0x7f1c040f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:54.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.808+0000 7f1c0ade9700 1 -- 172.21.15.27:0/3278802136 >> 172.21.15.27:0/3278802136 conn(0x7f1c040f0d40 msgr2=0x7f1c040f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:16:54.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.808+0000 7f1c0ade9700 1 -- 172.21.15.27:0/3278802136 shutdown_connections 2021-11-03T18:16:54.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.808+0000 7f1c0ade9700 1 -- 172.21.15.27:0/3278802136 wait complete. 2021-11-03T18:16:54.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.809+0000 7f1c0ade9700 1 Processor -- start 2021-11-03T18:16:54.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.809+0000 7f1c0ade9700 1 -- start start 2021-11-03T18:16:54.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.809+0000 7f1c0ade9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c040f55c0 0x7f1c04109c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:16:54.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.809+0000 7f1c0ade9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c040f6860 0x7f1c0410c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:16:54.811 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.809+0000 7f1c0ade9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c040603c0 con 0x7f1c040f55c0 2021-11-03T18:16:54.811 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.809+0000 7f1c0ade9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c04060500 con 0x7f1c040f6860 2021-11-03T18:16:54.811 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.809+0000 7f1c08b85700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c040f55c0 0x7f1c04109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:16:54.811 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.809+0000 7f1c08b85700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c040f55c0 0x7f1c04109c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33984/0 (socket says 172.21.15.27:33984) 2021-11-03T18:16:54.812 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.809+0000 7f1c08b85700 1 -- 172.21.15.27:0/3978022819 learned_addr learned my addr 172.21.15.27:0/3978022819 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:16:54.812 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.809+0000 7f1c08b85700 1 -- 172.21.15.27:0/3978022819 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c040f6860 msgr2=0x7f1c0410c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:16:54.812 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.809+0000 7f1c08b85700 1 --2- 172.21.15.27:0/3978022819 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c040f6860 0x7f1c0410c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:16:54.813 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.809+0000 7f1c08b85700 1 -- 172.21.15.27:0/3978022819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bf0005040 con 0x7f1c040f55c0 2021-11-03T18:16:54.813 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.809+0000 7f1c08b85700 1 --2- 172.21.15.27:0/3978022819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c040f55c0 0x7f1c04109c70 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 rx=0x7f1bf800c530 tx=0x7f1bf8007e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:16:54.813 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.810+0000 7f1c017fa700 1 -- 172.21.15.27:0/3978022819 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bf800f650 con 0x7f1c040f55c0 2021-11-03T18:16:54.813 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.810+0000 7f1c017fa700 1 -- 172.21.15.27:0/3978022819 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f1bf800fc60 con 0x7f1c040f55c0 2021-11-03T18:16:54.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.810+0000 7f1c0ade9700 1 -- 172.21.15.27:0/3978022819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c0410c720 con 0x7f1c040f55c0 2021-11-03T18:16:54.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.810+0000 7f1c017fa700 1 -- 172.21.15.27:0/3978022819 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1bf8012d20 con 0x7f1c040f55c0 2021-11-03T18:16:54.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.810+0000 7f1c0ade9700 1 -- 172.21.15.27:0/3978022819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c0410ccb0 con 0x7f1c040f55c0 2021-11-03T18:16:54.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.811+0000 7f1c0ade9700 1 -- 172.21.15.27:0/3978022819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c040f49b0 con 0x7f1c040f55c0 2021-11-03T18:16:54.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.811+0000 7f1c017fa700 1 -- 172.21.15.27:0/3978022819 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f1bf8019030 con 0x7f1c040f55c0 2021-11-03T18:16:54.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.811+0000 7f1c017fa700 1 --2- 172.21.15.27:0/3978022819 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1bf40640b0 0x7f1bf4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:16:54.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.811+0000 7f1c017fa700 1 -- 172.21.15.27:0/3978022819 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f1bf80806b0 con 0x7f1c040f55c0 2021-11-03T18:16:54.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.812+0000 7f1c03fff700 1 --2- 172.21.15.27:0/3978022819 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1bf40640b0 0x7f1bf4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:16:54.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.812+0000 7f1c03fff700 1 --2- 172.21.15.27:0/3978022819 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1bf40640b0 0x7f1bf4066570 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7f1bf0004e30 tx=0x7f1bf0009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:16:54.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:54.815+0000 7f1c017fa700 1 -- 172.21.15.27:0/3978022819 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f1bf80529d0 con 0x7f1c040f55c0 2021-11-03T18:16:55.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.046+0000 7f1c0ade9700 1 -- 172.21.15.27:0/3978022819 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1c04045ad0 con 0x7f1c040f55c0 2021-11-03T18:16:55.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.047+0000 7f1c017fa700 1 -- 172.21.15.27:0/3978022819 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f1bf80525c0 con 0x7f1c040f55c0 2021-11-03T18:16:55.045 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:16:55.045 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:16:55.045 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:16:55.045 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:16:55.046 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:16:55.046 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:16:55.046 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:16:55.046 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:16:55.047 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:16:55.047 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:16:55.047 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:16:55.047 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:16:55.048 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:16:55.048 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:16:55.048 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:16:55.048 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:16:55.048 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:16:55.049 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:16:55.050 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.048+0000 7f1c0ade9700 1 -- 172.21.15.27:0/3978022819 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1bf40640b0 msgr2=0x7f1bf4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:16:55.050 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.048+0000 7f1c0ade9700 1 --2- 172.21.15.27:0/3978022819 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1bf40640b0 0x7f1bf4066570 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7f1bf0004e30 tx=0x7f1bf0009b30).stop 2021-11-03T18:16:55.050 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.048+0000 7f1c0ade9700 1 -- 172.21.15.27:0/3978022819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c040f55c0 msgr2=0x7f1c04109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:16:55.050 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.048+0000 7f1c0ade9700 1 --2- 172.21.15.27:0/3978022819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c040f55c0 0x7f1c04109c70 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 rx=0x7f1bf800c530 tx=0x7f1bf8007e10).stop 2021-11-03T18:16:55.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.048+0000 7f1c0ade9700 1 -- 172.21.15.27:0/3978022819 shutdown_connections 2021-11-03T18:16:55.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.048+0000 7f1c0ade9700 1 --2- 172.21.15.27:0/3978022819 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1bf40640b0 0x7f1bf4066570 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:55.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.048+0000 7f1c0ade9700 1 --2- 172.21.15.27:0/3978022819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1c040f55c0 0x7f1c04109c70 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:55.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.048+0000 7f1c0ade9700 1 --2- 172.21.15.27:0/3978022819 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1c040f6860 0x7f1c0410c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:16:55.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.048+0000 7f1c0ade9700 1 -- 172.21.15.27:0/3978022819 >> 172.21.15.27:0/3978022819 conn(0x7f1c040f0d40 msgr2=0x7f1c040fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:16:55.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.048+0000 7f1c0ade9700 1 -- 172.21.15.27:0/3978022819 shutdown_connections 2021-11-03T18:16:55.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.048+0000 7f1c0ade9700 1 -- 172.21.15.27:0/3978022819 wait complete. 2021-11-03T18:16:55.147 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.149+0000 7fa7aab45700 1 -- 172.21.15.27:0/2608779553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7a40f66e0 msgr2=0x7fa7a40f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:16:55.147 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.149+0000 7fa7aab45700 1 --2- 172.21.15.27:0/2608779553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7a40f66e0 0x7fa7a40f6b00 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 rx=0x7fa79c004660 tx=0x7fa79c009b30).stop 2021-11-03T18:16:55.147 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.149+0000 7fa7aab45700 1 -- 172.21.15.27:0/2608779553 shutdown_connections 2021-11-03T18:16:55.148 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.149+0000 7fa7aab45700 1 --2- 172.21.15.27:0/2608779553 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7a40f6ff0 0x7fa7a40f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:55.148 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.149+0000 7fa7aab45700 1 --2- 172.21.15.27:0/2608779553 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7a40f66e0 0x7fa7a40f6b00 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:55.148 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.149+0000 7fa7aab45700 1 -- 172.21.15.27:0/2608779553 >> 172.21.15.27:0/2608779553 conn(0x7fa7a40f1f00 msgr2=0x7fa7a40f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:16:55.148 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.149+0000 7fa7aab45700 1 -- 172.21.15.27:0/2608779553 shutdown_connections 2021-11-03T18:16:55.149 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.149+0000 7fa7aab45700 1 -- 172.21.15.27:0/2608779553 wait complete. 2021-11-03T18:16:55.149 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.149+0000 7fa7aab45700 1 Processor -- start 2021-11-03T18:16:55.149 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.149+0000 7fa7aab45700 1 -- start start 2021-11-03T18:16:55.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.150+0000 7fa7aab45700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7a40f66e0 0x7fa7a4106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:16:55.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.150+0000 7fa7aab45700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7a40f6ff0 0x7fa7a4107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:16:55.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.150+0000 7fa7aab45700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a4107a00 con 0x7fa7a40f6ff0 2021-11-03T18:16:55.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.150+0000 7fa7aab45700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7a4107b40 con 0x7fa7a40f66e0 2021-11-03T18:16:55.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.150+0000 7fa7a3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7a40f6ff0 0x7fa7a4107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:16:55.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.150+0000 7fa7a3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7a40f6ff0 0x7fa7a4107450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:33996/0 (socket says 172.21.15.27:33996) 2021-11-03T18:16:55.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.150+0000 7fa7a3fff700 1 -- 172.21.15.27:0/2366323516 learned_addr learned my addr 172.21.15.27:0/2366323516 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:16:55.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.150+0000 7fa7a88e1700 1 --2- 172.21.15.27:0/2366323516 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7a40f66e0 0x7fa7a4106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:16:55.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.150+0000 7fa7a3fff700 1 -- 172.21.15.27:0/2366323516 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7a40f66e0 msgr2=0x7fa7a4106f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:16:55.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.150+0000 7fa7a3fff700 1 --2- 172.21.15.27:0/2366323516 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7a40f66e0 0x7fa7a4106f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:55.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.150+0000 7fa7a3fff700 1 -- 172.21.15.27:0/2366323516 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa79c005040 con 0x7fa7a40f6ff0 2021-11-03T18:16:55.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.150+0000 7fa7a88e1700 1 --2- 172.21.15.27:0/2366323516 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7a40f66e0 0x7fa7a4106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T18:16:55.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.150+0000 7fa7a3fff700 1 --2- 172.21.15.27:0/2366323516 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7a40f6ff0 0x7fa7a4107450 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 rx=0x7fa79400c7c0 tx=0x7fa794004d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:16:55.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.150+0000 7fa7a17fa700 1 -- 172.21.15.27:0/2366323516 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa79400a6e0 con 0x7fa7a40f6ff0 2021-11-03T18:16:55.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.150+0000 7fa7a17fa700 1 -- 172.21.15.27:0/2366323516 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fa79400acf0 con 0x7fa7a40f6ff0 2021-11-03T18:16:55.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.150+0000 7fa7aab45700 1 -- 172.21.15.27:0/2366323516 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a41084e0 con 0x7fa7a40f6ff0 2021-11-03T18:16:55.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.150+0000 7fa7a17fa700 1 -- 172.21.15.27:0/2366323516 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa794012df0 con 0x7fa7a40f6ff0 2021-11-03T18:16:55.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.151+0000 7fa7aab45700 1 -- 172.21.15.27:0/2366323516 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7a4108aa0 con 0x7fa7a40f6ff0 2021-11-03T18:16:55.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.152+0000 7fa7aab45700 1 -- 172.21.15.27:0/2366323516 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7a4045ad0 con 0x7fa7a40f6ff0 2021-11-03T18:16:55.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.152+0000 7fa7a17fa700 1 -- 172.21.15.27:0/2366323516 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fa794019030 con 0x7fa7a40f6ff0 2021-11-03T18:16:55.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.152+0000 7fa7a17fa700 1 --2- 172.21.15.27:0/2366323516 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa790064090 0x7fa790066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:16:55.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.152+0000 7fa7a17fa700 1 -- 172.21.15.27:0/2366323516 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fa794080af0 con 0x7fa7a40f6ff0 2021-11-03T18:16:55.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.152+0000 7fa7a88e1700 1 --2- 172.21.15.27:0/2366323516 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa790064090 0x7fa790066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:16:55.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.153+0000 7fa7a88e1700 1 --2- 172.21.15.27:0/2366323516 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa790064090 0x7fa790066550 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7fa79c004630 tx=0x7fa79c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:16:55.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.156+0000 7fa7a17fa700 1 -- 172.21.15.27:0/2366323516 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fa794052db0 con 0x7fa7a40f6ff0 2021-11-03T18:16:55.362 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.364+0000 7fa7aab45700 1 -- 172.21.15.27:0/2366323516 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa7a40fef70 con 0x7fa7a40f6ff0 2021-11-03T18:16:55.363 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.365+0000 7fa7a17fa700 1 -- 172.21.15.27:0/2366323516 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7fa794017070 con 0x7fa7a40f6ff0 2021-11-03T18:16:55.363 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:16:55.363 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:16:55.364 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:16:55.364 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:16:55.364 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:16:55.364 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:16:55.365 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:16:55.365 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:16:55.365 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:16:55.365 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:16:55.365 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:16:55.366 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:16:55.366 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:16:55.366 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:16:55.366 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:16:55.367 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:16:55.367 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:16:55.367 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:16:55.367 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:16:55.367 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:16:55.368 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:16:55.368 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:16:55.368 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:16:55.368 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:16:55.369 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:16:55.369 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:16:55.369 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:16:55.369 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:16:55.370 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:16:55.370 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:16:55.370 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:16:55.370 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:16:55.370 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:16:55.371 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:16:55.371 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:16:55.371 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:16:55.371 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:16:55.372 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:16:55.372 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:16:55.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.367+0000 7fa7aab45700 1 -- 172.21.15.27:0/2366323516 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa790064090 msgr2=0x7fa790066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:16:55.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.367+0000 7fa7aab45700 1 --2- 172.21.15.27:0/2366323516 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa790064090 0x7fa790066550 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 rx=0x7fa79c004630 tx=0x7fa79c009b30).stop 2021-11-03T18:16:55.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.367+0000 7fa7aab45700 1 -- 172.21.15.27:0/2366323516 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7a40f6ff0 msgr2=0x7fa7a4107450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:16:55.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.367+0000 7fa7aab45700 1 --2- 172.21.15.27:0/2366323516 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7a40f6ff0 0x7fa7a4107450 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 rx=0x7fa79400c7c0 tx=0x7fa794004d70).stop 2021-11-03T18:16:55.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.367+0000 7fa7aab45700 1 -- 172.21.15.27:0/2366323516 shutdown_connections 2021-11-03T18:16:55.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.367+0000 7fa7aab45700 1 --2- 172.21.15.27:0/2366323516 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa790064090 0x7fa790066550 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:55.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.367+0000 7fa7aab45700 1 --2- 172.21.15.27:0/2366323516 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7a40f66e0 0x7fa7a4106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:55.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.367+0000 7fa7aab45700 1 --2- 172.21.15.27:0/2366323516 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7a40f6ff0 0x7fa7a4107450 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:16:55.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.367+0000 7fa7aab45700 1 -- 172.21.15.27:0/2366323516 >> 172.21.15.27:0/2366323516 conn(0x7fa7a40f1f00 msgr2=0x7fa7a40fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:16:55.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.367+0000 7fa7aab45700 1 -- 172.21.15.27:0/2366323516 shutdown_connections 2021-11-03T18:16:55.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:16:55.367+0000 7fa7aab45700 1 -- 172.21.15.27:0/2366323516 wait complete. 2021-11-03T18:16:55.376 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:16:55.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3657.447475 secs 2021-11-03T18:16:55.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:55 smithi115 ceph-mon[65453]: from='client.36024 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:16:55.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:55 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3978022819' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:55.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3657.447475 secs 2021-11-03T18:16:55.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:55 smithi027 ceph-mon[97244]: from='client.36024 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:16:55.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:55 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3978022819' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:16:56.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:56 smithi115 ceph-mon[65453]: from='client.36028 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:16:56.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:56 smithi115 ceph-mon[65453]: pgmap v1868: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:56.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:56 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2366323516' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:16:56.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:16:56.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:16:56.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:56 smithi027 ceph-mon[97244]: from='client.36028 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:16:56.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:56 smithi027 ceph-mon[97244]: pgmap v1868: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:56.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:56 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2366323516' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:16:56.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:16:56.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:16:58.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:16:58 smithi115 ceph-mon[65453]: pgmap v1869: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:16:58.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:16:58 smithi027 ceph-mon[97244]: pgmap v1869: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:00.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:00 smithi115 ceph-mon[65453]: pgmap v1870: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:00.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3662.447613 secs 2021-11-03T18:17:00.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:17:00.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:17:00.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:17:00.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:00.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:00.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:00.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:00.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:00 smithi027 ceph-mon[97244]: pgmap v1870: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:00.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3662.447613 secs 2021-11-03T18:17:00.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:17:00.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:17:00.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:17:00.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:00.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:00.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:00.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:01.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:01 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:17:01.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:01 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:17:02.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:02 smithi115 ceph-mon[65453]: pgmap v1871: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:02.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:02 smithi027 ceph-mon[97244]: pgmap v1871: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:04.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:04 smithi115 ceph-mon[65453]: pgmap v1872: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:04.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:04 smithi027 ceph-mon[97244]: pgmap v1872: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:05.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3667.447690 secs 2021-11-03T18:17:05.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3667.447690 secs 2021-11-03T18:17:06.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:06 smithi115 ceph-mon[65453]: pgmap v1873: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:06.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:06 smithi027 ceph-mon[97244]: pgmap v1873: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:08.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:08 smithi115 ceph-mon[65453]: pgmap v1874: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:08.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:08 smithi027 ceph-mon[97244]: pgmap v1874: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:10.667 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:10 smithi115 ceph-mon[65453]: pgmap v1875: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:10.667 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3672.447837 secs 2021-11-03T18:17:10.667 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:17:10.671 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:10 smithi027 ceph-mon[97244]: pgmap v1875: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:10.671 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3672.447837 secs 2021-11-03T18:17:10.672 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:17:11.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:11 smithi115 ceph-mon[65453]: poll took 50155.994 ms: 1 events 2021-11-03T18:17:11.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:17:11.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:17:11.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:11.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:11.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:11.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:11.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:11 smithi027 ceph-mon[97244]: poll took 50155.994 ms: 1 events 2021-11-03T18:17:11.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:17:11.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:17:11.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:11.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:11.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:11.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:12.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:12 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:17:12.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:12 smithi115 ceph-mon[65453]: pgmap v1876: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:12.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:12 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:17:12.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:12 smithi027 ceph-mon[97244]: pgmap v1876: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:14.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:14 smithi115 ceph-mon[65453]: pgmap v1877: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:14.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:14 smithi027 ceph-mon[97244]: pgmap v1877: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:15.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3677.447951 secs 2021-11-03T18:17:15.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3677.447951 secs 2021-11-03T18:17:16.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:16 smithi115 ceph-mon[65453]: pgmap v1878: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:16.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:16 smithi027 ceph-mon[97244]: pgmap v1878: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:18.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:18 smithi115 ceph-mon[65453]: pgmap v1879: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:18.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:18 smithi027 ceph-mon[97244]: pgmap v1879: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:20.729 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:20 smithi027 ceph-mon[97244]: pgmap v1880: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:20.729 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3682.448015 secs 2021-11-03T18:17:20.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:20 smithi115 ceph-mon[65453]: pgmap v1880: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:20.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3682.448015 secs 2021-11-03T18:17:21.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:17:21.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:17:21.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:17:21.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:17:21.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:21.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:21.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:21.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:21.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:17:21.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:17:21.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:17:21.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:17:21.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:21.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:21.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:21.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:22.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:22 smithi115 ceph-mon[65453]: poll took 10027.052 ms: 1 events 2021-11-03T18:17:22.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:22 smithi115 ceph-mon[65453]: pgmap v1881: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:22.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:22 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:17:22.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:22 smithi027 ceph-mon[97244]: poll took 10027.052 ms: 1 events 2021-11-03T18:17:22.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:22 smithi027 ceph-mon[97244]: pgmap v1881: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:22.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:22 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:17:24.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:24 smithi115 ceph-mon[65453]: pgmap v1882: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:24.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:24 smithi027 ceph-mon[97244]: pgmap v1882: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:25.482 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.483+0000 7fa8c4f67700 1 -- 172.21.15.27:0/209582872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8c00f66e0 msgr2=0x7fa8c00f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:17:25.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.483+0000 7fa8c4f67700 1 --2- 172.21.15.27:0/209582872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8c00f66e0 0x7fa8c00f6b40 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 rx=0x7fa8b4004660 tx=0x7fa8b4009b30).stop 2021-11-03T18:17:25.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.483+0000 7fa8c4f67700 1 -- 172.21.15.27:0/209582872 shutdown_connections 2021-11-03T18:17:25.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.483+0000 7fa8c4f67700 1 --2- 172.21.15.27:0/209582872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8c00f66e0 0x7fa8c00f6b40 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:25.484 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.483+0000 7fa8c4f67700 1 --2- 172.21.15.27:0/209582872 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8c00f5440 0x7fa8c00f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:25.484 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.483+0000 7fa8c4f67700 1 -- 172.21.15.27:0/209582872 >> 172.21.15.27:0/209582872 conn(0x7fa8c00f0ba0 msgr2=0x7fa8c00f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:17:25.484 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.483+0000 7fa8c4f67700 1 -- 172.21.15.27:0/209582872 shutdown_connections 2021-11-03T18:17:25.484 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.483+0000 7fa8c4f67700 1 -- 172.21.15.27:0/209582872 wait complete. 2021-11-03T18:17:25.485 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.484+0000 7fa8c4f67700 1 Processor -- start 2021-11-03T18:17:25.485 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.484+0000 7fa8c4f67700 1 -- start start 2021-11-03T18:17:25.485 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.484+0000 7fa8c4f67700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8c00f5440 0x7fa8c01099b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:17:25.485 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.484+0000 7fa8c4f67700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8c00f66e0 0x7fa8c010beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:17:25.485 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.484+0000 7fa8c4f67700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c00603e0 con 0x7fa8c00f5440 2021-11-03T18:17:25.486 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.484+0000 7fa8c4f67700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c0060520 con 0x7fa8c00f66e0 2021-11-03T18:17:25.486 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.484+0000 7fa8bf7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8c00f5440 0x7fa8c01099b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:17:25.486 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.484+0000 7fa8bf7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8c00f5440 0x7fa8c01099b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34014/0 (socket says 172.21.15.27:34014) 2021-11-03T18:17:25.486 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.484+0000 7fa8bf7fe700 1 -- 172.21.15.27:0/1133323446 learned_addr learned my addr 172.21.15.27:0/1133323446 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:17:25.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.484+0000 7fa8beffd700 1 --2- 172.21.15.27:0/1133323446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8c00f66e0 0x7fa8c010beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:17:25.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.484+0000 7fa8bf7fe700 1 -- 172.21.15.27:0/1133323446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8c00f66e0 msgr2=0x7fa8c010beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:17:25.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.485+0000 7fa8bf7fe700 1 --2- 172.21.15.27:0/1133323446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8c00f66e0 0x7fa8c010beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:25.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.485+0000 7fa8bf7fe700 1 -- 172.21.15.27:0/1133323446 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b4005040 con 0x7fa8c00f5440 2021-11-03T18:17:25.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.485+0000 7fa8bf7fe700 1 --2- 172.21.15.27:0/1133323446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8c00f5440 0x7fa8c01099b0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 rx=0x7fa8b000e7c0 tx=0x7fa8b000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:17:25.488 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.485+0000 7fa8a7fff700 1 -- 172.21.15.27:0/1133323446 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8b00092e0 con 0x7fa8c00f5440 2021-11-03T18:17:25.488 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.485+0000 7fa8c4f67700 1 -- 172.21.15.27:0/1133323446 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c010c460 con 0x7fa8c00f5440 2021-11-03T18:17:25.488 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.485+0000 7fa8a7fff700 1 -- 172.21.15.27:0/1133323446 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fa8b000f040 con 0x7fa8c00f5440 2021-11-03T18:17:25.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.485+0000 7fa8a7fff700 1 -- 172.21.15.27:0/1133323446 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa8b00147e0 con 0x7fa8c00f5440 2021-11-03T18:17:25.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.485+0000 7fa8c4f67700 1 -- 172.21.15.27:0/1133323446 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c010ca20 con 0x7fa8c00f5440 2021-11-03T18:17:25.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.486+0000 7fa8c4f67700 1 -- 172.21.15.27:0/1133323446 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c0045ad0 con 0x7fa8c00f5440 2021-11-03T18:17:25.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.486+0000 7fa8a7fff700 1 -- 172.21.15.27:0/1133323446 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fa8b001b030 con 0x7fa8c00f5440 2021-11-03T18:17:25.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.487+0000 7fa8a7fff700 1 --2- 172.21.15.27:0/1133323446 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa8a8064100 0x7fa8a80665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:17:25.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.487+0000 7fa8a7fff700 1 -- 172.21.15.27:0/1133323446 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fa8b0080640 con 0x7fa8c00f5440 2021-11-03T18:17:25.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.487+0000 7fa8beffd700 1 --2- 172.21.15.27:0/1133323446 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa8a8064100 0x7fa8a80665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:17:25.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.487+0000 7fa8beffd700 1 --2- 172.21.15.27:0/1133323446 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa8a8064100 0x7fa8a80665c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7fa8b4005010 tx=0x7fa8b4016040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:17:25.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.491+0000 7fa8a7fff700 1 -- 172.21.15.27:0/1133323446 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fa8b0052900 con 0x7fa8c00f5440 2021-11-03T18:17:25.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.671+0000 7fa8c4f67700 1 -- 172.21.15.27:0/1133323446 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa8c00fdd40 con 0x7fa8a8064100 2021-11-03T18:17:25.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.672+0000 7fa8a7fff700 1 -- 172.21.15.27:0/1133323446 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fa8c00fdd40 con 0x7fa8a8064100 2021-11-03T18:17:25.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.674+0000 7fa8c4f67700 1 -- 172.21.15.27:0/1133323446 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa8a8064100 msgr2=0x7fa8a80665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:17:25.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.674+0000 7fa8c4f67700 1 --2- 172.21.15.27:0/1133323446 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa8a8064100 0x7fa8a80665c0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7fa8b4005010 tx=0x7fa8b4016040).stop 2021-11-03T18:17:25.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.674+0000 7fa8c4f67700 1 -- 172.21.15.27:0/1133323446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8c00f5440 msgr2=0x7fa8c01099b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:17:25.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.674+0000 7fa8c4f67700 1 --2- 172.21.15.27:0/1133323446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8c00f5440 0x7fa8c01099b0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 rx=0x7fa8b000e7c0 tx=0x7fa8b000b040).stop 2021-11-03T18:17:25.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.674+0000 7fa8c4f67700 1 -- 172.21.15.27:0/1133323446 shutdown_connections 2021-11-03T18:17:25.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.674+0000 7fa8c4f67700 1 --2- 172.21.15.27:0/1133323446 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa8a8064100 0x7fa8a80665c0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:25.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.674+0000 7fa8c4f67700 1 --2- 172.21.15.27:0/1133323446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8c00f5440 0x7fa8c01099b0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:25.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.674+0000 7fa8c4f67700 1 --2- 172.21.15.27:0/1133323446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8c00f66e0 0x7fa8c010beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:25.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.674+0000 7fa8c4f67700 1 -- 172.21.15.27:0/1133323446 >> 172.21.15.27:0/1133323446 conn(0x7fa8c00f0ba0 msgr2=0x7fa8c00fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:17:25.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.674+0000 7fa8c4f67700 1 -- 172.21.15.27:0/1133323446 shutdown_connections 2021-11-03T18:17:25.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.674+0000 7fa8c4f67700 1 -- 172.21.15.27:0/1133323446 wait complete. 2021-11-03T18:17:25.682 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:17:25.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3687.448153 secs 2021-11-03T18:17:25.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3687.448153 secs 2021-11-03T18:17:25.774 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.775+0000 7fa7f3fca700 1 -- 172.21.15.27:0/1144875192 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7ec0f55e0 msgr2=0x7fa7ec0f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:17:25.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.775+0000 7fa7f3fca700 1 --2- 172.21.15.27:0/1144875192 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7ec0f55e0 0x7fa7ec0f5a00 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 rx=0x7fa7dc0099e0 tx=0x7fa7dc004d10).stop 2021-11-03T18:17:25.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.775+0000 7fa7f3fca700 1 -- 172.21.15.27:0/1144875192 shutdown_connections 2021-11-03T18:17:25.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.775+0000 7fa7f3fca700 1 --2- 172.21.15.27:0/1144875192 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7ec0f5ef0 0x7fa7ec0f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:25.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.775+0000 7fa7f3fca700 1 --2- 172.21.15.27:0/1144875192 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7ec0f55e0 0x7fa7ec0f5a00 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:25.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.775+0000 7fa7f3fca700 1 -- 172.21.15.27:0/1144875192 >> 172.21.15.27:0/1144875192 conn(0x7fa7ec0f0de0 msgr2=0x7fa7ec0f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:17:25.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.775+0000 7fa7f3fca700 1 -- 172.21.15.27:0/1144875192 shutdown_connections 2021-11-03T18:17:25.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.775+0000 7fa7f3fca700 1 -- 172.21.15.27:0/1144875192 wait complete. 2021-11-03T18:17:25.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.776+0000 7fa7f3fca700 1 Processor -- start 2021-11-03T18:17:25.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.776+0000 7fa7f3fca700 1 -- start start 2021-11-03T18:17:25.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.776+0000 7fa7f3fca700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7ec0f55e0 0x7fa7ec109d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:17:25.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.776+0000 7fa7f3fca700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7ec0f5ef0 0x7fa7ec10c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:17:25.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.776+0000 7fa7f3fca700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ec0604e0 con 0x7fa7ec0f55e0 2021-11-03T18:17:25.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.776+0000 7fa7f3fca700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7ec060620 con 0x7fa7ec0f5ef0 2021-11-03T18:17:25.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.776+0000 7fa7f1d66700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7ec0f55e0 0x7fa7ec109d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:17:25.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.776+0000 7fa7f1d66700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7ec0f55e0 0x7fa7ec109d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34024/0 (socket says 172.21.15.27:34024) 2021-11-03T18:17:25.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.776+0000 7fa7f1d66700 1 -- 172.21.15.27:0/3320512061 learned_addr learned my addr 172.21.15.27:0/3320512061 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:17:25.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.776+0000 7fa7f1d66700 1 -- 172.21.15.27:0/3320512061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7ec0f5ef0 msgr2=0x7fa7ec10c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:17:25.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.776+0000 7fa7f1d66700 1 --2- 172.21.15.27:0/3320512061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7ec0f5ef0 0x7fa7ec10c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:17:25.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.776+0000 7fa7f1d66700 1 -- 172.21.15.27:0/3320512061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7dc009640 con 0x7fa7ec0f55e0 2021-11-03T18:17:25.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.776+0000 7fa7f1d66700 1 --2- 172.21.15.27:0/3320512061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7ec0f55e0 0x7fa7ec109d60 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 rx=0x7fa7dc004b70 tx=0x7fa7dc004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:17:25.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.776+0000 7fa7ea7fc700 1 -- 172.21.15.27:0/3320512061 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7dc00e030 con 0x7fa7ec0f55e0 2021-11-03T18:17:25.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.777+0000 7fa7ea7fc700 1 -- 172.21.15.27:0/3320512061 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fa7dc0049b0 con 0x7fa7ec0f55e0 2021-11-03T18:17:25.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.777+0000 7fa7f3fca700 1 -- 172.21.15.27:0/3320512061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7ec10c7b0 con 0x7fa7ec0f55e0 2021-11-03T18:17:25.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.777+0000 7fa7ea7fc700 1 -- 172.21.15.27:0/3320512061 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7dc018070 con 0x7fa7ec0f55e0 2021-11-03T18:17:25.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.777+0000 7fa7f3fca700 1 -- 172.21.15.27:0/3320512061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7ec10cc90 con 0x7fa7ec0f55e0 2021-11-03T18:17:25.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.777+0000 7fa7f3fca700 1 -- 172.21.15.27:0/3320512061 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7ec045ad0 con 0x7fa7ec0f55e0 2021-11-03T18:17:25.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.778+0000 7fa7ea7fc700 1 -- 172.21.15.27:0/3320512061 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fa7dc01db40 con 0x7fa7ec0f55e0 2021-11-03T18:17:25.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.778+0000 7fa7ea7fc700 1 --2- 172.21.15.27:0/3320512061 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa7d40640b0 0x7fa7d4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:17:25.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.778+0000 7fa7ea7fc700 1 -- 172.21.15.27:0/3320512061 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fa7dc0827d0 con 0x7fa7ec0f55e0 2021-11-03T18:17:25.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.778+0000 7fa7f1565700 1 --2- 172.21.15.27:0/3320512061 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa7d40640b0 0x7fa7d4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:17:25.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.779+0000 7fa7f1565700 1 --2- 172.21.15.27:0/3320512061 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa7d40640b0 0x7fa7d4066570 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7fa7e00047d0 tx=0x7fa7e0007a40).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:17:25.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.782+0000 7fa7ea7fc700 1 -- 172.21.15.27:0/3320512061 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fa7dc054a70 con 0x7fa7ec0f55e0 2021-11-03T18:17:25.954 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.956+0000 7fa7f3fca700 1 -- 172.21.15.27:0/3320512061 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7ec057f10 con 0x7fa7d40640b0 2021-11-03T18:17:25.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.971+0000 7fa7ea7fc700 1 -- 172.21.15.27:0/3320512061 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fa7ec057f10 con 0x7fa7d40640b0 2021-11-03T18:17:25.969 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:17:25.969 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (69m) 8m ago 70m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:17:25.970 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (60m) 8m ago 70m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:17:25.970 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (60m) 7m ago 69m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:17:25.970 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (68m) 8m ago 69m 42.1M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:17:25.970 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (62m) 8m ago 65m 26.3M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:17:25.971 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (65m) 8m ago 65m 3847M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:17:25.971 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (61m) 7m ago 65m 25.9M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:17:25.971 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (62m) 7m ago 65m 27.0M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:17:25.971 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (62m) 8m ago 71m 572M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:17:25.971 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (62m) 7m ago 69m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:17:25.972 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (61m) 8m ago 71m 273M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:17:25.972 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (61m) 7m ago 69m 129M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:17:25.972 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (69m) 8m ago 69m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:17:25.972 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (69m) 7m ago 69m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:17:25.973 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (60m) 8m ago 68m 371M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:17:25.973 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (59m) 8m ago 68m 405M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:17:25.973 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (59m) 8m ago 67m 302M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:17:25.973 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (58m) 7m ago 67m 429M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:17:25.974 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (58m) 7m ago 67m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:17:25.974 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (57m) 7m ago 66m 348M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:17:25.974 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (68m) 8m ago 69m 57.6M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:17:25.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.973+0000 7fa7f3fca700 1 -- 172.21.15.27:0/3320512061 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa7d40640b0 msgr2=0x7fa7d4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:17:25.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.973+0000 7fa7f3fca700 1 --2- 172.21.15.27:0/3320512061 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa7d40640b0 0x7fa7d4066570 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7fa7e00047d0 tx=0x7fa7e0007a40).stop 2021-11-03T18:17:25.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.973+0000 7fa7f3fca700 1 -- 172.21.15.27:0/3320512061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7ec0f55e0 msgr2=0x7fa7ec109d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:17:25.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.973+0000 7fa7f3fca700 1 --2- 172.21.15.27:0/3320512061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7ec0f55e0 0x7fa7ec109d60 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 rx=0x7fa7dc004b70 tx=0x7fa7dc004d10).stop 2021-11-03T18:17:25.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.973+0000 7fa7f3fca700 1 -- 172.21.15.27:0/3320512061 shutdown_connections 2021-11-03T18:17:25.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.973+0000 7fa7f3fca700 1 --2- 172.21.15.27:0/3320512061 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa7d40640b0 0x7fa7d4066570 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:25.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.973+0000 7fa7f3fca700 1 --2- 172.21.15.27:0/3320512061 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7ec0f55e0 0x7fa7ec109d60 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:25.977 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.973+0000 7fa7f3fca700 1 --2- 172.21.15.27:0/3320512061 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7ec0f5ef0 0x7fa7ec10c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:17:25.977 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.973+0000 7fa7f3fca700 1 -- 172.21.15.27:0/3320512061 >> 172.21.15.27:0/3320512061 conn(0x7fa7ec0f0de0 msgr2=0x7fa7ec0fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:17:25.977 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.973+0000 7fa7f3fca700 1 -- 172.21.15.27:0/3320512061 shutdown_connections 2021-11-03T18:17:25.977 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:25.973+0000 7fa7f3fca700 1 -- 172.21.15.27:0/3320512061 wait complete. 2021-11-03T18:17:26.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.072+0000 7fb077a18700 1 -- 172.21.15.27:0/3650453540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0700f55c0 msgr2=0x7fb0700f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:17:26.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.072+0000 7fb077a18700 1 --2- 172.21.15.27:0/3650453540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0700f55c0 0x7fb0700f59e0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 rx=0x7fb060004660 tx=0x7fb060009b30).stop 2021-11-03T18:17:26.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.072+0000 7fb077a18700 1 -- 172.21.15.27:0/3650453540 shutdown_connections 2021-11-03T18:17:26.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.072+0000 7fb077a18700 1 --2- 172.21.15.27:0/3650453540 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0700f6860 0x7fb0700f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:26.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.072+0000 7fb077a18700 1 --2- 172.21.15.27:0/3650453540 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0700f55c0 0x7fb0700f59e0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:26.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.072+0000 7fb077a18700 1 -- 172.21.15.27:0/3650453540 >> 172.21.15.27:0/3650453540 conn(0x7fb0700f0d40 msgr2=0x7fb0700f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:17:26.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.072+0000 7fb077a18700 1 -- 172.21.15.27:0/3650453540 shutdown_connections 2021-11-03T18:17:26.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.072+0000 7fb077a18700 1 -- 172.21.15.27:0/3650453540 wait complete. 2021-11-03T18:17:26.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.073+0000 7fb077a18700 1 Processor -- start 2021-11-03T18:17:26.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.073+0000 7fb077a18700 1 -- start start 2021-11-03T18:17:26.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.073+0000 7fb077a18700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0700f55c0 0x7fb0700fd740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:17:26.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.073+0000 7fb077a18700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0700f6860 0x7fb0700fbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:17:26.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.073+0000 7fb077a18700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0700fc120 con 0x7fb0700f55c0 2021-11-03T18:17:26.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.073+0000 7fb077a18700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0700fc260 con 0x7fb0700f6860 2021-11-03T18:17:26.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.073+0000 7fb0757b4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0700f55c0 0x7fb0700fd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:17:26.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.073+0000 7fb0757b4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0700f55c0 0x7fb0700fd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34034/0 (socket says 172.21.15.27:34034) 2021-11-03T18:17:26.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.073+0000 7fb0757b4700 1 -- 172.21.15.27:0/2062577221 learned_addr learned my addr 172.21.15.27:0/2062577221 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:17:26.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.073+0000 7fb074fb3700 1 --2- 172.21.15.27:0/2062577221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0700f6860 0x7fb0700fbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:17:26.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.074+0000 7fb0757b4700 1 -- 172.21.15.27:0/2062577221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0700f6860 msgr2=0x7fb0700fbbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:17:26.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.074+0000 7fb0757b4700 1 --2- 172.21.15.27:0/2062577221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0700f6860 0x7fb0700fbbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:26.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.074+0000 7fb0757b4700 1 -- 172.21.15.27:0/2062577221 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb060005040 con 0x7fb0700f55c0 2021-11-03T18:17:26.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.074+0000 7fb0757b4700 1 --2- 172.21.15.27:0/2062577221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0700f55c0 0x7fb0700fd740 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 rx=0x7fb060013a70 tx=0x7fb060009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:17:26.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.074+0000 7fb065ffb700 1 -- 172.21.15.27:0/2062577221 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb060018070 con 0x7fb0700f55c0 2021-11-03T18:17:26.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.074+0000 7fb065ffb700 1 -- 172.21.15.27:0/2062577221 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fb0600097d0 con 0x7fb0700f55c0 2021-11-03T18:17:26.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.074+0000 7fb077a18700 1 -- 172.21.15.27:0/2062577221 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0700fc4e0 con 0x7fb0700f55c0 2021-11-03T18:17:26.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.074+0000 7fb065ffb700 1 -- 172.21.15.27:0/2062577221 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb06001de80 con 0x7fb0700f55c0 2021-11-03T18:17:26.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.074+0000 7fb077a18700 1 -- 172.21.15.27:0/2062577221 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0700fca40 con 0x7fb0700f55c0 2021-11-03T18:17:26.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.075+0000 7fb077a18700 1 -- 172.21.15.27:0/2062577221 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb070045ad0 con 0x7fb0700f55c0 2021-11-03T18:17:26.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.076+0000 7fb065ffb700 1 -- 172.21.15.27:0/2062577221 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fb060007ba0 con 0x7fb0700f55c0 2021-11-03T18:17:26.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.076+0000 7fb065ffb700 1 --2- 172.21.15.27:0/2062577221 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb05c0640b0 0x7fb05c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:17:26.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.076+0000 7fb065ffb700 1 -- 172.21.15.27:0/2062577221 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fb060082360 con 0x7fb0700f55c0 2021-11-03T18:17:26.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.076+0000 7fb074fb3700 1 --2- 172.21.15.27:0/2062577221 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb05c0640b0 0x7fb05c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:17:26.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.077+0000 7fb074fb3700 1 --2- 172.21.15.27:0/2062577221 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb05c0640b0 0x7fb05c066570 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7fb06c004d80 tx=0x7fb06c00b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:17:26.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.080+0000 7fb065ffb700 1 -- 172.21.15.27:0/2062577221 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fb060054720 con 0x7fb0700f55c0 2021-11-03T18:17:26.308 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.311+0000 7fb077a18700 1 -- 172.21.15.27:0/2062577221 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb0700fef90 con 0x7fb0700f55c0 2021-11-03T18:17:26.309 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.311+0000 7fb065ffb700 1 -- 172.21.15.27:0/2062577221 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fb060054310 con 0x7fb0700f55c0 2021-11-03T18:17:26.309 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:17:26.310 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:17:26.310 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:17:26.310 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:17:26.310 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:17:26.311 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:17:26.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:17:26.311 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:17:26.311 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:17:26.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:17:26.312 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:17:26.312 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:17:26.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:17:26.313 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:17:26.313 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:17:26.313 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:17:26.314 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:17:26.314 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:17:26.314 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.313+0000 7fb077a18700 1 -- 172.21.15.27:0/2062577221 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb05c0640b0 msgr2=0x7fb05c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:17:26.315 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.313+0000 7fb077a18700 1 --2- 172.21.15.27:0/2062577221 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb05c0640b0 0x7fb05c066570 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7fb06c004d80 tx=0x7fb06c00b040).stop 2021-11-03T18:17:26.315 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.313+0000 7fb077a18700 1 -- 172.21.15.27:0/2062577221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0700f55c0 msgr2=0x7fb0700fd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:17:26.315 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.313+0000 7fb077a18700 1 --2- 172.21.15.27:0/2062577221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0700f55c0 0x7fb0700fd740 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 rx=0x7fb060013a70 tx=0x7fb060009b30).stop 2021-11-03T18:17:26.315 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.313+0000 7fb077a18700 1 -- 172.21.15.27:0/2062577221 shutdown_connections 2021-11-03T18:17:26.316 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.313+0000 7fb077a18700 1 --2- 172.21.15.27:0/2062577221 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb05c0640b0 0x7fb05c066570 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:26.316 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.313+0000 7fb077a18700 1 --2- 172.21.15.27:0/2062577221 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0700f55c0 0x7fb0700fd740 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:26.316 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.313+0000 7fb077a18700 1 --2- 172.21.15.27:0/2062577221 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0700f6860 0x7fb0700fbbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:26.316 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.313+0000 7fb077a18700 1 -- 172.21.15.27:0/2062577221 >> 172.21.15.27:0/2062577221 conn(0x7fb0700f0d40 msgr2=0x7fb0700fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:17:26.316 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.313+0000 7fb077a18700 1 -- 172.21.15.27:0/2062577221 shutdown_connections 2021-11-03T18:17:26.317 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.313+0000 7fb077a18700 1 -- 172.21.15.27:0/2062577221 wait complete. 2021-11-03T18:17:26.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.413+0000 7f6b0c21b700 1 -- 172.21.15.27:0/2768127050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b040f55c0 msgr2=0x7f6b040f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:17:26.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.413+0000 7f6b0c21b700 1 --2- 172.21.15.27:0/2768127050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b040f55c0 0x7f6b040f59e0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 rx=0x7f6af8004660 tx=0x7f6af8009b30).stop 2021-11-03T18:17:26.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.413+0000 7f6b0c21b700 1 -- 172.21.15.27:0/2768127050 shutdown_connections 2021-11-03T18:17:26.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.413+0000 7f6b0c21b700 1 --2- 172.21.15.27:0/2768127050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b040f6860 0x7f6b040f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:26.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.413+0000 7f6b0c21b700 1 --2- 172.21.15.27:0/2768127050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b040f55c0 0x7f6b040f59e0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:26.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.413+0000 7f6b0c21b700 1 -- 172.21.15.27:0/2768127050 >> 172.21.15.27:0/2768127050 conn(0x7f6b040f0d20 msgr2=0x7f6b040f3140 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:17:26.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.413+0000 7f6b0c21b700 1 -- 172.21.15.27:0/2768127050 shutdown_connections 2021-11-03T18:17:26.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.413+0000 7f6b0c21b700 1 -- 172.21.15.27:0/2768127050 wait complete. 2021-11-03T18:17:26.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.413+0000 7f6b0c21b700 1 Processor -- start 2021-11-03T18:17:26.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.413+0000 7f6b0c21b700 1 -- start start 2021-11-03T18:17:26.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.413+0000 7f6b0c21b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b040f55c0 0x7f6b04106fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:17:26.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.413+0000 7f6b0c21b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b040f6860 0x7f6b041074c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:17:26.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.413+0000 7f6b0c21b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b04107a70 con 0x7f6b040f55c0 2021-11-03T18:17:26.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.414+0000 7f6b0c21b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b04107bb0 con 0x7f6b040f6860 2021-11-03T18:17:26.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.414+0000 7f6b09fb7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b040f55c0 0x7f6b04106fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:17:26.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.414+0000 7f6b09fb7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b040f55c0 0x7f6b04106fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34044/0 (socket says 172.21.15.27:34044) 2021-11-03T18:17:26.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.414+0000 7f6b09fb7700 1 -- 172.21.15.27:0/3513489207 learned_addr learned my addr 172.21.15.27:0/3513489207 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:17:26.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.414+0000 7f6b097b6700 1 --2- 172.21.15.27:0/3513489207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b040f6860 0x7f6b041074c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:17:26.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.414+0000 7f6b09fb7700 1 -- 172.21.15.27:0/3513489207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b040f6860 msgr2=0x7f6b041074c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:17:26.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.414+0000 7f6b09fb7700 1 --2- 172.21.15.27:0/3513489207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b040f6860 0x7f6b041074c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:26.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.414+0000 7f6b09fb7700 1 -- 172.21.15.27:0/3513489207 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af8005040 con 0x7f6b040f55c0 2021-11-03T18:17:26.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.414+0000 7f6b097b6700 1 --2- 172.21.15.27:0/3513489207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b040f6860 0x7f6b041074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T18:17:26.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.414+0000 7f6b09fb7700 1 --2- 172.21.15.27:0/3513489207 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b040f55c0 0x7f6b04106fd0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 rx=0x7f6af8004f60 tx=0x7f6af8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:17:26.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.414+0000 7f6af67fc700 1 -- 172.21.15.27:0/3513489207 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6af8018070 con 0x7f6b040f55c0 2021-11-03T18:17:26.419 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.414+0000 7f6af67fc700 1 -- 172.21.15.27:0/3513489207 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f6af8007ba0 con 0x7f6b040f55c0 2021-11-03T18:17:26.419 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.414+0000 7f6b0c21b700 1 -- 172.21.15.27:0/3513489207 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b041084f0 con 0x7f6b040f55c0 2021-11-03T18:17:26.419 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.414+0000 7f6af67fc700 1 -- 172.21.15.27:0/3513489207 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6af801c620 con 0x7f6b040f55c0 2021-11-03T18:17:26.420 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.414+0000 7f6b0c21b700 1 -- 172.21.15.27:0/3513489207 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b04108a50 con 0x7f6b040f55c0 2021-11-03T18:17:26.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.415+0000 7f6b0c21b700 1 -- 172.21.15.27:0/3513489207 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b04045ad0 con 0x7f6b040f55c0 2021-11-03T18:17:26.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.416+0000 7f6af67fc700 1 -- 172.21.15.27:0/3513489207 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f6af801c8e0 con 0x7f6b040f55c0 2021-11-03T18:17:26.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.416+0000 7f6af67fc700 1 --2- 172.21.15.27:0/3513489207 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6af00640b0 0x7f6af0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:17:26.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.416+0000 7f6af67fc700 1 -- 172.21.15.27:0/3513489207 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f6af8082720 con 0x7f6b040f55c0 2021-11-03T18:17:26.423 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.416+0000 7f6b097b6700 1 --2- 172.21.15.27:0/3513489207 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6af00640b0 0x7f6af0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:17:26.424 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.416+0000 7f6b097b6700 1 --2- 172.21.15.27:0/3513489207 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6af00640b0 0x7f6af0066570 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f6b000041d0 tx=0x7f6b00009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:17:26.424 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.420+0000 7f6af67fc700 1 -- 172.21.15.27:0/3513489207 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f6af8054a20 con 0x7f6b040f55c0 2021-11-03T18:17:26.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.625+0000 7f6b0c21b700 1 -- 172.21.15.27:0/3513489207 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6b04109220 con 0x7f6b040f55c0 2021-11-03T18:17:26.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.625+0000 7f6af67fc700 1 -- 172.21.15.27:0/3513489207 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f6b04109220 con 0x7f6b040f55c0 2021-11-03T18:17:26.624 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:17:26.624 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:17:26.624 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:17:26.625 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:17:26.625 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:17:26.625 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:17:26.625 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:17:26.625 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:17:26.626 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:17:26.626 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:17:26.626 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:17:26.626 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:17:26.627 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:17:26.627 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:17:26.627 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:17:26.627 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:17:26.628 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:17:26.628 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:17:26.628 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:17:26.628 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:17:26.628 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:17:26.629 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:17:26.629 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:17:26.629 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:17:26.629 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:17:26.630 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:17:26.630 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:17:26.630 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:17:26.630 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:17:26.630 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:17:26.631 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:17:26.631 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:17:26.631 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:17:26.631 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:17:26.632 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:17:26.632 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:17:26.632 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:17:26.632 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:17:26.632 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:17:26.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.627+0000 7f6b0c21b700 1 -- 172.21.15.27:0/3513489207 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6af00640b0 msgr2=0x7f6af0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:17:26.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.627+0000 7f6b0c21b700 1 --2- 172.21.15.27:0/3513489207 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6af00640b0 0x7f6af0066570 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f6b000041d0 tx=0x7f6b00009040).stop 2021-11-03T18:17:26.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.627+0000 7f6b0c21b700 1 -- 172.21.15.27:0/3513489207 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b040f55c0 msgr2=0x7f6b04106fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:17:26.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.627+0000 7f6b0c21b700 1 --2- 172.21.15.27:0/3513489207 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b040f55c0 0x7f6b04106fd0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 rx=0x7f6af8004f60 tx=0x7f6af8009b30).stop 2021-11-03T18:17:26.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.627+0000 7f6b0c21b700 1 -- 172.21.15.27:0/3513489207 shutdown_connections 2021-11-03T18:17:26.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.627+0000 7f6b0c21b700 1 --2- 172.21.15.27:0/3513489207 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6af00640b0 0x7f6af0066570 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:26.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.627+0000 7f6b0c21b700 1 --2- 172.21.15.27:0/3513489207 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6b040f55c0 0x7f6b04106fd0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:26.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.627+0000 7f6b0c21b700 1 --2- 172.21.15.27:0/3513489207 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6b040f6860 0x7f6b041074c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:26.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.627+0000 7f6b0c21b700 1 -- 172.21.15.27:0/3513489207 >> 172.21.15.27:0/3513489207 conn(0x7f6b040f0d20 msgr2=0x7f6b0405f780 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:17:26.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.627+0000 7f6b0c21b700 1 -- 172.21.15.27:0/3513489207 shutdown_connections 2021-11-03T18:17:26.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:26.627+0000 7f6b0c21b700 1 -- 172.21.15.27:0/3513489207 wait complete. 2021-11-03T18:17:26.637 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:17:26.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:26 smithi115 ceph-mon[65453]: pgmap v1883: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:26.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:26 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2062577221' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:26.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:26 smithi027 ceph-mon[97244]: pgmap v1883: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:26.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:26 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2062577221' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:27.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:27 smithi115 ceph-mon[65453]: from='client.36040 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:17:27.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:27 smithi115 ceph-mon[65453]: from='client.36044 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:17:27.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:27 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3513489207' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:17:27.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:27 smithi027 ceph-mon[97244]: from='client.36040 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:17:27.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:27 smithi027 ceph-mon[97244]: from='client.36044 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:17:27.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:27 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3513489207' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:17:28.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:28 smithi115 ceph-mon[65453]: pgmap v1884: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:28.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:28 smithi027 ceph-mon[97244]: pgmap v1884: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:30.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:30 smithi115 ceph-mon[65453]: pgmap v1885: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:30.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3692.448302 secs 2021-11-03T18:17:30.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:30 smithi027 ceph-mon[97244]: pgmap v1885: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:30.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3692.448302 secs 2021-11-03T18:17:31.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:17:31.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:17:31.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:17:31.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:31.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:31.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:31.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:31.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:17:31.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:17:31.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:17:31.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:31.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:31.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:31.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:32.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:32 smithi115 ceph-mon[65453]: pgmap v1886: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:32.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:32 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:17:32.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:32 smithi027 ceph-mon[97244]: pgmap v1886: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:32.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:32 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:17:34.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:34 smithi115 ceph-mon[65453]: pgmap v1887: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:34.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:34 smithi027 ceph-mon[97244]: pgmap v1887: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:35.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3697.448455 secs 2021-11-03T18:17:35.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3697.448455 secs 2021-11-03T18:17:36.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:36 smithi115 ceph-mon[65453]: pgmap v1888: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:36.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:36 smithi027 ceph-mon[97244]: pgmap v1888: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:38.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:38 smithi115 ceph-mon[65453]: pgmap v1889: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:38.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:38 smithi027 ceph-mon[97244]: pgmap v1889: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:40.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:40 smithi115 ceph-mon[65453]: pgmap v1890: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:40.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3702.448596 secs 2021-11-03T18:17:40.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:40 smithi027 ceph-mon[97244]: pgmap v1890: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:40.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3702.448596 secs 2021-11-03T18:17:41.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:17:41.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:17:41.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:17:41.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:41.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:41.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:41.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:41.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:17:41.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:17:41.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:17:41.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:41.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:41.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:41.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:42.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:42 smithi115 ceph-mon[65453]: pgmap v1891: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:42.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:42 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:17:42.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:42 smithi027 ceph-mon[97244]: pgmap v1891: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:42.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:42 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:17:44.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:44 smithi115 ceph-mon[65453]: pgmap v1892: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:44.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:44 smithi027 ceph-mon[97244]: pgmap v1892: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:45.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3707.448725 secs 2021-11-03T18:17:45.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3707.448725 secs 2021-11-03T18:17:46.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:46 smithi027 ceph-mon[97244]: pgmap v1893: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:46.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:46 smithi115 ceph-mon[65453]: pgmap v1893: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:48.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:48 smithi027 ceph-mon[97244]: pgmap v1894: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:48.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:48 smithi115 ceph-mon[65453]: pgmap v1894: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:50.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:50 smithi027 ceph-mon[97244]: pgmap v1895: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:50.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3712.448848 secs 2021-11-03T18:17:50.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:50 smithi115 ceph-mon[65453]: pgmap v1895: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:50.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3712.448848 secs 2021-11-03T18:17:51.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:17:51.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:17:51.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:17:51.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:51.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:51.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:51.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:51.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:17:51.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:17:51.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:17:51.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:51.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:51.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:51.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:52.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:52 smithi027 ceph-mon[97244]: pgmap v1896: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:52.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:52 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:17:52.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:52 smithi115 ceph-mon[65453]: pgmap v1896: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:52.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:52 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:17:54.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:54 smithi027 ceph-mon[97244]: pgmap v1897: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:54.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:54 smithi115 ceph-mon[65453]: pgmap v1897: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:55.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3717.448992 secs 2021-11-03T18:17:55.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3717.448992 secs 2021-11-03T18:17:56.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.740+0000 7f37aec0f700 1 -- 172.21.15.27:0/523578670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37a80f66e0 msgr2=0x7f37a80f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:17:56.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.740+0000 7f37aec0f700 1 --2- 172.21.15.27:0/523578670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37a80f66e0 0x7f37a80f6b00 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 rx=0x7f379c004660 tx=0x7f379c009b30).stop 2021-11-03T18:17:56.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.741+0000 7f37aec0f700 1 -- 172.21.15.27:0/523578670 shutdown_connections 2021-11-03T18:17:56.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.741+0000 7f37aec0f700 1 --2- 172.21.15.27:0/523578670 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37a80f6ff0 0x7f37a80f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:56.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.741+0000 7f37aec0f700 1 --2- 172.21.15.27:0/523578670 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37a80f66e0 0x7f37a80f6b00 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:56.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.741+0000 7f37aec0f700 1 -- 172.21.15.27:0/523578670 >> 172.21.15.27:0/523578670 conn(0x7f37a80f1f00 msgr2=0x7f37a80f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:17:56.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.741+0000 7f37aec0f700 1 -- 172.21.15.27:0/523578670 shutdown_connections 2021-11-03T18:17:56.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.741+0000 7f37aec0f700 1 -- 172.21.15.27:0/523578670 wait complete. 2021-11-03T18:17:56.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.741+0000 7f37aec0f700 1 Processor -- start 2021-11-03T18:17:56.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.741+0000 7f37aec0f700 1 -- start start 2021-11-03T18:17:56.744 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.742+0000 7f37aec0f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37a80f66e0 0x7f37a8106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:17:56.744 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.742+0000 7f37aec0f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37a80f6ff0 0x7f37a8107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:17:56.744 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.742+0000 7f37aec0f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a8107a00 con 0x7f37a80f66e0 2021-11-03T18:17:56.745 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.742+0000 7f37aec0f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a8107b40 con 0x7f37a80f6ff0 2021-11-03T18:17:56.745 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.742+0000 7f37ac9ab700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37a80f66e0 0x7f37a8106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:17:56.745 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.742+0000 7f37ac9ab700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37a80f66e0 0x7f37a8106f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34062/0 (socket says 172.21.15.27:34062) 2021-11-03T18:17:56.745 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.742+0000 7f37ac9ab700 1 -- 172.21.15.27:0/131689004 learned_addr learned my addr 172.21.15.27:0/131689004 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:17:56.746 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.742+0000 7f37ac9ab700 1 -- 172.21.15.27:0/131689004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37a80f6ff0 msgr2=0x7f37a8107450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:17:56.746 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.742+0000 7f37ac9ab700 1 --2- 172.21.15.27:0/131689004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37a80f6ff0 0x7f37a8107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:17:56.746 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.742+0000 7f37ac9ab700 1 -- 172.21.15.27:0/131689004 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f379c005040 con 0x7f37a80f66e0 2021-11-03T18:17:56.746 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.742+0000 7f37ac9ab700 1 --2- 172.21.15.27:0/131689004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37a80f66e0 0x7f37a8106f60 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 rx=0x7f379c009990 tx=0x7f379c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:17:56.746 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.742+0000 7f37a57fa700 1 -- 172.21.15.27:0/131689004 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f379c018070 con 0x7f37a80f66e0 2021-11-03T18:17:56.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.742+0000 7f37a57fa700 1 -- 172.21.15.27:0/131689004 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f379c007ca0 con 0x7f37a80f66e0 2021-11-03T18:17:56.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.742+0000 7f37aec0f700 1 -- 172.21.15.27:0/131689004 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37a8108480 con 0x7f37a80f66e0 2021-11-03T18:17:56.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.742+0000 7f37a57fa700 1 -- 172.21.15.27:0/131689004 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f379c01c620 con 0x7f37a80f66e0 2021-11-03T18:17:56.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.742+0000 7f37aec0f700 1 -- 172.21.15.27:0/131689004 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37a81089e0 con 0x7f37a80f66e0 2021-11-03T18:17:56.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.743+0000 7f37aec0f700 1 -- 172.21.15.27:0/131689004 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37a8045ad0 con 0x7f37a80f66e0 2021-11-03T18:17:56.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.744+0000 7f37a57fa700 1 -- 172.21.15.27:0/131689004 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f379c01d960 con 0x7f37a80f66e0 2021-11-03T18:17:56.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.744+0000 7f37a57fa700 1 --2- 172.21.15.27:0/131689004 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f37980640b0 0x7f3798066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:17:56.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.745+0000 7f37a57fa700 1 -- 172.21.15.27:0/131689004 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f379c082580 con 0x7f37a80f66e0 2021-11-03T18:17:56.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.745+0000 7f37a7fff700 1 --2- 172.21.15.27:0/131689004 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f37980640b0 0x7f3798066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:17:56.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.745+0000 7f37a7fff700 1 --2- 172.21.15.27:0/131689004 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f37980640b0 0x7f3798066570 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7f3794004870 tx=0x7f37940079c0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:17:56.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.749+0000 7f37a57fa700 1 -- 172.21.15.27:0/131689004 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f379c054940 con 0x7f37a80f66e0 2021-11-03T18:17:56.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:56 smithi027 ceph-mon[97244]: pgmap v1898: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:56.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:17:56.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:17:56.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.929+0000 7f37aec0f700 1 -- 172.21.15.27:0/131689004 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f37a80fefe0 con 0x7f37980640b0 2021-11-03T18:17:56.929 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.931+0000 7f37a57fa700 1 -- 172.21.15.27:0/131689004 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f37a80fefe0 con 0x7f37980640b0 2021-11-03T18:17:56.931 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.933+0000 7f37aec0f700 1 -- 172.21.15.27:0/131689004 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f37980640b0 msgr2=0x7f3798066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:17:56.932 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.933+0000 7f37aec0f700 1 --2- 172.21.15.27:0/131689004 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f37980640b0 0x7f3798066570 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 rx=0x7f3794004870 tx=0x7f37940079c0).stop 2021-11-03T18:17:56.932 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.933+0000 7f37aec0f700 1 -- 172.21.15.27:0/131689004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37a80f66e0 msgr2=0x7f37a8106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:17:56.932 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.933+0000 7f37aec0f700 1 --2- 172.21.15.27:0/131689004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37a80f66e0 0x7f37a8106f60 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 rx=0x7f379c009990 tx=0x7f379c009b30).stop 2021-11-03T18:17:56.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.933+0000 7f37aec0f700 1 -- 172.21.15.27:0/131689004 shutdown_connections 2021-11-03T18:17:56.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.933+0000 7f37aec0f700 1 --2- 172.21.15.27:0/131689004 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f37980640b0 0x7f3798066570 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:56.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.933+0000 7f37aec0f700 1 --2- 172.21.15.27:0/131689004 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f37a80f66e0 0x7f37a8106f60 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:56.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.933+0000 7f37aec0f700 1 --2- 172.21.15.27:0/131689004 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f37a80f6ff0 0x7f37a8107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:17:56.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.933+0000 7f37aec0f700 1 -- 172.21.15.27:0/131689004 >> 172.21.15.27:0/131689004 conn(0x7f37a80f1f00 msgr2=0x7f37a80fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:17:56.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.933+0000 7f37aec0f700 1 -- 172.21.15.27:0/131689004 shutdown_connections 2021-11-03T18:17:56.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:56.933+0000 7f37aec0f700 1 -- 172.21.15.27:0/131689004 wait complete. 2021-11-03T18:17:56.941 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:17:56.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:56 smithi115 ceph-mon[65453]: pgmap v1898: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:56.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:17:56.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:17:57.032 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.032+0000 7fcb0895f700 1 -- 172.21.15.27:0/4117463695 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb040f66e0 msgr2=0x7fcb040f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:17:57.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.032+0000 7fcb0895f700 1 --2- 172.21.15.27:0/4117463695 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb040f66e0 0x7fcb040f6b40 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 rx=0x7fcaf8004660 tx=0x7fcaf8009b30).stop 2021-11-03T18:17:57.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.032+0000 7fcb0895f700 1 -- 172.21.15.27:0/4117463695 shutdown_connections 2021-11-03T18:17:57.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.032+0000 7fcb0895f700 1 --2- 172.21.15.27:0/4117463695 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb040f66e0 0x7fcb040f6b40 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:57.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.032+0000 7fcb0895f700 1 --2- 172.21.15.27:0/4117463695 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb040f5440 0x7fcb040f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T18:17:57.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.032+0000 7fcb0895f700 1 -- 172.21.15.27:0/4117463695 >> 172.21.15.27:0/4117463695 conn(0x7fcb040f0ba0 msgr2=0x7fcb040f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:17:57.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.032+0000 7fcb0895f700 1 -- 172.21.15.27:0/4117463695 shutdown_connections 2021-11-03T18:17:57.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.032+0000 7fcb0895f700 1 -- 172.21.15.27:0/4117463695 wait complete. 2021-11-03T18:17:57.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.033+0000 7fcb0895f700 1 Processor -- start 2021-11-03T18:17:57.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.033+0000 7fcb0895f700 1 -- start start 2021-11-03T18:17:57.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.033+0000 7fcb0895f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb040f5440 0x7fcb04105d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:17:57.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.033+0000 7fcb0895f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb040f66e0 0x7fcb04106210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:17:57.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.033+0000 7fcb0895f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb04106850 con 0x7fcb040f66e0 2021-11-03T18:17:57.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.033+0000 7fcb0895f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb04106fc0 con 0x7fcb040f5440 2021-11-03T18:17:57.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.033+0000 7fcb02ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb040f66e0 0x7fcb04106210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:17:57.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.033+0000 7fcb02ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb040f66e0 0x7fcb04106210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34074/0 (socket says 172.21.15.27:34074) 2021-11-03T18:17:57.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.033+0000 7fcb02ffd700 1 -- 172.21.15.27:0/3300350956 learned_addr learned my addr 172.21.15.27:0/3300350956 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:17:57.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.033+0000 7fcb037fe700 1 --2- 172.21.15.27:0/3300350956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb040f5440 0x7fcb04105d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:17:57.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.034+0000 7fcb02ffd700 1 -- 172.21.15.27:0/3300350956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb040f5440 msgr2=0x7fcb04105d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:17:57.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.034+0000 7fcb02ffd700 1 --2- 172.21.15.27:0/3300350956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb040f5440 0x7fcb04105d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:57.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.034+0000 7fcb02ffd700 1 -- 172.21.15.27:0/3300350956 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf8005040 con 0x7fcb040f66e0 2021-11-03T18:17:57.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.034+0000 7fcb037fe700 1 --2- 172.21.15.27:0/3300350956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb040f5440 0x7fcb04105d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:17:57.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.034+0000 7fcb02ffd700 1 --2- 172.21.15.27:0/3300350956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb040f66e0 0x7fcb04106210 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0x7fcaf8009990 tx=0x7fcaf8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:17:57.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.034+0000 7fcaebfff700 1 -- 172.21.15.27:0/3300350956 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcaf8018070 con 0x7fcb040f66e0 2021-11-03T18:17:57.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.034+0000 7fcb0895f700 1 -- 172.21.15.27:0/3300350956 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb04107240 con 0x7fcb040f66e0 2021-11-03T18:17:57.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.034+0000 7fcaebfff700 1 -- 172.21.15.27:0/3300350956 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fcaf8007ca0 con 0x7fcb040f66e0 2021-11-03T18:17:57.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.034+0000 7fcaebfff700 1 -- 172.21.15.27:0/3300350956 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcaf801c620 con 0x7fcb040f66e0 2021-11-03T18:17:57.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.034+0000 7fcb0895f700 1 -- 172.21.15.27:0/3300350956 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb041077a0 con 0x7fcb040f66e0 2021-11-03T18:17:57.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.035+0000 7fcb0895f700 1 -- 172.21.15.27:0/3300350956 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb04045ad0 con 0x7fcb040f66e0 2021-11-03T18:17:57.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.036+0000 7fcaebfff700 1 -- 172.21.15.27:0/3300350956 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fcaf801d960 con 0x7fcb040f66e0 2021-11-03T18:17:57.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.036+0000 7fcaebfff700 1 --2- 172.21.15.27:0/3300350956 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcaec064100 0x7fcaec0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:17:57.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.036+0000 7fcaebfff700 1 -- 172.21.15.27:0/3300350956 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fcaf8082700 con 0x7fcb040f66e0 2021-11-03T18:17:57.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.036+0000 7fcb037fe700 1 --2- 172.21.15.27:0/3300350956 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcaec064100 0x7fcaec0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:17:57.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.037+0000 7fcb037fe700 1 --2- 172.21.15.27:0/3300350956 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcaec064100 0x7fcaec0665c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7fcb040f6540 tx=0x7fcaf4008040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:17:57.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.040+0000 7fcaebfff700 1 -- 172.21.15.27:0/3300350956 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fcaf8054a00 con 0x7fcb040f66e0 2021-11-03T18:17:57.213 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.213+0000 7fcb0895f700 1 -- 172.21.15.27:0/3300350956 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb040fdd10 con 0x7fcaec064100 2021-11-03T18:17:57.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.227+0000 7fcaebfff700 1 -- 172.21.15.27:0/3300350956 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fcb040fdd10 con 0x7fcaec064100 2021-11-03T18:17:57.226 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:17:57.227 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (69m) 8m ago 70m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:17:57.227 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (61m) 8m ago 70m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:17:57.227 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (61m) 7m ago 70m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:17:57.227 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (69m) 8m ago 70m 42.1M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:17:57.228 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (63m) 8m ago 65m 26.3M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:17:57.228 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (65m) 8m ago 65m 3847M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:17:57.229 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (61m) 7m ago 65m 25.9M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:17:57.229 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (63m) 7m ago 66m 27.0M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:17:57.230 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (63m) 8m ago 72m 572M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:17:57.230 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (62m) 7m ago 69m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:17:57.230 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (62m) 8m ago 72m 273M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:17:57.230 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (61m) 7m ago 69m 129M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:17:57.231 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (70m) 8m ago 70m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:17:57.231 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (69m) 7m ago 69m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:17:57.231 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (60m) 8m ago 69m 371M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:17:57.231 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (60m) 8m ago 68m 405M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:17:57.232 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (59m) 8m ago 68m 302M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:17:57.232 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (59m) 7m ago 68m 429M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:17:57.232 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (58m) 7m ago 67m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:17:57.232 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (58m) 7m ago 67m 348M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:17:57.233 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (69m) 8m ago 70m 57.6M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:17:57.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.229+0000 7fcb0895f700 1 -- 172.21.15.27:0/3300350956 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcaec064100 msgr2=0x7fcaec0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:17:57.234 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.229+0000 7fcb0895f700 1 --2- 172.21.15.27:0/3300350956 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcaec064100 0x7fcaec0665c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7fcb040f6540 tx=0x7fcaf4008040).stop 2021-11-03T18:17:57.234 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.229+0000 7fcb0895f700 1 -- 172.21.15.27:0/3300350956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb040f66e0 msgr2=0x7fcb04106210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:17:57.235 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.229+0000 7fcb0895f700 1 --2- 172.21.15.27:0/3300350956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb040f66e0 0x7fcb04106210 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0x7fcaf8009990 tx=0x7fcaf8009b30).stop 2021-11-03T18:17:57.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.229+0000 7fcb0895f700 1 -- 172.21.15.27:0/3300350956 shutdown_connections 2021-11-03T18:17:57.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.229+0000 7fcb0895f700 1 --2- 172.21.15.27:0/3300350956 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcaec064100 0x7fcaec0665c0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:57.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.229+0000 7fcb0895f700 1 --2- 172.21.15.27:0/3300350956 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcb040f5440 0x7fcb04105d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:57.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.229+0000 7fcb0895f700 1 --2- 172.21.15.27:0/3300350956 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcb040f66e0 0x7fcb04106210 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:57.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.229+0000 7fcb0895f700 1 -- 172.21.15.27:0/3300350956 >> 172.21.15.27:0/3300350956 conn(0x7fcb040f0ba0 msgr2=0x7fcb040fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:17:57.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.229+0000 7fcb0895f700 1 -- 172.21.15.27:0/3300350956 shutdown_connections 2021-11-03T18:17:57.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.229+0000 7fcb0895f700 1 -- 172.21.15.27:0/3300350956 wait complete. 2021-11-03T18:17:57.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.328+0000 7fdfa376b700 1 -- 172.21.15.27:0/673873506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf9c0f66e0 msgr2=0x7fdf9c0f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:17:57.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.328+0000 7fdfa376b700 1 --2- 172.21.15.27:0/673873506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf9c0f66e0 0x7fdf9c0f6b00 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 rx=0x7fdf940099e0 tx=0x7fdf94004d10).stop 2021-11-03T18:17:57.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.328+0000 7fdfa376b700 1 -- 172.21.15.27:0/673873506 shutdown_connections 2021-11-03T18:17:57.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.328+0000 7fdfa376b700 1 --2- 172.21.15.27:0/673873506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf9c0f6ff0 0x7fdf9c0f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:57.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.328+0000 7fdfa376b700 1 --2- 172.21.15.27:0/673873506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf9c0f66e0 0x7fdf9c0f6b00 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:57.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.328+0000 7fdfa376b700 1 -- 172.21.15.27:0/673873506 >> 172.21.15.27:0/673873506 conn(0x7fdf9c0f1f00 msgr2=0x7fdf9c0f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:17:57.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.328+0000 7fdfa376b700 1 -- 172.21.15.27:0/673873506 shutdown_connections 2021-11-03T18:17:57.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.328+0000 7fdfa376b700 1 -- 172.21.15.27:0/673873506 wait complete. 2021-11-03T18:17:57.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.329+0000 7fdfa376b700 1 Processor -- start 2021-11-03T18:17:57.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.329+0000 7fdfa376b700 1 -- start start 2021-11-03T18:17:57.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.329+0000 7fdfa376b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf9c0f66e0 0x7fdf9c107070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:17:57.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.329+0000 7fdfa376b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf9c0f6ff0 0x7fdf9c107560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:17:57.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.329+0000 7fdfa376b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c107b10 con 0x7fdf9c0f6ff0 2021-11-03T18:17:57.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.329+0000 7fdfa376b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf9c107c50 con 0x7fdf9c0f66e0 2021-11-03T18:17:57.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.329+0000 7fdfa0d06700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf9c0f6ff0 0x7fdf9c107560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:17:57.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.329+0000 7fdfa0d06700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf9c0f6ff0 0x7fdf9c107560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34084/0 (socket says 172.21.15.27:34084) 2021-11-03T18:17:57.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.329+0000 7fdfa0d06700 1 -- 172.21.15.27:0/1682474918 learned_addr learned my addr 172.21.15.27:0/1682474918 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:17:57.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.329+0000 7fdfa1507700 1 --2- 172.21.15.27:0/1682474918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf9c0f66e0 0x7fdf9c107070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:17:57.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.329+0000 7fdfa0d06700 1 -- 172.21.15.27:0/1682474918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf9c0f66e0 msgr2=0x7fdf9c107070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:17:57.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.329+0000 7fdfa0d06700 1 --2- 172.21.15.27:0/1682474918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf9c0f66e0 0x7fdf9c107070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:57.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.329+0000 7fdfa0d06700 1 -- 172.21.15.27:0/1682474918 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf94009640 con 0x7fdf9c0f6ff0 2021-11-03T18:17:57.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.330+0000 7fdfa0d06700 1 --2- 172.21.15.27:0/1682474918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf9c0f6ff0 0x7fdf9c107560 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 rx=0x7fdf9000c790 tx=0x7fdf90007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:17:57.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.330+0000 7fdf99ffb700 1 -- 172.21.15.27:0/1682474918 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf9000e680 con 0x7fdf9c0f6ff0 2021-11-03T18:17:57.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.330+0000 7fdf99ffb700 1 -- 172.21.15.27:0/1682474918 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fdf9000ec90 con 0x7fdf9c0f6ff0 2021-11-03T18:17:57.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.330+0000 7fdf99ffb700 1 -- 172.21.15.27:0/1682474918 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdf90013d60 con 0x7fdf9c0f6ff0 2021-11-03T18:17:57.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.330+0000 7fdfa376b700 1 -- 172.21.15.27:0/1682474918 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf9c1085f0 con 0x7fdf9c0f6ff0 2021-11-03T18:17:57.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.330+0000 7fdfa376b700 1 -- 172.21.15.27:0/1682474918 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf9c108b30 con 0x7fdf9c0f6ff0 2021-11-03T18:17:57.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.331+0000 7fdfa376b700 1 -- 172.21.15.27:0/1682474918 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf9c045ad0 con 0x7fdf9c0f6ff0 2021-11-03T18:17:57.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.331+0000 7fdf99ffb700 1 -- 172.21.15.27:0/1682474918 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fdf9001a060 con 0x7fdf9c0f6ff0 2021-11-03T18:17:57.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.332+0000 7fdf99ffb700 1 --2- 172.21.15.27:0/1682474918 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdf840640b0 0x7fdf84066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:17:57.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.332+0000 7fdf99ffb700 1 -- 172.21.15.27:0/1682474918 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fdf90080a80 con 0x7fdf9c0f6ff0 2021-11-03T18:17:57.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.332+0000 7fdfa1507700 1 --2- 172.21.15.27:0/1682474918 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdf840640b0 0x7fdf84066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:17:57.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.332+0000 7fdfa1507700 1 --2- 172.21.15.27:0/1682474918 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdf840640b0 0x7fdf84066570 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7fdf9400b3b0 tx=0x7fdf94004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:17:57.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.336+0000 7fdf99ffb700 1 -- 172.21.15.27:0/1682474918 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fdf90052c30 con 0x7fdf9c0f6ff0 2021-11-03T18:17:57.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.563+0000 7fdfa376b700 1 -- 172.21.15.27:0/1682474918 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fdf9c027a00 con 0x7fdf9c0f6ff0 2021-11-03T18:17:57.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.563+0000 7fdf99ffb700 1 -- 172.21.15.27:0/1682474918 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fdf90018070 con 0x7fdf9c0f6ff0 2021-11-03T18:17:57.562 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:17:57.563 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:17:57.563 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:17:57.563 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:17:57.563 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:17:57.564 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:17:57.564 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:17:57.564 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:17:57.564 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:17:57.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:17:57.565 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:17:57.565 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:17:57.565 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:17:57.565 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:17:57.566 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:17:57.566 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:17:57.566 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:17:57.566 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:17:57.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.565+0000 7fdfa376b700 1 -- 172.21.15.27:0/1682474918 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdf840640b0 msgr2=0x7fdf84066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:17:57.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.565+0000 7fdfa376b700 1 --2- 172.21.15.27:0/1682474918 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdf840640b0 0x7fdf84066570 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7fdf9400b3b0 tx=0x7fdf94004d10).stop 2021-11-03T18:17:57.568 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.565+0000 7fdfa376b700 1 -- 172.21.15.27:0/1682474918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf9c0f6ff0 msgr2=0x7fdf9c107560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:17:57.568 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.565+0000 7fdfa376b700 1 --2- 172.21.15.27:0/1682474918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf9c0f6ff0 0x7fdf9c107560 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 rx=0x7fdf9000c790 tx=0x7fdf90007d70).stop 2021-11-03T18:17:57.568 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.565+0000 7fdfa376b700 1 -- 172.21.15.27:0/1682474918 shutdown_connections 2021-11-03T18:17:57.568 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.565+0000 7fdfa376b700 1 --2- 172.21.15.27:0/1682474918 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdf840640b0 0x7fdf84066570 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:57.569 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.565+0000 7fdfa376b700 1 --2- 172.21.15.27:0/1682474918 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf9c0f66e0 0x7fdf9c107070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:57.569 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.565+0000 7fdfa376b700 1 --2- 172.21.15.27:0/1682474918 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf9c0f6ff0 0x7fdf9c107560 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:57.569 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.565+0000 7fdfa376b700 1 -- 172.21.15.27:0/1682474918 >> 172.21.15.27:0/1682474918 conn(0x7fdf9c0f1f00 msgr2=0x7fdf9c0fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:17:57.569 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.565+0000 7fdfa376b700 1 -- 172.21.15.27:0/1682474918 shutdown_connections 2021-11-03T18:17:57.570 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.565+0000 7fdfa376b700 1 -- 172.21.15.27:0/1682474918 wait complete. 2021-11-03T18:17:57.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.665+0000 7f17cce9d700 1 -- 172.21.15.27:0/2943309965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17c80f66e0 msgr2=0x7f17c80f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:17:57.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.665+0000 7f17cce9d700 1 --2- 172.21.15.27:0/2943309965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17c80f66e0 0x7f17c80f6b40 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 rx=0x7f17b8004660 tx=0x7f17b8009b30).stop 2021-11-03T18:17:57.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.665+0000 7f17cce9d700 1 -- 172.21.15.27:0/2943309965 shutdown_connections 2021-11-03T18:17:57.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.665+0000 7f17cce9d700 1 --2- 172.21.15.27:0/2943309965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17c80f66e0 0x7f17c80f6b40 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:57.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.665+0000 7f17cce9d700 1 --2- 172.21.15.27:0/2943309965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17c80f5440 0x7f17c80f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T18:17:57.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.665+0000 7f17cce9d700 1 -- 172.21.15.27:0/2943309965 >> 172.21.15.27:0/2943309965 conn(0x7f17c80f0ba0 msgr2=0x7f17c80f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:17:57.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.665+0000 7f17cce9d700 1 -- 172.21.15.27:0/2943309965 shutdown_connections 2021-11-03T18:17:57.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.665+0000 7f17cce9d700 1 -- 172.21.15.27:0/2943309965 wait complete. 2021-11-03T18:17:57.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.666+0000 7f17cce9d700 1 Processor -- start 2021-11-03T18:17:57.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.666+0000 7f17cce9d700 1 -- start start 2021-11-03T18:17:57.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.666+0000 7f17cce9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17c80f5440 0x7f17c8109a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:17:57.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.666+0000 7f17cce9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17c80f66e0 0x7f17c810bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:17:57.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.666+0000 7f17cce9d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c8060470 con 0x7f17c80f5440 2021-11-03T18:17:57.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.666+0000 7f17cce9d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c80605b0 con 0x7f17c80f66e0 2021-11-03T18:17:57.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.666+0000 7f17c77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17c80f5440 0x7f17c8109a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:17:57.670 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.666+0000 7f17c77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17c80f5440 0x7f17c8109a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34092/0 (socket says 172.21.15.27:34092) 2021-11-03T18:17:57.670 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.666+0000 7f17c77fe700 1 -- 172.21.15.27:0/2912874963 learned_addr learned my addr 172.21.15.27:0/2912874963 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:17:57.670 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.667+0000 7f17c6ffd700 1 --2- 172.21.15.27:0/2912874963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17c80f66e0 0x7f17c810bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:17:57.670 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.667+0000 7f17c77fe700 1 -- 172.21.15.27:0/2912874963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17c80f66e0 msgr2=0x7f17c810bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:17:57.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.667+0000 7f17c77fe700 1 --2- 172.21.15.27:0/2912874963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17c80f66e0 0x7f17c810bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:57.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.667+0000 7f17c77fe700 1 -- 172.21.15.27:0/2912874963 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b8005040 con 0x7f17c80f5440 2021-11-03T18:17:57.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.667+0000 7f17c77fe700 1 --2- 172.21.15.27:0/2912874963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17c80f5440 0x7f17c8109a80 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 rx=0x7f17bc0093f0 tx=0x7f17bc006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:17:57.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.667+0000 7f17affff700 1 -- 172.21.15.27:0/2912874963 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17bc009ea0 con 0x7f17c80f5440 2021-11-03T18:17:57.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.667+0000 7f17affff700 1 -- 172.21.15.27:0/2912874963 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f17bc013af0 con 0x7f17c80f5440 2021-11-03T18:17:57.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.667+0000 7f17cce9d700 1 -- 172.21.15.27:0/2912874963 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c810c5e0 con 0x7f17c80f5440 2021-11-03T18:17:57.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.667+0000 7f17affff700 1 -- 172.21.15.27:0/2912874963 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17bc00db10 con 0x7f17c80f5440 2021-11-03T18:17:57.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.667+0000 7f17cce9d700 1 -- 172.21.15.27:0/2912874963 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c810cb70 con 0x7f17c80f5440 2021-11-03T18:17:57.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.668+0000 7f17cce9d700 1 -- 172.21.15.27:0/2912874963 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17c8059100 con 0x7f17c80f5440 2021-11-03T18:17:57.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.668+0000 7f17affff700 1 -- 172.21.15.27:0/2912874963 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f17bc01a060 con 0x7f17c80f5440 2021-11-03T18:17:57.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.669+0000 7f17affff700 1 --2- 172.21.15.27:0/2912874963 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f17b0064100 0x7f17b00665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:17:57.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.669+0000 7f17affff700 1 -- 172.21.15.27:0/2912874963 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f17bc080940 con 0x7f17c80f5440 2021-11-03T18:17:57.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.669+0000 7f17c6ffd700 1 --2- 172.21.15.27:0/2912874963 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f17b0064100 0x7f17b00665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:17:57.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.669+0000 7f17c6ffd700 1 --2- 172.21.15.27:0/2912874963 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f17b0064100 0x7f17b00665c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f17b8000c00 tx=0x7f17b8009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:17:57.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.673+0000 7f17affff700 1 -- 172.21.15.27:0/2912874963 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f17bc052b50 con 0x7f17c80f5440 2021-11-03T18:17:57.874 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.875+0000 7f17cce9d700 1 -- 172.21.15.27:0/2912874963 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f17c8045ad0 con 0x7f17c80f5440 2021-11-03T18:17:57.874 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.875+0000 7f17affff700 1 -- 172.21.15.27:0/2912874963 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f17bc018710 con 0x7f17c80f5440 2021-11-03T18:17:57.875 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:17:57.875 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:17:57.876 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:17:57.876 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:17:57.877 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:17:57.877 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:17:57.877 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:17:57.877 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:17:57.878 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:17:57.878 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:17:57.878 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:17:57.878 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:17:57.879 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:17:57.879 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:17:57.879 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:17:57.879 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:17:57.880 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:17:57.880 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:17:57.880 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:17:57.880 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:17:57.881 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:17:57.881 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:17:57.881 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:17:57.881 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:17:57.882 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:17:57.882 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:17:57.882 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:17:57.882 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:17:57.883 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:17:57.883 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:17:57.883 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:17:57.883 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:17:57.884 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:17:57.884 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:17:57.884 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:17:57.884 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:17:57.885 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:17:57.885 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:17:57.885 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:17:57.886 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.877+0000 7f17cce9d700 1 -- 172.21.15.27:0/2912874963 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f17b0064100 msgr2=0x7f17b00665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:17:57.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.877+0000 7f17cce9d700 1 --2- 172.21.15.27:0/2912874963 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f17b0064100 0x7f17b00665c0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f17b8000c00 tx=0x7f17b8009b30).stop 2021-11-03T18:17:57.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.877+0000 7f17cce9d700 1 -- 172.21.15.27:0/2912874963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17c80f5440 msgr2=0x7f17c8109a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:17:57.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.877+0000 7f17cce9d700 1 --2- 172.21.15.27:0/2912874963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17c80f5440 0x7f17c8109a80 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 rx=0x7f17bc0093f0 tx=0x7f17bc006ec0).stop 2021-11-03T18:17:57.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.877+0000 7f17cce9d700 1 -- 172.21.15.27:0/2912874963 shutdown_connections 2021-11-03T18:17:57.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.877+0000 7f17cce9d700 1 --2- 172.21.15.27:0/2912874963 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f17b0064100 0x7f17b00665c0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:57.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.877+0000 7f17cce9d700 1 --2- 172.21.15.27:0/2912874963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f17c80f5440 0x7f17c8109a80 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:57.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.877+0000 7f17cce9d700 1 --2- 172.21.15.27:0/2912874963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f17c80f66e0 0x7f17c810bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:17:57.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.877+0000 7f17cce9d700 1 -- 172.21.15.27:0/2912874963 >> 172.21.15.27:0/2912874963 conn(0x7f17c80f0ba0 msgr2=0x7f17c80fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:17:57.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.877+0000 7f17cce9d700 1 -- 172.21.15.27:0/2912874963 shutdown_connections 2021-11-03T18:17:57.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:17:57.878+0000 7f17cce9d700 1 -- 172.21.15.27:0/2912874963 wait complete. 2021-11-03T18:17:57.893 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:17:58.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:58 smithi027 ceph-mon[97244]: pgmap v1899: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:58.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:58 smithi027 ceph-mon[97244]: from='client.36056 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:17:58.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:58 smithi027 ceph-mon[97244]: from='client.36060 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:17:58.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:58 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1682474918' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:58.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:17:58 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2912874963' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:17:58.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:58 smithi115 ceph-mon[65453]: pgmap v1899: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:17:58.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:58 smithi115 ceph-mon[65453]: from='client.36056 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:17:58.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:58 smithi115 ceph-mon[65453]: from='client.36060 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:17:58.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:58 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1682474918' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:17:58.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:17:58 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2912874963' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:18:00.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:00 smithi027 ceph-mon[97244]: pgmap v1900: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:00.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3722.449125 secs 2021-11-03T18:18:00.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:00 smithi115 ceph-mon[65453]: pgmap v1900: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:00.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3722.449125 secs 2021-11-03T18:18:01.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:18:01.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:18:01.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:18:01.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:01.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:01.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:01.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:02.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:18:02.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:18:02.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:18:02.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:02.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:02.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:02.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:02.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:02 smithi027 ceph-mon[97244]: pgmap v1901: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:02.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:02 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:18:02.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:02 smithi115 ceph-mon[65453]: pgmap v1901: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:02.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:02 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:18:04.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:04 smithi115 ceph-mon[65453]: pgmap v1902: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:05.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:04 smithi027 ceph-mon[97244]: pgmap v1902: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:05.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3727.449248 secs 2021-11-03T18:18:06.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3727.449248 secs 2021-11-03T18:18:06.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:06 smithi115 ceph-mon[65453]: pgmap v1903: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:07.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:06 smithi027 ceph-mon[97244]: pgmap v1903: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:08.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:08 smithi115 ceph-mon[65453]: pgmap v1904: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:09.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:08 smithi027 ceph-mon[97244]: pgmap v1904: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:10.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:10 smithi115 ceph-mon[65453]: pgmap v1905: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:10.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3732.449374 secs 2021-11-03T18:18:11.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:10 smithi027 ceph-mon[97244]: pgmap v1905: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:11.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3732.449374 secs 2021-11-03T18:18:11.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:18:11.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:18:12.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:18:12.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:18:12.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:12 smithi027 ceph-mon[97244]: pgmap v1906: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:12.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:12 smithi027 ceph-mon[97244]: poll took 50155.058 ms: 1 events 2021-11-03T18:18:12.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:18:12.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:12.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:12.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:12.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:12.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:12 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:18:12.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:12 smithi115 ceph-mon[65453]: pgmap v1906: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:12.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:12 smithi115 ceph-mon[65453]: poll took 50155.058 ms: 1 events 2021-11-03T18:18:12.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:18:12.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:12.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:12.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:12.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:12.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:12 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:18:14.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:14 smithi115 ceph-mon[65453]: pgmap v1907: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:15.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:14 smithi027 ceph-mon[97244]: pgmap v1907: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:15.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3737.449503 secs 2021-11-03T18:18:16.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3737.449503 secs 2021-11-03T18:18:16.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:16 smithi115 ceph-mon[65453]: pgmap v1908: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:17.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:16 smithi027 ceph-mon[97244]: pgmap v1908: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:18.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:18 smithi115 ceph-mon[65453]: pgmap v1909: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:19.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:18 smithi027 ceph-mon[97244]: pgmap v1909: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:20.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:20 smithi115 ceph-mon[65453]: pgmap v1910: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:20.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3742.449592 secs 2021-11-03T18:18:21.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:20 smithi027 ceph-mon[97244]: pgmap v1910: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:21.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3742.449592 secs 2021-11-03T18:18:21.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:18:22.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:18:22.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:22 smithi027 ceph-mon[97244]: pgmap v1911: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:22.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:18:22.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:18:22.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:22.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:22.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:22.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:22.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:22 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:18:22.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:22 smithi115 ceph-mon[65453]: pgmap v1911: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:22.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:18:22.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:18:22.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:22.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:22.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:22.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:22.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:22 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:18:24.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:24 smithi115 ceph-mon[65453]: pgmap v1912: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:25.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:24 smithi027 ceph-mon[97244]: pgmap v1912: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:25.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3747.449656 secs 2021-11-03T18:18:26.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3747.449656 secs 2021-11-03T18:18:26.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:26 smithi115 ceph-mon[65453]: pgmap v1913: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:27.027 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:26 smithi027 ceph-mon[97244]: pgmap v1913: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:27.992 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:27.993+0000 7f6ed9565700 1 -- 172.21.15.27:0/1409248432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ed40f6860 msgr2=0x7f6ed40f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:18:27.992 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:27.993+0000 7f6ed9565700 1 --2- 172.21.15.27:0/1409248432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ed40f6860 0x7f6ed40f6cc0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 rx=0x7f6ebc004660 tx=0x7f6ebc009b30).stop 2021-11-03T18:18:27.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:27.993+0000 7f6ed9565700 1 -- 172.21.15.27:0/1409248432 shutdown_connections 2021-11-03T18:18:27.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:27.993+0000 7f6ed9565700 1 --2- 172.21.15.27:0/1409248432 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ed40f6860 0x7f6ed40f6cc0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:18:27.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:27.993+0000 7f6ed9565700 1 --2- 172.21.15.27:0/1409248432 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ed40f55c0 0x7f6ed40f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:18:27.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:27.993+0000 7f6ed9565700 1 -- 172.21.15.27:0/1409248432 >> 172.21.15.27:0/1409248432 conn(0x7f6ed40f0d40 msgr2=0x7f6ed40f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:18:27.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:27.993+0000 7f6ed9565700 1 -- 172.21.15.27:0/1409248432 shutdown_connections 2021-11-03T18:18:27.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:27.993+0000 7f6ed9565700 1 -- 172.21.15.27:0/1409248432 wait complete. 2021-11-03T18:18:27.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:27.994+0000 7f6ed9565700 1 Processor -- start 2021-11-03T18:18:27.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:27.994+0000 7f6ed9565700 1 -- start start 2021-11-03T18:18:27.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:27.994+0000 7f6ed9565700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ed40f55c0 0x7f6ed4109180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:18:27.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:27.994+0000 7f6ed9565700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ed40f6860 0x7f6ed4109670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:18:27.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:27.994+0000 7f6ed9565700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed4109c20 con 0x7f6ed40f55c0 2021-11-03T18:18:27.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:27.994+0000 7f6ed9565700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ed4109d60 con 0x7f6ed40f6860 2021-11-03T18:18:27.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:27.994+0000 7f6ed2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ed40f55c0 0x7f6ed4109180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:18:27.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:27.994+0000 7f6ed2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ed40f55c0 0x7f6ed4109180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34112/0 (socket says 172.21.15.27:34112) 2021-11-03T18:18:27.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:27.994+0000 7f6ed2ffd700 1 -- 172.21.15.27:0/3776564859 learned_addr learned my addr 172.21.15.27:0/3776564859 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:18:27.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:27.994+0000 7f6ed27fc700 1 --2- 172.21.15.27:0/3776564859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ed40f6860 0x7f6ed4109670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:18:27.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:27.994+0000 7f6ed2ffd700 1 -- 172.21.15.27:0/3776564859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ed40f6860 msgr2=0x7f6ed4109670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:18:27.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:27.994+0000 7f6ed2ffd700 1 --2- 172.21.15.27:0/3776564859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ed40f6860 0x7f6ed4109670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:18:27.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:27.994+0000 7f6ed2ffd700 1 -- 172.21.15.27:0/3776564859 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ebc005040 con 0x7f6ed40f55c0 2021-11-03T18:18:27.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:27.995+0000 7f6ed2ffd700 1 --2- 172.21.15.27:0/3776564859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ed40f55c0 0x7f6ed4109180 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 rx=0x7f6ecc00d7f0 tx=0x7f6ecc004e80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:18:28.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:27.995+0000 7f6ecb7fe700 1 -- 172.21.15.27:0/3776564859 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ecc00b4f0 con 0x7f6ed40f55c0 2021-11-03T18:18:28.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:27.995+0000 7f6ecb7fe700 1 -- 172.21.15.27:0/3776564859 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f6ecc00f040 con 0x7f6ed40f55c0 2021-11-03T18:18:28.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:27.995+0000 7f6ecb7fe700 1 -- 172.21.15.27:0/3776564859 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ecc013980 con 0x7f6ed40f55c0 2021-11-03T18:18:28.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:27.995+0000 7f6ed9565700 1 -- 172.21.15.27:0/3776564859 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ed410a700 con 0x7f6ed40f55c0 2021-11-03T18:18:28.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:27.995+0000 7f6ed9565700 1 -- 172.21.15.27:0/3776564859 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ed410acc0 con 0x7f6ed40f55c0 2021-11-03T18:18:28.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:27.996+0000 7f6ed9565700 1 -- 172.21.15.27:0/3776564859 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ed4045ad0 con 0x7f6ed40f55c0 2021-11-03T18:18:28.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:27.996+0000 7f6ecb7fe700 1 -- 172.21.15.27:0/3776564859 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f6ecc01a030 con 0x7f6ed40f55c0 2021-11-03T18:18:28.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:27.997+0000 7f6ecb7fe700 1 --2- 172.21.15.27:0/3776564859 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6ec00640b0 0x7f6ec0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:18:28.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:27.997+0000 7f6ecb7fe700 1 -- 172.21.15.27:0/3776564859 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f6ecc0807e0 con 0x7f6ed40f55c0 2021-11-03T18:18:28.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:27.997+0000 7f6ed27fc700 1 --2- 172.21.15.27:0/3776564859 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6ec00640b0 0x7f6ec0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:18:28.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:27.997+0000 7f6ed27fc700 1 --2- 172.21.15.27:0/3776564859 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6ec00640b0 0x7f6ec0066570 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7f6ebc009da0 tx=0x7f6ebc009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:18:28.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.001+0000 7f6ecb7fe700 1 -- 172.21.15.27:0/3776564859 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f6ecc0529f0 con 0x7f6ed40f55c0 2021-11-03T18:18:28.180 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.182+0000 7f6ed9565700 1 -- 172.21.15.27:0/3776564859 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ed40faaa0 con 0x7f6ec00640b0 2021-11-03T18:18:28.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.183+0000 7f6ecb7fe700 1 -- 172.21.15.27:0/3776564859 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f6ed40faaa0 con 0x7f6ec00640b0 2021-11-03T18:18:28.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.185+0000 7f6ed9565700 1 -- 172.21.15.27:0/3776564859 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6ec00640b0 msgr2=0x7f6ec0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:18:28.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.185+0000 7f6ed9565700 1 --2- 172.21.15.27:0/3776564859 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6ec00640b0 0x7f6ec0066570 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 rx=0x7f6ebc009da0 tx=0x7f6ebc009b30).stop 2021-11-03T18:18:28.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.185+0000 7f6ed9565700 1 -- 172.21.15.27:0/3776564859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ed40f55c0 msgr2=0x7f6ed4109180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:18:28.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.185+0000 7f6ed9565700 1 --2- 172.21.15.27:0/3776564859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ed40f55c0 0x7f6ed4109180 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 rx=0x7f6ecc00d7f0 tx=0x7f6ecc004e80).stop 2021-11-03T18:18:28.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.185+0000 7f6ed9565700 1 -- 172.21.15.27:0/3776564859 shutdown_connections 2021-11-03T18:18:28.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.185+0000 7f6ed9565700 1 --2- 172.21.15.27:0/3776564859 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6ec00640b0 0x7f6ec0066570 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:18:28.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.185+0000 7f6ed9565700 1 --2- 172.21.15.27:0/3776564859 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ed40f55c0 0x7f6ed4109180 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:18:28.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.185+0000 7f6ed9565700 1 --2- 172.21.15.27:0/3776564859 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ed40f6860 0x7f6ed4109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:18:28.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.185+0000 7f6ed9565700 1 -- 172.21.15.27:0/3776564859 >> 172.21.15.27:0/3776564859 conn(0x7f6ed40f0d40 msgr2=0x7f6ed405fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:18:28.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.185+0000 7f6ed9565700 1 -- 172.21.15.27:0/3776564859 shutdown_connections 2021-11-03T18:18:28.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.185+0000 7f6ed9565700 1 -- 172.21.15.27:0/3776564859 wait complete. 2021-11-03T18:18:28.193 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:18:28.286 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.285+0000 7f9024c94700 1 -- 172.21.15.27:0/3302132994 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90200f55e0 msgr2=0x7f90200f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:18:28.286 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.285+0000 7f9024c94700 1 --2- 172.21.15.27:0/3302132994 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90200f55e0 0x7f90200f5a00 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 rx=0x7f9008004660 tx=0x7f9008009b30).stop 2021-11-03T18:18:28.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.285+0000 7f9024c94700 1 -- 172.21.15.27:0/3302132994 shutdown_connections 2021-11-03T18:18:28.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.285+0000 7f9024c94700 1 --2- 172.21.15.27:0/3302132994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90200f5ef0 0x7f90200f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:18:28.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.285+0000 7f9024c94700 1 --2- 172.21.15.27:0/3302132994 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90200f55e0 0x7f90200f5a00 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:18:28.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.285+0000 7f9024c94700 1 -- 172.21.15.27:0/3302132994 >> 172.21.15.27:0/3302132994 conn(0x7f90200f0de0 msgr2=0x7f90200f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:18:28.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.285+0000 7f9024c94700 1 -- 172.21.15.27:0/3302132994 shutdown_connections 2021-11-03T18:18:28.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.285+0000 7f9024c94700 1 -- 172.21.15.27:0/3302132994 wait complete. 2021-11-03T18:18:28.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.285+0000 7f9024c94700 1 Processor -- start 2021-11-03T18:18:28.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.285+0000 7f9024c94700 1 -- start start 2021-11-03T18:18:28.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.286+0000 7f9024c94700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90200f55e0 0x7f9020109c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:18:28.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.286+0000 7f9024c94700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90200f5ef0 0x7f902010c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:18:28.290 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.286+0000 7f9024c94700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90200604e0 con 0x7f90200f55e0 2021-11-03T18:18:28.290 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.286+0000 7f9024c94700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9020060620 con 0x7f90200f5ef0 2021-11-03T18:18:28.290 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.286+0000 7f901e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90200f55e0 0x7f9020109c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:18:28.291 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.286+0000 7f901e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90200f55e0 0x7f9020109c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34122/0 (socket says 172.21.15.27:34122) 2021-11-03T18:18:28.291 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.286+0000 7f901e59c700 1 -- 172.21.15.27:0/1278619843 learned_addr learned my addr 172.21.15.27:0/1278619843 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:18:28.291 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.286+0000 7f901dd9b700 1 --2- 172.21.15.27:0/1278619843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90200f5ef0 0x7f902010c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:18:28.291 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.286+0000 7f901e59c700 1 -- 172.21.15.27:0/1278619843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90200f5ef0 msgr2=0x7f902010c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:18:28.291 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.286+0000 7f901e59c700 1 --2- 172.21.15.27:0/1278619843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90200f5ef0 0x7f902010c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:18:28.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.286+0000 7f901e59c700 1 -- 172.21.15.27:0/1278619843 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9008005040 con 0x7f90200f55e0 2021-11-03T18:18:28.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.286+0000 7f901dd9b700 1 --2- 172.21.15.27:0/1278619843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90200f5ef0 0x7f902010c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T18:18:28.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.286+0000 7f901e59c700 1 --2- 172.21.15.27:0/1278619843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90200f55e0 0x7f9020109c90 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 rx=0x7f9008009990 tx=0x7f9008009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:18:28.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.286+0000 7f9016ffd700 1 -- 172.21.15.27:0/1278619843 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9008018070 con 0x7f90200f55e0 2021-11-03T18:18:28.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.286+0000 7f9024c94700 1 -- 172.21.15.27:0/1278619843 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f902010c6e0 con 0x7f90200f55e0 2021-11-03T18:18:28.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.286+0000 7f9016ffd700 1 -- 172.21.15.27:0/1278619843 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f9008007ca0 con 0x7f90200f55e0 2021-11-03T18:18:28.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.286+0000 7f9016ffd700 1 -- 172.21.15.27:0/1278619843 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f900801c620 con 0x7f90200f55e0 2021-11-03T18:18:28.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.286+0000 7f9024c94700 1 -- 172.21.15.27:0/1278619843 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f902010cc40 con 0x7f90200f55e0 2021-11-03T18:18:28.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.287+0000 7f9024c94700 1 -- 172.21.15.27:0/1278619843 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9020045ad0 con 0x7f90200f55e0 2021-11-03T18:18:28.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.288+0000 7f9016ffd700 1 -- 172.21.15.27:0/1278619843 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f900801d960 con 0x7f90200f55e0 2021-11-03T18:18:28.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.288+0000 7f9016ffd700 1 --2- 172.21.15.27:0/1278619843 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f900c0640b0 0x7f900c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:18:28.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.288+0000 7f9016ffd700 1 -- 172.21.15.27:0/1278619843 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f9008082640 con 0x7f90200f55e0 2021-11-03T18:18:28.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.288+0000 7f901dd9b700 1 --2- 172.21.15.27:0/1278619843 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f900c0640b0 0x7f900c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:18:28.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.289+0000 7f901dd9b700 1 --2- 172.21.15.27:0/1278619843 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f900c0640b0 0x7f900c066570 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7f90100041d0 tx=0x7f9010009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:18:28.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.292+0000 7f9016ffd700 1 -- 172.21.15.27:0/1278619843 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f9008054940 con 0x7f90200f55e0 2021-11-03T18:18:28.463 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.464+0000 7f9024c94700 1 -- 172.21.15.27:0/1278619843 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f90200fdf50 con 0x7f900c0640b0 2021-11-03T18:18:28.477 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.479+0000 7f9016ffd700 1 -- 172.21.15.27:0/1278619843 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f90200fdf50 con 0x7f900c0640b0 2021-11-03T18:18:28.478 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:18:28.478 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (70m) 9m ago 71m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:18:28.478 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (61m) 9m ago 71m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:18:28.479 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (61m) 8m ago 70m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:18:28.480 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (69m) 9m ago 70m 42.1M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:18:28.480 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (63m) 9m ago 66m 26.3M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:18:28.481 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (66m) 9m ago 66m 3847M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:18:28.481 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (62m) 8m ago 66m 25.9M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:18:28.481 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (63m) 8m ago 66m 27.0M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:18:28.481 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (63m) 9m ago 72m 572M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:18:28.481 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (63m) 8m ago 70m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:18:28.482 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (62m) 9m ago 72m 273M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:18:28.482 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (62m) 8m ago 70m 129M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:18:28.482 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (70m) 9m ago 70m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:18:28.482 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (70m) 8m ago 70m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:18:28.483 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (61m) 9m ago 69m 371M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:18:28.483 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (60m) 9m ago 69m 405M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:18:28.483 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (60m) 9m ago 68m 302M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:18:28.483 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (59m) 8m ago 68m 429M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:18:28.484 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (59m) 8m ago 68m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:18:28.484 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (58m) 8m ago 68m 348M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:18:28.484 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (69m) 9m ago 70m 57.6M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:18:28.485 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.480+0000 7f9024c94700 1 -- 172.21.15.27:0/1278619843 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f900c0640b0 msgr2=0x7f900c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:18:28.485 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.480+0000 7f9024c94700 1 --2- 172.21.15.27:0/1278619843 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f900c0640b0 0x7f900c066570 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7f90100041d0 tx=0x7f9010009040).stop 2021-11-03T18:18:28.486 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.480+0000 7f9024c94700 1 -- 172.21.15.27:0/1278619843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90200f55e0 msgr2=0x7f9020109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:18:28.486 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.480+0000 7f9024c94700 1 --2- 172.21.15.27:0/1278619843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90200f55e0 0x7f9020109c90 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 rx=0x7f9008009990 tx=0x7f9008009b30).stop 2021-11-03T18:18:28.486 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.481+0000 7f9024c94700 1 -- 172.21.15.27:0/1278619843 shutdown_connections 2021-11-03T18:18:28.486 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.481+0000 7f9024c94700 1 --2- 172.21.15.27:0/1278619843 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f900c0640b0 0x7f900c066570 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:18:28.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.481+0000 7f9024c94700 1 --2- 172.21.15.27:0/1278619843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90200f55e0 0x7f9020109c90 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:18:28.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.481+0000 7f9024c94700 1 --2- 172.21.15.27:0/1278619843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90200f5ef0 0x7f902010c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:18:28.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.481+0000 7f9024c94700 1 -- 172.21.15.27:0/1278619843 >> 172.21.15.27:0/1278619843 conn(0x7f90200f0de0 msgr2=0x7f90200fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:18:28.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.481+0000 7f9024c94700 1 -- 172.21.15.27:0/1278619843 shutdown_connections 2021-11-03T18:18:28.488 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.481+0000 7f9024c94700 1 -- 172.21.15.27:0/1278619843 wait complete. 2021-11-03T18:18:28.580 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.580+0000 7f00fefcb700 1 -- 172.21.15.27:0/36403117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00f80f6ff0 msgr2=0x7f00f80f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:18:28.581 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.580+0000 7f00fefcb700 1 --2- 172.21.15.27:0/36403117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00f80f6ff0 0x7f00f80f7a20 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 rx=0x7f00ec004660 tx=0x7f00ec009b30).stop 2021-11-03T18:18:28.581 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.580+0000 7f00fefcb700 1 -- 172.21.15.27:0/36403117 shutdown_connections 2021-11-03T18:18:28.581 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.580+0000 7f00fefcb700 1 --2- 172.21.15.27:0/36403117 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00f80f6ff0 0x7f00f80f7a20 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:18:28.582 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.580+0000 7f00fefcb700 1 --2- 172.21.15.27:0/36403117 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00f80f66e0 0x7f00f80f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:18:28.582 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.580+0000 7f00fefcb700 1 -- 172.21.15.27:0/36403117 >> 172.21.15.27:0/36403117 conn(0x7f00f80f1f00 msgr2=0x7f00f80f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:18:28.582 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.580+0000 7f00fefcb700 1 -- 172.21.15.27:0/36403117 shutdown_connections 2021-11-03T18:18:28.582 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.580+0000 7f00fefcb700 1 -- 172.21.15.27:0/36403117 wait complete. 2021-11-03T18:18:28.582 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.580+0000 7f00fefcb700 1 Processor -- start 2021-11-03T18:18:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.580+0000 7f00fefcb700 1 -- start start 2021-11-03T18:18:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.581+0000 7f00fefcb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00f80f66e0 0x7f00f806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:18:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.581+0000 7f00fefcb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00f80f6ff0 0x7f00f8069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:18:28.583 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.581+0000 7f00fefcb700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f806a380 con 0x7f00f80f66e0 2021-11-03T18:18:28.584 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.581+0000 7f00fefcb700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f806a4c0 con 0x7f00f80f6ff0 2021-11-03T18:18:28.584 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.581+0000 7f00fcd67700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00f80f66e0 0x7f00f806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:18:28.584 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.581+0000 7f00fcd67700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00f80f66e0 0x7f00f806b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34132/0 (socket says 172.21.15.27:34132) 2021-11-03T18:18:28.585 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.581+0000 7f00fcd67700 1 -- 172.21.15.27:0/320491732 learned_addr learned my addr 172.21.15.27:0/320491732 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:18:28.585 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.581+0000 7f00f7fff700 1 --2- 172.21.15.27:0/320491732 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00f80f6ff0 0x7f00f8069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:18:28.585 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.581+0000 7f00fcd67700 1 -- 172.21.15.27:0/320491732 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00f80f6ff0 msgr2=0x7f00f8069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:18:28.585 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.581+0000 7f00fcd67700 1 --2- 172.21.15.27:0/320491732 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00f80f6ff0 0x7f00f8069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:18:28.586 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.581+0000 7f00fcd67700 1 -- 172.21.15.27:0/320491732 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ec005040 con 0x7f00f80f66e0 2021-11-03T18:18:28.586 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.581+0000 7f00fcd67700 1 --2- 172.21.15.27:0/320491732 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00f80f66e0 0x7f00f806b9a0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 rx=0x7f00f000eb70 tx=0x7f00f000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:18:28.586 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.581+0000 7f00f57fa700 1 -- 172.21.15.27:0/320491732 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00f00092e0 con 0x7f00f80f66e0 2021-11-03T18:18:28.586 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.581+0000 7f00f57fa700 1 -- 172.21.15.27:0/320491732 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f00f000f040 con 0x7f00f80f66e0 2021-11-03T18:18:28.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.582+0000 7f00f57fa700 1 -- 172.21.15.27:0/320491732 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00f000c930 con 0x7f00f80f66e0 2021-11-03T18:18:28.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.582+0000 7f00fefcb700 1 -- 172.21.15.27:0/320491732 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00f806a7a0 con 0x7f00f80f66e0 2021-11-03T18:18:28.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.582+0000 7f00fefcb700 1 -- 172.21.15.27:0/320491732 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00f810ef20 con 0x7f00f80f66e0 2021-11-03T18:18:28.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.583+0000 7f00fefcb700 1 -- 172.21.15.27:0/320491732 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00f80f5b70 con 0x7f00f80f66e0 2021-11-03T18:18:28.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.583+0000 7f00f57fa700 1 -- 172.21.15.27:0/320491732 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f00f001b030 con 0x7f00f80f66e0 2021-11-03T18:18:28.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.583+0000 7f00f57fa700 1 --2- 172.21.15.27:0/320491732 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f00e00640b0 0x7f00e0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:18:28.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.583+0000 7f00f57fa700 1 -- 172.21.15.27:0/320491732 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f00f0080ce0 con 0x7f00f80f66e0 2021-11-03T18:18:28.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.584+0000 7f00f7fff700 1 --2- 172.21.15.27:0/320491732 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f00e00640b0 0x7f00e0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:18:28.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.584+0000 7f00f7fff700 1 --2- 172.21.15.27:0/320491732 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f00e00640b0 0x7f00e0066570 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f00ec005010 tx=0x7f00ec016040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:18:28.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.588+0000 7f00f57fa700 1 -- 172.21.15.27:0/320491732 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f00f0052ef0 con 0x7f00f80f66e0 2021-11-03T18:18:28.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.820+0000 7f00fefcb700 1 -- 172.21.15.27:0/320491732 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f00f80fef70 con 0x7f00f80f66e0 2021-11-03T18:18:28.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.821+0000 7f00f57fa700 1 -- 172.21.15.27:0/320491732 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f00f00192e0 con 0x7f00f80f66e0 2021-11-03T18:18:28.821 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:18:28.822 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:18:28.822 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:18:28.823 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:18:28.823 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:18:28.824 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:18:28.824 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:18:28.824 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:18:28.825 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:18:28.825 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:18:28.825 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:18:28.826 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:18:28.826 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:18:28.827 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:18:28.827 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:18:28.828 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:18:28.828 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:18:28.829 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:18:28.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.823+0000 7f00fefcb700 1 -- 172.21.15.27:0/320491732 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f00e00640b0 msgr2=0x7f00e0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:18:28.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.823+0000 7f00fefcb700 1 --2- 172.21.15.27:0/320491732 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f00e00640b0 0x7f00e0066570 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7f00ec005010 tx=0x7f00ec016040).stop 2021-11-03T18:18:28.831 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.823+0000 7f00fefcb700 1 -- 172.21.15.27:0/320491732 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00f80f66e0 msgr2=0x7f00f806b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:18:28.831 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.823+0000 7f00fefcb700 1 --2- 172.21.15.27:0/320491732 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00f80f66e0 0x7f00f806b9a0 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 rx=0x7f00f000eb70 tx=0x7f00f000b040).stop 2021-11-03T18:18:28.831 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.823+0000 7f00fefcb700 1 -- 172.21.15.27:0/320491732 shutdown_connections 2021-11-03T18:18:28.831 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.823+0000 7f00fefcb700 1 --2- 172.21.15.27:0/320491732 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f00e00640b0 0x7f00e0066570 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:18:28.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.823+0000 7f00fefcb700 1 --2- 172.21.15.27:0/320491732 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00f80f66e0 0x7f00f806b9a0 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:18:28.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.823+0000 7f00fefcb700 1 --2- 172.21.15.27:0/320491732 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00f80f6ff0 0x7f00f8069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:18:28.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.823+0000 7f00fefcb700 1 -- 172.21.15.27:0/320491732 >> 172.21.15.27:0/320491732 conn(0x7f00f80f1f00 msgr2=0x7f00f80fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:18:28.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.823+0000 7f00fefcb700 1 -- 172.21.15.27:0/320491732 shutdown_connections 2021-11-03T18:18:28.833 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.823+0000 7f00fefcb700 1 -- 172.21.15.27:0/320491732 wait complete. 2021-11-03T18:18:28.927 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.923+0000 7f36b11dc700 1 -- 172.21.15.27:0/2703305843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36ac0f6530 msgr2=0x7f36ac0f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:18:28.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.923+0000 7f36b11dc700 1 --2- 172.21.15.27:0/2703305843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36ac0f6530 0x7f36ac0f6950 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 rx=0x7f369c004660 tx=0x7f369c009b30).stop 2021-11-03T18:18:28.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.923+0000 7f36b11dc700 1 -- 172.21.15.27:0/2703305843 shutdown_connections 2021-11-03T18:18:28.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.923+0000 7f36b11dc700 1 --2- 172.21.15.27:0/2703305843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36ac0f77d0 0x7f36ac0f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:18:28.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.923+0000 7f36b11dc700 1 --2- 172.21.15.27:0/2703305843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36ac0f6530 0x7f36ac0f6950 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:18:28.929 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.923+0000 7f36b11dc700 1 -- 172.21.15.27:0/2703305843 >> 172.21.15.27:0/2703305843 conn(0x7f36ac0f1cd0 msgr2=0x7f36ac0f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:18:28.929 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.923+0000 7f36b11dc700 1 -- 172.21.15.27:0/2703305843 shutdown_connections 2021-11-03T18:18:28.929 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.923+0000 7f36b11dc700 1 -- 172.21.15.27:0/2703305843 wait complete. 2021-11-03T18:18:28.929 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.924+0000 7f36b11dc700 1 Processor -- start 2021-11-03T18:18:28.930 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.924+0000 7f36b11dc700 1 -- start start 2021-11-03T18:18:28.930 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.924+0000 7f36b11dc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36ac0f6530 0x7f36ac106dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:18:28.930 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.924+0000 7f36b11dc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36ac0f77d0 0x7f36ac1072c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:18:28.931 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.924+0000 7f36b11dc700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac1078c0 con 0x7f36ac0f77d0 2021-11-03T18:18:28.931 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.924+0000 7f36b11dc700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36ac108100 con 0x7f36ac0f6530 2021-11-03T18:18:28.931 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.925+0000 7f36ab7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36ac0f77d0 0x7f36ac1072c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:18:28.932 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.925+0000 7f36ab7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36ac0f77d0 0x7f36ac1072c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34142/0 (socket says 172.21.15.27:34142) 2021-11-03T18:18:28.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.925+0000 7f36ab7fe700 1 -- 172.21.15.27:0/2582335471 learned_addr learned my addr 172.21.15.27:0/2582335471 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:18:28.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.925+0000 7f36abfff700 1 --2- 172.21.15.27:0/2582335471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36ac0f6530 0x7f36ac106dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:18:28.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.925+0000 7f36ab7fe700 1 -- 172.21.15.27:0/2582335471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36ac0f6530 msgr2=0x7f36ac106dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:18:28.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.925+0000 7f36ab7fe700 1 --2- 172.21.15.27:0/2582335471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36ac0f6530 0x7f36ac106dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:18:28.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.925+0000 7f36ab7fe700 1 -- 172.21.15.27:0/2582335471 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f369c005040 con 0x7f36ac0f77d0 2021-11-03T18:18:28.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.925+0000 7f36ab7fe700 1 --2- 172.21.15.27:0/2582335471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36ac0f77d0 0x7f36ac1072c0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 rx=0x7f36a00026d0 tx=0x7f36a0009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:18:28.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.925+0000 7f36a8ff9700 1 -- 172.21.15.27:0/2582335471 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36a00079c0 con 0x7f36ac0f77d0 2021-11-03T18:18:28.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.925+0000 7f36b11dc700 1 -- 172.21.15.27:0/2582335471 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36ac1083e0 con 0x7f36ac0f77d0 2021-11-03T18:18:28.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.925+0000 7f36a8ff9700 1 -- 172.21.15.27:0/2582335471 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f36a000f040 con 0x7f36ac0f77d0 2021-11-03T18:18:28.935 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.925+0000 7f36a8ff9700 1 -- 172.21.15.27:0/2582335471 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36a00148b0 con 0x7f36ac0f77d0 2021-11-03T18:18:28.935 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.925+0000 7f36b11dc700 1 -- 172.21.15.27:0/2582335471 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36ac108920 con 0x7f36ac0f77d0 2021-11-03T18:18:28.935 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.926+0000 7f36b11dc700 1 -- 172.21.15.27:0/2582335471 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36ac059100 con 0x7f36ac0f77d0 2021-11-03T18:18:28.936 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.927+0000 7f36a8ff9700 1 -- 172.21.15.27:0/2582335471 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f36a0007b20 con 0x7f36ac0f77d0 2021-11-03T18:18:28.937 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.927+0000 7f36a8ff9700 1 --2- 172.21.15.27:0/2582335471 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3694064100 0x7f36940665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:18:28.937 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.927+0000 7f36a8ff9700 1 -- 172.21.15.27:0/2582335471 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f36a0080770 con 0x7f36ac0f77d0 2021-11-03T18:18:28.938 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.927+0000 7f36abfff700 1 --2- 172.21.15.27:0/2582335471 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3694064100 0x7f36940665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:18:28.938 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.927+0000 7f36abfff700 1 --2- 172.21.15.27:0/2582335471 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3694064100 0x7f36940665c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f369c013690 tx=0x7f369c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:18:28.939 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:28.931+0000 7f36a8ff9700 1 -- 172.21.15.27:0/2582335471 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f36a0052a70 con 0x7f36ac0f77d0 2021-11-03T18:18:28.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:28 smithi115 ceph-mon[65453]: pgmap v1914: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:29.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:28 smithi027 ceph-mon[97244]: pgmap v1914: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:29.136 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:29.137+0000 7f36b11dc700 1 -- 172.21.15.27:0/2582335471 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f36ac0fed20 con 0x7f36ac0f77d0 2021-11-03T18:18:29.137 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:29.138+0000 7f36a8ff9700 1 -- 172.21.15.27:0/2582335471 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f36a0052660 con 0x7f36ac0f77d0 2021-11-03T18:18:29.137 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:18:29.138 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:18:29.138 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:18:29.138 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:18:29.138 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:18:29.139 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:18:29.139 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:18:29.139 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:18:29.139 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:18:29.140 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:18:29.140 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:18:29.140 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:18:29.140 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:18:29.141 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:18:29.141 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:18:29.141 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:18:29.142 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:18:29.142 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:18:29.142 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:18:29.142 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:18:29.143 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:18:29.143 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:18:29.143 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:18:29.143 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:18:29.144 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:18:29.144 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:18:29.144 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:18:29.144 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:18:29.145 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:18:29.145 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:18:29.145 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:18:29.145 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:18:29.145 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:18:29.146 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:18:29.146 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:18:29.146 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:18:29.147 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:18:29.147 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:18:29.147 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:18:29.148 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:29.139+0000 7f36b11dc700 1 -- 172.21.15.27:0/2582335471 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3694064100 msgr2=0x7f36940665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:18:29.149 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:29.139+0000 7f36b11dc700 1 --2- 172.21.15.27:0/2582335471 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3694064100 0x7f36940665c0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 rx=0x7f369c013690 tx=0x7f369c009b30).stop 2021-11-03T18:18:29.149 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:29.139+0000 7f36b11dc700 1 -- 172.21.15.27:0/2582335471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36ac0f77d0 msgr2=0x7f36ac1072c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:18:29.149 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:29.139+0000 7f36b11dc700 1 --2- 172.21.15.27:0/2582335471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36ac0f77d0 0x7f36ac1072c0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 rx=0x7f36a00026d0 tx=0x7f36a0009de0).stop 2021-11-03T18:18:29.149 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:29.139+0000 7f36b11dc700 1 -- 172.21.15.27:0/2582335471 shutdown_connections 2021-11-03T18:18:29.150 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:29.139+0000 7f36b11dc700 1 --2- 172.21.15.27:0/2582335471 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3694064100 0x7f36940665c0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:18:29.150 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:29.139+0000 7f36b11dc700 1 --2- 172.21.15.27:0/2582335471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36ac0f6530 0x7f36ac106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:18:29.150 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:29.139+0000 7f36b11dc700 1 --2- 172.21.15.27:0/2582335471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36ac0f77d0 0x7f36ac1072c0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:18:29.150 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:29.139+0000 7f36b11dc700 1 -- 172.21.15.27:0/2582335471 >> 172.21.15.27:0/2582335471 conn(0x7f36ac0f1cd0 msgr2=0x7f36ac0fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:18:29.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:29.140+0000 7f36b11dc700 1 -- 172.21.15.27:0/2582335471 shutdown_connections 2021-11-03T18:18:29.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:29.140+0000 7f36b11dc700 1 -- 172.21.15.27:0/2582335471 wait complete. 2021-11-03T18:18:29.151 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:18:29.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:29 smithi115 ceph-mon[65453]: from='client.36072 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:18:29.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:29 smithi115 ceph-mon[65453]: from='client.36076 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:18:29.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:29 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/320491732' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:29.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:29 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2582335471' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:18:30.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:29 smithi027 ceph-mon[97244]: from='client.36072 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:18:30.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:29 smithi027 ceph-mon[97244]: from='client.36076 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:18:30.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:29 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/320491732' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:30.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:29 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2582335471' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:18:30.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:30 smithi115 ceph-mon[65453]: pgmap v1915: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:30.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3752.449777 secs 2021-11-03T18:18:31.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:30 smithi027 ceph-mon[97244]: pgmap v1915: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:31.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3752.449777 secs 2021-11-03T18:18:31.882 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:18:32.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:18:32.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:32 smithi027 ceph-mon[97244]: pgmap v1916: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:32.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:32 smithi027 ceph-mon[97244]: poll took 20056.532 ms: 1 events 2021-11-03T18:18:32.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:18:32.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:18:32.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:18:32.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:32.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:32.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:32.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:32.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:32 smithi115 ceph-mon[65453]: pgmap v1916: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:32.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:32 smithi115 ceph-mon[65453]: poll took 20056.532 ms: 1 events 2021-11-03T18:18:32.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:18:32.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:18:32.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:18:32.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:32.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:32.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:32.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:33.959 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:33 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:18:34.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:33 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:18:34.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:34 smithi115 ceph-mon[65453]: pgmap v1917: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:35.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:34 smithi027 ceph-mon[97244]: pgmap v1917: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:35.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3757.449897 secs 2021-11-03T18:18:36.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3757.449897 secs 2021-11-03T18:18:36.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:36 smithi115 ceph-mon[65453]: pgmap v1918: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:37.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:36 smithi027 ceph-mon[97244]: pgmap v1918: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:38.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:38 smithi115 ceph-mon[65453]: pgmap v1919: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:39.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:38 smithi027 ceph-mon[97244]: pgmap v1919: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:40.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:40 smithi115 ceph-mon[65453]: pgmap v1920: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:40.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3762.450037 secs 2021-11-03T18:18:41.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:40 smithi027 ceph-mon[97244]: pgmap v1920: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:41.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3762.450037 secs 2021-11-03T18:18:42.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:42 smithi027 ceph-mon[97244]: pgmap v1921: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:42.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:18:42.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:18:42.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:18:42.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:42.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:42.948 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:42.949 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:42.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:42 smithi115 ceph-mon[65453]: pgmap v1921: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:42.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:18:42.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:18:42.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:18:42.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:42.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:42.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:42.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:43.958 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:43 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:18:44.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:43 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:18:44.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:44 smithi115 ceph-mon[65453]: pgmap v1922: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:45.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:44 smithi027 ceph-mon[97244]: pgmap v1922: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:45.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3767.450178 secs 2021-11-03T18:18:45.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:45 smithi115 ceph-mon[65453]: pgmap v1923: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:46.026 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3767.450178 secs 2021-11-03T18:18:46.027 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:45 smithi027 ceph-mon[97244]: pgmap v1923: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:48.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:47 smithi115 ceph-mon[65453]: pgmap v1924: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:48.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:47 smithi027 ceph-mon[97244]: pgmap v1924: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:50.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:49 smithi115 ceph-mon[65453]: pgmap v1925: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:50.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:49 smithi027 ceph-mon[97244]: pgmap v1925: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:51.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3772.450296 secs 2021-11-03T18:18:51.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3772.450296 secs 2021-11-03T18:18:52.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:52 smithi027 ceph-mon[97244]: pgmap v1926: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:52.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:18:52.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:18:52.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:18:52.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:52.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:52.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:52.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:52.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:52 smithi115 ceph-mon[65453]: pgmap v1926: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:52.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:18:52.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:18:52.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:18:52.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:52.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:52.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:52.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:18:53.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:53 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:18:53.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:53 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:18:54.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:54 smithi027 ceph-mon[97244]: pgmap v1927: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:54.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:54 smithi115 ceph-mon[65453]: pgmap v1927: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:55.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3777.450384 secs 2021-11-03T18:18:55.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3777.450384 secs 2021-11-03T18:18:56.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:56 smithi027 ceph-mon[97244]: pgmap v1928: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:56.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:18:56.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:18:56.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:56 smithi115 ceph-mon[65453]: pgmap v1928: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:56.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:18:56.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:18:58.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:18:58 smithi115 ceph-mon[65453]: pgmap v1929: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:58.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:18:58 smithi027 ceph-mon[97244]: pgmap v1929: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:18:59.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.254+0000 7f55af6eb700 1 -- 172.21.15.27:0/4216965260 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55a80f6ff0 msgr2=0x7f55a80f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:18:59.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.254+0000 7f55af6eb700 1 --2- 172.21.15.27:0/4216965260 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55a80f6ff0 0x7f55a80f7a20 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 rx=0x7f55a4004660 tx=0x7f55a4009b30).stop 2021-11-03T18:18:59.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.254+0000 7f55af6eb700 1 -- 172.21.15.27:0/4216965260 shutdown_connections 2021-11-03T18:18:59.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.254+0000 7f55af6eb700 1 --2- 172.21.15.27:0/4216965260 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55a80f6ff0 0x7f55a80f7a20 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:18:59.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.254+0000 7f55af6eb700 1 --2- 172.21.15.27:0/4216965260 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55a80f66e0 0x7f55a80f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:18:59.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.254+0000 7f55af6eb700 1 -- 172.21.15.27:0/4216965260 >> 172.21.15.27:0/4216965260 conn(0x7f55a80f1f00 msgr2=0x7f55a80f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:18:59.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.254+0000 7f55af6eb700 1 -- 172.21.15.27:0/4216965260 shutdown_connections 2021-11-03T18:18:59.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.254+0000 7f55af6eb700 1 -- 172.21.15.27:0/4216965260 wait complete. 2021-11-03T18:18:59.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.255+0000 7f55af6eb700 1 Processor -- start 2021-11-03T18:18:59.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.255+0000 7f55af6eb700 1 -- start start 2021-11-03T18:18:59.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.255+0000 7f55af6eb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55a80f66e0 0x7f55a8106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:18:59.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.255+0000 7f55af6eb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55a80f6ff0 0x7f55a8107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:18:59.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.255+0000 7f55af6eb700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a8107a10 con 0x7f55a80f66e0 2021-11-03T18:18:59.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.255+0000 7f55af6eb700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55a8107b50 con 0x7f55a80f6ff0 2021-11-03T18:18:59.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.255+0000 7f55ad487700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55a80f66e0 0x7f55a8106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:18:59.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.255+0000 7f55ad487700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55a80f66e0 0x7f55a8106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34162/0 (socket says 172.21.15.27:34162) 2021-11-03T18:18:59.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.255+0000 7f55ad487700 1 -- 172.21.15.27:0/1504840787 learned_addr learned my addr 172.21.15.27:0/1504840787 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:18:59.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.255+0000 7f55acc86700 1 --2- 172.21.15.27:0/1504840787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55a80f6ff0 0x7f55a8107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:18:59.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.256+0000 7f55ad487700 1 -- 172.21.15.27:0/1504840787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55a80f6ff0 msgr2=0x7f55a8107460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:18:59.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.256+0000 7f55ad487700 1 --2- 172.21.15.27:0/1504840787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55a80f6ff0 0x7f55a8107460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:18:59.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.256+0000 7f55ad487700 1 -- 172.21.15.27:0/1504840787 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a4005040 con 0x7f55a80f66e0 2021-11-03T18:18:59.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.256+0000 7f55ad487700 1 --2- 172.21.15.27:0/1504840787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55a80f66e0 0x7f55a8106f70 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 rx=0x7f55980026d0 tx=0x7f5598009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:18:59.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.256+0000 7f559dffb700 1 -- 172.21.15.27:0/1504840787 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55980079c0 con 0x7f55a80f66e0 2021-11-03T18:18:59.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.256+0000 7f55af6eb700 1 -- 172.21.15.27:0/1504840787 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55a81084f0 con 0x7f55a80f66e0 2021-11-03T18:18:59.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.256+0000 7f559dffb700 1 -- 172.21.15.27:0/1504840787 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f559800f040 con 0x7f55a80f66e0 2021-11-03T18:18:59.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.256+0000 7f559dffb700 1 -- 172.21.15.27:0/1504840787 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f55980148b0 con 0x7f55a80f66e0 2021-11-03T18:18:59.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.256+0000 7f55af6eb700 1 -- 172.21.15.27:0/1504840787 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55a8108ab0 con 0x7f55a80f66e0 2021-11-03T18:18:59.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.257+0000 7f55af6eb700 1 -- 172.21.15.27:0/1504840787 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55a8045ad0 con 0x7f55a80f66e0 2021-11-03T18:18:59.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.258+0000 7f559dffb700 1 -- 172.21.15.27:0/1504840787 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f5598007b20 con 0x7f55a80f66e0 2021-11-03T18:18:59.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.258+0000 7f559dffb700 1 --2- 172.21.15.27:0/1504840787 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f55940640b0 0x7f5594066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:18:59.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.258+0000 7f559dffb700 1 -- 172.21.15.27:0/1504840787 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f5598080850 con 0x7f55a80f66e0 2021-11-03T18:18:59.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.259+0000 7f55acc86700 1 --2- 172.21.15.27:0/1504840787 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f55940640b0 0x7f5594066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:18:59.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.259+0000 7f55acc86700 1 --2- 172.21.15.27:0/1504840787 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f55940640b0 0x7f5594066570 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7f55a4004630 tx=0x7f55a4009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:18:59.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.263+0000 7f559dffb700 1 -- 172.21.15.27:0/1504840787 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f5598052a90 con 0x7f55a80f66e0 2021-11-03T18:18:59.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.447+0000 7f55af6eb700 1 -- 172.21.15.27:0/1504840787 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f55a80ff080 con 0x7f55940640b0 2021-11-03T18:18:59.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.448+0000 7f559dffb700 1 -- 172.21.15.27:0/1504840787 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f55a4019040 con 0x7f55940640b0 2021-11-03T18:18:59.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.450+0000 7f55af6eb700 1 -- 172.21.15.27:0/1504840787 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f55940640b0 msgr2=0x7f5594066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:18:59.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.450+0000 7f55af6eb700 1 --2- 172.21.15.27:0/1504840787 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f55940640b0 0x7f5594066570 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7f55a4004630 tx=0x7f55a4009b30).stop 2021-11-03T18:18:59.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.450+0000 7f55af6eb700 1 -- 172.21.15.27:0/1504840787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55a80f66e0 msgr2=0x7f55a8106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:18:59.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.450+0000 7f55af6eb700 1 --2- 172.21.15.27:0/1504840787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55a80f66e0 0x7f55a8106f70 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 rx=0x7f55980026d0 tx=0x7f5598009de0).stop 2021-11-03T18:18:59.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.450+0000 7f55af6eb700 1 -- 172.21.15.27:0/1504840787 shutdown_connections 2021-11-03T18:18:59.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.450+0000 7f55af6eb700 1 --2- 172.21.15.27:0/1504840787 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f55940640b0 0x7f5594066570 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:18:59.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.450+0000 7f55af6eb700 1 --2- 172.21.15.27:0/1504840787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f55a80f66e0 0x7f55a8106f70 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:18:59.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.450+0000 7f55af6eb700 1 --2- 172.21.15.27:0/1504840787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f55a80f6ff0 0x7f55a8107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:18:59.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.450+0000 7f55af6eb700 1 -- 172.21.15.27:0/1504840787 >> 172.21.15.27:0/1504840787 conn(0x7f55a80f1f00 msgr2=0x7f55a80fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:18:59.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.450+0000 7f55af6eb700 1 -- 172.21.15.27:0/1504840787 shutdown_connections 2021-11-03T18:18:59.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.450+0000 7f55af6eb700 1 -- 172.21.15.27:0/1504840787 wait complete. 2021-11-03T18:18:59.458 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:18:59.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.550+0000 7fc95f36b700 1 -- 172.21.15.27:0/4264734758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9580f55c0 msgr2=0x7fc9580f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:18:59.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.550+0000 7fc95f36b700 1 --2- 172.21.15.27:0/4264734758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9580f55c0 0x7fc9580f59e0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 rx=0x7fc9500099e0 tx=0x7fc950004d10).stop 2021-11-03T18:18:59.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.550+0000 7fc95f36b700 1 -- 172.21.15.27:0/4264734758 shutdown_connections 2021-11-03T18:18:59.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.550+0000 7fc95f36b700 1 --2- 172.21.15.27:0/4264734758 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9580f6860 0x7fc9580f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T18:18:59.550 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.550+0000 7fc95f36b700 1 --2- 172.21.15.27:0/4264734758 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9580f55c0 0x7fc9580f59e0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:18:59.550 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.550+0000 7fc95f36b700 1 -- 172.21.15.27:0/4264734758 >> 172.21.15.27:0/4264734758 conn(0x7fc9580f0d40 msgr2=0x7fc9580f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:18:59.550 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.550+0000 7fc95f36b700 1 -- 172.21.15.27:0/4264734758 shutdown_connections 2021-11-03T18:18:59.550 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.550+0000 7fc95f36b700 1 -- 172.21.15.27:0/4264734758 wait complete. 2021-11-03T18:18:59.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.550+0000 7fc95f36b700 1 Processor -- start 2021-11-03T18:18:59.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.550+0000 7fc95f36b700 1 -- start start 2021-11-03T18:18:59.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.551+0000 7fc95f36b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9580f55c0 0x7fc958109e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:18:59.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.551+0000 7fc95f36b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9580f6860 0x7fc95810c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:18:59.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.551+0000 7fc95f36b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc958060470 con 0x7fc9580f55c0 2021-11-03T18:18:59.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.551+0000 7fc95f36b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9580605b0 con 0x7fc9580f6860 2021-11-03T18:18:59.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.551+0000 7fc95d107700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9580f55c0 0x7fc958109e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:18:59.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.551+0000 7fc95d107700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9580f55c0 0x7fc958109e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34172/0 (socket says 172.21.15.27:34172) 2021-11-03T18:18:59.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.551+0000 7fc95d107700 1 -- 172.21.15.27:0/3924919774 learned_addr learned my addr 172.21.15.27:0/3924919774 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:18:59.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.551+0000 7fc95c906700 1 --2- 172.21.15.27:0/3924919774 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9580f6860 0x7fc95810c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:18:59.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.551+0000 7fc95d107700 1 -- 172.21.15.27:0/3924919774 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9580f6860 msgr2=0x7fc95810c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:18:59.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.551+0000 7fc95d107700 1 --2- 172.21.15.27:0/3924919774 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9580f6860 0x7fc95810c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:18:59.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.551+0000 7fc95d107700 1 -- 172.21.15.27:0/3924919774 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc950009640 con 0x7fc9580f55c0 2021-11-03T18:18:59.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.551+0000 7fc95c906700 1 --2- 172.21.15.27:0/3924919774 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9580f6860 0x7fc95810c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T18:18:59.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.551+0000 7fc95d107700 1 --2- 172.21.15.27:0/3924919774 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9580f55c0 0x7fc958109e30 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 rx=0x7fc95000c010 tx=0x7fc950004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:18:59.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.551+0000 7fc94dffb700 1 -- 172.21.15.27:0/3924919774 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc95000e070 con 0x7fc9580f55c0 2021-11-03T18:18:59.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.551+0000 7fc94dffb700 1 -- 172.21.15.27:0/3924919774 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fc950007dc0 con 0x7fc9580f55c0 2021-11-03T18:18:59.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.551+0000 7fc95f36b700 1 -- 172.21.15.27:0/3924919774 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc95810c930 con 0x7fc9580f55c0 2021-11-03T18:18:59.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.551+0000 7fc94dffb700 1 -- 172.21.15.27:0/3924919774 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc950018070 con 0x7fc9580f55c0 2021-11-03T18:18:59.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.552+0000 7fc95f36b700 1 -- 172.21.15.27:0/3924919774 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc95810cde0 con 0x7fc9580f55c0 2021-11-03T18:18:59.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.552+0000 7fc95f36b700 1 -- 172.21.15.27:0/3924919774 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9580f49b0 con 0x7fc9580f55c0 2021-11-03T18:18:59.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.553+0000 7fc94dffb700 1 -- 172.21.15.27:0/3924919774 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fc950024060 con 0x7fc9580f55c0 2021-11-03T18:18:59.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.553+0000 7fc94dffb700 1 --2- 172.21.15.27:0/3924919774 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc9440640b0 0x7fc944066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:18:59.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.553+0000 7fc94dffb700 1 -- 172.21.15.27:0/3924919774 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fc9500823a0 con 0x7fc9580f55c0 2021-11-03T18:18:59.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.553+0000 7fc95c906700 1 --2- 172.21.15.27:0/3924919774 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc9440640b0 0x7fc944066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:18:59.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.554+0000 7fc95c906700 1 --2- 172.21.15.27:0/3924919774 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc9440640b0 0x7fc944066570 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7fc94800a7e0 tx=0x7fc948008040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:18:59.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.557+0000 7fc94dffb700 1 -- 172.21.15.27:0/3924919774 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fc9500545e0 con 0x7fc9580f55c0 2021-11-03T18:18:59.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.730+0000 7fc95f36b700 1 -- 172.21.15.27:0/3924919774 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc9580fdf80 con 0x7fc9440640b0 2021-11-03T18:18:59.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.744+0000 7fc94dffb700 1 -- 172.21.15.27:0/3924919774 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fc9580fdf80 con 0x7fc9440640b0 2021-11-03T18:18:59.743 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:18:59.744 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (70m) 9m ago 71m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:18:59.744 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (62m) 9m ago 71m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:18:59.744 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (62m) 8m ago 71m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:18:59.744 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (70m) 9m ago 71m 42.1M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:18:59.745 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (64m) 9m ago 66m 26.3M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:18:59.745 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (67m) 9m ago 67m 3847M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:18:59.745 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (62m) 8m ago 66m 25.9M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:18:59.745 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (64m) 8m ago 67m 27.0M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:18:59.746 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (64m) 9m ago 73m 572M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:18:59.746 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (63m) 8m ago 70m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:18:59.746 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (63m) 9m ago 73m 273M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:18:59.746 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (62m) 8m ago 70m 129M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:18:59.746 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (71m) 9m ago 71m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:18:59.747 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (70m) 8m ago 70m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:18:59.747 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (61m) 9m ago 70m 371M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:18:59.747 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (61m) 9m ago 69m 405M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:18:59.747 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (60m) 9m ago 69m 302M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:18:59.748 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (60m) 8m ago 69m 429M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:18:59.748 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (59m) 8m ago 68m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:18:59.748 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (59m) 8m ago 68m 348M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:18:59.748 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (70m) 9m ago 71m 57.6M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:18:59.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.746+0000 7fc95f36b700 1 -- 172.21.15.27:0/3924919774 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc9440640b0 msgr2=0x7fc944066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:18:59.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.746+0000 7fc95f36b700 1 --2- 172.21.15.27:0/3924919774 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc9440640b0 0x7fc944066570 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7fc94800a7e0 tx=0x7fc948008040).stop 2021-11-03T18:18:59.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.746+0000 7fc95f36b700 1 -- 172.21.15.27:0/3924919774 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9580f55c0 msgr2=0x7fc958109e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:18:59.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.746+0000 7fc95f36b700 1 --2- 172.21.15.27:0/3924919774 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9580f55c0 0x7fc958109e30 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 rx=0x7fc95000c010 tx=0x7fc950004d10).stop 2021-11-03T18:18:59.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.746+0000 7fc95f36b700 1 -- 172.21.15.27:0/3924919774 shutdown_connections 2021-11-03T18:18:59.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.746+0000 7fc95f36b700 1 --2- 172.21.15.27:0/3924919774 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc9440640b0 0x7fc944066570 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:18:59.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.746+0000 7fc95f36b700 1 --2- 172.21.15.27:0/3924919774 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9580f55c0 0x7fc958109e30 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:18:59.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.746+0000 7fc95f36b700 1 --2- 172.21.15.27:0/3924919774 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9580f6860 0x7fc95810c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:18:59.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.746+0000 7fc95f36b700 1 -- 172.21.15.27:0/3924919774 >> 172.21.15.27:0/3924919774 conn(0x7fc9580f0d40 msgr2=0x7fc9580fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:18:59.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.746+0000 7fc95f36b700 1 -- 172.21.15.27:0/3924919774 shutdown_connections 2021-11-03T18:18:59.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.746+0000 7fc95f36b700 1 -- 172.21.15.27:0/3924919774 wait complete. 2021-11-03T18:18:59.846 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.845+0000 7fa3c112d700 1 -- 172.21.15.27:0/3354424671 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3bc0f5a10 msgr2=0x7fa3bc0f5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:18:59.846 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.845+0000 7fa3c112d700 1 --2- 172.21.15.27:0/3354424671 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3bc0f5a10 0x7fa3bc0f5e30 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 rx=0x7fa3b0004660 tx=0x7fa3b0009b30).stop 2021-11-03T18:18:59.846 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.845+0000 7fa3c112d700 1 -- 172.21.15.27:0/3354424671 shutdown_connections 2021-11-03T18:18:59.847 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.845+0000 7fa3c112d700 1 --2- 172.21.15.27:0/3354424671 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3bc0f63b0 0x7fa3bc0f6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T18:18:59.847 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.845+0000 7fa3c112d700 1 --2- 172.21.15.27:0/3354424671 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3bc0f5a10 0x7fa3bc0f5e30 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:18:59.847 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.845+0000 7fa3c112d700 1 -- 172.21.15.27:0/3354424671 >> 172.21.15.27:0/3354424671 conn(0x7fa3bc0f0320 msgr2=0x7fa3bc0f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:18:59.847 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.845+0000 7fa3c112d700 1 -- 172.21.15.27:0/3354424671 shutdown_connections 2021-11-03T18:18:59.847 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.845+0000 7fa3c112d700 1 -- 172.21.15.27:0/3354424671 wait complete. 2021-11-03T18:18:59.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.846+0000 7fa3c112d700 1 Processor -- start 2021-11-03T18:18:59.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.846+0000 7fa3c112d700 1 -- start start 2021-11-03T18:18:59.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.846+0000 7fa3c112d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3bc0f5a10 0x7fa3bc0fd740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:18:59.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.846+0000 7fa3c112d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3bc0f63b0 0x7fa3bc0fbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:18:59.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.846+0000 7fa3c112d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3bc0fc1b0 con 0x7fa3bc0f63b0 2021-11-03T18:18:59.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.846+0000 7fa3c112d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3bc0fc2f0 con 0x7fa3bc0f5a10 2021-11-03T18:18:59.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.846+0000 7fa3bb7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3bc0f63b0 0x7fa3bc0fbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:18:59.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.846+0000 7fa3bb7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3bc0f63b0 0x7fa3bc0fbbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34184/0 (socket says 172.21.15.27:34184) 2021-11-03T18:18:59.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.846+0000 7fa3bb7fe700 1 -- 172.21.15.27:0/2891638211 learned_addr learned my addr 172.21.15.27:0/2891638211 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:18:59.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.846+0000 7fa3bbfff700 1 --2- 172.21.15.27:0/2891638211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3bc0f5a10 0x7fa3bc0fd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:18:59.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.847+0000 7fa3bb7fe700 1 -- 172.21.15.27:0/2891638211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3bc0f5a10 msgr2=0x7fa3bc0fd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:18:59.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.847+0000 7fa3bb7fe700 1 --2- 172.21.15.27:0/2891638211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3bc0f5a10 0x7fa3bc0fd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:18:59.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.847+0000 7fa3bb7fe700 1 -- 172.21.15.27:0/2891638211 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b0005040 con 0x7fa3bc0f63b0 2021-11-03T18:18:59.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.847+0000 7fa3bb7fe700 1 --2- 172.21.15.27:0/2891638211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3bc0f63b0 0x7fa3bc0fbbc0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 rx=0x7fa3ac00b410 tx=0x7fa3ac006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:18:59.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.847+0000 7fa3b8ff9700 1 -- 172.21.15.27:0/2891638211 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3ac0125f0 con 0x7fa3bc0f63b0 2021-11-03T18:18:59.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.847+0000 7fa3b8ff9700 1 -- 172.21.15.27:0/2891638211 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fa3ac012c00 con 0x7fa3bc0f63b0 2021-11-03T18:18:59.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.847+0000 7fa3c112d700 1 -- 172.21.15.27:0/2891638211 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3bc0fc5d0 con 0x7fa3bc0f63b0 2021-11-03T18:18:59.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.847+0000 7fa3b8ff9700 1 -- 172.21.15.27:0/2891638211 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3ac002ae0 con 0x7fa3bc0f63b0 2021-11-03T18:18:59.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.847+0000 7fa3c112d700 1 -- 172.21.15.27:0/2891638211 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3bc10edf0 con 0x7fa3bc0f63b0 2021-11-03T18:18:59.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.848+0000 7fa3c112d700 1 -- 172.21.15.27:0/2891638211 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3bc045ad0 con 0x7fa3bc0f63b0 2021-11-03T18:18:59.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.848+0000 7fa3b8ff9700 1 -- 172.21.15.27:0/2891638211 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fa3ac00f030 con 0x7fa3bc0f63b0 2021-11-03T18:18:59.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.849+0000 7fa3b8ff9700 1 --2- 172.21.15.27:0/2891638211 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa3a4064100 0x7fa3a40665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:18:59.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.849+0000 7fa3b8ff9700 1 -- 172.21.15.27:0/2891638211 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fa3ac080cc0 con 0x7fa3bc0f63b0 2021-11-03T18:18:59.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.849+0000 7fa3bbfff700 1 --2- 172.21.15.27:0/2891638211 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa3a4064100 0x7fa3a40665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:18:59.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.849+0000 7fa3bbfff700 1 --2- 172.21.15.27:0/2891638211 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa3a4064100 0x7fa3a40665c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7fa3b0004e30 tx=0x7fa3b0009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:18:59.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:18:59.853+0000 7fa3b8ff9700 1 -- 172.21.15.27:0/2891638211 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fa3ac052fc0 con 0x7fa3bc0f63b0 2021-11-03T18:19:00.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.084+0000 7fa3c112d700 1 -- 172.21.15.27:0/2891638211 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa3bc027a00 con 0x7fa3bc0f63b0 2021-11-03T18:19:00.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.085+0000 7fa3b8ff9700 1 -- 172.21.15.27:0/2891638211 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fa3ac052bb0 con 0x7fa3bc0f63b0 2021-11-03T18:19:00.084 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:19:00.084 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:19:00.085 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:19:00.085 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:19:00.085 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:19:00.085 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:19:00.085 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:19:00.086 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:19:00.086 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:19:00.086 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:19:00.086 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:19:00.087 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:19:00.087 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:19:00.087 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:19:00.087 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:19:00.088 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:19:00.088 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:19:00.088 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:19:00.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.086+0000 7fa3c112d700 1 -- 172.21.15.27:0/2891638211 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa3a4064100 msgr2=0x7fa3a40665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:19:00.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.086+0000 7fa3c112d700 1 --2- 172.21.15.27:0/2891638211 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa3a4064100 0x7fa3a40665c0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 rx=0x7fa3b0004e30 tx=0x7fa3b0009b30).stop 2021-11-03T18:19:00.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.086+0000 7fa3c112d700 1 -- 172.21.15.27:0/2891638211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3bc0f63b0 msgr2=0x7fa3bc0fbbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:19:00.090 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.086+0000 7fa3c112d700 1 --2- 172.21.15.27:0/2891638211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3bc0f63b0 0x7fa3bc0fbbc0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 rx=0x7fa3ac00b410 tx=0x7fa3ac006ef0).stop 2021-11-03T18:19:00.090 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.086+0000 7fa3c112d700 1 -- 172.21.15.27:0/2891638211 shutdown_connections 2021-11-03T18:19:00.090 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.086+0000 7fa3c112d700 1 --2- 172.21.15.27:0/2891638211 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa3a4064100 0x7fa3a40665c0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:19:00.090 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.086+0000 7fa3c112d700 1 --2- 172.21.15.27:0/2891638211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa3bc0f5a10 0x7fa3bc0fd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:19:00.091 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.086+0000 7fa3c112d700 1 --2- 172.21.15.27:0/2891638211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa3bc0f63b0 0x7fa3bc0fbbc0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:19:00.091 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.086+0000 7fa3c112d700 1 -- 172.21.15.27:0/2891638211 >> 172.21.15.27:0/2891638211 conn(0x7fa3bc0f0320 msgr2=0x7fa3bc0fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:19:00.091 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.086+0000 7fa3c112d700 1 -- 172.21.15.27:0/2891638211 shutdown_connections 2021-11-03T18:19:00.091 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.087+0000 7fa3c112d700 1 -- 172.21.15.27:0/2891638211 wait complete. 2021-11-03T18:19:00.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.187+0000 7f34a8b7a700 1 -- 172.21.15.27:0/4248619427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34a40f4ae0 msgr2=0x7f34a40f4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:19:00.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.187+0000 7f34a8b7a700 1 --2- 172.21.15.27:0/4248619427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34a40f4ae0 0x7f34a40f4f20 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 rx=0x7f3494004690 tx=0x7f3494009b30).stop 2021-11-03T18:19:00.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.187+0000 7f34a8b7a700 1 -- 172.21.15.27:0/4248619427 shutdown_connections 2021-11-03T18:19:00.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.187+0000 7f34a8b7a700 1 --2- 172.21.15.27:0/4248619427 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34a40f5410 0x7f34a40fbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:19:00.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.187+0000 7f34a8b7a700 1 --2- 172.21.15.27:0/4248619427 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34a40f4ae0 0x7f34a40f4f20 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:19:00.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.187+0000 7f34a8b7a700 1 -- 172.21.15.27:0/4248619427 >> 172.21.15.27:0/4248619427 conn(0x7f34a40f0540 msgr2=0x7f34a40f2960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:19:00.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.187+0000 7f34a8b7a700 1 -- 172.21.15.27:0/4248619427 shutdown_connections 2021-11-03T18:19:00.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.187+0000 7f34a8b7a700 1 -- 172.21.15.27:0/4248619427 wait complete. 2021-11-03T18:19:00.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.187+0000 7f34a8b7a700 1 Processor -- start 2021-11-03T18:19:00.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.187+0000 7f34a8b7a700 1 -- start start 2021-11-03T18:19:00.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.187+0000 7f34a8b7a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34a40f4ae0 0x7f34a4106ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:19:00.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.187+0000 7f34a8b7a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34a40f5410 0x7f34a41073d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:19:00.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.187+0000 7f34a8b7a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a4107980 con 0x7f34a40f4ae0 2021-11-03T18:19:00.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.187+0000 7f34a8b7a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a4107ac0 con 0x7f34a40f5410 2021-11-03T18:19:00.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.188+0000 7f34a259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34a40f4ae0 0x7f34a4106ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:19:00.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.188+0000 7f34a259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34a40f4ae0 0x7f34a4106ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34192/0 (socket says 172.21.15.27:34192) 2021-11-03T18:19:00.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.188+0000 7f34a259c700 1 -- 172.21.15.27:0/2099315321 learned_addr learned my addr 172.21.15.27:0/2099315321 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:19:00.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.188+0000 7f34a1d9b700 1 --2- 172.21.15.27:0/2099315321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34a40f5410 0x7f34a41073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:19:00.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.188+0000 7f34a259c700 1 -- 172.21.15.27:0/2099315321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34a40f5410 msgr2=0x7f34a41073d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:19:00.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.188+0000 7f34a259c700 1 --2- 172.21.15.27:0/2099315321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34a40f5410 0x7f34a41073d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:19:00.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.188+0000 7f34a259c700 1 -- 172.21.15.27:0/2099315321 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3494005040 con 0x7f34a40f4ae0 2021-11-03T18:19:00.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.188+0000 7f34a1d9b700 1 --2- 172.21.15.27:0/2099315321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34a40f5410 0x7f34a41073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:19:00.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.188+0000 7f34a259c700 1 --2- 172.21.15.27:0/2099315321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34a40f4ae0 0x7f34a4106ee0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 rx=0x7f3494004e20 tx=0x7f3494009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:19:00.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.188+0000 7f3492ffd700 1 -- 172.21.15.27:0/2099315321 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3494018070 con 0x7f34a40f4ae0 2021-11-03T18:19:00.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.188+0000 7f34a8b7a700 1 -- 172.21.15.27:0/2099315321 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34a4108400 con 0x7f34a40f4ae0 2021-11-03T18:19:00.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.188+0000 7f34a8b7a700 1 -- 172.21.15.27:0/2099315321 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34a4108960 con 0x7f34a40f4ae0 2021-11-03T18:19:00.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.188+0000 7f3492ffd700 1 -- 172.21.15.27:0/2099315321 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f3494013e40 con 0x7f34a40f4ae0 2021-11-03T18:19:00.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.188+0000 7f3492ffd700 1 -- 172.21.15.27:0/2099315321 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f349401ddd0 con 0x7f34a40f4ae0 2021-11-03T18:19:00.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.189+0000 7f34a8b7a700 1 -- 172.21.15.27:0/2099315321 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3488004fa0 con 0x7f34a40f4ae0 2021-11-03T18:19:00.196 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.190+0000 7f3492ffd700 1 -- 172.21.15.27:0/2099315321 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f349401d420 con 0x7f34a40f4ae0 2021-11-03T18:19:00.196 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.190+0000 7f3492ffd700 1 --2- 172.21.15.27:0/2099315321 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f348c0640b0 0x7f348c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:19:00.196 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.190+0000 7f3492ffd700 1 -- 172.21.15.27:0/2099315321 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f3494082230 con 0x7f34a40f4ae0 2021-11-03T18:19:00.197 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.190+0000 7f34a1d9b700 1 --2- 172.21.15.27:0/2099315321 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f348c0640b0 0x7f348c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:19:00.197 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.190+0000 7f34a1d9b700 1 --2- 172.21.15.27:0/2099315321 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f348c0640b0 0x7f348c066570 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7f3498009740 tx=0x7f3498009be0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:19:00.198 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.194+0000 7f3492ffd700 1 -- 172.21.15.27:0/2099315321 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f34940544d0 con 0x7f34a40f4ae0 2021-11-03T18:19:00.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.400+0000 7f34a8b7a700 1 -- 172.21.15.27:0/2099315321 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3488005e80 con 0x7f34a40f4ae0 2021-11-03T18:19:00.400 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.401+0000 7f3492ffd700 1 -- 172.21.15.27:0/2099315321 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f34940540c0 con 0x7f34a40f4ae0 2021-11-03T18:19:00.400 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:19:00.401 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:19:00.401 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:19:00.401 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:19:00.402 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:19:00.402 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:19:00.402 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:19:00.402 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:19:00.403 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:19:00.403 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:19:00.403 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:19:00.403 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:19:00.403 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:19:00.404 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:19:00.404 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:19:00.404 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:19:00.404 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:19:00.405 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:19:00.405 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:19:00.405 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:19:00.405 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:19:00.406 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:19:00.406 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:19:00.406 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:19:00.407 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:19:00.407 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:19:00.407 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:19:00.407 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:19:00.408 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:19:00.408 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:19:00.408 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:19:00.408 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:19:00.409 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:19:00.409 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:19:00.410 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:19:00.410 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:19:00.410 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:19:00.410 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:19:00.411 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:19:00.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.402+0000 7f34a8b7a700 1 -- 172.21.15.27:0/2099315321 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f348c0640b0 msgr2=0x7f348c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:19:00.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.402+0000 7f34a8b7a700 1 --2- 172.21.15.27:0/2099315321 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f348c0640b0 0x7f348c066570 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 rx=0x7f3498009740 tx=0x7f3498009be0).stop 2021-11-03T18:19:00.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.402+0000 7f34a8b7a700 1 -- 172.21.15.27:0/2099315321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34a40f4ae0 msgr2=0x7f34a4106ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:19:00.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.402+0000 7f34a8b7a700 1 --2- 172.21.15.27:0/2099315321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34a40f4ae0 0x7f34a4106ee0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 rx=0x7f3494004e20 tx=0x7f3494009b30).stop 2021-11-03T18:19:00.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.402+0000 7f34a8b7a700 1 -- 172.21.15.27:0/2099315321 shutdown_connections 2021-11-03T18:19:00.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.402+0000 7f34a8b7a700 1 --2- 172.21.15.27:0/2099315321 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f348c0640b0 0x7f348c066570 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:19:00.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.402+0000 7f34a8b7a700 1 --2- 172.21.15.27:0/2099315321 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f34a40f4ae0 0x7f34a4106ee0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:19:00.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.403+0000 7f34a8b7a700 1 --2- 172.21.15.27:0/2099315321 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f34a40f5410 0x7f34a41073d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:19:00.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.403+0000 7f34a8b7a700 1 -- 172.21.15.27:0/2099315321 >> 172.21.15.27:0/2099315321 conn(0x7f34a40f0540 msgr2=0x7f34a40f84f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:19:00.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.403+0000 7f34a8b7a700 1 -- 172.21.15.27:0/2099315321 shutdown_connections 2021-11-03T18:19:00.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:00.403+0000 7f34a8b7a700 1 -- 172.21.15.27:0/2099315321 wait complete. 2021-11-03T18:19:00.414 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:19:00.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:00 smithi115 ceph-mon[65453]: pgmap v1930: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:00.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3782.450440 secs 2021-11-03T18:19:00.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:00 smithi115 ceph-mon[65453]: from='client.36088 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:19:00.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:00 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2891638211' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:00.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:00 smithi027 ceph-mon[97244]: pgmap v1930: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:00.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3782.450440 secs 2021-11-03T18:19:00.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:00 smithi027 ceph-mon[97244]: from='client.36088 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:19:00.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:00 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2891638211' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:01.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:01 smithi115 ceph-mon[65453]: from='client.36092 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:19:01.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:01 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2099315321' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:19:01.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:01 smithi027 ceph-mon[97244]: from='client.36092 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:19:01.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:01 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2099315321' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:19:02.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:02 smithi115 ceph-mon[65453]: pgmap v1931: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:02.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:19:02.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:19:02.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:19:02.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:02.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:02.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:02.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:02.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:02 smithi027 ceph-mon[97244]: pgmap v1931: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:02.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:19:02.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:19:02.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:19:02.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:02.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:02.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:02.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:03.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:03 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:19:03.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:03 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:19:04.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:04 smithi115 ceph-mon[65453]: pgmap v1932: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:04.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:04 smithi027 ceph-mon[97244]: pgmap v1932: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:05.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3787.450580 secs 2021-11-03T18:19:05.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3787.450580 secs 2021-11-03T18:19:06.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:06 smithi115 ceph-mon[65453]: pgmap v1933: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:06.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:06 smithi027 ceph-mon[97244]: pgmap v1933: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:08.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:08 smithi115 ceph-mon[65453]: pgmap v1934: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:08.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:08 smithi027 ceph-mon[97244]: pgmap v1934: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:10.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:10 smithi115 ceph-mon[65453]: pgmap v1935: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:10.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3792.450711 secs 2021-11-03T18:19:10.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:10 smithi027 ceph-mon[97244]: pgmap v1935: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:10.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3792.450711 secs 2021-11-03T18:19:12.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:12 smithi115 ceph-mon[65453]: pgmap v1936: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:12.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:19:12.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:12 smithi027 ceph-mon[97244]: pgmap v1936: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:12.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:19:13.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:13 smithi115 ceph-mon[65453]: poll took 40118.009 ms: 1 events 2021-11-03T18:19:13.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:19:13.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:19:13.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:13.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:13.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:13.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:13.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:13 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:19:13.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:13 smithi027 ceph-mon[97244]: poll took 40118.009 ms: 1 events 2021-11-03T18:19:13.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:19:13.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:19:13.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:13.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:13.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:13.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:13.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:13 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:19:14.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:14 smithi115 ceph-mon[65453]: pgmap v1937: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:14.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:14 smithi027 ceph-mon[97244]: pgmap v1937: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:15.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3797.450837 secs 2021-11-03T18:19:15.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3797.450837 secs 2021-11-03T18:19:16.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:16 smithi115 ceph-mon[65453]: pgmap v1938: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:16.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:16 smithi027 ceph-mon[97244]: pgmap v1938: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:18.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:18 smithi115 ceph-mon[65453]: pgmap v1939: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:18.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:18 smithi027 ceph-mon[97244]: pgmap v1939: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:20.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:20 smithi115 ceph-mon[65453]: pgmap v1940: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:20.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3802.450927 secs 2021-11-03T18:19:20.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:20 smithi027 ceph-mon[97244]: pgmap v1940: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:20.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3802.450927 secs 2021-11-03T18:19:22.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:22 smithi115 ceph-mon[65453]: pgmap v1941: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:22.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:19:22.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:22 smithi027 ceph-mon[97244]: pgmap v1941: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:22.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:19:23.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:23 smithi115 ceph-mon[65453]: poll took 10024.510 ms: 1 events 2021-11-03T18:19:23.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:23 smithi027 ceph-mon[97244]: poll took 10024.510 ms: 1 events 2021-11-03T18:19:24.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:24 smithi115 ceph-mon[65453]: pgmap v1942: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:24.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:24 smithi027 ceph-mon[97244]: pgmap v1942: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:25.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3807.450942 secs 2021-11-03T18:19:25.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3807.450942 secs 2021-11-03T18:19:26.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:26 smithi115 ceph-mon[65453]: pgmap v1943: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:26.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:26 smithi027 ceph-mon[97244]: pgmap v1943: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:28.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:28 smithi115 ceph-mon[65453]: pgmap v1944: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:28.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:28 smithi115 ceph-mon[65453]: poll took 4974.158 ms: 1 events 2021-11-03T18:19:28.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:19:28.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:19:28.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:19:28.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:28.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:28.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:28.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:28.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:28 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:19:28.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:28 smithi027 ceph-mon[97244]: pgmap v1944: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:28.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:28 smithi027 ceph-mon[97244]: poll took 4974.158 ms: 1 events 2021-11-03T18:19:28.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:19:28.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:19:28.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:19:28.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:28.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:28.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:28.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:28.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:28 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:19:30.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.519+0000 7f8ffa855700 1 -- 172.21.15.27:0/1824999062 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ff40f66e0 msgr2=0x7f8ff40f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:19:30.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.519+0000 7f8ffa855700 1 --2- 172.21.15.27:0/1824999062 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ff40f66e0 0x7f8ff40f6b00 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 rx=0x7f8fe4004660 tx=0x7f8fe4009b30).stop 2021-11-03T18:19:30.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.519+0000 7f8ffa855700 1 -- 172.21.15.27:0/1824999062 shutdown_connections 2021-11-03T18:19:30.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.520+0000 7f8ffa855700 1 --2- 172.21.15.27:0/1824999062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ff40f7980 0x7f8ff40f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:19:30.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.520+0000 7f8ffa855700 1 --2- 172.21.15.27:0/1824999062 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ff40f66e0 0x7f8ff40f6b00 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:19:30.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.520+0000 7f8ffa855700 1 -- 172.21.15.27:0/1824999062 >> 172.21.15.27:0/1824999062 conn(0x7f8ff40f1e40 msgr2=0x7f8ff40f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:19:30.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.520+0000 7f8ffa855700 1 -- 172.21.15.27:0/1824999062 shutdown_connections 2021-11-03T18:19:30.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.520+0000 7f8ffa855700 1 -- 172.21.15.27:0/1824999062 wait complete. 2021-11-03T18:19:30.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.520+0000 7f8ffa855700 1 Processor -- start 2021-11-03T18:19:30.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.520+0000 7f8ffa855700 1 -- start start 2021-11-03T18:19:30.524 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.520+0000 7f8ffa855700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ff40f66e0 0x7f8ff4106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:19:30.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.521+0000 7f8ffa855700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ff40f7980 0x7f8ff4107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:19:30.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.521+0000 7f8ffa855700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff4107a00 con 0x7f8ff40f66e0 2021-11-03T18:19:30.526 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.521+0000 7f8ffa855700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff4107b40 con 0x7f8ff40f7980 2021-11-03T18:19:30.526 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.521+0000 7f8ff3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ff40f66e0 0x7f8ff4106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:19:30.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.521+0000 7f8ff3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ff40f66e0 0x7f8ff4106f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34208/0 (socket says 172.21.15.27:34208) 2021-11-03T18:19:30.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.521+0000 7f8ff3fff700 1 -- 172.21.15.27:0/2107384675 learned_addr learned my addr 172.21.15.27:0/2107384675 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:19:30.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.521+0000 7f8ff37fe700 1 --2- 172.21.15.27:0/2107384675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ff40f7980 0x7f8ff4107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:19:30.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.521+0000 7f8ff3fff700 1 -- 172.21.15.27:0/2107384675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ff40f7980 msgr2=0x7f8ff4107450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:19:30.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.521+0000 7f8ff3fff700 1 --2- 172.21.15.27:0/2107384675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ff40f7980 0x7f8ff4107450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:19:30.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.521+0000 7f8ff3fff700 1 -- 172.21.15.27:0/2107384675 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe4005040 con 0x7f8ff40f66e0 2021-11-03T18:19:30.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.521+0000 7f8ff3fff700 1 --2- 172.21.15.27:0/2107384675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ff40f66e0 0x7f8ff4106f60 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 rx=0x7f8fe4013a70 tx=0x7f8fe4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:19:30.530 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.521+0000 7f8ff0ff9700 1 -- 172.21.15.27:0/2107384675 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fe4018070 con 0x7f8ff40f66e0 2021-11-03T18:19:30.530 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.521+0000 7f8ff0ff9700 1 -- 172.21.15.27:0/2107384675 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f8fe40097d0 con 0x7f8ff40f66e0 2021-11-03T18:19:30.531 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.521+0000 7f8ffa855700 1 -- 172.21.15.27:0/2107384675 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ff4108480 con 0x7f8ff40f66e0 2021-11-03T18:19:30.531 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.521+0000 7f8ff0ff9700 1 -- 172.21.15.27:0/2107384675 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fe401c3c0 con 0x7f8ff40f66e0 2021-11-03T18:19:30.532 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.521+0000 7f8ffa855700 1 -- 172.21.15.27:0/2107384675 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff41089e0 con 0x7f8ff40f66e0 2021-11-03T18:19:30.532 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.522+0000 7f8ffa855700 1 -- 172.21.15.27:0/2107384675 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ff4045ad0 con 0x7f8ff40f66e0 2021-11-03T18:19:30.532 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.523+0000 7f8ff0ff9700 1 -- 172.21.15.27:0/2107384675 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f8fe4007ba0 con 0x7f8ff40f66e0 2021-11-03T18:19:30.533 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.523+0000 7f8ff0ff9700 1 --2- 172.21.15.27:0/2107384675 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8fdc0640b0 0x7f8fdc066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:19:30.533 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.523+0000 7f8ff0ff9700 1 -- 172.21.15.27:0/2107384675 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f8fe40824d0 con 0x7f8ff40f66e0 2021-11-03T18:19:30.533 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.523+0000 7f8ff37fe700 1 --2- 172.21.15.27:0/2107384675 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8fdc0640b0 0x7f8fdc066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:19:30.534 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.524+0000 7f8ff37fe700 1 --2- 172.21.15.27:0/2107384675 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8fdc0640b0 0x7f8fdc066570 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f8fe8009740 tx=0x7f8fe8009c60).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:19:30.535 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.527+0000 7f8ff0ff9700 1 -- 172.21.15.27:0/2107384675 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f8fe40547d0 con 0x7f8ff40f66e0 2021-11-03T18:19:30.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.706+0000 7f8ffa855700 1 -- 172.21.15.27:0/2107384675 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ff40fefb0 con 0x7f8fdc0640b0 2021-11-03T18:19:30.704 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.707+0000 7f8ff0ff9700 1 -- 172.21.15.27:0/2107384675 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f8ff40fefb0 con 0x7f8fdc0640b0 2021-11-03T18:19:30.707 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.709+0000 7f8ffa855700 1 -- 172.21.15.27:0/2107384675 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8fdc0640b0 msgr2=0x7f8fdc066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:19:30.707 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.709+0000 7f8ffa855700 1 --2- 172.21.15.27:0/2107384675 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8fdc0640b0 0x7f8fdc066570 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 rx=0x7f8fe8009740 tx=0x7f8fe8009c60).stop 2021-11-03T18:19:30.707 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.709+0000 7f8ffa855700 1 -- 172.21.15.27:0/2107384675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ff40f66e0 msgr2=0x7f8ff4106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:19:30.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.709+0000 7f8ffa855700 1 --2- 172.21.15.27:0/2107384675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ff40f66e0 0x7f8ff4106f60 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 rx=0x7f8fe4013a70 tx=0x7f8fe4009b30).stop 2021-11-03T18:19:30.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.709+0000 7f8ffa855700 1 -- 172.21.15.27:0/2107384675 shutdown_connections 2021-11-03T18:19:30.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.709+0000 7f8ffa855700 1 --2- 172.21.15.27:0/2107384675 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8fdc0640b0 0x7f8fdc066570 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:19:30.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.709+0000 7f8ffa855700 1 --2- 172.21.15.27:0/2107384675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ff40f66e0 0x7f8ff4106f60 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:19:30.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.709+0000 7f8ffa855700 1 --2- 172.21.15.27:0/2107384675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ff40f7980 0x7f8ff4107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:19:30.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.709+0000 7f8ffa855700 1 -- 172.21.15.27:0/2107384675 >> 172.21.15.27:0/2107384675 conn(0x7f8ff40f1e40 msgr2=0x7f8ff40fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:19:30.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.709+0000 7f8ffa855700 1 -- 172.21.15.27:0/2107384675 shutdown_connections 2021-11-03T18:19:30.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.709+0000 7f8ffa855700 1 -- 172.21.15.27:0/2107384675 wait complete. 2021-11-03T18:19:30.716 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:19:30.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:30 smithi115 ceph-mon[65453]: pgmap v1945: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:30.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3812.451073 secs 2021-11-03T18:19:30.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:30 smithi027 ceph-mon[97244]: pgmap v1945: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:30.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3812.451073 secs 2021-11-03T18:19:30.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.809+0000 7f0f1bc21700 1 -- 172.21.15.27:0/3317190682 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f140f98b0 msgr2=0x7f0f140fbca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:19:30.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.809+0000 7f0f1bc21700 1 --2- 172.21.15.27:0/3317190682 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f140f98b0 0x7f0f140fbca0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 rx=0x7f0f10004660 tx=0x7f0f10009b30).stop 2021-11-03T18:19:30.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.809+0000 7f0f1bc21700 1 -- 172.21.15.27:0/3317190682 shutdown_connections 2021-11-03T18:19:30.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.809+0000 7f0f1bc21700 1 --2- 172.21.15.27:0/3317190682 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f140f98b0 0x7f0f140fbca0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:19:30.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.809+0000 7f0f1bc21700 1 --2- 172.21.15.27:0/3317190682 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f14060330 0x7f0f140f93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:19:30.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.809+0000 7f0f1bc21700 1 -- 172.21.15.27:0/3317190682 >> 172.21.15.27:0/3317190682 conn(0x7f0f140f0de0 msgr2=0x7f0f140f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:19:30.811 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.809+0000 7f0f1bc21700 1 -- 172.21.15.27:0/3317190682 shutdown_connections 2021-11-03T18:19:30.811 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.809+0000 7f0f1bc21700 1 -- 172.21.15.27:0/3317190682 wait complete. 2021-11-03T18:19:30.812 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.810+0000 7f0f1bc21700 1 Processor -- start 2021-11-03T18:19:30.812 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.810+0000 7f0f1bc21700 1 -- start start 2021-11-03T18:19:30.812 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.810+0000 7f0f1bc21700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f14060330 0x7f0f141070d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:19:30.812 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.810+0000 7f0f1bc21700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f141075c0 0x7f0f141084b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:19:30.813 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.810+0000 7f0f1bc21700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f14107a90 con 0x7f0f141075c0 2021-11-03T18:19:30.813 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.810+0000 7f0f1bc21700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f14107bd0 con 0x7f0f14060330 2021-11-03T18:19:30.813 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.810+0000 7f0f191bc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f141075c0 0x7f0f141084b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:19:30.813 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.810+0000 7f0f191bc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f141075c0 0x7f0f141084b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34220/0 (socket says 172.21.15.27:34220) 2021-11-03T18:19:30.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.810+0000 7f0f191bc700 1 -- 172.21.15.27:0/3743737159 learned_addr learned my addr 172.21.15.27:0/3743737159 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:19:30.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.810+0000 7f0f199bd700 1 --2- 172.21.15.27:0/3743737159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f14060330 0x7f0f141070d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:19:30.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.810+0000 7f0f191bc700 1 -- 172.21.15.27:0/3743737159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f14060330 msgr2=0x7f0f141070d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:19:30.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.810+0000 7f0f191bc700 1 --2- 172.21.15.27:0/3743737159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f14060330 0x7f0f141070d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:19:30.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.810+0000 7f0f191bc700 1 -- 172.21.15.27:0/3743737159 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f10005040 con 0x7f0f141075c0 2021-11-03T18:19:30.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.810+0000 7f0f199bd700 1 --2- 172.21.15.27:0/3743737159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f14060330 0x7f0f141070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:19:30.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.810+0000 7f0f191bc700 1 --2- 172.21.15.27:0/3743737159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f141075c0 0x7f0f141084b0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 rx=0x7f0f10000c00 tx=0x7f0f10009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:19:30.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.811+0000 7f0f0a7fc700 1 -- 172.21.15.27:0/3743737159 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f10019070 con 0x7f0f141075c0 2021-11-03T18:19:30.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.811+0000 7f0f0a7fc700 1 -- 172.21.15.27:0/3743737159 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f0f100097d0 con 0x7f0f141075c0 2021-11-03T18:19:30.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.811+0000 7f0f1bc21700 1 -- 172.21.15.27:0/3743737159 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f14108a00 con 0x7f0f141075c0 2021-11-03T18:19:30.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.811+0000 7f0f0a7fc700 1 -- 172.21.15.27:0/3743737159 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f10014590 con 0x7f0f141075c0 2021-11-03T18:19:30.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.811+0000 7f0f1bc21700 1 -- 172.21.15.27:0/3743737159 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f14108f00 con 0x7f0f141075c0 2021-11-03T18:19:30.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.812+0000 7f0f1bc21700 1 -- 172.21.15.27:0/3743737159 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f14045ad0 con 0x7f0f141075c0 2021-11-03T18:19:30.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.812+0000 7f0f0a7fc700 1 -- 172.21.15.27:0/3743737159 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f0f10014800 con 0x7f0f141075c0 2021-11-03T18:19:30.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.812+0000 7f0f0a7fc700 1 --2- 172.21.15.27:0/3743737159 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0f000640b0 0x7f0f00066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:19:30.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.813+0000 7f0f199bd700 1 --2- 172.21.15.27:0/3743737159 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0f000640b0 0x7f0f00066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:19:30.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.813+0000 7f0f0a7fc700 1 -- 172.21.15.27:0/3743737159 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f0f10081b70 con 0x7f0f141075c0 2021-11-03T18:19:30.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.813+0000 7f0f199bd700 1 --2- 172.21.15.27:0/3743737159 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0f000640b0 0x7f0f00066570 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7f0f14060100 tx=0x7f0f04009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:19:30.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.817+0000 7f0f0a7fc700 1 -- 172.21.15.27:0/3743737159 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f0f10053f30 con 0x7f0f141075c0 2021-11-03T18:19:30.987 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:30.989+0000 7f0f1bc21700 1 -- 172.21.15.27:0/3743737159 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f14109180 con 0x7f0f000640b0 2021-11-03T18:19:31.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.004+0000 7f0f0a7fc700 1 -- 172.21.15.27:0/3743737159 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f0f14109180 con 0x7f0f000640b0 2021-11-03T18:19:31.002 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:19:31.002 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (71m) 3s ago 72m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:19:31.003 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (62m) 3s ago 72m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:19:31.003 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (62m) 9m ago 71m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:19:31.003 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (70m) 3s ago 71m 42.4M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:19:31.003 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (64m) 3s ago 67m 26.5M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:19:31.004 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (67m) 3s ago 67m 3849M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:19:31.004 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (63m) 9m ago 67m 25.9M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:19:31.004 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (64m) 9m ago 67m 27.0M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:19:31.004 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (64m) 3s ago 73m 590M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:19:31.004 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (64m) 9m ago 71m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:19:31.005 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (63m) 3s ago 73m 312M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:19:31.005 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (63m) 9m ago 71m 129M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:19:31.005 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (71m) 3s ago 71m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:19:31.005 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (71m) 9m ago 71m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:19:31.006 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (62m) 3s ago 70m 392M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:19:31.006 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (61m) 3s ago 70m 427M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:19:31.006 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (61m) 3s ago 70m 320M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:19:31.006 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (60m) 9m ago 69m 429M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:19:31.007 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (60m) 9m ago 69m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:19:31.007 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (60m) 9m ago 69m 348M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:19:31.007 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (70m) 3s ago 71m 59.9M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:19:31.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.006+0000 7f0f1bc21700 1 -- 172.21.15.27:0/3743737159 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0f000640b0 msgr2=0x7f0f00066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:19:31.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.006+0000 7f0f1bc21700 1 --2- 172.21.15.27:0/3743737159 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0f000640b0 0x7f0f00066570 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 rx=0x7f0f14060100 tx=0x7f0f04009040).stop 2021-11-03T18:19:31.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.006+0000 7f0f1bc21700 1 -- 172.21.15.27:0/3743737159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f141075c0 msgr2=0x7f0f141084b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:19:31.009 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.006+0000 7f0f1bc21700 1 --2- 172.21.15.27:0/3743737159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f141075c0 0x7f0f141084b0 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 rx=0x7f0f10000c00 tx=0x7f0f10009b30).stop 2021-11-03T18:19:31.009 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.006+0000 7f0f1bc21700 1 -- 172.21.15.27:0/3743737159 shutdown_connections 2021-11-03T18:19:31.009 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.006+0000 7f0f1bc21700 1 --2- 172.21.15.27:0/3743737159 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0f000640b0 0x7f0f00066570 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:19:31.009 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.006+0000 7f0f1bc21700 1 --2- 172.21.15.27:0/3743737159 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f14060330 0x7f0f141070d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:19:31.010 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.006+0000 7f0f1bc21700 1 --2- 172.21.15.27:0/3743737159 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f141075c0 0x7f0f141084b0 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:19:31.010 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.006+0000 7f0f1bc21700 1 -- 172.21.15.27:0/3743737159 >> 172.21.15.27:0/3743737159 conn(0x7f0f140f0de0 msgr2=0x7f0f140f31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:19:31.011 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.006+0000 7f0f1bc21700 1 -- 172.21.15.27:0/3743737159 shutdown_connections 2021-11-03T18:19:31.011 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.006+0000 7f0f1bc21700 1 -- 172.21.15.27:0/3743737159 wait complete. 2021-11-03T18:19:31.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.105+0000 7f064d020700 1 -- 172.21.15.27:0/1125172345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06480f49a0 msgr2=0x7f06480f4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:19:31.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.105+0000 7f064d020700 1 --2- 172.21.15.27:0/1125172345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06480f49a0 0x7f06480f4dc0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 rx=0x7f06400099e0 tx=0x7f0640004d10).stop 2021-11-03T18:19:31.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.105+0000 7f064d020700 1 -- 172.21.15.27:0/1125172345 shutdown_connections 2021-11-03T18:19:31.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.105+0000 7f064d020700 1 --2- 172.21.15.27:0/1125172345 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06480f5340 0x7f06480f7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:19:31.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.105+0000 7f064d020700 1 --2- 172.21.15.27:0/1125172345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06480f49a0 0x7f06480f4dc0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:19:31.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.105+0000 7f064d020700 1 -- 172.21.15.27:0/1125172345 >> 172.21.15.27:0/1125172345 conn(0x7f06480f04c0 msgr2=0x7f06480f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:19:31.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.105+0000 7f064d020700 1 -- 172.21.15.27:0/1125172345 shutdown_connections 2021-11-03T18:19:31.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.105+0000 7f064d020700 1 -- 172.21.15.27:0/1125172345 wait complete. 2021-11-03T18:19:31.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.106+0000 7f064d020700 1 Processor -- start 2021-11-03T18:19:31.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.106+0000 7f064d020700 1 -- start start 2021-11-03T18:19:31.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.106+0000 7f064d020700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06480f49a0 0x7f06481070e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:19:31.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.106+0000 7f064d020700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06480f5340 0x7f06481075d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:19:31.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.106+0000 7f064d020700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0648107b80 con 0x7f06480f5340 2021-11-03T18:19:31.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.106+0000 7f064d020700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0648107cc0 con 0x7f06480f49a0 2021-11-03T18:19:31.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.106+0000 7f064659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06480f5340 0x7f06481075d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:19:31.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.106+0000 7f064659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06480f5340 0x7f06481075d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34230/0 (socket says 172.21.15.27:34230) 2021-11-03T18:19:31.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.106+0000 7f064659c700 1 -- 172.21.15.27:0/1530324481 learned_addr learned my addr 172.21.15.27:0/1530324481 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:19:31.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.106+0000 7f0646d9d700 1 --2- 172.21.15.27:0/1530324481 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06480f49a0 0x7f06481070e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:19:31.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.106+0000 7f064659c700 1 -- 172.21.15.27:0/1530324481 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06480f49a0 msgr2=0x7f06481070e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:19:31.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.106+0000 7f064659c700 1 --2- 172.21.15.27:0/1530324481 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06480f49a0 0x7f06481070e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:19:31.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.106+0000 7f064659c700 1 -- 172.21.15.27:0/1530324481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0640009640 con 0x7f06480f5340 2021-11-03T18:19:31.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.106+0000 7f064659c700 1 --2- 172.21.15.27:0/1530324481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06480f5340 0x7f06481075d0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 rx=0x7f063000c510 tx=0x7f0630007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:19:31.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.107+0000 7f063f7fe700 1 -- 172.21.15.27:0/1530324481 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f063000f650 con 0x7f06480f5340 2021-11-03T18:19:31.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.107+0000 7f063f7fe700 1 -- 172.21.15.27:0/1530324481 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f063000fc60 con 0x7f06480f5340 2021-11-03T18:19:31.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.107+0000 7f064d020700 1 -- 172.21.15.27:0/1530324481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0648108660 con 0x7f06480f5340 2021-11-03T18:19:31.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.107+0000 7f063f7fe700 1 -- 172.21.15.27:0/1530324481 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0630012cd0 con 0x7f06480f5340 2021-11-03T18:19:31.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.107+0000 7f064d020700 1 -- 172.21.15.27:0/1530324481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0648108ba0 con 0x7f06480f5340 2021-11-03T18:19:31.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.108+0000 7f064d020700 1 -- 172.21.15.27:0/1530324481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0648045ad0 con 0x7f06480f5340 2021-11-03T18:19:31.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.109+0000 7f063f7fe700 1 -- 172.21.15.27:0/1530324481 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f0630010070 con 0x7f06480f5340 2021-11-03T18:19:31.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.109+0000 7f063f7fe700 1 --2- 172.21.15.27:0/1530324481 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f06340640b0 0x7f0634066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:19:31.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.109+0000 7f063f7fe700 1 -- 172.21.15.27:0/1530324481 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f0630080f40 con 0x7f06480f5340 2021-11-03T18:19:31.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.109+0000 7f0646d9d700 1 --2- 172.21.15.27:0/1530324481 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f06340640b0 0x7f0634066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:19:31.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.110+0000 7f0646d9d700 1 --2- 172.21.15.27:0/1530324481 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f06340640b0 0x7f0634066570 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f0640009f50 tx=0x7f0640004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:19:31.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.113+0000 7f063f7fe700 1 -- 172.21.15.27:0/1530324481 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f0630053150 con 0x7f06480f5340 2021-11-03T18:19:31.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.343+0000 7f064d020700 1 -- 172.21.15.27:0/1530324481 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0648027a00 con 0x7f06480f5340 2021-11-03T18:19:31.342 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.344+0000 7f063f7fe700 1 -- 172.21.15.27:0/1530324481 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f0630020070 con 0x7f06480f5340 2021-11-03T18:19:31.342 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:19:31.343 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:19:31.343 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:19:31.343 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:19:31.343 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:19:31.343 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:19:31.344 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:19:31.344 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:19:31.344 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:19:31.345 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:19:31.346 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:19:31.346 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:19:31.346 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:19:31.347 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:19:31.347 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:19:31.347 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:19:31.347 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:19:31.348 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:19:31.348 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.345+0000 7f064d020700 1 -- 172.21.15.27:0/1530324481 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f06340640b0 msgr2=0x7f0634066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:19:31.349 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.345+0000 7f064d020700 1 --2- 172.21.15.27:0/1530324481 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f06340640b0 0x7f0634066570 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 rx=0x7f0640009f50 tx=0x7f0640004d10).stop 2021-11-03T18:19:31.349 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.345+0000 7f064d020700 1 -- 172.21.15.27:0/1530324481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06480f5340 msgr2=0x7f06481075d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:19:31.349 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.345+0000 7f064d020700 1 --2- 172.21.15.27:0/1530324481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06480f5340 0x7f06481075d0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 rx=0x7f063000c510 tx=0x7f0630007df0).stop 2021-11-03T18:19:31.349 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.345+0000 7f064d020700 1 -- 172.21.15.27:0/1530324481 shutdown_connections 2021-11-03T18:19:31.350 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.345+0000 7f064d020700 1 --2- 172.21.15.27:0/1530324481 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f06340640b0 0x7f0634066570 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:19:31.350 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.345+0000 7f064d020700 1 --2- 172.21.15.27:0/1530324481 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06480f49a0 0x7f06481070e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:19:31.350 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.345+0000 7f064d020700 1 --2- 172.21.15.27:0/1530324481 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06480f5340 0x7f06481075d0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:19:31.350 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.346+0000 7f064d020700 1 -- 172.21.15.27:0/1530324481 >> 172.21.15.27:0/1530324481 conn(0x7f06480f04c0 msgr2=0x7f06480fb510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:19:31.351 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.346+0000 7f064d020700 1 -- 172.21.15.27:0/1530324481 shutdown_connections 2021-11-03T18:19:31.351 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.346+0000 7f064d020700 1 -- 172.21.15.27:0/1530324481 wait complete. 2021-11-03T18:19:31.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.445+0000 7f0c68c60700 1 -- 172.21.15.27:0/2872764161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c640f66e0 msgr2=0x7f0c640f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:19:31.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.445+0000 7f0c68c60700 1 --2- 172.21.15.27:0/2872764161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c640f66e0 0x7f0c640f6b40 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 rx=0x7f0c58004660 tx=0x7f0c58009b30).stop 2021-11-03T18:19:31.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.445+0000 7f0c68c60700 1 -- 172.21.15.27:0/2872764161 shutdown_connections 2021-11-03T18:19:31.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.445+0000 7f0c68c60700 1 --2- 172.21.15.27:0/2872764161 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c640f66e0 0x7f0c640f6b40 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:19:31.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.445+0000 7f0c68c60700 1 --2- 172.21.15.27:0/2872764161 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c640f5440 0x7f0c640f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:19:31.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.445+0000 7f0c68c60700 1 -- 172.21.15.27:0/2872764161 >> 172.21.15.27:0/2872764161 conn(0x7f0c640f0ba0 msgr2=0x7f0c640f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:19:31.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.445+0000 7f0c68c60700 1 -- 172.21.15.27:0/2872764161 shutdown_connections 2021-11-03T18:19:31.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.445+0000 7f0c68c60700 1 -- 172.21.15.27:0/2872764161 wait complete. 2021-11-03T18:19:31.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.446+0000 7f0c68c60700 1 Processor -- start 2021-11-03T18:19:31.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.446+0000 7f0c68c60700 1 -- start start 2021-11-03T18:19:31.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.446+0000 7f0c68c60700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c640f5440 0x7f0c64106d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:19:31.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.446+0000 7f0c68c60700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c640f66e0 0x7f0c64107240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:19:31.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.446+0000 7f0c68c60700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c641077f0 con 0x7f0c640f5440 2021-11-03T18:19:31.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.446+0000 7f0c68c60700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c64107ff0 con 0x7f0c640f66e0 2021-11-03T18:19:31.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.446+0000 7f0c637fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c640f5440 0x7f0c64106d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:19:31.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.446+0000 7f0c637fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c640f5440 0x7f0c64106d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34238/0 (socket says 172.21.15.27:34238) 2021-11-03T18:19:31.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.446+0000 7f0c637fe700 1 -- 172.21.15.27:0/1995264370 learned_addr learned my addr 172.21.15.27:0/1995264370 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:19:31.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.447+0000 7f0c62ffd700 1 --2- 172.21.15.27:0/1995264370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c640f66e0 0x7f0c64107240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:19:31.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.447+0000 7f0c637fe700 1 -- 172.21.15.27:0/1995264370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c640f66e0 msgr2=0x7f0c64107240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:19:31.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.447+0000 7f0c637fe700 1 --2- 172.21.15.27:0/1995264370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c640f66e0 0x7f0c64107240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:19:31.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.447+0000 7f0c637fe700 1 -- 172.21.15.27:0/1995264370 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c58005040 con 0x7f0c640f5440 2021-11-03T18:19:31.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.447+0000 7f0c62ffd700 1 --2- 172.21.15.27:0/1995264370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c640f66e0 0x7f0c64107240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:19:31.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.447+0000 7f0c637fe700 1 --2- 172.21.15.27:0/1995264370 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c640f5440 0x7f0c64106d50 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 rx=0x7f0c5400e7c0 tx=0x7f0c5400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:19:31.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.447+0000 7f0c4bfff700 1 -- 172.21.15.27:0/1995264370 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c540092e0 con 0x7f0c640f5440 2021-11-03T18:19:31.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.447+0000 7f0c4bfff700 1 -- 172.21.15.27:0/1995264370 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f0c5400f040 con 0x7f0c640f5440 2021-11-03T18:19:31.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.447+0000 7f0c68c60700 1 -- 172.21.15.27:0/1995264370 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c641082d0 con 0x7f0c640f5440 2021-11-03T18:19:31.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.447+0000 7f0c4bfff700 1 -- 172.21.15.27:0/1995264370 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c54014750 con 0x7f0c640f5440 2021-11-03T18:19:31.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.447+0000 7f0c68c60700 1 -- 172.21.15.27:0/1995264370 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c64108890 con 0x7f0c640f5440 2021-11-03T18:19:31.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.448+0000 7f0c68c60700 1 -- 172.21.15.27:0/1995264370 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c64059100 con 0x7f0c640f5440 2021-11-03T18:19:31.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.448+0000 7f0c4bfff700 1 -- 172.21.15.27:0/1995264370 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f0c5401b030 con 0x7f0c640f5440 2021-11-03T18:19:31.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.449+0000 7f0c4bfff700 1 --2- 172.21.15.27:0/1995264370 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0c4c064100 0x7f0c4c0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:19:31.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.449+0000 7f0c4bfff700 1 -- 172.21.15.27:0/1995264370 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f0c540809c0 con 0x7f0c640f5440 2021-11-03T18:19:31.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.449+0000 7f0c62ffd700 1 --2- 172.21.15.27:0/1995264370 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0c4c064100 0x7f0c4c0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:19:31.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.449+0000 7f0c62ffd700 1 --2- 172.21.15.27:0/1995264370 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0c4c064100 0x7f0c4c0665c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7f0c58005010 tx=0x7f0c5800b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:19:31.457 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.453+0000 7f0c4bfff700 1 -- 172.21.15.27:0/1995264370 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f0c54052bd0 con 0x7f0c640f5440 2021-11-03T18:19:31.656 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.658+0000 7f0c68c60700 1 -- 172.21.15.27:0/1995264370 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0c640008d0 con 0x7f0c640f5440 2021-11-03T18:19:31.656 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.658+0000 7f0c4bfff700 1 -- 172.21.15.27:0/1995264370 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f0c54019710 con 0x7f0c640f5440 2021-11-03T18:19:31.657 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:19:31.657 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:19:31.657 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:19:31.657 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:19:31.657 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:19:31.658 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:19:31.658 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:19:31.658 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:19:31.658 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:19:31.659 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:19:31.659 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:19:31.659 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:19:31.660 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:19:31.660 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:19:31.660 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:19:31.660 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:19:31.661 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:19:31.661 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:19:31.661 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:19:31.661 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:19:31.662 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:19:31.662 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:19:31.662 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:19:31.662 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:19:31.662 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:19:31.663 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:19:31.663 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:19:31.663 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:19:31.663 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:19:31.664 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:19:31.664 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:19:31.664 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:19:31.664 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:19:31.664 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:19:31.665 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:19:31.665 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:19:31.665 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:19:31.665 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:19:31.665 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:19:31.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.660+0000 7f0c68c60700 1 -- 172.21.15.27:0/1995264370 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0c4c064100 msgr2=0x7f0c4c0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:19:31.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.660+0000 7f0c68c60700 1 --2- 172.21.15.27:0/1995264370 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0c4c064100 0x7f0c4c0665c0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 rx=0x7f0c58005010 tx=0x7f0c5800b040).stop 2021-11-03T18:19:31.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.660+0000 7f0c68c60700 1 -- 172.21.15.27:0/1995264370 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c640f5440 msgr2=0x7f0c64106d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:19:31.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.660+0000 7f0c68c60700 1 --2- 172.21.15.27:0/1995264370 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c640f5440 0x7f0c64106d50 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 rx=0x7f0c5400e7c0 tx=0x7f0c5400b040).stop 2021-11-03T18:19:31.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.660+0000 7f0c68c60700 1 -- 172.21.15.27:0/1995264370 shutdown_connections 2021-11-03T18:19:31.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.660+0000 7f0c68c60700 1 --2- 172.21.15.27:0/1995264370 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0c4c064100 0x7f0c4c0665c0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:19:31.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.660+0000 7f0c68c60700 1 --2- 172.21.15.27:0/1995264370 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0c640f5440 0x7f0c64106d50 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:19:31.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.660+0000 7f0c68c60700 1 --2- 172.21.15.27:0/1995264370 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0c640f66e0 0x7f0c64107240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:19:31.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.660+0000 7f0c68c60700 1 -- 172.21.15.27:0/1995264370 >> 172.21.15.27:0/1995264370 conn(0x7f0c640f0ba0 msgr2=0x7f0c640fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:19:31.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.660+0000 7f0c68c60700 1 -- 172.21.15.27:0/1995264370 shutdown_connections 2021-11-03T18:19:31.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:19:31.660+0000 7f0c68c60700 1 -- 172.21.15.27:0/1995264370 wait complete. 2021-11-03T18:19:31.669 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:19:31.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:31 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1530324481' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:31.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:31 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1530324481' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:32.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:32 smithi115 ceph-mon[65453]: from='client.36104 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:19:32.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:32 smithi115 ceph-mon[65453]: pgmap v1946: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:32.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:32 smithi115 ceph-mon[65453]: from='client.36108 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:19:32.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:32 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1995264370' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:19:32.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:32 smithi027 ceph-mon[97244]: from='client.36104 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:19:32.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:32 smithi027 ceph-mon[97244]: pgmap v1946: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:32.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:32 smithi027 ceph-mon[97244]: from='client.36108 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:19:32.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:32 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1995264370' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:19:34.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:34 smithi115 ceph-mon[65453]: pgmap v1947: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:34.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:34 smithi027 ceph-mon[97244]: pgmap v1947: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:35.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3817.451208 secs 2021-11-03T18:19:35.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3817.451208 secs 2021-11-03T18:19:36.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:36 smithi115 ceph-mon[65453]: pgmap v1948: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:36.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:36 smithi027 ceph-mon[97244]: pgmap v1948: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:37.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:19:37.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:19:37.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:19:37.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:37.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:37.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:37.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:37.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:19:37.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:19:37.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:19:37.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:37.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:37.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:37.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:38.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:38 smithi115 ceph-mon[65453]: pgmap v1949: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:38.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:38 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:19:39.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:38 smithi027 ceph-mon[97244]: pgmap v1949: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:39.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:38 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:19:40.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:40 smithi115 ceph-mon[65453]: pgmap v1950: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:40.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3822.451351 secs 2021-11-03T18:19:41.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:40 smithi027 ceph-mon[97244]: pgmap v1950: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:41.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3822.451351 secs 2021-11-03T18:19:42.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:42 smithi027 ceph-mon[97244]: pgmap v1951: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:42.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:42 smithi115 ceph-mon[65453]: pgmap v1951: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:44.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:44 smithi115 ceph-mon[65453]: pgmap v1952: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:45.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:44 smithi027 ceph-mon[97244]: pgmap v1952: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:45.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3827.451475 secs 2021-11-03T18:19:46.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3827.451475 secs 2021-11-03T18:19:46.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:46 smithi115 ceph-mon[65453]: pgmap v1953: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:47.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:46 smithi027 ceph-mon[97244]: pgmap v1953: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:47.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:19:47.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:19:47.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:19:47.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:47.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:47.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:47.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:48.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:19:48.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:19:48.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:19:48.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:48.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:48.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:48.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:48.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:48 smithi115 ceph-mon[65453]: pgmap v1954: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:48.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:48 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:19:49.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:48 smithi027 ceph-mon[97244]: pgmap v1954: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:49.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:48 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:19:50.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:50 smithi115 ceph-mon[65453]: pgmap v1955: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:50.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3832.451589 secs 2021-11-03T18:19:51.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:50 smithi027 ceph-mon[97244]: pgmap v1955: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:51.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3832.451589 secs 2021-11-03T18:19:52.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:52 smithi027 ceph-mon[97244]: pgmap v1956: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:52.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:52 smithi115 ceph-mon[65453]: pgmap v1956: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:54.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:54 smithi115 ceph-mon[65453]: pgmap v1957: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:55.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:54 smithi027 ceph-mon[97244]: pgmap v1957: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:55.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3837.451731 secs 2021-11-03T18:19:56.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3837.451731 secs 2021-11-03T18:19:56.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:56 smithi115 ceph-mon[65453]: pgmap v1958: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:56.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:19:56.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:19:57.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:56 smithi027 ceph-mon[97244]: pgmap v1958: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:57.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:19:57.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:19:57.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:19:57.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:19:57.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:19:57.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:57.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:57.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:57.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:58.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:19:58.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:19:58.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:19:58.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:58.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:58.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:58.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:19:58.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:58 smithi115 ceph-mon[65453]: pgmap v1959: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:58.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:19:58 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:19:59.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:58 smithi027 ceph-mon[97244]: pgmap v1959: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:19:59.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:19:58 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:20:00.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:00 smithi115 ceph-mon[65453]: pgmap v1960: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:00.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3842.451856 secs 2021-11-03T18:20:00.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:00 smithi115 ceph-mon[65453]: overall HEALTH_WARN 1 filesystem is degraded; 1 filesystem has a failed mds daemon; 1 MDSs report slow requests; 3 daemons have recently crashed 2021-11-03T18:20:01.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:00 smithi027 ceph-mon[97244]: pgmap v1960: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:01.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3842.451856 secs 2021-11-03T18:20:01.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:00 smithi027 ceph-mon[97244]: overall HEALTH_WARN 1 filesystem is degraded; 1 filesystem has a failed mds daemon; 1 MDSs report slow requests; 3 daemons have recently crashed 2021-11-03T18:20:01.774 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.775+0000 7fe21b26b700 1 -- 172.21.15.27:0/3905449363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2140f77d0 msgr2=0x7fe2140f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:20:01.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.775+0000 7fe21b26b700 1 --2- 172.21.15.27:0/3905449363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2140f77d0 0x7fe2140f7c30 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 rx=0x7fe204004660 tx=0x7fe204009b30).stop 2021-11-03T18:20:01.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.775+0000 7fe21b26b700 1 -- 172.21.15.27:0/3905449363 shutdown_connections 2021-11-03T18:20:01.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.775+0000 7fe21b26b700 1 --2- 172.21.15.27:0/3905449363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2140f77d0 0x7fe2140f7c30 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:01.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.775+0000 7fe21b26b700 1 --2- 172.21.15.27:0/3905449363 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2140f6530 0x7fe2140f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:01.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.775+0000 7fe21b26b700 1 -- 172.21.15.27:0/3905449363 >> 172.21.15.27:0/3905449363 conn(0x7fe2140f1cd0 msgr2=0x7fe2140f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:20:01.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.775+0000 7fe21b26b700 1 -- 172.21.15.27:0/3905449363 shutdown_connections 2021-11-03T18:20:01.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.775+0000 7fe21b26b700 1 -- 172.21.15.27:0/3905449363 wait complete. 2021-11-03T18:20:01.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.775+0000 7fe21b26b700 1 Processor -- start 2021-11-03T18:20:01.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.775+0000 7fe21b26b700 1 -- start start 2021-11-03T18:20:01.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.775+0000 7fe21b26b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2140f6530 0x7fe21406b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:20:01.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.775+0000 7fe21b26b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2140f77d0 0x7fe214069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:20:01.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.775+0000 7fe21b26b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21406a380 con 0x7fe2140f6530 2021-11-03T18:20:01.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.775+0000 7fe21b26b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21406a4c0 con 0x7fe2140f77d0 2021-11-03T18:20:01.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.776+0000 7fe21a269700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2140f6530 0x7fe21406b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:20:01.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.776+0000 7fe21a269700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2140f6530 0x7fe21406b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34256/0 (socket says 172.21.15.27:34256) 2021-11-03T18:20:01.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.776+0000 7fe21a269700 1 -- 172.21.15.27:0/1958338554 learned_addr learned my addr 172.21.15.27:0/1958338554 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:20:01.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.776+0000 7fe219a68700 1 --2- 172.21.15.27:0/1958338554 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2140f77d0 0x7fe214069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:20:01.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.776+0000 7fe21a269700 1 -- 172.21.15.27:0/1958338554 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2140f77d0 msgr2=0x7fe214069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:20:01.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.776+0000 7fe21a269700 1 --2- 172.21.15.27:0/1958338554 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2140f77d0 0x7fe214069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:01.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.776+0000 7fe21a269700 1 -- 172.21.15.27:0/1958338554 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe204005040 con 0x7fe2140f6530 2021-11-03T18:20:01.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.776+0000 7fe21a269700 1 --2- 172.21.15.27:0/1958338554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2140f6530 0x7fe21406b9a0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 rx=0x7fe210002700 tx=0x7fe210009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:20:01.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.776+0000 7fe20affd700 1 -- 172.21.15.27:0/1958338554 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2100079c0 con 0x7fe2140f6530 2021-11-03T18:20:01.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.776+0000 7fe20affd700 1 -- 172.21.15.27:0/1958338554 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fe21000f040 con 0x7fe2140f6530 2021-11-03T18:20:01.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.776+0000 7fe20affd700 1 -- 172.21.15.27:0/1958338554 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe210014860 con 0x7fe2140f6530 2021-11-03T18:20:01.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.776+0000 7fe21b26b700 1 -- 172.21.15.27:0/1958338554 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe21406a7a0 con 0x7fe2140f6530 2021-11-03T18:20:01.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.776+0000 7fe21b26b700 1 -- 172.21.15.27:0/1958338554 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe21410ed40 con 0x7fe2140f6530 2021-11-03T18:20:01.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.777+0000 7fe21b26b700 1 -- 172.21.15.27:0/1958338554 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe214045ad0 con 0x7fe2140f6530 2021-11-03T18:20:01.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.778+0000 7fe20affd700 1 -- 172.21.15.27:0/1958338554 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fe21001b060 con 0x7fe2140f6530 2021-11-03T18:20:01.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.778+0000 7fe20affd700 1 --2- 172.21.15.27:0/1958338554 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe200064100 0x7fe2000665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:20:01.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.778+0000 7fe20affd700 1 -- 172.21.15.27:0/1958338554 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fe2100806d0 con 0x7fe2140f6530 2021-11-03T18:20:01.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.778+0000 7fe219a68700 1 --2- 172.21.15.27:0/1958338554 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe200064100 0x7fe2000665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:20:01.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.779+0000 7fe219a68700 1 --2- 172.21.15.27:0/1958338554 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe200064100 0x7fe2000665c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7fe204004e30 tx=0x7fe204009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:20:01.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.783+0000 7fe20affd700 1 -- 172.21.15.27:0/1958338554 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fe2100529d0 con 0x7fe2140f6530 2021-11-03T18:20:01.962 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.964+0000 7fe21b26b700 1 -- 172.21.15.27:0/1958338554 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2140feea0 con 0x7fe200064100 2021-11-03T18:20:01.963 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.965+0000 7fe20affd700 1 -- 172.21.15.27:0/1958338554 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fe204019040 con 0x7fe200064100 2021-11-03T18:20:01.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.967+0000 7fe21b26b700 1 -- 172.21.15.27:0/1958338554 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe200064100 msgr2=0x7fe2000665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:20:01.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.967+0000 7fe21b26b700 1 --2- 172.21.15.27:0/1958338554 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe200064100 0x7fe2000665c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7fe204004e30 tx=0x7fe204009b30).stop 2021-11-03T18:20:01.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.967+0000 7fe21b26b700 1 -- 172.21.15.27:0/1958338554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2140f6530 msgr2=0x7fe21406b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:20:01.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.967+0000 7fe21b26b700 1 --2- 172.21.15.27:0/1958338554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2140f6530 0x7fe21406b9a0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 rx=0x7fe210002700 tx=0x7fe210009d80).stop 2021-11-03T18:20:01.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.967+0000 7fe21b26b700 1 -- 172.21.15.27:0/1958338554 shutdown_connections 2021-11-03T18:20:01.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.967+0000 7fe21b26b700 1 --2- 172.21.15.27:0/1958338554 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe200064100 0x7fe2000665c0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:01.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.967+0000 7fe21b26b700 1 --2- 172.21.15.27:0/1958338554 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2140f6530 0x7fe21406b9a0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:01.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.967+0000 7fe21b26b700 1 --2- 172.21.15.27:0/1958338554 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2140f77d0 0x7fe214069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:01.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.967+0000 7fe21b26b700 1 -- 172.21.15.27:0/1958338554 >> 172.21.15.27:0/1958338554 conn(0x7fe2140f1cd0 msgr2=0x7fe2140fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:20:01.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.968+0000 7fe21b26b700 1 -- 172.21.15.27:0/1958338554 shutdown_connections 2021-11-03T18:20:01.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:01.968+0000 7fe21b26b700 1 -- 172.21.15.27:0/1958338554 wait complete. 2021-11-03T18:20:01.975 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:20:02.068 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.068+0000 7f79622e9700 1 -- 172.21.15.27:0/2902379902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f795c0f55d0 msgr2=0x7f795c0f59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:20:02.068 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.068+0000 7f79622e9700 1 --2- 172.21.15.27:0/2902379902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f795c0f55d0 0x7f795c0f59f0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 rx=0x7f7950004660 tx=0x7f7950009b30).stop 2021-11-03T18:20:02.068 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.068+0000 7f79622e9700 1 -- 172.21.15.27:0/2902379902 shutdown_connections 2021-11-03T18:20:02.069 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.068+0000 7f79622e9700 1 --2- 172.21.15.27:0/2902379902 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f795c0f6870 0x7f795c0f6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:02.069 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.068+0000 7f79622e9700 1 --2- 172.21.15.27:0/2902379902 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f795c0f55d0 0x7f795c0f59f0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:02.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.068+0000 7f79622e9700 1 -- 172.21.15.27:0/2902379902 >> 172.21.15.27:0/2902379902 conn(0x7f795c0f0d70 msgr2=0x7f795c0f3190 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:20:02.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.068+0000 7f79622e9700 1 -- 172.21.15.27:0/2902379902 shutdown_connections 2021-11-03T18:20:02.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.068+0000 7f79622e9700 1 -- 172.21.15.27:0/2902379902 wait complete. 2021-11-03T18:20:02.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.068+0000 7f79622e9700 1 Processor -- start 2021-11-03T18:20:02.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.068+0000 7f79622e9700 1 -- start start 2021-11-03T18:20:02.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.068+0000 7f79622e9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f795c0f55d0 0x7f795c109c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:20:02.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.068+0000 7f79622e9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f795c0f6870 0x7f795c10c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:20:02.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.068+0000 7f79622e9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795c0604a0 con 0x7f795c0f55d0 2021-11-03T18:20:02.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.068+0000 7f79622e9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795c0605e0 con 0x7f795c0f6870 2021-11-03T18:20:02.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.069+0000 7f795bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f795c0f55d0 0x7f795c109c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:20:02.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.069+0000 7f795bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f795c0f55d0 0x7f795c109c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34266/0 (socket says 172.21.15.27:34266) 2021-11-03T18:20:02.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.069+0000 7f795bfff700 1 -- 172.21.15.27:0/2164780806 learned_addr learned my addr 172.21.15.27:0/2164780806 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:20:02.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.069+0000 7f795b7fe700 1 --2- 172.21.15.27:0/2164780806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f795c0f6870 0x7f795c10c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:20:02.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.069+0000 7f795bfff700 1 -- 172.21.15.27:0/2164780806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f795c0f6870 msgr2=0x7f795c10c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:20:02.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.069+0000 7f795bfff700 1 --2- 172.21.15.27:0/2164780806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f795c0f6870 0x7f795c10c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:02.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.069+0000 7f795bfff700 1 -- 172.21.15.27:0/2164780806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7950005040 con 0x7f795c0f55d0 2021-11-03T18:20:02.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.069+0000 7f795bfff700 1 --2- 172.21.15.27:0/2164780806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f795c0f55d0 0x7f795c109c90 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 rx=0x7f7950004f60 tx=0x7f7950009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:20:02.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.069+0000 7f7958ff9700 1 -- 172.21.15.27:0/2164780806 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7950018070 con 0x7f795c0f55d0 2021-11-03T18:20:02.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.069+0000 7f7958ff9700 1 -- 172.21.15.27:0/2164780806 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f7950009770 con 0x7f795c0f55d0 2021-11-03T18:20:02.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.069+0000 7f79622e9700 1 -- 172.21.15.27:0/2164780806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f795c10c6e0 con 0x7f795c0f55d0 2021-11-03T18:20:02.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.069+0000 7f7958ff9700 1 -- 172.21.15.27:0/2164780806 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f795001c3c0 con 0x7f795c0f55d0 2021-11-03T18:20:02.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.069+0000 7f79622e9700 1 -- 172.21.15.27:0/2164780806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f795c10cc10 con 0x7f795c0f55d0 2021-11-03T18:20:02.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.070+0000 7f79622e9700 1 -- 172.21.15.27:0/2164780806 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f795c045ad0 con 0x7f795c0f55d0 2021-11-03T18:20:02.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.071+0000 7f7958ff9700 1 -- 172.21.15.27:0/2164780806 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f795001d420 con 0x7f795c0f55d0 2021-11-03T18:20:02.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.071+0000 7f7958ff9700 1 --2- 172.21.15.27:0/2164780806 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f79440640b0 0x7f7944066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:20:02.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.071+0000 7f7958ff9700 1 -- 172.21.15.27:0/2164780806 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f7950082650 con 0x7f795c0f55d0 2021-11-03T18:20:02.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.071+0000 7f795b7fe700 1 --2- 172.21.15.27:0/2164780806 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f79440640b0 0x7f7944066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:20:02.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.072+0000 7f795b7fe700 1 --2- 172.21.15.27:0/2164780806 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f79440640b0 0x7f7944066570 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7f794c0041d0 tx=0x7f794c009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:20:02.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.075+0000 7f7958ff9700 1 -- 172.21.15.27:0/2164780806 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f7950054a10 con 0x7f795c0f55d0 2021-11-03T18:20:02.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.247+0000 7f79622e9700 1 -- 172.21.15.27:0/2164780806 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f795c057f10 con 0x7f79440640b0 2021-11-03T18:20:02.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.262+0000 7f7958ff9700 1 -- 172.21.15.27:0/2164780806 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f795c057f10 con 0x7f79440640b0 2021-11-03T18:20:02.260 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:20:02.260 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (71m) 34s ago 72m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:20:02.261 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (63m) 34s ago 72m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:20:02.261 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (63m) 9m ago 72m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:20:02.261 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (71m) 34s ago 72m 42.4M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:20:02.262 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (65m) 34s ago 67m 26.5M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:20:02.262 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (68m) 34s ago 68m 3849M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:20:02.262 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (63m) 9m ago 68m 25.9M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:20:02.262 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (65m) 9m ago 68m 27.0M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:20:02.263 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (65m) 34s ago 74m 590M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:20:02.263 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (64m) 9m ago 72m 421M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:20:02.263 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (64m) 34s ago 74m 312M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:20:02.263 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (63m) 9m ago 71m 129M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:20:02.264 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (72m) 34s ago 72m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:20:02.264 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (71m) 9m ago 71m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:20:02.264 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (62m) 34s ago 71m 392M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:20:02.265 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (62m) 34s ago 70m 427M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:20:02.265 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (61m) 34s ago 70m 320M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:20:02.265 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (61m) 9m ago 70m 429M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:20:02.265 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (60m) 9m ago 69m 340M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:20:02.266 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (60m) 9m ago 69m 348M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:20:02.266 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (71m) 34s ago 72m 59.9M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:20:02.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.264+0000 7f79622e9700 1 -- 172.21.15.27:0/2164780806 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f79440640b0 msgr2=0x7f7944066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:20:02.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.264+0000 7f79622e9700 1 --2- 172.21.15.27:0/2164780806 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f79440640b0 0x7f7944066570 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7f794c0041d0 tx=0x7f794c009040).stop 2021-11-03T18:20:02.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.264+0000 7f79622e9700 1 -- 172.21.15.27:0/2164780806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f795c0f55d0 msgr2=0x7f795c109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:20:02.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.264+0000 7f79622e9700 1 --2- 172.21.15.27:0/2164780806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f795c0f55d0 0x7f795c109c90 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 rx=0x7f7950004f60 tx=0x7f7950009b30).stop 2021-11-03T18:20:02.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.264+0000 7f79622e9700 1 -- 172.21.15.27:0/2164780806 shutdown_connections 2021-11-03T18:20:02.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.264+0000 7f79622e9700 1 --2- 172.21.15.27:0/2164780806 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f79440640b0 0x7f7944066570 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:02.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.264+0000 7f79622e9700 1 --2- 172.21.15.27:0/2164780806 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f795c0f55d0 0x7f795c109c90 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:02.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.264+0000 7f79622e9700 1 --2- 172.21.15.27:0/2164780806 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f795c0f6870 0x7f795c10c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:02.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.264+0000 7f79622e9700 1 -- 172.21.15.27:0/2164780806 >> 172.21.15.27:0/2164780806 conn(0x7f795c0f0d70 msgr2=0x7f795c0fa2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:20:02.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.264+0000 7f79622e9700 1 -- 172.21.15.27:0/2164780806 shutdown_connections 2021-11-03T18:20:02.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.264+0000 7f79622e9700 1 -- 172.21.15.27:0/2164780806 wait complete. 2021-11-03T18:20:02.363 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.364+0000 7f13717dd700 1 -- 172.21.15.27:0/2189454233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f136c0f6a10 msgr2=0x7f136c0f6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:20:02.363 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.364+0000 7f13717dd700 1 --2- 172.21.15.27:0/2189454233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f136c0f6a10 0x7f136c0f6e30 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 rx=0x7f135c004660 tx=0x7f135c009b30).stop 2021-11-03T18:20:02.363 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.365+0000 7f13717dd700 1 -- 172.21.15.27:0/2189454233 shutdown_connections 2021-11-03T18:20:02.363 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.365+0000 7f13717dd700 1 --2- 172.21.15.27:0/2189454233 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f136c0f73b0 0x7f136c0f4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:02.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.365+0000 7f13717dd700 1 --2- 172.21.15.27:0/2189454233 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f136c0f6a10 0x7f136c0f6e30 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:02.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.365+0000 7f13717dd700 1 -- 172.21.15.27:0/2189454233 >> 172.21.15.27:0/2189454233 conn(0x7f136c0f0320 msgr2=0x7f136c0f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:20:02.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.365+0000 7f13717dd700 1 -- 172.21.15.27:0/2189454233 shutdown_connections 2021-11-03T18:20:02.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.365+0000 7f13717dd700 1 -- 172.21.15.27:0/2189454233 wait complete. 2021-11-03T18:20:02.367 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.365+0000 7f13717dd700 1 Processor -- start 2021-11-03T18:20:02.367 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.365+0000 7f13717dd700 1 -- start start 2021-11-03T18:20:02.367 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.366+0000 7f13717dd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f136c0f6a10 0x7f136c108f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:20:02.367 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.366+0000 7f13717dd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f136c0f73b0 0x7f136c109450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:20:02.367 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.366+0000 7f13717dd700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136c109a00 con 0x7f136c0f6a10 2021-11-03T18:20:02.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.366+0000 7f13717dd700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136c109b40 con 0x7f136c0f73b0 2021-11-03T18:20:02.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.366+0000 7f136bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f136c0f6a10 0x7f136c108f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:20:02.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.366+0000 7f136bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f136c0f6a10 0x7f136c108f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34276/0 (socket says 172.21.15.27:34276) 2021-11-03T18:20:02.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.366+0000 7f136bfff700 1 -- 172.21.15.27:0/915728277 learned_addr learned my addr 172.21.15.27:0/915728277 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:20:02.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.366+0000 7f136b7fe700 1 --2- 172.21.15.27:0/915728277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f136c0f73b0 0x7f136c109450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:20:02.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.366+0000 7f136bfff700 1 -- 172.21.15.27:0/915728277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f136c0f73b0 msgr2=0x7f136c109450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:20:02.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.366+0000 7f136bfff700 1 --2- 172.21.15.27:0/915728277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f136c0f73b0 0x7f136c109450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:02.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.366+0000 7f136bfff700 1 -- 172.21.15.27:0/915728277 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f135c005040 con 0x7f136c0f6a10 2021-11-03T18:20:02.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.366+0000 7f136b7fe700 1 --2- 172.21.15.27:0/915728277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f136c0f73b0 0x7f136c109450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:20:02.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.366+0000 7f136bfff700 1 --2- 172.21.15.27:0/915728277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f136c0f6a10 0x7f136c108f60 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 rx=0x7f135c009990 tx=0x7f135c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:20:02.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.366+0000 7f1368ff9700 1 -- 172.21.15.27:0/915728277 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f135c018070 con 0x7f136c0f6a10 2021-11-03T18:20:02.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.367+0000 7f1368ff9700 1 -- 172.21.15.27:0/915728277 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f135c007ba0 con 0x7f136c0f6a10 2021-11-03T18:20:02.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.367+0000 7f1368ff9700 1 -- 172.21.15.27:0/915728277 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f135c01c3c0 con 0x7f136c0f6a10 2021-11-03T18:20:02.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.367+0000 7f13717dd700 1 -- 172.21.15.27:0/915728277 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f136c10a480 con 0x7f136c0f6a10 2021-11-03T18:20:02.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.367+0000 7f13717dd700 1 -- 172.21.15.27:0/915728277 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f136c10a9e0 con 0x7f136c0f6a10 2021-11-03T18:20:02.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.368+0000 7f13717dd700 1 -- 172.21.15.27:0/915728277 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f136c045ad0 con 0x7f136c0f6a10 2021-11-03T18:20:02.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.368+0000 7f1368ff9700 1 -- 172.21.15.27:0/915728277 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f135c01d420 con 0x7f136c0f6a10 2021-11-03T18:20:02.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.369+0000 7f1368ff9700 1 --2- 172.21.15.27:0/915728277 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1354064100 0x7f13540665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:20:02.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.369+0000 7f1368ff9700 1 -- 172.21.15.27:0/915728277 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f135c082630 con 0x7f136c0f6a10 2021-11-03T18:20:02.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.369+0000 7f136b7fe700 1 --2- 172.21.15.27:0/915728277 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1354064100 0x7f13540665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:20:02.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.369+0000 7f136b7fe700 1 --2- 172.21.15.27:0/915728277 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1354064100 0x7f13540665c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7f1360009740 tx=0x7f1360009be0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:20:02.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.373+0000 7f1368ff9700 1 -- 172.21.15.27:0/915728277 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f135c054990 con 0x7f136c0f6a10 2021-11-03T18:20:02.605 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.607+0000 7f13717dd700 1 -- 172.21.15.27:0/915728277 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f136c027a00 con 0x7f136c0f6a10 2021-11-03T18:20:02.606 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.608+0000 7f1368ff9700 1 -- 172.21.15.27:0/915728277 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f135c054580 con 0x7f136c0f6a10 2021-11-03T18:20:02.606 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:20:02.607 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:20:02.607 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:20:02.607 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:20:02.607 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:20:02.608 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:20:02.608 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:20:02.608 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:20:02.608 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:20:02.608 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:20:02.609 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:20:02.609 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:20:02.609 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:20:02.609 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:20:02.610 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:20:02.610 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:20:02.610 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:20:02.610 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:20:02.611 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.610+0000 7f13717dd700 1 -- 172.21.15.27:0/915728277 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1354064100 msgr2=0x7f13540665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:20:02.611 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.610+0000 7f13717dd700 1 --2- 172.21.15.27:0/915728277 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1354064100 0x7f13540665c0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 rx=0x7f1360009740 tx=0x7f1360009be0).stop 2021-11-03T18:20:02.612 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.610+0000 7f13717dd700 1 -- 172.21.15.27:0/915728277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f136c0f6a10 msgr2=0x7f136c108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:20:02.612 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.610+0000 7f13717dd700 1 --2- 172.21.15.27:0/915728277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f136c0f6a10 0x7f136c108f60 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 rx=0x7f135c009990 tx=0x7f135c009b30).stop 2021-11-03T18:20:02.612 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.610+0000 7f13717dd700 1 -- 172.21.15.27:0/915728277 shutdown_connections 2021-11-03T18:20:02.612 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.610+0000 7f13717dd700 1 --2- 172.21.15.27:0/915728277 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1354064100 0x7f13540665c0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:02.613 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.610+0000 7f13717dd700 1 --2- 172.21.15.27:0/915728277 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f136c0f6a10 0x7f136c108f60 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:02.613 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.610+0000 7f13717dd700 1 --2- 172.21.15.27:0/915728277 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f136c0f73b0 0x7f136c109450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:02.613 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.610+0000 7f13717dd700 1 -- 172.21.15.27:0/915728277 >> 172.21.15.27:0/915728277 conn(0x7f136c0f0320 msgr2=0x7f136c0fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:20:02.613 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.610+0000 7f13717dd700 1 -- 172.21.15.27:0/915728277 shutdown_connections 2021-11-03T18:20:02.613 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.610+0000 7f13717dd700 1 -- 172.21.15.27:0/915728277 wait complete. 2021-11-03T18:20:02.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.710+0000 7fbccfb22700 1 -- 172.21.15.27:0/3710204463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcc80f7980 msgr2=0x7fbcc80f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:20:02.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.710+0000 7fbccfb22700 1 --2- 172.21.15.27:0/3710204463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcc80f7980 0x7fbcc80f7de0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 rx=0x7fbcb8004660 tx=0x7fbcb8009b30).stop 2021-11-03T18:20:02.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.710+0000 7fbccfb22700 1 -- 172.21.15.27:0/3710204463 shutdown_connections 2021-11-03T18:20:02.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.710+0000 7fbccfb22700 1 --2- 172.21.15.27:0/3710204463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcc80f7980 0x7fbcc80f7de0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:02.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.710+0000 7fbccfb22700 1 --2- 172.21.15.27:0/3710204463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcc80f66e0 0x7fbcc80f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:02.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.710+0000 7fbccfb22700 1 -- 172.21.15.27:0/3710204463 >> 172.21.15.27:0/3710204463 conn(0x7fbcc80f1e40 msgr2=0x7fbcc80f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:20:02.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.710+0000 7fbccfb22700 1 -- 172.21.15.27:0/3710204463 shutdown_connections 2021-11-03T18:20:02.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.710+0000 7fbccfb22700 1 -- 172.21.15.27:0/3710204463 wait complete. 2021-11-03T18:20:02.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.710+0000 7fbccfb22700 1 Processor -- start 2021-11-03T18:20:02.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.710+0000 7fbccfb22700 1 -- start start 2021-11-03T18:20:02.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.711+0000 7fbccfb22700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcc80f66e0 0x7fbcc8106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:20:02.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.711+0000 7fbccfb22700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcc80f7980 0x7fbcc8107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:20:02.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.711+0000 7fbccfb22700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc8107a00 con 0x7fbcc80f7980 2021-11-03T18:20:02.715 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.711+0000 7fbccfb22700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc8107b40 con 0x7fbcc80f66e0 2021-11-03T18:20:02.715 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.711+0000 7fbccd0bd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcc80f7980 0x7fbcc8107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:20:02.715 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.711+0000 7fbccd0bd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcc80f7980 0x7fbcc8107450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34288/0 (socket says 172.21.15.27:34288) 2021-11-03T18:20:02.715 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.711+0000 7fbccd0bd700 1 -- 172.21.15.27:0/2878670486 learned_addr learned my addr 172.21.15.27:0/2878670486 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:20:02.716 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.711+0000 7fbccd8be700 1 --2- 172.21.15.27:0/2878670486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcc80f66e0 0x7fbcc8106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:20:02.716 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.711+0000 7fbccd0bd700 1 -- 172.21.15.27:0/2878670486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcc80f66e0 msgr2=0x7fbcc8106f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:20:02.716 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.711+0000 7fbccd0bd700 1 --2- 172.21.15.27:0/2878670486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcc80f66e0 0x7fbcc8106f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:02.716 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.711+0000 7fbccd0bd700 1 -- 172.21.15.27:0/2878670486 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb8005040 con 0x7fbcc80f7980 2021-11-03T18:20:02.717 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.711+0000 7fbccd0bd700 1 --2- 172.21.15.27:0/2878670486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcc80f7980 0x7fbcc8107450 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 rx=0x7fbcb8009990 tx=0x7fbcb8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:20:02.717 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.711+0000 7fbcbe7fc700 1 -- 172.21.15.27:0/2878670486 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcb8018070 con 0x7fbcc80f7980 2021-11-03T18:20:02.717 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.711+0000 7fbcbe7fc700 1 -- 172.21.15.27:0/2878670486 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fbcb8007ba0 con 0x7fbcc80f7980 2021-11-03T18:20:02.718 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.711+0000 7fbcbe7fc700 1 -- 172.21.15.27:0/2878670486 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcb801c620 con 0x7fbcc80f7980 2021-11-03T18:20:02.718 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.711+0000 7fbccfb22700 1 -- 172.21.15.27:0/2878670486 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcc8108480 con 0x7fbcc80f7980 2021-11-03T18:20:02.718 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.711+0000 7fbccfb22700 1 -- 172.21.15.27:0/2878670486 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcc81089e0 con 0x7fbcc80f7980 2021-11-03T18:20:02.718 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.712+0000 7fbccfb22700 1 -- 172.21.15.27:0/2878670486 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcc8045ad0 con 0x7fbcc80f7980 2021-11-03T18:20:02.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.713+0000 7fbcbe7fc700 1 -- 172.21.15.27:0/2878670486 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fbcb801c890 con 0x7fbcc80f7980 2021-11-03T18:20:02.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.713+0000 7fbcbe7fc700 1 --2- 172.21.15.27:0/2878670486 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbcb40640b0 0x7fbcb4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:20:02.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.713+0000 7fbcbe7fc700 1 -- 172.21.15.27:0/2878670486 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fbcb80825a0 con 0x7fbcc80f7980 2021-11-03T18:20:02.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.713+0000 7fbccd8be700 1 --2- 172.21.15.27:0/2878670486 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbcb40640b0 0x7fbcb4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:20:02.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.714+0000 7fbccd8be700 1 --2- 172.21.15.27:0/2878670486 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbcb40640b0 0x7fbcb4066570 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7fbcc80f77e0 tx=0x7fbcc0009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:20:02.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.717+0000 7fbcbe7fc700 1 -- 172.21.15.27:0/2878670486 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fbcb8054960 con 0x7fbcc80f7980 2021-11-03T18:20:02.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.922+0000 7fbccfb22700 1 -- 172.21.15.27:0/2878670486 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbcc80fefa0 con 0x7fbcc80f7980 2021-11-03T18:20:02.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.923+0000 7fbcbe7fc700 1 -- 172.21.15.27:0/2878670486 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7fbcb8054550 con 0x7fbcc80f7980 2021-11-03T18:20:02.921 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:20:02.921 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:20:02.922 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:20:02.922 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:20:02.922 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:20:02.923 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:20:02.923 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:20:02.923 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:20:02.923 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:20:02.924 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:20:02.924 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:20:02.924 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:20:02.924 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:20:02.924 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:20:02.925 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:20:02.925 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:20:02.925 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:20:02.925 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:20:02.926 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:20:02.926 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:20:02.926 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:20:02.926 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:20:02.926 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:20:02.927 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:20:02.927 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:20:02.927 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:20:02.927 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:20:02.928 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:20:02.928 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:20:02.928 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:20:02.928 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:20:02.928 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:20:02.929 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:20:02.929 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:20:02.929 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:20:02.930 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:20:02.930 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:20:02.930 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:20:02.930 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:20:02.931 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.924+0000 7fbccfb22700 1 -- 172.21.15.27:0/2878670486 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbcb40640b0 msgr2=0x7fbcb4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:20:02.931 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.924+0000 7fbccfb22700 1 --2- 172.21.15.27:0/2878670486 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbcb40640b0 0x7fbcb4066570 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7fbcc80f77e0 tx=0x7fbcc0009040).stop 2021-11-03T18:20:02.932 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.924+0000 7fbccfb22700 1 -- 172.21.15.27:0/2878670486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcc80f7980 msgr2=0x7fbcc8107450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:20:02.932 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.924+0000 7fbccfb22700 1 --2- 172.21.15.27:0/2878670486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcc80f7980 0x7fbcc8107450 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 rx=0x7fbcb8009990 tx=0x7fbcb8009b30).stop 2021-11-03T18:20:02.932 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.925+0000 7fbccfb22700 1 -- 172.21.15.27:0/2878670486 shutdown_connections 2021-11-03T18:20:02.932 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.925+0000 7fbccfb22700 1 --2- 172.21.15.27:0/2878670486 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbcb40640b0 0x7fbcb4066570 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:02.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.925+0000 7fbccfb22700 1 --2- 172.21.15.27:0/2878670486 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbcc80f66e0 0x7fbcc8106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:02.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.925+0000 7fbccfb22700 1 --2- 172.21.15.27:0/2878670486 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbcc80f7980 0x7fbcc8107450 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:02.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.925+0000 7fbccfb22700 1 -- 172.21.15.27:0/2878670486 >> 172.21.15.27:0/2878670486 conn(0x7fbcc80f1e40 msgr2=0x7fbcc80fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:20:02.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.925+0000 7fbccfb22700 1 -- 172.21.15.27:0/2878670486 shutdown_connections 2021-11-03T18:20:02.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:02.925+0000 7fbccfb22700 1 -- 172.21.15.27:0/2878670486 wait complete. 2021-11-03T18:20:02.934 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:20:02.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:02 smithi027 ceph-mon[97244]: pgmap v1961: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:02.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:02 smithi115 ceph-mon[65453]: pgmap v1961: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:03.958 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:03 smithi115 ceph-mon[65453]: from='client.36120 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:20:03.959 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:03 smithi115 ceph-mon[65453]: from='client.36124 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:20:03.959 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:03 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/915728277' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:03.959 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:03 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2878670486' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:20:04.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:03 smithi027 ceph-mon[97244]: from='client.36120 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:20:04.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:03 smithi027 ceph-mon[97244]: from='client.36124 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:20:04.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:03 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/915728277' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:04.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:03 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2878670486' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:20:04.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:04 smithi115 ceph-mon[65453]: pgmap v1962: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:05.099 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:04 smithi027 ceph-mon[97244]: pgmap v1962: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:05.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3847.451946 secs 2021-11-03T18:20:06.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3847.451946 secs 2021-11-03T18:20:06.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:06 smithi115 ceph-mon[65453]: pgmap v1963: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:07.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:06 smithi027 ceph-mon[97244]: pgmap v1963: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:07.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:20:08.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:20:08.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:08 smithi115 ceph-mon[65453]: pgmap v1964: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:08.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:08 smithi115 ceph-mon[65453]: poll took 40123.110 ms: 1 events 2021-11-03T18:20:09.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:08 smithi027 ceph-mon[97244]: pgmap v1964: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:09.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:08 smithi027 ceph-mon[97244]: poll took 40123.110 ms: 1 events 2021-11-03T18:20:10.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:10 smithi115 ceph-mon[65453]: pgmap v1965: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:10.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3852.452087 secs 2021-11-03T18:20:10.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:20:10.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:20:10.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:20:10.741 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:10.741 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:10.741 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:10.742 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:11.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:10 smithi027 ceph-mon[97244]: pgmap v1965: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:11.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3852.452087 secs 2021-11-03T18:20:11.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:20:11.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:20:11.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:20:11.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:11.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:11.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:11.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:11.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:11 smithi115 ceph-mon[65453]: poll took 2837.235 ms: 1 events 2021-11-03T18:20:11.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:11 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:20:12.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:11 smithi027 ceph-mon[97244]: poll took 2837.235 ms: 1 events 2021-11-03T18:20:12.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:11 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:20:12.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:12 smithi027 ceph-mon[97244]: pgmap v1966: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:12.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:12 smithi115 ceph-mon[65453]: pgmap v1966: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:14.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:14 smithi115 ceph-mon[65453]: pgmap v1967: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:15.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:14 smithi027 ceph-mon[97244]: pgmap v1967: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:15.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3857.452211 secs 2021-11-03T18:20:16.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3857.452211 secs 2021-11-03T18:20:16.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:16 smithi115 ceph-mon[65453]: pgmap v1968: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:17.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:16 smithi027 ceph-mon[97244]: pgmap v1968: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:18.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:18 smithi115 ceph-mon[65453]: pgmap v1969: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:19.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:18 smithi027 ceph-mon[97244]: pgmap v1969: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:20.859 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:20 smithi115 ceph-mon[65453]: pgmap v1970: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:20.859 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3862.452339 secs 2021-11-03T18:20:20.859 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:20:20.876 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:20 smithi027 ceph-mon[97244]: pgmap v1970: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:20.876 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3862.452339 secs 2021-11-03T18:20:20.877 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:20:21.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:21 smithi115 ceph-mon[65453]: poll took 10038.765 ms: 1 events 2021-11-03T18:20:21.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:20:21.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:20:21.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:21.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:21.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:21.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:21 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:22.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:21 smithi027 ceph-mon[97244]: poll took 10038.765 ms: 1 events 2021-11-03T18:20:22.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:20:22.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:20:22.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:22.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:22.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:22.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:22.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:22 smithi027 ceph-mon[97244]: pgmap v1971: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:22.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:22 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:20:22.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:22 smithi115 ceph-mon[65453]: pgmap v1971: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:22.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:22 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:20:24.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:24 smithi115 ceph-mon[65453]: pgmap v1972: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:25.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:24 smithi027 ceph-mon[97244]: pgmap v1972: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:25.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3867.452474 secs 2021-11-03T18:20:26.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3867.452474 secs 2021-11-03T18:20:26.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:26 smithi115 ceph-mon[65453]: pgmap v1973: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:27.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:26 smithi027 ceph-mon[97244]: pgmap v1973: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:28.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:28 smithi115 ceph-mon[65453]: pgmap v1974: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:29.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:28 smithi027 ceph-mon[97244]: pgmap v1974: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:30.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:30 smithi115 ceph-mon[65453]: pgmap v1975: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:30.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3872.452541 secs 2021-11-03T18:20:31.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:30 smithi027 ceph-mon[97244]: pgmap v1975: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:31.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3872.452541 secs 2021-11-03T18:20:31.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:20:31.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:20:31.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:20:31.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:31.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:31.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:31.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:31 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:32.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:20:32.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:20:32.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:20:32.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:32.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:32.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:32.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:31 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:32.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:32 smithi027 ceph-mon[97244]: pgmap v1976: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:32.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:32 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:20:32.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:32 smithi115 ceph-mon[65453]: pgmap v1976: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:32.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:32 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:20:33.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.041+0000 7ff9a99e1700 1 -- 172.21.15.27:0/2493041077 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9a40f9720 msgr2=0x7ff9a40fbb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:20:33.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.041+0000 7ff9a99e1700 1 --2- 172.21.15.27:0/2493041077 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9a40f9720 0x7ff9a40fbb10 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 rx=0x7ff98c004660 tx=0x7ff98c009b30).stop 2021-11-03T18:20:33.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.041+0000 7ff9a99e1700 1 -- 172.21.15.27:0/2493041077 shutdown_connections 2021-11-03T18:20:33.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.041+0000 7ff9a99e1700 1 --2- 172.21.15.27:0/2493041077 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9a40f9720 0x7ff9a40fbb10 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:33.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.041+0000 7ff9a99e1700 1 --2- 172.21.15.27:0/2493041077 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9a40602b0 0x7ff9a40f91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:33.046 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.041+0000 7ff9a99e1700 1 -- 172.21.15.27:0/2493041077 >> 172.21.15.27:0/2493041077 conn(0x7ff9a40f0bc0 msgr2=0x7ff9a40f2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:20:33.046 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.041+0000 7ff9a99e1700 1 -- 172.21.15.27:0/2493041077 shutdown_connections 2021-11-03T18:20:33.046 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.041+0000 7ff9a99e1700 1 -- 172.21.15.27:0/2493041077 wait complete. 2021-11-03T18:20:33.047 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.042+0000 7ff9a99e1700 1 Processor -- start 2021-11-03T18:20:33.047 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.042+0000 7ff9a99e1700 1 -- start start 2021-11-03T18:20:33.048 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.042+0000 7ff9a99e1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9a40602b0 0x7ff9a4106dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:20:33.048 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.042+0000 7ff9a99e1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9a40f9720 0x7ff9a41072c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:20:33.049 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.042+0000 7ff9a99e1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a41078c0 con 0x7ff9a40f9720 2021-11-03T18:20:33.049 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.042+0000 7ff9a99e1700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a4108100 con 0x7ff9a40602b0 2021-11-03T18:20:33.049 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.042+0000 7ff9a27fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9a40f9720 0x7ff9a41072c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:20:33.050 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.042+0000 7ff9a27fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9a40f9720 0x7ff9a41072c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34306/0 (socket says 172.21.15.27:34306) 2021-11-03T18:20:33.050 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.042+0000 7ff9a27fc700 1 -- 172.21.15.27:0/1976499062 learned_addr learned my addr 172.21.15.27:0/1976499062 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:20:33.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.042+0000 7ff9a2ffd700 1 --2- 172.21.15.27:0/1976499062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9a40602b0 0x7ff9a4106dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:20:33.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.042+0000 7ff9a27fc700 1 -- 172.21.15.27:0/1976499062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9a40602b0 msgr2=0x7ff9a4106dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:20:33.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.042+0000 7ff9a27fc700 1 --2- 172.21.15.27:0/1976499062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9a40602b0 0x7ff9a4106dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:33.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.042+0000 7ff9a27fc700 1 -- 172.21.15.27:0/1976499062 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff98c005040 con 0x7ff9a40f9720 2021-11-03T18:20:33.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.042+0000 7ff9a27fc700 1 --2- 172.21.15.27:0/1976499062 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9a40f9720 0x7ff9a41072c0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 rx=0x7ff98c013a70 tx=0x7ff98c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:20:33.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.043+0000 7ff99b7fe700 1 -- 172.21.15.27:0/1976499062 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff98c018070 con 0x7ff9a40f9720 2021-11-03T18:20:33.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.043+0000 7ff9a99e1700 1 -- 172.21.15.27:0/1976499062 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9a4108380 con 0x7ff9a40f9720 2021-11-03T18:20:33.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.043+0000 7ff99b7fe700 1 -- 172.21.15.27:0/1976499062 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7ff98c0097d0 con 0x7ff9a40f9720 2021-11-03T18:20:33.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.043+0000 7ff99b7fe700 1 -- 172.21.15.27:0/1976499062 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff98c01de80 con 0x7ff9a40f9720 2021-11-03T18:20:33.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.043+0000 7ff9a99e1700 1 -- 172.21.15.27:0/1976499062 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9a4108860 con 0x7ff9a40f9720 2021-11-03T18:20:33.055 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.044+0000 7ff9a99e1700 1 -- 172.21.15.27:0/1976499062 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9a4059100 con 0x7ff9a40f9720 2021-11-03T18:20:33.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.044+0000 7ff99b7fe700 1 -- 172.21.15.27:0/1976499062 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7ff98c007ba0 con 0x7ff9a40f9720 2021-11-03T18:20:33.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.044+0000 7ff99b7fe700 1 --2- 172.21.15.27:0/1976499062 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff990064100 0x7ff9900665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:20:33.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.045+0000 7ff99b7fe700 1 -- 172.21.15.27:0/1976499062 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7ff98c082430 con 0x7ff9a40f9720 2021-11-03T18:20:33.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.045+0000 7ff9a2ffd700 1 --2- 172.21.15.27:0/1976499062 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff990064100 0x7ff9900665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:20:33.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.045+0000 7ff9a2ffd700 1 --2- 172.21.15.27:0/1976499062 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff990064100 0x7ff9900665c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7ff9a4107b90 tx=0x7ff99c00b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:20:33.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.049+0000 7ff99b7fe700 1 -- 172.21.15.27:0/1976499062 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7ff98c0547f0 con 0x7ff9a40f9720 2021-11-03T18:20:33.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.226+0000 7ff9a99e1700 1 -- 172.21.15.27:0/1976499062 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9a40fed90 con 0x7ff990064100 2021-11-03T18:20:33.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.227+0000 7ff99b7fe700 1 -- 172.21.15.27:0/1976499062 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7ff9a40fed90 con 0x7ff990064100 2021-11-03T18:20:33.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.229+0000 7ff9a99e1700 1 -- 172.21.15.27:0/1976499062 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff990064100 msgr2=0x7ff9900665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:20:33.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.229+0000 7ff9a99e1700 1 --2- 172.21.15.27:0/1976499062 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff990064100 0x7ff9900665c0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7ff9a4107b90 tx=0x7ff99c00b040).stop 2021-11-03T18:20:33.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.229+0000 7ff9a99e1700 1 -- 172.21.15.27:0/1976499062 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9a40f9720 msgr2=0x7ff9a41072c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:20:33.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.229+0000 7ff9a99e1700 1 --2- 172.21.15.27:0/1976499062 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9a40f9720 0x7ff9a41072c0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 rx=0x7ff98c013a70 tx=0x7ff98c009b30).stop 2021-11-03T18:20:33.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.229+0000 7ff9a99e1700 1 -- 172.21.15.27:0/1976499062 shutdown_connections 2021-11-03T18:20:33.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.229+0000 7ff9a99e1700 1 --2- 172.21.15.27:0/1976499062 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff990064100 0x7ff9900665c0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:33.230 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.229+0000 7ff9a99e1700 1 --2- 172.21.15.27:0/1976499062 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9a40602b0 0x7ff9a4106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:33.231 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.229+0000 7ff9a99e1700 1 --2- 172.21.15.27:0/1976499062 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9a40f9720 0x7ff9a41072c0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:33.231 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.229+0000 7ff9a99e1700 1 -- 172.21.15.27:0/1976499062 >> 172.21.15.27:0/1976499062 conn(0x7ff9a40f0bc0 msgr2=0x7ff9a40f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:20:33.231 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.229+0000 7ff9a99e1700 1 -- 172.21.15.27:0/1976499062 shutdown_connections 2021-11-03T18:20:33.232 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.229+0000 7ff9a99e1700 1 -- 172.21.15.27:0/1976499062 wait complete. 2021-11-03T18:20:33.237 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:20:33.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.330+0000 7fe6eda55700 1 -- 172.21.15.27:0/1359689708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6e80f6860 msgr2=0x7fe6e80f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:20:33.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.330+0000 7fe6eda55700 1 --2- 172.21.15.27:0/1359689708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6e80f6860 0x7fe6e80f6cc0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 rx=0x7fe6dc004660 tx=0x7fe6dc009b30).stop 2021-11-03T18:20:33.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.330+0000 7fe6eda55700 1 -- 172.21.15.27:0/1359689708 shutdown_connections 2021-11-03T18:20:33.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.330+0000 7fe6eda55700 1 --2- 172.21.15.27:0/1359689708 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6e80f6860 0x7fe6e80f6cc0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:33.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.330+0000 7fe6eda55700 1 --2- 172.21.15.27:0/1359689708 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6e80f55c0 0x7fe6e80f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:33.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.330+0000 7fe6eda55700 1 -- 172.21.15.27:0/1359689708 >> 172.21.15.27:0/1359689708 conn(0x7fe6e80f0d40 msgr2=0x7fe6e80f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:20:33.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.330+0000 7fe6eda55700 1 -- 172.21.15.27:0/1359689708 shutdown_connections 2021-11-03T18:20:33.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.330+0000 7fe6eda55700 1 -- 172.21.15.27:0/1359689708 wait complete. 2021-11-03T18:20:33.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.331+0000 7fe6eda55700 1 Processor -- start 2021-11-03T18:20:33.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.331+0000 7fe6eda55700 1 -- start start 2021-11-03T18:20:33.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.331+0000 7fe6eda55700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6e80f55c0 0x7fe6e8109be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:20:33.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.331+0000 7fe6eda55700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6e80f6860 0x7fe6e810c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:20:33.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.331+0000 7fe6eda55700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e80603c0 con 0x7fe6e80f6860 2021-11-03T18:20:33.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.331+0000 7fe6eda55700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6e8060500 con 0x7fe6e80f55c0 2021-11-03T18:20:33.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.331+0000 7fe6e67fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6e80f6860 0x7fe6e810c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:20:33.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.331+0000 7fe6e67fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6e80f6860 0x7fe6e810c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34318/0 (socket says 172.21.15.27:34318) 2021-11-03T18:20:33.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.331+0000 7fe6e67fc700 1 -- 172.21.15.27:0/3735651706 learned_addr learned my addr 172.21.15.27:0/3735651706 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:20:33.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.331+0000 7fe6e6ffd700 1 --2- 172.21.15.27:0/3735651706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6e80f55c0 0x7fe6e8109be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:20:33.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.331+0000 7fe6e67fc700 1 -- 172.21.15.27:0/3735651706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6e80f55c0 msgr2=0x7fe6e8109be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:20:33.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.331+0000 7fe6e67fc700 1 --2- 172.21.15.27:0/3735651706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6e80f55c0 0x7fe6e8109be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:33.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.331+0000 7fe6e67fc700 1 -- 172.21.15.27:0/3735651706 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6dc005040 con 0x7fe6e80f6860 2021-11-03T18:20:33.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.331+0000 7fe6e6ffd700 1 --2- 172.21.15.27:0/3735651706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6e80f55c0 0x7fe6e8109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:20:33.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.331+0000 7fe6e67fc700 1 --2- 172.21.15.27:0/3735651706 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6e80f6860 0x7fe6e810c0e0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 rx=0x7fe6dc002040 tx=0x7fe6dc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:20:33.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.332+0000 7fe6cf7fe700 1 -- 172.21.15.27:0/3735651706 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6dc019070 con 0x7fe6e80f6860 2021-11-03T18:20:33.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.332+0000 7fe6cf7fe700 1 -- 172.21.15.27:0/3735651706 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fe6dc007e70 con 0x7fe6e80f6860 2021-11-03T18:20:33.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.332+0000 7fe6cf7fe700 1 -- 172.21.15.27:0/3735651706 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe6dc014e30 con 0x7fe6e80f6860 2021-11-03T18:20:33.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.332+0000 7fe6eda55700 1 -- 172.21.15.27:0/3735651706 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6e810c630 con 0x7fe6e80f6860 2021-11-03T18:20:33.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.332+0000 7fe6eda55700 1 -- 172.21.15.27:0/3735651706 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6e810cb60 con 0x7fe6e80f6860 2021-11-03T18:20:33.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.333+0000 7fe6eda55700 1 -- 172.21.15.27:0/3735651706 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6e80f49b0 con 0x7fe6e80f6860 2021-11-03T18:20:33.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.333+0000 7fe6cf7fe700 1 -- 172.21.15.27:0/3735651706 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fe6dc025070 con 0x7fe6e80f6860 2021-11-03T18:20:33.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.334+0000 7fe6cf7fe700 1 --2- 172.21.15.27:0/3735651706 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe6d0064030 0x7fe6d00664f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:20:33.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.334+0000 7fe6e6ffd700 1 --2- 172.21.15.27:0/3735651706 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe6d0064030 0x7fe6d00664f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:20:33.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.334+0000 7fe6cf7fe700 1 -- 172.21.15.27:0/3735651706 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fe6dc081840 con 0x7fe6e80f6860 2021-11-03T18:20:33.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.334+0000 7fe6e6ffd700 1 --2- 172.21.15.27:0/3735651706 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe6d0064030 0x7fe6d00664f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7fe6e8060040 tx=0x7fe6e000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:20:33.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.338+0000 7fe6cf7fe700 1 -- 172.21.15.27:0/3735651706 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fe6dc053ae0 con 0x7fe6e80f6860 2021-11-03T18:20:33.511 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.513+0000 7fe6eda55700 1 -- 172.21.15.27:0/3735651706 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe6e80fdf00 con 0x7fe6d0064030 2021-11-03T18:20:33.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.528+0000 7fe6cf7fe700 1 -- 172.21.15.27:0/3735651706 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fe6e80fdf00 con 0x7fe6d0064030 2021-11-03T18:20:33.526 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:20:33.527 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (72m) 66s ago 73m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:20:33.527 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (63m) 66s ago 73m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:20:33.527 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (63m) 23s ago 72m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:20:33.527 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (71m) 66s ago 73m 42.4M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:20:33.527 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (65m) 66s ago 68m 26.5M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:20:33.528 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (68m) 66s ago 68m 3849M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:20:33.528 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (64m) 23s ago 68m 26.1M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:20:33.528 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (65m) 23s ago 68m 27.2M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:20:33.528 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (65m) 66s ago 74m 590M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:20:33.529 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (65m) 23s ago 72m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:20:33.529 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (64m) 66s ago 74m 312M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:20:33.529 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (64m) 23s ago 72m 145M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:20:33.529 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (72m) 66s ago 72m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:20:33.530 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (72m) 23s ago 72m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:20:33.530 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (63m) 66s ago 71m 392M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:20:33.530 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (62m) 66s ago 71m 427M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:20:33.530 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (62m) 66s ago 71m 320M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:20:33.530 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (61m) 23s ago 70m 454M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:20:33.531 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (61m) 23s ago 70m 359M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:20:33.531 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (61m) 23s ago 70m 368M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:20:33.531 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (71m) 66s ago 72m 59.9M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:20:33.532 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.529+0000 7fe6eda55700 1 -- 172.21.15.27:0/3735651706 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe6d0064030 msgr2=0x7fe6d00664f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:20:33.532 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.529+0000 7fe6eda55700 1 --2- 172.21.15.27:0/3735651706 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe6d0064030 0x7fe6d00664f0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7fe6e8060040 tx=0x7fe6e000b040).stop 2021-11-03T18:20:33.533 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.529+0000 7fe6eda55700 1 -- 172.21.15.27:0/3735651706 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6e80f6860 msgr2=0x7fe6e810c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:20:33.533 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.529+0000 7fe6eda55700 1 --2- 172.21.15.27:0/3735651706 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6e80f6860 0x7fe6e810c0e0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 rx=0x7fe6dc002040 tx=0x7fe6dc00b040).stop 2021-11-03T18:20:33.533 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.529+0000 7fe6eda55700 1 -- 172.21.15.27:0/3735651706 shutdown_connections 2021-11-03T18:20:33.534 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.529+0000 7fe6eda55700 1 --2- 172.21.15.27:0/3735651706 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe6d0064030 0x7fe6d00664f0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:33.534 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.529+0000 7fe6eda55700 1 --2- 172.21.15.27:0/3735651706 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe6e80f55c0 0x7fe6e8109be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:33.534 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.529+0000 7fe6eda55700 1 --2- 172.21.15.27:0/3735651706 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe6e80f6860 0x7fe6e810c0e0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:33.535 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.529+0000 7fe6eda55700 1 -- 172.21.15.27:0/3735651706 >> 172.21.15.27:0/3735651706 conn(0x7fe6e80f0d40 msgr2=0x7fe6e80fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:20:33.535 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.530+0000 7fe6eda55700 1 -- 172.21.15.27:0/3735651706 shutdown_connections 2021-11-03T18:20:33.536 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.530+0000 7fe6eda55700 1 -- 172.21.15.27:0/3735651706 wait complete. 2021-11-03T18:20:33.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.629+0000 7fe9bea63700 1 -- 172.21.15.27:0/77584884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9b806b920 msgr2=0x7fe9b8069ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:20:33.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.629+0000 7fe9bea63700 1 --2- 172.21.15.27:0/77584884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9b806b920 0x7fe9b8069ec0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 rx=0x7fe9a0004690 tx=0x7fe9a0009b30).stop 2021-11-03T18:20:33.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.629+0000 7fe9bea63700 1 -- 172.21.15.27:0/77584884 shutdown_connections 2021-11-03T18:20:33.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.629+0000 7fe9bea63700 1 --2- 172.21.15.27:0/77584884 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9b806a3b0 0x7fe9b806a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:33.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.629+0000 7fe9bea63700 1 --2- 172.21.15.27:0/77584884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9b806b920 0x7fe9b8069ec0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:33.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.629+0000 7fe9bea63700 1 -- 172.21.15.27:0/77584884 >> 172.21.15.27:0/77584884 conn(0x7fe9b80f1f00 msgr2=0x7fe9b80f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:20:33.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.629+0000 7fe9bea63700 1 -- 172.21.15.27:0/77584884 shutdown_connections 2021-11-03T18:20:33.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.629+0000 7fe9bea63700 1 -- 172.21.15.27:0/77584884 wait complete. 2021-11-03T18:20:33.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.630+0000 7fe9bea63700 1 Processor -- start 2021-11-03T18:20:33.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.630+0000 7fe9bea63700 1 -- start start 2021-11-03T18:20:33.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.630+0000 7fe9bea63700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9b806a3b0 0x7fe9b8109170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:20:33.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.630+0000 7fe9bea63700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9b806b920 0x7fe9b8109660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:20:33.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.630+0000 7fe9bea63700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b8109c10 con 0x7fe9b806a3b0 2021-11-03T18:20:33.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.630+0000 7fe9bea63700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9b8109d50 con 0x7fe9b806b920 2021-11-03T18:20:33.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.630+0000 7fe9b7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9b806a3b0 0x7fe9b8109170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:20:33.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.630+0000 7fe9b7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9b806a3b0 0x7fe9b8109170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34326/0 (socket says 172.21.15.27:34326) 2021-11-03T18:20:33.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.630+0000 7fe9b7fff700 1 -- 172.21.15.27:0/3077775289 learned_addr learned my addr 172.21.15.27:0/3077775289 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:20:33.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.630+0000 7fe9b77fe700 1 --2- 172.21.15.27:0/3077775289 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9b806b920 0x7fe9b8109660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:20:33.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.630+0000 7fe9b7fff700 1 -- 172.21.15.27:0/3077775289 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9b806b920 msgr2=0x7fe9b8109660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:20:33.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.630+0000 7fe9b7fff700 1 --2- 172.21.15.27:0/3077775289 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9b806b920 0x7fe9b8109660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:33.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.630+0000 7fe9b7fff700 1 -- 172.21.15.27:0/3077775289 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9a0005040 con 0x7fe9b806a3b0 2021-11-03T18:20:33.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.631+0000 7fe9b7fff700 1 --2- 172.21.15.27:0/3077775289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9b806a3b0 0x7fe9b8109170 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 rx=0x7fe9a0004e20 tx=0x7fe9a0009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:20:33.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.631+0000 7fe9b4ff9700 1 -- 172.21.15.27:0/3077775289 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9a0018070 con 0x7fe9b806a3b0 2021-11-03T18:20:33.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.631+0000 7fe9b4ff9700 1 -- 172.21.15.27:0/3077775289 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fe9a0013e40 con 0x7fe9b806a3b0 2021-11-03T18:20:33.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.631+0000 7fe9bea63700 1 -- 172.21.15.27:0/3077775289 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9b810a690 con 0x7fe9b806a3b0 2021-11-03T18:20:33.637 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.631+0000 7fe9b4ff9700 1 -- 172.21.15.27:0/3077775289 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe9a001c440 con 0x7fe9b806a3b0 2021-11-03T18:20:33.637 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.631+0000 7fe9bea63700 1 -- 172.21.15.27:0/3077775289 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9b810abf0 con 0x7fe9b806a3b0 2021-11-03T18:20:33.638 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.632+0000 7fe9bea63700 1 -- 172.21.15.27:0/3077775289 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9b8045ad0 con 0x7fe9b806a3b0 2021-11-03T18:20:33.638 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.632+0000 7fe9b4ff9700 1 -- 172.21.15.27:0/3077775289 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fe9a001d420 con 0x7fe9b806a3b0 2021-11-03T18:20:33.639 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.632+0000 7fe9b4ff9700 1 --2- 172.21.15.27:0/3077775289 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe9a40640b0 0x7fe9a4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:20:33.640 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.633+0000 7fe9b4ff9700 1 -- 172.21.15.27:0/3077775289 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fe9a0082770 con 0x7fe9b806a3b0 2021-11-03T18:20:33.640 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.633+0000 7fe9b77fe700 1 --2- 172.21.15.27:0/3077775289 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe9a40640b0 0x7fe9a4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:20:33.641 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.633+0000 7fe9b77fe700 1 --2- 172.21.15.27:0/3077775289 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe9a40640b0 0x7fe9a4066570 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7fe9a80096c0 tx=0x7fe9a8009c20).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:20:33.641 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.636+0000 7fe9b4ff9700 1 -- 172.21.15.27:0/3077775289 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fe9a0054a70 con 0x7fe9b806a3b0 2021-11-03T18:20:33.862 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.864+0000 7fe9bea63700 1 -- 172.21.15.27:0/3077775289 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe9b8101190 con 0x7fe9b806a3b0 2021-11-03T18:20:33.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.865+0000 7fe9b4ff9700 1 -- 172.21.15.27:0/3077775289 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fe9a0054660 con 0x7fe9b806a3b0 2021-11-03T18:20:33.863 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:20:33.864 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:20:33.864 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:20:33.864 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:20:33.864 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:20:33.864 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:20:33.865 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:20:33.865 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:20:33.865 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:20:33.865 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:20:33.866 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:20:33.866 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:20:33.866 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:20:33.866 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:20:33.867 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:20:33.867 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:20:33.867 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:20:33.867 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:20:33.868 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.866+0000 7fe9bea63700 1 -- 172.21.15.27:0/3077775289 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe9a40640b0 msgr2=0x7fe9a4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:20:33.868 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.866+0000 7fe9bea63700 1 --2- 172.21.15.27:0/3077775289 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe9a40640b0 0x7fe9a4066570 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7fe9a80096c0 tx=0x7fe9a8009c20).stop 2021-11-03T18:20:33.869 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.866+0000 7fe9bea63700 1 -- 172.21.15.27:0/3077775289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9b806a3b0 msgr2=0x7fe9b8109170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:20:33.869 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.866+0000 7fe9bea63700 1 --2- 172.21.15.27:0/3077775289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9b806a3b0 0x7fe9b8109170 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 rx=0x7fe9a0004e20 tx=0x7fe9a0009b30).stop 2021-11-03T18:20:33.869 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.866+0000 7fe9bea63700 1 -- 172.21.15.27:0/3077775289 shutdown_connections 2021-11-03T18:20:33.869 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.866+0000 7fe9bea63700 1 --2- 172.21.15.27:0/3077775289 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe9a40640b0 0x7fe9a4066570 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:33.869 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.866+0000 7fe9bea63700 1 --2- 172.21.15.27:0/3077775289 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe9b806a3b0 0x7fe9b8109170 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:33.870 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.866+0000 7fe9bea63700 1 --2- 172.21.15.27:0/3077775289 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe9b806b920 0x7fe9b8109660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:33.870 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.866+0000 7fe9bea63700 1 -- 172.21.15.27:0/3077775289 >> 172.21.15.27:0/3077775289 conn(0x7fe9b80f1f00 msgr2=0x7fe9b80fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:20:33.870 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.866+0000 7fe9bea63700 1 -- 172.21.15.27:0/3077775289 shutdown_connections 2021-11-03T18:20:33.870 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.866+0000 7fe9bea63700 1 -- 172.21.15.27:0/3077775289 wait complete. 2021-11-03T18:20:33.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.966+0000 7f8d8d5f0700 1 -- 172.21.15.27:0/2502894819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d880f9720 msgr2=0x7f8d880fbb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:20:33.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.966+0000 7f8d8d5f0700 1 --2- 172.21.15.27:0/2502894819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d880f9720 0x7f8d880fbb10 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 rx=0x7f8d70004660 tx=0x7f8d70009b30).stop 2021-11-03T18:20:33.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.966+0000 7f8d8d5f0700 1 -- 172.21.15.27:0/2502894819 shutdown_connections 2021-11-03T18:20:33.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.966+0000 7f8d8d5f0700 1 --2- 172.21.15.27:0/2502894819 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d880f9720 0x7f8d880fbb10 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:33.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.966+0000 7f8d8d5f0700 1 --2- 172.21.15.27:0/2502894819 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d880602b0 0x7f8d880f91a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:33.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.967+0000 7f8d8d5f0700 1 -- 172.21.15.27:0/2502894819 >> 172.21.15.27:0/2502894819 conn(0x7f8d880f0bc0 msgr2=0x7f8d880f2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:20:33.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.967+0000 7f8d8d5f0700 1 -- 172.21.15.27:0/2502894819 shutdown_connections 2021-11-03T18:20:33.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.967+0000 7f8d8d5f0700 1 -- 172.21.15.27:0/2502894819 wait complete. 2021-11-03T18:20:33.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.967+0000 7f8d8d5f0700 1 Processor -- start 2021-11-03T18:20:33.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.967+0000 7f8d8d5f0700 1 -- start start 2021-11-03T18:20:33.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.967+0000 7f8d8d5f0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d880602b0 0x7f8d88106e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:20:33.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.967+0000 7f8d8d5f0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d880f9720 0x7f8d881072f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:20:33.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.967+0000 7f8d8d5f0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d881078f0 con 0x7f8d880602b0 2021-11-03T18:20:33.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.967+0000 7f8d8d5f0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d88108100 con 0x7f8d880f9720 2021-11-03T18:20:33.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.968+0000 7f8d86ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d880602b0 0x7f8d88106e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:20:33.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.968+0000 7f8d86ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d880602b0 0x7f8d88106e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34336/0 (socket says 172.21.15.27:34336) 2021-11-03T18:20:33.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.968+0000 7f8d86ffd700 1 -- 172.21.15.27:0/3858372148 learned_addr learned my addr 172.21.15.27:0/3858372148 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:20:33.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.968+0000 7f8d867fc700 1 --2- 172.21.15.27:0/3858372148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d880f9720 0x7f8d881072f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:20:33.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.968+0000 7f8d86ffd700 1 -- 172.21.15.27:0/3858372148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d880f9720 msgr2=0x7f8d881072f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:20:33.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.968+0000 7f8d86ffd700 1 --2- 172.21.15.27:0/3858372148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d880f9720 0x7f8d881072f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:33.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.968+0000 7f8d86ffd700 1 -- 172.21.15.27:0/3858372148 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d70005040 con 0x7f8d880602b0 2021-11-03T18:20:33.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.968+0000 7f8d86ffd700 1 --2- 172.21.15.27:0/3858372148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d880602b0 0x7f8d88106e00 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 rx=0x7f8d8000d7f0 tx=0x7f8d80004e60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:20:33.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.968+0000 7f8d7f7fe700 1 -- 172.21.15.27:0/3858372148 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d80004020 con 0x7f8d880602b0 2021-11-03T18:20:33.973 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.968+0000 7f8d7f7fe700 1 -- 172.21.15.27:0/3858372148 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f8d8000f040 con 0x7f8d880602b0 2021-11-03T18:20:33.973 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.968+0000 7f8d8d5f0700 1 -- 172.21.15.27:0/3858372148 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d88108380 con 0x7f8d880602b0 2021-11-03T18:20:33.973 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.968+0000 7f8d7f7fe700 1 -- 172.21.15.27:0/3858372148 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d8000fea0 con 0x7f8d880602b0 2021-11-03T18:20:33.973 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.968+0000 7f8d8d5f0700 1 -- 172.21.15.27:0/3858372148 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d88108940 con 0x7f8d880602b0 2021-11-03T18:20:33.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.969+0000 7f8d8d5f0700 1 -- 172.21.15.27:0/3858372148 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d88059100 con 0x7f8d880602b0 2021-11-03T18:20:33.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.970+0000 7f8d7f7fe700 1 -- 172.21.15.27:0/3858372148 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f8d80004180 con 0x7f8d880602b0 2021-11-03T18:20:33.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.970+0000 7f8d7f7fe700 1 --2- 172.21.15.27:0/3858372148 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8d74064100 0x7f8d740665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:20:33.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.970+0000 7f8d7f7fe700 1 -- 172.21.15.27:0/3858372148 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f8d800808f0 con 0x7f8d880602b0 2021-11-03T18:20:33.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.970+0000 7f8d867fc700 1 --2- 172.21.15.27:0/3858372148 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8d74064100 0x7f8d740665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:20:33.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.971+0000 7f8d867fc700 1 --2- 172.21.15.27:0/3858372148 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8d74064100 0x7f8d740665c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f8d70004630 tx=0x7f8d70009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:20:33.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:33.974+0000 7f8d7f7fe700 1 -- 172.21.15.27:0/3858372148 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f8d80052aa0 con 0x7f8d880602b0 2021-11-03T18:20:34.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:34.185+0000 7f8d8d5f0700 1 -- 172.21.15.27:0/3858372148 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8d880008d0 con 0x7f8d880602b0 2021-11-03T18:20:34.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:34.185+0000 7f8d7f7fe700 1 -- 172.21.15.27:0/3858372148 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f8d80052690 con 0x7f8d880602b0 2021-11-03T18:20:34.184 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:20:34.184 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:20:34.184 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:20:34.184 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:20:34.185 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:20:34.185 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:20:34.185 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:20:34.185 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:20:34.185 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:20:34.186 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:20:34.186 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:20:34.186 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:20:34.186 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:20:34.187 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:20:34.187 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:20:34.187 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:20:34.187 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:20:34.187 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:20:34.188 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:20:34.188 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:20:34.188 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:20:34.188 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:20:34.189 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:20:34.189 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:20:34.189 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:20:34.189 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:20:34.190 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:20:34.190 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:20:34.190 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:20:34.191 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:20:34.191 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:20:34.191 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:20:34.191 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:20:34.191 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:20:34.192 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:20:34.192 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:20:34.192 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:20:34.192 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:20:34.193 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:20:34.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:34.187+0000 7f8d8d5f0700 1 -- 172.21.15.27:0/3858372148 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8d74064100 msgr2=0x7f8d740665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:20:34.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:34.187+0000 7f8d8d5f0700 1 --2- 172.21.15.27:0/3858372148 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8d74064100 0x7f8d740665c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 rx=0x7f8d70004630 tx=0x7f8d70009b30).stop 2021-11-03T18:20:34.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:34.187+0000 7f8d8d5f0700 1 -- 172.21.15.27:0/3858372148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d880602b0 msgr2=0x7f8d88106e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:20:34.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:34.187+0000 7f8d8d5f0700 1 --2- 172.21.15.27:0/3858372148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d880602b0 0x7f8d88106e00 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 rx=0x7f8d8000d7f0 tx=0x7f8d80004e60).stop 2021-11-03T18:20:34.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:34.187+0000 7f8d8d5f0700 1 -- 172.21.15.27:0/3858372148 shutdown_connections 2021-11-03T18:20:34.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:34.187+0000 7f8d8d5f0700 1 --2- 172.21.15.27:0/3858372148 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8d74064100 0x7f8d740665c0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:34.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:34.187+0000 7f8d8d5f0700 1 --2- 172.21.15.27:0/3858372148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8d880602b0 0x7f8d88106e00 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:34.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:34.187+0000 7f8d8d5f0700 1 --2- 172.21.15.27:0/3858372148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8d880f9720 0x7f8d881072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:20:34.196 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:34.187+0000 7f8d8d5f0700 1 -- 172.21.15.27:0/3858372148 >> 172.21.15.27:0/3858372148 conn(0x7f8d880f0bc0 msgr2=0x7f8d880f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:20:34.196 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:34.187+0000 7f8d8d5f0700 1 -- 172.21.15.27:0/3858372148 shutdown_connections 2021-11-03T18:20:34.196 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:20:34.187+0000 7f8d8d5f0700 1 -- 172.21.15.27:0/3858372148 wait complete. 2021-11-03T18:20:34.196 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:20:34.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:34 smithi115 ceph-mon[65453]: pgmap v1977: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:34.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:34 smithi115 ceph-mon[65453]: from='client.36136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:20:34.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:34 smithi115 ceph-mon[65453]: from='client.36140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:20:34.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:34 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3077775289' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:34.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:34 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3858372148' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:20:35.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:34 smithi027 ceph-mon[97244]: pgmap v1977: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:35.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:34 smithi027 ceph-mon[97244]: from='client.36136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:20:35.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:34 smithi027 ceph-mon[97244]: from='client.36140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:20:35.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:34 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3077775289' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:35.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:34 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3858372148' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:20:35.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3877.452679 secs 2021-11-03T18:20:36.083 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3877.452679 secs 2021-11-03T18:20:36.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:36 smithi115 ceph-mon[65453]: pgmap v1978: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:37.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:36 smithi027 ceph-mon[97244]: pgmap v1978: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:37.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:37 smithi115 ceph-mon[65453]: pgmap v1979: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:38.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:37 smithi027 ceph-mon[97244]: pgmap v1979: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:40.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:39 smithi115 ceph-mon[65453]: pgmap v1980: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:40.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:39 smithi027 ceph-mon[97244]: pgmap v1980: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:41.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3882.452775 secs 2021-11-03T18:20:41.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3882.452775 secs 2021-11-03T18:20:42.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:41 smithi115 ceph-mon[65453]: pgmap v1981: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:42.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:20:42.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:20:42.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:20:42.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:42.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:42.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:42.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:41 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:42.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:41 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:20:42.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:41 smithi027 ceph-mon[97244]: pgmap v1981: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:42.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:20:42.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:20:42.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:20:42.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:42.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:42.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:42.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:41 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:42.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:41 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:20:44.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:43 smithi115 ceph-mon[65453]: pgmap v1982: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:44.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:43 smithi027 ceph-mon[97244]: pgmap v1982: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:46.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3887.452916 secs 2021-11-03T18:20:46.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:45 smithi115 ceph-mon[65453]: pgmap v1983: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:46.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3887.452916 secs 2021-11-03T18:20:46.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:45 smithi027 ceph-mon[97244]: pgmap v1983: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:48.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:47 smithi115 ceph-mon[65453]: pgmap v1984: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:48.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:47 smithi027 ceph-mon[97244]: pgmap v1984: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:50.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:49 smithi115 ceph-mon[65453]: pgmap v1985: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:50.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:49 smithi027 ceph-mon[97244]: pgmap v1985: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:51.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3892.453062 secs 2021-11-03T18:20:51.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3892.453062 secs 2021-11-03T18:20:52.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:51 smithi115 ceph-mon[65453]: pgmap v1986: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:52.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:20:52.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:20:52.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:20:52.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:52.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:52.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:52.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:52.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:51 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:20:52.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:51 smithi027 ceph-mon[97244]: pgmap v1986: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:52.275 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:20:52.275 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:20:52.275 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:20:52.276 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:52.276 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:52.277 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:52.277 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:20:52.278 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:51 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:20:54.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:53 smithi115 ceph-mon[65453]: pgmap v1987: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:54.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:53 smithi027 ceph-mon[97244]: pgmap v1987: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:56.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3897.453203 secs 2021-11-03T18:20:56.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:55 smithi115 ceph-mon[65453]: pgmap v1988: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:56.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:20:56.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:20:56.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3897.453203 secs 2021-11-03T18:20:56.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:55 smithi027 ceph-mon[97244]: pgmap v1988: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:56.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:20:56.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:20:58.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:57 smithi115 ceph-mon[65453]: pgmap v1989: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:20:58.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:57 smithi027 ceph-mon[97244]: pgmap v1989: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:00.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:20:59 smithi115 ceph-mon[65453]: pgmap v1990: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:00.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:20:59 smithi027 ceph-mon[97244]: pgmap v1990: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:01.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3902.453322 secs 2021-11-03T18:21:01.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3902.453322 secs 2021-11-03T18:21:02.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:01 smithi115 ceph-mon[65453]: pgmap v1991: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:02.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:21:02.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:21:02.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:21:02.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:02.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:02.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:02.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:02.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:01 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:21:02.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:01 smithi027 ceph-mon[97244]: pgmap v1991: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:02.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:21:02.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:21:02.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:21:02.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:02.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:02.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:02.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:02.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:01 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:21:04.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:03 smithi115 ceph-mon[65453]: pgmap v1992: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:04.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:03 smithi027 ceph-mon[97244]: pgmap v1992: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:04.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.301+0000 7fa2cc9c3700 1 -- 172.21.15.27:0/2377039355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2c80f6530 msgr2=0x7fa2c80f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:21:04.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.301+0000 7fa2cc9c3700 1 --2- 172.21.15.27:0/2377039355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2c80f6530 0x7fa2c80f6950 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 rx=0x7fa2b8004660 tx=0x7fa2b8009b30).stop 2021-11-03T18:21:04.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.301+0000 7fa2cc9c3700 1 -- 172.21.15.27:0/2377039355 shutdown_connections 2021-11-03T18:21:04.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.301+0000 7fa2cc9c3700 1 --2- 172.21.15.27:0/2377039355 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2c80f77d0 0x7fa2c80f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:04.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.301+0000 7fa2cc9c3700 1 --2- 172.21.15.27:0/2377039355 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2c80f6530 0x7fa2c80f6950 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:04.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.301+0000 7fa2cc9c3700 1 -- 172.21.15.27:0/2377039355 >> 172.21.15.27:0/2377039355 conn(0x7fa2c80f1cd0 msgr2=0x7fa2c80f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:21:04.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.301+0000 7fa2cc9c3700 1 -- 172.21.15.27:0/2377039355 shutdown_connections 2021-11-03T18:21:04.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.301+0000 7fa2cc9c3700 1 -- 172.21.15.27:0/2377039355 wait complete. 2021-11-03T18:21:04.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.301+0000 7fa2cc9c3700 1 Processor -- start 2021-11-03T18:21:04.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.302+0000 7fa2cc9c3700 1 -- start start 2021-11-03T18:21:04.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.302+0000 7fa2cc9c3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2c80f6530 0x7fa2c806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:21:04.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.302+0000 7fa2cc9c3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2c80f77d0 0x7fa2c8069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:21:04.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.302+0000 7fa2cc9c3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c806a380 con 0x7fa2c80f77d0 2021-11-03T18:21:04.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.302+0000 7fa2cc9c3700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2c806a4c0 con 0x7fa2c80f6530 2021-11-03T18:21:04.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.302+0000 7fa2c6ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2c80f77d0 0x7fa2c8069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:21:04.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.302+0000 7fa2c6ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2c80f77d0 0x7fa2c8069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34356/0 (socket says 172.21.15.27:34356) 2021-11-03T18:21:04.307 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.302+0000 7fa2c6ffd700 1 -- 172.21.15.27:0/319965927 learned_addr learned my addr 172.21.15.27:0/319965927 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:21:04.307 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.302+0000 7fa2c77fe700 1 --2- 172.21.15.27:0/319965927 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2c80f6530 0x7fa2c806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:21:04.307 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.302+0000 7fa2c6ffd700 1 -- 172.21.15.27:0/319965927 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2c80f6530 msgr2=0x7fa2c806b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:21:04.307 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.302+0000 7fa2c6ffd700 1 --2- 172.21.15.27:0/319965927 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2c80f6530 0x7fa2c806b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:04.308 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.302+0000 7fa2c6ffd700 1 -- 172.21.15.27:0/319965927 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2b8005040 con 0x7fa2c80f77d0 2021-11-03T18:21:04.308 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.302+0000 7fa2c6ffd700 1 --2- 172.21.15.27:0/319965927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2c80f77d0 0x7fa2c8069e20 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 rx=0x7fa2bc0026d0 tx=0x7fa2bc009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:21:04.308 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.303+0000 7fa2affff700 1 -- 172.21.15.27:0/319965927 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2bc0079c0 con 0x7fa2c80f77d0 2021-11-03T18:21:04.308 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.303+0000 7fa2cc9c3700 1 -- 172.21.15.27:0/319965927 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2c806a7a0 con 0x7fa2c80f77d0 2021-11-03T18:21:04.309 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.303+0000 7fa2affff700 1 -- 172.21.15.27:0/319965927 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fa2bc00f040 con 0x7fa2c80f77d0 2021-11-03T18:21:04.309 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.303+0000 7fa2cc9c3700 1 -- 172.21.15.27:0/319965927 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c810ed90 con 0x7fa2c80f77d0 2021-11-03T18:21:04.309 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.303+0000 7fa2affff700 1 -- 172.21.15.27:0/319965927 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa2bc0148b0 con 0x7fa2c80f77d0 2021-11-03T18:21:04.309 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.304+0000 7fa2cc9c3700 1 -- 172.21.15.27:0/319965927 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2c8045ad0 con 0x7fa2c80f77d0 2021-11-03T18:21:04.310 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.304+0000 7fa2affff700 1 -- 172.21.15.27:0/319965927 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fa2bc007b20 con 0x7fa2c80f77d0 2021-11-03T18:21:04.310 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.305+0000 7fa2affff700 1 --2- 172.21.15.27:0/319965927 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa2b0064100 0x7fa2b00665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:21:04.310 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.305+0000 7fa2affff700 1 -- 172.21.15.27:0/319965927 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fa2bc0806d0 con 0x7fa2c80f77d0 2021-11-03T18:21:04.310 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.305+0000 7fa2c77fe700 1 --2- 172.21.15.27:0/319965927 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa2b0064100 0x7fa2b00665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:21:04.311 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.305+0000 7fa2c77fe700 1 --2- 172.21.15.27:0/319965927 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa2b0064100 0x7fa2b00665c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7fa2b8013690 tx=0x7fa2b8009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:21:04.311 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.309+0000 7fa2affff700 1 -- 172.21.15.27:0/319965927 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fa2bc0529d0 con 0x7fa2c80f77d0 2021-11-03T18:21:04.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.488+0000 7fa2cc9c3700 1 -- 172.21.15.27:0/319965927 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa2c80fed60 con 0x7fa2b0064100 2021-11-03T18:21:04.488 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.489+0000 7fa2affff700 1 -- 172.21.15.27:0/319965927 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fa2c80fed60 con 0x7fa2b0064100 2021-11-03T18:21:04.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.491+0000 7fa2cc9c3700 1 -- 172.21.15.27:0/319965927 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa2b0064100 msgr2=0x7fa2b00665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:21:04.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.491+0000 7fa2cc9c3700 1 --2- 172.21.15.27:0/319965927 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa2b0064100 0x7fa2b00665c0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7fa2b8013690 tx=0x7fa2b8009b30).stop 2021-11-03T18:21:04.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.491+0000 7fa2cc9c3700 1 -- 172.21.15.27:0/319965927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2c80f77d0 msgr2=0x7fa2c8069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:21:04.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.491+0000 7fa2cc9c3700 1 --2- 172.21.15.27:0/319965927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2c80f77d0 0x7fa2c8069e20 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 rx=0x7fa2bc0026d0 tx=0x7fa2bc009de0).stop 2021-11-03T18:21:04.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.491+0000 7fa2cc9c3700 1 -- 172.21.15.27:0/319965927 shutdown_connections 2021-11-03T18:21:04.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.491+0000 7fa2cc9c3700 1 --2- 172.21.15.27:0/319965927 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa2b0064100 0x7fa2b00665c0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:04.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.491+0000 7fa2cc9c3700 1 --2- 172.21.15.27:0/319965927 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa2c80f6530 0x7fa2c806b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:04.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.491+0000 7fa2cc9c3700 1 --2- 172.21.15.27:0/319965927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa2c80f77d0 0x7fa2c8069e20 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:04.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.491+0000 7fa2cc9c3700 1 -- 172.21.15.27:0/319965927 >> 172.21.15.27:0/319965927 conn(0x7fa2c80f1cd0 msgr2=0x7fa2c80fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:21:04.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.492+0000 7fa2cc9c3700 1 -- 172.21.15.27:0/319965927 shutdown_connections 2021-11-03T18:21:04.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.492+0000 7fa2cc9c3700 1 -- 172.21.15.27:0/319965927 wait complete. 2021-11-03T18:21:04.500 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:21:04.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.592+0000 7fd73359e700 1 -- 172.21.15.27:0/3371295076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7340f51a0 msgr2=0x7fd7340f7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:21:04.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.592+0000 7fd73359e700 1 --2- 172.21.15.27:0/3371295076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7340f51a0 0x7fd7340f7920 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 rx=0x7fd71c004660 tx=0x7fd71c009b30).stop 2021-11-03T18:21:04.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.592+0000 7fd73359e700 1 -- 172.21.15.27:0/3371295076 shutdown_connections 2021-11-03T18:21:04.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.592+0000 7fd73359e700 1 --2- 172.21.15.27:0/3371295076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7340f51a0 0x7fd7340f7920 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:04.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.592+0000 7fd73359e700 1 --2- 172.21.15.27:0/3371295076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7340f4800 0x7fd7340f4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:04.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.592+0000 7fd73359e700 1 -- 172.21.15.27:0/3371295076 >> 172.21.15.27:0/3371295076 conn(0x7fd7340f0320 msgr2=0x7fd7340f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:21:04.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.592+0000 7fd73359e700 1 -- 172.21.15.27:0/3371295076 shutdown_connections 2021-11-03T18:21:04.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.592+0000 7fd73359e700 1 -- 172.21.15.27:0/3371295076 wait complete. 2021-11-03T18:21:04.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.592+0000 7fd73359e700 1 Processor -- start 2021-11-03T18:21:04.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.592+0000 7fd73359e700 1 -- start start 2021-11-03T18:21:04.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.593+0000 7fd73359e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7340f4800 0x7fd734106d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:21:04.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.593+0000 7fd73359e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7340f51a0 0x7fd734107270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:21:04.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.593+0000 7fd73359e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd734107870 con 0x7fd7340f51a0 2021-11-03T18:21:04.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.593+0000 7fd73359e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd734108080 con 0x7fd7340f4800 2021-11-03T18:21:04.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.593+0000 7fd731d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7340f51a0 0x7fd734107270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:21:04.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.593+0000 7fd731d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7340f51a0 0x7fd734107270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34366/0 (socket says 172.21.15.27:34366) 2021-11-03T18:21:04.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.593+0000 7fd731d9b700 1 -- 172.21.15.27:0/3970038135 learned_addr learned my addr 172.21.15.27:0/3970038135 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:21:04.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.593+0000 7fd73259c700 1 --2- 172.21.15.27:0/3970038135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7340f4800 0x7fd734106d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:21:04.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.593+0000 7fd731d9b700 1 -- 172.21.15.27:0/3970038135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7340f4800 msgr2=0x7fd734106d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:21:04.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.593+0000 7fd731d9b700 1 --2- 172.21.15.27:0/3970038135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7340f4800 0x7fd734106d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:04.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.593+0000 7fd731d9b700 1 -- 172.21.15.27:0/3970038135 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd71c005040 con 0x7fd7340f51a0 2021-11-03T18:21:04.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.593+0000 7fd731d9b700 1 --2- 172.21.15.27:0/3970038135 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7340f51a0 0x7fd734107270 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 rx=0x7fd71c000c00 tx=0x7fd71c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:21:04.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.593+0000 7fd72affd700 1 -- 172.21.15.27:0/3970038135 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd71c019070 con 0x7fd7340f51a0 2021-11-03T18:21:04.600 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.593+0000 7fd72affd700 1 -- 172.21.15.27:0/3970038135 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fd71c013380 con 0x7fd7340f51a0 2021-11-03T18:21:04.600 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.593+0000 7fd73359e700 1 -- 172.21.15.27:0/3970038135 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7341082a0 con 0x7fd7340f51a0 2021-11-03T18:21:04.600 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.593+0000 7fd72affd700 1 -- 172.21.15.27:0/3970038135 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd71c014cd0 con 0x7fd7340f51a0 2021-11-03T18:21:04.600 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.593+0000 7fd73359e700 1 -- 172.21.15.27:0/3970038135 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd734108800 con 0x7fd7340f51a0 2021-11-03T18:21:04.602 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.594+0000 7fd73359e700 1 -- 172.21.15.27:0/3970038135 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd734059100 con 0x7fd7340f51a0 2021-11-03T18:21:04.602 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.595+0000 7fd72affd700 1 -- 172.21.15.27:0/3970038135 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fd71c025070 con 0x7fd7340f51a0 2021-11-03T18:21:04.602 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.595+0000 7fd72affd700 1 --2- 172.21.15.27:0/3970038135 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd720064100 0x7fd7200665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:21:04.603 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.595+0000 7fd72affd700 1 -- 172.21.15.27:0/3970038135 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fd71c081780 con 0x7fd7340f51a0 2021-11-03T18:21:04.603 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.595+0000 7fd73259c700 1 --2- 172.21.15.27:0/3970038135 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd720064100 0x7fd7200665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:21:04.603 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.596+0000 7fd73259c700 1 --2- 172.21.15.27:0/3970038135 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd720064100 0x7fd7200665c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7fd734106af0 tx=0x7fd72400b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:21:04.603 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.599+0000 7fd72affd700 1 -- 172.21.15.27:0/3970038135 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fd71c053a80 con 0x7fd7340f51a0 2021-11-03T18:21:04.773 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.774+0000 7fd73359e700 1 -- 172.21.15.27:0/3970038135 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7340fee90 con 0x7fd720064100 2021-11-03T18:21:04.787 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.789+0000 7fd72affd700 1 -- 172.21.15.27:0/3970038135 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fd7340fee90 con 0x7fd720064100 2021-11-03T18:21:04.788 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:21:04.788 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (72m) 97s ago 73m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:21:04.789 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (64m) 97s ago 73m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:21:04.789 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (64m) 54s ago 73m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:21:04.789 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (72m) 97s ago 73m 42.4M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:21:04.789 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (66m) 97s ago 68m 26.5M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:21:04.789 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (69m) 97s ago 69m 3849M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:21:04.790 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (64m) 54s ago 69m 26.1M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:21:04.790 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (66m) 54s ago 69m 27.2M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:21:04.790 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (66m) 97s ago 75m 590M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:21:04.790 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (65m) 54s ago 73m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:21:04.791 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (65m) 97s ago 75m 312M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:21:04.791 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (64m) 54s ago 73m 145M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:21:04.791 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (73m) 97s ago 73m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:21:04.791 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (72m) 54s ago 72m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:21:04.792 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (63m) 97s ago 72m 392M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:21:04.792 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (63m) 97s ago 71m 427M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:21:04.792 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (62m) 97s ago 71m 320M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:21:04.792 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (62m) 54s ago 71m 454M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:21:04.793 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (62m) 54s ago 70m 359M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:21:04.793 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (61m) 54s ago 70m 368M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:21:04.793 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (72m) 97s ago 73m 59.9M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:21:04.795 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.790+0000 7fd73359e700 1 -- 172.21.15.27:0/3970038135 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd720064100 msgr2=0x7fd7200665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:21:04.795 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.790+0000 7fd73359e700 1 --2- 172.21.15.27:0/3970038135 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd720064100 0x7fd7200665c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7fd734106af0 tx=0x7fd72400b040).stop 2021-11-03T18:21:04.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.790+0000 7fd73359e700 1 -- 172.21.15.27:0/3970038135 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7340f51a0 msgr2=0x7fd734107270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:21:04.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.790+0000 7fd73359e700 1 --2- 172.21.15.27:0/3970038135 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7340f51a0 0x7fd734107270 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 rx=0x7fd71c000c00 tx=0x7fd71c00b040).stop 2021-11-03T18:21:04.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.790+0000 7fd73359e700 1 -- 172.21.15.27:0/3970038135 shutdown_connections 2021-11-03T18:21:04.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.790+0000 7fd73359e700 1 --2- 172.21.15.27:0/3970038135 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd720064100 0x7fd7200665c0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:04.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.790+0000 7fd73359e700 1 --2- 172.21.15.27:0/3970038135 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd7340f4800 0x7fd734106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:04.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.790+0000 7fd73359e700 1 --2- 172.21.15.27:0/3970038135 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd7340f51a0 0x7fd734107270 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:04.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.790+0000 7fd73359e700 1 -- 172.21.15.27:0/3970038135 >> 172.21.15.27:0/3970038135 conn(0x7fd7340f0320 msgr2=0x7fd7340fb370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:21:04.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.790+0000 7fd73359e700 1 -- 172.21.15.27:0/3970038135 shutdown_connections 2021-11-03T18:21:04.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.791+0000 7fd73359e700 1 -- 172.21.15.27:0/3970038135 wait complete. 2021-11-03T18:21:04.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.890+0000 7f8ed6bf4700 1 -- 172.21.15.27:0/1567314352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ed00f55e0 msgr2=0x7f8ed00f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:21:04.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.890+0000 7f8ed6bf4700 1 --2- 172.21.15.27:0/1567314352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ed00f55e0 0x7f8ed00f5a00 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 rx=0x7f8ec8004660 tx=0x7f8ec8009b30).stop 2021-11-03T18:21:04.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.890+0000 7f8ed6bf4700 1 -- 172.21.15.27:0/1567314352 shutdown_connections 2021-11-03T18:21:04.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.890+0000 7f8ed6bf4700 1 --2- 172.21.15.27:0/1567314352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ed00f5ef0 0x7f8ed00f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:04.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.890+0000 7f8ed6bf4700 1 --2- 172.21.15.27:0/1567314352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ed00f55e0 0x7f8ed00f5a00 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:04.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.890+0000 7f8ed6bf4700 1 -- 172.21.15.27:0/1567314352 >> 172.21.15.27:0/1567314352 conn(0x7f8ed00f0de0 msgr2=0x7f8ed00f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:21:04.891 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.890+0000 7f8ed6bf4700 1 -- 172.21.15.27:0/1567314352 shutdown_connections 2021-11-03T18:21:04.891 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.890+0000 7f8ed6bf4700 1 -- 172.21.15.27:0/1567314352 wait complete. 2021-11-03T18:21:04.891 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.890+0000 7f8ed6bf4700 1 Processor -- start 2021-11-03T18:21:04.891 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.890+0000 7f8ed6bf4700 1 -- start start 2021-11-03T18:21:04.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.891+0000 7f8ed6bf4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ed00f55e0 0x7f8ed0106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:21:04.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.891+0000 7f8ed6bf4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ed00f5ef0 0x7f8ed0107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:21:04.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.891+0000 7f8ed6bf4700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed0107a00 con 0x7f8ed00f5ef0 2021-11-03T18:21:04.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.891+0000 7f8ed6bf4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed0107b40 con 0x7f8ed00f55e0 2021-11-03T18:21:04.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.891+0000 7f8ecffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ed00f5ef0 0x7f8ed0107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:21:04.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.891+0000 7f8ecffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ed00f5ef0 0x7f8ed0107450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34376/0 (socket says 172.21.15.27:34376) 2021-11-03T18:21:04.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.891+0000 7f8ecffff700 1 -- 172.21.15.27:0/741709099 learned_addr learned my addr 172.21.15.27:0/741709099 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:21:04.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.891+0000 7f8ed4990700 1 --2- 172.21.15.27:0/741709099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ed00f55e0 0x7f8ed0106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:21:04.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.891+0000 7f8ecffff700 1 -- 172.21.15.27:0/741709099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ed00f55e0 msgr2=0x7f8ed0106f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:21:04.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.891+0000 7f8ecffff700 1 --2- 172.21.15.27:0/741709099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ed00f55e0 0x7f8ed0106f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:04.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.891+0000 7f8ecffff700 1 -- 172.21.15.27:0/741709099 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec8005040 con 0x7f8ed00f5ef0 2021-11-03T18:21:04.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.891+0000 7f8ed4990700 1 --2- 172.21.15.27:0/741709099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ed00f55e0 0x7f8ed0106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:21:04.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.891+0000 7f8ecffff700 1 --2- 172.21.15.27:0/741709099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ed00f5ef0 0x7f8ed0107450 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 rx=0x7f8ec000c760 tx=0x7f8ec0004d20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:21:04.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.891+0000 7f8ecd7fa700 1 -- 172.21.15.27:0/741709099 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ec0004200 con 0x7f8ed00f5ef0 2021-11-03T18:21:04.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.891+0000 7f8ecd7fa700 1 -- 172.21.15.27:0/741709099 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f8ec00109c0 con 0x7f8ed00f5ef0 2021-11-03T18:21:04.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.891+0000 7f8ed6bf4700 1 -- 172.21.15.27:0/741709099 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ed01084e0 con 0x7f8ed00f5ef0 2021-11-03T18:21:04.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.891+0000 7f8ecd7fa700 1 -- 172.21.15.27:0/741709099 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8ec0003920 con 0x7f8ed00f5ef0 2021-11-03T18:21:04.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.891+0000 7f8ed6bf4700 1 -- 172.21.15.27:0/741709099 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ed0108aa0 con 0x7f8ed00f5ef0 2021-11-03T18:21:04.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.892+0000 7f8ed6bf4700 1 -- 172.21.15.27:0/741709099 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ed0045ad0 con 0x7f8ed00f5ef0 2021-11-03T18:21:04.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.893+0000 7f8ecd7fa700 1 -- 172.21.15.27:0/741709099 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f8ec0018020 con 0x7f8ed00f5ef0 2021-11-03T18:21:04.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.893+0000 7f8ecd7fa700 1 --2- 172.21.15.27:0/741709099 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8ebc0640b0 0x7f8ebc066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:21:04.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.893+0000 7f8ecd7fa700 1 -- 172.21.15.27:0/741709099 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f8ec00807e0 con 0x7f8ed00f5ef0 2021-11-03T18:21:04.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.893+0000 7f8ed4990700 1 --2- 172.21.15.27:0/741709099 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8ebc0640b0 0x7f8ebc066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:21:04.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.894+0000 7f8ed4990700 1 --2- 172.21.15.27:0/741709099 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8ebc0640b0 0x7f8ebc066570 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7f8ec8004e30 tx=0x7f8ec8009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:21:04.900 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:04.897+0000 7f8ecd7fa700 1 -- 172.21.15.27:0/741709099 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f8ec00529f0 con 0x7f8ed00f5ef0 2021-11-03T18:21:05.129 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.129+0000 7f8ed6bf4700 1 -- 172.21.15.27:0/741709099 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f8ed01091a0 con 0x7f8ed00f5ef0 2021-11-03T18:21:05.130 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.129+0000 7f8ecd7fa700 1 -- 172.21.15.27:0/741709099 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f8ec00525e0 con 0x7f8ed00f5ef0 2021-11-03T18:21:05.130 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:21:05.131 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:21:05.131 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:21:05.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:21:05.131 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:21:05.132 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:21:05.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:21:05.132 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:21:05.132 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:21:05.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:21:05.133 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:21:05.133 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:21:05.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:21:05.133 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:21:05.134 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:21:05.134 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:21:05.134 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:21:05.135 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:21:05.135 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.131+0000 7f8ed6bf4700 1 -- 172.21.15.27:0/741709099 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8ebc0640b0 msgr2=0x7f8ebc066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:21:05.136 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.131+0000 7f8ed6bf4700 1 --2- 172.21.15.27:0/741709099 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8ebc0640b0 0x7f8ebc066570 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7f8ec8004e30 tx=0x7f8ec8009b30).stop 2021-11-03T18:21:05.136 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.131+0000 7f8ed6bf4700 1 -- 172.21.15.27:0/741709099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ed00f5ef0 msgr2=0x7f8ed0107450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:21:05.136 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.131+0000 7f8ed6bf4700 1 --2- 172.21.15.27:0/741709099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ed00f5ef0 0x7f8ed0107450 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 rx=0x7f8ec000c760 tx=0x7f8ec0004d20).stop 2021-11-03T18:21:05.137 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.131+0000 7f8ed6bf4700 1 -- 172.21.15.27:0/741709099 shutdown_connections 2021-11-03T18:21:05.137 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.131+0000 7f8ed6bf4700 1 --2- 172.21.15.27:0/741709099 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8ebc0640b0 0x7f8ebc066570 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:05.137 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.131+0000 7f8ed6bf4700 1 --2- 172.21.15.27:0/741709099 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8ed00f55e0 0x7f8ed0106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:05.138 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.131+0000 7f8ed6bf4700 1 --2- 172.21.15.27:0/741709099 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8ed00f5ef0 0x7f8ed0107450 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:05.138 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.131+0000 7f8ed6bf4700 1 -- 172.21.15.27:0/741709099 >> 172.21.15.27:0/741709099 conn(0x7f8ed00f0de0 msgr2=0x7f8ed005ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:21:05.138 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.131+0000 7f8ed6bf4700 1 -- 172.21.15.27:0/741709099 shutdown_connections 2021-11-03T18:21:05.139 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.131+0000 7f8ed6bf4700 1 -- 172.21.15.27:0/741709099 wait complete. 2021-11-03T18:21:05.235 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.231+0000 7f9745861700 1 -- 172.21.15.27:0/3037217125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97400f77d0 msgr2=0x7f97400f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:21:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.231+0000 7f9745861700 1 --2- 172.21.15.27:0/3037217125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97400f77d0 0x7f97400f7c30 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 rx=0x7f972c004660 tx=0x7f972c009b30).stop 2021-11-03T18:21:05.236 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.231+0000 7f9745861700 1 -- 172.21.15.27:0/3037217125 shutdown_connections 2021-11-03T18:21:05.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.231+0000 7f9745861700 1 --2- 172.21.15.27:0/3037217125 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97400f77d0 0x7f97400f7c30 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:05.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.231+0000 7f9745861700 1 --2- 172.21.15.27:0/3037217125 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f97400f6530 0x7f97400f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:05.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.231+0000 7f9745861700 1 -- 172.21.15.27:0/3037217125 >> 172.21.15.27:0/3037217125 conn(0x7f97400f1cd0 msgr2=0x7f97400f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:21:05.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.232+0000 7f9745861700 1 -- 172.21.15.27:0/3037217125 shutdown_connections 2021-11-03T18:21:05.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.232+0000 7f9745861700 1 -- 172.21.15.27:0/3037217125 wait complete. 2021-11-03T18:21:05.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.232+0000 7f9745861700 1 Processor -- start 2021-11-03T18:21:05.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.232+0000 7f9745861700 1 -- start start 2021-11-03T18:21:05.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.232+0000 7f9745861700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97400f6530 0x7f9740106d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:21:05.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.232+0000 7f9745861700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f97400f77d0 0x7f9740107210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:21:05.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.232+0000 7f9745861700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97401077c0 con 0x7f97400f6530 2021-11-03T18:21:05.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.232+0000 7f9745861700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9740107fc0 con 0x7f97400f77d0 2021-11-03T18:21:05.242 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.232+0000 7f974485f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97400f6530 0x7f9740106d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:21:05.242 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.233+0000 7f974485f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97400f6530 0x7f9740106d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34384/0 (socket says 172.21.15.27:34384) 2021-11-03T18:21:05.243 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.233+0000 7f974485f700 1 -- 172.21.15.27:0/3770646901 learned_addr learned my addr 172.21.15.27:0/3770646901 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:21:05.243 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.233+0000 7f973ffff700 1 --2- 172.21.15.27:0/3770646901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f97400f77d0 0x7f9740107210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:21:05.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.233+0000 7f974485f700 1 -- 172.21.15.27:0/3770646901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f97400f77d0 msgr2=0x7f9740107210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:21:05.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.233+0000 7f974485f700 1 --2- 172.21.15.27:0/3770646901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f97400f77d0 0x7f9740107210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:05.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.233+0000 7f974485f700 1 -- 172.21.15.27:0/3770646901 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f972c005040 con 0x7f97400f6530 2021-11-03T18:21:05.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.233+0000 7f974485f700 1 --2- 172.21.15.27:0/3770646901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97400f6530 0x7f9740106d20 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 rx=0x7f973400e7c0 tx=0x7f973400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:21:05.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.233+0000 7f973d7fa700 1 -- 172.21.15.27:0/3770646901 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97340092e0 con 0x7f97400f6530 2021-11-03T18:21:05.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.233+0000 7f973d7fa700 1 -- 172.21.15.27:0/3770646901 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f973400f040 con 0x7f97400f6530 2021-11-03T18:21:05.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.233+0000 7f973d7fa700 1 -- 172.21.15.27:0/3770646901 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f97340147e0 con 0x7f97400f6530 2021-11-03T18:21:05.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.233+0000 7f9745861700 1 -- 172.21.15.27:0/3770646901 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97401082a0 con 0x7f97400f6530 2021-11-03T18:21:05.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.233+0000 7f9745861700 1 -- 172.21.15.27:0/3770646901 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9740108860 con 0x7f97400f6530 2021-11-03T18:21:05.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.234+0000 7f9745861700 1 -- 172.21.15.27:0/3770646901 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9740059100 con 0x7f97400f6530 2021-11-03T18:21:05.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.235+0000 7f973d7fa700 1 -- 172.21.15.27:0/3770646901 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f973401b030 con 0x7f97400f6530 2021-11-03T18:21:05.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.235+0000 7f973d7fa700 1 --2- 172.21.15.27:0/3770646901 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9730064100 0x7f97300665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:21:05.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.235+0000 7f973d7fa700 1 -- 172.21.15.27:0/3770646901 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f97340807d0 con 0x7f97400f6530 2021-11-03T18:21:05.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.235+0000 7f973ffff700 1 --2- 172.21.15.27:0/3770646901 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9730064100 0x7f97300665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:21:05.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.235+0000 7f973ffff700 1 --2- 172.21.15.27:0/3770646901 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9730064100 0x7f97300665c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7f972c005010 tx=0x7f972c016040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:21:05.248 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:04 smithi115 ceph-mon[65453]: from='client.36152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:21:05.248 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.239+0000 7f973d7fa700 1 -- 172.21.15.27:0/3770646901 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f97340529e0 con 0x7f97400f6530 2021-11-03T18:21:05.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:04 smithi027 ceph-mon[97244]: from='client.36152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:21:05.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.443+0000 7f9745861700 1 -- 172.21.15.27:0/3770646901 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f97400fed90 con 0x7f97400f6530 2021-11-03T18:21:05.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.443+0000 7f973d7fa700 1 -- 172.21.15.27:0/3770646901 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f9734019020 con 0x7f97400f6530 2021-11-03T18:21:05.443 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:21:05.443 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:21:05.443 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:21:05.444 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:21:05.444 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:21:05.444 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:21:05.444 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:21:05.445 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:21:05.445 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:21:05.445 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:21:05.445 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:21:05.446 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:21:05.446 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:21:05.446 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:21:05.446 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:21:05.447 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:21:05.447 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:21:05.447 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:21:05.447 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:21:05.448 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:21:05.448 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:21:05.448 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:21:05.448 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:21:05.448 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:21:05.449 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:21:05.449 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:21:05.449 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:21:05.449 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:21:05.450 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:21:05.450 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:21:05.450 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:21:05.450 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:21:05.450 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:21:05.451 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:21:05.451 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:21:05.451 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:21:05.451 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:21:05.452 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:21:05.452 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:21:05.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.445+0000 7f9745861700 1 -- 172.21.15.27:0/3770646901 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9730064100 msgr2=0x7f97300665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:21:05.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.445+0000 7f9745861700 1 --2- 172.21.15.27:0/3770646901 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9730064100 0x7f97300665c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7f972c005010 tx=0x7f972c016040).stop 2021-11-03T18:21:05.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.445+0000 7f9745861700 1 -- 172.21.15.27:0/3770646901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97400f6530 msgr2=0x7f9740106d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:21:05.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.445+0000 7f9745861700 1 --2- 172.21.15.27:0/3770646901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97400f6530 0x7f9740106d20 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 rx=0x7f973400e7c0 tx=0x7f973400b040).stop 2021-11-03T18:21:05.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.445+0000 7f9745861700 1 -- 172.21.15.27:0/3770646901 shutdown_connections 2021-11-03T18:21:05.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.445+0000 7f9745861700 1 --2- 172.21.15.27:0/3770646901 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9730064100 0x7f97300665c0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:05.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.445+0000 7f9745861700 1 --2- 172.21.15.27:0/3770646901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f97400f6530 0x7f9740106d20 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:05.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.445+0000 7f9745861700 1 --2- 172.21.15.27:0/3770646901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f97400f77d0 0x7f9740107210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:05.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.445+0000 7f9745861700 1 -- 172.21.15.27:0/3770646901 >> 172.21.15.27:0/3770646901 conn(0x7f97400f1cd0 msgr2=0x7f97400fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:21:05.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.445+0000 7f9745861700 1 -- 172.21.15.27:0/3770646901 shutdown_connections 2021-11-03T18:21:05.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:05.445+0000 7f9745861700 1 -- 172.21.15.27:0/3770646901 wait complete. 2021-11-03T18:21:05.456 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:21:06.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3907.453404 secs 2021-11-03T18:21:06.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:05 smithi115 ceph-mon[65453]: from='client.36156 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:21:06.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:05 smithi115 ceph-mon[65453]: pgmap v1993: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:06.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:05 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/741709099' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:06.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:05 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3770646901' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:21:06.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3907.453404 secs 2021-11-03T18:21:06.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:05 smithi027 ceph-mon[97244]: from='client.36156 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:21:06.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:05 smithi027 ceph-mon[97244]: pgmap v1993: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:06.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:05 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/741709099' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:06.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:05 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3770646901' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:21:08.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:07 smithi115 ceph-mon[65453]: pgmap v1994: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:08.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:07 smithi027 ceph-mon[97244]: pgmap v1994: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:10.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:09 smithi115 ceph-mon[65453]: pgmap v1995: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:10.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:09 smithi027 ceph-mon[97244]: pgmap v1995: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:11.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3912.453554 secs 2021-11-03T18:21:11.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3912.453554 secs 2021-11-03T18:21:12.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:11 smithi115 ceph-mon[65453]: pgmap v1996: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:12.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:21:12.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:21:12.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:21:12.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:12.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:12.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:12.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:11 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:12.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:11 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:21:12.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:11 smithi027 ceph-mon[97244]: pgmap v1996: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:12.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:21:12.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:21:12.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:21:12.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:12.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:12.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:12.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:11 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:12.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:11 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:21:14.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:13 smithi115 ceph-mon[65453]: pgmap v1997: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:14.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:13 smithi027 ceph-mon[97244]: pgmap v1997: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:16.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3917.453694 secs 2021-11-03T18:21:16.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:15 smithi027 ceph-mon[97244]: pgmap v1998: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:16.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3917.453694 secs 2021-11-03T18:21:16.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:15 smithi115 ceph-mon[65453]: pgmap v1998: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:18.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:17 smithi027 ceph-mon[97244]: pgmap v1999: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:18.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:17 smithi115 ceph-mon[65453]: pgmap v1999: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:20.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:19 smithi027 ceph-mon[97244]: pgmap v2000: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:20.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:19 smithi115 ceph-mon[65453]: pgmap v2000: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:21.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3922.453758 secs 2021-11-03T18:21:21.411 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3922.453758 secs 2021-11-03T18:21:22.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:21 smithi027 ceph-mon[97244]: pgmap v2001: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:22.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:21:22.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:21 smithi027 ceph-mon[97244]: poll took 60177.732 ms: 1 events 2021-11-03T18:21:22.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:21:22.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:21:22.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:22.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:22.275 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:22.275 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:21 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:22.276 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:21 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:21:22.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:21 smithi115 ceph-mon[65453]: pgmap v2001: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:22.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:21:22.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:22 smithi115 ceph-mon[65453]: poll took 60177.732 ms: 1 events 2021-11-03T18:21:22.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:21:22.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:21:22.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:22.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:22.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:22.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:22.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:22 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:21:24.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:24 smithi115 ceph-mon[65453]: pgmap v2002: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:24.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:24 smithi027 ceph-mon[97244]: pgmap v2002: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:26.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:26 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3927.453894 secs 2021-11-03T18:21:26.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:26 smithi027 ceph-mon[97244]: pgmap v2003: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:26.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:26 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3927.453894 secs 2021-11-03T18:21:26.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:26 smithi115 ceph-mon[65453]: pgmap v2003: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:28.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:28 smithi027 ceph-mon[97244]: pgmap v2004: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:28.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:28 smithi115 ceph-mon[65453]: pgmap v2004: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:30.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:30 smithi027 ceph-mon[97244]: pgmap v2005: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:30.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:30 smithi115 ceph-mon[65453]: pgmap v2005: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:31.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:31 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3932.454011 secs 2021-11-03T18:21:31.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:31 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3932.454011 secs 2021-11-03T18:21:32.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:32 smithi115 ceph-mon[65453]: pgmap v2006: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:32.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:21:32.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:21:32.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:21:32.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:32.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:32.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:32.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:32.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:32 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:21:32.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:32 smithi027 ceph-mon[97244]: pgmap v2006: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:32.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:21:32.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:21:32.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:21:32.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:32.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:32.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:32.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:32.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:32 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:21:34.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:34 smithi115 ceph-mon[65453]: pgmap v2007: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:34.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:34 smithi027 ceph-mon[97244]: pgmap v2007: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:35.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.560+0000 7f01f5a46700 1 -- 172.21.15.27:0/2226664803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01f00f66e0 msgr2=0x7f01f00f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:21:35.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.560+0000 7f01f5a46700 1 --2- 172.21.15.27:0/2226664803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01f00f66e0 0x7f01f00f6b00 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 rx=0x7f01e0004660 tx=0x7f01e0009b30).stop 2021-11-03T18:21:35.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.560+0000 7f01f5a46700 1 -- 172.21.15.27:0/2226664803 shutdown_connections 2021-11-03T18:21:35.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.560+0000 7f01f5a46700 1 --2- 172.21.15.27:0/2226664803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01f00f6ff0 0x7f01f00f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:35.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.560+0000 7f01f5a46700 1 --2- 172.21.15.27:0/2226664803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01f00f66e0 0x7f01f00f6b00 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:35.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.560+0000 7f01f5a46700 1 -- 172.21.15.27:0/2226664803 >> 172.21.15.27:0/2226664803 conn(0x7f01f00f1f00 msgr2=0x7f01f00f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:21:35.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.561+0000 7f01f5a46700 1 -- 172.21.15.27:0/2226664803 shutdown_connections 2021-11-03T18:21:35.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.561+0000 7f01f5a46700 1 -- 172.21.15.27:0/2226664803 wait complete. 2021-11-03T18:21:35.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.562+0000 7f01f5a46700 1 Processor -- start 2021-11-03T18:21:35.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.562+0000 7f01f5a46700 1 -- start start 2021-11-03T18:21:35.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.562+0000 7f01f5a46700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01f00f66e0 0x7f01f006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:21:35.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.562+0000 7f01f5a46700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01f00f6ff0 0x7f01f0069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:21:35.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.562+0000 7f01f5a46700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f006a380 con 0x7f01f00f66e0 2021-11-03T18:21:35.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.562+0000 7f01f5a46700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f006a4c0 con 0x7f01f00f6ff0 2021-11-03T18:21:35.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.562+0000 7f01eeffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01f00f66e0 0x7f01f006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:21:35.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.562+0000 7f01eeffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01f00f66e0 0x7f01f006b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34402/0 (socket says 172.21.15.27:34402) 2021-11-03T18:21:35.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.562+0000 7f01eeffd700 1 -- 172.21.15.27:0/598032728 learned_addr learned my addr 172.21.15.27:0/598032728 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:21:35.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.562+0000 7f01ee7fc700 1 --2- 172.21.15.27:0/598032728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01f00f6ff0 0x7f01f0069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:21:35.568 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.562+0000 7f01eeffd700 1 -- 172.21.15.27:0/598032728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01f00f6ff0 msgr2=0x7f01f0069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:21:35.568 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.562+0000 7f01eeffd700 1 --2- 172.21.15.27:0/598032728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01f00f6ff0 0x7f01f0069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:35.568 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.562+0000 7f01eeffd700 1 -- 172.21.15.27:0/598032728 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e0005040 con 0x7f01f00f66e0 2021-11-03T18:21:35.569 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.563+0000 7f01eeffd700 1 --2- 172.21.15.27:0/598032728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01f00f66e0 0x7f01f006b9a0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 rx=0x7f01e0009990 tx=0x7f01e0009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:21:35.569 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.563+0000 7f01d77fe700 1 -- 172.21.15.27:0/598032728 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01e0018070 con 0x7f01f00f66e0 2021-11-03T18:21:35.570 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.563+0000 7f01d77fe700 1 -- 172.21.15.27:0/598032728 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f01e0007ba0 con 0x7f01f00f66e0 2021-11-03T18:21:35.570 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.563+0000 7f01f5a46700 1 -- 172.21.15.27:0/598032728 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f006a740 con 0x7f01f00f66e0 2021-11-03T18:21:35.570 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.563+0000 7f01d77fe700 1 -- 172.21.15.27:0/598032728 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01e001c3c0 con 0x7f01f00f66e0 2021-11-03T18:21:35.571 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.563+0000 7f01f5a46700 1 -- 172.21.15.27:0/598032728 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f006aca0 con 0x7f01f00f66e0 2021-11-03T18:21:35.571 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.564+0000 7f01f5a46700 1 -- 172.21.15.27:0/598032728 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01f0045ad0 con 0x7f01f00f66e0 2021-11-03T18:21:35.572 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.564+0000 7f01d77fe700 1 -- 172.21.15.27:0/598032728 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f01e001d420 con 0x7f01f00f66e0 2021-11-03T18:21:35.572 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.565+0000 7f01d77fe700 1 --2- 172.21.15.27:0/598032728 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f01d80640b0 0x7f01d8066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:21:35.572 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.565+0000 7f01d77fe700 1 -- 172.21.15.27:0/598032728 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f01e0082560 con 0x7f01f00f66e0 2021-11-03T18:21:35.572 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.565+0000 7f01ee7fc700 1 --2- 172.21.15.27:0/598032728 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f01d80640b0 0x7f01d8066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:21:35.573 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.565+0000 7f01ee7fc700 1 --2- 172.21.15.27:0/598032728 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f01d80640b0 0x7f01d8066570 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f01e4009760 tx=0x7f01e4009c60).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:21:35.573 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.569+0000 7f01d77fe700 1 -- 172.21.15.27:0/598032728 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f01e00548c0 con 0x7f01f00f66e0 2021-11-03T18:21:35.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3937.454143 secs 2021-11-03T18:21:35.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.746+0000 7f01f5a46700 1 -- 172.21.15.27:0/598032728 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f01f00ff010 con 0x7f01d80640b0 2021-11-03T18:21:35.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.748+0000 7f01d77fe700 1 -- 172.21.15.27:0/598032728 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f01f00ff010 con 0x7f01d80640b0 2021-11-03T18:21:35.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.750+0000 7f01f5a46700 1 -- 172.21.15.27:0/598032728 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f01d80640b0 msgr2=0x7f01d8066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:21:35.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.750+0000 7f01f5a46700 1 --2- 172.21.15.27:0/598032728 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f01d80640b0 0x7f01d8066570 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7f01e4009760 tx=0x7f01e4009c60).stop 2021-11-03T18:21:35.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.750+0000 7f01f5a46700 1 -- 172.21.15.27:0/598032728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01f00f66e0 msgr2=0x7f01f006b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:21:35.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.750+0000 7f01f5a46700 1 --2- 172.21.15.27:0/598032728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01f00f66e0 0x7f01f006b9a0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 rx=0x7f01e0009990 tx=0x7f01e0009b30).stop 2021-11-03T18:21:35.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.750+0000 7f01f5a46700 1 -- 172.21.15.27:0/598032728 shutdown_connections 2021-11-03T18:21:35.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.750+0000 7f01f5a46700 1 --2- 172.21.15.27:0/598032728 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f01d80640b0 0x7f01d8066570 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:35.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.750+0000 7f01f5a46700 1 --2- 172.21.15.27:0/598032728 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f01f00f66e0 0x7f01f006b9a0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:35.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.750+0000 7f01f5a46700 1 --2- 172.21.15.27:0/598032728 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f01f00f6ff0 0x7f01f0069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:35.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.750+0000 7f01f5a46700 1 -- 172.21.15.27:0/598032728 >> 172.21.15.27:0/598032728 conn(0x7f01f00f1f00 msgr2=0x7f01f00fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:21:35.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.750+0000 7f01f5a46700 1 -- 172.21.15.27:0/598032728 shutdown_connections 2021-11-03T18:21:35.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.750+0000 7f01f5a46700 1 -- 172.21.15.27:0/598032728 wait complete. 2021-11-03T18:21:35.759 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:21:35.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3937.454143 secs 2021-11-03T18:21:35.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.850+0000 7f3cfc0b2700 1 -- 172.21.15.27:0/3356557497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cf40f5bb0 msgr2=0x7f3cf40f5fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:21:35.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.850+0000 7f3cfc0b2700 1 --2- 172.21.15.27:0/3356557497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cf40f5bb0 0x7f3cf40f5fd0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 rx=0x7f3cec0099e0 tx=0x7f3cec004d10).stop 2021-11-03T18:21:35.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.850+0000 7f3cfc0b2700 1 -- 172.21.15.27:0/3356557497 shutdown_connections 2021-11-03T18:21:35.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.850+0000 7f3cfc0b2700 1 --2- 172.21.15.27:0/3356557497 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3cf40f6550 0x7f3cf40f69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:35.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.850+0000 7f3cfc0b2700 1 --2- 172.21.15.27:0/3356557497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cf40f5bb0 0x7f3cf40f5fd0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:35.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.850+0000 7f3cfc0b2700 1 -- 172.21.15.27:0/3356557497 >> 172.21.15.27:0/3356557497 conn(0x7f3cf40f04c0 msgr2=0x7f3cf40f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:21:35.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.850+0000 7f3cfc0b2700 1 -- 172.21.15.27:0/3356557497 shutdown_connections 2021-11-03T18:21:35.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.850+0000 7f3cfc0b2700 1 -- 172.21.15.27:0/3356557497 wait complete. 2021-11-03T18:21:35.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.850+0000 7f3cfc0b2700 1 Processor -- start 2021-11-03T18:21:35.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.850+0000 7f3cfc0b2700 1 -- start start 2021-11-03T18:21:35.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.850+0000 7f3cfc0b2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cf40f5bb0 0x7f3cf4109d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:21:35.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.850+0000 7f3cfc0b2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3cf40f6550 0x7f3cf410c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:21:35.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.850+0000 7f3cfc0b2700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf40f56a0 con 0x7f3cf40f5bb0 2021-11-03T18:21:35.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.850+0000 7f3cfc0b2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf40f57e0 con 0x7f3cf40f6550 2021-11-03T18:21:35.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.851+0000 7f3cf9e4e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cf40f5bb0 0x7f3cf4109d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:21:35.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.851+0000 7f3cf9e4e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cf40f5bb0 0x7f3cf4109d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34412/0 (socket says 172.21.15.27:34412) 2021-11-03T18:21:35.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.851+0000 7f3cf9e4e700 1 -- 172.21.15.27:0/3549273877 learned_addr learned my addr 172.21.15.27:0/3549273877 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:21:35.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.851+0000 7f3cf964d700 1 --2- 172.21.15.27:0/3549273877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3cf40f6550 0x7f3cf410c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:21:35.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.851+0000 7f3cf9e4e700 1 -- 172.21.15.27:0/3549273877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3cf40f6550 msgr2=0x7f3cf410c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:21:35.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.851+0000 7f3cf9e4e700 1 --2- 172.21.15.27:0/3549273877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3cf40f6550 0x7f3cf410c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:35.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.851+0000 7f3cf9e4e700 1 -- 172.21.15.27:0/3549273877 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cec009640 con 0x7f3cf40f5bb0 2021-11-03T18:21:35.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.851+0000 7f3cf9e4e700 1 --2- 172.21.15.27:0/3549273877 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cf40f5bb0 0x7f3cf4109d80 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 rx=0x7f3cec00c010 tx=0x7f3cec004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:21:35.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.851+0000 7f3cf27fc700 1 -- 172.21.15.27:0/3549273877 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cec00e070 con 0x7f3cf40f5bb0 2021-11-03T18:21:35.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.851+0000 7f3cf27fc700 1 -- 172.21.15.27:0/3549273877 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f3cec007dc0 con 0x7f3cf40f5bb0 2021-11-03T18:21:35.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.851+0000 7f3cfc0b2700 1 -- 172.21.15.27:0/3549273877 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cf410c7d0 con 0x7f3cf40f5bb0 2021-11-03T18:21:35.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.851+0000 7f3cf27fc700 1 -- 172.21.15.27:0/3549273877 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cec018070 con 0x7f3cf40f5bb0 2021-11-03T18:21:35.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.851+0000 7f3cfc0b2700 1 -- 172.21.15.27:0/3549273877 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cf410ccb0 con 0x7f3cf40f5bb0 2021-11-03T18:21:35.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.852+0000 7f3cfc0b2700 1 -- 172.21.15.27:0/3549273877 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cf4045ad0 con 0x7f3cf40f5bb0 2021-11-03T18:21:35.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.853+0000 7f3cf27fc700 1 -- 172.21.15.27:0/3549273877 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f3cec024060 con 0x7f3cf40f5bb0 2021-11-03T18:21:35.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.853+0000 7f3cf27fc700 1 --2- 172.21.15.27:0/3549273877 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3cdc0640b0 0x7f3cdc066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:21:35.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.853+0000 7f3cf27fc700 1 -- 172.21.15.27:0/3549273877 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f3cec082330 con 0x7f3cf40f5bb0 2021-11-03T18:21:35.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.853+0000 7f3cf964d700 1 --2- 172.21.15.27:0/3549273877 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3cdc0640b0 0x7f3cdc066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:21:35.860 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.853+0000 7f3cf964d700 1 --2- 172.21.15.27:0/3549273877 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3cdc0640b0 0x7f3cdc066570 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f3ce80041d0 tx=0x7f3ce8009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:21:35.860 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:35.857+0000 7f3cf27fc700 1 -- 172.21.15.27:0/3549273877 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f3cec054630 con 0x7f3cf40f5bb0 2021-11-03T18:21:36.030 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.031+0000 7f3cfc0b2700 1 -- 172.21.15.27:0/3549273877 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3cf40fde80 con 0x7f3cdc0640b0 2021-11-03T18:21:36.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.046+0000 7f3cf27fc700 1 -- 172.21.15.27:0/3549273877 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f3cf40fde80 con 0x7f3cdc0640b0 2021-11-03T18:21:36.045 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:21:36.045 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (73m) 2m ago 74m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:21:36.045 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (64m) 2m ago 74m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:21:36.046 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (64m) 85s ago 73m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:21:36.046 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (72m) 2m ago 74m 42.4M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:21:36.046 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (66m) 2m ago 69m 26.5M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:21:36.046 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (69m) 2m ago 69m 3849M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:21:36.047 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (65m) 85s ago 69m 26.1M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:21:36.047 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (67m) 85s ago 69m 27.2M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:21:36.047 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (66m) 2m ago 75m 590M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:21:36.047 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (66m) 85s ago 73m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:21:36.047 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (65m) 2m ago 75m 312M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:21:36.048 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (65m) 85s ago 73m 145M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:21:36.048 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (74m) 2m ago 74m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:21:36.048 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (73m) 85s ago 73m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:21:36.048 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (64m) 2m ago 72m 392M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:21:36.049 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (63m) 2m ago 72m 427M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:21:36.049 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (63m) 2m ago 72m 320M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:21:36.049 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (63m) 85s ago 71m 454M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:21:36.049 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (62m) 85s ago 71m 359M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:21:36.050 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (62m) 85s ago 71m 368M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:21:36.050 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (72m) 2m ago 73m 59.9M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:21:36.050 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.048+0000 7f3cfc0b2700 1 -- 172.21.15.27:0/3549273877 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3cdc0640b0 msgr2=0x7f3cdc066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:21:36.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.048+0000 7f3cfc0b2700 1 --2- 172.21.15.27:0/3549273877 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3cdc0640b0 0x7f3cdc066570 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f3ce80041d0 tx=0x7f3ce8009040).stop 2021-11-03T18:21:36.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.048+0000 7f3cfc0b2700 1 -- 172.21.15.27:0/3549273877 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cf40f5bb0 msgr2=0x7f3cf4109d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:21:36.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.048+0000 7f3cfc0b2700 1 --2- 172.21.15.27:0/3549273877 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cf40f5bb0 0x7f3cf4109d80 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 rx=0x7f3cec00c010 tx=0x7f3cec004d10).stop 2021-11-03T18:21:36.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.048+0000 7f3cfc0b2700 1 -- 172.21.15.27:0/3549273877 shutdown_connections 2021-11-03T18:21:36.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.048+0000 7f3cfc0b2700 1 --2- 172.21.15.27:0/3549273877 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3cdc0640b0 0x7f3cdc066570 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:36.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.048+0000 7f3cfc0b2700 1 --2- 172.21.15.27:0/3549273877 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3cf40f5bb0 0x7f3cf4109d80 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:36.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.048+0000 7f3cfc0b2700 1 --2- 172.21.15.27:0/3549273877 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3cf40f6550 0x7f3cf410c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:36.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.048+0000 7f3cfc0b2700 1 -- 172.21.15.27:0/3549273877 >> 172.21.15.27:0/3549273877 conn(0x7f3cf40f04c0 msgr2=0x7f3cf40fa450 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:21:36.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.048+0000 7f3cfc0b2700 1 -- 172.21.15.27:0/3549273877 shutdown_connections 2021-11-03T18:21:36.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.048+0000 7f3cfc0b2700 1 -- 172.21.15.27:0/3549273877 wait complete. 2021-11-03T18:21:36.147 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.146+0000 7f739d823700 1 -- 172.21.15.27:0/365258752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73980f55e0 msgr2=0x7f73980f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:21:36.147 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.146+0000 7f739d823700 1 --2- 172.21.15.27:0/365258752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73980f55e0 0x7f73980f5a00 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 rx=0x7f7388004660 tx=0x7f7388009b30).stop 2021-11-03T18:21:36.147 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.146+0000 7f739d823700 1 -- 172.21.15.27:0/365258752 shutdown_connections 2021-11-03T18:21:36.148 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.146+0000 7f739d823700 1 --2- 172.21.15.27:0/365258752 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73980f5ef0 0x7f73980f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:36.148 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.146+0000 7f739d823700 1 --2- 172.21.15.27:0/365258752 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73980f55e0 0x7f73980f5a00 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:36.148 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.146+0000 7f739d823700 1 -- 172.21.15.27:0/365258752 >> 172.21.15.27:0/365258752 conn(0x7f73980f0de0 msgr2=0x7f73980f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:21:36.148 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.146+0000 7f739d823700 1 -- 172.21.15.27:0/365258752 shutdown_connections 2021-11-03T18:21:36.148 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.146+0000 7f739d823700 1 -- 172.21.15.27:0/365258752 wait complete. 2021-11-03T18:21:36.149 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.147+0000 7f739d823700 1 Processor -- start 2021-11-03T18:21:36.149 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.147+0000 7f739d823700 1 -- start start 2021-11-03T18:21:36.149 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.147+0000 7f739d823700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73980f55e0 0x7f7398109cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:21:36.149 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.147+0000 7f739d823700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73980f5ef0 0x7f739810c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:21:36.150 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.147+0000 7f739d823700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73980604c0 con 0x7f73980f55e0 2021-11-03T18:21:36.150 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.147+0000 7f739d823700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7398060600 con 0x7f73980f5ef0 2021-11-03T18:21:36.150 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.147+0000 7f7396ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73980f55e0 0x7f7398109cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:21:36.150 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.147+0000 7f7396ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73980f55e0 0x7f7398109cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34422/0 (socket says 172.21.15.27:34422) 2021-11-03T18:21:36.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.147+0000 7f7396ffd700 1 -- 172.21.15.27:0/454169445 learned_addr learned my addr 172.21.15.27:0/454169445 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:21:36.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.148+0000 7f73967fc700 1 --2- 172.21.15.27:0/454169445 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73980f5ef0 0x7f739810c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:21:36.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.148+0000 7f7396ffd700 1 -- 172.21.15.27:0/454169445 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73980f5ef0 msgr2=0x7f739810c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:21:36.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.148+0000 7f7396ffd700 1 --2- 172.21.15.27:0/454169445 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73980f5ef0 0x7f739810c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:36.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.148+0000 7f7396ffd700 1 -- 172.21.15.27:0/454169445 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7388005040 con 0x7f73980f55e0 2021-11-03T18:21:36.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.148+0000 7f7396ffd700 1 --2- 172.21.15.27:0/454169445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73980f55e0 0x7f7398109cb0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 rx=0x7f7388009990 tx=0x7f7388009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:21:36.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.148+0000 7f737f7fe700 1 -- 172.21.15.27:0/454169445 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7388018070 con 0x7f73980f55e0 2021-11-03T18:21:36.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.148+0000 7f737f7fe700 1 -- 172.21.15.27:0/454169445 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f7388007ba0 con 0x7f73980f55e0 2021-11-03T18:21:36.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.148+0000 7f739d823700 1 -- 172.21.15.27:0/454169445 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f739810c700 con 0x7f73980f55e0 2021-11-03T18:21:36.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.148+0000 7f737f7fe700 1 -- 172.21.15.27:0/454169445 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f738801c3c0 con 0x7f73980f55e0 2021-11-03T18:21:36.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.148+0000 7f739d823700 1 -- 172.21.15.27:0/454169445 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7398105b10 con 0x7f73980f55e0 2021-11-03T18:21:36.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.149+0000 7f739d823700 1 -- 172.21.15.27:0/454169445 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73980f4a70 con 0x7f73980f55e0 2021-11-03T18:21:36.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.150+0000 7f737f7fe700 1 -- 172.21.15.27:0/454169445 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f738801d420 con 0x7f73980f55e0 2021-11-03T18:21:36.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.150+0000 7f737f7fe700 1 --2- 172.21.15.27:0/454169445 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f73800684c0 0x7f738006a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:21:36.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.150+0000 7f737f7fe700 1 -- 172.21.15.27:0/454169445 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f73880824a0 con 0x7f73980f55e0 2021-11-03T18:21:36.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.150+0000 7f73967fc700 1 --2- 172.21.15.27:0/454169445 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f73800684c0 0x7f738006a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:21:36.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.150+0000 7f73967fc700 1 --2- 172.21.15.27:0/454169445 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f73800684c0 0x7f738006a980 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7f738c004c40 tx=0x7f738c00b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:21:36.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.154+0000 7f737f7fe700 1 -- 172.21.15.27:0/454169445 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f7388054860 con 0x7f73980f55e0 2021-11-03T18:21:36.386 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.385+0000 7f739d823700 1 -- 172.21.15.27:0/454169445 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7398045ad0 con 0x7f73980f55e0 2021-11-03T18:21:36.387 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.385+0000 7f737f7fe700 1 -- 172.21.15.27:0/454169445 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f7388054450 con 0x7f73980f55e0 2021-11-03T18:21:36.387 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:21:36.390 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:21:36.391 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:21:36.391 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:21:36.391 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:21:36.392 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:21:36.392 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:21:36.393 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:21:36.393 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:21:36.394 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:21:36.394 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:21:36.395 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:21:36.395 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:21:36.396 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:21:36.396 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:21:36.397 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:21:36.397 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:21:36.397 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:21:36.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.387+0000 7f739d823700 1 -- 172.21.15.27:0/454169445 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f73800684c0 msgr2=0x7f738006a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:21:36.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.387+0000 7f739d823700 1 --2- 172.21.15.27:0/454169445 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f73800684c0 0x7f738006a980 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7f738c004c40 tx=0x7f738c00b040).stop 2021-11-03T18:21:36.400 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.387+0000 7f739d823700 1 -- 172.21.15.27:0/454169445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73980f55e0 msgr2=0x7f7398109cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:21:36.400 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.387+0000 7f739d823700 1 --2- 172.21.15.27:0/454169445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73980f55e0 0x7f7398109cb0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 rx=0x7f7388009990 tx=0x7f7388009b30).stop 2021-11-03T18:21:36.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.387+0000 7f739d823700 1 -- 172.21.15.27:0/454169445 shutdown_connections 2021-11-03T18:21:36.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.387+0000 7f739d823700 1 --2- 172.21.15.27:0/454169445 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f73800684c0 0x7f738006a980 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:36.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.387+0000 7f739d823700 1 --2- 172.21.15.27:0/454169445 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73980f55e0 0x7f7398109cb0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:36.402 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.387+0000 7f739d823700 1 --2- 172.21.15.27:0/454169445 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73980f5ef0 0x7f739810c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:36.402 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.387+0000 7f739d823700 1 -- 172.21.15.27:0/454169445 >> 172.21.15.27:0/454169445 conn(0x7f73980f0de0 msgr2=0x7f73980fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:21:36.402 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.388+0000 7f739d823700 1 -- 172.21.15.27:0/454169445 shutdown_connections 2021-11-03T18:21:36.402 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.388+0000 7f739d823700 1 -- 172.21.15.27:0/454169445 wait complete. 2021-11-03T18:21:36.488 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.488+0000 7f9c60d4b700 1 -- 172.21.15.27:0/863119090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c5c0f66e0 msgr2=0x7f9c5c0f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:21:36.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.488+0000 7f9c60d4b700 1 --2- 172.21.15.27:0/863119090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c5c0f66e0 0x7f9c5c0f6b00 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 rx=0x7f9c50004660 tx=0x7f9c50009b30).stop 2021-11-03T18:21:36.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.488+0000 7f9c60d4b700 1 -- 172.21.15.27:0/863119090 shutdown_connections 2021-11-03T18:21:36.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.488+0000 7f9c60d4b700 1 --2- 172.21.15.27:0/863119090 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c5c0f7980 0x7f9c5c0f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:36.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.488+0000 7f9c60d4b700 1 --2- 172.21.15.27:0/863119090 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c5c0f66e0 0x7f9c5c0f6b00 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:36.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.488+0000 7f9c60d4b700 1 -- 172.21.15.27:0/863119090 >> 172.21.15.27:0/863119090 conn(0x7f9c5c0f1e40 msgr2=0x7f9c5c0f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:21:36.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.488+0000 7f9c60d4b700 1 -- 172.21.15.27:0/863119090 shutdown_connections 2021-11-03T18:21:36.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.488+0000 7f9c60d4b700 1 -- 172.21.15.27:0/863119090 wait complete. 2021-11-03T18:21:36.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.488+0000 7f9c60d4b700 1 Processor -- start 2021-11-03T18:21:36.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.489+0000 7f9c60d4b700 1 -- start start 2021-11-03T18:21:36.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.489+0000 7f9c60d4b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c5c0f66e0 0x7f9c5c106ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:21:36.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.489+0000 7f9c60d4b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c5c0f7980 0x7f9c5c1074e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:21:36.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.489+0000 7f9c60d4b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c107a90 con 0x7f9c5c0f7980 2021-11-03T18:21:36.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.489+0000 7f9c60d4b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c5c107bd0 con 0x7f9c5c0f66e0 2021-11-03T18:21:36.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.489+0000 7f9c59d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c5c0f7980 0x7f9c5c1074e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:21:36.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.489+0000 7f9c59d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c5c0f7980 0x7f9c5c1074e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34432/0 (socket says 172.21.15.27:34432) 2021-11-03T18:21:36.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.489+0000 7f9c59d9b700 1 -- 172.21.15.27:0/626390886 learned_addr learned my addr 172.21.15.27:0/626390886 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:21:36.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.489+0000 7f9c5a59c700 1 --2- 172.21.15.27:0/626390886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c5c0f66e0 0x7f9c5c106ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:21:36.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.489+0000 7f9c59d9b700 1 -- 172.21.15.27:0/626390886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c5c0f66e0 msgr2=0x7f9c5c106ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:21:36.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.489+0000 7f9c59d9b700 1 --2- 172.21.15.27:0/626390886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c5c0f66e0 0x7f9c5c106ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:36.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.489+0000 7f9c59d9b700 1 -- 172.21.15.27:0/626390886 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c50005040 con 0x7f9c5c0f7980 2021-11-03T18:21:36.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.489+0000 7f9c59d9b700 1 --2- 172.21.15.27:0/626390886 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c5c0f7980 0x7f9c5c1074e0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 rx=0x7f9c4c00c510 tx=0x7f9c4c007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:21:36.494 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.489+0000 7f9c4affd700 1 -- 172.21.15.27:0/626390886 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c4c00f650 con 0x7f9c5c0f7980 2021-11-03T18:21:36.494 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.489+0000 7f9c4affd700 1 -- 172.21.15.27:0/626390886 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f9c4c00fc60 con 0x7f9c5c0f7980 2021-11-03T18:21:36.494 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.489+0000 7f9c60d4b700 1 -- 172.21.15.27:0/626390886 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c5c108570 con 0x7f9c5c0f7980 2021-11-03T18:21:36.494 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.489+0000 7f9c4affd700 1 -- 172.21.15.27:0/626390886 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9c4c012d00 con 0x7f9c5c0f7980 2021-11-03T18:21:36.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.490+0000 7f9c60d4b700 1 -- 172.21.15.27:0/626390886 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c5c108b30 con 0x7f9c5c0f7980 2021-11-03T18:21:36.496 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.490+0000 7f9c60d4b700 1 -- 172.21.15.27:0/626390886 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c5c045ad0 con 0x7f9c5c0f7980 2021-11-03T18:21:36.496 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.491+0000 7f9c4affd700 1 -- 172.21.15.27:0/626390886 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f9c4c010070 con 0x7f9c5c0f7980 2021-11-03T18:21:36.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.491+0000 7f9c4affd700 1 --2- 172.21.15.27:0/626390886 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9c440640b0 0x7f9c44066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:21:36.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.491+0000 7f9c4affd700 1 -- 172.21.15.27:0/626390886 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f9c4c080800 con 0x7f9c5c0f7980 2021-11-03T18:21:36.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.491+0000 7f9c5a59c700 1 --2- 172.21.15.27:0/626390886 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9c440640b0 0x7f9c44066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:21:36.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.492+0000 7f9c5a59c700 1 --2- 172.21.15.27:0/626390886 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9c440640b0 0x7f9c44066570 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7f9c50009da0 tx=0x7f9c50009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:21:36.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.495+0000 7f9c4affd700 1 -- 172.21.15.27:0/626390886 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f9c4c052a10 con 0x7f9c5c0f7980 2021-11-03T18:21:36.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.701+0000 7f9c60d4b700 1 -- 172.21.15.27:0/626390886 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9c5c027a00 con 0x7f9c5c0f7980 2021-11-03T18:21:36.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.702+0000 7f9c4affd700 1 -- 172.21.15.27:0/626390886 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f9c4c052600 con 0x7f9c5c0f7980 2021-11-03T18:21:36.701 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:21:36.702 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:21:36.702 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:21:36.702 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:21:36.702 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:21:36.703 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:21:36.703 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:21:36.703 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:21:36.703 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:21:36.704 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:21:36.704 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:21:36.704 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:21:36.704 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:21:36.705 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:21:36.705 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:21:36.705 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:21:36.705 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:21:36.705 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:21:36.706 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:21:36.706 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:21:36.706 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:21:36.706 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:21:36.707 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:21:36.707 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:21:36.707 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:21:36.707 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:21:36.708 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:21:36.708 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:21:36.708 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:21:36.708 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:21:36.709 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:21:36.709 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:21:36.709 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:21:36.709 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:21:36.709 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:21:36.710 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:21:36.710 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:21:36.710 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:21:36.710 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:21:36.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.704+0000 7f9c60d4b700 1 -- 172.21.15.27:0/626390886 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9c440640b0 msgr2=0x7f9c44066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:21:36.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.704+0000 7f9c60d4b700 1 --2- 172.21.15.27:0/626390886 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9c440640b0 0x7f9c44066570 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 rx=0x7f9c50009da0 tx=0x7f9c50009b30).stop 2021-11-03T18:21:36.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.704+0000 7f9c60d4b700 1 -- 172.21.15.27:0/626390886 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c5c0f7980 msgr2=0x7f9c5c1074e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:21:36.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.704+0000 7f9c60d4b700 1 --2- 172.21.15.27:0/626390886 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c5c0f7980 0x7f9c5c1074e0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 rx=0x7f9c4c00c510 tx=0x7f9c4c007e00).stop 2021-11-03T18:21:36.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.704+0000 7f9c60d4b700 1 -- 172.21.15.27:0/626390886 shutdown_connections 2021-11-03T18:21:36.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.704+0000 7f9c60d4b700 1 --2- 172.21.15.27:0/626390886 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9c440640b0 0x7f9c44066570 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:36.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.704+0000 7f9c60d4b700 1 --2- 172.21.15.27:0/626390886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9c5c0f66e0 0x7f9c5c106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:36.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.704+0000 7f9c60d4b700 1 --2- 172.21.15.27:0/626390886 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9c5c0f7980 0x7f9c5c1074e0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:21:36.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.704+0000 7f9c60d4b700 1 -- 172.21.15.27:0/626390886 >> 172.21.15.27:0/626390886 conn(0x7f9c5c0f1e40 msgr2=0x7f9c5c0fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:21:36.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.704+0000 7f9c60d4b700 1 -- 172.21.15.27:0/626390886 shutdown_connections 2021-11-03T18:21:36.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:21:36.704+0000 7f9c60d4b700 1 -- 172.21.15.27:0/626390886 wait complete. 2021-11-03T18:21:36.714 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:21:36.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:36 smithi115 ceph-mon[65453]: pgmap v2008: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:36.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:36 smithi027 ceph-mon[97244]: pgmap v2008: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:37.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:37 smithi115 ceph-mon[65453]: from='client.36168 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:21:37.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:37 smithi115 ceph-mon[65453]: from='client.36172 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:21:37.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:37 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/454169445' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:37.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:37 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/626390886' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:21:37.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:37 smithi027 ceph-mon[97244]: from='client.36168 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:21:37.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:37 smithi027 ceph-mon[97244]: from='client.36172 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:21:37.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:37 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/454169445' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:37.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:37 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/626390886' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:21:38.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:38 smithi115 ceph-mon[65453]: pgmap v2009: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:38.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:38 smithi027 ceph-mon[97244]: pgmap v2009: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:40.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:40 smithi115 ceph-mon[65453]: pgmap v2010: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:40.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3942.454286 secs 2021-11-03T18:21:40.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:40 smithi027 ceph-mon[97244]: pgmap v2010: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:40.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3942.454286 secs 2021-11-03T18:21:42.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:42 smithi115 ceph-mon[65453]: pgmap v2011: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:42.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:21:42.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:21:42.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:21:42.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:42.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:42.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:42.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:42.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:42 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:21:42.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:42 smithi027 ceph-mon[97244]: pgmap v2011: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:42.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:21:42.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:21:42.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:21:42.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:42.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:42.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:42.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:42.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:42 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:21:44.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:44 smithi115 ceph-mon[65453]: pgmap v2012: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:44.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:44 smithi027 ceph-mon[97244]: pgmap v2012: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:45.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3947.454432 secs 2021-11-03T18:21:45.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3947.454432 secs 2021-11-03T18:21:46.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:46 smithi115 ceph-mon[65453]: pgmap v2013: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:46.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:46 smithi027 ceph-mon[97244]: pgmap v2013: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:48.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:48 smithi115 ceph-mon[65453]: pgmap v2014: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:48.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:48 smithi027 ceph-mon[97244]: pgmap v2014: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:50.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:50 smithi115 ceph-mon[65453]: pgmap v2015: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:50.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3952.454556 secs 2021-11-03T18:21:50.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:50 smithi027 ceph-mon[97244]: pgmap v2015: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:50.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3952.454556 secs 2021-11-03T18:21:52.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:52 smithi115 ceph-mon[65453]: pgmap v2016: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:52.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:21:52.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:21:52.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:21:52.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:52.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:52.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:52.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:52.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:52 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:21:52.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:52 smithi027 ceph-mon[97244]: pgmap v2016: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:52.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:21:52.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:21:52.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:21:52.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:52.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:52.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:52.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:21:52.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:52 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:21:54.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:54 smithi115 ceph-mon[65453]: pgmap v2017: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:54.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:54 smithi027 ceph-mon[97244]: pgmap v2017: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:55.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3957.454686 secs 2021-11-03T18:21:55.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3957.454686 secs 2021-11-03T18:21:56.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:56 smithi115 ceph-mon[65453]: pgmap v2018: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:56.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:21:56.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:21:56.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:56 smithi027 ceph-mon[97244]: pgmap v2018: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:56.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:21:56.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:21:58.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:21:58 smithi115 ceph-mon[65453]: pgmap v2019: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:21:58.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:21:58 smithi027 ceph-mon[97244]: pgmap v2019: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:00.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:00 smithi115 ceph-mon[65453]: pgmap v2020: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:00.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3962.454814 secs 2021-11-03T18:22:00.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:00 smithi027 ceph-mon[97244]: pgmap v2020: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:00.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3962.454814 secs 2021-11-03T18:22:02.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:02 smithi115 ceph-mon[65453]: pgmap v2021: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:02.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:22:02.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:22:02.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:22:02.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:02.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:02.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:02.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:02.741 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:02 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:22:02.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:02 smithi027 ceph-mon[97244]: pgmap v2021: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:02.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:22:02.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:22:02.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:22:02.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:02.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:02.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:02.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:02.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:02 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:22:04.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:04 smithi115 ceph-mon[65453]: pgmap v2022: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:04.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:04 smithi027 ceph-mon[97244]: pgmap v2022: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:05.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3967.454935 secs 2021-11-03T18:22:05.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3967.454935 secs 2021-11-03T18:22:06.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:06 smithi115 ceph-mon[65453]: pgmap v2023: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:06.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:06 smithi027 ceph-mon[97244]: pgmap v2023: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:06.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:06.818+0000 7fcfb365e700 1 -- 172.21.15.27:0/2465330186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfac06b9c0 msgr2=0x7fcfac069ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:22:06.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:06.818+0000 7fcfb365e700 1 --2- 172.21.15.27:0/2465330186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfac06b9c0 0x7fcfac069ec0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 rx=0x7fcfa4004660 tx=0x7fcfa4009b30).stop 2021-11-03T18:22:06.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:06.819+0000 7fcfb365e700 1 -- 172.21.15.27:0/2465330186 shutdown_connections 2021-11-03T18:22:06.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:06.819+0000 7fcfb365e700 1 --2- 172.21.15.27:0/2465330186 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfac06a3b0 0x7fcfac06a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:06.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:06.819+0000 7fcfb365e700 1 --2- 172.21.15.27:0/2465330186 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfac06b9c0 0x7fcfac069ec0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:06.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:06.819+0000 7fcfb365e700 1 -- 172.21.15.27:0/2465330186 >> 172.21.15.27:0/2465330186 conn(0x7fcfac0f1e20 msgr2=0x7fcfac0f4240 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:22:06.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:06.819+0000 7fcfb365e700 1 -- 172.21.15.27:0/2465330186 shutdown_connections 2021-11-03T18:22:06.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:06.819+0000 7fcfb365e700 1 -- 172.21.15.27:0/2465330186 wait complete. 2021-11-03T18:22:06.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:06.819+0000 7fcfb365e700 1 Processor -- start 2021-11-03T18:22:06.823 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:06.819+0000 7fcfb365e700 1 -- start start 2021-11-03T18:22:06.823 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:06.819+0000 7fcfb365e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfac06a3b0 0x7fcfac10d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:22:06.823 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:06.819+0000 7fcfb365e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfac06b9c0 0x7fcfac10da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:22:06.823 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:06.819+0000 7fcfb365e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac10dfd0 con 0x7fcfac06a3b0 2021-11-03T18:22:06.824 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:06.819+0000 7fcfb365e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfac10e110 con 0x7fcfac06b9c0 2021-11-03T18:22:06.824 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:06.820+0000 7fcfb13fa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfac06a3b0 0x7fcfac10d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:22:06.824 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:06.820+0000 7fcfb13fa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfac06a3b0 0x7fcfac10d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34448/0 (socket says 172.21.15.27:34448) 2021-11-03T18:22:06.824 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:06.820+0000 7fcfb13fa700 1 -- 172.21.15.27:0/521527636 learned_addr learned my addr 172.21.15.27:0/521527636 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:22:06.825 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:06.820+0000 7fcfb0bf9700 1 --2- 172.21.15.27:0/521527636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfac06b9c0 0x7fcfac10da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:22:06.825 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:06.820+0000 7fcfb13fa700 1 -- 172.21.15.27:0/521527636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfac06b9c0 msgr2=0x7fcfac10da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:22:06.825 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:06.820+0000 7fcfb13fa700 1 --2- 172.21.15.27:0/521527636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfac06b9c0 0x7fcfac10da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:06.825 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:06.820+0000 7fcfb13fa700 1 -- 172.21.15.27:0/521527636 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfa4005040 con 0x7fcfac06a3b0 2021-11-03T18:22:06.826 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:06.820+0000 7fcfb0bf9700 1 --2- 172.21.15.27:0/521527636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfac06b9c0 0x7fcfac10da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:22:06.826 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:06.820+0000 7fcfb13fa700 1 --2- 172.21.15.27:0/521527636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfac06a3b0 0x7fcfac10d530 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 rx=0x7fcfa4009990 tx=0x7fcfa4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:22:06.826 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:06.820+0000 7fcfa9ffb700 1 -- 172.21.15.27:0/521527636 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfa4018070 con 0x7fcfac06a3b0 2021-11-03T18:22:06.826 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:06.820+0000 7fcfa9ffb700 1 -- 172.21.15.27:0/521527636 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fcfa4007ba0 con 0x7fcfac06a3b0 2021-11-03T18:22:06.827 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:06.820+0000 7fcfb365e700 1 -- 172.21.15.27:0/521527636 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfac10ea50 con 0x7fcfac06a3b0 2021-11-03T18:22:06.827 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:06.820+0000 7fcfa9ffb700 1 -- 172.21.15.27:0/521527636 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcfa401c3c0 con 0x7fcfac06a3b0 2021-11-03T18:22:06.827 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:06.820+0000 7fcfb365e700 1 -- 172.21.15.27:0/521527636 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfac10efb0 con 0x7fcfac06a3b0 2021-11-03T18:22:06.827 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:06.821+0000 7fcfb365e700 1 -- 172.21.15.27:0/521527636 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfac045ad0 con 0x7fcfac06a3b0 2021-11-03T18:22:06.828 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:06.822+0000 7fcfa9ffb700 1 -- 172.21.15.27:0/521527636 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fcfa401d420 con 0x7fcfac06a3b0 2021-11-03T18:22:06.828 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:06.822+0000 7fcfa9ffb700 1 --2- 172.21.15.27:0/521527636 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcf940640b0 0x7fcf94066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:22:06.828 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:06.822+0000 7fcfa9ffb700 1 -- 172.21.15.27:0/521527636 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fcfa4082490 con 0x7fcfac06a3b0 2021-11-03T18:22:06.828 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:06.822+0000 7fcfb0bf9700 1 --2- 172.21.15.27:0/521527636 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcf940640b0 0x7fcf94066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:22:06.829 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:06.823+0000 7fcfb0bf9700 1 --2- 172.21.15.27:0/521527636 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcf940640b0 0x7fcf94066570 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7fcfa0009740 tx=0x7fcfa0009be0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:22:06.829 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:06.827+0000 7fcfa9ffb700 1 -- 172.21.15.27:0/521527636 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fcfa40547f0 con 0x7fcfac06a3b0 2021-11-03T18:22:07.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.006+0000 7fcfb365e700 1 -- 172.21.15.27:0/521527636 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcfac105580 con 0x7fcf940640b0 2021-11-03T18:22:07.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.008+0000 7fcfa9ffb700 1 -- 172.21.15.27:0/521527636 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fcfac105580 con 0x7fcf940640b0 2021-11-03T18:22:07.009 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.009+0000 7fcfb365e700 1 -- 172.21.15.27:0/521527636 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcf940640b0 msgr2=0x7fcf94066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:22:07.009 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.009+0000 7fcfb365e700 1 --2- 172.21.15.27:0/521527636 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcf940640b0 0x7fcf94066570 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7fcfa0009740 tx=0x7fcfa0009be0).stop 2021-11-03T18:22:07.010 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.009+0000 7fcfb365e700 1 -- 172.21.15.27:0/521527636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfac06a3b0 msgr2=0x7fcfac10d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:22:07.010 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.009+0000 7fcfb365e700 1 --2- 172.21.15.27:0/521527636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfac06a3b0 0x7fcfac10d530 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 rx=0x7fcfa4009990 tx=0x7fcfa4009b30).stop 2021-11-03T18:22:07.010 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.010+0000 7fcfb365e700 1 -- 172.21.15.27:0/521527636 shutdown_connections 2021-11-03T18:22:07.011 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.010+0000 7fcfb365e700 1 --2- 172.21.15.27:0/521527636 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcf940640b0 0x7fcf94066570 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:07.011 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.010+0000 7fcfb365e700 1 --2- 172.21.15.27:0/521527636 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcfac06a3b0 0x7fcfac10d530 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:07.011 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.010+0000 7fcfb365e700 1 --2- 172.21.15.27:0/521527636 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcfac06b9c0 0x7fcfac10da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:07.011 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.010+0000 7fcfb365e700 1 -- 172.21.15.27:0/521527636 >> 172.21.15.27:0/521527636 conn(0x7fcfac0f1e20 msgr2=0x7fcfac0f89b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:22:07.012 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.010+0000 7fcfb365e700 1 -- 172.21.15.27:0/521527636 shutdown_connections 2021-11-03T18:22:07.012 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.010+0000 7fcfb365e700 1 -- 172.21.15.27:0/521527636 wait complete. 2021-11-03T18:22:07.018 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:22:07.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.110+0000 7f71cb705700 1 -- 172.21.15.27:0/2218541334 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71c40f9850 msgr2=0x7f71c40fbc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:22:07.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.110+0000 7f71cb705700 1 --2- 172.21.15.27:0/2218541334 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71c40f9850 0x7f71c40fbc40 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 rx=0x7f71c0004660 tx=0x7f71c0009b30).stop 2021-11-03T18:22:07.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.110+0000 7f71cb705700 1 -- 172.21.15.27:0/2218541334 shutdown_connections 2021-11-03T18:22:07.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.110+0000 7f71cb705700 1 --2- 172.21.15.27:0/2218541334 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71c40f9850 0x7f71c40fbc40 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:07.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.110+0000 7f71cb705700 1 --2- 172.21.15.27:0/2218541334 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71c4060390 0x7f71c40f9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T18:22:07.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.110+0000 7f71cb705700 1 -- 172.21.15.27:0/2218541334 >> 172.21.15.27:0/2218541334 conn(0x7f71c40f0de0 msgr2=0x7f71c40f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:22:07.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.110+0000 7f71cb705700 1 -- 172.21.15.27:0/2218541334 shutdown_connections 2021-11-03T18:22:07.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.110+0000 7f71cb705700 1 -- 172.21.15.27:0/2218541334 wait complete. 2021-11-03T18:22:07.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.111+0000 7f71cb705700 1 Processor -- start 2021-11-03T18:22:07.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.111+0000 7f71cb705700 1 -- start start 2021-11-03T18:22:07.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.111+0000 7f71cb705700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71c4060390 0x7f71c4106fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:22:07.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.111+0000 7f71cb705700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71c40f9850 0x7f71c41074a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:22:07.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.111+0000 7f71cb705700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c4107ae0 con 0x7f71c40f9850 2021-11-03T18:22:07.121 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.111+0000 7f71cb705700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71c4108390 con 0x7f71c4060390 2021-11-03T18:22:07.121 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.111+0000 7f71c8ca0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71c40f9850 0x7f71c41074a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:22:07.121 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.111+0000 7f71c8ca0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71c40f9850 0x7f71c41074a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34460/0 (socket says 172.21.15.27:34460) 2021-11-03T18:22:07.122 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.111+0000 7f71c8ca0700 1 -- 172.21.15.27:0/2460992628 learned_addr learned my addr 172.21.15.27:0/2460992628 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:22:07.122 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.111+0000 7f71c94a1700 1 --2- 172.21.15.27:0/2460992628 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71c4060390 0x7f71c4106fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:22:07.123 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.111+0000 7f71c8ca0700 1 -- 172.21.15.27:0/2460992628 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71c4060390 msgr2=0x7f71c4106fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:22:07.123 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.111+0000 7f71c8ca0700 1 --2- 172.21.15.27:0/2460992628 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71c4060390 0x7f71c4106fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:07.124 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.111+0000 7f71c8ca0700 1 -- 172.21.15.27:0/2460992628 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71c0005040 con 0x7f71c40f9850 2021-11-03T18:22:07.124 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.111+0000 7f71c8ca0700 1 --2- 172.21.15.27:0/2460992628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71c40f9850 0x7f71c41074a0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 rx=0x7f71c0009990 tx=0x7f71c0009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:22:07.125 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.111+0000 7f71b9ffb700 1 -- 172.21.15.27:0/2460992628 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71c0018070 con 0x7f71c40f9850 2021-11-03T18:22:07.125 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.111+0000 7f71b9ffb700 1 -- 172.21.15.27:0/2460992628 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f71c0007ca0 con 0x7f71c40f9850 2021-11-03T18:22:07.125 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.111+0000 7f71cb705700 1 -- 172.21.15.27:0/2460992628 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71c4108610 con 0x7f71c40f9850 2021-11-03T18:22:07.126 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.111+0000 7f71b9ffb700 1 -- 172.21.15.27:0/2460992628 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f71c001c620 con 0x7f71c40f9850 2021-11-03T18:22:07.126 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.112+0000 7f71cb705700 1 -- 172.21.15.27:0/2460992628 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71c4108b70 con 0x7f71c40f9850 2021-11-03T18:22:07.127 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.113+0000 7f71cb705700 1 -- 172.21.15.27:0/2460992628 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71c40f4a70 con 0x7f71c40f9850 2021-11-03T18:22:07.127 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.113+0000 7f71b9ffb700 1 -- 172.21.15.27:0/2460992628 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f71c001d960 con 0x7f71c40f9850 2021-11-03T18:22:07.128 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.113+0000 7f71b9ffb700 1 --2- 172.21.15.27:0/2460992628 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f71b00640b0 0x7f71b0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:22:07.128 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.114+0000 7f71b9ffb700 1 -- 172.21.15.27:0/2460992628 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f71c0082580 con 0x7f71c40f9850 2021-11-03T18:22:07.128 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.114+0000 7f71c94a1700 1 --2- 172.21.15.27:0/2460992628 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f71b00640b0 0x7f71b0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:22:07.129 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.114+0000 7f71c94a1700 1 --2- 172.21.15.27:0/2460992628 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f71b00640b0 0x7f71b0066570 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f71c4060160 tx=0x7f71b4008040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:22:07.130 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.118+0000 7f71b9ffb700 1 -- 172.21.15.27:0/2460992628 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f71c0054940 con 0x7f71c40f9850 2021-11-03T18:22:07.291 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.291+0000 7f71cb705700 1 -- 172.21.15.27:0/2460992628 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f71c40fef50 con 0x7f71b00640b0 2021-11-03T18:22:07.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.306+0000 7f71b9ffb700 1 -- 172.21.15.27:0/2460992628 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f71c40fef50 con 0x7f71b00640b0 2021-11-03T18:22:07.306 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:22:07.306 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (73m) 2m ago 74m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:22:07.306 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (65m) 2m ago 74m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:22:07.307 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (65m) 116s ago 74m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:22:07.307 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (73m) 2m ago 74m 42.4M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:22:07.307 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (67m) 2m ago 70m 26.5M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:22:07.307 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (70m) 2m ago 70m 3849M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:22:07.307 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (65m) 116s ago 70m 26.1M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:22:07.308 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (67m) 116s ago 70m 27.2M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:22:07.308 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (67m) 2m ago 76m 590M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:22:07.308 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (66m) 116s ago 74m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:22:07.308 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (66m) 2m ago 76m 312M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:22:07.309 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (65m) 116s ago 74m 145M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:22:07.309 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (74m) 2m ago 74m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:22:07.309 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (73m) 116s ago 73m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:22:07.309 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (64m) 2m ago 73m 392M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:22:07.309 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (64m) 2m ago 72m 427M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:22:07.310 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (64m) 2m ago 72m 320M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:22:07.310 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (63m) 116s ago 72m 454M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:22:07.310 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (63m) 116s ago 71m 359M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:22:07.310 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (62m) 116s ago 71m 368M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:22:07.311 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (73m) 2m ago 74m 59.9M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:22:07.312 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.308+0000 7f71cb705700 1 -- 172.21.15.27:0/2460992628 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f71b00640b0 msgr2=0x7f71b0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:22:07.312 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.308+0000 7f71cb705700 1 --2- 172.21.15.27:0/2460992628 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f71b00640b0 0x7f71b0066570 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 rx=0x7f71c4060160 tx=0x7f71b4008040).stop 2021-11-03T18:22:07.312 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.308+0000 7f71cb705700 1 -- 172.21.15.27:0/2460992628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71c40f9850 msgr2=0x7f71c41074a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:22:07.312 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.308+0000 7f71cb705700 1 --2- 172.21.15.27:0/2460992628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71c40f9850 0x7f71c41074a0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 rx=0x7f71c0009990 tx=0x7f71c0009b30).stop 2021-11-03T18:22:07.313 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.308+0000 7f71cb705700 1 -- 172.21.15.27:0/2460992628 shutdown_connections 2021-11-03T18:22:07.313 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.308+0000 7f71cb705700 1 --2- 172.21.15.27:0/2460992628 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f71b00640b0 0x7f71b0066570 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:07.313 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.308+0000 7f71cb705700 1 --2- 172.21.15.27:0/2460992628 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f71c4060390 0x7f71c4106fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:07.313 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.308+0000 7f71cb705700 1 --2- 172.21.15.27:0/2460992628 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f71c40f9850 0x7f71c41074a0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:07.314 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.308+0000 7f71cb705700 1 -- 172.21.15.27:0/2460992628 >> 172.21.15.27:0/2460992628 conn(0x7f71c40f0de0 msgr2=0x7f71c40f31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:22:07.314 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.308+0000 7f71cb705700 1 -- 172.21.15.27:0/2460992628 shutdown_connections 2021-11-03T18:22:07.314 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.308+0000 7f71cb705700 1 -- 172.21.15.27:0/2460992628 wait complete. 2021-11-03T18:22:07.408 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.407+0000 7fbec5995700 1 -- 172.21.15.27:0/2638902634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbec00f66e0 msgr2=0x7fbec00f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:22:07.409 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.408+0000 7fbec5995700 1 --2- 172.21.15.27:0/2638902634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbec00f66e0 0x7fbec00f6b00 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 rx=0x7fbeb80099e0 tx=0x7fbeb8004d10).stop 2021-11-03T18:22:07.409 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.408+0000 7fbec5995700 1 -- 172.21.15.27:0/2638902634 shutdown_connections 2021-11-03T18:22:07.409 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.408+0000 7fbec5995700 1 --2- 172.21.15.27:0/2638902634 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbec00f6ff0 0x7fbec00f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:07.410 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.408+0000 7fbec5995700 1 --2- 172.21.15.27:0/2638902634 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbec00f66e0 0x7fbec00f6b00 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:07.410 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.408+0000 7fbec5995700 1 -- 172.21.15.27:0/2638902634 >> 172.21.15.27:0/2638902634 conn(0x7fbec00f1f00 msgr2=0x7fbec00f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:22:07.410 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.408+0000 7fbec5995700 1 -- 172.21.15.27:0/2638902634 shutdown_connections 2021-11-03T18:22:07.410 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.408+0000 7fbec5995700 1 -- 172.21.15.27:0/2638902634 wait complete. 2021-11-03T18:22:07.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.408+0000 7fbec5995700 1 Processor -- start 2021-11-03T18:22:07.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.408+0000 7fbec5995700 1 -- start start 2021-11-03T18:22:07.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.409+0000 7fbec5995700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbec00f66e0 0x7fbec006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:22:07.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.409+0000 7fbec5995700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbec00f6ff0 0x7fbec0069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:22:07.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.409+0000 7fbec5995700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec006a380 con 0x7fbec00f66e0 2021-11-03T18:22:07.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.409+0000 7fbec5995700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec006a4c0 con 0x7fbec00f6ff0 2021-11-03T18:22:07.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.409+0000 7fbebeffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbec00f66e0 0x7fbec006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:22:07.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.409+0000 7fbebeffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbec00f66e0 0x7fbec006b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34468/0 (socket says 172.21.15.27:34468) 2021-11-03T18:22:07.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.409+0000 7fbebeffd700 1 -- 172.21.15.27:0/4162375871 learned_addr learned my addr 172.21.15.27:0/4162375871 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:22:07.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.409+0000 7fbebe7fc700 1 --2- 172.21.15.27:0/4162375871 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbec00f6ff0 0x7fbec0069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:22:07.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.409+0000 7fbebeffd700 1 -- 172.21.15.27:0/4162375871 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbec00f6ff0 msgr2=0x7fbec0069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:22:07.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.409+0000 7fbebeffd700 1 --2- 172.21.15.27:0/4162375871 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbec00f6ff0 0x7fbec0069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:07.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.409+0000 7fbebeffd700 1 -- 172.21.15.27:0/4162375871 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb8009640 con 0x7fbec00f66e0 2021-11-03T18:22:07.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.409+0000 7fbebeffd700 1 --2- 172.21.15.27:0/4162375871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbec00f66e0 0x7fbec006b9a0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 rx=0x7fbeb8003380 tx=0x7fbeb8004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:22:07.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.409+0000 7fbeb77fe700 1 -- 172.21.15.27:0/4162375871 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbeb800e070 con 0x7fbec00f66e0 2021-11-03T18:22:07.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.409+0000 7fbeb77fe700 1 -- 172.21.15.27:0/4162375871 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fbeb8007dc0 con 0x7fbec00f66e0 2021-11-03T18:22:07.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.409+0000 7fbec5995700 1 -- 172.21.15.27:0/4162375871 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbec006a740 con 0x7fbec00f66e0 2021-11-03T18:22:07.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.409+0000 7fbeb77fe700 1 -- 172.21.15.27:0/4162375871 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbeb8018070 con 0x7fbec00f66e0 2021-11-03T18:22:07.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.409+0000 7fbec5995700 1 -- 172.21.15.27:0/4162375871 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec006aca0 con 0x7fbec00f66e0 2021-11-03T18:22:07.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.410+0000 7fbec5995700 1 -- 172.21.15.27:0/4162375871 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbec0045ad0 con 0x7fbec00f66e0 2021-11-03T18:22:07.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.411+0000 7fbeb77fe700 1 -- 172.21.15.27:0/4162375871 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fbeb80139d0 con 0x7fbec00f66e0 2021-11-03T18:22:07.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.411+0000 7fbeb77fe700 1 --2- 172.21.15.27:0/4162375871 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbeac0640b0 0x7fbeac066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:22:07.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.411+0000 7fbeb77fe700 1 -- 172.21.15.27:0/4162375871 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fbeb8082170 con 0x7fbec00f66e0 2021-11-03T18:22:07.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.411+0000 7fbebe7fc700 1 --2- 172.21.15.27:0/4162375871 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbeac0640b0 0x7fbeac066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:22:07.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.412+0000 7fbebe7fc700 1 --2- 172.21.15.27:0/4162375871 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbeac0640b0 0x7fbeac066570 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7fbea8007fd0 tx=0x7fbea800b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:22:07.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.415+0000 7fbeb77fe700 1 -- 172.21.15.27:0/4162375871 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fbeb8054470 con 0x7fbec00f66e0 2021-11-03T18:22:07.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.643+0000 7fbec5995700 1 -- 172.21.15.27:0/4162375871 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbec00586e0 con 0x7fbec00f66e0 2021-11-03T18:22:07.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.644+0000 7fbeb77fe700 1 -- 172.21.15.27:0/4162375871 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fbeb8054060 con 0x7fbec00f66e0 2021-11-03T18:22:07.644 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:22:07.644 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:22:07.644 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:22:07.644 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:22:07.645 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:22:07.645 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:22:07.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:22:07.645 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:22:07.646 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:22:07.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:22:07.646 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:22:07.647 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:22:07.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:22:07.647 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:22:07.647 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:22:07.648 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:22:07.648 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:22:07.648 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:22:07.649 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.645+0000 7fbec5995700 1 -- 172.21.15.27:0/4162375871 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbeac0640b0 msgr2=0x7fbeac066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:22:07.650 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.645+0000 7fbec5995700 1 --2- 172.21.15.27:0/4162375871 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbeac0640b0 0x7fbeac066570 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7fbea8007fd0 tx=0x7fbea800b040).stop 2021-11-03T18:22:07.650 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.645+0000 7fbec5995700 1 -- 172.21.15.27:0/4162375871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbec00f66e0 msgr2=0x7fbec006b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:22:07.650 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.645+0000 7fbec5995700 1 --2- 172.21.15.27:0/4162375871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbec00f66e0 0x7fbec006b9a0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 rx=0x7fbeb8003380 tx=0x7fbeb8004d10).stop 2021-11-03T18:22:07.650 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.646+0000 7fbec5995700 1 -- 172.21.15.27:0/4162375871 shutdown_connections 2021-11-03T18:22:07.651 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.646+0000 7fbec5995700 1 --2- 172.21.15.27:0/4162375871 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbeac0640b0 0x7fbeac066570 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:07.651 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.646+0000 7fbec5995700 1 --2- 172.21.15.27:0/4162375871 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbec00f66e0 0x7fbec006b9a0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:07.651 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.646+0000 7fbec5995700 1 --2- 172.21.15.27:0/4162375871 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbec00f6ff0 0x7fbec0069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:07.652 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.646+0000 7fbec5995700 1 -- 172.21.15.27:0/4162375871 >> 172.21.15.27:0/4162375871 conn(0x7fbec00f1f00 msgr2=0x7fbec00fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:22:07.652 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.646+0000 7fbec5995700 1 -- 172.21.15.27:0/4162375871 shutdown_connections 2021-11-03T18:22:07.652 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.646+0000 7fbec5995700 1 -- 172.21.15.27:0/4162375871 wait complete. 2021-11-03T18:22:07.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.745+0000 7fcf4a7c3700 1 -- 172.21.15.27:0/766030157 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf440f5ed0 msgr2=0x7fcf440f6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:22:07.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.745+0000 7fcf4a7c3700 1 --2- 172.21.15.27:0/766030157 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf440f5ed0 0x7fcf440f6900 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 rx=0x7fcf34004660 tx=0x7fcf34009b30).stop 2021-11-03T18:22:07.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.745+0000 7fcf4a7c3700 1 -- 172.21.15.27:0/766030157 shutdown_connections 2021-11-03T18:22:07.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.745+0000 7fcf4a7c3700 1 --2- 172.21.15.27:0/766030157 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf440f5ed0 0x7fcf440f6900 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:07.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.745+0000 7fcf4a7c3700 1 --2- 172.21.15.27:0/766030157 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf440f55c0 0x7fcf440f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:07.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.745+0000 7fcf4a7c3700 1 -- 172.21.15.27:0/766030157 >> 172.21.15.27:0/766030157 conn(0x7fcf440f0de0 msgr2=0x7fcf440f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:22:07.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.746+0000 7fcf4a7c3700 1 -- 172.21.15.27:0/766030157 shutdown_connections 2021-11-03T18:22:07.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.746+0000 7fcf4a7c3700 1 -- 172.21.15.27:0/766030157 wait complete. 2021-11-03T18:22:07.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.746+0000 7fcf4a7c3700 1 Processor -- start 2021-11-03T18:22:07.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.746+0000 7fcf4a7c3700 1 -- start start 2021-11-03T18:22:07.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.746+0000 7fcf4a7c3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf440f55c0 0x7fcf44109c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:22:07.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.746+0000 7fcf4a7c3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf440f5ed0 0x7fcf4410c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:22:07.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.746+0000 7fcf4a7c3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf44060520 con 0x7fcf440f55c0 2021-11-03T18:22:07.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.746+0000 7fcf4a7c3700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf44060660 con 0x7fcf440f5ed0 2021-11-03T18:22:07.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.747+0000 7fcf43fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf440f55c0 0x7fcf44109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:22:07.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.747+0000 7fcf43fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf440f55c0 0x7fcf44109c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34478/0 (socket says 172.21.15.27:34478) 2021-11-03T18:22:07.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.747+0000 7fcf43fff700 1 -- 172.21.15.27:0/3845389422 learned_addr learned my addr 172.21.15.27:0/3845389422 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:22:07.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.747+0000 7fcf437fe700 1 --2- 172.21.15.27:0/3845389422 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf440f5ed0 0x7fcf4410c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:22:07.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.747+0000 7fcf43fff700 1 -- 172.21.15.27:0/3845389422 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf440f5ed0 msgr2=0x7fcf4410c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:22:07.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.747+0000 7fcf43fff700 1 --2- 172.21.15.27:0/3845389422 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf440f5ed0 0x7fcf4410c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:07.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.747+0000 7fcf43fff700 1 -- 172.21.15.27:0/3845389422 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf34005040 con 0x7fcf440f55c0 2021-11-03T18:22:07.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.747+0000 7fcf43fff700 1 --2- 172.21.15.27:0/3845389422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf440f55c0 0x7fcf44109c70 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0x7fcf380026d0 tx=0x7fcf38009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:22:07.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.747+0000 7fcf40ff9700 1 -- 172.21.15.27:0/3845389422 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf380079c0 con 0x7fcf440f55c0 2021-11-03T18:22:07.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.747+0000 7fcf40ff9700 1 -- 172.21.15.27:0/3845389422 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fcf3800f040 con 0x7fcf440f55c0 2021-11-03T18:22:07.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.747+0000 7fcf4a7c3700 1 -- 172.21.15.27:0/3845389422 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf4410c720 con 0x7fcf440f55c0 2021-11-03T18:22:07.754 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.747+0000 7fcf40ff9700 1 -- 172.21.15.27:0/3845389422 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcf380148b0 con 0x7fcf440f55c0 2021-11-03T18:22:07.754 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.747+0000 7fcf4a7c3700 1 -- 172.21.15.27:0/3845389422 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf4410cce0 con 0x7fcf440f55c0 2021-11-03T18:22:07.754 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.748+0000 7fcf4a7c3700 1 -- 172.21.15.27:0/3845389422 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf44045ad0 con 0x7fcf440f55c0 2021-11-03T18:22:07.754 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.749+0000 7fcf40ff9700 1 -- 172.21.15.27:0/3845389422 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fcf38007b20 con 0x7fcf440f55c0 2021-11-03T18:22:07.755 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.749+0000 7fcf40ff9700 1 --2- 172.21.15.27:0/3845389422 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcf2c0640b0 0x7fcf2c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:22:07.756 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.749+0000 7fcf40ff9700 1 -- 172.21.15.27:0/3845389422 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fcf380807a0 con 0x7fcf440f55c0 2021-11-03T18:22:07.756 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.749+0000 7fcf437fe700 1 --2- 172.21.15.27:0/3845389422 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcf2c0640b0 0x7fcf2c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:22:07.756 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.749+0000 7fcf437fe700 1 --2- 172.21.15.27:0/3845389422 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcf2c0640b0 0x7fcf2c066570 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7fcf34004630 tx=0x7fcf34009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:22:07.756 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.753+0000 7fcf40ff9700 1 -- 172.21.15.27:0/3845389422 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fcf38052aa0 con 0x7fcf440f55c0 2021-11-03T18:22:07.954 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.954+0000 7fcf4a7c3700 1 -- 172.21.15.27:0/3845389422 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fcf440fdeb0 con 0x7fcf440f55c0 2021-11-03T18:22:07.954 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.955+0000 7fcf40ff9700 1 -- 172.21.15.27:0/3845389422 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7fcf38052690 con 0x7fcf440f55c0 2021-11-03T18:22:07.955 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:22:07.955 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:22:07.955 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:22:07.955 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:22:07.956 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:22:07.956 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:22:07.956 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:22:07.956 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:22:07.956 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:22:07.957 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:22:07.957 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:22:07.957 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:22:07.957 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:22:07.958 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:22:07.958 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:22:07.958 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:22:07.958 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:22:07.959 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:22:07.959 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:22:07.959 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:22:07.959 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:22:07.959 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:22:07.960 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:22:07.960 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:22:07.960 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:22:07.960 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:22:07.961 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:22:07.961 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:22:07.961 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:22:07.961 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:22:07.962 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:22:07.962 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:22:07.962 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:22:07.962 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:22:07.963 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:22:07.963 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:22:07.963 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:22:07.963 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:22:07.963 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:22:07.964 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.956+0000 7fcf4a7c3700 1 -- 172.21.15.27:0/3845389422 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcf2c0640b0 msgr2=0x7fcf2c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:22:07.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.956+0000 7fcf4a7c3700 1 --2- 172.21.15.27:0/3845389422 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcf2c0640b0 0x7fcf2c066570 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 rx=0x7fcf34004630 tx=0x7fcf34009b30).stop 2021-11-03T18:22:07.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.956+0000 7fcf4a7c3700 1 -- 172.21.15.27:0/3845389422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf440f55c0 msgr2=0x7fcf44109c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:22:07.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.956+0000 7fcf4a7c3700 1 --2- 172.21.15.27:0/3845389422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf440f55c0 0x7fcf44109c70 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0x7fcf380026d0 tx=0x7fcf38009de0).stop 2021-11-03T18:22:07.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.957+0000 7fcf4a7c3700 1 -- 172.21.15.27:0/3845389422 shutdown_connections 2021-11-03T18:22:07.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.957+0000 7fcf4a7c3700 1 --2- 172.21.15.27:0/3845389422 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcf2c0640b0 0x7fcf2c066570 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:07.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.957+0000 7fcf4a7c3700 1 --2- 172.21.15.27:0/3845389422 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcf440f55c0 0x7fcf44109c70 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:07.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.957+0000 7fcf4a7c3700 1 --2- 172.21.15.27:0/3845389422 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcf440f5ed0 0x7fcf4410c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:07.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.957+0000 7fcf4a7c3700 1 -- 172.21.15.27:0/3845389422 >> 172.21.15.27:0/3845389422 conn(0x7fcf440f0de0 msgr2=0x7fcf440fa350 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:22:07.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.957+0000 7fcf4a7c3700 1 -- 172.21.15.27:0/3845389422 shutdown_connections 2021-11-03T18:22:07.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:07.957+0000 7fcf4a7c3700 1 -- 172.21.15.27:0/3845389422 wait complete. 2021-11-03T18:22:07.967 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:22:08.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:08 smithi115 ceph-mon[65453]: pgmap v2024: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:08.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:08 smithi115 ceph-mon[65453]: from='client.36184 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:22:08.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:08 smithi115 ceph-mon[65453]: from='client.36188 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:22:08.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:08 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/4162375871' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:08.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:08 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3845389422' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:22:08.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:08 smithi027 ceph-mon[97244]: pgmap v2024: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:08.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:08 smithi027 ceph-mon[97244]: from='client.36184 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:22:08.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:08 smithi027 ceph-mon[97244]: from='client.36188 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:22:08.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:08 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/4162375871' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:08.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:08 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3845389422' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:22:10.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:10 smithi115 ceph-mon[65453]: pgmap v2025: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:10.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3972.455069 secs 2021-11-03T18:22:10.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:10 smithi027 ceph-mon[97244]: pgmap v2025: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:10.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3972.455069 secs 2021-11-03T18:22:12.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:12 smithi115 ceph-mon[65453]: pgmap v2026: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:12.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:22:12.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:22:12.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:22:12.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:12.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:12.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:12.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:12.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:12 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:22:12.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:12 smithi027 ceph-mon[97244]: pgmap v2026: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:12.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:22:12.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:22:12.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:22:12.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:12.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:12.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:12.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:12.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:12 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:22:14.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:14 smithi115 ceph-mon[65453]: pgmap v2027: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:14.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:14 smithi027 ceph-mon[97244]: pgmap v2027: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:15.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3977.455205 secs 2021-11-03T18:22:15.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3977.455205 secs 2021-11-03T18:22:16.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:16 smithi115 ceph-mon[65453]: pgmap v2028: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:16.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:16 smithi027 ceph-mon[97244]: pgmap v2028: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:18.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:18 smithi115 ceph-mon[65453]: pgmap v2029: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:18.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:18 smithi027 ceph-mon[97244]: pgmap v2029: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:20.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:20 smithi115 ceph-mon[65453]: pgmap v2030: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:20.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3982.455341 secs 2021-11-03T18:22:20.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:20 smithi027 ceph-mon[97244]: pgmap v2030: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:20.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3982.455341 secs 2021-11-03T18:22:22.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:22 smithi115 ceph-mon[65453]: pgmap v2031: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:22.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:22:22.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:22 smithi115 ceph-mon[65453]: poll took 60162.247 ms: 1 events 2021-11-03T18:22:22.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:22:22.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:22:22.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:22.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:22.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:22.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:22.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:22 smithi027 ceph-mon[97244]: pgmap v2031: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:22.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:22:22.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:22 smithi027 ceph-mon[97244]: poll took 60162.247 ms: 1 events 2021-11-03T18:22:22.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:22:22.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:22:22.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:22.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:22.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:22.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:23.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:23 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:22:23.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:23 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:22:24.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:24 smithi115 ceph-mon[65453]: pgmap v2032: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:24.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:24 smithi027 ceph-mon[97244]: pgmap v2032: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:25.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3987.455483 secs 2021-11-03T18:22:25.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3987.455483 secs 2021-11-03T18:22:26.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:26 smithi115 ceph-mon[65453]: pgmap v2033: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:26.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:26 smithi027 ceph-mon[97244]: pgmap v2033: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:28.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:28 smithi115 ceph-mon[65453]: pgmap v2034: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:28.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:28 smithi027 ceph-mon[97244]: pgmap v2034: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:30.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:30 smithi115 ceph-mon[65453]: pgmap v2035: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:30.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3992.455567 secs 2021-11-03T18:22:30.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:30 smithi027 ceph-mon[97244]: pgmap v2035: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:30.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3992.455567 secs 2021-11-03T18:22:32.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:32 smithi115 ceph-mon[65453]: pgmap v2036: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:32.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:22:32.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:22:32.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:22:32.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:32.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:32.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:32.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:32.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:32 smithi027 ceph-mon[97244]: pgmap v2036: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:32.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:22:32.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:22:32.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:22:32.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:32.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:32.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:32.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:33.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:33 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:22:33.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:33 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:22:34.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:34 smithi115 ceph-mon[65453]: pgmap v2037: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:34.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:34 smithi027 ceph-mon[97244]: pgmap v2037: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:35.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 3997.455649 secs 2021-11-03T18:22:35.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 3997.455649 secs 2021-11-03T18:22:36.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:36 smithi115 ceph-mon[65453]: pgmap v2038: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:36.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:36 smithi027 ceph-mon[97244]: pgmap v2038: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:38.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.073+0000 7fa72f4a8700 1 -- 172.21.15.27:0/3969174300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7280f6530 msgr2=0x7fa7280f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:22:38.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.073+0000 7fa72f4a8700 1 --2- 172.21.15.27:0/3969174300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7280f6530 0x7fa7280f6950 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 rx=0x7fa7200099e0 tx=0x7fa720004d10).stop 2021-11-03T18:22:38.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.073+0000 7fa72f4a8700 1 -- 172.21.15.27:0/3969174300 shutdown_connections 2021-11-03T18:22:38.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.073+0000 7fa72f4a8700 1 --2- 172.21.15.27:0/3969174300 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7280f77d0 0x7fa7280f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:38.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.073+0000 7fa72f4a8700 1 --2- 172.21.15.27:0/3969174300 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7280f6530 0x7fa7280f6950 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:38.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.073+0000 7fa72f4a8700 1 -- 172.21.15.27:0/3969174300 >> 172.21.15.27:0/3969174300 conn(0x7fa7280f1c90 msgr2=0x7fa7280f40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:22:38.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.073+0000 7fa72f4a8700 1 -- 172.21.15.27:0/3969174300 shutdown_connections 2021-11-03T18:22:38.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.073+0000 7fa72f4a8700 1 -- 172.21.15.27:0/3969174300 wait complete. 2021-11-03T18:22:38.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.074+0000 7fa72f4a8700 1 Processor -- start 2021-11-03T18:22:38.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.074+0000 7fa72f4a8700 1 -- start start 2021-11-03T18:22:38.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.074+0000 7fa72f4a8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7280f6530 0x7fa728106e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:22:38.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.074+0000 7fa72f4a8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7280f77d0 0x7fa728107380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:22:38.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.074+0000 7fa72f4a8700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa728107930 con 0x7fa7280f77d0 2021-11-03T18:22:38.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.074+0000 7fa72f4a8700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa728107a70 con 0x7fa7280f6530 2021-11-03T18:22:38.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.074+0000 7fa72dca5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7280f77d0 0x7fa728107380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:22:38.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.074+0000 7fa72dca5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7280f77d0 0x7fa728107380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34500/0 (socket says 172.21.15.27:34500) 2021-11-03T18:22:38.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.074+0000 7fa72dca5700 1 -- 172.21.15.27:0/1096308058 learned_addr learned my addr 172.21.15.27:0/1096308058 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:22:38.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.074+0000 7fa72e4a6700 1 --2- 172.21.15.27:0/1096308058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7280f6530 0x7fa728106e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:22:38.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.075+0000 7fa72dca5700 1 -- 172.21.15.27:0/1096308058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7280f6530 msgr2=0x7fa728106e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:22:38.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.075+0000 7fa72dca5700 1 --2- 172.21.15.27:0/1096308058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7280f6530 0x7fa728106e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:38.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.075+0000 7fa72dca5700 1 -- 172.21.15.27:0/1096308058 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa720009640 con 0x7fa7280f77d0 2021-11-03T18:22:38.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.075+0000 7fa72e4a6700 1 --2- 172.21.15.27:0/1096308058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7280f6530 0x7fa728106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:22:38.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.075+0000 7fa72dca5700 1 --2- 172.21.15.27:0/1096308058 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7280f77d0 0x7fa728107380 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 rx=0x7fa71c00c530 tx=0x7fa71c007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:22:38.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.075+0000 7fa726ffd700 1 -- 172.21.15.27:0/1096308058 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa71c00f650 con 0x7fa7280f77d0 2021-11-03T18:22:38.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.075+0000 7fa726ffd700 1 -- 172.21.15.27:0/1096308058 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fa71c00fc60 con 0x7fa7280f77d0 2021-11-03T18:22:38.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.075+0000 7fa72f4a8700 1 -- 172.21.15.27:0/1096308058 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa728108410 con 0x7fa7280f77d0 2021-11-03T18:22:38.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.075+0000 7fa726ffd700 1 -- 172.21.15.27:0/1096308058 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa71c012cd0 con 0x7fa7280f77d0 2021-11-03T18:22:38.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.075+0000 7fa72f4a8700 1 -- 172.21.15.27:0/1096308058 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa728108950 con 0x7fa7280f77d0 2021-11-03T18:22:38.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.076+0000 7fa72f4a8700 1 -- 172.21.15.27:0/1096308058 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa728045ad0 con 0x7fa7280f77d0 2021-11-03T18:22:38.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.076+0000 7fa726ffd700 1 -- 172.21.15.27:0/1096308058 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fa71c010070 con 0x7fa7280f77d0 2021-11-03T18:22:38.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.077+0000 7fa726ffd700 1 --2- 172.21.15.27:0/1096308058 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa710064100 0x7fa7100665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:22:38.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.077+0000 7fa726ffd700 1 -- 172.21.15.27:0/1096308058 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fa71c080740 con 0x7fa7280f77d0 2021-11-03T18:22:38.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.077+0000 7fa72e4a6700 1 --2- 172.21.15.27:0/1096308058 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa710064100 0x7fa7100665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:22:38.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.077+0000 7fa72e4a6700 1 --2- 172.21.15.27:0/1096308058 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa710064100 0x7fa7100665c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7fa720009f50 tx=0x7fa720004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:22:38.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.081+0000 7fa726ffd700 1 -- 172.21.15.27:0/1096308058 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fa71c052950 con 0x7fa7280f77d0 2021-11-03T18:22:38.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.259+0000 7fa72f4a8700 1 -- 172.21.15.27:0/1096308058 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa728108b70 con 0x7fa710064100 2021-11-03T18:22:38.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.260+0000 7fa726ffd700 1 -- 172.21.15.27:0/1096308058 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fa728108b70 con 0x7fa710064100 2021-11-03T18:22:38.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.262+0000 7fa72f4a8700 1 -- 172.21.15.27:0/1096308058 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa710064100 msgr2=0x7fa7100665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:22:38.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.262+0000 7fa72f4a8700 1 --2- 172.21.15.27:0/1096308058 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa710064100 0x7fa7100665c0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7fa720009f50 tx=0x7fa720004d10).stop 2021-11-03T18:22:38.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.262+0000 7fa72f4a8700 1 -- 172.21.15.27:0/1096308058 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7280f77d0 msgr2=0x7fa728107380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:22:38.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.262+0000 7fa72f4a8700 1 --2- 172.21.15.27:0/1096308058 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7280f77d0 0x7fa728107380 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 rx=0x7fa71c00c530 tx=0x7fa71c007e20).stop 2021-11-03T18:22:38.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.262+0000 7fa72f4a8700 1 -- 172.21.15.27:0/1096308058 shutdown_connections 2021-11-03T18:22:38.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.262+0000 7fa72f4a8700 1 --2- 172.21.15.27:0/1096308058 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa710064100 0x7fa7100665c0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:38.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.262+0000 7fa72f4a8700 1 --2- 172.21.15.27:0/1096308058 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa7280f6530 0x7fa728106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:38.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.262+0000 7fa72f4a8700 1 --2- 172.21.15.27:0/1096308058 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa7280f77d0 0x7fa728107380 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:38.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.262+0000 7fa72f4a8700 1 -- 172.21.15.27:0/1096308058 >> 172.21.15.27:0/1096308058 conn(0x7fa7280f1c90 msgr2=0x7fa7280fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:22:38.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.262+0000 7fa72f4a8700 1 -- 172.21.15.27:0/1096308058 shutdown_connections 2021-11-03T18:22:38.265 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.262+0000 7fa72f4a8700 1 -- 172.21.15.27:0/1096308058 wait complete. 2021-11-03T18:22:38.270 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:22:38.362 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.362+0000 7f995135e700 1 -- 172.21.15.27:0/2884005664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f994c0f5ef0 msgr2=0x7f994c0f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:22:38.362 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.362+0000 7f995135e700 1 --2- 172.21.15.27:0/2884005664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f994c0f5ef0 0x7f994c0f6920 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 rx=0x7f9940004660 tx=0x7f9940009b30).stop 2021-11-03T18:22:38.363 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.362+0000 7f995135e700 1 -- 172.21.15.27:0/2884005664 shutdown_connections 2021-11-03T18:22:38.363 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.362+0000 7f995135e700 1 --2- 172.21.15.27:0/2884005664 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f994c0f5ef0 0x7f994c0f6920 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:38.363 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.362+0000 7f995135e700 1 --2- 172.21.15.27:0/2884005664 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f994c0f55e0 0x7f994c0f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:38.363 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.362+0000 7f995135e700 1 -- 172.21.15.27:0/2884005664 >> 172.21.15.27:0/2884005664 conn(0x7f994c0f0de0 msgr2=0x7f994c0f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:22:38.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.362+0000 7f995135e700 1 -- 172.21.15.27:0/2884005664 shutdown_connections 2021-11-03T18:22:38.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.362+0000 7f995135e700 1 -- 172.21.15.27:0/2884005664 wait complete. 2021-11-03T18:22:38.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.363+0000 7f995135e700 1 Processor -- start 2021-11-03T18:22:38.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.363+0000 7f995135e700 1 -- start start 2021-11-03T18:22:38.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.363+0000 7f995135e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f994c0f55e0 0x7f994c0fa000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:22:38.365 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.363+0000 7f995135e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f994c0f5ef0 0x7f994c0f8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:22:38.365 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.363+0000 7f995135e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994c0f89e0 con 0x7f994c0f5ef0 2021-11-03T18:22:38.365 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.363+0000 7f995135e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f994c0f8b20 con 0x7f994c0f55e0 2021-11-03T18:22:38.365 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.363+0000 7f994a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f994c0f5ef0 0x7f994c0f8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:22:38.366 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.363+0000 7f994a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f994c0f5ef0 0x7f994c0f8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34510/0 (socket says 172.21.15.27:34510) 2021-11-03T18:22:38.366 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.363+0000 7f994a7fc700 1 -- 172.21.15.27:0/1866883961 learned_addr learned my addr 172.21.15.27:0/1866883961 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:22:38.366 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.363+0000 7f994affd700 1 --2- 172.21.15.27:0/1866883961 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f994c0f55e0 0x7f994c0fa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:22:38.367 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.363+0000 7f994a7fc700 1 -- 172.21.15.27:0/1866883961 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f994c0f55e0 msgr2=0x7f994c0fa000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:22:38.367 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.363+0000 7f994a7fc700 1 --2- 172.21.15.27:0/1866883961 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f994c0f55e0 0x7f994c0fa000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:38.367 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.363+0000 7f994a7fc700 1 -- 172.21.15.27:0/1866883961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9940005040 con 0x7f994c0f5ef0 2021-11-03T18:22:38.367 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.363+0000 7f994a7fc700 1 --2- 172.21.15.27:0/1866883961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f994c0f5ef0 0x7f994c0f8480 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 rx=0x7f9940009990 tx=0x7f9940009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:22:38.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.363+0000 7f99337fe700 1 -- 172.21.15.27:0/1866883961 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9940018070 con 0x7f994c0f5ef0 2021-11-03T18:22:38.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.364+0000 7f995135e700 1 -- 172.21.15.27:0/1866883961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f994c0f8da0 con 0x7f994c0f5ef0 2021-11-03T18:22:38.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.364+0000 7f99337fe700 1 -- 172.21.15.27:0/1866883961 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f9940007ca0 con 0x7f994c0f5ef0 2021-11-03T18:22:38.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.364+0000 7f99337fe700 1 -- 172.21.15.27:0/1866883961 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f994001c620 con 0x7f994c0f5ef0 2021-11-03T18:22:38.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.364+0000 7f995135e700 1 -- 172.21.15.27:0/1866883961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f994c0f9300 con 0x7f994c0f5ef0 2021-11-03T18:22:38.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.364+0000 7f995135e700 1 -- 172.21.15.27:0/1866883961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f994c045ad0 con 0x7f994c0f5ef0 2021-11-03T18:22:38.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.365+0000 7f99337fe700 1 -- 172.21.15.27:0/1866883961 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f994001d960 con 0x7f994c0f5ef0 2021-11-03T18:22:38.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.365+0000 7f99337fe700 1 --2- 172.21.15.27:0/1866883961 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f99340640b0 0x7f9934066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:22:38.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.365+0000 7f99337fe700 1 -- 172.21.15.27:0/1866883961 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f99400825a0 con 0x7f994c0f5ef0 2021-11-03T18:22:38.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.365+0000 7f994affd700 1 --2- 172.21.15.27:0/1866883961 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f99340640b0 0x7f9934066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:22:38.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.366+0000 7f994affd700 1 --2- 172.21.15.27:0/1866883961 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f99340640b0 0x7f9934066570 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f993c0041d0 tx=0x7f993c009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:22:38.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.369+0000 7f99337fe700 1 -- 172.21.15.27:0/1866883961 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f9940054960 con 0x7f994c0f5ef0 2021-11-03T18:22:38.542 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.544+0000 7f995135e700 1 -- 172.21.15.27:0/1866883961 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f994c05fe50 con 0x7f99340640b0 2021-11-03T18:22:38.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.558+0000 7f99337fe700 1 -- 172.21.15.27:0/1866883961 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f994c05fe50 con 0x7f99340640b0 2021-11-03T18:22:38.557 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:22:38.558 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (74m) 3m ago 75m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:22:38.558 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (65m) 3m ago 75m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:22:38.559 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (65m) 2m ago 74m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:22:38.559 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (74m) 3m ago 75m 42.4M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:22:38.559 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (67m) 3m ago 70m 26.5M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:22:38.560 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (70m) 3m ago 70m 3849M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:22:38.560 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (66m) 2m ago 70m 26.1M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:22:38.561 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (68m) 2m ago 70m 27.2M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:22:38.561 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (67m) 3m ago 76m 590M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:22:38.561 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (67m) 2m ago 74m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:22:38.562 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (66m) 3m ago 76m 312M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:22:38.562 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (66m) 2m ago 74m 145M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:22:38.562 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (75m) 3m ago 75m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:22:38.563 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (74m) 2m ago 74m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:22:38.563 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (65m) 3m ago 73m 392M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:22:38.563 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (65m) 3m ago 73m 427M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:22:38.563 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (64m) 3m ago 73m 320M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:22:38.564 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (64m) 2m ago 72m 454M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:22:38.564 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (63m) 2m ago 72m 359M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:22:38.565 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (63m) 2m ago 72m 368M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:22:38.565 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (73m) 3m ago 74m 59.9M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:22:38.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.560+0000 7f995135e700 1 -- 172.21.15.27:0/1866883961 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f99340640b0 msgr2=0x7f9934066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:22:38.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.560+0000 7f995135e700 1 --2- 172.21.15.27:0/1866883961 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f99340640b0 0x7f9934066570 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f993c0041d0 tx=0x7f993c009040).stop 2021-11-03T18:22:38.568 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.560+0000 7f995135e700 1 -- 172.21.15.27:0/1866883961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f994c0f5ef0 msgr2=0x7f994c0f8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:22:38.568 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.560+0000 7f995135e700 1 --2- 172.21.15.27:0/1866883961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f994c0f5ef0 0x7f994c0f8480 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 rx=0x7f9940009990 tx=0x7f9940009b30).stop 2021-11-03T18:22:38.569 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.560+0000 7f995135e700 1 -- 172.21.15.27:0/1866883961 shutdown_connections 2021-11-03T18:22:38.569 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.560+0000 7f995135e700 1 --2- 172.21.15.27:0/1866883961 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f99340640b0 0x7f9934066570 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:38.570 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.560+0000 7f995135e700 1 --2- 172.21.15.27:0/1866883961 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f994c0f55e0 0x7f994c0fa000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:38.570 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.560+0000 7f995135e700 1 --2- 172.21.15.27:0/1866883961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f994c0f5ef0 0x7f994c0f8480 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:38.571 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.560+0000 7f995135e700 1 -- 172.21.15.27:0/1866883961 >> 172.21.15.27:0/1866883961 conn(0x7f994c0f0de0 msgr2=0x7f994c05f740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:22:38.571 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.560+0000 7f995135e700 1 -- 172.21.15.27:0/1866883961 shutdown_connections 2021-11-03T18:22:38.572 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.560+0000 7f995135e700 1 -- 172.21.15.27:0/1866883961 wait complete. 2021-11-03T18:22:38.658 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.659+0000 7fbefa588700 1 -- 172.21.15.27:0/3306773900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbef40f6b90 msgr2=0x7fbef40f6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:22:38.658 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.659+0000 7fbefa588700 1 --2- 172.21.15.27:0/3306773900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbef40f6b90 0x7fbef40f6fb0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 rx=0x7fbee4004660 tx=0x7fbee4009b30).stop 2021-11-03T18:22:38.658 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.659+0000 7fbefa588700 1 -- 172.21.15.27:0/3306773900 shutdown_connections 2021-11-03T18:22:38.659 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.659+0000 7fbefa588700 1 --2- 172.21.15.27:0/3306773900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbef40f74a0 0x7fbef40f5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:38.659 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.659+0000 7fbefa588700 1 --2- 172.21.15.27:0/3306773900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbef40f6b90 0x7fbef40f6fb0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:38.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.659+0000 7fbefa588700 1 -- 172.21.15.27:0/3306773900 >> 172.21.15.27:0/3306773900 conn(0x7fbef40f0540 msgr2=0x7fbef40f2960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:22:38.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.659+0000 7fbefa588700 1 -- 172.21.15.27:0/3306773900 shutdown_connections 2021-11-03T18:22:38.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.659+0000 7fbefa588700 1 -- 172.21.15.27:0/3306773900 wait complete. 2021-11-03T18:22:38.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.660+0000 7fbefa588700 1 Processor -- start 2021-11-03T18:22:38.661 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.660+0000 7fbefa588700 1 -- start start 2021-11-03T18:22:38.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.660+0000 7fbefa588700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbef40f6b90 0x7fbef40f9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:22:38.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.660+0000 7fbefa588700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbef40f74a0 0x7fbef40f7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:22:38.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.660+0000 7fbefa588700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef40f81e0 con 0x7fbef40f74a0 2021-11-03T18:22:38.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.660+0000 7fbefa588700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbef40f8320 con 0x7fbef40f6b90 2021-11-03T18:22:38.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.660+0000 7fbef37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbef40f74a0 0x7fbef40f7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:22:38.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.660+0000 7fbef37fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbef40f74a0 0x7fbef40f7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34518/0 (socket says 172.21.15.27:34518) 2021-11-03T18:22:38.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.660+0000 7fbef37fe700 1 -- 172.21.15.27:0/3131366522 learned_addr learned my addr 172.21.15.27:0/3131366522 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:22:38.665 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.661+0000 7fbef37fe700 1 -- 172.21.15.27:0/3131366522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbef40f6b90 msgr2=0x7fbef40f9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:22:38.665 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.661+0000 7fbef3fff700 1 --2- 172.21.15.27:0/3131366522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbef40f6b90 0x7fbef40f9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:22:38.665 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.661+0000 7fbef37fe700 1 --2- 172.21.15.27:0/3131366522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbef40f6b90 0x7fbef40f9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:38.665 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.661+0000 7fbef37fe700 1 -- 172.21.15.27:0/3131366522 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbee4005040 con 0x7fbef40f74a0 2021-11-03T18:22:38.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.661+0000 7fbef37fe700 1 --2- 172.21.15.27:0/3131366522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbef40f74a0 0x7fbef40f7c80 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 rx=0x7fbee800c530 tx=0x7fbee8007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:22:38.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.661+0000 7fbef0ff9700 1 -- 172.21.15.27:0/3131366522 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbee800f650 con 0x7fbef40f74a0 2021-11-03T18:22:38.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.661+0000 7fbef0ff9700 1 -- 172.21.15.27:0/3131366522 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fbee800fc60 con 0x7fbef40f74a0 2021-11-03T18:22:38.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.661+0000 7fbef0ff9700 1 -- 172.21.15.27:0/3131366522 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbee8012cd0 con 0x7fbef40f74a0 2021-11-03T18:22:38.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.661+0000 7fbefa588700 1 -- 172.21.15.27:0/3131366522 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbef40f8600 con 0x7fbef40f74a0 2021-11-03T18:22:38.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.661+0000 7fbefa588700 1 -- 172.21.15.27:0/3131366522 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbef410efb0 con 0x7fbef40f74a0 2021-11-03T18:22:38.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.662+0000 7fbefa588700 1 -- 172.21.15.27:0/3131366522 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbef4045ad0 con 0x7fbef40f74a0 2021-11-03T18:22:38.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.662+0000 7fbef0ff9700 1 -- 172.21.15.27:0/3131366522 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fbee8010070 con 0x7fbef40f74a0 2021-11-03T18:22:38.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.663+0000 7fbef0ff9700 1 --2- 172.21.15.27:0/3131366522 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbedc0640b0 0x7fbedc066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:22:38.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.663+0000 7fbef0ff9700 1 -- 172.21.15.27:0/3131366522 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fbee80806a0 con 0x7fbef40f74a0 2021-11-03T18:22:38.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.663+0000 7fbef3fff700 1 --2- 172.21.15.27:0/3131366522 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbedc0640b0 0x7fbedc066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:22:38.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.663+0000 7fbef3fff700 1 --2- 172.21.15.27:0/3131366522 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbedc0640b0 0x7fbedc066570 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7fbee4009da0 tx=0x7fbee4009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:22:38.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.667+0000 7fbef0ff9700 1 -- 172.21.15.27:0/3131366522 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fbee80528b0 con 0x7fbef40f74a0 2021-11-03T18:22:38.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:38 smithi115 ceph-mon[65453]: pgmap v2039: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:38.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:38 smithi027 ceph-mon[97244]: pgmap v2039: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:38.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.896+0000 7fbefa588700 1 -- 172.21.15.27:0/3131366522 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbef410f6c0 con 0x7fbef40f74a0 2021-11-03T18:22:38.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.897+0000 7fbef0ff9700 1 -- 172.21.15.27:0/3131366522 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fbee80524a0 con 0x7fbef40f74a0 2021-11-03T18:22:38.895 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:22:38.896 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:22:38.896 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:22:38.896 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:22:38.897 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:22:38.897 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:22:38.897 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:22:38.897 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:22:38.898 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:22:38.898 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:22:38.898 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:22:38.898 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:22:38.899 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:22:38.899 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:22:38.899 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:22:38.899 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:22:38.900 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:22:38.900 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:22:38.901 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.898+0000 7fbefa588700 1 -- 172.21.15.27:0/3131366522 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbedc0640b0 msgr2=0x7fbedc066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:22:38.902 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.898+0000 7fbefa588700 1 --2- 172.21.15.27:0/3131366522 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbedc0640b0 0x7fbedc066570 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 rx=0x7fbee4009da0 tx=0x7fbee4009b30).stop 2021-11-03T18:22:38.902 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.898+0000 7fbefa588700 1 -- 172.21.15.27:0/3131366522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbef40f74a0 msgr2=0x7fbef40f7c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:22:38.902 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.898+0000 7fbefa588700 1 --2- 172.21.15.27:0/3131366522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbef40f74a0 0x7fbef40f7c80 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 rx=0x7fbee800c530 tx=0x7fbee8007e20).stop 2021-11-03T18:22:38.902 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.899+0000 7fbefa588700 1 -- 172.21.15.27:0/3131366522 shutdown_connections 2021-11-03T18:22:38.903 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.899+0000 7fbefa588700 1 --2- 172.21.15.27:0/3131366522 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbedc0640b0 0x7fbedc066570 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:38.903 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.899+0000 7fbefa588700 1 --2- 172.21.15.27:0/3131366522 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbef40f6b90 0x7fbef40f9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:38.903 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.899+0000 7fbefa588700 1 --2- 172.21.15.27:0/3131366522 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbef40f74a0 0x7fbef40f7c80 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:38.903 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.899+0000 7fbefa588700 1 -- 172.21.15.27:0/3131366522 >> 172.21.15.27:0/3131366522 conn(0x7fbef40f0540 msgr2=0x7fbef40f5040 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:22:38.904 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.899+0000 7fbefa588700 1 -- 172.21.15.27:0/3131366522 shutdown_connections 2021-11-03T18:22:38.904 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.899+0000 7fbefa588700 1 -- 172.21.15.27:0/3131366522 wait complete. 2021-11-03T18:22:39.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.999+0000 7fd92c0d6700 1 -- 172.21.15.27:0/3060017800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9240f6860 msgr2=0x7fd9240f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:22:39.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.999+0000 7fd92c0d6700 1 --2- 172.21.15.27:0/3060017800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9240f6860 0x7fd9240f6cc0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 rx=0x7fd918004660 tx=0x7fd918009b30).stop 2021-11-03T18:22:39.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.999+0000 7fd92c0d6700 1 -- 172.21.15.27:0/3060017800 shutdown_connections 2021-11-03T18:22:39.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.999+0000 7fd92c0d6700 1 --2- 172.21.15.27:0/3060017800 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9240f6860 0x7fd9240f6cc0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:39.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.999+0000 7fd92c0d6700 1 --2- 172.21.15.27:0/3060017800 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9240f55c0 0x7fd9240f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:39.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.999+0000 7fd92c0d6700 1 -- 172.21.15.27:0/3060017800 >> 172.21.15.27:0/3060017800 conn(0x7fd9240f0d40 msgr2=0x7fd9240f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:22:39.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.999+0000 7fd92c0d6700 1 -- 172.21.15.27:0/3060017800 shutdown_connections 2021-11-03T18:22:39.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.999+0000 7fd92c0d6700 1 -- 172.21.15.27:0/3060017800 wait complete. 2021-11-03T18:22:39.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:38.999+0000 7fd92c0d6700 1 Processor -- start 2021-11-03T18:22:39.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:39.000+0000 7fd92c0d6700 1 -- start start 2021-11-03T18:22:39.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:39.000+0000 7fd92c0d6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9240f55c0 0x7fd924109c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:22:39.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:39.000+0000 7fd92c0d6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9240f6860 0x7fd92410c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:22:39.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:39.000+0000 7fd92c0d6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9240603e0 con 0x7fd9240f6860 2021-11-03T18:22:39.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:39.000+0000 7fd92c0d6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd924060520 con 0x7fd9240f55c0 2021-11-03T18:22:39.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:39.000+0000 7fd929671700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9240f6860 0x7fd92410c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:22:39.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:39.000+0000 7fd929671700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9240f6860 0x7fd92410c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34530/0 (socket says 172.21.15.27:34530) 2021-11-03T18:22:39.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:39.000+0000 7fd929671700 1 -- 172.21.15.27:0/863931580 learned_addr learned my addr 172.21.15.27:0/863931580 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:22:39.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:39.000+0000 7fd929e72700 1 --2- 172.21.15.27:0/863931580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9240f55c0 0x7fd924109c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:22:39.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:39.000+0000 7fd929671700 1 -- 172.21.15.27:0/863931580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9240f55c0 msgr2=0x7fd924109c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:22:39.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:39.000+0000 7fd929671700 1 --2- 172.21.15.27:0/863931580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9240f55c0 0x7fd924109c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:39.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:39.000+0000 7fd929671700 1 -- 172.21.15.27:0/863931580 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd918005040 con 0x7fd9240f6860 2021-11-03T18:22:39.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:39.000+0000 7fd929e72700 1 --2- 172.21.15.27:0/863931580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9240f55c0 0x7fd924109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:22:39.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:39.000+0000 7fd929671700 1 --2- 172.21.15.27:0/863931580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9240f6860 0x7fd92410c150 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 rx=0x7fd918009990 tx=0x7fd918009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:22:39.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:39.000+0000 7fd9227fc700 1 -- 172.21.15.27:0/863931580 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd918018070 con 0x7fd9240f6860 2021-11-03T18:22:39.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:39.000+0000 7fd9227fc700 1 -- 172.21.15.27:0/863931580 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fd918007ca0 con 0x7fd9240f6860 2021-11-03T18:22:39.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:39.000+0000 7fd92c0d6700 1 -- 172.21.15.27:0/863931580 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd92410c6a0 con 0x7fd9240f6860 2021-11-03T18:22:39.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:39.000+0000 7fd9227fc700 1 -- 172.21.15.27:0/863931580 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd91801c620 con 0x7fd9240f6860 2021-11-03T18:22:39.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:39.000+0000 7fd92c0d6700 1 -- 172.21.15.27:0/863931580 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd92410cc00 con 0x7fd9240f6860 2021-11-03T18:22:39.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:39.001+0000 7fd92c0d6700 1 -- 172.21.15.27:0/863931580 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd924045ad0 con 0x7fd9240f6860 2021-11-03T18:22:39.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:39.002+0000 7fd9227fc700 1 -- 172.21.15.27:0/863931580 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fd91801d960 con 0x7fd9240f6860 2021-11-03T18:22:39.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:39.002+0000 7fd9227fc700 1 --2- 172.21.15.27:0/863931580 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd90c0640b0 0x7fd90c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:22:39.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:39.002+0000 7fd9227fc700 1 -- 172.21.15.27:0/863931580 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fd9180824e0 con 0x7fd9240f6860 2021-11-03T18:22:39.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:39.002+0000 7fd929e72700 1 --2- 172.21.15.27:0/863931580 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd90c0640b0 0x7fd90c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:22:39.009 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:39.003+0000 7fd929e72700 1 --2- 172.21.15.27:0/863931580 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd90c0640b0 0x7fd90c066570 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7fd924060060 tx=0x7fd91c009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:22:39.009 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:39.006+0000 7fd9227fc700 1 -- 172.21.15.27:0/863931580 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fd9180548a0 con 0x7fd9240f6860 2021-11-03T18:22:39.208 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:39.210+0000 7fd92c0d6700 1 -- 172.21.15.27:0/863931580 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fd924027a00 con 0x7fd9240f6860 2021-11-03T18:22:39.209 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:39.211+0000 7fd9227fc700 1 -- 172.21.15.27:0/863931580 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7fd918054490 con 0x7fd9240f6860 2021-11-03T18:22:39.209 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:22:39.210 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:22:39.210 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:22:39.211 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:22:39.211 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:22:39.211 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:22:39.212 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:22:39.212 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:22:39.213 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:22:39.213 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:22:39.213 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:22:39.213 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:22:39.213 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:22:39.214 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:22:39.214 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:22:39.214 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:22:39.214 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:22:39.215 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:22:39.215 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:22:39.215 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:22:39.215 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:22:39.216 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:22:39.216 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:22:39.216 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:22:39.217 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:22:39.217 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:22:39.217 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:22:39.217 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:22:39.218 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:22:39.218 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:22:39.218 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:22:39.218 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:22:39.218 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:22:39.219 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:22:39.219 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:22:39.219 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:22:39.219 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:22:39.220 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:22:39.220 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:22:39.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:39.212+0000 7fd92c0d6700 1 -- 172.21.15.27:0/863931580 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd90c0640b0 msgr2=0x7fd90c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:22:39.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:39.212+0000 7fd92c0d6700 1 --2- 172.21.15.27:0/863931580 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd90c0640b0 0x7fd90c066570 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7fd924060060 tx=0x7fd91c009040).stop 2021-11-03T18:22:39.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:39.212+0000 7fd92c0d6700 1 -- 172.21.15.27:0/863931580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9240f6860 msgr2=0x7fd92410c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:22:39.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:39.212+0000 7fd92c0d6700 1 --2- 172.21.15.27:0/863931580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9240f6860 0x7fd92410c150 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 rx=0x7fd918009990 tx=0x7fd918009b30).stop 2021-11-03T18:22:39.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:39.212+0000 7fd92c0d6700 1 -- 172.21.15.27:0/863931580 shutdown_connections 2021-11-03T18:22:39.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:39.212+0000 7fd92c0d6700 1 --2- 172.21.15.27:0/863931580 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd90c0640b0 0x7fd90c066570 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:39.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:39.212+0000 7fd92c0d6700 1 --2- 172.21.15.27:0/863931580 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9240f55c0 0x7fd924109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:39.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:39.212+0000 7fd92c0d6700 1 --2- 172.21.15.27:0/863931580 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9240f6860 0x7fd92410c150 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:22:39.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:39.212+0000 7fd92c0d6700 1 -- 172.21.15.27:0/863931580 >> 172.21.15.27:0/863931580 conn(0x7fd9240f0d40 msgr2=0x7fd9240fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:22:39.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:39.212+0000 7fd92c0d6700 1 -- 172.21.15.27:0/863931580 shutdown_connections 2021-11-03T18:22:39.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:22:39.212+0000 7fd92c0d6700 1 -- 172.21.15.27:0/863931580 wait complete. 2021-11-03T18:22:39.224 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:22:39.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:39 smithi115 ceph-mon[65453]: from='client.36200 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:22:39.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:39 smithi115 ceph-mon[65453]: from='client.36204 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:22:39.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:39 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3131366522' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:39.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:39 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/863931580' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:22:39.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:39 smithi027 ceph-mon[97244]: from='client.36200 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:22:39.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:39 smithi027 ceph-mon[97244]: from='client.36204 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:22:39.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:39 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3131366522' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:39.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:39 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/863931580' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:22:40.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:40 smithi115 ceph-mon[65453]: pgmap v2040: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:40.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4002.455747 secs 2021-11-03T18:22:40.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:40 smithi027 ceph-mon[97244]: pgmap v2040: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:40.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4002.455747 secs 2021-11-03T18:22:42.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:42 smithi115 ceph-mon[65453]: pgmap v2041: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:42.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:22:42.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:22:42.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:22:42.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:42.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:42.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:42.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:42.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:42 smithi027 ceph-mon[97244]: pgmap v2041: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:42.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:22:42.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:22:42.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:22:42.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:42.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:42.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:42.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:43.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:43 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:22:43.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:43 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:22:44.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:44 smithi115 ceph-mon[65453]: pgmap v2042: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:44.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:44 smithi027 ceph-mon[97244]: pgmap v2042: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:45.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4007.455883 secs 2021-11-03T18:22:45.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4007.455883 secs 2021-11-03T18:22:46.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:46 smithi115 ceph-mon[65453]: pgmap v2043: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:46.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:46 smithi027 ceph-mon[97244]: pgmap v2043: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:48.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:48 smithi115 ceph-mon[65453]: pgmap v2044: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:48.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:48 smithi027 ceph-mon[97244]: pgmap v2044: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:50.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:50 smithi115 ceph-mon[65453]: pgmap v2045: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:50.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4012.456011 secs 2021-11-03T18:22:50.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:50 smithi027 ceph-mon[97244]: pgmap v2045: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:50.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4012.456011 secs 2021-11-03T18:22:52.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:52 smithi115 ceph-mon[65453]: pgmap v2046: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:52.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:22:52.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:22:52.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:22:52.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:52.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:52.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:52.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:52.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:52 smithi027 ceph-mon[97244]: pgmap v2046: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:52.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:22:52.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:22:52.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:22:52.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:52.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:52.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:52.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:22:53.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:53 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:22:53.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:53 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:22:54.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:54 smithi115 ceph-mon[65453]: pgmap v2047: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:54.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:54 smithi027 ceph-mon[97244]: pgmap v2047: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:55.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4017.456152 secs 2021-11-03T18:22:55.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4017.456152 secs 2021-11-03T18:22:56.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:56 smithi027 ceph-mon[97244]: pgmap v2048: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:56.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:22:56.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:22:56.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:56 smithi115 ceph-mon[65453]: pgmap v2048: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:56.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:22:56.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:22:58.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:22:58 smithi027 ceph-mon[97244]: pgmap v2049: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:22:58.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:22:58 smithi115 ceph-mon[65453]: pgmap v2049: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:00.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:00 smithi027 ceph-mon[97244]: pgmap v2050: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:00.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4022.456305 secs 2021-11-03T18:23:00.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:00 smithi115 ceph-mon[65453]: pgmap v2050: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:00.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4022.456305 secs 2021-11-03T18:23:02.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:02 smithi027 ceph-mon[97244]: pgmap v2051: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:02.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:23:02.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:23:02.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:23:02.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:02.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:02.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:02.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:02.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:02 smithi115 ceph-mon[65453]: pgmap v2051: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:02.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:23:02.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:23:02.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:23:02.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:02.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:02.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:02.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:03.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:03 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:23:03.958 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:03 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:23:04.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:04 smithi027 ceph-mon[97244]: pgmap v2052: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:04.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:04 smithi115 ceph-mon[65453]: pgmap v2052: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:05.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4027.456430 secs 2021-11-03T18:23:05.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4027.456430 secs 2021-11-03T18:23:06.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:06 smithi027 ceph-mon[97244]: pgmap v2053: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:06.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:06 smithi115 ceph-mon[65453]: pgmap v2053: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:08.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:08 smithi027 ceph-mon[97244]: pgmap v2054: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:08.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:08 smithi115 ceph-mon[65453]: pgmap v2054: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:09.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.327+0000 7f9fc3dc3700 1 -- 172.21.15.27:0/292391128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fbc06a3b0 msgr2=0x7f9fbc06a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:23:09.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.327+0000 7f9fc3dc3700 1 --2- 172.21.15.27:0/292391128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fbc06a3b0 0x7f9fbc06a830 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 rx=0x7f9fac004690 tx=0x7f9fac009b30).stop 2021-11-03T18:23:09.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.327+0000 7f9fc3dc3700 1 -- 172.21.15.27:0/292391128 shutdown_connections 2021-11-03T18:23:09.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.327+0000 7f9fc3dc3700 1 --2- 172.21.15.27:0/292391128 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fbc06a3b0 0x7f9fbc06a830 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:09.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.327+0000 7f9fc3dc3700 1 --2- 172.21.15.27:0/292391128 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fbc06b920 0x7f9fbc069ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:09.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.327+0000 7f9fc3dc3700 1 -- 172.21.15.27:0/292391128 >> 172.21.15.27:0/292391128 conn(0x7f9fbc0f1f00 msgr2=0x7f9fbc0f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:23:09.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.327+0000 7f9fc3dc3700 1 -- 172.21.15.27:0/292391128 shutdown_connections 2021-11-03T18:23:09.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.327+0000 7f9fc3dc3700 1 -- 172.21.15.27:0/292391128 wait complete. 2021-11-03T18:23:09.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.327+0000 7f9fc3dc3700 1 Processor -- start 2021-11-03T18:23:09.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.327+0000 7f9fc3dc3700 1 -- start start 2021-11-03T18:23:09.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.328+0000 7f9fc3dc3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fbc06a3b0 0x7f9fbc109110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:23:09.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.328+0000 7f9fc3dc3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fbc06b920 0x7f9fbc109600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:23:09.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.328+0000 7f9fc3dc3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc109bb0 con 0x7f9fbc06b920 2021-11-03T18:23:09.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.328+0000 7f9fc3dc3700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc109cf0 con 0x7f9fbc06a3b0 2021-11-03T18:23:09.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.328+0000 7f9fc135e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fbc06b920 0x7f9fbc109600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:23:09.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.328+0000 7f9fc135e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fbc06b920 0x7f9fbc109600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34548/0 (socket says 172.21.15.27:34548) 2021-11-03T18:23:09.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.328+0000 7f9fc135e700 1 -- 172.21.15.27:0/1982891880 learned_addr learned my addr 172.21.15.27:0/1982891880 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:23:09.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.328+0000 7f9fc1b5f700 1 --2- 172.21.15.27:0/1982891880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fbc06a3b0 0x7f9fbc109110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:23:09.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.328+0000 7f9fc135e700 1 -- 172.21.15.27:0/1982891880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fbc06a3b0 msgr2=0x7f9fbc109110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:23:09.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.328+0000 7f9fc135e700 1 --2- 172.21.15.27:0/1982891880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fbc06a3b0 0x7f9fbc109110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:09.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.328+0000 7f9fc135e700 1 -- 172.21.15.27:0/1982891880 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fac005040 con 0x7f9fbc06b920 2021-11-03T18:23:09.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.328+0000 7f9fc1b5f700 1 --2- 172.21.15.27:0/1982891880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fbc06a3b0 0x7f9fbc109110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:23:09.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.328+0000 7f9fc135e700 1 --2- 172.21.15.27:0/1982891880 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fbc06b920 0x7f9fbc109600 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 rx=0x7f9fac007a40 tx=0x7f9fac009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:23:09.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.328+0000 7f9fb27fc700 1 -- 172.21.15.27:0/1982891880 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9fac018070 con 0x7f9fbc06b920 2021-11-03T18:23:09.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.328+0000 7f9fb27fc700 1 -- 172.21.15.27:0/1982891880 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f9fac009630 con 0x7f9fbc06b920 2021-11-03T18:23:09.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.328+0000 7f9fc3dc3700 1 -- 172.21.15.27:0/1982891880 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fbc10a630 con 0x7f9fbc06b920 2021-11-03T18:23:09.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.328+0000 7f9fb27fc700 1 -- 172.21.15.27:0/1982891880 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9fac01c620 con 0x7f9fbc06b920 2021-11-03T18:23:09.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.328+0000 7f9fc3dc3700 1 -- 172.21.15.27:0/1982891880 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fbc10ab90 con 0x7f9fbc06b920 2021-11-03T18:23:09.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.330+0000 7f9fc3dc3700 1 -- 172.21.15.27:0/1982891880 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fbc045ad0 con 0x7f9fbc06b920 2021-11-03T18:23:09.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.330+0000 7f9fb27fc700 1 -- 172.21.15.27:0/1982891880 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f9fac01da60 con 0x7f9fbc06b920 2021-11-03T18:23:09.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.330+0000 7f9fb27fc700 1 --2- 172.21.15.27:0/1982891880 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9fa8064090 0x7f9fa8066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:23:09.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.330+0000 7f9fb27fc700 1 -- 172.21.15.27:0/1982891880 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f9fac0826b0 con 0x7f9fbc06b920 2021-11-03T18:23:09.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.330+0000 7f9fc1b5f700 1 --2- 172.21.15.27:0/1982891880 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9fa8064090 0x7f9fa8066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:23:09.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.331+0000 7f9fc1b5f700 1 --2- 172.21.15.27:0/1982891880 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9fa8064090 0x7f9fa8066550 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7f9fbc06b6f0 tx=0x7f9fb8009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:23:09.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.334+0000 7f9fb27fc700 1 -- 172.21.15.27:0/1982891880 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f9fac0549b0 con 0x7f9fbc06b920 2021-11-03T18:23:09.512 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.514+0000 7f9fc3dc3700 1 -- 172.21.15.27:0/1982891880 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9fbc10ae00 con 0x7f9fa8064090 2021-11-03T18:23:09.513 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.515+0000 7f9fb27fc700 1 -- 172.21.15.27:0/1982891880 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f9fbc10ae00 con 0x7f9fa8064090 2021-11-03T18:23:09.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.517+0000 7f9fc3dc3700 1 -- 172.21.15.27:0/1982891880 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9fa8064090 msgr2=0x7f9fa8066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:23:09.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.517+0000 7f9fc3dc3700 1 --2- 172.21.15.27:0/1982891880 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9fa8064090 0x7f9fa8066550 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7f9fbc06b6f0 tx=0x7f9fb8009040).stop 2021-11-03T18:23:09.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.517+0000 7f9fc3dc3700 1 -- 172.21.15.27:0/1982891880 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fbc06b920 msgr2=0x7f9fbc109600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:23:09.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.517+0000 7f9fc3dc3700 1 --2- 172.21.15.27:0/1982891880 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fbc06b920 0x7f9fbc109600 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 rx=0x7f9fac007a40 tx=0x7f9fac009b30).stop 2021-11-03T18:23:09.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.517+0000 7f9fc3dc3700 1 -- 172.21.15.27:0/1982891880 shutdown_connections 2021-11-03T18:23:09.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.517+0000 7f9fc3dc3700 1 --2- 172.21.15.27:0/1982891880 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9fa8064090 0x7f9fa8066550 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:09.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.517+0000 7f9fc3dc3700 1 --2- 172.21.15.27:0/1982891880 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9fbc06a3b0 0x7f9fbc109110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:09.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.517+0000 7f9fc3dc3700 1 --2- 172.21.15.27:0/1982891880 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9fbc06b920 0x7f9fbc109600 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:09.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.517+0000 7f9fc3dc3700 1 -- 172.21.15.27:0/1982891880 >> 172.21.15.27:0/1982891880 conn(0x7f9fbc0f1f00 msgr2=0x7f9fbc0fcec0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:23:09.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.517+0000 7f9fc3dc3700 1 -- 172.21.15.27:0/1982891880 shutdown_connections 2021-11-03T18:23:09.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.517+0000 7f9fc3dc3700 1 -- 172.21.15.27:0/1982891880 wait complete. 2021-11-03T18:23:09.525 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:23:09.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.618+0000 7f8ff41a8700 1 -- 172.21.15.27:0/543445941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fec06a3b0 msgr2=0x7f8fec06a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:23:09.618 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.618+0000 7f8ff41a8700 1 --2- 172.21.15.27:0/543445941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fec06a3b0 0x7f8fec06a830 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 rx=0x7f8fdc004660 tx=0x7f8fdc009b30).stop 2021-11-03T18:23:09.618 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.618+0000 7f8ff41a8700 1 -- 172.21.15.27:0/543445941 shutdown_connections 2021-11-03T18:23:09.618 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.618+0000 7f8ff41a8700 1 --2- 172.21.15.27:0/543445941 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fec06a3b0 0x7f8fec06a830 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:09.619 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.618+0000 7f8ff41a8700 1 --2- 172.21.15.27:0/543445941 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8fec06b9c0 0x7f8fec069ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:09.619 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.618+0000 7f8ff41a8700 1 -- 172.21.15.27:0/543445941 >> 172.21.15.27:0/543445941 conn(0x7f8fec0f1e20 msgr2=0x7f8fec0f4240 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:23:09.619 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.618+0000 7f8ff41a8700 1 -- 172.21.15.27:0/543445941 shutdown_connections 2021-11-03T18:23:09.620 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.618+0000 7f8ff41a8700 1 -- 172.21.15.27:0/543445941 wait complete. 2021-11-03T18:23:09.620 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.618+0000 7f8ff41a8700 1 Processor -- start 2021-11-03T18:23:09.620 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.618+0000 7f8ff41a8700 1 -- start start 2021-11-03T18:23:09.620 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.619+0000 7f8ff41a8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8fec06a3b0 0x7f8fec10d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:23:09.620 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.619+0000 7f8ff41a8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fec06b9c0 0x7f8fec10da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:23:09.621 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.619+0000 7f8ff41a8700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec10dfe0 con 0x7f8fec06b9c0 2021-11-03T18:23:09.621 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.619+0000 7f8ff41a8700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fec10e120 con 0x7f8fec06a3b0 2021-11-03T18:23:09.621 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.619+0000 7f8ff1743700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fec06b9c0 0x7f8fec10da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:23:09.621 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.619+0000 7f8ff1743700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fec06b9c0 0x7f8fec10da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34558/0 (socket says 172.21.15.27:34558) 2021-11-03T18:23:09.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.619+0000 7f8ff1743700 1 -- 172.21.15.27:0/3841493475 learned_addr learned my addr 172.21.15.27:0/3841493475 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:23:09.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.619+0000 7f8ff1f44700 1 --2- 172.21.15.27:0/3841493475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8fec06a3b0 0x7f8fec10d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:23:09.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.619+0000 7f8ff1743700 1 -- 172.21.15.27:0/3841493475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8fec06a3b0 msgr2=0x7f8fec10d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:23:09.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.619+0000 7f8ff1743700 1 --2- 172.21.15.27:0/3841493475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8fec06a3b0 0x7f8fec10d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:09.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.619+0000 7f8ff1743700 1 -- 172.21.15.27:0/3841493475 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fdc005040 con 0x7f8fec06b9c0 2021-11-03T18:23:09.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.619+0000 7f8ff1743700 1 --2- 172.21.15.27:0/3841493475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fec06b9c0 0x7f8fec10da30 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 rx=0x7f8fdc009990 tx=0x7f8fdc009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:23:09.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.619+0000 7f8fe27fc700 1 -- 172.21.15.27:0/3841493475 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fdc018070 con 0x7f8fec06b9c0 2021-11-03T18:23:09.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.619+0000 7f8fe27fc700 1 -- 172.21.15.27:0/3841493475 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f8fdc007ba0 con 0x7f8fec06b9c0 2021-11-03T18:23:09.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.619+0000 7f8ff41a8700 1 -- 172.21.15.27:0/3841493475 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fec10ea60 con 0x7f8fec06b9c0 2021-11-03T18:23:09.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.619+0000 7f8fe27fc700 1 -- 172.21.15.27:0/3841493475 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fdc01c3c0 con 0x7f8fec06b9c0 2021-11-03T18:23:09.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.619+0000 7f8ff41a8700 1 -- 172.21.15.27:0/3841493475 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fec10efc0 con 0x7f8fec06b9c0 2021-11-03T18:23:09.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.620+0000 7f8ff41a8700 1 -- 172.21.15.27:0/3841493475 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fec045ad0 con 0x7f8fec06b9c0 2021-11-03T18:23:09.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.621+0000 7f8fe27fc700 1 -- 172.21.15.27:0/3841493475 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f8fdc01d420 con 0x7f8fec06b9c0 2021-11-03T18:23:09.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.621+0000 7f8fe27fc700 1 --2- 172.21.15.27:0/3841493475 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8fd80640b0 0x7f8fd8066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:23:09.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.621+0000 7f8fe27fc700 1 -- 172.21.15.27:0/3841493475 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f8fdc0823c0 con 0x7f8fec06b9c0 2021-11-03T18:23:09.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.621+0000 7f8ff1f44700 1 --2- 172.21.15.27:0/3841493475 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8fd80640b0 0x7f8fd8066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:23:09.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.622+0000 7f8ff1f44700 1 --2- 172.21.15.27:0/3841493475 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8fd80640b0 0x7f8fd8066570 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f8fe8004c40 tx=0x7f8fe800b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:23:09.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.625+0000 7f8fe27fc700 1 -- 172.21.15.27:0/3841493475 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f8fdc054720 con 0x7f8fec06b9c0 2021-11-03T18:23:09.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.799+0000 7f8ff41a8700 1 -- 172.21.15.27:0/3841493475 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8fec10f290 con 0x7f8fd80640b0 2021-11-03T18:23:09.812 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.814+0000 7f8fe27fc700 1 -- 172.21.15.27:0/3841493475 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f8fec10f290 con 0x7f8fd80640b0 2021-11-03T18:23:09.812 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:23:09.812 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (74m) 3m ago 75m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:23:09.813 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (66m) 3m ago 75m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:23:09.813 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (66m) 2m ago 75m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:23:09.813 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (74m) 3m ago 75m 42.4M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:23:09.813 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (68m) 3m ago 71m 26.5M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:23:09.814 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (71m) 3m ago 71m 3849M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:23:09.814 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (66m) 2m ago 71m 26.1M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:23:09.814 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (68m) 2m ago 71m 27.2M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:23:09.814 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (68m) 3m ago 77m 590M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:23:09.814 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (67m) 2m ago 75m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:23:09.815 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (67m) 3m ago 77m 312M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:23:09.815 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (67m) 2m ago 75m 145M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:23:09.815 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (75m) 3m ago 75m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:23:09.815 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (74m) 2m ago 74m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:23:09.815 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (66m) 3m ago 74m 392M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:23:09.816 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (65m) 3m ago 74m 427M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:23:09.816 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (65m) 3m ago 73m 320M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:23:09.816 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (64m) 2m ago 73m 454M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:23:09.816 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (64m) 2m ago 73m 359M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:23:09.817 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (63m) 2m ago 72m 368M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:23:09.817 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (74m) 3m ago 75m 59.9M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:23:09.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.815+0000 7f8ff41a8700 1 -- 172.21.15.27:0/3841493475 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8fd80640b0 msgr2=0x7f8fd8066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:23:09.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.815+0000 7f8ff41a8700 1 --2- 172.21.15.27:0/3841493475 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8fd80640b0 0x7f8fd8066570 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 rx=0x7f8fe8004c40 tx=0x7f8fe800b040).stop 2021-11-03T18:23:09.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.816+0000 7f8ff41a8700 1 -- 172.21.15.27:0/3841493475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fec06b9c0 msgr2=0x7f8fec10da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:23:09.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.816+0000 7f8ff41a8700 1 --2- 172.21.15.27:0/3841493475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fec06b9c0 0x7f8fec10da30 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 rx=0x7f8fdc009990 tx=0x7f8fdc009b30).stop 2021-11-03T18:23:09.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.816+0000 7f8ff41a8700 1 -- 172.21.15.27:0/3841493475 shutdown_connections 2021-11-03T18:23:09.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.816+0000 7f8ff41a8700 1 --2- 172.21.15.27:0/3841493475 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8fd80640b0 0x7f8fd8066570 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:09.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.816+0000 7f8ff41a8700 1 --2- 172.21.15.27:0/3841493475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8fec06a3b0 0x7f8fec10d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:09.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.816+0000 7f8ff41a8700 1 --2- 172.21.15.27:0/3841493475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8fec06b9c0 0x7f8fec10da30 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:09.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.816+0000 7f8ff41a8700 1 -- 172.21.15.27:0/3841493475 >> 172.21.15.27:0/3841493475 conn(0x7f8fec0f1e20 msgr2=0x7f8fec0f88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:23:09.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.816+0000 7f8ff41a8700 1 -- 172.21.15.27:0/3841493475 shutdown_connections 2021-11-03T18:23:09.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.816+0000 7f8ff41a8700 1 -- 172.21.15.27:0/3841493475 wait complete. 2021-11-03T18:23:09.915 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.914+0000 7fbc4d61f700 1 -- 172.21.15.27:0/3093324362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc480f5710 msgr2=0x7fbc480f5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:23:09.915 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.914+0000 7fbc4d61f700 1 --2- 172.21.15.27:0/3093324362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc480f5710 0x7fbc480f5b90 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 rx=0x7fbc38004660 tx=0x7fbc38009b30).stop 2021-11-03T18:23:09.916 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.914+0000 7fbc4d61f700 1 -- 172.21.15.27:0/3093324362 shutdown_connections 2021-11-03T18:23:09.916 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.914+0000 7fbc4d61f700 1 --2- 172.21.15.27:0/3093324362 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc480f5710 0x7fbc480f5b90 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:09.916 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.914+0000 7fbc4d61f700 1 --2- 172.21.15.27:0/3093324362 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc480f6a10 0x7fbc480f6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:09.916 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.914+0000 7fbc4d61f700 1 -- 172.21.15.27:0/3093324362 >> 172.21.15.27:0/3093324362 conn(0x7fbc480f0320 msgr2=0x7fbc480f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:23:09.916 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.914+0000 7fbc4d61f700 1 -- 172.21.15.27:0/3093324362 shutdown_connections 2021-11-03T18:23:09.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.914+0000 7fbc4d61f700 1 -- 172.21.15.27:0/3093324362 wait complete. 2021-11-03T18:23:09.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.915+0000 7fbc4d61f700 1 Processor -- start 2021-11-03T18:23:09.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.915+0000 7fbc4d61f700 1 -- start start 2021-11-03T18:23:09.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.915+0000 7fbc4d61f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc480f5710 0x7fbc48109aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:23:09.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.915+0000 7fbc4d61f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc480f6a10 0x7fbc4810bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:23:09.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.915+0000 7fbc4d61f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc480f53c0 con 0x7fbc480f5710 2021-11-03T18:23:09.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.915+0000 7fbc4d61f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc480f5500 con 0x7fbc480f6a10 2021-11-03T18:23:09.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.915+0000 7fbc47fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc480f5710 0x7fbc48109aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:23:09.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.915+0000 7fbc47fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc480f5710 0x7fbc48109aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34566/0 (socket says 172.21.15.27:34566) 2021-11-03T18:23:09.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.915+0000 7fbc47fff700 1 -- 172.21.15.27:0/933787126 learned_addr learned my addr 172.21.15.27:0/933787126 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:23:09.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.916+0000 7fbc477fe700 1 --2- 172.21.15.27:0/933787126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc480f6a10 0x7fbc4810bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:23:09.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.916+0000 7fbc47fff700 1 -- 172.21.15.27:0/933787126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc480f6a10 msgr2=0x7fbc4810bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:23:09.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.916+0000 7fbc47fff700 1 --2- 172.21.15.27:0/933787126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc480f6a10 0x7fbc4810bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:09.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.916+0000 7fbc47fff700 1 -- 172.21.15.27:0/933787126 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc38005040 con 0x7fbc480f5710 2021-11-03T18:23:09.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.916+0000 7fbc47fff700 1 --2- 172.21.15.27:0/933787126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc480f5710 0x7fbc48109aa0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 rx=0x7fbc3c002700 tx=0x7fbc3c009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:23:09.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.916+0000 7fbc44ff9700 1 -- 172.21.15.27:0/933787126 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc3c009320 con 0x7fbc480f5710 2021-11-03T18:23:09.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.916+0000 7fbc44ff9700 1 -- 172.21.15.27:0/933787126 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fbc3c00f040 con 0x7fbc480f5710 2021-11-03T18:23:09.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.916+0000 7fbc4d61f700 1 -- 172.21.15.27:0/933787126 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc4810c550 con 0x7fbc480f5710 2021-11-03T18:23:09.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.916+0000 7fbc4d61f700 1 -- 172.21.15.27:0/933787126 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc4810cb10 con 0x7fbc480f5710 2021-11-03T18:23:09.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.916+0000 7fbc44ff9700 1 -- 172.21.15.27:0/933787126 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc3c014900 con 0x7fbc480f5710 2021-11-03T18:23:09.922 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.917+0000 7fbc4d61f700 1 -- 172.21.15.27:0/933787126 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc48045ad0 con 0x7fbc480f5710 2021-11-03T18:23:09.923 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.917+0000 7fbc44ff9700 1 -- 172.21.15.27:0/933787126 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fbc3c009480 con 0x7fbc480f5710 2021-11-03T18:23:09.923 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.918+0000 7fbc44ff9700 1 --2- 172.21.15.27:0/933787126 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbc30064100 0x7fbc300665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:23:09.923 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.918+0000 7fbc44ff9700 1 -- 172.21.15.27:0/933787126 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fbc3c080ab0 con 0x7fbc480f5710 2021-11-03T18:23:09.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.918+0000 7fbc477fe700 1 --2- 172.21.15.27:0/933787126 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbc30064100 0x7fbc300665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:23:09.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.918+0000 7fbc477fe700 1 --2- 172.21.15.27:0/933787126 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbc30064100 0x7fbc300665c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7fbc38004630 tx=0x7fbc38009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:23:09.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:09.922+0000 7fbc44ff9700 1 -- 172.21.15.27:0/933787126 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fbc3c052db0 con 0x7fbc480f5710 2021-11-03T18:23:10.150 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.152+0000 7fbc4d61f700 1 -- 172.21.15.27:0/933787126 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbc480fdd00 con 0x7fbc480f5710 2021-11-03T18:23:10.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.152+0000 7fbc44ff9700 1 -- 172.21.15.27:0/933787126 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fbc3c0529a0 con 0x7fbc480f5710 2021-11-03T18:23:10.152 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:23:10.153 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:23:10.153 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:23:10.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:23:10.154 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:23:10.155 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:23:10.155 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:23:10.156 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:23:10.156 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:23:10.157 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:23:10.157 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:23:10.157 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:23:10.158 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:23:10.158 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:23:10.159 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:23:10.159 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:23:10.160 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:23:10.161 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:23:10.162 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.154+0000 7fbc4d61f700 1 -- 172.21.15.27:0/933787126 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbc30064100 msgr2=0x7fbc300665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:23:10.163 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.154+0000 7fbc4d61f700 1 --2- 172.21.15.27:0/933787126 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbc30064100 0x7fbc300665c0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7fbc38004630 tx=0x7fbc38009b30).stop 2021-11-03T18:23:10.163 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.154+0000 7fbc4d61f700 1 -- 172.21.15.27:0/933787126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc480f5710 msgr2=0x7fbc48109aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:23:10.164 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.154+0000 7fbc4d61f700 1 --2- 172.21.15.27:0/933787126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc480f5710 0x7fbc48109aa0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 rx=0x7fbc3c002700 tx=0x7fbc3c009d80).stop 2021-11-03T18:23:10.164 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.154+0000 7fbc4d61f700 1 -- 172.21.15.27:0/933787126 shutdown_connections 2021-11-03T18:23:10.164 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.154+0000 7fbc4d61f700 1 --2- 172.21.15.27:0/933787126 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbc30064100 0x7fbc300665c0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:10.165 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.154+0000 7fbc4d61f700 1 --2- 172.21.15.27:0/933787126 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc480f5710 0x7fbc48109aa0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:10.167 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.154+0000 7fbc4d61f700 1 --2- 172.21.15.27:0/933787126 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc480f6a10 0x7fbc4810bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:10.167 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.154+0000 7fbc4d61f700 1 -- 172.21.15.27:0/933787126 >> 172.21.15.27:0/933787126 conn(0x7fbc480f0320 msgr2=0x7fbc480fa170 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:23:10.167 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.154+0000 7fbc4d61f700 1 -- 172.21.15.27:0/933787126 shutdown_connections 2021-11-03T18:23:10.167 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.154+0000 7fbc4d61f700 1 -- 172.21.15.27:0/933787126 wait complete. 2021-11-03T18:23:10.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.254+0000 7f08df0be700 1 -- 172.21.15.27:0/2654119731 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08d80f77d0 msgr2=0x7f08d80f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:23:10.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.254+0000 7f08df0be700 1 --2- 172.21.15.27:0/2654119731 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08d80f77d0 0x7f08d80f7c30 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 rx=0x7f08d4004660 tx=0x7f08d4009b30).stop 2021-11-03T18:23:10.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.254+0000 7f08df0be700 1 -- 172.21.15.27:0/2654119731 shutdown_connections 2021-11-03T18:23:10.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.254+0000 7f08df0be700 1 --2- 172.21.15.27:0/2654119731 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08d80f77d0 0x7f08d80f7c30 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:10.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.254+0000 7f08df0be700 1 --2- 172.21.15.27:0/2654119731 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08d80f6530 0x7f08d80f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:10.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.254+0000 7f08df0be700 1 -- 172.21.15.27:0/2654119731 >> 172.21.15.27:0/2654119731 conn(0x7f08d80f1cd0 msgr2=0x7f08d80f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:23:10.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.254+0000 7f08df0be700 1 -- 172.21.15.27:0/2654119731 shutdown_connections 2021-11-03T18:23:10.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.254+0000 7f08df0be700 1 -- 172.21.15.27:0/2654119731 wait complete. 2021-11-03T18:23:10.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.255+0000 7f08df0be700 1 Processor -- start 2021-11-03T18:23:10.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.255+0000 7f08df0be700 1 -- start start 2021-11-03T18:23:10.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.255+0000 7f08df0be700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08d80f6530 0x7f08d806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:23:10.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.255+0000 7f08df0be700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08d80f77d0 0x7f08d8069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:23:10.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.255+0000 7f08df0be700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d806a380 con 0x7f08d80f77d0 2021-11-03T18:23:10.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.255+0000 7f08df0be700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08d806a4c0 con 0x7f08d80f6530 2021-11-03T18:23:10.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.255+0000 7f08dd8bb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08d80f77d0 0x7f08d8069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:23:10.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.255+0000 7f08dd8bb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08d80f77d0 0x7f08d8069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34578/0 (socket says 172.21.15.27:34578) 2021-11-03T18:23:10.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.255+0000 7f08dd8bb700 1 -- 172.21.15.27:0/2522845271 learned_addr learned my addr 172.21.15.27:0/2522845271 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:23:10.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.255+0000 7f08de0bc700 1 --2- 172.21.15.27:0/2522845271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08d80f6530 0x7f08d806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:23:10.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.255+0000 7f08dd8bb700 1 -- 172.21.15.27:0/2522845271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08d80f6530 msgr2=0x7f08d806b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:23:10.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.255+0000 7f08dd8bb700 1 --2- 172.21.15.27:0/2522845271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08d80f6530 0x7f08d806b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:10.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.255+0000 7f08dd8bb700 1 -- 172.21.15.27:0/2522845271 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d4005040 con 0x7f08d80f77d0 2021-11-03T18:23:10.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.256+0000 7f08dd8bb700 1 --2- 172.21.15.27:0/2522845271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08d80f77d0 0x7f08d8069e20 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 rx=0x7f08d4009990 tx=0x7f08d4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:23:10.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.256+0000 7f08ceffd700 1 -- 172.21.15.27:0/2522845271 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08d4018070 con 0x7f08d80f77d0 2021-11-03T18:23:10.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.256+0000 7f08ceffd700 1 -- 172.21.15.27:0/2522845271 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f08d4007ba0 con 0x7f08d80f77d0 2021-11-03T18:23:10.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.256+0000 7f08df0be700 1 -- 172.21.15.27:0/2522845271 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08d806a740 con 0x7f08d80f77d0 2021-11-03T18:23:10.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.256+0000 7f08ceffd700 1 -- 172.21.15.27:0/2522845271 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08d401c3c0 con 0x7f08d80f77d0 2021-11-03T18:23:10.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.256+0000 7f08df0be700 1 -- 172.21.15.27:0/2522845271 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08d806aca0 con 0x7f08d80f77d0 2021-11-03T18:23:10.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.257+0000 7f08df0be700 1 -- 172.21.15.27:0/2522845271 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08d8045ad0 con 0x7f08d80f77d0 2021-11-03T18:23:10.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.257+0000 7f08ceffd700 1 -- 172.21.15.27:0/2522845271 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f08d401d420 con 0x7f08d80f77d0 2021-11-03T18:23:10.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.257+0000 7f08ceffd700 1 --2- 172.21.15.27:0/2522845271 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f08c4064100 0x7f08c40665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:23:10.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.258+0000 7f08ceffd700 1 -- 172.21.15.27:0/2522845271 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f08d4082560 con 0x7f08d80f77d0 2021-11-03T18:23:10.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.258+0000 7f08de0bc700 1 --2- 172.21.15.27:0/2522845271 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f08c4064100 0x7f08c40665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:23:10.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.258+0000 7f08de0bc700 1 --2- 172.21.15.27:0/2522845271 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f08c4064100 0x7f08c40665c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f08d806b730 tx=0x7f08c8009c00).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:23:10.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.262+0000 7f08ceffd700 1 -- 172.21.15.27:0/2522845271 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f08d40548c0 con 0x7f08d80f77d0 2021-11-03T18:23:10.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.470+0000 7f08df0be700 1 -- 172.21.15.27:0/2522845271 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f08d8027a00 con 0x7f08d80f77d0 2021-11-03T18:23:10.469 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.471+0000 7f08ceffd700 1 -- 172.21.15.27:0/2522845271 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f08d40544b0 con 0x7f08d80f77d0 2021-11-03T18:23:10.469 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:23:10.470 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:23:10.470 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:23:10.470 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:23:10.470 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:23:10.470 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:23:10.471 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:23:10.471 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:23:10.471 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:23:10.471 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:23:10.472 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:23:10.472 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:23:10.472 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:23:10.472 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:23:10.473 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:23:10.473 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:23:10.473 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:23:10.473 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:23:10.473 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:23:10.474 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:23:10.474 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:23:10.474 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:23:10.474 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:23:10.475 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:23:10.475 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:23:10.475 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:23:10.475 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:23:10.476 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:23:10.476 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:23:10.476 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:23:10.476 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:23:10.477 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:23:10.477 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:23:10.477 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:23:10.477 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:23:10.477 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:23:10.478 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:23:10.478 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:23:10.478 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:23:10.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.473+0000 7f08df0be700 1 -- 172.21.15.27:0/2522845271 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f08c4064100 msgr2=0x7f08c40665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:23:10.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.473+0000 7f08df0be700 1 --2- 172.21.15.27:0/2522845271 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f08c4064100 0x7f08c40665c0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7f08d806b730 tx=0x7f08c8009c00).stop 2021-11-03T18:23:10.480 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.473+0000 7f08df0be700 1 -- 172.21.15.27:0/2522845271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08d80f77d0 msgr2=0x7f08d8069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:23:10.480 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.473+0000 7f08df0be700 1 --2- 172.21.15.27:0/2522845271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08d80f77d0 0x7f08d8069e20 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 rx=0x7f08d4009990 tx=0x7f08d4009b30).stop 2021-11-03T18:23:10.480 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.473+0000 7f08df0be700 1 -- 172.21.15.27:0/2522845271 shutdown_connections 2021-11-03T18:23:10.480 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.473+0000 7f08df0be700 1 --2- 172.21.15.27:0/2522845271 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f08c4064100 0x7f08c40665c0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:10.480 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.473+0000 7f08df0be700 1 --2- 172.21.15.27:0/2522845271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08d80f6530 0x7f08d806b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:10.481 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.473+0000 7f08df0be700 1 --2- 172.21.15.27:0/2522845271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08d80f77d0 0x7f08d8069e20 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:10.481 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.473+0000 7f08df0be700 1 -- 172.21.15.27:0/2522845271 >> 172.21.15.27:0/2522845271 conn(0x7f08d80f1cd0 msgr2=0x7f08d80fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:23:10.481 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.473+0000 7f08df0be700 1 -- 172.21.15.27:0/2522845271 shutdown_connections 2021-11-03T18:23:10.481 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:10.473+0000 7f08df0be700 1 -- 172.21.15.27:0/2522845271 wait complete. 2021-11-03T18:23:10.482 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:23:10.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:10 smithi115 ceph-mon[65453]: pgmap v2055: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:10.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4032.456452 secs 2021-11-03T18:23:10.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:10 smithi115 ceph-mon[65453]: from='client.36216 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:23:10.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:10 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/933787126' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:10.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:10 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2522845271' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:23:11.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:10 smithi027 ceph-mon[97244]: pgmap v2055: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:11.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4032.456452 secs 2021-11-03T18:23:11.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:10 smithi027 ceph-mon[97244]: from='client.36216 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:23:11.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:10 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/933787126' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:11.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:10 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2522845271' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:23:11.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:11 smithi115 ceph-mon[65453]: from='client.36220 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:23:12.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:11 smithi027 ceph-mon[97244]: from='client.36220 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:23:12.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:12 smithi027 ceph-mon[97244]: pgmap v2056: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:12.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:23:12.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:23:12.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:23:12.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:12.943 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:12.943 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:12.944 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:12.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:12 smithi115 ceph-mon[65453]: pgmap v2056: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:12.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:23:12.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:23:12.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:23:12.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:12.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:12.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:12.991 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:13.959 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:13 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:23:14.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:13 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:23:14.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:14 smithi115 ceph-mon[65453]: pgmap v2057: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:15.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:14 smithi027 ceph-mon[97244]: pgmap v2057: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:15.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4037.456591 secs 2021-11-03T18:23:16.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4037.456591 secs 2021-11-03T18:23:16.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:16 smithi115 ceph-mon[65453]: pgmap v2058: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:17.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:16 smithi027 ceph-mon[97244]: pgmap v2058: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:18.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:18 smithi115 ceph-mon[65453]: pgmap v2059: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:19.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:18 smithi027 ceph-mon[97244]: pgmap v2059: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:20.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:20 smithi115 ceph-mon[65453]: pgmap v2060: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:20.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4042.456672 secs 2021-11-03T18:23:21.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:20 smithi027 ceph-mon[97244]: pgmap v2060: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:21.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4042.456672 secs 2021-11-03T18:23:22.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:22 smithi115 ceph-mon[65453]: pgmap v2061: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:22.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:23:22.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:23:22.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:23:22.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:22.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:22.741 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:22.741 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:22.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:22 smithi027 ceph-mon[97244]: pgmap v2061: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:22.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:23:22.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:23:22.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:23:22.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:22.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:22.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:22.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:23.959 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:23 smithi115 ceph-mon[65453]: poll took 60179.476 ms: 1 events 2021-11-03T18:23:23.959 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:23 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:23:24.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:23 smithi027 ceph-mon[97244]: poll took 60179.476 ms: 1 events 2021-11-03T18:23:24.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:23 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:23:24.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:24 smithi115 ceph-mon[65453]: pgmap v2062: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:25.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:24 smithi027 ceph-mon[97244]: pgmap v2062: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:25.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4047.456788 secs 2021-11-03T18:23:26.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4047.456788 secs 2021-11-03T18:23:26.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:26 smithi115 ceph-mon[65453]: pgmap v2063: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:27.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:26 smithi027 ceph-mon[97244]: pgmap v2063: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:28.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:28 smithi115 ceph-mon[65453]: pgmap v2064: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:29.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:28 smithi027 ceph-mon[97244]: pgmap v2064: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:30.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:30 smithi115 ceph-mon[65453]: pgmap v2065: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:30.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4052.456932 secs 2021-11-03T18:23:31.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:30 smithi027 ceph-mon[97244]: pgmap v2065: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:31.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4052.456932 secs 2021-11-03T18:23:32.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:32 smithi027 ceph-mon[97244]: pgmap v2066: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:32.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:23:32.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:23:32.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:23:32.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:32.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:32.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:32.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:32.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:32 smithi115 ceph-mon[65453]: pgmap v2066: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:32.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:23:32.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:23:32.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:23:32.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:32.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:32.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:32.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:33.958 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:33 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:23:34.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:33 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:23:34.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:34 smithi115 ceph-mon[65453]: pgmap v2067: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:35.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:34 smithi027 ceph-mon[97244]: pgmap v2067: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:35.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4057.457043 secs 2021-11-03T18:23:36.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4057.457043 secs 2021-11-03T18:23:36.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:36 smithi115 ceph-mon[65453]: pgmap v2068: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:37.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:36 smithi027 ceph-mon[97244]: pgmap v2068: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:38.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:38 smithi115 ceph-mon[65453]: pgmap v2069: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:39.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:38 smithi027 ceph-mon[97244]: pgmap v2069: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:40.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.588+0000 7f2bbcb45700 1 -- 172.21.15.27:0/1100958613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bb80f66e0 msgr2=0x7f2bb80f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:23:40.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.588+0000 7f2bbcb45700 1 --2- 172.21.15.27:0/1100958613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bb80f66e0 0x7f2bb80f6b40 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 rx=0x7f2bac004660 tx=0x7f2bac009b30).stop 2021-11-03T18:23:40.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.588+0000 7f2bbcb45700 1 -- 172.21.15.27:0/1100958613 shutdown_connections 2021-11-03T18:23:40.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.588+0000 7f2bbcb45700 1 --2- 172.21.15.27:0/1100958613 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bb80f66e0 0x7f2bb80f6b40 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:40.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.588+0000 7f2bbcb45700 1 --2- 172.21.15.27:0/1100958613 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bb80f5440 0x7f2bb80f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:40.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.588+0000 7f2bbcb45700 1 -- 172.21.15.27:0/1100958613 >> 172.21.15.27:0/1100958613 conn(0x7f2bb80f0ba0 msgr2=0x7f2bb80f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:23:40.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.588+0000 7f2bbcb45700 1 -- 172.21.15.27:0/1100958613 shutdown_connections 2021-11-03T18:23:40.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.588+0000 7f2bbcb45700 1 -- 172.21.15.27:0/1100958613 wait complete. 2021-11-03T18:23:40.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.588+0000 7f2bbcb45700 1 Processor -- start 2021-11-03T18:23:40.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.589+0000 7f2bbcb45700 1 -- start start 2021-11-03T18:23:40.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.589+0000 7f2bbcb45700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bb80f5440 0x7f2bb8105d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:23:40.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.589+0000 7f2bbcb45700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bb80f66e0 0x7f2bb8106220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:23:40.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.589+0000 7f2bbcb45700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb81067d0 con 0x7f2bb80f5440 2021-11-03T18:23:40.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.589+0000 7f2bbcb45700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb8106fd0 con 0x7f2bb80f66e0 2021-11-03T18:23:40.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.589+0000 7f2bb77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bb80f5440 0x7f2bb8105d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:23:40.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.589+0000 7f2bb77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bb80f5440 0x7f2bb8105d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34594/0 (socket says 172.21.15.27:34594) 2021-11-03T18:23:40.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.589+0000 7f2bb77fe700 1 -- 172.21.15.27:0/2073674564 learned_addr learned my addr 172.21.15.27:0/2073674564 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:23:40.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.589+0000 7f2bb6ffd700 1 --2- 172.21.15.27:0/2073674564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bb80f66e0 0x7f2bb8106220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:23:40.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.589+0000 7f2bb77fe700 1 -- 172.21.15.27:0/2073674564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bb80f66e0 msgr2=0x7f2bb8106220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:23:40.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.589+0000 7f2bb77fe700 1 --2- 172.21.15.27:0/2073674564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bb80f66e0 0x7f2bb8106220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:40.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.589+0000 7f2bb77fe700 1 -- 172.21.15.27:0/2073674564 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bac005040 con 0x7f2bb80f5440 2021-11-03T18:23:40.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.589+0000 7f2bb77fe700 1 --2- 172.21.15.27:0/2073674564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bb80f5440 0x7f2bb8105d30 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 rx=0x7f2ba800e9a0 tx=0x7f2ba8007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:23:40.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.590+0000 7f2b9ffff700 1 -- 172.21.15.27:0/2073674564 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ba800c980 con 0x7f2bb80f5440 2021-11-03T18:23:40.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.590+0000 7f2b9ffff700 1 -- 172.21.15.27:0/2073674564 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f2ba8011470 con 0x7f2bb80f5440 2021-11-03T18:23:40.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.590+0000 7f2bbcb45700 1 -- 172.21.15.27:0/2073674564 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bb8107250 con 0x7f2bb80f5440 2021-11-03T18:23:40.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.590+0000 7f2b9ffff700 1 -- 172.21.15.27:0/2073674564 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2ba8010630 con 0x7f2bb80f5440 2021-11-03T18:23:40.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.590+0000 7f2bbcb45700 1 -- 172.21.15.27:0/2073674564 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bb81077c0 con 0x7f2bb80f5440 2021-11-03T18:23:40.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.591+0000 7f2bbcb45700 1 -- 172.21.15.27:0/2073674564 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bb8059100 con 0x7f2bb80f5440 2021-11-03T18:23:40.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.591+0000 7f2b9ffff700 1 -- 172.21.15.27:0/2073674564 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f2ba80115e0 con 0x7f2bb80f5440 2021-11-03T18:23:40.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.591+0000 7f2b9ffff700 1 --2- 172.21.15.27:0/2073674564 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2ba00640a0 0x7f2ba0066560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:23:40.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.592+0000 7f2b9ffff700 1 -- 172.21.15.27:0/2073674564 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f2ba8083910 con 0x7f2bb80f5440 2021-11-03T18:23:40.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.592+0000 7f2bb6ffd700 1 --2- 172.21.15.27:0/2073674564 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2ba00640a0 0x7f2ba0066560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:23:40.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.592+0000 7f2bb6ffd700 1 --2- 172.21.15.27:0/2073674564 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2ba00640a0 0x7f2ba0066560 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7f2bac004e30 tx=0x7f2bac009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:23:40.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.596+0000 7f2b9ffff700 1 -- 172.21.15.27:0/2073674564 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f2ba8055ac0 con 0x7f2bb80f5440 2021-11-03T18:23:40.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.782+0000 7f2bbcb45700 1 -- 172.21.15.27:0/2073674564 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2bb805f620 con 0x7f2ba00640a0 2021-11-03T18:23:40.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.783+0000 7f2b9ffff700 1 -- 172.21.15.27:0/2073674564 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f2bb805f620 con 0x7f2ba00640a0 2021-11-03T18:23:40.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.785+0000 7f2bbcb45700 1 -- 172.21.15.27:0/2073674564 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2ba00640a0 msgr2=0x7f2ba0066560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:23:40.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.785+0000 7f2bbcb45700 1 --2- 172.21.15.27:0/2073674564 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2ba00640a0 0x7f2ba0066560 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7f2bac004e30 tx=0x7f2bac009b30).stop 2021-11-03T18:23:40.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.785+0000 7f2bbcb45700 1 -- 172.21.15.27:0/2073674564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bb80f5440 msgr2=0x7f2bb8105d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:23:40.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.785+0000 7f2bbcb45700 1 --2- 172.21.15.27:0/2073674564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bb80f5440 0x7f2bb8105d30 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 rx=0x7f2ba800e9a0 tx=0x7f2ba8007e20).stop 2021-11-03T18:23:40.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.785+0000 7f2bbcb45700 1 -- 172.21.15.27:0/2073674564 shutdown_connections 2021-11-03T18:23:40.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.785+0000 7f2bbcb45700 1 --2- 172.21.15.27:0/2073674564 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2ba00640a0 0x7f2ba0066560 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:40.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.785+0000 7f2bbcb45700 1 --2- 172.21.15.27:0/2073674564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bb80f5440 0x7f2bb8105d30 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:40.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.785+0000 7f2bbcb45700 1 --2- 172.21.15.27:0/2073674564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bb80f66e0 0x7f2bb8106220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:40.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.785+0000 7f2bbcb45700 1 -- 172.21.15.27:0/2073674564 >> 172.21.15.27:0/2073674564 conn(0x7f2bb80f0ba0 msgr2=0x7f2bb80fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:23:40.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.785+0000 7f2bbcb45700 1 -- 172.21.15.27:0/2073674564 shutdown_connections 2021-11-03T18:23:40.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.785+0000 7f2bbcb45700 1 -- 172.21.15.27:0/2073674564 wait complete. 2021-11-03T18:23:40.793 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:23:40.883 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.885+0000 7f18a1525700 1 -- 172.21.15.27:0/1310206352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f189c0f66e0 msgr2=0x7f189c0f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:23:40.883 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.885+0000 7f18a1525700 1 --2- 172.21.15.27:0/1310206352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f189c0f66e0 0x7f189c0f6b00 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 rx=0x7f1890004660 tx=0x7f1890009b30).stop 2021-11-03T18:23:40.884 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.885+0000 7f18a1525700 1 -- 172.21.15.27:0/1310206352 shutdown_connections 2021-11-03T18:23:40.884 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.885+0000 7f18a1525700 1 --2- 172.21.15.27:0/1310206352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f189c0f6ff0 0x7f189c0f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:40.884 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.885+0000 7f18a1525700 1 --2- 172.21.15.27:0/1310206352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f189c0f66e0 0x7f189c0f6b00 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:40.884 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.885+0000 7f18a1525700 1 -- 172.21.15.27:0/1310206352 >> 172.21.15.27:0/1310206352 conn(0x7f189c0f1f00 msgr2=0x7f189c0f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:23:40.885 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.885+0000 7f18a1525700 1 -- 172.21.15.27:0/1310206352 shutdown_connections 2021-11-03T18:23:40.885 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.885+0000 7f18a1525700 1 -- 172.21.15.27:0/1310206352 wait complete. 2021-11-03T18:23:40.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.886+0000 7f18a1525700 1 Processor -- start 2021-11-03T18:23:40.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.886+0000 7f18a1525700 1 -- start start 2021-11-03T18:23:40.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.886+0000 7f18a1525700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f189c0f66e0 0x7f189c06b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:23:40.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.886+0000 7f18a1525700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f189c0f6ff0 0x7f189c069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:23:40.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.886+0000 7f18a1525700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189c06a380 con 0x7f189c0f6ff0 2021-11-03T18:23:40.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.886+0000 7f18a1525700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189c06a4c0 con 0x7f189c0f66e0 2021-11-03T18:23:40.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.886+0000 7f189a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f189c0f6ff0 0x7f189c069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:23:40.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.886+0000 7f189a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f189c0f6ff0 0x7f189c069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34604/0 (socket says 172.21.15.27:34604) 2021-11-03T18:23:40.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.886+0000 7f189a7fc700 1 -- 172.21.15.27:0/2439916872 learned_addr learned my addr 172.21.15.27:0/2439916872 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:23:40.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.886+0000 7f189affd700 1 --2- 172.21.15.27:0/2439916872 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f189c0f66e0 0x7f189c06b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:23:40.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.886+0000 7f189a7fc700 1 -- 172.21.15.27:0/2439916872 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f189c0f66e0 msgr2=0x7f189c06b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:23:40.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.886+0000 7f189a7fc700 1 --2- 172.21.15.27:0/2439916872 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f189c0f66e0 0x7f189c06b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:40.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.886+0000 7f189a7fc700 1 -- 172.21.15.27:0/2439916872 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1890005040 con 0x7f189c0f6ff0 2021-11-03T18:23:40.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.886+0000 7f189a7fc700 1 --2- 172.21.15.27:0/2439916872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f189c0f6ff0 0x7f189c069e20 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 rx=0x7f188c0026d0 tx=0x7f188c009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:23:40.891 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.886+0000 7f18837fe700 1 -- 172.21.15.27:0/2439916872 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f188c0079c0 con 0x7f189c0f6ff0 2021-11-03T18:23:40.891 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.887+0000 7f18a1525700 1 -- 172.21.15.27:0/2439916872 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f189c06a7a0 con 0x7f189c0f6ff0 2021-11-03T18:23:40.891 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.887+0000 7f18837fe700 1 -- 172.21.15.27:0/2439916872 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f188c00f040 con 0x7f189c0f6ff0 2021-11-03T18:23:40.891 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.887+0000 7f18837fe700 1 -- 172.21.15.27:0/2439916872 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f188c0148b0 con 0x7f189c0f6ff0 2021-11-03T18:23:40.891 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.887+0000 7f18a1525700 1 -- 172.21.15.27:0/2439916872 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f189c10ef90 con 0x7f189c0f6ff0 2021-11-03T18:23:40.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.888+0000 7f18a1525700 1 -- 172.21.15.27:0/2439916872 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f189c045ad0 con 0x7f189c0f6ff0 2021-11-03T18:23:40.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.888+0000 7f18837fe700 1 -- 172.21.15.27:0/2439916872 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f188c007b20 con 0x7f189c0f6ff0 2021-11-03T18:23:40.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.888+0000 7f18837fe700 1 --2- 172.21.15.27:0/2439916872 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f18840640b0 0x7f1884066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:23:40.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.888+0000 7f18837fe700 1 -- 172.21.15.27:0/2439916872 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f188c080790 con 0x7f189c0f6ff0 2021-11-03T18:23:40.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.888+0000 7f189affd700 1 --2- 172.21.15.27:0/2439916872 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f18840640b0 0x7f1884066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:23:40.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.889+0000 7f189affd700 1 --2- 172.21.15.27:0/2439916872 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f18840640b0 0x7f1884066570 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7f1890004630 tx=0x7f1890009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:23:40.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:40.893+0000 7f18837fe700 1 -- 172.21.15.27:0/2439916872 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f188c052a90 con 0x7f189c0f6ff0 2021-11-03T18:23:40.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:40 smithi115 ceph-mon[65453]: pgmap v2070: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:40.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4062.457130 secs 2021-11-03T18:23:41.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:40 smithi027 ceph-mon[97244]: pgmap v2070: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:41.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4062.457130 secs 2021-11-03T18:23:41.062 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.064+0000 7f18a1525700 1 -- 172.21.15.27:0/2439916872 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f189c0fefe0 con 0x7f18840640b0 2021-11-03T18:23:41.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.079+0000 7f18837fe700 1 -- 172.21.15.27:0/2439916872 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f189c0fefe0 con 0x7f18840640b0 2021-11-03T18:23:41.077 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:23:41.078 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (75m) 4m ago 76m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:23:41.078 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (66m) 4m ago 76m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:23:41.078 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (66m) 3m ago 75m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:23:41.078 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (75m) 4m ago 76m 42.4M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:23:41.078 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (68m) 4m ago 71m 26.5M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:23:41.079 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (71m) 4m ago 71m 3849M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:23:41.079 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (67m) 3m ago 71m 26.1M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:23:41.079 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (69m) 3m ago 71m 27.2M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:23:41.079 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (68m) 4m ago 77m 590M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:23:41.080 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (68m) 3m ago 75m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:23:41.080 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (67m) 4m ago 77m 312M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:23:41.080 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (67m) 3m ago 75m 145M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:23:41.080 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (76m) 4m ago 76m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:23:41.081 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (75m) 3m ago 75m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:23:41.081 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (66m) 4m ago 74m 392M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:23:41.081 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (66m) 4m ago 74m 427M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:23:41.081 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (65m) 4m ago 74m 320M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:23:41.082 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (65m) 3m ago 73m 454M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:23:41.082 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (64m) 3m ago 73m 359M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:23:41.082 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (64m) 3m ago 73m 368M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:23:41.082 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (74m) 4m ago 75m 59.9M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:23:41.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.081+0000 7f18a1525700 1 -- 172.21.15.27:0/2439916872 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f18840640b0 msgr2=0x7f1884066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:23:41.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.081+0000 7f18a1525700 1 --2- 172.21.15.27:0/2439916872 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f18840640b0 0x7f1884066570 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 rx=0x7f1890004630 tx=0x7f1890009b30).stop 2021-11-03T18:23:41.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.081+0000 7f18a1525700 1 -- 172.21.15.27:0/2439916872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f189c0f6ff0 msgr2=0x7f189c069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:23:41.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.081+0000 7f18a1525700 1 --2- 172.21.15.27:0/2439916872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f189c0f6ff0 0x7f189c069e20 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 rx=0x7f188c0026d0 tx=0x7f188c009de0).stop 2021-11-03T18:23:41.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.081+0000 7f18a1525700 1 -- 172.21.15.27:0/2439916872 shutdown_connections 2021-11-03T18:23:41.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.081+0000 7f18a1525700 1 --2- 172.21.15.27:0/2439916872 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f18840640b0 0x7f1884066570 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:41.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.081+0000 7f18a1525700 1 --2- 172.21.15.27:0/2439916872 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f189c0f66e0 0x7f189c06b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:41.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.081+0000 7f18a1525700 1 --2- 172.21.15.27:0/2439916872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f189c0f6ff0 0x7f189c069e20 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:41.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.081+0000 7f18a1525700 1 -- 172.21.15.27:0/2439916872 >> 172.21.15.27:0/2439916872 conn(0x7f189c0f1f00 msgr2=0x7f189c0fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:23:41.086 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.081+0000 7f18a1525700 1 -- 172.21.15.27:0/2439916872 shutdown_connections 2021-11-03T18:23:41.086 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.081+0000 7f18a1525700 1 -- 172.21.15.27:0/2439916872 wait complete. 2021-11-03T18:23:41.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.180+0000 7f20a1026700 1 -- 172.21.15.27:0/3007481367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f209c0f53d0 msgr2=0x7f209c0f7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:23:41.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.180+0000 7f20a1026700 1 --2- 172.21.15.27:0/3007481367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f209c0f53d0 0x7f209c0f7d00 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 rx=0x7f2084004660 tx=0x7f2084009b30).stop 2021-11-03T18:23:41.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.180+0000 7f20a1026700 1 -- 172.21.15.27:0/3007481367 shutdown_connections 2021-11-03T18:23:41.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.180+0000 7f20a1026700 1 --2- 172.21.15.27:0/3007481367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f209c0f53d0 0x7f209c0f7d00 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:41.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.180+0000 7f20a1026700 1 --2- 172.21.15.27:0/3007481367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f209c0f4ac0 0x7f209c0f4ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:41.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.180+0000 7f20a1026700 1 -- 172.21.15.27:0/3007481367 >> 172.21.15.27:0/3007481367 conn(0x7f209c0f0540 msgr2=0x7f209c0f2960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:23:41.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.180+0000 7f20a1026700 1 -- 172.21.15.27:0/3007481367 shutdown_connections 2021-11-03T18:23:41.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.180+0000 7f20a1026700 1 -- 172.21.15.27:0/3007481367 wait complete. 2021-11-03T18:23:41.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.181+0000 7f20a1026700 1 Processor -- start 2021-11-03T18:23:41.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.181+0000 7f20a1026700 1 -- start start 2021-11-03T18:23:41.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.181+0000 7f20a1026700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f209c0f4ac0 0x7f209c1071b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:23:41.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.181+0000 7f20a1026700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f209c0f53d0 0x7f209c1076a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:23:41.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.181+0000 7f20a1026700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209c107c50 con 0x7f209c0f53d0 2021-11-03T18:23:41.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.181+0000 7f20a1026700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f209c107d90 con 0x7f209c0f4ac0 2021-11-03T18:23:41.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.181+0000 7f209a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f209c0f53d0 0x7f209c1076a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:23:41.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.181+0000 7f209a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f209c0f53d0 0x7f209c1076a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34614/0 (socket says 172.21.15.27:34614) 2021-11-03T18:23:41.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.181+0000 7f209a59c700 1 -- 172.21.15.27:0/13793100 learned_addr learned my addr 172.21.15.27:0/13793100 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:23:41.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.181+0000 7f209ad9d700 1 --2- 172.21.15.27:0/13793100 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f209c0f4ac0 0x7f209c1071b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:23:41.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.181+0000 7f209a59c700 1 -- 172.21.15.27:0/13793100 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f209c0f4ac0 msgr2=0x7f209c1071b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:23:41.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.181+0000 7f209a59c700 1 --2- 172.21.15.27:0/13793100 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f209c0f4ac0 0x7f209c1071b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:41.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.181+0000 7f209a59c700 1 -- 172.21.15.27:0/13793100 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2084005040 con 0x7f209c0f53d0 2021-11-03T18:23:41.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.181+0000 7f209ad9d700 1 --2- 172.21.15.27:0/13793100 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f209c0f4ac0 0x7f209c1071b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T18:23:41.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.181+0000 7f209a59c700 1 --2- 172.21.15.27:0/13793100 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f209c0f53d0 0x7f209c1076a0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 rx=0x7f2084009990 tx=0x7f2084009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:23:41.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.182+0000 7f20937fe700 1 -- 172.21.15.27:0/13793100 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2084018070 con 0x7f209c0f53d0 2021-11-03T18:23:41.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.182+0000 7f20937fe700 1 -- 172.21.15.27:0/13793100 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f2084007ca0 con 0x7f209c0f53d0 2021-11-03T18:23:41.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.182+0000 7f20937fe700 1 -- 172.21.15.27:0/13793100 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f208401c620 con 0x7f209c0f53d0 2021-11-03T18:23:41.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.182+0000 7f20a1026700 1 -- 172.21.15.27:0/13793100 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f209c1086d0 con 0x7f209c0f53d0 2021-11-03T18:23:41.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.182+0000 7f20a1026700 1 -- 172.21.15.27:0/13793100 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f209c108c30 con 0x7f209c0f53d0 2021-11-03T18:23:41.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.182+0000 7f20a1026700 1 -- 172.21.15.27:0/13793100 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f209c045ad0 con 0x7f209c0f53d0 2021-11-03T18:23:41.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.183+0000 7f20937fe700 1 -- 172.21.15.27:0/13793100 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f208401d960 con 0x7f209c0f53d0 2021-11-03T18:23:41.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.183+0000 7f20937fe700 1 --2- 172.21.15.27:0/13793100 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f20880640b0 0x7f2088066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:23:41.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.184+0000 7f209ad9d700 1 --2- 172.21.15.27:0/13793100 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f20880640b0 0x7f2088066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:23:41.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.184+0000 7f20937fe700 1 -- 172.21.15.27:0/13793100 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f20840825a0 con 0x7f209c0f53d0 2021-11-03T18:23:41.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.184+0000 7f209ad9d700 1 --2- 172.21.15.27:0/13793100 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f20880640b0 0x7f2088066570 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f2094004fd0 tx=0x7f2094009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:23:41.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.188+0000 7f20937fe700 1 -- 172.21.15.27:0/13793100 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f2084054960 con 0x7f209c0f53d0 2021-11-03T18:23:41.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.416+0000 7f20a1026700 1 -- 172.21.15.27:0/13793100 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f209c05f560 con 0x7f209c0f53d0 2021-11-03T18:23:41.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.416+0000 7f20937fe700 1 -- 172.21.15.27:0/13793100 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f2084054550 con 0x7f209c0f53d0 2021-11-03T18:23:41.415 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:23:41.415 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:23:41.416 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:23:41.416 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:23:41.416 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:23:41.416 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:23:41.417 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:23:41.417 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:23:41.417 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:23:41.417 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:23:41.418 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:23:41.418 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:23:41.418 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:23:41.418 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:23:41.418 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:23:41.419 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:23:41.419 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:23:41.419 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:23:41.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.418+0000 7f20a1026700 1 -- 172.21.15.27:0/13793100 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f20880640b0 msgr2=0x7f2088066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:23:41.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.418+0000 7f20a1026700 1 --2- 172.21.15.27:0/13793100 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f20880640b0 0x7f2088066570 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f2094004fd0 tx=0x7f2094009040).stop 2021-11-03T18:23:41.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.418+0000 7f20a1026700 1 -- 172.21.15.27:0/13793100 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f209c0f53d0 msgr2=0x7f209c1076a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:23:41.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.418+0000 7f20a1026700 1 --2- 172.21.15.27:0/13793100 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f209c0f53d0 0x7f209c1076a0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 rx=0x7f2084009990 tx=0x7f2084009b30).stop 2021-11-03T18:23:41.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.418+0000 7f20a1026700 1 -- 172.21.15.27:0/13793100 shutdown_connections 2021-11-03T18:23:41.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.418+0000 7f20a1026700 1 --2- 172.21.15.27:0/13793100 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f20880640b0 0x7f2088066570 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:41.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.418+0000 7f20a1026700 1 --2- 172.21.15.27:0/13793100 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f209c0f4ac0 0x7f209c1071b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:41.423 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.418+0000 7f20a1026700 1 --2- 172.21.15.27:0/13793100 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f209c0f53d0 0x7f209c1076a0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:41.423 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.418+0000 7f20a1026700 1 -- 172.21.15.27:0/13793100 >> 172.21.15.27:0/13793100 conn(0x7f209c0f0540 msgr2=0x7f209c0fb750 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:23:41.423 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.418+0000 7f20a1026700 1 -- 172.21.15.27:0/13793100 shutdown_connections 2021-11-03T18:23:41.424 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.418+0000 7f20a1026700 1 -- 172.21.15.27:0/13793100 wait complete. 2021-11-03T18:23:41.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.519+0000 7f4f7e5e9700 1 -- 172.21.15.27:0/837279913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f7806b920 msgr2=0x7f4f78069ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:23:41.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.519+0000 7f4f7e5e9700 1 --2- 172.21.15.27:0/837279913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f7806b920 0x7f4f78069ec0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 rx=0x7f4f68004660 tx=0x7f4f68009b30).stop 2021-11-03T18:23:41.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.519+0000 7f4f7e5e9700 1 -- 172.21.15.27:0/837279913 shutdown_connections 2021-11-03T18:23:41.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.519+0000 7f4f7e5e9700 1 --2- 172.21.15.27:0/837279913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f7806a3b0 0x7f4f7806a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:41.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.519+0000 7f4f7e5e9700 1 --2- 172.21.15.27:0/837279913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f7806b920 0x7f4f78069ec0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:41.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.519+0000 7f4f7e5e9700 1 -- 172.21.15.27:0/837279913 >> 172.21.15.27:0/837279913 conn(0x7f4f780f1f00 msgr2=0x7f4f780f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:23:41.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.519+0000 7f4f7e5e9700 1 -- 172.21.15.27:0/837279913 shutdown_connections 2021-11-03T18:23:41.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.519+0000 7f4f7e5e9700 1 -- 172.21.15.27:0/837279913 wait complete. 2021-11-03T18:23:41.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.519+0000 7f4f7e5e9700 1 Processor -- start 2021-11-03T18:23:41.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.519+0000 7f4f7e5e9700 1 -- start start 2021-11-03T18:23:41.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.520+0000 7f4f7e5e9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f7806a3b0 0x7f4f7810d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:23:41.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.520+0000 7f4f7e5e9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f7806b920 0x7f4f7810dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:23:41.524 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.520+0000 7f4f7e5e9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7810e070 con 0x7f4f7806a3b0 2021-11-03T18:23:41.524 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.520+0000 7f4f7e5e9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7810e1b0 con 0x7f4f7806b920 2021-11-03T18:23:41.524 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.520+0000 7f4f77fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f7806a3b0 0x7f4f7810d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:23:41.524 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.520+0000 7f4f77fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f7806a3b0 0x7f4f7810d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34622/0 (socket says 172.21.15.27:34622) 2021-11-03T18:23:41.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.520+0000 7f4f77fff700 1 -- 172.21.15.27:0/4023828532 learned_addr learned my addr 172.21.15.27:0/4023828532 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:23:41.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.520+0000 7f4f777fe700 1 --2- 172.21.15.27:0/4023828532 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f7806b920 0x7f4f7810dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:23:41.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.520+0000 7f4f77fff700 1 -- 172.21.15.27:0/4023828532 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f7806b920 msgr2=0x7f4f7810dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:23:41.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.520+0000 7f4f77fff700 1 --2- 172.21.15.27:0/4023828532 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f7806b920 0x7f4f7810dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:41.526 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.520+0000 7f4f77fff700 1 -- 172.21.15.27:0/4023828532 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f68005040 con 0x7f4f7806a3b0 2021-11-03T18:23:41.526 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.520+0000 7f4f77fff700 1 --2- 172.21.15.27:0/4023828532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f7806a3b0 0x7f4f7810d5d0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 rx=0x7f4f68013a70 tx=0x7f4f68009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:23:41.526 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.520+0000 7f4f74ff9700 1 -- 172.21.15.27:0/4023828532 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f68018070 con 0x7f4f7806a3b0 2021-11-03T18:23:41.526 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.520+0000 7f4f7e5e9700 1 -- 172.21.15.27:0/4023828532 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f7810eaf0 con 0x7f4f7806a3b0 2021-11-03T18:23:41.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.520+0000 7f4f74ff9700 1 -- 172.21.15.27:0/4023828532 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f4f68007ba0 con 0x7f4f7806a3b0 2021-11-03T18:23:41.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.520+0000 7f4f74ff9700 1 -- 172.21.15.27:0/4023828532 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f6801c3c0 con 0x7f4f7806a3b0 2021-11-03T18:23:41.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.520+0000 7f4f7e5e9700 1 -- 172.21.15.27:0/4023828532 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f7810f050 con 0x7f4f7806a3b0 2021-11-03T18:23:41.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.521+0000 7f4f7e5e9700 1 -- 172.21.15.27:0/4023828532 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f78045ad0 con 0x7f4f7806a3b0 2021-11-03T18:23:41.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.522+0000 7f4f74ff9700 1 -- 172.21.15.27:0/4023828532 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f4f6801d420 con 0x7f4f7806a3b0 2021-11-03T18:23:41.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.522+0000 7f4f74ff9700 1 --2- 172.21.15.27:0/4023828532 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4f640640f0 0x7f4f640665b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:23:41.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.522+0000 7f4f74ff9700 1 -- 172.21.15.27:0/4023828532 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f4f680826f0 con 0x7f4f7806a3b0 2021-11-03T18:23:41.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.522+0000 7f4f777fe700 1 --2- 172.21.15.27:0/4023828532 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4f640640f0 0x7f4f640665b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:23:41.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.523+0000 7f4f777fe700 1 --2- 172.21.15.27:0/4023828532 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4f640640f0 0x7f4f640665b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f4f60004a70 tx=0x7f4f6000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:23:41.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.526+0000 7f4f74ff9700 1 -- 172.21.15.27:0/4023828532 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f4f680549f0 con 0x7f4f7806a3b0 2021-11-03T18:23:41.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.731+0000 7f4f7e5e9700 1 -- 172.21.15.27:0/4023828532 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4f781054d0 con 0x7f4f7806a3b0 2021-11-03T18:23:41.730 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.732+0000 7f4f74ff9700 1 -- 172.21.15.27:0/4023828532 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f4f680545e0 con 0x7f4f7806a3b0 2021-11-03T18:23:41.730 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:23:41.731 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:23:41.731 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:23:41.731 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:23:41.731 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:23:41.731 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:23:41.732 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:23:41.732 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:23:41.732 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:23:41.732 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:23:41.733 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:23:41.733 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:23:41.733 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:23:41.733 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:23:41.733 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:23:41.734 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:23:41.734 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:23:41.734 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:23:41.734 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:23:41.735 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:23:41.735 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:23:41.735 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:23:41.735 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:23:41.736 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:23:41.736 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:23:41.736 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:23:41.736 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:23:41.736 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:23:41.737 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:23:41.737 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:23:41.737 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:23:41.737 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:23:41.738 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:23:41.738 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:23:41.738 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:23:41.738 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:23:41.738 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:23:41.739 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:23:41.739 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:23:41.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.733+0000 7f4f7e5e9700 1 -- 172.21.15.27:0/4023828532 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4f640640f0 msgr2=0x7f4f640665b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:23:41.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.733+0000 7f4f7e5e9700 1 --2- 172.21.15.27:0/4023828532 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4f640640f0 0x7f4f640665b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f4f60004a70 tx=0x7f4f6000b040).stop 2021-11-03T18:23:41.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.733+0000 7f4f7e5e9700 1 -- 172.21.15.27:0/4023828532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f7806a3b0 msgr2=0x7f4f7810d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:23:41.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.733+0000 7f4f7e5e9700 1 --2- 172.21.15.27:0/4023828532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f7806a3b0 0x7f4f7810d5d0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 rx=0x7f4f68013a70 tx=0x7f4f68009b30).stop 2021-11-03T18:23:41.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.733+0000 7f4f7e5e9700 1 -- 172.21.15.27:0/4023828532 shutdown_connections 2021-11-03T18:23:41.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.733+0000 7f4f7e5e9700 1 --2- 172.21.15.27:0/4023828532 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4f640640f0 0x7f4f640665b0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:41.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.733+0000 7f4f7e5e9700 1 --2- 172.21.15.27:0/4023828532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f7806a3b0 0x7f4f7810d5d0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:41.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.733+0000 7f4f7e5e9700 1 --2- 172.21.15.27:0/4023828532 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f7806b920 0x7f4f7810dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:23:41.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.733+0000 7f4f7e5e9700 1 -- 172.21.15.27:0/4023828532 >> 172.21.15.27:0/4023828532 conn(0x7f4f780f1f00 msgr2=0x7f4f780f8a70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:23:41.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.734+0000 7f4f7e5e9700 1 -- 172.21.15.27:0/4023828532 shutdown_connections 2021-11-03T18:23:41.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:23:41.734+0000 7f4f7e5e9700 1 -- 172.21.15.27:0/4023828532 wait complete. 2021-11-03T18:23:41.743 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:23:41.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:41 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/13793100' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:42.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:41 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/13793100' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:42.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:42 smithi027 ceph-mon[97244]: from='client.36232 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:23:42.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:42 smithi027 ceph-mon[97244]: pgmap v2071: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:42.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:42 smithi027 ceph-mon[97244]: from='client.36236 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:23:42.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:42 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/4023828532' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:23:42.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:23:42.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:23:42.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:23:42.943 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:42.943 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:42.943 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:42.943 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:42.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:42 smithi115 ceph-mon[65453]: from='client.36232 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:23:42.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:42 smithi115 ceph-mon[65453]: pgmap v2071: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:42.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:42 smithi115 ceph-mon[65453]: from='client.36236 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:23:42.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:42 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/4023828532' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:23:42.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:23:42.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:23:42.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:23:42.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:42.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:42.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:42.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:43.958 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:43 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:23:44.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:43 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:23:44.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:44 smithi115 ceph-mon[65453]: pgmap v2072: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:45.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:44 smithi027 ceph-mon[97244]: pgmap v2072: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:45.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4067.457256 secs 2021-11-03T18:23:46.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4067.457256 secs 2021-11-03T18:23:46.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:46 smithi115 ceph-mon[65453]: pgmap v2073: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:47.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:46 smithi027 ceph-mon[97244]: pgmap v2073: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:48.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:48 smithi115 ceph-mon[65453]: pgmap v2074: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:49.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:48 smithi027 ceph-mon[97244]: pgmap v2074: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:50.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:50 smithi115 ceph-mon[65453]: pgmap v2075: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:50.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4072.457394 secs 2021-11-03T18:23:51.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:50 smithi027 ceph-mon[97244]: pgmap v2075: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:51.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4072.457394 secs 2021-11-03T18:23:52.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:52 smithi027 ceph-mon[97244]: pgmap v2076: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:52.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:23:52.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:23:52.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:23:52.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:52.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:52.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:52.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:52.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:52 smithi115 ceph-mon[65453]: pgmap v2076: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:52.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:23:52.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:23:52.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:23:52.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:52.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:52.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:52.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:23:53.959 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:53 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:23:54.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:53 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:23:54.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:54 smithi115 ceph-mon[65453]: pgmap v2077: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:55.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:54 smithi027 ceph-mon[97244]: pgmap v2077: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:55.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4077.457521 secs 2021-11-03T18:23:56.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4077.457521 secs 2021-11-03T18:23:56.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:56 smithi115 ceph-mon[65453]: pgmap v2078: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:56.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:23:56.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:23:57.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:56 smithi027 ceph-mon[97244]: pgmap v2078: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:57.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:23:57.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:23:58.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:23:58 smithi115 ceph-mon[65453]: pgmap v2079: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:23:59.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:23:58 smithi027 ceph-mon[97244]: pgmap v2079: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:00.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:00 smithi115 ceph-mon[65453]: pgmap v2080: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:00.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4082.457666 secs 2021-11-03T18:24:01.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:00 smithi027 ceph-mon[97244]: pgmap v2080: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:01.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4082.457666 secs 2021-11-03T18:24:02.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:02 smithi027 ceph-mon[97244]: pgmap v2081: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:02.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:24:02.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:24:02.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:24:02.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:02.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:02.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:02.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:02.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:02 smithi115 ceph-mon[65453]: pgmap v2081: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:02.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:24:02.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:24:02.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:24:02.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:02.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:02.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:02.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:03.958 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:03 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:24:04.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:03 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:24:04.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:04 smithi115 ceph-mon[65453]: pgmap v2082: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:05.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:04 smithi027 ceph-mon[97244]: pgmap v2082: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:05.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4087.457817 secs 2021-11-03T18:24:06.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4087.457817 secs 2021-11-03T18:24:06.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:06 smithi115 ceph-mon[65453]: pgmap v2083: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:07.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:06 smithi027 ceph-mon[97244]: pgmap v2083: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:08.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:08 smithi115 ceph-mon[65453]: pgmap v2084: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:09.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:08 smithi027 ceph-mon[97244]: pgmap v2084: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:10.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:10 smithi115 ceph-mon[65453]: pgmap v2085: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:10.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4092.457951 secs 2021-11-03T18:24:11.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:10 smithi027 ceph-mon[97244]: pgmap v2085: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:11.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4092.457951 secs 2021-11-03T18:24:11.847 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:11.847+0000 7f0d358e6700 1 -- 172.21.15.27:0/2046673302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d300f66e0 msgr2=0x7f0d300f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:24:11.847 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:11.847+0000 7f0d358e6700 1 --2- 172.21.15.27:0/2046673302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d300f66e0 0x7f0d300f6b00 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 rx=0x7f0d20004660 tx=0x7f0d20009b30).stop 2021-11-03T18:24:11.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:11.848+0000 7f0d358e6700 1 -- 172.21.15.27:0/2046673302 shutdown_connections 2021-11-03T18:24:11.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:11.848+0000 7f0d358e6700 1 --2- 172.21.15.27:0/2046673302 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d300f6ff0 0x7f0d300f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:11.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:11.848+0000 7f0d358e6700 1 --2- 172.21.15.27:0/2046673302 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d300f66e0 0x7f0d300f6b00 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:11.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:11.848+0000 7f0d358e6700 1 -- 172.21.15.27:0/2046673302 >> 172.21.15.27:0/2046673302 conn(0x7f0d300f1f00 msgr2=0x7f0d300f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:24:11.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:11.848+0000 7f0d358e6700 1 -- 172.21.15.27:0/2046673302 shutdown_connections 2021-11-03T18:24:11.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:11.848+0000 7f0d358e6700 1 -- 172.21.15.27:0/2046673302 wait complete. 2021-11-03T18:24:11.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:11.848+0000 7f0d358e6700 1 Processor -- start 2021-11-03T18:24:11.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:11.848+0000 7f0d358e6700 1 -- start start 2021-11-03T18:24:11.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:11.849+0000 7f0d358e6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d300f66e0 0x7f0d30106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:24:11.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:11.849+0000 7f0d358e6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d300f6ff0 0x7f0d30107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:24:11.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:11.849+0000 7f0d358e6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d30107a00 con 0x7f0d300f6ff0 2021-11-03T18:24:11.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:11.849+0000 7f0d358e6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d30107b40 con 0x7f0d300f66e0 2021-11-03T18:24:11.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:11.849+0000 7f0d2e7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d300f6ff0 0x7f0d30107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:24:11.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:11.849+0000 7f0d2e7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d300f6ff0 0x7f0d30107450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34642/0 (socket says 172.21.15.27:34642) 2021-11-03T18:24:11.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:11.849+0000 7f0d2e7fc700 1 -- 172.21.15.27:0/3213591791 learned_addr learned my addr 172.21.15.27:0/3213591791 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:24:11.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:11.849+0000 7f0d2effd700 1 --2- 172.21.15.27:0/3213591791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d300f66e0 0x7f0d30106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:24:11.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:11.849+0000 7f0d2e7fc700 1 -- 172.21.15.27:0/3213591791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d300f66e0 msgr2=0x7f0d30106f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:24:11.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:11.849+0000 7f0d2e7fc700 1 --2- 172.21.15.27:0/3213591791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d300f66e0 0x7f0d30106f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:11.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:11.849+0000 7f0d2e7fc700 1 -- 172.21.15.27:0/3213591791 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d20005040 con 0x7f0d300f6ff0 2021-11-03T18:24:11.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:11.849+0000 7f0d2e7fc700 1 --2- 172.21.15.27:0/3213591791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d300f6ff0 0x7f0d30107450 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 rx=0x7f0d240026d0 tx=0x7f0d24009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:24:11.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:11.849+0000 7f0d177fe700 1 -- 172.21.15.27:0/3213591791 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d240079c0 con 0x7f0d300f6ff0 2021-11-03T18:24:11.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:11.849+0000 7f0d358e6700 1 -- 172.21.15.27:0/3213591791 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d301084e0 con 0x7f0d300f6ff0 2021-11-03T18:24:11.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:11.849+0000 7f0d177fe700 1 -- 172.21.15.27:0/3213591791 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f0d2400f040 con 0x7f0d300f6ff0 2021-11-03T18:24:11.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:11.849+0000 7f0d177fe700 1 -- 172.21.15.27:0/3213591791 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d240148b0 con 0x7f0d300f6ff0 2021-11-03T18:24:11.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:11.850+0000 7f0d358e6700 1 -- 172.21.15.27:0/3213591791 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d30108aa0 con 0x7f0d300f6ff0 2021-11-03T18:24:11.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:11.850+0000 7f0d358e6700 1 -- 172.21.15.27:0/3213591791 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d30045ad0 con 0x7f0d300f6ff0 2021-11-03T18:24:11.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:11.851+0000 7f0d177fe700 1 -- 172.21.15.27:0/3213591791 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f0d24007b20 con 0x7f0d300f6ff0 2021-11-03T18:24:11.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:11.851+0000 7f0d177fe700 1 --2- 172.21.15.27:0/3213591791 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0d180640b0 0x7f0d18066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:24:11.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:11.852+0000 7f0d177fe700 1 -- 172.21.15.27:0/3213591791 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7f0d240807a0 con 0x7f0d300f6ff0 2021-11-03T18:24:11.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:11.852+0000 7f0d2effd700 1 --2- 172.21.15.27:0/3213591791 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0d180640b0 0x7f0d18066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:24:11.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:11.852+0000 7f0d2effd700 1 --2- 172.21.15.27:0/3213591791 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0d180640b0 0x7f0d18066570 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7f0d20004630 tx=0x7f0d20009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:24:11.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:11.856+0000 7f0d177fe700 1 -- 172.21.15.27:0/3213591791 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f0d24052aa0 con 0x7f0d300f6ff0 2021-11-03T18:24:12.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.034+0000 7f0d358e6700 1 -- 172.21.15.27:0/3213591791 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d30108d80 con 0x7f0d180640b0 2021-11-03T18:24:12.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.035+0000 7f0d177fe700 1 -- 172.21.15.27:0/3213591791 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f0d30108d80 con 0x7f0d180640b0 2021-11-03T18:24:12.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.037+0000 7f0d358e6700 1 -- 172.21.15.27:0/3213591791 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0d180640b0 msgr2=0x7f0d18066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:24:12.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.037+0000 7f0d358e6700 1 --2- 172.21.15.27:0/3213591791 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0d180640b0 0x7f0d18066570 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 rx=0x7f0d20004630 tx=0x7f0d20009b30).stop 2021-11-03T18:24:12.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.037+0000 7f0d358e6700 1 -- 172.21.15.27:0/3213591791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d300f6ff0 msgr2=0x7f0d30107450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:24:12.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.037+0000 7f0d358e6700 1 --2- 172.21.15.27:0/3213591791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d300f6ff0 0x7f0d30107450 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 rx=0x7f0d240026d0 tx=0x7f0d24009de0).stop 2021-11-03T18:24:12.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.037+0000 7f0d358e6700 1 -- 172.21.15.27:0/3213591791 shutdown_connections 2021-11-03T18:24:12.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.037+0000 7f0d358e6700 1 --2- 172.21.15.27:0/3213591791 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0d180640b0 0x7f0d18066570 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:12.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.037+0000 7f0d358e6700 1 --2- 172.21.15.27:0/3213591791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d300f66e0 0x7f0d30106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:12.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.037+0000 7f0d358e6700 1 --2- 172.21.15.27:0/3213591791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d300f6ff0 0x7f0d30107450 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:12.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.037+0000 7f0d358e6700 1 -- 172.21.15.27:0/3213591791 >> 172.21.15.27:0/3213591791 conn(0x7f0d300f1f00 msgr2=0x7f0d300fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:24:12.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.038+0000 7f0d358e6700 1 -- 172.21.15.27:0/3213591791 shutdown_connections 2021-11-03T18:24:12.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.038+0000 7f0d358e6700 1 -- 172.21.15.27:0/3213591791 wait complete. 2021-11-03T18:24:12.046 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:24:12.139 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.138+0000 7feeaf332700 1 -- 172.21.15.27:0/1000730177 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feea8060370 msgr2=0x7feea80fb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:24:12.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.138+0000 7feeaf332700 1 --2- 172.21.15.27:0/1000730177 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feea8060370 0x7feea80fb760 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 rx=0x7feea0009a10 tx=0x7feea0004d10).stop 2021-11-03T18:24:12.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.138+0000 7feeaf332700 1 -- 172.21.15.27:0/1000730177 shutdown_connections 2021-11-03T18:24:12.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.138+0000 7feeaf332700 1 --2- 172.21.15.27:0/1000730177 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feea80fbc50 0x7feea80fe040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:12.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.138+0000 7feeaf332700 1 --2- 172.21.15.27:0/1000730177 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feea8060370 0x7feea80fb760 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:12.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.138+0000 7feeaf332700 1 -- 172.21.15.27:0/1000730177 >> 172.21.15.27:0/1000730177 conn(0x7feea80f0de0 msgr2=0x7feea80f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:24:12.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.138+0000 7feeaf332700 1 -- 172.21.15.27:0/1000730177 shutdown_connections 2021-11-03T18:24:12.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.138+0000 7feeaf332700 1 -- 172.21.15.27:0/1000730177 wait complete. 2021-11-03T18:24:12.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.138+0000 7feeaf332700 1 Processor -- start 2021-11-03T18:24:12.142 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.138+0000 7feeaf332700 1 -- start start 2021-11-03T18:24:12.142 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.138+0000 7feeaf332700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feea8060370 0x7feea81092b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:24:12.142 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.139+0000 7feeaf332700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feea80fbc50 0x7feea81097a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:24:12.142 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.139+0000 7feeaf332700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea8109d50 con 0x7feea80fbc50 2021-11-03T18:24:12.143 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.139+0000 7feeaf332700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea8109e90 con 0x7feea8060370 2021-11-03T18:24:12.143 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.139+0000 7feeac8cd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feea80fbc50 0x7feea81097a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:24:12.143 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.139+0000 7feeac8cd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feea80fbc50 0x7feea81097a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34652/0 (socket says 172.21.15.27:34652) 2021-11-03T18:24:12.143 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.139+0000 7feeac8cd700 1 -- 172.21.15.27:0/2242726738 learned_addr learned my addr 172.21.15.27:0/2242726738 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:24:12.143 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.139+0000 7feead0ce700 1 --2- 172.21.15.27:0/2242726738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feea8060370 0x7feea81092b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:24:12.144 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.139+0000 7feeac8cd700 1 -- 172.21.15.27:0/2242726738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feea8060370 msgr2=0x7feea81092b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:24:12.144 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.139+0000 7feeac8cd700 1 --2- 172.21.15.27:0/2242726738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feea8060370 0x7feea81092b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:12.144 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.139+0000 7feeac8cd700 1 -- 172.21.15.27:0/2242726738 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea0009640 con 0x7feea80fbc50 2021-11-03T18:24:12.144 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.139+0000 7feeac8cd700 1 --2- 172.21.15.27:0/2242726738 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feea80fbc50 0x7feea81097a0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 rx=0x7fee9c0026d0 tx=0x7fee9c009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:24:12.145 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.139+0000 7feea5ffb700 1 -- 172.21.15.27:0/2242726738 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee9c0079c0 con 0x7feea80fbc50 2021-11-03T18:24:12.145 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.139+0000 7feea5ffb700 1 -- 172.21.15.27:0/2242726738 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fee9c00f040 con 0x7feea80fbc50 2021-11-03T18:24:12.145 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.139+0000 7feeaf332700 1 -- 172.21.15.27:0/2242726738 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feea810a830 con 0x7feea80fbc50 2021-11-03T18:24:12.145 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.139+0000 7feea5ffb700 1 -- 172.21.15.27:0/2242726738 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee9c0148b0 con 0x7feea80fbc50 2021-11-03T18:24:12.145 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.140+0000 7feeaf332700 1 -- 172.21.15.27:0/2242726738 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feea810adf0 con 0x7feea80fbc50 2021-11-03T18:24:12.146 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.140+0000 7feeaf332700 1 -- 172.21.15.27:0/2242726738 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feea8045ad0 con 0x7feea80fbc50 2021-11-03T18:24:12.146 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.141+0000 7feea5ffb700 1 -- 172.21.15.27:0/2242726738 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fee9c007b20 con 0x7feea80fbc50 2021-11-03T18:24:12.147 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.141+0000 7feea5ffb700 1 --2- 172.21.15.27:0/2242726738 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fee900640b0 0x7fee90066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:24:12.147 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.141+0000 7feea5ffb700 1 -- 172.21.15.27:0/2242726738 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fee9c0806c0 con 0x7feea80fbc50 2021-11-03T18:24:12.147 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.141+0000 7feead0ce700 1 --2- 172.21.15.27:0/2242726738 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fee900640b0 0x7fee90066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:24:12.148 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.142+0000 7feead0ce700 1 --2- 172.21.15.27:0/2242726738 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fee900640b0 0x7fee90066570 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7feea0003380 tx=0x7feea0004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:24:12.148 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.145+0000 7feea5ffb700 1 -- 172.21.15.27:0/2242726738 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fee9c0529c0 con 0x7feea80fbc50 2021-11-03T18:24:12.314 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.315+0000 7feeaf332700 1 -- 172.21.15.27:0/2242726738 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7feea810b0c0 con 0x7fee900640b0 2021-11-03T18:24:12.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.330+0000 7feea5ffb700 1 -- 172.21.15.27:0/2242726738 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7feea810b0c0 con 0x7fee900640b0 2021-11-03T18:24:12.329 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:24:12.330 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (75m) 4m ago 77m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:24:12.330 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (67m) 4m ago 76m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:24:12.330 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (67m) 4m ago 76m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:24:12.330 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (75m) 4m ago 76m 42.4M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:24:12.330 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (69m) 4m ago 72m 26.5M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:24:12.331 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (72m) 4m ago 72m 3849M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:24:12.331 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (67m) 4m ago 72m 26.1M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:24:12.331 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (69m) 4m ago 72m 27.2M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:24:12.331 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (69m) 4m ago 78m 590M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:24:12.332 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (68m) 4m ago 76m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:24:12.332 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (68m) 4m ago 78m 312M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:24:12.332 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (68m) 4m ago 76m 145M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:24:12.332 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (76m) 4m ago 76m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:24:12.332 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (75m) 4m ago 75m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:24:12.333 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (67m) 4m ago 75m 392M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:24:12.333 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (66m) 4m ago 75m 427M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:24:12.333 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (66m) 4m ago 74m 320M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:24:12.333 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (65m) 4m ago 74m 454M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:24:12.334 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (65m) 4m ago 74m 359M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:24:12.334 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (64m) 4m ago 73m 368M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:24:12.334 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (75m) 4m ago 76m 59.9M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:24:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.332+0000 7feeaf332700 1 -- 172.21.15.27:0/2242726738 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fee900640b0 msgr2=0x7fee90066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:24:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.332+0000 7feeaf332700 1 --2- 172.21.15.27:0/2242726738 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fee900640b0 0x7fee90066570 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7feea0003380 tx=0x7feea0004d10).stop 2021-11-03T18:24:12.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.332+0000 7feeaf332700 1 -- 172.21.15.27:0/2242726738 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feea80fbc50 msgr2=0x7feea81097a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:24:12.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.332+0000 7feeaf332700 1 --2- 172.21.15.27:0/2242726738 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feea80fbc50 0x7feea81097a0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 rx=0x7fee9c0026d0 tx=0x7fee9c009de0).stop 2021-11-03T18:24:12.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.332+0000 7feeaf332700 1 -- 172.21.15.27:0/2242726738 shutdown_connections 2021-11-03T18:24:12.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.332+0000 7feeaf332700 1 --2- 172.21.15.27:0/2242726738 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fee900640b0 0x7fee90066570 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:12.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.332+0000 7feeaf332700 1 --2- 172.21.15.27:0/2242726738 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feea8060370 0x7feea81092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:12.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.332+0000 7feeaf332700 1 --2- 172.21.15.27:0/2242726738 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feea80fbc50 0x7feea81097a0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:12.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.332+0000 7feeaf332700 1 -- 172.21.15.27:0/2242726738 >> 172.21.15.27:0/2242726738 conn(0x7feea80f0de0 msgr2=0x7feea80f31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:24:12.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.332+0000 7feeaf332700 1 -- 172.21.15.27:0/2242726738 shutdown_connections 2021-11-03T18:24:12.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.332+0000 7feeaf332700 1 -- 172.21.15.27:0/2242726738 wait complete. 2021-11-03T18:24:12.434 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.433+0000 7fd98a5be700 1 -- 172.21.15.27:0/2309861076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9840f55c0 msgr2=0x7fd9840f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:24:12.434 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.433+0000 7fd98a5be700 1 --2- 172.21.15.27:0/2309861076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9840f55c0 0x7fd9840f59e0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 rx=0x7fd974004660 tx=0x7fd974009b30).stop 2021-11-03T18:24:12.434 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.433+0000 7fd98a5be700 1 -- 172.21.15.27:0/2309861076 shutdown_connections 2021-11-03T18:24:12.435 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.433+0000 7fd98a5be700 1 --2- 172.21.15.27:0/2309861076 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9840f6860 0x7fd9840f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:12.435 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.433+0000 7fd98a5be700 1 --2- 172.21.15.27:0/2309861076 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9840f55c0 0x7fd9840f59e0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:12.435 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.433+0000 7fd98a5be700 1 -- 172.21.15.27:0/2309861076 >> 172.21.15.27:0/2309861076 conn(0x7fd9840f0d40 msgr2=0x7fd9840f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:24:12.435 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.433+0000 7fd98a5be700 1 -- 172.21.15.27:0/2309861076 shutdown_connections 2021-11-03T18:24:12.436 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.433+0000 7fd98a5be700 1 -- 172.21.15.27:0/2309861076 wait complete. 2021-11-03T18:24:12.436 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.433+0000 7fd98a5be700 1 Processor -- start 2021-11-03T18:24:12.436 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.433+0000 7fd98a5be700 1 -- start start 2021-11-03T18:24:12.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.434+0000 7fd98a5be700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9840f55c0 0x7fd984106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:24:12.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.434+0000 7fd98a5be700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9840f6860 0x7fd984107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:24:12.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.434+0000 7fd98a5be700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd984107a10 con 0x7fd9840f55c0 2021-11-03T18:24:12.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.434+0000 7fd98a5be700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd984107b50 con 0x7fd9840f6860 2021-11-03T18:24:12.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.434+0000 7fd983fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9840f55c0 0x7fd984106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:24:12.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.434+0000 7fd983fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9840f55c0 0x7fd984106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34660/0 (socket says 172.21.15.27:34660) 2021-11-03T18:24:12.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.434+0000 7fd983fff700 1 -- 172.21.15.27:0/2148971663 learned_addr learned my addr 172.21.15.27:0/2148971663 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:24:12.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.434+0000 7fd9837fe700 1 --2- 172.21.15.27:0/2148971663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9840f6860 0x7fd984107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:24:12.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.434+0000 7fd983fff700 1 -- 172.21.15.27:0/2148971663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9840f6860 msgr2=0x7fd984107460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:24:12.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.434+0000 7fd983fff700 1 --2- 172.21.15.27:0/2148971663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9840f6860 0x7fd984107460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:12.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.434+0000 7fd983fff700 1 -- 172.21.15.27:0/2148971663 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd974005040 con 0x7fd9840f55c0 2021-11-03T18:24:12.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.434+0000 7fd983fff700 1 --2- 172.21.15.27:0/2148971663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9840f55c0 0x7fd984106f70 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 rx=0x7fd974004f60 tx=0x7fd974009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:24:12.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.434+0000 7fd980ff9700 1 -- 172.21.15.27:0/2148971663 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd974018070 con 0x7fd9840f55c0 2021-11-03T18:24:12.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.434+0000 7fd980ff9700 1 -- 172.21.15.27:0/2148971663 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fd974007ba0 con 0x7fd9840f55c0 2021-11-03T18:24:12.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.434+0000 7fd98a5be700 1 -- 172.21.15.27:0/2148971663 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd984108490 con 0x7fd9840f55c0 2021-11-03T18:24:12.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.434+0000 7fd980ff9700 1 -- 172.21.15.27:0/2148971663 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd97401c620 con 0x7fd9840f55c0 2021-11-03T18:24:12.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.434+0000 7fd98a5be700 1 -- 172.21.15.27:0/2148971663 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9841089f0 con 0x7fd9840f55c0 2021-11-03T18:24:12.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.435+0000 7fd98a5be700 1 -- 172.21.15.27:0/2148971663 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd984045ad0 con 0x7fd9840f55c0 2021-11-03T18:24:12.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.436+0000 7fd980ff9700 1 -- 172.21.15.27:0/2148971663 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fd97401c8e0 con 0x7fd9840f55c0 2021-11-03T18:24:12.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.436+0000 7fd980ff9700 1 --2- 172.21.15.27:0/2148971663 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd96c0640b0 0x7fd96c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:24:12.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.436+0000 7fd980ff9700 1 -- 172.21.15.27:0/2148971663 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fd974082720 con 0x7fd9840f55c0 2021-11-03T18:24:12.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.436+0000 7fd9837fe700 1 --2- 172.21.15.27:0/2148971663 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd96c0640b0 0x7fd96c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:24:12.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.437+0000 7fd9837fe700 1 --2- 172.21.15.27:0/2148971663 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd96c0640b0 0x7fd96c066570 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7fd9780041d0 tx=0x7fd978009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:24:12.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.440+0000 7fd980ff9700 1 -- 172.21.15.27:0/2148971663 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fd974054a20 con 0x7fd9840f55c0 2021-11-03T18:24:12.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.672+0000 7fd98a5be700 1 -- 172.21.15.27:0/2148971663 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd9841091b0 con 0x7fd9840f55c0 2021-11-03T18:24:12.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.673+0000 7fd980ff9700 1 -- 172.21.15.27:0/2148971663 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fd974054610 con 0x7fd9840f55c0 2021-11-03T18:24:12.673 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:24:12.673 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:24:12.673 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:24:12.673 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:24:12.674 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:24:12.674 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:24:12.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:24:12.674 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:24:12.675 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:24:12.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:24:12.675 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:24:12.675 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:24:12.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:24:12.676 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:24:12.676 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:24:12.676 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:24:12.676 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:24:12.677 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:24:12.677 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.675+0000 7fd98a5be700 1 -- 172.21.15.27:0/2148971663 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd96c0640b0 msgr2=0x7fd96c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:24:12.678 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.675+0000 7fd98a5be700 1 --2- 172.21.15.27:0/2148971663 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd96c0640b0 0x7fd96c066570 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 rx=0x7fd9780041d0 tx=0x7fd978009040).stop 2021-11-03T18:24:12.678 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.675+0000 7fd98a5be700 1 -- 172.21.15.27:0/2148971663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9840f55c0 msgr2=0x7fd984106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:24:12.678 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.675+0000 7fd98a5be700 1 --2- 172.21.15.27:0/2148971663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9840f55c0 0x7fd984106f70 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 rx=0x7fd974004f60 tx=0x7fd974009b30).stop 2021-11-03T18:24:12.678 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.675+0000 7fd98a5be700 1 -- 172.21.15.27:0/2148971663 shutdown_connections 2021-11-03T18:24:12.679 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.675+0000 7fd98a5be700 1 --2- 172.21.15.27:0/2148971663 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd96c0640b0 0x7fd96c066570 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:12.679 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.675+0000 7fd98a5be700 1 --2- 172.21.15.27:0/2148971663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9840f55c0 0x7fd984106f70 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:12.679 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.675+0000 7fd98a5be700 1 --2- 172.21.15.27:0/2148971663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9840f6860 0x7fd984107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:12.679 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.675+0000 7fd98a5be700 1 -- 172.21.15.27:0/2148971663 >> 172.21.15.27:0/2148971663 conn(0x7fd9840f0d40 msgr2=0x7fd98405fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:24:12.680 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.675+0000 7fd98a5be700 1 -- 172.21.15.27:0/2148971663 shutdown_connections 2021-11-03T18:24:12.680 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.675+0000 7fd98a5be700 1 -- 172.21.15.27:0/2148971663 wait complete. 2021-11-03T18:24:12.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.775+0000 7fa816aa4700 1 -- 172.21.15.27:0/1040957483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8100f5340 msgr2=0x7fa8100f7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:24:12.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.775+0000 7fa816aa4700 1 --2- 172.21.15.27:0/1040957483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8100f5340 0x7fa8100f7ac0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 rx=0x7fa7fc004660 tx=0x7fa7fc009b30).stop 2021-11-03T18:24:12.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.776+0000 7fa816aa4700 1 -- 172.21.15.27:0/1040957483 shutdown_connections 2021-11-03T18:24:12.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.776+0000 7fa816aa4700 1 --2- 172.21.15.27:0/1040957483 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8100f5340 0x7fa8100f7ac0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:12.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.776+0000 7fa816aa4700 1 --2- 172.21.15.27:0/1040957483 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8100f49a0 0x7fa8100f4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T18:24:12.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.776+0000 7fa816aa4700 1 -- 172.21.15.27:0/1040957483 >> 172.21.15.27:0/1040957483 conn(0x7fa8100f04c0 msgr2=0x7fa8100f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:24:12.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.776+0000 7fa816aa4700 1 -- 172.21.15.27:0/1040957483 shutdown_connections 2021-11-03T18:24:12.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.776+0000 7fa816aa4700 1 -- 172.21.15.27:0/1040957483 wait complete. 2021-11-03T18:24:12.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.776+0000 7fa816aa4700 1 Processor -- start 2021-11-03T18:24:12.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.776+0000 7fa816aa4700 1 -- start start 2021-11-03T18:24:12.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.776+0000 7fa816aa4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8100f49a0 0x7fa8100f7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:24:12.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.776+0000 7fa816aa4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8100f5340 0x7fa8100f59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:24:12.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.776+0000 7fa816aa4700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8100f5fc0 con 0x7fa8100f49a0 2021-11-03T18:24:12.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.776+0000 7fa816aa4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8100f6100 con 0x7fa8100f5340 2021-11-03T18:24:12.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.777+0000 7fa814840700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8100f49a0 0x7fa8100f7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:24:12.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.777+0000 7fa814840700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8100f49a0 0x7fa8100f7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34670/0 (socket says 172.21.15.27:34670) 2021-11-03T18:24:12.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.777+0000 7fa814840700 1 -- 172.21.15.27:0/3817136305 learned_addr learned my addr 172.21.15.27:0/3817136305 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:24:12.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.777+0000 7fa80ffff700 1 --2- 172.21.15.27:0/3817136305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8100f5340 0x7fa8100f59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:24:12.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.777+0000 7fa814840700 1 -- 172.21.15.27:0/3817136305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8100f5340 msgr2=0x7fa8100f59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:24:12.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.777+0000 7fa814840700 1 --2- 172.21.15.27:0/3817136305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8100f5340 0x7fa8100f59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:12.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.777+0000 7fa814840700 1 -- 172.21.15.27:0/3817136305 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7fc005040 con 0x7fa8100f49a0 2021-11-03T18:24:12.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.777+0000 7fa80ffff700 1 --2- 172.21.15.27:0/3817136305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8100f5340 0x7fa8100f59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T18:24:12.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.777+0000 7fa814840700 1 --2- 172.21.15.27:0/3817136305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8100f49a0 0x7fa8100f7550 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 rx=0x7fa8040093f0 tx=0x7fa804006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:24:12.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.777+0000 7fa80d7fa700 1 -- 172.21.15.27:0/3817136305 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa804009ea0 con 0x7fa8100f49a0 2021-11-03T18:24:12.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.777+0000 7fa80d7fa700 1 -- 172.21.15.27:0/3817136305 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fa804013af0 con 0x7fa8100f49a0 2021-11-03T18:24:12.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.777+0000 7fa816aa4700 1 -- 172.21.15.27:0/3817136305 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8100f63e0 con 0x7fa8100f49a0 2021-11-03T18:24:12.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.777+0000 7fa80d7fa700 1 -- 172.21.15.27:0/3817136305 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa80400db10 con 0x7fa8100f49a0 2021-11-03T18:24:12.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.777+0000 7fa816aa4700 1 -- 172.21.15.27:0/3817136305 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa81010f000 con 0x7fa8100f49a0 2021-11-03T18:24:12.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.778+0000 7fa816aa4700 1 -- 172.21.15.27:0/3817136305 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa810045ad0 con 0x7fa8100f49a0 2021-11-03T18:24:12.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.779+0000 7fa80d7fa700 1 -- 172.21.15.27:0/3817136305 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fa80401a060 con 0x7fa8100f49a0 2021-11-03T18:24:12.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.779+0000 7fa80d7fa700 1 --2- 172.21.15.27:0/3817136305 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa8000640b0 0x7fa800066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:24:12.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.779+0000 7fa80d7fa700 1 -- 172.21.15.27:0/3817136305 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6547+0+0 (secure 0 0 0) 0x7fa804080940 con 0x7fa8100f49a0 2021-11-03T18:24:12.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.779+0000 7fa80ffff700 1 --2- 172.21.15.27:0/3817136305 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa8000640b0 0x7fa800066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:24:12.787 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.779+0000 7fa80ffff700 1 --2- 172.21.15.27:0/3817136305 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa8000640b0 0x7fa800066570 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7fa7fc009da0 tx=0x7fa7fc009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:24:12.787 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.783+0000 7fa80d7fa700 1 -- 172.21.15.27:0/3817136305 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fa804052b50 con 0x7fa8100f49a0 2021-11-03T18:24:12.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:12 smithi027 ceph-mon[97244]: pgmap v2086: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:12.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:24:12.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:24:12.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:24:12.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:12.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:12.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:12.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:12.987 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.987+0000 7fa816aa4700 1 -- 172.21.15.27:0/3817136305 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa8100586e0 con 0x7fa8100f49a0 2021-11-03T18:24:12.987 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.988+0000 7fa80d7fa700 1 -- 172.21.15.27:0/3817136305 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7fa804018710 con 0x7fa8100f49a0 2021-11-03T18:24:12.987 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:24:12.988 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:24:12.988 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:24:12.988 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:24:12.988 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:24:12.989 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:24:12.989 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:24:12.989 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:24:12.989 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:24:12.989 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:24:12.990 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:24:12.990 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:24:12.990 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:24:12.990 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:24:12.991 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:24:12.991 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:24:12.991 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:24:12.991 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:24:12.992 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:24:12.992 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:24:12.992 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:24:12.992 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:24:12.992 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:24:12.993 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:24:12.993 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:24:12.993 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:24:12.993 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:24:12.994 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:24:12.994 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:24:12.994 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:24:12.994 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:24:12.995 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:24:12.995 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:24:12.995 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:24:12.995 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:24:12.995 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:24:12.996 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:24:12.996 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:24:12.996 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:24:12.997 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:12 smithi115 ceph-mon[65453]: pgmap v2086: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:12.997 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:24:12.997 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:24:12.998 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:24:12.998 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:12.998 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:12.998 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:12.999 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:12.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.990+0000 7fa816aa4700 1 -- 172.21.15.27:0/3817136305 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa8000640b0 msgr2=0x7fa800066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:24:12.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.990+0000 7fa816aa4700 1 --2- 172.21.15.27:0/3817136305 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa8000640b0 0x7fa800066570 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7fa7fc009da0 tx=0x7fa7fc009b30).stop 2021-11-03T18:24:13.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.990+0000 7fa816aa4700 1 -- 172.21.15.27:0/3817136305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8100f49a0 msgr2=0x7fa8100f7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:24:13.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.990+0000 7fa816aa4700 1 --2- 172.21.15.27:0/3817136305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8100f49a0 0x7fa8100f7550 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 rx=0x7fa8040093f0 tx=0x7fa804006ec0).stop 2021-11-03T18:24:13.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.990+0000 7fa816aa4700 1 -- 172.21.15.27:0/3817136305 shutdown_connections 2021-11-03T18:24:13.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.990+0000 7fa816aa4700 1 --2- 172.21.15.27:0/3817136305 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa8000640b0 0x7fa800066570 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:13.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.990+0000 7fa816aa4700 1 --2- 172.21.15.27:0/3817136305 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa8100f49a0 0x7fa8100f7550 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:13.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.990+0000 7fa816aa4700 1 --2- 172.21.15.27:0/3817136305 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa8100f5340 0x7fa8100f59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:13.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.990+0000 7fa816aa4700 1 -- 172.21.15.27:0/3817136305 >> 172.21.15.27:0/3817136305 conn(0x7fa8100f04c0 msgr2=0x7fa8100fb510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:24:13.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.990+0000 7fa816aa4700 1 -- 172.21.15.27:0/3817136305 shutdown_connections 2021-11-03T18:24:13.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:12.990+0000 7fa816aa4700 1 -- 172.21.15.27:0/3817136305 wait complete. 2021-11-03T18:24:13.002 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:24:13.959 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:13 smithi115 ceph-mon[65453]: from='client.36248 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:24:13.959 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:13 smithi115 ceph-mon[65453]: from='client.36252 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:24:13.959 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:13 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:24:13.959 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:13 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2148971663' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:13.960 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:13 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3817136305' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:24:14.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:13 smithi027 ceph-mon[97244]: from='client.36248 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:24:14.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:13 smithi027 ceph-mon[97244]: from='client.36252 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:24:14.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:13 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:24:14.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:13 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2148971663' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:14.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:13 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3817136305' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:24:14.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:14 smithi115 ceph-mon[65453]: pgmap v2087: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:15.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:14 smithi027 ceph-mon[97244]: pgmap v2087: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:15.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4097.458100 secs 2021-11-03T18:24:16.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4097.458100 secs 2021-11-03T18:24:16.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:16 smithi115 ceph-mon[65453]: pgmap v2088: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:17.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:16 smithi027 ceph-mon[97244]: pgmap v2088: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:18.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:18 smithi115 ceph-mon[65453]: pgmap v2089: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:19.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:18 smithi027 ceph-mon[97244]: pgmap v2089: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:20.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:20 smithi115 ceph-mon[65453]: pgmap v2090: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:20.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4102.458192 secs 2021-11-03T18:24:20.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:20 smithi115 ceph-mon[65453]: osdmap e87: 6 total, 6 up, 6 in 2021-11-03T18:24:21.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:20 smithi027 ceph-mon[97244]: pgmap v2090: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:21.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4102.458192 secs 2021-11-03T18:24:21.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:20 smithi027 ceph-mon[97244]: osdmap e87: 6 total, 6 up, 6 in 2021-11-03T18:24:22.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:22 smithi027 ceph-mon[97244]: pgmap v2092: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:22.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:24:22.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:22 smithi115 ceph-mon[65453]: pgmap v2092: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:22.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:24:23.958 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:23 smithi115 ceph-mon[65453]: poll took 60166.163 ms: 1 events 2021-11-03T18:24:23.959 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:24:23.959 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:24:23.959 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:23.960 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:23.960 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:23.960 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:24.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:23 smithi027 ceph-mon[97244]: poll took 60166.163 ms: 1 events 2021-11-03T18:24:24.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:24:24.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:24:24.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:24.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:24.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:24.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:24.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:24 smithi115 ceph-mon[65453]: pgmap v2093: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:24.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:24 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:24:25.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:24 smithi027 ceph-mon[97244]: pgmap v2093: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:25.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:24 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:24:25.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4107.458324 secs 2021-11-03T18:24:25.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:25 smithi115 ceph-mon[65453]: pgmap v2094: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:26.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4107.458324 secs 2021-11-03T18:24:26.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:25 smithi027 ceph-mon[97244]: pgmap v2094: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:28.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:28 smithi027 ceph-mon[97244]: pgmap v2095: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:28.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:28 smithi115 ceph-mon[65453]: pgmap v2095: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:30.271 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:30 smithi027 ceph-mon[97244]: pgmap v2096: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:30.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:30 smithi115 ceph-mon[65453]: pgmap v2096: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:31.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:31 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4112.458447 secs 2021-11-03T18:24:31.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:31 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4112.458447 secs 2021-11-03T18:24:32.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:32 smithi115 ceph-mon[65453]: pgmap v2097: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:32.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:32 smithi027 ceph-mon[97244]: pgmap v2097: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:33.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:24:33.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:24:33.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:24:33.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:33.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:33.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:33.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:33 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:33.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:24:33.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:24:33.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:24:33.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:33.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:33.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:33.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:33 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:34.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:34 smithi115 ceph-mon[65453]: pgmap v2098: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:34.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:34 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:24:34.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:34 smithi027 ceph-mon[97244]: pgmap v2098: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:34.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:34 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:24:35.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4117.458471 secs 2021-11-03T18:24:35.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:35 smithi115 ceph-mon[65453]: osdmap e88: 6 total, 6 up, 6 in 2021-11-03T18:24:35.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4117.458471 secs 2021-11-03T18:24:35.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:35 smithi027 ceph-mon[97244]: osdmap e88: 6 total, 6 up, 6 in 2021-11-03T18:24:36.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:36 smithi115 ceph-mon[65453]: pgmap v2099: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:36.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:36 smithi027 ceph-mon[97244]: pgmap v2099: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:38.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:38 smithi115 ceph-mon[65453]: pgmap v2101: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:38.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:38 smithi027 ceph-mon[97244]: pgmap v2101: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:40.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:40 smithi115 ceph-mon[65453]: pgmap v2102: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:40.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4122.458593 secs 2021-11-03T18:24:40.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:40 smithi027 ceph-mon[97244]: pgmap v2102: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:40.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4122.458593 secs 2021-11-03T18:24:42.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:42 smithi115 ceph-mon[65453]: pgmap v2103: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:42.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:42 smithi027 ceph-mon[97244]: pgmap v2103: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:43.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.106+0000 7f00fd13f700 1 -- 172.21.15.27:0/102733345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00f80f6ff0 msgr2=0x7f00f80f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:24:43.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.106+0000 7f00fd13f700 1 --2- 172.21.15.27:0/102733345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00f80f6ff0 0x7f00f80f7a20 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 rx=0x7f00e0004660 tx=0x7f00e0009b30).stop 2021-11-03T18:24:43.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.106+0000 7f00fd13f700 1 -- 172.21.15.27:0/102733345 shutdown_connections 2021-11-03T18:24:43.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.106+0000 7f00fd13f700 1 --2- 172.21.15.27:0/102733345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00f80f6ff0 0x7f00f80f7a20 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:43.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.106+0000 7f00fd13f700 1 --2- 172.21.15.27:0/102733345 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00f80f66e0 0x7f00f80f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:43.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.106+0000 7f00fd13f700 1 -- 172.21.15.27:0/102733345 >> 172.21.15.27:0/102733345 conn(0x7f00f80f1f00 msgr2=0x7f00f80f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:24:43.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.106+0000 7f00fd13f700 1 -- 172.21.15.27:0/102733345 shutdown_connections 2021-11-03T18:24:43.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.106+0000 7f00fd13f700 1 -- 172.21.15.27:0/102733345 wait complete. 2021-11-03T18:24:43.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.107+0000 7f00fd13f700 1 Processor -- start 2021-11-03T18:24:43.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.107+0000 7f00fd13f700 1 -- start start 2021-11-03T18:24:43.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.107+0000 7f00fd13f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00f80f66e0 0x7f00f8106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:24:43.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.107+0000 7f00fd13f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00f80f6ff0 0x7f00f8107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:24:43.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.107+0000 7f00fd13f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f8107a10 con 0x7f00f80f6ff0 2021-11-03T18:24:43.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.107+0000 7f00fd13f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f8107b50 con 0x7f00f80f66e0 2021-11-03T18:24:43.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.108+0000 7f00f659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00f80f6ff0 0x7f00f8107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:24:43.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.108+0000 7f00f659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00f80f6ff0 0x7f00f8107460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34702/0 (socket says 172.21.15.27:34702) 2021-11-03T18:24:43.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.108+0000 7f00f659c700 1 -- 172.21.15.27:0/3683506931 learned_addr learned my addr 172.21.15.27:0/3683506931 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:24:43.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.108+0000 7f00f6d9d700 1 --2- 172.21.15.27:0/3683506931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00f80f66e0 0x7f00f8106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:24:43.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.108+0000 7f00f659c700 1 -- 172.21.15.27:0/3683506931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00f80f66e0 msgr2=0x7f00f8106f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:24:43.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.108+0000 7f00f659c700 1 --2- 172.21.15.27:0/3683506931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00f80f66e0 0x7f00f8106f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:43.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.108+0000 7f00f659c700 1 -- 172.21.15.27:0/3683506931 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e0005040 con 0x7f00f80f6ff0 2021-11-03T18:24:43.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.108+0000 7f00f659c700 1 --2- 172.21.15.27:0/3683506931 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00f80f6ff0 0x7f00f8107460 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 rx=0x7f00e0009990 tx=0x7f00e0009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:24:43.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.108+0000 7f00ef7fe700 1 -- 172.21.15.27:0/3683506931 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00e0018070 con 0x7f00f80f6ff0 2021-11-03T18:24:43.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.108+0000 7f00ef7fe700 1 -- 172.21.15.27:0/3683506931 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f00e0007ba0 con 0x7f00f80f6ff0 2021-11-03T18:24:43.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.108+0000 7f00ef7fe700 1 -- 172.21.15.27:0/3683506931 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00e001c3c0 con 0x7f00f80f6ff0 2021-11-03T18:24:43.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.108+0000 7f00fd13f700 1 -- 172.21.15.27:0/3683506931 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00f8108490 con 0x7f00f80f6ff0 2021-11-03T18:24:43.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.108+0000 7f00fd13f700 1 -- 172.21.15.27:0/3683506931 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00f81089f0 con 0x7f00f80f6ff0 2021-11-03T18:24:43.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.109+0000 7f00fd13f700 1 -- 172.21.15.27:0/3683506931 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00f8045ad0 con 0x7f00f80f6ff0 2021-11-03T18:24:43.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.110+0000 7f00ef7fe700 1 -- 172.21.15.27:0/3683506931 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f00e001d420 con 0x7f00f80f6ff0 2021-11-03T18:24:43.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.110+0000 7f00ef7fe700 1 --2- 172.21.15.27:0/3683506931 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f00e40640b0 0x7f00e4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:24:43.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.110+0000 7f00ef7fe700 1 -- 172.21.15.27:0/3683506931 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f00e0082410 con 0x7f00f80f6ff0 2021-11-03T18:24:43.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.110+0000 7f00f6d9d700 1 --2- 172.21.15.27:0/3683506931 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f00e40640b0 0x7f00e4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:24:43.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.111+0000 7f00f6d9d700 1 --2- 172.21.15.27:0/3683506931 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f00e40640b0 0x7f00e4066570 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f00e8007db0 tx=0x7f00e8009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:24:43.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.114+0000 7f00ef7fe700 1 -- 172.21.15.27:0/3683506931 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f00e0054830 con 0x7f00f80f6ff0 2021-11-03T18:24:43.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.293+0000 7f00fd13f700 1 -- 172.21.15.27:0/3683506931 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f00f80ff040 con 0x7f00e40640b0 2021-11-03T18:24:43.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.294+0000 7f00ef7fe700 1 -- 172.21.15.27:0/3683506931 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f00f80ff040 con 0x7f00e40640b0 2021-11-03T18:24:43.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.296+0000 7f00fd13f700 1 -- 172.21.15.27:0/3683506931 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f00e40640b0 msgr2=0x7f00e4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:24:43.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.296+0000 7f00fd13f700 1 --2- 172.21.15.27:0/3683506931 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f00e40640b0 0x7f00e4066570 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7f00e8007db0 tx=0x7f00e8009040).stop 2021-11-03T18:24:43.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.296+0000 7f00fd13f700 1 -- 172.21.15.27:0/3683506931 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00f80f6ff0 msgr2=0x7f00f8107460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:24:43.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.296+0000 7f00fd13f700 1 --2- 172.21.15.27:0/3683506931 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00f80f6ff0 0x7f00f8107460 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 rx=0x7f00e0009990 tx=0x7f00e0009b30).stop 2021-11-03T18:24:43.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.296+0000 7f00fd13f700 1 -- 172.21.15.27:0/3683506931 shutdown_connections 2021-11-03T18:24:43.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.296+0000 7f00fd13f700 1 --2- 172.21.15.27:0/3683506931 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f00e40640b0 0x7f00e4066570 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:43.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.296+0000 7f00fd13f700 1 --2- 172.21.15.27:0/3683506931 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00f80f66e0 0x7f00f8106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:43.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.296+0000 7f00fd13f700 1 --2- 172.21.15.27:0/3683506931 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00f80f6ff0 0x7f00f8107460 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:43.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.296+0000 7f00fd13f700 1 -- 172.21.15.27:0/3683506931 >> 172.21.15.27:0/3683506931 conn(0x7f00f80f1f00 msgr2=0x7f00f80fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:24:43.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.296+0000 7f00fd13f700 1 -- 172.21.15.27:0/3683506931 shutdown_connections 2021-11-03T18:24:43.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.296+0000 7f00fd13f700 1 -- 172.21.15.27:0/3683506931 wait complete. 2021-11-03T18:24:43.305 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:24:43.397 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.398+0000 7f38ce5a2700 1 -- 172.21.15.27:0/1665730899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38c80f55e0 msgr2=0x7f38c80f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:24:43.397 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.398+0000 7f38ce5a2700 1 --2- 172.21.15.27:0/1665730899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38c80f55e0 0x7f38c80f5a00 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 rx=0x7f38b0004660 tx=0x7f38b0009b30).stop 2021-11-03T18:24:43.397 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.398+0000 7f38ce5a2700 1 -- 172.21.15.27:0/1665730899 shutdown_connections 2021-11-03T18:24:43.397 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.398+0000 7f38ce5a2700 1 --2- 172.21.15.27:0/1665730899 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38c80f5ef0 0x7f38c80f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:43.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.398+0000 7f38ce5a2700 1 --2- 172.21.15.27:0/1665730899 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38c80f55e0 0x7f38c80f5a00 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:43.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.398+0000 7f38ce5a2700 1 -- 172.21.15.27:0/1665730899 >> 172.21.15.27:0/1665730899 conn(0x7f38c80f0de0 msgr2=0x7f38c80f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:24:43.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.398+0000 7f38ce5a2700 1 -- 172.21.15.27:0/1665730899 shutdown_connections 2021-11-03T18:24:43.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.398+0000 7f38ce5a2700 1 -- 172.21.15.27:0/1665730899 wait complete. 2021-11-03T18:24:43.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.398+0000 7f38ce5a2700 1 Processor -- start 2021-11-03T18:24:43.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.398+0000 7f38ce5a2700 1 -- start start 2021-11-03T18:24:43.402 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.399+0000 7f38ce5a2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38c80f55e0 0x7f38c8109c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:24:43.402 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.399+0000 7f38ce5a2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38c80f5ef0 0x7f38c810c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:24:43.402 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.399+0000 7f38ce5a2700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c80604e0 con 0x7f38c80f5ef0 2021-11-03T18:24:43.402 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.399+0000 7f38ce5a2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c8060620 con 0x7f38c80f55e0 2021-11-03T18:24:43.403 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.399+0000 7f38c77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38c80f5ef0 0x7f38c810c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:24:43.403 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.399+0000 7f38c77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38c80f5ef0 0x7f38c810c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34712/0 (socket says 172.21.15.27:34712) 2021-11-03T18:24:43.403 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.399+0000 7f38c77fe700 1 -- 172.21.15.27:0/2781071907 learned_addr learned my addr 172.21.15.27:0/2781071907 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:24:43.403 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.399+0000 7f38c7fff700 1 --2- 172.21.15.27:0/2781071907 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38c80f55e0 0x7f38c8109c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:24:43.403 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.399+0000 7f38c77fe700 1 -- 172.21.15.27:0/2781071907 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38c80f55e0 msgr2=0x7f38c8109c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:24:43.404 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.399+0000 7f38c77fe700 1 --2- 172.21.15.27:0/2781071907 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38c80f55e0 0x7f38c8109c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:43.404 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.399+0000 7f38c77fe700 1 -- 172.21.15.27:0/2781071907 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38b0005040 con 0x7f38c80f5ef0 2021-11-03T18:24:43.404 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.399+0000 7f38c7fff700 1 --2- 172.21.15.27:0/2781071907 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38c80f55e0 0x7f38c8109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:24:43.404 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.399+0000 7f38c77fe700 1 --2- 172.21.15.27:0/2781071907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38c80f5ef0 0x7f38c810c190 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 rx=0x7f38b80026d0 tx=0x7f38b8009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:24:43.405 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.399+0000 7f38c4ff9700 1 -- 172.21.15.27:0/2781071907 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38b8009320 con 0x7f38c80f5ef0 2021-11-03T18:24:43.405 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.399+0000 7f38c4ff9700 1 -- 172.21.15.27:0/2781071907 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f38b800f040 con 0x7f38c80f5ef0 2021-11-03T18:24:43.405 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.399+0000 7f38ce5a2700 1 -- 172.21.15.27:0/2781071907 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38c810c740 con 0x7f38c80f5ef0 2021-11-03T18:24:43.405 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.399+0000 7f38c4ff9700 1 -- 172.21.15.27:0/2781071907 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38b80147e0 con 0x7f38c80f5ef0 2021-11-03T18:24:43.406 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.399+0000 7f38ce5a2700 1 -- 172.21.15.27:0/2781071907 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38c810cd00 con 0x7f38c80f5ef0 2021-11-03T18:24:43.406 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.400+0000 7f38ce5a2700 1 -- 172.21.15.27:0/2781071907 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38c8045ad0 con 0x7f38c80f5ef0 2021-11-03T18:24:43.406 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.401+0000 7f38c4ff9700 1 -- 172.21.15.27:0/2781071907 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f38b8009480 con 0x7f38c80f5ef0 2021-11-03T18:24:43.406 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.401+0000 7f38c4ff9700 1 --2- 172.21.15.27:0/2781071907 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f38b40640b0 0x7f38b4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:24:43.407 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.401+0000 7f38c4ff9700 1 -- 172.21.15.27:0/2781071907 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f38b8080df0 con 0x7f38c80f5ef0 2021-11-03T18:24:43.407 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.401+0000 7f38c7fff700 1 --2- 172.21.15.27:0/2781071907 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f38b40640b0 0x7f38b4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:24:43.407 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.401+0000 7f38c7fff700 1 --2- 172.21.15.27:0/2781071907 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f38b40640b0 0x7f38b4066570 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f38b0005010 tx=0x7f38b0009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:24:43.408 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.405+0000 7f38c4ff9700 1 -- 172.21.15.27:0/2781071907 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f38b8053210 con 0x7f38c80f5ef0 2021-11-03T18:24:43.575 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.576+0000 7f38ce5a2700 1 -- 172.21.15.27:0/2781071907 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f38c80fdf40 con 0x7f38b40640b0 2021-11-03T18:24:43.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.590+0000 7f38c4ff9700 1 -- 172.21.15.27:0/2781071907 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f38c80fdf40 con 0x7f38b40640b0 2021-11-03T18:24:43.590 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:24:43.590 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (76m) 5m ago 77m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:24:43.590 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (67m) 5m ago 77m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:24:43.590 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (67m) 4m ago 76m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:24:43.591 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (76m) 5m ago 77m 42.4M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:24:43.591 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (69m) 5m ago 72m 26.5M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:24:43.591 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (72m) 5m ago 72m 3849M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:24:43.591 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (68m) 4m ago 72m 26.1M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:24:43.592 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (70m) 4m ago 72m 27.2M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:24:43.592 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (69m) 5m ago 78m 590M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:24:43.592 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (69m) 4m ago 76m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:24:43.592 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (68m) 5m ago 78m 312M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:24:43.592 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (68m) 4m ago 76m 145M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:24:43.593 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (77m) 5m ago 77m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:24:43.593 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (76m) 4m ago 76m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:24:43.593 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (67m) 5m ago 75m 392M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:24:43.593 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (67m) 5m ago 75m 427M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:24:43.594 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (66m) 5m ago 75m 320M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:24:43.594 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (66m) 4m ago 74m 454M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:24:43.594 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (65m) 4m ago 74m 359M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:24:43.594 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (65m) 4m ago 74m 368M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:24:43.594 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (76m) 5m ago 76m 59.9M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:24:43.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.592+0000 7f38ce5a2700 1 -- 172.21.15.27:0/2781071907 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f38b40640b0 msgr2=0x7f38b4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:24:43.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.592+0000 7f38ce5a2700 1 --2- 172.21.15.27:0/2781071907 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f38b40640b0 0x7f38b4066570 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7f38b0005010 tx=0x7f38b0009b30).stop 2021-11-03T18:24:43.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.592+0000 7f38ce5a2700 1 -- 172.21.15.27:0/2781071907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38c80f5ef0 msgr2=0x7f38c810c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:24:43.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.592+0000 7f38ce5a2700 1 --2- 172.21.15.27:0/2781071907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38c80f5ef0 0x7f38c810c190 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 rx=0x7f38b80026d0 tx=0x7f38b8009de0).stop 2021-11-03T18:24:43.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.592+0000 7f38ce5a2700 1 -- 172.21.15.27:0/2781071907 shutdown_connections 2021-11-03T18:24:43.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.592+0000 7f38ce5a2700 1 --2- 172.21.15.27:0/2781071907 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f38b40640b0 0x7f38b4066570 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:43.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.592+0000 7f38ce5a2700 1 --2- 172.21.15.27:0/2781071907 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38c80f55e0 0x7f38c8109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:43.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.592+0000 7f38ce5a2700 1 --2- 172.21.15.27:0/2781071907 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38c80f5ef0 0x7f38c810c190 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:43.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.592+0000 7f38ce5a2700 1 -- 172.21.15.27:0/2781071907 >> 172.21.15.27:0/2781071907 conn(0x7f38c80f0de0 msgr2=0x7f38c80fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:24:43.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.592+0000 7f38ce5a2700 1 -- 172.21.15.27:0/2781071907 shutdown_connections 2021-11-03T18:24:43.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.592+0000 7f38ce5a2700 1 -- 172.21.15.27:0/2781071907 wait complete. 2021-11-03T18:24:43.692 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.691+0000 7fd46d362700 1 -- 172.21.15.27:0/4001955520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4680f74a0 msgr2=0x7fd4680f5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:24:43.692 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.691+0000 7fd46d362700 1 --2- 172.21.15.27:0/4001955520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4680f74a0 0x7fd4680f5930 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 rx=0x7fd45c004660 tx=0x7fd45c009b30).stop 2021-11-03T18:24:43.692 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.691+0000 7fd46d362700 1 -- 172.21.15.27:0/4001955520 shutdown_connections 2021-11-03T18:24:43.693 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.691+0000 7fd46d362700 1 --2- 172.21.15.27:0/4001955520 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4680f74a0 0x7fd4680f5930 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:43.693 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.691+0000 7fd46d362700 1 --2- 172.21.15.27:0/4001955520 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd4680f6b90 0x7fd4680f6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:43.693 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.691+0000 7fd46d362700 1 -- 172.21.15.27:0/4001955520 >> 172.21.15.27:0/4001955520 conn(0x7fd4680f0540 msgr2=0x7fd4680f2960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:24:43.693 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.691+0000 7fd46d362700 1 -- 172.21.15.27:0/4001955520 shutdown_connections 2021-11-03T18:24:43.694 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.691+0000 7fd46d362700 1 -- 172.21.15.27:0/4001955520 wait complete. 2021-11-03T18:24:43.694 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.692+0000 7fd46d362700 1 Processor -- start 2021-11-03T18:24:43.694 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.692+0000 7fd46d362700 1 -- start start 2021-11-03T18:24:43.694 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.692+0000 7fd46d362700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4680f6b90 0x7fd468105e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:24:43.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.692+0000 7fd46d362700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd4680f74a0 0x7fd468106370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:24:43.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.692+0000 7fd46d362700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd468106920 con 0x7fd4680f6b90 2021-11-03T18:24:43.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.692+0000 7fd46d362700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd468106a60 con 0x7fd4680f74a0 2021-11-03T18:24:43.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.692+0000 7fd466ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4680f6b90 0x7fd468105e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:24:43.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.692+0000 7fd466ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4680f6b90 0x7fd468105e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34720/0 (socket says 172.21.15.27:34720) 2021-11-03T18:24:43.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.692+0000 7fd466ffd700 1 -- 172.21.15.27:0/3850718210 learned_addr learned my addr 172.21.15.27:0/3850718210 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:24:43.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.693+0000 7fd4667fc700 1 --2- 172.21.15.27:0/3850718210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd4680f74a0 0x7fd468106370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:24:43.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.693+0000 7fd466ffd700 1 -- 172.21.15.27:0/3850718210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd4680f74a0 msgr2=0x7fd468106370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:24:43.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.693+0000 7fd466ffd700 1 --2- 172.21.15.27:0/3850718210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd4680f74a0 0x7fd468106370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:43.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.693+0000 7fd466ffd700 1 -- 172.21.15.27:0/3850718210 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd45c005040 con 0x7fd4680f6b90 2021-11-03T18:24:43.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.693+0000 7fd466ffd700 1 --2- 172.21.15.27:0/3850718210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4680f6b90 0x7fd468105e80 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 rx=0x7fd45800c790 tx=0x7fd458007d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:24:43.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.693+0000 7fd44f7fe700 1 -- 172.21.15.27:0/3850718210 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd45800e680 con 0x7fd4680f6b90 2021-11-03T18:24:43.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.693+0000 7fd44f7fe700 1 -- 172.21.15.27:0/3850718210 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fd45800ec90 con 0x7fd4680f6b90 2021-11-03T18:24:43.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.693+0000 7fd44f7fe700 1 -- 172.21.15.27:0/3850718210 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd458012d00 con 0x7fd4680f6b90 2021-11-03T18:24:43.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.693+0000 7fd46d362700 1 -- 172.21.15.27:0/3850718210 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd468107400 con 0x7fd4680f6b90 2021-11-03T18:24:43.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.693+0000 7fd46d362700 1 -- 172.21.15.27:0/3850718210 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4681079c0 con 0x7fd4680f6b90 2021-11-03T18:24:43.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.694+0000 7fd46d362700 1 -- 172.21.15.27:0/3850718210 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd468045ad0 con 0x7fd4680f6b90 2021-11-03T18:24:43.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.694+0000 7fd44f7fe700 1 -- 172.21.15.27:0/3850718210 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fd458010070 con 0x7fd4680f6b90 2021-11-03T18:24:43.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.695+0000 7fd44f7fe700 1 --2- 172.21.15.27:0/3850718210 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd4500640b0 0x7fd450066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:24:43.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.695+0000 7fd44f7fe700 1 -- 172.21.15.27:0/3850718210 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fd458080990 con 0x7fd4680f6b90 2021-11-03T18:24:43.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.695+0000 7fd4667fc700 1 --2- 172.21.15.27:0/3850718210 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd4500640b0 0x7fd450066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:24:43.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.695+0000 7fd4667fc700 1 --2- 172.21.15.27:0/3850718210 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd4500640b0 0x7fd450066570 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7fd45c004e30 tx=0x7fd45c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:24:43.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.699+0000 7fd44f7fe700 1 -- 172.21.15.27:0/3850718210 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fd458052c00 con 0x7fd4680f6b90 2021-11-03T18:24:43.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:24:43.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:24:43.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:24:43.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:43.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:43.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:43.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:43 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:43.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:24:43.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:24:43.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:24:43.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:43.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:43.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:43.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:43 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:43.927 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.928+0000 7fd46d362700 1 -- 172.21.15.27:0/3850718210 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd4680fde40 con 0x7fd4680f6b90 2021-11-03T18:24:43.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.929+0000 7fd44f7fe700 1 -- 172.21.15.27:0/3850718210 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fd4580527f0 con 0x7fd4680f6b90 2021-11-03T18:24:43.928 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:24:43.929 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:24:43.929 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:24:43.929 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:24:43.929 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:24:43.929 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:24:43.930 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:24:43.930 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:24:43.930 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:24:43.930 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:24:43.931 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:24:43.931 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:24:43.931 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:24:43.931 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:24:43.931 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:24:43.932 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:24:43.932 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:24:43.932 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:24:43.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.930+0000 7fd46d362700 1 -- 172.21.15.27:0/3850718210 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd4500640b0 msgr2=0x7fd450066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:24:43.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.930+0000 7fd46d362700 1 --2- 172.21.15.27:0/3850718210 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd4500640b0 0x7fd450066570 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 rx=0x7fd45c004e30 tx=0x7fd45c009b30).stop 2021-11-03T18:24:43.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.930+0000 7fd46d362700 1 -- 172.21.15.27:0/3850718210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4680f6b90 msgr2=0x7fd468105e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:24:43.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.930+0000 7fd46d362700 1 --2- 172.21.15.27:0/3850718210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4680f6b90 0x7fd468105e80 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 rx=0x7fd45800c790 tx=0x7fd458007d90).stop 2021-11-03T18:24:43.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.930+0000 7fd46d362700 1 -- 172.21.15.27:0/3850718210 shutdown_connections 2021-11-03T18:24:43.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.930+0000 7fd46d362700 1 --2- 172.21.15.27:0/3850718210 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd4500640b0 0x7fd450066570 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:43.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.930+0000 7fd46d362700 1 --2- 172.21.15.27:0/3850718210 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd4680f6b90 0x7fd468105e80 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:43.935 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.930+0000 7fd46d362700 1 --2- 172.21.15.27:0/3850718210 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd4680f74a0 0x7fd468106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:43.935 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.930+0000 7fd46d362700 1 -- 172.21.15.27:0/3850718210 >> 172.21.15.27:0/3850718210 conn(0x7fd4680f0540 msgr2=0x7fd4680fa390 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:24:43.935 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.931+0000 7fd46d362700 1 -- 172.21.15.27:0/3850718210 shutdown_connections 2021-11-03T18:24:43.935 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:43.931+0000 7fd46d362700 1 -- 172.21.15.27:0/3850718210 wait complete. 2021-11-03T18:24:44.032 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.031+0000 7f366aa07700 1 -- 172.21.15.27:0/4074104442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36640f5340 msgr2=0x7f36640f7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:24:44.032 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.031+0000 7f366aa07700 1 --2- 172.21.15.27:0/4074104442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36640f5340 0x7f36640f7ac0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 rx=0x7f364c004660 tx=0x7f364c009b30).stop 2021-11-03T18:24:44.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.031+0000 7f366aa07700 1 -- 172.21.15.27:0/4074104442 shutdown_connections 2021-11-03T18:24:44.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.031+0000 7f366aa07700 1 --2- 172.21.15.27:0/4074104442 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36640f5340 0x7f36640f7ac0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:44.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.031+0000 7f366aa07700 1 --2- 172.21.15.27:0/4074104442 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36640f49a0 0x7f36640f4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:44.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.031+0000 7f366aa07700 1 -- 172.21.15.27:0/4074104442 >> 172.21.15.27:0/4074104442 conn(0x7f36640f04c0 msgr2=0x7f36640f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:24:44.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.031+0000 7f366aa07700 1 -- 172.21.15.27:0/4074104442 shutdown_connections 2021-11-03T18:24:44.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.031+0000 7f366aa07700 1 -- 172.21.15.27:0/4074104442 wait complete. 2021-11-03T18:24:44.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.032+0000 7f366aa07700 1 Processor -- start 2021-11-03T18:24:44.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.032+0000 7f366aa07700 1 -- start start 2021-11-03T18:24:44.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.032+0000 7f366aa07700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36640f49a0 0x7f3664106fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:24:44.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.032+0000 7f366aa07700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36640f5340 0x7f3664107490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:24:44.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.032+0000 7f366aa07700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3664107a40 con 0x7f36640f49a0 2021-11-03T18:24:44.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.032+0000 7f366aa07700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3664107b80 con 0x7f36640f5340 2021-11-03T18:24:44.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.032+0000 7f3663fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36640f49a0 0x7f3664106fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:24:44.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.032+0000 7f3663fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36640f49a0 0x7f3664106fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34730/0 (socket says 172.21.15.27:34730) 2021-11-03T18:24:44.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.032+0000 7f3663fff700 1 -- 172.21.15.27:0/3191938523 learned_addr learned my addr 172.21.15.27:0/3191938523 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:24:44.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.032+0000 7f36637fe700 1 --2- 172.21.15.27:0/3191938523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36640f5340 0x7f3664107490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:24:44.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.032+0000 7f3663fff700 1 -- 172.21.15.27:0/3191938523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36640f5340 msgr2=0x7f3664107490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:24:44.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.032+0000 7f3663fff700 1 --2- 172.21.15.27:0/3191938523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36640f5340 0x7f3664107490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:44.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.032+0000 7f3663fff700 1 -- 172.21.15.27:0/3191938523 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f364c005040 con 0x7f36640f49a0 2021-11-03T18:24:44.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.032+0000 7f36637fe700 1 --2- 172.21.15.27:0/3191938523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36640f5340 0x7f3664107490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:24:44.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.033+0000 7f3663fff700 1 --2- 172.21.15.27:0/3191938523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36640f49a0 0x7f3664106fa0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 rx=0x7f3654002700 tx=0x7f3654009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:24:44.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.033+0000 7f3660ff9700 1 -- 172.21.15.27:0/3191938523 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3654009320 con 0x7f36640f49a0 2021-11-03T18:24:44.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.033+0000 7f3660ff9700 1 -- 172.21.15.27:0/3191938523 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f365400f040 con 0x7f36640f49a0 2021-11-03T18:24:44.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.033+0000 7f366aa07700 1 -- 172.21.15.27:0/3191938523 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3664108520 con 0x7f36640f49a0 2021-11-03T18:24:44.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.033+0000 7f3660ff9700 1 -- 172.21.15.27:0/3191938523 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3654014900 con 0x7f36640f49a0 2021-11-03T18:24:44.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.033+0000 7f366aa07700 1 -- 172.21.15.27:0/3191938523 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3664108ae0 con 0x7f36640f49a0 2021-11-03T18:24:44.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.034+0000 7f366aa07700 1 -- 172.21.15.27:0/3191938523 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3664045ad0 con 0x7f36640f49a0 2021-11-03T18:24:44.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.034+0000 7f3660ff9700 1 -- 172.21.15.27:0/3191938523 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f36540077b0 con 0x7f36640f49a0 2021-11-03T18:24:44.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.035+0000 7f3660ff9700 1 --2- 172.21.15.27:0/3191938523 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f36500640b0 0x7f3650066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:24:44.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.035+0000 7f3660ff9700 1 -- 172.21.15.27:0/3191938523 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f3654080980 con 0x7f36640f49a0 2021-11-03T18:24:44.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.035+0000 7f36637fe700 1 --2- 172.21.15.27:0/3191938523 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f36500640b0 0x7f3650066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:24:44.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.035+0000 7f36637fe700 1 --2- 172.21.15.27:0/3191938523 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f36500640b0 0x7f3650066570 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7f364c005010 tx=0x7f364c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:24:44.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.039+0000 7f3660ff9700 1 -- 172.21.15.27:0/3191938523 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f3654052ce0 con 0x7f36640f49a0 2021-11-03T18:24:44.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.245+0000 7f366aa07700 1 -- 172.21.15.27:0/3191938523 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3664027a00 con 0x7f36640f49a0 2021-11-03T18:24:44.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.245+0000 7f3660ff9700 1 -- 172.21.15.27:0/3191938523 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f36540528d0 con 0x7f36640f49a0 2021-11-03T18:24:44.244 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:24:44.245 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:24:44.245 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:24:44.245 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:24:44.245 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:24:44.246 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:24:44.246 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:24:44.246 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:24:44.246 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:24:44.246 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:24:44.247 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:24:44.247 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:24:44.247 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:24:44.247 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:24:44.247 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:24:44.248 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:24:44.248 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:24:44.248 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:24:44.248 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:24:44.248 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:24:44.249 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:24:44.249 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:24:44.249 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:24:44.249 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:24:44.250 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:24:44.250 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:24:44.250 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:24:44.250 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:24:44.250 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:24:44.251 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:24:44.251 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:24:44.251 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:24:44.251 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:24:44.252 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:24:44.252 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:24:44.252 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:24:44.252 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:24:44.252 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:24:44.253 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:24:44.253 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.247+0000 7f366aa07700 1 -- 172.21.15.27:0/3191938523 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f36500640b0 msgr2=0x7f3650066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:24:44.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.247+0000 7f366aa07700 1 --2- 172.21.15.27:0/3191938523 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f36500640b0 0x7f3650066570 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7f364c005010 tx=0x7f364c009b30).stop 2021-11-03T18:24:44.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.247+0000 7f366aa07700 1 -- 172.21.15.27:0/3191938523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36640f49a0 msgr2=0x7f3664106fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:24:44.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.247+0000 7f366aa07700 1 --2- 172.21.15.27:0/3191938523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36640f49a0 0x7f3664106fa0 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 rx=0x7f3654002700 tx=0x7f3654009d80).stop 2021-11-03T18:24:44.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.247+0000 7f366aa07700 1 -- 172.21.15.27:0/3191938523 shutdown_connections 2021-11-03T18:24:44.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.247+0000 7f366aa07700 1 --2- 172.21.15.27:0/3191938523 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f36500640b0 0x7f3650066570 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:44.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.247+0000 7f366aa07700 1 --2- 172.21.15.27:0/3191938523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36640f49a0 0x7f3664106fa0 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:44.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.247+0000 7f366aa07700 1 --2- 172.21.15.27:0/3191938523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36640f5340 0x7f3664107490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:24:44.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.247+0000 7f366aa07700 1 -- 172.21.15.27:0/3191938523 >> 172.21.15.27:0/3191938523 conn(0x7f36640f04c0 msgr2=0x7f36640fb510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:24:44.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.247+0000 7f366aa07700 1 -- 172.21.15.27:0/3191938523 shutdown_connections 2021-11-03T18:24:44.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:24:44.247+0000 7f366aa07700 1 -- 172.21.15.27:0/3191938523 wait complete. 2021-11-03T18:24:44.256 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:24:44.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:44 smithi115 ceph-mon[65453]: pgmap v2104: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:44.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:44 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:24:44.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:44 smithi115 ceph-mon[65453]: from='client.36264 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:24:44.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:44 smithi115 ceph-mon[65453]: from='client.36268 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:24:44.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:44 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3850718210' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:44.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:44 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3191938523' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:24:44.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:44 smithi027 ceph-mon[97244]: pgmap v2104: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:44.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:44 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:24:44.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:44 smithi027 ceph-mon[97244]: from='client.36264 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:24:44.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:44 smithi027 ceph-mon[97244]: from='client.36268 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:24:44.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:44 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3850718210' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:44.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:44 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3191938523' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:24:45.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4127.458710 secs 2021-11-03T18:24:45.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4127.458710 secs 2021-11-03T18:24:46.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:46 smithi115 ceph-mon[65453]: pgmap v2105: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:46.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:46 smithi027 ceph-mon[97244]: pgmap v2105: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:48.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:48 smithi115 ceph-mon[65453]: pgmap v2106: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:48.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:48 smithi027 ceph-mon[97244]: pgmap v2106: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:50.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:50 smithi115 ceph-mon[65453]: pgmap v2107: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:50.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4132.458841 secs 2021-11-03T18:24:50.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:50 smithi027 ceph-mon[97244]: pgmap v2107: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:50.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4132.458841 secs 2021-11-03T18:24:52.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:52 smithi115 ceph-mon[65453]: pgmap v2108: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:52.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:52 smithi027 ceph-mon[97244]: pgmap v2108: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:53.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:53 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:24:53.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:53 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:24:53.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:53 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:24:53.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:53 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:53.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:53 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:53.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:53 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:53.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:53 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:53.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:53 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:24:53.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:53 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:24:53.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:53 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:24:53.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:53 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:53.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:53 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:53.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:53 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:53.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:53 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:24:54.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:54 smithi115 ceph-mon[65453]: pgmap v2109: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:54.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:54 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:24:54.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:54 smithi027 ceph-mon[97244]: pgmap v2109: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:54.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:54 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:24:55.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4137.458971 secs 2021-11-03T18:24:55.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4137.458971 secs 2021-11-03T18:24:56.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:56 smithi115 ceph-mon[65453]: pgmap v2110: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:56.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:24:56.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:24:56.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:56 smithi027 ceph-mon[97244]: pgmap v2110: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:56.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:24:56.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:24:58.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:24:58 smithi115 ceph-mon[65453]: pgmap v2111: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:24:58.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:24:58 smithi027 ceph-mon[97244]: pgmap v2111: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:00.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:00 smithi115 ceph-mon[65453]: pgmap v2112: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:00.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4142.459098 secs 2021-11-03T18:25:00.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:00 smithi027 ceph-mon[97244]: pgmap v2112: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:00.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4142.459098 secs 2021-11-03T18:25:02.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:02 smithi115 ceph-mon[65453]: pgmap v2113: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:02.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:02 smithi027 ceph-mon[97244]: pgmap v2113: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:03.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:25:03.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:25:03.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:25:03.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:03.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:03.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:03.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:03 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:03.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:25:03.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:25:03.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:25:03.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:03.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:03.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:03.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:03 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:04.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:04 smithi115 ceph-mon[65453]: pgmap v2114: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:04.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:04 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:25:04.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:04 smithi027 ceph-mon[97244]: pgmap v2114: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:04.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:04 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:25:05.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4147.459219 secs 2021-11-03T18:25:05.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4147.459219 secs 2021-11-03T18:25:06.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:06 smithi115 ceph-mon[65453]: pgmap v2115: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:06.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:06 smithi027 ceph-mon[97244]: pgmap v2115: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:08.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:08 smithi115 ceph-mon[65453]: pgmap v2116: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:08.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:08 smithi027 ceph-mon[97244]: pgmap v2116: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:10.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:10 smithi115 ceph-mon[65453]: pgmap v2117: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:10.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4152.459347 secs 2021-11-03T18:25:10.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:10 smithi027 ceph-mon[97244]: pgmap v2117: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:10.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4152.459347 secs 2021-11-03T18:25:12.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:12 smithi115 ceph-mon[65453]: pgmap v2118: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:12.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:12 smithi027 ceph-mon[97244]: pgmap v2118: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:13.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:25:13.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:25:13.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:25:13.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:13.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:13.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:13.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:13 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:13.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:25:13.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:25:13.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:25:13.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:13.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:13.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:13.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:13 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:14.361 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.362+0000 7f74dde93700 1 -- 172.21.15.27:0/2922883924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74d80f4d60 msgr2=0x7f74d80f5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:25:14.362 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.362+0000 7f74dde93700 1 --2- 172.21.15.27:0/2922883924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74d80f4d60 0x7f74d80f5180 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 rx=0x7f74c8004770 tx=0x7f74c800ac40).stop 2021-11-03T18:25:14.362 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.362+0000 7f74dde93700 1 -- 172.21.15.27:0/2922883924 shutdown_connections 2021-11-03T18:25:14.362 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.362+0000 7f74dde93700 1 --2- 172.21.15.27:0/2922883924 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74d80f5f60 0x7f74d80f63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:14.362 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.362+0000 7f74dde93700 1 --2- 172.21.15.27:0/2922883924 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74d80f4d60 0x7f74d80f5180 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:14.363 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.362+0000 7f74dde93700 1 -- 172.21.15.27:0/2922883924 >> 172.21.15.27:0/2922883924 conn(0x7f74d80f04c0 msgr2=0x7f74d80f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:25:14.363 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.362+0000 7f74dde93700 1 -- 172.21.15.27:0/2922883924 shutdown_connections 2021-11-03T18:25:14.363 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.362+0000 7f74dde93700 1 -- 172.21.15.27:0/2922883924 wait complete. 2021-11-03T18:25:14.363 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.362+0000 7f74dde93700 1 Processor -- start 2021-11-03T18:25:14.363 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.363+0000 7f74dde93700 1 -- start start 2021-11-03T18:25:14.366 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.363+0000 7f74dde93700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74d80f4d60 0x7f74d8106f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:25:14.366 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.363+0000 7f74dde93700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74d80f5f60 0x7f74d8107440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:25:14.366 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.363+0000 7f74dde93700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d81079f0 con 0x7f74d80f4d60 2021-11-03T18:25:14.367 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.363+0000 7f74dde93700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d8107b30 con 0x7f74d80f5f60 2021-11-03T18:25:14.367 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.363+0000 7f74d77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74d80f4d60 0x7f74d8106f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:25:14.367 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.363+0000 7f74d77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74d80f4d60 0x7f74d8106f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34748/0 (socket says 172.21.15.27:34748) 2021-11-03T18:25:14.367 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.363+0000 7f74d77fe700 1 -- 172.21.15.27:0/1018792477 learned_addr learned my addr 172.21.15.27:0/1018792477 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:25:14.367 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.363+0000 7f74d6ffd700 1 --2- 172.21.15.27:0/1018792477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74d80f5f60 0x7f74d8107440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:25:14.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.363+0000 7f74d77fe700 1 -- 172.21.15.27:0/1018792477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74d80f5f60 msgr2=0x7f74d8107440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:25:14.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.363+0000 7f74d77fe700 1 --2- 172.21.15.27:0/1018792477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74d80f5f60 0x7f74d8107440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:14.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.363+0000 7f74d77fe700 1 -- 172.21.15.27:0/1018792477 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c8005040 con 0x7f74d80f4d60 2021-11-03T18:25:14.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.363+0000 7f74d77fe700 1 --2- 172.21.15.27:0/1018792477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74d80f4d60 0x7f74d8106f50 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 rx=0x7f74c800adc0 tx=0x7f74c800ac40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:25:14.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.364+0000 7f74bffff700 1 -- 172.21.15.27:0/1018792477 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74c800f030 con 0x7f74d80f4d60 2021-11-03T18:25:14.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.364+0000 7f74bffff700 1 -- 172.21.15.27:0/1018792477 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f74c800a590 con 0x7f74d80f4d60 2021-11-03T18:25:14.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.364+0000 7f74dde93700 1 -- 172.21.15.27:0/1018792477 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74d8108470 con 0x7f74d80f4d60 2021-11-03T18:25:14.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.364+0000 7f74bffff700 1 -- 172.21.15.27:0/1018792477 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74c8019070 con 0x7f74d80f4d60 2021-11-03T18:25:14.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.364+0000 7f74dde93700 1 -- 172.21.15.27:0/1018792477 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74d81089d0 con 0x7f74d80f4d60 2021-11-03T18:25:14.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.365+0000 7f74dde93700 1 -- 172.21.15.27:0/1018792477 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74d8045ad0 con 0x7f74d80f4d60 2021-11-03T18:25:14.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.365+0000 7f74bffff700 1 -- 172.21.15.27:0/1018792477 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f74c80085f0 con 0x7f74d80f4d60 2021-11-03T18:25:14.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.366+0000 7f74bffff700 1 --2- 172.21.15.27:0/1018792477 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f74c00640b0 0x7f74c0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:25:14.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.366+0000 7f74bffff700 1 -- 172.21.15.27:0/1018792477 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f74c80823d0 con 0x7f74d80f4d60 2021-11-03T18:25:14.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.366+0000 7f74d6ffd700 1 --2- 172.21.15.27:0/1018792477 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f74c00640b0 0x7f74c0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:25:14.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.366+0000 7f74d6ffd700 1 --2- 172.21.15.27:0/1018792477 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f74c00640b0 0x7f74c0066570 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f74cc0041d0 tx=0x7f74cc009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:25:14.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.370+0000 7f74bffff700 1 -- 172.21.15.27:0/1018792477 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f74c8054730 con 0x7f74d80f4d60 2021-11-03T18:25:14.547 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.548+0000 7f74dde93700 1 -- 172.21.15.27:0/1018792477 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f74d80fa350 con 0x7f74c00640b0 2021-11-03T18:25:14.548 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.549+0000 7f74bffff700 1 -- 172.21.15.27:0/1018792477 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f74d80fa350 con 0x7f74c00640b0 2021-11-03T18:25:14.550 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.551+0000 7f74dde93700 1 -- 172.21.15.27:0/1018792477 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f74c00640b0 msgr2=0x7f74c0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:25:14.550 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.551+0000 7f74dde93700 1 --2- 172.21.15.27:0/1018792477 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f74c00640b0 0x7f74c0066570 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7f74cc0041d0 tx=0x7f74cc009040).stop 2021-11-03T18:25:14.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.551+0000 7f74dde93700 1 -- 172.21.15.27:0/1018792477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74d80f4d60 msgr2=0x7f74d8106f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:25:14.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.551+0000 7f74dde93700 1 --2- 172.21.15.27:0/1018792477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74d80f4d60 0x7f74d8106f50 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 rx=0x7f74c800adc0 tx=0x7f74c800ac40).stop 2021-11-03T18:25:14.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.551+0000 7f74dde93700 1 -- 172.21.15.27:0/1018792477 shutdown_connections 2021-11-03T18:25:14.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.551+0000 7f74dde93700 1 --2- 172.21.15.27:0/1018792477 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f74c00640b0 0x7f74c0066570 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:14.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.551+0000 7f74dde93700 1 --2- 172.21.15.27:0/1018792477 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74d80f4d60 0x7f74d8106f50 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:14.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.551+0000 7f74dde93700 1 --2- 172.21.15.27:0/1018792477 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74d80f5f60 0x7f74d8107440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:14.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.551+0000 7f74dde93700 1 -- 172.21.15.27:0/1018792477 >> 172.21.15.27:0/1018792477 conn(0x7f74d80f04c0 msgr2=0x7f74d80f9c40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:25:14.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.551+0000 7f74dde93700 1 -- 172.21.15.27:0/1018792477 shutdown_connections 2021-11-03T18:25:14.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.552+0000 7f74dde93700 1 -- 172.21.15.27:0/1018792477 wait complete. 2021-11-03T18:25:14.560 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:25:14.652 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.653+0000 7f39ded7d700 1 -- 172.21.15.27:0/3403320847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39d80f55c0 msgr2=0x7f39d80f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:25:14.652 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.653+0000 7f39ded7d700 1 --2- 172.21.15.27:0/3403320847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39d80f55c0 0x7f39d80f59e0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 rx=0x7f39cc004660 tx=0x7f39cc009b30).stop 2021-11-03T18:25:14.652 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.653+0000 7f39ded7d700 1 -- 172.21.15.27:0/3403320847 shutdown_connections 2021-11-03T18:25:14.653 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.653+0000 7f39ded7d700 1 --2- 172.21.15.27:0/3403320847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39d80f6860 0x7f39d80f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:14.653 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.653+0000 7f39ded7d700 1 --2- 172.21.15.27:0/3403320847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39d80f55c0 0x7f39d80f59e0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:14.653 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.653+0000 7f39ded7d700 1 -- 172.21.15.27:0/3403320847 >> 172.21.15.27:0/3403320847 conn(0x7f39d80f0d20 msgr2=0x7f39d80f3140 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:25:14.653 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.653+0000 7f39ded7d700 1 -- 172.21.15.27:0/3403320847 shutdown_connections 2021-11-03T18:25:14.654 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.653+0000 7f39ded7d700 1 -- 172.21.15.27:0/3403320847 wait complete. 2021-11-03T18:25:14.654 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.653+0000 7f39ded7d700 1 Processor -- start 2021-11-03T18:25:14.654 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.653+0000 7f39ded7d700 1 -- start start 2021-11-03T18:25:14.657 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.654+0000 7f39ded7d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39d80f55c0 0x7f39d8105ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:25:14.657 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.654+0000 7f39ded7d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39d80f6860 0x7f39d81063c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:25:14.657 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.654+0000 7f39ded7d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d81069c0 con 0x7f39d80f6860 2021-11-03T18:25:14.657 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.654+0000 7f39ded7d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39d81071d0 con 0x7f39d80f55c0 2021-11-03T18:25:14.658 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.654+0000 7f39d7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39d80f6860 0x7f39d81063c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:25:14.658 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.654+0000 7f39d7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39d80f6860 0x7f39d81063c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34760/0 (socket says 172.21.15.27:34760) 2021-11-03T18:25:14.658 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.654+0000 7f39d7fff700 1 -- 172.21.15.27:0/2283300212 learned_addr learned my addr 172.21.15.27:0/2283300212 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:25:14.658 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.654+0000 7f39dcb19700 1 --2- 172.21.15.27:0/2283300212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39d80f55c0 0x7f39d8105ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:25:14.659 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.654+0000 7f39d7fff700 1 -- 172.21.15.27:0/2283300212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39d80f55c0 msgr2=0x7f39d8105ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:25:14.659 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.654+0000 7f39d7fff700 1 --2- 172.21.15.27:0/2283300212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39d80f55c0 0x7f39d8105ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:14.659 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.654+0000 7f39d7fff700 1 -- 172.21.15.27:0/2283300212 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39cc005040 con 0x7f39d80f6860 2021-11-03T18:25:14.659 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.654+0000 7f39d7fff700 1 --2- 172.21.15.27:0/2283300212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39d80f6860 0x7f39d81063c0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 rx=0x7f39c4010a00 tx=0x7f39c400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:25:14.659 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.654+0000 7f39d57fa700 1 -- 172.21.15.27:0/2283300212 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39c40094c0 con 0x7f39d80f6860 2021-11-03T18:25:14.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.654+0000 7f39d57fa700 1 -- 172.21.15.27:0/2283300212 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f39c4010e90 con 0x7f39d80f6860 2021-11-03T18:25:14.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.654+0000 7f39ded7d700 1 -- 172.21.15.27:0/2283300212 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39d81073f0 con 0x7f39d80f6860 2021-11-03T18:25:14.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.654+0000 7f39d57fa700 1 -- 172.21.15.27:0/2283300212 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f39c4007db0 con 0x7f39d80f6860 2021-11-03T18:25:14.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.655+0000 7f39ded7d700 1 -- 172.21.15.27:0/2283300212 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39d8107980 con 0x7f39d80f6860 2021-11-03T18:25:14.661 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.656+0000 7f39ded7d700 1 -- 172.21.15.27:0/2283300212 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39d80f49b0 con 0x7f39d80f6860 2021-11-03T18:25:14.661 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.656+0000 7f39d57fa700 1 -- 172.21.15.27:0/2283300212 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f39c4022070 con 0x7f39d80f6860 2021-11-03T18:25:14.661 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.656+0000 7f39d57fa700 1 --2- 172.21.15.27:0/2283300212 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f39c8064090 0x7f39c8066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:25:14.661 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.656+0000 7f39d57fa700 1 -- 172.21.15.27:0/2283300212 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f39c40830b0 con 0x7f39d80f6860 2021-11-03T18:25:14.662 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.656+0000 7f39dcb19700 1 --2- 172.21.15.27:0/2283300212 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f39c8064090 0x7f39c8066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:25:14.662 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.657+0000 7f39dcb19700 1 --2- 172.21.15.27:0/2283300212 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f39c8064090 0x7f39c8066550 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f39cc000c00 tx=0x7f39cc016040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:25:14.662 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.661+0000 7f39d57fa700 1 -- 172.21.15.27:0/2283300212 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f39c4055320 con 0x7f39d80f6860 2021-11-03T18:25:14.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:14 smithi115 ceph-mon[65453]: pgmap v2119: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:14.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:14 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:25:14.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:14 smithi027 ceph-mon[97244]: pgmap v2119: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:14.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:14 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:25:14.834 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.835+0000 7f39ded7d700 1 -- 172.21.15.27:0/2283300212 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f39d80ff830 con 0x7f39c8064090 2021-11-03T18:25:14.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.850+0000 7f39d57fa700 1 -- 172.21.15.27:0/2283300212 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f39d80ff830 con 0x7f39c8064090 2021-11-03T18:25:14.849 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:25:14.849 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (76m) 5m ago 78m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:25:14.849 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (68m) 5m ago 78m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:25:14.850 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (68m) 5m ago 77m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:25:14.850 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (76m) 5m ago 77m 42.4M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:25:14.850 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (70m) 5m ago 73m 26.5M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:25:14.850 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (73m) 5m ago 73m 3849M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:25:14.850 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (69m) 5m ago 73m 26.1M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:25:14.851 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (70m) 5m ago 73m 27.2M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:25:14.851 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (70m) 5m ago 79m 590M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:25:14.851 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (69m) 5m ago 77m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:25:14.851 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (69m) 5m ago 79m 312M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:25:14.852 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (69m) 5m ago 77m 145M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:25:14.852 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (77m) 5m ago 77m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:25:14.852 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (77m) 5m ago 77m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:25:14.852 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (68m) 5m ago 76m 392M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:25:14.852 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (67m) 5m ago 76m 427M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:25:14.853 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (67m) 5m ago 75m 320M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:25:14.853 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (66m) 5m ago 75m 454M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:25:14.853 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (66m) 5m ago 75m 359M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:25:14.853 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (65m) 5m ago 74m 368M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:25:14.854 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (76m) 5m ago 77m 59.9M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:25:14.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.851+0000 7f39ded7d700 1 -- 172.21.15.27:0/2283300212 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f39c8064090 msgr2=0x7f39c8066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:25:14.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.851+0000 7f39ded7d700 1 --2- 172.21.15.27:0/2283300212 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f39c8064090 0x7f39c8066550 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 rx=0x7f39cc000c00 tx=0x7f39cc016040).stop 2021-11-03T18:25:14.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.851+0000 7f39ded7d700 1 -- 172.21.15.27:0/2283300212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39d80f6860 msgr2=0x7f39d81063c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:25:14.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.851+0000 7f39ded7d700 1 --2- 172.21.15.27:0/2283300212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39d80f6860 0x7f39d81063c0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 rx=0x7f39c4010a00 tx=0x7f39c400b040).stop 2021-11-03T18:25:14.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.852+0000 7f39ded7d700 1 -- 172.21.15.27:0/2283300212 shutdown_connections 2021-11-03T18:25:14.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.852+0000 7f39ded7d700 1 --2- 172.21.15.27:0/2283300212 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f39c8064090 0x7f39c8066550 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:14.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.852+0000 7f39ded7d700 1 --2- 172.21.15.27:0/2283300212 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f39d80f55c0 0x7f39d8105ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:14.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.852+0000 7f39ded7d700 1 --2- 172.21.15.27:0/2283300212 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f39d80f6860 0x7f39d81063c0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:14.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.852+0000 7f39ded7d700 1 -- 172.21.15.27:0/2283300212 >> 172.21.15.27:0/2283300212 conn(0x7f39d80f0d20 msgr2=0x7f39d80fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:25:14.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.852+0000 7f39ded7d700 1 -- 172.21.15.27:0/2283300212 shutdown_connections 2021-11-03T18:25:14.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.852+0000 7f39ded7d700 1 -- 172.21.15.27:0/2283300212 wait complete. 2021-11-03T18:25:14.950 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.951+0000 7f023a6c2700 1 -- 172.21.15.27:0/3670299594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02340f55c0 msgr2=0x7f02340f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:25:14.950 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.951+0000 7f023a6c2700 1 --2- 172.21.15.27:0/3670299594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02340f55c0 0x7f02340f59e0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 rx=0x7f0224004660 tx=0x7f0224009b30).stop 2021-11-03T18:25:14.951 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.951+0000 7f023a6c2700 1 -- 172.21.15.27:0/3670299594 shutdown_connections 2021-11-03T18:25:14.951 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.951+0000 7f023a6c2700 1 --2- 172.21.15.27:0/3670299594 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02340f6860 0x7f02340f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:14.951 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.951+0000 7f023a6c2700 1 --2- 172.21.15.27:0/3670299594 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02340f55c0 0x7f02340f59e0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:14.951 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.951+0000 7f023a6c2700 1 -- 172.21.15.27:0/3670299594 >> 172.21.15.27:0/3670299594 conn(0x7f02340f0d40 msgr2=0x7f02340f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:25:14.952 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.951+0000 7f023a6c2700 1 -- 172.21.15.27:0/3670299594 shutdown_connections 2021-11-03T18:25:14.952 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.951+0000 7f023a6c2700 1 -- 172.21.15.27:0/3670299594 wait complete. 2021-11-03T18:25:14.952 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.951+0000 7f023a6c2700 1 Processor -- start 2021-11-03T18:25:14.956 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.952+0000 7f023a6c2700 1 -- start start 2021-11-03T18:25:14.957 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.952+0000 7f023a6c2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02340f55c0 0x7f0234106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:25:14.957 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.952+0000 7f023a6c2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02340f6860 0x7f0234107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:25:14.957 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.952+0000 7f023a6c2700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0234107a10 con 0x7f02340f55c0 2021-11-03T18:25:14.957 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.952+0000 7f023a6c2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0234107b50 con 0x7f02340f6860 2021-11-03T18:25:14.958 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.952+0000 7f0233fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02340f55c0 0x7f0234106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:25:14.958 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.952+0000 7f0233fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02340f55c0 0x7f0234106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34768/0 (socket says 172.21.15.27:34768) 2021-11-03T18:25:14.958 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.952+0000 7f0233fff700 1 -- 172.21.15.27:0/2231917026 learned_addr learned my addr 172.21.15.27:0/2231917026 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:25:14.958 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.952+0000 7f02337fe700 1 --2- 172.21.15.27:0/2231917026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02340f6860 0x7f0234107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:25:14.959 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.952+0000 7f0233fff700 1 -- 172.21.15.27:0/2231917026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02340f6860 msgr2=0x7f0234107460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:25:14.959 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.952+0000 7f0233fff700 1 --2- 172.21.15.27:0/2231917026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02340f6860 0x7f0234107460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:14.959 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.952+0000 7f0233fff700 1 -- 172.21.15.27:0/2231917026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0224005040 con 0x7f02340f55c0 2021-11-03T18:25:14.959 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.952+0000 7f02337fe700 1 --2- 172.21.15.27:0/2231917026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02340f6860 0x7f0234107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:25:14.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.952+0000 7f0233fff700 1 --2- 172.21.15.27:0/2231917026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02340f55c0 0x7f0234106f70 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 rx=0x7f0224013a70 tx=0x7f0224009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:25:14.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.953+0000 7f0230ff9700 1 -- 172.21.15.27:0/2231917026 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0224018070 con 0x7f02340f55c0 2021-11-03T18:25:14.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.953+0000 7f023a6c2700 1 -- 172.21.15.27:0/2231917026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0234108490 con 0x7f02340f55c0 2021-11-03T18:25:14.961 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.953+0000 7f0230ff9700 1 -- 172.21.15.27:0/2231917026 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f02240097d0 con 0x7f02340f55c0 2021-11-03T18:25:14.961 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.953+0000 7f0230ff9700 1 -- 172.21.15.27:0/2231917026 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f022401c3c0 con 0x7f02340f55c0 2021-11-03T18:25:14.961 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.953+0000 7f023a6c2700 1 -- 172.21.15.27:0/2231917026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02341089f0 con 0x7f02340f55c0 2021-11-03T18:25:14.961 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.954+0000 7f023a6c2700 1 -- 172.21.15.27:0/2231917026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0234045ad0 con 0x7f02340f55c0 2021-11-03T18:25:14.962 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.954+0000 7f0230ff9700 1 -- 172.21.15.27:0/2231917026 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f0224007ba0 con 0x7f02340f55c0 2021-11-03T18:25:14.962 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.955+0000 7f0230ff9700 1 --2- 172.21.15.27:0/2231917026 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f021c0640b0 0x7f021c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:25:14.962 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.955+0000 7f0230ff9700 1 -- 172.21.15.27:0/2231917026 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f02240823a0 con 0x7f02340f55c0 2021-11-03T18:25:14.962 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.955+0000 7f02337fe700 1 --2- 172.21.15.27:0/2231917026 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f021c0640b0 0x7f021c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:25:14.963 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.955+0000 7f02337fe700 1 --2- 172.21.15.27:0/2231917026 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f021c0640b0 0x7f021c066570 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7f0228009740 tx=0x7f0228009be0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:25:14.963 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:14.959+0000 7f0230ff9700 1 -- 172.21.15.27:0/2231917026 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f02240547c0 con 0x7f02340f55c0 2021-11-03T18:25:15.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.185+0000 7f023a6c2700 1 -- 172.21.15.27:0/2231917026 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f023410d590 con 0x7f02340f55c0 2021-11-03T18:25:15.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.186+0000 7f0230ff9700 1 -- 172.21.15.27:0/2231917026 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f02240543b0 con 0x7f02340f55c0 2021-11-03T18:25:15.186 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:25:15.186 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:25:15.186 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:25:15.186 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:25:15.187 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:25:15.187 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:25:15.187 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:25:15.187 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:25:15.188 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:25:15.188 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:25:15.188 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:25:15.188 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:25:15.189 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:25:15.189 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:25:15.189 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:25:15.189 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:25:15.190 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:25:15.190 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:25:15.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.188+0000 7f023a6c2700 1 -- 172.21.15.27:0/2231917026 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f021c0640b0 msgr2=0x7f021c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:25:15.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.188+0000 7f023a6c2700 1 --2- 172.21.15.27:0/2231917026 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f021c0640b0 0x7f021c066570 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7f0228009740 tx=0x7f0228009be0).stop 2021-11-03T18:25:15.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.188+0000 7f023a6c2700 1 -- 172.21.15.27:0/2231917026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02340f55c0 msgr2=0x7f0234106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:25:15.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.188+0000 7f023a6c2700 1 --2- 172.21.15.27:0/2231917026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02340f55c0 0x7f0234106f70 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 rx=0x7f0224013a70 tx=0x7f0224009b30).stop 2021-11-03T18:25:15.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.188+0000 7f023a6c2700 1 -- 172.21.15.27:0/2231917026 shutdown_connections 2021-11-03T18:25:15.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.188+0000 7f023a6c2700 1 --2- 172.21.15.27:0/2231917026 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f021c0640b0 0x7f021c066570 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:15.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.188+0000 7f023a6c2700 1 --2- 172.21.15.27:0/2231917026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f02340f55c0 0x7f0234106f70 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:15.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.188+0000 7f023a6c2700 1 --2- 172.21.15.27:0/2231917026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f02340f6860 0x7f0234107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:15.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.188+0000 7f023a6c2700 1 -- 172.21.15.27:0/2231917026 >> 172.21.15.27:0/2231917026 conn(0x7f02340f0d40 msgr2=0x7f023405f620 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:25:15.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.188+0000 7f023a6c2700 1 -- 172.21.15.27:0/2231917026 shutdown_connections 2021-11-03T18:25:15.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.188+0000 7f023a6c2700 1 -- 172.21.15.27:0/2231917026 wait complete. 2021-11-03T18:25:15.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.288+0000 7ff63de1a700 1 -- 172.21.15.27:0/3543306375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6380f66e0 msgr2=0x7ff6380f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:25:15.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.288+0000 7ff63de1a700 1 --2- 172.21.15.27:0/3543306375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6380f66e0 0x7ff6380f6b00 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 rx=0x7ff6300099e0 tx=0x7ff630004d10).stop 2021-11-03T18:25:15.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.288+0000 7ff63de1a700 1 -- 172.21.15.27:0/3543306375 shutdown_connections 2021-11-03T18:25:15.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.288+0000 7ff63de1a700 1 --2- 172.21.15.27:0/3543306375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6380f7980 0x7ff6380f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:15.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.288+0000 7ff63de1a700 1 --2- 172.21.15.27:0/3543306375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6380f66e0 0x7ff6380f6b00 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:15.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.288+0000 7ff63de1a700 1 -- 172.21.15.27:0/3543306375 >> 172.21.15.27:0/3543306375 conn(0x7ff6380f1e40 msgr2=0x7ff6380f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:25:15.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.288+0000 7ff63de1a700 1 -- 172.21.15.27:0/3543306375 shutdown_connections 2021-11-03T18:25:15.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.288+0000 7ff63de1a700 1 -- 172.21.15.27:0/3543306375 wait complete. 2021-11-03T18:25:15.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.288+0000 7ff63de1a700 1 Processor -- start 2021-11-03T18:25:15.290 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.289+0000 7ff63de1a700 1 -- start start 2021-11-03T18:25:15.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.289+0000 7ff63de1a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6380f66e0 0x7ff6381070d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:25:15.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.289+0000 7ff63de1a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6380f7980 0x7ff6381075c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:25:15.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.289+0000 7ff63de1a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff638107b70 con 0x7ff6380f66e0 2021-11-03T18:25:15.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.289+0000 7ff63de1a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff638107cb0 con 0x7ff6380f7980 2021-11-03T18:25:15.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.289+0000 7ff6377fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6380f66e0 0x7ff6381070d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:25:15.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.289+0000 7ff6377fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6380f66e0 0x7ff6381070d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34778/0 (socket says 172.21.15.27:34778) 2021-11-03T18:25:15.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.289+0000 7ff6377fe700 1 -- 172.21.15.27:0/2351857389 learned_addr learned my addr 172.21.15.27:0/2351857389 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:25:15.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.289+0000 7ff6377fe700 1 -- 172.21.15.27:0/2351857389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6380f7980 msgr2=0x7ff6381075c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:25:15.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.289+0000 7ff6377fe700 1 --2- 172.21.15.27:0/2351857389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6380f7980 0x7ff6381075c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:25:15.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.289+0000 7ff6377fe700 1 -- 172.21.15.27:0/2351857389 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff630009640 con 0x7ff6380f66e0 2021-11-03T18:25:15.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.289+0000 7ff6377fe700 1 --2- 172.21.15.27:0/2351857389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6380f66e0 0x7ff6381070d0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 rx=0x7ff63000c010 tx=0x7ff630004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:25:15.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.289+0000 7ff62ffff700 1 -- 172.21.15.27:0/2351857389 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff63000e070 con 0x7ff6380f66e0 2021-11-03T18:25:15.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.289+0000 7ff63de1a700 1 -- 172.21.15.27:0/2351857389 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6381085f0 con 0x7ff6380f66e0 2021-11-03T18:25:15.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.290+0000 7ff62ffff700 1 -- 172.21.15.27:0/2351857389 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7ff630007dc0 con 0x7ff6380f66e0 2021-11-03T18:25:15.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.290+0000 7ff62ffff700 1 -- 172.21.15.27:0/2351857389 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff630018070 con 0x7ff6380f66e0 2021-11-03T18:25:15.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.290+0000 7ff63de1a700 1 -- 172.21.15.27:0/2351857389 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff638108ad0 con 0x7ff6380f66e0 2021-11-03T18:25:15.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.290+0000 7ff63de1a700 1 -- 172.21.15.27:0/2351857389 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff638045ad0 con 0x7ff6380f66e0 2021-11-03T18:25:15.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.291+0000 7ff62ffff700 1 -- 172.21.15.27:0/2351857389 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7ff630024060 con 0x7ff6380f66e0 2021-11-03T18:25:15.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.291+0000 7ff62ffff700 1 --2- 172.21.15.27:0/2351857389 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff6240640b0 0x7ff624066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:25:15.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.291+0000 7ff62ffff700 1 -- 172.21.15.27:0/2351857389 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7ff6300820c0 con 0x7ff6380f66e0 2021-11-03T18:25:15.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.291+0000 7ff636ffd700 1 --2- 172.21.15.27:0/2351857389 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff6240640b0 0x7ff624066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:25:15.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.292+0000 7ff636ffd700 1 --2- 172.21.15.27:0/2351857389 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff6240640b0 0x7ff624066570 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7ff620004a10 tx=0x7ff6200099e0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:25:15.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.295+0000 7ff62ffff700 1 -- 172.21.15.27:0/2351857389 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7ff630054420 con 0x7ff6380f66e0 2021-11-03T18:25:15.503 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.504+0000 7ff63de1a700 1 -- 172.21.15.27:0/2351857389 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff638027a00 con 0x7ff6380f66e0 2021-11-03T18:25:15.504 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.504+0000 7ff62ffff700 1 -- 172.21.15.27:0/2351857389 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7ff630022070 con 0x7ff6380f66e0 2021-11-03T18:25:15.504 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:25:15.505 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:25:15.505 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:25:15.505 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:25:15.506 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:25:15.506 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:25:15.506 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:25:15.506 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:25:15.506 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:25:15.507 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:25:15.507 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:25:15.507 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:25:15.507 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:25:15.508 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:25:15.508 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:25:15.508 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:25:15.508 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:25:15.509 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:25:15.509 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:25:15.509 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:25:15.509 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:25:15.509 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:25:15.510 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:25:15.510 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:25:15.510 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:25:15.510 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:25:15.511 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:25:15.511 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:25:15.511 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:25:15.512 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:25:15.512 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:25:15.512 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:25:15.512 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:25:15.512 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:25:15.513 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:25:15.513 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:25:15.513 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:25:15.513 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:25:15.514 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:25:15.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.506+0000 7ff63de1a700 1 -- 172.21.15.27:0/2351857389 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff6240640b0 msgr2=0x7ff624066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:25:15.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.506+0000 7ff63de1a700 1 --2- 172.21.15.27:0/2351857389 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff6240640b0 0x7ff624066570 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7ff620004a10 tx=0x7ff6200099e0).stop 2021-11-03T18:25:15.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.506+0000 7ff63de1a700 1 -- 172.21.15.27:0/2351857389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6380f66e0 msgr2=0x7ff6381070d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:25:15.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.506+0000 7ff63de1a700 1 --2- 172.21.15.27:0/2351857389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6380f66e0 0x7ff6381070d0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 rx=0x7ff63000c010 tx=0x7ff630004d10).stop 2021-11-03T18:25:15.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.506+0000 7ff63de1a700 1 -- 172.21.15.27:0/2351857389 shutdown_connections 2021-11-03T18:25:15.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.506+0000 7ff63de1a700 1 --2- 172.21.15.27:0/2351857389 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff6240640b0 0x7ff624066570 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:15.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.506+0000 7ff63de1a700 1 --2- 172.21.15.27:0/2351857389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff6380f66e0 0x7ff6381070d0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:15.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.506+0000 7ff63de1a700 1 --2- 172.21.15.27:0/2351857389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff6380f7980 0x7ff6381075c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:25:15.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.506+0000 7ff63de1a700 1 -- 172.21.15.27:0/2351857389 >> 172.21.15.27:0/2351857389 conn(0x7ff6380f1e40 msgr2=0x7ff6380fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:25:15.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.506+0000 7ff63de1a700 1 -- 172.21.15.27:0/2351857389 shutdown_connections 2021-11-03T18:25:15.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:15.506+0000 7ff63de1a700 1 -- 172.21.15.27:0/2351857389 wait complete. 2021-11-03T18:25:15.517 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:25:15.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4157.459431 secs 2021-11-03T18:25:15.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:15 smithi115 ceph-mon[65453]: from='client.36280 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:25:15.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:15 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2231917026' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:15.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4157.459431 secs 2021-11-03T18:25:15.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:15 smithi027 ceph-mon[97244]: from='client.36280 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:25:15.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:15 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2231917026' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:16.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:16 smithi115 ceph-mon[65453]: from='client.36284 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:25:16.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:16 smithi115 ceph-mon[65453]: pgmap v2120: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:16.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:16 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2351857389' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:25:16.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:16 smithi027 ceph-mon[97244]: from='client.36284 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:25:16.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:16 smithi027 ceph-mon[97244]: pgmap v2120: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:16.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:16 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2351857389' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:25:18.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:18 smithi115 ceph-mon[65453]: pgmap v2121: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:18.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:18 smithi027 ceph-mon[97244]: pgmap v2121: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:20.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:20 smithi115 ceph-mon[65453]: pgmap v2122: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:20.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4162.459554 secs 2021-11-03T18:25:20.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:20 smithi027 ceph-mon[97244]: pgmap v2122: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:20.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4162.459554 secs 2021-11-03T18:25:22.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:22 smithi115 ceph-mon[65453]: pgmap v2123: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:22.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:22 smithi027 ceph-mon[97244]: pgmap v2123: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:23.471 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:23 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:25:23.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:23 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:25:24.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:24 smithi115 ceph-mon[65453]: pgmap v2124: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:24.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:24 smithi115 ceph-mon[65453]: poll took 60179.937 ms: 1 events 2021-11-03T18:25:24.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:24 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:25:24.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:24 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:25:24.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:24 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:24.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:24 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:24.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:24 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:24.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:24 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:24.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:24 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:25:25.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:24 smithi027 ceph-mon[97244]: pgmap v2124: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:25.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:24 smithi027 ceph-mon[97244]: poll took 60179.937 ms: 1 events 2021-11-03T18:25:25.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:24 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:25:25.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:24 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:25:25.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:24 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:25.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:24 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:25.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:24 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:25.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:24 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:25.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:24 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:25:25.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4167.459614 secs 2021-11-03T18:25:26.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4167.459614 secs 2021-11-03T18:25:26.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:26 smithi115 ceph-mon[65453]: pgmap v2125: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:27.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:26 smithi027 ceph-mon[97244]: pgmap v2125: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:28.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:28 smithi115 ceph-mon[65453]: pgmap v2126: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:29.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:28 smithi027 ceph-mon[97244]: pgmap v2126: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:30.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:30 smithi115 ceph-mon[65453]: pgmap v2127: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:30.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4172.459755 secs 2021-11-03T18:25:31.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:30 smithi027 ceph-mon[97244]: pgmap v2127: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:31.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4172.459755 secs 2021-11-03T18:25:32.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:32 smithi027 ceph-mon[97244]: pgmap v2128: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:32.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:32 smithi115 ceph-mon[65453]: pgmap v2128: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:34.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:34 smithi115 ceph-mon[65453]: pgmap v2129: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:34.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:25:34.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:25:34.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:25:34.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:34.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:34.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:34.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:34.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:34 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:25:35.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:34 smithi027 ceph-mon[97244]: pgmap v2129: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:35.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:25:35.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:25:35.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:25:35.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:35.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:35.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:35.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:35.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:34 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:25:35.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4177.459873 secs 2021-11-03T18:25:36.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4177.459873 secs 2021-11-03T18:25:36.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:36 smithi115 ceph-mon[65453]: pgmap v2130: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:37.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:36 smithi027 ceph-mon[97244]: pgmap v2130: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:38.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:38 smithi115 ceph-mon[65453]: pgmap v2131: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:39.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:38 smithi027 ceph-mon[97244]: pgmap v2131: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:40.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:40 smithi115 ceph-mon[65453]: pgmap v2132: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:40.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4182.460019 secs 2021-11-03T18:25:41.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:40 smithi027 ceph-mon[97244]: pgmap v2132: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:41.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4182.460019 secs 2021-11-03T18:25:42.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:42 smithi027 ceph-mon[97244]: pgmap v2133: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:42.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:42 smithi115 ceph-mon[65453]: pgmap v2133: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:44.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:44 smithi115 ceph-mon[65453]: pgmap v2134: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:44.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:25:44.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:25:44.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:25:44.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:44.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:44.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:44.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:44.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:44 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:25:45.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:44 smithi027 ceph-mon[97244]: pgmap v2134: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:45.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:25:45.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:25:45.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:25:45.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:45.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:45.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:45.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:45.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:44 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:25:45.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.624+0000 7fbc18953700 1 -- 172.21.15.27:0/1069280711 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc140f77d0 msgr2=0x7fbc140f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:25:45.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.624+0000 7fbc18953700 1 --2- 172.21.15.27:0/1069280711 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc140f77d0 0x7fbc140f7c30 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 rx=0x7fbc08004660 tx=0x7fbc08009b30).stop 2021-11-03T18:25:45.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.624+0000 7fbc18953700 1 -- 172.21.15.27:0/1069280711 shutdown_connections 2021-11-03T18:25:45.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.624+0000 7fbc18953700 1 --2- 172.21.15.27:0/1069280711 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc140f77d0 0x7fbc140f7c30 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:45.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.624+0000 7fbc18953700 1 --2- 172.21.15.27:0/1069280711 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc140f6530 0x7fbc140f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:45.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.624+0000 7fbc18953700 1 -- 172.21.15.27:0/1069280711 >> 172.21.15.27:0/1069280711 conn(0x7fbc140f1cd0 msgr2=0x7fbc140f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:25:45.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.624+0000 7fbc18953700 1 -- 172.21.15.27:0/1069280711 shutdown_connections 2021-11-03T18:25:45.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.624+0000 7fbc18953700 1 -- 172.21.15.27:0/1069280711 wait complete. 2021-11-03T18:25:45.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.624+0000 7fbc18953700 1 Processor -- start 2021-11-03T18:25:45.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.625+0000 7fbc18953700 1 -- start start 2021-11-03T18:25:45.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.625+0000 7fbc18953700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc140f6530 0x7fbc1406b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:25:45.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.625+0000 7fbc18953700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc140f77d0 0x7fbc14069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:25:45.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.625+0000 7fbc18953700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1406a380 con 0x7fbc140f77d0 2021-11-03T18:25:45.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.625+0000 7fbc18953700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1406a4c0 con 0x7fbc140f6530 2021-11-03T18:25:45.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.625+0000 7fbc12ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc140f77d0 0x7fbc14069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:25:45.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.625+0000 7fbc12ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc140f77d0 0x7fbc14069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34798/0 (socket says 172.21.15.27:34798) 2021-11-03T18:25:45.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.625+0000 7fbc12ffd700 1 -- 172.21.15.27:0/1759408437 learned_addr learned my addr 172.21.15.27:0/1759408437 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:25:45.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.625+0000 7fbc137fe700 1 --2- 172.21.15.27:0/1759408437 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc140f6530 0x7fbc1406b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:25:45.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.625+0000 7fbc12ffd700 1 -- 172.21.15.27:0/1759408437 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc140f6530 msgr2=0x7fbc1406b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:25:45.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.625+0000 7fbc12ffd700 1 --2- 172.21.15.27:0/1759408437 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc140f6530 0x7fbc1406b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:45.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.625+0000 7fbc12ffd700 1 -- 172.21.15.27:0/1759408437 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc08005040 con 0x7fbc140f77d0 2021-11-03T18:25:45.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.625+0000 7fbc12ffd700 1 --2- 172.21.15.27:0/1759408437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc140f77d0 0x7fbc14069e20 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 rx=0x7fbc08009990 tx=0x7fbc08009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:25:45.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.626+0000 7fbbfbfff700 1 -- 172.21.15.27:0/1759408437 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc08018070 con 0x7fbc140f77d0 2021-11-03T18:25:45.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.626+0000 7fbbfbfff700 1 -- 172.21.15.27:0/1759408437 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fbc08007ba0 con 0x7fbc140f77d0 2021-11-03T18:25:45.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.626+0000 7fbc18953700 1 -- 172.21.15.27:0/1759408437 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc1406a740 con 0x7fbc140f77d0 2021-11-03T18:25:45.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.626+0000 7fbbfbfff700 1 -- 172.21.15.27:0/1759408437 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc0801c3c0 con 0x7fbc140f77d0 2021-11-03T18:25:45.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.626+0000 7fbc18953700 1 -- 172.21.15.27:0/1759408437 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc1406aca0 con 0x7fbc140f77d0 2021-11-03T18:25:45.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.627+0000 7fbc18953700 1 -- 172.21.15.27:0/1759408437 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc14045ad0 con 0x7fbc140f77d0 2021-11-03T18:25:45.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.627+0000 7fbbfbfff700 1 -- 172.21.15.27:0/1759408437 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fbc0801d420 con 0x7fbc140f77d0 2021-11-03T18:25:45.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.628+0000 7fbbfbfff700 1 --2- 172.21.15.27:0/1759408437 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbbfc064100 0x7fbbfc0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:25:45.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.628+0000 7fbbfbfff700 1 -- 172.21.15.27:0/1759408437 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fbc08082500 con 0x7fbc140f77d0 2021-11-03T18:25:45.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.628+0000 7fbc137fe700 1 --2- 172.21.15.27:0/1759408437 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbbfc064100 0x7fbbfc0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:25:45.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.628+0000 7fbc137fe700 1 --2- 172.21.15.27:0/1759408437 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbbfc064100 0x7fbbfc0665c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7fbc1406b730 tx=0x7fbc04009c00).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:25:45.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.632+0000 7fbbfbfff700 1 -- 172.21.15.27:0/1759408437 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fbc080548c0 con 0x7fbc140f77d0 2021-11-03T18:25:45.811 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.813+0000 7fbc18953700 1 -- 172.21.15.27:0/1759408437 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc140fee00 con 0x7fbbfc064100 2021-11-03T18:25:45.812 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.814+0000 7fbbfbfff700 1 -- 172.21.15.27:0/1759408437 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fbc140fee00 con 0x7fbbfc064100 2021-11-03T18:25:45.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.816+0000 7fbc18953700 1 -- 172.21.15.27:0/1759408437 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbbfc064100 msgr2=0x7fbbfc0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:25:45.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.816+0000 7fbc18953700 1 --2- 172.21.15.27:0/1759408437 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbbfc064100 0x7fbbfc0665c0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7fbc1406b730 tx=0x7fbc04009c00).stop 2021-11-03T18:25:45.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.816+0000 7fbc18953700 1 -- 172.21.15.27:0/1759408437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc140f77d0 msgr2=0x7fbc14069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:25:45.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.816+0000 7fbc18953700 1 --2- 172.21.15.27:0/1759408437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc140f77d0 0x7fbc14069e20 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 rx=0x7fbc08009990 tx=0x7fbc08009b30).stop 2021-11-03T18:25:45.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.816+0000 7fbc18953700 1 -- 172.21.15.27:0/1759408437 shutdown_connections 2021-11-03T18:25:45.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.816+0000 7fbc18953700 1 --2- 172.21.15.27:0/1759408437 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbbfc064100 0x7fbbfc0665c0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:45.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.816+0000 7fbc18953700 1 --2- 172.21.15.27:0/1759408437 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbc140f6530 0x7fbc1406b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:45.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.816+0000 7fbc18953700 1 --2- 172.21.15.27:0/1759408437 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbc140f77d0 0x7fbc14069e20 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:45.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.816+0000 7fbc18953700 1 -- 172.21.15.27:0/1759408437 >> 172.21.15.27:0/1759408437 conn(0x7fbc140f1cd0 msgr2=0x7fbc140fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:25:45.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.816+0000 7fbc18953700 1 -- 172.21.15.27:0/1759408437 shutdown_connections 2021-11-03T18:25:45.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.816+0000 7fbc18953700 1 -- 172.21.15.27:0/1759408437 wait complete. 2021-11-03T18:25:45.824 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:25:45.916 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.917+0000 7f1e2d363700 1 -- 172.21.15.27:0/3672977770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e280f6860 msgr2=0x7f1e280f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:25:45.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.917+0000 7f1e2d363700 1 --2- 172.21.15.27:0/3672977770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e280f6860 0x7f1e280f6cc0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 rx=0x7f1e10004660 tx=0x7f1e10009b30).stop 2021-11-03T18:25:45.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.917+0000 7f1e2d363700 1 -- 172.21.15.27:0/3672977770 shutdown_connections 2021-11-03T18:25:45.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.917+0000 7f1e2d363700 1 --2- 172.21.15.27:0/3672977770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e280f6860 0x7f1e280f6cc0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:45.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.917+0000 7f1e2d363700 1 --2- 172.21.15.27:0/3672977770 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e280f55c0 0x7f1e280f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:45.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.917+0000 7f1e2d363700 1 -- 172.21.15.27:0/3672977770 >> 172.21.15.27:0/3672977770 conn(0x7f1e280f0d40 msgr2=0x7f1e280f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:25:45.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.917+0000 7f1e2d363700 1 -- 172.21.15.27:0/3672977770 shutdown_connections 2021-11-03T18:25:45.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.917+0000 7f1e2d363700 1 -- 172.21.15.27:0/3672977770 wait complete. 2021-11-03T18:25:45.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.917+0000 7f1e2d363700 1 Processor -- start 2021-11-03T18:25:45.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.917+0000 7f1e2d363700 1 -- start start 2021-11-03T18:25:45.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.918+0000 7f1e2d363700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e280f55c0 0x7f1e28109c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:25:45.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.918+0000 7f1e2d363700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e280f6860 0x7f1e2810c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:25:45.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.918+0000 7f1e2d363700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e280603e0 con 0x7f1e280f55c0 2021-11-03T18:25:45.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.918+0000 7f1e2d363700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e28060520 con 0x7f1e280f6860 2021-11-03T18:25:45.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.918+0000 7f1e26ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e280f55c0 0x7f1e28109c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:25:45.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.918+0000 7f1e26ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e280f55c0 0x7f1e28109c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34806/0 (socket says 172.21.15.27:34806) 2021-11-03T18:25:45.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.918+0000 7f1e26ffd700 1 -- 172.21.15.27:0/57568443 learned_addr learned my addr 172.21.15.27:0/57568443 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:25:45.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.918+0000 7f1e1ffff700 1 --2- 172.21.15.27:0/57568443 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e280f6860 0x7f1e2810c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:25:45.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.918+0000 7f1e26ffd700 1 -- 172.21.15.27:0/57568443 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e280f6860 msgr2=0x7f1e2810c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:25:45.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.918+0000 7f1e26ffd700 1 --2- 172.21.15.27:0/57568443 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e280f6860 0x7f1e2810c150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:45.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.918+0000 7f1e26ffd700 1 -- 172.21.15.27:0/57568443 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e10005040 con 0x7f1e280f55c0 2021-11-03T18:25:45.922 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.918+0000 7f1e26ffd700 1 --2- 172.21.15.27:0/57568443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e280f55c0 0x7f1e28109c50 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 rx=0x7f1e18002700 tx=0x7f1e18009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:25:45.922 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.918+0000 7f1e1f7fe700 1 -- 172.21.15.27:0/57568443 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e18009320 con 0x7f1e280f55c0 2021-11-03T18:25:45.922 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.918+0000 7f1e2d363700 1 -- 172.21.15.27:0/57568443 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e2810c700 con 0x7f1e280f55c0 2021-11-03T18:25:45.922 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.918+0000 7f1e1f7fe700 1 -- 172.21.15.27:0/57568443 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f1e1800f040 con 0x7f1e280f55c0 2021-11-03T18:25:45.923 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.918+0000 7f1e1f7fe700 1 -- 172.21.15.27:0/57568443 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e18014900 con 0x7f1e280f55c0 2021-11-03T18:25:45.923 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.918+0000 7f1e2d363700 1 -- 172.21.15.27:0/57568443 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e2810ccc0 con 0x7f1e280f55c0 2021-11-03T18:25:45.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.919+0000 7f1e2d363700 1 -- 172.21.15.27:0/57568443 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e28045ad0 con 0x7f1e280f55c0 2021-11-03T18:25:45.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.920+0000 7f1e1f7fe700 1 -- 172.21.15.27:0/57568443 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f1e18009480 con 0x7f1e280f55c0 2021-11-03T18:25:45.925 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.920+0000 7f1e1f7fe700 1 --2- 172.21.15.27:0/57568443 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1e140640b0 0x7f1e14066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:25:45.925 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.920+0000 7f1e1f7fe700 1 -- 172.21.15.27:0/57568443 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f1e18080a50 con 0x7f1e280f55c0 2021-11-03T18:25:45.925 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.920+0000 7f1e1ffff700 1 --2- 172.21.15.27:0/57568443 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1e140640b0 0x7f1e14066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:25:45.925 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.921+0000 7f1e1ffff700 1 --2- 172.21.15.27:0/57568443 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1e140640b0 0x7f1e14066570 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f1e10004630 tx=0x7f1e10009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:25:45.925 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:45.924+0000 7f1e1f7fe700 1 -- 172.21.15.27:0/57568443 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f1e18052db0 con 0x7f1e280f55c0 2021-11-03T18:25:45.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4187.460161 secs 2021-11-03T18:25:46.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4187.460161 secs 2021-11-03T18:25:46.097 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.099+0000 7f1e2d363700 1 -- 172.21.15.27:0/57568443 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e280fdf30 con 0x7f1e140640b0 2021-11-03T18:25:46.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.114+0000 7f1e1f7fe700 1 -- 172.21.15.27:0/57568443 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f1e10019040 con 0x7f1e140640b0 2021-11-03T18:25:46.112 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:25:46.113 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (77m) 6m ago 78m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:25:46.113 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (69m) 6m ago 78m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:25:46.113 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (68m) 5m ago 77m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:25:46.113 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (77m) 6m ago 78m 42.4M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:25:46.113 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (70m) 6m ago 73m 26.5M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:25:46.114 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (73m) 6m ago 73m 3849M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:25:46.114 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (69m) 5m ago 73m 26.1M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:25:46.114 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (71m) 5m ago 73m 27.2M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:25:46.114 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (71m) 6m ago 79m 590M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:25:46.115 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (70m) 5m ago 77m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:25:46.115 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (70m) 6m ago 79m 312M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:25:46.115 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (69m) 5m ago 77m 145M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:25:46.115 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (78m) 6m ago 78m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:25:46.115 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (77m) 5m ago 77m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:25:46.116 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (68m) 6m ago 77m 392M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:25:46.116 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (68m) 6m ago 76m 427M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:25:46.116 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (67m) 6m ago 76m 320M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:25:46.116 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (67m) 5m ago 75m 454M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:25:46.116 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (66m) 5m ago 75m 359M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:25:46.117 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (66m) 5m ago 75m 368M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:25:46.117 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (77m) 6m ago 77m 59.9M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:25:46.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.116+0000 7f1e2d363700 1 -- 172.21.15.27:0/57568443 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1e140640b0 msgr2=0x7f1e14066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:25:46.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.116+0000 7f1e2d363700 1 --2- 172.21.15.27:0/57568443 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1e140640b0 0x7f1e14066570 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f1e10004630 tx=0x7f1e10009b30).stop 2021-11-03T18:25:46.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.116+0000 7f1e2d363700 1 -- 172.21.15.27:0/57568443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e280f55c0 msgr2=0x7f1e28109c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:25:46.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.116+0000 7f1e2d363700 1 --2- 172.21.15.27:0/57568443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e280f55c0 0x7f1e28109c50 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 rx=0x7f1e18002700 tx=0x7f1e18009d80).stop 2021-11-03T18:25:46.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.116+0000 7f1e2d363700 1 -- 172.21.15.27:0/57568443 shutdown_connections 2021-11-03T18:25:46.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.116+0000 7f1e2d363700 1 --2- 172.21.15.27:0/57568443 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1e140640b0 0x7f1e14066570 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:46.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.116+0000 7f1e2d363700 1 --2- 172.21.15.27:0/57568443 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e280f55c0 0x7f1e28109c50 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:46.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.116+0000 7f1e2d363700 1 --2- 172.21.15.27:0/57568443 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e280f6860 0x7f1e2810c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:46.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.116+0000 7f1e2d363700 1 -- 172.21.15.27:0/57568443 >> 172.21.15.27:0/57568443 conn(0x7f1e280f0d40 msgr2=0x7f1e280fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:25:46.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.116+0000 7f1e2d363700 1 -- 172.21.15.27:0/57568443 shutdown_connections 2021-11-03T18:25:46.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.116+0000 7f1e2d363700 1 -- 172.21.15.27:0/57568443 wait complete. 2021-11-03T18:25:46.219 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.216+0000 7f310381e700 1 -- 172.21.15.27:0/2268516851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30fc0f66e0 msgr2=0x7f30fc0f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:25:46.219 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.216+0000 7f310381e700 1 --2- 172.21.15.27:0/2268516851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30fc0f66e0 0x7f30fc0f6b00 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 rx=0x7f30ec004660 tx=0x7f30ec009b30).stop 2021-11-03T18:25:46.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.216+0000 7f310381e700 1 -- 172.21.15.27:0/2268516851 shutdown_connections 2021-11-03T18:25:46.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.216+0000 7f310381e700 1 --2- 172.21.15.27:0/2268516851 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30fc0f7980 0x7f30fc0f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:46.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.216+0000 7f310381e700 1 --2- 172.21.15.27:0/2268516851 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30fc0f66e0 0x7f30fc0f6b00 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:46.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.216+0000 7f310381e700 1 -- 172.21.15.27:0/2268516851 >> 172.21.15.27:0/2268516851 conn(0x7f30fc0f1e40 msgr2=0x7f30fc0f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:25:46.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.216+0000 7f310381e700 1 -- 172.21.15.27:0/2268516851 shutdown_connections 2021-11-03T18:25:46.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.216+0000 7f310381e700 1 -- 172.21.15.27:0/2268516851 wait complete. 2021-11-03T18:25:46.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.216+0000 7f310381e700 1 Processor -- start 2021-11-03T18:25:46.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.216+0000 7f310381e700 1 -- start start 2021-11-03T18:25:46.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.217+0000 7f310381e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30fc0f66e0 0x7f30fc106fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:25:46.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.217+0000 7f310381e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30fc0f7980 0x7f30fc1074a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:25:46.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.217+0000 7f310381e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc107a50 con 0x7f30fc0f66e0 2021-11-03T18:25:46.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.217+0000 7f310381e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc107b90 con 0x7f30fc0f7980 2021-11-03T18:25:46.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.217+0000 7f31015ba700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30fc0f66e0 0x7f30fc106fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:25:46.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.217+0000 7f31015ba700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30fc0f66e0 0x7f30fc106fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34816/0 (socket says 172.21.15.27:34816) 2021-11-03T18:25:46.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.217+0000 7f31015ba700 1 -- 172.21.15.27:0/1703080464 learned_addr learned my addr 172.21.15.27:0/1703080464 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:25:46.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.217+0000 7f3100db9700 1 --2- 172.21.15.27:0/1703080464 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30fc0f7980 0x7f30fc1074a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:25:46.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.217+0000 7f31015ba700 1 -- 172.21.15.27:0/1703080464 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30fc0f7980 msgr2=0x7f30fc1074a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:25:46.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.217+0000 7f31015ba700 1 --2- 172.21.15.27:0/1703080464 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30fc0f7980 0x7f30fc1074a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:46.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.217+0000 7f31015ba700 1 -- 172.21.15.27:0/1703080464 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30ec005040 con 0x7f30fc0f66e0 2021-11-03T18:25:46.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.217+0000 7f31015ba700 1 --2- 172.21.15.27:0/1703080464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30fc0f66e0 0x7f30fc106fb0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 rx=0x7f30ec004f60 tx=0x7f30ec009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:25:46.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.217+0000 7f30f1ffb700 1 -- 172.21.15.27:0/1703080464 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30ec018070 con 0x7f30fc0f66e0 2021-11-03T18:25:46.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.217+0000 7f30f1ffb700 1 -- 172.21.15.27:0/1703080464 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f30ec007ba0 con 0x7f30fc0f66e0 2021-11-03T18:25:46.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.217+0000 7f310381e700 1 -- 172.21.15.27:0/1703080464 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30fc1084d0 con 0x7f30fc0f66e0 2021-11-03T18:25:46.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.217+0000 7f30f1ffb700 1 -- 172.21.15.27:0/1703080464 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f30ec01c620 con 0x7f30fc0f66e0 2021-11-03T18:25:46.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.217+0000 7f310381e700 1 -- 172.21.15.27:0/1703080464 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30fc108a30 con 0x7f30fc0f66e0 2021-11-03T18:25:46.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.218+0000 7f310381e700 1 -- 172.21.15.27:0/1703080464 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30fc045ad0 con 0x7f30fc0f66e0 2021-11-03T18:25:46.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.219+0000 7f30f1ffb700 1 -- 172.21.15.27:0/1703080464 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f30ec01d420 con 0x7f30fc0f66e0 2021-11-03T18:25:46.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.219+0000 7f30f1ffb700 1 --2- 172.21.15.27:0/1703080464 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f30e80640b0 0x7f30e8066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:25:46.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.219+0000 7f30f1ffb700 1 -- 172.21.15.27:0/1703080464 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f30ec082620 con 0x7f30fc0f66e0 2021-11-03T18:25:46.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.219+0000 7f3100db9700 1 --2- 172.21.15.27:0/1703080464 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f30e80640b0 0x7f30e8066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:25:46.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.220+0000 7f3100db9700 1 --2- 172.21.15.27:0/1703080464 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f30e80640b0 0x7f30e8066570 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7f30f80041d0 tx=0x7f30f8009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:25:46.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.223+0000 7f30f1ffb700 1 -- 172.21.15.27:0/1703080464 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f30ec054a40 con 0x7f30fc0f66e0 2021-11-03T18:25:46.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.454+0000 7f310381e700 1 -- 172.21.15.27:0/1703080464 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f30fc027a00 con 0x7f30fc0f66e0 2021-11-03T18:25:46.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.455+0000 7f30f1ffb700 1 -- 172.21.15.27:0/1703080464 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f30ec054630 con 0x7f30fc0f66e0 2021-11-03T18:25:46.455 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:25:46.455 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:25:46.456 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:25:46.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:25:46.456 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:25:46.456 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:25:46.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:25:46.457 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:25:46.457 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:25:46.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:25:46.458 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:25:46.458 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:25:46.459 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:25:46.459 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:25:46.460 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:25:46.460 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:25:46.460 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:25:46.461 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:25:46.462 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.457+0000 7f310381e700 1 -- 172.21.15.27:0/1703080464 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f30e80640b0 msgr2=0x7f30e8066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:25:46.463 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.457+0000 7f310381e700 1 --2- 172.21.15.27:0/1703080464 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f30e80640b0 0x7f30e8066570 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7f30f80041d0 tx=0x7f30f8009040).stop 2021-11-03T18:25:46.463 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.457+0000 7f310381e700 1 -- 172.21.15.27:0/1703080464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30fc0f66e0 msgr2=0x7f30fc106fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:25:46.463 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.457+0000 7f310381e700 1 --2- 172.21.15.27:0/1703080464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30fc0f66e0 0x7f30fc106fb0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 rx=0x7f30ec004f60 tx=0x7f30ec009b30).stop 2021-11-03T18:25:46.464 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.457+0000 7f310381e700 1 -- 172.21.15.27:0/1703080464 shutdown_connections 2021-11-03T18:25:46.464 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.457+0000 7f310381e700 1 --2- 172.21.15.27:0/1703080464 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f30e80640b0 0x7f30e8066570 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:46.464 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.457+0000 7f310381e700 1 --2- 172.21.15.27:0/1703080464 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f30fc0f66e0 0x7f30fc106fb0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:46.464 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.457+0000 7f310381e700 1 --2- 172.21.15.27:0/1703080464 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f30fc0f7980 0x7f30fc1074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:46.465 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.457+0000 7f310381e700 1 -- 172.21.15.27:0/1703080464 >> 172.21.15.27:0/1703080464 conn(0x7f30fc0f1e40 msgr2=0x7f30fc0fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:25:46.465 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.457+0000 7f310381e700 1 -- 172.21.15.27:0/1703080464 shutdown_connections 2021-11-03T18:25:46.465 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.457+0000 7f310381e700 1 -- 172.21.15.27:0/1703080464 wait complete. 2021-11-03T18:25:46.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.557+0000 7f964d467700 1 -- 172.21.15.27:0/2035761115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96480f66e0 msgr2=0x7f96480f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:25:46.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.557+0000 7f964d467700 1 --2- 172.21.15.27:0/2035761115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96480f66e0 0x7f96480f6b40 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 rx=0x7f963c004660 tx=0x7f963c009b30).stop 2021-11-03T18:25:46.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.557+0000 7f964d467700 1 -- 172.21.15.27:0/2035761115 shutdown_connections 2021-11-03T18:25:46.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.557+0000 7f964d467700 1 --2- 172.21.15.27:0/2035761115 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96480f66e0 0x7f96480f6b40 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:46.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.557+0000 7f964d467700 1 --2- 172.21.15.27:0/2035761115 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96480f5440 0x7f96480f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:46.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.557+0000 7f964d467700 1 -- 172.21.15.27:0/2035761115 >> 172.21.15.27:0/2035761115 conn(0x7f96480f0ba0 msgr2=0x7f96480f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:25:46.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.557+0000 7f964d467700 1 -- 172.21.15.27:0/2035761115 shutdown_connections 2021-11-03T18:25:46.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.557+0000 7f964d467700 1 -- 172.21.15.27:0/2035761115 wait complete. 2021-11-03T18:25:46.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.558+0000 7f964d467700 1 Processor -- start 2021-11-03T18:25:46.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.558+0000 7f964d467700 1 -- start start 2021-11-03T18:25:46.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.558+0000 7f964d467700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96480f5440 0x7f9648109ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:25:46.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.558+0000 7f964d467700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96480f66e0 0x7f964810bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:25:46.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.558+0000 7f964d467700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96480603c0 con 0x7f96480f66e0 2021-11-03T18:25:46.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.558+0000 7f964d467700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9648060500 con 0x7f96480f5440 2021-11-03T18:25:46.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.558+0000 7f96477fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96480f66e0 0x7f964810bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:25:46.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.558+0000 7f96477fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96480f66e0 0x7f964810bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34828/0 (socket says 172.21.15.27:34828) 2021-11-03T18:25:46.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.558+0000 7f96477fe700 1 -- 172.21.15.27:0/1909334574 learned_addr learned my addr 172.21.15.27:0/1909334574 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:25:46.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.558+0000 7f96477fe700 1 -- 172.21.15.27:0/1909334574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96480f5440 msgr2=0x7f9648109ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:25:46.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.558+0000 7f96477fe700 1 --2- 172.21.15.27:0/1909334574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96480f5440 0x7f9648109ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:25:46.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.559+0000 7f96477fe700 1 -- 172.21.15.27:0/1909334574 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f963c005040 con 0x7f96480f66e0 2021-11-03T18:25:46.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.559+0000 7f96477fe700 1 --2- 172.21.15.27:0/1909334574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96480f66e0 0x7f964810bfb0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 rx=0x7f963c009990 tx=0x7f963c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:25:46.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.559+0000 7f9644ff9700 1 -- 172.21.15.27:0/1909334574 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f963c018070 con 0x7f96480f66e0 2021-11-03T18:25:46.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.559+0000 7f964d467700 1 -- 172.21.15.27:0/1909334574 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f964810c500 con 0x7f96480f66e0 2021-11-03T18:25:46.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.559+0000 7f9644ff9700 1 -- 172.21.15.27:0/1909334574 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f963c007ca0 con 0x7f96480f66e0 2021-11-03T18:25:46.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.559+0000 7f9644ff9700 1 -- 172.21.15.27:0/1909334574 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f963c01c620 con 0x7f96480f66e0 2021-11-03T18:25:46.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.559+0000 7f964d467700 1 -- 172.21.15.27:0/1909334574 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f964810ca30 con 0x7f96480f66e0 2021-11-03T18:25:46.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.560+0000 7f964d467700 1 -- 172.21.15.27:0/1909334574 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9648059100 con 0x7f96480f66e0 2021-11-03T18:25:46.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.560+0000 7f9644ff9700 1 -- 172.21.15.27:0/1909334574 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f963c01d960 con 0x7f96480f66e0 2021-11-03T18:25:46.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.561+0000 7f9644ff9700 1 --2- 172.21.15.27:0/1909334574 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9630064100 0x7f96300665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:25:46.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.561+0000 7f9644ff9700 1 -- 172.21.15.27:0/1909334574 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f963c082520 con 0x7f96480f66e0 2021-11-03T18:25:46.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.561+0000 7f9647fff700 1 --2- 172.21.15.27:0/1909334574 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9630064100 0x7f96300665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:25:46.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.561+0000 7f9647fff700 1 --2- 172.21.15.27:0/1909334574 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9630064100 0x7f96300665c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f9648060040 tx=0x7f9638007a60).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:25:46.568 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.565+0000 7f9644ff9700 1 -- 172.21.15.27:0/1909334574 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f963c054940 con 0x7f96480f66e0 2021-11-03T18:25:46.767 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.768+0000 7f964d467700 1 -- 172.21.15.27:0/1909334574 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f96480fdd40 con 0x7f96480f66e0 2021-11-03T18:25:46.767 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.769+0000 7f9644ff9700 1 -- 172.21.15.27:0/1909334574 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f963c054530 con 0x7f96480f66e0 2021-11-03T18:25:46.767 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:25:46.768 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:25:46.768 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:25:46.768 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:25:46.768 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:25:46.769 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:25:46.769 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:25:46.769 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:25:46.769 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:25:46.769 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:25:46.770 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:25:46.770 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:25:46.770 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:25:46.770 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:25:46.771 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:25:46.771 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:25:46.771 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:25:46.771 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:25:46.772 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:25:46.772 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:25:46.772 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:25:46.772 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:25:46.772 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:25:46.773 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:25:46.773 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:25:46.773 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:25:46.773 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:25:46.773 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:25:46.774 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:25:46.774 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:25:46.774 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:25:46.775 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:25:46.775 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:25:46.776 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:25:46.776 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:25:46.776 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:25:46.777 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:25:46.777 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:25:46.777 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:25:46.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.770+0000 7f964d467700 1 -- 172.21.15.27:0/1909334574 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9630064100 msgr2=0x7f96300665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:25:46.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.770+0000 7f964d467700 1 --2- 172.21.15.27:0/1909334574 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9630064100 0x7f96300665c0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7f9648060040 tx=0x7f9638007a60).stop 2021-11-03T18:25:46.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.770+0000 7f964d467700 1 -- 172.21.15.27:0/1909334574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96480f66e0 msgr2=0x7f964810bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:25:46.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.770+0000 7f964d467700 1 --2- 172.21.15.27:0/1909334574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96480f66e0 0x7f964810bfb0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 rx=0x7f963c009990 tx=0x7f963c009b30).stop 2021-11-03T18:25:46.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.771+0000 7f964d467700 1 -- 172.21.15.27:0/1909334574 shutdown_connections 2021-11-03T18:25:46.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.771+0000 7f964d467700 1 --2- 172.21.15.27:0/1909334574 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9630064100 0x7f96300665c0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:46.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.771+0000 7f964d467700 1 --2- 172.21.15.27:0/1909334574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96480f5440 0x7f9648109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:25:46.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.771+0000 7f964d467700 1 --2- 172.21.15.27:0/1909334574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96480f66e0 0x7f964810bfb0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:25:46.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.771+0000 7f964d467700 1 -- 172.21.15.27:0/1909334574 >> 172.21.15.27:0/1909334574 conn(0x7f96480f0ba0 msgr2=0x7f96480fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:25:46.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.771+0000 7f964d467700 1 -- 172.21.15.27:0/1909334574 shutdown_connections 2021-11-03T18:25:46.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:25:46.771+0000 7f964d467700 1 -- 172.21.15.27:0/1909334574 wait complete. 2021-11-03T18:25:46.783 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:25:46.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:46 smithi115 ceph-mon[65453]: pgmap v2135: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:46.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:46 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1703080464' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:47.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:46 smithi027 ceph-mon[97244]: pgmap v2135: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:47.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:46 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1703080464' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:47.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:47 smithi115 ceph-mon[65453]: from='client.36296 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:25:47.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:47 smithi115 ceph-mon[65453]: from='client.36300 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:25:47.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:47 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1909334574' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:25:48.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:47 smithi027 ceph-mon[97244]: from='client.36296 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:25:48.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:47 smithi027 ceph-mon[97244]: from='client.36300 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:25:48.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:47 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1909334574' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:25:48.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:48 smithi115 ceph-mon[65453]: pgmap v2136: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:49.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:48 smithi027 ceph-mon[97244]: pgmap v2136: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:50.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:50 smithi115 ceph-mon[65453]: pgmap v2137: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:50.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4192.460303 secs 2021-11-03T18:25:51.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:50 smithi027 ceph-mon[97244]: pgmap v2137: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:51.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4192.460303 secs 2021-11-03T18:25:52.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:52 smithi027 ceph-mon[97244]: pgmap v2138: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:52.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:52 smithi115 ceph-mon[65453]: pgmap v2138: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:54.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:54 smithi115 ceph-mon[65453]: pgmap v2139: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:54.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:25:54.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:25:54.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:25:54.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:54.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:54.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:54.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:54 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:54.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:54 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:25:55.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:54 smithi027 ceph-mon[97244]: pgmap v2139: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:55.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:25:55.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:25:55.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:25:55.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:55.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:55.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:55.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:54 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:25:55.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:54 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:25:55.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4197.460426 secs 2021-11-03T18:25:56.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4197.460426 secs 2021-11-03T18:25:56.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:56 smithi115 ceph-mon[65453]: pgmap v2140: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:56.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:25:56.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:25:57.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:56 smithi027 ceph-mon[97244]: pgmap v2140: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:57.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:25:57.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:25:58.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:25:58 smithi115 ceph-mon[65453]: pgmap v2141: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:25:59.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:25:58 smithi027 ceph-mon[97244]: pgmap v2141: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:00.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:00 smithi115 ceph-mon[65453]: pgmap v2142: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:00.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4202.460566 secs 2021-11-03T18:26:01.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:00 smithi027 ceph-mon[97244]: pgmap v2142: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:01.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4202.460566 secs 2021-11-03T18:26:02.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:02 smithi027 ceph-mon[97244]: pgmap v2143: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:02.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:02 smithi115 ceph-mon[65453]: pgmap v2143: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:04.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:04 smithi115 ceph-mon[65453]: pgmap v2144: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:04.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:26:04.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:26:04.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:26:04.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:04.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:04.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:04.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:04.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:04 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:26:05.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:04 smithi027 ceph-mon[97244]: pgmap v2144: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:05.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:26:05.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:26:05.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:26:05.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:05.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:05.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:05.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:05.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:04 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:26:05.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4207.460647 secs 2021-11-03T18:26:06.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4207.460647 secs 2021-11-03T18:26:06.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:06 smithi115 ceph-mon[65453]: pgmap v2145: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:07.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:06 smithi027 ceph-mon[97244]: pgmap v2145: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:08.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:08 smithi115 ceph-mon[65453]: pgmap v2146: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:09.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:08 smithi027 ceph-mon[97244]: pgmap v2146: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:10.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:10 smithi115 ceph-mon[65453]: pgmap v2147: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:10.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4212.460734 secs 2021-11-03T18:26:11.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:10 smithi027 ceph-mon[97244]: pgmap v2147: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:11.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4212.460734 secs 2021-11-03T18:26:12.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:12 smithi027 ceph-mon[97244]: pgmap v2148: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:12.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:12 smithi115 ceph-mon[65453]: pgmap v2148: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:14.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:14 smithi115 ceph-mon[65453]: pgmap v2149: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:14.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:26:14.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:26:14.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:26:14.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:14.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:14.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:14.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:14.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:14 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:26:15.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:14 smithi027 ceph-mon[97244]: pgmap v2149: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:15.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:26:15.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:26:15.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:26:15.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:15.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:15.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:15.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:15.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:14 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:26:15.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4217.460878 secs 2021-11-03T18:26:16.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4217.460878 secs 2021-11-03T18:26:16.883 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:16.885+0000 7f2489703700 1 -- 172.21.15.27:0/729693301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24840f55c0 msgr2=0x7f24840f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:26:16.883 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:16.885+0000 7f2489703700 1 --2- 172.21.15.27:0/729693301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24840f55c0 0x7f24840f59e0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 rx=0x7f247c0099e0 tx=0x7f247c004d10).stop 2021-11-03T18:26:16.884 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:16.885+0000 7f2489703700 1 -- 172.21.15.27:0/729693301 shutdown_connections 2021-11-03T18:26:16.884 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:16.885+0000 7f2489703700 1 --2- 172.21.15.27:0/729693301 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24840f6860 0x7f24840f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T18:26:16.884 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:16.885+0000 7f2489703700 1 --2- 172.21.15.27:0/729693301 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24840f55c0 0x7f24840f59e0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:16.885 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:16.885+0000 7f2489703700 1 -- 172.21.15.27:0/729693301 >> 172.21.15.27:0/729693301 conn(0x7f24840f0d40 msgr2=0x7f24840f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:26:16.885 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:16.885+0000 7f2489703700 1 -- 172.21.15.27:0/729693301 shutdown_connections 2021-11-03T18:26:16.885 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:16.885+0000 7f2489703700 1 -- 172.21.15.27:0/729693301 wait complete. 2021-11-03T18:26:16.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:16.886+0000 7f2489703700 1 Processor -- start 2021-11-03T18:26:16.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:16.886+0000 7f2489703700 1 -- start start 2021-11-03T18:26:16.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:16.886+0000 7f2489703700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24840f55c0 0x7f2484109e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:26:16.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:16.886+0000 7f2489703700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24840f6860 0x7f248410c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:26:16.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:16.886+0000 7f2489703700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2484060470 con 0x7f24840f6860 2021-11-03T18:26:16.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:16.886+0000 7f2489703700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24840605b0 con 0x7f24840f55c0 2021-11-03T18:26:16.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:16.886+0000 7f24827fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24840f6860 0x7f248410c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:26:16.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:16.886+0000 7f24827fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24840f6860 0x7f248410c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34846/0 (socket says 172.21.15.27:34846) 2021-11-03T18:26:16.891 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:16.886+0000 7f24827fc700 1 -- 172.21.15.27:0/2683515200 learned_addr learned my addr 172.21.15.27:0/2683515200 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:26:16.891 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:16.886+0000 7f2482ffd700 1 --2- 172.21.15.27:0/2683515200 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24840f55c0 0x7f2484109e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:26:16.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:16.887+0000 7f24827fc700 1 -- 172.21.15.27:0/2683515200 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24840f55c0 msgr2=0x7f2484109e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:26:16.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:16.887+0000 7f24827fc700 1 --2- 172.21.15.27:0/2683515200 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24840f55c0 0x7f2484109e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:16.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:16.887+0000 7f24827fc700 1 -- 172.21.15.27:0/2683515200 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f247c009640 con 0x7f24840f6860 2021-11-03T18:26:16.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:16.887+0000 7f24827fc700 1 --2- 172.21.15.27:0/2683515200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24840f6860 0x7f248410c330 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 rx=0x7f247800b410 tx=0x7f2478006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:26:16.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:16.887+0000 7f246b7fe700 1 -- 172.21.15.27:0/2683515200 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f24780125f0 con 0x7f24840f6860 2021-11-03T18:26:16.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:16.887+0000 7f246b7fe700 1 -- 172.21.15.27:0/2683515200 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f247800be30 con 0x7f24840f6860 2021-11-03T18:26:16.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:16.887+0000 7f2489703700 1 -- 172.21.15.27:0/2683515200 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f248410c990 con 0x7f24840f6860 2021-11-03T18:26:16.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:16.887+0000 7f246b7fe700 1 -- 172.21.15.27:0/2683515200 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2478002a70 con 0x7f24840f6860 2021-11-03T18:26:16.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:16.887+0000 7f2489703700 1 -- 172.21.15.27:0/2683515200 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f248410cea0 con 0x7f24840f6860 2021-11-03T18:26:16.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:16.888+0000 7f2489703700 1 -- 172.21.15.27:0/2683515200 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24840f49b0 con 0x7f24840f6860 2021-11-03T18:26:16.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:16.888+0000 7f246b7fe700 1 -- 172.21.15.27:0/2683515200 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f247800f030 con 0x7f24840f6860 2021-11-03T18:26:16.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:16.889+0000 7f246b7fe700 1 --2- 172.21.15.27:0/2683515200 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f246c064090 0x7f246c066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:26:16.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:16.889+0000 7f246b7fe700 1 -- 172.21.15.27:0/2683515200 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f2478080b10 con 0x7f24840f6860 2021-11-03T18:26:16.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:16.889+0000 7f2482ffd700 1 --2- 172.21.15.27:0/2683515200 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f246c064090 0x7f246c066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:26:16.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:16.889+0000 7f2482ffd700 1 --2- 172.21.15.27:0/2683515200 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f246c064090 0x7f246c066550 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7f247c00b730 tx=0x7f247c004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:26:16.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:16.893+0000 7f246b7fe700 1 -- 172.21.15.27:0/2683515200 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f2478052e70 con 0x7f24840f6860 2021-11-03T18:26:16.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:16 smithi115 ceph-mon[65453]: pgmap v2150: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:17.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:16 smithi027 ceph-mon[97244]: pgmap v2150: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:17.069 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.072+0000 7f2489703700 1 -- 172.21.15.27:0/2683515200 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f24840fdf40 con 0x7f246c064090 2021-11-03T18:26:17.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.073+0000 7f246b7fe700 1 -- 172.21.15.27:0/2683515200 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f24840fdf40 con 0x7f246c064090 2021-11-03T18:26:17.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.075+0000 7f2489703700 1 -- 172.21.15.27:0/2683515200 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f246c064090 msgr2=0x7f246c066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:26:17.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.075+0000 7f2489703700 1 --2- 172.21.15.27:0/2683515200 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f246c064090 0x7f246c066550 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 rx=0x7f247c00b730 tx=0x7f247c004d10).stop 2021-11-03T18:26:17.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.075+0000 7f2489703700 1 -- 172.21.15.27:0/2683515200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24840f6860 msgr2=0x7f248410c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:26:17.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.075+0000 7f2489703700 1 --2- 172.21.15.27:0/2683515200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24840f6860 0x7f248410c330 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 rx=0x7f247800b410 tx=0x7f2478006ef0).stop 2021-11-03T18:26:17.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.075+0000 7f2489703700 1 -- 172.21.15.27:0/2683515200 shutdown_connections 2021-11-03T18:26:17.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.075+0000 7f2489703700 1 --2- 172.21.15.27:0/2683515200 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f246c064090 0x7f246c066550 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:17.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.075+0000 7f2489703700 1 --2- 172.21.15.27:0/2683515200 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f24840f55c0 0x7f2484109e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:17.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.075+0000 7f2489703700 1 --2- 172.21.15.27:0/2683515200 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f24840f6860 0x7f248410c330 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:17.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.075+0000 7f2489703700 1 -- 172.21.15.27:0/2683515200 >> 172.21.15.27:0/2683515200 conn(0x7f24840f0d40 msgr2=0x7f24840fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:26:17.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.075+0000 7f2489703700 1 -- 172.21.15.27:0/2683515200 shutdown_connections 2021-11-03T18:26:17.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.075+0000 7f2489703700 1 -- 172.21.15.27:0/2683515200 wait complete. 2021-11-03T18:26:17.082 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:26:17.173 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.175+0000 7f59a3e1c700 1 -- 172.21.15.27:0/3168344082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f599c0f5ef0 msgr2=0x7f599c0f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:26:17.173 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.175+0000 7f59a3e1c700 1 --2- 172.21.15.27:0/3168344082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f599c0f5ef0 0x7f599c0f6920 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 rx=0x7f5998004660 tx=0x7f5998009b30).stop 2021-11-03T18:26:17.174 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.175+0000 7f59a3e1c700 1 -- 172.21.15.27:0/3168344082 shutdown_connections 2021-11-03T18:26:17.174 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.175+0000 7f59a3e1c700 1 --2- 172.21.15.27:0/3168344082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f599c0f5ef0 0x7f599c0f6920 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:17.174 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.175+0000 7f59a3e1c700 1 --2- 172.21.15.27:0/3168344082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f599c0f55e0 0x7f599c0f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:17.174 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.175+0000 7f59a3e1c700 1 -- 172.21.15.27:0/3168344082 >> 172.21.15.27:0/3168344082 conn(0x7f599c0f0de0 msgr2=0x7f599c0f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:26:17.175 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.175+0000 7f59a3e1c700 1 -- 172.21.15.27:0/3168344082 shutdown_connections 2021-11-03T18:26:17.175 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.175+0000 7f59a3e1c700 1 -- 172.21.15.27:0/3168344082 wait complete. 2021-11-03T18:26:17.177 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.175+0000 7f59a3e1c700 1 Processor -- start 2021-11-03T18:26:17.177 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.176+0000 7f59a3e1c700 1 -- start start 2021-11-03T18:26:17.178 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.176+0000 7f59a3e1c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f599c0f55e0 0x7f599c0fa000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:26:17.178 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.176+0000 7f59a3e1c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f599c0f5ef0 0x7f599c0f8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:26:17.178 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.176+0000 7f59a3e1c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599c0f89e0 con 0x7f599c0f55e0 2021-11-03T18:26:17.178 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.176+0000 7f59a3e1c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599c0f8b20 con 0x7f599c0f5ef0 2021-11-03T18:26:17.179 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.176+0000 7f59a1bb8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f599c0f55e0 0x7f599c0fa000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:26:17.179 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.176+0000 7f59a1bb8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f599c0f55e0 0x7f599c0fa000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34854/0 (socket says 172.21.15.27:34854) 2021-11-03T18:26:17.179 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.176+0000 7f59a1bb8700 1 -- 172.21.15.27:0/747933245 learned_addr learned my addr 172.21.15.27:0/747933245 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:26:17.179 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.176+0000 7f59a13b7700 1 --2- 172.21.15.27:0/747933245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f599c0f5ef0 0x7f599c0f8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:26:17.180 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.176+0000 7f59a1bb8700 1 -- 172.21.15.27:0/747933245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f599c0f5ef0 msgr2=0x7f599c0f8480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:26:17.180 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.176+0000 7f59a1bb8700 1 --2- 172.21.15.27:0/747933245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f599c0f5ef0 0x7f599c0f8480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:17.180 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.176+0000 7f59a1bb8700 1 -- 172.21.15.27:0/747933245 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5998005040 con 0x7f599c0f55e0 2021-11-03T18:26:17.180 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.176+0000 7f59a1bb8700 1 --2- 172.21.15.27:0/747933245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f599c0f55e0 0x7f599c0fa000 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 rx=0x7f598c00c530 tx=0x7f598c007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:26:17.181 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.176+0000 7f59927fc700 1 -- 172.21.15.27:0/747933245 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f598c00f650 con 0x7f599c0f55e0 2021-11-03T18:26:17.181 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.176+0000 7f59a3e1c700 1 -- 172.21.15.27:0/747933245 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f599c0f8e00 con 0x7f599c0f55e0 2021-11-03T18:26:17.181 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.176+0000 7f59927fc700 1 -- 172.21.15.27:0/747933245 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f598c00fc60 con 0x7f599c0f55e0 2021-11-03T18:26:17.181 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.176+0000 7f59927fc700 1 -- 172.21.15.27:0/747933245 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f598c012c20 con 0x7f599c0f55e0 2021-11-03T18:26:17.181 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.177+0000 7f59a3e1c700 1 -- 172.21.15.27:0/747933245 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f599c10efa0 con 0x7f599c0f55e0 2021-11-03T18:26:17.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.177+0000 7f59a3e1c700 1 -- 172.21.15.27:0/747933245 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f599c045ad0 con 0x7f599c0f55e0 2021-11-03T18:26:17.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.178+0000 7f59927fc700 1 -- 172.21.15.27:0/747933245 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f598c010070 con 0x7f599c0f55e0 2021-11-03T18:26:17.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.178+0000 7f59927fc700 1 --2- 172.21.15.27:0/747933245 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f59880640b0 0x7f5988066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:26:17.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.178+0000 7f59927fc700 1 -- 172.21.15.27:0/747933245 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f598c080660 con 0x7f599c0f55e0 2021-11-03T18:26:17.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.178+0000 7f59a13b7700 1 --2- 172.21.15.27:0/747933245 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f59880640b0 0x7f5988066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:26:17.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.179+0000 7f59a13b7700 1 --2- 172.21.15.27:0/747933245 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f59880640b0 0x7f5988066570 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7f5998004630 tx=0x7f5998009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:26:17.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.182+0000 7f59927fc700 1 -- 172.21.15.27:0/747933245 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f598c0528d0 con 0x7f599c0f55e0 2021-11-03T18:26:17.352 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.354+0000 7f59a3e1c700 1 -- 172.21.15.27:0/747933245 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f599c05fe50 con 0x7f59880640b0 2021-11-03T18:26:17.367 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.369+0000 7f59927fc700 1 -- 172.21.15.27:0/747933245 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f599c05fe50 con 0x7f59880640b0 2021-11-03T18:26:17.367 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:26:17.367 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (77m) 6m ago 79m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:26:17.367 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (69m) 6m ago 79m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:26:17.368 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (69m) 6m ago 78m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:26:17.368 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (77m) 6m ago 78m 42.4M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:26:17.368 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (71m) 6m ago 74m 26.5M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:26:17.368 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (74m) 6m ago 74m 3849M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:26:17.369 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (70m) 6m ago 74m 26.1M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:26:17.369 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (71m) 6m ago 74m 27.2M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:26:17.369 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (71m) 6m ago 80m 590M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:26:17.369 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (70m) 6m ago 78m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:26:17.369 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (70m) 6m ago 80m 312M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:26:17.370 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (70m) 6m ago 78m 145M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:26:17.370 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (78m) 6m ago 78m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:26:17.370 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (78m) 6m ago 78m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:26:17.370 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (69m) 6m ago 77m 392M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:26:17.371 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (68m) 6m ago 77m 427M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:26:17.371 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (68m) 6m ago 76m 320M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:26:17.371 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (67m) 6m ago 76m 454M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:26:17.371 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (67m) 6m ago 76m 359M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:26:17.371 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (66m) 6m ago 75m 368M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:26:17.372 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (77m) 6m ago 78m 59.9M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:26:17.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.371+0000 7f59a3e1c700 1 -- 172.21.15.27:0/747933245 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f59880640b0 msgr2=0x7f5988066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:26:17.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.371+0000 7f59a3e1c700 1 --2- 172.21.15.27:0/747933245 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f59880640b0 0x7f5988066570 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7f5998004630 tx=0x7f5998009b30).stop 2021-11-03T18:26:17.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.371+0000 7f59a3e1c700 1 -- 172.21.15.27:0/747933245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f599c0f55e0 msgr2=0x7f599c0fa000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:26:17.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.371+0000 7f59a3e1c700 1 --2- 172.21.15.27:0/747933245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f599c0f55e0 0x7f599c0fa000 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 rx=0x7f598c00c530 tx=0x7f598c007e20).stop 2021-11-03T18:26:17.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.371+0000 7f59a3e1c700 1 -- 172.21.15.27:0/747933245 shutdown_connections 2021-11-03T18:26:17.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.371+0000 7f59a3e1c700 1 --2- 172.21.15.27:0/747933245 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f59880640b0 0x7f5988066570 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:17.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.371+0000 7f59a3e1c700 1 --2- 172.21.15.27:0/747933245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f599c0f55e0 0x7f599c0fa000 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:17.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.371+0000 7f59a3e1c700 1 --2- 172.21.15.27:0/747933245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f599c0f5ef0 0x7f599c0f8480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:17.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.371+0000 7f59a3e1c700 1 -- 172.21.15.27:0/747933245 >> 172.21.15.27:0/747933245 conn(0x7f599c0f0de0 msgr2=0x7f599c05f740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:26:17.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.371+0000 7f59a3e1c700 1 -- 172.21.15.27:0/747933245 shutdown_connections 2021-11-03T18:26:17.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.371+0000 7f59a3e1c700 1 -- 172.21.15.27:0/747933245 wait complete. 2021-11-03T18:26:17.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.469+0000 7fc9ad97f700 1 -- 172.21.15.27:0/1015654519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9a80f77f0 msgr2=0x7fc9a80f7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:26:17.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.469+0000 7fc9ad97f700 1 --2- 172.21.15.27:0/1015654519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9a80f77f0 0x7fc9a80f7c50 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 rx=0x7fc990004660 tx=0x7fc990009b30).stop 2021-11-03T18:26:17.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.470+0000 7fc9ad97f700 1 -- 172.21.15.27:0/1015654519 shutdown_connections 2021-11-03T18:26:17.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.470+0000 7fc9ad97f700 1 --2- 172.21.15.27:0/1015654519 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9a80f77f0 0x7fc9a80f7c50 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:17.469 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.470+0000 7fc9ad97f700 1 --2- 172.21.15.27:0/1015654519 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9a80f6550 0x7fc9a80f6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:17.469 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.470+0000 7fc9ad97f700 1 -- 172.21.15.27:0/1015654519 >> 172.21.15.27:0/1015654519 conn(0x7fc9a80f1cd0 msgr2=0x7fc9a80f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:26:17.469 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.470+0000 7fc9ad97f700 1 -- 172.21.15.27:0/1015654519 shutdown_connections 2021-11-03T18:26:17.469 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.470+0000 7fc9ad97f700 1 -- 172.21.15.27:0/1015654519 wait complete. 2021-11-03T18:26:17.472 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.470+0000 7fc9ad97f700 1 Processor -- start 2021-11-03T18:26:17.472 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.470+0000 7fc9ad97f700 1 -- start start 2021-11-03T18:26:17.473 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.470+0000 7fc9ad97f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9a80f6550 0x7fc9a8106db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:26:17.473 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.470+0000 7fc9ad97f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9a80f77f0 0x7fc9a81072a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:26:17.473 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.470+0000 7fc9ad97f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a81078a0 con 0x7fc9a80f77f0 2021-11-03T18:26:17.473 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.470+0000 7fc9ad97f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9a81080e0 con 0x7fc9a80f6550 2021-11-03T18:26:17.473 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.471+0000 7fc9a67fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9a80f77f0 0x7fc9a81072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:26:17.474 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.471+0000 7fc9a67fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9a80f77f0 0x7fc9a81072a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34866/0 (socket says 172.21.15.27:34866) 2021-11-03T18:26:17.474 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.471+0000 7fc9a67fc700 1 -- 172.21.15.27:0/1705261393 learned_addr learned my addr 172.21.15.27:0/1705261393 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:26:17.474 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.471+0000 7fc9a6ffd700 1 --2- 172.21.15.27:0/1705261393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9a80f6550 0x7fc9a8106db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:26:17.474 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.471+0000 7fc9a67fc700 1 -- 172.21.15.27:0/1705261393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9a80f6550 msgr2=0x7fc9a8106db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:26:17.475 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.471+0000 7fc9a67fc700 1 --2- 172.21.15.27:0/1705261393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9a80f6550 0x7fc9a8106db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:17.475 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.471+0000 7fc9a67fc700 1 -- 172.21.15.27:0/1705261393 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc990005040 con 0x7fc9a80f77f0 2021-11-03T18:26:17.475 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.471+0000 7fc9a67fc700 1 --2- 172.21.15.27:0/1705261393 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9a80f77f0 0x7fc9a81072a0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 rx=0x7fc990009990 tx=0x7fc990009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:26:17.476 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.471+0000 7fc99f7fe700 1 -- 172.21.15.27:0/1705261393 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc990018070 con 0x7fc9a80f77f0 2021-11-03T18:26:17.476 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.471+0000 7fc9ad97f700 1 -- 172.21.15.27:0/1705261393 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9a8108360 con 0x7fc9a80f77f0 2021-11-03T18:26:17.476 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.471+0000 7fc99f7fe700 1 -- 172.21.15.27:0/1705261393 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fc990007ba0 con 0x7fc9a80f77f0 2021-11-03T18:26:17.476 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.471+0000 7fc99f7fe700 1 -- 172.21.15.27:0/1705261393 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc99001c3c0 con 0x7fc9a80f77f0 2021-11-03T18:26:17.477 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.471+0000 7fc9ad97f700 1 -- 172.21.15.27:0/1705261393 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9a8108840 con 0x7fc9a80f77f0 2021-11-03T18:26:17.477 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.472+0000 7fc9ad97f700 1 -- 172.21.15.27:0/1705261393 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9a8059100 con 0x7fc9a80f77f0 2021-11-03T18:26:17.477 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.473+0000 7fc99f7fe700 1 -- 172.21.15.27:0/1705261393 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fc99001d420 con 0x7fc9a80f77f0 2021-11-03T18:26:17.477 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.473+0000 7fc99f7fe700 1 --2- 172.21.15.27:0/1705261393 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc994064100 0x7fc9940665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:26:17.477 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.473+0000 7fc99f7fe700 1 -- 172.21.15.27:0/1705261393 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fc990082380 con 0x7fc9a80f77f0 2021-11-03T18:26:17.478 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.473+0000 7fc9a6ffd700 1 --2- 172.21.15.27:0/1705261393 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc994064100 0x7fc9940665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:26:17.478 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.473+0000 7fc9a6ffd700 1 --2- 172.21.15.27:0/1705261393 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc994064100 0x7fc9940665c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7fc9a8107b70 tx=0x7fc9a000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:26:17.478 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.477+0000 7fc99f7fe700 1 -- 172.21.15.27:0/1705261393 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fc9900547a0 con 0x7fc9a80f77f0 2021-11-03T18:26:17.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.707+0000 7fc9ad97f700 1 -- 172.21.15.27:0/1705261393 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc9a80fedb0 con 0x7fc9a80f77f0 2021-11-03T18:26:17.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.707+0000 7fc99f7fe700 1 -- 172.21.15.27:0/1705261393 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fc990054390 con 0x7fc9a80f77f0 2021-11-03T18:26:17.706 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:26:17.706 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:26:17.706 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:26:17.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:26:17.707 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:26:17.707 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:26:17.707 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:26:17.708 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:26:17.708 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:26:17.708 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:26:17.708 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:26:17.709 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:26:17.709 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:26:17.709 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:26:17.709 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:26:17.710 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:26:17.710 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:26:17.710 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:26:17.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.709+0000 7fc9ad97f700 1 -- 172.21.15.27:0/1705261393 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc994064100 msgr2=0x7fc9940665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:26:17.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.709+0000 7fc9ad97f700 1 --2- 172.21.15.27:0/1705261393 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc994064100 0x7fc9940665c0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7fc9a8107b70 tx=0x7fc9a000b040).stop 2021-11-03T18:26:17.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.709+0000 7fc9ad97f700 1 -- 172.21.15.27:0/1705261393 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9a80f77f0 msgr2=0x7fc9a81072a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:26:17.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.709+0000 7fc9ad97f700 1 --2- 172.21.15.27:0/1705261393 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9a80f77f0 0x7fc9a81072a0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 rx=0x7fc990009990 tx=0x7fc990009b30).stop 2021-11-03T18:26:17.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.709+0000 7fc9ad97f700 1 -- 172.21.15.27:0/1705261393 shutdown_connections 2021-11-03T18:26:17.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.709+0000 7fc9ad97f700 1 --2- 172.21.15.27:0/1705261393 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc994064100 0x7fc9940665c0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:17.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.709+0000 7fc9ad97f700 1 --2- 172.21.15.27:0/1705261393 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc9a80f6550 0x7fc9a8106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:17.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.709+0000 7fc9ad97f700 1 --2- 172.21.15.27:0/1705261393 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc9a80f77f0 0x7fc9a81072a0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:17.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.709+0000 7fc9ad97f700 1 -- 172.21.15.27:0/1705261393 >> 172.21.15.27:0/1705261393 conn(0x7fc9a80f1cd0 msgr2=0x7fc9a80fb240 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:26:17.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.709+0000 7fc9ad97f700 1 -- 172.21.15.27:0/1705261393 shutdown_connections 2021-11-03T18:26:17.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.710+0000 7fc9ad97f700 1 -- 172.21.15.27:0/1705261393 wait complete. 2021-11-03T18:26:17.807 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.809+0000 7ff971910700 1 -- 172.21.15.27:0/3643106574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff96c0f8610 msgr2=0x7ff96c0f4820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:26:17.808 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.809+0000 7ff971910700 1 --2- 172.21.15.27:0/3643106574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff96c0f8610 0x7ff96c0f4820 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 rx=0x7ff954004660 tx=0x7ff954009b30).stop 2021-11-03T18:26:17.808 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.810+0000 7ff971910700 1 -- 172.21.15.27:0/3643106574 shutdown_connections 2021-11-03T18:26:17.808 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.810+0000 7ff971910700 1 --2- 172.21.15.27:0/3643106574 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff96c0f8610 0x7ff96c0f4820 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:17.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.810+0000 7ff971910700 1 --2- 172.21.15.27:0/3643106574 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff96c0f7d00 0x7ff96c0f8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:17.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.810+0000 7ff971910700 1 -- 172.21.15.27:0/3643106574 >> 172.21.15.27:0/3643106574 conn(0x7ff96c0f0540 msgr2=0x7ff96c0f2960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:26:17.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.810+0000 7ff971910700 1 -- 172.21.15.27:0/3643106574 shutdown_connections 2021-11-03T18:26:17.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.810+0000 7ff971910700 1 -- 172.21.15.27:0/3643106574 wait complete. 2021-11-03T18:26:17.812 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.810+0000 7ff971910700 1 Processor -- start 2021-11-03T18:26:17.812 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.810+0000 7ff971910700 1 -- start start 2021-11-03T18:26:17.812 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.810+0000 7ff971910700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff96c0f7d00 0x7ff96c106f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:26:17.812 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.811+0000 7ff971910700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff96c0f8610 0x7ff96c107400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:26:17.813 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.811+0000 7ff971910700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c1079b0 con 0x7ff96c0f8610 2021-11-03T18:26:17.813 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.811+0000 7ff971910700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c107af0 con 0x7ff96c0f7d00 2021-11-03T18:26:17.813 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.811+0000 7ff96a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff96c0f8610 0x7ff96c107400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:26:17.813 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.811+0000 7ff96a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff96c0f8610 0x7ff96c107400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34876/0 (socket says 172.21.15.27:34876) 2021-11-03T18:26:17.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.811+0000 7ff96a7fc700 1 -- 172.21.15.27:0/3325224897 learned_addr learned my addr 172.21.15.27:0/3325224897 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:26:17.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.811+0000 7ff96affd700 1 --2- 172.21.15.27:0/3325224897 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff96c0f7d00 0x7ff96c106f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:26:17.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.811+0000 7ff96a7fc700 1 -- 172.21.15.27:0/3325224897 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff96c0f7d00 msgr2=0x7ff96c106f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:26:17.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.811+0000 7ff96a7fc700 1 --2- 172.21.15.27:0/3325224897 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff96c0f7d00 0x7ff96c106f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:17.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.811+0000 7ff96a7fc700 1 -- 172.21.15.27:0/3325224897 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff954005040 con 0x7ff96c0f8610 2021-11-03T18:26:17.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.811+0000 7ff96a7fc700 1 --2- 172.21.15.27:0/3325224897 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff96c0f8610 0x7ff96c107400 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 rx=0x7ff954009990 tx=0x7ff954009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:26:17.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.811+0000 7ff9637fe700 1 -- 172.21.15.27:0/3325224897 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff954018070 con 0x7ff96c0f8610 2021-11-03T18:26:17.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.811+0000 7ff971910700 1 -- 172.21.15.27:0/3325224897 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff96c108430 con 0x7ff96c0f8610 2021-11-03T18:26:17.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.811+0000 7ff9637fe700 1 -- 172.21.15.27:0/3325224897 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7ff954007ba0 con 0x7ff96c0f8610 2021-11-03T18:26:17.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.811+0000 7ff9637fe700 1 -- 172.21.15.27:0/3325224897 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff95401c3c0 con 0x7ff96c0f8610 2021-11-03T18:26:17.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.811+0000 7ff971910700 1 -- 172.21.15.27:0/3325224897 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff96c108990 con 0x7ff96c0f8610 2021-11-03T18:26:17.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.812+0000 7ff971910700 1 -- 172.21.15.27:0/3325224897 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff96c045ad0 con 0x7ff96c0f8610 2021-11-03T18:26:17.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.813+0000 7ff9637fe700 1 -- 172.21.15.27:0/3325224897 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7ff95401d420 con 0x7ff96c0f8610 2021-11-03T18:26:17.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.813+0000 7ff9637fe700 1 --2- 172.21.15.27:0/3325224897 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff9580640b0 0x7ff958066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:26:17.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.813+0000 7ff9637fe700 1 -- 172.21.15.27:0/3325224897 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7ff954082410 con 0x7ff96c0f8610 2021-11-03T18:26:17.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.813+0000 7ff96affd700 1 --2- 172.21.15.27:0/3325224897 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff9580640b0 0x7ff958066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:26:17.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.813+0000 7ff96affd700 1 --2- 172.21.15.27:0/3325224897 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff9580640b0 0x7ff958066570 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7ff964007db0 tx=0x7ff964009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:26:17.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:17.817+0000 7ff9637fe700 1 -- 172.21.15.27:0/3325224897 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7ff954054710 con 0x7ff96c0f8610 2021-11-03T18:26:18.018 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:18.020+0000 7ff971910700 1 -- 172.21.15.27:0/3325224897 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff96c027a00 con 0x7ff96c0f8610 2021-11-03T18:26:18.019 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:18.020+0000 7ff9637fe700 1 -- 172.21.15.27:0/3325224897 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7ff954054300 con 0x7ff96c0f8610 2021-11-03T18:26:18.019 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:26:18.019 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:26:18.020 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:26:18.020 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:26:18.020 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:26:18.020 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:26:18.021 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:26:18.021 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:26:18.021 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:26:18.021 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:26:18.022 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:26:18.022 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:26:18.022 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:26:18.022 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:26:18.023 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:26:18.023 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:26:18.023 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:26:18.023 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:26:18.023 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:26:18.024 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:26:18.024 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:26:18.024 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:26:18.024 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:26:18.025 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:26:18.025 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:26:18.025 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:26:18.025 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:26:18.025 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:26:18.026 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:26:18.026 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:26:18.026 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:26:18.026 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:26:18.027 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:26:18.027 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:26:18.027 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:26:18.027 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:26:18.028 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:26:18.028 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:26:18.028 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:26:18.029 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:18.022+0000 7ff971910700 1 -- 172.21.15.27:0/3325224897 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff9580640b0 msgr2=0x7ff958066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:26:18.029 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:18.022+0000 7ff971910700 1 --2- 172.21.15.27:0/3325224897 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff9580640b0 0x7ff958066570 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 rx=0x7ff964007db0 tx=0x7ff964009040).stop 2021-11-03T18:26:18.030 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:18.022+0000 7ff971910700 1 -- 172.21.15.27:0/3325224897 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff96c0f8610 msgr2=0x7ff96c107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:26:18.030 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:18.022+0000 7ff971910700 1 --2- 172.21.15.27:0/3325224897 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff96c0f8610 0x7ff96c107400 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 rx=0x7ff954009990 tx=0x7ff954009b30).stop 2021-11-03T18:26:18.030 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:18.022+0000 7ff971910700 1 -- 172.21.15.27:0/3325224897 shutdown_connections 2021-11-03T18:26:18.030 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:18.023+0000 7ff971910700 1 --2- 172.21.15.27:0/3325224897 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff9580640b0 0x7ff958066570 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:18.030 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:18.023+0000 7ff971910700 1 --2- 172.21.15.27:0/3325224897 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff96c0f7d00 0x7ff96c106f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:18.031 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:18.023+0000 7ff971910700 1 --2- 172.21.15.27:0/3325224897 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff96c0f8610 0x7ff96c107400 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:18.031 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:18.023+0000 7ff971910700 1 -- 172.21.15.27:0/3325224897 >> 172.21.15.27:0/3325224897 conn(0x7ff96c0f0540 msgr2=0x7ff96c0fb590 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:26:18.031 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:18.023+0000 7ff971910700 1 -- 172.21.15.27:0/3325224897 shutdown_connections 2021-11-03T18:26:18.031 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:18.023+0000 7ff971910700 1 -- 172.21.15.27:0/3325224897 wait complete. 2021-11-03T18:26:18.032 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:26:18.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:18 smithi115 ceph-mon[65453]: pgmap v2151: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:18.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:18 smithi115 ceph-mon[65453]: from='client.36312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:26:18.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:18 smithi115 ceph-mon[65453]: from='client.36316 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:26:18.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:18 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1705261393' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:18.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:18 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3325224897' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:26:19.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:18 smithi027 ceph-mon[97244]: pgmap v2151: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:19.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:18 smithi027 ceph-mon[97244]: from='client.36312 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:26:19.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:18 smithi027 ceph-mon[97244]: from='client.36316 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:26:19.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:18 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1705261393' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:19.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:18 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3325224897' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:26:20.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:20 smithi115 ceph-mon[65453]: pgmap v2152: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:20.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4222.460980 secs 2021-11-03T18:26:21.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:20 smithi027 ceph-mon[97244]: pgmap v2152: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:21.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4222.460980 secs 2021-11-03T18:26:22.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:22 smithi027 ceph-mon[97244]: pgmap v2153: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:22.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:22 smithi115 ceph-mon[65453]: pgmap v2153: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:24.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:24 smithi115 ceph-mon[65453]: pgmap v2154: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:24.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:24 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:26:24.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:24 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:26:24.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:24 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:26:24.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:24 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:24.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:24 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:24.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:24 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:24.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:24 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:25.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:24 smithi027 ceph-mon[97244]: pgmap v2154: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:25.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:24 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:26:25.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:24 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:26:25.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:24 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:26:25.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:24 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:25.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:24 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:25.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:24 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:25.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:24 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:25.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:25 smithi115 ceph-mon[65453]: poll took 60175.028 ms: 1 events 2021-11-03T18:26:25.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:25 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:26:25.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4227.461116 secs 2021-11-03T18:26:26.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:25 smithi027 ceph-mon[97244]: poll took 60175.028 ms: 1 events 2021-11-03T18:26:26.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:25 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:26:26.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4227.461116 secs 2021-11-03T18:26:26.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:26 smithi115 ceph-mon[65453]: pgmap v2155: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:27.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:26 smithi027 ceph-mon[97244]: pgmap v2155: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:28.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:28 smithi115 ceph-mon[65453]: pgmap v2156: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:29.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:28 smithi027 ceph-mon[97244]: pgmap v2156: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:30.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:30 smithi115 ceph-mon[65453]: pgmap v2157: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:30.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4232.461253 secs 2021-11-03T18:26:31.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:30 smithi027 ceph-mon[97244]: pgmap v2157: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:31.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4232.461253 secs 2021-11-03T18:26:32.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:32 smithi115 ceph-mon[65453]: pgmap v2158: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:33.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:32 smithi027 ceph-mon[97244]: pgmap v2158: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:34.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:34 smithi115 ceph-mon[65453]: pgmap v2159: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:34.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:26:34.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"}]: dispatch 2021-11-03T18:26:34.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2021-11-03T18:26:34.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:26:34.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:26:34.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:34.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:34.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:34.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:34 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:35.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:34 smithi027 ceph-mon[97244]: pgmap v2159: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:35.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:26:35.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"}]: dispatch 2021-11-03T18:26:35.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2021-11-03T18:26:35.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:26:35.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:26:35.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:35.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:35.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:35.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:34 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:35.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:35 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:26:35.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4237.461423 secs 2021-11-03T18:26:36.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:35 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:26:36.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4237.461423 secs 2021-11-03T18:26:36.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:36 smithi115 ceph-mon[65453]: pgmap v2160: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:37.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:36 smithi027 ceph-mon[97244]: pgmap v2160: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:38.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:38 smithi115 ceph-mon[65453]: pgmap v2161: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:39.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:38 smithi027 ceph-mon[97244]: pgmap v2161: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:40.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:40 smithi115 ceph-mon[65453]: pgmap v2162: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:40.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4242.461538 secs 2021-11-03T18:26:41.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:40 smithi027 ceph-mon[97244]: pgmap v2162: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:41.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4242.461538 secs 2021-11-03T18:26:42.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:42 smithi115 ceph-mon[65453]: pgmap v2163: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:43.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:42 smithi027 ceph-mon[97244]: pgmap v2163: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:44.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:44 smithi115 ceph-mon[65453]: pgmap v2164: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:44.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:26:44.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:26:44.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:26:44.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:44.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:44.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:44.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:44 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:45.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:44 smithi027 ceph-mon[97244]: pgmap v2164: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:45.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:26:45.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:26:45.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:26:45.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:45.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:45.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:45.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:44 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:45.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:45 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:26:45.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4247.461676 secs 2021-11-03T18:26:46.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:45 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:26:46.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4247.461676 secs 2021-11-03T18:26:46.994 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:46 smithi115 ceph-mon[65453]: pgmap v2165: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:47.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:46 smithi027 ceph-mon[97244]: pgmap v2165: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:48.137 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.137+0000 7f1489cdb700 1 -- 172.21.15.27:0/2323610679 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14840f66e0 msgr2=0x7f14840f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:26:48.137 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.137+0000 7f1489cdb700 1 --2- 172.21.15.27:0/2323610679 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14840f66e0 0x7f14840f6b00 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 rx=0x7f1474004660 tx=0x7f1474009b30).stop 2021-11-03T18:26:48.137 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.137+0000 7f1489cdb700 1 -- 172.21.15.27:0/2323610679 shutdown_connections 2021-11-03T18:26:48.138 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.137+0000 7f1489cdb700 1 --2- 172.21.15.27:0/2323610679 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14840f6ff0 0x7f14840f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:48.138 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.137+0000 7f1489cdb700 1 --2- 172.21.15.27:0/2323610679 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14840f66e0 0x7f14840f6b00 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:48.138 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.137+0000 7f1489cdb700 1 -- 172.21.15.27:0/2323610679 >> 172.21.15.27:0/2323610679 conn(0x7f14840f1f00 msgr2=0x7f14840f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:26:48.138 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.137+0000 7f1489cdb700 1 -- 172.21.15.27:0/2323610679 shutdown_connections 2021-11-03T18:26:48.139 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.137+0000 7f1489cdb700 1 -- 172.21.15.27:0/2323610679 wait complete. 2021-11-03T18:26:48.139 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.138+0000 7f1489cdb700 1 Processor -- start 2021-11-03T18:26:48.139 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.138+0000 7f1489cdb700 1 -- start start 2021-11-03T18:26:48.139 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.138+0000 7f1489cdb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14840f66e0 0x7f1484106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:26:48.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.138+0000 7f1489cdb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14840f6ff0 0x7f1484107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:26:48.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.138+0000 7f1489cdb700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1484107a00 con 0x7f14840f6ff0 2021-11-03T18:26:48.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.138+0000 7f1489cdb700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1484107b40 con 0x7f14840f66e0 2021-11-03T18:26:48.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.138+0000 7f1482ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14840f6ff0 0x7f1484107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:26:48.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.138+0000 7f1482ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14840f6ff0 0x7f1484107450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34894/0 (socket says 172.21.15.27:34894) 2021-11-03T18:26:48.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.138+0000 7f1482ffd700 1 -- 172.21.15.27:0/918124638 learned_addr learned my addr 172.21.15.27:0/918124638 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:26:48.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.138+0000 7f14837fe700 1 --2- 172.21.15.27:0/918124638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14840f66e0 0x7f1484106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:26:48.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.138+0000 7f1482ffd700 1 -- 172.21.15.27:0/918124638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14840f66e0 msgr2=0x7f1484106f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:26:48.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.139+0000 7f1482ffd700 1 --2- 172.21.15.27:0/918124638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14840f66e0 0x7f1484106f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:48.142 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.139+0000 7f1482ffd700 1 -- 172.21.15.27:0/918124638 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1474005040 con 0x7f14840f6ff0 2021-11-03T18:26:48.142 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.139+0000 7f1482ffd700 1 --2- 172.21.15.27:0/918124638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14840f6ff0 0x7f1484107450 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 rx=0x7f14780026d0 tx=0x7f1478009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:26:48.142 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.139+0000 7f146bfff700 1 -- 172.21.15.27:0/918124638 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14780079c0 con 0x7f14840f6ff0 2021-11-03T18:26:48.142 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.139+0000 7f1489cdb700 1 -- 172.21.15.27:0/918124638 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14841084e0 con 0x7f14840f6ff0 2021-11-03T18:26:48.143 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.139+0000 7f146bfff700 1 -- 172.21.15.27:0/918124638 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f147800f040 con 0x7f14840f6ff0 2021-11-03T18:26:48.143 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.139+0000 7f146bfff700 1 -- 172.21.15.27:0/918124638 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f14780148b0 con 0x7f14840f6ff0 2021-11-03T18:26:48.143 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.139+0000 7f1489cdb700 1 -- 172.21.15.27:0/918124638 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1484108aa0 con 0x7f14840f6ff0 2021-11-03T18:26:48.144 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.140+0000 7f1489cdb700 1 -- 172.21.15.27:0/918124638 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1484045ad0 con 0x7f14840f6ff0 2021-11-03T18:26:48.145 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.140+0000 7f146bfff700 1 -- 172.21.15.27:0/918124638 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f1478007b20 con 0x7f14840f6ff0 2021-11-03T18:26:48.145 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.141+0000 7f146bfff700 1 --2- 172.21.15.27:0/918124638 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f146c0640b0 0x7f146c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:26:48.145 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.141+0000 7f146bfff700 1 -- 172.21.15.27:0/918124638 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f1478080740 con 0x7f14840f6ff0 2021-11-03T18:26:48.145 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.141+0000 7f14837fe700 1 --2- 172.21.15.27:0/918124638 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f146c0640b0 0x7f146c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:26:48.146 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.141+0000 7f14837fe700 1 --2- 172.21.15.27:0/918124638 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f146c0640b0 0x7f146c066570 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f1474004630 tx=0x7f1474009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:26:48.146 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.145+0000 7f146bfff700 1 -- 172.21.15.27:0/918124638 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f1478052aa0 con 0x7f14840f6ff0 2021-11-03T18:26:48.323 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.325+0000 7f1489cdb700 1 -- 172.21.15.27:0/918124638 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f14840ff080 con 0x7f146c0640b0 2021-11-03T18:26:48.324 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.326+0000 7f146bfff700 1 -- 172.21.15.27:0/918124638 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f1474019040 con 0x7f146c0640b0 2021-11-03T18:26:48.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.328+0000 7f1489cdb700 1 -- 172.21.15.27:0/918124638 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f146c0640b0 msgr2=0x7f146c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:26:48.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.328+0000 7f1489cdb700 1 --2- 172.21.15.27:0/918124638 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f146c0640b0 0x7f146c066570 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7f1474004630 tx=0x7f1474009b30).stop 2021-11-03T18:26:48.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.328+0000 7f1489cdb700 1 -- 172.21.15.27:0/918124638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14840f6ff0 msgr2=0x7f1484107450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:26:48.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.328+0000 7f1489cdb700 1 --2- 172.21.15.27:0/918124638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14840f6ff0 0x7f1484107450 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 rx=0x7f14780026d0 tx=0x7f1478009de0).stop 2021-11-03T18:26:48.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.328+0000 7f1489cdb700 1 -- 172.21.15.27:0/918124638 shutdown_connections 2021-11-03T18:26:48.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.328+0000 7f1489cdb700 1 --2- 172.21.15.27:0/918124638 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f146c0640b0 0x7f146c066570 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:48.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.328+0000 7f1489cdb700 1 --2- 172.21.15.27:0/918124638 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f14840f66e0 0x7f1484106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:48.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.328+0000 7f1489cdb700 1 --2- 172.21.15.27:0/918124638 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f14840f6ff0 0x7f1484107450 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:48.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.328+0000 7f1489cdb700 1 -- 172.21.15.27:0/918124638 >> 172.21.15.27:0/918124638 conn(0x7f14840f1f00 msgr2=0x7f14840fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:26:48.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.328+0000 7f1489cdb700 1 -- 172.21.15.27:0/918124638 shutdown_connections 2021-11-03T18:26:48.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.328+0000 7f1489cdb700 1 -- 172.21.15.27:0/918124638 wait complete. 2021-11-03T18:26:48.335 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:26:48.430 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.428+0000 7f44198e5700 1 -- 172.21.15.27:0/984686057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44140f66e0 msgr2=0x7f44140f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:26:48.430 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.428+0000 7f44198e5700 1 --2- 172.21.15.27:0/984686057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44140f66e0 0x7f44140f6b40 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 rx=0x7f4400004660 tx=0x7f4400009b30).stop 2021-11-03T18:26:48.431 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.428+0000 7f44198e5700 1 -- 172.21.15.27:0/984686057 shutdown_connections 2021-11-03T18:26:48.431 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.428+0000 7f44198e5700 1 --2- 172.21.15.27:0/984686057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44140f66e0 0x7f44140f6b40 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:48.431 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.428+0000 7f44198e5700 1 --2- 172.21.15.27:0/984686057 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44140f5440 0x7f44140f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:48.432 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.428+0000 7f44198e5700 1 -- 172.21.15.27:0/984686057 >> 172.21.15.27:0/984686057 conn(0x7f44140f0ba0 msgr2=0x7f44140f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:26:48.432 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.428+0000 7f44198e5700 1 -- 172.21.15.27:0/984686057 shutdown_connections 2021-11-03T18:26:48.432 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.428+0000 7f44198e5700 1 -- 172.21.15.27:0/984686057 wait complete. 2021-11-03T18:26:48.432 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.428+0000 7f44198e5700 1 Processor -- start 2021-11-03T18:26:48.433 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.429+0000 7f44198e5700 1 -- start start 2021-11-03T18:26:48.433 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.429+0000 7f44198e5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44140f5440 0x7f4414105cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:26:48.433 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.429+0000 7f44198e5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44140f66e0 0x7f44141061c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:26:48.434 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.429+0000 7f44198e5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4414106770 con 0x7f44140f5440 2021-11-03T18:26:48.434 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.429+0000 7f44198e5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4414106f70 con 0x7f44140f66e0 2021-11-03T18:26:48.434 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.429+0000 7f44188e3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44140f5440 0x7f4414105cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:26:48.435 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.429+0000 7f44188e3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44140f5440 0x7f4414105cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34904/0 (socket says 172.21.15.27:34904) 2021-11-03T18:26:48.435 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.429+0000 7f44188e3700 1 -- 172.21.15.27:0/3923044438 learned_addr learned my addr 172.21.15.27:0/3923044438 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:26:48.436 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.429+0000 7f4413fff700 1 --2- 172.21.15.27:0/3923044438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44140f66e0 0x7f44141061c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:26:48.436 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.429+0000 7f44188e3700 1 -- 172.21.15.27:0/3923044438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44140f66e0 msgr2=0x7f44141061c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:26:48.436 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.429+0000 7f44188e3700 1 --2- 172.21.15.27:0/3923044438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44140f66e0 0x7f44141061c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:48.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.429+0000 7f44188e3700 1 -- 172.21.15.27:0/3923044438 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4400005040 con 0x7f44140f5440 2021-11-03T18:26:48.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.429+0000 7f4413fff700 1 --2- 172.21.15.27:0/3923044438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44140f66e0 0x7f44141061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:26:48.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.429+0000 7f44188e3700 1 --2- 172.21.15.27:0/3923044438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44140f5440 0x7f4414105cd0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 rx=0x7f440800f910 tx=0x7f4408009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:26:48.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.430+0000 7f44117fa700 1 -- 172.21.15.27:0/3923044438 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4408009500 con 0x7f44140f5440 2021-11-03T18:26:48.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.430+0000 7f44117fa700 1 -- 172.21.15.27:0/3923044438 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f4408009660 con 0x7f44140f5440 2021-11-03T18:26:48.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.430+0000 7f44198e5700 1 -- 172.21.15.27:0/3923044438 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44141071f0 con 0x7f44140f5440 2021-11-03T18:26:48.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.430+0000 7f44117fa700 1 -- 172.21.15.27:0/3923044438 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f440800de20 con 0x7f44140f5440 2021-11-03T18:26:48.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.430+0000 7f44198e5700 1 -- 172.21.15.27:0/3923044438 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4414107760 con 0x7f44140f5440 2021-11-03T18:26:48.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.431+0000 7f44198e5700 1 -- 172.21.15.27:0/3923044438 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4414059100 con 0x7f44140f5440 2021-11-03T18:26:48.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.431+0000 7f44117fa700 1 -- 172.21.15.27:0/3923044438 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f440800e040 con 0x7f44140f5440 2021-11-03T18:26:48.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.431+0000 7f44117fa700 1 --2- 172.21.15.27:0/3923044438 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4404064100 0x7f44040665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:26:48.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.431+0000 7f44117fa700 1 -- 172.21.15.27:0/3923044438 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f44080830d0 con 0x7f44140f5440 2021-11-03T18:26:48.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.432+0000 7f4413fff700 1 --2- 172.21.15.27:0/3923044438 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4404064100 0x7f44040665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:26:48.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.432+0000 7f4413fff700 1 --2- 172.21.15.27:0/3923044438 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4404064100 0x7f44040665c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7f4400004630 tx=0x7f4400009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:26:48.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.436+0000 7f44117fa700 1 -- 172.21.15.27:0/3923044438 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f4408055340 con 0x7f44140f5440 2021-11-03T18:26:48.606 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.608+0000 7f44198e5700 1 -- 172.21.15.27:0/3923044438 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f44140ff680 con 0x7f4404064100 2021-11-03T18:26:48.621 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.623+0000 7f44117fa700 1 -- 172.21.15.27:0/3923044438 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f44140ff680 con 0x7f4404064100 2021-11-03T18:26:48.621 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:26:48.621 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (78m) 7m ago 79m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:26:48.622 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (70m) 7m ago 79m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:26:48.622 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (69m) 6m ago 78m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:26:48.622 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (78m) 7m ago 79m 42.4M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:26:48.622 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (71m) 7m ago 74m 26.5M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:26:48.623 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (74m) 7m ago 74m 3849M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:26:48.623 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (70m) 6m ago 74m 26.1M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:26:48.623 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (72m) 6m ago 74m 27.2M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:26:48.623 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (72m) 7m ago 80m 590M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:26:48.623 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (71m) 6m ago 78m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:26:48.624 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (71m) 7m ago 81m 312M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:26:48.624 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (70m) 6m ago 78m 145M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:26:48.624 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (79m) 7m ago 79m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:26:48.624 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (78m) 6m ago 78m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:26:48.625 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (69m) 7m ago 78m 392M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:26:48.625 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (69m) 7m ago 77m 427M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:26:48.625 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (68m) 7m ago 77m 320M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:26:48.625 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (68m) 6m ago 77m 454M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:26:48.625 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (67m) 6m ago 76m 359M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:26:48.626 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (67m) 6m ago 76m 368M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:26:48.626 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (78m) 7m ago 78m 59.9M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:26:48.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.625+0000 7f44198e5700 1 -- 172.21.15.27:0/3923044438 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4404064100 msgr2=0x7f44040665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:26:48.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.625+0000 7f44198e5700 1 --2- 172.21.15.27:0/3923044438 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4404064100 0x7f44040665c0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7f4400004630 tx=0x7f4400009b30).stop 2021-11-03T18:26:48.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.625+0000 7f44198e5700 1 -- 172.21.15.27:0/3923044438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44140f5440 msgr2=0x7f4414105cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:26:48.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.625+0000 7f44198e5700 1 --2- 172.21.15.27:0/3923044438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44140f5440 0x7f4414105cd0 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 rx=0x7f440800f910 tx=0x7f4408009d80).stop 2021-11-03T18:26:48.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.625+0000 7f44198e5700 1 -- 172.21.15.27:0/3923044438 shutdown_connections 2021-11-03T18:26:48.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.625+0000 7f44198e5700 1 --2- 172.21.15.27:0/3923044438 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4404064100 0x7f44040665c0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:48.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.625+0000 7f44198e5700 1 --2- 172.21.15.27:0/3923044438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44140f5440 0x7f4414105cd0 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:48.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.625+0000 7f44198e5700 1 --2- 172.21.15.27:0/3923044438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44140f66e0 0x7f44141061c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:48.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.625+0000 7f44198e5700 1 -- 172.21.15.27:0/3923044438 >> 172.21.15.27:0/3923044438 conn(0x7f44140f0ba0 msgr2=0x7f44140fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:26:48.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.625+0000 7f44198e5700 1 -- 172.21.15.27:0/3923044438 shutdown_connections 2021-11-03T18:26:48.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.625+0000 7f44198e5700 1 -- 172.21.15.27:0/3923044438 wait complete. 2021-11-03T18:26:48.727 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.725+0000 7fd6353e6700 1 -- 172.21.15.27:0/3099890067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6300f5ed0 msgr2=0x7fd6300f6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:26:48.727 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.725+0000 7fd6353e6700 1 --2- 172.21.15.27:0/3099890067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6300f5ed0 0x7fd6300f6900 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 rx=0x7fd618004660 tx=0x7fd618009b30).stop 2021-11-03T18:26:48.727 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.725+0000 7fd6353e6700 1 -- 172.21.15.27:0/3099890067 shutdown_connections 2021-11-03T18:26:48.728 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.725+0000 7fd6353e6700 1 --2- 172.21.15.27:0/3099890067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6300f5ed0 0x7fd6300f6900 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:48.728 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.725+0000 7fd6353e6700 1 --2- 172.21.15.27:0/3099890067 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6300f55c0 0x7fd6300f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:48.728 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.725+0000 7fd6353e6700 1 -- 172.21.15.27:0/3099890067 >> 172.21.15.27:0/3099890067 conn(0x7fd6300f0dc0 msgr2=0x7fd6300f31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:26:48.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.725+0000 7fd6353e6700 1 -- 172.21.15.27:0/3099890067 shutdown_connections 2021-11-03T18:26:48.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.725+0000 7fd6353e6700 1 -- 172.21.15.27:0/3099890067 wait complete. 2021-11-03T18:26:48.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.726+0000 7fd6353e6700 1 Processor -- start 2021-11-03T18:26:48.730 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.726+0000 7fd6353e6700 1 -- start start 2021-11-03T18:26:48.730 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.726+0000 7fd6353e6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6300f55c0 0x7fd630105e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:26:48.730 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.726+0000 7fd6353e6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6300f5ed0 0x7fd630106330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:26:48.731 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.726+0000 7fd6353e6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd630106930 con 0x7fd6300f55c0 2021-11-03T18:26:48.731 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.726+0000 7fd6353e6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd630107140 con 0x7fd6300f5ed0 2021-11-03T18:26:48.731 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.726+0000 7fd62effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6300f55c0 0x7fd630105e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:26:48.732 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.726+0000 7fd62effd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6300f55c0 0x7fd630105e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34914/0 (socket says 172.21.15.27:34914) 2021-11-03T18:26:48.732 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.726+0000 7fd62effd700 1 -- 172.21.15.27:0/2767386847 learned_addr learned my addr 172.21.15.27:0/2767386847 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:26:48.732 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.726+0000 7fd62e7fc700 1 --2- 172.21.15.27:0/2767386847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6300f5ed0 0x7fd630106330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:26:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.726+0000 7fd62effd700 1 -- 172.21.15.27:0/2767386847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6300f5ed0 msgr2=0x7fd630106330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:26:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.726+0000 7fd62effd700 1 --2- 172.21.15.27:0/2767386847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6300f5ed0 0x7fd630106330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:48.733 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.726+0000 7fd62effd700 1 -- 172.21.15.27:0/2767386847 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd618005040 con 0x7fd6300f55c0 2021-11-03T18:26:48.734 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.726+0000 7fd62effd700 1 --2- 172.21.15.27:0/2767386847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6300f55c0 0x7fd630105e40 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 rx=0x7fd62800e7f0 tx=0x7fd62800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:26:48.734 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.727+0000 7fd6277fe700 1 -- 172.21.15.27:0/2767386847 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6280092e0 con 0x7fd6300f55c0 2021-11-03T18:26:48.734 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.727+0000 7fd6277fe700 1 -- 172.21.15.27:0/2767386847 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fd62800f040 con 0x7fd6300f55c0 2021-11-03T18:26:48.735 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.727+0000 7fd6353e6700 1 -- 172.21.15.27:0/2767386847 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6301073c0 con 0x7fd6300f55c0 2021-11-03T18:26:48.735 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.727+0000 7fd6277fe700 1 -- 172.21.15.27:0/2767386847 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd628014750 con 0x7fd6300f55c0 2021-11-03T18:26:48.735 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.727+0000 7fd6353e6700 1 -- 172.21.15.27:0/2767386847 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd630107980 con 0x7fd6300f55c0 2021-11-03T18:26:48.736 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.727+0000 7fd6353e6700 1 -- 172.21.15.27:0/2767386847 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6300f4a50 con 0x7fd6300f55c0 2021-11-03T18:26:48.736 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.728+0000 7fd6277fe700 1 -- 172.21.15.27:0/2767386847 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fd62801b030 con 0x7fd6300f55c0 2021-11-03T18:26:48.736 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.728+0000 7fd6277fe700 1 --2- 172.21.15.27:0/2767386847 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd61c0640b0 0x7fd61c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:26:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.728+0000 7fd6277fe700 1 -- 172.21.15.27:0/2767386847 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fd628080580 con 0x7fd6300f55c0 2021-11-03T18:26:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.728+0000 7fd62e7fc700 1 --2- 172.21.15.27:0/2767386847 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd61c0640b0 0x7fd61c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:26:48.737 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.729+0000 7fd62e7fc700 1 --2- 172.21.15.27:0/2767386847 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd61c0640b0 0x7fd61c066570 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7fd6180043d0 tx=0x7fd61800b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:26:48.738 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.732+0000 7fd6277fe700 1 -- 172.21.15.27:0/2767386847 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fd6280528a0 con 0x7fd6300f55c0 2021-11-03T18:26:48.959 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.961+0000 7fd6353e6700 1 -- 172.21.15.27:0/2767386847 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd6300ff7c0 con 0x7fd6300f55c0 2021-11-03T18:26:48.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.961+0000 7fd6277fe700 1 -- 172.21.15.27:0/2767386847 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fd6280192e0 con 0x7fd6300f55c0 2021-11-03T18:26:48.960 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:26:48.961 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:26:48.961 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:26:48.961 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:26:48.962 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:26:48.962 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:26:48.962 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:26:48.963 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:26:48.963 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:26:48.963 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:26:48.964 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:26:48.964 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:26:48.964 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:26:48.964 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:26:48.965 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:26:48.965 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:26:48.965 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:26:48.966 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:26:48.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.963+0000 7fd6353e6700 1 -- 172.21.15.27:0/2767386847 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd61c0640b0 msgr2=0x7fd61c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:26:48.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.963+0000 7fd6353e6700 1 --2- 172.21.15.27:0/2767386847 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd61c0640b0 0x7fd61c066570 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 rx=0x7fd6180043d0 tx=0x7fd61800b040).stop 2021-11-03T18:26:48.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.963+0000 7fd6353e6700 1 -- 172.21.15.27:0/2767386847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6300f55c0 msgr2=0x7fd630105e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:26:48.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.963+0000 7fd6353e6700 1 --2- 172.21.15.27:0/2767386847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6300f55c0 0x7fd630105e40 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 rx=0x7fd62800e7f0 tx=0x7fd62800b040).stop 2021-11-03T18:26:48.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.963+0000 7fd6353e6700 1 -- 172.21.15.27:0/2767386847 shutdown_connections 2021-11-03T18:26:48.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.963+0000 7fd6353e6700 1 --2- 172.21.15.27:0/2767386847 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd61c0640b0 0x7fd61c066570 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:48.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.963+0000 7fd6353e6700 1 --2- 172.21.15.27:0/2767386847 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6300f55c0 0x7fd630105e40 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:48.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.963+0000 7fd6353e6700 1 --2- 172.21.15.27:0/2767386847 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6300f5ed0 0x7fd630106330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:48.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.963+0000 7fd6353e6700 1 -- 172.21.15.27:0/2767386847 >> 172.21.15.27:0/2767386847 conn(0x7fd6300f0dc0 msgr2=0x7fd6300fa350 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:26:48.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.963+0000 7fd6353e6700 1 -- 172.21.15.27:0/2767386847 shutdown_connections 2021-11-03T18:26:48.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:48.963+0000 7fd6353e6700 1 -- 172.21.15.27:0/2767386847 wait complete. 2021-11-03T18:26:48.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:48 smithi115 ceph-mon[65453]: pgmap v2166: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:48.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:48 smithi115 ceph-mon[65453]: from='client.36328 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:26:48.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:48 smithi115 ceph-mon[65453]: from='client.36332 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:26:49.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:48 smithi027 ceph-mon[97244]: pgmap v2166: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:49.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:48 smithi027 ceph-mon[97244]: from='client.36328 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:26:49.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:48 smithi027 ceph-mon[97244]: from='client.36332 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:26:49.062 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.063+0000 7f12ef10b700 1 -- 172.21.15.27:0/1605342470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12e80f4d60 msgr2=0x7f12e80f5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:26:49.062 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.063+0000 7f12ef10b700 1 --2- 172.21.15.27:0/1605342470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12e80f4d60 0x7f12e80f5180 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 rx=0x7f12dc004660 tx=0x7f12dc009b30).stop 2021-11-03T18:26:49.062 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.063+0000 7f12ef10b700 1 -- 172.21.15.27:0/1605342470 shutdown_connections 2021-11-03T18:26:49.062 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.063+0000 7f12ef10b700 1 --2- 172.21.15.27:0/1605342470 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12e80f5f60 0x7f12e80f63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:49.063 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.063+0000 7f12ef10b700 1 --2- 172.21.15.27:0/1605342470 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12e80f4d60 0x7f12e80f5180 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:49.063 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.063+0000 7f12ef10b700 1 -- 172.21.15.27:0/1605342470 >> 172.21.15.27:0/1605342470 conn(0x7f12e80f04c0 msgr2=0x7f12e80f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:26:49.063 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.063+0000 7f12ef10b700 1 -- 172.21.15.27:0/1605342470 shutdown_connections 2021-11-03T18:26:49.063 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.064+0000 7f12ef10b700 1 -- 172.21.15.27:0/1605342470 wait complete. 2021-11-03T18:26:49.064 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.064+0000 7f12ef10b700 1 Processor -- start 2021-11-03T18:26:49.064 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.064+0000 7f12ef10b700 1 -- start start 2021-11-03T18:26:49.067 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.064+0000 7f12ef10b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12e80f4d60 0x7f12e8109c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:26:49.068 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.064+0000 7f12ef10b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12e80f5f60 0x7f12e810c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:26:49.068 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.064+0000 7f12ef10b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e80fc9c0 con 0x7f12e80f5f60 2021-11-03T18:26:49.068 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.064+0000 7f12ef10b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e80fcb00 con 0x7f12e80f4d60 2021-11-03T18:26:49.069 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.064+0000 7f12e7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12e80f5f60 0x7f12e810c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:26:49.069 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.065+0000 7f12e7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12e80f5f60 0x7f12e810c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34926/0 (socket says 172.21.15.27:34926) 2021-11-03T18:26:49.069 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.065+0000 7f12e7fff700 1 -- 172.21.15.27:0/3793550963 learned_addr learned my addr 172.21.15.27:0/3793550963 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:26:49.069 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.065+0000 7f12ecea7700 1 --2- 172.21.15.27:0/3793550963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12e80f4d60 0x7f12e8109c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:26:49.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.065+0000 7f12e7fff700 1 -- 172.21.15.27:0/3793550963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12e80f4d60 msgr2=0x7f12e8109c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:26:49.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.065+0000 7f12e7fff700 1 --2- 172.21.15.27:0/3793550963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12e80f4d60 0x7f12e8109c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:49.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.065+0000 7f12e7fff700 1 -- 172.21.15.27:0/3793550963 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12dc005040 con 0x7f12e80f5f60 2021-11-03T18:26:49.070 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.065+0000 7f12e7fff700 1 --2- 172.21.15.27:0/3793550963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12e80f5f60 0x7f12e810c150 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 rx=0x7f12d40026d0 tx=0x7f12d4009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:26:49.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.065+0000 7f12e57fa700 1 -- 172.21.15.27:0/3793550963 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12d40079c0 con 0x7f12e80f5f60 2021-11-03T18:26:49.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.065+0000 7f12e57fa700 1 -- 172.21.15.27:0/3793550963 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f12d400f040 con 0x7f12e80f5f60 2021-11-03T18:26:49.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.065+0000 7f12ef10b700 1 -- 172.21.15.27:0/3793550963 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12e810c700 con 0x7f12e80f5f60 2021-11-03T18:26:49.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.065+0000 7f12e57fa700 1 -- 172.21.15.27:0/3793550963 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12d40148b0 con 0x7f12e80f5f60 2021-11-03T18:26:49.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.065+0000 7f12ef10b700 1 -- 172.21.15.27:0/3793550963 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e810ccc0 con 0x7f12e80f5f60 2021-11-03T18:26:49.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.066+0000 7f12ef10b700 1 -- 172.21.15.27:0/3793550963 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12e8045ad0 con 0x7f12e80f5f60 2021-11-03T18:26:49.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.067+0000 7f12e57fa700 1 -- 172.21.15.27:0/3793550963 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f12d4007b20 con 0x7f12e80f5f60 2021-11-03T18:26:49.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.067+0000 7f12e57fa700 1 --2- 172.21.15.27:0/3793550963 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f12d8064090 0x7f12d8066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:26:49.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.067+0000 7f12e57fa700 1 -- 172.21.15.27:0/3793550963 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f12d4080660 con 0x7f12e80f5f60 2021-11-03T18:26:49.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.067+0000 7f12ecea7700 1 --2- 172.21.15.27:0/3793550963 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f12d8064090 0x7f12d8066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:26:49.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.067+0000 7f12ecea7700 1 --2- 172.21.15.27:0/3793550963 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f12d8064090 0x7f12d8066550 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f12dc013690 tx=0x7f12dc009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:26:49.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.071+0000 7f12e57fa700 1 -- 172.21.15.27:0/3793550963 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f12d40529c0 con 0x7f12e80f5f60 2021-11-03T18:26:49.274 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.276+0000 7f12ef10b700 1 -- 172.21.15.27:0/3793550963 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f12e8027a00 con 0x7f12e80f5f60 2021-11-03T18:26:49.274 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.276+0000 7f12e57fa700 1 -- 172.21.15.27:0/3793550963 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f12d40525b0 con 0x7f12e80f5f60 2021-11-03T18:26:49.275 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:26:49.275 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:26:49.275 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:26:49.276 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:26:49.276 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:26:49.276 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:26:49.276 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:26:49.276 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:26:49.277 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:26:49.277 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:26:49.277 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:26:49.277 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:26:49.278 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:26:49.278 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:26:49.278 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:26:49.278 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:26:49.278 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:26:49.279 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:26:49.279 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:26:49.279 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:26:49.279 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:26:49.279 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:26:49.280 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:26:49.280 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:26:49.280 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:26:49.280 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:26:49.281 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:26:49.281 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:26:49.281 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:26:49.281 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:26:49.281 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:26:49.282 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:26:49.282 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:26:49.282 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:26:49.282 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:26:49.283 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:26:49.283 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:26:49.283 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:26:49.283 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:26:49.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.278+0000 7f12ef10b700 1 -- 172.21.15.27:0/3793550963 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f12d8064090 msgr2=0x7f12d8066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:26:49.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.278+0000 7f12ef10b700 1 --2- 172.21.15.27:0/3793550963 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f12d8064090 0x7f12d8066550 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f12dc013690 tx=0x7f12dc009b30).stop 2021-11-03T18:26:49.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.278+0000 7f12ef10b700 1 -- 172.21.15.27:0/3793550963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12e80f5f60 msgr2=0x7f12e810c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:26:49.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.278+0000 7f12ef10b700 1 --2- 172.21.15.27:0/3793550963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12e80f5f60 0x7f12e810c150 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 rx=0x7f12d40026d0 tx=0x7f12d4009de0).stop 2021-11-03T18:26:49.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.278+0000 7f12ef10b700 1 -- 172.21.15.27:0/3793550963 shutdown_connections 2021-11-03T18:26:49.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.278+0000 7f12ef10b700 1 --2- 172.21.15.27:0/3793550963 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f12d8064090 0x7f12d8066550 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:49.286 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.278+0000 7f12ef10b700 1 --2- 172.21.15.27:0/3793550963 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f12e80f4d60 0x7f12e8109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:49.286 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.278+0000 7f12ef10b700 1 --2- 172.21.15.27:0/3793550963 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f12e80f5f60 0x7f12e810c150 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:26:49.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.278+0000 7f12ef10b700 1 -- 172.21.15.27:0/3793550963 >> 172.21.15.27:0/3793550963 conn(0x7f12e80f04c0 msgr2=0x7f12e80f9a40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:26:49.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.278+0000 7f12ef10b700 1 -- 172.21.15.27:0/3793550963 shutdown_connections 2021-11-03T18:26:49.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:26:49.278+0000 7f12ef10b700 1 -- 172.21.15.27:0/3793550963 wait complete. 2021-11-03T18:26:49.287 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:26:49.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:49 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2767386847' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:49.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:49 smithi115 ceph-mon[65453]: pgmap v2167: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:49.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:49 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3793550963' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:26:50.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:49 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2767386847' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:50.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:49 smithi027 ceph-mon[97244]: pgmap v2167: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:50.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:49 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3793550963' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:26:51.027 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4252.461819 secs 2021-11-03T18:26:51.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4252.461819 secs 2021-11-03T18:26:52.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:51 smithi027 ceph-mon[97244]: pgmap v2168: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:52.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:51 smithi115 ceph-mon[65453]: pgmap v2168: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:54.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:54 smithi115 ceph-mon[65453]: pgmap v2169: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:54.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:54 smithi027 ceph-mon[97244]: pgmap v2169: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:55.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:26:55.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:26:55.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:26:55.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:55.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:55.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:55.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:55.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:55 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:26:55.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:26:55.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:26:55.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:26:55.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:55.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:55.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:55.524 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:26:55.524 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:55 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:26:56.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:56 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4257.461954 secs 2021-11-03T18:26:56.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:56 smithi115 ceph-mon[65453]: pgmap v2170: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:56.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:26:56.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:26:56.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:56 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4257.461954 secs 2021-11-03T18:26:56.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:56 smithi027 ceph-mon[97244]: pgmap v2170: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:56.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:26:56.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:26:58.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:26:58 smithi115 ceph-mon[65453]: pgmap v2171: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:26:58.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:26:58 smithi027 ceph-mon[97244]: pgmap v2171: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:00.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:00 smithi115 ceph-mon[65453]: pgmap v2172: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:00.521 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:00 smithi027 ceph-mon[97244]: pgmap v2172: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:01.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:01 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4262.462089 secs 2021-11-03T18:27:01.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:01 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4262.462089 secs 2021-11-03T18:27:02.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:02 smithi115 ceph-mon[65453]: pgmap v2173: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:02.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:02 smithi027 ceph-mon[97244]: pgmap v2173: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:04.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:04 smithi115 ceph-mon[65453]: pgmap v2174: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:04.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:27:04.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:27:04.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:27:04.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:04.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:04.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:04.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:04 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:04.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:04 smithi027 ceph-mon[97244]: pgmap v2174: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:04.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:27:04.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:27:04.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:27:04.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:04.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:04.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:04.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:04 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:05.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:05 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:27:05.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4267.462212 secs 2021-11-03T18:27:05.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:05 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:27:05.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4267.462212 secs 2021-11-03T18:27:06.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:06 smithi115 ceph-mon[65453]: pgmap v2175: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:06.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:06 smithi027 ceph-mon[97244]: pgmap v2175: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:08.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:08 smithi115 ceph-mon[65453]: pgmap v2176: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:08.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:08 smithi027 ceph-mon[97244]: pgmap v2176: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:10.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:10 smithi115 ceph-mon[65453]: pgmap v2177: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:10.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4272.462307 secs 2021-11-03T18:27:10.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:10 smithi027 ceph-mon[97244]: pgmap v2177: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:10.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4272.462307 secs 2021-11-03T18:27:12.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:12 smithi115 ceph-mon[65453]: pgmap v2178: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:12.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:12 smithi027 ceph-mon[97244]: pgmap v2178: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:14.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:14 smithi115 ceph-mon[65453]: pgmap v2179: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:14.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:27:14.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:27:14.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:27:14.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:14.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:14.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:14.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:14 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:14.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:14 smithi027 ceph-mon[97244]: pgmap v2179: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:14.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:27:14.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:27:14.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:27:14.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:14.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:14.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:14.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:14 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:15.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:15 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:27:15.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4277.462451 secs 2021-11-03T18:27:15.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:15 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:27:15.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4277.462451 secs 2021-11-03T18:27:16.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:16 smithi115 ceph-mon[65453]: pgmap v2180: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:16.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:16 smithi027 ceph-mon[97244]: pgmap v2180: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:18.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:18 smithi115 ceph-mon[65453]: pgmap v2181: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:18.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:18 smithi027 ceph-mon[97244]: pgmap v2181: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:19.393 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.392+0000 7f5121ec6700 1 -- 172.21.15.27:0/3510669415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f511c0f66e0 msgr2=0x7f511c0f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:27:19.393 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.392+0000 7f5121ec6700 1 --2- 172.21.15.27:0/3510669415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f511c0f66e0 0x7f511c0f6b00 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 rx=0x7f510c004660 tx=0x7f510c009b30).stop 2021-11-03T18:27:19.393 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.392+0000 7f5121ec6700 1 -- 172.21.15.27:0/3510669415 shutdown_connections 2021-11-03T18:27:19.394 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.392+0000 7f5121ec6700 1 --2- 172.21.15.27:0/3510669415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f511c0f6ff0 0x7f511c0f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:19.395 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.392+0000 7f5121ec6700 1 --2- 172.21.15.27:0/3510669415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f511c0f66e0 0x7f511c0f6b00 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:19.395 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.392+0000 7f5121ec6700 1 -- 172.21.15.27:0/3510669415 >> 172.21.15.27:0/3510669415 conn(0x7f511c0f1f00 msgr2=0x7f511c0f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:27:19.395 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.392+0000 7f5121ec6700 1 -- 172.21.15.27:0/3510669415 shutdown_connections 2021-11-03T18:27:19.395 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.392+0000 7f5121ec6700 1 -- 172.21.15.27:0/3510669415 wait complete. 2021-11-03T18:27:19.396 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.393+0000 7f5121ec6700 1 Processor -- start 2021-11-03T18:27:19.396 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.393+0000 7f5121ec6700 1 -- start start 2021-11-03T18:27:19.396 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.393+0000 7f5121ec6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f511c0f66e0 0x7f511c106f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:27:19.396 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.393+0000 7f5121ec6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f511c0f6ff0 0x7f511c1073f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:27:19.397 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.393+0000 7f5121ec6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511c1079a0 con 0x7f511c0f66e0 2021-11-03T18:27:19.397 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.393+0000 7f5121ec6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511c107ae0 con 0x7f511c0f6ff0 2021-11-03T18:27:19.397 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.393+0000 7f511b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f511c0f66e0 0x7f511c106f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:27:19.397 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.393+0000 7f511b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f511c0f66e0 0x7f511c106f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34942/0 (socket says 172.21.15.27:34942) 2021-11-03T18:27:19.397 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.393+0000 7f511b7fe700 1 -- 172.21.15.27:0/2950771278 learned_addr learned my addr 172.21.15.27:0/2950771278 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:27:19.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.393+0000 7f511affd700 1 --2- 172.21.15.27:0/2950771278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f511c0f6ff0 0x7f511c1073f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:27:19.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.393+0000 7f511b7fe700 1 -- 172.21.15.27:0/2950771278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f511c0f6ff0 msgr2=0x7f511c1073f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:27:19.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.394+0000 7f511b7fe700 1 --2- 172.21.15.27:0/2950771278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f511c0f6ff0 0x7f511c1073f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:19.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.394+0000 7f511b7fe700 1 -- 172.21.15.27:0/2950771278 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f510c005040 con 0x7f511c0f66e0 2021-11-03T18:27:19.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.394+0000 7f511b7fe700 1 --2- 172.21.15.27:0/2950771278 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f511c0f66e0 0x7f511c106f00 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 rx=0x7f510c009990 tx=0x7f510c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:27:19.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.394+0000 7f5103fff700 1 -- 172.21.15.27:0/2950771278 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f510c018070 con 0x7f511c0f66e0 2021-11-03T18:27:19.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.394+0000 7f5103fff700 1 -- 172.21.15.27:0/2950771278 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f510c007ba0 con 0x7f511c0f66e0 2021-11-03T18:27:19.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.394+0000 7f5121ec6700 1 -- 172.21.15.27:0/2950771278 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f511c108420 con 0x7f511c0f66e0 2021-11-03T18:27:19.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.394+0000 7f5103fff700 1 -- 172.21.15.27:0/2950771278 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f510c01c3c0 con 0x7f511c0f66e0 2021-11-03T18:27:19.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.394+0000 7f5121ec6700 1 -- 172.21.15.27:0/2950771278 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f511c108980 con 0x7f511c0f66e0 2021-11-03T18:27:19.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.395+0000 7f5121ec6700 1 -- 172.21.15.27:0/2950771278 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f511c045ad0 con 0x7f511c0f66e0 2021-11-03T18:27:19.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.395+0000 7f5103fff700 1 -- 172.21.15.27:0/2950771278 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f510c01d420 con 0x7f511c0f66e0 2021-11-03T18:27:19.402 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.396+0000 7f5103fff700 1 --2- 172.21.15.27:0/2950771278 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f51040640b0 0x7f5104066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:27:19.402 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.396+0000 7f5103fff700 1 -- 172.21.15.27:0/2950771278 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f510c082500 con 0x7f511c0f66e0 2021-11-03T18:27:19.402 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.396+0000 7f511affd700 1 --2- 172.21.15.27:0/2950771278 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f51040640b0 0x7f5104066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:27:19.402 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.396+0000 7f511affd700 1 --2- 172.21.15.27:0/2950771278 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f51040640b0 0x7f5104066570 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f5110009740 tx=0x7f5110009c60).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:27:19.403 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.400+0000 7f5103fff700 1 -- 172.21.15.27:0/2950771278 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f510c0548c0 con 0x7f511c0f66e0 2021-11-03T18:27:19.580 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.581+0000 7f5121ec6700 1 -- 172.21.15.27:0/2950771278 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f511c0fef40 con 0x7f51040640b0 2021-11-03T18:27:19.581 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.582+0000 7f5103fff700 1 -- 172.21.15.27:0/2950771278 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f511c0fef40 con 0x7f51040640b0 2021-11-03T18:27:19.583 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.584+0000 7f5121ec6700 1 -- 172.21.15.27:0/2950771278 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f51040640b0 msgr2=0x7f5104066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:27:19.583 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.584+0000 7f5121ec6700 1 --2- 172.21.15.27:0/2950771278 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f51040640b0 0x7f5104066570 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f5110009740 tx=0x7f5110009c60).stop 2021-11-03T18:27:19.583 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.584+0000 7f5121ec6700 1 -- 172.21.15.27:0/2950771278 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f511c0f66e0 msgr2=0x7f511c106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:27:19.584 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.584+0000 7f5121ec6700 1 --2- 172.21.15.27:0/2950771278 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f511c0f66e0 0x7f511c106f00 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 rx=0x7f510c009990 tx=0x7f510c009b30).stop 2021-11-03T18:27:19.584 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.584+0000 7f5121ec6700 1 -- 172.21.15.27:0/2950771278 shutdown_connections 2021-11-03T18:27:19.584 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.584+0000 7f5121ec6700 1 --2- 172.21.15.27:0/2950771278 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f51040640b0 0x7f5104066570 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:19.584 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.584+0000 7f5121ec6700 1 --2- 172.21.15.27:0/2950771278 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f511c0f66e0 0x7f511c106f00 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:19.585 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.584+0000 7f5121ec6700 1 --2- 172.21.15.27:0/2950771278 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f511c0f6ff0 0x7f511c1073f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:19.585 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.584+0000 7f5121ec6700 1 -- 172.21.15.27:0/2950771278 >> 172.21.15.27:0/2950771278 conn(0x7f511c0f1f00 msgr2=0x7f511c0fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:27:19.585 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.584+0000 7f5121ec6700 1 -- 172.21.15.27:0/2950771278 shutdown_connections 2021-11-03T18:27:19.585 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.584+0000 7f5121ec6700 1 -- 172.21.15.27:0/2950771278 wait complete. 2021-11-03T18:27:19.593 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:27:19.684 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.685+0000 7f75ed671700 1 -- 172.21.15.27:0/88852957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75e80f55c0 msgr2=0x7f75e80f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:27:19.685 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.685+0000 7f75ed671700 1 --2- 172.21.15.27:0/88852957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75e80f55c0 0x7f75e80f59e0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 rx=0x7f75e00099e0 tx=0x7f75e0004d10).stop 2021-11-03T18:27:19.685 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.685+0000 7f75ed671700 1 -- 172.21.15.27:0/88852957 shutdown_connections 2021-11-03T18:27:19.685 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.685+0000 7f75ed671700 1 --2- 172.21.15.27:0/88852957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75e80f6860 0x7f75e80f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:19.685 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.685+0000 7f75ed671700 1 --2- 172.21.15.27:0/88852957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75e80f55c0 0x7f75e80f59e0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:19.686 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.685+0000 7f75ed671700 1 -- 172.21.15.27:0/88852957 >> 172.21.15.27:0/88852957 conn(0x7f75e80f0d40 msgr2=0x7f75e80f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:27:19.686 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.685+0000 7f75ed671700 1 -- 172.21.15.27:0/88852957 shutdown_connections 2021-11-03T18:27:19.686 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.685+0000 7f75ed671700 1 -- 172.21.15.27:0/88852957 wait complete. 2021-11-03T18:27:19.686 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.686+0000 7f75ed671700 1 Processor -- start 2021-11-03T18:27:19.686 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.686+0000 7f75ed671700 1 -- start start 2021-11-03T18:27:19.689 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.686+0000 7f75ed671700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75e80f55c0 0x7f75e8107060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:27:19.689 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.686+0000 7f75ed671700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75e80f6860 0x7f75e8107550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:27:19.689 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.686+0000 7f75ed671700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e8107b00 con 0x7f75e80f55c0 2021-11-03T18:27:19.690 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.686+0000 7f75ed671700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75e8107c40 con 0x7f75e80f6860 2021-11-03T18:27:19.690 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.686+0000 7f75e6ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75e80f55c0 0x7f75e8107060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:27:19.690 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.686+0000 7f75e6ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75e80f55c0 0x7f75e8107060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34952/0 (socket says 172.21.15.27:34952) 2021-11-03T18:27:19.690 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.686+0000 7f75e6ffd700 1 -- 172.21.15.27:0/1435237415 learned_addr learned my addr 172.21.15.27:0/1435237415 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:27:19.690 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.686+0000 7f75e67fc700 1 --2- 172.21.15.27:0/1435237415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75e80f6860 0x7f75e8107550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:27:19.691 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.686+0000 7f75e6ffd700 1 -- 172.21.15.27:0/1435237415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75e80f6860 msgr2=0x7f75e8107550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:27:19.691 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.686+0000 7f75e6ffd700 1 --2- 172.21.15.27:0/1435237415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75e80f6860 0x7f75e8107550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:19.691 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.686+0000 7f75e6ffd700 1 -- 172.21.15.27:0/1435237415 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e0009640 con 0x7f75e80f55c0 2021-11-03T18:27:19.691 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.686+0000 7f75e6ffd700 1 --2- 172.21.15.27:0/1435237415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75e80f55c0 0x7f75e8107060 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 rx=0x7f75e000bb10 tx=0x7f75e0004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:27:19.692 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.687+0000 7f75cf7fe700 1 -- 172.21.15.27:0/1435237415 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75e000e070 con 0x7f75e80f55c0 2021-11-03T18:27:19.692 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.687+0000 7f75ed671700 1 -- 172.21.15.27:0/1435237415 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75e8108580 con 0x7f75e80f55c0 2021-11-03T18:27:19.692 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.687+0000 7f75cf7fe700 1 -- 172.21.15.27:0/1435237415 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f75e001d8d0 con 0x7f75e80f55c0 2021-11-03T18:27:19.692 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.687+0000 7f75cf7fe700 1 -- 172.21.15.27:0/1435237415 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75e0018070 con 0x7f75e80f55c0 2021-11-03T18:27:19.692 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.687+0000 7f75ed671700 1 -- 172.21.15.27:0/1435237415 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75e8108a60 con 0x7f75e80f55c0 2021-11-03T18:27:19.693 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.688+0000 7f75ed671700 1 -- 172.21.15.27:0/1435237415 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75e8045ad0 con 0x7f75e80f55c0 2021-11-03T18:27:19.693 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.688+0000 7f75cf7fe700 1 -- 172.21.15.27:0/1435237415 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f75e0024060 con 0x7f75e80f55c0 2021-11-03T18:27:19.693 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.688+0000 7f75cf7fe700 1 --2- 172.21.15.27:0/1435237415 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f75d0064090 0x7f75d0066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:27:19.693 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.689+0000 7f75cf7fe700 1 -- 172.21.15.27:0/1435237415 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f75e0082140 con 0x7f75e80f55c0 2021-11-03T18:27:19.693 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.689+0000 7f75e67fc700 1 --2- 172.21.15.27:0/1435237415 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f75d0064090 0x7f75d0066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:27:19.694 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.689+0000 7f75e67fc700 1 --2- 172.21.15.27:0/1435237415 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f75d0064090 0x7f75d0066550 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f75dc007fd0 tx=0x7f75dc009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:27:19.694 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.693+0000 7f75cf7fe700 1 -- 172.21.15.27:0/1435237415 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f75e00544a0 con 0x7f75e80f55c0 2021-11-03T18:27:19.866 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.867+0000 7f75ed671700 1 -- 172.21.15.27:0/1435237415 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f75e8109290 con 0x7f75d0064090 2021-11-03T18:27:19.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.882+0000 7f75cf7fe700 1 -- 172.21.15.27:0/1435237415 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f75e8109290 con 0x7f75d0064090 2021-11-03T18:27:19.881 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:27:19.881 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (78m) 7m ago 80m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:27:19.882 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (70m) 7m ago 80m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:27:19.882 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (70m) 7m ago 79m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:27:19.882 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (78m) 7m ago 79m 42.4M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:27:19.882 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (72m) 7m ago 75m 26.5M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:27:19.882 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (75m) 7m ago 75m 3849M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:27:19.883 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (71m) 7m ago 75m 26.1M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:27:19.883 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (72m) 7m ago 75m 27.2M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:27:19.883 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (72m) 7m ago 81m 590M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:27:19.883 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (71m) 7m ago 79m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:27:19.884 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (71m) 7m ago 81m 312M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:27:19.884 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (71m) 7m ago 79m 145M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:27:19.884 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (79m) 7m ago 79m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:27:19.884 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (79m) 7m ago 79m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:27:19.884 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (70m) 7m ago 78m 392M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:27:19.885 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (69m) 7m ago 78m 427M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:27:19.885 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (69m) 7m ago 77m 320M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:27:19.885 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (68m) 7m ago 77m 454M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:27:19.885 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (68m) 7m ago 77m 359M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:27:19.885 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (67m) 7m ago 76m 368M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:27:19.886 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (78m) 7m ago 79m 59.9M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:27:19.886 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.884+0000 7f75ed671700 1 -- 172.21.15.27:0/1435237415 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f75d0064090 msgr2=0x7f75d0066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:27:19.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.884+0000 7f75ed671700 1 --2- 172.21.15.27:0/1435237415 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f75d0064090 0x7f75d0066550 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f75dc007fd0 tx=0x7f75dc009040).stop 2021-11-03T18:27:19.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.884+0000 7f75ed671700 1 -- 172.21.15.27:0/1435237415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75e80f55c0 msgr2=0x7f75e8107060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:27:19.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.884+0000 7f75ed671700 1 --2- 172.21.15.27:0/1435237415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75e80f55c0 0x7f75e8107060 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 rx=0x7f75e000bb10 tx=0x7f75e0004d10).stop 2021-11-03T18:27:19.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.884+0000 7f75ed671700 1 -- 172.21.15.27:0/1435237415 shutdown_connections 2021-11-03T18:27:19.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.884+0000 7f75ed671700 1 --2- 172.21.15.27:0/1435237415 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f75d0064090 0x7f75d0066550 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:19.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.884+0000 7f75ed671700 1 --2- 172.21.15.27:0/1435237415 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75e80f55c0 0x7f75e8107060 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:19.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.884+0000 7f75ed671700 1 --2- 172.21.15.27:0/1435237415 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75e80f6860 0x7f75e8107550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:19.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.884+0000 7f75ed671700 1 -- 172.21.15.27:0/1435237415 >> 172.21.15.27:0/1435237415 conn(0x7f75e80f0d40 msgr2=0x7f75e805fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:27:19.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.884+0000 7f75ed671700 1 -- 172.21.15.27:0/1435237415 shutdown_connections 2021-11-03T18:27:19.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.884+0000 7f75ed671700 1 -- 172.21.15.27:0/1435237415 wait complete. 2021-11-03T18:27:19.982 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.983+0000 7f25c26a2700 1 -- 172.21.15.27:0/1284709652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25bc0f77d0 msgr2=0x7f25bc0f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:27:19.983 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.983+0000 7f25c26a2700 1 --2- 172.21.15.27:0/1284709652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25bc0f77d0 0x7f25bc0f7c30 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 rx=0x7f25ac004660 tx=0x7f25ac009b30).stop 2021-11-03T18:27:19.983 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.984+0000 7f25c26a2700 1 -- 172.21.15.27:0/1284709652 shutdown_connections 2021-11-03T18:27:19.983 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.984+0000 7f25c26a2700 1 --2- 172.21.15.27:0/1284709652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25bc0f77d0 0x7f25bc0f7c30 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:19.983 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.984+0000 7f25c26a2700 1 --2- 172.21.15.27:0/1284709652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25bc0f6530 0x7f25bc0f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:19.984 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.984+0000 7f25c26a2700 1 -- 172.21.15.27:0/1284709652 >> 172.21.15.27:0/1284709652 conn(0x7f25bc0f1cd0 msgr2=0x7f25bc0f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:27:19.984 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.984+0000 7f25c26a2700 1 -- 172.21.15.27:0/1284709652 shutdown_connections 2021-11-03T18:27:19.984 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.984+0000 7f25c26a2700 1 -- 172.21.15.27:0/1284709652 wait complete. 2021-11-03T18:27:19.986 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.984+0000 7f25c26a2700 1 Processor -- start 2021-11-03T18:27:19.987 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.984+0000 7f25c26a2700 1 -- start start 2021-11-03T18:27:19.987 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.984+0000 7f25c26a2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25bc0f6530 0x7f25bc106d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:27:19.987 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.984+0000 7f25c26a2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25bc0f77d0 0x7f25bc107270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:27:19.987 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.984+0000 7f25c26a2700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc107870 con 0x7f25bc0f6530 2021-11-03T18:27:19.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.984+0000 7f25c26a2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc108080 con 0x7f25bc0f77d0 2021-11-03T18:27:19.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.984+0000 7f25c16a0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25bc0f6530 0x7f25bc106d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:27:19.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.984+0000 7f25c16a0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25bc0f6530 0x7f25bc106d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34962/0 (socket says 172.21.15.27:34962) 2021-11-03T18:27:19.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.985+0000 7f25c16a0700 1 -- 172.21.15.27:0/2568548757 learned_addr learned my addr 172.21.15.27:0/2568548757 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:27:19.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.985+0000 7f25c0e9f700 1 --2- 172.21.15.27:0/2568548757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25bc0f77d0 0x7f25bc107270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:27:19.989 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.985+0000 7f25c16a0700 1 -- 172.21.15.27:0/2568548757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25bc0f77d0 msgr2=0x7f25bc107270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:27:19.989 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.985+0000 7f25c16a0700 1 --2- 172.21.15.27:0/2568548757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25bc0f77d0 0x7f25bc107270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:19.989 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.985+0000 7f25c16a0700 1 -- 172.21.15.27:0/2568548757 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25ac005040 con 0x7f25bc0f6530 2021-11-03T18:27:19.989 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.985+0000 7f25c16a0700 1 --2- 172.21.15.27:0/2568548757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25bc0f6530 0x7f25bc106d80 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 rx=0x7f25b8002700 tx=0x7f25b8009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:27:19.990 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.985+0000 7f25b1ffb700 1 -- 172.21.15.27:0/2568548757 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25b80079c0 con 0x7f25bc0f6530 2021-11-03T18:27:19.990 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.985+0000 7f25c26a2700 1 -- 172.21.15.27:0/2568548757 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25bc108300 con 0x7f25bc0f6530 2021-11-03T18:27:19.990 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.985+0000 7f25b1ffb700 1 -- 172.21.15.27:0/2568548757 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f25b800f040 con 0x7f25bc0f6530 2021-11-03T18:27:19.990 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.985+0000 7f25b1ffb700 1 -- 172.21.15.27:0/2568548757 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25b8014860 con 0x7f25bc0f6530 2021-11-03T18:27:19.990 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.985+0000 7f25c26a2700 1 -- 172.21.15.27:0/2568548757 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25bc1088c0 con 0x7f25bc0f6530 2021-11-03T18:27:19.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.986+0000 7f25c26a2700 1 -- 172.21.15.27:0/2568548757 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25bc059100 con 0x7f25bc0f6530 2021-11-03T18:27:19.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.987+0000 7f25b1ffb700 1 -- 172.21.15.27:0/2568548757 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f25b801b060 con 0x7f25bc0f6530 2021-11-03T18:27:19.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.987+0000 7f25b1ffb700 1 --2- 172.21.15.27:0/2568548757 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f25a8064100 0x7f25a80665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:27:19.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.987+0000 7f25b1ffb700 1 -- 172.21.15.27:0/2568548757 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f25b8080670 con 0x7f25bc0f6530 2021-11-03T18:27:19.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.987+0000 7f25c0e9f700 1 --2- 172.21.15.27:0/2568548757 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f25a8064100 0x7f25a80665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:27:19.992 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.987+0000 7f25c0e9f700 1 --2- 172.21.15.27:0/2568548757 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f25a8064100 0x7f25a80665c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f25ac004630 tx=0x7f25ac009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:27:19.992 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:19.991+0000 7f25b1ffb700 1 -- 172.21.15.27:0/2568548757 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f25b80529d0 con 0x7f25bc0f6530 2021-11-03T18:27:20.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.221+0000 7f25c26a2700 1 -- 172.21.15.27:0/2568548757 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f25bc045ad0 con 0x7f25bc0f6530 2021-11-03T18:27:20.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.222+0000 7f25b1ffb700 1 -- 172.21.15.27:0/2568548757 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f25b80192e0 con 0x7f25bc0f6530 2021-11-03T18:27:20.221 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:27:20.222 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:27:20.222 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:27:20.222 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:27:20.222 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:27:20.223 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:27:20.223 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:27:20.223 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:27:20.223 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:27:20.223 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:27:20.224 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:27:20.224 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:27:20.224 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:27:20.224 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:27:20.225 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:27:20.225 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:27:20.225 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:27:20.225 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:27:20.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.224+0000 7f25c26a2700 1 -- 172.21.15.27:0/2568548757 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f25a8064100 msgr2=0x7f25a80665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:27:20.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.224+0000 7f25c26a2700 1 --2- 172.21.15.27:0/2568548757 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f25a8064100 0x7f25a80665c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x7f25ac004630 tx=0x7f25ac009b30).stop 2021-11-03T18:27:20.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.224+0000 7f25c26a2700 1 -- 172.21.15.27:0/2568548757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25bc0f6530 msgr2=0x7f25bc106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:27:20.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.224+0000 7f25c26a2700 1 --2- 172.21.15.27:0/2568548757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25bc0f6530 0x7f25bc106d80 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 rx=0x7f25b8002700 tx=0x7f25b8009d80).stop 2021-11-03T18:27:20.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.224+0000 7f25c26a2700 1 -- 172.21.15.27:0/2568548757 shutdown_connections 2021-11-03T18:27:20.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.224+0000 7f25c26a2700 1 --2- 172.21.15.27:0/2568548757 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f25a8064100 0x7f25a80665c0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:20.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.224+0000 7f25c26a2700 1 --2- 172.21.15.27:0/2568548757 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f25bc0f6530 0x7f25bc106d80 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:20.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.224+0000 7f25c26a2700 1 --2- 172.21.15.27:0/2568548757 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f25bc0f77d0 0x7f25bc107270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:20.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.224+0000 7f25c26a2700 1 -- 172.21.15.27:0/2568548757 >> 172.21.15.27:0/2568548757 conn(0x7f25bc0f1cd0 msgr2=0x7f25bc0fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:27:20.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.224+0000 7f25c26a2700 1 -- 172.21.15.27:0/2568548757 shutdown_connections 2021-11-03T18:27:20.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.224+0000 7f25c26a2700 1 -- 172.21.15.27:0/2568548757 wait complete. 2021-11-03T18:27:20.324 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.324+0000 7f73cf3c0700 1 -- 172.21.15.27:0/515944111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73c80f6860 msgr2=0x7f73c80f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:27:20.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.324+0000 7f73cf3c0700 1 --2- 172.21.15.27:0/515944111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73c80f6860 0x7f73c80f6cc0 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 rx=0x7f73b8004660 tx=0x7f73b8009b30).stop 2021-11-03T18:27:20.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.324+0000 7f73cf3c0700 1 -- 172.21.15.27:0/515944111 shutdown_connections 2021-11-03T18:27:20.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.324+0000 7f73cf3c0700 1 --2- 172.21.15.27:0/515944111 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73c80f6860 0x7f73c80f6cc0 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:20.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.324+0000 7f73cf3c0700 1 --2- 172.21.15.27:0/515944111 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73c80f55c0 0x7f73c80f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:20.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.324+0000 7f73cf3c0700 1 -- 172.21.15.27:0/515944111 >> 172.21.15.27:0/515944111 conn(0x7f73c80f0d40 msgr2=0x7f73c80f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:27:20.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.324+0000 7f73cf3c0700 1 -- 172.21.15.27:0/515944111 shutdown_connections 2021-11-03T18:27:20.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.324+0000 7f73cf3c0700 1 -- 172.21.15.27:0/515944111 wait complete. 2021-11-03T18:27:20.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.324+0000 7f73cf3c0700 1 Processor -- start 2021-11-03T18:27:20.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.324+0000 7f73cf3c0700 1 -- start start 2021-11-03T18:27:20.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.325+0000 7f73cf3c0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73c80f55c0 0x7f73c8105ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:27:20.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.325+0000 7f73cf3c0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73c80f6860 0x7f73c81063b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:27:20.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.325+0000 7f73cf3c0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c81069b0 con 0x7f73c80f55c0 2021-11-03T18:27:20.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.325+0000 7f73cf3c0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73c81071c0 con 0x7f73c80f6860 2021-11-03T18:27:20.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.325+0000 7f73cd15c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73c80f55c0 0x7f73c8105ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:27:20.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.325+0000 7f73cd15c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73c80f55c0 0x7f73c8105ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34972/0 (socket says 172.21.15.27:34972) 2021-11-03T18:27:20.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.325+0000 7f73cd15c700 1 -- 172.21.15.27:0/3684167861 learned_addr learned my addr 172.21.15.27:0/3684167861 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:27:20.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.325+0000 7f73cc95b700 1 --2- 172.21.15.27:0/3684167861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73c80f6860 0x7f73c81063b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:27:20.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.325+0000 7f73cd15c700 1 -- 172.21.15.27:0/3684167861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73c80f6860 msgr2=0x7f73c81063b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:27:20.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.325+0000 7f73cd15c700 1 --2- 172.21.15.27:0/3684167861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73c80f6860 0x7f73c81063b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:20.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.325+0000 7f73cd15c700 1 -- 172.21.15.27:0/3684167861 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b8005040 con 0x7f73c80f55c0 2021-11-03T18:27:20.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.325+0000 7f73cd15c700 1 --2- 172.21.15.27:0/3684167861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73c80f55c0 0x7f73c8105ec0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 rx=0x7f73c4002700 tx=0x7f73c4009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:27:20.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.325+0000 7f73bdffb700 1 -- 172.21.15.27:0/3684167861 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73c40079c0 con 0x7f73c80f55c0 2021-11-03T18:27:20.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.325+0000 7f73bdffb700 1 -- 172.21.15.27:0/3684167861 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f73c400f040 con 0x7f73c80f55c0 2021-11-03T18:27:20.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.325+0000 7f73bdffb700 1 -- 172.21.15.27:0/3684167861 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f73c4014860 con 0x7f73c80f55c0 2021-11-03T18:27:20.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.325+0000 7f73cf3c0700 1 -- 172.21.15.27:0/3684167861 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73c8107440 con 0x7f73c80f55c0 2021-11-03T18:27:20.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.326+0000 7f73cf3c0700 1 -- 172.21.15.27:0/3684167861 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73c8107a00 con 0x7f73c80f55c0 2021-11-03T18:27:20.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.326+0000 7f73cf3c0700 1 -- 172.21.15.27:0/3684167861 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73c80f49b0 con 0x7f73c80f55c0 2021-11-03T18:27:20.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.327+0000 7f73bdffb700 1 -- 172.21.15.27:0/3684167861 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f73c401b060 con 0x7f73c80f55c0 2021-11-03T18:27:20.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.327+0000 7f73bdffb700 1 --2- 172.21.15.27:0/3684167861 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f73b40640b0 0x7f73b4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:27:20.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.327+0000 7f73bdffb700 1 -- 172.21.15.27:0/3684167861 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f73c4080660 con 0x7f73c80f55c0 2021-11-03T18:27:20.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.327+0000 7f73cc95b700 1 --2- 172.21.15.27:0/3684167861 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f73b40640b0 0x7f73b4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:27:20.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.328+0000 7f73cc95b700 1 --2- 172.21.15.27:0/3684167861 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f73b40640b0 0x7f73b4066570 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f73b8004630 tx=0x7f73b8009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:27:20.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.331+0000 7f73bdffb700 1 -- 172.21.15.27:0/3684167861 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f73c4052900 con 0x7f73c80f55c0 2021-11-03T18:27:20.539 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.540+0000 7f73cf3c0700 1 -- 172.21.15.27:0/3684167861 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f73c8045ad0 con 0x7f73c80f55c0 2021-11-03T18:27:20.539 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.540+0000 7f73bdffb700 1 -- 172.21.15.27:0/3684167861 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f73c4019070 con 0x7f73c80f55c0 2021-11-03T18:27:20.540 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:27:20.540 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:27:20.540 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:27:20.541 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:27:20.541 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:27:20.541 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:27:20.541 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:27:20.542 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:27:20.542 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:27:20.542 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:27:20.542 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:27:20.542 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:27:20.543 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:27:20.543 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:27:20.543 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:27:20.543 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:27:20.544 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:27:20.544 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:27:20.544 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:27:20.544 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:27:20.544 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:27:20.545 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:27:20.545 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:27:20.545 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:27:20.545 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:27:20.546 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:27:20.546 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:27:20.546 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:27:20.546 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:27:20.546 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:27:20.547 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:27:20.547 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:27:20.547 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:27:20.547 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:27:20.548 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:27:20.548 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:27:20.548 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:27:20.548 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:27:20.548 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:27:20.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.542+0000 7f73cf3c0700 1 -- 172.21.15.27:0/3684167861 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f73b40640b0 msgr2=0x7f73b4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:27:20.550 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.542+0000 7f73cf3c0700 1 --2- 172.21.15.27:0/3684167861 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f73b40640b0 0x7f73b4066570 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f73b8004630 tx=0x7f73b8009b30).stop 2021-11-03T18:27:20.550 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.542+0000 7f73cf3c0700 1 -- 172.21.15.27:0/3684167861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73c80f55c0 msgr2=0x7f73c8105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:27:20.550 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.542+0000 7f73cf3c0700 1 --2- 172.21.15.27:0/3684167861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73c80f55c0 0x7f73c8105ec0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 rx=0x7f73c4002700 tx=0x7f73c4009d80).stop 2021-11-03T18:27:20.550 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.542+0000 7f73cf3c0700 1 -- 172.21.15.27:0/3684167861 shutdown_connections 2021-11-03T18:27:20.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.542+0000 7f73cf3c0700 1 --2- 172.21.15.27:0/3684167861 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f73b40640b0 0x7f73b4066570 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:20.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.542+0000 7f73cf3c0700 1 --2- 172.21.15.27:0/3684167861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73c80f55c0 0x7f73c8105ec0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:20.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.542+0000 7f73cf3c0700 1 --2- 172.21.15.27:0/3684167861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73c80f6860 0x7f73c81063b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:20.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.542+0000 7f73cf3c0700 1 -- 172.21.15.27:0/3684167861 >> 172.21.15.27:0/3684167861 conn(0x7f73c80f0d40 msgr2=0x7f73c80fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:27:20.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.543+0000 7f73cf3c0700 1 -- 172.21.15.27:0/3684167861 shutdown_connections 2021-11-03T18:27:20.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:20.543+0000 7f73cf3c0700 1 -- 172.21.15.27:0/3684167861 wait complete. 2021-11-03T18:27:20.552 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:27:20.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:20 smithi115 ceph-mon[65453]: pgmap v2182: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:20.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4282.462513 secs 2021-11-03T18:27:20.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:20 smithi115 ceph-mon[65453]: from='client.36344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:27:20.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:20 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2568548757' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:20.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:20 smithi027 ceph-mon[97244]: pgmap v2182: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:20.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4282.462513 secs 2021-11-03T18:27:20.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:20 smithi027 ceph-mon[97244]: from='client.36344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:27:20.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:20 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2568548757' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:21.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:21 smithi115 ceph-mon[65453]: from='client.36348 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:27:21.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:21 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3684167861' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:27:21.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:21 smithi027 ceph-mon[97244]: from='client.36348 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:27:21.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:21 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3684167861' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:27:22.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:22 smithi115 ceph-mon[65453]: pgmap v2183: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:22.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:22 smithi027 ceph-mon[97244]: pgmap v2183: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:24.590 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:24 smithi027 ceph-mon[97244]: pgmap v2184: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:24.590 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:24 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:27:24.627 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:24 smithi115 ceph-mon[65453]: pgmap v2184: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:24.627 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:24 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:27:25.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:25 smithi115 ceph-mon[65453]: poll took 60180.703 ms: 1 events 2021-11-03T18:27:25.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4287.462583 secs 2021-11-03T18:27:25.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:27:25.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:27:25.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:27:25.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:25.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:25.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:25.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:26.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:25 smithi027 ceph-mon[97244]: poll took 60180.703 ms: 1 events 2021-11-03T18:27:26.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4287.462583 secs 2021-11-03T18:27:26.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:27:26.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:27:26.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:27:26.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:26.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:26.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:26.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:26.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:26 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:27:26.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:26 smithi115 ceph-mon[65453]: pgmap v2185: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:27.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:26 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:27:27.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:26 smithi027 ceph-mon[97244]: pgmap v2185: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:28.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:28 smithi115 ceph-mon[65453]: pgmap v2186: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:29.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:28 smithi027 ceph-mon[97244]: pgmap v2186: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:30.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:30 smithi115 ceph-mon[65453]: pgmap v2187: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:30.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4292.462647 secs 2021-11-03T18:27:31.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:30 smithi027 ceph-mon[97244]: pgmap v2187: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:31.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4292.462647 secs 2021-11-03T18:27:32.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:32 smithi027 ceph-mon[97244]: pgmap v2188: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:32.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:32 smithi115 ceph-mon[65453]: pgmap v2188: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:34.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:34 smithi115 ceph-mon[65453]: pgmap v2189: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:35.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:34 smithi027 ceph-mon[97244]: pgmap v2189: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:35.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4297.462724 secs 2021-11-03T18:27:35.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:27:35.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:27:35.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:27:35.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:35.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:35.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:35.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:36.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4297.462724 secs 2021-11-03T18:27:36.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:27:36.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:27:36.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:27:36.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:36.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:36.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:36.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:36.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:36 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:27:36.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:36 smithi115 ceph-mon[65453]: pgmap v2190: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:37.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:36 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:27:37.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:36 smithi027 ceph-mon[97244]: pgmap v2190: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:38.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:38 smithi115 ceph-mon[65453]: pgmap v2191: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:39.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:38 smithi027 ceph-mon[97244]: pgmap v2191: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:40.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:40 smithi115 ceph-mon[65453]: pgmap v2192: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:40.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4302.462788 secs 2021-11-03T18:27:41.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:40 smithi027 ceph-mon[97244]: pgmap v2192: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:41.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4302.462788 secs 2021-11-03T18:27:42.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:42 smithi027 ceph-mon[97244]: pgmap v2193: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:42.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:42 smithi115 ceph-mon[65453]: pgmap v2193: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:44.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:44 smithi115 ceph-mon[65453]: pgmap v2194: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:45.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:44 smithi027 ceph-mon[97244]: pgmap v2194: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:45.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4307.462903 secs 2021-11-03T18:27:45.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:27:45.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:27:45.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:27:45.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:45.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:45.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:45.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:46.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4307.462903 secs 2021-11-03T18:27:46.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:27:46.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:27:46.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:27:46.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:46.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:46.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:46.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:46.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:46 smithi115 ceph-mon[65453]: pgmap v2195: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:46.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:46 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:27:47.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:46 smithi027 ceph-mon[97244]: pgmap v2195: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:47.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:46 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:27:48.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:48 smithi115 ceph-mon[65453]: pgmap v2196: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:49.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:48 smithi027 ceph-mon[97244]: pgmap v2196: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:50.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.658+0000 7f73782c8700 1 -- 172.21.15.27:0/1714840575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7370060310 msgr2=0x7f73700fb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:27:50.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.659+0000 7f73782c8700 1 --2- 172.21.15.27:0/1714840575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7370060310 0x7f73700fb6a0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 rx=0x7f736c004660 tx=0x7f736c009b30).stop 2021-11-03T18:27:50.661 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.659+0000 7f73782c8700 1 -- 172.21.15.27:0/1714840575 shutdown_connections 2021-11-03T18:27:50.661 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.659+0000 7f73782c8700 1 --2- 172.21.15.27:0/1714840575 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f73700fbb90 0x7f73700fdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T18:27:50.661 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.659+0000 7f73782c8700 1 --2- 172.21.15.27:0/1714840575 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7370060310 0x7f73700fb6a0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:50.661 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.659+0000 7f73782c8700 1 -- 172.21.15.27:0/1714840575 >> 172.21.15.27:0/1714840575 conn(0x7f73700f0d40 msgr2=0x7f73700f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:27:50.662 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.659+0000 7f73782c8700 1 -- 172.21.15.27:0/1714840575 shutdown_connections 2021-11-03T18:27:50.662 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.659+0000 7f73782c8700 1 -- 172.21.15.27:0/1714840575 wait complete. 2021-11-03T18:27:50.662 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.659+0000 7f73782c8700 1 Processor -- start 2021-11-03T18:27:50.662 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.659+0000 7f73782c8700 1 -- start start 2021-11-03T18:27:50.662 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.659+0000 7f73782c8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7370060310 0x7f7370109170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:27:50.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.659+0000 7f73782c8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73700fbb90 0x7f7370109660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:27:50.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.659+0000 7f73782c8700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7370109ca0 con 0x7f73700fbb90 2021-11-03T18:27:50.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.659+0000 7f73782c8700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f737010a550 con 0x7f7370060310 2021-11-03T18:27:50.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.660+0000 7f7375863700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73700fbb90 0x7f7370109660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:27:50.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.660+0000 7f7375863700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73700fbb90 0x7f7370109660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:34992/0 (socket says 172.21.15.27:34992) 2021-11-03T18:27:50.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.660+0000 7f7375863700 1 -- 172.21.15.27:0/1829465391 learned_addr learned my addr 172.21.15.27:0/1829465391 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:27:50.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.660+0000 7f7376064700 1 --2- 172.21.15.27:0/1829465391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7370060310 0x7f7370109170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:27:50.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.660+0000 7f7375863700 1 -- 172.21.15.27:0/1829465391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7370060310 msgr2=0x7f7370109170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:27:50.665 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.660+0000 7f7375863700 1 --2- 172.21.15.27:0/1829465391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7370060310 0x7f7370109170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:50.665 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.660+0000 7f7375863700 1 -- 172.21.15.27:0/1829465391 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f736c005040 con 0x7f73700fbb90 2021-11-03T18:27:50.665 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.660+0000 7f7375863700 1 --2- 172.21.15.27:0/1829465391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73700fbb90 0x7f7370109660 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 rx=0x7f73600093f0 tx=0x7f7360006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:27:50.665 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.660+0000 7f7366ffd700 1 -- 172.21.15.27:0/1829465391 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7360013640 con 0x7f73700fbb90 2021-11-03T18:27:50.665 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.660+0000 7f73782c8700 1 -- 172.21.15.27:0/1829465391 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f737010a830 con 0x7f73700fbb90 2021-11-03T18:27:50.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.660+0000 7f7366ffd700 1 -- 172.21.15.27:0/1829465391 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f7360013c50 con 0x7f73700fbb90 2021-11-03T18:27:50.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.660+0000 7f7366ffd700 1 -- 172.21.15.27:0/1829465391 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7360012c70 con 0x7f73700fbb90 2021-11-03T18:27:50.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.660+0000 7f73782c8700 1 -- 172.21.15.27:0/1829465391 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f737010adf0 con 0x7f73700fbb90 2021-11-03T18:27:50.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.661+0000 7f73782c8700 1 -- 172.21.15.27:0/1829465391 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73700f49b0 con 0x7f73700fbb90 2021-11-03T18:27:50.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.662+0000 7f7366ffd700 1 -- 172.21.15.27:0/1829465391 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f736001a060 con 0x7f73700fbb90 2021-11-03T18:27:50.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.662+0000 7f7366ffd700 1 --2- 172.21.15.27:0/1829465391 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f735c0640b0 0x7f735c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:27:50.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.662+0000 7f7366ffd700 1 -- 172.21.15.27:0/1829465391 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f73600808d0 con 0x7f73700fbb90 2021-11-03T18:27:50.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.662+0000 7f7376064700 1 --2- 172.21.15.27:0/1829465391 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f735c0640b0 0x7f735c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:27:50.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.663+0000 7f7376064700 1 --2- 172.21.15.27:0/1829465391 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f735c0640b0 0x7f735c066570 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7f736c004630 tx=0x7f736c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:27:50.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.666+0000 7f7366ffd700 1 -- 172.21.15.27:0/1829465391 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f7360052ae0 con 0x7f73700fbb90 2021-11-03T18:27:50.846 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.847+0000 7f73782c8700 1 -- 172.21.15.27:0/1829465391 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f73701011a0 con 0x7f735c0640b0 2021-11-03T18:27:50.847 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.848+0000 7f7366ffd700 1 -- 172.21.15.27:0/1829465391 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f73701011a0 con 0x7f735c0640b0 2021-11-03T18:27:50.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.850+0000 7f73782c8700 1 -- 172.21.15.27:0/1829465391 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f735c0640b0 msgr2=0x7f735c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:27:50.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.850+0000 7f73782c8700 1 --2- 172.21.15.27:0/1829465391 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f735c0640b0 0x7f735c066570 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7f736c004630 tx=0x7f736c009b30).stop 2021-11-03T18:27:50.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.850+0000 7f73782c8700 1 -- 172.21.15.27:0/1829465391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73700fbb90 msgr2=0x7f7370109660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:27:50.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.850+0000 7f73782c8700 1 --2- 172.21.15.27:0/1829465391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73700fbb90 0x7f7370109660 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 rx=0x7f73600093f0 tx=0x7f7360006ec0).stop 2021-11-03T18:27:50.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.850+0000 7f73782c8700 1 -- 172.21.15.27:0/1829465391 shutdown_connections 2021-11-03T18:27:50.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.850+0000 7f73782c8700 1 --2- 172.21.15.27:0/1829465391 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f735c0640b0 0x7f735c066570 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:50.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.850+0000 7f73782c8700 1 --2- 172.21.15.27:0/1829465391 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7370060310 0x7f7370109170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:50.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.850+0000 7f73782c8700 1 --2- 172.21.15.27:0/1829465391 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f73700fbb90 0x7f7370109660 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:50.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.850+0000 7f73782c8700 1 -- 172.21.15.27:0/1829465391 >> 172.21.15.27:0/1829465391 conn(0x7f73700f0d40 msgr2=0x7f73700f3130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:27:50.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.851+0000 7f73782c8700 1 -- 172.21.15.27:0/1829465391 shutdown_connections 2021-11-03T18:27:50.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.851+0000 7f73782c8700 1 -- 172.21.15.27:0/1829465391 wait complete. 2021-11-03T18:27:50.859 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:27:50.951 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.950+0000 7f18873c0700 1 -- 172.21.15.27:0/1093677055 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18800f5240 msgr2=0x7f18800f7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:27:50.952 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.950+0000 7f18873c0700 1 --2- 172.21.15.27:0/1093677055 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18800f5240 0x7f18800f7b80 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 rx=0x7f1870004660 tx=0x7f1870009b30).stop 2021-11-03T18:27:50.952 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.950+0000 7f18873c0700 1 -- 172.21.15.27:0/1093677055 shutdown_connections 2021-11-03T18:27:50.952 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.950+0000 7f18873c0700 1 --2- 172.21.15.27:0/1093677055 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18800f5240 0x7f18800f7b80 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:50.952 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.950+0000 7f18873c0700 1 --2- 172.21.15.27:0/1093677055 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18800f48a0 0x7f18800f4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T18:27:50.953 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.950+0000 7f18873c0700 1 -- 172.21.15.27:0/1093677055 >> 172.21.15.27:0/1093677055 conn(0x7f18800f0320 msgr2=0x7f18800f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:27:50.953 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.951+0000 7f18873c0700 1 -- 172.21.15.27:0/1093677055 shutdown_connections 2021-11-03T18:27:50.953 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.951+0000 7f18873c0700 1 -- 172.21.15.27:0/1093677055 wait complete. 2021-11-03T18:27:50.953 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.951+0000 7f18873c0700 1 Processor -- start 2021-11-03T18:27:50.954 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.951+0000 7f18873c0700 1 -- start start 2021-11-03T18:27:50.954 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.951+0000 7f18873c0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18800f48a0 0x7f18800f7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:27:50.954 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.951+0000 7f18873c0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18800f5240 0x7f18800f5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:27:50.954 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.951+0000 7f18873c0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18800f5f00 con 0x7f18800f48a0 2021-11-03T18:27:50.955 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.951+0000 7f18873c0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18800f6040 con 0x7f18800f5240 2021-11-03T18:27:50.955 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.951+0000 7f18863be700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18800f48a0 0x7f18800f7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:27:50.955 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.951+0000 7f18863be700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18800f48a0 0x7f18800f7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35000/0 (socket says 172.21.15.27:35000) 2021-11-03T18:27:50.955 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.951+0000 7f18863be700 1 -- 172.21.15.27:0/1546493549 learned_addr learned my addr 172.21.15.27:0/1546493549 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:27:50.955 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.952+0000 7f1885bbd700 1 --2- 172.21.15.27:0/1546493549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18800f5240 0x7f18800f5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:27:50.956 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.952+0000 7f18863be700 1 -- 172.21.15.27:0/1546493549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18800f5240 msgr2=0x7f18800f5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:27:50.956 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.952+0000 7f18863be700 1 --2- 172.21.15.27:0/1546493549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18800f5240 0x7f18800f5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:50.956 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.952+0000 7f18863be700 1 -- 172.21.15.27:0/1546493549 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1870005040 con 0x7f18800f48a0 2021-11-03T18:27:50.956 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.952+0000 7f18863be700 1 --2- 172.21.15.27:0/1546493549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18800f48a0 0x7f18800f7490 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 rx=0x7f187c00b410 tx=0x7f187c006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:27:50.957 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.952+0000 7f1876ffd700 1 -- 172.21.15.27:0/1546493549 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f187c0125f0 con 0x7f18800f48a0 2021-11-03T18:27:50.957 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.952+0000 7f18873c0700 1 -- 172.21.15.27:0/1546493549 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18800f6320 con 0x7f18800f48a0 2021-11-03T18:27:50.957 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.952+0000 7f1876ffd700 1 -- 172.21.15.27:0/1546493549 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f187c012c00 con 0x7f18800f48a0 2021-11-03T18:27:50.957 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.952+0000 7f1876ffd700 1 -- 172.21.15.27:0/1546493549 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f187c002ae0 con 0x7f18800f48a0 2021-11-03T18:27:50.958 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.952+0000 7f18873c0700 1 -- 172.21.15.27:0/1546493549 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f188010efb0 con 0x7f18800f48a0 2021-11-03T18:27:50.959 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.953+0000 7f18873c0700 1 -- 172.21.15.27:0/1546493549 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1880045ad0 con 0x7f18800f48a0 2021-11-03T18:27:50.959 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.953+0000 7f1876ffd700 1 -- 172.21.15.27:0/1546493549 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f187c00f030 con 0x7f18800f48a0 2021-11-03T18:27:50.959 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.954+0000 7f1876ffd700 1 --2- 172.21.15.27:0/1546493549 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f186c064100 0x7f186c0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:27:50.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.954+0000 7f1876ffd700 1 -- 172.21.15.27:0/1546493549 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f187c080c90 con 0x7f18800f48a0 2021-11-03T18:27:50.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.954+0000 7f1885bbd700 1 --2- 172.21.15.27:0/1546493549 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f186c064100 0x7f186c0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:27:50.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.954+0000 7f1885bbd700 1 --2- 172.21.15.27:0/1546493549 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f186c064100 0x7f186c0665c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f1870004e30 tx=0x7f1870009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:27:50.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:50.958+0000 7f1876ffd700 1 -- 172.21.15.27:0/1546493549 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f187c052f30 con 0x7f18800f48a0 2021-11-03T18:27:50.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:50 smithi115 ceph-mon[65453]: pgmap v2197: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:50.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4312.463058 secs 2021-11-03T18:27:51.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:50 smithi027 ceph-mon[97244]: pgmap v2197: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:51.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4312.463058 secs 2021-11-03T18:27:51.131 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.132+0000 7f18873c0700 1 -- 172.21.15.27:0/1546493549 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f18800fefd0 con 0x7f186c064100 2021-11-03T18:27:51.145 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.147+0000 7f1876ffd700 1 -- 172.21.15.27:0/1546493549 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f18800fefd0 con 0x7f186c064100 2021-11-03T18:27:51.146 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:27:51.146 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (79m) 8m ago 80m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:27:51.147 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (71m) 8m ago 80m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:27:51.147 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (70m) 7m ago 79m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:27:51.147 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (79m) 8m ago 80m 42.4M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:27:51.147 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (72m) 8m ago 75m 26.5M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:27:51.147 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (75m) 8m ago 75m 3849M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:27:51.148 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (71m) 7m ago 75m 26.1M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:27:51.148 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (73m) 7m ago 75m 27.2M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:27:51.148 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (73m) 8m ago 81m 590M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:27:51.148 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (72m) 7m ago 79m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:27:51.149 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (72m) 8m ago 82m 312M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:27:51.149 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (71m) 7m ago 79m 145M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:27:51.149 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (80m) 8m ago 80m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:27:51.149 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (79m) 7m ago 79m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:27:51.149 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (70m) 8m ago 79m 392M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:27:51.150 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (70m) 8m ago 78m 427M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:27:51.150 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (69m) 8m ago 78m 320M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:27:51.150 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (69m) 7m ago 78m 454M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:27:51.150 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (68m) 7m ago 77m 359M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:27:51.151 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (68m) 7m ago 77m 368M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:27:51.151 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (79m) 8m ago 80m 59.9M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:27:51.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.148+0000 7f18873c0700 1 -- 172.21.15.27:0/1546493549 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f186c064100 msgr2=0x7f186c0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:27:51.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.148+0000 7f18873c0700 1 --2- 172.21.15.27:0/1546493549 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f186c064100 0x7f186c0665c0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7f1870004e30 tx=0x7f1870009b30).stop 2021-11-03T18:27:51.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.148+0000 7f18873c0700 1 -- 172.21.15.27:0/1546493549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18800f48a0 msgr2=0x7f18800f7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:27:51.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.148+0000 7f18873c0700 1 --2- 172.21.15.27:0/1546493549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18800f48a0 0x7f18800f7490 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 rx=0x7f187c00b410 tx=0x7f187c006ef0).stop 2021-11-03T18:27:51.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.149+0000 7f18873c0700 1 -- 172.21.15.27:0/1546493549 shutdown_connections 2021-11-03T18:27:51.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.149+0000 7f18873c0700 1 --2- 172.21.15.27:0/1546493549 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f186c064100 0x7f186c0665c0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:51.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.149+0000 7f18873c0700 1 --2- 172.21.15.27:0/1546493549 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18800f48a0 0x7f18800f7490 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:51.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.149+0000 7f18873c0700 1 --2- 172.21.15.27:0/1546493549 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18800f5240 0x7f18800f5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:51.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.149+0000 7f18873c0700 1 -- 172.21.15.27:0/1546493549 >> 172.21.15.27:0/1546493549 conn(0x7f18800f0320 msgr2=0x7f18800fb5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:27:51.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.149+0000 7f18873c0700 1 -- 172.21.15.27:0/1546493549 shutdown_connections 2021-11-03T18:27:51.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.149+0000 7f18873c0700 1 -- 172.21.15.27:0/1546493549 wait complete. 2021-11-03T18:27:51.249 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.248+0000 7fcd6c4a5700 1 -- 172.21.15.27:0/1913569206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd640f7960 msgr2=0x7fcd640f7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:27:51.249 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.248+0000 7fcd6c4a5700 1 --2- 172.21.15.27:0/1913569206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd640f7960 0x7fcd640f7dc0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 rx=0x7fcd58004660 tx=0x7fcd58009b30).stop 2021-11-03T18:27:51.249 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.248+0000 7fcd6c4a5700 1 -- 172.21.15.27:0/1913569206 shutdown_connections 2021-11-03T18:27:51.250 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.248+0000 7fcd6c4a5700 1 --2- 172.21.15.27:0/1913569206 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd640f7960 0x7fcd640f7dc0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:51.250 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.248+0000 7fcd6c4a5700 1 --2- 172.21.15.27:0/1913569206 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd640f66c0 0x7fcd640f6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:51.250 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.248+0000 7fcd6c4a5700 1 -- 172.21.15.27:0/1913569206 >> 172.21.15.27:0/1913569206 conn(0x7fcd640f1e20 msgr2=0x7fcd640f4240 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:27:51.250 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.248+0000 7fcd6c4a5700 1 -- 172.21.15.27:0/1913569206 shutdown_connections 2021-11-03T18:27:51.251 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.248+0000 7fcd6c4a5700 1 -- 172.21.15.27:0/1913569206 wait complete. 2021-11-03T18:27:51.251 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.248+0000 7fcd6c4a5700 1 Processor -- start 2021-11-03T18:27:51.251 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.248+0000 7fcd6c4a5700 1 -- start start 2021-11-03T18:27:51.251 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.249+0000 7fcd6c4a5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd640f66c0 0x7fcd64106f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:27:51.252 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.249+0000 7fcd6c4a5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd640f7960 0x7fcd64107440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:27:51.252 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.249+0000 7fcd6c4a5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd641079f0 con 0x7fcd640f66c0 2021-11-03T18:27:51.252 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.249+0000 7fcd6c4a5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd64107b30 con 0x7fcd640f7960 2021-11-03T18:27:51.252 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.249+0000 7fcd6a241700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd640f66c0 0x7fcd64106f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:27:51.252 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.249+0000 7fcd6a241700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd640f66c0 0x7fcd64106f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35010/0 (socket says 172.21.15.27:35010) 2021-11-03T18:27:51.253 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.249+0000 7fcd6a241700 1 -- 172.21.15.27:0/603474530 learned_addr learned my addr 172.21.15.27:0/603474530 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:27:51.253 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.249+0000 7fcd69a40700 1 --2- 172.21.15.27:0/603474530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd640f7960 0x7fcd64107440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:27:51.253 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.249+0000 7fcd6a241700 1 -- 172.21.15.27:0/603474530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd640f7960 msgr2=0x7fcd64107440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:27:51.253 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.249+0000 7fcd6a241700 1 --2- 172.21.15.27:0/603474530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd640f7960 0x7fcd64107440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:51.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.249+0000 7fcd6a241700 1 -- 172.21.15.27:0/603474530 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd58005040 con 0x7fcd640f66c0 2021-11-03T18:27:51.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.249+0000 7fcd69a40700 1 --2- 172.21.15.27:0/603474530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd640f7960 0x7fcd64107440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:27:51.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.249+0000 7fcd6a241700 1 --2- 172.21.15.27:0/603474530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd640f66c0 0x7fcd64106f50 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 rx=0x7fcd5c00c530 tx=0x7fcd5c007e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:27:51.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.249+0000 7fcd62ffd700 1 -- 172.21.15.27:0/603474530 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd5c00f650 con 0x7fcd640f66c0 2021-11-03T18:27:51.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.249+0000 7fcd62ffd700 1 -- 172.21.15.27:0/603474530 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fcd5c00fc60 con 0x7fcd640f66c0 2021-11-03T18:27:51.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.249+0000 7fcd6c4a5700 1 -- 172.21.15.27:0/603474530 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd641084d0 con 0x7fcd640f66c0 2021-11-03T18:27:51.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.249+0000 7fcd62ffd700 1 -- 172.21.15.27:0/603474530 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd5c012d20 con 0x7fcd640f66c0 2021-11-03T18:27:51.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.250+0000 7fcd6c4a5700 1 -- 172.21.15.27:0/603474530 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd64108a90 con 0x7fcd640f66c0 2021-11-03T18:27:51.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.250+0000 7fcd6c4a5700 1 -- 172.21.15.27:0/603474530 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd64045ad0 con 0x7fcd640f66c0 2021-11-03T18:27:51.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.251+0000 7fcd62ffd700 1 -- 172.21.15.27:0/603474530 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fcd5c019030 con 0x7fcd640f66c0 2021-11-03T18:27:51.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.251+0000 7fcd62ffd700 1 --2- 172.21.15.27:0/603474530 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcd4c0640b0 0x7fcd4c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:27:51.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.251+0000 7fcd62ffd700 1 -- 172.21.15.27:0/603474530 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fcd5c080630 con 0x7fcd640f66c0 2021-11-03T18:27:51.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.251+0000 7fcd69a40700 1 --2- 172.21.15.27:0/603474530 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcd4c0640b0 0x7fcd4c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:27:51.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.252+0000 7fcd69a40700 1 --2- 172.21.15.27:0/603474530 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcd4c0640b0 0x7fcd4c066570 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7fcd58004e30 tx=0x7fcd58009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:27:51.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.255+0000 7fcd62ffd700 1 -- 172.21.15.27:0/603474530 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fcd5c052830 con 0x7fcd640f66c0 2021-11-03T18:27:51.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.484+0000 7fcd6c4a5700 1 -- 172.21.15.27:0/603474530 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcd64108660 con 0x7fcd640f66c0 2021-11-03T18:27:51.484 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.484+0000 7fcd62ffd700 1 -- 172.21.15.27:0/603474530 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fcd64108660 con 0x7fcd640f66c0 2021-11-03T18:27:51.484 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:27:51.485 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:27:51.485 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:27:51.485 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:27:51.485 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:27:51.485 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:27:51.486 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:27:51.486 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:27:51.486 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:27:51.486 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:27:51.487 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:27:51.487 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:27:51.487 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:27:51.487 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:27:51.488 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:27:51.488 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:27:51.488 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:27:51.488 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:27:51.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.486+0000 7fcd6c4a5700 1 -- 172.21.15.27:0/603474530 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcd4c0640b0 msgr2=0x7fcd4c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:27:51.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.486+0000 7fcd6c4a5700 1 --2- 172.21.15.27:0/603474530 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcd4c0640b0 0x7fcd4c066570 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7fcd58004e30 tx=0x7fcd58009b30).stop 2021-11-03T18:27:51.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.486+0000 7fcd6c4a5700 1 -- 172.21.15.27:0/603474530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd640f66c0 msgr2=0x7fcd64106f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:27:51.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.486+0000 7fcd6c4a5700 1 --2- 172.21.15.27:0/603474530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd640f66c0 0x7fcd64106f50 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 rx=0x7fcd5c00c530 tx=0x7fcd5c007e10).stop 2021-11-03T18:27:51.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.486+0000 7fcd6c4a5700 1 -- 172.21.15.27:0/603474530 shutdown_connections 2021-11-03T18:27:51.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.487+0000 7fcd6c4a5700 1 --2- 172.21.15.27:0/603474530 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcd4c0640b0 0x7fcd4c066570 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:51.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.487+0000 7fcd6c4a5700 1 --2- 172.21.15.27:0/603474530 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd640f66c0 0x7fcd64106f50 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:51.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.487+0000 7fcd6c4a5700 1 --2- 172.21.15.27:0/603474530 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd640f7960 0x7fcd64107440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:51.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.487+0000 7fcd6c4a5700 1 -- 172.21.15.27:0/603474530 >> 172.21.15.27:0/603474530 conn(0x7fcd640f1e20 msgr2=0x7fcd640fb3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:27:51.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.487+0000 7fcd6c4a5700 1 -- 172.21.15.27:0/603474530 shutdown_connections 2021-11-03T18:27:51.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.487+0000 7fcd6c4a5700 1 -- 172.21.15.27:0/603474530 wait complete. 2021-11-03T18:27:51.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.586+0000 7fda9759e700 1 -- 172.21.15.27:0/1364057828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda980f5710 msgr2=0x7fda980f5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:27:51.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.586+0000 7fda9759e700 1 --2- 172.21.15.27:0/1364057828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda980f5710 0x7fda980f5b90 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 rx=0x7fda88004660 tx=0x7fda88009b30).stop 2021-11-03T18:27:51.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.586+0000 7fda9759e700 1 -- 172.21.15.27:0/1364057828 shutdown_connections 2021-11-03T18:27:51.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.586+0000 7fda9759e700 1 --2- 172.21.15.27:0/1364057828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda980f5710 0x7fda980f5b90 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:51.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.586+0000 7fda9759e700 1 --2- 172.21.15.27:0/1364057828 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda980f6a10 0x7fda980f6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:51.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.586+0000 7fda9759e700 1 -- 172.21.15.27:0/1364057828 >> 172.21.15.27:0/1364057828 conn(0x7fda980f0320 msgr2=0x7fda980f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:27:51.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.586+0000 7fda9759e700 1 -- 172.21.15.27:0/1364057828 shutdown_connections 2021-11-03T18:27:51.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.586+0000 7fda9759e700 1 -- 172.21.15.27:0/1364057828 wait complete. 2021-11-03T18:27:51.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.587+0000 7fda9759e700 1 Processor -- start 2021-11-03T18:27:51.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.587+0000 7fda9759e700 1 -- start start 2021-11-03T18:27:51.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.587+0000 7fda9759e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda980f5710 0x7fda981099b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:27:51.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.587+0000 7fda9759e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda980f6a10 0x7fda9810beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:27:51.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.587+0000 7fda9759e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda980f5400 con 0x7fda980f5710 2021-11-03T18:27:51.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.587+0000 7fda9759e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda980f5540 con 0x7fda980f6a10 2021-11-03T18:27:51.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.587+0000 7fda9659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda980f5710 0x7fda981099b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:27:51.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.587+0000 7fda9659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda980f5710 0x7fda981099b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35018/0 (socket says 172.21.15.27:35018) 2021-11-03T18:27:51.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.587+0000 7fda9659c700 1 -- 172.21.15.27:0/2329945217 learned_addr learned my addr 172.21.15.27:0/2329945217 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:27:51.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.587+0000 7fda95d9b700 1 --2- 172.21.15.27:0/2329945217 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda980f6a10 0x7fda9810beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:27:51.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.587+0000 7fda9659c700 1 -- 172.21.15.27:0/2329945217 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda980f6a10 msgr2=0x7fda9810beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:27:51.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.587+0000 7fda9659c700 1 --2- 172.21.15.27:0/2329945217 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda980f6a10 0x7fda9810beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:51.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.587+0000 7fda9659c700 1 -- 172.21.15.27:0/2329945217 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda88005040 con 0x7fda980f5710 2021-11-03T18:27:51.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.587+0000 7fda95d9b700 1 --2- 172.21.15.27:0/2329945217 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda980f6a10 0x7fda9810beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T18:27:51.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.588+0000 7fda9659c700 1 --2- 172.21.15.27:0/2329945217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda980f5710 0x7fda981099b0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 rx=0x7fda8c002700 tx=0x7fda8c009e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:27:51.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.588+0000 7fda86ffd700 1 -- 172.21.15.27:0/2329945217 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda8c009320 con 0x7fda980f5710 2021-11-03T18:27:51.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.588+0000 7fda86ffd700 1 -- 172.21.15.27:0/2329945217 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fda8c00f040 con 0x7fda980f5710 2021-11-03T18:27:51.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.588+0000 7fda9759e700 1 -- 172.21.15.27:0/2329945217 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda9810c4c0 con 0x7fda980f5710 2021-11-03T18:27:51.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.588+0000 7fda86ffd700 1 -- 172.21.15.27:0/2329945217 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda8c0148e0 con 0x7fda980f5710 2021-11-03T18:27:51.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.588+0000 7fda9759e700 1 -- 172.21.15.27:0/2329945217 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda9810ca50 con 0x7fda980f5710 2021-11-03T18:27:51.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.589+0000 7fda9759e700 1 -- 172.21.15.27:0/2329945217 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda98045ad0 con 0x7fda980f5710 2021-11-03T18:27:51.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.589+0000 7fda86ffd700 1 -- 172.21.15.27:0/2329945217 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fda8c009480 con 0x7fda980f5710 2021-11-03T18:27:51.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.590+0000 7fda86ffd700 1 --2- 172.21.15.27:0/2329945217 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fda80064100 0x7fda800665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:27:51.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.590+0000 7fda86ffd700 1 -- 172.21.15.27:0/2329945217 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fda8c080ae0 con 0x7fda980f5710 2021-11-03T18:27:51.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.590+0000 7fda95d9b700 1 --2- 172.21.15.27:0/2329945217 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fda80064100 0x7fda800665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:27:51.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.590+0000 7fda95d9b700 1 --2- 172.21.15.27:0/2329945217 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fda80064100 0x7fda800665c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7fda88009da0 tx=0x7fda88009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:27:51.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.594+0000 7fda86ffd700 1 -- 172.21.15.27:0/2329945217 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fda8c052e40 con 0x7fda980f5710 2021-11-03T18:27:51.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.799+0000 7fda9759e700 1 -- 172.21.15.27:0/2329945217 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fda980fddb0 con 0x7fda980f5710 2021-11-03T18:27:51.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.799+0000 7fda86ffd700 1 -- 172.21.15.27:0/2329945217 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7fda8c052a30 con 0x7fda980f5710 2021-11-03T18:27:51.799 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:27:51.799 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:27:51.800 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:27:51.800 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:27:51.800 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:27:51.800 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:27:51.800 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:27:51.801 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:27:51.801 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:27:51.801 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:27:51.802 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:27:51.802 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:27:51.802 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:27:51.802 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:27:51.803 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:27:51.803 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:27:51.803 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:27:51.803 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:27:51.804 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:27:51.804 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:27:51.804 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:27:51.804 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:27:51.804 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:27:51.805 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:27:51.805 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:27:51.805 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:27:51.805 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:27:51.806 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:27:51.806 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:27:51.806 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:27:51.807 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:27:51.807 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:27:51.807 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:27:51.807 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:27:51.807 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:27:51.808 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:27:51.808 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:27:51.808 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:27:51.808 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:27:51.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.801+0000 7fda9759e700 1 -- 172.21.15.27:0/2329945217 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fda80064100 msgr2=0x7fda800665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:27:51.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.801+0000 7fda9759e700 1 --2- 172.21.15.27:0/2329945217 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fda80064100 0x7fda800665c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7fda88009da0 tx=0x7fda88009b30).stop 2021-11-03T18:27:51.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.801+0000 7fda9759e700 1 -- 172.21.15.27:0/2329945217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda980f5710 msgr2=0x7fda981099b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:27:51.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.801+0000 7fda9759e700 1 --2- 172.21.15.27:0/2329945217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda980f5710 0x7fda981099b0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 rx=0x7fda8c002700 tx=0x7fda8c009e00).stop 2021-11-03T18:27:51.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.801+0000 7fda9759e700 1 -- 172.21.15.27:0/2329945217 shutdown_connections 2021-11-03T18:27:51.811 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.801+0000 7fda9759e700 1 --2- 172.21.15.27:0/2329945217 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fda80064100 0x7fda800665c0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:51.811 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.801+0000 7fda9759e700 1 --2- 172.21.15.27:0/2329945217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fda980f5710 0x7fda981099b0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:51.811 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.801+0000 7fda9759e700 1 --2- 172.21.15.27:0/2329945217 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fda980f6a10 0x7fda9810beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:27:51.812 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.801+0000 7fda9759e700 1 -- 172.21.15.27:0/2329945217 >> 172.21.15.27:0/2329945217 conn(0x7fda980f0320 msgr2=0x7fda980fa170 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:27:51.812 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.802+0000 7fda9759e700 1 -- 172.21.15.27:0/2329945217 shutdown_connections 2021-11-03T18:27:51.812 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:27:51.802+0000 7fda9759e700 1 -- 172.21.15.27:0/2329945217 wait complete. 2021-11-03T18:27:51.812 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:27:51.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:51 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/603474530' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:52.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:51 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/603474530' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:52.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:52 smithi027 ceph-mon[97244]: from='client.36360 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:27:52.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:52 smithi027 ceph-mon[97244]: pgmap v2198: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:52.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:52 smithi027 ceph-mon[97244]: from='client.36364 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:27:52.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:52 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2329945217' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:27:52.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:52 smithi115 ceph-mon[65453]: from='client.36360 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:27:52.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:52 smithi115 ceph-mon[65453]: pgmap v2198: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:52.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:52 smithi115 ceph-mon[65453]: from='client.36364 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:27:52.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:52 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2329945217' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:27:54.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:54 smithi115 ceph-mon[65453]: pgmap v2199: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:55.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:54 smithi027 ceph-mon[97244]: pgmap v2199: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:55.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4317.463176 secs 2021-11-03T18:27:55.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:27:55.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:27:55.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:27:55.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:55.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:55.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:55.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:56.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4317.463176 secs 2021-11-03T18:27:56.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:27:56.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:27:56.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:27:56.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:56.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:56.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:56.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:27:56.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:56 smithi115 ceph-mon[65453]: pgmap v2200: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:56.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:56 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:27:56.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:27:56.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:27:57.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:56 smithi027 ceph-mon[97244]: pgmap v2200: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:57.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:56 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:27:57.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:27:57.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:27:58.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:27:58 smithi115 ceph-mon[65453]: pgmap v2201: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:27:59.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:27:58 smithi027 ceph-mon[97244]: pgmap v2201: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:00.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:00 smithi115 ceph-mon[65453]: pgmap v2202: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:00.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4322.463304 secs 2021-11-03T18:28:01.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:00 smithi027 ceph-mon[97244]: pgmap v2202: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:01.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4322.463304 secs 2021-11-03T18:28:02.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:02 smithi027 ceph-mon[97244]: pgmap v2203: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:02.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:02 smithi115 ceph-mon[65453]: pgmap v2203: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:04.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:04 smithi115 ceph-mon[65453]: pgmap v2204: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:05.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:04 smithi027 ceph-mon[97244]: pgmap v2204: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:05.996 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4327.463451 secs 2021-11-03T18:28:05.996 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:28:05.997 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:28:05.997 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:28:05.997 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:05.997 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:05.997 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:05.998 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:06.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4327.463451 secs 2021-11-03T18:28:06.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:28:06.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:28:06.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:28:06.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:06.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:06.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:06.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:06.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:06 smithi115 ceph-mon[65453]: pgmap v2205: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:06.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:06 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:28:07.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:06 smithi027 ceph-mon[97244]: pgmap v2205: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:07.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:06 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:28:08.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:08 smithi115 ceph-mon[65453]: pgmap v2206: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:09.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:08 smithi027 ceph-mon[97244]: pgmap v2206: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:10.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:10 smithi115 ceph-mon[65453]: pgmap v2207: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:10.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4332.463542 secs 2021-11-03T18:28:11.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:10 smithi027 ceph-mon[97244]: pgmap v2207: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:11.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4332.463542 secs 2021-11-03T18:28:12.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:12 smithi115 ceph-mon[65453]: pgmap v2208: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:13.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:12 smithi027 ceph-mon[97244]: pgmap v2208: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:14.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:14 smithi115 ceph-mon[65453]: pgmap v2209: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:15.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:14 smithi027 ceph-mon[97244]: pgmap v2209: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:15.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4337.463652 secs 2021-11-03T18:28:15.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:28:15.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:28:15.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:28:15.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:15.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:15.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:15.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:16.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4337.463652 secs 2021-11-03T18:28:16.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:28:16.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:28:16.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:28:16.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:16.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:16.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:16.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:16.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:16 smithi115 ceph-mon[65453]: pgmap v2210: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:16.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:16 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:28:17.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:16 smithi027 ceph-mon[97244]: pgmap v2210: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:17.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:16 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:28:18.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:18 smithi115 ceph-mon[65453]: pgmap v2211: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:19.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:18 smithi027 ceph-mon[97244]: pgmap v2211: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:20.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:20 smithi115 ceph-mon[65453]: pgmap v2212: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:20.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4342.463784 secs 2021-11-03T18:28:21.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:20 smithi027 ceph-mon[97244]: pgmap v2212: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:21.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4342.463784 secs 2021-11-03T18:28:21.916 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:21.915+0000 7f18ae5cd700 1 -- 172.21.15.27:0/305823141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18a80f6ff0 msgr2=0x7f18a80f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:28:21.916 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:21.915+0000 7f18ae5cd700 1 --2- 172.21.15.27:0/305823141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18a80f6ff0 0x7f18a80f7a20 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 rx=0x7f1890004660 tx=0x7f1890009b30).stop 2021-11-03T18:28:21.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:21.915+0000 7f18ae5cd700 1 -- 172.21.15.27:0/305823141 shutdown_connections 2021-11-03T18:28:21.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:21.915+0000 7f18ae5cd700 1 --2- 172.21.15.27:0/305823141 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18a80f6ff0 0x7f18a80f7a20 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:21.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:21.915+0000 7f18ae5cd700 1 --2- 172.21.15.27:0/305823141 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18a80f66e0 0x7f18a80f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:21.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:21.915+0000 7f18ae5cd700 1 -- 172.21.15.27:0/305823141 >> 172.21.15.27:0/305823141 conn(0x7f18a80f1f00 msgr2=0x7f18a80f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:28:21.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:21.916+0000 7f18ae5cd700 1 -- 172.21.15.27:0/305823141 shutdown_connections 2021-11-03T18:28:21.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:21.916+0000 7f18ae5cd700 1 -- 172.21.15.27:0/305823141 wait complete. 2021-11-03T18:28:21.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:21.916+0000 7f18ae5cd700 1 Processor -- start 2021-11-03T18:28:21.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:21.916+0000 7f18ae5cd700 1 -- start start 2021-11-03T18:28:21.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:21.916+0000 7f18ae5cd700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18a80f66e0 0x7f18a8106ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:28:21.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:21.916+0000 7f18ae5cd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18a80f6ff0 0x7f18a81073e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:28:21.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:21.916+0000 7f18ae5cd700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a8107990 con 0x7f18a80f6ff0 2021-11-03T18:28:21.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:21.917+0000 7f18ae5cd700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a8107ad0 con 0x7f18a80f66e0 2021-11-03T18:28:21.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:21.917+0000 7f18a77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18a80f6ff0 0x7f18a81073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:28:21.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:21.917+0000 7f18a77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18a80f6ff0 0x7f18a81073e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35038/0 (socket says 172.21.15.27:35038) 2021-11-03T18:28:21.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:21.917+0000 7f18a7fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18a80f66e0 0x7f18a8106ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:28:21.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:21.917+0000 7f18a77fe700 1 -- 172.21.15.27:0/931898082 learned_addr learned my addr 172.21.15.27:0/931898082 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:28:21.922 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:21.917+0000 7f18a77fe700 1 -- 172.21.15.27:0/931898082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18a80f66e0 msgr2=0x7f18a8106ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:28:21.922 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:21.917+0000 7f18a77fe700 1 --2- 172.21.15.27:0/931898082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18a80f66e0 0x7f18a8106ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:21.922 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:21.917+0000 7f18a77fe700 1 -- 172.21.15.27:0/931898082 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1890005040 con 0x7f18a80f6ff0 2021-11-03T18:28:21.922 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:21.917+0000 7f18a77fe700 1 --2- 172.21.15.27:0/931898082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18a80f6ff0 0x7f18a81073e0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 rx=0x7f1890000c00 tx=0x7f189000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:28:21.923 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:21.917+0000 7f18a4ff9700 1 -- 172.21.15.27:0/931898082 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1890019070 con 0x7f18a80f6ff0 2021-11-03T18:28:21.923 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:21.917+0000 7f18ae5cd700 1 -- 172.21.15.27:0/931898082 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18a8108410 con 0x7f18a80f6ff0 2021-11-03T18:28:21.923 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:21.917+0000 7f18a4ff9700 1 -- 172.21.15.27:0/931898082 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f1890007e70 con 0x7f18a80f6ff0 2021-11-03T18:28:21.923 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:21.917+0000 7f18ae5cd700 1 -- 172.21.15.27:0/931898082 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18a8108970 con 0x7f18a80f6ff0 2021-11-03T18:28:21.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:21.917+0000 7f18a4ff9700 1 -- 172.21.15.27:0/931898082 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1890014ce0 con 0x7f18a80f6ff0 2021-11-03T18:28:21.926 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:21.918+0000 7f18ae5cd700 1 -- 172.21.15.27:0/931898082 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18a8045ad0 con 0x7f18a80f6ff0 2021-11-03T18:28:21.926 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:21.919+0000 7f18a4ff9700 1 -- 172.21.15.27:0/931898082 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f1890025070 con 0x7f18a80f6ff0 2021-11-03T18:28:21.926 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:21.919+0000 7f18a4ff9700 1 --2- 172.21.15.27:0/931898082 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f18940640b0 0x7f1894066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:28:21.927 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:21.919+0000 7f18a4ff9700 1 -- 172.21.15.27:0/931898082 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f1890081670 con 0x7f18a80f6ff0 2021-11-03T18:28:21.927 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:21.919+0000 7f18a7fff700 1 --2- 172.21.15.27:0/931898082 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f18940640b0 0x7f1894066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:28:21.927 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:21.920+0000 7f18a7fff700 1 --2- 172.21.15.27:0/931898082 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f18940640b0 0x7f1894066570 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f1898009fd0 tx=0x7f189800b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:28:21.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:21.923+0000 7f18a4ff9700 1 -- 172.21.15.27:0/931898082 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f18900539d0 con 0x7f18a80f6ff0 2021-11-03T18:28:22.103 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.104+0000 7f18ae5cd700 1 -- 172.21.15.27:0/931898082 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f18a80ff080 con 0x7f18940640b0 2021-11-03T18:28:22.104 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.105+0000 7f18a4ff9700 1 -- 172.21.15.27:0/931898082 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f189800f040 con 0x7f18940640b0 2021-11-03T18:28:22.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.107+0000 7f18ae5cd700 1 -- 172.21.15.27:0/931898082 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f18940640b0 msgr2=0x7f1894066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:28:22.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.107+0000 7f18ae5cd700 1 --2- 172.21.15.27:0/931898082 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f18940640b0 0x7f1894066570 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f1898009fd0 tx=0x7f189800b040).stop 2021-11-03T18:28:22.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.107+0000 7f18ae5cd700 1 -- 172.21.15.27:0/931898082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18a80f6ff0 msgr2=0x7f18a81073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:28:22.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.107+0000 7f18ae5cd700 1 --2- 172.21.15.27:0/931898082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18a80f6ff0 0x7f18a81073e0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 rx=0x7f1890000c00 tx=0x7f189000b040).stop 2021-11-03T18:28:22.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.107+0000 7f18ae5cd700 1 -- 172.21.15.27:0/931898082 shutdown_connections 2021-11-03T18:28:22.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.107+0000 7f18ae5cd700 1 --2- 172.21.15.27:0/931898082 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f18940640b0 0x7f1894066570 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:22.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.107+0000 7f18ae5cd700 1 --2- 172.21.15.27:0/931898082 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f18a80f66e0 0x7f18a8106ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:22.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.107+0000 7f18ae5cd700 1 --2- 172.21.15.27:0/931898082 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f18a80f6ff0 0x7f18a81073e0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:22.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.107+0000 7f18ae5cd700 1 -- 172.21.15.27:0/931898082 >> 172.21.15.27:0/931898082 conn(0x7f18a80f1f00 msgr2=0x7f18a80fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:28:22.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.107+0000 7f18ae5cd700 1 -- 172.21.15.27:0/931898082 shutdown_connections 2021-11-03T18:28:22.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.107+0000 7f18ae5cd700 1 -- 172.21.15.27:0/931898082 wait complete. 2021-11-03T18:28:22.116 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:28:22.213 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.211+0000 7f8e01dd7700 1 -- 172.21.15.27:0/564793104 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8dfc0f6ff0 msgr2=0x7f8dfc0f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:28:22.213 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.211+0000 7f8e01dd7700 1 --2- 172.21.15.27:0/564793104 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8dfc0f6ff0 0x7f8dfc0f7a20 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 rx=0x7f8df0004660 tx=0x7f8df0009b30).stop 2021-11-03T18:28:22.213 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.211+0000 7f8e01dd7700 1 -- 172.21.15.27:0/564793104 shutdown_connections 2021-11-03T18:28:22.213 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.211+0000 7f8e01dd7700 1 --2- 172.21.15.27:0/564793104 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8dfc0f6ff0 0x7f8dfc0f7a20 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:22.214 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.211+0000 7f8e01dd7700 1 --2- 172.21.15.27:0/564793104 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8dfc0f66e0 0x7f8dfc0f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:22.214 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.211+0000 7f8e01dd7700 1 -- 172.21.15.27:0/564793104 >> 172.21.15.27:0/564793104 conn(0x7f8dfc0f1f00 msgr2=0x7f8dfc0f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:28:22.214 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.211+0000 7f8e01dd7700 1 -- 172.21.15.27:0/564793104 shutdown_connections 2021-11-03T18:28:22.214 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.211+0000 7f8e01dd7700 1 -- 172.21.15.27:0/564793104 wait complete. 2021-11-03T18:28:22.215 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.212+0000 7f8e01dd7700 1 Processor -- start 2021-11-03T18:28:22.215 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.212+0000 7f8e01dd7700 1 -- start start 2021-11-03T18:28:22.215 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.212+0000 7f8e01dd7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8dfc0f66e0 0x7f8dfc106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:28:22.215 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.212+0000 7f8e01dd7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8dfc0f6ff0 0x7f8dfc107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:28:22.215 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.212+0000 7f8e01dd7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dfc107a10 con 0x7f8dfc0f66e0 2021-11-03T18:28:22.216 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.212+0000 7f8e01dd7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dfc107b50 con 0x7f8dfc0f6ff0 2021-11-03T18:28:22.216 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.212+0000 7f8dfb7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8dfc0f66e0 0x7f8dfc106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:28:22.216 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.212+0000 7f8dfb7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8dfc0f66e0 0x7f8dfc106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35046/0 (socket says 172.21.15.27:35046) 2021-11-03T18:28:22.216 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.212+0000 7f8dfb7fe700 1 -- 172.21.15.27:0/3266333389 learned_addr learned my addr 172.21.15.27:0/3266333389 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:28:22.217 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.212+0000 7f8dfaffd700 1 --2- 172.21.15.27:0/3266333389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8dfc0f6ff0 0x7f8dfc107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:28:22.217 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.212+0000 7f8dfb7fe700 1 -- 172.21.15.27:0/3266333389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8dfc0f6ff0 msgr2=0x7f8dfc107460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:28:22.217 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.212+0000 7f8dfb7fe700 1 --2- 172.21.15.27:0/3266333389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8dfc0f6ff0 0x7f8dfc107460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:22.217 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.212+0000 7f8dfb7fe700 1 -- 172.21.15.27:0/3266333389 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8df0005040 con 0x7f8dfc0f66e0 2021-11-03T18:28:22.218 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.212+0000 7f8dfaffd700 1 --2- 172.21.15.27:0/3266333389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8dfc0f6ff0 0x7f8dfc107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:28:22.218 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.213+0000 7f8dfb7fe700 1 --2- 172.21.15.27:0/3266333389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8dfc0f66e0 0x7f8dfc106f70 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 rx=0x7f8dec00c530 tx=0x7f8dec007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:28:22.218 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.213+0000 7f8de3fff700 1 -- 172.21.15.27:0/3266333389 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8dec00f650 con 0x7f8dfc0f66e0 2021-11-03T18:28:22.218 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.213+0000 7f8de3fff700 1 -- 172.21.15.27:0/3266333389 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f8dec00fc60 con 0x7f8dfc0f66e0 2021-11-03T18:28:22.218 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.213+0000 7f8de3fff700 1 -- 172.21.15.27:0/3266333389 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8dec012cd0 con 0x7f8dfc0f66e0 2021-11-03T18:28:22.219 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.213+0000 7f8e01dd7700 1 -- 172.21.15.27:0/3266333389 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dfc1084f0 con 0x7f8dfc0f66e0 2021-11-03T18:28:22.219 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.213+0000 7f8e01dd7700 1 -- 172.21.15.27:0/3266333389 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dfc108ab0 con 0x7f8dfc0f66e0 2021-11-03T18:28:22.219 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.214+0000 7f8e01dd7700 1 -- 172.21.15.27:0/3266333389 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dfc045ad0 con 0x7f8dfc0f66e0 2021-11-03T18:28:22.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.214+0000 7f8de3fff700 1 -- 172.21.15.27:0/3266333389 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f8dec010070 con 0x7f8dfc0f66e0 2021-11-03T18:28:22.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.214+0000 7f8de3fff700 1 --2- 172.21.15.27:0/3266333389 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8de40640b0 0x7f8de4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:28:22.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.215+0000 7f8de3fff700 1 -- 172.21.15.27:0/3266333389 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f8dec0806f0 con 0x7f8dfc0f66e0 2021-11-03T18:28:22.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.215+0000 7f8dfaffd700 1 --2- 172.21.15.27:0/3266333389 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8de40640b0 0x7f8de4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:28:22.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.215+0000 7f8dfaffd700 1 --2- 172.21.15.27:0/3266333389 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8de40640b0 0x7f8de4066570 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f8df0004e30 tx=0x7f8df0009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:28:22.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.219+0000 7f8de3fff700 1 -- 172.21.15.27:0/3266333389 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f8dec052a20 con 0x7f8dfc0f66e0 2021-11-03T18:28:22.396 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.397+0000 7f8e01dd7700 1 -- 172.21.15.27:0/3266333389 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8dfc0fefe0 con 0x7f8de40640b0 2021-11-03T18:28:22.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.412+0000 7f8de3fff700 1 -- 172.21.15.27:0/3266333389 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f8dfc0fefe0 con 0x7f8de40640b0 2021-11-03T18:28:22.411 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:28:22.412 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (79m) 8m ago 81m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:28:22.412 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (71m) 8m ago 81m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:28:22.412 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (71m) 8m ago 80m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:28:22.412 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (79m) 8m ago 80m 42.4M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:28:22.413 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (73m) 8m ago 76m 26.5M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:28:22.413 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (76m) 8m ago 76m 3849M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:28:22.413 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (72m) 8m ago 76m 26.1M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:28:22.413 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (73m) 8m ago 76m 27.2M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:28:22.413 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (73m) 8m ago 82m 590M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:28:22.414 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (72m) 8m ago 80m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:28:22.414 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (72m) 8m ago 82m 312M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:28:22.414 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (72m) 8m ago 80m 145M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:28:22.414 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (80m) 8m ago 80m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:28:22.415 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (80m) 8m ago 80m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:28:22.415 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (71m) 8m ago 79m 392M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:28:22.415 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (70m) 8m ago 79m 427M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:28:22.415 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (70m) 8m ago 78m 320M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:28:22.416 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (69m) 8m ago 78m 454M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:28:22.416 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (69m) 8m ago 78m 359M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:28:22.416 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (68m) 8m ago 77m 368M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:28:22.416 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (79m) 8m ago 80m 59.9M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:28:22.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.414+0000 7f8e01dd7700 1 -- 172.21.15.27:0/3266333389 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8de40640b0 msgr2=0x7f8de4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:28:22.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.414+0000 7f8e01dd7700 1 --2- 172.21.15.27:0/3266333389 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8de40640b0 0x7f8de4066570 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f8df0004e30 tx=0x7f8df0009b30).stop 2021-11-03T18:28:22.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.414+0000 7f8e01dd7700 1 -- 172.21.15.27:0/3266333389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8dfc0f66e0 msgr2=0x7f8dfc106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:28:22.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.414+0000 7f8e01dd7700 1 --2- 172.21.15.27:0/3266333389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8dfc0f66e0 0x7f8dfc106f70 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 rx=0x7f8dec00c530 tx=0x7f8dec007e20).stop 2021-11-03T18:28:22.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.414+0000 7f8e01dd7700 1 -- 172.21.15.27:0/3266333389 shutdown_connections 2021-11-03T18:28:22.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.414+0000 7f8e01dd7700 1 --2- 172.21.15.27:0/3266333389 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8de40640b0 0x7f8de4066570 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:22.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.414+0000 7f8e01dd7700 1 --2- 172.21.15.27:0/3266333389 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8dfc0f66e0 0x7f8dfc106f70 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:22.419 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.414+0000 7f8e01dd7700 1 --2- 172.21.15.27:0/3266333389 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8dfc0f6ff0 0x7f8dfc107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:22.419 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.414+0000 7f8e01dd7700 1 -- 172.21.15.27:0/3266333389 >> 172.21.15.27:0/3266333389 conn(0x7f8dfc0f1f00 msgr2=0x7f8dfc0fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:28:22.419 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.414+0000 7f8e01dd7700 1 -- 172.21.15.27:0/3266333389 shutdown_connections 2021-11-03T18:28:22.419 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.414+0000 7f8e01dd7700 1 -- 172.21.15.27:0/3266333389 wait complete. 2021-11-03T18:28:22.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.513+0000 7f308239e700 1 -- 172.21.15.27:0/760475953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f307c06b9c0 msgr2=0x7f307c069ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:28:22.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.513+0000 7f308239e700 1 --2- 172.21.15.27:0/760475953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f307c06b9c0 0x7f307c069ec0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 rx=0x7f3078004660 tx=0x7f3078009b30).stop 2021-11-03T18:28:22.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.514+0000 7f308239e700 1 -- 172.21.15.27:0/760475953 shutdown_connections 2021-11-03T18:28:22.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.514+0000 7f308239e700 1 --2- 172.21.15.27:0/760475953 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f307c06a3b0 0x7f307c06a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:22.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.514+0000 7f308239e700 1 --2- 172.21.15.27:0/760475953 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f307c06b9c0 0x7f307c069ec0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:22.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.514+0000 7f308239e700 1 -- 172.21.15.27:0/760475953 >> 172.21.15.27:0/760475953 conn(0x7f307c0f1cd0 msgr2=0x7f307c0f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:28:22.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.514+0000 7f308239e700 1 -- 172.21.15.27:0/760475953 shutdown_connections 2021-11-03T18:28:22.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.514+0000 7f308239e700 1 -- 172.21.15.27:0/760475953 wait complete. 2021-11-03T18:28:22.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.514+0000 7f308239e700 1 Processor -- start 2021-11-03T18:28:22.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.514+0000 7f308239e700 1 -- start start 2021-11-03T18:28:22.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.514+0000 7f308239e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f307c06a3b0 0x7f307c108f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:28:22.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.514+0000 7f308239e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f307c06b9c0 0x7f307c109440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:28:22.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.514+0000 7f308239e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c1099f0 con 0x7f307c06b9c0 2021-11-03T18:28:22.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.514+0000 7f308239e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f307c109b30 con 0x7f307c06a3b0 2021-11-03T18:28:22.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.514+0000 7f3080b9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f307c06b9c0 0x7f307c109440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:28:22.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.514+0000 7f3080b9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f307c06b9c0 0x7f307c109440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35058/0 (socket says 172.21.15.27:35058) 2021-11-03T18:28:22.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.514+0000 7f3080b9b700 1 -- 172.21.15.27:0/475289627 learned_addr learned my addr 172.21.15.27:0/475289627 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:28:22.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.515+0000 7f308139c700 1 --2- 172.21.15.27:0/475289627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f307c06a3b0 0x7f307c108f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:28:22.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.515+0000 7f3080b9b700 1 -- 172.21.15.27:0/475289627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f307c06a3b0 msgr2=0x7f307c108f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:28:22.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.515+0000 7f3080b9b700 1 --2- 172.21.15.27:0/475289627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f307c06a3b0 0x7f307c108f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:22.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.515+0000 7f3080b9b700 1 -- 172.21.15.27:0/475289627 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3078005040 con 0x7f307c06b9c0 2021-11-03T18:28:22.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.515+0000 7f308139c700 1 --2- 172.21.15.27:0/475289627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f307c06a3b0 0x7f307c108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T18:28:22.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.515+0000 7f3080b9b700 1 --2- 172.21.15.27:0/475289627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f307c06b9c0 0x7f307c109440 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 rx=0x7f306c0026d0 tx=0x7f306c009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:28:22.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.515+0000 7f3071ffb700 1 -- 172.21.15.27:0/475289627 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f306c009320 con 0x7f307c06b9c0 2021-11-03T18:28:22.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.515+0000 7f3071ffb700 1 -- 172.21.15.27:0/475289627 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f306c00f040 con 0x7f307c06b9c0 2021-11-03T18:28:22.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.515+0000 7f308239e700 1 -- 172.21.15.27:0/475289627 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f307c10a4d0 con 0x7f307c06b9c0 2021-11-03T18:28:22.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.515+0000 7f3071ffb700 1 -- 172.21.15.27:0/475289627 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f306c0148f0 con 0x7f307c06b9c0 2021-11-03T18:28:22.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.515+0000 7f308239e700 1 -- 172.21.15.27:0/475289627 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f307c10aa90 con 0x7f307c06b9c0 2021-11-03T18:28:22.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.516+0000 7f308239e700 1 -- 172.21.15.27:0/475289627 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f307c045ad0 con 0x7f307c06b9c0 2021-11-03T18:28:22.524 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.517+0000 7f3071ffb700 1 -- 172.21.15.27:0/475289627 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f306c01b060 con 0x7f307c06b9c0 2021-11-03T18:28:22.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.517+0000 7f3071ffb700 1 --2- 172.21.15.27:0/475289627 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3068064100 0x7f30680665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:28:22.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.517+0000 7f3071ffb700 1 -- 172.21.15.27:0/475289627 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f306c0807e0 con 0x7f307c06b9c0 2021-11-03T18:28:22.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.517+0000 7f308139c700 1 --2- 172.21.15.27:0/475289627 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3068064100 0x7f30680665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:28:22.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.518+0000 7f308139c700 1 --2- 172.21.15.27:0/475289627 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3068064100 0x7f30680665c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f3078009da0 tx=0x7f3078009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:28:22.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.521+0000 7f3071ffb700 1 -- 172.21.15.27:0/475289627 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f306c052a50 con 0x7f307c06b9c0 2021-11-03T18:28:22.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.751+0000 7f308239e700 1 -- 172.21.15.27:0/475289627 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f307c027a00 con 0x7f307c06b9c0 2021-11-03T18:28:22.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.751+0000 7f3071ffb700 1 -- 172.21.15.27:0/475289627 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f306c052640 con 0x7f307c06b9c0 2021-11-03T18:28:22.751 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:28:22.751 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:28:22.752 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:28:22.752 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:28:22.752 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:28:22.753 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:28:22.753 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:28:22.753 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:28:22.753 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:28:22.754 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:28:22.754 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:28:22.754 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:28:22.755 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:28:22.755 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:28:22.755 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:28:22.756 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:28:22.756 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:28:22.756 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:28:22.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.753+0000 7f308239e700 1 -- 172.21.15.27:0/475289627 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3068064100 msgr2=0x7f30680665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:28:22.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.753+0000 7f308239e700 1 --2- 172.21.15.27:0/475289627 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3068064100 0x7f30680665c0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f3078009da0 tx=0x7f3078009b30).stop 2021-11-03T18:28:22.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.753+0000 7f308239e700 1 -- 172.21.15.27:0/475289627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f307c06b9c0 msgr2=0x7f307c109440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:28:22.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.753+0000 7f308239e700 1 --2- 172.21.15.27:0/475289627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f307c06b9c0 0x7f307c109440 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 rx=0x7f306c0026d0 tx=0x7f306c009de0).stop 2021-11-03T18:28:22.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.753+0000 7f308239e700 1 -- 172.21.15.27:0/475289627 shutdown_connections 2021-11-03T18:28:22.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.753+0000 7f308239e700 1 --2- 172.21.15.27:0/475289627 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3068064100 0x7f30680665c0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:22.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.753+0000 7f308239e700 1 --2- 172.21.15.27:0/475289627 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f307c06a3b0 0x7f307c108f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:22.759 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.753+0000 7f308239e700 1 --2- 172.21.15.27:0/475289627 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f307c06b9c0 0x7f307c109440 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:22.759 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.753+0000 7f308239e700 1 -- 172.21.15.27:0/475289627 >> 172.21.15.27:0/475289627 conn(0x7f307c0f1cd0 msgr2=0x7f307c0fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:28:22.759 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.754+0000 7f308239e700 1 -- 172.21.15.27:0/475289627 shutdown_connections 2021-11-03T18:28:22.759 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.754+0000 7f308239e700 1 -- 172.21.15.27:0/475289627 wait complete. 2021-11-03T18:28:22.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.854+0000 7f0be3714700 1 -- 172.21.15.27:0/2286163694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bdc0f6ff0 msgr2=0x7f0bdc0f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:28:22.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.854+0000 7f0be3714700 1 --2- 172.21.15.27:0/2286163694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bdc0f6ff0 0x7f0bdc0f7a20 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 rx=0x7f0bd0004660 tx=0x7f0bd0009b30).stop 2021-11-03T18:28:22.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.854+0000 7f0be3714700 1 -- 172.21.15.27:0/2286163694 shutdown_connections 2021-11-03T18:28:22.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.854+0000 7f0be3714700 1 --2- 172.21.15.27:0/2286163694 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bdc0f6ff0 0x7f0bdc0f7a20 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:22.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.854+0000 7f0be3714700 1 --2- 172.21.15.27:0/2286163694 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0bdc0f66e0 0x7f0bdc0f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:22.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.854+0000 7f0be3714700 1 -- 172.21.15.27:0/2286163694 >> 172.21.15.27:0/2286163694 conn(0x7f0bdc0f1f00 msgr2=0x7f0bdc0f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:28:22.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.854+0000 7f0be3714700 1 -- 172.21.15.27:0/2286163694 shutdown_connections 2021-11-03T18:28:22.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.854+0000 7f0be3714700 1 -- 172.21.15.27:0/2286163694 wait complete. 2021-11-03T18:28:22.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.854+0000 7f0be3714700 1 Processor -- start 2021-11-03T18:28:22.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.854+0000 7f0be3714700 1 -- start start 2021-11-03T18:28:22.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.854+0000 7f0be3714700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bdc0f66e0 0x7f0bdc06b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:28:22.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.854+0000 7f0be3714700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0bdc0f6ff0 0x7f0bdc069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:28:22.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.854+0000 7f0be3714700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bdc06a380 con 0x7f0bdc0f66e0 2021-11-03T18:28:22.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.854+0000 7f0be3714700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bdc06a4c0 con 0x7f0bdc0f6ff0 2021-11-03T18:28:22.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.855+0000 7f0be14b0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bdc0f66e0 0x7f0bdc06b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:28:22.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.855+0000 7f0be14b0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bdc0f66e0 0x7f0bdc06b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35068/0 (socket says 172.21.15.27:35068) 2021-11-03T18:28:22.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.855+0000 7f0be14b0700 1 -- 172.21.15.27:0/609453311 learned_addr learned my addr 172.21.15.27:0/609453311 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:28:22.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.855+0000 7f0be0caf700 1 --2- 172.21.15.27:0/609453311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0bdc0f6ff0 0x7f0bdc069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:28:22.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.855+0000 7f0be14b0700 1 -- 172.21.15.27:0/609453311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0bdc0f6ff0 msgr2=0x7f0bdc069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:28:22.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.855+0000 7f0be14b0700 1 --2- 172.21.15.27:0/609453311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0bdc0f6ff0 0x7f0bdc069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:22.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.855+0000 7f0be14b0700 1 -- 172.21.15.27:0/609453311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bd0005040 con 0x7f0bdc0f66e0 2021-11-03T18:28:22.860 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.855+0000 7f0be14b0700 1 --2- 172.21.15.27:0/609453311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bdc0f66e0 0x7f0bdc06b9a0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 rx=0x7f0bd4007fd0 tx=0x7f0bd4004ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:28:22.860 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.855+0000 7f0bd9ffb700 1 -- 172.21.15.27:0/609453311 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bd400b4f0 con 0x7f0bdc0f66e0 2021-11-03T18:28:22.860 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.855+0000 7f0bd9ffb700 1 -- 172.21.15.27:0/609453311 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f0bd400f040 con 0x7f0bdc0f66e0 2021-11-03T18:28:22.860 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.855+0000 7f0bd9ffb700 1 -- 172.21.15.27:0/609453311 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0bd4013a50 con 0x7f0bdc0f66e0 2021-11-03T18:28:22.861 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.855+0000 7f0be3714700 1 -- 172.21.15.27:0/609453311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bdc06a7a0 con 0x7f0bdc0f66e0 2021-11-03T18:28:22.861 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.856+0000 7f0be3714700 1 -- 172.21.15.27:0/609453311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bdc10efa0 con 0x7f0bdc0f66e0 2021-11-03T18:28:22.862 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.856+0000 7f0be3714700 1 -- 172.21.15.27:0/609453311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bdc045ad0 con 0x7f0bdc0f66e0 2021-11-03T18:28:22.862 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.857+0000 7f0bd9ffb700 1 -- 172.21.15.27:0/609453311 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f0bd401a030 con 0x7f0bdc0f66e0 2021-11-03T18:28:22.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.857+0000 7f0bd9ffb700 1 --2- 172.21.15.27:0/609453311 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0bc40640b0 0x7f0bc4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:28:22.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.857+0000 7f0bd9ffb700 1 -- 172.21.15.27:0/609453311 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f0bd40806e0 con 0x7f0bdc0f66e0 2021-11-03T18:28:22.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.857+0000 7f0be0caf700 1 --2- 172.21.15.27:0/609453311 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0bc40640b0 0x7f0bc4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:28:22.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.858+0000 7f0be0caf700 1 --2- 172.21.15.27:0/609453311 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0bc40640b0 0x7f0bc4066570 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7f0bd0004630 tx=0x7f0bd0009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:28:22.864 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:22.861+0000 7f0bd9ffb700 1 -- 172.21.15.27:0/609453311 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f0bd4052a00 con 0x7f0bdc0f66e0 2021-11-03T18:28:22.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:22 smithi115 ceph-mon[65453]: pgmap v2213: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:23.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:22 smithi027 ceph-mon[97244]: pgmap v2213: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:23.061 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:23.063+0000 7f0be3714700 1 -- 172.21.15.27:0/609453311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0bdc0586e0 con 0x7f0bdc0f66e0 2021-11-03T18:28:23.062 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:23.063+0000 7f0bd9ffb700 1 -- 172.21.15.27:0/609453311 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f0bd4018020 con 0x7f0bdc0f66e0 2021-11-03T18:28:23.062 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:28:23.063 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:28:23.063 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:28:23.063 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:28:23.063 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:28:23.064 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:28:23.064 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:28:23.064 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:28:23.064 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:28:23.065 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:28:23.065 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:28:23.065 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:28:23.065 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:28:23.065 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:28:23.066 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:28:23.066 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:28:23.066 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:28:23.066 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:28:23.066 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:28:23.067 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:28:23.067 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:28:23.067 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:28:23.067 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:28:23.068 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:28:23.068 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:28:23.068 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:28:23.068 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:28:23.068 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:28:23.069 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:28:23.069 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:28:23.069 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:28:23.069 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:28:23.070 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:28:23.070 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:28:23.070 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:28:23.070 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:28:23.070 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:28:23.071 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:28:23.071 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:28:23.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:23.065+0000 7f0be3714700 1 -- 172.21.15.27:0/609453311 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0bc40640b0 msgr2=0x7f0bc4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:28:23.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:23.065+0000 7f0be3714700 1 --2- 172.21.15.27:0/609453311 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0bc40640b0 0x7f0bc4066570 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7f0bd0004630 tx=0x7f0bd0009b30).stop 2021-11-03T18:28:23.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:23.065+0000 7f0be3714700 1 -- 172.21.15.27:0/609453311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bdc0f66e0 msgr2=0x7f0bdc06b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:28:23.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:23.065+0000 7f0be3714700 1 --2- 172.21.15.27:0/609453311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bdc0f66e0 0x7f0bdc06b9a0 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 rx=0x7f0bd4007fd0 tx=0x7f0bd4004ee0).stop 2021-11-03T18:28:23.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:23.065+0000 7f0be3714700 1 -- 172.21.15.27:0/609453311 shutdown_connections 2021-11-03T18:28:23.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:23.065+0000 7f0be3714700 1 --2- 172.21.15.27:0/609453311 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0bc40640b0 0x7f0bc4066570 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:23.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:23.065+0000 7f0be3714700 1 --2- 172.21.15.27:0/609453311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0bdc0f66e0 0x7f0bdc06b9a0 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:23.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:23.065+0000 7f0be3714700 1 --2- 172.21.15.27:0/609453311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0bdc0f6ff0 0x7f0bdc069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:23.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:23.065+0000 7f0be3714700 1 -- 172.21.15.27:0/609453311 >> 172.21.15.27:0/609453311 conn(0x7f0bdc0f1f00 msgr2=0x7f0bdc0fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:28:23.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:23.065+0000 7f0be3714700 1 -- 172.21.15.27:0/609453311 shutdown_connections 2021-11-03T18:28:23.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:23.065+0000 7f0be3714700 1 -- 172.21.15.27:0/609453311 wait complete. 2021-11-03T18:28:23.074 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:28:23.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:23 smithi115 ceph-mon[65453]: from='client.36376 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:28:23.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:23 smithi115 ceph-mon[65453]: from='client.36380 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:28:23.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:23 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/475289627' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:23.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:23 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/609453311' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:28:24.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:23 smithi027 ceph-mon[97244]: from='client.36376 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:28:24.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:23 smithi027 ceph-mon[97244]: from='client.36380 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:28:24.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:23 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/475289627' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:24.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:23 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/609453311' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:28:24.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:24 smithi115 ceph-mon[65453]: pgmap v2214: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:25.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:24 smithi027 ceph-mon[97244]: pgmap v2214: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:25.734 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4347.463913 secs 2021-11-03T18:28:25.734 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:28:25.734 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:28:25.734 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:28:25.735 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:25.735 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:25.735 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:25.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4347.463913 secs 2021-11-03T18:28:25.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:28:25.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:28:25.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:28:25.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:25.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:25.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:25.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:26.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:26.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:26 smithi115 ceph-mon[65453]: pgmap v2215: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:26.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:26 smithi115 ceph-mon[65453]: poll took 60169.628 ms: 1 events 2021-11-03T18:28:26.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:26 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:28:27.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:26 smithi027 ceph-mon[97244]: pgmap v2215: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:27.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:26 smithi027 ceph-mon[97244]: poll took 60169.628 ms: 1 events 2021-11-03T18:28:27.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:26 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:28:29.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:28 smithi027 ceph-mon[97244]: pgmap v2216: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:29.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:28 smithi115 ceph-mon[65453]: pgmap v2216: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:31.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:30 smithi027 ceph-mon[97244]: pgmap v2217: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:31.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4352.464005 secs 2021-11-03T18:28:31.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:30 smithi115 ceph-mon[65453]: pgmap v2217: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:31.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4352.464005 secs 2021-11-03T18:28:33.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:32 smithi027 ceph-mon[97244]: pgmap v2218: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:33.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:32 smithi115 ceph-mon[65453]: pgmap v2218: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:35.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:34 smithi027 ceph-mon[97244]: pgmap v2219: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:35.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:34 smithi115 ceph-mon[65453]: pgmap v2219: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:35.802 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4357.464144 secs 2021-11-03T18:28:35.802 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:28:36.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4357.464144 secs 2021-11-03T18:28:36.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:28:37.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:36 smithi027 ceph-mon[97244]: pgmap v2220: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:37.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:36 smithi027 ceph-mon[97244]: poll took 10025.860 ms: 1 events 2021-11-03T18:28:37.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:28:37.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:28:37.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:28:37.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:37.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:37.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:37.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:37.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:36 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:28:37.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:36 smithi115 ceph-mon[65453]: pgmap v2220: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:37.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:36 smithi115 ceph-mon[65453]: poll took 10025.860 ms: 1 events 2021-11-03T18:28:37.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:28:37.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:28:37.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:28:37.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:37.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:37.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:37.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:37.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:36 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:28:38.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:37 smithi027 ceph-mon[97244]: pgmap v2221: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:38.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:37 smithi115 ceph-mon[65453]: pgmap v2221: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:40.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:40 smithi115 ceph-mon[65453]: pgmap v2222: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:40.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:40 smithi027 ceph-mon[97244]: pgmap v2222: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:41.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:41 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4362.464282 secs 2021-11-03T18:28:41.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:41 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4362.464282 secs 2021-11-03T18:28:42.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:42 smithi115 ceph-mon[65453]: pgmap v2223: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:42.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:42 smithi027 ceph-mon[97244]: pgmap v2223: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:44.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:44 smithi115 ceph-mon[65453]: pgmap v2224: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:44.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:44 smithi027 ceph-mon[97244]: pgmap v2224: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:45.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4367.464442 secs 2021-11-03T18:28:45.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4367.464442 secs 2021-11-03T18:28:46.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:46 smithi115 ceph-mon[65453]: pgmap v2225: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:46.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:28:46.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:28:46.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:28:46.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:46.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:46.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:46.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:46.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:46 smithi027 ceph-mon[97244]: pgmap v2225: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:46.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:28:46.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:28:46.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:28:46.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:46.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:46.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:46.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:47.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:47 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:28:47.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:47 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:28:48.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:48 smithi115 ceph-mon[65453]: pgmap v2226: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:48.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:48 smithi027 ceph-mon[97244]: pgmap v2226: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:50.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:50 smithi115 ceph-mon[65453]: pgmap v2227: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:50.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4372.464566 secs 2021-11-03T18:28:50.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:50 smithi027 ceph-mon[97244]: pgmap v2227: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:50.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4372.464566 secs 2021-11-03T18:28:52.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:52 smithi115 ceph-mon[65453]: pgmap v2228: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:52.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:52 smithi027 ceph-mon[97244]: pgmap v2228: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:53.178 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.180+0000 7fcd31c8b700 1 -- 172.21.15.27:0/2056999259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd2c0f6860 msgr2=0x7fcd2c0f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:28:53.179 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.180+0000 7fcd31c8b700 1 --2- 172.21.15.27:0/2056999259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd2c0f6860 0x7fcd2c0f6cc0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 rx=0x7fcd20004660 tx=0x7fcd20009b30).stop 2021-11-03T18:28:53.179 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.180+0000 7fcd31c8b700 1 -- 172.21.15.27:0/2056999259 shutdown_connections 2021-11-03T18:28:53.179 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.180+0000 7fcd31c8b700 1 --2- 172.21.15.27:0/2056999259 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd2c0f6860 0x7fcd2c0f6cc0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:53.179 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.180+0000 7fcd31c8b700 1 --2- 172.21.15.27:0/2056999259 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd2c0f55c0 0x7fcd2c0f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:53.180 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.180+0000 7fcd31c8b700 1 -- 172.21.15.27:0/2056999259 >> 172.21.15.27:0/2056999259 conn(0x7fcd2c0f0d40 msgr2=0x7fcd2c0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:28:53.180 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.180+0000 7fcd31c8b700 1 -- 172.21.15.27:0/2056999259 shutdown_connections 2021-11-03T18:28:53.180 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.180+0000 7fcd31c8b700 1 -- 172.21.15.27:0/2056999259 wait complete. 2021-11-03T18:28:53.180 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.181+0000 7fcd31c8b700 1 Processor -- start 2021-11-03T18:28:53.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.181+0000 7fcd31c8b700 1 -- start start 2021-11-03T18:28:53.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.181+0000 7fcd31c8b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd2c0f55c0 0x7fcd2c106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:28:53.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.181+0000 7fcd31c8b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd2c0f6860 0x7fcd2c107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:28:53.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.181+0000 7fcd31c8b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c107a10 con 0x7fcd2c0f6860 2021-11-03T18:28:53.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.181+0000 7fcd31c8b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd2c107b50 con 0x7fcd2c0f55c0 2021-11-03T18:28:53.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.181+0000 7fcd2affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd2c0f6860 0x7fcd2c107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:28:53.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.181+0000 7fcd2affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd2c0f6860 0x7fcd2c107460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35092/0 (socket says 172.21.15.27:35092) 2021-11-03T18:28:53.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.181+0000 7fcd2affd700 1 -- 172.21.15.27:0/1159600501 learned_addr learned my addr 172.21.15.27:0/1159600501 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:28:53.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.181+0000 7fcd2b7fe700 1 --2- 172.21.15.27:0/1159600501 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd2c0f55c0 0x7fcd2c106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:28:53.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.182+0000 7fcd2affd700 1 -- 172.21.15.27:0/1159600501 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd2c0f55c0 msgr2=0x7fcd2c106f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:28:53.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.182+0000 7fcd2affd700 1 --2- 172.21.15.27:0/1159600501 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd2c0f55c0 0x7fcd2c106f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:53.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.182+0000 7fcd2affd700 1 -- 172.21.15.27:0/1159600501 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd20005040 con 0x7fcd2c0f6860 2021-11-03T18:28:53.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.182+0000 7fcd2b7fe700 1 --2- 172.21.15.27:0/1159600501 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd2c0f55c0 0x7fcd2c106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:28:53.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.182+0000 7fcd2affd700 1 --2- 172.21.15.27:0/1159600501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd2c0f6860 0x7fcd2c107460 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 rx=0x7fcd20009990 tx=0x7fcd20009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:28:53.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.182+0000 7fcd13fff700 1 -- 172.21.15.27:0/1159600501 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd20018070 con 0x7fcd2c0f6860 2021-11-03T18:28:53.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.182+0000 7fcd13fff700 1 -- 172.21.15.27:0/1159600501 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fcd20007ba0 con 0x7fcd2c0f6860 2021-11-03T18:28:53.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.182+0000 7fcd31c8b700 1 -- 172.21.15.27:0/1159600501 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd2c108490 con 0x7fcd2c0f6860 2021-11-03T18:28:53.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.182+0000 7fcd13fff700 1 -- 172.21.15.27:0/1159600501 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcd2001c3c0 con 0x7fcd2c0f6860 2021-11-03T18:28:53.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.182+0000 7fcd31c8b700 1 -- 172.21.15.27:0/1159600501 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd2c1089f0 con 0x7fcd2c0f6860 2021-11-03T18:28:53.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.183+0000 7fcd31c8b700 1 -- 172.21.15.27:0/1159600501 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd2c045ad0 con 0x7fcd2c0f6860 2021-11-03T18:28:53.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.183+0000 7fcd13fff700 1 -- 172.21.15.27:0/1159600501 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fcd2001d420 con 0x7fcd2c0f6860 2021-11-03T18:28:53.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.184+0000 7fcd13fff700 1 --2- 172.21.15.27:0/1159600501 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcd140640b0 0x7fcd14066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:28:53.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.184+0000 7fcd13fff700 1 -- 172.21.15.27:0/1159600501 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fcd20082500 con 0x7fcd2c0f6860 2021-11-03T18:28:53.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.184+0000 7fcd2b7fe700 1 --2- 172.21.15.27:0/1159600501 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcd140640b0 0x7fcd14066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:28:53.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.184+0000 7fcd2b7fe700 1 --2- 172.21.15.27:0/1159600501 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcd140640b0 0x7fcd14066570 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7fcd2c0f66c0 tx=0x7fcd1c009c00).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:28:53.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.188+0000 7fcd13fff700 1 -- 172.21.15.27:0/1159600501 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fcd200548c0 con 0x7fcd2c0f6860 2021-11-03T18:28:53.365 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.367+0000 7fcd31c8b700 1 -- 172.21.15.27:0/1159600501 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd2c0faca0 con 0x7fcd140640b0 2021-11-03T18:28:53.366 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.369+0000 7fcd13fff700 1 -- 172.21.15.27:0/1159600501 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fcd2c0faca0 con 0x7fcd140640b0 2021-11-03T18:28:53.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.370+0000 7fcd31c8b700 1 -- 172.21.15.27:0/1159600501 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcd140640b0 msgr2=0x7fcd14066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:28:53.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.370+0000 7fcd31c8b700 1 --2- 172.21.15.27:0/1159600501 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcd140640b0 0x7fcd14066570 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7fcd2c0f66c0 tx=0x7fcd1c009c00).stop 2021-11-03T18:28:53.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.370+0000 7fcd31c8b700 1 -- 172.21.15.27:0/1159600501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd2c0f6860 msgr2=0x7fcd2c107460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:28:53.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.371+0000 7fcd31c8b700 1 --2- 172.21.15.27:0/1159600501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd2c0f6860 0x7fcd2c107460 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 rx=0x7fcd20009990 tx=0x7fcd20009b30).stop 2021-11-03T18:28:53.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.371+0000 7fcd31c8b700 1 -- 172.21.15.27:0/1159600501 shutdown_connections 2021-11-03T18:28:53.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.371+0000 7fcd31c8b700 1 --2- 172.21.15.27:0/1159600501 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcd140640b0 0x7fcd14066570 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:53.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.371+0000 7fcd31c8b700 1 --2- 172.21.15.27:0/1159600501 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcd2c0f55c0 0x7fcd2c106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:53.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.371+0000 7fcd31c8b700 1 --2- 172.21.15.27:0/1159600501 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcd2c0f6860 0x7fcd2c107460 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:53.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.371+0000 7fcd31c8b700 1 -- 172.21.15.27:0/1159600501 >> 172.21.15.27:0/1159600501 conn(0x7fcd2c0f0d40 msgr2=0x7fcd2c05fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:28:53.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.371+0000 7fcd31c8b700 1 -- 172.21.15.27:0/1159600501 shutdown_connections 2021-11-03T18:28:53.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.371+0000 7fcd31c8b700 1 -- 172.21.15.27:0/1159600501 wait complete. 2021-11-03T18:28:53.378 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:28:53.471 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.471+0000 7eff7d9fe700 1 -- 172.21.15.27:0/1371590632 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff780f5ef0 msgr2=0x7eff780f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:28:53.471 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.471+0000 7eff7d9fe700 1 --2- 172.21.15.27:0/1371590632 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff780f5ef0 0x7eff780f6920 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 rx=0x7eff60004660 tx=0x7eff60009b30).stop 2021-11-03T18:28:53.471 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.471+0000 7eff7d9fe700 1 -- 172.21.15.27:0/1371590632 shutdown_connections 2021-11-03T18:28:53.472 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.471+0000 7eff7d9fe700 1 --2- 172.21.15.27:0/1371590632 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff780f5ef0 0x7eff780f6920 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:53.472 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.471+0000 7eff7d9fe700 1 --2- 172.21.15.27:0/1371590632 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff780f55e0 0x7eff780f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:53.472 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.471+0000 7eff7d9fe700 1 -- 172.21.15.27:0/1371590632 >> 172.21.15.27:0/1371590632 conn(0x7eff780f0de0 msgr2=0x7eff780f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:28:53.472 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.471+0000 7eff7d9fe700 1 -- 172.21.15.27:0/1371590632 shutdown_connections 2021-11-03T18:28:53.473 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.471+0000 7eff7d9fe700 1 -- 172.21.15.27:0/1371590632 wait complete. 2021-11-03T18:28:53.473 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.472+0000 7eff7d9fe700 1 Processor -- start 2021-11-03T18:28:53.473 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.472+0000 7eff7d9fe700 1 -- start start 2021-11-03T18:28:53.473 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.472+0000 7eff7d9fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff780f55e0 0x7eff78106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:28:53.473 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.472+0000 7eff7d9fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff780f5ef0 0x7eff78107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:28:53.474 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.472+0000 7eff7d9fe700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff78107a10 con 0x7eff780f55e0 2021-11-03T18:28:53.474 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.472+0000 7eff7d9fe700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff78107b50 con 0x7eff780f5ef0 2021-11-03T18:28:53.474 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.472+0000 7eff76ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff780f55e0 0x7eff78106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:28:53.474 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.472+0000 7eff76ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff780f55e0 0x7eff78106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35100/0 (socket says 172.21.15.27:35100) 2021-11-03T18:28:53.475 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.472+0000 7eff76ffd700 1 -- 172.21.15.27:0/1729834057 learned_addr learned my addr 172.21.15.27:0/1729834057 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:28:53.475 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.472+0000 7eff767fc700 1 --2- 172.21.15.27:0/1729834057 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff780f5ef0 0x7eff78107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:28:53.475 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.472+0000 7eff76ffd700 1 -- 172.21.15.27:0/1729834057 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff780f5ef0 msgr2=0x7eff78107460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:28:53.475 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.472+0000 7eff76ffd700 1 --2- 172.21.15.27:0/1729834057 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff780f5ef0 0x7eff78107460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:53.475 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.472+0000 7eff76ffd700 1 -- 172.21.15.27:0/1729834057 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff60005040 con 0x7eff780f55e0 2021-11-03T18:28:53.476 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.472+0000 7eff767fc700 1 --2- 172.21.15.27:0/1729834057 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff780f5ef0 0x7eff78107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:28:53.476 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.472+0000 7eff76ffd700 1 --2- 172.21.15.27:0/1729834057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff780f55e0 0x7eff78106f70 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 rx=0x7eff70007fd0 tx=0x7eff70004ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:28:53.476 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.473+0000 7eff6f7fe700 1 -- 172.21.15.27:0/1729834057 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff7000b5a0 con 0x7eff780f55e0 2021-11-03T18:28:53.476 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.473+0000 7eff6f7fe700 1 -- 172.21.15.27:0/1729834057 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7eff7000f040 con 0x7eff780f55e0 2021-11-03T18:28:53.477 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.473+0000 7eff7d9fe700 1 -- 172.21.15.27:0/1729834057 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff781084f0 con 0x7eff780f55e0 2021-11-03T18:28:53.477 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.473+0000 7eff6f7fe700 1 -- 172.21.15.27:0/1729834057 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7eff70013b60 con 0x7eff780f55e0 2021-11-03T18:28:53.477 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.473+0000 7eff7d9fe700 1 -- 172.21.15.27:0/1729834057 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff78108ab0 con 0x7eff780f55e0 2021-11-03T18:28:53.478 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.474+0000 7eff7d9fe700 1 -- 172.21.15.27:0/1729834057 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff78045ad0 con 0x7eff780f55e0 2021-11-03T18:28:53.478 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.474+0000 7eff6f7fe700 1 -- 172.21.15.27:0/1729834057 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7eff7001a030 con 0x7eff780f55e0 2021-11-03T18:28:53.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.474+0000 7eff6f7fe700 1 --2- 172.21.15.27:0/1729834057 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7eff640640b0 0x7eff64066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:28:53.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.474+0000 7eff6f7fe700 1 -- 172.21.15.27:0/1729834057 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7eff70080720 con 0x7eff780f55e0 2021-11-03T18:28:53.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.474+0000 7eff767fc700 1 --2- 172.21.15.27:0/1729834057 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7eff640640b0 0x7eff64066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:28:53.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.475+0000 7eff767fc700 1 --2- 172.21.15.27:0/1729834057 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7eff640640b0 0x7eff64066570 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7eff60004630 tx=0x7eff60009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:28:53.480 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.479+0000 7eff6f7fe700 1 -- 172.21.15.27:0/1729834057 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7eff70052a40 con 0x7eff780f55e0 2021-11-03T18:28:53.650 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.652+0000 7eff7d9fe700 1 -- 172.21.15.27:0/1729834057 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7eff7805fe50 con 0x7eff640640b0 2021-11-03T18:28:53.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.666+0000 7eff6f7fe700 1 -- 172.21.15.27:0/1729834057 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7eff7805fe50 con 0x7eff640640b0 2021-11-03T18:28:53.664 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:28:53.665 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (80m) 9m ago 81m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:28:53.665 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (72m) 9m ago 81m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:28:53.665 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (71m) 8m ago 80m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:28:53.666 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (80m) 9m ago 81m 42.4M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:28:53.666 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (73m) 9m ago 76m 26.5M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:28:53.666 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (76m) 9m ago 76m 3849M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:28:53.666 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (72m) 8m ago 76m 26.1M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:28:53.666 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (74m) 8m ago 76m 27.2M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:28:53.667 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (74m) 9m ago 82m 590M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:28:53.667 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (73m) 8m ago 80m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:28:53.667 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (73m) 9m ago 83m 312M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:28:53.667 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (72m) 8m ago 80m 145M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:28:53.668 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (81m) 9m ago 81m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:28:53.668 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (80m) 8m ago 80m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:28:53.668 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (71m) 9m ago 80m 392M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:28:53.668 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (71m) 9m ago 79m 427M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:28:53.669 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (70m) 9m ago 79m 320M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:28:53.669 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (70m) 8m ago 79m 454M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:28:53.669 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (69m) 8m ago 78m 359M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:28:53.669 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (69m) 8m ago 78m 368M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:28:53.669 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (80m) 9m ago 81m 59.9M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:28:53.670 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.668+0000 7eff7d9fe700 1 -- 172.21.15.27:0/1729834057 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7eff640640b0 msgr2=0x7eff64066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:28:53.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.668+0000 7eff7d9fe700 1 --2- 172.21.15.27:0/1729834057 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7eff640640b0 0x7eff64066570 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 rx=0x7eff60004630 tx=0x7eff60009b30).stop 2021-11-03T18:28:53.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.668+0000 7eff7d9fe700 1 -- 172.21.15.27:0/1729834057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff780f55e0 msgr2=0x7eff78106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:28:53.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.668+0000 7eff7d9fe700 1 --2- 172.21.15.27:0/1729834057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff780f55e0 0x7eff78106f70 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 rx=0x7eff70007fd0 tx=0x7eff70004ee0).stop 2021-11-03T18:28:53.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.668+0000 7eff7d9fe700 1 -- 172.21.15.27:0/1729834057 shutdown_connections 2021-11-03T18:28:53.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.668+0000 7eff7d9fe700 1 --2- 172.21.15.27:0/1729834057 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7eff640640b0 0x7eff64066570 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:53.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.668+0000 7eff7d9fe700 1 --2- 172.21.15.27:0/1729834057 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7eff780f55e0 0x7eff78106f70 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:53.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.668+0000 7eff7d9fe700 1 --2- 172.21.15.27:0/1729834057 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7eff780f5ef0 0x7eff78107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:53.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.668+0000 7eff7d9fe700 1 -- 172.21.15.27:0/1729834057 >> 172.21.15.27:0/1729834057 conn(0x7eff780f0de0 msgr2=0x7eff7805f740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:28:53.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.668+0000 7eff7d9fe700 1 -- 172.21.15.27:0/1729834057 shutdown_connections 2021-11-03T18:28:53.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.668+0000 7eff7d9fe700 1 -- 172.21.15.27:0/1729834057 wait complete. 2021-11-03T18:28:53.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.768+0000 7f5a2ec4b700 1 -- 172.21.15.27:0/846886241 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a280f77d0 msgr2=0x7f5a280f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:28:53.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.768+0000 7f5a2ec4b700 1 --2- 172.21.15.27:0/846886241 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a280f77d0 0x7f5a280f7c30 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 rx=0x7f5a18004660 tx=0x7f5a18009b30).stop 2021-11-03T18:28:53.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.768+0000 7f5a2ec4b700 1 -- 172.21.15.27:0/846886241 shutdown_connections 2021-11-03T18:28:53.770 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.768+0000 7f5a2ec4b700 1 --2- 172.21.15.27:0/846886241 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a280f77d0 0x7f5a280f7c30 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:53.770 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.768+0000 7f5a2ec4b700 1 --2- 172.21.15.27:0/846886241 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a280f6530 0x7f5a280f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:53.770 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.768+0000 7f5a2ec4b700 1 -- 172.21.15.27:0/846886241 >> 172.21.15.27:0/846886241 conn(0x7f5a280f1c90 msgr2=0x7f5a280f40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:28:53.770 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.768+0000 7f5a2ec4b700 1 -- 172.21.15.27:0/846886241 shutdown_connections 2021-11-03T18:28:53.771 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.768+0000 7f5a2ec4b700 1 -- 172.21.15.27:0/846886241 wait complete. 2021-11-03T18:28:53.771 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.768+0000 7f5a2ec4b700 1 Processor -- start 2021-11-03T18:28:53.771 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.768+0000 7f5a2ec4b700 1 -- start start 2021-11-03T18:28:53.771 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.769+0000 7f5a2ec4b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a280f6530 0x7f5a2806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:28:53.771 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.769+0000 7f5a2ec4b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a280f77d0 0x7f5a28069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:28:53.772 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.769+0000 7f5a2ec4b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2806a380 con 0x7f5a280f77d0 2021-11-03T18:28:53.772 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.769+0000 7f5a2ec4b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a2806a4c0 con 0x7f5a280f6530 2021-11-03T18:28:53.773 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.769+0000 7f5a2d448700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a280f77d0 0x7f5a28069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:28:53.773 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.769+0000 7f5a2d448700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a280f77d0 0x7f5a28069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35112/0 (socket says 172.21.15.27:35112) 2021-11-03T18:28:53.773 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.769+0000 7f5a2d448700 1 -- 172.21.15.27:0/3563246019 learned_addr learned my addr 172.21.15.27:0/3563246019 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:28:53.774 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.769+0000 7f5a2dc49700 1 --2- 172.21.15.27:0/3563246019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a280f6530 0x7f5a2806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:28:53.774 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.769+0000 7f5a2d448700 1 -- 172.21.15.27:0/3563246019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a280f6530 msgr2=0x7f5a2806b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:28:53.774 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.769+0000 7f5a2d448700 1 --2- 172.21.15.27:0/3563246019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a280f6530 0x7f5a2806b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:53.774 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.769+0000 7f5a2d448700 1 -- 172.21.15.27:0/3563246019 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a18005040 con 0x7f5a280f77d0 2021-11-03T18:28:53.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.769+0000 7f5a2dc49700 1 --2- 172.21.15.27:0/3563246019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a280f6530 0x7f5a2806b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:28:53.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.769+0000 7f5a2d448700 1 --2- 172.21.15.27:0/3563246019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a280f77d0 0x7f5a28069e20 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 rx=0x7f5a18009990 tx=0x7f5a18009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:28:53.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.769+0000 7f5a1e7fc700 1 -- 172.21.15.27:0/3563246019 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a18018070 con 0x7f5a280f77d0 2021-11-03T18:28:53.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.769+0000 7f5a1e7fc700 1 -- 172.21.15.27:0/3563246019 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f5a18007ca0 con 0x7f5a280f77d0 2021-11-03T18:28:53.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.769+0000 7f5a2ec4b700 1 -- 172.21.15.27:0/3563246019 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a2806a740 con 0x7f5a280f77d0 2021-11-03T18:28:53.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.769+0000 7f5a1e7fc700 1 -- 172.21.15.27:0/3563246019 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5a1801c620 con 0x7f5a280f77d0 2021-11-03T18:28:53.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.769+0000 7f5a2ec4b700 1 -- 172.21.15.27:0/3563246019 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a2806aca0 con 0x7f5a280f77d0 2021-11-03T18:28:53.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.770+0000 7f5a2ec4b700 1 -- 172.21.15.27:0/3563246019 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a28045ad0 con 0x7f5a280f77d0 2021-11-03T18:28:53.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.771+0000 7f5a1e7fc700 1 -- 172.21.15.27:0/3563246019 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f5a1801d960 con 0x7f5a280f77d0 2021-11-03T18:28:53.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.771+0000 7f5a1e7fc700 1 --2- 172.21.15.27:0/3563246019 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5a14064100 0x7f5a140665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:28:53.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.771+0000 7f5a1e7fc700 1 -- 172.21.15.27:0/3563246019 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f5a18082600 con 0x7f5a280f77d0 2021-11-03T18:28:53.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.771+0000 7f5a2dc49700 1 --2- 172.21.15.27:0/3563246019 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5a14064100 0x7f5a140665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:28:53.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.772+0000 7f5a2dc49700 1 --2- 172.21.15.27:0/3563246019 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5a14064100 0x7f5a140665c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f5a2806b730 tx=0x7f5a24009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:28:53.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:53.776+0000 7f5a1e7fc700 1 -- 172.21.15.27:0/3563246019 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f5a18054960 con 0x7f5a280f77d0 2021-11-03T18:28:54.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.007+0000 7f5a2ec4b700 1 -- 172.21.15.27:0/3563246019 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f5a28027a00 con 0x7f5a280f77d0 2021-11-03T18:28:54.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.008+0000 7f5a1e7fc700 1 -- 172.21.15.27:0/3563246019 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f5a18054550 con 0x7f5a280f77d0 2021-11-03T18:28:54.006 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:28:54.007 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:28:54.007 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:28:54.007 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:28:54.007 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:28:54.008 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:28:54.008 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:28:54.008 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:28:54.008 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:28:54.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:28:54.009 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:28:54.009 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:28:54.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:28:54.009 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:28:54.010 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:28:54.010 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:28:54.010 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:28:54.010 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:28:54.011 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.010+0000 7f5a2ec4b700 1 -- 172.21.15.27:0/3563246019 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5a14064100 msgr2=0x7f5a140665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:28:54.011 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.010+0000 7f5a2ec4b700 1 --2- 172.21.15.27:0/3563246019 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5a14064100 0x7f5a140665c0 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 rx=0x7f5a2806b730 tx=0x7f5a24009040).stop 2021-11-03T18:28:54.012 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.010+0000 7f5a2ec4b700 1 -- 172.21.15.27:0/3563246019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a280f77d0 msgr2=0x7f5a28069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:28:54.012 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.010+0000 7f5a2ec4b700 1 --2- 172.21.15.27:0/3563246019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a280f77d0 0x7f5a28069e20 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 rx=0x7f5a18009990 tx=0x7f5a18009b30).stop 2021-11-03T18:28:54.012 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.010+0000 7f5a2ec4b700 1 -- 172.21.15.27:0/3563246019 shutdown_connections 2021-11-03T18:28:54.012 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.010+0000 7f5a2ec4b700 1 --2- 172.21.15.27:0/3563246019 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5a14064100 0x7f5a140665c0 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:54.013 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.010+0000 7f5a2ec4b700 1 --2- 172.21.15.27:0/3563246019 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5a280f6530 0x7f5a2806b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:54.013 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.010+0000 7f5a2ec4b700 1 --2- 172.21.15.27:0/3563246019 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5a280f77d0 0x7f5a28069e20 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:54.013 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.010+0000 7f5a2ec4b700 1 -- 172.21.15.27:0/3563246019 >> 172.21.15.27:0/3563246019 conn(0x7f5a280f1c90 msgr2=0x7f5a280fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:28:54.013 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.010+0000 7f5a2ec4b700 1 -- 172.21.15.27:0/3563246019 shutdown_connections 2021-11-03T18:28:54.013 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.010+0000 7f5a2ec4b700 1 -- 172.21.15.27:0/3563246019 wait complete. 2021-11-03T18:28:54.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.110+0000 7f8e474f9700 1 -- 172.21.15.27:0/948573746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e400f5ef0 msgr2=0x7f8e400f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:28:54.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.110+0000 7f8e474f9700 1 --2- 172.21.15.27:0/948573746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e400f5ef0 0x7f8e400f6920 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 rx=0x7f8e34004660 tx=0x7f8e34009b30).stop 2021-11-03T18:28:54.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.110+0000 7f8e474f9700 1 -- 172.21.15.27:0/948573746 shutdown_connections 2021-11-03T18:28:54.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.110+0000 7f8e474f9700 1 --2- 172.21.15.27:0/948573746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e400f5ef0 0x7f8e400f6920 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:54.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.110+0000 7f8e474f9700 1 --2- 172.21.15.27:0/948573746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e400f55e0 0x7f8e400f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:54.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.110+0000 7f8e474f9700 1 -- 172.21.15.27:0/948573746 >> 172.21.15.27:0/948573746 conn(0x7f8e400f0de0 msgr2=0x7f8e400f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:28:54.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.110+0000 7f8e474f9700 1 -- 172.21.15.27:0/948573746 shutdown_connections 2021-11-03T18:28:54.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.110+0000 7f8e474f9700 1 -- 172.21.15.27:0/948573746 wait complete. 2021-11-03T18:28:54.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.110+0000 7f8e474f9700 1 Processor -- start 2021-11-03T18:28:54.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.110+0000 7f8e474f9700 1 -- start start 2021-11-03T18:28:54.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.111+0000 7f8e474f9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e400f55e0 0x7f8e400fd800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:28:54.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.111+0000 7f8e474f9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e400f5ef0 0x7f8e400fbc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:28:54.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.111+0000 7f8e474f9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e400fc1e0 con 0x7f8e400f5ef0 2021-11-03T18:28:54.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.111+0000 7f8e474f9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e400fc320 con 0x7f8e400f55e0 2021-11-03T18:28:54.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.111+0000 7f8e44a94700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e400f5ef0 0x7f8e400fbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:28:54.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.111+0000 7f8e44a94700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e400f5ef0 0x7f8e400fbc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35122/0 (socket says 172.21.15.27:35122) 2021-11-03T18:28:54.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.111+0000 7f8e44a94700 1 -- 172.21.15.27:0/2612358652 learned_addr learned my addr 172.21.15.27:0/2612358652 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:28:54.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.111+0000 7f8e45295700 1 --2- 172.21.15.27:0/2612358652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e400f55e0 0x7f8e400fd800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:28:54.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.111+0000 7f8e44a94700 1 -- 172.21.15.27:0/2612358652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e400f55e0 msgr2=0x7f8e400fd800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:28:54.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.111+0000 7f8e44a94700 1 --2- 172.21.15.27:0/2612358652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e400f55e0 0x7f8e400fd800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:54.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.111+0000 7f8e44a94700 1 -- 172.21.15.27:0/2612358652 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e34005040 con 0x7f8e400f5ef0 2021-11-03T18:28:54.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.111+0000 7f8e44a94700 1 --2- 172.21.15.27:0/2612358652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e400f5ef0 0x7f8e400fbc80 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 rx=0x7f8e34009990 tx=0x7f8e34009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:28:54.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.111+0000 7f8e3dffb700 1 -- 172.21.15.27:0/2612358652 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e34018070 con 0x7f8e400f5ef0 2021-11-03T18:28:54.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.111+0000 7f8e3dffb700 1 -- 172.21.15.27:0/2612358652 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f8e34007ba0 con 0x7f8e400f5ef0 2021-11-03T18:28:54.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.111+0000 7f8e474f9700 1 -- 172.21.15.27:0/2612358652 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e400fc5a0 con 0x7f8e400f5ef0 2021-11-03T18:28:54.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.111+0000 7f8e3dffb700 1 -- 172.21.15.27:0/2612358652 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e3401c620 con 0x7f8e400f5ef0 2021-11-03T18:28:54.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.111+0000 7f8e474f9700 1 -- 172.21.15.27:0/2612358652 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e400fcb00 con 0x7f8e400f5ef0 2021-11-03T18:28:54.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.112+0000 7f8e474f9700 1 -- 172.21.15.27:0/2612358652 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e40045ad0 con 0x7f8e400f5ef0 2021-11-03T18:28:54.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.113+0000 7f8e3dffb700 1 -- 172.21.15.27:0/2612358652 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f8e3401c890 con 0x7f8e400f5ef0 2021-11-03T18:28:54.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.113+0000 7f8e3dffb700 1 --2- 172.21.15.27:0/2612358652 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8e280640b0 0x7f8e28066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:28:54.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.113+0000 7f8e3dffb700 1 -- 172.21.15.27:0/2612358652 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f8e34082520 con 0x7f8e400f5ef0 2021-11-03T18:28:54.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.113+0000 7f8e45295700 1 --2- 172.21.15.27:0/2612358652 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8e280640b0 0x7f8e28066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:28:54.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.114+0000 7f8e45295700 1 --2- 172.21.15.27:0/2612358652 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8e280640b0 0x7f8e28066570 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f8e38004fd0 tx=0x7f8e38009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:28:54.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.117+0000 7f8e3dffb700 1 -- 172.21.15.27:0/2612358652 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f8e34054940 con 0x7f8e400f5ef0 2021-11-03T18:28:54.319 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.321+0000 7f8e474f9700 1 -- 172.21.15.27:0/2612358652 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8e40027a00 con 0x7f8e400f5ef0 2021-11-03T18:28:54.319 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.321+0000 7f8e3dffb700 1 -- 172.21.15.27:0/2612358652 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f8e34054530 con 0x7f8e400f5ef0 2021-11-03T18:28:54.320 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:28:54.320 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:28:54.320 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:28:54.320 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:28:54.321 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:28:54.321 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:28:54.321 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:28:54.321 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:28:54.322 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:28:54.322 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:28:54.322 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:28:54.322 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:28:54.323 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:28:54.323 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:28:54.323 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:28:54.323 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:28:54.323 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:28:54.324 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:28:54.324 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:28:54.324 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:28:54.324 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:28:54.324 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:28:54.325 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:28:54.325 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:28:54.325 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:28:54.325 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:28:54.326 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:28:54.326 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:28:54.326 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:28:54.326 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:28:54.326 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:28:54.327 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:28:54.327 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:28:54.327 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:28:54.327 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:28:54.327 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:28:54.328 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:28:54.328 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:28:54.328 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:28:54.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.323+0000 7f8e474f9700 1 -- 172.21.15.27:0/2612358652 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8e280640b0 msgr2=0x7f8e28066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:28:54.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.323+0000 7f8e474f9700 1 --2- 172.21.15.27:0/2612358652 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8e280640b0 0x7f8e28066570 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f8e38004fd0 tx=0x7f8e38009040).stop 2021-11-03T18:28:54.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.323+0000 7f8e474f9700 1 -- 172.21.15.27:0/2612358652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e400f5ef0 msgr2=0x7f8e400fbc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:28:54.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.323+0000 7f8e474f9700 1 --2- 172.21.15.27:0/2612358652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e400f5ef0 0x7f8e400fbc80 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 rx=0x7f8e34009990 tx=0x7f8e34009b30).stop 2021-11-03T18:28:54.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.323+0000 7f8e474f9700 1 -- 172.21.15.27:0/2612358652 shutdown_connections 2021-11-03T18:28:54.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.323+0000 7f8e474f9700 1 --2- 172.21.15.27:0/2612358652 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8e280640b0 0x7f8e28066570 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:54.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.323+0000 7f8e474f9700 1 --2- 172.21.15.27:0/2612358652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e400f55e0 0x7f8e400fd800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:54.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.323+0000 7f8e474f9700 1 --2- 172.21.15.27:0/2612358652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e400f5ef0 0x7f8e400fbc80 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:28:54.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.323+0000 7f8e474f9700 1 -- 172.21.15.27:0/2612358652 >> 172.21.15.27:0/2612358652 conn(0x7f8e400f0de0 msgr2=0x7f8e400fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:28:54.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.323+0000 7f8e474f9700 1 -- 172.21.15.27:0/2612358652 shutdown_connections 2021-11-03T18:28:54.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:28:54.323+0000 7f8e474f9700 1 -- 172.21.15.27:0/2612358652 wait complete. 2021-11-03T18:28:54.332 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:28:54.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:54 smithi115 ceph-mon[65453]: pgmap v2229: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:54.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:54 smithi115 ceph-mon[65453]: from='client.36392 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:28:54.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:54 smithi115 ceph-mon[65453]: from='client.36396 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:28:54.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:54 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3563246019' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:54.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:54 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2612358652' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:28:54.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:54 smithi027 ceph-mon[97244]: pgmap v2229: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:54.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:54 smithi027 ceph-mon[97244]: from='client.36392 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:28:54.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:54 smithi027 ceph-mon[97244]: from='client.36396 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:28:54.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:54 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3563246019' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:54.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:54 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2612358652' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:28:55.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4377.464628 secs 2021-11-03T18:28:55.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4377.464628 secs 2021-11-03T18:28:56.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:56 smithi115 ceph-mon[65453]: pgmap v2230: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:56.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:28:56.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:28:56.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:28:56.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:28:56.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:28:56.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:56.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:56.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:56.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:56.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:56 smithi027 ceph-mon[97244]: pgmap v2230: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:56.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:28:56.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:28:56.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:28:56.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:28:56.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:28:56.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:56.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:56.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:56.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:28:57.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:57 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:28:57.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:57 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:28:58.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:28:58 smithi115 ceph-mon[65453]: pgmap v2231: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:28:58.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:28:58 smithi027 ceph-mon[97244]: pgmap v2231: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:00.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:00 smithi115 ceph-mon[65453]: pgmap v2232: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:00.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4382.464715 secs 2021-11-03T18:29:00.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:00 smithi027 ceph-mon[97244]: pgmap v2232: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:00.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4382.464715 secs 2021-11-03T18:29:02.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:02 smithi115 ceph-mon[65453]: pgmap v2233: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:02.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:02 smithi027 ceph-mon[97244]: pgmap v2233: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:04.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:04 smithi115 ceph-mon[65453]: pgmap v2234: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:04.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:04 smithi027 ceph-mon[97244]: pgmap v2234: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:05.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4387.464837 secs 2021-11-03T18:29:05.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4387.464837 secs 2021-11-03T18:29:06.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:06 smithi115 ceph-mon[65453]: pgmap v2235: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:06.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:29:06.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:29:06.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:29:06.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:06.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:06.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:06.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:06.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:06 smithi027 ceph-mon[97244]: pgmap v2235: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:06.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:29:06.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:29:06.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:29:06.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:06.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:06.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:06.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:07.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:07 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:29:07.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:07 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:29:08.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:08 smithi115 ceph-mon[65453]: pgmap v2236: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:08.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:08 smithi027 ceph-mon[97244]: pgmap v2236: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:10.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:10 smithi115 ceph-mon[65453]: pgmap v2237: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:10.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4392.464976 secs 2021-11-03T18:29:10.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:10 smithi027 ceph-mon[97244]: pgmap v2237: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:10.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4392.464976 secs 2021-11-03T18:29:12.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:12 smithi115 ceph-mon[65453]: pgmap v2238: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:12.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:12 smithi027 ceph-mon[97244]: pgmap v2238: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:14.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:14 smithi115 ceph-mon[65453]: pgmap v2239: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:14.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:14 smithi027 ceph-mon[97244]: pgmap v2239: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:15.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4397.465115 secs 2021-11-03T18:29:15.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4397.465115 secs 2021-11-03T18:29:16.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:16 smithi115 ceph-mon[65453]: pgmap v2240: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:16.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:29:16.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:29:16.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:29:16.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:16.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:16.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:16.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:16.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:16 smithi027 ceph-mon[97244]: pgmap v2240: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:16.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:29:16.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:29:16.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:29:16.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:16.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:16.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:16.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:17.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:17 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:29:17.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:17 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:29:18.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:18 smithi115 ceph-mon[65453]: pgmap v2241: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:18.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:18 smithi027 ceph-mon[97244]: pgmap v2241: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:20.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:20 smithi115 ceph-mon[65453]: pgmap v2242: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:20.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4402.465252 secs 2021-11-03T18:29:20.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:20 smithi027 ceph-mon[97244]: pgmap v2242: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:20.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4402.465252 secs 2021-11-03T18:29:22.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:22 smithi115 ceph-mon[65453]: pgmap v2243: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:22.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:22 smithi027 ceph-mon[97244]: pgmap v2243: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:24.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.438+0000 7f816a0e7700 1 -- 172.21.15.27:0/1959959409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81640f6530 msgr2=0x7f81640f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:29:24.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.438+0000 7f816a0e7700 1 --2- 172.21.15.27:0/1959959409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81640f6530 0x7f81640f6950 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 rx=0x7f8160004660 tx=0x7f8160009b30).stop 2021-11-03T18:29:24.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.438+0000 7f816a0e7700 1 -- 172.21.15.27:0/1959959409 shutdown_connections 2021-11-03T18:29:24.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.438+0000 7f816a0e7700 1 --2- 172.21.15.27:0/1959959409 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81640f77d0 0x7f81640f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:24.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.438+0000 7f816a0e7700 1 --2- 172.21.15.27:0/1959959409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81640f6530 0x7f81640f6950 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:24.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.438+0000 7f816a0e7700 1 -- 172.21.15.27:0/1959959409 >> 172.21.15.27:0/1959959409 conn(0x7f81640f1cd0 msgr2=0x7f81640f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:29:24.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.438+0000 7f816a0e7700 1 -- 172.21.15.27:0/1959959409 shutdown_connections 2021-11-03T18:29:24.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.438+0000 7f816a0e7700 1 -- 172.21.15.27:0/1959959409 wait complete. 2021-11-03T18:29:24.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.438+0000 7f816a0e7700 1 Processor -- start 2021-11-03T18:29:24.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.438+0000 7f816a0e7700 1 -- start start 2021-11-03T18:29:24.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.439+0000 7f816a0e7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81640f6530 0x7f8164106dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:29:24.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.439+0000 7f816a0e7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81640f77d0 0x7f81641072c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:29:24.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.439+0000 7f816a0e7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81641078c0 con 0x7f81640f6530 2021-11-03T18:29:24.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.439+0000 7f816a0e7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8164108100 con 0x7f81640f77d0 2021-11-03T18:29:24.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.439+0000 7f81690e5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81640f6530 0x7f8164106dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:29:24.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.439+0000 7f81690e5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81640f6530 0x7f8164106dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35138/0 (socket says 172.21.15.27:35138) 2021-11-03T18:29:24.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.439+0000 7f81690e5700 1 -- 172.21.15.27:0/2561947937 learned_addr learned my addr 172.21.15.27:0/2561947937 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:29:24.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.439+0000 7f81688e4700 1 --2- 172.21.15.27:0/2561947937 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81640f77d0 0x7f81641072c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:29:24.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.439+0000 7f81690e5700 1 -- 172.21.15.27:0/2561947937 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81640f77d0 msgr2=0x7f81641072c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:29:24.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.439+0000 7f81690e5700 1 --2- 172.21.15.27:0/2561947937 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81640f77d0 0x7f81641072c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:24.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.439+0000 7f81690e5700 1 -- 172.21.15.27:0/2561947937 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8160005040 con 0x7f81640f6530 2021-11-03T18:29:24.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.439+0000 7f81690e5700 1 --2- 172.21.15.27:0/2561947937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81640f6530 0x7f8164106dd0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 rx=0x7f8160009830 tx=0x7f8160009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:29:24.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.439+0000 7f8159ffb700 1 -- 172.21.15.27:0/2561947937 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8160018070 con 0x7f81640f6530 2021-11-03T18:29:24.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.439+0000 7f816a0e7700 1 -- 172.21.15.27:0/2561947937 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8164108380 con 0x7f81640f6530 2021-11-03T18:29:24.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.439+0000 7f8159ffb700 1 -- 172.21.15.27:0/2561947937 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f8160007ba0 con 0x7f81640f6530 2021-11-03T18:29:24.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.439+0000 7f8159ffb700 1 -- 172.21.15.27:0/2561947937 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f816001c620 con 0x7f81640f6530 2021-11-03T18:29:24.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.440+0000 7f816a0e7700 1 -- 172.21.15.27:0/2561947937 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8164108860 con 0x7f81640f6530 2021-11-03T18:29:24.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.441+0000 7f816a0e7700 1 -- 172.21.15.27:0/2561947937 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8164059100 con 0x7f81640f6530 2021-11-03T18:29:24.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.441+0000 7f8159ffb700 1 -- 172.21.15.27:0/2561947937 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f8160007d10 con 0x7f81640f6530 2021-11-03T18:29:24.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.441+0000 7f8159ffb700 1 --2- 172.21.15.27:0/2561947937 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8150064100 0x7f81500665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:29:24.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.442+0000 7f8159ffb700 1 -- 172.21.15.27:0/2561947937 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f8160082540 con 0x7f81640f6530 2021-11-03T18:29:24.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.442+0000 7f81688e4700 1 --2- 172.21.15.27:0/2561947937 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8150064100 0x7f81500665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:29:24.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.442+0000 7f81688e4700 1 --2- 172.21.15.27:0/2561947937 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8150064100 0x7f81500665c0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f8164106b60 tx=0x7f815400b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:29:24.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.446+0000 7f8159ffb700 1 -- 172.21.15.27:0/2561947937 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f8160054960 con 0x7f81640f6530 2021-11-03T18:29:24.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.624+0000 7f816a0e7700 1 -- 172.21.15.27:0/2561947937 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f81640fee20 con 0x7f8150064100 2021-11-03T18:29:24.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.626+0000 7f8159ffb700 1 -- 172.21.15.27:0/2561947937 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f81640fee20 con 0x7f8150064100 2021-11-03T18:29:24.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.628+0000 7f816a0e7700 1 -- 172.21.15.27:0/2561947937 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8150064100 msgr2=0x7f81500665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:29:24.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.628+0000 7f816a0e7700 1 --2- 172.21.15.27:0/2561947937 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8150064100 0x7f81500665c0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f8164106b60 tx=0x7f815400b040).stop 2021-11-03T18:29:24.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.628+0000 7f816a0e7700 1 -- 172.21.15.27:0/2561947937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81640f6530 msgr2=0x7f8164106dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:29:24.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.628+0000 7f816a0e7700 1 --2- 172.21.15.27:0/2561947937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81640f6530 0x7f8164106dd0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 rx=0x7f8160009830 tx=0x7f8160009b30).stop 2021-11-03T18:29:24.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.628+0000 7f816a0e7700 1 -- 172.21.15.27:0/2561947937 shutdown_connections 2021-11-03T18:29:24.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.628+0000 7f816a0e7700 1 --2- 172.21.15.27:0/2561947937 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8150064100 0x7f81500665c0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:24.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.628+0000 7f816a0e7700 1 --2- 172.21.15.27:0/2561947937 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f81640f6530 0x7f8164106dd0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:24.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.628+0000 7f816a0e7700 1 --2- 172.21.15.27:0/2561947937 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f81640f77d0 0x7f81641072c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:24.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.628+0000 7f816a0e7700 1 -- 172.21.15.27:0/2561947937 >> 172.21.15.27:0/2561947937 conn(0x7f81640f1cd0 msgr2=0x7f81640fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:29:24.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.628+0000 7f816a0e7700 1 -- 172.21.15.27:0/2561947937 shutdown_connections 2021-11-03T18:29:24.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.628+0000 7f816a0e7700 1 -- 172.21.15.27:0/2561947937 wait complete. 2021-11-03T18:29:24.635 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:29:24.727 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.728+0000 7f1d25a48700 1 -- 172.21.15.27:0/282449545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d200f77d0 msgr2=0x7f1d200f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:29:24.727 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.728+0000 7f1d25a48700 1 --2- 172.21.15.27:0/282449545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d200f77d0 0x7f1d200f7c30 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 rx=0x7f1d0c004660 tx=0x7f1d0c009b30).stop 2021-11-03T18:29:24.728 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.728+0000 7f1d25a48700 1 -- 172.21.15.27:0/282449545 shutdown_connections 2021-11-03T18:29:24.728 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.728+0000 7f1d25a48700 1 --2- 172.21.15.27:0/282449545 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d200f77d0 0x7f1d200f7c30 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:24.728 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.728+0000 7f1d25a48700 1 --2- 172.21.15.27:0/282449545 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d200f6530 0x7f1d200f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:24.728 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.728+0000 7f1d25a48700 1 -- 172.21.15.27:0/282449545 >> 172.21.15.27:0/282449545 conn(0x7f1d200f1cd0 msgr2=0x7f1d200f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:29:24.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.728+0000 7f1d25a48700 1 -- 172.21.15.27:0/282449545 shutdown_connections 2021-11-03T18:29:24.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.729+0000 7f1d25a48700 1 -- 172.21.15.27:0/282449545 wait complete. 2021-11-03T18:29:24.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.729+0000 7f1d25a48700 1 Processor -- start 2021-11-03T18:29:24.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.729+0000 7f1d25a48700 1 -- start start 2021-11-03T18:29:24.730 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.729+0000 7f1d25a48700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d200f6530 0x7f1d20106d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:29:24.730 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.729+0000 7f1d25a48700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d200f77d0 0x7f1d20107270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:29:24.730 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.729+0000 7f1d25a48700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d20107870 con 0x7f1d200f77d0 2021-11-03T18:29:24.730 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.729+0000 7f1d25a48700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d20108080 con 0x7f1d200f6530 2021-11-03T18:29:24.731 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.730+0000 7f1d1ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d200f77d0 0x7f1d20107270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:29:24.731 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.730+0000 7f1d1ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d200f77d0 0x7f1d20107270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35150/0 (socket says 172.21.15.27:35150) 2021-11-03T18:29:24.731 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.730+0000 7f1d1ffff700 1 -- 172.21.15.27:0/4074876673 learned_addr learned my addr 172.21.15.27:0/4074876673 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:29:24.733 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.730+0000 7f1d1ffff700 1 -- 172.21.15.27:0/4074876673 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d200f6530 msgr2=0x7f1d20106d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:29:24.733 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.730+0000 7f1d24a46700 1 --2- 172.21.15.27:0/4074876673 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d200f6530 0x7f1d20106d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:29:24.734 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.730+0000 7f1d1ffff700 1 --2- 172.21.15.27:0/4074876673 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d200f6530 0x7f1d20106d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:24.734 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.730+0000 7f1d1ffff700 1 -- 172.21.15.27:0/4074876673 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d0c005040 con 0x7f1d200f77d0 2021-11-03T18:29:24.734 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.730+0000 7f1d1ffff700 1 --2- 172.21.15.27:0/4074876673 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d200f77d0 0x7f1d20107270 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 rx=0x7f1d0c009da0 tx=0x7f1d0c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:29:24.734 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.730+0000 7f1d1d7fa700 1 -- 172.21.15.27:0/4074876673 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d0c018070 con 0x7f1d200f77d0 2021-11-03T18:29:24.735 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.730+0000 7f1d1d7fa700 1 -- 172.21.15.27:0/4074876673 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f1d0c007d70 con 0x7f1d200f77d0 2021-11-03T18:29:24.735 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.730+0000 7f1d25a48700 1 -- 172.21.15.27:0/4074876673 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d201082a0 con 0x7f1d200f77d0 2021-11-03T18:29:24.735 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.730+0000 7f1d1d7fa700 1 -- 172.21.15.27:0/4074876673 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d0c01dea0 con 0x7f1d200f77d0 2021-11-03T18:29:24.735 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.730+0000 7f1d25a48700 1 -- 172.21.15.27:0/4074876673 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d20108800 con 0x7f1d200f77d0 2021-11-03T18:29:24.736 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.731+0000 7f1d25a48700 1 -- 172.21.15.27:0/4074876673 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d20059100 con 0x7f1d200f77d0 2021-11-03T18:29:24.736 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.732+0000 7f1d1d7fa700 1 -- 172.21.15.27:0/4074876673 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f1d0c01d420 con 0x7f1d200f77d0 2021-11-03T18:29:24.736 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.732+0000 7f1d1d7fa700 1 --2- 172.21.15.27:0/4074876673 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1d10064100 0x7f1d100665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:29:24.736 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.732+0000 7f1d1d7fa700 1 -- 172.21.15.27:0/4074876673 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f1d0c082670 con 0x7f1d200f77d0 2021-11-03T18:29:24.736 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.732+0000 7f1d24a46700 1 --2- 172.21.15.27:0/4074876673 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1d10064100 0x7f1d100665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:29:24.737 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.732+0000 7f1d24a46700 1 --2- 172.21.15.27:0/4074876673 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1d10064100 0x7f1d100665c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7f1d20106af0 tx=0x7f1d14009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:29:24.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:24 smithi115 ceph-mon[65453]: pgmap v2244: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:24.738 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.736+0000 7f1d1d7fa700 1 -- 172.21.15.27:0/4074876673 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f1d0c054a90 con 0x7f1d200f77d0 2021-11-03T18:29:24.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:24 smithi027 ceph-mon[97244]: pgmap v2244: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:24.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.908+0000 7f1d25a48700 1 -- 172.21.15.27:0/4074876673 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d200008d0 con 0x7f1d10064100 2021-11-03T18:29:24.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.922+0000 7f1d1d7fa700 1 -- 172.21.15.27:0/4074876673 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f1d200008d0 con 0x7f1d10064100 2021-11-03T18:29:24.920 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:29:24.921 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (80m) 9m ago 82m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:29:24.921 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (72m) 9m ago 82m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:29:24.921 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (72m) 9m ago 81m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:29:24.921 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (80m) 9m ago 81m 42.4M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:29:24.922 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (74m) 9m ago 77m 26.5M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:29:24.922 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (77m) 9m ago 77m 3849M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:29:24.922 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (73m) 9m ago 77m 26.1M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:29:24.923 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (74m) 9m ago 77m 27.2M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:29:24.923 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (74m) 9m ago 83m 590M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:29:24.923 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (74m) 9m ago 81m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:29:24.923 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (73m) 9m ago 83m 312M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:29:24.924 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (73m) 9m ago 81m 145M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:29:24.924 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (81m) 9m ago 81m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:29:24.924 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (81m) 9m ago 81m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:29:24.924 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (72m) 9m ago 80m 392M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:29:24.924 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (71m) 9m ago 80m 427M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:29:24.925 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (71m) 9m ago 79m 320M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:29:24.925 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (70m) 9m ago 79m 454M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:29:24.925 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (70m) 9m ago 79m 359M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:29:24.925 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (69m) 9m ago 78m 368M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:29:24.926 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (80m) 9m ago 81m 59.9M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:29:24.927 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.924+0000 7f1d25a48700 1 -- 172.21.15.27:0/4074876673 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1d10064100 msgr2=0x7f1d100665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:29:24.927 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.924+0000 7f1d25a48700 1 --2- 172.21.15.27:0/4074876673 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1d10064100 0x7f1d100665c0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 rx=0x7f1d20106af0 tx=0x7f1d14009040).stop 2021-11-03T18:29:24.927 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.924+0000 7f1d25a48700 1 -- 172.21.15.27:0/4074876673 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d200f77d0 msgr2=0x7f1d20107270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:29:24.927 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.924+0000 7f1d25a48700 1 --2- 172.21.15.27:0/4074876673 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d200f77d0 0x7f1d20107270 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 rx=0x7f1d0c009da0 tx=0x7f1d0c009b30).stop 2021-11-03T18:29:24.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.924+0000 7f1d25a48700 1 -- 172.21.15.27:0/4074876673 shutdown_connections 2021-11-03T18:29:24.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.924+0000 7f1d25a48700 1 --2- 172.21.15.27:0/4074876673 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1d10064100 0x7f1d100665c0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:24.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.924+0000 7f1d25a48700 1 --2- 172.21.15.27:0/4074876673 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1d200f6530 0x7f1d20106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:24.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.924+0000 7f1d25a48700 1 --2- 172.21.15.27:0/4074876673 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1d200f77d0 0x7f1d20107270 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:24.929 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.924+0000 7f1d25a48700 1 -- 172.21.15.27:0/4074876673 >> 172.21.15.27:0/4074876673 conn(0x7f1d200f1cd0 msgr2=0x7f1d200fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:29:24.929 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.924+0000 7f1d25a48700 1 -- 172.21.15.27:0/4074876673 shutdown_connections 2021-11-03T18:29:24.929 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:24.924+0000 7f1d25a48700 1 -- 172.21.15.27:0/4074876673 wait complete. 2021-11-03T18:29:25.023 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.023+0000 7f700eed5700 1 -- 172.21.15.27:0/1410479891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70080f66e0 msgr2=0x7f70080f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:29:25.024 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.023+0000 7f700eed5700 1 --2- 172.21.15.27:0/1410479891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70080f66e0 0x7f70080f6b40 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 rx=0x7f6ff8004660 tx=0x7f6ff8009b30).stop 2021-11-03T18:29:25.024 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.023+0000 7f700eed5700 1 -- 172.21.15.27:0/1410479891 shutdown_connections 2021-11-03T18:29:25.024 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.023+0000 7f700eed5700 1 --2- 172.21.15.27:0/1410479891 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70080f66e0 0x7f70080f6b40 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:25.024 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.023+0000 7f700eed5700 1 --2- 172.21.15.27:0/1410479891 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70080f5440 0x7f70080f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:25.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.023+0000 7f700eed5700 1 -- 172.21.15.27:0/1410479891 >> 172.21.15.27:0/1410479891 conn(0x7f70080f0ba0 msgr2=0x7f70080f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:29:25.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.023+0000 7f700eed5700 1 -- 172.21.15.27:0/1410479891 shutdown_connections 2021-11-03T18:29:25.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.023+0000 7f700eed5700 1 -- 172.21.15.27:0/1410479891 wait complete. 2021-11-03T18:29:25.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.024+0000 7f700eed5700 1 Processor -- start 2021-11-03T18:29:25.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.024+0000 7f700eed5700 1 -- start start 2021-11-03T18:29:25.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.024+0000 7f700eed5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70080f5440 0x7f7008109a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:29:25.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.024+0000 7f700eed5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70080f66e0 0x7f700810bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:29:25.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.024+0000 7f700eed5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70080603e0 con 0x7f70080f66e0 2021-11-03T18:29:25.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.024+0000 7f700eed5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7008060520 con 0x7f70080f5440 2021-11-03T18:29:25.027 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.024+0000 7f700d6d2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70080f66e0 0x7f700810bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:29:25.027 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.024+0000 7f700d6d2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70080f66e0 0x7f700810bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35160/0 (socket says 172.21.15.27:35160) 2021-11-03T18:29:25.027 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.024+0000 7f700d6d2700 1 -- 172.21.15.27:0/1526095817 learned_addr learned my addr 172.21.15.27:0/1526095817 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:29:25.027 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.024+0000 7f700ded3700 1 --2- 172.21.15.27:0/1526095817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70080f5440 0x7f7008109a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:29:25.028 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.025+0000 7f700d6d2700 1 -- 172.21.15.27:0/1526095817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70080f5440 msgr2=0x7f7008109a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:29:25.028 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.025+0000 7f700d6d2700 1 --2- 172.21.15.27:0/1526095817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70080f5440 0x7f7008109a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:25.028 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.025+0000 7f700d6d2700 1 -- 172.21.15.27:0/1526095817 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ff8005040 con 0x7f70080f66e0 2021-11-03T18:29:25.028 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.025+0000 7f700d6d2700 1 --2- 172.21.15.27:0/1526095817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70080f66e0 0x7f700810bf90 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 rx=0x7f6ff8009990 tx=0x7f6ff8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:29:25.029 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.025+0000 7f6ffe7fc700 1 -- 172.21.15.27:0/1526095817 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ff8018070 con 0x7f70080f66e0 2021-11-03T18:29:25.029 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.025+0000 7f6ffe7fc700 1 -- 172.21.15.27:0/1526095817 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f6ff8007ba0 con 0x7f70080f66e0 2021-11-03T18:29:25.029 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.025+0000 7f700eed5700 1 -- 172.21.15.27:0/1526095817 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f700810c4e0 con 0x7f70080f66e0 2021-11-03T18:29:25.030 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.025+0000 7f6ffe7fc700 1 -- 172.21.15.27:0/1526095817 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6ff801c620 con 0x7f70080f66e0 2021-11-03T18:29:25.030 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.025+0000 7f700eed5700 1 -- 172.21.15.27:0/1526095817 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f700810ca40 con 0x7f70080f66e0 2021-11-03T18:29:25.032 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.026+0000 7f700eed5700 1 -- 172.21.15.27:0/1526095817 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7008045ad0 con 0x7f70080f66e0 2021-11-03T18:29:25.032 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.026+0000 7f6ffe7fc700 1 -- 172.21.15.27:0/1526095817 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f6ff801c8e0 con 0x7f70080f66e0 2021-11-03T18:29:25.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.027+0000 7f6ffe7fc700 1 --2- 172.21.15.27:0/1526095817 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6ff4064100 0x7f6ff40665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:29:25.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.027+0000 7f6ffe7fc700 1 -- 172.21.15.27:0/1526095817 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f6ff8082600 con 0x7f70080f66e0 2021-11-03T18:29:25.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.027+0000 7f700ded3700 1 --2- 172.21.15.27:0/1526095817 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6ff4064100 0x7f6ff40665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:29:25.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.027+0000 7f700ded3700 1 --2- 172.21.15.27:0/1526095817 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6ff4064100 0x7f6ff40665c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f7008060060 tx=0x7f7004009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:29:25.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.031+0000 7f6ffe7fc700 1 -- 172.21.15.27:0/1526095817 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f6ff8054a20 con 0x7f70080f66e0 2021-11-03T18:29:25.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.261+0000 7f700eed5700 1 -- 172.21.15.27:0/1526095817 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f70080fdca0 con 0x7f70080f66e0 2021-11-03T18:29:25.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.262+0000 7f6ffe7fc700 1 -- 172.21.15.27:0/1526095817 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f6ff8054610 con 0x7f70080f66e0 2021-11-03T18:29:25.260 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:29:25.261 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:29:25.261 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:29:25.261 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:29:25.261 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:29:25.261 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:29:25.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:29:25.262 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:29:25.262 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:29:25.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:29:25.263 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:29:25.263 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:29:25.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:29:25.263 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:29:25.264 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:29:25.264 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:29:25.265 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:29:25.265 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:29:25.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.263+0000 7f700eed5700 1 -- 172.21.15.27:0/1526095817 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6ff4064100 msgr2=0x7f6ff40665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:29:25.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.263+0000 7f700eed5700 1 --2- 172.21.15.27:0/1526095817 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6ff4064100 0x7f6ff40665c0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 rx=0x7f7008060060 tx=0x7f7004009040).stop 2021-11-03T18:29:25.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.263+0000 7f700eed5700 1 -- 172.21.15.27:0/1526095817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70080f66e0 msgr2=0x7f700810bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:29:25.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.263+0000 7f700eed5700 1 --2- 172.21.15.27:0/1526095817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70080f66e0 0x7f700810bf90 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 rx=0x7f6ff8009990 tx=0x7f6ff8009b30).stop 2021-11-03T18:29:25.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.264+0000 7f700eed5700 1 -- 172.21.15.27:0/1526095817 shutdown_connections 2021-11-03T18:29:25.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.264+0000 7f700eed5700 1 --2- 172.21.15.27:0/1526095817 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6ff4064100 0x7f6ff40665c0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:25.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.264+0000 7f700eed5700 1 --2- 172.21.15.27:0/1526095817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f70080f5440 0x7f7008109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:25.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.264+0000 7f700eed5700 1 --2- 172.21.15.27:0/1526095817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f70080f66e0 0x7f700810bf90 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:25.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.264+0000 7f700eed5700 1 -- 172.21.15.27:0/1526095817 >> 172.21.15.27:0/1526095817 conn(0x7f70080f0ba0 msgr2=0x7f70080fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:29:25.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.264+0000 7f700eed5700 1 -- 172.21.15.27:0/1526095817 shutdown_connections 2021-11-03T18:29:25.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.264+0000 7f700eed5700 1 -- 172.21.15.27:0/1526095817 wait complete. 2021-11-03T18:29:25.363 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.363+0000 7f88b6cff700 1 -- 172.21.15.27:0/582151234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88b00f96b0 msgr2=0x7f88b00fbb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:29:25.363 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.363+0000 7f88b6cff700 1 --2- 172.21.15.27:0/582151234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88b00f96b0 0x7f88b00fbb00 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 rx=0x7f88a4004660 tx=0x7f88a4009b30).stop 2021-11-03T18:29:25.363 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.363+0000 7f88b6cff700 1 -- 172.21.15.27:0/582151234 shutdown_connections 2021-11-03T18:29:25.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.363+0000 7f88b6cff700 1 --2- 172.21.15.27:0/582151234 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88b00f96b0 0x7f88b00fbb00 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:25.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.363+0000 7f88b6cff700 1 --2- 172.21.15.27:0/582151234 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88b0060270 0x7f88b00f91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:25.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.363+0000 7f88b6cff700 1 -- 172.21.15.27:0/582151234 >> 172.21.15.27:0/582151234 conn(0x7f88b00f0ba0 msgr2=0x7f88b00f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:29:25.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.363+0000 7f88b6cff700 1 -- 172.21.15.27:0/582151234 shutdown_connections 2021-11-03T18:29:25.365 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.363+0000 7f88b6cff700 1 -- 172.21.15.27:0/582151234 wait complete. 2021-11-03T18:29:25.365 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.364+0000 7f88b6cff700 1 Processor -- start 2021-11-03T18:29:25.365 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.364+0000 7f88b6cff700 1 -- start start 2021-11-03T18:29:25.365 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.364+0000 7f88b6cff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88b0060270 0x7f88b0106db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:29:25.366 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.364+0000 7f88b6cff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88b00f96b0 0x7f88b01072a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:29:25.366 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.364+0000 7f88b6cff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b01078a0 con 0x7f88b0060270 2021-11-03T18:29:25.366 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.364+0000 7f88b6cff700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b01080e0 con 0x7f88b00f96b0 2021-11-03T18:29:25.366 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.364+0000 7f88b5cfd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88b0060270 0x7f88b0106db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:29:25.367 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.364+0000 7f88b5cfd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88b0060270 0x7f88b0106db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35168/0 (socket says 172.21.15.27:35168) 2021-11-03T18:29:25.367 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.364+0000 7f88b5cfd700 1 -- 172.21.15.27:0/904407675 learned_addr learned my addr 172.21.15.27:0/904407675 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:29:25.367 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.365+0000 7f88b5cfd700 1 -- 172.21.15.27:0/904407675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88b00f96b0 msgr2=0x7f88b01072a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:29:25.367 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.365+0000 7f88b5cfd700 1 --2- 172.21.15.27:0/904407675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88b00f96b0 0x7f88b01072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:29:25.367 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.365+0000 7f88b5cfd700 1 -- 172.21.15.27:0/904407675 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a4005040 con 0x7f88b0060270 2021-11-03T18:29:25.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.365+0000 7f88b5cfd700 1 --2- 172.21.15.27:0/904407675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88b0060270 0x7f88b0106db0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 rx=0x7f88ac0026d0 tx=0x7f88ac009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:29:25.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.365+0000 7f88a27fc700 1 -- 172.21.15.27:0/904407675 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88ac009320 con 0x7f88b0060270 2021-11-03T18:29:25.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.365+0000 7f88a27fc700 1 -- 172.21.15.27:0/904407675 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f88ac00f040 con 0x7f88b0060270 2021-11-03T18:29:25.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.365+0000 7f88a27fc700 1 -- 172.21.15.27:0/904407675 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88ac0147e0 con 0x7f88b0060270 2021-11-03T18:29:25.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.365+0000 7f88b6cff700 1 -- 172.21.15.27:0/904407675 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88b01083c0 con 0x7f88b0060270 2021-11-03T18:29:25.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.365+0000 7f88b6cff700 1 -- 172.21.15.27:0/904407675 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b0108900 con 0x7f88b0060270 2021-11-03T18:29:25.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.366+0000 7f88b6cff700 1 -- 172.21.15.27:0/904407675 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88b0059100 con 0x7f88b0060270 2021-11-03T18:29:25.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.366+0000 7f88a27fc700 1 -- 172.21.15.27:0/904407675 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f88ac009480 con 0x7f88b0060270 2021-11-03T18:29:25.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.367+0000 7f88a27fc700 1 --2- 172.21.15.27:0/904407675 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f889c064100 0x7f889c0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:29:25.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.367+0000 7f88a27fc700 1 -- 172.21.15.27:0/904407675 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f88ac080eb0 con 0x7f88b0060270 2021-11-03T18:29:25.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.367+0000 7f88b54fc700 1 --2- 172.21.15.27:0/904407675 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f889c064100 0x7f889c0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:29:25.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.367+0000 7f88b54fc700 1 --2- 172.21.15.27:0/904407675 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f889c064100 0x7f889c0665c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f88a4004e30 tx=0x7f88a4009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:29:25.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.371+0000 7f88a27fc700 1 -- 172.21.15.27:0/904407675 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f88ac053210 con 0x7f88b0060270 2021-11-03T18:29:25.578 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.580+0000 7f88b6cff700 1 -- 172.21.15.27:0/904407675 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f88b00fee50 con 0x7f88b0060270 2021-11-03T18:29:25.578 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.580+0000 7f88a27fc700 1 -- 172.21.15.27:0/904407675 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f88ac052e00 con 0x7f88b0060270 2021-11-03T18:29:25.579 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:29:25.579 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:29:25.580 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:29:25.580 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:29:25.580 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:29:25.580 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:29:25.580 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:29:25.581 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:29:25.581 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:29:25.581 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:29:25.581 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:29:25.582 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:29:25.582 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:29:25.582 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:29:25.582 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:29:25.583 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:29:25.583 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:29:25.583 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:29:25.583 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:29:25.583 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:29:25.584 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:29:25.584 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:29:25.584 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:29:25.584 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:29:25.585 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:29:25.585 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:29:25.585 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:29:25.585 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:29:25.585 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:29:25.586 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:29:25.586 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:29:25.586 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:29:25.586 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:29:25.587 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:29:25.587 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:29:25.587 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:29:25.587 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:29:25.588 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:29:25.588 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:29:25.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.582+0000 7f88b6cff700 1 -- 172.21.15.27:0/904407675 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f889c064100 msgr2=0x7f889c0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:29:25.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.582+0000 7f88b6cff700 1 --2- 172.21.15.27:0/904407675 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f889c064100 0x7f889c0665c0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 rx=0x7f88a4004e30 tx=0x7f88a4009b30).stop 2021-11-03T18:29:25.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.582+0000 7f88b6cff700 1 -- 172.21.15.27:0/904407675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88b0060270 msgr2=0x7f88b0106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:29:25.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.582+0000 7f88b6cff700 1 --2- 172.21.15.27:0/904407675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88b0060270 0x7f88b0106db0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 rx=0x7f88ac0026d0 tx=0x7f88ac009de0).stop 2021-11-03T18:29:25.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.582+0000 7f88b6cff700 1 -- 172.21.15.27:0/904407675 shutdown_connections 2021-11-03T18:29:25.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.582+0000 7f88b6cff700 1 --2- 172.21.15.27:0/904407675 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f889c064100 0x7f889c0665c0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:25.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.582+0000 7f88b6cff700 1 --2- 172.21.15.27:0/904407675 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88b0060270 0x7f88b0106db0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:25.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.582+0000 7f88b6cff700 1 --2- 172.21.15.27:0/904407675 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88b00f96b0 0x7f88b01072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:29:25.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.582+0000 7f88b6cff700 1 -- 172.21.15.27:0/904407675 >> 172.21.15.27:0/904407675 conn(0x7f88b00f0ba0 msgr2=0x7f88b00f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:29:25.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.582+0000 7f88b6cff700 1 -- 172.21.15.27:0/904407675 shutdown_connections 2021-11-03T18:29:25.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:25.582+0000 7f88b6cff700 1 -- 172.21.15.27:0/904407675 wait complete. 2021-11-03T18:29:25.591 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:29:25.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4407.465330 secs 2021-11-03T18:29:25.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:25 smithi115 ceph-mon[65453]: from='client.36408 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:29:25.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:25 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1526095817' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:25.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4407.465330 secs 2021-11-03T18:29:25.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:25 smithi027 ceph-mon[97244]: from='client.36408 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:29:25.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:25 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1526095817' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:26.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:26 smithi115 ceph-mon[65453]: from='client.36412 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:29:26.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:26 smithi115 ceph-mon[65453]: pgmap v2245: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:26.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:26 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/904407675' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:29:26.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:29:26.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:29:26.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:29:26.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:26.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:26.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:26.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:26.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:26 smithi027 ceph-mon[97244]: from='client.36412 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:29:26.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:26 smithi027 ceph-mon[97244]: pgmap v2245: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:26.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:26 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/904407675' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:29:26.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:29:26.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:29:26.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:29:26.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:26.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:26.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:26.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:27.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:27 smithi027 ceph-mon[97244]: poll took 50158.009 ms: 1 events 2021-11-03T18:29:27.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:27 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:29:27.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:27 smithi115 ceph-mon[65453]: poll took 50158.009 ms: 1 events 2021-11-03T18:29:27.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:27 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:29:28.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:28 smithi027 ceph-mon[97244]: pgmap v2246: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:28.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:28 smithi115 ceph-mon[65453]: pgmap v2246: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:30.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:30 smithi027 ceph-mon[97244]: pgmap v2247: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:30.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4412.465472 secs 2021-11-03T18:29:30.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:30 smithi115 ceph-mon[65453]: pgmap v2247: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:30.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4412.465472 secs 2021-11-03T18:29:32.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:32 smithi027 ceph-mon[97244]: pgmap v2248: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:32.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:32 smithi115 ceph-mon[65453]: pgmap v2248: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:34.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:34 smithi027 ceph-mon[97244]: pgmap v2249: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:34.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:34 smithi115 ceph-mon[65453]: pgmap v2249: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:35.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4417.465614 secs 2021-11-03T18:29:35.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4417.465614 secs 2021-11-03T18:29:36.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:36 smithi027 ceph-mon[97244]: pgmap v2250: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:36.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:29:36.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:36 smithi115 ceph-mon[65453]: pgmap v2250: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:36.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:29:37.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:37 smithi027 ceph-mon[97244]: poll took 10018.316 ms: 1 events 2021-11-03T18:29:37.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:37 smithi115 ceph-mon[65453]: poll took 10018.316 ms: 1 events 2021-11-03T18:29:38.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:38 smithi027 ceph-mon[97244]: pgmap v2251: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:38.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:38 smithi115 ceph-mon[65453]: pgmap v2251: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:40.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:40 smithi027 ceph-mon[97244]: pgmap v2252: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:40.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4422.465693 secs 2021-11-03T18:29:40.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:40 smithi115 ceph-mon[65453]: pgmap v2252: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:40.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4422.465693 secs 2021-11-03T18:29:42.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:42 smithi115 ceph-mon[65453]: pgmap v2253: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:42.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:42 smithi115 ceph-mon[65453]: poll took 5036.239 ms: 1 events 2021-11-03T18:29:42.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:29:42.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:29:42.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:29:42.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:42.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:42.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:42.741 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:42.741 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:42 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:29:42.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:42 smithi027 ceph-mon[97244]: pgmap v2253: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:42.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:42 smithi027 ceph-mon[97244]: poll took 5036.239 ms: 1 events 2021-11-03T18:29:42.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:29:42.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:29:42.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:29:42.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:42.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:42.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:42.776 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:42.776 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:42 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:29:44.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:44 smithi115 ceph-mon[65453]: pgmap v2254: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:44.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:44 smithi027 ceph-mon[97244]: pgmap v2254: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:45.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4427.465813 secs 2021-11-03T18:29:45.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4427.465813 secs 2021-11-03T18:29:46.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:46 smithi115 ceph-mon[65453]: pgmap v2255: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:46.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:46 smithi027 ceph-mon[97244]: pgmap v2255: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:48.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:48 smithi115 ceph-mon[65453]: pgmap v2256: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:48.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:48 smithi027 ceph-mon[97244]: pgmap v2256: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:50.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:50 smithi115 ceph-mon[65453]: pgmap v2257: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:50.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4432.465956 secs 2021-11-03T18:29:50.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:50 smithi027 ceph-mon[97244]: pgmap v2257: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:50.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4432.465956 secs 2021-11-03T18:29:52.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:52 smithi115 ceph-mon[65453]: pgmap v2258: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:52.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:29:52.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:29:52.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:29:52.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:52.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:52.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:52.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:52.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:52 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:29:52.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:52 smithi027 ceph-mon[97244]: pgmap v2258: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:52.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:29:52.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:29:52.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:29:52.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:52.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:52.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:52.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:52.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:52 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:29:54.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:54 smithi115 ceph-mon[65453]: pgmap v2259: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:54.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:54 smithi027 ceph-mon[97244]: pgmap v2259: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:55.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.698+0000 7f8f31015700 1 -- 172.21.15.27:0/1044441375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f2c0f77d0 msgr2=0x7f8f2c0f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:29:55.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.698+0000 7f8f31015700 1 --2- 172.21.15.27:0/1044441375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f2c0f77d0 0x7f8f2c0f7c30 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 rx=0x7f8f20004660 tx=0x7f8f20009b30).stop 2021-11-03T18:29:55.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.698+0000 7f8f31015700 1 -- 172.21.15.27:0/1044441375 shutdown_connections 2021-11-03T18:29:55.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.698+0000 7f8f31015700 1 --2- 172.21.15.27:0/1044441375 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f2c0f77d0 0x7f8f2c0f7c30 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:55.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.698+0000 7f8f31015700 1 --2- 172.21.15.27:0/1044441375 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f2c0f6530 0x7f8f2c0f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:55.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.698+0000 7f8f31015700 1 -- 172.21.15.27:0/1044441375 >> 172.21.15.27:0/1044441375 conn(0x7f8f2c0f1cd0 msgr2=0x7f8f2c0f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:29:55.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.698+0000 7f8f31015700 1 -- 172.21.15.27:0/1044441375 shutdown_connections 2021-11-03T18:29:55.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.698+0000 7f8f31015700 1 -- 172.21.15.27:0/1044441375 wait complete. 2021-11-03T18:29:55.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.698+0000 7f8f31015700 1 Processor -- start 2021-11-03T18:29:55.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.699+0000 7f8f31015700 1 -- start start 2021-11-03T18:29:55.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.699+0000 7f8f31015700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f2c0f6530 0x7f8f2c106d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:29:55.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.699+0000 7f8f31015700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f2c0f77d0 0x7f8f2c107230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:29:55.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.699+0000 7f8f31015700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2c1077e0 con 0x7f8f2c0f77d0 2021-11-03T18:29:55.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.699+0000 7f8f31015700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2c107fe0 con 0x7f8f2c0f6530 2021-11-03T18:29:55.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.699+0000 7f8f2b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f2c0f77d0 0x7f8f2c107230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:29:55.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.699+0000 7f8f2b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f2c0f77d0 0x7f8f2c107230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35188/0 (socket says 172.21.15.27:35188) 2021-11-03T18:29:55.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.699+0000 7f8f2b7fe700 1 -- 172.21.15.27:0/2624232836 learned_addr learned my addr 172.21.15.27:0/2624232836 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:29:55.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.699+0000 7f8f2bfff700 1 --2- 172.21.15.27:0/2624232836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f2c0f6530 0x7f8f2c106d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:29:55.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.699+0000 7f8f2b7fe700 1 -- 172.21.15.27:0/2624232836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f2c0f6530 msgr2=0x7f8f2c106d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:29:55.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.699+0000 7f8f2b7fe700 1 --2- 172.21.15.27:0/2624232836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f2c0f6530 0x7f8f2c106d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:55.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.699+0000 7f8f2b7fe700 1 -- 172.21.15.27:0/2624232836 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f20005040 con 0x7f8f2c0f77d0 2021-11-03T18:29:55.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.699+0000 7f8f2b7fe700 1 --2- 172.21.15.27:0/2624232836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f2c0f77d0 0x7f8f2c107230 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 rx=0x7f8f20009830 tx=0x7f8f20009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:29:55.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.700+0000 7f8f28ff9700 1 -- 172.21.15.27:0/2624232836 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f20018070 con 0x7f8f2c0f77d0 2021-11-03T18:29:55.704 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.700+0000 7f8f28ff9700 1 -- 172.21.15.27:0/2624232836 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f8f20007ba0 con 0x7f8f2c0f77d0 2021-11-03T18:29:55.704 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.700+0000 7f8f31015700 1 -- 172.21.15.27:0/2624232836 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f2c108260 con 0x7f8f2c0f77d0 2021-11-03T18:29:55.704 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.700+0000 7f8f28ff9700 1 -- 172.21.15.27:0/2624232836 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8f2001c3c0 con 0x7f8f2c0f77d0 2021-11-03T18:29:55.704 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.700+0000 7f8f31015700 1 -- 172.21.15.27:0/2624232836 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f2c1087c0 con 0x7f8f2c0f77d0 2021-11-03T18:29:55.704 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.701+0000 7f8f31015700 1 -- 172.21.15.27:0/2624232836 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f2c059100 con 0x7f8f2c0f77d0 2021-11-03T18:29:55.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.701+0000 7f8f28ff9700 1 -- 172.21.15.27:0/2624232836 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f8f20007d10 con 0x7f8f2c0f77d0 2021-11-03T18:29:55.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.702+0000 7f8f28ff9700 1 --2- 172.21.15.27:0/2624232836 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8f14064100 0x7f8f140665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:29:55.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.702+0000 7f8f28ff9700 1 -- 172.21.15.27:0/2624232836 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f8f20082360 con 0x7f8f2c0f77d0 2021-11-03T18:29:55.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.702+0000 7f8f2bfff700 1 --2- 172.21.15.27:0/2624232836 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8f14064100 0x7f8f140665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:29:55.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.702+0000 7f8f2bfff700 1 --2- 172.21.15.27:0/2624232836 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8f14064100 0x7f8f140665c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f8f2c0f7630 tx=0x7f8f1c00b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:29:55.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.706+0000 7f8f28ff9700 1 -- 172.21.15.27:0/2624232836 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f8f20054780 con 0x7f8f2c0f77d0 2021-11-03T18:29:55.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4437.466081 secs 2021-11-03T18:29:55.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4437.466081 secs 2021-11-03T18:29:55.883 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.886+0000 7f8f31015700 1 -- 172.21.15.27:0/2624232836 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8f2c0fed60 con 0x7f8f14064100 2021-11-03T18:29:55.885 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.887+0000 7f8f28ff9700 1 -- 172.21.15.27:0/2624232836 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f8f2c0fed60 con 0x7f8f14064100 2021-11-03T18:29:55.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.889+0000 7f8f31015700 1 -- 172.21.15.27:0/2624232836 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8f14064100 msgr2=0x7f8f140665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:29:55.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.889+0000 7f8f31015700 1 --2- 172.21.15.27:0/2624232836 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8f14064100 0x7f8f140665c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 rx=0x7f8f2c0f7630 tx=0x7f8f1c00b040).stop 2021-11-03T18:29:55.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.889+0000 7f8f31015700 1 -- 172.21.15.27:0/2624232836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f2c0f77d0 msgr2=0x7f8f2c107230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:29:55.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.889+0000 7f8f31015700 1 --2- 172.21.15.27:0/2624232836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f2c0f77d0 0x7f8f2c107230 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 rx=0x7f8f20009830 tx=0x7f8f20009b30).stop 2021-11-03T18:29:55.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.889+0000 7f8f31015700 1 -- 172.21.15.27:0/2624232836 shutdown_connections 2021-11-03T18:29:55.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.889+0000 7f8f31015700 1 --2- 172.21.15.27:0/2624232836 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8f14064100 0x7f8f140665c0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:55.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.889+0000 7f8f31015700 1 --2- 172.21.15.27:0/2624232836 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8f2c0f6530 0x7f8f2c106d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:55.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.889+0000 7f8f31015700 1 --2- 172.21.15.27:0/2624232836 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8f2c0f77d0 0x7f8f2c107230 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:55.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.889+0000 7f8f31015700 1 -- 172.21.15.27:0/2624232836 >> 172.21.15.27:0/2624232836 conn(0x7f8f2c0f1cd0 msgr2=0x7f8f2c0fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:29:55.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.889+0000 7f8f31015700 1 -- 172.21.15.27:0/2624232836 shutdown_connections 2021-11-03T18:29:55.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.889+0000 7f8f31015700 1 -- 172.21.15.27:0/2624232836 wait complete. 2021-11-03T18:29:55.896 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:29:55.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.990+0000 7f0924263700 1 -- 172.21.15.27:0/819363676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f091c0f66e0 msgr2=0x7f091c0f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:29:55.989 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.990+0000 7f0924263700 1 --2- 172.21.15.27:0/819363676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f091c0f66e0 0x7f091c0f6b00 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 rx=0x7f0918004660 tx=0x7f0918009b30).stop 2021-11-03T18:29:55.989 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.990+0000 7f0924263700 1 -- 172.21.15.27:0/819363676 shutdown_connections 2021-11-03T18:29:55.989 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.990+0000 7f0924263700 1 --2- 172.21.15.27:0/819363676 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f091c0f7980 0x7f091c0f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:55.989 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.990+0000 7f0924263700 1 --2- 172.21.15.27:0/819363676 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f091c0f66e0 0x7f091c0f6b00 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:55.990 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.990+0000 7f0924263700 1 -- 172.21.15.27:0/819363676 >> 172.21.15.27:0/819363676 conn(0x7f091c0f1e40 msgr2=0x7f091c0f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:29:55.990 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.990+0000 7f0924263700 1 -- 172.21.15.27:0/819363676 shutdown_connections 2021-11-03T18:29:55.990 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.990+0000 7f0924263700 1 -- 172.21.15.27:0/819363676 wait complete. 2021-11-03T18:29:55.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.991+0000 7f0924263700 1 Processor -- start 2021-11-03T18:29:55.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.991+0000 7f0924263700 1 -- start start 2021-11-03T18:29:55.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.991+0000 7f0924263700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f091c0f66e0 0x7f091c06b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:29:55.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.991+0000 7f0924263700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f091c0f7980 0x7f091c069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:29:55.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.991+0000 7f0924263700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091c06a380 con 0x7f091c0f7980 2021-11-03T18:29:55.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.991+0000 7f0924263700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091c06a4c0 con 0x7f091c0f66e0 2021-11-03T18:29:55.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.991+0000 7f09217fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f091c0f7980 0x7f091c069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:29:55.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.991+0000 7f09217fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f091c0f7980 0x7f091c069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35198/0 (socket says 172.21.15.27:35198) 2021-11-03T18:29:55.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.991+0000 7f09217fe700 1 -- 172.21.15.27:0/187321290 learned_addr learned my addr 172.21.15.27:0/187321290 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:29:55.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.991+0000 7f0921fff700 1 --2- 172.21.15.27:0/187321290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f091c0f66e0 0x7f091c06b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:29:55.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.992+0000 7f09217fe700 1 -- 172.21.15.27:0/187321290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f091c0f66e0 msgr2=0x7f091c06b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:29:55.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.992+0000 7f09217fe700 1 --2- 172.21.15.27:0/187321290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f091c0f66e0 0x7f091c06b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:55.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.992+0000 7f09217fe700 1 -- 172.21.15.27:0/187321290 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0918005040 con 0x7f091c0f7980 2021-11-03T18:29:55.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.992+0000 7f0921fff700 1 --2- 172.21.15.27:0/187321290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f091c0f66e0 0x7f091c06b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:29:55.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.992+0000 7f09217fe700 1 --2- 172.21.15.27:0/187321290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f091c0f7980 0x7f091c069e20 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 rx=0x7f091000c790 tx=0x7f0910007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:29:55.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.992+0000 7f090e7fc700 1 -- 172.21.15.27:0/187321290 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f091000e680 con 0x7f091c0f7980 2021-11-03T18:29:55.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.992+0000 7f090e7fc700 1 -- 172.21.15.27:0/187321290 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f091000ec90 con 0x7f091c0f7980 2021-11-03T18:29:55.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.992+0000 7f0924263700 1 -- 172.21.15.27:0/187321290 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f091c06a7a0 con 0x7f091c0f7980 2021-11-03T18:29:55.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.992+0000 7f090e7fc700 1 -- 172.21.15.27:0/187321290 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0910013d60 con 0x7f091c0f7980 2021-11-03T18:29:55.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.992+0000 7f0924263700 1 -- 172.21.15.27:0/187321290 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f091c10efe0 con 0x7f091c0f7980 2021-11-03T18:29:55.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.993+0000 7f0924263700 1 -- 172.21.15.27:0/187321290 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f091c045ad0 con 0x7f091c0f7980 2021-11-03T18:29:55.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.993+0000 7f090e7fc700 1 -- 172.21.15.27:0/187321290 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f091001a060 con 0x7f091c0f7980 2021-11-03T18:29:55.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.994+0000 7f090e7fc700 1 --2- 172.21.15.27:0/187321290 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f09080640b0 0x7f0908066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:29:55.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.994+0000 7f090e7fc700 1 -- 172.21.15.27:0/187321290 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f0910080b00 con 0x7f091c0f7980 2021-11-03T18:29:55.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.994+0000 7f0921fff700 1 --2- 172.21.15.27:0/187321290 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f09080640b0 0x7f0908066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:29:55.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.994+0000 7f0921fff700 1 --2- 172.21.15.27:0/187321290 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f09080640b0 0x7f0908066570 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f0918004e30 tx=0x7f0918009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:29:55.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:55.998+0000 7f090e7fc700 1 -- 172.21.15.27:0/187321290 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f0910052d70 con 0x7f091c0f7980 2021-11-03T18:29:56.168 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.170+0000 7f0924263700 1 -- 172.21.15.27:0/187321290 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f091c0fefb0 con 0x7f09080640b0 2021-11-03T18:29:56.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.184+0000 7f090e7fc700 1 -- 172.21.15.27:0/187321290 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f091c0fefb0 con 0x7f09080640b0 2021-11-03T18:29:56.182 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:29:56.183 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (81m) 14s ago 82m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:29:56.183 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (73m) 14s ago 82m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:29:56.183 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (73m) 9m ago 81m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:29:56.183 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (81m) 14s ago 82m 42.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:29:56.184 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (74m) 14s ago 77m 26.7M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:29:56.184 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (77m) 14s ago 77m 3850M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:29:56.184 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (73m) 9m ago 77m 26.1M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:29:56.184 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (75m) 9m ago 78m 27.2M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:29:56.185 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (75m) 14s ago 84m 608M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:29:56.185 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (74m) 9m ago 81m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:29:56.185 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (74m) 14s ago 84m 350M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:29:56.185 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (73m) 9m ago 81m 145M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:29:56.186 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (82m) 14s ago 82m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:29:56.186 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (81m) 9m ago 81m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:29:56.186 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (72m) 14s ago 81m 413M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:29:56.186 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (72m) 14s ago 80m 450M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:29:56.186 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (71m) 14s ago 80m 338M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:29:56.187 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (71m) 9m ago 80m 454M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:29:56.187 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (70m) 9m ago 79m 359M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:29:56.187 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (70m) 9m ago 79m 368M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:29:56.187 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (81m) 14s ago 82m 64.4M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:29:56.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.186+0000 7f0924263700 1 -- 172.21.15.27:0/187321290 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f09080640b0 msgr2=0x7f0908066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:29:56.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.186+0000 7f0924263700 1 --2- 172.21.15.27:0/187321290 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f09080640b0 0x7f0908066570 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f0918004e30 tx=0x7f0918009b30).stop 2021-11-03T18:29:56.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.186+0000 7f0924263700 1 -- 172.21.15.27:0/187321290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f091c0f7980 msgr2=0x7f091c069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:29:56.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.186+0000 7f0924263700 1 --2- 172.21.15.27:0/187321290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f091c0f7980 0x7f091c069e20 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 rx=0x7f091000c790 tx=0x7f0910007d70).stop 2021-11-03T18:29:56.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.186+0000 7f0924263700 1 -- 172.21.15.27:0/187321290 shutdown_connections 2021-11-03T18:29:56.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.186+0000 7f0924263700 1 --2- 172.21.15.27:0/187321290 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f09080640b0 0x7f0908066570 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:56.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.186+0000 7f0924263700 1 --2- 172.21.15.27:0/187321290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f091c0f66e0 0x7f091c06b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:56.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.186+0000 7f0924263700 1 --2- 172.21.15.27:0/187321290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f091c0f7980 0x7f091c069e20 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:56.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.186+0000 7f0924263700 1 -- 172.21.15.27:0/187321290 >> 172.21.15.27:0/187321290 conn(0x7f091c0f1e40 msgr2=0x7f091c0fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:29:56.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.186+0000 7f0924263700 1 -- 172.21.15.27:0/187321290 shutdown_connections 2021-11-03T18:29:56.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.186+0000 7f0924263700 1 -- 172.21.15.27:0/187321290 wait complete. 2021-11-03T18:29:56.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.285+0000 7fc80f1aa700 1 -- 172.21.15.27:0/1310646458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8080f7980 msgr2=0x7fc8080f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:29:56.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.285+0000 7fc80f1aa700 1 --2- 172.21.15.27:0/1310646458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8080f7980 0x7fc8080f7de0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 rx=0x7fc7f8004660 tx=0x7fc7f8009b30).stop 2021-11-03T18:29:56.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.285+0000 7fc80f1aa700 1 -- 172.21.15.27:0/1310646458 shutdown_connections 2021-11-03T18:29:56.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.285+0000 7fc80f1aa700 1 --2- 172.21.15.27:0/1310646458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8080f7980 0x7fc8080f7de0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:56.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.285+0000 7fc80f1aa700 1 --2- 172.21.15.27:0/1310646458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8080f66e0 0x7fc8080f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:56.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.285+0000 7fc80f1aa700 1 -- 172.21.15.27:0/1310646458 >> 172.21.15.27:0/1310646458 conn(0x7fc8080f1e40 msgr2=0x7fc8080f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:29:56.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.285+0000 7fc80f1aa700 1 -- 172.21.15.27:0/1310646458 shutdown_connections 2021-11-03T18:29:56.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.285+0000 7fc80f1aa700 1 -- 172.21.15.27:0/1310646458 wait complete. 2021-11-03T18:29:56.286 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.286+0000 7fc80f1aa700 1 Processor -- start 2021-11-03T18:29:56.286 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.286+0000 7fc80f1aa700 1 -- start start 2021-11-03T18:29:56.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.286+0000 7fc80f1aa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8080f66e0 0x7fc80806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:29:56.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.286+0000 7fc80f1aa700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8080f7980 0x7fc808069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:29:56.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.286+0000 7fc80f1aa700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80806a380 con 0x7fc8080f66e0 2021-11-03T18:29:56.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.286+0000 7fc80f1aa700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80806a4c0 con 0x7fc8080f7980 2021-11-03T18:29:56.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.286+0000 7fc80cf46700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8080f66e0 0x7fc80806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:29:56.290 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.286+0000 7fc80cf46700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8080f66e0 0x7fc80806b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35206/0 (socket says 172.21.15.27:35206) 2021-11-03T18:29:56.290 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.286+0000 7fc80cf46700 1 -- 172.21.15.27:0/2564016783 learned_addr learned my addr 172.21.15.27:0/2564016783 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:29:56.290 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.287+0000 7fc807fff700 1 --2- 172.21.15.27:0/2564016783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8080f7980 0x7fc808069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:29:56.290 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.287+0000 7fc80cf46700 1 -- 172.21.15.27:0/2564016783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8080f7980 msgr2=0x7fc808069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:29:56.291 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.287+0000 7fc80cf46700 1 --2- 172.21.15.27:0/2564016783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8080f7980 0x7fc808069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:56.291 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.287+0000 7fc80cf46700 1 -- 172.21.15.27:0/2564016783 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f8005040 con 0x7fc8080f66e0 2021-11-03T18:29:56.291 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.287+0000 7fc807fff700 1 --2- 172.21.15.27:0/2564016783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8080f7980 0x7fc808069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:29:56.291 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.287+0000 7fc80cf46700 1 --2- 172.21.15.27:0/2564016783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8080f66e0 0x7fc80806b9a0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 rx=0x7fc80000eb70 tx=0x7fc80000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:29:56.291 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.287+0000 7fc8057fa700 1 -- 172.21.15.27:0/2564016783 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8000092e0 con 0x7fc8080f66e0 2021-11-03T18:29:56.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.287+0000 7fc8057fa700 1 -- 172.21.15.27:0/2564016783 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fc80000f040 con 0x7fc8080f66e0 2021-11-03T18:29:56.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.287+0000 7fc80f1aa700 1 -- 172.21.15.27:0/2564016783 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc80806a7a0 con 0x7fc8080f66e0 2021-11-03T18:29:56.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.287+0000 7fc8057fa700 1 -- 172.21.15.27:0/2564016783 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc80000c9a0 con 0x7fc8080f66e0 2021-11-03T18:29:56.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.287+0000 7fc80f1aa700 1 -- 172.21.15.27:0/2564016783 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc80810ef40 con 0x7fc8080f66e0 2021-11-03T18:29:56.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.288+0000 7fc80f1aa700 1 -- 172.21.15.27:0/2564016783 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc808045ad0 con 0x7fc8080f66e0 2021-11-03T18:29:56.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.289+0000 7fc8057fa700 1 -- 172.21.15.27:0/2564016783 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fc80001b030 con 0x7fc8080f66e0 2021-11-03T18:29:56.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.289+0000 7fc8057fa700 1 --2- 172.21.15.27:0/2564016783 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc7f40640b0 0x7fc7f4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:29:56.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.289+0000 7fc8057fa700 1 -- 172.21.15.27:0/2564016783 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fc800080980 con 0x7fc8080f66e0 2021-11-03T18:29:56.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.289+0000 7fc807fff700 1 --2- 172.21.15.27:0/2564016783 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc7f40640b0 0x7fc7f4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:29:56.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.289+0000 7fc807fff700 1 --2- 172.21.15.27:0/2564016783 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc7f40640b0 0x7fc7f4066570 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7fc7f8004d80 tx=0x7fc7f800b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:29:56.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.293+0000 7fc8057fa700 1 -- 172.21.15.27:0/2564016783 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fc800052bf0 con 0x7fc8080f66e0 2021-11-03T18:29:56.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.523+0000 7fc80f1aa700 1 -- 172.21.15.27:0/2564016783 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc8080fefa0 con 0x7fc8080f66e0 2021-11-03T18:29:56.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.524+0000 7fc8057fa700 1 -- 172.21.15.27:0/2564016783 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fc8000527e0 con 0x7fc8080f66e0 2021-11-03T18:29:56.522 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:29:56.522 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:29:56.523 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:29:56.523 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:29:56.523 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:29:56.523 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:29:56.523 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:29:56.524 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:29:56.524 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:29:56.524 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:29:56.524 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:29:56.525 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:29:56.525 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:29:56.525 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:29:56.525 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:29:56.525 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:29:56.526 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:29:56.526 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:29:56.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.525+0000 7fc80f1aa700 1 -- 172.21.15.27:0/2564016783 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc7f40640b0 msgr2=0x7fc7f4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:29:56.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.525+0000 7fc80f1aa700 1 --2- 172.21.15.27:0/2564016783 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc7f40640b0 0x7fc7f4066570 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 rx=0x7fc7f8004d80 tx=0x7fc7f800b040).stop 2021-11-03T18:29:56.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.525+0000 7fc80f1aa700 1 -- 172.21.15.27:0/2564016783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8080f66e0 msgr2=0x7fc80806b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:29:56.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.525+0000 7fc80f1aa700 1 --2- 172.21.15.27:0/2564016783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8080f66e0 0x7fc80806b9a0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 rx=0x7fc80000eb70 tx=0x7fc80000b040).stop 2021-11-03T18:29:56.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.526+0000 7fc80f1aa700 1 -- 172.21.15.27:0/2564016783 shutdown_connections 2021-11-03T18:29:56.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.526+0000 7fc80f1aa700 1 --2- 172.21.15.27:0/2564016783 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc7f40640b0 0x7fc7f4066570 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:56.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.526+0000 7fc80f1aa700 1 --2- 172.21.15.27:0/2564016783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc8080f66e0 0x7fc80806b9a0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:56.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.526+0000 7fc80f1aa700 1 --2- 172.21.15.27:0/2564016783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc8080f7980 0x7fc808069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:56.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.526+0000 7fc80f1aa700 1 -- 172.21.15.27:0/2564016783 >> 172.21.15.27:0/2564016783 conn(0x7fc8080f1e40 msgr2=0x7fc8080fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:29:56.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.526+0000 7fc80f1aa700 1 -- 172.21.15.27:0/2564016783 shutdown_connections 2021-11-03T18:29:56.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.526+0000 7fc80f1aa700 1 -- 172.21.15.27:0/2564016783 wait complete. 2021-11-03T18:29:56.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.626+0000 7f2a91ddb700 1 -- 172.21.15.27:0/1146340467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a8c0fed80 msgr2=0x7f2a8c0ff1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:29:56.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.626+0000 7f2a91ddb700 1 --2- 172.21.15.27:0/1146340467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a8c0fed80 0x7f2a8c0ff1a0 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 rx=0x7f2a80004660 tx=0x7f2a80009b30).stop 2021-11-03T18:29:56.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.626+0000 7f2a91ddb700 1 -- 172.21.15.27:0/1146340467 shutdown_connections 2021-11-03T18:29:56.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.626+0000 7f2a91ddb700 1 --2- 172.21.15.27:0/1146340467 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a8c0ffff0 0x7f2a8c100450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:56.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.626+0000 7f2a91ddb700 1 --2- 172.21.15.27:0/1146340467 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a8c0fed80 0x7f2a8c0ff1a0 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:56.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.626+0000 7f2a91ddb700 1 -- 172.21.15.27:0/1146340467 >> 172.21.15.27:0/1146340467 conn(0x7f2a8c065720 msgr2=0x7f2a8c067b40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:29:56.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.626+0000 7f2a91ddb700 1 -- 172.21.15.27:0/1146340467 shutdown_connections 2021-11-03T18:29:56.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.626+0000 7f2a91ddb700 1 -- 172.21.15.27:0/1146340467 wait complete. 2021-11-03T18:29:56.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.627+0000 7f2a91ddb700 1 Processor -- start 2021-11-03T18:29:56.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.627+0000 7f2a91ddb700 1 -- start start 2021-11-03T18:29:56.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.627+0000 7f2a91ddb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a8c0fed80 0x7f2a8c111750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:29:56.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.628+0000 7f2a91ddb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a8c0ffff0 0x7f2a8c111c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:29:56.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.628+0000 7f2a91ddb700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c1121f0 con 0x7f2a8c0fed80 2021-11-03T18:29:56.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.628+0000 7f2a91ddb700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a8c112330 con 0x7f2a8c0ffff0 2021-11-03T18:29:56.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.628+0000 7f2a90dd9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a8c0fed80 0x7f2a8c111750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:29:56.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.628+0000 7f2a90dd9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a8c0fed80 0x7f2a8c111750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35216/0 (socket says 172.21.15.27:35216) 2021-11-03T18:29:56.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.628+0000 7f2a90dd9700 1 -- 172.21.15.27:0/1242662944 learned_addr learned my addr 172.21.15.27:0/1242662944 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:29:56.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.628+0000 7f2a8bfff700 1 --2- 172.21.15.27:0/1242662944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a8c0ffff0 0x7f2a8c111c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:29:56.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.628+0000 7f2a90dd9700 1 -- 172.21.15.27:0/1242662944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a8c0ffff0 msgr2=0x7f2a8c111c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:29:56.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.628+0000 7f2a90dd9700 1 --2- 172.21.15.27:0/1242662944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a8c0ffff0 0x7f2a8c111c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:56.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.628+0000 7f2a90dd9700 1 -- 172.21.15.27:0/1242662944 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a80005040 con 0x7f2a8c0fed80 2021-11-03T18:29:56.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.628+0000 7f2a8bfff700 1 --2- 172.21.15.27:0/1242662944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a8c0ffff0 0x7f2a8c111c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:29:56.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.628+0000 7f2a90dd9700 1 --2- 172.21.15.27:0/1242662944 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a8c0fed80 0x7f2a8c111750 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 rx=0x7f2a80009830 tx=0x7f2a80009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:29:56.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.628+0000 7f2a897fa700 1 -- 172.21.15.27:0/1242662944 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a80018070 con 0x7f2a8c0fed80 2021-11-03T18:29:56.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.628+0000 7f2a91ddb700 1 -- 172.21.15.27:0/1242662944 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a8c112c70 con 0x7f2a8c0fed80 2021-11-03T18:29:56.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.628+0000 7f2a897fa700 1 -- 172.21.15.27:0/1242662944 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f2a80007ba0 con 0x7f2a8c0fed80 2021-11-03T18:29:56.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.628+0000 7f2a897fa700 1 -- 172.21.15.27:0/1242662944 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a8001c620 con 0x7f2a8c0fed80 2021-11-03T18:29:56.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.628+0000 7f2a91ddb700 1 -- 172.21.15.27:0/1242662944 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a8c1131d0 con 0x7f2a8c0fed80 2021-11-03T18:29:56.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.629+0000 7f2a91ddb700 1 -- 172.21.15.27:0/1242662944 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a8c045ad0 con 0x7f2a8c0fed80 2021-11-03T18:29:56.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.630+0000 7f2a897fa700 1 -- 172.21.15.27:0/1242662944 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f2a80007d10 con 0x7f2a8c0fed80 2021-11-03T18:29:56.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.630+0000 7f2a897fa700 1 --2- 172.21.15.27:0/1242662944 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2a7c064100 0x7f2a7c0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:29:56.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.630+0000 7f2a897fa700 1 -- 172.21.15.27:0/1242662944 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f2a80082710 con 0x7f2a8c0fed80 2021-11-03T18:29:56.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.630+0000 7f2a8bfff700 1 --2- 172.21.15.27:0/1242662944 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2a7c064100 0x7f2a7c0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:29:56.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.631+0000 7f2a8bfff700 1 --2- 172.21.15.27:0/1242662944 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2a7c064100 0x7f2a7c0665c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7f2a78009740 tx=0x7f2a78009be0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:29:56.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.634+0000 7f2a897fa700 1 -- 172.21.15.27:0/1242662944 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f2a80054b30 con 0x7f2a8c0fed80 2021-11-03T18:29:56.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:56 smithi115 ceph-mon[65453]: pgmap v2260: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:56.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:29:56.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:29:56.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:56 smithi027 ceph-mon[97244]: pgmap v2260: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:56.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:29:56.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:29:56.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.843+0000 7f2a91ddb700 1 -- 172.21.15.27:0/1242662944 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2a8c107620 con 0x7f2a8c0fed80 2021-11-03T18:29:56.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.844+0000 7f2a897fa700 1 -- 172.21.15.27:0/1242662944 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f2a80054720 con 0x7f2a8c0fed80 2021-11-03T18:29:56.842 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:29:56.843 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:29:56.843 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:29:56.843 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:29:56.843 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:29:56.844 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:29:56.844 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:29:56.844 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:29:56.844 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:29:56.844 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:29:56.845 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:29:56.845 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:29:56.845 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:29:56.845 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:29:56.846 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:29:56.846 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:29:56.846 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:29:56.846 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:29:56.847 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:29:56.847 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:29:56.847 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:29:56.847 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:29:56.847 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:29:56.848 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:29:56.848 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:29:56.848 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:29:56.848 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:29:56.849 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:29:56.849 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:29:56.849 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:29:56.849 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:29:56.849 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:29:56.850 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:29:56.850 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:29:56.850 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:29:56.850 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:29:56.851 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:29:56.851 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:29:56.851 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:29:56.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.846+0000 7f2a91ddb700 1 -- 172.21.15.27:0/1242662944 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2a7c064100 msgr2=0x7f2a7c0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:29:56.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.846+0000 7f2a91ddb700 1 --2- 172.21.15.27:0/1242662944 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2a7c064100 0x7f2a7c0665c0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 rx=0x7f2a78009740 tx=0x7f2a78009be0).stop 2021-11-03T18:29:56.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.846+0000 7f2a91ddb700 1 -- 172.21.15.27:0/1242662944 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a8c0fed80 msgr2=0x7f2a8c111750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:29:56.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.846+0000 7f2a91ddb700 1 --2- 172.21.15.27:0/1242662944 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a8c0fed80 0x7f2a8c111750 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 rx=0x7f2a80009830 tx=0x7f2a80009b30).stop 2021-11-03T18:29:56.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.846+0000 7f2a91ddb700 1 -- 172.21.15.27:0/1242662944 shutdown_connections 2021-11-03T18:29:56.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.846+0000 7f2a91ddb700 1 --2- 172.21.15.27:0/1242662944 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2a7c064100 0x7f2a7c0665c0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:56.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.846+0000 7f2a91ddb700 1 --2- 172.21.15.27:0/1242662944 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2a8c0fed80 0x7f2a8c111750 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:56.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.846+0000 7f2a91ddb700 1 --2- 172.21.15.27:0/1242662944 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2a8c0ffff0 0x7f2a8c111c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:29:56.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.846+0000 7f2a91ddb700 1 -- 172.21.15.27:0/1242662944 >> 172.21.15.27:0/1242662944 conn(0x7f2a8c065720 msgr2=0x7f2a8c060f50 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:29:56.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.846+0000 7f2a91ddb700 1 -- 172.21.15.27:0/1242662944 shutdown_connections 2021-11-03T18:29:56.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:29:56.846+0000 7f2a91ddb700 1 -- 172.21.15.27:0/1242662944 wait complete. 2021-11-03T18:29:56.855 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:29:57.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:57 smithi115 ceph-mon[65453]: from='client.36424 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:29:57.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:57 smithi115 ceph-mon[65453]: from='client.36428 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:29:57.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:57 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2564016783' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:57.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:57 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1242662944' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:29:57.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:57 smithi027 ceph-mon[97244]: from='client.36424 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:29:57.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:57 smithi027 ceph-mon[97244]: from='client.36428 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:29:57.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:57 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2564016783' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:29:57.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:57 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1242662944' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:29:58.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:29:58 smithi115 ceph-mon[65453]: pgmap v2261: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:29:58.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:29:58 smithi027 ceph-mon[97244]: pgmap v2261: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:00.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:00 smithi115 ceph-mon[65453]: pgmap v2262: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:00.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4442.466204 secs 2021-11-03T18:30:00.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:00 smithi115 ceph-mon[65453]: overall HEALTH_WARN 1 filesystem is degraded; 1 filesystem has a failed mds daemon; 1 MDSs report slow requests; 3 daemons have recently crashed 2021-11-03T18:30:00.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:00 smithi027 ceph-mon[97244]: pgmap v2262: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:00.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4442.466204 secs 2021-11-03T18:30:00.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:00 smithi027 ceph-mon[97244]: overall HEALTH_WARN 1 filesystem is degraded; 1 filesystem has a failed mds daemon; 1 MDSs report slow requests; 3 daemons have recently crashed 2021-11-03T18:30:02.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:02 smithi115 ceph-mon[65453]: pgmap v2263: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:02.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:30:02.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:30:02.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:30:02.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:02.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:02.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:02.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:02 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:02.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:02 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:30:02.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:02 smithi027 ceph-mon[97244]: pgmap v2263: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:02.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:30:02.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:30:02.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:30:02.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:02.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:02.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:02.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:02 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:02.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:02 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:30:04.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:04 smithi027 ceph-mon[97244]: pgmap v2264: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:04.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:04 smithi115 ceph-mon[65453]: pgmap v2264: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:05.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4447.466323 secs 2021-11-03T18:30:05.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4447.466323 secs 2021-11-03T18:30:06.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:06 smithi027 ceph-mon[97244]: pgmap v2265: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:06.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:06 smithi115 ceph-mon[65453]: pgmap v2265: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:08.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:08 smithi027 ceph-mon[97244]: pgmap v2266: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:08.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:08 smithi115 ceph-mon[65453]: pgmap v2266: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:10.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:10 smithi027 ceph-mon[97244]: pgmap v2267: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:10.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4452.466446 secs 2021-11-03T18:30:10.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:10 smithi115 ceph-mon[65453]: pgmap v2267: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:10.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4452.466446 secs 2021-11-03T18:30:12.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:12 smithi027 ceph-mon[97244]: pgmap v2268: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:12.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:12 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:30:12.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:12 smithi027 ceph-mon[97244]: poll took 30094.570 ms: 1 events 2021-11-03T18:30:12.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:12 smithi115 ceph-mon[65453]: pgmap v2268: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:12.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:12 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:30:12.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:12 smithi115 ceph-mon[65453]: poll took 30094.570 ms: 1 events 2021-11-03T18:30:14.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:14 smithi027 ceph-mon[97244]: pgmap v2269: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:14.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:14 smithi115 ceph-mon[65453]: pgmap v2269: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:15.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4457.466557 secs 2021-11-03T18:30:15.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:15 smithi027 ceph-mon[97244]: poll took 2912.902 ms: 1 events 2021-11-03T18:30:15.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:30:15.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:30:15.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:30:15.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:15.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:15.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:15.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:15.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:15 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:30:15.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4457.466557 secs 2021-11-03T18:30:15.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:15 smithi115 ceph-mon[65453]: poll took 2912.902 ms: 1 events 2021-11-03T18:30:15.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:30:15.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:30:15.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:30:15.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:15.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:15.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:15.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:15.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:15 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:30:16.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:16 smithi115 ceph-mon[65453]: pgmap v2270: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:17.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:16 smithi027 ceph-mon[97244]: pgmap v2270: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:18.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:18 smithi115 ceph-mon[65453]: pgmap v2271: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:19.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:18 smithi027 ceph-mon[97244]: pgmap v2271: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:20.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:20 smithi115 ceph-mon[65453]: pgmap v2272: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:20.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4462.466686 secs 2021-11-03T18:30:21.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:20 smithi027 ceph-mon[97244]: pgmap v2272: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:21.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4462.466686 secs 2021-11-03T18:30:22.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:22 smithi027 ceph-mon[97244]: pgmap v2273: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:22.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:22 smithi115 ceph-mon[65453]: pgmap v2273: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:24.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:24 smithi115 ceph-mon[65453]: pgmap v2274: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:25.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:24 smithi027 ceph-mon[97244]: pgmap v2274: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:25.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4467.466805 secs 2021-11-03T18:30:25.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:30:25.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:30:25.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:30:25.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:25.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:25.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:25.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:25.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:25 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:30:26.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4467.466805 secs 2021-11-03T18:30:26.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:30:26.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:30:26.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:30:26.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:26.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:26.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:26.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:26.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:25 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:30:26.959 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:26.959+0000 7f21ee7fa700 1 -- 172.21.15.27:0/143449050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21e80f6ff0 msgr2=0x7f21e80f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:30:26.959 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:26.959+0000 7f21ee7fa700 1 --2- 172.21.15.27:0/143449050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21e80f6ff0 0x7f21e80f7a20 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 rx=0x7f21dc004660 tx=0x7f21dc009b30).stop 2021-11-03T18:30:26.959 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:26.959+0000 7f21ee7fa700 1 -- 172.21.15.27:0/143449050 shutdown_connections 2021-11-03T18:30:26.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:26.959+0000 7f21ee7fa700 1 --2- 172.21.15.27:0/143449050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21e80f6ff0 0x7f21e80f7a20 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:26.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:26.959+0000 7f21ee7fa700 1 --2- 172.21.15.27:0/143449050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21e80f66e0 0x7f21e80f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:26.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:26.959+0000 7f21ee7fa700 1 -- 172.21.15.27:0/143449050 >> 172.21.15.27:0/143449050 conn(0x7f21e80f1f00 msgr2=0x7f21e80f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:30:26.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:26.959+0000 7f21ee7fa700 1 -- 172.21.15.27:0/143449050 shutdown_connections 2021-11-03T18:30:26.961 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:26.959+0000 7f21ee7fa700 1 -- 172.21.15.27:0/143449050 wait complete. 2021-11-03T18:30:26.961 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:26.960+0000 7f21ee7fa700 1 Processor -- start 2021-11-03T18:30:26.961 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:26.960+0000 7f21ee7fa700 1 -- start start 2021-11-03T18:30:26.963 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:26.960+0000 7f21ee7fa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21e80f66e0 0x7f21e8106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:30:26.964 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:26.960+0000 7f21ee7fa700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21e80f6ff0 0x7f21e8107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:30:26.964 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:26.960+0000 7f21ee7fa700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e8107a10 con 0x7f21e80f66e0 2021-11-03T18:30:26.964 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:26.960+0000 7f21ee7fa700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e8107b50 con 0x7f21e80f6ff0 2021-11-03T18:30:26.964 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:26.961+0000 7f21e7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21e80f66e0 0x7f21e8106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:30:26.964 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:26.961+0000 7f21e7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21e80f66e0 0x7f21e8106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35232/0 (socket says 172.21.15.27:35232) 2021-11-03T18:30:26.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:26.961+0000 7f21e7fff700 1 -- 172.21.15.27:0/1762981525 learned_addr learned my addr 172.21.15.27:0/1762981525 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:30:26.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:26.961+0000 7f21e77fe700 1 --2- 172.21.15.27:0/1762981525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21e80f6ff0 0x7f21e8107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:30:26.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:26.961+0000 7f21e7fff700 1 -- 172.21.15.27:0/1762981525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21e80f6ff0 msgr2=0x7f21e8107460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:30:26.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:26.961+0000 7f21e7fff700 1 --2- 172.21.15.27:0/1762981525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21e80f6ff0 0x7f21e8107460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:26.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:26.961+0000 7f21e7fff700 1 -- 172.21.15.27:0/1762981525 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21dc005040 con 0x7f21e80f66e0 2021-11-03T18:30:26.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:26.961+0000 7f21e77fe700 1 --2- 172.21.15.27:0/1762981525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21e80f6ff0 0x7f21e8107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:30:26.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:26.961+0000 7f21e7fff700 1 --2- 172.21.15.27:0/1762981525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21e80f66e0 0x7f21e8106f70 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 rx=0x7f21d80026d0 tx=0x7f21d8009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:30:26.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:26.961+0000 7f21e4ff9700 1 -- 172.21.15.27:0/1762981525 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21d8009320 con 0x7f21e80f66e0 2021-11-03T18:30:26.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:26.961+0000 7f21e4ff9700 1 -- 172.21.15.27:0/1762981525 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f21d800f040 con 0x7f21e80f66e0 2021-11-03T18:30:26.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:26.961+0000 7f21ee7fa700 1 -- 172.21.15.27:0/1762981525 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e81084f0 con 0x7f21e80f66e0 2021-11-03T18:30:26.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:26.961+0000 7f21e4ff9700 1 -- 172.21.15.27:0/1762981525 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f21d80147e0 con 0x7f21e80f66e0 2021-11-03T18:30:26.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:26.961+0000 7f21ee7fa700 1 -- 172.21.15.27:0/1762981525 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e8108ab0 con 0x7f21e80f66e0 2021-11-03T18:30:26.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:26.962+0000 7f21ee7fa700 1 -- 172.21.15.27:0/1762981525 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e8045ad0 con 0x7f21e80f66e0 2021-11-03T18:30:26.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:26.963+0000 7f21e4ff9700 1 -- 172.21.15.27:0/1762981525 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f21d8009480 con 0x7f21e80f66e0 2021-11-03T18:30:26.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:26.963+0000 7f21e4ff9700 1 --2- 172.21.15.27:0/1762981525 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f21d00640b0 0x7f21d0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:30:26.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:26.963+0000 7f21e4ff9700 1 -- 172.21.15.27:0/1762981525 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f21d8080df0 con 0x7f21e80f66e0 2021-11-03T18:30:26.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:26.963+0000 7f21e77fe700 1 --2- 172.21.15.27:0/1762981525 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f21d00640b0 0x7f21d0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:30:26.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:26.964+0000 7f21e77fe700 1 --2- 172.21.15.27:0/1762981525 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f21d00640b0 0x7f21d0066570 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7f21dc004630 tx=0x7f21dc009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:30:26.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:26.967+0000 7f21e4ff9700 1 -- 172.21.15.27:0/1762981525 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f21d8053210 con 0x7f21e80f66e0 2021-11-03T18:30:26.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:26 smithi115 ceph-mon[65453]: pgmap v2275: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:27.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:26 smithi027 ceph-mon[97244]: pgmap v2275: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:27.148 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.149+0000 7f21ee7fa700 1 -- 172.21.15.27:0/1762981525 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f21e80ff050 con 0x7f21d00640b0 2021-11-03T18:30:27.149 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.150+0000 7f21e4ff9700 1 -- 172.21.15.27:0/1762981525 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f21e80ff050 con 0x7f21d00640b0 2021-11-03T18:30:27.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.152+0000 7f21ee7fa700 1 -- 172.21.15.27:0/1762981525 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f21d00640b0 msgr2=0x7f21d0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:30:27.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.152+0000 7f21ee7fa700 1 --2- 172.21.15.27:0/1762981525 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f21d00640b0 0x7f21d0066570 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 rx=0x7f21dc004630 tx=0x7f21dc009b30).stop 2021-11-03T18:30:27.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.152+0000 7f21ee7fa700 1 -- 172.21.15.27:0/1762981525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21e80f66e0 msgr2=0x7f21e8106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:30:27.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.152+0000 7f21ee7fa700 1 --2- 172.21.15.27:0/1762981525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21e80f66e0 0x7f21e8106f70 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 rx=0x7f21d80026d0 tx=0x7f21d8009de0).stop 2021-11-03T18:30:27.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.152+0000 7f21ee7fa700 1 -- 172.21.15.27:0/1762981525 shutdown_connections 2021-11-03T18:30:27.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.152+0000 7f21ee7fa700 1 --2- 172.21.15.27:0/1762981525 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f21d00640b0 0x7f21d0066570 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:27.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.152+0000 7f21ee7fa700 1 --2- 172.21.15.27:0/1762981525 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f21e80f66e0 0x7f21e8106f70 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:27.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.152+0000 7f21ee7fa700 1 --2- 172.21.15.27:0/1762981525 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f21e80f6ff0 0x7f21e8107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:27.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.152+0000 7f21ee7fa700 1 -- 172.21.15.27:0/1762981525 >> 172.21.15.27:0/1762981525 conn(0x7f21e80f1f00 msgr2=0x7f21e80fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:30:27.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.152+0000 7f21ee7fa700 1 -- 172.21.15.27:0/1762981525 shutdown_connections 2021-11-03T18:30:27.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.152+0000 7f21ee7fa700 1 -- 172.21.15.27:0/1762981525 wait complete. 2021-11-03T18:30:27.161 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:30:27.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.253+0000 7f4091b77700 1 -- 172.21.15.27:0/478575610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f408c0f6860 msgr2=0x7f408c0f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:30:27.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.253+0000 7f4091b77700 1 --2- 172.21.15.27:0/478575610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f408c0f6860 0x7f408c0f6cc0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 rx=0x7f4080004660 tx=0x7f4080009b30).stop 2021-11-03T18:30:27.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.253+0000 7f4091b77700 1 -- 172.21.15.27:0/478575610 shutdown_connections 2021-11-03T18:30:27.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.253+0000 7f4091b77700 1 --2- 172.21.15.27:0/478575610 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f408c0f6860 0x7f408c0f6cc0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:27.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.253+0000 7f4091b77700 1 --2- 172.21.15.27:0/478575610 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f408c0f55c0 0x7f408c0f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:27.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.253+0000 7f4091b77700 1 -- 172.21.15.27:0/478575610 >> 172.21.15.27:0/478575610 conn(0x7f408c0f0d40 msgr2=0x7f408c0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:30:27.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.253+0000 7f4091b77700 1 -- 172.21.15.27:0/478575610 shutdown_connections 2021-11-03T18:30:27.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.253+0000 7f4091b77700 1 -- 172.21.15.27:0/478575610 wait complete. 2021-11-03T18:30:27.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.253+0000 7f4091b77700 1 Processor -- start 2021-11-03T18:30:27.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.254+0000 7f4091b77700 1 -- start start 2021-11-03T18:30:27.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.254+0000 7f4091b77700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f408c0f55c0 0x7f408c109c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:30:27.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.254+0000 7f4091b77700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f408c0f6860 0x7f408c10c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:30:27.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.254+0000 7f4091b77700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408c0603e0 con 0x7f408c0f6860 2021-11-03T18:30:27.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.254+0000 7f4091b77700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408c060520 con 0x7f408c0f55c0 2021-11-03T18:30:27.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.254+0000 7f408affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f408c0f6860 0x7f408c10c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:30:27.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.254+0000 7f408affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f408c0f6860 0x7f408c10c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35244/0 (socket says 172.21.15.27:35244) 2021-11-03T18:30:27.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.254+0000 7f408affd700 1 -- 172.21.15.27:0/1025474813 learned_addr learned my addr 172.21.15.27:0/1025474813 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:30:27.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.254+0000 7f408b7fe700 1 --2- 172.21.15.27:0/1025474813 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f408c0f55c0 0x7f408c109c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:30:27.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.254+0000 7f408affd700 1 -- 172.21.15.27:0/1025474813 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f408c0f55c0 msgr2=0x7f408c109c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:30:27.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.254+0000 7f408affd700 1 --2- 172.21.15.27:0/1025474813 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f408c0f55c0 0x7f408c109c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:27.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.254+0000 7f408affd700 1 -- 172.21.15.27:0/1025474813 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4080005040 con 0x7f408c0f6860 2021-11-03T18:30:27.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.254+0000 7f408b7fe700 1 --2- 172.21.15.27:0/1025474813 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f408c0f55c0 0x7f408c109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:30:27.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.254+0000 7f408affd700 1 --2- 172.21.15.27:0/1025474813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f408c0f6860 0x7f408c10c150 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 rx=0x7f4080009990 tx=0x7f4080009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:30:27.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.255+0000 7f4073fff700 1 -- 172.21.15.27:0/1025474813 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4080018070 con 0x7f408c0f6860 2021-11-03T18:30:27.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.255+0000 7f4073fff700 1 -- 172.21.15.27:0/1025474813 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f4080007ca0 con 0x7f408c0f6860 2021-11-03T18:30:27.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.255+0000 7f4091b77700 1 -- 172.21.15.27:0/1025474813 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f408c10c6a0 con 0x7f408c0f6860 2021-11-03T18:30:27.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.255+0000 7f4073fff700 1 -- 172.21.15.27:0/1025474813 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f408001c620 con 0x7f408c0f6860 2021-11-03T18:30:27.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.255+0000 7f4091b77700 1 -- 172.21.15.27:0/1025474813 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f408c10cc00 con 0x7f408c0f6860 2021-11-03T18:30:27.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.256+0000 7f4091b77700 1 -- 172.21.15.27:0/1025474813 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f408c045ad0 con 0x7f408c0f6860 2021-11-03T18:30:27.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.256+0000 7f4073fff700 1 -- 172.21.15.27:0/1025474813 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f408001d960 con 0x7f408c0f6860 2021-11-03T18:30:27.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.256+0000 7f4073fff700 1 --2- 172.21.15.27:0/1025474813 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f40740640b0 0x7f4074066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:30:27.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.257+0000 7f4073fff700 1 -- 172.21.15.27:0/1025474813 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f4080082480 con 0x7f408c0f6860 2021-11-03T18:30:27.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.257+0000 7f408b7fe700 1 --2- 172.21.15.27:0/1025474813 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f40740640b0 0x7f4074066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:30:27.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.257+0000 7f408b7fe700 1 --2- 172.21.15.27:0/1025474813 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f40740640b0 0x7f4074066570 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7f408c060060 tx=0x7f407c009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:30:27.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.261+0000 7f4073fff700 1 -- 172.21.15.27:0/1025474813 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f40800548a0 con 0x7f408c0f6860 2021-11-03T18:30:27.434 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.435+0000 7f4091b77700 1 -- 172.21.15.27:0/1025474813 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f408c0fde90 con 0x7f40740640b0 2021-11-03T18:30:27.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.449+0000 7f4073fff700 1 -- 172.21.15.27:0/1025474813 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f408c0fde90 con 0x7f40740640b0 2021-11-03T18:30:27.449 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:30:27.449 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (82m) 46s ago 83m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:30:27.449 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (73m) 46s ago 83m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:30:27.449 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (73m) 12s ago 82m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:30:27.450 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (81m) 46s ago 82m 42.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:30:27.450 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (75m) 46s ago 78m 26.7M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:30:27.450 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (78m) 46s ago 78m 3850M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:30:27.450 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (74m) 12s ago 78m 26.3M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:30:27.451 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (75m) 12s ago 78m 27.4M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:30:27.451 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (75m) 46s ago 84m 608M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:30:27.451 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (75m) 12s ago 82m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:30:27.451 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (74m) 46s ago 84m 350M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:30:27.451 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (74m) 12s ago 82m 160M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:30:27.452 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (82m) 46s ago 82m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:30:27.452 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (82m) 12s ago 82m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:30:27.452 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (73m) 46s ago 81m 413M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:30:27.452 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (72m) 46s ago 81m 450M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:30:27.453 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (72m) 46s ago 80m 338M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:30:27.453 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (71m) 12s ago 80m 479M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:30:27.453 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (71m) 12s ago 80m 378M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:30:27.453 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (70m) 12s ago 79m 388M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:30:27.454 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (81m) 46s ago 82m 64.4M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:30:27.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.451+0000 7f4091b77700 1 -- 172.21.15.27:0/1025474813 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f40740640b0 msgr2=0x7f4074066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:30:27.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.451+0000 7f4091b77700 1 --2- 172.21.15.27:0/1025474813 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f40740640b0 0x7f4074066570 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 rx=0x7f408c060060 tx=0x7f407c009040).stop 2021-11-03T18:30:27.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.451+0000 7f4091b77700 1 -- 172.21.15.27:0/1025474813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f408c0f6860 msgr2=0x7f408c10c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:30:27.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.451+0000 7f4091b77700 1 --2- 172.21.15.27:0/1025474813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f408c0f6860 0x7f408c10c150 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 rx=0x7f4080009990 tx=0x7f4080009b30).stop 2021-11-03T18:30:27.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.451+0000 7f4091b77700 1 -- 172.21.15.27:0/1025474813 shutdown_connections 2021-11-03T18:30:27.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.451+0000 7f4091b77700 1 --2- 172.21.15.27:0/1025474813 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f40740640b0 0x7f4074066570 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:27.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.451+0000 7f4091b77700 1 --2- 172.21.15.27:0/1025474813 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f408c0f55c0 0x7f408c109c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:27.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.451+0000 7f4091b77700 1 --2- 172.21.15.27:0/1025474813 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f408c0f6860 0x7f408c10c150 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:27.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.451+0000 7f4091b77700 1 -- 172.21.15.27:0/1025474813 >> 172.21.15.27:0/1025474813 conn(0x7f408c0f0d40 msgr2=0x7f408c0fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:30:27.457 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.451+0000 7f4091b77700 1 -- 172.21.15.27:0/1025474813 shutdown_connections 2021-11-03T18:30:27.457 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.451+0000 7f4091b77700 1 -- 172.21.15.27:0/1025474813 wait complete. 2021-11-03T18:30:27.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.550+0000 7ff1b59e0700 1 -- 172.21.15.27:0/3843187078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1b00f66e0 msgr2=0x7ff1b00f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:30:27.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.550+0000 7ff1b59e0700 1 --2- 172.21.15.27:0/3843187078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1b00f66e0 0x7ff1b00f6b40 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 rx=0x7ff19c004660 tx=0x7ff19c009b30).stop 2021-11-03T18:30:27.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.550+0000 7ff1b59e0700 1 -- 172.21.15.27:0/3843187078 shutdown_connections 2021-11-03T18:30:27.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.550+0000 7ff1b59e0700 1 --2- 172.21.15.27:0/3843187078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1b00f66e0 0x7ff1b00f6b40 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:27.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.550+0000 7ff1b59e0700 1 --2- 172.21.15.27:0/3843187078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1b00f5440 0x7ff1b00f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:27.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.551+0000 7ff1b59e0700 1 -- 172.21.15.27:0/3843187078 >> 172.21.15.27:0/3843187078 conn(0x7ff1b00f0ba0 msgr2=0x7ff1b00f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:30:27.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.551+0000 7ff1b59e0700 1 -- 172.21.15.27:0/3843187078 shutdown_connections 2021-11-03T18:30:27.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.551+0000 7ff1b59e0700 1 -- 172.21.15.27:0/3843187078 wait complete. 2021-11-03T18:30:27.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.551+0000 7ff1b59e0700 1 Processor -- start 2021-11-03T18:30:27.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.551+0000 7ff1b59e0700 1 -- start start 2021-11-03T18:30:27.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.551+0000 7ff1b59e0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1b00f5440 0x7ff1b0109a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:30:27.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.551+0000 7ff1b59e0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1b00f66e0 0x7ff1b010bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:30:27.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.551+0000 7ff1b59e0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b00603e0 con 0x7ff1b00f66e0 2021-11-03T18:30:27.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.551+0000 7ff1b59e0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b0060520 con 0x7ff1b00f5440 2021-11-03T18:30:27.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.552+0000 7ff1affff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1b00f66e0 0x7ff1b010bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:30:27.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.552+0000 7ff1affff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1b00f66e0 0x7ff1b010bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35254/0 (socket says 172.21.15.27:35254) 2021-11-03T18:30:27.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.552+0000 7ff1affff700 1 -- 172.21.15.27:0/1705845822 learned_addr learned my addr 172.21.15.27:0/1705845822 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:30:27.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.552+0000 7ff1b49de700 1 --2- 172.21.15.27:0/1705845822 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1b00f5440 0x7ff1b0109a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:30:27.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.552+0000 7ff1affff700 1 -- 172.21.15.27:0/1705845822 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1b00f5440 msgr2=0x7ff1b0109a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:30:27.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.552+0000 7ff1affff700 1 --2- 172.21.15.27:0/1705845822 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1b00f5440 0x7ff1b0109a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:27.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.552+0000 7ff1affff700 1 -- 172.21.15.27:0/1705845822 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff19c005040 con 0x7ff1b00f66e0 2021-11-03T18:30:27.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.552+0000 7ff1affff700 1 --2- 172.21.15.27:0/1705845822 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1b00f66e0 0x7ff1b010bf90 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 rx=0x7ff19c009990 tx=0x7ff19c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:30:27.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.552+0000 7ff1ad7fa700 1 -- 172.21.15.27:0/1705845822 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff19c018070 con 0x7ff1b00f66e0 2021-11-03T18:30:27.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.552+0000 7ff1ad7fa700 1 -- 172.21.15.27:0/1705845822 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7ff19c007ba0 con 0x7ff1b00f66e0 2021-11-03T18:30:27.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.552+0000 7ff1b59e0700 1 -- 172.21.15.27:0/1705845822 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1b010c4e0 con 0x7ff1b00f66e0 2021-11-03T18:30:27.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.552+0000 7ff1ad7fa700 1 -- 172.21.15.27:0/1705845822 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff19c01c620 con 0x7ff1b00f66e0 2021-11-03T18:30:27.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.553+0000 7ff1b59e0700 1 -- 172.21.15.27:0/1705845822 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1b010ca40 con 0x7ff1b00f66e0 2021-11-03T18:30:27.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.554+0000 7ff1b59e0700 1 -- 172.21.15.27:0/1705845822 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1b0045ad0 con 0x7ff1b00f66e0 2021-11-03T18:30:27.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.554+0000 7ff1ad7fa700 1 -- 172.21.15.27:0/1705845822 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7ff19c01c8e0 con 0x7ff1b00f66e0 2021-11-03T18:30:27.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.555+0000 7ff1ad7fa700 1 --2- 172.21.15.27:0/1705845822 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff1a0064100 0x7ff1a00665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:30:27.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.555+0000 7ff1ad7fa700 1 -- 172.21.15.27:0/1705845822 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7ff19c082600 con 0x7ff1b00f66e0 2021-11-03T18:30:27.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.555+0000 7ff1b49de700 1 --2- 172.21.15.27:0/1705845822 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff1a0064100 0x7ff1a00665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:30:27.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.555+0000 7ff1b49de700 1 --2- 172.21.15.27:0/1705845822 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff1a0064100 0x7ff1a00665c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7ff1b0060060 tx=0x7ff1a4009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:30:27.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.560+0000 7ff1ad7fa700 1 -- 172.21.15.27:0/1705845822 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7ff19c054a20 con 0x7ff1b00f66e0 2021-11-03T18:30:27.792 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.793+0000 7ff1b59e0700 1 -- 172.21.15.27:0/1705845822 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff1b00fdca0 con 0x7ff1b00f66e0 2021-11-03T18:30:27.793 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.794+0000 7ff1ad7fa700 1 -- 172.21.15.27:0/1705845822 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7ff19c054610 con 0x7ff1b00f66e0 2021-11-03T18:30:27.793 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:30:27.793 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:30:27.794 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:30:27.794 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:30:27.794 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:30:27.794 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:30:27.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:30:27.795 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:30:27.795 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:30:27.795 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:30:27.795 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:30:27.796 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:30:27.796 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:30:27.796 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:30:27.796 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:30:27.797 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:30:27.797 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:30:27.797 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:30:27.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.795+0000 7ff1b59e0700 1 -- 172.21.15.27:0/1705845822 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff1a0064100 msgr2=0x7ff1a00665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:30:27.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.795+0000 7ff1b59e0700 1 --2- 172.21.15.27:0/1705845822 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff1a0064100 0x7ff1a00665c0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7ff1b0060060 tx=0x7ff1a4009040).stop 2021-11-03T18:30:27.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.795+0000 7ff1b59e0700 1 -- 172.21.15.27:0/1705845822 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1b00f66e0 msgr2=0x7ff1b010bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:30:27.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.795+0000 7ff1b59e0700 1 --2- 172.21.15.27:0/1705845822 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1b00f66e0 0x7ff1b010bf90 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 rx=0x7ff19c009990 tx=0x7ff19c009b30).stop 2021-11-03T18:30:27.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.796+0000 7ff1b59e0700 1 -- 172.21.15.27:0/1705845822 shutdown_connections 2021-11-03T18:30:27.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.796+0000 7ff1b59e0700 1 --2- 172.21.15.27:0/1705845822 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff1a0064100 0x7ff1a00665c0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:27.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.796+0000 7ff1b59e0700 1 --2- 172.21.15.27:0/1705845822 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1b00f5440 0x7ff1b0109a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:27.800 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.796+0000 7ff1b59e0700 1 --2- 172.21.15.27:0/1705845822 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1b00f66e0 0x7ff1b010bf90 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:27.800 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.796+0000 7ff1b59e0700 1 -- 172.21.15.27:0/1705845822 >> 172.21.15.27:0/1705845822 conn(0x7ff1b00f0ba0 msgr2=0x7ff1b00fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:30:27.800 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.796+0000 7ff1b59e0700 1 -- 172.21.15.27:0/1705845822 shutdown_connections 2021-11-03T18:30:27.800 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.796+0000 7ff1b59e0700 1 -- 172.21.15.27:0/1705845822 wait complete. 2021-11-03T18:30:27.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.895+0000 7f369d145700 1 -- 172.21.15.27:0/2839522674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36980f55e0 msgr2=0x7f36980f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:30:27.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.895+0000 7f369d145700 1 --2- 172.21.15.27:0/2839522674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36980f55e0 0x7f36980f5a00 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 rx=0x7f36900099e0 tx=0x7f3690004d10).stop 2021-11-03T18:30:27.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.895+0000 7f369d145700 1 -- 172.21.15.27:0/2839522674 shutdown_connections 2021-11-03T18:30:27.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.895+0000 7f369d145700 1 --2- 172.21.15.27:0/2839522674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36980f5ef0 0x7f36980f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:27.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.895+0000 7f369d145700 1 --2- 172.21.15.27:0/2839522674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36980f55e0 0x7f36980f5a00 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:27.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.895+0000 7f369d145700 1 -- 172.21.15.27:0/2839522674 >> 172.21.15.27:0/2839522674 conn(0x7f36980f0de0 msgr2=0x7f36980f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:30:27.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.896+0000 7f369d145700 1 -- 172.21.15.27:0/2839522674 shutdown_connections 2021-11-03T18:30:27.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.896+0000 7f369d145700 1 -- 172.21.15.27:0/2839522674 wait complete. 2021-11-03T18:30:27.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.896+0000 7f369d145700 1 Processor -- start 2021-11-03T18:30:27.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.896+0000 7f369d145700 1 -- start start 2021-11-03T18:30:27.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.896+0000 7f369d145700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36980f55e0 0x7f3698109d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:30:27.900 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.896+0000 7f369d145700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36980f5ef0 0x7f369810c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:30:27.900 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.896+0000 7f369d145700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36980604e0 con 0x7f36980f5ef0 2021-11-03T18:30:27.900 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.896+0000 7f369d145700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3698060620 con 0x7f36980f55e0 2021-11-03T18:30:27.900 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.897+0000 7f369659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36980f5ef0 0x7f369810c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:30:27.900 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.897+0000 7f369659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36980f5ef0 0x7f369810c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35264/0 (socket says 172.21.15.27:35264) 2021-11-03T18:30:27.901 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.897+0000 7f369659c700 1 -- 172.21.15.27:0/884374735 learned_addr learned my addr 172.21.15.27:0/884374735 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:30:27.901 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.897+0000 7f3696d9d700 1 --2- 172.21.15.27:0/884374735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36980f55e0 0x7f3698109d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:30:27.901 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.897+0000 7f369659c700 1 -- 172.21.15.27:0/884374735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36980f55e0 msgr2=0x7f3698109d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:30:27.901 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.897+0000 7f369659c700 1 --2- 172.21.15.27:0/884374735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36980f55e0 0x7f3698109d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:27.902 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.897+0000 7f369659c700 1 -- 172.21.15.27:0/884374735 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3690009640 con 0x7f36980f5ef0 2021-11-03T18:30:27.902 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.897+0000 7f369659c700 1 --2- 172.21.15.27:0/884374735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36980f5ef0 0x7f369810c260 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 rx=0x7f36800026d0 tx=0x7f3680009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:30:27.902 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.897+0000 7f368f7fe700 1 -- 172.21.15.27:0/884374735 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36800079c0 con 0x7f36980f5ef0 2021-11-03T18:30:27.902 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.897+0000 7f368f7fe700 1 -- 172.21.15.27:0/884374735 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f368000f040 con 0x7f36980f5ef0 2021-11-03T18:30:27.903 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.897+0000 7f369d145700 1 -- 172.21.15.27:0/884374735 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f369810c810 con 0x7f36980f5ef0 2021-11-03T18:30:27.903 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.897+0000 7f368f7fe700 1 -- 172.21.15.27:0/884374735 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36800148b0 con 0x7f36980f5ef0 2021-11-03T18:30:27.903 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.897+0000 7f369d145700 1 -- 172.21.15.27:0/884374735 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f369810cd50 con 0x7f36980f5ef0 2021-11-03T18:30:27.903 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.898+0000 7f369d145700 1 -- 172.21.15.27:0/884374735 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3698045ad0 con 0x7f36980f5ef0 2021-11-03T18:30:27.904 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.898+0000 7f368f7fe700 1 -- 172.21.15.27:0/884374735 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f3680007b20 con 0x7f36980f5ef0 2021-11-03T18:30:27.904 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.899+0000 7f368f7fe700 1 --2- 172.21.15.27:0/884374735 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f36840640b0 0x7f3684066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:30:27.904 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.899+0000 7f368f7fe700 1 -- 172.21.15.27:0/884374735 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f3680080740 con 0x7f36980f5ef0 2021-11-03T18:30:27.904 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.899+0000 7f3696d9d700 1 --2- 172.21.15.27:0/884374735 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f36840640b0 0x7f3684066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:30:27.904 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.899+0000 7f3696d9d700 1 --2- 172.21.15.27:0/884374735 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f36840640b0 0x7f3684066570 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7f369000b730 tx=0x7f3690004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:30:27.905 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:27.903+0000 7f368f7fe700 1 -- 172.21.15.27:0/884374735 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f3680052aa0 con 0x7f36980f5ef0 2021-11-03T18:30:28.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:28.109+0000 7f369d145700 1 -- 172.21.15.27:0/884374735 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f36980fdea0 con 0x7f36980f5ef0 2021-11-03T18:30:28.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:28.109+0000 7f368f7fe700 1 -- 172.21.15.27:0/884374735 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f3680052690 con 0x7f36980f5ef0 2021-11-03T18:30:28.109 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:30:28.109 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:30:28.109 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:30:28.110 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:30:28.110 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:30:28.110 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:30:28.110 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:30:28.111 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:30:28.111 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:30:28.111 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:30:28.111 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:30:28.112 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:30:28.112 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:30:28.112 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:30:28.112 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:30:28.113 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:30:28.113 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:30:28.113 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:30:28.113 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:30:28.114 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:30:28.114 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:30:28.114 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:30:28.114 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:30:28.115 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:30:28.115 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:30:28.115 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:30:28.115 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:30:28.115 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:30:28.116 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:30:28.116 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:30:28.116 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:30:28.116 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:30:28.117 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:30:28.117 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:30:28.117 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:30:28.117 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:30:28.117 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:30:28.118 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:30:28.118 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:30:28.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:28.111+0000 7f369d145700 1 -- 172.21.15.27:0/884374735 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f36840640b0 msgr2=0x7f3684066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:30:28.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:28.111+0000 7f369d145700 1 --2- 172.21.15.27:0/884374735 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f36840640b0 0x7f3684066570 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7f369000b730 tx=0x7f3690004d10).stop 2021-11-03T18:30:28.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:28.111+0000 7f369d145700 1 -- 172.21.15.27:0/884374735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36980f5ef0 msgr2=0x7f369810c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:30:28.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:28.111+0000 7f369d145700 1 --2- 172.21.15.27:0/884374735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36980f5ef0 0x7f369810c260 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 rx=0x7f36800026d0 tx=0x7f3680009de0).stop 2021-11-03T18:30:28.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:28.111+0000 7f369d145700 1 -- 172.21.15.27:0/884374735 shutdown_connections 2021-11-03T18:30:28.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:28.111+0000 7f369d145700 1 --2- 172.21.15.27:0/884374735 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f36840640b0 0x7f3684066570 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:28.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:28.111+0000 7f369d145700 1 --2- 172.21.15.27:0/884374735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f36980f55e0 0x7f3698109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:28.121 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:28.111+0000 7f369d145700 1 --2- 172.21.15.27:0/884374735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f36980f5ef0 0x7f369810c260 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:28.122 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:28.111+0000 7f369d145700 1 -- 172.21.15.27:0/884374735 >> 172.21.15.27:0/884374735 conn(0x7f36980f0de0 msgr2=0x7f36980fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:30:28.122 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:28.111+0000 7f369d145700 1 -- 172.21.15.27:0/884374735 shutdown_connections 2021-11-03T18:30:28.122 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:28.111+0000 7f369d145700 1 -- 172.21.15.27:0/884374735 wait complete. 2021-11-03T18:30:28.122 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:30:28.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:28 smithi115 ceph-mon[65453]: pgmap v2276: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:28.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:28 smithi115 ceph-mon[65453]: from='client.36440 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:30:28.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:28 smithi115 ceph-mon[65453]: from='client.36444 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:30:28.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:28 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1705845822' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:28.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:28 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/884374735' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:30:29.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:28 smithi027 ceph-mon[97244]: pgmap v2276: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:29.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:28 smithi027 ceph-mon[97244]: from='client.36440 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:30:29.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:28 smithi027 ceph-mon[97244]: from='client.36444 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:30:29.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:28 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1705845822' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:29.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:28 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/884374735' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:30:30.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:30 smithi115 ceph-mon[65453]: pgmap v2277: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:30.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4472.466927 secs 2021-11-03T18:30:31.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:30 smithi027 ceph-mon[97244]: pgmap v2277: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:31.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4472.466927 secs 2021-11-03T18:30:32.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:32 smithi027 ceph-mon[97244]: pgmap v2278: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:32.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:32 smithi115 ceph-mon[65453]: pgmap v2278: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:34.932 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:34 smithi115 ceph-mon[65453]: pgmap v2279: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:34.937 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:34 smithi027 ceph-mon[97244]: pgmap v2279: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:35.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4477.467069 secs 2021-11-03T18:30:35.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:30:35.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:35 smithi115 ceph-mon[65453]: poll took 20076.832 ms: 1 events 2021-11-03T18:30:35.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:30:35.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:30:35.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:35.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:35.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:35.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:36.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4477.467069 secs 2021-11-03T18:30:36.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:30:36.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:35 smithi027 ceph-mon[97244]: poll took 20076.832 ms: 1 events 2021-11-03T18:30:36.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:30:36.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:30:36.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:36.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:36.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:36.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:36.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:36 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:30:36.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:36 smithi115 ceph-mon[65453]: pgmap v2280: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:37.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:36 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:30:37.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:36 smithi027 ceph-mon[97244]: pgmap v2280: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:38.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:38 smithi115 ceph-mon[65453]: pgmap v2281: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:39.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:38 smithi027 ceph-mon[97244]: pgmap v2281: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:40.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:40 smithi115 ceph-mon[65453]: pgmap v2282: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:40.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4482.467198 secs 2021-11-03T18:30:41.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:40 smithi027 ceph-mon[97244]: pgmap v2282: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:41.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4482.467198 secs 2021-11-03T18:30:42.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:42 smithi027 ceph-mon[97244]: pgmap v2283: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:42.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:42 smithi115 ceph-mon[65453]: pgmap v2283: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:44.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:44 smithi115 ceph-mon[65453]: pgmap v2284: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:45.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:44 smithi027 ceph-mon[97244]: pgmap v2284: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:45.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4487.467337 secs 2021-11-03T18:30:45.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:30:45.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:30:45.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:30:45.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:45.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:45.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:45.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:46.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4487.467337 secs 2021-11-03T18:30:46.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:30:46.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:30:46.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:30:46.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:46.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:46.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:46.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:46.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:46 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:30:46.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:46 smithi115 ceph-mon[65453]: pgmap v2285: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:47.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:46 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:30:47.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:46 smithi027 ceph-mon[97244]: pgmap v2285: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:48.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:48 smithi115 ceph-mon[65453]: pgmap v2286: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:49.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:48 smithi027 ceph-mon[97244]: pgmap v2286: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:50.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:50 smithi115 ceph-mon[65453]: pgmap v2287: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:50.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4492.467460 secs 2021-11-03T18:30:51.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:50 smithi027 ceph-mon[97244]: pgmap v2287: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:51.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4492.467460 secs 2021-11-03T18:30:52.946 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:52 smithi027 ceph-mon[97244]: pgmap v2288: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:52.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:52 smithi115 ceph-mon[65453]: pgmap v2288: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:54.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:54 smithi115 ceph-mon[65453]: pgmap v2289: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:55.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:54 smithi027 ceph-mon[97244]: pgmap v2289: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:55.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4497.467588 secs 2021-11-03T18:30:55.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:30:55.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:30:55.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:30:55.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:55.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:55.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:55.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:56.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4497.467588 secs 2021-11-03T18:30:56.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:30:56.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:30:56.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:30:56.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:56.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:56.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:56.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:56.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:56 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:30:56.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:56 smithi115 ceph-mon[65453]: pgmap v2290: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:56.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:30:56.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:30:57.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:56 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:30:57.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:56 smithi027 ceph-mon[97244]: pgmap v2290: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:57.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:30:57.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:30:58.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.227+0000 7f8476296700 1 -- 172.21.15.27:0/2727299639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84700f6530 msgr2=0x7f84700f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:30:58.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.227+0000 7f8476296700 1 --2- 172.21.15.27:0/2727299639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84700f6530 0x7f84700f6950 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 rx=0x7f846c004660 tx=0x7f846c009b30).stop 2021-11-03T18:30:58.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.227+0000 7f8476296700 1 -- 172.21.15.27:0/2727299639 shutdown_connections 2021-11-03T18:30:58.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.227+0000 7f8476296700 1 --2- 172.21.15.27:0/2727299639 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84700f77d0 0x7f84700f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:58.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.227+0000 7f8476296700 1 --2- 172.21.15.27:0/2727299639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84700f6530 0x7f84700f6950 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:58.230 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.227+0000 7f8476296700 1 -- 172.21.15.27:0/2727299639 >> 172.21.15.27:0/2727299639 conn(0x7f84700f1cd0 msgr2=0x7f84700f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:30:58.230 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.227+0000 7f8476296700 1 -- 172.21.15.27:0/2727299639 shutdown_connections 2021-11-03T18:30:58.230 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.227+0000 7f8476296700 1 -- 172.21.15.27:0/2727299639 wait complete. 2021-11-03T18:30:58.230 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.227+0000 7f8476296700 1 Processor -- start 2021-11-03T18:30:58.231 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.228+0000 7f8476296700 1 -- start start 2021-11-03T18:30:58.231 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.228+0000 7f8476296700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84700f6530 0x7f8470108f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:30:58.231 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.228+0000 7f8476296700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84700f77d0 0x7f8470109450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:30:58.231 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.228+0000 7f8476296700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8470109a00 con 0x7f84700f77d0 2021-11-03T18:30:58.232 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.228+0000 7f8476296700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8470109b40 con 0x7f84700f6530 2021-11-03T18:30:58.232 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.228+0000 7f8474a93700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84700f77d0 0x7f8470109450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:30:58.232 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.228+0000 7f8474a93700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84700f77d0 0x7f8470109450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35296/0 (socket says 172.21.15.27:35296) 2021-11-03T18:30:58.232 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.228+0000 7f8474a93700 1 -- 172.21.15.27:0/867906017 learned_addr learned my addr 172.21.15.27:0/867906017 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:30:58.232 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.228+0000 7f8475294700 1 --2- 172.21.15.27:0/867906017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84700f6530 0x7f8470108f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:30:58.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.228+0000 7f8474a93700 1 -- 172.21.15.27:0/867906017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84700f6530 msgr2=0x7f8470108f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:30:58.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.228+0000 7f8474a93700 1 --2- 172.21.15.27:0/867906017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84700f6530 0x7f8470108f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:58.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.228+0000 7f8474a93700 1 -- 172.21.15.27:0/867906017 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f846c005040 con 0x7f84700f77d0 2021-11-03T18:30:58.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.228+0000 7f8474a93700 1 --2- 172.21.15.27:0/867906017 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84700f77d0 0x7f8470109450 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 rx=0x7f846000c790 tx=0x7f8460007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:30:58.234 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.228+0000 7f8465ffb700 1 -- 172.21.15.27:0/867906017 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f846000e680 con 0x7f84700f77d0 2021-11-03T18:30:58.234 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.229+0000 7f8465ffb700 1 -- 172.21.15.27:0/867906017 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f846000ec90 con 0x7f84700f77d0 2021-11-03T18:30:58.234 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.229+0000 7f8476296700 1 -- 172.21.15.27:0/867906017 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f847010a4e0 con 0x7f84700f77d0 2021-11-03T18:30:58.235 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.229+0000 7f8465ffb700 1 -- 172.21.15.27:0/867906017 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8460013d60 con 0x7f84700f77d0 2021-11-03T18:30:58.235 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.229+0000 7f8476296700 1 -- 172.21.15.27:0/867906017 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f847010aaa0 con 0x7f84700f77d0 2021-11-03T18:30:58.236 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.230+0000 7f8476296700 1 -- 172.21.15.27:0/867906017 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8470045ad0 con 0x7f84700f77d0 2021-11-03T18:30:58.236 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.230+0000 7f8465ffb700 1 -- 172.21.15.27:0/867906017 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f846001a060 con 0x7f84700f77d0 2021-11-03T18:30:58.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.231+0000 7f8465ffb700 1 --2- 172.21.15.27:0/867906017 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f845c064100 0x7f845c0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:30:58.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.231+0000 7f8465ffb700 1 -- 172.21.15.27:0/867906017 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f8460080ad0 con 0x7f84700f77d0 2021-11-03T18:30:58.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.231+0000 7f8475294700 1 --2- 172.21.15.27:0/867906017 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f845c064100 0x7f845c0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:30:58.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.231+0000 7f8475294700 1 --2- 172.21.15.27:0/867906017 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f845c064100 0x7f845c0665c0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7f846c009da0 tx=0x7f846c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:30:58.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.235+0000 7f8465ffb700 1 -- 172.21.15.27:0/867906017 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f8460052d40 con 0x7f84700f77d0 2021-11-03T18:30:58.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.415+0000 7f8476296700 1 -- 172.21.15.27:0/867906017 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f84700008d0 con 0x7f845c064100 2021-11-03T18:30:58.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.416+0000 7f8465ffb700 1 -- 172.21.15.27:0/867906017 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f84700008d0 con 0x7f845c064100 2021-11-03T18:30:58.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.418+0000 7f8476296700 1 -- 172.21.15.27:0/867906017 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f845c064100 msgr2=0x7f845c0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:30:58.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.418+0000 7f8476296700 1 --2- 172.21.15.27:0/867906017 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f845c064100 0x7f845c0665c0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 rx=0x7f846c009da0 tx=0x7f846c009b30).stop 2021-11-03T18:30:58.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.418+0000 7f8476296700 1 -- 172.21.15.27:0/867906017 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84700f77d0 msgr2=0x7f8470109450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:30:58.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.418+0000 7f8476296700 1 --2- 172.21.15.27:0/867906017 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84700f77d0 0x7f8470109450 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 rx=0x7f846000c790 tx=0x7f8460007d70).stop 2021-11-03T18:30:58.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.418+0000 7f8476296700 1 -- 172.21.15.27:0/867906017 shutdown_connections 2021-11-03T18:30:58.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.418+0000 7f8476296700 1 --2- 172.21.15.27:0/867906017 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f845c064100 0x7f845c0665c0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:58.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.418+0000 7f8476296700 1 --2- 172.21.15.27:0/867906017 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84700f6530 0x7f8470108f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:58.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.418+0000 7f8476296700 1 --2- 172.21.15.27:0/867906017 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84700f77d0 0x7f8470109450 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:58.419 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.418+0000 7f8476296700 1 -- 172.21.15.27:0/867906017 >> 172.21.15.27:0/867906017 conn(0x7f84700f1cd0 msgr2=0x7f847006a520 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:30:58.419 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.418+0000 7f8476296700 1 -- 172.21.15.27:0/867906017 shutdown_connections 2021-11-03T18:30:58.419 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.418+0000 7f8476296700 1 -- 172.21.15.27:0/867906017 wait complete. 2021-11-03T18:30:58.426 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:30:58.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.518+0000 7f0418ec8700 1 -- 172.21.15.27:0/3199891012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04140f5440 msgr2=0x7f04140f5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:30:58.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.518+0000 7f0418ec8700 1 --2- 172.21.15.27:0/3199891012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04140f5440 0x7f04140f5860 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 rx=0x7f0404004660 tx=0x7f0404009b30).stop 2021-11-03T18:30:58.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.518+0000 7f0418ec8700 1 -- 172.21.15.27:0/3199891012 shutdown_connections 2021-11-03T18:30:58.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.518+0000 7f0418ec8700 1 --2- 172.21.15.27:0/3199891012 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04140f66e0 0x7f04140f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T18:30:58.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.518+0000 7f0418ec8700 1 --2- 172.21.15.27:0/3199891012 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04140f5440 0x7f04140f5860 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:58.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.518+0000 7f0418ec8700 1 -- 172.21.15.27:0/3199891012 >> 172.21.15.27:0/3199891012 conn(0x7f04140f0ba0 msgr2=0x7f04140f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:30:58.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.518+0000 7f0418ec8700 1 -- 172.21.15.27:0/3199891012 shutdown_connections 2021-11-03T18:30:58.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.518+0000 7f0418ec8700 1 -- 172.21.15.27:0/3199891012 wait complete. 2021-11-03T18:30:58.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.519+0000 7f0418ec8700 1 Processor -- start 2021-11-03T18:30:58.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.519+0000 7f0418ec8700 1 -- start start 2021-11-03T18:30:58.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.519+0000 7f0418ec8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04140f5440 0x7f04140fd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:30:58.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.519+0000 7f0418ec8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04140f66e0 0x7f04140fba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:30:58.524 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.519+0000 7f0418ec8700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041410aaa0 con 0x7f04140f66e0 2021-11-03T18:30:58.524 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.519+0000 7f0418ec8700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f041410abe0 con 0x7f04140f5440 2021-11-03T18:30:58.524 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.519+0000 7f0412ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04140f66e0 0x7f04140fba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:30:58.524 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.519+0000 7f0412ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04140f66e0 0x7f04140fba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35306/0 (socket says 172.21.15.27:35306) 2021-11-03T18:30:58.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.519+0000 7f0412ffd700 1 -- 172.21.15.27:0/3816445886 learned_addr learned my addr 172.21.15.27:0/3816445886 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:30:58.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.519+0000 7f04137fe700 1 --2- 172.21.15.27:0/3816445886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04140f5440 0x7f04140fd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:30:58.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.520+0000 7f0412ffd700 1 -- 172.21.15.27:0/3816445886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04140f5440 msgr2=0x7f04140fd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:30:58.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.520+0000 7f0412ffd700 1 --2- 172.21.15.27:0/3816445886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04140f5440 0x7f04140fd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:58.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.520+0000 7f0412ffd700 1 -- 172.21.15.27:0/3816445886 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0404005040 con 0x7f04140f66e0 2021-11-03T18:30:58.526 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.520+0000 7f04137fe700 1 --2- 172.21.15.27:0/3816445886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04140f5440 0x7f04140fd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T18:30:58.526 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.520+0000 7f0412ffd700 1 --2- 172.21.15.27:0/3816445886 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04140f66e0 0x7f04140fba40 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 rx=0x7f040800b410 tx=0x7f0408006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:30:58.526 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.520+0000 7f03fbfff700 1 -- 172.21.15.27:0/3816445886 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04080125f0 con 0x7f04140f66e0 2021-11-03T18:30:58.526 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.520+0000 7f0418ec8700 1 -- 172.21.15.27:0/3816445886 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04140fc080 con 0x7f04140f66e0 2021-11-03T18:30:58.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.520+0000 7f03fbfff700 1 -- 172.21.15.27:0/3816445886 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f0408012c00 con 0x7f04140f66e0 2021-11-03T18:30:58.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.520+0000 7f03fbfff700 1 -- 172.21.15.27:0/3816445886 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0408002ae0 con 0x7f04140f66e0 2021-11-03T18:30:58.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.520+0000 7f0418ec8700 1 -- 172.21.15.27:0/3816445886 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04140fc640 con 0x7f04140f66e0 2021-11-03T18:30:58.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.521+0000 7f0418ec8700 1 -- 172.21.15.27:0/3816445886 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0414045ad0 con 0x7f04140f66e0 2021-11-03T18:30:58.527 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.521+0000 7f03fbfff700 1 -- 172.21.15.27:0/3816445886 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f040800f030 con 0x7f04140f66e0 2021-11-03T18:30:58.528 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.522+0000 7f03fbfff700 1 --2- 172.21.15.27:0/3816445886 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f03fc064100 0x7f03fc0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:30:58.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.522+0000 7f03fbfff700 1 -- 172.21.15.27:0/3816445886 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f0408080c80 con 0x7f04140f66e0 2021-11-03T18:30:58.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.522+0000 7f04137fe700 1 --2- 172.21.15.27:0/3816445886 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f03fc064100 0x7f03fc0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:30:58.529 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.522+0000 7f04137fe700 1 --2- 172.21.15.27:0/3816445886 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f03fc064100 0x7f03fc0665c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f0404004e30 tx=0x7f0404009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:30:58.530 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.526+0000 7f03fbfff700 1 -- 172.21.15.27:0/3816445886 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f0408052fe0 con 0x7f04140f66e0 2021-11-03T18:30:58.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.698+0000 7f0418ec8700 1 -- 172.21.15.27:0/3816445886 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f04140fed80 con 0x7f03fc064100 2021-11-03T18:30:58.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.713+0000 7f03fbfff700 1 -- 172.21.15.27:0/3816445886 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f04140fed80 con 0x7f03fc064100 2021-11-03T18:30:58.712 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:30:58.713 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (82m) 77s ago 83m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:30:58.713 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (74m) 77s ago 83m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:30:58.713 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (74m) 44s ago 83m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:30:58.713 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (82m) 77s ago 83m 42.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:30:58.713 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (76m) 77s ago 78m 26.7M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:30:58.714 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (79m) 77s ago 78m 3850M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:30:58.714 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (74m) 44s ago 78m 26.3M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:30:58.714 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (76m) 44s ago 79m 27.4M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:30:58.714 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (76m) 77s ago 85m 608M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:30:58.715 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (75m) 44s ago 82m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:30:58.715 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (75m) 77s ago 85m 350M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:30:58.715 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (74m) 44s ago 82m 160M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:30:58.715 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (83m) 77s ago 83m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:30:58.716 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (82m) 44s ago 82m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:30:58.716 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (73m) 77s ago 82m 413M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:30:58.716 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (73m) 77s ago 81m 450M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:30:58.716 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (72m) 77s ago 81m 338M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:30:58.716 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (72m) 44s ago 81m 479M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:30:58.717 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (71m) 44s ago 80m 378M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:30:58.717 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (71m) 44s ago 80m 388M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:30:58.717 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (82m) 77s ago 83m 64.4M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:30:58.718 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.715+0000 7f0418ec8700 1 -- 172.21.15.27:0/3816445886 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f03fc064100 msgr2=0x7f03fc0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:30:58.718 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.715+0000 7f0418ec8700 1 --2- 172.21.15.27:0/3816445886 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f03fc064100 0x7f03fc0665c0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 rx=0x7f0404004e30 tx=0x7f0404009b30).stop 2021-11-03T18:30:58.718 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.715+0000 7f0418ec8700 1 -- 172.21.15.27:0/3816445886 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04140f66e0 msgr2=0x7f04140fba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:30:58.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.715+0000 7f0418ec8700 1 --2- 172.21.15.27:0/3816445886 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04140f66e0 0x7f04140fba40 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 rx=0x7f040800b410 tx=0x7f0408006ef0).stop 2021-11-03T18:30:58.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.715+0000 7f0418ec8700 1 -- 172.21.15.27:0/3816445886 shutdown_connections 2021-11-03T18:30:58.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.715+0000 7f0418ec8700 1 --2- 172.21.15.27:0/3816445886 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f03fc064100 0x7f03fc0665c0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:58.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.715+0000 7f0418ec8700 1 --2- 172.21.15.27:0/3816445886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04140f5440 0x7f04140fd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:58.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.715+0000 7f0418ec8700 1 --2- 172.21.15.27:0/3816445886 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04140f66e0 0x7f04140fba40 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:58.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.715+0000 7f0418ec8700 1 -- 172.21.15.27:0/3816445886 >> 172.21.15.27:0/3816445886 conn(0x7f04140f0ba0 msgr2=0x7f04140fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:30:58.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.715+0000 7f0418ec8700 1 -- 172.21.15.27:0/3816445886 shutdown_connections 2021-11-03T18:30:58.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.715+0000 7f0418ec8700 1 -- 172.21.15.27:0/3816445886 wait complete. 2021-11-03T18:30:58.813 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.814+0000 7fcc0fd8a700 1 -- 172.21.15.27:0/229115395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc080f5ef0 msgr2=0x7fcc080f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:30:58.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.814+0000 7fcc0fd8a700 1 --2- 172.21.15.27:0/229115395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc080f5ef0 0x7fcc080f6920 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 rx=0x7fcbfc004660 tx=0x7fcbfc009b30).stop 2021-11-03T18:30:58.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.814+0000 7fcc0fd8a700 1 -- 172.21.15.27:0/229115395 shutdown_connections 2021-11-03T18:30:58.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.814+0000 7fcc0fd8a700 1 --2- 172.21.15.27:0/229115395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc080f5ef0 0x7fcc080f6920 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:58.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.814+0000 7fcc0fd8a700 1 --2- 172.21.15.27:0/229115395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc080f55e0 0x7fcc080f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:58.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.814+0000 7fcc0fd8a700 1 -- 172.21.15.27:0/229115395 >> 172.21.15.27:0/229115395 conn(0x7fcc080f0de0 msgr2=0x7fcc080f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:30:58.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.814+0000 7fcc0fd8a700 1 -- 172.21.15.27:0/229115395 shutdown_connections 2021-11-03T18:30:58.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.814+0000 7fcc0fd8a700 1 -- 172.21.15.27:0/229115395 wait complete. 2021-11-03T18:30:58.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.815+0000 7fcc0fd8a700 1 Processor -- start 2021-11-03T18:30:58.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.815+0000 7fcc0fd8a700 1 -- start start 2021-11-03T18:30:58.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.815+0000 7fcc0fd8a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc080f55e0 0x7fcc08109c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:30:58.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.815+0000 7fcc0fd8a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc080f5ef0 0x7fcc0810c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:30:58.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.815+0000 7fcc0fd8a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc08060500 con 0x7fcc080f5ef0 2021-11-03T18:30:58.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.815+0000 7fcc0fd8a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc08060640 con 0x7fcc080f55e0 2021-11-03T18:30:58.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.815+0000 7fcc0d325700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc080f5ef0 0x7fcc0810c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:30:58.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.815+0000 7fcc0d325700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc080f5ef0 0x7fcc0810c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35316/0 (socket says 172.21.15.27:35316) 2021-11-03T18:30:58.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.815+0000 7fcc0d325700 1 -- 172.21.15.27:0/532238352 learned_addr learned my addr 172.21.15.27:0/532238352 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:30:58.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.815+0000 7fcc0db26700 1 --2- 172.21.15.27:0/532238352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc080f55e0 0x7fcc08109c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:30:58.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.815+0000 7fcc0d325700 1 -- 172.21.15.27:0/532238352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc080f55e0 msgr2=0x7fcc08109c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:30:58.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.815+0000 7fcc0d325700 1 --2- 172.21.15.27:0/532238352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc080f55e0 0x7fcc08109c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:58.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.815+0000 7fcc0d325700 1 -- 172.21.15.27:0/532238352 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbfc005040 con 0x7fcc080f5ef0 2021-11-03T18:30:58.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.816+0000 7fcc0d325700 1 --2- 172.21.15.27:0/532238352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc080f5ef0 0x7fcc0810c190 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 rx=0x7fcbfc009990 tx=0x7fcbfc009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:30:58.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.816+0000 7fcc067fc700 1 -- 172.21.15.27:0/532238352 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbfc018070 con 0x7fcc080f5ef0 2021-11-03T18:30:58.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.816+0000 7fcc067fc700 1 -- 172.21.15.27:0/532238352 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fcbfc007ba0 con 0x7fcc080f5ef0 2021-11-03T18:30:58.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.816+0000 7fcc0fd8a700 1 -- 172.21.15.27:0/532238352 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc0810c6e0 con 0x7fcc080f5ef0 2021-11-03T18:30:58.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.816+0000 7fcc067fc700 1 -- 172.21.15.27:0/532238352 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbfc01c3c0 con 0x7fcc080f5ef0 2021-11-03T18:30:58.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.816+0000 7fcc0fd8a700 1 -- 172.21.15.27:0/532238352 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc0810cc40 con 0x7fcc080f5ef0 2021-11-03T18:30:58.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.817+0000 7fcc0fd8a700 1 -- 172.21.15.27:0/532238352 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc08045ad0 con 0x7fcc080f5ef0 2021-11-03T18:30:58.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.818+0000 7fcc067fc700 1 -- 172.21.15.27:0/532238352 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fcbfc01d420 con 0x7fcc080f5ef0 2021-11-03T18:30:58.823 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.818+0000 7fcc067fc700 1 --2- 172.21.15.27:0/532238352 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcbf00640b0 0x7fcbf0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:30:58.823 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.818+0000 7fcc067fc700 1 -- 172.21.15.27:0/532238352 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fcbfc0826a0 con 0x7fcc080f5ef0 2021-11-03T18:30:58.823 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.818+0000 7fcc0db26700 1 --2- 172.21.15.27:0/532238352 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcbf00640b0 0x7fcbf0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:30:58.823 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.819+0000 7fcc0db26700 1 --2- 172.21.15.27:0/532238352 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcbf00640b0 0x7fcbf0066570 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7fcc08060180 tx=0x7fcc00009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:30:58.824 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:58.822+0000 7fcc067fc700 1 -- 172.21.15.27:0/532238352 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fcbfc0549c0 con 0x7fcc080f5ef0 2021-11-03T18:30:58.994 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:58 smithi115 ceph-mon[65453]: pgmap v2291: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:59.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:58 smithi027 ceph-mon[97244]: pgmap v2291: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:30:59.050 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.051+0000 7fcc0fd8a700 1 -- 172.21.15.27:0/532238352 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcc08027a00 con 0x7fcc080f5ef0 2021-11-03T18:30:59.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.052+0000 7fcc067fc700 1 -- 172.21.15.27:0/532238352 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fcbfc0545b0 con 0x7fcc080f5ef0 2021-11-03T18:30:59.051 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:30:59.052 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:30:59.052 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:30:59.052 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:30:59.052 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:30:59.053 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:30:59.053 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:30:59.053 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:30:59.053 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:30:59.054 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:30:59.054 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:30:59.054 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:30:59.054 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:30:59.054 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:30:59.055 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:30:59.055 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:30:59.055 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:30:59.056 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:30:59.056 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.053+0000 7fcc0fd8a700 1 -- 172.21.15.27:0/532238352 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcbf00640b0 msgr2=0x7fcbf0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:30:59.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.053+0000 7fcc0fd8a700 1 --2- 172.21.15.27:0/532238352 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcbf00640b0 0x7fcbf0066570 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 rx=0x7fcc08060180 tx=0x7fcc00009040).stop 2021-11-03T18:30:59.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.054+0000 7fcc0fd8a700 1 -- 172.21.15.27:0/532238352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc080f5ef0 msgr2=0x7fcc0810c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:30:59.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.054+0000 7fcc0fd8a700 1 --2- 172.21.15.27:0/532238352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc080f5ef0 0x7fcc0810c190 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 rx=0x7fcbfc009990 tx=0x7fcbfc009b30).stop 2021-11-03T18:30:59.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.054+0000 7fcc0fd8a700 1 -- 172.21.15.27:0/532238352 shutdown_connections 2021-11-03T18:30:59.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.054+0000 7fcc0fd8a700 1 --2- 172.21.15.27:0/532238352 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcbf00640b0 0x7fcbf0066570 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:59.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.054+0000 7fcc0fd8a700 1 --2- 172.21.15.27:0/532238352 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcc080f55e0 0x7fcc08109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:59.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.054+0000 7fcc0fd8a700 1 --2- 172.21.15.27:0/532238352 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcc080f5ef0 0x7fcc0810c190 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:59.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.054+0000 7fcc0fd8a700 1 -- 172.21.15.27:0/532238352 >> 172.21.15.27:0/532238352 conn(0x7fcc080f0de0 msgr2=0x7fcc080fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:30:59.059 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.054+0000 7fcc0fd8a700 1 -- 172.21.15.27:0/532238352 shutdown_connections 2021-11-03T18:30:59.059 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.054+0000 7fcc0fd8a700 1 -- 172.21.15.27:0/532238352 wait complete. 2021-11-03T18:30:59.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.154+0000 7fc009b45700 1 -- 172.21.15.27:0/788732201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0040f55e0 msgr2=0x7fc0040f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:30:59.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.154+0000 7fc009b45700 1 --2- 172.21.15.27:0/788732201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0040f55e0 0x7fc0040f5a00 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 rx=0x7fbff4004660 tx=0x7fbff4009b30).stop 2021-11-03T18:30:59.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.154+0000 7fc009b45700 1 -- 172.21.15.27:0/788732201 shutdown_connections 2021-11-03T18:30:59.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.154+0000 7fc009b45700 1 --2- 172.21.15.27:0/788732201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0040f5ef0 0x7fc0040f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:59.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.154+0000 7fc009b45700 1 --2- 172.21.15.27:0/788732201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0040f55e0 0x7fc0040f5a00 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:59.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.154+0000 7fc009b45700 1 -- 172.21.15.27:0/788732201 >> 172.21.15.27:0/788732201 conn(0x7fc0040f0de0 msgr2=0x7fc0040f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:30:59.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.154+0000 7fc009b45700 1 -- 172.21.15.27:0/788732201 shutdown_connections 2021-11-03T18:30:59.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.154+0000 7fc009b45700 1 -- 172.21.15.27:0/788732201 wait complete. 2021-11-03T18:30:59.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.154+0000 7fc009b45700 1 Processor -- start 2021-11-03T18:30:59.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.155+0000 7fc009b45700 1 -- start start 2021-11-03T18:30:59.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.155+0000 7fc009b45700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0040f55e0 0x7fc0040f97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:30:59.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.155+0000 7fc009b45700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0040f5ef0 0x7fc0040f7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:30:59.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.155+0000 7fc009b45700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0040f81c0 con 0x7fc0040f55e0 2021-11-03T18:30:59.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.155+0000 7fc009b45700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0040f8300 con 0x7fc0040f5ef0 2021-11-03T18:30:59.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.155+0000 7fc0037fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0040f55e0 0x7fc0040f97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:30:59.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.155+0000 7fc0037fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0040f55e0 0x7fc0040f97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35324/0 (socket says 172.21.15.27:35324) 2021-11-03T18:30:59.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.155+0000 7fc0037fe700 1 -- 172.21.15.27:0/753749721 learned_addr learned my addr 172.21.15.27:0/753749721 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:30:59.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.155+0000 7fc002ffd700 1 --2- 172.21.15.27:0/753749721 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0040f5ef0 0x7fc0040f7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:30:59.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.155+0000 7fc0037fe700 1 -- 172.21.15.27:0/753749721 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0040f5ef0 msgr2=0x7fc0040f7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:30:59.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.155+0000 7fc0037fe700 1 --2- 172.21.15.27:0/753749721 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0040f5ef0 0x7fc0040f7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:59.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.155+0000 7fc0037fe700 1 -- 172.21.15.27:0/753749721 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbff4005040 con 0x7fc0040f55e0 2021-11-03T18:30:59.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.155+0000 7fc0037fe700 1 --2- 172.21.15.27:0/753749721 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0040f55e0 0x7fc0040f97e0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 rx=0x7fbff4009990 tx=0x7fbff4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:30:59.161 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.156+0000 7fbfebfff700 1 -- 172.21.15.27:0/753749721 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbff4018070 con 0x7fc0040f55e0 2021-11-03T18:30:59.161 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.156+0000 7fc009b45700 1 -- 172.21.15.27:0/753749721 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0040f8580 con 0x7fc0040f55e0 2021-11-03T18:30:59.161 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.156+0000 7fbfebfff700 1 -- 172.21.15.27:0/753749721 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fbff4007ba0 con 0x7fc0040f55e0 2021-11-03T18:30:59.162 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.156+0000 7fbfebfff700 1 -- 172.21.15.27:0/753749721 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbff401c620 con 0x7fc0040f55e0 2021-11-03T18:30:59.162 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.156+0000 7fc009b45700 1 -- 172.21.15.27:0/753749721 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0040f8ae0 con 0x7fc0040f55e0 2021-11-03T18:30:59.162 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.157+0000 7fc009b45700 1 -- 172.21.15.27:0/753749721 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc004045ad0 con 0x7fc0040f55e0 2021-11-03T18:30:59.163 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.158+0000 7fbfebfff700 1 -- 172.21.15.27:0/753749721 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fbff401c8e0 con 0x7fc0040f55e0 2021-11-03T18:30:59.163 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.158+0000 7fbfebfff700 1 --2- 172.21.15.27:0/753749721 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbfec0640b0 0x7fbfec066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:30:59.163 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.158+0000 7fbfebfff700 1 -- 172.21.15.27:0/753749721 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fbff4082870 con 0x7fc0040f55e0 2021-11-03T18:30:59.163 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.158+0000 7fc002ffd700 1 --2- 172.21.15.27:0/753749721 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbfec0640b0 0x7fbfec066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:30:59.163 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.158+0000 7fc002ffd700 1 --2- 172.21.15.27:0/753749721 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbfec0640b0 0x7fbfec066570 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7fbff80041d0 tx=0x7fbff8009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:30:59.164 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.162+0000 7fbfebfff700 1 -- 172.21.15.27:0/753749721 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fbff4054bd0 con 0x7fc0040f55e0 2021-11-03T18:30:59.365 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.366+0000 7fc009b45700 1 -- 172.21.15.27:0/753749721 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc00410f690 con 0x7fc0040f55e0 2021-11-03T18:30:59.366 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.367+0000 7fbfebfff700 1 -- 172.21.15.27:0/753749721 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7fbff40547c0 con 0x7fc0040f55e0 2021-11-03T18:30:59.367 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:30:59.367 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:30:59.367 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:30:59.367 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:30:59.368 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:30:59.368 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:30:59.368 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:30:59.368 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:30:59.369 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:30:59.369 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:30:59.369 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:30:59.369 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:30:59.370 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:30:59.370 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:30:59.370 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:30:59.370 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:30:59.371 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:30:59.371 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:30:59.371 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:30:59.371 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:30:59.372 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:30:59.372 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:30:59.372 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:30:59.372 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:30:59.372 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:30:59.373 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:30:59.373 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:30:59.373 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:30:59.374 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:30:59.374 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:30:59.374 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:30:59.375 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:30:59.375 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:30:59.375 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:30:59.375 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:30:59.375 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:30:59.376 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:30:59.376 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:30:59.376 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:30:59.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.368+0000 7fc009b45700 1 -- 172.21.15.27:0/753749721 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbfec0640b0 msgr2=0x7fbfec066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:30:59.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.368+0000 7fc009b45700 1 --2- 172.21.15.27:0/753749721 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbfec0640b0 0x7fbfec066570 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7fbff80041d0 tx=0x7fbff8009040).stop 2021-11-03T18:30:59.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.368+0000 7fc009b45700 1 -- 172.21.15.27:0/753749721 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0040f55e0 msgr2=0x7fc0040f97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:30:59.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.368+0000 7fc009b45700 1 --2- 172.21.15.27:0/753749721 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0040f55e0 0x7fc0040f97e0 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 rx=0x7fbff4009990 tx=0x7fbff4009b30).stop 2021-11-03T18:30:59.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.369+0000 7fc009b45700 1 -- 172.21.15.27:0/753749721 shutdown_connections 2021-11-03T18:30:59.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.369+0000 7fc009b45700 1 --2- 172.21.15.27:0/753749721 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbfec0640b0 0x7fbfec066570 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:59.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.369+0000 7fc009b45700 1 --2- 172.21.15.27:0/753749721 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc0040f55e0 0x7fc0040f97e0 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:59.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.369+0000 7fc009b45700 1 --2- 172.21.15.27:0/753749721 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc0040f5ef0 0x7fc0040f7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:30:59.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.369+0000 7fc009b45700 1 -- 172.21.15.27:0/753749721 >> 172.21.15.27:0/753749721 conn(0x7fc0040f0de0 msgr2=0x7fc00405ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:30:59.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.369+0000 7fc009b45700 1 -- 172.21.15.27:0/753749721 shutdown_connections 2021-11-03T18:30:59.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:30:59.369+0000 7fc009b45700 1 -- 172.21.15.27:0/753749721 wait complete. 2021-11-03T18:30:59.381 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:30:59.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:59 smithi115 ceph-mon[65453]: from='client.36456 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:30:59.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:59 smithi115 ceph-mon[65453]: from='client.36460 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:30:59.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:59 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/532238352' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:30:59.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:30:59 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/753749721' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:31:00.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:59 smithi027 ceph-mon[97244]: from='client.36456 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:31:00.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:59 smithi027 ceph-mon[97244]: from='client.36460 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:31:00.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:59 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/532238352' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:00.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:30:59 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/753749721' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:31:00.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:00 smithi115 ceph-mon[65453]: pgmap v2292: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:00.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4502.467696 secs 2021-11-03T18:31:01.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:00 smithi027 ceph-mon[97244]: pgmap v2292: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:01.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4502.467696 secs 2021-11-03T18:31:02.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:02 smithi027 ceph-mon[97244]: pgmap v2293: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:02.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:02 smithi115 ceph-mon[65453]: pgmap v2293: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:04.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:04 smithi115 ceph-mon[65453]: pgmap v2294: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:05.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:04 smithi027 ceph-mon[97244]: pgmap v2294: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:05.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4507.467811 secs 2021-11-03T18:31:05.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:31:05.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:31:05.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:31:05.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:05.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:05.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:05.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:06.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4507.467811 secs 2021-11-03T18:31:06.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:31:06.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:31:06.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:31:06.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:06.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:06.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:06.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:06.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:06 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:31:06.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:06 smithi115 ceph-mon[65453]: pgmap v2295: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:07.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:06 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:31:07.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:06 smithi027 ceph-mon[97244]: pgmap v2295: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:08.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:08 smithi115 ceph-mon[65453]: pgmap v2296: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:09.394 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:08 smithi027 ceph-mon[97244]: pgmap v2296: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:10.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:10 smithi115 ceph-mon[65453]: pgmap v2297: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:10.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4512.467958 secs 2021-11-03T18:31:11.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:10 smithi027 ceph-mon[97244]: pgmap v2297: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:11.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4512.467958 secs 2021-11-03T18:31:12.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:12 smithi027 ceph-mon[97244]: pgmap v2298: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:12.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:12 smithi115 ceph-mon[65453]: pgmap v2298: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:14.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:14 smithi115 ceph-mon[65453]: pgmap v2299: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:15.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:14 smithi027 ceph-mon[97244]: pgmap v2299: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:15.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4517.468070 secs 2021-11-03T18:31:15.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:31:15.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:31:15.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:31:15.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:15.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:15.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:15.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:16.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4517.468070 secs 2021-11-03T18:31:16.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:31:16.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:31:16.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:31:16.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:16.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:16.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:16.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:16.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:16 smithi115 ceph-mon[65453]: pgmap v2300: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:16.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:16 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:31:17.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:16 smithi027 ceph-mon[97244]: pgmap v2300: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:17.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:16 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:31:18.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:18 smithi115 ceph-mon[65453]: pgmap v2301: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:19.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:18 smithi027 ceph-mon[97244]: pgmap v2301: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:20.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:20 smithi115 ceph-mon[65453]: pgmap v2302: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:20.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4522.468213 secs 2021-11-03T18:31:21.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:20 smithi027 ceph-mon[97244]: pgmap v2302: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:21.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4522.468213 secs 2021-11-03T18:31:22.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:22 smithi027 ceph-mon[97244]: pgmap v2303: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:22.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:22 smithi115 ceph-mon[65453]: pgmap v2303: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:24.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:24 smithi115 ceph-mon[65453]: pgmap v2304: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:25.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:24 smithi027 ceph-mon[97244]: pgmap v2304: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:25.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4527.468355 secs 2021-11-03T18:31:25.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:31:25.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:31:25.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:31:25.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:25.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:25.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:25.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:26.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4527.468355 secs 2021-11-03T18:31:26.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:31:26.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:31:26.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:31:26.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:26.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:26.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:26.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:26.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:26 smithi115 ceph-mon[65453]: pgmap v2305: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:26.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:26 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:31:27.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:26 smithi027 ceph-mon[97244]: pgmap v2305: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:27.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:26 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:31:28.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:28 smithi115 ceph-mon[65453]: pgmap v2306: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:29.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:28 smithi027 ceph-mon[97244]: pgmap v2306: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:29.482 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.482+0000 7f88e63d4700 1 -- 172.21.15.27:0/357192787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88e00fba10 msgr2=0x7f88e00fde00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:31:29.482 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.482+0000 7f88e63d4700 1 --2- 172.21.15.27:0/357192787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88e00fba10 0x7f88e00fde00 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 rx=0x7f88d0004660 tx=0x7f88d0009b30).stop 2021-11-03T18:31:29.482 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.482+0000 7f88e63d4700 1 -- 172.21.15.27:0/357192787 shutdown_connections 2021-11-03T18:31:29.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.482+0000 7f88e63d4700 1 --2- 172.21.15.27:0/357192787 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88e00fba10 0x7f88e00fde00 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:31:29.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.483+0000 7f88e63d4700 1 --2- 172.21.15.27:0/357192787 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88e0060310 0x7f88e00fb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:31:29.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.483+0000 7f88e63d4700 1 -- 172.21.15.27:0/357192787 >> 172.21.15.27:0/357192787 conn(0x7f88e00f0ba0 msgr2=0x7f88e00f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:31:29.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.483+0000 7f88e63d4700 1 -- 172.21.15.27:0/357192787 shutdown_connections 2021-11-03T18:31:29.484 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.483+0000 7f88e63d4700 1 -- 172.21.15.27:0/357192787 wait complete. 2021-11-03T18:31:29.486 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.483+0000 7f88e63d4700 1 Processor -- start 2021-11-03T18:31:29.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.483+0000 7f88e63d4700 1 -- start start 2021-11-03T18:31:29.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.484+0000 7f88e63d4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88e0060310 0x7f88e0108f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:31:29.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.484+0000 7f88e63d4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88e00fba10 0x7f88e0109420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:31:29.487 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.484+0000 7f88e63d4700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e01099d0 con 0x7f88e0060310 2021-11-03T18:31:29.488 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.484+0000 7f88e63d4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e0109b10 con 0x7f88e00fba10 2021-11-03T18:31:29.488 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.484+0000 7f88e53d2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88e0060310 0x7f88e0108f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:31:29.488 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.484+0000 7f88e53d2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88e0060310 0x7f88e0108f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35342/0 (socket says 172.21.15.27:35342) 2021-11-03T18:31:29.488 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.484+0000 7f88e53d2700 1 -- 172.21.15.27:0/637222807 learned_addr learned my addr 172.21.15.27:0/637222807 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:31:29.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.484+0000 7f88e4bd1700 1 --2- 172.21.15.27:0/637222807 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88e00fba10 0x7f88e0109420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:31:29.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.484+0000 7f88e53d2700 1 -- 172.21.15.27:0/637222807 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88e00fba10 msgr2=0x7f88e0109420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:31:29.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.484+0000 7f88e53d2700 1 --2- 172.21.15.27:0/637222807 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88e00fba10 0x7f88e0109420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:31:29.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.484+0000 7f88e53d2700 1 -- 172.21.15.27:0/637222807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d0005040 con 0x7f88e0060310 2021-11-03T18:31:29.489 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.484+0000 7f88e4bd1700 1 --2- 172.21.15.27:0/637222807 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88e00fba10 0x7f88e0109420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:31:29.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.484+0000 7f88e53d2700 1 --2- 172.21.15.27:0/637222807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88e0060310 0x7f88e0108f30 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 rx=0x7f88dc0026d0 tx=0x7f88dc009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:31:29.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.484+0000 7f88d5ffb700 1 -- 172.21.15.27:0/637222807 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88dc009320 con 0x7f88e0060310 2021-11-03T18:31:29.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.484+0000 7f88d5ffb700 1 -- 172.21.15.27:0/637222807 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f88dc00f040 con 0x7f88e0060310 2021-11-03T18:31:29.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.484+0000 7f88e63d4700 1 -- 172.21.15.27:0/637222807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88e010a4b0 con 0x7f88e0060310 2021-11-03T18:31:29.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.484+0000 7f88d5ffb700 1 -- 172.21.15.27:0/637222807 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88dc0147e0 con 0x7f88e0060310 2021-11-03T18:31:29.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.485+0000 7f88e63d4700 1 -- 172.21.15.27:0/637222807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88e010aa70 con 0x7f88e0060310 2021-11-03T18:31:29.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.486+0000 7f88e63d4700 1 -- 172.21.15.27:0/637222807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88e0045ad0 con 0x7f88e0060310 2021-11-03T18:31:29.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.486+0000 7f88d5ffb700 1 -- 172.21.15.27:0/637222807 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f88dc014aa0 con 0x7f88e0060310 2021-11-03T18:31:29.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.486+0000 7f88d5ffb700 1 --2- 172.21.15.27:0/637222807 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f88cc064100 0x7f88cc0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:31:29.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.487+0000 7f88d5ffb700 1 -- 172.21.15.27:0/637222807 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f88dc080ec0 con 0x7f88e0060310 2021-11-03T18:31:29.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.487+0000 7f88e4bd1700 1 --2- 172.21.15.27:0/637222807 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f88cc064100 0x7f88cc0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:31:29.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.487+0000 7f88e4bd1700 1 --2- 172.21.15.27:0/637222807 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f88cc064100 0x7f88cc0665c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7f88d0013690 tx=0x7f88d0009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:31:29.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.491+0000 7f88d5ffb700 1 -- 172.21.15.27:0/637222807 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f88dc053370 con 0x7f88e0060310 2021-11-03T18:31:29.670 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.671+0000 7f88e63d4700 1 -- 172.21.15.27:0/637222807 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f88e01010c0 con 0x7f88cc064100 2021-11-03T18:31:29.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.672+0000 7f88d5ffb700 1 -- 172.21.15.27:0/637222807 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f88d0019040 con 0x7f88cc064100 2021-11-03T18:31:29.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.674+0000 7f88e63d4700 1 -- 172.21.15.27:0/637222807 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f88cc064100 msgr2=0x7f88cc0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:31:29.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.674+0000 7f88e63d4700 1 --2- 172.21.15.27:0/637222807 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f88cc064100 0x7f88cc0665c0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7f88d0013690 tx=0x7f88d0009b30).stop 2021-11-03T18:31:29.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.674+0000 7f88e63d4700 1 -- 172.21.15.27:0/637222807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88e0060310 msgr2=0x7f88e0108f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:31:29.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.674+0000 7f88e63d4700 1 --2- 172.21.15.27:0/637222807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88e0060310 0x7f88e0108f30 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 rx=0x7f88dc0026d0 tx=0x7f88dc009de0).stop 2021-11-03T18:31:29.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.674+0000 7f88e63d4700 1 -- 172.21.15.27:0/637222807 shutdown_connections 2021-11-03T18:31:29.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.674+0000 7f88e63d4700 1 --2- 172.21.15.27:0/637222807 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f88cc064100 0x7f88cc0665c0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:31:29.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.674+0000 7f88e63d4700 1 --2- 172.21.15.27:0/637222807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88e0060310 0x7f88e0108f30 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:31:29.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.674+0000 7f88e63d4700 1 --2- 172.21.15.27:0/637222807 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88e00fba10 0x7f88e0109420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:31:29.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.674+0000 7f88e63d4700 1 -- 172.21.15.27:0/637222807 >> 172.21.15.27:0/637222807 conn(0x7f88e00f0ba0 msgr2=0x7f88e00f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:31:29.676 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.674+0000 7f88e63d4700 1 -- 172.21.15.27:0/637222807 shutdown_connections 2021-11-03T18:31:29.676 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.674+0000 7f88e63d4700 1 -- 172.21.15.27:0/637222807 wait complete. 2021-11-03T18:31:29.683 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:31:29.773 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.774+0000 7fd654800700 1 -- 172.21.15.27:0/3670625962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6500f77d0 msgr2=0x7fd6500f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:31:29.774 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.774+0000 7fd654800700 1 --2- 172.21.15.27:0/3670625962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6500f77d0 0x7fd6500f7c30 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 rx=0x7fd640004660 tx=0x7fd640009b30).stop 2021-11-03T18:31:29.774 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.774+0000 7fd654800700 1 -- 172.21.15.27:0/3670625962 shutdown_connections 2021-11-03T18:31:29.774 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.774+0000 7fd654800700 1 --2- 172.21.15.27:0/3670625962 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6500f77d0 0x7fd6500f7c30 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:31:29.774 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.774+0000 7fd654800700 1 --2- 172.21.15.27:0/3670625962 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6500f6530 0x7fd6500f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:31:29.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.774+0000 7fd654800700 1 -- 172.21.15.27:0/3670625962 >> 172.21.15.27:0/3670625962 conn(0x7fd6500f1cd0 msgr2=0x7fd6500f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:31:29.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.774+0000 7fd654800700 1 -- 172.21.15.27:0/3670625962 shutdown_connections 2021-11-03T18:31:29.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.774+0000 7fd654800700 1 -- 172.21.15.27:0/3670625962 wait complete. 2021-11-03T18:31:29.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.775+0000 7fd654800700 1 Processor -- start 2021-11-03T18:31:29.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.775+0000 7fd654800700 1 -- start start 2021-11-03T18:31:29.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.775+0000 7fd654800700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6500f6530 0x7fd650106d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:31:29.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.775+0000 7fd654800700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6500f77d0 0x7fd650107270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:31:29.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.775+0000 7fd654800700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd650107870 con 0x7fd6500f6530 2021-11-03T18:31:29.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.775+0000 7fd654800700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd650108080 con 0x7fd6500f77d0 2021-11-03T18:31:29.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.775+0000 7fd64ed9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6500f6530 0x7fd650106d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:31:29.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.775+0000 7fd64ed9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6500f6530 0x7fd650106d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35352/0 (socket says 172.21.15.27:35352) 2021-11-03T18:31:29.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.775+0000 7fd64ed9d700 1 -- 172.21.15.27:0/779118345 learned_addr learned my addr 172.21.15.27:0/779118345 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:31:29.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.775+0000 7fd64e59c700 1 --2- 172.21.15.27:0/779118345 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6500f77d0 0x7fd650107270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:31:29.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.775+0000 7fd64ed9d700 1 -- 172.21.15.27:0/779118345 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6500f77d0 msgr2=0x7fd650107270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:31:29.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.775+0000 7fd64ed9d700 1 --2- 172.21.15.27:0/779118345 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6500f77d0 0x7fd650107270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:31:29.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.775+0000 7fd64ed9d700 1 -- 172.21.15.27:0/779118345 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd640005040 con 0x7fd6500f6530 2021-11-03T18:31:29.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.776+0000 7fd64ed9d700 1 --2- 172.21.15.27:0/779118345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6500f6530 0x7fd650106d80 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 rx=0x7fd638002700 tx=0x7fd638009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:31:29.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.776+0000 7fd6477fe700 1 -- 172.21.15.27:0/779118345 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6380079c0 con 0x7fd6500f6530 2021-11-03T18:31:29.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.776+0000 7fd6477fe700 1 -- 172.21.15.27:0/779118345 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fd63800f040 con 0x7fd6500f6530 2021-11-03T18:31:29.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.776+0000 7fd6477fe700 1 -- 172.21.15.27:0/779118345 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd638014860 con 0x7fd6500f6530 2021-11-03T18:31:29.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.776+0000 7fd654800700 1 -- 172.21.15.27:0/779118345 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd650108300 con 0x7fd6500f6530 2021-11-03T18:31:29.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.776+0000 7fd654800700 1 -- 172.21.15.27:0/779118345 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6501088c0 con 0x7fd6500f6530 2021-11-03T18:31:29.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.777+0000 7fd654800700 1 -- 172.21.15.27:0/779118345 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd650059100 con 0x7fd6500f6530 2021-11-03T18:31:29.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.777+0000 7fd6477fe700 1 -- 172.21.15.27:0/779118345 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fd63801b060 con 0x7fd6500f6530 2021-11-03T18:31:29.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.778+0000 7fd6477fe700 1 --2- 172.21.15.27:0/779118345 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd63c064100 0x7fd63c0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:31:29.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.778+0000 7fd6477fe700 1 -- 172.21.15.27:0/779118345 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fd638080730 con 0x7fd6500f6530 2021-11-03T18:31:29.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.778+0000 7fd64e59c700 1 --2- 172.21.15.27:0/779118345 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd63c064100 0x7fd63c0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:31:29.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.778+0000 7fd64e59c700 1 --2- 172.21.15.27:0/779118345 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd63c064100 0x7fd63c0665c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7fd640004630 tx=0x7fd640009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:31:29.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.782+0000 7fd6477fe700 1 -- 172.21.15.27:0/779118345 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fd6380529d0 con 0x7fd6500f6530 2021-11-03T18:31:29.955 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.956+0000 7fd654800700 1 -- 172.21.15.27:0/779118345 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd6500fedd0 con 0x7fd63c064100 2021-11-03T18:31:29.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.970+0000 7fd6477fe700 1 -- 172.21.15.27:0/779118345 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fd6500fedd0 con 0x7fd63c064100 2021-11-03T18:31:29.970 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:31:29.970 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (83m) 108s ago 84m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:31:29.970 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (74m) 108s ago 84m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:31:29.970 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (74m) 75s ago 83m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:31:29.971 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (82m) 108s ago 83m 42.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:31:29.971 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (76m) 108s ago 79m 26.7M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:31:29.971 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (79m) 108s ago 79m 3850M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:31:29.971 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (75m) 75s ago 79m 26.3M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:31:29.972 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (76m) 75s ago 79m 27.4M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:31:29.972 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (76m) 108s ago 85m 608M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:31:29.972 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (76m) 75s ago 83m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:31:29.972 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (75m) 108s ago 85m 350M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:31:29.972 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (75m) 75s ago 83m 160M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:31:29.973 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (83m) 108s ago 83m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:31:29.973 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (83m) 75s ago 83m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:31:29.973 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (74m) 108s ago 82m 413M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:31:29.973 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (73m) 108s ago 82m 450M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:31:29.974 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (73m) 108s ago 82m 338M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:31:29.974 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (72m) 75s ago 81m 479M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:31:29.974 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (72m) 75s ago 81m 378M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:31:29.974 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (72m) 75s ago 81m 388M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:31:29.975 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (82m) 108s ago 83m 64.4M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:31:29.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.972+0000 7fd654800700 1 -- 172.21.15.27:0/779118345 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd63c064100 msgr2=0x7fd63c0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:31:29.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.972+0000 7fd654800700 1 --2- 172.21.15.27:0/779118345 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd63c064100 0x7fd63c0665c0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 rx=0x7fd640004630 tx=0x7fd640009b30).stop 2021-11-03T18:31:29.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.972+0000 7fd654800700 1 -- 172.21.15.27:0/779118345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6500f6530 msgr2=0x7fd650106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:31:29.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.972+0000 7fd654800700 1 --2- 172.21.15.27:0/779118345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6500f6530 0x7fd650106d80 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 rx=0x7fd638002700 tx=0x7fd638009d80).stop 2021-11-03T18:31:29.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.972+0000 7fd654800700 1 -- 172.21.15.27:0/779118345 shutdown_connections 2021-11-03T18:31:29.977 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.972+0000 7fd654800700 1 --2- 172.21.15.27:0/779118345 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd63c064100 0x7fd63c0665c0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:31:29.977 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.972+0000 7fd654800700 1 --2- 172.21.15.27:0/779118345 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd6500f6530 0x7fd650106d80 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:31:29.977 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.972+0000 7fd654800700 1 --2- 172.21.15.27:0/779118345 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd6500f77d0 0x7fd650107270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:31:29.977 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.972+0000 7fd654800700 1 -- 172.21.15.27:0/779118345 >> 172.21.15.27:0/779118345 conn(0x7fd6500f1cd0 msgr2=0x7fd6500fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:31:29.978 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.972+0000 7fd654800700 1 -- 172.21.15.27:0/779118345 shutdown_connections 2021-11-03T18:31:29.978 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:29.972+0000 7fd654800700 1 -- 172.21.15.27:0/779118345 wait complete. 2021-11-03T18:31:30.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.071+0000 7f51cdd0e700 1 -- 172.21.15.27:0/593091601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51c80f77d0 msgr2=0x7f51c80f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:31:30.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.071+0000 7f51cdd0e700 1 --2- 172.21.15.27:0/593091601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51c80f77d0 0x7f51c80f7c30 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 rx=0x7f51b80099e0 tx=0x7f51b8004d10).stop 2021-11-03T18:31:30.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.071+0000 7f51cdd0e700 1 -- 172.21.15.27:0/593091601 shutdown_connections 2021-11-03T18:31:30.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.071+0000 7f51cdd0e700 1 --2- 172.21.15.27:0/593091601 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51c80f77d0 0x7f51c80f7c30 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:31:30.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.071+0000 7f51cdd0e700 1 --2- 172.21.15.27:0/593091601 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51c80f6530 0x7f51c80f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:31:30.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.071+0000 7f51cdd0e700 1 -- 172.21.15.27:0/593091601 >> 172.21.15.27:0/593091601 conn(0x7f51c80f1cd0 msgr2=0x7f51c80f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:31:30.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.071+0000 7f51cdd0e700 1 -- 172.21.15.27:0/593091601 shutdown_connections 2021-11-03T18:31:30.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.071+0000 7f51cdd0e700 1 -- 172.21.15.27:0/593091601 wait complete. 2021-11-03T18:31:30.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.072+0000 7f51cdd0e700 1 Processor -- start 2021-11-03T18:31:30.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.072+0000 7f51cdd0e700 1 -- start start 2021-11-03T18:31:30.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.072+0000 7f51cdd0e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51c80f6530 0x7f51c8106e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:31:30.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.072+0000 7f51cdd0e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51c80f77d0 0x7f51c8107350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:31:30.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.072+0000 7f51cdd0e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c8107950 con 0x7f51c80f6530 2021-11-03T18:31:30.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.072+0000 7f51cdd0e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c8108160 con 0x7f51c80f77d0 2021-11-03T18:31:30.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.072+0000 7f51ccd0c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51c80f6530 0x7f51c8106e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:31:30.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.072+0000 7f51ccd0c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51c80f6530 0x7f51c8106e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35362/0 (socket says 172.21.15.27:35362) 2021-11-03T18:31:30.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.072+0000 7f51ccd0c700 1 -- 172.21.15.27:0/2046797909 learned_addr learned my addr 172.21.15.27:0/2046797909 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:31:30.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.073+0000 7f51c7fff700 1 --2- 172.21.15.27:0/2046797909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51c80f77d0 0x7f51c8107350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:31:30.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.073+0000 7f51ccd0c700 1 -- 172.21.15.27:0/2046797909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51c80f77d0 msgr2=0x7f51c8107350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:31:30.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.073+0000 7f51ccd0c700 1 --2- 172.21.15.27:0/2046797909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51c80f77d0 0x7f51c8107350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:31:30.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.073+0000 7f51ccd0c700 1 -- 172.21.15.27:0/2046797909 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b8009640 con 0x7f51c80f6530 2021-11-03T18:31:30.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.073+0000 7f51ccd0c700 1 --2- 172.21.15.27:0/2046797909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51c80f6530 0x7f51c8106e60 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 rx=0x7f51c000c790 tx=0x7f51c0007d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:31:30.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.073+0000 7f51c57fa700 1 -- 172.21.15.27:0/2046797909 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51c000e680 con 0x7f51c80f6530 2021-11-03T18:31:30.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.073+0000 7f51c57fa700 1 -- 172.21.15.27:0/2046797909 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f51c000ec90 con 0x7f51c80f6530 2021-11-03T18:31:30.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.073+0000 7f51cdd0e700 1 -- 172.21.15.27:0/2046797909 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51c81083e0 con 0x7f51c80f6530 2021-11-03T18:31:30.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.073+0000 7f51c57fa700 1 -- 172.21.15.27:0/2046797909 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51c0012d00 con 0x7f51c80f6530 2021-11-03T18:31:30.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.074+0000 7f51cdd0e700 1 -- 172.21.15.27:0/2046797909 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c8108920 con 0x7f51c80f6530 2021-11-03T18:31:30.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.074+0000 7f51c57fa700 1 -- 172.21.15.27:0/2046797909 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f51c0010070 con 0x7f51c80f6530 2021-11-03T18:31:30.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.074+0000 7f51cdd0e700 1 -- 172.21.15.27:0/2046797909 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51c8059100 con 0x7f51c80f6530 2021-11-03T18:31:30.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.075+0000 7f51c57fa700 1 --2- 172.21.15.27:0/2046797909 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f51b4064100 0x7f51b40665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:31:30.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.075+0000 7f51c57fa700 1 -- 172.21.15.27:0/2046797909 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f51c0080560 con 0x7f51c80f6530 2021-11-03T18:31:30.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.075+0000 7f51c7fff700 1 --2- 172.21.15.27:0/2046797909 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f51b4064100 0x7f51b40665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:31:30.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.075+0000 7f51c7fff700 1 --2- 172.21.15.27:0/2046797909 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f51b4064100 0x7f51b40665c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7f51b800b3b0 tx=0x7f51b8004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:31:30.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.079+0000 7f51c57fa700 1 -- 172.21.15.27:0/2046797909 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f51c00527d0 con 0x7f51c80f6530 2021-11-03T18:31:30.309 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.310+0000 7f51cdd0e700 1 -- 172.21.15.27:0/2046797909 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f51c80fecf0 con 0x7f51c80f6530 2021-11-03T18:31:30.310 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.311+0000 7f51c57fa700 1 -- 172.21.15.27:0/2046797909 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f51c00523c0 con 0x7f51c80f6530 2021-11-03T18:31:30.310 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:31:30.311 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:31:30.311 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:31:30.311 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:31:30.311 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:31:30.312 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:31:30.312 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:31:30.312 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:31:30.312 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:31:30.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:31:30.313 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:31:30.313 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:31:30.313 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:31:30.314 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:31:30.314 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:31:30.314 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:31:30.314 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:31:30.315 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:31:30.315 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.312+0000 7f51cdd0e700 1 -- 172.21.15.27:0/2046797909 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f51b4064100 msgr2=0x7f51b40665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:31:30.316 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.312+0000 7f51cdd0e700 1 --2- 172.21.15.27:0/2046797909 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f51b4064100 0x7f51b40665c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7f51b800b3b0 tx=0x7f51b8004d10).stop 2021-11-03T18:31:30.316 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.313+0000 7f51cdd0e700 1 -- 172.21.15.27:0/2046797909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51c80f6530 msgr2=0x7f51c8106e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:31:30.316 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.313+0000 7f51cdd0e700 1 --2- 172.21.15.27:0/2046797909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51c80f6530 0x7f51c8106e60 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 rx=0x7f51c000c790 tx=0x7f51c0007d90).stop 2021-11-03T18:31:30.316 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.313+0000 7f51cdd0e700 1 -- 172.21.15.27:0/2046797909 shutdown_connections 2021-11-03T18:31:30.317 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.313+0000 7f51cdd0e700 1 --2- 172.21.15.27:0/2046797909 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f51b4064100 0x7f51b40665c0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:31:30.317 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.313+0000 7f51cdd0e700 1 --2- 172.21.15.27:0/2046797909 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51c80f6530 0x7f51c8106e60 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:31:30.317 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.313+0000 7f51cdd0e700 1 --2- 172.21.15.27:0/2046797909 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51c80f77d0 0x7f51c8107350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:31:30.317 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.313+0000 7f51cdd0e700 1 -- 172.21.15.27:0/2046797909 >> 172.21.15.27:0/2046797909 conn(0x7f51c80f1cd0 msgr2=0x7f51c80fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:31:30.317 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.313+0000 7f51cdd0e700 1 -- 172.21.15.27:0/2046797909 shutdown_connections 2021-11-03T18:31:30.318 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.313+0000 7f51cdd0e700 1 -- 172.21.15.27:0/2046797909 wait complete. 2021-11-03T18:31:30.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.412+0000 7f4bea349700 1 -- 172.21.15.27:0/1794497252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4be40f5440 msgr2=0x7f4be40f5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:31:30.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.412+0000 7f4bea349700 1 --2- 172.21.15.27:0/1794497252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4be40f5440 0x7f4be40f5860 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 rx=0x7f4bd4004660 tx=0x7f4bd4009b30).stop 2021-11-03T18:31:30.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.412+0000 7f4bea349700 1 -- 172.21.15.27:0/1794497252 shutdown_connections 2021-11-03T18:31:30.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.412+0000 7f4bea349700 1 --2- 172.21.15.27:0/1794497252 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4be40f66e0 0x7f4be40f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:31:30.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.412+0000 7f4bea349700 1 --2- 172.21.15.27:0/1794497252 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4be40f5440 0x7f4be40f5860 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:31:30.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.412+0000 7f4bea349700 1 -- 172.21.15.27:0/1794497252 >> 172.21.15.27:0/1794497252 conn(0x7f4be40f0ba0 msgr2=0x7f4be40f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:31:30.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.412+0000 7f4bea349700 1 -- 172.21.15.27:0/1794497252 shutdown_connections 2021-11-03T18:31:30.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.412+0000 7f4bea349700 1 -- 172.21.15.27:0/1794497252 wait complete. 2021-11-03T18:31:30.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.413+0000 7f4bea349700 1 Processor -- start 2021-11-03T18:31:30.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.413+0000 7f4bea349700 1 -- start start 2021-11-03T18:31:30.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.413+0000 7f4bea349700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4be40f5440 0x7f4be4106d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:31:30.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.413+0000 7f4bea349700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4be40f66e0 0x7f4be4107270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:31:30.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.413+0000 7f4bea349700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be4107870 con 0x7f4be40f66e0 2021-11-03T18:31:30.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.413+0000 7f4bea349700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be41080b0 con 0x7f4be40f5440 2021-11-03T18:31:30.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.413+0000 7f4be8b46700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4be40f66e0 0x7f4be4107270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:31:30.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.413+0000 7f4be8b46700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4be40f66e0 0x7f4be4107270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35374/0 (socket says 172.21.15.27:35374) 2021-11-03T18:31:30.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.413+0000 7f4be8b46700 1 -- 172.21.15.27:0/2021817002 learned_addr learned my addr 172.21.15.27:0/2021817002 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:31:30.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.413+0000 7f4be9347700 1 --2- 172.21.15.27:0/2021817002 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4be40f5440 0x7f4be4106d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:31:30.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.414+0000 7f4be8b46700 1 -- 172.21.15.27:0/2021817002 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4be40f5440 msgr2=0x7f4be4106d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:31:30.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.414+0000 7f4be8b46700 1 --2- 172.21.15.27:0/2021817002 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4be40f5440 0x7f4be4106d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:31:30.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.414+0000 7f4be8b46700 1 -- 172.21.15.27:0/2021817002 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bd4005040 con 0x7f4be40f66e0 2021-11-03T18:31:30.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.414+0000 7f4be8b46700 1 --2- 172.21.15.27:0/2021817002 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4be40f66e0 0x7f4be4107270 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 rx=0x7f4be0002700 tx=0x7f4be0009e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:31:30.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.414+0000 7f4bd9ffb700 1 -- 172.21.15.27:0/2021817002 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4be0009320 con 0x7f4be40f66e0 2021-11-03T18:31:30.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.414+0000 7f4bd9ffb700 1 -- 172.21.15.27:0/2021817002 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f4be000f040 con 0x7f4be40f66e0 2021-11-03T18:31:30.419 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.414+0000 7f4bea349700 1 -- 172.21.15.27:0/2021817002 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be4108390 con 0x7f4be40f66e0 2021-11-03T18:31:30.419 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.414+0000 7f4bd9ffb700 1 -- 172.21.15.27:0/2021817002 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4be0014870 con 0x7f4be40f66e0 2021-11-03T18:31:30.419 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.414+0000 7f4bea349700 1 -- 172.21.15.27:0/2021817002 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be41088d0 con 0x7f4be40f66e0 2021-11-03T18:31:30.420 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.415+0000 7f4bea349700 1 -- 172.21.15.27:0/2021817002 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be4059100 con 0x7f4be40f66e0 2021-11-03T18:31:30.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.416+0000 7f4bd9ffb700 1 -- 172.21.15.27:0/2021817002 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f4be0009480 con 0x7f4be40f66e0 2021-11-03T18:31:30.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.416+0000 7f4bd9ffb700 1 --2- 172.21.15.27:0/2021817002 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4bd0064100 0x7f4bd00665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:31:30.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.416+0000 7f4bd9ffb700 1 -- 172.21.15.27:0/2021817002 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f4be00809b0 con 0x7f4be40f66e0 2021-11-03T18:31:30.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.416+0000 7f4be9347700 1 --2- 172.21.15.27:0/2021817002 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4bd0064100 0x7f4bd00665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:31:30.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.416+0000 7f4be9347700 1 --2- 172.21.15.27:0/2021817002 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4bd0064100 0x7f4bd00665c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7f4bd4013690 tx=0x7f4bd4009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:31:30.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.420+0000 7f4bd9ffb700 1 -- 172.21.15.27:0/2021817002 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f4be0052d10 con 0x7f4be40f66e0 2021-11-03T18:31:30.621 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.622+0000 7f4bea349700 1 -- 172.21.15.27:0/2021817002 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4be410d3e0 con 0x7f4be40f66e0 2021-11-03T18:31:30.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.622+0000 7f4bd9ffb700 1 -- 172.21.15.27:0/2021817002 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f4be0052900 con 0x7f4be40f66e0 2021-11-03T18:31:30.622 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:31:30.622 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:31:30.623 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:31:30.623 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:31:30.623 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:31:30.623 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:31:30.623 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:31:30.624 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:31:30.624 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:31:30.624 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:31:30.624 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:31:30.625 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:31:30.625 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:31:30.625 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:31:30.625 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:31:30.625 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:31:30.626 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:31:30.626 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:31:30.626 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:31:30.626 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:31:30.627 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:31:30.627 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:31:30.627 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:31:30.627 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:31:30.627 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:31:30.628 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:31:30.628 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:31:30.628 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:31:30.628 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:31:30.628 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:31:30.629 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:31:30.629 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:31:30.629 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:31:30.629 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:31:30.630 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:31:30.630 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:31:30.630 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:31:30.630 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:31:30.631 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:31:30.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.624+0000 7f4bea349700 1 -- 172.21.15.27:0/2021817002 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4bd0064100 msgr2=0x7f4bd00665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:31:30.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.624+0000 7f4bea349700 1 --2- 172.21.15.27:0/2021817002 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4bd0064100 0x7f4bd00665c0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7f4bd4013690 tx=0x7f4bd4009b30).stop 2021-11-03T18:31:30.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.624+0000 7f4bea349700 1 -- 172.21.15.27:0/2021817002 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4be40f66e0 msgr2=0x7f4be4107270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:31:30.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.624+0000 7f4bea349700 1 --2- 172.21.15.27:0/2021817002 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4be40f66e0 0x7f4be4107270 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 rx=0x7f4be0002700 tx=0x7f4be0009e00).stop 2021-11-03T18:31:30.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.624+0000 7f4bea349700 1 -- 172.21.15.27:0/2021817002 shutdown_connections 2021-11-03T18:31:30.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.624+0000 7f4bea349700 1 --2- 172.21.15.27:0/2021817002 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4bd0064100 0x7f4bd00665c0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:31:30.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.624+0000 7f4bea349700 1 --2- 172.21.15.27:0/2021817002 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4be40f5440 0x7f4be4106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:31:30.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.624+0000 7f4bea349700 1 --2- 172.21.15.27:0/2021817002 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4be40f66e0 0x7f4be4107270 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:31:30.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.624+0000 7f4bea349700 1 -- 172.21.15.27:0/2021817002 >> 172.21.15.27:0/2021817002 conn(0x7f4be40f0ba0 msgr2=0x7f4be405f620 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:31:30.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.624+0000 7f4bea349700 1 -- 172.21.15.27:0/2021817002 shutdown_connections 2021-11-03T18:31:30.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:31:30.624+0000 7f4bea349700 1 -- 172.21.15.27:0/2021817002 wait complete. 2021-11-03T18:31:30.634 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:31:30.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:30 smithi115 ceph-mon[65453]: pgmap v2307: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:30.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4532.468438 secs 2021-11-03T18:31:30.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:30 smithi115 ceph-mon[65453]: from='client.36472 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:31:30.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:30 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2046797909' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:30.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:30 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2021817002' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:31:31.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:30 smithi027 ceph-mon[97244]: pgmap v2307: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:31.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4532.468438 secs 2021-11-03T18:31:31.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:30 smithi027 ceph-mon[97244]: from='client.36472 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:31:31.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:30 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2046797909' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:31.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:30 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2021817002' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:31:31.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:31 smithi115 ceph-mon[65453]: from='client.36476 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:31:32.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:31 smithi027 ceph-mon[97244]: from='client.36476 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:31:32.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:32 smithi027 ceph-mon[97244]: pgmap v2308: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:32.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:32 smithi115 ceph-mon[65453]: pgmap v2308: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:34.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:34 smithi115 ceph-mon[65453]: pgmap v2309: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:35.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:34 smithi027 ceph-mon[97244]: pgmap v2309: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:35.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4537.468551 secs 2021-11-03T18:31:35.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:31:35.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:31:35.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:31:35.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:35.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:35.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:35.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:35.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4537.468551 secs 2021-11-03T18:31:35.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:31:35.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:31:35.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:31:35.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:35.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:35.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:35.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:36.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:36 smithi115 ceph-mon[65453]: pgmap v2310: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:36.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:36 smithi115 ceph-mon[65453]: poll took 60176.245 ms: 1 events 2021-11-03T18:31:36.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:36 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:31:37.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:36 smithi027 ceph-mon[97244]: pgmap v2310: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:37.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:36 smithi027 ceph-mon[97244]: poll took 60176.245 ms: 1 events 2021-11-03T18:31:37.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:36 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:31:38.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:38 smithi115 ceph-mon[65453]: pgmap v2311: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:39.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:38 smithi027 ceph-mon[97244]: pgmap v2311: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:40.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:40 smithi115 ceph-mon[65453]: pgmap v2312: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:40.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4542.468618 secs 2021-11-03T18:31:41.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:40 smithi027 ceph-mon[97244]: pgmap v2312: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:41.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4542.468618 secs 2021-11-03T18:31:42.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:42 smithi115 ceph-mon[65453]: pgmap v2313: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:43.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:42 smithi027 ceph-mon[97244]: pgmap v2313: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:44.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:44 smithi115 ceph-mon[65453]: pgmap v2314: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:45.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:44 smithi027 ceph-mon[97244]: pgmap v2314: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:45.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4547.468743 secs 2021-11-03T18:31:45.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:31:45.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:31:45.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:31:45.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:45.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:45.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:45.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:46.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4547.468743 secs 2021-11-03T18:31:46.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:31:46.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:31:46.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:31:46.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:46.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:46.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:46.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:46.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:46 smithi115 ceph-mon[65453]: pgmap v2315: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:46.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:46 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:31:47.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:46 smithi027 ceph-mon[97244]: pgmap v2315: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:47.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:46 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:31:48.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:48 smithi115 ceph-mon[65453]: pgmap v2316: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:49.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:48 smithi027 ceph-mon[97244]: pgmap v2316: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:50.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:50 smithi115 ceph-mon[65453]: pgmap v2317: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:50.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4552.468885 secs 2021-11-03T18:31:51.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:50 smithi027 ceph-mon[97244]: pgmap v2317: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:51.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4552.468885 secs 2021-11-03T18:31:52.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:52 smithi115 ceph-mon[65453]: pgmap v2318: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:53.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:52 smithi027 ceph-mon[97244]: pgmap v2318: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:54.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:54 smithi115 ceph-mon[65453]: pgmap v2319: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:55.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:54 smithi027 ceph-mon[97244]: pgmap v2319: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:55.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4557.469016 secs 2021-11-03T18:31:55.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:31:55.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:31:55.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:31:55.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:55.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:55.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:55.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:56.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4557.469016 secs 2021-11-03T18:31:56.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:31:56.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:31:56.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:31:56.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:56.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:56.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:56.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:31:56.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:56 smithi115 ceph-mon[65453]: pgmap v2320: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:56.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:56 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:31:56.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:31:56.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:31:57.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:56 smithi027 ceph-mon[97244]: pgmap v2320: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:57.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:56 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:31:57.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:31:57.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:31:58.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:31:58 smithi115 ceph-mon[65453]: pgmap v2321: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:31:59.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:31:58 smithi027 ceph-mon[97244]: pgmap v2321: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:00.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.744+0000 7f5773fff700 1 -- 172.21.15.27:0/3739135688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57740f51a0 msgr2=0x7f57740f7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:32:00.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.744+0000 7f5773fff700 1 --2- 172.21.15.27:0/3739135688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57740f51a0 0x7f57740f7920 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 rx=0x7f575c004660 tx=0x7f575c009b30).stop 2021-11-03T18:32:00.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.744+0000 7f5773fff700 1 -- 172.21.15.27:0/3739135688 shutdown_connections 2021-11-03T18:32:00.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.744+0000 7f5773fff700 1 --2- 172.21.15.27:0/3739135688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57740f51a0 0x7f57740f7920 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:00.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.744+0000 7f5773fff700 1 --2- 172.21.15.27:0/3739135688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57740f4800 0x7f57740f4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:00.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.744+0000 7f5773fff700 1 -- 172.21.15.27:0/3739135688 >> 172.21.15.27:0/3739135688 conn(0x7f57740f0320 msgr2=0x7f57740f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:32:00.743 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.744+0000 7f5773fff700 1 -- 172.21.15.27:0/3739135688 shutdown_connections 2021-11-03T18:32:00.744 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.744+0000 7f5773fff700 1 -- 172.21.15.27:0/3739135688 wait complete. 2021-11-03T18:32:00.746 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.744+0000 7f5773fff700 1 Processor -- start 2021-11-03T18:32:00.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.745+0000 7f5773fff700 1 -- start start 2021-11-03T18:32:00.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.745+0000 7f5773fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57740f4800 0x7f5774106d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:32:00.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.745+0000 7f5773fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57740f51a0 0x7f5774107270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:32:00.747 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.745+0000 7f5773fff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5774107870 con 0x7f57740f51a0 2021-11-03T18:32:00.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.745+0000 7f5773fff700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5774108080 con 0x7f57740f4800 2021-11-03T18:32:00.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.745+0000 7f57727fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57740f51a0 0x7f5774107270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:32:00.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.745+0000 7f57727fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57740f51a0 0x7f5774107270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35392/0 (socket says 172.21.15.27:35392) 2021-11-03T18:32:00.748 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.745+0000 7f57727fc700 1 -- 172.21.15.27:0/884342046 learned_addr learned my addr 172.21.15.27:0/884342046 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:32:00.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.745+0000 7f5772ffd700 1 --2- 172.21.15.27:0/884342046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57740f4800 0x7f5774106d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:32:00.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.745+0000 7f57727fc700 1 -- 172.21.15.27:0/884342046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57740f4800 msgr2=0x7f5774106d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:32:00.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.745+0000 7f57727fc700 1 --2- 172.21.15.27:0/884342046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57740f4800 0x7f5774106d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:00.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.745+0000 7f57727fc700 1 -- 172.21.15.27:0/884342046 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f575c005040 con 0x7f57740f51a0 2021-11-03T18:32:00.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.745+0000 7f57727fc700 1 --2- 172.21.15.27:0/884342046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57740f51a0 0x7f5774107270 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 rx=0x7f575c014040 tx=0x7f575c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:32:00.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.745+0000 7f576b7fe700 1 -- 172.21.15.27:0/884342046 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f575c019070 con 0x7f57740f51a0 2021-11-03T18:32:00.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.746+0000 7f5773fff700 1 -- 172.21.15.27:0/884342046 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57741082a0 con 0x7f57740f51a0 2021-11-03T18:32:00.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.746+0000 7f576b7fe700 1 -- 172.21.15.27:0/884342046 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f575c007e70 con 0x7f57740f51a0 2021-11-03T18:32:00.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.746+0000 7f576b7fe700 1 -- 172.21.15.27:0/884342046 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f575c014d50 con 0x7f57740f51a0 2021-11-03T18:32:00.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.746+0000 7f5773fff700 1 -- 172.21.15.27:0/884342046 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5774108800 con 0x7f57740f51a0 2021-11-03T18:32:00.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.747+0000 7f5773fff700 1 -- 172.21.15.27:0/884342046 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5774059100 con 0x7f57740f51a0 2021-11-03T18:32:00.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.748+0000 7f576b7fe700 1 -- 172.21.15.27:0/884342046 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f575c025070 con 0x7f57740f51a0 2021-11-03T18:32:00.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.748+0000 7f576b7fe700 1 --2- 172.21.15.27:0/884342046 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5760064100 0x7f57600665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:32:00.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.748+0000 7f576b7fe700 1 -- 172.21.15.27:0/884342046 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f575c081bc0 con 0x7f57740f51a0 2021-11-03T18:32:00.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.748+0000 7f5772ffd700 1 --2- 172.21.15.27:0/884342046 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5760064100 0x7f57600665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:32:00.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.749+0000 7f5772ffd700 1 --2- 172.21.15.27:0/884342046 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5760064100 0x7f57600665c0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7f5764004ec0 tx=0x7f576400b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:32:00.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.752+0000 7f576b7fe700 1 -- 172.21.15.27:0/884342046 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f575c053f20 con 0x7f57740f51a0 2021-11-03T18:32:00.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.930+0000 7f5773fff700 1 -- 172.21.15.27:0/884342046 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f57740fee90 con 0x7f5760064100 2021-11-03T18:32:00.929 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.931+0000 7f576b7fe700 1 -- 172.21.15.27:0/884342046 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f57740fee90 con 0x7f5760064100 2021-11-03T18:32:00.931 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.933+0000 7f5773fff700 1 -- 172.21.15.27:0/884342046 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5760064100 msgr2=0x7f57600665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:32:00.932 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.933+0000 7f5773fff700 1 --2- 172.21.15.27:0/884342046 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5760064100 0x7f57600665c0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 rx=0x7f5764004ec0 tx=0x7f576400b040).stop 2021-11-03T18:32:00.932 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.933+0000 7f5773fff700 1 -- 172.21.15.27:0/884342046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57740f51a0 msgr2=0x7f5774107270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:32:00.932 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.933+0000 7f5773fff700 1 --2- 172.21.15.27:0/884342046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57740f51a0 0x7f5774107270 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 rx=0x7f575c014040 tx=0x7f575c00b040).stop 2021-11-03T18:32:00.932 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.933+0000 7f5773fff700 1 -- 172.21.15.27:0/884342046 shutdown_connections 2021-11-03T18:32:00.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.933+0000 7f5773fff700 1 --2- 172.21.15.27:0/884342046 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5760064100 0x7f57600665c0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:00.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.933+0000 7f5773fff700 1 --2- 172.21.15.27:0/884342046 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57740f4800 0x7f5774106d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:00.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.933+0000 7f5773fff700 1 --2- 172.21.15.27:0/884342046 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57740f51a0 0x7f5774107270 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:00.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.933+0000 7f5773fff700 1 -- 172.21.15.27:0/884342046 >> 172.21.15.27:0/884342046 conn(0x7f57740f0320 msgr2=0x7f57740fb370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:32:00.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.933+0000 7f5773fff700 1 -- 172.21.15.27:0/884342046 shutdown_connections 2021-11-03T18:32:00.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:00.933+0000 7f5773fff700 1 -- 172.21.15.27:0/884342046 wait complete. 2021-11-03T18:32:00.941 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:32:00.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:00 smithi115 ceph-mon[65453]: pgmap v2322: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:00.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4562.469107 secs 2021-11-03T18:32:01.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:00 smithi027 ceph-mon[97244]: pgmap v2322: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:01.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4562.469107 secs 2021-11-03T18:32:01.032 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.034+0000 7f3d382e8700 1 -- 172.21.15.27:0/1715391491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d300f5340 msgr2=0x7f3d300f7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:32:01.032 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.034+0000 7f3d382e8700 1 --2- 172.21.15.27:0/1715391491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d300f5340 0x7f3d300f7ac0 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 rx=0x7f3d20004660 tx=0x7f3d20009b30).stop 2021-11-03T18:32:01.032 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.034+0000 7f3d382e8700 1 -- 172.21.15.27:0/1715391491 shutdown_connections 2021-11-03T18:32:01.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.034+0000 7f3d382e8700 1 --2- 172.21.15.27:0/1715391491 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d300f5340 0x7f3d300f7ac0 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:01.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.034+0000 7f3d382e8700 1 --2- 172.21.15.27:0/1715391491 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d300f49a0 0x7f3d300f4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:01.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.034+0000 7f3d382e8700 1 -- 172.21.15.27:0/1715391491 >> 172.21.15.27:0/1715391491 conn(0x7f3d300f04c0 msgr2=0x7f3d300f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:32:01.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.034+0000 7f3d382e8700 1 -- 172.21.15.27:0/1715391491 shutdown_connections 2021-11-03T18:32:01.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.034+0000 7f3d382e8700 1 -- 172.21.15.27:0/1715391491 wait complete. 2021-11-03T18:32:01.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.034+0000 7f3d382e8700 1 Processor -- start 2021-11-03T18:32:01.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.035+0000 7f3d382e8700 1 -- start start 2021-11-03T18:32:01.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.035+0000 7f3d382e8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d300f49a0 0x7f3d300f7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:32:01.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.035+0000 7f3d382e8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d300f5340 0x7f3d300f59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:32:01.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.035+0000 7f3d382e8700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d300f5f30 con 0x7f3d300f49a0 2021-11-03T18:32:01.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.035+0000 7f3d382e8700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d300f6070 con 0x7f3d300f5340 2021-11-03T18:32:01.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.035+0000 7f3d36084700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d300f49a0 0x7f3d300f7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:32:01.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.035+0000 7f3d36084700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d300f49a0 0x7f3d300f7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35400/0 (socket says 172.21.15.27:35400) 2021-11-03T18:32:01.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.035+0000 7f3d36084700 1 -- 172.21.15.27:0/1589374383 learned_addr learned my addr 172.21.15.27:0/1589374383 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:32:01.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.035+0000 7f3d35883700 1 --2- 172.21.15.27:0/1589374383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d300f5340 0x7f3d300f59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:32:01.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.035+0000 7f3d36084700 1 -- 172.21.15.27:0/1589374383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d300f5340 msgr2=0x7f3d300f59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:32:01.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.035+0000 7f3d36084700 1 --2- 172.21.15.27:0/1589374383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d300f5340 0x7f3d300f59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:01.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.035+0000 7f3d36084700 1 -- 172.21.15.27:0/1589374383 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d20005040 con 0x7f3d300f49a0 2021-11-03T18:32:01.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.035+0000 7f3d36084700 1 --2- 172.21.15.27:0/1589374383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d300f49a0 0x7f3d300f7550 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 rx=0x7f3d2c00e7c0 tx=0x7f3d2c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:32:01.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.035+0000 7f3d26ffd700 1 -- 172.21.15.27:0/1589374383 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d2c0092e0 con 0x7f3d300f49a0 2021-11-03T18:32:01.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.035+0000 7f3d26ffd700 1 -- 172.21.15.27:0/1589374383 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f3d2c00f040 con 0x7f3d300f49a0 2021-11-03T18:32:01.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.036+0000 7f3d382e8700 1 -- 172.21.15.27:0/1589374383 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d300f6350 con 0x7f3d300f49a0 2021-11-03T18:32:01.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.036+0000 7f3d26ffd700 1 -- 172.21.15.27:0/1589374383 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d2c0147e0 con 0x7f3d300f49a0 2021-11-03T18:32:01.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.036+0000 7f3d382e8700 1 -- 172.21.15.27:0/1589374383 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d3010ef60 con 0x7f3d300f49a0 2021-11-03T18:32:01.042 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.036+0000 7f3d382e8700 1 -- 172.21.15.27:0/1589374383 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d30045ad0 con 0x7f3d300f49a0 2021-11-03T18:32:01.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.037+0000 7f3d26ffd700 1 -- 172.21.15.27:0/1589374383 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f3d2c01b030 con 0x7f3d300f49a0 2021-11-03T18:32:01.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.037+0000 7f3d26ffd700 1 --2- 172.21.15.27:0/1589374383 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3d1c0640b0 0x7f3d1c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:32:01.043 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.037+0000 7f3d26ffd700 1 -- 172.21.15.27:0/1589374383 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f3d2c080720 con 0x7f3d300f49a0 2021-11-03T18:32:01.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.037+0000 7f3d35883700 1 --2- 172.21.15.27:0/1589374383 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3d1c0640b0 0x7f3d1c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:32:01.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.038+0000 7f3d35883700 1 --2- 172.21.15.27:0/1589374383 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3d1c0640b0 0x7f3d1c066570 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7f3d20004d80 tx=0x7f3d20016040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:32:01.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.041+0000 7f3d26ffd700 1 -- 172.21.15.27:0/1589374383 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f3d2c052990 con 0x7f3d300f49a0 2021-11-03T18:32:01.218 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.220+0000 7f3d382e8700 1 -- 172.21.15.27:0/1589374383 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d300ff000 con 0x7f3d1c0640b0 2021-11-03T18:32:01.232 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.234+0000 7f3d26ffd700 1 -- 172.21.15.27:0/1589374383 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f3d300ff000 con 0x7f3d1c0640b0 2021-11-03T18:32:01.233 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:32:01.233 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (83m) 2m ago 84m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:32:01.234 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (75m) 2m ago 84m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:32:01.234 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (75m) 106s ago 84m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:32:01.234 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (83m) 2m ago 84m 42.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:32:01.234 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (77m) 2m ago 79m 26.7M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:32:01.234 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (80m) 2m ago 80m 3850M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:32:01.235 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (75m) 106s ago 79m 26.3M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:32:01.235 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (77m) 106s ago 80m 27.4M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:32:01.235 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (77m) 2m ago 86m 608M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:32:01.235 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (76m) 106s ago 84m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:32:01.236 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (76m) 2m ago 86m 350M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:32:01.236 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (75m) 106s ago 83m 160M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:32:01.236 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (84m) 2m ago 84m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:32:01.236 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (83m) 106s ago 83m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:32:01.236 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (74m) 2m ago 83m 413M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:32:01.237 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (74m) 2m ago 82m 450M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:32:01.237 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (73m) 2m ago 82m 338M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:32:01.237 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (73m) 106s ago 82m 479M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:32:01.237 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (72m) 106s ago 81m 378M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:32:01.237 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (72m) 106s ago 81m 388M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:32:01.238 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (83m) 2m ago 84m 64.4M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:32:01.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.236+0000 7f3d382e8700 1 -- 172.21.15.27:0/1589374383 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3d1c0640b0 msgr2=0x7f3d1c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:32:01.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.236+0000 7f3d382e8700 1 --2- 172.21.15.27:0/1589374383 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3d1c0640b0 0x7f3d1c066570 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7f3d20004d80 tx=0x7f3d20016040).stop 2021-11-03T18:32:01.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.236+0000 7f3d382e8700 1 -- 172.21.15.27:0/1589374383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d300f49a0 msgr2=0x7f3d300f7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:32:01.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.236+0000 7f3d382e8700 1 --2- 172.21.15.27:0/1589374383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d300f49a0 0x7f3d300f7550 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 rx=0x7f3d2c00e7c0 tx=0x7f3d2c00b040).stop 2021-11-03T18:32:01.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.236+0000 7f3d382e8700 1 -- 172.21.15.27:0/1589374383 shutdown_connections 2021-11-03T18:32:01.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.236+0000 7f3d382e8700 1 --2- 172.21.15.27:0/1589374383 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3d1c0640b0 0x7f3d1c066570 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:01.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.236+0000 7f3d382e8700 1 --2- 172.21.15.27:0/1589374383 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3d300f49a0 0x7f3d300f7550 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:01.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.236+0000 7f3d382e8700 1 --2- 172.21.15.27:0/1589374383 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3d300f5340 0x7f3d300f59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:01.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.236+0000 7f3d382e8700 1 -- 172.21.15.27:0/1589374383 >> 172.21.15.27:0/1589374383 conn(0x7f3d300f04c0 msgr2=0x7f3d300fb510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:32:01.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.236+0000 7f3d382e8700 1 -- 172.21.15.27:0/1589374383 shutdown_connections 2021-11-03T18:32:01.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.236+0000 7f3d382e8700 1 -- 172.21.15.27:0/1589374383 wait complete. 2021-11-03T18:32:01.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.335+0000 7f59b9619700 1 -- 172.21.15.27:0/3929840742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59b40f7980 msgr2=0x7f59b40f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:32:01.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.335+0000 7f59b9619700 1 --2- 172.21.15.27:0/3929840742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59b40f7980 0x7f59b40f7de0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 rx=0x7f599c004660 tx=0x7f599c009b30).stop 2021-11-03T18:32:01.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.335+0000 7f59b9619700 1 -- 172.21.15.27:0/3929840742 shutdown_connections 2021-11-03T18:32:01.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.335+0000 7f59b9619700 1 --2- 172.21.15.27:0/3929840742 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59b40f7980 0x7f59b40f7de0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:01.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.335+0000 7f59b9619700 1 --2- 172.21.15.27:0/3929840742 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59b40f66e0 0x7f59b40f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:01.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.335+0000 7f59b9619700 1 -- 172.21.15.27:0/3929840742 >> 172.21.15.27:0/3929840742 conn(0x7f59b40f1e40 msgr2=0x7f59b40f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:32:01.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.335+0000 7f59b9619700 1 -- 172.21.15.27:0/3929840742 shutdown_connections 2021-11-03T18:32:01.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.335+0000 7f59b9619700 1 -- 172.21.15.27:0/3929840742 wait complete. 2021-11-03T18:32:01.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.336+0000 7f59b9619700 1 Processor -- start 2021-11-03T18:32:01.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.336+0000 7f59b9619700 1 -- start start 2021-11-03T18:32:01.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.336+0000 7f59b9619700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59b40f66e0 0x7f59b4106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:32:01.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.336+0000 7f59b9619700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59b40f7980 0x7f59b4107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:32:01.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.336+0000 7f59b9619700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b4107a00 con 0x7f59b40f7980 2021-11-03T18:32:01.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.336+0000 7f59b9619700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b4107b40 con 0x7f59b40f66e0 2021-11-03T18:32:01.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.336+0000 7f59b27fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59b40f7980 0x7f59b4107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:32:01.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.336+0000 7f59b27fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59b40f7980 0x7f59b4107450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35412/0 (socket says 172.21.15.27:35412) 2021-11-03T18:32:01.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.336+0000 7f59b27fc700 1 -- 172.21.15.27:0/2076218506 learned_addr learned my addr 172.21.15.27:0/2076218506 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:32:01.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.336+0000 7f59b2ffd700 1 --2- 172.21.15.27:0/2076218506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59b40f66e0 0x7f59b4106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:32:01.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.337+0000 7f59b27fc700 1 -- 172.21.15.27:0/2076218506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59b40f66e0 msgr2=0x7f59b4106f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:32:01.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.337+0000 7f59b27fc700 1 --2- 172.21.15.27:0/2076218506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59b40f66e0 0x7f59b4106f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:01.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.337+0000 7f59b27fc700 1 -- 172.21.15.27:0/2076218506 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f599c005040 con 0x7f59b40f7980 2021-11-03T18:32:01.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.337+0000 7f59b27fc700 1 --2- 172.21.15.27:0/2076218506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59b40f7980 0x7f59b4107450 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 rx=0x7f599c009990 tx=0x7f599c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:32:01.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.337+0000 7f59ab7fe700 1 -- 172.21.15.27:0/2076218506 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f599c018070 con 0x7f59b40f7980 2021-11-03T18:32:01.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.337+0000 7f59b9619700 1 -- 172.21.15.27:0/2076218506 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59b4108480 con 0x7f59b40f7980 2021-11-03T18:32:01.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.337+0000 7f59ab7fe700 1 -- 172.21.15.27:0/2076218506 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f599c007ba0 con 0x7f59b40f7980 2021-11-03T18:32:01.342 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.337+0000 7f59ab7fe700 1 -- 172.21.15.27:0/2076218506 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f599c01c3c0 con 0x7f59b40f7980 2021-11-03T18:32:01.342 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.337+0000 7f59b9619700 1 -- 172.21.15.27:0/2076218506 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59b41089e0 con 0x7f59b40f7980 2021-11-03T18:32:01.342 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.338+0000 7f59b9619700 1 -- 172.21.15.27:0/2076218506 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59b4045ad0 con 0x7f59b40f7980 2021-11-03T18:32:01.343 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.338+0000 7f59ab7fe700 1 -- 172.21.15.27:0/2076218506 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f599c01d420 con 0x7f59b40f7980 2021-11-03T18:32:01.344 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.339+0000 7f59ab7fe700 1 --2- 172.21.15.27:0/2076218506 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f59a00640b0 0x7f59a0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:32:01.344 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.339+0000 7f59ab7fe700 1 -- 172.21.15.27:0/2076218506 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f599c082410 con 0x7f59b40f7980 2021-11-03T18:32:01.344 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.339+0000 7f59b2ffd700 1 --2- 172.21.15.27:0/2076218506 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f59a00640b0 0x7f59a0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:32:01.344 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.339+0000 7f59b2ffd700 1 --2- 172.21.15.27:0/2076218506 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f59a00640b0 0x7f59a0066570 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7f59b40f77e0 tx=0x7f59a4009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:32:01.345 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.343+0000 7f59ab7fe700 1 -- 172.21.15.27:0/2076218506 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f599c054830 con 0x7f59b40f7980 2021-11-03T18:32:01.570 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.572+0000 7f59b9619700 1 -- 172.21.15.27:0/2076218506 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f59b4027a00 con 0x7f59b40f7980 2021-11-03T18:32:01.571 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.572+0000 7f59ab7fe700 1 -- 172.21.15.27:0/2076218506 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f599c054420 con 0x7f59b40f7980 2021-11-03T18:32:01.571 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:32:01.572 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:32:01.572 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:32:01.572 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:32:01.572 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:32:01.572 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:32:01.573 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:32:01.573 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:32:01.573 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:32:01.573 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:32:01.574 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:32:01.574 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:32:01.574 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:32:01.574 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:32:01.575 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:32:01.575 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:32:01.575 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:32:01.575 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:32:01.576 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.574+0000 7f59b9619700 1 -- 172.21.15.27:0/2076218506 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f59a00640b0 msgr2=0x7f59a0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:32:01.576 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.574+0000 7f59b9619700 1 --2- 172.21.15.27:0/2076218506 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f59a00640b0 0x7f59a0066570 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7f59b40f77e0 tx=0x7f59a4009040).stop 2021-11-03T18:32:01.577 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.574+0000 7f59b9619700 1 -- 172.21.15.27:0/2076218506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59b40f7980 msgr2=0x7f59b4107450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:32:01.577 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.574+0000 7f59b9619700 1 --2- 172.21.15.27:0/2076218506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59b40f7980 0x7f59b4107450 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 rx=0x7f599c009990 tx=0x7f599c009b30).stop 2021-11-03T18:32:01.577 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.574+0000 7f59b9619700 1 -- 172.21.15.27:0/2076218506 shutdown_connections 2021-11-03T18:32:01.577 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.574+0000 7f59b9619700 1 --2- 172.21.15.27:0/2076218506 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f59a00640b0 0x7f59a0066570 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:01.578 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.574+0000 7f59b9619700 1 --2- 172.21.15.27:0/2076218506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59b40f66e0 0x7f59b4106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:01.578 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.574+0000 7f59b9619700 1 --2- 172.21.15.27:0/2076218506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59b40f7980 0x7f59b4107450 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:01.578 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.574+0000 7f59b9619700 1 -- 172.21.15.27:0/2076218506 >> 172.21.15.27:0/2076218506 conn(0x7f59b40f1e40 msgr2=0x7f59b40fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:32:01.578 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.574+0000 7f59b9619700 1 -- 172.21.15.27:0/2076218506 shutdown_connections 2021-11-03T18:32:01.578 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.574+0000 7f59b9619700 1 -- 172.21.15.27:0/2076218506 wait complete. 2021-11-03T18:32:01.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.674+0000 7f57f1346700 1 -- 172.21.15.27:0/4086696822 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57ec0f66e0 msgr2=0x7f57ec0f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:32:01.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.674+0000 7f57f1346700 1 --2- 172.21.15.27:0/4086696822 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57ec0f66e0 0x7f57ec0f6b40 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 rx=0x7f57e0004660 tx=0x7f57e0009b30).stop 2021-11-03T18:32:01.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.674+0000 7f57f1346700 1 -- 172.21.15.27:0/4086696822 shutdown_connections 2021-11-03T18:32:01.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.674+0000 7f57f1346700 1 --2- 172.21.15.27:0/4086696822 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57ec0f66e0 0x7f57ec0f6b40 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:01.676 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.674+0000 7f57f1346700 1 --2- 172.21.15.27:0/4086696822 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57ec0f5440 0x7f57ec0f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:01.676 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.674+0000 7f57f1346700 1 -- 172.21.15.27:0/4086696822 >> 172.21.15.27:0/4086696822 conn(0x7f57ec0f0ba0 msgr2=0x7f57ec0f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:32:01.676 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.674+0000 7f57f1346700 1 -- 172.21.15.27:0/4086696822 shutdown_connections 2021-11-03T18:32:01.676 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.674+0000 7f57f1346700 1 -- 172.21.15.27:0/4086696822 wait complete. 2021-11-03T18:32:01.677 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.675+0000 7f57f1346700 1 Processor -- start 2021-11-03T18:32:01.677 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.675+0000 7f57f1346700 1 -- start start 2021-11-03T18:32:01.677 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.675+0000 7f57f1346700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57ec0f5440 0x7f57ec1099b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:32:01.677 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.675+0000 7f57f1346700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57ec0f66e0 0x7f57ec10beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:32:01.677 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.675+0000 7f57f1346700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec0603e0 con 0x7f57ec0f5440 2021-11-03T18:32:01.678 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.675+0000 7f57f1346700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57ec060520 con 0x7f57ec0f66e0 2021-11-03T18:32:01.678 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.675+0000 7f57ebfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57ec0f5440 0x7f57ec1099b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:32:01.678 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.675+0000 7f57ebfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57ec0f5440 0x7f57ec1099b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35420/0 (socket says 172.21.15.27:35420) 2021-11-03T18:32:01.678 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.675+0000 7f57ebfff700 1 -- 172.21.15.27:0/858413106 learned_addr learned my addr 172.21.15.27:0/858413106 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:32:01.679 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.676+0000 7f57eb7fe700 1 --2- 172.21.15.27:0/858413106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57ec0f66e0 0x7f57ec10beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:32:01.679 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.676+0000 7f57ebfff700 1 -- 172.21.15.27:0/858413106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57ec0f66e0 msgr2=0x7f57ec10beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:32:01.679 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.676+0000 7f57ebfff700 1 --2- 172.21.15.27:0/858413106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57ec0f66e0 0x7f57ec10beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:01.679 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.676+0000 7f57ebfff700 1 -- 172.21.15.27:0/858413106 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e0005040 con 0x7f57ec0f5440 2021-11-03T18:32:01.680 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.676+0000 7f57ebfff700 1 --2- 172.21.15.27:0/858413106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57ec0f5440 0x7f57ec1099b0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 rx=0x7f57dc00e7c0 tx=0x7f57dc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:32:01.680 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.676+0000 7f57e8ff9700 1 -- 172.21.15.27:0/858413106 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57dc0092e0 con 0x7f57ec0f5440 2021-11-03T18:32:01.680 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.676+0000 7f57e8ff9700 1 -- 172.21.15.27:0/858413106 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f57dc00f040 con 0x7f57ec0f5440 2021-11-03T18:32:01.680 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.676+0000 7f57f1346700 1 -- 172.21.15.27:0/858413106 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57ec10c460 con 0x7f57ec0f5440 2021-11-03T18:32:01.681 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.676+0000 7f57e8ff9700 1 -- 172.21.15.27:0/858413106 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f57dc0147e0 con 0x7f57ec0f5440 2021-11-03T18:32:01.681 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.676+0000 7f57f1346700 1 -- 172.21.15.27:0/858413106 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57ec10ca20 con 0x7f57ec0f5440 2021-11-03T18:32:01.682 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.677+0000 7f57f1346700 1 -- 172.21.15.27:0/858413106 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57ec045ad0 con 0x7f57ec0f5440 2021-11-03T18:32:01.682 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.677+0000 7f57e8ff9700 1 -- 172.21.15.27:0/858413106 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f57dc01b030 con 0x7f57ec0f5440 2021-11-03T18:32:01.682 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.678+0000 7f57e8ff9700 1 --2- 172.21.15.27:0/858413106 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f57d4064100 0x7f57d40665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:32:01.683 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.678+0000 7f57e8ff9700 1 -- 172.21.15.27:0/858413106 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f57dc080770 con 0x7f57ec0f5440 2021-11-03T18:32:01.683 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.678+0000 7f57eb7fe700 1 --2- 172.21.15.27:0/858413106 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f57d4064100 0x7f57d40665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:32:01.683 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.678+0000 7f57eb7fe700 1 --2- 172.21.15.27:0/858413106 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f57d4064100 0x7f57d40665c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f57e0005010 tx=0x7f57e0016040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:32:01.683 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.682+0000 7f57e8ff9700 1 -- 172.21.15.27:0/858413106 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f57dc0529e0 con 0x7f57ec0f5440 2021-11-03T18:32:01.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.889+0000 7f57f1346700 1 -- 172.21.15.27:0/858413106 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f57ec027a00 con 0x7f57ec0f5440 2021-11-03T18:32:01.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.889+0000 7f57e8ff9700 1 -- 172.21.15.27:0/858413106 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f57dc019020 con 0x7f57ec0f5440 2021-11-03T18:32:01.888 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:32:01.888 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:32:01.888 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:32:01.889 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:32:01.889 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:32:01.889 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:32:01.889 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:32:01.890 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:32:01.890 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:32:01.890 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:32:01.891 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:32:01.891 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:32:01.891 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:32:01.892 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:32:01.892 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:32:01.892 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:32:01.893 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:32:01.893 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:32:01.893 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:32:01.893 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:32:01.894 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:32:01.894 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:32:01.894 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:32:01.895 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:32:01.895 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:32:01.895 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:32:01.895 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:32:01.896 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:32:01.896 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:32:01.896 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:32:01.896 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:32:01.897 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:32:01.897 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:32:01.897 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:32:01.897 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:32:01.898 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:32:01.898 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:32:01.898 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:32:01.899 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:32:01.900 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.891+0000 7f57f1346700 1 -- 172.21.15.27:0/858413106 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f57d4064100 msgr2=0x7f57d40665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:32:01.900 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.891+0000 7f57f1346700 1 --2- 172.21.15.27:0/858413106 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f57d4064100 0x7f57d40665c0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 rx=0x7f57e0005010 tx=0x7f57e0016040).stop 2021-11-03T18:32:01.900 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.891+0000 7f57f1346700 1 -- 172.21.15.27:0/858413106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57ec0f5440 msgr2=0x7f57ec1099b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:32:01.901 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.891+0000 7f57f1346700 1 --2- 172.21.15.27:0/858413106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57ec0f5440 0x7f57ec1099b0 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 rx=0x7f57dc00e7c0 tx=0x7f57dc00b040).stop 2021-11-03T18:32:01.901 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.891+0000 7f57f1346700 1 -- 172.21.15.27:0/858413106 shutdown_connections 2021-11-03T18:32:01.901 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.891+0000 7f57f1346700 1 --2- 172.21.15.27:0/858413106 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f57d4064100 0x7f57d40665c0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:01.901 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.891+0000 7f57f1346700 1 --2- 172.21.15.27:0/858413106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f57ec0f5440 0x7f57ec1099b0 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:01.902 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.891+0000 7f57f1346700 1 --2- 172.21.15.27:0/858413106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f57ec0f66e0 0x7f57ec10beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:01.902 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.891+0000 7f57f1346700 1 -- 172.21.15.27:0/858413106 >> 172.21.15.27:0/858413106 conn(0x7f57ec0f0ba0 msgr2=0x7f57ec0fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:32:01.902 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.891+0000 7f57f1346700 1 -- 172.21.15.27:0/858413106 shutdown_connections 2021-11-03T18:32:01.903 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:01.891+0000 7f57f1346700 1 -- 172.21.15.27:0/858413106 wait complete. 2021-11-03T18:32:01.903 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:32:01.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:01 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2076218506' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:02.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:01 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2076218506' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:02.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:02 smithi115 ceph-mon[65453]: from='client.36488 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:32:02.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:02 smithi115 ceph-mon[65453]: pgmap v2323: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:02.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:02 smithi115 ceph-mon[65453]: from='client.36492 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:32:02.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:02 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/858413106' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:32:03.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:02 smithi027 ceph-mon[97244]: from='client.36488 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:32:03.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:02 smithi027 ceph-mon[97244]: pgmap v2323: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:03.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:02 smithi027 ceph-mon[97244]: from='client.36492 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:32:03.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:02 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/858413106' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:32:05.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:04 smithi027 ceph-mon[97244]: pgmap v2324: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:05.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:04 smithi115 ceph-mon[65453]: pgmap v2324: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:06.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4567.469226 secs 2021-11-03T18:32:06.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:32:06.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:32:06.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:32:06.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:06.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:06.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:06.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:06.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4567.469226 secs 2021-11-03T18:32:06.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:32:06.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:32:06.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:32:06.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:06.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:06.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:06.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:07.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:06 smithi027 ceph-mon[97244]: pgmap v2325: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:07.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:06 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:32:07.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:06 smithi115 ceph-mon[65453]: pgmap v2325: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:07.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:06 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:32:09.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:08 smithi027 ceph-mon[97244]: pgmap v2326: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:09.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:08 smithi115 ceph-mon[65453]: pgmap v2326: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:11.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:10 smithi027 ceph-mon[97244]: pgmap v2327: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:11.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4572.469361 secs 2021-11-03T18:32:11.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:10 smithi115 ceph-mon[65453]: pgmap v2327: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:11.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4572.469361 secs 2021-11-03T18:32:13.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:12 smithi027 ceph-mon[97244]: pgmap v2328: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:13.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:12 smithi115 ceph-mon[65453]: pgmap v2328: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:15.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:14 smithi027 ceph-mon[97244]: pgmap v2329: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:15.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:14 smithi115 ceph-mon[65453]: pgmap v2329: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:16.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4577.469482 secs 2021-11-03T18:32:16.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:32:16.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:32:16.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:32:16.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:16.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:16.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:16.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:16.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4577.469482 secs 2021-11-03T18:32:16.315 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:32:16.316 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:32:16.316 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:32:16.316 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:16.316 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:16.317 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:16.317 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:17.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:16 smithi027 ceph-mon[97244]: pgmap v2330: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:17.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:16 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:32:17.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:16 smithi115 ceph-mon[65453]: pgmap v2330: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:17.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:16 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:32:19.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:18 smithi027 ceph-mon[97244]: pgmap v2331: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:19.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:18 smithi115 ceph-mon[65453]: pgmap v2331: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:21.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:20 smithi027 ceph-mon[97244]: pgmap v2332: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:21.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4582.469624 secs 2021-11-03T18:32:21.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:20 smithi115 ceph-mon[65453]: pgmap v2332: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:21.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4582.469624 secs 2021-11-03T18:32:23.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:22 smithi115 ceph-mon[65453]: pgmap v2333: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:23.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:22 smithi027 ceph-mon[97244]: pgmap v2333: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:25.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:24 smithi115 ceph-mon[65453]: pgmap v2334: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:25.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:24 smithi027 ceph-mon[97244]: pgmap v2334: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:26.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4587.469764 secs 2021-11-03T18:32:26.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:32:26.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:32:26.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:32:26.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:26.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:26.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:26.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:26.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4587.469764 secs 2021-11-03T18:32:26.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:32:26.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:32:26.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:32:26.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:26.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:26.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:26.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:27.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:26 smithi115 ceph-mon[65453]: pgmap v2335: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:27.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:26 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:32:27.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:26 smithi027 ceph-mon[97244]: pgmap v2335: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:27.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:26 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:32:29.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:28 smithi115 ceph-mon[65453]: pgmap v2336: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:29.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:28 smithi027 ceph-mon[97244]: pgmap v2336: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:31.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:30 smithi115 ceph-mon[65453]: pgmap v2337: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:31.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4592.469908 secs 2021-11-03T18:32:31.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:30 smithi027 ceph-mon[97244]: pgmap v2337: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:31.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4592.469908 secs 2021-11-03T18:32:32.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.005+0000 7fd382951700 1 -- 172.21.15.27:0/1189447942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd37c0f66e0 msgr2=0x7fd37c0f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:32:32.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.005+0000 7fd382951700 1 --2- 172.21.15.27:0/1189447942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd37c0f66e0 0x7fd37c0f6b00 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 rx=0x7fd364004660 tx=0x7fd364009b30).stop 2021-11-03T18:32:32.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.005+0000 7fd382951700 1 -- 172.21.15.27:0/1189447942 shutdown_connections 2021-11-03T18:32:32.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.005+0000 7fd382951700 1 --2- 172.21.15.27:0/1189447942 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd37c0f6ff0 0x7fd37c0f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:32.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.005+0000 7fd382951700 1 --2- 172.21.15.27:0/1189447942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd37c0f66e0 0x7fd37c0f6b00 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:32.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.005+0000 7fd382951700 1 -- 172.21.15.27:0/1189447942 >> 172.21.15.27:0/1189447942 conn(0x7fd37c0f1f00 msgr2=0x7fd37c0f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:32:32.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.005+0000 7fd382951700 1 -- 172.21.15.27:0/1189447942 shutdown_connections 2021-11-03T18:32:32.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.005+0000 7fd382951700 1 -- 172.21.15.27:0/1189447942 wait complete. 2021-11-03T18:32:32.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.006+0000 7fd382951700 1 Processor -- start 2021-11-03T18:32:32.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.006+0000 7fd382951700 1 -- start start 2021-11-03T18:32:32.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.006+0000 7fd382951700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd37c0f66e0 0x7fd37c106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:32:32.009 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.006+0000 7fd382951700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd37c0f6ff0 0x7fd37c107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:32:32.009 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.006+0000 7fd382951700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c107a00 con 0x7fd37c0f6ff0 2021-11-03T18:32:32.009 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.006+0000 7fd382951700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37c107b40 con 0x7fd37c0f66e0 2021-11-03T18:32:32.009 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.006+0000 7fd37b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd37c0f6ff0 0x7fd37c107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:32:32.010 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.006+0000 7fd37b7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd37c0f6ff0 0x7fd37c107450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35440/0 (socket says 172.21.15.27:35440) 2021-11-03T18:32:32.010 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.007+0000 7fd37b7fe700 1 -- 172.21.15.27:0/295215975 learned_addr learned my addr 172.21.15.27:0/295215975 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:32:32.010 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.007+0000 7fd37bfff700 1 --2- 172.21.15.27:0/295215975 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd37c0f66e0 0x7fd37c106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:32:32.010 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.007+0000 7fd37b7fe700 1 -- 172.21.15.27:0/295215975 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd37c0f66e0 msgr2=0x7fd37c106f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:32:32.011 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.007+0000 7fd37b7fe700 1 --2- 172.21.15.27:0/295215975 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd37c0f66e0 0x7fd37c106f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:32.011 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.007+0000 7fd37b7fe700 1 -- 172.21.15.27:0/295215975 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd364005040 con 0x7fd37c0f6ff0 2021-11-03T18:32:32.011 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.007+0000 7fd37b7fe700 1 --2- 172.21.15.27:0/295215975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd37c0f6ff0 0x7fd37c107450 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 rx=0x7fd36c00c790 tx=0x7fd36c007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:32:32.011 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.007+0000 7fd378ff9700 1 -- 172.21.15.27:0/295215975 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd36c00e680 con 0x7fd37c0f6ff0 2021-11-03T18:32:32.012 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.007+0000 7fd382951700 1 -- 172.21.15.27:0/295215975 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd37c1084e0 con 0x7fd37c0f6ff0 2021-11-03T18:32:32.012 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.007+0000 7fd378ff9700 1 -- 172.21.15.27:0/295215975 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fd36c00ec90 con 0x7fd37c0f6ff0 2021-11-03T18:32:32.012 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.007+0000 7fd378ff9700 1 -- 172.21.15.27:0/295215975 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd36c013d60 con 0x7fd37c0f6ff0 2021-11-03T18:32:32.013 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.007+0000 7fd382951700 1 -- 172.21.15.27:0/295215975 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd37c108aa0 con 0x7fd37c0f6ff0 2021-11-03T18:32:32.013 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.008+0000 7fd382951700 1 -- 172.21.15.27:0/295215975 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd37c045ad0 con 0x7fd37c0f6ff0 2021-11-03T18:32:32.014 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.009+0000 7fd378ff9700 1 -- 172.21.15.27:0/295215975 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fd36c01a060 con 0x7fd37c0f6ff0 2021-11-03T18:32:32.014 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.009+0000 7fd378ff9700 1 --2- 172.21.15.27:0/295215975 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd3680640b0 0x7fd368066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:32:32.015 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.010+0000 7fd378ff9700 1 -- 172.21.15.27:0/295215975 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fd36c080c80 con 0x7fd37c0f6ff0 2021-11-03T18:32:32.015 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.010+0000 7fd37bfff700 1 --2- 172.21.15.27:0/295215975 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd3680640b0 0x7fd368066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:32:32.015 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.010+0000 7fd37bfff700 1 --2- 172.21.15.27:0/295215975 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd3680640b0 0x7fd368066570 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7fd364004e30 tx=0x7fd364009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:32:32.016 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.013+0000 7fd378ff9700 1 -- 172.21.15.27:0/295215975 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fd36c052e30 con 0x7fd37c0f6ff0 2021-11-03T18:32:32.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.191+0000 7fd382951700 1 -- 172.21.15.27:0/295215975 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fd37c0ff040 con 0x7fd3680640b0 2021-11-03T18:32:32.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.192+0000 7fd378ff9700 1 -- 172.21.15.27:0/295215975 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fd37c0ff040 con 0x7fd3680640b0 2021-11-03T18:32:32.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.194+0000 7fd382951700 1 -- 172.21.15.27:0/295215975 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd3680640b0 msgr2=0x7fd368066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:32:32.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.194+0000 7fd382951700 1 --2- 172.21.15.27:0/295215975 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd3680640b0 0x7fd368066570 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7fd364004e30 tx=0x7fd364009b30).stop 2021-11-03T18:32:32.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.194+0000 7fd382951700 1 -- 172.21.15.27:0/295215975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd37c0f6ff0 msgr2=0x7fd37c107450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:32:32.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.194+0000 7fd382951700 1 --2- 172.21.15.27:0/295215975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd37c0f6ff0 0x7fd37c107450 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 rx=0x7fd36c00c790 tx=0x7fd36c007d70).stop 2021-11-03T18:32:32.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.194+0000 7fd382951700 1 -- 172.21.15.27:0/295215975 shutdown_connections 2021-11-03T18:32:32.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.194+0000 7fd382951700 1 --2- 172.21.15.27:0/295215975 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd3680640b0 0x7fd368066570 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:32.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.194+0000 7fd382951700 1 --2- 172.21.15.27:0/295215975 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd37c0f66e0 0x7fd37c106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:32.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.194+0000 7fd382951700 1 --2- 172.21.15.27:0/295215975 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd37c0f6ff0 0x7fd37c107450 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:32.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.194+0000 7fd382951700 1 -- 172.21.15.27:0/295215975 >> 172.21.15.27:0/295215975 conn(0x7fd37c0f1f00 msgr2=0x7fd37c0fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:32:32.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.194+0000 7fd382951700 1 -- 172.21.15.27:0/295215975 shutdown_connections 2021-11-03T18:32:32.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.194+0000 7fd382951700 1 -- 172.21.15.27:0/295215975 wait complete. 2021-11-03T18:32:32.201 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:32:32.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.294+0000 7f00e52ae700 1 -- 172.21.15.27:0/2419807179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00e00f66e0 msgr2=0x7f00e00f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:32:32.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.294+0000 7f00e52ae700 1 --2- 172.21.15.27:0/2419807179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00e00f66e0 0x7f00e00f6b00 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 rx=0x7f00d4004660 tx=0x7f00d4009b30).stop 2021-11-03T18:32:32.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.294+0000 7f00e52ae700 1 -- 172.21.15.27:0/2419807179 shutdown_connections 2021-11-03T18:32:32.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.294+0000 7f00e52ae700 1 --2- 172.21.15.27:0/2419807179 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00e00f6ff0 0x7f00e00f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:32.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.294+0000 7f00e52ae700 1 --2- 172.21.15.27:0/2419807179 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00e00f66e0 0x7f00e00f6b00 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:32.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.294+0000 7f00e52ae700 1 -- 172.21.15.27:0/2419807179 >> 172.21.15.27:0/2419807179 conn(0x7f00e00f1f00 msgr2=0x7f00e00f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:32:32.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.294+0000 7f00e52ae700 1 -- 172.21.15.27:0/2419807179 shutdown_connections 2021-11-03T18:32:32.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.294+0000 7f00e52ae700 1 -- 172.21.15.27:0/2419807179 wait complete. 2021-11-03T18:32:32.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.295+0000 7f00e52ae700 1 Processor -- start 2021-11-03T18:32:32.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.295+0000 7f00e52ae700 1 -- start start 2021-11-03T18:32:32.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.295+0000 7f00e52ae700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00e00f66e0 0x7f00e0106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:32:32.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.295+0000 7f00e52ae700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00e00f6ff0 0x7f00e0107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:32:32.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.295+0000 7f00e52ae700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e0107a00 con 0x7f00e00f66e0 2021-11-03T18:32:32.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.295+0000 7f00e52ae700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00e0107b40 con 0x7f00e00f6ff0 2021-11-03T18:32:32.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.295+0000 7f00deffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00e00f66e0 0x7f00e0106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:32:32.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.295+0000 7f00deffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00e00f66e0 0x7f00e0106f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35448/0 (socket says 172.21.15.27:35448) 2021-11-03T18:32:32.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.295+0000 7f00deffd700 1 -- 172.21.15.27:0/4018658050 learned_addr learned my addr 172.21.15.27:0/4018658050 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:32:32.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.295+0000 7f00deffd700 1 -- 172.21.15.27:0/4018658050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00e00f6ff0 msgr2=0x7f00e0107450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:32:32.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.295+0000 7f00deffd700 1 --2- 172.21.15.27:0/4018658050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00e00f6ff0 0x7f00e0107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:32:32.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.295+0000 7f00deffd700 1 -- 172.21.15.27:0/4018658050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00d4005040 con 0x7f00e00f66e0 2021-11-03T18:32:32.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.295+0000 7f00deffd700 1 --2- 172.21.15.27:0/4018658050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00e00f66e0 0x7f00e0106f60 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 rx=0x7f00d4009990 tx=0x7f00d4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:32:32.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.296+0000 7f00c77fe700 1 -- 172.21.15.27:0/4018658050 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00d4018070 con 0x7f00e00f66e0 2021-11-03T18:32:32.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.296+0000 7f00e52ae700 1 -- 172.21.15.27:0/4018658050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00e0108480 con 0x7f00e00f66e0 2021-11-03T18:32:32.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.296+0000 7f00c77fe700 1 -- 172.21.15.27:0/4018658050 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f00d4007ba0 con 0x7f00e00f66e0 2021-11-03T18:32:32.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.296+0000 7f00c77fe700 1 -- 172.21.15.27:0/4018658050 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f00d401c3c0 con 0x7f00e00f66e0 2021-11-03T18:32:32.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.296+0000 7f00e52ae700 1 -- 172.21.15.27:0/4018658050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00e01089e0 con 0x7f00e00f66e0 2021-11-03T18:32:32.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.296+0000 7f00e52ae700 1 -- 172.21.15.27:0/4018658050 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00e0045ad0 con 0x7f00e00f66e0 2021-11-03T18:32:32.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.297+0000 7f00c77fe700 1 -- 172.21.15.27:0/4018658050 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f00d401d420 con 0x7f00e00f66e0 2021-11-03T18:32:32.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.297+0000 7f00c77fe700 1 --2- 172.21.15.27:0/4018658050 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f00c80640b0 0x7f00c8066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:32:32.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.297+0000 7f00c77fe700 1 -- 172.21.15.27:0/4018658050 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f00d4082500 con 0x7f00e00f66e0 2021-11-03T18:32:32.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.298+0000 7f00de7fc700 1 --2- 172.21.15.27:0/4018658050 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f00c80640b0 0x7f00c8066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:32:32.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.298+0000 7f00de7fc700 1 --2- 172.21.15.27:0/4018658050 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f00c80640b0 0x7f00c8066570 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7f00d0004a10 tx=0x7f00d00099e0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:32:32.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.301+0000 7f00c77fe700 1 -- 172.21.15.27:0/4018658050 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f00d40548c0 con 0x7f00e00f66e0 2021-11-03T18:32:32.476 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.478+0000 7f00e52ae700 1 -- 172.21.15.27:0/4018658050 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f00e0108cc0 con 0x7f00c80640b0 2021-11-03T18:32:32.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.493+0000 7f00c77fe700 1 -- 172.21.15.27:0/4018658050 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f00e0108cc0 con 0x7f00c80640b0 2021-11-03T18:32:32.491 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:32:32.492 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (84m) 2m ago 85m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:32:32.492 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (75m) 2m ago 85m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:32:32.492 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (75m) 2m ago 84m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:32:32.492 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (83m) 2m ago 85m 42.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:32:32.492 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (77m) 2m ago 80m 26.7M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:32:32.493 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (80m) 2m ago 80m 3850M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:32:32.493 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (76m) 2m ago 80m 26.3M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:32:32.493 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (77m) 2m ago 80m 27.4M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:32:32.493 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (77m) 2m ago 86m 608M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:32:32.494 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (77m) 2m ago 84m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:32:32.494 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (76m) 2m ago 86m 350M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:32:32.494 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (76m) 2m ago 84m 160M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:32:32.494 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (84m) 2m ago 84m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:32:32.494 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (84m) 2m ago 84m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:32:32.495 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (75m) 2m ago 83m 413M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:32:32.495 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (74m) 2m ago 83m 450M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:32:32.495 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (74m) 2m ago 83m 338M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:32:32.495 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (73m) 2m ago 82m 479M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:32:32.497 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (73m) 2m ago 82m 378M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:32:32.497 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (73m) 2m ago 82m 388M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:32:32.497 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (83m) 2m ago 84m 64.4M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:32:32.498 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.494+0000 7f00e52ae700 1 -- 172.21.15.27:0/4018658050 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f00c80640b0 msgr2=0x7f00c8066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:32:32.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.494+0000 7f00e52ae700 1 --2- 172.21.15.27:0/4018658050 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f00c80640b0 0x7f00c8066570 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 rx=0x7f00d0004a10 tx=0x7f00d00099e0).stop 2021-11-03T18:32:32.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.494+0000 7f00e52ae700 1 -- 172.21.15.27:0/4018658050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00e00f66e0 msgr2=0x7f00e0106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:32:32.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.494+0000 7f00e52ae700 1 --2- 172.21.15.27:0/4018658050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00e00f66e0 0x7f00e0106f60 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 rx=0x7f00d4009990 tx=0x7f00d4009b30).stop 2021-11-03T18:32:32.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.495+0000 7f00e52ae700 1 -- 172.21.15.27:0/4018658050 shutdown_connections 2021-11-03T18:32:32.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.495+0000 7f00e52ae700 1 --2- 172.21.15.27:0/4018658050 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f00c80640b0 0x7f00c8066570 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:32.500 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.495+0000 7f00e52ae700 1 --2- 172.21.15.27:0/4018658050 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f00e00f66e0 0x7f00e0106f60 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:32.500 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.495+0000 7f00e52ae700 1 --2- 172.21.15.27:0/4018658050 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f00e00f6ff0 0x7f00e0107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:32:32.500 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.495+0000 7f00e52ae700 1 -- 172.21.15.27:0/4018658050 >> 172.21.15.27:0/4018658050 conn(0x7f00e00f1f00 msgr2=0x7f00e00fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:32:32.500 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.495+0000 7f00e52ae700 1 -- 172.21.15.27:0/4018658050 shutdown_connections 2021-11-03T18:32:32.501 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.495+0000 7f00e52ae700 1 -- 172.21.15.27:0/4018658050 wait complete. 2021-11-03T18:32:32.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.593+0000 7f393f022700 1 -- 172.21.15.27:0/4219916833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f393806a3b0 msgr2=0x7f393806a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:32:32.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.593+0000 7f393f022700 1 --2- 172.21.15.27:0/4219916833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f393806a3b0 0x7f393806a830 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 rx=0x7f3928004660 tx=0x7f3928009b30).stop 2021-11-03T18:32:32.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.594+0000 7f393f022700 1 -- 172.21.15.27:0/4219916833 shutdown_connections 2021-11-03T18:32:32.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.594+0000 7f393f022700 1 --2- 172.21.15.27:0/4219916833 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f393806a3b0 0x7f393806a830 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:32.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.594+0000 7f393f022700 1 --2- 172.21.15.27:0/4219916833 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f393806b9c0 0x7f3938069ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:32.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.594+0000 7f393f022700 1 -- 172.21.15.27:0/4219916833 >> 172.21.15.27:0/4219916833 conn(0x7f39380f1cd0 msgr2=0x7f39380f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:32:32.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.594+0000 7f393f022700 1 -- 172.21.15.27:0/4219916833 shutdown_connections 2021-11-03T18:32:32.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.594+0000 7f393f022700 1 -- 172.21.15.27:0/4219916833 wait complete. 2021-11-03T18:32:32.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.594+0000 7f393f022700 1 Processor -- start 2021-11-03T18:32:32.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.594+0000 7f393f022700 1 -- start start 2021-11-03T18:32:32.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.594+0000 7f393f022700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f393806a3b0 0x7f3938108f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:32:32.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.594+0000 7f393f022700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f393806b9c0 0x7f3938109410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:32:32.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.594+0000 7f393f022700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39381099c0 con 0x7f393806b9c0 2021-11-03T18:32:32.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.594+0000 7f393f022700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3938109b00 con 0x7f393806a3b0 2021-11-03T18:32:32.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.594+0000 7f393d81f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f393806b9c0 0x7f3938109410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:32:32.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.594+0000 7f393d81f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f393806b9c0 0x7f3938109410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35460/0 (socket says 172.21.15.27:35460) 2021-11-03T18:32:32.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.594+0000 7f393d81f700 1 -- 172.21.15.27:0/360659923 learned_addr learned my addr 172.21.15.27:0/360659923 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:32:32.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.595+0000 7f393e020700 1 --2- 172.21.15.27:0/360659923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f393806a3b0 0x7f3938108f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:32:32.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.595+0000 7f393d81f700 1 -- 172.21.15.27:0/360659923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f393806a3b0 msgr2=0x7f3938108f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:32:32.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.595+0000 7f393d81f700 1 --2- 172.21.15.27:0/360659923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f393806a3b0 0x7f3938108f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:32.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.595+0000 7f393d81f700 1 -- 172.21.15.27:0/360659923 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3928005040 con 0x7f393806b9c0 2021-11-03T18:32:32.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.595+0000 7f393d81f700 1 --2- 172.21.15.27:0/360659923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f393806b9c0 0x7f3938109410 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 rx=0x7f3928009990 tx=0x7f3928009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:32:32.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.595+0000 7f392effd700 1 -- 172.21.15.27:0/360659923 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3928018070 con 0x7f393806b9c0 2021-11-03T18:32:32.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.595+0000 7f392effd700 1 -- 172.21.15.27:0/360659923 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f3928007ba0 con 0x7f393806b9c0 2021-11-03T18:32:32.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.595+0000 7f393f022700 1 -- 172.21.15.27:0/360659923 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f393810a440 con 0x7f393806b9c0 2021-11-03T18:32:32.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.595+0000 7f392effd700 1 -- 172.21.15.27:0/360659923 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f392801c3c0 con 0x7f393806b9c0 2021-11-03T18:32:32.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.595+0000 7f393f022700 1 -- 172.21.15.27:0/360659923 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f393810a9a0 con 0x7f393806b9c0 2021-11-03T18:32:32.600 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.596+0000 7f393f022700 1 -- 172.21.15.27:0/360659923 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3938045ad0 con 0x7f393806b9c0 2021-11-03T18:32:32.600 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.597+0000 7f392effd700 1 -- 172.21.15.27:0/360659923 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f392801d420 con 0x7f393806b9c0 2021-11-03T18:32:32.601 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.597+0000 7f392effd700 1 --2- 172.21.15.27:0/360659923 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3924064100 0x7f39240665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:32:32.601 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.597+0000 7f392effd700 1 -- 172.21.15.27:0/360659923 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f3928082360 con 0x7f393806b9c0 2021-11-03T18:32:32.601 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.597+0000 7f393e020700 1 --2- 172.21.15.27:0/360659923 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3924064100 0x7f39240665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:32:32.601 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.598+0000 7f393e020700 1 --2- 172.21.15.27:0/360659923 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3924064100 0x7f39240665c0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7f3934004c40 tx=0x7f393400b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:32:32.601 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.601+0000 7f392effd700 1 -- 172.21.15.27:0/360659923 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f3928054720 con 0x7f393806b9c0 2021-11-03T18:32:32.829 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.831+0000 7f393f022700 1 -- 172.21.15.27:0/360659923 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3938027a00 con 0x7f393806b9c0 2021-11-03T18:32:32.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.832+0000 7f392effd700 1 -- 172.21.15.27:0/360659923 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f3928054310 con 0x7f393806b9c0 2021-11-03T18:32:32.830 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:32:32.831 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:32:32.831 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:32:32.831 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:32:32.831 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:32:32.831 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:32:32.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:32:32.832 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:32:32.832 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:32:32.832 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:32:32.833 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:32:32.833 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:32:32.833 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:32:32.833 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:32:32.833 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:32:32.834 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:32:32.834 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:32:32.834 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:32:32.835 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.834+0000 7f393f022700 1 -- 172.21.15.27:0/360659923 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3924064100 msgr2=0x7f39240665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:32:32.835 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.834+0000 7f393f022700 1 --2- 172.21.15.27:0/360659923 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3924064100 0x7f39240665c0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7f3934004c40 tx=0x7f393400b040).stop 2021-11-03T18:32:32.835 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.834+0000 7f393f022700 1 -- 172.21.15.27:0/360659923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f393806b9c0 msgr2=0x7f3938109410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:32:32.836 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.834+0000 7f393f022700 1 --2- 172.21.15.27:0/360659923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f393806b9c0 0x7f3938109410 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 rx=0x7f3928009990 tx=0x7f3928009b30).stop 2021-11-03T18:32:32.836 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.834+0000 7f393f022700 1 -- 172.21.15.27:0/360659923 shutdown_connections 2021-11-03T18:32:32.836 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.834+0000 7f393f022700 1 --2- 172.21.15.27:0/360659923 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3924064100 0x7f39240665c0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:32.836 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.834+0000 7f393f022700 1 --2- 172.21.15.27:0/360659923 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f393806a3b0 0x7f3938108f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:32.837 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.834+0000 7f393f022700 1 --2- 172.21.15.27:0/360659923 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f393806b9c0 0x7f3938109410 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:32.837 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.834+0000 7f393f022700 1 -- 172.21.15.27:0/360659923 >> 172.21.15.27:0/360659923 conn(0x7f39380f1cd0 msgr2=0x7f39380fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:32:32.837 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.834+0000 7f393f022700 1 -- 172.21.15.27:0/360659923 shutdown_connections 2021-11-03T18:32:32.837 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.834+0000 7f393f022700 1 -- 172.21.15.27:0/360659923 wait complete. 2021-11-03T18:32:32.932 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.934+0000 7f08868d6700 1 -- 172.21.15.27:0/924031465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08800f5ed0 msgr2=0x7f08800f6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:32:32.932 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.934+0000 7f08868d6700 1 --2- 172.21.15.27:0/924031465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08800f5ed0 0x7f08800f6900 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 rx=0x7f0870004660 tx=0x7f0870009b30).stop 2021-11-03T18:32:32.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.934+0000 7f08868d6700 1 -- 172.21.15.27:0/924031465 shutdown_connections 2021-11-03T18:32:32.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.934+0000 7f08868d6700 1 --2- 172.21.15.27:0/924031465 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08800f5ed0 0x7f08800f6900 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:32.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.934+0000 7f08868d6700 1 --2- 172.21.15.27:0/924031465 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08800f55c0 0x7f08800f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:32.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.934+0000 7f08868d6700 1 -- 172.21.15.27:0/924031465 >> 172.21.15.27:0/924031465 conn(0x7f08800f0dc0 msgr2=0x7f08800f31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:32:32.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.934+0000 7f08868d6700 1 -- 172.21.15.27:0/924031465 shutdown_connections 2021-11-03T18:32:32.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.934+0000 7f08868d6700 1 -- 172.21.15.27:0/924031465 wait complete. 2021-11-03T18:32:32.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.934+0000 7f08868d6700 1 Processor -- start 2021-11-03T18:32:32.934 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.934+0000 7f08868d6700 1 -- start start 2021-11-03T18:32:32.936 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.935+0000 7f08868d6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08800f55c0 0x7f0880106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:32:32.937 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.935+0000 7f08868d6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08800f5ed0 0x7f0880107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:32:32.937 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.935+0000 7f08868d6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0880107a00 con 0x7f08800f55c0 2021-11-03T18:32:32.937 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.935+0000 7f08868d6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0880107b40 con 0x7f08800f5ed0 2021-11-03T18:32:32.937 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.935+0000 7f087ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08800f55c0 0x7f0880106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:32:32.938 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.935+0000 7f087ffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08800f55c0 0x7f0880106f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35468/0 (socket says 172.21.15.27:35468) 2021-11-03T18:32:32.938 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.935+0000 7f087ffff700 1 -- 172.21.15.27:0/3423066934 learned_addr learned my addr 172.21.15.27:0/3423066934 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:32:32.938 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.935+0000 7f087f7fe700 1 --2- 172.21.15.27:0/3423066934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08800f5ed0 0x7f0880107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:32:32.938 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.935+0000 7f087ffff700 1 -- 172.21.15.27:0/3423066934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08800f5ed0 msgr2=0x7f0880107450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:32:32.939 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.935+0000 7f087ffff700 1 --2- 172.21.15.27:0/3423066934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08800f5ed0 0x7f0880107450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:32.939 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.935+0000 7f087ffff700 1 -- 172.21.15.27:0/3423066934 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0870005040 con 0x7f08800f55c0 2021-11-03T18:32:32.939 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.935+0000 7f087ffff700 1 --2- 172.21.15.27:0/3423066934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08800f55c0 0x7f0880106f60 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 rx=0x7f08740026d0 tx=0x7f0874009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:32:32.939 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.935+0000 7f087cff9700 1 -- 172.21.15.27:0/3423066934 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08740079c0 con 0x7f08800f55c0 2021-11-03T18:32:32.939 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.935+0000 7f087cff9700 1 -- 172.21.15.27:0/3423066934 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f087400f040 con 0x7f08800f55c0 2021-11-03T18:32:32.940 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.935+0000 7f08868d6700 1 -- 172.21.15.27:0/3423066934 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08801084e0 con 0x7f08800f55c0 2021-11-03T18:32:32.940 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.935+0000 7f087cff9700 1 -- 172.21.15.27:0/3423066934 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08740148b0 con 0x7f08800f55c0 2021-11-03T18:32:32.940 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.935+0000 7f08868d6700 1 -- 172.21.15.27:0/3423066934 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0880108aa0 con 0x7f08800f55c0 2021-11-03T18:32:32.940 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.936+0000 7f08868d6700 1 -- 172.21.15.27:0/3423066934 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0880045ad0 con 0x7f08800f55c0 2021-11-03T18:32:32.940 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.937+0000 7f087cff9700 1 -- 172.21.15.27:0/3423066934 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f0874007b20 con 0x7f08800f55c0 2021-11-03T18:32:32.941 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.937+0000 7f087cff9700 1 --2- 172.21.15.27:0/3423066934 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0868064180 0x7f0868066640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:32:32.941 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.937+0000 7f087cff9700 1 -- 172.21.15.27:0/3423066934 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f0874080640 con 0x7f08800f55c0 2021-11-03T18:32:32.941 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.937+0000 7f087f7fe700 1 --2- 172.21.15.27:0/3423066934 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0868064180 0x7f0868066640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:32:32.941 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.938+0000 7f087f7fe700 1 --2- 172.21.15.27:0/3423066934 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0868064180 0x7f0868066640 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f0870004630 tx=0x7f0870009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:32:32.942 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:32.941+0000 7f087cff9700 1 -- 172.21.15.27:0/3423066934 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f0874052ad0 con 0x7f08800f55c0 2021-11-03T18:32:33.144 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:33.146+0000 7f08868d6700 1 -- 172.21.15.27:0/3423066934 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f08800ff060 con 0x7f08800f55c0 2021-11-03T18:32:33.144 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:33.146+0000 7f087cff9700 1 -- 172.21.15.27:0/3423066934 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f08740526c0 con 0x7f08800f55c0 2021-11-03T18:32:33.145 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:32:33.145 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:32:33.145 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:32:33.145 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:32:33.146 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:32:33.146 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:32:33.146 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:32:33.146 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:32:33.147 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:32:33.147 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:32:33.147 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:32:33.147 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:32:33.148 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:32:33.148 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:32:33.148 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:32:33.148 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:32:33.149 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:32:33.149 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:32:33.149 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:32:33.149 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:32:33.149 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:32:33.150 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:32:33.150 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:32:33.150 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:32:33.150 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:32:33.151 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:32:33.151 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:32:33.151 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:32:33.151 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:32:33.151 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:32:33.152 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:32:33.152 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:32:33.152 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:32:33.152 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:32:33.153 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:32:33.153 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:32:33.153 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:32:33.153 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:32:33.154 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:32:33.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:33.148+0000 7f08868d6700 1 -- 172.21.15.27:0/3423066934 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0868064180 msgr2=0x7f0868066640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:32:33.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:33.148+0000 7f08868d6700 1 --2- 172.21.15.27:0/3423066934 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0868064180 0x7f0868066640 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f0870004630 tx=0x7f0870009b30).stop 2021-11-03T18:32:33.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:33.148+0000 7f08868d6700 1 -- 172.21.15.27:0/3423066934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08800f55c0 msgr2=0x7f0880106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:32:33.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:33.148+0000 7f08868d6700 1 --2- 172.21.15.27:0/3423066934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08800f55c0 0x7f0880106f60 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 rx=0x7f08740026d0 tx=0x7f0874009de0).stop 2021-11-03T18:32:33.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:33.148+0000 7f08868d6700 1 -- 172.21.15.27:0/3423066934 shutdown_connections 2021-11-03T18:32:33.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:33.148+0000 7f08868d6700 1 --2- 172.21.15.27:0/3423066934 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0868064180 0x7f0868066640 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:33.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:33.148+0000 7f08868d6700 1 --2- 172.21.15.27:0/3423066934 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08800f55c0 0x7f0880106f60 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:33.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:33.148+0000 7f08868d6700 1 --2- 172.21.15.27:0/3423066934 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08800f5ed0 0x7f0880107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:32:33.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:33.148+0000 7f08868d6700 1 -- 172.21.15.27:0/3423066934 >> 172.21.15.27:0/3423066934 conn(0x7f08800f0dc0 msgr2=0x7f08800fa350 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:32:33.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:33.148+0000 7f08868d6700 1 -- 172.21.15.27:0/3423066934 shutdown_connections 2021-11-03T18:32:33.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:32:33.148+0000 7f08868d6700 1 -- 172.21.15.27:0/3423066934 wait complete. 2021-11-03T18:32:33.157 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:32:33.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:32 smithi115 ceph-mon[65453]: pgmap v2338: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:33.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:32 smithi027 ceph-mon[97244]: pgmap v2338: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:34.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:33 smithi115 ceph-mon[65453]: from='client.36504 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:32:34.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:33 smithi115 ceph-mon[65453]: from='client.36508 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:32:34.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:33 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/360659923' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:34.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:33 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3423066934' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:32:34.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:33 smithi027 ceph-mon[97244]: from='client.36504 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:32:34.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:33 smithi027 ceph-mon[97244]: from='client.36508 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:32:34.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:33 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/360659923' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:34.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:33 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3423066934' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:32:35.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:34 smithi115 ceph-mon[65453]: pgmap v2339: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:35.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:34 smithi027 ceph-mon[97244]: pgmap v2339: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:35.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4597.470049 secs 2021-11-03T18:32:35.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:35 smithi115 ceph-mon[65453]: pgmap v2340: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:35.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:32:35.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:35 smithi115 ceph-mon[65453]: poll took 60186.817 ms: 1 events 2021-11-03T18:32:36.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4597.470049 secs 2021-11-03T18:32:36.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:35 smithi027 ceph-mon[97244]: pgmap v2340: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:36.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:32:36.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:35 smithi027 ceph-mon[97244]: poll took 60186.817 ms: 1 events 2021-11-03T18:32:37.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:32:37.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:32:37.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:37.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:37.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:37.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:37.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:37 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:32:37.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:32:37.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:32:37.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:37.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:37.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:37.523 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:37.524 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:37 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:32:38.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:38 smithi115 ceph-mon[65453]: pgmap v2341: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:38.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:38 smithi027 ceph-mon[97244]: pgmap v2341: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:40.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:40 smithi115 ceph-mon[65453]: pgmap v2342: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:40.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4602.470155 secs 2021-11-03T18:32:40.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:40 smithi027 ceph-mon[97244]: pgmap v2342: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:40.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4602.470155 secs 2021-11-03T18:32:42.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:42 smithi115 ceph-mon[65453]: pgmap v2343: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:42.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:42 smithi027 ceph-mon[97244]: pgmap v2343: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:44.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:44 smithi115 ceph-mon[65453]: pgmap v2344: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:44.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:44 smithi027 ceph-mon[97244]: pgmap v2344: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:45.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4607.470254 secs 2021-11-03T18:32:45.771 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4607.470254 secs 2021-11-03T18:32:46.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:46 smithi115 ceph-mon[65453]: pgmap v2345: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:46.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:32:46.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:32:46.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:32:46.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:46.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:46.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:46.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:46.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:46 smithi027 ceph-mon[97244]: pgmap v2345: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:46.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:32:46.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:32:46.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:32:46.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:46.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:46.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:46.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:47.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:47 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:32:47.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:47 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:32:48.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:48 smithi115 ceph-mon[65453]: pgmap v2346: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:48.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:48 smithi027 ceph-mon[97244]: pgmap v2346: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:50.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:50 smithi115 ceph-mon[65453]: pgmap v2347: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:50.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4612.470378 secs 2021-11-03T18:32:50.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:50 smithi027 ceph-mon[97244]: pgmap v2347: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:50.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4612.470378 secs 2021-11-03T18:32:52.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:52 smithi115 ceph-mon[65453]: pgmap v2348: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:52.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:52 smithi027 ceph-mon[97244]: pgmap v2348: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:54.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:54 smithi115 ceph-mon[65453]: pgmap v2349: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:54.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:54 smithi027 ceph-mon[97244]: pgmap v2349: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:55.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4617.470512 secs 2021-11-03T18:32:55.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4617.470512 secs 2021-11-03T18:32:56.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:56 smithi115 ceph-mon[65453]: pgmap v2350: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:56.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:32:56.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:32:56.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:32:56.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:32:56.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:32:56.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:56.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:56.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:56.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:56.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:56 smithi027 ceph-mon[97244]: pgmap v2350: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:56.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:32:56.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:32:56.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:32:56.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:32:56.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:32:56.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:56.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:56.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:56.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:32:57.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:57 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:32:57.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:57 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:32:58.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:32:58 smithi115 ceph-mon[65453]: pgmap v2351: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:32:58.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:32:58 smithi027 ceph-mon[97244]: pgmap v2351: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:00.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:00 smithi027 ceph-mon[97244]: pgmap v2352: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:00.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4622.470607 secs 2021-11-03T18:33:00.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:00 smithi115 ceph-mon[65453]: pgmap v2352: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:00.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4622.470607 secs 2021-11-03T18:33:02.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:02 smithi027 ceph-mon[97244]: pgmap v2353: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:02.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:02 smithi115 ceph-mon[65453]: pgmap v2353: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:03.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.262+0000 7f6ba97bf700 1 -- 172.21.15.27:0/3263195773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ba406b9c0 msgr2=0x7f6ba4069ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:33:03.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.262+0000 7f6ba97bf700 1 --2- 172.21.15.27:0/3263195773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ba406b9c0 0x7f6ba4069ec0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 rx=0x7f6b94004660 tx=0x7f6b94009b30).stop 2021-11-03T18:33:03.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.263+0000 7f6ba97bf700 1 -- 172.21.15.27:0/3263195773 shutdown_connections 2021-11-03T18:33:03.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.263+0000 7f6ba97bf700 1 --2- 172.21.15.27:0/3263195773 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ba406a3b0 0x7f6ba406a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:03.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.263+0000 7f6ba97bf700 1 --2- 172.21.15.27:0/3263195773 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ba406b9c0 0x7f6ba4069ec0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:03.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.263+0000 7f6ba97bf700 1 -- 172.21.15.27:0/3263195773 >> 172.21.15.27:0/3263195773 conn(0x7f6ba40f1cd0 msgr2=0x7f6ba40f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:33:03.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.263+0000 7f6ba97bf700 1 -- 172.21.15.27:0/3263195773 shutdown_connections 2021-11-03T18:33:03.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.263+0000 7f6ba97bf700 1 -- 172.21.15.27:0/3263195773 wait complete. 2021-11-03T18:33:03.265 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.263+0000 7f6ba97bf700 1 Processor -- start 2021-11-03T18:33:03.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.263+0000 7f6ba97bf700 1 -- start start 2021-11-03T18:33:03.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.264+0000 7f6ba97bf700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ba406a3b0 0x7f6ba410d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:33:03.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.264+0000 7f6ba97bf700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ba406b9c0 0x7f6ba410d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:33:03.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.264+0000 7f6ba97bf700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba410de30 con 0x7f6ba406a3b0 2021-11-03T18:33:03.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.264+0000 7f6ba97bf700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ba410df70 con 0x7f6ba406b9c0 2021-11-03T18:33:03.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.264+0000 7f6ba2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ba406a3b0 0x7f6ba410d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:33:03.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.264+0000 7f6ba2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ba406a3b0 0x7f6ba410d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35488/0 (socket says 172.21.15.27:35488) 2021-11-03T18:33:03.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.264+0000 7f6ba2ffd700 1 -- 172.21.15.27:0/3700267262 learned_addr learned my addr 172.21.15.27:0/3700267262 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:33:03.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.264+0000 7f6ba27fc700 1 --2- 172.21.15.27:0/3700267262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ba406b9c0 0x7f6ba410d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:33:03.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.264+0000 7f6ba2ffd700 1 -- 172.21.15.27:0/3700267262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ba406b9c0 msgr2=0x7f6ba410d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:33:03.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.264+0000 7f6ba2ffd700 1 --2- 172.21.15.27:0/3700267262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ba406b9c0 0x7f6ba410d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:03.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.264+0000 7f6ba2ffd700 1 -- 172.21.15.27:0/3700267262 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b94005040 con 0x7f6ba406a3b0 2021-11-03T18:33:03.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.264+0000 7f6ba2ffd700 1 --2- 172.21.15.27:0/3700267262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ba406a3b0 0x7f6ba410d390 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 rx=0x7f6b94009990 tx=0x7f6b94009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:33:03.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.265+0000 7f6b8b7fe700 1 -- 172.21.15.27:0/3700267262 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b94018070 con 0x7f6ba406a3b0 2021-11-03T18:33:03.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.265+0000 7f6b8b7fe700 1 -- 172.21.15.27:0/3700267262 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f6b94007ba0 con 0x7f6ba406a3b0 2021-11-03T18:33:03.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.265+0000 7f6ba97bf700 1 -- 172.21.15.27:0/3700267262 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ba410e8b0 con 0x7f6ba406a3b0 2021-11-03T18:33:03.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.265+0000 7f6b8b7fe700 1 -- 172.21.15.27:0/3700267262 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b9401c3c0 con 0x7f6ba406a3b0 2021-11-03T18:33:03.270 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.265+0000 7f6ba97bf700 1 -- 172.21.15.27:0/3700267262 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ba410ee10 con 0x7f6ba406a3b0 2021-11-03T18:33:03.270 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.266+0000 7f6ba97bf700 1 -- 172.21.15.27:0/3700267262 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ba4045ad0 con 0x7f6ba406a3b0 2021-11-03T18:33:03.270 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.266+0000 7f6b8b7fe700 1 -- 172.21.15.27:0/3700267262 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f6b9401d420 con 0x7f6ba406a3b0 2021-11-03T18:33:03.270 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.267+0000 7f6b8b7fe700 1 --2- 172.21.15.27:0/3700267262 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6b8c064100 0x7f6b8c0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:33:03.271 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.267+0000 7f6b8b7fe700 1 -- 172.21.15.27:0/3700267262 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f6b94082340 con 0x7f6ba406a3b0 2021-11-03T18:33:03.271 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.267+0000 7f6ba27fc700 1 --2- 172.21.15.27:0/3700267262 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6b8c064100 0x7f6b8c0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:33:03.271 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.267+0000 7f6ba27fc700 1 --2- 172.21.15.27:0/3700267262 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6b8c064100 0x7f6b8c0665c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7f6b98004c40 tx=0x7f6b9800b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:33:03.272 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.271+0000 7f6b8b7fe700 1 -- 172.21.15.27:0/3700267262 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f6b94054760 con 0x7f6ba406a3b0 2021-11-03T18:33:03.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.448+0000 7f6ba97bf700 1 -- 172.21.15.27:0/3700267262 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ba4105440 con 0x7f6b8c064100 2021-11-03T18:33:03.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.450+0000 7f6b8b7fe700 1 -- 172.21.15.27:0/3700267262 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f6ba4105440 con 0x7f6b8c064100 2021-11-03T18:33:03.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.451+0000 7f6ba97bf700 1 -- 172.21.15.27:0/3700267262 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6b8c064100 msgr2=0x7f6b8c0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:33:03.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.451+0000 7f6ba97bf700 1 --2- 172.21.15.27:0/3700267262 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6b8c064100 0x7f6b8c0665c0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7f6b98004c40 tx=0x7f6b9800b040).stop 2021-11-03T18:33:03.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.451+0000 7f6ba97bf700 1 -- 172.21.15.27:0/3700267262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ba406a3b0 msgr2=0x7f6ba410d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:33:03.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.451+0000 7f6ba97bf700 1 --2- 172.21.15.27:0/3700267262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ba406a3b0 0x7f6ba410d390 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 rx=0x7f6b94009990 tx=0x7f6b94009b30).stop 2021-11-03T18:33:03.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.451+0000 7f6ba97bf700 1 -- 172.21.15.27:0/3700267262 shutdown_connections 2021-11-03T18:33:03.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.451+0000 7f6ba97bf700 1 --2- 172.21.15.27:0/3700267262 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6b8c064100 0x7f6b8c0665c0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:03.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.451+0000 7f6ba97bf700 1 --2- 172.21.15.27:0/3700267262 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6ba406a3b0 0x7f6ba410d390 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:03.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.451+0000 7f6ba97bf700 1 --2- 172.21.15.27:0/3700267262 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6ba406b9c0 0x7f6ba410d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:03.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.451+0000 7f6ba97bf700 1 -- 172.21.15.27:0/3700267262 >> 172.21.15.27:0/3700267262 conn(0x7f6ba40f1cd0 msgr2=0x7f6ba40f8840 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:33:03.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.451+0000 7f6ba97bf700 1 -- 172.21.15.27:0/3700267262 shutdown_connections 2021-11-03T18:33:03.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.451+0000 7f6ba97bf700 1 -- 172.21.15.27:0/3700267262 wait complete. 2021-11-03T18:33:03.459 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:33:03.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.552+0000 7fae1db9c700 1 -- 172.21.15.27:0/1145169597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae180f49a0 msgr2=0x7fae180f4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:33:03.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.552+0000 7fae1db9c700 1 --2- 172.21.15.27:0/1145169597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae180f49a0 0x7fae180f4dc0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 rx=0x7fae08004660 tx=0x7fae08009b30).stop 2021-11-03T18:33:03.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.553+0000 7fae1db9c700 1 -- 172.21.15.27:0/1145169597 shutdown_connections 2021-11-03T18:33:03.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.553+0000 7fae1db9c700 1 --2- 172.21.15.27:0/1145169597 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae180f5340 0x7fae180f7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:03.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.553+0000 7fae1db9c700 1 --2- 172.21.15.27:0/1145169597 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae180f49a0 0x7fae180f4dc0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:03.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.553+0000 7fae1db9c700 1 -- 172.21.15.27:0/1145169597 >> 172.21.15.27:0/1145169597 conn(0x7fae180f04c0 msgr2=0x7fae180f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:33:03.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.553+0000 7fae1db9c700 1 -- 172.21.15.27:0/1145169597 shutdown_connections 2021-11-03T18:33:03.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.553+0000 7fae1db9c700 1 -- 172.21.15.27:0/1145169597 wait complete. 2021-11-03T18:33:03.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.553+0000 7fae1db9c700 1 Processor -- start 2021-11-03T18:33:03.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.553+0000 7fae1db9c700 1 -- start start 2021-11-03T18:33:03.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.553+0000 7fae1db9c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae180f49a0 0x7fae18106fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:33:03.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.553+0000 7fae1db9c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae180f5340 0x7fae181074b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:33:03.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.553+0000 7fae1db9c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae18107a60 con 0x7fae180f49a0 2021-11-03T18:33:03.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.553+0000 7fae1db9c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae18107ba0 con 0x7fae180f5340 2021-11-03T18:33:03.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.554+0000 7fae177fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae180f49a0 0x7fae18106fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:33:03.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.554+0000 7fae177fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae180f49a0 0x7fae18106fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35500/0 (socket says 172.21.15.27:35500) 2021-11-03T18:33:03.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.554+0000 7fae177fe700 1 -- 172.21.15.27:0/2369438603 learned_addr learned my addr 172.21.15.27:0/2369438603 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:33:03.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.554+0000 7fae16ffd700 1 --2- 172.21.15.27:0/2369438603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae180f5340 0x7fae181074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:33:03.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.554+0000 7fae177fe700 1 -- 172.21.15.27:0/2369438603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae180f5340 msgr2=0x7fae181074b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:33:03.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.554+0000 7fae177fe700 1 --2- 172.21.15.27:0/2369438603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae180f5340 0x7fae181074b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:03.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.554+0000 7fae177fe700 1 -- 172.21.15.27:0/2369438603 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae08005040 con 0x7fae180f49a0 2021-11-03T18:33:03.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.554+0000 7fae16ffd700 1 --2- 172.21.15.27:0/2369438603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae180f5340 0x7fae181074b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T18:33:03.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.554+0000 7fae177fe700 1 --2- 172.21.15.27:0/2369438603 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae180f49a0 0x7fae18106fc0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 rx=0x7fae08013a70 tx=0x7fae08009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:33:03.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.554+0000 7fadfffff700 1 -- 172.21.15.27:0/2369438603 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae08018070 con 0x7fae180f49a0 2021-11-03T18:33:03.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.554+0000 7fae1db9c700 1 -- 172.21.15.27:0/2369438603 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae181084e0 con 0x7fae180f49a0 2021-11-03T18:33:03.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.554+0000 7fadfffff700 1 -- 172.21.15.27:0/2369438603 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fae080097d0 con 0x7fae180f49a0 2021-11-03T18:33:03.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.554+0000 7fadfffff700 1 -- 172.21.15.27:0/2369438603 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae0801c3c0 con 0x7fae180f49a0 2021-11-03T18:33:03.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.554+0000 7fae1db9c700 1 -- 172.21.15.27:0/2369438603 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae18108a40 con 0x7fae180f49a0 2021-11-03T18:33:03.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.555+0000 7fae1db9c700 1 -- 172.21.15.27:0/2369438603 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae18045ad0 con 0x7fae180f49a0 2021-11-03T18:33:03.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.555+0000 7fadfffff700 1 -- 172.21.15.27:0/2369438603 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fae08007ba0 con 0x7fae180f49a0 2021-11-03T18:33:03.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.556+0000 7fadfffff700 1 --2- 172.21.15.27:0/2369438603 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fae000640b0 0x7fae00066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:33:03.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.556+0000 7fadfffff700 1 -- 172.21.15.27:0/2369438603 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fae08082470 con 0x7fae180f49a0 2021-11-03T18:33:03.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.556+0000 7fae16ffd700 1 --2- 172.21.15.27:0/2369438603 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fae000640b0 0x7fae00066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:33:03.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.556+0000 7fae16ffd700 1 --2- 172.21.15.27:0/2369438603 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fae000640b0 0x7fae00066570 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7fae0c009fd0 tx=0x7fae0c00b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:33:03.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.560+0000 7fadfffff700 1 -- 172.21.15.27:0/2369438603 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fae080547d0 con 0x7fae180f49a0 2021-11-03T18:33:03.733 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.735+0000 7fae1db9c700 1 -- 172.21.15.27:0/2369438603 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fae180feff0 con 0x7fae000640b0 2021-11-03T18:33:03.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.749+0000 7fadfffff700 1 -- 172.21.15.27:0/2369438603 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fae180feff0 con 0x7fae000640b0 2021-11-03T18:33:03.749 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:33:03.750 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (84m) 3m ago 85m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:33:03.750 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (76m) 3m ago 85m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:33:03.750 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (76m) 2m ago 85m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:33:03.751 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (84m) 3m ago 85m 42.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:33:03.752 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (78m) 3m ago 80m 26.7M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:33:03.752 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (81m) 3m ago 81m 3850M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:33:03.752 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (76m) 2m ago 81m 26.3M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:33:03.752 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (78m) 2m ago 81m 27.4M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:33:03.753 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (78m) 3m ago 87m 608M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:33:03.753 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (77m) 2m ago 85m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:33:03.753 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (77m) 3m ago 87m 350M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:33:03.753 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (76m) 2m ago 84m 160M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:33:03.754 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (85m) 3m ago 85m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:33:03.754 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (84m) 2m ago 84m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:33:03.754 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (75m) 3m ago 84m 413M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:33:03.754 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (75m) 3m ago 83m 450M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:33:03.754 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (74m) 3m ago 83m 338M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:33:03.755 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (74m) 2m ago 83m 479M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:33:03.755 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (74m) 2m ago 82m 378M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:33:03.755 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (73m) 2m ago 82m 388M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:33:03.755 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (84m) 3m ago 85m 64.4M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:33:03.756 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.751+0000 7fae1db9c700 1 -- 172.21.15.27:0/2369438603 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fae000640b0 msgr2=0x7fae00066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:33:03.756 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.751+0000 7fae1db9c700 1 --2- 172.21.15.27:0/2369438603 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fae000640b0 0x7fae00066570 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7fae0c009fd0 tx=0x7fae0c00b040).stop 2021-11-03T18:33:03.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.751+0000 7fae1db9c700 1 -- 172.21.15.27:0/2369438603 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae180f49a0 msgr2=0x7fae18106fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:33:03.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.751+0000 7fae1db9c700 1 --2- 172.21.15.27:0/2369438603 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae180f49a0 0x7fae18106fc0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 rx=0x7fae08013a70 tx=0x7fae08009b30).stop 2021-11-03T18:33:03.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.751+0000 7fae1db9c700 1 -- 172.21.15.27:0/2369438603 shutdown_connections 2021-11-03T18:33:03.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.751+0000 7fae1db9c700 1 --2- 172.21.15.27:0/2369438603 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fae000640b0 0x7fae00066570 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:03.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.751+0000 7fae1db9c700 1 --2- 172.21.15.27:0/2369438603 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae180f49a0 0x7fae18106fc0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:03.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.751+0000 7fae1db9c700 1 --2- 172.21.15.27:0/2369438603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae180f5340 0x7fae181074b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:03.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.751+0000 7fae1db9c700 1 -- 172.21.15.27:0/2369438603 >> 172.21.15.27:0/2369438603 conn(0x7fae180f04c0 msgr2=0x7fae180fb510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:33:03.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.751+0000 7fae1db9c700 1 -- 172.21.15.27:0/2369438603 shutdown_connections 2021-11-03T18:33:03.759 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.751+0000 7fae1db9c700 1 -- 172.21.15.27:0/2369438603 wait complete. 2021-11-03T18:33:03.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.850+0000 7fa621df7700 1 -- 172.21.15.27:0/663126462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa61c06a350 msgr2=0x7fa61c06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:33:03.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.850+0000 7fa621df7700 1 --2- 172.21.15.27:0/663126462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa61c06a350 0x7fa61c06a7d0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 rx=0x7fa604004660 tx=0x7fa604009b30).stop 2021-11-03T18:33:03.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.850+0000 7fa621df7700 1 -- 172.21.15.27:0/663126462 shutdown_connections 2021-11-03T18:33:03.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.850+0000 7fa621df7700 1 --2- 172.21.15.27:0/663126462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa61c06a350 0x7fa61c06a7d0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:03.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.850+0000 7fa621df7700 1 --2- 172.21.15.27:0/663126462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa61c06b8c0 0x7fa61c069e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:03.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.850+0000 7fa621df7700 1 -- 172.21.15.27:0/663126462 >> 172.21.15.27:0/663126462 conn(0x7fa61c0f1f00 msgr2=0x7fa61c0f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:33:03.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.850+0000 7fa621df7700 1 -- 172.21.15.27:0/663126462 shutdown_connections 2021-11-03T18:33:03.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.850+0000 7fa621df7700 1 -- 172.21.15.27:0/663126462 wait complete. 2021-11-03T18:33:03.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.851+0000 7fa621df7700 1 Processor -- start 2021-11-03T18:33:03.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.851+0000 7fa621df7700 1 -- start start 2021-11-03T18:33:03.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.851+0000 7fa621df7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa61c06a350 0x7fa61c109160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:33:03.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.851+0000 7fa621df7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa61c06b8c0 0x7fa61c109650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:33:03.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.851+0000 7fa621df7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c109c00 con 0x7fa61c06b8c0 2021-11-03T18:33:03.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.851+0000 7fa621df7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c109d40 con 0x7fa61c06a350 2021-11-03T18:33:03.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.851+0000 7fa61affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa61c06b8c0 0x7fa61c109650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:33:03.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.851+0000 7fa61affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa61c06b8c0 0x7fa61c109650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35512/0 (socket says 172.21.15.27:35512) 2021-11-03T18:33:03.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.851+0000 7fa61affd700 1 -- 172.21.15.27:0/2571688663 learned_addr learned my addr 172.21.15.27:0/2571688663 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:33:03.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.852+0000 7fa61b7fe700 1 --2- 172.21.15.27:0/2571688663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa61c06a350 0x7fa61c109160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:33:03.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.852+0000 7fa61affd700 1 -- 172.21.15.27:0/2571688663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa61c06a350 msgr2=0x7fa61c109160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:33:03.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.852+0000 7fa61affd700 1 --2- 172.21.15.27:0/2571688663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa61c06a350 0x7fa61c109160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:03.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.852+0000 7fa61affd700 1 -- 172.21.15.27:0/2571688663 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa604005040 con 0x7fa61c06b8c0 2021-11-03T18:33:03.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.852+0000 7fa61b7fe700 1 --2- 172.21.15.27:0/2571688663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa61c06a350 0x7fa61c109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T18:33:03.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.852+0000 7fa61affd700 1 --2- 172.21.15.27:0/2571688663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa61c06b8c0 0x7fa61c109650 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 rx=0x7fa6040045a0 tx=0x7fa604009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:33:03.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.852+0000 7fa613fff700 1 -- 172.21.15.27:0/2571688663 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa604018070 con 0x7fa61c06b8c0 2021-11-03T18:33:03.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.852+0000 7fa613fff700 1 -- 172.21.15.27:0/2571688663 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fa60401d8d0 con 0x7fa61c06b8c0 2021-11-03T18:33:03.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.852+0000 7fa621df7700 1 -- 172.21.15.27:0/2571688663 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa61c10a680 con 0x7fa61c06b8c0 2021-11-03T18:33:03.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.852+0000 7fa613fff700 1 -- 172.21.15.27:0/2571688663 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6040126c0 con 0x7fa61c06b8c0 2021-11-03T18:33:03.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.852+0000 7fa621df7700 1 -- 172.21.15.27:0/2571688663 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa61c10abe0 con 0x7fa61c06b8c0 2021-11-03T18:33:03.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.853+0000 7fa621df7700 1 -- 172.21.15.27:0/2571688663 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa61c045ad0 con 0x7fa61c06b8c0 2021-11-03T18:33:03.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.853+0000 7fa613fff700 1 -- 172.21.15.27:0/2571688663 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fa60401da40 con 0x7fa61c06b8c0 2021-11-03T18:33:03.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.854+0000 7fa613fff700 1 --2- 172.21.15.27:0/2571688663 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa608064090 0x7fa608066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:33:03.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.854+0000 7fa613fff700 1 -- 172.21.15.27:0/2571688663 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fa6040828e0 con 0x7fa61c06b8c0 2021-11-03T18:33:03.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.854+0000 7fa61b7fe700 1 --2- 172.21.15.27:0/2571688663 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa608064090 0x7fa608066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:33:03.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.854+0000 7fa61b7fe700 1 --2- 172.21.15.27:0/2571688663 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa608064090 0x7fa608066550 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7fa61c06b690 tx=0x7fa60c009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:33:03.860 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:03.858+0000 7fa613fff700 1 -- 172.21.15.27:0/2571688663 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fa604054c40 con 0x7fa61c06b8c0 2021-11-03T18:33:04.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.083+0000 7fa621df7700 1 -- 172.21.15.27:0/2571688663 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa61c027a00 con 0x7fa61c06b8c0 2021-11-03T18:33:04.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.084+0000 7fa613fff700 1 -- 172.21.15.27:0/2571688663 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fa604054830 con 0x7fa61c06b8c0 2021-11-03T18:33:04.083 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:33:04.083 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:33:04.083 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:33:04.083 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:33:04.084 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:33:04.084 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:33:04.084 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:33:04.084 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:33:04.084 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:33:04.085 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:33:04.085 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:33:04.085 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:33:04.085 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:33:04.085 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:33:04.086 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:33:04.086 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:33:04.086 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:33:04.086 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:33:04.087 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.086+0000 7fa621df7700 1 -- 172.21.15.27:0/2571688663 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa608064090 msgr2=0x7fa608066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:33:04.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.086+0000 7fa621df7700 1 --2- 172.21.15.27:0/2571688663 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa608064090 0x7fa608066550 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 rx=0x7fa61c06b690 tx=0x7fa60c009040).stop 2021-11-03T18:33:04.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.086+0000 7fa621df7700 1 -- 172.21.15.27:0/2571688663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa61c06b8c0 msgr2=0x7fa61c109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:33:04.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.086+0000 7fa621df7700 1 --2- 172.21.15.27:0/2571688663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa61c06b8c0 0x7fa61c109650 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 rx=0x7fa6040045a0 tx=0x7fa604009b30).stop 2021-11-03T18:33:04.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.086+0000 7fa621df7700 1 -- 172.21.15.27:0/2571688663 shutdown_connections 2021-11-03T18:33:04.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.086+0000 7fa621df7700 1 --2- 172.21.15.27:0/2571688663 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa608064090 0x7fa608066550 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:04.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.086+0000 7fa621df7700 1 --2- 172.21.15.27:0/2571688663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa61c06a350 0x7fa61c109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:04.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.086+0000 7fa621df7700 1 --2- 172.21.15.27:0/2571688663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa61c06b8c0 0x7fa61c109650 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:04.090 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.086+0000 7fa621df7700 1 -- 172.21.15.27:0/2571688663 >> 172.21.15.27:0/2571688663 conn(0x7fa61c0f1f00 msgr2=0x7fa61c0fcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:33:04.090 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.086+0000 7fa621df7700 1 -- 172.21.15.27:0/2571688663 shutdown_connections 2021-11-03T18:33:04.090 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.086+0000 7fa621df7700 1 -- 172.21.15.27:0/2571688663 wait complete. 2021-11-03T18:33:04.190 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.190+0000 7f7f06d6f700 1 -- 172.21.15.27:0/113458283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f000f6ff0 msgr2=0x7f7f000f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:33:04.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.190+0000 7f7f06d6f700 1 --2- 172.21.15.27:0/113458283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f000f6ff0 0x7f7f000f7a20 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 rx=0x7f7eec004660 tx=0x7f7eec009b30).stop 2021-11-03T18:33:04.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.191+0000 7f7f06d6f700 1 -- 172.21.15.27:0/113458283 shutdown_connections 2021-11-03T18:33:04.191 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.191+0000 7f7f06d6f700 1 --2- 172.21.15.27:0/113458283 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f000f6ff0 0x7f7f000f7a20 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:04.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.191+0000 7f7f06d6f700 1 --2- 172.21.15.27:0/113458283 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f000f66e0 0x7f7f000f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:04.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.191+0000 7f7f06d6f700 1 -- 172.21.15.27:0/113458283 >> 172.21.15.27:0/113458283 conn(0x7f7f000f1f00 msgr2=0x7f7f000f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:33:04.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.191+0000 7f7f06d6f700 1 -- 172.21.15.27:0/113458283 shutdown_connections 2021-11-03T18:33:04.192 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.191+0000 7f7f06d6f700 1 -- 172.21.15.27:0/113458283 wait complete. 2021-11-03T18:33:04.193 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.191+0000 7f7f06d6f700 1 Processor -- start 2021-11-03T18:33:04.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.191+0000 7f7f06d6f700 1 -- start start 2021-11-03T18:33:04.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.191+0000 7f7f06d6f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f000f66e0 0x7f7f00106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:33:04.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.191+0000 7f7f06d6f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f000f6ff0 0x7f7f00107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:33:04.194 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.191+0000 7f7f06d6f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f00107a10 con 0x7f7f000f6ff0 2021-11-03T18:33:04.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.191+0000 7f7f06d6f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f00107b50 con 0x7f7f000f66e0 2021-11-03T18:33:04.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.192+0000 7f7efffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f000f6ff0 0x7f7f00107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:33:04.195 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.192+0000 7f7efffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f000f6ff0 0x7f7f00107460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35522/0 (socket says 172.21.15.27:35522) 2021-11-03T18:33:04.196 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.192+0000 7f7efffff700 1 -- 172.21.15.27:0/1511578635 learned_addr learned my addr 172.21.15.27:0/1511578635 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:33:04.196 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.192+0000 7f7f04b0b700 1 --2- 172.21.15.27:0/1511578635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f000f66e0 0x7f7f00106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:33:04.197 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.192+0000 7f7efffff700 1 -- 172.21.15.27:0/1511578635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f000f66e0 msgr2=0x7f7f00106f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:33:04.197 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.192+0000 7f7efffff700 1 --2- 172.21.15.27:0/1511578635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f000f66e0 0x7f7f00106f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:04.197 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.192+0000 7f7efffff700 1 -- 172.21.15.27:0/1511578635 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eec005040 con 0x7f7f000f6ff0 2021-11-03T18:33:04.197 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.192+0000 7f7f04b0b700 1 --2- 172.21.15.27:0/1511578635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f000f66e0 0x7f7f00106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T18:33:04.198 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.192+0000 7f7efffff700 1 --2- 172.21.15.27:0/1511578635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f000f6ff0 0x7f7f00107460 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 rx=0x7f7eec009990 tx=0x7f7eec009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:33:04.198 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.192+0000 7f7efd7fa700 1 -- 172.21.15.27:0/1511578635 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7eec018070 con 0x7f7f000f6ff0 2021-11-03T18:33:04.198 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.192+0000 7f7efd7fa700 1 -- 172.21.15.27:0/1511578635 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f7eec007ca0 con 0x7f7f000f6ff0 2021-11-03T18:33:04.198 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.192+0000 7f7efd7fa700 1 -- 172.21.15.27:0/1511578635 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7eec01c620 con 0x7f7f000f6ff0 2021-11-03T18:33:04.199 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.192+0000 7f7f06d6f700 1 -- 172.21.15.27:0/1511578635 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f00108490 con 0x7f7f000f6ff0 2021-11-03T18:33:04.199 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.192+0000 7f7f06d6f700 1 -- 172.21.15.27:0/1511578635 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f001089f0 con 0x7f7f000f6ff0 2021-11-03T18:33:04.200 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.193+0000 7f7f06d6f700 1 -- 172.21.15.27:0/1511578635 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f00045ad0 con 0x7f7f000f6ff0 2021-11-03T18:33:04.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.194+0000 7f7efd7fa700 1 -- 172.21.15.27:0/1511578635 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f7eec01d960 con 0x7f7f000f6ff0 2021-11-03T18:33:04.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.194+0000 7f7efd7fa700 1 --2- 172.21.15.27:0/1511578635 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7ef00640b0 0x7f7ef0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:33:04.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.194+0000 7f7efd7fa700 1 -- 172.21.15.27:0/1511578635 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f7eec082470 con 0x7f7f000f6ff0 2021-11-03T18:33:04.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.194+0000 7f7f04b0b700 1 --2- 172.21.15.27:0/1511578635 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7ef00640b0 0x7f7ef0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:33:04.202 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.195+0000 7f7f04b0b700 1 --2- 172.21.15.27:0/1511578635 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7ef00640b0 0x7f7ef0066570 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7f7ef40041d0 tx=0x7f7ef4009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:33:04.202 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.198+0000 7f7efd7fa700 1 -- 172.21.15.27:0/1511578635 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f7eec054890 con 0x7f7f000f6ff0 2021-11-03T18:33:04.397 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.399+0000 7f7f06d6f700 1 -- 172.21.15.27:0/1511578635 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f7f00027a00 con 0x7f7f000f6ff0 2021-11-03T18:33:04.397 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.399+0000 7f7efd7fa700 1 -- 172.21.15.27:0/1511578635 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f7eec054480 con 0x7f7f000f6ff0 2021-11-03T18:33:04.398 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:33:04.398 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:33:04.398 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:33:04.398 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:33:04.399 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:33:04.399 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:33:04.399 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:33:04.399 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:33:04.400 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:33:04.400 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:33:04.400 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:33:04.400 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:33:04.400 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:33:04.401 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:33:04.401 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:33:04.401 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:33:04.401 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:33:04.402 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:33:04.402 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:33:04.402 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:33:04.402 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:33:04.402 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:33:04.403 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:33:04.403 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:33:04.403 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:33:04.403 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:33:04.404 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:33:04.404 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:33:04.404 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:33:04.404 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:33:04.404 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:33:04.405 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:33:04.405 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:33:04.405 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:33:04.405 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:33:04.405 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:33:04.406 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:33:04.406 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:33:04.406 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:33:04.407 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.401+0000 7f7f06d6f700 1 -- 172.21.15.27:0/1511578635 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7ef00640b0 msgr2=0x7f7ef0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:33:04.407 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.401+0000 7f7f06d6f700 1 --2- 172.21.15.27:0/1511578635 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7ef00640b0 0x7f7ef0066570 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7f7ef40041d0 tx=0x7f7ef4009040).stop 2021-11-03T18:33:04.408 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.401+0000 7f7f06d6f700 1 -- 172.21.15.27:0/1511578635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f000f6ff0 msgr2=0x7f7f00107460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:33:04.408 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.401+0000 7f7f06d6f700 1 --2- 172.21.15.27:0/1511578635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f000f6ff0 0x7f7f00107460 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 rx=0x7f7eec009990 tx=0x7f7eec009b30).stop 2021-11-03T18:33:04.408 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.401+0000 7f7f06d6f700 1 -- 172.21.15.27:0/1511578635 shutdown_connections 2021-11-03T18:33:04.408 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.401+0000 7f7f06d6f700 1 --2- 172.21.15.27:0/1511578635 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7ef00640b0 0x7f7ef0066570 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:04.409 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.401+0000 7f7f06d6f700 1 --2- 172.21.15.27:0/1511578635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7f000f66e0 0x7f7f00106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:04.409 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.401+0000 7f7f06d6f700 1 --2- 172.21.15.27:0/1511578635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7f000f6ff0 0x7f7f00107460 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:04.409 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.401+0000 7f7f06d6f700 1 -- 172.21.15.27:0/1511578635 >> 172.21.15.27:0/1511578635 conn(0x7f7f000f1f00 msgr2=0x7f7f000fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:33:04.409 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.401+0000 7f7f06d6f700 1 -- 172.21.15.27:0/1511578635 shutdown_connections 2021-11-03T18:33:04.410 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:04.401+0000 7f7f06d6f700 1 -- 172.21.15.27:0/1511578635 wait complete. 2021-11-03T18:33:04.410 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:33:04.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:04 smithi027 ceph-mon[97244]: pgmap v2354: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:04.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:04 smithi027 ceph-mon[97244]: from='client.36520 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:33:04.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:04 smithi027 ceph-mon[97244]: from='client.36524 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:33:04.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:04 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2571688663' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:04.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:04 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1511578635' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:33:04.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:04 smithi115 ceph-mon[65453]: pgmap v2354: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:04.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:04 smithi115 ceph-mon[65453]: from='client.36520 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:33:04.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:04 smithi115 ceph-mon[65453]: from='client.36524 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:33:04.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:04 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2571688663' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:04.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:04 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1511578635' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:33:05.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4627.470674 secs 2021-11-03T18:33:05.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4627.470674 secs 2021-11-03T18:33:06.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:06 smithi027 ceph-mon[97244]: pgmap v2355: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:06.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:33:06.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:33:06.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:33:06.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:06.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:06.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:06.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:06.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:06 smithi115 ceph-mon[65453]: pgmap v2355: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:06.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:33:06.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:33:06.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:33:06.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:06.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:06.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:06.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:07.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:07 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:33:07.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:07 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:33:08.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:08 smithi027 ceph-mon[97244]: pgmap v2356: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:08.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:08 smithi115 ceph-mon[65453]: pgmap v2356: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:10.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:10 smithi027 ceph-mon[97244]: pgmap v2357: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:10.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4632.470792 secs 2021-11-03T18:33:10.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:10 smithi115 ceph-mon[65453]: pgmap v2357: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:10.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4632.470792 secs 2021-11-03T18:33:12.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:12 smithi027 ceph-mon[97244]: pgmap v2358: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:12.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:12 smithi115 ceph-mon[65453]: pgmap v2358: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:14.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:14 smithi027 ceph-mon[97244]: pgmap v2359: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:14.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:14 smithi115 ceph-mon[65453]: pgmap v2359: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:15.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4637.470905 secs 2021-11-03T18:33:16.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4637.470905 secs 2021-11-03T18:33:16.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:16 smithi115 ceph-mon[65453]: pgmap v2360: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:16.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:33:16.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:33:16.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:33:16.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:16.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:16.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:16.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:17.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:16 smithi027 ceph-mon[97244]: pgmap v2360: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:17.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:33:17.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:33:17.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:33:17.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:17.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:17.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:17.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:17.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:17 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:33:18.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:17 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:33:18.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:18 smithi115 ceph-mon[65453]: pgmap v2361: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:19.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:18 smithi027 ceph-mon[97244]: pgmap v2361: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:20.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:20 smithi115 ceph-mon[65453]: pgmap v2362: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:20.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4642.471056 secs 2021-11-03T18:33:21.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:20 smithi027 ceph-mon[97244]: pgmap v2362: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:21.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4642.471056 secs 2021-11-03T18:33:22.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:22 smithi027 ceph-mon[97244]: pgmap v2363: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:22.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:22 smithi115 ceph-mon[65453]: pgmap v2363: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:24.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:24 smithi115 ceph-mon[65453]: pgmap v2364: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:25.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:24 smithi027 ceph-mon[97244]: pgmap v2364: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:25.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4647.471167 secs 2021-11-03T18:33:26.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4647.471167 secs 2021-11-03T18:33:26.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:26 smithi115 ceph-mon[65453]: pgmap v2365: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:26.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:33:26.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:33:26.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:33:26.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:26.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:26.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:26.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:27.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:26 smithi027 ceph-mon[97244]: pgmap v2365: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:27.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:33:27.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:33:27.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:33:27.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:27.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:27.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:27.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:27.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:27 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:33:28.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:27 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:33:28.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:28 smithi115 ceph-mon[65453]: pgmap v2366: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:29.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:28 smithi027 ceph-mon[97244]: pgmap v2366: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:30.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:30 smithi115 ceph-mon[65453]: pgmap v2367: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:30.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4652.471286 secs 2021-11-03T18:33:31.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:30 smithi027 ceph-mon[97244]: pgmap v2367: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:31.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4652.471286 secs 2021-11-03T18:33:32.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:32 smithi027 ceph-mon[97244]: pgmap v2368: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:32.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:32 smithi115 ceph-mon[65453]: pgmap v2368: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:34.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.514+0000 7ff22b5af700 1 -- 172.21.15.27:0/674692471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2240f6860 msgr2=0x7ff2240f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:33:34.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.514+0000 7ff22b5af700 1 --2- 172.21.15.27:0/674692471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2240f6860 0x7ff2240f6cc0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 rx=0x7ff218004660 tx=0x7ff218009b30).stop 2021-11-03T18:33:34.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.514+0000 7ff22b5af700 1 -- 172.21.15.27:0/674692471 shutdown_connections 2021-11-03T18:33:34.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.514+0000 7ff22b5af700 1 --2- 172.21.15.27:0/674692471 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2240f6860 0x7ff2240f6cc0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:34.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.514+0000 7ff22b5af700 1 --2- 172.21.15.27:0/674692471 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2240f55c0 0x7ff2240f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:34.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.514+0000 7ff22b5af700 1 -- 172.21.15.27:0/674692471 >> 172.21.15.27:0/674692471 conn(0x7ff2240f0d40 msgr2=0x7ff2240f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:33:34.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.514+0000 7ff22b5af700 1 -- 172.21.15.27:0/674692471 shutdown_connections 2021-11-03T18:33:34.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.514+0000 7ff22b5af700 1 -- 172.21.15.27:0/674692471 wait complete. 2021-11-03T18:33:34.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.515+0000 7ff22b5af700 1 Processor -- start 2021-11-03T18:33:34.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.515+0000 7ff22b5af700 1 -- start start 2021-11-03T18:33:34.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.515+0000 7ff22b5af700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2240f55c0 0x7ff22410c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:33:34.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.515+0000 7ff22b5af700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2240f6860 0x7ff22410e570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:33:34.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.515+0000 7ff22b5af700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2240602b0 con 0x7ff2240f6860 2021-11-03T18:33:34.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.515+0000 7ff22b5af700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2240603f0 con 0x7ff2240f55c0 2021-11-03T18:33:34.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.515+0000 7ff228b4a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2240f6860 0x7ff22410e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:33:34.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.515+0000 7ff228b4a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2240f6860 0x7ff22410e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35542/0 (socket says 172.21.15.27:35542) 2021-11-03T18:33:34.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.515+0000 7ff228b4a700 1 -- 172.21.15.27:0/2907786506 learned_addr learned my addr 172.21.15.27:0/2907786506 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:33:34.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.515+0000 7ff22934b700 1 --2- 172.21.15.27:0/2907786506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2240f55c0 0x7ff22410c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:33:34.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.515+0000 7ff228b4a700 1 -- 172.21.15.27:0/2907786506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2240f55c0 msgr2=0x7ff22410c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:33:34.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.515+0000 7ff228b4a700 1 --2- 172.21.15.27:0/2907786506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2240f55c0 0x7ff22410c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:34.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.515+0000 7ff228b4a700 1 -- 172.21.15.27:0/2907786506 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff218005040 con 0x7ff2240f6860 2021-11-03T18:33:34.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.515+0000 7ff22934b700 1 --2- 172.21.15.27:0/2907786506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2240f55c0 0x7ff22410c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:33:34.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.516+0000 7ff228b4a700 1 --2- 172.21.15.27:0/2907786506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2240f6860 0x7ff22410e570 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 rx=0x7ff218009990 tx=0x7ff218009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:33:34.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.516+0000 7ff221ffb700 1 -- 172.21.15.27:0/2907786506 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff218018070 con 0x7ff2240f6860 2021-11-03T18:33:34.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.516+0000 7ff22b5af700 1 -- 172.21.15.27:0/2907786506 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff22410ea60 con 0x7ff2240f6860 2021-11-03T18:33:34.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.516+0000 7ff221ffb700 1 -- 172.21.15.27:0/2907786506 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7ff218007ca0 con 0x7ff2240f6860 2021-11-03T18:33:34.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.516+0000 7ff221ffb700 1 -- 172.21.15.27:0/2907786506 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff21801c620 con 0x7ff2240f6860 2021-11-03T18:33:34.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.516+0000 7ff22b5af700 1 -- 172.21.15.27:0/2907786506 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff22410ee30 con 0x7ff2240f6860 2021-11-03T18:33:34.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.517+0000 7ff22b5af700 1 -- 172.21.15.27:0/2907786506 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2240f49b0 con 0x7ff2240f6860 2021-11-03T18:33:34.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.517+0000 7ff221ffb700 1 -- 172.21.15.27:0/2907786506 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7ff21801d960 con 0x7ff2240f6860 2021-11-03T18:33:34.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.518+0000 7ff221ffb700 1 --2- 172.21.15.27:0/2907786506 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff20c0640b0 0x7ff20c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:33:34.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.518+0000 7ff221ffb700 1 -- 172.21.15.27:0/2907786506 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7ff218082520 con 0x7ff2240f6860 2021-11-03T18:33:34.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.518+0000 7ff22934b700 1 --2- 172.21.15.27:0/2907786506 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff20c0640b0 0x7ff20c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:33:34.525 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.519+0000 7ff22934b700 1 --2- 172.21.15.27:0/2907786506 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff20c0640b0 0x7ff20c066570 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7ff224060040 tx=0x7ff21c009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:33:34.526 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.522+0000 7ff221ffb700 1 -- 172.21.15.27:0/2907786506 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7ff218054940 con 0x7ff2240f6860 2021-11-03T18:33:34.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.702+0000 7ff22b5af700 1 -- 172.21.15.27:0/2907786506 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2240fdd80 con 0x7ff20c0640b0 2021-11-03T18:33:34.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.704+0000 7ff221ffb700 1 -- 172.21.15.27:0/2907786506 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7ff2240fdd80 con 0x7ff20c0640b0 2021-11-03T18:33:34.704 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.705+0000 7ff22b5af700 1 -- 172.21.15.27:0/2907786506 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff20c0640b0 msgr2=0x7ff20c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:33:34.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.705+0000 7ff22b5af700 1 --2- 172.21.15.27:0/2907786506 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff20c0640b0 0x7ff20c066570 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 rx=0x7ff224060040 tx=0x7ff21c009040).stop 2021-11-03T18:33:34.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.705+0000 7ff22b5af700 1 -- 172.21.15.27:0/2907786506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2240f6860 msgr2=0x7ff22410e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:33:34.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.705+0000 7ff22b5af700 1 --2- 172.21.15.27:0/2907786506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2240f6860 0x7ff22410e570 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 rx=0x7ff218009990 tx=0x7ff218009b30).stop 2021-11-03T18:33:34.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.706+0000 7ff22b5af700 1 -- 172.21.15.27:0/2907786506 shutdown_connections 2021-11-03T18:33:34.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.706+0000 7ff22b5af700 1 --2- 172.21.15.27:0/2907786506 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff20c0640b0 0x7ff20c066570 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:34.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.706+0000 7ff22b5af700 1 --2- 172.21.15.27:0/2907786506 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2240f55c0 0x7ff22410c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:34.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.706+0000 7ff22b5af700 1 --2- 172.21.15.27:0/2907786506 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2240f6860 0x7ff22410e570 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:34.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.706+0000 7ff22b5af700 1 -- 172.21.15.27:0/2907786506 >> 172.21.15.27:0/2907786506 conn(0x7ff2240f0d40 msgr2=0x7ff2240fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:33:34.707 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.706+0000 7ff22b5af700 1 -- 172.21.15.27:0/2907786506 shutdown_connections 2021-11-03T18:33:34.707 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.706+0000 7ff22b5af700 1 -- 172.21.15.27:0/2907786506 wait complete. 2021-11-03T18:33:34.714 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:33:34.806 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.805+0000 7f7457fff700 1 -- 172.21.15.27:0/2010636537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74580f96a0 msgr2=0x7f74580fbaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:33:34.806 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.805+0000 7f7457fff700 1 --2- 172.21.15.27:0/2010636537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74580f96a0 0x7f74580fbaf0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 rx=0x7f744c004660 tx=0x7f744c009b30).stop 2021-11-03T18:33:34.807 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.805+0000 7f7457fff700 1 -- 172.21.15.27:0/2010636537 shutdown_connections 2021-11-03T18:33:34.807 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.805+0000 7f7457fff700 1 --2- 172.21.15.27:0/2010636537 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f74580f96a0 0x7f74580fbaf0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:34.807 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.805+0000 7f7457fff700 1 --2- 172.21.15.27:0/2010636537 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7458060310 0x7f74580f9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T18:33:34.808 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.805+0000 7f7457fff700 1 -- 172.21.15.27:0/2010636537 >> 172.21.15.27:0/2010636537 conn(0x7f74580f0ba0 msgr2=0x7f74580f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:33:34.808 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.805+0000 7f7457fff700 1 -- 172.21.15.27:0/2010636537 shutdown_connections 2021-11-03T18:33:34.808 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.805+0000 7f7457fff700 1 -- 172.21.15.27:0/2010636537 wait complete. 2021-11-03T18:33:34.808 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.806+0000 7f7457fff700 1 Processor -- start 2021-11-03T18:33:34.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.806+0000 7f7457fff700 1 -- start start 2021-11-03T18:33:34.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.806+0000 7f7457fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7458060310 0x7f7458106dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:33:34.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.806+0000 7f7457fff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74580f96a0 0x7f74581072b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:33:34.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.806+0000 7f7457fff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7458107940 con 0x7f7458060310 2021-11-03T18:33:34.809 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.806+0000 7f7457fff700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74581080f0 con 0x7f74580f96a0 2021-11-03T18:33:34.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.806+0000 7f7456ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7458060310 0x7f7458106dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:33:34.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.806+0000 7f7456ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7458060310 0x7f7458106dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35550/0 (socket says 172.21.15.27:35550) 2021-11-03T18:33:34.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.806+0000 7f7456ffd700 1 -- 172.21.15.27:0/3867484106 learned_addr learned my addr 172.21.15.27:0/3867484106 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:33:34.810 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.806+0000 7f74567fc700 1 --2- 172.21.15.27:0/3867484106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74580f96a0 0x7f74581072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:33:34.811 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.807+0000 7f7456ffd700 1 -- 172.21.15.27:0/3867484106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74580f96a0 msgr2=0x7f74581072b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:33:34.811 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.807+0000 7f7456ffd700 1 --2- 172.21.15.27:0/3867484106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74580f96a0 0x7f74581072b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:34.811 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.807+0000 7f7456ffd700 1 -- 172.21.15.27:0/3867484106 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f744c005040 con 0x7f7458060310 2021-11-03T18:33:34.811 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.807+0000 7f7456ffd700 1 --2- 172.21.15.27:0/3867484106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7458060310 0x7f7458106dc0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 rx=0x7f744800b410 tx=0x7f7448006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:33:34.811 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.807+0000 7f743f7fe700 1 -- 172.21.15.27:0/3867484106 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f74480125f0 con 0x7f7458060310 2021-11-03T18:33:34.812 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.807+0000 7f743f7fe700 1 -- 172.21.15.27:0/3867484106 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f744800be30 con 0x7f7458060310 2021-11-03T18:33:34.812 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.807+0000 7f7457fff700 1 -- 172.21.15.27:0/3867484106 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74581083d0 con 0x7f7458060310 2021-11-03T18:33:34.812 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.807+0000 7f743f7fe700 1 -- 172.21.15.27:0/3867484106 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7448002a70 con 0x7f7458060310 2021-11-03T18:33:34.812 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.807+0000 7f7457fff700 1 -- 172.21.15.27:0/3867484106 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7458108990 con 0x7f7458060310 2021-11-03T18:33:34.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.808+0000 7f7457fff700 1 -- 172.21.15.27:0/3867484106 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7458059100 con 0x7f7458060310 2021-11-03T18:33:34.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.808+0000 7f743f7fe700 1 -- 172.21.15.27:0/3867484106 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f744800f030 con 0x7f7458060310 2021-11-03T18:33:34.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.809+0000 7f743f7fe700 1 --2- 172.21.15.27:0/3867484106 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7440064100 0x7f74400665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:33:34.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.809+0000 7f743f7fe700 1 -- 172.21.15.27:0/3867484106 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f7448080b20 con 0x7f7458060310 2021-11-03T18:33:34.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.809+0000 7f74567fc700 1 --2- 172.21.15.27:0/3867484106 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7440064100 0x7f74400665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:33:34.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.809+0000 7f74567fc700 1 --2- 172.21.15.27:0/3867484106 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7440064100 0x7f74400665c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f7458106b50 tx=0x7f744c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:33:34.815 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.813+0000 7f743f7fe700 1 -- 172.21.15.27:0/3867484106 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f7448052e80 con 0x7f7458060310 2021-11-03T18:33:34.984 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.985+0000 7f7457fff700 1 -- 172.21.15.27:0/3867484106 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f74580008d0 con 0x7f7440064100 2021-11-03T18:33:34.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:34 smithi115 ceph-mon[65453]: pgmap v2369: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:34.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:34.999+0000 7f743f7fe700 1 -- 172.21.15.27:0/3867484106 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f74580008d0 con 0x7f7440064100 2021-11-03T18:33:34.999 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:33:34.999 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (85m) 3m ago 86m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:33:34.999 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (76m) 3m ago 86m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:33:35.000 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (76m) 3m ago 85m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:33:35.000 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (84m) 3m ago 86m 42.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:33:35.001 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (78m) 3m ago 81m 26.7M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:33:35.001 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (81m) 3m ago 81m 3850M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:33:35.001 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (77m) 3m ago 81m 26.3M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:33:35.002 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (78m) 3m ago 81m 27.4M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:33:35.002 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (78m) 3m ago 87m 608M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:33:35.002 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (78m) 3m ago 85m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:33:35.002 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (77m) 3m ago 87m 350M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:33:35.003 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (77m) 3m ago 85m 160M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:33:35.003 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (86m) 3m ago 85m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:33:35.003 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (85m) 3m ago 85m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:33:35.003 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (76m) 3m ago 84m 413M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:33:35.003 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (75m) 3m ago 84m 450M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:33:35.004 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (75m) 3m ago 84m 338M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:33:35.004 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (74m) 3m ago 83m 479M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:33:35.004 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (74m) 3m ago 83m 378M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:33:35.005 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (74m) 3m ago 83m 388M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:33:35.005 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (84m) 3m ago 85m 64.4M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:33:35.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.001+0000 7f7457fff700 1 -- 172.21.15.27:0/3867484106 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7440064100 msgr2=0x7f74400665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:33:35.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.001+0000 7f7457fff700 1 --2- 172.21.15.27:0/3867484106 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7440064100 0x7f74400665c0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f7458106b50 tx=0x7f744c009b30).stop 2021-11-03T18:33:35.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.001+0000 7f7457fff700 1 -- 172.21.15.27:0/3867484106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7458060310 msgr2=0x7f7458106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:33:35.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.001+0000 7f7457fff700 1 --2- 172.21.15.27:0/3867484106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7458060310 0x7f7458106dc0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 rx=0x7f744800b410 tx=0x7f7448006ef0).stop 2021-11-03T18:33:35.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.001+0000 7f7457fff700 1 -- 172.21.15.27:0/3867484106 shutdown_connections 2021-11-03T18:33:35.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.001+0000 7f7457fff700 1 --2- 172.21.15.27:0/3867484106 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7440064100 0x7f74400665c0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:35.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.001+0000 7f7457fff700 1 --2- 172.21.15.27:0/3867484106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7458060310 0x7f7458106dc0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:35.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.001+0000 7f7457fff700 1 --2- 172.21.15.27:0/3867484106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f74580f96a0 0x7f74581072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:35.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.001+0000 7f7457fff700 1 -- 172.21.15.27:0/3867484106 >> 172.21.15.27:0/3867484106 conn(0x7f74580f0ba0 msgr2=0x7f74580f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:33:35.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.001+0000 7f7457fff700 1 -- 172.21.15.27:0/3867484106 shutdown_connections 2021-11-03T18:33:35.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.001+0000 7f7457fff700 1 -- 172.21.15.27:0/3867484106 wait complete. 2021-11-03T18:33:35.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:34 smithi027 ceph-mon[97244]: pgmap v2369: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:35.102 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.100+0000 7f22f4b47700 1 -- 172.21.15.27:0/1449346985 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22f00f5ef0 msgr2=0x7f22f00f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:33:35.103 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.100+0000 7f22f4b47700 1 --2- 172.21.15.27:0/1449346985 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22f00f5ef0 0x7f22f00f6920 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 rx=0x7f22d8004660 tx=0x7f22d8009b30).stop 2021-11-03T18:33:35.103 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.100+0000 7f22f4b47700 1 -- 172.21.15.27:0/1449346985 shutdown_connections 2021-11-03T18:33:35.103 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.100+0000 7f22f4b47700 1 --2- 172.21.15.27:0/1449346985 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22f00f5ef0 0x7f22f00f6920 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:35.104 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.100+0000 7f22f4b47700 1 --2- 172.21.15.27:0/1449346985 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22f00f55e0 0x7f22f00f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:35.104 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.100+0000 7f22f4b47700 1 -- 172.21.15.27:0/1449346985 >> 172.21.15.27:0/1449346985 conn(0x7f22f00f0de0 msgr2=0x7f22f00f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:33:35.104 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.100+0000 7f22f4b47700 1 -- 172.21.15.27:0/1449346985 shutdown_connections 2021-11-03T18:33:35.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.100+0000 7f22f4b47700 1 -- 172.21.15.27:0/1449346985 wait complete. 2021-11-03T18:33:35.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.101+0000 7f22f4b47700 1 Processor -- start 2021-11-03T18:33:35.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.101+0000 7f22f4b47700 1 -- start start 2021-11-03T18:33:35.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.101+0000 7f22f4b47700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22f00f55e0 0x7f22f0109c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:33:35.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.101+0000 7f22f4b47700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22f00f5ef0 0x7f22f010c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:33:35.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.101+0000 7f22f4b47700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f00604e0 con 0x7f22f00f55e0 2021-11-03T18:33:35.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.101+0000 7f22f4b47700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22f0060620 con 0x7f22f00f5ef0 2021-11-03T18:33:35.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.101+0000 7f22ee59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22f00f55e0 0x7f22f0109c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:33:35.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.101+0000 7f22ee59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22f00f55e0 0x7f22f0109c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35560/0 (socket says 172.21.15.27:35560) 2021-11-03T18:33:35.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.101+0000 7f22ee59c700 1 -- 172.21.15.27:0/513775067 learned_addr learned my addr 172.21.15.27:0/513775067 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:33:35.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.101+0000 7f22edd9b700 1 --2- 172.21.15.27:0/513775067 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22f00f5ef0 0x7f22f010c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:33:35.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.102+0000 7f22ee59c700 1 -- 172.21.15.27:0/513775067 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22f00f5ef0 msgr2=0x7f22f010c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:33:35.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.102+0000 7f22ee59c700 1 --2- 172.21.15.27:0/513775067 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22f00f5ef0 0x7f22f010c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:35.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.102+0000 7f22ee59c700 1 -- 172.21.15.27:0/513775067 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d8005040 con 0x7f22f00f55e0 2021-11-03T18:33:35.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.102+0000 7f22ee59c700 1 --2- 172.21.15.27:0/513775067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22f00f55e0 0x7f22f0109c90 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 rx=0x7f22e80026d0 tx=0x7f22e8009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:33:35.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.102+0000 7f22e6ffd700 1 -- 172.21.15.27:0/513775067 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22e80079c0 con 0x7f22f00f55e0 2021-11-03T18:33:35.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.102+0000 7f22e6ffd700 1 -- 172.21.15.27:0/513775067 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f22e800f040 con 0x7f22f00f55e0 2021-11-03T18:33:35.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.102+0000 7f22e6ffd700 1 -- 172.21.15.27:0/513775067 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22e80148b0 con 0x7f22f00f55e0 2021-11-03T18:33:35.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.102+0000 7f22f4b47700 1 -- 172.21.15.27:0/513775067 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22f010c740 con 0x7f22f00f55e0 2021-11-03T18:33:35.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.102+0000 7f22f4b47700 1 -- 172.21.15.27:0/513775067 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22f010cd00 con 0x7f22f00f55e0 2021-11-03T18:33:35.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.103+0000 7f22f4b47700 1 -- 172.21.15.27:0/513775067 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22f0045ad0 con 0x7f22f00f55e0 2021-11-03T18:33:35.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.103+0000 7f22e6ffd700 1 -- 172.21.15.27:0/513775067 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f22e8007b20 con 0x7f22f00f55e0 2021-11-03T18:33:35.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.104+0000 7f22e6ffd700 1 --2- 172.21.15.27:0/513775067 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f22dc0640b0 0x7f22dc066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:33:35.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.104+0000 7f22e6ffd700 1 -- 172.21.15.27:0/513775067 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f22e8080730 con 0x7f22f00f55e0 2021-11-03T18:33:35.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.104+0000 7f22edd9b700 1 --2- 172.21.15.27:0/513775067 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f22dc0640b0 0x7f22dc066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:33:35.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.104+0000 7f22edd9b700 1 --2- 172.21.15.27:0/513775067 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f22dc0640b0 0x7f22dc066570 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7f22d8004630 tx=0x7f22d8009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:33:35.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.108+0000 7f22e6ffd700 1 -- 172.21.15.27:0/513775067 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f22e8052a90 con 0x7f22f00f55e0 2021-11-03T18:33:35.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.335+0000 7f22f4b47700 1 -- 172.21.15.27:0/513775067 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f22f00fded0 con 0x7f22f00f55e0 2021-11-03T18:33:35.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.335+0000 7f22e6ffd700 1 -- 172.21.15.27:0/513775067 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f22e8052680 con 0x7f22f00f55e0 2021-11-03T18:33:35.336 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:33:35.336 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:33:35.337 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:33:35.337 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:33:35.338 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:33:35.338 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:33:35.339 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:33:35.339 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:33:35.339 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:33:35.340 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:33:35.340 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:33:35.341 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:33:35.341 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:33:35.342 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:33:35.342 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:33:35.343 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:33:35.343 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:33:35.343 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:33:35.345 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.337+0000 7f22f4b47700 1 -- 172.21.15.27:0/513775067 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f22dc0640b0 msgr2=0x7f22dc066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:33:35.345 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.337+0000 7f22f4b47700 1 --2- 172.21.15.27:0/513775067 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f22dc0640b0 0x7f22dc066570 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 rx=0x7f22d8004630 tx=0x7f22d8009b30).stop 2021-11-03T18:33:35.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.337+0000 7f22f4b47700 1 -- 172.21.15.27:0/513775067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22f00f55e0 msgr2=0x7f22f0109c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:33:35.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.337+0000 7f22f4b47700 1 --2- 172.21.15.27:0/513775067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22f00f55e0 0x7f22f0109c90 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 rx=0x7f22e80026d0 tx=0x7f22e8009de0).stop 2021-11-03T18:33:35.347 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.337+0000 7f22f4b47700 1 -- 172.21.15.27:0/513775067 shutdown_connections 2021-11-03T18:33:35.347 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.337+0000 7f22f4b47700 1 --2- 172.21.15.27:0/513775067 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f22dc0640b0 0x7f22dc066570 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:35.348 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.337+0000 7f22f4b47700 1 --2- 172.21.15.27:0/513775067 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22f00f55e0 0x7f22f0109c90 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:35.348 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.337+0000 7f22f4b47700 1 --2- 172.21.15.27:0/513775067 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22f00f5ef0 0x7f22f010c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:35.348 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.337+0000 7f22f4b47700 1 -- 172.21.15.27:0/513775067 >> 172.21.15.27:0/513775067 conn(0x7f22f00f0de0 msgr2=0x7f22f00fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:33:35.349 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.337+0000 7f22f4b47700 1 -- 172.21.15.27:0/513775067 shutdown_connections 2021-11-03T18:33:35.349 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.337+0000 7f22f4b47700 1 -- 172.21.15.27:0/513775067 wait complete. 2021-11-03T18:33:35.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.437+0000 7ff42a20c700 1 -- 172.21.15.27:0/2892476218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff424060310 msgr2=0x7ff4240f92a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:33:35.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.437+0000 7ff42a20c700 1 --2- 172.21.15.27:0/2892476218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff424060310 0x7ff4240f92a0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 rx=0x7ff41c0099e0 tx=0x7ff41c004d10).stop 2021-11-03T18:33:35.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.437+0000 7ff42a20c700 1 -- 172.21.15.27:0/2892476218 shutdown_connections 2021-11-03T18:33:35.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.437+0000 7ff42a20c700 1 --2- 172.21.15.27:0/2892476218 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4240f9820 0x7ff4240fbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:35.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.437+0000 7ff42a20c700 1 --2- 172.21.15.27:0/2892476218 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff424060310 0x7ff4240f92a0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:35.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.437+0000 7ff42a20c700 1 -- 172.21.15.27:0/2892476218 >> 172.21.15.27:0/2892476218 conn(0x7ff4240f0d40 msgr2=0x7ff4240f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:33:35.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.437+0000 7ff42a20c700 1 -- 172.21.15.27:0/2892476218 shutdown_connections 2021-11-03T18:33:35.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.437+0000 7ff42a20c700 1 -- 172.21.15.27:0/2892476218 wait complete. 2021-11-03T18:33:35.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.437+0000 7ff42a20c700 1 Processor -- start 2021-11-03T18:33:35.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.437+0000 7ff42a20c700 1 -- start start 2021-11-03T18:33:35.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.438+0000 7ff42a20c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff424060310 0x7ff4240f6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:33:35.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.438+0000 7ff42a20c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4240f9820 0x7ff4240f51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:33:35.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.438+0000 7ff42a20c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4240f5700 con 0x7ff424060310 2021-11-03T18:33:35.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.438+0000 7ff42a20c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4240f5840 con 0x7ff4240f9820 2021-11-03T18:33:35.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.438+0000 7ff4237fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff424060310 0x7ff4240f6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:33:35.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.438+0000 7ff4237fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff424060310 0x7ff4240f6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35570/0 (socket says 172.21.15.27:35570) 2021-11-03T18:33:35.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.438+0000 7ff4237fe700 1 -- 172.21.15.27:0/3384643188 learned_addr learned my addr 172.21.15.27:0/3384643188 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:33:35.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.438+0000 7ff422ffd700 1 --2- 172.21.15.27:0/3384643188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4240f9820 0x7ff4240f51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:33:35.449 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.438+0000 7ff4237fe700 1 -- 172.21.15.27:0/3384643188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4240f9820 msgr2=0x7ff4240f51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:33:35.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.438+0000 7ff4237fe700 1 --2- 172.21.15.27:0/3384643188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4240f9820 0x7ff4240f51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:35.450 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.438+0000 7ff4237fe700 1 -- 172.21.15.27:0/3384643188 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff41c009640 con 0x7ff424060310 2021-11-03T18:33:35.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.438+0000 7ff422ffd700 1 --2- 172.21.15.27:0/3384643188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4240f9820 0x7ff4240f51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T18:33:35.451 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.438+0000 7ff4237fe700 1 --2- 172.21.15.27:0/3384643188 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff424060310 0x7ff4240f6d20 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 rx=0x7ff41c00b3b0 tx=0x7ff41c004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:33:35.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.438+0000 7ff41bfff700 1 -- 172.21.15.27:0/3384643188 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff41c00e030 con 0x7ff424060310 2021-11-03T18:33:35.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.438+0000 7ff42a20c700 1 -- 172.21.15.27:0/3384643188 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4240f5ac0 con 0x7ff424060310 2021-11-03T18:33:35.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.438+0000 7ff41bfff700 1 -- 172.21.15.27:0/3384643188 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7ff41c0049b0 con 0x7ff424060310 2021-11-03T18:33:35.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.438+0000 7ff41bfff700 1 -- 172.21.15.27:0/3384643188 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff41c018070 con 0x7ff424060310 2021-11-03T18:33:35.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.438+0000 7ff42a20c700 1 -- 172.21.15.27:0/3384643188 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4240f5fa0 con 0x7ff424060310 2021-11-03T18:33:35.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.439+0000 7ff42a20c700 1 -- 172.21.15.27:0/3384643188 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff424045ad0 con 0x7ff424060310 2021-11-03T18:33:35.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.440+0000 7ff41bfff700 1 -- 172.21.15.27:0/3384643188 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7ff41c007d40 con 0x7ff424060310 2021-11-03T18:33:35.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.440+0000 7ff41bfff700 1 --2- 172.21.15.27:0/3384643188 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff4100640b0 0x7ff410066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:33:35.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.440+0000 7ff41bfff700 1 -- 172.21.15.27:0/3384643188 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7ff41c082540 con 0x7ff424060310 2021-11-03T18:33:35.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.440+0000 7ff422ffd700 1 --2- 172.21.15.27:0/3384643188 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff4100640b0 0x7ff410066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:33:35.456 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.441+0000 7ff422ffd700 1 --2- 172.21.15.27:0/3384643188 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff4100640b0 0x7ff410066570 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7ff4240600e0 tx=0x7ff40c00b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:33:35.457 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.444+0000 7ff41bfff700 1 -- 172.21.15.27:0/3384643188 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7ff41c0548a0 con 0x7ff424060310 2021-11-03T18:33:35.648 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.649+0000 7ff42a20c700 1 -- 172.21.15.27:0/3384643188 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff424027a00 con 0x7ff424060310 2021-11-03T18:33:35.648 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.649+0000 7ff41bfff700 1 -- 172.21.15.27:0/3384643188 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7ff41c054490 con 0x7ff424060310 2021-11-03T18:33:35.649 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:33:35.649 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:33:35.649 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:33:35.650 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:33:35.650 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:33:35.650 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:33:35.650 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:33:35.651 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:33:35.651 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:33:35.651 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:33:35.651 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:33:35.652 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:33:35.652 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:33:35.652 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:33:35.652 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:33:35.653 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:33:35.653 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:33:35.653 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:33:35.654 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:33:35.655 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:33:35.655 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:33:35.655 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:33:35.655 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:33:35.656 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:33:35.656 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:33:35.656 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:33:35.657 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:33:35.657 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:33:35.657 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:33:35.657 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:33:35.658 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:33:35.658 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:33:35.658 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:33:35.658 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:33:35.658 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:33:35.659 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:33:35.659 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:33:35.659 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:33:35.659 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:33:35.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.651+0000 7ff42a20c700 1 -- 172.21.15.27:0/3384643188 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff4100640b0 msgr2=0x7ff410066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:33:35.661 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.651+0000 7ff42a20c700 1 --2- 172.21.15.27:0/3384643188 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff4100640b0 0x7ff410066570 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7ff4240600e0 tx=0x7ff40c00b040).stop 2021-11-03T18:33:35.661 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.651+0000 7ff42a20c700 1 -- 172.21.15.27:0/3384643188 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff424060310 msgr2=0x7ff4240f6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:33:35.661 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.651+0000 7ff42a20c700 1 --2- 172.21.15.27:0/3384643188 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff424060310 0x7ff4240f6d20 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 rx=0x7ff41c00b3b0 tx=0x7ff41c004d10).stop 2021-11-03T18:33:35.661 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.651+0000 7ff42a20c700 1 -- 172.21.15.27:0/3384643188 shutdown_connections 2021-11-03T18:33:35.662 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.651+0000 7ff42a20c700 1 --2- 172.21.15.27:0/3384643188 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff4100640b0 0x7ff410066570 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:35.662 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.651+0000 7ff42a20c700 1 --2- 172.21.15.27:0/3384643188 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff424060310 0x7ff4240f6d20 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:35.662 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.651+0000 7ff42a20c700 1 --2- 172.21.15.27:0/3384643188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff4240f9820 0x7ff4240f51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:33:35.662 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.651+0000 7ff42a20c700 1 -- 172.21.15.27:0/3384643188 >> 172.21.15.27:0/3384643188 conn(0x7ff4240f0d40 msgr2=0x7ff4240f3130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:33:35.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.651+0000 7ff42a20c700 1 -- 172.21.15.27:0/3384643188 shutdown_connections 2021-11-03T18:33:35.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:33:35.651+0000 7ff42a20c700 1 -- 172.21.15.27:0/3384643188 wait complete. 2021-11-03T18:33:35.663 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:33:35.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4657.471377 secs 2021-11-03T18:33:35.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:35 smithi115 ceph-mon[65453]: from='client.36536 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:33:35.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:35 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/513775067' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:36.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4657.471377 secs 2021-11-03T18:33:36.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:35 smithi027 ceph-mon[97244]: from='client.36536 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:33:36.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:35 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/513775067' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:36.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:36 smithi115 ceph-mon[65453]: from='client.36540 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:33:36.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:36 smithi115 ceph-mon[65453]: pgmap v2370: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:36.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:36 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3384643188' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:33:36.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:33:37.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:36 smithi027 ceph-mon[97244]: from='client.36540 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:33:37.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:36 smithi027 ceph-mon[97244]: pgmap v2370: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:37.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:36 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3384643188' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:33:37.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:33:37.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:37 smithi115 ceph-mon[65453]: poll took 60169.482 ms: 1 events 2021-11-03T18:33:37.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:33:37.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:33:37.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:37.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:37.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:37.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:37.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:37 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:33:38.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:37 smithi027 ceph-mon[97244]: poll took 60169.482 ms: 1 events 2021-11-03T18:33:38.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:33:38.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:33:38.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:38.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:38.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:38.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:38.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:37 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:33:38.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:38 smithi115 ceph-mon[65453]: pgmap v2371: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:39.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:38 smithi027 ceph-mon[97244]: pgmap v2371: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:40.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:40 smithi115 ceph-mon[65453]: pgmap v2372: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:40.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4662.471539 secs 2021-11-03T18:33:41.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:40 smithi027 ceph-mon[97244]: pgmap v2372: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:41.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4662.471539 secs 2021-11-03T18:33:42.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:42 smithi027 ceph-mon[97244]: pgmap v2373: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:42.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:42 smithi115 ceph-mon[65453]: pgmap v2373: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:44.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:44 smithi115 ceph-mon[65453]: pgmap v2374: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:45.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:44 smithi027 ceph-mon[97244]: pgmap v2374: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:45.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4667.471659 secs 2021-11-03T18:33:46.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4667.471659 secs 2021-11-03T18:33:46.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:46 smithi115 ceph-mon[65453]: pgmap v2375: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:46.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:33:47.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:46 smithi027 ceph-mon[97244]: pgmap v2375: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:47.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:33:47.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:33:47.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:33:47.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:47.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:47.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:47.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:47.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:47 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:33:48.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:33:48.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:33:48.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:48.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:48.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:48.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:48.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:47 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:33:48.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:48 smithi115 ceph-mon[65453]: pgmap v2376: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:49.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:48 smithi027 ceph-mon[97244]: pgmap v2376: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:50.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:50 smithi115 ceph-mon[65453]: pgmap v2377: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:50.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4672.471810 secs 2021-11-03T18:33:51.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:50 smithi027 ceph-mon[97244]: pgmap v2377: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:51.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4672.471810 secs 2021-11-03T18:33:52.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:52 smithi027 ceph-mon[97244]: pgmap v2378: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:52.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:52 smithi115 ceph-mon[65453]: pgmap v2378: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:54.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:54 smithi115 ceph-mon[65453]: pgmap v2379: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:55.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:54 smithi027 ceph-mon[97244]: pgmap v2379: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:55.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4677.471898 secs 2021-11-03T18:33:56.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4677.471898 secs 2021-11-03T18:33:56.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:56 smithi115 ceph-mon[65453]: pgmap v2380: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:56.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:33:56.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:33:57.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:56 smithi027 ceph-mon[97244]: pgmap v2380: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:57.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:33:57.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:33:57.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:33:57.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:33:57.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:33:57.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:57.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:57.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:57.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:57.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:57 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:33:58.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:33:58.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:33:58.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:33:58.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:58.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:58.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:58.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:33:58.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:57 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:33:58.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:33:58 smithi115 ceph-mon[65453]: pgmap v2381: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:33:59.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:33:58 smithi027 ceph-mon[97244]: pgmap v2381: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:00.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:00 smithi115 ceph-mon[65453]: pgmap v2382: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:00.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4682.472040 secs 2021-11-03T18:34:01.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:00 smithi027 ceph-mon[97244]: pgmap v2382: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:01.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4682.472040 secs 2021-11-03T18:34:02.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:02 smithi027 ceph-mon[97244]: pgmap v2383: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:02.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:02 smithi115 ceph-mon[65453]: pgmap v2383: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:04.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:04 smithi115 ceph-mon[65453]: pgmap v2384: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:05.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:04 smithi027 ceph-mon[97244]: pgmap v2384: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:05.767 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.766+0000 7f0ce5d44700 1 -- 172.21.15.27:0/1948582619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ce00f55c0 msgr2=0x7f0ce00f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:34:05.768 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.766+0000 7f0ce5d44700 1 --2- 172.21.15.27:0/1948582619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ce00f55c0 0x7f0ce00f59e0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 rx=0x7f0cd00099e0 tx=0x7f0cd0004d10).stop 2021-11-03T18:34:05.768 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.766+0000 7f0ce5d44700 1 -- 172.21.15.27:0/1948582619 shutdown_connections 2021-11-03T18:34:05.768 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.766+0000 7f0ce5d44700 1 --2- 172.21.15.27:0/1948582619 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ce00f6860 0x7f0ce00f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:05.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.766+0000 7f0ce5d44700 1 --2- 172.21.15.27:0/1948582619 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ce00f55c0 0x7f0ce00f59e0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:05.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.766+0000 7f0ce5d44700 1 -- 172.21.15.27:0/1948582619 >> 172.21.15.27:0/1948582619 conn(0x7f0ce00f0d40 msgr2=0x7f0ce00f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:34:05.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.766+0000 7f0ce5d44700 1 -- 172.21.15.27:0/1948582619 shutdown_connections 2021-11-03T18:34:05.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.766+0000 7f0ce5d44700 1 -- 172.21.15.27:0/1948582619 wait complete. 2021-11-03T18:34:05.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.767+0000 7f0ce5d44700 1 Processor -- start 2021-11-03T18:34:05.770 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.767+0000 7f0ce5d44700 1 -- start start 2021-11-03T18:34:05.770 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.767+0000 7f0ce5d44700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ce00f55c0 0x7f0ce0105f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:34:05.770 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.767+0000 7f0ce5d44700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ce00f6860 0x7f0ce0106450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:34:05.770 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.767+0000 7f0ce5d44700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce0106a00 con 0x7f0ce00f55c0 2021-11-03T18:34:05.771 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.767+0000 7f0ce5d44700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce0106b40 con 0x7f0ce00f6860 2021-11-03T18:34:05.771 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.767+0000 7f0cdf7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ce00f55c0 0x7f0ce0105f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:34:05.771 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.767+0000 7f0cdf7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ce00f55c0 0x7f0ce0105f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35588/0 (socket says 172.21.15.27:35588) 2021-11-03T18:34:05.771 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.768+0000 7f0cdf7fe700 1 -- 172.21.15.27:0/208485137 learned_addr learned my addr 172.21.15.27:0/208485137 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:34:05.771 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.768+0000 7f0cdeffd700 1 --2- 172.21.15.27:0/208485137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ce00f6860 0x7f0ce0106450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:34:05.772 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.768+0000 7f0cdf7fe700 1 -- 172.21.15.27:0/208485137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ce00f6860 msgr2=0x7f0ce0106450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:34:05.772 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.768+0000 7f0cdf7fe700 1 --2- 172.21.15.27:0/208485137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ce00f6860 0x7f0ce0106450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:05.772 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.768+0000 7f0cdf7fe700 1 -- 172.21.15.27:0/208485137 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cd0009640 con 0x7f0ce00f55c0 2021-11-03T18:34:05.772 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.768+0000 7f0cdf7fe700 1 --2- 172.21.15.27:0/208485137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ce00f55c0 0x7f0ce0105f60 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 rx=0x7f0cd000bb10 tx=0x7f0cd0004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:34:05.772 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.768+0000 7f0cc7fff700 1 -- 172.21.15.27:0/208485137 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0cd000e070 con 0x7f0ce00f55c0 2021-11-03T18:34:05.773 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.768+0000 7f0ce5d44700 1 -- 172.21.15.27:0/208485137 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce0107480 con 0x7f0ce00f55c0 2021-11-03T18:34:05.773 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.768+0000 7f0cc7fff700 1 -- 172.21.15.27:0/208485137 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f0cd0007d40 con 0x7f0ce00f55c0 2021-11-03T18:34:05.773 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.768+0000 7f0cc7fff700 1 -- 172.21.15.27:0/208485137 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0cd0018070 con 0x7f0ce00f55c0 2021-11-03T18:34:05.773 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.768+0000 7f0ce5d44700 1 -- 172.21.15.27:0/208485137 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce0107960 con 0x7f0ce00f55c0 2021-11-03T18:34:05.774 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.769+0000 7f0ce5d44700 1 -- 172.21.15.27:0/208485137 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ce0045ad0 con 0x7f0ce00f55c0 2021-11-03T18:34:05.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.770+0000 7f0cc7fff700 1 -- 172.21.15.27:0/208485137 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f0cd0024060 con 0x7f0ce00f55c0 2021-11-03T18:34:05.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.770+0000 7f0cc7fff700 1 --2- 172.21.15.27:0/208485137 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0cc8064090 0x7f0cc8066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:34:05.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.770+0000 7f0cc7fff700 1 -- 172.21.15.27:0/208485137 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f0cd0081f70 con 0x7f0ce00f55c0 2021-11-03T18:34:05.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.770+0000 7f0cdeffd700 1 --2- 172.21.15.27:0/208485137 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0cc8064090 0x7f0cc8066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:34:05.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.770+0000 7f0cdeffd700 1 --2- 172.21.15.27:0/208485137 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0cc8064090 0x7f0cc8066550 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f0cd4004ce0 tx=0x7f0cd400b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:34:05.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.774+0000 7f0cc7fff700 1 -- 172.21.15.27:0/208485137 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f0cd00aedf0 con 0x7f0ce00f55c0 2021-11-03T18:34:05.953 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.954+0000 7f0ce5d44700 1 -- 172.21.15.27:0/208485137 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ce00ffda0 con 0x7f0cc8064090 2021-11-03T18:34:05.954 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.956+0000 7f0cc7fff700 1 -- 172.21.15.27:0/208485137 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f0ce00ffda0 con 0x7f0cc8064090 2021-11-03T18:34:05.956 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.957+0000 7f0ce5d44700 1 -- 172.21.15.27:0/208485137 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0cc8064090 msgr2=0x7f0cc8066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:34:05.957 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.957+0000 7f0ce5d44700 1 --2- 172.21.15.27:0/208485137 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0cc8064090 0x7f0cc8066550 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f0cd4004ce0 tx=0x7f0cd400b040).stop 2021-11-03T18:34:05.957 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.957+0000 7f0ce5d44700 1 -- 172.21.15.27:0/208485137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ce00f55c0 msgr2=0x7f0ce0105f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:34:05.957 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.957+0000 7f0ce5d44700 1 --2- 172.21.15.27:0/208485137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ce00f55c0 0x7f0ce0105f60 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 rx=0x7f0cd000bb10 tx=0x7f0cd0004d10).stop 2021-11-03T18:34:05.957 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.957+0000 7f0ce5d44700 1 -- 172.21.15.27:0/208485137 shutdown_connections 2021-11-03T18:34:05.958 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.957+0000 7f0ce5d44700 1 --2- 172.21.15.27:0/208485137 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0cc8064090 0x7f0cc8066550 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:05.958 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.957+0000 7f0ce5d44700 1 --2- 172.21.15.27:0/208485137 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ce00f55c0 0x7f0ce0105f60 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:05.958 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.957+0000 7f0ce5d44700 1 --2- 172.21.15.27:0/208485137 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ce00f6860 0x7f0ce0106450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:05.958 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.957+0000 7f0ce5d44700 1 -- 172.21.15.27:0/208485137 >> 172.21.15.27:0/208485137 conn(0x7f0ce00f0d40 msgr2=0x7f0ce00fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:34:05.959 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.958+0000 7f0ce5d44700 1 -- 172.21.15.27:0/208485137 shutdown_connections 2021-11-03T18:34:05.959 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:05.958+0000 7f0ce5d44700 1 -- 172.21.15.27:0/208485137 wait complete. 2021-11-03T18:34:05.966 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:34:05.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4687.472180 secs 2021-11-03T18:34:06.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4687.472180 secs 2021-11-03T18:34:06.060 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.059+0000 7fdf0cfda700 1 -- 172.21.15.27:0/2618388647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf080f7980 msgr2=0x7fdf080f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:34:06.060 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.059+0000 7fdf0cfda700 1 --2- 172.21.15.27:0/2618388647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf080f7980 0x7fdf080f7de0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 rx=0x7fdef0004660 tx=0x7fdef0009b30).stop 2021-11-03T18:34:06.060 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.059+0000 7fdf0cfda700 1 -- 172.21.15.27:0/2618388647 shutdown_connections 2021-11-03T18:34:06.060 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.059+0000 7fdf0cfda700 1 --2- 172.21.15.27:0/2618388647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf080f7980 0x7fdf080f7de0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:06.061 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.059+0000 7fdf0cfda700 1 --2- 172.21.15.27:0/2618388647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf080f66e0 0x7fdf080f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:06.061 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.059+0000 7fdf0cfda700 1 -- 172.21.15.27:0/2618388647 >> 172.21.15.27:0/2618388647 conn(0x7fdf080f1e40 msgr2=0x7fdf080f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:34:06.061 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.059+0000 7fdf0cfda700 1 -- 172.21.15.27:0/2618388647 shutdown_connections 2021-11-03T18:34:06.062 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.059+0000 7fdf0cfda700 1 -- 172.21.15.27:0/2618388647 wait complete. 2021-11-03T18:34:06.062 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.059+0000 7fdf0cfda700 1 Processor -- start 2021-11-03T18:34:06.062 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.059+0000 7fdf0cfda700 1 -- start start 2021-11-03T18:34:06.063 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.060+0000 7fdf0cfda700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf080f66e0 0x7fdf08106f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:34:06.063 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.060+0000 7fdf0cfda700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf080f7980 0x7fdf08107400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:34:06.063 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.060+0000 7fdf0cfda700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf081079b0 con 0x7fdf080f7980 2021-11-03T18:34:06.063 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.060+0000 7fdf0cfda700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf08107af0 con 0x7fdf080f66e0 2021-11-03T18:34:06.063 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.060+0000 7fdf05d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf080f7980 0x7fdf08107400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:34:06.064 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.060+0000 7fdf05d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf080f7980 0x7fdf08107400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35600/0 (socket says 172.21.15.27:35600) 2021-11-03T18:34:06.064 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.060+0000 7fdf05d9b700 1 -- 172.21.15.27:0/2011319405 learned_addr learned my addr 172.21.15.27:0/2011319405 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:34:06.064 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.060+0000 7fdf0659c700 1 --2- 172.21.15.27:0/2011319405 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf080f66e0 0x7fdf08106f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:34:06.064 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.060+0000 7fdf05d9b700 1 -- 172.21.15.27:0/2011319405 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf080f66e0 msgr2=0x7fdf08106f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:34:06.065 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.060+0000 7fdf05d9b700 1 --2- 172.21.15.27:0/2011319405 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf080f66e0 0x7fdf08106f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:06.065 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.060+0000 7fdf05d9b700 1 -- 172.21.15.27:0/2011319405 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef0005040 con 0x7fdf080f7980 2021-11-03T18:34:06.065 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.060+0000 7fdf0659c700 1 --2- 172.21.15.27:0/2011319405 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf080f66e0 0x7fdf08106f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T18:34:06.065 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.060+0000 7fdf05d9b700 1 --2- 172.21.15.27:0/2011319405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf080f7980 0x7fdf08107400 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 rx=0x7fdef0000c00 tx=0x7fdef000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:34:06.065 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.060+0000 7fdefeffd700 1 -- 172.21.15.27:0/2011319405 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdef0019070 con 0x7fdf080f7980 2021-11-03T18:34:06.066 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.060+0000 7fdefeffd700 1 -- 172.21.15.27:0/2011319405 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fdef0007e70 con 0x7fdf080f7980 2021-11-03T18:34:06.066 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.060+0000 7fdf0cfda700 1 -- 172.21.15.27:0/2011319405 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf08108430 con 0x7fdf080f7980 2021-11-03T18:34:06.066 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.060+0000 7fdefeffd700 1 -- 172.21.15.27:0/2011319405 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdef0014d90 con 0x7fdf080f7980 2021-11-03T18:34:06.066 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.060+0000 7fdf0cfda700 1 -- 172.21.15.27:0/2011319405 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf08108990 con 0x7fdf080f7980 2021-11-03T18:34:06.068 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.061+0000 7fdf0cfda700 1 -- 172.21.15.27:0/2011319405 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf08045ad0 con 0x7fdf080f7980 2021-11-03T18:34:06.068 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.062+0000 7fdefeffd700 1 -- 172.21.15.27:0/2011319405 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fdef0025070 con 0x7fdf080f7980 2021-11-03T18:34:06.068 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.062+0000 7fdefeffd700 1 --2- 172.21.15.27:0/2011319405 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdef40640b0 0x7fdef4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:34:06.068 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.062+0000 7fdefeffd700 1 -- 172.21.15.27:0/2011319405 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fdef0081800 con 0x7fdf080f7980 2021-11-03T18:34:06.069 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.062+0000 7fdf0659c700 1 --2- 172.21.15.27:0/2011319405 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdef40640b0 0x7fdef4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:34:06.069 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.063+0000 7fdf0659c700 1 --2- 172.21.15.27:0/2011319405 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdef40640b0 0x7fdef4066570 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7fdef8009da0 tx=0x7fdef800b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:34:06.069 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.066+0000 7fdefeffd700 1 -- 172.21.15.27:0/2011319405 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fdef0053b00 con 0x7fdf080f7980 2021-11-03T18:34:06.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.238+0000 7fdf0cfda700 1 -- 172.21.15.27:0/2011319405 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf080fefb0 con 0x7fdef40640b0 2021-11-03T18:34:06.251 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.252+0000 7fdefeffd700 1 -- 172.21.15.27:0/2011319405 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fdf080fefb0 con 0x7fdef40640b0 2021-11-03T18:34:06.252 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:34:06.252 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (85m) 4m ago 86m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:34:06.252 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (77m) 4m ago 86m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:34:06.252 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (77m) 3m ago 86m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:34:06.253 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (85m) 4m ago 86m 42.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:34:06.253 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (79m) 4m ago 82m 26.7M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:34:06.253 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (82m) 4m ago 82m 3850M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:34:06.253 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (77m) 3m ago 82m 26.3M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:34:06.254 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (79m) 3m ago 82m 27.4M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:34:06.254 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (79m) 4m ago 88m 608M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:34:06.254 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (78m) 3m ago 86m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:34:06.254 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (78m) 4m ago 88m 350M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:34:06.255 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (77m) 3m ago 86m 160M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:34:06.255 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (86m) 4m ago 86m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:34:06.255 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (85m) 3m ago 85m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:34:06.255 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (76m) 4m ago 85m 413M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:34:06.255 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (76m) 4m ago 84m 450M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:34:06.256 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (75m) 4m ago 84m 338M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:34:06.256 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (75m) 3m ago 84m 479M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:34:06.256 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (75m) 3m ago 83m 378M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:34:06.256 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (74m) 3m ago 83m 388M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:34:06.257 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (85m) 4m ago 86m 64.4M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:34:06.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.254+0000 7fdf0cfda700 1 -- 172.21.15.27:0/2011319405 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdef40640b0 msgr2=0x7fdef4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:34:06.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.254+0000 7fdf0cfda700 1 --2- 172.21.15.27:0/2011319405 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdef40640b0 0x7fdef4066570 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 rx=0x7fdef8009da0 tx=0x7fdef800b040).stop 2021-11-03T18:34:06.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.254+0000 7fdf0cfda700 1 -- 172.21.15.27:0/2011319405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf080f7980 msgr2=0x7fdf08107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:34:06.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.254+0000 7fdf0cfda700 1 --2- 172.21.15.27:0/2011319405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf080f7980 0x7fdf08107400 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 rx=0x7fdef0000c00 tx=0x7fdef000b040).stop 2021-11-03T18:34:06.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.254+0000 7fdf0cfda700 1 -- 172.21.15.27:0/2011319405 shutdown_connections 2021-11-03T18:34:06.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.254+0000 7fdf0cfda700 1 --2- 172.21.15.27:0/2011319405 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdef40640b0 0x7fdef4066570 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:06.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.254+0000 7fdf0cfda700 1 --2- 172.21.15.27:0/2011319405 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf080f66e0 0x7fdf08106f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:06.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.254+0000 7fdf0cfda700 1 --2- 172.21.15.27:0/2011319405 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf080f7980 0x7fdf08107400 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:06.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.254+0000 7fdf0cfda700 1 -- 172.21.15.27:0/2011319405 >> 172.21.15.27:0/2011319405 conn(0x7fdf080f1e40 msgr2=0x7fdf080fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:34:06.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.254+0000 7fdf0cfda700 1 -- 172.21.15.27:0/2011319405 shutdown_connections 2021-11-03T18:34:06.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.254+0000 7fdf0cfda700 1 -- 172.21.15.27:0/2011319405 wait complete. 2021-11-03T18:34:06.355 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.354+0000 7fb2ab196700 1 -- 172.21.15.27:0/3532096166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2a40f5440 msgr2=0x7fb2a40f5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:34:06.355 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.354+0000 7fb2ab196700 1 --2- 172.21.15.27:0/3532096166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2a40f5440 0x7fb2a40f5860 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 rx=0x7fb2a0004660 tx=0x7fb2a0009b30).stop 2021-11-03T18:34:06.356 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.354+0000 7fb2ab196700 1 -- 172.21.15.27:0/3532096166 shutdown_connections 2021-11-03T18:34:06.356 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.354+0000 7fb2ab196700 1 --2- 172.21.15.27:0/3532096166 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2a40f66e0 0x7fb2a40f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:06.356 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.354+0000 7fb2ab196700 1 --2- 172.21.15.27:0/3532096166 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2a40f5440 0x7fb2a40f5860 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:06.357 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.354+0000 7fb2ab196700 1 -- 172.21.15.27:0/3532096166 >> 172.21.15.27:0/3532096166 conn(0x7fb2a40f0ba0 msgr2=0x7fb2a40f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:34:06.357 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.354+0000 7fb2ab196700 1 -- 172.21.15.27:0/3532096166 shutdown_connections 2021-11-03T18:34:06.357 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.354+0000 7fb2ab196700 1 -- 172.21.15.27:0/3532096166 wait complete. 2021-11-03T18:34:06.357 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.354+0000 7fb2ab196700 1 Processor -- start 2021-11-03T18:34:06.358 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.354+0000 7fb2ab196700 1 -- start start 2021-11-03T18:34:06.358 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.354+0000 7fb2ab196700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2a40f5440 0x7fb2a4106e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:34:06.358 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.354+0000 7fb2ab196700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2a40f66e0 0x7fb2a4107300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:34:06.358 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.354+0000 7fb2ab196700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a4107900 con 0x7fb2a40f66e0 2021-11-03T18:34:06.359 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.354+0000 7fb2ab196700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a4108110 con 0x7fb2a40f5440 2021-11-03T18:34:06.359 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.355+0000 7fb2a9993700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2a40f66e0 0x7fb2a4107300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:34:06.359 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.355+0000 7fb2a9993700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2a40f66e0 0x7fb2a4107300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35608/0 (socket says 172.21.15.27:35608) 2021-11-03T18:34:06.359 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.355+0000 7fb2a9993700 1 -- 172.21.15.27:0/1373603 learned_addr learned my addr 172.21.15.27:0/1373603 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:34:06.360 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.355+0000 7fb2aa194700 1 --2- 172.21.15.27:0/1373603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2a40f5440 0x7fb2a4106e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:34:06.360 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.355+0000 7fb2a9993700 1 -- 172.21.15.27:0/1373603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2a40f5440 msgr2=0x7fb2a4106e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:34:06.360 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.355+0000 7fb2a9993700 1 --2- 172.21.15.27:0/1373603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2a40f5440 0x7fb2a4106e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:06.360 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.355+0000 7fb2a9993700 1 -- 172.21.15.27:0/1373603 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a0005040 con 0x7fb2a40f66e0 2021-11-03T18:34:06.360 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.355+0000 7fb2a9993700 1 --2- 172.21.15.27:0/1373603 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2a40f66e0 0x7fb2a4107300 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 rx=0x7fb2940026d0 tx=0x7fb294009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:34:06.361 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.355+0000 7fb29affd700 1 -- 172.21.15.27:0/1373603 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2940079c0 con 0x7fb2a40f66e0 2021-11-03T18:34:06.361 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.355+0000 7fb29affd700 1 -- 172.21.15.27:0/1373603 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fb29400f040 con 0x7fb2a40f66e0 2021-11-03T18:34:06.361 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.355+0000 7fb2ab196700 1 -- 172.21.15.27:0/1373603 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2a4108390 con 0x7fb2a40f66e0 2021-11-03T18:34:06.361 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.355+0000 7fb29affd700 1 -- 172.21.15.27:0/1373603 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2940148b0 con 0x7fb2a40f66e0 2021-11-03T18:34:06.362 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.355+0000 7fb2ab196700 1 -- 172.21.15.27:0/1373603 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a4108950 con 0x7fb2a40f66e0 2021-11-03T18:34:06.362 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.356+0000 7fb2ab196700 1 -- 172.21.15.27:0/1373603 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2a4059100 con 0x7fb2a40f66e0 2021-11-03T18:34:06.363 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.357+0000 7fb29affd700 1 -- 172.21.15.27:0/1373603 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fb294007b20 con 0x7fb2a40f66e0 2021-11-03T18:34:06.363 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.357+0000 7fb29affd700 1 --2- 172.21.15.27:0/1373603 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb290064100 0x7fb2900665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:34:06.363 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.358+0000 7fb29affd700 1 -- 172.21.15.27:0/1373603 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fb294080b20 con 0x7fb2a40f66e0 2021-11-03T18:34:06.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.358+0000 7fb2aa194700 1 --2- 172.21.15.27:0/1373603 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb290064100 0x7fb2900665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:34:06.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.358+0000 7fb2aa194700 1 --2- 172.21.15.27:0/1373603 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb290064100 0x7fb2900665c0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7fb2a0013690 tx=0x7fb2a0009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:34:06.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.361+0000 7fb29affd700 1 -- 172.21.15.27:0/1373603 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fb294052e80 con 0x7fb2a40f66e0 2021-11-03T18:34:06.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.591+0000 7fb2ab196700 1 -- 172.21.15.27:0/1373603 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb2a40fed60 con 0x7fb2a40f66e0 2021-11-03T18:34:06.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.591+0000 7fb29affd700 1 -- 172.21.15.27:0/1373603 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fb294052a70 con 0x7fb2a40f66e0 2021-11-03T18:34:06.591 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:34:06.592 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:34:06.592 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:34:06.592 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:34:06.592 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:34:06.593 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:34:06.593 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:34:06.593 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:34:06.593 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:34:06.593 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:34:06.594 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:34:06.594 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:34:06.594 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:34:06.594 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:34:06.595 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:34:06.595 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:34:06.595 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:34:06.596 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:34:06.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.593+0000 7fb2ab196700 1 -- 172.21.15.27:0/1373603 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb290064100 msgr2=0x7fb2900665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:34:06.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.593+0000 7fb2ab196700 1 --2- 172.21.15.27:0/1373603 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb290064100 0x7fb2900665c0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 rx=0x7fb2a0013690 tx=0x7fb2a0009b30).stop 2021-11-03T18:34:06.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.593+0000 7fb2ab196700 1 -- 172.21.15.27:0/1373603 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2a40f66e0 msgr2=0x7fb2a4107300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:34:06.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.593+0000 7fb2ab196700 1 --2- 172.21.15.27:0/1373603 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2a40f66e0 0x7fb2a4107300 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 rx=0x7fb2940026d0 tx=0x7fb294009de0).stop 2021-11-03T18:34:06.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.593+0000 7fb2ab196700 1 -- 172.21.15.27:0/1373603 shutdown_connections 2021-11-03T18:34:06.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.593+0000 7fb2ab196700 1 --2- 172.21.15.27:0/1373603 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb290064100 0x7fb2900665c0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:06.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.593+0000 7fb2ab196700 1 --2- 172.21.15.27:0/1373603 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb2a40f5440 0x7fb2a4106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:06.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.593+0000 7fb2ab196700 1 --2- 172.21.15.27:0/1373603 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb2a40f66e0 0x7fb2a4107300 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:06.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.593+0000 7fb2ab196700 1 -- 172.21.15.27:0/1373603 >> 172.21.15.27:0/1373603 conn(0x7fb2a40f0ba0 msgr2=0x7fb2a40fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:34:06.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.593+0000 7fb2ab196700 1 -- 172.21.15.27:0/1373603 shutdown_connections 2021-11-03T18:34:06.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.593+0000 7fb2ab196700 1 -- 172.21.15.27:0/1373603 wait complete. 2021-11-03T18:34:06.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.696+0000 7f39115b0700 1 -- 172.21.15.27:0/3024055632 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f390c0f6530 msgr2=0x7f390c0f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:34:06.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.696+0000 7f39115b0700 1 --2- 172.21.15.27:0/3024055632 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f390c0f6530 0x7f390c0f6950 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 rx=0x7f38fc004660 tx=0x7f38fc009b30).stop 2021-11-03T18:34:06.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.696+0000 7f39115b0700 1 -- 172.21.15.27:0/3024055632 shutdown_connections 2021-11-03T18:34:06.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.696+0000 7f39115b0700 1 --2- 172.21.15.27:0/3024055632 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f390c0f77d0 0x7f390c0f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T18:34:06.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.696+0000 7f39115b0700 1 --2- 172.21.15.27:0/3024055632 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f390c0f6530 0x7f390c0f6950 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:06.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.696+0000 7f39115b0700 1 -- 172.21.15.27:0/3024055632 >> 172.21.15.27:0/3024055632 conn(0x7f390c0f1c90 msgr2=0x7f390c0f40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:34:06.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.696+0000 7f39115b0700 1 -- 172.21.15.27:0/3024055632 shutdown_connections 2021-11-03T18:34:06.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.696+0000 7f39115b0700 1 -- 172.21.15.27:0/3024055632 wait complete. 2021-11-03T18:34:06.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.697+0000 7f39115b0700 1 Processor -- start 2021-11-03T18:34:06.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.697+0000 7f39115b0700 1 -- start start 2021-11-03T18:34:06.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.697+0000 7f39115b0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f390c0f6530 0x7f390c06b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:34:06.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.697+0000 7f39115b0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f390c0f77d0 0x7f390c069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:34:06.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.697+0000 7f39115b0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390c06a410 con 0x7f390c0f6530 2021-11-03T18:34:06.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.697+0000 7f39115b0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f390c06a550 con 0x7f390c0f77d0 2021-11-03T18:34:06.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.697+0000 7f390bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f390c0f6530 0x7f390c06b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:34:06.704 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.697+0000 7f390bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f390c0f6530 0x7f390c06b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35616/0 (socket says 172.21.15.27:35616) 2021-11-03T18:34:06.704 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.697+0000 7f390bfff700 1 -- 172.21.15.27:0/3954389900 learned_addr learned my addr 172.21.15.27:0/3954389900 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:34:06.704 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.697+0000 7f390b7fe700 1 --2- 172.21.15.27:0/3954389900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f390c0f77d0 0x7f390c069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:34:06.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.698+0000 7f390bfff700 1 -- 172.21.15.27:0/3954389900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f390c0f77d0 msgr2=0x7f390c069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:34:06.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.698+0000 7f390bfff700 1 --2- 172.21.15.27:0/3954389900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f390c0f77d0 0x7f390c069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:06.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.698+0000 7f390bfff700 1 -- 172.21.15.27:0/3954389900 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38fc005040 con 0x7f390c0f6530 2021-11-03T18:34:06.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.698+0000 7f390b7fe700 1 --2- 172.21.15.27:0/3954389900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f390c0f77d0 0x7f390c069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:34:06.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.698+0000 7f390bfff700 1 --2- 172.21.15.27:0/3954389900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f390c0f6530 0x7f390c06b9a0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 rx=0x7f38fc000c00 tx=0x7f38fc009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:34:06.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.698+0000 7f3908ff9700 1 -- 172.21.15.27:0/3954389900 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38fc018070 con 0x7f390c0f6530 2021-11-03T18:34:06.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.698+0000 7f3908ff9700 1 -- 172.21.15.27:0/3954389900 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f38fc007d70 con 0x7f390c0f6530 2021-11-03T18:34:06.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.698+0000 7f3908ff9700 1 -- 172.21.15.27:0/3954389900 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38fc01c620 con 0x7f390c0f6530 2021-11-03T18:34:06.707 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.698+0000 7f39115b0700 1 -- 172.21.15.27:0/3954389900 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f390c06a7d0 con 0x7f390c0f6530 2021-11-03T18:34:06.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.698+0000 7f39115b0700 1 -- 172.21.15.27:0/3954389900 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f390c10ede0 con 0x7f390c0f6530 2021-11-03T18:34:06.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.699+0000 7f39115b0700 1 -- 172.21.15.27:0/3954389900 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f390c045ad0 con 0x7f390c0f6530 2021-11-03T18:34:06.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.700+0000 7f3908ff9700 1 -- 172.21.15.27:0/3954389900 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f38fc01d420 con 0x7f390c0f6530 2021-11-03T18:34:06.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.700+0000 7f3908ff9700 1 --2- 172.21.15.27:0/3954389900 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f38f4064100 0x7f38f40665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:34:06.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.700+0000 7f3908ff9700 1 -- 172.21.15.27:0/3954389900 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f38fc082560 con 0x7f390c0f6530 2021-11-03T18:34:06.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.700+0000 7f390b7fe700 1 --2- 172.21.15.27:0/3954389900 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f38f4064100 0x7f38f40665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:34:06.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.701+0000 7f390b7fe700 1 --2- 172.21.15.27:0/3954389900 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f38f4064100 0x7f38f40665c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7f390000b8d0 tx=0x7f3900008040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:34:06.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.705+0000 7f3908ff9700 1 -- 172.21.15.27:0/3954389900 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f38fc054980 con 0x7f390c0f6530 2021-11-03T18:34:06.909 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.910+0000 7f39115b0700 1 -- 172.21.15.27:0/3954389900 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f390c0fed70 con 0x7f390c0f6530 2021-11-03T18:34:06.909 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.910+0000 7f3908ff9700 1 -- 172.21.15.27:0/3954389900 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f38fc054570 con 0x7f390c0f6530 2021-11-03T18:34:06.910 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:34:06.910 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:34:06.910 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:34:06.911 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:34:06.911 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:34:06.911 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:34:06.911 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:34:06.912 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:34:06.912 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:34:06.912 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:34:06.912 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:34:06.913 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:34:06.913 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:34:06.913 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:34:06.913 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:34:06.913 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:34:06.914 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:34:06.914 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:34:06.914 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:34:06.914 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:34:06.915 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:34:06.915 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:34:06.915 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:34:06.915 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:34:06.915 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:34:06.916 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:34:06.916 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:34:06.916 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:34:06.916 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:34:06.917 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:34:06.917 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:34:06.917 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:34:06.917 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:34:06.917 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:34:06.918 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:34:06.918 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:34:06.918 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:34:06.918 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:34:06.918 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:34:06.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.912+0000 7f39115b0700 1 -- 172.21.15.27:0/3954389900 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f38f4064100 msgr2=0x7f38f40665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:34:06.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.912+0000 7f39115b0700 1 --2- 172.21.15.27:0/3954389900 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f38f4064100 0x7f38f40665c0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 rx=0x7f390000b8d0 tx=0x7f3900008040).stop 2021-11-03T18:34:06.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.912+0000 7f39115b0700 1 -- 172.21.15.27:0/3954389900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f390c0f6530 msgr2=0x7f390c06b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:34:06.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.912+0000 7f39115b0700 1 --2- 172.21.15.27:0/3954389900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f390c0f6530 0x7f390c06b9a0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 rx=0x7f38fc000c00 tx=0x7f38fc009b30).stop 2021-11-03T18:34:06.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.912+0000 7f39115b0700 1 -- 172.21.15.27:0/3954389900 shutdown_connections 2021-11-03T18:34:06.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.912+0000 7f39115b0700 1 --2- 172.21.15.27:0/3954389900 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f38f4064100 0x7f38f40665c0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:06.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.912+0000 7f39115b0700 1 --2- 172.21.15.27:0/3954389900 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f390c0f6530 0x7f390c06b9a0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:06.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.912+0000 7f39115b0700 1 --2- 172.21.15.27:0/3954389900 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f390c0f77d0 0x7f390c069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:06.922 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.912+0000 7f39115b0700 1 -- 172.21.15.27:0/3954389900 >> 172.21.15.27:0/3954389900 conn(0x7f390c0f1c90 msgr2=0x7f390c0fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:34:06.922 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.912+0000 7f39115b0700 1 -- 172.21.15.27:0/3954389900 shutdown_connections 2021-11-03T18:34:06.922 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:06.912+0000 7f39115b0700 1 -- 172.21.15.27:0/3954389900 wait complete. 2021-11-03T18:34:06.922 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:34:06.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:06 smithi115 ceph-mon[65453]: pgmap v2385: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:06.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:06 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1373603' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:07.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:06 smithi027 ceph-mon[97244]: pgmap v2385: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:07.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:06 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1373603' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:07.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:07 smithi115 ceph-mon[65453]: from='client.36552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:34:07.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:07 smithi115 ceph-mon[65453]: from='client.36556 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:34:07.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:34:07.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:34:07.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:34:07.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:07.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:07.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:07.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:07.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:07 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:34:07.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:07 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3954389900' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:34:08.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:07 smithi027 ceph-mon[97244]: from='client.36552 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:34:08.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:07 smithi027 ceph-mon[97244]: from='client.36556 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:34:08.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:34:08.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:34:08.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:34:08.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:08.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:08.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:08.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:08.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:07 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:34:08.026 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:07 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3954389900' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:34:08.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:08 smithi115 ceph-mon[65453]: pgmap v2386: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:09.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:08 smithi027 ceph-mon[97244]: pgmap v2386: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:10.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:10 smithi115 ceph-mon[65453]: pgmap v2387: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:10.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4692.472326 secs 2021-11-03T18:34:11.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:10 smithi027 ceph-mon[97244]: pgmap v2387: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:11.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4692.472326 secs 2021-11-03T18:34:12.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:12 smithi115 ceph-mon[65453]: pgmap v2388: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:13.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:12 smithi027 ceph-mon[97244]: pgmap v2388: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:14.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:14 smithi115 ceph-mon[65453]: pgmap v2389: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:15.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:14 smithi027 ceph-mon[97244]: pgmap v2389: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:15.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4697.472473 secs 2021-11-03T18:34:16.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4697.472473 secs 2021-11-03T18:34:16.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:16 smithi115 ceph-mon[65453]: pgmap v2390: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:16.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:34:17.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:16 smithi027 ceph-mon[97244]: pgmap v2390: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:17.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:34:17.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:34:17.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:34:17.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:17.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:17.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:17.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:17.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:17 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:34:18.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:34:18.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:34:18.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:18.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:18.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:18.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:18.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:17 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:34:18.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:18 smithi115 ceph-mon[65453]: pgmap v2391: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:19.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:18 smithi027 ceph-mon[97244]: pgmap v2391: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:20.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:20 smithi115 ceph-mon[65453]: pgmap v2392: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:20.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4702.472594 secs 2021-11-03T18:34:21.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:20 smithi027 ceph-mon[97244]: pgmap v2392: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:21.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4702.472594 secs 2021-11-03T18:34:22.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:22 smithi115 ceph-mon[65453]: pgmap v2393: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:23.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:22 smithi027 ceph-mon[97244]: pgmap v2393: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:24.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:24 smithi115 ceph-mon[65453]: pgmap v2394: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:25.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:24 smithi027 ceph-mon[97244]: pgmap v2394: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:25.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4707.472734 secs 2021-11-03T18:34:26.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4707.472734 secs 2021-11-03T18:34:26.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:26 smithi115 ceph-mon[65453]: pgmap v2395: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:27.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:26 smithi027 ceph-mon[97244]: pgmap v2395: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:27.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:34:27.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:34:27.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:34:27.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:27.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:27.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:27.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:27.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:27 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:34:28.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:34:28.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:34:28.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:34:28.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:28.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:28.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:28.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:28.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:27 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:34:29.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:28 smithi027 ceph-mon[97244]: pgmap v2396: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:29.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:28 smithi115 ceph-mon[65453]: pgmap v2396: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:31.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:30 smithi027 ceph-mon[97244]: pgmap v2397: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:31.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4712.472873 secs 2021-11-03T18:34:31.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:30 smithi115 ceph-mon[65453]: pgmap v2397: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:31.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4712.472873 secs 2021-11-03T18:34:33.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:32 smithi027 ceph-mon[97244]: pgmap v2398: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:33.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:32 smithi115 ceph-mon[65453]: pgmap v2398: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:35.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:34 smithi027 ceph-mon[97244]: pgmap v2399: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:35.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:34 smithi115 ceph-mon[65453]: pgmap v2399: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:36.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4717.472999 secs 2021-11-03T18:34:36.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4717.472999 secs 2021-11-03T18:34:37.020 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.021+0000 7ff195355700 1 -- 172.21.15.27:0/2431386746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1900f66e0 msgr2=0x7ff1900f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:34:37.020 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.021+0000 7ff195355700 1 --2- 172.21.15.27:0/2431386746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1900f66e0 0x7ff1900f6b00 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 rx=0x7ff180004660 tx=0x7ff180009b30).stop 2021-11-03T18:34:37.021 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.021+0000 7ff195355700 1 -- 172.21.15.27:0/2431386746 shutdown_connections 2021-11-03T18:34:37.021 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.021+0000 7ff195355700 1 --2- 172.21.15.27:0/2431386746 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1900f6ff0 0x7ff1900f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:37.021 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.021+0000 7ff195355700 1 --2- 172.21.15.27:0/2431386746 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1900f66e0 0x7ff1900f6b00 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:37.021 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.021+0000 7ff195355700 1 -- 172.21.15.27:0/2431386746 >> 172.21.15.27:0/2431386746 conn(0x7ff1900f1f00 msgr2=0x7ff1900f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:34:37.021 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.021+0000 7ff195355700 1 -- 172.21.15.27:0/2431386746 shutdown_connections 2021-11-03T18:34:37.022 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.021+0000 7ff195355700 1 -- 172.21.15.27:0/2431386746 wait complete. 2021-11-03T18:34:37.024 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.021+0000 7ff195355700 1 Processor -- start 2021-11-03T18:34:37.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.021+0000 7ff195355700 1 -- start start 2021-11-03T18:34:37.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.022+0000 7ff195355700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1900f66e0 0x7ff19006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:34:37.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.022+0000 7ff195355700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1900f6ff0 0x7ff190069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:34:37.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.022+0000 7ff195355700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19006a380 con 0x7ff1900f6ff0 2021-11-03T18:34:37.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.022+0000 7ff195355700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff19006a4c0 con 0x7ff1900f66e0 2021-11-03T18:34:37.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.022+0000 7ff18e7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1900f6ff0 0x7ff190069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:34:37.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.022+0000 7ff18e7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1900f6ff0 0x7ff190069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35636/0 (socket says 172.21.15.27:35636) 2021-11-03T18:34:37.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.022+0000 7ff18e7fc700 1 -- 172.21.15.27:0/125480663 learned_addr learned my addr 172.21.15.27:0/125480663 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:34:37.027 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.022+0000 7ff18effd700 1 --2- 172.21.15.27:0/125480663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1900f66e0 0x7ff19006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:34:37.027 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.022+0000 7ff18e7fc700 1 -- 172.21.15.27:0/125480663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1900f66e0 msgr2=0x7ff19006b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:34:37.027 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.022+0000 7ff18e7fc700 1 --2- 172.21.15.27:0/125480663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1900f66e0 0x7ff19006b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:37.027 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.022+0000 7ff18e7fc700 1 -- 172.21.15.27:0/125480663 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff180005040 con 0x7ff1900f6ff0 2021-11-03T18:34:37.027 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.022+0000 7ff18e7fc700 1 --2- 172.21.15.27:0/125480663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1900f6ff0 0x7ff190069e20 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 rx=0x7ff1780026d0 tx=0x7ff178009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:34:37.028 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.022+0000 7ff1877fe700 1 -- 172.21.15.27:0/125480663 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1780079c0 con 0x7ff1900f6ff0 2021-11-03T18:34:37.028 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.022+0000 7ff1877fe700 1 -- 172.21.15.27:0/125480663 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7ff17800f040 con 0x7ff1900f6ff0 2021-11-03T18:34:37.028 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.022+0000 7ff1877fe700 1 -- 172.21.15.27:0/125480663 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff1780148b0 con 0x7ff1900f6ff0 2021-11-03T18:34:37.029 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:36 smithi027 ceph-mon[97244]: pgmap v2400: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:37.029 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:34:37.030 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.023+0000 7ff195355700 1 -- 172.21.15.27:0/125480663 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff19006a7a0 con 0x7ff1900f6ff0 2021-11-03T18:34:37.030 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.023+0000 7ff195355700 1 -- 172.21.15.27:0/125480663 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff19010efc0 con 0x7ff1900f6ff0 2021-11-03T18:34:37.030 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.024+0000 7ff195355700 1 -- 172.21.15.27:0/125480663 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff190045ad0 con 0x7ff1900f6ff0 2021-11-03T18:34:37.031 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.024+0000 7ff1877fe700 1 -- 172.21.15.27:0/125480663 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7ff178007b20 con 0x7ff1900f6ff0 2021-11-03T18:34:37.031 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.025+0000 7ff1877fe700 1 --2- 172.21.15.27:0/125480663 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff17c0640b0 0x7ff17c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:34:37.031 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.025+0000 7ff1877fe700 1 -- 172.21.15.27:0/125480663 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7ff178080740 con 0x7ff1900f6ff0 2021-11-03T18:34:37.032 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.025+0000 7ff18effd700 1 --2- 172.21.15.27:0/125480663 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff17c0640b0 0x7ff17c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:34:37.032 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.025+0000 7ff18effd700 1 --2- 172.21.15.27:0/125480663 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff17c0640b0 0x7ff17c066570 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7ff180004630 tx=0x7ff180009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:34:37.032 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.029+0000 7ff1877fe700 1 -- 172.21.15.27:0/125480663 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7ff178052aa0 con 0x7ff1900f6ff0 2021-11-03T18:34:37.101 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:36 smithi115 ceph-mon[65453]: pgmap v2400: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:37.102 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:34:37.212 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.212+0000 7ff195355700 1 -- 172.21.15.27:0/125480663 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1900ff040 con 0x7ff17c0640b0 2021-11-03T18:34:37.212 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.214+0000 7ff1877fe700 1 -- 172.21.15.27:0/125480663 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7ff1900ff040 con 0x7ff17c0640b0 2021-11-03T18:34:37.215 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.215+0000 7ff195355700 1 -- 172.21.15.27:0/125480663 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff17c0640b0 msgr2=0x7ff17c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:34:37.215 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.216+0000 7ff195355700 1 --2- 172.21.15.27:0/125480663 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff17c0640b0 0x7ff17c066570 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 rx=0x7ff180004630 tx=0x7ff180009b30).stop 2021-11-03T18:34:37.215 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.216+0000 7ff195355700 1 -- 172.21.15.27:0/125480663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1900f6ff0 msgr2=0x7ff190069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:34:37.216 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.216+0000 7ff195355700 1 --2- 172.21.15.27:0/125480663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1900f6ff0 0x7ff190069e20 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 rx=0x7ff1780026d0 tx=0x7ff178009de0).stop 2021-11-03T18:34:37.216 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.216+0000 7ff195355700 1 -- 172.21.15.27:0/125480663 shutdown_connections 2021-11-03T18:34:37.216 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.216+0000 7ff195355700 1 --2- 172.21.15.27:0/125480663 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff17c0640b0 0x7ff17c066570 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:37.216 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.216+0000 7ff195355700 1 --2- 172.21.15.27:0/125480663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff1900f66e0 0x7ff19006b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:37.217 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.216+0000 7ff195355700 1 --2- 172.21.15.27:0/125480663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff1900f6ff0 0x7ff190069e20 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:37.217 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.216+0000 7ff195355700 1 -- 172.21.15.27:0/125480663 >> 172.21.15.27:0/125480663 conn(0x7ff1900f1f00 msgr2=0x7ff1900fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:34:37.217 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.216+0000 7ff195355700 1 -- 172.21.15.27:0/125480663 shutdown_connections 2021-11-03T18:34:37.218 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.216+0000 7ff195355700 1 -- 172.21.15.27:0/125480663 wait complete. 2021-11-03T18:34:37.225 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:34:37.315 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.316+0000 7f8496264700 1 -- 172.21.15.27:0/3386117576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84900f7970 msgr2=0x7f84900f7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:34:37.316 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.316+0000 7f8496264700 1 --2- 172.21.15.27:0/3386117576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84900f7970 0x7f84900f7dd0 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 rx=0x7f8480004660 tx=0x7f8480009b30).stop 2021-11-03T18:34:37.316 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.316+0000 7f8496264700 1 -- 172.21.15.27:0/3386117576 shutdown_connections 2021-11-03T18:34:37.316 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.316+0000 7f8496264700 1 --2- 172.21.15.27:0/3386117576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84900f7970 0x7f84900f7dd0 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:37.316 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.316+0000 7f8496264700 1 --2- 172.21.15.27:0/3386117576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84900f66d0 0x7f84900f6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:37.317 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.316+0000 7f8496264700 1 -- 172.21.15.27:0/3386117576 >> 172.21.15.27:0/3386117576 conn(0x7f84900f1e50 msgr2=0x7f84900f4270 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:34:37.317 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.316+0000 7f8496264700 1 -- 172.21.15.27:0/3386117576 shutdown_connections 2021-11-03T18:34:37.317 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.316+0000 7f8496264700 1 -- 172.21.15.27:0/3386117576 wait complete. 2021-11-03T18:34:37.318 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.317+0000 7f8496264700 1 Processor -- start 2021-11-03T18:34:37.320 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.317+0000 7f8496264700 1 -- start start 2021-11-03T18:34:37.320 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.317+0000 7f8496264700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84900f66d0 0x7f849006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:34:37.320 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.317+0000 7f8496264700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84900f7970 0x7f8490069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:34:37.321 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.317+0000 7f8496264700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849006a380 con 0x7f84900f7970 2021-11-03T18:34:37.321 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.317+0000 7f8496264700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f849006a4c0 con 0x7f84900f66d0 2021-11-03T18:34:37.321 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.317+0000 7f848f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84900f7970 0x7f8490069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:34:37.321 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.317+0000 7f848f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84900f7970 0x7f8490069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35646/0 (socket says 172.21.15.27:35646) 2021-11-03T18:34:37.322 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.317+0000 7f848f7fe700 1 -- 172.21.15.27:0/1962928392 learned_addr learned my addr 172.21.15.27:0/1962928392 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:34:37.322 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.317+0000 7f848ffff700 1 --2- 172.21.15.27:0/1962928392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84900f66d0 0x7f849006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:34:37.322 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.317+0000 7f848f7fe700 1 -- 172.21.15.27:0/1962928392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84900f66d0 msgr2=0x7f849006b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:34:37.322 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.317+0000 7f848f7fe700 1 --2- 172.21.15.27:0/1962928392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84900f66d0 0x7f849006b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:37.323 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.317+0000 7f848f7fe700 1 -- 172.21.15.27:0/1962928392 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8480005040 con 0x7f84900f7970 2021-11-03T18:34:37.323 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.317+0000 7f848f7fe700 1 --2- 172.21.15.27:0/1962928392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84900f7970 0x7f8490069e20 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 rx=0x7f8480000c00 tx=0x7f848000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:34:37.323 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.318+0000 7f848cff9700 1 -- 172.21.15.27:0/1962928392 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8480019070 con 0x7f84900f7970 2021-11-03T18:34:37.323 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.318+0000 7f8496264700 1 -- 172.21.15.27:0/1962928392 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f849006a740 con 0x7f84900f7970 2021-11-03T18:34:37.324 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.318+0000 7f848cff9700 1 -- 172.21.15.27:0/1962928392 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f8480007e70 con 0x7f84900f7970 2021-11-03T18:34:37.324 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.318+0000 7f848cff9700 1 -- 172.21.15.27:0/1962928392 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8480014ce0 con 0x7f84900f7970 2021-11-03T18:34:37.324 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.318+0000 7f8496264700 1 -- 172.21.15.27:0/1962928392 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f849006aca0 con 0x7f84900f7970 2021-11-03T18:34:37.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.319+0000 7f8496264700 1 -- 172.21.15.27:0/1962928392 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8490045ad0 con 0x7f84900f7970 2021-11-03T18:34:37.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.319+0000 7f848cff9700 1 -- 172.21.15.27:0/1962928392 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f8480025070 con 0x7f84900f7970 2021-11-03T18:34:37.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.319+0000 7f848cff9700 1 --2- 172.21.15.27:0/1962928392 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f84780640b0 0x7f8478066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:34:37.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.320+0000 7f848cff9700 1 -- 172.21.15.27:0/1962928392 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f8480081750 con 0x7f84900f7970 2021-11-03T18:34:37.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.320+0000 7f848ffff700 1 --2- 172.21.15.27:0/1962928392 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f84780640b0 0x7f8478066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:34:37.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.320+0000 7f848ffff700 1 --2- 172.21.15.27:0/1962928392 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f84780640b0 0x7f8478066570 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f849006b730 tx=0x7f848400b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:34:37.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.324+0000 7f848cff9700 1 -- 172.21.15.27:0/1962928392 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f8480053ab0 con 0x7f84900f7970 2021-11-03T18:34:37.496 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.497+0000 7f8496264700 1 -- 172.21.15.27:0/1962928392 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f84900fefa0 con 0x7f84780640b0 2021-11-03T18:34:37.510 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.511+0000 7f848cff9700 1 -- 172.21.15.27:0/1962928392 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f84900fefa0 con 0x7f84780640b0 2021-11-03T18:34:37.511 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:34:37.511 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (86m) 4m ago 87m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:34:37.511 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (77m) 4m ago 87m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:34:37.511 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (77m) 4m ago 86m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:34:37.512 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (86m) 4m ago 87m 42.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:34:37.512 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (79m) 4m ago 82m 26.7M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:34:37.512 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (82m) 4m ago 82m 3850M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:34:37.512 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (78m) 4m ago 82m 26.3M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:34:37.512 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (80m) 4m ago 82m 27.4M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:34:37.513 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (79m) 4m ago 88m 608M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:34:37.513 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (79m) 4m ago 86m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:34:37.513 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (78m) 4m ago 88m 350M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:34:37.513 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (78m) 4m ago 86m 160M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:34:37.514 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (87m) 4m ago 87m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:34:37.514 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (86m) 4m ago 86m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:34:37.514 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (77m) 4m ago 85m 413M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:34:37.514 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (77m) 4m ago 85m 450M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:34:37.514 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (76m) 4m ago 85m 338M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:34:37.515 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (76m) 4m ago 84m 479M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:34:37.515 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (75m) 4m ago 84m 378M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:34:37.515 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (75m) 4m ago 84m 388M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:34:37.515 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (85m) 4m ago 86m 64.4M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:34:37.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.513+0000 7f8496264700 1 -- 172.21.15.27:0/1962928392 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f84780640b0 msgr2=0x7f8478066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:34:37.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.513+0000 7f8496264700 1 --2- 172.21.15.27:0/1962928392 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f84780640b0 0x7f8478066570 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 rx=0x7f849006b730 tx=0x7f848400b040).stop 2021-11-03T18:34:37.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.513+0000 7f8496264700 1 -- 172.21.15.27:0/1962928392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84900f7970 msgr2=0x7f8490069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:34:37.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.513+0000 7f8496264700 1 --2- 172.21.15.27:0/1962928392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84900f7970 0x7f8490069e20 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 rx=0x7f8480000c00 tx=0x7f848000b040).stop 2021-11-03T18:34:37.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.513+0000 7f8496264700 1 -- 172.21.15.27:0/1962928392 shutdown_connections 2021-11-03T18:34:37.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.513+0000 7f8496264700 1 --2- 172.21.15.27:0/1962928392 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f84780640b0 0x7f8478066570 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:37.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.513+0000 7f8496264700 1 --2- 172.21.15.27:0/1962928392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84900f66d0 0x7f849006b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:37.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.513+0000 7f8496264700 1 --2- 172.21.15.27:0/1962928392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84900f7970 0x7f8490069e20 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:37.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.513+0000 7f8496264700 1 -- 172.21.15.27:0/1962928392 >> 172.21.15.27:0/1962928392 conn(0x7f84900f1e50 msgr2=0x7f84900fb3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:34:37.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.513+0000 7f8496264700 1 -- 172.21.15.27:0/1962928392 shutdown_connections 2021-11-03T18:34:37.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.513+0000 7f8496264700 1 -- 172.21.15.27:0/1962928392 wait complete. 2021-11-03T18:34:37.613 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.613+0000 7f404f96d700 1 -- 172.21.15.27:0/2126285780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40480f55d0 msgr2=0x7f40480f59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:34:37.614 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.613+0000 7f404f96d700 1 --2- 172.21.15.27:0/2126285780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40480f55d0 0x7f40480f59f0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 rx=0x7f40400099e0 tx=0x7f4040004d10).stop 2021-11-03T18:34:37.614 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.613+0000 7f404f96d700 1 -- 172.21.15.27:0/2126285780 shutdown_connections 2021-11-03T18:34:37.614 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.613+0000 7f404f96d700 1 --2- 172.21.15.27:0/2126285780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40480f6870 0x7f40480f6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:37.615 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.613+0000 7f404f96d700 1 --2- 172.21.15.27:0/2126285780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40480f55d0 0x7f40480f59f0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:37.615 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.613+0000 7f404f96d700 1 -- 172.21.15.27:0/2126285780 >> 172.21.15.27:0/2126285780 conn(0x7f40480f0d70 msgr2=0x7f40480f3190 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:34:37.615 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.613+0000 7f404f96d700 1 -- 172.21.15.27:0/2126285780 shutdown_connections 2021-11-03T18:34:37.615 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.613+0000 7f404f96d700 1 -- 172.21.15.27:0/2126285780 wait complete. 2021-11-03T18:34:37.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.613+0000 7f404f96d700 1 Processor -- start 2021-11-03T18:34:37.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.613+0000 7f404f96d700 1 -- start start 2021-11-03T18:34:37.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.613+0000 7f404f96d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40480f55d0 0x7f4048107080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:34:37.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.614+0000 7f404f96d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40480f6870 0x7f4048107570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:34:37.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.614+0000 7f404f96d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4048107b20 con 0x7f40480f55d0 2021-11-03T18:34:37.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.614+0000 7f404f96d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4048107c60 con 0x7f40480f6870 2021-11-03T18:34:37.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.614+0000 7f404d709700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40480f55d0 0x7f4048107080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:34:37.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.614+0000 7f404d709700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40480f55d0 0x7f4048107080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35654/0 (socket says 172.21.15.27:35654) 2021-11-03T18:34:37.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.614+0000 7f404d709700 1 -- 172.21.15.27:0/3406725725 learned_addr learned my addr 172.21.15.27:0/3406725725 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:34:37.618 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.614+0000 7f404cf08700 1 --2- 172.21.15.27:0/3406725725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40480f6870 0x7f4048107570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:34:37.618 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.614+0000 7f404d709700 1 -- 172.21.15.27:0/3406725725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40480f6870 msgr2=0x7f4048107570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:34:37.618 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.614+0000 7f404d709700 1 --2- 172.21.15.27:0/3406725725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40480f6870 0x7f4048107570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:37.618 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.614+0000 7f404d709700 1 -- 172.21.15.27:0/3406725725 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4040009640 con 0x7f40480f55d0 2021-11-03T18:34:37.619 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.614+0000 7f404d709700 1 --2- 172.21.15.27:0/3406725725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40480f55d0 0x7f4048107080 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 rx=0x7f404000bb10 tx=0x7f4040004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:34:37.619 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.614+0000 7f4045ffb700 1 -- 172.21.15.27:0/3406725725 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f404000e070 con 0x7f40480f55d0 2021-11-03T18:34:37.619 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.614+0000 7f4045ffb700 1 -- 172.21.15.27:0/3406725725 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f4040007df0 con 0x7f40480f55d0 2021-11-03T18:34:37.619 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.614+0000 7f404f96d700 1 -- 172.21.15.27:0/3406725725 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40481085a0 con 0x7f40480f55d0 2021-11-03T18:34:37.620 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.614+0000 7f4045ffb700 1 -- 172.21.15.27:0/3406725725 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4040018070 con 0x7f40480f55d0 2021-11-03T18:34:37.620 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.614+0000 7f404f96d700 1 -- 172.21.15.27:0/3406725725 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4048108a80 con 0x7f40480f55d0 2021-11-03T18:34:37.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.615+0000 7f404f96d700 1 -- 172.21.15.27:0/3406725725 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4048045ad0 con 0x7f40480f55d0 2021-11-03T18:34:37.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.616+0000 7f4045ffb700 1 -- 172.21.15.27:0/3406725725 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f4040024060 con 0x7f40480f55d0 2021-11-03T18:34:37.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.616+0000 7f4045ffb700 1 --2- 172.21.15.27:0/3406725725 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4030064090 0x7f4030066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:34:37.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.616+0000 7f4045ffb700 1 -- 172.21.15.27:0/3406725725 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f40400820c0 con 0x7f40480f55d0 2021-11-03T18:34:37.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.616+0000 7f404cf08700 1 --2- 172.21.15.27:0/3406725725 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4030064090 0x7f4030066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:34:37.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.617+0000 7f404cf08700 1 --2- 172.21.15.27:0/3406725725 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4030064090 0x7f4030066550 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7f403c0096c0 tx=0x7f403c009be0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:34:37.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.620+0000 7f4045ffb700 1 -- 172.21.15.27:0/3406725725 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f40400543c0 con 0x7f40480f55d0 2021-11-03T18:34:37.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.851+0000 7f404f96d700 1 -- 172.21.15.27:0/3406725725 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f40480fde90 con 0x7f40480f55d0 2021-11-03T18:34:37.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.851+0000 7f4045ffb700 1 -- 172.21.15.27:0/3406725725 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f40400222e0 con 0x7f40480f55d0 2021-11-03T18:34:37.851 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:34:37.851 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:34:37.852 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:34:37.852 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:34:37.852 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:34:37.852 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:34:37.853 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:34:37.853 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:34:37.853 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:34:37.853 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:34:37.853 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:34:37.854 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:34:37.854 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:34:37.854 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:34:37.854 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:34:37.855 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:34:37.855 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:34:37.855 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:34:37.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.853+0000 7f404f96d700 1 -- 172.21.15.27:0/3406725725 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4030064090 msgr2=0x7f4030066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:34:37.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.853+0000 7f404f96d700 1 --2- 172.21.15.27:0/3406725725 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4030064090 0x7f4030066550 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 rx=0x7f403c0096c0 tx=0x7f403c009be0).stop 2021-11-03T18:34:37.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.853+0000 7f404f96d700 1 -- 172.21.15.27:0/3406725725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40480f55d0 msgr2=0x7f4048107080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:34:37.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.853+0000 7f404f96d700 1 --2- 172.21.15.27:0/3406725725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40480f55d0 0x7f4048107080 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 rx=0x7f404000bb10 tx=0x7f4040004d10).stop 2021-11-03T18:34:37.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.853+0000 7f404f96d700 1 -- 172.21.15.27:0/3406725725 shutdown_connections 2021-11-03T18:34:37.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.853+0000 7f404f96d700 1 --2- 172.21.15.27:0/3406725725 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4030064090 0x7f4030066550 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:37.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.853+0000 7f404f96d700 1 --2- 172.21.15.27:0/3406725725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f40480f55d0 0x7f4048107080 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:37.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.853+0000 7f404f96d700 1 --2- 172.21.15.27:0/3406725725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f40480f6870 0x7f4048107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:37.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.853+0000 7f404f96d700 1 -- 172.21.15.27:0/3406725725 >> 172.21.15.27:0/3406725725 conn(0x7f40480f0d70 msgr2=0x7f404805f700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:34:37.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.853+0000 7f404f96d700 1 -- 172.21.15.27:0/3406725725 shutdown_connections 2021-11-03T18:34:37.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.853+0000 7f404f96d700 1 -- 172.21.15.27:0/3406725725 wait complete. 2021-11-03T18:34:37.954 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.953+0000 7f4fceab1700 1 -- 172.21.15.27:0/1637153275 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fc80f6ff0 msgr2=0x7f4fc80f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:34:37.954 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.953+0000 7f4fceab1700 1 --2- 172.21.15.27:0/1637153275 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fc80f6ff0 0x7f4fc80f7a20 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 rx=0x7f4fb4004660 tx=0x7f4fb4009b30).stop 2021-11-03T18:34:37.955 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.953+0000 7f4fceab1700 1 -- 172.21.15.27:0/1637153275 shutdown_connections 2021-11-03T18:34:37.955 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.953+0000 7f4fceab1700 1 --2- 172.21.15.27:0/1637153275 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fc80f6ff0 0x7f4fc80f7a20 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:37.955 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.953+0000 7f4fceab1700 1 --2- 172.21.15.27:0/1637153275 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4fc80f66e0 0x7f4fc80f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:37.955 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.953+0000 7f4fceab1700 1 -- 172.21.15.27:0/1637153275 >> 172.21.15.27:0/1637153275 conn(0x7f4fc80f1f00 msgr2=0x7f4fc80f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:34:37.956 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.953+0000 7f4fceab1700 1 -- 172.21.15.27:0/1637153275 shutdown_connections 2021-11-03T18:34:37.956 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.953+0000 7f4fceab1700 1 -- 172.21.15.27:0/1637153275 wait complete. 2021-11-03T18:34:37.956 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.954+0000 7f4fceab1700 1 Processor -- start 2021-11-03T18:34:37.956 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.954+0000 7f4fceab1700 1 -- start start 2021-11-03T18:34:37.957 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.954+0000 7f4fceab1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4fc80f66e0 0x7f4fc8106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:34:37.957 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.954+0000 7f4fceab1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fc80f6ff0 0x7f4fc8107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:34:37.957 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.954+0000 7f4fceab1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc8107a10 con 0x7f4fc80f6ff0 2021-11-03T18:34:37.958 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.954+0000 7f4fceab1700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fc8107b50 con 0x7f4fc80f66e0 2021-11-03T18:34:37.958 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.954+0000 7f4fc7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fc80f6ff0 0x7f4fc8107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:34:37.958 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.954+0000 7f4fc7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fc80f6ff0 0x7f4fc8107460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35668/0 (socket says 172.21.15.27:35668) 2021-11-03T18:34:37.958 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.954+0000 7f4fc7fff700 1 -- 172.21.15.27:0/1180668433 learned_addr learned my addr 172.21.15.27:0/1180668433 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:34:37.959 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.954+0000 7f4fc7fff700 1 -- 172.21.15.27:0/1180668433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4fc80f66e0 msgr2=0x7f4fc8106f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:34:37.959 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.954+0000 7f4fc7fff700 1 --2- 172.21.15.27:0/1180668433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4fc80f66e0 0x7f4fc8106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:34:37.959 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.954+0000 7f4fc7fff700 1 -- 172.21.15.27:0/1180668433 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb4005040 con 0x7f4fc80f6ff0 2021-11-03T18:34:37.959 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.955+0000 7f4fc7fff700 1 --2- 172.21.15.27:0/1180668433 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fc80f6ff0 0x7f4fc8107460 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 rx=0x7f4fb4009990 tx=0x7f4fb4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:34:37.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.955+0000 7f4fc57fa700 1 -- 172.21.15.27:0/1180668433 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fb4018070 con 0x7f4fc80f6ff0 2021-11-03T18:34:37.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.955+0000 7f4fc57fa700 1 -- 172.21.15.27:0/1180668433 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f4fb4007ba0 con 0x7f4fc80f6ff0 2021-11-03T18:34:37.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.955+0000 7f4fceab1700 1 -- 172.21.15.27:0/1180668433 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fc8108490 con 0x7f4fc80f6ff0 2021-11-03T18:34:37.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.955+0000 7f4fc57fa700 1 -- 172.21.15.27:0/1180668433 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4fb401c3c0 con 0x7f4fc80f6ff0 2021-11-03T18:34:37.961 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.955+0000 7f4fceab1700 1 -- 172.21.15.27:0/1180668433 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fc81089f0 con 0x7f4fc80f6ff0 2021-11-03T18:34:37.962 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.956+0000 7f4fceab1700 1 -- 172.21.15.27:0/1180668433 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4fc8045ad0 con 0x7f4fc80f6ff0 2021-11-03T18:34:37.962 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.956+0000 7f4fc57fa700 1 -- 172.21.15.27:0/1180668433 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f4fb401d420 con 0x7f4fc80f6ff0 2021-11-03T18:34:37.962 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.957+0000 7f4fc57fa700 1 --2- 172.21.15.27:0/1180668433 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4fb80640b0 0x7f4fb8066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:34:37.963 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.957+0000 7f4fc57fa700 1 -- 172.21.15.27:0/1180668433 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f4fb40825d0 con 0x7f4fc80f6ff0 2021-11-03T18:34:37.963 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.957+0000 7f4fcc84d700 1 --2- 172.21.15.27:0/1180668433 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4fb80640b0 0x7f4fb8066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:34:37.963 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.957+0000 7f4fcc84d700 1 --2- 172.21.15.27:0/1180668433 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4fb80640b0 0x7f4fb8066570 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7f4fbc004a10 tx=0x7f4fbc0099e0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:34:37.963 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:37.961+0000 7f4fc57fa700 1 -- 172.21.15.27:0/1180668433 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f4fb4054990 con 0x7f4fc80f6ff0 2021-11-03T18:34:38.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:37 smithi027 ceph-mon[97244]: poll took 60170.877 ms: 1 events 2021-11-03T18:34:38.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:34:38.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:34:38.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:38.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:38.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:38.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:38.166 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:38.167+0000 7f4fceab1700 1 -- 172.21.15.27:0/1180668433 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4fc8108c10 con 0x7f4fc80f6ff0 2021-11-03T18:34:38.166 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:38.167+0000 7f4fc57fa700 1 -- 172.21.15.27:0/1180668433 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f4fb4054580 con 0x7f4fc80f6ff0 2021-11-03T18:34:38.167 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:34:38.167 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:34:38.167 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:34:38.167 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:34:38.167 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:34:38.168 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:34:38.168 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:34:38.168 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:34:38.168 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:34:38.169 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:34:38.169 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:34:38.169 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:34:38.169 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:34:38.169 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:34:38.170 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:34:38.170 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:34:38.170 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:34:38.170 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:34:38.171 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:34:38.171 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:34:38.171 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:34:38.171 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:34:38.172 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:34:38.172 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:34:38.172 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:34:38.172 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:34:38.172 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:34:38.173 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:34:38.173 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:34:38.173 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:34:38.174 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:34:38.174 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:34:38.174 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:34:38.174 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:34:38.175 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:34:38.175 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:34:38.175 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:34:38.175 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:34:38.175 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:34:38.176 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:38.169+0000 7f4fceab1700 1 -- 172.21.15.27:0/1180668433 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4fb80640b0 msgr2=0x7f4fb8066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:34:38.177 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:38.169+0000 7f4fceab1700 1 --2- 172.21.15.27:0/1180668433 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4fb80640b0 0x7f4fb8066570 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 rx=0x7f4fbc004a10 tx=0x7f4fbc0099e0).stop 2021-11-03T18:34:38.177 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:38.169+0000 7f4fceab1700 1 -- 172.21.15.27:0/1180668433 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fc80f6ff0 msgr2=0x7f4fc8107460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:34:38.177 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:38.169+0000 7f4fceab1700 1 --2- 172.21.15.27:0/1180668433 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fc80f6ff0 0x7f4fc8107460 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 rx=0x7f4fb4009990 tx=0x7f4fb4009b30).stop 2021-11-03T18:34:38.177 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:38.169+0000 7f4fceab1700 1 -- 172.21.15.27:0/1180668433 shutdown_connections 2021-11-03T18:34:38.178 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:38.169+0000 7f4fceab1700 1 --2- 172.21.15.27:0/1180668433 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4fb80640b0 0x7f4fb8066570 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:38.178 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:38.169+0000 7f4fceab1700 1 --2- 172.21.15.27:0/1180668433 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4fc80f66e0 0x7f4fc8106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:34:38.178 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:38.169+0000 7f4fceab1700 1 --2- 172.21.15.27:0/1180668433 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4fc80f6ff0 0x7f4fc8107460 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:34:38.178 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:38.169+0000 7f4fceab1700 1 -- 172.21.15.27:0/1180668433 >> 172.21.15.27:0/1180668433 conn(0x7f4fc80f1f00 msgr2=0x7f4fc80fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:34:38.179 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:38.169+0000 7f4fceab1700 1 -- 172.21.15.27:0/1180668433 shutdown_connections 2021-11-03T18:34:38.179 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:34:38.169+0000 7f4fceab1700 1 -- 172.21.15.27:0/1180668433 wait complete. 2021-11-03T18:34:38.179 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:34:38.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:37 smithi115 ceph-mon[65453]: poll took 60170.877 ms: 1 events 2021-11-03T18:34:38.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:34:38.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:34:38.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:38.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:38.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:38.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:39.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:38 smithi027 ceph-mon[97244]: pgmap v2401: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:39.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:38 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:34:39.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:38 smithi027 ceph-mon[97244]: from='client.36568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:34:39.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:38 smithi027 ceph-mon[97244]: from='client.36572 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:34:39.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:38 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3406725725' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:39.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:38 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1180668433' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:34:39.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:38 smithi115 ceph-mon[65453]: pgmap v2401: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:39.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:38 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:34:39.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:38 smithi115 ceph-mon[65453]: from='client.36568 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:34:39.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:38 smithi115 ceph-mon[65453]: from='client.36572 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:34:39.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:38 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3406725725' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:39.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:38 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1180668433' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:34:41.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:40 smithi027 ceph-mon[97244]: pgmap v2402: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:41.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4722.473140 secs 2021-11-03T18:34:41.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:40 smithi115 ceph-mon[65453]: pgmap v2402: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:41.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4722.473140 secs 2021-11-03T18:34:43.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:42 smithi115 ceph-mon[65453]: pgmap v2403: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:43.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:42 smithi027 ceph-mon[97244]: pgmap v2403: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:45.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:44 smithi115 ceph-mon[65453]: pgmap v2404: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:45.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:44 smithi027 ceph-mon[97244]: pgmap v2404: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:46.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4727.473273 secs 2021-11-03T18:34:46.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4727.473273 secs 2021-11-03T18:34:47.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:46 smithi115 ceph-mon[65453]: pgmap v2405: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:47.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:46 smithi027 ceph-mon[97244]: pgmap v2405: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:48.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:34:48.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:34:48.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:34:48.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:48.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:48.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:48.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:48.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:34:48.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:34:48.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:34:48.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:48.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:48.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:48.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:49.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:48 smithi115 ceph-mon[65453]: pgmap v2406: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:49.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:48 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:34:49.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:48 smithi027 ceph-mon[97244]: pgmap v2406: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:49.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:48 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:34:51.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:50 smithi115 ceph-mon[65453]: pgmap v2407: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:51.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4732.473425 secs 2021-11-03T18:34:51.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:50 smithi027 ceph-mon[97244]: pgmap v2407: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:51.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4732.473425 secs 2021-11-03T18:34:53.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:52 smithi115 ceph-mon[65453]: pgmap v2408: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:53.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:52 smithi027 ceph-mon[97244]: pgmap v2408: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:55.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:54 smithi115 ceph-mon[65453]: pgmap v2409: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:55.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:54 smithi027 ceph-mon[97244]: pgmap v2409: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:56.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4737.473549 secs 2021-11-03T18:34:56.314 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:34:56.315 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:34:56.317 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4737.473549 secs 2021-11-03T18:34:56.317 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:34:56.317 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:34:57.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:56 smithi115 ceph-mon[65453]: pgmap v2410: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:57.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:56 smithi027 ceph-mon[97244]: pgmap v2410: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:58.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:34:58.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:34:58.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:34:58.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:58.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:58.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:58.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:58.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:34:58.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:34:58.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:34:58.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:58.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:58.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:58.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:34:59.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:58 smithi115 ceph-mon[65453]: pgmap v2411: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:59.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:34:58 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:34:59.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:58 smithi027 ceph-mon[97244]: pgmap v2411: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:34:59.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:34:58 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:35:01.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:00 smithi115 ceph-mon[65453]: pgmap v2412: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:01.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4742.473630 secs 2021-11-03T18:35:01.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:00 smithi027 ceph-mon[97244]: pgmap v2412: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:01.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4742.473630 secs 2021-11-03T18:35:03.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:02 smithi115 ceph-mon[65453]: pgmap v2413: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:03.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:02 smithi027 ceph-mon[97244]: pgmap v2413: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:05.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:04 smithi115 ceph-mon[65453]: pgmap v2414: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:05.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:04 smithi027 ceph-mon[97244]: pgmap v2414: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:06.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4747.473762 secs 2021-11-03T18:35:06.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4747.473762 secs 2021-11-03T18:35:07.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:06 smithi115 ceph-mon[65453]: pgmap v2415: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:07.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:06 smithi027 ceph-mon[97244]: pgmap v2415: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:08.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:07 smithi115 ceph-mon[65453]: pgmap v2416: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:08.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:35:08.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:35:08.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:35:08.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:08.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:08.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:08.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:08.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:07 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:35:08.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:07 smithi027 ceph-mon[97244]: pgmap v2416: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:08.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:35:08.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:35:08.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:35:08.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:08.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:08.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:08.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:08.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:07 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:35:08.283 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.284+0000 7f4464fb7700 1 -- 172.21.15.27:0/1185425548 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44600f57c0 msgr2=0x7f44600f5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:35:08.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.284+0000 7f4464fb7700 1 --2- 172.21.15.27:0/1185425548 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44600f57c0 0x7f44600f5c40 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 rx=0x7f4454004660 tx=0x7f4454009b30).stop 2021-11-03T18:35:08.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.284+0000 7f4464fb7700 1 -- 172.21.15.27:0/1185425548 shutdown_connections 2021-11-03T18:35:08.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.284+0000 7f4464fb7700 1 --2- 172.21.15.27:0/1185425548 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44600f57c0 0x7f44600f5c40 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:08.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.284+0000 7f4464fb7700 1 --2- 172.21.15.27:0/1185425548 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44600f6ab0 0x7f44600f6ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:08.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.284+0000 7f4464fb7700 1 -- 172.21.15.27:0/1185425548 >> 172.21.15.27:0/1185425548 conn(0x7f44600f0330 msgr2=0x7f44600f2750 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:35:08.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.284+0000 7f4464fb7700 1 -- 172.21.15.27:0/1185425548 shutdown_connections 2021-11-03T18:35:08.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.284+0000 7f4464fb7700 1 -- 172.21.15.27:0/1185425548 wait complete. 2021-11-03T18:35:08.286 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.284+0000 7f4464fb7700 1 Processor -- start 2021-11-03T18:35:08.286 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.284+0000 7f4464fb7700 1 -- start start 2021-11-03T18:35:08.286 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.285+0000 7f4464fb7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44600f57c0 0x7f4460109a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:35:08.286 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.285+0000 7f4464fb7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44600f6ab0 0x7f446010bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:35:08.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.285+0000 7f4464fb7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44600f5450 con 0x7f44600f57c0 2021-11-03T18:35:08.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.285+0000 7f4464fb7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44600f5590 con 0x7f44600f6ab0 2021-11-03T18:35:08.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.285+0000 7f445f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44600f57c0 0x7f4460109a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:35:08.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.285+0000 7f445f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44600f57c0 0x7f4460109a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35684/0 (socket says 172.21.15.27:35684) 2021-11-03T18:35:08.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.285+0000 7f445f7fe700 1 -- 172.21.15.27:0/1290920913 learned_addr learned my addr 172.21.15.27:0/1290920913 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:35:08.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.285+0000 7f445effd700 1 --2- 172.21.15.27:0/1290920913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44600f6ab0 0x7f446010bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:35:08.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.285+0000 7f445f7fe700 1 -- 172.21.15.27:0/1290920913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44600f6ab0 msgr2=0x7f446010bf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:35:08.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.285+0000 7f445f7fe700 1 --2- 172.21.15.27:0/1290920913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44600f6ab0 0x7f446010bf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:08.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.285+0000 7f445f7fe700 1 -- 172.21.15.27:0/1290920913 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4454005040 con 0x7f44600f57c0 2021-11-03T18:35:08.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.285+0000 7f445f7fe700 1 --2- 172.21.15.27:0/1290920913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44600f57c0 0x7f4460109a40 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 rx=0x7f44500026d0 tx=0x7f4450009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:35:08.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.285+0000 7f4447fff700 1 -- 172.21.15.27:0/1290920913 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4450009320 con 0x7f44600f57c0 2021-11-03T18:35:08.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.285+0000 7f4447fff700 1 -- 172.21.15.27:0/1290920913 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f445000f040 con 0x7f44600f57c0 2021-11-03T18:35:08.290 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.285+0000 7f4464fb7700 1 -- 172.21.15.27:0/1290920913 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f446010c4f0 con 0x7f44600f57c0 2021-11-03T18:35:08.291 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.285+0000 7f4447fff700 1 -- 172.21.15.27:0/1290920913 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44500147e0 con 0x7f44600f57c0 2021-11-03T18:35:08.291 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.286+0000 7f4464fb7700 1 -- 172.21.15.27:0/1290920913 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f446010ca80 con 0x7f44600f57c0 2021-11-03T18:35:08.291 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.287+0000 7f4464fb7700 1 -- 172.21.15.27:0/1290920913 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4460059100 con 0x7f44600f57c0 2021-11-03T18:35:08.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.287+0000 7f4447fff700 1 -- 172.21.15.27:0/1290920913 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f4450009480 con 0x7f44600f57c0 2021-11-03T18:35:08.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.287+0000 7f4447fff700 1 --2- 172.21.15.27:0/1290920913 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4448064100 0x7f44480665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:35:08.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.287+0000 7f4447fff700 1 -- 172.21.15.27:0/1290920913 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f4450080df0 con 0x7f44600f57c0 2021-11-03T18:35:08.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.287+0000 7f445effd700 1 --2- 172.21.15.27:0/1290920913 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4448064100 0x7f44480665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:35:08.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.288+0000 7f445effd700 1 --2- 172.21.15.27:0/1290920913 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4448064100 0x7f44480665c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7f4454004630 tx=0x7f4454009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:35:08.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.292+0000 7f4447fff700 1 -- 172.21.15.27:0/1290920913 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f4450053210 con 0x7f44600f57c0 2021-11-03T18:35:08.469 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.471+0000 7f4464fb7700 1 -- 172.21.15.27:0/1290920913 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f44600fde40 con 0x7f4448064100 2021-11-03T18:35:08.470 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.472+0000 7f4447fff700 1 -- 172.21.15.27:0/1290920913 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f4454019040 con 0x7f4448064100 2021-11-03T18:35:08.472 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.474+0000 7f4464fb7700 1 -- 172.21.15.27:0/1290920913 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4448064100 msgr2=0x7f44480665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:35:08.472 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.474+0000 7f4464fb7700 1 --2- 172.21.15.27:0/1290920913 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4448064100 0x7f44480665c0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 rx=0x7f4454004630 tx=0x7f4454009b30).stop 2021-11-03T18:35:08.472 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.474+0000 7f4464fb7700 1 -- 172.21.15.27:0/1290920913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44600f57c0 msgr2=0x7f4460109a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:35:08.473 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.474+0000 7f4464fb7700 1 --2- 172.21.15.27:0/1290920913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44600f57c0 0x7f4460109a40 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 rx=0x7f44500026d0 tx=0x7f4450009de0).stop 2021-11-03T18:35:08.473 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.474+0000 7f4464fb7700 1 -- 172.21.15.27:0/1290920913 shutdown_connections 2021-11-03T18:35:08.473 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.474+0000 7f4464fb7700 1 --2- 172.21.15.27:0/1290920913 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4448064100 0x7f44480665c0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:08.473 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.474+0000 7f4464fb7700 1 --2- 172.21.15.27:0/1290920913 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f44600f57c0 0x7f4460109a40 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:08.474 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.474+0000 7f4464fb7700 1 --2- 172.21.15.27:0/1290920913 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f44600f6ab0 0x7f446010bf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:08.474 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.474+0000 7f4464fb7700 1 -- 172.21.15.27:0/1290920913 >> 172.21.15.27:0/1290920913 conn(0x7f44600f0330 msgr2=0x7f44600fa220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:35:08.474 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.474+0000 7f4464fb7700 1 -- 172.21.15.27:0/1290920913 shutdown_connections 2021-11-03T18:35:08.474 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.474+0000 7f4464fb7700 1 -- 172.21.15.27:0/1290920913 wait complete. 2021-11-03T18:35:08.482 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:35:08.575 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.574+0000 7f95e05a0700 1 -- 172.21.15.27:0/4060808822 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95d80f7550 msgr2=0x7f95d80f47e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:35:08.575 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.574+0000 7f95e05a0700 1 --2- 172.21.15.27:0/4060808822 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95d80f7550 0x7f95d80f47e0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 rx=0x7f95c8004660 tx=0x7f95c8009b30).stop 2021-11-03T18:35:08.575 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.575+0000 7f95e05a0700 1 -- 172.21.15.27:0/4060808822 shutdown_connections 2021-11-03T18:35:08.576 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.575+0000 7f95e05a0700 1 --2- 172.21.15.27:0/4060808822 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95d80f7550 0x7f95d80f47e0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:08.576 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.575+0000 7f95e05a0700 1 --2- 172.21.15.27:0/4060808822 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95d80f6bb0 0x7f95d80f6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:08.576 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.575+0000 7f95e05a0700 1 -- 172.21.15.27:0/4060808822 >> 172.21.15.27:0/4060808822 conn(0x7f95d80f04c0 msgr2=0x7f95d80f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:35:08.576 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.575+0000 7f95e05a0700 1 -- 172.21.15.27:0/4060808822 shutdown_connections 2021-11-03T18:35:08.577 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.575+0000 7f95e05a0700 1 -- 172.21.15.27:0/4060808822 wait complete. 2021-11-03T18:35:08.577 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.575+0000 7f95e05a0700 1 Processor -- start 2021-11-03T18:35:08.577 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.575+0000 7f95e05a0700 1 -- start start 2021-11-03T18:35:08.578 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.575+0000 7f95e05a0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95d80f6bb0 0x7f95d8109150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:35:08.578 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.575+0000 7f95e05a0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95d80f7550 0x7f95d8109640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:35:08.578 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.575+0000 7f95e05a0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d8109bf0 con 0x7f95d80f7550 2021-11-03T18:35:08.578 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.575+0000 7f95e05a0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95d8109d30 con 0x7f95d80f6bb0 2021-11-03T18:35:08.579 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.576+0000 7f95ddb3b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95d80f7550 0x7f95d8109640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:35:08.579 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.576+0000 7f95ddb3b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95d80f7550 0x7f95d8109640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35696/0 (socket says 172.21.15.27:35696) 2021-11-03T18:35:08.579 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.576+0000 7f95ddb3b700 1 -- 172.21.15.27:0/2261144309 learned_addr learned my addr 172.21.15.27:0/2261144309 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:35:08.579 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.576+0000 7f95de33c700 1 --2- 172.21.15.27:0/2261144309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95d80f6bb0 0x7f95d8109150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:35:08.580 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.576+0000 7f95ddb3b700 1 -- 172.21.15.27:0/2261144309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95d80f6bb0 msgr2=0x7f95d8109150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:35:08.580 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.576+0000 7f95ddb3b700 1 --2- 172.21.15.27:0/2261144309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95d80f6bb0 0x7f95d8109150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:08.580 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.576+0000 7f95ddb3b700 1 -- 172.21.15.27:0/2261144309 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95c8005040 con 0x7f95d80f7550 2021-11-03T18:35:08.580 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.576+0000 7f95ddb3b700 1 --2- 172.21.15.27:0/2261144309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95d80f7550 0x7f95d8109640 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 rx=0x7f95c8009990 tx=0x7f95c8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:35:08.580 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.576+0000 7f95ceffd700 1 -- 172.21.15.27:0/2261144309 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95c8018070 con 0x7f95d80f7550 2021-11-03T18:35:08.581 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.576+0000 7f95ceffd700 1 -- 172.21.15.27:0/2261144309 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f95c8007ba0 con 0x7f95d80f7550 2021-11-03T18:35:08.581 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.576+0000 7f95ceffd700 1 -- 172.21.15.27:0/2261144309 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95c801c3c0 con 0x7f95d80f7550 2021-11-03T18:35:08.582 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.576+0000 7f95e05a0700 1 -- 172.21.15.27:0/2261144309 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95d810a670 con 0x7f95d80f7550 2021-11-03T18:35:08.582 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.576+0000 7f95e05a0700 1 -- 172.21.15.27:0/2261144309 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95d810abd0 con 0x7f95d80f7550 2021-11-03T18:35:08.584 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.577+0000 7f95e05a0700 1 -- 172.21.15.27:0/2261144309 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95d8045ad0 con 0x7f95d80f7550 2021-11-03T18:35:08.584 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.578+0000 7f95ceffd700 1 -- 172.21.15.27:0/2261144309 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f95c801d420 con 0x7f95d80f7550 2021-11-03T18:35:08.584 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.578+0000 7f95ceffd700 1 --2- 172.21.15.27:0/2261144309 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f95c40640b0 0x7f95c4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:35:08.585 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.578+0000 7f95ceffd700 1 -- 172.21.15.27:0/2261144309 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f95c8082410 con 0x7f95d80f7550 2021-11-03T18:35:08.585 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.578+0000 7f95de33c700 1 --2- 172.21.15.27:0/2261144309 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f95c40640b0 0x7f95c4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:35:08.585 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.579+0000 7f95de33c700 1 --2- 172.21.15.27:0/2261144309 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f95c40640b0 0x7f95c4066570 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7f95d4004c40 tx=0x7f95d400b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:35:08.585 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.582+0000 7f95ceffd700 1 -- 172.21.15.27:0/2261144309 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f95c8054830 con 0x7f95d80f7550 2021-11-03T18:35:08.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.755+0000 7f95e05a0700 1 -- 172.21.15.27:0/2261144309 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f95d810adf0 con 0x7f95c40640b0 2021-11-03T18:35:08.768 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.770+0000 7f95ceffd700 1 -- 172.21.15.27:0/2261144309 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f95d810adf0 con 0x7f95c40640b0 2021-11-03T18:35:08.768 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:35:08.768 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (86m) 5m ago 87m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:35:08.769 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (78m) 5m ago 87m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:35:08.769 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (78m) 4m ago 87m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:35:08.769 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (86m) 5m ago 87m 42.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:35:08.769 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (80m) 5m ago 83m 26.7M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:35:08.770 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (83m) 5m ago 83m 3850M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:35:08.770 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (78m) 4m ago 83m 26.3M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:35:08.770 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (80m) 4m ago 83m 27.4M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:35:08.770 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (80m) 5m ago 89m 608M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:35:08.770 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (79m) 4m ago 87m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:35:08.771 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (79m) 5m ago 89m 350M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:35:08.771 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (78m) 4m ago 87m 160M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:35:08.771 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (87m) 5m ago 87m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:35:08.771 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (86m) 4m ago 86m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:35:08.772 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (78m) 5m ago 86m 413M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:35:08.772 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (77m) 5m ago 86m 450M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:35:08.772 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (77m) 5m ago 85m 338M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:35:08.772 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (76m) 4m ago 85m 479M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:35:08.772 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (76m) 4m ago 85m 378M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:35:08.773 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (75m) 4m ago 84m 388M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:35:08.773 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (86m) 5m ago 87m 64.4M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:35:08.774 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.772+0000 7f95e05a0700 1 -- 172.21.15.27:0/2261144309 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f95c40640b0 msgr2=0x7f95c4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:35:08.774 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.772+0000 7f95e05a0700 1 --2- 172.21.15.27:0/2261144309 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f95c40640b0 0x7f95c4066570 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 rx=0x7f95d4004c40 tx=0x7f95d400b040).stop 2021-11-03T18:35:08.774 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.772+0000 7f95e05a0700 1 -- 172.21.15.27:0/2261144309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95d80f7550 msgr2=0x7f95d8109640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:35:08.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.772+0000 7f95e05a0700 1 --2- 172.21.15.27:0/2261144309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95d80f7550 0x7f95d8109640 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 rx=0x7f95c8009990 tx=0x7f95c8009b30).stop 2021-11-03T18:35:08.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.772+0000 7f95e05a0700 1 -- 172.21.15.27:0/2261144309 shutdown_connections 2021-11-03T18:35:08.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.772+0000 7f95e05a0700 1 --2- 172.21.15.27:0/2261144309 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f95c40640b0 0x7f95c4066570 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:08.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.772+0000 7f95e05a0700 1 --2- 172.21.15.27:0/2261144309 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95d80f6bb0 0x7f95d8109150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:08.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.772+0000 7f95e05a0700 1 --2- 172.21.15.27:0/2261144309 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95d80f7550 0x7f95d8109640 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:08.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.772+0000 7f95e05a0700 1 -- 172.21.15.27:0/2261144309 >> 172.21.15.27:0/2261144309 conn(0x7f95d80f04c0 msgr2=0x7f95d80fe700 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:35:08.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.772+0000 7f95e05a0700 1 -- 172.21.15.27:0/2261144309 shutdown_connections 2021-11-03T18:35:08.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.772+0000 7f95e05a0700 1 -- 172.21.15.27:0/2261144309 wait complete. 2021-11-03T18:35:08.872 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.871+0000 7f1e32af9700 1 -- 172.21.15.27:0/600133668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e2c0f7960 msgr2=0x7f1e2c0f7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:35:08.872 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.871+0000 7f1e32af9700 1 --2- 172.21.15.27:0/600133668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e2c0f7960 0x7f1e2c0f7dc0 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 rx=0x7f1e1c0099e0 tx=0x7f1e1c004d10).stop 2021-11-03T18:35:08.872 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.871+0000 7f1e32af9700 1 -- 172.21.15.27:0/600133668 shutdown_connections 2021-11-03T18:35:08.872 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.871+0000 7f1e32af9700 1 --2- 172.21.15.27:0/600133668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e2c0f7960 0x7f1e2c0f7dc0 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:08.873 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.871+0000 7f1e32af9700 1 --2- 172.21.15.27:0/600133668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e2c0f66c0 0x7f1e2c0f6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:08.873 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.871+0000 7f1e32af9700 1 -- 172.21.15.27:0/600133668 >> 172.21.15.27:0/600133668 conn(0x7f1e2c0f1e20 msgr2=0x7f1e2c0f4240 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:35:08.873 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.871+0000 7f1e32af9700 1 -- 172.21.15.27:0/600133668 shutdown_connections 2021-11-03T18:35:08.873 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.871+0000 7f1e32af9700 1 -- 172.21.15.27:0/600133668 wait complete. 2021-11-03T18:35:08.874 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.872+0000 7f1e32af9700 1 Processor -- start 2021-11-03T18:35:08.874 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.872+0000 7f1e32af9700 1 -- start start 2021-11-03T18:35:08.874 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.872+0000 7f1e32af9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e2c0f66c0 0x7f1e2c107030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:35:08.874 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.872+0000 7f1e32af9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e2c0f7960 0x7f1e2c107520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:35:08.875 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.872+0000 7f1e32af9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2c107ad0 con 0x7f1e2c0f66c0 2021-11-03T18:35:08.875 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.872+0000 7f1e32af9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2c107c10 con 0x7f1e2c0f7960 2021-11-03T18:35:08.876 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.872+0000 7f1e30895700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e2c0f66c0 0x7f1e2c107030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:35:08.876 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.872+0000 7f1e30895700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e2c0f66c0 0x7f1e2c107030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35704/0 (socket says 172.21.15.27:35704) 2021-11-03T18:35:08.876 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.872+0000 7f1e30895700 1 -- 172.21.15.27:0/1974934674 learned_addr learned my addr 172.21.15.27:0/1974934674 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:35:08.876 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.872+0000 7f1e2bfff700 1 --2- 172.21.15.27:0/1974934674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e2c0f7960 0x7f1e2c107520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:35:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.873+0000 7f1e30895700 1 -- 172.21.15.27:0/1974934674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e2c0f7960 msgr2=0x7f1e2c107520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:35:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.873+0000 7f1e30895700 1 --2- 172.21.15.27:0/1974934674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e2c0f7960 0x7f1e2c107520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.873+0000 7f1e30895700 1 -- 172.21.15.27:0/1974934674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e1c009640 con 0x7f1e2c0f66c0 2021-11-03T18:35:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.873+0000 7f1e30895700 1 --2- 172.21.15.27:0/1974934674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e2c0f66c0 0x7f1e2c107030 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 rx=0x7f1e24002700 tx=0x7f1e24009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:35:08.877 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.873+0000 7f1e297fa700 1 -- 172.21.15.27:0/1974934674 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e240079c0 con 0x7f1e2c0f66c0 2021-11-03T18:35:08.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.873+0000 7f1e32af9700 1 -- 172.21.15.27:0/1974934674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e2c1085b0 con 0x7f1e2c0f66c0 2021-11-03T18:35:08.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.873+0000 7f1e297fa700 1 -- 172.21.15.27:0/1974934674 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f1e2400f040 con 0x7f1e2c0f66c0 2021-11-03T18:35:08.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.873+0000 7f1e297fa700 1 -- 172.21.15.27:0/1974934674 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1e24014860 con 0x7f1e2c0f66c0 2021-11-03T18:35:08.879 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.873+0000 7f1e32af9700 1 -- 172.21.15.27:0/1974934674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e2c108af0 con 0x7f1e2c0f66c0 2021-11-03T18:35:08.879 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.874+0000 7f1e32af9700 1 -- 172.21.15.27:0/1974934674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e2c045ad0 con 0x7f1e2c0f66c0 2021-11-03T18:35:08.880 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.874+0000 7f1e297fa700 1 -- 172.21.15.27:0/1974934674 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f1e2401b060 con 0x7f1e2c0f66c0 2021-11-03T18:35:08.880 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.875+0000 7f1e297fa700 1 --2- 172.21.15.27:0/1974934674 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1e180640b0 0x7f1e18066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:35:08.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.875+0000 7f1e297fa700 1 -- 172.21.15.27:0/1974934674 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f1e24080870 con 0x7f1e2c0f66c0 2021-11-03T18:35:08.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.875+0000 7f1e2bfff700 1 --2- 172.21.15.27:0/1974934674 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1e180640b0 0x7f1e18066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:35:08.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.875+0000 7f1e2bfff700 1 --2- 172.21.15.27:0/1974934674 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1e180640b0 0x7f1e18066570 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7f1e1c003380 tx=0x7f1e1c004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:35:08.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:08.878+0000 7f1e297fa700 1 -- 172.21.15.27:0/1974934674 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f1e24052bd0 con 0x7f1e2c0f66c0 2021-11-03T18:35:09.104 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.106+0000 7f1e32af9700 1 -- 172.21.15.27:0/1974934674 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1e2c108d70 con 0x7f1e2c0f66c0 2021-11-03T18:35:09.105 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.107+0000 7f1e297fa700 1 -- 172.21.15.27:0/1974934674 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f1e240527c0 con 0x7f1e2c0f66c0 2021-11-03T18:35:09.106 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:35:09.106 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:35:09.106 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:35:09.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:35:09.107 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:35:09.107 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:35:09.107 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:35:09.108 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:35:09.108 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:35:09.108 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:35:09.108 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:35:09.109 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:35:09.109 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:35:09.109 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:35:09.109 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:35:09.110 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:35:09.110 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:35:09.110 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:35:09.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.109+0000 7f1e32af9700 1 -- 172.21.15.27:0/1974934674 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1e180640b0 msgr2=0x7f1e18066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:35:09.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.109+0000 7f1e32af9700 1 --2- 172.21.15.27:0/1974934674 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1e180640b0 0x7f1e18066570 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7f1e1c003380 tx=0x7f1e1c004d10).stop 2021-11-03T18:35:09.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.109+0000 7f1e32af9700 1 -- 172.21.15.27:0/1974934674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e2c0f66c0 msgr2=0x7f1e2c107030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:35:09.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.109+0000 7f1e32af9700 1 --2- 172.21.15.27:0/1974934674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e2c0f66c0 0x7f1e2c107030 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 rx=0x7f1e24002700 tx=0x7f1e24009d80).stop 2021-11-03T18:35:09.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.109+0000 7f1e32af9700 1 -- 172.21.15.27:0/1974934674 shutdown_connections 2021-11-03T18:35:09.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.109+0000 7f1e32af9700 1 --2- 172.21.15.27:0/1974934674 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1e180640b0 0x7f1e18066570 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:09.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.109+0000 7f1e32af9700 1 --2- 172.21.15.27:0/1974934674 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f1e2c0f66c0 0x7f1e2c107030 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:09.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.109+0000 7f1e32af9700 1 --2- 172.21.15.27:0/1974934674 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f1e2c0f7960 0x7f1e2c107520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:09.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.109+0000 7f1e32af9700 1 -- 172.21.15.27:0/1974934674 >> 172.21.15.27:0/1974934674 conn(0x7f1e2c0f1e20 msgr2=0x7f1e2c0fb3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:35:09.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.109+0000 7f1e32af9700 1 -- 172.21.15.27:0/1974934674 shutdown_connections 2021-11-03T18:35:09.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.109+0000 7f1e32af9700 1 -- 172.21.15.27:0/1974934674 wait complete. 2021-11-03T18:35:09.212 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.209+0000 7f0635b82700 1 -- 172.21.15.27:0/982605235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06300f55e0 msgr2=0x7f06300f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:35:09.212 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.209+0000 7f0635b82700 1 --2- 172.21.15.27:0/982605235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06300f55e0 0x7f06300f5a00 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 rx=0x7f0620004660 tx=0x7f0620009b30).stop 2021-11-03T18:35:09.213 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.209+0000 7f0635b82700 1 -- 172.21.15.27:0/982605235 shutdown_connections 2021-11-03T18:35:09.213 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.209+0000 7f0635b82700 1 --2- 172.21.15.27:0/982605235 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06300f5ef0 0x7f06300f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:09.214 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.209+0000 7f0635b82700 1 --2- 172.21.15.27:0/982605235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06300f55e0 0x7f06300f5a00 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:09.214 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.209+0000 7f0635b82700 1 -- 172.21.15.27:0/982605235 >> 172.21.15.27:0/982605235 conn(0x7f06300f0de0 msgr2=0x7f06300f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:35:09.215 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.209+0000 7f0635b82700 1 -- 172.21.15.27:0/982605235 shutdown_connections 2021-11-03T18:35:09.215 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.209+0000 7f0635b82700 1 -- 172.21.15.27:0/982605235 wait complete. 2021-11-03T18:35:09.215 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.209+0000 7f0635b82700 1 Processor -- start 2021-11-03T18:35:09.216 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.210+0000 7f0635b82700 1 -- start start 2021-11-03T18:35:09.216 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.210+0000 7f0635b82700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06300f55e0 0x7f06300f97e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:35:09.216 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.210+0000 7f0635b82700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06300f5ef0 0x7f06300f7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:35:09.216 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.210+0000 7f0635b82700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06300f81c0 con 0x7f06300f55e0 2021-11-03T18:35:09.217 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.210+0000 7f0635b82700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06300f8300 con 0x7f06300f5ef0 2021-11-03T18:35:09.217 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.210+0000 7f062f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06300f55e0 0x7f06300f97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:35:09.217 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.210+0000 7f062f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06300f55e0 0x7f06300f97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35714/0 (socket says 172.21.15.27:35714) 2021-11-03T18:35:09.217 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.210+0000 7f062f7fe700 1 -- 172.21.15.27:0/1442015290 learned_addr learned my addr 172.21.15.27:0/1442015290 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:35:09.218 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.210+0000 7f062effd700 1 --2- 172.21.15.27:0/1442015290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06300f5ef0 0x7f06300f7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:35:09.218 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.210+0000 7f062f7fe700 1 -- 172.21.15.27:0/1442015290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06300f5ef0 msgr2=0x7f06300f7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:35:09.218 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.210+0000 7f062f7fe700 1 --2- 172.21.15.27:0/1442015290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06300f5ef0 0x7f06300f7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:09.219 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.210+0000 7f062f7fe700 1 -- 172.21.15.27:0/1442015290 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0620005040 con 0x7f06300f55e0 2021-11-03T18:35:09.219 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.210+0000 7f062effd700 1 --2- 172.21.15.27:0/1442015290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06300f5ef0 0x7f06300f7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:35:09.219 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.210+0000 7f062f7fe700 1 --2- 172.21.15.27:0/1442015290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06300f55e0 0x7f06300f97e0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 rx=0x7f0620009990 tx=0x7f0620009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:35:09.219 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.211+0000 7f0617fff700 1 -- 172.21.15.27:0/1442015290 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0620018070 con 0x7f06300f55e0 2021-11-03T18:35:09.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.211+0000 7f0617fff700 1 -- 172.21.15.27:0/1442015290 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f0620007ca0 con 0x7f06300f55e0 2021-11-03T18:35:09.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.211+0000 7f0635b82700 1 -- 172.21.15.27:0/1442015290 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06300f8580 con 0x7f06300f55e0 2021-11-03T18:35:09.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.211+0000 7f0617fff700 1 -- 172.21.15.27:0/1442015290 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f062001c620 con 0x7f06300f55e0 2021-11-03T18:35:09.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.211+0000 7f0635b82700 1 -- 172.21.15.27:0/1442015290 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06300f8ae0 con 0x7f06300f55e0 2021-11-03T18:35:09.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.212+0000 7f0635b82700 1 -- 172.21.15.27:0/1442015290 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0630045ad0 con 0x7f06300f55e0 2021-11-03T18:35:09.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.212+0000 7f0617fff700 1 -- 172.21.15.27:0/1442015290 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f062001d960 con 0x7f06300f55e0 2021-11-03T18:35:09.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.212+0000 7f0617fff700 1 --2- 172.21.15.27:0/1442015290 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f06180640b0 0x7f0618066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:35:09.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.212+0000 7f0617fff700 1 -- 172.21.15.27:0/1442015290 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f0620082520 con 0x7f06300f55e0 2021-11-03T18:35:09.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.212+0000 7f062effd700 1 --2- 172.21.15.27:0/1442015290 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f06180640b0 0x7f0618066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:35:09.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.213+0000 7f062effd700 1 --2- 172.21.15.27:0/1442015290 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f06180640b0 0x7f0618066570 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7f0624004a90 tx=0x7f0624009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:35:09.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.216+0000 7f0617fff700 1 -- 172.21.15.27:0/1442015290 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f0620054940 con 0x7f06300f55e0 2021-11-03T18:35:09.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:08 smithi115 ceph-mon[65453]: from='client.36584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:35:09.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:08 smithi115 ceph-mon[65453]: from='client.36588 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:35:09.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:08 smithi027 ceph-mon[97244]: from='client.36584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:35:09.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:08 smithi027 ceph-mon[97244]: from='client.36588 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:35:09.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.418+0000 7f0635b82700 1 -- 172.21.15.27:0/1442015290 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f063010f5d0 con 0x7f06300f55e0 2021-11-03T18:35:09.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.419+0000 7f0617fff700 1 -- 172.21.15.27:0/1442015290 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f0620054530 con 0x7f06300f55e0 2021-11-03T18:35:09.417 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:35:09.417 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:35:09.418 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:35:09.418 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:35:09.418 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:35:09.419 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:35:09.419 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:35:09.419 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:35:09.420 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:35:09.420 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:35:09.420 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:35:09.420 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:35:09.421 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:35:09.421 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:35:09.421 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:35:09.422 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:35:09.422 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:35:09.423 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:35:09.423 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:35:09.423 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:35:09.424 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:35:09.424 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:35:09.424 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:35:09.424 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:35:09.425 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:35:09.425 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:35:09.425 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:35:09.426 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:35:09.426 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:35:09.426 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:35:09.427 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:35:09.427 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:35:09.427 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:35:09.427 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:35:09.428 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:35:09.428 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:35:09.428 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:35:09.429 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:35:09.429 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:35:09.431 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.420+0000 7f0635b82700 1 -- 172.21.15.27:0/1442015290 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f06180640b0 msgr2=0x7f0618066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:35:09.431 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.420+0000 7f0635b82700 1 --2- 172.21.15.27:0/1442015290 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f06180640b0 0x7f0618066570 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7f0624004a90 tx=0x7f0624009040).stop 2021-11-03T18:35:09.432 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.420+0000 7f0635b82700 1 -- 172.21.15.27:0/1442015290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06300f55e0 msgr2=0x7f06300f97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:35:09.433 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.420+0000 7f0635b82700 1 --2- 172.21.15.27:0/1442015290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06300f55e0 0x7f06300f97e0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 rx=0x7f0620009990 tx=0x7f0620009b30).stop 2021-11-03T18:35:09.433 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.420+0000 7f0635b82700 1 -- 172.21.15.27:0/1442015290 shutdown_connections 2021-11-03T18:35:09.434 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.420+0000 7f0635b82700 1 --2- 172.21.15.27:0/1442015290 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f06180640b0 0x7f0618066570 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:09.434 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.420+0000 7f0635b82700 1 --2- 172.21.15.27:0/1442015290 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f06300f55e0 0x7f06300f97e0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:09.434 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.420+0000 7f0635b82700 1 --2- 172.21.15.27:0/1442015290 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f06300f5ef0 0x7f06300f7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:09.436 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.420+0000 7f0635b82700 1 -- 172.21.15.27:0/1442015290 >> 172.21.15.27:0/1442015290 conn(0x7f06300f0de0 msgr2=0x7f063005ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:35:09.436 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.421+0000 7f0635b82700 1 -- 172.21.15.27:0/1442015290 shutdown_connections 2021-11-03T18:35:09.436 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:09.421+0000 7f0635b82700 1 -- 172.21.15.27:0/1442015290 wait complete. 2021-11-03T18:35:09.436 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:35:10.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:09 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1974934674' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:10.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:09 smithi115 ceph-mon[65453]: pgmap v2417: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:10.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:09 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1442015290' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:35:10.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:09 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1974934674' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:10.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:09 smithi027 ceph-mon[97244]: pgmap v2417: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:10.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:09 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1442015290' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:35:11.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4752.473846 secs 2021-11-03T18:35:11.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4752.473846 secs 2021-11-03T18:35:12.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:11 smithi115 ceph-mon[65453]: pgmap v2418: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:12.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:11 smithi027 ceph-mon[97244]: pgmap v2418: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:14.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:14 smithi115 ceph-mon[65453]: pgmap v2419: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:14.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:14 smithi027 ceph-mon[97244]: pgmap v2419: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:15.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4757.473968 secs 2021-11-03T18:35:15.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4757.473968 secs 2021-11-03T18:35:16.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:16 smithi115 ceph-mon[65453]: pgmap v2420: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:16.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:16 smithi027 ceph-mon[97244]: pgmap v2420: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:17.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:35:17.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:35:17.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:35:17.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:17.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:17.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:17.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:17.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:35:17.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:35:17.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:35:17.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:17.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:17.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:17.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:18.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:18 smithi027 ceph-mon[97244]: pgmap v2421: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:18.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:18 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:35:18.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:18 smithi115 ceph-mon[65453]: pgmap v2421: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:18.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:18 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:35:20.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:20 smithi027 ceph-mon[97244]: pgmap v2422: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:20.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4762.474095 secs 2021-11-03T18:35:20.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:20 smithi115 ceph-mon[65453]: pgmap v2422: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:20.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4762.474095 secs 2021-11-03T18:35:22.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:22 smithi027 ceph-mon[97244]: pgmap v2423: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:22.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:22 smithi115 ceph-mon[65453]: pgmap v2423: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:24.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:24 smithi027 ceph-mon[97244]: pgmap v2424: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:24.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:24 smithi115 ceph-mon[65453]: pgmap v2424: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:25.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4767.474217 secs 2021-11-03T18:35:25.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4767.474217 secs 2021-11-03T18:35:26.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:26 smithi027 ceph-mon[97244]: pgmap v2425: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:26.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:26 smithi115 ceph-mon[65453]: pgmap v2425: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:27.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:35:27.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:35:27.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:35:27.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:27.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:27.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:27.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:27.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:35:27.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:35:27.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:35:27.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:27.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:27.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:27.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:28.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:28 smithi027 ceph-mon[97244]: pgmap v2426: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:28.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:28 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:35:28.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:28 smithi115 ceph-mon[65453]: pgmap v2426: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:28.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:28 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:35:30.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:30 smithi027 ceph-mon[97244]: pgmap v2427: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:30.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4772.474298 secs 2021-11-03T18:35:30.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:30 smithi115 ceph-mon[65453]: pgmap v2427: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:30.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4772.474298 secs 2021-11-03T18:35:32.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:32 smithi027 ceph-mon[97244]: pgmap v2428: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:32.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:32 smithi115 ceph-mon[65453]: pgmap v2428: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:34.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:34 smithi115 ceph-mon[65453]: pgmap v2429: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:35.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:34 smithi027 ceph-mon[97244]: pgmap v2429: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:35.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4777.474451 secs 2021-11-03T18:35:36.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4777.474451 secs 2021-11-03T18:35:36.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:36 smithi115 ceph-mon[65453]: pgmap v2430: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:37.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:36 smithi027 ceph-mon[97244]: pgmap v2430: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:37.665 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:35:37.684 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:35:38.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:38 smithi115 ceph-mon[65453]: pgmap v2431: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:38.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:38 smithi115 ceph-mon[65453]: poll took 60170.635 ms: 1 events 2021-11-03T18:35:38.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:35:38.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:35:38.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:38.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:38.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:38.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:38.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:38 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:35:39.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:38 smithi027 ceph-mon[97244]: pgmap v2431: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:39.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:38 smithi027 ceph-mon[97244]: poll took 60170.635 ms: 1 events 2021-11-03T18:35:39.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:35:39.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:35:39.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:39.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:39.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:39.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:39.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:38 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:35:39.535 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.534+0000 7f95e5691700 1 -- 172.21.15.27:0/867884565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95e00f6530 msgr2=0x7f95e00f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:35:39.536 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.534+0000 7f95e5691700 1 --2- 172.21.15.27:0/867884565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95e00f6530 0x7f95e00f6950 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 rx=0x7f95d0004660 tx=0x7f95d0009b30).stop 2021-11-03T18:35:39.536 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.535+0000 7f95e5691700 1 -- 172.21.15.27:0/867884565 shutdown_connections 2021-11-03T18:35:39.536 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.535+0000 7f95e5691700 1 --2- 172.21.15.27:0/867884565 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95e00f77d0 0x7f95e00f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:39.536 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.535+0000 7f95e5691700 1 --2- 172.21.15.27:0/867884565 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95e00f6530 0x7f95e00f6950 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:39.537 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.535+0000 7f95e5691700 1 -- 172.21.15.27:0/867884565 >> 172.21.15.27:0/867884565 conn(0x7f95e00f1cd0 msgr2=0x7f95e00f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:35:39.537 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.535+0000 7f95e5691700 1 -- 172.21.15.27:0/867884565 shutdown_connections 2021-11-03T18:35:39.537 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.535+0000 7f95e5691700 1 -- 172.21.15.27:0/867884565 wait complete. 2021-11-03T18:35:39.537 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.535+0000 7f95e5691700 1 Processor -- start 2021-11-03T18:35:39.538 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.535+0000 7f95e5691700 1 -- start start 2021-11-03T18:35:39.538 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.536+0000 7f95e5691700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95e00f6530 0x7f95e0106db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:35:39.538 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.536+0000 7f95e5691700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95e00f77d0 0x7f95e01072a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:35:39.538 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.536+0000 7f95e5691700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e01078a0 con 0x7f95e00f6530 2021-11-03T18:35:39.539 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.536+0000 7f95e5691700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95e01080e0 con 0x7f95e00f77d0 2021-11-03T18:35:39.539 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.536+0000 7f95dffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95e00f6530 0x7f95e0106db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:35:39.539 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.536+0000 7f95dffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95e00f6530 0x7f95e0106db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35732/0 (socket says 172.21.15.27:35732) 2021-11-03T18:35:39.539 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.536+0000 7f95dffff700 1 -- 172.21.15.27:0/488064202 learned_addr learned my addr 172.21.15.27:0/488064202 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:35:39.539 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.536+0000 7f95df7fe700 1 --2- 172.21.15.27:0/488064202 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95e00f77d0 0x7f95e01072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:35:39.540 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.536+0000 7f95dffff700 1 -- 172.21.15.27:0/488064202 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95e00f77d0 msgr2=0x7f95e01072a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:35:39.540 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.536+0000 7f95dffff700 1 --2- 172.21.15.27:0/488064202 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95e00f77d0 0x7f95e01072a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:39.540 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.536+0000 7f95dffff700 1 -- 172.21.15.27:0/488064202 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95d0005040 con 0x7f95e00f6530 2021-11-03T18:35:39.540 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.536+0000 7f95dffff700 1 --2- 172.21.15.27:0/488064202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95e00f6530 0x7f95e0106db0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 rx=0x7f95d0004f60 tx=0x7f95d0009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:35:39.541 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.536+0000 7f95dcff9700 1 -- 172.21.15.27:0/488064202 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95d0018070 con 0x7f95e00f6530 2021-11-03T18:35:39.541 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.536+0000 7f95dcff9700 1 -- 172.21.15.27:0/488064202 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f95d0009770 con 0x7f95e00f6530 2021-11-03T18:35:39.541 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.536+0000 7f95e5691700 1 -- 172.21.15.27:0/488064202 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95e0108360 con 0x7f95e00f6530 2021-11-03T18:35:39.541 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.536+0000 7f95dcff9700 1 -- 172.21.15.27:0/488064202 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f95d001c3c0 con 0x7f95e00f6530 2021-11-03T18:35:39.542 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.536+0000 7f95e5691700 1 -- 172.21.15.27:0/488064202 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95e0108840 con 0x7f95e00f6530 2021-11-03T18:35:39.543 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.538+0000 7f95e5691700 1 -- 172.21.15.27:0/488064202 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95e0059100 con 0x7f95e00f6530 2021-11-03T18:35:39.543 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.538+0000 7f95dcff9700 1 -- 172.21.15.27:0/488064202 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f95d001d420 con 0x7f95e00f6530 2021-11-03T18:35:39.543 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.538+0000 7f95dcff9700 1 --2- 172.21.15.27:0/488064202 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f95c8064100 0x7f95c80665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:35:39.543 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.539+0000 7f95dcff9700 1 -- 172.21.15.27:0/488064202 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f95d0082580 con 0x7f95e00f6530 2021-11-03T18:35:39.544 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.539+0000 7f95df7fe700 1 --2- 172.21.15.27:0/488064202 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f95c8064100 0x7f95c80665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:35:39.544 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.539+0000 7f95df7fe700 1 --2- 172.21.15.27:0/488064202 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f95c8064100 0x7f95c80665c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7f95d4007fd0 tx=0x7f95d4009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:35:39.544 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.543+0000 7f95dcff9700 1 -- 172.21.15.27:0/488064202 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f95d00549a0 con 0x7f95e00f6530 2021-11-03T18:35:39.721 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.723+0000 7f95e5691700 1 -- 172.21.15.27:0/488064202 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f95e00fecf0 con 0x7f95c8064100 2021-11-03T18:35:39.722 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.724+0000 7f95dcff9700 1 -- 172.21.15.27:0/488064202 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f95e00fecf0 con 0x7f95c8064100 2021-11-03T18:35:39.724 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.726+0000 7f95e5691700 1 -- 172.21.15.27:0/488064202 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f95c8064100 msgr2=0x7f95c80665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:35:39.725 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.726+0000 7f95e5691700 1 --2- 172.21.15.27:0/488064202 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f95c8064100 0x7f95c80665c0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 rx=0x7f95d4007fd0 tx=0x7f95d4009040).stop 2021-11-03T18:35:39.725 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.726+0000 7f95e5691700 1 -- 172.21.15.27:0/488064202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95e00f6530 msgr2=0x7f95e0106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:35:39.725 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.726+0000 7f95e5691700 1 --2- 172.21.15.27:0/488064202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95e00f6530 0x7f95e0106db0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 rx=0x7f95d0004f60 tx=0x7f95d0009b30).stop 2021-11-03T18:35:39.725 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.726+0000 7f95e5691700 1 -- 172.21.15.27:0/488064202 shutdown_connections 2021-11-03T18:35:39.726 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.726+0000 7f95e5691700 1 --2- 172.21.15.27:0/488064202 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f95c8064100 0x7f95c80665c0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:39.726 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.726+0000 7f95e5691700 1 --2- 172.21.15.27:0/488064202 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f95e00f6530 0x7f95e0106db0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:39.726 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.726+0000 7f95e5691700 1 --2- 172.21.15.27:0/488064202 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f95e00f77d0 0x7f95e01072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:39.726 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.726+0000 7f95e5691700 1 -- 172.21.15.27:0/488064202 >> 172.21.15.27:0/488064202 conn(0x7f95e00f1cd0 msgr2=0x7f95e00fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:35:39.727 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.726+0000 7f95e5691700 1 -- 172.21.15.27:0/488064202 shutdown_connections 2021-11-03T18:35:39.727 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.726+0000 7f95e5691700 1 -- 172.21.15.27:0/488064202 wait complete. 2021-11-03T18:35:39.734 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:35:39.827 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.826+0000 7f90200b0700 1 -- 172.21.15.27:0/1154562438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90180f7dc0 msgr2=0x7f90180f81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:35:39.827 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.826+0000 7f90200b0700 1 --2- 172.21.15.27:0/1154562438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90180f7dc0 0x7f90180f81e0 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 rx=0x7f9014004660 tx=0x7f9014009b30).stop 2021-11-03T18:35:39.827 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.826+0000 7f90200b0700 1 -- 172.21.15.27:0/1154562438 shutdown_connections 2021-11-03T18:35:39.828 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.826+0000 7f90200b0700 1 --2- 172.21.15.27:0/1154562438 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90180f86d0 0x7f90180f48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:39.828 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.826+0000 7f90200b0700 1 --2- 172.21.15.27:0/1154562438 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90180f7dc0 0x7f90180f81e0 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:39.828 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.826+0000 7f90200b0700 1 -- 172.21.15.27:0/1154562438 >> 172.21.15.27:0/1154562438 conn(0x7f90180f0540 msgr2=0x7f90180f2960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:35:39.828 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.827+0000 7f90200b0700 1 -- 172.21.15.27:0/1154562438 shutdown_connections 2021-11-03T18:35:39.829 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.827+0000 7f90200b0700 1 -- 172.21.15.27:0/1154562438 wait complete. 2021-11-03T18:35:39.829 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.827+0000 7f90200b0700 1 Processor -- start 2021-11-03T18:35:39.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.827+0000 7f90200b0700 1 -- start start 2021-11-03T18:35:39.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.827+0000 7f90200b0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90180f7dc0 0x7f9018106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:35:39.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.827+0000 7f90200b0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90180f86d0 0x7f9018107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:35:39.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.827+0000 7f90200b0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9018107a10 con 0x7f90180f7dc0 2021-11-03T18:35:39.831 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.827+0000 7f90200b0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9018107b50 con 0x7f90180f86d0 2021-11-03T18:35:39.831 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.827+0000 7f901de4c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90180f7dc0 0x7f9018106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:35:39.831 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.827+0000 7f901de4c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90180f7dc0 0x7f9018106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35742/0 (socket says 172.21.15.27:35742) 2021-11-03T18:35:39.831 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.827+0000 7f901de4c700 1 -- 172.21.15.27:0/3013391688 learned_addr learned my addr 172.21.15.27:0/3013391688 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:35:39.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.828+0000 7f901d64b700 1 --2- 172.21.15.27:0/3013391688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90180f86d0 0x7f9018107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:35:39.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.828+0000 7f901de4c700 1 -- 172.21.15.27:0/3013391688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90180f86d0 msgr2=0x7f9018107460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:35:39.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.828+0000 7f901de4c700 1 --2- 172.21.15.27:0/3013391688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90180f86d0 0x7f9018107460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:39.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.828+0000 7f901de4c700 1 -- 172.21.15.27:0/3013391688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9014005040 con 0x7f90180f7dc0 2021-11-03T18:35:39.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.828+0000 7f901de4c700 1 --2- 172.21.15.27:0/3013391688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90180f7dc0 0x7f9018106f70 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 rx=0x7f9014009990 tx=0x7f9014009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:35:39.833 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.828+0000 7f900e7fc700 1 -- 172.21.15.27:0/3013391688 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9014018070 con 0x7f90180f7dc0 2021-11-03T18:35:39.833 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.828+0000 7f90200b0700 1 -- 172.21.15.27:0/3013391688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9018108490 con 0x7f90180f7dc0 2021-11-03T18:35:39.833 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.828+0000 7f900e7fc700 1 -- 172.21.15.27:0/3013391688 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f9014007ba0 con 0x7f90180f7dc0 2021-11-03T18:35:39.833 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.828+0000 7f900e7fc700 1 -- 172.21.15.27:0/3013391688 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f901401c3c0 con 0x7f90180f7dc0 2021-11-03T18:35:39.834 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.828+0000 7f90200b0700 1 -- 172.21.15.27:0/3013391688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90181089f0 con 0x7f90180f7dc0 2021-11-03T18:35:39.835 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.829+0000 7f90200b0700 1 -- 172.21.15.27:0/3013391688 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9018045ad0 con 0x7f90180f7dc0 2021-11-03T18:35:39.835 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.830+0000 7f900e7fc700 1 -- 172.21.15.27:0/3013391688 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f901401d420 con 0x7f90180f7dc0 2021-11-03T18:35:39.836 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.830+0000 7f900e7fc700 1 --2- 172.21.15.27:0/3013391688 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f90040640b0 0x7f9004066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:35:39.836 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.830+0000 7f900e7fc700 1 -- 172.21.15.27:0/3013391688 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f9014082360 con 0x7f90180f7dc0 2021-11-03T18:35:39.836 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.830+0000 7f901d64b700 1 --2- 172.21.15.27:0/3013391688 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f90040640b0 0x7f9004066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:35:39.836 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.830+0000 7f901d64b700 1 --2- 172.21.15.27:0/3013391688 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f90040640b0 0x7f9004066570 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f9008004c40 tx=0x7f900800b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:35:39.837 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:39.834+0000 7f900e7fc700 1 -- 172.21.15.27:0/3013391688 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f9014054720 con 0x7f90180f7dc0 2021-11-03T18:35:40.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.008+0000 7f90200b0700 1 -- 172.21.15.27:0/3013391688 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f90180fefc0 con 0x7f90040640b0 2021-11-03T18:35:40.021 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.023+0000 7f900e7fc700 1 -- 172.21.15.27:0/3013391688 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f90180fefc0 con 0x7f90040640b0 2021-11-03T18:35:40.021 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:35:40.022 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (87m) 5m ago 88m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:35:40.022 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (78m) 5m ago 88m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:35:40.022 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (78m) 5m ago 87m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:35:40.022 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (87m) 5m ago 88m 42.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:35:40.023 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (80m) 5m ago 83m 26.7M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:35:40.023 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (83m) 5m ago 83m 3850M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:35:40.023 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (79m) 5m ago 83m 26.3M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:35:40.023 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (81m) 5m ago 83m 27.4M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:35:40.024 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (80m) 5m ago 89m 608M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:35:40.024 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (80m) 5m ago 87m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:35:40.024 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (79m) 5m ago 89m 350M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:35:40.024 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (79m) 5m ago 87m 160M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:35:40.025 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (88m) 5m ago 88m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:35:40.025 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (87m) 5m ago 87m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:35:40.025 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (78m) 5m ago 86m 413M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:35:40.025 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (78m) 5m ago 86m 450M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:35:40.025 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (77m) 5m ago 86m 338M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:35:40.026 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (77m) 5m ago 85m 479M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:35:40.026 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (76m) 5m ago 85m 378M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:35:40.026 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (76m) 5m ago 85m 388M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:35:40.026 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (86m) 5m ago 87m 64.4M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:35:40.028 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.025+0000 7f90200b0700 1 -- 172.21.15.27:0/3013391688 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f90040640b0 msgr2=0x7f9004066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:35:40.028 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.025+0000 7f90200b0700 1 --2- 172.21.15.27:0/3013391688 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f90040640b0 0x7f9004066570 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f9008004c40 tx=0x7f900800b040).stop 2021-11-03T18:35:40.028 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.025+0000 7f90200b0700 1 -- 172.21.15.27:0/3013391688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90180f7dc0 msgr2=0x7f9018106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:35:40.029 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.025+0000 7f90200b0700 1 --2- 172.21.15.27:0/3013391688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90180f7dc0 0x7f9018106f70 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 rx=0x7f9014009990 tx=0x7f9014009b30).stop 2021-11-03T18:35:40.029 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.025+0000 7f90200b0700 1 -- 172.21.15.27:0/3013391688 shutdown_connections 2021-11-03T18:35:40.029 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.025+0000 7f90200b0700 1 --2- 172.21.15.27:0/3013391688 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f90040640b0 0x7f9004066570 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:40.029 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.025+0000 7f90200b0700 1 --2- 172.21.15.27:0/3013391688 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f90180f7dc0 0x7f9018106f70 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:40.029 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.025+0000 7f90200b0700 1 --2- 172.21.15.27:0/3013391688 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f90180f86d0 0x7f9018107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:40.030 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.025+0000 7f90200b0700 1 -- 172.21.15.27:0/3013391688 >> 172.21.15.27:0/3013391688 conn(0x7f90180f0540 msgr2=0x7f90180fb650 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:35:40.030 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.025+0000 7f90200b0700 1 -- 172.21.15.27:0/3013391688 shutdown_connections 2021-11-03T18:35:40.030 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.025+0000 7f90200b0700 1 -- 172.21.15.27:0/3013391688 wait complete. 2021-11-03T18:35:40.123 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.124+0000 7f9a1b4c5700 1 -- 172.21.15.27:0/2471144942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a140f7980 msgr2=0x7f9a140f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:35:40.124 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.124+0000 7f9a1b4c5700 1 --2- 172.21.15.27:0/2471144942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a140f7980 0x7f9a140f7de0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 rx=0x7f9a04004660 tx=0x7f9a04009b30).stop 2021-11-03T18:35:40.124 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.124+0000 7f9a1b4c5700 1 -- 172.21.15.27:0/2471144942 shutdown_connections 2021-11-03T18:35:40.124 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.124+0000 7f9a1b4c5700 1 --2- 172.21.15.27:0/2471144942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a140f7980 0x7f9a140f7de0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:40.124 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.124+0000 7f9a1b4c5700 1 --2- 172.21.15.27:0/2471144942 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a140f66e0 0x7f9a140f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:40.125 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.124+0000 7f9a1b4c5700 1 -- 172.21.15.27:0/2471144942 >> 172.21.15.27:0/2471144942 conn(0x7f9a140f1e40 msgr2=0x7f9a140f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:35:40.125 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.124+0000 7f9a1b4c5700 1 -- 172.21.15.27:0/2471144942 shutdown_connections 2021-11-03T18:35:40.125 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.124+0000 7f9a1b4c5700 1 -- 172.21.15.27:0/2471144942 wait complete. 2021-11-03T18:35:40.125 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.125+0000 7f9a1b4c5700 1 Processor -- start 2021-11-03T18:35:40.126 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.125+0000 7f9a1b4c5700 1 -- start start 2021-11-03T18:35:40.126 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.125+0000 7f9a1b4c5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a140f66e0 0x7f9a14104d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:35:40.126 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.125+0000 7f9a1b4c5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a140f7980 0x7f9a14105200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:35:40.126 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.125+0000 7f9a1b4c5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a14105800 con 0x7f9a140f66e0 2021-11-03T18:35:40.127 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.125+0000 7f9a1b4c5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a14105940 con 0x7f9a140f7980 2021-11-03T18:35:40.127 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.125+0000 7f9a18a60700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a140f7980 0x7f9a14105200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:35:40.127 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.125+0000 7f9a19261700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a140f66e0 0x7f9a14104d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:35:40.127 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.125+0000 7f9a18a60700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a140f7980 0x7f9a14105200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.115:3300/0 says I am v2:172.21.15.27:55152/0 (socket says 172.21.15.27:55152) 2021-11-03T18:35:40.128 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.125+0000 7f9a18a60700 1 -- 172.21.15.27:0/6198883 learned_addr learned my addr 172.21.15.27:0/6198883 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:35:40.129 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.126+0000 7f9a18a60700 1 -- 172.21.15.27:0/6198883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a140f66e0 msgr2=0x7f9a14104d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:35:40.130 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.126+0000 7f9a18a60700 1 --2- 172.21.15.27:0/6198883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a140f66e0 0x7f9a14104d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:40.130 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.126+0000 7f9a18a60700 1 -- 172.21.15.27:0/6198883 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a04005040 con 0x7f9a140f7980 2021-11-03T18:35:40.130 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.126+0000 7f9a18a60700 1 --2- 172.21.15.27:0/6198883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a140f7980 0x7f9a14105200 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f9a04009990 tx=0x7f9a04009b30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:35:40.130 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.126+0000 7f9a09ffb700 1 -- 172.21.15.27:0/6198883 <== mon.1 v2:172.21.15.115:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a04018070 con 0x7f9a140f7980 2021-11-03T18:35:40.131 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.126+0000 7f9a1b4c5700 1 -- 172.21.15.27:0/6198883 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a14106250 con 0x7f9a140f7980 2021-11-03T18:35:40.131 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.126+0000 7f9a09ffb700 1 -- 172.21.15.27:0/6198883 <== mon.1 v2:172.21.15.115:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f9a04007ba0 con 0x7f9a140f7980 2021-11-03T18:35:40.131 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.126+0000 7f9a09ffb700 1 -- 172.21.15.27:0/6198883 <== mon.1 v2:172.21.15.115:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9a0401c3c0 con 0x7f9a140f7980 2021-11-03T18:35:40.131 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.126+0000 7f9a1b4c5700 1 -- 172.21.15.27:0/6198883 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a141067b0 con 0x7f9a140f7980 2021-11-03T18:35:40.132 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.128+0000 7f9a09ffb700 1 -- 172.21.15.27:0/6198883 <== mon.1 v2:172.21.15.115:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f9a0401d420 con 0x7f9a140f7980 2021-11-03T18:35:40.132 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.128+0000 7f9a09ffb700 1 --2- 172.21.15.27:0/6198883 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f99f8064040 0x7f99f8066500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:35:40.132 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.128+0000 7f9a09ffb700 1 -- 172.21.15.27:0/6198883 <== mon.1 v2:172.21.15.115:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f9a04082360 con 0x7f9a140f7980 2021-11-03T18:35:40.132 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.129+0000 7f9a19261700 1 --2- 172.21.15.27:0/6198883 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f99f8064040 0x7f99f8066500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:35:40.132 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.129+0000 7f9a1b4c5700 1 -- 172.21.15.27:0/6198883 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a14045ad0 con 0x7f9a140f7980 2021-11-03T18:35:40.133 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.129+0000 7f9a19261700 1 --2- 172.21.15.27:0/6198883 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f99f8064040 0x7f99f8066500 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f9a14104aa0 tx=0x7f9a1000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:35:40.133 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.134+0000 7f9a09ffb700 1 -- 172.21.15.27:0/6198883 <== mon.1 v2:172.21.15.115:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f9a04054720 con 0x7f9a140f7980 2021-11-03T18:35:40.360 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.362+0000 7f9a1b4c5700 1 -- 172.21.15.27:0/6198883 --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f9a140fccd0 con 0x7f9a140f7980 2021-11-03T18:35:40.361 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.363+0000 7f9a09ffb700 1 -- 172.21.15.27:0/6198883 <== mon.1 v2:172.21.15.115:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f9a04054310 con 0x7f9a140f7980 2021-11-03T18:35:40.361 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:35:40.362 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:35:40.362 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:35:40.362 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:35:40.362 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:35:40.363 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:35:40.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:35:40.363 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:35:40.363 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:35:40.363 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:35:40.364 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:35:40.364 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:35:40.364 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:35:40.364 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:35:40.365 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:35:40.365 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:35:40.365 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:35:40.365 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:35:40.366 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.365+0000 7f9a1b4c5700 1 -- 172.21.15.27:0/6198883 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f99f8064040 msgr2=0x7f99f8066500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:35:40.367 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.365+0000 7f9a1b4c5700 1 --2- 172.21.15.27:0/6198883 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f99f8064040 0x7f99f8066500 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f9a14104aa0 tx=0x7f9a1000b040).stop 2021-11-03T18:35:40.367 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.365+0000 7f9a1b4c5700 1 -- 172.21.15.27:0/6198883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a140f7980 msgr2=0x7f9a14105200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:35:40.367 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.365+0000 7f9a1b4c5700 1 --2- 172.21.15.27:0/6198883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a140f7980 0x7f9a14105200 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f9a04009990 tx=0x7f9a04009b30).stop 2021-11-03T18:35:40.367 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.365+0000 7f9a1b4c5700 1 -- 172.21.15.27:0/6198883 shutdown_connections 2021-11-03T18:35:40.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.365+0000 7f9a1b4c5700 1 --2- 172.21.15.27:0/6198883 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f99f8064040 0x7f99f8066500 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:40.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.365+0000 7f9a1b4c5700 1 --2- 172.21.15.27:0/6198883 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9a140f66e0 0x7f9a14104d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:40.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.365+0000 7f9a1b4c5700 1 --2- 172.21.15.27:0/6198883 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9a140f7980 0x7f9a14105200 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:40.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.365+0000 7f9a1b4c5700 1 -- 172.21.15.27:0/6198883 >> 172.21.15.27:0/6198883 conn(0x7f9a140f1e40 msgr2=0x7f9a140fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:35:40.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.365+0000 7f9a1b4c5700 1 -- 172.21.15.27:0/6198883 shutdown_connections 2021-11-03T18:35:40.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.365+0000 7f9a1b4c5700 1 -- 172.21.15.27:0/6198883 wait complete. 2021-11-03T18:35:40.465 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.464+0000 7f4d46940700 1 -- 172.21.15.27:0/405521475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d400f5440 msgr2=0x7f4d400f5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:35:40.465 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.464+0000 7f4d46940700 1 --2- 172.21.15.27:0/405521475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d400f5440 0x7f4d400f5860 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 rx=0x7f4d3c004660 tx=0x7f4d3c009b30).stop 2021-11-03T18:35:40.465 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.464+0000 7f4d46940700 1 -- 172.21.15.27:0/405521475 shutdown_connections 2021-11-03T18:35:40.466 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.464+0000 7f4d46940700 1 --2- 172.21.15.27:0/405521475 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d400f66e0 0x7f4d400f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:40.466 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.464+0000 7f4d46940700 1 --2- 172.21.15.27:0/405521475 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d400f5440 0x7f4d400f5860 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:40.466 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.464+0000 7f4d46940700 1 -- 172.21.15.27:0/405521475 >> 172.21.15.27:0/405521475 conn(0x7f4d400f0ba0 msgr2=0x7f4d400f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:35:40.466 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.465+0000 7f4d46940700 1 -- 172.21.15.27:0/405521475 shutdown_connections 2021-11-03T18:35:40.467 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.465+0000 7f4d46940700 1 -- 172.21.15.27:0/405521475 wait complete. 2021-11-03T18:35:40.467 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.465+0000 7f4d46940700 1 Processor -- start 2021-11-03T18:35:40.467 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.465+0000 7f4d46940700 1 -- start start 2021-11-03T18:35:40.467 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.465+0000 7f4d46940700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d400f5440 0x7f4d40109aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:35:40.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.465+0000 7f4d46940700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d400f66e0 0x7f4d4010bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:35:40.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.465+0000 7f4d46940700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d400603e0 con 0x7f4d400f66e0 2021-11-03T18:35:40.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.465+0000 7f4d46940700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d40060520 con 0x7f4d400f5440 2021-11-03T18:35:40.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.465+0000 7f4d4513d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d400f66e0 0x7f4d4010bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:35:40.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.465+0000 7f4d4513d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d400f66e0 0x7f4d4010bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35764/0 (socket says 172.21.15.27:35764) 2021-11-03T18:35:40.469 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.466+0000 7f4d4513d700 1 -- 172.21.15.27:0/517002786 learned_addr learned my addr 172.21.15.27:0/517002786 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:35:40.469 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.466+0000 7f4d4593e700 1 --2- 172.21.15.27:0/517002786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d400f5440 0x7f4d40109aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:35:40.469 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.466+0000 7f4d4513d700 1 -- 172.21.15.27:0/517002786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d400f5440 msgr2=0x7f4d40109aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:35:40.469 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.466+0000 7f4d4513d700 1 --2- 172.21.15.27:0/517002786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d400f5440 0x7f4d40109aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:40.470 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.466+0000 7f4d4513d700 1 -- 172.21.15.27:0/517002786 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d3c005040 con 0x7f4d400f66e0 2021-11-03T18:35:40.470 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.466+0000 7f4d4513d700 1 --2- 172.21.15.27:0/517002786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d400f66e0 0x7f4d4010bfa0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 rx=0x7f4d3000c530 tx=0x7f4d30007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:35:40.470 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.466+0000 7f4d367fc700 1 -- 172.21.15.27:0/517002786 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d3000f650 con 0x7f4d400f66e0 2021-11-03T18:35:40.470 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.466+0000 7f4d367fc700 1 -- 172.21.15.27:0/517002786 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f4d3000fc60 con 0x7f4d400f66e0 2021-11-03T18:35:40.471 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.466+0000 7f4d46940700 1 -- 172.21.15.27:0/517002786 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d4010c550 con 0x7f4d400f66e0 2021-11-03T18:35:40.471 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.466+0000 7f4d367fc700 1 -- 172.21.15.27:0/517002786 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4d30012c20 con 0x7f4d400f66e0 2021-11-03T18:35:40.471 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.466+0000 7f4d46940700 1 -- 172.21.15.27:0/517002786 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d4010cb10 con 0x7f4d400f66e0 2021-11-03T18:35:40.471 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.467+0000 7f4d46940700 1 -- 172.21.15.27:0/517002786 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d40045ad0 con 0x7f4d400f66e0 2021-11-03T18:35:40.472 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.468+0000 7f4d367fc700 1 -- 172.21.15.27:0/517002786 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f4d30010070 con 0x7f4d400f66e0 2021-11-03T18:35:40.472 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.468+0000 7f4d367fc700 1 --2- 172.21.15.27:0/517002786 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4d2c064100 0x7f4d2c0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:35:40.473 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.468+0000 7f4d367fc700 1 -- 172.21.15.27:0/517002786 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f4d30080550 con 0x7f4d400f66e0 2021-11-03T18:35:40.473 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.468+0000 7f4d4593e700 1 --2- 172.21.15.27:0/517002786 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4d2c064100 0x7f4d2c0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:35:40.473 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.468+0000 7f4d4593e700 1 --2- 172.21.15.27:0/517002786 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4d2c064100 0x7f4d2c0665c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7f4d3c013690 tx=0x7f4d3c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:35:40.473 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.472+0000 7f4d367fc700 1 -- 172.21.15.27:0/517002786 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f4d300527c0 con 0x7f4d400f66e0 2021-11-03T18:35:40.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.675+0000 7f4d46940700 1 -- 172.21.15.27:0/517002786 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4d40027a00 con 0x7f4d400f66e0 2021-11-03T18:35:40.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.675+0000 7f4d367fc700 1 -- 172.21.15.27:0/517002786 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f4d300523b0 con 0x7f4d400f66e0 2021-11-03T18:35:40.675 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:35:40.676 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:35:40.676 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:35:40.676 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:35:40.676 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:35:40.677 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:35:40.677 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:35:40.677 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:35:40.677 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:35:40.678 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:35:40.678 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:35:40.678 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:35:40.678 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:35:40.679 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:35:40.679 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:35:40.679 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:35:40.679 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:35:40.679 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:35:40.680 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:35:40.680 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:35:40.680 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:35:40.680 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:35:40.681 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:35:40.681 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:35:40.681 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:35:40.681 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:35:40.681 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:35:40.682 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:35:40.682 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:35:40.682 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:35:40.683 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:35:40.683 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:35:40.683 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:35:40.683 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:35:40.684 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:35:40.684 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:35:40.684 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:35:40.684 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:35:40.685 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:35:40.686 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.677+0000 7f4d46940700 1 -- 172.21.15.27:0/517002786 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4d2c064100 msgr2=0x7f4d2c0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:35:40.686 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.677+0000 7f4d46940700 1 --2- 172.21.15.27:0/517002786 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4d2c064100 0x7f4d2c0665c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 rx=0x7f4d3c013690 tx=0x7f4d3c009b30).stop 2021-11-03T18:35:40.686 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.677+0000 7f4d46940700 1 -- 172.21.15.27:0/517002786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d400f66e0 msgr2=0x7f4d4010bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:35:40.687 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.677+0000 7f4d46940700 1 --2- 172.21.15.27:0/517002786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d400f66e0 0x7f4d4010bfa0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 rx=0x7f4d3000c530 tx=0x7f4d30007e20).stop 2021-11-03T18:35:40.687 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.677+0000 7f4d46940700 1 -- 172.21.15.27:0/517002786 shutdown_connections 2021-11-03T18:35:40.687 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.677+0000 7f4d46940700 1 --2- 172.21.15.27:0/517002786 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4d2c064100 0x7f4d2c0665c0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:40.687 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.677+0000 7f4d46940700 1 --2- 172.21.15.27:0/517002786 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4d400f5440 0x7f4d40109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:40.687 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.677+0000 7f4d46940700 1 --2- 172.21.15.27:0/517002786 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4d400f66e0 0x7f4d4010bfa0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:35:40.688 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.677+0000 7f4d46940700 1 -- 172.21.15.27:0/517002786 >> 172.21.15.27:0/517002786 conn(0x7f4d400f0ba0 msgr2=0x7f4d400fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:35:40.688 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.677+0000 7f4d46940700 1 -- 172.21.15.27:0/517002786 shutdown_connections 2021-11-03T18:35:40.688 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:35:40.677+0000 7f4d46940700 1 -- 172.21.15.27:0/517002786 wait complete. 2021-11-03T18:35:40.688 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:35:40.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:40 smithi115 ceph-mon[65453]: pgmap v2432: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:40.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4782.474553 secs 2021-11-03T18:35:40.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:40 smithi115 ceph-mon[65453]: from='client.36600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:35:40.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:40 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/6198883' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:40.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:40 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/517002786' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:35:41.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:40 smithi027 ceph-mon[97244]: pgmap v2432: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:41.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4782.474553 secs 2021-11-03T18:35:41.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:40 smithi027 ceph-mon[97244]: from='client.36600 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:35:41.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:40 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/6198883' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:41.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:40 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/517002786' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:35:41.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:41 smithi115 ceph-mon[65453]: from='client.36604 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:35:42.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:41 smithi027 ceph-mon[97244]: from='client.36604 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:35:42.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:42 smithi115 ceph-mon[65453]: pgmap v2433: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:43.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:42 smithi027 ceph-mon[97244]: pgmap v2433: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:44.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:44 smithi115 ceph-mon[65453]: pgmap v2434: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:45.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:44 smithi027 ceph-mon[97244]: pgmap v2434: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:45.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4787.474679 secs 2021-11-03T18:35:46.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4787.474679 secs 2021-11-03T18:35:46.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:46 smithi115 ceph-mon[65453]: pgmap v2435: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:47.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:46 smithi027 ceph-mon[97244]: pgmap v2435: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:48.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:35:48.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:35:48.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:35:48.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:48.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:48.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:48.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:48.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:35:48.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:35:48.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:35:48.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:48.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:48.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:48.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:49.028 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:48 smithi027 ceph-mon[97244]: pgmap v2436: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:49.028 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:48 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:35:49.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:48 smithi115 ceph-mon[65453]: pgmap v2436: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:49.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:48 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:35:51.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:50 smithi027 ceph-mon[97244]: pgmap v2437: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:51.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4792.474806 secs 2021-11-03T18:35:51.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:50 smithi115 ceph-mon[65453]: pgmap v2437: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:51.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4792.474806 secs 2021-11-03T18:35:53.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:52 smithi115 ceph-mon[65453]: pgmap v2438: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:53.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:52 smithi027 ceph-mon[97244]: pgmap v2438: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:55.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:54 smithi115 ceph-mon[65453]: pgmap v2439: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:55.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:54 smithi027 ceph-mon[97244]: pgmap v2439: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:56.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4797.474927 secs 2021-11-03T18:35:56.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4797.474927 secs 2021-11-03T18:35:57.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:56 smithi115 ceph-mon[65453]: pgmap v2440: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:57.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:35:57.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:35:57.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:56 smithi027 ceph-mon[97244]: pgmap v2440: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:57.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:35:57.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:35:58.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:35:58.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:35:58.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:35:58.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:58.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:58.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:58.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:58.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:35:58.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:35:58.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:35:58.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:58.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:58.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:58.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:35:59.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:58 smithi115 ceph-mon[65453]: pgmap v2441: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:59.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:35:58 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:35:59.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:58 smithi027 ceph-mon[97244]: pgmap v2441: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:35:59.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:35:58 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:36:01.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:00 smithi115 ceph-mon[65453]: pgmap v2442: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:01.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4802.475070 secs 2021-11-03T18:36:01.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:00 smithi027 ceph-mon[97244]: pgmap v2442: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:01.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4802.475070 secs 2021-11-03T18:36:03.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:02 smithi115 ceph-mon[65453]: pgmap v2443: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:03.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:02 smithi027 ceph-mon[97244]: pgmap v2443: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:05.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:04 smithi115 ceph-mon[65453]: pgmap v2444: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:05.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:04 smithi027 ceph-mon[97244]: pgmap v2444: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:06.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4807.475196 secs 2021-11-03T18:36:06.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4807.475196 secs 2021-11-03T18:36:07.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:06 smithi115 ceph-mon[65453]: pgmap v2445: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:07.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:06 smithi027 ceph-mon[97244]: pgmap v2445: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:08.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:36:08.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:36:08.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:36:08.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:08.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:08.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:08.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:08.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:36:08.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:36:08.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:36:08.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:08.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:08.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:08.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:09.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:08 smithi115 ceph-mon[65453]: pgmap v2446: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:09.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:08 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:36:09.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:08 smithi027 ceph-mon[97244]: pgmap v2446: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:09.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:08 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:36:10.789 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.788+0000 7f03aa210700 1 -- 172.21.15.27:0/1561629936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03a40f9830 msgr2=0x7f03a40fbc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:36:10.789 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.788+0000 7f03aa210700 1 --2- 172.21.15.27:0/1561629936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03a40f9830 0x7f03a40fbc80 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 rx=0x7f038c004660 tx=0x7f038c009b30).stop 2021-11-03T18:36:10.789 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.788+0000 7f03aa210700 1 -- 172.21.15.27:0/1561629936 shutdown_connections 2021-11-03T18:36:10.790 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.788+0000 7f03aa210700 1 --2- 172.21.15.27:0/1561629936 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03a40f9830 0x7f03a40fbc80 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:10.790 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.788+0000 7f03aa210700 1 --2- 172.21.15.27:0/1561629936 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03a4060270 0x7f03a40f9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:10.790 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.788+0000 7f03aa210700 1 -- 172.21.15.27:0/1561629936 >> 172.21.15.27:0/1561629936 conn(0x7f03a40f0d40 msgr2=0x7f03a40f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:36:10.790 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.788+0000 7f03aa210700 1 -- 172.21.15.27:0/1561629936 shutdown_connections 2021-11-03T18:36:10.791 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.788+0000 7f03aa210700 1 -- 172.21.15.27:0/1561629936 wait complete. 2021-11-03T18:36:10.791 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.788+0000 7f03aa210700 1 Processor -- start 2021-11-03T18:36:10.791 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.789+0000 7f03aa210700 1 -- start start 2021-11-03T18:36:10.791 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.789+0000 7f03aa210700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03a4060270 0x7f03a40f6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:36:10.792 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.789+0000 7f03aa210700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03a40f9830 0x7f03a40f5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:36:10.792 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.789+0000 7f03aa210700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a40f57a0 con 0x7f03a40f9830 2021-11-03T18:36:10.792 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.789+0000 7f03aa210700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a40f58e0 con 0x7f03a4060270 2021-11-03T18:36:10.792 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.789+0000 7f03a2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03a40f9830 0x7f03a40f5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:36:10.793 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.789+0000 7f03a2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03a40f9830 0x7f03a40f5240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35782/0 (socket says 172.21.15.27:35782) 2021-11-03T18:36:10.793 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.789+0000 7f03a2ffd700 1 -- 172.21.15.27:0/874148649 learned_addr learned my addr 172.21.15.27:0/874148649 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:36:10.794 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.789+0000 7f03a37fe700 1 --2- 172.21.15.27:0/874148649 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03a4060270 0x7f03a40f6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:36:10.794 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.789+0000 7f03a2ffd700 1 -- 172.21.15.27:0/874148649 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03a4060270 msgr2=0x7f03a40f6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:36:10.794 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.789+0000 7f03a2ffd700 1 --2- 172.21.15.27:0/874148649 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03a4060270 0x7f03a40f6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:10.795 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.789+0000 7f03a2ffd700 1 -- 172.21.15.27:0/874148649 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f038c005040 con 0x7f03a40f9830 2021-11-03T18:36:10.795 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.789+0000 7f03a2ffd700 1 --2- 172.21.15.27:0/874148649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03a40f9830 0x7f03a40f5240 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 rx=0x7f038c013a70 tx=0x7f038c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:36:10.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.790+0000 7f039bfff700 1 -- 172.21.15.27:0/874148649 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f038c018070 con 0x7f03a40f9830 2021-11-03T18:36:10.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.790+0000 7f039bfff700 1 -- 172.21.15.27:0/874148649 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f038c007ba0 con 0x7f03a40f9830 2021-11-03T18:36:10.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.790+0000 7f03aa210700 1 -- 172.21.15.27:0/874148649 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a40f5b60 con 0x7f03a40f9830 2021-11-03T18:36:10.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.790+0000 7f039bfff700 1 -- 172.21.15.27:0/874148649 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f038c01c3c0 con 0x7f03a40f9830 2021-11-03T18:36:10.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.790+0000 7f03aa210700 1 -- 172.21.15.27:0/874148649 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a40f60c0 con 0x7f03a40f9830 2021-11-03T18:36:10.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.791+0000 7f03aa210700 1 -- 172.21.15.27:0/874148649 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03a4045ad0 con 0x7f03a40f9830 2021-11-03T18:36:10.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.791+0000 7f039bfff700 1 -- 172.21.15.27:0/874148649 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f038c01d420 con 0x7f03a40f9830 2021-11-03T18:36:10.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.792+0000 7f039bfff700 1 --2- 172.21.15.27:0/874148649 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f03900640b0 0x7f0390066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:36:10.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.792+0000 7f039bfff700 1 -- 172.21.15.27:0/874148649 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f038c082430 con 0x7f03a40f9830 2021-11-03T18:36:10.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.792+0000 7f03a37fe700 1 --2- 172.21.15.27:0/874148649 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f03900640b0 0x7f0390066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:36:10.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.792+0000 7f03a37fe700 1 --2- 172.21.15.27:0/874148649 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f03900640b0 0x7f0390066570 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7f03a40f6b50 tx=0x7f0394009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:36:10.800 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.796+0000 7f039bfff700 1 -- 172.21.15.27:0/874148649 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f038c054850 con 0x7f03a40f9830 2021-11-03T18:36:10.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.976+0000 7f03aa210700 1 -- 172.21.15.27:0/874148649 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f03a4057f10 con 0x7f03900640b0 2021-11-03T18:36:10.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.977+0000 7f039bfff700 1 -- 172.21.15.27:0/874148649 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f03a4057f10 con 0x7f03900640b0 2021-11-03T18:36:10.977 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.979+0000 7f03aa210700 1 -- 172.21.15.27:0/874148649 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f03900640b0 msgr2=0x7f0390066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:36:10.978 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.979+0000 7f03aa210700 1 --2- 172.21.15.27:0/874148649 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f03900640b0 0x7f0390066570 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7f03a40f6b50 tx=0x7f0394009040).stop 2021-11-03T18:36:10.978 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.979+0000 7f03aa210700 1 -- 172.21.15.27:0/874148649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03a40f9830 msgr2=0x7f03a40f5240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:36:10.978 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.979+0000 7f03aa210700 1 --2- 172.21.15.27:0/874148649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03a40f9830 0x7f03a40f5240 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 rx=0x7f038c013a70 tx=0x7f038c009b30).stop 2021-11-03T18:36:10.979 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.979+0000 7f03aa210700 1 -- 172.21.15.27:0/874148649 shutdown_connections 2021-11-03T18:36:10.979 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.979+0000 7f03aa210700 1 --2- 172.21.15.27:0/874148649 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f03900640b0 0x7f0390066570 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:10.979 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.979+0000 7f03aa210700 1 --2- 172.21.15.27:0/874148649 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f03a4060270 0x7f03a40f6dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:10.979 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.979+0000 7f03aa210700 1 --2- 172.21.15.27:0/874148649 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f03a40f9830 0x7f03a40f5240 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:10.980 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.979+0000 7f03aa210700 1 -- 172.21.15.27:0/874148649 >> 172.21.15.27:0/874148649 conn(0x7f03a40f0d40 msgr2=0x7f03a40f3130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:36:10.980 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.979+0000 7f03aa210700 1 -- 172.21.15.27:0/874148649 shutdown_connections 2021-11-03T18:36:10.980 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:10.979+0000 7f03aa210700 1 -- 172.21.15.27:0/874148649 wait complete. 2021-11-03T18:36:10.987 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:36:11.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.079+0000 7f081c117700 1 -- 172.21.15.27:0/3868840550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08140f66e0 msgr2=0x7f08140f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:36:11.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.079+0000 7f081c117700 1 --2- 172.21.15.27:0/3868840550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08140f66e0 0x7f08140f6b00 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 rx=0x7f0808004660 tx=0x7f0808009b30).stop 2021-11-03T18:36:11.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.079+0000 7f081c117700 1 -- 172.21.15.27:0/3868840550 shutdown_connections 2021-11-03T18:36:11.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.079+0000 7f081c117700 1 --2- 172.21.15.27:0/3868840550 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08140f7980 0x7f08140f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:11.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.079+0000 7f081c117700 1 --2- 172.21.15.27:0/3868840550 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08140f66e0 0x7f08140f6b00 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:11.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.079+0000 7f081c117700 1 -- 172.21.15.27:0/3868840550 >> 172.21.15.27:0/3868840550 conn(0x7f08140f1e40 msgr2=0x7f08140f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:36:11.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.079+0000 7f081c117700 1 -- 172.21.15.27:0/3868840550 shutdown_connections 2021-11-03T18:36:11.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.079+0000 7f081c117700 1 -- 172.21.15.27:0/3868840550 wait complete. 2021-11-03T18:36:11.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.080+0000 7f081c117700 1 Processor -- start 2021-11-03T18:36:11.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.080+0000 7f081c117700 1 -- start start 2021-11-03T18:36:11.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.080+0000 7f081c117700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08140f66e0 0x7f0814106fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:36:11.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.080+0000 7f081c117700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08140f7980 0x7f08141074a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:36:11.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.080+0000 7f081c117700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0814107a50 con 0x7f08140f66e0 2021-11-03T18:36:11.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.080+0000 7f081c117700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0814107b90 con 0x7f08140f7980 2021-11-03T18:36:11.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.080+0000 7f0819eb3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08140f66e0 0x7f0814106fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:36:11.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.080+0000 7f0819eb3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08140f66e0 0x7f0814106fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35790/0 (socket says 172.21.15.27:35790) 2021-11-03T18:36:11.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.080+0000 7f0819eb3700 1 -- 172.21.15.27:0/460150785 learned_addr learned my addr 172.21.15.27:0/460150785 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:36:11.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.080+0000 7f08196b2700 1 --2- 172.21.15.27:0/460150785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08140f7980 0x7f08141074a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:36:11.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.080+0000 7f0819eb3700 1 -- 172.21.15.27:0/460150785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08140f7980 msgr2=0x7f08141074a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:36:11.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.080+0000 7f0819eb3700 1 --2- 172.21.15.27:0/460150785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08140f7980 0x7f08141074a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:11.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.080+0000 7f0819eb3700 1 -- 172.21.15.27:0/460150785 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0808005040 con 0x7f08140f66e0 2021-11-03T18:36:11.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.080+0000 7f08196b2700 1 --2- 172.21.15.27:0/460150785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08140f7980 0x7f08141074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:36:11.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.081+0000 7f0819eb3700 1 --2- 172.21.15.27:0/460150785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08140f66e0 0x7f0814106fb0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 rx=0x7f0808013a70 tx=0x7f0808009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:36:11.086 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.081+0000 7f08067fc700 1 -- 172.21.15.27:0/460150785 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0808018070 con 0x7f08140f66e0 2021-11-03T18:36:11.086 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.081+0000 7f081c117700 1 -- 172.21.15.27:0/460150785 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08141084d0 con 0x7f08140f66e0 2021-11-03T18:36:11.086 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.081+0000 7f08067fc700 1 -- 172.21.15.27:0/460150785 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f08080097d0 con 0x7f08140f66e0 2021-11-03T18:36:11.086 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.081+0000 7f08067fc700 1 -- 172.21.15.27:0/460150785 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f080801c3c0 con 0x7f08140f66e0 2021-11-03T18:36:11.087 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.081+0000 7f081c117700 1 -- 172.21.15.27:0/460150785 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0814108a30 con 0x7f08140f66e0 2021-11-03T18:36:11.087 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.082+0000 7f081c117700 1 -- 172.21.15.27:0/460150785 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0814045ad0 con 0x7f08140f66e0 2021-11-03T18:36:11.087 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.082+0000 7f08067fc700 1 -- 172.21.15.27:0/460150785 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f0808007ba0 con 0x7f08140f66e0 2021-11-03T18:36:11.087 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.083+0000 7f08067fc700 1 --2- 172.21.15.27:0/460150785 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f08000640b0 0x7f0800066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:36:11.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.083+0000 7f08067fc700 1 -- 172.21.15.27:0/460150785 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f0808082470 con 0x7f08140f66e0 2021-11-03T18:36:11.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.083+0000 7f08196b2700 1 --2- 172.21.15.27:0/460150785 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f08000640b0 0x7f0800066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:36:11.088 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.083+0000 7f08196b2700 1 --2- 172.21.15.27:0/460150785 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f08000640b0 0x7f0800066570 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7f0810009740 tx=0x7f0810009be0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:36:11.089 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.087+0000 7f08067fc700 1 -- 172.21.15.27:0/460150785 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f08080547d0 con 0x7f08140f66e0 2021-11-03T18:36:11.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:10 smithi115 ceph-mon[65453]: pgmap v2447: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:11.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4812.475336 secs 2021-11-03T18:36:11.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.259+0000 7f081c117700 1 -- 172.21.15.27:0/460150785 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f08140fefb0 con 0x7f08000640b0 2021-11-03T18:36:11.276 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:10 smithi027 ceph-mon[97244]: pgmap v2447: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:11.276 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4812.475336 secs 2021-11-03T18:36:11.277 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.274+0000 7f08067fc700 1 -- 172.21.15.27:0/460150785 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f08140fefb0 con 0x7f08000640b0 2021-11-03T18:36:11.277 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:36:11.277 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (87m) 6m ago 88m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:36:11.278 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (79m) 6m ago 88m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:36:11.278 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (79m) 5m ago 88m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:36:11.278 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (87m) 6m ago 88m 42.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:36:11.278 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (81m) 6m ago 84m 26.7M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:36:11.279 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (84m) 6m ago 84m 3850M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:36:11.279 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (79m) 5m ago 84m 26.3M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:36:11.279 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (81m) 5m ago 84m 27.4M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:36:11.279 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (81m) 6m ago 90m 608M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:36:11.279 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (80m) 5m ago 88m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:36:11.280 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (80m) 6m ago 90m 350M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:36:11.280 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (80m) 5m ago 88m 160M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:36:11.280 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (88m) 6m ago 88m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:36:11.280 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (87m) 5m ago 87m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:36:11.280 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (79m) 6m ago 87m 413M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:36:11.281 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (78m) 6m ago 87m 450M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:36:11.281 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (78m) 6m ago 86m 338M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:36:11.281 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (77m) 5m ago 86m 479M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:36:11.281 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (77m) 5m ago 86m 378M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:36:11.282 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (76m) 5m ago 85m 388M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:36:11.282 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (87m) 6m ago 88m 64.4M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:36:11.282 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.276+0000 7f081c117700 1 -- 172.21.15.27:0/460150785 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f08000640b0 msgr2=0x7f0800066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:36:11.283 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.276+0000 7f081c117700 1 --2- 172.21.15.27:0/460150785 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f08000640b0 0x7f0800066570 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7f0810009740 tx=0x7f0810009be0).stop 2021-11-03T18:36:11.283 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.276+0000 7f081c117700 1 -- 172.21.15.27:0/460150785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08140f66e0 msgr2=0x7f0814106fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:36:11.283 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.276+0000 7f081c117700 1 --2- 172.21.15.27:0/460150785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08140f66e0 0x7f0814106fb0 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 rx=0x7f0808013a70 tx=0x7f0808009b30).stop 2021-11-03T18:36:11.283 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.276+0000 7f081c117700 1 -- 172.21.15.27:0/460150785 shutdown_connections 2021-11-03T18:36:11.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.276+0000 7f081c117700 1 --2- 172.21.15.27:0/460150785 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f08000640b0 0x7f0800066570 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:11.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.276+0000 7f081c117700 1 --2- 172.21.15.27:0/460150785 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f08140f66e0 0x7f0814106fb0 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:11.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.276+0000 7f081c117700 1 --2- 172.21.15.27:0/460150785 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f08140f7980 0x7f08141074a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:11.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.276+0000 7f081c117700 1 -- 172.21.15.27:0/460150785 >> 172.21.15.27:0/460150785 conn(0x7f08140f1e40 msgr2=0x7f08140fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:36:11.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.276+0000 7f081c117700 1 -- 172.21.15.27:0/460150785 shutdown_connections 2021-11-03T18:36:11.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.276+0000 7f081c117700 1 -- 172.21.15.27:0/460150785 wait complete. 2021-11-03T18:36:11.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.375+0000 7f3731289700 1 -- 172.21.15.27:0/1709904397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f372c0f55e0 msgr2=0x7f372c0f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:36:11.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.375+0000 7f3731289700 1 --2- 172.21.15.27:0/1709904397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f372c0f55e0 0x7f372c0f5a00 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 rx=0x7f37240099e0 tx=0x7f3724004d10).stop 2021-11-03T18:36:11.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.375+0000 7f3731289700 1 -- 172.21.15.27:0/1709904397 shutdown_connections 2021-11-03T18:36:11.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.375+0000 7f3731289700 1 --2- 172.21.15.27:0/1709904397 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f372c0f5ef0 0x7f372c0f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:11.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.375+0000 7f3731289700 1 --2- 172.21.15.27:0/1709904397 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f372c0f55e0 0x7f372c0f5a00 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:11.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.375+0000 7f3731289700 1 -- 172.21.15.27:0/1709904397 >> 172.21.15.27:0/1709904397 conn(0x7f372c0f0de0 msgr2=0x7f372c0f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:36:11.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.375+0000 7f3731289700 1 -- 172.21.15.27:0/1709904397 shutdown_connections 2021-11-03T18:36:11.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.375+0000 7f3731289700 1 -- 172.21.15.27:0/1709904397 wait complete. 2021-11-03T18:36:11.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.376+0000 7f3731289700 1 Processor -- start 2021-11-03T18:36:11.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.376+0000 7f3731289700 1 -- start start 2021-11-03T18:36:11.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.376+0000 7f3731289700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f372c0f55e0 0x7f372c107060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:36:11.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.376+0000 7f3731289700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f372c0f5ef0 0x7f372c107550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:36:11.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.376+0000 7f3731289700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372c107b00 con 0x7f372c0f5ef0 2021-11-03T18:36:11.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.376+0000 7f3731289700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372c107c40 con 0x7f372c0f55e0 2021-11-03T18:36:11.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.376+0000 7f372a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f372c0f5ef0 0x7f372c107550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:36:11.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.376+0000 7f372a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f372c0f5ef0 0x7f372c107550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35802/0 (socket says 172.21.15.27:35802) 2021-11-03T18:36:11.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.376+0000 7f372a7fc700 1 -- 172.21.15.27:0/1075757623 learned_addr learned my addr 172.21.15.27:0/1075757623 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:36:11.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.376+0000 7f372affd700 1 --2- 172.21.15.27:0/1075757623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f372c0f55e0 0x7f372c107060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:36:11.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.376+0000 7f372a7fc700 1 -- 172.21.15.27:0/1075757623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f372c0f55e0 msgr2=0x7f372c107060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:36:11.382 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.376+0000 7f372a7fc700 1 --2- 172.21.15.27:0/1075757623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f372c0f55e0 0x7f372c107060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:11.382 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.376+0000 7f372a7fc700 1 -- 172.21.15.27:0/1075757623 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3724009640 con 0x7f372c0f5ef0 2021-11-03T18:36:11.383 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.377+0000 7f372a7fc700 1 --2- 172.21.15.27:0/1075757623 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f372c0f5ef0 0x7f372c107550 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 rx=0x7f371400c790 tx=0x7f3714007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:36:11.383 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.377+0000 7f37237fe700 1 -- 172.21.15.27:0/1075757623 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f371400e680 con 0x7f372c0f5ef0 2021-11-03T18:36:11.383 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.377+0000 7f37237fe700 1 -- 172.21.15.27:0/1075757623 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f371400ec90 con 0x7f372c0f5ef0 2021-11-03T18:36:11.383 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.377+0000 7f3731289700 1 -- 172.21.15.27:0/1075757623 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f372c1085e0 con 0x7f372c0f5ef0 2021-11-03T18:36:11.384 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.377+0000 7f37237fe700 1 -- 172.21.15.27:0/1075757623 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3714013d60 con 0x7f372c0f5ef0 2021-11-03T18:36:11.384 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.377+0000 7f3731289700 1 -- 172.21.15.27:0/1075757623 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f372c108b20 con 0x7f372c0f5ef0 2021-11-03T18:36:11.384 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.378+0000 7f3731289700 1 -- 172.21.15.27:0/1075757623 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f372c045ad0 con 0x7f372c0f5ef0 2021-11-03T18:36:11.384 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.378+0000 7f37237fe700 1 -- 172.21.15.27:0/1075757623 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f371401a060 con 0x7f372c0f5ef0 2021-11-03T18:36:11.384 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.379+0000 7f37237fe700 1 --2- 172.21.15.27:0/1075757623 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f37180640b0 0x7f3718066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:36:11.385 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.379+0000 7f37237fe700 1 -- 172.21.15.27:0/1075757623 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f3714080a20 con 0x7f372c0f5ef0 2021-11-03T18:36:11.385 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.379+0000 7f372affd700 1 --2- 172.21.15.27:0/1075757623 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f37180640b0 0x7f3718066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:36:11.385 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.379+0000 7f372affd700 1 --2- 172.21.15.27:0/1075757623 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f37180640b0 0x7f3718066570 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7f372400b3b0 tx=0x7f3724004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:36:11.386 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.383+0000 7f37237fe700 1 -- 172.21.15.27:0/1075757623 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f3714052c30 con 0x7f372c0f5ef0 2021-11-03T18:36:11.613 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.614+0000 7f3731289700 1 -- 172.21.15.27:0/1075757623 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f372c0fdf60 con 0x7f372c0f5ef0 2021-11-03T18:36:11.613 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.615+0000 7f37237fe700 1 -- 172.21.15.27:0/1075757623 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f372c0fdf60 con 0x7f372c0f5ef0 2021-11-03T18:36:11.614 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:36:11.614 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:36:11.614 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:36:11.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:36:11.615 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:36:11.615 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:36:11.615 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:36:11.615 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:36:11.616 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:36:11.616 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:36:11.616 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:36:11.616 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:36:11.617 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:36:11.617 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:36:11.617 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:36:11.617 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:36:11.617 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:36:11.618 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:36:11.619 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.616+0000 7f3731289700 1 -- 172.21.15.27:0/1075757623 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f37180640b0 msgr2=0x7f3718066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:36:11.619 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.616+0000 7f3731289700 1 --2- 172.21.15.27:0/1075757623 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f37180640b0 0x7f3718066570 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 rx=0x7f372400b3b0 tx=0x7f3724004d10).stop 2021-11-03T18:36:11.619 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.616+0000 7f3731289700 1 -- 172.21.15.27:0/1075757623 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f372c0f5ef0 msgr2=0x7f372c107550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:36:11.619 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.616+0000 7f3731289700 1 --2- 172.21.15.27:0/1075757623 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f372c0f5ef0 0x7f372c107550 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 rx=0x7f371400c790 tx=0x7f3714007d70).stop 2021-11-03T18:36:11.620 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.617+0000 7f3731289700 1 -- 172.21.15.27:0/1075757623 shutdown_connections 2021-11-03T18:36:11.620 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.617+0000 7f3731289700 1 --2- 172.21.15.27:0/1075757623 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f37180640b0 0x7f3718066570 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:11.620 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.617+0000 7f3731289700 1 --2- 172.21.15.27:0/1075757623 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f372c0f55e0 0x7f372c107060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:11.620 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.617+0000 7f3731289700 1 --2- 172.21.15.27:0/1075757623 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f372c0f5ef0 0x7f372c107550 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:11.621 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.617+0000 7f3731289700 1 -- 172.21.15.27:0/1075757623 >> 172.21.15.27:0/1075757623 conn(0x7f372c0f0de0 msgr2=0x7f372c05ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:36:11.621 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.617+0000 7f3731289700 1 -- 172.21.15.27:0/1075757623 shutdown_connections 2021-11-03T18:36:11.621 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.617+0000 7f3731289700 1 -- 172.21.15.27:0/1075757623 wait complete. 2021-11-03T18:36:11.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.717+0000 7faff18ed700 1 -- 172.21.15.27:0/407556368 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafec0f5ef0 msgr2=0x7fafec0f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:36:11.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.717+0000 7faff18ed700 1 --2- 172.21.15.27:0/407556368 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafec0f5ef0 0x7fafec0f6920 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 rx=0x7fafd4004660 tx=0x7fafd4009b30).stop 2021-11-03T18:36:11.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.717+0000 7faff18ed700 1 -- 172.21.15.27:0/407556368 shutdown_connections 2021-11-03T18:36:11.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.717+0000 7faff18ed700 1 --2- 172.21.15.27:0/407556368 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafec0f5ef0 0x7fafec0f6920 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:11.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.717+0000 7faff18ed700 1 --2- 172.21.15.27:0/407556368 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafec0f55e0 0x7fafec0f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:11.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.717+0000 7faff18ed700 1 -- 172.21.15.27:0/407556368 >> 172.21.15.27:0/407556368 conn(0x7fafec0f0de0 msgr2=0x7fafec0f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:36:11.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.717+0000 7faff18ed700 1 -- 172.21.15.27:0/407556368 shutdown_connections 2021-11-03T18:36:11.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.717+0000 7faff18ed700 1 -- 172.21.15.27:0/407556368 wait complete. 2021-11-03T18:36:11.721 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.718+0000 7faff18ed700 1 Processor -- start 2021-11-03T18:36:11.721 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.718+0000 7faff18ed700 1 -- start start 2021-11-03T18:36:11.721 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.718+0000 7faff18ed700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafec0f55e0 0x7fafec106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:36:11.721 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.718+0000 7faff18ed700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafec0f5ef0 0x7fafec107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:36:11.722 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.718+0000 7faff18ed700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafec107a10 con 0x7fafec0f5ef0 2021-11-03T18:36:11.722 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.718+0000 7faff18ed700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafec107b50 con 0x7fafec0f55e0 2021-11-03T18:36:11.722 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.718+0000 7fafea7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafec0f5ef0 0x7fafec107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:36:11.722 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.718+0000 7fafea7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafec0f5ef0 0x7fafec107460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35812/0 (socket says 172.21.15.27:35812) 2021-11-03T18:36:11.722 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.718+0000 7fafea7fc700 1 -- 172.21.15.27:0/475980299 learned_addr learned my addr 172.21.15.27:0/475980299 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:36:11.723 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.718+0000 7fafeaffd700 1 --2- 172.21.15.27:0/475980299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafec0f55e0 0x7fafec106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:36:11.723 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.718+0000 7fafea7fc700 1 -- 172.21.15.27:0/475980299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafec0f55e0 msgr2=0x7fafec106f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:36:11.723 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.718+0000 7fafea7fc700 1 --2- 172.21.15.27:0/475980299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafec0f55e0 0x7fafec106f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:11.723 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.718+0000 7fafea7fc700 1 -- 172.21.15.27:0/475980299 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd4005040 con 0x7fafec0f5ef0 2021-11-03T18:36:11.724 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.718+0000 7fafeaffd700 1 --2- 172.21.15.27:0/475980299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafec0f55e0 0x7fafec106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T18:36:11.724 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.718+0000 7fafea7fc700 1 --2- 172.21.15.27:0/475980299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafec0f5ef0 0x7fafec107460 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 rx=0x7fafd4009990 tx=0x7fafd4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:36:11.724 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.719+0000 7fafe37fe700 1 -- 172.21.15.27:0/475980299 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafd4018070 con 0x7fafec0f5ef0 2021-11-03T18:36:11.724 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.719+0000 7fafe37fe700 1 -- 172.21.15.27:0/475980299 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fafd4007ba0 con 0x7fafec0f5ef0 2021-11-03T18:36:11.725 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.719+0000 7faff18ed700 1 -- 172.21.15.27:0/475980299 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafec108490 con 0x7fafec0f5ef0 2021-11-03T18:36:11.725 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.719+0000 7fafe37fe700 1 -- 172.21.15.27:0/475980299 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fafd401c3c0 con 0x7fafec0f5ef0 2021-11-03T18:36:11.725 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.719+0000 7faff18ed700 1 -- 172.21.15.27:0/475980299 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafec1089f0 con 0x7fafec0f5ef0 2021-11-03T18:36:11.725 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.720+0000 7faff18ed700 1 -- 172.21.15.27:0/475980299 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafec045ad0 con 0x7fafec0f5ef0 2021-11-03T18:36:11.725 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.720+0000 7fafe37fe700 1 -- 172.21.15.27:0/475980299 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fafd401d420 con 0x7fafec0f5ef0 2021-11-03T18:36:11.726 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.720+0000 7fafe37fe700 1 --2- 172.21.15.27:0/475980299 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fafd80640b0 0x7fafd8066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:36:11.726 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.720+0000 7fafe37fe700 1 -- 172.21.15.27:0/475980299 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fafd40825d0 con 0x7fafec0f5ef0 2021-11-03T18:36:11.726 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.721+0000 7fafeaffd700 1 --2- 172.21.15.27:0/475980299 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fafd80640b0 0x7fafd8066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:36:11.726 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.721+0000 7fafeaffd700 1 --2- 172.21.15.27:0/475980299 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fafd80640b0 0x7fafd8066570 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7fafdc009fd0 tx=0x7fafdc00b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:36:11.727 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.724+0000 7fafe37fe700 1 -- 172.21.15.27:0/475980299 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fafd4054990 con 0x7fafec0f5ef0 2021-11-03T18:36:11.929 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.931+0000 7faff18ed700 1 -- 172.21.15.27:0/475980299 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fafec0fddd0 con 0x7fafec0f5ef0 2021-11-03T18:36:11.929 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.931+0000 7fafe37fe700 1 -- 172.21.15.27:0/475980299 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7fafd4054580 con 0x7fafec0f5ef0 2021-11-03T18:36:11.930 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:36:11.930 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:36:11.930 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:36:11.931 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:36:11.931 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:36:11.931 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:36:11.931 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:36:11.931 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:36:11.932 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:36:11.933 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:36:11.933 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:36:11.933 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:36:11.933 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:36:11.934 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:36:11.934 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:36:11.934 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:36:11.934 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:36:11.934 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:36:11.935 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:36:11.935 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:36:11.935 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:36:11.935 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:36:11.936 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:36:11.936 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:36:11.936 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:36:11.936 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:36:11.937 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:36:11.937 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:36:11.937 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:36:11.937 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:36:11.937 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:36:11.938 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:36:11.938 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:36:11.938 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:36:11.938 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:36:11.939 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:36:11.939 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:36:11.939 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:36:11.939 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:36:11.940 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.933+0000 7faff18ed700 1 -- 172.21.15.27:0/475980299 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fafd80640b0 msgr2=0x7fafd8066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:36:11.941 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.933+0000 7faff18ed700 1 --2- 172.21.15.27:0/475980299 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fafd80640b0 0x7fafd8066570 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7fafdc009fd0 tx=0x7fafdc00b040).stop 2021-11-03T18:36:11.941 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.933+0000 7faff18ed700 1 -- 172.21.15.27:0/475980299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafec0f5ef0 msgr2=0x7fafec107460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:36:11.941 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.933+0000 7faff18ed700 1 --2- 172.21.15.27:0/475980299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafec0f5ef0 0x7fafec107460 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 rx=0x7fafd4009990 tx=0x7fafd4009b30).stop 2021-11-03T18:36:11.941 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.933+0000 7faff18ed700 1 -- 172.21.15.27:0/475980299 shutdown_connections 2021-11-03T18:36:11.942 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.933+0000 7faff18ed700 1 --2- 172.21.15.27:0/475980299 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fafd80640b0 0x7fafd8066570 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:11.942 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.933+0000 7faff18ed700 1 --2- 172.21.15.27:0/475980299 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafec0f55e0 0x7fafec106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:11.942 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.933+0000 7faff18ed700 1 --2- 172.21.15.27:0/475980299 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafec0f5ef0 0x7fafec107460 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:11.942 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.933+0000 7faff18ed700 1 -- 172.21.15.27:0/475980299 >> 172.21.15.27:0/475980299 conn(0x7fafec0f0de0 msgr2=0x7fafec05f740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:36:11.943 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.933+0000 7faff18ed700 1 -- 172.21.15.27:0/475980299 shutdown_connections 2021-11-03T18:36:11.943 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:11.933+0000 7faff18ed700 1 -- 172.21.15.27:0/475980299 wait complete. 2021-11-03T18:36:11.944 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:36:12.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:11 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1075757623' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:12.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:11 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1075757623' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:13.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:12 smithi115 ceph-mon[65453]: from='client.36616 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:36:13.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:12 smithi115 ceph-mon[65453]: pgmap v2448: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:13.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:12 smithi115 ceph-mon[65453]: from='client.36620 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:36:13.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:12 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/475980299' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:36:13.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:12 smithi027 ceph-mon[97244]: from='client.36616 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:36:13.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:12 smithi027 ceph-mon[97244]: pgmap v2448: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:13.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:12 smithi027 ceph-mon[97244]: from='client.36620 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:36:13.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:12 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/475980299' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:36:15.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:14 smithi115 ceph-mon[65453]: pgmap v2449: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:15.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:14 smithi027 ceph-mon[97244]: pgmap v2449: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:16.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4817.475477 secs 2021-11-03T18:36:16.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4817.475477 secs 2021-11-03T18:36:17.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:16 smithi115 ceph-mon[65453]: pgmap v2450: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:17.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:16 smithi027 ceph-mon[97244]: pgmap v2450: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:18.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:17 smithi115 ceph-mon[65453]: pgmap v2451: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:18.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:36:18.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:36:18.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:36:18.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:18.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:18.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:18.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:18.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:17 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:36:18.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:17 smithi027 ceph-mon[97244]: pgmap v2451: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:18.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:36:18.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:36:18.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:36:18.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:18.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:18.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:18.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:18.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:17 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:36:20.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:20 smithi115 ceph-mon[65453]: pgmap v2452: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:20.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4822.475546 secs 2021-11-03T18:36:20.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:20 smithi027 ceph-mon[97244]: pgmap v2452: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:20.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4822.475546 secs 2021-11-03T18:36:22.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:22 smithi027 ceph-mon[97244]: pgmap v2453: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:22.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:22 smithi115 ceph-mon[65453]: pgmap v2453: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:24.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:24 smithi027 ceph-mon[97244]: pgmap v2454: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:24.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:24 smithi115 ceph-mon[65453]: pgmap v2454: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:25.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4827.475685 secs 2021-11-03T18:36:25.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4827.475685 secs 2021-11-03T18:36:26.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:26 smithi027 ceph-mon[97244]: pgmap v2455: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:26.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:26 smithi115 ceph-mon[65453]: pgmap v2455: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:28.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:28 smithi027 ceph-mon[97244]: pgmap v2456: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:28.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:36:28.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:36:28.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:36:28.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:28.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:28.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:28.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:28.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:28 smithi115 ceph-mon[65453]: pgmap v2456: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:28.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:36:28.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:36:28.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:36:28.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:28.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:28.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:28.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:29.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:29 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:36:29.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:29 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:36:30.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:30 smithi027 ceph-mon[97244]: pgmap v2457: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:30.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4832.475823 secs 2021-11-03T18:36:30.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:30 smithi115 ceph-mon[65453]: pgmap v2457: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:30.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4832.475823 secs 2021-11-03T18:36:32.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:32 smithi027 ceph-mon[97244]: pgmap v2458: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:32.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:32 smithi115 ceph-mon[65453]: pgmap v2458: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:34.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:34 smithi027 ceph-mon[97244]: pgmap v2459: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:34.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:34 smithi115 ceph-mon[65453]: pgmap v2459: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:35.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4837.475950 secs 2021-11-03T18:36:36.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4837.475950 secs 2021-11-03T18:36:36.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:36 smithi115 ceph-mon[65453]: pgmap v2460: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:37.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:36 smithi027 ceph-mon[97244]: pgmap v2460: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:38.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:38 smithi115 ceph-mon[65453]: pgmap v2461: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:38.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:36:38.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2021-11-03T18:36:38.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"}]: dispatch 2021-11-03T18:36:38.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:36:38.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:36:38.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:38.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:38.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:38.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:39.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:38 smithi027 ceph-mon[97244]: pgmap v2461: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:39.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:36:39.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2021-11-03T18:36:39.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"}]: dispatch 2021-11-03T18:36:39.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:36:39.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:36:39.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:39.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:39.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:39.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:39.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:39 smithi115 ceph-mon[65453]: poll took 60183.566 ms: 1 events 2021-11-03T18:36:39.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:39 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:36:40.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:39 smithi027 ceph-mon[97244]: poll took 60183.566 ms: 1 events 2021-11-03T18:36:40.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:39 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:36:40.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:40 smithi115 ceph-mon[65453]: pgmap v2462: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:40.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4842.476091 secs 2021-11-03T18:36:41.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:40 smithi027 ceph-mon[97244]: pgmap v2462: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:41.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4842.476091 secs 2021-11-03T18:36:42.048 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.047+0000 7fba3c4da700 1 -- 172.21.15.27:0/90661416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba340f7980 msgr2=0x7fba340f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:36:42.048 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.047+0000 7fba3c4da700 1 --2- 172.21.15.27:0/90661416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba340f7980 0x7fba340f7de0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 rx=0x7fba24004660 tx=0x7fba24009b30).stop 2021-11-03T18:36:42.049 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.047+0000 7fba3c4da700 1 -- 172.21.15.27:0/90661416 shutdown_connections 2021-11-03T18:36:42.049 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.047+0000 7fba3c4da700 1 --2- 172.21.15.27:0/90661416 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba340f7980 0x7fba340f7de0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:42.049 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.047+0000 7fba3c4da700 1 --2- 172.21.15.27:0/90661416 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba340f66e0 0x7fba340f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:42.049 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.047+0000 7fba3c4da700 1 -- 172.21.15.27:0/90661416 >> 172.21.15.27:0/90661416 conn(0x7fba340f1e40 msgr2=0x7fba340f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:36:42.049 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.047+0000 7fba3c4da700 1 -- 172.21.15.27:0/90661416 shutdown_connections 2021-11-03T18:36:42.050 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.047+0000 7fba3c4da700 1 -- 172.21.15.27:0/90661416 wait complete. 2021-11-03T18:36:42.050 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.048+0000 7fba3c4da700 1 Processor -- start 2021-11-03T18:36:42.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.048+0000 7fba3c4da700 1 -- start start 2021-11-03T18:36:42.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.048+0000 7fba3c4da700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba340f66e0 0x7fba3406b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:36:42.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.048+0000 7fba3c4da700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba340f7980 0x7fba34069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:36:42.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.048+0000 7fba3c4da700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3406a380 con 0x7fba340f7980 2021-11-03T18:36:42.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.048+0000 7fba3c4da700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3406a4c0 con 0x7fba340f66e0 2021-11-03T18:36:42.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.048+0000 7fba39a75700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba340f7980 0x7fba34069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:36:42.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.048+0000 7fba39a75700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba340f7980 0x7fba34069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35830/0 (socket says 172.21.15.27:35830) 2021-11-03T18:36:42.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.048+0000 7fba39a75700 1 -- 172.21.15.27:0/4061503009 learned_addr learned my addr 172.21.15.27:0/4061503009 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:36:42.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.048+0000 7fba3a276700 1 --2- 172.21.15.27:0/4061503009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba340f66e0 0x7fba3406b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:36:42.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.048+0000 7fba39a75700 1 -- 172.21.15.27:0/4061503009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba340f66e0 msgr2=0x7fba3406b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:36:42.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.048+0000 7fba39a75700 1 --2- 172.21.15.27:0/4061503009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba340f66e0 0x7fba3406b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:42.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.048+0000 7fba39a75700 1 -- 172.21.15.27:0/4061503009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba24005040 con 0x7fba340f7980 2021-11-03T18:36:42.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.049+0000 7fba39a75700 1 --2- 172.21.15.27:0/4061503009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba340f7980 0x7fba34069e20 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 rx=0x7fba24009990 tx=0x7fba24009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:36:42.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.049+0000 7fba2affd700 1 -- 172.21.15.27:0/4061503009 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba24018070 con 0x7fba340f7980 2021-11-03T18:36:42.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.049+0000 7fba2affd700 1 -- 172.21.15.27:0/4061503009 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fba24007ca0 con 0x7fba340f7980 2021-11-03T18:36:42.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.049+0000 7fba3c4da700 1 -- 172.21.15.27:0/4061503009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba3406a740 con 0x7fba340f7980 2021-11-03T18:36:42.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.049+0000 7fba2affd700 1 -- 172.21.15.27:0/4061503009 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fba2401c620 con 0x7fba340f7980 2021-11-03T18:36:42.055 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.049+0000 7fba3c4da700 1 -- 172.21.15.27:0/4061503009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba3406aca0 con 0x7fba340f7980 2021-11-03T18:36:42.056 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.050+0000 7fba3c4da700 1 -- 172.21.15.27:0/4061503009 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba34045ad0 con 0x7fba340f7980 2021-11-03T18:36:42.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.050+0000 7fba2affd700 1 -- 172.21.15.27:0/4061503009 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fba2401d960 con 0x7fba340f7980 2021-11-03T18:36:42.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.051+0000 7fba2affd700 1 --2- 172.21.15.27:0/4061503009 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fba200640b0 0x7fba20066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:36:42.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.051+0000 7fba2affd700 1 -- 172.21.15.27:0/4061503009 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fba24082490 con 0x7fba340f7980 2021-11-03T18:36:42.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.051+0000 7fba3a276700 1 --2- 172.21.15.27:0/4061503009 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fba200640b0 0x7fba20066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:36:42.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.051+0000 7fba3a276700 1 --2- 172.21.15.27:0/4061503009 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fba200640b0 0x7fba20066570 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7fba3406b730 tx=0x7fba30009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:36:42.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.055+0000 7fba2affd700 1 -- 172.21.15.27:0/4061503009 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fba240548b0 con 0x7fba340f7980 2021-11-03T18:36:42.234 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.235+0000 7fba3c4da700 1 -- 172.21.15.27:0/4061503009 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fba340008d0 con 0x7fba200640b0 2021-11-03T18:36:42.235 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.237+0000 7fba2affd700 1 -- 172.21.15.27:0/4061503009 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fba340008d0 con 0x7fba200640b0 2021-11-03T18:36:42.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.238+0000 7fba3c4da700 1 -- 172.21.15.27:0/4061503009 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fba200640b0 msgr2=0x7fba20066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:36:42.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.238+0000 7fba3c4da700 1 --2- 172.21.15.27:0/4061503009 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fba200640b0 0x7fba20066570 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7fba3406b730 tx=0x7fba30009040).stop 2021-11-03T18:36:42.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.238+0000 7fba3c4da700 1 -- 172.21.15.27:0/4061503009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba340f7980 msgr2=0x7fba34069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:36:42.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.238+0000 7fba3c4da700 1 --2- 172.21.15.27:0/4061503009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba340f7980 0x7fba34069e20 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 rx=0x7fba24009990 tx=0x7fba24009b30).stop 2021-11-03T18:36:42.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.239+0000 7fba3c4da700 1 -- 172.21.15.27:0/4061503009 shutdown_connections 2021-11-03T18:36:42.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.239+0000 7fba3c4da700 1 --2- 172.21.15.27:0/4061503009 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fba200640b0 0x7fba20066570 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:42.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.239+0000 7fba3c4da700 1 --2- 172.21.15.27:0/4061503009 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fba340f66e0 0x7fba3406b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:42.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.239+0000 7fba3c4da700 1 --2- 172.21.15.27:0/4061503009 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fba340f7980 0x7fba34069e20 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:42.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.239+0000 7fba3c4da700 1 -- 172.21.15.27:0/4061503009 >> 172.21.15.27:0/4061503009 conn(0x7fba340f1e40 msgr2=0x7fba340fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:36:42.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.239+0000 7fba3c4da700 1 -- 172.21.15.27:0/4061503009 shutdown_connections 2021-11-03T18:36:42.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.239+0000 7fba3c4da700 1 -- 172.21.15.27:0/4061503009 wait complete. 2021-11-03T18:36:42.249 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:36:42.343 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.339+0000 7f51e5add700 1 -- 172.21.15.27:0/2052723502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51e00f6860 msgr2=0x7f51e00f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:36:42.343 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.339+0000 7f51e5add700 1 --2- 172.21.15.27:0/2052723502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51e00f6860 0x7f51e00f6cc0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 rx=0x7f51d4004660 tx=0x7f51d4009b30).stop 2021-11-03T18:36:42.343 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.339+0000 7f51e5add700 1 -- 172.21.15.27:0/2052723502 shutdown_connections 2021-11-03T18:36:42.344 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.339+0000 7f51e5add700 1 --2- 172.21.15.27:0/2052723502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51e00f6860 0x7f51e00f6cc0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:42.344 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.339+0000 7f51e5add700 1 --2- 172.21.15.27:0/2052723502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51e00f55c0 0x7f51e00f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:42.344 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.339+0000 7f51e5add700 1 -- 172.21.15.27:0/2052723502 >> 172.21.15.27:0/2052723502 conn(0x7f51e00f0d40 msgr2=0x7f51e00f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:36:42.345 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.339+0000 7f51e5add700 1 -- 172.21.15.27:0/2052723502 shutdown_connections 2021-11-03T18:36:42.345 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.339+0000 7f51e5add700 1 -- 172.21.15.27:0/2052723502 wait complete. 2021-11-03T18:36:42.345 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.339+0000 7f51e5add700 1 Processor -- start 2021-11-03T18:36:42.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.340+0000 7f51e5add700 1 -- start start 2021-11-03T18:36:42.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.340+0000 7f51e5add700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51e00f55c0 0x7f51e0109c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:36:42.347 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.340+0000 7f51e5add700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51e00f6860 0x7f51e010c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:36:42.347 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.340+0000 7f51e5add700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e00603c0 con 0x7f51e00f6860 2021-11-03T18:36:42.347 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.340+0000 7f51e5add700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e0060500 con 0x7f51e00f55c0 2021-11-03T18:36:42.347 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.340+0000 7f51deffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51e00f6860 0x7f51e010c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:36:42.348 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.340+0000 7f51deffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51e00f6860 0x7f51e010c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35840/0 (socket says 172.21.15.27:35840) 2021-11-03T18:36:42.348 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.340+0000 7f51deffd700 1 -- 172.21.15.27:0/3853376776 learned_addr learned my addr 172.21.15.27:0/3853376776 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:36:42.348 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.340+0000 7f51df7fe700 1 --2- 172.21.15.27:0/3853376776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51e00f55c0 0x7f51e0109c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:36:42.349 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.340+0000 7f51deffd700 1 -- 172.21.15.27:0/3853376776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51e00f55c0 msgr2=0x7f51e0109c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:36:42.349 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.340+0000 7f51deffd700 1 --2- 172.21.15.27:0/3853376776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51e00f55c0 0x7f51e0109c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:42.350 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.340+0000 7f51deffd700 1 -- 172.21.15.27:0/3853376776 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d4005040 con 0x7f51e00f6860 2021-11-03T18:36:42.350 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.340+0000 7f51deffd700 1 --2- 172.21.15.27:0/3853376776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51e00f6860 0x7f51e010c180 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 rx=0x7f51d4009990 tx=0x7f51d4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:36:42.350 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.341+0000 7f51c7fff700 1 -- 172.21.15.27:0/3853376776 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51d4018070 con 0x7f51e00f6860 2021-11-03T18:36:42.351 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.341+0000 7f51c7fff700 1 -- 172.21.15.27:0/3853376776 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f51d4007ca0 con 0x7f51e00f6860 2021-11-03T18:36:42.351 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.341+0000 7f51e5add700 1 -- 172.21.15.27:0/3853376776 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51e010c6d0 con 0x7f51e00f6860 2021-11-03T18:36:42.351 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.341+0000 7f51c7fff700 1 -- 172.21.15.27:0/3853376776 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51d401c620 con 0x7f51e00f6860 2021-11-03T18:36:42.351 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.341+0000 7f51e5add700 1 -- 172.21.15.27:0/3853376776 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e010cc00 con 0x7f51e00f6860 2021-11-03T18:36:42.352 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.342+0000 7f51e5add700 1 -- 172.21.15.27:0/3853376776 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51e00f49b0 con 0x7f51e00f6860 2021-11-03T18:36:42.352 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.342+0000 7f51c7fff700 1 -- 172.21.15.27:0/3853376776 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f51d401d960 con 0x7f51e00f6860 2021-11-03T18:36:42.353 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.343+0000 7f51c7fff700 1 --2- 172.21.15.27:0/3853376776 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f51c80640b0 0x7f51c8066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:36:42.353 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.343+0000 7f51df7fe700 1 --2- 172.21.15.27:0/3853376776 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f51c80640b0 0x7f51c8066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:36:42.353 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.343+0000 7f51c7fff700 1 -- 172.21.15.27:0/3853376776 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f51d4082520 con 0x7f51e00f6860 2021-11-03T18:36:42.353 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.343+0000 7f51df7fe700 1 --2- 172.21.15.27:0/3853376776 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f51c80640b0 0x7f51c8066570 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7f51d00041d0 tx=0x7f51d0009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:36:42.354 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.347+0000 7f51c7fff700 1 -- 172.21.15.27:0/3853376776 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f51d4054940 con 0x7f51e00f6860 2021-11-03T18:36:42.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.523+0000 7f51e5add700 1 -- 172.21.15.27:0/3853376776 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f51e00fdf10 con 0x7f51c80640b0 2021-11-03T18:36:42.537 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.537+0000 7f51c7fff700 1 -- 172.21.15.27:0/3853376776 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f51e00fdf10 con 0x7f51c80640b0 2021-11-03T18:36:42.537 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:36:42.538 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (88m) 7m ago 89m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:36:42.538 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (79m) 7m ago 89m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:36:42.538 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (79m) 6m ago 88m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:36:42.538 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (88m) 7m ago 89m 42.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:36:42.539 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (81m) 7m ago 84m 26.7M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:36:42.539 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (84m) 7m ago 84m 3850M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:36:42.539 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (80m) 6m ago 84m 26.3M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:36:42.540 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (82m) 6m ago 84m 27.4M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:36:42.540 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (81m) 7m ago 90m 608M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:36:42.540 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (81m) 6m ago 88m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:36:42.540 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (80m) 7m ago 90m 350M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:36:42.541 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (80m) 6m ago 88m 160M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:36:42.541 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (89m) 7m ago 89m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:36:42.541 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (88m) 6m ago 88m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:36:42.541 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (79m) 7m ago 87m 413M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:36:42.542 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (79m) 7m ago 87m 450M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:36:42.542 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (78m) 7m ago 87m 338M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:36:42.542 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (78m) 6m ago 86m 479M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:36:42.542 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (77m) 6m ago 86m 378M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:36:42.542 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (77m) 6m ago 86m 388M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:36:42.543 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (88m) 7m ago 88m 64.4M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:36:42.544 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.539+0000 7f51e5add700 1 -- 172.21.15.27:0/3853376776 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f51c80640b0 msgr2=0x7f51c8066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:36:42.544 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.539+0000 7f51e5add700 1 --2- 172.21.15.27:0/3853376776 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f51c80640b0 0x7f51c8066570 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7f51d00041d0 tx=0x7f51d0009040).stop 2021-11-03T18:36:42.544 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.539+0000 7f51e5add700 1 -- 172.21.15.27:0/3853376776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51e00f6860 msgr2=0x7f51e010c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:36:42.544 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.539+0000 7f51e5add700 1 --2- 172.21.15.27:0/3853376776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51e00f6860 0x7f51e010c180 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 rx=0x7f51d4009990 tx=0x7f51d4009b30).stop 2021-11-03T18:36:42.545 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.539+0000 7f51e5add700 1 -- 172.21.15.27:0/3853376776 shutdown_connections 2021-11-03T18:36:42.545 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.540+0000 7f51e5add700 1 --2- 172.21.15.27:0/3853376776 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f51c80640b0 0x7f51c8066570 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:42.545 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.540+0000 7f51e5add700 1 --2- 172.21.15.27:0/3853376776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51e00f55c0 0x7f51e0109c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:42.546 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.540+0000 7f51e5add700 1 --2- 172.21.15.27:0/3853376776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51e00f6860 0x7f51e010c180 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:42.546 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.540+0000 7f51e5add700 1 -- 172.21.15.27:0/3853376776 >> 172.21.15.27:0/3853376776 conn(0x7f51e00f0d40 msgr2=0x7f51e00fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:36:42.546 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.540+0000 7f51e5add700 1 -- 172.21.15.27:0/3853376776 shutdown_connections 2021-11-03T18:36:42.547 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.540+0000 7f51e5add700 1 -- 172.21.15.27:0/3853376776 wait complete. 2021-11-03T18:36:42.640 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.639+0000 7f7868346700 1 -- 172.21.15.27:0/4023367460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7860060390 msgr2=0x7f78600f9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:36:42.640 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.639+0000 7f7868346700 1 --2- 172.21.15.27:0/4023367460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7860060390 0x7f78600f9360 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 rx=0x7f7858004660 tx=0x7f7858009b30).stop 2021-11-03T18:36:42.641 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.639+0000 7f7868346700 1 -- 172.21.15.27:0/4023367460 shutdown_connections 2021-11-03T18:36:42.641 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.639+0000 7f7868346700 1 --2- 172.21.15.27:0/4023367460 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f78600f9850 0x7f78600fbca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:42.641 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.639+0000 7f7868346700 1 --2- 172.21.15.27:0/4023367460 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7860060390 0x7f78600f9360 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:42.642 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.639+0000 7f7868346700 1 -- 172.21.15.27:0/4023367460 >> 172.21.15.27:0/4023367460 conn(0x7f78600f0d40 msgr2=0x7f78600f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:36:42.642 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.639+0000 7f7868346700 1 -- 172.21.15.27:0/4023367460 shutdown_connections 2021-11-03T18:36:42.642 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.639+0000 7f7868346700 1 -- 172.21.15.27:0/4023367460 wait complete. 2021-11-03T18:36:42.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.639+0000 7f7868346700 1 Processor -- start 2021-11-03T18:36:42.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.639+0000 7f7868346700 1 -- start start 2021-11-03T18:36:42.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.640+0000 7f7868346700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7860060390 0x7f7860106fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:36:42.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.640+0000 7f7868346700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78600f9850 0x7f7860107490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:36:42.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.640+0000 7f7868346700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7860107a40 con 0x7f78600f9850 2021-11-03T18:36:42.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.640+0000 7f7868346700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7860107b80 con 0x7f7860060390 2021-11-03T18:36:42.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.640+0000 7f78658e1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78600f9850 0x7f7860107490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:36:42.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.640+0000 7f78658e1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78600f9850 0x7f7860107490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35850/0 (socket says 172.21.15.27:35850) 2021-11-03T18:36:42.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.640+0000 7f78658e1700 1 -- 172.21.15.27:0/1407668010 learned_addr learned my addr 172.21.15.27:0/1407668010 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:36:42.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.640+0000 7f78660e2700 1 --2- 172.21.15.27:0/1407668010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7860060390 0x7f7860106fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:36:42.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.640+0000 7f78658e1700 1 -- 172.21.15.27:0/1407668010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7860060390 msgr2=0x7f7860106fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:36:42.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.640+0000 7f78658e1700 1 --2- 172.21.15.27:0/1407668010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7860060390 0x7f7860106fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:42.646 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.640+0000 7f78658e1700 1 -- 172.21.15.27:0/1407668010 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7858005040 con 0x7f78600f9850 2021-11-03T18:36:42.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.640+0000 7f78658e1700 1 --2- 172.21.15.27:0/1407668010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78600f9850 0x7f7860107490 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 rx=0x7f7854002700 tx=0x7f7854009d60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:36:42.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.640+0000 7f785effd700 1 -- 172.21.15.27:0/1407668010 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78540079c0 con 0x7f78600f9850 2021-11-03T18:36:42.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.640+0000 7f785effd700 1 -- 172.21.15.27:0/1407668010 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f785400f040 con 0x7f78600f9850 2021-11-03T18:36:42.647 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.640+0000 7f7868346700 1 -- 172.21.15.27:0/1407668010 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7860108520 con 0x7f78600f9850 2021-11-03T18:36:42.648 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.640+0000 7f785effd700 1 -- 172.21.15.27:0/1407668010 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7854014740 con 0x7f78600f9850 2021-11-03T18:36:42.648 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.640+0000 7f7868346700 1 -- 172.21.15.27:0/1407668010 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7860108ae0 con 0x7f78600f9850 2021-11-03T18:36:42.648 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.641+0000 7f7868346700 1 -- 172.21.15.27:0/1407668010 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7860045ad0 con 0x7f78600f9850 2021-11-03T18:36:42.650 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.642+0000 7f785effd700 1 -- 172.21.15.27:0/1407668010 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f785401b060 con 0x7f78600f9850 2021-11-03T18:36:42.650 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.642+0000 7f785effd700 1 --2- 172.21.15.27:0/1407668010 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7848064180 0x7f7848066640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:36:42.650 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.642+0000 7f785effd700 1 -- 172.21.15.27:0/1407668010 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f7854080510 con 0x7f78600f9850 2021-11-03T18:36:42.651 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.642+0000 7f78660e2700 1 --2- 172.21.15.27:0/1407668010 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7848064180 0x7f7848066640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:36:42.651 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.643+0000 7f78660e2700 1 --2- 172.21.15.27:0/1407668010 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7848064180 0x7f7848066640 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7f7858005010 tx=0x7f7858009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:36:42.651 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.646+0000 7f785effd700 1 -- 172.21.15.27:0/1407668010 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f78540529a0 con 0x7f78600f9850 2021-11-03T18:36:42.872 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.872+0000 7f7868346700 1 -- 172.21.15.27:0/1407668010 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f78601086b0 con 0x7f78600f9850 2021-11-03T18:36:42.872 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.873+0000 7f785effd700 1 -- 172.21.15.27:0/1407668010 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f78540192e0 con 0x7f78600f9850 2021-11-03T18:36:42.872 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:36:42.873 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:36:42.873 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:36:42.873 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:36:42.873 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:36:42.874 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:36:42.874 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:36:42.874 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:36:42.874 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:36:42.875 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:36:42.875 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:36:42.875 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:36:42.875 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:36:42.875 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:36:42.876 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:36:42.876 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:36:42.876 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:36:42.876 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:36:42.877 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.874+0000 7f7868346700 1 -- 172.21.15.27:0/1407668010 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7848064180 msgr2=0x7f7848066640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:36:42.877 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.874+0000 7f7868346700 1 --2- 172.21.15.27:0/1407668010 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7848064180 0x7f7848066640 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7f7858005010 tx=0x7f7858009b30).stop 2021-11-03T18:36:42.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.874+0000 7f7868346700 1 -- 172.21.15.27:0/1407668010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78600f9850 msgr2=0x7f7860107490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:36:42.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.874+0000 7f7868346700 1 --2- 172.21.15.27:0/1407668010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78600f9850 0x7f7860107490 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 rx=0x7f7854002700 tx=0x7f7854009d60).stop 2021-11-03T18:36:42.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.874+0000 7f7868346700 1 -- 172.21.15.27:0/1407668010 shutdown_connections 2021-11-03T18:36:42.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.874+0000 7f7868346700 1 --2- 172.21.15.27:0/1407668010 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7848064180 0x7f7848066640 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:42.879 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.874+0000 7f7868346700 1 --2- 172.21.15.27:0/1407668010 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7860060390 0x7f7860106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:42.880 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.874+0000 7f7868346700 1 --2- 172.21.15.27:0/1407668010 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f78600f9850 0x7f7860107490 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:42.880 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.874+0000 7f7868346700 1 -- 172.21.15.27:0/1407668010 >> 172.21.15.27:0/1407668010 conn(0x7f78600f0d40 msgr2=0x7f78600f3130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:36:42.880 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.875+0000 7f7868346700 1 -- 172.21.15.27:0/1407668010 shutdown_connections 2021-11-03T18:36:42.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.875+0000 7f7868346700 1 -- 172.21.15.27:0/1407668010 wait complete. 2021-11-03T18:36:42.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:42 smithi027 ceph-mon[97244]: pgmap v2463: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:42.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.975+0000 7f528a374700 1 -- 172.21.15.27:0/4270155698 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52840f77d0 msgr2=0x7f52840f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:36:42.977 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.975+0000 7f528a374700 1 --2- 172.21.15.27:0/4270155698 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52840f77d0 0x7f52840f7c30 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 rx=0x7f5274004660 tx=0x7f5274009b30).stop 2021-11-03T18:36:42.977 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.975+0000 7f528a374700 1 -- 172.21.15.27:0/4270155698 shutdown_connections 2021-11-03T18:36:42.977 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.975+0000 7f528a374700 1 --2- 172.21.15.27:0/4270155698 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52840f77d0 0x7f52840f7c30 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:42.977 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.975+0000 7f528a374700 1 --2- 172.21.15.27:0/4270155698 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52840f6530 0x7f52840f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:42.978 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.975+0000 7f528a374700 1 -- 172.21.15.27:0/4270155698 >> 172.21.15.27:0/4270155698 conn(0x7f52840f1cd0 msgr2=0x7f52840f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:36:42.978 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.975+0000 7f528a374700 1 -- 172.21.15.27:0/4270155698 shutdown_connections 2021-11-03T18:36:42.978 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.975+0000 7f528a374700 1 -- 172.21.15.27:0/4270155698 wait complete. 2021-11-03T18:36:42.978 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.976+0000 7f528a374700 1 Processor -- start 2021-11-03T18:36:42.979 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.976+0000 7f528a374700 1 -- start start 2021-11-03T18:36:42.979 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.976+0000 7f528a374700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52840f6530 0x7f528406b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:36:42.979 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.976+0000 7f528a374700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52840f77d0 0x7f5284069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:36:42.979 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.976+0000 7f528a374700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528406a380 con 0x7f52840f77d0 2021-11-03T18:36:42.980 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.976+0000 7f528a374700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f528406a4c0 con 0x7f52840f6530 2021-11-03T18:36:42.980 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.976+0000 7f5288b71700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52840f77d0 0x7f5284069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:36:42.980 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.976+0000 7f5288b71700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52840f77d0 0x7f5284069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35860/0 (socket says 172.21.15.27:35860) 2021-11-03T18:36:42.980 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.976+0000 7f5288b71700 1 -- 172.21.15.27:0/1976891523 learned_addr learned my addr 172.21.15.27:0/1976891523 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:36:42.981 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.976+0000 7f5289372700 1 --2- 172.21.15.27:0/1976891523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52840f6530 0x7f528406b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:36:42.981 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.977+0000 7f5288b71700 1 -- 172.21.15.27:0/1976891523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52840f6530 msgr2=0x7f528406b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:36:42.981 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.977+0000 7f5288b71700 1 --2- 172.21.15.27:0/1976891523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52840f6530 0x7f528406b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:42.981 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.977+0000 7f5288b71700 1 -- 172.21.15.27:0/1976891523 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5274005040 con 0x7f52840f77d0 2021-11-03T18:36:42.982 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.977+0000 7f5288b71700 1 --2- 172.21.15.27:0/1976891523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52840f77d0 0x7f5284069e20 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 rx=0x7f5274009990 tx=0x7f5274009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:36:42.982 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.977+0000 7f5279ffb700 1 -- 172.21.15.27:0/1976891523 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5274018070 con 0x7f52840f77d0 2021-11-03T18:36:42.982 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.977+0000 7f5279ffb700 1 -- 172.21.15.27:0/1976891523 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f5274007ba0 con 0x7f52840f77d0 2021-11-03T18:36:42.982 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.977+0000 7f528a374700 1 -- 172.21.15.27:0/1976891523 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f528406a740 con 0x7f52840f77d0 2021-11-03T18:36:42.982 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.977+0000 7f5279ffb700 1 -- 172.21.15.27:0/1976891523 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f527401c3c0 con 0x7f52840f77d0 2021-11-03T18:36:42.983 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.977+0000 7f528a374700 1 -- 172.21.15.27:0/1976891523 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f528406aca0 con 0x7f52840f77d0 2021-11-03T18:36:42.984 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.978+0000 7f528a374700 1 -- 172.21.15.27:0/1976891523 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5284045ad0 con 0x7f52840f77d0 2021-11-03T18:36:42.984 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.978+0000 7f5279ffb700 1 -- 172.21.15.27:0/1976891523 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f527401d420 con 0x7f52840f77d0 2021-11-03T18:36:42.985 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.979+0000 7f5279ffb700 1 --2- 172.21.15.27:0/1976891523 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5270064100 0x7f52700665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:36:42.985 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.979+0000 7f5279ffb700 1 -- 172.21.15.27:0/1976891523 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f5274082500 con 0x7f52840f77d0 2021-11-03T18:36:42.985 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.979+0000 7f5289372700 1 --2- 172.21.15.27:0/1976891523 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5270064100 0x7f52700665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:36:42.985 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.979+0000 7f5289372700 1 --2- 172.21.15.27:0/1976891523 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5270064100 0x7f52700665c0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7f528406b730 tx=0x7f5280009c00).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:36:42.986 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:42.983+0000 7f5279ffb700 1 -- 172.21.15.27:0/1976891523 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f52740548c0 con 0x7f52840f77d0 2021-11-03T18:36:42.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:42 smithi115 ceph-mon[65453]: pgmap v2463: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:43.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:43.189+0000 7f528a374700 1 -- 172.21.15.27:0/1976891523 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5284027a00 con 0x7f52840f77d0 2021-11-03T18:36:43.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:43.190+0000 7f5279ffb700 1 -- 172.21.15.27:0/1976891523 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f52740544b0 con 0x7f52840f77d0 2021-11-03T18:36:43.189 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:36:43.190 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:36:43.190 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:36:43.190 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:36:43.190 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:36:43.191 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:36:43.191 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:36:43.191 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:36:43.191 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:36:43.191 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:36:43.192 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:36:43.192 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:36:43.192 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:36:43.192 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:36:43.193 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:36:43.193 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:36:43.193 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:36:43.193 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:36:43.194 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:36:43.194 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:36:43.194 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:36:43.194 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:36:43.195 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:36:43.195 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:36:43.195 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:36:43.195 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:36:43.195 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:36:43.196 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:36:43.196 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:36:43.196 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:36:43.196 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:36:43.197 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:36:43.197 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:36:43.197 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:36:43.197 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:36:43.197 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:36:43.198 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:36:43.198 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:36:43.198 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:36:43.199 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:43.191+0000 7f528a374700 1 -- 172.21.15.27:0/1976891523 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5270064100 msgr2=0x7f52700665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:36:43.199 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:43.191+0000 7f528a374700 1 --2- 172.21.15.27:0/1976891523 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5270064100 0x7f52700665c0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 rx=0x7f528406b730 tx=0x7f5280009c00).stop 2021-11-03T18:36:43.200 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:43.191+0000 7f528a374700 1 -- 172.21.15.27:0/1976891523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52840f77d0 msgr2=0x7f5284069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:36:43.200 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:43.191+0000 7f528a374700 1 --2- 172.21.15.27:0/1976891523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52840f77d0 0x7f5284069e20 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 rx=0x7f5274009990 tx=0x7f5274009b30).stop 2021-11-03T18:36:43.200 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:43.192+0000 7f528a374700 1 -- 172.21.15.27:0/1976891523 shutdown_connections 2021-11-03T18:36:43.200 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:43.192+0000 7f528a374700 1 --2- 172.21.15.27:0/1976891523 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5270064100 0x7f52700665c0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:43.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:43.192+0000 7f528a374700 1 --2- 172.21.15.27:0/1976891523 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f52840f6530 0x7f528406b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:43.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:43.192+0000 7f528a374700 1 --2- 172.21.15.27:0/1976891523 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f52840f77d0 0x7f5284069e20 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:36:43.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:43.192+0000 7f528a374700 1 -- 172.21.15.27:0/1976891523 >> 172.21.15.27:0/1976891523 conn(0x7f52840f1cd0 msgr2=0x7f52840fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:36:43.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:43.192+0000 7f528a374700 1 -- 172.21.15.27:0/1976891523 shutdown_connections 2021-11-03T18:36:43.201 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:36:43.192+0000 7f528a374700 1 -- 172.21.15.27:0/1976891523 wait complete. 2021-11-03T18:36:43.202 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:36:43.959 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:43 smithi115 ceph-mon[65453]: from='client.36632 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:36:43.959 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:43 smithi115 ceph-mon[65453]: from='client.36636 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:36:43.960 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:43 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1407668010' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:43.960 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:43 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1976891523' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:36:44.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:43 smithi027 ceph-mon[97244]: from='client.36632 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:36:44.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:43 smithi027 ceph-mon[97244]: from='client.36636 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:36:44.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:43 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1407668010' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:44.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:43 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1976891523' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:36:44.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:44 smithi115 ceph-mon[65453]: pgmap v2464: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:45.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:44 smithi027 ceph-mon[97244]: pgmap v2464: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:45.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4847.476221 secs 2021-11-03T18:36:46.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4847.476221 secs 2021-11-03T18:36:46.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:46 smithi115 ceph-mon[65453]: pgmap v2465: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:47.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:46 smithi027 ceph-mon[97244]: pgmap v2465: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:48.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:48 smithi115 ceph-mon[65453]: pgmap v2466: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:48.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:36:48.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:36:48.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:36:48.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:48.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:48.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:48.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:49.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:48 smithi027 ceph-mon[97244]: pgmap v2466: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:49.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:36:49.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:36:49.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:36:49.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:49.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:49.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:49.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:49.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:49 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:36:50.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:49 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:36:50.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:50 smithi115 ceph-mon[65453]: pgmap v2467: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:50.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4852.476364 secs 2021-11-03T18:36:51.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:50 smithi027 ceph-mon[97244]: pgmap v2467: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:51.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4852.476364 secs 2021-11-03T18:36:52.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:52 smithi027 ceph-mon[97244]: pgmap v2468: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:52.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:52 smithi115 ceph-mon[65453]: pgmap v2468: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:54.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:54 smithi115 ceph-mon[65453]: pgmap v2469: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:55.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:54 smithi027 ceph-mon[97244]: pgmap v2469: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:55.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4857.476505 secs 2021-11-03T18:36:56.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4857.476505 secs 2021-11-03T18:36:56.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:56 smithi115 ceph-mon[65453]: pgmap v2470: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:56.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:36:56.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:36:57.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:56 smithi027 ceph-mon[97244]: pgmap v2470: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:57.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:36:57.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:36:58.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:58 smithi115 ceph-mon[65453]: pgmap v2471: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:58.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:36:58.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:36:58.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:36:58.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:58.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:58.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:58.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:59.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:58 smithi027 ceph-mon[97244]: pgmap v2471: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:36:59.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:36:59.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:36:59.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:36:59.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:59.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:59.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:59.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:36:59.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:36:59 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:37:00.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:36:59 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:37:00.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:00 smithi115 ceph-mon[65453]: pgmap v2472: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:00.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4862.476653 secs 2021-11-03T18:37:01.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:00 smithi027 ceph-mon[97244]: pgmap v2472: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:01.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4862.476653 secs 2021-11-03T18:37:02.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:02 smithi027 ceph-mon[97244]: pgmap v2473: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:02.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:02 smithi115 ceph-mon[65453]: pgmap v2473: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:04.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:04 smithi115 ceph-mon[65453]: pgmap v2474: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:05.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:04 smithi027 ceph-mon[97244]: pgmap v2474: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:05.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4867.476792 secs 2021-11-03T18:37:06.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4867.476792 secs 2021-11-03T18:37:06.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:06 smithi115 ceph-mon[65453]: pgmap v2475: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:07.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:06 smithi027 ceph-mon[97244]: pgmap v2475: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:08.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:08 smithi115 ceph-mon[65453]: pgmap v2476: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:08.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:37:08.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:37:08.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:37:08.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:08.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:08.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:08.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:09.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:08 smithi027 ceph-mon[97244]: pgmap v2476: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:09.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:37:09.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:37:09.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:37:09.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:09.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:09.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:09.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:09.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:09 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:37:10.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:09 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:37:10.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:10 smithi115 ceph-mon[65453]: pgmap v2477: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:10.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4872.476897 secs 2021-11-03T18:37:11.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:10 smithi027 ceph-mon[97244]: pgmap v2477: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:11.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4872.476897 secs 2021-11-03T18:37:12.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:12 smithi027 ceph-mon[97244]: pgmap v2478: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:12.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:12 smithi115 ceph-mon[65453]: pgmap v2478: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:13.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.303+0000 7f7cf9712700 1 -- 172.21.15.27:0/608284644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cf40f5ed0 msgr2=0x7f7cf40f6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:37:13.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.303+0000 7f7cf9712700 1 --2- 172.21.15.27:0/608284644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cf40f5ed0 0x7f7cf40f6900 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 rx=0x7f7ce4004660 tx=0x7f7ce4009b30).stop 2021-11-03T18:37:13.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.303+0000 7f7cf9712700 1 -- 172.21.15.27:0/608284644 shutdown_connections 2021-11-03T18:37:13.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.303+0000 7f7cf9712700 1 --2- 172.21.15.27:0/608284644 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cf40f5ed0 0x7f7cf40f6900 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:13.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.303+0000 7f7cf9712700 1 --2- 172.21.15.27:0/608284644 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cf40f55c0 0x7f7cf40f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:13.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.303+0000 7f7cf9712700 1 -- 172.21.15.27:0/608284644 >> 172.21.15.27:0/608284644 conn(0x7f7cf40f0de0 msgr2=0x7f7cf40f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:37:13.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.303+0000 7f7cf9712700 1 -- 172.21.15.27:0/608284644 shutdown_connections 2021-11-03T18:37:13.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.303+0000 7f7cf9712700 1 -- 172.21.15.27:0/608284644 wait complete. 2021-11-03T18:37:13.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.304+0000 7f7cf9712700 1 Processor -- start 2021-11-03T18:37:13.307 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.304+0000 7f7cf9712700 1 -- start start 2021-11-03T18:37:13.307 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.304+0000 7f7cf9712700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cf40f55c0 0x7f7cf4109c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:37:13.307 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.304+0000 7f7cf9712700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cf40f5ed0 0x7f7cf410c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:37:13.307 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.304+0000 7f7cf9712700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf40604a0 con 0x7f7cf40f5ed0 2021-11-03T18:37:13.308 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.304+0000 7f7cf9712700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf40605e0 con 0x7f7cf40f55c0 2021-11-03T18:37:13.308 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.304+0000 7f7cf27fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cf40f5ed0 0x7f7cf410c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:37:13.308 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.304+0000 7f7cf27fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cf40f5ed0 0x7f7cf410c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35878/0 (socket says 172.21.15.27:35878) 2021-11-03T18:37:13.308 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.304+0000 7f7cf27fc700 1 -- 172.21.15.27:0/757237697 learned_addr learned my addr 172.21.15.27:0/757237697 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:37:13.309 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.304+0000 7f7cf2ffd700 1 --2- 172.21.15.27:0/757237697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cf40f55c0 0x7f7cf4109c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:37:13.309 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.305+0000 7f7cf27fc700 1 -- 172.21.15.27:0/757237697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cf40f55c0 msgr2=0x7f7cf4109c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:37:13.309 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.305+0000 7f7cf27fc700 1 --2- 172.21.15.27:0/757237697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cf40f55c0 0x7f7cf4109c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:13.309 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.305+0000 7f7cf27fc700 1 -- 172.21.15.27:0/757237697 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ce4005040 con 0x7f7cf40f5ed0 2021-11-03T18:37:13.310 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.305+0000 7f7cf27fc700 1 --2- 172.21.15.27:0/757237697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cf40f5ed0 0x7f7cf410c190 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 rx=0x7f7ce4009990 tx=0x7f7ce4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:37:13.310 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.305+0000 7f7ceb7fe700 1 -- 172.21.15.27:0/757237697 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ce4018070 con 0x7f7cf40f5ed0 2021-11-03T18:37:13.310 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.305+0000 7f7ceb7fe700 1 -- 172.21.15.27:0/757237697 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f7ce4007ba0 con 0x7f7cf40f5ed0 2021-11-03T18:37:13.310 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.305+0000 7f7cf9712700 1 -- 172.21.15.27:0/757237697 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cf410c6e0 con 0x7f7cf40f5ed0 2021-11-03T18:37:13.310 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.305+0000 7f7ceb7fe700 1 -- 172.21.15.27:0/757237697 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7ce401c3c0 con 0x7f7cf40f5ed0 2021-11-03T18:37:13.311 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.305+0000 7f7cf9712700 1 -- 172.21.15.27:0/757237697 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cf410cc10 con 0x7f7cf40f5ed0 2021-11-03T18:37:13.311 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.306+0000 7f7cf9712700 1 -- 172.21.15.27:0/757237697 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cf40f4a50 con 0x7f7cf40f5ed0 2021-11-03T18:37:13.311 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.307+0000 7f7ceb7fe700 1 -- 172.21.15.27:0/757237697 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f7ce401d420 con 0x7f7cf40f5ed0 2021-11-03T18:37:13.311 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.307+0000 7f7ceb7fe700 1 --2- 172.21.15.27:0/757237697 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7ce00640b0 0x7f7ce0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:37:13.312 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.307+0000 7f7ceb7fe700 1 -- 172.21.15.27:0/757237697 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f7ce4082390 con 0x7f7cf40f5ed0 2021-11-03T18:37:13.312 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.307+0000 7f7cf2ffd700 1 --2- 172.21.15.27:0/757237697 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7ce00640b0 0x7f7ce0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:37:13.312 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.307+0000 7f7cf2ffd700 1 --2- 172.21.15.27:0/757237697 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7ce00640b0 0x7f7ce0066570 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7f7cf4060120 tx=0x7f7cdc00b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:37:13.313 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.311+0000 7f7ceb7fe700 1 -- 172.21.15.27:0/757237697 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f7ce40547b0 con 0x7f7cf40f5ed0 2021-11-03T18:37:13.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.491+0000 7f7cf9712700 1 -- 172.21.15.27:0/757237697 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7cf40fdf20 con 0x7f7ce00640b0 2021-11-03T18:37:13.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.493+0000 7f7ceb7fe700 1 -- 172.21.15.27:0/757237697 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f7cf40fdf20 con 0x7f7ce00640b0 2021-11-03T18:37:13.494 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.495+0000 7f7cf9712700 1 -- 172.21.15.27:0/757237697 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7ce00640b0 msgr2=0x7f7ce0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:37:13.494 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.495+0000 7f7cf9712700 1 --2- 172.21.15.27:0/757237697 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7ce00640b0 0x7f7ce0066570 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7f7cf4060120 tx=0x7f7cdc00b040).stop 2021-11-03T18:37:13.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.495+0000 7f7cf9712700 1 -- 172.21.15.27:0/757237697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cf40f5ed0 msgr2=0x7f7cf410c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:37:13.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.495+0000 7f7cf9712700 1 --2- 172.21.15.27:0/757237697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cf40f5ed0 0x7f7cf410c190 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 rx=0x7f7ce4009990 tx=0x7f7ce4009b30).stop 2021-11-03T18:37:13.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.495+0000 7f7cf9712700 1 -- 172.21.15.27:0/757237697 shutdown_connections 2021-11-03T18:37:13.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.495+0000 7f7cf9712700 1 --2- 172.21.15.27:0/757237697 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7ce00640b0 0x7f7ce0066570 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:13.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.495+0000 7f7cf9712700 1 --2- 172.21.15.27:0/757237697 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7cf40f55c0 0x7f7cf4109c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:13.496 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.495+0000 7f7cf9712700 1 --2- 172.21.15.27:0/757237697 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7cf40f5ed0 0x7f7cf410c190 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:13.496 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.495+0000 7f7cf9712700 1 -- 172.21.15.27:0/757237697 >> 172.21.15.27:0/757237697 conn(0x7f7cf40f0de0 msgr2=0x7f7cf40fa350 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:37:13.496 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.495+0000 7f7cf9712700 1 -- 172.21.15.27:0/757237697 shutdown_connections 2021-11-03T18:37:13.496 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.495+0000 7f7cf9712700 1 -- 172.21.15.27:0/757237697 wait complete. 2021-11-03T18:37:13.504 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:37:13.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.595+0000 7fd978be6700 1 -- 172.21.15.27:0/2837037395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9740f66e0 msgr2=0x7fd9740f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:37:13.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.595+0000 7fd978be6700 1 --2- 172.21.15.27:0/2837037395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9740f66e0 0x7fd9740f6b00 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 rx=0x7fd95c004660 tx=0x7fd95c009b30).stop 2021-11-03T18:37:13.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.595+0000 7fd978be6700 1 -- 172.21.15.27:0/2837037395 shutdown_connections 2021-11-03T18:37:13.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.595+0000 7fd978be6700 1 --2- 172.21.15.27:0/2837037395 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9740f6ff0 0x7fd9740f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:13.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.595+0000 7fd978be6700 1 --2- 172.21.15.27:0/2837037395 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9740f66e0 0x7fd9740f6b00 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:13.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.595+0000 7fd978be6700 1 -- 172.21.15.27:0/2837037395 >> 172.21.15.27:0/2837037395 conn(0x7fd9740f1f00 msgr2=0x7fd9740f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:37:13.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.595+0000 7fd978be6700 1 -- 172.21.15.27:0/2837037395 shutdown_connections 2021-11-03T18:37:13.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.595+0000 7fd978be6700 1 -- 172.21.15.27:0/2837037395 wait complete. 2021-11-03T18:37:13.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.596+0000 7fd978be6700 1 Processor -- start 2021-11-03T18:37:13.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.596+0000 7fd978be6700 1 -- start start 2021-11-03T18:37:13.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.596+0000 7fd978be6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9740f66e0 0x7fd97406b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:37:13.600 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.596+0000 7fd978be6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9740f6ff0 0x7fd974069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:37:13.600 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.596+0000 7fd978be6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97406a380 con 0x7fd9740f66e0 2021-11-03T18:37:13.600 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.596+0000 7fd978be6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97406a4c0 con 0x7fd9740f6ff0 2021-11-03T18:37:13.601 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.596+0000 7fd97259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9740f66e0 0x7fd97406b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:37:13.601 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.596+0000 7fd97259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9740f66e0 0x7fd97406b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35886/0 (socket says 172.21.15.27:35886) 2021-11-03T18:37:13.601 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.596+0000 7fd97259c700 1 -- 172.21.15.27:0/2898798503 learned_addr learned my addr 172.21.15.27:0/2898798503 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:37:13.602 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.596+0000 7fd97259c700 1 -- 172.21.15.27:0/2898798503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9740f6ff0 msgr2=0x7fd974069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:37:13.602 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.596+0000 7fd97259c700 1 --2- 172.21.15.27:0/2898798503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9740f6ff0 0x7fd974069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:37:13.603 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.596+0000 7fd97259c700 1 -- 172.21.15.27:0/2898798503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd95c005040 con 0x7fd9740f66e0 2021-11-03T18:37:13.603 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.596+0000 7fd97259c700 1 --2- 172.21.15.27:0/2898798503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9740f66e0 0x7fd97406b9a0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 rx=0x7fd95c009990 tx=0x7fd95c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:37:13.603 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.596+0000 7fd96affd700 1 -- 172.21.15.27:0/2898798503 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd95c018070 con 0x7fd9740f66e0 2021-11-03T18:37:13.604 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.596+0000 7fd96affd700 1 -- 172.21.15.27:0/2898798503 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fd95c007ca0 con 0x7fd9740f66e0 2021-11-03T18:37:13.604 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.596+0000 7fd978be6700 1 -- 172.21.15.27:0/2898798503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd97406a740 con 0x7fd9740f66e0 2021-11-03T18:37:13.604 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.596+0000 7fd96affd700 1 -- 172.21.15.27:0/2898798503 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd95c01c620 con 0x7fd9740f66e0 2021-11-03T18:37:13.605 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.596+0000 7fd978be6700 1 -- 172.21.15.27:0/2898798503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd97406aca0 con 0x7fd9740f66e0 2021-11-03T18:37:13.605 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.597+0000 7fd978be6700 1 -- 172.21.15.27:0/2898798503 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd974045ad0 con 0x7fd9740f66e0 2021-11-03T18:37:13.605 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.598+0000 7fd96affd700 1 -- 172.21.15.27:0/2898798503 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fd95c01d960 con 0x7fd9740f66e0 2021-11-03T18:37:13.606 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.598+0000 7fd96affd700 1 --2- 172.21.15.27:0/2898798503 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd9600640b0 0x7fd960066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:37:13.606 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.598+0000 7fd96affd700 1 -- 172.21.15.27:0/2898798503 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fd95c082540 con 0x7fd9740f66e0 2021-11-03T18:37:13.606 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.598+0000 7fd971d9b700 1 --2- 172.21.15.27:0/2898798503 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd9600640b0 0x7fd960066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:37:13.607 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.599+0000 7fd971d9b700 1 --2- 172.21.15.27:0/2898798503 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd9600640b0 0x7fd960066570 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7fd964004880 tx=0x7fd964007ae0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:37:13.607 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.603+0000 7fd96affd700 1 -- 172.21.15.27:0/2898798503 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fd95c054960 con 0x7fd9740f66e0 2021-11-03T18:37:13.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.777+0000 7fd978be6700 1 -- 172.21.15.27:0/2898798503 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd974057f10 con 0x7fd9600640b0 2021-11-03T18:37:13.791 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.792+0000 7fd96affd700 1 -- 172.21.15.27:0/2898798503 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fd974057f10 con 0x7fd9600640b0 2021-11-03T18:37:13.791 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:37:13.791 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (88m) 7m ago 90m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:37:13.792 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (80m) 7m ago 89m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:37:13.792 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (80m) 6m ago 89m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:37:13.792 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (88m) 7m ago 89m 42.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:37:13.792 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (82m) 7m ago 85m 26.7M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:37:13.793 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (85m) 7m ago 85m 3850M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:37:13.793 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (81m) 6m ago 85m 26.3M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:37:13.793 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (82m) 6m ago 85m 27.4M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:37:13.793 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (82m) 7m ago 91m 608M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:37:13.793 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (81m) 6m ago 89m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:37:13.794 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (81m) 7m ago 91m 350M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:37:13.794 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (81m) 6m ago 89m 160M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:37:13.794 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (89m) 7m ago 89m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:37:13.794 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (89m) 6m ago 89m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:37:13.795 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (80m) 7m ago 88m 413M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:37:13.795 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (79m) 7m ago 88m 450M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:37:13.795 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (79m) 7m ago 87m 338M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:37:13.795 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (78m) 6m ago 87m 479M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:37:13.795 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (78m) 6m ago 87m 378M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:37:13.796 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (77m) 6m ago 86m 388M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:37:13.796 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (88m) 7m ago 89m 64.4M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:37:13.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.793+0000 7fd978be6700 1 -- 172.21.15.27:0/2898798503 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd9600640b0 msgr2=0x7fd960066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:37:13.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.793+0000 7fd978be6700 1 --2- 172.21.15.27:0/2898798503 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd9600640b0 0x7fd960066570 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7fd964004880 tx=0x7fd964007ae0).stop 2021-11-03T18:37:13.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.793+0000 7fd978be6700 1 -- 172.21.15.27:0/2898798503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9740f66e0 msgr2=0x7fd97406b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:37:13.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.793+0000 7fd978be6700 1 --2- 172.21.15.27:0/2898798503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9740f66e0 0x7fd97406b9a0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 rx=0x7fd95c009990 tx=0x7fd95c009b30).stop 2021-11-03T18:37:13.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.794+0000 7fd978be6700 1 -- 172.21.15.27:0/2898798503 shutdown_connections 2021-11-03T18:37:13.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.794+0000 7fd978be6700 1 --2- 172.21.15.27:0/2898798503 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd9600640b0 0x7fd960066570 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:13.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.794+0000 7fd978be6700 1 --2- 172.21.15.27:0/2898798503 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd9740f66e0 0x7fd97406b9a0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:13.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.794+0000 7fd978be6700 1 --2- 172.21.15.27:0/2898798503 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd9740f6ff0 0x7fd974069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:37:13.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.794+0000 7fd978be6700 1 -- 172.21.15.27:0/2898798503 >> 172.21.15.27:0/2898798503 conn(0x7fd9740f1f00 msgr2=0x7fd9740fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:37:13.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.794+0000 7fd978be6700 1 -- 172.21.15.27:0/2898798503 shutdown_connections 2021-11-03T18:37:13.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.794+0000 7fd978be6700 1 -- 172.21.15.27:0/2898798503 wait complete. 2021-11-03T18:37:13.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.893+0000 7fe8f6157700 1 -- 172.21.15.27:0/2075775633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8f00f66e0 msgr2=0x7fe8f00f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:37:13.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.893+0000 7fe8f6157700 1 --2- 172.21.15.27:0/2075775633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8f00f66e0 0x7fe8f00f6b00 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 rx=0x7fe8e0004660 tx=0x7fe8e0009b30).stop 2021-11-03T18:37:13.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.893+0000 7fe8f6157700 1 -- 172.21.15.27:0/2075775633 shutdown_connections 2021-11-03T18:37:13.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.893+0000 7fe8f6157700 1 --2- 172.21.15.27:0/2075775633 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8f00f6ff0 0x7fe8f00f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:13.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.893+0000 7fe8f6157700 1 --2- 172.21.15.27:0/2075775633 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8f00f66e0 0x7fe8f00f6b00 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:13.895 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.893+0000 7fe8f6157700 1 -- 172.21.15.27:0/2075775633 >> 172.21.15.27:0/2075775633 conn(0x7fe8f00f1f00 msgr2=0x7fe8f00f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:37:13.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.893+0000 7fe8f6157700 1 -- 172.21.15.27:0/2075775633 shutdown_connections 2021-11-03T18:37:13.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.893+0000 7fe8f6157700 1 -- 172.21.15.27:0/2075775633 wait complete. 2021-11-03T18:37:13.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.894+0000 7fe8f6157700 1 Processor -- start 2021-11-03T18:37:13.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.894+0000 7fe8f6157700 1 -- start start 2021-11-03T18:37:13.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.894+0000 7fe8f6157700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8f00f66e0 0x7fe8f0106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:37:13.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.894+0000 7fe8f6157700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8f00f6ff0 0x7fe8f0107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:37:13.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.894+0000 7fe8f6157700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f0107a00 con 0x7fe8f00f6ff0 2021-11-03T18:37:13.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.894+0000 7fe8f6157700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8f0107b40 con 0x7fe8f00f66e0 2021-11-03T18:37:13.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.894+0000 7fe8eeffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8f00f6ff0 0x7fe8f0107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:37:13.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.894+0000 7fe8eeffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8f00f6ff0 0x7fe8f0107450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35898/0 (socket says 172.21.15.27:35898) 2021-11-03T18:37:13.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.894+0000 7fe8eeffd700 1 -- 172.21.15.27:0/953336957 learned_addr learned my addr 172.21.15.27:0/953336957 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:37:13.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.894+0000 7fe8eeffd700 1 -- 172.21.15.27:0/953336957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8f00f66e0 msgr2=0x7fe8f0106f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:37:13.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.894+0000 7fe8eeffd700 1 --2- 172.21.15.27:0/953336957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8f00f66e0 0x7fe8f0106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:37:13.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.894+0000 7fe8eeffd700 1 -- 172.21.15.27:0/953336957 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e0005040 con 0x7fe8f00f6ff0 2021-11-03T18:37:13.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.895+0000 7fe8eeffd700 1 --2- 172.21.15.27:0/953336957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8f00f6ff0 0x7fe8f0107450 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 rx=0x7fe8e40026d0 tx=0x7fe8e4009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:37:13.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.895+0000 7fe8d7fff700 1 -- 172.21.15.27:0/953336957 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8e4009320 con 0x7fe8f00f6ff0 2021-11-03T18:37:13.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.895+0000 7fe8f6157700 1 -- 172.21.15.27:0/953336957 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8f01084e0 con 0x7fe8f00f6ff0 2021-11-03T18:37:13.900 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.895+0000 7fe8d7fff700 1 -- 172.21.15.27:0/953336957 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fe8e400f040 con 0x7fe8f00f6ff0 2021-11-03T18:37:13.900 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.895+0000 7fe8d7fff700 1 -- 172.21.15.27:0/953336957 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe8e40147e0 con 0x7fe8f00f6ff0 2021-11-03T18:37:13.900 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.895+0000 7fe8f6157700 1 -- 172.21.15.27:0/953336957 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8f0108aa0 con 0x7fe8f00f6ff0 2021-11-03T18:37:13.901 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.896+0000 7fe8f6157700 1 -- 172.21.15.27:0/953336957 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8f0045ad0 con 0x7fe8f00f6ff0 2021-11-03T18:37:13.902 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.896+0000 7fe8d7fff700 1 -- 172.21.15.27:0/953336957 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fe8e4009480 con 0x7fe8f00f6ff0 2021-11-03T18:37:13.902 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.897+0000 7fe8d7fff700 1 --2- 172.21.15.27:0/953336957 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe8d80640b0 0x7fe8d8066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:37:13.902 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.897+0000 7fe8d7fff700 1 -- 172.21.15.27:0/953336957 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fe8e4080eb0 con 0x7fe8f00f6ff0 2021-11-03T18:37:13.903 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.897+0000 7fe8ef7fe700 1 --2- 172.21.15.27:0/953336957 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe8d80640b0 0x7fe8d8066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:37:13.903 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.897+0000 7fe8ef7fe700 1 --2- 172.21.15.27:0/953336957 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe8d80640b0 0x7fe8d8066570 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7fe8e0004e30 tx=0x7fe8e0009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:37:13.903 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:13.901+0000 7fe8d7fff700 1 -- 172.21.15.27:0/953336957 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fe8e4053210 con 0x7fe8f00f6ff0 2021-11-03T18:37:14.128 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.129+0000 7fe8f6157700 1 -- 172.21.15.27:0/953336957 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe8f0108d20 con 0x7fe8f00f6ff0 2021-11-03T18:37:14.129 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.130+0000 7fe8d7fff700 1 -- 172.21.15.27:0/953336957 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fe8e4052e00 con 0x7fe8f00f6ff0 2021-11-03T18:37:14.129 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:37:14.130 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:37:14.130 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:37:14.130 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:37:14.130 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:37:14.131 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:37:14.131 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:37:14.131 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:37:14.132 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:37:14.132 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:37:14.133 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:37:14.133 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:37:14.133 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:37:14.133 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:37:14.134 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:37:14.134 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:37:14.135 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:37:14.135 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:37:14.136 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.131+0000 7fe8f6157700 1 -- 172.21.15.27:0/953336957 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe8d80640b0 msgr2=0x7fe8d8066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:37:14.136 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.131+0000 7fe8f6157700 1 --2- 172.21.15.27:0/953336957 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe8d80640b0 0x7fe8d8066570 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 rx=0x7fe8e0004e30 tx=0x7fe8e0009b30).stop 2021-11-03T18:37:14.137 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.131+0000 7fe8f6157700 1 -- 172.21.15.27:0/953336957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8f00f6ff0 msgr2=0x7fe8f0107450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:37:14.137 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.131+0000 7fe8f6157700 1 --2- 172.21.15.27:0/953336957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8f00f6ff0 0x7fe8f0107450 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 rx=0x7fe8e40026d0 tx=0x7fe8e4009de0).stop 2021-11-03T18:37:14.137 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.131+0000 7fe8f6157700 1 -- 172.21.15.27:0/953336957 shutdown_connections 2021-11-03T18:37:14.137 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.131+0000 7fe8f6157700 1 --2- 172.21.15.27:0/953336957 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe8d80640b0 0x7fe8d8066570 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:14.138 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.131+0000 7fe8f6157700 1 --2- 172.21.15.27:0/953336957 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8f00f66e0 0x7fe8f0106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:37:14.138 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.131+0000 7fe8f6157700 1 --2- 172.21.15.27:0/953336957 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8f00f6ff0 0x7fe8f0107450 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:14.138 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.131+0000 7fe8f6157700 1 -- 172.21.15.27:0/953336957 >> 172.21.15.27:0/953336957 conn(0x7fe8f00f1f00 msgr2=0x7fe8f00fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:37:14.138 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.131+0000 7fe8f6157700 1 -- 172.21.15.27:0/953336957 shutdown_connections 2021-11-03T18:37:14.139 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.131+0000 7fe8f6157700 1 -- 172.21.15.27:0/953336957 wait complete. 2021-11-03T18:37:14.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.232+0000 7feb4d934700 1 -- 172.21.15.27:0/1279066042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb480f6860 msgr2=0x7feb480f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:37:14.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.232+0000 7feb4d934700 1 --2- 172.21.15.27:0/1279066042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb480f6860 0x7feb480f6cc0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 rx=0x7feb30004660 tx=0x7feb30009b30).stop 2021-11-03T18:37:14.234 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.232+0000 7feb4d934700 1 -- 172.21.15.27:0/1279066042 shutdown_connections 2021-11-03T18:37:14.234 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.232+0000 7feb4d934700 1 --2- 172.21.15.27:0/1279066042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb480f6860 0x7feb480f6cc0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:14.234 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.232+0000 7feb4d934700 1 --2- 172.21.15.27:0/1279066042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb480f55c0 0x7feb480f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:14.234 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.232+0000 7feb4d934700 1 -- 172.21.15.27:0/1279066042 >> 172.21.15.27:0/1279066042 conn(0x7feb480f0d40 msgr2=0x7feb480f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:37:14.235 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.232+0000 7feb4d934700 1 -- 172.21.15.27:0/1279066042 shutdown_connections 2021-11-03T18:37:14.235 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.232+0000 7feb4d934700 1 -- 172.21.15.27:0/1279066042 wait complete. 2021-11-03T18:37:14.235 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.232+0000 7feb4d934700 1 Processor -- start 2021-11-03T18:37:14.235 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.232+0000 7feb4d934700 1 -- start start 2021-11-03T18:37:14.236 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.233+0000 7feb4d934700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb480f55c0 0x7feb48105ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:37:14.236 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.233+0000 7feb4d934700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb480f6860 0x7feb481063b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:37:14.236 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.233+0000 7feb4d934700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb481069b0 con 0x7feb480f6860 2021-11-03T18:37:14.236 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.233+0000 7feb4d934700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb481071c0 con 0x7feb480f55c0 2021-11-03T18:37:14.236 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.233+0000 7feb467fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb480f6860 0x7feb481063b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:37:14.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.233+0000 7feb467fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb480f6860 0x7feb481063b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35908/0 (socket says 172.21.15.27:35908) 2021-11-03T18:37:14.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.233+0000 7feb467fc700 1 -- 172.21.15.27:0/39794118 learned_addr learned my addr 172.21.15.27:0/39794118 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:37:14.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.233+0000 7feb46ffd700 1 --2- 172.21.15.27:0/39794118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb480f55c0 0x7feb48105ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:37:14.237 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.233+0000 7feb467fc700 1 -- 172.21.15.27:0/39794118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb480f55c0 msgr2=0x7feb48105ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:37:14.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.233+0000 7feb467fc700 1 --2- 172.21.15.27:0/39794118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb480f55c0 0x7feb48105ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:14.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.233+0000 7feb467fc700 1 -- 172.21.15.27:0/39794118 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb30005040 con 0x7feb480f6860 2021-11-03T18:37:14.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.233+0000 7feb46ffd700 1 --2- 172.21.15.27:0/39794118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb480f55c0 0x7feb48105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T18:37:14.238 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.233+0000 7feb467fc700 1 --2- 172.21.15.27:0/39794118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb480f6860 0x7feb481063b0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 rx=0x7feb30009990 tx=0x7feb30009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:37:14.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.233+0000 7feb3f7fe700 1 -- 172.21.15.27:0/39794118 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb3001a070 con 0x7feb480f6860 2021-11-03T18:37:14.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.233+0000 7feb3f7fe700 1 -- 172.21.15.27:0/39794118 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7feb30012a80 con 0x7feb480f6860 2021-11-03T18:37:14.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.233+0000 7feb4d934700 1 -- 172.21.15.27:0/39794118 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb481073e0 con 0x7feb480f6860 2021-11-03T18:37:14.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.233+0000 7feb3f7fe700 1 -- 172.21.15.27:0/39794118 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb3001f9a0 con 0x7feb480f6860 2021-11-03T18:37:14.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.233+0000 7feb4d934700 1 -- 172.21.15.27:0/39794118 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb48107940 con 0x7feb480f6860 2021-11-03T18:37:14.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.235+0000 7feb3f7fe700 1 -- 172.21.15.27:0/39794118 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7feb30012bf0 con 0x7feb480f6860 2021-11-03T18:37:14.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.235+0000 7feb4d934700 1 -- 172.21.15.27:0/39794118 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb480f49b0 con 0x7feb480f6860 2021-11-03T18:37:14.242 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.235+0000 7feb3f7fe700 1 --2- 172.21.15.27:0/39794118 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7feb340640b0 0x7feb34066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:37:14.242 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.235+0000 7feb3f7fe700 1 -- 172.21.15.27:0/39794118 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7feb30084990 con 0x7feb480f6860 2021-11-03T18:37:14.242 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.235+0000 7feb46ffd700 1 --2- 172.21.15.27:0/39794118 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7feb340640b0 0x7feb34066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:37:14.243 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.236+0000 7feb46ffd700 1 --2- 172.21.15.27:0/39794118 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7feb340640b0 0x7feb34066570 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7feb48105c30 tx=0x7feb40009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:37:14.243 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.239+0000 7feb3f7fe700 1 -- 172.21.15.27:0/39794118 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7feb30056cf0 con 0x7feb480f6860 2021-11-03T18:37:14.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.443+0000 7feb4d934700 1 -- 172.21.15.27:0/39794118 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7feb480ff790 con 0x7feb480f6860 2021-11-03T18:37:14.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.443+0000 7feb3f7fe700 1 -- 172.21.15.27:0/39794118 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7feb300568e0 con 0x7feb480f6860 2021-11-03T18:37:14.443 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:37:14.443 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:37:14.443 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:37:14.444 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:37:14.444 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:37:14.444 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:37:14.444 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:37:14.444 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:37:14.445 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:37:14.445 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:37:14.445 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:37:14.445 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:37:14.446 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:37:14.446 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:37:14.446 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:37:14.446 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:37:14.446 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:37:14.447 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:37:14.447 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:37:14.447 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:37:14.447 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:37:14.448 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:37:14.448 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:37:14.448 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:37:14.448 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:37:14.448 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:37:14.449 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:37:14.449 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:37:14.449 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:37:14.449 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:37:14.450 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:37:14.450 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:37:14.450 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:37:14.450 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:37:14.450 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:37:14.451 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:37:14.451 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:37:14.451 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:37:14.451 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:37:14.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.445+0000 7feb4d934700 1 -- 172.21.15.27:0/39794118 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7feb340640b0 msgr2=0x7feb34066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:37:14.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.445+0000 7feb4d934700 1 --2- 172.21.15.27:0/39794118 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7feb340640b0 0x7feb34066570 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7feb48105c30 tx=0x7feb40009040).stop 2021-11-03T18:37:14.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.445+0000 7feb4d934700 1 -- 172.21.15.27:0/39794118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb480f6860 msgr2=0x7feb481063b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:37:14.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.445+0000 7feb4d934700 1 --2- 172.21.15.27:0/39794118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb480f6860 0x7feb481063b0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 rx=0x7feb30009990 tx=0x7feb30009b30).stop 2021-11-03T18:37:14.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.445+0000 7feb4d934700 1 -- 172.21.15.27:0/39794118 shutdown_connections 2021-11-03T18:37:14.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.445+0000 7feb4d934700 1 --2- 172.21.15.27:0/39794118 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7feb340640b0 0x7feb34066570 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:14.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.445+0000 7feb4d934700 1 --2- 172.21.15.27:0/39794118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feb480f55c0 0x7feb48105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:14.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.445+0000 7feb4d934700 1 --2- 172.21.15.27:0/39794118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feb480f6860 0x7feb481063b0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:14.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.445+0000 7feb4d934700 1 -- 172.21.15.27:0/39794118 >> 172.21.15.27:0/39794118 conn(0x7feb480f0d40 msgr2=0x7feb480fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:37:14.454 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.445+0000 7feb4d934700 1 -- 172.21.15.27:0/39794118 shutdown_connections 2021-11-03T18:37:14.455 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:14.445+0000 7feb4d934700 1 -- 172.21.15.27:0/39794118 wait complete. 2021-11-03T18:37:14.455 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:37:14.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:14 smithi115 ceph-mon[65453]: pgmap v2479: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:15.084 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:14 smithi115 ceph-mon[65453]: from='client.36648 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:37:15.085 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:14 smithi115 ceph-mon[65453]: from='client.36652 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:37:15.086 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:14 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/953336957' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:15.086 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:14 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/39794118' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:37:15.087 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:14 smithi027 ceph-mon[97244]: pgmap v2479: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:15.087 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:14 smithi027 ceph-mon[97244]: from='client.36648 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:37:15.087 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:14 smithi027 ceph-mon[97244]: from='client.36652 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:37:15.088 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:14 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/953336957' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:15.088 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:14 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/39794118' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:37:15.991 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4877.476981 secs 2021-11-03T18:37:16.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4877.476981 secs 2021-11-03T18:37:16.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:16 smithi115 ceph-mon[65453]: pgmap v2480: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:17.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:16 smithi027 ceph-mon[97244]: pgmap v2480: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:18.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:18 smithi115 ceph-mon[65453]: pgmap v2481: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:18.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:37:18.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:37:18.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:37:18.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:18.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:18.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:18.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:19.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:18 smithi027 ceph-mon[97244]: pgmap v2481: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:19.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:37:19.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:37:19.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:37:19.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:19.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:19.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:19.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:19.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:19 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:37:20.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:19 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:37:20.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:20 smithi115 ceph-mon[65453]: pgmap v2482: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:20.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4882.477121 secs 2021-11-03T18:37:21.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:20 smithi027 ceph-mon[97244]: pgmap v2482: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:21.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4882.477121 secs 2021-11-03T18:37:22.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:22 smithi027 ceph-mon[97244]: pgmap v2483: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:22.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:22 smithi115 ceph-mon[65453]: pgmap v2483: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:24.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:24 smithi115 ceph-mon[65453]: pgmap v2484: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:25.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:24 smithi027 ceph-mon[97244]: pgmap v2484: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:25.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4887.477266 secs 2021-11-03T18:37:26.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4887.477266 secs 2021-11-03T18:37:26.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:26 smithi115 ceph-mon[65453]: pgmap v2485: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:27.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:26 smithi027 ceph-mon[97244]: pgmap v2485: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:28.707 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:28 smithi027 ceph-mon[97244]: pgmap v2486: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:28.708 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:37:28.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:28 smithi115 ceph-mon[65453]: pgmap v2486: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:28.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:37:29.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:29 smithi115 ceph-mon[65453]: poll took 50150.715 ms: 1 events 2021-11-03T18:37:29.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:37:29.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:37:29.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:37:29.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:29.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:29.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:29.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:29.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:29 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:37:30.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:29 smithi027 ceph-mon[97244]: poll took 50150.715 ms: 1 events 2021-11-03T18:37:30.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:37:30.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:37:30.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:37:30.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:30.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:30.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:30.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:30.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:29 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:37:30.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:30 smithi115 ceph-mon[65453]: pgmap v2487: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:30.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4892.477408 secs 2021-11-03T18:37:31.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:30 smithi027 ceph-mon[97244]: pgmap v2487: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:31.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4892.477408 secs 2021-11-03T18:37:32.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:32 smithi115 ceph-mon[65453]: pgmap v2488: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:33.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:32 smithi027 ceph-mon[97244]: pgmap v2488: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:34.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:34 smithi115 ceph-mon[65453]: pgmap v2489: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:35.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:34 smithi027 ceph-mon[97244]: pgmap v2489: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:35.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4897.477570 secs 2021-11-03T18:37:36.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4897.477570 secs 2021-11-03T18:37:36.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:36 smithi115 ceph-mon[65453]: pgmap v2490: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:37.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:36 smithi027 ceph-mon[97244]: pgmap v2490: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:39.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:38 smithi027 ceph-mon[97244]: pgmap v2491: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:39.097 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:38 smithi115 ceph-mon[65453]: pgmap v2491: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:40.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:37:40.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:39 smithi027 ceph-mon[97244]: poll took 10033.365 ms: 1 events 2021-11-03T18:37:40.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:37:40.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:37:40.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:40.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:40.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:40.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:40.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:37:40.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:39 smithi115 ceph-mon[65453]: poll took 10033.365 ms: 1 events 2021-11-03T18:37:40.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:37:40.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:37:40.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:40.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:40.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:40.241 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:41.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:40 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:37:41.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:40 smithi027 ceph-mon[97244]: pgmap v2492: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:41.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4902.477677 secs 2021-11-03T18:37:41.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:40 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:37:41.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:40 smithi115 ceph-mon[65453]: pgmap v2492: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:41.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4902.477677 secs 2021-11-03T18:37:43.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:42 smithi027 ceph-mon[97244]: pgmap v2493: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:43.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:42 smithi115 ceph-mon[65453]: pgmap v2493: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:44.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.562+0000 7fec43017700 1 -- 172.21.15.27:0/2378918776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec3c0f6530 msgr2=0x7fec3c0f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:37:44.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.562+0000 7fec43017700 1 --2- 172.21.15.27:0/2378918776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec3c0f6530 0x7fec3c0f6950 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 rx=0x7fec38004660 tx=0x7fec38009b30).stop 2021-11-03T18:37:44.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.562+0000 7fec43017700 1 -- 172.21.15.27:0/2378918776 shutdown_connections 2021-11-03T18:37:44.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.562+0000 7fec43017700 1 --2- 172.21.15.27:0/2378918776 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec3c0f77d0 0x7fec3c0f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:44.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.562+0000 7fec43017700 1 --2- 172.21.15.27:0/2378918776 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec3c0f6530 0x7fec3c0f6950 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:44.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.562+0000 7fec43017700 1 -- 172.21.15.27:0/2378918776 >> 172.21.15.27:0/2378918776 conn(0x7fec3c0f1cd0 msgr2=0x7fec3c0f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:37:44.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.562+0000 7fec43017700 1 -- 172.21.15.27:0/2378918776 shutdown_connections 2021-11-03T18:37:44.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.562+0000 7fec43017700 1 -- 172.21.15.27:0/2378918776 wait complete. 2021-11-03T18:37:44.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.562+0000 7fec43017700 1 Processor -- start 2021-11-03T18:37:44.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.562+0000 7fec43017700 1 -- start start 2021-11-03T18:37:44.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.563+0000 7fec43017700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec3c0f6530 0x7fec3c106db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:37:44.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.563+0000 7fec43017700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec3c0f77d0 0x7fec3c1072a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:37:44.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.563+0000 7fec43017700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c1078a0 con 0x7fec3c0f77d0 2021-11-03T18:37:44.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.563+0000 7fec43017700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec3c1080e0 con 0x7fec3c0f6530 2021-11-03T18:37:44.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.563+0000 7fec41814700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec3c0f77d0 0x7fec3c1072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:37:44.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.563+0000 7fec41814700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec3c0f77d0 0x7fec3c1072a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35926/0 (socket says 172.21.15.27:35926) 2021-11-03T18:37:44.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.563+0000 7fec41814700 1 -- 172.21.15.27:0/2036841245 learned_addr learned my addr 172.21.15.27:0/2036841245 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:37:44.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.563+0000 7fec42015700 1 --2- 172.21.15.27:0/2036841245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec3c0f6530 0x7fec3c106db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:37:44.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.563+0000 7fec41814700 1 -- 172.21.15.27:0/2036841245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec3c0f6530 msgr2=0x7fec3c106db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:37:44.568 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.563+0000 7fec41814700 1 --2- 172.21.15.27:0/2036841245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec3c0f6530 0x7fec3c106db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:44.568 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.563+0000 7fec41814700 1 -- 172.21.15.27:0/2036841245 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec38005040 con 0x7fec3c0f77d0 2021-11-03T18:37:44.568 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.563+0000 7fec41814700 1 --2- 172.21.15.27:0/2036841245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec3c0f77d0 0x7fec3c1072a0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 rx=0x7fec2c00c530 tx=0x7fec2c007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:37:44.568 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.563+0000 7fec32ffd700 1 -- 172.21.15.27:0/2036841245 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec2c00f650 con 0x7fec3c0f77d0 2021-11-03T18:37:44.569 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.564+0000 7fec32ffd700 1 -- 172.21.15.27:0/2036841245 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fec2c00fc60 con 0x7fec3c0f77d0 2021-11-03T18:37:44.569 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.564+0000 7fec43017700 1 -- 172.21.15.27:0/2036841245 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec3c1083c0 con 0x7fec3c0f77d0 2021-11-03T18:37:44.569 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.564+0000 7fec32ffd700 1 -- 172.21.15.27:0/2036841245 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec2c012cd0 con 0x7fec3c0f77d0 2021-11-03T18:37:44.569 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.564+0000 7fec43017700 1 -- 172.21.15.27:0/2036841245 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec3c108900 con 0x7fec3c0f77d0 2021-11-03T18:37:44.570 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.565+0000 7fec43017700 1 -- 172.21.15.27:0/2036841245 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec3c059100 con 0x7fec3c0f77d0 2021-11-03T18:37:44.571 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.565+0000 7fec32ffd700 1 -- 172.21.15.27:0/2036841245 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fec2c010070 con 0x7fec3c0f77d0 2021-11-03T18:37:44.571 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.565+0000 7fec32ffd700 1 --2- 172.21.15.27:0/2036841245 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fec28064100 0x7fec280665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:37:44.571 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.565+0000 7fec32ffd700 1 -- 172.21.15.27:0/2036841245 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fec2c080640 con 0x7fec3c0f77d0 2021-11-03T18:37:44.571 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.565+0000 7fec42015700 1 --2- 172.21.15.27:0/2036841245 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fec28064100 0x7fec280665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:37:44.572 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.566+0000 7fec42015700 1 --2- 172.21.15.27:0/2036841245 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fec28064100 0x7fec280665c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7fec38004e30 tx=0x7fec38009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:37:44.572 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.570+0000 7fec32ffd700 1 -- 172.21.15.27:0/2036841245 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fec2c0528b0 con 0x7fec3c0f77d0 2021-11-03T18:37:44.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.751+0000 7fec43017700 1 -- 172.21.15.27:0/2036841245 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fec3c0fee80 con 0x7fec28064100 2021-11-03T18:37:44.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.752+0000 7fec32ffd700 1 -- 172.21.15.27:0/2036841245 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fec3c0fee80 con 0x7fec28064100 2021-11-03T18:37:44.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.754+0000 7fec43017700 1 -- 172.21.15.27:0/2036841245 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fec28064100 msgr2=0x7fec280665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:37:44.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.754+0000 7fec43017700 1 --2- 172.21.15.27:0/2036841245 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fec28064100 0x7fec280665c0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 rx=0x7fec38004e30 tx=0x7fec38009b30).stop 2021-11-03T18:37:44.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.754+0000 7fec43017700 1 -- 172.21.15.27:0/2036841245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec3c0f77d0 msgr2=0x7fec3c1072a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:37:44.754 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.754+0000 7fec43017700 1 --2- 172.21.15.27:0/2036841245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec3c0f77d0 0x7fec3c1072a0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 rx=0x7fec2c00c530 tx=0x7fec2c007e20).stop 2021-11-03T18:37:44.754 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.754+0000 7fec43017700 1 -- 172.21.15.27:0/2036841245 shutdown_connections 2021-11-03T18:37:44.754 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.754+0000 7fec43017700 1 --2- 172.21.15.27:0/2036841245 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fec28064100 0x7fec280665c0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:44.754 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.754+0000 7fec43017700 1 --2- 172.21.15.27:0/2036841245 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec3c0f6530 0x7fec3c106db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:44.755 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.754+0000 7fec43017700 1 --2- 172.21.15.27:0/2036841245 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec3c0f77d0 0x7fec3c1072a0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:44.755 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.754+0000 7fec43017700 1 -- 172.21.15.27:0/2036841245 >> 172.21.15.27:0/2036841245 conn(0x7fec3c0f1cd0 msgr2=0x7fec3c0fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:37:44.755 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.754+0000 7fec43017700 1 -- 172.21.15.27:0/2036841245 shutdown_connections 2021-11-03T18:37:44.755 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.754+0000 7fec43017700 1 -- 172.21.15.27:0/2036841245 wait complete. 2021-11-03T18:37:44.762 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:37:44.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.854+0000 7f924b0a8700 1 -- 172.21.15.27:0/1099350791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92440f66e0 msgr2=0x7f92440f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:37:44.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.854+0000 7f924b0a8700 1 --2- 172.21.15.27:0/1099350791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92440f66e0 0x7f92440f6b00 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 rx=0x7f923c004660 tx=0x7f923c009b30).stop 2021-11-03T18:37:44.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.854+0000 7f924b0a8700 1 -- 172.21.15.27:0/1099350791 shutdown_connections 2021-11-03T18:37:44.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.854+0000 7f924b0a8700 1 --2- 172.21.15.27:0/1099350791 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92440f6ff0 0x7f92440f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:44.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.854+0000 7f924b0a8700 1 --2- 172.21.15.27:0/1099350791 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92440f66e0 0x7f92440f6b00 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:44.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.854+0000 7f924b0a8700 1 -- 172.21.15.27:0/1099350791 >> 172.21.15.27:0/1099350791 conn(0x7f92440f1f00 msgr2=0x7f92440f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:37:44.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.854+0000 7f924b0a8700 1 -- 172.21.15.27:0/1099350791 shutdown_connections 2021-11-03T18:37:44.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.854+0000 7f924b0a8700 1 -- 172.21.15.27:0/1099350791 wait complete. 2021-11-03T18:37:44.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.854+0000 7f924b0a8700 1 Processor -- start 2021-11-03T18:37:44.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.854+0000 7f924b0a8700 1 -- start start 2021-11-03T18:37:44.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.855+0000 7f924b0a8700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92440f66e0 0x7f9244106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:37:44.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.855+0000 7f924b0a8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92440f6ff0 0x7f9244107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:37:44.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.855+0000 7f924b0a8700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9244107a00 con 0x7f92440f6ff0 2021-11-03T18:37:44.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.855+0000 7f924b0a8700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9244107b40 con 0x7f92440f66e0 2021-11-03T18:37:44.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.855+0000 7f9243fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92440f6ff0 0x7f9244107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:37:44.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.855+0000 7f9243fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92440f6ff0 0x7f9244107450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35936/0 (socket says 172.21.15.27:35936) 2021-11-03T18:37:44.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.855+0000 7f9243fff700 1 -- 172.21.15.27:0/2421736630 learned_addr learned my addr 172.21.15.27:0/2421736630 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:37:44.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.855+0000 7f9248e44700 1 --2- 172.21.15.27:0/2421736630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92440f66e0 0x7f9244106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:37:44.860 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.855+0000 7f9243fff700 1 -- 172.21.15.27:0/2421736630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92440f66e0 msgr2=0x7f9244106f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:37:44.860 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.855+0000 7f9243fff700 1 --2- 172.21.15.27:0/2421736630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92440f66e0 0x7f9244106f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:44.860 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.855+0000 7f9243fff700 1 -- 172.21.15.27:0/2421736630 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f923c005040 con 0x7f92440f6ff0 2021-11-03T18:37:44.860 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.855+0000 7f9243fff700 1 --2- 172.21.15.27:0/2421736630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92440f6ff0 0x7f9244107450 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 rx=0x7f9234007fd0 tx=0x7f9234004ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:37:44.861 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.855+0000 7f92417fa700 1 -- 172.21.15.27:0/2421736630 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f923400b4f0 con 0x7f92440f6ff0 2021-11-03T18:37:44.861 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.855+0000 7f924b0a8700 1 -- 172.21.15.27:0/2421736630 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92441084e0 con 0x7f92440f6ff0 2021-11-03T18:37:44.861 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.856+0000 7f92417fa700 1 -- 172.21.15.27:0/2421736630 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f923400f040 con 0x7f92440f6ff0 2021-11-03T18:37:44.861 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.856+0000 7f92417fa700 1 -- 172.21.15.27:0/2421736630 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9234013a50 con 0x7f92440f6ff0 2021-11-03T18:37:44.862 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.856+0000 7f924b0a8700 1 -- 172.21.15.27:0/2421736630 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9244108aa0 con 0x7f92440f6ff0 2021-11-03T18:37:44.862 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.856+0000 7f924b0a8700 1 -- 172.21.15.27:0/2421736630 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9244045ad0 con 0x7f92440f6ff0 2021-11-03T18:37:44.862 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.857+0000 7f92417fa700 1 -- 172.21.15.27:0/2421736630 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f923401a030 con 0x7f92440f6ff0 2021-11-03T18:37:44.862 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.857+0000 7f92417fa700 1 --2- 172.21.15.27:0/2421736630 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f92300640b0 0x7f9230066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:37:44.862 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.857+0000 7f92417fa700 1 -- 172.21.15.27:0/2421736630 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f9234080710 con 0x7f92440f6ff0 2021-11-03T18:37:44.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.857+0000 7f9248e44700 1 --2- 172.21.15.27:0/2421736630 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f92300640b0 0x7f9230066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:37:44.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.858+0000 7f9248e44700 1 --2- 172.21.15.27:0/2421736630 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f92300640b0 0x7f9230066570 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 rx=0x7f923c004630 tx=0x7f923c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:37:44.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:44.861+0000 7f92417fa700 1 -- 172.21.15.27:0/2421736630 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f9234052af0 con 0x7f92440f6ff0 2021-11-03T18:37:45.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:44 smithi027 ceph-mon[97244]: pgmap v2494: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:45.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.034+0000 7f924b0a8700 1 -- 172.21.15.27:0/2421736630 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f92440ff040 con 0x7f92300640b0 2021-11-03T18:37:45.048 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.049+0000 7f92417fa700 1 -- 172.21.15.27:0/2421736630 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f92440ff040 con 0x7f92300640b0 2021-11-03T18:37:45.048 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:37:45.049 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (89m) 8m ago 90m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:37:45.049 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (81m) 8m ago 90m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:37:45.049 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (80m) 7m ago 89m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:37:45.049 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (89m) 8m ago 90m 42.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:37:45.050 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (82m) 8m ago 85m 26.7M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:37:45.051 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (85m) 8m ago 85m 3850M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:37:45.051 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (81m) 7m ago 85m 26.3M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:37:45.051 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (83m) 7m ago 85m 27.4M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:37:45.051 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (83m) 8m ago 91m 608M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:37:45.052 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (82m) 7m ago 89m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:37:45.052 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (82m) 8m ago 91m 350M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:37:45.052 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (81m) 7m ago 89m 160M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:37:45.052 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (90m) 8m ago 90m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:37:45.053 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (89m) 7m ago 89m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:37:45.053 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (80m) 8m ago 88m 413M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:37:45.053 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (80m) 8m ago 88m 450M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:37:45.053 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (79m) 8m ago 88m 338M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:37:45.053 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (79m) 7m ago 87m 479M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:37:45.054 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (78m) 7m ago 87m 378M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:37:45.054 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (78m) 7m ago 87m 388M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:37:45.054 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (89m) 8m ago 89m 64.4M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:37:45.055 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.051+0000 7f924b0a8700 1 -- 172.21.15.27:0/2421736630 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f92300640b0 msgr2=0x7f9230066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:37:45.055 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.051+0000 7f924b0a8700 1 --2- 172.21.15.27:0/2421736630 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f92300640b0 0x7f9230066570 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 rx=0x7f923c004630 tx=0x7f923c009b30).stop 2021-11-03T18:37:45.056 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.051+0000 7f924b0a8700 1 -- 172.21.15.27:0/2421736630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92440f6ff0 msgr2=0x7f9244107450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:37:45.056 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.051+0000 7f924b0a8700 1 --2- 172.21.15.27:0/2421736630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92440f6ff0 0x7f9244107450 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 rx=0x7f9234007fd0 tx=0x7f9234004ee0).stop 2021-11-03T18:37:45.056 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.051+0000 7f924b0a8700 1 -- 172.21.15.27:0/2421736630 shutdown_connections 2021-11-03T18:37:45.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.051+0000 7f924b0a8700 1 --2- 172.21.15.27:0/2421736630 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f92300640b0 0x7f9230066570 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:45.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.051+0000 7f924b0a8700 1 --2- 172.21.15.27:0/2421736630 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92440f66e0 0x7f9244106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:45.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.051+0000 7f924b0a8700 1 --2- 172.21.15.27:0/2421736630 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92440f6ff0 0x7f9244107450 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:45.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.051+0000 7f924b0a8700 1 -- 172.21.15.27:0/2421736630 >> 172.21.15.27:0/2421736630 conn(0x7f92440f1f00 msgr2=0x7f92440fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:37:45.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.051+0000 7f924b0a8700 1 -- 172.21.15.27:0/2421736630 shutdown_connections 2021-11-03T18:37:45.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.051+0000 7f924b0a8700 1 -- 172.21.15.27:0/2421736630 wait complete. 2021-11-03T18:37:45.149 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.150+0000 7fa1aad1b700 1 -- 172.21.15.27:0/2654158489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1a40f4d60 msgr2=0x7fa1a40f5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:37:45.150 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.150+0000 7fa1aad1b700 1 --2- 172.21.15.27:0/2654158489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1a40f4d60 0x7fa1a40f5180 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 rx=0x7fa19c004770 tx=0x7fa19c00ac40).stop 2021-11-03T18:37:45.150 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.150+0000 7fa1aad1b700 1 -- 172.21.15.27:0/2654158489 shutdown_connections 2021-11-03T18:37:45.150 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.150+0000 7fa1aad1b700 1 --2- 172.21.15.27:0/2654158489 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1a40f5f60 0x7fa1a40f63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:45.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.150+0000 7fa1aad1b700 1 --2- 172.21.15.27:0/2654158489 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1a40f4d60 0x7fa1a40f5180 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:45.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.150+0000 7fa1aad1b700 1 -- 172.21.15.27:0/2654158489 >> 172.21.15.27:0/2654158489 conn(0x7fa1a40f04c0 msgr2=0x7fa1a40f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:37:45.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.150+0000 7fa1aad1b700 1 -- 172.21.15.27:0/2654158489 shutdown_connections 2021-11-03T18:37:45.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.150+0000 7fa1aad1b700 1 -- 172.21.15.27:0/2654158489 wait complete. 2021-11-03T18:37:45.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.151+0000 7fa1aad1b700 1 Processor -- start 2021-11-03T18:37:45.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.151+0000 7fa1aad1b700 1 -- start start 2021-11-03T18:37:45.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.151+0000 7fa1aad1b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1a40f4d60 0x7fa1a4106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:37:45.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.151+0000 7fa1aad1b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1a40f5f60 0x7fa1a4107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:37:45.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.151+0000 7fa1aad1b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a4107a00 con 0x7fa1a40f5f60 2021-11-03T18:37:45.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.151+0000 7fa1aad1b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a4107b40 con 0x7fa1a40f4d60 2021-11-03T18:37:45.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.151+0000 7fa1a3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1a40f5f60 0x7fa1a4107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:37:45.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.151+0000 7fa1a3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1a40f5f60 0x7fa1a4107450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35946/0 (socket says 172.21.15.27:35946) 2021-11-03T18:37:45.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.151+0000 7fa1a3fff700 1 -- 172.21.15.27:0/1559753691 learned_addr learned my addr 172.21.15.27:0/1559753691 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:37:45.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.151+0000 7fa1a8ab7700 1 --2- 172.21.15.27:0/1559753691 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1a40f4d60 0x7fa1a4106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:37:45.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.151+0000 7fa1a3fff700 1 -- 172.21.15.27:0/1559753691 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1a40f4d60 msgr2=0x7fa1a4106f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:37:45.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.151+0000 7fa1a3fff700 1 --2- 172.21.15.27:0/1559753691 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1a40f4d60 0x7fa1a4106f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:45.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.151+0000 7fa1a3fff700 1 -- 172.21.15.27:0/1559753691 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa19c005040 con 0x7fa1a40f5f60 2021-11-03T18:37:45.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.152+0000 7fa1a3fff700 1 --2- 172.21.15.27:0/1559753691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1a40f5f60 0x7fa1a4107450 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 rx=0x7fa194007fd0 tx=0x7fa194004ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:37:45.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.152+0000 7fa1a17fa700 1 -- 172.21.15.27:0/1559753691 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa19400b4f0 con 0x7fa1a40f5f60 2021-11-03T18:37:45.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.152+0000 7fa1a17fa700 1 -- 172.21.15.27:0/1559753691 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fa19400f040 con 0x7fa1a40f5f60 2021-11-03T18:37:45.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.152+0000 7fa1aad1b700 1 -- 172.21.15.27:0/1559753691 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1a41084e0 con 0x7fa1a40f5f60 2021-11-03T18:37:45.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.152+0000 7fa1a17fa700 1 -- 172.21.15.27:0/1559753691 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa194013a50 con 0x7fa1a40f5f60 2021-11-03T18:37:45.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.152+0000 7fa1aad1b700 1 -- 172.21.15.27:0/1559753691 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1a4108aa0 con 0x7fa1a40f5f60 2021-11-03T18:37:45.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.153+0000 7fa1aad1b700 1 -- 172.21.15.27:0/1559753691 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1a4045ad0 con 0x7fa1a40f5f60 2021-11-03T18:37:45.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.153+0000 7fa1a17fa700 1 -- 172.21.15.27:0/1559753691 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fa19401a030 con 0x7fa1a40f5f60 2021-11-03T18:37:45.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.154+0000 7fa1a17fa700 1 --2- 172.21.15.27:0/1559753691 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa190064090 0x7fa190066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:37:45.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.154+0000 7fa1a17fa700 1 -- 172.21.15.27:0/1559753691 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fa194080710 con 0x7fa1a40f5f60 2021-11-03T18:37:45.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.154+0000 7fa1a8ab7700 1 --2- 172.21.15.27:0/1559753691 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa190064090 0x7fa190066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:37:45.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.154+0000 7fa1a8ab7700 1 --2- 172.21.15.27:0/1559753691 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa190064090 0x7fa190066550 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7fa19c014450 tx=0x7fa19c016040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:37:45.160 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.158+0000 7fa1a17fa700 1 -- 172.21.15.27:0/1559753691 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fa194052af0 con 0x7fa1a40f5f60 2021-11-03T18:37:45.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:44 smithi115 ceph-mon[65453]: pgmap v2494: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:45.385 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.386+0000 7fa1aad1b700 1 -- 172.21.15.27:0/1559753691 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa1a4027a00 con 0x7fa1a40f5f60 2021-11-03T18:37:45.386 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.387+0000 7fa1a17fa700 1 -- 172.21.15.27:0/1559753691 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fa194018070 con 0x7fa1a40f5f60 2021-11-03T18:37:45.387 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:37:45.387 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:37:45.387 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:37:45.387 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:37:45.388 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:37:45.388 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:37:45.388 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:37:45.388 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:37:45.388 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:37:45.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:37:45.389 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:37:45.389 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:37:45.389 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:37:45.389 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:37:45.390 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:37:45.390 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:37:45.390 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:37:45.390 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:37:45.391 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.389+0000 7fa1aad1b700 1 -- 172.21.15.27:0/1559753691 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa190064090 msgr2=0x7fa190066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:37:45.392 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.389+0000 7fa1aad1b700 1 --2- 172.21.15.27:0/1559753691 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa190064090 0x7fa190066550 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7fa19c014450 tx=0x7fa19c016040).stop 2021-11-03T18:37:45.392 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.389+0000 7fa1aad1b700 1 -- 172.21.15.27:0/1559753691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1a40f5f60 msgr2=0x7fa1a4107450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:37:45.392 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.389+0000 7fa1aad1b700 1 --2- 172.21.15.27:0/1559753691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1a40f5f60 0x7fa1a4107450 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 rx=0x7fa194007fd0 tx=0x7fa194004ee0).stop 2021-11-03T18:37:45.392 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.389+0000 7fa1aad1b700 1 -- 172.21.15.27:0/1559753691 shutdown_connections 2021-11-03T18:37:45.392 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.389+0000 7fa1aad1b700 1 --2- 172.21.15.27:0/1559753691 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa190064090 0x7fa190066550 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:45.393 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.389+0000 7fa1aad1b700 1 --2- 172.21.15.27:0/1559753691 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa1a40f4d60 0x7fa1a4106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:45.393 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.389+0000 7fa1aad1b700 1 --2- 172.21.15.27:0/1559753691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa1a40f5f60 0x7fa1a4107450 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:45.393 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.389+0000 7fa1aad1b700 1 -- 172.21.15.27:0/1559753691 >> 172.21.15.27:0/1559753691 conn(0x7fa1a40f04c0 msgr2=0x7fa1a40f9a40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:37:45.393 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.389+0000 7fa1aad1b700 1 -- 172.21.15.27:0/1559753691 shutdown_connections 2021-11-03T18:37:45.394 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.389+0000 7fa1aad1b700 1 -- 172.21.15.27:0/1559753691 wait complete. 2021-11-03T18:37:45.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.489+0000 7efe60b04700 1 -- 172.21.15.27:0/2672516463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe5c0f66e0 msgr2=0x7efe5c0f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:37:45.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.489+0000 7efe60b04700 1 --2- 172.21.15.27:0/2672516463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe5c0f66e0 0x7efe5c0f6b00 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 rx=0x7efe540099e0 tx=0x7efe54004d10).stop 2021-11-03T18:37:45.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.489+0000 7efe60b04700 1 -- 172.21.15.27:0/2672516463 shutdown_connections 2021-11-03T18:37:45.490 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.489+0000 7efe60b04700 1 --2- 172.21.15.27:0/2672516463 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe5c0f6ff0 0x7efe5c0f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:45.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.489+0000 7efe60b04700 1 --2- 172.21.15.27:0/2672516463 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe5c0f66e0 0x7efe5c0f6b00 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:45.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.489+0000 7efe60b04700 1 -- 172.21.15.27:0/2672516463 >> 172.21.15.27:0/2672516463 conn(0x7efe5c0f1f00 msgr2=0x7efe5c0f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:37:45.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.489+0000 7efe60b04700 1 -- 172.21.15.27:0/2672516463 shutdown_connections 2021-11-03T18:37:45.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.489+0000 7efe60b04700 1 -- 172.21.15.27:0/2672516463 wait complete. 2021-11-03T18:37:45.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.489+0000 7efe60b04700 1 Processor -- start 2021-11-03T18:37:45.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.489+0000 7efe60b04700 1 -- start start 2021-11-03T18:37:45.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.489+0000 7efe60b04700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe5c0f66e0 0x7efe5c107070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:37:45.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.489+0000 7efe60b04700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe5c0f6ff0 0x7efe5c107560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:37:45.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.489+0000 7efe60b04700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5c107b10 con 0x7efe5c0f66e0 2021-11-03T18:37:45.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.489+0000 7efe60b04700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5c107c50 con 0x7efe5c0f6ff0 2021-11-03T18:37:45.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.490+0000 7efe5a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe5c0f66e0 0x7efe5c107070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:37:45.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.490+0000 7efe5a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe5c0f66e0 0x7efe5c107070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35954/0 (socket says 172.21.15.27:35954) 2021-11-03T18:37:45.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.490+0000 7efe5a59c700 1 -- 172.21.15.27:0/4165137315 learned_addr learned my addr 172.21.15.27:0/4165137315 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:37:45.494 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.490+0000 7efe59d9b700 1 --2- 172.21.15.27:0/4165137315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe5c0f6ff0 0x7efe5c107560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:37:45.494 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.490+0000 7efe5a59c700 1 -- 172.21.15.27:0/4165137315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe5c0f6ff0 msgr2=0x7efe5c107560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:37:45.494 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.490+0000 7efe5a59c700 1 --2- 172.21.15.27:0/4165137315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe5c0f6ff0 0x7efe5c107560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:45.494 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.490+0000 7efe5a59c700 1 -- 172.21.15.27:0/4165137315 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe54009640 con 0x7efe5c0f66e0 2021-11-03T18:37:45.494 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.490+0000 7efe59d9b700 1 --2- 172.21.15.27:0/4165137315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe5c0f6ff0 0x7efe5c107560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:37:45.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.490+0000 7efe5a59c700 1 --2- 172.21.15.27:0/4165137315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe5c0f66e0 0x7efe5c107070 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 rx=0x7efe5400b3b0 tx=0x7efe54004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:37:45.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.490+0000 7efe4affd700 1 -- 172.21.15.27:0/4165137315 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe5400e030 con 0x7efe5c0f66e0 2021-11-03T18:37:45.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.490+0000 7efe4affd700 1 -- 172.21.15.27:0/4165137315 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7efe540049b0 con 0x7efe5c0f66e0 2021-11-03T18:37:45.495 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.490+0000 7efe4affd700 1 -- 172.21.15.27:0/4165137315 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe54018070 con 0x7efe5c0f66e0 2021-11-03T18:37:45.496 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.490+0000 7efe60b04700 1 -- 172.21.15.27:0/4165137315 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe5c108590 con 0x7efe5c0f66e0 2021-11-03T18:37:45.496 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.490+0000 7efe60b04700 1 -- 172.21.15.27:0/4165137315 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe5c108a70 con 0x7efe5c0f66e0 2021-11-03T18:37:45.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.491+0000 7efe60b04700 1 -- 172.21.15.27:0/4165137315 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe5c045ad0 con 0x7efe5c0f66e0 2021-11-03T18:37:45.497 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.492+0000 7efe4affd700 1 -- 172.21.15.27:0/4165137315 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7efe54007d40 con 0x7efe5c0f66e0 2021-11-03T18:37:45.498 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.492+0000 7efe4affd700 1 --2- 172.21.15.27:0/4165137315 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7efe440640b0 0x7efe44066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:37:45.498 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.492+0000 7efe4affd700 1 -- 172.21.15.27:0/4165137315 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7efe54082540 con 0x7efe5c0f66e0 2021-11-03T18:37:45.498 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.492+0000 7efe59d9b700 1 --2- 172.21.15.27:0/4165137315 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7efe440640b0 0x7efe44066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:37:45.498 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.492+0000 7efe59d9b700 1 --2- 172.21.15.27:0/4165137315 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7efe440640b0 0x7efe44066570 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 rx=0x7efe50009740 tx=0x7efe50009be0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:37:45.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.496+0000 7efe4affd700 1 -- 172.21.15.27:0/4165137315 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7efe540548a0 con 0x7efe5c0f66e0 2021-11-03T18:37:45.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.699+0000 7efe60b04700 1 -- 172.21.15.27:0/4165137315 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7efe5c100e60 con 0x7efe5c0f66e0 2021-11-03T18:37:45.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.700+0000 7efe4affd700 1 -- 172.21.15.27:0/4165137315 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7efe54054490 con 0x7efe5c0f66e0 2021-11-03T18:37:45.700 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:37:45.700 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:37:45.700 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:37:45.701 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:37:45.701 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:37:45.701 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:37:45.702 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:37:45.702 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:37:45.702 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:37:45.702 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:37:45.703 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:37:45.703 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:37:45.703 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:37:45.703 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:37:45.704 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:37:45.704 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:37:45.704 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:37:45.704 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:37:45.705 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:37:45.705 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:37:45.705 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:37:45.705 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:37:45.706 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:37:45.706 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:37:45.706 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:37:45.707 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:37:45.707 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:37:45.707 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:37:45.708 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:37:45.708 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:37:45.709 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:37:45.709 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:37:45.710 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:37:45.710 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:37:45.710 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:37:45.710 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:37:45.711 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:37:45.711 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:37:45.711 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:37:45.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.702+0000 7efe60b04700 1 -- 172.21.15.27:0/4165137315 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7efe440640b0 msgr2=0x7efe44066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:37:45.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.702+0000 7efe60b04700 1 --2- 172.21.15.27:0/4165137315 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7efe440640b0 0x7efe44066570 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 rx=0x7efe50009740 tx=0x7efe50009be0).stop 2021-11-03T18:37:45.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.702+0000 7efe60b04700 1 -- 172.21.15.27:0/4165137315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe5c0f66e0 msgr2=0x7efe5c107070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:37:45.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.702+0000 7efe60b04700 1 --2- 172.21.15.27:0/4165137315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe5c0f66e0 0x7efe5c107070 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 rx=0x7efe5400b3b0 tx=0x7efe54004d10).stop 2021-11-03T18:37:45.713 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.702+0000 7efe60b04700 1 -- 172.21.15.27:0/4165137315 shutdown_connections 2021-11-03T18:37:45.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.702+0000 7efe60b04700 1 --2- 172.21.15.27:0/4165137315 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7efe440640b0 0x7efe44066570 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:45.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.702+0000 7efe60b04700 1 --2- 172.21.15.27:0/4165137315 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efe5c0f66e0 0x7efe5c107070 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:45.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.702+0000 7efe60b04700 1 --2- 172.21.15.27:0/4165137315 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efe5c0f6ff0 0x7efe5c107560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:37:45.714 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.702+0000 7efe60b04700 1 -- 172.21.15.27:0/4165137315 >> 172.21.15.27:0/4165137315 conn(0x7efe5c0f1f00 msgr2=0x7efe5c0fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:37:45.715 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.702+0000 7efe60b04700 1 -- 172.21.15.27:0/4165137315 shutdown_connections 2021-11-03T18:37:45.715 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:37:45.702+0000 7efe60b04700 1 -- 172.21.15.27:0/4165137315 wait complete. 2021-11-03T18:37:45.715 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:37:46.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4907.477755 secs 2021-11-03T18:37:46.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:45 smithi027 ceph-mon[97244]: from='client.36664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:37:46.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:45 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1559753691' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:46.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:45 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/4165137315' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:37:46.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4907.477755 secs 2021-11-03T18:37:46.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:45 smithi115 ceph-mon[65453]: from='client.36664 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:37:46.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:45 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1559753691' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:46.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:45 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/4165137315' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:37:47.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:46 smithi027 ceph-mon[97244]: from='client.36668 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:37:47.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:46 smithi027 ceph-mon[97244]: pgmap v2495: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:47.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:46 smithi115 ceph-mon[65453]: from='client.36668 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:37:47.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:46 smithi115 ceph-mon[65453]: pgmap v2495: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:49.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:48 smithi027 ceph-mon[97244]: pgmap v2496: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:49.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:48 smithi115 ceph-mon[65453]: pgmap v2496: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:50.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:37:50.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:37:50.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:37:50.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:50.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:50.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:50.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:50.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:37:50.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:37:50.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:37:50.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:50.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:50.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:50.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:37:51.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:50 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:37:51.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:50 smithi027 ceph-mon[97244]: pgmap v2497: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:51.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4912.477839 secs 2021-11-03T18:37:51.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:50 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:37:51.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:50 smithi115 ceph-mon[65453]: pgmap v2497: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:51.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4912.477839 secs 2021-11-03T18:37:53.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:52 smithi027 ceph-mon[97244]: pgmap v2498: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:53.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:52 smithi115 ceph-mon[65453]: pgmap v2498: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:55.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:54 smithi027 ceph-mon[97244]: pgmap v2499: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:55.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:54 smithi115 ceph-mon[65453]: pgmap v2499: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:56.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4917.477978 secs 2021-11-03T18:37:56.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4917.477978 secs 2021-11-03T18:37:57.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:56 smithi027 ceph-mon[97244]: pgmap v2500: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:57.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:37:57.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:37:57.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:56 smithi115 ceph-mon[65453]: pgmap v2500: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:57.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:37:57.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:37:59.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:58 smithi115 ceph-mon[65453]: pgmap v2501: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:37:59.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:58 smithi027 ceph-mon[97244]: pgmap v2501: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:00.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:38:00.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:38:00.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:38:00.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:00.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:00.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:00.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:37:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:00.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:38:00.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:38:00.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:38:00.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:00.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:00.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:00.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:37:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:01.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:00 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:38:01.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:00 smithi115 ceph-mon[65453]: pgmap v2502: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:01.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4922.478126 secs 2021-11-03T18:38:01.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:00 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:38:01.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:00 smithi027 ceph-mon[97244]: pgmap v2502: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:01.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4922.478126 secs 2021-11-03T18:38:03.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:02 smithi115 ceph-mon[65453]: pgmap v2503: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:03.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:02 smithi027 ceph-mon[97244]: pgmap v2503: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:05.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:04 smithi115 ceph-mon[65453]: pgmap v2504: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:05.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:04 smithi027 ceph-mon[97244]: pgmap v2504: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:06.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4927.478226 secs 2021-11-03T18:38:06.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4927.478226 secs 2021-11-03T18:38:07.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:06 smithi115 ceph-mon[65453]: pgmap v2505: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:07.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:06 smithi027 ceph-mon[97244]: pgmap v2505: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:09.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:08 smithi115 ceph-mon[65453]: pgmap v2506: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:09.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:08 smithi027 ceph-mon[97244]: pgmap v2506: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:10.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:38:10.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:38:10.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:38:10.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:10.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:10.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:10.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:10.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:38:10.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:38:10.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:38:10.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:10.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:10.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:10.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:11.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:10 smithi115 ceph-mon[65453]: pgmap v2507: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:11.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:10 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:38:11.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4932.478359 secs 2021-11-03T18:38:11.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:10 smithi027 ceph-mon[97244]: pgmap v2507: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:11.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:10 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:38:11.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4932.478359 secs 2021-11-03T18:38:13.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:12 smithi115 ceph-mon[65453]: pgmap v2508: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:13.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:12 smithi027 ceph-mon[97244]: pgmap v2508: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:15.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:14 smithi115 ceph-mon[65453]: pgmap v2509: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:15.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:14 smithi027 ceph-mon[97244]: pgmap v2509: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:15.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:15.819+0000 7fafb883b700 1 -- 172.21.15.27:0/1552347377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafb40f6530 msgr2=0x7fafb40f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:38:15.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:15.819+0000 7fafb883b700 1 --2- 172.21.15.27:0/1552347377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafb40f6530 0x7fafb40f6950 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 rx=0x7fafac0099e0 tx=0x7fafac004d10).stop 2021-11-03T18:38:15.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:15.819+0000 7fafb883b700 1 -- 172.21.15.27:0/1552347377 shutdown_connections 2021-11-03T18:38:15.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:15.819+0000 7fafb883b700 1 --2- 172.21.15.27:0/1552347377 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafb40f77d0 0x7fafb40f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:15.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:15.819+0000 7fafb883b700 1 --2- 172.21.15.27:0/1552347377 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafb40f6530 0x7fafb40f6950 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:15.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:15.819+0000 7fafb883b700 1 -- 172.21.15.27:0/1552347377 >> 172.21.15.27:0/1552347377 conn(0x7fafb40f1c90 msgr2=0x7fafb40f40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:38:15.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:15.819+0000 7fafb883b700 1 -- 172.21.15.27:0/1552347377 shutdown_connections 2021-11-03T18:38:15.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:15.819+0000 7fafb883b700 1 -- 172.21.15.27:0/1552347377 wait complete. 2021-11-03T18:38:15.824 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:15.820+0000 7fafb883b700 1 Processor -- start 2021-11-03T18:38:15.825 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:15.820+0000 7fafb883b700 1 -- start start 2021-11-03T18:38:15.826 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:15.820+0000 7fafb883b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafb40f6530 0x7fafb4106e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:38:15.826 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:15.820+0000 7fafb883b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafb40f77d0 0x7fafb4107380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:38:15.826 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:15.820+0000 7fafb883b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb4107930 con 0x7fafb40f77d0 2021-11-03T18:38:15.827 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:15.820+0000 7fafb883b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafb4107a70 con 0x7fafb40f6530 2021-11-03T18:38:15.827 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:15.820+0000 7fafb259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafb40f77d0 0x7fafb4107380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:38:15.827 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:15.820+0000 7fafb259c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafb40f77d0 0x7fafb4107380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35974/0 (socket says 172.21.15.27:35974) 2021-11-03T18:38:15.827 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:15.820+0000 7fafb259c700 1 -- 172.21.15.27:0/335548214 learned_addr learned my addr 172.21.15.27:0/335548214 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:38:15.827 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:15.820+0000 7fafb2d9d700 1 --2- 172.21.15.27:0/335548214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafb40f6530 0x7fafb4106e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:38:15.828 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:15.820+0000 7fafb259c700 1 -- 172.21.15.27:0/335548214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafb40f6530 msgr2=0x7fafb4106e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:38:15.828 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:15.820+0000 7fafb259c700 1 --2- 172.21.15.27:0/335548214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafb40f6530 0x7fafb4106e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:15.828 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:15.820+0000 7fafb259c700 1 -- 172.21.15.27:0/335548214 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafac009640 con 0x7fafb40f77d0 2021-11-03T18:38:15.828 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:15.820+0000 7fafb2d9d700 1 --2- 172.21.15.27:0/335548214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafb40f6530 0x7fafb4106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T18:38:15.829 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:15.821+0000 7fafb259c700 1 --2- 172.21.15.27:0/335548214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafb40f77d0 0x7fafb4107380 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 rx=0x7faf9c00e7f0 tx=0x7faf9c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:38:15.829 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:15.821+0000 7fafab7fe700 1 -- 172.21.15.27:0/335548214 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf9c0092e0 con 0x7fafb40f77d0 2021-11-03T18:38:15.829 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:15.821+0000 7fafab7fe700 1 -- 172.21.15.27:0/335548214 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7faf9c00f040 con 0x7fafb40f77d0 2021-11-03T18:38:15.829 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:15.821+0000 7fafb883b700 1 -- 172.21.15.27:0/335548214 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafb4108410 con 0x7fafb40f77d0 2021-11-03T18:38:15.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:15.821+0000 7fafab7fe700 1 -- 172.21.15.27:0/335548214 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf9c014750 con 0x7fafb40f77d0 2021-11-03T18:38:15.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:15.821+0000 7fafb883b700 1 -- 172.21.15.27:0/335548214 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafb41089d0 con 0x7fafb40f77d0 2021-11-03T18:38:15.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:15.822+0000 7fafb883b700 1 -- 172.21.15.27:0/335548214 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafb4045ad0 con 0x7fafb40f77d0 2021-11-03T18:38:15.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:15.823+0000 7fafab7fe700 1 -- 172.21.15.27:0/335548214 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7faf9c01b030 con 0x7fafb40f77d0 2021-11-03T18:38:15.830 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:15.823+0000 7fafab7fe700 1 --2- 172.21.15.27:0/335548214 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fafa0064100 0x7fafa00665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:38:15.831 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:15.823+0000 7fafab7fe700 1 -- 172.21.15.27:0/335548214 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7faf9c080790 con 0x7fafb40f77d0 2021-11-03T18:38:15.831 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:15.823+0000 7fafb2d9d700 1 --2- 172.21.15.27:0/335548214 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fafa0064100 0x7fafa00665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:38:15.831 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:15.824+0000 7fafb2d9d700 1 --2- 172.21.15.27:0/335548214 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fafa0064100 0x7fafa00665c0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7fafac00b3b0 tx=0x7fafac004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:38:15.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:15.827+0000 7fafab7fe700 1 -- 172.21.15.27:0/335548214 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7faf9c052ab0 con 0x7fafb40f77d0 2021-11-03T18:38:16.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.009+0000 7fafb883b700 1 -- 172.21.15.27:0/335548214 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fafb40fee60 con 0x7fafa0064100 2021-11-03T18:38:16.011 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.010+0000 7fafab7fe700 1 -- 172.21.15.27:0/335548214 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fafb40fee60 con 0x7fafa0064100 2021-11-03T18:38:16.011 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.012+0000 7fafb883b700 1 -- 172.21.15.27:0/335548214 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fafa0064100 msgr2=0x7fafa00665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:38:16.012 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.012+0000 7fafb883b700 1 --2- 172.21.15.27:0/335548214 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fafa0064100 0x7fafa00665c0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7fafac00b3b0 tx=0x7fafac004d10).stop 2021-11-03T18:38:16.012 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.012+0000 7fafb883b700 1 -- 172.21.15.27:0/335548214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafb40f77d0 msgr2=0x7fafb4107380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:38:16.012 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.012+0000 7fafb883b700 1 --2- 172.21.15.27:0/335548214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafb40f77d0 0x7fafb4107380 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 rx=0x7faf9c00e7f0 tx=0x7faf9c00b040).stop 2021-11-03T18:38:16.012 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.012+0000 7fafb883b700 1 -- 172.21.15.27:0/335548214 shutdown_connections 2021-11-03T18:38:16.013 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.012+0000 7fafb883b700 1 --2- 172.21.15.27:0/335548214 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fafa0064100 0x7fafa00665c0 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:16.013 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.012+0000 7fafb883b700 1 --2- 172.21.15.27:0/335548214 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fafb40f6530 0x7fafb4106e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:16.013 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.012+0000 7fafb883b700 1 --2- 172.21.15.27:0/335548214 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fafb40f77d0 0x7fafb4107380 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:16.013 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.012+0000 7fafb883b700 1 -- 172.21.15.27:0/335548214 >> 172.21.15.27:0/335548214 conn(0x7fafb40f1c90 msgr2=0x7fafb40fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:38:16.014 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.012+0000 7fafb883b700 1 -- 172.21.15.27:0/335548214 shutdown_connections 2021-11-03T18:38:16.015 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.012+0000 7fafb883b700 1 -- 172.21.15.27:0/335548214 wait complete. 2021-11-03T18:38:16.020 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:38:16.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.112+0000 7fdf096f3700 1 -- 172.21.15.27:0/3376227047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf040f66c0 msgr2=0x7fdf040f6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:38:16.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.112+0000 7fdf096f3700 1 --2- 172.21.15.27:0/3376227047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf040f66c0 0x7fdf040f6ae0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 rx=0x7fdef8004660 tx=0x7fdef8009b30).stop 2021-11-03T18:38:16.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.112+0000 7fdf096f3700 1 -- 172.21.15.27:0/3376227047 shutdown_connections 2021-11-03T18:38:16.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.112+0000 7fdf096f3700 1 --2- 172.21.15.27:0/3376227047 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf040f7960 0x7fdf040f7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:16.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.112+0000 7fdf096f3700 1 --2- 172.21.15.27:0/3376227047 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf040f66c0 0x7fdf040f6ae0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:16.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.112+0000 7fdf096f3700 1 -- 172.21.15.27:0/3376227047 >> 172.21.15.27:0/3376227047 conn(0x7fdf040f1e20 msgr2=0x7fdf040f4240 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:38:16.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.112+0000 7fdf096f3700 1 -- 172.21.15.27:0/3376227047 shutdown_connections 2021-11-03T18:38:16.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.112+0000 7fdf096f3700 1 -- 172.21.15.27:0/3376227047 wait complete. 2021-11-03T18:38:16.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.113+0000 7fdf096f3700 1 Processor -- start 2021-11-03T18:38:16.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.113+0000 7fdf096f3700 1 -- start start 2021-11-03T18:38:16.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.113+0000 7fdf096f3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf040f66c0 0x7fdf0406b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:38:16.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.113+0000 7fdf096f3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf040f7960 0x7fdf04069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:38:16.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.113+0000 7fdf096f3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0406a380 con 0x7fdf040f66c0 2021-11-03T18:38:16.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.113+0000 7fdf096f3700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0406a4c0 con 0x7fdf040f7960 2021-11-03T18:38:16.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.113+0000 7fdf02ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf040f66c0 0x7fdf0406b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:38:16.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.113+0000 7fdf02ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf040f66c0 0x7fdf0406b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35982/0 (socket says 172.21.15.27:35982) 2021-11-03T18:38:16.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.113+0000 7fdf02ffd700 1 -- 172.21.15.27:0/3499537863 learned_addr learned my addr 172.21.15.27:0/3499537863 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:38:16.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.113+0000 7fdf027fc700 1 --2- 172.21.15.27:0/3499537863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf040f7960 0x7fdf04069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:38:16.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.114+0000 7fdf02ffd700 1 -- 172.21.15.27:0/3499537863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf040f7960 msgr2=0x7fdf04069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:38:16.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.114+0000 7fdf02ffd700 1 --2- 172.21.15.27:0/3499537863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf040f7960 0x7fdf04069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:16.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.114+0000 7fdf02ffd700 1 -- 172.21.15.27:0/3499537863 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef8005040 con 0x7fdf040f66c0 2021-11-03T18:38:16.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.114+0000 7fdf02ffd700 1 --2- 172.21.15.27:0/3499537863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf040f66c0 0x7fdf0406b9a0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 rx=0x7fdef8004f60 tx=0x7fdef8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:38:16.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.114+0000 7fdeeb7fe700 1 -- 172.21.15.27:0/3499537863 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdef8018070 con 0x7fdf040f66c0 2021-11-03T18:38:16.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.114+0000 7fdeeb7fe700 1 -- 172.21.15.27:0/3499537863 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fdef8007ba0 con 0x7fdf040f66c0 2021-11-03T18:38:16.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.114+0000 7fdeeb7fe700 1 -- 172.21.15.27:0/3499537863 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdef801c620 con 0x7fdf040f66c0 2021-11-03T18:38:16.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.114+0000 7fdf096f3700 1 -- 172.21.15.27:0/3499537863 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf0406a740 con 0x7fdf040f66c0 2021-11-03T18:38:16.121 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.114+0000 7fdf096f3700 1 -- 172.21.15.27:0/3499537863 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf0406aca0 con 0x7fdf040f66c0 2021-11-03T18:38:16.121 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.115+0000 7fdf096f3700 1 -- 172.21.15.27:0/3499537863 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf04045ad0 con 0x7fdf040f66c0 2021-11-03T18:38:16.122 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.115+0000 7fdeeb7fe700 1 -- 172.21.15.27:0/3499537863 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fdef801d420 con 0x7fdf040f66c0 2021-11-03T18:38:16.122 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.116+0000 7fdeeb7fe700 1 --2- 172.21.15.27:0/3499537863 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdeec0640b0 0x7fdeec066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:38:16.122 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.116+0000 7fdeeb7fe700 1 -- 172.21.15.27:0/3499537863 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fdef8082610 con 0x7fdf040f66c0 2021-11-03T18:38:16.122 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.116+0000 7fdf027fc700 1 --2- 172.21.15.27:0/3499537863 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdeec0640b0 0x7fdeec066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:38:16.122 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.116+0000 7fdf027fc700 1 --2- 172.21.15.27:0/3499537863 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdeec0640b0 0x7fdeec066570 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0x7fdef40041d0 tx=0x7fdef4009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:38:16.123 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.120+0000 7fdeeb7fe700 1 -- 172.21.15.27:0/3499537863 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fdef8054a30 con 0x7fdf040f66c0 2021-11-03T18:38:16.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4937.478605 secs 2021-11-03T18:38:16.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4937.478605 secs 2021-11-03T18:38:16.291 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.293+0000 7fdf096f3700 1 -- 172.21.15.27:0/3499537863 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf04057f10 con 0x7fdeec0640b0 2021-11-03T18:38:16.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.308+0000 7fdeeb7fe700 1 -- 172.21.15.27:0/3499537863 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fdf04057f10 con 0x7fdeec0640b0 2021-11-03T18:38:16.306 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:38:16.307 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (89m) 8m ago 91m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:38:16.307 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (81m) 8m ago 91m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:38:16.307 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (81m) 8m ago 90m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:38:16.307 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (89m) 8m ago 90m 42.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:38:16.308 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (83m) 8m ago 86m 26.7M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:38:16.308 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (86m) 8m ago 86m 3850M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:38:16.308 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (82m) 8m ago 86m 26.3M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:38:16.308 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (83m) 8m ago 86m 27.4M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:38:16.309 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (83m) 8m ago 92m 608M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:38:16.309 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (82m) 8m ago 90m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:38:16.309 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (82m) 8m ago 92m 350M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:38:16.309 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (82m) 8m ago 90m 160M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:38:16.309 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (90m) 8m ago 90m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:38:16.310 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (90m) 8m ago 90m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:38:16.310 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (81m) 8m ago 89m 413M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:38:16.310 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (80m) 8m ago 89m 450M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:38:16.310 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (80m) 8m ago 88m 338M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:38:16.311 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (79m) 8m ago 88m 479M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:38:16.311 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (79m) 8m ago 88m 378M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:38:16.311 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (78m) 8m ago 87m 388M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:38:16.311 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (89m) 8m ago 90m 64.4M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:38:16.313 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.310+0000 7fdf096f3700 1 -- 172.21.15.27:0/3499537863 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdeec0640b0 msgr2=0x7fdeec066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:38:16.313 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.310+0000 7fdf096f3700 1 --2- 172.21.15.27:0/3499537863 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdeec0640b0 0x7fdeec066570 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0x7fdef40041d0 tx=0x7fdef4009040).stop 2021-11-03T18:38:16.313 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.310+0000 7fdf096f3700 1 -- 172.21.15.27:0/3499537863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf040f66c0 msgr2=0x7fdf0406b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:38:16.314 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.310+0000 7fdf096f3700 1 --2- 172.21.15.27:0/3499537863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf040f66c0 0x7fdf0406b9a0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 rx=0x7fdef8004f60 tx=0x7fdef8009b30).stop 2021-11-03T18:38:16.314 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.310+0000 7fdf096f3700 1 -- 172.21.15.27:0/3499537863 shutdown_connections 2021-11-03T18:38:16.314 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.310+0000 7fdf096f3700 1 --2- 172.21.15.27:0/3499537863 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdeec0640b0 0x7fdeec066570 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:16.314 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.310+0000 7fdf096f3700 1 --2- 172.21.15.27:0/3499537863 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdf040f66c0 0x7fdf0406b9a0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:16.315 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.310+0000 7fdf096f3700 1 --2- 172.21.15.27:0/3499537863 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdf040f7960 0x7fdf04069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:16.315 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.310+0000 7fdf096f3700 1 -- 172.21.15.27:0/3499537863 >> 172.21.15.27:0/3499537863 conn(0x7fdf040f1e20 msgr2=0x7fdf040fb3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:38:16.315 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.310+0000 7fdf096f3700 1 -- 172.21.15.27:0/3499537863 shutdown_connections 2021-11-03T18:38:16.315 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.310+0000 7fdf096f3700 1 -- 172.21.15.27:0/3499537863 wait complete. 2021-11-03T18:38:16.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.409+0000 7f53df7af700 1 -- 172.21.15.27:0/2238387848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53d80f55c0 msgr2=0x7f53d80f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:38:16.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.409+0000 7f53df7af700 1 --2- 172.21.15.27:0/2238387848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53d80f55c0 0x7f53d80f59e0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 rx=0x7f53d00099e0 tx=0x7f53d0004d10).stop 2021-11-03T18:38:16.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.409+0000 7f53df7af700 1 -- 172.21.15.27:0/2238387848 shutdown_connections 2021-11-03T18:38:16.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.409+0000 7f53df7af700 1 --2- 172.21.15.27:0/2238387848 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53d80f6860 0x7f53d80f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:16.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.409+0000 7f53df7af700 1 --2- 172.21.15.27:0/2238387848 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53d80f55c0 0x7f53d80f59e0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:16.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.409+0000 7f53df7af700 1 -- 172.21.15.27:0/2238387848 >> 172.21.15.27:0/2238387848 conn(0x7f53d80f0d40 msgr2=0x7f53d80f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:38:16.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.409+0000 7f53df7af700 1 -- 172.21.15.27:0/2238387848 shutdown_connections 2021-11-03T18:38:16.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.409+0000 7f53df7af700 1 -- 172.21.15.27:0/2238387848 wait complete. 2021-11-03T18:38:16.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.409+0000 7f53df7af700 1 Processor -- start 2021-11-03T18:38:16.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.410+0000 7f53df7af700 1 -- start start 2021-11-03T18:38:16.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.410+0000 7f53df7af700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53d80f55c0 0x7f53d8109d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:38:16.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.410+0000 7f53df7af700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53d80f6860 0x7f53d810c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:38:16.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.410+0000 7f53df7af700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d80603e0 con 0x7f53d80f6860 2021-11-03T18:38:16.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.410+0000 7f53df7af700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d8060520 con 0x7f53d80f55c0 2021-11-03T18:38:16.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.410+0000 7f53dcd4a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53d80f6860 0x7f53d810c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:38:16.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.410+0000 7f53dcd4a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53d80f6860 0x7f53d810c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:35994/0 (socket says 172.21.15.27:35994) 2021-11-03T18:38:16.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.410+0000 7f53dcd4a700 1 -- 172.21.15.27:0/3236284820 learned_addr learned my addr 172.21.15.27:0/3236284820 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:38:16.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.410+0000 7f53dd54b700 1 --2- 172.21.15.27:0/3236284820 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53d80f55c0 0x7f53d8109d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:38:16.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.410+0000 7f53dcd4a700 1 -- 172.21.15.27:0/3236284820 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53d80f55c0 msgr2=0x7f53d8109d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:38:16.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.410+0000 7f53dcd4a700 1 --2- 172.21.15.27:0/3236284820 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53d80f55c0 0x7f53d8109d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:16.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.410+0000 7f53dcd4a700 1 -- 172.21.15.27:0/3236284820 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d0009640 con 0x7f53d80f6860 2021-11-03T18:38:16.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.410+0000 7f53dcd4a700 1 --2- 172.21.15.27:0/3236284820 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53d80f6860 0x7f53d810c260 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 rx=0x7f53cc0026d0 tx=0x7f53cc009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:38:16.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.410+0000 7f53d5ffb700 1 -- 172.21.15.27:0/3236284820 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53cc0079c0 con 0x7f53d80f6860 2021-11-03T18:38:16.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.410+0000 7f53d5ffb700 1 -- 172.21.15.27:0/3236284820 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f53cc00f040 con 0x7f53d80f6860 2021-11-03T18:38:16.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.410+0000 7f53df7af700 1 -- 172.21.15.27:0/3236284820 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53d810c810 con 0x7f53d80f6860 2021-11-03T18:38:16.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.411+0000 7f53d5ffb700 1 -- 172.21.15.27:0/3236284820 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53cc0148b0 con 0x7f53d80f6860 2021-11-03T18:38:16.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.411+0000 7f53df7af700 1 -- 172.21.15.27:0/3236284820 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53d810cd50 con 0x7f53d80f6860 2021-11-03T18:38:16.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.412+0000 7f53df7af700 1 -- 172.21.15.27:0/3236284820 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53d8045ad0 con 0x7f53d80f6860 2021-11-03T18:38:16.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.412+0000 7f53d5ffb700 1 -- 172.21.15.27:0/3236284820 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f53cc007b20 con 0x7f53d80f6860 2021-11-03T18:38:16.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.412+0000 7f53d5ffb700 1 --2- 172.21.15.27:0/3236284820 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f53c00640b0 0x7f53c0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:38:16.419 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.412+0000 7f53d5ffb700 1 -- 172.21.15.27:0/3236284820 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f53cc080660 con 0x7f53d80f6860 2021-11-03T18:38:16.419 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.413+0000 7f53dd54b700 1 --2- 172.21.15.27:0/3236284820 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f53c00640b0 0x7f53c0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:38:16.419 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.413+0000 7f53dd54b700 1 --2- 172.21.15.27:0/3236284820 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f53c00640b0 0x7f53c0066570 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 rx=0x7f53d00099b0 tx=0x7f53d0004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:38:16.420 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.416+0000 7f53d5ffb700 1 -- 172.21.15.27:0/3236284820 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f53cc0529c0 con 0x7f53d80f6860 2021-11-03T18:38:16.643 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.645+0000 7f53df7af700 1 -- 172.21.15.27:0/3236284820 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f53d8027a00 con 0x7f53d80f6860 2021-11-03T18:38:16.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.646+0000 7f53d5ffb700 1 -- 172.21.15.27:0/3236284820 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f53cc0525b0 con 0x7f53d80f6860 2021-11-03T18:38:16.645 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:38:16.645 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:38:16.645 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:38:16.645 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:38:16.646 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:38:16.646 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:38:16.646 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:38:16.646 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:38:16.647 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:38:16.647 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:38:16.647 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:38:16.647 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:38:16.648 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:38:16.648 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:38:16.648 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:38:16.648 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:38:16.649 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:38:16.649 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:38:16.650 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.647+0000 7f53df7af700 1 -- 172.21.15.27:0/3236284820 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f53c00640b0 msgr2=0x7f53c0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:38:16.650 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.647+0000 7f53df7af700 1 --2- 172.21.15.27:0/3236284820 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f53c00640b0 0x7f53c0066570 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 rx=0x7f53d00099b0 tx=0x7f53d0004d10).stop 2021-11-03T18:38:16.650 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.647+0000 7f53df7af700 1 -- 172.21.15.27:0/3236284820 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53d80f6860 msgr2=0x7f53d810c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:38:16.650 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.647+0000 7f53df7af700 1 --2- 172.21.15.27:0/3236284820 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53d80f6860 0x7f53d810c260 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 rx=0x7f53cc0026d0 tx=0x7f53cc009de0).stop 2021-11-03T18:38:16.651 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.648+0000 7f53df7af700 1 -- 172.21.15.27:0/3236284820 shutdown_connections 2021-11-03T18:38:16.651 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.648+0000 7f53df7af700 1 --2- 172.21.15.27:0/3236284820 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f53c00640b0 0x7f53c0066570 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:16.651 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.648+0000 7f53df7af700 1 --2- 172.21.15.27:0/3236284820 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53d80f55c0 0x7f53d8109d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:16.651 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.648+0000 7f53df7af700 1 --2- 172.21.15.27:0/3236284820 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53d80f6860 0x7f53d810c260 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:16.652 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.648+0000 7f53df7af700 1 -- 172.21.15.27:0/3236284820 >> 172.21.15.27:0/3236284820 conn(0x7f53d80f0d40 msgr2=0x7f53d80fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:38:16.652 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.648+0000 7f53df7af700 1 -- 172.21.15.27:0/3236284820 shutdown_connections 2021-11-03T18:38:16.652 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.648+0000 7f53df7af700 1 -- 172.21.15.27:0/3236284820 wait complete. 2021-11-03T18:38:16.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.748+0000 7fab7e9fa700 1 -- 172.21.15.27:0/2936114155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab780f6530 msgr2=0x7fab780f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:38:16.749 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.748+0000 7fab7e9fa700 1 --2- 172.21.15.27:0/2936114155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab780f6530 0x7fab780f6950 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 rx=0x7fab74004660 tx=0x7fab74009b30).stop 2021-11-03T18:38:16.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.749+0000 7fab7e9fa700 1 -- 172.21.15.27:0/2936114155 shutdown_connections 2021-11-03T18:38:16.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.749+0000 7fab7e9fa700 1 --2- 172.21.15.27:0/2936114155 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab780f77d0 0x7fab780f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:16.750 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.749+0000 7fab7e9fa700 1 --2- 172.21.15.27:0/2936114155 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab780f6530 0x7fab780f6950 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:16.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.749+0000 7fab7e9fa700 1 -- 172.21.15.27:0/2936114155 >> 172.21.15.27:0/2936114155 conn(0x7fab780f1cd0 msgr2=0x7fab780f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:38:16.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.749+0000 7fab7e9fa700 1 -- 172.21.15.27:0/2936114155 shutdown_connections 2021-11-03T18:38:16.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.749+0000 7fab7e9fa700 1 -- 172.21.15.27:0/2936114155 wait complete. 2021-11-03T18:38:16.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.749+0000 7fab7e9fa700 1 Processor -- start 2021-11-03T18:38:16.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.749+0000 7fab7e9fa700 1 -- start start 2021-11-03T18:38:16.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.749+0000 7fab7e9fa700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab780f6530 0x7fab7806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:38:16.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.749+0000 7fab7e9fa700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab780f77d0 0x7fab78069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:38:16.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.749+0000 7fab7e9fa700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7806a380 con 0x7fab780f77d0 2021-11-03T18:38:16.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.749+0000 7fab7e9fa700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab7806a4c0 con 0x7fab780f6530 2021-11-03T18:38:16.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.749+0000 7fab7d1f7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab780f77d0 0x7fab78069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:38:16.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.749+0000 7fab7d1f7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab780f77d0 0x7fab78069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36004/0 (socket says 172.21.15.27:36004) 2021-11-03T18:38:16.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.749+0000 7fab7d1f7700 1 -- 172.21.15.27:0/1719995843 learned_addr learned my addr 172.21.15.27:0/1719995843 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:38:16.754 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.750+0000 7fab7d9f8700 1 --2- 172.21.15.27:0/1719995843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab780f6530 0x7fab7806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:38:16.754 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.750+0000 7fab7d1f7700 1 -- 172.21.15.27:0/1719995843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab780f6530 msgr2=0x7fab7806b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:38:16.754 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.750+0000 7fab7d1f7700 1 --2- 172.21.15.27:0/1719995843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab780f6530 0x7fab7806b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:16.755 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.750+0000 7fab7d1f7700 1 -- 172.21.15.27:0/1719995843 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab74005040 con 0x7fab780f77d0 2021-11-03T18:38:16.755 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.750+0000 7fab7d1f7700 1 --2- 172.21.15.27:0/1719995843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab780f77d0 0x7fab78069e20 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 rx=0x7fab680026d0 tx=0x7fab68009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:38:16.755 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.750+0000 7fab6e7fc700 1 -- 172.21.15.27:0/1719995843 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab680079c0 con 0x7fab780f77d0 2021-11-03T18:38:16.756 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.750+0000 7fab6e7fc700 1 -- 172.21.15.27:0/1719995843 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fab6800f040 con 0x7fab780f77d0 2021-11-03T18:38:16.756 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.750+0000 7fab7e9fa700 1 -- 172.21.15.27:0/1719995843 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab7806a7a0 con 0x7fab780f77d0 2021-11-03T18:38:16.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.750+0000 7fab6e7fc700 1 -- 172.21.15.27:0/1719995843 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab680148b0 con 0x7fab780f77d0 2021-11-03T18:38:16.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.750+0000 7fab7e9fa700 1 -- 172.21.15.27:0/1719995843 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab7810ed70 con 0x7fab780f77d0 2021-11-03T18:38:16.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.751+0000 7fab7e9fa700 1 -- 172.21.15.27:0/1719995843 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab78045ad0 con 0x7fab780f77d0 2021-11-03T18:38:16.760 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.751+0000 7fab6e7fc700 1 -- 172.21.15.27:0/1719995843 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fab68007b20 con 0x7fab780f77d0 2021-11-03T18:38:16.760 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.752+0000 7fab6e7fc700 1 --2- 172.21.15.27:0/1719995843 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fab64064100 0x7fab640665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:38:16.761 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.752+0000 7fab6e7fc700 1 -- 172.21.15.27:0/1719995843 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fab68080660 con 0x7fab780f77d0 2021-11-03T18:38:16.761 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.752+0000 7fab7d9f8700 1 --2- 172.21.15.27:0/1719995843 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fab64064100 0x7fab640665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:38:16.762 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.752+0000 7fab7d9f8700 1 --2- 172.21.15.27:0/1719995843 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fab64064100 0x7fab640665c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7fab74013690 tx=0x7fab74009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:38:16.762 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.756+0000 7fab6e7fc700 1 -- 172.21.15.27:0/1719995843 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fab680529c0 con 0x7fab780f77d0 2021-11-03T18:38:16.957 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.959+0000 7fab7e9fa700 1 -- 172.21.15.27:0/1719995843 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fab780586e0 con 0x7fab780f77d0 2021-11-03T18:38:16.958 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.959+0000 7fab6e7fc700 1 -- 172.21.15.27:0/1719995843 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7fab680525b0 con 0x7fab780f77d0 2021-11-03T18:38:16.958 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:38:16.958 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:38:16.959 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:38:16.959 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:38:16.959 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:38:16.959 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:38:16.960 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:38:16.960 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:38:16.960 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:38:16.960 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:38:16.960 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:38:16.961 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:38:16.961 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:38:16.961 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:38:16.961 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:38:16.962 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:38:16.962 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:38:16.962 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:38:16.962 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:38:16.963 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:38:16.963 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:38:16.963 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:38:16.963 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:38:16.964 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:38:16.964 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:38:16.964 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:38:16.964 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:38:16.965 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:38:16.965 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:38:16.965 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:38:16.965 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:38:16.966 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:38:16.966 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:38:16.966 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:38:16.966 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:38:16.967 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:38:16.967 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:38:16.967 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:38:16.967 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:38:16.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.961+0000 7fab7e9fa700 1 -- 172.21.15.27:0/1719995843 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fab64064100 msgr2=0x7fab640665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:38:16.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.961+0000 7fab7e9fa700 1 --2- 172.21.15.27:0/1719995843 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fab64064100 0x7fab640665c0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 rx=0x7fab74013690 tx=0x7fab74009b30).stop 2021-11-03T18:38:16.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.961+0000 7fab7e9fa700 1 -- 172.21.15.27:0/1719995843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab780f77d0 msgr2=0x7fab78069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:38:16.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.961+0000 7fab7e9fa700 1 --2- 172.21.15.27:0/1719995843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab780f77d0 0x7fab78069e20 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 rx=0x7fab680026d0 tx=0x7fab68009de0).stop 2021-11-03T18:38:16.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.961+0000 7fab7e9fa700 1 -- 172.21.15.27:0/1719995843 shutdown_connections 2021-11-03T18:38:16.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.961+0000 7fab7e9fa700 1 --2- 172.21.15.27:0/1719995843 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fab64064100 0x7fab640665c0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:16.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.961+0000 7fab7e9fa700 1 --2- 172.21.15.27:0/1719995843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab780f6530 0x7fab7806b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:16.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.961+0000 7fab7e9fa700 1 --2- 172.21.15.27:0/1719995843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab780f77d0 0x7fab78069e20 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:16.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.961+0000 7fab7e9fa700 1 -- 172.21.15.27:0/1719995843 >> 172.21.15.27:0/1719995843 conn(0x7fab780f1cd0 msgr2=0x7fab780fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:38:16.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.961+0000 7fab7e9fa700 1 -- 172.21.15.27:0/1719995843 shutdown_connections 2021-11-03T18:38:16.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:16.961+0000 7fab7e9fa700 1 -- 172.21.15.27:0/1719995843 wait complete. 2021-11-03T18:38:16.971 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:38:17.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:16 smithi115 ceph-mon[65453]: pgmap v2510: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:17.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:16 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3236284820' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:17.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:16 smithi027 ceph-mon[97244]: pgmap v2510: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:17.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:16 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3236284820' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:18.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:17 smithi115 ceph-mon[65453]: from='client.36680 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:38:18.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:17 smithi115 ceph-mon[65453]: from='client.36684 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:38:18.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:17 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1719995843' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:38:18.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:17 smithi027 ceph-mon[97244]: from='client.36680 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:38:18.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:17 smithi027 ceph-mon[97244]: from='client.36684 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:38:18.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:17 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1719995843' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:38:19.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:18 smithi115 ceph-mon[65453]: pgmap v2511: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:19.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:18 smithi027 ceph-mon[97244]: pgmap v2511: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:20.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:38:20.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:38:20.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:38:20.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:20.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:20.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:20.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:20.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:38:20.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:38:20.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:38:20.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:20.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:20.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:20.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:21.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:20 smithi115 ceph-mon[65453]: pgmap v2512: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:21.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:20 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:38:21.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4942.478688 secs 2021-11-03T18:38:21.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:20 smithi027 ceph-mon[97244]: pgmap v2512: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:21.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:20 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:38:21.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4942.478688 secs 2021-11-03T18:38:23.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:22 smithi115 ceph-mon[65453]: pgmap v2513: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:23.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:22 smithi027 ceph-mon[97244]: pgmap v2513: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:25.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:24 smithi115 ceph-mon[65453]: pgmap v2514: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:25.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:24 smithi027 ceph-mon[97244]: pgmap v2514: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:26.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4947.478772 secs 2021-11-03T18:38:26.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4947.478772 secs 2021-11-03T18:38:27.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:26 smithi115 ceph-mon[65453]: pgmap v2515: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:27.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:26 smithi027 ceph-mon[97244]: pgmap v2515: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:29.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:28 smithi115 ceph-mon[65453]: pgmap v2516: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:29.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:28 smithi027 ceph-mon[97244]: pgmap v2516: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:30.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:38:30.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:38:30.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:38:30.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:30.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:30.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:30.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:30.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:38:30.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:38:30.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:38:30.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:30.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:30.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:30.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:31.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:30 smithi115 ceph-mon[65453]: pgmap v2517: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:31.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:30 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:38:31.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4952.478918 secs 2021-11-03T18:38:31.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:30 smithi027 ceph-mon[97244]: pgmap v2517: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:31.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:30 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:38:31.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4952.478918 secs 2021-11-03T18:38:33.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:32 smithi115 ceph-mon[65453]: pgmap v2518: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:33.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:32 smithi027 ceph-mon[97244]: pgmap v2518: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:35.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:34 smithi115 ceph-mon[65453]: pgmap v2519: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:35.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:34 smithi027 ceph-mon[97244]: pgmap v2519: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:36.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4957.479047 secs 2021-11-03T18:38:36.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4957.479047 secs 2021-11-03T18:38:37.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:36 smithi115 ceph-mon[65453]: pgmap v2520: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:37.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:36 smithi027 ceph-mon[97244]: pgmap v2520: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:39.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:38 smithi115 ceph-mon[65453]: pgmap v2521: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:39.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:38 smithi027 ceph-mon[97244]: pgmap v2521: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:40.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:38:40.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:38:40.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:38:40.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:38:41.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:40 smithi115 ceph-mon[65453]: pgmap v2522: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:41.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:40 smithi115 ceph-mon[65453]: poll took 60185.602 ms: 1 events 2021-11-03T18:38:41.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4962.479134 secs 2021-11-03T18:38:41.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:38:41.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:38:41.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:41.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:41.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:41.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:41.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:40 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:38:41.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:40 smithi027 ceph-mon[97244]: pgmap v2522: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:41.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:40 smithi027 ceph-mon[97244]: poll took 60185.602 ms: 1 events 2021-11-03T18:38:41.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4962.479134 secs 2021-11-03T18:38:41.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:38:41.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:38:41.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:41.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:41.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:41.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:41.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:40 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:38:42.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:41 smithi115 ceph-mon[65453]: pgmap v2523: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:42.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:41 smithi027 ceph-mon[97244]: pgmap v2523: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:44.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:44 smithi027 ceph-mon[97244]: pgmap v2524: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:44.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:44 smithi115 ceph-mon[65453]: pgmap v2524: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:45.522 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4967.479271 secs 2021-11-03T18:38:45.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4967.479271 secs 2021-11-03T18:38:46.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:46 smithi115 ceph-mon[65453]: pgmap v2525: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:47.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:46 smithi027 ceph-mon[97244]: pgmap v2525: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:47.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.076+0000 7f925c764700 1 -- 172.21.15.27:0/4218739132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92540f55c0 msgr2=0x7f92540f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:38:47.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.076+0000 7f925c764700 1 --2- 172.21.15.27:0/4218739132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92540f55c0 0x7f92540f59e0 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 rx=0x7f924c0099e0 tx=0x7f924c004d10).stop 2021-11-03T18:38:47.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.076+0000 7f925c764700 1 -- 172.21.15.27:0/4218739132 shutdown_connections 2021-11-03T18:38:47.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.076+0000 7f925c764700 1 --2- 172.21.15.27:0/4218739132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92540f6860 0x7f92540f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:47.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.076+0000 7f925c764700 1 --2- 172.21.15.27:0/4218739132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92540f55c0 0x7f92540f59e0 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:47.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.076+0000 7f925c764700 1 -- 172.21.15.27:0/4218739132 >> 172.21.15.27:0/4218739132 conn(0x7f92540f0d40 msgr2=0x7f92540f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:38:47.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.076+0000 7f925c764700 1 -- 172.21.15.27:0/4218739132 shutdown_connections 2021-11-03T18:38:47.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.076+0000 7f925c764700 1 -- 172.21.15.27:0/4218739132 wait complete. 2021-11-03T18:38:47.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.077+0000 7f925c764700 1 Processor -- start 2021-11-03T18:38:47.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.077+0000 7f925c764700 1 -- start start 2021-11-03T18:38:47.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.077+0000 7f925c764700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92540f55c0 0x7f9254109d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:38:47.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.077+0000 7f925c764700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92540f6860 0x7f925410c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:38:47.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.077+0000 7f925c764700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9254060420 con 0x7f92540f55c0 2021-11-03T18:38:47.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.077+0000 7f925c764700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9254060560 con 0x7f92540f6860 2021-11-03T18:38:47.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.077+0000 7f925a500700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92540f55c0 0x7f9254109d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:38:47.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.077+0000 7f925a500700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92540f55c0 0x7f9254109d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36024/0 (socket says 172.21.15.27:36024) 2021-11-03T18:38:47.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.077+0000 7f925a500700 1 -- 172.21.15.27:0/1675029235 learned_addr learned my addr 172.21.15.27:0/1675029235 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:38:47.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.077+0000 7f9259cff700 1 --2- 172.21.15.27:0/1675029235 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92540f6860 0x7f925410c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:38:47.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.077+0000 7f925a500700 1 -- 172.21.15.27:0/1675029235 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92540f6860 msgr2=0x7f925410c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:38:47.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.077+0000 7f925a500700 1 --2- 172.21.15.27:0/1675029235 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92540f6860 0x7f925410c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:47.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.077+0000 7f925a500700 1 -- 172.21.15.27:0/1675029235 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f924c009640 con 0x7f92540f55c0 2021-11-03T18:38:47.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.077+0000 7f925a500700 1 --2- 172.21.15.27:0/1675029235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92540f55c0 0x7f9254109d20 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 rx=0x7f924c00c010 tx=0x7f924c004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:38:47.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.078+0000 7f9252ffd700 1 -- 172.21.15.27:0/1675029235 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f924c00e070 con 0x7f92540f55c0 2021-11-03T18:38:47.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.078+0000 7f925c764700 1 -- 172.21.15.27:0/1675029235 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f925410c7d0 con 0x7f92540f55c0 2021-11-03T18:38:47.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.078+0000 7f9252ffd700 1 -- 172.21.15.27:0/1675029235 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f924c007d10 con 0x7f92540f55c0 2021-11-03T18:38:47.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.078+0000 7f9252ffd700 1 -- 172.21.15.27:0/1675029235 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f924c018070 con 0x7f92540f55c0 2021-11-03T18:38:47.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.078+0000 7f925c764700 1 -- 172.21.15.27:0/1675029235 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f925410cc80 con 0x7f92540f55c0 2021-11-03T18:38:47.083 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.079+0000 7f925c764700 1 -- 172.21.15.27:0/1675029235 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9254045ad0 con 0x7f92540f55c0 2021-11-03T18:38:47.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.079+0000 7f9252ffd700 1 -- 172.21.15.27:0/1675029235 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f924c024060 con 0x7f92540f55c0 2021-11-03T18:38:47.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.079+0000 7f9252ffd700 1 --2- 172.21.15.27:0/1675029235 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f923c0640b0 0x7f923c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:38:47.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.080+0000 7f9252ffd700 1 -- 172.21.15.27:0/1675029235 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f924c081fe0 con 0x7f92540f55c0 2021-11-03T18:38:47.084 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.080+0000 7f9259cff700 1 --2- 172.21.15.27:0/1675029235 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f923c0640b0 0x7f923c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:38:47.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.080+0000 7f9259cff700 1 --2- 172.21.15.27:0/1675029235 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f923c0640b0 0x7f923c066570 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 rx=0x7f9248004c40 tx=0x7f924800b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:38:47.085 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.084+0000 7f9252ffd700 1 -- 172.21.15.27:0/1675029235 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f924c054340 con 0x7f92540f55c0 2021-11-03T18:38:47.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.265+0000 7f925c764700 1 -- 172.21.15.27:0/1675029235 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f92540fdf00 con 0x7f923c0640b0 2021-11-03T18:38:47.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.266+0000 7f9252ffd700 1 -- 172.21.15.27:0/1675029235 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f92540fdf00 con 0x7f923c0640b0 2021-11-03T18:38:47.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.268+0000 7f925c764700 1 -- 172.21.15.27:0/1675029235 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f923c0640b0 msgr2=0x7f923c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:38:47.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.268+0000 7f925c764700 1 --2- 172.21.15.27:0/1675029235 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f923c0640b0 0x7f923c066570 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 rx=0x7f9248004c40 tx=0x7f924800b040).stop 2021-11-03T18:38:47.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.268+0000 7f925c764700 1 -- 172.21.15.27:0/1675029235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92540f55c0 msgr2=0x7f9254109d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:38:47.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.268+0000 7f925c764700 1 --2- 172.21.15.27:0/1675029235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92540f55c0 0x7f9254109d20 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 rx=0x7f924c00c010 tx=0x7f924c004d10).stop 2021-11-03T18:38:47.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.268+0000 7f925c764700 1 -- 172.21.15.27:0/1675029235 shutdown_connections 2021-11-03T18:38:47.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.268+0000 7f925c764700 1 --2- 172.21.15.27:0/1675029235 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f923c0640b0 0x7f923c066570 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:47.270 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.268+0000 7f925c764700 1 --2- 172.21.15.27:0/1675029235 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f92540f55c0 0x7f9254109d20 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:47.270 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.268+0000 7f925c764700 1 --2- 172.21.15.27:0/1675029235 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f92540f6860 0x7f925410c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:47.271 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.268+0000 7f925c764700 1 -- 172.21.15.27:0/1675029235 >> 172.21.15.27:0/1675029235 conn(0x7f92540f0d40 msgr2=0x7f92540fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:38:47.271 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.268+0000 7f925c764700 1 -- 172.21.15.27:0/1675029235 shutdown_connections 2021-11-03T18:38:47.272 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.268+0000 7f925c764700 1 -- 172.21.15.27:0/1675029235 wait complete. 2021-11-03T18:38:47.276 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:38:47.367 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.368+0000 7f0d1d39d700 1 -- 172.21.15.27:0/2744087069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d180f51a0 msgr2=0x7f0d180f7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:38:47.367 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.368+0000 7f0d1d39d700 1 --2- 172.21.15.27:0/2744087069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d180f51a0 0x7f0d180f7920 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 rx=0x7f0d0c004660 tx=0x7f0d0c009b30).stop 2021-11-03T18:38:47.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.368+0000 7f0d1d39d700 1 -- 172.21.15.27:0/2744087069 shutdown_connections 2021-11-03T18:38:47.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.368+0000 7f0d1d39d700 1 --2- 172.21.15.27:0/2744087069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d180f51a0 0x7f0d180f7920 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:47.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.368+0000 7f0d1d39d700 1 --2- 172.21.15.27:0/2744087069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d180f4800 0x7f0d180f4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:47.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.368+0000 7f0d1d39d700 1 -- 172.21.15.27:0/2744087069 >> 172.21.15.27:0/2744087069 conn(0x7f0d180f0320 msgr2=0x7f0d180f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:38:47.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.368+0000 7f0d1d39d700 1 -- 172.21.15.27:0/2744087069 shutdown_connections 2021-11-03T18:38:47.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.368+0000 7f0d1d39d700 1 -- 172.21.15.27:0/2744087069 wait complete. 2021-11-03T18:38:47.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.369+0000 7f0d1d39d700 1 Processor -- start 2021-11-03T18:38:47.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.369+0000 7f0d1d39d700 1 -- start start 2021-11-03T18:38:47.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.369+0000 7f0d1d39d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d180f4800 0x7f0d18106e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:38:47.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.369+0000 7f0d1d39d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d180f51a0 0x7f0d18107310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:38:47.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.369+0000 7f0d1d39d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d18107910 con 0x7f0d180f51a0 2021-11-03T18:38:47.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.369+0000 7f0d1d39d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d18108120 con 0x7f0d180f4800 2021-11-03T18:38:47.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.369+0000 7f0d177fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d180f51a0 0x7f0d18107310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:38:47.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.369+0000 7f0d177fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d180f51a0 0x7f0d18107310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36036/0 (socket says 172.21.15.27:36036) 2021-11-03T18:38:47.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.369+0000 7f0d177fe700 1 -- 172.21.15.27:0/3465463344 learned_addr learned my addr 172.21.15.27:0/3465463344 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:38:47.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.369+0000 7f0d177fe700 1 -- 172.21.15.27:0/3465463344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d180f4800 msgr2=0x7f0d18106e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:38:47.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.369+0000 7f0d17fff700 1 --2- 172.21.15.27:0/3465463344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d180f4800 0x7f0d18106e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:38:47.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.369+0000 7f0d177fe700 1 --2- 172.21.15.27:0/3465463344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d180f4800 0x7f0d18106e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:47.374 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.369+0000 7f0d177fe700 1 -- 172.21.15.27:0/3465463344 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d0c005040 con 0x7f0d180f51a0 2021-11-03T18:38:47.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.369+0000 7f0d17fff700 1 --2- 172.21.15.27:0/3465463344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d180f4800 0x7f0d18106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T18:38:47.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.370+0000 7f0d177fe700 1 --2- 172.21.15.27:0/3465463344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d180f51a0 0x7f0d18107310 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 rx=0x7f0d0c004f60 tx=0x7f0d0c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:38:47.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.370+0000 7f0d14ff9700 1 -- 172.21.15.27:0/3465463344 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d0c018070 con 0x7f0d180f51a0 2021-11-03T18:38:47.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.370+0000 7f0d14ff9700 1 -- 172.21.15.27:0/3465463344 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f0d0c007ba0 con 0x7f0d180f51a0 2021-11-03T18:38:47.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.370+0000 7f0d1d39d700 1 -- 172.21.15.27:0/3465463344 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d18108340 con 0x7f0d180f51a0 2021-11-03T18:38:47.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.370+0000 7f0d14ff9700 1 -- 172.21.15.27:0/3465463344 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0d0c01c620 con 0x7f0d180f51a0 2021-11-03T18:38:47.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.370+0000 7f0d1d39d700 1 -- 172.21.15.27:0/3465463344 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d181088a0 con 0x7f0d180f51a0 2021-11-03T18:38:47.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.371+0000 7f0d1d39d700 1 -- 172.21.15.27:0/3465463344 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d18059100 con 0x7f0d180f51a0 2021-11-03T18:38:47.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.371+0000 7f0d14ff9700 1 -- 172.21.15.27:0/3465463344 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f0d0c01c8e0 con 0x7f0d180f51a0 2021-11-03T18:38:47.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.371+0000 7f0d14ff9700 1 --2- 172.21.15.27:0/3465463344 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0d00064100 0x7f0d000665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:38:47.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.372+0000 7f0d14ff9700 1 -- 172.21.15.27:0/3465463344 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f0d0c0826f0 con 0x7f0d180f51a0 2021-11-03T18:38:47.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.372+0000 7f0d17fff700 1 --2- 172.21.15.27:0/3465463344 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0d00064100 0x7f0d000665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:38:47.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.372+0000 7f0d17fff700 1 --2- 172.21.15.27:0/3465463344 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0d00064100 0x7f0d000665c0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x7f0d08007fd0 tx=0x7f0d08009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:38:47.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.376+0000 7f0d14ff9700 1 -- 172.21.15.27:0/3465463344 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f0d0c054a50 con 0x7f0d180f51a0 2021-11-03T18:38:47.547 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.548+0000 7f0d1d39d700 1 -- 172.21.15.27:0/3465463344 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d180feeb0 con 0x7f0d00064100 2021-11-03T18:38:47.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.563+0000 7f0d14ff9700 1 -- 172.21.15.27:0/3465463344 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f0d180feeb0 con 0x7f0d00064100 2021-11-03T18:38:47.561 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:38:47.561 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (90m) 9m ago 91m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:38:47.562 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (82m) 9m ago 91m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:38:47.562 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (81m) 8m ago 90m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:38:47.562 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (90m) 9m ago 91m 42.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:38:47.562 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (83m) 9m ago 86m 26.7M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:38:47.563 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (86m) 9m ago 86m 3850M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:38:47.563 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (82m) 8m ago 86m 26.3M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:38:47.563 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (84m) 8m ago 86m 27.4M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:38:47.563 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (84m) 9m ago 92m 608M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:38:47.563 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (83m) 8m ago 90m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:38:47.564 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (83m) 9m ago 93m 350M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:38:47.564 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (82m) 8m ago 90m 160M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:38:47.564 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (91m) 9m ago 91m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:38:47.564 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (90m) 8m ago 90m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:38:47.565 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (81m) 9m ago 90m 413M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:38:47.565 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (81m) 9m ago 89m 450M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:38:47.565 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (80m) 9m ago 89m 338M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:38:47.565 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (80m) 8m ago 88m 479M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:38:47.565 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (79m) 8m ago 88m 378M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:38:47.566 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (79m) 8m ago 88m 388M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:38:47.566 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (90m) 9m ago 90m 64.4M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:38:47.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.565+0000 7f0d1d39d700 1 -- 172.21.15.27:0/3465463344 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0d00064100 msgr2=0x7f0d000665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:38:47.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.565+0000 7f0d1d39d700 1 --2- 172.21.15.27:0/3465463344 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0d00064100 0x7f0d000665c0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 rx=0x7f0d08007fd0 tx=0x7f0d08009040).stop 2021-11-03T18:38:47.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.565+0000 7f0d1d39d700 1 -- 172.21.15.27:0/3465463344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d180f51a0 msgr2=0x7f0d18107310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:38:47.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.565+0000 7f0d1d39d700 1 --2- 172.21.15.27:0/3465463344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d180f51a0 0x7f0d18107310 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 rx=0x7f0d0c004f60 tx=0x7f0d0c009b30).stop 2021-11-03T18:38:47.567 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.565+0000 7f0d1d39d700 1 -- 172.21.15.27:0/3465463344 shutdown_connections 2021-11-03T18:38:47.568 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.565+0000 7f0d1d39d700 1 --2- 172.21.15.27:0/3465463344 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0d00064100 0x7f0d000665c0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:47.568 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.565+0000 7f0d1d39d700 1 --2- 172.21.15.27:0/3465463344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0d180f4800 0x7f0d18106e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:47.568 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.565+0000 7f0d1d39d700 1 --2- 172.21.15.27:0/3465463344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0d180f51a0 0x7f0d18107310 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:47.568 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.565+0000 7f0d1d39d700 1 -- 172.21.15.27:0/3465463344 >> 172.21.15.27:0/3465463344 conn(0x7f0d180f0320 msgr2=0x7f0d180fb370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:38:47.569 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.565+0000 7f0d1d39d700 1 -- 172.21.15.27:0/3465463344 shutdown_connections 2021-11-03T18:38:47.569 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.565+0000 7f0d1d39d700 1 -- 172.21.15.27:0/3465463344 wait complete. 2021-11-03T18:38:47.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.664+0000 7f09b2eae700 1 -- 172.21.15.27:0/1653458942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09ac0f55e0 msgr2=0x7f09ac0f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:38:47.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.664+0000 7f09b2eae700 1 --2- 172.21.15.27:0/1653458942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09ac0f55e0 0x7f09ac0f5a00 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 rx=0x7f09a0004660 tx=0x7f09a0009b30).stop 2021-11-03T18:38:47.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.664+0000 7f09b2eae700 1 -- 172.21.15.27:0/1653458942 shutdown_connections 2021-11-03T18:38:47.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.664+0000 7f09b2eae700 1 --2- 172.21.15.27:0/1653458942 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09ac0f6880 0x7f09ac0f6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:47.665 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.664+0000 7f09b2eae700 1 --2- 172.21.15.27:0/1653458942 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09ac0f55e0 0x7f09ac0f5a00 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:47.665 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.664+0000 7f09b2eae700 1 -- 172.21.15.27:0/1653458942 >> 172.21.15.27:0/1653458942 conn(0x7f09ac0f0d40 msgr2=0x7f09ac0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:38:47.665 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.664+0000 7f09b2eae700 1 -- 172.21.15.27:0/1653458942 shutdown_connections 2021-11-03T18:38:47.665 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.664+0000 7f09b2eae700 1 -- 172.21.15.27:0/1653458942 wait complete. 2021-11-03T18:38:47.665 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.664+0000 7f09b2eae700 1 Processor -- start 2021-11-03T18:38:47.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.664+0000 7f09b2eae700 1 -- start start 2021-11-03T18:38:47.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.664+0000 7f09b2eae700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09ac0f55e0 0x7f09ac0f9f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:38:47.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.664+0000 7f09b2eae700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09ac0f6880 0x7f09ac0f83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:38:47.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.664+0000 7f09b2eae700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac0f8940 con 0x7f09ac0f55e0 2021-11-03T18:38:47.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.664+0000 7f09b2eae700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ac0f8a80 con 0x7f09ac0f6880 2021-11-03T18:38:47.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.665+0000 7f09b0c4a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09ac0f55e0 0x7f09ac0f9f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:38:47.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.665+0000 7f09b0c4a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09ac0f55e0 0x7f09ac0f9f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36044/0 (socket says 172.21.15.27:36044) 2021-11-03T18:38:47.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.665+0000 7f09b0c4a700 1 -- 172.21.15.27:0/2255105951 learned_addr learned my addr 172.21.15.27:0/2255105951 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:38:47.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.665+0000 7f09abfff700 1 --2- 172.21.15.27:0/2255105951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09ac0f6880 0x7f09ac0f83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:38:47.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.665+0000 7f09b0c4a700 1 -- 172.21.15.27:0/2255105951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09ac0f6880 msgr2=0x7f09ac0f83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:38:47.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.665+0000 7f09b0c4a700 1 --2- 172.21.15.27:0/2255105951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09ac0f6880 0x7f09ac0f83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:47.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.665+0000 7f09b0c4a700 1 -- 172.21.15.27:0/2255105951 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09a0005040 con 0x7f09ac0f55e0 2021-11-03T18:38:47.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.665+0000 7f09b0c4a700 1 --2- 172.21.15.27:0/2255105951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09ac0f55e0 0x7f09ac0f9f60 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 rx=0x7f09a0013a70 tx=0x7f09a0009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:38:47.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.665+0000 7f09a97fa700 1 -- 172.21.15.27:0/2255105951 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09a0018070 con 0x7f09ac0f55e0 2021-11-03T18:38:47.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.665+0000 7f09b2eae700 1 -- 172.21.15.27:0/2255105951 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09ac0f8d00 con 0x7f09ac0f55e0 2021-11-03T18:38:47.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.665+0000 7f09a97fa700 1 -- 172.21.15.27:0/2255105951 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f09a00097d0 con 0x7f09ac0f55e0 2021-11-03T18:38:47.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.665+0000 7f09a97fa700 1 -- 172.21.15.27:0/2255105951 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f09a001de80 con 0x7f09ac0f55e0 2021-11-03T18:38:47.670 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.665+0000 7f09b2eae700 1 -- 172.21.15.27:0/2255105951 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09ac0f9260 con 0x7f09ac0f55e0 2021-11-03T18:38:47.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.666+0000 7f09b2eae700 1 -- 172.21.15.27:0/2255105951 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09ac045ad0 con 0x7f09ac0f55e0 2021-11-03T18:38:47.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.667+0000 7f09a97fa700 1 -- 172.21.15.27:0/2255105951 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f09a0007ba0 con 0x7f09ac0f55e0 2021-11-03T18:38:47.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.667+0000 7f09a97fa700 1 --2- 172.21.15.27:0/2255105951 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f099c0640b0 0x7f099c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:38:47.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.667+0000 7f09a97fa700 1 -- 172.21.15.27:0/2255105951 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f09a00823d0 con 0x7f09ac0f55e0 2021-11-03T18:38:47.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.667+0000 7f09abfff700 1 --2- 172.21.15.27:0/2255105951 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f099c0640b0 0x7f099c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:38:47.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.668+0000 7f09abfff700 1 --2- 172.21.15.27:0/2255105951 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f099c0640b0 0x7f099c066570 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0x7f0998004e20 tx=0x7f099800b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:38:47.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.671+0000 7f09a97fa700 1 -- 172.21.15.27:0/2255105951 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f09a00547f0 con 0x7f09ac0f55e0 2021-11-03T18:38:47.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.901+0000 7f09b2eae700 1 -- 172.21.15.27:0/2255105951 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f09ac10f6a0 con 0x7f09ac0f55e0 2021-11-03T18:38:47.900 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.902+0000 7f09a97fa700 1 -- 172.21.15.27:0/2255105951 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f09a00543e0 con 0x7f09ac0f55e0 2021-11-03T18:38:47.900 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:38:47.901 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:38:47.901 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:38:47.901 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:38:47.901 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:38:47.902 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:38:47.902 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:38:47.902 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:38:47.902 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:38:47.902 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:38:47.903 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:38:47.903 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:38:47.903 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:38:47.903 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:38:47.904 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:38:47.904 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:38:47.904 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:38:47.904 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:38:47.905 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.903+0000 7f09b2eae700 1 -- 172.21.15.27:0/2255105951 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f099c0640b0 msgr2=0x7f099c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:38:47.905 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.903+0000 7f09b2eae700 1 --2- 172.21.15.27:0/2255105951 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f099c0640b0 0x7f099c066570 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 rx=0x7f0998004e20 tx=0x7f099800b040).stop 2021-11-03T18:38:47.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.903+0000 7f09b2eae700 1 -- 172.21.15.27:0/2255105951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09ac0f55e0 msgr2=0x7f09ac0f9f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:38:47.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.903+0000 7f09b2eae700 1 --2- 172.21.15.27:0/2255105951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09ac0f55e0 0x7f09ac0f9f60 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 rx=0x7f09a0013a70 tx=0x7f09a0009b30).stop 2021-11-03T18:38:47.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.904+0000 7f09b2eae700 1 -- 172.21.15.27:0/2255105951 shutdown_connections 2021-11-03T18:38:47.906 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.904+0000 7f09b2eae700 1 --2- 172.21.15.27:0/2255105951 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f099c0640b0 0x7f099c066570 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:47.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.904+0000 7f09b2eae700 1 --2- 172.21.15.27:0/2255105951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09ac0f55e0 0x7f09ac0f9f60 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:47.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.904+0000 7f09b2eae700 1 --2- 172.21.15.27:0/2255105951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09ac0f6880 0x7f09ac0f83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:47.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.904+0000 7f09b2eae700 1 -- 172.21.15.27:0/2255105951 >> 172.21.15.27:0/2255105951 conn(0x7f09ac0f0d40 msgr2=0x7f09ac05f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:38:47.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.904+0000 7f09b2eae700 1 -- 172.21.15.27:0/2255105951 shutdown_connections 2021-11-03T18:38:47.908 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:47.904+0000 7f09b2eae700 1 -- 172.21.15.27:0/2255105951 wait complete. 2021-11-03T18:38:48.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.003+0000 7f0f5b59e700 1 -- 172.21.15.27:0/3912183872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f5c0f77d0 msgr2=0x7f0f5c0f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:38:48.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.003+0000 7f0f5b59e700 1 --2- 172.21.15.27:0/3912183872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f5c0f77d0 0x7f0f5c0f7c30 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 rx=0x7f0f50004660 tx=0x7f0f50009b30).stop 2021-11-03T18:38:48.004 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.003+0000 7f0f5b59e700 1 -- 172.21.15.27:0/3912183872 shutdown_connections 2021-11-03T18:38:48.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.003+0000 7f0f5b59e700 1 --2- 172.21.15.27:0/3912183872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f5c0f77d0 0x7f0f5c0f7c30 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:48.005 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.003+0000 7f0f5b59e700 1 --2- 172.21.15.27:0/3912183872 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f5c0f6530 0x7f0f5c0f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:48.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.003+0000 7f0f5b59e700 1 -- 172.21.15.27:0/3912183872 >> 172.21.15.27:0/3912183872 conn(0x7f0f5c0f1cd0 msgr2=0x7f0f5c0f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:38:48.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.004+0000 7f0f5b59e700 1 -- 172.21.15.27:0/3912183872 shutdown_connections 2021-11-03T18:38:48.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.004+0000 7f0f5b59e700 1 -- 172.21.15.27:0/3912183872 wait complete. 2021-11-03T18:38:48.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.004+0000 7f0f5b59e700 1 Processor -- start 2021-11-03T18:38:48.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.004+0000 7f0f5b59e700 1 -- start start 2021-11-03T18:38:48.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.004+0000 7f0f5b59e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f5c0f6530 0x7f0f5c106d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:38:48.007 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.004+0000 7f0f5b59e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f5c0f77d0 0x7f0f5c107270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:38:48.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.004+0000 7f0f5b59e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c107870 con 0x7f0f5c0f6530 2021-11-03T18:38:48.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.004+0000 7f0f5b59e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c108080 con 0x7f0f5c0f77d0 2021-11-03T18:38:48.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.004+0000 7f0f5a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f5c0f6530 0x7f0f5c106d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:38:48.008 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.004+0000 7f0f5a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f5c0f6530 0x7f0f5c106d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36054/0 (socket says 172.21.15.27:36054) 2021-11-03T18:38:48.009 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.004+0000 7f0f5a59c700 1 -- 172.21.15.27:0/2289972078 learned_addr learned my addr 172.21.15.27:0/2289972078 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:38:48.009 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.005+0000 7f0f59d9b700 1 --2- 172.21.15.27:0/2289972078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f5c0f77d0 0x7f0f5c107270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:38:48.009 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.005+0000 7f0f5a59c700 1 -- 172.21.15.27:0/2289972078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f5c0f77d0 msgr2=0x7f0f5c107270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:38:48.009 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.005+0000 7f0f5a59c700 1 --2- 172.21.15.27:0/2289972078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f5c0f77d0 0x7f0f5c107270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:48.009 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.005+0000 7f0f5a59c700 1 -- 172.21.15.27:0/2289972078 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f50005040 con 0x7f0f5c0f6530 2021-11-03T18:38:48.010 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.005+0000 7f0f5a59c700 1 --2- 172.21.15.27:0/2289972078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f5c0f6530 0x7f0f5c106d80 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 rx=0x7f0f4c00c530 tx=0x7f0f4c007e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:38:48.010 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.005+0000 7f0f4affd700 1 -- 172.21.15.27:0/2289972078 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f4c00f650 con 0x7f0f5c0f6530 2021-11-03T18:38:48.010 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.005+0000 7f0f4affd700 1 -- 172.21.15.27:0/2289972078 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f0f4c00fc60 con 0x7f0f5c0f6530 2021-11-03T18:38:48.010 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.005+0000 7f0f4affd700 1 -- 172.21.15.27:0/2289972078 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f4c012d20 con 0x7f0f5c0f6530 2021-11-03T18:38:48.011 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.005+0000 7f0f5b59e700 1 -- 172.21.15.27:0/2289972078 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f5c108300 con 0x7f0f5c0f6530 2021-11-03T18:38:48.011 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.005+0000 7f0f5b59e700 1 -- 172.21.15.27:0/2289972078 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f5c1088c0 con 0x7f0f5c0f6530 2021-11-03T18:38:48.011 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.006+0000 7f0f5b59e700 1 -- 172.21.15.27:0/2289972078 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f5c059100 con 0x7f0f5c0f6530 2021-11-03T18:38:48.012 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.006+0000 7f0f4affd700 1 -- 172.21.15.27:0/2289972078 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f0f4c019030 con 0x7f0f5c0f6530 2021-11-03T18:38:48.013 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.007+0000 7f0f4affd700 1 --2- 172.21.15.27:0/2289972078 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0f44064100 0x7f0f440665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:38:48.013 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.007+0000 7f0f4affd700 1 -- 172.21.15.27:0/2289972078 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f0f4c080620 con 0x7f0f5c0f6530 2021-11-03T18:38:48.013 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.007+0000 7f0f59d9b700 1 --2- 172.21.15.27:0/2289972078 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0f44064100 0x7f0f440665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:38:48.014 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.007+0000 7f0f59d9b700 1 --2- 172.21.15.27:0/2289972078 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0f44064100 0x7f0f440665c0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 rx=0x7f0f50009da0 tx=0x7f0f50009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:38:48.014 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.011+0000 7f0f4affd700 1 -- 172.21.15.27:0/2289972078 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f0f4c0528e0 con 0x7f0f5c0f6530 2021-11-03T18:38:48.212 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.213+0000 7f0f5b59e700 1 -- 172.21.15.27:0/2289972078 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0f5c0fecf0 con 0x7f0f5c0f6530 2021-11-03T18:38:48.212 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.214+0000 7f0f4affd700 1 -- 172.21.15.27:0/2289972078 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f0f4c017710 con 0x7f0f5c0f6530 2021-11-03T18:38:48.213 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:38:48.213 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:38:48.214 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:38:48.214 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:38:48.214 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:38:48.214 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:38:48.215 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:38:48.215 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:38:48.215 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:38:48.215 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:38:48.216 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:38:48.216 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:38:48.216 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:38:48.216 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:38:48.216 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:38:48.217 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:38:48.217 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:38:48.217 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:38:48.217 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:38:48.218 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:38:48.218 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:38:48.218 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:38:48.218 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:38:48.218 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:38:48.219 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:38:48.219 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:38:48.219 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:38:48.219 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:38:48.220 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:38:48.220 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:38:48.220 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:38:48.220 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:38:48.220 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:38:48.221 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:38:48.221 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:38:48.221 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:38:48.221 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:38:48.221 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:38:48.222 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:38:48.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.216+0000 7f0f5b59e700 1 -- 172.21.15.27:0/2289972078 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0f44064100 msgr2=0x7f0f440665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:38:48.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.216+0000 7f0f5b59e700 1 --2- 172.21.15.27:0/2289972078 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0f44064100 0x7f0f440665c0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 rx=0x7f0f50009da0 tx=0x7f0f50009b30).stop 2021-11-03T18:38:48.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.216+0000 7f0f5b59e700 1 -- 172.21.15.27:0/2289972078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f5c0f6530 msgr2=0x7f0f5c106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:38:48.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.216+0000 7f0f5b59e700 1 --2- 172.21.15.27:0/2289972078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f5c0f6530 0x7f0f5c106d80 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 rx=0x7f0f4c00c530 tx=0x7f0f4c007e10).stop 2021-11-03T18:38:48.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.216+0000 7f0f5b59e700 1 -- 172.21.15.27:0/2289972078 shutdown_connections 2021-11-03T18:38:48.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.216+0000 7f0f5b59e700 1 --2- 172.21.15.27:0/2289972078 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0f44064100 0x7f0f440665c0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:48.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.216+0000 7f0f5b59e700 1 --2- 172.21.15.27:0/2289972078 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0f5c0f6530 0x7f0f5c106d80 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:48.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.216+0000 7f0f5b59e700 1 --2- 172.21.15.27:0/2289972078 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0f5c0f77d0 0x7f0f5c107270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:38:48.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.216+0000 7f0f5b59e700 1 -- 172.21.15.27:0/2289972078 >> 172.21.15.27:0/2289972078 conn(0x7f0f5c0f1cd0 msgr2=0x7f0f5c0fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:38:48.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.216+0000 7f0f5b59e700 1 -- 172.21.15.27:0/2289972078 shutdown_connections 2021-11-03T18:38:48.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:38:48.216+0000 7f0f5b59e700 1 -- 172.21.15.27:0/2289972078 wait complete. 2021-11-03T18:38:48.225 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:38:48.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:48 smithi115 ceph-mon[65453]: pgmap v2526: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:48.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:48 smithi115 ceph-mon[65453]: from='client.36696 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:38:48.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:48 smithi115 ceph-mon[65453]: from='client.36700 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:38:48.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:48 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2255105951' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:48.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:48 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2289972078' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:38:49.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:48 smithi027 ceph-mon[97244]: pgmap v2526: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:49.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:48 smithi027 ceph-mon[97244]: from='client.36696 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:38:49.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:48 smithi027 ceph-mon[97244]: from='client.36700 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:38:49.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:48 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2255105951' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:49.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:48 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2289972078' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:38:50.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:50 smithi115 ceph-mon[65453]: pgmap v2527: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:50.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4972.479411 secs 2021-11-03T18:38:50.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:38:50.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:38:50.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:38:50.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:50.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:50.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:50.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:51.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:50 smithi027 ceph-mon[97244]: pgmap v2527: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:51.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4972.479411 secs 2021-11-03T18:38:51.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:38:51.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:38:51.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:38:51.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:51.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:51.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:51.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:38:51.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:51 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:38:52.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:51 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:38:52.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:52 smithi027 ceph-mon[97244]: pgmap v2528: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:52.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:52 smithi115 ceph-mon[65453]: pgmap v2528: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:54.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:54 smithi115 ceph-mon[65453]: pgmap v2529: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:55.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:54 smithi027 ceph-mon[97244]: pgmap v2529: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:55.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4977.479532 secs 2021-11-03T18:38:56.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4977.479532 secs 2021-11-03T18:38:56.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:56 smithi115 ceph-mon[65453]: pgmap v2530: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:56.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:38:56.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:38:57.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:56 smithi027 ceph-mon[97244]: pgmap v2530: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:57.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:38:57.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:38:58.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:38:58 smithi115 ceph-mon[65453]: pgmap v2531: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:38:59.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:38:58 smithi027 ceph-mon[97244]: pgmap v2531: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:00.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:00 smithi115 ceph-mon[65453]: pgmap v2532: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:00.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4982.479678 secs 2021-11-03T18:39:00.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:39:00.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:39:00.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:39:00.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:00.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:00.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:00.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:01.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:00 smithi027 ceph-mon[97244]: pgmap v2532: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:01.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4982.479678 secs 2021-11-03T18:39:01.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:39:01.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:39:01.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:39:01.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:01.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:01.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:01.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:01.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:01 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:39:02.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:01 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:39:02.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:02 smithi027 ceph-mon[97244]: pgmap v2533: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:02.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:02 smithi115 ceph-mon[65453]: pgmap v2533: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:04.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:04 smithi115 ceph-mon[65453]: pgmap v2534: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:05.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:04 smithi027 ceph-mon[97244]: pgmap v2534: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:05.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4987.479826 secs 2021-11-03T18:39:06.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4987.479826 secs 2021-11-03T18:39:06.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:06 smithi115 ceph-mon[65453]: pgmap v2535: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:07.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:06 smithi027 ceph-mon[97244]: pgmap v2535: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:08.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:08 smithi115 ceph-mon[65453]: pgmap v2536: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:09.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:08 smithi027 ceph-mon[97244]: pgmap v2536: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:10.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:10 smithi115 ceph-mon[65453]: pgmap v2537: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:10.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4992.479956 secs 2021-11-03T18:39:10.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:39:10.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:39:10.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:39:10.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:10.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:10.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:10.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:11.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:10 smithi027 ceph-mon[97244]: pgmap v2537: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:11.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4992.479956 secs 2021-11-03T18:39:11.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:39:11.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:39:11.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:39:11.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:11.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:11.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:11.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:11.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:11 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:39:12.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:11 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:39:12.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:12 smithi027 ceph-mon[97244]: pgmap v2538: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:12.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:12 smithi115 ceph-mon[65453]: pgmap v2538: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:14.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:14 smithi115 ceph-mon[65453]: pgmap v2539: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:15.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:14 smithi027 ceph-mon[97244]: pgmap v2539: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:15.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 4997.480090 secs 2021-11-03T18:39:16.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 4997.480090 secs 2021-11-03T18:39:16.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:16 smithi115 ceph-mon[65453]: pgmap v2540: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:17.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:16 smithi027 ceph-mon[97244]: pgmap v2540: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:18.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.330+0000 7fab29ecf700 1 -- 172.21.15.27:0/900487823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab240f9850 msgr2=0x7fab240fbc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:39:18.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.330+0000 7fab29ecf700 1 --2- 172.21.15.27:0/900487823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab240f9850 0x7fab240fbc80 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 rx=0x7fab18004660 tx=0x7fab18009b30).stop 2021-11-03T18:39:18.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.330+0000 7fab29ecf700 1 -- 172.21.15.27:0/900487823 shutdown_connections 2021-11-03T18:39:18.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.330+0000 7fab29ecf700 1 --2- 172.21.15.27:0/900487823 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab240f9850 0x7fab240fbc80 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:18.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.330+0000 7fab29ecf700 1 --2- 172.21.15.27:0/900487823 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab24060250 0x7fab240f9360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:18.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.330+0000 7fab29ecf700 1 -- 172.21.15.27:0/900487823 >> 172.21.15.27:0/900487823 conn(0x7fab240f0d40 msgr2=0x7fab240f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:39:18.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.330+0000 7fab29ecf700 1 -- 172.21.15.27:0/900487823 shutdown_connections 2021-11-03T18:39:18.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.330+0000 7fab29ecf700 1 -- 172.21.15.27:0/900487823 wait complete. 2021-11-03T18:39:18.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.331+0000 7fab29ecf700 1 Processor -- start 2021-11-03T18:39:18.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.331+0000 7fab29ecf700 1 -- start start 2021-11-03T18:39:18.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.331+0000 7fab29ecf700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab24060250 0x7fab240f6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:39:18.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.331+0000 7fab29ecf700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab240f9850 0x7fab240f5260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:39:18.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.331+0000 7fab29ecf700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab240f57c0 con 0x7fab240f9850 2021-11-03T18:39:18.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.331+0000 7fab29ecf700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab240f5900 con 0x7fab24060250 2021-11-03T18:39:18.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.331+0000 7fab22ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab240f9850 0x7fab240f5260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:39:18.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.331+0000 7fab22ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab240f9850 0x7fab240f5260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36076/0 (socket says 172.21.15.27:36076) 2021-11-03T18:39:18.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.331+0000 7fab22ffd700 1 -- 172.21.15.27:0/3978411984 learned_addr learned my addr 172.21.15.27:0/3978411984 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:39:18.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.331+0000 7fab237fe700 1 --2- 172.21.15.27:0/3978411984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab24060250 0x7fab240f6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:39:18.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.332+0000 7fab22ffd700 1 -- 172.21.15.27:0/3978411984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab24060250 msgr2=0x7fab240f6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:39:18.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.332+0000 7fab22ffd700 1 --2- 172.21.15.27:0/3978411984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab24060250 0x7fab240f6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:18.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.332+0000 7fab22ffd700 1 -- 172.21.15.27:0/3978411984 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab18005040 con 0x7fab240f9850 2021-11-03T18:39:18.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.332+0000 7fab22ffd700 1 --2- 172.21.15.27:0/3978411984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab240f9850 0x7fab240f5260 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 rx=0x7fab18000c00 tx=0x7fab18009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:39:18.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.332+0000 7fab0bfff700 1 -- 172.21.15.27:0/3978411984 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab18018070 con 0x7fab240f9850 2021-11-03T18:39:18.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.332+0000 7fab0bfff700 1 -- 172.21.15.27:0/3978411984 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fab180097d0 con 0x7fab240f9850 2021-11-03T18:39:18.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.332+0000 7fab29ecf700 1 -- 172.21.15.27:0/3978411984 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab240f5b80 con 0x7fab240f9850 2021-11-03T18:39:18.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.332+0000 7fab0bfff700 1 -- 172.21.15.27:0/3978411984 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab1801de40 con 0x7fab240f9850 2021-11-03T18:39:18.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.332+0000 7fab29ecf700 1 -- 172.21.15.27:0/3978411984 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab240f60e0 con 0x7fab240f9850 2021-11-03T18:39:18.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.333+0000 7fab29ecf700 1 -- 172.21.15.27:0/3978411984 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab24045ad0 con 0x7fab240f9850 2021-11-03T18:39:18.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.333+0000 7fab0bfff700 1 -- 172.21.15.27:0/3978411984 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fab1801d420 con 0x7fab240f9850 2021-11-03T18:39:18.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.334+0000 7fab0bfff700 1 --2- 172.21.15.27:0/3978411984 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fab0c0640b0 0x7fab0c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:39:18.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.334+0000 7fab0bfff700 1 -- 172.21.15.27:0/3978411984 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fab180824a0 con 0x7fab240f9850 2021-11-03T18:39:18.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.334+0000 7fab237fe700 1 --2- 172.21.15.27:0/3978411984 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fab0c0640b0 0x7fab0c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:39:18.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.334+0000 7fab237fe700 1 --2- 172.21.15.27:0/3978411984 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fab0c0640b0 0x7fab0c066570 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0x7fab240f6b70 tx=0x7fab14009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:39:18.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.338+0000 7fab0bfff700 1 -- 172.21.15.27:0/3978411984 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fab180548c0 con 0x7fab240f9850 2021-11-03T18:39:18.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.516+0000 7fab29ecf700 1 -- 172.21.15.27:0/3978411984 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fab240ff000 con 0x7fab0c0640b0 2021-11-03T18:39:18.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.518+0000 7fab0bfff700 1 -- 172.21.15.27:0/3978411984 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fab240ff000 con 0x7fab0c0640b0 2021-11-03T18:39:18.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.520+0000 7fab29ecf700 1 -- 172.21.15.27:0/3978411984 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fab0c0640b0 msgr2=0x7fab0c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:39:18.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.520+0000 7fab29ecf700 1 --2- 172.21.15.27:0/3978411984 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fab0c0640b0 0x7fab0c066570 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 rx=0x7fab240f6b70 tx=0x7fab14009040).stop 2021-11-03T18:39:18.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.520+0000 7fab29ecf700 1 -- 172.21.15.27:0/3978411984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab240f9850 msgr2=0x7fab240f5260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:39:18.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.520+0000 7fab29ecf700 1 --2- 172.21.15.27:0/3978411984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab240f9850 0x7fab240f5260 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 rx=0x7fab18000c00 tx=0x7fab18009b30).stop 2021-11-03T18:39:18.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.520+0000 7fab29ecf700 1 -- 172.21.15.27:0/3978411984 shutdown_connections 2021-11-03T18:39:18.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.520+0000 7fab29ecf700 1 --2- 172.21.15.27:0/3978411984 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fab0c0640b0 0x7fab0c066570 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:18.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.520+0000 7fab29ecf700 1 --2- 172.21.15.27:0/3978411984 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fab24060250 0x7fab240f6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:18.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.520+0000 7fab29ecf700 1 --2- 172.21.15.27:0/3978411984 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fab240f9850 0x7fab240f5260 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:18.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.520+0000 7fab29ecf700 1 -- 172.21.15.27:0/3978411984 >> 172.21.15.27:0/3978411984 conn(0x7fab240f0d40 msgr2=0x7fab240f3130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:39:18.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.520+0000 7fab29ecf700 1 -- 172.21.15.27:0/3978411984 shutdown_connections 2021-11-03T18:39:18.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.520+0000 7fab29ecf700 1 -- 172.21.15.27:0/3978411984 wait complete. 2021-11-03T18:39:18.528 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:39:18.620 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.620+0000 7f67ab6ff700 1 -- 172.21.15.27:0/1999813079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67a40f5ef0 msgr2=0x7f67a40f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:39:18.620 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.620+0000 7f67ab6ff700 1 --2- 172.21.15.27:0/1999813079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67a40f5ef0 0x7f67a40f6920 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 rx=0x7f6798004660 tx=0x7f6798009b30).stop 2021-11-03T18:39:18.621 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.620+0000 7f67ab6ff700 1 -- 172.21.15.27:0/1999813079 shutdown_connections 2021-11-03T18:39:18.621 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.620+0000 7f67ab6ff700 1 --2- 172.21.15.27:0/1999813079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67a40f5ef0 0x7f67a40f6920 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:18.621 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.620+0000 7f67ab6ff700 1 --2- 172.21.15.27:0/1999813079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67a40f55e0 0x7f67a40f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:18.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.620+0000 7f67ab6ff700 1 -- 172.21.15.27:0/1999813079 >> 172.21.15.27:0/1999813079 conn(0x7f67a40f0de0 msgr2=0x7f67a40f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:39:18.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.620+0000 7f67ab6ff700 1 -- 172.21.15.27:0/1999813079 shutdown_connections 2021-11-03T18:39:18.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.620+0000 7f67ab6ff700 1 -- 172.21.15.27:0/1999813079 wait complete. 2021-11-03T18:39:18.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.621+0000 7f67ab6ff700 1 Processor -- start 2021-11-03T18:39:18.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.621+0000 7f67ab6ff700 1 -- start start 2021-11-03T18:39:18.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.621+0000 7f67ab6ff700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67a40f55e0 0x7f67a4105e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:39:18.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.621+0000 7f67ab6ff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67a40f5ef0 0x7f67a4106380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:39:18.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.621+0000 7f67ab6ff700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a4106930 con 0x7f67a40f5ef0 2021-11-03T18:39:18.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.621+0000 7f67ab6ff700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67a4106a70 con 0x7f67a40f55e0 2021-11-03T18:39:18.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.621+0000 7f67a8c9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67a40f5ef0 0x7f67a4106380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:39:18.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.621+0000 7f67a8c9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67a40f5ef0 0x7f67a4106380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36086/0 (socket says 172.21.15.27:36086) 2021-11-03T18:39:18.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.621+0000 7f67a8c9a700 1 -- 172.21.15.27:0/2859553185 learned_addr learned my addr 172.21.15.27:0/2859553185 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:39:18.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.621+0000 7f67a949b700 1 --2- 172.21.15.27:0/2859553185 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67a40f55e0 0x7f67a4105e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:39:18.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.621+0000 7f67a8c9a700 1 -- 172.21.15.27:0/2859553185 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67a40f55e0 msgr2=0x7f67a4105e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:39:18.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.621+0000 7f67a8c9a700 1 --2- 172.21.15.27:0/2859553185 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67a40f55e0 0x7f67a4105e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:18.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.621+0000 7f67a8c9a700 1 -- 172.21.15.27:0/2859553185 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6798005040 con 0x7f67a40f5ef0 2021-11-03T18:39:18.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.621+0000 7f67a8c9a700 1 --2- 172.21.15.27:0/2859553185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67a40f5ef0 0x7f67a4106380 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 rx=0x7f6798009830 tx=0x7f6798009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:39:18.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.621+0000 7f67a1ffb700 1 -- 172.21.15.27:0/2859553185 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6798018070 con 0x7f67a40f5ef0 2021-11-03T18:39:18.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.622+0000 7f67a1ffb700 1 -- 172.21.15.27:0/2859553185 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f6798007ba0 con 0x7f67a40f5ef0 2021-11-03T18:39:18.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.622+0000 7f67ab6ff700 1 -- 172.21.15.27:0/2859553185 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67a41073b0 con 0x7f67a40f5ef0 2021-11-03T18:39:18.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.622+0000 7f67a1ffb700 1 -- 172.21.15.27:0/2859553185 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f679801c3c0 con 0x7f67a40f5ef0 2021-11-03T18:39:18.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.622+0000 7f67ab6ff700 1 -- 172.21.15.27:0/2859553185 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67a4107910 con 0x7f67a40f5ef0 2021-11-03T18:39:18.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.623+0000 7f67ab6ff700 1 -- 172.21.15.27:0/2859553185 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67a4045ad0 con 0x7f67a40f5ef0 2021-11-03T18:39:18.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.623+0000 7f67a1ffb700 1 -- 172.21.15.27:0/2859553185 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f679801d420 con 0x7f67a40f5ef0 2021-11-03T18:39:18.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.624+0000 7f67a1ffb700 1 --2- 172.21.15.27:0/2859553185 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f678c064090 0x7f678c066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:39:18.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.624+0000 7f67a1ffb700 1 -- 172.21.15.27:0/2859553185 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f67980822f0 con 0x7f67a40f5ef0 2021-11-03T18:39:18.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.624+0000 7f67a949b700 1 --2- 172.21.15.27:0/2859553185 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f678c064090 0x7f678c066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:39:18.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.624+0000 7f67a949b700 1 --2- 172.21.15.27:0/2859553185 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f678c064090 0x7f678c066550 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7f679c007db0 tx=0x7f679c009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:39:18.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.628+0000 7f67a1ffb700 1 -- 172.21.15.27:0/2859553185 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f6798054650 con 0x7f67a40f5ef0 2021-11-03T18:39:18.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.799+0000 7f67ab6ff700 1 -- 172.21.15.27:0/2859553185 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f67a40fde40 con 0x7f678c064090 2021-11-03T18:39:18.812 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.814+0000 7f67a1ffb700 1 -- 172.21.15.27:0/2859553185 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f67a40fde40 con 0x7f678c064090 2021-11-03T18:39:18.812 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:39:18.813 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (90m) 9m ago 92m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:39:18.813 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (82m) 9m ago 92m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:39:18.813 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (82m) 9m ago 91m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:39:18.814 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (90m) 9m ago 91m 42.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:39:18.814 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (84m) 9m ago 87m 26.7M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:39:18.814 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (87m) 9m ago 87m 3850M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:39:18.814 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (83m) 9m ago 87m 26.3M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:39:18.814 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (84m) 9m ago 87m 27.4M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:39:18.815 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (84m) 9m ago 93m 608M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:39:18.815 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (83m) 9m ago 91m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:39:18.815 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (83m) 9m ago 93m 350M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:39:18.815 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (83m) 9m ago 91m 160M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:39:18.816 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (91m) 9m ago 91m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:39:18.816 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (91m) 9m ago 91m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:39:18.816 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (82m) 9m ago 90m 413M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:39:18.816 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (81m) 9m ago 90m 450M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:39:18.816 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (81m) 9m ago 89m 338M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:39:18.817 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (80m) 9m ago 89m 479M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:39:18.817 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (80m) 9m ago 89m 378M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:39:18.817 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (79m) 9m ago 88m 388M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:39:18.817 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (90m) 9m ago 91m 64.4M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:39:18.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.816+0000 7f67ab6ff700 1 -- 172.21.15.27:0/2859553185 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f678c064090 msgr2=0x7f678c066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:39:18.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.816+0000 7f67ab6ff700 1 --2- 172.21.15.27:0/2859553185 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f678c064090 0x7f678c066550 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 rx=0x7f679c007db0 tx=0x7f679c009040).stop 2021-11-03T18:39:18.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.816+0000 7f67ab6ff700 1 -- 172.21.15.27:0/2859553185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67a40f5ef0 msgr2=0x7f67a4106380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:39:18.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.816+0000 7f67ab6ff700 1 --2- 172.21.15.27:0/2859553185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67a40f5ef0 0x7f67a4106380 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 rx=0x7f6798009830 tx=0x7f6798009b30).stop 2021-11-03T18:39:18.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.816+0000 7f67ab6ff700 1 -- 172.21.15.27:0/2859553185 shutdown_connections 2021-11-03T18:39:18.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.816+0000 7f67ab6ff700 1 --2- 172.21.15.27:0/2859553185 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f678c064090 0x7f678c066550 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:18.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.816+0000 7f67ab6ff700 1 --2- 172.21.15.27:0/2859553185 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67a40f55e0 0x7f67a4105e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:18.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.816+0000 7f67ab6ff700 1 --2- 172.21.15.27:0/2859553185 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67a40f5ef0 0x7f67a4106380 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:18.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.816+0000 7f67ab6ff700 1 -- 172.21.15.27:0/2859553185 >> 172.21.15.27:0/2859553185 conn(0x7f67a40f0de0 msgr2=0x7f67a40fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:39:18.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.816+0000 7f67ab6ff700 1 -- 172.21.15.27:0/2859553185 shutdown_connections 2021-11-03T18:39:18.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.816+0000 7f67ab6ff700 1 -- 172.21.15.27:0/2859553185 wait complete. 2021-11-03T18:39:18.915 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.915+0000 7f82d0db4700 1 -- 172.21.15.27:0/3950703439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82cc0f5ed0 msgr2=0x7f82cc0f6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:39:18.915 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.915+0000 7f82d0db4700 1 --2- 172.21.15.27:0/3950703439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82cc0f5ed0 0x7f82cc0f6900 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 rx=0x7f82b4004660 tx=0x7f82b4009b30).stop 2021-11-03T18:39:18.916 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.915+0000 7f82d0db4700 1 -- 172.21.15.27:0/3950703439 shutdown_connections 2021-11-03T18:39:18.916 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.915+0000 7f82d0db4700 1 --2- 172.21.15.27:0/3950703439 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82cc0f5ed0 0x7f82cc0f6900 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:18.916 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.915+0000 7f82d0db4700 1 --2- 172.21.15.27:0/3950703439 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82cc0f55c0 0x7f82cc0f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:18.916 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.915+0000 7f82d0db4700 1 -- 172.21.15.27:0/3950703439 >> 172.21.15.27:0/3950703439 conn(0x7f82cc0f0de0 msgr2=0x7f82cc0f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:39:18.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.915+0000 7f82d0db4700 1 -- 172.21.15.27:0/3950703439 shutdown_connections 2021-11-03T18:39:18.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.915+0000 7f82d0db4700 1 -- 172.21.15.27:0/3950703439 wait complete. 2021-11-03T18:39:18.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.915+0000 7f82d0db4700 1 Processor -- start 2021-11-03T18:39:18.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.916+0000 7f82d0db4700 1 -- start start 2021-11-03T18:39:18.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.916+0000 7f82d0db4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82cc0f55c0 0x7f82cc109bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:39:18.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.916+0000 7f82d0db4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82cc0f5ed0 0x7f82cc10c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:39:18.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.916+0000 7f82d0db4700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc0604a0 con 0x7f82cc0f5ed0 2021-11-03T18:39:18.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.916+0000 7f82d0db4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82cc0605e0 con 0x7f82cc0f55c0 2021-11-03T18:39:18.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.916+0000 7f82c9d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82cc0f5ed0 0x7f82cc10c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:39:18.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.916+0000 7f82c9d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82cc0f5ed0 0x7f82cc10c0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36096/0 (socket says 172.21.15.27:36096) 2021-11-03T18:39:18.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.916+0000 7f82c9d9b700 1 -- 172.21.15.27:0/1360180817 learned_addr learned my addr 172.21.15.27:0/1360180817 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:39:18.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.916+0000 7f82ca59c700 1 --2- 172.21.15.27:0/1360180817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82cc0f55c0 0x7f82cc109bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:39:18.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.916+0000 7f82c9d9b700 1 -- 172.21.15.27:0/1360180817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82cc0f55c0 msgr2=0x7f82cc109bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:39:18.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.916+0000 7f82c9d9b700 1 --2- 172.21.15.27:0/1360180817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82cc0f55c0 0x7f82cc109bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:18.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.916+0000 7f82c9d9b700 1 -- 172.21.15.27:0/1360180817 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82b4005040 con 0x7f82cc0f5ed0 2021-11-03T18:39:18.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.916+0000 7f82c9d9b700 1 --2- 172.21.15.27:0/1360180817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82cc0f5ed0 0x7f82cc10c0f0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 rx=0x7f82b4009990 tx=0x7f82b4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:39:18.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.916+0000 7f82c2ffd700 1 -- 172.21.15.27:0/1360180817 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82b4018070 con 0x7f82cc0f5ed0 2021-11-03T18:39:18.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.916+0000 7f82c2ffd700 1 -- 172.21.15.27:0/1360180817 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f82b4007ba0 con 0x7f82cc0f5ed0 2021-11-03T18:39:18.922 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.916+0000 7f82c2ffd700 1 -- 172.21.15.27:0/1360180817 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f82b401c3c0 con 0x7f82cc0f5ed0 2021-11-03T18:39:18.922 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.916+0000 7f82d0db4700 1 -- 172.21.15.27:0/1360180817 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82cc10c640 con 0x7f82cc0f5ed0 2021-11-03T18:39:18.922 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.917+0000 7f82d0db4700 1 -- 172.21.15.27:0/1360180817 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82cc10cb70 con 0x7f82cc0f5ed0 2021-11-03T18:39:18.923 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.917+0000 7f82d0db4700 1 -- 172.21.15.27:0/1360180817 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82cc045ad0 con 0x7f82cc0f5ed0 2021-11-03T18:39:18.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.918+0000 7f82c2ffd700 1 -- 172.21.15.27:0/1360180817 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f82b401d420 con 0x7f82cc0f5ed0 2021-11-03T18:39:18.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.918+0000 7f82c2ffd700 1 --2- 172.21.15.27:0/1360180817 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f82b80640b0 0x7f82b8066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:39:18.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.918+0000 7f82c2ffd700 1 -- 172.21.15.27:0/1360180817 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f82b4082670 con 0x7f82cc0f5ed0 2021-11-03T18:39:18.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.918+0000 7f82ca59c700 1 --2- 172.21.15.27:0/1360180817 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f82b80640b0 0x7f82b8066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:39:18.925 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.919+0000 7f82ca59c700 1 --2- 172.21.15.27:0/1360180817 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f82b80640b0 0x7f82b8066570 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 rx=0x7f82cc060120 tx=0x7f82c4004d80).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:39:18.925 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:18.922+0000 7f82c2ffd700 1 -- 172.21.15.27:0/1360180817 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f82b4054970 con 0x7f82cc0f5ed0 2021-11-03T18:39:18.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:18 smithi115 ceph-mon[65453]: pgmap v2541: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:19.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:18 smithi027 ceph-mon[97244]: pgmap v2541: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:19.150 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.152+0000 7f82d0db4700 1 -- 172.21.15.27:0/1360180817 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f82cc0fdeb0 con 0x7f82cc0f5ed0 2021-11-03T18:39:19.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.152+0000 7f82c2ffd700 1 -- 172.21.15.27:0/1360180817 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f82b4054560 con 0x7f82cc0f5ed0 2021-11-03T18:39:19.152 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:39:19.152 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:39:19.152 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:39:19.152 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:39:19.153 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:39:19.153 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:39:19.153 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:39:19.153 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:39:19.153 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:39:19.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:39:19.154 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:39:19.154 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:39:19.154 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:39:19.154 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:39:19.155 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:39:19.155 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:39:19.155 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:39:19.155 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:39:19.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.154+0000 7f82d0db4700 1 -- 172.21.15.27:0/1360180817 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f82b80640b0 msgr2=0x7f82b8066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:39:19.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.154+0000 7f82d0db4700 1 --2- 172.21.15.27:0/1360180817 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f82b80640b0 0x7f82b8066570 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 rx=0x7f82cc060120 tx=0x7f82c4004d80).stop 2021-11-03T18:39:19.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.154+0000 7f82d0db4700 1 -- 172.21.15.27:0/1360180817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82cc0f5ed0 msgr2=0x7f82cc10c0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:39:19.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.154+0000 7f82d0db4700 1 --2- 172.21.15.27:0/1360180817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82cc0f5ed0 0x7f82cc10c0f0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 rx=0x7f82b4009990 tx=0x7f82b4009b30).stop 2021-11-03T18:39:19.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.154+0000 7f82d0db4700 1 -- 172.21.15.27:0/1360180817 shutdown_connections 2021-11-03T18:39:19.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.154+0000 7f82d0db4700 1 --2- 172.21.15.27:0/1360180817 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f82b80640b0 0x7f82b8066570 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:19.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.154+0000 7f82d0db4700 1 --2- 172.21.15.27:0/1360180817 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f82cc0f55c0 0x7f82cc109bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:19.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.154+0000 7f82d0db4700 1 --2- 172.21.15.27:0/1360180817 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f82cc0f5ed0 0x7f82cc10c0f0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:19.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.154+0000 7f82d0db4700 1 -- 172.21.15.27:0/1360180817 >> 172.21.15.27:0/1360180817 conn(0x7f82cc0f0de0 msgr2=0x7f82cc0fa350 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:39:19.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.154+0000 7f82d0db4700 1 -- 172.21.15.27:0/1360180817 shutdown_connections 2021-11-03T18:39:19.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.154+0000 7f82d0db4700 1 -- 172.21.15.27:0/1360180817 wait complete. 2021-11-03T18:39:19.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.255+0000 7f4f6eeac700 1 -- 172.21.15.27:0/3881000360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f680f55e0 msgr2=0x7f4f680f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:39:19.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.255+0000 7f4f6eeac700 1 --2- 172.21.15.27:0/3881000360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f680f55e0 0x7f4f680f5a00 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 rx=0x7f4f60004660 tx=0x7f4f60009b30).stop 2021-11-03T18:39:19.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.255+0000 7f4f6eeac700 1 -- 172.21.15.27:0/3881000360 shutdown_connections 2021-11-03T18:39:19.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.255+0000 7f4f6eeac700 1 --2- 172.21.15.27:0/3881000360 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f680f5ef0 0x7f4f680f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:19.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.255+0000 7f4f6eeac700 1 --2- 172.21.15.27:0/3881000360 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f680f55e0 0x7f4f680f5a00 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:19.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.255+0000 7f4f6eeac700 1 -- 172.21.15.27:0/3881000360 >> 172.21.15.27:0/3881000360 conn(0x7f4f680f0de0 msgr2=0x7f4f680f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:39:19.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.255+0000 7f4f6eeac700 1 -- 172.21.15.27:0/3881000360 shutdown_connections 2021-11-03T18:39:19.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.255+0000 7f4f6eeac700 1 -- 172.21.15.27:0/3881000360 wait complete. 2021-11-03T18:39:19.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.255+0000 7f4f6eeac700 1 Processor -- start 2021-11-03T18:39:19.259 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.255+0000 7f4f6eeac700 1 -- start start 2021-11-03T18:39:19.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.256+0000 7f4f6eeac700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f680f55e0 0x7f4f68109bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:39:19.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.256+0000 7f4f6eeac700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f680f5ef0 0x7f4f6810c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:39:19.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.256+0000 7f4f6eeac700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f680604e0 con 0x7f4f680f55e0 2021-11-03T18:39:19.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.256+0000 7f4f6eeac700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f68060620 con 0x7f4f680f5ef0 2021-11-03T18:39:19.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.256+0000 7f4f6cc48700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f680f55e0 0x7f4f68109bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:39:19.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.256+0000 7f4f6cc48700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f680f55e0 0x7f4f68109bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36104/0 (socket says 172.21.15.27:36104) 2021-11-03T18:39:19.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.256+0000 7f4f6cc48700 1 -- 172.21.15.27:0/2459258338 learned_addr learned my addr 172.21.15.27:0/2459258338 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:39:19.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.256+0000 7f4f67fff700 1 --2- 172.21.15.27:0/2459258338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f680f5ef0 0x7f4f6810c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:39:19.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.256+0000 7f4f6cc48700 1 -- 172.21.15.27:0/2459258338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f680f5ef0 msgr2=0x7f4f6810c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:39:19.262 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.256+0000 7f4f6cc48700 1 --2- 172.21.15.27:0/2459258338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f680f5ef0 0x7f4f6810c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:19.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.256+0000 7f4f6cc48700 1 -- 172.21.15.27:0/2459258338 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f60005040 con 0x7f4f680f55e0 2021-11-03T18:39:19.263 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.256+0000 7f4f67fff700 1 --2- 172.21.15.27:0/2459258338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f680f5ef0 0x7f4f6810c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T18:39:19.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.256+0000 7f4f6cc48700 1 --2- 172.21.15.27:0/2459258338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f680f55e0 0x7f4f68109bf0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 rx=0x7f4f60009990 tx=0x7f4f60009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:39:19.264 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.256+0000 7f4f657fa700 1 -- 172.21.15.27:0/2459258338 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f60018070 con 0x7f4f680f55e0 2021-11-03T18:39:19.265 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.256+0000 7f4f657fa700 1 -- 172.21.15.27:0/2459258338 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f4f60007ba0 con 0x7f4f680f55e0 2021-11-03T18:39:19.265 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.256+0000 7f4f6eeac700 1 -- 172.21.15.27:0/2459258338 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f6810c640 con 0x7f4f680f55e0 2021-11-03T18:39:19.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.256+0000 7f4f657fa700 1 -- 172.21.15.27:0/2459258338 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f6001c3c0 con 0x7f4f680f55e0 2021-11-03T18:39:19.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.256+0000 7f4f6eeac700 1 -- 172.21.15.27:0/2459258338 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f6810cba0 con 0x7f4f680f55e0 2021-11-03T18:39:19.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.257+0000 7f4f6eeac700 1 -- 172.21.15.27:0/2459258338 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f68045ad0 con 0x7f4f680f55e0 2021-11-03T18:39:19.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.258+0000 7f4f657fa700 1 -- 172.21.15.27:0/2459258338 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f4f6001d420 con 0x7f4f680f55e0 2021-11-03T18:39:19.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.258+0000 7f4f657fa700 1 --2- 172.21.15.27:0/2459258338 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4f540640b0 0x7f4f54066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:39:19.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.258+0000 7f4f657fa700 1 -- 172.21.15.27:0/2459258338 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f4f60082500 con 0x7f4f680f55e0 2021-11-03T18:39:19.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.258+0000 7f4f67fff700 1 --2- 172.21.15.27:0/2459258338 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4f540640b0 0x7f4f54066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:39:19.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.259+0000 7f4f67fff700 1 --2- 172.21.15.27:0/2459258338 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4f540640b0 0x7f4f54066570 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7f4f580049a0 tx=0x7f4f5800b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:39:19.269 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.262+0000 7f4f657fa700 1 -- 172.21.15.27:0/2459258338 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f4f600548c0 con 0x7f4f680f55e0 2021-11-03T18:39:19.464 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.465+0000 7f4f6eeac700 1 -- 172.21.15.27:0/2459258338 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4f680fde70 con 0x7f4f680f55e0 2021-11-03T18:39:19.464 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.466+0000 7f4f657fa700 1 -- 172.21.15.27:0/2459258338 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f4f600544b0 con 0x7f4f680f55e0 2021-11-03T18:39:19.464 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:39:19.465 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:39:19.465 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:39:19.465 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:39:19.465 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:39:19.466 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:39:19.466 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:39:19.466 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:39:19.466 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:39:19.467 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:39:19.467 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:39:19.467 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:39:19.467 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:39:19.468 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:39:19.468 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:39:19.468 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:39:19.468 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:39:19.468 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:39:19.469 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:39:19.469 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:39:19.469 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:39:19.469 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:39:19.470 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:39:19.470 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:39:19.470 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:39:19.470 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:39:19.470 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:39:19.471 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:39:19.471 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:39:19.471 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:39:19.471 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:39:19.472 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:39:19.472 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:39:19.472 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:39:19.472 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:39:19.473 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:39:19.473 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:39:19.473 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:39:19.473 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:39:19.474 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.467+0000 7f4f6eeac700 1 -- 172.21.15.27:0/2459258338 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4f540640b0 msgr2=0x7f4f54066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:39:19.474 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.467+0000 7f4f6eeac700 1 --2- 172.21.15.27:0/2459258338 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4f540640b0 0x7f4f54066570 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7f4f580049a0 tx=0x7f4f5800b040).stop 2021-11-03T18:39:19.475 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.467+0000 7f4f6eeac700 1 -- 172.21.15.27:0/2459258338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f680f55e0 msgr2=0x7f4f68109bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:39:19.475 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.467+0000 7f4f6eeac700 1 --2- 172.21.15.27:0/2459258338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f680f55e0 0x7f4f68109bf0 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 rx=0x7f4f60009990 tx=0x7f4f60009b30).stop 2021-11-03T18:39:19.475 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.467+0000 7f4f6eeac700 1 -- 172.21.15.27:0/2459258338 shutdown_connections 2021-11-03T18:39:19.475 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.467+0000 7f4f6eeac700 1 --2- 172.21.15.27:0/2459258338 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4f540640b0 0x7f4f54066570 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:19.476 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.468+0000 7f4f6eeac700 1 --2- 172.21.15.27:0/2459258338 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4f680f55e0 0x7f4f68109bf0 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:19.476 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.468+0000 7f4f6eeac700 1 --2- 172.21.15.27:0/2459258338 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4f680f5ef0 0x7f4f6810c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:19.476 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.468+0000 7f4f6eeac700 1 -- 172.21.15.27:0/2459258338 >> 172.21.15.27:0/2459258338 conn(0x7f4f680f0de0 msgr2=0x7f4f680fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:39:19.476 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.468+0000 7f4f6eeac700 1 -- 172.21.15.27:0/2459258338 shutdown_connections 2021-11-03T18:39:19.476 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:19.468+0000 7f4f6eeac700 1 -- 172.21.15.27:0/2459258338 wait complete. 2021-11-03T18:39:19.477 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:39:19.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:19 smithi115 ceph-mon[65453]: from='client.36712 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:39:19.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:19 smithi115 ceph-mon[65453]: from='client.36716 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:39:19.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:19 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1360180817' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:19.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:19 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2459258338' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:39:20.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:19 smithi027 ceph-mon[97244]: from='client.36712 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:39:20.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:19 smithi027 ceph-mon[97244]: from='client.36716 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:39:20.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:19 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1360180817' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:20.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:19 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2459258338' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:39:20.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:20 smithi115 ceph-mon[65453]: pgmap v2542: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:20.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5002.480175 secs 2021-11-03T18:39:20.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:39:20.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:39:20.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:39:20.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:20.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:20.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:20.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:21.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:20 smithi027 ceph-mon[97244]: pgmap v2542: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:21.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5002.480175 secs 2021-11-03T18:39:21.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:39:21.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:39:21.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:39:21.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:21.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:21.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:21.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:21.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:21 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:39:22.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:21 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:39:22.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:22 smithi027 ceph-mon[97244]: pgmap v2543: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:22.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:22 smithi115 ceph-mon[65453]: pgmap v2543: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:24.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:24 smithi115 ceph-mon[65453]: pgmap v2544: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:25.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:24 smithi027 ceph-mon[97244]: pgmap v2544: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:25.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5007.480292 secs 2021-11-03T18:39:26.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5007.480292 secs 2021-11-03T18:39:26.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:26 smithi115 ceph-mon[65453]: pgmap v2545: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:27.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:26 smithi027 ceph-mon[97244]: pgmap v2545: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:28.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:28 smithi115 ceph-mon[65453]: pgmap v2546: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:29.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:28 smithi027 ceph-mon[97244]: pgmap v2546: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:30.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:30 smithi115 ceph-mon[65453]: pgmap v2547: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:30.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5012.480456 secs 2021-11-03T18:39:30.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:39:30.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:39:30.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:39:30.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:30.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:30.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:30.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:31.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:30 smithi027 ceph-mon[97244]: pgmap v2547: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:31.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5012.480456 secs 2021-11-03T18:39:31.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:39:31.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:39:31.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:39:31.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:31.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:31.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:31.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:31.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:31 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:39:32.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:31 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:39:32.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:32 smithi027 ceph-mon[97244]: pgmap v2548: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:32.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:32 smithi115 ceph-mon[65453]: pgmap v2548: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:34.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:34 smithi115 ceph-mon[65453]: pgmap v2549: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:35.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:34 smithi027 ceph-mon[97244]: pgmap v2549: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:35.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5017.480595 secs 2021-11-03T18:39:36.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5017.480595 secs 2021-11-03T18:39:36.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:36 smithi115 ceph-mon[65453]: pgmap v2550: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:37.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:36 smithi027 ceph-mon[97244]: pgmap v2550: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:38.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:38 smithi115 ceph-mon[65453]: pgmap v2551: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:39.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:38 smithi027 ceph-mon[97244]: pgmap v2551: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:40.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:40 smithi115 ceph-mon[65453]: pgmap v2552: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:40.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5022.480727 secs 2021-11-03T18:39:40.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:39:40.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:39:40.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:39:40.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:40.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:40.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:40.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:40.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:40 smithi027 ceph-mon[97244]: pgmap v2552: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:40.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5022.480727 secs 2021-11-03T18:39:40.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:39:40.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:39:40.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:39:40.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:40.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:40.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:40.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:41.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:41 smithi115 ceph-mon[65453]: poll took 60181.798 ms: 1 events 2021-11-03T18:39:41.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:41 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:39:42.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:41 smithi027 ceph-mon[97244]: poll took 60181.798 ms: 1 events 2021-11-03T18:39:42.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:41 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:39:42.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:42 smithi027 ceph-mon[97244]: pgmap v2553: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:42.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:42 smithi115 ceph-mon[65453]: pgmap v2553: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:44.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:44 smithi115 ceph-mon[65453]: pgmap v2554: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:45.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:44 smithi027 ceph-mon[97244]: pgmap v2554: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:45.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5027.480857 secs 2021-11-03T18:39:46.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5027.480857 secs 2021-11-03T18:39:46.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:46 smithi115 ceph-mon[65453]: pgmap v2555: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:47.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:46 smithi027 ceph-mon[97244]: pgmap v2555: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:48.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:48 smithi115 ceph-mon[65453]: pgmap v2556: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:49.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:48 smithi027 ceph-mon[97244]: pgmap v2556: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:49.582 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.582+0000 7f31b759e700 1 -- 172.21.15.27:0/2102963639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31b8060270 msgr2=0x7f31b80f91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:39:49.582 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.582+0000 7f31b759e700 1 --2- 172.21.15.27:0/2102963639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31b8060270 0x7f31b80f91c0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 rx=0x7f31a8004660 tx=0x7f31a8009b30).stop 2021-11-03T18:39:49.582 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.582+0000 7f31b759e700 1 -- 172.21.15.27:0/2102963639 shutdown_connections 2021-11-03T18:39:49.583 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.582+0000 7f31b759e700 1 --2- 172.21.15.27:0/2102963639 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31b80f96b0 0x7f31b80fbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:49.583 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.582+0000 7f31b759e700 1 --2- 172.21.15.27:0/2102963639 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31b8060270 0x7f31b80f91c0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:49.583 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.582+0000 7f31b759e700 1 -- 172.21.15.27:0/2102963639 >> 172.21.15.27:0/2102963639 conn(0x7f31b80f0ba0 msgr2=0x7f31b80f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:39:49.583 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.582+0000 7f31b759e700 1 -- 172.21.15.27:0/2102963639 shutdown_connections 2021-11-03T18:39:49.584 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.582+0000 7f31b759e700 1 -- 172.21.15.27:0/2102963639 wait complete. 2021-11-03T18:39:49.584 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.582+0000 7f31b759e700 1 Processor -- start 2021-11-03T18:39:49.584 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.583+0000 7f31b759e700 1 -- start start 2021-11-03T18:39:49.584 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.583+0000 7f31b759e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31b8060270 0x7f31b8106e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:39:49.584 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.583+0000 7f31b759e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31b80f96b0 0x7f31b8107310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:39:49.585 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.583+0000 7f31b759e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b8107910 con 0x7f31b8060270 2021-11-03T18:39:49.585 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.583+0000 7f31b759e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31b8108120 con 0x7f31b80f96b0 2021-11-03T18:39:49.585 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.583+0000 7f31b659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31b8060270 0x7f31b8106e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:39:49.585 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.583+0000 7f31b659c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31b8060270 0x7f31b8106e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36134/0 (socket says 172.21.15.27:36134) 2021-11-03T18:39:49.586 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.583+0000 7f31b659c700 1 -- 172.21.15.27:0/1453749173 learned_addr learned my addr 172.21.15.27:0/1453749173 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:39:49.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.583+0000 7f31b5d9b700 1 --2- 172.21.15.27:0/1453749173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31b80f96b0 0x7f31b8107310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:39:49.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.583+0000 7f31b659c700 1 -- 172.21.15.27:0/1453749173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31b80f96b0 msgr2=0x7f31b8107310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:39:49.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.583+0000 7f31b659c700 1 --2- 172.21.15.27:0/1453749173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31b80f96b0 0x7f31b8107310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:49.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.583+0000 7f31b659c700 1 -- 172.21.15.27:0/1453749173 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a8005040 con 0x7f31b8060270 2021-11-03T18:39:49.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.583+0000 7f31b659c700 1 --2- 172.21.15.27:0/1453749173 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31b8060270 0x7f31b8106e20 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 rx=0x7f31a8009990 tx=0x7f31a8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:39:49.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.583+0000 7f31a6ffd700 1 -- 172.21.15.27:0/1453749173 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31a8018070 con 0x7f31b8060270 2021-11-03T18:39:49.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.584+0000 7f31b759e700 1 -- 172.21.15.27:0/1453749173 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31b8108340 con 0x7f31b8060270 2021-11-03T18:39:49.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.584+0000 7f31a6ffd700 1 -- 172.21.15.27:0/1453749173 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f31a8007ba0 con 0x7f31b8060270 2021-11-03T18:39:49.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.584+0000 7f31a6ffd700 1 -- 172.21.15.27:0/1453749173 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f31a801c3c0 con 0x7f31b8060270 2021-11-03T18:39:49.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.584+0000 7f31b759e700 1 -- 172.21.15.27:0/1453749173 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31b81088a0 con 0x7f31b8060270 2021-11-03T18:39:49.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.584+0000 7f31b759e700 1 -- 172.21.15.27:0/1453749173 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31b8059100 con 0x7f31b8060270 2021-11-03T18:39:49.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.585+0000 7f31a6ffd700 1 -- 172.21.15.27:0/1453749173 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f31a801d420 con 0x7f31b8060270 2021-11-03T18:39:49.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.585+0000 7f31a6ffd700 1 --2- 172.21.15.27:0/1453749173 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f31a0064100 0x7f31a00665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:39:49.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.585+0000 7f31a6ffd700 1 -- 172.21.15.27:0/1453749173 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f31a80825d0 con 0x7f31b8060270 2021-11-03T18:39:49.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.585+0000 7f31b5d9b700 1 --2- 172.21.15.27:0/1453749173 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f31a0064100 0x7f31a00665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:39:49.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.586+0000 7f31b5d9b700 1 --2- 172.21.15.27:0/1453749173 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f31a0064100 0x7f31a00665c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7f31b8060040 tx=0x7f31ac009be0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:39:49.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.590+0000 7f31a6ffd700 1 -- 172.21.15.27:0/1453749173 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f31a80549f0 con 0x7f31b8060270 2021-11-03T18:39:49.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.770+0000 7f31b759e700 1 -- 172.21.15.27:0/1453749173 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f31b80008d0 con 0x7f31a0064100 2021-11-03T18:39:49.770 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.771+0000 7f31a6ffd700 1 -- 172.21.15.27:0/1453749173 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f31b80008d0 con 0x7f31a0064100 2021-11-03T18:39:49.772 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.773+0000 7f31b759e700 1 -- 172.21.15.27:0/1453749173 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f31a0064100 msgr2=0x7f31a00665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:39:49.773 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.773+0000 7f31b759e700 1 --2- 172.21.15.27:0/1453749173 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f31a0064100 0x7f31a00665c0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 rx=0x7f31b8060040 tx=0x7f31ac009be0).stop 2021-11-03T18:39:49.773 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.773+0000 7f31b759e700 1 -- 172.21.15.27:0/1453749173 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31b8060270 msgr2=0x7f31b8106e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:39:49.773 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.773+0000 7f31b759e700 1 --2- 172.21.15.27:0/1453749173 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31b8060270 0x7f31b8106e20 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 rx=0x7f31a8009990 tx=0x7f31a8009b30).stop 2021-11-03T18:39:49.773 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.773+0000 7f31b759e700 1 -- 172.21.15.27:0/1453749173 shutdown_connections 2021-11-03T18:39:49.774 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.773+0000 7f31b759e700 1 --2- 172.21.15.27:0/1453749173 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f31a0064100 0x7f31a00665c0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:49.774 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.773+0000 7f31b759e700 1 --2- 172.21.15.27:0/1453749173 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f31b8060270 0x7f31b8106e20 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:49.774 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.773+0000 7f31b759e700 1 --2- 172.21.15.27:0/1453749173 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f31b80f96b0 0x7f31b8107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:49.774 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.773+0000 7f31b759e700 1 -- 172.21.15.27:0/1453749173 >> 172.21.15.27:0/1453749173 conn(0x7f31b80f0ba0 msgr2=0x7f31b80f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:39:49.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.773+0000 7f31b759e700 1 -- 172.21.15.27:0/1453749173 shutdown_connections 2021-11-03T18:39:49.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.773+0000 7f31b759e700 1 -- 172.21.15.27:0/1453749173 wait complete. 2021-11-03T18:39:49.782 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:39:49.874 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.873+0000 7f3ab501d700 1 -- 172.21.15.27:0/1466750576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ab00f5ed0 msgr2=0x7f3ab00f6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:39:49.874 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.873+0000 7f3ab501d700 1 --2- 172.21.15.27:0/1466750576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ab00f5ed0 0x7f3ab00f6900 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 rx=0x7f3aa0004660 tx=0x7f3aa0009b30).stop 2021-11-03T18:39:49.874 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.873+0000 7f3ab501d700 1 -- 172.21.15.27:0/1466750576 shutdown_connections 2021-11-03T18:39:49.875 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.873+0000 7f3ab501d700 1 --2- 172.21.15.27:0/1466750576 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ab00f5ed0 0x7f3ab00f6900 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:49.875 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.873+0000 7f3ab501d700 1 --2- 172.21.15.27:0/1466750576 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ab00f55c0 0x7f3ab00f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:49.875 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.873+0000 7f3ab501d700 1 -- 172.21.15.27:0/1466750576 >> 172.21.15.27:0/1466750576 conn(0x7f3ab00f0dc0 msgr2=0x7f3ab00f31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:39:49.875 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.873+0000 7f3ab501d700 1 -- 172.21.15.27:0/1466750576 shutdown_connections 2021-11-03T18:39:49.875 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.873+0000 7f3ab501d700 1 -- 172.21.15.27:0/1466750576 wait complete. 2021-11-03T18:39:49.876 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.874+0000 7f3ab501d700 1 Processor -- start 2021-11-03T18:39:49.876 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.874+0000 7f3ab501d700 1 -- start start 2021-11-03T18:39:49.876 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.874+0000 7f3ab501d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ab00f55c0 0x7f3ab0109c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:39:49.876 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.874+0000 7f3ab501d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ab00f5ed0 0x7f3ab010c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:39:49.877 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.874+0000 7f3ab501d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab0060560 con 0x7f3ab00f5ed0 2021-11-03T18:39:49.877 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.874+0000 7f3ab501d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ab00606a0 con 0x7f3ab00f55c0 2021-11-03T18:39:49.877 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.874+0000 7f3aae59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ab00f5ed0 0x7f3ab010c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:39:49.877 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.874+0000 7f3aae59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ab00f5ed0 0x7f3ab010c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36146/0 (socket says 172.21.15.27:36146) 2021-11-03T18:39:49.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.874+0000 7f3aae59c700 1 -- 172.21.15.27:0/2104010130 learned_addr learned my addr 172.21.15.27:0/2104010130 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:39:49.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.874+0000 7f3aaed9d700 1 --2- 172.21.15.27:0/2104010130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ab00f55c0 0x7f3ab0109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:39:49.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.874+0000 7f3aae59c700 1 -- 172.21.15.27:0/2104010130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ab00f55c0 msgr2=0x7f3ab0109c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:39:49.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.874+0000 7f3aae59c700 1 --2- 172.21.15.27:0/2104010130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ab00f55c0 0x7f3ab0109c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:49.879 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.874+0000 7f3aae59c700 1 -- 172.21.15.27:0/2104010130 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3aa0005040 con 0x7f3ab00f5ed0 2021-11-03T18:39:49.879 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.874+0000 7f3aae59c700 1 --2- 172.21.15.27:0/2104010130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ab00f5ed0 0x7f3ab010c170 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 rx=0x7f3aa0009990 tx=0x7f3aa0009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:39:49.879 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.875+0000 7f3a977fe700 1 -- 172.21.15.27:0/2104010130 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3aa0018070 con 0x7f3ab00f5ed0 2021-11-03T18:39:49.879 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.875+0000 7f3a977fe700 1 -- 172.21.15.27:0/2104010130 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f3aa0007ba0 con 0x7f3ab00f5ed0 2021-11-03T18:39:49.880 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.875+0000 7f3ab501d700 1 -- 172.21.15.27:0/2104010130 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ab010c6c0 con 0x7f3ab00f5ed0 2021-11-03T18:39:49.880 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.875+0000 7f3a977fe700 1 -- 172.21.15.27:0/2104010130 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3aa001c3c0 con 0x7f3ab00f5ed0 2021-11-03T18:39:49.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.875+0000 7f3ab501d700 1 -- 172.21.15.27:0/2104010130 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ab010cc20 con 0x7f3ab00f5ed0 2021-11-03T18:39:49.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.876+0000 7f3ab501d700 1 -- 172.21.15.27:0/2104010130 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ab0045ad0 con 0x7f3ab00f5ed0 2021-11-03T18:39:49.882 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.876+0000 7f3a977fe700 1 -- 172.21.15.27:0/2104010130 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f3aa001d420 con 0x7f3ab00f5ed0 2021-11-03T18:39:49.882 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.876+0000 7f3a977fe700 1 --2- 172.21.15.27:0/2104010130 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3a980640b0 0x7f3a98066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:39:49.882 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.877+0000 7f3a977fe700 1 -- 172.21.15.27:0/2104010130 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f3aa0082500 con 0x7f3ab00f5ed0 2021-11-03T18:39:49.882 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.877+0000 7f3aaed9d700 1 --2- 172.21.15.27:0/2104010130 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3a980640b0 0x7f3a98066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:39:49.883 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.877+0000 7f3aaed9d700 1 --2- 172.21.15.27:0/2104010130 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3a980640b0 0x7f3a98066570 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 rx=0x7f3ab00601e0 tx=0x7f3aa4009be0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:39:49.883 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:49.881+0000 7f3a977fe700 1 -- 172.21.15.27:0/2104010130 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f3aa00548c0 con 0x7f3ab00f5ed0 2021-11-03T18:39:50.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.054+0000 7f3ab501d700 1 -- 172.21.15.27:0/2104010130 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f3ab00fdec0 con 0x7f3a980640b0 2021-11-03T18:39:50.068 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.069+0000 7f3a977fe700 1 -- 172.21.15.27:0/2104010130 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f3ab00fdec0 con 0x7f3a980640b0 2021-11-03T18:39:50.068 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:39:50.069 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (91m) 10m ago 92m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:39:50.069 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (83m) 10m ago 92m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:39:50.069 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (82m) 9m ago 91m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:39:50.069 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (91m) 10m ago 92m 42.7M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:39:50.069 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (84m) 10m ago 87m 26.7M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:39:50.070 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (87m) 10m ago 87m 3850M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:39:50.070 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (83m) 9m ago 87m 26.3M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:39:50.070 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (85m) 9m ago 87m 27.4M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:39:50.070 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (85m) 10m ago 93m 608M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:39:50.071 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (84m) 9m ago 91m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:39:50.071 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (84m) 10m ago 94m 350M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:39:50.071 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (83m) 9m ago 91m 160M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:39:50.071 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (92m) 10m ago 92m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:39:50.071 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (91m) 9m ago 91m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:39:50.072 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (82m) 10m ago 91m 413M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:39:50.072 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (82m) 10m ago 90m 450M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:39:50.072 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (81m) 10m ago 90m 338M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:39:50.072 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (81m) 9m ago 90m 479M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:39:50.073 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (80m) 9m ago 89m 378M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:39:50.073 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (80m) 9m ago 89m 388M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:39:50.073 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (91m) 10m ago 92m 64.4M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:39:50.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.071+0000 7f3ab501d700 1 -- 172.21.15.27:0/2104010130 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3a980640b0 msgr2=0x7f3a98066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:39:50.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.071+0000 7f3ab501d700 1 --2- 172.21.15.27:0/2104010130 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3a980640b0 0x7f3a98066570 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 rx=0x7f3ab00601e0 tx=0x7f3aa4009be0).stop 2021-11-03T18:39:50.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.071+0000 7f3ab501d700 1 -- 172.21.15.27:0/2104010130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ab00f5ed0 msgr2=0x7f3ab010c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:39:50.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.071+0000 7f3ab501d700 1 --2- 172.21.15.27:0/2104010130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ab00f5ed0 0x7f3ab010c170 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 rx=0x7f3aa0009990 tx=0x7f3aa0009b30).stop 2021-11-03T18:39:50.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.071+0000 7f3ab501d700 1 -- 172.21.15.27:0/2104010130 shutdown_connections 2021-11-03T18:39:50.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.071+0000 7f3ab501d700 1 --2- 172.21.15.27:0/2104010130 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3a980640b0 0x7f3a98066570 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:50.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.071+0000 7f3ab501d700 1 --2- 172.21.15.27:0/2104010130 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3ab00f55c0 0x7f3ab0109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:50.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.071+0000 7f3ab501d700 1 --2- 172.21.15.27:0/2104010130 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3ab00f5ed0 0x7f3ab010c170 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:50.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.071+0000 7f3ab501d700 1 -- 172.21.15.27:0/2104010130 >> 172.21.15.27:0/2104010130 conn(0x7f3ab00f0dc0 msgr2=0x7f3ab00fa350 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:39:50.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.071+0000 7f3ab501d700 1 -- 172.21.15.27:0/2104010130 shutdown_connections 2021-11-03T18:39:50.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.071+0000 7f3ab501d700 1 -- 172.21.15.27:0/2104010130 wait complete. 2021-11-03T18:39:50.171 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.170+0000 7fb2a0997700 1 -- 172.21.15.27:0/4142888535 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb29c0f49a0 msgr2=0x7fb29c0f4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:39:50.171 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.170+0000 7fb2a0997700 1 --2- 172.21.15.27:0/4142888535 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb29c0f49a0 0x7fb29c0f4dc0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 rx=0x7fb2940099e0 tx=0x7fb294004d10).stop 2021-11-03T18:39:50.171 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.170+0000 7fb2a0997700 1 -- 172.21.15.27:0/4142888535 shutdown_connections 2021-11-03T18:39:50.172 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.170+0000 7fb2a0997700 1 --2- 172.21.15.27:0/4142888535 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb29c0f5340 0x7fb29c0f7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:50.172 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.170+0000 7fb2a0997700 1 --2- 172.21.15.27:0/4142888535 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb29c0f49a0 0x7fb29c0f4dc0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:50.172 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.170+0000 7fb2a0997700 1 -- 172.21.15.27:0/4142888535 >> 172.21.15.27:0/4142888535 conn(0x7fb29c0f04c0 msgr2=0x7fb29c0f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:39:50.172 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.170+0000 7fb2a0997700 1 -- 172.21.15.27:0/4142888535 shutdown_connections 2021-11-03T18:39:50.172 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.170+0000 7fb2a0997700 1 -- 172.21.15.27:0/4142888535 wait complete. 2021-11-03T18:39:50.173 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.171+0000 7fb2a0997700 1 Processor -- start 2021-11-03T18:39:50.173 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.171+0000 7fb2a0997700 1 -- start start 2021-11-03T18:39:50.173 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.171+0000 7fb2a0997700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb29c0f49a0 0x7fb29c0f7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:39:50.173 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.171+0000 7fb2a0997700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb29c0f5340 0x7fb29c0f59d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:39:50.174 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.171+0000 7fb2a0997700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c0f5f30 con 0x7fb29c0f49a0 2021-11-03T18:39:50.174 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.171+0000 7fb2a0997700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c0f6070 con 0x7fb29c0f5340 2021-11-03T18:39:50.174 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.171+0000 7fb29a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb29c0f49a0 0x7fb29c0f7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:39:50.174 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.171+0000 7fb29a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb29c0f49a0 0x7fb29c0f7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36154/0 (socket says 172.21.15.27:36154) 2021-11-03T18:39:50.175 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.171+0000 7fb29a59c700 1 -- 172.21.15.27:0/37681770 learned_addr learned my addr 172.21.15.27:0/37681770 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:39:50.175 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.171+0000 7fb299d9b700 1 --2- 172.21.15.27:0/37681770 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb29c0f5340 0x7fb29c0f59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:39:50.175 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.171+0000 7fb29a59c700 1 -- 172.21.15.27:0/37681770 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb29c0f5340 msgr2=0x7fb29c0f59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:39:50.175 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.171+0000 7fb29a59c700 1 --2- 172.21.15.27:0/37681770 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb29c0f5340 0x7fb29c0f59d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:50.175 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.171+0000 7fb29a59c700 1 -- 172.21.15.27:0/37681770 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb294009640 con 0x7fb29c0f49a0 2021-11-03T18:39:50.176 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.171+0000 7fb29a59c700 1 --2- 172.21.15.27:0/37681770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb29c0f49a0 0x7fb29c0f7550 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 rx=0x7fb29400baa0 tx=0x7fb294004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:39:50.176 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.171+0000 7fb292ffd700 1 -- 172.21.15.27:0/37681770 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb29400e070 con 0x7fb29c0f49a0 2021-11-03T18:39:50.176 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.171+0000 7fb292ffd700 1 -- 172.21.15.27:0/37681770 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fb294007df0 con 0x7fb29c0f49a0 2021-11-03T18:39:50.176 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.171+0000 7fb2a0997700 1 -- 172.21.15.27:0/37681770 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb29c0f62f0 con 0x7fb29c0f49a0 2021-11-03T18:39:50.177 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.172+0000 7fb292ffd700 1 -- 172.21.15.27:0/37681770 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb294018070 con 0x7fb29c0f49a0 2021-11-03T18:39:50.177 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.172+0000 7fb2a0997700 1 -- 172.21.15.27:0/37681770 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb29c0f6850 con 0x7fb29c0f49a0 2021-11-03T18:39:50.178 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.172+0000 7fb2a0997700 1 -- 172.21.15.27:0/37681770 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb29c045ad0 con 0x7fb29c0f49a0 2021-11-03T18:39:50.178 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.173+0000 7fb292ffd700 1 -- 172.21.15.27:0/37681770 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fb294024060 con 0x7fb29c0f49a0 2021-11-03T18:39:50.178 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.173+0000 7fb292ffd700 1 --2- 172.21.15.27:0/37681770 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb2880640b0 0x7fb288066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:39:50.179 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.173+0000 7fb292ffd700 1 -- 172.21.15.27:0/37681770 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fb294081f30 con 0x7fb29c0f49a0 2021-11-03T18:39:50.179 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.173+0000 7fb299d9b700 1 --2- 172.21.15.27:0/37681770 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb2880640b0 0x7fb288066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:39:50.179 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.174+0000 7fb299d9b700 1 --2- 172.21.15.27:0/37681770 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb2880640b0 0x7fb288066570 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0x7fb29c0f5780 tx=0x7fb28400b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:39:50.179 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.177+0000 7fb292ffd700 1 -- 172.21.15.27:0/37681770 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fb2940aedf0 con 0x7fb29c0f49a0 2021-11-03T18:39:50.405 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.406+0000 7fb2a0997700 1 -- 172.21.15.27:0/37681770 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb29c0586e0 con 0x7fb29c0f49a0 2021-11-03T18:39:50.405 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.406+0000 7fb292ffd700 1 -- 172.21.15.27:0/37681770 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fb2940222e0 con 0x7fb29c0f49a0 2021-11-03T18:39:50.406 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:39:50.406 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:39:50.406 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:39:50.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:39:50.407 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:39:50.407 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:39:50.407 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:39:50.408 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:39:50.408 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:39:50.408 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:39:50.408 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:39:50.408 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:39:50.409 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:39:50.409 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:39:50.409 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:39:50.409 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:39:50.410 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:39:50.410 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:39:50.410 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.408+0000 7fb2a0997700 1 -- 172.21.15.27:0/37681770 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb2880640b0 msgr2=0x7fb288066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:39:50.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.408+0000 7fb2a0997700 1 --2- 172.21.15.27:0/37681770 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb2880640b0 0x7fb288066570 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0x7fb29c0f5780 tx=0x7fb28400b040).stop 2021-11-03T18:39:50.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.408+0000 7fb2a0997700 1 -- 172.21.15.27:0/37681770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb29c0f49a0 msgr2=0x7fb29c0f7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:39:50.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.408+0000 7fb2a0997700 1 --2- 172.21.15.27:0/37681770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb29c0f49a0 0x7fb29c0f7550 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 rx=0x7fb29400baa0 tx=0x7fb294004d10).stop 2021-11-03T18:39:50.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.408+0000 7fb2a0997700 1 -- 172.21.15.27:0/37681770 shutdown_connections 2021-11-03T18:39:50.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.408+0000 7fb2a0997700 1 --2- 172.21.15.27:0/37681770 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb2880640b0 0x7fb288066570 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:50.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.408+0000 7fb2a0997700 1 --2- 172.21.15.27:0/37681770 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb29c0f49a0 0x7fb29c0f7550 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:50.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.408+0000 7fb2a0997700 1 --2- 172.21.15.27:0/37681770 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb29c0f5340 0x7fb29c0f59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:50.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.408+0000 7fb2a0997700 1 -- 172.21.15.27:0/37681770 >> 172.21.15.27:0/37681770 conn(0x7fb29c0f04c0 msgr2=0x7fb29c0fb510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:39:50.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.408+0000 7fb2a0997700 1 -- 172.21.15.27:0/37681770 shutdown_connections 2021-11-03T18:39:50.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.408+0000 7fb2a0997700 1 -- 172.21.15.27:0/37681770 wait complete. 2021-11-03T18:39:50.508 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.509+0000 7fb02999f700 1 -- 172.21.15.27:0/68675306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0240f77d0 msgr2=0x7fb0240f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:39:50.508 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.509+0000 7fb02999f700 1 --2- 172.21.15.27:0/68675306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0240f77d0 0x7fb0240f7c30 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 rx=0x7fb010004660 tx=0x7fb010009b30).stop 2021-11-03T18:39:50.509 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.509+0000 7fb02999f700 1 -- 172.21.15.27:0/68675306 shutdown_connections 2021-11-03T18:39:50.509 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.509+0000 7fb02999f700 1 --2- 172.21.15.27:0/68675306 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0240f77d0 0x7fb0240f7c30 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:50.509 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.509+0000 7fb02999f700 1 --2- 172.21.15.27:0/68675306 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0240f6530 0x7fb0240f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:50.509 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.509+0000 7fb02999f700 1 -- 172.21.15.27:0/68675306 >> 172.21.15.27:0/68675306 conn(0x7fb0240f1c90 msgr2=0x7fb0240f40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:39:50.510 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.509+0000 7fb02999f700 1 -- 172.21.15.27:0/68675306 shutdown_connections 2021-11-03T18:39:50.510 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.509+0000 7fb02999f700 1 -- 172.21.15.27:0/68675306 wait complete. 2021-11-03T18:39:50.510 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.509+0000 7fb02999f700 1 Processor -- start 2021-11-03T18:39:50.510 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.509+0000 7fb02999f700 1 -- start start 2021-11-03T18:39:50.513 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.510+0000 7fb02999f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0240f6530 0x7fb024106d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:39:50.513 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.510+0000 7fb02999f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0240f77d0 0x7fb024107240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:39:50.513 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.510+0000 7fb02999f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0241077f0 con 0x7fb0240f6530 2021-11-03T18:39:50.513 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.510+0000 7fb02999f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb024107ff0 con 0x7fb0240f77d0 2021-11-03T18:39:50.514 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.510+0000 7fb02899d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0240f6530 0x7fb024106d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:39:50.514 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.510+0000 7fb02899d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0240f6530 0x7fb024106d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36164/0 (socket says 172.21.15.27:36164) 2021-11-03T18:39:50.514 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.510+0000 7fb02899d700 1 -- 172.21.15.27:0/393720867 learned_addr learned my addr 172.21.15.27:0/393720867 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:39:50.514 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.510+0000 7fb023fff700 1 --2- 172.21.15.27:0/393720867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0240f77d0 0x7fb024107240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:39:50.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.510+0000 7fb02899d700 1 -- 172.21.15.27:0/393720867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0240f77d0 msgr2=0x7fb024107240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:39:50.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.510+0000 7fb02899d700 1 --2- 172.21.15.27:0/393720867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0240f77d0 0x7fb024107240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:50.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.510+0000 7fb02899d700 1 -- 172.21.15.27:0/393720867 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb010005040 con 0x7fb0240f6530 2021-11-03T18:39:50.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.510+0000 7fb02899d700 1 --2- 172.21.15.27:0/393720867 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0240f6530 0x7fb024106d50 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 rx=0x7fb018002700 tx=0x7fb018009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:39:50.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.510+0000 7fb0217fa700 1 -- 172.21.15.27:0/393720867 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0180079c0 con 0x7fb0240f6530 2021-11-03T18:39:50.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.510+0000 7fb0217fa700 1 -- 172.21.15.27:0/393720867 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fb01800f040 con 0x7fb0240f6530 2021-11-03T18:39:50.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.510+0000 7fb02999f700 1 -- 172.21.15.27:0/393720867 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0241082d0 con 0x7fb0240f6530 2021-11-03T18:39:50.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.510+0000 7fb0217fa700 1 -- 172.21.15.27:0/393720867 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb018014860 con 0x7fb0240f6530 2021-11-03T18:39:50.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.510+0000 7fb02999f700 1 -- 172.21.15.27:0/393720867 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb024108890 con 0x7fb0240f6530 2021-11-03T18:39:50.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.511+0000 7fb02999f700 1 -- 172.21.15.27:0/393720867 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb024059100 con 0x7fb0240f6530 2021-11-03T18:39:50.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.512+0000 7fb0217fa700 1 -- 172.21.15.27:0/393720867 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fb01801b060 con 0x7fb0240f6530 2021-11-03T18:39:50.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.512+0000 7fb0217fa700 1 --2- 172.21.15.27:0/393720867 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb014064100 0x7fb0140665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:39:50.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.512+0000 7fb0217fa700 1 -- 172.21.15.27:0/393720867 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fb018080660 con 0x7fb0240f6530 2021-11-03T18:39:50.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.512+0000 7fb023fff700 1 --2- 172.21.15.27:0/393720867 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb014064100 0x7fb0140665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:39:50.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.513+0000 7fb023fff700 1 --2- 172.21.15.27:0/393720867 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb014064100 0x7fb0140665c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7fb010004630 tx=0x7fb010009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:39:50.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.516+0000 7fb0217fa700 1 -- 172.21.15.27:0/393720867 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fb018052900 con 0x7fb0240f6530 2021-11-03T18:39:50.721 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.722+0000 7fb02999f700 1 -- 172.21.15.27:0/393720867 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fb024027a00 con 0x7fb0240f6530 2021-11-03T18:39:50.722 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.722+0000 7fb0217fa700 1 -- 172.21.15.27:0/393720867 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7fb018019070 con 0x7fb0240f6530 2021-11-03T18:39:50.722 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:39:50.722 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:39:50.723 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:39:50.723 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:39:50.723 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:39:50.723 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:39:50.724 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:39:50.724 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:39:50.724 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:39:50.724 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:39:50.724 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:39:50.725 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:39:50.725 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:39:50.725 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:39:50.725 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:39:50.726 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:39:50.726 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:39:50.726 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:39:50.726 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:39:50.727 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:39:50.727 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:39:50.727 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:39:50.727 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:39:50.727 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:39:50.728 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:39:50.728 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:39:50.728 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:39:50.728 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:39:50.729 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:39:50.729 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:39:50.729 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:39:50.729 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:39:50.729 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:39:50.730 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:39:50.730 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:39:50.730 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:39:50.730 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:39:50.731 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:39:50.731 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:39:50.732 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.725+0000 7fb02999f700 1 -- 172.21.15.27:0/393720867 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb014064100 msgr2=0x7fb0140665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:39:50.732 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.725+0000 7fb02999f700 1 --2- 172.21.15.27:0/393720867 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb014064100 0x7fb0140665c0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7fb010004630 tx=0x7fb010009b30).stop 2021-11-03T18:39:50.732 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.725+0000 7fb02999f700 1 -- 172.21.15.27:0/393720867 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0240f6530 msgr2=0x7fb024106d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:39:50.733 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.725+0000 7fb02999f700 1 --2- 172.21.15.27:0/393720867 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0240f6530 0x7fb024106d50 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 rx=0x7fb018002700 tx=0x7fb018009d80).stop 2021-11-03T18:39:50.733 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.725+0000 7fb02999f700 1 -- 172.21.15.27:0/393720867 shutdown_connections 2021-11-03T18:39:50.733 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.725+0000 7fb02999f700 1 --2- 172.21.15.27:0/393720867 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb014064100 0x7fb0140665c0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:50.733 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.725+0000 7fb02999f700 1 --2- 172.21.15.27:0/393720867 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb0240f6530 0x7fb024106d50 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:50.733 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.725+0000 7fb02999f700 1 --2- 172.21.15.27:0/393720867 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb0240f77d0 0x7fb024107240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:39:50.734 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.725+0000 7fb02999f700 1 -- 172.21.15.27:0/393720867 >> 172.21.15.27:0/393720867 conn(0x7fb0240f1c90 msgr2=0x7fb0240fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:39:50.734 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.725+0000 7fb02999f700 1 -- 172.21.15.27:0/393720867 shutdown_connections 2021-11-03T18:39:50.734 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:39:50.725+0000 7fb02999f700 1 -- 172.21.15.27:0/393720867 wait complete. 2021-11-03T18:39:50.734 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:39:50.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:50 smithi027 ceph-mon[97244]: pgmap v2557: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:50.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5032.481003 secs 2021-11-03T18:39:50.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:50 smithi027 ceph-mon[97244]: from='client.36728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:39:50.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:50 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/37681770' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:50.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:39:50.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:50 smithi115 ceph-mon[65453]: pgmap v2557: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:50.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5032.481003 secs 2021-11-03T18:39:50.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:50 smithi115 ceph-mon[65453]: from='client.36728 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:39:50.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:50 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/37681770' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:39:50.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:39:51.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:51 smithi115 ceph-mon[65453]: from='client.36732 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:39:51.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:51 smithi115 ceph-mon[65453]: poll took 10026.077 ms: 1 events 2021-11-03T18:39:51.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:51 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/393720867' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:39:52.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:51 smithi027 ceph-mon[97244]: from='client.36732 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:39:52.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:51 smithi027 ceph-mon[97244]: poll took 10026.077 ms: 1 events 2021-11-03T18:39:52.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:51 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/393720867' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:39:52.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:52 smithi115 ceph-mon[65453]: pgmap v2558: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:53.021 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:52 smithi027 ceph-mon[97244]: pgmap v2558: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:54.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:54 smithi115 ceph-mon[65453]: pgmap v2559: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:55.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:54 smithi027 ceph-mon[97244]: pgmap v2559: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:55.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5037.481087 secs 2021-11-03T18:39:55.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:39:56.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5037.481087 secs 2021-11-03T18:39:56.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:39:56.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:56 smithi027 ceph-mon[97244]: pgmap v2560: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:56.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:56 smithi027 ceph-mon[97244]: poll took 5021.810 ms: 1 events 2021-11-03T18:39:56.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:39:56.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:39:56.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:56 smithi115 ceph-mon[65453]: pgmap v2560: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:56.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:56 smithi115 ceph-mon[65453]: poll took 5021.810 ms: 1 events 2021-11-03T18:39:56.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:39:56.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:39:58.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:58 smithi115 ceph-mon[65453]: pgmap v2561: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:39:59.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:58 smithi027 ceph-mon[97244]: pgmap v2561: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:00.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:59 smithi115 ceph-mon[65453]: poll took 2876.687 ms: 1 events 2021-11-03T18:40:00.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:40:00.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:40:00.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:40:00.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:00.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:00.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:00.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:00.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:39:59 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:40:00.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:59 smithi027 ceph-mon[97244]: poll took 2876.687 ms: 1 events 2021-11-03T18:40:00.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:40:00.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:40:00.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:40:00.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:00.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:00.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:00.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:00.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:39:59 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:40:01.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:00 smithi115 ceph-mon[65453]: pgmap v2562: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:01.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5042.481178 secs 2021-11-03T18:40:01.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:00 smithi115 ceph-mon[65453]: overall HEALTH_WARN 1 filesystem is degraded; 1 filesystem has a failed mds daemon; 1 MDSs report slow requests; 3 daemons have recently crashed 2021-11-03T18:40:01.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:00 smithi027 ceph-mon[97244]: pgmap v2562: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:01.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5042.481178 secs 2021-11-03T18:40:01.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:00 smithi027 ceph-mon[97244]: overall HEALTH_WARN 1 filesystem is degraded; 1 filesystem has a failed mds daemon; 1 MDSs report slow requests; 3 daemons have recently crashed 2021-11-03T18:40:03.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:02 smithi115 ceph-mon[65453]: pgmap v2563: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:03.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:02 smithi027 ceph-mon[97244]: pgmap v2563: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:05.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:04 smithi115 ceph-mon[65453]: pgmap v2564: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:05.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:04 smithi027 ceph-mon[97244]: pgmap v2564: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:06.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5047.481256 secs 2021-11-03T18:40:06.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5047.481256 secs 2021-11-03T18:40:07.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:06 smithi115 ceph-mon[65453]: pgmap v2565: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:07.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:06 smithi027 ceph-mon[97244]: pgmap v2565: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:09.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:08 smithi115 ceph-mon[65453]: pgmap v2566: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:09.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:40:09.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:08 smithi027 ceph-mon[97244]: pgmap v2566: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:09.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:40:10.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:40:10.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:40:10.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:10.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:10.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:10.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:10.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:09 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:40:10.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:40:10.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:40:10.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:10.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:10.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:10.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:10.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:09 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:40:11.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:10 smithi115 ceph-mon[65453]: pgmap v2567: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:11.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5052.481396 secs 2021-11-03T18:40:11.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:10 smithi027 ceph-mon[97244]: pgmap v2567: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:11.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5052.481396 secs 2021-11-03T18:40:13.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:12 smithi115 ceph-mon[65453]: pgmap v2568: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:13.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:12 smithi027 ceph-mon[97244]: pgmap v2568: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:15.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:14 smithi115 ceph-mon[65453]: pgmap v2569: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:15.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:14 smithi027 ceph-mon[97244]: pgmap v2569: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:16.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5057.481517 secs 2021-11-03T18:40:16.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5057.481517 secs 2021-11-03T18:40:17.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:16 smithi115 ceph-mon[65453]: pgmap v2570: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:17.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:16 smithi027 ceph-mon[97244]: pgmap v2570: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:18.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:18 smithi115 ceph-mon[65453]: pgmap v2571: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:18.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:40:19.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:18 smithi027 ceph-mon[97244]: pgmap v2571: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:19.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:40:20.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:19 smithi115 ceph-mon[65453]: poll took 20057.721 ms: 1 events 2021-11-03T18:40:20.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:19 smithi027 ceph-mon[97244]: poll took 20057.721 ms: 1 events 2021-11-03T18:40:20.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:20.842+0000 7fac8e1e9700 1 -- 172.21.15.27:0/2998949928 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac880f6ff0 msgr2=0x7fac880f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:40:20.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:20.842+0000 7fac8e1e9700 1 --2- 172.21.15.27:0/2998949928 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac880f6ff0 0x7fac880f7a20 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 rx=0x7fac70004660 tx=0x7fac70009b30).stop 2021-11-03T18:40:20.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:20.842+0000 7fac8e1e9700 1 -- 172.21.15.27:0/2998949928 shutdown_connections 2021-11-03T18:40:20.843 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:20.842+0000 7fac8e1e9700 1 --2- 172.21.15.27:0/2998949928 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac880f6ff0 0x7fac880f7a20 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:20.843 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:20.842+0000 7fac8e1e9700 1 --2- 172.21.15.27:0/2998949928 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac880f66e0 0x7fac880f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:20.843 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:20.842+0000 7fac8e1e9700 1 -- 172.21.15.27:0/2998949928 >> 172.21.15.27:0/2998949928 conn(0x7fac880f1f00 msgr2=0x7fac880f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:40:20.843 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:20.842+0000 7fac8e1e9700 1 -- 172.21.15.27:0/2998949928 shutdown_connections 2021-11-03T18:40:20.844 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:20.842+0000 7fac8e1e9700 1 -- 172.21.15.27:0/2998949928 wait complete. 2021-11-03T18:40:20.844 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:20.843+0000 7fac8e1e9700 1 Processor -- start 2021-11-03T18:40:20.846 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:20.843+0000 7fac8e1e9700 1 -- start start 2021-11-03T18:40:20.846 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:20.843+0000 7fac8e1e9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac880f66e0 0x7fac8806b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:40:20.847 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:20.843+0000 7fac8e1e9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac880f6ff0 0x7fac88069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:40:20.847 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:20.843+0000 7fac8e1e9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac8806a380 con 0x7fac880f6ff0 2021-11-03T18:40:20.847 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:20.843+0000 7fac8e1e9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac8806a4c0 con 0x7fac880f66e0 2021-11-03T18:40:20.847 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:20.843+0000 7fac86ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac880f6ff0 0x7fac88069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:40:20.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:20.843+0000 7fac86ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac880f6ff0 0x7fac88069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36184/0 (socket says 172.21.15.27:36184) 2021-11-03T18:40:20.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:20.843+0000 7fac86ffd700 1 -- 172.21.15.27:0/604630801 learned_addr learned my addr 172.21.15.27:0/604630801 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:40:20.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:20.843+0000 7fac877fe700 1 --2- 172.21.15.27:0/604630801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac880f66e0 0x7fac8806b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:40:20.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:20.844+0000 7fac86ffd700 1 -- 172.21.15.27:0/604630801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac880f66e0 msgr2=0x7fac8806b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:40:20.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:20.844+0000 7fac86ffd700 1 --2- 172.21.15.27:0/604630801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac880f66e0 0x7fac8806b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:20.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:20.844+0000 7fac86ffd700 1 -- 172.21.15.27:0/604630801 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac70005040 con 0x7fac880f6ff0 2021-11-03T18:40:20.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:20.844+0000 7fac86ffd700 1 --2- 172.21.15.27:0/604630801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac880f6ff0 0x7fac88069e20 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 rx=0x7fac70000c00 tx=0x7fac7000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:40:20.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:20.844+0000 7fac7ffff700 1 -- 172.21.15.27:0/604630801 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac70019070 con 0x7fac880f6ff0 2021-11-03T18:40:20.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:20.844+0000 7fac7ffff700 1 -- 172.21.15.27:0/604630801 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fac70007e70 con 0x7fac880f6ff0 2021-11-03T18:40:20.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:20.844+0000 7fac8e1e9700 1 -- 172.21.15.27:0/604630801 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac8806a740 con 0x7fac880f6ff0 2021-11-03T18:40:20.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:20.844+0000 7fac7ffff700 1 -- 172.21.15.27:0/604630801 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac70014ce0 con 0x7fac880f6ff0 2021-11-03T18:40:20.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:20.844+0000 7fac8e1e9700 1 -- 172.21.15.27:0/604630801 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac8806aca0 con 0x7fac880f6ff0 2021-11-03T18:40:20.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:20.845+0000 7fac8e1e9700 1 -- 172.21.15.27:0/604630801 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac88045ad0 con 0x7fac880f6ff0 2021-11-03T18:40:20.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:20.845+0000 7fac7ffff700 1 -- 172.21.15.27:0/604630801 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fac70025070 con 0x7fac880f6ff0 2021-11-03T18:40:20.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:20.846+0000 7fac7ffff700 1 --2- 172.21.15.27:0/604630801 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fac740640b0 0x7fac74066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:40:20.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:20.846+0000 7fac7ffff700 1 -- 172.21.15.27:0/604630801 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fac70081710 con 0x7fac880f6ff0 2021-11-03T18:40:20.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:20.846+0000 7fac877fe700 1 --2- 172.21.15.27:0/604630801 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fac740640b0 0x7fac74066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:40:20.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:20.846+0000 7fac877fe700 1 --2- 172.21.15.27:0/604630801 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fac740640b0 0x7fac74066570 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 rx=0x7fac8806b730 tx=0x7fac8000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:40:20.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:20.850+0000 7fac7ffff700 1 -- 172.21.15.27:0/604630801 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fac700539b0 con 0x7fac880f6ff0 2021-11-03T18:40:21.031 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.031+0000 7fac8e1e9700 1 -- 172.21.15.27:0/604630801 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fac880fefe0 con 0x7fac740640b0 2021-11-03T18:40:21.032 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.033+0000 7fac7ffff700 1 -- 172.21.15.27:0/604630801 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fac880fefe0 con 0x7fac740640b0 2021-11-03T18:40:21.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.034+0000 7fac8e1e9700 1 -- 172.21.15.27:0/604630801 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fac740640b0 msgr2=0x7fac74066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:40:21.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.034+0000 7fac8e1e9700 1 --2- 172.21.15.27:0/604630801 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fac740640b0 0x7fac74066570 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 rx=0x7fac8806b730 tx=0x7fac8000b040).stop 2021-11-03T18:40:21.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.034+0000 7fac8e1e9700 1 -- 172.21.15.27:0/604630801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac880f6ff0 msgr2=0x7fac88069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:40:21.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.034+0000 7fac8e1e9700 1 --2- 172.21.15.27:0/604630801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac880f6ff0 0x7fac88069e20 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 rx=0x7fac70000c00 tx=0x7fac7000b040).stop 2021-11-03T18:40:21.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.035+0000 7fac8e1e9700 1 -- 172.21.15.27:0/604630801 shutdown_connections 2021-11-03T18:40:21.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.035+0000 7fac8e1e9700 1 --2- 172.21.15.27:0/604630801 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fac740640b0 0x7fac74066570 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:21.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.035+0000 7fac8e1e9700 1 --2- 172.21.15.27:0/604630801 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fac880f66e0 0x7fac8806b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:21.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.035+0000 7fac8e1e9700 1 --2- 172.21.15.27:0/604630801 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fac880f6ff0 0x7fac88069e20 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:21.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.035+0000 7fac8e1e9700 1 -- 172.21.15.27:0/604630801 >> 172.21.15.27:0/604630801 conn(0x7fac880f1f00 msgr2=0x7fac880fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:40:21.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.035+0000 7fac8e1e9700 1 -- 172.21.15.27:0/604630801 shutdown_connections 2021-11-03T18:40:21.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.035+0000 7fac8e1e9700 1 -- 172.21.15.27:0/604630801 wait complete. 2021-11-03T18:40:21.043 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:40:21.135 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.135+0000 7f4651c97700 1 -- 172.21.15.27:0/1800082532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f464c0f55c0 msgr2=0x7f464c0f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:40:21.135 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.135+0000 7f4651c97700 1 --2- 172.21.15.27:0/1800082532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f464c0f55c0 0x7f464c0f59e0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 rx=0x7f463c004660 tx=0x7f463c009b30).stop 2021-11-03T18:40:21.135 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.135+0000 7f4651c97700 1 -- 172.21.15.27:0/1800082532 shutdown_connections 2021-11-03T18:40:21.135 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.135+0000 7f4651c97700 1 --2- 172.21.15.27:0/1800082532 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f464c0f5ed0 0x7f464c0f6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:21.136 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.135+0000 7f4651c97700 1 --2- 172.21.15.27:0/1800082532 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f464c0f55c0 0x7f464c0f59e0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:21.136 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.135+0000 7f4651c97700 1 -- 172.21.15.27:0/1800082532 >> 172.21.15.27:0/1800082532 conn(0x7f464c0f0de0 msgr2=0x7f464c0f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:40:21.136 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.135+0000 7f4651c97700 1 -- 172.21.15.27:0/1800082532 shutdown_connections 2021-11-03T18:40:21.136 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.135+0000 7f4651c97700 1 -- 172.21.15.27:0/1800082532 wait complete. 2021-11-03T18:40:21.136 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.136+0000 7f4651c97700 1 Processor -- start 2021-11-03T18:40:21.137 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.136+0000 7f4651c97700 1 -- start start 2021-11-03T18:40:21.139 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.136+0000 7f4651c97700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f464c0f55c0 0x7f464c0f97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:40:21.139 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.136+0000 7f4651c97700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f464c0f5ed0 0x7f464c0f7c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:40:21.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.136+0000 7f4651c97700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464c0f81a0 con 0x7f464c0f5ed0 2021-11-03T18:40:21.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.136+0000 7f4651c97700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464c0f82e0 con 0x7f464c0f55c0 2021-11-03T18:40:21.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.136+0000 7f4642ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f464c0f5ed0 0x7f464c0f7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:40:21.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.136+0000 7f4642ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f464c0f5ed0 0x7f464c0f7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36194/0 (socket says 172.21.15.27:36194) 2021-11-03T18:40:21.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.136+0000 7f4642ffd700 1 -- 172.21.15.27:0/2272683172 learned_addr learned my addr 172.21.15.27:0/2272683172 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:40:21.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.136+0000 7f464b7fe700 1 --2- 172.21.15.27:0/2272683172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f464c0f55c0 0x7f464c0f97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:40:21.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.137+0000 7f4642ffd700 1 -- 172.21.15.27:0/2272683172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f464c0f55c0 msgr2=0x7f464c0f97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:40:21.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.137+0000 7f4642ffd700 1 --2- 172.21.15.27:0/2272683172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f464c0f55c0 0x7f464c0f97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:21.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.137+0000 7f4642ffd700 1 -- 172.21.15.27:0/2272683172 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f463c005040 con 0x7f464c0f5ed0 2021-11-03T18:40:21.142 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.137+0000 7f4642ffd700 1 --2- 172.21.15.27:0/2272683172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f464c0f5ed0 0x7f464c0f7c40 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 rx=0x7f463400e7f0 tx=0x7f463400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:40:21.142 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.137+0000 7f4648ff9700 1 -- 172.21.15.27:0/2272683172 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f46340092e0 con 0x7f464c0f5ed0 2021-11-03T18:40:21.142 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.137+0000 7f4648ff9700 1 -- 172.21.15.27:0/2272683172 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f463400f040 con 0x7f464c0f5ed0 2021-11-03T18:40:21.142 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.137+0000 7f4648ff9700 1 -- 172.21.15.27:0/2272683172 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4634014750 con 0x7f464c0f5ed0 2021-11-03T18:40:21.142 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.137+0000 7f4651c97700 1 -- 172.21.15.27:0/2272683172 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f464c0f85c0 con 0x7f464c0f5ed0 2021-11-03T18:40:21.143 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.137+0000 7f4651c97700 1 -- 172.21.15.27:0/2272683172 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f464c10ef00 con 0x7f464c0f5ed0 2021-11-03T18:40:21.143 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.138+0000 7f4651c97700 1 -- 172.21.15.27:0/2272683172 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f464c045ad0 con 0x7f464c0f5ed0 2021-11-03T18:40:21.143 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.138+0000 7f4648ff9700 1 -- 172.21.15.27:0/2272683172 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f463401b030 con 0x7f464c0f5ed0 2021-11-03T18:40:21.143 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.139+0000 7f4648ff9700 1 --2- 172.21.15.27:0/2272683172 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f46380640b0 0x7f4638066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:40:21.144 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.139+0000 7f4648ff9700 1 -- 172.21.15.27:0/2272683172 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f46340805d0 con 0x7f464c0f5ed0 2021-11-03T18:40:21.144 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.139+0000 7f464b7fe700 1 --2- 172.21.15.27:0/2272683172 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f46380640b0 0x7f4638066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:40:21.144 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.139+0000 7f464b7fe700 1 --2- 172.21.15.27:0/2272683172 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f46380640b0 0x7f4638066570 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0x7f463c004d80 tx=0x7f463c016040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:40:21.145 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.143+0000 7f4648ff9700 1 -- 172.21.15.27:0/2272683172 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f46340528f0 con 0x7f464c0f5ed0 2021-11-03T18:40:21.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:20 smithi115 ceph-mon[65453]: pgmap v2572: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:21.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5062.481640 secs 2021-11-03T18:40:21.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:20 smithi027 ceph-mon[97244]: pgmap v2572: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:21.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5062.481640 secs 2021-11-03T18:40:21.318 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.319+0000 7f4651c97700 1 -- 172.21.15.27:0/2272683172 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f464c0fad40 con 0x7f46380640b0 2021-11-03T18:40:21.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.334+0000 7f4648ff9700 1 -- 172.21.15.27:0/2272683172 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f464c0fad40 con 0x7f46380640b0 2021-11-03T18:40:21.333 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:40:21.333 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (91m) 25s ago 93m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:40:21.334 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (83m) 25s ago 93m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:40:21.334 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (83m) 10m ago 92m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:40:21.334 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (91m) 25s ago 92m 42.9M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:40:21.334 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (85m) 25s ago 88m 26.9M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:40:21.334 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (88m) 25s ago 88m 3852M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:40:21.335 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (84m) 10m ago 88m 26.3M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:40:21.335 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (85m) 10m ago 88m 27.4M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:40:21.335 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (85m) 25s ago 94m 625M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:40:21.335 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (84m) 10m ago 92m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:40:21.336 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (84m) 25s ago 94m 387M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:40:21.336 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (84m) 10m ago 92m 160M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:40:21.336 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (92m) 25s ago 92m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:40:21.336 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (92m) 10m ago 92m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:40:21.337 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (83m) 25s ago 91m 433M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:40:21.337 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (82m) 25s ago 91m 471M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:40:21.337 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (82m) 25s ago 90m 356M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:40:21.337 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (81m) 10m ago 90m 479M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:40:21.337 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (81m) 10m ago 90m 378M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:40:21.338 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (80m) 10m ago 89m 388M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:40:21.338 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (91m) 25s ago 92m 66.8M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:40:21.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.336+0000 7f4651c97700 1 -- 172.21.15.27:0/2272683172 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f46380640b0 msgr2=0x7f4638066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:40:21.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.336+0000 7f4651c97700 1 --2- 172.21.15.27:0/2272683172 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f46380640b0 0x7f4638066570 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0x7f463c004d80 tx=0x7f463c016040).stop 2021-11-03T18:40:21.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.336+0000 7f4651c97700 1 -- 172.21.15.27:0/2272683172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f464c0f5ed0 msgr2=0x7f464c0f7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:40:21.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.336+0000 7f4651c97700 1 --2- 172.21.15.27:0/2272683172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f464c0f5ed0 0x7f464c0f7c40 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 rx=0x7f463400e7f0 tx=0x7f463400b040).stop 2021-11-03T18:40:21.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.336+0000 7f4651c97700 1 -- 172.21.15.27:0/2272683172 shutdown_connections 2021-11-03T18:40:21.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.336+0000 7f4651c97700 1 --2- 172.21.15.27:0/2272683172 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f46380640b0 0x7f4638066570 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:21.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.336+0000 7f4651c97700 1 --2- 172.21.15.27:0/2272683172 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f464c0f55c0 0x7f464c0f97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:21.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.336+0000 7f4651c97700 1 --2- 172.21.15.27:0/2272683172 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f464c0f5ed0 0x7f464c0f7c40 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:21.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.336+0000 7f4651c97700 1 -- 172.21.15.27:0/2272683172 >> 172.21.15.27:0/2272683172 conn(0x7f464c0f0de0 msgr2=0x7f464c05ffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:40:21.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.336+0000 7f4651c97700 1 -- 172.21.15.27:0/2272683172 shutdown_connections 2021-11-03T18:40:21.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.336+0000 7f4651c97700 1 -- 172.21.15.27:0/2272683172 wait complete. 2021-11-03T18:40:21.436 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.435+0000 7f59ea3b3700 1 -- 172.21.15.27:0/2387340925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59e40f7960 msgr2=0x7f59e40f7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:40:21.436 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.435+0000 7f59ea3b3700 1 --2- 172.21.15.27:0/2387340925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59e40f7960 0x7f59e40f7dc0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 rx=0x7f59d4004660 tx=0x7f59d4009b30).stop 2021-11-03T18:40:21.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.435+0000 7f59ea3b3700 1 -- 172.21.15.27:0/2387340925 shutdown_connections 2021-11-03T18:40:21.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.435+0000 7f59ea3b3700 1 --2- 172.21.15.27:0/2387340925 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59e40f7960 0x7f59e40f7dc0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:21.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.435+0000 7f59ea3b3700 1 --2- 172.21.15.27:0/2387340925 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59e40f66c0 0x7f59e40f6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:21.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.435+0000 7f59ea3b3700 1 -- 172.21.15.27:0/2387340925 >> 172.21.15.27:0/2387340925 conn(0x7f59e40f1e20 msgr2=0x7f59e40f4240 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:40:21.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.435+0000 7f59ea3b3700 1 -- 172.21.15.27:0/2387340925 shutdown_connections 2021-11-03T18:40:21.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.435+0000 7f59ea3b3700 1 -- 172.21.15.27:0/2387340925 wait complete. 2021-11-03T18:40:21.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.436+0000 7f59ea3b3700 1 Processor -- start 2021-11-03T18:40:21.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.436+0000 7f59ea3b3700 1 -- start start 2021-11-03T18:40:21.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.436+0000 7f59ea3b3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59e40f66c0 0x7f59e4106ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:40:21.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.436+0000 7f59ea3b3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59e40f7960 0x7f59e41073b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:40:21.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.436+0000 7f59ea3b3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e4107960 con 0x7f59e40f66c0 2021-11-03T18:40:21.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.436+0000 7f59ea3b3700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e4107aa0 con 0x7f59e40f7960 2021-11-03T18:40:21.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.436+0000 7f59e3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59e40f66c0 0x7f59e4106ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:40:21.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.436+0000 7f59e37fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59e40f7960 0x7f59e41073b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:40:21.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.436+0000 7f59e3fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59e40f66c0 0x7f59e4106ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36202/0 (socket says 172.21.15.27:36202) 2021-11-03T18:40:21.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.436+0000 7f59e3fff700 1 -- 172.21.15.27:0/3248762793 learned_addr learned my addr 172.21.15.27:0/3248762793 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:40:21.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.436+0000 7f59e3fff700 1 -- 172.21.15.27:0/3248762793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59e40f7960 msgr2=0x7f59e41073b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:40:21.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.436+0000 7f59e3fff700 1 --2- 172.21.15.27:0/3248762793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59e40f7960 0x7f59e41073b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:21.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.437+0000 7f59e3fff700 1 -- 172.21.15.27:0/3248762793 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d4005040 con 0x7f59e40f66c0 2021-11-03T18:40:21.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.437+0000 7f59e37fe700 1 --2- 172.21.15.27:0/3248762793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59e40f7960 0x7f59e41073b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-03T18:40:21.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.437+0000 7f59e3fff700 1 --2- 172.21.15.27:0/3248762793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59e40f66c0 0x7f59e4106ec0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 rx=0x7f59cc00e7c0 tx=0x7f59cc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:40:21.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.437+0000 7f59e0ff9700 1 -- 172.21.15.27:0/3248762793 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59cc0092e0 con 0x7f59e40f66c0 2021-11-03T18:40:21.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.437+0000 7f59e0ff9700 1 -- 172.21.15.27:0/3248762793 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f59cc00f040 con 0x7f59e40f66c0 2021-11-03T18:40:21.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.437+0000 7f59e0ff9700 1 -- 172.21.15.27:0/3248762793 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f59cc0147e0 con 0x7f59e40f66c0 2021-11-03T18:40:21.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.437+0000 7f59ea3b3700 1 -- 172.21.15.27:0/3248762793 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59e4108440 con 0x7f59e40f66c0 2021-11-03T18:40:21.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.437+0000 7f59ea3b3700 1 -- 172.21.15.27:0/3248762793 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e4108a00 con 0x7f59e40f66c0 2021-11-03T18:40:21.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.438+0000 7f59ea3b3700 1 -- 172.21.15.27:0/3248762793 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59e4045ad0 con 0x7f59e40f66c0 2021-11-03T18:40:21.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.438+0000 7f59e0ff9700 1 -- 172.21.15.27:0/3248762793 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f59cc01b030 con 0x7f59e40f66c0 2021-11-03T18:40:21.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.439+0000 7f59e0ff9700 1 --2- 172.21.15.27:0/3248762793 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f59d00640b0 0x7f59d0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:40:21.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.439+0000 7f59e0ff9700 1 -- 172.21.15.27:0/3248762793 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f59cc080770 con 0x7f59e40f66c0 2021-11-03T18:40:21.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.439+0000 7f59e37fe700 1 --2- 172.21.15.27:0/3248762793 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f59d00640b0 0x7f59d0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:40:21.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.439+0000 7f59e37fe700 1 --2- 172.21.15.27:0/3248762793 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f59d00640b0 0x7f59d0066570 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 rx=0x7f59d4005010 tx=0x7f59d4016040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:40:21.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.443+0000 7f59e0ff9700 1 -- 172.21.15.27:0/3248762793 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f59cc0529e0 con 0x7f59e40f66c0 2021-11-03T18:40:21.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.673+0000 7f59ea3b3700 1 -- 172.21.15.27:0/3248762793 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f59e40fef50 con 0x7f59e40f66c0 2021-11-03T18:40:21.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.673+0000 7f59e0ff9700 1 -- 172.21.15.27:0/3248762793 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f59cc019070 con 0x7f59e40f66c0 2021-11-03T18:40:21.673 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:40:21.674 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:40:21.674 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:40:21.674 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:40:21.674 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:40:21.675 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:40:21.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:40:21.675 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:40:21.675 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:40:21.675 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:40:21.676 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:40:21.676 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:40:21.676 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:40:21.676 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:40:21.677 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:40:21.677 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:40:21.677 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:40:21.677 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:40:21.678 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.675+0000 7f59ea3b3700 1 -- 172.21.15.27:0/3248762793 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f59d00640b0 msgr2=0x7f59d0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:40:21.678 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.675+0000 7f59ea3b3700 1 --2- 172.21.15.27:0/3248762793 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f59d00640b0 0x7f59d0066570 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 rx=0x7f59d4005010 tx=0x7f59d4016040).stop 2021-11-03T18:40:21.679 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.675+0000 7f59ea3b3700 1 -- 172.21.15.27:0/3248762793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59e40f66c0 msgr2=0x7f59e4106ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:40:21.679 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.675+0000 7f59ea3b3700 1 --2- 172.21.15.27:0/3248762793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59e40f66c0 0x7f59e4106ec0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 rx=0x7f59cc00e7c0 tx=0x7f59cc00b040).stop 2021-11-03T18:40:21.679 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.676+0000 7f59ea3b3700 1 -- 172.21.15.27:0/3248762793 shutdown_connections 2021-11-03T18:40:21.679 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.676+0000 7f59ea3b3700 1 --2- 172.21.15.27:0/3248762793 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f59d00640b0 0x7f59d0066570 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:21.680 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.676+0000 7f59ea3b3700 1 --2- 172.21.15.27:0/3248762793 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f59e40f66c0 0x7f59e4106ec0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:21.680 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.676+0000 7f59ea3b3700 1 --2- 172.21.15.27:0/3248762793 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f59e40f7960 0x7f59e41073b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:21.680 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.676+0000 7f59ea3b3700 1 -- 172.21.15.27:0/3248762793 >> 172.21.15.27:0/3248762793 conn(0x7f59e40f1e20 msgr2=0x7f59e40fb3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:40:21.680 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.676+0000 7f59ea3b3700 1 -- 172.21.15.27:0/3248762793 shutdown_connections 2021-11-03T18:40:21.681 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.676+0000 7f59ea3b3700 1 -- 172.21.15.27:0/3248762793 wait complete. 2021-11-03T18:40:21.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.776+0000 7f51fe6a3700 1 -- 172.21.15.27:0/2238981230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51f80f7960 msgr2=0x7f51f80f7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:40:21.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.776+0000 7f51fe6a3700 1 --2- 172.21.15.27:0/2238981230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51f80f7960 0x7f51f80f7dc0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 rx=0x7f51e8004660 tx=0x7f51e8009b30).stop 2021-11-03T18:40:21.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.776+0000 7f51fe6a3700 1 -- 172.21.15.27:0/2238981230 shutdown_connections 2021-11-03T18:40:21.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.776+0000 7f51fe6a3700 1 --2- 172.21.15.27:0/2238981230 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51f80f7960 0x7f51f80f7dc0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:21.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.776+0000 7f51fe6a3700 1 --2- 172.21.15.27:0/2238981230 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51f80f66c0 0x7f51f80f6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:21.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.776+0000 7f51fe6a3700 1 -- 172.21.15.27:0/2238981230 >> 172.21.15.27:0/2238981230 conn(0x7f51f80f1e20 msgr2=0x7f51f80f4240 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:40:21.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.776+0000 7f51fe6a3700 1 -- 172.21.15.27:0/2238981230 shutdown_connections 2021-11-03T18:40:21.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.776+0000 7f51fe6a3700 1 -- 172.21.15.27:0/2238981230 wait complete. 2021-11-03T18:40:21.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.776+0000 7f51fe6a3700 1 Processor -- start 2021-11-03T18:40:21.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.777+0000 7f51fe6a3700 1 -- start start 2021-11-03T18:40:21.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.777+0000 7f51fe6a3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51f80f66c0 0x7f51f8106ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:40:21.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.777+0000 7f51fe6a3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51f80f7960 0x7f51f81073b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:40:21.780 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.777+0000 7f51fe6a3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f8107960 con 0x7f51f80f7960 2021-11-03T18:40:21.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.777+0000 7f51fe6a3700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51f8107aa0 con 0x7f51f80f66c0 2021-11-03T18:40:21.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.777+0000 7f51f77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51f80f7960 0x7f51f81073b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:40:21.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.777+0000 7f51f77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51f80f7960 0x7f51f81073b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36214/0 (socket says 172.21.15.27:36214) 2021-11-03T18:40:21.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.777+0000 7f51f77fe700 1 -- 172.21.15.27:0/2944464774 learned_addr learned my addr 172.21.15.27:0/2944464774 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:40:21.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.777+0000 7f51f7fff700 1 --2- 172.21.15.27:0/2944464774 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51f80f66c0 0x7f51f8106ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:40:21.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.777+0000 7f51f77fe700 1 -- 172.21.15.27:0/2944464774 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51f80f66c0 msgr2=0x7f51f8106ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:40:21.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.777+0000 7f51f77fe700 1 --2- 172.21.15.27:0/2944464774 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51f80f66c0 0x7f51f8106ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:21.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.777+0000 7f51f77fe700 1 -- 172.21.15.27:0/2944464774 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e8005040 con 0x7f51f80f7960 2021-11-03T18:40:21.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.777+0000 7f51f77fe700 1 --2- 172.21.15.27:0/2944464774 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51f80f7960 0x7f51f81073b0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 rx=0x7f51e8009990 tx=0x7f51e8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:40:21.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.777+0000 7f51f4ff9700 1 -- 172.21.15.27:0/2944464774 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51e8018070 con 0x7f51f80f7960 2021-11-03T18:40:21.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.777+0000 7f51f4ff9700 1 -- 172.21.15.27:0/2944464774 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f51e8007ba0 con 0x7f51f80f7960 2021-11-03T18:40:21.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.778+0000 7f51fe6a3700 1 -- 172.21.15.27:0/2944464774 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51f81083e0 con 0x7f51f80f7960 2021-11-03T18:40:21.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.778+0000 7f51f4ff9700 1 -- 172.21.15.27:0/2944464774 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51e801c3c0 con 0x7f51f80f7960 2021-11-03T18:40:21.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.778+0000 7f51fe6a3700 1 -- 172.21.15.27:0/2944464774 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51f8108940 con 0x7f51f80f7960 2021-11-03T18:40:21.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.779+0000 7f51fe6a3700 1 -- 172.21.15.27:0/2944464774 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51f8045ad0 con 0x7f51f80f7960 2021-11-03T18:40:21.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.779+0000 7f51f4ff9700 1 -- 172.21.15.27:0/2944464774 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f51e801d420 con 0x7f51f80f7960 2021-11-03T18:40:21.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.779+0000 7f51f4ff9700 1 --2- 172.21.15.27:0/2944464774 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f51e00640b0 0x7f51e0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:40:21.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.779+0000 7f51f4ff9700 1 -- 172.21.15.27:0/2944464774 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f51e8082360 con 0x7f51f80f7960 2021-11-03T18:40:21.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.780+0000 7f51f7fff700 1 --2- 172.21.15.27:0/2944464774 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f51e00640b0 0x7f51e0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:40:21.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.780+0000 7f51f7fff700 1 --2- 172.21.15.27:0/2944464774 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f51e00640b0 0x7f51e0066570 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0x7f51f80f77c0 tx=0x7f51ec00b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:40:21.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.784+0000 7f51f4ff9700 1 -- 172.21.15.27:0/2944464774 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f51e8054720 con 0x7f51f80f7960 2021-11-03T18:40:21.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.991+0000 7f51fe6a3700 1 -- 172.21.15.27:0/2944464774 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f51f8027a00 con 0x7f51f80f7960 2021-11-03T18:40:21.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.992+0000 7f51f4ff9700 1 -- 172.21.15.27:0/2944464774 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f51e8054310 con 0x7f51f80f7960 2021-11-03T18:40:21.991 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:40:21.992 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:40:21.992 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:40:21.992 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:40:21.992 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:40:21.993 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:40:21.993 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:40:21.993 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:40:21.993 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:40:21.994 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:40:21.994 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:40:21.994 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:40:21.994 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:40:21.994 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:40:21.995 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:40:21.995 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:40:21.995 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:40:21.995 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:40:21.996 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:40:21.996 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:40:21.996 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:40:21.996 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:40:21.996 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:40:21.997 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:40:21.997 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:40:21.997 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:40:21.997 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:40:21.997 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:40:21.998 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:40:21.998 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:40:21.998 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:40:21.998 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:40:21.999 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:40:21.999 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:40:21.999 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:40:21.999 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:40:22.000 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:40:22.000 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:40:22.000 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:40:22.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.994+0000 7f51fe6a3700 1 -- 172.21.15.27:0/2944464774 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f51e00640b0 msgr2=0x7f51e0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:40:22.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.994+0000 7f51fe6a3700 1 --2- 172.21.15.27:0/2944464774 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f51e00640b0 0x7f51e0066570 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0x7f51f80f77c0 tx=0x7f51ec00b040).stop 2021-11-03T18:40:22.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.994+0000 7f51fe6a3700 1 -- 172.21.15.27:0/2944464774 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51f80f7960 msgr2=0x7f51f81073b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:40:22.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.994+0000 7f51fe6a3700 1 --2- 172.21.15.27:0/2944464774 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51f80f7960 0x7f51f81073b0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 rx=0x7f51e8009990 tx=0x7f51e8009b30).stop 2021-11-03T18:40:22.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.994+0000 7f51fe6a3700 1 -- 172.21.15.27:0/2944464774 shutdown_connections 2021-11-03T18:40:22.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.994+0000 7f51fe6a3700 1 --2- 172.21.15.27:0/2944464774 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f51e00640b0 0x7f51e0066570 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:22.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.994+0000 7f51fe6a3700 1 --2- 172.21.15.27:0/2944464774 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f51f80f66c0 0x7f51f8106ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:22.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.994+0000 7f51fe6a3700 1 --2- 172.21.15.27:0/2944464774 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f51f80f7960 0x7f51f81073b0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:22.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.994+0000 7f51fe6a3700 1 -- 172.21.15.27:0/2944464774 >> 172.21.15.27:0/2944464774 conn(0x7f51f80f1e20 msgr2=0x7f51f80fb3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:40:22.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.994+0000 7f51fe6a3700 1 -- 172.21.15.27:0/2944464774 shutdown_connections 2021-11-03T18:40:22.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:21.994+0000 7f51fe6a3700 1 -- 172.21.15.27:0/2944464774 wait complete. 2021-11-03T18:40:22.004 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:40:22.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:21 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3248762793' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:22.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:21 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3248762793' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:23.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:22 smithi115 ceph-mon[65453]: from='client.36744 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:40:23.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:22 smithi115 ceph-mon[65453]: pgmap v2573: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:23.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:22 smithi115 ceph-mon[65453]: from='client.36748 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:40:23.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:40:23.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:40:23.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:40:23.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:23.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:23.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:23.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:22 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:23.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:22 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2944464774' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:40:23.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:22 smithi027 ceph-mon[97244]: from='client.36744 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:40:23.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:22 smithi027 ceph-mon[97244]: pgmap v2573: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:23.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:22 smithi027 ceph-mon[97244]: from='client.36748 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:40:23.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:40:23.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:40:23.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:40:23.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:23.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:23.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:23.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:22 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:23.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:22 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2944464774' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:40:24.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:23 smithi115 ceph-mon[65453]: poll took 3012.878 ms: 1 events 2021-11-03T18:40:24.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:23 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:40:24.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:23 smithi027 ceph-mon[97244]: poll took 3012.878 ms: 1 events 2021-11-03T18:40:24.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:23 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:40:25.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:24 smithi115 ceph-mon[65453]: pgmap v2574: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:25.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:24 smithi027 ceph-mon[97244]: pgmap v2574: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:26.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5067.481761 secs 2021-11-03T18:40:26.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5067.481761 secs 2021-11-03T18:40:27.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:26 smithi115 ceph-mon[65453]: pgmap v2575: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:27.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:26 smithi027 ceph-mon[97244]: pgmap v2575: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:28.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:27 smithi115 ceph-mon[65453]: pgmap v2576: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:28.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:27 smithi027 ceph-mon[97244]: pgmap v2576: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:30.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:30 smithi115 ceph-mon[65453]: pgmap v2577: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:30.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5072.481876 secs 2021-11-03T18:40:30.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:30 smithi027 ceph-mon[97244]: pgmap v2577: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:30.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5072.481876 secs 2021-11-03T18:40:32.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:32 smithi027 ceph-mon[97244]: pgmap v2578: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:32.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:40:32.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:40:32.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:40:32.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:32.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:32.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:32.943 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:32 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:32.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:32 smithi115 ceph-mon[65453]: pgmap v2578: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:32.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:40:32.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:40:32.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:40:32.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:32.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:32.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:32.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:32 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:33.959 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:33 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:40:34.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:33 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:40:34.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:34 smithi115 ceph-mon[65453]: pgmap v2579: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:35.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:34 smithi027 ceph-mon[97244]: pgmap v2579: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:35.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5077.481996 secs 2021-11-03T18:40:36.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5077.481996 secs 2021-11-03T18:40:36.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:36 smithi115 ceph-mon[65453]: pgmap v2580: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:37.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:36 smithi027 ceph-mon[97244]: pgmap v2580: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:38.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:38 smithi115 ceph-mon[65453]: pgmap v2581: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:39.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:38 smithi027 ceph-mon[97244]: pgmap v2581: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:40.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:40 smithi115 ceph-mon[65453]: pgmap v2582: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:40.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5082.482123 secs 2021-11-03T18:40:41.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:40 smithi027 ceph-mon[97244]: pgmap v2582: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:41.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5082.482123 secs 2021-11-03T18:40:42.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:42 smithi115 ceph-mon[65453]: pgmap v2583: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:42.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:40:42.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:40:42.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:40:42.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:42.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:42.741 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:42.741 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:42 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:42.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:42 smithi027 ceph-mon[97244]: pgmap v2583: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:42.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:40:42.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:40:42.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:40:42.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:42.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:42.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:42.776 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:42 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:43.959 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:43 smithi115 ceph-mon[65453]: poll took 20057.368 ms: 1 events 2021-11-03T18:40:43.959 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:43 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:40:44.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:43 smithi027 ceph-mon[97244]: poll took 20057.368 ms: 1 events 2021-11-03T18:40:44.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:43 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:40:44.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:44 smithi115 ceph-mon[65453]: pgmap v2584: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:45.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:44 smithi027 ceph-mon[97244]: pgmap v2584: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:45.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5087.482246 secs 2021-11-03T18:40:46.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5087.482246 secs 2021-11-03T18:40:46.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:46 smithi115 ceph-mon[65453]: pgmap v2585: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:47.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:46 smithi027 ceph-mon[97244]: pgmap v2585: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:48.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:48 smithi115 ceph-mon[65453]: pgmap v2586: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:49.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:48 smithi027 ceph-mon[97244]: pgmap v2586: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:50.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:50 smithi115 ceph-mon[65453]: pgmap v2587: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:50.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5092.482371 secs 2021-11-03T18:40:51.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:50 smithi027 ceph-mon[97244]: pgmap v2587: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:51.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5092.482371 secs 2021-11-03T18:40:52.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.110+0000 7ff9cd355700 1 -- 172.21.15.27:0/1541523349 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9c80f6530 msgr2=0x7ff9c80f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:40:52.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.110+0000 7ff9cd355700 1 --2- 172.21.15.27:0/1541523349 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9c80f6530 0x7ff9c80f6950 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 rx=0x7ff9b8004660 tx=0x7ff9b8009b30).stop 2021-11-03T18:40:52.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.110+0000 7ff9cd355700 1 -- 172.21.15.27:0/1541523349 shutdown_connections 2021-11-03T18:40:52.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.110+0000 7ff9cd355700 1 --2- 172.21.15.27:0/1541523349 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9c80f77d0 0x7ff9c80f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:52.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.110+0000 7ff9cd355700 1 --2- 172.21.15.27:0/1541523349 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9c80f6530 0x7ff9c80f6950 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:52.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.110+0000 7ff9cd355700 1 -- 172.21.15.27:0/1541523349 >> 172.21.15.27:0/1541523349 conn(0x7ff9c80f1cd0 msgr2=0x7ff9c80f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:40:52.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.110+0000 7ff9cd355700 1 -- 172.21.15.27:0/1541523349 shutdown_connections 2021-11-03T18:40:52.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.110+0000 7ff9cd355700 1 -- 172.21.15.27:0/1541523349 wait complete. 2021-11-03T18:40:52.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.110+0000 7ff9cd355700 1 Processor -- start 2021-11-03T18:40:52.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.111+0000 7ff9cd355700 1 -- start start 2021-11-03T18:40:52.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.111+0000 7ff9cd355700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9c80f6530 0x7ff9c8106db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:40:52.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.111+0000 7ff9cd355700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9c80f77d0 0x7ff9c81072a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:40:52.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.111+0000 7ff9cd355700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c81078a0 con 0x7ff9c80f6530 2021-11-03T18:40:52.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.111+0000 7ff9cd355700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c81080e0 con 0x7ff9c80f77d0 2021-11-03T18:40:52.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.111+0000 7ff9c7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9c80f6530 0x7ff9c8106db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:40:52.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.111+0000 7ff9c7fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9c80f6530 0x7ff9c8106db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36230/0 (socket says 172.21.15.27:36230) 2021-11-03T18:40:52.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.111+0000 7ff9c7fff700 1 -- 172.21.15.27:0/984045350 learned_addr learned my addr 172.21.15.27:0/984045350 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:40:52.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.111+0000 7ff9c77fe700 1 --2- 172.21.15.27:0/984045350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9c80f77d0 0x7ff9c81072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:40:52.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.111+0000 7ff9c7fff700 1 -- 172.21.15.27:0/984045350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9c80f77d0 msgr2=0x7ff9c81072a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:40:52.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.111+0000 7ff9c7fff700 1 --2- 172.21.15.27:0/984045350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9c80f77d0 0x7ff9c81072a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:52.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.111+0000 7ff9c7fff700 1 -- 172.21.15.27:0/984045350 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b8005040 con 0x7ff9c80f6530 2021-11-03T18:40:52.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.111+0000 7ff9c7fff700 1 --2- 172.21.15.27:0/984045350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9c80f6530 0x7ff9c8106db0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 rx=0x7ff9b8013a70 tx=0x7ff9b8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:40:52.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.112+0000 7ff9c4ff9700 1 -- 172.21.15.27:0/984045350 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9b8018070 con 0x7ff9c80f6530 2021-11-03T18:40:52.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.112+0000 7ff9cd355700 1 -- 172.21.15.27:0/984045350 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9c8108360 con 0x7ff9c80f6530 2021-11-03T18:40:52.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.112+0000 7ff9c4ff9700 1 -- 172.21.15.27:0/984045350 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7ff9b80097d0 con 0x7ff9c80f6530 2021-11-03T18:40:52.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.112+0000 7ff9c4ff9700 1 -- 172.21.15.27:0/984045350 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff9b801c3c0 con 0x7ff9c80f6530 2021-11-03T18:40:52.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.112+0000 7ff9cd355700 1 -- 172.21.15.27:0/984045350 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9c8108840 con 0x7ff9c80f6530 2021-11-03T18:40:52.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.113+0000 7ff9cd355700 1 -- 172.21.15.27:0/984045350 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9c8059100 con 0x7ff9c80f6530 2021-11-03T18:40:52.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.113+0000 7ff9c4ff9700 1 -- 172.21.15.27:0/984045350 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7ff9b8007ba0 con 0x7ff9c80f6530 2021-11-03T18:40:52.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.114+0000 7ff9c4ff9700 1 --2- 172.21.15.27:0/984045350 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff9b4064100 0x7ff9b40665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:40:52.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.114+0000 7ff9c4ff9700 1 -- 172.21.15.27:0/984045350 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7ff9b80824b0 con 0x7ff9c80f6530 2021-11-03T18:40:52.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.114+0000 7ff9c77fe700 1 --2- 172.21.15.27:0/984045350 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff9b4064100 0x7ff9b40665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:40:52.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.114+0000 7ff9c77fe700 1 --2- 172.21.15.27:0/984045350 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff9b4064100 0x7ff9b40665c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 rx=0x7ff9c8106b40 tx=0x7ff9b0009be0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:40:52.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.118+0000 7ff9c4ff9700 1 -- 172.21.15.27:0/984045350 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7ff9b8054810 con 0x7ff9c80f6530 2021-11-03T18:40:52.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.298+0000 7ff9cd355700 1 -- 172.21.15.27:0/984045350 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9c80fee00 con 0x7ff9b4064100 2021-11-03T18:40:52.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.299+0000 7ff9c4ff9700 1 -- 172.21.15.27:0/984045350 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7ff9c80fee00 con 0x7ff9b4064100 2021-11-03T18:40:52.300 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.301+0000 7ff9cd355700 1 -- 172.21.15.27:0/984045350 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff9b4064100 msgr2=0x7ff9b40665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:40:52.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.301+0000 7ff9cd355700 1 --2- 172.21.15.27:0/984045350 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff9b4064100 0x7ff9b40665c0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 rx=0x7ff9c8106b40 tx=0x7ff9b0009be0).stop 2021-11-03T18:40:52.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.301+0000 7ff9cd355700 1 -- 172.21.15.27:0/984045350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9c80f6530 msgr2=0x7ff9c8106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:40:52.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.301+0000 7ff9cd355700 1 --2- 172.21.15.27:0/984045350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9c80f6530 0x7ff9c8106db0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 rx=0x7ff9b8013a70 tx=0x7ff9b8009b30).stop 2021-11-03T18:40:52.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.301+0000 7ff9cd355700 1 -- 172.21.15.27:0/984045350 shutdown_connections 2021-11-03T18:40:52.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.301+0000 7ff9cd355700 1 --2- 172.21.15.27:0/984045350 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff9b4064100 0x7ff9b40665c0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:52.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.301+0000 7ff9cd355700 1 --2- 172.21.15.27:0/984045350 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff9c80f6530 0x7ff9c8106db0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:52.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.301+0000 7ff9cd355700 1 --2- 172.21.15.27:0/984045350 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff9c80f77d0 0x7ff9c81072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:52.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.301+0000 7ff9cd355700 1 -- 172.21.15.27:0/984045350 >> 172.21.15.27:0/984045350 conn(0x7ff9c80f1cd0 msgr2=0x7ff9c80fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:40:52.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.301+0000 7ff9cd355700 1 -- 172.21.15.27:0/984045350 shutdown_connections 2021-11-03T18:40:52.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.301+0000 7ff9cd355700 1 -- 172.21.15.27:0/984045350 wait complete. 2021-11-03T18:40:52.310 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:40:52.403 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.403+0000 7f4b21160700 1 -- 172.21.15.27:0/1468873564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b1c0f7980 msgr2=0x7f4b1c0f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:40:52.404 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.403+0000 7f4b21160700 1 --2- 172.21.15.27:0/1468873564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b1c0f7980 0x7f4b1c0f7de0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 rx=0x7f4b04004660 tx=0x7f4b04009b30).stop 2021-11-03T18:40:52.404 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.404+0000 7f4b21160700 1 -- 172.21.15.27:0/1468873564 shutdown_connections 2021-11-03T18:40:52.404 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.404+0000 7f4b21160700 1 --2- 172.21.15.27:0/1468873564 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b1c0f7980 0x7f4b1c0f7de0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:52.405 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.404+0000 7f4b21160700 1 --2- 172.21.15.27:0/1468873564 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b1c0f66e0 0x7f4b1c0f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:52.405 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.404+0000 7f4b21160700 1 -- 172.21.15.27:0/1468873564 >> 172.21.15.27:0/1468873564 conn(0x7f4b1c0f1e40 msgr2=0x7f4b1c0f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:40:52.405 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.404+0000 7f4b21160700 1 -- 172.21.15.27:0/1468873564 shutdown_connections 2021-11-03T18:40:52.405 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.404+0000 7f4b21160700 1 -- 172.21.15.27:0/1468873564 wait complete. 2021-11-03T18:40:52.405 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.404+0000 7f4b21160700 1 Processor -- start 2021-11-03T18:40:52.406 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.404+0000 7f4b21160700 1 -- start start 2021-11-03T18:40:52.406 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.404+0000 7f4b21160700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b1c0f66e0 0x7f4b1c106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:40:52.406 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.404+0000 7f4b21160700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b1c0f7980 0x7f4b1c107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:40:52.406 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.404+0000 7f4b21160700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1c107a00 con 0x7f4b1c0f7980 2021-11-03T18:40:52.407 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.404+0000 7f4b21160700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b1c107b40 con 0x7f4b1c0f66e0 2021-11-03T18:40:52.407 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.405+0000 7f4b1a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b1c0f7980 0x7f4b1c107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:40:52.407 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.405+0000 7f4b1a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b1c0f7980 0x7f4b1c107450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36242/0 (socket says 172.21.15.27:36242) 2021-11-03T18:40:52.407 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.405+0000 7f4b1a59c700 1 -- 172.21.15.27:0/2697039598 learned_addr learned my addr 172.21.15.27:0/2697039598 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:40:52.408 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.405+0000 7f4b1ad9d700 1 --2- 172.21.15.27:0/2697039598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b1c0f66e0 0x7f4b1c106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:40:52.409 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.405+0000 7f4b1a59c700 1 -- 172.21.15.27:0/2697039598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b1c0f66e0 msgr2=0x7f4b1c106f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:40:52.410 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.405+0000 7f4b1a59c700 1 --2- 172.21.15.27:0/2697039598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b1c0f66e0 0x7f4b1c106f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:52.410 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.405+0000 7f4b1a59c700 1 -- 172.21.15.27:0/2697039598 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b04005040 con 0x7f4b1c0f7980 2021-11-03T18:40:52.410 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.405+0000 7f4b1ad9d700 1 --2- 172.21.15.27:0/2697039598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b1c0f66e0 0x7f4b1c106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:40:52.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.405+0000 7f4b1a59c700 1 --2- 172.21.15.27:0/2697039598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b1c0f7980 0x7f4b1c107450 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 rx=0x7f4b04009990 tx=0x7f4b04009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:40:52.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.405+0000 7f4b137fe700 1 -- 172.21.15.27:0/2697039598 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b04018070 con 0x7f4b1c0f7980 2021-11-03T18:40:52.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.405+0000 7f4b137fe700 1 -- 172.21.15.27:0/2697039598 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f4b04007ba0 con 0x7f4b1c0f7980 2021-11-03T18:40:52.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.405+0000 7f4b21160700 1 -- 172.21.15.27:0/2697039598 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b1c108480 con 0x7f4b1c0f7980 2021-11-03T18:40:52.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.405+0000 7f4b137fe700 1 -- 172.21.15.27:0/2697039598 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b0401c3c0 con 0x7f4b1c0f7980 2021-11-03T18:40:52.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.405+0000 7f4b21160700 1 -- 172.21.15.27:0/2697039598 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b1c1089e0 con 0x7f4b1c0f7980 2021-11-03T18:40:52.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.406+0000 7f4b21160700 1 -- 172.21.15.27:0/2697039598 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b1c045ad0 con 0x7f4b1c0f7980 2021-11-03T18:40:52.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.407+0000 7f4b137fe700 1 -- 172.21.15.27:0/2697039598 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f4b0401d420 con 0x7f4b1c0f7980 2021-11-03T18:40:52.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.407+0000 7f4b137fe700 1 --2- 172.21.15.27:0/2697039598 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4b080640b0 0x7f4b08066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:40:52.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.407+0000 7f4b137fe700 1 -- 172.21.15.27:0/2697039598 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f4b040823e0 con 0x7f4b1c0f7980 2021-11-03T18:40:52.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.407+0000 7f4b1ad9d700 1 --2- 172.21.15.27:0/2697039598 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4b080640b0 0x7f4b08066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:40:52.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.407+0000 7f4b1ad9d700 1 --2- 172.21.15.27:0/2697039598 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4b080640b0 0x7f4b08066570 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 rx=0x7f4b1c0f77e0 tx=0x7f4b14004d00).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:40:52.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.411+0000 7f4b137fe700 1 -- 172.21.15.27:0/2697039598 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f4b040547a0 con 0x7f4b1c0f7980 2021-11-03T18:40:52.584 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.585+0000 7f4b21160700 1 -- 172.21.15.27:0/2697039598 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b1c108c50 con 0x7f4b080640b0 2021-11-03T18:40:52.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.600+0000 7f4b137fe700 1 -- 172.21.15.27:0/2697039598 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f4b1c108c50 con 0x7f4b080640b0 2021-11-03T18:40:52.599 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:40:52.600 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (92m) 57s ago 93m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:40:52.600 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (84m) 57s ago 93m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:40:52.600 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (83m) 30s ago 92m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:40:52.600 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (92m) 57s ago 93m 42.9M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:40:52.601 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (85m) 57s ago 88m 26.9M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:40:52.601 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (88m) 57s ago 88m 3852M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:40:52.601 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (84m) 30s ago 88m 26.5M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:40:52.601 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (86m) 30s ago 88m 27.6M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:40:52.601 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (86m) 57s ago 94m 625M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:40:52.602 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (85m) 30s ago 92m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:40:52.602 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (85m) 57s ago 95m 387M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:40:52.602 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (84m) 30s ago 92m 175M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:40:52.602 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (93m) 57s ago 93m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:40:52.603 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (92m) 30s ago 92m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:40:52.603 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (83m) 57s ago 92m 433M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:40:52.603 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (83m) 57s ago 91m 471M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:40:52.603 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (82m) 57s ago 91m 356M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:40:52.603 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (82m) 30s ago 91m 504M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:40:52.604 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (81m) 30s ago 90m 396M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:40:52.604 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (81m) 30s ago 90m 408M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:40:52.604 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (92m) 57s ago 93m 66.8M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:40:52.606 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.601+0000 7f4b21160700 1 -- 172.21.15.27:0/2697039598 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4b080640b0 msgr2=0x7f4b08066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:40:52.606 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.601+0000 7f4b21160700 1 --2- 172.21.15.27:0/2697039598 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4b080640b0 0x7f4b08066570 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 rx=0x7f4b1c0f77e0 tx=0x7f4b14004d00).stop 2021-11-03T18:40:52.606 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.601+0000 7f4b21160700 1 -- 172.21.15.27:0/2697039598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b1c0f7980 msgr2=0x7f4b1c107450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:40:52.607 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.601+0000 7f4b21160700 1 --2- 172.21.15.27:0/2697039598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b1c0f7980 0x7f4b1c107450 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 rx=0x7f4b04009990 tx=0x7f4b04009b30).stop 2021-11-03T18:40:52.607 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.601+0000 7f4b21160700 1 -- 172.21.15.27:0/2697039598 shutdown_connections 2021-11-03T18:40:52.607 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.601+0000 7f4b21160700 1 --2- 172.21.15.27:0/2697039598 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4b080640b0 0x7f4b08066570 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:52.607 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.601+0000 7f4b21160700 1 --2- 172.21.15.27:0/2697039598 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4b1c0f66e0 0x7f4b1c106f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:52.608 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.601+0000 7f4b21160700 1 --2- 172.21.15.27:0/2697039598 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4b1c0f7980 0x7f4b1c107450 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:52.608 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.602+0000 7f4b21160700 1 -- 172.21.15.27:0/2697039598 >> 172.21.15.27:0/2697039598 conn(0x7f4b1c0f1e40 msgr2=0x7f4b1c0fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:40:52.608 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.602+0000 7f4b21160700 1 -- 172.21.15.27:0/2697039598 shutdown_connections 2021-11-03T18:40:52.608 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.602+0000 7f4b21160700 1 -- 172.21.15.27:0/2697039598 wait complete. 2021-11-03T18:40:52.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.701+0000 7fe41f04b700 1 -- 172.21.15.27:0/2155774652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4180f55c0 msgr2=0x7fe4180f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:40:52.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.701+0000 7fe41f04b700 1 --2- 172.21.15.27:0/2155774652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4180f55c0 0x7fe4180f59e0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 rx=0x7fe4100099e0 tx=0x7fe410004d10).stop 2021-11-03T18:40:52.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.701+0000 7fe41f04b700 1 -- 172.21.15.27:0/2155774652 shutdown_connections 2021-11-03T18:40:52.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.701+0000 7fe41f04b700 1 --2- 172.21.15.27:0/2155774652 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4180f6860 0x7fe4180f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:52.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.701+0000 7fe41f04b700 1 --2- 172.21.15.27:0/2155774652 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4180f55c0 0x7fe4180f59e0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:52.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.701+0000 7fe41f04b700 1 -- 172.21.15.27:0/2155774652 >> 172.21.15.27:0/2155774652 conn(0x7fe4180f0d40 msgr2=0x7fe4180f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:40:52.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.701+0000 7fe41f04b700 1 -- 172.21.15.27:0/2155774652 shutdown_connections 2021-11-03T18:40:52.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.701+0000 7fe41f04b700 1 -- 172.21.15.27:0/2155774652 wait complete. 2021-11-03T18:40:52.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.701+0000 7fe41f04b700 1 Processor -- start 2021-11-03T18:40:52.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.701+0000 7fe41f04b700 1 -- start start 2021-11-03T18:40:52.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.702+0000 7fe41f04b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4180f55c0 0x7fe4180f9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:40:52.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.702+0000 7fe41f04b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4180f6860 0x7fe4180f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:40:52.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.702+0000 7fe41f04b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4180f8100 con 0x7fe4180f6860 2021-11-03T18:40:52.705 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.702+0000 7fe41f04b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4180f8240 con 0x7fe4180f55c0 2021-11-03T18:40:52.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.702+0000 7fe417fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4180f6860 0x7fe4180f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:40:52.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.702+0000 7fe417fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4180f6860 0x7fe4180f7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36252/0 (socket says 172.21.15.27:36252) 2021-11-03T18:40:52.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.702+0000 7fe417fff700 1 -- 172.21.15.27:0/4280224807 learned_addr learned my addr 172.21.15.27:0/4280224807 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:40:52.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.702+0000 7fe41cde7700 1 --2- 172.21.15.27:0/4280224807 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4180f55c0 0x7fe4180f9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:40:52.706 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.702+0000 7fe417fff700 1 -- 172.21.15.27:0/4280224807 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4180f55c0 msgr2=0x7fe4180f9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:40:52.707 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.702+0000 7fe417fff700 1 --2- 172.21.15.27:0/4280224807 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4180f55c0 0x7fe4180f9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:52.707 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.702+0000 7fe417fff700 1 -- 172.21.15.27:0/4280224807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe410009640 con 0x7fe4180f6860 2021-11-03T18:40:52.707 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.702+0000 7fe417fff700 1 --2- 172.21.15.27:0/4280224807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4180f6860 0x7fe4180f7ba0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 rx=0x7fe40c00e7f0 tx=0x7fe40c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:40:52.707 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.702+0000 7fe4157fa700 1 -- 172.21.15.27:0/4280224807 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe40c0092e0 con 0x7fe4180f6860 2021-11-03T18:40:52.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.703+0000 7fe41f04b700 1 -- 172.21.15.27:0/4280224807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4180f8520 con 0x7fe4180f6860 2021-11-03T18:40:52.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.703+0000 7fe4157fa700 1 -- 172.21.15.27:0/4280224807 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fe40c00f040 con 0x7fe4180f6860 2021-11-03T18:40:52.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.703+0000 7fe4157fa700 1 -- 172.21.15.27:0/4280224807 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe40c014750 con 0x7fe4180f6860 2021-11-03T18:40:52.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.703+0000 7fe41f04b700 1 -- 172.21.15.27:0/4280224807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe41810f0d0 con 0x7fe4180f6860 2021-11-03T18:40:52.708 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.704+0000 7fe41f04b700 1 -- 172.21.15.27:0/4280224807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe418045ad0 con 0x7fe4180f6860 2021-11-03T18:40:52.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.704+0000 7fe4157fa700 1 -- 172.21.15.27:0/4280224807 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fe40c01b030 con 0x7fe4180f6860 2021-11-03T18:40:52.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.704+0000 7fe4157fa700 1 --2- 172.21.15.27:0/4280224807 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe4000640b0 0x7fe400066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:40:52.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.704+0000 7fe4157fa700 1 -- 172.21.15.27:0/4280224807 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fe40c080680 con 0x7fe4180f6860 2021-11-03T18:40:52.709 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.705+0000 7fe41cde7700 1 --2- 172.21.15.27:0/4280224807 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe4000640b0 0x7fe400066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:40:52.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.705+0000 7fe41cde7700 1 --2- 172.21.15.27:0/4280224807 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe4000640b0 0x7fe400066570 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 rx=0x7fe41000b6c0 tx=0x7fe410015040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:40:52.710 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.709+0000 7fe4157fa700 1 -- 172.21.15.27:0/4280224807 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fe40c0529a0 con 0x7fe4180f6860 2021-11-03T18:40:52.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:52 smithi115 ceph-mon[65453]: pgmap v2588: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:52.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:52 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:40:52.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:52 smithi027 ceph-mon[97244]: pgmap v2588: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:52.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:52 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:40:52.942 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.942+0000 7fe41f04b700 1 -- 172.21.15.27:0/4280224807 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe4180fdd80 con 0x7fe4180f6860 2021-11-03T18:40:52.942 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.942+0000 7fe4157fa700 1 -- 172.21.15.27:0/4280224807 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fe40c052590 con 0x7fe4180f6860 2021-11-03T18:40:52.942 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:40:52.943 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:40:52.943 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:40:52.943 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:40:52.944 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:40:52.944 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:40:52.944 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:40:52.944 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:40:52.944 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:40:52.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:40:52.945 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:40:52.945 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:40:52.945 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:40:52.945 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:40:52.946 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:40:52.946 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:40:52.946 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:40:52.946 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:40:52.947 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.944+0000 7fe41f04b700 1 -- 172.21.15.27:0/4280224807 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe4000640b0 msgr2=0x7fe400066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:40:52.947 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.944+0000 7fe41f04b700 1 --2- 172.21.15.27:0/4280224807 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe4000640b0 0x7fe400066570 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 rx=0x7fe41000b6c0 tx=0x7fe410015040).stop 2021-11-03T18:40:52.948 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.944+0000 7fe41f04b700 1 -- 172.21.15.27:0/4280224807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4180f6860 msgr2=0x7fe4180f7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:40:52.948 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.944+0000 7fe41f04b700 1 --2- 172.21.15.27:0/4280224807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4180f6860 0x7fe4180f7ba0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 rx=0x7fe40c00e7f0 tx=0x7fe40c00b040).stop 2021-11-03T18:40:52.948 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.944+0000 7fe41f04b700 1 -- 172.21.15.27:0/4280224807 shutdown_connections 2021-11-03T18:40:52.948 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.944+0000 7fe41f04b700 1 --2- 172.21.15.27:0/4280224807 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe4000640b0 0x7fe400066570 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:52.948 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.944+0000 7fe41f04b700 1 --2- 172.21.15.27:0/4280224807 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4180f55c0 0x7fe4180f9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:52.949 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.944+0000 7fe41f04b700 1 --2- 172.21.15.27:0/4280224807 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4180f6860 0x7fe4180f7ba0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:52.949 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.944+0000 7fe41f04b700 1 -- 172.21.15.27:0/4280224807 >> 172.21.15.27:0/4280224807 conn(0x7fe4180f0d40 msgr2=0x7fe41805ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:40:52.949 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.944+0000 7fe41f04b700 1 -- 172.21.15.27:0/4280224807 shutdown_connections 2021-11-03T18:40:52.949 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:52.944+0000 7fe41f04b700 1 -- 172.21.15.27:0/4280224807 wait complete. 2021-11-03T18:40:53.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.045+0000 7f9e25224700 1 -- 172.21.15.27:0/3112116473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e200f5ef0 msgr2=0x7f9e200f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:40:53.044 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.045+0000 7f9e25224700 1 --2- 172.21.15.27:0/3112116473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e200f5ef0 0x7f9e200f6920 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 rx=0x7f9e14004660 tx=0x7f9e14009b30).stop 2021-11-03T18:40:53.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.045+0000 7f9e25224700 1 -- 172.21.15.27:0/3112116473 shutdown_connections 2021-11-03T18:40:53.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.045+0000 7f9e25224700 1 --2- 172.21.15.27:0/3112116473 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e200f5ef0 0x7f9e200f6920 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:53.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.045+0000 7f9e25224700 1 --2- 172.21.15.27:0/3112116473 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e200f55e0 0x7f9e200f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:53.045 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.045+0000 7f9e25224700 1 -- 172.21.15.27:0/3112116473 >> 172.21.15.27:0/3112116473 conn(0x7f9e200f0de0 msgr2=0x7f9e200f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:40:53.046 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.045+0000 7f9e25224700 1 -- 172.21.15.27:0/3112116473 shutdown_connections 2021-11-03T18:40:53.046 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.045+0000 7f9e25224700 1 -- 172.21.15.27:0/3112116473 wait complete. 2021-11-03T18:40:53.048 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.045+0000 7f9e25224700 1 Processor -- start 2021-11-03T18:40:53.048 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.045+0000 7f9e25224700 1 -- start start 2021-11-03T18:40:53.049 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.046+0000 7f9e25224700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e200f55e0 0x7f9e20109120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:40:53.049 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.046+0000 7f9e25224700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e200f5ef0 0x7f9e20109610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:40:53.049 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.046+0000 7f9e25224700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e20109bc0 con 0x7f9e200f5ef0 2021-11-03T18:40:53.049 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.046+0000 7f9e25224700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e20109d00 con 0x7f9e200f55e0 2021-11-03T18:40:53.050 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.046+0000 7f9e1e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e200f5ef0 0x7f9e20109610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:40:53.050 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.046+0000 7f9e1e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e200f5ef0 0x7f9e20109610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36264/0 (socket says 172.21.15.27:36264) 2021-11-03T18:40:53.050 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.046+0000 7f9e1ed9d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e200f55e0 0x7f9e20109120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:40:53.050 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.046+0000 7f9e1e59c700 1 -- 172.21.15.27:0/2148338721 learned_addr learned my addr 172.21.15.27:0/2148338721 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:40:53.050 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.046+0000 7f9e1e59c700 1 -- 172.21.15.27:0/2148338721 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e200f55e0 msgr2=0x7f9e20109120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:40:53.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.046+0000 7f9e1e59c700 1 --2- 172.21.15.27:0/2148338721 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e200f55e0 0x7f9e20109120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:53.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.046+0000 7f9e1e59c700 1 -- 172.21.15.27:0/2148338721 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e14005040 con 0x7f9e200f5ef0 2021-11-03T18:40:53.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.046+0000 7f9e1ed9d700 1 --2- 172.21.15.27:0/2148338721 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e200f55e0 0x7f9e20109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2021-11-03T18:40:53.051 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.046+0000 7f9e1e59c700 1 --2- 172.21.15.27:0/2148338721 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e200f5ef0 0x7f9e20109610 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 rx=0x7f9e14000c00 tx=0x7f9e1400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:40:53.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.046+0000 7f9e077fe700 1 -- 172.21.15.27:0/2148338721 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e14019070 con 0x7f9e200f5ef0 2021-11-03T18:40:53.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.046+0000 7f9e25224700 1 -- 172.21.15.27:0/2148338721 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e2010a640 con 0x7f9e200f5ef0 2021-11-03T18:40:53.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.046+0000 7f9e077fe700 1 -- 172.21.15.27:0/2148338721 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f9e14007e70 con 0x7f9e200f5ef0 2021-11-03T18:40:53.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.046+0000 7f9e077fe700 1 -- 172.21.15.27:0/2148338721 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9e14014ce0 con 0x7f9e200f5ef0 2021-11-03T18:40:53.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.047+0000 7f9e25224700 1 -- 172.21.15.27:0/2148338721 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e2010aba0 con 0x7f9e200f5ef0 2021-11-03T18:40:53.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.047+0000 7f9e25224700 1 -- 172.21.15.27:0/2148338721 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e20045ad0 con 0x7f9e200f5ef0 2021-11-03T18:40:53.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.048+0000 7f9e077fe700 1 -- 172.21.15.27:0/2148338721 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f9e14025070 con 0x7f9e200f5ef0 2021-11-03T18:40:53.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.048+0000 7f9e077fe700 1 --2- 172.21.15.27:0/2148338721 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9e080640b0 0x7f9e08066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:40:53.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.048+0000 7f9e077fe700 1 -- 172.21.15.27:0/2148338721 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f9e14081750 con 0x7f9e200f5ef0 2021-11-03T18:40:53.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.048+0000 7f9e1ed9d700 1 --2- 172.21.15.27:0/2148338721 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9e080640b0 0x7f9e08066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:40:53.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.049+0000 7f9e1ed9d700 1 --2- 172.21.15.27:0/2148338721 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9e080640b0 0x7f9e08066570 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 rx=0x7f9e10009fd0 tx=0x7f9e1000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:40:53.055 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.052+0000 7f9e077fe700 1 -- 172.21.15.27:0/2148338721 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f9e14053ab0 con 0x7f9e200f5ef0 2021-11-03T18:40:53.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.255+0000 7f9e25224700 1 -- 172.21.15.27:0/2148338721 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f9e2010b2a0 con 0x7f9e200f5ef0 2021-11-03T18:40:53.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.256+0000 7f9e077fe700 1 -- 172.21.15.27:0/2148338721 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f9e14023020 con 0x7f9e200f5ef0 2021-11-03T18:40:53.257 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:40:53.257 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:40:53.257 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:40:53.258 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:40:53.258 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:40:53.259 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:40:53.259 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:40:53.260 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:40:53.260 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:40:53.260 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:40:53.261 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:40:53.261 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:40:53.262 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:40:53.262 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:40:53.263 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:40:53.263 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:40:53.263 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:40:53.264 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:40:53.264 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:40:53.265 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:40:53.265 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:40:53.266 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:40:53.266 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:40:53.266 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:40:53.267 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:40:53.267 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:40:53.268 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:40:53.268 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:40:53.269 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:40:53.269 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:40:53.269 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:40:53.270 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:40:53.270 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:40:53.271 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:40:53.271 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:40:53.272 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:40:53.272 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:40:53.273 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:40:53.273 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:40:53.275 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.257+0000 7f9e25224700 1 -- 172.21.15.27:0/2148338721 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9e080640b0 msgr2=0x7f9e08066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:40:53.275 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.257+0000 7f9e25224700 1 --2- 172.21.15.27:0/2148338721 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9e080640b0 0x7f9e08066570 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 rx=0x7f9e10009fd0 tx=0x7f9e1000b040).stop 2021-11-03T18:40:53.276 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.257+0000 7f9e25224700 1 -- 172.21.15.27:0/2148338721 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e200f5ef0 msgr2=0x7f9e20109610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:40:53.276 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.257+0000 7f9e25224700 1 --2- 172.21.15.27:0/2148338721 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e200f5ef0 0x7f9e20109610 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 rx=0x7f9e14000c00 tx=0x7f9e1400b040).stop 2021-11-03T18:40:53.277 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.257+0000 7f9e25224700 1 -- 172.21.15.27:0/2148338721 shutdown_connections 2021-11-03T18:40:53.277 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.257+0000 7f9e25224700 1 --2- 172.21.15.27:0/2148338721 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f9e080640b0 0x7f9e08066570 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:53.278 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.257+0000 7f9e25224700 1 --2- 172.21.15.27:0/2148338721 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f9e200f55e0 0x7f9e20109120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:53.278 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.257+0000 7f9e25224700 1 --2- 172.21.15.27:0/2148338721 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f9e200f5ef0 0x7f9e20109610 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:40:53.279 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.257+0000 7f9e25224700 1 -- 172.21.15.27:0/2148338721 >> 172.21.15.27:0/2148338721 conn(0x7f9e200f0de0 msgr2=0x7f9e2005f720 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:40:53.279 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.257+0000 7f9e25224700 1 -- 172.21.15.27:0/2148338721 shutdown_connections 2021-11-03T18:40:53.279 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:40:53.257+0000 7f9e25224700 1 -- 172.21.15.27:0/2148338721 wait complete. 2021-11-03T18:40:53.280 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:40:53.958 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:53 smithi115 ceph-mon[65453]: from='client.36760 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:40:53.959 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:53 smithi115 ceph-mon[65453]: poll took 10027.190 ms: 1 events 2021-11-03T18:40:53.959 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:53 smithi115 ceph-mon[65453]: from='client.36764 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:40:53.959 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:53 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/4280224807' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:53.960 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:53 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2148338721' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:40:54.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:53 smithi027 ceph-mon[97244]: from='client.36760 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:40:54.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:53 smithi027 ceph-mon[97244]: poll took 10027.190 ms: 1 events 2021-11-03T18:40:54.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:53 smithi027 ceph-mon[97244]: from='client.36764 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:40:54.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:53 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/4280224807' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:54.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:53 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2148338721' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:40:54.908 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:54 smithi115 ceph-mon[65453]: pgmap v2589: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:55.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:54 smithi027 ceph-mon[97244]: pgmap v2589: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:55.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5097.482501 secs 2021-11-03T18:40:55.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:40:55.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:40:55.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:40:55.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:55.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:55.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:55.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:56.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5097.482501 secs 2021-11-03T18:40:56.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:40:56.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:40:56.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:40:56.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:56.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:56.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:56.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:40:56.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:56 smithi115 ceph-mon[65453]: pgmap v2590: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:56.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:56 smithi115 ceph-mon[65453]: poll took 2591.710 ms: 1 events 2021-11-03T18:40:56.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:56 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:40:56.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:40:56.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:40:57.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:56 smithi027 ceph-mon[97244]: pgmap v2590: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:57.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:56 smithi027 ceph-mon[97244]: poll took 2591.710 ms: 1 events 2021-11-03T18:40:57.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:56 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:40:57.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:40:57.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:40:58.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:40:58 smithi115 ceph-mon[65453]: pgmap v2591: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:40:59.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:40:58 smithi027 ceph-mon[97244]: pgmap v2591: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:00.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:00 smithi115 ceph-mon[65453]: pgmap v2592: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:00.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5102.482634 secs 2021-11-03T18:41:01.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:00 smithi027 ceph-mon[97244]: pgmap v2592: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:01.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5102.482634 secs 2021-11-03T18:41:02.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:02 smithi027 ceph-mon[97244]: pgmap v2593: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:02.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:02 smithi115 ceph-mon[65453]: pgmap v2593: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:04.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:04 smithi115 ceph-mon[65453]: pgmap v2594: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:05.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:04 smithi027 ceph-mon[97244]: pgmap v2594: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:05.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5107.482758 secs 2021-11-03T18:41:05.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:41:05.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:41:05.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:41:05.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:05.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:05.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:05.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:05 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:06.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5107.482758 secs 2021-11-03T18:41:06.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:41:06.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:41:06.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:41:06.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:06.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:06.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:06.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:05 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:06.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:06 smithi115 ceph-mon[65453]: pgmap v2595: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:06.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:06 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:41:07.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:06 smithi027 ceph-mon[97244]: pgmap v2595: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:07.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:06 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:41:08.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:08 smithi115 ceph-mon[65453]: pgmap v2596: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:09.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:08 smithi027 ceph-mon[97244]: pgmap v2596: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:10.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:10 smithi115 ceph-mon[65453]: pgmap v2597: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:10.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5112.482912 secs 2021-11-03T18:41:11.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:10 smithi027 ceph-mon[97244]: pgmap v2597: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:11.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5112.482912 secs 2021-11-03T18:41:12.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:12 smithi027 ceph-mon[97244]: pgmap v2598: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:12.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:12 smithi115 ceph-mon[65453]: pgmap v2598: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:14.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:14 smithi115 ceph-mon[65453]: pgmap v2599: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:15.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:14 smithi027 ceph-mon[97244]: pgmap v2599: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:15.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5117.483052 secs 2021-11-03T18:41:16.132 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:41:16.132 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:41:16.132 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:41:16.133 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:16.133 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:16.133 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:16.133 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:15 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:16.134 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5117.483052 secs 2021-11-03T18:41:16.134 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:41:16.135 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:41:16.135 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:41:16.135 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:16.135 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:16.135 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:16.136 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:15 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:16.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:16 smithi115 ceph-mon[65453]: pgmap v2600: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:16.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:16 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:41:17.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:16 smithi027 ceph-mon[97244]: pgmap v2600: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:17.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:16 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:41:18.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:18 smithi115 ceph-mon[65453]: pgmap v2601: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:19.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:18 smithi027 ceph-mon[97244]: pgmap v2601: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:20.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:20 smithi115 ceph-mon[65453]: pgmap v2602: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:20.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5122.483170 secs 2021-11-03T18:41:21.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:20 smithi027 ceph-mon[97244]: pgmap v2602: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:21.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5122.483170 secs 2021-11-03T18:41:22.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:22 smithi027 ceph-mon[97244]: pgmap v2603: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:22.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:22 smithi115 ceph-mon[65453]: pgmap v2603: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:23.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.372+0000 7fdd1e504700 1 -- 172.21.15.27:0/4014711373 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd180f55e0 msgr2=0x7fdd180f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:41:23.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.372+0000 7fdd1e504700 1 --2- 172.21.15.27:0/4014711373 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd180f55e0 0x7fdd180f5a00 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 rx=0x7fdd08004660 tx=0x7fdd08009b30).stop 2021-11-03T18:41:23.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.372+0000 7fdd1e504700 1 -- 172.21.15.27:0/4014711373 shutdown_connections 2021-11-03T18:41:23.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.372+0000 7fdd1e504700 1 --2- 172.21.15.27:0/4014711373 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd180f5ef0 0x7fdd180f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:23.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.372+0000 7fdd1e504700 1 --2- 172.21.15.27:0/4014711373 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd180f55e0 0x7fdd180f5a00 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:23.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.372+0000 7fdd1e504700 1 -- 172.21.15.27:0/4014711373 >> 172.21.15.27:0/4014711373 conn(0x7fdd180f0de0 msgr2=0x7fdd180f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:41:23.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.373+0000 7fdd1e504700 1 -- 172.21.15.27:0/4014711373 shutdown_connections 2021-11-03T18:41:23.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.373+0000 7fdd1e504700 1 -- 172.21.15.27:0/4014711373 wait complete. 2021-11-03T18:41:23.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.373+0000 7fdd1e504700 1 Processor -- start 2021-11-03T18:41:23.375 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.373+0000 7fdd1e504700 1 -- start start 2021-11-03T18:41:23.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.374+0000 7fdd1e504700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd180f55e0 0x7fdd18109cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:41:23.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.374+0000 7fdd1e504700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd180f5ef0 0x7fdd1810c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:41:23.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.374+0000 7fdd1e504700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd180604c0 con 0x7fdd180f5ef0 2021-11-03T18:41:23.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.374+0000 7fdd1e504700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd18060600 con 0x7fdd180f55e0 2021-11-03T18:41:23.376 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.374+0000 7fdd177fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd180f5ef0 0x7fdd1810c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:41:23.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.374+0000 7fdd177fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd180f5ef0 0x7fdd1810c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36282/0 (socket says 172.21.15.27:36282) 2021-11-03T18:41:23.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.374+0000 7fdd177fe700 1 -- 172.21.15.27:0/4303132 learned_addr learned my addr 172.21.15.27:0/4303132 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:41:23.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.374+0000 7fdd17fff700 1 --2- 172.21.15.27:0/4303132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd180f55e0 0x7fdd18109cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:41:23.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.374+0000 7fdd177fe700 1 -- 172.21.15.27:0/4303132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd180f55e0 msgr2=0x7fdd18109cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:41:23.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.374+0000 7fdd177fe700 1 --2- 172.21.15.27:0/4303132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd180f55e0 0x7fdd18109cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:23.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.374+0000 7fdd177fe700 1 -- 172.21.15.27:0/4303132 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd08005040 con 0x7fdd180f5ef0 2021-11-03T18:41:23.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.374+0000 7fdd177fe700 1 --2- 172.21.15.27:0/4303132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd180f5ef0 0x7fdd1810c1b0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 rx=0x7fdd0c00c530 tx=0x7fdd0c007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:41:23.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.374+0000 7fdd14ff9700 1 -- 172.21.15.27:0/4303132 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd0c00f650 con 0x7fdd180f5ef0 2021-11-03T18:41:23.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.374+0000 7fdd14ff9700 1 -- 172.21.15.27:0/4303132 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fdd0c00fc60 con 0x7fdd180f5ef0 2021-11-03T18:41:23.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.374+0000 7fdd1e504700 1 -- 172.21.15.27:0/4303132 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd1810c760 con 0x7fdd180f5ef0 2021-11-03T18:41:23.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.374+0000 7fdd14ff9700 1 -- 172.21.15.27:0/4303132 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdd0c012c20 con 0x7fdd180f5ef0 2021-11-03T18:41:23.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.375+0000 7fdd1e504700 1 -- 172.21.15.27:0/4303132 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd1810ccf0 con 0x7fdd180f5ef0 2021-11-03T18:41:23.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.375+0000 7fdd1e504700 1 -- 172.21.15.27:0/4303132 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd18045ad0 con 0x7fdd180f5ef0 2021-11-03T18:41:23.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.376+0000 7fdd14ff9700 1 -- 172.21.15.27:0/4303132 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fdd0c010070 con 0x7fdd180f5ef0 2021-11-03T18:41:23.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.377+0000 7fdd14ff9700 1 --2- 172.21.15.27:0/4303132 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdd000640b0 0x7fdd00066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:41:23.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.377+0000 7fdd17fff700 1 --2- 172.21.15.27:0/4303132 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdd000640b0 0x7fdd00066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:41:23.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.377+0000 7fdd14ff9700 1 -- 172.21.15.27:0/4303132 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fdd0c080540 con 0x7fdd180f5ef0 2021-11-03T18:41:23.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.377+0000 7fdd17fff700 1 --2- 172.21.15.27:0/4303132 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdd000640b0 0x7fdd00066570 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7fdd08004630 tx=0x7fdd08009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:41:23.381 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.381+0000 7fdd14ff9700 1 -- 172.21.15.27:0/4303132 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fdd0c052870 con 0x7fdd180f5ef0 2021-11-03T18:41:23.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.559+0000 7fdd1e504700 1 -- 172.21.15.27:0/4303132 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd18057f10 con 0x7fdd000640b0 2021-11-03T18:41:23.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.560+0000 7fdd14ff9700 1 -- 172.21.15.27:0/4303132 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fdd18057f10 con 0x7fdd000640b0 2021-11-03T18:41:23.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.562+0000 7fdd1e504700 1 -- 172.21.15.27:0/4303132 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdd000640b0 msgr2=0x7fdd00066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:41:23.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.562+0000 7fdd1e504700 1 --2- 172.21.15.27:0/4303132 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdd000640b0 0x7fdd00066570 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7fdd08004630 tx=0x7fdd08009b30).stop 2021-11-03T18:41:23.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.562+0000 7fdd1e504700 1 -- 172.21.15.27:0/4303132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd180f5ef0 msgr2=0x7fdd1810c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:41:23.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.562+0000 7fdd1e504700 1 --2- 172.21.15.27:0/4303132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd180f5ef0 0x7fdd1810c1b0 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 rx=0x7fdd0c00c530 tx=0x7fdd0c007e20).stop 2021-11-03T18:41:23.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.562+0000 7fdd1e504700 1 -- 172.21.15.27:0/4303132 shutdown_connections 2021-11-03T18:41:23.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.562+0000 7fdd1e504700 1 --2- 172.21.15.27:0/4303132 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdd000640b0 0x7fdd00066570 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:23.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.562+0000 7fdd1e504700 1 --2- 172.21.15.27:0/4303132 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdd180f55e0 0x7fdd18109cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:23.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.562+0000 7fdd1e504700 1 --2- 172.21.15.27:0/4303132 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdd180f5ef0 0x7fdd1810c1b0 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:23.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.562+0000 7fdd1e504700 1 -- 172.21.15.27:0/4303132 >> 172.21.15.27:0/4303132 conn(0x7fdd180f0de0 msgr2=0x7fdd180fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:41:23.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.562+0000 7fdd1e504700 1 -- 172.21.15.27:0/4303132 shutdown_connections 2021-11-03T18:41:23.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.562+0000 7fdd1e504700 1 -- 172.21.15.27:0/4303132 wait complete. 2021-11-03T18:41:23.570 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:41:23.662 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.662+0000 7fec2d7b6700 1 -- 172.21.15.27:0/1683384658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec280f77d0 msgr2=0x7fec280f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:41:23.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.662+0000 7fec2d7b6700 1 --2- 172.21.15.27:0/1683384658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec280f77d0 0x7fec280f7c30 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 rx=0x7fec1c004660 tx=0x7fec1c009b30).stop 2021-11-03T18:41:23.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.662+0000 7fec2d7b6700 1 -- 172.21.15.27:0/1683384658 shutdown_connections 2021-11-03T18:41:23.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.662+0000 7fec2d7b6700 1 --2- 172.21.15.27:0/1683384658 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec280f77d0 0x7fec280f7c30 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:23.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.662+0000 7fec2d7b6700 1 --2- 172.21.15.27:0/1683384658 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec280f6530 0x7fec280f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:23.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.662+0000 7fec2d7b6700 1 -- 172.21.15.27:0/1683384658 >> 172.21.15.27:0/1683384658 conn(0x7fec280f1c90 msgr2=0x7fec280f40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:41:23.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.662+0000 7fec2d7b6700 1 -- 172.21.15.27:0/1683384658 shutdown_connections 2021-11-03T18:41:23.665 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.662+0000 7fec2d7b6700 1 -- 172.21.15.27:0/1683384658 wait complete. 2021-11-03T18:41:23.665 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.663+0000 7fec2d7b6700 1 Processor -- start 2021-11-03T18:41:23.665 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.663+0000 7fec2d7b6700 1 -- start start 2021-11-03T18:41:23.665 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.663+0000 7fec2d7b6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec280f6530 0x7fec28106d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:41:23.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.663+0000 7fec2d7b6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec280f77d0 0x7fec28107280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:41:23.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.663+0000 7fec2d7b6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec28107880 con 0x7fec280f6530 2021-11-03T18:41:23.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.663+0000 7fec2d7b6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec28108090 con 0x7fec280f77d0 2021-11-03T18:41:23.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.663+0000 7fec27fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec280f6530 0x7fec28106d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:41:23.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.663+0000 7fec27fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec280f6530 0x7fec28106d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36290/0 (socket says 172.21.15.27:36290) 2021-11-03T18:41:23.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.663+0000 7fec27fff700 1 -- 172.21.15.27:0/2952908502 learned_addr learned my addr 172.21.15.27:0/2952908502 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:41:23.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.663+0000 7fec277fe700 1 --2- 172.21.15.27:0/2952908502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec280f77d0 0x7fec28107280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:41:23.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.664+0000 7fec27fff700 1 -- 172.21.15.27:0/2952908502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec280f77d0 msgr2=0x7fec28107280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:41:23.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.664+0000 7fec27fff700 1 --2- 172.21.15.27:0/2952908502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec280f77d0 0x7fec28107280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:23.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.664+0000 7fec27fff700 1 -- 172.21.15.27:0/2952908502 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec1c005040 con 0x7fec280f6530 2021-11-03T18:41:23.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.664+0000 7fec277fe700 1 --2- 172.21.15.27:0/2952908502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec280f77d0 0x7fec28107280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:41:23.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.664+0000 7fec27fff700 1 --2- 172.21.15.27:0/2952908502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec280f6530 0x7fec28106d90 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 rx=0x7fec18002700 tx=0x7fec18009d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:41:23.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.664+0000 7fec24ff9700 1 -- 172.21.15.27:0/2952908502 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec18009320 con 0x7fec280f6530 2021-11-03T18:41:23.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.664+0000 7fec24ff9700 1 -- 172.21.15.27:0/2952908502 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fec1800f040 con 0x7fec280f6530 2021-11-03T18:41:23.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.664+0000 7fec2d7b6700 1 -- 172.21.15.27:0/2952908502 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec28108310 con 0x7fec280f6530 2021-11-03T18:41:23.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.664+0000 7fec24ff9700 1 -- 172.21.15.27:0/2952908502 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fec18014900 con 0x7fec280f6530 2021-11-03T18:41:23.670 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.664+0000 7fec2d7b6700 1 -- 172.21.15.27:0/2952908502 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec281088d0 con 0x7fec280f6530 2021-11-03T18:41:23.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.665+0000 7fec2d7b6700 1 -- 172.21.15.27:0/2952908502 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec28059100 con 0x7fec280f6530 2021-11-03T18:41:23.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.665+0000 7fec24ff9700 1 -- 172.21.15.27:0/2952908502 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fec18009480 con 0x7fec280f6530 2021-11-03T18:41:23.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.666+0000 7fec24ff9700 1 --2- 172.21.15.27:0/2952908502 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fec10064100 0x7fec100665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:41:23.671 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.666+0000 7fec24ff9700 1 -- 172.21.15.27:0/2952908502 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fec18080a80 con 0x7fec280f6530 2021-11-03T18:41:23.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.666+0000 7fec277fe700 1 --2- 172.21.15.27:0/2952908502 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fec10064100 0x7fec100665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:41:23.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.666+0000 7fec277fe700 1 --2- 172.21.15.27:0/2952908502 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fec10064100 0x7fec100665c0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 rx=0x7fec1c004e30 tx=0x7fec1c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:41:23.672 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.670+0000 7fec24ff9700 1 -- 172.21.15.27:0/2952908502 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fec18052de0 con 0x7fec280f6530 2021-11-03T18:41:23.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.844+0000 7fec2d7b6700 1 -- 172.21.15.27:0/2952908502 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fec280008d0 con 0x7fec10064100 2021-11-03T18:41:23.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.858+0000 7fec24ff9700 1 -- 172.21.15.27:0/2952908502 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fec280008d0 con 0x7fec10064100 2021-11-03T18:41:23.857 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:41:23.857 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (92m) 88s ago 94m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:41:23.857 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (84m) 88s ago 94m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:41:23.857 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (84m) 61s ago 93m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:41:23.858 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (92m) 88s ago 93m 42.9M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:41:23.858 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (86m) 88s ago 89m 26.9M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:41:23.858 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (89m) 88s ago 89m 3852M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:41:23.858 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (85m) 61s ago 89m 26.5M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:41:23.859 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (86m) 61s ago 89m 27.6M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:41:23.859 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (86m) 88s ago 95m 625M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:41:23.859 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (86m) 61s ago 93m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:41:23.859 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (85m) 88s ago 95m 387M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:41:23.860 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (85m) 61s ago 93m 175M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:41:23.860 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (93m) 88s ago 93m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:41:23.860 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (93m) 61s ago 93m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:41:23.860 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (84m) 88s ago 92m 433M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:41:23.861 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (83m) 88s ago 92m 471M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:41:23.861 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (83m) 88s ago 91m 356M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:41:23.861 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (82m) 61s ago 91m 504M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:41:23.861 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (82m) 61s ago 91m 396M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:41:23.862 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (81m) 61s ago 90m 408M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:41:23.862 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (92m) 88s ago 93m 66.8M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:41:23.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.860+0000 7fec2d7b6700 1 -- 172.21.15.27:0/2952908502 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fec10064100 msgr2=0x7fec100665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:41:23.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.860+0000 7fec2d7b6700 1 --2- 172.21.15.27:0/2952908502 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fec10064100 0x7fec100665c0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 rx=0x7fec1c004e30 tx=0x7fec1c009b30).stop 2021-11-03T18:41:23.864 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.860+0000 7fec2d7b6700 1 -- 172.21.15.27:0/2952908502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec280f6530 msgr2=0x7fec28106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:41:23.864 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.860+0000 7fec2d7b6700 1 --2- 172.21.15.27:0/2952908502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec280f6530 0x7fec28106d90 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 rx=0x7fec18002700 tx=0x7fec18009d80).stop 2021-11-03T18:41:23.864 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.860+0000 7fec2d7b6700 1 -- 172.21.15.27:0/2952908502 shutdown_connections 2021-11-03T18:41:23.864 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.860+0000 7fec2d7b6700 1 --2- 172.21.15.27:0/2952908502 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fec10064100 0x7fec100665c0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:23.865 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.860+0000 7fec2d7b6700 1 --2- 172.21.15.27:0/2952908502 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fec280f6530 0x7fec28106d90 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:23.865 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.860+0000 7fec2d7b6700 1 --2- 172.21.15.27:0/2952908502 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fec280f77d0 0x7fec28107280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:23.865 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.860+0000 7fec2d7b6700 1 -- 172.21.15.27:0/2952908502 >> 172.21.15.27:0/2952908502 conn(0x7fec280f1c90 msgr2=0x7fec280fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:41:23.865 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.860+0000 7fec2d7b6700 1 -- 172.21.15.27:0/2952908502 shutdown_connections 2021-11-03T18:41:23.865 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.861+0000 7fec2d7b6700 1 -- 172.21.15.27:0/2952908502 wait complete. 2021-11-03T18:41:23.961 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.961+0000 7fb7a9dc7700 1 -- 172.21.15.27:0/811521725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7a40f6860 msgr2=0x7fb7a40f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:41:23.961 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.961+0000 7fb7a9dc7700 1 --2- 172.21.15.27:0/811521725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7a40f6860 0x7fb7a40f6cc0 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 rx=0x7fb798004660 tx=0x7fb798009b30).stop 2021-11-03T18:41:23.961 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.961+0000 7fb7a9dc7700 1 -- 172.21.15.27:0/811521725 shutdown_connections 2021-11-03T18:41:23.962 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.961+0000 7fb7a9dc7700 1 --2- 172.21.15.27:0/811521725 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7a40f6860 0x7fb7a40f6cc0 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:23.962 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.961+0000 7fb7a9dc7700 1 --2- 172.21.15.27:0/811521725 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7a40f55c0 0x7fb7a40f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:23.962 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.961+0000 7fb7a9dc7700 1 -- 172.21.15.27:0/811521725 >> 172.21.15.27:0/811521725 conn(0x7fb7a40f0d40 msgr2=0x7fb7a40f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:41:23.963 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.961+0000 7fb7a9dc7700 1 -- 172.21.15.27:0/811521725 shutdown_connections 2021-11-03T18:41:23.963 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.961+0000 7fb7a9dc7700 1 -- 172.21.15.27:0/811521725 wait complete. 2021-11-03T18:41:23.963 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.962+0000 7fb7a9dc7700 1 Processor -- start 2021-11-03T18:41:23.963 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.962+0000 7fb7a9dc7700 1 -- start start 2021-11-03T18:41:23.964 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.962+0000 7fb7a9dc7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7a40f55c0 0x7fb7a4106ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:41:23.964 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.962+0000 7fb7a9dc7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7a40f6860 0x7fb7a41073c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:41:23.964 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.962+0000 7fb7a9dc7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a4107970 con 0x7fb7a40f6860 2021-11-03T18:41:23.964 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.962+0000 7fb7a9dc7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a4107ab0 con 0x7fb7a40f55c0 2021-11-03T18:41:23.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.962+0000 7fb7a2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7a40f6860 0x7fb7a41073c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:41:23.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.962+0000 7fb7a2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7a40f6860 0x7fb7a41073c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36302/0 (socket says 172.21.15.27:36302) 2021-11-03T18:41:23.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.962+0000 7fb7a2ffd700 1 -- 172.21.15.27:0/2047115035 learned_addr learned my addr 172.21.15.27:0/2047115035 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:41:23.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.962+0000 7fb7a37fe700 1 --2- 172.21.15.27:0/2047115035 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7a40f55c0 0x7fb7a4106ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:41:23.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.962+0000 7fb7a2ffd700 1 -- 172.21.15.27:0/2047115035 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7a40f55c0 msgr2=0x7fb7a4106ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:41:23.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.962+0000 7fb7a2ffd700 1 --2- 172.21.15.27:0/2047115035 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7a40f55c0 0x7fb7a4106ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:23.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.962+0000 7fb7a2ffd700 1 -- 172.21.15.27:0/2047115035 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb798005040 con 0x7fb7a40f6860 2021-11-03T18:41:23.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.962+0000 7fb7a2ffd700 1 --2- 172.21.15.27:0/2047115035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7a40f6860 0x7fb7a41073c0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 rx=0x7fb798009990 tx=0x7fb798009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:41:23.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.963+0000 7fb78bfff700 1 -- 172.21.15.27:0/2047115035 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb798018070 con 0x7fb7a40f6860 2021-11-03T18:41:23.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.963+0000 7fb78bfff700 1 -- 172.21.15.27:0/2047115035 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fb798007ba0 con 0x7fb7a40f6860 2021-11-03T18:41:23.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.963+0000 7fb7a9dc7700 1 -- 172.21.15.27:0/2047115035 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a41083f0 con 0x7fb7a40f6860 2021-11-03T18:41:23.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.963+0000 7fb78bfff700 1 -- 172.21.15.27:0/2047115035 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb79801c3c0 con 0x7fb7a40f6860 2021-11-03T18:41:23.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.963+0000 7fb7a9dc7700 1 -- 172.21.15.27:0/2047115035 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a4108950 con 0x7fb7a40f6860 2021-11-03T18:41:23.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.964+0000 7fb7a9dc7700 1 -- 172.21.15.27:0/2047115035 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7a4045ad0 con 0x7fb7a40f6860 2021-11-03T18:41:23.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.964+0000 7fb78bfff700 1 -- 172.21.15.27:0/2047115035 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fb79801d420 con 0x7fb7a40f6860 2021-11-03T18:41:23.970 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.964+0000 7fb78bfff700 1 --2- 172.21.15.27:0/2047115035 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb78c0640b0 0x7fb78c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:41:23.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.965+0000 7fb78bfff700 1 -- 172.21.15.27:0/2047115035 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fb798082410 con 0x7fb7a40f6860 2021-11-03T18:41:23.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.965+0000 7fb7a37fe700 1 --2- 172.21.15.27:0/2047115035 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb78c0640b0 0x7fb78c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:41:23.971 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.965+0000 7fb7a37fe700 1 --2- 172.21.15.27:0/2047115035 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb78c0640b0 0x7fb78c066570 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 rx=0x7fb7a40f66c0 tx=0x7fb79400b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:41:23.972 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:23.969+0000 7fb78bfff700 1 -- 172.21.15.27:0/2047115035 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fb798054830 con 0x7fb7a40f6860 2021-11-03T18:41:24.197 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.199+0000 7fb7a9dc7700 1 -- 172.21.15.27:0/2047115035 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb7a40fdd80 con 0x7fb7a40f6860 2021-11-03T18:41:24.197 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.199+0000 7fb78bfff700 1 -- 172.21.15.27:0/2047115035 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fb798054420 con 0x7fb7a40f6860 2021-11-03T18:41:24.198 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:41:24.198 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:41:24.198 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:41:24.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:41:24.199 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:41:24.199 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:41:24.199 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:41:24.199 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:41:24.200 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:41:24.200 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:41:24.200 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:41:24.200 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:41:24.201 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:41:24.201 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:41:24.201 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:41:24.201 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:41:24.201 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:41:24.202 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:41:24.203 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.201+0000 7fb7a9dc7700 1 -- 172.21.15.27:0/2047115035 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb78c0640b0 msgr2=0x7fb78c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:41:24.203 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.201+0000 7fb7a9dc7700 1 --2- 172.21.15.27:0/2047115035 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb78c0640b0 0x7fb78c066570 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 rx=0x7fb7a40f66c0 tx=0x7fb79400b040).stop 2021-11-03T18:41:24.203 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.201+0000 7fb7a9dc7700 1 -- 172.21.15.27:0/2047115035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7a40f6860 msgr2=0x7fb7a41073c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:41:24.203 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.201+0000 7fb7a9dc7700 1 --2- 172.21.15.27:0/2047115035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7a40f6860 0x7fb7a41073c0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 rx=0x7fb798009990 tx=0x7fb798009b30).stop 2021-11-03T18:41:24.204 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.201+0000 7fb7a9dc7700 1 -- 172.21.15.27:0/2047115035 shutdown_connections 2021-11-03T18:41:24.204 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.201+0000 7fb7a9dc7700 1 --2- 172.21.15.27:0/2047115035 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb78c0640b0 0x7fb78c066570 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:24.204 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.201+0000 7fb7a9dc7700 1 --2- 172.21.15.27:0/2047115035 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb7a40f55c0 0x7fb7a4106ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:24.204 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.201+0000 7fb7a9dc7700 1 --2- 172.21.15.27:0/2047115035 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb7a40f6860 0x7fb7a41073c0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:24.204 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.201+0000 7fb7a9dc7700 1 -- 172.21.15.27:0/2047115035 >> 172.21.15.27:0/2047115035 conn(0x7fb7a40f0d40 msgr2=0x7fb7a405f620 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:41:24.205 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.201+0000 7fb7a9dc7700 1 -- 172.21.15.27:0/2047115035 shutdown_connections 2021-11-03T18:41:24.205 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.201+0000 7fb7a9dc7700 1 -- 172.21.15.27:0/2047115035 wait complete. 2021-11-03T18:41:24.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.301+0000 7f75b72ae700 1 -- 172.21.15.27:0/2339246940 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75b00f66c0 msgr2=0x7f75b00f6ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:41:24.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.301+0000 7f75b72ae700 1 --2- 172.21.15.27:0/2339246940 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75b00f66c0 0x7f75b00f6ae0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 rx=0x7f75a0004660 tx=0x7f75a0009b30).stop 2021-11-03T18:41:24.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.301+0000 7f75b72ae700 1 -- 172.21.15.27:0/2339246940 shutdown_connections 2021-11-03T18:41:24.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.301+0000 7f75b72ae700 1 --2- 172.21.15.27:0/2339246940 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75b00f7960 0x7f75b00f7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:24.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.301+0000 7f75b72ae700 1 --2- 172.21.15.27:0/2339246940 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75b00f66c0 0x7f75b00f6ae0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:24.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.301+0000 7f75b72ae700 1 -- 172.21.15.27:0/2339246940 >> 172.21.15.27:0/2339246940 conn(0x7f75b00f1e20 msgr2=0x7f75b00f4240 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:41:24.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.301+0000 7f75b72ae700 1 -- 172.21.15.27:0/2339246940 shutdown_connections 2021-11-03T18:41:24.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.301+0000 7f75b72ae700 1 -- 172.21.15.27:0/2339246940 wait complete. 2021-11-03T18:41:24.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.302+0000 7f75b72ae700 1 Processor -- start 2021-11-03T18:41:24.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.302+0000 7f75b72ae700 1 -- start start 2021-11-03T18:41:24.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.302+0000 7f75b72ae700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75b00f66c0 0x7f75b006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:41:24.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.302+0000 7f75b72ae700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75b00f7960 0x7f75b0069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:41:24.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.302+0000 7f75b72ae700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b006a380 con 0x7f75b00f7960 2021-11-03T18:41:24.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.302+0000 7f75b72ae700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b006a4c0 con 0x7f75b00f66c0 2021-11-03T18:41:24.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.302+0000 7f75b4849700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75b00f7960 0x7f75b0069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:41:24.304 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.302+0000 7f75b4849700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75b00f7960 0x7f75b0069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36310/0 (socket says 172.21.15.27:36310) 2021-11-03T18:41:24.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.302+0000 7f75b4849700 1 -- 172.21.15.27:0/4269693043 learned_addr learned my addr 172.21.15.27:0/4269693043 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:41:24.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.302+0000 7f75b504a700 1 --2- 172.21.15.27:0/4269693043 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75b00f66c0 0x7f75b006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:41:24.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.302+0000 7f75b4849700 1 -- 172.21.15.27:0/4269693043 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75b00f66c0 msgr2=0x7f75b006b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:41:24.305 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.302+0000 7f75b4849700 1 --2- 172.21.15.27:0/4269693043 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75b00f66c0 0x7f75b006b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:24.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.302+0000 7f75b4849700 1 -- 172.21.15.27:0/4269693043 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75a0005040 con 0x7f75b00f7960 2021-11-03T18:41:24.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.302+0000 7f75b504a700 1 --2- 172.21.15.27:0/4269693043 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75b00f66c0 0x7f75b006b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:41:24.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.303+0000 7f75b4849700 1 --2- 172.21.15.27:0/4269693043 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75b00f7960 0x7f75b0069e20 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 rx=0x7f75ac0026d0 tx=0x7f75ac009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:41:24.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.303+0000 7f75a5ffb700 1 -- 172.21.15.27:0/4269693043 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75ac009320 con 0x7f75b00f7960 2021-11-03T18:41:24.306 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.303+0000 7f75a5ffb700 1 -- 172.21.15.27:0/4269693043 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f75ac00f040 con 0x7f75b00f7960 2021-11-03T18:41:24.307 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.303+0000 7f75b72ae700 1 -- 172.21.15.27:0/4269693043 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75b006a7a0 con 0x7f75b00f7960 2021-11-03T18:41:24.307 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.303+0000 7f75a5ffb700 1 -- 172.21.15.27:0/4269693043 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75ac0147e0 con 0x7f75b00f7960 2021-11-03T18:41:24.307 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.303+0000 7f75b72ae700 1 -- 172.21.15.27:0/4269693043 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75b010efc0 con 0x7f75b00f7960 2021-11-03T18:41:24.308 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.304+0000 7f75b72ae700 1 -- 172.21.15.27:0/4269693043 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75b0045ad0 con 0x7f75b00f7960 2021-11-03T18:41:24.309 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.304+0000 7f75a5ffb700 1 -- 172.21.15.27:0/4269693043 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f75ac009480 con 0x7f75b00f7960 2021-11-03T18:41:24.309 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.305+0000 7f75a5ffb700 1 --2- 172.21.15.27:0/4269693043 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f759c0640b0 0x7f759c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:41:24.309 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.305+0000 7f75a5ffb700 1 -- 172.21.15.27:0/4269693043 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f75ac080df0 con 0x7f75b00f7960 2021-11-03T18:41:24.309 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.305+0000 7f75b504a700 1 --2- 172.21.15.27:0/4269693043 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f759c0640b0 0x7f759c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:41:24.310 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.305+0000 7f75b504a700 1 --2- 172.21.15.27:0/4269693043 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f759c0640b0 0x7f759c066570 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 rx=0x7f75a0004e30 tx=0x7f75a0009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:41:24.310 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.309+0000 7f75a5ffb700 1 -- 172.21.15.27:0/4269693043 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f75ac053210 con 0x7f75b00f7960 2021-11-03T18:41:24.510 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.512+0000 7f75b72ae700 1 -- 172.21.15.27:0/4269693043 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f75b006a930 con 0x7f75b00f7960 2021-11-03T18:41:24.510 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.512+0000 7f75a5ffb700 1 -- 172.21.15.27:0/4269693043 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f75b006a930 con 0x7f75b00f7960 2021-11-03T18:41:24.511 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:41:24.511 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:41:24.511 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:41:24.512 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:41:24.512 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:41:24.512 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:41:24.512 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:41:24.512 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:41:24.513 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:41:24.513 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:41:24.513 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:41:24.513 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:41:24.514 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:41:24.514 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:41:24.514 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:41:24.514 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:41:24.514 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:41:24.515 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:41:24.515 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:41:24.515 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:41:24.515 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:41:24.515 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:41:24.516 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:41:24.516 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:41:24.516 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:41:24.516 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:41:24.517 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:41:24.517 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:41:24.517 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:41:24.517 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:41:24.517 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:41:24.518 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:41:24.518 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:41:24.518 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:41:24.518 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:41:24.518 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:41:24.519 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:41:24.519 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:41:24.519 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:41:24.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.514+0000 7f75b72ae700 1 -- 172.21.15.27:0/4269693043 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f759c0640b0 msgr2=0x7f759c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:41:24.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.514+0000 7f75b72ae700 1 --2- 172.21.15.27:0/4269693043 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f759c0640b0 0x7f759c066570 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 rx=0x7f75a0004e30 tx=0x7f75a0009b30).stop 2021-11-03T18:41:24.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.514+0000 7f75b72ae700 1 -- 172.21.15.27:0/4269693043 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75b00f7960 msgr2=0x7f75b0069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:41:24.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.514+0000 7f75b72ae700 1 --2- 172.21.15.27:0/4269693043 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75b00f7960 0x7f75b0069e20 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 rx=0x7f75ac0026d0 tx=0x7f75ac009de0).stop 2021-11-03T18:41:24.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.514+0000 7f75b72ae700 1 -- 172.21.15.27:0/4269693043 shutdown_connections 2021-11-03T18:41:24.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.514+0000 7f75b72ae700 1 --2- 172.21.15.27:0/4269693043 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f759c0640b0 0x7f759c066570 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:24.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.514+0000 7f75b72ae700 1 --2- 172.21.15.27:0/4269693043 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f75b00f66c0 0x7f75b006b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:24.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.514+0000 7f75b72ae700 1 --2- 172.21.15.27:0/4269693043 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f75b00f7960 0x7f75b0069e20 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:24.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.514+0000 7f75b72ae700 1 -- 172.21.15.27:0/4269693043 >> 172.21.15.27:0/4269693043 conn(0x7f75b00f1e20 msgr2=0x7f75b00fb3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:41:24.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.514+0000 7f75b72ae700 1 -- 172.21.15.27:0/4269693043 shutdown_connections 2021-11-03T18:41:24.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:24.514+0000 7f75b72ae700 1 -- 172.21.15.27:0/4269693043 wait complete. 2021-11-03T18:41:24.523 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:41:24.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:24 smithi115 ceph-mon[65453]: pgmap v2604: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:24.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:24 smithi115 ceph-mon[65453]: from='client.36776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:41:24.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:24 smithi115 ceph-mon[65453]: from='client.36780 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:41:24.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:24 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2047115035' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:24.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:24 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/4269693043' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:41:25.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:24 smithi027 ceph-mon[97244]: pgmap v2604: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:25.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:24 smithi027 ceph-mon[97244]: from='client.36776 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:41:25.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:24 smithi027 ceph-mon[97244]: from='client.36780 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:41:25.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:24 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2047115035' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:25.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:24 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/4269693043' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:41:25.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5127.483255 secs 2021-11-03T18:41:25.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:41:25.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:41:25.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:41:25.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:25.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:25.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:25.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:25 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:26.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5127.483255 secs 2021-11-03T18:41:26.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:41:26.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:41:26.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:41:26.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:26.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:26.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:26.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:25 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:26.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:26 smithi115 ceph-mon[65453]: pgmap v2605: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:26.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:26 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:41:27.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:26 smithi027 ceph-mon[97244]: pgmap v2605: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:27.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:26 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:41:28.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:28 smithi115 ceph-mon[65453]: pgmap v2606: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:29.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:28 smithi027 ceph-mon[97244]: pgmap v2606: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:30.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:30 smithi115 ceph-mon[65453]: pgmap v2607: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:30.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5132.483399 secs 2021-11-03T18:41:31.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:30 smithi027 ceph-mon[97244]: pgmap v2607: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:31.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5132.483399 secs 2021-11-03T18:41:32.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:32 smithi027 ceph-mon[97244]: pgmap v2608: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:32.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:32 smithi115 ceph-mon[65453]: pgmap v2608: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:34.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:34 smithi115 ceph-mon[65453]: pgmap v2609: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:35.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:34 smithi027 ceph-mon[97244]: pgmap v2609: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:35.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5137.483532 secs 2021-11-03T18:41:35.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:41:35.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:41:35.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:41:35.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:35.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:35.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:35.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:35 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:36.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5137.483532 secs 2021-11-03T18:41:36.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:41:36.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:41:36.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:41:36.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:36.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:36.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:36.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:35 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:36.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:36 smithi115 ceph-mon[65453]: pgmap v2610: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:36.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:36 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:41:37.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:36 smithi027 ceph-mon[97244]: pgmap v2610: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:37.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:36 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:41:38.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:38 smithi115 ceph-mon[65453]: pgmap v2611: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:39.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:38 smithi027 ceph-mon[97244]: pgmap v2611: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:40.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:40 smithi115 ceph-mon[65453]: pgmap v2612: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:40.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5142.483650 secs 2021-11-03T18:41:41.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:40 smithi027 ceph-mon[97244]: pgmap v2612: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:41.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5142.483650 secs 2021-11-03T18:41:42.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:42 smithi115 ceph-mon[65453]: pgmap v2613: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:43.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:42 smithi027 ceph-mon[97244]: pgmap v2613: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:44.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:44 smithi115 ceph-mon[65453]: pgmap v2614: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:45.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:44 smithi027 ceph-mon[97244]: pgmap v2614: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:45.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5147.483760 secs 2021-11-03T18:41:45.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:45 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:41:45.819 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5147.483760 secs 2021-11-03T18:41:45.819 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:45 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:41:47.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:46 smithi115 ceph-mon[65453]: pgmap v2615: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:47.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:46 smithi115 ceph-mon[65453]: poll took 50153.171 ms: 1 events 2021-11-03T18:41:47.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:41:47.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:41:47.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:47.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:47.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:47.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:47.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:46 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:41:47.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:46 smithi027 ceph-mon[97244]: pgmap v2615: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:47.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:46 smithi027 ceph-mon[97244]: poll took 50153.171 ms: 1 events 2021-11-03T18:41:47.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:41:47.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:41:47.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:47.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:47.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:47.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:47.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:46 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:41:49.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:48 smithi115 ceph-mon[65453]: pgmap v2616: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:49.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:48 smithi027 ceph-mon[97244]: pgmap v2616: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:51.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:50 smithi115 ceph-mon[65453]: pgmap v2617: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:51.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5152.483903 secs 2021-11-03T18:41:51.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:50 smithi027 ceph-mon[97244]: pgmap v2617: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:51.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5152.483903 secs 2021-11-03T18:41:53.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:52 smithi115 ceph-mon[65453]: pgmap v2618: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:53.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:52 smithi027 ceph-mon[97244]: pgmap v2618: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:54.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.629+0000 7fe2b5c6c700 1 -- 172.21.15.27:0/3782140008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2b00f6530 msgr2=0x7fe2b00f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:41:54.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.629+0000 7fe2b5c6c700 1 --2- 172.21.15.27:0/3782140008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2b00f6530 0x7fe2b00f6950 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 rx=0x7fe2a4004660 tx=0x7fe2a4009b30).stop 2021-11-03T18:41:54.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.629+0000 7fe2b5c6c700 1 -- 172.21.15.27:0/3782140008 shutdown_connections 2021-11-03T18:41:54.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.629+0000 7fe2b5c6c700 1 --2- 172.21.15.27:0/3782140008 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2b00f77d0 0x7fe2b00f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:54.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.629+0000 7fe2b5c6c700 1 --2- 172.21.15.27:0/3782140008 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2b00f6530 0x7fe2b00f6950 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:54.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.629+0000 7fe2b5c6c700 1 -- 172.21.15.27:0/3782140008 >> 172.21.15.27:0/3782140008 conn(0x7fe2b00f1cd0 msgr2=0x7fe2b00f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:41:54.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.629+0000 7fe2b5c6c700 1 -- 172.21.15.27:0/3782140008 shutdown_connections 2021-11-03T18:41:54.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.629+0000 7fe2b5c6c700 1 -- 172.21.15.27:0/3782140008 wait complete. 2021-11-03T18:41:54.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.629+0000 7fe2b5c6c700 1 Processor -- start 2021-11-03T18:41:54.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.629+0000 7fe2b5c6c700 1 -- start start 2021-11-03T18:41:54.632 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.630+0000 7fe2b5c6c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2b00f6530 0x7fe2b0106db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:41:54.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.630+0000 7fe2b5c6c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2b00f77d0 0x7fe2b01072a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:41:54.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.630+0000 7fe2b5c6c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b01078a0 con 0x7fe2b00f6530 2021-11-03T18:41:54.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.630+0000 7fe2b5c6c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b01080e0 con 0x7fe2b00f77d0 2021-11-03T18:41:54.633 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.630+0000 7fe2b4c6a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2b00f6530 0x7fe2b0106db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:41:54.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.630+0000 7fe2b4c6a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2b00f6530 0x7fe2b0106db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36326/0 (socket says 172.21.15.27:36326) 2021-11-03T18:41:54.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.630+0000 7fe2b4c6a700 1 -- 172.21.15.27:0/2286532316 learned_addr learned my addr 172.21.15.27:0/2286532316 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:41:54.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.630+0000 7fe2affff700 1 --2- 172.21.15.27:0/2286532316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2b00f77d0 0x7fe2b01072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:41:54.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.630+0000 7fe2b4c6a700 1 -- 172.21.15.27:0/2286532316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2b00f77d0 msgr2=0x7fe2b01072a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:41:54.634 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.630+0000 7fe2b4c6a700 1 --2- 172.21.15.27:0/2286532316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2b00f77d0 0x7fe2b01072a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:54.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.630+0000 7fe2b4c6a700 1 -- 172.21.15.27:0/2286532316 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2a4005040 con 0x7fe2b00f6530 2021-11-03T18:41:54.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.630+0000 7fe2b4c6a700 1 --2- 172.21.15.27:0/2286532316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2b00f6530 0x7fe2b0106db0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 rx=0x7fe2a4013a70 tx=0x7fe2a4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:41:54.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.630+0000 7fe2ad7fa700 1 -- 172.21.15.27:0/2286532316 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2a4018070 con 0x7fe2b00f6530 2021-11-03T18:41:54.635 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.631+0000 7fe2ad7fa700 1 -- 172.21.15.27:0/2286532316 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fe2a40097d0 con 0x7fe2b00f6530 2021-11-03T18:41:54.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.631+0000 7fe2b5c6c700 1 -- 172.21.15.27:0/2286532316 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b0108360 con 0x7fe2b00f6530 2021-11-03T18:41:54.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.631+0000 7fe2ad7fa700 1 -- 172.21.15.27:0/2286532316 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2a401de80 con 0x7fe2b00f6530 2021-11-03T18:41:54.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.631+0000 7fe2b5c6c700 1 -- 172.21.15.27:0/2286532316 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b0108840 con 0x7fe2b00f6530 2021-11-03T18:41:54.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.632+0000 7fe2b5c6c700 1 -- 172.21.15.27:0/2286532316 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2b0059100 con 0x7fe2b00f6530 2021-11-03T18:41:54.637 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.632+0000 7fe2ad7fa700 1 -- 172.21.15.27:0/2286532316 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fe2a4007ba0 con 0x7fe2b00f6530 2021-11-03T18:41:54.637 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.632+0000 7fe2ad7fa700 1 --2- 172.21.15.27:0/2286532316 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe2a0064100 0x7fe2a00665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:41:54.637 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.633+0000 7fe2ad7fa700 1 -- 172.21.15.27:0/2286532316 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fe2a4082430 con 0x7fe2b00f6530 2021-11-03T18:41:54.638 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.633+0000 7fe2affff700 1 --2- 172.21.15.27:0/2286532316 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe2a0064100 0x7fe2a00665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:41:54.638 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.633+0000 7fe2affff700 1 --2- 172.21.15.27:0/2286532316 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe2a0064100 0x7fe2a00665c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7fe2b0106b40 tx=0x7fe29c00b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:41:54.639 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.637+0000 7fe2ad7fa700 1 -- 172.21.15.27:0/2286532316 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fe2a4054850 con 0x7fe2b00f6530 2021-11-03T18:41:54.813 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.815+0000 7fe2b5c6c700 1 -- 172.21.15.27:0/2286532316 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2b00fee00 con 0x7fe2a0064100 2021-11-03T18:41:54.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.816+0000 7fe2ad7fa700 1 -- 172.21.15.27:0/2286532316 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fe2b00fee00 con 0x7fe2a0064100 2021-11-03T18:41:54.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.818+0000 7fe2b5c6c700 1 -- 172.21.15.27:0/2286532316 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe2a0064100 msgr2=0x7fe2a00665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:41:54.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.818+0000 7fe2b5c6c700 1 --2- 172.21.15.27:0/2286532316 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe2a0064100 0x7fe2a00665c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7fe2b0106b40 tx=0x7fe29c00b040).stop 2021-11-03T18:41:54.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.818+0000 7fe2b5c6c700 1 -- 172.21.15.27:0/2286532316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2b00f6530 msgr2=0x7fe2b0106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:41:54.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.818+0000 7fe2b5c6c700 1 --2- 172.21.15.27:0/2286532316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2b00f6530 0x7fe2b0106db0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 rx=0x7fe2a4013a70 tx=0x7fe2a4009b30).stop 2021-11-03T18:41:54.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.818+0000 7fe2b5c6c700 1 -- 172.21.15.27:0/2286532316 shutdown_connections 2021-11-03T18:41:54.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.818+0000 7fe2b5c6c700 1 --2- 172.21.15.27:0/2286532316 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe2a0064100 0x7fe2a00665c0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:54.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.818+0000 7fe2b5c6c700 1 --2- 172.21.15.27:0/2286532316 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe2b00f6530 0x7fe2b0106db0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:54.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.818+0000 7fe2b5c6c700 1 --2- 172.21.15.27:0/2286532316 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe2b00f77d0 0x7fe2b01072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:54.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.818+0000 7fe2b5c6c700 1 -- 172.21.15.27:0/2286532316 >> 172.21.15.27:0/2286532316 conn(0x7fe2b00f1cd0 msgr2=0x7fe2b00fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:41:54.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.818+0000 7fe2b5c6c700 1 -- 172.21.15.27:0/2286532316 shutdown_connections 2021-11-03T18:41:54.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.818+0000 7fe2b5c6c700 1 -- 172.21.15.27:0/2286532316 wait complete. 2021-11-03T18:41:54.826 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:41:54.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.919+0000 7f4352cdc700 1 -- 172.21.15.27:0/379156660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f434c0f55c0 msgr2=0x7f434c0f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:41:54.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.919+0000 7f4352cdc700 1 --2- 172.21.15.27:0/379156660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f434c0f55c0 0x7f434c0f59e0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 rx=0x7f4340004660 tx=0x7f4340009b30).stop 2021-11-03T18:41:54.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.920+0000 7f4352cdc700 1 -- 172.21.15.27:0/379156660 shutdown_connections 2021-11-03T18:41:54.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.920+0000 7f4352cdc700 1 --2- 172.21.15.27:0/379156660 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f434c0f5ed0 0x7f434c0f6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:54.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.920+0000 7f4352cdc700 1 --2- 172.21.15.27:0/379156660 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f434c0f55c0 0x7f434c0f59e0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:54.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.920+0000 7f4352cdc700 1 -- 172.21.15.27:0/379156660 >> 172.21.15.27:0/379156660 conn(0x7f434c0f0dc0 msgr2=0x7f434c0f31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:41:54.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.920+0000 7f4352cdc700 1 -- 172.21.15.27:0/379156660 shutdown_connections 2021-11-03T18:41:54.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.920+0000 7f4352cdc700 1 -- 172.21.15.27:0/379156660 wait complete. 2021-11-03T18:41:54.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.920+0000 7f4352cdc700 1 Processor -- start 2021-11-03T18:41:54.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.920+0000 7f4352cdc700 1 -- start start 2021-11-03T18:41:54.923 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.920+0000 7f4352cdc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f434c0f55c0 0x7f434c109c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:41:54.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.920+0000 7f4352cdc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f434c0f5ed0 0x7f434c10c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:41:54.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.920+0000 7f4352cdc700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434c060560 con 0x7f434c0f5ed0 2021-11-03T18:41:54.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.920+0000 7f4352cdc700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434c0606a0 con 0x7f434c0f55c0 2021-11-03T18:41:54.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.921+0000 7f434bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f434c0f5ed0 0x7f434c10c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:41:54.925 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.921+0000 7f434bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f434c0f5ed0 0x7f434c10c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36338/0 (socket says 172.21.15.27:36338) 2021-11-03T18:41:54.925 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.921+0000 7f434bfff700 1 -- 172.21.15.27:0/1472399497 learned_addr learned my addr 172.21.15.27:0/1472399497 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:41:54.925 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.921+0000 7f4350a78700 1 --2- 172.21.15.27:0/1472399497 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f434c0f55c0 0x7f434c109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:41:54.926 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.921+0000 7f434bfff700 1 -- 172.21.15.27:0/1472399497 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f434c0f55c0 msgr2=0x7f434c109c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:41:54.926 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.921+0000 7f434bfff700 1 --2- 172.21.15.27:0/1472399497 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f434c0f55c0 0x7f434c109c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:54.926 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.921+0000 7f434bfff700 1 -- 172.21.15.27:0/1472399497 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4340005040 con 0x7f434c0f5ed0 2021-11-03T18:41:54.926 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.921+0000 7f434bfff700 1 --2- 172.21.15.27:0/1472399497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f434c0f5ed0 0x7f434c10c170 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 rx=0x7f43380026d0 tx=0x7f4338009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:41:54.927 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.921+0000 7f43497fa700 1 -- 172.21.15.27:0/1472399497 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4338009320 con 0x7f434c0f5ed0 2021-11-03T18:41:54.927 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.921+0000 7f4352cdc700 1 -- 172.21.15.27:0/1472399497 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f434c10c720 con 0x7f434c0f5ed0 2021-11-03T18:41:54.927 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.921+0000 7f43497fa700 1 -- 172.21.15.27:0/1472399497 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f433800f040 con 0x7f434c0f5ed0 2021-11-03T18:41:54.927 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.921+0000 7f43497fa700 1 -- 172.21.15.27:0/1472399497 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43380147e0 con 0x7f434c0f5ed0 2021-11-03T18:41:54.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.921+0000 7f4352cdc700 1 -- 172.21.15.27:0/1472399497 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f434c10cce0 con 0x7f434c0f5ed0 2021-11-03T18:41:54.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.922+0000 7f4352cdc700 1 -- 172.21.15.27:0/1472399497 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f434c045ad0 con 0x7f434c0f5ed0 2021-11-03T18:41:54.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.922+0000 7f43497fa700 1 -- 172.21.15.27:0/1472399497 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f4338009480 con 0x7f434c0f5ed0 2021-11-03T18:41:54.929 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.923+0000 7f43497fa700 1 --2- 172.21.15.27:0/1472399497 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f433c0640b0 0x7f433c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:41:54.929 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.923+0000 7f43497fa700 1 -- 172.21.15.27:0/1472399497 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f4338080eb0 con 0x7f434c0f5ed0 2021-11-03T18:41:54.929 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.923+0000 7f4350a78700 1 --2- 172.21.15.27:0/1472399497 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f433c0640b0 0x7f433c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:41:54.929 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.923+0000 7f4350a78700 1 --2- 172.21.15.27:0/1472399497 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f433c0640b0 0x7f433c066570 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 rx=0x7f4340004630 tx=0x7f4340009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:41:54.930 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:54.927+0000 7f43497fa700 1 -- 172.21.15.27:0/1472399497 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f4338053210 con 0x7f434c0f5ed0 2021-11-03T18:41:55.098 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.100+0000 7f4352cdc700 1 -- 172.21.15.27:0/1472399497 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f434c0fdf20 con 0x7f433c0640b0 2021-11-03T18:41:55.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.114+0000 7f43497fa700 1 -- 172.21.15.27:0/1472399497 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f434c0fdf20 con 0x7f433c0640b0 2021-11-03T18:41:55.113 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:41:55.113 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (93m) 119s ago 94m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:41:55.113 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (85m) 119s ago 94m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:41:55.113 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (85m) 93s ago 93m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:41:55.114 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (93m) 119s ago 94m 42.9M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:41:55.114 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (86m) 119s ago 89m 26.9M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:41:55.114 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (89m) 119s ago 89m 3852M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:41:55.114 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (85m) 93s ago 89m 26.5M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:41:55.115 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (87m) 93s ago 89m 27.6M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:41:55.115 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (87m) 119s ago 96m 625M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:41:55.115 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (86m) 93s ago 93m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:41:55.115 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (86m) 119s ago 96m 387M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:41:55.116 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (85m) 93s ago 93m 175M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:41:55.116 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (94m) 119s ago 94m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:41:55.116 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (93m) 93s ago 93m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:41:55.116 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (84m) 119s ago 93m 433M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:41:55.116 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (84m) 119s ago 92m 471M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:41:55.117 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (83m) 119s ago 92m 356M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:41:55.117 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (83m) 93s ago 92m 504M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:41:55.117 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (82m) 93s ago 91m 396M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:41:55.117 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (82m) 93s ago 91m 408M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:41:55.118 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (93m) 119s ago 94m 66.8M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:41:55.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.116+0000 7f4352cdc700 1 -- 172.21.15.27:0/1472399497 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f433c0640b0 msgr2=0x7f433c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:41:55.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.116+0000 7f4352cdc700 1 --2- 172.21.15.27:0/1472399497 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f433c0640b0 0x7f433c066570 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 rx=0x7f4340004630 tx=0x7f4340009b30).stop 2021-11-03T18:41:55.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.116+0000 7f4352cdc700 1 -- 172.21.15.27:0/1472399497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f434c0f5ed0 msgr2=0x7f434c10c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:41:55.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.116+0000 7f4352cdc700 1 --2- 172.21.15.27:0/1472399497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f434c0f5ed0 0x7f434c10c170 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 rx=0x7f43380026d0 tx=0x7f4338009de0).stop 2021-11-03T18:41:55.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.116+0000 7f4352cdc700 1 -- 172.21.15.27:0/1472399497 shutdown_connections 2021-11-03T18:41:55.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.116+0000 7f4352cdc700 1 --2- 172.21.15.27:0/1472399497 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f433c0640b0 0x7f433c066570 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:55.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.116+0000 7f4352cdc700 1 --2- 172.21.15.27:0/1472399497 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f434c0f55c0 0x7f434c109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:55.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.116+0000 7f4352cdc700 1 --2- 172.21.15.27:0/1472399497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f434c0f5ed0 0x7f434c10c170 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:55.121 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.116+0000 7f4352cdc700 1 -- 172.21.15.27:0/1472399497 >> 172.21.15.27:0/1472399497 conn(0x7f434c0f0dc0 msgr2=0x7f434c0fa350 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:41:55.121 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.116+0000 7f4352cdc700 1 -- 172.21.15.27:0/1472399497 shutdown_connections 2021-11-03T18:41:55.121 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.116+0000 7f4352cdc700 1 -- 172.21.15.27:0/1472399497 wait complete. 2021-11-03T18:41:55.218 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.218+0000 7fcbc3b50700 1 -- 172.21.15.27:0/3558189890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbbc0f5340 msgr2=0x7fcbbc0f7ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:41:55.218 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.218+0000 7fcbc3b50700 1 --2- 172.21.15.27:0/3558189890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbbc0f5340 0x7fcbbc0f7ac0 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 rx=0x7fcbb0004660 tx=0x7fcbb0009b30).stop 2021-11-03T18:41:55.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.218+0000 7fcbc3b50700 1 -- 172.21.15.27:0/3558189890 shutdown_connections 2021-11-03T18:41:55.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.218+0000 7fcbc3b50700 1 --2- 172.21.15.27:0/3558189890 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbbc0f5340 0x7fcbbc0f7ac0 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:55.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.218+0000 7fcbc3b50700 1 --2- 172.21.15.27:0/3558189890 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbbc0f49a0 0x7fcbbc0f4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:55.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.218+0000 7fcbc3b50700 1 -- 172.21.15.27:0/3558189890 >> 172.21.15.27:0/3558189890 conn(0x7fcbbc0f04c0 msgr2=0x7fcbbc0f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:41:55.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.218+0000 7fcbc3b50700 1 -- 172.21.15.27:0/3558189890 shutdown_connections 2021-11-03T18:41:55.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.218+0000 7fcbc3b50700 1 -- 172.21.15.27:0/3558189890 wait complete. 2021-11-03T18:41:55.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.218+0000 7fcbc3b50700 1 Processor -- start 2021-11-03T18:41:55.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.219+0000 7fcbc3b50700 1 -- start start 2021-11-03T18:41:55.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.219+0000 7fcbc3b50700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbbc0f49a0 0x7fcbbc106fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:41:55.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.219+0000 7fcbc3b50700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbbc0f5340 0x7fcbbc107490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:41:55.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.219+0000 7fcbc3b50700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc107a40 con 0x7fcbbc0f5340 2021-11-03T18:41:55.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.219+0000 7fcbc3b50700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbbc107b80 con 0x7fcbbc0f49a0 2021-11-03T18:41:55.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.219+0000 7fcbc10eb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbbc0f5340 0x7fcbbc107490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:41:55.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.219+0000 7fcbc10eb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbbc0f5340 0x7fcbbc107490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36348/0 (socket says 172.21.15.27:36348) 2021-11-03T18:41:55.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.219+0000 7fcbc10eb700 1 -- 172.21.15.27:0/1333742803 learned_addr learned my addr 172.21.15.27:0/1333742803 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:41:55.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.219+0000 7fcbc18ec700 1 --2- 172.21.15.27:0/1333742803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbbc0f49a0 0x7fcbbc106fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:41:55.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.219+0000 7fcbc10eb700 1 -- 172.21.15.27:0/1333742803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbbc0f49a0 msgr2=0x7fcbbc106fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:41:55.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.219+0000 7fcbc10eb700 1 --2- 172.21.15.27:0/1333742803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbbc0f49a0 0x7fcbbc106fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:55.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.219+0000 7fcbc10eb700 1 -- 172.21.15.27:0/1333742803 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbb0005040 con 0x7fcbbc0f5340 2021-11-03T18:41:55.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.219+0000 7fcbc10eb700 1 --2- 172.21.15.27:0/1333742803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbbc0f5340 0x7fcbbc107490 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 rx=0x7fcbb0004f60 tx=0x7fcbb0009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:41:55.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.219+0000 7fcbba7fc700 1 -- 172.21.15.27:0/1333742803 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbb0018070 con 0x7fcbbc0f5340 2021-11-03T18:41:55.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.219+0000 7fcbba7fc700 1 -- 172.21.15.27:0/1333742803 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fcbb0007ba0 con 0x7fcbbc0f5340 2021-11-03T18:41:55.230 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.219+0000 7fcbc3b50700 1 -- 172.21.15.27:0/1333742803 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbbc1084c0 con 0x7fcbbc0f5340 2021-11-03T18:41:55.230 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.219+0000 7fcbba7fc700 1 -- 172.21.15.27:0/1333742803 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcbb001c620 con 0x7fcbbc0f5340 2021-11-03T18:41:55.231 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.219+0000 7fcbc3b50700 1 -- 172.21.15.27:0/1333742803 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbbc108a20 con 0x7fcbbc0f5340 2021-11-03T18:41:55.232 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.220+0000 7fcbc3b50700 1 -- 172.21.15.27:0/1333742803 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbbc045ad0 con 0x7fcbbc0f5340 2021-11-03T18:41:55.232 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.221+0000 7fcbba7fc700 1 -- 172.21.15.27:0/1333742803 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fcbb001d420 con 0x7fcbbc0f5340 2021-11-03T18:41:55.232 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.221+0000 7fcbba7fc700 1 --2- 172.21.15.27:0/1333742803 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcba40640b0 0x7fcba4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:41:55.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.221+0000 7fcbba7fc700 1 -- 172.21.15.27:0/1333742803 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fcbb0082620 con 0x7fcbbc0f5340 2021-11-03T18:41:55.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.221+0000 7fcbc18ec700 1 --2- 172.21.15.27:0/1333742803 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcba40640b0 0x7fcba4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:41:55.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.222+0000 7fcbc18ec700 1 --2- 172.21.15.27:0/1333742803 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcba40640b0 0x7fcba4066570 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 rx=0x7fcbbc05f4c0 tx=0x7fcbb4009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:41:55.233 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.225+0000 7fcbba7fc700 1 -- 172.21.15.27:0/1333742803 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fcbb0054a40 con 0x7fcbbc0f5340 2021-11-03T18:41:55.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:54 smithi115 ceph-mon[65453]: pgmap v2619: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:55.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:54 smithi027 ceph-mon[97244]: pgmap v2619: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:55.452 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.453+0000 7fcbc3b50700 1 -- 172.21.15.27:0/1333742803 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcbbc0fefc0 con 0x7fcbbc0f5340 2021-11-03T18:41:55.453 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.454+0000 7fcbba7fc700 1 -- 172.21.15.27:0/1333742803 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fcbb0054630 con 0x7fcbbc0f5340 2021-11-03T18:41:55.454 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:41:55.454 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:41:55.454 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:41:55.455 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:41:55.455 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:41:55.455 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:41:55.456 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:41:55.456 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:41:55.456 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:41:55.457 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:41:55.457 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:41:55.457 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:41:55.458 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:41:55.458 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:41:55.458 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:41:55.458 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:41:55.459 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:41:55.459 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:41:55.460 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.456+0000 7fcbc3b50700 1 -- 172.21.15.27:0/1333742803 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcba40640b0 msgr2=0x7fcba4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:41:55.460 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.456+0000 7fcbc3b50700 1 --2- 172.21.15.27:0/1333742803 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcba40640b0 0x7fcba4066570 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 rx=0x7fcbbc05f4c0 tx=0x7fcbb4009040).stop 2021-11-03T18:41:55.460 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.456+0000 7fcbc3b50700 1 -- 172.21.15.27:0/1333742803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbbc0f5340 msgr2=0x7fcbbc107490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:41:55.461 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.456+0000 7fcbc3b50700 1 --2- 172.21.15.27:0/1333742803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbbc0f5340 0x7fcbbc107490 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 rx=0x7fcbb0004f60 tx=0x7fcbb0009b30).stop 2021-11-03T18:41:55.461 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.456+0000 7fcbc3b50700 1 -- 172.21.15.27:0/1333742803 shutdown_connections 2021-11-03T18:41:55.461 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.456+0000 7fcbc3b50700 1 --2- 172.21.15.27:0/1333742803 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fcba40640b0 0x7fcba4066570 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:55.461 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.456+0000 7fcbc3b50700 1 --2- 172.21.15.27:0/1333742803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fcbbc0f49a0 0x7fcbbc106fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:55.462 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.456+0000 7fcbc3b50700 1 --2- 172.21.15.27:0/1333742803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fcbbc0f5340 0x7fcbbc107490 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:55.462 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.456+0000 7fcbc3b50700 1 -- 172.21.15.27:0/1333742803 >> 172.21.15.27:0/1333742803 conn(0x7fcbbc0f04c0 msgr2=0x7fcbbc0fb510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:41:55.462 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.456+0000 7fcbc3b50700 1 -- 172.21.15.27:0/1333742803 shutdown_connections 2021-11-03T18:41:55.462 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.456+0000 7fcbc3b50700 1 -- 172.21.15.27:0/1333742803 wait complete. 2021-11-03T18:41:55.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.556+0000 7f2e89ba9700 1 -- 172.21.15.27:0/3876917796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e840f55e0 msgr2=0x7f2e840f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:41:55.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.556+0000 7f2e89ba9700 1 --2- 172.21.15.27:0/3876917796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e840f55e0 0x7f2e840f5a00 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 rx=0x7f2e74004660 tx=0x7f2e74009b30).stop 2021-11-03T18:41:55.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.556+0000 7f2e89ba9700 1 -- 172.21.15.27:0/3876917796 shutdown_connections 2021-11-03T18:41:55.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.556+0000 7f2e89ba9700 1 --2- 172.21.15.27:0/3876917796 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e840f5ef0 0x7f2e840f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:55.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.556+0000 7f2e89ba9700 1 --2- 172.21.15.27:0/3876917796 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e840f55e0 0x7f2e840f5a00 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:55.557 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.556+0000 7f2e89ba9700 1 -- 172.21.15.27:0/3876917796 >> 172.21.15.27:0/3876917796 conn(0x7f2e840f0de0 msgr2=0x7f2e840f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:41:55.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.556+0000 7f2e89ba9700 1 -- 172.21.15.27:0/3876917796 shutdown_connections 2021-11-03T18:41:55.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.556+0000 7f2e89ba9700 1 -- 172.21.15.27:0/3876917796 wait complete. 2021-11-03T18:41:55.558 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.556+0000 7f2e89ba9700 1 Processor -- start 2021-11-03T18:41:55.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.557+0000 7f2e89ba9700 1 -- start start 2021-11-03T18:41:55.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.557+0000 7f2e89ba9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e840f55e0 0x7f2e84109c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:41:55.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.557+0000 7f2e89ba9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e840f5ef0 0x7f2e8410c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:41:55.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.557+0000 7f2e89ba9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e840604e0 con 0x7f2e840f5ef0 2021-11-03T18:41:55.559 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.557+0000 7f2e89ba9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e84060620 con 0x7f2e840f55e0 2021-11-03T18:41:55.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.557+0000 7f2e82ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e840f5ef0 0x7f2e8410c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:41:55.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.557+0000 7f2e82ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e840f5ef0 0x7f2e8410c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36358/0 (socket says 172.21.15.27:36358) 2021-11-03T18:41:55.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.557+0000 7f2e82ffd700 1 -- 172.21.15.27:0/2225132951 learned_addr learned my addr 172.21.15.27:0/2225132951 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:41:55.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.557+0000 7f2e837fe700 1 --2- 172.21.15.27:0/2225132951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e840f55e0 0x7f2e84109c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:41:55.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.557+0000 7f2e82ffd700 1 -- 172.21.15.27:0/2225132951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e840f55e0 msgr2=0x7f2e84109c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:41:55.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.557+0000 7f2e82ffd700 1 --2- 172.21.15.27:0/2225132951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e840f55e0 0x7f2e84109c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:55.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.557+0000 7f2e82ffd700 1 -- 172.21.15.27:0/2225132951 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e74005040 con 0x7f2e840f5ef0 2021-11-03T18:41:55.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.557+0000 7f2e837fe700 1 --2- 172.21.15.27:0/2225132951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e840f55e0 0x7f2e84109c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:41:55.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.557+0000 7f2e82ffd700 1 --2- 172.21.15.27:0/2225132951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e840f5ef0 0x7f2e8410c110 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 rx=0x7f2e7800e7f0 tx=0x7f2e7800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:41:55.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.558+0000 7f2e6bfff700 1 -- 172.21.15.27:0/2225132951 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e780092e0 con 0x7f2e840f5ef0 2021-11-03T18:41:55.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.558+0000 7f2e89ba9700 1 -- 172.21.15.27:0/2225132951 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e8410c6c0 con 0x7f2e840f5ef0 2021-11-03T18:41:55.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.558+0000 7f2e6bfff700 1 -- 172.21.15.27:0/2225132951 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f2e7800f040 con 0x7f2e840f5ef0 2021-11-03T18:41:55.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.558+0000 7f2e6bfff700 1 -- 172.21.15.27:0/2225132951 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2e78014750 con 0x7f2e840f5ef0 2021-11-03T18:41:55.563 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.558+0000 7f2e89ba9700 1 -- 172.21.15.27:0/2225132951 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e8410cc50 con 0x7f2e840f5ef0 2021-11-03T18:41:55.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.559+0000 7f2e89ba9700 1 -- 172.21.15.27:0/2225132951 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e840f4a70 con 0x7f2e840f5ef0 2021-11-03T18:41:55.564 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.559+0000 7f2e6bfff700 1 -- 172.21.15.27:0/2225132951 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f2e7801b030 con 0x7f2e840f5ef0 2021-11-03T18:41:55.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.559+0000 7f2e6bfff700 1 --2- 172.21.15.27:0/2225132951 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2e6c0640b0 0x7f2e6c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:41:55.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.560+0000 7f2e837fe700 1 --2- 172.21.15.27:0/2225132951 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2e6c0640b0 0x7f2e6c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:41:55.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.560+0000 7f2e6bfff700 1 -- 172.21.15.27:0/2225132951 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f2e780806e0 con 0x7f2e840f5ef0 2021-11-03T18:41:55.565 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.560+0000 7f2e837fe700 1 --2- 172.21.15.27:0/2225132951 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2e6c0640b0 0x7f2e6c066570 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7f2e74005010 tx=0x7f2e7400b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:41:55.566 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.563+0000 7f2e6bfff700 1 -- 172.21.15.27:0/2225132951 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f2e78052940 con 0x7f2e840f5ef0 2021-11-03T18:41:55.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.771+0000 7f2e89ba9700 1 -- 172.21.15.27:0/2225132951 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f2e84045ad0 con 0x7f2e840f5ef0 2021-11-03T18:41:55.770 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.771+0000 7f2e6bfff700 1 -- 172.21.15.27:0/2225132951 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f2e78019070 con 0x7f2e840f5ef0 2021-11-03T18:41:55.770 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:41:55.770 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:41:55.771 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:41:55.771 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:41:55.771 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:41:55.771 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:41:55.772 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:41:55.772 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:41:55.772 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:41:55.773 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:41:55.773 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:41:55.773 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:41:55.774 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:41:55.774 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:41:55.774 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:41:55.774 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:41:55.775 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:41:55.775 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:41:55.775 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:41:55.775 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:41:55.776 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:41:55.776 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:41:55.776 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:41:55.776 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:41:55.776 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:41:55.777 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:41:55.777 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:41:55.777 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:41:55.777 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:41:55.778 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:41:55.778 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:41:55.778 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:41:55.778 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:41:55.779 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:41:55.779 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:41:55.779 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:41:55.779 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:41:55.780 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:41:55.780 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:41:55.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.773+0000 7f2e89ba9700 1 -- 172.21.15.27:0/2225132951 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2e6c0640b0 msgr2=0x7f2e6c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:41:55.781 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.773+0000 7f2e89ba9700 1 --2- 172.21.15.27:0/2225132951 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2e6c0640b0 0x7f2e6c066570 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7f2e74005010 tx=0x7f2e7400b040).stop 2021-11-03T18:41:55.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.773+0000 7f2e89ba9700 1 -- 172.21.15.27:0/2225132951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e840f5ef0 msgr2=0x7f2e8410c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:41:55.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.773+0000 7f2e89ba9700 1 --2- 172.21.15.27:0/2225132951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e840f5ef0 0x7f2e8410c110 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 rx=0x7f2e7800e7f0 tx=0x7f2e7800b040).stop 2021-11-03T18:41:55.782 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.773+0000 7f2e89ba9700 1 -- 172.21.15.27:0/2225132951 shutdown_connections 2021-11-03T18:41:55.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.773+0000 7f2e89ba9700 1 --2- 172.21.15.27:0/2225132951 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2e6c0640b0 0x7f2e6c066570 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:55.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.773+0000 7f2e89ba9700 1 --2- 172.21.15.27:0/2225132951 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2e840f55e0 0x7f2e84109c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:55.783 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.773+0000 7f2e89ba9700 1 --2- 172.21.15.27:0/2225132951 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2e840f5ef0 0x7f2e8410c110 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:41:55.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.773+0000 7f2e89ba9700 1 -- 172.21.15.27:0/2225132951 >> 172.21.15.27:0/2225132951 conn(0x7f2e840f0de0 msgr2=0x7f2e840fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:41:55.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.773+0000 7f2e89ba9700 1 -- 172.21.15.27:0/2225132951 shutdown_connections 2021-11-03T18:41:55.784 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:41:55.773+0000 7f2e89ba9700 1 -- 172.21.15.27:0/2225132951 wait complete. 2021-11-03T18:41:55.784 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:41:56.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5157.484038 secs 2021-11-03T18:41:56.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:55 smithi115 ceph-mon[65453]: from='client.36792 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:41:56.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:55 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1333742803' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:56.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:55 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2225132951' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:41:56.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:41:56.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:41:56.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5157.484038 secs 2021-11-03T18:41:56.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:55 smithi027 ceph-mon[97244]: from='client.36792 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:41:56.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:55 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1333742803' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:56.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:55 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2225132951' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:41:56.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:41:56.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:41:57.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:56 smithi115 ceph-mon[65453]: from='client.36796 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:41:57.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:56 smithi115 ceph-mon[65453]: pgmap v2620: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:57.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:41:57.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:41:57.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:41:57.241 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:57.241 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:57.242 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:57.242 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:57.242 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:56 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:41:57.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:56 smithi027 ceph-mon[97244]: from='client.36796 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:41:57.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:56 smithi027 ceph-mon[97244]: pgmap v2620: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:57.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:41:57.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:41:57.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:41:57.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:57.275 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:57.275 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:57.276 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:41:57.276 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:56 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:41:59.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:41:58 smithi115 ceph-mon[65453]: pgmap v2621: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:41:59.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:41:58 smithi027 ceph-mon[97244]: pgmap v2621: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:01.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:00 smithi115 ceph-mon[65453]: pgmap v2622: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:01.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5162.484162 secs 2021-11-03T18:42:01.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:00 smithi027 ceph-mon[97244]: pgmap v2622: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:01.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5162.484162 secs 2021-11-03T18:42:03.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:02 smithi115 ceph-mon[65453]: pgmap v2623: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:03.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:02 smithi027 ceph-mon[97244]: pgmap v2623: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:05.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:04 smithi115 ceph-mon[65453]: pgmap v2624: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:05.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:04 smithi027 ceph-mon[97244]: pgmap v2624: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:06.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5167.484308 secs 2021-11-03T18:42:06.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5167.484308 secs 2021-11-03T18:42:07.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:06 smithi115 ceph-mon[65453]: pgmap v2625: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:07.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:42:07.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:42:07.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:42:07.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:07.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:07.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:07.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:07.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:06 smithi027 ceph-mon[97244]: pgmap v2625: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:07.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:42:07.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:42:07.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:42:07.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:07.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:07.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:07.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:08.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:07 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:42:08.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:07 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:42:09.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:08 smithi115 ceph-mon[65453]: pgmap v2626: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:09.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:08 smithi027 ceph-mon[97244]: pgmap v2626: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:11.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:10 smithi115 ceph-mon[65453]: pgmap v2627: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:11.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5172.484448 secs 2021-11-03T18:42:11.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:10 smithi027 ceph-mon[97244]: pgmap v2627: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:11.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5172.484448 secs 2021-11-03T18:42:13.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:12 smithi115 ceph-mon[65453]: pgmap v2628: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:13.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:12 smithi027 ceph-mon[97244]: pgmap v2628: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:15.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:14 smithi115 ceph-mon[65453]: pgmap v2629: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:15.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:14 smithi027 ceph-mon[97244]: pgmap v2629: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:16.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5177.484604 secs 2021-11-03T18:42:16.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5177.484604 secs 2021-11-03T18:42:17.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:16 smithi115 ceph-mon[65453]: pgmap v2630: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:17.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:42:17.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:42:17.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:42:17.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:17.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:17.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:17.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:17.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:16 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:42:17.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:16 smithi027 ceph-mon[97244]: pgmap v2630: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:17.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:42:17.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:42:17.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:42:17.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:17.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:17.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:17.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:17.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:16 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:42:18.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:17 smithi115 ceph-mon[65453]: pgmap v2631: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:18.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:17 smithi027 ceph-mon[97244]: pgmap v2631: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:20.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:20 smithi115 ceph-mon[65453]: pgmap v2632: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:20.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5182.484727 secs 2021-11-03T18:42:20.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:20 smithi027 ceph-mon[97244]: pgmap v2632: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:20.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5182.484727 secs 2021-11-03T18:42:22.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:22 smithi027 ceph-mon[97244]: pgmap v2633: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:22.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:22 smithi115 ceph-mon[65453]: pgmap v2633: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:24.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:24 smithi115 ceph-mon[65453]: pgmap v2634: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:25.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:24 smithi027 ceph-mon[97244]: pgmap v2634: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:25.885 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:25.886+0000 7f22bb6a6700 1 -- 172.21.15.27:0/2739874948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22b40f55c0 msgr2=0x7f22b40f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:42:25.885 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:25.886+0000 7f22bb6a6700 1 --2- 172.21.15.27:0/2739874948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22b40f55c0 0x7f22b40f59e0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 rx=0x7f22a4004660 tx=0x7f22a4009b30).stop 2021-11-03T18:42:25.885 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:25.886+0000 7f22bb6a6700 1 -- 172.21.15.27:0/2739874948 shutdown_connections 2021-11-03T18:42:25.886 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:25.886+0000 7f22bb6a6700 1 --2- 172.21.15.27:0/2739874948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22b40f6860 0x7f22b40f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:25.886 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:25.886+0000 7f22bb6a6700 1 --2- 172.21.15.27:0/2739874948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22b40f55c0 0x7f22b40f59e0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:25.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:25.886+0000 7f22bb6a6700 1 -- 172.21.15.27:0/2739874948 >> 172.21.15.27:0/2739874948 conn(0x7f22b40f0d40 msgr2=0x7f22b40f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:42:25.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:25.886+0000 7f22bb6a6700 1 -- 172.21.15.27:0/2739874948 shutdown_connections 2021-11-03T18:42:25.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:25.886+0000 7f22bb6a6700 1 -- 172.21.15.27:0/2739874948 wait complete. 2021-11-03T18:42:25.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:25.887+0000 7f22bb6a6700 1 Processor -- start 2021-11-03T18:42:25.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:25.887+0000 7f22bb6a6700 1 -- start start 2021-11-03T18:42:25.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:25.887+0000 7f22bb6a6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22b40f55c0 0x7f22b4109c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:42:25.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:25.887+0000 7f22bb6a6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22b40f6860 0x7f22b410c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:42:25.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:25.887+0000 7f22bb6a6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b40603e0 con 0x7f22b40f55c0 2021-11-03T18:42:25.891 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:25.887+0000 7f22bb6a6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22b4060520 con 0x7f22b40f6860 2021-11-03T18:42:25.891 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:25.887+0000 7f22b9442700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22b40f55c0 0x7f22b4109c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:42:25.891 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:25.887+0000 7f22b9442700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22b40f55c0 0x7f22b4109c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36374/0 (socket says 172.21.15.27:36374) 2021-11-03T18:42:25.891 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:25.887+0000 7f22b9442700 1 -- 172.21.15.27:0/1268784762 learned_addr learned my addr 172.21.15.27:0/1268784762 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:42:25.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:25.887+0000 7f22b8c41700 1 --2- 172.21.15.27:0/1268784762 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22b40f6860 0x7f22b410c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:42:25.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:25.887+0000 7f22b9442700 1 -- 172.21.15.27:0/1268784762 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22b40f6860 msgr2=0x7f22b410c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:42:25.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:25.887+0000 7f22b9442700 1 --2- 172.21.15.27:0/1268784762 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22b40f6860 0x7f22b410c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:25.892 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:25.887+0000 7f22b9442700 1 -- 172.21.15.27:0/1268784762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a4005040 con 0x7f22b40f55c0 2021-11-03T18:42:25.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:25.888+0000 7f22b9442700 1 --2- 172.21.15.27:0/1268784762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22b40f55c0 0x7f22b4109c00 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 rx=0x7f22a4013a70 tx=0x7f22a4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:42:25.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:25.888+0000 7f22a9ffb700 1 -- 172.21.15.27:0/1268784762 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22a4018070 con 0x7f22b40f55c0 2021-11-03T18:42:25.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:25.888+0000 7f22a9ffb700 1 -- 172.21.15.27:0/1268784762 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f22a40097d0 con 0x7f22b40f55c0 2021-11-03T18:42:25.893 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:25.888+0000 7f22bb6a6700 1 -- 172.21.15.27:0/1268784762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22b410c650 con 0x7f22b40f55c0 2021-11-03T18:42:25.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:25.888+0000 7f22a9ffb700 1 -- 172.21.15.27:0/1268784762 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f22a401c3c0 con 0x7f22b40f55c0 2021-11-03T18:42:25.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:25.888+0000 7f22bb6a6700 1 -- 172.21.15.27:0/1268784762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22b410cbb0 con 0x7f22b40f55c0 2021-11-03T18:42:25.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:25.889+0000 7f22bb6a6700 1 -- 172.21.15.27:0/1268784762 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22b4045ad0 con 0x7f22b40f55c0 2021-11-03T18:42:25.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:25.890+0000 7f22a9ffb700 1 -- 172.21.15.27:0/1268784762 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f22a4007ba0 con 0x7f22b40f55c0 2021-11-03T18:42:25.894 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:25.890+0000 7f22a9ffb700 1 --2- 172.21.15.27:0/1268784762 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f22a00640b0 0x7f22a0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:42:25.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:25.890+0000 7f22a9ffb700 1 -- 172.21.15.27:0/1268784762 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f22a4082470 con 0x7f22b40f55c0 2021-11-03T18:42:25.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:25.890+0000 7f22b8c41700 1 --2- 172.21.15.27:0/1268784762 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f22a00640b0 0x7f22a0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:42:25.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:25.890+0000 7f22b8c41700 1 --2- 172.21.15.27:0/1268784762 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f22a00640b0 0x7f22a0066570 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 rx=0x7f22b00096c0 tx=0x7f22b0009be0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:42:25.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:25.894+0000 7f22a9ffb700 1 -- 172.21.15.27:0/1268784762 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f22a40547d0 con 0x7f22b40f55c0 2021-11-03T18:42:25.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5187.484843 secs 2021-11-03T18:42:26.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5187.484843 secs 2021-11-03T18:42:26.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.076+0000 7f22bb6a6700 1 -- 172.21.15.27:0/1268784762 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f22b40fdec0 con 0x7f22a00640b0 2021-11-03T18:42:26.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.077+0000 7f22a9ffb700 1 -- 172.21.15.27:0/1268784762 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f22b40fdec0 con 0x7f22a00640b0 2021-11-03T18:42:26.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.079+0000 7f22bb6a6700 1 -- 172.21.15.27:0/1268784762 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f22a00640b0 msgr2=0x7f22a0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:42:26.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.079+0000 7f22bb6a6700 1 --2- 172.21.15.27:0/1268784762 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f22a00640b0 0x7f22a0066570 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 rx=0x7f22b00096c0 tx=0x7f22b0009be0).stop 2021-11-03T18:42:26.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.079+0000 7f22bb6a6700 1 -- 172.21.15.27:0/1268784762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22b40f55c0 msgr2=0x7f22b4109c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:42:26.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.079+0000 7f22bb6a6700 1 --2- 172.21.15.27:0/1268784762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22b40f55c0 0x7f22b4109c00 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 rx=0x7f22a4013a70 tx=0x7f22a4009b30).stop 2021-11-03T18:42:26.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.079+0000 7f22bb6a6700 1 -- 172.21.15.27:0/1268784762 shutdown_connections 2021-11-03T18:42:26.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.079+0000 7f22bb6a6700 1 --2- 172.21.15.27:0/1268784762 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f22a00640b0 0x7f22a0066570 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:26.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.079+0000 7f22bb6a6700 1 --2- 172.21.15.27:0/1268784762 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f22b40f55c0 0x7f22b4109c00 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:26.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.079+0000 7f22bb6a6700 1 --2- 172.21.15.27:0/1268784762 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f22b40f6860 0x7f22b410c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:26.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.079+0000 7f22bb6a6700 1 -- 172.21.15.27:0/1268784762 >> 172.21.15.27:0/1268784762 conn(0x7f22b40f0d40 msgr2=0x7f22b40fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:42:26.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.079+0000 7f22bb6a6700 1 -- 172.21.15.27:0/1268784762 shutdown_connections 2021-11-03T18:42:26.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.080+0000 7f22bb6a6700 1 -- 172.21.15.27:0/1268784762 wait complete. 2021-11-03T18:42:26.087 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:42:26.179 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.180+0000 7fb531347700 1 -- 172.21.15.27:0/3107409879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb52c0f74a0 msgr2=0x7fb52c0f5930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:42:26.180 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.180+0000 7fb531347700 1 --2- 172.21.15.27:0/3107409879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb52c0f74a0 0x7fb52c0f5930 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 rx=0x7fb514004660 tx=0x7fb514009b30).stop 2021-11-03T18:42:26.180 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.180+0000 7fb531347700 1 -- 172.21.15.27:0/3107409879 shutdown_connections 2021-11-03T18:42:26.180 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.180+0000 7fb531347700 1 --2- 172.21.15.27:0/3107409879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb52c0f74a0 0x7fb52c0f5930 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:26.181 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.180+0000 7fb531347700 1 --2- 172.21.15.27:0/3107409879 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb52c0f6b90 0x7fb52c0f6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:26.181 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.180+0000 7fb531347700 1 -- 172.21.15.27:0/3107409879 >> 172.21.15.27:0/3107409879 conn(0x7fb52c0f0540 msgr2=0x7fb52c0f2960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:42:26.181 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.180+0000 7fb531347700 1 -- 172.21.15.27:0/3107409879 shutdown_connections 2021-11-03T18:42:26.181 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.180+0000 7fb531347700 1 -- 172.21.15.27:0/3107409879 wait complete. 2021-11-03T18:42:26.181 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.180+0000 7fb531347700 1 Processor -- start 2021-11-03T18:42:26.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.180+0000 7fb531347700 1 -- start start 2021-11-03T18:42:26.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.181+0000 7fb531347700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb52c0f6b90 0x7fb52c106ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:42:26.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.181+0000 7fb531347700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb52c0f74a0 0x7fb52c1073e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:42:26.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.181+0000 7fb531347700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c107990 con 0x7fb52c0f74a0 2021-11-03T18:42:26.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.181+0000 7fb531347700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c107ad0 con 0x7fb52c0f6b90 2021-11-03T18:42:26.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.181+0000 7fb52a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb52c0f74a0 0x7fb52c1073e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:42:26.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.181+0000 7fb52a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb52c0f74a0 0x7fb52c1073e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36386/0 (socket says 172.21.15.27:36386) 2021-11-03T18:42:26.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.181+0000 7fb52a7fc700 1 -- 172.21.15.27:0/1126541927 learned_addr learned my addr 172.21.15.27:0/1126541927 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:42:26.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.181+0000 7fb52a7fc700 1 -- 172.21.15.27:0/1126541927 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb52c0f6b90 msgr2=0x7fb52c106ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:42:26.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.181+0000 7fb52a7fc700 1 --2- 172.21.15.27:0/1126541927 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb52c0f6b90 0x7fb52c106ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:42:26.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.181+0000 7fb52a7fc700 1 -- 172.21.15.27:0/1126541927 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb514005040 con 0x7fb52c0f74a0 2021-11-03T18:42:26.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.181+0000 7fb52a7fc700 1 --2- 172.21.15.27:0/1126541927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb52c0f74a0 0x7fb52c1073e0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 rx=0x7fb514000c00 tx=0x7fb51400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:42:26.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.181+0000 7fb5237fe700 1 -- 172.21.15.27:0/1126541927 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb514019070 con 0x7fb52c0f74a0 2021-11-03T18:42:26.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.181+0000 7fb5237fe700 1 -- 172.21.15.27:0/1126541927 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fb514007e70 con 0x7fb52c0f74a0 2021-11-03T18:42:26.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.181+0000 7fb531347700 1 -- 172.21.15.27:0/1126541927 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb52c108410 con 0x7fb52c0f74a0 2021-11-03T18:42:26.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.181+0000 7fb5237fe700 1 -- 172.21.15.27:0/1126541927 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb514014d90 con 0x7fb52c0f74a0 2021-11-03T18:42:26.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.181+0000 7fb531347700 1 -- 172.21.15.27:0/1126541927 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb52c108970 con 0x7fb52c0f74a0 2021-11-03T18:42:26.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.182+0000 7fb531347700 1 -- 172.21.15.27:0/1126541927 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb52c045ad0 con 0x7fb52c0f74a0 2021-11-03T18:42:26.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.183+0000 7fb5237fe700 1 -- 172.21.15.27:0/1126541927 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fb514025070 con 0x7fb52c0f74a0 2021-11-03T18:42:26.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.183+0000 7fb5237fe700 1 --2- 172.21.15.27:0/1126541927 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb5180640b0 0x7fb518066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:42:26.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.183+0000 7fb5237fe700 1 -- 172.21.15.27:0/1126541927 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fb514081720 con 0x7fb52c0f74a0 2021-11-03T18:42:26.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.183+0000 7fb52affd700 1 --2- 172.21.15.27:0/1126541927 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb5180640b0 0x7fb518066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:42:26.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.184+0000 7fb52affd700 1 --2- 172.21.15.27:0/1126541927 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb5180640b0 0x7fb518066570 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7fb524004fd0 tx=0x7fb52400b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:42:26.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.187+0000 7fb5237fe700 1 -- 172.21.15.27:0/1126541927 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fb514053a20 con 0x7fb52c0f74a0 2021-11-03T18:42:26.357 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.358+0000 7fb531347700 1 -- 172.21.15.27:0/1126541927 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fb52c0f4f30 con 0x7fb5180640b0 2021-11-03T18:42:26.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.373+0000 7fb5237fe700 1 -- 172.21.15.27:0/1126541927 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fb52c0f4f30 con 0x7fb5180640b0 2021-11-03T18:42:26.371 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:42:26.372 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (94m) 2m ago 95m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:42:26.372 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (85m) 2m ago 95m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:42:26.372 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (85m) 2m ago 94m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:42:26.373 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (93m) 2m ago 94m 42.9M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:42:26.373 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (87m) 2m ago 90m 26.9M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:42:26.373 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (90m) 2m ago 90m 3852M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:42:26.373 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (86m) 2m ago 90m 26.5M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:42:26.373 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (87m) 2m ago 90m 27.6M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:42:26.374 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (87m) 2m ago 96m 625M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:42:26.374 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (87m) 2m ago 94m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:42:26.374 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (86m) 2m ago 96m 387M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:42:26.374 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (86m) 2m ago 94m 175M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:42:26.375 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (94m) 2m ago 94m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:42:26.375 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (94m) 2m ago 94m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:42:26.375 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (85m) 2m ago 93m 433M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:42:26.375 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (84m) 2m ago 93m 471M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:42:26.376 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (84m) 2m ago 92m 356M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:42:26.376 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (83m) 2m ago 92m 504M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:42:26.376 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (83m) 2m ago 92m 396M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:42:26.376 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (82m) 2m ago 91m 408M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:42:26.376 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (93m) 2m ago 94m 66.8M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:42:26.377 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.375+0000 7fb531347700 1 -- 172.21.15.27:0/1126541927 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb5180640b0 msgr2=0x7fb518066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:42:26.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.375+0000 7fb531347700 1 --2- 172.21.15.27:0/1126541927 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb5180640b0 0x7fb518066570 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7fb524004fd0 tx=0x7fb52400b040).stop 2021-11-03T18:42:26.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.375+0000 7fb531347700 1 -- 172.21.15.27:0/1126541927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb52c0f74a0 msgr2=0x7fb52c1073e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:42:26.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.375+0000 7fb531347700 1 --2- 172.21.15.27:0/1126541927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb52c0f74a0 0x7fb52c1073e0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 rx=0x7fb514000c00 tx=0x7fb51400b040).stop 2021-11-03T18:42:26.378 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.375+0000 7fb531347700 1 -- 172.21.15.27:0/1126541927 shutdown_connections 2021-11-03T18:42:26.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.375+0000 7fb531347700 1 --2- 172.21.15.27:0/1126541927 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fb5180640b0 0x7fb518066570 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:26.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.375+0000 7fb531347700 1 --2- 172.21.15.27:0/1126541927 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fb52c0f6b90 0x7fb52c106ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:42:26.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.375+0000 7fb531347700 1 --2- 172.21.15.27:0/1126541927 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fb52c0f74a0 0x7fb52c1073e0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:26.379 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.375+0000 7fb531347700 1 -- 172.21.15.27:0/1126541927 >> 172.21.15.27:0/1126541927 conn(0x7fb52c0f0540 msgr2=0x7fb52c0f4820 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:42:26.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.375+0000 7fb531347700 1 -- 172.21.15.27:0/1126541927 shutdown_connections 2021-11-03T18:42:26.380 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.375+0000 7fb531347700 1 -- 172.21.15.27:0/1126541927 wait complete. 2021-11-03T18:42:26.473 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.473+0000 7f3f45441700 1 -- 172.21.15.27:0/1338407363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f400f77d0 msgr2=0x7f3f400f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:42:26.474 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.473+0000 7f3f45441700 1 --2- 172.21.15.27:0/1338407363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f400f77d0 0x7f3f400f7c30 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 rx=0x7f3f34004660 tx=0x7f3f34009b30).stop 2021-11-03T18:42:26.474 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.473+0000 7f3f45441700 1 -- 172.21.15.27:0/1338407363 shutdown_connections 2021-11-03T18:42:26.474 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.473+0000 7f3f45441700 1 --2- 172.21.15.27:0/1338407363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f400f77d0 0x7f3f400f7c30 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:26.475 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.473+0000 7f3f45441700 1 --2- 172.21.15.27:0/1338407363 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f400f6530 0x7f3f400f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:26.475 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.473+0000 7f3f45441700 1 -- 172.21.15.27:0/1338407363 >> 172.21.15.27:0/1338407363 conn(0x7f3f400f1c90 msgr2=0x7f3f400f40b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:42:26.475 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.474+0000 7f3f45441700 1 -- 172.21.15.27:0/1338407363 shutdown_connections 2021-11-03T18:42:26.475 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.474+0000 7f3f45441700 1 -- 172.21.15.27:0/1338407363 wait complete. 2021-11-03T18:42:26.476 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.474+0000 7f3f45441700 1 Processor -- start 2021-11-03T18:42:26.476 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.474+0000 7f3f45441700 1 -- start start 2021-11-03T18:42:26.476 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.474+0000 7f3f45441700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f400f6530 0x7f3f4006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:42:26.476 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.474+0000 7f3f45441700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f400f77d0 0x7f3f40069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:42:26.476 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.474+0000 7f3f45441700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4006a380 con 0x7f3f400f77d0 2021-11-03T18:42:26.477 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.474+0000 7f3f45441700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4006a4c0 con 0x7f3f400f6530 2021-11-03T18:42:26.477 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.474+0000 7f3f3f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f400f77d0 0x7f3f40069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:42:26.477 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.474+0000 7f3f3f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f400f77d0 0x7f3f40069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36396/0 (socket says 172.21.15.27:36396) 2021-11-03T18:42:26.477 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.474+0000 7f3f3f7fe700 1 -- 172.21.15.27:0/2511149879 learned_addr learned my addr 172.21.15.27:0/2511149879 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:42:26.478 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.475+0000 7f3f3ffff700 1 --2- 172.21.15.27:0/2511149879 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f400f6530 0x7f3f4006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:42:26.478 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.475+0000 7f3f3f7fe700 1 -- 172.21.15.27:0/2511149879 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f400f6530 msgr2=0x7f3f4006b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:42:26.478 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.475+0000 7f3f3f7fe700 1 --2- 172.21.15.27:0/2511149879 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f400f6530 0x7f3f4006b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:26.478 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.475+0000 7f3f3f7fe700 1 -- 172.21.15.27:0/2511149879 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f34005040 con 0x7f3f400f77d0 2021-11-03T18:42:26.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.475+0000 7f3f3ffff700 1 --2- 172.21.15.27:0/2511149879 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f400f6530 0x7f3f4006b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:42:26.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.475+0000 7f3f3f7fe700 1 --2- 172.21.15.27:0/2511149879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f400f77d0 0x7f3f40069e20 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 rx=0x7f3f34009990 tx=0x7f3f34009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:42:26.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.475+0000 7f3f3cff9700 1 -- 172.21.15.27:0/2511149879 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f34018070 con 0x7f3f400f77d0 2021-11-03T18:42:26.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.475+0000 7f3f3cff9700 1 -- 172.21.15.27:0/2511149879 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f3f34007ba0 con 0x7f3f400f77d0 2021-11-03T18:42:26.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.475+0000 7f3f45441700 1 -- 172.21.15.27:0/2511149879 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f4006a740 con 0x7f3f400f77d0 2021-11-03T18:42:26.480 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.475+0000 7f3f3cff9700 1 -- 172.21.15.27:0/2511149879 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3f3401c3c0 con 0x7f3f400f77d0 2021-11-03T18:42:26.480 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.475+0000 7f3f45441700 1 -- 172.21.15.27:0/2511149879 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f4006aca0 con 0x7f3f400f77d0 2021-11-03T18:42:26.481 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.476+0000 7f3f45441700 1 -- 172.21.15.27:0/2511149879 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f40045ad0 con 0x7f3f400f77d0 2021-11-03T18:42:26.481 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.476+0000 7f3f3cff9700 1 -- 172.21.15.27:0/2511149879 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f3f3401d420 con 0x7f3f400f77d0 2021-11-03T18:42:26.482 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.477+0000 7f3f3cff9700 1 --2- 172.21.15.27:0/2511149879 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3f28064100 0x7f3f280665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:42:26.482 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.477+0000 7f3f3cff9700 1 -- 172.21.15.27:0/2511149879 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f3f34082430 con 0x7f3f400f77d0 2021-11-03T18:42:26.482 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.477+0000 7f3f3ffff700 1 --2- 172.21.15.27:0/2511149879 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3f28064100 0x7f3f280665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:42:26.482 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.477+0000 7f3f3ffff700 1 --2- 172.21.15.27:0/2511149879 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3f28064100 0x7f3f280665c0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 rx=0x7f3f4006b730 tx=0x7f3f30009c00).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:42:26.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.481+0000 7f3f3cff9700 1 -- 172.21.15.27:0/2511149879 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f3f340547f0 con 0x7f3f400f77d0 2021-11-03T18:42:26.711 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.713+0000 7f3f45441700 1 -- 172.21.15.27:0/2511149879 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f3f40027a00 con 0x7f3f400f77d0 2021-11-03T18:42:26.712 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.714+0000 7f3f3cff9700 1 -- 172.21.15.27:0/2511149879 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f3f340543e0 con 0x7f3f400f77d0 2021-11-03T18:42:26.712 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:42:26.713 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:42:26.713 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:42:26.713 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:42:26.713 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:42:26.714 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:42:26.714 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:42:26.714 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:42:26.714 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:42:26.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:42:26.715 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:42:26.715 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:42:26.715 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:42:26.716 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:42:26.716 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:42:26.716 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:42:26.716 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:42:26.717 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:42:26.717 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.716+0000 7f3f45441700 1 -- 172.21.15.27:0/2511149879 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3f28064100 msgr2=0x7f3f280665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:42:26.718 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.716+0000 7f3f45441700 1 --2- 172.21.15.27:0/2511149879 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3f28064100 0x7f3f280665c0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 rx=0x7f3f4006b730 tx=0x7f3f30009c00).stop 2021-11-03T18:42:26.718 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.716+0000 7f3f45441700 1 -- 172.21.15.27:0/2511149879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f400f77d0 msgr2=0x7f3f40069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:42:26.718 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.716+0000 7f3f45441700 1 --2- 172.21.15.27:0/2511149879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f400f77d0 0x7f3f40069e20 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 rx=0x7f3f34009990 tx=0x7f3f34009b30).stop 2021-11-03T18:42:26.718 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.716+0000 7f3f45441700 1 -- 172.21.15.27:0/2511149879 shutdown_connections 2021-11-03T18:42:26.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.716+0000 7f3f45441700 1 --2- 172.21.15.27:0/2511149879 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3f28064100 0x7f3f280665c0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:26.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.716+0000 7f3f45441700 1 --2- 172.21.15.27:0/2511149879 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3f400f6530 0x7f3f4006b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:26.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.716+0000 7f3f45441700 1 --2- 172.21.15.27:0/2511149879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3f400f77d0 0x7f3f40069e20 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:26.719 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.716+0000 7f3f45441700 1 -- 172.21.15.27:0/2511149879 >> 172.21.15.27:0/2511149879 conn(0x7f3f400f1c90 msgr2=0x7f3f400fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:42:26.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.716+0000 7f3f45441700 1 -- 172.21.15.27:0/2511149879 shutdown_connections 2021-11-03T18:42:26.720 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.716+0000 7f3f45441700 1 -- 172.21.15.27:0/2511149879 wait complete. 2021-11-03T18:42:26.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.816+0000 7fee48618700 1 -- 172.21.15.27:0/923602614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee400f66d0 msgr2=0x7fee400f6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:42:26.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.816+0000 7fee48618700 1 --2- 172.21.15.27:0/923602614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee400f66d0 0x7fee400f6af0 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 rx=0x7fee3c004660 tx=0x7fee3c009b30).stop 2021-11-03T18:42:26.816 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.816+0000 7fee48618700 1 -- 172.21.15.27:0/923602614 shutdown_connections 2021-11-03T18:42:26.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.816+0000 7fee48618700 1 --2- 172.21.15.27:0/923602614 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee400f7970 0x7fee400f7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:26.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.816+0000 7fee48618700 1 --2- 172.21.15.27:0/923602614 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee400f66d0 0x7fee400f6af0 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:26.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.816+0000 7fee48618700 1 -- 172.21.15.27:0/923602614 >> 172.21.15.27:0/923602614 conn(0x7fee400f1e50 msgr2=0x7fee400f4270 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:42:26.817 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.816+0000 7fee48618700 1 -- 172.21.15.27:0/923602614 shutdown_connections 2021-11-03T18:42:26.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.816+0000 7fee48618700 1 -- 172.21.15.27:0/923602614 wait complete. 2021-11-03T18:42:26.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.816+0000 7fee48618700 1 Processor -- start 2021-11-03T18:42:26.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.817+0000 7fee48618700 1 -- start start 2021-11-03T18:42:26.818 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.817+0000 7fee48618700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee400f66d0 0x7fee40106f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:42:26.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.817+0000 7fee48618700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee400f7970 0x7fee40107470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:42:26.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.817+0000 7fee48618700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee40107a20 con 0x7fee400f7970 2021-11-03T18:42:26.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.817+0000 7fee48618700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee40107b60 con 0x7fee400f66d0 2021-11-03T18:42:26.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.817+0000 7fee45bb3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee400f7970 0x7fee40107470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:42:26.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.817+0000 7fee45bb3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee400f7970 0x7fee40107470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36406/0 (socket says 172.21.15.27:36406) 2021-11-03T18:42:26.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.817+0000 7fee45bb3700 1 -- 172.21.15.27:0/425229631 learned_addr learned my addr 172.21.15.27:0/425229631 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:42:26.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.817+0000 7fee463b4700 1 --2- 172.21.15.27:0/425229631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee400f66d0 0x7fee40106f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:42:26.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.817+0000 7fee45bb3700 1 -- 172.21.15.27:0/425229631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee400f66d0 msgr2=0x7fee40106f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:42:26.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.817+0000 7fee45bb3700 1 --2- 172.21.15.27:0/425229631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee400f66d0 0x7fee40106f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:26.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.817+0000 7fee45bb3700 1 -- 172.21.15.27:0/425229631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3c005040 con 0x7fee400f7970 2021-11-03T18:42:26.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.817+0000 7fee45bb3700 1 --2- 172.21.15.27:0/425229631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee400f7970 0x7fee40107470 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 rx=0x7fee300026d0 tx=0x7fee30009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:42:26.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.817+0000 7fee36ffd700 1 -- 172.21.15.27:0/425229631 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee300079c0 con 0x7fee400f7970 2021-11-03T18:42:26.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.817+0000 7fee36ffd700 1 -- 172.21.15.27:0/425229631 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fee3000f040 con 0x7fee400f7970 2021-11-03T18:42:26.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.817+0000 7fee48618700 1 -- 172.21.15.27:0/425229631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee40108500 con 0x7fee400f7970 2021-11-03T18:42:26.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.817+0000 7fee36ffd700 1 -- 172.21.15.27:0/425229631 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fee300148b0 con 0x7fee400f7970 2021-11-03T18:42:26.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.818+0000 7fee48618700 1 -- 172.21.15.27:0/425229631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee40108ac0 con 0x7fee400f7970 2021-11-03T18:42:26.823 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.818+0000 7fee48618700 1 -- 172.21.15.27:0/425229631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee40045ad0 con 0x7fee400f7970 2021-11-03T18:42:26.824 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.819+0000 7fee36ffd700 1 -- 172.21.15.27:0/425229631 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fee30007b20 con 0x7fee400f7970 2021-11-03T18:42:26.824 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.819+0000 7fee36ffd700 1 --2- 172.21.15.27:0/425229631 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fee2c0640b0 0x7fee2c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:42:26.824 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.819+0000 7fee36ffd700 1 -- 172.21.15.27:0/425229631 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fee30080660 con 0x7fee400f7970 2021-11-03T18:42:26.824 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.819+0000 7fee463b4700 1 --2- 172.21.15.27:0/425229631 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fee2c0640b0 0x7fee2c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:42:26.825 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.820+0000 7fee463b4700 1 --2- 172.21.15.27:0/425229631 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fee2c0640b0 0x7fee2c066570 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0x7fee3c004630 tx=0x7fee3c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:42:26.825 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:26.823+0000 7fee36ffd700 1 -- 172.21.15.27:0/425229631 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fee300529c0 con 0x7fee400f7970 2021-11-03T18:42:26.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:26 smithi115 ceph-mon[65453]: pgmap v2635: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:26.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:42:26.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:42:26.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:42:26.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:26.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:26.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:26.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:27.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:26 smithi027 ceph-mon[97244]: pgmap v2635: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:27.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:42:27.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:42:27.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:42:27.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:27.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:27.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:27.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:27.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:27.026+0000 7fee48618700 1 -- 172.21.15.27:0/425229631 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fee40027a00 con 0x7fee400f7970 2021-11-03T18:42:27.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:27.027+0000 7fee36ffd700 1 -- 172.21.15.27:0/425229631 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7fee300525b0 con 0x7fee400f7970 2021-11-03T18:42:27.025 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:42:27.026 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:42:27.026 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:42:27.026 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:42:27.027 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:42:27.027 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:42:27.027 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:42:27.027 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:42:27.028 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:42:27.028 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:42:27.029 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:42:27.029 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:42:27.029 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:42:27.029 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:42:27.030 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:42:27.030 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:42:27.030 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:42:27.030 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:42:27.031 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:42:27.031 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:42:27.031 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:42:27.031 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:42:27.032 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:42:27.032 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:42:27.032 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:42:27.032 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:42:27.032 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:42:27.033 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:42:27.033 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:42:27.033 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:42:27.033 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:42:27.034 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:42:27.034 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:42:27.034 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:42:27.034 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:42:27.034 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:42:27.035 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:42:27.035 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:42:27.035 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:42:27.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:27.029+0000 7fee48618700 1 -- 172.21.15.27:0/425229631 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fee2c0640b0 msgr2=0x7fee2c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:42:27.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:27.029+0000 7fee48618700 1 --2- 172.21.15.27:0/425229631 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fee2c0640b0 0x7fee2c066570 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0x7fee3c004630 tx=0x7fee3c009b30).stop 2021-11-03T18:42:27.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:27.029+0000 7fee48618700 1 -- 172.21.15.27:0/425229631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee400f7970 msgr2=0x7fee40107470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:42:27.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:27.029+0000 7fee48618700 1 --2- 172.21.15.27:0/425229631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee400f7970 0x7fee40107470 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 rx=0x7fee300026d0 tx=0x7fee30009de0).stop 2021-11-03T18:42:27.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:27.029+0000 7fee48618700 1 -- 172.21.15.27:0/425229631 shutdown_connections 2021-11-03T18:42:27.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:27.029+0000 7fee48618700 1 --2- 172.21.15.27:0/425229631 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fee2c0640b0 0x7fee2c066570 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:27.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:27.029+0000 7fee48618700 1 --2- 172.21.15.27:0/425229631 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fee400f66d0 0x7fee40106f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:27.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:27.029+0000 7fee48618700 1 --2- 172.21.15.27:0/425229631 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fee400f7970 0x7fee40107470 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:27.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:27.029+0000 7fee48618700 1 -- 172.21.15.27:0/425229631 >> 172.21.15.27:0/425229631 conn(0x7fee400f1e50 msgr2=0x7fee400fb3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:42:27.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:27.029+0000 7fee48618700 1 -- 172.21.15.27:0/425229631 shutdown_connections 2021-11-03T18:42:27.038 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:27.029+0000 7fee48618700 1 -- 172.21.15.27:0/425229631 wait complete. 2021-11-03T18:42:27.038 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:42:27.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:27 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:42:27.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:27 smithi115 ceph-mon[65453]: from='client.36808 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:42:27.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:27 smithi115 ceph-mon[65453]: from='client.36812 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:42:27.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:27 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2511149879' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:27.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:27 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/425229631' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:42:28.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:27 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:42:28.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:27 smithi027 ceph-mon[97244]: from='client.36808 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:42:28.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:27 smithi027 ceph-mon[97244]: from='client.36812 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:42:28.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:27 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2511149879' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:28.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:27 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/425229631' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:42:28.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:28 smithi115 ceph-mon[65453]: pgmap v2636: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:29.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:28 smithi027 ceph-mon[97244]: pgmap v2636: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:30.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:30 smithi115 ceph-mon[65453]: pgmap v2637: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:30.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5192.485002 secs 2021-11-03T18:42:31.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:30 smithi027 ceph-mon[97244]: pgmap v2637: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:31.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5192.485002 secs 2021-11-03T18:42:32.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:32 smithi027 ceph-mon[97244]: pgmap v2638: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:32.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:32 smithi115 ceph-mon[65453]: pgmap v2638: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:34.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:34 smithi115 ceph-mon[65453]: pgmap v2639: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:35.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:34 smithi027 ceph-mon[97244]: pgmap v2639: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:35.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5197.485130 secs 2021-11-03T18:42:36.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5197.485130 secs 2021-11-03T18:42:36.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:36 smithi115 ceph-mon[65453]: pgmap v2640: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:36.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:42:36.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:42:36.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:42:36.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:36.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:36.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:36.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:37.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:36 smithi027 ceph-mon[97244]: pgmap v2640: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:37.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:42:37.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:42:37.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:42:37.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:37.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:37.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:37.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:37.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:37 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:42:38.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:37 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:42:38.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:38 smithi115 ceph-mon[65453]: pgmap v2641: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:39.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:38 smithi027 ceph-mon[97244]: pgmap v2641: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:40.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:40 smithi115 ceph-mon[65453]: pgmap v2642: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:40.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5202.485266 secs 2021-11-03T18:42:41.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:40 smithi027 ceph-mon[97244]: pgmap v2642: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:41.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5202.485266 secs 2021-11-03T18:42:42.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:42 smithi027 ceph-mon[97244]: pgmap v2643: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:42.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:42 smithi115 ceph-mon[65453]: pgmap v2643: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:44.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:44 smithi115 ceph-mon[65453]: pgmap v2644: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:45.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:44 smithi027 ceph-mon[97244]: pgmap v2644: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:45.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5207.485404 secs 2021-11-03T18:42:46.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5207.485404 secs 2021-11-03T18:42:46.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:46 smithi115 ceph-mon[65453]: pgmap v2645: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:46.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:42:46.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:42:46.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:42:46.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:46.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:46.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:46.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:47.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:46 smithi027 ceph-mon[97244]: pgmap v2645: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:47.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:42:47.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:42:47.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:42:47.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:47.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:47.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:47.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:47.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:47 smithi115 ceph-mon[65453]: poll took 60170.730 ms: 1 events 2021-11-03T18:42:47.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:47 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:42:48.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:47 smithi027 ceph-mon[97244]: poll took 60170.730 ms: 1 events 2021-11-03T18:42:48.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:47 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:42:48.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:48 smithi115 ceph-mon[65453]: pgmap v2646: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:49.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:48 smithi027 ceph-mon[97244]: pgmap v2646: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:50.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:50 smithi115 ceph-mon[65453]: pgmap v2647: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:50.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5212.485529 secs 2021-11-03T18:42:51.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:50 smithi027 ceph-mon[97244]: pgmap v2647: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:51.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5212.485529 secs 2021-11-03T18:42:52.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:52 smithi027 ceph-mon[97244]: pgmap v2648: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:52.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:52 smithi115 ceph-mon[65453]: pgmap v2648: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:54.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:54 smithi115 ceph-mon[65453]: pgmap v2649: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:55.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:54 smithi027 ceph-mon[97244]: pgmap v2649: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:55.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5217.485680 secs 2021-11-03T18:42:56.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5217.485680 secs 2021-11-03T18:42:56.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:56 smithi115 ceph-mon[65453]: pgmap v2650: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:56.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:42:56.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:42:56.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:42:56.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:42:56.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:42:56.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:56.991 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:56.991 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:56.992 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:57.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:56 smithi027 ceph-mon[97244]: pgmap v2650: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:57.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:42:57.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:42:57.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:42:57.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:42:57.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:42:57.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:57.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:57.026 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:57.027 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:57.147 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.143+0000 7fe4b3640700 1 -- 172.21.15.27:0/1049436878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4ac0f6860 msgr2=0x7fe4ac0f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:42:57.148 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.143+0000 7fe4b3640700 1 --2- 172.21.15.27:0/1049436878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4ac0f6860 0x7fe4ac0f6cc0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 rx=0x7fe49c004660 tx=0x7fe49c009b30).stop 2021-11-03T18:42:57.148 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.143+0000 7fe4b3640700 1 -- 172.21.15.27:0/1049436878 shutdown_connections 2021-11-03T18:42:57.149 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.143+0000 7fe4b3640700 1 --2- 172.21.15.27:0/1049436878 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4ac0f6860 0x7fe4ac0f6cc0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:57.149 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.143+0000 7fe4b3640700 1 --2- 172.21.15.27:0/1049436878 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4ac0f55c0 0x7fe4ac0f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:57.150 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.143+0000 7fe4b3640700 1 -- 172.21.15.27:0/1049436878 >> 172.21.15.27:0/1049436878 conn(0x7fe4ac0f0d40 msgr2=0x7fe4ac0f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:42:57.150 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.143+0000 7fe4b3640700 1 -- 172.21.15.27:0/1049436878 shutdown_connections 2021-11-03T18:42:57.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.143+0000 7fe4b3640700 1 -- 172.21.15.27:0/1049436878 wait complete. 2021-11-03T18:42:57.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.144+0000 7fe4b3640700 1 Processor -- start 2021-11-03T18:42:57.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.144+0000 7fe4b3640700 1 -- start start 2021-11-03T18:42:57.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.144+0000 7fe4b3640700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4ac0f55c0 0x7fe4ac106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:42:57.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.144+0000 7fe4b3640700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4ac0f6860 0x7fe4ac107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:42:57.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.144+0000 7fe4b3640700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac107a10 con 0x7fe4ac0f55c0 2021-11-03T18:42:57.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.144+0000 7fe4b3640700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ac107b50 con 0x7fe4ac0f6860 2021-11-03T18:42:57.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.144+0000 7fe4b13dc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4ac0f55c0 0x7fe4ac106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:42:57.154 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.144+0000 7fe4b13dc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4ac0f55c0 0x7fe4ac106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36422/0 (socket says 172.21.15.27:36422) 2021-11-03T18:42:57.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.144+0000 7fe4b13dc700 1 -- 172.21.15.27:0/3719892552 learned_addr learned my addr 172.21.15.27:0/3719892552 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:42:57.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.145+0000 7fe4b0bdb700 1 --2- 172.21.15.27:0/3719892552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4ac0f6860 0x7fe4ac107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:42:57.155 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.145+0000 7fe4b13dc700 1 -- 172.21.15.27:0/3719892552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4ac0f6860 msgr2=0x7fe4ac107460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:42:57.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.145+0000 7fe4b13dc700 1 --2- 172.21.15.27:0/3719892552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4ac0f6860 0x7fe4ac107460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:57.156 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.145+0000 7fe4b13dc700 1 -- 172.21.15.27:0/3719892552 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe49c005040 con 0x7fe4ac0f55c0 2021-11-03T18:42:57.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.145+0000 7fe4b13dc700 1 --2- 172.21.15.27:0/3719892552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4ac0f55c0 0x7fe4ac106f70 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 rx=0x7fe4a400c710 tx=0x7fe4a4004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:42:57.157 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.145+0000 7fe4a1ffb700 1 -- 172.21.15.27:0/3719892552 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4a40106f0 con 0x7fe4ac0f55c0 2021-11-03T18:42:57.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.145+0000 7fe4a1ffb700 1 -- 172.21.15.27:0/3719892552 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fe4a4010d00 con 0x7fe4ac0f55c0 2021-11-03T18:42:57.158 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.145+0000 7fe4b3640700 1 -- 172.21.15.27:0/3719892552 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4ac1084f0 con 0x7fe4ac0f55c0 2021-11-03T18:42:57.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.145+0000 7fe4a1ffb700 1 -- 172.21.15.27:0/3719892552 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe4a4012d90 con 0x7fe4ac0f55c0 2021-11-03T18:42:57.159 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.145+0000 7fe4b3640700 1 -- 172.21.15.27:0/3719892552 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4ac108ab0 con 0x7fe4ac0f55c0 2021-11-03T18:42:57.160 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.146+0000 7fe4b3640700 1 -- 172.21.15.27:0/3719892552 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4ac045ad0 con 0x7fe4ac0f55c0 2021-11-03T18:42:57.160 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.147+0000 7fe4a1ffb700 1 -- 172.21.15.27:0/3719892552 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fe4a4010070 con 0x7fe4ac0f55c0 2021-11-03T18:42:57.160 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.147+0000 7fe4a1ffb700 1 --2- 172.21.15.27:0/3719892552 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe4980640b0 0x7fe498066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:42:57.161 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.147+0000 7fe4a1ffb700 1 -- 172.21.15.27:0/3719892552 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fe4a4080ce0 con 0x7fe4ac0f55c0 2021-11-03T18:42:57.161 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.147+0000 7fe4b0bdb700 1 --2- 172.21.15.27:0/3719892552 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe4980640b0 0x7fe498066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:42:57.162 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.148+0000 7fe4b0bdb700 1 --2- 172.21.15.27:0/3719892552 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe4980640b0 0x7fe498066570 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x7fe49c004e30 tx=0x7fe49c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:42:57.163 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.151+0000 7fe4a1ffb700 1 -- 172.21.15.27:0/3719892552 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fe4a4052f50 con 0x7fe4ac0f55c0 2021-11-03T18:42:57.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.331+0000 7fe4b3640700 1 -- 172.21.15.27:0/3719892552 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe4ac05fd30 con 0x7fe4980640b0 2021-11-03T18:42:57.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.333+0000 7fe4a1ffb700 1 -- 172.21.15.27:0/3719892552 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fe4ac05fd30 con 0x7fe4980640b0 2021-11-03T18:42:57.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.334+0000 7fe4b3640700 1 -- 172.21.15.27:0/3719892552 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe4980640b0 msgr2=0x7fe498066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:42:57.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.334+0000 7fe4b3640700 1 --2- 172.21.15.27:0/3719892552 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe4980640b0 0x7fe498066570 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x7fe49c004e30 tx=0x7fe49c009b30).stop 2021-11-03T18:42:57.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.334+0000 7fe4b3640700 1 -- 172.21.15.27:0/3719892552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4ac0f55c0 msgr2=0x7fe4ac106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:42:57.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.334+0000 7fe4b3640700 1 --2- 172.21.15.27:0/3719892552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4ac0f55c0 0x7fe4ac106f70 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 rx=0x7fe4a400c710 tx=0x7fe4a4004d10).stop 2021-11-03T18:42:57.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.334+0000 7fe4b3640700 1 -- 172.21.15.27:0/3719892552 shutdown_connections 2021-11-03T18:42:57.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.334+0000 7fe4b3640700 1 --2- 172.21.15.27:0/3719892552 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe4980640b0 0x7fe498066570 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:57.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.334+0000 7fe4b3640700 1 --2- 172.21.15.27:0/3719892552 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe4ac0f55c0 0x7fe4ac106f70 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:57.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.334+0000 7fe4b3640700 1 --2- 172.21.15.27:0/3719892552 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe4ac0f6860 0x7fe4ac107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:57.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.334+0000 7fe4b3640700 1 -- 172.21.15.27:0/3719892552 >> 172.21.15.27:0/3719892552 conn(0x7fe4ac0f0d40 msgr2=0x7fe4ac05f620 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:42:57.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.335+0000 7fe4b3640700 1 -- 172.21.15.27:0/3719892552 shutdown_connections 2021-11-03T18:42:57.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.335+0000 7fe4b3640700 1 -- 172.21.15.27:0/3719892552 wait complete. 2021-11-03T18:42:57.343 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:42:57.436 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.435+0000 7f36002b4700 1 -- 172.21.15.27:0/2345497253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35f80f55c0 msgr2=0x7f35f80f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:42:57.436 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.435+0000 7f36002b4700 1 --2- 172.21.15.27:0/2345497253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35f80f55c0 0x7f35f80f59e0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 rx=0x7f35f4004660 tx=0x7f35f4009b30).stop 2021-11-03T18:42:57.436 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.435+0000 7f36002b4700 1 -- 172.21.15.27:0/2345497253 shutdown_connections 2021-11-03T18:42:57.436 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.435+0000 7f36002b4700 1 --2- 172.21.15.27:0/2345497253 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35f80f6860 0x7f35f80f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:57.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.435+0000 7f36002b4700 1 --2- 172.21.15.27:0/2345497253 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35f80f55c0 0x7f35f80f59e0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:57.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.435+0000 7f36002b4700 1 -- 172.21.15.27:0/2345497253 >> 172.21.15.27:0/2345497253 conn(0x7f35f80f0d40 msgr2=0x7f35f80f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:42:57.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.435+0000 7f36002b4700 1 -- 172.21.15.27:0/2345497253 shutdown_connections 2021-11-03T18:42:57.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.435+0000 7f36002b4700 1 -- 172.21.15.27:0/2345497253 wait complete. 2021-11-03T18:42:57.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.435+0000 7f36002b4700 1 Processor -- start 2021-11-03T18:42:57.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.435+0000 7f36002b4700 1 -- start start 2021-11-03T18:42:57.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.436+0000 7f36002b4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35f80f55c0 0x7f35f8106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:42:57.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.436+0000 7f36002b4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35f80f6860 0x7f35f8107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:42:57.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.436+0000 7f36002b4700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f8107a10 con 0x7f35f80f55c0 2021-11-03T18:42:57.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.436+0000 7f36002b4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35f8107b50 con 0x7f35f80f6860 2021-11-03T18:42:57.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.436+0000 7f35fe050700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35f80f55c0 0x7f35f8106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:42:57.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.436+0000 7f35fe050700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35f80f55c0 0x7f35f8106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36432/0 (socket says 172.21.15.27:36432) 2021-11-03T18:42:57.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.436+0000 7f35fe050700 1 -- 172.21.15.27:0/4256647294 learned_addr learned my addr 172.21.15.27:0/4256647294 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:42:57.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.436+0000 7f35fe050700 1 -- 172.21.15.27:0/4256647294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35f80f6860 msgr2=0x7f35f8107460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:42:57.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.436+0000 7f35fe050700 1 --2- 172.21.15.27:0/4256647294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35f80f6860 0x7f35f8107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:42:57.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.436+0000 7f35fe050700 1 -- 172.21.15.27:0/4256647294 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35f4005040 con 0x7f35f80f55c0 2021-11-03T18:42:57.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.436+0000 7f35fe050700 1 --2- 172.21.15.27:0/4256647294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35f80f55c0 0x7f35f8106f70 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 rx=0x7f35f4013a70 tx=0x7f35f4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:42:57.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.436+0000 7f35eaffd700 1 -- 172.21.15.27:0/4256647294 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35f4018070 con 0x7f35f80f55c0 2021-11-03T18:42:57.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.436+0000 7f35eaffd700 1 -- 172.21.15.27:0/4256647294 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f35f40097d0 con 0x7f35f80f55c0 2021-11-03T18:42:57.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.436+0000 7f35eaffd700 1 -- 172.21.15.27:0/4256647294 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f35f401c3c0 con 0x7f35f80f55c0 2021-11-03T18:42:57.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.436+0000 7f36002b4700 1 -- 172.21.15.27:0/4256647294 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35f8108490 con 0x7f35f80f55c0 2021-11-03T18:42:57.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.436+0000 7f36002b4700 1 -- 172.21.15.27:0/4256647294 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35f81089f0 con 0x7f35f80f55c0 2021-11-03T18:42:57.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.437+0000 7f36002b4700 1 -- 172.21.15.27:0/4256647294 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35f8045ad0 con 0x7f35f80f55c0 2021-11-03T18:42:57.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.438+0000 7f35eaffd700 1 -- 172.21.15.27:0/4256647294 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f35f4007ba0 con 0x7f35f80f55c0 2021-11-03T18:42:57.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.438+0000 7f35eaffd700 1 --2- 172.21.15.27:0/4256647294 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f35e40640b0 0x7f35e4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:42:57.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.438+0000 7f35eaffd700 1 -- 172.21.15.27:0/4256647294 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f35f4082470 con 0x7f35f80f55c0 2021-11-03T18:42:57.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.438+0000 7f35fd84f700 1 --2- 172.21.15.27:0/4256647294 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f35e40640b0 0x7f35e4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:42:57.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.439+0000 7f35fd84f700 1 --2- 172.21.15.27:0/4256647294 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f35e40640b0 0x7f35e4066570 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 rx=0x7f35ec004a10 tx=0x7f35ec0099e0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:42:57.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.442+0000 7f35eaffd700 1 -- 172.21.15.27:0/4256647294 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f35f40547d0 con 0x7f35f80f55c0 2021-11-03T18:42:57.615 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.616+0000 7f36002b4700 1 -- 172.21.15.27:0/4256647294 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f35f805fd30 con 0x7f35e40640b0 2021-11-03T18:42:57.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.630+0000 7f35eaffd700 1 -- 172.21.15.27:0/4256647294 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f35f805fd30 con 0x7f35e40640b0 2021-11-03T18:42:57.630 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:42:57.630 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (94m) 3m ago 95m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:42:57.631 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (86m) 3m ago 95m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:42:57.631 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (86m) 2m ago 95m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:42:57.631 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (94m) 3m ago 95m 42.9M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:42:57.631 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (88m) 3m ago 90m 26.9M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:42:57.632 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (90m) 3m ago 90m 3852M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:42:57.632 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (86m) 2m ago 90m 26.5M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:42:57.632 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (88m) 2m ago 91m 27.6M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:42:57.632 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (88m) 3m ago 97m 625M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:42:57.633 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (87m) 2m ago 94m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:42:57.633 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (87m) 3m ago 97m 387M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:42:57.633 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (86m) 2m ago 94m 175M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:42:57.633 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (95m) 3m ago 95m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:42:57.633 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (94m) 2m ago 94m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:42:57.634 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (85m) 3m ago 94m 433M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:42:57.634 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (85m) 3m ago 93m 471M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:42:57.634 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (84m) 3m ago 93m 356M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:42:57.634 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (84m) 2m ago 93m 504M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:42:57.635 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (83m) 2m ago 92m 396M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:42:57.635 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (83m) 2m ago 92m 408M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:42:57.635 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (94m) 3m ago 95m 66.8M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:42:57.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.632+0000 7f36002b4700 1 -- 172.21.15.27:0/4256647294 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f35e40640b0 msgr2=0x7f35e4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:42:57.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.632+0000 7f36002b4700 1 --2- 172.21.15.27:0/4256647294 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f35e40640b0 0x7f35e4066570 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 rx=0x7f35ec004a10 tx=0x7f35ec0099e0).stop 2021-11-03T18:42:57.636 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.632+0000 7f36002b4700 1 -- 172.21.15.27:0/4256647294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35f80f55c0 msgr2=0x7f35f8106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:42:57.637 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.632+0000 7f36002b4700 1 --2- 172.21.15.27:0/4256647294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35f80f55c0 0x7f35f8106f70 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 rx=0x7f35f4013a70 tx=0x7f35f4009b30).stop 2021-11-03T18:42:57.637 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.632+0000 7f36002b4700 1 -- 172.21.15.27:0/4256647294 shutdown_connections 2021-11-03T18:42:57.637 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.632+0000 7f36002b4700 1 --2- 172.21.15.27:0/4256647294 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f35e40640b0 0x7f35e4066570 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:57.637 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.632+0000 7f36002b4700 1 --2- 172.21.15.27:0/4256647294 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f35f80f55c0 0x7f35f8106f70 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:57.638 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.632+0000 7f36002b4700 1 --2- 172.21.15.27:0/4256647294 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f35f80f6860 0x7f35f8107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:42:57.638 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.632+0000 7f36002b4700 1 -- 172.21.15.27:0/4256647294 >> 172.21.15.27:0/4256647294 conn(0x7f35f80f0d40 msgr2=0x7f35f805f620 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:42:57.638 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.632+0000 7f36002b4700 1 -- 172.21.15.27:0/4256647294 shutdown_connections 2021-11-03T18:42:57.638 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.632+0000 7f36002b4700 1 -- 172.21.15.27:0/4256647294 wait complete. 2021-11-03T18:42:57.733 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.732+0000 7f293f4dc700 1 -- 172.21.15.27:0/361014223 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29380f66e0 msgr2=0x7f29380f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:42:57.734 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.732+0000 7f293f4dc700 1 --2- 172.21.15.27:0/361014223 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29380f66e0 0x7f29380f6b00 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 rx=0x7f29280099e0 tx=0x7f2928004d10).stop 2021-11-03T18:42:57.734 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.732+0000 7f293f4dc700 1 -- 172.21.15.27:0/361014223 shutdown_connections 2021-11-03T18:42:57.734 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.732+0000 7f293f4dc700 1 --2- 172.21.15.27:0/361014223 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29380f6ff0 0x7f29380f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:57.734 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.732+0000 7f293f4dc700 1 --2- 172.21.15.27:0/361014223 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29380f66e0 0x7f29380f6b00 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:57.735 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.732+0000 7f293f4dc700 1 -- 172.21.15.27:0/361014223 >> 172.21.15.27:0/361014223 conn(0x7f29380f1f00 msgr2=0x7f29380f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:42:57.735 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.732+0000 7f293f4dc700 1 -- 172.21.15.27:0/361014223 shutdown_connections 2021-11-03T18:42:57.735 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.732+0000 7f293f4dc700 1 -- 172.21.15.27:0/361014223 wait complete. 2021-11-03T18:42:57.736 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.733+0000 7f293f4dc700 1 Processor -- start 2021-11-03T18:42:57.736 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.733+0000 7f293f4dc700 1 -- start start 2021-11-03T18:42:57.736 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.733+0000 7f293f4dc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29380f66e0 0x7f2938107070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:42:57.736 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.733+0000 7f293f4dc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29380f6ff0 0x7f2938107560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:42:57.736 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.733+0000 7f293f4dc700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2938107b10 con 0x7f29380f66e0 2021-11-03T18:42:57.737 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.733+0000 7f293f4dc700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2938107c50 con 0x7f29380f6ff0 2021-11-03T18:42:57.737 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.733+0000 7f293d278700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29380f66e0 0x7f2938107070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:42:57.737 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.733+0000 7f293d278700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29380f66e0 0x7f2938107070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36442/0 (socket says 172.21.15.27:36442) 2021-11-03T18:42:57.737 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.733+0000 7f293d278700 1 -- 172.21.15.27:0/2094324917 learned_addr learned my addr 172.21.15.27:0/2094324917 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:42:57.738 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.733+0000 7f293ca77700 1 --2- 172.21.15.27:0/2094324917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29380f6ff0 0x7f2938107560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:42:57.738 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.733+0000 7f293d278700 1 -- 172.21.15.27:0/2094324917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29380f6ff0 msgr2=0x7f2938107560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:42:57.738 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.733+0000 7f293d278700 1 --2- 172.21.15.27:0/2094324917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29380f6ff0 0x7f2938107560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:57.738 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.733+0000 7f293d278700 1 -- 172.21.15.27:0/2094324917 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2928009640 con 0x7f29380f66e0 2021-11-03T18:42:57.738 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.733+0000 7f293ca77700 1 --2- 172.21.15.27:0/2094324917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29380f6ff0 0x7f2938107560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T18:42:57.739 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.734+0000 7f293d278700 1 --2- 172.21.15.27:0/2094324917 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29380f66e0 0x7f2938107070 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 rx=0x7f2928004b70 tx=0x7f2928004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:42:57.739 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.734+0000 7f2935ffb700 1 -- 172.21.15.27:0/2094324917 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f292800e030 con 0x7f29380f66e0 2021-11-03T18:42:57.739 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.734+0000 7f2935ffb700 1 -- 172.21.15.27:0/2094324917 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f29280049b0 con 0x7f29380f66e0 2021-11-03T18:42:57.739 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.734+0000 7f293f4dc700 1 -- 172.21.15.27:0/2094324917 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2938108590 con 0x7f29380f66e0 2021-11-03T18:42:57.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.734+0000 7f2935ffb700 1 -- 172.21.15.27:0/2094324917 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2928018070 con 0x7f29380f66e0 2021-11-03T18:42:57.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.734+0000 7f293f4dc700 1 -- 172.21.15.27:0/2094324917 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2938108a70 con 0x7f29380f66e0 2021-11-03T18:42:57.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.735+0000 7f293f4dc700 1 -- 172.21.15.27:0/2094324917 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2938045ad0 con 0x7f29380f66e0 2021-11-03T18:42:57.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.735+0000 7f2935ffb700 1 -- 172.21.15.27:0/2094324917 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f292801db40 con 0x7f29380f66e0 2021-11-03T18:42:57.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.736+0000 7f2935ffb700 1 --2- 172.21.15.27:0/2094324917 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f29200640b0 0x7f2920066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:42:57.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.736+0000 7f2935ffb700 1 -- 172.21.15.27:0/2094324917 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f29280827c0 con 0x7f29380f66e0 2021-11-03T18:42:57.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.736+0000 7f293ca77700 1 --2- 172.21.15.27:0/2094324917 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f29200640b0 0x7f2920066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:42:57.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.736+0000 7f293ca77700 1 --2- 172.21.15.27:0/2094324917 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f29200640b0 0x7f2920066570 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7f292c0041d0 tx=0x7f292c009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:42:57.742 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.740+0000 7f2935ffb700 1 -- 172.21.15.27:0/2094324917 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f2928054ac0 con 0x7f29380f66e0 2021-11-03T18:42:57.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.969+0000 7f293f4dc700 1 -- 172.21.15.27:0/2094324917 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f29380fef70 con 0x7f29380f66e0 2021-11-03T18:42:57.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.970+0000 7f2935ffb700 1 -- 172.21.15.27:0/2094324917 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f29280546b0 con 0x7f29380f66e0 2021-11-03T18:42:57.969 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:42:57.970 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:42:57.970 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:42:57.970 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:42:57.970 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:42:57.971 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:42:57.971 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:42:57.971 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:42:57.971 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:42:57.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:42:57.972 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:42:57.972 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:42:57.972 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:42:57.972 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:42:57.973 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:42:57.973 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:42:57.973 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:42:57.973 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:42:57.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.971+0000 7f293f4dc700 1 -- 172.21.15.27:0/2094324917 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f29200640b0 msgr2=0x7f2920066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:42:57.974 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.971+0000 7f293f4dc700 1 --2- 172.21.15.27:0/2094324917 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f29200640b0 0x7f2920066570 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7f292c0041d0 tx=0x7f292c009040).stop 2021-11-03T18:42:57.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.971+0000 7f293f4dc700 1 -- 172.21.15.27:0/2094324917 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29380f66e0 msgr2=0x7f2938107070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:42:57.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.971+0000 7f293f4dc700 1 --2- 172.21.15.27:0/2094324917 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29380f66e0 0x7f2938107070 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 rx=0x7f2928004b70 tx=0x7f2928004d10).stop 2021-11-03T18:42:57.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.971+0000 7f293f4dc700 1 -- 172.21.15.27:0/2094324917 shutdown_connections 2021-11-03T18:42:57.975 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.971+0000 7f293f4dc700 1 --2- 172.21.15.27:0/2094324917 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f29200640b0 0x7f2920066570 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:57.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.971+0000 7f293f4dc700 1 --2- 172.21.15.27:0/2094324917 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f29380f66e0 0x7f2938107070 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:57.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.971+0000 7f293f4dc700 1 --2- 172.21.15.27:0/2094324917 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f29380f6ff0 0x7f2938107560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:57.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.971+0000 7f293f4dc700 1 -- 172.21.15.27:0/2094324917 >> 172.21.15.27:0/2094324917 conn(0x7f29380f1f00 msgr2=0x7f29380fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:42:57.976 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.971+0000 7f293f4dc700 1 -- 172.21.15.27:0/2094324917 shutdown_connections 2021-11-03T18:42:57.977 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:57.972+0000 7f293f4dc700 1 -- 172.21.15.27:0/2094324917 wait complete. 2021-11-03T18:42:57.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:57 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:42:58.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:57 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:42:58.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.072+0000 7ff737f4d700 1 -- 172.21.15.27:0/3809241994 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7300f55c0 msgr2=0x7ff7300f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:42:58.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.072+0000 7ff737f4d700 1 --2- 172.21.15.27:0/3809241994 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7300f55c0 0x7ff7300f59e0 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 rx=0x7ff724004660 tx=0x7ff724009b30).stop 2021-11-03T18:42:58.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.072+0000 7ff737f4d700 1 -- 172.21.15.27:0/3809241994 shutdown_connections 2021-11-03T18:42:58.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.072+0000 7ff737f4d700 1 --2- 172.21.15.27:0/3809241994 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7300f6860 0x7ff7300f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:58.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.072+0000 7ff737f4d700 1 --2- 172.21.15.27:0/3809241994 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7300f55c0 0x7ff7300f59e0 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:58.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.072+0000 7ff737f4d700 1 -- 172.21.15.27:0/3809241994 >> 172.21.15.27:0/3809241994 conn(0x7ff7300f0d40 msgr2=0x7ff7300f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:42:58.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.072+0000 7ff737f4d700 1 -- 172.21.15.27:0/3809241994 shutdown_connections 2021-11-03T18:42:58.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.072+0000 7ff737f4d700 1 -- 172.21.15.27:0/3809241994 wait complete. 2021-11-03T18:42:58.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.072+0000 7ff737f4d700 1 Processor -- start 2021-11-03T18:42:58.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.072+0000 7ff737f4d700 1 -- start start 2021-11-03T18:42:58.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.073+0000 7ff737f4d700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7300f55c0 0x7ff7300f9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:42:58.075 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.073+0000 7ff737f4d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7300f6860 0x7ff7300f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:42:58.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.073+0000 7ff737f4d700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7300f8100 con 0x7ff7300f6860 2021-11-03T18:42:58.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.073+0000 7ff737f4d700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7300f8240 con 0x7ff7300f55c0 2021-11-03T18:42:58.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.073+0000 7ff7354e8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7300f6860 0x7ff7300f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:42:58.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.073+0000 7ff7354e8700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7300f6860 0x7ff7300f7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36454/0 (socket says 172.21.15.27:36454) 2021-11-03T18:42:58.076 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.073+0000 7ff7354e8700 1 -- 172.21.15.27:0/104007409 learned_addr learned my addr 172.21.15.27:0/104007409 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:42:58.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.073+0000 7ff735ce9700 1 --2- 172.21.15.27:0/104007409 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7300f55c0 0x7ff7300f9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:42:58.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.073+0000 7ff7354e8700 1 -- 172.21.15.27:0/104007409 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7300f55c0 msgr2=0x7ff7300f9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:42:58.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.073+0000 7ff7354e8700 1 --2- 172.21.15.27:0/104007409 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7300f55c0 0x7ff7300f9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:58.077 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.073+0000 7ff7354e8700 1 -- 172.21.15.27:0/104007409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff724005040 con 0x7ff7300f6860 2021-11-03T18:42:58.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.073+0000 7ff7354e8700 1 --2- 172.21.15.27:0/104007409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7300f6860 0x7ff7300f7ba0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 rx=0x7ff72c0026d0 tx=0x7ff72c009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:42:58.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.073+0000 7ff7227fc700 1 -- 172.21.15.27:0/104007409 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff72c009320 con 0x7ff7300f6860 2021-11-03T18:42:58.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.073+0000 7ff7227fc700 1 -- 172.21.15.27:0/104007409 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7ff72c00f040 con 0x7ff7300f6860 2021-11-03T18:42:58.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.073+0000 7ff7227fc700 1 -- 172.21.15.27:0/104007409 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff72c0147e0 con 0x7ff7300f6860 2021-11-03T18:42:58.078 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.073+0000 7ff737f4d700 1 -- 172.21.15.27:0/104007409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7300f8520 con 0x7ff7300f6860 2021-11-03T18:42:58.079 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.073+0000 7ff737f4d700 1 -- 172.21.15.27:0/104007409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff73010efa0 con 0x7ff7300f6860 2021-11-03T18:42:58.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.074+0000 7ff737f4d700 1 -- 172.21.15.27:0/104007409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff730045ad0 con 0x7ff7300f6860 2021-11-03T18:42:58.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.075+0000 7ff7227fc700 1 -- 172.21.15.27:0/104007409 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7ff72c009480 con 0x7ff7300f6860 2021-11-03T18:42:58.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.075+0000 7ff7227fc700 1 --2- 172.21.15.27:0/104007409 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff71c064090 0x7ff71c066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:42:58.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.075+0000 7ff7227fc700 1 -- 172.21.15.27:0/104007409 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7ff72c080e10 con 0x7ff7300f6860 2021-11-03T18:42:58.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.075+0000 7ff735ce9700 1 --2- 172.21.15.27:0/104007409 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff71c064090 0x7ff71c066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:42:58.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.076+0000 7ff735ce9700 1 --2- 172.21.15.27:0/104007409 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff71c064090 0x7ff71c066550 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 rx=0x7ff7300f94b0 tx=0x7ff724009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:42:58.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.079+0000 7ff7227fc700 1 -- 172.21.15.27:0/104007409 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7ff72c053230 con 0x7ff7300f6860 2021-11-03T18:42:58.282 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.283+0000 7ff737f4d700 1 -- 172.21.15.27:0/104007409 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff7300fdd80 con 0x7ff7300f6860 2021-11-03T18:42:58.283 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.284+0000 7ff7227fc700 1 -- 172.21.15.27:0/104007409 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7ff72c052e20 con 0x7ff7300f6860 2021-11-03T18:42:58.283 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:42:58.284 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:42:58.284 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:42:58.284 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:42:58.285 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:42:58.285 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:42:58.285 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:42:58.285 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:42:58.285 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:42:58.286 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:42:58.286 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:42:58.286 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:42:58.286 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:42:58.287 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:42:58.287 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:42:58.287 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:42:58.287 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:42:58.288 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:42:58.288 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:42:58.288 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:42:58.288 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:42:58.289 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:42:58.289 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:42:58.289 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:42:58.290 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:42:58.290 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:42:58.290 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:42:58.290 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:42:58.291 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:42:58.292 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:42:58.292 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:42:58.293 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:42:58.293 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:42:58.293 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:42:58.294 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:42:58.294 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:42:58.294 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:42:58.294 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:42:58.295 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:42:58.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.285+0000 7ff737f4d700 1 -- 172.21.15.27:0/104007409 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff71c064090 msgr2=0x7ff71c066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:42:58.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.285+0000 7ff737f4d700 1 --2- 172.21.15.27:0/104007409 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff71c064090 0x7ff71c066550 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 rx=0x7ff7300f94b0 tx=0x7ff724009b30).stop 2021-11-03T18:42:58.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.285+0000 7ff737f4d700 1 -- 172.21.15.27:0/104007409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7300f6860 msgr2=0x7ff7300f7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:42:58.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.285+0000 7ff737f4d700 1 --2- 172.21.15.27:0/104007409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7300f6860 0x7ff7300f7ba0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 rx=0x7ff72c0026d0 tx=0x7ff72c009de0).stop 2021-11-03T18:42:58.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.285+0000 7ff737f4d700 1 -- 172.21.15.27:0/104007409 shutdown_connections 2021-11-03T18:42:58.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.285+0000 7ff737f4d700 1 --2- 172.21.15.27:0/104007409 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff71c064090 0x7ff71c066550 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:58.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.285+0000 7ff737f4d700 1 --2- 172.21.15.27:0/104007409 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff7300f55c0 0x7ff7300f9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:58.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.285+0000 7ff737f4d700 1 --2- 172.21.15.27:0/104007409 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff7300f6860 0x7ff7300f7ba0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:42:58.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.285+0000 7ff737f4d700 1 -- 172.21.15.27:0/104007409 >> 172.21.15.27:0/104007409 conn(0x7ff7300f0d40 msgr2=0x7ff73005fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:42:58.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.286+0000 7ff737f4d700 1 -- 172.21.15.27:0/104007409 shutdown_connections 2021-11-03T18:42:58.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:42:58.286+0000 7ff737f4d700 1 -- 172.21.15.27:0/104007409 wait complete. 2021-11-03T18:42:58.299 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:42:58.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:58 smithi115 ceph-mon[65453]: pgmap v2651: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:58.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:58 smithi115 ceph-mon[65453]: from='client.36824 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:42:58.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:58 smithi115 ceph-mon[65453]: from='client.36828 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:42:58.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:58 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2094324917' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:58.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:42:58 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/104007409' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:42:59.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:58 smithi027 ceph-mon[97244]: pgmap v2651: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:42:59.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:58 smithi027 ceph-mon[97244]: from='client.36824 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:42:59.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:58 smithi027 ceph-mon[97244]: from='client.36828 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:42:59.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:58 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2094324917' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:42:59.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:42:58 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/104007409' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:43:00.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:00 smithi115 ceph-mon[65453]: pgmap v2652: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:00.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5222.485798 secs 2021-11-03T18:43:01.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:00 smithi027 ceph-mon[97244]: pgmap v2652: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:01.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5222.485798 secs 2021-11-03T18:43:02.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:02 smithi027 ceph-mon[97244]: pgmap v2653: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:02.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:02 smithi115 ceph-mon[65453]: pgmap v2653: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:04.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:04 smithi115 ceph-mon[65453]: pgmap v2654: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:05.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:04 smithi027 ceph-mon[97244]: pgmap v2654: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:05.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5227.485913 secs 2021-11-03T18:43:06.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5227.485913 secs 2021-11-03T18:43:06.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:06 smithi115 ceph-mon[65453]: pgmap v2655: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:06.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:43:06.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:43:06.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:43:06.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:06.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:06.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:06.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:07.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:06 smithi027 ceph-mon[97244]: pgmap v2655: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:07.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:43:07.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:43:07.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:43:07.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:07.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:07.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:07.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:07.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:07 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:43:08.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:07 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:43:08.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:08 smithi115 ceph-mon[65453]: pgmap v2656: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:09.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:08 smithi027 ceph-mon[97244]: pgmap v2656: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:10.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:10 smithi115 ceph-mon[65453]: pgmap v2657: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:10.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5232.486043 secs 2021-11-03T18:43:11.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:10 smithi027 ceph-mon[97244]: pgmap v2657: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:11.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5232.486043 secs 2021-11-03T18:43:12.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:12 smithi115 ceph-mon[65453]: pgmap v2658: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:13.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:12 smithi027 ceph-mon[97244]: pgmap v2658: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:14.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:14 smithi115 ceph-mon[65453]: pgmap v2659: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:15.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:14 smithi027 ceph-mon[97244]: pgmap v2659: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:15.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5237.486162 secs 2021-11-03T18:43:16.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5237.486162 secs 2021-11-03T18:43:16.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:16 smithi115 ceph-mon[65453]: pgmap v2660: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:16.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:43:16.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:43:16.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:43:16.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:16.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:16.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:16.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:17.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:16 smithi027 ceph-mon[97244]: pgmap v2660: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:17.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:43:17.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:43:17.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:43:17.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:17.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:17.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:17.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:17.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:17 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:43:18.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:17 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:43:18.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:18 smithi115 ceph-mon[65453]: pgmap v2661: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:19.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:18 smithi027 ceph-mon[97244]: pgmap v2661: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:20.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:20 smithi115 ceph-mon[65453]: pgmap v2662: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:20.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5242.486281 secs 2021-11-03T18:43:21.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:20 smithi027 ceph-mon[97244]: pgmap v2662: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:21.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5242.486281 secs 2021-11-03T18:43:22.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:22 smithi115 ceph-mon[65453]: pgmap v2663: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:23.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:22 smithi027 ceph-mon[97244]: pgmap v2663: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:24.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:24 smithi115 ceph-mon[65453]: pgmap v2664: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:25.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:24 smithi027 ceph-mon[97244]: pgmap v2664: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:26.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5247.486403 secs 2021-11-03T18:43:26.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5247.486403 secs 2021-11-03T18:43:27.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:26 smithi027 ceph-mon[97244]: pgmap v2665: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:27.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:43:27.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:43:27.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:43:27.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:27.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:27.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:27.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:27.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:26 smithi115 ceph-mon[65453]: pgmap v2665: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:27.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:43:27.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:43:27.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:43:27.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:27.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:27.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:27.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:28.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:27 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:43:28.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:27 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:43:28.406 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.400+0000 7fae480fe700 1 -- 172.21.15.27:0/557924171 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae400f55c0 msgr2=0x7fae400f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:43:28.406 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.400+0000 7fae480fe700 1 --2- 172.21.15.27:0/557924171 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae400f55c0 0x7fae400f59e0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 rx=0x7fae3c004660 tx=0x7fae3c009b30).stop 2021-11-03T18:43:28.407 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.400+0000 7fae480fe700 1 -- 172.21.15.27:0/557924171 shutdown_connections 2021-11-03T18:43:28.407 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.400+0000 7fae480fe700 1 --2- 172.21.15.27:0/557924171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae400f6860 0x7fae400f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:43:28.408 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.400+0000 7fae480fe700 1 --2- 172.21.15.27:0/557924171 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae400f55c0 0x7fae400f59e0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:43:28.408 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.400+0000 7fae480fe700 1 -- 172.21.15.27:0/557924171 >> 172.21.15.27:0/557924171 conn(0x7fae400f0d40 msgr2=0x7fae400f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:43:28.409 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.401+0000 7fae480fe700 1 -- 172.21.15.27:0/557924171 shutdown_connections 2021-11-03T18:43:28.409 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.401+0000 7fae480fe700 1 -- 172.21.15.27:0/557924171 wait complete. 2021-11-03T18:43:28.410 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.401+0000 7fae480fe700 1 Processor -- start 2021-11-03T18:43:28.410 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.401+0000 7fae480fe700 1 -- start start 2021-11-03T18:43:28.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.401+0000 7fae480fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae400f55c0 0x7fae4010c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:43:28.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.401+0000 7fae480fe700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae400f6860 0x7fae4010e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:43:28.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.401+0000 7fae480fe700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae400602b0 con 0x7fae400f55c0 2021-11-03T18:43:28.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.401+0000 7fae480fe700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae400603f0 con 0x7fae400f6860 2021-11-03T18:43:28.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.402+0000 7fae45e9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae400f55c0 0x7fae4010c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:43:28.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.402+0000 7fae45e9a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae400f55c0 0x7fae4010c090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36470/0 (socket says 172.21.15.27:36470) 2021-11-03T18:43:28.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.402+0000 7fae45e9a700 1 -- 172.21.15.27:0/3076911555 learned_addr learned my addr 172.21.15.27:0/3076911555 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:43:28.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.402+0000 7fae45699700 1 --2- 172.21.15.27:0/3076911555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae400f6860 0x7fae4010e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:43:28.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.402+0000 7fae45e9a700 1 -- 172.21.15.27:0/3076911555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae400f6860 msgr2=0x7fae4010e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:43:28.415 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.402+0000 7fae45e9a700 1 --2- 172.21.15.27:0/3076911555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae400f6860 0x7fae4010e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:43:28.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.402+0000 7fae45e9a700 1 -- 172.21.15.27:0/3076911555 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae3c005040 con 0x7fae400f55c0 2021-11-03T18:43:28.416 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.402+0000 7fae45e9a700 1 --2- 172.21.15.27:0/3076911555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae400f55c0 0x7fae4010c090 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 rx=0x7fae3c004f60 tx=0x7fae3c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:43:28.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.402+0000 7fae367fc700 1 -- 172.21.15.27:0/3076911555 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae3c018070 con 0x7fae400f55c0 2021-11-03T18:43:28.417 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.402+0000 7fae480fe700 1 -- 172.21.15.27:0/3076911555 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae4010ea80 con 0x7fae400f55c0 2021-11-03T18:43:28.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.402+0000 7fae480fe700 1 -- 172.21.15.27:0/3076911555 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae4010ee50 con 0x7fae400f55c0 2021-11-03T18:43:28.418 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.402+0000 7fae367fc700 1 -- 172.21.15.27:0/3076911555 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fae3c007ba0 con 0x7fae400f55c0 2021-11-03T18:43:28.419 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.402+0000 7fae367fc700 1 -- 172.21.15.27:0/3076911555 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fae3c01c620 con 0x7fae400f55c0 2021-11-03T18:43:28.419 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.403+0000 7fae480fe700 1 -- 172.21.15.27:0/3076911555 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae400f49b0 con 0x7fae400f55c0 2021-11-03T18:43:28.420 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.404+0000 7fae367fc700 1 -- 172.21.15.27:0/3076911555 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fae3c01c8e0 con 0x7fae400f55c0 2021-11-03T18:43:28.420 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.404+0000 7fae367fc700 1 --2- 172.21.15.27:0/3076911555 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fae2c0640b0 0x7fae2c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:43:28.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.404+0000 7fae367fc700 1 -- 172.21.15.27:0/3076911555 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fae3c0826c0 con 0x7fae400f55c0 2021-11-03T18:43:28.421 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.404+0000 7fae45699700 1 --2- 172.21.15.27:0/3076911555 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fae2c0640b0 0x7fae2c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:43:28.422 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.405+0000 7fae45699700 1 --2- 172.21.15.27:0/3076911555 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fae2c0640b0 0x7fae2c066570 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 rx=0x7fae300049e0 tx=0x7fae30009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:43:28.423 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.408+0000 7fae367fc700 1 -- 172.21.15.27:0/3076911555 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fae3c054a20 con 0x7fae400f55c0 2021-11-03T18:43:28.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.587+0000 7fae480fe700 1 -- 172.21.15.27:0/3076911555 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fae400fde50 con 0x7fae2c0640b0 2021-11-03T18:43:28.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.589+0000 7fae367fc700 1 -- 172.21.15.27:0/3076911555 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fae400fde50 con 0x7fae2c0640b0 2021-11-03T18:43:28.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.591+0000 7fae480fe700 1 -- 172.21.15.27:0/3076911555 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fae2c0640b0 msgr2=0x7fae2c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:43:28.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.591+0000 7fae480fe700 1 --2- 172.21.15.27:0/3076911555 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fae2c0640b0 0x7fae2c066570 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 rx=0x7fae300049e0 tx=0x7fae30009040).stop 2021-11-03T18:43:28.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.591+0000 7fae480fe700 1 -- 172.21.15.27:0/3076911555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae400f55c0 msgr2=0x7fae4010c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:43:28.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.591+0000 7fae480fe700 1 --2- 172.21.15.27:0/3076911555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae400f55c0 0x7fae4010c090 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 rx=0x7fae3c004f60 tx=0x7fae3c009b30).stop 2021-11-03T18:43:28.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.591+0000 7fae480fe700 1 -- 172.21.15.27:0/3076911555 shutdown_connections 2021-11-03T18:43:28.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.591+0000 7fae480fe700 1 --2- 172.21.15.27:0/3076911555 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fae2c0640b0 0x7fae2c066570 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:43:28.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.591+0000 7fae480fe700 1 --2- 172.21.15.27:0/3076911555 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fae400f55c0 0x7fae4010c090 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:43:28.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.591+0000 7fae480fe700 1 --2- 172.21.15.27:0/3076911555 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fae400f6860 0x7fae4010e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:43:28.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.591+0000 7fae480fe700 1 -- 172.21.15.27:0/3076911555 >> 172.21.15.27:0/3076911555 conn(0x7fae400f0d40 msgr2=0x7fae400fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:43:28.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.591+0000 7fae480fe700 1 -- 172.21.15.27:0/3076911555 shutdown_connections 2021-11-03T18:43:28.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.591+0000 7fae480fe700 1 -- 172.21.15.27:0/3076911555 wait complete. 2021-11-03T18:43:28.599 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:43:28.693 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.692+0000 7fe1ff888700 1 -- 172.21.15.27:0/3848577328 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1f80f7940 msgr2=0x7fe1f80f7da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:43:28.693 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.692+0000 7fe1ff888700 1 --2- 172.21.15.27:0/3848577328 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1f80f7940 0x7fe1f80f7da0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 rx=0x7fe1e8004660 tx=0x7fe1e8009b30).stop 2021-11-03T18:43:28.693 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.692+0000 7fe1ff888700 1 -- 172.21.15.27:0/3848577328 shutdown_connections 2021-11-03T18:43:28.694 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.692+0000 7fe1ff888700 1 --2- 172.21.15.27:0/3848577328 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1f80f7940 0x7fe1f80f7da0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:43:28.694 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.692+0000 7fe1ff888700 1 --2- 172.21.15.27:0/3848577328 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1f80f66a0 0x7fe1f80f6ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:43:28.694 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.692+0000 7fe1ff888700 1 -- 172.21.15.27:0/3848577328 >> 172.21.15.27:0/3848577328 conn(0x7fe1f80f1e00 msgr2=0x7fe1f80f4220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:43:28.694 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.692+0000 7fe1ff888700 1 -- 172.21.15.27:0/3848577328 shutdown_connections 2021-11-03T18:43:28.694 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.692+0000 7fe1ff888700 1 -- 172.21.15.27:0/3848577328 wait complete. 2021-11-03T18:43:28.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.692+0000 7fe1ff888700 1 Processor -- start 2021-11-03T18:43:28.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.692+0000 7fe1ff888700 1 -- start start 2021-11-03T18:43:28.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.693+0000 7fe1ff888700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1f80f66a0 0x7fe1f8106e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:43:28.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.693+0000 7fe1ff888700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1f80f7940 0x7fe1f8107340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:43:28.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.693+0000 7fe1ff888700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f8107940 con 0x7fe1f80f66a0 2021-11-03T18:43:28.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.693+0000 7fe1ff888700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f8108150 con 0x7fe1f80f7940 2021-11-03T18:43:28.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.693+0000 7fe1fd624700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1f80f66a0 0x7fe1f8106e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:43:28.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.693+0000 7fe1fd624700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1f80f66a0 0x7fe1f8106e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36480/0 (socket says 172.21.15.27:36480) 2021-11-03T18:43:28.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.693+0000 7fe1fd624700 1 -- 172.21.15.27:0/2177890511 learned_addr learned my addr 172.21.15.27:0/2177890511 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:43:28.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.693+0000 7fe1fce23700 1 --2- 172.21.15.27:0/2177890511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1f80f7940 0x7fe1f8107340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:43:28.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.693+0000 7fe1fd624700 1 -- 172.21.15.27:0/2177890511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1f80f7940 msgr2=0x7fe1f8107340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:43:28.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.693+0000 7fe1fd624700 1 --2- 172.21.15.27:0/2177890511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1f80f7940 0x7fe1f8107340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:43:28.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.693+0000 7fe1fd624700 1 -- 172.21.15.27:0/2177890511 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1e8005040 con 0x7fe1f80f66a0 2021-11-03T18:43:28.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.693+0000 7fe1fd624700 1 --2- 172.21.15.27:0/2177890511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1f80f66a0 0x7fe1f8106e50 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 rx=0x7fe1f400e7c0 tx=0x7fe1f400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:43:28.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.693+0000 7fe1edffb700 1 -- 172.21.15.27:0/2177890511 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1f40092e0 con 0x7fe1f80f66a0 2021-11-03T18:43:28.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.693+0000 7fe1edffb700 1 -- 172.21.15.27:0/2177890511 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fe1f400f040 con 0x7fe1f80f66a0 2021-11-03T18:43:28.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.693+0000 7fe1ff888700 1 -- 172.21.15.27:0/2177890511 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1f81083d0 con 0x7fe1f80f66a0 2021-11-03T18:43:28.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.693+0000 7fe1edffb700 1 -- 172.21.15.27:0/2177890511 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1f4014750 con 0x7fe1f80f66a0 2021-11-03T18:43:28.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.694+0000 7fe1ff888700 1 -- 172.21.15.27:0/2177890511 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1f8108990 con 0x7fe1f80f66a0 2021-11-03T18:43:28.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.695+0000 7fe1ff888700 1 -- 172.21.15.27:0/2177890511 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1f80f5a90 con 0x7fe1f80f66a0 2021-11-03T18:43:28.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.695+0000 7fe1edffb700 1 -- 172.21.15.27:0/2177890511 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fe1f401b030 con 0x7fe1f80f66a0 2021-11-03T18:43:28.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.695+0000 7fe1edffb700 1 --2- 172.21.15.27:0/2177890511 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe1e4064100 0x7fe1e40665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:43:28.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.695+0000 7fe1edffb700 1 -- 172.21.15.27:0/2177890511 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fe1f4080640 con 0x7fe1f80f66a0 2021-11-03T18:43:28.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.695+0000 7fe1fce23700 1 --2- 172.21.15.27:0/2177890511 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe1e4064100 0x7fe1e40665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:43:28.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.696+0000 7fe1fce23700 1 --2- 172.21.15.27:0/2177890511 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe1e4064100 0x7fe1e40665c0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7fe1e8004d80 tx=0x7fe1e800b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:43:28.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.700+0000 7fe1edffb700 1 -- 172.21.15.27:0/2177890511 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fe1f4052900 con 0x7fe1f80f66a0 2021-11-03T18:43:28.875 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.876+0000 7fe1ff888700 1 -- 172.21.15.27:0/2177890511 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe1f80fef70 con 0x7fe1e4064100 2021-11-03T18:43:28.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.890+0000 7fe1edffb700 1 -- 172.21.15.27:0/2177890511 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fe1f80fef70 con 0x7fe1e4064100 2021-11-03T18:43:28.890 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:43:28.890 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (95m) 3m ago 96m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:43:28.890 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (86m) 3m ago 96m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:43:28.891 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (86m) 3m ago 95m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:43:28.891 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (94m) 3m ago 95m 42.9M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:43:28.891 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (88m) 3m ago 91m 26.9M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:43:28.891 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (91m) 3m ago 91m 3852M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:43:28.891 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (87m) 3m ago 91m 26.5M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:43:28.892 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (88m) 3m ago 91m 27.6M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:43:28.892 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (88m) 3m ago 97m 625M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:43:28.892 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (88m) 3m ago 95m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:43:28.892 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (87m) 3m ago 97m 387M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:43:28.893 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (87m) 3m ago 95m 175M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:43:28.893 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (95m) 3m ago 95m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:43:28.893 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (95m) 3m ago 95m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:43:28.893 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (86m) 3m ago 94m 433M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:43:28.894 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (85m) 3m ago 94m 471M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:43:28.894 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (85m) 3m ago 93m 356M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:43:28.894 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (84m) 3m ago 93m 504M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:43:28.894 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (84m) 3m ago 93m 396M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:43:28.895 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (83m) 3m ago 93m 408M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:43:28.895 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (94m) 3m ago 95m 66.8M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:43:28.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.892+0000 7fe1ff888700 1 -- 172.21.15.27:0/2177890511 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe1e4064100 msgr2=0x7fe1e40665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:43:28.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.892+0000 7fe1ff888700 1 --2- 172.21.15.27:0/2177890511 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe1e4064100 0x7fe1e40665c0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7fe1e8004d80 tx=0x7fe1e800b040).stop 2021-11-03T18:43:28.896 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.892+0000 7fe1ff888700 1 -- 172.21.15.27:0/2177890511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1f80f66a0 msgr2=0x7fe1f8106e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:43:28.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.892+0000 7fe1ff888700 1 --2- 172.21.15.27:0/2177890511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1f80f66a0 0x7fe1f8106e50 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 rx=0x7fe1f400e7c0 tx=0x7fe1f400b040).stop 2021-11-03T18:43:28.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.892+0000 7fe1ff888700 1 -- 172.21.15.27:0/2177890511 shutdown_connections 2021-11-03T18:43:28.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.892+0000 7fe1ff888700 1 --2- 172.21.15.27:0/2177890511 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe1e4064100 0x7fe1e40665c0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:43:28.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.892+0000 7fe1ff888700 1 --2- 172.21.15.27:0/2177890511 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe1f80f66a0 0x7fe1f8106e50 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:43:28.897 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.892+0000 7fe1ff888700 1 --2- 172.21.15.27:0/2177890511 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe1f80f7940 0x7fe1f8107340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:43:28.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.892+0000 7fe1ff888700 1 -- 172.21.15.27:0/2177890511 >> 172.21.15.27:0/2177890511 conn(0x7fe1f80f1e00 msgr2=0x7fe1f80fb390 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:43:28.898 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.892+0000 7fe1ff888700 1 -- 172.21.15.27:0/2177890511 shutdown_connections 2021-11-03T18:43:28.899 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.892+0000 7fe1ff888700 1 -- 172.21.15.27:0/2177890511 wait complete. 2021-11-03T18:43:28.992 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.991+0000 7ff0a2d0c700 1 -- 172.21.15.27:0/3716069441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff09c0f55e0 msgr2=0x7ff09c0f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:43:28.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.991+0000 7ff0a2d0c700 1 --2- 172.21.15.27:0/3716069441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff09c0f55e0 0x7ff09c0f5a00 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 rx=0x7ff094004660 tx=0x7ff094009b30).stop 2021-11-03T18:43:28.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.991+0000 7ff0a2d0c700 1 -- 172.21.15.27:0/3716069441 shutdown_connections 2021-11-03T18:43:28.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.991+0000 7ff0a2d0c700 1 --2- 172.21.15.27:0/3716069441 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff09c0f5ef0 0x7ff09c0f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:43:28.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.991+0000 7ff0a2d0c700 1 --2- 172.21.15.27:0/3716069441 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff09c0f55e0 0x7ff09c0f5a00 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:43:28.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.991+0000 7ff0a2d0c700 1 -- 172.21.15.27:0/3716069441 >> 172.21.15.27:0/3716069441 conn(0x7ff09c0f0de0 msgr2=0x7ff09c0f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:43:28.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.991+0000 7ff0a2d0c700 1 -- 172.21.15.27:0/3716069441 shutdown_connections 2021-11-03T18:43:28.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.991+0000 7ff0a2d0c700 1 -- 172.21.15.27:0/3716069441 wait complete. 2021-11-03T18:43:28.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.992+0000 7ff0a2d0c700 1 Processor -- start 2021-11-03T18:43:28.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.992+0000 7ff0a2d0c700 1 -- start start 2021-11-03T18:43:28.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.992+0000 7ff0a2d0c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff09c0f55e0 0x7ff09c106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:43:28.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.992+0000 7ff0a2d0c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff09c0f5ef0 0x7ff09c107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:43:28.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.992+0000 7ff0a2d0c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c107a10 con 0x7ff09c0f5ef0 2021-11-03T18:43:28.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.992+0000 7ff0a2d0c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff09c107b50 con 0x7ff09c0f55e0 2021-11-03T18:43:28.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.992+0000 7ff09bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff09c0f5ef0 0x7ff09c107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:43:28.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.992+0000 7ff09bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff09c0f5ef0 0x7ff09c107460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36492/0 (socket says 172.21.15.27:36492) 2021-11-03T18:43:28.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.992+0000 7ff09bfff700 1 -- 172.21.15.27:0/3720064635 learned_addr learned my addr 172.21.15.27:0/3720064635 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:43:28.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.992+0000 7ff0a0aa8700 1 --2- 172.21.15.27:0/3720064635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff09c0f55e0 0x7ff09c106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:43:28.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.992+0000 7ff09bfff700 1 -- 172.21.15.27:0/3720064635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff09c0f55e0 msgr2=0x7ff09c106f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:43:28.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.992+0000 7ff09bfff700 1 --2- 172.21.15.27:0/3720064635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff09c0f55e0 0x7ff09c106f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:43:28.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.992+0000 7ff09bfff700 1 -- 172.21.15.27:0/3720064635 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff094005040 con 0x7ff09c0f5ef0 2021-11-03T18:43:28.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.992+0000 7ff0a0aa8700 1 --2- 172.21.15.27:0/3720064635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff09c0f55e0 0x7ff09c106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T18:43:28.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.992+0000 7ff09bfff700 1 --2- 172.21.15.27:0/3720064635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff09c0f5ef0 0x7ff09c107460 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 rx=0x7ff08c00c760 tx=0x7ff08c004d20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:43:28.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.993+0000 7ff0997fa700 1 -- 172.21.15.27:0/3720064635 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff08c004200 con 0x7ff09c0f5ef0 2021-11-03T18:43:28.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.993+0000 7ff0997fa700 1 -- 172.21.15.27:0/3720064635 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7ff08c0109c0 con 0x7ff09c0f5ef0 2021-11-03T18:43:29.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.993+0000 7ff0a2d0c700 1 -- 172.21.15.27:0/3720064635 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff09c1084f0 con 0x7ff09c0f5ef0 2021-11-03T18:43:29.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.993+0000 7ff0997fa700 1 -- 172.21.15.27:0/3720064635 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff08c003920 con 0x7ff09c0f5ef0 2021-11-03T18:43:29.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.993+0000 7ff0a2d0c700 1 -- 172.21.15.27:0/3720064635 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff09c108ab0 con 0x7ff09c0f5ef0 2021-11-03T18:43:29.001 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.994+0000 7ff0a2d0c700 1 -- 172.21.15.27:0/3720064635 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff09c045ad0 con 0x7ff09c0f5ef0 2021-11-03T18:43:29.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.994+0000 7ff0997fa700 1 -- 172.21.15.27:0/3720064635 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7ff08c018020 con 0x7ff09c0f5ef0 2021-11-03T18:43:29.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.994+0000 7ff0997fa700 1 --2- 172.21.15.27:0/3720064635 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff0880640b0 0x7ff088066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:43:29.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.995+0000 7ff0997fa700 1 -- 172.21.15.27:0/3720064635 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7ff08c080790 con 0x7ff09c0f5ef0 2021-11-03T18:43:29.002 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.995+0000 7ff0a0aa8700 1 --2- 172.21.15.27:0/3720064635 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff0880640b0 0x7ff088066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:43:29.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.995+0000 7ff0a0aa8700 1 --2- 172.21.15.27:0/3720064635 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff0880640b0 0x7ff088066570 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 rx=0x7ff094009da0 tx=0x7ff094009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:43:29.003 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:28.998+0000 7ff0997fa700 1 -- 172.21.15.27:0/3720064635 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7ff08c052a00 con 0x7ff09c0f5ef0 2021-11-03T18:43:29.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:28 smithi027 ceph-mon[97244]: pgmap v2666: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:29.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.229+0000 7ff0a2d0c700 1 -- 172.21.15.27:0/3720064635 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ff09c10d650 con 0x7ff09c0f5ef0 2021-11-03T18:43:29.230 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.230+0000 7ff0997fa700 1 -- 172.21.15.27:0/3720064635 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7ff08c0525f0 con 0x7ff09c0f5ef0 2021-11-03T18:43:29.231 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:43:29.231 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:43:29.231 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:43:29.232 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:43:29.232 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:43:29.233 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:43:29.233 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:43:29.234 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:43:29.234 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:43:29.235 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:43:29.235 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:43:29.236 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:43:29.236 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:43:29.236 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:43:29.237 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:43:29.237 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:43:29.238 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:43:29.238 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:43:29.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.231+0000 7ff0a2d0c700 1 -- 172.21.15.27:0/3720064635 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff0880640b0 msgr2=0x7ff088066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:43:29.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.231+0000 7ff0a2d0c700 1 --2- 172.21.15.27:0/3720064635 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff0880640b0 0x7ff088066570 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 rx=0x7ff094009da0 tx=0x7ff094009b30).stop 2021-11-03T18:43:29.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.231+0000 7ff0a2d0c700 1 -- 172.21.15.27:0/3720064635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff09c0f5ef0 msgr2=0x7ff09c107460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:43:29.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.231+0000 7ff0a2d0c700 1 --2- 172.21.15.27:0/3720064635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff09c0f5ef0 0x7ff09c107460 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 rx=0x7ff08c00c760 tx=0x7ff08c004d20).stop 2021-11-03T18:43:29.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.232+0000 7ff0a2d0c700 1 -- 172.21.15.27:0/3720064635 shutdown_connections 2021-11-03T18:43:29.242 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.232+0000 7ff0a2d0c700 1 --2- 172.21.15.27:0/3720064635 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff0880640b0 0x7ff088066570 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:43:29.242 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.232+0000 7ff0a2d0c700 1 --2- 172.21.15.27:0/3720064635 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff09c0f55e0 0x7ff09c106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:43:29.243 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.232+0000 7ff0a2d0c700 1 --2- 172.21.15.27:0/3720064635 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff09c0f5ef0 0x7ff09c107460 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:43:29.243 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.232+0000 7ff0a2d0c700 1 -- 172.21.15.27:0/3720064635 >> 172.21.15.27:0/3720064635 conn(0x7ff09c0f0de0 msgr2=0x7ff09c05f740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:43:29.243 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.232+0000 7ff0a2d0c700 1 -- 172.21.15.27:0/3720064635 shutdown_connections 2021-11-03T18:43:29.243 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.232+0000 7ff0a2d0c700 1 -- 172.21.15.27:0/3720064635 wait complete. 2021-11-03T18:43:29.244 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:28 smithi115 ceph-mon[65453]: pgmap v2666: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:29.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.332+0000 7f4ba9e22700 1 -- 172.21.15.27:0/1910432879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ba40f7980 msgr2=0x7f4ba40f7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:43:29.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.332+0000 7f4ba9e22700 1 --2- 172.21.15.27:0/1910432879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ba40f7980 0x7f4ba40f7de0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 rx=0x7f4b94004660 tx=0x7f4b94009b30).stop 2021-11-03T18:43:29.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.332+0000 7f4ba9e22700 1 -- 172.21.15.27:0/1910432879 shutdown_connections 2021-11-03T18:43:29.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.332+0000 7f4ba9e22700 1 --2- 172.21.15.27:0/1910432879 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ba40f7980 0x7f4ba40f7de0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:43:29.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.332+0000 7f4ba9e22700 1 --2- 172.21.15.27:0/1910432879 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ba40f66e0 0x7f4ba40f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:43:29.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.332+0000 7f4ba9e22700 1 -- 172.21.15.27:0/1910432879 >> 172.21.15.27:0/1910432879 conn(0x7f4ba40f1e40 msgr2=0x7f4ba40f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:43:29.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.332+0000 7f4ba9e22700 1 -- 172.21.15.27:0/1910432879 shutdown_connections 2021-11-03T18:43:29.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.332+0000 7f4ba9e22700 1 -- 172.21.15.27:0/1910432879 wait complete. 2021-11-03T18:43:29.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.332+0000 7f4ba9e22700 1 Processor -- start 2021-11-03T18:43:29.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.333+0000 7f4ba9e22700 1 -- start start 2021-11-03T18:43:29.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.333+0000 7f4ba9e22700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ba40f66e0 0x7f4ba4106f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:43:29.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.333+0000 7f4ba9e22700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ba40f7980 0x7f4ba4107400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:43:29.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.333+0000 7f4ba9e22700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba41079b0 con 0x7f4ba40f7980 2021-11-03T18:43:29.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.333+0000 7f4ba9e22700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ba4107af0 con 0x7f4ba40f66e0 2021-11-03T18:43:29.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.333+0000 7f4ba2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ba40f7980 0x7f4ba4107400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:43:29.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.333+0000 7f4ba2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ba40f7980 0x7f4ba4107400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36502/0 (socket says 172.21.15.27:36502) 2021-11-03T18:43:29.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.333+0000 7f4ba2ffd700 1 -- 172.21.15.27:0/1305466567 learned_addr learned my addr 172.21.15.27:0/1305466567 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:43:29.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.333+0000 7f4ba37fe700 1 --2- 172.21.15.27:0/1305466567 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ba40f66e0 0x7f4ba4106f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:43:29.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.333+0000 7f4ba2ffd700 1 -- 172.21.15.27:0/1305466567 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ba40f66e0 msgr2=0x7f4ba4106f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:43:29.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.333+0000 7f4ba2ffd700 1 --2- 172.21.15.27:0/1305466567 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ba40f66e0 0x7f4ba4106f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:43:29.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.333+0000 7f4ba2ffd700 1 -- 172.21.15.27:0/1305466567 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b94005040 con 0x7f4ba40f7980 2021-11-03T18:43:29.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.333+0000 7f4ba37fe700 1 --2- 172.21.15.27:0/1305466567 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ba40f66e0 0x7f4ba4106f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T18:43:29.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.333+0000 7f4ba2ffd700 1 --2- 172.21.15.27:0/1305466567 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ba40f7980 0x7f4ba4107400 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 rx=0x7f4b94009990 tx=0x7f4b94009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:43:29.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.333+0000 7f4b8bfff700 1 -- 172.21.15.27:0/1305466567 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b94018070 con 0x7f4ba40f7980 2021-11-03T18:43:29.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.334+0000 7f4b8bfff700 1 -- 172.21.15.27:0/1305466567 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f4b94007ba0 con 0x7f4ba40f7980 2021-11-03T18:43:29.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.334+0000 7f4ba9e22700 1 -- 172.21.15.27:0/1305466567 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ba4108430 con 0x7f4ba40f7980 2021-11-03T18:43:29.339 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.334+0000 7f4b8bfff700 1 -- 172.21.15.27:0/1305466567 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b9401c3c0 con 0x7f4ba40f7980 2021-11-03T18:43:29.340 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.334+0000 7f4ba9e22700 1 -- 172.21.15.27:0/1305466567 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ba4108990 con 0x7f4ba40f7980 2021-11-03T18:43:29.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.335+0000 7f4ba9e22700 1 -- 172.21.15.27:0/1305466567 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ba4045ad0 con 0x7f4ba40f7980 2021-11-03T18:43:29.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.335+0000 7f4b8bfff700 1 -- 172.21.15.27:0/1305466567 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f4b9401d420 con 0x7f4ba40f7980 2021-11-03T18:43:29.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.335+0000 7f4b8bfff700 1 --2- 172.21.15.27:0/1305466567 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4b8c0640b0 0x7f4b8c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:43:29.341 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.336+0000 7f4b8bfff700 1 -- 172.21.15.27:0/1305466567 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f4b94082490 con 0x7f4ba40f7980 2021-11-03T18:43:29.342 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.336+0000 7f4ba37fe700 1 --2- 172.21.15.27:0/1305466567 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4b8c0640b0 0x7f4b8c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:43:29.342 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.336+0000 7f4ba37fe700 1 --2- 172.21.15.27:0/1305466567 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4b8c0640b0 0x7f4b8c066570 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7f4b98009880 tx=0x7f4b9800b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:43:29.342 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.340+0000 7f4b8bfff700 1 -- 172.21.15.27:0/1305466567 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f4b940548b0 con 0x7f4ba40f7980 2021-11-03T18:43:29.542 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.542+0000 7f4ba9e22700 1 -- 172.21.15.27:0/1305466567 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f4ba40fef40 con 0x7f4ba40f7980 2021-11-03T18:43:29.542 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.543+0000 7f4b8bfff700 1 -- 172.21.15.27:0/1305466567 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f4b940544a0 con 0x7f4ba40f7980 2021-11-03T18:43:29.543 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:43:29.543 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:43:29.543 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:43:29.543 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:43:29.544 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:43:29.544 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:43:29.544 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:43:29.544 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:43:29.544 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:43:29.545 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:43:29.545 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:43:29.545 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:43:29.545 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:43:29.546 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:43:29.546 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:43:29.546 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:43:29.546 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:43:29.546 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:43:29.547 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:43:29.547 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:43:29.547 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:43:29.547 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:43:29.548 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:43:29.548 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:43:29.548 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:43:29.548 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:43:29.548 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:43:29.549 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:43:29.549 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:43:29.549 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:43:29.549 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:43:29.549 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:43:29.550 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:43:29.550 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:43:29.550 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:43:29.550 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:43:29.551 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:43:29.551 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:43:29.551 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:43:29.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.544+0000 7f4ba9e22700 1 -- 172.21.15.27:0/1305466567 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4b8c0640b0 msgr2=0x7f4b8c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:43:29.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.544+0000 7f4ba9e22700 1 --2- 172.21.15.27:0/1305466567 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4b8c0640b0 0x7f4b8c066570 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7f4b98009880 tx=0x7f4b9800b040).stop 2021-11-03T18:43:29.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.544+0000 7f4ba9e22700 1 -- 172.21.15.27:0/1305466567 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ba40f7980 msgr2=0x7f4ba4107400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:43:29.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.544+0000 7f4ba9e22700 1 --2- 172.21.15.27:0/1305466567 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ba40f7980 0x7f4ba4107400 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 rx=0x7f4b94009990 tx=0x7f4b94009b30).stop 2021-11-03T18:43:29.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.545+0000 7f4ba9e22700 1 -- 172.21.15.27:0/1305466567 shutdown_connections 2021-11-03T18:43:29.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.545+0000 7f4ba9e22700 1 --2- 172.21.15.27:0/1305466567 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4b8c0640b0 0x7f4b8c066570 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:43:29.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.545+0000 7f4ba9e22700 1 --2- 172.21.15.27:0/1305466567 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f4ba40f66e0 0x7f4ba4106f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:43:29.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.545+0000 7f4ba9e22700 1 --2- 172.21.15.27:0/1305466567 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f4ba40f7980 0x7f4ba4107400 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:43:29.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.545+0000 7f4ba9e22700 1 -- 172.21.15.27:0/1305466567 >> 172.21.15.27:0/1305466567 conn(0x7f4ba40f1e40 msgr2=0x7f4ba40fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:43:29.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.545+0000 7f4ba9e22700 1 -- 172.21.15.27:0/1305466567 shutdown_connections 2021-11-03T18:43:29.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:29.545+0000 7f4ba9e22700 1 -- 172.21.15.27:0/1305466567 wait complete. 2021-11-03T18:43:29.555 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:43:30.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:29 smithi027 ceph-mon[97244]: from='client.36840 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:43:30.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:29 smithi027 ceph-mon[97244]: from='client.36844 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:43:30.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:29 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3720064635' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:30.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:29 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1305466567' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:43:30.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:29 smithi115 ceph-mon[65453]: from='client.36840 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:43:30.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:29 smithi115 ceph-mon[65453]: from='client.36844 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:43:30.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:29 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3720064635' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:30.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:29 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1305466567' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:43:31.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:30 smithi027 ceph-mon[97244]: pgmap v2667: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:31.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5252.486434 secs 2021-11-03T18:43:31.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:30 smithi115 ceph-mon[65453]: pgmap v2667: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:31.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5252.486434 secs 2021-11-03T18:43:33.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:32 smithi027 ceph-mon[97244]: pgmap v2668: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:33.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:32 smithi115 ceph-mon[65453]: pgmap v2668: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:35.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:34 smithi027 ceph-mon[97244]: pgmap v2669: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:35.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:34 smithi115 ceph-mon[65453]: pgmap v2669: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:36.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5257.486539 secs 2021-11-03T18:43:36.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5257.486539 secs 2021-11-03T18:43:37.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:36 smithi115 ceph-mon[65453]: pgmap v2670: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:37.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:43:37.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:43:37.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:43:37.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:37.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:37.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:37.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:36 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:37.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:36 smithi027 ceph-mon[97244]: pgmap v2670: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:37.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:43:37.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:43:37.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:43:37.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:37.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:37.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:37.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:36 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:38.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:37 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:43:38.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:37 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:43:39.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:38 smithi115 ceph-mon[65453]: pgmap v2671: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:39.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:38 smithi027 ceph-mon[97244]: pgmap v2671: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:41.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:40 smithi115 ceph-mon[65453]: pgmap v2672: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:41.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5262.486629 secs 2021-11-03T18:43:41.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:40 smithi027 ceph-mon[97244]: pgmap v2672: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:41.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5262.486629 secs 2021-11-03T18:43:43.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:42 smithi115 ceph-mon[65453]: pgmap v2673: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:43.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:42 smithi027 ceph-mon[97244]: pgmap v2673: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:45.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:44 smithi115 ceph-mon[65453]: pgmap v2674: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:45.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:44 smithi027 ceph-mon[97244]: pgmap v2674: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:46.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5267.486706 secs 2021-11-03T18:43:46.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5267.486706 secs 2021-11-03T18:43:46.894 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:46 smithi115 ceph-mon[65453]: pgmap v2675: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:46.895 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:46 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:43:46.899 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:46 smithi027 ceph-mon[97244]: pgmap v2675: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:46.900 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:46 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:43:48.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:47 smithi115 ceph-mon[65453]: poll took 60164.519 ms: 1 events 2021-11-03T18:43:48.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:43:48.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:43:48.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:48.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:48.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:48.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:48.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:47 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:43:48.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:47 smithi027 ceph-mon[97244]: poll took 60164.519 ms: 1 events 2021-11-03T18:43:48.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:43:48.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:43:48.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:48.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:48.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:48.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:48.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:47 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:43:49.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:48 smithi115 ceph-mon[65453]: pgmap v2676: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:49.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:48 smithi027 ceph-mon[97244]: pgmap v2676: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:51.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:50 smithi115 ceph-mon[65453]: pgmap v2677: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:51.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5272.486811 secs 2021-11-03T18:43:51.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:50 smithi027 ceph-mon[97244]: pgmap v2677: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:51.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5272.486811 secs 2021-11-03T18:43:53.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:52 smithi115 ceph-mon[65453]: pgmap v2678: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:53.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:52 smithi027 ceph-mon[97244]: pgmap v2678: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:55.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:54 smithi115 ceph-mon[65453]: pgmap v2679: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:55.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:54 smithi027 ceph-mon[97244]: pgmap v2679: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:56.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5277.486952 secs 2021-11-03T18:43:56.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:43:56.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:43:56.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5277.486952 secs 2021-11-03T18:43:56.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:43:56.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:43:57.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:56 smithi115 ceph-mon[65453]: pgmap v2680: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:57.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:43:57.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:43:57.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:56 smithi027 ceph-mon[97244]: pgmap v2680: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:57.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:43:57.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:43:58.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:43:58.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:58.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:58.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:58.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:58.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:57 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:43:58.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:43:57 smithi115 ceph-mon[65453]: pgmap v2681: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:58.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:43:58.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:58.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:58.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:58.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:43:58.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:57 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:43:58.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:43:57 smithi027 ceph-mon[97244]: pgmap v2681: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:43:59.659 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.659+0000 7f2bbdb72700 1 -- 172.21.15.27:0/3177347740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bb80f6ff0 msgr2=0x7f2bb80f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:43:59.659 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.659+0000 7f2bbdb72700 1 --2- 172.21.15.27:0/3177347740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bb80f6ff0 0x7f2bb80f7a20 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 rx=0x7f2bac004660 tx=0x7f2bac009b30).stop 2021-11-03T18:43:59.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.659+0000 7f2bbdb72700 1 -- 172.21.15.27:0/3177347740 shutdown_connections 2021-11-03T18:43:59.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.659+0000 7f2bbdb72700 1 --2- 172.21.15.27:0/3177347740 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bb80f6ff0 0x7f2bb80f7a20 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:43:59.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.659+0000 7f2bbdb72700 1 --2- 172.21.15.27:0/3177347740 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bb80f66e0 0x7f2bb80f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:43:59.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.659+0000 7f2bbdb72700 1 -- 172.21.15.27:0/3177347740 >> 172.21.15.27:0/3177347740 conn(0x7f2bb80f1f00 msgr2=0x7f2bb80f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:43:59.661 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.659+0000 7f2bbdb72700 1 -- 172.21.15.27:0/3177347740 shutdown_connections 2021-11-03T18:43:59.661 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.659+0000 7f2bbdb72700 1 -- 172.21.15.27:0/3177347740 wait complete. 2021-11-03T18:43:59.661 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.660+0000 7f2bbdb72700 1 Processor -- start 2021-11-03T18:43:59.661 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.660+0000 7f2bbdb72700 1 -- start start 2021-11-03T18:43:59.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.660+0000 7f2bbdb72700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bb80f66e0 0x7f2bb8106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:43:59.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.660+0000 7f2bbdb72700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bb80f6ff0 0x7f2bb8107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:43:59.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.660+0000 7f2bbdb72700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb8107a10 con 0x7f2bb80f6ff0 2021-11-03T18:43:59.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.660+0000 7f2bbdb72700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb8107b50 con 0x7f2bb80f66e0 2021-11-03T18:43:59.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.660+0000 7f2bb6ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bb80f6ff0 0x7f2bb8107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:43:59.665 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.660+0000 7f2bb6ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bb80f6ff0 0x7f2bb8107460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36520/0 (socket says 172.21.15.27:36520) 2021-11-03T18:43:59.665 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.660+0000 7f2bb6ffd700 1 -- 172.21.15.27:0/3027795069 learned_addr learned my addr 172.21.15.27:0/3027795069 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:43:59.665 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.661+0000 7f2bb77fe700 1 --2- 172.21.15.27:0/3027795069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bb80f66e0 0x7f2bb8106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:43:59.665 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.661+0000 7f2bb6ffd700 1 -- 172.21.15.27:0/3027795069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bb80f66e0 msgr2=0x7f2bb8106f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:43:59.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.661+0000 7f2bb6ffd700 1 --2- 172.21.15.27:0/3027795069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bb80f66e0 0x7f2bb8106f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:43:59.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.661+0000 7f2bb6ffd700 1 -- 172.21.15.27:0/3027795069 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bac005040 con 0x7f2bb80f6ff0 2021-11-03T18:43:59.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.661+0000 7f2bb6ffd700 1 --2- 172.21.15.27:0/3027795069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bb80f6ff0 0x7f2bb8107460 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 rx=0x7f2bac009990 tx=0x7f2bac009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:43:59.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.661+0000 7f2b9ffff700 1 -- 172.21.15.27:0/3027795069 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bac018070 con 0x7f2bb80f6ff0 2021-11-03T18:43:59.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.661+0000 7f2bbdb72700 1 -- 172.21.15.27:0/3027795069 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bb8108490 con 0x7f2bb80f6ff0 2021-11-03T18:43:59.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.661+0000 7f2b9ffff700 1 -- 172.21.15.27:0/3027795069 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f2bac007ba0 con 0x7f2bb80f6ff0 2021-11-03T18:43:59.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.661+0000 7f2bbdb72700 1 -- 172.21.15.27:0/3027795069 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bb81089f0 con 0x7f2bb80f6ff0 2021-11-03T18:43:59.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.661+0000 7f2b9ffff700 1 -- 172.21.15.27:0/3027795069 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2bac01c3c0 con 0x7f2bb80f6ff0 2021-11-03T18:43:59.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.662+0000 7f2bbdb72700 1 -- 172.21.15.27:0/3027795069 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bb8045ad0 con 0x7f2bb80f6ff0 2021-11-03T18:43:59.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.663+0000 7f2b9ffff700 1 -- 172.21.15.27:0/3027795069 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f2bac01d420 con 0x7f2bb80f6ff0 2021-11-03T18:43:59.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.663+0000 7f2b9ffff700 1 --2- 172.21.15.27:0/3027795069 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2ba00640b0 0x7f2ba0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:43:59.668 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.663+0000 7f2b9ffff700 1 -- 172.21.15.27:0/3027795069 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f2bac082340 con 0x7f2bb80f6ff0 2021-11-03T18:43:59.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.663+0000 7f2bb77fe700 1 --2- 172.21.15.27:0/3027795069 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2ba00640b0 0x7f2ba0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:43:59.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.664+0000 7f2bb77fe700 1 --2- 172.21.15.27:0/3027795069 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2ba00640b0 0x7f2ba0066570 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7f2ba8004c40 tx=0x7f2ba800b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:43:59.669 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.667+0000 7f2b9ffff700 1 -- 172.21.15.27:0/3027795069 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f2bac054760 con 0x7f2bb80f6ff0 2021-11-03T18:43:59.846 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.847+0000 7f2bbdb72700 1 -- 172.21.15.27:0/3027795069 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2bb80ff010 con 0x7f2ba00640b0 2021-11-03T18:43:59.848 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.848+0000 7f2b9ffff700 1 -- 172.21.15.27:0/3027795069 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f2bb80ff010 con 0x7f2ba00640b0 2021-11-03T18:43:59.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.850+0000 7f2bbdb72700 1 -- 172.21.15.27:0/3027795069 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2ba00640b0 msgr2=0x7f2ba0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:43:59.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.850+0000 7f2bbdb72700 1 --2- 172.21.15.27:0/3027795069 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2ba00640b0 0x7f2ba0066570 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7f2ba8004c40 tx=0x7f2ba800b040).stop 2021-11-03T18:43:59.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.850+0000 7f2bbdb72700 1 -- 172.21.15.27:0/3027795069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bb80f6ff0 msgr2=0x7f2bb8107460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:43:59.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.850+0000 7f2bbdb72700 1 --2- 172.21.15.27:0/3027795069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bb80f6ff0 0x7f2bb8107460 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 rx=0x7f2bac009990 tx=0x7f2bac009b30).stop 2021-11-03T18:43:59.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.850+0000 7f2bbdb72700 1 -- 172.21.15.27:0/3027795069 shutdown_connections 2021-11-03T18:43:59.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.850+0000 7f2bbdb72700 1 --2- 172.21.15.27:0/3027795069 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2ba00640b0 0x7f2ba0066570 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:43:59.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.850+0000 7f2bbdb72700 1 --2- 172.21.15.27:0/3027795069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f2bb80f66e0 0x7f2bb8106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:43:59.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.850+0000 7f2bbdb72700 1 --2- 172.21.15.27:0/3027795069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f2bb80f6ff0 0x7f2bb8107460 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:43:59.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.850+0000 7f2bbdb72700 1 -- 172.21.15.27:0/3027795069 >> 172.21.15.27:0/3027795069 conn(0x7f2bb80f1f00 msgr2=0x7f2bb80fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:43:59.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.850+0000 7f2bbdb72700 1 -- 172.21.15.27:0/3027795069 shutdown_connections 2021-11-03T18:43:59.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.850+0000 7f2bbdb72700 1 -- 172.21.15.27:0/3027795069 wait complete. 2021-11-03T18:43:59.859 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:43:59.950 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.950+0000 7fc406427700 1 -- 172.21.15.27:0/2098547195 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4000f6530 msgr2=0x7fc4000f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:43:59.950 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.950+0000 7fc406427700 1 --2- 172.21.15.27:0/2098547195 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4000f6530 0x7fc4000f6950 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 rx=0x7fc3fc004660 tx=0x7fc3fc009b30).stop 2021-11-03T18:43:59.950 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.950+0000 7fc406427700 1 -- 172.21.15.27:0/2098547195 shutdown_connections 2021-11-03T18:43:59.951 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.950+0000 7fc406427700 1 --2- 172.21.15.27:0/2098547195 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4000f77d0 0x7fc4000f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T18:43:59.951 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.950+0000 7fc406427700 1 --2- 172.21.15.27:0/2098547195 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4000f6530 0x7fc4000f6950 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:43:59.951 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.950+0000 7fc406427700 1 -- 172.21.15.27:0/2098547195 >> 172.21.15.27:0/2098547195 conn(0x7fc4000f1cd0 msgr2=0x7fc4000f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:43:59.952 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.950+0000 7fc406427700 1 -- 172.21.15.27:0/2098547195 shutdown_connections 2021-11-03T18:43:59.952 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.950+0000 7fc406427700 1 -- 172.21.15.27:0/2098547195 wait complete. 2021-11-03T18:43:59.956 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.951+0000 7fc406427700 1 Processor -- start 2021-11-03T18:43:59.956 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.951+0000 7fc406427700 1 -- start start 2021-11-03T18:43:59.956 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.951+0000 7fc406427700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4000f6530 0x7fc400106da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:43:59.957 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.951+0000 7fc406427700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4000f77d0 0x7fc400107290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:43:59.957 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.951+0000 7fc406427700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc400107920 con 0x7fc4000f6530 2021-11-03T18:43:59.957 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.951+0000 7fc406427700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4001080d0 con 0x7fc4000f77d0 2021-11-03T18:43:59.957 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.951+0000 7fc405425700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4000f6530 0x7fc400106da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:43:59.958 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.951+0000 7fc405425700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4000f6530 0x7fc400106da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36528/0 (socket says 172.21.15.27:36528) 2021-11-03T18:43:59.958 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.951+0000 7fc405425700 1 -- 172.21.15.27:0/183348696 learned_addr learned my addr 172.21.15.27:0/183348696 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:43:59.959 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.952+0000 7fc405425700 1 -- 172.21.15.27:0/183348696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4000f77d0 msgr2=0x7fc400107290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:43:59.959 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.952+0000 7fc405425700 1 --2- 172.21.15.27:0/183348696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4000f77d0 0x7fc400107290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:43:59.959 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.952+0000 7fc405425700 1 -- 172.21.15.27:0/183348696 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3fc005040 con 0x7fc4000f6530 2021-11-03T18:43:59.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.952+0000 7fc405425700 1 --2- 172.21.15.27:0/183348696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4000f6530 0x7fc400106da0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 rx=0x7fc3fc004f60 tx=0x7fc3fc009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:43:59.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.952+0000 7fc3f5ffb700 1 -- 172.21.15.27:0/183348696 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3fc018070 con 0x7fc4000f6530 2021-11-03T18:43:59.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.952+0000 7fc3f5ffb700 1 -- 172.21.15.27:0/183348696 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fc3fc007ba0 con 0x7fc4000f6530 2021-11-03T18:43:59.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.952+0000 7fc406427700 1 -- 172.21.15.27:0/183348696 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc400108350 con 0x7fc4000f6530 2021-11-03T18:43:59.961 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.952+0000 7fc3f5ffb700 1 -- 172.21.15.27:0/183348696 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3fc01c620 con 0x7fc4000f6530 2021-11-03T18:43:59.961 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.952+0000 7fc406427700 1 -- 172.21.15.27:0/183348696 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4001088b0 con 0x7fc4000f6530 2021-11-03T18:43:59.961 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.953+0000 7fc406427700 1 -- 172.21.15.27:0/183348696 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc400059100 con 0x7fc4000f6530 2021-11-03T18:43:59.961 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.954+0000 7fc3f5ffb700 1 -- 172.21.15.27:0/183348696 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fc3fc01d420 con 0x7fc4000f6530 2021-11-03T18:43:59.962 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.954+0000 7fc3f5ffb700 1 --2- 172.21.15.27:0/183348696 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc3ec064100 0x7fc3ec0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:43:59.962 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.954+0000 7fc3f5ffb700 1 -- 172.21.15.27:0/183348696 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fc3fc082620 con 0x7fc4000f6530 2021-11-03T18:43:59.962 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.954+0000 7fc404c24700 1 --2- 172.21.15.27:0/183348696 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc3ec064100 0x7fc3ec0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:43:59.962 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.954+0000 7fc404c24700 1 --2- 172.21.15.27:0/183348696 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc3ec064100 0x7fc3ec0665c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 rx=0x7fc400106b30 tx=0x7fc3f0006ab0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:43:59.963 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:43:59.958+0000 7fc3f5ffb700 1 -- 172.21.15.27:0/183348696 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fc3fc054a40 con 0x7fc4000f6530 2021-11-03T18:44:00.129 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.130+0000 7fc406427700 1 -- 172.21.15.27:0/183348696 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4000fede0 con 0x7fc3ec064100 2021-11-03T18:44:00.144 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.145+0000 7fc3f5ffb700 1 -- 172.21.15.27:0/183348696 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7fc4000fede0 con 0x7fc3ec064100 2021-11-03T18:44:00.145 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:44:00.145 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (95m) 4m ago 96m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:44:00.145 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (87m) 4m ago 96m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:44:00.145 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (87m) 3m ago 96m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:44:00.146 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (95m) 4m ago 96m 42.9M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:44:00.146 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (89m) 4m ago 91m 26.9M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:44:00.146 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (92m) 4m ago 92m 3852M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:44:00.146 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (87m) 3m ago 91m 26.5M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:44:00.147 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (89m) 3m ago 92m 27.6M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:44:00.147 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (89m) 4m ago 98m 625M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:44:00.147 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (88m) 3m ago 96m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:44:00.147 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (88m) 4m ago 98m 387M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:44:00.148 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (87m) 3m ago 95m 175M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:44:00.148 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (96m) 4m ago 96m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:44:00.148 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (95m) 3m ago 95m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:44:00.148 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (86m) 4m ago 95m 433M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:44:00.148 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (86m) 4m ago 94m 471M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:44:00.149 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (85m) 4m ago 94m 356M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:44:00.149 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (85m) 3m ago 94m 504M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:44:00.149 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (84m) 3m ago 93m 396M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:44:00.149 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (84m) 3m ago 93m 408M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:44:00.150 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (95m) 4m ago 96m 66.8M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:44:00.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.146+0000 7fc406427700 1 -- 172.21.15.27:0/183348696 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc3ec064100 msgr2=0x7fc3ec0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:44:00.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.146+0000 7fc406427700 1 --2- 172.21.15.27:0/183348696 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc3ec064100 0x7fc3ec0665c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 rx=0x7fc400106b30 tx=0x7fc3f0006ab0).stop 2021-11-03T18:44:00.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.146+0000 7fc406427700 1 -- 172.21.15.27:0/183348696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4000f6530 msgr2=0x7fc400106da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:44:00.151 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.146+0000 7fc406427700 1 --2- 172.21.15.27:0/183348696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4000f6530 0x7fc400106da0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 rx=0x7fc3fc004f60 tx=0x7fc3fc009b30).stop 2021-11-03T18:44:00.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.147+0000 7fc406427700 1 -- 172.21.15.27:0/183348696 shutdown_connections 2021-11-03T18:44:00.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.147+0000 7fc406427700 1 --2- 172.21.15.27:0/183348696 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc3ec064100 0x7fc3ec0665c0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:44:00.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.147+0000 7fc406427700 1 --2- 172.21.15.27:0/183348696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc4000f6530 0x7fc400106da0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:44:00.152 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.147+0000 7fc406427700 1 --2- 172.21.15.27:0/183348696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc4000f77d0 0x7fc400107290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:44:00.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.147+0000 7fc406427700 1 -- 172.21.15.27:0/183348696 >> 172.21.15.27:0/183348696 conn(0x7fc4000f1cd0 msgr2=0x7fc4000fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:44:00.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.147+0000 7fc406427700 1 -- 172.21.15.27:0/183348696 shutdown_connections 2021-11-03T18:44:00.153 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.147+0000 7fc406427700 1 -- 172.21.15.27:0/183348696 wait complete. 2021-11-03T18:44:00.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.246+0000 7f7dac8e3700 1 -- 172.21.15.27:0/1623455837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7da80f5ef0 msgr2=0x7f7da80f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:44:00.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.246+0000 7f7dac8e3700 1 --2- 172.21.15.27:0/1623455837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7da80f5ef0 0x7f7da80f6920 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 rx=0x7f7d9c004660 tx=0x7f7d9c009b30).stop 2021-11-03T18:44:00.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.246+0000 7f7dac8e3700 1 -- 172.21.15.27:0/1623455837 shutdown_connections 2021-11-03T18:44:00.248 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.246+0000 7f7dac8e3700 1 --2- 172.21.15.27:0/1623455837 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7da80f5ef0 0x7f7da80f6920 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:44:00.248 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.246+0000 7f7dac8e3700 1 --2- 172.21.15.27:0/1623455837 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7da80f55e0 0x7f7da80f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:44:00.248 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.246+0000 7f7dac8e3700 1 -- 172.21.15.27:0/1623455837 >> 172.21.15.27:0/1623455837 conn(0x7f7da80f0de0 msgr2=0x7f7da80f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:44:00.248 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.246+0000 7f7dac8e3700 1 -- 172.21.15.27:0/1623455837 shutdown_connections 2021-11-03T18:44:00.249 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.246+0000 7f7dac8e3700 1 -- 172.21.15.27:0/1623455837 wait complete. 2021-11-03T18:44:00.249 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.246+0000 7f7dac8e3700 1 Processor -- start 2021-11-03T18:44:00.249 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.246+0000 7f7dac8e3700 1 -- start start 2021-11-03T18:44:00.249 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.247+0000 7f7dac8e3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7da80f55e0 0x7f7da810c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:44:00.249 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.247+0000 7f7dac8e3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7da80f5ef0 0x7f7da810e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:44:00.250 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.247+0000 7f7dac8e3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da80603b0 con 0x7f7da80f5ef0 2021-11-03T18:44:00.250 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.247+0000 7f7dac8e3700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da80604f0 con 0x7f7da80f55e0 2021-11-03T18:44:00.250 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.247+0000 7f7da5d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7da80f5ef0 0x7f7da810e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:44:00.251 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.247+0000 7f7da5d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7da80f5ef0 0x7f7da810e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36540/0 (socket says 172.21.15.27:36540) 2021-11-03T18:44:00.251 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.247+0000 7f7da5d9b700 1 -- 172.21.15.27:0/493361023 learned_addr learned my addr 172.21.15.27:0/493361023 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:44:00.251 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.247+0000 7f7da659c700 1 --2- 172.21.15.27:0/493361023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7da80f55e0 0x7f7da810c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:44:00.251 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.247+0000 7f7da5d9b700 1 -- 172.21.15.27:0/493361023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7da80f55e0 msgr2=0x7f7da810c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:44:00.251 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.247+0000 7f7da5d9b700 1 --2- 172.21.15.27:0/493361023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7da80f55e0 0x7f7da810c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:44:00.252 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.247+0000 7f7da5d9b700 1 -- 172.21.15.27:0/493361023 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d9c005040 con 0x7f7da80f5ef0 2021-11-03T18:44:00.252 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.247+0000 7f7da5d9b700 1 --2- 172.21.15.27:0/493361023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7da80f5ef0 0x7f7da810e5b0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 rx=0x7f7d9c009990 tx=0x7f7d9c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:44:00.252 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.247+0000 7f7d96ffd700 1 -- 172.21.15.27:0/493361023 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d9c018070 con 0x7f7da80f5ef0 2021-11-03T18:44:00.252 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.247+0000 7f7d96ffd700 1 -- 172.21.15.27:0/493361023 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f7d9c007ba0 con 0x7f7da80f5ef0 2021-11-03T18:44:00.253 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.247+0000 7f7dac8e3700 1 -- 172.21.15.27:0/493361023 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7da810eaa0 con 0x7f7da80f5ef0 2021-11-03T18:44:00.253 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.247+0000 7f7d96ffd700 1 -- 172.21.15.27:0/493361023 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7d9c01c3c0 con 0x7f7da80f5ef0 2021-11-03T18:44:00.253 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.247+0000 7f7dac8e3700 1 -- 172.21.15.27:0/493361023 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7da810ee70 con 0x7f7da80f5ef0 2021-11-03T18:44:00.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.248+0000 7f7dac8e3700 1 -- 172.21.15.27:0/493361023 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7da8045ad0 con 0x7f7da80f5ef0 2021-11-03T18:44:00.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.249+0000 7f7d96ffd700 1 -- 172.21.15.27:0/493361023 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f7d9c01d420 con 0x7f7da80f5ef0 2021-11-03T18:44:00.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.249+0000 7f7d96ffd700 1 --2- 172.21.15.27:0/493361023 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7d900640b0 0x7f7d90066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:44:00.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.249+0000 7f7d96ffd700 1 -- 172.21.15.27:0/493361023 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f7d9c0823e0 con 0x7f7da80f5ef0 2021-11-03T18:44:00.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.249+0000 7f7da659c700 1 --2- 172.21.15.27:0/493361023 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7d900640b0 0x7f7d90066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:44:00.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.250+0000 7f7da659c700 1 --2- 172.21.15.27:0/493361023 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7d900640b0 0x7f7d90066570 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7f7da8060140 tx=0x7f7da000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:44:00.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.253+0000 7f7d96ffd700 1 -- 172.21.15.27:0/493361023 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f7d9c0547a0 con 0x7f7da80f5ef0 2021-11-03T18:44:00.485 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.485+0000 7f7dac8e3700 1 -- 172.21.15.27:0/493361023 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f7da8027a00 con 0x7f7da80f5ef0 2021-11-03T18:44:00.485 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.486+0000 7f7d96ffd700 1 -- 172.21.15.27:0/493361023 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f7d9c054390 con 0x7f7da80f5ef0 2021-11-03T18:44:00.486 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:44:00.486 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:44:00.486 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:44:00.486 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:44:00.487 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:44:00.487 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:44:00.487 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:44:00.487 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:44:00.487 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:44:00.488 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:44:00.488 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:44:00.488 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:44:00.488 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:44:00.489 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:44:00.489 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:44:00.489 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:44:00.489 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:44:00.490 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:44:00.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.488+0000 7f7dac8e3700 1 -- 172.21.15.27:0/493361023 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7d900640b0 msgr2=0x7f7d90066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:44:00.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.488+0000 7f7dac8e3700 1 --2- 172.21.15.27:0/493361023 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7d900640b0 0x7f7d90066570 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 rx=0x7f7da8060140 tx=0x7f7da000b040).stop 2021-11-03T18:44:00.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.488+0000 7f7dac8e3700 1 -- 172.21.15.27:0/493361023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7da80f5ef0 msgr2=0x7f7da810e5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:44:00.491 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.488+0000 7f7dac8e3700 1 --2- 172.21.15.27:0/493361023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7da80f5ef0 0x7f7da810e5b0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 rx=0x7f7d9c009990 tx=0x7f7d9c009b30).stop 2021-11-03T18:44:00.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.488+0000 7f7dac8e3700 1 -- 172.21.15.27:0/493361023 shutdown_connections 2021-11-03T18:44:00.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.488+0000 7f7dac8e3700 1 --2- 172.21.15.27:0/493361023 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f7d900640b0 0x7f7d90066570 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:44:00.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.488+0000 7f7dac8e3700 1 --2- 172.21.15.27:0/493361023 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f7da80f55e0 0x7f7da810c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:44:00.492 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.488+0000 7f7dac8e3700 1 --2- 172.21.15.27:0/493361023 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f7da80f5ef0 0x7f7da810e5b0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:44:00.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.488+0000 7f7dac8e3700 1 -- 172.21.15.27:0/493361023 >> 172.21.15.27:0/493361023 conn(0x7f7da80f0de0 msgr2=0x7f7da80fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:44:00.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.488+0000 7f7dac8e3700 1 -- 172.21.15.27:0/493361023 shutdown_connections 2021-11-03T18:44:00.493 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.488+0000 7f7dac8e3700 1 -- 172.21.15.27:0/493361023 wait complete. 2021-11-03T18:44:00.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.588+0000 7f323accb700 1 -- 172.21.15.27:0/1323153496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32340f5ef0 msgr2=0x7f32340f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:44:00.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.588+0000 7f323accb700 1 --2- 172.21.15.27:0/1323153496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32340f5ef0 0x7f32340f6920 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 rx=0x7f3220004660 tx=0x7f3220009b30).stop 2021-11-03T18:44:00.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.588+0000 7f323accb700 1 -- 172.21.15.27:0/1323153496 shutdown_connections 2021-11-03T18:44:00.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.588+0000 7f323accb700 1 --2- 172.21.15.27:0/1323153496 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32340f5ef0 0x7f32340f6920 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:44:00.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.588+0000 7f323accb700 1 --2- 172.21.15.27:0/1323153496 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32340f55e0 0x7f32340f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:44:00.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.588+0000 7f323accb700 1 -- 172.21.15.27:0/1323153496 >> 172.21.15.27:0/1323153496 conn(0x7f32340f0de0 msgr2=0x7f32340f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:44:00.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.588+0000 7f323accb700 1 -- 172.21.15.27:0/1323153496 shutdown_connections 2021-11-03T18:44:00.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.588+0000 7f323accb700 1 -- 172.21.15.27:0/1323153496 wait complete. 2021-11-03T18:44:00.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.589+0000 7f323accb700 1 Processor -- start 2021-11-03T18:44:00.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.589+0000 7f323accb700 1 -- start start 2021-11-03T18:44:00.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.589+0000 7f323accb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32340f55e0 0x7f3234105e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:44:00.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.589+0000 7f323accb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32340f5ef0 0x7f3234106380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:44:00.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.589+0000 7f323accb700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3234106930 con 0x7f32340f55e0 2021-11-03T18:44:00.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.589+0000 7f323accb700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3234106a70 con 0x7f32340f5ef0 2021-11-03T18:44:00.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.589+0000 7f3238a67700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32340f55e0 0x7f3234105e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:44:00.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.589+0000 7f3238a67700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32340f55e0 0x7f3234105e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36548/0 (socket says 172.21.15.27:36548) 2021-11-03T18:44:00.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.589+0000 7f3238a67700 1 -- 172.21.15.27:0/3024733568 learned_addr learned my addr 172.21.15.27:0/3024733568 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:44:00.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.589+0000 7f3233fff700 1 --2- 172.21.15.27:0/3024733568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32340f5ef0 0x7f3234106380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:44:00.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.589+0000 7f3238a67700 1 -- 172.21.15.27:0/3024733568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32340f5ef0 msgr2=0x7f3234106380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:44:00.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.589+0000 7f3238a67700 1 --2- 172.21.15.27:0/3024733568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32340f5ef0 0x7f3234106380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:44:00.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.589+0000 7f3238a67700 1 -- 172.21.15.27:0/3024733568 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3220005040 con 0x7f32340f55e0 2021-11-03T18:44:00.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.589+0000 7f3233fff700 1 --2- 172.21.15.27:0/3024733568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32340f5ef0 0x7f3234106380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:44:00.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.590+0000 7f3238a67700 1 --2- 172.21.15.27:0/3024733568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32340f55e0 0x7f3234105e90 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 rx=0x7f32280026d0 tx=0x7f3228009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:44:00.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.590+0000 7f32317fa700 1 -- 172.21.15.27:0/3024733568 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3228009320 con 0x7f32340f55e0 2021-11-03T18:44:00.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.590+0000 7f323accb700 1 -- 172.21.15.27:0/3024733568 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3234107410 con 0x7f32340f55e0 2021-11-03T18:44:00.595 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.590+0000 7f32317fa700 1 -- 172.21.15.27:0/3024733568 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f322800f040 con 0x7f32340f55e0 2021-11-03T18:44:00.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.590+0000 7f32317fa700 1 -- 172.21.15.27:0/3024733568 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f32280147e0 con 0x7f32340f55e0 2021-11-03T18:44:00.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.590+0000 7f323accb700 1 -- 172.21.15.27:0/3024733568 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32341079d0 con 0x7f32340f55e0 2021-11-03T18:44:00.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.591+0000 7f323accb700 1 -- 172.21.15.27:0/3024733568 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3234045ad0 con 0x7f32340f55e0 2021-11-03T18:44:00.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.591+0000 7f32317fa700 1 -- 172.21.15.27:0/3024733568 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f3228009480 con 0x7f32340f55e0 2021-11-03T18:44:00.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.592+0000 7f32317fa700 1 --2- 172.21.15.27:0/3024733568 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f32240662c0 0x7f3224068780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:44:00.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.592+0000 7f32317fa700 1 -- 172.21.15.27:0/3024733568 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f3228080eb0 con 0x7f32340f55e0 2021-11-03T18:44:00.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.592+0000 7f3233fff700 1 --2- 172.21.15.27:0/3024733568 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f32240662c0 0x7f3224068780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:44:00.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.592+0000 7f3233fff700 1 --2- 172.21.15.27:0/3024733568 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f32240662c0 0x7f3224068780 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7f3220004630 tx=0x7f3220009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:44:00.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.596+0000 7f32317fa700 1 -- 172.21.15.27:0/3024733568 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f3228053210 con 0x7f32340f55e0 2021-11-03T18:44:00.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:00 smithi115 ceph-mon[65453]: pgmap v2682: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:00.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5282.487081 secs 2021-11-03T18:44:00.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:00 smithi115 ceph-mon[65453]: from='client.36856 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:44:00.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:00 smithi027 ceph-mon[97244]: pgmap v2682: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:00.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5282.487081 secs 2021-11-03T18:44:00.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:00 smithi027 ceph-mon[97244]: from='client.36856 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:44:00.804 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.804+0000 7f323accb700 1 -- 172.21.15.27:0/3024733568 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f32340ff7e0 con 0x7f32340f55e0 2021-11-03T18:44:00.804 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.805+0000 7f32317fa700 1 -- 172.21.15.27:0/3024733568 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f3228052e00 con 0x7f32340f55e0 2021-11-03T18:44:00.805 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:44:00.805 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:44:00.805 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:44:00.805 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:44:00.806 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:44:00.806 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:44:00.806 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:44:00.807 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:44:00.807 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:44:00.807 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:44:00.807 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:44:00.808 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:44:00.808 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:44:00.808 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:44:00.809 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:44:00.809 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:44:00.809 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:44:00.810 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:44:00.810 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:44:00.810 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:44:00.811 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:44:00.811 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:44:00.812 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:44:00.812 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:44:00.813 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:44:00.813 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:44:00.813 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:44:00.814 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:44:00.814 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:44:00.815 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:44:00.815 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:44:00.816 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:44:00.816 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:44:00.816 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:44:00.817 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:44:00.817 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:44:00.818 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:44:00.818 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:44:00.819 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:44:00.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.807+0000 7f323accb700 1 -- 172.21.15.27:0/3024733568 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f32240662c0 msgr2=0x7f3224068780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:44:00.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.807+0000 7f323accb700 1 --2- 172.21.15.27:0/3024733568 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f32240662c0 0x7f3224068780 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7f3220004630 tx=0x7f3220009b30).stop 2021-11-03T18:44:00.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.807+0000 7f323accb700 1 -- 172.21.15.27:0/3024733568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32340f55e0 msgr2=0x7f3234105e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:44:00.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.807+0000 7f323accb700 1 --2- 172.21.15.27:0/3024733568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32340f55e0 0x7f3234105e90 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 rx=0x7f32280026d0 tx=0x7f3228009de0).stop 2021-11-03T18:44:00.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.807+0000 7f323accb700 1 -- 172.21.15.27:0/3024733568 shutdown_connections 2021-11-03T18:44:00.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.807+0000 7f323accb700 1 --2- 172.21.15.27:0/3024733568 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f32240662c0 0x7f3224068780 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:44:00.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.807+0000 7f323accb700 1 --2- 172.21.15.27:0/3024733568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f32340f55e0 0x7f3234105e90 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:44:00.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.807+0000 7f323accb700 1 --2- 172.21.15.27:0/3024733568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f32340f5ef0 0x7f3234106380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:44:00.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.807+0000 7f323accb700 1 -- 172.21.15.27:0/3024733568 >> 172.21.15.27:0/3024733568 conn(0x7f32340f0de0 msgr2=0x7f32340fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:44:00.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.807+0000 7f323accb700 1 -- 172.21.15.27:0/3024733568 shutdown_connections 2021-11-03T18:44:00.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:00.807+0000 7f323accb700 1 -- 172.21.15.27:0/3024733568 wait complete. 2021-11-03T18:44:00.822 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:44:01.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:01 smithi115 ceph-mon[65453]: from='client.36860 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:44:01.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:01 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/493361023' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:01.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:01 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3024733568' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:44:01.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:01 smithi027 ceph-mon[97244]: from='client.36860 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:44:01.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:01 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/493361023' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:01.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:01 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3024733568' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:44:02.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:02 smithi027 ceph-mon[97244]: pgmap v2683: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:02.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:02 smithi115 ceph-mon[65453]: pgmap v2683: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:04.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:04 smithi115 ceph-mon[65453]: pgmap v2684: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:05.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:04 smithi027 ceph-mon[97244]: pgmap v2684: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:05.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5287.487179 secs 2021-11-03T18:44:06.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5287.487179 secs 2021-11-03T18:44:06.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:06 smithi115 ceph-mon[65453]: pgmap v2685: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:07.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:06 smithi027 ceph-mon[97244]: pgmap v2685: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:07.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:44:07.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:44:07.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:44:07.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:07.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:07.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:07.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:08.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:44:08.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:44:08.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:44:08.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:08.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:08.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:08.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:08.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:08 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:44:08.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:08 smithi115 ceph-mon[65453]: pgmap v2686: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:09.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:08 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:44:09.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:08 smithi027 ceph-mon[97244]: pgmap v2686: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:10.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:10 smithi115 ceph-mon[65453]: pgmap v2687: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:10.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5292.487308 secs 2021-11-03T18:44:11.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:10 smithi027 ceph-mon[97244]: pgmap v2687: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:11.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5292.487308 secs 2021-11-03T18:44:12.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:12 smithi027 ceph-mon[97244]: pgmap v2688: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:12.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:12 smithi115 ceph-mon[65453]: pgmap v2688: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:14.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:14 smithi115 ceph-mon[65453]: pgmap v2689: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:15.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:14 smithi027 ceph-mon[97244]: pgmap v2689: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:15.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5297.487444 secs 2021-11-03T18:44:16.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5297.487444 secs 2021-11-03T18:44:16.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:16 smithi115 ceph-mon[65453]: pgmap v2690: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:17.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:16 smithi027 ceph-mon[97244]: pgmap v2690: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:17.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:44:17.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:44:17.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:44:17.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:17.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:17.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:17.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:18.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:44:18.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:44:18.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:44:18.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:18.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:18.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:18.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:18.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:18 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:44:18.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:18 smithi115 ceph-mon[65453]: pgmap v2691: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:19.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:18 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:44:19.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:18 smithi027 ceph-mon[97244]: pgmap v2691: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:20.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:20 smithi115 ceph-mon[65453]: pgmap v2692: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:20.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5302.487587 secs 2021-11-03T18:44:21.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:20 smithi027 ceph-mon[97244]: pgmap v2692: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:21.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5302.487587 secs 2021-11-03T18:44:22.940 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:22 smithi027 ceph-mon[97244]: pgmap v2693: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:22.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:22 smithi115 ceph-mon[65453]: pgmap v2693: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:24.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:24 smithi115 ceph-mon[65453]: pgmap v2694: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:25.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:24 smithi027 ceph-mon[97244]: pgmap v2694: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:25.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5307.487668 secs 2021-11-03T18:44:26.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5307.487668 secs 2021-11-03T18:44:26.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:26 smithi115 ceph-mon[65453]: pgmap v2695: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:27.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:26 smithi027 ceph-mon[97244]: pgmap v2695: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:27.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:44:27.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:44:27.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:44:27.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:27.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:27.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:27.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:28.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:44:28.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:44:28.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:44:28.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:28.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:28.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:28.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:28.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:28 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:44:28.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:28 smithi115 ceph-mon[65453]: pgmap v2696: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:29.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:28 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:44:29.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:28 smithi027 ceph-mon[97244]: pgmap v2696: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:30.923 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:30.922+0000 7f883e791700 1 -- 172.21.15.27:0/1430184392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88380f5ef0 msgr2=0x7f88380f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:44:30.923 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:30.922+0000 7f883e791700 1 --2- 172.21.15.27:0/1430184392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88380f5ef0 0x7f88380f6920 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 rx=0x7f882c004660 tx=0x7f882c009b30).stop 2021-11-03T18:44:30.923 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:30.922+0000 7f883e791700 1 -- 172.21.15.27:0/1430184392 shutdown_connections 2021-11-03T18:44:30.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:30.922+0000 7f883e791700 1 --2- 172.21.15.27:0/1430184392 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88380f5ef0 0x7f88380f6920 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:44:30.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:30.922+0000 7f883e791700 1 --2- 172.21.15.27:0/1430184392 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88380f55e0 0x7f88380f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:44:30.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:30.922+0000 7f883e791700 1 -- 172.21.15.27:0/1430184392 >> 172.21.15.27:0/1430184392 conn(0x7f88380f0de0 msgr2=0x7f88380f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:44:30.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:30.922+0000 7f883e791700 1 -- 172.21.15.27:0/1430184392 shutdown_connections 2021-11-03T18:44:30.925 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:30.922+0000 7f883e791700 1 -- 172.21.15.27:0/1430184392 wait complete. 2021-11-03T18:44:30.925 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:30.923+0000 7f883e791700 1 Processor -- start 2021-11-03T18:44:30.925 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:30.923+0000 7f883e791700 1 -- start start 2021-11-03T18:44:30.925 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:30.923+0000 7f883e791700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88380f55e0 0x7f8838109150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:44:30.925 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:30.923+0000 7f883e791700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88380f5ef0 0x7f8838109640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:44:30.926 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:30.923+0000 7f883e791700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8838109bf0 con 0x7f88380f5ef0 2021-11-03T18:44:30.926 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:30.923+0000 7f883e791700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8838109d30 con 0x7f88380f55e0 2021-11-03T18:44:30.926 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:30.924+0000 7f88377fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88380f5ef0 0x7f8838109640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:44:30.926 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:30.924+0000 7f88377fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88380f5ef0 0x7f8838109640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36568/0 (socket says 172.21.15.27:36568) 2021-11-03T18:44:30.927 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:30.924+0000 7f88377fe700 1 -- 172.21.15.27:0/2251962454 learned_addr learned my addr 172.21.15.27:0/2251962454 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:44:30.927 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:30.924+0000 7f8837fff700 1 --2- 172.21.15.27:0/2251962454 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88380f55e0 0x7f8838109150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:44:30.927 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:30.924+0000 7f88377fe700 1 -- 172.21.15.27:0/2251962454 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88380f55e0 msgr2=0x7f8838109150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:44:30.927 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:30.924+0000 7f88377fe700 1 --2- 172.21.15.27:0/2251962454 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88380f55e0 0x7f8838109150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:44:30.927 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:30.924+0000 7f88377fe700 1 -- 172.21.15.27:0/2251962454 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f882c005040 con 0x7f88380f5ef0 2021-11-03T18:44:30.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:30.924+0000 7f88377fe700 1 --2- 172.21.15.27:0/2251962454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88380f5ef0 0x7f8838109640 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 rx=0x7f882c009990 tx=0x7f882c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:44:30.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:30.924+0000 7f8834ff9700 1 -- 172.21.15.27:0/2251962454 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f882c018070 con 0x7f88380f5ef0 2021-11-03T18:44:30.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:30.924+0000 7f883e791700 1 -- 172.21.15.27:0/2251962454 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f883810a670 con 0x7f88380f5ef0 2021-11-03T18:44:30.928 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:30.924+0000 7f8834ff9700 1 -- 172.21.15.27:0/2251962454 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f882c007ba0 con 0x7f88380f5ef0 2021-11-03T18:44:30.929 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:30.924+0000 7f8834ff9700 1 -- 172.21.15.27:0/2251962454 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f882c01c620 con 0x7f88380f5ef0 2021-11-03T18:44:30.929 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:30.924+0000 7f883e791700 1 -- 172.21.15.27:0/2251962454 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f883810abd0 con 0x7f88380f5ef0 2021-11-03T18:44:30.930 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:30.925+0000 7f883e791700 1 -- 172.21.15.27:0/2251962454 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8838045ad0 con 0x7f88380f5ef0 2021-11-03T18:44:30.930 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:30.926+0000 7f8834ff9700 1 -- 172.21.15.27:0/2251962454 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f882c01c8e0 con 0x7f88380f5ef0 2021-11-03T18:44:30.930 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:30.926+0000 7f8834ff9700 1 --2- 172.21.15.27:0/2251962454 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f88200640b0 0x7f8820066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:44:30.931 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:30.926+0000 7f8834ff9700 1 -- 172.21.15.27:0/2251962454 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f882c0826c0 con 0x7f88380f5ef0 2021-11-03T18:44:30.931 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:30.926+0000 7f8837fff700 1 --2- 172.21.15.27:0/2251962454 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f88200640b0 0x7f8820066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:44:30.931 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:30.927+0000 7f8837fff700 1 --2- 172.21.15.27:0/2251962454 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f88200640b0 0x7f8820066570 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7f8828007fd0 tx=0x7f8828009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:44:30.931 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:30.931+0000 7f8834ff9700 1 -- 172.21.15.27:0/2251962454 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f882c054a20 con 0x7f88380f5ef0 2021-11-03T18:44:30.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:30 smithi115 ceph-mon[65453]: pgmap v2697: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:30.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5312.487809 secs 2021-11-03T18:44:31.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:30 smithi027 ceph-mon[97244]: pgmap v2697: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:31.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5312.487809 secs 2021-11-03T18:44:31.106 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.108+0000 7f883e791700 1 -- 172.21.15.27:0/2251962454 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f883810b3c0 con 0x7f88200640b0 2021-11-03T18:44:31.107 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.110+0000 7f8834ff9700 1 -- 172.21.15.27:0/2251962454 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f882800f710 con 0x7f88200640b0 2021-11-03T18:44:31.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.111+0000 7f883e791700 1 -- 172.21.15.27:0/2251962454 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f88200640b0 msgr2=0x7f8820066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:44:31.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.111+0000 7f883e791700 1 --2- 172.21.15.27:0/2251962454 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f88200640b0 0x7f8820066570 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7f8828007fd0 tx=0x7f8828009040).stop 2021-11-03T18:44:31.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.111+0000 7f883e791700 1 -- 172.21.15.27:0/2251962454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88380f5ef0 msgr2=0x7f8838109640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:44:31.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.111+0000 7f883e791700 1 --2- 172.21.15.27:0/2251962454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88380f5ef0 0x7f8838109640 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 rx=0x7f882c009990 tx=0x7f882c009b30).stop 2021-11-03T18:44:31.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.111+0000 7f883e791700 1 -- 172.21.15.27:0/2251962454 shutdown_connections 2021-11-03T18:44:31.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.111+0000 7f883e791700 1 --2- 172.21.15.27:0/2251962454 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f88200640b0 0x7f8820066570 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:44:31.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.111+0000 7f883e791700 1 --2- 172.21.15.27:0/2251962454 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f88380f55e0 0x7f8838109150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:44:31.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.111+0000 7f883e791700 1 --2- 172.21.15.27:0/2251962454 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f88380f5ef0 0x7f8838109640 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:44:31.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.111+0000 7f883e791700 1 -- 172.21.15.27:0/2251962454 >> 172.21.15.27:0/2251962454 conn(0x7f88380f0de0 msgr2=0x7f883805ff60 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:44:31.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.111+0000 7f883e791700 1 -- 172.21.15.27:0/2251962454 shutdown_connections 2021-11-03T18:44:31.112 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.111+0000 7f883e791700 1 -- 172.21.15.27:0/2251962454 wait complete. 2021-11-03T18:44:31.119 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:44:31.210 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.212+0000 7feff66ef700 1 -- 172.21.15.27:0/3823804828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feff00f6ff0 msgr2=0x7feff00f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:44:31.211 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.212+0000 7feff66ef700 1 --2- 172.21.15.27:0/3823804828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feff00f6ff0 0x7feff00f7a20 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 rx=0x7fefe4004660 tx=0x7fefe4009b30).stop 2021-11-03T18:44:31.211 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.212+0000 7feff66ef700 1 -- 172.21.15.27:0/3823804828 shutdown_connections 2021-11-03T18:44:31.211 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.212+0000 7feff66ef700 1 --2- 172.21.15.27:0/3823804828 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feff00f6ff0 0x7feff00f7a20 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:44:31.211 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.212+0000 7feff66ef700 1 --2- 172.21.15.27:0/3823804828 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feff00f66e0 0x7feff00f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:44:31.212 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.212+0000 7feff66ef700 1 -- 172.21.15.27:0/3823804828 >> 172.21.15.27:0/3823804828 conn(0x7feff00f1f00 msgr2=0x7feff00f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:44:31.212 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.212+0000 7feff66ef700 1 -- 172.21.15.27:0/3823804828 shutdown_connections 2021-11-03T18:44:31.212 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.212+0000 7feff66ef700 1 -- 172.21.15.27:0/3823804828 wait complete. 2021-11-03T18:44:31.212 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.212+0000 7feff66ef700 1 Processor -- start 2021-11-03T18:44:31.215 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.213+0000 7feff66ef700 1 -- start start 2021-11-03T18:44:31.215 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.213+0000 7feff66ef700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feff00f66e0 0x7feff0106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:44:31.216 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.213+0000 7feff66ef700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feff00f6ff0 0x7feff0107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:44:31.216 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.213+0000 7feff66ef700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff0107a10 con 0x7feff00f6ff0 2021-11-03T18:44:31.216 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.213+0000 7feff66ef700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feff0107b50 con 0x7feff00f66e0 2021-11-03T18:44:31.216 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.213+0000 7fefef7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feff00f6ff0 0x7feff0107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:44:31.217 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.213+0000 7fefef7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feff00f6ff0 0x7feff0107460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36578/0 (socket says 172.21.15.27:36578) 2021-11-03T18:44:31.217 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.213+0000 7fefef7fe700 1 -- 172.21.15.27:0/106176582 learned_addr learned my addr 172.21.15.27:0/106176582 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:44:31.217 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.213+0000 7fefeffff700 1 --2- 172.21.15.27:0/106176582 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feff00f66e0 0x7feff0106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:44:31.217 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.213+0000 7fefef7fe700 1 -- 172.21.15.27:0/106176582 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feff00f66e0 msgr2=0x7feff0106f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:44:31.218 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.213+0000 7fefef7fe700 1 --2- 172.21.15.27:0/106176582 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feff00f66e0 0x7feff0106f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:44:31.218 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.213+0000 7fefef7fe700 1 -- 172.21.15.27:0/106176582 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe4005040 con 0x7feff00f6ff0 2021-11-03T18:44:31.218 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.213+0000 7fefef7fe700 1 --2- 172.21.15.27:0/106176582 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feff00f6ff0 0x7feff0107460 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 rx=0x7fefe4009990 tx=0x7fefe4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:44:31.218 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.214+0000 7fefecff9700 1 -- 172.21.15.27:0/106176582 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefe4018070 con 0x7feff00f6ff0 2021-11-03T18:44:31.219 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.214+0000 7fefecff9700 1 -- 172.21.15.27:0/106176582 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fefe4007ca0 con 0x7feff00f6ff0 2021-11-03T18:44:31.219 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.214+0000 7feff66ef700 1 -- 172.21.15.27:0/106176582 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feff0108490 con 0x7feff00f6ff0 2021-11-03T18:44:31.219 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.214+0000 7fefecff9700 1 -- 172.21.15.27:0/106176582 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fefe401c620 con 0x7feff00f6ff0 2021-11-03T18:44:31.219 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.214+0000 7feff66ef700 1 -- 172.21.15.27:0/106176582 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feff01089f0 con 0x7feff00f6ff0 2021-11-03T18:44:31.219 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.215+0000 7feff66ef700 1 -- 172.21.15.27:0/106176582 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feff0045ad0 con 0x7feff00f6ff0 2021-11-03T18:44:31.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.215+0000 7fefecff9700 1 -- 172.21.15.27:0/106176582 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fefe401d960 con 0x7feff00f6ff0 2021-11-03T18:44:31.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.215+0000 7fefecff9700 1 --2- 172.21.15.27:0/106176582 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fefd80640b0 0x7fefd8066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:44:31.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.215+0000 7fefecff9700 1 -- 172.21.15.27:0/106176582 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fefe4082540 con 0x7feff00f6ff0 2021-11-03T18:44:31.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.216+0000 7fefeffff700 1 --2- 172.21.15.27:0/106176582 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fefd80640b0 0x7fefd8066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:44:31.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.216+0000 7fefeffff700 1 --2- 172.21.15.27:0/106176582 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fefd80640b0 0x7fefd8066570 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7fefe00041d0 tx=0x7fefe0009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:44:31.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.219+0000 7fefecff9700 1 -- 172.21.15.27:0/106176582 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fefe4054960 con 0x7feff00f6ff0 2021-11-03T18:44:31.391 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.393+0000 7feff66ef700 1 -- 172.21.15.27:0/106176582 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7feff00ff040 con 0x7fefd80640b0 2021-11-03T18:44:31.405 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.407+0000 7fefecff9700 1 -- 172.21.15.27:0/106176582 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7feff00ff040 con 0x7fefd80640b0 2021-11-03T18:44:31.406 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:44:31.406 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (96m) 4m ago 97m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:44:31.406 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (87m) 4m ago 97m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:44:31.406 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (87m) 4m ago 96m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:44:31.407 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (95m) 4m ago 97m 42.9M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:44:31.407 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (89m) 4m ago 92m 26.9M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:44:31.407 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (92m) 4m ago 92m 3852M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:44:31.407 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (88m) 4m ago 92m 26.5M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:44:31.408 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (89m) 4m ago 92m 27.6M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:44:31.408 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (89m) 4m ago 98m 625M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:44:31.408 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (89m) 4m ago 96m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:44:31.408 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (88m) 4m ago 98m 387M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:44:31.408 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (88m) 4m ago 96m 175M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:44:31.409 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (96m) 4m ago 96m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:44:31.409 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (96m) 4m ago 96m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:44:31.409 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (87m) 4m ago 95m 433M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:44:31.409 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (86m) 4m ago 95m 471M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:44:31.410 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (86m) 4m ago 95m 356M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:44:31.410 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (85m) 4m ago 94m 504M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:44:31.410 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (85m) 4m ago 94m 396M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:44:31.410 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (85m) 4m ago 94m 408M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:44:31.411 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (95m) 4m ago 96m 66.8M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:44:31.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.409+0000 7feff66ef700 1 -- 172.21.15.27:0/106176582 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fefd80640b0 msgr2=0x7fefd8066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:44:31.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.409+0000 7feff66ef700 1 --2- 172.21.15.27:0/106176582 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fefd80640b0 0x7fefd8066570 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7fefe00041d0 tx=0x7fefe0009040).stop 2021-11-03T18:44:31.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.409+0000 7feff66ef700 1 -- 172.21.15.27:0/106176582 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feff00f6ff0 msgr2=0x7feff0107460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:44:31.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.409+0000 7feff66ef700 1 --2- 172.21.15.27:0/106176582 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feff00f6ff0 0x7feff0107460 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 rx=0x7fefe4009990 tx=0x7fefe4009b30).stop 2021-11-03T18:44:31.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.409+0000 7feff66ef700 1 -- 172.21.15.27:0/106176582 shutdown_connections 2021-11-03T18:44:31.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.409+0000 7feff66ef700 1 --2- 172.21.15.27:0/106176582 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fefd80640b0 0x7fefd8066570 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:44:31.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.409+0000 7feff66ef700 1 --2- 172.21.15.27:0/106176582 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7feff00f66e0 0x7feff0106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:44:31.413 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.409+0000 7feff66ef700 1 --2- 172.21.15.27:0/106176582 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7feff00f6ff0 0x7feff0107460 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:44:31.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.409+0000 7feff66ef700 1 -- 172.21.15.27:0/106176582 >> 172.21.15.27:0/106176582 conn(0x7feff00f1f00 msgr2=0x7feff00fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:44:31.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.409+0000 7feff66ef700 1 -- 172.21.15.27:0/106176582 shutdown_connections 2021-11-03T18:44:31.414 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.409+0000 7feff66ef700 1 -- 172.21.15.27:0/106176582 wait complete. 2021-11-03T18:44:31.507 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.509+0000 7f07e0ea0700 1 -- 172.21.15.27:0/1714349271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07dc0f6530 msgr2=0x7f07dc0f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:44:31.507 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.509+0000 7f07e0ea0700 1 --2- 172.21.15.27:0/1714349271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07dc0f6530 0x7f07dc0f6950 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 rx=0x7f07d40099e0 tx=0x7f07d4004d10).stop 2021-11-03T18:44:31.508 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.509+0000 7f07e0ea0700 1 -- 172.21.15.27:0/1714349271 shutdown_connections 2021-11-03T18:44:31.508 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.509+0000 7f07e0ea0700 1 --2- 172.21.15.27:0/1714349271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07dc0f77d0 0x7f07dc0f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:44:31.508 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.509+0000 7f07e0ea0700 1 --2- 172.21.15.27:0/1714349271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07dc0f6530 0x7f07dc0f6950 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:44:31.508 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.509+0000 7f07e0ea0700 1 -- 172.21.15.27:0/1714349271 >> 172.21.15.27:0/1714349271 conn(0x7f07dc0f1cd0 msgr2=0x7f07dc0f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:44:31.509 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.509+0000 7f07e0ea0700 1 -- 172.21.15.27:0/1714349271 shutdown_connections 2021-11-03T18:44:31.509 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.509+0000 7f07e0ea0700 1 -- 172.21.15.27:0/1714349271 wait complete. 2021-11-03T18:44:31.509 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.509+0000 7f07e0ea0700 1 Processor -- start 2021-11-03T18:44:31.511 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.509+0000 7f07e0ea0700 1 -- start start 2021-11-03T18:44:31.512 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.510+0000 7f07e0ea0700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07dc0f6530 0x7f07dc06b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:44:31.512 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.510+0000 7f07e0ea0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07dc0f77d0 0x7f07dc069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:44:31.512 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.510+0000 7f07e0ea0700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc06a380 con 0x7f07dc0f77d0 2021-11-03T18:44:31.512 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.510+0000 7f07e0ea0700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07dc06a4c0 con 0x7f07dc0f6530 2021-11-03T18:44:31.513 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.510+0000 7f07daffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07dc0f77d0 0x7f07dc069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:44:31.513 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.510+0000 7f07daffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07dc0f77d0 0x7f07dc069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36588/0 (socket says 172.21.15.27:36588) 2021-11-03T18:44:31.513 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.510+0000 7f07daffd700 1 -- 172.21.15.27:0/3460658965 learned_addr learned my addr 172.21.15.27:0/3460658965 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:44:31.514 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.510+0000 7f07db7fe700 1 --2- 172.21.15.27:0/3460658965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07dc0f6530 0x7f07dc06b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:44:31.514 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.510+0000 7f07daffd700 1 -- 172.21.15.27:0/3460658965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07dc0f6530 msgr2=0x7f07dc06b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:44:31.514 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.510+0000 7f07daffd700 1 --2- 172.21.15.27:0/3460658965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07dc0f6530 0x7f07dc06b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:44:31.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.510+0000 7f07daffd700 1 -- 172.21.15.27:0/3460658965 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d4009640 con 0x7f07dc0f77d0 2021-11-03T18:44:31.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.510+0000 7f07daffd700 1 --2- 172.21.15.27:0/3460658965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07dc0f77d0 0x7f07dc069e20 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 rx=0x7f07c400c790 tx=0x7f07c4007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:44:31.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.510+0000 7f07d3fff700 1 -- 172.21.15.27:0/3460658965 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07c4002590 con 0x7f07dc0f77d0 2021-11-03T18:44:31.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.510+0000 7f07e0ea0700 1 -- 172.21.15.27:0/3460658965 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07dc06a7a0 con 0x7f07dc0f77d0 2021-11-03T18:44:31.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.510+0000 7f07d3fff700 1 -- 172.21.15.27:0/3460658965 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f07c4002ba0 con 0x7f07dc0f77d0 2021-11-03T18:44:31.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.510+0000 7f07d3fff700 1 -- 172.21.15.27:0/3460658965 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f07c4013d90 con 0x7f07dc0f77d0 2021-11-03T18:44:31.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.510+0000 7f07e0ea0700 1 -- 172.21.15.27:0/3460658965 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07dc10eea0 con 0x7f07dc0f77d0 2021-11-03T18:44:31.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.511+0000 7f07e0ea0700 1 -- 172.21.15.27:0/3460658965 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07dc045ad0 con 0x7f07dc0f77d0 2021-11-03T18:44:31.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.512+0000 7f07d3fff700 1 -- 172.21.15.27:0/3460658965 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f07c401a060 con 0x7f07dc0f77d0 2021-11-03T18:44:31.517 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.512+0000 7f07d3fff700 1 --2- 172.21.15.27:0/3460658965 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f07c8064100 0x7f07c80665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:44:31.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.512+0000 7f07d3fff700 1 -- 172.21.15.27:0/3460658965 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f07c4080b40 con 0x7f07dc0f77d0 2021-11-03T18:44:31.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.512+0000 7f07db7fe700 1 --2- 172.21.15.27:0/3460658965 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f07c8064100 0x7f07c80665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:44:31.518 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.513+0000 7f07db7fe700 1 --2- 172.21.15.27:0/3460658965 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f07c8064100 0x7f07c80665c0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7f07d40099b0 tx=0x7f07d4004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:44:31.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.516+0000 7f07d3fff700 1 -- 172.21.15.27:0/3460658965 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f07c4052db0 con 0x7f07dc0f77d0 2021-11-03T18:44:31.745 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.747+0000 7f07e0ea0700 1 -- 172.21.15.27:0/3460658965 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f07dc0fecf0 con 0x7f07dc0f77d0 2021-11-03T18:44:31.746 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.747+0000 7f07d3fff700 1 -- 172.21.15.27:0/3460658965 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f07c40182e0 con 0x7f07dc0f77d0 2021-11-03T18:44:31.746 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:44:31.746 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:44:31.747 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:44:31.747 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:44:31.747 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:44:31.747 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:44:31.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:44:31.748 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:44:31.748 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:44:31.748 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:44:31.748 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:44:31.749 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:44:31.749 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:44:31.749 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:44:31.749 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:44:31.749 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:44:31.750 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:44:31.750 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:44:31.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.749+0000 7f07e0ea0700 1 -- 172.21.15.27:0/3460658965 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f07c8064100 msgr2=0x7f07c80665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:44:31.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.749+0000 7f07e0ea0700 1 --2- 172.21.15.27:0/3460658965 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f07c8064100 0x7f07c80665c0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7f07d40099b0 tx=0x7f07d4004d10).stop 2021-11-03T18:44:31.751 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.749+0000 7f07e0ea0700 1 -- 172.21.15.27:0/3460658965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07dc0f77d0 msgr2=0x7f07dc069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:44:31.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.749+0000 7f07e0ea0700 1 --2- 172.21.15.27:0/3460658965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07dc0f77d0 0x7f07dc069e20 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 rx=0x7f07c400c790 tx=0x7f07c4007d70).stop 2021-11-03T18:44:31.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.749+0000 7f07e0ea0700 1 -- 172.21.15.27:0/3460658965 shutdown_connections 2021-11-03T18:44:31.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.749+0000 7f07e0ea0700 1 --2- 172.21.15.27:0/3460658965 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f07c8064100 0x7f07c80665c0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:44:31.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.749+0000 7f07e0ea0700 1 --2- 172.21.15.27:0/3460658965 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f07dc0f6530 0x7f07dc06b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:44:31.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.749+0000 7f07e0ea0700 1 --2- 172.21.15.27:0/3460658965 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f07dc0f77d0 0x7f07dc069e20 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:44:31.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.749+0000 7f07e0ea0700 1 -- 172.21.15.27:0/3460658965 >> 172.21.15.27:0/3460658965 conn(0x7f07dc0f1cd0 msgr2=0x7f07dc0fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:44:31.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.749+0000 7f07e0ea0700 1 -- 172.21.15.27:0/3460658965 shutdown_connections 2021-11-03T18:44:31.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.749+0000 7f07e0ea0700 1 -- 172.21.15.27:0/3460658965 wait complete. 2021-11-03T18:44:31.849 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.849+0000 7f8b51937700 1 -- 172.21.15.27:0/4241110446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b4c0f8530 msgr2=0x7f8b4c0f46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:44:31.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.849+0000 7f8b51937700 1 --2- 172.21.15.27:0/4241110446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b4c0f8530 0x7f8b4c0f46b0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 rx=0x7f8b40004660 tx=0x7f8b40009b30).stop 2021-11-03T18:44:31.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.849+0000 7f8b51937700 1 -- 172.21.15.27:0/4241110446 shutdown_connections 2021-11-03T18:44:31.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.849+0000 7f8b51937700 1 --2- 172.21.15.27:0/4241110446 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b4c0f8530 0x7f8b4c0f46b0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:44:31.850 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.849+0000 7f8b51937700 1 --2- 172.21.15.27:0/4241110446 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b4c0f7b90 0x7f8b4c0f7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:44:31.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.849+0000 7f8b51937700 1 -- 172.21.15.27:0/4241110446 >> 172.21.15.27:0/4241110446 conn(0x7f8b4c0f0330 msgr2=0x7f8b4c0f2750 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:44:31.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.849+0000 7f8b51937700 1 -- 172.21.15.27:0/4241110446 shutdown_connections 2021-11-03T18:44:31.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.849+0000 7f8b51937700 1 -- 172.21.15.27:0/4241110446 wait complete. 2021-11-03T18:44:31.851 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.850+0000 7f8b51937700 1 Processor -- start 2021-11-03T18:44:31.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.850+0000 7f8b51937700 1 -- start start 2021-11-03T18:44:31.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.850+0000 7f8b51937700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b4c0f7b90 0x7f8b4c0f7460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:44:31.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.850+0000 7f8b51937700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b4c0f8530 0x7f8b4c0f58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:44:31.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.850+0000 7f8b51937700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c0f5e40 con 0x7f8b4c0f8530 2021-11-03T18:44:31.852 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.850+0000 7f8b51937700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b4c0f5f80 con 0x7f8b4c0f7b90 2021-11-03T18:44:31.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.850+0000 7f8b4a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b4c0f8530 0x7f8b4c0f58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:44:31.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.850+0000 7f8b4a7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b4c0f8530 0x7f8b4c0f58e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36598/0 (socket says 172.21.15.27:36598) 2021-11-03T18:44:31.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.850+0000 7f8b4a7fc700 1 -- 172.21.15.27:0/4229916783 learned_addr learned my addr 172.21.15.27:0/4229916783 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:44:31.853 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.850+0000 7f8b4a7fc700 1 -- 172.21.15.27:0/4229916783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b4c0f7b90 msgr2=0x7f8b4c0f7460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:44:31.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.850+0000 7f8b4a7fc700 1 --2- 172.21.15.27:0/4229916783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b4c0f7b90 0x7f8b4c0f7460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:44:31.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.850+0000 7f8b4a7fc700 1 -- 172.21.15.27:0/4229916783 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b40005040 con 0x7f8b4c0f8530 2021-11-03T18:44:31.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.851+0000 7f8b4a7fc700 1 --2- 172.21.15.27:0/4229916783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b4c0f8530 0x7f8b4c0f58e0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 rx=0x7f8b40009da0 tx=0x7f8b40009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:44:31.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.851+0000 7f8b337fe700 1 -- 172.21.15.27:0/4229916783 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b40018070 con 0x7f8b4c0f8530 2021-11-03T18:44:31.854 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.851+0000 7f8b51937700 1 -- 172.21.15.27:0/4229916783 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b4c0f6200 con 0x7f8b4c0f8530 2021-11-03T18:44:31.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.851+0000 7f8b337fe700 1 -- 172.21.15.27:0/4229916783 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f8b40007d70 con 0x7f8b4c0f8530 2021-11-03T18:44:31.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.851+0000 7f8b337fe700 1 -- 172.21.15.27:0/4229916783 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b4001dea0 con 0x7f8b4c0f8530 2021-11-03T18:44:31.855 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.851+0000 7f8b51937700 1 -- 172.21.15.27:0/4229916783 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b4c0f6760 con 0x7f8b4c0f8530 2021-11-03T18:44:31.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.852+0000 7f8b51937700 1 -- 172.21.15.27:0/4229916783 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b4c045ad0 con 0x7f8b4c0f8530 2021-11-03T18:44:31.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.852+0000 7f8b337fe700 1 -- 172.21.15.27:0/4229916783 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f8b4001d420 con 0x7f8b4c0f8530 2021-11-03T18:44:31.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.853+0000 7f8b337fe700 1 --2- 172.21.15.27:0/4229916783 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8b34064100 0x7f8b340665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:44:31.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.853+0000 7f8b337fe700 1 -- 172.21.15.27:0/4229916783 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f8b40082670 con 0x7f8b4c0f8530 2021-11-03T18:44:31.858 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.853+0000 7f8b4affd700 1 --2- 172.21.15.27:0/4229916783 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8b34064100 0x7f8b340665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:44:31.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.853+0000 7f8b4affd700 1 --2- 172.21.15.27:0/4229916783 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8b34064100 0x7f8b340665c0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7f8b4c0f71f0 tx=0x7f8b3c007a60).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:44:31.859 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:31.857+0000 7f8b337fe700 1 -- 172.21.15.27:0/4229916783 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f8b40054a90 con 0x7f8b4c0f8530 2021-11-03T18:44:32.060 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:32.061+0000 7f8b51937700 1 -- 172.21.15.27:0/4229916783 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8b4c027a00 con 0x7f8b4c0f8530 2021-11-03T18:44:32.061 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:32.062+0000 7f8b337fe700 1 -- 172.21.15.27:0/4229916783 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f8b40054680 con 0x7f8b4c0f8530 2021-11-03T18:44:32.061 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:44:32.061 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:44:32.061 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:44:32.062 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:44:32.062 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:44:32.062 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:44:32.062 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:44:32.063 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:44:32.063 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:44:32.063 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:44:32.063 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:44:32.064 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:44:32.064 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:44:32.064 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:44:32.064 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:44:32.064 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:44:32.065 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:44:32.065 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:44:32.065 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:44:32.065 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:44:32.066 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:44:32.066 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:44:32.066 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:44:32.066 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:44:32.067 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:44:32.067 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:44:32.067 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:44:32.067 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:44:32.067 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:44:32.068 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:44:32.068 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:44:32.068 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:44:32.068 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:44:32.069 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:44:32.069 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:44:32.069 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:44:32.069 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:44:32.069 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:44:32.070 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:44:32.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:32.064+0000 7f8b51937700 1 -- 172.21.15.27:0/4229916783 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8b34064100 msgr2=0x7f8b340665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:44:32.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:32.064+0000 7f8b51937700 1 --2- 172.21.15.27:0/4229916783 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8b34064100 0x7f8b340665c0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 rx=0x7f8b4c0f71f0 tx=0x7f8b3c007a60).stop 2021-11-03T18:44:32.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:32.064+0000 7f8b51937700 1 -- 172.21.15.27:0/4229916783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b4c0f8530 msgr2=0x7f8b4c0f58e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:44:32.071 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:32.064+0000 7f8b51937700 1 --2- 172.21.15.27:0/4229916783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b4c0f8530 0x7f8b4c0f58e0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 rx=0x7f8b40009da0 tx=0x7f8b40009b30).stop 2021-11-03T18:44:32.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:32.064+0000 7f8b51937700 1 -- 172.21.15.27:0/4229916783 shutdown_connections 2021-11-03T18:44:32.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:32.064+0000 7f8b51937700 1 --2- 172.21.15.27:0/4229916783 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8b34064100 0x7f8b340665c0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:44:32.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:32.064+0000 7f8b51937700 1 --2- 172.21.15.27:0/4229916783 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8b4c0f7b90 0x7f8b4c0f7460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:44:32.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:32.064+0000 7f8b51937700 1 --2- 172.21.15.27:0/4229916783 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8b4c0f8530 0x7f8b4c0f58e0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:44:32.072 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:32.064+0000 7f8b51937700 1 -- 172.21.15.27:0/4229916783 >> 172.21.15.27:0/4229916783 conn(0x7f8b4c0f0330 msgr2=0x7f8b4c0fb420 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:44:32.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:32.064+0000 7f8b51937700 1 -- 172.21.15.27:0/4229916783 shutdown_connections 2021-11-03T18:44:32.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:44:32.064+0000 7f8b51937700 1 -- 172.21.15.27:0/4229916783 wait complete. 2021-11-03T18:44:32.073 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:44:32.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:32 smithi027 ceph-mon[97244]: from='client.36872 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:44:32.942 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:32 smithi027 ceph-mon[97244]: pgmap v2698: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:32.943 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:32 smithi027 ceph-mon[97244]: from='client.36876 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:44:32.943 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:32 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3460658965' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:32.943 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:32 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/4229916783' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:44:32.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:32 smithi115 ceph-mon[65453]: from='client.36872 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:44:32.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:32 smithi115 ceph-mon[65453]: pgmap v2698: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:32.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:32 smithi115 ceph-mon[65453]: from='client.36876 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:44:32.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:32 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3460658965' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:32.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:32 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/4229916783' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:44:34.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:34 smithi115 ceph-mon[65453]: pgmap v2699: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:35.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:34 smithi027 ceph-mon[97244]: pgmap v2699: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:35.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5317.487934 secs 2021-11-03T18:44:36.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5317.487934 secs 2021-11-03T18:44:36.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:36 smithi115 ceph-mon[65453]: pgmap v2700: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:37.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:36 smithi027 ceph-mon[97244]: pgmap v2700: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:37.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:44:37.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:44:37.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:44:37.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:37.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:37.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:37.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:38.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:44:38.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:44:38.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:44:38.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:38.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:38.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:38.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:38.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:38 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:44:38.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:38 smithi115 ceph-mon[65453]: pgmap v2701: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:39.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:38 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:44:39.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:38 smithi027 ceph-mon[97244]: pgmap v2701: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:40.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:40 smithi115 ceph-mon[65453]: pgmap v2702: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:40.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5322.488063 secs 2021-11-03T18:44:41.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:40 smithi027 ceph-mon[97244]: pgmap v2702: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:41.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5322.488063 secs 2021-11-03T18:44:42.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:42 smithi027 ceph-mon[97244]: pgmap v2703: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:42.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:42 smithi115 ceph-mon[65453]: pgmap v2703: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:44.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:44 smithi115 ceph-mon[65453]: pgmap v2704: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:45.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:44 smithi027 ceph-mon[97244]: pgmap v2704: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:45.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5327.488198 secs 2021-11-03T18:44:46.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5327.488198 secs 2021-11-03T18:44:46.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:46 smithi115 ceph-mon[65453]: pgmap v2705: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:47.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:46 smithi027 ceph-mon[97244]: pgmap v2705: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:47.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:44:47.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:44:47.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:44:47.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:47.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:47.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:47.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:48.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:44:48.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:44:48.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:44:48.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:48.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:48.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:48.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:48.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:48 smithi115 ceph-mon[65453]: poll took 60165.282 ms: 1 events 2021-11-03T18:44:48.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:48 smithi115 ceph-mon[65453]: pgmap v2706: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:48.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:48 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:44:49.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:48 smithi027 ceph-mon[97244]: poll took 60165.282 ms: 1 events 2021-11-03T18:44:49.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:48 smithi027 ceph-mon[97244]: pgmap v2706: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:49.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:48 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:44:50.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:50 smithi115 ceph-mon[65453]: pgmap v2707: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:50.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5332.488344 secs 2021-11-03T18:44:51.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:50 smithi027 ceph-mon[97244]: pgmap v2707: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:51.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5332.488344 secs 2021-11-03T18:44:52.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:52 smithi115 ceph-mon[65453]: pgmap v2708: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:53.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:52 smithi027 ceph-mon[97244]: pgmap v2708: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:54.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:54 smithi115 ceph-mon[65453]: pgmap v2709: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:55.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:54 smithi027 ceph-mon[97244]: pgmap v2709: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:55.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5337.488458 secs 2021-11-03T18:44:56.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5337.488458 secs 2021-11-03T18:44:56.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:56 smithi115 ceph-mon[65453]: pgmap v2710: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:56.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:44:56.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:44:57.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:56 smithi027 ceph-mon[97244]: pgmap v2710: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:57.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:44:57.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:44:57.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:44:57.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:44:57.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:44:57.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:57.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:57.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:57.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:57 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:58.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:44:58.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:44:58.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:44:58.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:58.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:58.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:58.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:57 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:44:58.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:58 smithi115 ceph-mon[65453]: pgmap v2711: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:58.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:44:58 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:44:59.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:58 smithi027 ceph-mon[97244]: pgmap v2711: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:44:59.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:44:58 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:45:00.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:00 smithi115 ceph-mon[65453]: pgmap v2712: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:00.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5342.488579 secs 2021-11-03T18:45:01.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:00 smithi027 ceph-mon[97244]: pgmap v2712: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:01.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5342.488579 secs 2021-11-03T18:45:02.178 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.178+0000 7f43cdec4700 1 -- 172.21.15.27:0/2010626026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43c8060310 msgr2=0x7f43c80fb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:45:02.178 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.178+0000 7f43cdec4700 1 --2- 172.21.15.27:0/2010626026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43c8060310 0x7f43c80fb7c0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 rx=0x7f43b8004660 tx=0x7f43b8009b30).stop 2021-11-03T18:45:02.179 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.178+0000 7f43cdec4700 1 -- 172.21.15.27:0/2010626026 shutdown_connections 2021-11-03T18:45:02.179 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.178+0000 7f43cdec4700 1 --2- 172.21.15.27:0/2010626026 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43c80fbcb0 0x7f43c80fe0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T18:45:02.179 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.178+0000 7f43cdec4700 1 --2- 172.21.15.27:0/2010626026 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43c8060310 0x7f43c80fb7c0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:02.179 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.178+0000 7f43cdec4700 1 -- 172.21.15.27:0/2010626026 >> 172.21.15.27:0/2010626026 conn(0x7f43c80f0de0 msgr2=0x7f43c80f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:45:02.179 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.178+0000 7f43cdec4700 1 -- 172.21.15.27:0/2010626026 shutdown_connections 2021-11-03T18:45:02.180 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.178+0000 7f43cdec4700 1 -- 172.21.15.27:0/2010626026 wait complete. 2021-11-03T18:45:02.180 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.178+0000 7f43cdec4700 1 Processor -- start 2021-11-03T18:45:02.180 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.179+0000 7f43cdec4700 1 -- start start 2021-11-03T18:45:02.180 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.179+0000 7f43cdec4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43c8060310 0x7f43c81091c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:45:02.181 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.179+0000 7f43cdec4700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43c80fbcb0 0x7f43c81096b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:45:02.181 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.179+0000 7f43cdec4700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c8109cf0 con 0x7f43c8060310 2021-11-03T18:45:02.181 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.179+0000 7f43cdec4700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43c810a5a0 con 0x7f43c80fbcb0 2021-11-03T18:45:02.181 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.179+0000 7f43c77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43c8060310 0x7f43c81091c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:45:02.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.179+0000 7f43c77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43c8060310 0x7f43c81091c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36614/0 (socket says 172.21.15.27:36614) 2021-11-03T18:45:02.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.179+0000 7f43c77fe700 1 -- 172.21.15.27:0/2176165337 learned_addr learned my addr 172.21.15.27:0/2176165337 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:45:02.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.179+0000 7f43c6ffd700 1 --2- 172.21.15.27:0/2176165337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43c80fbcb0 0x7f43c81096b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:45:02.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.179+0000 7f43c77fe700 1 -- 172.21.15.27:0/2176165337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43c80fbcb0 msgr2=0x7f43c81096b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:45:02.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.179+0000 7f43c77fe700 1 --2- 172.21.15.27:0/2176165337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43c80fbcb0 0x7f43c81096b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:02.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.179+0000 7f43c77fe700 1 -- 172.21.15.27:0/2176165337 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b8005040 con 0x7f43c8060310 2021-11-03T18:45:02.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.179+0000 7f43c77fe700 1 --2- 172.21.15.27:0/2176165337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43c8060310 0x7f43c81091c0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 rx=0x7f43b8004f60 tx=0x7f43b8009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:45:02.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.180+0000 7f43affff700 1 -- 172.21.15.27:0/2176165337 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43b8018070 con 0x7f43c8060310 2021-11-03T18:45:02.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.180+0000 7f43affff700 1 -- 172.21.15.27:0/2176165337 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f43b8009770 con 0x7f43c8060310 2021-11-03T18:45:02.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.180+0000 7f43cdec4700 1 -- 172.21.15.27:0/2176165337 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43c810a820 con 0x7f43c8060310 2021-11-03T18:45:02.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.180+0000 7f43affff700 1 -- 172.21.15.27:0/2176165337 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43b801c3c0 con 0x7f43c8060310 2021-11-03T18:45:02.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.180+0000 7f43cdec4700 1 -- 172.21.15.27:0/2176165337 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43c810ad80 con 0x7f43c8060310 2021-11-03T18:45:02.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.181+0000 7f43cdec4700 1 -- 172.21.15.27:0/2176165337 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43c80f4a70 con 0x7f43c8060310 2021-11-03T18:45:02.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.182+0000 7f43affff700 1 -- 172.21.15.27:0/2176165337 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f43b801d420 con 0x7f43c8060310 2021-11-03T18:45:02.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.182+0000 7f43affff700 1 --2- 172.21.15.27:0/2176165337 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f43b00640b0 0x7f43b0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:45:02.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.182+0000 7f43affff700 1 -- 172.21.15.27:0/2176165337 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f43b8082b10 con 0x7f43c8060310 2021-11-03T18:45:02.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.182+0000 7f43c6ffd700 1 --2- 172.21.15.27:0/2176165337 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f43b00640b0 0x7f43b0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:45:02.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.183+0000 7f43c6ffd700 1 --2- 172.21.15.27:0/2176165337 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f43b00640b0 0x7f43b0066570 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7f43bc006fd0 tx=0x7f43bc008040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:45:02.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.186+0000 7f43affff700 1 -- 172.21.15.27:0/2176165337 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f43b8054ef0 con 0x7f43c8060310 2021-11-03T18:45:02.365 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.367+0000 7f43cdec4700 1 -- 172.21.15.27:0/2176165337 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f43c81011a0 con 0x7f43b00640b0 2021-11-03T18:45:02.366 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.368+0000 7f43affff700 1 -- 172.21.15.27:0/2176165337 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f43c81011a0 con 0x7f43b00640b0 2021-11-03T18:45:02.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.370+0000 7f43cdec4700 1 -- 172.21.15.27:0/2176165337 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f43b00640b0 msgr2=0x7f43b0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:45:02.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.370+0000 7f43cdec4700 1 --2- 172.21.15.27:0/2176165337 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f43b00640b0 0x7f43b0066570 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7f43bc006fd0 tx=0x7f43bc008040).stop 2021-11-03T18:45:02.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.370+0000 7f43cdec4700 1 -- 172.21.15.27:0/2176165337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43c8060310 msgr2=0x7f43c81091c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:45:02.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.370+0000 7f43cdec4700 1 --2- 172.21.15.27:0/2176165337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43c8060310 0x7f43c81091c0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 rx=0x7f43b8004f60 tx=0x7f43b8009b30).stop 2021-11-03T18:45:02.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.370+0000 7f43cdec4700 1 -- 172.21.15.27:0/2176165337 shutdown_connections 2021-11-03T18:45:02.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.371+0000 7f43cdec4700 1 --2- 172.21.15.27:0/2176165337 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f43b00640b0 0x7f43b0066570 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:02.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.371+0000 7f43cdec4700 1 --2- 172.21.15.27:0/2176165337 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43c8060310 0x7f43c81091c0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:02.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.371+0000 7f43cdec4700 1 --2- 172.21.15.27:0/2176165337 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43c80fbcb0 0x7f43c81096b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:02.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.371+0000 7f43cdec4700 1 -- 172.21.15.27:0/2176165337 >> 172.21.15.27:0/2176165337 conn(0x7f43c80f0de0 msgr2=0x7f43c80f31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:45:02.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.371+0000 7f43cdec4700 1 -- 172.21.15.27:0/2176165337 shutdown_connections 2021-11-03T18:45:02.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.371+0000 7f43cdec4700 1 -- 172.21.15.27:0/2176165337 wait complete. 2021-11-03T18:45:02.378 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:45:02.472 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.471+0000 7f2920979700 1 -- 172.21.15.27:0/2061552667 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f291c0f4800 msgr2=0x7f291c0f4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:45:02.473 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.471+0000 7f2920979700 1 --2- 172.21.15.27:0/2061552667 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f291c0f4800 0x7f291c0f4c20 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 rx=0x7f290c004660 tx=0x7f290c009b30).stop 2021-11-03T18:45:02.473 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.471+0000 7f2920979700 1 -- 172.21.15.27:0/2061552667 shutdown_connections 2021-11-03T18:45:02.473 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.471+0000 7f2920979700 1 --2- 172.21.15.27:0/2061552667 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f291c0f51a0 0x7f291c0f7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:02.474 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.471+0000 7f2920979700 1 --2- 172.21.15.27:0/2061552667 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f291c0f4800 0x7f291c0f4c20 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:02.474 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.471+0000 7f2920979700 1 -- 172.21.15.27:0/2061552667 >> 172.21.15.27:0/2061552667 conn(0x7f291c0f0320 msgr2=0x7f291c0f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:45:02.474 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.472+0000 7f2920979700 1 -- 172.21.15.27:0/2061552667 shutdown_connections 2021-11-03T18:45:02.474 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.472+0000 7f2920979700 1 -- 172.21.15.27:0/2061552667 wait complete. 2021-11-03T18:45:02.475 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.472+0000 7f2920979700 1 Processor -- start 2021-11-03T18:45:02.475 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.472+0000 7f2920979700 1 -- start start 2021-11-03T18:45:02.475 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.472+0000 7f2920979700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f291c0f4800 0x7f291c0f73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:45:02.475 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.472+0000 7f2920979700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f291c0f51a0 0x7f291c0f5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:45:02.476 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.472+0000 7f2920979700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f291c0f5d90 con 0x7f291c0f51a0 2021-11-03T18:45:02.476 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.472+0000 7f2920979700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f291c0f5ed0 con 0x7f291c0f4800 2021-11-03T18:45:02.476 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.473+0000 7f291affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f291c0f51a0 0x7f291c0f5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:45:02.476 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.473+0000 7f291affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f291c0f51a0 0x7f291c0f5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36624/0 (socket says 172.21.15.27:36624) 2021-11-03T18:45:02.477 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.473+0000 7f291affd700 1 -- 172.21.15.27:0/4040120663 learned_addr learned my addr 172.21.15.27:0/4040120663 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:45:02.477 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.473+0000 7f291b7fe700 1 --2- 172.21.15.27:0/4040120663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f291c0f4800 0x7f291c0f73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:45:02.477 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.473+0000 7f291affd700 1 -- 172.21.15.27:0/4040120663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f291c0f4800 msgr2=0x7f291c0f73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:45:02.477 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.473+0000 7f291affd700 1 --2- 172.21.15.27:0/4040120663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f291c0f4800 0x7f291c0f73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:02.477 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.473+0000 7f291affd700 1 -- 172.21.15.27:0/4040120663 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f290c005040 con 0x7f291c0f51a0 2021-11-03T18:45:02.478 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.473+0000 7f291affd700 1 --2- 172.21.15.27:0/4040120663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f291c0f51a0 0x7f291c0f5830 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 rx=0x7f290400e7c0 tx=0x7f290400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:45:02.478 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.473+0000 7f2913fff700 1 -- 172.21.15.27:0/4040120663 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29040092e0 con 0x7f291c0f51a0 2021-11-03T18:45:02.478 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.473+0000 7f2913fff700 1 -- 172.21.15.27:0/4040120663 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f290400f040 con 0x7f291c0f51a0 2021-11-03T18:45:02.478 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.473+0000 7f2920979700 1 -- 172.21.15.27:0/4040120663 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f291c0f61b0 con 0x7f291c0f51a0 2021-11-03T18:45:02.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.473+0000 7f2913fff700 1 -- 172.21.15.27:0/4040120663 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f29040147a0 con 0x7f291c0f51a0 2021-11-03T18:45:02.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.473+0000 7f2920979700 1 -- 172.21.15.27:0/4040120663 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f291c10ed30 con 0x7f291c0f51a0 2021-11-03T18:45:02.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.474+0000 7f2920979700 1 -- 172.21.15.27:0/4040120663 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f291c045ad0 con 0x7f291c0f51a0 2021-11-03T18:45:02.480 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.475+0000 7f2913fff700 1 -- 172.21.15.27:0/4040120663 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f29040077b0 con 0x7f291c0f51a0 2021-11-03T18:45:02.480 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.475+0000 7f2913fff700 1 --2- 172.21.15.27:0/4040120663 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2908064100 0x7f29080665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:45:02.481 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.476+0000 7f2913fff700 1 -- 172.21.15.27:0/4040120663 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f2904081310 con 0x7f291c0f51a0 2021-11-03T18:45:02.481 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.476+0000 7f291b7fe700 1 --2- 172.21.15.27:0/4040120663 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2908064100 0x7f29080665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:45:02.481 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.476+0000 7f291b7fe700 1 --2- 172.21.15.27:0/4040120663 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2908064100 0x7f29080665c0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 rx=0x7f290c000c00 tx=0x7f290c016040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:45:02.481 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.479+0000 7f2913fff700 1 -- 172.21.15.27:0/4040120663 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f2904053580 con 0x7f291c0f51a0 2021-11-03T18:45:02.652 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.654+0000 7f2920979700 1 -- 172.21.15.27:0/4040120663 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f291c0feef0 con 0x7f2908064100 2021-11-03T18:45:02.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.668+0000 7f2913fff700 1 -- 172.21.15.27:0/4040120663 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f290c00f070 con 0x7f2908064100 2021-11-03T18:45:02.667 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:45:02.667 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (96m) 5m ago 97m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:45:02.667 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (88m) 5m ago 97m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:45:02.668 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (88m) 4m ago 97m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:45:02.668 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (96m) 5m ago 97m 42.9M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:45:02.668 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (90m) 5m ago 92m 26.9M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:45:02.668 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (93m) 5m ago 93m 3852M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:45:02.669 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (88m) 4m ago 93m 26.5M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:45:02.669 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (90m) 4m ago 93m 27.6M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:45:02.669 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (90m) 5m ago 99m 625M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:45:02.669 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (89m) 4m ago 97m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:45:02.670 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (89m) 5m ago 99m 387M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:45:02.670 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (88m) 4m ago 96m 175M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:45:02.670 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (97m) 5m ago 97m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:45:02.670 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (96m) 4m ago 96m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:45:02.670 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (87m) 5m ago 96m 433M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:45:02.671 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (87m) 5m ago 95m 471M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:45:02.671 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (86m) 5m ago 95m 356M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:45:02.671 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (86m) 4m ago 95m 504M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:45:02.671 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (85m) 4m ago 94m 396M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:45:02.672 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (85m) 4m ago 94m 408M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:45:02.672 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (96m) 5m ago 97m 66.8M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:45:02.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.670+0000 7f2920979700 1 -- 172.21.15.27:0/4040120663 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2908064100 msgr2=0x7f29080665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:45:02.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.670+0000 7f2920979700 1 --2- 172.21.15.27:0/4040120663 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2908064100 0x7f29080665c0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 rx=0x7f290c000c00 tx=0x7f290c016040).stop 2021-11-03T18:45:02.673 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.670+0000 7f2920979700 1 -- 172.21.15.27:0/4040120663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f291c0f51a0 msgr2=0x7f291c0f5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:45:02.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.670+0000 7f2920979700 1 --2- 172.21.15.27:0/4040120663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f291c0f51a0 0x7f291c0f5830 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 rx=0x7f290400e7c0 tx=0x7f290400b040).stop 2021-11-03T18:45:02.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.670+0000 7f2920979700 1 -- 172.21.15.27:0/4040120663 shutdown_connections 2021-11-03T18:45:02.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.670+0000 7f2920979700 1 --2- 172.21.15.27:0/4040120663 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f2908064100 0x7f29080665c0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:02.674 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.670+0000 7f2920979700 1 --2- 172.21.15.27:0/4040120663 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f291c0f4800 0x7f291c0f73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:02.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.670+0000 7f2920979700 1 --2- 172.21.15.27:0/4040120663 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f291c0f51a0 0x7f291c0f5830 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:02.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.670+0000 7f2920979700 1 -- 172.21.15.27:0/4040120663 >> 172.21.15.27:0/4040120663 conn(0x7f291c0f0320 msgr2=0x7f291c0fb370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:45:02.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.670+0000 7f2920979700 1 -- 172.21.15.27:0/4040120663 shutdown_connections 2021-11-03T18:45:02.675 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.670+0000 7f2920979700 1 -- 172.21.15.27:0/4040120663 wait complete. 2021-11-03T18:45:02.768 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.770+0000 7efd2ad5f700 1 -- 172.21.15.27:0/1897929625 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd240f5ef0 msgr2=0x7efd240f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:45:02.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.770+0000 7efd2ad5f700 1 --2- 172.21.15.27:0/1897929625 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd240f5ef0 0x7efd240f6920 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 rx=0x7efd140099e0 tx=0x7efd14004d10).stop 2021-11-03T18:45:02.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.770+0000 7efd2ad5f700 1 -- 172.21.15.27:0/1897929625 shutdown_connections 2021-11-03T18:45:02.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.770+0000 7efd2ad5f700 1 --2- 172.21.15.27:0/1897929625 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd240f5ef0 0x7efd240f6920 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:02.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.770+0000 7efd2ad5f700 1 --2- 172.21.15.27:0/1897929625 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd240f55e0 0x7efd240f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:02.770 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.770+0000 7efd2ad5f700 1 -- 172.21.15.27:0/1897929625 >> 172.21.15.27:0/1897929625 conn(0x7efd240f0de0 msgr2=0x7efd240f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:45:02.770 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.770+0000 7efd2ad5f700 1 -- 172.21.15.27:0/1897929625 shutdown_connections 2021-11-03T18:45:02.770 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.770+0000 7efd2ad5f700 1 -- 172.21.15.27:0/1897929625 wait complete. 2021-11-03T18:45:02.770 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.770+0000 7efd2ad5f700 1 Processor -- start 2021-11-03T18:45:02.771 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.770+0000 7efd2ad5f700 1 -- start start 2021-11-03T18:45:02.773 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.771+0000 7efd2ad5f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd240f55e0 0x7efd24109d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:45:02.773 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.771+0000 7efd2ad5f700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd240f5ef0 0x7efd2410c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:45:02.773 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.771+0000 7efd2ad5f700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd24060500 con 0x7efd240f55e0 2021-11-03T18:45:02.774 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.771+0000 7efd2ad5f700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd24060640 con 0x7efd240f5ef0 2021-11-03T18:45:02.774 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.771+0000 7efd28afb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd240f55e0 0x7efd24109d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:45:02.774 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.771+0000 7efd28afb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd240f55e0 0x7efd24109d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36632/0 (socket says 172.21.15.27:36632) 2021-11-03T18:45:02.774 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.771+0000 7efd28afb700 1 -- 172.21.15.27:0/1970017492 learned_addr learned my addr 172.21.15.27:0/1970017492 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:45:02.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.771+0000 7efd23fff700 1 --2- 172.21.15.27:0/1970017492 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd240f5ef0 0x7efd2410c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:45:02.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.771+0000 7efd28afb700 1 -- 172.21.15.27:0/1970017492 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd240f5ef0 msgr2=0x7efd2410c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:45:02.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.771+0000 7efd28afb700 1 --2- 172.21.15.27:0/1970017492 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd240f5ef0 0x7efd2410c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:02.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.771+0000 7efd28afb700 1 -- 172.21.15.27:0/1970017492 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd14009640 con 0x7efd240f55e0 2021-11-03T18:45:02.775 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.771+0000 7efd28afb700 1 --2- 172.21.15.27:0/1970017492 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd240f55e0 0x7efd24109d60 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 rx=0x7efd1c00c530 tx=0x7efd1c007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:45:02.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.771+0000 7efd217fa700 1 -- 172.21.15.27:0/1970017492 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd1c00f650 con 0x7efd240f55e0 2021-11-03T18:45:02.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.771+0000 7efd2ad5f700 1 -- 172.21.15.27:0/1970017492 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd2410c810 con 0x7efd240f55e0 2021-11-03T18:45:02.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.771+0000 7efd217fa700 1 -- 172.21.15.27:0/1970017492 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7efd1c00fc60 con 0x7efd240f55e0 2021-11-03T18:45:02.776 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.771+0000 7efd217fa700 1 -- 172.21.15.27:0/1970017492 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efd1c012cd0 con 0x7efd240f55e0 2021-11-03T18:45:02.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.771+0000 7efd2ad5f700 1 -- 172.21.15.27:0/1970017492 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd2410cd50 con 0x7efd240f55e0 2021-11-03T18:45:02.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.772+0000 7efd2ad5f700 1 -- 172.21.15.27:0/1970017492 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd24045ad0 con 0x7efd240f55e0 2021-11-03T18:45:02.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.773+0000 7efd217fa700 1 -- 172.21.15.27:0/1970017492 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7efd1c010070 con 0x7efd240f55e0 2021-11-03T18:45:02.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.773+0000 7efd217fa700 1 --2- 172.21.15.27:0/1970017492 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7efd100640b0 0x7efd10066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:45:02.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.773+0000 7efd217fa700 1 -- 172.21.15.27:0/1970017492 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7efd1c0806e0 con 0x7efd240f55e0 2021-11-03T18:45:02.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.773+0000 7efd23fff700 1 --2- 172.21.15.27:0/1970017492 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7efd100640b0 0x7efd10066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:45:02.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.774+0000 7efd23fff700 1 --2- 172.21.15.27:0/1970017492 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7efd100640b0 0x7efd10066570 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7efd1400b3b0 tx=0x7efd14004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:45:02.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:02.777+0000 7efd217fa700 1 -- 172.21.15.27:0/1970017492 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7efd1c052950 con 0x7efd240f55e0 2021-11-03T18:45:02.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:02 smithi115 ceph-mon[65453]: pgmap v2713: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:03.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.007+0000 7efd2ad5f700 1 -- 172.21.15.27:0/1970017492 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7efd240fded0 con 0x7efd240f55e0 2021-11-03T18:45:03.006 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.008+0000 7efd217fa700 1 -- 172.21.15.27:0/1970017492 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7efd1c052540 con 0x7efd240f55e0 2021-11-03T18:45:03.007 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:45:03.007 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:45:03.007 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:45:03.007 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:45:03.008 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:45:03.008 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:45:03.008 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:45:03.008 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:45:03.008 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:45:03.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:45:03.009 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:45:03.009 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:45:03.009 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:45:03.010 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:45:03.010 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:45:03.010 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:45:03.010 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:45:03.010 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:45:03.011 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.009+0000 7efd2ad5f700 1 -- 172.21.15.27:0/1970017492 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7efd100640b0 msgr2=0x7efd10066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:45:03.012 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.009+0000 7efd2ad5f700 1 --2- 172.21.15.27:0/1970017492 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7efd100640b0 0x7efd10066570 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7efd1400b3b0 tx=0x7efd14004d10).stop 2021-11-03T18:45:03.012 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.009+0000 7efd2ad5f700 1 -- 172.21.15.27:0/1970017492 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd240f55e0 msgr2=0x7efd24109d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:45:03.012 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.009+0000 7efd2ad5f700 1 --2- 172.21.15.27:0/1970017492 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd240f55e0 0x7efd24109d60 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 rx=0x7efd1c00c530 tx=0x7efd1c007e20).stop 2021-11-03T18:45:03.012 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.009+0000 7efd2ad5f700 1 -- 172.21.15.27:0/1970017492 shutdown_connections 2021-11-03T18:45:03.013 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.009+0000 7efd2ad5f700 1 --2- 172.21.15.27:0/1970017492 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7efd100640b0 0x7efd10066570 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:03.013 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.009+0000 7efd2ad5f700 1 --2- 172.21.15.27:0/1970017492 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7efd240f55e0 0x7efd24109d60 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:03.013 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.009+0000 7efd2ad5f700 1 --2- 172.21.15.27:0/1970017492 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7efd240f5ef0 0x7efd2410c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:03.013 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.009+0000 7efd2ad5f700 1 -- 172.21.15.27:0/1970017492 >> 172.21.15.27:0/1970017492 conn(0x7efd240f0de0 msgr2=0x7efd240fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:45:03.013 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.010+0000 7efd2ad5f700 1 -- 172.21.15.27:0/1970017492 shutdown_connections 2021-11-03T18:45:03.014 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.010+0000 7efd2ad5f700 1 -- 172.21.15.27:0/1970017492 wait complete. 2021-11-03T18:45:03.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:02 smithi027 ceph-mon[97244]: pgmap v2713: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:03.108 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.109+0000 7fce6d7ca700 1 -- 172.21.15.27:0/2032732133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce680f5460 msgr2=0x7fce680f5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:45:03.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.109+0000 7fce6d7ca700 1 --2- 172.21.15.27:0/2032732133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce680f5460 0x7fce680f5880 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 rx=0x7fce5c004660 tx=0x7fce5c009b30).stop 2021-11-03T18:45:03.109 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.109+0000 7fce6d7ca700 1 -- 172.21.15.27:0/2032732133 shutdown_connections 2021-11-03T18:45:03.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.109+0000 7fce6d7ca700 1 --2- 172.21.15.27:0/2032732133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce680f6700 0x7fce680f6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T18:45:03.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.109+0000 7fce6d7ca700 1 --2- 172.21.15.27:0/2032732133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce680f5460 0x7fce680f5880 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:03.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.109+0000 7fce6d7ca700 1 -- 172.21.15.27:0/2032732133 >> 172.21.15.27:0/2032732133 conn(0x7fce680f0bc0 msgr2=0x7fce680f2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:45:03.110 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.109+0000 7fce6d7ca700 1 -- 172.21.15.27:0/2032732133 shutdown_connections 2021-11-03T18:45:03.111 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.109+0000 7fce6d7ca700 1 -- 172.21.15.27:0/2032732133 wait complete. 2021-11-03T18:45:03.113 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.110+0000 7fce6d7ca700 1 Processor -- start 2021-11-03T18:45:03.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.110+0000 7fce6d7ca700 1 -- start start 2021-11-03T18:45:03.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.110+0000 7fce6d7ca700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce680f5460 0x7fce68109ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:45:03.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.110+0000 7fce6d7ca700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce680f6700 0x7fce6810bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:45:03.114 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.110+0000 7fce6d7ca700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce68060450 con 0x7fce680f6700 2021-11-03T18:45:03.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.110+0000 7fce6d7ca700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce68060590 con 0x7fce680f5460 2021-11-03T18:45:03.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.110+0000 7fce667fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce680f6700 0x7fce6810bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:45:03.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.110+0000 7fce667fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce680f6700 0x7fce6810bfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36644/0 (socket says 172.21.15.27:36644) 2021-11-03T18:45:03.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.110+0000 7fce667fc700 1 -- 172.21.15.27:0/2104541693 learned_addr learned my addr 172.21.15.27:0/2104541693 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:45:03.115 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.111+0000 7fce66ffd700 1 --2- 172.21.15.27:0/2104541693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce680f5460 0x7fce68109ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:45:03.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.111+0000 7fce667fc700 1 -- 172.21.15.27:0/2104541693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce680f5460 msgr2=0x7fce68109ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:45:03.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.111+0000 7fce667fc700 1 --2- 172.21.15.27:0/2104541693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce680f5460 0x7fce68109ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:03.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.111+0000 7fce667fc700 1 -- 172.21.15.27:0/2104541693 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce5c005040 con 0x7fce680f6700 2021-11-03T18:45:03.116 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.111+0000 7fce667fc700 1 --2- 172.21.15.27:0/2104541693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce680f6700 0x7fce6810bfb0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 rx=0x7fce5800b410 tx=0x7fce58006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:45:03.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.111+0000 7fce4f7fe700 1 -- 172.21.15.27:0/2104541693 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce580125f0 con 0x7fce680f6700 2021-11-03T18:45:03.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.111+0000 7fce4f7fe700 1 -- 172.21.15.27:0/2104541693 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fce5800be80 con 0x7fce680f6700 2021-11-03T18:45:03.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.111+0000 7fce6d7ca700 1 -- 172.21.15.27:0/2104541693 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce6810c5c0 con 0x7fce680f6700 2021-11-03T18:45:03.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.111+0000 7fce4f7fe700 1 -- 172.21.15.27:0/2104541693 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fce58002ac0 con 0x7fce680f6700 2021-11-03T18:45:03.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.111+0000 7fce6d7ca700 1 -- 172.21.15.27:0/2104541693 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce6810cb80 con 0x7fce680f6700 2021-11-03T18:45:03.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.112+0000 7fce6d7ca700 1 -- 172.21.15.27:0/2104541693 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce68045ad0 con 0x7fce680f6700 2021-11-03T18:45:03.118 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.112+0000 7fce4f7fe700 1 -- 172.21.15.27:0/2104541693 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fce5800f030 con 0x7fce680f6700 2021-11-03T18:45:03.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.113+0000 7fce4f7fe700 1 --2- 172.21.15.27:0/2104541693 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fce50064100 0x7fce500665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:45:03.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.113+0000 7fce4f7fe700 1 -- 172.21.15.27:0/2104541693 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fce58080cc0 con 0x7fce680f6700 2021-11-03T18:45:03.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.113+0000 7fce66ffd700 1 --2- 172.21.15.27:0/2104541693 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fce50064100 0x7fce500665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:45:03.119 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.113+0000 7fce66ffd700 1 --2- 172.21.15.27:0/2104541693 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fce50064100 0x7fce500665c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7fce5c004e30 tx=0x7fce5c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:45:03.120 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.117+0000 7fce4f7fe700 1 -- 172.21.15.27:0/2104541693 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fce58052fc0 con 0x7fce680f6700 2021-11-03T18:45:03.319 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.320+0000 7fce6d7ca700 1 -- 172.21.15.27:0/2104541693 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fce68027a00 con 0x7fce680f6700 2021-11-03T18:45:03.320 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.321+0000 7fce4f7fe700 1 -- 172.21.15.27:0/2104541693 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7fce58052bb0 con 0x7fce680f6700 2021-11-03T18:45:03.320 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:45:03.320 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:45:03.321 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:45:03.321 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:45:03.321 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:45:03.322 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:45:03.322 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:45:03.322 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:45:03.322 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:45:03.322 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:45:03.323 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:45:03.323 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:45:03.323 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:45:03.324 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:45:03.324 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:45:03.324 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:45:03.325 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:45:03.325 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:45:03.325 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:45:03.325 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:45:03.326 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:45:03.326 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:45:03.326 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:45:03.326 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:45:03.327 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:45:03.327 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:45:03.327 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:45:03.327 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:45:03.328 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:45:03.328 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:45:03.328 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:45:03.328 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:45:03.328 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:45:03.329 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:45:03.329 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:45:03.329 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:45:03.329 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:45:03.330 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:45:03.330 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:45:03.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.323+0000 7fce6d7ca700 1 -- 172.21.15.27:0/2104541693 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fce50064100 msgr2=0x7fce500665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:45:03.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.323+0000 7fce6d7ca700 1 --2- 172.21.15.27:0/2104541693 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fce50064100 0x7fce500665c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7fce5c004e30 tx=0x7fce5c009b30).stop 2021-11-03T18:45:03.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.323+0000 7fce6d7ca700 1 -- 172.21.15.27:0/2104541693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce680f6700 msgr2=0x7fce6810bfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:45:03.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.323+0000 7fce6d7ca700 1 --2- 172.21.15.27:0/2104541693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce680f6700 0x7fce6810bfb0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 rx=0x7fce5800b410 tx=0x7fce58006ef0).stop 2021-11-03T18:45:03.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.323+0000 7fce6d7ca700 1 -- 172.21.15.27:0/2104541693 shutdown_connections 2021-11-03T18:45:03.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.323+0000 7fce6d7ca700 1 --2- 172.21.15.27:0/2104541693 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fce50064100 0x7fce500665c0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:03.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.323+0000 7fce6d7ca700 1 --2- 172.21.15.27:0/2104541693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fce680f5460 0x7fce68109ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:03.333 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.323+0000 7fce6d7ca700 1 --2- 172.21.15.27:0/2104541693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fce680f6700 0x7fce6810bfb0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:03.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.323+0000 7fce6d7ca700 1 -- 172.21.15.27:0/2104541693 >> 172.21.15.27:0/2104541693 conn(0x7fce680f0bc0 msgr2=0x7fce680fa150 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:45:03.334 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.323+0000 7fce6d7ca700 1 -- 172.21.15.27:0/2104541693 shutdown_connections 2021-11-03T18:45:03.335 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:03.323+0000 7fce6d7ca700 1 -- 172.21.15.27:0/2104541693 wait complete. 2021-11-03T18:45:03.335 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:45:03.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:03 smithi115 ceph-mon[65453]: from='client.36888 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:45:03.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:03 smithi115 ceph-mon[65453]: from='client.36892 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:45:03.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:03 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1970017492' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:03.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:03 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2104541693' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:45:04.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:03 smithi027 ceph-mon[97244]: from='client.36888 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:45:04.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:03 smithi027 ceph-mon[97244]: from='client.36892 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:45:04.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:03 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1970017492' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:04.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:03 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2104541693' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:45:04.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:04 smithi115 ceph-mon[65453]: pgmap v2714: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:05.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:04 smithi027 ceph-mon[97244]: pgmap v2714: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:06.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5347.488726 secs 2021-11-03T18:45:06.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5347.488726 secs 2021-11-03T18:45:07.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:06 smithi027 ceph-mon[97244]: pgmap v2715: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:07.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:06 smithi115 ceph-mon[65453]: pgmap v2715: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:08.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:45:08.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:45:08.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:45:08.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:08.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:08.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:08.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:07 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:08.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:45:08.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:45:08.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:45:08.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:08.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:08.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:08.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:07 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:09.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:08 smithi027 ceph-mon[97244]: pgmap v2716: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:09.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:08 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:45:09.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:08 smithi115 ceph-mon[65453]: pgmap v2716: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:09.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:08 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:45:11.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:10 smithi027 ceph-mon[97244]: pgmap v2717: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:11.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5352.488864 secs 2021-11-03T18:45:11.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:10 smithi115 ceph-mon[65453]: pgmap v2717: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:11.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5352.488864 secs 2021-11-03T18:45:13.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:12 smithi027 ceph-mon[97244]: pgmap v2718: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:13.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:12 smithi115 ceph-mon[65453]: pgmap v2718: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:15.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:14 smithi027 ceph-mon[97244]: pgmap v2719: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:15.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:14 smithi115 ceph-mon[65453]: pgmap v2719: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:16.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5357.488950 secs 2021-11-03T18:45:16.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5357.488950 secs 2021-11-03T18:45:17.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:16 smithi027 ceph-mon[97244]: pgmap v2720: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:17.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:16 smithi115 ceph-mon[65453]: pgmap v2720: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:18.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:45:18.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:45:18.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:45:18.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:18.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:18.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:18.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:17 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:18.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:45:18.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:45:18.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:45:18.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:18.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:18.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:18.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:17 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:19.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:18 smithi115 ceph-mon[65453]: pgmap v2721: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:19.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:18 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:45:19.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:18 smithi027 ceph-mon[97244]: pgmap v2721: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:19.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:18 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:45:21.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:20 smithi115 ceph-mon[65453]: pgmap v2722: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:21.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5362.489082 secs 2021-11-03T18:45:21.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:20 smithi027 ceph-mon[97244]: pgmap v2722: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:21.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5362.489082 secs 2021-11-03T18:45:23.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:22 smithi115 ceph-mon[65453]: pgmap v2723: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:23.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:22 smithi027 ceph-mon[97244]: pgmap v2723: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:25.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:24 smithi115 ceph-mon[65453]: pgmap v2724: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:25.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:24 smithi027 ceph-mon[97244]: pgmap v2724: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:26.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5367.489211 secs 2021-11-03T18:45:26.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5367.489211 secs 2021-11-03T18:45:27.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:26 smithi115 ceph-mon[65453]: pgmap v2725: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:27.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:26 smithi027 ceph-mon[97244]: pgmap v2725: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:28.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:45:28.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:45:28.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:45:28.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:28.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:28.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:28.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:27 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:28.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:45:28.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:45:28.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:45:28.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:28.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:28.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:28.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:27 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:29.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:28 smithi115 ceph-mon[65453]: pgmap v2726: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:29.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:28 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:45:29.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:28 smithi027 ceph-mon[97244]: pgmap v2726: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:29.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:28 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:45:31.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:30 smithi115 ceph-mon[65453]: pgmap v2727: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:31.502 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5372.489350 secs 2021-11-03T18:45:31.503 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:30 smithi027 ceph-mon[97244]: pgmap v2727: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:31.503 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5372.489350 secs 2021-11-03T18:45:33.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:32 smithi115 ceph-mon[65453]: pgmap v2728: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:33.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:32 smithi027 ceph-mon[97244]: pgmap v2728: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:33.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.438+0000 7f0e054d2700 1 -- 172.21.15.27:0/119524288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e00060270 msgr2=0x7f0e000f91c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:45:33.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.438+0000 7f0e054d2700 1 --2- 172.21.15.27:0/119524288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e00060270 0x7f0e000f91c0 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 rx=0x7f0de8004660 tx=0x7f0de8009b30).stop 2021-11-03T18:45:33.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.438+0000 7f0e054d2700 1 -- 172.21.15.27:0/119524288 shutdown_connections 2021-11-03T18:45:33.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.438+0000 7f0e054d2700 1 --2- 172.21.15.27:0/119524288 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e000f96b0 0x7f0e000fbb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:33.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.438+0000 7f0e054d2700 1 --2- 172.21.15.27:0/119524288 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e00060270 0x7f0e000f91c0 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:33.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.438+0000 7f0e054d2700 1 -- 172.21.15.27:0/119524288 >> 172.21.15.27:0/119524288 conn(0x7f0e000f0ba0 msgr2=0x7f0e000f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:45:33.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.438+0000 7f0e054d2700 1 -- 172.21.15.27:0/119524288 shutdown_connections 2021-11-03T18:45:33.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.438+0000 7f0e054d2700 1 -- 172.21.15.27:0/119524288 wait complete. 2021-11-03T18:45:33.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.438+0000 7f0e054d2700 1 Processor -- start 2021-11-03T18:45:33.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.439+0000 7f0e054d2700 1 -- start start 2021-11-03T18:45:33.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.439+0000 7f0e054d2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e00060270 0x7f0e000f6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:45:33.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.439+0000 7f0e054d2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e000f96b0 0x7f0e000f50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:45:33.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.439+0000 7f0e054d2700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e000f5620 con 0x7f0e00060270 2021-11-03T18:45:33.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.439+0000 7f0e054d2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e000f5760 con 0x7f0e000f96b0 2021-11-03T18:45:33.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.439+0000 7f0dfffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e00060270 0x7f0e000f6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:45:33.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.439+0000 7f0dfffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e00060270 0x7f0e000f6c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36662/0 (socket says 172.21.15.27:36662) 2021-11-03T18:45:33.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.439+0000 7f0dfffff700 1 -- 172.21.15.27:0/2837905568 learned_addr learned my addr 172.21.15.27:0/2837905568 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:45:33.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.439+0000 7f0dff7fe700 1 --2- 172.21.15.27:0/2837905568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e000f96b0 0x7f0e000f50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:45:33.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.439+0000 7f0dfffff700 1 -- 172.21.15.27:0/2837905568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e000f96b0 msgr2=0x7f0e000f50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:45:33.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.439+0000 7f0dfffff700 1 --2- 172.21.15.27:0/2837905568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e000f96b0 0x7f0e000f50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:33.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.439+0000 7f0dfffff700 1 -- 172.21.15.27:0/2837905568 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de8005040 con 0x7f0e00060270 2021-11-03T18:45:33.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.439+0000 7f0dff7fe700 1 --2- 172.21.15.27:0/2837905568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e000f96b0 0x7f0e000f50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:45:33.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.439+0000 7f0dfffff700 1 --2- 172.21.15.27:0/2837905568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e00060270 0x7f0e000f6c40 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 rx=0x7f0de8000c00 tx=0x7f0de800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:45:33.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.440+0000 7f0dfcff9700 1 -- 172.21.15.27:0/2837905568 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0de8019070 con 0x7f0e00060270 2021-11-03T18:45:33.444 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.440+0000 7f0dfcff9700 1 -- 172.21.15.27:0/2837905568 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f0de8007e70 con 0x7f0e00060270 2021-11-03T18:45:33.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.440+0000 7f0e054d2700 1 -- 172.21.15.27:0/2837905568 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e000f59e0 con 0x7f0e00060270 2021-11-03T18:45:33.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.440+0000 7f0dfcff9700 1 -- 172.21.15.27:0/2837905568 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0de8014e20 con 0x7f0e00060270 2021-11-03T18:45:33.445 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.440+0000 7f0e054d2700 1 -- 172.21.15.27:0/2837905568 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e000f5f40 con 0x7f0e00060270 2021-11-03T18:45:33.446 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.441+0000 7f0e054d2700 1 -- 172.21.15.27:0/2837905568 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e00045ad0 con 0x7f0e00060270 2021-11-03T18:45:33.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.441+0000 7f0dfcff9700 1 -- 172.21.15.27:0/2837905568 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f0de8025070 con 0x7f0e00060270 2021-11-03T18:45:33.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.442+0000 7f0dfcff9700 1 --2- 172.21.15.27:0/2837905568 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0dec0640a0 0x7f0dec066560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:45:33.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.442+0000 7f0dfcff9700 1 -- 172.21.15.27:0/2837905568 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f0de80818a0 con 0x7f0e00060270 2021-11-03T18:45:33.447 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.442+0000 7f0dff7fe700 1 --2- 172.21.15.27:0/2837905568 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0dec0640a0 0x7f0dec066560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:45:33.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.442+0000 7f0dff7fe700 1 --2- 172.21.15.27:0/2837905568 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0dec0640a0 0x7f0dec066560 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7f0e00060040 tx=0x7f0df000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:45:33.448 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.446+0000 7f0dfcff9700 1 -- 172.21.15.27:0/2837905568 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f0de8053ba0 con 0x7f0e00060270 2021-11-03T18:45:33.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.624+0000 7f0e054d2700 1 -- 172.21.15.27:0/2837905568 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e00057f10 con 0x7f0dec0640a0 2021-11-03T18:45:33.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.625+0000 7f0dfcff9700 1 -- 172.21.15.27:0/2837905568 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f0e00057f10 con 0x7f0dec0640a0 2021-11-03T18:45:33.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.627+0000 7f0e054d2700 1 -- 172.21.15.27:0/2837905568 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0dec0640a0 msgr2=0x7f0dec066560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:45:33.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.627+0000 7f0e054d2700 1 --2- 172.21.15.27:0/2837905568 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0dec0640a0 0x7f0dec066560 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7f0e00060040 tx=0x7f0df000b040).stop 2021-11-03T18:45:33.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.627+0000 7f0e054d2700 1 -- 172.21.15.27:0/2837905568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e00060270 msgr2=0x7f0e000f6c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:45:33.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.627+0000 7f0e054d2700 1 --2- 172.21.15.27:0/2837905568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e00060270 0x7f0e000f6c40 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 rx=0x7f0de8000c00 tx=0x7f0de800b040).stop 2021-11-03T18:45:33.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.627+0000 7f0e054d2700 1 -- 172.21.15.27:0/2837905568 shutdown_connections 2021-11-03T18:45:33.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.627+0000 7f0e054d2700 1 --2- 172.21.15.27:0/2837905568 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0dec0640a0 0x7f0dec066560 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:33.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.627+0000 7f0e054d2700 1 --2- 172.21.15.27:0/2837905568 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0e00060270 0x7f0e000f6c40 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:33.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.627+0000 7f0e054d2700 1 --2- 172.21.15.27:0/2837905568 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0e000f96b0 0x7f0e000f50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:33.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.627+0000 7f0e054d2700 1 -- 172.21.15.27:0/2837905568 >> 172.21.15.27:0/2837905568 conn(0x7f0e000f0ba0 msgr2=0x7f0e000f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:45:33.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.627+0000 7f0e054d2700 1 -- 172.21.15.27:0/2837905568 shutdown_connections 2021-11-03T18:45:33.629 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.627+0000 7f0e054d2700 1 -- 172.21.15.27:0/2837905568 wait complete. 2021-11-03T18:45:33.635 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:45:33.728 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.728+0000 7f38f7f25700 1 -- 172.21.15.27:0/560527455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38f00f66e0 msgr2=0x7f38f00f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:45:33.728 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.728+0000 7f38f7f25700 1 --2- 172.21.15.27:0/560527455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38f00f66e0 0x7f38f00f6b00 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 rx=0x7f38e80099e0 tx=0x7f38e8004d10).stop 2021-11-03T18:45:33.728 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.728+0000 7f38f7f25700 1 -- 172.21.15.27:0/560527455 shutdown_connections 2021-11-03T18:45:33.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.728+0000 7f38f7f25700 1 --2- 172.21.15.27:0/560527455 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38f00f7980 0x7f38f00f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:33.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.728+0000 7f38f7f25700 1 --2- 172.21.15.27:0/560527455 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38f00f66e0 0x7f38f00f6b00 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:33.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.728+0000 7f38f7f25700 1 -- 172.21.15.27:0/560527455 >> 172.21.15.27:0/560527455 conn(0x7f38f00f1e40 msgr2=0x7f38f00f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:45:33.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.728+0000 7f38f7f25700 1 -- 172.21.15.27:0/560527455 shutdown_connections 2021-11-03T18:45:33.729 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.728+0000 7f38f7f25700 1 -- 172.21.15.27:0/560527455 wait complete. 2021-11-03T18:45:33.730 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.728+0000 7f38f7f25700 1 Processor -- start 2021-11-03T18:45:33.730 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.728+0000 7f38f7f25700 1 -- start start 2021-11-03T18:45:33.730 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.728+0000 7f38f7f25700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38f00f66e0 0x7f38f01070d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:45:33.731 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.729+0000 7f38f7f25700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38f00f7980 0x7f38f01075c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:45:33.731 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.729+0000 7f38f7f25700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f0107b70 con 0x7f38f00f66e0 2021-11-03T18:45:33.731 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.729+0000 7f38f7f25700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f0107cb0 con 0x7f38f00f7980 2021-11-03T18:45:33.731 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.729+0000 7f38f5cc1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38f00f66e0 0x7f38f01070d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:45:33.732 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.729+0000 7f38f5cc1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38f00f66e0 0x7f38f01070d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36672/0 (socket says 172.21.15.27:36672) 2021-11-03T18:45:33.732 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.729+0000 7f38f5cc1700 1 -- 172.21.15.27:0/883835133 learned_addr learned my addr 172.21.15.27:0/883835133 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:45:33.732 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.729+0000 7f38f54c0700 1 --2- 172.21.15.27:0/883835133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38f00f7980 0x7f38f01075c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:45:33.732 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.729+0000 7f38f5cc1700 1 -- 172.21.15.27:0/883835133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38f00f7980 msgr2=0x7f38f01075c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:45:33.733 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.729+0000 7f38f5cc1700 1 --2- 172.21.15.27:0/883835133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38f00f7980 0x7f38f01075c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:33.733 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.729+0000 7f38f5cc1700 1 -- 172.21.15.27:0/883835133 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e8009640 con 0x7f38f00f66e0 2021-11-03T18:45:33.733 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.729+0000 7f38f5cc1700 1 --2- 172.21.15.27:0/883835133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38f00f66e0 0x7f38f01070d0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 rx=0x7f38e800c010 tx=0x7f38e8004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:45:33.733 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.729+0000 7f38e67fc700 1 -- 172.21.15.27:0/883835133 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38e800e070 con 0x7f38f00f66e0 2021-11-03T18:45:33.733 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.729+0000 7f38e67fc700 1 -- 172.21.15.27:0/883835133 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f38e8007dc0 con 0x7f38f00f66e0 2021-11-03T18:45:33.734 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.729+0000 7f38f7f25700 1 -- 172.21.15.27:0/883835133 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f01085f0 con 0x7f38f00f66e0 2021-11-03T18:45:33.734 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.729+0000 7f38e67fc700 1 -- 172.21.15.27:0/883835133 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38e8018070 con 0x7f38f00f66e0 2021-11-03T18:45:33.735 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.729+0000 7f38f7f25700 1 -- 172.21.15.27:0/883835133 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f0108ad0 con 0x7f38f00f66e0 2021-11-03T18:45:33.737 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.730+0000 7f38f7f25700 1 -- 172.21.15.27:0/883835133 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38f0045ad0 con 0x7f38f00f66e0 2021-11-03T18:45:33.737 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.731+0000 7f38e67fc700 1 -- 172.21.15.27:0/883835133 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f38e8024060 con 0x7f38f00f66e0 2021-11-03T18:45:33.737 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.731+0000 7f38e67fc700 1 --2- 172.21.15.27:0/883835133 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f38dc0640b0 0x7f38dc066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:45:33.738 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.731+0000 7f38e67fc700 1 -- 172.21.15.27:0/883835133 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f38e8082080 con 0x7f38f00f66e0 2021-11-03T18:45:33.738 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.731+0000 7f38f54c0700 1 --2- 172.21.15.27:0/883835133 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f38dc0640b0 0x7f38dc066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:45:33.739 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.732+0000 7f38f54c0700 1 --2- 172.21.15.27:0/883835133 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f38dc0640b0 0x7f38dc066570 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 rx=0x7f38e00096c0 tx=0x7f38e0009be0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:45:33.739 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.735+0000 7f38e67fc700 1 -- 172.21.15.27:0/883835133 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f38e80543e0 con 0x7f38f00f66e0 2021-11-03T18:45:33.907 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.909+0000 7f38f7f25700 1 -- 172.21.15.27:0/883835133 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f38f0100ec0 con 0x7f38dc0640b0 2021-11-03T18:45:33.923 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.923+0000 7f38e67fc700 1 -- 172.21.15.27:0/883835133 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3498 (secure 0 0 0) 0x7f38f0100ec0 con 0x7f38dc0640b0 2021-11-03T18:45:33.924 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:45:33.924 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (97m) 5m ago 98m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:45:33.925 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (88m) 5m ago 98m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:45:33.925 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (88m) 5m ago 97m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:45:33.925 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (96m) 5m ago 98m 42.9M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:45:33.926 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (90m) 5m ago 93m 26.9M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:45:33.926 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (93m) 5m ago 93m 3852M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:45:33.927 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (89m) 5m ago 93m 26.5M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:45:33.927 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (90m) 5m ago 93m 27.6M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:45:33.928 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (90m) 5m ago 99m 625M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:45:33.928 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (90m) 5m ago 97m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:45:33.929 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (89m) 5m ago 99m 387M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:45:33.929 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (89m) 5m ago 97m 175M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:45:33.929 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (97m) 5m ago 97m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:45:33.930 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (97m) 5m ago 97m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:45:33.930 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (88m) 5m ago 96m 433M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:45:33.931 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (87m) 5m ago 96m 471M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:45:33.931 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (87m) 5m ago 96m 356M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:45:33.938 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (86m) 5m ago 95m 504M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:45:33.938 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (86m) 5m ago 95m 396M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:45:33.939 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (86m) 5m ago 95m 408M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:45:33.939 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (96m) 5m ago 97m 66.8M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:45:33.940 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.925+0000 7f38f7f25700 1 -- 172.21.15.27:0/883835133 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f38dc0640b0 msgr2=0x7f38dc066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:45:33.940 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.925+0000 7f38f7f25700 1 --2- 172.21.15.27:0/883835133 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f38dc0640b0 0x7f38dc066570 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 rx=0x7f38e00096c0 tx=0x7f38e0009be0).stop 2021-11-03T18:45:33.941 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.925+0000 7f38f7f25700 1 -- 172.21.15.27:0/883835133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38f00f66e0 msgr2=0x7f38f01070d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:45:33.941 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.925+0000 7f38f7f25700 1 --2- 172.21.15.27:0/883835133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38f00f66e0 0x7f38f01070d0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 rx=0x7f38e800c010 tx=0x7f38e8004d10).stop 2021-11-03T18:45:33.941 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.925+0000 7f38f7f25700 1 -- 172.21.15.27:0/883835133 shutdown_connections 2021-11-03T18:45:33.942 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.925+0000 7f38f7f25700 1 --2- 172.21.15.27:0/883835133 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f38dc0640b0 0x7f38dc066570 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:33.942 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.925+0000 7f38f7f25700 1 --2- 172.21.15.27:0/883835133 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f38f00f66e0 0x7f38f01070d0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:33.942 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.925+0000 7f38f7f25700 1 --2- 172.21.15.27:0/883835133 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f38f00f7980 0x7f38f01075c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:33.943 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.925+0000 7f38f7f25700 1 -- 172.21.15.27:0/883835133 >> 172.21.15.27:0/883835133 conn(0x7f38f00f1e40 msgr2=0x7f38f00fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:45:33.943 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.925+0000 7f38f7f25700 1 -- 172.21.15.27:0/883835133 shutdown_connections 2021-11-03T18:45:33.943 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:33.925+0000 7f38f7f25700 1 -- 172.21.15.27:0/883835133 wait complete. 2021-11-03T18:45:34.024 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.024+0000 7f0621973700 1 -- 172.21.15.27:0/2514671260 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f061c0f73b0 msgr2=0x7f061c0f4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:45:34.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.024+0000 7f0621973700 1 --2- 172.21.15.27:0/2514671260 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f061c0f73b0 0x7f061c0f4640 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 rx=0x7f0608004660 tx=0x7f0608009b30).stop 2021-11-03T18:45:34.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.024+0000 7f0621973700 1 -- 172.21.15.27:0/2514671260 shutdown_connections 2021-11-03T18:45:34.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.024+0000 7f0621973700 1 --2- 172.21.15.27:0/2514671260 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f061c0f73b0 0x7f061c0f4640 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:34.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.024+0000 7f0621973700 1 --2- 172.21.15.27:0/2514671260 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f061c0f6a10 0x7f061c0f6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:34.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.024+0000 7f0621973700 1 -- 172.21.15.27:0/2514671260 >> 172.21.15.27:0/2514671260 conn(0x7f061c0f0320 msgr2=0x7f061c0f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:45:34.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.025+0000 7f0621973700 1 -- 172.21.15.27:0/2514671260 shutdown_connections 2021-11-03T18:45:34.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.025+0000 7f0621973700 1 -- 172.21.15.27:0/2514671260 wait complete. 2021-11-03T18:45:34.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.025+0000 7f0621973700 1 Processor -- start 2021-11-03T18:45:34.027 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.025+0000 7f0621973700 1 -- start start 2021-11-03T18:45:34.027 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.025+0000 7f0621973700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f061c0f6a10 0x7f061c108f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:45:34.027 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.025+0000 7f0621973700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f061c0f73b0 0x7f061c109430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:45:34.027 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.025+0000 7f0621973700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c1099e0 con 0x7f061c0f6a10 2021-11-03T18:45:34.028 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.025+0000 7f0621973700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f061c109b20 con 0x7f061c0f73b0 2021-11-03T18:45:34.028 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.025+0000 7f0620971700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f061c0f6a10 0x7f061c108f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:45:34.028 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.025+0000 7f0620971700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f061c0f6a10 0x7f061c108f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36682/0 (socket says 172.21.15.27:36682) 2021-11-03T18:45:34.028 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.026+0000 7f0620971700 1 -- 172.21.15.27:0/725104118 learned_addr learned my addr 172.21.15.27:0/725104118 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:45:34.029 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.026+0000 7f061bfff700 1 --2- 172.21.15.27:0/725104118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f061c0f73b0 0x7f061c109430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:45:34.029 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.026+0000 7f0620971700 1 -- 172.21.15.27:0/725104118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f061c0f73b0 msgr2=0x7f061c109430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:45:34.029 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.026+0000 7f0620971700 1 --2- 172.21.15.27:0/725104118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f061c0f73b0 0x7f061c109430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:34.029 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.026+0000 7f0620971700 1 -- 172.21.15.27:0/725104118 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0608005040 con 0x7f061c0f6a10 2021-11-03T18:45:34.030 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.026+0000 7f0620971700 1 --2- 172.21.15.27:0/725104118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f061c0f6a10 0x7f061c108f40 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 rx=0x7f061000c790 tx=0x7f0610007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:45:34.030 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.026+0000 7f06197fa700 1 -- 172.21.15.27:0/725104118 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0610002590 con 0x7f061c0f6a10 2021-11-03T18:45:34.030 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.026+0000 7f06197fa700 1 -- 172.21.15.27:0/725104118 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f0610002ba0 con 0x7f061c0f6a10 2021-11-03T18:45:34.031 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.026+0000 7f06197fa700 1 -- 172.21.15.27:0/725104118 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0610013d90 con 0x7f061c0f6a10 2021-11-03T18:45:34.031 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.026+0000 7f0621973700 1 -- 172.21.15.27:0/725104118 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f061c10a4c0 con 0x7f061c0f6a10 2021-11-03T18:45:34.031 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.026+0000 7f0621973700 1 -- 172.21.15.27:0/725104118 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f061c10aa80 con 0x7f061c0f6a10 2021-11-03T18:45:34.032 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.027+0000 7f0621973700 1 -- 172.21.15.27:0/725104118 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f061c045ad0 con 0x7f061c0f6a10 2021-11-03T18:45:34.032 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.027+0000 7f06197fa700 1 -- 172.21.15.27:0/725104118 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f061001a060 con 0x7f061c0f6a10 2021-11-03T18:45:34.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.028+0000 7f06197fa700 1 --2- 172.21.15.27:0/725104118 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f060c064100 0x7f060c0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:45:34.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.028+0000 7f06197fa700 1 -- 172.21.15.27:0/725104118 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f0610080b30 con 0x7f061c0f6a10 2021-11-03T18:45:34.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.028+0000 7f061bfff700 1 --2- 172.21.15.27:0/725104118 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f060c064100 0x7f060c0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:45:34.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.028+0000 7f061bfff700 1 --2- 172.21.15.27:0/725104118 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f060c064100 0x7f060c0665c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7f0608004630 tx=0x7f0608009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:45:34.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.032+0000 7f06197fa700 1 -- 172.21.15.27:0/725104118 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f0610052d40 con 0x7f061c0f6a10 2021-11-03T18:45:34.260 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.262+0000 7f0621973700 1 -- 172.21.15.27:0/725104118 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f061c027a00 con 0x7f061c0f6a10 2021-11-03T18:45:34.261 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.262+0000 7f06197fa700 1 -- 172.21.15.27:0/725104118 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f06100182e0 con 0x7f061c0f6a10 2021-11-03T18:45:34.261 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:45:34.261 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:45:34.261 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:45:34.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:45:34.262 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:45:34.262 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:45:34.262 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:45:34.263 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:45:34.263 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:45:34.263 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:45:34.263 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:45:34.263 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:45:34.264 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:45:34.264 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:45:34.264 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:45:34.264 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:45:34.265 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:45:34.265 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:45:34.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.264+0000 7f0621973700 1 -- 172.21.15.27:0/725104118 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f060c064100 msgr2=0x7f060c0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:45:34.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.264+0000 7f0621973700 1 --2- 172.21.15.27:0/725104118 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f060c064100 0x7f060c0665c0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7f0608004630 tx=0x7f0608009b30).stop 2021-11-03T18:45:34.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.264+0000 7f0621973700 1 -- 172.21.15.27:0/725104118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f061c0f6a10 msgr2=0x7f061c108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:45:34.266 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.264+0000 7f0621973700 1 --2- 172.21.15.27:0/725104118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f061c0f6a10 0x7f061c108f40 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 rx=0x7f061000c790 tx=0x7f0610007d70).stop 2021-11-03T18:45:34.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.264+0000 7f0621973700 1 -- 172.21.15.27:0/725104118 shutdown_connections 2021-11-03T18:45:34.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.264+0000 7f0621973700 1 --2- 172.21.15.27:0/725104118 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f060c064100 0x7f060c0665c0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:34.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.264+0000 7f0621973700 1 --2- 172.21.15.27:0/725104118 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f061c0f6a10 0x7f061c108f40 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:34.267 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.264+0000 7f0621973700 1 --2- 172.21.15.27:0/725104118 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f061c0f73b0 0x7f061c109430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:34.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.264+0000 7f0621973700 1 -- 172.21.15.27:0/725104118 >> 172.21.15.27:0/725104118 conn(0x7f061c0f0320 msgr2=0x7f061c0fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:45:34.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.264+0000 7f0621973700 1 -- 172.21.15.27:0/725104118 shutdown_connections 2021-11-03T18:45:34.268 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.264+0000 7f0621973700 1 -- 172.21.15.27:0/725104118 wait complete. 2021-11-03T18:45:34.363 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.365+0000 7f872202a700 1 -- 172.21.15.27:0/1453978458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f871c0f6ff0 msgr2=0x7f871c0f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:45:34.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.365+0000 7f872202a700 1 --2- 172.21.15.27:0/1453978458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f871c0f6ff0 0x7f871c0f7a20 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 rx=0x7f8710004660 tx=0x7f8710009b30).stop 2021-11-03T18:45:34.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.365+0000 7f872202a700 1 -- 172.21.15.27:0/1453978458 shutdown_connections 2021-11-03T18:45:34.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.365+0000 7f872202a700 1 --2- 172.21.15.27:0/1453978458 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f871c0f6ff0 0x7f871c0f7a20 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:34.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.365+0000 7f872202a700 1 --2- 172.21.15.27:0/1453978458 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f871c0f66e0 0x7f871c0f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:34.364 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.365+0000 7f872202a700 1 -- 172.21.15.27:0/1453978458 >> 172.21.15.27:0/1453978458 conn(0x7f871c0f1f00 msgr2=0x7f871c0f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:45:34.365 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.365+0000 7f872202a700 1 -- 172.21.15.27:0/1453978458 shutdown_connections 2021-11-03T18:45:34.365 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.365+0000 7f872202a700 1 -- 172.21.15.27:0/1453978458 wait complete. 2021-11-03T18:45:34.365 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.365+0000 7f872202a700 1 Processor -- start 2021-11-03T18:45:34.365 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.365+0000 7f872202a700 1 -- start start 2021-11-03T18:45:34.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.366+0000 7f872202a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f871c0f66e0 0x7f871c06b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:45:34.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.366+0000 7f872202a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f871c0f6ff0 0x7f871c069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:45:34.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.366+0000 7f872202a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871c06a380 con 0x7f871c0f6ff0 2021-11-03T18:45:34.368 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.366+0000 7f872202a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871c06a4c0 con 0x7f871c0f66e0 2021-11-03T18:45:34.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.366+0000 7f871affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f871c0f6ff0 0x7f871c069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:45:34.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.366+0000 7f871affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f871c0f6ff0 0x7f871c069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36694/0 (socket says 172.21.15.27:36694) 2021-11-03T18:45:34.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.366+0000 7f871affd700 1 -- 172.21.15.27:0/1477674167 learned_addr learned my addr 172.21.15.27:0/1477674167 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:45:34.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.366+0000 7f871b7fe700 1 --2- 172.21.15.27:0/1477674167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f871c0f66e0 0x7f871c06b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:45:34.369 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.366+0000 7f871affd700 1 -- 172.21.15.27:0/1477674167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f871c0f66e0 msgr2=0x7f871c06b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:45:34.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.366+0000 7f871affd700 1 --2- 172.21.15.27:0/1477674167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f871c0f66e0 0x7f871c06b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:34.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.366+0000 7f871affd700 1 -- 172.21.15.27:0/1477674167 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8710005040 con 0x7f871c0f6ff0 2021-11-03T18:45:34.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.366+0000 7f871affd700 1 --2- 172.21.15.27:0/1477674167 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f871c0f6ff0 0x7f871c069e20 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 rx=0x7f8710009990 tx=0x7f8710009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:45:34.370 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.366+0000 7f8703fff700 1 -- 172.21.15.27:0/1477674167 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8710018070 con 0x7f871c0f6ff0 2021-11-03T18:45:34.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.366+0000 7f8703fff700 1 -- 172.21.15.27:0/1477674167 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f8710007ba0 con 0x7f871c0f6ff0 2021-11-03T18:45:34.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.366+0000 7f872202a700 1 -- 172.21.15.27:0/1477674167 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f871c06a740 con 0x7f871c0f6ff0 2021-11-03T18:45:34.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.366+0000 7f8703fff700 1 -- 172.21.15.27:0/1477674167 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f871001c3c0 con 0x7f871c0f6ff0 2021-11-03T18:45:34.371 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.366+0000 7f872202a700 1 -- 172.21.15.27:0/1477674167 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f871c06aca0 con 0x7f871c0f6ff0 2021-11-03T18:45:34.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.367+0000 7f872202a700 1 -- 172.21.15.27:0/1477674167 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f871c045ad0 con 0x7f871c0f6ff0 2021-11-03T18:45:34.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.368+0000 7f8703fff700 1 -- 172.21.15.27:0/1477674167 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f871001d420 con 0x7f871c0f6ff0 2021-11-03T18:45:34.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.368+0000 7f8703fff700 1 --2- 172.21.15.27:0/1477674167 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f87040640b0 0x7f8704066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:45:34.372 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.368+0000 7f8703fff700 1 -- 172.21.15.27:0/1477674167 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f87100825d0 con 0x7f871c0f6ff0 2021-11-03T18:45:34.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.368+0000 7f871b7fe700 1 --2- 172.21.15.27:0/1477674167 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f87040640b0 0x7f8704066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:45:34.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.369+0000 7f871b7fe700 1 --2- 172.21.15.27:0/1477674167 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f87040640b0 0x7f8704066570 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0x7f871c06b730 tx=0x7f870c009be0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:45:34.373 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.372+0000 7f8703fff700 1 -- 172.21.15.27:0/1477674167 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f8710054990 con 0x7f871c0f6ff0 2021-11-03T18:45:34.575 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.577+0000 7f872202a700 1 -- 172.21.15.27:0/1477674167 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f871c027a00 con 0x7f871c0f6ff0 2021-11-03T18:45:34.576 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.577+0000 7f8703fff700 1 -- 172.21.15.27:0/1477674167 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f8710054580 con 0x7f871c0f6ff0 2021-11-03T18:45:34.576 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:45:34.576 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:45:34.577 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:45:34.577 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:45:34.577 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:45:34.577 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:45:34.578 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:45:34.578 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:45:34.578 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:45:34.578 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:45:34.578 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:45:34.579 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:45:34.579 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:45:34.579 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:45:34.579 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:45:34.580 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:45:34.580 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:45:34.580 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:45:34.580 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:45:34.580 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:45:34.581 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:45:34.581 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:45:34.581 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:45:34.582 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:45:34.582 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:45:34.582 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:45:34.582 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:45:34.582 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:45:34.583 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:45:34.583 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:45:34.583 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:45:34.583 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:45:34.584 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:45:34.584 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:45:34.584 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:45:34.584 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:45:34.584 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:45:34.585 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:45:34.585 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:45:34.586 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.579+0000 7f872202a700 1 -- 172.21.15.27:0/1477674167 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f87040640b0 msgr2=0x7f8704066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:45:34.586 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.579+0000 7f872202a700 1 --2- 172.21.15.27:0/1477674167 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f87040640b0 0x7f8704066570 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0x7f871c06b730 tx=0x7f870c009be0).stop 2021-11-03T18:45:34.586 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.579+0000 7f872202a700 1 -- 172.21.15.27:0/1477674167 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f871c0f6ff0 msgr2=0x7f871c069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:45:34.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.579+0000 7f872202a700 1 --2- 172.21.15.27:0/1477674167 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f871c0f6ff0 0x7f871c069e20 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 rx=0x7f8710009990 tx=0x7f8710009b30).stop 2021-11-03T18:45:34.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.579+0000 7f872202a700 1 -- 172.21.15.27:0/1477674167 shutdown_connections 2021-11-03T18:45:34.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.579+0000 7f872202a700 1 --2- 172.21.15.27:0/1477674167 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f87040640b0 0x7f8704066570 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:34.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.579+0000 7f872202a700 1 --2- 172.21.15.27:0/1477674167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f871c0f66e0 0x7f871c06b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:34.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.579+0000 7f872202a700 1 --2- 172.21.15.27:0/1477674167 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f871c0f6ff0 0x7f871c069e20 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:45:34.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.579+0000 7f872202a700 1 -- 172.21.15.27:0/1477674167 >> 172.21.15.27:0/1477674167 conn(0x7f871c0f1f00 msgr2=0x7f871c0fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:45:34.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.579+0000 7f872202a700 1 -- 172.21.15.27:0/1477674167 shutdown_connections 2021-11-03T18:45:34.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:45:34.580+0000 7f872202a700 1 -- 172.21.15.27:0/1477674167 wait complete. 2021-11-03T18:45:34.589 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:45:35.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:34 smithi115 ceph-mon[65453]: pgmap v2729: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:35.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:34 smithi115 ceph-mon[65453]: from='client.36904 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:45:35.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:34 smithi115 ceph-mon[65453]: from='client.36908 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:45:35.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:34 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/725104118' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:35.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:34 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1477674167' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:45:35.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:34 smithi027 ceph-mon[97244]: pgmap v2729: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:35.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:34 smithi027 ceph-mon[97244]: from='client.36904 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:45:35.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:34 smithi027 ceph-mon[97244]: from='client.36908 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:45:35.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:34 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/725104118' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:35.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:34 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1477674167' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:45:36.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5377.489433 secs 2021-11-03T18:45:36.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5377.489433 secs 2021-11-03T18:45:37.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:36 smithi115 ceph-mon[65453]: pgmap v2730: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:37.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:36 smithi027 ceph-mon[97244]: pgmap v2730: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:38.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:45:38.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:45:38.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:45:38.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:38.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:38.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:38.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:37 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:38.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:45:38.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:45:38.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:45:38.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:38.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:38.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:38.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:37 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:39.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:38 smithi115 ceph-mon[65453]: pgmap v2731: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:39.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:38 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:45:39.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:38 smithi027 ceph-mon[97244]: pgmap v2731: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:39.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:38 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:45:41.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:40 smithi115 ceph-mon[65453]: pgmap v2732: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:41.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5382.489518 secs 2021-11-03T18:45:41.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:40 smithi027 ceph-mon[97244]: pgmap v2732: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:41.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5382.489518 secs 2021-11-03T18:45:43.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:42 smithi115 ceph-mon[65453]: pgmap v2733: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:43.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:42 smithi027 ceph-mon[97244]: pgmap v2733: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:45.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:44 smithi115 ceph-mon[65453]: pgmap v2734: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:45.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:44 smithi027 ceph-mon[97244]: pgmap v2734: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:46.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5387.489654 secs 2021-11-03T18:45:46.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5387.489654 secs 2021-11-03T18:45:47.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:46 smithi115 ceph-mon[65453]: pgmap v2735: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:47.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:46 smithi027 ceph-mon[97244]: pgmap v2735: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:47.952 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:47 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:45:47.968 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:47 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:45:49.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:48 smithi115 ceph-mon[65453]: pgmap v2736: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:49.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:48 smithi115 ceph-mon[65453]: poll took 60181.287 ms: 1 events 2021-11-03T18:45:49.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:45:49.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:45:49.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:49.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:49.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:49.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:49.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:48 smithi027 ceph-mon[97244]: pgmap v2736: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:49.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:48 smithi027 ceph-mon[97244]: poll took 60181.287 ms: 1 events 2021-11-03T18:45:49.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:45:49.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:45:49.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:49.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:49.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:49.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:50.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:49 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:45:50.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:49 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:45:51.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:50 smithi115 ceph-mon[65453]: pgmap v2737: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:51.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5392.489768 secs 2021-11-03T18:45:51.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:50 smithi027 ceph-mon[97244]: pgmap v2737: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:51.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5392.489768 secs 2021-11-03T18:45:53.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:52 smithi115 ceph-mon[65453]: pgmap v2738: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:53.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:52 smithi027 ceph-mon[97244]: pgmap v2738: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:55.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:54 smithi115 ceph-mon[65453]: pgmap v2739: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:55.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:54 smithi027 ceph-mon[97244]: pgmap v2739: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:56.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5397.489907 secs 2021-11-03T18:45:56.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:45:56.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:45:56.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5397.489907 secs 2021-11-03T18:45:56.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:45:56.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:45:57.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:56 smithi115 ceph-mon[65453]: pgmap v2740: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:57.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:56 smithi027 ceph-mon[97244]: pgmap v2740: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:59.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:58 smithi115 ceph-mon[65453]: pgmap v2741: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:59.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:45:59.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:45:59.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:45:59.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:59.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:59.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:59.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:58 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:59.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:58 smithi027 ceph-mon[97244]: pgmap v2741: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:45:59.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:45:59.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:45:59.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:45:59.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:59.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:59.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:45:59.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:58 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:00.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:45:59 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:46:00.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:45:59 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:46:01.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:00 smithi115 ceph-mon[65453]: pgmap v2742: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:01.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5402.490047 secs 2021-11-03T18:46:01.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:00 smithi027 ceph-mon[97244]: pgmap v2742: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:01.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5402.490047 secs 2021-11-03T18:46:03.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:02 smithi115 ceph-mon[65453]: pgmap v2743: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:03.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:02 smithi027 ceph-mon[97244]: pgmap v2743: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:04.694 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.693+0000 7fa6e7a96700 1 -- 172.21.15.27:0/2542930449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6e00f6ff0 msgr2=0x7fa6e00f7a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:46:04.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.693+0000 7fa6e7a96700 1 --2- 172.21.15.27:0/2542930449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6e00f6ff0 0x7fa6e00f7a20 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 rx=0x7fa6dc004660 tx=0x7fa6dc009b30).stop 2021-11-03T18:46:04.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.693+0000 7fa6e7a96700 1 -- 172.21.15.27:0/2542930449 shutdown_connections 2021-11-03T18:46:04.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.693+0000 7fa6e7a96700 1 --2- 172.21.15.27:0/2542930449 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6e00f6ff0 0x7fa6e00f7a20 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:04.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.693+0000 7fa6e7a96700 1 --2- 172.21.15.27:0/2542930449 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6e00f66e0 0x7fa6e00f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:04.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.693+0000 7fa6e7a96700 1 -- 172.21.15.27:0/2542930449 >> 172.21.15.27:0/2542930449 conn(0x7fa6e00f1f00 msgr2=0x7fa6e00f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:46:04.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.693+0000 7fa6e7a96700 1 -- 172.21.15.27:0/2542930449 shutdown_connections 2021-11-03T18:46:04.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.694+0000 7fa6e7a96700 1 -- 172.21.15.27:0/2542930449 wait complete. 2021-11-03T18:46:04.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.694+0000 7fa6e7a96700 1 Processor -- start 2021-11-03T18:46:04.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.694+0000 7fa6e7a96700 1 -- start start 2021-11-03T18:46:04.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.694+0000 7fa6e7a96700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6e00f66e0 0x7fa6e006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:46:04.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.694+0000 7fa6e7a96700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6e00f6ff0 0x7fa6e0069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:46:04.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.694+0000 7fa6e7a96700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e006a380 con 0x7fa6e00f66e0 2021-11-03T18:46:04.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.694+0000 7fa6e7a96700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e006a4c0 con 0x7fa6e00f6ff0 2021-11-03T18:46:04.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.694+0000 7fa6e5832700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6e00f66e0 0x7fa6e006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:46:04.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.694+0000 7fa6e5832700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6e00f66e0 0x7fa6e006b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36726/0 (socket says 172.21.15.27:36726) 2021-11-03T18:46:04.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.694+0000 7fa6e5832700 1 -- 172.21.15.27:0/2033598844 learned_addr learned my addr 172.21.15.27:0/2033598844 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:46:04.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.695+0000 7fa6e5031700 1 --2- 172.21.15.27:0/2033598844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6e00f6ff0 0x7fa6e0069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:46:04.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.695+0000 7fa6e5832700 1 -- 172.21.15.27:0/2033598844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6e00f6ff0 msgr2=0x7fa6e0069e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:46:04.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.695+0000 7fa6e5832700 1 --2- 172.21.15.27:0/2033598844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6e00f6ff0 0x7fa6e0069e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:04.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.695+0000 7fa6e5832700 1 -- 172.21.15.27:0/2033598844 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6dc005040 con 0x7fa6e00f66e0 2021-11-03T18:46:04.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.695+0000 7fa6e5832700 1 --2- 172.21.15.27:0/2033598844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6e00f66e0 0x7fa6e006b9a0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 rx=0x7fa6d000e7f0 tx=0x7fa6d000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:46:04.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.695+0000 7fa6d67fc700 1 -- 172.21.15.27:0/2033598844 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6d00092e0 con 0x7fa6e00f66e0 2021-11-03T18:46:04.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.695+0000 7fa6d67fc700 1 -- 172.21.15.27:0/2033598844 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fa6d000f040 con 0x7fa6e00f66e0 2021-11-03T18:46:04.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.695+0000 7fa6e7a96700 1 -- 172.21.15.27:0/2033598844 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6e006a7a0 con 0x7fa6e00f66e0 2021-11-03T18:46:04.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.695+0000 7fa6d67fc700 1 -- 172.21.15.27:0/2033598844 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6d0014750 con 0x7fa6e00f66e0 2021-11-03T18:46:04.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.695+0000 7fa6e7a96700 1 -- 172.21.15.27:0/2033598844 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6e010ef20 con 0x7fa6e00f66e0 2021-11-03T18:46:04.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.696+0000 7fa6e7a96700 1 -- 172.21.15.27:0/2033598844 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6e0045ad0 con 0x7fa6e00f66e0 2021-11-03T18:46:04.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.696+0000 7fa6d67fc700 1 -- 172.21.15.27:0/2033598844 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fa6d001b030 con 0x7fa6e00f66e0 2021-11-03T18:46:04.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.697+0000 7fa6d67fc700 1 --2- 172.21.15.27:0/2033598844 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa6cc0640b0 0x7fa6cc066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:46:04.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.697+0000 7fa6d67fc700 1 -- 172.21.15.27:0/2033598844 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fa6d0080650 con 0x7fa6e00f66e0 2021-11-03T18:46:04.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.697+0000 7fa6e5031700 1 --2- 172.21.15.27:0/2033598844 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa6cc0640b0 0x7fa6cc066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:46:04.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.697+0000 7fa6e5031700 1 --2- 172.21.15.27:0/2033598844 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa6cc0640b0 0x7fa6cc066570 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0x7fa6dc004d80 tx=0x7fa6dc00b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:46:04.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.701+0000 7fa6d67fc700 1 -- 172.21.15.27:0/2033598844 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fa6d0052970 con 0x7fa6e00f66e0 2021-11-03T18:46:04.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.879+0000 7fa6e7a96700 1 -- 172.21.15.27:0/2033598844 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6e00ff050 con 0x7fa6cc0640b0 2021-11-03T18:46:04.879 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.880+0000 7fa6d67fc700 1 -- 172.21.15.27:0/2033598844 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fa6e00ff050 con 0x7fa6cc0640b0 2021-11-03T18:46:04.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.882+0000 7fa6e7a96700 1 -- 172.21.15.27:0/2033598844 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa6cc0640b0 msgr2=0x7fa6cc066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:46:04.882 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.882+0000 7fa6e7a96700 1 --2- 172.21.15.27:0/2033598844 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa6cc0640b0 0x7fa6cc066570 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 rx=0x7fa6dc004d80 tx=0x7fa6dc00b040).stop 2021-11-03T18:46:04.882 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.882+0000 7fa6e7a96700 1 -- 172.21.15.27:0/2033598844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6e00f66e0 msgr2=0x7fa6e006b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:46:04.882 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.882+0000 7fa6e7a96700 1 --2- 172.21.15.27:0/2033598844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6e00f66e0 0x7fa6e006b9a0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 rx=0x7fa6d000e7f0 tx=0x7fa6d000b040).stop 2021-11-03T18:46:04.882 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.882+0000 7fa6e7a96700 1 -- 172.21.15.27:0/2033598844 shutdown_connections 2021-11-03T18:46:04.883 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.882+0000 7fa6e7a96700 1 --2- 172.21.15.27:0/2033598844 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fa6cc0640b0 0x7fa6cc066570 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:04.883 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.882+0000 7fa6e7a96700 1 --2- 172.21.15.27:0/2033598844 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fa6e00f66e0 0x7fa6e006b9a0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:04.883 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.882+0000 7fa6e7a96700 1 --2- 172.21.15.27:0/2033598844 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fa6e00f6ff0 0x7fa6e0069e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:04.883 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.882+0000 7fa6e7a96700 1 -- 172.21.15.27:0/2033598844 >> 172.21.15.27:0/2033598844 conn(0x7fa6e00f1f00 msgr2=0x7fa6e00fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:46:04.884 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.882+0000 7fa6e7a96700 1 -- 172.21.15.27:0/2033598844 shutdown_connections 2021-11-03T18:46:04.884 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.882+0000 7fa6e7a96700 1 -- 172.21.15.27:0/2033598844 wait complete. 2021-11-03T18:46:04.892 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:46:04.982 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.982+0000 7faad40ec700 1 -- 172.21.15.27:0/1450628563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faacc0f5ef0 msgr2=0x7faacc0f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:46:04.982 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.982+0000 7faad40ec700 1 --2- 172.21.15.27:0/1450628563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faacc0f5ef0 0x7faacc0f6920 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 rx=0x7faabc004660 tx=0x7faabc009b30).stop 2021-11-03T18:46:04.983 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.983+0000 7faad40ec700 1 -- 172.21.15.27:0/1450628563 shutdown_connections 2021-11-03T18:46:04.983 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.983+0000 7faad40ec700 1 --2- 172.21.15.27:0/1450628563 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faacc0f5ef0 0x7faacc0f6920 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:04.983 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.983+0000 7faad40ec700 1 --2- 172.21.15.27:0/1450628563 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faacc0f55e0 0x7faacc0f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:04.983 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.983+0000 7faad40ec700 1 -- 172.21.15.27:0/1450628563 >> 172.21.15.27:0/1450628563 conn(0x7faacc0f0de0 msgr2=0x7faacc0f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:46:04.984 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.983+0000 7faad40ec700 1 -- 172.21.15.27:0/1450628563 shutdown_connections 2021-11-03T18:46:04.984 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.983+0000 7faad40ec700 1 -- 172.21.15.27:0/1450628563 wait complete. 2021-11-03T18:46:04.986 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.983+0000 7faad40ec700 1 Processor -- start 2021-11-03T18:46:04.987 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.983+0000 7faad40ec700 1 -- start start 2021-11-03T18:46:04.987 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.983+0000 7faad40ec700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faacc0f55e0 0x7faacc106f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:46:04.987 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.983+0000 7faad40ec700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faacc0f5ef0 0x7faacc107410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:46:04.987 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.983+0000 7faad40ec700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faacc1079c0 con 0x7faacc0f5ef0 2021-11-03T18:46:04.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.983+0000 7faad40ec700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faacc107b00 con 0x7faacc0f55e0 2021-11-03T18:46:04.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.984+0000 7faad1687700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faacc0f5ef0 0x7faacc107410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:46:04.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.984+0000 7faad1687700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faacc0f5ef0 0x7faacc107410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36738/0 (socket says 172.21.15.27:36738) 2021-11-03T18:46:04.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.984+0000 7faad1687700 1 -- 172.21.15.27:0/2051191354 learned_addr learned my addr 172.21.15.27:0/2051191354 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:46:04.989 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.984+0000 7faad1e88700 1 --2- 172.21.15.27:0/2051191354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faacc0f55e0 0x7faacc106f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:46:04.989 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.984+0000 7faad1687700 1 -- 172.21.15.27:0/2051191354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faacc0f55e0 msgr2=0x7faacc106f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:46:04.989 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.984+0000 7faad1687700 1 --2- 172.21.15.27:0/2051191354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faacc0f55e0 0x7faacc106f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:04.989 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.984+0000 7faad1687700 1 -- 172.21.15.27:0/2051191354 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faabc005040 con 0x7faacc0f5ef0 2021-11-03T18:46:04.990 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.984+0000 7faad1687700 1 --2- 172.21.15.27:0/2051191354 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faacc0f5ef0 0x7faacc107410 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 rx=0x7faabc000c00 tx=0x7faabc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:46:04.990 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.984+0000 7faac27fc700 1 -- 172.21.15.27:0/2051191354 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faabc019070 con 0x7faacc0f5ef0 2021-11-03T18:46:04.990 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.984+0000 7faac27fc700 1 -- 172.21.15.27:0/2051191354 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7faabc007e70 con 0x7faacc0f5ef0 2021-11-03T18:46:04.990 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.984+0000 7faad40ec700 1 -- 172.21.15.27:0/2051191354 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faacc108440 con 0x7faacc0f5ef0 2021-11-03T18:46:04.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.984+0000 7faac27fc700 1 -- 172.21.15.27:0/2051191354 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faabc014c90 con 0x7faacc0f5ef0 2021-11-03T18:46:04.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.984+0000 7faad40ec700 1 -- 172.21.15.27:0/2051191354 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faacc1089a0 con 0x7faacc0f5ef0 2021-11-03T18:46:04.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.985+0000 7faad40ec700 1 -- 172.21.15.27:0/2051191354 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faacc045ad0 con 0x7faacc0f5ef0 2021-11-03T18:46:04.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.986+0000 7faac27fc700 1 -- 172.21.15.27:0/2051191354 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7faabc025070 con 0x7faacc0f5ef0 2021-11-03T18:46:04.992 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.986+0000 7faac27fc700 1 --2- 172.21.15.27:0/2051191354 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7faab8064180 0x7faab8066640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:46:04.992 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.986+0000 7faac27fc700 1 -- 172.21.15.27:0/2051191354 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7faabc0815b0 con 0x7faacc0f5ef0 2021-11-03T18:46:04.992 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.986+0000 7faad1e88700 1 --2- 172.21.15.27:0/2051191354 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7faab8064180 0x7faab8066640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:46:04.992 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.986+0000 7faad1e88700 1 --2- 172.21.15.27:0/2051191354 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7faab8064180 0x7faab8066640 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0x7faac8009fd0 tx=0x7faac800b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:46:04.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:04.990+0000 7faac27fc700 1 -- 172.21.15.27:0/2051191354 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7faabc053990 con 0x7faacc0f5ef0 2021-11-03T18:46:05.162 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.162+0000 7faad40ec700 1 -- 172.21.15.27:0/2051191354 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7faacc05fe30 con 0x7faab8064180 2021-11-03T18:46:05.180 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.177+0000 7faac27fc700 1 -- 172.21.15.27:0/2051191354 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7faacc05fe30 con 0x7faab8064180 2021-11-03T18:46:05.181 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:46:05.181 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (97m) 6m ago 98m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:46:05.182 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (89m) 6m ago 98m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:46:05.182 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (89m) 5m ago 98m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:46:05.182 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (97m) 6m ago 98m 42.9M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:46:05.182 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (91m) 6m ago 93m 26.9M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:46:05.183 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (94m) 6m ago 94m 3852M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:46:05.183 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (89m) 5m ago 94m 26.5M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:46:05.183 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (91m) 5m ago 94m 27.6M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:46:05.183 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (91m) 6m ago 100m 625M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:46:05.184 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (90m) 5m ago 98m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:46:05.184 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (90m) 6m ago 100m 387M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:46:05.184 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (89m) 5m ago 98m 175M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:46:05.184 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (98m) 6m ago 98m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:46:05.185 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (97m) 5m ago 97m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:46:05.185 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (88m) 6m ago 97m 433M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:46:05.185 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (88m) 6m ago 96m 471M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:46:05.185 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (87m) 6m ago 96m 356M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:46:05.185 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (87m) 5m ago 96m 504M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:46:05.186 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (87m) 5m ago 95m 396M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:46:05.186 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (86m) 5m ago 95m 408M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:46:05.186 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (97m) 6m ago 98m 66.8M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:46:05.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.179+0000 7faad40ec700 1 -- 172.21.15.27:0/2051191354 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7faab8064180 msgr2=0x7faab8066640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:46:05.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.179+0000 7faad40ec700 1 --2- 172.21.15.27:0/2051191354 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7faab8064180 0x7faab8066640 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 rx=0x7faac8009fd0 tx=0x7faac800b040).stop 2021-11-03T18:46:05.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.179+0000 7faad40ec700 1 -- 172.21.15.27:0/2051191354 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faacc0f5ef0 msgr2=0x7faacc107410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:46:05.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.179+0000 7faad40ec700 1 --2- 172.21.15.27:0/2051191354 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faacc0f5ef0 0x7faacc107410 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 rx=0x7faabc000c00 tx=0x7faabc00b040).stop 2021-11-03T18:46:05.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.179+0000 7faad40ec700 1 -- 172.21.15.27:0/2051191354 shutdown_connections 2021-11-03T18:46:05.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.179+0000 7faad40ec700 1 --2- 172.21.15.27:0/2051191354 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7faab8064180 0x7faab8066640 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:05.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.179+0000 7faad40ec700 1 --2- 172.21.15.27:0/2051191354 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faacc0f55e0 0x7faacc106f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:05.188 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.179+0000 7faad40ec700 1 --2- 172.21.15.27:0/2051191354 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faacc0f5ef0 0x7faacc107410 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:05.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.179+0000 7faad40ec700 1 -- 172.21.15.27:0/2051191354 >> 172.21.15.27:0/2051191354 conn(0x7faacc0f0de0 msgr2=0x7faacc05f720 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:46:05.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.179+0000 7faad40ec700 1 -- 172.21.15.27:0/2051191354 shutdown_connections 2021-11-03T18:46:05.189 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.179+0000 7faad40ec700 1 -- 172.21.15.27:0/2051191354 wait complete. 2021-11-03T18:46:05.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:04 smithi115 ceph-mon[65453]: pgmap v2744: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:05.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:04 smithi027 ceph-mon[97244]: pgmap v2744: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:05.278 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.278+0000 7fc3257ac700 1 -- 172.21.15.27:0/3432760696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3200f51b0 msgr2=0x7fc3200f7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:46:05.278 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.278+0000 7fc3257ac700 1 --2- 172.21.15.27:0/3432760696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3200f51b0 0x7fc3200f7930 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 rx=0x7fc308004660 tx=0x7fc308009b30).stop 2021-11-03T18:46:05.278 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.278+0000 7fc3257ac700 1 -- 172.21.15.27:0/3432760696 shutdown_connections 2021-11-03T18:46:05.279 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.278+0000 7fc3257ac700 1 --2- 172.21.15.27:0/3432760696 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3200f51b0 0x7fc3200f7930 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:05.279 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.278+0000 7fc3257ac700 1 --2- 172.21.15.27:0/3432760696 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3200f4810 0x7fc3200f4c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:05.279 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.278+0000 7fc3257ac700 1 -- 172.21.15.27:0/3432760696 >> 172.21.15.27:0/3432760696 conn(0x7fc3200f0330 msgr2=0x7fc3200f2750 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:46:05.279 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.278+0000 7fc3257ac700 1 -- 172.21.15.27:0/3432760696 shutdown_connections 2021-11-03T18:46:05.279 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.279+0000 7fc3257ac700 1 -- 172.21.15.27:0/3432760696 wait complete. 2021-11-03T18:46:05.283 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.279+0000 7fc3257ac700 1 Processor -- start 2021-11-03T18:46:05.283 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.279+0000 7fc3257ac700 1 -- start start 2021-11-03T18:46:05.283 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.279+0000 7fc3257ac700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3200f4810 0x7fc320106e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:46:05.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.279+0000 7fc3257ac700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3200f51b0 0x7fc320107320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:46:05.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.279+0000 7fc3257ac700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc320107920 con 0x7fc3200f51b0 2021-11-03T18:46:05.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.279+0000 7fc3257ac700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc320108130 con 0x7fc3200f4810 2021-11-03T18:46:05.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.279+0000 7fc31e7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3200f51b0 0x7fc320107320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:46:05.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.280+0000 7fc31e7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3200f51b0 0x7fc320107320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36748/0 (socket says 172.21.15.27:36748) 2021-11-03T18:46:05.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.280+0000 7fc31e7fc700 1 -- 172.21.15.27:0/1497022065 learned_addr learned my addr 172.21.15.27:0/1497022065 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:46:05.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.280+0000 7fc31effd700 1 --2- 172.21.15.27:0/1497022065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3200f4810 0x7fc320106e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:46:05.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.280+0000 7fc31e7fc700 1 -- 172.21.15.27:0/1497022065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3200f4810 msgr2=0x7fc320106e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:46:05.286 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.280+0000 7fc31e7fc700 1 --2- 172.21.15.27:0/1497022065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3200f4810 0x7fc320106e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:05.286 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.280+0000 7fc31e7fc700 1 -- 172.21.15.27:0/1497022065 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc308005040 con 0x7fc3200f51b0 2021-11-03T18:46:05.286 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.280+0000 7fc31e7fc700 1 --2- 172.21.15.27:0/1497022065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3200f51b0 0x7fc320107320 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 rx=0x7fc308004f60 tx=0x7fc308009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:46:05.286 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.280+0000 7fc3177fe700 1 -- 172.21.15.27:0/1497022065 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc308018070 con 0x7fc3200f51b0 2021-11-03T18:46:05.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.280+0000 7fc3177fe700 1 -- 172.21.15.27:0/1497022065 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fc308007ba0 con 0x7fc3200f51b0 2021-11-03T18:46:05.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.280+0000 7fc3257ac700 1 -- 172.21.15.27:0/1497022065 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc320108350 con 0x7fc3200f51b0 2021-11-03T18:46:05.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.280+0000 7fc3177fe700 1 -- 172.21.15.27:0/1497022065 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc30801c620 con 0x7fc3200f51b0 2021-11-03T18:46:05.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.280+0000 7fc3257ac700 1 -- 172.21.15.27:0/1497022065 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3201088b0 con 0x7fc3200f51b0 2021-11-03T18:46:05.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.281+0000 7fc3257ac700 1 -- 172.21.15.27:0/1497022065 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc320059100 con 0x7fc3200f51b0 2021-11-03T18:46:05.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.281+0000 7fc3177fe700 1 -- 172.21.15.27:0/1497022065 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fc30801c8e0 con 0x7fc3200f51b0 2021-11-03T18:46:05.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.282+0000 7fc3177fe700 1 --2- 172.21.15.27:0/1497022065 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc30c064100 0x7fc30c0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:46:05.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.282+0000 7fc3177fe700 1 -- 172.21.15.27:0/1497022065 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fc3080826e0 con 0x7fc3200f51b0 2021-11-03T18:46:05.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.282+0000 7fc31effd700 1 --2- 172.21.15.27:0/1497022065 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc30c064100 0x7fc30c0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:46:05.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.282+0000 7fc31effd700 1 --2- 172.21.15.27:0/1497022065 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc30c064100 0x7fc30c0665c0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7fc320106ba0 tx=0x7fc318009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:46:05.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.286+0000 7fc3177fe700 1 -- 172.21.15.27:0/1497022065 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fc308054a40 con 0x7fc3200f51b0 2021-11-03T18:46:05.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.516+0000 7fc3257ac700 1 -- 172.21.15.27:0/1497022065 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fc3200fed30 con 0x7fc3200f51b0 2021-11-03T18:46:05.516 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.516+0000 7fc3177fe700 1 -- 172.21.15.27:0/1497022065 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fc308054630 con 0x7fc3200f51b0 2021-11-03T18:46:05.516 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:46:05.516 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:46:05.517 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:46:05.517 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:46:05.517 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:46:05.517 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:46:05.517 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:46:05.518 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:46:05.518 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:46:05.518 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:46:05.518 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:46:05.519 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:46:05.519 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:46:05.519 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:46:05.519 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:46:05.519 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:46:05.520 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:46:05.520 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:46:05.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.518+0000 7fc3257ac700 1 -- 172.21.15.27:0/1497022065 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc30c064100 msgr2=0x7fc30c0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:46:05.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.518+0000 7fc3257ac700 1 --2- 172.21.15.27:0/1497022065 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc30c064100 0x7fc30c0665c0 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7fc320106ba0 tx=0x7fc318009040).stop 2021-11-03T18:46:05.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.518+0000 7fc3257ac700 1 -- 172.21.15.27:0/1497022065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3200f51b0 msgr2=0x7fc320107320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:46:05.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.518+0000 7fc3257ac700 1 --2- 172.21.15.27:0/1497022065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3200f51b0 0x7fc320107320 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 rx=0x7fc308004f60 tx=0x7fc308009b30).stop 2021-11-03T18:46:05.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.518+0000 7fc3257ac700 1 -- 172.21.15.27:0/1497022065 shutdown_connections 2021-11-03T18:46:05.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.518+0000 7fc3257ac700 1 --2- 172.21.15.27:0/1497022065 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fc30c064100 0x7fc30c0665c0 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:05.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.518+0000 7fc3257ac700 1 --2- 172.21.15.27:0/1497022065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fc3200f4810 0x7fc320106e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:05.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.518+0000 7fc3257ac700 1 --2- 172.21.15.27:0/1497022065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fc3200f51b0 0x7fc320107320 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:05.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.518+0000 7fc3257ac700 1 -- 172.21.15.27:0/1497022065 >> 172.21.15.27:0/1497022065 conn(0x7fc3200f0330 msgr2=0x7fc3200fb380 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:46:05.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.518+0000 7fc3257ac700 1 -- 172.21.15.27:0/1497022065 shutdown_connections 2021-11-03T18:46:05.523 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.518+0000 7fc3257ac700 1 -- 172.21.15.27:0/1497022065 wait complete. 2021-11-03T18:46:05.620 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.619+0000 7f284215b700 1 -- 172.21.15.27:0/2858651886 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f283c0f7d00 msgr2=0x7f283c0f8120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:46:05.621 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.619+0000 7f284215b700 1 --2- 172.21.15.27:0/2858651886 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f283c0f7d00 0x7f283c0f8120 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 rx=0x7f282c004660 tx=0x7f282c009b30).stop 2021-11-03T18:46:05.621 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.619+0000 7f284215b700 1 -- 172.21.15.27:0/2858651886 shutdown_connections 2021-11-03T18:46:05.621 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.619+0000 7f284215b700 1 --2- 172.21.15.27:0/2858651886 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f283c0f8610 0x7f283c0f4820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:05.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.619+0000 7f284215b700 1 --2- 172.21.15.27:0/2858651886 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f283c0f7d00 0x7f283c0f8120 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:05.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.619+0000 7f284215b700 1 -- 172.21.15.27:0/2858651886 >> 172.21.15.27:0/2858651886 conn(0x7f283c0f0540 msgr2=0x7f283c0f2960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:46:05.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.619+0000 7f284215b700 1 -- 172.21.15.27:0/2858651886 shutdown_connections 2021-11-03T18:46:05.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.619+0000 7f284215b700 1 -- 172.21.15.27:0/2858651886 wait complete. 2021-11-03T18:46:05.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.619+0000 7f284215b700 1 Processor -- start 2021-11-03T18:46:05.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.620+0000 7f284215b700 1 -- start start 2021-11-03T18:46:05.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.620+0000 7f284215b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f283c0f7d00 0x7f283c0f75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:46:05.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.620+0000 7f284215b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f283c0f8610 0x7f283c0f5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:46:05.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.620+0000 7f284215b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283c0f5fb0 con 0x7f283c0f8610 2021-11-03T18:46:05.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.620+0000 7f284215b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f283c0f60f0 con 0x7f283c0f7d00 2021-11-03T18:46:05.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.620+0000 7f283affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f283c0f8610 0x7f283c0f5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:46:05.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.620+0000 7f283affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f283c0f8610 0x7f283c0f5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36758/0 (socket says 172.21.15.27:36758) 2021-11-03T18:46:05.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.620+0000 7f283affd700 1 -- 172.21.15.27:0/2357192798 learned_addr learned my addr 172.21.15.27:0/2357192798 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:46:05.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.620+0000 7f283b7fe700 1 --2- 172.21.15.27:0/2357192798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f283c0f7d00 0x7f283c0f75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:46:05.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.620+0000 7f283affd700 1 -- 172.21.15.27:0/2357192798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f283c0f7d00 msgr2=0x7f283c0f75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:46:05.626 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.620+0000 7f283affd700 1 --2- 172.21.15.27:0/2357192798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f283c0f7d00 0x7f283c0f75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:05.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.620+0000 7f283affd700 1 -- 172.21.15.27:0/2357192798 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f282c005040 con 0x7f283c0f8610 2021-11-03T18:46:05.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.620+0000 7f283affd700 1 --2- 172.21.15.27:0/2357192798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f283c0f8610 0x7f283c0f5a50 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 rx=0x7f283000e7f0 tx=0x7f283000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:46:05.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.620+0000 7f2823fff700 1 -- 172.21.15.27:0/2357192798 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f28300092e0 con 0x7f283c0f8610 2021-11-03T18:46:05.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.621+0000 7f2823fff700 1 -- 172.21.15.27:0/2357192798 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f283000f040 con 0x7f283c0f8610 2021-11-03T18:46:05.627 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.621+0000 7f284215b700 1 -- 172.21.15.27:0/2357192798 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f283c0f63d0 con 0x7f283c0f8610 2021-11-03T18:46:05.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.621+0000 7f2823fff700 1 -- 172.21.15.27:0/2357192798 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2830014750 con 0x7f283c0f8610 2021-11-03T18:46:05.628 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.621+0000 7f284215b700 1 -- 172.21.15.27:0/2357192798 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f283c10ef00 con 0x7f283c0f8610 2021-11-03T18:46:05.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.622+0000 7f284215b700 1 -- 172.21.15.27:0/2357192798 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f283c045ad0 con 0x7f283c0f8610 2021-11-03T18:46:05.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.622+0000 7f2823fff700 1 -- 172.21.15.27:0/2357192798 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f283001b030 con 0x7f283c0f8610 2021-11-03T18:46:05.630 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.622+0000 7f2823fff700 1 --2- 172.21.15.27:0/2357192798 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f28240640b0 0x7f2824066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:46:05.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.622+0000 7f2823fff700 1 -- 172.21.15.27:0/2357192798 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f2830080680 con 0x7f283c0f8610 2021-11-03T18:46:05.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.622+0000 7f283b7fe700 1 --2- 172.21.15.27:0/2357192798 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f28240640b0 0x7f2824066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:46:05.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.623+0000 7f283b7fe700 1 --2- 172.21.15.27:0/2357192798 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f28240640b0 0x7f2824066570 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7f282c004d80 tx=0x7f282c016040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:46:05.631 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.627+0000 7f2823fff700 1 -- 172.21.15.27:0/2357192798 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f28300529a0 con 0x7f283c0f8610 2021-11-03T18:46:05.828 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.828+0000 7f284215b700 1 -- 172.21.15.27:0/2357192798 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f283c027a00 con 0x7f283c0f8610 2021-11-03T18:46:05.828 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.829+0000 7f2823fff700 1 -- 172.21.15.27:0/2357192798 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f2830019710 con 0x7f283c0f8610 2021-11-03T18:46:05.829 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:46:05.829 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:46:05.829 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:46:05.830 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:46:05.830 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:46:05.830 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:46:05.830 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:46:05.831 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:46:05.831 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:46:05.831 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:46:05.831 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:46:05.831 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:46:05.832 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:46:05.832 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:46:05.832 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:46:05.832 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:46:05.832 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:46:05.833 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:46:05.833 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:46:05.833 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:46:05.833 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:46:05.834 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:46:05.834 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:46:05.834 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:46:05.834 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:46:05.835 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:46:05.836 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:46:05.836 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:46:05.836 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:46:05.836 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:46:05.837 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:46:05.837 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:46:05.837 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:46:05.837 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:46:05.838 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:46:05.838 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:46:05.838 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:46:05.839 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:46:05.839 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:46:05.840 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.830+0000 7f284215b700 1 -- 172.21.15.27:0/2357192798 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f28240640b0 msgr2=0x7f2824066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:46:05.840 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.830+0000 7f284215b700 1 --2- 172.21.15.27:0/2357192798 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f28240640b0 0x7f2824066570 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7f282c004d80 tx=0x7f282c016040).stop 2021-11-03T18:46:05.841 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.830+0000 7f284215b700 1 -- 172.21.15.27:0/2357192798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f283c0f8610 msgr2=0x7f283c0f5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:46:05.841 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.830+0000 7f284215b700 1 --2- 172.21.15.27:0/2357192798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f283c0f8610 0x7f283c0f5a50 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 rx=0x7f283000e7f0 tx=0x7f283000b040).stop 2021-11-03T18:46:05.841 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.831+0000 7f284215b700 1 -- 172.21.15.27:0/2357192798 shutdown_connections 2021-11-03T18:46:05.841 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.831+0000 7f284215b700 1 --2- 172.21.15.27:0/2357192798 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f28240640b0 0x7f2824066570 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:05.841 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.831+0000 7f284215b700 1 --2- 172.21.15.27:0/2357192798 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f283c0f7d00 0x7f283c0f75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:05.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.831+0000 7f284215b700 1 --2- 172.21.15.27:0/2357192798 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f283c0f8610 0x7f283c0f5a50 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:05.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.831+0000 7f284215b700 1 -- 172.21.15.27:0/2357192798 >> 172.21.15.27:0/2357192798 conn(0x7f283c0f0540 msgr2=0x7f283c0fb590 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:46:05.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.831+0000 7f284215b700 1 -- 172.21.15.27:0/2357192798 shutdown_connections 2021-11-03T18:46:05.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:05.831+0000 7f284215b700 1 -- 172.21.15.27:0/2357192798 wait complete. 2021-11-03T18:46:05.843 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:46:06.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5407.490170 secs 2021-11-03T18:46:06.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:05 smithi115 ceph-mon[65453]: from='client.36920 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:46:06.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:05 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1497022065' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:06.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:05 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2357192798' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:46:06.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5407.490170 secs 2021-11-03T18:46:06.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:05 smithi027 ceph-mon[97244]: from='client.36920 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:46:06.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:05 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1497022065' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:06.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:05 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2357192798' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:46:07.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:06 smithi115 ceph-mon[65453]: from='client.36924 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:46:07.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:06 smithi115 ceph-mon[65453]: pgmap v2745: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:07.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:06 smithi027 ceph-mon[97244]: from='client.36924 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:46:07.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:06 smithi027 ceph-mon[97244]: pgmap v2745: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:09.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:08 smithi115 ceph-mon[65453]: pgmap v2746: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:09.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:46:09.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:46:09.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:46:09.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:09.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:09.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:09.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:08 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:09.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:08 smithi027 ceph-mon[97244]: pgmap v2746: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:09.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:46:09.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:46:09.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:46:09.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:09.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:09.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:09.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:08 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:10.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:09 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:46:10.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:09 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:46:11.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:10 smithi115 ceph-mon[65453]: pgmap v2747: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:11.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5412.490298 secs 2021-11-03T18:46:11.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:10 smithi027 ceph-mon[97244]: pgmap v2747: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:11.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5412.490298 secs 2021-11-03T18:46:13.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:12 smithi115 ceph-mon[65453]: pgmap v2748: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:13.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:12 smithi027 ceph-mon[97244]: pgmap v2748: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:15.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:14 smithi115 ceph-mon[65453]: pgmap v2749: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:15.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:14 smithi027 ceph-mon[97244]: pgmap v2749: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:16.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5417.490463 secs 2021-11-03T18:46:16.278 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5417.490463 secs 2021-11-03T18:46:17.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:16 smithi115 ceph-mon[65453]: pgmap v2750: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:17.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:16 smithi027 ceph-mon[97244]: pgmap v2750: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:19.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:18 smithi115 ceph-mon[65453]: pgmap v2751: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:19.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:46:19.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:46:19.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:46:19.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:19.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:19.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:19.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:18 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:19.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:18 smithi027 ceph-mon[97244]: pgmap v2751: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:19.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:46:19.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:46:19.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:46:19.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:19.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:19.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:19.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:18 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:20.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:19 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:46:20.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:19 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:46:21.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:20 smithi115 ceph-mon[65453]: pgmap v2752: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:21.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5422.490607 secs 2021-11-03T18:46:21.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:20 smithi027 ceph-mon[97244]: pgmap v2752: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:21.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5422.490607 secs 2021-11-03T18:46:23.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:22 smithi115 ceph-mon[65453]: pgmap v2753: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:23.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:22 smithi027 ceph-mon[97244]: pgmap v2753: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:25.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:24 smithi115 ceph-mon[65453]: pgmap v2754: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:25.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:24 smithi027 ceph-mon[97244]: pgmap v2754: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:26.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5427.490715 secs 2021-11-03T18:46:26.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5427.490715 secs 2021-11-03T18:46:27.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:26 smithi115 ceph-mon[65453]: pgmap v2755: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:27.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:26 smithi027 ceph-mon[97244]: pgmap v2755: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:29.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:28 smithi115 ceph-mon[65453]: pgmap v2756: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:29.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:46:29.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:46:29.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:46:29.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:46:29.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:46:29.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:29.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:29.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:29.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:28 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:29.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:28 smithi027 ceph-mon[97244]: pgmap v2756: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:29.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:46:29.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:46:29.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:46:29.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:46:29.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:46:29.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:29.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:29.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:29.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:28 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:30.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:29 smithi115 ceph-mon[65453]: poll took 40113.882 ms: 1 events 2021-11-03T18:46:30.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:29 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:46:30.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:29 smithi027 ceph-mon[97244]: poll took 40113.882 ms: 1 events 2021-11-03T18:46:30.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:29 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:46:31.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:30 smithi115 ceph-mon[65453]: pgmap v2757: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:31.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5432.490838 secs 2021-11-03T18:46:31.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:30 smithi027 ceph-mon[97244]: pgmap v2757: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:31.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5432.490838 secs 2021-11-03T18:46:33.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:32 smithi115 ceph-mon[65453]: pgmap v2758: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:33.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:32 smithi027 ceph-mon[97244]: pgmap v2758: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:35.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:34 smithi115 ceph-mon[65453]: pgmap v2759: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:35.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:34 smithi027 ceph-mon[97244]: pgmap v2759: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:35.945 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:35.946+0000 7f87e59e7700 1 -- 172.21.15.27:0/1960639167 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87e00f5460 msgr2=0x7f87e00f5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:46:35.946 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:35.946+0000 7f87e59e7700 1 --2- 172.21.15.27:0/1960639167 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87e00f5460 0x7f87e00f5880 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 rx=0x7f87d00099e0 tx=0x7f87d0004d10).stop 2021-11-03T18:46:35.946 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:35.946+0000 7f87e59e7700 1 -- 172.21.15.27:0/1960639167 shutdown_connections 2021-11-03T18:46:35.946 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:35.946+0000 7f87e59e7700 1 --2- 172.21.15.27:0/1960639167 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87e00f6700 0x7f87e00f6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:35.946 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:35.946+0000 7f87e59e7700 1 --2- 172.21.15.27:0/1960639167 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87e00f5460 0x7f87e00f5880 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:35.947 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:35.946+0000 7f87e59e7700 1 -- 172.21.15.27:0/1960639167 >> 172.21.15.27:0/1960639167 conn(0x7f87e00f0bc0 msgr2=0x7f87e00f2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:46:35.947 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:35.946+0000 7f87e59e7700 1 -- 172.21.15.27:0/1960639167 shutdown_connections 2021-11-03T18:46:35.947 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:35.946+0000 7f87e59e7700 1 -- 172.21.15.27:0/1960639167 wait complete. 2021-11-03T18:46:35.948 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:35.947+0000 7f87e59e7700 1 Processor -- start 2021-11-03T18:46:35.949 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:35.947+0000 7f87e59e7700 1 -- start start 2021-11-03T18:46:35.949 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:35.947+0000 7f87e59e7700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87e00f5460 0x7f87e0106ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:46:35.949 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:35.947+0000 7f87e59e7700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87e00f6700 0x7f87e01073d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:46:35.949 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:35.947+0000 7f87e59e7700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e0107980 con 0x7f87e00f6700 2021-11-03T18:46:35.949 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:35.947+0000 7f87e59e7700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e0107ac0 con 0x7f87e00f5460 2021-11-03T18:46:35.950 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:35.947+0000 7f87de7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87e00f6700 0x7f87e01073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:46:35.950 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:35.947+0000 7f87de7fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87e00f6700 0x7f87e01073d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36776/0 (socket says 172.21.15.27:36776) 2021-11-03T18:46:35.950 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:35.947+0000 7f87de7fc700 1 -- 172.21.15.27:0/4247778311 learned_addr learned my addr 172.21.15.27:0/4247778311 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:46:35.950 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:35.947+0000 7f87deffd700 1 --2- 172.21.15.27:0/4247778311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87e00f5460 0x7f87e0106ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:46:35.951 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:35.948+0000 7f87de7fc700 1 -- 172.21.15.27:0/4247778311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87e00f5460 msgr2=0x7f87e0106ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:46:35.951 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:35.948+0000 7f87de7fc700 1 --2- 172.21.15.27:0/4247778311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87e00f5460 0x7f87e0106ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:35.951 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:35.948+0000 7f87de7fc700 1 -- 172.21.15.27:0/4247778311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d0009640 con 0x7f87e00f6700 2021-11-03T18:46:35.951 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:35.948+0000 7f87de7fc700 1 --2- 172.21.15.27:0/4247778311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87e00f6700 0x7f87e01073d0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 rx=0x7f87c800c790 tx=0x7f87c8007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:46:35.953 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:35.949+0000 7f87d77fe700 1 -- 172.21.15.27:0/4247778311 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87c800e680 con 0x7f87e00f6700 2021-11-03T18:46:35.953 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:35.950+0000 7f87d77fe700 1 -- 172.21.15.27:0/4247778311 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f87c800ec90 con 0x7f87e00f6700 2021-11-03T18:46:35.953 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:35.950+0000 7f87e59e7700 1 -- 172.21.15.27:0/4247778311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87e0108460 con 0x7f87e00f6700 2021-11-03T18:46:35.953 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:35.950+0000 7f87d77fe700 1 -- 172.21.15.27:0/4247778311 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87c8013d60 con 0x7f87e00f6700 2021-11-03T18:46:35.954 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:35.950+0000 7f87e59e7700 1 -- 172.21.15.27:0/4247778311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e01089a0 con 0x7f87e00f6700 2021-11-03T18:46:35.954 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:35.951+0000 7f87e59e7700 1 -- 172.21.15.27:0/4247778311 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87e0045ad0 con 0x7f87e00f6700 2021-11-03T18:46:35.954 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:35.951+0000 7f87d77fe700 1 -- 172.21.15.27:0/4247778311 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f87c801a060 con 0x7f87e00f6700 2021-11-03T18:46:35.955 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:35.951+0000 7f87d77fe700 1 --2- 172.21.15.27:0/4247778311 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f87cc064100 0x7f87cc0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:46:35.955 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:35.952+0000 7f87d77fe700 1 -- 172.21.15.27:0/4247778311 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f87c8080b30 con 0x7f87e00f6700 2021-11-03T18:46:35.955 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:35.952+0000 7f87deffd700 1 --2- 172.21.15.27:0/4247778311 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f87cc064100 0x7f87cc0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:46:35.955 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:35.952+0000 7f87deffd700 1 --2- 172.21.15.27:0/4247778311 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f87cc064100 0x7f87cc0665c0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 rx=0x7f87d000b3b0 tx=0x7f87d0004d10).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:46:35.956 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:35.956+0000 7f87d77fe700 1 -- 172.21.15.27:0/4247778311 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f87c8052ef0 con 0x7f87e00f6700 2021-11-03T18:46:36.136 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.136+0000 7f87e59e7700 1 -- 172.21.15.27:0/4247778311 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f87e00fddb0 con 0x7f87cc064100 2021-11-03T18:46:36.137 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.138+0000 7f87d77fe700 1 -- 172.21.15.27:0/4247778311 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f87e00fddb0 con 0x7f87cc064100 2021-11-03T18:46:36.139 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.139+0000 7f87e59e7700 1 -- 172.21.15.27:0/4247778311 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f87cc064100 msgr2=0x7f87cc0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:46:36.139 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.139+0000 7f87e59e7700 1 --2- 172.21.15.27:0/4247778311 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f87cc064100 0x7f87cc0665c0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 rx=0x7f87d000b3b0 tx=0x7f87d0004d10).stop 2021-11-03T18:46:36.139 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.139+0000 7f87e59e7700 1 -- 172.21.15.27:0/4247778311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87e00f6700 msgr2=0x7f87e01073d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:46:36.139 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.139+0000 7f87e59e7700 1 --2- 172.21.15.27:0/4247778311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87e00f6700 0x7f87e01073d0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 rx=0x7f87c800c790 tx=0x7f87c8007d70).stop 2021-11-03T18:46:36.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.139+0000 7f87e59e7700 1 -- 172.21.15.27:0/4247778311 shutdown_connections 2021-11-03T18:46:36.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.140+0000 7f87e59e7700 1 --2- 172.21.15.27:0/4247778311 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f87cc064100 0x7f87cc0665c0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:36.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.140+0000 7f87e59e7700 1 --2- 172.21.15.27:0/4247778311 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f87e00f5460 0x7f87e0106ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:36.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.140+0000 7f87e59e7700 1 --2- 172.21.15.27:0/4247778311 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f87e00f6700 0x7f87e01073d0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:36.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.140+0000 7f87e59e7700 1 -- 172.21.15.27:0/4247778311 >> 172.21.15.27:0/4247778311 conn(0x7f87e00f0bc0 msgr2=0x7f87e005fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:46:36.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.140+0000 7f87e59e7700 1 -- 172.21.15.27:0/4247778311 shutdown_connections 2021-11-03T18:46:36.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.140+0000 7f87e59e7700 1 -- 172.21.15.27:0/4247778311 wait complete. 2021-11-03T18:46:36.148 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:46:36.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5437.490980 secs 2021-11-03T18:46:36.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.240+0000 7f99d3441700 1 -- 172.21.15.27:0/2359075178 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99cc0f66d0 msgr2=0x7f99cc0f6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:46:36.239 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.240+0000 7f99d3441700 1 --2- 172.21.15.27:0/2359075178 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99cc0f66d0 0x7f99cc0f6af0 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 rx=0x7f99c8004660 tx=0x7f99c8009b30).stop 2021-11-03T18:46:36.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.240+0000 7f99d3441700 1 -- 172.21.15.27:0/2359075178 shutdown_connections 2021-11-03T18:46:36.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.240+0000 7f99d3441700 1 --2- 172.21.15.27:0/2359075178 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99cc0f7970 0x7f99cc0f7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:36.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.240+0000 7f99d3441700 1 --2- 172.21.15.27:0/2359075178 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99cc0f66d0 0x7f99cc0f6af0 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:36.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.240+0000 7f99d3441700 1 -- 172.21.15.27:0/2359075178 >> 172.21.15.27:0/2359075178 conn(0x7f99cc0f1e50 msgr2=0x7f99cc0f4270 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:46:36.240 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.240+0000 7f99d3441700 1 -- 172.21.15.27:0/2359075178 shutdown_connections 2021-11-03T18:46:36.241 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.240+0000 7f99d3441700 1 -- 172.21.15.27:0/2359075178 wait complete. 2021-11-03T18:46:36.243 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.240+0000 7f99d3441700 1 Processor -- start 2021-11-03T18:46:36.243 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.240+0000 7f99d3441700 1 -- start start 2021-11-03T18:46:36.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.241+0000 7f99d3441700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99cc0f66d0 0x7f99cc106f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:46:36.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.241+0000 7f99d3441700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99cc0f7970 0x7f99cc1073f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:46:36.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.241+0000 7f99d3441700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc1079a0 con 0x7f99cc0f66d0 2021-11-03T18:46:36.244 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.241+0000 7f99d3441700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc107ae0 con 0x7f99cc0f7970 2021-11-03T18:46:36.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.241+0000 7f99d11dd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99cc0f66d0 0x7f99cc106f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:46:36.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.241+0000 7f99d11dd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99cc0f66d0 0x7f99cc106f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36784/0 (socket says 172.21.15.27:36784) 2021-11-03T18:46:36.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.241+0000 7f99d11dd700 1 -- 172.21.15.27:0/3100504073 learned_addr learned my addr 172.21.15.27:0/3100504073 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:46:36.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.241+0000 7f99d09dc700 1 --2- 172.21.15.27:0/3100504073 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99cc0f7970 0x7f99cc1073f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:46:36.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.241+0000 7f99d11dd700 1 -- 172.21.15.27:0/3100504073 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99cc0f7970 msgr2=0x7f99cc1073f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:46:36.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.241+0000 7f99d11dd700 1 --2- 172.21.15.27:0/3100504073 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99cc0f7970 0x7f99cc1073f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:36.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.241+0000 7f99d11dd700 1 -- 172.21.15.27:0/3100504073 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99c8005040 con 0x7f99cc0f66d0 2021-11-03T18:46:36.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.241+0000 7f99d09dc700 1 --2- 172.21.15.27:0/3100504073 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99cc0f7970 0x7f99cc1073f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:46:36.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.241+0000 7f99d11dd700 1 --2- 172.21.15.27:0/3100504073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99cc0f66d0 0x7f99cc106f00 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 rx=0x7f99c8014040 tx=0x7f99c800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:46:36.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.241+0000 7f99c1ffb700 1 -- 172.21.15.27:0/3100504073 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99c8019070 con 0x7f99cc0f66d0 2021-11-03T18:46:36.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.241+0000 7f99c1ffb700 1 -- 172.21.15.27:0/3100504073 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f99c8007e70 con 0x7f99cc0f66d0 2021-11-03T18:46:36.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.241+0000 7f99d3441700 1 -- 172.21.15.27:0/3100504073 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99cc108420 con 0x7f99cc0f66d0 2021-11-03T18:46:36.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.241+0000 7f99c1ffb700 1 -- 172.21.15.27:0/3100504073 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99c8014d50 con 0x7f99cc0f66d0 2021-11-03T18:46:36.248 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.242+0000 7f99d3441700 1 -- 172.21.15.27:0/3100504073 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99cc108980 con 0x7f99cc0f66d0 2021-11-03T18:46:36.248 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.242+0000 7f99d3441700 1 -- 172.21.15.27:0/3100504073 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99cc045ad0 con 0x7f99cc0f66d0 2021-11-03T18:46:36.248 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.243+0000 7f99c1ffb700 1 -- 172.21.15.27:0/3100504073 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f99c8025070 con 0x7f99cc0f66d0 2021-11-03T18:46:36.248 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.243+0000 7f99c1ffb700 1 --2- 172.21.15.27:0/3100504073 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f99b80640b0 0x7f99b8066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:46:36.249 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.243+0000 7f99c1ffb700 1 -- 172.21.15.27:0/3100504073 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f99c8081710 con 0x7f99cc0f66d0 2021-11-03T18:46:36.249 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.243+0000 7f99d09dc700 1 --2- 172.21.15.27:0/3100504073 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f99b80640b0 0x7f99b8066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:46:36.249 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.244+0000 7f99d09dc700 1 --2- 172.21.15.27:0/3100504073 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f99b80640b0 0x7f99b8066570 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7f99bc007fc0 tx=0x7f99bc00b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:46:36.249 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.247+0000 7f99c1ffb700 1 -- 172.21.15.27:0/3100504073 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f99c8053a70 con 0x7f99cc0f66d0 2021-11-03T18:46:36.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5437.490980 secs 2021-11-03T18:46:36.419 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.419+0000 7f99d3441700 1 -- 172.21.15.27:0/3100504073 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f99cc0fefd0 con 0x7f99b80640b0 2021-11-03T18:46:36.434 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.434+0000 7f99c1ffb700 1 -- 172.21.15.27:0/3100504073 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7f99cc0fefd0 con 0x7f99b80640b0 2021-11-03T18:46:36.434 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:46:36.434 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (98m) 6m ago 99m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:46:36.435 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (89m) 6m ago 99m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:46:36.435 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (89m) 6m ago 98m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:46:36.435 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (97m) 6m ago 99m 42.9M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:46:36.435 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (91m) 6m ago 94m 26.9M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:46:36.436 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (94m) 6m ago 94m 3852M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:46:36.436 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (90m) 6m ago 94m 26.5M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:46:36.436 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (92m) 6m ago 94m 27.6M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:46:36.436 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (91m) 6m ago 100m 625M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:46:36.436 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (91m) 6m ago 98m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:46:36.437 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (90m) 6m ago 100m 387M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:46:36.437 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (90m) 6m ago 98m 175M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:46:36.437 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (99m) 6m ago 99m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:46:36.437 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (98m) 6m ago 98m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:46:36.438 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (89m) 6m ago 97m 433M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:46:36.438 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (88m) 6m ago 97m 471M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:46:36.438 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (88m) 6m ago 97m 356M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:46:36.438 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (88m) 6m ago 96m 504M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:46:36.438 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (87m) 6m ago 96m 396M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:46:36.439 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (87m) 6m ago 96m 408M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:46:36.439 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (97m) 6m ago 98m 66.8M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:46:36.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.436+0000 7f99d3441700 1 -- 172.21.15.27:0/3100504073 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f99b80640b0 msgr2=0x7f99b8066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:46:36.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.436+0000 7f99d3441700 1 --2- 172.21.15.27:0/3100504073 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f99b80640b0 0x7f99b8066570 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 rx=0x7f99bc007fc0 tx=0x7f99bc00b040).stop 2021-11-03T18:46:36.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.436+0000 7f99d3441700 1 -- 172.21.15.27:0/3100504073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99cc0f66d0 msgr2=0x7f99cc106f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:46:36.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.436+0000 7f99d3441700 1 --2- 172.21.15.27:0/3100504073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99cc0f66d0 0x7f99cc106f00 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 rx=0x7f99c8014040 tx=0x7f99c800b040).stop 2021-11-03T18:46:36.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.436+0000 7f99d3441700 1 -- 172.21.15.27:0/3100504073 shutdown_connections 2021-11-03T18:46:36.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.436+0000 7f99d3441700 1 --2- 172.21.15.27:0/3100504073 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f99b80640b0 0x7f99b8066570 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:36.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.436+0000 7f99d3441700 1 --2- 172.21.15.27:0/3100504073 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f99cc0f66d0 0x7f99cc106f00 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:36.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.436+0000 7f99d3441700 1 --2- 172.21.15.27:0/3100504073 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f99cc0f7970 0x7f99cc1073f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:36.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.436+0000 7f99d3441700 1 -- 172.21.15.27:0/3100504073 >> 172.21.15.27:0/3100504073 conn(0x7f99cc0f1e50 msgr2=0x7f99cc0fb3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:46:36.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.436+0000 7f99d3441700 1 -- 172.21.15.27:0/3100504073 shutdown_connections 2021-11-03T18:46:36.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.436+0000 7f99d3441700 1 -- 172.21.15.27:0/3100504073 wait complete. 2021-11-03T18:46:36.536 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.535+0000 7faf2f59e700 1 -- 172.21.15.27:0/291419693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf300f4800 msgr2=0x7faf300f4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:46:36.537 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.535+0000 7faf2f59e700 1 --2- 172.21.15.27:0/291419693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf300f4800 0x7faf300f4c20 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 rx=0x7faf20004660 tx=0x7faf20009b30).stop 2021-11-03T18:46:36.537 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.535+0000 7faf2f59e700 1 -- 172.21.15.27:0/291419693 shutdown_connections 2021-11-03T18:46:36.537 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.535+0000 7faf2f59e700 1 --2- 172.21.15.27:0/291419693 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf300f51a0 0x7faf300f7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:36.537 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.535+0000 7faf2f59e700 1 --2- 172.21.15.27:0/291419693 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf300f4800 0x7faf300f4c20 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:36.538 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.535+0000 7faf2f59e700 1 -- 172.21.15.27:0/291419693 >> 172.21.15.27:0/291419693 conn(0x7faf300f0320 msgr2=0x7faf300f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:46:36.538 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.536+0000 7faf2f59e700 1 -- 172.21.15.27:0/291419693 shutdown_connections 2021-11-03T18:46:36.538 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.536+0000 7faf2f59e700 1 -- 172.21.15.27:0/291419693 wait complete. 2021-11-03T18:46:36.538 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.536+0000 7faf2f59e700 1 Processor -- start 2021-11-03T18:46:36.539 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.536+0000 7faf2f59e700 1 -- start start 2021-11-03T18:46:36.539 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.536+0000 7faf2f59e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf300f4800 0x7faf30106dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:46:36.539 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.536+0000 7faf2f59e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf300f51a0 0x7faf301072b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:46:36.539 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.536+0000 7faf2f59e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf301078b0 con 0x7faf300f4800 2021-11-03T18:46:36.539 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.536+0000 7faf2f59e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf301080f0 con 0x7faf300f51a0 2021-11-03T18:46:36.540 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.536+0000 7faf2e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf300f4800 0x7faf30106dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:46:36.540 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.536+0000 7faf2e59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf300f4800 0x7faf30106dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36794/0 (socket says 172.21.15.27:36794) 2021-11-03T18:46:36.540 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.536+0000 7faf2e59c700 1 -- 172.21.15.27:0/309414188 learned_addr learned my addr 172.21.15.27:0/309414188 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:46:36.540 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.537+0000 7faf2dd9b700 1 --2- 172.21.15.27:0/309414188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf300f51a0 0x7faf301072b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:46:36.541 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.537+0000 7faf2e59c700 1 -- 172.21.15.27:0/309414188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf300f51a0 msgr2=0x7faf301072b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:46:36.541 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.537+0000 7faf2e59c700 1 --2- 172.21.15.27:0/309414188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf300f51a0 0x7faf301072b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:36.541 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.537+0000 7faf2e59c700 1 -- 172.21.15.27:0/309414188 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf20005040 con 0x7faf300f4800 2021-11-03T18:46:36.541 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.537+0000 7faf2e59c700 1 --2- 172.21.15.27:0/309414188 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf300f4800 0x7faf30106dc0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 rx=0x7faf20013a70 tx=0x7faf20009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:46:36.541 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.537+0000 7faf1effd700 1 -- 172.21.15.27:0/309414188 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf20018070 con 0x7faf300f4800 2021-11-03T18:46:36.542 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.537+0000 7faf1effd700 1 -- 172.21.15.27:0/309414188 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7faf200097d0 con 0x7faf300f4800 2021-11-03T18:46:36.542 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.537+0000 7faf2f59e700 1 -- 172.21.15.27:0/309414188 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf30108370 con 0x7faf300f4800 2021-11-03T18:46:36.542 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.537+0000 7faf2f59e700 1 -- 172.21.15.27:0/309414188 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf30108850 con 0x7faf300f4800 2021-11-03T18:46:36.542 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.537+0000 7faf1effd700 1 -- 172.21.15.27:0/309414188 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf2001c3c0 con 0x7faf300f4800 2021-11-03T18:46:36.544 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.538+0000 7faf2f59e700 1 -- 172.21.15.27:0/309414188 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf30059100 con 0x7faf300f4800 2021-11-03T18:46:36.544 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.538+0000 7faf1effd700 1 -- 172.21.15.27:0/309414188 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7faf20007ba0 con 0x7faf300f4800 2021-11-03T18:46:36.544 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.539+0000 7faf1effd700 1 --2- 172.21.15.27:0/309414188 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7faf18064100 0x7faf180665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:46:36.544 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.539+0000 7faf1effd700 1 -- 172.21.15.27:0/309414188 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7faf20082470 con 0x7faf300f4800 2021-11-03T18:46:36.545 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.539+0000 7faf2dd9b700 1 --2- 172.21.15.27:0/309414188 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7faf18064100 0x7faf180665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:46:36.545 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.539+0000 7faf2dd9b700 1 --2- 172.21.15.27:0/309414188 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7faf18064100 0x7faf180665c0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7faf30106b50 tx=0x7faf2400b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:46:36.545 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.543+0000 7faf1effd700 1 -- 172.21.15.27:0/309414188 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7faf200547d0 con 0x7faf300f4800 2021-11-03T18:46:36.771 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.771+0000 7faf2f59e700 1 -- 172.21.15.27:0/309414188 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7faf300fede0 con 0x7faf300f4800 2021-11-03T18:46:36.772 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.772+0000 7faf1effd700 1 -- 172.21.15.27:0/309414188 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7faf200543c0 con 0x7faf300f4800 2021-11-03T18:46:36.772 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:46:36.772 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:46:36.773 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:46:36.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:46:36.773 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:46:36.773 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:46:36.773 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:46:36.774 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:46:36.774 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:46:36.774 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:46:36.774 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:46:36.775 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:46:36.775 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:46:36.775 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:46:36.775 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:46:36.776 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:46:36.776 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:46:36.776 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:46:36.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.774+0000 7faf2f59e700 1 -- 172.21.15.27:0/309414188 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7faf18064100 msgr2=0x7faf180665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:46:36.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.774+0000 7faf2f59e700 1 --2- 172.21.15.27:0/309414188 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7faf18064100 0x7faf180665c0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 rx=0x7faf30106b50 tx=0x7faf2400b040).stop 2021-11-03T18:46:36.777 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.774+0000 7faf2f59e700 1 -- 172.21.15.27:0/309414188 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf300f4800 msgr2=0x7faf30106dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:46:36.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.774+0000 7faf2f59e700 1 --2- 172.21.15.27:0/309414188 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf300f4800 0x7faf30106dc0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 rx=0x7faf20013a70 tx=0x7faf20009b30).stop 2021-11-03T18:46:36.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.774+0000 7faf2f59e700 1 -- 172.21.15.27:0/309414188 shutdown_connections 2021-11-03T18:46:36.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.774+0000 7faf2f59e700 1 --2- 172.21.15.27:0/309414188 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7faf18064100 0x7faf180665c0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:36.778 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.774+0000 7faf2f59e700 1 --2- 172.21.15.27:0/309414188 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7faf300f4800 0x7faf30106dc0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:36.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.774+0000 7faf2f59e700 1 --2- 172.21.15.27:0/309414188 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7faf300f51a0 0x7faf301072b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:36.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.774+0000 7faf2f59e700 1 -- 172.21.15.27:0/309414188 >> 172.21.15.27:0/309414188 conn(0x7faf300f0320 msgr2=0x7faf300fb370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:46:36.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.774+0000 7faf2f59e700 1 -- 172.21.15.27:0/309414188 shutdown_connections 2021-11-03T18:46:36.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.774+0000 7faf2f59e700 1 -- 172.21.15.27:0/309414188 wait complete. 2021-11-03T18:46:36.876 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.874+0000 7ff2b09bb700 1 -- 172.21.15.27:0/3741715892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2ac0f96d0 msgr2=0x7ff2ac0fbb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:46:36.876 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.874+0000 7ff2b09bb700 1 --2- 172.21.15.27:0/3741715892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2ac0f96d0 0x7ff2ac0fbb00 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 rx=0x7ff29c004660 tx=0x7ff29c009b30).stop 2021-11-03T18:46:36.876 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.874+0000 7ff2b09bb700 1 -- 172.21.15.27:0/3741715892 shutdown_connections 2021-11-03T18:46:36.876 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.874+0000 7ff2b09bb700 1 --2- 172.21.15.27:0/3741715892 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2ac0f96d0 0x7ff2ac0fbb00 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:36.877 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.874+0000 7ff2b09bb700 1 --2- 172.21.15.27:0/3741715892 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2ac060250 0x7ff2ac0f91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:36.877 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.874+0000 7ff2b09bb700 1 -- 172.21.15.27:0/3741715892 >> 172.21.15.27:0/3741715892 conn(0x7ff2ac0f0ba0 msgr2=0x7ff2ac0f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:46:36.877 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.874+0000 7ff2b09bb700 1 -- 172.21.15.27:0/3741715892 shutdown_connections 2021-11-03T18:46:36.877 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.874+0000 7ff2b09bb700 1 -- 172.21.15.27:0/3741715892 wait complete. 2021-11-03T18:46:36.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.875+0000 7ff2b09bb700 1 Processor -- start 2021-11-03T18:46:36.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.875+0000 7ff2b09bb700 1 -- start start 2021-11-03T18:46:36.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.875+0000 7ff2b09bb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2ac060250 0x7ff2ac0f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:46:36.878 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.875+0000 7ff2b09bb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2ac0f96d0 0x7ff2ac0f50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:46:36.879 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.875+0000 7ff2b09bb700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac0f5640 con 0x7ff2ac0f96d0 2021-11-03T18:46:36.879 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.875+0000 7ff2b09bb700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2ac0f5780 con 0x7ff2ac060250 2021-11-03T18:46:36.879 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.875+0000 7ff2aaffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2ac0f96d0 0x7ff2ac0f50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:46:36.879 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.875+0000 7ff2aaffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2ac0f96d0 0x7ff2ac0f50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36806/0 (socket says 172.21.15.27:36806) 2021-11-03T18:46:36.880 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.875+0000 7ff2aaffd700 1 -- 172.21.15.27:0/2345253094 learned_addr learned my addr 172.21.15.27:0/2345253094 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:46:36.880 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.876+0000 7ff2ab7fe700 1 --2- 172.21.15.27:0/2345253094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2ac060250 0x7ff2ac0f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:46:36.880 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.876+0000 7ff2aaffd700 1 -- 172.21.15.27:0/2345253094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2ac060250 msgr2=0x7ff2ac0f6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:46:36.880 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.876+0000 7ff2aaffd700 1 --2- 172.21.15.27:0/2345253094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2ac060250 0x7ff2ac0f6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:36.880 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.876+0000 7ff2aaffd700 1 -- 172.21.15.27:0/2345253094 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff29c005040 con 0x7ff2ac0f96d0 2021-11-03T18:46:36.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.876+0000 7ff2aaffd700 1 --2- 172.21.15.27:0/2345253094 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2ac0f96d0 0x7ff2ac0f50e0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 rx=0x7ff29c000c00 tx=0x7ff29c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:46:36.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.876+0000 7ff2a3fff700 1 -- 172.21.15.27:0/2345253094 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff29c018070 con 0x7ff2ac0f96d0 2021-11-03T18:46:36.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.876+0000 7ff2a3fff700 1 -- 172.21.15.27:0/2345253094 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7ff29c0097d0 con 0x7ff2ac0f96d0 2021-11-03T18:46:36.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.876+0000 7ff2b09bb700 1 -- 172.21.15.27:0/2345253094 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2ac0f5a00 con 0x7ff2ac0f96d0 2021-11-03T18:46:36.882 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.876+0000 7ff2a3fff700 1 -- 172.21.15.27:0/2345253094 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff29c01de40 con 0x7ff2ac0f96d0 2021-11-03T18:46:36.882 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.876+0000 7ff2b09bb700 1 -- 172.21.15.27:0/2345253094 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2ac0f5f60 con 0x7ff2ac0f96d0 2021-11-03T18:46:36.883 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.877+0000 7ff2b09bb700 1 -- 172.21.15.27:0/2345253094 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2ac045ad0 con 0x7ff2ac0f96d0 2021-11-03T18:46:36.883 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.878+0000 7ff2a3fff700 1 -- 172.21.15.27:0/2345253094 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7ff29c01d420 con 0x7ff2ac0f96d0 2021-11-03T18:46:36.884 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.878+0000 7ff2a3fff700 1 --2- 172.21.15.27:0/2345253094 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff298064100 0x7ff2980665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:46:36.884 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.878+0000 7ff2a3fff700 1 -- 172.21.15.27:0/2345253094 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7ff29c0824b0 con 0x7ff2ac0f96d0 2021-11-03T18:46:36.884 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.878+0000 7ff2ab7fe700 1 --2- 172.21.15.27:0/2345253094 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff298064100 0x7ff2980665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:46:36.884 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.879+0000 7ff2ab7fe700 1 --2- 172.21.15.27:0/2345253094 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff298064100 0x7ff2980665c0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0x7ff2ac0f69f0 tx=0x7ff29400b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:46:36.885 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:36.882+0000 7ff2a3fff700 1 -- 172.21.15.27:0/2345253094 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7ff29c0548d0 con 0x7ff2ac0f96d0 2021-11-03T18:46:37.090 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:37.091+0000 7ff2b09bb700 1 -- 172.21.15.27:0/2345253094 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7ff2ac027a00 con 0x7ff2ac0f96d0 2021-11-03T18:46:37.091 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:37.091+0000 7ff2a3fff700 1 -- 172.21.15.27:0/2345253094 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7ff29c0544c0 con 0x7ff2ac0f96d0 2021-11-03T18:46:37.091 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:46:37.092 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:46:37.092 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:46:37.092 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:46:37.092 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:46:37.093 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:46:37.093 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:46:37.093 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:46:37.093 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:46:37.094 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:46:37.094 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:46:37.094 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:46:37.094 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:46:37.094 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:46:37.095 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:46:37.095 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:46:37.095 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:46:37.095 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:46:37.096 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:46:37.096 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:46:37.096 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:46:37.096 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:46:37.097 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:46:37.097 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:46:37.097 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:46:37.097 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:46:37.097 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:46:37.098 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:46:37.098 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:46:37.098 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:46:37.098 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:46:37.099 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:46:37.099 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:46:37.099 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:46:37.099 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:46:37.100 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:46:37.100 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:46:37.100 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:46:37.100 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:46:37.101 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:37.093+0000 7ff2b09bb700 1 -- 172.21.15.27:0/2345253094 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff298064100 msgr2=0x7ff2980665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:46:37.101 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:37.093+0000 7ff2b09bb700 1 --2- 172.21.15.27:0/2345253094 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff298064100 0x7ff2980665c0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 rx=0x7ff2ac0f69f0 tx=0x7ff29400b040).stop 2021-11-03T18:46:37.102 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:37.093+0000 7ff2b09bb700 1 -- 172.21.15.27:0/2345253094 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2ac0f96d0 msgr2=0x7ff2ac0f50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:46:37.102 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:37.093+0000 7ff2b09bb700 1 --2- 172.21.15.27:0/2345253094 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2ac0f96d0 0x7ff2ac0f50e0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 rx=0x7ff29c000c00 tx=0x7ff29c009b30).stop 2021-11-03T18:46:37.102 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:37.093+0000 7ff2b09bb700 1 -- 172.21.15.27:0/2345253094 shutdown_connections 2021-11-03T18:46:37.102 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:37.093+0000 7ff2b09bb700 1 --2- 172.21.15.27:0/2345253094 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff298064100 0x7ff2980665c0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:37.103 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:37.093+0000 7ff2b09bb700 1 --2- 172.21.15.27:0/2345253094 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff2ac060250 0x7ff2ac0f6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:37.103 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:37.093+0000 7ff2b09bb700 1 --2- 172.21.15.27:0/2345253094 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff2ac0f96d0 0x7ff2ac0f50e0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:46:37.103 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:37.093+0000 7ff2b09bb700 1 -- 172.21.15.27:0/2345253094 >> 172.21.15.27:0/2345253094 conn(0x7ff2ac0f0ba0 msgr2=0x7ff2ac0f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:46:37.103 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:37.093+0000 7ff2b09bb700 1 -- 172.21.15.27:0/2345253094 shutdown_connections 2021-11-03T18:46:37.104 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:46:37.093+0000 7ff2b09bb700 1 -- 172.21.15.27:0/2345253094 wait complete. 2021-11-03T18:46:37.104 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:46:37.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:36 smithi115 ceph-mon[65453]: pgmap v2760: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:37.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:36 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/309414188' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:37.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:36 smithi027 ceph-mon[97244]: pgmap v2760: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:37.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:36 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/309414188' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:38.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:37 smithi115 ceph-mon[65453]: from='client.36936 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:46:38.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:37 smithi115 ceph-mon[65453]: from='client.36940 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:46:38.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:37 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2345253094' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:46:38.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:37 smithi027 ceph-mon[97244]: from='client.36936 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:46:38.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:37 smithi027 ceph-mon[97244]: from='client.36940 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:46:38.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:37 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2345253094' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:46:39.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:38 smithi115 ceph-mon[65453]: pgmap v2761: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:39.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:46:39.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"}]: dispatch 2021-11-03T18:46:39.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2021-11-03T18:46:39.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:46:39.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:46:39.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:39.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:39.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:39.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:38 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:39.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:38 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:46:39.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:38 smithi027 ceph-mon[97244]: pgmap v2761: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:39.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:46:39.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi027", "name": "osd_memory_target"}]: dispatch 2021-11-03T18:46:39.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config rm", "who": "osd/host:smithi115", "name": "osd_memory_target"}]: dispatch 2021-11-03T18:46:39.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:46:39.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:46:39.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:39.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:39.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:39.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:38 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:39.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:38 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:46:40.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:39 smithi115 ceph-mon[65453]: pgmap v2762: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:40.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:39 smithi027 ceph-mon[97244]: pgmap v2762: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:41.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5442.491119 secs 2021-11-03T18:46:41.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5442.491119 secs 2021-11-03T18:46:42.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:41 smithi115 ceph-mon[65453]: pgmap v2763: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:42.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:41 smithi027 ceph-mon[97244]: pgmap v2763: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:44.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:44 smithi115 ceph-mon[65453]: pgmap v2764: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:44.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:44 smithi027 ceph-mon[97244]: pgmap v2764: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:45.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5447.491261 secs 2021-11-03T18:46:45.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5447.491261 secs 2021-11-03T18:46:46.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:46 smithi115 ceph-mon[65453]: pgmap v2765: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:47.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:46 smithi027 ceph-mon[97244]: pgmap v2765: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:48.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:48 smithi115 ceph-mon[65453]: pgmap v2766: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:48.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:48 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:46:48.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:48 smithi027 ceph-mon[97244]: pgmap v2766: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:48.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:48 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:46:50.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:49 smithi115 ceph-mon[65453]: poll took 20068.873 ms: 1 events 2021-11-03T18:46:50.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:46:50.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:46:50.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:50.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:50.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:50.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:50.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:49 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:46:50.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:49 smithi027 ceph-mon[97244]: poll took 20068.873 ms: 1 events 2021-11-03T18:46:50.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:46:50.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:46:50.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:50.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:50.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:50.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:46:50.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:49 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:46:51.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:50 smithi115 ceph-mon[65453]: pgmap v2767: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:51.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5452.491394 secs 2021-11-03T18:46:51.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:50 smithi027 ceph-mon[97244]: pgmap v2767: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:51.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5452.491394 secs 2021-11-03T18:46:53.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:52 smithi115 ceph-mon[65453]: pgmap v2768: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:53.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:52 smithi027 ceph-mon[97244]: pgmap v2768: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:55.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:54 smithi115 ceph-mon[65453]: pgmap v2769: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:55.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:54 smithi027 ceph-mon[97244]: pgmap v2769: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:56.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5457.491536 secs 2021-11-03T18:46:56.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:46:56.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:46:56.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5457.491536 secs 2021-11-03T18:46:56.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:46:56.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:46:57.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:56 smithi115 ceph-mon[65453]: pgmap v2770: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:57.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:56 smithi027 ceph-mon[97244]: pgmap v2770: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:59.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:58 smithi115 ceph-mon[65453]: pgmap v2771: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:46:59.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:58 smithi027 ceph-mon[97244]: pgmap v2771: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:00.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:47:00.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:47:00.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:47:00.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:00.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:00.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:00.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:00.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:46:59 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:47:00.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:47:00.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:47:00.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:47:00.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:00.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:00.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:00.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:00.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:46:59 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:47:01.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:00 smithi115 ceph-mon[65453]: pgmap v2772: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:01.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5462.491684 secs 2021-11-03T18:47:01.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:00 smithi027 ceph-mon[97244]: pgmap v2772: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:01.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5462.491684 secs 2021-11-03T18:47:03.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:02 smithi115 ceph-mon[65453]: pgmap v2773: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:03.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:02 smithi027 ceph-mon[97244]: pgmap v2773: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:05.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:04 smithi115 ceph-mon[65453]: pgmap v2774: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:05.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:04 smithi027 ceph-mon[97244]: pgmap v2774: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:06.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5467.491838 secs 2021-11-03T18:47:06.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5467.491838 secs 2021-11-03T18:47:07.208 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.208+0000 7f5fe2dab700 1 -- 172.21.15.27:0/2513122106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5fdc0f5ef0 msgr2=0x7f5fdc0f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:47:07.209 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.208+0000 7f5fe2dab700 1 --2- 172.21.15.27:0/2513122106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5fdc0f5ef0 0x7f5fdc0f6920 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 rx=0x7f5fd0004660 tx=0x7f5fd0009b30).stop 2021-11-03T18:47:07.209 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.208+0000 7f5fe2dab700 1 -- 172.21.15.27:0/2513122106 shutdown_connections 2021-11-03T18:47:07.209 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.208+0000 7f5fe2dab700 1 --2- 172.21.15.27:0/2513122106 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5fdc0f5ef0 0x7f5fdc0f6920 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:07.209 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.208+0000 7f5fe2dab700 1 --2- 172.21.15.27:0/2513122106 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5fdc0f55e0 0x7f5fdc0f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:07.209 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.208+0000 7f5fe2dab700 1 -- 172.21.15.27:0/2513122106 >> 172.21.15.27:0/2513122106 conn(0x7f5fdc0f0de0 msgr2=0x7f5fdc0f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:47:07.210 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.208+0000 7f5fe2dab700 1 -- 172.21.15.27:0/2513122106 shutdown_connections 2021-11-03T18:47:07.210 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.208+0000 7f5fe2dab700 1 -- 172.21.15.27:0/2513122106 wait complete. 2021-11-03T18:47:07.210 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.209+0000 7f5fe2dab700 1 Processor -- start 2021-11-03T18:47:07.210 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.209+0000 7f5fe2dab700 1 -- start start 2021-11-03T18:47:07.214 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.209+0000 7f5fe2dab700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5fdc0f55e0 0x7f5fdc109150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:47:07.214 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.209+0000 7f5fe2dab700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5fdc0f5ef0 0x7f5fdc109640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:47:07.215 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.209+0000 7f5fe2dab700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fdc109bf0 con 0x7f5fdc0f5ef0 2021-11-03T18:47:07.215 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.209+0000 7f5fe2dab700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fdc109d30 con 0x7f5fdc0f55e0 2021-11-03T18:47:07.216 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.209+0000 7f5fdbfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5fdc0f5ef0 0x7f5fdc109640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:47:07.216 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.210+0000 7f5fdbfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5fdc0f5ef0 0x7f5fdc109640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36824/0 (socket says 172.21.15.27:36824) 2021-11-03T18:47:07.217 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.210+0000 7f5fdbfff700 1 -- 172.21.15.27:0/503222803 learned_addr learned my addr 172.21.15.27:0/503222803 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:47:07.217 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.210+0000 7f5fe0b47700 1 --2- 172.21.15.27:0/503222803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5fdc0f55e0 0x7f5fdc109150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:47:07.217 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.210+0000 7f5fdbfff700 1 -- 172.21.15.27:0/503222803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5fdc0f55e0 msgr2=0x7f5fdc109150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:47:07.218 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.210+0000 7f5fdbfff700 1 --2- 172.21.15.27:0/503222803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5fdc0f55e0 0x7f5fdc109150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:07.218 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.210+0000 7f5fdbfff700 1 -- 172.21.15.27:0/503222803 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd0005040 con 0x7f5fdc0f5ef0 2021-11-03T18:47:07.219 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.210+0000 7f5fdbfff700 1 --2- 172.21.15.27:0/503222803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5fdc0f5ef0 0x7f5fdc109640 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 rx=0x7f5fd0009990 tx=0x7f5fd0009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:47:07.219 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.210+0000 7f5fd97fa700 1 -- 172.21.15.27:0/503222803 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5fd0018070 con 0x7f5fdc0f5ef0 2021-11-03T18:47:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.210+0000 7f5fd97fa700 1 -- 172.21.15.27:0/503222803 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f5fd0007ca0 con 0x7f5fdc0f5ef0 2021-11-03T18:47:07.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.210+0000 7f5fe2dab700 1 -- 172.21.15.27:0/503222803 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fdc10a670 con 0x7f5fdc0f5ef0 2021-11-03T18:47:07.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.210+0000 7f5fd97fa700 1 -- 172.21.15.27:0/503222803 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5fd001c620 con 0x7f5fdc0f5ef0 2021-11-03T18:47:07.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.210+0000 7f5fe2dab700 1 -- 172.21.15.27:0/503222803 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fdc10abd0 con 0x7f5fdc0f5ef0 2021-11-03T18:47:07.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.211+0000 7f5fe2dab700 1 -- 172.21.15.27:0/503222803 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fdc045ad0 con 0x7f5fdc0f5ef0 2021-11-03T18:47:07.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.212+0000 7f5fd97fa700 1 -- 172.21.15.27:0/503222803 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f5fd001d960 con 0x7f5fdc0f5ef0 2021-11-03T18:47:07.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.212+0000 7f5fd97fa700 1 --2- 172.21.15.27:0/503222803 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5fc40640b0 0x7f5fc4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:47:07.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.212+0000 7f5fd97fa700 1 -- 172.21.15.27:0/503222803 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f5fd0082520 con 0x7f5fdc0f5ef0 2021-11-03T18:47:07.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.212+0000 7f5fe0b47700 1 --2- 172.21.15.27:0/503222803 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5fc40640b0 0x7f5fc4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:47:07.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.213+0000 7f5fe0b47700 1 --2- 172.21.15.27:0/503222803 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5fc40640b0 0x7f5fc4066570 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7f5fd4004550 tx=0x7f5fd4009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:47:07.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.216+0000 7f5fd97fa700 1 -- 172.21.15.27:0/503222803 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f5fd0054940 con 0x7f5fdc0f5ef0 2021-11-03T18:47:07.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:06 smithi115 ceph-mon[65453]: pgmap v2775: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:07.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:06 smithi027 ceph-mon[97244]: pgmap v2775: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:07.396 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.396+0000 7f5fe2dab700 1 -- 172.21.15.27:0/503222803 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5fdc0fab60 con 0x7f5fc40640b0 2021-11-03T18:47:07.397 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.398+0000 7f5fd97fa700 1 -- 172.21.15.27:0/503222803 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f5fdc0fab60 con 0x7f5fc40640b0 2021-11-03T18:47:07.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.399+0000 7f5fe2dab700 1 -- 172.21.15.27:0/503222803 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5fc40640b0 msgr2=0x7f5fc4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:47:07.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.399+0000 7f5fe2dab700 1 --2- 172.21.15.27:0/503222803 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5fc40640b0 0x7f5fc4066570 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 rx=0x7f5fd4004550 tx=0x7f5fd4009040).stop 2021-11-03T18:47:07.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.399+0000 7f5fe2dab700 1 -- 172.21.15.27:0/503222803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5fdc0f5ef0 msgr2=0x7f5fdc109640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:47:07.400 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.399+0000 7f5fe2dab700 1 --2- 172.21.15.27:0/503222803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5fdc0f5ef0 0x7f5fdc109640 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 rx=0x7f5fd0009990 tx=0x7f5fd0009b30).stop 2021-11-03T18:47:07.400 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.400+0000 7f5fe2dab700 1 -- 172.21.15.27:0/503222803 shutdown_connections 2021-11-03T18:47:07.400 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.400+0000 7f5fe2dab700 1 --2- 172.21.15.27:0/503222803 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5fc40640b0 0x7f5fc4066570 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:07.400 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.400+0000 7f5fe2dab700 1 --2- 172.21.15.27:0/503222803 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5fdc0f55e0 0x7f5fdc109150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:07.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.400+0000 7f5fe2dab700 1 --2- 172.21.15.27:0/503222803 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5fdc0f5ef0 0x7f5fdc109640 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:07.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.400+0000 7f5fe2dab700 1 -- 172.21.15.27:0/503222803 >> 172.21.15.27:0/503222803 conn(0x7f5fdc0f0de0 msgr2=0x7f5fdc05ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:47:07.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.400+0000 7f5fe2dab700 1 -- 172.21.15.27:0/503222803 shutdown_connections 2021-11-03T18:47:07.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.400+0000 7f5fe2dab700 1 -- 172.21.15.27:0/503222803 wait complete. 2021-11-03T18:47:07.409 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:47:07.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.500+0000 7fdbb926b700 1 -- 172.21.15.27:0/1247460171 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbb40f7960 msgr2=0x7fdbb40f7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:47:07.500 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.500+0000 7fdbb926b700 1 --2- 172.21.15.27:0/1247460171 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbb40f7960 0x7fdbb40f7dc0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 rx=0x7fdb9c004660 tx=0x7fdb9c009b30).stop 2021-11-03T18:47:07.500 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.500+0000 7fdbb926b700 1 -- 172.21.15.27:0/1247460171 shutdown_connections 2021-11-03T18:47:07.500 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.500+0000 7fdbb926b700 1 --2- 172.21.15.27:0/1247460171 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbb40f7960 0x7fdbb40f7dc0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:07.500 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.500+0000 7fdbb926b700 1 --2- 172.21.15.27:0/1247460171 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdbb40f66c0 0x7fdbb40f6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:07.501 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.500+0000 7fdbb926b700 1 -- 172.21.15.27:0/1247460171 >> 172.21.15.27:0/1247460171 conn(0x7fdbb40f1e20 msgr2=0x7fdbb40f4240 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:47:07.501 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.500+0000 7fdbb926b700 1 -- 172.21.15.27:0/1247460171 shutdown_connections 2021-11-03T18:47:07.501 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.500+0000 7fdbb926b700 1 -- 172.21.15.27:0/1247460171 wait complete. 2021-11-03T18:47:07.503 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.501+0000 7fdbb926b700 1 Processor -- start 2021-11-03T18:47:07.504 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.501+0000 7fdbb926b700 1 -- start start 2021-11-03T18:47:07.504 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.501+0000 7fdbb926b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbb40f66c0 0x7fdbb4106f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:47:07.504 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.501+0000 7fdbb926b700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdbb40f7960 0x7fdbb4107440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:47:07.504 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.501+0000 7fdbb926b700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb41079f0 con 0x7fdbb40f66c0 2021-11-03T18:47:07.505 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.501+0000 7fdbb926b700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb4107b30 con 0x7fdbb40f7960 2021-11-03T18:47:07.505 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.501+0000 7fdbb2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbb40f66c0 0x7fdbb4106f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:47:07.505 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.501+0000 7fdbb2ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbb40f66c0 0x7fdbb4106f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36832/0 (socket says 172.21.15.27:36832) 2021-11-03T18:47:07.505 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.501+0000 7fdbb2ffd700 1 -- 172.21.15.27:0/1933367318 learned_addr learned my addr 172.21.15.27:0/1933367318 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:47:07.505 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.501+0000 7fdbb27fc700 1 --2- 172.21.15.27:0/1933367318 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdbb40f7960 0x7fdbb4107440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:47:07.506 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.501+0000 7fdbb2ffd700 1 -- 172.21.15.27:0/1933367318 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdbb40f7960 msgr2=0x7fdbb4107440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:47:07.506 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.501+0000 7fdbb2ffd700 1 --2- 172.21.15.27:0/1933367318 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdbb40f7960 0x7fdbb4107440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:07.506 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.501+0000 7fdbb2ffd700 1 -- 172.21.15.27:0/1933367318 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb9c005040 con 0x7fdbb40f66c0 2021-11-03T18:47:07.506 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.501+0000 7fdbb27fc700 1 --2- 172.21.15.27:0/1933367318 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdbb40f7960 0x7fdbb4107440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:47:07.507 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.501+0000 7fdbb2ffd700 1 --2- 172.21.15.27:0/1933367318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbb40f66c0 0x7fdbb4106f50 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 rx=0x7fdbac00c710 tx=0x7fdbac004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:47:07.507 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.501+0000 7fdbab7fe700 1 -- 172.21.15.27:0/1933367318 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbac0106f0 con 0x7fdbb40f66c0 2021-11-03T18:47:07.507 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.501+0000 7fdbab7fe700 1 -- 172.21.15.27:0/1933367318 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fdbac010d00 con 0x7fdbb40f66c0 2021-11-03T18:47:07.507 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.501+0000 7fdbb926b700 1 -- 172.21.15.27:0/1933367318 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbb41084d0 con 0x7fdbb40f66c0 2021-11-03T18:47:07.507 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.502+0000 7fdbab7fe700 1 -- 172.21.15.27:0/1933367318 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdbac012d90 con 0x7fdbb40f66c0 2021-11-03T18:47:07.508 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.502+0000 7fdbb926b700 1 -- 172.21.15.27:0/1933367318 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb4108a90 con 0x7fdbb40f66c0 2021-11-03T18:47:07.508 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.502+0000 7fdbb926b700 1 -- 172.21.15.27:0/1933367318 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbb4045ad0 con 0x7fdbb40f66c0 2021-11-03T18:47:07.508 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.503+0000 7fdbab7fe700 1 -- 172.21.15.27:0/1933367318 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fdbac010070 con 0x7fdbb40f66c0 2021-11-03T18:47:07.508 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.503+0000 7fdbab7fe700 1 --2- 172.21.15.27:0/1933367318 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdba00640b0 0x7fdba0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:47:07.509 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.503+0000 7fdbab7fe700 1 -- 172.21.15.27:0/1933367318 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fdbac080d80 con 0x7fdbb40f66c0 2021-11-03T18:47:07.509 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.504+0000 7fdbb27fc700 1 --2- 172.21.15.27:0/1933367318 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdba00640b0 0x7fdba0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:47:07.509 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.504+0000 7fdbb27fc700 1 --2- 172.21.15.27:0/1933367318 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdba00640b0 0x7fdba0066570 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 rx=0x7fdb9c004e30 tx=0x7fdb9c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:47:07.510 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.507+0000 7fdbab7fe700 1 -- 172.21.15.27:0/1933367318 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fdbac0530b0 con 0x7fdbb40f66c0 2021-11-03T18:47:07.678 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.679+0000 7fdbb926b700 1 -- 172.21.15.27:0/1933367318 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdbb4108d10 con 0x7fdba00640b0 2021-11-03T18:47:07.692 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.693+0000 7fdbab7fe700 1 -- 172.21.15.27:0/1933367318 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3520 (secure 0 0 0) 0x7fdbb4108d10 con 0x7fdba00640b0 2021-11-03T18:47:07.693 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:47:07.693 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (98m) 7m ago 99m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:47:07.693 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (90m) 7m ago 99m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:47:07.693 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (90m) 6m ago 99m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:47:07.693 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (98m) 7m ago 99m 42.9M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:47:07.694 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (92m) 7m ago 95m 26.9M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:47:07.694 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (95m) 7m ago 95m 3852M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:47:07.694 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (90m) 6m ago 95m 26.5M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:47:07.694 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (92m) 6m ago 95m 27.6M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:47:07.695 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (92m) 7m ago 101m 625M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:47:07.695 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (91m) 6m ago 99m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:47:07.695 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (91m) 7m ago 101m 387M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:47:07.695 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (90m) 6m ago 99m 175M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:47:07.695 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (99m) 7m ago 99m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:47:07.696 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (98m) 6m ago 98m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:47:07.696 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (90m) 7m ago 98m 433M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:47:07.696 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (89m) 7m ago 98m 471M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:47:07.696 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (89m) 7m ago 97m 356M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:47:07.696 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (88m) 6m ago 97m 504M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:47:07.697 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (88m) 6m ago 96m 396M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:47:07.697 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (87m) 6m ago 96m 408M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:47:07.697 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (98m) 7m ago 99m 66.8M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:47:07.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.695+0000 7fdbb926b700 1 -- 172.21.15.27:0/1933367318 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdba00640b0 msgr2=0x7fdba0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:47:07.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.695+0000 7fdbb926b700 1 --2- 172.21.15.27:0/1933367318 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdba00640b0 0x7fdba0066570 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 rx=0x7fdb9c004e30 tx=0x7fdb9c009b30).stop 2021-11-03T18:47:07.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.695+0000 7fdbb926b700 1 -- 172.21.15.27:0/1933367318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbb40f66c0 msgr2=0x7fdbb4106f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:47:07.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.695+0000 7fdbb926b700 1 --2- 172.21.15.27:0/1933367318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbb40f66c0 0x7fdbb4106f50 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 rx=0x7fdbac00c710 tx=0x7fdbac004d10).stop 2021-11-03T18:47:07.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.695+0000 7fdbb926b700 1 -- 172.21.15.27:0/1933367318 shutdown_connections 2021-11-03T18:47:07.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.695+0000 7fdbb926b700 1 --2- 172.21.15.27:0/1933367318 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fdba00640b0 0x7fdba0066570 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:07.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.695+0000 7fdbb926b700 1 --2- 172.21.15.27:0/1933367318 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fdbb40f66c0 0x7fdbb4106f50 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:07.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.695+0000 7fdbb926b700 1 --2- 172.21.15.27:0/1933367318 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fdbb40f7960 0x7fdbb4107440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:07.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.695+0000 7fdbb926b700 1 -- 172.21.15.27:0/1933367318 >> 172.21.15.27:0/1933367318 conn(0x7fdbb40f1e20 msgr2=0x7fdbb40fb3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:47:07.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.695+0000 7fdbb926b700 1 -- 172.21.15.27:0/1933367318 shutdown_connections 2021-11-03T18:47:07.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.695+0000 7fdbb926b700 1 -- 172.21.15.27:0/1933367318 wait complete. 2021-11-03T18:47:07.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.794+0000 7f438f59e700 1 -- 172.21.15.27:0/938898901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43900f5440 msgr2=0x7f43900f5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:47:07.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.794+0000 7f438f59e700 1 --2- 172.21.15.27:0/938898901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43900f5440 0x7f43900f5860 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 rx=0x7f4380004660 tx=0x7f4380009b30).stop 2021-11-03T18:47:07.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.795+0000 7f438f59e700 1 -- 172.21.15.27:0/938898901 shutdown_connections 2021-11-03T18:47:07.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.795+0000 7f438f59e700 1 --2- 172.21.15.27:0/938898901 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43900f66e0 0x7f43900f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:07.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.795+0000 7f438f59e700 1 --2- 172.21.15.27:0/938898901 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43900f5440 0x7f43900f5860 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:07.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.795+0000 7f438f59e700 1 -- 172.21.15.27:0/938898901 >> 172.21.15.27:0/938898901 conn(0x7f43900f0ba0 msgr2=0x7f43900f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:47:07.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.795+0000 7f438f59e700 1 -- 172.21.15.27:0/938898901 shutdown_connections 2021-11-03T18:47:07.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.795+0000 7f438f59e700 1 -- 172.21.15.27:0/938898901 wait complete. 2021-11-03T18:47:07.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.795+0000 7f438f59e700 1 Processor -- start 2021-11-03T18:47:07.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.795+0000 7f438f59e700 1 -- start start 2021-11-03T18:47:07.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.795+0000 7f438f59e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43900f5440 0x7f4390106e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:47:07.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.795+0000 7f438f59e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43900f66e0 0x7f4390107300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:47:07.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.795+0000 7f438f59e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4390107900 con 0x7f43900f66e0 2021-11-03T18:47:07.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.795+0000 7f438f59e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4390108110 con 0x7f43900f5440 2021-11-03T18:47:07.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.795+0000 7f438dd9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43900f66e0 0x7f4390107300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:47:07.800 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.795+0000 7f438dd9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43900f66e0 0x7f4390107300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36844/0 (socket says 172.21.15.27:36844) 2021-11-03T18:47:07.800 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.795+0000 7f438dd9b700 1 -- 172.21.15.27:0/616347271 learned_addr learned my addr 172.21.15.27:0/616347271 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:47:07.800 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.796+0000 7f438e59c700 1 --2- 172.21.15.27:0/616347271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43900f5440 0x7f4390106e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:47:07.800 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.796+0000 7f438dd9b700 1 -- 172.21.15.27:0/616347271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43900f5440 msgr2=0x7f4390106e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:47:07.801 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.796+0000 7f438dd9b700 1 --2- 172.21.15.27:0/616347271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43900f5440 0x7f4390106e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:07.801 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.796+0000 7f438dd9b700 1 -- 172.21.15.27:0/616347271 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4380005040 con 0x7f43900f66e0 2021-11-03T18:47:07.801 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.796+0000 7f438e59c700 1 --2- 172.21.15.27:0/616347271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43900f5440 0x7f4390106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T18:47:07.801 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.796+0000 7f438dd9b700 1 --2- 172.21.15.27:0/616347271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43900f66e0 0x7f4390107300 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 rx=0x7f43840026d0 tx=0x7f4384009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:47:07.801 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.796+0000 7f437effd700 1 -- 172.21.15.27:0/616347271 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4384009320 con 0x7f43900f66e0 2021-11-03T18:47:07.802 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.796+0000 7f437effd700 1 -- 172.21.15.27:0/616347271 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f438400f040 con 0x7f43900f66e0 2021-11-03T18:47:07.802 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.796+0000 7f438f59e700 1 -- 172.21.15.27:0/616347271 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4390108390 con 0x7f43900f66e0 2021-11-03T18:47:07.802 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.796+0000 7f437effd700 1 -- 172.21.15.27:0/616347271 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43840148f0 con 0x7f43900f66e0 2021-11-03T18:47:07.802 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.796+0000 7f438f59e700 1 -- 172.21.15.27:0/616347271 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4390108950 con 0x7f43900f66e0 2021-11-03T18:47:07.804 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.797+0000 7f438f59e700 1 -- 172.21.15.27:0/616347271 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4390059100 con 0x7f43900f66e0 2021-11-03T18:47:07.804 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.797+0000 7f437effd700 1 -- 172.21.15.27:0/616347271 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f438401b060 con 0x7f43900f66e0 2021-11-03T18:47:07.804 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.798+0000 7f437effd700 1 --2- 172.21.15.27:0/616347271 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4378064100 0x7f43780665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:47:07.804 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.798+0000 7f437effd700 1 -- 172.21.15.27:0/616347271 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f4384080820 con 0x7f43900f66e0 2021-11-03T18:47:07.805 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.798+0000 7f438e59c700 1 --2- 172.21.15.27:0/616347271 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4378064100 0x7f43780665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:47:07.805 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.798+0000 7f438e59c700 1 --2- 172.21.15.27:0/616347271 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4378064100 0x7f43780665c0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 rx=0x7f4380004e30 tx=0x7f4380009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:47:07.805 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:07.802+0000 7f437effd700 1 -- 172.21.15.27:0/616347271 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f4384052a90 con 0x7f43900f66e0 2021-11-03T18:47:08.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.034+0000 7f438f59e700 1 -- 172.21.15.27:0/616347271 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f43900fdc20 con 0x7f43900f66e0 2021-11-03T18:47:08.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.034+0000 7f437effd700 1 -- 172.21.15.27:0/616347271 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f4384019070 con 0x7f43900f66e0 2021-11-03T18:47:08.034 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:47:08.034 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:47:08.035 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:47:08.035 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:47:08.035 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:47:08.035 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:47:08.035 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:47:08.036 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:47:08.036 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:47:08.036 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:47:08.036 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:47:08.037 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:47:08.037 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:47:08.037 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:47:08.037 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:47:08.037 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:47:08.038 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:47:08.038 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:47:08.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.036+0000 7f438f59e700 1 -- 172.21.15.27:0/616347271 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4378064100 msgr2=0x7f43780665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:47:08.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.036+0000 7f438f59e700 1 --2- 172.21.15.27:0/616347271 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4378064100 0x7f43780665c0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 rx=0x7f4380004e30 tx=0x7f4380009b30).stop 2021-11-03T18:47:08.039 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.036+0000 7f438f59e700 1 -- 172.21.15.27:0/616347271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43900f66e0 msgr2=0x7f4390107300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:47:08.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.036+0000 7f438f59e700 1 --2- 172.21.15.27:0/616347271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43900f66e0 0x7f4390107300 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 rx=0x7f43840026d0 tx=0x7f4384009de0).stop 2021-11-03T18:47:08.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.036+0000 7f438f59e700 1 -- 172.21.15.27:0/616347271 shutdown_connections 2021-11-03T18:47:08.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.036+0000 7f438f59e700 1 --2- 172.21.15.27:0/616347271 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f4378064100 0x7f43780665c0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:08.040 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.036+0000 7f438f59e700 1 --2- 172.21.15.27:0/616347271 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f43900f5440 0x7f4390106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:08.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.036+0000 7f438f59e700 1 --2- 172.21.15.27:0/616347271 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f43900f66e0 0x7f4390107300 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:08.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.036+0000 7f438f59e700 1 -- 172.21.15.27:0/616347271 >> 172.21.15.27:0/616347271 conn(0x7f43900f0ba0 msgr2=0x7f439005f620 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:47:08.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.036+0000 7f438f59e700 1 -- 172.21.15.27:0/616347271 shutdown_connections 2021-11-03T18:47:08.041 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.036+0000 7f438f59e700 1 -- 172.21.15.27:0/616347271 wait complete. 2021-11-03T18:47:08.136 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.136+0000 7f3b693c1700 1 -- 172.21.15.27:0/590119272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b640f66e0 msgr2=0x7f3b640f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:47:08.136 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.136+0000 7f3b693c1700 1 --2- 172.21.15.27:0/590119272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b640f66e0 0x7f3b640f6b00 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 rx=0x7f3b54004660 tx=0x7f3b54009b30).stop 2021-11-03T18:47:08.136 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.136+0000 7f3b693c1700 1 -- 172.21.15.27:0/590119272 shutdown_connections 2021-11-03T18:47:08.136 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.136+0000 7f3b693c1700 1 --2- 172.21.15.27:0/590119272 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b640f6ff0 0x7f3b640f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:08.137 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.136+0000 7f3b693c1700 1 --2- 172.21.15.27:0/590119272 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b640f66e0 0x7f3b640f6b00 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:08.137 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.136+0000 7f3b693c1700 1 -- 172.21.15.27:0/590119272 >> 172.21.15.27:0/590119272 conn(0x7f3b640f1f00 msgr2=0x7f3b640f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:47:08.137 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.136+0000 7f3b693c1700 1 -- 172.21.15.27:0/590119272 shutdown_connections 2021-11-03T18:47:08.137 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.136+0000 7f3b693c1700 1 -- 172.21.15.27:0/590119272 wait complete. 2021-11-03T18:47:08.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.137+0000 7f3b693c1700 1 Processor -- start 2021-11-03T18:47:08.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.137+0000 7f3b693c1700 1 -- start start 2021-11-03T18:47:08.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.137+0000 7f3b693c1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b640f66e0 0x7f3b6406b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:47:08.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.137+0000 7f3b693c1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b640f6ff0 0x7f3b64069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:47:08.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.137+0000 7f3b693c1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6406a380 con 0x7f3b640f6ff0 2021-11-03T18:47:08.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.137+0000 7f3b693c1700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6406a4c0 con 0x7f3b640f66e0 2021-11-03T18:47:08.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.137+0000 7f3b627fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b640f6ff0 0x7f3b64069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:47:08.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.137+0000 7f3b627fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b640f6ff0 0x7f3b64069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36856/0 (socket says 172.21.15.27:36856) 2021-11-03T18:47:08.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.137+0000 7f3b627fc700 1 -- 172.21.15.27:0/3930301517 learned_addr learned my addr 172.21.15.27:0/3930301517 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:47:08.142 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.137+0000 7f3b62ffd700 1 --2- 172.21.15.27:0/3930301517 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b640f66e0 0x7f3b6406b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:47:08.142 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.137+0000 7f3b627fc700 1 -- 172.21.15.27:0/3930301517 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b640f66e0 msgr2=0x7f3b6406b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:47:08.142 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.137+0000 7f3b627fc700 1 --2- 172.21.15.27:0/3930301517 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b640f66e0 0x7f3b6406b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:08.142 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.137+0000 7f3b627fc700 1 -- 172.21.15.27:0/3930301517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b54005040 con 0x7f3b640f6ff0 2021-11-03T18:47:08.143 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.138+0000 7f3b627fc700 1 --2- 172.21.15.27:0/3930301517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b640f6ff0 0x7f3b64069e20 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 rx=0x7f3b580026d0 tx=0x7f3b58009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:47:08.143 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.138+0000 7f3b4b7fe700 1 -- 172.21.15.27:0/3930301517 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b580079c0 con 0x7f3b640f6ff0 2021-11-03T18:47:08.143 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.138+0000 7f3b4b7fe700 1 -- 172.21.15.27:0/3930301517 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f3b5800f040 con 0x7f3b640f6ff0 2021-11-03T18:47:08.143 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.138+0000 7f3b693c1700 1 -- 172.21.15.27:0/3930301517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b6406a7a0 con 0x7f3b640f6ff0 2021-11-03T18:47:08.144 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.138+0000 7f3b4b7fe700 1 -- 172.21.15.27:0/3930301517 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3b580148b0 con 0x7f3b640f6ff0 2021-11-03T18:47:08.144 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.138+0000 7f3b693c1700 1 -- 172.21.15.27:0/3930301517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b6410ef90 con 0x7f3b640f6ff0 2021-11-03T18:47:08.144 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.139+0000 7f3b693c1700 1 -- 172.21.15.27:0/3930301517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b64045ad0 con 0x7f3b640f6ff0 2021-11-03T18:47:08.144 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.139+0000 7f3b4b7fe700 1 -- 172.21.15.27:0/3930301517 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f3b58007b20 con 0x7f3b640f6ff0 2021-11-03T18:47:08.144 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.140+0000 7f3b4b7fe700 1 --2- 172.21.15.27:0/3930301517 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3b4c0640b0 0x7f3b4c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:47:08.145 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.140+0000 7f3b4b7fe700 1 -- 172.21.15.27:0/3930301517 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f3b58080660 con 0x7f3b640f6ff0 2021-11-03T18:47:08.145 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.140+0000 7f3b62ffd700 1 --2- 172.21.15.27:0/3930301517 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3b4c0640b0 0x7f3b4c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:47:08.145 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.140+0000 7f3b62ffd700 1 --2- 172.21.15.27:0/3930301517 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3b4c0640b0 0x7f3b4c066570 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 rx=0x7f3b54004630 tx=0x7f3b54009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:47:08.146 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.144+0000 7f3b4b7fe700 1 -- 172.21.15.27:0/3930301517 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f3b580529c0 con 0x7f3b640f6ff0 2021-11-03T18:47:08.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.346+0000 7f3b693c1700 1 -- 172.21.15.27:0/3930301517 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3b640fef70 con 0x7f3b640f6ff0 2021-11-03T18:47:08.346 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.347+0000 7f3b4b7fe700 1 -- 172.21.15.27:0/3930301517 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f3b580525b0 con 0x7f3b640f6ff0 2021-11-03T18:47:08.347 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:47:08.347 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:47:08.347 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:47:08.347 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:47:08.348 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:47:08.348 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:47:08.348 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:47:08.348 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:47:08.349 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:47:08.349 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:47:08.349 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:47:08.349 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:47:08.349 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:47:08.350 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:47:08.350 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:47:08.350 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:47:08.350 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:47:08.351 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:47:08.351 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:47:08.351 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:47:08.351 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:47:08.351 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:47:08.352 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:47:08.352 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:47:08.352 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:47:08.352 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:47:08.352 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:47:08.353 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:47:08.353 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:47:08.353 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:47:08.353 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:47:08.354 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:47:08.354 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:47:08.354 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:47:08.354 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:47:08.354 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:47:08.355 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:47:08.355 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:47:08.355 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:47:08.356 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.348+0000 7f3b693c1700 1 -- 172.21.15.27:0/3930301517 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3b4c0640b0 msgr2=0x7f3b4c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:47:08.357 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.348+0000 7f3b693c1700 1 --2- 172.21.15.27:0/3930301517 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3b4c0640b0 0x7f3b4c066570 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 rx=0x7f3b54004630 tx=0x7f3b54009b30).stop 2021-11-03T18:47:08.357 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.348+0000 7f3b693c1700 1 -- 172.21.15.27:0/3930301517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b640f6ff0 msgr2=0x7f3b64069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:47:08.357 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.348+0000 7f3b693c1700 1 --2- 172.21.15.27:0/3930301517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b640f6ff0 0x7f3b64069e20 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 rx=0x7f3b580026d0 tx=0x7f3b58009de0).stop 2021-11-03T18:47:08.357 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.348+0000 7f3b693c1700 1 -- 172.21.15.27:0/3930301517 shutdown_connections 2021-11-03T18:47:08.357 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.348+0000 7f3b693c1700 1 --2- 172.21.15.27:0/3930301517 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f3b4c0640b0 0x7f3b4c066570 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:08.358 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.348+0000 7f3b693c1700 1 --2- 172.21.15.27:0/3930301517 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f3b640f66e0 0x7f3b6406b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:08.358 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.348+0000 7f3b693c1700 1 --2- 172.21.15.27:0/3930301517 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f3b640f6ff0 0x7f3b64069e20 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:08.358 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.348+0000 7f3b693c1700 1 -- 172.21.15.27:0/3930301517 >> 172.21.15.27:0/3930301517 conn(0x7f3b640f1f00 msgr2=0x7f3b640fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:47:08.358 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.349+0000 7f3b693c1700 1 -- 172.21.15.27:0/3930301517 shutdown_connections 2021-11-03T18:47:08.359 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:08.349+0000 7f3b693c1700 1 -- 172.21.15.27:0/3930301517 wait complete. 2021-11-03T18:47:08.359 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:47:09.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:08 smithi115 ceph-mon[65453]: pgmap v2776: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:09.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:08 smithi115 ceph-mon[65453]: from='client.36952 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:47:09.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:08 smithi115 ceph-mon[65453]: from='client.36956 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:47:09.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:08 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/616347271' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:09.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:08 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3930301517' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:47:09.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:08 smithi027 ceph-mon[97244]: pgmap v2776: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:09.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:08 smithi027 ceph-mon[97244]: from='client.36952 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:47:09.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:08 smithi027 ceph-mon[97244]: from='client.36956 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:47:09.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:08 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/616347271' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:09.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:08 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3930301517' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:47:10.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:47:10.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:47:10.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:47:10.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:10.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:10.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:10.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:10.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:09 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:47:10.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:47:10.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:47:10.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:47:10.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:10.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:10.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:10.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:10.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:09 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:47:11.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:10 smithi115 ceph-mon[65453]: pgmap v2777: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:11.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5472.491916 secs 2021-11-03T18:47:11.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:10 smithi027 ceph-mon[97244]: pgmap v2777: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:11.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5472.491916 secs 2021-11-03T18:47:13.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:12 smithi115 ceph-mon[65453]: pgmap v2778: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:13.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:12 smithi027 ceph-mon[97244]: pgmap v2778: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:15.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:14 smithi115 ceph-mon[65453]: pgmap v2779: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:15.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:14 smithi027 ceph-mon[97244]: pgmap v2779: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:16.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5477.492046 secs 2021-11-03T18:47:16.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5477.492046 secs 2021-11-03T18:47:17.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:16 smithi115 ceph-mon[65453]: pgmap v2780: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:17.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:16 smithi027 ceph-mon[97244]: pgmap v2780: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:19.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:18 smithi115 ceph-mon[65453]: pgmap v2781: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:19.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:18 smithi027 ceph-mon[97244]: pgmap v2781: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:20.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:47:20.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:47:20.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:47:20.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:20.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:20.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:20.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:19 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:20.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:19 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:47:20.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:47:20.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:47:20.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:47:20.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:20.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:20.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:20.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:19 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:20.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:19 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:47:21.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:20 smithi115 ceph-mon[65453]: pgmap v2782: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:21.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5482.492187 secs 2021-11-03T18:47:21.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:20 smithi027 ceph-mon[97244]: pgmap v2782: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:21.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5482.492187 secs 2021-11-03T18:47:23.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:22 smithi115 ceph-mon[65453]: pgmap v2783: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:23.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:22 smithi027 ceph-mon[97244]: pgmap v2783: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:25.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:24 smithi115 ceph-mon[65453]: pgmap v2784: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:25.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:24 smithi027 ceph-mon[97244]: pgmap v2784: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:26.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5487.492304 secs 2021-11-03T18:47:26.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5487.492304 secs 2021-11-03T18:47:27.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:26 smithi115 ceph-mon[65453]: pgmap v2785: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:27.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:26 smithi027 ceph-mon[97244]: pgmap v2785: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:29.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:28 smithi115 ceph-mon[65453]: pgmap v2786: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:29.256 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:28 smithi027 ceph-mon[97244]: pgmap v2786: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:30.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:47:30.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:29 smithi115 ceph-mon[65453]: poll took 40121.661 ms: 1 events 2021-11-03T18:47:30.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:47:30.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:47:30.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:47:30.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:30.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:30.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:30.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:29 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:30.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:47:30.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:29 smithi027 ceph-mon[97244]: poll took 40121.661 ms: 1 events 2021-11-03T18:47:30.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:47:30.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:47:30.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:47:30.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:30.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:30.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:30.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:29 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:31.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:30 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:47:31.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:30 smithi115 ceph-mon[65453]: pgmap v2787: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:31.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5492.492464 secs 2021-11-03T18:47:31.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:30 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:47:31.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:30 smithi027 ceph-mon[97244]: pgmap v2787: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:31.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5492.492464 secs 2021-11-03T18:47:33.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:32 smithi115 ceph-mon[65453]: pgmap v2788: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:33.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:32 smithi027 ceph-mon[97244]: pgmap v2788: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:35.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:34 smithi115 ceph-mon[65453]: pgmap v2789: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:35.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:34 smithi027 ceph-mon[97244]: pgmap v2789: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:36.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5497.492599 secs 2021-11-03T18:47:36.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5497.492599 secs 2021-11-03T18:47:37.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:36 smithi115 ceph-mon[65453]: pgmap v2790: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:37.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:36 smithi027 ceph-mon[97244]: pgmap v2790: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:38.464 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.464+0000 7f96c0d79700 1 -- 172.21.15.27:0/1528239344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96bc0603f0 msgr2=0x7f96bc0f9340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:47:38.465 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.464+0000 7f96c0d79700 1 --2- 172.21.15.27:0/1528239344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96bc0603f0 0x7f96bc0f9340 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 rx=0x7f96b0004660 tx=0x7f96b0009b30).stop 2021-11-03T18:47:38.465 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.464+0000 7f96c0d79700 1 -- 172.21.15.27:0/1528239344 shutdown_connections 2021-11-03T18:47:38.465 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.464+0000 7f96c0d79700 1 --2- 172.21.15.27:0/1528239344 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96bc0f9830 0x7f96bc0fbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:38.465 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.464+0000 7f96c0d79700 1 --2- 172.21.15.27:0/1528239344 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96bc0603f0 0x7f96bc0f9340 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:38.466 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.464+0000 7f96c0d79700 1 -- 172.21.15.27:0/1528239344 >> 172.21.15.27:0/1528239344 conn(0x7f96bc0f0dc0 msgr2=0x7f96bc0f31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:47:38.466 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.464+0000 7f96c0d79700 1 -- 172.21.15.27:0/1528239344 shutdown_connections 2021-11-03T18:47:38.466 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.464+0000 7f96c0d79700 1 -- 172.21.15.27:0/1528239344 wait complete. 2021-11-03T18:47:38.466 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.464+0000 7f96c0d79700 1 Processor -- start 2021-11-03T18:47:38.467 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.464+0000 7f96c0d79700 1 -- start start 2021-11-03T18:47:38.467 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.465+0000 7f96c0d79700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96bc0603f0 0x7f96bc106f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:47:38.467 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.465+0000 7f96c0d79700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96bc0f9830 0x7f96bc107450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:47:38.467 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.465+0000 7f96c0d79700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc107a00 con 0x7f96bc0603f0 2021-11-03T18:47:38.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.465+0000 7f96c0d79700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96bc107b40 con 0x7f96bc0f9830 2021-11-03T18:47:38.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.465+0000 7f96ba59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96bc0603f0 0x7f96bc106f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:47:38.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.465+0000 7f96ba59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96bc0603f0 0x7f96bc106f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36874/0 (socket says 172.21.15.27:36874) 2021-11-03T18:47:38.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.465+0000 7f96ba59c700 1 -- 172.21.15.27:0/3012995967 learned_addr learned my addr 172.21.15.27:0/3012995967 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:47:38.468 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.465+0000 7f96b9d9b700 1 --2- 172.21.15.27:0/3012995967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96bc0f9830 0x7f96bc107450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:47:38.469 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.465+0000 7f96ba59c700 1 -- 172.21.15.27:0/3012995967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96bc0f9830 msgr2=0x7f96bc107450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:47:38.469 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.465+0000 7f96ba59c700 1 --2- 172.21.15.27:0/3012995967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96bc0f9830 0x7f96bc107450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:38.469 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.465+0000 7f96ba59c700 1 -- 172.21.15.27:0/3012995967 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b0005040 con 0x7f96bc0603f0 2021-11-03T18:47:38.469 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.465+0000 7f96b9d9b700 1 --2- 172.21.15.27:0/3012995967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96bc0f9830 0x7f96bc107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:47:38.469 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.465+0000 7f96ba59c700 1 --2- 172.21.15.27:0/3012995967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96bc0603f0 0x7f96bc106f60 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 rx=0x7f96b0004f60 tx=0x7f96b0009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:47:38.470 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.465+0000 7f96aaffd700 1 -- 172.21.15.27:0/3012995967 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96b0018070 con 0x7f96bc0603f0 2021-11-03T18:47:38.470 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.465+0000 7f96c0d79700 1 -- 172.21.15.27:0/3012995967 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96bc108480 con 0x7f96bc0603f0 2021-11-03T18:47:38.470 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.465+0000 7f96aaffd700 1 -- 172.21.15.27:0/3012995967 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f96b0007ba0 con 0x7f96bc0603f0 2021-11-03T18:47:38.470 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.465+0000 7f96aaffd700 1 -- 172.21.15.27:0/3012995967 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f96b001c620 con 0x7f96bc0603f0 2021-11-03T18:47:38.471 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.465+0000 7f96c0d79700 1 -- 172.21.15.27:0/3012995967 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96bc1089e0 con 0x7f96bc0603f0 2021-11-03T18:47:38.472 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.466+0000 7f96c0d79700 1 -- 172.21.15.27:0/3012995967 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96bc045ad0 con 0x7f96bc0603f0 2021-11-03T18:47:38.472 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.467+0000 7f96aaffd700 1 -- 172.21.15.27:0/3012995967 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f96b001c8e0 con 0x7f96bc0603f0 2021-11-03T18:47:38.472 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.467+0000 7f96aaffd700 1 --2- 172.21.15.27:0/3012995967 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f96a40640b0 0x7f96a4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:47:38.472 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.467+0000 7f96aaffd700 1 -- 172.21.15.27:0/3012995967 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f96b00826c0 con 0x7f96bc0603f0 2021-11-03T18:47:38.473 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.467+0000 7f96b9d9b700 1 --2- 172.21.15.27:0/3012995967 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f96a40640b0 0x7f96a4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:47:38.473 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.468+0000 7f96b9d9b700 1 --2- 172.21.15.27:0/3012995967 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f96a40640b0 0x7f96a4066570 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x7f96ac0041d0 tx=0x7f96ac009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:47:38.473 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.471+0000 7f96aaffd700 1 -- 172.21.15.27:0/3012995967 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f96b0054a20 con 0x7f96bc0603f0 2021-11-03T18:47:38.649 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.650+0000 7f96c0d79700 1 -- 172.21.15.27:0/3012995967 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f96bc0ff010 con 0x7f96a40640b0 2021-11-03T18:47:38.650 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.652+0000 7f96aaffd700 1 -- 172.21.15.27:0/3012995967 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f96bc0ff010 con 0x7f96a40640b0 2021-11-03T18:47:38.652 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.653+0000 7f96c0d79700 1 -- 172.21.15.27:0/3012995967 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f96a40640b0 msgr2=0x7f96a4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:47:38.652 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.653+0000 7f96c0d79700 1 --2- 172.21.15.27:0/3012995967 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f96a40640b0 0x7f96a4066570 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x7f96ac0041d0 tx=0x7f96ac009040).stop 2021-11-03T18:47:38.652 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.653+0000 7f96c0d79700 1 -- 172.21.15.27:0/3012995967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96bc0603f0 msgr2=0x7f96bc106f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:47:38.652 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.653+0000 7f96c0d79700 1 --2- 172.21.15.27:0/3012995967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96bc0603f0 0x7f96bc106f60 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 rx=0x7f96b0004f60 tx=0x7f96b0009b30).stop 2021-11-03T18:47:38.653 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.653+0000 7f96c0d79700 1 -- 172.21.15.27:0/3012995967 shutdown_connections 2021-11-03T18:47:38.653 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.653+0000 7f96c0d79700 1 --2- 172.21.15.27:0/3012995967 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f96a40640b0 0x7f96a4066570 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:38.653 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.653+0000 7f96c0d79700 1 --2- 172.21.15.27:0/3012995967 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f96bc0603f0 0x7f96bc106f60 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:38.653 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.653+0000 7f96c0d79700 1 --2- 172.21.15.27:0/3012995967 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f96bc0f9830 0x7f96bc107450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:38.653 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.653+0000 7f96c0d79700 1 -- 172.21.15.27:0/3012995967 >> 172.21.15.27:0/3012995967 conn(0x7f96bc0f0dc0 msgr2=0x7f96bc0f31b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:47:38.654 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.654+0000 7f96c0d79700 1 -- 172.21.15.27:0/3012995967 shutdown_connections 2021-11-03T18:47:38.654 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.654+0000 7f96c0d79700 1 -- 172.21.15.27:0/3012995967 wait complete. 2021-11-03T18:47:38.661 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:47:38.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.753+0000 7f145142c700 1 -- 172.21.15.27:0/3254293237 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f144c0fba30 msgr2=0x7f144c0fde20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:47:38.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.753+0000 7f145142c700 1 --2- 172.21.15.27:0/3254293237 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f144c0fba30 0x7f144c0fde20 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 rx=0x7f1440004660 tx=0x7f1440009b30).stop 2021-11-03T18:47:38.752 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.753+0000 7f145142c700 1 -- 172.21.15.27:0/3254293237 shutdown_connections 2021-11-03T18:47:38.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.753+0000 7f145142c700 1 --2- 172.21.15.27:0/3254293237 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f144c0fba30 0x7f144c0fde20 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:38.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.753+0000 7f145142c700 1 --2- 172.21.15.27:0/3254293237 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f144c060310 0x7f144c0fb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:38.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.753+0000 7f145142c700 1 -- 172.21.15.27:0/3254293237 >> 172.21.15.27:0/3254293237 conn(0x7f144c0f0bc0 msgr2=0x7f144c0f2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:47:38.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.753+0000 7f145142c700 1 -- 172.21.15.27:0/3254293237 shutdown_connections 2021-11-03T18:47:38.753 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.753+0000 7f145142c700 1 -- 172.21.15.27:0/3254293237 wait complete. 2021-11-03T18:47:38.756 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.754+0000 7f145142c700 1 Processor -- start 2021-11-03T18:47:38.756 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.754+0000 7f145142c700 1 -- start start 2021-11-03T18:47:38.756 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.754+0000 7f145142c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f144c060310 0x7f144c108f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:47:38.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.754+0000 7f145142c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f144c0fba30 0x7f144c109450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:47:38.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.754+0000 7f145142c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144c109a00 con 0x7f144c060310 2021-11-03T18:47:38.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.754+0000 7f145142c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f144c109b40 con 0x7f144c0fba30 2021-11-03T18:47:38.757 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.754+0000 7f144affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f144c060310 0x7f144c108f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:47:38.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.754+0000 7f144affd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f144c060310 0x7f144c108f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36884/0 (socket says 172.21.15.27:36884) 2021-11-03T18:47:38.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.754+0000 7f144affd700 1 -- 172.21.15.27:0/1178633705 learned_addr learned my addr 172.21.15.27:0/1178633705 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:47:38.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.755+0000 7f144a7fc700 1 --2- 172.21.15.27:0/1178633705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f144c0fba30 0x7f144c109450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:47:38.758 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.755+0000 7f144affd700 1 -- 172.21.15.27:0/1178633705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f144c0fba30 msgr2=0x7f144c109450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:47:38.759 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.755+0000 7f144affd700 1 --2- 172.21.15.27:0/1178633705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f144c0fba30 0x7f144c109450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:38.759 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.755+0000 7f144affd700 1 -- 172.21.15.27:0/1178633705 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1440005040 con 0x7f144c060310 2021-11-03T18:47:38.759 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.755+0000 7f144a7fc700 1 --2- 172.21.15.27:0/1178633705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f144c0fba30 0x7f144c109450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:47:38.759 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.755+0000 7f144affd700 1 --2- 172.21.15.27:0/1178633705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f144c060310 0x7f144c108f60 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 rx=0x7f143c00c530 tx=0x7f143c007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:47:38.759 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.755+0000 7f14337fe700 1 -- 172.21.15.27:0/1178633705 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f143c00f650 con 0x7f144c060310 2021-11-03T18:47:38.760 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.755+0000 7f14337fe700 1 -- 172.21.15.27:0/1178633705 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f143c00fc60 con 0x7f144c060310 2021-11-03T18:47:38.760 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.755+0000 7f145142c700 1 -- 172.21.15.27:0/1178633705 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f144c10a4e0 con 0x7f144c060310 2021-11-03T18:47:38.760 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.755+0000 7f14337fe700 1 -- 172.21.15.27:0/1178633705 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f143c012cd0 con 0x7f144c060310 2021-11-03T18:47:38.760 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.755+0000 7f145142c700 1 -- 172.21.15.27:0/1178633705 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f144c10aaa0 con 0x7f144c060310 2021-11-03T18:47:38.760 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.756+0000 7f145142c700 1 -- 172.21.15.27:0/1178633705 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f144c045ad0 con 0x7f144c060310 2021-11-03T18:47:38.761 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.756+0000 7f14337fe700 1 -- 172.21.15.27:0/1178633705 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f143c010070 con 0x7f144c060310 2021-11-03T18:47:38.761 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.757+0000 7f14337fe700 1 --2- 172.21.15.27:0/1178633705 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1434064100 0x7f14340665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:47:38.761 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.757+0000 7f14337fe700 1 -- 172.21.15.27:0/1178633705 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f143c0806e0 con 0x7f144c060310 2021-11-03T18:47:38.761 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.757+0000 7f144a7fc700 1 --2- 172.21.15.27:0/1178633705 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1434064100 0x7f14340665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:47:38.762 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.757+0000 7f144a7fc700 1 --2- 172.21.15.27:0/1178633705 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1434064100 0x7f14340665c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0x7f1440004e30 tx=0x7f1440009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:47:38.762 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.761+0000 7f14337fe700 1 -- 172.21.15.27:0/1178633705 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f143c052950 con 0x7f144c060310 2021-11-03T18:47:38.933 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.935+0000 7f145142c700 1 -- 172.21.15.27:0/1178633705 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f144c1010a0 con 0x7f1434064100 2021-11-03T18:47:38.948 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.950+0000 7f14337fe700 1 -- 172.21.15.27:0/1178633705 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3542 (secure 0 0 0) 0x7f144c1010a0 con 0x7f1434064100 2021-11-03T18:47:38.949 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:47:38.949 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (99m) 7m ago 100m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:47:38.949 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (90m) 7m ago 100m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:47:38.950 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (90m) 7m ago 99m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:47:38.950 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (99m) 7m ago 100m 42.9M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:47:38.950 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (92m) 7m ago 95m 26.9M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:47:38.950 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (95m) 7m ago 95m 3852M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:47:38.951 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (91m) 7m ago 95m 26.5M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:47:38.951 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (93m) 7m ago 95m 27.6M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:47:38.951 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (92m) 7m ago 101m 625M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:47:38.951 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (92m) 7m ago 99m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:47:38.952 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (91m) 7m ago 101m 387M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:47:38.952 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (91m) 7m ago 99m 175M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:47:38.952 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (100m) 7m ago 100m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:47:38.952 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (99m) 7m ago 99m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:47:38.952 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (90m) 7m ago 98m 433M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:47:38.953 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (90m) 7m ago 98m 471M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:47:38.953 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (89m) 7m ago 98m 356M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:47:38.953 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (89m) 7m ago 97m 504M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:47:38.953 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (88m) 7m ago 97m 396M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:47:38.954 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (88m) 7m ago 97m 408M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:47:38.954 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (98m) 7m ago 99m 66.8M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:47:38.955 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.952+0000 7f145142c700 1 -- 172.21.15.27:0/1178633705 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1434064100 msgr2=0x7f14340665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:47:38.955 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.952+0000 7f145142c700 1 --2- 172.21.15.27:0/1178633705 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1434064100 0x7f14340665c0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0x7f1440004e30 tx=0x7f1440009b30).stop 2021-11-03T18:47:38.955 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.952+0000 7f145142c700 1 -- 172.21.15.27:0/1178633705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f144c060310 msgr2=0x7f144c108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:47:38.955 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.952+0000 7f145142c700 1 --2- 172.21.15.27:0/1178633705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f144c060310 0x7f144c108f60 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 rx=0x7f143c00c530 tx=0x7f143c007e20).stop 2021-11-03T18:47:38.956 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.952+0000 7f145142c700 1 -- 172.21.15.27:0/1178633705 shutdown_connections 2021-11-03T18:47:38.956 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.952+0000 7f145142c700 1 --2- 172.21.15.27:0/1178633705 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f1434064100 0x7f14340665c0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:38.956 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.952+0000 7f145142c700 1 --2- 172.21.15.27:0/1178633705 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f144c060310 0x7f144c108f60 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:38.956 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.952+0000 7f145142c700 1 --2- 172.21.15.27:0/1178633705 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f144c0fba30 0x7f144c109450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:38.957 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.952+0000 7f145142c700 1 -- 172.21.15.27:0/1178633705 >> 172.21.15.27:0/1178633705 conn(0x7f144c0f0bc0 msgr2=0x7f144c0f2fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:47:38.957 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.952+0000 7f145142c700 1 -- 172.21.15.27:0/1178633705 shutdown_connections 2021-11-03T18:47:38.957 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:38.952+0000 7f145142c700 1 -- 172.21.15.27:0/1178633705 wait complete. 2021-11-03T18:47:39.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.051+0000 7f096b59e700 1 -- 172.21.15.27:0/3337359911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f096c0f77d0 msgr2=0x7f096c0f7c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:47:39.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.051+0000 7f096b59e700 1 --2- 172.21.15.27:0/3337359911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f096c0f77d0 0x7f096c0f7c30 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 rx=0x7f0954004660 tx=0x7f0954009b30).stop 2021-11-03T18:47:39.052 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.051+0000 7f096b59e700 1 -- 172.21.15.27:0/3337359911 shutdown_connections 2021-11-03T18:47:39.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.051+0000 7f096b59e700 1 --2- 172.21.15.27:0/3337359911 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f096c0f77d0 0x7f096c0f7c30 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:39.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.051+0000 7f096b59e700 1 --2- 172.21.15.27:0/3337359911 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f096c0f6530 0x7f096c0f6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:39.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.051+0000 7f096b59e700 1 -- 172.21.15.27:0/3337359911 >> 172.21.15.27:0/3337359911 conn(0x7f096c0f1cd0 msgr2=0x7f096c0f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:47:39.053 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.051+0000 7f096b59e700 1 -- 172.21.15.27:0/3337359911 shutdown_connections 2021-11-03T18:47:39.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.051+0000 7f096b59e700 1 -- 172.21.15.27:0/3337359911 wait complete. 2021-11-03T18:47:39.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.052+0000 7f096b59e700 1 Processor -- start 2021-11-03T18:47:39.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.052+0000 7f096b59e700 1 -- start start 2021-11-03T18:47:39.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.052+0000 7f096b59e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f096c0f6530 0x7f096c106d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:47:39.055 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.052+0000 7f096b59e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f096c0f77d0 0x7f096c107270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:47:39.055 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.052+0000 7f096b59e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096c107870 con 0x7f096c0f6530 2021-11-03T18:47:39.055 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.052+0000 7f096b59e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096c108080 con 0x7f096c0f77d0 2021-11-03T18:47:39.055 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.052+0000 7f096a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f096c0f6530 0x7f096c106d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:47:39.055 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.052+0000 7f096a59c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f096c0f6530 0x7f096c106d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36894/0 (socket says 172.21.15.27:36894) 2021-11-03T18:47:39.056 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.052+0000 7f096a59c700 1 -- 172.21.15.27:0/3361083462 learned_addr learned my addr 172.21.15.27:0/3361083462 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:47:39.056 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.052+0000 7f0969d9b700 1 --2- 172.21.15.27:0/3361083462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f096c0f77d0 0x7f096c107270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:47:39.056 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.052+0000 7f096a59c700 1 -- 172.21.15.27:0/3361083462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f096c0f77d0 msgr2=0x7f096c107270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:47:39.056 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.052+0000 7f096a59c700 1 --2- 172.21.15.27:0/3361083462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f096c0f77d0 0x7f096c107270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:39.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.052+0000 7f096a59c700 1 -- 172.21.15.27:0/3361083462 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0954005040 con 0x7f096c0f6530 2021-11-03T18:47:39.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.052+0000 7f0969d9b700 1 --2- 172.21.15.27:0/3361083462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f096c0f77d0 0x7f096c107270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:47:39.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.052+0000 7f096a59c700 1 --2- 172.21.15.27:0/3361083462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f096c0f6530 0x7f096c106d80 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 rx=0x7f096400c810 tx=0x7f0964004d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:47:39.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.052+0000 7f0962ffd700 1 -- 172.21.15.27:0/3361083462 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f096400a700 con 0x7f096c0f6530 2021-11-03T18:47:39.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.053+0000 7f0962ffd700 1 -- 172.21.15.27:0/3361083462 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f096400ad10 con 0x7f096c0f6530 2021-11-03T18:47:39.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.053+0000 7f096b59e700 1 -- 172.21.15.27:0/3361083462 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f096c108300 con 0x7f096c0f6530 2021-11-03T18:47:39.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.053+0000 7f0962ffd700 1 -- 172.21.15.27:0/3361083462 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0964012e50 con 0x7f096c0f6530 2021-11-03T18:47:39.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.053+0000 7f096b59e700 1 -- 172.21.15.27:0/3361083462 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f096c1088c0 con 0x7f096c0f6530 2021-11-03T18:47:39.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.053+0000 7f096b59e700 1 -- 172.21.15.27:0/3361083462 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f096c059100 con 0x7f096c0f6530 2021-11-03T18:47:39.060 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.054+0000 7f0962ffd700 1 -- 172.21.15.27:0/3361083462 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f0964019030 con 0x7f096c0f6530 2021-11-03T18:47:39.060 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.054+0000 7f0962ffd700 1 --2- 172.21.15.27:0/3361083462 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f09580640a0 0x7f0958066560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:47:39.060 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.054+0000 7f0962ffd700 1 -- 172.21.15.27:0/3361083462 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f0964080ae0 con 0x7f096c0f6530 2021-11-03T18:47:39.061 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.054+0000 7f0969d9b700 1 --2- 172.21.15.27:0/3361083462 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f09580640a0 0x7f0958066560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:47:39.061 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.055+0000 7f0969d9b700 1 --2- 172.21.15.27:0/3361083462 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f09580640a0 0x7f0958066560 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 rx=0x7f0954004e30 tx=0x7f0954009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:47:39.061 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.058+0000 7f0962ffd700 1 -- 172.21.15.27:0/3361083462 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f0964052e00 con 0x7f096c0f6530 2021-11-03T18:47:39.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:38 smithi115 ceph-mon[65453]: pgmap v2791: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:39.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:38 smithi027 ceph-mon[97244]: pgmap v2791: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:39.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.289+0000 7f096b59e700 1 -- 172.21.15.27:0/3361083462 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f096c045ad0 con 0x7f096c0f6530 2021-11-03T18:47:39.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.289+0000 7f0962ffd700 1 -- 172.21.15.27:0/3361083462 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f09640172e0 con 0x7f096c0f6530 2021-11-03T18:47:39.288 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:47:39.289 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:47:39.289 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:47:39.289 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:47:39.289 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:47:39.290 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:47:39.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:47:39.290 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:47:39.290 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:47:39.290 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:47:39.291 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:47:39.291 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:47:39.291 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:47:39.291 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:47:39.292 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:47:39.292 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:47:39.292 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:47:39.292 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:47:39.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.291+0000 7f096b59e700 1 -- 172.21.15.27:0/3361083462 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f09580640a0 msgr2=0x7f0958066560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:47:39.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.291+0000 7f096b59e700 1 --2- 172.21.15.27:0/3361083462 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f09580640a0 0x7f0958066560 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 rx=0x7f0954004e30 tx=0x7f0954009b30).stop 2021-11-03T18:47:39.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.291+0000 7f096b59e700 1 -- 172.21.15.27:0/3361083462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f096c0f6530 msgr2=0x7f096c106d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:47:39.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.291+0000 7f096b59e700 1 --2- 172.21.15.27:0/3361083462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f096c0f6530 0x7f096c106d80 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 rx=0x7f096400c810 tx=0x7f0964004d90).stop 2021-11-03T18:47:39.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.291+0000 7f096b59e700 1 -- 172.21.15.27:0/3361083462 shutdown_connections 2021-11-03T18:47:39.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.291+0000 7f096b59e700 1 --2- 172.21.15.27:0/3361083462 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f09580640a0 0x7f0958066560 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:39.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.291+0000 7f096b59e700 1 --2- 172.21.15.27:0/3361083462 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f096c0f6530 0x7f096c106d80 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:39.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.291+0000 7f096b59e700 1 --2- 172.21.15.27:0/3361083462 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f096c0f77d0 0x7f096c107270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:39.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.291+0000 7f096b59e700 1 -- 172.21.15.27:0/3361083462 >> 172.21.15.27:0/3361083462 conn(0x7f096c0f1cd0 msgr2=0x7f096c0fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:47:39.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.291+0000 7f096b59e700 1 -- 172.21.15.27:0/3361083462 shutdown_connections 2021-11-03T18:47:39.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.291+0000 7f096b59e700 1 -- 172.21.15.27:0/3361083462 wait complete. 2021-11-03T18:47:39.397 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.393+0000 7f265976a700 1 -- 172.21.15.27:0/3397201991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26540f5440 msgr2=0x7f26540f5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:47:39.397 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.393+0000 7f265976a700 1 --2- 172.21.15.27:0/3397201991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26540f5440 0x7f26540f5860 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 rx=0x7f2644004660 tx=0x7f2644009b30).stop 2021-11-03T18:47:39.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.393+0000 7f265976a700 1 -- 172.21.15.27:0/3397201991 shutdown_connections 2021-11-03T18:47:39.398 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.393+0000 7f265976a700 1 --2- 172.21.15.27:0/3397201991 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26540f66e0 0x7f26540f6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T18:47:39.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.393+0000 7f265976a700 1 --2- 172.21.15.27:0/3397201991 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26540f5440 0x7f26540f5860 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:39.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.393+0000 7f265976a700 1 -- 172.21.15.27:0/3397201991 >> 172.21.15.27:0/3397201991 conn(0x7f26540f0ba0 msgr2=0x7f26540f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:47:39.400 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.393+0000 7f265976a700 1 -- 172.21.15.27:0/3397201991 shutdown_connections 2021-11-03T18:47:39.400 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.393+0000 7f265976a700 1 -- 172.21.15.27:0/3397201991 wait complete. 2021-11-03T18:47:39.400 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.394+0000 7f265976a700 1 Processor -- start 2021-11-03T18:47:39.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.394+0000 7f265976a700 1 -- start start 2021-11-03T18:47:39.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.394+0000 7f265976a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26540f5440 0x7f2654108f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:47:39.402 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.394+0000 7f265976a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26540f66e0 0x7f2654109450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:47:39.402 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.394+0000 7f265976a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2654109a90 con 0x7f26540f5440 2021-11-03T18:47:39.403 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.394+0000 7f265976a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f265410a340 con 0x7f26540f66e0 2021-11-03T18:47:39.403 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.394+0000 7f2653fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26540f5440 0x7f2654108f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:47:39.403 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.394+0000 7f2653fff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26540f5440 0x7f2654108f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36904/0 (socket says 172.21.15.27:36904) 2021-11-03T18:47:39.404 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.394+0000 7f2653fff700 1 -- 172.21.15.27:0/1416275843 learned_addr learned my addr 172.21.15.27:0/1416275843 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:47:39.404 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.394+0000 7f2653fff700 1 -- 172.21.15.27:0/1416275843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26540f66e0 msgr2=0x7f2654109450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:47:39.405 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.394+0000 7f2653fff700 1 --2- 172.21.15.27:0/1416275843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26540f66e0 0x7f2654109450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:47:39.405 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.394+0000 7f2653fff700 1 -- 172.21.15.27:0/1416275843 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2644005040 con 0x7f26540f5440 2021-11-03T18:47:39.406 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.395+0000 7f2653fff700 1 --2- 172.21.15.27:0/1416275843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26540f5440 0x7f2654108f60 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 rx=0x7f2644004f60 tx=0x7f2644009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:47:39.406 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.395+0000 7f2650ff9700 1 -- 172.21.15.27:0/1416275843 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2644018070 con 0x7f26540f5440 2021-11-03T18:47:39.407 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.395+0000 7f2650ff9700 1 -- 172.21.15.27:0/1416275843 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f2644007ba0 con 0x7f26540f5440 2021-11-03T18:47:39.407 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.395+0000 7f2650ff9700 1 -- 172.21.15.27:0/1416275843 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f264401c620 con 0x7f26540f5440 2021-11-03T18:47:39.407 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.395+0000 7f265976a700 1 -- 172.21.15.27:0/1416275843 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f265410a5c0 con 0x7f26540f5440 2021-11-03T18:47:39.408 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.395+0000 7f265976a700 1 -- 172.21.15.27:0/1416275843 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f265410ab20 con 0x7f26540f5440 2021-11-03T18:47:39.408 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.396+0000 7f265976a700 1 -- 172.21.15.27:0/1416275843 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2654059100 con 0x7f26540f5440 2021-11-03T18:47:39.409 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.396+0000 7f2650ff9700 1 -- 172.21.15.27:0/1416275843 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f264401d420 con 0x7f26540f5440 2021-11-03T18:47:39.409 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.397+0000 7f2650ff9700 1 --2- 172.21.15.27:0/1416275843 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f263c064100 0x7f263c0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:47:39.410 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.397+0000 7f2650ff9700 1 -- 172.21.15.27:0/1416275843 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f26440826d0 con 0x7f26540f5440 2021-11-03T18:47:39.410 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.397+0000 7f26537fe700 1 --2- 172.21.15.27:0/1416275843 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f263c064100 0x7f263c0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:47:39.411 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.397+0000 7f26537fe700 1 --2- 172.21.15.27:0/1416275843 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f263c064100 0x7f263c0665c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 rx=0x7f2648009610 tx=0x7f2648006ab0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:47:39.412 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.401+0000 7f2650ff9700 1 -- 172.21.15.27:0/1416275843 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f2644054a30 con 0x7f26540f5440 2021-11-03T18:47:39.605 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.606+0000 7f265976a700 1 -- 172.21.15.27:0/1416275843 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f265410f580 con 0x7f26540f5440 2021-11-03T18:47:39.605 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.607+0000 7f2650ff9700 1 -- 172.21.15.27:0/1416275843 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f2644054620 con 0x7f26540f5440 2021-11-03T18:47:39.606 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:47:39.606 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:47:39.606 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:47:39.607 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:47:39.607 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:47:39.607 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:47:39.607 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:47:39.608 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:47:39.608 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:47:39.608 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:47:39.608 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:47:39.609 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:47:39.609 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:47:39.609 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:47:39.609 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:47:39.609 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:47:39.610 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:47:39.610 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:47:39.611 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:47:39.611 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:47:39.611 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:47:39.612 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:47:39.612 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:47:39.612 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:47:39.612 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:47:39.612 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:47:39.613 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:47:39.613 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:47:39.613 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:47:39.613 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:47:39.614 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:47:39.614 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:47:39.614 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:47:39.614 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:47:39.615 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:47:39.615 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:47:39.615 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:47:39.615 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:47:39.615 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:47:39.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.609+0000 7f265976a700 1 -- 172.21.15.27:0/1416275843 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f263c064100 msgr2=0x7f263c0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:47:39.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.609+0000 7f265976a700 1 --2- 172.21.15.27:0/1416275843 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f263c064100 0x7f263c0665c0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 rx=0x7f2648009610 tx=0x7f2648006ab0).stop 2021-11-03T18:47:39.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.609+0000 7f265976a700 1 -- 172.21.15.27:0/1416275843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26540f5440 msgr2=0x7f2654108f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:47:39.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.609+0000 7f265976a700 1 --2- 172.21.15.27:0/1416275843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26540f5440 0x7f2654108f60 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 rx=0x7f2644004f60 tx=0x7f2644009b30).stop 2021-11-03T18:47:39.617 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.609+0000 7f265976a700 1 -- 172.21.15.27:0/1416275843 shutdown_connections 2021-11-03T18:47:39.618 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.609+0000 7f265976a700 1 --2- 172.21.15.27:0/1416275843 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f263c064100 0x7f263c0665c0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:39.618 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.609+0000 7f265976a700 1 --2- 172.21.15.27:0/1416275843 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f26540f5440 0x7f2654108f60 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:47:39.618 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.609+0000 7f265976a700 1 --2- 172.21.15.27:0/1416275843 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f26540f66e0 0x7f2654109450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:47:39.618 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.609+0000 7f265976a700 1 -- 172.21.15.27:0/1416275843 >> 172.21.15.27:0/1416275843 conn(0x7f26540f0ba0 msgr2=0x7f265405fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:47:39.619 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.609+0000 7f265976a700 1 -- 172.21.15.27:0/1416275843 shutdown_connections 2021-11-03T18:47:39.619 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:47:39.609+0000 7f265976a700 1 -- 172.21.15.27:0/1416275843 wait complete. 2021-11-03T18:47:39.619 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:47:40.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:39 smithi115 ceph-mon[65453]: from='client.36968 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:47:40.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:39 smithi115 ceph-mon[65453]: from='client.36972 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:47:40.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:39 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3361083462' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:40.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:47:40.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:47:40.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:47:40.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:40.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:40.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:40.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:40.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:39 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1416275843' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:47:40.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:39 smithi027 ceph-mon[97244]: from='client.36968 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:47:40.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:39 smithi027 ceph-mon[97244]: from='client.36972 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:47:40.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:39 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3361083462' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:40.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:47:40.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:47:40.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:47:40.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:40.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:40.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:40.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:40.275 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:39 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1416275843' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:47:41.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:40 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:47:41.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:40 smithi115 ceph-mon[65453]: pgmap v2792: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:41.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5502.492682 secs 2021-11-03T18:47:41.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:40 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:47:41.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:40 smithi027 ceph-mon[97244]: pgmap v2792: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:41.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5502.492682 secs 2021-11-03T18:47:43.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:42 smithi115 ceph-mon[65453]: pgmap v2793: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:43.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:42 smithi027 ceph-mon[97244]: pgmap v2793: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:45.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:44 smithi115 ceph-mon[65453]: pgmap v2794: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:45.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:44 smithi027 ceph-mon[97244]: pgmap v2794: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:46.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5507.492820 secs 2021-11-03T18:47:46.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5507.492820 secs 2021-11-03T18:47:47.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:46 smithi115 ceph-mon[65453]: pgmap v2795: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:47.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:46 smithi027 ceph-mon[97244]: pgmap v2795: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:48.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:47 smithi115 ceph-mon[65453]: pgmap v2796: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:48.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:47 smithi027 ceph-mon[97244]: pgmap v2796: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:49.659 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:47:49.676 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:47:50.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:50 smithi115 ceph-mon[65453]: poll took 20051.503 ms: 1 events 2021-11-03T18:47:50.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:50 smithi115 ceph-mon[65453]: pgmap v2797: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:50.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5512.492909 secs 2021-11-03T18:47:50.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:47:50.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:47:50.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:50.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:50.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:50.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:50.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:50 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:47:51.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:50 smithi027 ceph-mon[97244]: poll took 20051.503 ms: 1 events 2021-11-03T18:47:51.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:50 smithi027 ceph-mon[97244]: pgmap v2797: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:51.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5512.492909 secs 2021-11-03T18:47:51.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:47:51.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:47:51.026 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:51.026 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:51.026 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:51.026 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:47:51.027 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:50 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:47:52.941 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:52 smithi027 ceph-mon[97244]: pgmap v2798: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:52.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:52 smithi115 ceph-mon[65453]: pgmap v2798: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:54.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:54 smithi115 ceph-mon[65453]: pgmap v2799: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:55.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:54 smithi027 ceph-mon[97244]: pgmap v2799: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:55.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5517.492994 secs 2021-11-03T18:47:56.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5517.492994 secs 2021-11-03T18:47:56.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:56 smithi115 ceph-mon[65453]: pgmap v2800: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:56.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:47:56.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:47:57.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:56 smithi027 ceph-mon[97244]: pgmap v2800: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:57.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:47:57.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:47:58.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:47:58 smithi115 ceph-mon[65453]: pgmap v2801: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:47:59.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:47:58 smithi027 ceph-mon[97244]: pgmap v2801: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:00.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:00 smithi115 ceph-mon[65453]: pgmap v2802: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:00.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5522.493131 secs 2021-11-03T18:48:00.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:48:00.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:48:00.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:48:00.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:00.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:00.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:00.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:00.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:00 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:48:01.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:00 smithi027 ceph-mon[97244]: pgmap v2802: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:01.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5522.493131 secs 2021-11-03T18:48:01.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:48:01.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:48:01.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:48:01.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:01.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:01.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:01.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:01.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:00 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:48:02.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:02 smithi115 ceph-mon[65453]: pgmap v2803: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:03.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:02 smithi027 ceph-mon[97244]: pgmap v2803: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:04.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:04 smithi115 ceph-mon[65453]: pgmap v2804: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:05.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:04 smithi027 ceph-mon[97244]: pgmap v2804: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:05.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5527.493247 secs 2021-11-03T18:48:06.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5527.493247 secs 2021-11-03T18:48:06.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:06 smithi115 ceph-mon[65453]: pgmap v2805: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:07.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:06 smithi027 ceph-mon[97244]: pgmap v2805: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:08.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:08 smithi115 ceph-mon[65453]: pgmap v2806: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:09.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:08 smithi027 ceph-mon[97244]: pgmap v2806: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:09.723 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.723+0000 7f4031164700 1 -- 172.21.15.27:0/3342063861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f402c0f5ed0 msgr2=0x7f402c0f6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:48:09.724 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.723+0000 7f4031164700 1 --2- 172.21.15.27:0/3342063861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f402c0f5ed0 0x7f402c0f6900 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 rx=0x7f4020004660 tx=0x7f4020009b30).stop 2021-11-03T18:48:09.724 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.723+0000 7f4031164700 1 -- 172.21.15.27:0/3342063861 shutdown_connections 2021-11-03T18:48:09.724 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.723+0000 7f4031164700 1 --2- 172.21.15.27:0/3342063861 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f402c0f5ed0 0x7f402c0f6900 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:09.725 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.723+0000 7f4031164700 1 --2- 172.21.15.27:0/3342063861 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f402c0f55c0 0x7f402c0f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:09.725 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.723+0000 7f4031164700 1 -- 172.21.15.27:0/3342063861 >> 172.21.15.27:0/3342063861 conn(0x7f402c0f0dc0 msgr2=0x7f402c0f31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:48:09.725 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.724+0000 7f4031164700 1 -- 172.21.15.27:0/3342063861 shutdown_connections 2021-11-03T18:48:09.725 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.724+0000 7f4031164700 1 -- 172.21.15.27:0/3342063861 wait complete. 2021-11-03T18:48:09.725 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.724+0000 7f4031164700 1 Processor -- start 2021-11-03T18:48:09.726 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.724+0000 7f4031164700 1 -- start start 2021-11-03T18:48:09.726 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.724+0000 7f4031164700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f402c0f55c0 0x7f402c0f9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:48:09.726 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.725+0000 7f4031164700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f402c0f5ed0 0x7f402c0f8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:48:09.726 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.725+0000 7f4031164700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402c0f89c0 con 0x7f402c0f55c0 2021-11-03T18:48:09.727 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.725+0000 7f4031164700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402c0f8b00 con 0x7f402c0f5ed0 2021-11-03T18:48:09.727 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.725+0000 7f402ad9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f402c0f55c0 0x7f402c0f9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:48:09.727 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.725+0000 7f402ad9d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f402c0f55c0 0x7f402c0f9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36924/0 (socket says 172.21.15.27:36924) 2021-11-03T18:48:09.727 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.725+0000 7f402ad9d700 1 -- 172.21.15.27:0/377147961 learned_addr learned my addr 172.21.15.27:0/377147961 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:48:09.727 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.725+0000 7f402a59c700 1 --2- 172.21.15.27:0/377147961 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f402c0f5ed0 0x7f402c0f8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:48:09.728 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.725+0000 7f402ad9d700 1 -- 172.21.15.27:0/377147961 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f402c0f5ed0 msgr2=0x7f402c0f8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:48:09.728 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.725+0000 7f402ad9d700 1 --2- 172.21.15.27:0/377147961 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f402c0f5ed0 0x7f402c0f8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:09.728 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.725+0000 7f402ad9d700 1 -- 172.21.15.27:0/377147961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4020005040 con 0x7f402c0f55c0 2021-11-03T18:48:09.728 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.725+0000 7f402ad9d700 1 --2- 172.21.15.27:0/377147961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f402c0f55c0 0x7f402c0f9fe0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 rx=0x7f401c00c530 tx=0x7f401c007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:48:09.730 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.727+0000 7f40137fe700 1 -- 172.21.15.27:0/377147961 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f401c00f650 con 0x7f402c0f55c0 2021-11-03T18:48:09.730 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.727+0000 7f40137fe700 1 -- 172.21.15.27:0/377147961 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f401c00fc60 con 0x7f402c0f55c0 2021-11-03T18:48:09.730 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.727+0000 7f40137fe700 1 -- 172.21.15.27:0/377147961 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f401c012c20 con 0x7f402c0f55c0 2021-11-03T18:48:09.731 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.727+0000 7f4031164700 1 -- 172.21.15.27:0/377147961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f402c0f8de0 con 0x7f402c0f55c0 2021-11-03T18:48:09.731 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.728+0000 7f4031164700 1 -- 172.21.15.27:0/377147961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f402c10ef80 con 0x7f402c0f55c0 2021-11-03T18:48:09.731 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.728+0000 7f4031164700 1 -- 172.21.15.27:0/377147961 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f402c045ad0 con 0x7f402c0f55c0 2021-11-03T18:48:09.731 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.730+0000 7f40137fe700 1 -- 172.21.15.27:0/377147961 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f401c010070 con 0x7f402c0f55c0 2021-11-03T18:48:09.731 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.731+0000 7f40137fe700 1 --2- 172.21.15.27:0/377147961 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f40140640b0 0x7f4014066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:48:09.732 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.731+0000 7f40137fe700 1 -- 172.21.15.27:0/377147961 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f401c080f40 con 0x7f402c0f55c0 2021-11-03T18:48:09.732 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.731+0000 7f402a59c700 1 --2- 172.21.15.27:0/377147961 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f40140640b0 0x7f4014066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:48:09.732 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.731+0000 7f402a59c700 1 --2- 172.21.15.27:0/377147961 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f40140640b0 0x7f4014066570 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 rx=0x7f4020004630 tx=0x7f4020009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:48:09.733 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.734+0000 7f40137fe700 1 -- 172.21.15.27:0/377147961 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f401c053230 con 0x7f402c0f55c0 2021-11-03T18:48:09.915 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.916+0000 7f4031164700 1 -- 172.21.15.27:0/377147961 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f402c05feb0 con 0x7f40140640b0 2021-11-03T18:48:09.916 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.918+0000 7f40137fe700 1 -- 172.21.15.27:0/377147961 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f402c05feb0 con 0x7f40140640b0 2021-11-03T18:48:09.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.919+0000 7f4031164700 1 -- 172.21.15.27:0/377147961 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f40140640b0 msgr2=0x7f4014066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:48:09.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.919+0000 7f4031164700 1 --2- 172.21.15.27:0/377147961 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f40140640b0 0x7f4014066570 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 rx=0x7f4020004630 tx=0x7f4020009b30).stop 2021-11-03T18:48:09.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.919+0000 7f4031164700 1 -- 172.21.15.27:0/377147961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f402c0f55c0 msgr2=0x7f402c0f9fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:48:09.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.919+0000 7f4031164700 1 --2- 172.21.15.27:0/377147961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f402c0f55c0 0x7f402c0f9fe0 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 rx=0x7f401c00c530 tx=0x7f401c007e20).stop 2021-11-03T18:48:09.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.919+0000 7f4031164700 1 -- 172.21.15.27:0/377147961 shutdown_connections 2021-11-03T18:48:09.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.919+0000 7f4031164700 1 --2- 172.21.15.27:0/377147961 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f40140640b0 0x7f4014066570 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:09.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.919+0000 7f4031164700 1 --2- 172.21.15.27:0/377147961 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f402c0f55c0 0x7f402c0f9fe0 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:09.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.919+0000 7f4031164700 1 --2- 172.21.15.27:0/377147961 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f402c0f5ed0 0x7f402c0f8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:09.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.919+0000 7f4031164700 1 -- 172.21.15.27:0/377147961 >> 172.21.15.27:0/377147961 conn(0x7f402c0f0dc0 msgr2=0x7f402c05f7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:48:09.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.919+0000 7f4031164700 1 -- 172.21.15.27:0/377147961 shutdown_connections 2021-11-03T18:48:09.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:09.919+0000 7f4031164700 1 -- 172.21.15.27:0/377147961 wait complete. 2021-11-03T18:48:09.927 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:48:10.020 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.020+0000 7ff84cae1700 1 -- 172.21.15.27:0/501513734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8480f55c0 msgr2=0x7ff8480f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:48:10.021 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.020+0000 7ff84cae1700 1 --2- 172.21.15.27:0/501513734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8480f55c0 0x7ff8480f59e0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 rx=0x7ff830004660 tx=0x7ff830009b30).stop 2021-11-03T18:48:10.021 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.020+0000 7ff84cae1700 1 -- 172.21.15.27:0/501513734 shutdown_connections 2021-11-03T18:48:10.021 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.020+0000 7ff84cae1700 1 --2- 172.21.15.27:0/501513734 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8480f6860 0x7ff8480f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:10.022 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.020+0000 7ff84cae1700 1 --2- 172.21.15.27:0/501513734 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8480f55c0 0x7ff8480f59e0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:10.022 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.020+0000 7ff84cae1700 1 -- 172.21.15.27:0/501513734 >> 172.21.15.27:0/501513734 conn(0x7ff8480f0d40 msgr2=0x7ff8480f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:48:10.022 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.020+0000 7ff84cae1700 1 -- 172.21.15.27:0/501513734 shutdown_connections 2021-11-03T18:48:10.023 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.020+0000 7ff84cae1700 1 -- 172.21.15.27:0/501513734 wait complete. 2021-11-03T18:48:10.023 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.020+0000 7ff84cae1700 1 Processor -- start 2021-11-03T18:48:10.023 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.020+0000 7ff84cae1700 1 -- start start 2021-11-03T18:48:10.023 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.021+0000 7ff84cae1700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8480f55c0 0x7ff848109160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:48:10.024 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.021+0000 7ff84cae1700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8480f6860 0x7ff848109650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:48:10.024 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.021+0000 7ff84cae1700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff848109c00 con 0x7ff8480f6860 2021-11-03T18:48:10.024 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.021+0000 7ff84cae1700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff848109d40 con 0x7ff8480f55c0 2021-11-03T18:48:10.024 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.021+0000 7ff845d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8480f6860 0x7ff848109650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:48:10.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.021+0000 7ff845d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8480f6860 0x7ff848109650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36934/0 (socket says 172.21.15.27:36934) 2021-11-03T18:48:10.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.021+0000 7ff845d9b700 1 -- 172.21.15.27:0/2786334754 learned_addr learned my addr 172.21.15.27:0/2786334754 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:48:10.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.021+0000 7ff84659c700 1 --2- 172.21.15.27:0/2786334754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8480f55c0 0x7ff848109160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:48:10.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.021+0000 7ff845d9b700 1 -- 172.21.15.27:0/2786334754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8480f55c0 msgr2=0x7ff848109160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:48:10.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.021+0000 7ff845d9b700 1 --2- 172.21.15.27:0/2786334754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8480f55c0 0x7ff848109160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:10.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.021+0000 7ff845d9b700 1 -- 172.21.15.27:0/2786334754 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff830005040 con 0x7ff8480f6860 2021-11-03T18:48:10.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.021+0000 7ff84659c700 1 --2- 172.21.15.27:0/2786334754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8480f55c0 0x7ff848109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T18:48:10.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.021+0000 7ff845d9b700 1 --2- 172.21.15.27:0/2786334754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8480f6860 0x7ff848109650 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 rx=0x7ff8380026d0 tx=0x7ff838009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:48:10.026 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.021+0000 7ff83effd700 1 -- 172.21.15.27:0/2786334754 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff838009320 con 0x7ff8480f6860 2021-11-03T18:48:10.027 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.021+0000 7ff83effd700 1 -- 172.21.15.27:0/2786334754 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7ff83800f040 con 0x7ff8480f6860 2021-11-03T18:48:10.027 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.021+0000 7ff84cae1700 1 -- 172.21.15.27:0/2786334754 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff84810a6e0 con 0x7ff8480f6860 2021-11-03T18:48:10.027 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.021+0000 7ff83effd700 1 -- 172.21.15.27:0/2786334754 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8380147e0 con 0x7ff8480f6860 2021-11-03T18:48:10.028 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.021+0000 7ff84cae1700 1 -- 172.21.15.27:0/2786334754 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff84810aca0 con 0x7ff8480f6860 2021-11-03T18:48:10.028 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.022+0000 7ff84cae1700 1 -- 172.21.15.27:0/2786334754 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff848045ad0 con 0x7ff8480f6860 2021-11-03T18:48:10.029 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.023+0000 7ff83effd700 1 -- 172.21.15.27:0/2786334754 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7ff838009480 con 0x7ff8480f6860 2021-11-03T18:48:10.029 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.023+0000 7ff83effd700 1 --2- 172.21.15.27:0/2786334754 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff8340640b0 0x7ff834066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:48:10.029 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.023+0000 7ff83effd700 1 -- 172.21.15.27:0/2786334754 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7ff838080df0 con 0x7ff8480f6860 2021-11-03T18:48:10.030 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.023+0000 7ff84659c700 1 --2- 172.21.15.27:0/2786334754 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff8340640b0 0x7ff834066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:48:10.030 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.024+0000 7ff84659c700 1 --2- 172.21.15.27:0/2786334754 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff8340640b0 0x7ff834066570 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0x7ff830009da0 tx=0x7ff830009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:48:10.030 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.028+0000 7ff83effd700 1 -- 172.21.15.27:0/2786334754 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7ff838053210 con 0x7ff8480f6860 2021-11-03T18:48:10.203 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.204+0000 7ff84cae1700 1 -- 172.21.15.27:0/2786334754 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7ff8480faaa0 con 0x7ff8340640b0 2021-11-03T18:48:10.217 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.219+0000 7ff83effd700 1 -- 172.21.15.27:0/2786334754 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3542 (secure 0 0 0) 0x7ff8480faaa0 con 0x7ff8340640b0 2021-11-03T18:48:10.217 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:48:10.218 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (99m) 8m ago 100m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:48:10.218 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (91m) 8m ago 100m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:48:10.218 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (91m) 7m ago 100m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:48:10.219 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (99m) 8m ago 100m 42.9M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:48:10.219 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (93m) 8m ago 96m 26.9M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:48:10.219 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (96m) 8m ago 96m 3852M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:48:10.219 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (91m) 7m ago 96m 26.5M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:48:10.219 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (93m) 7m ago 96m 27.6M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:48:10.220 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (93m) 8m ago 102m 625M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:48:10.220 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (92m) 7m ago 100m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:48:10.220 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (92m) 8m ago 102m 387M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:48:10.220 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (92m) 7m ago 100m 175M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:48:10.221 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (100m) 8m ago 100m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:48:10.221 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (99m) 7m ago 99m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:48:10.221 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (91m) 8m ago 99m 433M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:48:10.221 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (90m) 8m ago 99m 471M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:48:10.222 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (90m) 8m ago 98m 356M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:48:10.222 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (89m) 7m ago 98m 504M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:48:10.222 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (89m) 7m ago 98m 396M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:48:10.222 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (88m) 7m ago 97m 408M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:48:10.223 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (99m) 8m ago 100m 66.8M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:48:10.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.221+0000 7ff84cae1700 1 -- 172.21.15.27:0/2786334754 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff8340640b0 msgr2=0x7ff834066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:48:10.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.221+0000 7ff84cae1700 1 --2- 172.21.15.27:0/2786334754 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff8340640b0 0x7ff834066570 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0x7ff830009da0 tx=0x7ff830009b30).stop 2021-11-03T18:48:10.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.221+0000 7ff84cae1700 1 -- 172.21.15.27:0/2786334754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8480f6860 msgr2=0x7ff848109650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:48:10.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.221+0000 7ff84cae1700 1 --2- 172.21.15.27:0/2786334754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8480f6860 0x7ff848109650 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 rx=0x7ff8380026d0 tx=0x7ff838009de0).stop 2021-11-03T18:48:10.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.221+0000 7ff84cae1700 1 -- 172.21.15.27:0/2786334754 shutdown_connections 2021-11-03T18:48:10.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.221+0000 7ff84cae1700 1 --2- 172.21.15.27:0/2786334754 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ff8340640b0 0x7ff834066570 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:10.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.221+0000 7ff84cae1700 1 --2- 172.21.15.27:0/2786334754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ff8480f55c0 0x7ff848109160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:10.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.221+0000 7ff84cae1700 1 --2- 172.21.15.27:0/2786334754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ff8480f6860 0x7ff848109650 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:10.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.221+0000 7ff84cae1700 1 -- 172.21.15.27:0/2786334754 >> 172.21.15.27:0/2786334754 conn(0x7ff8480f0d40 msgr2=0x7ff84805fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:48:10.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.221+0000 7ff84cae1700 1 -- 172.21.15.27:0/2786334754 shutdown_connections 2021-11-03T18:48:10.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.221+0000 7ff84cae1700 1 -- 172.21.15.27:0/2786334754 wait complete. 2021-11-03T18:48:10.318 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.319+0000 7ffb61676700 1 -- 172.21.15.27:0/3793039647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb5c0f51a0 msgr2=0x7ffb5c0f7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:48:10.319 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.319+0000 7ffb61676700 1 --2- 172.21.15.27:0/3793039647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb5c0f51a0 0x7ffb5c0f7920 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 rx=0x7ffb50004660 tx=0x7ffb50009b30).stop 2021-11-03T18:48:10.319 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.319+0000 7ffb61676700 1 -- 172.21.15.27:0/3793039647 shutdown_connections 2021-11-03T18:48:10.319 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.319+0000 7ffb61676700 1 --2- 172.21.15.27:0/3793039647 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb5c0f51a0 0x7ffb5c0f7920 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:10.319 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.319+0000 7ffb61676700 1 --2- 172.21.15.27:0/3793039647 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb5c0f4800 0x7ffb5c0f4c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T18:48:10.320 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.319+0000 7ffb61676700 1 -- 172.21.15.27:0/3793039647 >> 172.21.15.27:0/3793039647 conn(0x7ffb5c0f0320 msgr2=0x7ffb5c0f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:48:10.320 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.319+0000 7ffb61676700 1 -- 172.21.15.27:0/3793039647 shutdown_connections 2021-11-03T18:48:10.320 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.320+0000 7ffb61676700 1 -- 172.21.15.27:0/3793039647 wait complete. 2021-11-03T18:48:10.320 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.320+0000 7ffb61676700 1 Processor -- start 2021-11-03T18:48:10.320 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.320+0000 7ffb61676700 1 -- start start 2021-11-03T18:48:10.323 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.320+0000 7ffb61676700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb5c0f4800 0x7ffb5c0f73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:48:10.323 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.320+0000 7ffb61676700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb5c0f51a0 0x7ffb5c0f5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:48:10.324 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.320+0000 7ffb61676700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5c0f5e20 con 0x7ffb5c0f4800 2021-11-03T18:48:10.324 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.320+0000 7ffb61676700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb5c0f5f60 con 0x7ffb5c0f51a0 2021-11-03T18:48:10.324 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.320+0000 7ffb5bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb5c0f4800 0x7ffb5c0f73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:48:10.324 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.321+0000 7ffb5bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb5c0f4800 0x7ffb5c0f73b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36942/0 (socket says 172.21.15.27:36942) 2021-11-03T18:48:10.324 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.321+0000 7ffb5bfff700 1 -- 172.21.15.27:0/2654054148 learned_addr learned my addr 172.21.15.27:0/2654054148 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:48:10.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.321+0000 7ffb5b7fe700 1 --2- 172.21.15.27:0/2654054148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb5c0f51a0 0x7ffb5c0f5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:48:10.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.321+0000 7ffb5bfff700 1 -- 172.21.15.27:0/2654054148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb5c0f51a0 msgr2=0x7ffb5c0f5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:48:10.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.321+0000 7ffb5bfff700 1 --2- 172.21.15.27:0/2654054148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb5c0f51a0 0x7ffb5c0f5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:10.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.321+0000 7ffb5bfff700 1 -- 172.21.15.27:0/2654054148 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb50005040 con 0x7ffb5c0f4800 2021-11-03T18:48:10.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.321+0000 7ffb5bfff700 1 --2- 172.21.15.27:0/2654054148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb5c0f4800 0x7ffb5c0f73b0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 rx=0x7ffb4c0093f0 tx=0x7ffb4c006ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:48:10.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.321+0000 7ffb58ff9700 1 -- 172.21.15.27:0/2654054148 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb4c013640 con 0x7ffb5c0f4800 2021-11-03T18:48:10.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.321+0000 7ffb58ff9700 1 -- 172.21.15.27:0/2654054148 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7ffb4c013c50 con 0x7ffb5c0f4800 2021-11-03T18:48:10.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.321+0000 7ffb61676700 1 -- 172.21.15.27:0/2654054148 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb5c0f6240 con 0x7ffb5c0f4800 2021-11-03T18:48:10.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.321+0000 7ffb61676700 1 -- 172.21.15.27:0/2654054148 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb5c10edd0 con 0x7ffb5c0f4800 2021-11-03T18:48:10.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.321+0000 7ffb58ff9700 1 -- 172.21.15.27:0/2654054148 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb4c012c70 con 0x7ffb5c0f4800 2021-11-03T18:48:10.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.322+0000 7ffb61676700 1 -- 172.21.15.27:0/2654054148 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb3c004fa0 con 0x7ffb5c0f4800 2021-11-03T18:48:10.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.323+0000 7ffb58ff9700 1 -- 172.21.15.27:0/2654054148 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7ffb4c01a060 con 0x7ffb5c0f4800 2021-11-03T18:48:10.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.323+0000 7ffb58ff9700 1 --2- 172.21.15.27:0/2654054148 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffb44064100 0x7ffb440665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:48:10.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.323+0000 7ffb58ff9700 1 -- 172.21.15.27:0/2654054148 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7ffb4c080800 con 0x7ffb5c0f4800 2021-11-03T18:48:10.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.323+0000 7ffb5b7fe700 1 --2- 172.21.15.27:0/2654054148 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffb44064100 0x7ffb440665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:48:10.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.323+0000 7ffb5b7fe700 1 --2- 172.21.15.27:0/2654054148 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffb44064100 0x7ffb440665c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 rx=0x7ffb50004630 tx=0x7ffb50009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:48:10.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.327+0000 7ffb58ff9700 1 -- 172.21.15.27:0/2654054148 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7ffb4c052a10 con 0x7ffb5c0f4800 2021-11-03T18:48:10.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.556+0000 7ffb61676700 1 -- 172.21.15.27:0/2654054148 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7ffb3c005e80 con 0x7ffb5c0f4800 2021-11-03T18:48:10.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.556+0000 7ffb58ff9700 1 -- 172.21.15.27:0/2654054148 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7ffb4c052600 con 0x7ffb5c0f4800 2021-11-03T18:48:10.555 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:48:10.555 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:48:10.556 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:48:10.556 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:48:10.556 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:48:10.556 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:48:10.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:48:10.557 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:48:10.557 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:48:10.557 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:48:10.558 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:48:10.558 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:48:10.558 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:48:10.558 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:48:10.558 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:48:10.559 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:48:10.559 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:48:10.559 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:48:10.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.558+0000 7ffb61676700 1 -- 172.21.15.27:0/2654054148 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffb44064100 msgr2=0x7ffb440665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:48:10.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.558+0000 7ffb61676700 1 --2- 172.21.15.27:0/2654054148 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffb44064100 0x7ffb440665c0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 rx=0x7ffb50004630 tx=0x7ffb50009b30).stop 2021-11-03T18:48:10.560 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.558+0000 7ffb61676700 1 -- 172.21.15.27:0/2654054148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb5c0f4800 msgr2=0x7ffb5c0f73b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:48:10.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.558+0000 7ffb61676700 1 --2- 172.21.15.27:0/2654054148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb5c0f4800 0x7ffb5c0f73b0 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 rx=0x7ffb4c0093f0 tx=0x7ffb4c006ec0).stop 2021-11-03T18:48:10.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.558+0000 7ffb61676700 1 -- 172.21.15.27:0/2654054148 shutdown_connections 2021-11-03T18:48:10.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.558+0000 7ffb61676700 1 --2- 172.21.15.27:0/2654054148 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffb44064100 0x7ffb440665c0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:10.561 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.558+0000 7ffb61676700 1 --2- 172.21.15.27:0/2654054148 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffb5c0f4800 0x7ffb5c0f73b0 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:10.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.558+0000 7ffb61676700 1 --2- 172.21.15.27:0/2654054148 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffb5c0f51a0 0x7ffb5c0f5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:10.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.558+0000 7ffb61676700 1 -- 172.21.15.27:0/2654054148 >> 172.21.15.27:0/2654054148 conn(0x7ffb5c0f0320 msgr2=0x7ffb5c0fb370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:48:10.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.558+0000 7ffb61676700 1 -- 172.21.15.27:0/2654054148 shutdown_connections 2021-11-03T18:48:10.562 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.558+0000 7ffb61676700 1 -- 172.21.15.27:0/2654054148 wait complete. 2021-11-03T18:48:10.658 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.658+0000 7f6e7e329700 1 -- 172.21.15.27:0/2273079079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e780f55c0 msgr2=0x7f6e780f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:48:10.658 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.658+0000 7f6e7e329700 1 --2- 172.21.15.27:0/2273079079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e780f55c0 0x7f6e780f59e0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 rx=0x7f6e68004660 tx=0x7f6e68009b30).stop 2021-11-03T18:48:10.659 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.658+0000 7f6e7e329700 1 -- 172.21.15.27:0/2273079079 shutdown_connections 2021-11-03T18:48:10.659 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.658+0000 7f6e7e329700 1 --2- 172.21.15.27:0/2273079079 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e780f6860 0x7f6e780f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:10.659 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.658+0000 7f6e7e329700 1 --2- 172.21.15.27:0/2273079079 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e780f55c0 0x7f6e780f59e0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:10.659 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.658+0000 7f6e7e329700 1 -- 172.21.15.27:0/2273079079 >> 172.21.15.27:0/2273079079 conn(0x7f6e780f0d40 msgr2=0x7f6e780f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:48:10.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.658+0000 7f6e7e329700 1 -- 172.21.15.27:0/2273079079 shutdown_connections 2021-11-03T18:48:10.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.658+0000 7f6e7e329700 1 -- 172.21.15.27:0/2273079079 wait complete. 2021-11-03T18:48:10.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.658+0000 7f6e7e329700 1 Processor -- start 2021-11-03T18:48:10.660 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.659+0000 7f6e7e329700 1 -- start start 2021-11-03T18:48:10.661 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.659+0000 7f6e7e329700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e780f55c0 0x7f6e780f9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:48:10.661 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.659+0000 7f6e7e329700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e780f6860 0x7f6e780f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:48:10.661 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.659+0000 7f6e7e329700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e780f8100 con 0x7f6e780f6860 2021-11-03T18:48:10.661 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.659+0000 7f6e7e329700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e780f8240 con 0x7f6e780f55c0 2021-11-03T18:48:10.661 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.659+0000 7f6e777fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e780f6860 0x7f6e780f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:48:10.662 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.659+0000 7f6e777fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e780f6860 0x7f6e780f7ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36954/0 (socket says 172.21.15.27:36954) 2021-11-03T18:48:10.662 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.659+0000 7f6e777fe700 1 -- 172.21.15.27:0/706420029 learned_addr learned my addr 172.21.15.27:0/706420029 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:48:10.662 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.659+0000 7f6e77fff700 1 --2- 172.21.15.27:0/706420029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e780f55c0 0x7f6e780f9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:48:10.662 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.659+0000 7f6e777fe700 1 -- 172.21.15.27:0/706420029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e780f55c0 msgr2=0x7f6e780f9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:48:10.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.659+0000 7f6e777fe700 1 --2- 172.21.15.27:0/706420029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e780f55c0 0x7f6e780f9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:10.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.659+0000 7f6e777fe700 1 -- 172.21.15.27:0/706420029 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e68005040 con 0x7f6e780f6860 2021-11-03T18:48:10.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.659+0000 7f6e777fe700 1 --2- 172.21.15.27:0/706420029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e780f6860 0x7f6e780f7ba0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 rx=0x7f6e6c0026d0 tx=0x7f6e6c009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:48:10.663 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.660+0000 7f6e74ff9700 1 -- 172.21.15.27:0/706420029 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e6c009320 con 0x7f6e780f6860 2021-11-03T18:48:10.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.660+0000 7f6e74ff9700 1 -- 172.21.15.27:0/706420029 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f6e6c00f040 con 0x7f6e780f6860 2021-11-03T18:48:10.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.660+0000 7f6e7e329700 1 -- 172.21.15.27:0/706420029 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e780f8520 con 0x7f6e780f6860 2021-11-03T18:48:10.664 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.660+0000 7f6e74ff9700 1 -- 172.21.15.27:0/706420029 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e6c0147e0 con 0x7f6e780f6860 2021-11-03T18:48:10.665 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.660+0000 7f6e7e329700 1 -- 172.21.15.27:0/706420029 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e7810efa0 con 0x7f6e780f6860 2021-11-03T18:48:10.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.661+0000 7f6e7e329700 1 -- 172.21.15.27:0/706420029 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e78045ad0 con 0x7f6e780f6860 2021-11-03T18:48:10.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.662+0000 7f6e74ff9700 1 -- 172.21.15.27:0/706420029 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f6e6c009480 con 0x7f6e780f6860 2021-11-03T18:48:10.666 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.662+0000 7f6e74ff9700 1 --2- 172.21.15.27:0/706420029 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6e600640b0 0x7f6e60066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:48:10.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.662+0000 7f6e74ff9700 1 -- 172.21.15.27:0/706420029 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f6e6c0811d0 con 0x7f6e780f6860 2021-11-03T18:48:10.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.662+0000 7f6e77fff700 1 --2- 172.21.15.27:0/706420029 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6e600640b0 0x7f6e60066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:48:10.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.662+0000 7f6e77fff700 1 --2- 172.21.15.27:0/706420029 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6e600640b0 0x7f6e60066570 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 rx=0x7f6e68004e30 tx=0x7f6e68009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:48:10.667 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.666+0000 7f6e74ff9700 1 -- 172.21.15.27:0/706420029 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f6e6c053530 con 0x7f6e780f6860 2021-11-03T18:48:10.870 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.871+0000 7f6e7e329700 1 -- 172.21.15.27:0/706420029 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f6e780fdd80 con 0x7f6e780f6860 2021-11-03T18:48:10.870 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.872+0000 7f6e74ff9700 1 -- 172.21.15.27:0/706420029 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f6e6c053120 con 0x7f6e780f6860 2021-11-03T18:48:10.871 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:48:10.871 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:48:10.871 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:48:10.872 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:48:10.872 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:48:10.872 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:48:10.872 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:48:10.873 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:48:10.873 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:48:10.873 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:48:10.873 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:48:10.873 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:48:10.874 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:48:10.874 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:48:10.874 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:48:10.874 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:48:10.875 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:48:10.875 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:48:10.875 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:48:10.875 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:48:10.875 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:48:10.876 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:48:10.876 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:48:10.876 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:48:10.876 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:48:10.877 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:48:10.877 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:48:10.877 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:48:10.877 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:48:10.877 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:48:10.878 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:48:10.878 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:48:10.878 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:48:10.878 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:48:10.879 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:48:10.879 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:48:10.879 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:48:10.879 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:48:10.879 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:48:10.880 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.873+0000 7f6e7e329700 1 -- 172.21.15.27:0/706420029 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6e600640b0 msgr2=0x7f6e60066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:48:10.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.873+0000 7f6e7e329700 1 --2- 172.21.15.27:0/706420029 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6e600640b0 0x7f6e60066570 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 rx=0x7f6e68004e30 tx=0x7f6e68009b30).stop 2021-11-03T18:48:10.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.873+0000 7f6e7e329700 1 -- 172.21.15.27:0/706420029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e780f6860 msgr2=0x7f6e780f7ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:48:10.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.873+0000 7f6e7e329700 1 --2- 172.21.15.27:0/706420029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e780f6860 0x7f6e780f7ba0 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 rx=0x7f6e6c0026d0 tx=0x7f6e6c009de0).stop 2021-11-03T18:48:10.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.873+0000 7f6e7e329700 1 -- 172.21.15.27:0/706420029 shutdown_connections 2021-11-03T18:48:10.882 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.874+0000 7f6e7e329700 1 --2- 172.21.15.27:0/706420029 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6e600640b0 0x7f6e60066570 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:10.882 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.874+0000 7f6e7e329700 1 --2- 172.21.15.27:0/706420029 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6e780f55c0 0x7f6e780f9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:10.882 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.874+0000 7f6e7e329700 1 --2- 172.21.15.27:0/706420029 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6e780f6860 0x7f6e780f7ba0 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:10.882 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.874+0000 7f6e7e329700 1 -- 172.21.15.27:0/706420029 >> 172.21.15.27:0/706420029 conn(0x7f6e780f0d40 msgr2=0x7f6e7805fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:48:10.883 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.874+0000 7f6e7e329700 1 -- 172.21.15.27:0/706420029 shutdown_connections 2021-11-03T18:48:10.883 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:10.874+0000 7f6e7e329700 1 -- 172.21.15.27:0/706420029 wait complete. 2021-11-03T18:48:10.883 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:48:10.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:10 smithi115 ceph-mon[65453]: pgmap v2807: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:10.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5532.493391 secs 2021-11-03T18:48:10.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:48:10.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:48:10.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:48:10.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:10.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:10.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:10.990 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:10.991 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:10 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:48:10.991 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:10 smithi115 ceph-mon[65453]: from='client.36984 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:48:10.991 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:10 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2654054148' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:11.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:10 smithi027 ceph-mon[97244]: pgmap v2807: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:11.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5532.493391 secs 2021-11-03T18:48:11.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:48:11.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:48:11.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:48:11.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:11.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:11.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:11.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:11.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:10 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:48:11.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:10 smithi027 ceph-mon[97244]: from='client.36984 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:48:11.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:10 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2654054148' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:11.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:11 smithi115 ceph-mon[65453]: from='client.36988 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:48:11.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:11 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/706420029' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:48:12.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:11 smithi027 ceph-mon[97244]: from='client.36988 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:48:12.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:11 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/706420029' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:48:13.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:12 smithi027 ceph-mon[97244]: pgmap v2808: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:13.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:12 smithi115 ceph-mon[65453]: pgmap v2808: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:15.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:14 smithi027 ceph-mon[97244]: pgmap v2809: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:15.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:14 smithi115 ceph-mon[65453]: pgmap v2809: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:16.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5537.493519 secs 2021-11-03T18:48:16.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5537.493519 secs 2021-11-03T18:48:17.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:16 smithi027 ceph-mon[97244]: pgmap v2810: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:17.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:16 smithi115 ceph-mon[65453]: pgmap v2810: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:19.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:18 smithi027 ceph-mon[97244]: pgmap v2811: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:19.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:18 smithi115 ceph-mon[65453]: pgmap v2811: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:21.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:20 smithi027 ceph-mon[97244]: pgmap v2812: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:21.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5542.493656 secs 2021-11-03T18:48:21.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:48:21.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:48:21.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:48:21.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:21.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:21.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:21.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:21.025 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:20 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:48:21.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:20 smithi115 ceph-mon[65453]: pgmap v2812: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:21.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5542.493656 secs 2021-11-03T18:48:21.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:48:21.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:48:21.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:48:21.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:21.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:21.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:21.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:21.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:20 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:48:23.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:22 smithi027 ceph-mon[97244]: pgmap v2813: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:23.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:22 smithi115 ceph-mon[65453]: pgmap v2813: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:25.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:24 smithi027 ceph-mon[97244]: pgmap v2814: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:25.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:24 smithi115 ceph-mon[65453]: pgmap v2814: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:26.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5547.493737 secs 2021-11-03T18:48:26.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5547.493737 secs 2021-11-03T18:48:27.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:26 smithi027 ceph-mon[97244]: pgmap v2815: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:27.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:26 smithi115 ceph-mon[65453]: pgmap v2815: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:29.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:28 smithi027 ceph-mon[97244]: pgmap v2816: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:29.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:28 smithi115 ceph-mon[65453]: pgmap v2816: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:31.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:30 smithi115 ceph-mon[65453]: pgmap v2817: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:31.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5552.493855 secs 2021-11-03T18:48:31.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:48:31.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:48:31.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:48:31.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:31.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:31.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:31.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:31.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:30 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:48:31.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:30 smithi027 ceph-mon[97244]: pgmap v2817: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:31.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5552.493855 secs 2021-11-03T18:48:31.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:48:31.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:48:31.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:48:31.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:31.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:31.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:31.275 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:31.275 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:30 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:48:33.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:32 smithi115 ceph-mon[65453]: pgmap v2818: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:33.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:32 smithi027 ceph-mon[97244]: pgmap v2818: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:35.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:34 smithi115 ceph-mon[65453]: pgmap v2819: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:35.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:34 smithi027 ceph-mon[97244]: pgmap v2819: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:36.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5557.493920 secs 2021-11-03T18:48:36.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5557.493920 secs 2021-11-03T18:48:37.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:36 smithi115 ceph-mon[65453]: pgmap v2820: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:37.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:36 smithi027 ceph-mon[97244]: pgmap v2820: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:39.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:38 smithi115 ceph-mon[65453]: pgmap v2821: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:39.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:38 smithi027 ceph-mon[97244]: pgmap v2821: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:40.986 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:40.987+0000 7f5af0ff2700 1 -- 172.21.15.27:0/59610531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5aec0f6820 msgr2=0x7f5aec0f6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:48:40.986 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:40.987+0000 7f5af0ff2700 1 --2- 172.21.15.27:0/59610531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5aec0f6820 0x7f5aec0f6c80 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 rx=0x7f5ad4004660 tx=0x7f5ad4009b30).stop 2021-11-03T18:48:40.987 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:40.988+0000 7f5af0ff2700 1 -- 172.21.15.27:0/59610531 shutdown_connections 2021-11-03T18:48:40.987 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:40.988+0000 7f5af0ff2700 1 --2- 172.21.15.27:0/59610531 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5aec0f6820 0x7f5aec0f6c80 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:40.987 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:40.988+0000 7f5af0ff2700 1 --2- 172.21.15.27:0/59610531 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5aec0f5580 0x7f5aec0f59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:40.987 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:40.988+0000 7f5af0ff2700 1 -- 172.21.15.27:0/59610531 >> 172.21.15.27:0/59610531 conn(0x7f5aec0f0ce0 msgr2=0x7f5aec0f3100 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:48:40.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:40.988+0000 7f5af0ff2700 1 -- 172.21.15.27:0/59610531 shutdown_connections 2021-11-03T18:48:40.988 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:40.988+0000 7f5af0ff2700 1 -- 172.21.15.27:0/59610531 wait complete. 2021-11-03T18:48:40.990 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:40.988+0000 7f5af0ff2700 1 Processor -- start 2021-11-03T18:48:40.990 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:40.988+0000 7f5af0ff2700 1 -- start start 2021-11-03T18:48:40.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:40.988+0000 7f5af0ff2700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5aec0f5580 0x7f5aec109bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:48:40.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:40.988+0000 7f5af0ff2700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5aec0f6820 0x7f5aec10c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:48:40.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:40.988+0000 7f5af0ff2700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec0604e0 con 0x7f5aec0f6820 2021-11-03T18:48:40.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:40.988+0000 7f5af0ff2700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aec060620 con 0x7f5aec0f5580 2021-11-03T18:48:40.992 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:40.988+0000 7f5ae9d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5aec0f6820 0x7f5aec10c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:48:40.992 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:40.989+0000 7f5ae9d9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5aec0f6820 0x7f5aec10c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36976/0 (socket says 172.21.15.27:36976) 2021-11-03T18:48:40.992 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:40.989+0000 7f5ae9d9b700 1 -- 172.21.15.27:0/3708418002 learned_addr learned my addr 172.21.15.27:0/3708418002 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:48:40.992 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:40.989+0000 7f5aea59c700 1 --2- 172.21.15.27:0/3708418002 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5aec0f5580 0x7f5aec109bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:48:40.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:40.989+0000 7f5ae9d9b700 1 -- 172.21.15.27:0/3708418002 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5aec0f5580 msgr2=0x7f5aec109bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:48:40.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:40.989+0000 7f5ae9d9b700 1 --2- 172.21.15.27:0/3708418002 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5aec0f5580 0x7f5aec109bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:40.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:40.989+0000 7f5ae9d9b700 1 -- 172.21.15.27:0/3708418002 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ad4005040 con 0x7f5aec0f6820 2021-11-03T18:48:40.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:40.989+0000 7f5ae9d9b700 1 --2- 172.21.15.27:0/3708418002 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5aec0f6820 0x7f5aec10c0b0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 rx=0x7f5ad4009990 tx=0x7f5ad4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:48:40.993 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:40.989+0000 7f5ae2ffd700 1 -- 172.21.15.27:0/3708418002 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ad4018070 con 0x7f5aec0f6820 2021-11-03T18:48:40.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:40.989+0000 7f5ae2ffd700 1 -- 172.21.15.27:0/3708418002 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f5ad4007ba0 con 0x7f5aec0f6820 2021-11-03T18:48:40.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:40.989+0000 7f5af0ff2700 1 -- 172.21.15.27:0/3708418002 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aec10c600 con 0x7f5aec0f6820 2021-11-03T18:48:40.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:40.989+0000 7f5ae2ffd700 1 -- 172.21.15.27:0/3708418002 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5ad401c3c0 con 0x7f5aec0f6820 2021-11-03T18:48:40.994 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:40.989+0000 7f5af0ff2700 1 -- 172.21.15.27:0/3708418002 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aec10cb30 con 0x7f5aec0f6820 2021-11-03T18:48:40.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:40.990+0000 7f5af0ff2700 1 -- 172.21.15.27:0/3708418002 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aec045ad0 con 0x7f5aec0f6820 2021-11-03T18:48:40.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:40.991+0000 7f5ae2ffd700 1 -- 172.21.15.27:0/3708418002 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f5ad401d420 con 0x7f5aec0f6820 2021-11-03T18:48:40.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:40.991+0000 7f5ae2ffd700 1 --2- 172.21.15.27:0/3708418002 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5ad80641c0 0x7f5ad8066680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:48:40.995 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:40.991+0000 7f5ae2ffd700 1 -- 172.21.15.27:0/3708418002 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f5ad4082450 con 0x7f5aec0f6820 2021-11-03T18:48:40.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:40.991+0000 7f5aea59c700 1 --2- 172.21.15.27:0/3708418002 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5ad80641c0 0x7f5ad8066680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:48:40.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:40.992+0000 7f5aea59c700 1 --2- 172.21.15.27:0/3708418002 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5ad80641c0 0x7f5ad8066680 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 rx=0x7f5aec060160 tx=0x7f5adc00b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:48:40.996 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:40.995+0000 7f5ae2ffd700 1 -- 172.21.15.27:0/3708418002 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f5ad40547b0 con 0x7f5aec0f6820 2021-11-03T18:48:41.177 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.178+0000 7f5af0ff2700 1 -- 172.21.15.27:0/3708418002 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5aec057f10 con 0x7f5ad80641c0 2021-11-03T18:48:41.178 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.180+0000 7f5ae2ffd700 1 -- 172.21.15.27:0/3708418002 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f5aec057f10 con 0x7f5ad80641c0 2021-11-03T18:48:41.180 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.181+0000 7f5af0ff2700 1 -- 172.21.15.27:0/3708418002 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5ad80641c0 msgr2=0x7f5ad8066680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:48:41.180 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.181+0000 7f5af0ff2700 1 --2- 172.21.15.27:0/3708418002 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5ad80641c0 0x7f5ad8066680 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 rx=0x7f5aec060160 tx=0x7f5adc00b040).stop 2021-11-03T18:48:41.181 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.182+0000 7f5af0ff2700 1 -- 172.21.15.27:0/3708418002 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5aec0f6820 msgr2=0x7f5aec10c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:48:41.181 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.182+0000 7f5af0ff2700 1 --2- 172.21.15.27:0/3708418002 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5aec0f6820 0x7f5aec10c0b0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 rx=0x7f5ad4009990 tx=0x7f5ad4009b30).stop 2021-11-03T18:48:41.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.182+0000 7f5af0ff2700 1 -- 172.21.15.27:0/3708418002 shutdown_connections 2021-11-03T18:48:41.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.182+0000 7f5af0ff2700 1 --2- 172.21.15.27:0/3708418002 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5ad80641c0 0x7f5ad8066680 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:41.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.182+0000 7f5af0ff2700 1 --2- 172.21.15.27:0/3708418002 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5aec0f5580 0x7f5aec109bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:41.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.182+0000 7f5af0ff2700 1 --2- 172.21.15.27:0/3708418002 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5aec0f6820 0x7f5aec10c0b0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:41.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.182+0000 7f5af0ff2700 1 -- 172.21.15.27:0/3708418002 >> 172.21.15.27:0/3708418002 conn(0x7f5aec0f0ce0 msgr2=0x7f5aec0fa270 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:48:41.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.182+0000 7f5af0ff2700 1 -- 172.21.15.27:0/3708418002 shutdown_connections 2021-11-03T18:48:41.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.182+0000 7f5af0ff2700 1 -- 172.21.15.27:0/3708418002 wait complete. 2021-11-03T18:48:41.190 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:48:41.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:40 smithi115 ceph-mon[65453]: pgmap v2822: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:41.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5562.494062 secs 2021-11-03T18:48:41.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:48:41.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:40 smithi115 ceph-mon[65453]: poll took 50148.178 ms: 1 events 2021-11-03T18:48:41.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:48:41.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:48:41.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:48:41.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:41.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:41.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:41.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:41.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:40 smithi027 ceph-mon[97244]: pgmap v2822: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:41.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5562.494062 secs 2021-11-03T18:48:41.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:48:41.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:40 smithi027 ceph-mon[97244]: poll took 50148.178 ms: 1 events 2021-11-03T18:48:41.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:48:41.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:48:41.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:48:41.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:41.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:41.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:41.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:41.280 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.282+0000 7f53c60db700 1 -- 172.21.15.27:0/2749991170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c00f7590 msgr2=0x7f53c00f59b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:48:41.281 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.282+0000 7f53c60db700 1 --2- 172.21.15.27:0/2749991170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c00f7590 0x7f53c00f59b0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 rx=0x7f53b4004660 tx=0x7f53b4009b30).stop 2021-11-03T18:48:41.281 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.282+0000 7f53c60db700 1 -- 172.21.15.27:0/2749991170 shutdown_connections 2021-11-03T18:48:41.281 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.282+0000 7f53c60db700 1 --2- 172.21.15.27:0/2749991170 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c00f7590 0x7f53c00f59b0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:41.281 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.282+0000 7f53c60db700 1 --2- 172.21.15.27:0/2749991170 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53c00f6bf0 0x7f53c00f7010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:41.282 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.282+0000 7f53c60db700 1 -- 172.21.15.27:0/2749991170 >> 172.21.15.27:0/2749991170 conn(0x7f53c00f04c0 msgr2=0x7f53c00f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:48:41.282 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.282+0000 7f53c60db700 1 -- 172.21.15.27:0/2749991170 shutdown_connections 2021-11-03T18:48:41.282 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.282+0000 7f53c60db700 1 -- 172.21.15.27:0/2749991170 wait complete. 2021-11-03T18:48:41.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.283+0000 7f53c60db700 1 Processor -- start 2021-11-03T18:48:41.284 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.283+0000 7f53c60db700 1 -- start start 2021-11-03T18:48:41.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.283+0000 7f53c60db700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53c00f6bf0 0x7f53c0106f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:48:41.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.283+0000 7f53c60db700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c00f7590 0x7f53c0107440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:48:41.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.283+0000 7f53c60db700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c01079f0 con 0x7f53c00f7590 2021-11-03T18:48:41.285 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.283+0000 7f53c60db700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53c0107b30 con 0x7f53c00f6bf0 2021-11-03T18:48:41.286 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.283+0000 7f53beffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c00f7590 0x7f53c0107440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:48:41.286 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.283+0000 7f53beffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c00f7590 0x7f53c0107440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36986/0 (socket says 172.21.15.27:36986) 2021-11-03T18:48:41.286 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.283+0000 7f53beffd700 1 -- 172.21.15.27:0/2139975661 learned_addr learned my addr 172.21.15.27:0/2139975661 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:48:41.286 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.283+0000 7f53beffd700 1 -- 172.21.15.27:0/2139975661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53c00f6bf0 msgr2=0x7f53c0106f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:48:41.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.283+0000 7f53beffd700 1 --2- 172.21.15.27:0/2139975661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53c00f6bf0 0x7f53c0106f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:48:41.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.283+0000 7f53beffd700 1 -- 172.21.15.27:0/2139975661 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53b4005040 con 0x7f53c00f7590 2021-11-03T18:48:41.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.283+0000 7f53beffd700 1 --2- 172.21.15.27:0/2139975661 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c00f7590 0x7f53c0107440 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 rx=0x7f53b4009990 tx=0x7f53b4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:48:41.287 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.284+0000 7f53a7fff700 1 -- 172.21.15.27:0/2139975661 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53b4018070 con 0x7f53c00f7590 2021-11-03T18:48:41.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.284+0000 7f53c60db700 1 -- 172.21.15.27:0/2139975661 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53c0108470 con 0x7f53c00f7590 2021-11-03T18:48:41.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.284+0000 7f53a7fff700 1 -- 172.21.15.27:0/2139975661 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f53b4007ca0 con 0x7f53c00f7590 2021-11-03T18:48:41.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.284+0000 7f53a7fff700 1 -- 172.21.15.27:0/2139975661 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53b401c620 con 0x7f53c00f7590 2021-11-03T18:48:41.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.284+0000 7f53c60db700 1 -- 172.21.15.27:0/2139975661 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53c01089d0 con 0x7f53c00f7590 2021-11-03T18:48:41.288 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.285+0000 7f53c60db700 1 -- 172.21.15.27:0/2139975661 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53c0045ad0 con 0x7f53c00f7590 2021-11-03T18:48:41.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.285+0000 7f53a7fff700 1 -- 172.21.15.27:0/2139975661 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f53b401d960 con 0x7f53c00f7590 2021-11-03T18:48:41.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.285+0000 7f53a7fff700 1 --2- 172.21.15.27:0/2139975661 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f53a80640b0 0x7f53a8066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:48:41.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.286+0000 7f53a7fff700 1 -- 172.21.15.27:0/2139975661 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f53b4082520 con 0x7f53c00f7590 2021-11-03T18:48:41.289 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.286+0000 7f53bf7fe700 1 --2- 172.21.15.27:0/2139975661 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f53a80640b0 0x7f53a8066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:48:41.290 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.286+0000 7f53bf7fe700 1 --2- 172.21.15.27:0/2139975661 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f53a80640b0 0x7f53a8066570 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 rx=0x7f53c005f4e0 tx=0x7f53b0007a60).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:48:41.290 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.290+0000 7f53a7fff700 1 -- 172.21.15.27:0/2139975661 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f53b4054940 con 0x7f53c00f7590 2021-11-03T18:48:41.460 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.461+0000 7f53c60db700 1 -- 172.21.15.27:0/2139975661 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f53c00fade0 con 0x7f53a80640b0 2021-11-03T18:48:41.474 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.476+0000 7f53a7fff700 1 -- 172.21.15.27:0/2139975661 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3542 (secure 0 0 0) 0x7f53c00fade0 con 0x7f53a80640b0 2021-11-03T18:48:41.475 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:48:41.475 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (100m) 8m ago 101m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:48:41.475 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (91m) 8m ago 101m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:48:41.476 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (91m) 8m ago 100m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:48:41.476 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (100m) 8m ago 101m 42.9M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:48:41.476 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (93m) 8m ago 96m 26.9M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:48:41.476 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (96m) 8m ago 96m 3852M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:48:41.477 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (92m) 8m ago 96m 26.5M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:48:41.477 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (94m) 8m ago 96m 27.6M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:48:41.477 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (93m) 8m ago 102m 625M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:48:41.477 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (93m) 8m ago 100m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:48:41.478 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (92m) 8m ago 102m 387M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:48:41.478 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (92m) 8m ago 100m 175M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:48:41.478 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (101m) 8m ago 101m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:48:41.478 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (100m) 8m ago 100m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:48:41.479 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (91m) 8m ago 99m 433M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:48:41.479 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (91m) 8m ago 99m 471M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:48:41.479 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (90m) 8m ago 99m 356M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:48:41.480 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (90m) 8m ago 98m 504M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:48:41.480 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (89m) 8m ago 98m 396M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:48:41.480 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (89m) 8m ago 98m 408M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:48:41.480 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (99m) 8m ago 100m 66.8M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:48:41.481 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.478+0000 7f53c60db700 1 -- 172.21.15.27:0/2139975661 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f53a80640b0 msgr2=0x7f53a8066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:48:41.482 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.478+0000 7f53c60db700 1 --2- 172.21.15.27:0/2139975661 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f53a80640b0 0x7f53a8066570 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 rx=0x7f53c005f4e0 tx=0x7f53b0007a60).stop 2021-11-03T18:48:41.482 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.478+0000 7f53c60db700 1 -- 172.21.15.27:0/2139975661 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c00f7590 msgr2=0x7f53c0107440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:48:41.482 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.478+0000 7f53c60db700 1 --2- 172.21.15.27:0/2139975661 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c00f7590 0x7f53c0107440 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 rx=0x7f53b4009990 tx=0x7f53b4009b30).stop 2021-11-03T18:48:41.482 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.478+0000 7f53c60db700 1 -- 172.21.15.27:0/2139975661 shutdown_connections 2021-11-03T18:48:41.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.478+0000 7f53c60db700 1 --2- 172.21.15.27:0/2139975661 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f53a80640b0 0x7f53a8066570 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:41.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.478+0000 7f53c60db700 1 --2- 172.21.15.27:0/2139975661 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f53c00f6bf0 0x7f53c0106f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:48:41.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.478+0000 7f53c60db700 1 --2- 172.21.15.27:0/2139975661 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f53c00f7590 0x7f53c0107440 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:41.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.478+0000 7f53c60db700 1 -- 172.21.15.27:0/2139975661 >> 172.21.15.27:0/2139975661 conn(0x7f53c00f04c0 msgr2=0x7f53c00f50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:48:41.484 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.478+0000 7f53c60db700 1 -- 172.21.15.27:0/2139975661 shutdown_connections 2021-11-03T18:48:41.484 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.478+0000 7f53c60db700 1 -- 172.21.15.27:0/2139975661 wait complete. 2021-11-03T18:48:41.577 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.577+0000 7fbcae3c5700 1 -- 172.21.15.27:0/2799858224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbca80f6860 msgr2=0x7fbca80f6cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:48:41.577 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.577+0000 7fbcae3c5700 1 --2- 172.21.15.27:0/2799858224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbca80f6860 0x7fbca80f6cc0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 rx=0x7fbc9c004660 tx=0x7fbc9c009b30).stop 2021-11-03T18:48:41.578 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.577+0000 7fbcae3c5700 1 -- 172.21.15.27:0/2799858224 shutdown_connections 2021-11-03T18:48:41.578 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.577+0000 7fbcae3c5700 1 --2- 172.21.15.27:0/2799858224 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbca80f6860 0x7fbca80f6cc0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:41.578 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.577+0000 7fbcae3c5700 1 --2- 172.21.15.27:0/2799858224 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbca80f55c0 0x7fbca80f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:41.578 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.577+0000 7fbcae3c5700 1 -- 172.21.15.27:0/2799858224 >> 172.21.15.27:0/2799858224 conn(0x7fbca80f0d40 msgr2=0x7fbca80f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:48:41.579 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.577+0000 7fbcae3c5700 1 -- 172.21.15.27:0/2799858224 shutdown_connections 2021-11-03T18:48:41.579 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.577+0000 7fbcae3c5700 1 -- 172.21.15.27:0/2799858224 wait complete. 2021-11-03T18:48:41.579 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.577+0000 7fbcae3c5700 1 Processor -- start 2021-11-03T18:48:41.579 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.577+0000 7fbcae3c5700 1 -- start start 2021-11-03T18:48:41.580 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.578+0000 7fbcae3c5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbca80f55c0 0x7fbca8109c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:48:41.580 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.578+0000 7fbcae3c5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbca80f6860 0x7fbca810c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:48:41.580 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.578+0000 7fbcae3c5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca80603c0 con 0x7fbca80f6860 2021-11-03T18:48:41.580 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.578+0000 7fbcae3c5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca8060500 con 0x7fbca80f55c0 2021-11-03T18:48:41.581 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.578+0000 7fbca77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbca80f6860 0x7fbca810c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:48:41.581 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.578+0000 7fbca77fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbca80f6860 0x7fbca810c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:36996/0 (socket says 172.21.15.27:36996) 2021-11-03T18:48:41.581 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.578+0000 7fbca77fe700 1 -- 172.21.15.27:0/336502657 learned_addr learned my addr 172.21.15.27:0/336502657 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:48:41.582 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.578+0000 7fbca7fff700 1 --2- 172.21.15.27:0/336502657 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbca80f55c0 0x7fbca8109c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:48:41.582 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.578+0000 7fbca77fe700 1 -- 172.21.15.27:0/336502657 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbca80f55c0 msgr2=0x7fbca8109c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:48:41.582 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.578+0000 7fbca77fe700 1 --2- 172.21.15.27:0/336502657 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbca80f55c0 0x7fbca8109c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:41.582 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.578+0000 7fbca77fe700 1 -- 172.21.15.27:0/336502657 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc9c005040 con 0x7fbca80f6860 2021-11-03T18:48:41.583 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.578+0000 7fbca77fe700 1 --2- 172.21.15.27:0/336502657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbca80f6860 0x7fbca810c170 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 rx=0x7fbc9c009990 tx=0x7fbc9c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:48:41.583 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.578+0000 7fbca4ff9700 1 -- 172.21.15.27:0/336502657 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc9c018070 con 0x7fbca80f6860 2021-11-03T18:48:41.583 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.578+0000 7fbca4ff9700 1 -- 172.21.15.27:0/336502657 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fbc9c007ba0 con 0x7fbca80f6860 2021-11-03T18:48:41.583 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.578+0000 7fbcae3c5700 1 -- 172.21.15.27:0/336502657 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbca810c6c0 con 0x7fbca80f6860 2021-11-03T18:48:41.583 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.578+0000 7fbca4ff9700 1 -- 172.21.15.27:0/336502657 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc9c01c3c0 con 0x7fbca80f6860 2021-11-03T18:48:41.584 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.578+0000 7fbcae3c5700 1 -- 172.21.15.27:0/336502657 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca810cbf0 con 0x7fbca80f6860 2021-11-03T18:48:41.585 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.579+0000 7fbcae3c5700 1 -- 172.21.15.27:0/336502657 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbca80f49b0 con 0x7fbca80f6860 2021-11-03T18:48:41.585 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.580+0000 7fbca4ff9700 1 -- 172.21.15.27:0/336502657 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fbc9c01d420 con 0x7fbca80f6860 2021-11-03T18:48:41.585 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.580+0000 7fbca4ff9700 1 --2- 172.21.15.27:0/336502657 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbc900640b0 0x7fbc90066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:48:41.586 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.580+0000 7fbca4ff9700 1 -- 172.21.15.27:0/336502657 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fbc9c0824e0 con 0x7fbca80f6860 2021-11-03T18:48:41.586 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.580+0000 7fbca7fff700 1 --2- 172.21.15.27:0/336502657 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbc900640b0 0x7fbc90066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:48:41.586 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.581+0000 7fbca7fff700 1 --2- 172.21.15.27:0/336502657 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbc900640b0 0x7fbc90066570 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 rx=0x7fbca8060040 tx=0x7fbc98009c00).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:48:41.586 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.584+0000 7fbca4ff9700 1 -- 172.21.15.27:0/336502657 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fbc9c0548a0 con 0x7fbca80f6860 2021-11-03T18:48:41.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.815+0000 7fbcae3c5700 1 -- 172.21.15.27:0/336502657 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbca8045ad0 con 0x7fbca80f6860 2021-11-03T18:48:41.814 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.815+0000 7fbca4ff9700 1 -- 172.21.15.27:0/336502657 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fbc9c054490 con 0x7fbca80f6860 2021-11-03T18:48:41.814 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:48:41.815 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:48:41.815 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:48:41.815 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:48:41.815 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:48:41.816 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:48:41.816 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:48:41.816 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:48:41.816 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:48:41.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:48:41.817 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:48:41.817 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:48:41.817 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:48:41.818 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:48:41.818 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:48:41.818 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:48:41.818 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:48:41.818 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:48:41.819 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.817+0000 7fbcae3c5700 1 -- 172.21.15.27:0/336502657 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbc900640b0 msgr2=0x7fbc90066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:48:41.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.817+0000 7fbcae3c5700 1 --2- 172.21.15.27:0/336502657 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbc900640b0 0x7fbc90066570 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 rx=0x7fbca8060040 tx=0x7fbc98009c00).stop 2021-11-03T18:48:41.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.817+0000 7fbcae3c5700 1 -- 172.21.15.27:0/336502657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbca80f6860 msgr2=0x7fbca810c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:48:41.820 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.817+0000 7fbcae3c5700 1 --2- 172.21.15.27:0/336502657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbca80f6860 0x7fbca810c170 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 rx=0x7fbc9c009990 tx=0x7fbc9c009b30).stop 2021-11-03T18:48:41.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.817+0000 7fbcae3c5700 1 -- 172.21.15.27:0/336502657 shutdown_connections 2021-11-03T18:48:41.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.817+0000 7fbcae3c5700 1 --2- 172.21.15.27:0/336502657 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbc900640b0 0x7fbc90066570 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:41.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.817+0000 7fbcae3c5700 1 --2- 172.21.15.27:0/336502657 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbca80f55c0 0x7fbca8109c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:41.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.817+0000 7fbcae3c5700 1 --2- 172.21.15.27:0/336502657 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbca80f6860 0x7fbca810c170 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:41.821 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.817+0000 7fbcae3c5700 1 -- 172.21.15.27:0/336502657 >> 172.21.15.27:0/336502657 conn(0x7fbca80f0d40 msgr2=0x7fbca80fa2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:48:41.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.817+0000 7fbcae3c5700 1 -- 172.21.15.27:0/336502657 shutdown_connections 2021-11-03T18:48:41.822 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.818+0000 7fbcae3c5700 1 -- 172.21.15.27:0/336502657 wait complete. 2021-11-03T18:48:41.918 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.918+0000 7f48334d3700 1 -- 172.21.15.27:0/2314383872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f482c0f66e0 msgr2=0x7f482c0f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:48:41.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.918+0000 7f48334d3700 1 --2- 172.21.15.27:0/2314383872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f482c0f66e0 0x7f482c0f6b00 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 rx=0x7f481c004660 tx=0x7f481c009b30).stop 2021-11-03T18:48:41.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.918+0000 7f48334d3700 1 -- 172.21.15.27:0/2314383872 shutdown_connections 2021-11-03T18:48:41.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.918+0000 7f48334d3700 1 --2- 172.21.15.27:0/2314383872 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f482c0f7980 0x7f482c0f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T18:48:41.919 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.918+0000 7f48334d3700 1 --2- 172.21.15.27:0/2314383872 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f482c0f66e0 0x7f482c0f6b00 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:41.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.918+0000 7f48334d3700 1 -- 172.21.15.27:0/2314383872 >> 172.21.15.27:0/2314383872 conn(0x7f482c0f1e40 msgr2=0x7f482c0f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:48:41.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.918+0000 7f48334d3700 1 -- 172.21.15.27:0/2314383872 shutdown_connections 2021-11-03T18:48:41.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.918+0000 7f48334d3700 1 -- 172.21.15.27:0/2314383872 wait complete. 2021-11-03T18:48:41.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.919+0000 7f48334d3700 1 Processor -- start 2021-11-03T18:48:41.920 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.919+0000 7f48334d3700 1 -- start start 2021-11-03T18:48:41.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.919+0000 7f48334d3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f482c0f66e0 0x7f482c06b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:48:41.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.919+0000 7f48334d3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f482c0f7980 0x7f482c069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:48:41.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.919+0000 7f48334d3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482c06a410 con 0x7f482c0f7980 2021-11-03T18:48:41.921 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.919+0000 7f48334d3700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482c06a550 con 0x7f482c0f66e0 2021-11-03T18:48:41.922 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.919+0000 7f4830a6e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f482c0f7980 0x7f482c069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:48:41.922 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.919+0000 7f4830a6e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f482c0f7980 0x7f482c069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37006/0 (socket says 172.21.15.27:37006) 2021-11-03T18:48:41.922 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.919+0000 7f4830a6e700 1 -- 172.21.15.27:0/2139562193 learned_addr learned my addr 172.21.15.27:0/2139562193 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:48:41.922 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.919+0000 7f483126f700 1 --2- 172.21.15.27:0/2139562193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f482c0f66e0 0x7f482c06b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:48:41.923 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.919+0000 7f4830a6e700 1 -- 172.21.15.27:0/2139562193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f482c0f66e0 msgr2=0x7f482c06b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:48:41.923 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.919+0000 7f4830a6e700 1 --2- 172.21.15.27:0/2139562193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f482c0f66e0 0x7f482c06b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:41.923 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.919+0000 7f4830a6e700 1 -- 172.21.15.27:0/2139562193 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f481c005040 con 0x7f482c0f7980 2021-11-03T18:48:41.923 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.919+0000 7f4830a6e700 1 --2- 172.21.15.27:0/2139562193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f482c0f7980 0x7f482c069e20 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 rx=0x7f482800b410 tx=0x7f4828006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:48:41.923 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.919+0000 7f4821ffb700 1 -- 172.21.15.27:0/2139562193 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48280125f0 con 0x7f482c0f7980 2021-11-03T18:48:41.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.920+0000 7f48334d3700 1 -- 172.21.15.27:0/2139562193 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f482c06a830 con 0x7f482c0f7980 2021-11-03T18:48:41.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.920+0000 7f4821ffb700 1 -- 172.21.15.27:0/2139562193 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f4828012c00 con 0x7f482c0f7980 2021-11-03T18:48:41.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.920+0000 7f4821ffb700 1 -- 172.21.15.27:0/2139562193 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4828002ae0 con 0x7f482c0f7980 2021-11-03T18:48:41.924 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.920+0000 7f48334d3700 1 -- 172.21.15.27:0/2139562193 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f482c10efe0 con 0x7f482c0f7980 2021-11-03T18:48:41.926 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.921+0000 7f48334d3700 1 -- 172.21.15.27:0/2139562193 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f482c045ad0 con 0x7f482c0f7980 2021-11-03T18:48:41.926 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.921+0000 7f4821ffb700 1 -- 172.21.15.27:0/2139562193 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f482800f030 con 0x7f482c0f7980 2021-11-03T18:48:41.926 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.921+0000 7f4821ffb700 1 --2- 172.21.15.27:0/2139562193 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f48180640b0 0x7f4818066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:48:41.927 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.921+0000 7f4821ffb700 1 -- 172.21.15.27:0/2139562193 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f4828080c60 con 0x7f482c0f7980 2021-11-03T18:48:41.927 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.921+0000 7f483126f700 1 --2- 172.21.15.27:0/2139562193 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f48180640b0 0x7f4818066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:48:41.927 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.922+0000 7f483126f700 1 --2- 172.21.15.27:0/2139562193 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f48180640b0 0x7f4818066570 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 rx=0x7f481c009da0 tx=0x7f481c009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:48:41.927 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:41.926+0000 7f4821ffb700 1 -- 172.21.15.27:0/2139562193 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f4828052fc0 con 0x7f482c0f7980 2021-11-03T18:48:42.127 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:42.129+0000 7f48334d3700 1 -- 172.21.15.27:0/2139562193 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f482c027a00 con 0x7f482c0f7980 2021-11-03T18:48:42.128 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:42.129+0000 7f4821ffb700 1 -- 172.21.15.27:0/2139562193 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f4828052bb0 con 0x7f482c0f7980 2021-11-03T18:48:42.129 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:48:42.129 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:48:42.129 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:48:42.129 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:48:42.130 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:48:42.131 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:48:42.131 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:48:42.131 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:48:42.131 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:48:42.132 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:48:42.132 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:48:42.132 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:48:42.132 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:48:42.133 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:48:42.133 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:48:42.133 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:48:42.133 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:48:42.134 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:48:42.134 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:48:42.134 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:48:42.134 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:48:42.134 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:48:42.135 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:48:42.135 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:48:42.135 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:48:42.135 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:48:42.136 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:48:42.136 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:48:42.136 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:48:42.136 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:48:42.137 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:48:42.137 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:48:42.137 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:48:42.137 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:48:42.138 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:48:42.138 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:48:42.138 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:48:42.138 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:48:42.138 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:48:42.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:42.131+0000 7f48334d3700 1 -- 172.21.15.27:0/2139562193 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f48180640b0 msgr2=0x7f4818066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:48:42.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:42.131+0000 7f48334d3700 1 --2- 172.21.15.27:0/2139562193 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f48180640b0 0x7f4818066570 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 rx=0x7f481c009da0 tx=0x7f481c009b30).stop 2021-11-03T18:48:42.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:42.131+0000 7f48334d3700 1 -- 172.21.15.27:0/2139562193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f482c0f7980 msgr2=0x7f482c069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:48:42.140 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:42.131+0000 7f48334d3700 1 --2- 172.21.15.27:0/2139562193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f482c0f7980 0x7f482c069e20 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 rx=0x7f482800b410 tx=0x7f4828006ef0).stop 2021-11-03T18:48:42.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:42.131+0000 7f48334d3700 1 -- 172.21.15.27:0/2139562193 shutdown_connections 2021-11-03T18:48:42.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:42.131+0000 7f48334d3700 1 --2- 172.21.15.27:0/2139562193 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f48180640b0 0x7f4818066570 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:42.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:42.131+0000 7f48334d3700 1 --2- 172.21.15.27:0/2139562193 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f482c0f66e0 0x7f482c06b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:42.141 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:42.131+0000 7f48334d3700 1 --2- 172.21.15.27:0/2139562193 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f482c0f7980 0x7f482c069e20 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:48:42.142 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:42.131+0000 7f48334d3700 1 -- 172.21.15.27:0/2139562193 >> 172.21.15.27:0/2139562193 conn(0x7f482c0f1e40 msgr2=0x7f482c0fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:48:42.142 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:42.132+0000 7f48334d3700 1 -- 172.21.15.27:0/2139562193 shutdown_connections 2021-11-03T18:48:42.142 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:48:42.132+0000 7f48334d3700 1 -- 172.21.15.27:0/2139562193 wait complete. 2021-11-03T18:48:42.142 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:48:42.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:41 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:48:42.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:41 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:48:43.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:42 smithi115 ceph-mon[65453]: from='client.37000 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:48:43.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:42 smithi115 ceph-mon[65453]: pgmap v2823: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:43.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:42 smithi115 ceph-mon[65453]: from='client.37004 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:48:43.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:42 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/336502657' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:43.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:42 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2139562193' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:48:43.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:42 smithi027 ceph-mon[97244]: from='client.37000 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:48:43.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:42 smithi027 ceph-mon[97244]: pgmap v2823: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:43.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:42 smithi027 ceph-mon[97244]: from='client.37004 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:48:43.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:42 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/336502657' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:43.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:42 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2139562193' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:48:45.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:44 smithi115 ceph-mon[65453]: pgmap v2824: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:45.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:44 smithi027 ceph-mon[97244]: pgmap v2824: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:46.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5567.494187 secs 2021-11-03T18:48:46.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5567.494187 secs 2021-11-03T18:48:47.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:46 smithi115 ceph-mon[65453]: pgmap v2825: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:47.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:46 smithi027 ceph-mon[97244]: pgmap v2825: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:49.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:48 smithi115 ceph-mon[65453]: pgmap v2826: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:49.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:48 smithi027 ceph-mon[97244]: pgmap v2826: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:51.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:50 smithi115 ceph-mon[65453]: pgmap v2827: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:51.270 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5572.494312 secs 2021-11-03T18:48:51.270 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:48:51.271 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:48:51.271 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:48:51.271 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:51.271 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:51.272 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:51.272 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:51.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:50 smithi027 ceph-mon[97244]: pgmap v2827: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:51.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5572.494312 secs 2021-11-03T18:48:51.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:48:51.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:48:51.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:48:51.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:51.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:51.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:51.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:48:52.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:51 smithi115 ceph-mon[65453]: poll took 10037.752 ms: 1 events 2021-11-03T18:48:52.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:51 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:48:52.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:51 smithi027 ceph-mon[97244]: poll took 10037.752 ms: 1 events 2021-11-03T18:48:52.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:51 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:48:53.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:52 smithi115 ceph-mon[65453]: pgmap v2828: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:53.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:52 smithi027 ceph-mon[97244]: pgmap v2828: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:55.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:54 smithi115 ceph-mon[65453]: pgmap v2829: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:55.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:54 smithi027 ceph-mon[97244]: pgmap v2829: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:56.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5577.494452 secs 2021-11-03T18:48:56.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5577.494452 secs 2021-11-03T18:48:57.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:56 smithi115 ceph-mon[65453]: pgmap v2830: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:57.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:48:57.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:48:57.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:56 smithi027 ceph-mon[97244]: pgmap v2830: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:57.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:48:57.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:48:59.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:48:58 smithi115 ceph-mon[65453]: pgmap v2831: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:48:59.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:48:58 smithi027 ceph-mon[97244]: pgmap v2831: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:01.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:00 smithi115 ceph-mon[65453]: pgmap v2832: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:01.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5582.494569 secs 2021-11-03T18:49:01.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:49:01.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:49:01.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:49:01.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:01.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:01.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:01.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:01.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:00 smithi027 ceph-mon[97244]: pgmap v2832: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:01.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5582.494569 secs 2021-11-03T18:49:01.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:49:01.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:49:01.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:49:01.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:01.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:01.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:01.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:02.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:01 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:49:02.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:01 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:49:03.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:02 smithi115 ceph-mon[65453]: pgmap v2833: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:03.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:02 smithi027 ceph-mon[97244]: pgmap v2833: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:05.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:04 smithi115 ceph-mon[65453]: pgmap v2834: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:05.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:04 smithi027 ceph-mon[97244]: pgmap v2834: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:06.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5587.494631 secs 2021-11-03T18:49:06.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5587.494631 secs 2021-11-03T18:49:07.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:06 smithi115 ceph-mon[65453]: pgmap v2835: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:07.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:06 smithi027 ceph-mon[97244]: pgmap v2835: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:09.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:08 smithi115 ceph-mon[65453]: pgmap v2836: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:09.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:08 smithi027 ceph-mon[97244]: pgmap v2836: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:11.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:10 smithi115 ceph-mon[65453]: pgmap v2837: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:11.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5592.494756 secs 2021-11-03T18:49:11.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:49:11.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:49:11.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:49:11.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:11.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:11.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:11.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:11.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:10 smithi027 ceph-mon[97244]: pgmap v2837: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:11.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5592.494756 secs 2021-11-03T18:49:11.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:49:11.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:49:11.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:49:11.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:11.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:11.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:11.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:12.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:11 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:49:12.245 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.246+0000 7f6be148c700 1 -- 172.21.15.27:0/3392033691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bdc0f7dc0 msgr2=0x7f6bdc0f81e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:49:12.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.246+0000 7f6be148c700 1 --2- 172.21.15.27:0/3392033691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bdc0f7dc0 0x7f6bdc0f81e0 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 rx=0x7f6bd4004660 tx=0x7f6bd4009b30).stop 2021-11-03T18:49:12.246 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.246+0000 7f6be148c700 1 -- 172.21.15.27:0/3392033691 shutdown_connections 2021-11-03T18:49:12.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.246+0000 7f6be148c700 1 --2- 172.21.15.27:0/3392033691 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bdc0f86d0 0x7f6bdc0f48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:12.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.246+0000 7f6be148c700 1 --2- 172.21.15.27:0/3392033691 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bdc0f7dc0 0x7f6bdc0f81e0 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:12.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.246+0000 7f6be148c700 1 -- 172.21.15.27:0/3392033691 >> 172.21.15.27:0/3392033691 conn(0x7f6bdc0f0540 msgr2=0x7f6bdc0f2960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:49:12.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.246+0000 7f6be148c700 1 -- 172.21.15.27:0/3392033691 shutdown_connections 2021-11-03T18:49:12.248 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.246+0000 7f6be148c700 1 -- 172.21.15.27:0/3392033691 wait complete. 2021-11-03T18:49:12.251 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.247+0000 7f6be148c700 1 Processor -- start 2021-11-03T18:49:12.251 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.247+0000 7f6be148c700 1 -- start start 2021-11-03T18:49:12.251 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.247+0000 7f6be148c700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bdc0f7dc0 0x7f6bdc106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:49:12.252 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.247+0000 7f6be148c700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bdc0f86d0 0x7f6bdc107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:49:12.252 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.247+0000 7f6be148c700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc107a10 con 0x7f6bdc0f7dc0 2021-11-03T18:49:12.252 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.247+0000 7f6be148c700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bdc107b50 con 0x7f6bdc0f86d0 2021-11-03T18:49:12.253 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.247+0000 7f6bdaffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bdc0f7dc0 0x7f6bdc106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:49:12.253 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.247+0000 7f6bdaffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bdc0f7dc0 0x7f6bdc106f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37022/0 (socket says 172.21.15.27:37022) 2021-11-03T18:49:12.253 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.247+0000 7f6bdaffd700 1 -- 172.21.15.27:0/4165302668 learned_addr learned my addr 172.21.15.27:0/4165302668 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:49:12.253 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.247+0000 7f6bdaffd700 1 -- 172.21.15.27:0/4165302668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bdc0f86d0 msgr2=0x7f6bdc107460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:49:12.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.247+0000 7f6bda7fc700 1 --2- 172.21.15.27:0/4165302668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bdc0f86d0 0x7f6bdc107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:49:12.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.247+0000 7f6bdaffd700 1 --2- 172.21.15.27:0/4165302668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bdc0f86d0 0x7f6bdc107460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:12.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.247+0000 7f6bdaffd700 1 -- 172.21.15.27:0/4165302668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd4005040 con 0x7f6bdc0f7dc0 2021-11-03T18:49:12.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.247+0000 7f6bdaffd700 1 --2- 172.21.15.27:0/4165302668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bdc0f7dc0 0x7f6bdc106f70 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 rx=0x7f6bd4009990 tx=0x7f6bd4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:49:12.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.248+0000 7f6bd37fe700 1 -- 172.21.15.27:0/4165302668 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bd4018070 con 0x7f6bdc0f7dc0 2021-11-03T18:49:12.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.248+0000 7f6be148c700 1 -- 172.21.15.27:0/4165302668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bdc108490 con 0x7f6bdc0f7dc0 2021-11-03T18:49:12.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.248+0000 7f6bd37fe700 1 -- 172.21.15.27:0/4165302668 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f6bd4007ba0 con 0x7f6bdc0f7dc0 2021-11-03T18:49:12.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.248+0000 7f6bd37fe700 1 -- 172.21.15.27:0/4165302668 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bd401c3c0 con 0x7f6bdc0f7dc0 2021-11-03T18:49:12.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.248+0000 7f6be148c700 1 -- 172.21.15.27:0/4165302668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bdc1089f0 con 0x7f6bdc0f7dc0 2021-11-03T18:49:12.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.249+0000 7f6be148c700 1 -- 172.21.15.27:0/4165302668 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bdc045ad0 con 0x7f6bdc0f7dc0 2021-11-03T18:49:12.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.249+0000 7f6bd37fe700 1 -- 172.21.15.27:0/4165302668 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f6bd401d420 con 0x7f6bdc0f7dc0 2021-11-03T18:49:12.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.249+0000 7f6bd37fe700 1 --2- 172.21.15.27:0/4165302668 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6bc80640b0 0x7f6bc8066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:49:12.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.250+0000 7f6bd37fe700 1 -- 172.21.15.27:0/4165302668 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f6bd4082500 con 0x7f6bdc0f7dc0 2021-11-03T18:49:12.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.250+0000 7f6bda7fc700 1 --2- 172.21.15.27:0/4165302668 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6bc80640b0 0x7f6bc8066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:49:12.257 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.250+0000 7f6bda7fc700 1 --2- 172.21.15.27:0/4165302668 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6bc80640b0 0x7f6bc8066570 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 rx=0x7f6bc4004a70 tx=0x7f6bc400b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:49:12.258 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.254+0000 7f6bd37fe700 1 -- 172.21.15.27:0/4165302668 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f6bd40548c0 con 0x7f6bdc0f7dc0 2021-11-03T18:49:12.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:11 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:49:12.433 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.434+0000 7f6be148c700 1 -- 172.21.15.27:0/4165302668 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6bdc0fefc0 con 0x7f6bc80640b0 2021-11-03T18:49:12.434 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.435+0000 7f6bd37fe700 1 -- 172.21.15.27:0/4165302668 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f6bdc0fefc0 con 0x7f6bc80640b0 2021-11-03T18:49:12.436 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.437+0000 7f6be148c700 1 -- 172.21.15.27:0/4165302668 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6bc80640b0 msgr2=0x7f6bc8066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:49:12.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.437+0000 7f6be148c700 1 --2- 172.21.15.27:0/4165302668 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6bc80640b0 0x7f6bc8066570 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 rx=0x7f6bc4004a70 tx=0x7f6bc400b040).stop 2021-11-03T18:49:12.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.437+0000 7f6be148c700 1 -- 172.21.15.27:0/4165302668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bdc0f7dc0 msgr2=0x7f6bdc106f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:49:12.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.437+0000 7f6be148c700 1 --2- 172.21.15.27:0/4165302668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bdc0f7dc0 0x7f6bdc106f70 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 rx=0x7f6bd4009990 tx=0x7f6bd4009b30).stop 2021-11-03T18:49:12.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.437+0000 7f6be148c700 1 -- 172.21.15.27:0/4165302668 shutdown_connections 2021-11-03T18:49:12.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.437+0000 7f6be148c700 1 --2- 172.21.15.27:0/4165302668 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6bc80640b0 0x7f6bc8066570 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:12.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.437+0000 7f6be148c700 1 --2- 172.21.15.27:0/4165302668 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6bdc0f7dc0 0x7f6bdc106f70 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:12.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.437+0000 7f6be148c700 1 --2- 172.21.15.27:0/4165302668 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6bdc0f86d0 0x7f6bdc107460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:12.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.437+0000 7f6be148c700 1 -- 172.21.15.27:0/4165302668 >> 172.21.15.27:0/4165302668 conn(0x7f6bdc0f0540 msgr2=0x7f6bdc0fb650 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:49:12.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.437+0000 7f6be148c700 1 -- 172.21.15.27:0/4165302668 shutdown_connections 2021-11-03T18:49:12.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.437+0000 7f6be148c700 1 -- 172.21.15.27:0/4165302668 wait complete. 2021-11-03T18:49:12.446 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:49:12.542 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.537+0000 7fe876c64700 1 -- 172.21.15.27:0/267642488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8700f55c0 msgr2=0x7fe8700f59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:49:12.543 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.537+0000 7fe876c64700 1 --2- 172.21.15.27:0/267642488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8700f55c0 0x7fe8700f59e0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 rx=0x7fe864004660 tx=0x7fe864009b30).stop 2021-11-03T18:49:12.543 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.537+0000 7fe876c64700 1 -- 172.21.15.27:0/267642488 shutdown_connections 2021-11-03T18:49:12.544 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.537+0000 7fe876c64700 1 --2- 172.21.15.27:0/267642488 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8700f6860 0x7fe8700f6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2021-11-03T18:49:12.544 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.537+0000 7fe876c64700 1 --2- 172.21.15.27:0/267642488 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8700f55c0 0x7fe8700f59e0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:12.545 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.537+0000 7fe876c64700 1 -- 172.21.15.27:0/267642488 >> 172.21.15.27:0/267642488 conn(0x7fe8700f0d40 msgr2=0x7fe8700f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:49:12.545 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.537+0000 7fe876c64700 1 -- 172.21.15.27:0/267642488 shutdown_connections 2021-11-03T18:49:12.545 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.537+0000 7fe876c64700 1 -- 172.21.15.27:0/267642488 wait complete. 2021-11-03T18:49:12.546 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.538+0000 7fe876c64700 1 Processor -- start 2021-11-03T18:49:12.546 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.538+0000 7fe876c64700 1 -- start start 2021-11-03T18:49:12.547 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.538+0000 7fe876c64700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8700f55c0 0x7fe8700f9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:49:12.547 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.538+0000 7fe876c64700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8700f6860 0x7fe8700f7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:49:12.548 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.538+0000 7fe876c64700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8700f8190 con 0x7fe8700f55c0 2021-11-03T18:49:12.548 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.538+0000 7fe876c64700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8700f82d0 con 0x7fe8700f6860 2021-11-03T18:49:12.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.538+0000 7fe874a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8700f55c0 0x7fe8700f9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:49:12.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.538+0000 7fe874a00700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8700f55c0 0x7fe8700f9720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37032/0 (socket says 172.21.15.27:37032) 2021-11-03T18:49:12.549 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.538+0000 7fe874a00700 1 -- 172.21.15.27:0/1192822211 learned_addr learned my addr 172.21.15.27:0/1192822211 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:49:12.550 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.539+0000 7fe86ffff700 1 --2- 172.21.15.27:0/1192822211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8700f6860 0x7fe8700f7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:49:12.550 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.539+0000 7fe874a00700 1 -- 172.21.15.27:0/1192822211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8700f6860 msgr2=0x7fe8700f7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:49:12.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.539+0000 7fe874a00700 1 --2- 172.21.15.27:0/1192822211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8700f6860 0x7fe8700f7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:12.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.539+0000 7fe874a00700 1 -- 172.21.15.27:0/1192822211 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe864005040 con 0x7fe8700f55c0 2021-11-03T18:49:12.551 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.539+0000 7fe874a00700 1 --2- 172.21.15.27:0/1192822211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8700f55c0 0x7fe8700f9720 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 rx=0x7fe864000c00 tx=0x7fe864009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:49:12.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.539+0000 7fe86d7fa700 1 -- 172.21.15.27:0/1192822211 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe864018070 con 0x7fe8700f55c0 2021-11-03T18:49:12.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.539+0000 7fe86d7fa700 1 -- 172.21.15.27:0/1192822211 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fe864007cf0 con 0x7fe8700f55c0 2021-11-03T18:49:12.552 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.539+0000 7fe876c64700 1 -- 172.21.15.27:0/1192822211 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8700f8550 con 0x7fe8700f55c0 2021-11-03T18:49:12.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.539+0000 7fe86d7fa700 1 -- 172.21.15.27:0/1192822211 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe86401c3c0 con 0x7fe8700f55c0 2021-11-03T18:49:12.553 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.539+0000 7fe876c64700 1 -- 172.21.15.27:0/1192822211 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe87010efc0 con 0x7fe8700f55c0 2021-11-03T18:49:12.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.540+0000 7fe876c64700 1 -- 172.21.15.27:0/1192822211 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe870045ad0 con 0x7fe8700f55c0 2021-11-03T18:49:12.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.540+0000 7fe86d7fa700 1 -- 172.21.15.27:0/1192822211 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fe86401d420 con 0x7fe8700f55c0 2021-11-03T18:49:12.554 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.541+0000 7fe86d7fa700 1 --2- 172.21.15.27:0/1192822211 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe860064170 0x7fe860066630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:49:12.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.541+0000 7fe86d7fa700 1 -- 172.21.15.27:0/1192822211 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fe8640824a0 con 0x7fe8700f55c0 2021-11-03T18:49:12.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.541+0000 7fe86ffff700 1 --2- 172.21.15.27:0/1192822211 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe860064170 0x7fe860066630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:49:12.555 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.541+0000 7fe86ffff700 1 --2- 172.21.15.27:0/1192822211 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe860064170 0x7fe860066630 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 rx=0x7fe85c006fd0 tx=0x7fe85c008040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:49:12.556 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.545+0000 7fe86d7fa700 1 -- 172.21.15.27:0/1192822211 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fe864054940 con 0x7fe8700f55c0 2021-11-03T18:49:12.716 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.716+0000 7fe876c64700 1 -- 172.21.15.27:0/1192822211 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe8700faca0 con 0x7fe860064170 2021-11-03T18:49:12.731 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.731+0000 7fe86d7fa700 1 -- 172.21.15.27:0/1192822211 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3542 (secure 0 0 0) 0x7fe8700faca0 con 0x7fe860064170 2021-11-03T18:49:12.731 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:49:12.731 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (100m) 9m ago 102m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:49:12.732 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (92m) 9m ago 101m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:49:12.732 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (92m) 8m ago 101m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:49:12.732 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (100m) 9m ago 101m 42.9M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:49:12.733 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (94m) 9m ago 97m 26.9M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:49:12.733 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (97m) 9m ago 97m 3852M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:49:12.733 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (93m) 8m ago 97m 26.5M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:49:12.733 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (94m) 8m ago 97m 27.6M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:49:12.734 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (94m) 9m ago 103m 625M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:49:12.734 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (93m) 8m ago 101m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:49:12.734 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (93m) 9m ago 103m 387M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:49:12.735 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (93m) 8m ago 101m 175M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:49:12.735 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (101m) 9m ago 101m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:49:12.735 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (101m) 8m ago 100m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:49:12.735 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (92m) 9m ago 100m 433M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:49:12.736 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (91m) 9m ago 100m 471M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:49:12.736 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (91m) 9m ago 99m 356M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:49:12.736 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (90m) 8m ago 99m 504M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:49:12.737 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (90m) 8m ago 99m 396M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:49:12.737 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (89m) 8m ago 98m 408M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:49:12.737 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (100m) 9m ago 101m 66.8M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:49:12.738 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.733+0000 7fe876c64700 1 -- 172.21.15.27:0/1192822211 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe860064170 msgr2=0x7fe860066630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:49:12.738 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.733+0000 7fe876c64700 1 --2- 172.21.15.27:0/1192822211 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe860064170 0x7fe860066630 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 rx=0x7fe85c006fd0 tx=0x7fe85c008040).stop 2021-11-03T18:49:12.739 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.733+0000 7fe876c64700 1 -- 172.21.15.27:0/1192822211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8700f55c0 msgr2=0x7fe8700f9720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:49:12.739 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.733+0000 7fe876c64700 1 --2- 172.21.15.27:0/1192822211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8700f55c0 0x7fe8700f9720 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 rx=0x7fe864000c00 tx=0x7fe864009b30).stop 2021-11-03T18:49:12.739 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.733+0000 7fe876c64700 1 -- 172.21.15.27:0/1192822211 shutdown_connections 2021-11-03T18:49:12.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.733+0000 7fe876c64700 1 --2- 172.21.15.27:0/1192822211 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe860064170 0x7fe860066630 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:12.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.733+0000 7fe876c64700 1 --2- 172.21.15.27:0/1192822211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe8700f55c0 0x7fe8700f9720 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:12.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.733+0000 7fe876c64700 1 --2- 172.21.15.27:0/1192822211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe8700f6860 0x7fe8700f7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:12.740 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.733+0000 7fe876c64700 1 -- 172.21.15.27:0/1192822211 >> 172.21.15.27:0/1192822211 conn(0x7fe8700f0d40 msgr2=0x7fe87005fe40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:49:12.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.733+0000 7fe876c64700 1 -- 172.21.15.27:0/1192822211 shutdown_connections 2021-11-03T18:49:12.741 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.733+0000 7fe876c64700 1 -- 172.21.15.27:0/1192822211 wait complete. 2021-11-03T18:49:12.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.832+0000 7fbeb52f5700 1 -- 172.21.15.27:0/299248948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeb00602b0 msgr2=0x7fbeb00f9180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:49:12.832 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.832+0000 7fbeb52f5700 1 --2- 172.21.15.27:0/299248948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeb00602b0 0x7fbeb00f9180 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 rx=0x7fbea4004660 tx=0x7fbea4009b30).stop 2021-11-03T18:49:12.833 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.832+0000 7fbeb52f5700 1 -- 172.21.15.27:0/299248948 shutdown_connections 2021-11-03T18:49:12.833 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.832+0000 7fbeb52f5700 1 --2- 172.21.15.27:0/299248948 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbeb00f9700 0x7fbeb00fbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:12.833 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.832+0000 7fbeb52f5700 1 --2- 172.21.15.27:0/299248948 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeb00602b0 0x7fbeb00f9180 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:12.833 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.832+0000 7fbeb52f5700 1 -- 172.21.15.27:0/299248948 >> 172.21.15.27:0/299248948 conn(0x7fbeb00f0ba0 msgr2=0x7fbeb00f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:49:12.834 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.832+0000 7fbeb52f5700 1 -- 172.21.15.27:0/299248948 shutdown_connections 2021-11-03T18:49:12.834 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.832+0000 7fbeb52f5700 1 -- 172.21.15.27:0/299248948 wait complete. 2021-11-03T18:49:12.834 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.833+0000 7fbeb52f5700 1 Processor -- start 2021-11-03T18:49:12.834 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.833+0000 7fbeb52f5700 1 -- start start 2021-11-03T18:49:12.838 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.833+0000 7fbeb52f5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeb00602b0 0x7fbeb0106db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:49:12.838 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.833+0000 7fbeb52f5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbeb00f9700 0x7fbeb01072a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:49:12.838 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.833+0000 7fbeb52f5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb01078a0 con 0x7fbeb00602b0 2021-11-03T18:49:12.839 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.833+0000 7fbeb52f5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb01080e0 con 0x7fbeb00f9700 2021-11-03T18:49:12.839 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.833+0000 7fbeaffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeb00602b0 0x7fbeb0106db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:49:12.839 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.833+0000 7fbeaffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeb00602b0 0x7fbeb0106db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37042/0 (socket says 172.21.15.27:37042) 2021-11-03T18:49:12.839 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.833+0000 7fbeaffff700 1 -- 172.21.15.27:0/2034086968 learned_addr learned my addr 172.21.15.27:0/2034086968 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:49:12.840 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.834+0000 7fbeaffff700 1 -- 172.21.15.27:0/2034086968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbeb00f9700 msgr2=0x7fbeb01072a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:49:12.840 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.834+0000 7fbeaf7fe700 1 --2- 172.21.15.27:0/2034086968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbeb00f9700 0x7fbeb01072a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:49:12.840 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.834+0000 7fbeaffff700 1 --2- 172.21.15.27:0/2034086968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbeb00f9700 0x7fbeb01072a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:12.840 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.834+0000 7fbeaffff700 1 -- 172.21.15.27:0/2034086968 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea4005040 con 0x7fbeb00602b0 2021-11-03T18:49:12.841 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.834+0000 7fbeaf7fe700 1 --2- 172.21.15.27:0/2034086968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbeb00f9700 0x7fbeb01072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T18:49:12.841 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.834+0000 7fbeaffff700 1 --2- 172.21.15.27:0/2034086968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeb00602b0 0x7fbeb0106db0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 rx=0x7fbea4009da0 tx=0x7fbea4009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:49:12.841 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.834+0000 7fbeacff9700 1 -- 172.21.15.27:0/2034086968 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbea4018070 con 0x7fbeb00602b0 2021-11-03T18:49:12.841 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.834+0000 7fbeb52f5700 1 -- 172.21.15.27:0/2034086968 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeb0108360 con 0x7fbeb00602b0 2021-11-03T18:49:12.841 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.834+0000 7fbeacff9700 1 -- 172.21.15.27:0/2034086968 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fbea4007d70 con 0x7fbeb00602b0 2021-11-03T18:49:12.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.834+0000 7fbeacff9700 1 -- 172.21.15.27:0/2034086968 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbea401dea0 con 0x7fbeb00602b0 2021-11-03T18:49:12.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.834+0000 7fbeb52f5700 1 -- 172.21.15.27:0/2034086968 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeb0108840 con 0x7fbeb00602b0 2021-11-03T18:49:12.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.835+0000 7fbeacff9700 1 -- 172.21.15.27:0/2034086968 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fbea401d420 con 0x7fbeb00602b0 2021-11-03T18:49:12.842 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.835+0000 7fbeb52f5700 1 -- 172.21.15.27:0/2034086968 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeb0059100 con 0x7fbeb00602b0 2021-11-03T18:49:12.843 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.836+0000 7fbeacff9700 1 --2- 172.21.15.27:0/2034086968 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbe98064100 0x7fbe980665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:49:12.843 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.836+0000 7fbeacff9700 1 -- 172.21.15.27:0/2034086968 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fbea4082580 con 0x7fbeb00602b0 2021-11-03T18:49:12.843 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.836+0000 7fbeaf7fe700 1 --2- 172.21.15.27:0/2034086968 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbe98064100 0x7fbe980665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:49:12.843 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.836+0000 7fbeaf7fe700 1 --2- 172.21.15.27:0/2034086968 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbe98064100 0x7fbe980665c0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 rx=0x7fbeb0106b40 tx=0x7fbea0009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:49:12.844 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:12.840+0000 7fbeacff9700 1 -- 172.21.15.27:0/2034086968 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fbea40549a0 con 0x7fbeb00602b0 2021-11-03T18:49:13.073 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.074+0000 7fbeb52f5700 1 -- 172.21.15.27:0/2034086968 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbeb00fee20 con 0x7fbeb00602b0 2021-11-03T18:49:13.074 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.074+0000 7fbeacff9700 1 -- 172.21.15.27:0/2034086968 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fbea4054590 con 0x7fbeb00602b0 2021-11-03T18:49:13.075 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:49:13.075 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:49:13.075 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:49:13.075 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:49:13.075 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:49:13.076 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:49:13.076 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:49:13.076 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:49:13.076 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:49:13.077 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:49:13.077 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:49:13.077 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:49:13.077 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:49:13.078 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:49:13.078 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:49:13.078 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:49:13.078 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:49:13.079 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:49:13.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.076+0000 7fbeb52f5700 1 -- 172.21.15.27:0/2034086968 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbe98064100 msgr2=0x7fbe980665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:49:13.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.076+0000 7fbeb52f5700 1 --2- 172.21.15.27:0/2034086968 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbe98064100 0x7fbe980665c0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 rx=0x7fbeb0106b40 tx=0x7fbea0009040).stop 2021-11-03T18:49:13.080 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.076+0000 7fbeb52f5700 1 -- 172.21.15.27:0/2034086968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeb00602b0 msgr2=0x7fbeb0106db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:49:13.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.076+0000 7fbeb52f5700 1 --2- 172.21.15.27:0/2034086968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeb00602b0 0x7fbeb0106db0 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 rx=0x7fbea4009da0 tx=0x7fbea4009b30).stop 2021-11-03T18:49:13.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.076+0000 7fbeb52f5700 1 -- 172.21.15.27:0/2034086968 shutdown_connections 2021-11-03T18:49:13.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.076+0000 7fbeb52f5700 1 --2- 172.21.15.27:0/2034086968 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbe98064100 0x7fbe980665c0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:13.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.076+0000 7fbeb52f5700 1 --2- 172.21.15.27:0/2034086968 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbeb00602b0 0x7fbeb0106db0 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:13.081 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.076+0000 7fbeb52f5700 1 --2- 172.21.15.27:0/2034086968 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbeb00f9700 0x7fbeb01072a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:13.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.076+0000 7fbeb52f5700 1 -- 172.21.15.27:0/2034086968 >> 172.21.15.27:0/2034086968 conn(0x7fbeb00f0ba0 msgr2=0x7fbeb00f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:49:13.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.076+0000 7fbeb52f5700 1 -- 172.21.15.27:0/2034086968 shutdown_connections 2021-11-03T18:49:13.082 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.077+0000 7fbeb52f5700 1 -- 172.21.15.27:0/2034086968 wait complete. 2021-11-03T18:49:13.178 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.177+0000 7f8a7d913700 1 -- 172.21.15.27:0/3619207557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a780602f0 msgr2=0x7f8a780f9400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:49:13.178 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.177+0000 7f8a7d913700 1 --2- 172.21.15.27:0/3619207557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a780602f0 0x7f8a780f9400 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 rx=0x7f8a68004660 tx=0x7f8a68009b30).stop 2021-11-03T18:49:13.179 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.177+0000 7f8a7d913700 1 -- 172.21.15.27:0/3619207557 shutdown_connections 2021-11-03T18:49:13.179 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.177+0000 7f8a7d913700 1 --2- 172.21.15.27:0/3619207557 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a780f98f0 0x7f8a780fbce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:13.179 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.177+0000 7f8a7d913700 1 --2- 172.21.15.27:0/3619207557 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a780602f0 0x7f8a780f9400 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:13.179 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.177+0000 7f8a7d913700 1 -- 172.21.15.27:0/3619207557 >> 172.21.15.27:0/3619207557 conn(0x7f8a780f0de0 msgr2=0x7f8a780f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:49:13.180 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.177+0000 7f8a7d913700 1 -- 172.21.15.27:0/3619207557 shutdown_connections 2021-11-03T18:49:13.180 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.177+0000 7f8a7d913700 1 -- 172.21.15.27:0/3619207557 wait complete. 2021-11-03T18:49:13.180 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.178+0000 7f8a7d913700 1 Processor -- start 2021-11-03T18:49:13.180 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.178+0000 7f8a7d913700 1 -- start start 2021-11-03T18:49:13.180 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.178+0000 7f8a7d913700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a780602f0 0x7f8a780f6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:49:13.181 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.178+0000 7f8a7d913700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a780f98f0 0x7f8a780f5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:49:13.181 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.178+0000 7f8a7d913700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a780f5860 con 0x7f8a780602f0 2021-11-03T18:49:13.181 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.178+0000 7f8a7d913700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a780f59a0 con 0x7f8a780f98f0 2021-11-03T18:49:13.181 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.178+0000 7f8a76ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a780602f0 0x7f8a780f6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:49:13.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.178+0000 7f8a76ffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a780602f0 0x7f8a780f6e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37052/0 (socket says 172.21.15.27:37052) 2021-11-03T18:49:13.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.178+0000 7f8a76ffd700 1 -- 172.21.15.27:0/280075293 learned_addr learned my addr 172.21.15.27:0/280075293 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:49:13.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.178+0000 7f8a76ffd700 1 -- 172.21.15.27:0/280075293 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a780f98f0 msgr2=0x7f8a780f5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:49:13.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.178+0000 7f8a76ffd700 1 --2- 172.21.15.27:0/280075293 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a780f98f0 0x7f8a780f5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:49:13.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.178+0000 7f8a76ffd700 1 -- 172.21.15.27:0/280075293 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a68005040 con 0x7f8a780602f0 2021-11-03T18:49:13.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.178+0000 7f8a76ffd700 1 --2- 172.21.15.27:0/280075293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a780602f0 0x7f8a780f6e80 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 rx=0x7f8a68004f60 tx=0x7f8a68009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:49:13.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.179+0000 7f8a6f7fe700 1 -- 172.21.15.27:0/280075293 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a68018070 con 0x7f8a780602f0 2021-11-03T18:49:13.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.179+0000 7f8a6f7fe700 1 -- 172.21.15.27:0/280075293 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f8a68007ba0 con 0x7f8a780602f0 2021-11-03T18:49:13.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.179+0000 7f8a7d913700 1 -- 172.21.15.27:0/280075293 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a780f5c20 con 0x7f8a780602f0 2021-11-03T18:49:13.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.179+0000 7f8a6f7fe700 1 -- 172.21.15.27:0/280075293 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8a6801c620 con 0x7f8a780602f0 2021-11-03T18:49:13.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.179+0000 7f8a7d913700 1 -- 172.21.15.27:0/280075293 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a780f6180 con 0x7f8a780602f0 2021-11-03T18:49:13.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.179+0000 7f8a7d913700 1 -- 172.21.15.27:0/280075293 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a78045ad0 con 0x7f8a780602f0 2021-11-03T18:49:13.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.180+0000 7f8a6f7fe700 1 -- 172.21.15.27:0/280075293 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f8a6801d420 con 0x7f8a780602f0 2021-11-03T18:49:13.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.180+0000 7f8a6f7fe700 1 --2- 172.21.15.27:0/280075293 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8a640640b0 0x7f8a64066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:49:13.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.181+0000 7f8a6f7fe700 1 -- 172.21.15.27:0/280075293 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f8a68082610 con 0x7f8a780602f0 2021-11-03T18:49:13.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.181+0000 7f8a6ffff700 1 --2- 172.21.15.27:0/280075293 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8a640640b0 0x7f8a64066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:49:13.186 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.181+0000 7f8a6ffff700 1 --2- 172.21.15.27:0/280075293 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8a640640b0 0x7f8a64066570 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 rx=0x7f8a60004880 tx=0x7f8a60007ae0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:49:13.187 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.185+0000 7f8a6f7fe700 1 -- 172.21.15.27:0/280075293 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f8a68054a30 con 0x7f8a780602f0 2021-11-03T18:49:13.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:12 smithi115 ceph-mon[65453]: pgmap v2838: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:13.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:12 smithi027 ceph-mon[97244]: pgmap v2838: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:13.388 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.388+0000 7f8a7d913700 1 -- 172.21.15.27:0/280075293 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8a780586e0 con 0x7f8a780602f0 2021-11-03T18:49:13.388 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.389+0000 7f8a6f7fe700 1 -- 172.21.15.27:0/280075293 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f8a68054620 con 0x7f8a780602f0 2021-11-03T18:49:13.389 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:49:13.389 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:49:13.389 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:49:13.390 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:49:13.390 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:49:13.390 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:49:13.390 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:49:13.391 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:49:13.391 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:49:13.391 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:49:13.391 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:49:13.391 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:49:13.392 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:49:13.392 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:49:13.392 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:49:13.392 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:49:13.393 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:49:13.393 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:49:13.393 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:49:13.393 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:49:13.394 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:49:13.394 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:49:13.394 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:49:13.394 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:49:13.394 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:49:13.395 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:49:13.395 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:49:13.395 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:49:13.395 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:49:13.396 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:49:13.396 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:49:13.396 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:49:13.396 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:49:13.397 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:49:13.397 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:49:13.397 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:49:13.397 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:49:13.397 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:49:13.398 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:49:13.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.390+0000 7f8a7d913700 1 -- 172.21.15.27:0/280075293 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8a640640b0 msgr2=0x7f8a64066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:49:13.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.390+0000 7f8a7d913700 1 --2- 172.21.15.27:0/280075293 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8a640640b0 0x7f8a64066570 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 rx=0x7f8a60004880 tx=0x7f8a60007ae0).stop 2021-11-03T18:49:13.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.390+0000 7f8a7d913700 1 -- 172.21.15.27:0/280075293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a780602f0 msgr2=0x7f8a780f6e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:49:13.399 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.390+0000 7f8a7d913700 1 --2- 172.21.15.27:0/280075293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a780602f0 0x7f8a780f6e80 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 rx=0x7f8a68004f60 tx=0x7f8a68009b30).stop 2021-11-03T18:49:13.400 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.391+0000 7f8a7d913700 1 -- 172.21.15.27:0/280075293 shutdown_connections 2021-11-03T18:49:13.400 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.391+0000 7f8a7d913700 1 --2- 172.21.15.27:0/280075293 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8a640640b0 0x7f8a64066570 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:13.400 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.391+0000 7f8a7d913700 1 --2- 172.21.15.27:0/280075293 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8a780602f0 0x7f8a780f6e80 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:13.400 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.391+0000 7f8a7d913700 1 --2- 172.21.15.27:0/280075293 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8a780f98f0 0x7f8a780f5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:49:13.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.391+0000 7f8a7d913700 1 -- 172.21.15.27:0/280075293 >> 172.21.15.27:0/280075293 conn(0x7f8a780f0de0 msgr2=0x7f8a780f31d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:49:13.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.391+0000 7f8a7d913700 1 -- 172.21.15.27:0/280075293 shutdown_connections 2021-11-03T18:49:13.401 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:13.391+0000 7f8a7d913700 1 -- 172.21.15.27:0/280075293 wait complete. 2021-11-03T18:49:13.401 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:49:14.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:13 smithi115 ceph-mon[65453]: from='client.37016 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:49:14.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:13 smithi115 ceph-mon[65453]: from='client.37020 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:49:14.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:13 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2034086968' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:14.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:13 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/280075293' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:49:14.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:13 smithi027 ceph-mon[97244]: from='client.37016 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:49:14.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:13 smithi027 ceph-mon[97244]: from='client.37020 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:49:14.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:13 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2034086968' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:14.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:13 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/280075293' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:49:15.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:14 smithi115 ceph-mon[65453]: pgmap v2839: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:15.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:14 smithi027 ceph-mon[97244]: pgmap v2839: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:16.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5597.494849 secs 2021-11-03T18:49:16.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5597.494849 secs 2021-11-03T18:49:17.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:16 smithi115 ceph-mon[65453]: pgmap v2840: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:17.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:16 smithi027 ceph-mon[97244]: pgmap v2840: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:19.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:18 smithi115 ceph-mon[65453]: pgmap v2841: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:19.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:18 smithi027 ceph-mon[97244]: pgmap v2841: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:21.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:20 smithi115 ceph-mon[65453]: pgmap v2842: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:21.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5602.494989 secs 2021-11-03T18:49:21.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:49:21.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:49:21.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:49:21.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:21.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:21.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:21.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:21.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:20 smithi027 ceph-mon[97244]: pgmap v2842: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:21.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5602.494989 secs 2021-11-03T18:49:21.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:49:21.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:49:21.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:49:21.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:21.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:21.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:21.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:22.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:21 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:49:22.279 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:21 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:49:23.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:22 smithi115 ceph-mon[65453]: pgmap v2843: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:23.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:22 smithi027 ceph-mon[97244]: pgmap v2843: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:25.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:24 smithi115 ceph-mon[65453]: pgmap v2844: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:25.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:24 smithi027 ceph-mon[97244]: pgmap v2844: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:26.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5607.495064 secs 2021-11-03T18:49:26.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5607.495064 secs 2021-11-03T18:49:27.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:26 smithi115 ceph-mon[65453]: pgmap v2845: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:27.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:26 smithi027 ceph-mon[97244]: pgmap v2845: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:29.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:28 smithi115 ceph-mon[65453]: pgmap v2846: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:29.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:28 smithi027 ceph-mon[97244]: pgmap v2846: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:31.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:30 smithi115 ceph-mon[65453]: pgmap v2847: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:31.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5612.495202 secs 2021-11-03T18:49:31.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:49:31.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:49:31.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:49:31.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:31.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:31.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:31.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:31.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:30 smithi027 ceph-mon[97244]: pgmap v2847: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:31.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5612.495202 secs 2021-11-03T18:49:31.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:49:31.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:49:31.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:49:31.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:31.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:31.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:31.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:32.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:31 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:49:32.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:31 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:49:33.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:32 smithi115 ceph-mon[65453]: pgmap v2848: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:33.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:32 smithi027 ceph-mon[97244]: pgmap v2848: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:35.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:34 smithi115 ceph-mon[65453]: pgmap v2849: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:35.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:34 smithi027 ceph-mon[97244]: pgmap v2849: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:36.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5617.495297 secs 2021-11-03T18:49:36.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5617.495297 secs 2021-11-03T18:49:37.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:36 smithi115 ceph-mon[65453]: pgmap v2850: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:37.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:36 smithi027 ceph-mon[97244]: pgmap v2850: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:39.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:38 smithi115 ceph-mon[65453]: pgmap v2851: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:39.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:38 smithi027 ceph-mon[97244]: pgmap v2851: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:41.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:40 smithi115 ceph-mon[65453]: pgmap v2852: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:41.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5622.495438 secs 2021-11-03T18:49:41.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:49:41.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:49:41.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:49:41.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:41.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:41.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:41.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:40 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:41.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:40 smithi027 ceph-mon[97244]: pgmap v2852: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:41.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5622.495438 secs 2021-11-03T18:49:41.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:49:41.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:49:41.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:49:41.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:41.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:41.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:41.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:40 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:42.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:41 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:49:42.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:41 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:49:43.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:42 smithi115 ceph-mon[65453]: pgmap v2853: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:43.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:42 smithi027 ceph-mon[97244]: pgmap v2853: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:43.506 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.505+0000 7ffa33752700 1 -- 172.21.15.27:0/2480106754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa2c0f55e0 msgr2=0x7ffa2c0f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:49:43.507 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.505+0000 7ffa33752700 1 --2- 172.21.15.27:0/2480106754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa2c0f55e0 0x7ffa2c0f5a00 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 rx=0x7ffa240099e0 tx=0x7ffa24004d10).stop 2021-11-03T18:49:43.507 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.505+0000 7ffa33752700 1 -- 172.21.15.27:0/2480106754 shutdown_connections 2021-11-03T18:49:43.507 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.505+0000 7ffa33752700 1 --2- 172.21.15.27:0/2480106754 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa2c0f5ef0 0x7ffa2c0f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:43.507 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.505+0000 7ffa33752700 1 --2- 172.21.15.27:0/2480106754 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa2c0f55e0 0x7ffa2c0f5a00 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:43.508 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.505+0000 7ffa33752700 1 -- 172.21.15.27:0/2480106754 >> 172.21.15.27:0/2480106754 conn(0x7ffa2c0f0de0 msgr2=0x7ffa2c0f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:49:43.508 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.505+0000 7ffa33752700 1 -- 172.21.15.27:0/2480106754 shutdown_connections 2021-11-03T18:49:43.508 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.505+0000 7ffa33752700 1 -- 172.21.15.27:0/2480106754 wait complete. 2021-11-03T18:49:43.508 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.506+0000 7ffa33752700 1 Processor -- start 2021-11-03T18:49:43.509 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.506+0000 7ffa33752700 1 -- start start 2021-11-03T18:49:43.509 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.506+0000 7ffa33752700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa2c0f55e0 0x7ffa2c107060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:49:43.509 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.506+0000 7ffa33752700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa2c0f5ef0 0x7ffa2c107550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:49:43.509 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.506+0000 7ffa33752700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c107b00 con 0x7ffa2c0f55e0 2021-11-03T18:49:43.510 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.506+0000 7ffa33752700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa2c107c40 con 0x7ffa2c0f5ef0 2021-11-03T18:49:43.510 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.506+0000 7ffa314ee700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa2c0f55e0 0x7ffa2c107060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:49:43.510 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.506+0000 7ffa314ee700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa2c0f55e0 0x7ffa2c107060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37070/0 (socket says 172.21.15.27:37070) 2021-11-03T18:49:43.510 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.506+0000 7ffa314ee700 1 -- 172.21.15.27:0/1288851363 learned_addr learned my addr 172.21.15.27:0/1288851363 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:49:43.511 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.506+0000 7ffa30ced700 1 --2- 172.21.15.27:0/1288851363 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa2c0f5ef0 0x7ffa2c107550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:49:43.511 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.506+0000 7ffa314ee700 1 -- 172.21.15.27:0/1288851363 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa2c0f5ef0 msgr2=0x7ffa2c107550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:49:43.511 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.506+0000 7ffa314ee700 1 --2- 172.21.15.27:0/1288851363 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa2c0f5ef0 0x7ffa2c107550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:43.511 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.506+0000 7ffa314ee700 1 -- 172.21.15.27:0/1288851363 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa24009640 con 0x7ffa2c0f55e0 2021-11-03T18:49:43.512 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.506+0000 7ffa314ee700 1 --2- 172.21.15.27:0/1288851363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa2c0f55e0 0x7ffa2c107060 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 rx=0x7ffa24003380 tx=0x7ffa24004d10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:49:43.512 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.507+0000 7ffa29ffb700 1 -- 172.21.15.27:0/1288851363 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa2400e070 con 0x7ffa2c0f55e0 2021-11-03T18:49:43.512 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.507+0000 7ffa29ffb700 1 -- 172.21.15.27:0/1288851363 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7ffa24007d10 con 0x7ffa2c0f55e0 2021-11-03T18:49:43.512 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.507+0000 7ffa33752700 1 -- 172.21.15.27:0/1288851363 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa2c108580 con 0x7ffa2c0f55e0 2021-11-03T18:49:43.512 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.507+0000 7ffa29ffb700 1 -- 172.21.15.27:0/1288851363 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa24018070 con 0x7ffa2c0f55e0 2021-11-03T18:49:43.513 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.507+0000 7ffa33752700 1 -- 172.21.15.27:0/1288851363 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa2c108ae0 con 0x7ffa2c0f55e0 2021-11-03T18:49:43.514 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.508+0000 7ffa33752700 1 -- 172.21.15.27:0/1288851363 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa2c045ad0 con 0x7ffa2c0f55e0 2021-11-03T18:49:43.514 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.509+0000 7ffa29ffb700 1 -- 172.21.15.27:0/1288851363 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7ffa24013420 con 0x7ffa2c0f55e0 2021-11-03T18:49:43.514 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.509+0000 7ffa29ffb700 1 --2- 172.21.15.27:0/1288851363 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffa140640b0 0x7ffa14066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:49:43.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.509+0000 7ffa29ffb700 1 -- 172.21.15.27:0/1288851363 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7ffa24082050 con 0x7ffa2c0f55e0 2021-11-03T18:49:43.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.509+0000 7ffa30ced700 1 --2- 172.21.15.27:0/1288851363 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffa140640b0 0x7ffa14066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:49:43.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.510+0000 7ffa30ced700 1 --2- 172.21.15.27:0/1288851363 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffa140640b0 0x7ffa14066570 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 rx=0x7ffa20009fd0 tx=0x7ffa2000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:49:43.515 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.513+0000 7ffa29ffb700 1 -- 172.21.15.27:0/1288851363 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7ffa240543b0 con 0x7ffa2c0f55e0 2021-11-03T18:49:43.691 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.692+0000 7ffa33752700 1 -- 172.21.15.27:0/1288851363 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa2c0fad60 con 0x7ffa140640b0 2021-11-03T18:49:43.692 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.693+0000 7ffa29ffb700 1 -- 172.21.15.27:0/1288851363 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7ffa2c0fad60 con 0x7ffa140640b0 2021-11-03T18:49:43.694 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.695+0000 7ffa33752700 1 -- 172.21.15.27:0/1288851363 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffa140640b0 msgr2=0x7ffa14066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:49:43.694 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.695+0000 7ffa33752700 1 --2- 172.21.15.27:0/1288851363 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffa140640b0 0x7ffa14066570 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 rx=0x7ffa20009fd0 tx=0x7ffa2000b040).stop 2021-11-03T18:49:43.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.695+0000 7ffa33752700 1 -- 172.21.15.27:0/1288851363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa2c0f55e0 msgr2=0x7ffa2c107060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:49:43.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.695+0000 7ffa33752700 1 --2- 172.21.15.27:0/1288851363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa2c0f55e0 0x7ffa2c107060 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 rx=0x7ffa24003380 tx=0x7ffa24004d10).stop 2021-11-03T18:49:43.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.695+0000 7ffa33752700 1 -- 172.21.15.27:0/1288851363 shutdown_connections 2021-11-03T18:49:43.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.695+0000 7ffa33752700 1 --2- 172.21.15.27:0/1288851363 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7ffa140640b0 0x7ffa14066570 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:43.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.695+0000 7ffa33752700 1 --2- 172.21.15.27:0/1288851363 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7ffa2c0f55e0 0x7ffa2c107060 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:43.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.695+0000 7ffa33752700 1 --2- 172.21.15.27:0/1288851363 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7ffa2c0f5ef0 0x7ffa2c107550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:43.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.695+0000 7ffa33752700 1 -- 172.21.15.27:0/1288851363 >> 172.21.15.27:0/1288851363 conn(0x7ffa2c0f0de0 msgr2=0x7ffa2c05ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:49:43.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.695+0000 7ffa33752700 1 -- 172.21.15.27:0/1288851363 shutdown_connections 2021-11-03T18:49:43.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.695+0000 7ffa33752700 1 -- 172.21.15.27:0/1288851363 wait complete. 2021-11-03T18:49:43.704 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:49:43.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.795+0000 7f54c1cef700 1 -- 172.21.15.27:0/3371323042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54bc0f7960 msgr2=0x7f54bc0f7dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:49:43.796 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.795+0000 7f54c1cef700 1 --2- 172.21.15.27:0/3371323042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54bc0f7960 0x7f54bc0f7dc0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 rx=0x7f54b0004660 tx=0x7f54b0009b30).stop 2021-11-03T18:49:43.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.795+0000 7f54c1cef700 1 -- 172.21.15.27:0/3371323042 shutdown_connections 2021-11-03T18:49:43.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.795+0000 7f54c1cef700 1 --2- 172.21.15.27:0/3371323042 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54bc0f7960 0x7f54bc0f7dc0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:43.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.795+0000 7f54c1cef700 1 --2- 172.21.15.27:0/3371323042 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54bc0f66c0 0x7f54bc0f6ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:43.797 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.795+0000 7f54c1cef700 1 -- 172.21.15.27:0/3371323042 >> 172.21.15.27:0/3371323042 conn(0x7f54bc0f1e20 msgr2=0x7f54bc0f4240 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:49:43.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.795+0000 7f54c1cef700 1 -- 172.21.15.27:0/3371323042 shutdown_connections 2021-11-03T18:49:43.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.795+0000 7f54c1cef700 1 -- 172.21.15.27:0/3371323042 wait complete. 2021-11-03T18:49:43.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.795+0000 7f54c1cef700 1 Processor -- start 2021-11-03T18:49:43.798 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.796+0000 7f54c1cef700 1 -- start start 2021-11-03T18:49:43.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.796+0000 7f54c1cef700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54bc0f66c0 0x7f54bc106f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:49:43.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.796+0000 7f54c1cef700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54bc0f7960 0x7f54bc107440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:49:43.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.796+0000 7f54c1cef700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54bc1079f0 con 0x7f54bc0f66c0 2021-11-03T18:49:43.799 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.796+0000 7f54c1cef700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54bc107b30 con 0x7f54bc0f7960 2021-11-03T18:49:43.800 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.796+0000 7f54bb7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54bc0f66c0 0x7f54bc106f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:49:43.800 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.796+0000 7f54bb7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54bc0f66c0 0x7f54bc106f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37080/0 (socket says 172.21.15.27:37080) 2021-11-03T18:49:43.800 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.796+0000 7f54bb7fe700 1 -- 172.21.15.27:0/230067595 learned_addr learned my addr 172.21.15.27:0/230067595 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:49:43.800 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.796+0000 7f54baffd700 1 --2- 172.21.15.27:0/230067595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54bc0f7960 0x7f54bc107440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:49:43.800 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.796+0000 7f54bb7fe700 1 -- 172.21.15.27:0/230067595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54bc0f7960 msgr2=0x7f54bc107440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:49:43.801 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.796+0000 7f54bb7fe700 1 --2- 172.21.15.27:0/230067595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54bc0f7960 0x7f54bc107440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:43.801 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.796+0000 7f54bb7fe700 1 -- 172.21.15.27:0/230067595 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54b0005040 con 0x7f54bc0f66c0 2021-11-03T18:49:43.801 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.796+0000 7f54baffd700 1 --2- 172.21.15.27:0/230067595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54bc0f7960 0x7f54bc107440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T18:49:43.801 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.796+0000 7f54bb7fe700 1 --2- 172.21.15.27:0/230067595 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54bc0f66c0 0x7f54bc106f50 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 rx=0x7f54ac00c530 tx=0x7f54ac007e10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:49:43.802 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.796+0000 7f54a3fff700 1 -- 172.21.15.27:0/230067595 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54ac00f650 con 0x7f54bc0f66c0 2021-11-03T18:49:43.802 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.797+0000 7f54a3fff700 1 -- 172.21.15.27:0/230067595 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f54ac00fc60 con 0x7f54bc0f66c0 2021-11-03T18:49:43.802 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.797+0000 7f54c1cef700 1 -- 172.21.15.27:0/230067595 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54bc1084d0 con 0x7f54bc0f66c0 2021-11-03T18:49:43.802 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.797+0000 7f54a3fff700 1 -- 172.21.15.27:0/230067595 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f54ac012d20 con 0x7f54bc0f66c0 2021-11-03T18:49:43.802 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.797+0000 7f54c1cef700 1 -- 172.21.15.27:0/230067595 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54bc108a90 con 0x7f54bc0f66c0 2021-11-03T18:49:43.804 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.798+0000 7f54c1cef700 1 -- 172.21.15.27:0/230067595 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54bc045ad0 con 0x7f54bc0f66c0 2021-11-03T18:49:43.804 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.798+0000 7f54a3fff700 1 -- 172.21.15.27:0/230067595 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f54ac019030 con 0x7f54bc0f66c0 2021-11-03T18:49:43.804 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.798+0000 7f54a3fff700 1 --2- 172.21.15.27:0/230067595 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f54a40640b0 0x7f54a4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:49:43.804 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.799+0000 7f54a3fff700 1 -- 172.21.15.27:0/230067595 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f54ac080620 con 0x7f54bc0f66c0 2021-11-03T18:49:43.805 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.799+0000 7f54baffd700 1 --2- 172.21.15.27:0/230067595 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f54a40640b0 0x7f54a4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:49:43.805 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.799+0000 7f54baffd700 1 --2- 172.21.15.27:0/230067595 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f54a40640b0 0x7f54a4066570 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7f54b0009da0 tx=0x7f54b0009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:49:43.805 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.802+0000 7f54a3fff700 1 -- 172.21.15.27:0/230067595 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f54ac0528e0 con 0x7f54bc0f66c0 2021-11-03T18:49:43.977 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.977+0000 7f54c1cef700 1 -- 172.21.15.27:0/230067595 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f54bc108d10 con 0x7f54a40640b0 2021-11-03T18:49:43.991 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.992+0000 7f54a3fff700 1 -- 172.21.15.27:0/230067595 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3542 (secure 0 0 0) 0x7f54bc108d10 con 0x7f54a40640b0 2021-11-03T18:49:43.992 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:49:43.992 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (101m) 9m ago 102m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:49:43.992 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (92m) 9m ago 102m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:49:43.992 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (92m) 9m ago 101m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:49:43.993 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (101m) 9m ago 102m 42.9M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:49:43.993 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (94m) 9m ago 97m 26.9M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:49:43.993 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (97m) 9m ago 97m 3852M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:49:43.993 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (93m) 9m ago 97m 26.5M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:49:43.994 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (95m) 9m ago 97m 27.6M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:49:43.994 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (95m) 9m ago 103m 625M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:49:43.994 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (94m) 9m ago 101m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:49:43.994 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (93m) 9m ago 103m 387M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:49:43.994 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (93m) 9m ago 101m 175M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:49:43.995 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (102m) 9m ago 102m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:49:43.995 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (101m) 9m ago 101m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:49:43.995 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (92m) 9m ago 100m 433M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:49:43.995 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (92m) 9m ago 100m 471M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:49:43.996 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (91m) 9m ago 100m 356M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:49:43.996 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (91m) 9m ago 99m 504M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:49:43.996 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (90m) 9m ago 99m 396M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:49:43.996 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (90m) 9m ago 99m 408M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:49:43.996 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (101m) 9m ago 101m 66.8M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:49:43.997 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.994+0000 7f54c1cef700 1 -- 172.21.15.27:0/230067595 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f54a40640b0 msgr2=0x7f54a4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:49:43.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.994+0000 7f54c1cef700 1 --2- 172.21.15.27:0/230067595 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f54a40640b0 0x7f54a4066570 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7f54b0009da0 tx=0x7f54b0009b30).stop 2021-11-03T18:49:43.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.994+0000 7f54c1cef700 1 -- 172.21.15.27:0/230067595 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54bc0f66c0 msgr2=0x7f54bc106f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:49:43.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.994+0000 7f54c1cef700 1 --2- 172.21.15.27:0/230067595 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54bc0f66c0 0x7f54bc106f50 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 rx=0x7f54ac00c530 tx=0x7f54ac007e10).stop 2021-11-03T18:49:43.998 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.994+0000 7f54c1cef700 1 -- 172.21.15.27:0/230067595 shutdown_connections 2021-11-03T18:49:43.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.994+0000 7f54c1cef700 1 --2- 172.21.15.27:0/230067595 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f54a40640b0 0x7f54a4066570 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:43.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.994+0000 7f54c1cef700 1 --2- 172.21.15.27:0/230067595 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f54bc0f66c0 0x7f54bc106f50 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:43.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.994+0000 7f54c1cef700 1 --2- 172.21.15.27:0/230067595 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f54bc0f7960 0x7f54bc107440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:43.999 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.994+0000 7f54c1cef700 1 -- 172.21.15.27:0/230067595 >> 172.21.15.27:0/230067595 conn(0x7f54bc0f1e20 msgr2=0x7f54bc0fb3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:49:44.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.994+0000 7f54c1cef700 1 -- 172.21.15.27:0/230067595 shutdown_connections 2021-11-03T18:49:44.000 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:43.994+0000 7f54c1cef700 1 -- 172.21.15.27:0/230067595 wait complete. 2021-11-03T18:49:44.093 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.093+0000 7fd306df9700 1 -- 172.21.15.27:0/1839726884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3000f66e0 msgr2=0x7fd3000f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:49:44.093 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.093+0000 7fd306df9700 1 --2- 172.21.15.27:0/1839726884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3000f66e0 0x7fd3000f6b00 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 rx=0x7fd2f8004660 tx=0x7fd2f8009b30).stop 2021-11-03T18:49:44.093 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.093+0000 7fd306df9700 1 -- 172.21.15.27:0/1839726884 shutdown_connections 2021-11-03T18:49:44.093 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.093+0000 7fd306df9700 1 --2- 172.21.15.27:0/1839726884 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd3000f7980 0x7fd3000f7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:44.094 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.093+0000 7fd306df9700 1 --2- 172.21.15.27:0/1839726884 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3000f66e0 0x7fd3000f6b00 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:44.094 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.093+0000 7fd306df9700 1 -- 172.21.15.27:0/1839726884 >> 172.21.15.27:0/1839726884 conn(0x7fd3000f1e40 msgr2=0x7fd3000f4260 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:49:44.094 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.093+0000 7fd306df9700 1 -- 172.21.15.27:0/1839726884 shutdown_connections 2021-11-03T18:49:44.095 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.093+0000 7fd306df9700 1 -- 172.21.15.27:0/1839726884 wait complete. 2021-11-03T18:49:44.095 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.093+0000 7fd306df9700 1 Processor -- start 2021-11-03T18:49:44.096 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.093+0000 7fd306df9700 1 -- start start 2021-11-03T18:49:44.098 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.094+0000 7fd306df9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd3000f66e0 0x7fd30006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:49:44.098 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.094+0000 7fd306df9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3000f7980 0x7fd300069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:49:44.098 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.094+0000 7fd306df9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30006a380 con 0x7fd3000f7980 2021-11-03T18:49:44.098 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.094+0000 7fd306df9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30006a4c0 con 0x7fd3000f66e0 2021-11-03T18:49:44.099 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.094+0000 7fd2fffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3000f7980 0x7fd300069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:49:44.099 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.094+0000 7fd2fffff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3000f7980 0x7fd300069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37092/0 (socket says 172.21.15.27:37092) 2021-11-03T18:49:44.099 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.094+0000 7fd2fffff700 1 -- 172.21.15.27:0/2534200780 learned_addr learned my addr 172.21.15.27:0/2534200780 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:49:44.100 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.094+0000 7fd2fffff700 1 -- 172.21.15.27:0/2534200780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd3000f66e0 msgr2=0x7fd30006b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:49:44.100 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.094+0000 7fd2fffff700 1 --2- 172.21.15.27:0/2534200780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd3000f66e0 0x7fd30006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:49:44.100 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.094+0000 7fd2fffff700 1 -- 172.21.15.27:0/2534200780 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2f8005040 con 0x7fd3000f7980 2021-11-03T18:49:44.100 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.094+0000 7fd2fffff700 1 --2- 172.21.15.27:0/2534200780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3000f7980 0x7fd300069e20 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 rx=0x7fd2f000c760 tx=0x7fd2f0004d20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:49:44.101 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.094+0000 7fd2fd7fa700 1 -- 172.21.15.27:0/2534200780 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2f0004200 con 0x7fd3000f7980 2021-11-03T18:49:44.101 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.094+0000 7fd2fd7fa700 1 -- 172.21.15.27:0/2534200780 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fd2f00109c0 con 0x7fd3000f7980 2021-11-03T18:49:44.101 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.094+0000 7fd306df9700 1 -- 172.21.15.27:0/2534200780 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd30006a7a0 con 0x7fd3000f7980 2021-11-03T18:49:44.101 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.095+0000 7fd2fd7fa700 1 -- 172.21.15.27:0/2534200780 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2f0003920 con 0x7fd3000f7980 2021-11-03T18:49:44.102 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.095+0000 7fd306df9700 1 -- 172.21.15.27:0/2534200780 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd30010efe0 con 0x7fd3000f7980 2021-11-03T18:49:44.102 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.095+0000 7fd306df9700 1 -- 172.21.15.27:0/2534200780 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd300045ad0 con 0x7fd3000f7980 2021-11-03T18:49:44.102 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.096+0000 7fd2fd7fa700 1 -- 172.21.15.27:0/2534200780 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fd2f0018020 con 0x7fd3000f7980 2021-11-03T18:49:44.102 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.097+0000 7fd2fd7fa700 1 --2- 172.21.15.27:0/2534200780 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd2ec0640b0 0x7fd2ec066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:49:44.103 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.097+0000 7fd2fd7fa700 1 -- 172.21.15.27:0/2534200780 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fd2f0080bd0 con 0x7fd3000f7980 2021-11-03T18:49:44.103 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.097+0000 7fd304b95700 1 --2- 172.21.15.27:0/2534200780 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd2ec0640b0 0x7fd2ec066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:49:44.103 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.097+0000 7fd304b95700 1 --2- 172.21.15.27:0/2534200780 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd2ec0640b0 0x7fd2ec066570 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0x7fd2f8004e30 tx=0x7fd2f8009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:49:44.104 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.100+0000 7fd2fd7fa700 1 -- 172.21.15.27:0/2534200780 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fd2f0052e40 con 0x7fd3000f7980 2021-11-03T18:49:44.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.330+0000 7fd306df9700 1 -- 172.21.15.27:0/2534200780 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd30006a930 con 0x7fd3000f7980 2021-11-03T18:49:44.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.331+0000 7fd2fd7fa700 1 -- 172.21.15.27:0/2534200780 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7fd2f00212e0 con 0x7fd3000f7980 2021-11-03T18:49:44.331 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:49:44.331 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:49:44.332 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:49:44.332 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:49:44.332 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:49:44.332 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:49:44.332 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:49:44.333 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:49:44.333 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:49:44.333 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:49:44.333 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:49:44.334 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:49:44.334 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:49:44.334 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:49:44.334 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:49:44.335 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:49:44.335 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:49:44.335 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:49:44.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.333+0000 7fd306df9700 1 -- 172.21.15.27:0/2534200780 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd2ec0640b0 msgr2=0x7fd2ec066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:49:44.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.333+0000 7fd306df9700 1 --2- 172.21.15.27:0/2534200780 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd2ec0640b0 0x7fd2ec066570 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0x7fd2f8004e30 tx=0x7fd2f8009b30).stop 2021-11-03T18:49:44.336 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.333+0000 7fd306df9700 1 -- 172.21.15.27:0/2534200780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3000f7980 msgr2=0x7fd300069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:49:44.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.333+0000 7fd306df9700 1 --2- 172.21.15.27:0/2534200780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3000f7980 0x7fd300069e20 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 rx=0x7fd2f000c760 tx=0x7fd2f0004d20).stop 2021-11-03T18:49:44.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.333+0000 7fd306df9700 1 -- 172.21.15.27:0/2534200780 shutdown_connections 2021-11-03T18:49:44.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.333+0000 7fd306df9700 1 --2- 172.21.15.27:0/2534200780 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fd2ec0640b0 0x7fd2ec066570 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:44.337 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.333+0000 7fd306df9700 1 --2- 172.21.15.27:0/2534200780 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fd3000f66e0 0x7fd30006b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:49:44.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.333+0000 7fd306df9700 1 --2- 172.21.15.27:0/2534200780 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fd3000f7980 0x7fd300069e20 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:44.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.333+0000 7fd306df9700 1 -- 172.21.15.27:0/2534200780 >> 172.21.15.27:0/2534200780 conn(0x7fd3000f1e40 msgr2=0x7fd3000fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:49:44.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.333+0000 7fd306df9700 1 -- 172.21.15.27:0/2534200780 shutdown_connections 2021-11-03T18:49:44.338 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.333+0000 7fd306df9700 1 -- 172.21.15.27:0/2534200780 wait complete. 2021-11-03T18:49:44.434 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.433+0000 7fca83b91700 1 -- 172.21.15.27:0/137560482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca7c0f6b90 msgr2=0x7fca7c0f6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:49:44.434 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.433+0000 7fca83b91700 1 --2- 172.21.15.27:0/137560482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca7c0f6b90 0x7fca7c0f6fb0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 rx=0x7fca6c004660 tx=0x7fca6c009b30).stop 2021-11-03T18:49:44.435 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.433+0000 7fca83b91700 1 -- 172.21.15.27:0/137560482 shutdown_connections 2021-11-03T18:49:44.435 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.433+0000 7fca83b91700 1 --2- 172.21.15.27:0/137560482 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca7c0f74a0 0x7fca7c0f5930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:44.435 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.433+0000 7fca83b91700 1 --2- 172.21.15.27:0/137560482 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca7c0f6b90 0x7fca7c0f6fb0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:44.436 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.433+0000 7fca83b91700 1 -- 172.21.15.27:0/137560482 >> 172.21.15.27:0/137560482 conn(0x7fca7c0f0540 msgr2=0x7fca7c0f2960 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:49:44.436 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.433+0000 7fca83b91700 1 -- 172.21.15.27:0/137560482 shutdown_connections 2021-11-03T18:49:44.436 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.433+0000 7fca83b91700 1 -- 172.21.15.27:0/137560482 wait complete. 2021-11-03T18:49:44.436 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.433+0000 7fca83b91700 1 Processor -- start 2021-11-03T18:49:44.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.433+0000 7fca83b91700 1 -- start start 2021-11-03T18:49:44.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.434+0000 7fca83b91700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca7c0f6b90 0x7fca7c106f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:49:44.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.434+0000 7fca83b91700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca7c0f74a0 0x7fca7c107470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:49:44.437 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.434+0000 7fca83b91700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c107a20 con 0x7fca7c0f6b90 2021-11-03T18:49:44.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.434+0000 7fca83b91700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c107b60 con 0x7fca7c0f74a0 2021-11-03T18:49:44.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.434+0000 7fca8192d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca7c0f6b90 0x7fca7c106f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:49:44.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.434+0000 7fca8192d700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca7c0f6b90 0x7fca7c106f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37098/0 (socket says 172.21.15.27:37098) 2021-11-03T18:49:44.438 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.434+0000 7fca8192d700 1 -- 172.21.15.27:0/3317620567 learned_addr learned my addr 172.21.15.27:0/3317620567 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:49:44.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.434+0000 7fca8112c700 1 --2- 172.21.15.27:0/3317620567 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca7c0f74a0 0x7fca7c107470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:49:44.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.434+0000 7fca8192d700 1 -- 172.21.15.27:0/3317620567 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca7c0f74a0 msgr2=0x7fca7c107470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:49:44.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.434+0000 7fca8192d700 1 --2- 172.21.15.27:0/3317620567 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca7c0f74a0 0x7fca7c107470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:44.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.434+0000 7fca8192d700 1 -- 172.21.15.27:0/3317620567 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca6c005040 con 0x7fca7c0f6b90 2021-11-03T18:49:44.439 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.434+0000 7fca8192d700 1 --2- 172.21.15.27:0/3317620567 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca7c0f6b90 0x7fca7c106f80 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 rx=0x7fca6c004f60 tx=0x7fca6c009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:49:44.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.434+0000 7fca727fc700 1 -- 172.21.15.27:0/3317620567 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca6c018070 con 0x7fca7c0f6b90 2021-11-03T18:49:44.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.434+0000 7fca727fc700 1 -- 172.21.15.27:0/3317620567 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fca6c009770 con 0x7fca7c0f6b90 2021-11-03T18:49:44.440 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.434+0000 7fca83b91700 1 -- 172.21.15.27:0/3317620567 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca7c1084a0 con 0x7fca7c0f6b90 2021-11-03T18:49:44.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.435+0000 7fca727fc700 1 -- 172.21.15.27:0/3317620567 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fca6c01c3c0 con 0x7fca7c0f6b90 2021-11-03T18:49:44.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.435+0000 7fca83b91700 1 -- 172.21.15.27:0/3317620567 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca7c108a00 con 0x7fca7c0f6b90 2021-11-03T18:49:44.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.436+0000 7fca83b91700 1 -- 172.21.15.27:0/3317620567 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca7c045ad0 con 0x7fca7c0f6b90 2021-11-03T18:49:44.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.436+0000 7fca727fc700 1 -- 172.21.15.27:0/3317620567 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fca6c01d420 con 0x7fca7c0f6b90 2021-11-03T18:49:44.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.436+0000 7fca727fc700 1 --2- 172.21.15.27:0/3317620567 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fca680640b0 0x7fca68066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:49:44.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.437+0000 7fca727fc700 1 -- 172.21.15.27:0/3317620567 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fca6c082510 con 0x7fca7c0f6b90 2021-11-03T18:49:44.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.437+0000 7fca8112c700 1 --2- 172.21.15.27:0/3317620567 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fca680640b0 0x7fca68066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:49:44.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.437+0000 7fca8112c700 1 --2- 172.21.15.27:0/3317620567 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fca680640b0 0x7fca68066570 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 rx=0x7fca780041d0 tx=0x7fca78009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:49:44.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.441+0000 7fca727fc700 1 -- 172.21.15.27:0/3317620567 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fca6c054930 con 0x7fca7c0f6b90 2021-11-03T18:49:44.644 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.645+0000 7fca83b91700 1 -- 172.21.15.27:0/3317620567 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fca7c10d5a0 con 0x7fca7c0f6b90 2021-11-03T18:49:44.645 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.645+0000 7fca727fc700 1 -- 172.21.15.27:0/3317620567 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7fca6c054520 con 0x7fca7c0f6b90 2021-11-03T18:49:44.645 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:49:44.645 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:49:44.646 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:49:44.646 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:49:44.646 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:49:44.646 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:49:44.646 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:49:44.647 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:49:44.647 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:49:44.647 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:49:44.647 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:49:44.648 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:49:44.648 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:49:44.648 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:49:44.648 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:49:44.648 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:49:44.649 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:49:44.649 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:49:44.649 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:49:44.649 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:49:44.650 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:49:44.650 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:49:44.650 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:49:44.650 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:49:44.650 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:49:44.651 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:49:44.651 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:49:44.651 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:49:44.651 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:49:44.652 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:49:44.652 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:49:44.652 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:49:44.652 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:49:44.652 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:49:44.653 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:49:44.653 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:49:44.653 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:49:44.653 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:49:44.654 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:49:44.654 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.647+0000 7fca83b91700 1 -- 172.21.15.27:0/3317620567 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fca680640b0 msgr2=0x7fca68066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:49:44.655 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.647+0000 7fca83b91700 1 --2- 172.21.15.27:0/3317620567 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fca680640b0 0x7fca68066570 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 rx=0x7fca780041d0 tx=0x7fca78009040).stop 2021-11-03T18:49:44.655 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.647+0000 7fca83b91700 1 -- 172.21.15.27:0/3317620567 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca7c0f6b90 msgr2=0x7fca7c106f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:49:44.655 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.647+0000 7fca83b91700 1 --2- 172.21.15.27:0/3317620567 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca7c0f6b90 0x7fca7c106f80 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 rx=0x7fca6c004f60 tx=0x7fca6c009b30).stop 2021-11-03T18:49:44.655 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.647+0000 7fca83b91700 1 -- 172.21.15.27:0/3317620567 shutdown_connections 2021-11-03T18:49:44.656 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.647+0000 7fca83b91700 1 --2- 172.21.15.27:0/3317620567 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fca680640b0 0x7fca68066570 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:44.656 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.647+0000 7fca83b91700 1 --2- 172.21.15.27:0/3317620567 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fca7c0f6b90 0x7fca7c106f80 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:44.656 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.647+0000 7fca83b91700 1 --2- 172.21.15.27:0/3317620567 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fca7c0f74a0 0x7fca7c107470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:49:44.656 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.647+0000 7fca83b91700 1 -- 172.21.15.27:0/3317620567 >> 172.21.15.27:0/3317620567 conn(0x7fca7c0f0540 msgr2=0x7fca7c0f5040 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:49:44.657 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.647+0000 7fca83b91700 1 -- 172.21.15.27:0/3317620567 shutdown_connections 2021-11-03T18:49:44.657 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:49:44.647+0000 7fca83b91700 1 -- 172.21.15.27:0/3317620567 wait complete. 2021-11-03T18:49:44.657 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:49:45.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:44 smithi115 ceph-mon[65453]: pgmap v2854: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:45.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:44 smithi115 ceph-mon[65453]: from='client.37032 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:49:45.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:44 smithi115 ceph-mon[65453]: from='client.37036 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:49:45.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:44 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/2534200780' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:45.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:44 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/3317620567' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:49:45.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:44 smithi027 ceph-mon[97244]: pgmap v2854: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:45.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:44 smithi027 ceph-mon[97244]: from='client.37032 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:49:45.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:44 smithi027 ceph-mon[97244]: from='client.37036 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:49:45.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:44 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/2534200780' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:45.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:44 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/3317620567' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:49:46.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5627.495522 secs 2021-11-03T18:49:46.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5627.495522 secs 2021-11-03T18:49:47.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:46 smithi115 ceph-mon[65453]: pgmap v2855: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:47.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:46 smithi027 ceph-mon[97244]: pgmap v2855: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:49.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:48 smithi115 ceph-mon[65453]: pgmap v2856: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:49.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:48 smithi027 ceph-mon[97244]: pgmap v2856: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:51.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:50 smithi027 ceph-mon[97244]: pgmap v2857: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:51.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5632.495645 secs 2021-11-03T18:49:51.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:50 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:49:51.067 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:50 smithi115 ceph-mon[65453]: pgmap v2857: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:51.067 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5632.495645 secs 2021-11-03T18:49:51.067 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:50 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:49:52.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:51 smithi115 ceph-mon[65453]: poll took 60194.715 ms: 1 events 2021-11-03T18:49:52.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:49:52.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:49:52.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:52.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:52.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:52.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:51 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:52.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:51 smithi027 ceph-mon[97244]: poll took 60194.715 ms: 1 events 2021-11-03T18:49:52.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:49:52.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:49:52.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:52.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:52.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:52.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:51 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:49:53.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:52 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:49:53.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:52 smithi115 ceph-mon[65453]: pgmap v2858: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:53.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:52 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:49:53.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:52 smithi027 ceph-mon[97244]: pgmap v2858: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:55.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:54 smithi115 ceph-mon[65453]: pgmap v2859: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:55.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:54 smithi027 ceph-mon[97244]: pgmap v2859: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:56.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5637.495781 secs 2021-11-03T18:49:56.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:49:56.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:55 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:49:56.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5637.495781 secs 2021-11-03T18:49:56.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:49:56.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:55 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:49:57.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:56 smithi115 ceph-mon[65453]: pgmap v2860: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:57.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:56 smithi027 ceph-mon[97244]: pgmap v2860: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:59.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:49:58 smithi115 ceph-mon[65453]: pgmap v2861: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:49:59.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:49:58 smithi027 ceph-mon[97244]: pgmap v2861: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:01.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:00 smithi115 ceph-mon[65453]: pgmap v2862: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:01.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5642.495922 secs 2021-11-03T18:50:01.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:00 smithi115 ceph-mon[65453]: overall HEALTH_WARN 1 filesystem is degraded; 1 filesystem has a failed mds daemon; 1 MDSs report slow requests; 3 daemons have recently crashed 2021-11-03T18:50:01.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:00 smithi027 ceph-mon[97244]: pgmap v2862: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:01.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5642.495922 secs 2021-11-03T18:50:01.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:00 smithi027 ceph-mon[97244]: overall HEALTH_WARN 1 filesystem is degraded; 1 filesystem has a failed mds daemon; 1 MDSs report slow requests; 3 daemons have recently crashed 2021-11-03T18:50:02.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:01 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:50:02.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:01 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:50:03.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:02 smithi115 ceph-mon[65453]: poll took 10027.656 ms: 1 events 2021-11-03T18:50:03.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:02 smithi115 ceph-mon[65453]: pgmap v2863: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:03.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:02 smithi027 ceph-mon[97244]: poll took 10027.656 ms: 1 events 2021-11-03T18:50:03.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:02 smithi027 ceph-mon[97244]: pgmap v2863: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:05.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:04 smithi027 ceph-mon[97244]: pgmap v2864: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:05.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:04 smithi115 ceph-mon[65453]: pgmap v2864: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:06.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5647.495991 secs 2021-11-03T18:50:06.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5647.495991 secs 2021-11-03T18:50:07.275 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:06 smithi027 ceph-mon[97244]: pgmap v2865: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:07.276 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:50:07.276 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:50:07.276 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:50:07.276 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:07.277 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:07.277 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:07.277 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:06 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:07.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:06 smithi115 ceph-mon[65453]: pgmap v2865: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:07.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:50:07.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:50:07.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:50:07.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:07.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:07.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:07.489 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:06 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:08.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:08 smithi027 ceph-mon[97244]: poll took 5153.729 ms: 1 events 2021-11-03T18:50:08.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:08 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:50:08.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:08 smithi115 ceph-mon[65453]: poll took 5153.729 ms: 1 events 2021-11-03T18:50:08.488 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:08 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:50:09.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:09 smithi027 ceph-mon[97244]: pgmap v2866: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:09.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:09 smithi115 ceph-mon[65453]: pgmap v2866: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:10.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:10 smithi027 ceph-mon[97244]: pgmap v2867: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:10.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:10 smithi115 ceph-mon[65453]: pgmap v2867: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:11.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:11 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5652.496132 secs 2021-11-03T18:50:11.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:11 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5652.496132 secs 2021-11-03T18:50:12.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:12 smithi027 ceph-mon[97244]: pgmap v2868: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:12.487 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:12 smithi115 ceph-mon[65453]: pgmap v2868: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:14.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:14 smithi115 ceph-mon[65453]: pgmap v2869: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:14.764 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.762+0000 7f5dc0635700 1 -- 172.21.15.27:0/967878382 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5db80f49a0 msgr2=0x7f5db80f4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:50:14.764 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.762+0000 7f5dc0635700 1 --2- 172.21.15.27:0/967878382 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5db80f49a0 0x7f5db80f4dc0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 rx=0x7f5db0004660 tx=0x7f5db0009b30).stop 2021-11-03T18:50:14.765 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.762+0000 7f5dc0635700 1 -- 172.21.15.27:0/967878382 shutdown_connections 2021-11-03T18:50:14.765 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.762+0000 7f5dc0635700 1 --2- 172.21.15.27:0/967878382 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5db80f5340 0x7f5db80f7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:14.765 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.762+0000 7f5dc0635700 1 --2- 172.21.15.27:0/967878382 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5db80f49a0 0x7f5db80f4dc0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:14.765 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.762+0000 7f5dc0635700 1 -- 172.21.15.27:0/967878382 >> 172.21.15.27:0/967878382 conn(0x7f5db80f04c0 msgr2=0x7f5db80f28e0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:50:14.766 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.762+0000 7f5dc0635700 1 -- 172.21.15.27:0/967878382 shutdown_connections 2021-11-03T18:50:14.766 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.762+0000 7f5dc0635700 1 -- 172.21.15.27:0/967878382 wait complete. 2021-11-03T18:50:14.766 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.763+0000 7f5dc0635700 1 Processor -- start 2021-11-03T18:50:14.766 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.763+0000 7f5dc0635700 1 -- start start 2021-11-03T18:50:14.766 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.763+0000 7f5dc0635700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5db80f49a0 0x7f5db8106fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:50:14.767 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.763+0000 7f5dc0635700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5db80f5340 0x7f5db81074b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:50:14.767 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.763+0000 7f5dc0635700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db8107a60 con 0x7f5db80f5340 2021-11-03T18:50:14.767 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.763+0000 7f5dc0635700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5db8107ba0 con 0x7f5db80f49a0 2021-11-03T18:50:14.767 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.763+0000 7f5dbdbd0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5db80f5340 0x7f5db81074b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:50:14.768 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.763+0000 7f5dbdbd0700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5db80f5340 0x7f5db81074b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37118/0 (socket says 172.21.15.27:37118) 2021-11-03T18:50:14.768 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.763+0000 7f5dbdbd0700 1 -- 172.21.15.27:0/880256735 learned_addr learned my addr 172.21.15.27:0/880256735 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:50:14.768 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.763+0000 7f5dbe3d1700 1 --2- 172.21.15.27:0/880256735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5db80f49a0 0x7f5db8106fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:50:14.768 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.764+0000 7f5dbdbd0700 1 -- 172.21.15.27:0/880256735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5db80f49a0 msgr2=0x7f5db8106fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:50:14.768 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.764+0000 7f5dbdbd0700 1 --2- 172.21.15.27:0/880256735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5db80f49a0 0x7f5db8106fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:14.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.764+0000 7f5dbdbd0700 1 -- 172.21.15.27:0/880256735 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5db0005040 con 0x7f5db80f5340 2021-11-03T18:50:14.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.764+0000 7f5dbe3d1700 1 --2- 172.21.15.27:0/880256735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5db80f49a0 0x7f5db8106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:50:14.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.764+0000 7f5dbdbd0700 1 --2- 172.21.15.27:0/880256735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5db80f5340 0x7f5db81074b0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 rx=0x7f5dac002700 tx=0x7f5dac009d60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:50:14.769 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.764+0000 7f5db6ffd700 1 -- 172.21.15.27:0/880256735 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5dac009320 con 0x7f5db80f5340 2021-11-03T18:50:14.770 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.764+0000 7f5db6ffd700 1 -- 172.21.15.27:0/880256735 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f5dac00f040 con 0x7f5db80f5340 2021-11-03T18:50:14.770 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.764+0000 7f5dc0635700 1 -- 172.21.15.27:0/880256735 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5db8108540 con 0x7f5db80f5340 2021-11-03T18:50:14.770 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.764+0000 7f5db6ffd700 1 -- 172.21.15.27:0/880256735 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5dac014810 con 0x7f5db80f5340 2021-11-03T18:50:14.770 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.764+0000 7f5dc0635700 1 -- 172.21.15.27:0/880256735 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5db8108b00 con 0x7f5db80f5340 2021-11-03T18:50:14.771 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.765+0000 7f5dc0635700 1 -- 172.21.15.27:0/880256735 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5db8045ad0 con 0x7f5db80f5340 2021-11-03T18:50:14.772 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.766+0000 7f5db6ffd700 1 -- 172.21.15.27:0/880256735 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f5dac0077b0 con 0x7f5db80f5340 2021-11-03T18:50:14.772 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.766+0000 7f5db6ffd700 1 --2- 172.21.15.27:0/880256735 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5da00640b0 0x7f5da0066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:50:14.772 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.766+0000 7f5db6ffd700 1 -- 172.21.15.27:0/880256735 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f5dac080e10 con 0x7f5db80f5340 2021-11-03T18:50:14.772 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.766+0000 7f5dbe3d1700 1 --2- 172.21.15.27:0/880256735 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5da00640b0 0x7f5da0066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:50:14.773 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.766+0000 7f5dbe3d1700 1 --2- 172.21.15.27:0/880256735 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5da00640b0 0x7f5da0066570 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 rx=0x7f5db0005010 tx=0x7f5db0009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:50:14.773 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.770+0000 7f5db6ffd700 1 -- 172.21.15.27:0/880256735 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f5dac053170 con 0x7f5db80f5340 2021-11-03T18:50:14.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:14 smithi027 ceph-mon[97244]: pgmap v2869: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:14.948 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.949+0000 7f5dc0635700 1 -- 172.21.15.27:0/880256735 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5db8108d80 con 0x7f5da00640b0 2021-11-03T18:50:14.949 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.950+0000 7f5db6ffd700 1 -- 172.21.15.27:0/880256735 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f5db8108d80 con 0x7f5da00640b0 2021-11-03T18:50:14.951 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.952+0000 7f5dc0635700 1 -- 172.21.15.27:0/880256735 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5da00640b0 msgr2=0x7f5da0066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:50:14.952 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.952+0000 7f5dc0635700 1 --2- 172.21.15.27:0/880256735 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5da00640b0 0x7f5da0066570 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 rx=0x7f5db0005010 tx=0x7f5db0009b30).stop 2021-11-03T18:50:14.952 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.952+0000 7f5dc0635700 1 -- 172.21.15.27:0/880256735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5db80f5340 msgr2=0x7f5db81074b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:50:14.952 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.952+0000 7f5dc0635700 1 --2- 172.21.15.27:0/880256735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5db80f5340 0x7f5db81074b0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 rx=0x7f5dac002700 tx=0x7f5dac009d60).stop 2021-11-03T18:50:14.952 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.952+0000 7f5dc0635700 1 -- 172.21.15.27:0/880256735 shutdown_connections 2021-11-03T18:50:14.953 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.952+0000 7f5dc0635700 1 --2- 172.21.15.27:0/880256735 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f5da00640b0 0x7f5da0066570 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:14.953 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.952+0000 7f5dc0635700 1 --2- 172.21.15.27:0/880256735 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f5db80f49a0 0x7f5db8106fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:14.953 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.952+0000 7f5dc0635700 1 --2- 172.21.15.27:0/880256735 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f5db80f5340 0x7f5db81074b0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:14.953 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.952+0000 7f5dc0635700 1 -- 172.21.15.27:0/880256735 >> 172.21.15.27:0/880256735 conn(0x7f5db80f04c0 msgr2=0x7f5db80fb510 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:50:14.954 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.952+0000 7f5dc0635700 1 -- 172.21.15.27:0/880256735 shutdown_connections 2021-11-03T18:50:14.954 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:14.952+0000 7f5dc0635700 1 -- 172.21.15.27:0/880256735 wait complete. 2021-11-03T18:50:14.961 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:50:15.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.052+0000 7f43a1be5700 1 -- 172.21.15.27:0/1850937201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f439c0f5710 msgr2=0x7f439c0f5b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:50:15.054 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.052+0000 7f43a1be5700 1 --2- 172.21.15.27:0/1850937201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f439c0f5710 0x7f439c0f5b90 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 rx=0x7f4388004660 tx=0x7f4388009b30).stop 2021-11-03T18:50:15.055 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.052+0000 7f43a1be5700 1 -- 172.21.15.27:0/1850937201 shutdown_connections 2021-11-03T18:50:15.055 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.052+0000 7f43a1be5700 1 --2- 172.21.15.27:0/1850937201 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f439c0f5710 0x7f439c0f5b90 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:15.055 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.052+0000 7f43a1be5700 1 --2- 172.21.15.27:0/1850937201 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f439c0f6a10 0x7f439c0f6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:15.055 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.052+0000 7f43a1be5700 1 -- 172.21.15.27:0/1850937201 >> 172.21.15.27:0/1850937201 conn(0x7f439c0f0320 msgr2=0x7f439c0f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:50:15.055 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.052+0000 7f43a1be5700 1 -- 172.21.15.27:0/1850937201 shutdown_connections 2021-11-03T18:50:15.056 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.052+0000 7f43a1be5700 1 -- 172.21.15.27:0/1850937201 wait complete. 2021-11-03T18:50:15.056 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.053+0000 7f43a1be5700 1 Processor -- start 2021-11-03T18:50:15.056 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.053+0000 7f43a1be5700 1 -- start start 2021-11-03T18:50:15.056 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.053+0000 7f43a1be5700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f439c0f5710 0x7f439c105c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:50:15.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.053+0000 7f43a1be5700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f439c0f6a10 0x7f439c1060f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:50:15.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.053+0000 7f43a1be5700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f439c1066a0 con 0x7f439c0f6a10 2021-11-03T18:50:15.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.053+0000 7f43a1be5700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f439c1067e0 con 0x7f439c0f5710 2021-11-03T18:50:15.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.053+0000 7f439bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f439c0f6a10 0x7f439c1060f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:50:15.057 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.053+0000 7f439bfff700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f439c0f6a10 0x7f439c1060f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37126/0 (socket says 172.21.15.27:37126) 2021-11-03T18:50:15.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.053+0000 7f439bfff700 1 -- 172.21.15.27:0/3299279625 learned_addr learned my addr 172.21.15.27:0/3299279625 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:50:15.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.054+0000 7f43a0be3700 1 --2- 172.21.15.27:0/3299279625 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f439c0f5710 0x7f439c105c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:50:15.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.054+0000 7f439bfff700 1 -- 172.21.15.27:0/3299279625 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f439c0f5710 msgr2=0x7f439c105c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:50:15.058 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.054+0000 7f439bfff700 1 --2- 172.21.15.27:0/3299279625 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f439c0f5710 0x7f439c105c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:15.059 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.054+0000 7f439bfff700 1 -- 172.21.15.27:0/3299279625 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4388005040 con 0x7f439c0f6a10 2021-11-03T18:50:15.059 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.054+0000 7f43a0be3700 1 --2- 172.21.15.27:0/3299279625 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f439c0f5710 0x7f439c105c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T18:50:15.059 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.054+0000 7f439bfff700 1 --2- 172.21.15.27:0/3299279625 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f439c0f6a10 0x7f439c1060f0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 rx=0x7f4388014040 tx=0x7f438800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:50:15.059 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.054+0000 7f43997fa700 1 -- 172.21.15.27:0/3299279625 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4388019070 con 0x7f439c0f6a10 2021-11-03T18:50:15.060 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.054+0000 7f43997fa700 1 -- 172.21.15.27:0/3299279625 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f4388007e70 con 0x7f439c0f6a10 2021-11-03T18:50:15.060 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.054+0000 7f43a1be5700 1 -- 172.21.15.27:0/3299279625 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f439c107120 con 0x7f439c0f6a10 2021-11-03T18:50:15.060 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.054+0000 7f43997fa700 1 -- 172.21.15.27:0/3299279625 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4388014d50 con 0x7f439c0f6a10 2021-11-03T18:50:15.060 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.054+0000 7f43a1be5700 1 -- 172.21.15.27:0/3299279625 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f439c107680 con 0x7f439c0f6a10 2021-11-03T18:50:15.061 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.055+0000 7f43a1be5700 1 -- 172.21.15.27:0/3299279625 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f439c045ad0 con 0x7f439c0f6a10 2021-11-03T18:50:15.062 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.056+0000 7f43997fa700 1 -- 172.21.15.27:0/3299279625 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f4388025070 con 0x7f439c0f6a10 2021-11-03T18:50:15.062 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.056+0000 7f43997fa700 1 --2- 172.21.15.27:0/3299279625 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f438c064100 0x7f438c0665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:50:15.062 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.056+0000 7f43997fa700 1 -- 172.21.15.27:0/3299279625 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f43880817f0 con 0x7f439c0f6a10 2021-11-03T18:50:15.062 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.056+0000 7f43a0be3700 1 --2- 172.21.15.27:0/3299279625 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f438c064100 0x7f438c0665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:50:15.063 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.057+0000 7f43a0be3700 1 --2- 172.21.15.27:0/3299279625 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f438c064100 0x7f438c0665c0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 rx=0x7f4390009da0 tx=0x7f439000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:50:15.063 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.060+0000 7f43997fa700 1 -- 172.21.15.27:0/3299279625 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f4388053b50 con 0x7f439c0f6a10 2021-11-03T18:50:15.232 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.233+0000 7f43a1be5700 1 -- 172.21.15.27:0/3299279625 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f439c1078a0 con 0x7f438c064100 2021-11-03T18:50:15.247 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.247+0000 7f43997fa700 1 -- 172.21.15.27:0/3299279625 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3542 (secure 0 0 0) 0x7f439c1078a0 con 0x7f438c064100 2021-11-03T18:50:15.247 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:50:15.248 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (101m) 8s ago 103m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:50:15.248 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (93m) 8s ago 103m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:50:15.248 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (93m) 9m ago 102m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:50:15.248 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (101m) 8s ago 102m 42.8M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:50:15.248 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (95m) 8s ago 98m 27.1M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:50:15.249 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (98m) 8s ago 98m 3853M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:50:15.249 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (94m) 9m ago 98m 26.5M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:50:15.249 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (95m) 9m ago 98m 27.6M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:50:15.249 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (95m) 8s ago 104m 644M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:50:15.250 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (94m) 9m ago 102m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:50:15.250 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (94m) 8s ago 104m 424M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:50:15.250 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (94m) 9m ago 102m 175M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:50:15.250 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (102m) 8s ago 102m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:50:15.251 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (102m) 9m ago 102m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:50:15.251 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (93m) 8s ago 101m 453M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:50:15.251 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (92m) 8s ago 101m 493M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:50:15.251 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (92m) 8s ago 100m 374M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:50:15.251 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (91m) 9m ago 100m 504M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:50:15.252 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (91m) 9m ago 100m 396M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:50:15.252 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (90m) 9m ago 99m 408M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:50:15.252 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (101m) 8s ago 102m 69.1M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:50:15.253 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.249+0000 7f43a1be5700 1 -- 172.21.15.27:0/3299279625 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f438c064100 msgr2=0x7f438c0665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:50:15.253 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.249+0000 7f43a1be5700 1 --2- 172.21.15.27:0/3299279625 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f438c064100 0x7f438c0665c0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 rx=0x7f4390009da0 tx=0x7f439000b040).stop 2021-11-03T18:50:15.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.249+0000 7f43a1be5700 1 -- 172.21.15.27:0/3299279625 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f439c0f6a10 msgr2=0x7f439c1060f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:50:15.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.249+0000 7f43a1be5700 1 --2- 172.21.15.27:0/3299279625 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f439c0f6a10 0x7f439c1060f0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 rx=0x7f4388014040 tx=0x7f438800b040).stop 2021-11-03T18:50:15.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.250+0000 7f43a1be5700 1 -- 172.21.15.27:0/3299279625 shutdown_connections 2021-11-03T18:50:15.254 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.250+0000 7f43a1be5700 1 --2- 172.21.15.27:0/3299279625 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f438c064100 0x7f438c0665c0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:15.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.250+0000 7f43a1be5700 1 --2- 172.21.15.27:0/3299279625 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f439c0f5710 0x7f439c105c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:15.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.250+0000 7f43a1be5700 1 --2- 172.21.15.27:0/3299279625 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f439c0f6a10 0x7f439c1060f0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:15.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.250+0000 7f43a1be5700 1 -- 172.21.15.27:0/3299279625 >> 172.21.15.27:0/3299279625 conn(0x7f439c0f0320 msgr2=0x7f439c0fa170 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:50:15.255 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.250+0000 7f43a1be5700 1 -- 172.21.15.27:0/3299279625 shutdown_connections 2021-11-03T18:50:15.256 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.250+0000 7f43a1be5700 1 -- 172.21.15.27:0/3299279625 wait complete. 2021-11-03T18:50:15.348 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.348+0000 7f10020d9700 1 -- 172.21.15.27:0/1599533065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ffc0fcce0 msgr2=0x7f0ffc0fd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:50:15.349 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.348+0000 7f10020d9700 1 --2- 172.21.15.27:0/1599533065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ffc0fcce0 0x7f0ffc0fd100 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 rx=0x7f0fec004660 tx=0x7f0fec009b30).stop 2021-11-03T18:50:15.349 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.349+0000 7f10020d9700 1 -- 172.21.15.27:0/1599533065 shutdown_connections 2021-11-03T18:50:15.349 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.349+0000 7f10020d9700 1 --2- 172.21.15.27:0/1599533065 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ffc0fdf80 0x7f0ffc0fe3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:15.349 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.349+0000 7f10020d9700 1 --2- 172.21.15.27:0/1599533065 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ffc0fcce0 0x7f0ffc0fd100 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:15.350 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.349+0000 7f10020d9700 1 -- 172.21.15.27:0/1599533065 >> 172.21.15.27:0/1599533065 conn(0x7f0ffc06c000 msgr2=0x7f0ffc06e420 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:50:15.350 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.349+0000 7f10020d9700 1 -- 172.21.15.27:0/1599533065 shutdown_connections 2021-11-03T18:50:15.350 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.349+0000 7f10020d9700 1 -- 172.21.15.27:0/1599533065 wait complete. 2021-11-03T18:50:15.350 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.349+0000 7f10020d9700 1 Processor -- start 2021-11-03T18:50:15.351 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.349+0000 7f10020d9700 1 -- start start 2021-11-03T18:50:15.353 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.349+0000 7f10020d9700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ffc0fcce0 0x7f0ffc10d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:50:15.353 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.349+0000 7f10020d9700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ffc0fdf80 0x7f0ffc10daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:50:15.353 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.349+0000 7f10020d9700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ffc10e050 con 0x7f0ffc0fdf80 2021-11-03T18:50:15.354 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.349+0000 7f10020d9700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ffc10e190 con 0x7f0ffc0fcce0 2021-11-03T18:50:15.354 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.350+0000 7f0ffaffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ffc0fdf80 0x7f0ffc10daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:50:15.354 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.350+0000 7f0ffaffd700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ffc0fdf80 0x7f0ffc10daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37138/0 (socket says 172.21.15.27:37138) 2021-11-03T18:50:15.354 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.350+0000 7f0ffaffd700 1 -- 172.21.15.27:0/987061768 learned_addr learned my addr 172.21.15.27:0/987061768 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:50:15.355 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.350+0000 7f0ffb7fe700 1 --2- 172.21.15.27:0/987061768 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ffc0fcce0 0x7f0ffc10d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:50:15.355 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.350+0000 7f0ffaffd700 1 -- 172.21.15.27:0/987061768 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ffc0fcce0 msgr2=0x7f0ffc10d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:50:15.355 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.350+0000 7f0ffaffd700 1 --2- 172.21.15.27:0/987061768 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ffc0fcce0 0x7f0ffc10d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:15.355 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.350+0000 7f0ffaffd700 1 -- 172.21.15.27:0/987061768 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fec005040 con 0x7f0ffc0fdf80 2021-11-03T18:50:15.356 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.350+0000 7f0ffaffd700 1 --2- 172.21.15.27:0/987061768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ffc0fdf80 0x7f0ffc10daa0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 rx=0x7f0ff000c530 tx=0x7f0ff0007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:50:15.356 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.350+0000 7f0fe3fff700 1 -- 172.21.15.27:0/987061768 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ff000f650 con 0x7f0ffc0fdf80 2021-11-03T18:50:15.356 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.350+0000 7f0fe3fff700 1 -- 172.21.15.27:0/987061768 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f0ff000fc60 con 0x7f0ffc0fdf80 2021-11-03T18:50:15.356 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.350+0000 7f10020d9700 1 -- 172.21.15.27:0/987061768 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ffc10eb30 con 0x7f0ffc0fdf80 2021-11-03T18:50:15.356 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.350+0000 7f0fe3fff700 1 -- 172.21.15.27:0/987061768 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0ff0012c20 con 0x7f0ffc0fdf80 2021-11-03T18:50:15.357 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.350+0000 7f10020d9700 1 -- 172.21.15.27:0/987061768 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ffc10f0f0 con 0x7f0ffc0fdf80 2021-11-03T18:50:15.357 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.351+0000 7f10020d9700 1 -- 172.21.15.27:0/987061768 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ffc045ad0 con 0x7f0ffc0fdf80 2021-11-03T18:50:15.357 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.352+0000 7f0fe3fff700 1 -- 172.21.15.27:0/987061768 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f0ff0010070 con 0x7f0ffc0fdf80 2021-11-03T18:50:15.357 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.352+0000 7f0fe3fff700 1 --2- 172.21.15.27:0/987061768 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0fe40640b0 0x7f0fe4066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:50:15.358 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.352+0000 7f0fe3fff700 1 -- 172.21.15.27:0/987061768 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f0ff0080570 con 0x7f0ffc0fdf80 2021-11-03T18:50:15.358 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.352+0000 7f0ffb7fe700 1 --2- 172.21.15.27:0/987061768 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0fe40640b0 0x7f0fe4066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:50:15.358 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.353+0000 7f0ffb7fe700 1 --2- 172.21.15.27:0/987061768 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0fe40640b0 0x7f0fe4066570 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 rx=0x7f0fec004630 tx=0x7f0fec009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:50:15.359 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.356+0000 7f0fe3fff700 1 -- 172.21.15.27:0/987061768 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f0ff00528a0 con 0x7f0ffc0fdf80 2021-11-03T18:50:15.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.590+0000 7f10020d9700 1 -- 172.21.15.27:0/987061768 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0ffc027a00 con 0x7f0ffc0fdf80 2021-11-03T18:50:15.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.591+0000 7f0fe3fff700 1 -- 172.21.15.27:0/987061768 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f0ff0052490 con 0x7f0ffc0fdf80 2021-11-03T18:50:15.591 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:50:15.591 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:50:15.592 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:50:15.592 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:50:15.592 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:50:15.592 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:50:15.593 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:50:15.593 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:50:15.593 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:50:15.593 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:50:15.594 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:50:15.594 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:50:15.594 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:50:15.594 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:50:15.594 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:50:15.595 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:50:15.595 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:50:15.595 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:50:15.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.593+0000 7f10020d9700 1 -- 172.21.15.27:0/987061768 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0fe40640b0 msgr2=0x7f0fe4066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:50:15.596 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.593+0000 7f10020d9700 1 --2- 172.21.15.27:0/987061768 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0fe40640b0 0x7f0fe4066570 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 rx=0x7f0fec004630 tx=0x7f0fec009b30).stop 2021-11-03T18:50:15.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.593+0000 7f10020d9700 1 -- 172.21.15.27:0/987061768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ffc0fdf80 msgr2=0x7f0ffc10daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:50:15.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.593+0000 7f10020d9700 1 --2- 172.21.15.27:0/987061768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ffc0fdf80 0x7f0ffc10daa0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 rx=0x7f0ff000c530 tx=0x7f0ff0007e20).stop 2021-11-03T18:50:15.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.593+0000 7f10020d9700 1 -- 172.21.15.27:0/987061768 shutdown_connections 2021-11-03T18:50:15.597 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.593+0000 7f10020d9700 1 --2- 172.21.15.27:0/987061768 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f0fe40640b0 0x7f0fe4066570 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:15.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.593+0000 7f10020d9700 1 --2- 172.21.15.27:0/987061768 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f0ffc0fcce0 0x7f0ffc10d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:15.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.593+0000 7f10020d9700 1 --2- 172.21.15.27:0/987061768 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f0ffc0fdf80 0x7f0ffc10daa0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:15.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.593+0000 7f10020d9700 1 -- 172.21.15.27:0/987061768 >> 172.21.15.27:0/987061768 conn(0x7f0ffc06c000 msgr2=0x7f0ffc1019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:50:15.598 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.593+0000 7f10020d9700 1 -- 172.21.15.27:0/987061768 shutdown_connections 2021-11-03T18:50:15.599 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.593+0000 7f10020d9700 1 -- 172.21.15.27:0/987061768 wait complete. 2021-11-03T18:50:15.695 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.694+0000 7f685c89a700 1 -- 172.21.15.27:0/3518887429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68580f6530 msgr2=0x7f68580f6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:50:15.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.694+0000 7f685c89a700 1 --2- 172.21.15.27:0/3518887429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68580f6530 0x7f68580f6950 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 rx=0x7f6850004660 tx=0x7f6850009b30).stop 2021-11-03T18:50:15.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.694+0000 7f685c89a700 1 -- 172.21.15.27:0/3518887429 shutdown_connections 2021-11-03T18:50:15.696 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.694+0000 7f685c89a700 1 --2- 172.21.15.27:0/3518887429 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68580f77d0 0x7f68580f7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:15.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.694+0000 7f685c89a700 1 --2- 172.21.15.27:0/3518887429 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68580f6530 0x7f68580f6950 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:15.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.694+0000 7f685c89a700 1 -- 172.21.15.27:0/3518887429 >> 172.21.15.27:0/3518887429 conn(0x7f68580f1cd0 msgr2=0x7f68580f40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:50:15.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.694+0000 7f685c89a700 1 -- 172.21.15.27:0/3518887429 shutdown_connections 2021-11-03T18:50:15.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.694+0000 7f685c89a700 1 -- 172.21.15.27:0/3518887429 wait complete. 2021-11-03T18:50:15.697 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.694+0000 7f685c89a700 1 Processor -- start 2021-11-03T18:50:15.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.694+0000 7f685c89a700 1 -- start start 2021-11-03T18:50:15.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.695+0000 7f685c89a700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68580f6530 0x7f6858106d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:50:15.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.695+0000 7f685c89a700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68580f77d0 0x7f6858107260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:50:15.698 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.695+0000 7f685c89a700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6858107860 con 0x7f68580f6530 2021-11-03T18:50:15.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.695+0000 7f685c89a700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6858108070 con 0x7f68580f77d0 2021-11-03T18:50:15.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.695+0000 7f68577fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68580f6530 0x7f6858106d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:50:15.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.695+0000 7f68577fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68580f6530 0x7f6858106d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37144/0 (socket says 172.21.15.27:37144) 2021-11-03T18:50:15.699 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.695+0000 7f68577fe700 1 -- 172.21.15.27:0/51464825 learned_addr learned my addr 172.21.15.27:0/51464825 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:50:15.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.695+0000 7f6856ffd700 1 --2- 172.21.15.27:0/51464825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68580f77d0 0x7f6858107260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:50:15.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.695+0000 7f68577fe700 1 -- 172.21.15.27:0/51464825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68580f77d0 msgr2=0x7f6858107260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:50:15.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.695+0000 7f68577fe700 1 --2- 172.21.15.27:0/51464825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68580f77d0 0x7f6858107260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:15.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.695+0000 7f68577fe700 1 -- 172.21.15.27:0/51464825 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6850005040 con 0x7f68580f6530 2021-11-03T18:50:15.700 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.695+0000 7f6856ffd700 1 --2- 172.21.15.27:0/51464825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68580f77d0 0x7f6858107260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2021-11-03T18:50:15.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.695+0000 7f68577fe700 1 --2- 172.21.15.27:0/51464825 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68580f6530 0x7f6858106d70 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 rx=0x7f6850009830 tx=0x7f6850009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:50:15.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.695+0000 7f684ffff700 1 -- 172.21.15.27:0/51464825 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6850018070 con 0x7f68580f6530 2021-11-03T18:50:15.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.695+0000 7f684ffff700 1 -- 172.21.15.27:0/51464825 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f6850007ba0 con 0x7f68580f6530 2021-11-03T18:50:15.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.695+0000 7f685c89a700 1 -- 172.21.15.27:0/51464825 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6858108290 con 0x7f68580f6530 2021-11-03T18:50:15.701 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.695+0000 7f684ffff700 1 -- 172.21.15.27:0/51464825 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f685001c620 con 0x7f68580f6530 2021-11-03T18:50:15.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.695+0000 7f685c89a700 1 -- 172.21.15.27:0/51464825 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68581087f0 con 0x7f68580f6530 2021-11-03T18:50:15.702 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.696+0000 7f685c89a700 1 -- 172.21.15.27:0/51464825 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6858059100 con 0x7f68580f6530 2021-11-03T18:50:15.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.697+0000 7f684ffff700 1 -- 172.21.15.27:0/51464825 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f6850007d10 con 0x7f68580f6530 2021-11-03T18:50:15.703 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.697+0000 7f684ffff700 1 --2- 172.21.15.27:0/51464825 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6844064100 0x7f68440665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:50:15.704 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.697+0000 7f684ffff700 1 -- 172.21.15.27:0/51464825 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f6850082600 con 0x7f68580f6530 2021-11-03T18:50:15.704 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.697+0000 7f6856ffd700 1 --2- 172.21.15.27:0/51464825 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6844064100 0x7f68440665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:50:15.704 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.698+0000 7f6856ffd700 1 --2- 172.21.15.27:0/51464825 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6844064100 0x7f68440665c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 rx=0x7f6840009880 tx=0x7f684000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:50:15.704 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.701+0000 7f684ffff700 1 -- 172.21.15.27:0/51464825 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f6850054960 con 0x7f68580f6530 2021-11-03T18:50:15.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5657.496226 secs 2021-11-03T18:50:15.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:15 smithi115 ceph-mon[65453]: from='client.37048 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:50:15.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5657.496226 secs 2021-11-03T18:50:15.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:15 smithi027 ceph-mon[97244]: from='client.37048 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:50:15.904 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.905+0000 7f685c89a700 1 -- 172.21.15.27:0/51464825 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f68580fed90 con 0x7f68580f6530 2021-11-03T18:50:15.905 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.905+0000 7f684ffff700 1 -- 172.21.15.27:0/51464825 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f6850054550 con 0x7f68580f6530 2021-11-03T18:50:15.905 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:50:15.906 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:50:15.906 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:50:15.906 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:50:15.906 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:50:15.907 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:50:15.907 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:50:15.907 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:50:15.907 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:50:15.907 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:50:15.908 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:50:15.908 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:50:15.908 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:50:15.908 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:50:15.909 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:50:15.909 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:50:15.909 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:50:15.909 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:50:15.909 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:50:15.910 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:50:15.910 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:50:15.910 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:50:15.910 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:50:15.911 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:50:15.911 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:50:15.911 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:50:15.911 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:50:15.912 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:50:15.912 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:50:15.912 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:50:15.912 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:50:15.912 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:50:15.913 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:50:15.913 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:50:15.913 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:50:15.913 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:50:15.914 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:50:15.914 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:50:15.914 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:50:15.915 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.907+0000 7f685c89a700 1 -- 172.21.15.27:0/51464825 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6844064100 msgr2=0x7f68440665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:50:15.915 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.907+0000 7f685c89a700 1 --2- 172.21.15.27:0/51464825 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6844064100 0x7f68440665c0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 rx=0x7f6840009880 tx=0x7f684000b040).stop 2021-11-03T18:50:15.915 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.907+0000 7f685c89a700 1 -- 172.21.15.27:0/51464825 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68580f6530 msgr2=0x7f6858106d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:50:15.916 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.907+0000 7f685c89a700 1 --2- 172.21.15.27:0/51464825 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68580f6530 0x7f6858106d70 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 rx=0x7f6850009830 tx=0x7f6850009b30).stop 2021-11-03T18:50:15.916 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.907+0000 7f685c89a700 1 -- 172.21.15.27:0/51464825 shutdown_connections 2021-11-03T18:50:15.916 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.907+0000 7f685c89a700 1 --2- 172.21.15.27:0/51464825 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6844064100 0x7f68440665c0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:15.916 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.907+0000 7f685c89a700 1 --2- 172.21.15.27:0/51464825 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f68580f6530 0x7f6858106d70 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:15.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.907+0000 7f685c89a700 1 --2- 172.21.15.27:0/51464825 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f68580f77d0 0x7f6858107260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:15.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.907+0000 7f685c89a700 1 -- 172.21.15.27:0/51464825 >> 172.21.15.27:0/51464825 conn(0x7f68580f1cd0 msgr2=0x7f68580fb220 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:50:15.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.907+0000 7f685c89a700 1 -- 172.21.15.27:0/51464825 shutdown_connections 2021-11-03T18:50:15.917 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:15.907+0000 7f685c89a700 1 -- 172.21.15.27:0/51464825 wait complete. 2021-11-03T18:50:15.918 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:50:16.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:16 smithi115 ceph-mon[65453]: from='client.37052 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:50:16.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:16 smithi115 ceph-mon[65453]: pgmap v2870: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:16.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:16 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/987061768' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:16.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:16 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/51464825' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:50:16.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:50:16.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:50:16.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:50:16.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:16.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:16.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:16.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:16 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:16.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:16 smithi027 ceph-mon[97244]: from='client.37052 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:50:16.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:16 smithi027 ceph-mon[97244]: pgmap v2870: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:16.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:16 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/987061768' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:16.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:16 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/51464825' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:50:16.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:50:16.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:50:16.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:50:16.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:16.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:16.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:16.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:16 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:17.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:17 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:50:17.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:17 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:50:18.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:18 smithi115 ceph-mon[65453]: pgmap v2871: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:18.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:18 smithi027 ceph-mon[97244]: pgmap v2871: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:20.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:20 smithi115 ceph-mon[65453]: pgmap v2872: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:20.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5662.496347 secs 2021-11-03T18:50:20.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:20 smithi027 ceph-mon[97244]: pgmap v2872: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:20.772 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5662.496347 secs 2021-11-03T18:50:22.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:22 smithi115 ceph-mon[65453]: pgmap v2873: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:23.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:22 smithi027 ceph-mon[97244]: pgmap v2873: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:24.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:24 smithi115 ceph-mon[65453]: pgmap v2874: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:25.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:24 smithi027 ceph-mon[97244]: pgmap v2874: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:25.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:25 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5667.496487 secs 2021-11-03T18:50:26.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:25 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5667.496487 secs 2021-11-03T18:50:26.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:26 smithi115 ceph-mon[65453]: pgmap v2875: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:26.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:26 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:50:27.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:26 smithi027 ceph-mon[97244]: pgmap v2875: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:27.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:26 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:50:27.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:27 smithi115 ceph-mon[65453]: poll took 20055.008 ms: 1 events 2021-11-03T18:50:28.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:27 smithi027 ceph-mon[97244]: poll took 20055.008 ms: 1 events 2021-11-03T18:50:28.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:28 smithi115 ceph-mon[65453]: pgmap v2876: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:29.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:28 smithi027 ceph-mon[97244]: pgmap v2876: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:30.737 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:30 smithi115 ceph-mon[65453]: poll took 2902.311 ms: 1 events 2021-11-03T18:50:30.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:50:30.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:50:30.738 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:50:30.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:30.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:30.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:30.739 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:30 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:30.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:30 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:50:30.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:30 smithi115 ceph-mon[65453]: pgmap v2877: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:30.740 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:30 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5672.496635 secs 2021-11-03T18:50:30.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:30 smithi027 ceph-mon[97244]: poll took 2902.311 ms: 1 events 2021-11-03T18:50:30.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:50:30.773 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:50:30.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:50:30.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:30.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:30.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:30.774 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:30 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:30.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:30 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:50:30.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:30 smithi027 ceph-mon[97244]: pgmap v2877: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:30.775 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:30 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5672.496635 secs 2021-11-03T18:50:32.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:32 smithi115 ceph-mon[65453]: pgmap v2878: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:33.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:32 smithi027 ceph-mon[97244]: pgmap v2878: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:34.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:34 smithi115 ceph-mon[65453]: pgmap v2879: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:35.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:34 smithi027 ceph-mon[97244]: pgmap v2879: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:35.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:35 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5677.496761 secs 2021-11-03T18:50:36.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:35 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5677.496761 secs 2021-11-03T18:50:36.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:36 smithi115 ceph-mon[65453]: pgmap v2880: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:37.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:36 smithi027 ceph-mon[97244]: pgmap v2880: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:38.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:38 smithi115 ceph-mon[65453]: pgmap v2881: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:39.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:38 smithi027 ceph-mon[97244]: pgmap v2881: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:39.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:50:39.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:50:39.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:50:39.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:39.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:39.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:39.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:39 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:40.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:50:40.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:50:40.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:50:40.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:40.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:40.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:40.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:39 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:40.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:40 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:50:40.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:40 smithi115 ceph-mon[65453]: pgmap v2882: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:40.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:40 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5682.496889 secs 2021-11-03T18:50:41.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:40 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:50:41.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:40 smithi027 ceph-mon[97244]: pgmap v2882: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:41.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:40 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5682.496889 secs 2021-11-03T18:50:42.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:42 smithi115 ceph-mon[65453]: pgmap v2883: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:43.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:42 smithi027 ceph-mon[97244]: pgmap v2883: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:44.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:44 smithi115 ceph-mon[65453]: pgmap v2884: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:45.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:44 smithi027 ceph-mon[97244]: pgmap v2884: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:45.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:45 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5687.497020 secs 2021-11-03T18:50:46.022 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.023+0000 7f092fe58700 1 -- 172.21.15.27:0/1887824217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09280f55e0 msgr2=0x7f09280f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:50:46.023 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.023+0000 7f092fe58700 1 --2- 172.21.15.27:0/1887824217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09280f55e0 0x7f09280f5a00 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 rx=0x7f0920004660 tx=0x7f0920009b30).stop 2021-11-03T18:50:46.023 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.023+0000 7f092fe58700 1 -- 172.21.15.27:0/1887824217 shutdown_connections 2021-11-03T18:50:46.024 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.023+0000 7f092fe58700 1 --2- 172.21.15.27:0/1887824217 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09280f5ef0 0x7f09280f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:46.024 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.023+0000 7f092fe58700 1 --2- 172.21.15.27:0/1887824217 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09280f55e0 0x7f09280f5a00 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:46.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.023+0000 7f092fe58700 1 -- 172.21.15.27:0/1887824217 >> 172.21.15.27:0/1887824217 conn(0x7f09280f0de0 msgr2=0x7f09280f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:50:46.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.023+0000 7f092fe58700 1 -- 172.21.15.27:0/1887824217 shutdown_connections 2021-11-03T18:50:46.025 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.023+0000 7f092fe58700 1 -- 172.21.15.27:0/1887824217 wait complete. 2021-11-03T18:50:46.026 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:45 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5687.497020 secs 2021-11-03T18:50:46.030 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.024+0000 7f092fe58700 1 Processor -- start 2021-11-03T18:50:46.030 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.024+0000 7f092fe58700 1 -- start start 2021-11-03T18:50:46.031 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.024+0000 7f092fe58700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09280f55e0 0x7f0928105e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:50:46.031 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.024+0000 7f092fe58700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09280f5ef0 0x7f0928106380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:50:46.031 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.024+0000 7f092fe58700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0928106930 con 0x7f09280f55e0 2021-11-03T18:50:46.031 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.024+0000 7f092fe58700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0928106a70 con 0x7f09280f5ef0 2021-11-03T18:50:46.032 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.024+0000 7f092dbf4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09280f55e0 0x7f0928105e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:50:46.032 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.024+0000 7f092dbf4700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09280f55e0 0x7f0928105e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37162/0 (socket says 172.21.15.27:37162) 2021-11-03T18:50:46.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.024+0000 7f092dbf4700 1 -- 172.21.15.27:0/430010822 learned_addr learned my addr 172.21.15.27:0/430010822 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:50:46.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.024+0000 7f092d3f3700 1 --2- 172.21.15.27:0/430010822 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09280f5ef0 0x7f0928106380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:50:46.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.025+0000 7f092dbf4700 1 -- 172.21.15.27:0/430010822 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09280f5ef0 msgr2=0x7f0928106380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:50:46.033 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.025+0000 7f092dbf4700 1 --2- 172.21.15.27:0/430010822 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09280f5ef0 0x7f0928106380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:46.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.025+0000 7f092dbf4700 1 -- 172.21.15.27:0/430010822 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0920005040 con 0x7f09280f55e0 2021-11-03T18:50:46.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.025+0000 7f092d3f3700 1 --2- 172.21.15.27:0/430010822 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09280f5ef0 0x7f0928106380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2021-11-03T18:50:46.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.025+0000 7f092dbf4700 1 --2- 172.21.15.27:0/430010822 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09280f55e0 0x7f0928105e90 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 rx=0x7f0920009990 tx=0x7f0920009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:50:46.034 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.025+0000 7f09267fc700 1 -- 172.21.15.27:0/430010822 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0920018070 con 0x7f09280f55e0 2021-11-03T18:50:46.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.025+0000 7f092fe58700 1 -- 172.21.15.27:0/430010822 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09281073b0 con 0x7f09280f55e0 2021-11-03T18:50:46.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.025+0000 7f09267fc700 1 -- 172.21.15.27:0/430010822 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f0920007ca0 con 0x7f09280f55e0 2021-11-03T18:50:46.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.025+0000 7f09267fc700 1 -- 172.21.15.27:0/430010822 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f092001c620 con 0x7f09280f55e0 2021-11-03T18:50:46.035 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.025+0000 7f092fe58700 1 -- 172.21.15.27:0/430010822 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0928107910 con 0x7f09280f55e0 2021-11-03T18:50:46.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.026+0000 7f092fe58700 1 -- 172.21.15.27:0/430010822 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0928045ad0 con 0x7f09280f55e0 2021-11-03T18:50:46.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.026+0000 7f09267fc700 1 -- 172.21.15.27:0/430010822 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f092001d960 con 0x7f09280f55e0 2021-11-03T18:50:46.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.027+0000 7f09267fc700 1 --2- 172.21.15.27:0/430010822 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f09100640b0 0x7f0910066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:50:46.036 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.027+0000 7f09267fc700 1 -- 172.21.15.27:0/430010822 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f0920082540 con 0x7f09280f55e0 2021-11-03T18:50:46.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.027+0000 7f092d3f3700 1 --2- 172.21.15.27:0/430010822 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f09100640b0 0x7f0910066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:50:46.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.027+0000 7f092d3f3700 1 --2- 172.21.15.27:0/430010822 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f09100640b0 0x7f0910066570 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 rx=0x7f091c0041d0 tx=0x7f091c009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:50:46.037 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.031+0000 7f09267fc700 1 -- 172.21.15.27:0/430010822 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f0920054960 con 0x7f09280f55e0 2021-11-03T18:50:46.217 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.219+0000 7f092fe58700 1 -- 172.21.15.27:0/430010822 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f09280fdf80 con 0x7f09100640b0 2021-11-03T18:50:46.218 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.220+0000 7f09267fc700 1 -- 172.21.15.27:0/430010822 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f091c004600 con 0x7f09100640b0 2021-11-03T18:50:46.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.222+0000 7f092fe58700 1 -- 172.21.15.27:0/430010822 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f09100640b0 msgr2=0x7f0910066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:50:46.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.222+0000 7f092fe58700 1 --2- 172.21.15.27:0/430010822 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f09100640b0 0x7f0910066570 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 rx=0x7f091c0041d0 tx=0x7f091c009040).stop 2021-11-03T18:50:46.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.222+0000 7f092fe58700 1 -- 172.21.15.27:0/430010822 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09280f55e0 msgr2=0x7f0928105e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:50:46.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.222+0000 7f092fe58700 1 --2- 172.21.15.27:0/430010822 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09280f55e0 0x7f0928105e90 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 rx=0x7f0920009990 tx=0x7f0920009b30).stop 2021-11-03T18:50:46.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.222+0000 7f092fe58700 1 -- 172.21.15.27:0/430010822 shutdown_connections 2021-11-03T18:50:46.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.222+0000 7f092fe58700 1 --2- 172.21.15.27:0/430010822 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f09100640b0 0x7f0910066570 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:46.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.222+0000 7f092fe58700 1 --2- 172.21.15.27:0/430010822 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f09280f55e0 0x7f0928105e90 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:46.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.222+0000 7f092fe58700 1 --2- 172.21.15.27:0/430010822 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f09280f5ef0 0x7f0928106380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:46.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.222+0000 7f092fe58700 1 -- 172.21.15.27:0/430010822 >> 172.21.15.27:0/430010822 conn(0x7f09280f0de0 msgr2=0x7f09280fa370 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:50:46.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.222+0000 7f092fe58700 1 -- 172.21.15.27:0/430010822 shutdown_connections 2021-11-03T18:50:46.223 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.222+0000 7f092fe58700 1 -- 172.21.15.27:0/430010822 wait complete. 2021-11-03T18:50:46.230 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:50:46.321 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.322+0000 7f67352fc700 1 -- 172.21.15.27:0/794929476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67300f66e0 msgr2=0x7f67300f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:50:46.321 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.322+0000 7f67352fc700 1 --2- 172.21.15.27:0/794929476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67300f66e0 0x7f67300f6b40 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 rx=0x7f6720004660 tx=0x7f6720009b30).stop 2021-11-03T18:50:46.321 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.322+0000 7f67352fc700 1 -- 172.21.15.27:0/794929476 shutdown_connections 2021-11-03T18:50:46.322 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.322+0000 7f67352fc700 1 --2- 172.21.15.27:0/794929476 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67300f66e0 0x7f67300f6b40 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:46.322 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.322+0000 7f67352fc700 1 --2- 172.21.15.27:0/794929476 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67300f5440 0x7f67300f5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:46.322 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.322+0000 7f67352fc700 1 -- 172.21.15.27:0/794929476 >> 172.21.15.27:0/794929476 conn(0x7f67300f0ba0 msgr2=0x7f67300f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:50:46.322 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.322+0000 7f67352fc700 1 -- 172.21.15.27:0/794929476 shutdown_connections 2021-11-03T18:50:46.323 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.322+0000 7f67352fc700 1 -- 172.21.15.27:0/794929476 wait complete. 2021-11-03T18:50:46.325 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.323+0000 7f67352fc700 1 Processor -- start 2021-11-03T18:50:46.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.323+0000 7f67352fc700 1 -- start start 2021-11-03T18:50:46.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.323+0000 7f67352fc700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67300f5440 0x7f6730106e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:50:46.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.323+0000 7f67352fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67300f66e0 0x7f67301072f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:50:46.326 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.323+0000 7f67352fc700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67301078f0 con 0x7f67300f66e0 2021-11-03T18:50:46.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.323+0000 7f67352fc700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6730108100 con 0x7f67300f5440 2021-11-03T18:50:46.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.323+0000 7f672f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67300f66e0 0x7f67301072f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:50:46.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.323+0000 7f672f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67300f66e0 0x7f67301072f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37174/0 (socket says 172.21.15.27:37174) 2021-11-03T18:50:46.327 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.323+0000 7f672f7fe700 1 -- 172.21.15.27:0/171162211 learned_addr learned my addr 172.21.15.27:0/171162211 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:50:46.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.324+0000 7f672ffff700 1 --2- 172.21.15.27:0/171162211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67300f5440 0x7f6730106e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:50:46.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.324+0000 7f672f7fe700 1 -- 172.21.15.27:0/171162211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67300f5440 msgr2=0x7f6730106e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:50:46.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.324+0000 7f672f7fe700 1 --2- 172.21.15.27:0/171162211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67300f5440 0x7f6730106e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:46.328 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.324+0000 7f672f7fe700 1 -- 172.21.15.27:0/171162211 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6720005040 con 0x7f67300f66e0 2021-11-03T18:50:46.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.324+0000 7f672f7fe700 1 --2- 172.21.15.27:0/171162211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67300f66e0 0x7f67301072f0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 rx=0x7f6720009990 tx=0x7f6720009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:50:46.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.324+0000 7f672cff9700 1 -- 172.21.15.27:0/171162211 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6720018070 con 0x7f67300f66e0 2021-11-03T18:50:46.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.324+0000 7f672cff9700 1 -- 172.21.15.27:0/171162211 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f6720007ca0 con 0x7f67300f66e0 2021-11-03T18:50:46.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.324+0000 7f67352fc700 1 -- 172.21.15.27:0/171162211 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6730108320 con 0x7f67300f66e0 2021-11-03T18:50:46.329 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.324+0000 7f672cff9700 1 -- 172.21.15.27:0/171162211 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f672001c620 con 0x7f67300f66e0 2021-11-03T18:50:46.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.324+0000 7f67352fc700 1 -- 172.21.15.27:0/171162211 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6730108880 con 0x7f67300f66e0 2021-11-03T18:50:46.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.325+0000 7f67352fc700 1 -- 172.21.15.27:0/171162211 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6730059100 con 0x7f67300f66e0 2021-11-03T18:50:46.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.325+0000 7f672cff9700 1 -- 172.21.15.27:0/171162211 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f672001d960 con 0x7f67300f66e0 2021-11-03T18:50:46.330 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.326+0000 7f672cff9700 1 --2- 172.21.15.27:0/171162211 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6718064100 0x7f67180665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:50:46.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.326+0000 7f672cff9700 1 -- 172.21.15.27:0/171162211 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f6720082480 con 0x7f67300f66e0 2021-11-03T18:50:46.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.326+0000 7f672ffff700 1 --2- 172.21.15.27:0/171162211 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6718064100 0x7f67180665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:50:46.331 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.326+0000 7f672ffff700 1 --2- 172.21.15.27:0/171162211 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6718064100 0x7f67180665c0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 rx=0x7f6730106b70 tx=0x7f6724009040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:50:46.332 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.330+0000 7f672cff9700 1 -- 172.21.15.27:0/171162211 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f67200548a0 con 0x7f67300f66e0 2021-11-03T18:50:46.499 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.500+0000 7f67352fc700 1 -- 172.21.15.27:0/171162211 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f67300fede0 con 0x7f6718064100 2021-11-03T18:50:46.513 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.515+0000 7f672cff9700 1 -- 172.21.15.27:0/171162211 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3542 (secure 0 0 0) 0x7f67300fede0 con 0x7f6718064100 2021-11-03T18:50:46.514 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:50:46.514 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (102m) 40s ago 103m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:50:46.514 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (94m) 40s ago 103m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:50:46.514 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (93m) 17s ago 102m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:50:46.515 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (102m) 40s ago 103m 42.8M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:50:46.515 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (95m) 40s ago 98m 27.1M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:50:46.515 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (98m) 40s ago 98m 3853M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:50:46.515 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (94m) 17s ago 98m 26.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:50:46.516 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (96m) 17s ago 98m 27.8M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:50:46.516 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (96m) 40s ago 104m 644M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:50:46.516 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (95m) 17s ago 102m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:50:46.516 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (95m) 40s ago 105m 424M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:50:46.517 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (94m) 17s ago 102m 191M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:50:46.517 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (103m) 40s ago 103m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:50:46.517 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (102m) 17s ago 102m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:50:46.517 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (93m) 40s ago 102m 453M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:50:46.517 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (93m) 40s ago 101m 493M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:50:46.518 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (92m) 40s ago 101m 374M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:50:46.518 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (92m) 17s ago 100m 529M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:50:46.518 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (91m) 17s ago 100m 415M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:50:46.518 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (91m) 17s ago 100m 428M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:50:46.519 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (102m) 40s ago 102m 69.1M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:50:46.519 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.517+0000 7f67352fc700 1 -- 172.21.15.27:0/171162211 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6718064100 msgr2=0x7f67180665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:50:46.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.517+0000 7f67352fc700 1 --2- 172.21.15.27:0/171162211 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6718064100 0x7f67180665c0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 rx=0x7f6730106b70 tx=0x7f6724009040).stop 2021-11-03T18:50:46.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.517+0000 7f67352fc700 1 -- 172.21.15.27:0/171162211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67300f66e0 msgr2=0x7f67301072f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:50:46.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.517+0000 7f67352fc700 1 --2- 172.21.15.27:0/171162211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67300f66e0 0x7f67301072f0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 rx=0x7f6720009990 tx=0x7f6720009b30).stop 2021-11-03T18:50:46.520 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.517+0000 7f67352fc700 1 -- 172.21.15.27:0/171162211 shutdown_connections 2021-11-03T18:50:46.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.517+0000 7f67352fc700 1 --2- 172.21.15.27:0/171162211 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6718064100 0x7f67180665c0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:46.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.517+0000 7f67352fc700 1 --2- 172.21.15.27:0/171162211 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f67300f5440 0x7f6730106e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:46.521 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.517+0000 7f67352fc700 1 --2- 172.21.15.27:0/171162211 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f67300f66e0 0x7f67301072f0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:46.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.517+0000 7f67352fc700 1 -- 172.21.15.27:0/171162211 >> 172.21.15.27:0/171162211 conn(0x7f67300f0ba0 msgr2=0x7f67300fa130 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:50:46.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.517+0000 7f67352fc700 1 -- 172.21.15.27:0/171162211 shutdown_connections 2021-11-03T18:50:46.522 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.517+0000 7f67352fc700 1 -- 172.21.15.27:0/171162211 wait complete. 2021-11-03T18:50:46.615 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.616+0000 7f6d4d9cb700 1 -- 172.21.15.27:0/3501462069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d480f55e0 msgr2=0x7f6d480f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:50:46.615 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.616+0000 7f6d4d9cb700 1 --2- 172.21.15.27:0/3501462069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d480f55e0 0x7f6d480f5a00 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 rx=0x7f6d40004660 tx=0x7f6d40009b30).stop 2021-11-03T18:50:46.615 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.616+0000 7f6d4d9cb700 1 -- 172.21.15.27:0/3501462069 shutdown_connections 2021-11-03T18:50:46.615 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.616+0000 7f6d4d9cb700 1 --2- 172.21.15.27:0/3501462069 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d480f6880 0x7f6d480f6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:46.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.616+0000 7f6d4d9cb700 1 --2- 172.21.15.27:0/3501462069 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d480f55e0 0x7f6d480f5a00 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:46.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.616+0000 7f6d4d9cb700 1 -- 172.21.15.27:0/3501462069 >> 172.21.15.27:0/3501462069 conn(0x7f6d480f0d40 msgr2=0x7f6d480f3160 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:50:46.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.616+0000 7f6d4d9cb700 1 -- 172.21.15.27:0/3501462069 shutdown_connections 2021-11-03T18:50:46.616 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.616+0000 7f6d4d9cb700 1 -- 172.21.15.27:0/3501462069 wait complete. 2021-11-03T18:50:46.619 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.617+0000 7f6d4d9cb700 1 Processor -- start 2021-11-03T18:50:46.619 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.617+0000 7f6d4d9cb700 1 -- start start 2021-11-03T18:50:46.619 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.617+0000 7f6d4d9cb700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d480f55e0 0x7f6d48109cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:50:46.620 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.617+0000 7f6d4d9cb700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d480f6880 0x7f6d4810c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:50:46.620 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.617+0000 7f6d4d9cb700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d480604e0 con 0x7f6d480f6880 2021-11-03T18:50:46.620 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.617+0000 7f6d4d9cb700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d48060620 con 0x7f6d480f55e0 2021-11-03T18:50:46.620 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.617+0000 7f6d467fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d480f6880 0x7f6d4810c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:50:46.621 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.617+0000 7f6d467fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d480f6880 0x7f6d4810c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37184/0 (socket says 172.21.15.27:37184) 2021-11-03T18:50:46.621 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.617+0000 7f6d467fc700 1 -- 172.21.15.27:0/1526318669 learned_addr learned my addr 172.21.15.27:0/1526318669 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:50:46.621 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.617+0000 7f6d46ffd700 1 --2- 172.21.15.27:0/1526318669 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d480f55e0 0x7f6d48109cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:50:46.621 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.617+0000 7f6d467fc700 1 -- 172.21.15.27:0/1526318669 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d480f55e0 msgr2=0x7f6d48109cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:50:46.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.617+0000 7f6d467fc700 1 --2- 172.21.15.27:0/1526318669 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d480f55e0 0x7f6d48109cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:46.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.617+0000 7f6d467fc700 1 -- 172.21.15.27:0/1526318669 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d40005040 con 0x7f6d480f6880 2021-11-03T18:50:46.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.617+0000 7f6d467fc700 1 --2- 172.21.15.27:0/1526318669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d480f6880 0x7f6d4810c1d0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 rx=0x7f6d300026d0 tx=0x7f6d30009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:50:46.622 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.618+0000 7f6d3f7fe700 1 -- 172.21.15.27:0/1526318669 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d300079c0 con 0x7f6d480f6880 2021-11-03T18:50:46.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.618+0000 7f6d3f7fe700 1 -- 172.21.15.27:0/1526318669 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f6d3000f040 con 0x7f6d480f6880 2021-11-03T18:50:46.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.618+0000 7f6d4d9cb700 1 -- 172.21.15.27:0/1526318669 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d4810c780 con 0x7f6d480f6880 2021-11-03T18:50:46.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.618+0000 7f6d3f7fe700 1 -- 172.21.15.27:0/1526318669 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d300148b0 con 0x7f6d480f6880 2021-11-03T18:50:46.623 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.618+0000 7f6d4d9cb700 1 -- 172.21.15.27:0/1526318669 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d4810cd10 con 0x7f6d480f6880 2021-11-03T18:50:46.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.619+0000 7f6d4d9cb700 1 -- 172.21.15.27:0/1526318669 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d480f49d0 con 0x7f6d480f6880 2021-11-03T18:50:46.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.619+0000 7f6d3f7fe700 1 -- 172.21.15.27:0/1526318669 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f6d30007b20 con 0x7f6d480f6880 2021-11-03T18:50:46.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.619+0000 7f6d3f7fe700 1 --2- 172.21.15.27:0/1526318669 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6d34064090 0x7f6d34066550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:50:46.624 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.619+0000 7f6d3f7fe700 1 -- 172.21.15.27:0/1526318669 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f6d30080670 con 0x7f6d480f6880 2021-11-03T18:50:46.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.620+0000 7f6d46ffd700 1 --2- 172.21.15.27:0/1526318669 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6d34064090 0x7f6d34066550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:50:46.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.620+0000 7f6d46ffd700 1 --2- 172.21.15.27:0/1526318669 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6d34064090 0x7f6d34066550 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 rx=0x7f6d48060160 tx=0x7f6d40009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:50:46.625 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.623+0000 7f6d3f7fe700 1 -- 172.21.15.27:0/1526318669 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f6d300529d0 con 0x7f6d480f6880 2021-11-03T18:50:46.856 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.858+0000 7f6d4d9cb700 1 -- 172.21.15.27:0/1526318669 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f6d48045ad0 con 0x7f6d480f6880 2021-11-03T18:50:46.857 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.858+0000 7f6d3f7fe700 1 -- 172.21.15.27:0/1526318669 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f6d300525c0 con 0x7f6d480f6880 2021-11-03T18:50:46.858 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:50:46.858 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:50:46.858 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:50:46.858 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:50:46.859 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:50:46.859 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:50:46.859 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:50:46.859 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:50:46.860 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:50:46.860 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:50:46.860 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:50:46.860 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:50:46.861 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:50:46.861 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:50:46.861 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:50:46.861 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:50:46.861 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:50:46.862 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:50:46.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.860+0000 7f6d4d9cb700 1 -- 172.21.15.27:0/1526318669 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6d34064090 msgr2=0x7f6d34066550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:50:46.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.860+0000 7f6d4d9cb700 1 --2- 172.21.15.27:0/1526318669 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6d34064090 0x7f6d34066550 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 rx=0x7f6d48060160 tx=0x7f6d40009b30).stop 2021-11-03T18:50:46.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.860+0000 7f6d4d9cb700 1 -- 172.21.15.27:0/1526318669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d480f6880 msgr2=0x7f6d4810c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:50:46.863 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.860+0000 7f6d4d9cb700 1 --2- 172.21.15.27:0/1526318669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d480f6880 0x7f6d4810c1d0 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 rx=0x7f6d300026d0 tx=0x7f6d30009de0).stop 2021-11-03T18:50:46.864 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.860+0000 7f6d4d9cb700 1 -- 172.21.15.27:0/1526318669 shutdown_connections 2021-11-03T18:50:46.864 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.860+0000 7f6d4d9cb700 1 --2- 172.21.15.27:0/1526318669 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f6d34064090 0x7f6d34066550 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:46.864 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.860+0000 7f6d4d9cb700 1 --2- 172.21.15.27:0/1526318669 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f6d480f55e0 0x7f6d48109cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:46.864 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.860+0000 7f6d4d9cb700 1 --2- 172.21.15.27:0/1526318669 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f6d480f6880 0x7f6d4810c1d0 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:46.865 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.860+0000 7f6d4d9cb700 1 -- 172.21.15.27:0/1526318669 >> 172.21.15.27:0/1526318669 conn(0x7f6d480f0d40 msgr2=0x7f6d480fa2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:50:46.865 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.860+0000 7f6d4d9cb700 1 -- 172.21.15.27:0/1526318669 shutdown_connections 2021-11-03T18:50:46.865 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.860+0000 7f6d4d9cb700 1 -- 172.21.15.27:0/1526318669 wait complete. 2021-11-03T18:50:46.959 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.960+0000 7fbd0d560700 1 -- 172.21.15.27:0/2115943131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd080f6700 msgr2=0x7fbd080f6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:50:46.959 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.960+0000 7fbd0d560700 1 --2- 172.21.15.27:0/2115943131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd080f6700 0x7fbd080f6b60 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 rx=0x7fbcfc004660 tx=0x7fbcfc009b30).stop 2021-11-03T18:50:46.959 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.960+0000 7fbd0d560700 1 -- 172.21.15.27:0/2115943131 shutdown_connections 2021-11-03T18:50:46.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.960+0000 7fbd0d560700 1 --2- 172.21.15.27:0/2115943131 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd080f6700 0x7fbd080f6b60 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:46.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.960+0000 7fbd0d560700 1 --2- 172.21.15.27:0/2115943131 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd080f5460 0x7fbd080f5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:46.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.960+0000 7fbd0d560700 1 -- 172.21.15.27:0/2115943131 >> 172.21.15.27:0/2115943131 conn(0x7fbd080f0bc0 msgr2=0x7fbd080f2fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:50:46.960 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.960+0000 7fbd0d560700 1 -- 172.21.15.27:0/2115943131 shutdown_connections 2021-11-03T18:50:46.961 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.960+0000 7fbd0d560700 1 -- 172.21.15.27:0/2115943131 wait complete. 2021-11-03T18:50:46.963 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.961+0000 7fbd0d560700 1 Processor -- start 2021-11-03T18:50:46.963 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.961+0000 7fbd0d560700 1 -- start start 2021-11-03T18:50:46.963 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.961+0000 7fbd0d560700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd080f5460 0x7fbd08105d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:50:46.964 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.961+0000 7fbd0d560700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd080f6700 0x7fbd08106240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:50:46.964 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.961+0000 7fbd0d560700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd081067f0 con 0x7fbd080f6700 2021-11-03T18:50:46.964 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.961+0000 7fbd0d560700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd08106ff0 con 0x7fbd080f5460 2021-11-03T18:50:46.964 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.961+0000 7fbd067fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd080f6700 0x7fbd08106240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:50:46.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.961+0000 7fbd067fc700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd080f6700 0x7fbd08106240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37194/0 (socket says 172.21.15.27:37194) 2021-11-03T18:50:46.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.961+0000 7fbd067fc700 1 -- 172.21.15.27:0/4153155765 learned_addr learned my addr 172.21.15.27:0/4153155765 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:50:46.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.962+0000 7fbd067fc700 1 -- 172.21.15.27:0/4153155765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd080f5460 msgr2=0x7fbd08105d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:50:46.965 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.962+0000 7fbd067fc700 1 --2- 172.21.15.27:0/4153155765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd080f5460 0x7fbd08105d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:50:46.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.962+0000 7fbd067fc700 1 -- 172.21.15.27:0/4153155765 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcfc005040 con 0x7fbd080f6700 2021-11-03T18:50:46.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.962+0000 7fbd067fc700 1 --2- 172.21.15.27:0/4153155765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd080f6700 0x7fbd08106240 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 rx=0x7fbcfc009da0 tx=0x7fbcfc009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:50:46.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.962+0000 7fbcef7fe700 1 -- 172.21.15.27:0/4153155765 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcfc018070 con 0x7fbd080f6700 2021-11-03T18:50:46.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.962+0000 7fbcef7fe700 1 -- 172.21.15.27:0/4153155765 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fbcfc007d70 con 0x7fbd080f6700 2021-11-03T18:50:46.966 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.962+0000 7fbd0d560700 1 -- 172.21.15.27:0/4153155765 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd08107270 con 0x7fbd080f6700 2021-11-03T18:50:46.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.962+0000 7fbcef7fe700 1 -- 172.21.15.27:0/4153155765 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcfc01dea0 con 0x7fbd080f6700 2021-11-03T18:50:46.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.962+0000 7fbd0d560700 1 -- 172.21.15.27:0/4153155765 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd081077d0 con 0x7fbd080f6700 2021-11-03T18:50:46.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.963+0000 7fbd0d560700 1 -- 172.21.15.27:0/4153155765 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd08059100 con 0x7fbd080f6700 2021-11-03T18:50:46.967 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.963+0000 7fbcef7fe700 1 -- 172.21.15.27:0/4153155765 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fbcfc01d420 con 0x7fbd080f6700 2021-11-03T18:50:46.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.964+0000 7fbcef7fe700 1 --2- 172.21.15.27:0/4153155765 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbcf0064100 0x7fbcf00665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:50:46.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.964+0000 7fbcef7fe700 1 -- 172.21.15.27:0/4153155765 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fbcfc0826e0 con 0x7fbd080f6700 2021-11-03T18:50:46.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.964+0000 7fbd06ffd700 1 --2- 172.21.15.27:0/4153155765 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbcf0064100 0x7fbcf00665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:50:46.968 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.964+0000 7fbd06ffd700 1 --2- 172.21.15.27:0/4153155765 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbcf0064100 0x7fbcf00665c0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 rx=0x7fbd080f6560 tx=0x7fbcf8007a60).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:50:46.969 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:46.968+0000 7fbcef7fe700 1 -- 172.21.15.27:0/4153155765 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fbcfc054b00 con 0x7fbd080f6700 2021-11-03T18:50:46.987 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:46 smithi115 ceph-mon[65453]: pgmap v2885: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:47.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:46 smithi027 ceph-mon[97244]: pgmap v2885: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:47.172 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:47.173+0000 7fbd0d560700 1 -- 172.21.15.27:0/4153155765 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fbd08027a00 con 0x7fbd080f6700 2021-11-03T18:50:47.172 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:47.174+0000 7fbcef7fe700 1 -- 172.21.15.27:0/4153155765 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7fbcfc0546f0 con 0x7fbd080f6700 2021-11-03T18:50:47.173 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:50:47.173 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:50:47.173 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:50:47.174 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:50:47.174 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:50:47.174 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:50:47.174 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:50:47.175 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:50:47.175 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:50:47.175 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:50:47.175 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:50:47.175 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:50:47.176 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:50:47.176 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:50:47.176 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:50:47.176 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:50:47.177 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:50:47.177 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:50:47.177 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:50:47.177 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:50:47.177 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:50:47.178 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:50:47.178 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:50:47.178 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:50:47.178 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:50:47.179 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:50:47.179 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:50:47.179 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:50:47.179 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:50:47.179 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:50:47.180 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:50:47.180 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:50:47.180 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:50:47.180 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:50:47.180 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:50:47.181 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:50:47.181 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:50:47.181 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:50:47.181 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:50:47.182 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:47.176+0000 7fbd0d560700 1 -- 172.21.15.27:0/4153155765 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbcf0064100 msgr2=0x7fbcf00665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:50:47.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:47.176+0000 7fbd0d560700 1 --2- 172.21.15.27:0/4153155765 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbcf0064100 0x7fbcf00665c0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 rx=0x7fbd080f6560 tx=0x7fbcf8007a60).stop 2021-11-03T18:50:47.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:47.176+0000 7fbd0d560700 1 -- 172.21.15.27:0/4153155765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd080f6700 msgr2=0x7fbd08106240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:50:47.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:47.176+0000 7fbd0d560700 1 --2- 172.21.15.27:0/4153155765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd080f6700 0x7fbd08106240 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 rx=0x7fbcfc009da0 tx=0x7fbcfc009b30).stop 2021-11-03T18:50:47.183 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:47.176+0000 7fbd0d560700 1 -- 172.21.15.27:0/4153155765 shutdown_connections 2021-11-03T18:50:47.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:47.176+0000 7fbd0d560700 1 --2- 172.21.15.27:0/4153155765 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fbcf0064100 0x7fbcf00665c0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:47.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:47.176+0000 7fbd0d560700 1 --2- 172.21.15.27:0/4153155765 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fbd080f5460 0x7fbd08105d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2021-11-03T18:50:47.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:47.176+0000 7fbd0d560700 1 --2- 172.21.15.27:0/4153155765 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fbd080f6700 0x7fbd08106240 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:50:47.184 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:47.176+0000 7fbd0d560700 1 -- 172.21.15.27:0/4153155765 >> 172.21.15.27:0/4153155765 conn(0x7fbd080f0bc0 msgr2=0x7fbd080fa150 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:50:47.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:47.176+0000 7fbd0d560700 1 -- 172.21.15.27:0/4153155765 shutdown_connections 2021-11-03T18:50:47.185 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:50:47.176+0000 7fbd0d560700 1 -- 172.21.15.27:0/4153155765 wait complete. 2021-11-03T18:50:47.185 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:50:47.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:47 smithi115 ceph-mon[65453]: from='client.37064 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:50:47.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:47 smithi115 ceph-mon[65453]: from='client.37068 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:50:47.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:47 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1526318669' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:47.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:47 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/4153155765' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:50:48.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:47 smithi027 ceph-mon[97244]: from='client.37064 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:50:48.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:47 smithi027 ceph-mon[97244]: from='client.37068 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:50:48.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:47 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1526318669' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:48.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:47 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/4153155765' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:50:48.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:48 smithi115 ceph-mon[65453]: pgmap v2886: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:49.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:48 smithi027 ceph-mon[97244]: pgmap v2886: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:49.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:50:49.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:50:49.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:50:49.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:49.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:49.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:49.989 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:49 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:50.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:50:50.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:50:50.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:50:50.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:50.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:50.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:50.024 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:49 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:50:50.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:50 smithi115 ceph-mon[65453]: pgmap v2887: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:50.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:50 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:50:50.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:50 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5692.497149 secs 2021-11-03T18:50:51.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:50 smithi027 ceph-mon[97244]: pgmap v2887: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:51.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:50 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:50:51.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:50 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5692.497149 secs 2021-11-03T18:50:52.988 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:52 smithi115 ceph-mon[65453]: pgmap v2888: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:53.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:52 smithi027 ceph-mon[97244]: pgmap v2888: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:55.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:54 smithi027 ceph-mon[97244]: pgmap v2889: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:55.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:54 smithi115 ceph-mon[65453]: pgmap v2889: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:56.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:55 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5697.497269 secs 2021-11-03T18:50:56.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:55 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5697.497269 secs 2021-11-03T18:50:57.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:56 smithi027 ceph-mon[97244]: pgmap v2890: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:57.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:50:57.023 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:56 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:50:57.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:56 smithi115 ceph-mon[65453]: pgmap v2890: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:57.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/trash_purge_schedule"}]: dispatch 2021-11-03T18:50:57.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:56 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi027.ufhhvb/mirror_snapshot_schedule"}]: dispatch 2021-11-03T18:50:59.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:58 smithi027 ceph-mon[97244]: pgmap v2891: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:50:59.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:58 smithi115 ceph-mon[65453]: pgmap v2891: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:51:00.022 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:50:59 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:51:00.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:50:59 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:51:01.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:00 smithi115 ceph-mon[65453]: pgmap v2892: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:51:01.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:00 smithi115 ceph-mon[65453]: poll took 30091.613 ms: 1 events 2021-11-03T18:51:01.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:00 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5702.497364 secs 2021-11-03T18:51:01.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:51:01.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:51:01.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:51:01.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:51:01.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:51:01.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:00 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:51:01.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:00 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:51:01.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:00 smithi027 ceph-mon[97244]: pgmap v2892: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:51:01.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:00 smithi027 ceph-mon[97244]: poll took 30091.613 ms: 1 events 2021-11-03T18:51:01.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:00 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5702.497364 secs 2021-11-03T18:51:01.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:51:01.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:51:01.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:51:01.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:51:01.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:51:01.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:00 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:51:01.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:00 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:51:03.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:02 smithi115 ceph-mon[65453]: pgmap v2893: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:51:03.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:02 smithi027 ceph-mon[97244]: pgmap v2893: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:51:05.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:04 smithi115 ceph-mon[65453]: pgmap v2894: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:51:05.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:04 smithi027 ceph-mon[97244]: pgmap v2894: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:51:06.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:05 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5707.497485 secs 2021-11-03T18:51:06.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:05 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5707.497485 secs 2021-11-03T18:51:07.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:06 smithi115 ceph-mon[65453]: pgmap v2895: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:51:07.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:06 smithi027 ceph-mon[97244]: pgmap v2895: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:51:09.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:08 smithi115 ceph-mon[65453]: pgmap v2896: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:51:09.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:08 smithi027 ceph-mon[97244]: pgmap v2896: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:51:10.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:09 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:51:10.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:09 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:51:11.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:10 smithi115 ceph-mon[65453]: pgmap v2897: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:51:11.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:10 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5712.497637 secs 2021-11-03T18:51:11.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:51:11.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:51:11.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:51:11.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:51:11.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:51:11.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:10 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:51:11.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:10 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:51:11.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:10 smithi027 ceph-mon[97244]: pgmap v2897: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:51:11.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:10 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5712.497637 secs 2021-11-03T18:51:11.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:51:11.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:51:11.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:51:11.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:51:11.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:51:11.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:10 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:51:11.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:10 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:51:13.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:12 smithi115 ceph-mon[65453]: pgmap v2898: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:51:13.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:12 smithi027 ceph-mon[97244]: pgmap v2898: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:51:15.237 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:14 smithi115 ceph-mon[65453]: pgmap v2899: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:51:15.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:14 smithi027 ceph-mon[97244]: pgmap v2899: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:51:16.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:15 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5717.497762 secs 2021-11-03T18:51:16.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:15 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5717.497762 secs 2021-11-03T18:51:17.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:16 smithi115 ceph-mon[65453]: pgmap v2900: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:51:17.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:16 smithi027 ceph-mon[97244]: pgmap v2900: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:51:17.292 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.292+0000 7f8e6bdd3700 1 -- 172.21.15.27:0/1581707939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e640f5ef0 msgr2=0x7f8e640f6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:51:17.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.292+0000 7f8e6bdd3700 1 --2- 172.21.15.27:0/1581707939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e640f5ef0 0x7f8e640f6920 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 rx=0x7f8e54004660 tx=0x7f8e54009b30).stop 2021-11-03T18:51:17.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.292+0000 7f8e6bdd3700 1 -- 172.21.15.27:0/1581707939 shutdown_connections 2021-11-03T18:51:17.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.292+0000 7f8e6bdd3700 1 --2- 172.21.15.27:0/1581707939 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e640f5ef0 0x7f8e640f6920 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:51:17.293 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.292+0000 7f8e6bdd3700 1 --2- 172.21.15.27:0/1581707939 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e640f55e0 0x7f8e640f5a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:51:17.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.292+0000 7f8e6bdd3700 1 -- 172.21.15.27:0/1581707939 >> 172.21.15.27:0/1581707939 conn(0x7f8e640f0de0 msgr2=0x7f8e640f3200 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:51:17.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.292+0000 7f8e6bdd3700 1 -- 172.21.15.27:0/1581707939 shutdown_connections 2021-11-03T18:51:17.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.292+0000 7f8e6bdd3700 1 -- 172.21.15.27:0/1581707939 wait complete. 2021-11-03T18:51:17.294 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.292+0000 7f8e6bdd3700 1 Processor -- start 2021-11-03T18:51:17.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.292+0000 7f8e6bdd3700 1 -- start start 2021-11-03T18:51:17.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.293+0000 7f8e6bdd3700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e640f55e0 0x7f8e64106f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:51:17.295 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.293+0000 7f8e6bdd3700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e640f5ef0 0x7f8e64107460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:51:17.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.293+0000 7f8e6bdd3700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e64107a10 con 0x7f8e640f5ef0 2021-11-03T18:51:17.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.293+0000 7f8e6bdd3700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e64107b50 con 0x7f8e640f55e0 2021-11-03T18:51:17.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.293+0000 7f8e6936e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e640f5ef0 0x7f8e64107460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:51:17.296 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.293+0000 7f8e6936e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e640f5ef0 0x7f8e64107460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37212/0 (socket says 172.21.15.27:37212) 2021-11-03T18:51:17.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.293+0000 7f8e6936e700 1 -- 172.21.15.27:0/274774367 learned_addr learned my addr 172.21.15.27:0/274774367 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:51:17.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.293+0000 7f8e69b6f700 1 --2- 172.21.15.27:0/274774367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e640f55e0 0x7f8e64106f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:51:17.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.293+0000 7f8e6936e700 1 -- 172.21.15.27:0/274774367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e640f55e0 msgr2=0x7f8e64106f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:51:17.297 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.293+0000 7f8e6936e700 1 --2- 172.21.15.27:0/274774367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e640f55e0 0x7f8e64106f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:51:17.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.293+0000 7f8e6936e700 1 -- 172.21.15.27:0/274774367 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e54005040 con 0x7f8e640f5ef0 2021-11-03T18:51:17.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.293+0000 7f8e6936e700 1 --2- 172.21.15.27:0/274774367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e640f5ef0 0x7f8e64107460 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 rx=0x7f8e54009990 tx=0x7f8e54009b30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:51:17.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.293+0000 7f8e5a7fc700 1 -- 172.21.15.27:0/274774367 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e54018070 con 0x7f8e640f5ef0 2021-11-03T18:51:17.298 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.293+0000 7f8e5a7fc700 1 -- 172.21.15.27:0/274774367 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f8e54007ba0 con 0x7f8e640f5ef0 2021-11-03T18:51:17.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.294+0000 7f8e6bdd3700 1 -- 172.21.15.27:0/274774367 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e64108490 con 0x7f8e640f5ef0 2021-11-03T18:51:17.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.294+0000 7f8e5a7fc700 1 -- 172.21.15.27:0/274774367 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8e5401c3c0 con 0x7f8e640f5ef0 2021-11-03T18:51:17.299 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.294+0000 7f8e6bdd3700 1 -- 172.21.15.27:0/274774367 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e641089f0 con 0x7f8e640f5ef0 2021-11-03T18:51:17.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.294+0000 7f8e6bdd3700 1 -- 172.21.15.27:0/274774367 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e64045ad0 con 0x7f8e640f5ef0 2021-11-03T18:51:17.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.295+0000 7f8e5a7fc700 1 -- 172.21.15.27:0/274774367 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f8e5401d420 con 0x7f8e640f5ef0 2021-11-03T18:51:17.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.295+0000 7f8e5a7fc700 1 --2- 172.21.15.27:0/274774367 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8e500640b0 0x7f8e50066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:51:17.301 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.295+0000 7f8e5a7fc700 1 -- 172.21.15.27:0/274774367 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f8e54082360 con 0x7f8e640f5ef0 2021-11-03T18:51:17.302 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.295+0000 7f8e69b6f700 1 --2- 172.21.15.27:0/274774367 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8e500640b0 0x7f8e50066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:51:17.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.296+0000 7f8e69b6f700 1 --2- 172.21.15.27:0/274774367 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8e500640b0 0x7f8e50066570 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 rx=0x7f8e60004c40 tx=0x7f8e6000b040).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:51:17.303 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.300+0000 7f8e5a7fc700 1 -- 172.21.15.27:0/274774367 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f8e54054720 con 0x7f8e640f5ef0 2021-11-03T18:51:17.479 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.480+0000 7f8e6bdd3700 1 -- 172.21.15.27:0/274774367 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e6405fe30 con 0x7f8e500640b0 2021-11-03T18:51:17.480 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.482+0000 7f8e5a7fc700 1 -- 172.21.15.27:0/274774367 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7f8e6405fe30 con 0x7f8e500640b0 2021-11-03T18:51:17.482 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.484+0000 7f8e6bdd3700 1 -- 172.21.15.27:0/274774367 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8e500640b0 msgr2=0x7f8e50066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:51:17.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.484+0000 7f8e6bdd3700 1 --2- 172.21.15.27:0/274774367 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8e500640b0 0x7f8e50066570 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 rx=0x7f8e60004c40 tx=0x7f8e6000b040).stop 2021-11-03T18:51:17.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.484+0000 7f8e6bdd3700 1 -- 172.21.15.27:0/274774367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e640f5ef0 msgr2=0x7f8e64107460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:51:17.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.484+0000 7f8e6bdd3700 1 --2- 172.21.15.27:0/274774367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e640f5ef0 0x7f8e64107460 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 rx=0x7f8e54009990 tx=0x7f8e54009b30).stop 2021-11-03T18:51:17.483 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.484+0000 7f8e6bdd3700 1 -- 172.21.15.27:0/274774367 shutdown_connections 2021-11-03T18:51:17.484 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.484+0000 7f8e6bdd3700 1 --2- 172.21.15.27:0/274774367 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8e500640b0 0x7f8e50066570 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:51:17.484 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.484+0000 7f8e6bdd3700 1 --2- 172.21.15.27:0/274774367 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f8e640f55e0 0x7f8e64106f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:51:17.484 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.484+0000 7f8e6bdd3700 1 --2- 172.21.15.27:0/274774367 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f8e640f5ef0 0x7f8e64107460 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:51:17.484 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.484+0000 7f8e6bdd3700 1 -- 172.21.15.27:0/274774367 >> 172.21.15.27:0/274774367 conn(0x7f8e640f0de0 msgr2=0x7f8e6405f720 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:51:17.484 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.484+0000 7f8e6bdd3700 1 -- 172.21.15.27:0/274774367 shutdown_connections 2021-11-03T18:51:17.485 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.484+0000 7f8e6bdd3700 1 -- 172.21.15.27:0/274774367 wait complete. 2021-11-03T18:51:17.492 INFO:teuthology.orchestra.run.smithi027.stdout:true 2021-11-03T18:51:17.585 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.584+0000 7fe72f59e700 1 -- 172.21.15.27:0/1051145295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe730060330 msgr2=0x7fe7300f9120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:51:17.585 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.584+0000 7fe72f59e700 1 --2- 172.21.15.27:0/1051145295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe730060330 0x7fe7300f9120 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 rx=0x7fe720004660 tx=0x7fe720009b30).stop 2021-11-03T18:51:17.586 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.585+0000 7fe72f59e700 1 -- 172.21.15.27:0/1051145295 shutdown_connections 2021-11-03T18:51:17.586 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.585+0000 7fe72f59e700 1 --2- 172.21.15.27:0/1051145295 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe7300f96a0 0x7fe7300fbaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:51:17.586 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.585+0000 7fe72f59e700 1 --2- 172.21.15.27:0/1051145295 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe730060330 0x7fe7300f9120 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:51:17.586 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.585+0000 7fe72f59e700 1 -- 172.21.15.27:0/1051145295 >> 172.21.15.27:0/1051145295 conn(0x7fe7300f0ba0 msgr2=0x7fe7300f2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:51:17.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.585+0000 7fe72f59e700 1 -- 172.21.15.27:0/1051145295 shutdown_connections 2021-11-03T18:51:17.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.585+0000 7fe72f59e700 1 -- 172.21.15.27:0/1051145295 wait complete. 2021-11-03T18:51:17.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.585+0000 7fe72f59e700 1 Processor -- start 2021-11-03T18:51:17.587 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.585+0000 7fe72f59e700 1 -- start start 2021-11-03T18:51:17.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.585+0000 7fe72f59e700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe730060330 0x7fe730106dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:51:17.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.585+0000 7fe72f59e700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7300f96a0 0x7fe7301072c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:51:17.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.585+0000 7fe72f59e700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7301078c0 con 0x7fe7300f96a0 2021-11-03T18:51:17.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.585+0000 7fe72f59e700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe730108100 con 0x7fe730060330 2021-11-03T18:51:17.588 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.585+0000 7fe72dd9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7300f96a0 0x7fe7301072c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:51:17.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.585+0000 7fe72dd9b700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7300f96a0 0x7fe7301072c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37222/0 (socket says 172.21.15.27:37222) 2021-11-03T18:51:17.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.585+0000 7fe72dd9b700 1 -- 172.21.15.27:0/1650633926 learned_addr learned my addr 172.21.15.27:0/1650633926 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:51:17.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.586+0000 7fe72e59c700 1 --2- 172.21.15.27:0/1650633926 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe730060330 0x7fe730106dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:51:17.589 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.586+0000 7fe72dd9b700 1 -- 172.21.15.27:0/1650633926 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe730060330 msgr2=0x7fe730106dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:51:17.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.586+0000 7fe72dd9b700 1 --2- 172.21.15.27:0/1650633926 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe730060330 0x7fe730106dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:51:17.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.586+0000 7fe72dd9b700 1 -- 172.21.15.27:0/1650633926 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe720005040 con 0x7fe7300f96a0 2021-11-03T18:51:17.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.586+0000 7fe72dd9b700 1 --2- 172.21.15.27:0/1650633926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7300f96a0 0x7fe7301072c0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 rx=0x7fe724002700 tx=0x7fe724009d60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:51:17.590 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.586+0000 7fe71effd700 1 -- 172.21.15.27:0/1650633926 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe7240079c0 con 0x7fe7300f96a0 2021-11-03T18:51:17.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.586+0000 7fe71effd700 1 -- 172.21.15.27:0/1650633926 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7fe72400f040 con 0x7fe7300f96a0 2021-11-03T18:51:17.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.586+0000 7fe72f59e700 1 -- 172.21.15.27:0/1650633926 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7301083e0 con 0x7fe7300f96a0 2021-11-03T18:51:17.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.586+0000 7fe71effd700 1 -- 172.21.15.27:0/1650633926 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe724014740 con 0x7fe7300f96a0 2021-11-03T18:51:17.591 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.586+0000 7fe72f59e700 1 -- 172.21.15.27:0/1650633926 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe730108920 con 0x7fe7300f96a0 2021-11-03T18:51:17.592 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.587+0000 7fe72f59e700 1 -- 172.21.15.27:0/1650633926 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe730059100 con 0x7fe7300f96a0 2021-11-03T18:51:17.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.588+0000 7fe71effd700 1 -- 172.21.15.27:0/1650633926 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7fe72401b060 con 0x7fe7300f96a0 2021-11-03T18:51:17.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.588+0000 7fe71effd700 1 --2- 172.21.15.27:0/1650633926 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe718064100 0x7fe7180665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:51:17.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.588+0000 7fe71effd700 1 -- 172.21.15.27:0/1650633926 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7fe724080690 con 0x7fe7300f96a0 2021-11-03T18:51:17.593 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.588+0000 7fe72e59c700 1 --2- 172.21.15.27:0/1650633926 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe718064100 0x7fe7180665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:51:17.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.588+0000 7fe72e59c700 1 --2- 172.21.15.27:0/1650633926 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe718064100 0x7fe7180665c0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 rx=0x7fe720004630 tx=0x7fe720009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:51:17.594 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.592+0000 7fe71effd700 1 -- 172.21.15.27:0/1650633926 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7fe724052990 con 0x7fe7300f96a0 2021-11-03T18:51:17.764 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.765+0000 7fe72f59e700 1 -- 172.21.15.27:0/1650633926 --> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe7300feeb0 con 0x7fe718064100 2021-11-03T18:51:17.779 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.780+0000 7fe71effd700 1 -- 172.21.15.27:0/1650633926 <== mgr.34104 v2:172.21.15.27:6828/2812551029 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3542 (secure 0 0 0) 0x7fe7300feeb0 con 0x7fe718064100 2021-11-03T18:51:17.779 INFO:teuthology.orchestra.run.smithi027.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2021-11-03T18:51:17.779 INFO:teuthology.orchestra.run.smithi027.stdout:alertmanager.smithi027 smithi027 *:9093,9094 running (102m) 71s ago 104m 19.8M - 0.20.0 0881eb8f169f f3c51b310eb7 2021-11-03T18:51:17.779 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi027 smithi027 running (94m) 71s ago 104m 7189k - 17.0.0-8776-g6e90212d 50416d063274 ef141d2afe22 2021-11-03T18:51:17.780 INFO:teuthology.orchestra.run.smithi027.stdout:crash.smithi115 smithi115 running (94m) 48s ago 103m 7235k - 17.0.0-8776-g6e90212d 50416d063274 e85b0146d460 2021-11-03T18:51:17.780 INFO:teuthology.orchestra.run.smithi027.stdout:grafana.smithi027 smithi027 *:3000 running (102m) 71s ago 103m 42.8M - 6.7.4 557c83e11646 d11df98c024d 2021-11-03T18:51:17.780 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.fksixo smithi027 running (96m) 71s ago 99m 27.1M - 16.2.4 8d91d370c2b8 b7eb5661f5db 2021-11-03T18:51:17.780 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi027.jdwohn smithi027 running (99m) 71s ago 99m 3853M - 16.2.4 8d91d370c2b8 24b2f5428798 2021-11-03T18:51:17.781 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.semywj smithi115 running (95m) 48s ago 99m 26.7M - 16.2.4 8d91d370c2b8 ad2f9ab3055c 2021-11-03T18:51:17.781 INFO:teuthology.orchestra.run.smithi027.stdout:mds.cephfs.smithi115.zsfgnv smithi115 running (96m) 48s ago 99m 27.8M - 16.2.4 8d91d370c2b8 7ca4a56723cb 2021-11-03T18:51:17.781 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi027.ufhhvb smithi027 *:8499,8443,9283 running (96m) 71s ago 105m 644M - 17.0.0-8776-g6e90212d 50416d063274 fc8e5ab66fa8 2021-11-03T18:51:17.781 INFO:teuthology.orchestra.run.smithi027.stdout:mgr.smithi115.tcyodz smithi115 *:8499,8443,9283 running (95m) 48s ago 103m 422M - 17.0.0-8776-g6e90212d 50416d063274 124851ba05d9 2021-11-03T18:51:17.781 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi027 smithi027 running (95m) 71s ago 105m 424M 2048M 17.0.0-8776-g6e90212d 50416d063274 ce77ef590f9e 2021-11-03T18:51:17.782 INFO:teuthology.orchestra.run.smithi027.stdout:mon.smithi115 smithi115 running (95m) 48s ago 103m 191M 2048M 17.0.0-8776-g6e90212d 50416d063274 e3f7967a24d5 2021-11-03T18:51:17.782 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi027 smithi027 *:9100 running (103m) 71s ago 103m 18.1M - 0.18.1 e5a616e4b9cf ec9f5df57dd3 2021-11-03T18:51:17.782 INFO:teuthology.orchestra.run.smithi027.stdout:node-exporter.smithi115 smithi115 *:9100 running (103m) 48s ago 103m 18.4M - 0.18.1 e5a616e4b9cf c0cfb6c210da 2021-11-03T18:51:17.782 INFO:teuthology.orchestra.run.smithi027.stdout:osd.0 smithi027 running (94m) 71s ago 102m 453M 4096M 17.0.0-8776-g6e90212d 50416d063274 da109dc25a05 2021-11-03T18:51:17.783 INFO:teuthology.orchestra.run.smithi027.stdout:osd.1 smithi027 running (93m) 71s ago 102m 493M 4096M 17.0.0-8776-g6e90212d 50416d063274 cf75166ddccf 2021-11-03T18:51:17.783 INFO:teuthology.orchestra.run.smithi027.stdout:osd.2 smithi027 running (93m) 71s ago 101m 374M 4096M 17.0.0-8776-g6e90212d 50416d063274 8e134c0e6fe1 2021-11-03T18:51:17.783 INFO:teuthology.orchestra.run.smithi027.stdout:osd.3 smithi115 running (92m) 48s ago 101m 529M 4096M 17.0.0-8776-g6e90212d 50416d063274 731e672c66d0 2021-11-03T18:51:17.783 INFO:teuthology.orchestra.run.smithi027.stdout:osd.4 smithi115 running (92m) 48s ago 101m 415M 4096M 17.0.0-8776-g6e90212d 50416d063274 3a8535b47627 2021-11-03T18:51:17.783 INFO:teuthology.orchestra.run.smithi027.stdout:osd.5 smithi115 running (91m) 48s ago 100m 428M 4096M 17.0.0-8776-g6e90212d 50416d063274 059cccf0e5c8 2021-11-03T18:51:17.784 INFO:teuthology.orchestra.run.smithi027.stdout:prometheus.smithi027 smithi027 *:9095 running (102m) 71s ago 103m 69.1M - 2.18.1 de242295e225 65df1ba48e2f 2021-11-03T18:51:17.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.782+0000 7fe72f59e700 1 -- 172.21.15.27:0/1650633926 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe718064100 msgr2=0x7fe7180665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:51:17.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.782+0000 7fe72f59e700 1 --2- 172.21.15.27:0/1650633926 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe718064100 0x7fe7180665c0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 rx=0x7fe720004630 tx=0x7fe720009b30).stop 2021-11-03T18:51:17.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.782+0000 7fe72f59e700 1 -- 172.21.15.27:0/1650633926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7300f96a0 msgr2=0x7fe7301072c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:51:17.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.782+0000 7fe72f59e700 1 --2- 172.21.15.27:0/1650633926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7300f96a0 0x7fe7301072c0 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 rx=0x7fe724002700 tx=0x7fe724009d60).stop 2021-11-03T18:51:17.785 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.782+0000 7fe72f59e700 1 -- 172.21.15.27:0/1650633926 shutdown_connections 2021-11-03T18:51:17.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.782+0000 7fe72f59e700 1 --2- 172.21.15.27:0/1650633926 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7fe718064100 0x7fe7180665c0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:51:17.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.782+0000 7fe72f59e700 1 --2- 172.21.15.27:0/1650633926 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7fe730060330 0x7fe730106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:51:17.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.782+0000 7fe72f59e700 1 --2- 172.21.15.27:0/1650633926 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7fe7300f96a0 0x7fe7301072c0 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:51:17.786 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.782+0000 7fe72f59e700 1 -- 172.21.15.27:0/1650633926 >> 172.21.15.27:0/1650633926 conn(0x7fe7300f0ba0 msgr2=0x7fe7300f2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:51:17.787 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.782+0000 7fe72f59e700 1 -- 172.21.15.27:0/1650633926 shutdown_connections 2021-11-03T18:51:17.787 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.782+0000 7fe72f59e700 1 -- 172.21.15.27:0/1650633926 wait complete. 2021-11-03T18:51:17.880 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.881+0000 7f847a681700 1 -- 172.21.15.27:0/3081252795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84740f73b0 msgr2=0x7f84740f4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:51:17.880 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.881+0000 7f847a681700 1 --2- 172.21.15.27:0/3081252795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84740f73b0 0x7f84740f4640 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 rx=0x7f8464004660 tx=0x7f8464009b30).stop 2021-11-03T18:51:17.880 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.881+0000 7f847a681700 1 -- 172.21.15.27:0/3081252795 shutdown_connections 2021-11-03T18:51:17.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.881+0000 7f847a681700 1 --2- 172.21.15.27:0/3081252795 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84740f73b0 0x7f84740f4640 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:51:17.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.881+0000 7f847a681700 1 --2- 172.21.15.27:0/3081252795 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84740f6a10 0x7f84740f6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:51:17.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.881+0000 7f847a681700 1 -- 172.21.15.27:0/3081252795 >> 172.21.15.27:0/3081252795 conn(0x7f84740f0320 msgr2=0x7f84740f2740 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:51:17.881 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.882+0000 7f847a681700 1 -- 172.21.15.27:0/3081252795 shutdown_connections 2021-11-03T18:51:17.882 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.882+0000 7f847a681700 1 -- 172.21.15.27:0/3081252795 wait complete. 2021-11-03T18:51:17.884 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.882+0000 7f847a681700 1 Processor -- start 2021-11-03T18:51:17.884 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.882+0000 7f847a681700 1 -- start start 2021-11-03T18:51:17.885 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.882+0000 7f847a681700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84740f6a10 0x7f8474108f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:51:17.885 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.882+0000 7f847a681700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84740f73b0 0x7f8474109430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:51:17.885 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.882+0000 7f847a681700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84741099e0 con 0x7f84740f6a10 2021-11-03T18:51:17.885 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.882+0000 7f847a681700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8474109b20 con 0x7f84740f73b0 2021-11-03T18:51:17.885 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.882+0000 7f847967f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84740f6a10 0x7f8474108f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:51:17.886 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.882+0000 7f847967f700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84740f6a10 0x7f8474108f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37230/0 (socket says 172.21.15.27:37230) 2021-11-03T18:51:17.886 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.882+0000 7f847967f700 1 -- 172.21.15.27:0/1376718291 learned_addr learned my addr 172.21.15.27:0/1376718291 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:51:17.886 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.883+0000 7f847967f700 1 -- 172.21.15.27:0/1376718291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84740f73b0 msgr2=0x7f8474109430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:51:17.886 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.883+0000 7f8478e7e700 1 --2- 172.21.15.27:0/1376718291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84740f73b0 0x7f8474109430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:51:17.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.883+0000 7f847967f700 1 --2- 172.21.15.27:0/1376718291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84740f73b0 0x7f8474109430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:51:17.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.883+0000 7f847967f700 1 -- 172.21.15.27:0/1376718291 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8464005040 con 0x7f84740f6a10 2021-11-03T18:51:17.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.883+0000 7f847967f700 1 --2- 172.21.15.27:0/1376718291 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84740f6a10 0x7f8474108f40 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 rx=0x7f847000c530 tx=0x7f8470007e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:51:17.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.883+0000 7f8469ffb700 1 -- 172.21.15.27:0/1376718291 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f847000f650 con 0x7f84740f6a10 2021-11-03T18:51:17.887 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.883+0000 7f8469ffb700 1 -- 172.21.15.27:0/1376718291 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f847000fc60 con 0x7f84740f6a10 2021-11-03T18:51:17.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.883+0000 7f8469ffb700 1 -- 172.21.15.27:0/1376718291 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8470012cd0 con 0x7f84740f6a10 2021-11-03T18:51:17.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.883+0000 7f847a681700 1 -- 172.21.15.27:0/1376718291 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f847410a4c0 con 0x7f84740f6a10 2021-11-03T18:51:17.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.883+0000 7f847a681700 1 -- 172.21.15.27:0/1376718291 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f847410aa80 con 0x7f84740f6a10 2021-11-03T18:51:17.888 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.884+0000 7f847a681700 1 -- 172.21.15.27:0/1376718291 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8474045ad0 con 0x7f84740f6a10 2021-11-03T18:51:17.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.885+0000 7f8469ffb700 1 -- 172.21.15.27:0/1376718291 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f8470010070 con 0x7f84740f6a10 2021-11-03T18:51:17.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.885+0000 7f8469ffb700 1 --2- 172.21.15.27:0/1376718291 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8460064100 0x7f84600665c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:51:17.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.885+0000 7f8469ffb700 1 -- 172.21.15.27:0/1376718291 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f8470080640 con 0x7f84740f6a10 2021-11-03T18:51:17.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.885+0000 7f8478e7e700 1 --2- 172.21.15.27:0/1376718291 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8460064100 0x7f84600665c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:51:17.889 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.885+0000 7f8478e7e700 1 --2- 172.21.15.27:0/1376718291 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8460064100 0x7f84600665c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 rx=0x7f8464009da0 tx=0x7f8464009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:51:17.890 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:17.889+0000 7f8469ffb700 1 -- 172.21.15.27:0/1376718291 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f84700528b0 con 0x7f84740f6a10 2021-11-03T18:51:18.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.118+0000 7f847a681700 1 -- 172.21.15.27:0/1376718291 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f84741010b0 con 0x7f84740f6a10 2021-11-03T18:51:18.117 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.119+0000 7f8469ffb700 1 -- 172.21.15.27:0/1376718291 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+711 (secure 0 0 0) 0x7f84700524a0 con 0x7f84740f6a10 2021-11-03T18:51:18.118 INFO:teuthology.orchestra.run.smithi027.stdout:{ 2021-11-03T18:51:18.118 INFO:teuthology.orchestra.run.smithi027.stdout: "mon": { 2021-11-03T18:51:18.118 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:51:18.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:51:18.119 INFO:teuthology.orchestra.run.smithi027.stdout: "mgr": { 2021-11-03T18:51:18.119 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 2 2021-11-03T18:51:18.119 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:51:18.119 INFO:teuthology.orchestra.run.smithi027.stdout: "osd": { 2021-11-03T18:51:18.120 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 6 2021-11-03T18:51:18.120 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:51:18.120 INFO:teuthology.orchestra.run.smithi027.stdout: "mds": { 2021-11-03T18:51:18.120 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4 2021-11-03T18:51:18.121 INFO:teuthology.orchestra.run.smithi027.stdout: }, 2021-11-03T18:51:18.121 INFO:teuthology.orchestra.run.smithi027.stdout: "overall": { 2021-11-03T18:51:18.121 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 16.2.4 (3cbe25cde3cfa028984618ad32de9edc4c1eaed0) pacific (stable)": 4, 2021-11-03T18:51:18.121 INFO:teuthology.orchestra.run.smithi027.stdout: "ceph version 17.0.0-8776-g6e90212d (6e90212d8510e2d506395a8c82057d0d1787f3a4) quincy (dev)": 10 2021-11-03T18:51:18.122 INFO:teuthology.orchestra.run.smithi027.stdout: } 2021-11-03T18:51:18.122 INFO:teuthology.orchestra.run.smithi027.stdout:} 2021-11-03T18:51:18.123 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.120+0000 7f847a681700 1 -- 172.21.15.27:0/1376718291 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8460064100 msgr2=0x7f84600665c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:51:18.123 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.120+0000 7f847a681700 1 --2- 172.21.15.27:0/1376718291 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8460064100 0x7f84600665c0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 rx=0x7f8464009da0 tx=0x7f8464009b30).stop 2021-11-03T18:51:18.123 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.120+0000 7f847a681700 1 -- 172.21.15.27:0/1376718291 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84740f6a10 msgr2=0x7f8474108f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:51:18.123 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.120+0000 7f847a681700 1 --2- 172.21.15.27:0/1376718291 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84740f6a10 0x7f8474108f40 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 rx=0x7f847000c530 tx=0x7f8470007e20).stop 2021-11-03T18:51:18.124 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.121+0000 7f847a681700 1 -- 172.21.15.27:0/1376718291 shutdown_connections 2021-11-03T18:51:18.124 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.121+0000 7f847a681700 1 --2- 172.21.15.27:0/1376718291 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f8460064100 0x7f84600665c0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:51:18.124 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.121+0000 7f847a681700 1 --2- 172.21.15.27:0/1376718291 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f84740f6a10 0x7f8474108f40 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:51:18.124 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.121+0000 7f847a681700 1 --2- 172.21.15.27:0/1376718291 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f84740f73b0 0x7f8474109430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:51:18.125 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.121+0000 7f847a681700 1 -- 172.21.15.27:0/1376718291 >> 172.21.15.27:0/1376718291 conn(0x7f84740f0320 msgr2=0x7f84740fe560 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:51:18.125 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.121+0000 7f847a681700 1 -- 172.21.15.27:0/1376718291 shutdown_connections 2021-11-03T18:51:18.125 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.121+0000 7f847a681700 1 -- 172.21.15.27:0/1376718291 wait complete. 2021-11-03T18:51:18.219 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.221+0000 7f04363f6700 1 -- 172.21.15.27:0/1414524205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04300f66e0 msgr2=0x7f04300f6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:51:18.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.221+0000 7f04363f6700 1 --2- 172.21.15.27:0/1414524205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04300f66e0 0x7f04300f6b00 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 rx=0x7f0420004660 tx=0x7f0420009b30).stop 2021-11-03T18:51:18.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.221+0000 7f04363f6700 1 -- 172.21.15.27:0/1414524205 shutdown_connections 2021-11-03T18:51:18.220 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.221+0000 7f04363f6700 1 --2- 172.21.15.27:0/1414524205 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04300f6ff0 0x7f04300f7a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2021-11-03T18:51:18.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.221+0000 7f04363f6700 1 --2- 172.21.15.27:0/1414524205 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04300f66e0 0x7f04300f6b00 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:51:18.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.221+0000 7f04363f6700 1 -- 172.21.15.27:0/1414524205 >> 172.21.15.27:0/1414524205 conn(0x7f04300f1f00 msgr2=0x7f04300f4320 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:51:18.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.221+0000 7f04363f6700 1 -- 172.21.15.27:0/1414524205 shutdown_connections 2021-11-03T18:51:18.221 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.221+0000 7f04363f6700 1 -- 172.21.15.27:0/1414524205 wait complete. 2021-11-03T18:51:18.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.221+0000 7f04363f6700 1 Processor -- start 2021-11-03T18:51:18.222 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.221+0000 7f04363f6700 1 -- start start 2021-11-03T18:51:18.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.222+0000 7f04363f6700 1 --2- >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04300f66e0 0x7f043006b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:51:18.224 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.222+0000 7f04363f6700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04300f6ff0 0x7f0430069e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:51:18.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.222+0000 7f04363f6700 1 -- --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043006a380 con 0x7f04300f6ff0 2021-11-03T18:51:18.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.222+0000 7f04363f6700 1 -- --> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f043006a4c0 con 0x7f04300f66e0 2021-11-03T18:51:18.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.222+0000 7f042f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04300f6ff0 0x7f0430069e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:51:18.225 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.222+0000 7f042f7fe700 1 --2- >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04300f6ff0 0x7f0430069e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.27:3300/0 says I am v2:172.21.15.27:37242/0 (socket says 172.21.15.27:37242) 2021-11-03T18:51:18.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.222+0000 7f042f7fe700 1 -- 172.21.15.27:0/1822846497 learned_addr learned my addr 172.21.15.27:0/1822846497 (peer_addr_for_me v2:172.21.15.27:0/0) 2021-11-03T18:51:18.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.222+0000 7f042ffff700 1 --2- 172.21.15.27:0/1822846497 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04300f66e0 0x7f043006b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:51:18.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.222+0000 7f042f7fe700 1 -- 172.21.15.27:0/1822846497 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04300f66e0 msgr2=0x7f043006b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:51:18.226 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.222+0000 7f042f7fe700 1 --2- 172.21.15.27:0/1822846497 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04300f66e0 0x7f043006b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:51:18.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.222+0000 7f042f7fe700 1 -- 172.21.15.27:0/1822846497 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0420005040 con 0x7f04300f6ff0 2021-11-03T18:51:18.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.222+0000 7f042f7fe700 1 --2- 172.21.15.27:0/1822846497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04300f6ff0 0x7f0430069e20 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 rx=0x7f04180026d0 tx=0x7f0418009de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:51:18.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.222+0000 7f042cff9700 1 -- 172.21.15.27:0/1822846497 <== mon.0 v2:172.21.15.27:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04180079c0 con 0x7f04300f6ff0 2021-11-03T18:51:18.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.222+0000 7f042cff9700 1 -- 172.21.15.27:0/1822846497 <== mon.0 v2:172.21.15.27:3300/0 2 ==== config(26 keys) v1 ==== 1075+0+0 (secure 0 0 0) 0x7f041800f040 con 0x7f04300f6ff0 2021-11-03T18:51:18.227 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.222+0000 7f04363f6700 1 -- 172.21.15.27:0/1822846497 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f043006a7a0 con 0x7f04300f6ff0 2021-11-03T18:51:18.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.222+0000 7f042cff9700 1 -- 172.21.15.27:0/1822846497 <== mon.0 v2:172.21.15.27:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f04180148b0 con 0x7f04300f6ff0 2021-11-03T18:51:18.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.222+0000 7f04363f6700 1 -- 172.21.15.27:0/1822846497 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f043010ef90 con 0x7f04300f6ff0 2021-11-03T18:51:18.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.223+0000 7f04363f6700 1 -- 172.21.15.27:0/1822846497 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0430045ad0 con 0x7f04300f6ff0 2021-11-03T18:51:18.228 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.224+0000 7f042cff9700 1 -- 172.21.15.27:0/1822846497 <== mon.0 v2:172.21.15.27:3300/0 4 ==== mgrmap(e 36) v1 ==== 81917+0+0 (secure 0 0 0) 0x7f0418007b20 con 0x7f04300f6ff0 2021-11-03T18:51:18.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.224+0000 7f042cff9700 1 --2- 172.21.15.27:0/1822846497 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f041c0640b0 0x7f041c066570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2021-11-03T18:51:18.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.224+0000 7f042cff9700 1 -- 172.21.15.27:0/1822846497 <== mon.0 v2:172.21.15.27:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 6461+0+0 (secure 0 0 0) 0x7f0418080670 con 0x7f04300f6ff0 2021-11-03T18:51:18.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.224+0000 7f042ffff700 1 --2- 172.21.15.27:0/1822846497 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f041c0640b0 0x7f041c066570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2021-11-03T18:51:18.229 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.225+0000 7f042ffff700 1 --2- 172.21.15.27:0/1822846497 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f041c0640b0 0x7f041c066570 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 rx=0x7f0420004630 tx=0x7f0420009b30).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2021-11-03T18:51:18.230 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.228+0000 7f042cff9700 1 -- 172.21.15.27:0/1822846497 <== mon.0 v2:172.21.15.27:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+155104 (secure 0 0 0) 0x7f04180529d0 con 0x7f04300f6ff0 2021-11-03T18:51:18.430 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.432+0000 7f04363f6700 1 -- 172.21.15.27:0/1822846497 --> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0430027a00 con 0x7f04300f6ff0 2021-11-03T18:51:18.431 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.432+0000 7f042cff9700 1 -- 172.21.15.27:0/1822846497 <== mon.0 v2:172.21.15.27:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 31 v31) v1 ==== 76+0+1780 (secure 0 0 0) 0x7f04180525c0 con 0x7f04300f6ff0 2021-11-03T18:51:18.431 INFO:teuthology.orchestra.run.smithi027.stdout:e31 2021-11-03T18:51:18.431 INFO:teuthology.orchestra.run.smithi027.stdout:enable_multiple, ever_enabled_multiple: 1,1 2021-11-03T18:51:18.432 INFO:teuthology.orchestra.run.smithi027.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:51:18.432 INFO:teuthology.orchestra.run.smithi027.stdout:legacy client fscid: 1 2021-11-03T18:51:18.432 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:51:18.432 INFO:teuthology.orchestra.run.smithi027.stdout:Filesystem 'cephfs' (1) 2021-11-03T18:51:18.433 INFO:teuthology.orchestra.run.smithi027.stdout:fs_name cephfs 2021-11-03T18:51:18.433 INFO:teuthology.orchestra.run.smithi027.stdout:epoch 31 2021-11-03T18:51:18.433 INFO:teuthology.orchestra.run.smithi027.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2021-11-03T18:51:18.433 INFO:teuthology.orchestra.run.smithi027.stdout:created 2021-11-03T17:11:53.078170+0000 2021-11-03T18:51:18.434 INFO:teuthology.orchestra.run.smithi027.stdout:modified 2021-11-03T17:19:34.682010+0000 2021-11-03T18:51:18.434 INFO:teuthology.orchestra.run.smithi027.stdout:tableserver 0 2021-11-03T18:51:18.434 INFO:teuthology.orchestra.run.smithi027.stdout:root 0 2021-11-03T18:51:18.434 INFO:teuthology.orchestra.run.smithi027.stdout:session_timeout 60 2021-11-03T18:51:18.434 INFO:teuthology.orchestra.run.smithi027.stdout:session_autoclose 300 2021-11-03T18:51:18.435 INFO:teuthology.orchestra.run.smithi027.stdout:max_file_size 1099511627776 2021-11-03T18:51:18.435 INFO:teuthology.orchestra.run.smithi027.stdout:required_client_features {} 2021-11-03T18:51:18.435 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure 0 2021-11-03T18:51:18.435 INFO:teuthology.orchestra.run.smithi027.stdout:last_failure_osd_epoch 53 2021-11-03T18:51:18.436 INFO:teuthology.orchestra.run.smithi027.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2021-11-03T18:51:18.436 INFO:teuthology.orchestra.run.smithi027.stdout:max_mds 1 2021-11-03T18:51:18.436 INFO:teuthology.orchestra.run.smithi027.stdout:in 0,1 2021-11-03T18:51:18.436 INFO:teuthology.orchestra.run.smithi027.stdout:up {1=14436} 2021-11-03T18:51:18.437 INFO:teuthology.orchestra.run.smithi027.stdout:failed 0 2021-11-03T18:51:18.437 INFO:teuthology.orchestra.run.smithi027.stdout:damaged 2021-11-03T18:51:18.437 INFO:teuthology.orchestra.run.smithi027.stdout:stopped 2021-11-03T18:51:18.437 INFO:teuthology.orchestra.run.smithi027.stdout:data_pools [3] 2021-11-03T18:51:18.437 INFO:teuthology.orchestra.run.smithi027.stdout:metadata_pool 2 2021-11-03T18:51:18.438 INFO:teuthology.orchestra.run.smithi027.stdout:inline_data disabled 2021-11-03T18:51:18.438 INFO:teuthology.orchestra.run.smithi027.stdout:balancer 2021-11-03T18:51:18.438 INFO:teuthology.orchestra.run.smithi027.stdout:standby_count_wanted 1 2021-11-03T18:51:18.438 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.jdwohn{1:14436} state up:active seq 107 join_fscid=1 addr [v2:172.21.15.27:6826/536348060,v1:172.21.15.27:6827/536348060] compat {c=[1],r=[1],i=[77f]}] 2021-11-03T18:51:18.439 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:51:18.439 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:51:18.439 INFO:teuthology.orchestra.run.smithi027.stdout:Standby daemons: 2021-11-03T18:51:18.439 INFO:teuthology.orchestra.run.smithi027.stdout: 2021-11-03T18:51:18.439 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi027.fksixo{-1:14626} state up:standby seq 3 join_fscid=1 addr [v2:172.21.15.27:6800/944356635,v1:172.21.15.27:6801/944356635] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:51:18.440 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.zsfgnv{-1:24427} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.115:6824/1283095060,v1:172.21.15.115:6825/1283095060] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:51:18.440 INFO:teuthology.orchestra.run.smithi027.stdout:[mds.cephfs.smithi115.semywj{-1:34134} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.115:6826/78432028,v1:172.21.15.115:6827/78432028] compat {c=[1],r=[1],i=[1]}] 2021-11-03T18:51:18.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.434+0000 7f04363f6700 1 -- 172.21.15.27:0/1822846497 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f041c0640b0 msgr2=0x7f041c066570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:51:18.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.434+0000 7f04363f6700 1 --2- 172.21.15.27:0/1822846497 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f041c0640b0 0x7f041c066570 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 rx=0x7f0420004630 tx=0x7f0420009b30).stop 2021-11-03T18:51:18.441 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.434+0000 7f04363f6700 1 -- 172.21.15.27:0/1822846497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04300f6ff0 msgr2=0x7f0430069e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2021-11-03T18:51:18.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.434+0000 7f04363f6700 1 --2- 172.21.15.27:0/1822846497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04300f6ff0 0x7f0430069e20 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 rx=0x7f04180026d0 tx=0x7f0418009de0).stop 2021-11-03T18:51:18.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.434+0000 7f04363f6700 1 -- 172.21.15.27:0/1822846497 shutdown_connections 2021-11-03T18:51:18.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.434+0000 7f04363f6700 1 --2- 172.21.15.27:0/1822846497 >> [v2:172.21.15.27:6828/2812551029,v1:172.21.15.27:6829/2812551029] conn(0x7f041c0640b0 0x7f041c066570 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:51:18.442 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.434+0000 7f04363f6700 1 --2- 172.21.15.27:0/1822846497 >> [v2:172.21.15.115:3300/0,v1:172.21.15.115:6789/0] conn(0x7f04300f66e0 0x7f043006b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:51:18.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.434+0000 7f04363f6700 1 --2- 172.21.15.27:0/1822846497 >> [v2:172.21.15.27:3300/0,v1:172.21.15.27:6789/0] conn(0x7f04300f6ff0 0x7f0430069e20 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2021-11-03T18:51:18.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.434+0000 7f04363f6700 1 -- 172.21.15.27:0/1822846497 >> 172.21.15.27:0/1822846497 conn(0x7f04300f1f00 msgr2=0x7f04300fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2021-11-03T18:51:18.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.434+0000 7f04363f6700 1 -- 172.21.15.27:0/1822846497 shutdown_connections 2021-11-03T18:51:18.443 INFO:teuthology.orchestra.run.smithi027.stderr:2021-11-03T18:51:18.434+0000 7f04363f6700 1 -- 172.21.15.27:0/1822846497 wait complete. 2021-11-03T18:51:18.443 INFO:teuthology.orchestra.run.smithi027.stderr:dumped fsmap epoch 31 2021-11-03T18:51:19.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:18 smithi115 ceph-mon[65453]: pgmap v2901: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:51:19.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:18 smithi115 ceph-mon[65453]: from='client.37080 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:51:19.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:18 smithi115 ceph-mon[65453]: from='client.37084 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:51:19.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:18 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1376718291' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:51:19.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:18 smithi115 ceph-mon[65453]: from='client.? 172.21.15.27:0/1822846497' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:51:19.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:18 smithi027 ceph-mon[97244]: pgmap v2901: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:51:19.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:18 smithi027 ceph-mon[97244]: from='client.37080 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:51:19.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:18 smithi027 ceph-mon[97244]: from='client.37084 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2021-11-03T18:51:19.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:18 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1376718291' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:51:19.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:18 smithi027 ceph-mon[97244]: from='client.? 172.21.15.27:0/1822846497' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2021-11-03T18:51:21.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:20 smithi115 ceph-mon[65453]: pgmap v2902: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:51:21.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:20 smithi115 ceph-mon[65453]: 1 slow requests, 0 included below; oldest blocked for > 5722.497899 secs 2021-11-03T18:51:21.238 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:51:21.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:51:21.239 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:51:21.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:51:21.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:51:21.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:51:21.240 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:20 smithi115 ceph-mon[65453]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:51:21.241 INFO:journalctl@ceph.mon.smithi115.smithi115.stdout:Nov 03 18:51:20 smithi115 ceph-mon[65453]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:51:21.272 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:20 smithi027 ceph-mon[97244]: pgmap v2902: 65 pgs: 65 active+clean; 564 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail 2021-11-03T18:51:21.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:20 smithi027 ceph-mon[97244]: 1 slow requests, 0 included below; oldest blocked for > 5722.497899 secs 2021-11-03T18:51:21.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:51:21.273 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' 2021-11-03T18:51:21.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2021-11-03T18:51:21.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:51:21.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:51:21.274 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:51:21.275 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:20 smithi027 ceph-mon[97244]: from='mgr.34104 172.21.15.27:0/5408693' entity='mgr.smithi027.ufhhvb' cmd=[{"prefix": "versions"}]: dispatch 2021-11-03T18:51:21.275 INFO:journalctl@ceph.mon.smithi027.smithi027.stdout:Nov 03 18:51:20 smithi027 ceph-mon[97244]: Upgrade: Waiting for fs cephfs to scale down to reach 1 MDS 2021-11-03T18:51:22.006 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2021-11-03T18:51:22.009 DEBUG:teuthology.task.console_log:Killing console logger for smithi027 2021-11-03T18:51:22.010 DEBUG:teuthology.task.console_log:Killing console logger for smithi115 2021-11-03T18:51:22.010 DEBUG:teuthology.exit:Finished running handlers